2024-04-21T08:20:59.111 INFO:root:teuthology version: 0.0.1.dev269+g9481b1d 2024-04-21T08:20:59.112 DEBUG:teuthology.run:Teuthology command: teuthology --verbose --description fs/workload/{0-ubuntu_22.04 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} --name leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi --owner scheduled_leonidus@teuthology --archive /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666511 -- /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666511/orig.config.yaml 2024-04-21T08:20:59.174 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-21T08:20:59.273 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666511 branch: wip-lusov-quiescer description: fs/workload/{0-ubuntu_22.04 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} email: null first_in_suite: false job_id: '7666511' kernel: client: sha1: distro kdb: true sha1: distro ktype: distro last_in_suite: false log-rotate: ceph-mds: 10G ceph-osd: 10G machine_type: smithi mgrmodules: sequential: - print: Enabling mgr modules - exec: mon.a: - ceph mgr module enable snap_schedule - ceph config set mgr mgr/snap_schedule/allow_m_granularity true - ceph config set mgr mgr/snap_schedule/dump_on_update true name: leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi no_nested_subset: false nuke-on-error: true openstack: - volumes: count: 4 size: 20 - machine: disk: 200 os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-lusov-quiescer ceph: cephfs: ec_profile: - m=2 - k=2 - crush-failure-domain=osd max_mds: 5 session_timeout: 300 standby_replay: true conf: client: client mount timeout: 600 debug client: 20 debug ms: 1 rados mon op timeout: 900 rados osd op timeout: 900 global: ms die on skipped message: false mds: debug mds: 20 debug mds balancer: 20 debug ms: 1 mds bal fragment size max: 10000 mds bal merge size: 5 mds bal split bits: 3 mds bal split size: 100 mds debug frag: true mds debug scatterstat: true mds op complaint time: 180 mds verify scatter: true mds_bal_replicate_threshold: 1 osd op complaint time: 180 rados mon op timeout: 900 rados osd op timeout: 900 mgr: cephsqlite lock renewal timeout: 900000 debug cephsqlite: 20 debug client: 20 debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon down mkfs grace: 300 mon op complaint time: 120 osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug ms: 1 debug osd: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore osd op complaint time: 180 osd shutdown pgref assert: true osd_max_omap_entries_per_request: 10000 flavor: default fs: xfs log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - FS_DEGRADED - FS_INLINE_DATA_DEPRECATED - FS_WITH_FAILED_MDS - MDS_ALL_DOWN - MDS_DAMAGE - MDS_DEGRADED - MDS_FAILED - MDS_INSUFFICIENT_STANDBY - MDS_UP_LESS_THAN_MAX - filesystem is online with fewer MDS than max_mds - POOL_APP_NOT_ENABLED - do not have an application enabled - overall HEALTH_ - Replacing daemon - deprecated feature inline_data - overall HEALTH_ - \(OSD_DOWN\) - \(OSD_ - but it is still running - is not responding - PG_AVAILABILITY - PG_DEGRADED - Reduced data availability - slow metadata IO - SLOW_OPS - slow request sha1: a630765c1f93c6e61d9b40c4b76437611d5ef29e subvols: create: 2 subvol_options: --size 25000000000 ceph-deploy: bluestore: true conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} osd: bluestore block size: 96636764160 bluestore fsck on mount: true debug bluefs: 20 debug bluestore: 20 debug rocksdb: 10 mon osd backfillfull_ratio: 0.85 mon osd full ratio: 0.9 mon osd nearfull ratio: 0.8 osd failsafe full ratio: 0.95 osd objectstore: bluestore fs: xfs ceph-fuse: client.0: mount_subvol_num: 0 check-counter: counters: mds: - mds.exported - mds.imported - min: 3 name: mds_cache.dir_update - min: 3 name: mds_cache.dir_update_receipt - mds.root_rsnaps - mds_server.req_mksnap_latency.avgcount - mds_server.req_rmsnap_latency.avgcount dry_run: true install: ceph: flavor: default sha1: a630765c1f93c6e61d9b40c4b76437611d5ef29e kclient: client.0: mount_subvol_num: 1 mntopts: - ms_mode=secure - wsync syntax: v1 selinux: allowlist: - scontext=system_u:system_r:getty_t:s0 thrashosds: bdev_inject_crash: 2 bdev_inject_crash_probability: 0.5 workunit: branch: wip-lusov-quiescer sha1: 4f6bc5273643aaf5660909e85400647a8951c10b owner: scheduled_leonidus@teuthology priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - mds.a - mds.d - mds.g - mds.j - osd.0 - osd.3 - osd.6 - osd.9 - client.0 - - mon.b - mgr.y - mds.b - mds.e - mds.h - mds.k - osd.1 - osd.4 - osd.7 - osd.10 - - mon.c - mgr.z - mds.c - mds.f - mds.i - mds.l - osd.2 - osd.5 - osd.8 - osd.11 seed: 7599 sha1: a630765c1f93c6e61d9b40c4b76437611d5ef29e sleep_before_teardown: 0 subset: 111/111111 suite: fs suite_branch: wip-lusov-quiescer suite_path: /home/teuthworker/src/github.com_leonid-s-usov_ceph-ceph_4f6bc5273643aaf5660909e85400647a8951c10b/qa suite_relpath: qa suite_repo: https://github.com/leonid-s-usov/ceph-ceph suite_sha1: 4f6bc5273643aaf5660909e85400647a8951c10b syslog: ignorelist: - WARNING*.*check_session_state - WARNING*.*__ceph_remove_cap targets: smithi086.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBYfbXPoZ7D8BJKoi2VUL62N9ZlIfp6xJxHHCbXcCmEINmZC3+9zF50s3gdqfg271tP9z3/iq7Bx5j4Hf0/NLoU= smithi088.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI1Hyc0xyjsQLdKgYCVOBgiyqDPLM5T0eiIiym5swZ7i9ORD0a10OvDztobC2iwb4Aa3ggytXWvbSMkahIgj4Ug= smithi179.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPnr8CAdOQWzcezeu0TweNh/qMpja8vU18k8zRiz2xPUnqGVrCgFU078WM4ypon/wpPvxgc+yvadBIfb7w8XJTY= tasks: - install: extra_packages: deb: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror rpm: - python3-cephfs - cephfs-shell - cephfs-top - cephfs-mirror extra_system_packages: deb: - bison - flex - libelf-dev - libssl-dev - network-manager - iproute2 - util-linux - dump - indent - libaio-dev - libtool-bin - uuid-dev - xfslibs-dev - postgresql - postgresql-client - postgresql-common - postgresql-contrib rpm: - bison - flex - elfutils-libelf-devel - openssl-devel - NetworkManager - iproute - util-linux - libacl-devel - libaio-devel - libattr-devel - libtool - libuuid-devel - xfsdump - xfsprogs - xfsprogs-devel - libaio-devel - libtool - libuuid-devel - xfsprogs-devel - postgresql - postgresql-server - postgresql-contrib - cephadm: roleless: false - cephadm.shell: mon.a: - ceph orch status - ceph orch ps - ceph orch ls - ceph orch host ls - ceph orch device ls - cephadm.shell: mon.a: - ceph fs dump - ceph osd dump - fs.ready: timeout: 300 - sequential: - mgrmodules - kclient: null - exec: mon.a: - ceph fs set cephfs balance_automate true - check-counter: {} - fwd_scrub: scrub_timeout: 900 sleep_between_iterations: 1 - exec: mon.a: - ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m - ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - date +%s > START_TIME - full_sequential_finally: - exec: mon.a: - date +%s > END_TIME - START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi - ceph fs snap-schedule status --fs=cephfs --path=/ - ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true - background_exec: mon.a: - while sleep 13; do ceph tell mds.cephfs:0 flush journal; done - quiescer: null - workunit: clients: all: - fs/test_o_trunc.sh teuthology: fragments_dropped: [] meta: {} postmerge: - local function is_kupstream() return yaml.ktype == 'upstream' end - local function is_kdistro() return yaml.ktype == 'distro' end - local function is_fuse() return false end - local function syntax_version() return yaml.overrides.kclient.syntax end teuthology_branch: main teuthology_sha1: 9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17 timestamp: 2024-04-21_08:12:55 tube: smithi user: leonidus verbose: false worker_log: /home/teuthworker/archive/worker_logs/dispatcher.smithi.2226885 2024-04-21T08:20:59.273 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_leonid-s-usov_ceph-ceph_4f6bc5273643aaf5660909e85400647a8951c10b/qa; will attempt to use it 2024-04-21T08:20:59.274 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_leonid-s-usov_ceph-ceph_4f6bc5273643aaf5660909e85400647a8951c10b/qa/tasks 2024-04-21T08:20:59.274 INFO:teuthology.run_tasks:Running task internal.check_packages... 2024-04-21T08:20:59.276 INFO:teuthology.task.internal:Checking packages... 2024-04-21T08:20:59.300 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'default' and ceph hash 'a630765c1f93c6e61d9b40c4b76437611d5ef29e' 2024-04-21T08:20:59.300 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2024-04-21T08:20:59.300 INFO:teuthology.packaging:ref: None 2024-04-21T08:20:59.300 INFO:teuthology.packaging:tag: None 2024-04-21T08:20:59.300 INFO:teuthology.packaging:branch: wip-lusov-quiescer 2024-04-21T08:20:59.300 INFO:teuthology.packaging:sha1: a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:20:59.301 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&ref=wip-lusov-quiescer 2024-04-21T08:20:59.466 INFO:teuthology.task.internal:Found packages for ceph version 19.0.0-3154.ga630765c 2024-04-21T08:20:59.467 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2024-04-21T08:20:59.475 INFO:teuthology.task.internal:no buildpackages task found 2024-04-21T08:20:59.475 INFO:teuthology.run_tasks:Running task internal.save_config... 2024-04-21T08:20:59.493 INFO:teuthology.task.internal:Saving configuration 2024-04-21T08:20:59.521 INFO:teuthology.run_tasks:Running task internal.check_lock... 2024-04-21T08:20:59.527 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-21T08:20:59.551 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi086.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666511', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-21 08:15:32.418863', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBYfbXPoZ7D8BJKoi2VUL62N9ZlIfp6xJxHHCbXcCmEINmZC3+9zF50s3gdqfg271tP9z3/iq7Bx5j4Hf0/NLoU='} 2024-04-21T08:20:59.573 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi088.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666511', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-21 08:15:32.421991', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI1Hyc0xyjsQLdKgYCVOBgiyqDPLM5T0eiIiym5swZ7i9ORD0a10OvDztobC2iwb4Aa3ggytXWvbSMkahIgj4Ug='} 2024-04-21T08:20:59.596 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi179.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666511', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-21 08:15:32.420592', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPnr8CAdOQWzcezeu0TweNh/qMpja8vU18k8zRiz2xPUnqGVrCgFU078WM4ypon/wpPvxgc+yvadBIfb7w8XJTY='} 2024-04-21T08:20:59.597 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2024-04-21T08:20:59.603 INFO:teuthology.task.internal:roles: ubuntu@smithi086.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'mds.a', 'mds.d', 'mds.g', 'mds.j', 'osd.0', 'osd.3', 'osd.6', 'osd.9', 'client.0'] 2024-04-21T08:20:59.603 INFO:teuthology.task.internal:roles: ubuntu@smithi088.front.sepia.ceph.com - ['mon.b', 'mgr.y', 'mds.b', 'mds.e', 'mds.h', 'mds.k', 'osd.1', 'osd.4', 'osd.7', 'osd.10'] 2024-04-21T08:20:59.604 INFO:teuthology.task.internal:roles: ubuntu@smithi179.front.sepia.ceph.com - ['mon.c', 'mgr.z', 'mds.c', 'mds.f', 'mds.i', 'mds.l', 'osd.2', 'osd.5', 'osd.8', 'osd.11'] 2024-04-21T08:20:59.604 INFO:teuthology.run_tasks:Running task console_log... 2024-04-21T08:20:59.710 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7fa5a91a60d0>, signals=[15]) 2024-04-21T08:20:59.710 INFO:teuthology.run_tasks:Running task internal.connect... 2024-04-21T08:20:59.717 INFO:teuthology.task.internal:Opening connections... 2024-04-21T08:20:59.717 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi086.front.sepia.ceph.com 2024-04-21T08:20:59.719 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-21T08:20:59.796 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi088.front.sepia.ceph.com 2024-04-21T08:20:59.797 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-21T08:20:59.869 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi179.front.sepia.ceph.com 2024-04-21T08:20:59.870 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-21T08:20:59.943 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2024-04-21T08:20:59.950 DEBUG:teuthology.orchestra.run.smithi086:> uname -m 2024-04-21T08:20:59.969 INFO:teuthology.orchestra.run.smithi086.stdout:x86_64 2024-04-21T08:20:59.969 DEBUG:teuthology.orchestra.run.smithi086:> cat /etc/os-release 2024-04-21T08:21:00.027 INFO:teuthology.orchestra.run.smithi086.stdout:NAME="CentOS Stream" 2024-04-21T08:21:00.027 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION="9" 2024-04-21T08:21:00.027 INFO:teuthology.orchestra.run.smithi086.stdout:ID="centos" 2024-04-21T08:21:00.027 INFO:teuthology.orchestra.run.smithi086.stdout:ID_LIKE="rhel fedora" 2024-04-21T08:21:00.027 INFO:teuthology.orchestra.run.smithi086.stdout:VERSION_ID="9" 2024-04-21T08:21:00.027 INFO:teuthology.orchestra.run.smithi086.stdout:PLATFORM_ID="platform:el9" 2024-04-21T08:21:00.027 INFO:teuthology.orchestra.run.smithi086.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-21T08:21:00.027 INFO:teuthology.orchestra.run.smithi086.stdout:ANSI_COLOR="0;31" 2024-04-21T08:21:00.027 INFO:teuthology.orchestra.run.smithi086.stdout:LOGO="fedora-logo-icon" 2024-04-21T08:21:00.027 INFO:teuthology.orchestra.run.smithi086.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-21T08:21:00.027 INFO:teuthology.orchestra.run.smithi086.stdout:HOME_URL="https://centos.org/" 2024-04-21T08:21:00.027 INFO:teuthology.orchestra.run.smithi086.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-21T08:21:00.027 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-21T08:21:00.028 INFO:teuthology.orchestra.run.smithi086.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-21T08:21:00.028 INFO:teuthology.lock.ops:Updating smithi086.front.sepia.ceph.com on lock server 2024-04-21T08:21:00.045 DEBUG:teuthology.orchestra.run.smithi088:> uname -m 2024-04-21T08:21:00.064 INFO:teuthology.orchestra.run.smithi088.stdout:x86_64 2024-04-21T08:21:00.064 DEBUG:teuthology.orchestra.run.smithi088:> cat /etc/os-release 2024-04-21T08:21:00.121 INFO:teuthology.orchestra.run.smithi088.stdout:NAME="CentOS Stream" 2024-04-21T08:21:00.121 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION="9" 2024-04-21T08:21:00.121 INFO:teuthology.orchestra.run.smithi088.stdout:ID="centos" 2024-04-21T08:21:00.121 INFO:teuthology.orchestra.run.smithi088.stdout:ID_LIKE="rhel fedora" 2024-04-21T08:21:00.122 INFO:teuthology.orchestra.run.smithi088.stdout:VERSION_ID="9" 2024-04-21T08:21:00.122 INFO:teuthology.orchestra.run.smithi088.stdout:PLATFORM_ID="platform:el9" 2024-04-21T08:21:00.122 INFO:teuthology.orchestra.run.smithi088.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-21T08:21:00.122 INFO:teuthology.orchestra.run.smithi088.stdout:ANSI_COLOR="0;31" 2024-04-21T08:21:00.122 INFO:teuthology.orchestra.run.smithi088.stdout:LOGO="fedora-logo-icon" 2024-04-21T08:21:00.122 INFO:teuthology.orchestra.run.smithi088.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-21T08:21:00.122 INFO:teuthology.orchestra.run.smithi088.stdout:HOME_URL="https://centos.org/" 2024-04-21T08:21:00.122 INFO:teuthology.orchestra.run.smithi088.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-21T08:21:00.122 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-21T08:21:00.122 INFO:teuthology.orchestra.run.smithi088.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-21T08:21:00.123 INFO:teuthology.lock.ops:Updating smithi088.front.sepia.ceph.com on lock server 2024-04-21T08:21:00.137 DEBUG:teuthology.orchestra.run.smithi179:> uname -m 2024-04-21T08:21:00.155 INFO:teuthology.orchestra.run.smithi179.stdout:x86_64 2024-04-21T08:21:00.155 DEBUG:teuthology.orchestra.run.smithi179:> cat /etc/os-release 2024-04-21T08:21:00.212 INFO:teuthology.orchestra.run.smithi179.stdout:NAME="CentOS Stream" 2024-04-21T08:21:00.212 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION="9" 2024-04-21T08:21:00.212 INFO:teuthology.orchestra.run.smithi179.stdout:ID="centos" 2024-04-21T08:21:00.212 INFO:teuthology.orchestra.run.smithi179.stdout:ID_LIKE="rhel fedora" 2024-04-21T08:21:00.213 INFO:teuthology.orchestra.run.smithi179.stdout:VERSION_ID="9" 2024-04-21T08:21:00.213 INFO:teuthology.orchestra.run.smithi179.stdout:PLATFORM_ID="platform:el9" 2024-04-21T08:21:00.213 INFO:teuthology.orchestra.run.smithi179.stdout:PRETTY_NAME="CentOS Stream 9" 2024-04-21T08:21:00.213 INFO:teuthology.orchestra.run.smithi179.stdout:ANSI_COLOR="0;31" 2024-04-21T08:21:00.213 INFO:teuthology.orchestra.run.smithi179.stdout:LOGO="fedora-logo-icon" 2024-04-21T08:21:00.213 INFO:teuthology.orchestra.run.smithi179.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2024-04-21T08:21:00.213 INFO:teuthology.orchestra.run.smithi179.stdout:HOME_URL="https://centos.org/" 2024-04-21T08:21:00.213 INFO:teuthology.orchestra.run.smithi179.stdout:BUG_REPORT_URL="https://bugzilla.redhat.com/" 2024-04-21T08:21:00.213 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2024-04-21T08:21:00.213 INFO:teuthology.orchestra.run.smithi179.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2024-04-21T08:21:00.213 INFO:teuthology.lock.ops:Updating smithi179.front.sepia.ceph.com on lock server 2024-04-21T08:21:00.237 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2024-04-21T08:21:00.248 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2024-04-21T08:21:00.276 INFO:teuthology.task.internal:Checking for old test directory... 2024-04-21T08:21:00.276 DEBUG:teuthology.orchestra.run.smithi086:> test '!' -e /home/ubuntu/cephtest 2024-04-21T08:21:00.279 DEBUG:teuthology.orchestra.run.smithi088:> test '!' -e /home/ubuntu/cephtest 2024-04-21T08:21:00.281 DEBUG:teuthology.orchestra.run.smithi179:> test '!' -e /home/ubuntu/cephtest 2024-04-21T08:21:00.297 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2024-04-21T08:21:00.303 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2024-04-21T08:21:00.303 DEBUG:teuthology.orchestra.run.smithi086:> test -z $(ls -A /var/lib/ceph) 2024-04-21T08:21:00.336 DEBUG:teuthology.orchestra.run.smithi088:> test -z $(ls -A /var/lib/ceph) 2024-04-21T08:21:00.338 DEBUG:teuthology.orchestra.run.smithi179:> test -z $(ls -A /var/lib/ceph) 2024-04-21T08:21:00.444 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2024-04-21T08:21:00.524 INFO:teuthology.run_tasks:Running task kernel... 2024-04-21T08:21:00.541 INFO:teuthology.task.kernel:normalize config orig: {'client': {'sha1': 'distro'}, 'kdb': True, 'sha1': 'distro'} 2024-04-21T08:21:00.541 INFO:teuthology.task.kernel:normalize config final: {'client.0': {'sha1': 'distro'}} 2024-04-21T08:21:00.542 INFO:teuthology.task.kernel:config {'client.0': {'sha1': 'distro'}}, timeout 300 2024-04-21T08:21:00.542 DEBUG:teuthology.orchestra.run.smithi086:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-21T08:21:00.558 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T08:21:00.558 DEBUG:teuthology.orchestra.run.smithi086:> uname -r 2024-04-21T08:21:00.616 INFO:teuthology.orchestra.run.smithi086.stdout:5.14.0-437.el9.x86_64 2024-04-21T08:21:00.616 INFO:teuthology.task.kernel:Running kernel on smithi086: 5.14.0-437.el9.x86_64 2024-04-21T08:21:00.617 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum install -y kernel 2024-04-21T08:21:03.075 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - BaseOS 9.1 MB/s | 8.0 MB 00:00 2024-04-21T08:21:04.951 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - AppStream 24 MB/s | 19 MB 00:00 2024-04-21T08:21:11.118 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - CRB 2.4 MB/s | 6.2 MB 00:02 2024-04-21T08:21:12.702 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - Extras packages 69 kB/s | 16 kB 00:00 2024-04-21T08:21:14.780 INFO:teuthology.orchestra.run.smithi086.stdout:Extra Packages for Enterprise Linux 11 MB/s | 21 MB 00:01 2024-04-21T08:21:19.836 INFO:teuthology.orchestra.run.smithi086.stdout:lab-extras 13 kB/s | 1.7 kB 00:00 2024-04-21T08:21:21.763 INFO:teuthology.orchestra.run.smithi086.stdout:Package kernel-5.14.0-239.el9.x86_64 is already installed. 2024-04-21T08:21:21.764 INFO:teuthology.orchestra.run.smithi086.stdout:Package kernel-5.14.0-437.el9.x86_64 is already installed. 2024-04-21T08:21:21.842 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-04-21T08:21:21.842 INFO:teuthology.orchestra.run.smithi086.stdout:Nothing to do. 2024-04-21T08:21:21.843 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-04-21T08:21:21.969 DEBUG:teuthology.orchestra.run.smithi086:> echo no | sudo yum reinstall kernel || true 2024-04-21T08:21:22.466 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:00:03 ago on Sun 21 Apr 2024 08:21:19 AM UTC. 2024-04-21T08:21:22.591 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-04-21T08:21:22.610 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-04-21T08:21:22.610 INFO:teuthology.orchestra.run.smithi086.stdout: Package Architecture Version Repository Size 2024-04-21T08:21:22.610 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-04-21T08:21:22.610 INFO:teuthology.orchestra.run.smithi086.stdout:Reinstalling: 2024-04-21T08:21:22.610 INFO:teuthology.orchestra.run.smithi086.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-21T08:21:22.610 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:21:22.610 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-04-21T08:21:22.610 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-04-21T08:21:22.610 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:21:22.611 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 255 k 2024-04-21T08:21:22.611 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 0 2024-04-21T08:21:22.611 INFO:teuthology.orchestra.run.smithi086.stderr:Operation aborted. 2024-04-21T08:21:22.675 INFO:teuthology.orchestra.run.smithi086.stdout:Is this ok [y/N]: 2024-04-21T08:21:22.675 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum reinstall -y kernel || true 2024-04-21T08:21:23.163 INFO:teuthology.orchestra.run.smithi086.stdout:Last metadata expiration check: 0:00:04 ago on Sun 21 Apr 2024 08:21:19 AM UTC. 2024-04-21T08:21:23.278 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-04-21T08:21:23.279 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-04-21T08:21:23.279 INFO:teuthology.orchestra.run.smithi086.stdout: Package Architecture Version Repository Size 2024-04-21T08:21:23.279 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-04-21T08:21:23.279 INFO:teuthology.orchestra.run.smithi086.stdout:Reinstalling: 2024-04-21T08:21:23.279 INFO:teuthology.orchestra.run.smithi086.stdout: kernel x86_64 5.14.0-437.el9 baseos 255 k 2024-04-21T08:21:23.279 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:21:23.280 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-04-21T08:21:23.280 INFO:teuthology.orchestra.run.smithi086.stdout:================================================================================ 2024-04-21T08:21:23.280 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:21:23.280 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 255 k 2024-04-21T08:21:23.280 INFO:teuthology.orchestra.run.smithi086.stdout:Installed size: 0 2024-04-21T08:21:23.280 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-04-21T08:21:23.663 INFO:teuthology.orchestra.run.smithi086.stdout:kernel-5.14.0-437.el9.x86_64.rpm 2.2 MB/s | 255 kB 00:00 2024-04-21T08:21:23.663 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-04-21T08:21:23.664 INFO:teuthology.orchestra.run.smithi086.stdout:Total 663 kB/s | 255 kB 00:00 2024-04-21T08:21:23.680 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-04-21T08:21:23.854 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-04-21T08:21:23.854 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-04-21T08:21:23.859 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-04-21T08:21:23.859 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-04-21T08:21:23.988 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-04-21T08:21:24.056 INFO:teuthology.orchestra.run.smithi086.stdout: Reinstalling : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-21T08:21:24.056 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-21T08:21:24.743 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: kernel-5.14.0-437.el9.x86_64 2/2 2024-04-21T08:21:24.743 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 1/2 2024-04-21T08:21:25.437 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : kernel-5.14.0-437.el9.x86_64 2/2 2024-04-21T08:21:25.437 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:21:25.437 INFO:teuthology.orchestra.run.smithi086.stdout:Reinstalled: 2024-04-21T08:21:25.437 INFO:teuthology.orchestra.run.smithi086.stdout: kernel-5.14.0-437.el9.x86_64 2024-04-21T08:21:25.437 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:21:25.437 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-04-21T08:21:25.603 DEBUG:teuthology.orchestra.run.smithi086:> rpm -q kernel | sort -rV | head -n 1 2024-04-21T08:21:25.651 INFO:teuthology.orchestra.run.smithi086.stdout:kernel-5.14.0-437.el9.x86_64 2024-04-21T08:21:25.651 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-437.el9.x86_64 2024-04-21T08:21:25.652 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2024-04-21T08:21:25.652 DEBUG:teuthology.parallel:result is None 2024-04-21T08:21:25.652 INFO:teuthology.run_tasks:Running task internal.base... 2024-04-21T08:21:25.661 INFO:teuthology.task.internal:Creating test directory... 2024-04-21T08:21:25.661 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-21T08:21:25.664 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-21T08:21:25.667 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2024-04-21T08:21:25.686 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2024-04-21T08:21:25.695 INFO:teuthology.run_tasks:Running task internal.archive... 2024-04-21T08:21:25.705 INFO:teuthology.task.internal:Creating archive directory... 2024-04-21T08:21:25.706 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-21T08:21:25.720 DEBUG:teuthology.orchestra.run.smithi088:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-21T08:21:25.725 DEBUG:teuthology.orchestra.run.smithi179:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2024-04-21T08:21:26.028 INFO:teuthology.run_tasks:Running task internal.coredump... 2024-04-21T08:21:26.036 INFO:teuthology.task.internal:Enabling coredump saving... 2024-04-21T08:21:26.036 DEBUG:teuthology.orchestra.run.smithi088:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-21T08:21:26.053 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T08:21:26.053 DEBUG:teuthology.orchestra.run.smithi179:> test -f /run/.containerenv -o -f /.dockerenv 2024-04-21T08:21:26.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T08:21:26.070 DEBUG:teuthology.orchestra.run.smithi086:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-21T08:21:26.073 DEBUG:teuthology.orchestra.run.smithi088:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-21T08:21:26.097 DEBUG:teuthology.orchestra.run.smithi179:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2024-04-21T08:21:26.126 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-21T08:21:26.138 INFO:teuthology.orchestra.run.smithi088.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-21T08:21:26.144 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-21T08:21:26.146 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-21T08:21:26.163 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-21T08:21:26.177 INFO:teuthology.orchestra.run.smithi088.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2024-04-21T08:21:26.179 INFO:teuthology.run_tasks:Running task internal.sudo... 2024-04-21T08:21:26.187 INFO:teuthology.task.internal:Configuring sudo... 2024-04-21T08:21:26.187 DEBUG:teuthology.orchestra.run.smithi086:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-21T08:21:26.189 DEBUG:teuthology.orchestra.run.smithi088:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-21T08:21:26.222 DEBUG:teuthology.orchestra.run.smithi179:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2024-04-21T08:21:26.248 INFO:teuthology.run_tasks:Running task internal.syslog... 2024-04-21T08:21:26.258 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2024-04-21T08:21:26.258 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-21T08:21:26.261 DEBUG:teuthology.orchestra.run.smithi088:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-21T08:21:26.288 DEBUG:teuthology.orchestra.run.smithi179:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2024-04-21T08:21:26.304 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-21T08:21:26.354 DEBUG:teuthology.orchestra.run.smithi086:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-21T08:21:26.397 DEBUG:teuthology.orchestra.run.smithi086:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-21T08:21:26.479 DEBUG:teuthology.orchestra.run.smithi086:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-21T08:21:26.516 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-21T08:21:26.517 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-21T08:21:26.580 DEBUG:teuthology.orchestra.run.smithi088:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-21T08:21:26.622 DEBUG:teuthology.orchestra.run.smithi088:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-21T08:21:26.674 DEBUG:teuthology.orchestra.run.smithi088:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-21T08:21:26.777 DEBUG:teuthology.orchestra.run.smithi088:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-21T08:21:26.811 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-21T08:21:26.811 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-21T08:21:26.885 DEBUG:teuthology.orchestra.run.smithi179:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-21T08:21:26.930 DEBUG:teuthology.orchestra.run.smithi179:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2024-04-21T08:21:26.981 DEBUG:teuthology.orchestra.run.smithi179:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-21T08:21:27.067 DEBUG:teuthology.orchestra.run.smithi179:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2024-04-21T08:21:27.110 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-21T08:21:27.110 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2024-04-21T08:21:27.177 DEBUG:teuthology.orchestra.run.smithi086:> sudo service rsyslog restart 2024-04-21T08:21:27.179 DEBUG:teuthology.orchestra.run.smithi088:> sudo service rsyslog restart 2024-04-21T08:21:27.182 DEBUG:teuthology.orchestra.run.smithi179:> sudo service rsyslog restart 2024-04-21T08:21:27.219 INFO:teuthology.orchestra.run.smithi086.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-21T08:21:27.226 INFO:teuthology.orchestra.run.smithi088.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-21T08:21:27.255 INFO:teuthology.orchestra.run.smithi179.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-21T08:21:27.646 INFO:teuthology.run_tasks:Running task internal.timer... 2024-04-21T08:21:27.669 INFO:teuthology.task.internal:Starting timer... 2024-04-21T08:21:27.669 INFO:teuthology.run_tasks:Running task pcp... 2024-04-21T08:21:27.680 INFO:teuthology.run_tasks:Running task selinux... 2024-04-21T08:21:27.689 DEBUG:teuthology.task:Applying overrides for task selinux: {'allowlist': ['scontext=system_u:system_r:getty_t:s0']} 2024-04-21T08:21:27.689 DEBUG:teuthology.orchestra.run.smithi086:> sudo service auditd rotate 2024-04-21T08:21:27.730 INFO:teuthology.orchestra.run.smithi086.stdout:Rotating logs: 2024-04-21T08:21:27.732 DEBUG:teuthology.orchestra.run.smithi088:> sudo service auditd rotate 2024-04-21T08:21:27.797 INFO:teuthology.orchestra.run.smithi088.stdout:Rotating logs: 2024-04-21T08:21:27.798 DEBUG:teuthology.orchestra.run.smithi179:> sudo service auditd rotate 2024-04-21T08:21:27.850 INFO:teuthology.orchestra.run.smithi179.stdout:Rotating logs: 2024-04-21T08:21:27.852 DEBUG:teuthology.task.selinux:Getting current SELinux state 2024-04-21T08:21:27.852 DEBUG:teuthology.orchestra.run.smithi086:> /usr/sbin/getenforce 2024-04-21T08:21:27.877 INFO:teuthology.orchestra.run.smithi086.stdout:Permissive 2024-04-21T08:21:27.877 DEBUG:teuthology.orchestra.run.smithi088:> /usr/sbin/getenforce 2024-04-21T08:21:27.906 INFO:teuthology.orchestra.run.smithi088.stdout:Permissive 2024-04-21T08:21:27.906 DEBUG:teuthology.orchestra.run.smithi179:> /usr/sbin/getenforce 2024-04-21T08:21:27.939 INFO:teuthology.orchestra.run.smithi179.stdout:Permissive 2024-04-21T08:21:27.939 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi086.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi088.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi179.front.sepia.ceph.com': 'permissive'} 2024-04-21T08:21:27.939 DEBUG:teuthology.orchestra.run.smithi086:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-21T08:21:27.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T08:21:27.963 DEBUG:teuthology.orchestra.run.smithi088:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-21T08:21:27.990 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T08:21:27.991 DEBUG:teuthology.orchestra.run.smithi179:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-21T08:21:28.017 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T08:21:28.018 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2024-04-21T08:21:28.018 DEBUG:teuthology.orchestra.run.smithi086:> sudo /usr/sbin/setenforce permissive 2024-04-21T08:21:28.041 DEBUG:teuthology.orchestra.run.smithi088:> sudo /usr/sbin/setenforce permissive 2024-04-21T08:21:28.068 DEBUG:teuthology.orchestra.run.smithi179:> sudo /usr/sbin/setenforce permissive 2024-04-21T08:21:28.100 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2024-04-21T08:21:28.110 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2024-04-21T08:21:28.115 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2024-04-21T08:21:28.253 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2024-04-21T08:21:28.269 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2024-04-21T08:21:28.271 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi086.front.sepia.ceph.com,smithi088.front.sepia.ceph.com,smithi179.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2024-04-21T08:25:22.131 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi086.front.sepia.ceph.com'), Remote(name='ubuntu@smithi088.front.sepia.ceph.com'), Remote(name='ubuntu@smithi179.front.sepia.ceph.com')] 2024-04-21T08:25:22.133 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-21T08:25:22.134 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi086.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-21T08:25:22.221 DEBUG:teuthology.orchestra.run.smithi086:> true 2024-04-21T08:25:22.301 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi086.front.sepia.ceph.com' 2024-04-21T08:25:22.302 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-04-21T08:25:22.302 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi088.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-21T08:25:22.377 DEBUG:teuthology.orchestra.run.smithi088:> true 2024-04-21T08:25:22.454 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi088.front.sepia.ceph.com' 2024-04-21T08:25:22.455 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-04-21T08:25:22.455 DEBUG:teuthology.orchestra.connection:{'hostname': 'smithi179.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2024-04-21T08:25:22.537 DEBUG:teuthology.orchestra.run.smithi179:> true 2024-04-21T08:25:22.624 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@smithi179.front.sepia.ceph.com' 2024-04-21T08:25:22.624 INFO:teuthology.run_tasks:Running task clock... 2024-04-21T08:25:22.639 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2024-04-21T08:25:22.640 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-21T08:25:22.640 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-21T08:25:22.642 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-21T08:25:22.643 DEBUG:teuthology.orchestra.run.smithi088:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-21T08:25:22.645 INFO:teuthology.orchestra.run:Running command with timeout 360 2024-04-21T08:25:22.645 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-21T08:25:22.671 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-21T08:25:22.672 INFO:teuthology.orchestra.run.smithi088.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-21T08:25:22.686 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-21T08:25:22.686 INFO:teuthology.orchestra.run.smithi088.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-21T08:25:22.695 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2024-04-21T08:25:22.710 INFO:teuthology.orchestra.run.smithi088.stderr:sudo: ntpd: command not found 2024-04-21T08:25:22.710 INFO:teuthology.orchestra.run.smithi086.stderr:sudo: ntpd: command not found 2024-04-21T08:25:22.710 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2024-04-21T08:25:22.721 INFO:teuthology.orchestra.run.smithi088.stdout:506 Cannot talk to daemon 2024-04-21T08:25:22.721 INFO:teuthology.orchestra.run.smithi086.stdout:506 Cannot talk to daemon 2024-04-21T08:25:22.734 INFO:teuthology.orchestra.run.smithi088.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-21T08:25:22.734 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-21T08:25:22.735 INFO:teuthology.orchestra.run.smithi179.stderr:sudo: ntpd: command not found 2024-04-21T08:25:22.745 INFO:teuthology.orchestra.run.smithi179.stdout:506 Cannot talk to daemon 2024-04-21T08:25:22.746 INFO:teuthology.orchestra.run.smithi088.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-21T08:25:22.748 INFO:teuthology.orchestra.run.smithi086.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-21T08:25:22.759 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to start ntp.service: Unit ntp.service not found. 2024-04-21T08:25:22.772 INFO:teuthology.orchestra.run.smithi179.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2024-04-21T08:25:22.799 INFO:teuthology.orchestra.run.smithi088.stderr:bash: line 1: ntpq: command not found 2024-04-21T08:25:22.801 INFO:teuthology.orchestra.run.smithi088.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-21T08:25:22.801 INFO:teuthology.orchestra.run.smithi088.stdout:=============================================================================== 2024-04-21T08:25:22.801 INFO:teuthology.orchestra.run.smithi088.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:25:22.802 INFO:teuthology.orchestra.run.smithi088.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:25:22.802 INFO:teuthology.orchestra.run.smithi088.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:25:22.802 INFO:teuthology.orchestra.run.smithi088.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:25:22.802 INFO:teuthology.orchestra.run.smithi086.stderr:bash: line 1: ntpq: command not found 2024-04-21T08:25:22.806 INFO:teuthology.orchestra.run.smithi086.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-21T08:25:22.806 INFO:teuthology.orchestra.run.smithi086.stdout:=============================================================================== 2024-04-21T08:25:22.806 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:25:22.806 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:25:22.806 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:25:22.806 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:25:22.825 INFO:teuthology.orchestra.run.smithi179.stderr:bash: line 1: ntpq: command not found 2024-04-21T08:25:22.828 INFO:teuthology.orchestra.run.smithi179.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-21T08:25:22.828 INFO:teuthology.orchestra.run.smithi179.stdout:=============================================================================== 2024-04-21T08:25:22.828 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv01.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:25:22.828 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv02.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:25:22.828 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv03.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:25:22.828 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv04.front.sepia.ceph.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:25:22.829 INFO:teuthology.run_tasks:Running task install... 2024-04-21T08:25:22.838 DEBUG:teuthology.task.install:project ceph 2024-04-21T08:25:22.838 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'default', 'sha1': 'a630765c1f93c6e61d9b40c4b76437611d5ef29e'}} 2024-04-21T08:25:22.838 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'flavor': 'default', 'sha1': 'a630765c1f93c6e61d9b40c4b76437611d5ef29e'} 2024-04-21T08:25:22.838 INFO:teuthology.task.install:Using flavor: default 2024-04-21T08:25:22.843 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2024-04-21T08:25:22.843 INFO:teuthology.task.install:extra packages: {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']} 2024-04-21T08:25:22.844 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a630765c1f93c6e61d9b40c4b76437611d5ef29e', 'tag': None, 'wait_for_package': False} 2024-04-21T08:25:22.844 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:25:22.846 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a630765c1f93c6e61d9b40c4b76437611d5ef29e', 'tag': None, 'wait_for_package': False} 2024-04-21T08:25:22.846 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:25:22.847 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror'], 'rpm': ['python3-cephfs', 'cephfs-shell', 'cephfs-top', 'cephfs-mirror']}, 'extra_system_packages': {'deb': ['bison', 'flex', 'libelf-dev', 'libssl-dev', 'network-manager', 'iproute2', 'util-linux', 'dump', 'indent', 'libaio-dev', 'libtool-bin', 'uuid-dev', 'xfslibs-dev', 'postgresql', 'postgresql-client', 'postgresql-common', 'postgresql-contrib'], 'rpm': ['bison', 'flex', 'elfutils-libelf-devel', 'openssl-devel', 'NetworkManager', 'iproute', 'util-linux', 'libacl-devel', 'libaio-devel', 'libattr-devel', 'libtool', 'libuuid-devel', 'xfsdump', 'xfsprogs', 'xfsprogs-devel', 'libaio-devel', 'libtool', 'libuuid-devel', 'xfsprogs-devel', 'postgresql', 'postgresql-server', 'postgresql-contrib']}, 'extras': None, 'enable_coprs': [], 'flavor': 'default', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'a630765c1f93c6e61d9b40c4b76437611d5ef29e', 'tag': None, 'wait_for_package': False} 2024-04-21T08:25:22.847 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:25:23.077 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/ 2024-04-21T08:25:23.078 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3154.ga630765c 2024-04-21T08:25:23.081 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/ 2024-04-21T08:25:23.081 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3154.ga630765c 2024-04-21T08:25:23.084 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/ 2024-04-21T08:25:23.084 INFO:teuthology.task.install.rpm:Package version is 19.0.0-3154.ga630765c 2024-04-21T08:25:23.250 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-21T08:25:23.251 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-21T08:25:23.251 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-21T08:25:23.275 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-21T08:25:23.275 DEBUG:teuthology.orchestra.run.smithi086:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a630765c1f93c6e61d9b40c4b76437611d5ef29e/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-21T08:25:23.328 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-21T08:25:23.328 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-21T08:25:23.328 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-21T08:25:23.338 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2024-04-21T08:25:23.338 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-21T08:25:23.338 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/yum.repos.d/ceph.repo 2024-04-21T08:25:23.339 DEBUG:teuthology.orchestra.run.smithi086:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-21T08:25:23.354 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-21T08:25:23.354 DEBUG:teuthology.orchestra.run.smithi179:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a630765c1f93c6e61d9b40c4b76437611d5ef29e/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-21T08:25:23.364 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-cephfs, cephfs-shell, cephfs-top, cephfs-mirror, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib, bison, flex, elfutils-libelf-devel, openssl-devel, NetworkManager, iproute, util-linux, libacl-devel, libaio-devel, libattr-devel, libtool, libuuid-devel, xfsdump, xfsprogs, xfsprogs-devel, libaio-devel, libtool, libuuid-devel, xfsprogs-devel, postgresql, postgresql-server, postgresql-contrib on remote rpm x86_64 2024-04-21T08:25:23.365 DEBUG:teuthology.orchestra.run.smithi088:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/a630765c1f93c6e61d9b40c4b76437611d5ef29e/;g' /etc/yum.repos.d/ceph.repo ; fi 2024-04-21T08:25:23.412 DEBUG:teuthology.orchestra.run.smithi086:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-21T08:25:23.420 DEBUG:teuthology.orchestra.run.smithi179:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-21T08:25:23.430 DEBUG:teuthology.orchestra.run.smithi088:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2024-04-21T08:25:23.475 INFO:teuthology.orchestra.run.smithi086.stdout:check_obsoletes = 1 2024-04-21T08:25:23.477 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum clean all 2024-04-21T08:25:23.493 DEBUG:teuthology.orchestra.run.smithi179:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-21T08:25:23.504 DEBUG:teuthology.orchestra.run.smithi088:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2024-04-21T08:25:23.557 INFO:teuthology.orchestra.run.smithi179.stdout:check_obsoletes = 1 2024-04-21T08:25:23.559 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum clean all 2024-04-21T08:25:23.567 INFO:teuthology.orchestra.run.smithi088.stdout:check_obsoletes = 1 2024-04-21T08:25:23.568 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum clean all 2024-04-21T08:25:23.703 INFO:teuthology.orchestra.run.smithi086.stdout:48 files removed 2024-04-21T08:25:23.731 DEBUG:teuthology.orchestra.run.smithi086:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-21T08:25:23.784 INFO:teuthology.orchestra.run.smithi179.stdout:48 files removed 2024-04-21T08:25:23.791 INFO:teuthology.orchestra.run.smithi088.stdout:48 files removed 2024-04-21T08:25:23.811 DEBUG:teuthology.orchestra.run.smithi179:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-21T08:25:23.818 DEBUG:teuthology.orchestra.run.smithi088:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-cephfs cephfs-shell cephfs-top cephfs-mirror bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib bison flex elfutils-libelf-devel openssl-devel NetworkManager iproute util-linux libacl-devel libaio-devel libattr-devel libtool libuuid-devel xfsdump xfsprogs xfsprogs-devel libaio-devel libtool libuuid-devel xfsprogs-devel postgresql postgresql-server postgresql-contrib 2024-04-21T08:25:24.741 INFO:teuthology.orchestra.run.smithi086.stdout:ceph packages for x86_64 111 kB/s | 83 kB 00:00 2024-04-21T08:25:24.796 INFO:teuthology.orchestra.run.smithi179.stdout:ceph packages for x86_64 115 kB/s | 83 kB 00:00 2024-04-21T08:25:24.836 INFO:teuthology.orchestra.run.smithi088.stdout:ceph packages for x86_64 110 kB/s | 83 kB 00:00 2024-04-21T08:25:25.258 INFO:teuthology.orchestra.run.smithi086.stdout:ceph noarch packages 25 kB/s | 12 kB 00:00 2024-04-21T08:25:25.328 INFO:teuthology.orchestra.run.smithi088.stdout:ceph noarch packages 26 kB/s | 12 kB 00:00 2024-04-21T08:25:25.422 INFO:teuthology.orchestra.run.smithi179.stdout:ceph noarch packages 20 kB/s | 12 kB 00:00 2024-04-21T08:25:25.867 INFO:teuthology.orchestra.run.smithi086.stdout:ceph source packages 3.3 kB/s | 1.9 kB 00:00 2024-04-21T08:25:25.987 INFO:teuthology.orchestra.run.smithi088.stdout:ceph source packages 3.0 kB/s | 1.9 kB 00:00 2024-04-21T08:25:26.022 INFO:teuthology.orchestra.run.smithi179.stdout:ceph source packages 3.4 kB/s | 1.9 kB 00:00 2024-04-21T08:25:26.400 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.0 MB 00:00 2024-04-21T08:25:26.529 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - BaseOS 16 MB/s | 8.0 MB 00:00 2024-04-21T08:25:27.581 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - BaseOS 5.3 MB/s | 8.0 MB 00:01 2024-04-21T08:25:28.443 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - AppStream 19 MB/s | 19 MB 00:00 2024-04-21T08:25:29.229 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - AppStream 12 MB/s | 19 MB 00:01 2024-04-21T08:25:32.670 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - CRB 8.7 MB/s | 6.2 MB 00:00 2024-04-21T08:25:33.779 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - CRB 6.2 MB/s | 6.2 MB 00:00 2024-04-21T08:25:34.020 INFO:teuthology.orchestra.run.smithi086.stdout:CentOS Stream 9 - Extras packages 70 kB/s | 16 kB 00:00 2024-04-21T08:25:35.490 INFO:teuthology.orchestra.run.smithi088.stdout:CentOS Stream 9 - Extras packages 29 kB/s | 16 kB 00:00 2024-04-21T08:25:35.714 INFO:teuthology.orchestra.run.smithi086.stdout:Extra Packages for Enterprise Linux 13 MB/s | 21 MB 00:01 2024-04-21T08:25:36.817 INFO:teuthology.orchestra.run.smithi088.stdout:Extra Packages for Enterprise Linux 17 MB/s | 21 MB 00:01 2024-04-21T08:25:40.721 INFO:teuthology.orchestra.run.smithi086.stdout:lab-extras 29 kB/s | 1.7 kB 00:00 2024-04-21T08:25:41.822 INFO:teuthology.orchestra.run.smithi088.stdout:lab-extras 16 kB/s | 1.7 kB 00:00 2024-04-21T08:25:42.872 INFO:teuthology.orchestra.run.smithi086.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-21T08:25:42.873 INFO:teuthology.orchestra.run.smithi086.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-21T08:25:42.883 INFO:teuthology.orchestra.run.smithi086.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-21T08:25:42.884 INFO:teuthology.orchestra.run.smithi086.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-21T08:25:42.885 INFO:teuthology.orchestra.run.smithi086.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-21T08:25:42.886 INFO:teuthology.orchestra.run.smithi086.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-21T08:25:42.887 INFO:teuthology.orchestra.run.smithi086.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-21T08:25:42.889 INFO:teuthology.orchestra.run.smithi086.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-21T08:25:42.890 INFO:teuthology.orchestra.run.smithi086.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-21T08:25:42.949 INFO:teuthology.orchestra.run.smithi086.stdout:Dependencies resolved. 2024-04-21T08:25:42.958 INFO:teuthology.orchestra.run.smithi086.stdout:======================================================================================= 2024-04-21T08:25:42.958 INFO:teuthology.orchestra.run.smithi086.stdout: Package Arch Version Repository Size 2024-04-21T08:25:42.958 INFO:teuthology.orchestra.run.smithi086.stdout:======================================================================================= 2024-04-21T08:25:42.958 INFO:teuthology.orchestra.run.smithi086.stdout:Installing: 2024-04-21T08:25:42.958 INFO:teuthology.orchestra.run.smithi086.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-21T08:25:42.958 INFO:teuthology.orchestra.run.smithi086.stdout: ceph x86_64 2:19.0.0-3154.ga630765c.el9 ceph 6.6 k 2024-04-21T08:25:42.959 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base x86_64 2:19.0.0-3154.ga630765c.el9 ceph 5.4 M 2024-04-21T08:25:42.959 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-fuse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 890 k 2024-04-21T08:25:42.959 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3154.ga630765c.el9 ceph 145 k 2024-04-21T08:25:42.959 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr x86_64 2:19.0.0-3154.ga630765c.el9 ceph 1.5 M 2024-04-21T08:25:42.959 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 142 k 2024-04-21T08:25:42.959 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 3.5 M 2024-04-21T08:25:42.959 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 7.4 M 2024-04-21T08:25:42.959 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-rook noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 49 k 2024-04-21T08:25:42.959 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-radosgw x86_64 2:19.0.0-3154.ga630765c.el9 ceph 11 M 2024-04-21T08:25:42.959 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-test x86_64 2:19.0.0-3154.ga630765c.el9 ceph 49 M 2024-04-21T08:25:42.959 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-volume noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 273 k 2024-04-21T08:25:42.959 INFO:teuthology.orchestra.run.smithi086.stdout: cephadm noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 762 k 2024-04-21T08:25:42.959 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-mirror x86_64 2:19.0.0-3154.ga630765c.el9 ceph 225 k 2024-04-21T08:25:42.959 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-shell x86_64 2:19.0.0-3154.ga630765c.el9 ceph 27 k 2024-04-21T08:25:42.960 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-top noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 18 k 2024-04-21T08:25:42.960 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-21T08:25:42.960 INFO:teuthology.orchestra.run.smithi086.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-21T08:25:42.960 INFO:teuthology.orchestra.run.smithi086.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-21T08:25:42.960 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs-devel x86_64 2:19.0.0-3154.ga630765c.el9 ceph 32 k 2024-04-21T08:25:42.960 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 746 k 2024-04-21T08:25:42.960 INFO:teuthology.orchestra.run.smithi086.stdout: librados-devel x86_64 2:19.0.0-3154.ga630765c.el9 ceph 127 k 2024-04-21T08:25:42.960 INFO:teuthology.orchestra.run.smithi086.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-21T08:25:42.960 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-21T08:25:42.960 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-21T08:25:42.960 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-21T08:25:42.960 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-21T08:25:42.960 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-21T08:25:42.960 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cephfs x86_64 2:19.0.0-3154.ga630765c.el9 ceph 162 k 2024-04-21T08:25:42.961 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rados x86_64 2:19.0.0-3154.ga630765c.el9 ceph 321 k 2024-04-21T08:25:42.961 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rbd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 298 k 2024-04-21T08:25:42.961 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rgw x86_64 2:19.0.0-3154.ga630765c.el9 ceph 100 k 2024-04-21T08:25:42.961 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-fuse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 85 k 2024-04-21T08:25:42.961 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-mirror x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.1 M 2024-04-21T08:25:42.961 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-nbd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 171 k 2024-04-21T08:25:42.961 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-21T08:25:42.961 INFO:teuthology.orchestra.run.smithi086.stdout:Upgrading: 2024-04-21T08:25:42.961 INFO:teuthology.orchestra.run.smithi086.stdout: librados2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.4 M 2024-04-21T08:25:42.961 INFO:teuthology.orchestra.run.smithi086.stdout: librbd1 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.1 M 2024-04-21T08:25:42.961 INFO:teuthology.orchestra.run.smithi086.stdout:Installing dependencies: 2024-04-21T08:25:42.961 INFO:teuthology.orchestra.run.smithi086.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-21T08:25:42.961 INFO:teuthology.orchestra.run.smithi086.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-21T08:25:42.961 INFO:teuthology.orchestra.run.smithi086.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-21T08:25:42.962 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-common x86_64 2:19.0.0-3154.ga630765c.el9 ceph 21 M 2024-04-21T08:25:42.962 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 34 k 2024-04-21T08:25:42.962 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mds x86_64 2:19.0.0-3154.ga630765c.el9 ceph 2.4 M 2024-04-21T08:25:42.962 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 249 k 2024-04-21T08:25:42.962 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mon x86_64 2:19.0.0-3154.ga630765c.el9 ceph 4.7 M 2024-04-21T08:25:42.962 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-osd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 17 M 2024-04-21T08:25:42.962 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 16 k 2024-04-21T08:25:42.962 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-selinux x86_64 2:19.0.0-3154.ga630765c.el9 ceph 25 k 2024-04-21T08:25:42.962 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-21T08:25:42.962 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-21T08:25:42.962 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-21T08:25:42.962 INFO:teuthology.orchestra.run.smithi086.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-21T08:25:42.962 INFO:teuthology.orchestra.run.smithi086.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-21T08:25:42.962 INFO:teuthology.orchestra.run.smithi086.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-21T08:25:42.963 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-21T08:25:42.963 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-21T08:25:42.963 INFO:teuthology.orchestra.run.smithi086.stdout: libcephsqlite x86_64 2:19.0.0-3154.ga630765c.el9 ceph 163 k 2024-04-21T08:25:42.963 INFO:teuthology.orchestra.run.smithi086.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-21T08:25:42.963 INFO:teuthology.orchestra.run.smithi086.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-21T08:25:42.963 INFO:teuthology.orchestra.run.smithi086.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-21T08:25:42.963 INFO:teuthology.orchestra.run.smithi086.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-21T08:25:42.963 INFO:teuthology.orchestra.run.smithi086.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-21T08:25:42.963 INFO:teuthology.orchestra.run.smithi086.stdout: libradosstriper1 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 501 k 2024-04-21T08:25:42.963 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-21T08:25:42.963 INFO:teuthology.orchestra.run.smithi086.stdout: librgw2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 5.4 M 2024-04-21T08:25:42.963 INFO:teuthology.orchestra.run.smithi086.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-21T08:25:42.963 INFO:teuthology.orchestra.run.smithi086.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-21T08:25:42.963 INFO:teuthology.orchestra.run.smithi086.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-21T08:25:42.964 INFO:teuthology.orchestra.run.smithi086.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-21T08:25:42.964 INFO:teuthology.orchestra.run.smithi086.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-21T08:25:42.964 INFO:teuthology.orchestra.run.smithi086.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-21T08:25:42.964 INFO:teuthology.orchestra.run.smithi086.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-21T08:25:42.964 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-21T08:25:42.964 INFO:teuthology.orchestra.run.smithi086.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-21T08:25:42.964 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-21T08:25:42.964 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-21T08:25:42.964 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-21T08:25:42.964 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-21T08:25:42.964 INFO:teuthology.orchestra.run.smithi086.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-21T08:25:42.964 INFO:teuthology.orchestra.run.smithi086.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-21T08:25:42.964 INFO:teuthology.orchestra.run.smithi086.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-21T08:25:42.964 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-21T08:25:42.965 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 45 k 2024-04-21T08:25:42.965 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-common x86_64 2:19.0.0-3154.ga630765c.el9 ceph 135 k 2024-04-21T08:25:42.965 INFO:teuthology.orchestra.run.smithi086.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-21T08:25:42.965 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-21T08:25:42.965 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-21T08:25:42.965 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-21T08:25:42.965 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-21T08:25:42.965 INFO:teuthology.orchestra.run.smithi086.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-21T08:25:42.965 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-21T08:25:42.965 INFO:teuthology.orchestra.run.smithi086.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-21T08:25:42.965 INFO:teuthology.orchestra.run.smithi086.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-21T08:25:42.965 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-21T08:25:42.965 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-21T08:25:42.965 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-21T08:25:42.966 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-21T08:25:42.966 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-21T08:25:42.966 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-21T08:25:42.966 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-21T08:25:42.966 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-21T08:25:42.966 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-21T08:25:42.966 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-21T08:25:42.966 INFO:teuthology.orchestra.run.smithi086.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-21T08:25:42.966 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-21T08:25:42.966 INFO:teuthology.orchestra.run.smithi086.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-21T08:25:42.966 INFO:teuthology.orchestra.run.smithi086.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-21T08:25:42.966 INFO:teuthology.orchestra.run.smithi086.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-21T08:25:42.966 INFO:teuthology.orchestra.run.smithi086.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-21T08:25:42.967 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-21T08:25:42.967 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-21T08:25:42.967 INFO:teuthology.orchestra.run.smithi086.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-21T08:25:42.967 INFO:teuthology.orchestra.run.smithi086.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-21T08:25:42.967 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-21T08:25:42.967 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-21T08:25:42.967 INFO:teuthology.orchestra.run.smithi086.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-21T08:25:42.967 INFO:teuthology.orchestra.run.smithi086.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-21T08:25:42.967 INFO:teuthology.orchestra.run.smithi086.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-21T08:25:42.967 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-21T08:25:42.967 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-21T08:25:42.967 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-21T08:25:42.967 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-21T08:25:42.967 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-21T08:25:42.967 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-21T08:25:42.968 INFO:teuthology.orchestra.run.smithi086.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-21T08:25:42.968 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-21T08:25:42.968 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-21T08:25:42.968 INFO:teuthology.orchestra.run.smithi086.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-21T08:25:42.968 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-21T08:25:42.968 INFO:teuthology.orchestra.run.smithi086.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-21T08:25:42.968 INFO:teuthology.orchestra.run.smithi086.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-21T08:25:42.968 INFO:teuthology.orchestra.run.smithi086.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-21T08:25:42.968 INFO:teuthology.orchestra.run.smithi086.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-21T08:25:42.968 INFO:teuthology.orchestra.run.smithi086.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-21T08:25:42.968 INFO:teuthology.orchestra.run.smithi086.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-21T08:25:42.968 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-21T08:25:42.968 INFO:teuthology.orchestra.run.smithi086.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-21T08:25:42.968 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-21T08:25:42.969 INFO:teuthology.orchestra.run.smithi086.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-21T08:25:42.969 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-21T08:25:42.969 INFO:teuthology.orchestra.run.smithi086.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-21T08:25:42.969 INFO:teuthology.orchestra.run.smithi086.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-21T08:25:42.969 INFO:teuthology.orchestra.run.smithi086.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-21T08:25:42.969 INFO:teuthology.orchestra.run.smithi086.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-21T08:25:42.969 INFO:teuthology.orchestra.run.smithi086.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-21T08:25:42.969 INFO:teuthology.orchestra.run.smithi086.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-21T08:25:42.969 INFO:teuthology.orchestra.run.smithi086.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-21T08:25:42.969 INFO:teuthology.orchestra.run.smithi086.stdout:Installing weak dependencies: 2024-04-21T08:25:42.969 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-21T08:25:42.969 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:25:42.969 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction Summary 2024-04-21T08:25:42.969 INFO:teuthology.orchestra.run.smithi086.stdout:======================================================================================= 2024-04-21T08:25:42.969 INFO:teuthology.orchestra.run.smithi086.stdout:Install 148 Packages 2024-04-21T08:25:42.970 INFO:teuthology.orchestra.run.smithi086.stdout:Upgrade 2 Packages 2024-04-21T08:25:42.970 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:25:42.970 INFO:teuthology.orchestra.run.smithi086.stdout:Total download size: 218 M 2024-04-21T08:25:42.970 INFO:teuthology.orchestra.run.smithi086.stdout:Downloading Packages: 2024-04-21T08:25:43.806 INFO:teuthology.orchestra.run.smithi086.stdout:(1/150): ceph-19.0.0-3154.ga630765c.el9.x86_64. 29 kB/s | 6.6 kB 00:00 2024-04-21T08:25:43.956 INFO:teuthology.orchestra.run.smithi088.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-21T08:25:43.956 INFO:teuthology.orchestra.run.smithi088.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-21T08:25:43.967 INFO:teuthology.orchestra.run.smithi088.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-21T08:25:43.968 INFO:teuthology.orchestra.run.smithi088.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-21T08:25:43.968 INFO:teuthology.orchestra.run.smithi088.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-21T08:25:43.969 INFO:teuthology.orchestra.run.smithi088.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-21T08:25:43.971 INFO:teuthology.orchestra.run.smithi088.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-21T08:25:43.973 INFO:teuthology.orchestra.run.smithi088.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-21T08:25:43.974 INFO:teuthology.orchestra.run.smithi088.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-21T08:25:44.032 INFO:teuthology.orchestra.run.smithi088.stdout:Dependencies resolved. 2024-04-21T08:25:44.046 INFO:teuthology.orchestra.run.smithi088.stdout:======================================================================================= 2024-04-21T08:25:44.047 INFO:teuthology.orchestra.run.smithi088.stdout: Package Arch Version Repository Size 2024-04-21T08:25:44.047 INFO:teuthology.orchestra.run.smithi088.stdout:======================================================================================= 2024-04-21T08:25:44.047 INFO:teuthology.orchestra.run.smithi088.stdout:Installing: 2024-04-21T08:25:44.047 INFO:teuthology.orchestra.run.smithi088.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-21T08:25:44.047 INFO:teuthology.orchestra.run.smithi088.stdout: ceph x86_64 2:19.0.0-3154.ga630765c.el9 ceph 6.6 k 2024-04-21T08:25:44.047 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-base x86_64 2:19.0.0-3154.ga630765c.el9 ceph 5.4 M 2024-04-21T08:25:44.047 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-fuse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 890 k 2024-04-21T08:25:44.047 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3154.ga630765c.el9 ceph 145 k 2024-04-21T08:25:44.047 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr x86_64 2:19.0.0-3154.ga630765c.el9 ceph 1.5 M 2024-04-21T08:25:44.047 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 142 k 2024-04-21T08:25:44.047 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 3.5 M 2024-04-21T08:25:44.047 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 7.4 M 2024-04-21T08:25:44.047 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-rook noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 49 k 2024-04-21T08:25:44.047 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-radosgw x86_64 2:19.0.0-3154.ga630765c.el9 ceph 11 M 2024-04-21T08:25:44.048 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-test x86_64 2:19.0.0-3154.ga630765c.el9 ceph 49 M 2024-04-21T08:25:44.048 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-volume noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 273 k 2024-04-21T08:25:44.048 INFO:teuthology.orchestra.run.smithi088.stdout: cephadm noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 762 k 2024-04-21T08:25:44.048 INFO:teuthology.orchestra.run.smithi088.stdout: cephfs-mirror x86_64 2:19.0.0-3154.ga630765c.el9 ceph 225 k 2024-04-21T08:25:44.048 INFO:teuthology.orchestra.run.smithi088.stdout: cephfs-shell x86_64 2:19.0.0-3154.ga630765c.el9 ceph 27 k 2024-04-21T08:25:44.048 INFO:teuthology.orchestra.run.smithi088.stdout: cephfs-top noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 18 k 2024-04-21T08:25:44.048 INFO:teuthology.orchestra.run.smithi088.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-21T08:25:44.048 INFO:teuthology.orchestra.run.smithi088.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-21T08:25:44.048 INFO:teuthology.orchestra.run.smithi088.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-21T08:25:44.048 INFO:teuthology.orchestra.run.smithi088.stdout: libcephfs-devel x86_64 2:19.0.0-3154.ga630765c.el9 ceph 32 k 2024-04-21T08:25:44.048 INFO:teuthology.orchestra.run.smithi088.stdout: libcephfs2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 746 k 2024-04-21T08:25:44.048 INFO:teuthology.orchestra.run.smithi088.stdout: librados-devel x86_64 2:19.0.0-3154.ga630765c.el9 ceph 127 k 2024-04-21T08:25:44.048 INFO:teuthology.orchestra.run.smithi088.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-21T08:25:44.049 INFO:teuthology.orchestra.run.smithi088.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-21T08:25:44.049 INFO:teuthology.orchestra.run.smithi088.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-21T08:25:44.049 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-21T08:25:44.049 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-21T08:25:44.049 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-21T08:25:44.049 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cephfs x86_64 2:19.0.0-3154.ga630765c.el9 ceph 162 k 2024-04-21T08:25:44.049 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rados x86_64 2:19.0.0-3154.ga630765c.el9 ceph 321 k 2024-04-21T08:25:44.049 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rbd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 298 k 2024-04-21T08:25:44.049 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rgw x86_64 2:19.0.0-3154.ga630765c.el9 ceph 100 k 2024-04-21T08:25:44.049 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-fuse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 85 k 2024-04-21T08:25:44.049 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-mirror x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.1 M 2024-04-21T08:25:44.049 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-nbd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 171 k 2024-04-21T08:25:44.049 INFO:teuthology.orchestra.run.smithi088.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-21T08:25:44.049 INFO:teuthology.orchestra.run.smithi088.stdout:Upgrading: 2024-04-21T08:25:44.050 INFO:teuthology.orchestra.run.smithi088.stdout: librados2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.4 M 2024-04-21T08:25:44.050 INFO:teuthology.orchestra.run.smithi088.stdout: librbd1 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.1 M 2024-04-21T08:25:44.050 INFO:teuthology.orchestra.run.smithi088.stdout:Installing dependencies: 2024-04-21T08:25:44.050 INFO:teuthology.orchestra.run.smithi088.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-21T08:25:44.050 INFO:teuthology.orchestra.run.smithi088.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-21T08:25:44.050 INFO:teuthology.orchestra.run.smithi088.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-21T08:25:44.050 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-common x86_64 2:19.0.0-3154.ga630765c.el9 ceph 21 M 2024-04-21T08:25:44.050 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 34 k 2024-04-21T08:25:44.050 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mds x86_64 2:19.0.0-3154.ga630765c.el9 ceph 2.4 M 2024-04-21T08:25:44.050 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 249 k 2024-04-21T08:25:44.050 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mon x86_64 2:19.0.0-3154.ga630765c.el9 ceph 4.7 M 2024-04-21T08:25:44.050 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-osd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 17 M 2024-04-21T08:25:44.050 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 16 k 2024-04-21T08:25:44.051 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-selinux x86_64 2:19.0.0-3154.ga630765c.el9 ceph 25 k 2024-04-21T08:25:44.051 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-21T08:25:44.051 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-21T08:25:44.051 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-21T08:25:44.051 INFO:teuthology.orchestra.run.smithi088.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-21T08:25:44.051 INFO:teuthology.orchestra.run.smithi088.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-21T08:25:44.051 INFO:teuthology.orchestra.run.smithi088.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-21T08:25:44.051 INFO:teuthology.orchestra.run.smithi088.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-21T08:25:44.051 INFO:teuthology.orchestra.run.smithi088.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-21T08:25:44.051 INFO:teuthology.orchestra.run.smithi088.stdout: libcephsqlite x86_64 2:19.0.0-3154.ga630765c.el9 ceph 163 k 2024-04-21T08:25:44.051 INFO:teuthology.orchestra.run.smithi088.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-21T08:25:44.051 INFO:teuthology.orchestra.run.smithi088.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-21T08:25:44.051 INFO:teuthology.orchestra.run.smithi088.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-21T08:25:44.051 INFO:teuthology.orchestra.run.smithi088.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-21T08:25:44.052 INFO:teuthology.orchestra.run.smithi088.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-21T08:25:44.052 INFO:teuthology.orchestra.run.smithi088.stdout: libradosstriper1 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 501 k 2024-04-21T08:25:44.052 INFO:teuthology.orchestra.run.smithi088.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-21T08:25:44.052 INFO:teuthology.orchestra.run.smithi088.stdout: librgw2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 5.4 M 2024-04-21T08:25:44.052 INFO:teuthology.orchestra.run.smithi088.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-21T08:25:44.052 INFO:teuthology.orchestra.run.smithi088.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-21T08:25:44.052 INFO:teuthology.orchestra.run.smithi088.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-21T08:25:44.052 INFO:teuthology.orchestra.run.smithi088.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-21T08:25:44.052 INFO:teuthology.orchestra.run.smithi088.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-21T08:25:44.052 INFO:teuthology.orchestra.run.smithi088.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-21T08:25:44.052 INFO:teuthology.orchestra.run.smithi088.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-21T08:25:44.052 INFO:teuthology.orchestra.run.smithi088.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-21T08:25:44.052 INFO:teuthology.orchestra.run.smithi088.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-21T08:25:44.053 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-21T08:25:44.053 INFO:teuthology.orchestra.run.smithi088.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-21T08:25:44.053 INFO:teuthology.orchestra.run.smithi088.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-21T08:25:44.053 INFO:teuthology.orchestra.run.smithi088.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-21T08:25:44.053 INFO:teuthology.orchestra.run.smithi088.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-21T08:25:44.053 INFO:teuthology.orchestra.run.smithi088.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-21T08:25:44.053 INFO:teuthology.orchestra.run.smithi088.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-21T08:25:44.053 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-21T08:25:44.053 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ceph-argparse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 45 k 2024-04-21T08:25:44.053 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ceph-common x86_64 2:19.0.0-3154.ga630765c.el9 ceph 135 k 2024-04-21T08:25:44.053 INFO:teuthology.orchestra.run.smithi088.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-21T08:25:44.053 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-21T08:25:44.053 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-21T08:25:44.053 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-21T08:25:44.053 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-21T08:25:44.054 INFO:teuthology.orchestra.run.smithi088.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-21T08:25:44.054 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-21T08:25:44.054 INFO:teuthology.orchestra.run.smithi088.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-21T08:25:44.054 INFO:teuthology.orchestra.run.smithi088.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-21T08:25:44.054 INFO:teuthology.orchestra.run.smithi088.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-21T08:25:44.054 INFO:teuthology.orchestra.run.smithi088.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-21T08:25:44.054 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-21T08:25:44.054 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-21T08:25:44.054 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-21T08:25:44.054 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-21T08:25:44.054 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-21T08:25:44.054 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-21T08:25:44.054 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-21T08:25:44.055 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-21T08:25:44.055 INFO:teuthology.orchestra.run.smithi088.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-21T08:25:44.055 INFO:teuthology.orchestra.run.smithi088.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-21T08:25:44.055 INFO:teuthology.orchestra.run.smithi088.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-21T08:25:44.055 INFO:teuthology.orchestra.run.smithi088.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-21T08:25:44.055 INFO:teuthology.orchestra.run.smithi088.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-21T08:25:44.055 INFO:teuthology.orchestra.run.smithi088.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-21T08:25:44.055 INFO:teuthology.orchestra.run.smithi088.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-21T08:25:44.055 INFO:teuthology.orchestra.run.smithi088.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-21T08:25:44.055 INFO:teuthology.orchestra.run.smithi088.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-21T08:25:44.055 INFO:teuthology.orchestra.run.smithi088.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-21T08:25:44.055 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-21T08:25:44.055 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-21T08:25:44.055 INFO:teuthology.orchestra.run.smithi088.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-21T08:25:44.055 INFO:teuthology.orchestra.run.smithi088.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-21T08:25:44.056 INFO:teuthology.orchestra.run.smithi088.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-21T08:25:44.056 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-21T08:25:44.056 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-21T08:25:44.056 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-21T08:25:44.056 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-21T08:25:44.056 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-21T08:25:44.056 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-21T08:25:44.056 INFO:teuthology.orchestra.run.smithi088.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-21T08:25:44.056 INFO:teuthology.orchestra.run.smithi088.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-21T08:25:44.056 INFO:teuthology.orchestra.run.smithi088.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-21T08:25:44.056 INFO:teuthology.orchestra.run.smithi088.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-21T08:25:44.056 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-21T08:25:44.056 INFO:teuthology.orchestra.run.smithi088.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-21T08:25:44.056 INFO:teuthology.orchestra.run.smithi088.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-21T08:25:44.057 INFO:teuthology.orchestra.run.smithi088.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-21T08:25:44.057 INFO:teuthology.orchestra.run.smithi088.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-21T08:25:44.057 INFO:teuthology.orchestra.run.smithi088.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-21T08:25:44.057 INFO:teuthology.orchestra.run.smithi088.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-21T08:25:44.057 INFO:teuthology.orchestra.run.smithi088.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-21T08:25:44.057 INFO:teuthology.orchestra.run.smithi088.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-21T08:25:44.057 INFO:teuthology.orchestra.run.smithi088.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-21T08:25:44.057 INFO:teuthology.orchestra.run.smithi088.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-21T08:25:44.057 INFO:teuthology.orchestra.run.smithi088.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-21T08:25:44.057 INFO:teuthology.orchestra.run.smithi088.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-21T08:25:44.057 INFO:teuthology.orchestra.run.smithi088.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-21T08:25:44.057 INFO:teuthology.orchestra.run.smithi088.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-21T08:25:44.057 INFO:teuthology.orchestra.run.smithi088.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-21T08:25:44.058 INFO:teuthology.orchestra.run.smithi088.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-21T08:25:44.058 INFO:teuthology.orchestra.run.smithi088.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-21T08:25:44.058 INFO:teuthology.orchestra.run.smithi088.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-21T08:25:44.058 INFO:teuthology.orchestra.run.smithi088.stdout:Installing weak dependencies: 2024-04-21T08:25:44.058 INFO:teuthology.orchestra.run.smithi088.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-21T08:25:44.058 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-21T08:25:44.058 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction Summary 2024-04-21T08:25:44.058 INFO:teuthology.orchestra.run.smithi088.stdout:======================================================================================= 2024-04-21T08:25:44.058 INFO:teuthology.orchestra.run.smithi088.stdout:Install 148 Packages 2024-04-21T08:25:44.058 INFO:teuthology.orchestra.run.smithi088.stdout:Upgrade 2 Packages 2024-04-21T08:25:44.058 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-21T08:25:44.059 INFO:teuthology.orchestra.run.smithi088.stdout:Total download size: 218 M 2024-04-21T08:25:44.059 INFO:teuthology.orchestra.run.smithi088.stdout:Downloading Packages: 2024-04-21T08:25:44.164 INFO:teuthology.orchestra.run.smithi086.stdout:(2/150): ceph-fuse-19.0.0-3154.ga630765c.el9.x8 2.4 MB/s | 890 kB 00:00 2024-04-21T08:25:44.248 INFO:teuthology.orchestra.run.smithi086.stdout:(3/150): ceph-immutable-object-cache-19.0.0-315 1.7 MB/s | 145 kB 00:00 2024-04-21T08:25:44.798 INFO:teuthology.orchestra.run.smithi086.stdout:(4/150): ceph-mds-19.0.0-3154.ga630765c.el9.x86 4.3 MB/s | 2.4 MB 00:00 2024-04-21T08:25:44.865 INFO:teuthology.orchestra.run.smithi086.stdout:(5/150): ceph-base-19.0.0-3154.ga630765c.el9.x8 4.2 MB/s | 5.4 MB 00:01 2024-04-21T08:25:45.081 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - AppStream 1.2 MB/s | 19 MB 00:16 2024-04-21T08:25:45.265 INFO:teuthology.orchestra.run.smithi086.stdout:(6/150): ceph-common-19.0.0-3154.ga630765c.el9. 13 MB/s | 21 MB 00:01 2024-04-21T08:25:45.292 INFO:teuthology.orchestra.run.smithi088.stdout:(1/150): ceph-19.0.0-3154.ga630765c.el9.x86_64. 25 kB/s | 6.6 kB 00:00 2024-04-21T08:25:45.307 INFO:teuthology.orchestra.run.smithi086.stdout:(7/150): ceph-mgr-19.0.0-3154.ga630765c.el9.x86 2.9 MB/s | 1.5 MB 00:00 2024-04-21T08:25:45.749 INFO:teuthology.orchestra.run.smithi086.stdout:(8/150): ceph-mon-19.0.0-3154.ga630765c.el9.x86 5.3 MB/s | 4.7 MB 00:00 2024-04-21T08:25:45.841 INFO:teuthology.orchestra.run.smithi086.stdout:(9/150): ceph-selinux-19.0.0-3154.ga630765c.el9 275 kB/s | 25 kB 00:00 2024-04-21T08:25:46.684 INFO:teuthology.orchestra.run.smithi088.stdout:(2/150): ceph-fuse-19.0.0-3154.ga630765c.el9.x8 640 kB/s | 890 kB 00:01 2024-04-21T08:25:46.826 INFO:teuthology.orchestra.run.smithi088.stdout:(3/150): ceph-immutable-object-cache-19.0.0-315 1.0 MB/s | 145 kB 00:00 2024-04-21T08:25:46.858 INFO:teuthology.orchestra.run.smithi086.stdout:(10/150): ceph-radosgw-19.0.0-3154.ga630765c.el 6.8 MB/s | 11 MB 00:01 2024-04-21T08:25:47.008 INFO:teuthology.orchestra.run.smithi086.stdout:(11/150): ceph-osd-19.0.0-3154.ga630765c.el9.x8 9.7 MB/s | 17 MB 00:01 2024-04-21T08:25:47.033 INFO:teuthology.orchestra.run.smithi086.stdout:(12/150): cephfs-mirror-19.0.0-3154.ga630765c.e 1.3 MB/s | 225 kB 00:00 2024-04-21T08:25:47.059 INFO:teuthology.orchestra.run.smithi086.stdout:(13/150): cephfs-shell-19.0.0-3154.ga630765c.el 542 kB/s | 27 kB 00:00 2024-04-21T08:25:47.067 INFO:teuthology.orchestra.run.smithi088.stdout:(4/150): ceph-base-19.0.0-3154.ga630765c.el9.x8 2.6 MB/s | 5.4 MB 00:02 2024-04-21T08:25:47.092 INFO:teuthology.orchestra.run.smithi086.stdout:(14/150): libcephfs-devel-19.0.0-3154.ga630765c 542 kB/s | 32 kB 00:00 2024-04-21T08:25:47.159 INFO:teuthology.orchestra.run.smithi086.stdout:(15/150): libcephfs2-19.0.0-3154.ga630765c.el9. 7.3 MB/s | 746 kB 00:00 2024-04-21T08:25:47.185 INFO:teuthology.orchestra.run.smithi086.stdout:(16/150): libcephsqlite-19.0.0-3154.ga630765c.e 1.7 MB/s | 163 kB 00:00 2024-04-21T08:25:47.210 INFO:teuthology.orchestra.run.smithi086.stdout:(17/150): librados-devel-19.0.0-3154.ga630765c. 2.5 MB/s | 127 kB 00:00 2024-04-21T08:25:47.277 INFO:teuthology.orchestra.run.smithi086.stdout:(18/150): libradosstriper1-19.0.0-3154.ga630765 5.3 MB/s | 501 kB 00:00 2024-04-21T08:25:47.360 INFO:teuthology.orchestra.run.smithi086.stdout:(19/150): python3-ceph-argparse-19.0.0-3154.ga6 542 kB/s | 45 kB 00:00 2024-04-21T08:25:47.469 INFO:teuthology.orchestra.run.smithi086.stdout:(20/150): python3-ceph-common-19.0.0-3154.ga630 1.2 MB/s | 135 kB 00:00 2024-04-21T08:25:47.553 INFO:teuthology.orchestra.run.smithi086.stdout:(21/150): python3-cephfs-19.0.0-3154.ga630765c. 1.9 MB/s | 162 kB 00:00 2024-04-21T08:25:47.593 INFO:teuthology.orchestra.run.smithi088.stdout:(5/150): ceph-mgr-19.0.0-3154.ga630765c.el9.x86 2.8 MB/s | 1.5 MB 00:00 2024-04-21T08:25:47.653 INFO:teuthology.orchestra.run.smithi086.stdout:(22/150): python3-rados-19.0.0-3154.ga630765c.e 3.1 MB/s | 321 kB 00:00 2024-04-21T08:25:47.720 INFO:teuthology.orchestra.run.smithi086.stdout:(23/150): librgw2-19.0.0-3154.ga630765c.el9.x86 11 MB/s | 5.4 MB 00:00 2024-04-21T08:25:47.745 INFO:teuthology.orchestra.run.smithi086.stdout:(24/150): python3-rbd-19.0.0-3154.ga630765c.el9 3.2 MB/s | 298 kB 00:00 2024-04-21T08:25:47.771 INFO:teuthology.orchestra.run.smithi086.stdout:(25/150): python3-rgw-19.0.0-3154.ga630765c.el9 1.9 MB/s | 100 kB 00:00 2024-04-21T08:25:47.801 INFO:teuthology.orchestra.run.smithi088.stdout:(6/150): ceph-mds-19.0.0-3154.ga630765c.el9.x86 2.4 MB/s | 2.4 MB 00:00 2024-04-21T08:25:47.813 INFO:teuthology.orchestra.run.smithi086.stdout:(26/150): rbd-fuse-19.0.0-3154.ga630765c.el9.x8 1.2 MB/s | 85 kB 00:00 2024-04-21T08:25:47.913 INFO:teuthology.orchestra.run.smithi086.stdout:(27/150): rbd-nbd-19.0.0-3154.ga630765c.el9.x86 1.7 MB/s | 171 kB 00:00 2024-04-21T08:25:48.005 INFO:teuthology.orchestra.run.smithi086.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3154.g 373 kB/s | 34 kB 00:00 2024-04-21T08:25:48.072 INFO:teuthology.orchestra.run.smithi086.stdout:(29/150): rbd-mirror-19.0.0-3154.ga630765c.el9. 10 MB/s | 3.1 MB 00:00 2024-04-21T08:25:48.097 INFO:teuthology.orchestra.run.smithi086.stdout:(30/150): ceph-mgr-cephadm-19.0.0-3154.ga630765 1.5 MB/s | 142 kB 00:00 2024-04-21T08:25:48.614 INFO:teuthology.orchestra.run.smithi086.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3154.ga6307 6.5 MB/s | 3.5 MB 00:00 2024-04-21T08:25:48.698 INFO:teuthology.orchestra.run.smithi086.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3154.ga6 2.9 MB/s | 249 kB 00:00 2024-04-21T08:25:48.756 INFO:teuthology.orchestra.run.smithi086.stdout:(33/150): ceph-mgr-rook-19.0.0-3154.ga630765c.e 846 kB/s | 49 kB 00:00 2024-04-21T08:25:48.841 INFO:teuthology.orchestra.run.smithi086.stdout:(34/150): ceph-prometheus-alerts-19.0.0-3154.ga 197 kB/s | 16 kB 00:00 2024-04-21T08:25:49.001 INFO:teuthology.orchestra.run.smithi086.stdout:(35/150): ceph-mgr-diskprediction-local-19.0.0- 8.2 MB/s | 7.4 MB 00:00 2024-04-21T08:25:49.024 INFO:teuthology.orchestra.run.smithi086.stdout:(36/150): ceph-volume-19.0.0-3154.ga630765c.el9 1.5 MB/s | 273 kB 00:00 2024-04-21T08:25:49.100 INFO:teuthology.orchestra.run.smithi086.stdout:(37/150): cephfs-top-19.0.0-3154.ga630765c.el9. 243 kB/s | 18 kB 00:00 2024-04-21T08:25:49.141 INFO:teuthology.orchestra.run.smithi086.stdout:(38/150): cephadm-19.0.0-3154.ga630765c.el9.noa 5.3 MB/s | 762 kB 00:00 2024-04-21T08:25:49.233 INFO:teuthology.orchestra.run.smithi086.stdout:(39/150): fuse-2.9.9-15.el9.x86_64.rpm 602 kB/s | 80 kB 00:00 2024-04-21T08:25:49.292 INFO:teuthology.orchestra.run.smithi086.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 3.3 MB/s | 195 kB 00:00 2024-04-21T08:25:49.342 INFO:teuthology.orchestra.run.smithi086.stdout:(41/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.0 MB/s | 253 kB 00:00 2024-04-21T08:25:49.376 INFO:teuthology.orchestra.run.smithi086.stdout:(42/150): libgfortran-11.4.1-3.el9.x86_64.rpm 3.4 MB/s | 803 kB 00:00 2024-04-21T08:25:49.426 INFO:teuthology.orchestra.run.smithi086.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2024-04-21T08:25:49.434 INFO:teuthology.orchestra.run.smithi088.stdout:(7/150): ceph-mon-19.0.0-3154.ga630765c.el9.x86 2.6 MB/s | 4.7 MB 00:01 2024-04-21T08:25:49.460 INFO:teuthology.orchestra.run.smithi086.stdout:(44/150): python3-cryptography-36.0.1-4.el9.x86 10 MB/s | 1.2 MB 00:00 2024-04-21T08:25:49.485 INFO:teuthology.orchestra.run.smithi086.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 2.3 MB/s | 135 kB 00:00 2024-04-21T08:25:49.510 INFO:teuthology.orchestra.run.smithi086.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-21T08:25:49.536 INFO:teuthology.orchestra.run.smithi086.stdout:(47/150): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-21T08:25:49.685 INFO:teuthology.orchestra.run.smithi088.stdout:(8/150): ceph-common-19.0.0-3154.ga630765c.el9. 4.5 MB/s | 21 MB 00:04 2024-04-21T08:25:49.735 INFO:teuthology.orchestra.run.smithi088.stdout:(9/150): ceph-selinux-19.0.0-3154.ga630765c.el9 505 kB/s | 25 kB 00:00 2024-04-21T08:25:49.811 INFO:teuthology.orchestra.run.smithi086.stdout:(48/150): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 689 kB 00:00 2024-04-21T08:25:49.861 INFO:teuthology.orchestra.run.smithi086.stdout:(49/150): bison-3.7.4-5.el9.x86_64.rpm 2.8 MB/s | 943 kB 00:00 2024-04-21T08:25:49.887 INFO:teuthology.orchestra.run.smithi086.stdout:(50/150): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 107 kB 00:00 2024-04-21T08:25:49.920 INFO:teuthology.orchestra.run.smithi086.stdout:(51/150): elfutils-libelf-devel-0.190-2.el9.x86 367 kB/s | 21 kB 00:00 2024-04-21T08:25:49.954 INFO:teuthology.orchestra.run.smithi086.stdout:(52/150): flex-2.6.4-9.el9.x86_64.rpm 4.6 MB/s | 313 kB 00:00 2024-04-21T08:25:49.979 INFO:teuthology.orchestra.run.smithi086.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 560 kB/s | 33 kB 00:00 2024-04-21T08:25:50.038 INFO:teuthology.orchestra.run.smithi086.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 313 kB/s | 18 kB 00:00 2024-04-21T08:25:50.094 INFO:teuthology.orchestra.run.smithi088.stdout:(10/150): ceph-osd-19.0.0-3154.ga630765c.el9.x8 7.3 MB/s | 17 MB 00:02 2024-04-21T08:25:50.104 INFO:teuthology.orchestra.run.smithi086.stdout:(55/150): libaio-devel-0.3.111-13.el9.x86_64.rp 167 kB/s | 11 kB 00:00 2024-04-21T08:25:50.165 INFO:teuthology.orchestra.run.smithi086.stdout:(56/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 14 MB/s | 3.0 MB 00:00 2024-04-21T08:25:50.169 INFO:teuthology.orchestra.run.smithi088.stdout:(11/150): cephfs-mirror-19.0.0-3154.ga630765c.e 2.9 MB/s | 225 kB 00:00 2024-04-21T08:25:50.180 INFO:teuthology.orchestra.run.smithi086.stdout:(57/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.1 MB/s | 160 kB 00:00 2024-04-21T08:25:50.231 INFO:teuthology.orchestra.run.smithi086.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 677 kB/s | 45 kB 00:00 2024-04-21T08:25:50.236 INFO:teuthology.orchestra.run.smithi088.stdout:(12/150): cephfs-shell-19.0.0-3154.ga630765c.el 405 kB/s | 27 kB 00:00 2024-04-21T08:25:50.264 INFO:teuthology.orchestra.run.smithi086.stdout:(59/150): librdkafka-1.6.1-102.el9.x86_64.rpm 7.7 MB/s | 662 kB 00:00 2024-04-21T08:25:50.294 INFO:teuthology.orchestra.run.smithi088.stdout:(13/150): libcephfs-devel-19.0.0-3154.ga630765c 544 kB/s | 32 kB 00:00 2024-04-21T08:25:50.297 INFO:teuthology.orchestra.run.smithi086.stdout:(60/150): libtool-2.4.6-45.el9.x86_64.rpm 8.5 MB/s | 582 kB 00:00 2024-04-21T08:25:50.323 INFO:teuthology.orchestra.run.smithi086.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 435 kB/s | 25 kB 00:00 2024-04-21T08:25:50.356 INFO:teuthology.orchestra.run.smithi086.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 810 kB/s | 47 kB 00:00 2024-04-21T08:25:50.387 INFO:teuthology.orchestra.run.smithi088.stdout:(14/150): libcephfs2-19.0.0-3154.ga630765c.el9. 8.0 MB/s | 746 kB 00:00 2024-04-21T08:25:50.390 INFO:teuthology.orchestra.run.smithi086.stdout:(63/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.3 MB/s | 292 kB 00:00 2024-04-21T08:25:50.424 INFO:teuthology.orchestra.run.smithi086.stdout:(64/150): lua-5.4.4-4.el9.x86_64.rpm 2.7 MB/s | 188 kB 00:00 2024-04-21T08:25:50.449 INFO:teuthology.orchestra.run.smithi086.stdout:(65/150): openblas-0.3.21-2.el9.x86_64.rpm 601 kB/s | 35 kB 00:00 2024-04-21T08:25:50.453 INFO:teuthology.orchestra.run.smithi088.stdout:(15/150): libcephsqlite-19.0.0-3154.ga630765c.e 2.4 MB/s | 163 kB 00:00 2024-04-21T08:25:50.529 INFO:teuthology.orchestra.run.smithi088.stdout:(16/150): librados-devel-19.0.0-3154.ga630765c. 1.7 MB/s | 127 kB 00:00 2024-04-21T08:25:50.591 INFO:teuthology.orchestra.run.smithi086.stdout:(66/150): openblas-openmp-0.3.21-2.el9.x86_64.r 28 MB/s | 4.7 MB 00:00 2024-04-21T08:25:50.612 INFO:teuthology.orchestra.run.smithi088.stdout:(17/150): libradosstriper1-19.0.0-3154.ga630765 5.9 MB/s | 501 kB 00:00 2024-04-21T08:25:50.666 INFO:teuthology.orchestra.run.smithi086.stdout:(67/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 19 MB/s | 4.1 MB 00:00 2024-04-21T08:25:50.708 INFO:teuthology.orchestra.run.smithi086.stdout:(68/150): postgresql-13.14-1.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-04-21T08:25:50.750 INFO:teuthology.orchestra.run.smithi086.stdout:(69/150): postgresql-contrib-13.14-1.el9.x86_64 11 MB/s | 911 kB 00:00 2024-04-21T08:25:50.784 INFO:teuthology.orchestra.run.smithi086.stdout:(70/150): postgresql-private-libs-13.14-1.el9.x 1.7 MB/s | 133 kB 00:00 2024-04-21T08:25:50.884 INFO:teuthology.orchestra.run.smithi086.stdout:(71/150): protobuf-3.14.0-13.el9.x86_64.rpm 10 MB/s | 1.0 MB 00:00 2024-04-21T08:25:50.968 INFO:teuthology.orchestra.run.smithi086.stdout:(72/150): postgresql-server-13.14-1.el9.x86_64. 27 MB/s | 5.8 MB 00:00 2024-04-21T08:25:50.993 INFO:teuthology.orchestra.run.smithi086.stdout:(73/150): python3-attrs-20.3.0-7.el9.noarch.rpm 804 kB/s | 87 kB 00:00 2024-04-21T08:25:51.012 INFO:teuthology.orchestra.run.smithi088.stdout:(18/150): librgw2-19.0.0-3154.ga630765c.el9.x86 13 MB/s | 5.4 MB 00:00 2024-04-21T08:25:51.060 INFO:teuthology.orchestra.run.smithi086.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 3.6 MB/s | 244 kB 00:00 2024-04-21T08:25:51.079 INFO:teuthology.orchestra.run.smithi088.stdout:(19/150): python3-ceph-argparse-19.0.0-3154.ga6 680 kB/s | 45 kB 00:00 2024-04-21T08:25:51.144 INFO:teuthology.orchestra.run.smithi086.stdout:(75/150): python3-babel-2.9.1-2.el9.noarch.rpm 34 MB/s | 6.0 MB 00:00 2024-04-21T08:25:51.146 INFO:teuthology.orchestra.run.smithi088.stdout:(20/150): python3-ceph-common-19.0.0-3154.ga630 2.0 MB/s | 135 kB 00:00 2024-04-21T08:25:51.169 INFO:teuthology.orchestra.run.smithi086.stdout:(76/150): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-04-21T08:25:51.202 INFO:teuthology.orchestra.run.smithi086.stdout:(77/150): python3-jsonpatch-1.21-16.el9.noarch. 437 kB/s | 26 kB 00:00 2024-04-21T08:25:51.213 INFO:teuthology.orchestra.run.smithi088.stdout:(21/150): python3-cephfs-19.0.0-3154.ga630765c. 2.4 MB/s | 162 kB 00:00 2024-04-21T08:25:51.227 INFO:teuthology.orchestra.run.smithi086.stdout:(78/150): python3-jsonpointer-2.0-4.el9.noarch. 326 kB/s | 19 kB 00:00 2024-04-21T08:25:51.269 INFO:teuthology.orchestra.run.smithi086.stdout:(79/150): python3-mako-1.1.4-6.el9.noarch.rpm 2.5 MB/s | 172 kB 00:00 2024-04-21T08:25:51.288 INFO:teuthology.orchestra.run.smithi088.stdout:(22/150): python3-rados-19.0.0-3154.ga630765c.e 4.2 MB/s | 321 kB 00:00 2024-04-21T08:25:51.295 INFO:teuthology.orchestra.run.smithi086.stdout:(80/150): python3-markupsafe-1.1.1-12.el9.x86_6 519 kB/s | 35 kB 00:00 2024-04-21T08:25:51.362 INFO:teuthology.orchestra.run.smithi086.stdout:(81/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 3.7 MB/s | 250 kB 00:00 2024-04-21T08:25:51.422 INFO:teuthology.orchestra.run.smithi088.stdout:(23/150): python3-rbd-19.0.0-3154.ga630765c.el9 2.2 MB/s | 298 kB 00:00 2024-04-21T08:25:51.429 INFO:teuthology.orchestra.run.smithi086.stdout:(82/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 32 MB/s | 5.1 MB 00:00 2024-04-21T08:25:51.454 INFO:teuthology.orchestra.run.smithi086.stdout:(83/150): python3-oauthlib-3.1.1-5.el9.noarch.r 2.4 MB/s | 222 kB 00:00 2024-04-21T08:25:51.488 INFO:teuthology.orchestra.run.smithi086.stdout:(84/150): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-21T08:25:51.489 INFO:teuthology.orchestra.run.smithi088.stdout:(24/150): python3-rgw-19.0.0-3154.ga630765c.el9 1.5 MB/s | 100 kB 00:00 2024-04-21T08:25:51.514 INFO:teuthology.orchestra.run.smithi086.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 722 kB/s | 42 kB 00:00 2024-04-21T08:25:51.555 INFO:teuthology.orchestra.run.smithi086.stdout:(86/150): python3-protobuf-3.14.0-13.el9.noarch 3.9 MB/s | 269 kB 00:00 2024-04-21T08:25:51.556 INFO:teuthology.orchestra.run.smithi088.stdout:(25/150): rbd-fuse-19.0.0-3154.ga630765c.el9.x8 1.3 MB/s | 85 kB 00:00 2024-04-21T08:25:51.580 INFO:teuthology.orchestra.run.smithi086.stdout:(87/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 2.3 MB/s | 159 kB 00:00 2024-04-21T08:25:51.622 INFO:teuthology.orchestra.run.smithi086.stdout:(88/150): python3-pyasn1-modules-0.4.8-6.el9.no 4.1 MB/s | 279 kB 00:00 2024-04-21T08:25:51.647 INFO:teuthology.orchestra.run.smithi086.stdout:(89/150): python3-pytz-2021.1-5.el9.noarch.rpm 760 kB/s | 51 kB 00:00 2024-04-21T08:25:51.681 INFO:teuthology.orchestra.run.smithi086.stdout:(90/150): python3-requests-oauthlib-1.3.0-12.el 913 kB/s | 54 kB 00:00 2024-04-21T08:25:51.748 INFO:teuthology.orchestra.run.smithi086.stdout:(91/150): python3-toml-0.10.2-6.el9.noarch.rpm 627 kB/s | 42 kB 00:00 2024-04-21T08:25:51.806 INFO:teuthology.orchestra.run.smithi086.stdout:(92/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 757 kB/s | 44 kB 00:00 2024-04-21T08:25:51.840 INFO:teuthology.orchestra.run.smithi088.stdout:(26/150): rbd-mirror-19.0.0-3154.ga630765c.el9. 11 MB/s | 3.1 MB 00:00 2024-04-21T08:25:51.873 INFO:teuthology.orchestra.run.smithi086.stdout:(93/150): qatlib-24.02.0-1.el9.x86_64.rpm 3.2 MB/s | 221 kB 00:00 2024-04-21T08:25:51.915 INFO:teuthology.orchestra.run.smithi088.stdout:(27/150): rbd-nbd-19.0.0-3154.ga630765c.el9.x86 2.2 MB/s | 171 kB 00:00 2024-04-21T08:25:51.932 INFO:teuthology.orchestra.run.smithi086.stdout:(94/150): qatlib-service-24.02.0-1.el9.x86_64.r 616 kB/s | 36 kB 00:00 2024-04-21T08:25:52.057 INFO:teuthology.orchestra.run.smithi088.stdout:(28/150): ceph-radosgw-19.0.0-3154.ga630765c.el 4.0 MB/s | 11 MB 00:02 2024-04-21T08:25:52.083 INFO:teuthology.orchestra.run.smithi088.stdout:(29/150): ceph-grafana-dashboards-19.0.0-3154.g 204 kB/s | 34 kB 00:00 2024-04-21T08:25:52.108 INFO:teuthology.orchestra.run.smithi088.stdout:(30/150): ceph-mgr-cephadm-19.0.0-3154.ga630765 2.8 MB/s | 142 kB 00:00 2024-04-21T08:25:52.149 INFO:teuthology.orchestra.run.smithi086.stdout:(95/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 32 MB/s | 16 MB 00:00 2024-04-21T08:25:52.174 INFO:teuthology.orchestra.run.smithi086.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 196 kB/s | 47 kB 00:00 2024-04-21T08:25:52.216 INFO:teuthology.orchestra.run.smithi086.stdout:(97/150): socat-1.7.4.1-5.el9.x86_64.rpm 4.5 MB/s | 305 kB 00:00 2024-04-21T08:25:52.241 INFO:teuthology.orchestra.run.smithi086.stdout:(98/150): uuid-1.6.2-55.el9.x86_64.rpm 863 kB/s | 58 kB 00:00 2024-04-21T08:25:52.275 INFO:teuthology.orchestra.run.smithi086.stdout:(99/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.6 MB/s | 94 kB 00:00 2024-04-21T08:25:52.300 INFO:teuthology.orchestra.run.smithi086.stdout:(100/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.1 MB/s | 64 kB 00:00 2024-04-21T08:25:52.334 INFO:teuthology.orchestra.run.smithi086.stdout:(101/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 765 kB/s | 45 kB 00:00 2024-04-21T08:25:52.341 INFO:teuthology.orchestra.run.smithi088.stdout:(31/150): ceph-mgr-dashboard-19.0.0-3154.ga6307 14 MB/s | 3.5 MB 00:00 2024-04-21T08:25:52.368 INFO:teuthology.orchestra.run.smithi086.stdout:(102/150): lua-devel-5.4.4-4.el9.x86_64.rpm 332 kB/s | 22 kB 00:00 2024-04-21T08:25:52.408 INFO:teuthology.orchestra.run.smithi088.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3154.ga6 3.7 MB/s | 249 kB 00:00 2024-04-21T08:25:52.475 INFO:teuthology.orchestra.run.smithi088.stdout:(33/150): ceph-mgr-rook-19.0.0-3154.ga630765c.e 742 kB/s | 49 kB 00:00 2024-04-21T08:25:52.518 INFO:teuthology.orchestra.run.smithi086.stdout:(103/150): protobuf-compiler-3.14.0-13.el9.x86_ 4.6 MB/s | 863 kB 00:00 2024-04-21T08:25:52.542 INFO:teuthology.orchestra.run.smithi088.stdout:(34/150): ceph-prometheus-alerts-19.0.0-3154.ga 247 kB/s | 16 kB 00:00 2024-04-21T08:25:52.618 INFO:teuthology.orchestra.run.smithi088.stdout:(35/150): ceph-volume-19.0.0-3154.ga630765c.el9 3.6 MB/s | 273 kB 00:00 2024-04-21T08:25:52.709 INFO:teuthology.orchestra.run.smithi088.stdout:(36/150): cephadm-19.0.0-3154.ga630765c.el9.noa 8.1 MB/s | 762 kB 00:00 2024-04-21T08:25:52.776 INFO:teuthology.orchestra.run.smithi088.stdout:(37/150): cephfs-top-19.0.0-3154.ga630765c.el9. 273 kB/s | 18 kB 00:00 2024-04-21T08:25:52.852 INFO:teuthology.orchestra.run.smithi086.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 1.1 MB/s | 548 kB 00:00 2024-04-21T08:25:52.885 INFO:teuthology.orchestra.run.smithi088.stdout:(38/150): fuse-2.9.9-15.el9.x86_64.rpm 743 kB/s | 80 kB 00:00 2024-04-21T08:25:52.944 INFO:teuthology.orchestra.run.smithi086.stdout:(105/150): grpc-data-1.46.7-10.el9.noarch.rpm 212 kB/s | 19 kB 00:00 2024-04-21T08:25:52.969 INFO:teuthology.orchestra.run.smithi086.stdout:(106/150): gperftools-libs-2.9.1-2.el9.x86_64.r 686 kB/s | 309 kB 00:00 2024-04-21T08:25:52.993 INFO:teuthology.orchestra.run.smithi088.stdout:(39/150): libgfortran-11.4.1-3.el9.x86_64.rpm 7.2 MB/s | 803 kB 00:00 2024-04-21T08:25:53.035 INFO:teuthology.orchestra.run.smithi088.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 4.6 MB/s | 195 kB 00:00 2024-04-21T08:25:53.052 INFO:teuthology.orchestra.run.smithi086.stdout:(107/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 312 kB/s | 26 kB 00:00 2024-04-21T08:25:53.077 INFO:teuthology.orchestra.run.smithi088.stdout:(41/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.9 MB/s | 253 kB 00:00 2024-04-21T08:25:53.145 INFO:teuthology.orchestra.run.smithi088.stdout:(42/150): python3-cryptography-36.0.1-4.el9.x86 18 MB/s | 1.2 MB 00:00 2024-04-21T08:25:53.146 INFO:teuthology.orchestra.run.smithi086.stdout:(108/150): liboath-2.6.7-2.el9.x86_64.rpm 530 kB/s | 49 kB 00:00 2024-04-21T08:25:53.203 INFO:teuthology.orchestra.run.smithi088.stdout:(43/150): python3-ply-3.11-14.el9.noarch.rpm 1.8 MB/s | 106 kB 00:00 2024-04-21T08:25:53.236 INFO:teuthology.orchestra.run.smithi086.stdout:(109/150): libunwind-1.6.2-1.el9.x86_64.rpm 737 kB/s | 67 kB 00:00 2024-04-21T08:25:53.245 INFO:teuthology.orchestra.run.smithi088.stdout:(44/150): python3-pycparser-2.20-6.el9.noarch.r 3.2 MB/s | 135 kB 00:00 2024-04-21T08:25:53.287 INFO:teuthology.orchestra.run.smithi088.stdout:(45/150): python3-requests-2.25.1-8.el9.noarch. 2.9 MB/s | 125 kB 00:00 2024-04-21T08:25:53.295 INFO:teuthology.orchestra.run.smithi086.stdout:(110/150): libarrow-9.0.0-10.el9.x86_64.rpm 13 MB/s | 4.4 MB 00:00 2024-04-21T08:25:53.329 INFO:teuthology.orchestra.run.smithi088.stdout:(46/150): python3-urllib3-1.26.5-5.el9.noarch.r 5.0 MB/s | 215 kB 00:00 2024-04-21T08:25:53.336 INFO:teuthology.orchestra.run.smithi086.stdout:(111/150): luarocks-3.9.2-1.el9.noarch.rpm 1.5 MB/s | 151 kB 00:00 2024-04-21T08:25:53.395 INFO:teuthology.orchestra.run.smithi086.stdout:(112/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 8.2 MB/s | 838 kB 00:00 2024-04-21T08:25:53.471 INFO:teuthology.orchestra.run.smithi086.stdout:(113/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 580 kB/s | 43 kB 00:00 2024-04-21T08:25:53.479 INFO:teuthology.orchestra.run.smithi088.stdout:(47/150): automake-1.16.2-8.el9.noarch.rpm 4.5 MB/s | 689 kB 00:00 2024-04-21T08:25:53.504 INFO:teuthology.orchestra.run.smithi086.stdout:(114/150): python3-asyncssh-2.13.2-5.el9.noarch 3.2 MB/s | 548 kB 00:00 2024-04-21T08:25:53.554 INFO:teuthology.orchestra.run.smithi086.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 385 kB/s | 32 kB 00:00 2024-04-21T08:25:53.579 INFO:teuthology.orchestra.run.smithi088.stdout:(48/150): ceph-mgr-diskprediction-local-19.0.0- 5.0 MB/s | 7.4 MB 00:01 2024-04-21T08:25:53.603 INFO:teuthology.orchestra.run.smithi086.stdout:(116/150): python3-certifi-2021.10.8-2.el9.noar 153 kB/s | 15 kB 00:00 2024-04-21T08:25:53.638 INFO:teuthology.orchestra.run.smithi086.stdout:(117/150): python3-cheroot-8.6.0-4.el9.noarch.r 2.0 MB/s | 172 kB 00:00 2024-04-21T08:25:53.655 INFO:teuthology.orchestra.run.smithi088.stdout:(49/150): boost-program-options-1.75.0-8.el9.x8 1.4 MB/s | 107 kB 00:00 2024-04-21T08:25:53.697 INFO:teuthology.orchestra.run.smithi088.stdout:(50/150): elfutils-libelf-devel-0.190-2.el9.x86 518 kB/s | 21 kB 00:00 2024-04-21T08:25:53.705 INFO:teuthology.orchestra.run.smithi086.stdout:(118/150): python3-cherrypy-18.6.1-2.el9.noarch 3.4 MB/s | 358 kB 00:00 2024-04-21T08:25:53.730 INFO:teuthology.orchestra.run.smithi086.stdout:(119/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.2 MB/s | 303 kB 00:00 2024-04-21T08:25:53.781 INFO:teuthology.orchestra.run.smithi086.stdout:(120/150): python3-colorama-0.4.4-12.el9.noarch 471 kB/s | 35 kB 00:00 2024-04-21T08:25:53.797 INFO:teuthology.orchestra.run.smithi088.stdout:(51/150): bison-3.7.4-5.el9.x86_64.rpm 2.9 MB/s | 943 kB 00:00 2024-04-21T08:25:53.814 INFO:teuthology.orchestra.run.smithi086.stdout:(121/150): python3-google-auth-2.28.2-1.el9.noa 2.4 MB/s | 203 kB 00:00 2024-04-21T08:25:53.838 INFO:teuthology.orchestra.run.smithi088.stdout:(52/150): flex-2.6.4-9.el9.x86_64.rpm 2.2 MB/s | 313 kB 00:00 2024-04-21T08:25:53.855 INFO:teuthology.orchestra.run.smithi088.stdout:(53/150): flexiblas-3.0.4-8.el9.x86_64.rpm 559 kB/s | 33 kB 00:00 2024-04-21T08:25:53.898 INFO:teuthology.orchestra.run.smithi086.stdout:(122/150): python3-grpcio-tools-1.46.7-10.el9.x 1.7 MB/s | 144 kB 00:00 2024-04-21T08:25:53.906 INFO:teuthology.orchestra.run.smithi088.stdout:(54/150): flexiblas-openblas-openmp-3.0.4-8.el9 366 kB/s | 18 kB 00:00 2024-04-21T08:25:53.990 INFO:teuthology.orchestra.run.smithi086.stdout:(123/150): python3-grpcio-1.46.7-10.el9.x86_64. 9.8 MB/s | 2.0 MB 00:00 2024-04-21T08:25:54.015 INFO:teuthology.orchestra.run.smithi086.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 91 kB/s | 11 kB 00:00 2024-04-21T08:25:54.031 INFO:teuthology.orchestra.run.smithi088.stdout:(55/150): libaio-devel-0.3.111-13.el9.x86_64.rp 89 kB/s | 11 kB 00:00 2024-04-21T08:25:54.074 INFO:teuthology.orchestra.run.smithi086.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 212 kB/s | 18 kB 00:00 2024-04-21T08:25:54.099 INFO:teuthology.orchestra.run.smithi086.stdout:(126/150): python3-jaraco-collections-3.0.0-8.e 277 kB/s | 23 kB 00:00 2024-04-21T08:25:54.115 INFO:teuthology.orchestra.run.smithi088.stdout:(56/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.9 MB/s | 160 kB 00:00 2024-04-21T08:25:54.157 INFO:teuthology.orchestra.run.smithi086.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 233 kB/s | 19 kB 00:00 2024-04-21T08:25:54.174 INFO:teuthology.orchestra.run.smithi088.stdout:(57/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 780 kB/s | 45 kB 00:00 2024-04-21T08:25:54.183 INFO:teuthology.orchestra.run.smithi086.stdout:(128/150): python3-jaraco-text-3.2.0-6.el9.noar 236 kB/s | 20 kB 00:00 2024-04-21T08:25:54.241 INFO:teuthology.orchestra.run.smithi088.stdout:(58/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 7.4 MB/s | 3.0 MB 00:00 2024-04-21T08:25:54.258 INFO:teuthology.orchestra.run.smithi086.stdout:(129/150): python3-kubernetes-26.1.0-2.el9.noar 10 MB/s | 1.0 MB 00:00 2024-04-21T08:25:54.283 INFO:teuthology.orchestra.run.smithi086.stdout:(130/150): python3-logutils-0.3.5-21.el9.noarch 462 kB/s | 46 kB 00:00 2024-04-21T08:25:54.342 INFO:teuthology.orchestra.run.smithi086.stdout:(131/150): python3-more-itertools-8.12.0-2.el9. 944 kB/s | 79 kB 00:00 2024-04-21T08:25:54.349 INFO:teuthology.orchestra.run.smithi088.stdout:(59/150): libtool-2.4.6-45.el9.x86_64.rpm 5.3 MB/s | 582 kB 00:00 2024-04-21T08:25:54.367 INFO:teuthology.orchestra.run.smithi086.stdout:(132/150): python3-natsort-7.1.1-5.el9.noarch.r 691 kB/s | 58 kB 00:00 2024-04-21T08:25:54.374 INFO:teuthology.orchestra.run.smithi088.stdout:(60/150): librdkafka-1.6.1-102.el9.x86_64.rpm 3.2 MB/s | 662 kB 00:00 2024-04-21T08:25:54.433 INFO:teuthology.orchestra.run.smithi088.stdout:(61/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 812 kB/s | 47 kB 00:00 2024-04-21T08:25:54.458 INFO:teuthology.orchestra.run.smithi088.stdout:(62/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 234 kB/s | 25 kB 00:00 2024-04-21T08:25:54.567 INFO:teuthology.orchestra.run.smithi088.stdout:(63/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.1 MB/s | 292 kB 00:00 2024-04-21T08:25:54.592 INFO:teuthology.orchestra.run.smithi088.stdout:(64/150): lua-5.4.4-4.el9.x86_64.rpm 1.4 MB/s | 188 kB 00:00 2024-04-21T08:25:54.643 INFO:teuthology.orchestra.run.smithi088.stdout:(65/150): openblas-0.3.21-2.el9.x86_64.rpm 469 kB/s | 35 kB 00:00 2024-04-21T08:25:54.784 INFO:teuthology.orchestra.run.smithi086.stdout:(133/150): ceph-test-19.0.0-3154.ga630765c.el9. 5.5 MB/s | 49 MB 00:08 2024-04-21T08:25:54.812 INFO:teuthology.orchestra.run.smithi086.stdout:(134/150): python3-portend-3.1.0-2.el9.noarch.r 37 kB/s | 16 kB 00:00 2024-04-21T08:25:54.836 INFO:teuthology.orchestra.run.smithi086.stdout:(135/150): python3-pecan-1.4.2-3.el9.noarch.rpm 551 kB/s | 272 kB 00:00 2024-04-21T08:25:54.895 INFO:teuthology.orchestra.run.smithi086.stdout:(136/150): python3-pyperclip-1.8.2-2.el9.noarch 338 kB/s | 28 kB 00:00 2024-04-21T08:25:54.921 INFO:teuthology.orchestra.run.smithi086.stdout:(137/150): python3-repoze-lru-0.7-16.el9.noarch 367 kB/s | 31 kB 00:00 2024-04-21T08:25:54.979 INFO:teuthology.orchestra.run.smithi086.stdout:(138/150): python3-routes-2.5.1-5.el9.noarch.rp 2.2 MB/s | 188 kB 00:00 2024-04-21T08:25:55.005 INFO:teuthology.orchestra.run.smithi086.stdout:(139/150): python3-rsa-4.9-2.el9.noarch.rpm 709 kB/s | 59 kB 00:00 2024-04-21T08:25:55.063 INFO:teuthology.orchestra.run.smithi086.stdout:(140/150): python3-tempora-5.0.0-2.el9.noarch.r 428 kB/s | 36 kB 00:00 2024-04-21T08:25:55.088 INFO:teuthology.orchestra.run.smithi086.stdout:(141/150): python3-typing-extensions-4.4.0-2.el 610 kB/s | 51 kB 00:00 2024-04-21T08:25:55.114 INFO:teuthology.orchestra.run.smithi086.stdout:(142/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 273 kB/s | 90 kB 00:00 2024-04-21T08:25:55.147 INFO:teuthology.orchestra.run.smithi086.stdout:(143/150): python3-webob-1.8.7-6.el9.noarch.rpm 2.7 MB/s | 230 kB 00:00 2024-04-21T08:25:55.168 INFO:teuthology.orchestra.run.smithi088.stdout:(66/150): openblas-openmp-0.3.21-2.el9.x86_64.r 8.2 MB/s | 4.7 MB 00:00 2024-04-21T08:25:55.172 INFO:teuthology.orchestra.run.smithi086.stdout:(144/150): python3-websocket-client-1.2.3-2.el9 1.0 MB/s | 90 kB 00:00 2024-04-21T08:25:55.231 INFO:teuthology.orchestra.run.smithi086.stdout:(145/150): python3-zc-lockfile-2.0-10.el9.noarc 239 kB/s | 20 kB 00:00 2024-04-21T08:25:55.256 INFO:teuthology.orchestra.run.smithi086.stdout:(146/150): re2-20211101-3.el9.x86_64.rpm 2.2 MB/s | 192 kB 00:00 2024-04-21T08:25:55.326 INFO:teuthology.orchestra.run.smithi088.stdout:(67/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 6.0 MB/s | 4.1 MB 00:00 2024-04-21T08:25:55.348 INFO:teuthology.orchestra.run.smithi086.stdout:(147/150): thrift-0.15.0-2.el9.x86_64.rpm 14 MB/s | 1.6 MB 00:00 2024-04-21T08:25:55.382 INFO:teuthology.orchestra.run.smithi086.stdout:(148/150): python3-werkzeug-2.0.3-3.el9.1.noarc 1.6 MB/s | 427 kB 00:00 2024-04-21T08:25:55.690 INFO:teuthology.orchestra.run.smithi086.stdout:(149/150): librados2-19.0.0-3154.ga630765c.el9. 7.8 MB/s | 3.4 MB 00:00 2024-04-21T08:25:55.768 INFO:teuthology.orchestra.run.smithi088.stdout:(68/150): ceph-test-19.0.0-3154.ga630765c.el9.x 8.1 MB/s | 49 MB 00:06 2024-04-21T08:25:55.852 INFO:teuthology.orchestra.run.smithi088.stdout:(69/150): postgresql-13.14-1.el9.x86_64.rpm 2.3 MB/s | 1.6 MB 00:00 2024-04-21T08:25:55.910 INFO:teuthology.orchestra.run.smithi088.stdout:(70/150): postgresql-private-libs-13.14-1.el9.x 943 kB/s | 133 kB 00:00 2024-04-21T08:25:55.915 INFO:teuthology.orchestra.run.smithi086.stdout:(150/150): librbd1-19.0.0-3154.ga630765c.el9.x8 5.5 MB/s | 3.1 MB 00:00 2024-04-21T08:25:55.918 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-04-21T08:25:55.918 INFO:teuthology.orchestra.run.smithi086.stdout:Total 17 MB/s | 218 MB 00:12 2024-04-21T08:25:56.027 INFO:teuthology.orchestra.run.smithi088.stdout:(71/150): postgresql-contrib-13.14-1.el9.x86_64 1.3 MB/s | 911 kB 00:00 2024-04-21T08:25:56.136 INFO:teuthology.orchestra.run.smithi088.stdout:(72/150): python3-attrs-20.3.0-7.el9.noarch.rpm 808 kB/s | 87 kB 00:00 2024-04-21T08:25:56.169 INFO:teuthology.orchestra.run.smithi088.stdout:(73/150): protobuf-3.14.0-13.el9.x86_64.rpm 3.9 MB/s | 1.0 MB 00:00 2024-04-21T08:25:56.303 INFO:teuthology.orchestra.run.smithi088.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 1.8 MB/s | 244 kB 00:00 2024-04-21T08:25:56.420 INFO:teuthology.orchestra.run.smithi088.stdout:(75/150): python3-jinja2-2.11.3-5.el9.noarch.rp 2.1 MB/s | 248 kB 00:00 2024-04-21T08:25:56.562 INFO:teuthology.orchestra.run.smithi088.stdout:(76/150): postgresql-server-13.14-1.el9.x86_64. 8.1 MB/s | 5.8 MB 00:00 2024-04-21T08:25:56.587 INFO:teuthology.orchestra.run.smithi088.stdout:(77/150): python3-jsonpatch-1.21-16.el9.noarch. 154 kB/s | 26 kB 00:00 2024-04-21T08:25:56.654 INFO:teuthology.orchestra.run.smithi088.stdout:(78/150): python3-jsonpointer-2.0-4.el9.noarch. 209 kB/s | 19 kB 00:00 2024-04-21T08:25:56.704 INFO:teuthology.orchestra.run.smithi088.stdout:(79/150): python3-mako-1.1.4-6.el9.noarch.rpm 1.4 MB/s | 172 kB 00:00 2024-04-21T08:25:56.729 INFO:teuthology.orchestra.run.smithi088.stdout:(80/150): python3-markupsafe-1.1.1-12.el9.x86_6 461 kB/s | 35 kB 00:00 2024-04-21T08:25:56.748 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - CRB 786 kB/s | 6.2 MB 00:08 2024-04-21T08:25:56.831 INFO:teuthology.orchestra.run.smithi088.stdout:(81/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-04-21T08:25:56.922 INFO:teuthology.orchestra.run.smithi088.stdout:(82/150): python3-babel-2.9.1-2.el9.noarch.rpm 7.6 MB/s | 6.0 MB 00:00 2024-04-21T08:25:56.971 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction check 2024-04-21T08:25:56.988 INFO:teuthology.orchestra.run.smithi088.stdout:(83/150): python3-oauthlib-3.1.1-5.el9.noarch.r 1.4 MB/s | 222 kB 00:00 2024-04-21T08:25:57.014 INFO:teuthology.orchestra.run.smithi088.stdout:(84/150): python3-packaging-20.9-5.el9.noarch.r 840 kB/s | 77 kB 00:00 2024-04-21T08:25:57.056 INFO:teuthology.orchestra.run.smithi088.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 632 kB/s | 42 kB 00:00 2024-04-21T08:25:57.096 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction check succeeded. 2024-04-21T08:25:57.097 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction test 2024-04-21T08:25:57.114 INFO:teuthology.orchestra.run.smithi088.stdout:(86/150): python3-protobuf-3.14.0-13.el9.noarch 2.6 MB/s | 269 kB 00:00 2024-04-21T08:25:57.156 INFO:teuthology.orchestra.run.smithi088.stdout:(87/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.6 MB/s | 159 kB 00:00 2024-04-21T08:25:57.265 INFO:teuthology.orchestra.run.smithi088.stdout:(88/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 9.1 MB/s | 5.1 MB 00:00 2024-04-21T08:25:57.291 INFO:teuthology.orchestra.run.smithi088.stdout:(89/150): python3-pytz-2021.1-5.el9.noarch.rpm 381 kB/s | 51 kB 00:00 2024-04-21T08:25:57.315 INFO:teuthology.orchestra.run.smithi088.stdout:(90/150): python3-pyasn1-modules-0.4.8-6.el9.no 1.4 MB/s | 279 kB 00:00 2024-04-21T08:25:57.349 INFO:teuthology.orchestra.run.smithi088.stdout:(91/150): python3-requests-oauthlib-1.3.0-12.el 642 kB/s | 54 kB 00:00 2024-04-21T08:25:57.408 INFO:teuthology.orchestra.run.smithi088.stdout:(92/150): python3-toml-0.10.2-6.el9.noarch.rpm 453 kB/s | 42 kB 00:00 2024-04-21T08:25:57.433 INFO:teuthology.orchestra.run.smithi088.stdout:(93/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 528 kB/s | 44 kB 00:00 2024-04-21T08:25:57.500 INFO:teuthology.orchestra.run.smithi088.stdout:(94/150): qatlib-service-24.02.0-1.el9.x86_64.r 541 kB/s | 36 kB 00:00 2024-04-21T08:25:57.542 INFO:teuthology.orchestra.run.smithi088.stdout:(95/150): qatlib-24.02.0-1.el9.x86_64.rpm 1.6 MB/s | 221 kB 00:00 2024-04-21T08:25:57.617 INFO:teuthology.orchestra.run.smithi088.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 406 kB/s | 47 kB 00:00 2024-04-21T08:25:57.684 INFO:teuthology.orchestra.run.smithi088.stdout:(97/150): socat-1.7.4.1-5.el9.x86_64.rpm 2.1 MB/s | 305 kB 00:00 2024-04-21T08:25:57.709 INFO:teuthology.orchestra.run.smithi088.stdout:(98/150): uuid-1.6.2-55.el9.x86_64.rpm 628 kB/s | 58 kB 00:00 2024-04-21T08:25:57.785 INFO:teuthology.orchestra.run.smithi088.stdout:(99/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 851 kB/s | 64 kB 00:00 2024-04-21T08:25:57.810 INFO:teuthology.orchestra.run.smithi088.stdout:(100/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rp 747 kB/s | 94 kB 00:00 2024-04-21T08:25:57.868 INFO:teuthology.orchestra.run.smithi088.stdout:(101/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 537 kB/s | 45 kB 00:00 2024-04-21T08:25:57.894 INFO:teuthology.orchestra.run.smithi088.stdout:(102/150): lua-devel-5.4.4-4.el9.x86_64.rpm 267 kB/s | 22 kB 00:00 2024-04-21T08:25:58.198 INFO:teuthology.orchestra.run.smithi179.stdout:CentOS Stream 9 - Extras packages 53 kB/s | 16 kB 00:00 2024-04-21T08:25:58.202 INFO:teuthology.orchestra.run.smithi088.stdout:(103/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 18 MB/s | 16 MB 00:00 2024-04-21T08:25:58.269 INFO:teuthology.orchestra.run.smithi088.stdout:(104/150): protobuf-compiler-3.14.0-13.el9.x86_ 2.1 MB/s | 863 kB 00:00 2024-04-21T08:25:58.494 INFO:teuthology.orchestra.run.smithi088.stdout:(105/150): abseil-cpp-20211102.0-3.el9.x86_64.r 913 kB/s | 548 kB 00:00 2024-04-21T08:25:58.520 INFO:teuthology.orchestra.run.smithi088.stdout:(106/150): grpc-data-1.46.7-10.el9.noarch.rpm 78 kB/s | 19 kB 00:00 2024-04-21T08:25:58.620 INFO:teuthology.orchestra.run.smithi088.stdout:(107/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 260 kB/s | 26 kB 00:00 2024-04-21T08:25:58.662 INFO:teuthology.orchestra.run.smithi088.stdout:(108/150): gperftools-libs-2.9.1-2.el9.x86_64.r 673 kB/s | 309 kB 00:00 2024-04-21T08:25:58.712 INFO:teuthology.orchestra.run.smithi088.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 528 kB/s | 49 kB 00:00 2024-04-21T08:25:58.762 INFO:teuthology.orchestra.run.smithi088.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 671 kB/s | 67 kB 00:00 2024-04-21T08:25:58.797 INFO:teuthology.orchestra.run.smithi086.stdout:Transaction test succeeded. 2024-04-21T08:25:58.797 INFO:teuthology.orchestra.run.smithi086.stdout:Running transaction 2024-04-21T08:25:58.888 INFO:teuthology.orchestra.run.smithi088.stdout:(111/150): luarocks-3.9.2-1.el9.noarch.rpm 862 kB/s | 151 kB 00:00 2024-04-21T08:25:58.946 INFO:teuthology.orchestra.run.smithi088.stdout:(112/150): libarrow-9.0.0-10.el9.x86_64.rpm 9.8 MB/s | 4.4 MB 00:00 2024-04-21T08:25:58.971 INFO:teuthology.orchestra.run.smithi088.stdout:(113/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 3.9 MB/s | 838 kB 00:00 2024-04-21T08:25:58.997 INFO:teuthology.orchestra.run.smithi088.stdout:(114/150): python3-asyncssh-2.13.2-5.el9.noarch 4.9 MB/s | 548 kB 00:00 2024-04-21T08:25:59.039 INFO:teuthology.orchestra.run.smithi088.stdout:(115/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 469 kB/s | 43 kB 00:00 2024-04-21T08:25:59.064 INFO:teuthology.orchestra.run.smithi088.stdout:(116/150): python3-cachetools-4.2.4-1.el9.noarc 349 kB/s | 32 kB 00:00 2024-04-21T08:25:59.089 INFO:teuthology.orchestra.run.smithi088.stdout:(117/150): python3-certifi-2021.10.8-2.el9.noar 163 kB/s | 15 kB 00:00 2024-04-21T08:25:59.140 INFO:teuthology.orchestra.run.smithi088.stdout:(118/150): python3-cheroot-8.6.0-4.el9.noarch.r 1.7 MB/s | 172 kB 00:00 2024-04-21T08:25:59.173 INFO:teuthology.orchestra.run.smithi088.stdout:(119/150): python3-cherrypy-18.6.1-2.el9.noarch 3.2 MB/s | 358 kB 00:00 2024-04-21T08:25:59.199 INFO:teuthology.orchestra.run.smithi088.stdout:(120/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 2.7 MB/s | 303 kB 00:00 2024-04-21T08:25:59.224 INFO:teuthology.orchestra.run.smithi088.stdout:(121/150): python3-colorama-0.4.4-12.el9.noarch 420 kB/s | 35 kB 00:00 2024-04-21T08:25:59.274 INFO:teuthology.orchestra.run.smithi088.stdout:(122/150): python3-google-auth-2.28.2-1.el9.noa 2.0 MB/s | 203 kB 00:00 2024-04-21T08:25:59.333 INFO:teuthology.orchestra.run.smithi088.stdout:(123/150): python3-grpcio-1.46.7-10.el9.x86_64. 15 MB/s | 2.0 MB 00:00 2024-04-21T08:25:59.358 INFO:teuthology.orchestra.run.smithi088.stdout:(124/150): python3-grpcio-tools-1.46.7-10.el9.x 1.1 MB/s | 144 kB 00:00 2024-04-21T08:25:59.383 INFO:teuthology.orchestra.run.smithi088.stdout:(125/150): python3-jaraco-8.2.1-3.el9.noarch.rp 98 kB/s | 11 kB 00:00 2024-04-21T08:25:59.425 INFO:teuthology.orchestra.run.smithi088.stdout:(126/150): python3-jaraco-classes-3.2.1-5.el9.n 192 kB/s | 18 kB 00:00 2024-04-21T08:25:59.450 INFO:teuthology.orchestra.run.smithi088.stdout:(127/150): python3-jaraco-collections-3.0.0-8.e 251 kB/s | 23 kB 00:00 2024-04-21T08:25:59.476 INFO:teuthology.orchestra.run.smithi088.stdout:(128/150): python3-jaraco-functools-3.5.0-2.el9 210 kB/s | 19 kB 00:00 2024-04-21T08:25:59.518 INFO:teuthology.orchestra.run.smithi088.stdout:(129/150): python3-jaraco-text-3.2.0-6.el9.noar 214 kB/s | 20 kB 00:00 2024-04-21T08:25:59.618 INFO:teuthology.orchestra.run.smithi088.stdout:(130/150): python3-kubernetes-26.1.0-2.el9.noar 6.1 MB/s | 1.0 MB 00:00 2024-04-21T08:25:59.643 INFO:teuthology.orchestra.run.smithi088.stdout:(131/150): python3-logutils-0.3.5-21.el9.noarch 277 kB/s | 46 kB 00:00 2024-04-21T08:25:59.668 INFO:teuthology.orchestra.run.smithi088.stdout:(132/150): python3-more-itertools-8.12.0-2.el9. 524 kB/s | 79 kB 00:00 2024-04-21T08:25:59.710 INFO:teuthology.orchestra.run.smithi088.stdout:(133/150): python3-natsort-7.1.1-5.el9.noarch.r 626 kB/s | 58 kB 00:00 2024-04-21T08:25:59.744 INFO:teuthology.orchestra.run.smithi088.stdout:(134/150): python3-pecan-1.4.2-3.el9.noarch.rpm 2.6 MB/s | 272 kB 00:00 2024-04-21T08:25:59.769 INFO:teuthology.orchestra.run.smithi088.stdout:(135/150): python3-portend-3.1.0-2.el9.noarch.r 163 kB/s | 16 kB 00:00 2024-04-21T08:25:59.811 INFO:teuthology.orchestra.run.smithi088.stdout:(136/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 896 kB/s | 90 kB 00:00 2024-04-21T08:25:59.837 INFO:teuthology.orchestra.run.smithi088.stdout:(137/150): python3-pyperclip-1.8.2-2.el9.noarch 306 kB/s | 28 kB 00:00 2024-04-21T08:25:59.862 INFO:teuthology.orchestra.run.smithi088.stdout:(138/150): python3-repoze-lru-0.7-16.el9.noarch 334 kB/s | 31 kB 00:00 2024-04-21T08:25:59.912 INFO:teuthology.orchestra.run.smithi088.stdout:(139/150): python3-routes-2.5.1-5.el9.noarch.rp 1.8 MB/s | 188 kB 00:00 2024-04-21T08:25:59.937 INFO:teuthology.orchestra.run.smithi088.stdout:(140/150): python3-rsa-4.9-2.el9.noarch.rpm 588 kB/s | 59 kB 00:00 2024-04-21T08:25:59.963 INFO:teuthology.orchestra.run.smithi088.stdout:(141/150): python3-tempora-5.0.0-2.el9.noarch.r 356 kB/s | 36 kB 00:00 2024-04-21T08:26:00.004 INFO:teuthology.orchestra.run.smithi088.stdout:(142/150): python3-typing-extensions-4.4.0-2.el 554 kB/s | 51 kB 00:00 2024-04-21T08:26:00.066 INFO:teuthology.orchestra.run.smithi088.stdout:(143/150): python3-websocket-client-1.2.3-2.el9 896 kB/s | 90 kB 00:00 2024-04-21T08:26:00.105 INFO:teuthology.orchestra.run.smithi088.stdout:(144/150): python3-webob-1.8.7-6.el9.noarch.rpm 1.3 MB/s | 230 kB 00:00 2024-04-21T08:26:00.130 INFO:teuthology.orchestra.run.smithi088.stdout:(145/150): python3-werkzeug-2.0.3-3.el9.1.noarc 3.3 MB/s | 427 kB 00:00 2024-04-21T08:26:00.180 INFO:teuthology.orchestra.run.smithi088.stdout:(146/150): python3-zc-lockfile-2.0-10.el9.noarc 170 kB/s | 20 kB 00:00 2024-04-21T08:26:00.222 INFO:teuthology.orchestra.run.smithi088.stdout:(147/150): re2-20211101-3.el9.x86_64.rpm 2.0 MB/s | 192 kB 00:00 2024-04-21T08:26:00.339 INFO:teuthology.orchestra.run.smithi088.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 7.6 MB/s | 1.6 MB 00:00 2024-04-21T08:26:00.598 INFO:teuthology.orchestra.run.smithi088.stdout:(149/150): librados2-19.0.0-3154.ga630765c.el9. 8.1 MB/s | 3.4 MB 00:00 2024-04-21T08:26:00.740 INFO:teuthology.orchestra.run.smithi088.stdout:(150/150): librbd1-19.0.0-3154.ga630765c.el9.x8 6.1 MB/s | 3.1 MB 00:00 2024-04-21T08:26:00.742 INFO:teuthology.orchestra.run.smithi088.stdout:-------------------------------------------------------------------------------- 2024-04-21T08:26:00.743 INFO:teuthology.orchestra.run.smithi088.stdout:Total 13 MB/s | 218 MB 00:16 2024-04-21T08:26:01.794 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction check 2024-04-21T08:26:01.916 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction check succeeded. 2024-04-21T08:26:01.917 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction test 2024-04-21T08:26:03.261 INFO:teuthology.orchestra.run.smithi179.stdout:Extra Packages for Enterprise Linux 4.2 MB/s | 21 MB 00:04 2024-04-21T08:26:03.619 INFO:teuthology.orchestra.run.smithi088.stdout:Transaction test succeeded. 2024-04-21T08:26:03.619 INFO:teuthology.orchestra.run.smithi088.stdout:Running transaction 2024-04-21T08:26:04.636 INFO:teuthology.orchestra.run.smithi086.stdout: Preparing : 1/1 2024-04-21T08:26:04.718 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-21T08:26:04.743 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-21T08:26:04.785 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-21T08:26:05.122 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-21T08:26:05.236 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : librados2-2:19.0.0-3154.ga630765c.el9.x86_64 5/152 2024-04-21T08:26:05.358 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librados2-2:19.0.0-3154.ga630765c.el9.x86_64 5/152 2024-04-21T08:26:05.387 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T08:26:05.478 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T08:26:05.520 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 7/152 2024-04-21T08:26:05.549 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-21T08:26:05.575 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-21T08:26:05.725 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-21T08:26:05.780 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-21T08:26:05.808 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T08:26:05.912 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T08:26:05.934 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 13/152 2024-04-21T08:26:05.997 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 13/152 2024-04-21T08:26:06.019 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-21T08:26:06.042 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-21T08:26:06.244 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-21T08:26:06.277 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-21T08:26:06.383 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-21T08:26:06.473 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-21T08:26:06.500 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-21T08:26:06.536 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-21T08:26:06.609 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-21T08:26:06.643 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-21T08:26:06.687 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-21T08:26:06.719 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-21T08:26:06.747 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-21T08:26:06.818 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-21T08:26:06.872 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-21T08:26:06.909 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-21T08:26:06.953 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-21T08:26:07.033 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-21T08:26:07.064 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ceph-common-2:19.0.0-3154.ga630765c.el9. 30/152 2024-04-21T08:26:07.107 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ceph-argparse-2:19.0.0-3154.ga630765c.el 31/152 2024-04-21T08:26:07.136 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_6 32/152 2024-04-21T08:26:07.207 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-21T08:26:07.251 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-21T08:26:07.326 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-21T08:26:07.462 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-21T08:26:07.512 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-21T08:26:08.052 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-21T08:26:08.142 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-21T08:26:08.168 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-21T08:26:08.203 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-21T08:26:08.241 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librados-devel-2:19.0.0-3154.ga630765c.el9.x86_6 42/152 2024-04-21T08:26:08.274 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-21T08:26:08.285 INFO:teuthology.orchestra.run.smithi179.stdout:lab-extras 28 kB/s | 1.7 kB 00:00 2024-04-21T08:26:08.330 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-21T08:26:08.404 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-21T08:26:08.442 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-21T08:26:08.478 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-21T08:26:08.525 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-21T08:26:08.561 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-21T08:26:08.604 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-21T08:26:08.646 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-21T08:26:08.783 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-21T08:26:08.827 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-21T08:26:08.857 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-21T08:26:08.898 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-21T08:26:08.993 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-21T08:26:09.031 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-21T08:26:09.071 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-21T08:26:09.097 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-21T08:26:09.122 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-21T08:26:09.196 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-21T08:26:09.325 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-21T08:26:09.362 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-21T08:26:09.390 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-21T08:26:09.424 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-21T08:26:09.468 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-21T08:26:09.507 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-21T08:26:09.546 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-21T08:26:09.576 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-21T08:26:09.610 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-21T08:26:09.644 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-21T08:26:09.686 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-21T08:26:09.695 INFO:teuthology.orchestra.run.smithi088.stdout: Preparing : 1/1 2024-04-21T08:26:09.720 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-21T08:26:09.758 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-21T08:26:09.791 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-21T08:26:09.807 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-21T08:26:09.833 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-21T08:26:09.858 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-21T08:26:09.890 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-21T08:26:09.937 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-21T08:26:10.080 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-21T08:26:10.164 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-21T08:26:10.284 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : librados2-2:19.0.0-3154.ga630765c.el9.x86_64 5/152 2024-04-21T08:26:10.406 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librados2-2:19.0.0-3154.ga630765c.el9.x86_64 5/152 2024-04-21T08:26:10.420 INFO:teuthology.orchestra.run.smithi179.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-21T08:26:10.420 INFO:teuthology.orchestra.run.smithi179.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2024-04-21T08:26:10.426 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T08:26:10.431 INFO:teuthology.orchestra.run.smithi179.stdout:Package NetworkManager-1:1.47.4-1.el9.x86_64 is already installed. 2024-04-21T08:26:10.432 INFO:teuthology.orchestra.run.smithi179.stdout:Package iproute-6.2.0-5.el9.x86_64 is already installed. 2024-04-21T08:26:10.432 INFO:teuthology.orchestra.run.smithi179.stdout:Package util-linux-2.37.4-18.el9.x86_64 is already installed. 2024-04-21T08:26:10.433 INFO:teuthology.orchestra.run.smithi179.stdout:Package libacl-devel-2.3.1-4.el9.x86_64 is already installed. 2024-04-21T08:26:10.435 INFO:teuthology.orchestra.run.smithi179.stdout:Package libattr-devel-2.5.1-3.el9.x86_64 is already installed. 2024-04-21T08:26:10.437 INFO:teuthology.orchestra.run.smithi179.stdout:Package xfsdump-3.1.12-4.el9.x86_64 is already installed. 2024-04-21T08:26:10.438 INFO:teuthology.orchestra.run.smithi179.stdout:Package xfsprogs-6.3.0-1.el9.x86_64 is already installed. 2024-04-21T08:26:10.496 INFO:teuthology.orchestra.run.smithi179.stdout:Dependencies resolved. 2024-04-21T08:26:10.510 INFO:teuthology.orchestra.run.smithi179.stdout:======================================================================================= 2024-04-21T08:26:10.510 INFO:teuthology.orchestra.run.smithi179.stdout: Package Arch Version Repository Size 2024-04-21T08:26:10.511 INFO:teuthology.orchestra.run.smithi179.stdout:======================================================================================= 2024-04-21T08:26:10.511 INFO:teuthology.orchestra.run.smithi179.stdout:Installing: 2024-04-21T08:26:10.511 INFO:teuthology.orchestra.run.smithi179.stdout: bison x86_64 3.7.4-5.el9 appstream 943 k 2024-04-21T08:26:10.511 INFO:teuthology.orchestra.run.smithi179.stdout: ceph x86_64 2:19.0.0-3154.ga630765c.el9 ceph 6.6 k 2024-04-21T08:26:10.511 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-base x86_64 2:19.0.0-3154.ga630765c.el9 ceph 5.4 M 2024-04-21T08:26:10.511 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-fuse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 890 k 2024-04-21T08:26:10.511 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-immutable-object-cache x86_64 2:19.0.0-3154.ga630765c.el9 ceph 145 k 2024-04-21T08:26:10.511 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr x86_64 2:19.0.0-3154.ga630765c.el9 ceph 1.5 M 2024-04-21T08:26:10.511 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-cephadm noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 142 k 2024-04-21T08:26:10.511 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-dashboard noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 3.5 M 2024-04-21T08:26:10.511 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-diskprediction-local noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 7.4 M 2024-04-21T08:26:10.511 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-rook noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 49 k 2024-04-21T08:26:10.511 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-radosgw x86_64 2:19.0.0-3154.ga630765c.el9 ceph 11 M 2024-04-21T08:26:10.512 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-test x86_64 2:19.0.0-3154.ga630765c.el9 ceph 49 M 2024-04-21T08:26:10.512 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-volume noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 273 k 2024-04-21T08:26:10.512 INFO:teuthology.orchestra.run.smithi179.stdout: cephadm noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 762 k 2024-04-21T08:26:10.512 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-mirror x86_64 2:19.0.0-3154.ga630765c.el9 ceph 225 k 2024-04-21T08:26:10.512 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-shell x86_64 2:19.0.0-3154.ga630765c.el9 ceph 27 k 2024-04-21T08:26:10.512 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-top noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 18 k 2024-04-21T08:26:10.512 INFO:teuthology.orchestra.run.smithi179.stdout: elfutils-libelf-devel x86_64 0.190-2.el9 appstream 21 k 2024-04-21T08:26:10.512 INFO:teuthology.orchestra.run.smithi179.stdout: flex x86_64 2.6.4-9.el9 appstream 313 k 2024-04-21T08:26:10.512 INFO:teuthology.orchestra.run.smithi179.stdout: libaio-devel x86_64 0.3.111-13.el9 appstream 11 k 2024-04-21T08:26:10.512 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs-devel x86_64 2:19.0.0-3154.ga630765c.el9 ceph 32 k 2024-04-21T08:26:10.512 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 746 k 2024-04-21T08:26:10.512 INFO:teuthology.orchestra.run.smithi179.stdout: librados-devel x86_64 2:19.0.0-3154.ga630765c.el9 ceph 127 k 2024-04-21T08:26:10.512 INFO:teuthology.orchestra.run.smithi179.stdout: libtool x86_64 2.4.6-45.el9 appstream 582 k 2024-04-21T08:26:10.513 INFO:teuthology.orchestra.run.smithi179.stdout: libuuid-devel x86_64 2.37.4-18.el9 appstream 25 k 2024-04-21T08:26:10.513 INFO:teuthology.orchestra.run.smithi179.stdout: openssl-devel x86_64 1:3.0.7-27.el9 appstream 4.1 M 2024-04-21T08:26:10.513 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql x86_64 13.14-1.el9 appstream 1.6 M 2024-04-21T08:26:10.513 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-contrib x86_64 13.14-1.el9 appstream 911 k 2024-04-21T08:26:10.513 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-server x86_64 13.14-1.el9 appstream 5.8 M 2024-04-21T08:26:10.513 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cephfs x86_64 2:19.0.0-3154.ga630765c.el9 ceph 162 k 2024-04-21T08:26:10.513 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rados x86_64 2:19.0.0-3154.ga630765c.el9 ceph 321 k 2024-04-21T08:26:10.513 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rbd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 298 k 2024-04-21T08:26:10.513 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rgw x86_64 2:19.0.0-3154.ga630765c.el9 ceph 100 k 2024-04-21T08:26:10.513 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-fuse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 85 k 2024-04-21T08:26:10.513 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-mirror x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.1 M 2024-04-21T08:26:10.513 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-nbd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 171 k 2024-04-21T08:26:10.514 INFO:teuthology.orchestra.run.smithi179.stdout: xfsprogs-devel x86_64 6.3.0-1.el9 appstream 94 k 2024-04-21T08:26:10.514 INFO:teuthology.orchestra.run.smithi179.stdout:Upgrading: 2024-04-21T08:26:10.514 INFO:teuthology.orchestra.run.smithi179.stdout: librados2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.4 M 2024-04-21T08:26:10.514 INFO:teuthology.orchestra.run.smithi179.stdout: librbd1 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 3.1 M 2024-04-21T08:26:10.514 INFO:teuthology.orchestra.run.smithi179.stdout:Installing dependencies: 2024-04-21T08:26:10.514 INFO:teuthology.orchestra.run.smithi179.stdout: abseil-cpp x86_64 20211102.0-3.el9 epel 548 k 2024-04-21T08:26:10.514 INFO:teuthology.orchestra.run.smithi179.stdout: automake noarch 1.16.2-8.el9 appstream 689 k 2024-04-21T08:26:10.514 INFO:teuthology.orchestra.run.smithi179.stdout: boost-program-options x86_64 1.75.0-8.el9 appstream 107 k 2024-04-21T08:26:10.514 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-common x86_64 2:19.0.0-3154.ga630765c.el9 ceph 21 M 2024-04-21T08:26:10.514 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-grafana-dashboards noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 34 k 2024-04-21T08:26:10.514 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mds x86_64 2:19.0.0-3154.ga630765c.el9 ceph 2.4 M 2024-04-21T08:26:10.514 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-modules-core noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 249 k 2024-04-21T08:26:10.515 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mon x86_64 2:19.0.0-3154.ga630765c.el9 ceph 4.7 M 2024-04-21T08:26:10.515 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-osd x86_64 2:19.0.0-3154.ga630765c.el9 ceph 17 M 2024-04-21T08:26:10.515 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-prometheus-alerts noarch 2:19.0.0-3154.ga630765c.el9 ceph-noarch 16 k 2024-04-21T08:26:10.515 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-selinux x86_64 2:19.0.0-3154.ga630765c.el9 ceph 25 k 2024-04-21T08:26:10.515 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2024-04-21T08:26:10.515 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2024-04-21T08:26:10.515 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2024-04-21T08:26:10.515 INFO:teuthology.orchestra.run.smithi179.stdout: fuse x86_64 2.9.9-15.el9 baseos 80 k 2024-04-21T08:26:10.515 INFO:teuthology.orchestra.run.smithi179.stdout: gperftools-libs x86_64 2.9.1-2.el9 epel 309 k 2024-04-21T08:26:10.515 INFO:teuthology.orchestra.run.smithi179.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2024-04-21T08:26:10.515 INFO:teuthology.orchestra.run.smithi179.stdout: libarrow x86_64 9.0.0-10.el9 epel 4.4 M 2024-04-21T08:26:10.515 INFO:teuthology.orchestra.run.smithi179.stdout: libarrow-doc noarch 9.0.0-10.el9 epel 26 k 2024-04-21T08:26:10.515 INFO:teuthology.orchestra.run.smithi179.stdout: libcephsqlite x86_64 2:19.0.0-3154.ga630765c.el9 ceph 163 k 2024-04-21T08:26:10.515 INFO:teuthology.orchestra.run.smithi179.stdout: libgfortran x86_64 11.4.1-3.el9 baseos 803 k 2024-04-21T08:26:10.516 INFO:teuthology.orchestra.run.smithi179.stdout: liboath x86_64 2.6.7-2.el9 epel 49 k 2024-04-21T08:26:10.516 INFO:teuthology.orchestra.run.smithi179.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2024-04-21T08:26:10.516 INFO:teuthology.orchestra.run.smithi179.stdout: libquadmath x86_64 11.4.1-3.el9 baseos 195 k 2024-04-21T08:26:10.516 INFO:teuthology.orchestra.run.smithi179.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2024-04-21T08:26:10.516 INFO:teuthology.orchestra.run.smithi179.stdout: libradosstriper1 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 501 k 2024-04-21T08:26:10.516 INFO:teuthology.orchestra.run.smithi179.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2024-04-21T08:26:10.516 INFO:teuthology.orchestra.run.smithi179.stdout: librgw2 x86_64 2:19.0.0-3154.ga630765c.el9 ceph 5.4 M 2024-04-21T08:26:10.516 INFO:teuthology.orchestra.run.smithi179.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2024-04-21T08:26:10.516 INFO:teuthology.orchestra.run.smithi179.stdout: libzstd-devel x86_64 1.5.1-2.el9 appstream 47 k 2024-04-21T08:26:10.516 INFO:teuthology.orchestra.run.smithi179.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2024-04-21T08:26:10.516 INFO:teuthology.orchestra.run.smithi179.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2024-04-21T08:26:10.516 INFO:teuthology.orchestra.run.smithi179.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2024-04-21T08:26:10.516 INFO:teuthology.orchestra.run.smithi179.stdout: luarocks noarch 3.9.2-1.el9 epel 151 k 2024-04-21T08:26:10.517 INFO:teuthology.orchestra.run.smithi179.stdout: openblas x86_64 0.3.21-2.el9 appstream 35 k 2024-04-21T08:26:10.517 INFO:teuthology.orchestra.run.smithi179.stdout: openblas-openmp x86_64 0.3.21-2.el9 appstream 4.7 M 2024-04-21T08:26:10.517 INFO:teuthology.orchestra.run.smithi179.stdout: parquet-libs x86_64 9.0.0-10.el9 epel 838 k 2024-04-21T08:26:10.517 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-private-libs x86_64 13.14-1.el9 appstream 133 k 2024-04-21T08:26:10.517 INFO:teuthology.orchestra.run.smithi179.stdout: protobuf x86_64 3.14.0-13.el9 appstream 1.0 M 2024-04-21T08:26:10.517 INFO:teuthology.orchestra.run.smithi179.stdout: protobuf-compiler x86_64 3.14.0-13.el9 crb 863 k 2024-04-21T08:26:10.517 INFO:teuthology.orchestra.run.smithi179.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2024-04-21T08:26:10.517 INFO:teuthology.orchestra.run.smithi179.stdout: python3-attrs noarch 20.3.0-7.el9 appstream 87 k 2024-04-21T08:26:10.517 INFO:teuthology.orchestra.run.smithi179.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2024-04-21T08:26:10.517 INFO:teuthology.orchestra.run.smithi179.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2024-04-21T08:26:10.517 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2024-04-21T08:26:10.517 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-argparse x86_64 2:19.0.0-3154.ga630765c.el9 ceph 45 k 2024-04-21T08:26:10.517 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-common x86_64 2:19.0.0-3154.ga630765c.el9 ceph 135 k 2024-04-21T08:26:10.517 INFO:teuthology.orchestra.run.smithi179.stdout: python3-certifi noarch 2021.10.8-2.el9 epel 15 k 2024-04-21T08:26:10.517 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2024-04-21T08:26:10.518 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cheroot noarch 8.6.0-4.el9 epel 172 k 2024-04-21T08:26:10.518 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2024-04-21T08:26:10.518 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cmd2 noarch 2.4.2-2.el9 epel 303 k 2024-04-21T08:26:10.518 INFO:teuthology.orchestra.run.smithi179.stdout: python3-colorama noarch 0.4.4-12.el9 epel 35 k 2024-04-21T08:26:10.518 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cryptography x86_64 36.0.1-4.el9 baseos 1.2 M 2024-04-21T08:26:10.518 INFO:teuthology.orchestra.run.smithi179.stdout: python3-devel x86_64 3.9.18-3.el9 appstream 244 k 2024-04-21T08:26:10.518 INFO:teuthology.orchestra.run.smithi179.stdout: python3-google-auth noarch 1:2.28.2-1.el9 epel 203 k 2024-04-21T08:26:10.518 INFO:teuthology.orchestra.run.smithi179.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2024-04-21T08:26:10.518 INFO:teuthology.orchestra.run.smithi179.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2024-04-21T08:26:10.518 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2024-04-21T08:26:10.518 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2024-04-21T08:26:10.518 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2024-04-21T08:26:10.519 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2024-04-21T08:26:10.519 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-text noarch 3.2.0-6.el9 epel 20 k 2024-04-21T08:26:10.519 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jinja2 noarch 2.11.3-5.el9 appstream 248 k 2024-04-21T08:26:10.519 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpatch noarch 1.21-16.el9 appstream 26 k 2024-04-21T08:26:10.519 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpointer noarch 2.0-4.el9 appstream 19 k 2024-04-21T08:26:10.519 INFO:teuthology.orchestra.run.smithi179.stdout: python3-kubernetes noarch 1:26.1.0-2.el9 epel 1.0 M 2024-04-21T08:26:10.519 INFO:teuthology.orchestra.run.smithi179.stdout: python3-logutils noarch 0.3.5-21.el9 epel 46 k 2024-04-21T08:26:10.519 INFO:teuthology.orchestra.run.smithi179.stdout: python3-mako noarch 1.1.4-6.el9 appstream 172 k 2024-04-21T08:26:10.519 INFO:teuthology.orchestra.run.smithi179.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2024-04-21T08:26:10.519 INFO:teuthology.orchestra.run.smithi179.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2024-04-21T08:26:10.519 INFO:teuthology.orchestra.run.smithi179.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2024-04-21T08:26:10.519 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy x86_64 1:1.20.1-5.el9 appstream 5.1 M 2024-04-21T08:26:10.519 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy-f2py x86_64 1:1.20.1-5.el9 appstream 250 k 2024-04-21T08:26:10.520 INFO:teuthology.orchestra.run.smithi179.stdout: python3-oauthlib noarch 3.1.1-5.el9 appstream 222 k 2024-04-21T08:26:10.520 INFO:teuthology.orchestra.run.smithi179.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2024-04-21T08:26:10.520 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pecan noarch 1.4.2-3.el9 epel 272 k 2024-04-21T08:26:10.520 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2024-04-21T08:26:10.520 INFO:teuthology.orchestra.run.smithi179.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2024-04-21T08:26:10.520 INFO:teuthology.orchestra.run.smithi179.stdout: python3-prettytable noarch 0.7.2-27.el9 appstream 42 k 2024-04-21T08:26:10.520 INFO:teuthology.orchestra.run.smithi179.stdout: python3-protobuf noarch 3.14.0-13.el9 appstream 269 k 2024-04-21T08:26:10.520 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2024-04-21T08:26:10.520 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2024-04-21T08:26:10.520 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2024-04-21T08:26:10.520 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2024-04-21T08:26:10.520 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyperclip noarch 1.8.2-2.el9 epel 28 k 2024-04-21T08:26:10.520 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pytz noarch 2021.1-5.el9 appstream 51 k 2024-04-21T08:26:10.520 INFO:teuthology.orchestra.run.smithi179.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2024-04-21T08:26:10.521 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests noarch 2.25.1-8.el9 baseos 125 k 2024-04-21T08:26:10.521 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2024-04-21T08:26:10.521 INFO:teuthology.orchestra.run.smithi179.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2024-04-21T08:26:10.521 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2024-04-21T08:26:10.521 INFO:teuthology.orchestra.run.smithi179.stdout: python3-scipy x86_64 1.6.2-8.el9 appstream 16 M 2024-04-21T08:26:10.521 INFO:teuthology.orchestra.run.smithi179.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2024-04-21T08:26:10.521 INFO:teuthology.orchestra.run.smithi179.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2024-04-21T08:26:10.521 INFO:teuthology.orchestra.run.smithi179.stdout: python3-typing-extensions noarch 4.4.0-2.el9 epel 51 k 2024-04-21T08:26:10.521 INFO:teuthology.orchestra.run.smithi179.stdout: python3-urllib3 noarch 1.26.5-5.el9 baseos 215 k 2024-04-21T08:26:10.522 INFO:teuthology.orchestra.run.smithi179.stdout: python3-wcwidth noarch 0.2.5-8.el9 appstream 44 k 2024-04-21T08:26:10.522 INFO:teuthology.orchestra.run.smithi179.stdout: python3-webob noarch 1.8.7-6.el9 epel 230 k 2024-04-21T08:26:10.522 INFO:teuthology.orchestra.run.smithi179.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2024-04-21T08:26:10.522 INFO:teuthology.orchestra.run.smithi179.stdout: python3-werkzeug noarch 2.0.3-3.el9.1 epel 427 k 2024-04-21T08:26:10.522 INFO:teuthology.orchestra.run.smithi179.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2024-04-21T08:26:10.522 INFO:teuthology.orchestra.run.smithi179.stdout: qatlib x86_64 24.02.0-1.el9 appstream 221 k 2024-04-21T08:26:10.522 INFO:teuthology.orchestra.run.smithi179.stdout: qatzip-libs x86_64 1.2.0-1.el9 appstream 47 k 2024-04-21T08:26:10.522 INFO:teuthology.orchestra.run.smithi179.stdout: re2 x86_64 1:20211101-3.el9 epel 192 k 2024-04-21T08:26:10.522 INFO:teuthology.orchestra.run.smithi179.stdout: socat x86_64 1.7.4.1-5.el9 appstream 305 k 2024-04-21T08:26:10.522 INFO:teuthology.orchestra.run.smithi179.stdout: thrift x86_64 0.15.0-2.el9 epel 1.6 M 2024-04-21T08:26:10.522 INFO:teuthology.orchestra.run.smithi179.stdout: uuid x86_64 1.6.2-55.el9 appstream 58 k 2024-04-21T08:26:10.522 INFO:teuthology.orchestra.run.smithi179.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2024-04-21T08:26:10.522 INFO:teuthology.orchestra.run.smithi179.stdout: zlib-devel x86_64 1.2.11-41.el9 appstream 45 k 2024-04-21T08:26:10.523 INFO:teuthology.orchestra.run.smithi179.stdout:Installing weak dependencies: 2024-04-21T08:26:10.523 INFO:teuthology.orchestra.run.smithi179.stdout: qatlib-service x86_64 24.02.0-1.el9 appstream 36 k 2024-04-21T08:26:10.523 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-21T08:26:10.523 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction Summary 2024-04-21T08:26:10.523 INFO:teuthology.orchestra.run.smithi179.stdout:======================================================================================= 2024-04-21T08:26:10.523 INFO:teuthology.orchestra.run.smithi179.stdout:Install 148 Packages 2024-04-21T08:26:10.523 INFO:teuthology.orchestra.run.smithi179.stdout:Upgrade 2 Packages 2024-04-21T08:26:10.523 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-21T08:26:10.524 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T08:26:10.524 INFO:teuthology.orchestra.run.smithi179.stdout:Total download size: 218 M 2024-04-21T08:26:10.524 INFO:teuthology.orchestra.run.smithi179.stdout:Downloading Packages: 2024-04-21T08:26:10.546 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-21T08:26:10.559 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 7/152 2024-04-21T08:26:10.589 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-21T08:26:10.615 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-21T08:26:10.765 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-21T08:26:10.829 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-21T08:26:10.857 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T08:26:10.960 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T08:26:10.983 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 13/152 2024-04-21T08:26:11.045 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 13/152 2024-04-21T08:26:11.067 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-21T08:26:11.090 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-21T08:26:11.275 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-21T08:26:11.278 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-21T08:26:11.308 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-21T08:26:11.321 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-21T08:26:11.358 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-21T08:26:11.414 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-21T08:26:11.456 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-21T08:26:11.513 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-21T08:26:11.513 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-21T08:26:11.529 INFO:teuthology.orchestra.run.smithi179.stdout:(1/150): ceph-19.0.0-3154.ga630765c.el9.x86_64. 39 kB/s | 6.6 kB 00:00 2024-04-21T08:26:11.549 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-21T08:26:11.585 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-21T08:26:11.602 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-21T08:26:11.641 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-21T08:26:11.658 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-21T08:26:11.700 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-21T08:26:11.744 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-21T08:26:11.776 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-21T08:26:11.787 INFO:teuthology.orchestra.run.smithi179.stdout:(2/150): ceph-fuse-19.0.0-3154.ga630765c.el9.x8 3.4 MB/s | 890 kB 00:00 2024-04-21T08:26:11.804 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-21T08:26:11.816 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-21T08:26:11.838 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-21T08:26:11.854 INFO:teuthology.orchestra.run.smithi179.stdout:(3/150): ceph-immutable-object-cache-19.0.0-315 2.1 MB/s | 145 kB 00:00 2024-04-21T08:26:11.867 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-21T08:26:11.907 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-21T08:26:11.921 INFO:teuthology.orchestra.run.smithi179.stdout:(4/150): ceph-base-19.0.0-3154.ga630765c.el9.x8 9.6 MB/s | 5.4 MB 00:00 2024-04-21T08:26:11.927 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-21T08:26:11.957 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-21T08:26:12.001 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-21T08:26:12.064 INFO:teuthology.orchestra.run.smithi179.stdout:(5/150): ceph-mgr-19.0.0-3154.ga630765c.el9.x86 11 MB/s | 1.5 MB 00:00 2024-04-21T08:26:12.078 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-21T08:26:12.111 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-ceph-common-2:19.0.0-3154.ga630765c.el9. 30/152 2024-04-21T08:26:12.130 INFO:teuthology.orchestra.run.smithi179.stdout:(6/150): ceph-mds-19.0.0-3154.ga630765c.el9.x86 8.6 MB/s | 2.4 MB 00:00 2024-04-21T08:26:12.155 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-ceph-argparse-2:19.0.0-3154.ga630765c.el 31/152 2024-04-21T08:26:12.183 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_6 32/152 2024-04-21T08:26:12.255 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-21T08:26:12.298 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-21T08:26:12.374 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-21T08:26:12.406 INFO:teuthology.orchestra.run.smithi179.stdout:(7/150): ceph-mon-19.0.0-3154.ga630765c.el9.x86 14 MB/s | 4.7 MB 00:00 2024-04-21T08:26:12.509 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-21T08:26:12.561 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-21T08:26:12.567 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-21T08:26:12.679 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-21T08:26:13.101 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-21T08:26:13.187 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-21T08:26:13.217 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-21T08:26:13.248 INFO:teuthology.orchestra.run.smithi179.stdout:(8/150): ceph-radosgw-19.0.0-3154.ga630765c.el9 13 MB/s | 11 MB 00:00 2024-04-21T08:26:13.259 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-21T08:26:13.297 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : librados-devel-2:19.0.0-3154.ga630765c.el9.x86_6 42/152 2024-04-21T08:26:13.298 INFO:teuthology.orchestra.run.smithi179.stdout:(9/150): ceph-selinux-19.0.0-3154.ga630765c.el9 505 kB/s | 25 kB 00:00 2024-04-21T08:26:13.330 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-21T08:26:13.385 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-21T08:26:13.458 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-21T08:26:13.490 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-21T08:26:13.526 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-21T08:26:13.572 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-21T08:26:13.609 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-21T08:26:13.652 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-21T08:26:13.693 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-21T08:26:13.829 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-21T08:26:13.866 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-21T08:26:13.896 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-21T08:26:13.923 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-21T08:26:13.937 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-21T08:26:14.009 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-21T08:26:14.032 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-21T08:26:14.037 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-21T08:26:14.070 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-21T08:26:14.079 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-21T08:26:14.119 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-21T08:26:14.145 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-21T08:26:14.165 INFO:teuthology.orchestra.run.smithi179.stdout:(10/150): ceph-osd-19.0.0-3154.ga630765c.el9.x8 8.3 MB/s | 17 MB 00:02 2024-04-21T08:26:14.169 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-21T08:26:14.224 INFO:teuthology.orchestra.run.smithi179.stdout:(11/150): cephfs-mirror-19.0.0-3154.ga630765c.e 3.8 MB/s | 225 kB 00:00 2024-04-21T08:26:14.245 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-21T08:26:14.291 INFO:teuthology.orchestra.run.smithi179.stdout:(12/150): cephfs-shell-19.0.0-3154.ga630765c.el 405 kB/s | 27 kB 00:00 2024-04-21T08:26:14.358 INFO:teuthology.orchestra.run.smithi179.stdout:(13/150): libcephfs-devel-19.0.0-3154.ga630765c 475 kB/s | 32 kB 00:00 2024-04-21T08:26:14.372 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-21T08:26:14.382 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-21T08:26:14.410 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-21T08:26:14.433 INFO:teuthology.orchestra.run.smithi086.stdout: Upgrading : librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 97/152 2024-04-21T08:26:14.438 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-21T08:26:14.473 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-21T08:26:14.508 INFO:teuthology.orchestra.run.smithi179.stdout:(14/150): libcephfs2-19.0.0-3154.ga630765c.el9. 4.9 MB/s | 746 kB 00:00 2024-04-21T08:26:14.517 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-21T08:26:14.523 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 97/152 2024-04-21T08:26:14.554 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 98/152 2024-04-21T08:26:14.563 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-21T08:26:14.583 INFO:teuthology.orchestra.run.smithi179.stdout:(15/150): libcephsqlite-19.0.0-3154.ga630765c.e 2.1 MB/s | 163 kB 00:00 2024-04-21T08:26:14.590 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-21T08:26:14.594 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-21T08:26:14.634 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-21T08:26:14.659 INFO:teuthology.orchestra.run.smithi179.stdout:(16/150): librados-devel-19.0.0-3154.ga630765c. 1.6 MB/s | 127 kB 00:00 2024-04-21T08:26:14.666 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-21T08:26:14.700 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-21T08:26:14.741 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-21T08:26:14.751 INFO:teuthology.orchestra.run.smithi179.stdout:(17/150): libradosstriper1-19.0.0-3154.ga630765 5.3 MB/s | 501 kB 00:00 2024-04-21T08:26:14.776 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-21T08:26:14.863 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-21T08:26:14.906 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-21T08:26:15.069 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-21T08:26:15.102 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 101/152 2024-04-21T08:26:15.114 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-21T08:26:15.173 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 101/152 2024-04-21T08:26:15.195 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 102/152 2024-04-21T08:26:15.204 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-21T08:26:15.314 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-21T08:26:15.365 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-21T08:26:15.618 INFO:teuthology.orchestra.run.smithi179.stdout:(18/150): ceph-common-19.0.0-3154.ga630765c.el9 5.0 MB/s | 21 MB 00:04 2024-04-21T08:26:15.685 INFO:teuthology.orchestra.run.smithi179.stdout:(19/150): librgw2-19.0.0-3154.ga630765c.el9.x86 5.8 MB/s | 5.4 MB 00:00 2024-04-21T08:26:15.710 INFO:teuthology.orchestra.run.smithi179.stdout:(20/150): python3-ceph-argparse-19.0.0-3154.ga6 491 kB/s | 45 kB 00:00 2024-04-21T08:26:15.744 INFO:teuthology.orchestra.run.smithi179.stdout:(21/150): python3-ceph-common-19.0.0-3154.ga630 2.3 MB/s | 135 kB 00:00 2024-04-21T08:26:15.769 INFO:teuthology.orchestra.run.smithi179.stdout:(22/150): python3-cephfs-19.0.0-3154.ga630765c. 2.7 MB/s | 162 kB 00:00 2024-04-21T08:26:15.795 INFO:teuthology.orchestra.run.smithi179.stdout:(23/150): python3-rados-19.0.0-3154.ga630765c.e 6.2 MB/s | 321 kB 00:00 2024-04-21T08:26:15.854 INFO:teuthology.orchestra.run.smithi179.stdout:(24/150): python3-rbd-19.0.0-3154.ga630765c.el9 3.5 MB/s | 298 kB 00:00 2024-04-21T08:26:15.879 INFO:teuthology.orchestra.run.smithi179.stdout:(25/150): python3-rgw-19.0.0-3154.ga630765c.el9 1.2 MB/s | 100 kB 00:00 2024-04-21T08:26:15.929 INFO:teuthology.orchestra.run.smithi179.stdout:(26/150): rbd-fuse-19.0.0-3154.ga630765c.el9.x8 1.1 MB/s | 85 kB 00:00 2024-04-21T08:26:16.038 INFO:teuthology.orchestra.run.smithi179.stdout:(27/150): rbd-nbd-19.0.0-3154.ga630765c.el9.x86 1.5 MB/s | 171 kB 00:00 2024-04-21T08:26:16.087 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-21T08:26:16.136 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-21T08:26:16.178 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-21T08:26:16.205 INFO:teuthology.orchestra.run.smithi179.stdout:(28/150): ceph-grafana-dashboards-19.0.0-3154.g 205 kB/s | 34 kB 00:00 2024-04-21T08:26:16.278 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-21T08:26:16.336 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-21T08:26:16.423 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-21T08:26:16.455 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-21T08:26:16.632 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-21T08:26:16.647 INFO:teuthology.orchestra.run.smithi179.stdout:(29/150): ceph-test-19.0.0-3154.ga630765c.el9.x 15 MB/s | 49 MB 00:03 2024-04-21T08:26:16.661 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-21T08:26:16.672 INFO:teuthology.orchestra.run.smithi179.stdout:(30/150): ceph-mgr-cephadm-19.0.0-3154.ga630765 305 kB/s | 142 kB 00:00 2024-04-21T08:26:16.731 INFO:teuthology.orchestra.run.smithi179.stdout:(31/150): rbd-mirror-19.0.0-3154.ga630765c.el9. 3.6 MB/s | 3.1 MB 00:00 2024-04-21T08:26:16.738 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-21T08:26:16.865 INFO:teuthology.orchestra.run.smithi179.stdout:(32/150): ceph-mgr-modules-core-19.0.0-3154.ga6 1.8 MB/s | 249 kB 00:00 2024-04-21T08:26:16.923 INFO:teuthology.orchestra.run.smithi179.stdout:(33/150): ceph-mgr-dashboard-19.0.0-3154.ga6307 13 MB/s | 3.5 MB 00:00 2024-04-21T08:26:16.949 INFO:teuthology.orchestra.run.smithi179.stdout:(34/150): ceph-mgr-rook-19.0.0-3154.ga630765c.e 589 kB/s | 49 kB 00:00 2024-04-21T08:26:16.974 INFO:teuthology.orchestra.run.smithi179.stdout:(35/150): ceph-prometheus-alerts-19.0.0-3154.ga 328 kB/s | 16 kB 00:00 2024-04-21T08:26:16.999 INFO:teuthology.orchestra.run.smithi179.stdout:(36/150): ceph-volume-19.0.0-3154.ga630765c.el9 5.3 MB/s | 273 kB 00:00 2024-04-21T08:26:17.050 INFO:teuthology.orchestra.run.smithi179.stdout:(37/150): cephadm-19.0.0-3154.ga630765c.el9.noa 9.9 MB/s | 762 kB 00:00 2024-04-21T08:26:17.067 INFO:teuthology.orchestra.run.smithi179.stdout:(38/150): cephfs-top-19.0.0-3154.ga630765c.el9. 273 kB/s | 18 kB 00:00 2024-04-21T08:26:17.175 INFO:teuthology.orchestra.run.smithi179.stdout:(39/150): fuse-2.9.9-15.el9.x86_64.rpm 641 kB/s | 80 kB 00:00 2024-04-21T08:26:17.243 INFO:teuthology.orchestra.run.smithi179.stdout:(40/150): libquadmath-11.4.1-3.el9.x86_64.rpm 2.8 MB/s | 195 kB 00:00 2024-04-21T08:26:17.268 INFO:teuthology.orchestra.run.smithi179.stdout:(41/150): libgfortran-11.4.1-3.el9.x86_64.rpm 3.9 MB/s | 803 kB 00:00 2024-04-21T08:26:17.293 INFO:teuthology.orchestra.run.smithi179.stdout:(42/150): python3-cffi-1.14.5-5.el9.x86_64.rpm 4.9 MB/s | 253 kB 00:00 2024-04-21T08:26:17.352 INFO:teuthology.orchestra.run.smithi179.stdout:(43/150): python3-cryptography-36.0.1-4.el9.x86 15 MB/s | 1.2 MB 00:00 2024-04-21T08:26:17.372 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-21T08:26:17.377 INFO:teuthology.orchestra.run.smithi179.stdout:(44/150): python3-ply-3.11-14.el9.noarch.rpm 1.2 MB/s | 106 kB 00:00 2024-04-21T08:26:17.402 INFO:teuthology.orchestra.run.smithi179.stdout:(45/150): python3-pycparser-2.20-6.el9.noarch.r 2.6 MB/s | 135 kB 00:00 2024-04-21T08:26:17.428 INFO:teuthology.orchestra.run.smithi179.stdout:(46/150): python3-requests-2.25.1-8.el9.noarch. 2.4 MB/s | 125 kB 00:00 2024-04-21T08:26:17.453 INFO:teuthology.orchestra.run.smithi179.stdout:(47/150): python3-urllib3-1.26.5-5.el9.noarch.r 4.2 MB/s | 215 kB 00:00 2024-04-21T08:26:17.493 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-21T08:26:17.599 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T08:26:17.642 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T08:26:17.648 INFO:teuthology.orchestra.run.smithi179.stdout:(48/150): ceph-mgr-diskprediction-local-19.0.0- 7.6 MB/s | 7.4 MB 00:00 2024-04-21T08:26:17.729 INFO:teuthology.orchestra.run.smithi179.stdout:(49/150): automake-1.16.2-8.el9.noarch.rpm 2.2 MB/s | 689 kB 00:00 2024-04-21T08:26:17.771 INFO:teuthology.orchestra.run.smithi179.stdout:(50/150): bison-3.7.4-5.el9.x86_64.rpm 2.9 MB/s | 943 kB 00:00 2024-04-21T08:26:17.796 INFO:teuthology.orchestra.run.smithi179.stdout:(51/150): boost-program-options-1.75.0-8.el9.x8 726 kB/s | 107 kB 00:00 2024-04-21T08:26:17.821 INFO:teuthology.orchestra.run.smithi179.stdout:(52/150): elfutils-libelf-devel-0.190-2.el9.x86 234 kB/s | 21 kB 00:00 2024-04-21T08:26:17.846 INFO:teuthology.orchestra.run.smithi179.stdout:(53/150): flex-2.6.4-9.el9.x86_64.rpm 4.1 MB/s | 313 kB 00:00 2024-04-21T08:26:17.872 INFO:teuthology.orchestra.run.smithi179.stdout:(54/150): flexiblas-3.0.4-8.el9.x86_64.rpm 651 kB/s | 33 kB 00:00 2024-04-21T08:26:17.897 INFO:teuthology.orchestra.run.smithi179.stdout:(55/150): flexiblas-openblas-openmp-3.0.4-8.el9 361 kB/s | 18 kB 00:00 2024-04-21T08:26:17.922 INFO:teuthology.orchestra.run.smithi179.stdout:(56/150): libaio-devel-0.3.111-13.el9.x86_64.rp 223 kB/s | 11 kB 00:00 2024-04-21T08:26:17.948 INFO:teuthology.orchestra.run.smithi179.stdout:(57/150): libpmemobj-1.12.1-1.el9.x86_64.rpm 3.1 MB/s | 160 kB 00:00 2024-04-21T08:26:17.973 INFO:teuthology.orchestra.run.smithi179.stdout:(58/150): librabbitmq-0.11.0-7.el9.x86_64.rpm 897 kB/s | 45 kB 00:00 2024-04-21T08:26:18.015 INFO:teuthology.orchestra.run.smithi179.stdout:(59/150): librdkafka-1.6.1-102.el9.x86_64.rpm 9.7 MB/s | 662 kB 00:00 2024-04-21T08:26:18.057 INFO:teuthology.orchestra.run.smithi179.stdout:(60/150): libtool-2.4.6-45.el9.x86_64.rpm 6.8 MB/s | 582 kB 00:00 2024-04-21T08:26:18.082 INFO:teuthology.orchestra.run.smithi179.stdout:(61/150): libuuid-devel-2.37.4-18.el9.x86_64.rp 382 kB/s | 25 kB 00:00 2024-04-21T08:26:18.108 INFO:teuthology.orchestra.run.smithi179.stdout:(62/150): libzstd-devel-1.5.1-2.el9.x86_64.rpm 941 kB/s | 47 kB 00:00 2024-04-21T08:26:18.158 INFO:teuthology.orchestra.run.smithi179.stdout:(63/150): flexiblas-netlib-3.0.4-8.el9.x86_64.r 8.9 MB/s | 3.0 MB 00:00 2024-04-21T08:26:18.183 INFO:teuthology.orchestra.run.smithi179.stdout:(64/150): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.8 MB/s | 292 kB 00:00 2024-04-21T08:26:18.189 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T08:26:18.209 INFO:teuthology.orchestra.run.smithi179.stdout:(65/150): openblas-0.3.21-2.el9.x86_64.rpm 701 kB/s | 35 kB 00:00 2024-04-21T08:26:18.229 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 104/152 2024-04-21T08:26:18.234 INFO:teuthology.orchestra.run.smithi179.stdout:(66/150): lua-5.4.4-4.el9.x86_64.rpm 1.5 MB/s | 188 kB 00:00 2024-04-21T08:26:18.321 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 104/152 2024-04-21T08:26:18.321 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-21T08:26:18.321 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-21T08:26:18.321 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:26:18.343 INFO:teuthology.orchestra.run.smithi179.stdout:(67/150): openssl-devel-3.0.7-27.el9.x86_64.rpm 31 MB/s | 4.1 MB 00:00 2024-04-21T08:26:18.355 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 105/152 2024-04-21T08:26:18.402 INFO:teuthology.orchestra.run.smithi179.stdout:(68/150): postgresql-contrib-13.14-1.el9.x86_64 15 MB/s | 911 kB 00:00 2024-04-21T08:26:18.435 INFO:teuthology.orchestra.run.smithi179.stdout:(69/150): postgresql-13.14-1.el9.x86_64.rpm 7.9 MB/s | 1.6 MB 00:00 2024-04-21T08:26:18.460 INFO:teuthology.orchestra.run.smithi179.stdout:(70/150): postgresql-private-libs-13.14-1.el9.x 2.2 MB/s | 133 kB 00:00 2024-04-21T08:26:18.519 INFO:teuthology.orchestra.run.smithi179.stdout:(71/150): openblas-openmp-0.3.21-2.el9.x86_64.r 14 MB/s | 4.7 MB 00:00 2024-04-21T08:26:18.553 INFO:teuthology.orchestra.run.smithi179.stdout:(72/150): protobuf-3.14.0-13.el9.x86_64.rpm 11 MB/s | 1.0 MB 00:00 2024-04-21T08:26:18.578 INFO:teuthology.orchestra.run.smithi179.stdout:(73/150): python3-attrs-20.3.0-7.el9.noarch.rpm 1.5 MB/s | 87 kB 00:00 2024-04-21T08:26:18.653 INFO:teuthology.orchestra.run.smithi179.stdout:(74/150): python3-devel-3.9.18-3.el9.x86_64.rpm 3.2 MB/s | 244 kB 00:00 2024-04-21T08:26:18.737 INFO:teuthology.orchestra.run.smithi179.stdout:(75/150): python3-babel-2.9.1-2.el9.noarch.rpm 32 MB/s | 6.0 MB 00:00 2024-04-21T08:26:18.762 INFO:teuthology.orchestra.run.smithi179.stdout:(76/150): python3-jinja2-2.11.3-5.el9.noarch.rp 2.2 MB/s | 248 kB 00:00 2024-04-21T08:26:18.787 INFO:teuthology.orchestra.run.smithi179.stdout:(77/150): python3-jsonpatch-1.21-16.el9.noarch. 512 kB/s | 26 kB 00:00 2024-04-21T08:26:18.813 INFO:teuthology.orchestra.run.smithi179.stdout:(78/150): python3-jsonpointer-2.0-4.el9.noarch. 380 kB/s | 19 kB 00:00 2024-04-21T08:26:18.838 INFO:teuthology.orchestra.run.smithi179.stdout:(79/150): python3-mako-1.1.4-6.el9.noarch.rpm 3.3 MB/s | 172 kB 00:00 2024-04-21T08:26:18.844 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-21T08:26:18.858 INFO:teuthology.orchestra.run.smithi179.stdout:(80/150): python3-markupsafe-1.1.1-12.el9.x86_6 827 kB/s | 35 kB 00:00 2024-04-21T08:26:18.931 INFO:teuthology.orchestra.run.smithi179.stdout:(81/150): postgresql-server-13.14-1.el9.x86_64. 12 MB/s | 5.8 MB 00:00 2024-04-21T08:26:18.931 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-21T08:26:18.956 INFO:teuthology.orchestra.run.smithi179.stdout:(82/150): python3-numpy-f2py-1.20.1-5.el9.x86_6 2.4 MB/s | 250 kB 00:00 2024-04-21T08:26:18.959 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-21T08:26:18.989 INFO:teuthology.orchestra.run.smithi179.stdout:(83/150): python3-oauthlib-3.1.1-5.el9.noarch.r 3.7 MB/s | 222 kB 00:00 2024-04-21T08:26:18.992 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-21T08:26:19.015 INFO:teuthology.orchestra.run.smithi179.stdout:(84/150): python3-packaging-20.9-5.el9.noarch.r 1.3 MB/s | 77 kB 00:00 2024-04-21T08:26:19.040 INFO:teuthology.orchestra.run.smithi179.stdout:(85/150): python3-prettytable-0.7.2-27.el9.noar 839 kB/s | 42 kB 00:00 2024-04-21T08:26:19.065 INFO:teuthology.orchestra.run.smithi179.stdout:(86/150): python3-protobuf-3.14.0-13.el9.noarch 5.2 MB/s | 269 kB 00:00 2024-04-21T08:26:19.091 INFO:teuthology.orchestra.run.smithi179.stdout:(87/150): python3-pyasn1-0.4.8-6.el9.noarch.rpm 3.1 MB/s | 159 kB 00:00 2024-04-21T08:26:19.144 INFO:teuthology.orchestra.run.smithi179.stdout:(88/150): python3-pyasn1-modules-0.4.8-6.el9.no 3.5 MB/s | 279 kB 00:00 2024-04-21T08:26:19.166 INFO:teuthology.orchestra.run.smithi179.stdout:(89/150): python3-pytz-2021.1-5.el9.noarch.rpm 677 kB/s | 51 kB 00:00 2024-04-21T08:26:19.192 INFO:teuthology.orchestra.run.smithi179.stdout:(90/150): python3-requests-oauthlib-1.3.0-12.el 1.1 MB/s | 54 kB 00:00 2024-04-21T08:26:19.267 INFO:teuthology.orchestra.run.smithi179.stdout:(91/150): python3-numpy-1.20.1-5.el9.x86_64.rpm 12 MB/s | 5.1 MB 00:00 2024-04-21T08:26:19.293 INFO:teuthology.orchestra.run.smithi179.stdout:(92/150): python3-toml-0.10.2-6.el9.noarch.rpm 414 kB/s | 42 kB 00:00 2024-04-21T08:26:19.305 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-21T08:26:19.347 INFO:teuthology.orchestra.run.smithi088.stdout: Upgrading : librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 97/152 2024-04-21T08:26:19.348 INFO:teuthology.orchestra.run.smithi179.stdout:(93/150): python3-wcwidth-0.2.5-8.el9.noarch.rp 877 kB/s | 44 kB 00:00 2024-04-21T08:26:19.348 INFO:teuthology.orchestra.run.smithi179.stdout:(94/150): qatlib-24.02.0-1.el9.x86_64.rpm 4.3 MB/s | 221 kB 00:00 2024-04-21T08:26:19.369 INFO:teuthology.orchestra.run.smithi179.stdout:(95/150): qatlib-service-24.02.0-1.el9.x86_64.r 712 kB/s | 36 kB 00:00 2024-04-21T08:26:19.394 INFO:teuthology.orchestra.run.smithi179.stdout:(96/150): qatzip-libs-1.2.0-1.el9.x86_64.rpm 942 kB/s | 47 kB 00:00 2024-04-21T08:26:19.419 INFO:teuthology.orchestra.run.smithi179.stdout:(97/150): socat-1.7.4.1-5.el9.x86_64.rpm 5.9 MB/s | 305 kB 00:00 2024-04-21T08:26:19.438 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 97/152 2024-04-21T08:26:19.445 INFO:teuthology.orchestra.run.smithi179.stdout:(98/150): uuid-1.6.2-55.el9.x86_64.rpm 1.1 MB/s | 58 kB 00:00 2024-04-21T08:26:19.469 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 98/152 2024-04-21T08:26:19.470 INFO:teuthology.orchestra.run.smithi179.stdout:(99/150): xfsprogs-devel-6.3.0-1.el9.x86_64.rpm 1.8 MB/s | 94 kB 00:00 2024-04-21T08:26:19.495 INFO:teuthology.orchestra.run.smithi179.stdout:(100/150): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.2 MB/s | 64 kB 00:00 2024-04-21T08:26:19.505 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-21T08:26:19.521 INFO:teuthology.orchestra.run.smithi179.stdout:(101/150): zlib-devel-1.2.11-41.el9.x86_64.rpm 887 kB/s | 45 kB 00:00 2024-04-21T08:26:19.896 INFO:teuthology.orchestra.run.smithi179.stdout:(102/150): python3-scipy-1.6.2-8.el9.x86_64.rpm 22 MB/s | 16 MB 00:00 2024-04-21T08:26:19.985 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-21T08:26:20.016 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 101/152 2024-04-21T08:26:20.088 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 101/152 2024-04-21T08:26:20.118 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 102/152 2024-04-21T08:26:20.296 INFO:teuthology.orchestra.run.smithi179.stdout:(103/150): lua-devel-5.4.4-4.el9.x86_64.rpm 28 kB/s | 22 kB 00:00 2024-04-21T08:26:20.397 INFO:teuthology.orchestra.run.smithi179.stdout:(104/150): abseil-cpp-20211102.0-3.el9.x86_64.r 1.2 MB/s | 548 kB 00:00 2024-04-21T08:26:20.422 INFO:teuthology.orchestra.run.smithi179.stdout:(105/150): gperftools-libs-2.9.1-2.el9.x86_64.r 2.4 MB/s | 309 kB 00:00 2024-04-21T08:26:20.472 INFO:teuthology.orchestra.run.smithi179.stdout:(106/150): grpc-data-1.46.7-10.el9.noarch.rpm 259 kB/s | 19 kB 00:00 2024-04-21T08:26:20.615 INFO:teuthology.orchestra.run.smithi179.stdout:(107/150): libarrow-doc-9.0.0-10.el9.noarch.rpm 184 kB/s | 26 kB 00:00 2024-04-21T08:26:20.673 INFO:teuthology.orchestra.run.smithi179.stdout:(108/150): libarrow-9.0.0-10.el9.x86_64.rpm 18 MB/s | 4.4 MB 00:00 2024-04-21T08:26:20.698 INFO:teuthology.orchestra.run.smithi179.stdout:(109/150): liboath-2.6.7-2.el9.x86_64.rpm 585 kB/s | 49 kB 00:00 2024-04-21T08:26:20.732 INFO:teuthology.orchestra.run.smithi179.stdout:(110/150): libunwind-1.6.2-1.el9.x86_64.rpm 1.1 MB/s | 67 kB 00:00 2024-04-21T08:26:20.774 INFO:teuthology.orchestra.run.smithi179.stdout:(111/150): luarocks-3.9.2-1.el9.noarch.rpm 2.0 MB/s | 151 kB 00:00 2024-04-21T08:26:20.816 INFO:teuthology.orchestra.run.smithi179.stdout:(112/150): parquet-libs-9.0.0-10.el9.x86_64.rpm 9.8 MB/s | 838 kB 00:00 2024-04-21T08:26:20.849 INFO:teuthology.orchestra.run.smithi179.stdout:(113/150): python3-asyncssh-2.13.2-5.el9.noarch 7.1 MB/s | 548 kB 00:00 2024-04-21T08:26:20.883 INFO:teuthology.orchestra.run.smithi179.stdout:(114/150): python3-bcrypt-3.2.2-1.el9.x86_64.rp 646 kB/s | 43 kB 00:00 2024-04-21T08:26:20.917 INFO:teuthology.orchestra.run.smithi179.stdout:(115/150): python3-cachetools-4.2.4-1.el9.noarc 480 kB/s | 32 kB 00:00 2024-04-21T08:26:20.950 INFO:teuthology.orchestra.run.smithi179.stdout:(116/150): python3-certifi-2021.10.8-2.el9.noar 224 kB/s | 15 kB 00:00 2024-04-21T08:26:20.993 INFO:teuthology.orchestra.run.smithi179.stdout:(117/150): python3-cheroot-8.6.0-4.el9.noarch.r 2.2 MB/s | 172 kB 00:00 2024-04-21T08:26:21.026 INFO:teuthology.orchestra.run.smithi179.stdout:(118/150): python3-cherrypy-18.6.1-2.el9.noarch 4.6 MB/s | 358 kB 00:00 2024-04-21T08:26:21.068 INFO:teuthology.orchestra.run.smithi179.stdout:(119/150): python3-cmd2-2.4.2-2.el9.noarch.rpm 3.9 MB/s | 303 kB 00:00 2024-04-21T08:26:21.085 INFO:teuthology.orchestra.run.smithi179.stdout:(120/150): python3-colorama-0.4.4-12.el9.noarch 600 kB/s | 35 kB 00:00 2024-04-21T08:26:21.144 INFO:teuthology.orchestra.run.smithi179.stdout:(121/150): python3-google-auth-2.28.2-1.el9.noa 2.6 MB/s | 203 kB 00:00 2024-04-21T08:26:21.194 INFO:teuthology.orchestra.run.smithi179.stdout:(122/150): python3-grpcio-1.46.7-10.el9.x86_64. 19 MB/s | 2.0 MB 00:00 2024-04-21T08:26:21.211 INFO:teuthology.orchestra.run.smithi179.stdout:(123/150): python3-grpcio-tools-1.46.7-10.el9.x 2.2 MB/s | 144 kB 00:00 2024-04-21T08:26:21.345 INFO:teuthology.orchestra.run.smithi179.stdout:(124/150): python3-jaraco-8.2.1-3.el9.noarch.rp 71 kB/s | 11 kB 00:00 2024-04-21T08:26:21.371 INFO:teuthology.orchestra.run.smithi179.stdout:(125/150): python3-jaraco-classes-3.2.1-5.el9.n 111 kB/s | 18 kB 00:00 2024-04-21T08:26:21.413 INFO:teuthology.orchestra.run.smithi179.stdout:(126/150): python3-jaraco-collections-3.0.0-8.e 346 kB/s | 23 kB 00:00 2024-04-21T08:26:21.438 INFO:teuthology.orchestra.run.smithi179.stdout:(127/150): python3-jaraco-functools-3.5.0-2.el9 290 kB/s | 19 kB 00:00 2024-04-21T08:26:21.480 INFO:teuthology.orchestra.run.smithi179.stdout:(128/150): protobuf-compiler-3.14.0-13.el9.x86_ 441 kB/s | 863 kB 00:01 2024-04-21T08:26:21.505 INFO:teuthology.orchestra.run.smithi179.stdout:(129/150): python3-jaraco-text-3.2.0-6.el9.noar 213 kB/s | 20 kB 00:00 2024-04-21T08:26:21.539 INFO:teuthology.orchestra.run.smithi179.stdout:(130/150): python3-kubernetes-26.1.0-2.el9.noar 10 MB/s | 1.0 MB 00:00 2024-04-21T08:26:21.564 INFO:teuthology.orchestra.run.smithi179.stdout:(131/150): python3-logutils-0.3.5-21.el9.noarch 554 kB/s | 46 kB 00:00 2024-04-21T08:26:21.590 INFO:teuthology.orchestra.run.smithi179.stdout:(132/150): python3-more-itertools-8.12.0-2.el9. 963 kB/s | 79 kB 00:00 2024-04-21T08:26:21.615 INFO:teuthology.orchestra.run.smithi179.stdout:(133/150): python3-natsort-7.1.1-5.el9.noarch.r 766 kB/s | 58 kB 00:00 2024-04-21T08:26:21.640 INFO:teuthology.orchestra.run.smithi179.stdout:(134/150): python3-pecan-1.4.2-3.el9.noarch.rpm 3.5 MB/s | 272 kB 00:00 2024-04-21T08:26:21.666 INFO:teuthology.orchestra.run.smithi179.stdout:(135/150): python3-portend-3.1.0-2.el9.noarch.r 217 kB/s | 16 kB 00:00 2024-04-21T08:26:21.691 INFO:teuthology.orchestra.run.smithi179.stdout:(136/150): python3-pyOpenSSL-21.0.0-1.el9.noarc 1.2 MB/s | 90 kB 00:00 2024-04-21T08:26:21.716 INFO:teuthology.orchestra.run.smithi179.stdout:(137/150): python3-pyperclip-1.8.2-2.el9.noarch 373 kB/s | 28 kB 00:00 2024-04-21T08:26:21.741 INFO:teuthology.orchestra.run.smithi179.stdout:(138/150): python3-repoze-lru-0.7-16.el9.noarch 408 kB/s | 31 kB 00:00 2024-04-21T08:26:21.767 INFO:teuthology.orchestra.run.smithi179.stdout:(139/150): python3-routes-2.5.1-5.el9.noarch.rp 2.4 MB/s | 188 kB 00:00 2024-04-21T08:26:21.792 INFO:teuthology.orchestra.run.smithi179.stdout:(140/150): python3-rsa-4.9-2.el9.noarch.rpm 784 kB/s | 59 kB 00:00 2024-04-21T08:26:21.817 INFO:teuthology.orchestra.run.smithi179.stdout:(141/150): python3-tempora-5.0.0-2.el9.noarch.r 476 kB/s | 36 kB 00:00 2024-04-21T08:26:21.843 INFO:teuthology.orchestra.run.smithi179.stdout:(142/150): python3-typing-extensions-4.4.0-2.el 673 kB/s | 51 kB 00:00 2024-04-21T08:26:21.868 INFO:teuthology.orchestra.run.smithi179.stdout:(143/150): python3-webob-1.8.7-6.el9.noarch.rpm 3.0 MB/s | 230 kB 00:00 2024-04-21T08:26:21.893 INFO:teuthology.orchestra.run.smithi179.stdout:(144/150): python3-websocket-client-1.2.3-2.el9 1.2 MB/s | 90 kB 00:00 2024-04-21T08:26:21.927 INFO:teuthology.orchestra.run.smithi179.stdout:(145/150): python3-werkzeug-2.0.3-3.el9.1.noarc 5.0 MB/s | 427 kB 00:00 2024-04-21T08:26:21.953 INFO:teuthology.orchestra.run.smithi179.stdout:(146/150): python3-zc-lockfile-2.0-10.el9.noarc 238 kB/s | 20 kB 00:00 2024-04-21T08:26:21.978 INFO:teuthology.orchestra.run.smithi179.stdout:(147/150): re2-20211101-3.el9.x86_64.rpm 2.3 MB/s | 192 kB 00:00 2024-04-21T08:26:22.028 INFO:teuthology.orchestra.run.smithi179.stdout:(148/150): thrift-0.15.0-2.el9.x86_64.rpm 16 MB/s | 1.6 MB 00:00 2024-04-21T08:26:22.295 INFO:teuthology.orchestra.run.smithi179.stdout:(149/150): librbd1-19.0.0-3154.ga630765c.el9.x8 10 MB/s | 3.1 MB 00:00 2024-04-21T08:26:22.351 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T08:26:22.379 INFO:teuthology.orchestra.run.smithi179.stdout:(150/150): librados2-19.0.0-3154.ga630765c.el9. 8.0 MB/s | 3.4 MB 00:00 2024-04-21T08:26:22.382 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T08:26:22.384 INFO:teuthology.orchestra.run.smithi179.stdout:-------------------------------------------------------------------------------- 2024-04-21T08:26:22.384 INFO:teuthology.orchestra.run.smithi179.stdout:Total 18 MB/s | 218 MB 00:11 2024-04-21T08:26:22.927 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T08:26:22.960 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 104/152 2024-04-21T08:26:23.034 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 104/152 2024-04-21T08:26:23.034 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-21T08:26:23.034 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-21T08:26:23.037 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-21T08:26:23.062 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 105/152 2024-04-21T08:26:23.413 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction check 2024-04-21T08:26:23.539 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction check succeeded. 2024-04-21T08:26:23.539 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction test 2024-04-21T08:26:25.253 INFO:teuthology.orchestra.run.smithi179.stdout:Transaction test succeeded. 2024-04-21T08:26:25.254 INFO:teuthology.orchestra.run.smithi179.stdout:Running transaction 2024-04-21T08:26:29.838 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 105/152 2024-04-21T08:26:29.838 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /sys 2024-04-21T08:26:29.838 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /proc 2024-04-21T08:26:29.838 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /mnt 2024-04-21T08:26:29.838 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /var/tmp 2024-04-21T08:26:29.838 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /home 2024-04-21T08:26:29.838 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /root 2024-04-21T08:26:29.838 INFO:teuthology.orchestra.run.smithi086.stdout:skipping the directory /tmp 2024-04-21T08:26:29.838 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:26:30.007 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 106/152 2024-04-21T08:26:30.024 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 106/152 2024-04-21T08:26:30.024 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:26:30.024 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-21T08:26:30.024 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:26:31.093 INFO:teuthology.orchestra.run.smithi179.stdout: Preparing : 1/1 2024-04-21T08:26:31.173 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : thrift-0.15.0-2.el9.x86_64 1/152 2024-04-21T08:26:31.198 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 2/152 2024-04-21T08:26:31.241 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 3/152 2024-04-21T08:26:31.529 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 107/152 2024-04-21T08:26:31.554 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 107/152 2024-04-21T08:26:31.554 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:26:31.554 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-21T08:26:31.554 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-21T08:26:31.554 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-21T08:26:31.555 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:26:31.575 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/152 2024-04-21T08:26:31.691 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : librados2-2:19.0.0-3154.ga630765c.el9.x86_64 5/152 2024-04-21T08:26:31.771 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 108/152 2024-04-21T08:26:31.794 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 108/152 2024-04-21T08:26:31.794 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:26:31.794 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-21T08:26:31.794 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-21T08:26:31.794 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-21T08:26:31.794 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:26:31.813 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librados2-2:19.0.0-3154.ga630765c.el9.x86_64 5/152 2024-04-21T08:26:31.843 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T08:26:31.918 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T08:26:31.960 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 7/152 2024-04-21T08:26:31.989 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/152 2024-04-21T08:26:32.015 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/152 2024-04-21T08:26:32.170 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : postgresql-private-libs-13.14-1.el9.x86_64 10/152 2024-04-21T08:26:32.221 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : postgresql-13.14-1.el9.x86_64 11/152 2024-04-21T08:26:32.230 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 109/152 2024-04-21T08:26:32.240 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T08:26:32.253 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 109/152 2024-04-21T08:26:32.253 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:26:32.253 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-21T08:26:32.253 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-21T08:26:32.253 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-21T08:26:32.253 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:26:32.345 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T08:26:32.360 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-21T08:26:32.367 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 13/152 2024-04-21T08:26:32.416 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-21T08:26:32.421 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 13/152 2024-04-21T08:26:32.443 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-21T08:26:32.463 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: qatlib-service-24.02.0-1.el9.x86_64 14/152 2024-04-21T08:26:32.473 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-21T08:26:32.657 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-21T08:26:32.660 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-21T08:26:32.701 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : qatlib-24.02.0-1.el9.x86_64 15/152 2024-04-21T08:26:32.705 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-21T08:26:32.783 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-21T08:26:32.808 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : re2-1:20211101-3.el9.x86_64 16/152 2024-04-21T08:26:32.871 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-21T08:26:32.907 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libarrow-9.0.0-10.el9.x86_64 17/152 2024-04-21T08:26:32.942 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-werkzeug-2.0.3-3.el9.1.noarch 18/152 2024-04-21T08:26:32.977 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : liboath-2.6.7-2.el9.x86_64 19/152 2024-04-21T08:26:33.010 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-21T08:26:33.055 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-21T08:26:33.060 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pytz-2021.1-5.el9.noarch 20/152 2024-04-21T08:26:33.096 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-21T08:26:33.102 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 21/152 2024-04-21T08:26:33.123 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-21T08:26:33.146 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 22/152 2024-04-21T08:26:33.178 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : protobuf-3.14.0-13.el9.x86_64 23/152 2024-04-21T08:26:33.207 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : lua-5.4.4-4.el9.x86_64 24/152 2024-04-21T08:26:33.278 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 25/152 2024-04-21T08:26:33.330 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-urllib3-1.26.5-5.el9.noarch 26/152 2024-04-21T08:26:33.360 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-requests-2.25.1-8.el9.noarch 27/152 2024-04-21T08:26:33.404 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libquadmath-11.4.1-3.el9.x86_64 28/152 2024-04-21T08:26:33.489 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libgfortran-11.4.1-3.el9.x86_64 29/152 2024-04-21T08:26:33.523 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-ceph-common-2:19.0.0-3154.ga630765c.el9. 30/152 2024-04-21T08:26:33.558 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: cephadm-2:19.0.0-3154.ga630765c.el9.noarch 121/152 2024-04-21T08:26:33.566 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-ceph-argparse-2:19.0.0-3154.ga630765c.el 31/152 2024-04-21T08:26:33.588 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephadm-2:19.0.0-3154.ga630765c.el9.noarch 121/152 2024-04-21T08:26:33.595 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_6 32/152 2024-04-21T08:26:33.621 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.e 122/152 2024-04-21T08:26:33.681 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3154.ga630765c. 123/152 2024-04-21T08:26:33.682 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 33/152 2024-04-21T08:26:33.703 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 124/152 2024-04-21T08:26:33.727 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : luarocks-3.9.2-1.el9.noarch 34/152 2024-04-21T08:26:33.803 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : protobuf-compiler-3.14.0-13.el9.x86_64 35/152 2024-04-21T08:26:33.940 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-mako-1.1.4-6.el9.noarch 36/152 2024-04-21T08:26:33.997 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 37/152 2024-04-21T08:26:34.481 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 105/152 2024-04-21T08:26:34.481 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /sys 2024-04-21T08:26:34.481 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /proc 2024-04-21T08:26:34.481 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /mnt 2024-04-21T08:26:34.481 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /var/tmp 2024-04-21T08:26:34.481 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /home 2024-04-21T08:26:34.481 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /root 2024-04-21T08:26:34.481 INFO:teuthology.orchestra.run.smithi088.stdout:skipping the directory /tmp 2024-04-21T08:26:34.481 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-21T08:26:34.546 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rsa-4.9-2.el9.noarch 38/152 2024-04-21T08:26:34.655 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 106/152 2024-04-21T08:26:34.671 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 106/152 2024-04-21T08:26:34.671 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:26:34.671 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-21T08:26:34.671 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-21T08:26:34.685 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 39/152 2024-04-21T08:26:34.713 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jinja2-2.11.3-5.el9.noarch 40/152 2024-04-21T08:26:34.733 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 124/152 2024-04-21T08:26:34.764 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : qatzip-libs-1.2.0-1.el9.x86_64 41/152 2024-04-21T08:26:34.780 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 125/152 2024-04-21T08:26:34.802 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librados-devel-2:19.0.0-3154.ga630765c.el9.x86_6 42/152 2024-04-21T08:26:34.835 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 43/152 2024-04-21T08:26:34.891 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 44/152 2024-04-21T08:26:34.965 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 45/152 2024-04-21T08:26:35.012 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-webob-1.8.7-6.el9.noarch 46/152 2024-04-21T08:26:35.048 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-typing-extensions-4.4.0-2.el9.noarch 47/152 2024-04-21T08:26:35.096 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 48/152 2024-04-21T08:26:35.134 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 49/152 2024-04-21T08:26:35.175 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyperclip-1.8.2-2.el9.noarch 50/152 2024-04-21T08:26:35.217 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 51/152 2024-04-21T08:26:35.356 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-logutils-0.3.5-21.el9.noarch 52/152 2024-04-21T08:26:35.397 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pecan-1.4.2-3.el9.noarch 53/152 2024-04-21T08:26:35.428 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-colorama-0.4.4-12.el9.noarch 54/152 2024-04-21T08:26:35.478 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-certifi-2021.10.8-2.el9.noarch 55/152 2024-04-21T08:26:35.582 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 56/152 2024-04-21T08:26:35.619 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-google-auth-1:2.28.2-1.el9.noarch 57/152 2024-04-21T08:26:35.659 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 58/152 2024-04-21T08:26:35.685 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : gperftools-libs-2.9.1-2.el9.x86_64 59/152 2024-04-21T08:26:35.710 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libarrow-doc-9.0.0-10.el9.noarch 60/152 2024-04-21T08:26:35.739 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 125/152 2024-04-21T08:26:35.763 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 126/152 2024-04-21T08:26:35.786 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 61/152 2024-04-21T08:26:35.865 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 126/152 2024-04-21T08:26:35.915 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : abseil-cpp-20211102.0-3.el9.x86_64 62/152 2024-04-21T08:26:35.952 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 63/152 2024-04-21T08:26:35.980 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : zlib-devel-1.2.11-41.el9.x86_64 64/152 2024-04-21T08:26:36.014 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 65/152 2024-04-21T08:26:36.026 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el 127/152 2024-04-21T08:26:36.057 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 128/152 2024-04-21T08:26:36.059 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : uuid-1.6.2-55.el9.x86_64 66/152 2024-04-21T08:26:36.080 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 128/152 2024-04-21T08:26:36.080 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:26:36.080 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-21T08:26:36.080 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-21T08:26:36.080 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-21T08:26:36.080 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:26:36.096 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : socat-1.7.4.1-5.el9.x86_64 67/152 2024-04-21T08:26:36.133 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 129/152 2024-04-21T08:26:36.136 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-wcwidth-0.2.5-8.el9.noarch 68/152 2024-04-21T08:26:36.144 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 129/152 2024-04-21T08:26:36.166 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 69/152 2024-04-21T08:26:36.166 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 107/152 2024-04-21T08:26:36.192 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 107/152 2024-04-21T08:26:36.192 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:26:36.192 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-21T08:26:36.192 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-21T08:26:36.192 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-21T08:26:36.192 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-21T08:26:36.200 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 70/152 2024-04-21T08:26:36.234 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-text-3.2.0-6.el9.noarch 71/152 2024-04-21T08:26:36.242 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-2:19.0.0-3154.ga630765c.el9.x86_64 130/152 2024-04-21T08:26:36.275 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 72/152 2024-04-21T08:26:36.289 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 131/152 2024-04-21T08:26:36.318 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 73/152 2024-04-21T08:26:36.407 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 74/152 2024-04-21T08:26:36.417 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 108/152 2024-04-21T08:26:36.440 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 108/152 2024-04-21T08:26:36.440 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:26:36.440 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-21T08:26:36.440 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-21T08:26:36.440 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-21T08:26:36.440 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-21T08:26:36.458 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-protobuf-3.14.0-13.el9.noarch 75/152 2024-04-21T08:26:36.489 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 76/152 2024-04-21T08:26:36.536 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-prettytable-0.7.2-27.el9.noarch 77/152 2024-04-21T08:26:36.644 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-packaging-20.9-5.el9.noarch 78/152 2024-04-21T08:26:36.699 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-oauthlib-3.1.1-5.el9.noarch 79/152 2024-04-21T08:26:36.893 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 109/152 2024-04-21T08:26:36.916 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 109/152 2024-04-21T08:26:36.916 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:26:36.916 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-21T08:26:36.916 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-21T08:26:36.916 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-21T08:26:36.916 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-21T08:26:37.022 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-21T08:26:37.070 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-21T08:26:37.135 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-21T08:26:37.215 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-21T08:26:37.260 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 133/152 2024-04-21T08:26:37.283 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 133/152 2024-04-21T08:26:37.284 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:26:37.284 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-21T08:26:37.284 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-21T08:26:37.284 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-21T08:26:37.284 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:26:37.320 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-21T08:26:37.328 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3154.ga6307 134/152 2024-04-21T08:26:37.349 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3154.ga6307 134/152 2024-04-21T08:26:37.349 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:26:37.350 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-21T08:26:37.350 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:26:37.376 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-21T08:26:37.395 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 135/152 2024-04-21T08:26:37.417 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 135/152 2024-04-21T08:26:37.417 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:26:37.417 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-21T08:26:37.417 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-21T08:26:37.417 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-21T08:26:37.418 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:26:37.443 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 80/152 2024-04-21T08:26:37.454 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-21T08:26:37.488 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-kubernetes-1:26.1.0-2.el9.noarch 81/152 2024-04-21T08:26:37.522 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jsonpointer-2.0-4.el9.noarch 82/152 2024-04-21T08:26:37.542 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-21T08:26:37.625 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-jsonpatch-1.21-16.el9.noarch 83/152 2024-04-21T08:26:37.674 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-21T08:26:37.681 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-devel-3.9.18-3.el9.x86_64 84/152 2024-04-21T08:26:37.718 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-21T08:26:37.720 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 136/152 2024-04-21T08:26:37.743 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 136/152 2024-04-21T08:26:37.744 INFO:teuthology.orchestra.run.smithi086.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:26:37.744 INFO:teuthology.orchestra.run.smithi086.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-21T08:26:37.744 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-21T08:26:37.744 INFO:teuthology.orchestra.run.smithi086.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-21T08:26:37.744 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:26:37.749 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-21T08:26:37.771 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-attrs-20.3.0-7.el9.noarch 85/152 2024-04-21T08:26:37.777 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-21T08:26:37.808 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cmd2-2.4.2-2.el9.noarch 86/152 2024-04-21T08:26:37.986 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : openblas-0.3.21-2.el9.x86_64 87/152 2024-04-21T08:26:38.022 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : openblas-openmp-0.3.21-2.el9.x86_64 88/152 2024-04-21T08:26:38.100 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 89/152 2024-04-21T08:26:38.197 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: cephadm-2:19.0.0-3154.ga630765c.el9.noarch 121/152 2024-04-21T08:26:38.226 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : cephadm-2:19.0.0-3154.ga630765c.el9.noarch 121/152 2024-04-21T08:26:38.250 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.e 122/152 2024-04-21T08:26:38.311 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3154.ga630765c. 123/152 2024-04-21T08:26:38.341 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 124/152 2024-04-21T08:26:38.784 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 90/152 2024-04-21T08:26:38.899 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-numpy-1:1.20.1-5.el9.x86_64 91/152 2024-04-21T08:26:39.374 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 124/152 2024-04-21T08:26:39.409 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 125/152 2024-04-21T08:26:40.169 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 92/152 2024-04-21T08:26:40.255 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-scipy-1.6.2-8.el9.x86_64 93/152 2024-04-21T08:26:40.283 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libzstd-devel-1.5.1-2.el9.x86_64 94/152 2024-04-21T08:26:40.317 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libuuid-devel-2.37.4-18.el9.x86_64 95/152 2024-04-21T08:26:40.375 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 125/152 2024-04-21T08:26:40.401 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 126/152 2024-04-21T08:26:40.502 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 126/152 2024-04-21T08:26:40.632 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 96/152 2024-04-21T08:26:40.664 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el 127/152 2024-04-21T08:26:40.680 INFO:teuthology.orchestra.run.smithi179.stdout: Upgrading : librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 97/152 2024-04-21T08:26:40.703 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 128/152 2024-04-21T08:26:40.726 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 128/152 2024-04-21T08:26:40.727 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:26:40.727 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-21T08:26:40.727 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-21T08:26:40.727 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-21T08:26:40.727 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-21T08:26:40.770 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 97/152 2024-04-21T08:26:40.770 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 129/152 2024-04-21T08:26:40.782 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 129/152 2024-04-21T08:26:40.801 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 98/152 2024-04-21T08:26:40.837 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : boost-program-options-1.75.0-8.el9.x86_64 99/152 2024-04-21T08:26:40.882 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-2:19.0.0-3154.ga630765c.el9.x86_64 130/152 2024-04-21T08:26:40.927 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 131/152 2024-04-21T08:26:41.318 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : parquet-libs-9.0.0-10.el9.x86_64 100/152 2024-04-21T08:26:41.349 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 101/152 2024-04-21T08:26:41.421 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 101/152 2024-04-21T08:26:41.451 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 102/152 2024-04-21T08:26:41.858 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-21T08:26:41.889 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 133/152 2024-04-21T08:26:41.913 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 133/152 2024-04-21T08:26:41.913 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:26:41.913 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-21T08:26:41.913 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-21T08:26:41.913 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-21T08:26:41.913 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-21T08:26:41.957 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3154.ga6307 134/152 2024-04-21T08:26:41.978 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3154.ga6307 134/152 2024-04-21T08:26:41.979 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:26:41.979 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-21T08:26:41.979 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-21T08:26:42.024 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 135/152 2024-04-21T08:26:42.046 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 135/152 2024-04-21T08:26:42.046 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:26:42.046 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-21T08:26:42.046 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-21T08:26:42.046 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-21T08:26:42.046 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-21T08:26:42.325 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 137/152 2024-04-21T08:26:42.349 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 136/152 2024-04-21T08:26:42.365 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 138/152 2024-04-21T08:26:42.372 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 136/152 2024-04-21T08:26:42.372 INFO:teuthology.orchestra.run.smithi088.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:26:42.372 INFO:teuthology.orchestra.run.smithi088.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-21T08:26:42.372 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-21T08:26:42.373 INFO:teuthology.orchestra.run.smithi088.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-21T08:26:42.373 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-21T08:26:42.406 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 139/152 2024-04-21T08:26:42.438 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-21T08:26:42.474 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-21T08:26:42.644 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 142/152 2024-04-21T08:26:42.695 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-21T08:26:42.725 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_ 144/152 2024-04-21T08:26:43.325 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T08:26:43.364 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T08:26:43.383 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T08:26:43.746 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T08:26:43.782 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T08:26:44.158 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 146/152 2024-04-21T08:26:44.235 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-21T08:26:44.300 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-21T08:26:44.334 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 103/152 2024-04-21T08:26:44.370 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 104/152 2024-04-21T08:26:44.380 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-21T08:26:44.422 INFO:teuthology.orchestra.run.smithi086.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-21T08:26:44.422 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-21T08:26:44.461 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 104/152 2024-04-21T08:26:44.462 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2024-04-21T08:26:44.462 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2024-04-21T08:26:44.462 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-21T08:26:44.496 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 105/152 2024-04-21T08:26:44.514 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-21T08:26:44.514 INFO:teuthology.orchestra.run.smithi086.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T08:26:46.986 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 137/152 2024-04-21T08:26:47.030 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 138/152 2024-04-21T08:26:47.077 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 139/152 2024-04-21T08:26:47.108 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-21T08:26:47.136 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-21T08:26:47.326 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 142/152 2024-04-21T08:26:47.373 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-21T08:26:47.404 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_ 144/152 2024-04-21T08:26:47.973 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T08:26:48.017 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T08:26:48.037 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T08:26:48.807 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 146/152 2024-04-21T08:26:48.897 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-21T08:26:48.937 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-21T08:26:49.017 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-21T08:26:49.050 INFO:teuthology.orchestra.run.smithi088.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-21T08:26:49.050 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-21T08:26:49.142 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-21T08:26:49.142 INFO:teuthology.orchestra.run.smithi088.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T08:26:50.408 INFO:teuthology.orchestra.run.smithi086.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T08:26:50.408 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-2:19.0.0-3154.ga630765c.el9.x86_64 1/152 2024-04-21T08:26:50.408 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 2/152 2024-04-21T08:26:50.408 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 3/152 2024-04-21T08:26:50.408 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 4/152 2024-04-21T08:26:50.408 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3154.ga6307 5/152 2024-04-21T08:26:50.408 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T08:26:50.409 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 7/152 2024-04-21T08:26:50.409 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 8/152 2024-04-21T08:26:50.409 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 9/152 2024-04-21T08:26:50.409 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 10/152 2024-04-21T08:26:50.409 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 11/152 2024-04-21T08:26:50.409 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T08:26:50.409 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 13/152 2024-04-21T08:26:50.409 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 14/152 2024-04-21T08:26:50.409 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_ 15/152 2024-04-21T08:26:50.409 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 16/152 2024-04-21T08:26:50.409 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 17/152 2024-04-21T08:26:50.410 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados-devel-2:19.0.0-3154.ga630765c.el9.x86_6 18/152 2024-04-21T08:26:50.410 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 19/152 2024-04-21T08:26:50.410 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 20/152 2024-04-21T08:26:50.410 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3154.ga630765c.el 21/152 2024-04-21T08:26:50.410 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ceph-common-2:19.0.0-3154.ga630765c.el9. 22/152 2024-04-21T08:26:50.410 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_6 23/152 2024-04-21T08:26:50.410 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 24/152 2024-04-21T08:26:50.410 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 25/152 2024-04-21T08:26:50.410 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 26/152 2024-04-21T08:26:50.421 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 27/152 2024-04-21T08:26:50.421 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 28/152 2024-04-21T08:26:50.422 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 29/152 2024-04-21T08:26:50.422 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3154.ga630765c. 30/152 2024-04-21T08:26:50.422 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 31/152 2024-04-21T08:26:50.422 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 32/152 2024-04-21T08:26:50.422 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 33/152 2024-04-21T08:26:50.422 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el 34/152 2024-04-21T08:26:50.422 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 35/152 2024-04-21T08:26:50.422 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.e 36/152 2024-04-21T08:26:50.422 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 37/152 2024-04-21T08:26:50.422 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephadm-2:19.0.0-3154.ga630765c.el9.noarch 38/152 2024-04-21T08:26:50.422 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 39/152 2024-04-21T08:26:50.422 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-21T08:26:50.422 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-21T08:26:50.423 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-21T08:26:50.423 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-21T08:26:50.423 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-21T08:26:50.423 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-21T08:26:50.423 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-21T08:26:50.423 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-21T08:26:50.423 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-21T08:26:50.423 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-21T08:26:50.423 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-21T08:26:50.423 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-21T08:26:50.423 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-21T08:26:50.423 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-21T08:26:50.423 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-21T08:26:50.423 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-21T08:26:50.424 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-21T08:26:50.424 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-21T08:26:50.424 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-21T08:26:50.424 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-21T08:26:50.424 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-21T08:26:50.424 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-21T08:26:50.424 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-21T08:26:50.424 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-21T08:26:50.424 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-21T08:26:50.424 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-21T08:26:50.424 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-21T08:26:50.424 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-21T08:26:50.424 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-21T08:26:50.424 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-21T08:26:50.424 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-21T08:26:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-21T08:26:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-21T08:26:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-21T08:26:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-21T08:26:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-21T08:26:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-21T08:26:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-21T08:26:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-21T08:26:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-21T08:26:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-21T08:26:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-21T08:26:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-21T08:26:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-21T08:26:50.425 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-21T08:26:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-21T08:26:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-21T08:26:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-21T08:26:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-21T08:26:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-21T08:26:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-21T08:26:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-21T08:26:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-21T08:26:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-21T08:26:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-21T08:26:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-21T08:26:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-21T08:26:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-21T08:26:50.426 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-21T08:26:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-21T08:26:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-21T08:26:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-21T08:26:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-21T08:26:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-21T08:26:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-21T08:26:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-21T08:26:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-21T08:26:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-21T08:26:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-21T08:26:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-21T08:26:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-21T08:26:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-21T08:26:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-21T08:26:50.427 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-21T08:26:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-21T08:26:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-21T08:26:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-21T08:26:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-21T08:26:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-21T08:26:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-21T08:26:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-21T08:26:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-21T08:26:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-21T08:26:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-21T08:26:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-21T08:26:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-21T08:26:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-21T08:26:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-21T08:26:50.428 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-21T08:26:50.429 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-21T08:26:50.429 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-21T08:26:50.429 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-21T08:26:50.429 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-21T08:26:50.429 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-21T08:26:50.429 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-21T08:26:50.429 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-21T08:26:50.429 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-21T08:26:50.429 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-21T08:26:50.429 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-21T08:26:50.429 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-21T08:26:50.429 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-21T08:26:50.429 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-21T08:26:50.429 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-21T08:26:50.430 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-21T08:26:50.430 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-21T08:26:50.430 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-21T08:26:50.430 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-21T08:26:50.430 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-21T08:26:50.430 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-21T08:26:50.430 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados2-2:19.0.0-3154.ga630765c.el9.x86_64 149/152 2024-04-21T08:26:50.430 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-21T08:26:50.430 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 151/152 2024-04-21T08:26:52.398 INFO:teuthology.orchestra.run.smithi086.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T08:26:52.399 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:26:52.399 INFO:teuthology.orchestra.run.smithi086.stdout:Upgraded: 2024-04-21T08:26:52.399 INFO:teuthology.orchestra.run.smithi086.stdout: librados2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:52.399 INFO:teuthology.orchestra.run.smithi086.stdout: librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:52.399 INFO:teuthology.orchestra.run.smithi086.stdout:Installed: 2024-04-21T08:26:52.399 INFO:teuthology.orchestra.run.smithi086.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-21T08:26:52.399 INFO:teuthology.orchestra.run.smithi086.stdout: automake-1.16.2-8.el9.noarch 2024-04-21T08:26:52.399 INFO:teuthology.orchestra.run.smithi086.stdout: bison-3.7.4-5.el9.x86_64 2024-04-21T08:26:52.399 INFO:teuthology.orchestra.run.smithi086.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-21T08:26:52.399 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:52.399 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:52.399 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:52.399 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:52.400 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-grafana-dashboards-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:52.400 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-immutable-object-cache-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:52.400 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:52.400 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:52.400 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:52.400 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:52.400 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:52.400 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:52.400 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:52.400 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:52.400 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:52.400 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:52.400 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:52.400 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:52.401 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:52.401 INFO:teuthology.orchestra.run.smithi086.stdout: ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:52.401 INFO:teuthology.orchestra.run.smithi086.stdout: cephadm-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:52.401 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:52.401 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:52.401 INFO:teuthology.orchestra.run.smithi086.stdout: cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:52.401 INFO:teuthology.orchestra.run.smithi086.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-21T08:26:52.401 INFO:teuthology.orchestra.run.smithi086.stdout: flex-2.6.4-9.el9.x86_64 2024-04-21T08:26:52.401 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-21T08:26:52.401 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-21T08:26:52.401 INFO:teuthology.orchestra.run.smithi086.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-21T08:26:52.401 INFO:teuthology.orchestra.run.smithi086.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-21T08:26:52.401 INFO:teuthology.orchestra.run.smithi086.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-21T08:26:52.401 INFO:teuthology.orchestra.run.smithi086.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-21T08:26:52.402 INFO:teuthology.orchestra.run.smithi086.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-21T08:26:52.402 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-21T08:26:52.402 INFO:teuthology.orchestra.run.smithi086.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-21T08:26:52.402 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:52.402 INFO:teuthology.orchestra.run.smithi086.stdout: libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:52.402 INFO:teuthology.orchestra.run.smithi086.stdout: libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:52.402 INFO:teuthology.orchestra.run.smithi086.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-21T08:26:52.402 INFO:teuthology.orchestra.run.smithi086.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-21T08:26:52.402 INFO:teuthology.orchestra.run.smithi086.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-21T08:26:52.402 INFO:teuthology.orchestra.run.smithi086.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-21T08:26:52.402 INFO:teuthology.orchestra.run.smithi086.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-21T08:26:52.402 INFO:teuthology.orchestra.run.smithi086.stdout: librados-devel-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:52.402 INFO:teuthology.orchestra.run.smithi086.stdout: libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:52.402 INFO:teuthology.orchestra.run.smithi086.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-21T08:26:52.403 INFO:teuthology.orchestra.run.smithi086.stdout: librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:52.403 INFO:teuthology.orchestra.run.smithi086.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-21T08:26:52.403 INFO:teuthology.orchestra.run.smithi086.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-21T08:26:52.403 INFO:teuthology.orchestra.run.smithi086.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-21T08:26:52.403 INFO:teuthology.orchestra.run.smithi086.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-21T08:26:52.403 INFO:teuthology.orchestra.run.smithi086.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-21T08:26:52.403 INFO:teuthology.orchestra.run.smithi086.stdout: lua-5.4.4-4.el9.x86_64 2024-04-21T08:26:52.403 INFO:teuthology.orchestra.run.smithi086.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-21T08:26:52.403 INFO:teuthology.orchestra.run.smithi086.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-21T08:26:52.403 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-21T08:26:52.403 INFO:teuthology.orchestra.run.smithi086.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-21T08:26:52.403 INFO:teuthology.orchestra.run.smithi086.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-21T08:26:52.403 INFO:teuthology.orchestra.run.smithi086.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-21T08:26:52.403 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-13.14-1.el9.x86_64 2024-04-21T08:26:52.403 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-21T08:26:52.404 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-21T08:26:52.404 INFO:teuthology.orchestra.run.smithi086.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-21T08:26:52.404 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-21T08:26:52.404 INFO:teuthology.orchestra.run.smithi086.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-21T08:26:52.404 INFO:teuthology.orchestra.run.smithi086.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-21T08:26:52.404 INFO:teuthology.orchestra.run.smithi086.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-21T08:26:52.404 INFO:teuthology.orchestra.run.smithi086.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-21T08:26:52.404 INFO:teuthology.orchestra.run.smithi086.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-21T08:26:52.404 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-21T08:26:52.404 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-argparse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:52.404 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:52.404 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:52.404 INFO:teuthology.orchestra.run.smithi086.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-21T08:26:52.404 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-21T08:26:52.405 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-21T08:26:52.405 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-21T08:26:52.405 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-21T08:26:52.405 INFO:teuthology.orchestra.run.smithi086.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-21T08:26:52.405 INFO:teuthology.orchestra.run.smithi086.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-21T08:26:52.405 INFO:teuthology.orchestra.run.smithi086.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-21T08:26:52.405 INFO:teuthology.orchestra.run.smithi086.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-21T08:26:52.405 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-21T08:26:52.405 INFO:teuthology.orchestra.run.smithi086.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-21T08:26:52.405 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-21T08:26:52.405 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-21T08:26:52.405 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-21T08:26:52.405 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-21T08:26:52.405 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-21T08:26:52.405 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-21T08:26:52.406 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-21T08:26:52.406 INFO:teuthology.orchestra.run.smithi086.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-21T08:26:52.406 INFO:teuthology.orchestra.run.smithi086.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-21T08:26:52.406 INFO:teuthology.orchestra.run.smithi086.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-21T08:26:52.406 INFO:teuthology.orchestra.run.smithi086.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-21T08:26:52.406 INFO:teuthology.orchestra.run.smithi086.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-21T08:26:52.406 INFO:teuthology.orchestra.run.smithi086.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-21T08:26:52.406 INFO:teuthology.orchestra.run.smithi086.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-21T08:26:52.406 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-21T08:26:52.407 INFO:teuthology.orchestra.run.smithi086.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-21T08:26:52.407 INFO:teuthology.orchestra.run.smithi086.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-21T08:26:52.407 INFO:teuthology.orchestra.run.smithi086.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-21T08:26:52.407 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-21T08:26:52.407 INFO:teuthology.orchestra.run.smithi086.stdout: python3-ply-3.11-14.el9.noarch 2024-04-21T08:26:52.407 INFO:teuthology.orchestra.run.smithi086.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-21T08:26:52.407 INFO:teuthology.orchestra.run.smithi086.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-21T08:26:52.407 INFO:teuthology.orchestra.run.smithi086.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-21T08:26:52.407 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-21T08:26:52.407 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-21T08:26:52.407 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-21T08:26:52.407 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-21T08:26:52.407 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-21T08:26:52.408 INFO:teuthology.orchestra.run.smithi086.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-21T08:26:52.408 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:52.408 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:52.408 INFO:teuthology.orchestra.run.smithi086.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-21T08:26:52.408 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-21T08:26:52.408 INFO:teuthology.orchestra.run.smithi086.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-21T08:26:52.408 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:52.408 INFO:teuthology.orchestra.run.smithi086.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-21T08:26:52.408 INFO:teuthology.orchestra.run.smithi086.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-21T08:26:52.408 INFO:teuthology.orchestra.run.smithi086.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-21T08:26:52.408 INFO:teuthology.orchestra.run.smithi086.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-21T08:26:52.408 INFO:teuthology.orchestra.run.smithi086.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-21T08:26:52.408 INFO:teuthology.orchestra.run.smithi086.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-21T08:26:52.408 INFO:teuthology.orchestra.run.smithi086.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-21T08:26:52.409 INFO:teuthology.orchestra.run.smithi086.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-21T08:26:52.409 INFO:teuthology.orchestra.run.smithi086.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-21T08:26:52.409 INFO:teuthology.orchestra.run.smithi086.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-21T08:26:52.409 INFO:teuthology.orchestra.run.smithi086.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-21T08:26:52.409 INFO:teuthology.orchestra.run.smithi086.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-21T08:26:52.409 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-21T08:26:52.409 INFO:teuthology.orchestra.run.smithi086.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-21T08:26:52.409 INFO:teuthology.orchestra.run.smithi086.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-21T08:26:52.409 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:52.409 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:52.409 INFO:teuthology.orchestra.run.smithi086.stdout: rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:52.409 INFO:teuthology.orchestra.run.smithi086.stdout: re2-1:20211101-3.el9.x86_64 2024-04-21T08:26:52.409 INFO:teuthology.orchestra.run.smithi086.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-21T08:26:52.409 INFO:teuthology.orchestra.run.smithi086.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-21T08:26:52.409 INFO:teuthology.orchestra.run.smithi086.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-21T08:26:52.410 INFO:teuthology.orchestra.run.smithi086.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-21T08:26:52.410 INFO:teuthology.orchestra.run.smithi086.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-21T08:26:52.410 INFO:teuthology.orchestra.run.smithi086.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-21T08:26:52.410 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:26:52.410 INFO:teuthology.orchestra.run.smithi086.stdout:Complete! 2024-04-21T08:26:52.774 DEBUG:teuthology.parallel:result is None 2024-04-21T08:26:55.125 INFO:teuthology.orchestra.run.smithi088.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T08:26:55.125 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-2:19.0.0-3154.ga630765c.el9.x86_64 1/152 2024-04-21T08:26:55.125 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 2/152 2024-04-21T08:26:55.125 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 3/152 2024-04-21T08:26:55.126 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 4/152 2024-04-21T08:26:55.126 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3154.ga6307 5/152 2024-04-21T08:26:55.126 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T08:26:55.126 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 7/152 2024-04-21T08:26:55.126 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 8/152 2024-04-21T08:26:55.126 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 9/152 2024-04-21T08:26:55.126 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 10/152 2024-04-21T08:26:55.126 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 11/152 2024-04-21T08:26:55.126 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T08:26:55.126 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 13/152 2024-04-21T08:26:55.126 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 14/152 2024-04-21T08:26:55.126 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_ 15/152 2024-04-21T08:26:55.126 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 16/152 2024-04-21T08:26:55.127 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 17/152 2024-04-21T08:26:55.127 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librados-devel-2:19.0.0-3154.ga630765c.el9.x86_6 18/152 2024-04-21T08:26:55.127 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 19/152 2024-04-21T08:26:55.127 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 20/152 2024-04-21T08:26:55.127 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3154.ga630765c.el 21/152 2024-04-21T08:26:55.127 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-ceph-common-2:19.0.0-3154.ga630765c.el9. 22/152 2024-04-21T08:26:55.127 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_6 23/152 2024-04-21T08:26:55.127 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 24/152 2024-04-21T08:26:55.127 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 25/152 2024-04-21T08:26:55.127 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 26/152 2024-04-21T08:26:55.127 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 27/152 2024-04-21T08:26:55.127 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 28/152 2024-04-21T08:26:55.127 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 29/152 2024-04-21T08:26:55.127 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3154.ga630765c. 30/152 2024-04-21T08:26:55.128 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 31/152 2024-04-21T08:26:55.128 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 32/152 2024-04-21T08:26:55.128 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 33/152 2024-04-21T08:26:55.128 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el 34/152 2024-04-21T08:26:55.128 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 35/152 2024-04-21T08:26:55.128 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.e 36/152 2024-04-21T08:26:55.128 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 37/152 2024-04-21T08:26:55.128 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : cephadm-2:19.0.0-3154.ga630765c.el9.noarch 38/152 2024-04-21T08:26:55.128 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 39/152 2024-04-21T08:26:55.128 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-21T08:26:55.128 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-21T08:26:55.128 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-21T08:26:55.128 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-21T08:26:55.128 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-21T08:26:55.129 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-21T08:26:55.129 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-21T08:26:55.129 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-21T08:26:55.129 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-21T08:26:55.129 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-21T08:26:55.129 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-21T08:26:55.129 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-21T08:26:55.129 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-21T08:26:55.129 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-21T08:26:55.129 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-21T08:26:55.129 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-21T08:26:55.129 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-21T08:26:55.129 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-21T08:26:55.129 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-21T08:26:55.130 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-21T08:26:55.130 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-21T08:26:55.130 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-21T08:26:55.130 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-21T08:26:55.130 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-21T08:26:55.130 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-21T08:26:55.130 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-21T08:26:55.130 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-21T08:26:55.130 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-21T08:26:55.130 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-21T08:26:55.130 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-21T08:26:55.130 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-21T08:26:55.130 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-21T08:26:55.131 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-21T08:26:55.131 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-21T08:26:55.131 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-21T08:26:55.131 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-21T08:26:55.131 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-21T08:26:55.131 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-21T08:26:55.131 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-21T08:26:55.131 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-21T08:26:55.131 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-21T08:26:55.131 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-21T08:26:55.131 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-21T08:26:55.131 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-21T08:26:55.131 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-21T08:26:55.131 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-21T08:26:55.132 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-21T08:26:55.132 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-21T08:26:55.132 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-21T08:26:55.132 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-21T08:26:55.132 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-21T08:26:55.132 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-21T08:26:55.132 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-21T08:26:55.132 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-21T08:26:55.132 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-21T08:26:55.132 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-21T08:26:55.132 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-21T08:26:55.132 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-21T08:26:55.132 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-21T08:26:55.132 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-21T08:26:55.133 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-21T08:26:55.133 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-21T08:26:55.133 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-21T08:26:55.133 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-21T08:26:55.133 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-21T08:26:55.133 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-21T08:26:55.133 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-21T08:26:55.133 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-21T08:26:55.133 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-21T08:26:55.133 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-21T08:26:55.133 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-21T08:26:55.133 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-21T08:26:55.133 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-21T08:26:55.134 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-21T08:26:55.134 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-21T08:26:55.134 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-21T08:26:55.134 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-21T08:26:55.134 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-21T08:26:55.134 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-21T08:26:55.134 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-21T08:26:55.134 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-21T08:26:55.134 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-21T08:26:55.134 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-21T08:26:55.134 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-21T08:26:55.134 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-21T08:26:55.134 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-21T08:26:55.134 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-21T08:26:55.135 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-21T08:26:55.135 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-21T08:26:55.135 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-21T08:26:55.135 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-21T08:26:55.135 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-21T08:26:55.135 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-21T08:26:55.135 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-21T08:26:55.135 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-21T08:26:55.135 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-21T08:26:55.135 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-21T08:26:55.135 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-21T08:26:55.135 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-21T08:26:55.135 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-21T08:26:55.135 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-21T08:26:55.136 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-21T08:26:55.136 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-21T08:26:55.136 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-21T08:26:55.136 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-21T08:26:55.136 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-21T08:26:55.136 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-21T08:26:55.136 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-21T08:26:55.136 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-21T08:26:55.136 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librados2-2:19.0.0-3154.ga630765c.el9.x86_64 149/152 2024-04-21T08:26:55.136 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-21T08:26:55.136 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 151/152 2024-04-21T08:26:55.780 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 105/152 2024-04-21T08:26:55.781 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /sys 2024-04-21T08:26:55.781 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /proc 2024-04-21T08:26:55.781 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /mnt 2024-04-21T08:26:55.781 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /var/tmp 2024-04-21T08:26:55.781 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /home 2024-04-21T08:26:55.781 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /root 2024-04-21T08:26:55.781 INFO:teuthology.orchestra.run.smithi179.stdout:skipping the directory /tmp 2024-04-21T08:26:55.781 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-21T08:26:55.965 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 106/152 2024-04-21T08:26:55.980 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 106/152 2024-04-21T08:26:55.980 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:26:55.980 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2024-04-21T08:26:55.980 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-21T08:26:57.205 INFO:teuthology.orchestra.run.smithi088.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T08:26:57.206 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-21T08:26:57.206 INFO:teuthology.orchestra.run.smithi088.stdout:Upgraded: 2024-04-21T08:26:57.206 INFO:teuthology.orchestra.run.smithi088.stdout: librados2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:57.206 INFO:teuthology.orchestra.run.smithi088.stdout: librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:57.206 INFO:teuthology.orchestra.run.smithi088.stdout:Installed: 2024-04-21T08:26:57.206 INFO:teuthology.orchestra.run.smithi088.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-21T08:26:57.206 INFO:teuthology.orchestra.run.smithi088.stdout: automake-1.16.2-8.el9.noarch 2024-04-21T08:26:57.206 INFO:teuthology.orchestra.run.smithi088.stdout: bison-3.7.4-5.el9.x86_64 2024-04-21T08:26:57.206 INFO:teuthology.orchestra.run.smithi088.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-21T08:26:57.206 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:57.206 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:57.207 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:57.207 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:57.207 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-grafana-dashboards-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:57.207 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-immutable-object-cache-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:57.207 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:57.207 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:57.207 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:57.207 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:57.207 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:57.207 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:57.207 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:57.208 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:57.208 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:57.208 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:57.208 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:57.208 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:57.208 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:57.208 INFO:teuthology.orchestra.run.smithi088.stdout: ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:57.208 INFO:teuthology.orchestra.run.smithi088.stdout: cephadm-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:57.208 INFO:teuthology.orchestra.run.smithi088.stdout: cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:57.209 INFO:teuthology.orchestra.run.smithi088.stdout: cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:57.209 INFO:teuthology.orchestra.run.smithi088.stdout: cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:26:57.209 INFO:teuthology.orchestra.run.smithi088.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-21T08:26:57.209 INFO:teuthology.orchestra.run.smithi088.stdout: flex-2.6.4-9.el9.x86_64 2024-04-21T08:26:57.209 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-21T08:26:57.209 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-21T08:26:57.209 INFO:teuthology.orchestra.run.smithi088.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-21T08:26:57.209 INFO:teuthology.orchestra.run.smithi088.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-21T08:26:57.209 INFO:teuthology.orchestra.run.smithi088.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-21T08:26:57.209 INFO:teuthology.orchestra.run.smithi088.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-21T08:26:57.209 INFO:teuthology.orchestra.run.smithi088.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-21T08:26:57.210 INFO:teuthology.orchestra.run.smithi088.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-21T08:26:57.210 INFO:teuthology.orchestra.run.smithi088.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-21T08:26:57.210 INFO:teuthology.orchestra.run.smithi088.stdout: libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:57.210 INFO:teuthology.orchestra.run.smithi088.stdout: libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:57.210 INFO:teuthology.orchestra.run.smithi088.stdout: libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:57.210 INFO:teuthology.orchestra.run.smithi088.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-21T08:26:57.210 INFO:teuthology.orchestra.run.smithi088.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-21T08:26:57.210 INFO:teuthology.orchestra.run.smithi088.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-21T08:26:57.210 INFO:teuthology.orchestra.run.smithi088.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-21T08:26:57.210 INFO:teuthology.orchestra.run.smithi088.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-21T08:26:57.210 INFO:teuthology.orchestra.run.smithi088.stdout: librados-devel-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:57.211 INFO:teuthology.orchestra.run.smithi088.stdout: libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:57.211 INFO:teuthology.orchestra.run.smithi088.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-21T08:26:57.211 INFO:teuthology.orchestra.run.smithi088.stdout: librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:57.211 INFO:teuthology.orchestra.run.smithi088.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-21T08:26:57.211 INFO:teuthology.orchestra.run.smithi088.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-21T08:26:57.211 INFO:teuthology.orchestra.run.smithi088.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-21T08:26:57.211 INFO:teuthology.orchestra.run.smithi088.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-21T08:26:57.211 INFO:teuthology.orchestra.run.smithi088.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-21T08:26:57.211 INFO:teuthology.orchestra.run.smithi088.stdout: lua-5.4.4-4.el9.x86_64 2024-04-21T08:26:57.211 INFO:teuthology.orchestra.run.smithi088.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-21T08:26:57.211 INFO:teuthology.orchestra.run.smithi088.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-21T08:26:57.212 INFO:teuthology.orchestra.run.smithi088.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-21T08:26:57.212 INFO:teuthology.orchestra.run.smithi088.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-21T08:26:57.212 INFO:teuthology.orchestra.run.smithi088.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-21T08:26:57.212 INFO:teuthology.orchestra.run.smithi088.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-21T08:26:57.212 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-13.14-1.el9.x86_64 2024-04-21T08:26:57.212 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-21T08:26:57.212 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-21T08:26:57.212 INFO:teuthology.orchestra.run.smithi088.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-21T08:26:57.212 INFO:teuthology.orchestra.run.smithi088.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-21T08:26:57.212 INFO:teuthology.orchestra.run.smithi088.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-21T08:26:57.212 INFO:teuthology.orchestra.run.smithi088.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-21T08:26:57.212 INFO:teuthology.orchestra.run.smithi088.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-21T08:26:57.212 INFO:teuthology.orchestra.run.smithi088.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-21T08:26:57.213 INFO:teuthology.orchestra.run.smithi088.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-21T08:26:57.213 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-21T08:26:57.213 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ceph-argparse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:57.213 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:57.213 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:57.213 INFO:teuthology.orchestra.run.smithi088.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-21T08:26:57.213 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-21T08:26:57.213 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-21T08:26:57.213 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-21T08:26:57.213 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-21T08:26:57.213 INFO:teuthology.orchestra.run.smithi088.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-21T08:26:57.213 INFO:teuthology.orchestra.run.smithi088.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-21T08:26:57.214 INFO:teuthology.orchestra.run.smithi088.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-21T08:26:57.214 INFO:teuthology.orchestra.run.smithi088.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-21T08:26:57.214 INFO:teuthology.orchestra.run.smithi088.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-21T08:26:57.214 INFO:teuthology.orchestra.run.smithi088.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-21T08:26:57.214 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-21T08:26:57.214 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-21T08:26:57.214 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-21T08:26:57.214 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-21T08:26:57.214 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-21T08:26:57.214 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-21T08:26:57.214 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-21T08:26:57.215 INFO:teuthology.orchestra.run.smithi088.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-21T08:26:57.215 INFO:teuthology.orchestra.run.smithi088.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-21T08:26:57.215 INFO:teuthology.orchestra.run.smithi088.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-21T08:26:57.215 INFO:teuthology.orchestra.run.smithi088.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-21T08:26:57.215 INFO:teuthology.orchestra.run.smithi088.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-21T08:26:57.215 INFO:teuthology.orchestra.run.smithi088.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-21T08:26:57.215 INFO:teuthology.orchestra.run.smithi088.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-21T08:26:57.215 INFO:teuthology.orchestra.run.smithi088.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-21T08:26:57.215 INFO:teuthology.orchestra.run.smithi088.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-21T08:26:57.215 INFO:teuthology.orchestra.run.smithi088.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-21T08:26:57.215 INFO:teuthology.orchestra.run.smithi088.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-21T08:26:57.215 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-21T08:26:57.216 INFO:teuthology.orchestra.run.smithi088.stdout: python3-ply-3.11-14.el9.noarch 2024-04-21T08:26:57.216 INFO:teuthology.orchestra.run.smithi088.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-21T08:26:57.216 INFO:teuthology.orchestra.run.smithi088.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-21T08:26:57.216 INFO:teuthology.orchestra.run.smithi088.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-21T08:26:57.216 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-21T08:26:57.216 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-21T08:26:57.216 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-21T08:26:57.216 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-21T08:26:57.216 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-21T08:26:57.216 INFO:teuthology.orchestra.run.smithi088.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-21T08:26:57.216 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:57.217 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:57.217 INFO:teuthology.orchestra.run.smithi088.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-21T08:26:57.217 INFO:teuthology.orchestra.run.smithi088.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-21T08:26:57.217 INFO:teuthology.orchestra.run.smithi088.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-21T08:26:57.217 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:57.217 INFO:teuthology.orchestra.run.smithi088.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-21T08:26:57.217 INFO:teuthology.orchestra.run.smithi088.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-21T08:26:57.217 INFO:teuthology.orchestra.run.smithi088.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-21T08:26:57.217 INFO:teuthology.orchestra.run.smithi088.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-21T08:26:57.217 INFO:teuthology.orchestra.run.smithi088.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-21T08:26:57.217 INFO:teuthology.orchestra.run.smithi088.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-21T08:26:57.217 INFO:teuthology.orchestra.run.smithi088.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-21T08:26:57.217 INFO:teuthology.orchestra.run.smithi088.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-21T08:26:57.218 INFO:teuthology.orchestra.run.smithi088.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-21T08:26:57.218 INFO:teuthology.orchestra.run.smithi088.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-21T08:26:57.218 INFO:teuthology.orchestra.run.smithi088.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-21T08:26:57.218 INFO:teuthology.orchestra.run.smithi088.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-21T08:26:57.218 INFO:teuthology.orchestra.run.smithi088.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-21T08:26:57.218 INFO:teuthology.orchestra.run.smithi088.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-21T08:26:57.218 INFO:teuthology.orchestra.run.smithi088.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-21T08:26:57.218 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:57.218 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:57.218 INFO:teuthology.orchestra.run.smithi088.stdout: rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:26:57.218 INFO:teuthology.orchestra.run.smithi088.stdout: re2-1:20211101-3.el9.x86_64 2024-04-21T08:26:57.218 INFO:teuthology.orchestra.run.smithi088.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-21T08:26:57.219 INFO:teuthology.orchestra.run.smithi088.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-21T08:26:57.219 INFO:teuthology.orchestra.run.smithi088.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-21T08:26:57.219 INFO:teuthology.orchestra.run.smithi088.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-21T08:26:57.219 INFO:teuthology.orchestra.run.smithi088.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-21T08:26:57.219 INFO:teuthology.orchestra.run.smithi088.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-21T08:26:57.220 INFO:teuthology.orchestra.run.smithi088.stdout: 2024-04-21T08:26:57.220 INFO:teuthology.orchestra.run.smithi088.stdout:Complete! 2024-04-21T08:26:57.452 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 107/152 2024-04-21T08:26:57.478 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 107/152 2024-04-21T08:26:57.478 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:26:57.478 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2024-04-21T08:26:57.478 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-21T08:26:57.478 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2024-04-21T08:26:57.478 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-21T08:26:57.608 DEBUG:teuthology.parallel:result is None 2024-04-21T08:26:57.703 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 108/152 2024-04-21T08:26:57.725 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 108/152 2024-04-21T08:26:57.726 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:26:57.726 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2024-04-21T08:26:57.726 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-21T08:26:57.726 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2024-04-21T08:26:57.726 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-21T08:26:58.170 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 109/152 2024-04-21T08:26:58.193 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 109/152 2024-04-21T08:26:58.193 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:26:58.193 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2024-04-21T08:26:58.193 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-21T08:26:58.194 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2024-04-21T08:26:58.194 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-21T08:26:58.301 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : automake-1.16.2-8.el9.noarch 110/152 2024-04-21T08:26:58.366 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-ply-3.11-14.el9.noarch 111/152 2024-04-21T08:26:58.430 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 112/152 2024-04-21T08:26:58.617 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 113/152 2024-04-21T08:26:58.673 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cryptography-36.0.1-4.el9.x86_64 114/152 2024-04-21T08:26:58.752 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 115/152 2024-04-21T08:26:58.848 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cheroot-8.6.0-4.el9.noarch 116/152 2024-04-21T08:26:58.989 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 117/152 2024-04-21T08:26:59.040 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 118/152 2024-04-21T08:26:59.072 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 119/152 2024-04-21T08:26:59.108 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : fuse-2.9.9-15.el9.x86_64 120/152 2024-04-21T08:26:59.561 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: cephadm-2:19.0.0-3154.ga630765c.el9.noarch 121/152 2024-04-21T08:26:59.591 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : cephadm-2:19.0.0-3154.ga630765c.el9.noarch 121/152 2024-04-21T08:26:59.624 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.e 122/152 2024-04-21T08:26:59.685 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-grafana-dashboards-2:19.0.0-3154.ga630765c. 123/152 2024-04-21T08:26:59.714 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 124/152 2024-04-21T08:27:00.752 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 124/152 2024-04-21T08:27:00.800 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 125/152 2024-04-21T08:27:01.768 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 125/152 2024-04-21T08:27:01.801 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 126/152 2024-04-21T08:27:01.905 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 126/152 2024-04-21T08:27:02.073 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el 127/152 2024-04-21T08:27:02.103 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 128/152 2024-04-21T08:27:02.126 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 128/152 2024-04-21T08:27:02.126 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:27:02.126 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2024-04-21T08:27:02.126 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-21T08:27:02.126 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2024-04-21T08:27:02.127 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-21T08:27:02.171 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 129/152 2024-04-21T08:27:02.182 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 129/152 2024-04-21T08:27:02.281 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-2:19.0.0-3154.ga630765c.el9.x86_64 130/152 2024-04-21T08:27:02.328 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 131/152 2024-04-21T08:27:03.259 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libtool-2.4.6-45.el9.x86_64 132/152 2024-04-21T08:27:03.307 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 133/152 2024-04-21T08:27:03.330 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 133/152 2024-04-21T08:27:03.330 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:27:03.330 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2024-04-21T08:27:03.330 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-21T08:27:03.330 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2024-04-21T08:27:03.331 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-21T08:27:03.374 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-immutable-object-cache-2:19.0.0-3154.ga6307 134/152 2024-04-21T08:27:03.396 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: ceph-immutable-object-cache-2:19.0.0-3154.ga6307 134/152 2024-04-21T08:27:03.397 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:27:03.397 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2024-04-21T08:27:03.397 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-21T08:27:03.450 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 135/152 2024-04-21T08:27:03.473 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 135/152 2024-04-21T08:27:03.473 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:27:03.473 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "cephfs-mirror@*.service" escaped as "cephfs-mirror@\x2a.service". 2024-04-21T08:27:03.473 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-21T08:27:03.474 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/cephfs-mirror.target → /usr/lib/systemd/system/cephfs-mirror.target. 2024-04-21T08:27:03.474 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-21T08:27:03.777 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 136/152 2024-04-21T08:27:03.799 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 136/152 2024-04-21T08:27:03.799 INFO:teuthology.orchestra.run.smithi179.stdout:Glob pattern passed to enable, but globs are not supported for this. 2024-04-21T08:27:03.799 INFO:teuthology.orchestra.run.smithi179.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2024-04-21T08:27:03.799 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-21T08:27:03.799 INFO:teuthology.orchestra.run.smithi179.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2024-04-21T08:27:03.800 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-21T08:27:08.424 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 137/152 2024-04-21T08:27:08.465 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 138/152 2024-04-21T08:27:08.506 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 139/152 2024-04-21T08:27:08.538 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : xfsprogs-devel-6.3.0-1.el9.x86_64 140/152 2024-04-21T08:27:08.566 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : elfutils-libelf-devel-0.190-2.el9.x86_64 141/152 2024-04-21T08:27:08.775 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 142/152 2024-04-21T08:27:08.821 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : postgresql-contrib-13.14-1.el9.x86_64 143/152 2024-04-21T08:27:08.851 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_ 144/152 2024-04-21T08:27:09.442 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T08:27:09.484 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T08:27:09.503 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: postgresql-server-13.14-1.el9.x86_64 145/152 2024-04-21T08:27:10.295 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 146/152 2024-04-21T08:27:10.383 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : openssl-devel-1:3.0.7-27.el9.x86_64 147/152 2024-04-21T08:27:10.424 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : libaio-devel-0.3.111-13.el9.x86_64 148/152 2024-04-21T08:27:10.508 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : flex-2.6.4-9.el9.x86_64 149/152 2024-04-21T08:27:10.663 INFO:teuthology.orchestra.run.smithi179.stdout: Installing : bison-3.7.4-5.el9.x86_64 150/152 2024-04-21T08:27:10.663 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-21T08:27:10.664 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 151/152 2024-04-21T08:27:10.664 INFO:teuthology.orchestra.run.smithi179.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T08:27:16.691 INFO:teuthology.orchestra.run.smithi179.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T08:27:16.691 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-2:19.0.0-3154.ga630765c.el9.x86_64 1/152 2024-04-21T08:27:16.692 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 2/152 2024-04-21T08:27:16.692 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 3/152 2024-04-21T08:27:16.692 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 4/152 2024-04-21T08:27:16.692 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-immutable-object-cache-2:19.0.0-3154.ga6307 5/152 2024-04-21T08:27:16.692 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 6/152 2024-04-21T08:27:16.692 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 7/152 2024-04-21T08:27:16.692 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 8/152 2024-04-21T08:27:16.692 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 9/152 2024-04-21T08:27:16.692 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 10/152 2024-04-21T08:27:16.692 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 11/152 2024-04-21T08:27:16.692 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 12/152 2024-04-21T08:27:16.692 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 13/152 2024-04-21T08:27:16.692 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 14/152 2024-04-21T08:27:16.693 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_ 15/152 2024-04-21T08:27:16.693 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 16/152 2024-04-21T08:27:16.693 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 17/152 2024-04-21T08:27:16.693 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librados-devel-2:19.0.0-3154.ga630765c.el9.x86_6 18/152 2024-04-21T08:27:16.693 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86 19/152 2024-04-21T08:27:16.693 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 20/152 2024-04-21T08:27:16.693 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-ceph-argparse-2:19.0.0-3154.ga630765c.el 21/152 2024-04-21T08:27:16.693 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-ceph-common-2:19.0.0-3154.ga630765c.el9. 22/152 2024-04-21T08:27:16.693 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_6 23/152 2024-04-21T08:27:16.693 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 24/152 2024-04-21T08:27:16.693 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 25/152 2024-04-21T08:27:16.693 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 26/152 2024-04-21T08:27:16.693 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 27/152 2024-04-21T08:27:16.694 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 28/152 2024-04-21T08:27:16.694 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 29/152 2024-04-21T08:27:16.694 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-grafana-dashboards-2:19.0.0-3154.ga630765c. 30/152 2024-04-21T08:27:16.694 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noa 31/152 2024-04-21T08:27:16.694 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.n 32/152 2024-04-21T08:27:16.694 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-diskprediction-local-2:19.0.0-3154.ga63 33/152 2024-04-21T08:27:16.694 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el 34/152 2024-04-21T08:27:16.694 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 35/152 2024-04-21T08:27:16.694 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.e 36/152 2024-04-21T08:27:16.694 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 37/152 2024-04-21T08:27:16.694 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : cephadm-2:19.0.0-3154.ga630765c.el9.noarch 38/152 2024-04-21T08:27:16.694 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 39/152 2024-04-21T08:27:16.694 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : fuse-2.9.9-15.el9.x86_64 40/152 2024-04-21T08:27:16.695 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libgfortran-11.4.1-3.el9.x86_64 41/152 2024-04-21T08:27:16.695 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libquadmath-11.4.1-3.el9.x86_64 42/152 2024-04-21T08:27:16.695 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 43/152 2024-04-21T08:27:16.695 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cryptography-36.0.1-4.el9.x86_64 44/152 2024-04-21T08:27:16.695 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-ply-3.11-14.el9.noarch 45/152 2024-04-21T08:27:16.695 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 46/152 2024-04-21T08:27:16.695 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-requests-2.25.1-8.el9.noarch 47/152 2024-04-21T08:27:16.695 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-urllib3-1.26.5-5.el9.noarch 48/152 2024-04-21T08:27:16.695 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : automake-1.16.2-8.el9.noarch 49/152 2024-04-21T08:27:16.695 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : bison-3.7.4-5.el9.x86_64 50/152 2024-04-21T08:27:16.695 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : boost-program-options-1.75.0-8.el9.x86_64 51/152 2024-04-21T08:27:16.695 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : elfutils-libelf-devel-0.190-2.el9.x86_64 52/152 2024-04-21T08:27:16.695 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : flex-2.6.4-9.el9.x86_64 53/152 2024-04-21T08:27:16.696 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 54/152 2024-04-21T08:27:16.696 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 55/152 2024-04-21T08:27:16.696 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 56/152 2024-04-21T08:27:16.696 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libaio-devel-0.3.111-13.el9.x86_64 57/152 2024-04-21T08:27:16.696 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 58/152 2024-04-21T08:27:16.696 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 59/152 2024-04-21T08:27:16.696 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 60/152 2024-04-21T08:27:16.696 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libtool-2.4.6-45.el9.x86_64 61/152 2024-04-21T08:27:16.696 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libuuid-devel-2.37.4-18.el9.x86_64 62/152 2024-04-21T08:27:16.696 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libzstd-devel-1.5.1-2.el9.x86_64 63/152 2024-04-21T08:27:16.696 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 64/152 2024-04-21T08:27:16.696 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : lua-5.4.4-4.el9.x86_64 65/152 2024-04-21T08:27:16.696 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openblas-0.3.21-2.el9.x86_64 66/152 2024-04-21T08:27:16.696 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openblas-openmp-0.3.21-2.el9.x86_64 67/152 2024-04-21T08:27:16.697 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : openssl-devel-1:3.0.7-27.el9.x86_64 68/152 2024-04-21T08:27:16.697 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : postgresql-13.14-1.el9.x86_64 69/152 2024-04-21T08:27:16.697 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : postgresql-contrib-13.14-1.el9.x86_64 70/152 2024-04-21T08:27:16.697 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : postgresql-private-libs-13.14-1.el9.x86_64 71/152 2024-04-21T08:27:16.697 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : postgresql-server-13.14-1.el9.x86_64 72/152 2024-04-21T08:27:16.697 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : protobuf-3.14.0-13.el9.x86_64 73/152 2024-04-21T08:27:16.697 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-attrs-20.3.0-7.el9.noarch 74/152 2024-04-21T08:27:16.697 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 75/152 2024-04-21T08:27:16.697 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-devel-3.9.18-3.el9.x86_64 76/152 2024-04-21T08:27:16.697 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jinja2-2.11.3-5.el9.noarch 77/152 2024-04-21T08:27:16.697 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jsonpatch-1.21-16.el9.noarch 78/152 2024-04-21T08:27:16.697 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jsonpointer-2.0-4.el9.noarch 79/152 2024-04-21T08:27:16.697 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-mako-1.1.4-6.el9.noarch 80/152 2024-04-21T08:27:16.697 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 81/152 2024-04-21T08:27:16.698 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-numpy-1:1.20.1-5.el9.x86_64 82/152 2024-04-21T08:27:16.698 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-numpy-f2py-1:1.20.1-5.el9.x86_64 83/152 2024-04-21T08:27:16.698 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-oauthlib-3.1.1-5.el9.noarch 84/152 2024-04-21T08:27:16.698 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 85/152 2024-04-21T08:27:16.698 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-prettytable-0.7.2-27.el9.noarch 86/152 2024-04-21T08:27:16.698 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-protobuf-3.14.0-13.el9.noarch 87/152 2024-04-21T08:27:16.698 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 88/152 2024-04-21T08:27:16.698 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 89/152 2024-04-21T08:27:16.698 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pytz-2021.1-5.el9.noarch 90/152 2024-04-21T08:27:16.698 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 91/152 2024-04-21T08:27:16.698 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-scipy-1.6.2-8.el9.x86_64 92/152 2024-04-21T08:27:16.698 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 93/152 2024-04-21T08:27:16.698 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-wcwidth-0.2.5-8.el9.noarch 94/152 2024-04-21T08:27:16.699 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : qatlib-24.02.0-1.el9.x86_64 95/152 2024-04-21T08:27:16.699 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : qatlib-service-24.02.0-1.el9.x86_64 96/152 2024-04-21T08:27:16.699 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : qatzip-libs-1.2.0-1.el9.x86_64 97/152 2024-04-21T08:27:16.699 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : socat-1.7.4.1-5.el9.x86_64 98/152 2024-04-21T08:27:16.699 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : uuid-1.6.2-55.el9.x86_64 99/152 2024-04-21T08:27:16.699 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : xfsprogs-devel-6.3.0-1.el9.x86_64 100/152 2024-04-21T08:27:16.699 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 101/152 2024-04-21T08:27:16.699 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : zlib-devel-1.2.11-41.el9.x86_64 102/152 2024-04-21T08:27:16.699 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 103/152 2024-04-21T08:27:16.699 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : protobuf-compiler-3.14.0-13.el9.x86_64 104/152 2024-04-21T08:27:16.699 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : abseil-cpp-20211102.0-3.el9.x86_64 105/152 2024-04-21T08:27:16.699 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : gperftools-libs-2.9.1-2.el9.x86_64 106/152 2024-04-21T08:27:16.699 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 107/152 2024-04-21T08:27:16.700 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libarrow-9.0.0-10.el9.x86_64 108/152 2024-04-21T08:27:16.700 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libarrow-doc-9.0.0-10.el9.noarch 109/152 2024-04-21T08:27:16.700 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : liboath-2.6.7-2.el9.x86_64 110/152 2024-04-21T08:27:16.700 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 111/152 2024-04-21T08:27:16.700 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : luarocks-3.9.2-1.el9.noarch 112/152 2024-04-21T08:27:16.700 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : parquet-libs-9.0.0-10.el9.x86_64 113/152 2024-04-21T08:27:16.700 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 114/152 2024-04-21T08:27:16.700 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/152 2024-04-21T08:27:16.700 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/152 2024-04-21T08:27:16.700 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-certifi-2021.10.8-2.el9.noarch 117/152 2024-04-21T08:27:16.700 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cheroot-8.6.0-4.el9.noarch 118/152 2024-04-21T08:27:16.700 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/152 2024-04-21T08:27:16.700 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-cmd2-2.4.2-2.el9.noarch 120/152 2024-04-21T08:27:16.700 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-colorama-0.4.4-12.el9.noarch 121/152 2024-04-21T08:27:16.701 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-google-auth-1:2.28.2-1.el9.noarch 122/152 2024-04-21T08:27:16.701 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 123/152 2024-04-21T08:27:16.701 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 124/152 2024-04-21T08:27:16.701 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 125/152 2024-04-21T08:27:16.701 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 126/152 2024-04-21T08:27:16.701 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 127/152 2024-04-21T08:27:16.701 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 128/152 2024-04-21T08:27:16.701 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-jaraco-text-3.2.0-6.el9.noarch 129/152 2024-04-21T08:27:16.701 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-kubernetes-1:26.1.0-2.el9.noarch 130/152 2024-04-21T08:27:16.701 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-logutils-0.3.5-21.el9.noarch 131/152 2024-04-21T08:27:16.701 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 132/152 2024-04-21T08:27:16.701 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 133/152 2024-04-21T08:27:16.702 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pecan-1.4.2-3.el9.noarch 134/152 2024-04-21T08:27:16.702 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 135/152 2024-04-21T08:27:16.702 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 136/152 2024-04-21T08:27:16.702 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-pyperclip-1.8.2-2.el9.noarch 137/152 2024-04-21T08:27:16.702 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 138/152 2024-04-21T08:27:16.702 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 139/152 2024-04-21T08:27:16.702 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 140/152 2024-04-21T08:27:16.702 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 141/152 2024-04-21T08:27:16.702 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-typing-extensions-4.4.0-2.el9.noarch 142/152 2024-04-21T08:27:16.702 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-webob-1.8.7-6.el9.noarch 143/152 2024-04-21T08:27:16.702 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 144/152 2024-04-21T08:27:16.702 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-werkzeug-2.0.3-3.el9.1.noarch 145/152 2024-04-21T08:27:16.702 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 146/152 2024-04-21T08:27:16.703 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : re2-1:20211101-3.el9.x86_64 147/152 2024-04-21T08:27:16.703 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : thrift-0.15.0-2.el9.x86_64 148/152 2024-04-21T08:27:16.703 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librados2-2:19.0.0-3154.ga630765c.el9.x86_64 149/152 2024-04-21T08:27:16.703 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 150/152 2024-04-21T08:27:16.703 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 151/152 2024-04-21T08:27:18.795 INFO:teuthology.orchestra.run.smithi179.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 152/152 2024-04-21T08:27:18.795 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-21T08:27:18.796 INFO:teuthology.orchestra.run.smithi179.stdout:Upgraded: 2024-04-21T08:27:18.796 INFO:teuthology.orchestra.run.smithi179.stdout: librados2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:27:18.796 INFO:teuthology.orchestra.run.smithi179.stdout: librbd1-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:27:18.796 INFO:teuthology.orchestra.run.smithi179.stdout:Installed: 2024-04-21T08:27:18.796 INFO:teuthology.orchestra.run.smithi179.stdout: abseil-cpp-20211102.0-3.el9.x86_64 2024-04-21T08:27:18.796 INFO:teuthology.orchestra.run.smithi179.stdout: automake-1.16.2-8.el9.noarch 2024-04-21T08:27:18.796 INFO:teuthology.orchestra.run.smithi179.stdout: bison-3.7.4-5.el9.x86_64 2024-04-21T08:27:18.796 INFO:teuthology.orchestra.run.smithi179.stdout: boost-program-options-1.75.0-8.el9.x86_64 2024-04-21T08:27:18.796 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:27:18.796 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-base-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:27:18.796 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:27:18.796 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:27:18.797 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-grafana-dashboards-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:27:18.797 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-immutable-object-cache-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:27:18.797 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mds-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:27:18.797 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:27:18.797 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-cephadm-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:27:18.797 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-dashboard-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:27:18.797 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-diskprediction-local-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:27:18.797 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-modules-core-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:27:18.797 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mgr-rook-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:27:18.797 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-mon-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:27:18.797 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-osd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:27:18.797 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-prometheus-alerts-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:27:18.797 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-radosgw-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:27:18.797 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-selinux-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:27:18.798 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-test-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:27:18.798 INFO:teuthology.orchestra.run.smithi179.stdout: ceph-volume-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:27:18.798 INFO:teuthology.orchestra.run.smithi179.stdout: cephadm-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:27:18.798 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:27:18.798 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-shell-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:27:18.798 INFO:teuthology.orchestra.run.smithi179.stdout: cephfs-top-2:19.0.0-3154.ga630765c.el9.noarch 2024-04-21T08:27:18.798 INFO:teuthology.orchestra.run.smithi179.stdout: elfutils-libelf-devel-0.190-2.el9.x86_64 2024-04-21T08:27:18.798 INFO:teuthology.orchestra.run.smithi179.stdout: flex-2.6.4-9.el9.x86_64 2024-04-21T08:27:18.798 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-3.0.4-8.el9.x86_64 2024-04-21T08:27:18.798 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2024-04-21T08:27:18.798 INFO:teuthology.orchestra.run.smithi179.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2024-04-21T08:27:18.798 INFO:teuthology.orchestra.run.smithi179.stdout: fuse-2.9.9-15.el9.x86_64 2024-04-21T08:27:18.799 INFO:teuthology.orchestra.run.smithi179.stdout: gperftools-libs-2.9.1-2.el9.x86_64 2024-04-21T08:27:18.799 INFO:teuthology.orchestra.run.smithi179.stdout: grpc-data-1.46.7-10.el9.noarch 2024-04-21T08:27:18.799 INFO:teuthology.orchestra.run.smithi179.stdout: libaio-devel-0.3.111-13.el9.x86_64 2024-04-21T08:27:18.799 INFO:teuthology.orchestra.run.smithi179.stdout: libarrow-9.0.0-10.el9.x86_64 2024-04-21T08:27:18.799 INFO:teuthology.orchestra.run.smithi179.stdout: libarrow-doc-9.0.0-10.el9.noarch 2024-04-21T08:27:18.799 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs-devel-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:27:18.799 INFO:teuthology.orchestra.run.smithi179.stdout: libcephfs2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:27:18.799 INFO:teuthology.orchestra.run.smithi179.stdout: libcephsqlite-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:27:18.799 INFO:teuthology.orchestra.run.smithi179.stdout: libgfortran-11.4.1-3.el9.x86_64 2024-04-21T08:27:18.799 INFO:teuthology.orchestra.run.smithi179.stdout: liboath-2.6.7-2.el9.x86_64 2024-04-21T08:27:18.799 INFO:teuthology.orchestra.run.smithi179.stdout: libpmemobj-1.12.1-1.el9.x86_64 2024-04-21T08:27:18.799 INFO:teuthology.orchestra.run.smithi179.stdout: libquadmath-11.4.1-3.el9.x86_64 2024-04-21T08:27:18.799 INFO:teuthology.orchestra.run.smithi179.stdout: librabbitmq-0.11.0-7.el9.x86_64 2024-04-21T08:27:18.799 INFO:teuthology.orchestra.run.smithi179.stdout: librados-devel-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:27:18.800 INFO:teuthology.orchestra.run.smithi179.stdout: libradosstriper1-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:27:18.800 INFO:teuthology.orchestra.run.smithi179.stdout: librdkafka-1.6.1-102.el9.x86_64 2024-04-21T08:27:18.800 INFO:teuthology.orchestra.run.smithi179.stdout: librgw2-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:27:18.800 INFO:teuthology.orchestra.run.smithi179.stdout: libtool-2.4.6-45.el9.x86_64 2024-04-21T08:27:18.800 INFO:teuthology.orchestra.run.smithi179.stdout: libunwind-1.6.2-1.el9.x86_64 2024-04-21T08:27:18.800 INFO:teuthology.orchestra.run.smithi179.stdout: libuuid-devel-2.37.4-18.el9.x86_64 2024-04-21T08:27:18.800 INFO:teuthology.orchestra.run.smithi179.stdout: libzstd-devel-1.5.1-2.el9.x86_64 2024-04-21T08:27:18.800 INFO:teuthology.orchestra.run.smithi179.stdout: lttng-ust-2.12.0-6.el9.x86_64 2024-04-21T08:27:18.800 INFO:teuthology.orchestra.run.smithi179.stdout: lua-5.4.4-4.el9.x86_64 2024-04-21T08:27:18.800 INFO:teuthology.orchestra.run.smithi179.stdout: lua-devel-5.4.4-4.el9.x86_64 2024-04-21T08:27:18.800 INFO:teuthology.orchestra.run.smithi179.stdout: luarocks-3.9.2-1.el9.noarch 2024-04-21T08:27:18.800 INFO:teuthology.orchestra.run.smithi179.stdout: openblas-0.3.21-2.el9.x86_64 2024-04-21T08:27:18.801 INFO:teuthology.orchestra.run.smithi179.stdout: openblas-openmp-0.3.21-2.el9.x86_64 2024-04-21T08:27:18.801 INFO:teuthology.orchestra.run.smithi179.stdout: openssl-devel-1:3.0.7-27.el9.x86_64 2024-04-21T08:27:18.801 INFO:teuthology.orchestra.run.smithi179.stdout: parquet-libs-9.0.0-10.el9.x86_64 2024-04-21T08:27:18.801 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-13.14-1.el9.x86_64 2024-04-21T08:27:18.801 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-contrib-13.14-1.el9.x86_64 2024-04-21T08:27:18.801 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-private-libs-13.14-1.el9.x86_64 2024-04-21T08:27:18.801 INFO:teuthology.orchestra.run.smithi179.stdout: postgresql-server-13.14-1.el9.x86_64 2024-04-21T08:27:18.801 INFO:teuthology.orchestra.run.smithi179.stdout: protobuf-3.14.0-13.el9.x86_64 2024-04-21T08:27:18.801 INFO:teuthology.orchestra.run.smithi179.stdout: protobuf-compiler-3.14.0-13.el9.x86_64 2024-04-21T08:27:18.801 INFO:teuthology.orchestra.run.smithi179.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2024-04-21T08:27:18.801 INFO:teuthology.orchestra.run.smithi179.stdout: python3-attrs-20.3.0-7.el9.noarch 2024-04-21T08:27:18.801 INFO:teuthology.orchestra.run.smithi179.stdout: python3-babel-2.9.1-2.el9.noarch 2024-04-21T08:27:18.801 INFO:teuthology.orchestra.run.smithi179.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2024-04-21T08:27:18.801 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cachetools-4.2.4-1.el9.noarch 2024-04-21T08:27:18.801 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-argparse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:27:18.802 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ceph-common-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:27:18.802 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cephfs-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:27:18.802 INFO:teuthology.orchestra.run.smithi179.stdout: python3-certifi-2021.10.8-2.el9.noarch 2024-04-21T08:27:18.802 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cffi-1.14.5-5.el9.x86_64 2024-04-21T08:27:18.802 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cheroot-8.6.0-4.el9.noarch 2024-04-21T08:27:18.802 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2024-04-21T08:27:18.802 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cmd2-2.4.2-2.el9.noarch 2024-04-21T08:27:18.802 INFO:teuthology.orchestra.run.smithi179.stdout: python3-colorama-0.4.4-12.el9.noarch 2024-04-21T08:27:18.802 INFO:teuthology.orchestra.run.smithi179.stdout: python3-cryptography-36.0.1-4.el9.x86_64 2024-04-21T08:27:18.802 INFO:teuthology.orchestra.run.smithi179.stdout: python3-devel-3.9.18-3.el9.x86_64 2024-04-21T08:27:18.802 INFO:teuthology.orchestra.run.smithi179.stdout: python3-google-auth-1:2.28.2-1.el9.noarch 2024-04-21T08:27:18.802 INFO:teuthology.orchestra.run.smithi179.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2024-04-21T08:27:18.802 INFO:teuthology.orchestra.run.smithi179.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2024-04-21T08:27:18.803 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-8.2.1-3.el9.noarch 2024-04-21T08:27:18.803 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2024-04-21T08:27:18.803 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2024-04-21T08:27:18.803 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2024-04-21T08:27:18.803 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jaraco-text-3.2.0-6.el9.noarch 2024-04-21T08:27:18.803 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jinja2-2.11.3-5.el9.noarch 2024-04-21T08:27:18.803 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpatch-1.21-16.el9.noarch 2024-04-21T08:27:18.803 INFO:teuthology.orchestra.run.smithi179.stdout: python3-jsonpointer-2.0-4.el9.noarch 2024-04-21T08:27:18.803 INFO:teuthology.orchestra.run.smithi179.stdout: python3-kubernetes-1:26.1.0-2.el9.noarch 2024-04-21T08:27:18.803 INFO:teuthology.orchestra.run.smithi179.stdout: python3-logutils-0.3.5-21.el9.noarch 2024-04-21T08:27:18.803 INFO:teuthology.orchestra.run.smithi179.stdout: python3-mako-1.1.4-6.el9.noarch 2024-04-21T08:27:18.803 INFO:teuthology.orchestra.run.smithi179.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2024-04-21T08:27:18.803 INFO:teuthology.orchestra.run.smithi179.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2024-04-21T08:27:18.803 INFO:teuthology.orchestra.run.smithi179.stdout: python3-natsort-7.1.1-5.el9.noarch 2024-04-21T08:27:18.804 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy-1:1.20.1-5.el9.x86_64 2024-04-21T08:27:18.804 INFO:teuthology.orchestra.run.smithi179.stdout: python3-numpy-f2py-1:1.20.1-5.el9.x86_64 2024-04-21T08:27:18.804 INFO:teuthology.orchestra.run.smithi179.stdout: python3-oauthlib-3.1.1-5.el9.noarch 2024-04-21T08:27:18.804 INFO:teuthology.orchestra.run.smithi179.stdout: python3-packaging-20.9-5.el9.noarch 2024-04-21T08:27:18.804 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pecan-1.4.2-3.el9.noarch 2024-04-21T08:27:18.804 INFO:teuthology.orchestra.run.smithi179.stdout: python3-ply-3.11-14.el9.noarch 2024-04-21T08:27:18.804 INFO:teuthology.orchestra.run.smithi179.stdout: python3-portend-3.1.0-2.el9.noarch 2024-04-21T08:27:18.804 INFO:teuthology.orchestra.run.smithi179.stdout: python3-prettytable-0.7.2-27.el9.noarch 2024-04-21T08:27:18.804 INFO:teuthology.orchestra.run.smithi179.stdout: python3-protobuf-3.14.0-13.el9.noarch 2024-04-21T08:27:18.804 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2024-04-21T08:27:18.804 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2024-04-21T08:27:18.804 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2024-04-21T08:27:18.805 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pycparser-2.20-6.el9.noarch 2024-04-21T08:27:18.805 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pyperclip-1.8.2-2.el9.noarch 2024-04-21T08:27:18.805 INFO:teuthology.orchestra.run.smithi179.stdout: python3-pytz-2021.1-5.el9.noarch 2024-04-21T08:27:18.805 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rados-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:27:18.805 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rbd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:27:18.805 INFO:teuthology.orchestra.run.smithi179.stdout: python3-repoze-lru-0.7-16.el9.noarch 2024-04-21T08:27:18.805 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests-2.25.1-8.el9.noarch 2024-04-21T08:27:18.805 INFO:teuthology.orchestra.run.smithi179.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2024-04-21T08:27:18.805 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rgw-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:27:18.805 INFO:teuthology.orchestra.run.smithi179.stdout: python3-routes-2.5.1-5.el9.noarch 2024-04-21T08:27:18.805 INFO:teuthology.orchestra.run.smithi179.stdout: python3-rsa-4.9-2.el9.noarch 2024-04-21T08:27:18.805 INFO:teuthology.orchestra.run.smithi179.stdout: python3-scipy-1.6.2-8.el9.x86_64 2024-04-21T08:27:18.805 INFO:teuthology.orchestra.run.smithi179.stdout: python3-tempora-5.0.0-2.el9.noarch 2024-04-21T08:27:18.806 INFO:teuthology.orchestra.run.smithi179.stdout: python3-toml-0.10.2-6.el9.noarch 2024-04-21T08:27:18.806 INFO:teuthology.orchestra.run.smithi179.stdout: python3-typing-extensions-4.4.0-2.el9.noarch 2024-04-21T08:27:18.806 INFO:teuthology.orchestra.run.smithi179.stdout: python3-urllib3-1.26.5-5.el9.noarch 2024-04-21T08:27:18.806 INFO:teuthology.orchestra.run.smithi179.stdout: python3-wcwidth-0.2.5-8.el9.noarch 2024-04-21T08:27:18.806 INFO:teuthology.orchestra.run.smithi179.stdout: python3-webob-1.8.7-6.el9.noarch 2024-04-21T08:27:18.806 INFO:teuthology.orchestra.run.smithi179.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2024-04-21T08:27:18.806 INFO:teuthology.orchestra.run.smithi179.stdout: python3-werkzeug-2.0.3-3.el9.1.noarch 2024-04-21T08:27:18.806 INFO:teuthology.orchestra.run.smithi179.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2024-04-21T08:27:18.806 INFO:teuthology.orchestra.run.smithi179.stdout: qatlib-24.02.0-1.el9.x86_64 2024-04-21T08:27:18.806 INFO:teuthology.orchestra.run.smithi179.stdout: qatlib-service-24.02.0-1.el9.x86_64 2024-04-21T08:27:18.806 INFO:teuthology.orchestra.run.smithi179.stdout: qatzip-libs-1.2.0-1.el9.x86_64 2024-04-21T08:27:18.806 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-fuse-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:27:18.806 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-mirror-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:27:18.807 INFO:teuthology.orchestra.run.smithi179.stdout: rbd-nbd-2:19.0.0-3154.ga630765c.el9.x86_64 2024-04-21T08:27:18.807 INFO:teuthology.orchestra.run.smithi179.stdout: re2-1:20211101-3.el9.x86_64 2024-04-21T08:27:18.807 INFO:teuthology.orchestra.run.smithi179.stdout: socat-1.7.4.1-5.el9.x86_64 2024-04-21T08:27:18.807 INFO:teuthology.orchestra.run.smithi179.stdout: thrift-0.15.0-2.el9.x86_64 2024-04-21T08:27:18.807 INFO:teuthology.orchestra.run.smithi179.stdout: uuid-1.6.2-55.el9.x86_64 2024-04-21T08:27:18.807 INFO:teuthology.orchestra.run.smithi179.stdout: xfsprogs-devel-6.3.0-1.el9.x86_64 2024-04-21T08:27:18.807 INFO:teuthology.orchestra.run.smithi179.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2024-04-21T08:27:18.807 INFO:teuthology.orchestra.run.smithi179.stdout: zlib-devel-1.2.11-41.el9.x86_64 2024-04-21T08:27:18.808 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-21T08:27:18.808 INFO:teuthology.orchestra.run.smithi179.stdout:Complete! 2024-04-21T08:27:19.160 DEBUG:teuthology.parallel:result is None 2024-04-21T08:27:19.160 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:27:19.377 DEBUG:teuthology.orchestra.run.smithi086:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-21T08:27:19.403 INFO:teuthology.orchestra.run.smithi086.stdout:19.0.0-3154.ga630765c.el9 2024-04-21T08:27:19.404 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3154.ga630765c.el9 2024-04-21T08:27:19.404 INFO:teuthology.task.install:The correct ceph version 19.0.0-3154.ga630765c is installed. 2024-04-21T08:27:19.406 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:27:19.620 DEBUG:teuthology.orchestra.run.smithi088:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-21T08:27:19.646 INFO:teuthology.orchestra.run.smithi088.stdout:19.0.0-3154.ga630765c.el9 2024-04-21T08:27:19.646 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3154.ga630765c.el9 2024-04-21T08:27:19.646 INFO:teuthology.task.install:The correct ceph version 19.0.0-3154.ga630765c is installed. 2024-04-21T08:27:19.648 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:27:19.799 DEBUG:teuthology.orchestra.run.smithi179:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2024-04-21T08:27:19.821 INFO:teuthology.orchestra.run.smithi179.stdout:19.0.0-3154.ga630765c.el9 2024-04-21T08:27:19.821 INFO:teuthology.packaging:The installed version of ceph is 19.0.0-3154.ga630765c.el9 2024-04-21T08:27:19.822 INFO:teuthology.task.install:The correct ceph version 19.0.0-3154.ga630765c is installed. 2024-04-21T08:27:19.823 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2024-04-21T08:27:19.823 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-21T08:27:19.824 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-21T08:27:19.855 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-21T08:27:19.855 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-21T08:27:19.886 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-21T08:27:19.886 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2024-04-21T08:27:19.913 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2024-04-21T08:27:19.913 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-21T08:27:19.913 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/daemon-helper 2024-04-21T08:27:19.941 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-21T08:27:20.007 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-21T08:27:20.007 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/usr/bin/daemon-helper 2024-04-21T08:27:20.035 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-21T08:27:20.103 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-21T08:27:20.103 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/daemon-helper 2024-04-21T08:27:20.127 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/daemon-helper 2024-04-21T08:27:20.193 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2024-04-21T08:27:20.193 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-21T08:27:20.193 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-21T08:27:20.222 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-21T08:27:20.285 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-21T08:27:20.285 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-21T08:27:20.310 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-21T08:27:20.374 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-21T08:27:20.374 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/adjust-ulimits 2024-04-21T08:27:20.397 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2024-04-21T08:27:20.462 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2024-04-21T08:27:20.462 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-21T08:27:20.462 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/usr/bin/stdin-killer 2024-04-21T08:27:20.486 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-21T08:27:20.559 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-21T08:27:20.560 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/usr/bin/stdin-killer 2024-04-21T08:27:20.584 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-21T08:27:20.648 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-21T08:27:20.648 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/usr/bin/stdin-killer 2024-04-21T08:27:20.673 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod a=rx -- /usr/bin/stdin-killer 2024-04-21T08:27:20.737 INFO:teuthology.run_tasks:Running task cephadm... 2024-04-21T08:27:20.842 INFO:tasks.cephadm:Config: {'roleless': False, 'cephfs': {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True}, 'conf': {'client': {'client mount timeout': 600, 'debug client': 20, 'debug ms': 1, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'global': {'ms die on skipped message': False}, 'mds': {'debug mds': 20, 'debug mds balancer': 20, 'debug ms': 1, 'mds bal fragment size max': 10000, 'mds bal merge size': 5, 'mds bal split bits': 3, 'mds bal split size': 100, 'mds debug frag': True, 'mds debug scatterstat': True, 'mds op complaint time': 180, 'mds verify scatter': True, 'mds_bal_replicate_threshold': 1, 'osd op complaint time': 180, 'rados mon op timeout': 900, 'rados osd op timeout': 900}, 'mgr': {'cephsqlite lock renewal timeout': 900000, 'debug cephsqlite': 20, 'debug client': 20, 'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'mon down mkfs grace': 300, 'mon op complaint time': 120}, 'osd': {'bluestore block size': 96636764160, 'bluestore fsck on mount': True, 'debug bluefs': 20, 'debug bluestore': 20, 'debug ms': 1, 'debug osd': 20, 'debug rocksdb': 10, 'mon osd backfillfull_ratio': 0.85, 'mon osd full ratio': 0.9, 'mon osd nearfull ratio': 0.8, 'osd failsafe full ratio': 0.95, 'osd objectstore': 'bluestore', 'osd op complaint time': 180, 'osd shutdown pgref assert': True, 'osd_max_omap_entries_per_request': 10000}}, 'flavor': 'default', 'fs': 'xfs', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', 'FS_DEGRADED', 'FS_INLINE_DATA_DEPRECATED', 'FS_WITH_FAILED_MDS', 'MDS_ALL_DOWN', 'MDS_DAMAGE', 'MDS_DEGRADED', 'MDS_FAILED', 'MDS_INSUFFICIENT_STANDBY', 'MDS_UP_LESS_THAN_MAX', 'filesystem is online with fewer MDS than max_mds', 'POOL_APP_NOT_ENABLED', 'do not have an application enabled', 'overall HEALTH_', 'Replacing daemon', 'deprecated feature inline_data', 'overall HEALTH_', '\\(OSD_DOWN\\)', '\\(OSD_', 'but it is still running', 'is not responding', 'PG_AVAILABILITY', 'PG_DEGRADED', 'Reduced data availability', 'slow metadata IO', 'SLOW_OPS', 'slow request'], 'sha1': 'a630765c1f93c6e61d9b40c4b76437611d5ef29e', 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-21T08:27:20.842 INFO:tasks.cephadm:Cluster image is quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:27:20.843 INFO:tasks.cephadm:Cluster fsid is f8cec7cc-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:27:20.843 INFO:tasks.cephadm:Choosing monitor IPs and ports... 2024-04-21T08:27:20.843 INFO:tasks.cephadm:Monitor IPs: {'mon.a': '172.21.15.86', 'mon.b': '172.21.15.88', 'mon.c': '172.21.15.179'} 2024-04-21T08:27:20.843 INFO:tasks.cephadm:First mon is mon.a on smithi086 2024-04-21T08:27:20.843 INFO:tasks.cephadm:First mgr is x 2024-04-21T08:27:20.843 INFO:tasks.cephadm:Normalizing hostnames... 2024-04-21T08:27:20.844 DEBUG:teuthology.orchestra.run.smithi086:> sudo hostname $(hostname -s) 2024-04-21T08:27:20.868 DEBUG:teuthology.orchestra.run.smithi088:> sudo hostname $(hostname -s) 2024-04-21T08:27:20.891 DEBUG:teuthology.orchestra.run.smithi179:> sudo hostname $(hostname -s) 2024-04-21T08:27:20.916 INFO:tasks.cephadm:Downloading "compiled" cephadm from cachra 2024-04-21T08:27:20.916 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=default&distros=centos%2F9%2Fx86_64&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:27:21.057 INFO:tasks.cephadm:builder_project result: [{'status': 'ready', 'sha1': 'a630765c1f93c6e61d9b40c4b76437611d5ef29e', 'extra': {'build_url': 'https://jenkins.ceph.com/job/ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic/78740/', 'root_build_cause': 'SCMTRIGGER', 'version': '19.0.0-3154-ga630765c', 'node_name': '172.21.2.1+braggi01', 'job_name': 'ceph-dev-new-build/ARCH=x86_64,AVAILABLE_ARCH=x86_64,AVAILABLE_DIST=centos9,DIST=centos9,MACHINE_SIZE=gigantic', 'package_manager_version': '19.0.0-3154.ga630765c'}, 'url': 'https://2.chacra.ceph.com/r/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/', 'distro_codename': None, 'modified': '2024-04-19 11:13:53.331234', 'distro_version': '9', 'project': 'ceph', 'flavor': 'default', 'ref': 'wip-lusov-quiescer', 'chacra_url': 'https://2.chacra.ceph.com/repos/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/flavors/default/', 'archs': ['source', 'x86_64'], 'distro': 'centos'}] 2024-04-21T08:27:21.294 INFO:tasks.util.chacra:got chacra host 2.chacra.ceph.com, ref wip-lusov-quiescer, sha1 a630765c1f93c6e61d9b40c4b76437611d5ef29e from https://shaman.ceph.com/api/search/?project=ceph&distros=centos%2F9%2Fx86_64&flavor=default&sha1=a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:27:21.295 INFO:tasks.cephadm:Discovered cachra url: https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/x86_64/flavors/default/cephadm 2024-04-21T08:27:21.295 DEBUG:teuthology.orchestra.run.smithi086:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-21T08:27:21.947 INFO:teuthology.orchestra.run.smithi086.stdout:-rw-r--r--. 1 ubuntu ubuntu 781532 Apr 21 08:27 /home/ubuntu/cephtest/cephadm 2024-04-21T08:27:21.947 DEBUG:teuthology.orchestra.run.smithi088:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-21T08:27:22.457 INFO:teuthology.orchestra.run.smithi088.stdout:-rw-r--r--. 1 ubuntu ubuntu 781532 Apr 21 08:27 /home/ubuntu/cephtest/cephadm 2024-04-21T08:27:22.457 DEBUG:teuthology.orchestra.run.smithi179:> curl --silent -L https://2.chacra.ceph.com/binaries/ceph/wip-lusov-quiescer/a630765c1f93c6e61d9b40c4b76437611d5ef29e/centos/9/x86_64/flavors/default/cephadm > /home/ubuntu/cephtest/cephadm && ls -l /home/ubuntu/cephtest/cephadm 2024-04-21T08:27:23.098 INFO:teuthology.orchestra.run.smithi179.stdout:-rw-r--r--. 1 ubuntu ubuntu 781532 Apr 21 08:27 /home/ubuntu/cephtest/cephadm 2024-04-21T08:27:23.098 DEBUG:teuthology.orchestra.run.smithi086:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-21T08:27:23.117 DEBUG:teuthology.orchestra.run.smithi088:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-21T08:27:23.136 DEBUG:teuthology.orchestra.run.smithi179:> test -s /home/ubuntu/cephtest/cephadm && test $(stat -c%s /home/ubuntu/cephtest/cephadm) -gt 1000 && chmod +x /home/ubuntu/cephtest/cephadm 2024-04-21T08:27:23.171 INFO:tasks.cephadm:Pulling image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e on all hosts... 2024-04-21T08:27:23.171 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e pull 2024-04-21T08:27:23.174 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e pull 2024-04-21T08:27:23.179 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e pull 2024-04-21T08:27:24.776 INFO:teuthology.orchestra.run.smithi086.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e... 2024-04-21T08:27:24.788 INFO:teuthology.orchestra.run.smithi179.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e... 2024-04-21T08:27:24.881 INFO:teuthology.orchestra.run.smithi088.stderr:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e... 2024-04-21T08:27:53.401 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-21T08:27:53.402 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_version": "ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev)", 2024-04-21T08:27:53.402 INFO:teuthology.orchestra.run.smithi088.stdout: "image_id": "b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192", 2024-04-21T08:27:53.402 INFO:teuthology.orchestra.run.smithi088.stdout: "repo_digests": [ 2024-04-21T08:27:53.402 INFO:teuthology.orchestra.run.smithi088.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda" 2024-04-21T08:27:53.402 INFO:teuthology.orchestra.run.smithi088.stdout: ] 2024-04-21T08:27:53.402 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-21T08:28:11.084 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:28:11.084 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev)", 2024-04-21T08:28:11.084 INFO:teuthology.orchestra.run.smithi086.stdout: "image_id": "b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192", 2024-04-21T08:28:11.085 INFO:teuthology.orchestra.run.smithi086.stdout: "repo_digests": [ 2024-04-21T08:28:11.085 INFO:teuthology.orchestra.run.smithi086.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda" 2024-04-21T08:28:11.085 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-21T08:28:11.085 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:28:14.068 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-04-21T08:28:14.068 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_version": "ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev)", 2024-04-21T08:28:14.068 INFO:teuthology.orchestra.run.smithi179.stdout: "image_id": "b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192", 2024-04-21T08:28:14.068 INFO:teuthology.orchestra.run.smithi179.stdout: "repo_digests": [ 2024-04-21T08:28:14.068 INFO:teuthology.orchestra.run.smithi179.stdout: "quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda" 2024-04-21T08:28:14.068 INFO:teuthology.orchestra.run.smithi179.stdout: ] 2024-04-21T08:28:14.068 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-04-21T08:28:14.093 DEBUG:teuthology.orchestra.run.smithi086:> sudo mkdir -p /etc/ceph 2024-04-21T08:28:14.124 DEBUG:teuthology.orchestra.run.smithi088:> sudo mkdir -p /etc/ceph 2024-04-21T08:28:14.156 DEBUG:teuthology.orchestra.run.smithi179:> sudo mkdir -p /etc/ceph 2024-04-21T08:28:14.184 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 777 /etc/ceph 2024-04-21T08:28:14.211 DEBUG:teuthology.orchestra.run.smithi088:> sudo chmod 777 /etc/ceph 2024-04-21T08:28:14.239 DEBUG:teuthology.orchestra.run.smithi179:> sudo chmod 777 /etc/ceph 2024-04-21T08:28:14.263 INFO:tasks.cephadm:Writing seed config... 2024-04-21T08:28:14.264 INFO:tasks.cephadm: override: [client] client mount timeout = 600 2024-04-21T08:28:14.265 INFO:tasks.cephadm: override: [client] debug client = 20 2024-04-21T08:28:14.265 INFO:tasks.cephadm: override: [client] debug ms = 1 2024-04-21T08:28:14.265 INFO:tasks.cephadm: override: [client] rados mon op timeout = 900 2024-04-21T08:28:14.265 INFO:tasks.cephadm: override: [client] rados osd op timeout = 900 2024-04-21T08:28:14.265 INFO:tasks.cephadm: override: [global] ms die on skipped message = False 2024-04-21T08:28:14.265 INFO:tasks.cephadm: override: [mds] debug mds = 20 2024-04-21T08:28:14.265 INFO:tasks.cephadm: override: [mds] debug mds balancer = 20 2024-04-21T08:28:14.265 INFO:tasks.cephadm: override: [mds] debug ms = 1 2024-04-21T08:28:14.265 INFO:tasks.cephadm: override: [mds] mds bal fragment size max = 10000 2024-04-21T08:28:14.265 INFO:tasks.cephadm: override: [mds] mds bal merge size = 5 2024-04-21T08:28:14.265 INFO:tasks.cephadm: override: [mds] mds bal split bits = 3 2024-04-21T08:28:14.265 INFO:tasks.cephadm: override: [mds] mds bal split size = 100 2024-04-21T08:28:14.266 INFO:tasks.cephadm: override: [mds] mds debug frag = True 2024-04-21T08:28:14.266 INFO:tasks.cephadm: override: [mds] mds debug scatterstat = True 2024-04-21T08:28:14.266 INFO:tasks.cephadm: override: [mds] mds op complaint time = 180 2024-04-21T08:28:14.266 INFO:tasks.cephadm: override: [mds] mds verify scatter = True 2024-04-21T08:28:14.266 INFO:tasks.cephadm: override: [mds] mds_bal_replicate_threshold = 1 2024-04-21T08:28:14.266 INFO:tasks.cephadm: override: [mds] osd op complaint time = 180 2024-04-21T08:28:14.266 INFO:tasks.cephadm: override: [mds] rados mon op timeout = 900 2024-04-21T08:28:14.266 INFO:tasks.cephadm: override: [mds] rados osd op timeout = 900 2024-04-21T08:28:14.267 INFO:tasks.cephadm: override: [mgr] cephsqlite lock renewal timeout = 900000 2024-04-21T08:28:14.267 INFO:tasks.cephadm: override: [mgr] debug cephsqlite = 20 2024-04-21T08:28:14.267 INFO:tasks.cephadm: override: [mgr] debug client = 20 2024-04-21T08:28:14.267 INFO:tasks.cephadm: override: [mgr] debug mgr = 20 2024-04-21T08:28:14.267 INFO:tasks.cephadm: override: [mgr] debug ms = 1 2024-04-21T08:28:14.267 INFO:tasks.cephadm: override: [mon] debug mon = 20 2024-04-21T08:28:14.267 INFO:tasks.cephadm: override: [mon] debug ms = 1 2024-04-21T08:28:14.267 INFO:tasks.cephadm: override: [mon] debug paxos = 20 2024-04-21T08:28:14.267 INFO:tasks.cephadm: override: [mon] mon down mkfs grace = 300 2024-04-21T08:28:14.267 INFO:tasks.cephadm: override: [mon] mon op complaint time = 120 2024-04-21T08:28:14.268 INFO:tasks.cephadm: override: [osd] bluestore block size = 96636764160 2024-04-21T08:28:14.268 INFO:tasks.cephadm: override: [osd] bluestore fsck on mount = True 2024-04-21T08:28:14.268 INFO:tasks.cephadm: override: [osd] debug bluefs = 20 2024-04-21T08:28:14.268 INFO:tasks.cephadm: override: [osd] debug bluestore = 20 2024-04-21T08:28:14.268 INFO:tasks.cephadm: override: [osd] debug ms = 1 2024-04-21T08:28:14.268 INFO:tasks.cephadm: override: [osd] debug osd = 20 2024-04-21T08:28:14.268 INFO:tasks.cephadm: override: [osd] debug rocksdb = 10 2024-04-21T08:28:14.268 INFO:tasks.cephadm: override: [osd] mon osd backfillfull_ratio = 0.85 2024-04-21T08:28:14.268 INFO:tasks.cephadm: override: [osd] mon osd full ratio = 0.9 2024-04-21T08:28:14.268 INFO:tasks.cephadm: override: [osd] mon osd nearfull ratio = 0.8 2024-04-21T08:28:14.268 INFO:tasks.cephadm: override: [osd] osd failsafe full ratio = 0.95 2024-04-21T08:28:14.268 INFO:tasks.cephadm: override: [osd] osd objectstore = bluestore 2024-04-21T08:28:14.268 INFO:tasks.cephadm: override: [osd] osd op complaint time = 180 2024-04-21T08:28:14.268 INFO:tasks.cephadm: override: [osd] osd shutdown pgref assert = True 2024-04-21T08:28:14.269 INFO:tasks.cephadm: override: [osd] osd_max_omap_entries_per_request = 10000 2024-04-21T08:28:14.270 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-21T08:28:14.270 DEBUG:teuthology.orchestra.run.smithi086:> dd of=/home/ubuntu/cephtest/seed.ceph.conf 2024-04-21T08:28:14.287 DEBUG:tasks.cephadm:Final config: [global] # make logging friendly to teuthology log_to_file = true log_to_stderr = false log to journald = false mon cluster log to file = true mon cluster log file level = debug mon clock drift allowed = 1.000 # replicate across OSDs, not hosts osd crush chooseleaf type = 0 #osd pool default size = 2 osd pool default erasure code profile = plugin=jerasure technique=reed_sol_van k=2 m=1 crush-failure-domain=osd # enable some debugging auth debug = true ms die on old message = true ms die on bug = true debug asserts on shutdown = true # adjust warnings mon max pg per osd = 10000# >= luminous mon pg warn max object skew = 0 mon osd allow primary affinity = true mon osd allow pg remap = true mon warn on legacy crush tunables = false mon warn on crush straw calc version zero = false mon warn on no sortbitwise = false mon warn on osd down out interval zero = false mon warn on too few osds = false mon_warn_on_pool_pg_num_not_power_of_two = false # disable pg_autoscaler by default for new pools osd_pool_default_pg_autoscale_mode = off # tests delete pools mon allow pool delete = true fsid = f8cec7cc-ffb8-11ee-bc93-c7b262605968 ms die on skipped message = False [osd] osd scrub load threshold = 5.0 osd scrub max interval = 600 osd mclock profile = high_recovery_ops osd recover clone overlap = true osd recovery max chunk = 1048576 osd deep scrub update digest min age = 30 osd map max advance = 10 osd memory target autotune = true # debugging osd debug shutdown = true osd debug op order = true osd debug verify stray on activate = true osd debug pg log writeout = true osd debug verify cached snaps = true osd debug verify missing on start = true osd debug misdirected ops = true osd op queue = debug_random osd op queue cut off = debug_random osd shutdown pgref assert = True bdev debug aio = true osd sloppy crc = true bluestore block size = 96636764160 bluestore fsck on mount = True debug bluefs = 20 debug bluestore = 20 debug ms = 1 debug osd = 20 debug rocksdb = 10 mon osd backfillfull_ratio = 0.85 mon osd full ratio = 0.9 mon osd nearfull ratio = 0.8 osd failsafe full ratio = 0.95 osd objectstore = bluestore osd op complaint time = 180 osd_max_omap_entries_per_request = 10000 [mgr] mon reweight min pgs per osd = 4 mon reweight min bytes per osd = 10 mgr/telemetry/nag = false cephsqlite lock renewal timeout = 900000 debug cephsqlite = 20 debug client = 20 debug mgr = 20 debug ms = 1 [mon] mon data avail warn = 5 mon mgr mkfs grace = 240 mon reweight min pgs per osd = 4 mon osd reporter subtree level = osd mon osd prime pg temp = true mon reweight min bytes per osd = 10 # rotate auth tickets quickly to exercise renewal paths auth mon ticket ttl = 660# 11m auth service ticket ttl = 240# 4m # don't complain about global id reclaim mon_warn_on_insecure_global_id_reclaim = false mon_warn_on_insecure_global_id_reclaim_allowed = false debug mon = 20 debug ms = 1 debug paxos = 20 mon down mkfs grace = 300 mon op complaint time = 120 [client.rgw] rgw cache enabled = true rgw enable ops log = true rgw enable usage log = true [client] client mount timeout = 600 debug client = 20 debug ms = 1 rados mon op timeout = 900 rados osd op timeout = 900 [mds] debug mds = 20 debug mds balancer = 20 debug ms = 1 mds bal fragment size max = 10000 mds bal merge size = 5 mds bal split bits = 3 mds bal split size = 100 mds debug frag = True mds debug scatterstat = True mds op complaint time = 180 mds verify scatter = True mds_bal_replicate_threshold = 1 osd op complaint time = 180 rados mon op timeout = 900 rados osd op timeout = 900 2024-04-21T08:28:14.287 DEBUG:teuthology.orchestra.run.smithi086:mon.a> sudo journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mon.a.service 2024-04-21T08:28:14.331 DEBUG:teuthology.orchestra.run.smithi086:mgr.x> sudo journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mgr.x.service 2024-04-21T08:28:14.374 INFO:tasks.cephadm:Bootstrapping... 2024-04-21T08:28:14.374 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e -v bootstrap --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 --config /home/ubuntu/cephtest/seed.ceph.conf --output-config /etc/ceph/ceph.conf --output-keyring /etc/ceph/ceph.client.admin.keyring --output-pub-ssh-key /home/ubuntu/cephtest/ceph.pub --mon-id a --mgr-id x --orphan-initial-daemons --skip-monitoring-stack --mon-ip 172.21.15.86 --skip-admin-label && sudo chmod +r /etc/ceph/ceph.client.admin.keyring 2024-04-21T08:28:14.563 INFO:teuthology.orchestra.run.smithi086.stdout:-------------------------------------------------------------------------------- 2024-04-21T08:28:14.564 INFO:teuthology.orchestra.run.smithi086.stdout:cephadm ['--image', 'quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e', '-v', 'bootstrap', '--fsid', 'f8cec7cc-ffb8-11ee-bc93-c7b262605968', '--config', '/home/ubuntu/cephtest/seed.ceph.conf', '--output-config', '/etc/ceph/ceph.conf', '--output-keyring', '/etc/ceph/ceph.client.admin.keyring', '--output-pub-ssh-key', '/home/ubuntu/cephtest/ceph.pub', '--mon-id', 'a', '--mgr-id', 'x', '--orphan-initial-daemons', '--skip-monitoring-stack', '--mon-ip', '172.21.15.86', '--skip-admin-label'] 2024-04-21T08:28:14.564 INFO:teuthology.orchestra.run.smithi086.stderr:Specifying an fsid for your cluster offers no advantages and may increase the likelihood of fsid conflicts. 2024-04-21T08:28:14.564 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying podman|docker is present... 2024-04-21T08:28:14.588 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-21T08:28:14.589 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying lvm2 is present... 2024-04-21T08:28:14.589 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying time synchronization is in place... 2024-04-21T08:28:14.596 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-21T08:28:14.596 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-21T08:28:14.602 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-21T08:28:14.602 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-04-21T08:28:14.608 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout enabled 2024-04-21T08:28:14.614 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout active 2024-04-21T08:28:14.614 INFO:teuthology.orchestra.run.smithi086.stdout:Unit chronyd.service is enabled and running 2024-04-21T08:28:14.614 INFO:teuthology.orchestra.run.smithi086.stdout:Repeating the final host check... 2024-04-21T08:28:14.638 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stdout 4.9.4-dev 2024-04-21T08:28:14.639 INFO:teuthology.orchestra.run.smithi086.stdout:podman (/bin/podman) version 4.9.4 is present 2024-04-21T08:28:14.639 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl is present 2024-04-21T08:28:14.639 INFO:teuthology.orchestra.run.smithi086.stdout:lvcreate is present 2024-04-21T08:28:14.645 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled chrony.service 2024-04-21T08:28:14.645 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Failed to get unit file state for chrony.service: No such file or directory 2024-04-21T08:28:14.650 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active chrony.service 2024-04-21T08:28:14.650 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-04-21T08:28:14.656 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout enabled 2024-04-21T08:28:14.663 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout active 2024-04-21T08:28:14.663 INFO:teuthology.orchestra.run.smithi086.stdout:Unit chronyd.service is enabled and running 2024-04-21T08:28:14.663 INFO:teuthology.orchestra.run.smithi086.stdout:Host looks OK 2024-04-21T08:28:14.663 INFO:teuthology.orchestra.run.smithi086.stdout:Cluster fsid: f8cec7cc-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:28:14.663 INFO:teuthology.orchestra.run.smithi086.stdout:Acquiring lock 140472363150496 on /run/cephadm/f8cec7cc-ffb8-11ee-bc93-c7b262605968.lock 2024-04-21T08:28:14.663 INFO:teuthology.orchestra.run.smithi086.stdout:Lock 140472363150496 acquired on /run/cephadm/f8cec7cc-ffb8-11ee-bc93-c7b262605968.lock 2024-04-21T08:28:14.663 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying IP 172.21.15.86 port 3300 ... 2024-04-21T08:28:14.664 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying IP 172.21.15.86 port 6789 ... 2024-04-21T08:28:14.664 INFO:teuthology.orchestra.run.smithi086.stdout:Base mon IP(s) is [172.21.15.86:3300, 172.21.15.86:6789], mon addrv is [v2:172.21.15.86:3300,v1:172.21.15.86:6789] 2024-04-21T08:28:14.667 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout default via 172.21.15.254 dev enp3s0f1 proto dhcp src 172.21.15.86 metric 100 2024-04-21T08:28:14.667 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout 172.21.0.0/20 dev enp3s0f1 proto kernel scope link src 172.21.15.86 metric 100 2024-04-21T08:28:14.669 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout ::1 dev lo proto kernel metric 256 pref medium 2024-04-21T08:28:14.669 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout fe80::/64 dev enp3s0f1 proto kernel metric 1024 pref medium 2024-04-21T08:28:14.669 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout default via fe80::327c:5e00:6487:71e0 dev enp3s0f1 proto ra metric 100 pref medium 2024-04-21T08:28:14.672 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout 1: lo: mtu 65536 state UNKNOWN qlen 1000 2024-04-21T08:28:14.672 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout inet6 ::1/128 scope host 2024-04-21T08:28:14.672 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-21T08:28:14.672 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout 5: enp3s0f1: mtu 1500 state UP qlen 1000 2024-04-21T08:28:14.672 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout inet6 fe80::ecad:5405:e8c2:c812/64 scope link noprefixroute 2024-04-21T08:28:14.672 INFO:teuthology.orchestra.run.smithi086.stdout:/sbin/ip: stdout valid_lft forever preferred_lft forever 2024-04-21T08:28:14.673 INFO:teuthology.orchestra.run.smithi086.stdout:Mon IP `172.21.15.86` is in CIDR network `172.21.0.0/20` 2024-04-21T08:28:14.673 INFO:teuthology.orchestra.run.smithi086.stdout:Mon IP `172.21.15.86` is in CIDR network `172.21.0.0/20` 2024-04-21T08:28:14.673 INFO:teuthology.orchestra.run.smithi086.stdout:Inferred mon public CIDR from local network configuration ['172.21.0.0/20', '172.21.0.0/20'] 2024-04-21T08:28:14.673 INFO:teuthology.orchestra.run.smithi086.stdout:Internal network (--cluster-network) has not been provided, OSD replication will default to the public_network 2024-04-21T08:28:14.674 INFO:teuthology.orchestra.run.smithi086.stdout:Pulling container image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e... 2024-04-21T08:28:15.094 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stdout b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 2024-04-21T08:28:15.095 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Trying to pull quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e... 2024-04-21T08:28:15.095 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Getting image source signatures 2024-04-21T08:28:15.095 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Copying blob sha256:7eab8a2da10d06ac88aea1fd8998a2559b4de9d07c585e3b1e6ee6c4d35fc119 2024-04-21T08:28:15.095 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Copying blob sha256:426fede9fffa2c7dfe40a4f9322a34794b72405170134d54d2c30d9cc6c6a5be 2024-04-21T08:28:15.095 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Copying config sha256:b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 2024-04-21T08:28:15.095 INFO:teuthology.orchestra.run.smithi086.stdout:/bin/podman: stderr Writing manifest to image destination 2024-04-21T08:28:16.084 INFO:teuthology.orchestra.run.smithi086.stdout:ceph: stdout ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev) 2024-04-21T08:28:16.084 INFO:teuthology.orchestra.run.smithi086.stdout:Ceph version: ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev) 2024-04-21T08:28:16.084 INFO:teuthology.orchestra.run.smithi086.stdout:Extracting ceph user uid/gid from container image... 2024-04-21T08:28:17.037 INFO:teuthology.orchestra.run.smithi086.stdout:stat: stdout 167 167 2024-04-21T08:28:17.037 INFO:teuthology.orchestra.run.smithi086.stdout:Creating initial keys... 2024-04-21T08:28:17.913 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph-authtool: stdout AQAhziRmRUllFxAACaaCCuV1O61dyePW9BHdZQ== 2024-04-21T08:28:18.802 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph-authtool: stdout AQAiziRm3pxQDBAAVxzR4olkPzwWOoJk34DfWQ== 2024-04-21T08:28:19.583 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph-authtool: stdout AQAjziRm8OzGBRAAePGICiVYx3ljxpV/xtV9Ww== 2024-04-21T08:28:19.584 INFO:teuthology.orchestra.run.smithi086.stdout:Creating initial monmap... 2024-04-21T08:28:20.414 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-21T08:28:20.414 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: stdout setting min_mon_release = quincy 2024-04-21T08:28:20.414 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: set fsid to f8cec7cc-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:28:20.414 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: stdout /usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-21T08:28:20.414 INFO:teuthology.orchestra.run.smithi086.stdout:monmaptool for a [v2:172.21.15.86:3300,v1:172.21.15.86:6789] on /usr/bin/monmaptool: monmap file /tmp/monmap 2024-04-21T08:28:20.414 INFO:teuthology.orchestra.run.smithi086.stdout:setting min_mon_release = quincy 2024-04-21T08:28:20.414 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: set fsid to f8cec7cc-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:28:20.415 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/monmaptool: writing epoch 0 to /tmp/monmap (1 monitors) 2024-04-21T08:28:20.415 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:28:20.415 INFO:teuthology.orchestra.run.smithi086.stdout:Creating mon... 2024-04-21T08:28:21.287 INFO:teuthology.orchestra.run.smithi086.stdout:create mon.a on 2024-04-21T08:28:21.501 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Removed "/etc/systemd/system/multi-user.target.wants/ceph.target". 2024-04-21T08:28:21.653 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /etc/systemd/system/ceph.target. 2024-04-21T08:28:21.830 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Created symlink /etc/systemd/system/multi-user.target.wants/ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968.target → /etc/systemd/system/ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968.target. 2024-04-21T08:28:21.830 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph.target.wants/ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968.target → /etc/systemd/system/ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968.target. 2024-04-21T08:28:22.081 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mon.a 2024-04-21T08:28:22.081 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mon.a.service: Unit ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mon.a.service not loaded. 2024-04-21T08:28:22.238 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968.target.wants/ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mon.a.service → /etc/systemd/system/ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@.service. 2024-04-21T08:28:22.714 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:22 smithi086 ceph-mon[27570]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-21T08:28:22.714 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:22 smithi086 bash[27551]: 81099027997be48f0ea6a63052ba0e2c1529ea314ab487c067c3adb10a186c78 2024-04-21T08:28:22.727 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-21T08:28:22.727 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout disabled 2024-04-21T08:28:22.734 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-21T08:28:22.734 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-04-21T08:28:22.734 INFO:teuthology.orchestra.run.smithi086.stdout:firewalld.service is not enabled 2024-04-21T08:28:22.734 INFO:teuthology.orchestra.run.smithi086.stdout:Not possible to enable service . firewalld.service is not available 2024-04-21T08:28:22.735 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mon to start... 2024-04-21T08:28:22.735 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mon... 2024-04-21T08:28:22.973 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:22 smithi086 systemd[1]: Started Ceph mon.a for f8cec7cc-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:28:23.804 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout cluster: 2024-04-21T08:28:23.805 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout id: f8cec7cc-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:28:23.805 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout health: HEALTH_OK 2024-04-21T08:28:23.805 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-21T08:28:23.805 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout services: 2024-04-21T08:28:23.805 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon: 1 daemons, quorum a (age 0.541021s) 2024-04-21T08:28:23.805 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mgr: no daemons active 2024-04-21T08:28:23.805 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd: 0 osds: 0 up, 0 in 2024-04-21T08:28:23.805 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-21T08:28:23.805 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout data: 2024-04-21T08:28:23.805 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout pools: 0 pools, 0 pgs 2024-04-21T08:28:23.805 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout objects: 0 objects, 0 B 2024-04-21T08:28:23.805 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout usage: 0 B used, 0 B / 0 B avail 2024-04-21T08:28:23.806 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout pgs: 2024-04-21T08:28:23.806 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-21T08:28:23.806 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.150+0000 7fa08d373700 1 Processor -- start 2024-04-21T08:28:23.806 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.150+0000 7fa08d373700 1 -- start start 2024-04-21T08:28:23.806 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.150+0000 7fa08d373700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa088108800 0x7fa088108c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:23.806 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.150+0000 7fa08d373700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0881091f0 con 0x7fa088108800 2024-04-21T08:28:23.806 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.151+0000 7fa086ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa088108800 0x7fa088108c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:23.806 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.151+0000 7fa086ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa088108800 0x7fa088108c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59760/0 (socket says 172.21.15.86:59760) 2024-04-21T08:28:23.806 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.151+0000 7fa086ffd700 1 -- 172.21.15.86:0/3987315975 learned_addr learned my addr 172.21.15.86:0/3987315975 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:28:23.806 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.151+0000 7fa086ffd700 1 -- 172.21.15.86:0/3987315975 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa088107460 con 0x7fa088108800 2024-04-21T08:28:23.806 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.151+0000 7fa086ffd700 1 --2- 172.21.15.86:0/3987315975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa088108800 0x7fa088108c20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa07800bb40 tx=0x7fa07800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37aa0414173c262d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:23.806 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.151+0000 7fa085ffb700 1 -- 172.21.15.86:0/3987315975 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa07800dbe0 con 0x7fa088108800 2024-04-21T08:28:23.807 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.151+0000 7fa085ffb700 1 -- 172.21.15.86:0/3987315975 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa07800dd80 con 0x7fa088108800 2024-04-21T08:28:23.807 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.151+0000 7fa085ffb700 1 -- 172.21.15.86:0/3987315975 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa078004180 con 0x7fa088108800 2024-04-21T08:28:23.807 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.152+0000 7fa08d373700 1 -- 172.21.15.86:0/3987315975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa088108800 msgr2=0x7fa088108c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:23.807 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.152+0000 7fa08d373700 1 --2- 172.21.15.86:0/3987315975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa088108800 0x7fa088108c20 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7fa07800bb40 tx=0x7fa07800be50 comp rx=0 tx=0).stop 2024-04-21T08:28:23.807 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.152+0000 7fa08d373700 1 -- 172.21.15.86:0/3987315975 shutdown_connections 2024-04-21T08:28:23.807 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.152+0000 7fa08d373700 1 --2- 172.21.15.86:0/3987315975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa088108800 0x7fa088108c20 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:23.807 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.152+0000 7fa08d373700 1 -- 172.21.15.86:0/3987315975 >> 172.21.15.86:0/3987315975 conn(0x7fa088101f70 msgr2=0x7fa0881043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:23.807 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.152+0000 7fa08d373700 1 -- 172.21.15.86:0/3987315975 shutdown_connections 2024-04-21T08:28:23.807 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.152+0000 7fa08d373700 1 -- 172.21.15.86:0/3987315975 wait complete. 2024-04-21T08:28:23.807 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.153+0000 7fa08d373700 1 Processor -- start 2024-04-21T08:28:23.807 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.153+0000 7fa08d373700 1 -- start start 2024-04-21T08:28:23.807 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.153+0000 7fa08d373700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa088108800 0x7fa0881997d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:23.807 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.153+0000 7fa08d373700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0881078d0 con 0x7fa088108800 2024-04-21T08:28:23.807 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.153+0000 7fa086ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa088108800 0x7fa0881997d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:23.808 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.153+0000 7fa086ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa088108800 0x7fa0881997d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59762/0 (socket says 172.21.15.86:59762) 2024-04-21T08:28:23.808 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.153+0000 7fa086ffd700 1 -- 172.21.15.86:0/1918976145 learned_addr learned my addr 172.21.15.86:0/1918976145 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:28:23.808 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.153+0000 7fa086ffd700 1 -- 172.21.15.86:0/1918976145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa088199d10 con 0x7fa088108800 2024-04-21T08:28:23.808 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.154+0000 7fa086ffd700 1 --2- 172.21.15.86:0/1918976145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa088108800 0x7fa0881997d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa07800b340 tx=0x7fa078004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:23.808 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.154+0000 7fa06ffff700 1 -- 172.21.15.86:0/1918976145 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa078029070 con 0x7fa088108800 2024-04-21T08:28:23.808 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.154+0000 7fa06ffff700 1 -- 172.21.15.86:0/1918976145 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa078004030 con 0x7fa088108800 2024-04-21T08:28:23.808 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.154+0000 7fa08d373700 1 -- 172.21.15.86:0/1918976145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa088199fa0 con 0x7fa088108800 2024-04-21T08:28:23.808 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.154+0000 7fa06ffff700 1 -- 172.21.15.86:0/1918976145 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa078010470 con 0x7fa088108800 2024-04-21T08:28:23.808 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.154+0000 7fa08d373700 1 -- 172.21.15.86:0/1918976145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa08819cc90 con 0x7fa088108800 2024-04-21T08:28:23.808 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.154+0000 7fa06ffff700 1 -- 172.21.15.86:0/1918976145 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa07802c070 con 0x7fa088108800 2024-04-21T08:28:23.808 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.154+0000 7fa06ffff700 1 -- 172.21.15.86:0/1918976145 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa078021070 con 0x7fa088108800 2024-04-21T08:28:23.808 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.155+0000 7fa08d373700 1 -- 172.21.15.86:0/1918976145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0880640a0 con 0x7fa088108800 2024-04-21T08:28:23.808 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.157+0000 7fa06ffff700 1 -- 172.21.15.86:0/1918976145 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fa078033020 con 0x7fa088108800 2024-04-21T08:28:23.808 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.226+0000 7fa08d373700 1 -- 172.21.15.86:0/1918976145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status"} v 0) -- 0x7fa08802e170 con 0x7fa088108800 2024-04-21T08:28:23.809 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.226+0000 7fa06ffff700 1 -- 172.21.15.86:0/1918976145 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status"}]=0 v0) ==== 54+0+317 (secure 0 0 0) 0x7fa078015070 con 0x7fa088108800 2024-04-21T08:28:23.809 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.227+0000 7fa08d373700 1 -- 172.21.15.86:0/1918976145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa088108800 msgr2=0x7fa0881997d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:23.809 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.227+0000 7fa08d373700 1 --2- 172.21.15.86:0/1918976145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa088108800 0x7fa0881997d0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7fa07800b340 tx=0x7fa078004750 comp rx=0 tx=0).stop 2024-04-21T08:28:23.809 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.227+0000 7fa08d373700 1 -- 172.21.15.86:0/1918976145 shutdown_connections 2024-04-21T08:28:23.809 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.227+0000 7fa08d373700 1 --2- 172.21.15.86:0/1918976145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa088108800 0x7fa0881997d0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:23.809 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.227+0000 7fa08d373700 1 -- 172.21.15.86:0/1918976145 >> 172.21.15.86:0/1918976145 conn(0x7fa088101f70 msgr2=0x7fa0881904d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:23.809 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.227+0000 7fa08d373700 1 -- 172.21.15.86:0/1918976145 shutdown_connections 2024-04-21T08:28:23.809 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:23.227+0000 7fa08d373700 1 -- 172.21.15.86:0/1918976145 wait complete. 2024-04-21T08:28:23.809 INFO:teuthology.orchestra.run.smithi086.stdout:mon is available 2024-04-21T08:28:23.809 INFO:teuthology.orchestra.run.smithi086.stdout:Assimilating anything we can from ceph.conf... 2024-04-21T08:28:23.905 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:23 smithi086 ceph-mon[27570]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-21T08:28:23.906 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:23 smithi086 ceph-mon[27570]: monmap epoch 1 2024-04-21T08:28:23.906 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:23 smithi086 ceph-mon[27570]: fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:28:23.906 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:23 smithi086 ceph-mon[27570]: last_changed 2024-04-21T08:28:19.891280+0000 2024-04-21T08:28:23.906 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:23 smithi086 ceph-mon[27570]: created 2024-04-21T08:28:19.891280+0000 2024-04-21T08:28:23.906 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:23 smithi086 ceph-mon[27570]: min_mon_release 19 (squid) 2024-04-21T08:28:23.907 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:23 smithi086 ceph-mon[27570]: election_strategy: 1 2024-04-21T08:28:23.907 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:23 smithi086 ceph-mon[27570]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-21T08:28:23.907 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:23 smithi086 ceph-mon[27570]: fsmap 2024-04-21T08:28:23.907 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:23 smithi086 ceph-mon[27570]: osdmap e1: 0 total, 0 up, 0 in 2024-04-21T08:28:23.907 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:23 smithi086 ceph-mon[27570]: mgrmap e1: no daemons active 2024-04-21T08:28:23.907 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:23 smithi086 ceph-mon[27570]: from='client.? 172.21.15.86:0/1918976145' entity='client.admin' cmd=[{"prefix": "status"}]: dispatch 2024-04-21T08:28:24.841 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-21T08:28:24.842 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [global] 2024-04-21T08:28:24.842 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout fsid = f8cec7cc-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:28:24.842 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-21T08:28:24.842 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_host = [v2:172.21.15.86:3300,v1:172.21.15.86:6789] 2024-04-21T08:28:24.842 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-21T08:28:24.842 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-21T08:28:24.842 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-21T08:28:24.842 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-21T08:28:24.842 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-21T08:28:24.842 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-21T08:28:24.842 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-21T08:28:24.842 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-21T08:28:24.843 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [osd] 2024-04-21T08:28:24.843 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-21T08:28:24.843 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-21T08:28:24.843 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-21T08:28:24.843 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-21T08:28:24.843 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-21T08:28:24.843 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.191+0000 7fa884895700 1 Processor -- start 2024-04-21T08:28:24.843 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.191+0000 7fa884895700 1 -- start start 2024-04-21T08:28:24.843 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.191+0000 7fa884895700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa880108410 0x7fa880108830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:24.843 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.191+0000 7fa884895700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa880108e00 con 0x7fa880108410 2024-04-21T08:28:24.843 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.191+0000 7fa87e59c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa880108410 0x7fa880108830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:24.843 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.192+0000 7fa87e59c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa880108410 0x7fa880108830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59778/0 (socket says 172.21.15.86:59778) 2024-04-21T08:28:24.843 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.192+0000 7fa87e59c700 1 -- 172.21.15.86:0/299464212 learned_addr learned my addr 172.21.15.86:0/299464212 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:28:24.843 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.192+0000 7fa87e59c700 1 -- 172.21.15.86:0/299464212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa8801095c0 con 0x7fa880108410 2024-04-21T08:28:24.843 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.192+0000 7fa87e59c700 1 --2- 172.21.15.86:0/299464212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa880108410 0x7fa880108830 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa8700098a0 tx=0x7fa870009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55ae4e6676827b9d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:24.844 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.192+0000 7fa87d59a700 1 -- 172.21.15.86:0/299464212 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa870004160 con 0x7fa880108410 2024-04-21T08:28:24.844 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.192+0000 7fa87d59a700 1 -- 172.21.15.86:0/299464212 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa870004300 con 0x7fa880108410 2024-04-21T08:28:24.844 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.192+0000 7fa87d59a700 1 -- 172.21.15.86:0/299464212 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa8700036a0 con 0x7fa880108410 2024-04-21T08:28:24.844 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.193+0000 7fa884895700 1 -- 172.21.15.86:0/299464212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa880108410 msgr2=0x7fa880108830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:24.844 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.193+0000 7fa884895700 1 --2- 172.21.15.86:0/299464212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa880108410 0x7fa880108830 secure :-1 s=READY pgs=3 cs=0 l=1 rev1=1 crypto rx=0x7fa8700098a0 tx=0x7fa870009bb0 comp rx=0 tx=0).stop 2024-04-21T08:28:24.844 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.193+0000 7fa884895700 1 -- 172.21.15.86:0/299464212 shutdown_connections 2024-04-21T08:28:24.844 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.193+0000 7fa884895700 1 --2- 172.21.15.86:0/299464212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa880108410 0x7fa880108830 unknown :-1 s=CLOSED pgs=3 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:24.844 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.193+0000 7fa884895700 1 -- 172.21.15.86:0/299464212 >> 172.21.15.86:0/299464212 conn(0x7fa8801039b0 msgr2=0x7fa880105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:24.844 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.193+0000 7fa884895700 1 -- 172.21.15.86:0/299464212 shutdown_connections 2024-04-21T08:28:24.844 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.193+0000 7fa884895700 1 -- 172.21.15.86:0/299464212 wait complete. 2024-04-21T08:28:24.844 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.193+0000 7fa884895700 1 Processor -- start 2024-04-21T08:28:24.844 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.193+0000 7fa884895700 1 -- start start 2024-04-21T08:28:24.844 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.194+0000 7fa884895700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa880108410 0x7fa88019b970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:24.844 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.194+0000 7fa884895700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa880109af0 con 0x7fa880108410 2024-04-21T08:28:24.845 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.194+0000 7fa87e59c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa880108410 0x7fa88019b970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:24.845 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.194+0000 7fa87e59c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa880108410 0x7fa88019b970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59780/0 (socket says 172.21.15.86:59780) 2024-04-21T08:28:24.845 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.194+0000 7fa87e59c700 1 -- 172.21.15.86:0/3470580810 learned_addr learned my addr 172.21.15.86:0/3470580810 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:28:24.845 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.194+0000 7fa87e59c700 1 -- 172.21.15.86:0/3470580810 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa88019beb0 con 0x7fa880108410 2024-04-21T08:28:24.845 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.194+0000 7fa87e59c700 1 --2- 172.21.15.86:0/3470580810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa880108410 0x7fa88019b970 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa870000c00 tx=0x7fa870004750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:24.845 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.194+0000 7fa86f7fe700 1 -- 172.21.15.86:0/3470580810 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa870003db0 con 0x7fa880108410 2024-04-21T08:28:24.845 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.194+0000 7fa86f7fe700 1 -- 172.21.15.86:0/3470580810 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(0 keys) ==== 4+0+0 (secure 0 0 0) 0x7fa870011630 con 0x7fa880108410 2024-04-21T08:28:24.845 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.194+0000 7fa86f7fe700 1 -- 172.21.15.86:0/3470580810 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fa870011950 con 0x7fa880108410 2024-04-21T08:28:24.845 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.194+0000 7fa884895700 1 -- 172.21.15.86:0/3470580810 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa88019c140 con 0x7fa880108410 2024-04-21T08:28:24.845 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.194+0000 7fa884895700 1 -- 172.21.15.86:0/3470580810 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa88019ee30 con 0x7fa880108410 2024-04-21T08:28:24.845 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.195+0000 7fa86f7fe700 1 -- 172.21.15.86:0/3470580810 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7fa870011b80 con 0x7fa880108410 2024-04-21T08:28:24.845 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.195+0000 7fa86f7fe700 1 -- 172.21.15.86:0/3470580810 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fa87001c450 con 0x7fa880108410 2024-04-21T08:28:24.845 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.195+0000 7fa884895700 1 -- 172.21.15.86:0/3470580810 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa8800640a0 con 0x7fa880108410 2024-04-21T08:28:24.845 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.198+0000 7fa86f7fe700 1 -- 172.21.15.86:0/3470580810 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7fa870032020 con 0x7fa880108410 2024-04-21T08:28:24.846 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.264+0000 7fa884895700 1 -- 172.21.15.86:0/3470580810 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7fa88002e1b0 con 0x7fa880108410 2024-04-21T08:28:24.846 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.267+0000 7fa86f7fe700 1 -- 172.21.15.86:0/3470580810 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v2) ==== 70+0+465 (secure 0 0 0) 0x7fa870004260 con 0x7fa880108410 2024-04-21T08:28:24.846 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.268+0000 7fa86f7fe700 1 -- 172.21.15.86:0/3470580810 <== mon.0 v2:172.21.15.86:3300/0 8 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fa87001c650 con 0x7fa880108410 2024-04-21T08:28:24.846 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.269+0000 7fa884895700 1 -- 172.21.15.86:0/3470580810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa880108410 msgr2=0x7fa88019b970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:24.846 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.269+0000 7fa884895700 1 --2- 172.21.15.86:0/3470580810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa880108410 0x7fa88019b970 secure :-1 s=READY pgs=4 cs=0 l=1 rev1=1 crypto rx=0x7fa870000c00 tx=0x7fa870004750 comp rx=0 tx=0).stop 2024-04-21T08:28:24.846 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.269+0000 7fa884895700 1 -- 172.21.15.86:0/3470580810 shutdown_connections 2024-04-21T08:28:24.846 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.269+0000 7fa884895700 1 --2- 172.21.15.86:0/3470580810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa880108410 0x7fa88019b970 unknown :-1 s=CLOSED pgs=4 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:24.846 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.269+0000 7fa884895700 1 -- 172.21.15.86:0/3470580810 >> 172.21.15.86:0/3470580810 conn(0x7fa8801039b0 msgr2=0x7fa880192630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:24.846 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.269+0000 7fa884895700 1 -- 172.21.15.86:0/3470580810 shutdown_connections 2024-04-21T08:28:24.846 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:24.269+0000 7fa884895700 1 -- 172.21.15.86:0/3470580810 wait complete. 2024-04-21T08:28:24.846 INFO:teuthology.orchestra.run.smithi086.stdout:Generating new minimal ceph.conf... 2024-04-21T08:28:24.958 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:24 smithi086 ceph-mon[27570]: from='client.? 172.21.15.86:0/3470580810' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-21T08:28:24.958 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:24 smithi086 ceph-mon[27570]: from='client.? 172.21.15.86:0/3470580810' entity='client.admin' cmd='[{"prefix": "config assimilate-conf"}]': finished 2024-04-21T08:28:26.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:25 smithi086 ceph-mon[27570]: from='client.? 172.21.15.86:0/2431564519' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:28:31.908 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.538+0000 7f6eeacfe700 1 Processor -- start 2024-04-21T08:28:31.914 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.538+0000 7f6eeacfe700 1 -- start start 2024-04-21T08:28:31.914 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.538+0000 7f6eeacfe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ee4107310 0x7f6ee4107730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:31.914 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.538+0000 7f6eeacfe700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee4107d00 con 0x7f6ee4107310 2024-04-21T08:28:31.914 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.538+0000 7f6ee8a9a700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ee4107310 0x7f6ee4107730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:31.914 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.538+0000 7f6ee8a9a700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ee4107310 0x7f6ee4107730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59796/0 (socket says 172.21.15.86:59796) 2024-04-21T08:28:31.915 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.538+0000 7f6ee8a9a700 1 -- 172.21.15.86:0/1084725125 learned_addr learned my addr 172.21.15.86:0/1084725125 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:28:31.915 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.538+0000 7f6ee8a9a700 1 -- 172.21.15.86:0/1084725125 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee4108510 con 0x7f6ee4107310 2024-04-21T08:28:31.915 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.538+0000 7f6ee8a9a700 1 --2- 172.21.15.86:0/1084725125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ee4107310 0x7f6ee4107730 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80098a0 tx=0x7f6ed8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1263538fa478ea7e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:31.915 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.539+0000 7f6ee37fe700 1 -- 172.21.15.86:0/1084725125 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ed8004160 con 0x7f6ee4107310 2024-04-21T08:28:31.915 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.539+0000 7f6ee37fe700 1 -- 172.21.15.86:0/1084725125 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6ed8004300 con 0x7f6ee4107310 2024-04-21T08:28:31.915 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.539+0000 7f6ee37fe700 1 -- 172.21.15.86:0/1084725125 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ed8011af0 con 0x7f6ee4107310 2024-04-21T08:28:31.915 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.539+0000 7f6eeacfe700 1 -- 172.21.15.86:0/1084725125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ee4107310 msgr2=0x7f6ee4107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:31.915 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.539+0000 7f6eeacfe700 1 --2- 172.21.15.86:0/1084725125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ee4107310 0x7f6ee4107730 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80098a0 tx=0x7f6ed8009bb0 comp rx=0 tx=0).stop 2024-04-21T08:28:31.915 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.539+0000 7f6eeacfe700 1 -- 172.21.15.86:0/1084725125 shutdown_connections 2024-04-21T08:28:31.915 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.539+0000 7f6eeacfe700 1 --2- 172.21.15.86:0/1084725125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ee4107310 0x7f6ee4107730 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:31.915 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.539+0000 7f6eeacfe700 1 -- 172.21.15.86:0/1084725125 >> 172.21.15.86:0/1084725125 conn(0x7f6ee41028b0 msgr2=0x7f6ee4104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:31.915 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.539+0000 7f6eeacfe700 1 -- 172.21.15.86:0/1084725125 shutdown_connections 2024-04-21T08:28:31.915 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.539+0000 7f6eeacfe700 1 -- 172.21.15.86:0/1084725125 wait complete. 2024-04-21T08:28:31.916 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.540+0000 7f6eeacfe700 1 Processor -- start 2024-04-21T08:28:31.916 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.540+0000 7f6eeacfe700 1 -- start start 2024-04-21T08:28:31.916 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.540+0000 7f6eeacfe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ee4107310 0x7f6ee419c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:31.916 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.540+0000 7f6eeacfe700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6ee410b740 con 0x7f6ee4107310 2024-04-21T08:28:31.916 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.540+0000 7f6ee8a9a700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ee4107310 0x7f6ee419c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:31.916 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.540+0000 7f6ee8a9a700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ee4107310 0x7f6ee419c2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59800/0 (socket says 172.21.15.86:59800) 2024-04-21T08:28:31.916 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.540+0000 7f6ee8a9a700 1 -- 172.21.15.86:0/2431564519 learned_addr learned my addr 172.21.15.86:0/2431564519 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:28:31.916 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.540+0000 7f6ee8a9a700 1 -- 172.21.15.86:0/2431564519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ee419c7e0 con 0x7f6ee4107310 2024-04-21T08:28:31.916 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.540+0000 7f6ee8a9a700 1 --2- 172.21.15.86:0/2431564519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ee4107310 0x7f6ee419c2a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80117b0 tx=0x7f6ed8011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:31.916 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.541+0000 7f6ee1ffb700 1 -- 172.21.15.86:0/2431564519 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ed8003af0 con 0x7f6ee4107310 2024-04-21T08:28:31.916 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.541+0000 7f6ee1ffb700 1 -- 172.21.15.86:0/2431564519 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6ed8020070 con 0x7f6ee4107310 2024-04-21T08:28:31.916 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.541+0000 7f6eeacfe700 1 -- 172.21.15.86:0/2431564519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ee419ca70 con 0x7f6ee4107310 2024-04-21T08:28:31.916 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.541+0000 7f6ee1ffb700 1 -- 172.21.15.86:0/2431564519 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6ed8019630 con 0x7f6ee4107310 2024-04-21T08:28:31.916 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.541+0000 7f6eeacfe700 1 -- 172.21.15.86:0/2431564519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ee419f760 con 0x7f6ee4107310 2024-04-21T08:28:31.917 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.541+0000 7f6ee1ffb700 1 -- 172.21.15.86:0/2431564519 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f6ed8019c80 con 0x7f6ee4107310 2024-04-21T08:28:31.917 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.541+0000 7f6ee1ffb700 1 -- 172.21.15.86:0/2431564519 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f6ed800b720 con 0x7f6ee4107310 2024-04-21T08:28:31.917 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.542+0000 7f6eeacfe700 1 -- 172.21.15.86:0/2431564519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ee410b8c0 con 0x7f6ee4107310 2024-04-21T08:28:31.917 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.544+0000 7f6ee1ffb700 1 -- 172.21.15.86:0/2431564519 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f6ed80042a0 con 0x7f6ee4107310 2024-04-21T08:28:31.917 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.612+0000 7f6eeacfe700 1 -- 172.21.15.86:0/2431564519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7f6ee402e0a0 con 0x7f6ee4107310 2024-04-21T08:28:31.917 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.613+0000 7f6ee1ffb700 1 -- 172.21.15.86:0/2431564519 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v2) ==== 76+0+175 (secure 0 0 0) 0x7f6ed801a9c0 con 0x7f6ee4107310 2024-04-21T08:28:31.917 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.613+0000 7f6eeacfe700 1 -- 172.21.15.86:0/2431564519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ee4107310 msgr2=0x7f6ee419c2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:31.917 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.613+0000 7f6eeacfe700 1 --2- 172.21.15.86:0/2431564519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ee4107310 0x7f6ee419c2a0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f6ed80117b0 tx=0x7f6ed8011b70 comp rx=0 tx=0).stop 2024-04-21T08:28:31.917 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.613+0000 7f6eeacfe700 1 -- 172.21.15.86:0/2431564519 shutdown_connections 2024-04-21T08:28:31.917 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.613+0000 7f6eeacfe700 1 --2- 172.21.15.86:0/2431564519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ee4107310 0x7f6ee419c2a0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:31.917 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.613+0000 7f6eeacfe700 1 -- 172.21.15.86:0/2431564519 >> 172.21.15.86:0/2431564519 conn(0x7f6ee41028b0 msgr2=0x7f6ee4103700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:31.917 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.613+0000 7f6eeacfe700 1 -- 172.21.15.86:0/2431564519 shutdown_connections 2024-04-21T08:28:31.917 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:25.613+0000 7f6eeacfe700 1 -- 172.21.15.86:0/2431564519 wait complete. 2024-04-21T08:28:31.917 INFO:teuthology.orchestra.run.smithi086.stdout:Restarting the monitor... 2024-04-21T08:28:32.339 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:31 smithi086 systemd[1]: Stopping Ceph mon.a for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:28:33.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:33 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mon-a[27566]: 2024-04-21T08:28:33.243+0000 7f5c521c2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:28:33.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:33 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mon-a[27566]: 2024-04-21T08:28:33.243+0000 7f5c521c2700 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2024-04-21T08:28:33.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:33 smithi086 podman[27813]: 2024-04-21 08:28:33.304983382 +0000 UTC m=+1.225007113 container died 81099027997be48f0ea6a63052ba0e2c1529ea314ab487c067c3adb10a186c78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mon-a, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-21T08:28:34.810 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:34 smithi086 podman[27813]: 2024-04-21 08:28:34.567423582 +0000 UTC m=+2.487447320 container cleanup 81099027997be48f0ea6a63052ba0e2c1529ea314ab487c067c3adb10a186c78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mon-a, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-21T08:28:34.811 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:34 smithi086 bash[27813]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mon-a 2024-04-21T08:28:35.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:34 smithi086 podman[27828]: 2024-04-21 08:28:34.812291438 +0000 UTC m=+1.503923069 container remove 81099027997be48f0ea6a63052ba0e2c1529ea314ab487c067c3adb10a186c78 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mon-a, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD) 2024-04-21T08:28:35.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:34 smithi086 systemd[1]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mon.a.service: Deactivated successfully. 2024-04-21T08:28:35.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:34 smithi086 systemd[1]: Stopped Ceph mon.a for f8cec7cc-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:28:35.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:34 smithi086 systemd[1]: Starting Ceph mon.a for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:28:35.372 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 podman[27920]: 2024-04-21 08:28:35.121339446 +0000 UTC m=+0.090528669 container create f70ad6da7b69cfc5da26ec15d10a6252045d3822626a62d03d5308d8f5a73090 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mon-a, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:28:35.373 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 podman[27920]: 2024-04-21 08:28:35.048759102 +0000 UTC m=+0.017948281 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e 2024-04-21T08:28:35.373 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 podman[27920]: 2024-04-21 08:28:35.298566445 +0000 UTC m=+0.267755623 container init f70ad6da7b69cfc5da26ec15d10a6252045d3822626a62d03d5308d8f5a73090 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mon-a, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-21T08:28:35.373 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 podman[27920]: 2024-04-21 08:28:35.301389669 +0000 UTC m=+0.270578856 container start f70ad6da7b69cfc5da26ec15d10a6252045d3822626a62d03d5308d8f5a73090 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mon-a, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True) 2024-04-21T08:28:35.373 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: set uid:gid to 167:167 (ceph:ceph) 2024-04-21T08:28:35.373 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev), process ceph-mon, pid 2 2024-04-21T08:28:35.373 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: pidfile_write: ignore empty --pid-file 2024-04-21T08:28:35.373 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: load: jerasure load: lrc 2024-04-21T08:28:35.373 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: RocksDB version: 7.9.2 2024-04-21T08:28:35.373 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Git sha 0 2024-04-21T08:28:35.373 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Compile date 2024-04-19 14:19:22 2024-04-21T08:28:35.373 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: DB SUMMARY 2024-04-21T08:28:35.373 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: DB Session ID: 48FKKKF12EN45K5G5B53 2024-04-21T08:28:35.374 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: CURRENT file: CURRENT 2024-04-21T08:28:35.374 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: IDENTITY file: IDENTITY 2024-04-21T08:28:35.374 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: MANIFEST file: MANIFEST-000010 size: 179 Bytes 2024-04-21T08:28:35.374 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: SST files in /var/lib/ceph/mon/ceph-a/store.db dir, Total Num: 1, files: 000008.sst 2024-04-21T08:28:35.374 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-a/store.db: 000009.log size: 104526 ; 2024-04-21T08:28:35.374 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.error_if_exists: 0 2024-04-21T08:28:35.374 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.create_if_missing: 0 2024-04-21T08:28:35.374 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.paranoid_checks: 1 2024-04-21T08:28:35.374 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-21T08:28:35.374 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-21T08:28:35.374 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-21T08:28:35.374 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.env: 0x558de85ddcc0 2024-04-21T08:28:35.374 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.fs: PosixFileSystem 2024-04-21T08:28:35.374 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.info_log: 0x558dea543be0 2024-04-21T08:28:35.375 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.max_file_opening_threads: 16 2024-04-21T08:28:35.375 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.statistics: (nil) 2024-04-21T08:28:35.375 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.use_fsync: 0 2024-04-21T08:28:35.375 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.max_log_file_size: 0 2024-04-21T08:28:35.375 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-21T08:28:35.375 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-21T08:28:35.375 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.keep_log_file_num: 1000 2024-04-21T08:28:35.375 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.recycle_log_file_num: 0 2024-04-21T08:28:35.375 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.allow_fallocate: 1 2024-04-21T08:28:35.375 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.allow_mmap_reads: 0 2024-04-21T08:28:35.375 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.allow_mmap_writes: 0 2024-04-21T08:28:35.375 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.use_direct_reads: 0 2024-04-21T08:28:35.375 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-21T08:28:35.375 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.create_missing_column_families: 0 2024-04-21T08:28:35.375 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.db_log_dir: 2024-04-21T08:28:35.376 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.wal_dir: 2024-04-21T08:28:35.376 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-21T08:28:35.376 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-21T08:28:35.376 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-21T08:28:35.376 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-21T08:28:35.376 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-21T08:28:35.376 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-21T08:28:35.376 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.advise_random_on_open: 1 2024-04-21T08:28:35.376 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.db_write_buffer_size: 0 2024-04-21T08:28:35.376 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.write_buffer_manager: 0x558dea4d70e0 2024-04-21T08:28:35.376 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-21T08:28:35.376 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-21T08:28:35.376 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-21T08:28:35.376 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.rate_limiter: (nil) 2024-04-21T08:28:35.376 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-21T08:28:35.378 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.wal_recovery_mode: 2 2024-04-21T08:28:35.378 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.enable_thread_tracking: 0 2024-04-21T08:28:35.378 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.enable_pipelined_write: 0 2024-04-21T08:28:35.378 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.unordered_write: 0 2024-04-21T08:28:35.378 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-21T08:28:35.378 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-21T08:28:35.378 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-21T08:28:35.378 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-21T08:28:35.378 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.row_cache: None 2024-04-21T08:28:35.379 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.wal_filter: None 2024-04-21T08:28:35.379 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-21T08:28:35.379 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.allow_ingest_behind: 0 2024-04-21T08:28:35.379 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.two_write_queues: 0 2024-04-21T08:28:35.379 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.manual_wal_flush: 0 2024-04-21T08:28:35.379 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.wal_compression: 0 2024-04-21T08:28:35.379 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.atomic_flush: 0 2024-04-21T08:28:35.379 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-21T08:28:35.379 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-21T08:28:35.379 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-21T08:28:35.379 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.log_readahead_size: 0 2024-04-21T08:28:35.379 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-21T08:28:35.379 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.best_efforts_recovery: 0 2024-04-21T08:28:35.379 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-21T08:28:35.379 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-21T08:28:35.380 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.allow_data_in_errors: 0 2024-04-21T08:28:35.380 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.db_host_id: __hostname__ 2024-04-21T08:28:35.380 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-21T08:28:35.380 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.max_background_jobs: 2 2024-04-21T08:28:35.380 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.max_background_compactions: -1 2024-04-21T08:28:35.380 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.max_subcompactions: 1 2024-04-21T08:28:35.380 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-21T08:28:35.380 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-21T08:28:35.380 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-21T08:28:35.380 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.max_total_wal_size: 0 2024-04-21T08:28:35.380 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-21T08:28:35.380 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-21T08:28:35.380 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-21T08:28:35.380 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-21T08:28:35.381 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.max_open_files: -1 2024-04-21T08:28:35.381 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.bytes_per_sync: 0 2024-04-21T08:28:35.381 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-21T08:28:35.381 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-21T08:28:35.381 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.compaction_readahead_size: 0 2024-04-21T08:28:35.381 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.max_background_flushes: -1 2024-04-21T08:28:35.381 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Compression algorithms supported: 2024-04-21T08:28:35.381 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-21T08:28:35.381 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: kZSTD supported: 0 2024-04-21T08:28:35.381 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: kXpressCompression supported: 0 2024-04-21T08:28:35.381 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: kLZ4HCCompression supported: 1 2024-04-21T08:28:35.381 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: kZlibCompression supported: 1 2024-04-21T08:28:35.381 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: kSnappyCompression supported: 1 2024-04-21T08:28:35.381 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: kLZ4Compression supported: 1 2024-04-21T08:28:35.382 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: kBZip2Compression supported: 0 2024-04-21T08:28:35.382 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-21T08:28:35.382 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-21T08:28:35.382 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 2024-04-21T08:28:35.382 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-21T08:28:35.382 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-21T08:28:35.382 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.merge_operator: 2024-04-21T08:28:35.382 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.compaction_filter: None 2024-04-21T08:28:35.382 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.compaction_filter_factory: None 2024-04-21T08:28:35.382 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.sst_partitioner_factory: None 2024-04-21T08:28:35.382 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-21T08:28:35.382 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-21T08:28:35.382 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x558dea543dc0) 2024-04-21T08:28:35.382 INFO:journalctl@ceph.mon.a.smithi086.stdout: cache_index_and_filter_blocks: 1 2024-04-21T08:28:35.383 INFO:journalctl@ceph.mon.a.smithi086.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-21T08:28:35.383 INFO:journalctl@ceph.mon.a.smithi086.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-21T08:28:35.383 INFO:journalctl@ceph.mon.a.smithi086.stdout: pin_top_level_index_and_filter: 1 2024-04-21T08:28:35.383 INFO:journalctl@ceph.mon.a.smithi086.stdout: index_type: 0 2024-04-21T08:28:35.383 INFO:journalctl@ceph.mon.a.smithi086.stdout: data_block_index_type: 0 2024-04-21T08:28:35.383 INFO:journalctl@ceph.mon.a.smithi086.stdout: index_shortening: 1 2024-04-21T08:28:35.383 INFO:journalctl@ceph.mon.a.smithi086.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-21T08:28:35.383 INFO:journalctl@ceph.mon.a.smithi086.stdout: checksum: 4 2024-04-21T08:28:35.383 INFO:journalctl@ceph.mon.a.smithi086.stdout: no_block_cache: 0 2024-04-21T08:28:35.383 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_cache: 0x558dea5091f0 2024-04-21T08:28:35.383 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_cache_name: BinnedLRUCache 2024-04-21T08:28:35.383 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_cache_options: 2024-04-21T08:28:35.383 INFO:journalctl@ceph.mon.a.smithi086.stdout: capacity : 536870912 2024-04-21T08:28:35.383 INFO:journalctl@ceph.mon.a.smithi086.stdout: num_shard_bits : 4 2024-04-21T08:28:35.383 INFO:journalctl@ceph.mon.a.smithi086.stdout: strict_capacity_limit : 0 2024-04-21T08:28:35.384 INFO:journalctl@ceph.mon.a.smithi086.stdout: high_pri_pool_ratio: 0.000 2024-04-21T08:28:35.384 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_cache_compressed: (nil) 2024-04-21T08:28:35.384 INFO:journalctl@ceph.mon.a.smithi086.stdout: persistent_cache: (nil) 2024-04-21T08:28:35.384 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_size: 4096 2024-04-21T08:28:35.384 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_size_deviation: 10 2024-04-21T08:28:35.384 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_restart_interval: 16 2024-04-21T08:28:35.384 INFO:journalctl@ceph.mon.a.smithi086.stdout: index_block_restart_interval: 1 2024-04-21T08:28:35.384 INFO:journalctl@ceph.mon.a.smithi086.stdout: metadata_block_size: 4096 2024-04-21T08:28:35.384 INFO:journalctl@ceph.mon.a.smithi086.stdout: partition_filters: 0 2024-04-21T08:28:35.384 INFO:journalctl@ceph.mon.a.smithi086.stdout: use_delta_encoding: 1 2024-04-21T08:28:35.384 INFO:journalctl@ceph.mon.a.smithi086.stdout: filter_policy: bloomfilter 2024-04-21T08:28:35.384 INFO:journalctl@ceph.mon.a.smithi086.stdout: whole_key_filtering: 1 2024-04-21T08:28:35.384 INFO:journalctl@ceph.mon.a.smithi086.stdout: verify_compression: 0 2024-04-21T08:28:35.384 INFO:journalctl@ceph.mon.a.smithi086.stdout: read_amp_bytes_per_bit: 0 2024-04-21T08:28:35.385 INFO:journalctl@ceph.mon.a.smithi086.stdout: format_version: 5 2024-04-21T08:28:35.385 INFO:journalctl@ceph.mon.a.smithi086.stdout: enable_index_compression: 1 2024-04-21T08:28:35.385 INFO:journalctl@ceph.mon.a.smithi086.stdout: block_align: 0 2024-04-21T08:28:35.385 INFO:journalctl@ceph.mon.a.smithi086.stdout: max_auto_readahead_size: 262144 2024-04-21T08:28:35.385 INFO:journalctl@ceph.mon.a.smithi086.stdout: prepopulate_block_cache: 0 2024-04-21T08:28:35.385 INFO:journalctl@ceph.mon.a.smithi086.stdout: initial_auto_readahead_size: 8192 2024-04-21T08:28:35.385 INFO:journalctl@ceph.mon.a.smithi086.stdout: num_file_reads_for_auto_readahead: 2 2024-04-21T08:28:35.385 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.write_buffer_size: 33554432 2024-04-21T08:28:35.385 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.max_write_buffer_number: 2 2024-04-21T08:28:35.386 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.compression: NoCompression 2024-04-21T08:28:35.386 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.bottommost_compression: Disabled 2024-04-21T08:28:35.386 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.prefix_extractor: nullptr 2024-04-21T08:28:35.386 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-21T08:28:35.386 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.num_levels: 7 2024-04-21T08:28:35.386 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-21T08:28:35.386 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-21T08:28:35.386 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-21T08:28:35.386 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-21T08:28:35.386 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-21T08:28:35.387 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-21T08:28:35.387 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-21T08:28:35.387 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-21T08:28:35.387 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-21T08:28:35.387 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-21T08:28:35.387 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-21T08:28:35.387 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-21T08:28:35.387 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-21T08:28:35.387 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.compression_opts.level: 32767 2024-04-21T08:28:35.387 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.compression_opts.strategy: 0 2024-04-21T08:28:35.387 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-21T08:28:35.387 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-21T08:28:35.387 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-21T08:28:35.387 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-21T08:28:35.387 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.compression_opts.enabled: false 2024-04-21T08:28:35.388 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-21T08:28:35.388 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-21T08:28:35.388 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-21T08:28:35.388 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-21T08:28:35.388 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.target_file_size_base: 67108864 2024-04-21T08:28:35.388 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-21T08:28:35.388 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-21T08:28:35.388 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-21T08:28:35.388 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-21T08:28:35.388 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-21T08:28:35.388 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-21T08:28:35.388 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-21T08:28:35.388 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-21T08:28:35.388 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-21T08:28:35.389 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-21T08:28:35.389 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-21T08:28:35.389 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-21T08:28:35.389 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-21T08:28:35.389 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-21T08:28:35.389 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.arena_block_size: 1048576 2024-04-21T08:28:35.389 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-21T08:28:35.389 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-21T08:28:35.389 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.disable_auto_compactions: 0 2024-04-21T08:28:35.389 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-21T08:28:35.389 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-21T08:28:35.389 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-21T08:28:35.389 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-21T08:28:35.389 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-21T08:28:35.389 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-21T08:28:35.390 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-21T08:28:35.390 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-21T08:28:35.390 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-21T08:28:35.390 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-21T08:28:35.390 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-21T08:28:35.390 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.inplace_update_support: 0 2024-04-21T08:28:35.390 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-21T08:28:35.390 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-21T08:28:35.390 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-21T08:28:35.391 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-21T08:28:35.391 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.bloom_locality: 0 2024-04-21T08:28:35.391 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.max_successive_merges: 0 2024-04-21T08:28:35.391 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-21T08:28:35.391 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.paranoid_file_checks: 0 2024-04-21T08:28:35.391 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.force_consistency_checks: 1 2024-04-21T08:28:35.391 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.report_bg_io_stats: 0 2024-04-21T08:28:35.391 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.ttl: 2592000 2024-04-21T08:28:35.391 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-21T08:28:35.391 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-21T08:28:35.391 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-21T08:28:35.391 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.enable_blob_files: false 2024-04-21T08:28:35.391 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.min_blob_size: 0 2024-04-21T08:28:35.391 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.blob_file_size: 268435456 2024-04-21T08:28:35.392 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-21T08:28:35.392 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-21T08:28:35.392 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-21T08:28:35.392 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-21T08:28:35.392 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-21T08:28:35.392 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.blob_file_starting_level: 0 2024-04-21T08:28:35.392 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-21T08:28:35.392 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-a/store.db/MANIFEST-000010 succeeded,manifest_file_number is 10, next_file_number is 12, last_sequence is 5, log_number is 5,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 5 2024-04-21T08:28:35.392 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 5 2024-04-21T08:28:35.392 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: 7208e148-466c-4821-ad44-60200c2f2af6 2024-04-21T08:28:35.392 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713688115329842, "job": 1, "event": "recovery_started", "wal_files": [9]} 2024-04-21T08:28:35.392 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #9 mode 2 2024-04-21T08:28:35.392 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713688115330861, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 13, "file_size": 99747, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 8, "largest_seqno": 324, "table_properties": {"data_size": 97629, "index_size": 310, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 773, "raw_key_size": 14030, "raw_average_key_size": 49, "raw_value_size": 89991, "raw_average_value_size": 317, "num_data_blocks": 12, "num_entries": 283, "num_filter_entries": 283, "num_deletions": 3, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713688115, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "7208e148-466c-4821-ad44-60200c2f2af6", "db_session_id": "48FKKKF12EN45K5G5B53", "orig_file_number": 13, "seqno_to_time_mapping": "N/A"}} 2024-04-21T08:28:35.392 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713688115330944, "job": 1, "event": "recovery_finished"} 2024-04-21T08:28:35.393 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: [db/version_set.cc:5047] Creating manifest 15 2024-04-21T08:28:35.393 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-a/store.db/000009.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-21T08:28:35.393 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x558dea61e000 2024-04-21T08:28:35.393 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: DB pointer 0x558dea608000 2024-04-21T08:28:35.393 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-21T08:28:35.393 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-21T08:28:35.393 INFO:journalctl@ceph.mon.a.smithi086.stdout: ** DB Stats ** 2024-04-21T08:28:35.393 INFO:journalctl@ceph.mon.a.smithi086.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-21T08:28:35.393 INFO:journalctl@ceph.mon.a.smithi086.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-21T08:28:35.393 INFO:journalctl@ceph.mon.a.smithi086.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-21T08:28:35.393 INFO:journalctl@ceph.mon.a.smithi086.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-21T08:28:35.393 INFO:journalctl@ceph.mon.a.smithi086.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-21T08:28:35.393 INFO:journalctl@ceph.mon.a.smithi086.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-21T08:28:35.393 INFO:journalctl@ceph.mon.a.smithi086.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-21T08:28:35.394 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-04-21T08:28:35.394 INFO:journalctl@ceph.mon.a.smithi086.stdout: ** Compaction Stats [default] ** 2024-04-21T08:28:35.394 INFO:journalctl@ceph.mon.a.smithi086.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-21T08:28:35.394 INFO:journalctl@ceph.mon.a.smithi086.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-21T08:28:35.394 INFO:journalctl@ceph.mon.a.smithi086.stdout: L0 2/0 99.27 KB 0.5 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 145.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-21T08:28:35.394 INFO:journalctl@ceph.mon.a.smithi086.stdout: Sum 2/0 99.27 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 145.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-21T08:28:35.394 INFO:journalctl@ceph.mon.a.smithi086.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 145.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-21T08:28:35.394 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-04-21T08:28:35.394 INFO:journalctl@ceph.mon.a.smithi086.stdout: ** Compaction Stats [default] ** 2024-04-21T08:28:35.394 INFO:journalctl@ceph.mon.a.smithi086.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-21T08:28:35.394 INFO:journalctl@ceph.mon.a.smithi086.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-21T08:28:35.394 INFO:journalctl@ceph.mon.a.smithi086.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 145.5 0.00 0.00 1 0.001 0 0 0.0 0.0 2024-04-21T08:28:35.394 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-04-21T08:28:35.395 INFO:journalctl@ceph.mon.a.smithi086.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-21T08:28:35.395 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-04-21T08:28:35.395 INFO:journalctl@ceph.mon.a.smithi086.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-21T08:28:35.395 INFO:journalctl@ceph.mon.a.smithi086.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-21T08:28:35.395 INFO:journalctl@ceph.mon.a.smithi086.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-21T08:28:35.395 INFO:journalctl@ceph.mon.a.smithi086.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-21T08:28:35.395 INFO:journalctl@ceph.mon.a.smithi086.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-21T08:28:35.395 INFO:journalctl@ceph.mon.a.smithi086.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-21T08:28:35.395 INFO:journalctl@ceph.mon.a.smithi086.stdout: Cumulative compaction: 0.00 GB write, 31.22 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-21T08:28:35.395 INFO:journalctl@ceph.mon.a.smithi086.stdout: Interval compaction: 0.00 GB write, 31.22 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-21T08:28:35.395 INFO:journalctl@ceph.mon.a.smithi086.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-21T08:28:35.395 INFO:journalctl@ceph.mon.a.smithi086.stdout: Block cache BinnedLRUCache@0x558dea5091f0#2 capacity: 512.00 MB usage: 1.52 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 9e-06 secs_since: 0 2024-04-21T08:28:35.395 INFO:journalctl@ceph.mon.a.smithi086.stdout: Block cache entry stats(count,size,portion): FilterBlock(2,1.02 KB,0.000193715%) IndexBlock(2,0.50 KB,9.53674e-05%) Misc(1,0.00 KB,0%) 2024-04-21T08:28:35.396 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-04-21T08:28:35.396 INFO:journalctl@ceph.mon.a.smithi086.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-21T08:28:35.396 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: starting mon.a rank 0 at public addrs [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] at bind addrs [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon_data /var/lib/ceph/mon/ceph-a fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:28:35.396 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: mon.a@-1(???) e1 preinit fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:28:35.396 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: mon.a@-1(???).mds e1 new map 2024-04-21T08:28:35.396 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: mon.a@-1(???).mds e1 print_map 2024-04-21T08:28:35.396 INFO:journalctl@ceph.mon.a.smithi086.stdout: e1 2024-04-21T08:28:35.396 INFO:journalctl@ceph.mon.a.smithi086.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-21T08:28:35.396 INFO:journalctl@ceph.mon.a.smithi086.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-21T08:28:35.396 INFO:journalctl@ceph.mon.a.smithi086.stdout: legacy client fscid: -1 2024-04-21T08:28:35.396 INFO:journalctl@ceph.mon.a.smithi086.stdout: 2024-04-21T08:28:35.396 INFO:journalctl@ceph.mon.a.smithi086.stdout: No filesystems configured 2024-04-21T08:28:35.396 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: mon.a@-1(???).osd e1 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-21T08:28:35.396 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-21T08:28:35.397 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-21T08:28:35.397 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: mon.a@-1(???).osd e1 crush map has features 288514050185494528, adjusting msgr requires 2024-04-21T08:28:35.397 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: mon.a@-1(???).paxosservice(auth 1..2) refresh upgraded, format 0 -> 3 2024-04-21T08:28:35.397 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 bash[27920]: f70ad6da7b69cfc5da26ec15d10a6252045d3822626a62d03d5308d8f5a73090 2024-04-21T08:28:35.397 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 systemd[1]: Started Ceph mon.a for f8cec7cc-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:28:35.397 INFO:teuthology.orchestra.run.smithi086.stdout:Setting public_network to 172.21.0.0/20 in mon config section 2024-04-21T08:28:35.675 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: mon.a is new leader, mons a in quorum (ranks 0) 2024-04-21T08:28:35.676 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: monmap epoch 1 2024-04-21T08:28:35.676 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:28:35.676 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: last_changed 2024-04-21T08:28:19.891280+0000 2024-04-21T08:28:35.676 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: created 2024-04-21T08:28:19.891280+0000 2024-04-21T08:28:35.676 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: min_mon_release 19 (squid) 2024-04-21T08:28:35.676 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: election_strategy: 1 2024-04-21T08:28:35.677 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-21T08:28:35.677 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: fsmap 2024-04-21T08:28:35.677 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: osdmap e1: 0 total, 0 up, 0 in 2024-04-21T08:28:35.677 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:35 smithi086 ceph-mon[27939]: mgrmap e1: no daemons active 2024-04-21T08:28:36.397 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.777+0000 7f45af59e700 1 Processor -- start 2024-04-21T08:28:36.404 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.777+0000 7f45af59e700 1 -- start start 2024-04-21T08:28:36.404 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.777+0000 7f45af59e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b0106b20 0x7f45b0106f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:36.404 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.777+0000 7f45af59e700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f45b007be20 con 0x7f45b0106b20 2024-04-21T08:28:36.404 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.777+0000 7f45ae59c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b0106b20 0x7f45b0106f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:36.405 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.777+0000 7f45ae59c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b0106b20 0x7f45b0106f40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50934/0 (socket says 172.21.15.86:50934) 2024-04-21T08:28:36.405 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.777+0000 7f45ae59c700 1 -- 172.21.15.86:0/2148679924 learned_addr learned my addr 172.21.15.86:0/2148679924 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:28:36.405 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.778+0000 7f45ae59c700 1 -- 172.21.15.86:0/2148679924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b0107480 con 0x7f45b0106b20 2024-04-21T08:28:36.405 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.778+0000 7f45ae59c700 1 --2- 172.21.15.86:0/2148679924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b0106b20 0x7f45b0106f40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f45a00098a0 tx=0x7f45a0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=685689c8718720ba server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:36.405 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.778+0000 7f45ad59a700 1 -- 172.21.15.86:0/2148679924 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45a0004160 con 0x7f45b0106b20 2024-04-21T08:28:36.406 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.778+0000 7f45ad59a700 1 -- 172.21.15.86:0/2148679924 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f45a0004300 con 0x7f45b0106b20 2024-04-21T08:28:36.406 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.778+0000 7f45ad59a700 1 -- 172.21.15.86:0/2148679924 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45a0011af0 con 0x7f45b0106b20 2024-04-21T08:28:36.406 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.779+0000 7f45af59e700 1 -- 172.21.15.86:0/2148679924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b0106b20 msgr2=0x7f45b0106f40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:36.406 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.779+0000 7f45af59e700 1 --2- 172.21.15.86:0/2148679924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b0106b20 0x7f45b0106f40 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f45a00098a0 tx=0x7f45a0009bb0 comp rx=0 tx=0).stop 2024-04-21T08:28:36.406 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.779+0000 7f45af59e700 1 -- 172.21.15.86:0/2148679924 shutdown_connections 2024-04-21T08:28:36.406 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.779+0000 7f45af59e700 1 --2- 172.21.15.86:0/2148679924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b0106b20 0x7f45b0106f40 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:36.406 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.779+0000 7f45af59e700 1 -- 172.21.15.86:0/2148679924 >> 172.21.15.86:0/2148679924 conn(0x7f45b01026e0 msgr2=0x7f45b0104b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:36.406 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.779+0000 7f45af59e700 1 -- 172.21.15.86:0/2148679924 shutdown_connections 2024-04-21T08:28:36.407 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.779+0000 7f45af59e700 1 -- 172.21.15.86:0/2148679924 wait complete. 2024-04-21T08:28:36.407 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.779+0000 7f45af59e700 1 Processor -- start 2024-04-21T08:28:36.407 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.780+0000 7f45af59e700 1 -- start start 2024-04-21T08:28:36.407 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.780+0000 7f45af59e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b0106b20 0x7f45b019e2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:36.407 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.780+0000 7f45af59e700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f45b0108440 con 0x7f45b0106b20 2024-04-21T08:28:36.407 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.780+0000 7f45ae59c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b0106b20 0x7f45b019e2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:36.407 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.780+0000 7f45ae59c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b0106b20 0x7f45b019e2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50950/0 (socket says 172.21.15.86:50950) 2024-04-21T08:28:36.407 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.780+0000 7f45ae59c700 1 -- 172.21.15.86:0/2005620182 learned_addr learned my addr 172.21.15.86:0/2005620182 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:28:36.407 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.780+0000 7f45ae59c700 1 -- 172.21.15.86:0/2005620182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45b019e7e0 con 0x7f45b0106b20 2024-04-21T08:28:36.408 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.780+0000 7f45ae59c700 1 --2- 172.21.15.86:0/2005620182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b0106b20 0x7f45b019e2a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f45a0009fb0 tx=0x7f45a0011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:36.408 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.780+0000 7f459f7fe700 1 -- 172.21.15.86:0/2005620182 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45a00036a0 con 0x7f45b0106b20 2024-04-21T08:28:36.408 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.780+0000 7f459f7fe700 1 -- 172.21.15.86:0/2005620182 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f45a001a430 con 0x7f45b0106b20 2024-04-21T08:28:36.408 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.780+0000 7f45af59e700 1 -- 172.21.15.86:0/2005620182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45b019ea70 con 0x7f45b0106b20 2024-04-21T08:28:36.408 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.780+0000 7f459f7fe700 1 -- 172.21.15.86:0/2005620182 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f45a0019550 con 0x7f45b0106b20 2024-04-21T08:28:36.408 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.780+0000 7f45af59e700 1 -- 172.21.15.86:0/2005620182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45b01a1760 con 0x7f45b0106b20 2024-04-21T08:28:36.408 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.781+0000 7f459f7fe700 1 -- 172.21.15.86:0/2005620182 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f45a0019be0 con 0x7f45b0106b20 2024-04-21T08:28:36.408 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.781+0000 7f459f7fe700 1 -- 172.21.15.86:0/2005620182 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f45a0024900 con 0x7f45b0106b20 2024-04-21T08:28:36.408 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.781+0000 7f45af59e700 1 -- 172.21.15.86:0/2005620182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45b0069b50 con 0x7f45b0106b20 2024-04-21T08:28:36.409 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.784+0000 7f459f7fe700 1 -- 172.21.15.86:0/2005620182 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f45a0043e50 con 0x7f45b0106b20 2024-04-21T08:28:36.409 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.851+0000 7f45af59e700 1 -- 172.21.15.86:0/2005620182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=public_network}] v 0) -- 0x7f45b002e1f0 con 0x7f45b0106b20 2024-04-21T08:28:36.409 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.853+0000 7f459f7fe700 1 -- 172.21.15.86:0/2005620182 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=public_network}]=0 v3)=0 v3) ==== 124+0+0 (secure 0 0 0) 0x7f45a0032020 con 0x7f45b0106b20 2024-04-21T08:28:36.409 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.854+0000 7f45af59e700 1 -- 172.21.15.86:0/2005620182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b0106b20 msgr2=0x7f45b019e2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:36.409 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.854+0000 7f45af59e700 1 --2- 172.21.15.86:0/2005620182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b0106b20 0x7f45b019e2a0 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f45a0009fb0 tx=0x7f45a0011940 comp rx=0 tx=0).stop 2024-04-21T08:28:36.409 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.854+0000 7f45af59e700 1 -- 172.21.15.86:0/2005620182 shutdown_connections 2024-04-21T08:28:36.409 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.854+0000 7f45af59e700 1 --2- 172.21.15.86:0/2005620182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45b0106b20 0x7f45b019e2a0 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:36.409 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.854+0000 7f45af59e700 1 -- 172.21.15.86:0/2005620182 >> 172.21.15.86:0/2005620182 conn(0x7f45b01026e0 msgr2=0x7f45b0194ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:36.409 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.854+0000 7f45af59e700 1 -- 172.21.15.86:0/2005620182 shutdown_connections 2024-04-21T08:28:36.410 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:35.854+0000 7f45af59e700 1 -- 172.21.15.86:0/2005620182 wait complete. 2024-04-21T08:28:36.410 INFO:teuthology.orchestra.run.smithi086.stdout:Wrote config to /etc/ceph/ceph.conf 2024-04-21T08:28:36.427 INFO:teuthology.orchestra.run.smithi086.stdout:Wrote keyring to /etc/ceph/ceph.client.admin.keyring 2024-04-21T08:28:36.427 INFO:teuthology.orchestra.run.smithi086.stdout:Creating mgr... 2024-04-21T08:28:36.428 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying port 0.0.0.0:9283 ... 2024-04-21T08:28:36.428 INFO:teuthology.orchestra.run.smithi086.stdout:Verifying port 0.0.0.0:8765 ... 2024-04-21T08:28:36.658 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl reset-failed ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mgr.x 2024-04-21T08:28:36.658 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Failed to reset failed state of unit ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mgr.x.service: Unit ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mgr.x.service not loaded. 2024-04-21T08:28:36.802 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stderr Created symlink /etc/systemd/system/ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968.target.wants/ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mgr.x.service → /etc/systemd/system/ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@.service. 2024-04-21T08:28:37.021 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:36 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2005620182' entity='client.admin' 2024-04-21T08:28:37.307 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:37 smithi086 podman[28154]: 2024-04-21 08:28:37.227684914 +0000 UTC m=+0.303787303 container start bbba55e9c9c1b85cd1aec736267dea8abbc1129d1a742d308c5dbd82f4799707 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux ) 2024-04-21T08:28:37.322 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-21T08:28:37.322 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout disabled 2024-04-21T08:28:37.329 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-21T08:28:37.329 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-04-21T08:28:37.329 INFO:teuthology.orchestra.run.smithi086.stdout:firewalld.service is not enabled 2024-04-21T08:28:37.329 INFO:teuthology.orchestra.run.smithi086.stdout:Not possible to enable service . firewalld.service is not available 2024-04-21T08:28:37.337 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-21T08:28:37.337 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout disabled 2024-04-21T08:28:37.344 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-21T08:28:37.345 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-04-21T08:28:37.345 INFO:teuthology.orchestra.run.smithi086.stdout:firewalld.service is not enabled 2024-04-21T08:28:37.345 INFO:teuthology.orchestra.run.smithi086.stdout:Not possible to open ports <[9283, 8765]>. firewalld.service is not available 2024-04-21T08:28:37.345 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mgr to start... 2024-04-21T08:28:37.345 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mgr... 2024-04-21T08:28:37.564 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:37 smithi086 bash[28154]: bbba55e9c9c1b85cd1aec736267dea8abbc1129d1a742d308c5dbd82f4799707 2024-04-21T08:28:37.564 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:37 smithi086 systemd[1]: Started Ceph mgr.x for f8cec7cc-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:28:37.564 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:37 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:37.379+0000 7fe3c346c200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-21T08:28:37.564 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:37 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:37.564+0000 7fe3c346c200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:28:37.866 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:37 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:37.671+0000 7fe3c346c200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-21T08:28:38.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:37 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/711683467' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-21T08:28:38.367 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:37 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:37.991+0000 7fe3c346c200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-21T08:28:38.367 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:38 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:38.076+0000 7fe3c346c200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T08:28:38.525 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-21T08:28:38.526 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-04-21T08:28:38.526 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsid": "f8cec7cc-ffb8-11ee-bc93-c7b262605968", 2024-04-21T08:28:38.526 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "health": { 2024-04-21T08:28:38.526 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-21T08:28:38.526 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-21T08:28:38.526 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-21T08:28:38.526 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:38.526 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-21T08:28:38.526 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-21T08:28:38.526 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 0 2024-04-21T08:28:38.526 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:28:38.526 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-21T08:28:38.527 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "a" 2024-04-21T08:28:38.529 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:28:38.529 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_age": 2, 2024-04-21T08:28:38.529 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-21T08:28:38.529 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:28:38.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-21T08:28:38.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-21T08:28:38.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:38.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-21T08:28:38.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:28:38.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-21T08:28:38.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-21T08:28:38.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-21T08:28:38.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-21T08:28:38.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-21T08:28:38.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-21T08:28:38.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:38.530 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-21T08:28:38.531 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-21T08:28:38.531 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-21T08:28:38.531 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-21T08:28:38.531 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-21T08:28:38.531 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-21T08:28:38.531 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-21T08:28:38.531 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-21T08:28:38.531 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-21T08:28:38.531 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:38.531 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-21T08:28:38.531 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:28:38.531 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-21T08:28:38.531 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-21T08:28:38.532 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:38.532 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-21T08:28:38.532 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-21T08:28:38.532 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-21T08:28:38.532 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-21T08:28:38.532 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-21T08:28:38.532 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-21T08:28:38.532 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "restful" 2024-04-21T08:28:38.532 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:28:38.532 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-21T08:28:38.532 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:38.532 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-21T08:28:38.532 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:28:38.532 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modified": "2024-04-21T08:28:22.687798+0000", 2024-04-21T08:28:38.532 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-21T08:28:38.533 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:38.533 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-21T08:28:38.533 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-04-21T08:28:38.533 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.803+0000 7f9c8ac93700 1 Processor -- start 2024-04-21T08:28:38.533 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.803+0000 7f9c8ac93700 1 -- start start 2024-04-21T08:28:38.533 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.803+0000 7f9c8ac93700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c84108440 0x7f9c84108860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:38.533 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.803+0000 7f9c8ac93700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9c84108e30 con 0x7f9c84108440 2024-04-21T08:28:38.533 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.803+0000 7f9c88a2f700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c84108440 0x7f9c84108860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:38.533 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.804+0000 7f9c88a2f700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c84108440 0x7f9c84108860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50968/0 (socket says 172.21.15.86:50968) 2024-04-21T08:28:38.533 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.804+0000 7f9c88a2f700 1 -- 172.21.15.86:0/1070270251 learned_addr learned my addr 172.21.15.86:0/1070270251 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:28:38.533 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.804+0000 7f9c88a2f700 1 -- 172.21.15.86:0/1070270251 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c841095f0 con 0x7f9c84108440 2024-04-21T08:28:38.533 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.804+0000 7f9c88a2f700 1 --2- 172.21.15.86:0/1070270251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c84108440 0x7f9c84108860 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9c780098a0 tx=0x7f9c78009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e9a0d4d4d573401 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:38.533 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.804+0000 7f9c837fe700 1 -- 172.21.15.86:0/1070270251 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c78004160 con 0x7f9c84108440 2024-04-21T08:28:38.534 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.804+0000 7f9c837fe700 1 -- 172.21.15.86:0/1070270251 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9c78004300 con 0x7f9c84108440 2024-04-21T08:28:38.534 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.804+0000 7f9c837fe700 1 -- 172.21.15.86:0/1070270251 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c78011af0 con 0x7f9c84108440 2024-04-21T08:28:38.534 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.805+0000 7f9c8ac93700 1 -- 172.21.15.86:0/1070270251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c84108440 msgr2=0x7f9c84108860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:38.534 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.805+0000 7f9c8ac93700 1 --2- 172.21.15.86:0/1070270251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c84108440 0x7f9c84108860 secure :-1 s=READY pgs=5 cs=0 l=1 rev1=1 crypto rx=0x7f9c780098a0 tx=0x7f9c78009bb0 comp rx=0 tx=0).stop 2024-04-21T08:28:38.534 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.805+0000 7f9c8ac93700 1 -- 172.21.15.86:0/1070270251 shutdown_connections 2024-04-21T08:28:38.534 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.805+0000 7f9c8ac93700 1 --2- 172.21.15.86:0/1070270251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c84108440 0x7f9c84108860 unknown :-1 s=CLOSED pgs=5 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:38.534 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.805+0000 7f9c8ac93700 1 -- 172.21.15.86:0/1070270251 >> 172.21.15.86:0/1070270251 conn(0x7f9c84103a60 msgr2=0x7f9c84105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:38.534 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.805+0000 7f9c8ac93700 1 -- 172.21.15.86:0/1070270251 shutdown_connections 2024-04-21T08:28:38.534 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.805+0000 7f9c8ac93700 1 -- 172.21.15.86:0/1070270251 wait complete. 2024-04-21T08:28:38.534 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.805+0000 7f9c8ac93700 1 Processor -- start 2024-04-21T08:28:38.534 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.806+0000 7f9c8ac93700 1 -- start start 2024-04-21T08:28:38.534 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.806+0000 7f9c8ac93700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c84108440 0x7f9c8419c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:38.534 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.806+0000 7f9c8ac93700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9c8410c820 con 0x7f9c84108440 2024-04-21T08:28:38.534 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.806+0000 7f9c88a2f700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c84108440 0x7f9c8419c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:38.535 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.806+0000 7f9c88a2f700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c84108440 0x7f9c8419c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50978/0 (socket says 172.21.15.86:50978) 2024-04-21T08:28:38.535 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.806+0000 7f9c88a2f700 1 -- 172.21.15.86:0/711683467 learned_addr learned my addr 172.21.15.86:0/711683467 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:28:38.535 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.806+0000 7f9c88a2f700 1 -- 172.21.15.86:0/711683467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9c8419c7d0 con 0x7f9c84108440 2024-04-21T08:28:38.535 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.806+0000 7f9c88a2f700 1 --2- 172.21.15.86:0/711683467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c84108440 0x7f9c8419c290 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9c78009fb0 tx=0x7f9c78011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:38.535 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.806+0000 7f9c81ffb700 1 -- 172.21.15.86:0/711683467 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c780036a0 con 0x7f9c84108440 2024-04-21T08:28:38.535 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.806+0000 7f9c81ffb700 1 -- 172.21.15.86:0/711683467 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f9c7801a430 con 0x7f9c84108440 2024-04-21T08:28:38.535 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.806+0000 7f9c8ac93700 1 -- 172.21.15.86:0/711683467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9c8419ca60 con 0x7f9c84108440 2024-04-21T08:28:38.535 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.806+0000 7f9c81ffb700 1 -- 172.21.15.86:0/711683467 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9c78019550 con 0x7f9c84108440 2024-04-21T08:28:38.535 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.807+0000 7f9c81ffb700 1 -- 172.21.15.86:0/711683467 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f9c78019be0 con 0x7f9c84108440 2024-04-21T08:28:38.535 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.807+0000 7f9c8ac93700 1 -- 172.21.15.86:0/711683467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9c8419f750 con 0x7f9c84108440 2024-04-21T08:28:38.535 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.807+0000 7f9c81ffb700 1 -- 172.21.15.86:0/711683467 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f9c7800b910 con 0x7f9c84108440 2024-04-21T08:28:38.535 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.808+0000 7f9c8ac93700 1 -- 172.21.15.86:0/711683467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9c8410c9a0 con 0x7f9c84108440 2024-04-21T08:28:38.535 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.810+0000 7f9c81ffb700 1 -- 172.21.15.86:0/711683467 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f9c78032070 con 0x7f9c84108440 2024-04-21T08:28:38.535 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.880+0000 7f9c8ac93700 1 -- 172.21.15.86:0/711683467 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f9c8419f920 con 0x7f9c84108440 2024-04-21T08:28:38.536 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.881+0000 7f9c81ffb700 1 -- 172.21.15.86:0/711683467 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f9c7801a5e0 con 0x7f9c84108440 2024-04-21T08:28:38.536 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.882+0000 7f9c8ac93700 1 -- 172.21.15.86:0/711683467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c84108440 msgr2=0x7f9c8419c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:38.536 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.882+0000 7f9c8ac93700 1 --2- 172.21.15.86:0/711683467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c84108440 0x7f9c8419c290 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f9c78009fb0 tx=0x7f9c78011940 comp rx=0 tx=0).stop 2024-04-21T08:28:38.536 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.882+0000 7f9c8ac93700 1 -- 172.21.15.86:0/711683467 shutdown_connections 2024-04-21T08:28:38.536 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.882+0000 7f9c8ac93700 1 --2- 172.21.15.86:0/711683467 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9c84108440 0x7f9c8419c290 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:38.536 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.882+0000 7f9c8ac93700 1 -- 172.21.15.86:0/711683467 >> 172.21.15.86:0/711683467 conn(0x7f9c84103a60 msgr2=0x7f9c84104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:38.536 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.882+0000 7f9c8ac93700 1 -- 172.21.15.86:0/711683467 shutdown_connections 2024-04-21T08:28:38.536 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:37.882+0000 7f9c8ac93700 1 -- 172.21.15.86:0/711683467 wait complete. 2024-04-21T08:28:38.536 INFO:teuthology.orchestra.run.smithi086.stdout:mgr not available, waiting (1/15)... 2024-04-21T08:28:39.588 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:39 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:39.254+0000 7fe3c346c200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T08:28:39.588 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:39 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:39.419+0000 7fe3c346c200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T08:28:39.588 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:39 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:39.587+0000 7fe3c346c200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T08:28:40.170 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:39 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:39.874+0000 7fe3c346c200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T08:28:40.171 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:39 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:39.967+0000 7fe3c346c200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T08:28:40.171 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:40 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:40.170+0000 7fe3c346c200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T08:28:40.458 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:40 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:40.369+0000 7fe3c346c200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:28:41.352 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:41 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:41.001+0000 7fe3c346c200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T08:28:41.616 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:41 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:41.428+0000 7fe3c346c200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T08:28:41.616 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:41 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:41.528+0000 7fe3c346c200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T08:28:42.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:41 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1537062984' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-21T08:28:42.116 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:41 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:41.741+0000 7fe3c346c200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T08:28:42.249 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-21T08:28:42.249 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-04-21T08:28:42.249 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsid": "f8cec7cc-ffb8-11ee-bc93-c7b262605968", 2024-04-21T08:28:42.249 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "health": { 2024-04-21T08:28:42.249 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-21T08:28:42.250 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-21T08:28:42.250 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-21T08:28:42.250 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:42.250 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-21T08:28:42.250 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-21T08:28:42.250 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 0 2024-04-21T08:28:42.250 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:28:42.250 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-21T08:28:42.250 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "a" 2024-04-21T08:28:42.250 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:28:42.251 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_age": 6, 2024-04-21T08:28:42.251 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-21T08:28:42.251 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:28:42.251 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-21T08:28:42.251 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-21T08:28:42.251 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:42.251 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-21T08:28:42.251 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:28:42.251 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-21T08:28:42.251 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-21T08:28:42.251 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-21T08:28:42.251 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-21T08:28:42.251 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-21T08:28:42.251 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-21T08:28:42.251 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:42.252 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-21T08:28:42.252 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-21T08:28:42.252 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-21T08:28:42.252 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-21T08:28:42.252 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-21T08:28:42.252 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-21T08:28:42.252 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-21T08:28:42.252 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-21T08:28:42.252 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-21T08:28:42.252 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:42.252 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-21T08:28:42.252 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:28:42.252 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-21T08:28:42.252 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-21T08:28:42.253 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:42.253 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-21T08:28:42.253 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-21T08:28:42.253 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-21T08:28:42.253 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-21T08:28:42.253 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-21T08:28:42.253 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-21T08:28:42.253 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "restful" 2024-04-21T08:28:42.253 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:28:42.253 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-21T08:28:42.253 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:42.253 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-21T08:28:42.253 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:28:42.253 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modified": "2024-04-21T08:28:22.687798+0000", 2024-04-21T08:28:42.253 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-21T08:28:42.254 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:42.254 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-21T08:28:42.254 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-04-21T08:28:42.254 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.547+0000 7f672dba2700 1 Processor -- start 2024-04-21T08:28:42.254 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.547+0000 7f672dba2700 1 -- start start 2024-04-21T08:28:42.254 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.547+0000 7f672dba2700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67281081c0 0x7f67281085e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:42.254 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.547+0000 7f672dba2700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6728108bb0 con 0x7f67281081c0 2024-04-21T08:28:42.254 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.547+0000 7f672cba0700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67281081c0 0x7f67281085e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:42.254 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.547+0000 7f672cba0700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67281081c0 0x7f67281085e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53304/0 (socket says 172.21.15.86:53304) 2024-04-21T08:28:42.254 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.547+0000 7f672cba0700 1 -- 172.21.15.86:0/3674982683 learned_addr learned my addr 172.21.15.86:0/3674982683 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:28:42.254 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.548+0000 7f672cba0700 1 -- 172.21.15.86:0/3674982683 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6728109370 con 0x7f67281081c0 2024-04-21T08:28:42.254 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.548+0000 7f672cba0700 1 --2- 172.21.15.86:0/3674982683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67281081c0 0x7f67281085e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f671c0098a0 tx=0x7f671c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b7d26299cb4f467 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:42.254 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.548+0000 7f67277fe700 1 -- 172.21.15.86:0/3674982683 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f671c004160 con 0x7f67281081c0 2024-04-21T08:28:42.255 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.548+0000 7f67277fe700 1 -- 172.21.15.86:0/3674982683 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f671c004300 con 0x7f67281081c0 2024-04-21T08:28:42.255 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.548+0000 7f67277fe700 1 -- 172.21.15.86:0/3674982683 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f671c011af0 con 0x7f67281081c0 2024-04-21T08:28:42.255 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.549+0000 7f672dba2700 1 -- 172.21.15.86:0/3674982683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67281081c0 msgr2=0x7f67281085e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:42.255 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.549+0000 7f672dba2700 1 --2- 172.21.15.86:0/3674982683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67281081c0 0x7f67281085e0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f671c0098a0 tx=0x7f671c009bb0 comp rx=0 tx=0).stop 2024-04-21T08:28:42.255 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.549+0000 7f672dba2700 1 -- 172.21.15.86:0/3674982683 shutdown_connections 2024-04-21T08:28:42.255 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.549+0000 7f672dba2700 1 --2- 172.21.15.86:0/3674982683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67281081c0 0x7f67281085e0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:42.255 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.549+0000 7f672dba2700 1 -- 172.21.15.86:0/3674982683 >> 172.21.15.86:0/3674982683 conn(0x7f6728103760 msgr2=0x7f6728105ba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:42.255 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.549+0000 7f672dba2700 1 -- 172.21.15.86:0/3674982683 shutdown_connections 2024-04-21T08:28:42.255 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.549+0000 7f672dba2700 1 -- 172.21.15.86:0/3674982683 wait complete. 2024-04-21T08:28:42.255 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.550+0000 7f672dba2700 1 Processor -- start 2024-04-21T08:28:42.255 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.550+0000 7f672dba2700 1 -- start start 2024-04-21T08:28:42.255 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.550+0000 7f672dba2700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67281081c0 0x7f672819c030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:42.255 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.550+0000 7f672dba2700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f672810c720 con 0x7f67281081c0 2024-04-21T08:28:42.256 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.550+0000 7f672cba0700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67281081c0 0x7f672819c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:42.256 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.550+0000 7f672cba0700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67281081c0 0x7f672819c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53306/0 (socket says 172.21.15.86:53306) 2024-04-21T08:28:42.256 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.550+0000 7f672cba0700 1 -- 172.21.15.86:0/1537062984 learned_addr learned my addr 172.21.15.86:0/1537062984 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:28:42.256 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.550+0000 7f672cba0700 1 -- 172.21.15.86:0/1537062984 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f672819c570 con 0x7f67281081c0 2024-04-21T08:28:42.256 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.550+0000 7f672cba0700 1 --2- 172.21.15.86:0/1537062984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67281081c0 0x7f672819c030 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f671c003f40 tx=0x7f671c011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:42.256 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.551+0000 7f6725ffb700 1 -- 172.21.15.86:0/1537062984 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f671c0038b0 con 0x7f67281081c0 2024-04-21T08:28:42.256 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.551+0000 7f6725ffb700 1 -- 172.21.15.86:0/1537062984 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f671c003a50 con 0x7f67281081c0 2024-04-21T08:28:42.256 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.551+0000 7f672dba2700 1 -- 172.21.15.86:0/1537062984 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f672819c800 con 0x7f67281081c0 2024-04-21T08:28:42.256 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.551+0000 7f6725ffb700 1 -- 172.21.15.86:0/1537062984 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f671c01a5c0 con 0x7f67281081c0 2024-04-21T08:28:42.256 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.551+0000 7f672dba2700 1 -- 172.21.15.86:0/1537062984 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f672819f4f0 con 0x7f67281081c0 2024-04-21T08:28:42.256 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.551+0000 7f6725ffb700 1 -- 172.21.15.86:0/1537062984 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 1) ==== 807+0+0 (secure 0 0 0) 0x7f671c01ac50 con 0x7f67281081c0 2024-04-21T08:28:42.256 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.551+0000 7f6725ffb700 1 -- 172.21.15.86:0/1537062984 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f671c019700 con 0x7f67281081c0 2024-04-21T08:28:42.256 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.552+0000 7f672dba2700 1 -- 172.21.15.86:0/1537062984 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6728069b50 con 0x7f67281081c0 2024-04-21T08:28:42.256 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.554+0000 7f6725ffb700 1 -- 172.21.15.86:0/1537062984 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f671c0199a0 con 0x7f67281081c0 2024-04-21T08:28:42.256 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.627+0000 7f672dba2700 1 -- 172.21.15.86:0/1537062984 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f672819fa40 con 0x7f67281081c0 2024-04-21T08:28:42.257 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.628+0000 7f6725ffb700 1 -- 172.21.15.86:0/1537062984 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f671c019b80 con 0x7f67281081c0 2024-04-21T08:28:42.257 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.629+0000 7f672dba2700 1 -- 172.21.15.86:0/1537062984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67281081c0 msgr2=0x7f672819c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:42.257 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.629+0000 7f672dba2700 1 --2- 172.21.15.86:0/1537062984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67281081c0 0x7f672819c030 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f671c003f40 tx=0x7f671c011890 comp rx=0 tx=0).stop 2024-04-21T08:28:42.257 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.629+0000 7f672dba2700 1 -- 172.21.15.86:0/1537062984 shutdown_connections 2024-04-21T08:28:42.257 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.629+0000 7f672dba2700 1 --2- 172.21.15.86:0/1537062984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67281081c0 0x7f672819c030 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:42.257 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.629+0000 7f672dba2700 1 -- 172.21.15.86:0/1537062984 >> 172.21.15.86:0/1537062984 conn(0x7f6728103760 msgr2=0x7f6728104210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:42.257 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.629+0000 7f672dba2700 1 -- 172.21.15.86:0/1537062984 shutdown_connections 2024-04-21T08:28:42.257 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:41.629+0000 7f672dba2700 1 -- 172.21.15.86:0/1537062984 wait complete. 2024-04-21T08:28:42.257 INFO:teuthology.orchestra.run.smithi086.stdout:mgr not available, waiting (2/15)... 2024-04-21T08:28:42.597 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:42 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:42.268+0000 7fe3c346c200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T08:28:42.597 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:42 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:42.359+0000 7fe3c346c200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T08:28:42.866 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:42 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:42.596+0000 7fe3c346c200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T08:28:42.866 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:42 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:42.729+0000 7fe3c346c200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T08:28:42.866 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:42 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:42.823+0000 7fe3c346c200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T08:28:43.238 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:42 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:42.937+0000 7fe3c346c200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T08:28:43.238 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:43 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:43.237+0000 7fe3c346c200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T08:28:44.096 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:43 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:43.827+0000 7fe3c346c200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T08:28:44.096 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:44 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:44.040+0000 7fe3c346c200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T08:28:44.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:44 smithi086 ceph-mon[27939]: Activating manager daemon x 2024-04-21T08:28:44.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:44 smithi086 ceph-mon[27939]: mgrmap e2: x(active, starting, since 0.00339508s) 2024-04-21T08:28:44.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:44 smithi086 ceph-mon[27939]: from='mgr.14100 172.21.15.86:0/4136241698' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-21T08:28:44.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:44 smithi086 ceph-mon[27939]: from='mgr.14100 172.21.15.86:0/4136241698' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-21T08:28:44.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:44 smithi086 ceph-mon[27939]: from='mgr.14100 172.21.15.86:0/4136241698' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-21T08:28:44.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:44 smithi086 ceph-mon[27939]: from='mgr.14100 172.21.15.86:0/4136241698' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:28:44.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:44 smithi086 ceph-mon[27939]: from='mgr.14100 172.21.15.86:0/4136241698' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-21T08:28:44.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:44 smithi086 ceph-mon[27939]: Manager daemon x is now available 2024-04-21T08:28:44.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:44 smithi086 ceph-mon[27939]: from='mgr.14100 172.21.15.86:0/4136241698' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-21T08:28:44.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:44 smithi086 ceph-mon[27939]: from='mgr.14100 172.21.15.86:0/4136241698' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-21T08:28:44.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:44 smithi086 ceph-mon[27939]: from='mgr.14100 172.21.15.86:0/4136241698' entity='mgr.x' 2024-04-21T08:28:44.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:44 smithi086 ceph-mon[27939]: from='mgr.14100 172.21.15.86:0/4136241698' entity='mgr.x' 2024-04-21T08:28:44.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:44 smithi086 ceph-mon[27939]: from='mgr.14100 172.21.15.86:0/4136241698' entity='mgr.x' 2024-04-21T08:28:45.315 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:45 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1066964210' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-21T08:28:45.316 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:45 smithi086 ceph-mon[27939]: mgrmap e3: x(active, since 1.00639s) 2024-04-21T08:28:45.320 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-21T08:28:45.320 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-04-21T08:28:45.320 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsid": "f8cec7cc-ffb8-11ee-bc93-c7b262605968", 2024-04-21T08:28:45.320 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "health": { 2024-04-21T08:28:45.320 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-21T08:28:45.320 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-21T08:28:45.320 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-21T08:28:45.320 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:45.320 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-21T08:28:45.321 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-21T08:28:45.321 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 0 2024-04-21T08:28:45.321 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:28:45.321 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-21T08:28:45.321 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "a" 2024-04-21T08:28:45.321 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:28:45.321 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_age": 9, 2024-04-21T08:28:45.321 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-21T08:28:45.321 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:28:45.321 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-21T08:28:45.321 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-21T08:28:45.321 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:45.321 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-21T08:28:45.321 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:28:45.321 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-21T08:28:45.321 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-21T08:28:45.322 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-21T08:28:45.330 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-21T08:28:45.330 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-21T08:28:45.330 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-21T08:28:45.330 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:45.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-21T08:28:45.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-21T08:28:45.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-21T08:28:45.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-21T08:28:45.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-21T08:28:45.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-21T08:28:45.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-21T08:28:45.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-21T08:28:45.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-21T08:28:45.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:45.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-21T08:28:45.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:28:45.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-21T08:28:45.331 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-21T08:28:45.332 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:45.332 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-21T08:28:45.332 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "available": false, 2024-04-21T08:28:45.332 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-21T08:28:45.332 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-21T08:28:45.332 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-21T08:28:45.332 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-21T08:28:45.332 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "restful" 2024-04-21T08:28:45.332 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:28:45.332 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-21T08:28:45.332 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:45.332 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-21T08:28:45.332 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:28:45.332 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modified": "2024-04-21T08:28:22.687798+0000", 2024-04-21T08:28:45.333 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-21T08:28:45.333 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:45.333 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-21T08:28:45.333 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-04-21T08:28:45.333 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.641+0000 7f4cfd9ec700 1 Processor -- start 2024-04-21T08:28:45.333 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.641+0000 7f4cfd9ec700 1 -- start start 2024-04-21T08:28:45.333 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.641+0000 7f4cfd9ec700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cf8107380 0x7f4cf81077a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:45.333 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.641+0000 7f4cfd9ec700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf8107d70 con 0x7f4cf8107380 2024-04-21T08:28:45.333 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.641+0000 7f4cf6ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cf8107380 0x7f4cf81077a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:45.333 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.642+0000 7f4cf6ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cf8107380 0x7f4cf81077a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53354/0 (socket says 172.21.15.86:53354) 2024-04-21T08:28:45.333 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.642+0000 7f4cf6ffd700 1 -- 172.21.15.86:0/573985748 learned_addr learned my addr 172.21.15.86:0/573985748 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:28:45.333 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.642+0000 7f4cf6ffd700 1 -- 172.21.15.86:0/573985748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf8108580 con 0x7f4cf8107380 2024-04-21T08:28:45.333 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.642+0000 7f4cf6ffd700 1 --2- 172.21.15.86:0/573985748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cf8107380 0x7f4cf81077a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4ce80098a0 tx=0x7f4ce8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9004c6b2134eb20 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:45.334 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.642+0000 7f4cf67fc700 1 -- 172.21.15.86:0/573985748 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ce8004160 con 0x7f4cf8107380 2024-04-21T08:28:45.334 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.642+0000 7f4cf67fc700 1 -- 172.21.15.86:0/573985748 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4ce8004300 con 0x7f4cf8107380 2024-04-21T08:28:45.334 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.642+0000 7f4cf67fc700 1 -- 172.21.15.86:0/573985748 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ce8011af0 con 0x7f4cf8107380 2024-04-21T08:28:45.334 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.643+0000 7f4cfd9ec700 1 -- 172.21.15.86:0/573985748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cf8107380 msgr2=0x7f4cf81077a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:45.334 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.643+0000 7f4cfd9ec700 1 --2- 172.21.15.86:0/573985748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cf8107380 0x7f4cf81077a0 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4ce80098a0 tx=0x7f4ce8009bb0 comp rx=0 tx=0).stop 2024-04-21T08:28:45.334 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.643+0000 7f4cfd9ec700 1 -- 172.21.15.86:0/573985748 shutdown_connections 2024-04-21T08:28:45.334 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.643+0000 7f4cfd9ec700 1 --2- 172.21.15.86:0/573985748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cf8107380 0x7f4cf81077a0 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:45.334 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.643+0000 7f4cfd9ec700 1 -- 172.21.15.86:0/573985748 >> 172.21.15.86:0/573985748 conn(0x7f4cf8102980 msgr2=0x7f4cf8104da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:45.334 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.643+0000 7f4cfd9ec700 1 -- 172.21.15.86:0/573985748 shutdown_connections 2024-04-21T08:28:45.334 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.643+0000 7f4cfd9ec700 1 -- 172.21.15.86:0/573985748 wait complete. 2024-04-21T08:28:45.334 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.643+0000 7f4cfd9ec700 1 Processor -- start 2024-04-21T08:28:45.334 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.643+0000 7f4cfd9ec700 1 -- start start 2024-04-21T08:28:45.334 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.644+0000 7f4cfd9ec700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cf8107380 0x7f4cf819c230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:45.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.644+0000 7f4cfd9ec700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4cf810b7b0 con 0x7f4cf8107380 2024-04-21T08:28:45.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.644+0000 7f4cf6ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cf8107380 0x7f4cf819c230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:45.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.644+0000 7f4cf6ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cf8107380 0x7f4cf819c230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53366/0 (socket says 172.21.15.86:53366) 2024-04-21T08:28:45.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.644+0000 7f4cf6ffd700 1 -- 172.21.15.86:0/1066964210 learned_addr learned my addr 172.21.15.86:0/1066964210 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:28:45.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.644+0000 7f4cf6ffd700 1 -- 172.21.15.86:0/1066964210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cf819c770 con 0x7f4cf8107380 2024-04-21T08:28:45.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.644+0000 7f4cf6ffd700 1 --2- 172.21.15.86:0/1066964210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cf8107380 0x7f4cf819c230 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8003f40 tx=0x7f4ce8011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:45.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.644+0000 7f4cfc9ea700 1 -- 172.21.15.86:0/1066964210 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ce80038b0 con 0x7f4cf8107380 2024-04-21T08:28:45.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.644+0000 7f4cfd9ec700 1 -- 172.21.15.86:0/1066964210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cf819ca00 con 0x7f4cf8107380 2024-04-21T08:28:45.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.644+0000 7f4cfc9ea700 1 -- 172.21.15.86:0/1066964210 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4ce8003a50 con 0x7f4cf8107380 2024-04-21T08:28:45.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.644+0000 7f4cfc9ea700 1 -- 172.21.15.86:0/1066964210 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4ce801a5c0 con 0x7f4cf8107380 2024-04-21T08:28:45.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.644+0000 7f4cfd9ec700 1 -- 172.21.15.86:0/1066964210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cf819f6f0 con 0x7f4cf8107380 2024-04-21T08:28:45.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.645+0000 7f4cfc9ea700 1 -- 172.21.15.86:0/1066964210 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 2) ==== 46314+0+0 (secure 0 0 0) 0x7f4ce801a8c0 con 0x7f4cf8107380 2024-04-21T08:28:45.335 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.645+0000 7f4cfd9ec700 1 -- 172.21.15.86:0/1066964210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4cf80640a0 con 0x7f4cf8107380 2024-04-21T08:28:45.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.645+0000 7f4cfc9ea700 1 -- 172.21.15.86:0/1066964210 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f4ce804d6f0 con 0x7f4cf8107380 2024-04-21T08:28:45.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.647+0000 7f4cfc9ea700 1 -- 172.21.15.86:0/1066964210 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+73265 (secure 0 0 0) 0x7f4ce8019c70 con 0x7f4cf8107380 2024-04-21T08:28:45.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.717+0000 7f4cfd9ec700 1 -- 172.21.15.86:0/1066964210 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f4cf80008d0 con 0x7f4cf8107380 2024-04-21T08:28:45.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.717+0000 7f4cfc9ea700 1 -- 172.21.15.86:0/1066964210 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f4ce8020940 con 0x7f4cf8107380 2024-04-21T08:28:45.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.718+0000 7f4cfd9ec700 1 -- 172.21.15.86:0/1066964210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cf8107380 msgr2=0x7f4cf819c230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:45.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.718+0000 7f4cfd9ec700 1 --2- 172.21.15.86:0/1066964210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cf8107380 0x7f4cf819c230 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7f4ce8003f40 tx=0x7f4ce8011890 comp rx=0 tx=0).stop 2024-04-21T08:28:45.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.718+0000 7f4cfd9ec700 1 -- 172.21.15.86:0/1066964210 shutdown_connections 2024-04-21T08:28:45.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.718+0000 7f4cfd9ec700 1 --2- 172.21.15.86:0/1066964210 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cf8107380 0x7f4cf819c230 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:45.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.718+0000 7f4cfd9ec700 1 -- 172.21.15.86:0/1066964210 >> 172.21.15.86:0/1066964210 conn(0x7f4cf8102980 msgr2=0x7f4cf8103730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:45.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.718+0000 7f4cfd9ec700 1 -- 172.21.15.86:0/1066964210 shutdown_connections 2024-04-21T08:28:45.336 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:44.718+0000 7f4cfd9ec700 1 -- 172.21.15.86:0/1066964210 wait complete. 2024-04-21T08:28:45.336 INFO:teuthology.orchestra.run.smithi086.stdout:mgr not available, waiting (3/15)... 2024-04-21T08:28:47.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:47 smithi086 ceph-mon[27939]: mgrmap e4: x(active, since 2s) 2024-04-21T08:28:48.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:48 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1842393109' entity='client.admin' cmd=[{"prefix": "status", "format": "json-pretty"}]: dispatch 2024-04-21T08:28:48.651 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-21T08:28:48.651 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-04-21T08:28:48.651 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsid": "f8cec7cc-ffb8-11ee-bc93-c7b262605968", 2024-04-21T08:28:48.651 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "health": { 2024-04-21T08:28:48.651 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "status": "HEALTH_OK", 2024-04-21T08:28:48.651 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "checks": {}, 2024-04-21T08:28:48.651 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mutes": [] 2024-04-21T08:28:48.651 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:48.651 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "election_epoch": 5, 2024-04-21T08:28:48.651 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum": [ 2024-04-21T08:28:48.651 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 0 2024-04-21T08:28:48.652 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:28:48.652 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_names": [ 2024-04-21T08:28:48.652 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "a" 2024-04-21T08:28:48.652 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:28:48.652 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "quorum_age": 12, 2024-04-21T08:28:48.652 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "monmap": { 2024-04-21T08:28:48.652 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:28:48.652 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "min_mon_release_name": "squid", 2024-04-21T08:28:48.652 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_mons": 1 2024-04-21T08:28:48.652 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:48.652 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osdmap": { 2024-04-21T08:28:48.652 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:28:48.652 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_osds": 0, 2024-04-21T08:28:48.652 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_up_osds": 0, 2024-04-21T08:28:48.652 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_up_since": 0, 2024-04-21T08:28:48.661 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_in_osds": 0, 2024-04-21T08:28:48.661 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "osd_in_since": 0, 2024-04-21T08:28:48.661 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_remapped_pgs": 0 2024-04-21T08:28:48.661 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:48.661 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgmap": { 2024-04-21T08:28:48.661 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "pgs_by_state": [], 2024-04-21T08:28:48.662 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pgs": 0, 2024-04-21T08:28:48.662 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_pools": 0, 2024-04-21T08:28:48.662 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_objects": 0, 2024-04-21T08:28:48.662 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "data_bytes": 0, 2024-04-21T08:28:48.662 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_used": 0, 2024-04-21T08:28:48.662 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_avail": 0, 2024-04-21T08:28:48.662 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "bytes_total": 0 2024-04-21T08:28:48.662 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:48.662 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "fsmap": { 2024-04-21T08:28:48.662 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:28:48.662 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "by_rank": [], 2024-04-21T08:28:48.662 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "up:standby": 0 2024-04-21T08:28:48.662 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:48.663 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mgrmap": { 2024-04-21T08:28:48.663 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-21T08:28:48.663 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_standbys": 0, 2024-04-21T08:28:48.663 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modules": [ 2024-04-21T08:28:48.663 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "iostat", 2024-04-21T08:28:48.663 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "nfs", 2024-04-21T08:28:48.663 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "restful" 2024-04-21T08:28:48.663 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ], 2024-04-21T08:28:48.663 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-21T08:28:48.663 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:48.663 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "servicemap": { 2024-04-21T08:28:48.663 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 1, 2024-04-21T08:28:48.663 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "modified": "2024-04-21T08:28:22.687798+0000", 2024-04-21T08:28:48.663 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "services": {} 2024-04-21T08:28:48.663 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout }, 2024-04-21T08:28:48.664 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "progress_events": {} 2024-04-21T08:28:48.664 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-04-21T08:28:48.664 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.771+0000 7f445a8f9700 1 Processor -- start 2024-04-21T08:28:48.664 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.771+0000 7f445a8f9700 1 -- start start 2024-04-21T08:28:48.664 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.771+0000 7f445a8f9700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4454108840 0x7f4454108c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:48.664 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.771+0000 7f445a8f9700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4454109230 con 0x7f4454108840 2024-04-21T08:28:48.664 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.771+0000 7f4453fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4454108840 0x7f4454108c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:48.664 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.771+0000 7f4453fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4454108840 0x7f4454108c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53382/0 (socket says 172.21.15.86:53382) 2024-04-21T08:28:48.664 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.771+0000 7f4453fff700 1 -- 172.21.15.86:0/4236124044 learned_addr learned my addr 172.21.15.86:0/4236124044 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:28:48.664 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.772+0000 7f4453fff700 1 -- 172.21.15.86:0/4236124044 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44541074a0 con 0x7f4454108840 2024-04-21T08:28:48.664 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.772+0000 7f4453fff700 1 --2- 172.21.15.86:0/4236124044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4454108840 0x7f4454108c60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f44440098a0 tx=0x7f4444009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8483255e6a0a3aa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:48.664 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.772+0000 7f4452ffd700 1 -- 172.21.15.86:0/4236124044 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4444004160 con 0x7f4454108840 2024-04-21T08:28:48.664 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.772+0000 7f4452ffd700 1 -- 172.21.15.86:0/4236124044 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4444004300 con 0x7f4454108840 2024-04-21T08:28:48.664 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.772+0000 7f4452ffd700 1 -- 172.21.15.86:0/4236124044 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4444011af0 con 0x7f4454108840 2024-04-21T08:28:48.665 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.772+0000 7f445a8f9700 1 -- 172.21.15.86:0/4236124044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4454108840 msgr2=0x7f4454108c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:48.665 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.773+0000 7f445a8f9700 1 --2- 172.21.15.86:0/4236124044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4454108840 0x7f4454108c60 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7f44440098a0 tx=0x7f4444009bb0 comp rx=0 tx=0).stop 2024-04-21T08:28:48.665 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.773+0000 7f445a8f9700 1 -- 172.21.15.86:0/4236124044 shutdown_connections 2024-04-21T08:28:48.665 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.773+0000 7f445a8f9700 1 --2- 172.21.15.86:0/4236124044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4454108840 0x7f4454108c60 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:48.665 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.773+0000 7f445a8f9700 1 -- 172.21.15.86:0/4236124044 >> 172.21.15.86:0/4236124044 conn(0x7f4454101fb0 msgr2=0x7f44541043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:48.665 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.773+0000 7f445a8f9700 1 -- 172.21.15.86:0/4236124044 shutdown_connections 2024-04-21T08:28:48.665 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.773+0000 7f445a8f9700 1 -- 172.21.15.86:0/4236124044 wait complete. 2024-04-21T08:28:48.665 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.773+0000 7f445a8f9700 1 Processor -- start 2024-04-21T08:28:48.665 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.773+0000 7f445a8f9700 1 -- start start 2024-04-21T08:28:48.665 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.774+0000 7f445a8f9700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4454108840 0x7f445419a010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:48.665 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.774+0000 7f445a8f9700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4454106390 con 0x7f4454108840 2024-04-21T08:28:48.665 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.774+0000 7f4453fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4454108840 0x7f445419a010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:48.665 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.774+0000 7f4453fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4454108840 0x7f445419a010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53396/0 (socket says 172.21.15.86:53396) 2024-04-21T08:28:48.665 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.774+0000 7f4453fff700 1 -- 172.21.15.86:0/1842393109 learned_addr learned my addr 172.21.15.86:0/1842393109 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:28:48.665 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.774+0000 7f4453fff700 1 -- 172.21.15.86:0/1842393109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f445419a550 con 0x7f4454108840 2024-04-21T08:28:48.666 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.774+0000 7f4453fff700 1 --2- 172.21.15.86:0/1842393109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4454108840 0x7f445419a010 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4444003f40 tx=0x7f4444011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:48.666 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.774+0000 7f44517fa700 1 -- 172.21.15.86:0/1842393109 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f44440038b0 con 0x7f4454108840 2024-04-21T08:28:48.666 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.774+0000 7f44517fa700 1 -- 172.21.15.86:0/1842393109 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f4444003a50 con 0x7f4454108840 2024-04-21T08:28:48.666 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.774+0000 7f44517fa700 1 -- 172.21.15.86:0/1842393109 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f444401a5c0 con 0x7f4454108840 2024-04-21T08:28:48.666 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.774+0000 7f445a8f9700 1 -- 172.21.15.86:0/1842393109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f445419a7e0 con 0x7f4454108840 2024-04-21T08:28:48.666 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.774+0000 7f445a8f9700 1 -- 172.21.15.86:0/1842393109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f445419d480 con 0x7f4454108840 2024-04-21T08:28:48.666 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.775+0000 7f44517fa700 1 -- 172.21.15.86:0/1842393109 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f444401a8c0 con 0x7f4454108840 2024-04-21T08:28:48.666 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.775+0000 7f44517fa700 1 --2- 172.21.15.86:0/1842393109 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7f443c03b7e0 0x7f443c03dca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:48.666 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.775+0000 7f44517fa700 1 -- 172.21.15.86:0/1842393109 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f444404da60 con 0x7f4454108840 2024-04-21T08:28:48.666 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.775+0000 7f44537fe700 1 --2- 172.21.15.86:0/1842393109 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7f443c03b7e0 0x7f443c03dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:48.666 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.775+0000 7f445a8f9700 1 -- 172.21.15.86:0/1842393109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f44540640a0 con 0x7f4454108840 2024-04-21T08:28:48.666 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.776+0000 7f44537fe700 1 --2- 172.21.15.86:0/1842393109 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7f443c03b7e0 0x7f443c03dca0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4448006fd0 tx=0x7f4448006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:48.666 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:47.781+0000 7f44517fa700 1 -- 172.21.15.86:0/1842393109 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4444019c70 con 0x7f4454108840 2024-04-21T08:28:48.667 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.060+0000 7f445a8f9700 1 -- 172.21.15.86:0/1842393109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "status", "format": "json-pretty"} v 0) -- 0x7f4454193570 con 0x7f4454108840 2024-04-21T08:28:48.667 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.061+0000 7f44517fa700 1 -- 172.21.15.86:0/1842393109 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "status", "format": "json-pretty"}]=0 v0) ==== 79+0+1239 (secure 0 0 0) 0x7f4454193570 con 0x7f4454108840 2024-04-21T08:28:48.667 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.064+0000 7f445a8f9700 1 -- 172.21.15.86:0/1842393109 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7f443c03b7e0 msgr2=0x7f443c03dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:48.667 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.064+0000 7f445a8f9700 1 --2- 172.21.15.86:0/1842393109 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7f443c03b7e0 0x7f443c03dca0 secure :-1 s=READY pgs=6 cs=0 l=1 rev1=1 crypto rx=0x7f4448006fd0 tx=0x7f4448006e40 comp rx=0 tx=0).stop 2024-04-21T08:28:48.667 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.064+0000 7f445a8f9700 1 -- 172.21.15.86:0/1842393109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4454108840 msgr2=0x7f445419a010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:48.667 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.064+0000 7f445a8f9700 1 --2- 172.21.15.86:0/1842393109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4454108840 0x7f445419a010 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f4444003f40 tx=0x7f4444011890 comp rx=0 tx=0).stop 2024-04-21T08:28:48.667 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.064+0000 7f445a8f9700 1 -- 172.21.15.86:0/1842393109 shutdown_connections 2024-04-21T08:28:48.667 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.064+0000 7f445a8f9700 1 --2- 172.21.15.86:0/1842393109 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7f443c03b7e0 0x7f443c03dca0 unknown :-1 s=CLOSED pgs=6 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:48.667 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.064+0000 7f445a8f9700 1 --2- 172.21.15.86:0/1842393109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4454108840 0x7f445419a010 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:48.667 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.064+0000 7f445a8f9700 1 -- 172.21.15.86:0/1842393109 >> 172.21.15.86:0/1842393109 conn(0x7f4454101fb0 msgr2=0x7f4454190630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:48.667 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.064+0000 7f445a8f9700 1 -- 172.21.15.86:0/1842393109 shutdown_connections 2024-04-21T08:28:48.667 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:48.064+0000 7f445a8f9700 1 -- 172.21.15.86:0/1842393109 wait complete. 2024-04-21T08:28:48.667 INFO:teuthology.orchestra.run.smithi086.stdout:mgr is available 2024-04-21T08:28:49.585 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:49 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/399228401' entity='client.admin' cmd=[{"prefix": "config assimilate-conf"}]: dispatch 2024-04-21T08:28:49.845 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-21T08:28:49.845 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [global] 2024-04-21T08:28:49.845 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout fsid = f8cec7cc-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:28:49.845 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_cluster_log_file_level = debug 2024-04-21T08:28:49.845 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_allow_pg_remap = true 2024-04-21T08:28:49.845 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_allow_primary_affinity = true 2024-04-21T08:28:49.846 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_warn_on_no_sortbitwise = false 2024-04-21T08:28:49.846 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_crush_chooseleaf_type = 0 2024-04-21T08:28:49.846 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-21T08:28:49.846 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [mgr] 2024-04-21T08:28:49.846 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mgr/telemetry/nag = false 2024-04-21T08:28:49.846 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 2024-04-21T08:28:49.846 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout [osd] 2024-04-21T08:28:49.846 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_backfillfull_ratio = 0.85 2024-04-21T08:28:49.846 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_full_ratio = 0.9 2024-04-21T08:28:49.846 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout mon_osd_nearfull_ratio = 0.8 2024-04-21T08:28:49.846 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_map_max_advance = 10 2024-04-21T08:28:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout osd_sloppy_crc = true 2024-04-21T08:28:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.077+0000 7f606af5f700 1 Processor -- start 2024-04-21T08:28:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.077+0000 7f606af5f700 1 -- start start 2024-04-21T08:28:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.077+0000 7f606af5f700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60641069f0 0x7f6064106e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.077+0000 7f606af5f700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60641073e0 con 0x7f60641069f0 2024-04-21T08:28:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.077+0000 7f6068cfb700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60641069f0 0x7f6064106e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.077+0000 7f6068cfb700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60641069f0 0x7f6064106e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53412/0 (socket says 172.21.15.86:53412) 2024-04-21T08:28:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.078+0000 7f6068cfb700 1 -- 172.21.15.86:0/2506598545 learned_addr learned my addr 172.21.15.86:0/2506598545 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:28:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.078+0000 7f6068cfb700 1 -- 172.21.15.86:0/2506598545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6064107bf0 con 0x7f60641069f0 2024-04-21T08:28:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.078+0000 7f6068cfb700 1 --2- 172.21.15.86:0/2506598545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60641069f0 0x7f6064106e10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f606000aa40 tx=0x7f606000ad50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5292d78802be409c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.078+0000 7f605b7fe700 1 -- 172.21.15.86:0/2506598545 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f606000b040 con 0x7f60641069f0 2024-04-21T08:28:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.078+0000 7f605b7fe700 1 -- 172.21.15.86:0/2506598545 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f606000c8f0 con 0x7f60641069f0 2024-04-21T08:28:49.847 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.078+0000 7f605b7fe700 1 -- 172.21.15.86:0/2506598545 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6060003b80 con 0x7f60641069f0 2024-04-21T08:28:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.079+0000 7f606af5f700 1 -- 172.21.15.86:0/2506598545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60641069f0 msgr2=0x7f6064106e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.079+0000 7f606af5f700 1 --2- 172.21.15.86:0/2506598545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60641069f0 0x7f6064106e10 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f606000aa40 tx=0x7f606000ad50 comp rx=0 tx=0).stop 2024-04-21T08:28:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.079+0000 7f606af5f700 1 -- 172.21.15.86:0/2506598545 shutdown_connections 2024-04-21T08:28:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.079+0000 7f606af5f700 1 --2- 172.21.15.86:0/2506598545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60641069f0 0x7f6064106e10 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.079+0000 7f606af5f700 1 -- 172.21.15.86:0/2506598545 >> 172.21.15.86:0/2506598545 conn(0x7f6064101fb0 msgr2=0x7f60641043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.079+0000 7f606af5f700 1 -- 172.21.15.86:0/2506598545 shutdown_connections 2024-04-21T08:28:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.079+0000 7f606af5f700 1 -- 172.21.15.86:0/2506598545 wait complete. 2024-04-21T08:28:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.079+0000 7f606af5f700 1 Processor -- start 2024-04-21T08:28:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.079+0000 7f606af5f700 1 -- start start 2024-04-21T08:28:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.080+0000 7f606af5f700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60641069f0 0x7f6064199fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.080+0000 7f606af5f700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6064108310 con 0x7f60641069f0 2024-04-21T08:28:49.848 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.080+0000 7f6068cfb700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60641069f0 0x7f6064199fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.080+0000 7f6068cfb700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60641069f0 0x7f6064199fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53420/0 (socket says 172.21.15.86:53420) 2024-04-21T08:28:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.080+0000 7f6068cfb700 1 -- 172.21.15.86:0/399228401 learned_addr learned my addr 172.21.15.86:0/399228401 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:28:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.080+0000 7f6068cfb700 1 -- 172.21.15.86:0/399228401 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606419a510 con 0x7f60641069f0 2024-04-21T08:28:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.080+0000 7f6068cfb700 1 --2- 172.21.15.86:0/399228401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60641069f0 0x7f6064199fd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6060006fa0 tx=0x7f606000cfd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.080+0000 7f6059ffb700 1 -- 172.21.15.86:0/399228401 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6060027070 con 0x7f60641069f0 2024-04-21T08:28:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.080+0000 7f606af5f700 1 -- 172.21.15.86:0/399228401 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f606419a7a0 con 0x7f60641069f0 2024-04-21T08:28:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.080+0000 7f6059ffb700 1 -- 172.21.15.86:0/399228401 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f60600043b0 con 0x7f60641069f0 2024-04-21T08:28:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.080+0000 7f6059ffb700 1 -- 172.21.15.86:0/399228401 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6060010560 con 0x7f60641069f0 2024-04-21T08:28:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.080+0000 7f606af5f700 1 -- 172.21.15.86:0/399228401 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f606419d490 con 0x7f60641069f0 2024-04-21T08:28:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.081+0000 7f6059ffb700 1 -- 172.21.15.86:0/399228401 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f606002a070 con 0x7f60641069f0 2024-04-21T08:28:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.081+0000 7f606af5f700 1 -- 172.21.15.86:0/399228401 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60640640a0 con 0x7f60641069f0 2024-04-21T08:28:49.849 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.081+0000 7f6059ffb700 1 --2- 172.21.15.86:0/399228401 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7f604c039620 0x7f604c03bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:49.850 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.082+0000 7f6059ffb700 1 -- 172.21.15.86:0/399228401 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f606001f070 con 0x7f60641069f0 2024-04-21T08:28:49.850 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.082+0000 7f605bfff700 1 --2- 172.21.15.86:0/399228401 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7f604c039620 0x7f604c03bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:49.850 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.082+0000 7f605bfff700 1 --2- 172.21.15.86:0/399228401 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7f604c039620 0x7f604c03bae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6054006fd0 tx=0x7f6054006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:49.850 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.087+0000 7f6059ffb700 1 -- 172.21.15.86:0/399228401 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6060014070 con 0x7f60641069f0 2024-04-21T08:28:49.850 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.281+0000 7f606af5f700 1 -- 172.21.15.86:0/399228401 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config assimilate-conf"} v 0) -- 0x7f60640008d0 con 0x7f60641069f0 2024-04-21T08:28:49.850 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.282+0000 7f6059ffb700 1 -- 172.21.15.86:0/399228401 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config assimilate-conf"}]=0 v3) ==== 70+0+409 (secure 0 0 0) 0x7f606002b7b0 con 0x7f60641069f0 2024-04-21T08:28:49.850 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.284+0000 7f606af5f700 1 -- 172.21.15.86:0/399228401 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7f604c039620 msgr2=0x7f604c03bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:49.850 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.284+0000 7f606af5f700 1 --2- 172.21.15.86:0/399228401 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7f604c039620 0x7f604c03bae0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f6054006fd0 tx=0x7f6054006e40 comp rx=0 tx=0).stop 2024-04-21T08:28:49.850 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.284+0000 7f606af5f700 1 -- 172.21.15.86:0/399228401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60641069f0 msgr2=0x7f6064199fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:49.850 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.284+0000 7f606af5f700 1 --2- 172.21.15.86:0/399228401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60641069f0 0x7f6064199fd0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f6060006fa0 tx=0x7f606000cfd0 comp rx=0 tx=0).stop 2024-04-21T08:28:49.850 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.284+0000 7f606af5f700 1 -- 172.21.15.86:0/399228401 shutdown_connections 2024-04-21T08:28:49.850 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.284+0000 7f606af5f700 1 --2- 172.21.15.86:0/399228401 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7f604c039620 0x7f604c03bae0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:49.851 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.284+0000 7f606af5f700 1 --2- 172.21.15.86:0/399228401 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60641069f0 0x7f6064199fd0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:49.851 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.284+0000 7f606af5f700 1 -- 172.21.15.86:0/399228401 >> 172.21.15.86:0/399228401 conn(0x7f6064101fb0 msgr2=0x7f6064190c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:49.851 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.285+0000 7f606af5f700 1 -- 172.21.15.86:0/399228401 shutdown_connections 2024-04-21T08:28:49.851 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:49.285+0000 7f606af5f700 1 -- 172.21.15.86:0/399228401 wait complete. 2024-04-21T08:28:49.851 INFO:teuthology.orchestra.run.smithi086.stdout:Enabling cephadm module... 2024-04-21T08:28:50.824 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:50 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/168784971' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "cephadm"}]: dispatch 2024-04-21T08:28:50.824 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:50 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: ignoring --setuser ceph since I am not root 2024-04-21T08:28:50.824 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:50 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: ignoring --setgroup ceph since I am not root 2024-04-21T08:28:50.824 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:50 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:50.726+0000 7f4936bc3200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-21T08:28:51.116 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:50 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:50.914+0000 7f4936bc3200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:28:51.116 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:51 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:51.015+0000 7f4936bc3200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-21T08:28:51.140 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.250+0000 7f66e778b700 1 Processor -- start 2024-04-21T08:28:51.140 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.250+0000 7f66e778b700 1 -- start start 2024-04-21T08:28:51.140 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.250+0000 7f66e778b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66e0106660 0x7f66e0106a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:51.140 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.250+0000 7f66e778b700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f66e0107050 con 0x7f66e0106660 2024-04-21T08:28:51.140 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.250+0000 7f66e5527700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66e0106660 0x7f66e0106a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:51.140 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.250+0000 7f66e5527700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66e0106660 0x7f66e0106a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49702/0 (socket says 172.21.15.86:49702) 2024-04-21T08:28:51.140 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.250+0000 7f66e5527700 1 -- 172.21.15.86:0/277525044 learned_addr learned my addr 172.21.15.86:0/277525044 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:28:51.140 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.250+0000 7f66e5527700 1 -- 172.21.15.86:0/277525044 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e01095f0 con 0x7f66e0106660 2024-04-21T08:28:51.140 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.251+0000 7f66e5527700 1 --2- 172.21.15.86:0/277525044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66e0106660 0x7f66e0106a80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f66d00098a0 tx=0x7f66d0009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc4f6023cd27a8e1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:51.140 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.251+0000 7f66d7fff700 1 -- 172.21.15.86:0/277525044 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f66d0004160 con 0x7f66e0106660 2024-04-21T08:28:51.140 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.251+0000 7f66d7fff700 1 -- 172.21.15.86:0/277525044 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f66d0004300 con 0x7f66e0106660 2024-04-21T08:28:51.140 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.251+0000 7f66d7fff700 1 -- 172.21.15.86:0/277525044 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f66d0011af0 con 0x7f66e0106660 2024-04-21T08:28:51.140 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.251+0000 7f66e778b700 1 -- 172.21.15.86:0/277525044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66e0106660 msgr2=0x7f66e0106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:51.141 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.251+0000 7f66e778b700 1 --2- 172.21.15.86:0/277525044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66e0106660 0x7f66e0106a80 secure :-1 s=READY pgs=22 cs=0 l=1 rev1=1 crypto rx=0x7f66d00098a0 tx=0x7f66d0009bb0 comp rx=0 tx=0).stop 2024-04-21T08:28:51.141 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.251+0000 7f66e778b700 1 -- 172.21.15.86:0/277525044 shutdown_connections 2024-04-21T08:28:51.141 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.251+0000 7f66e778b700 1 --2- 172.21.15.86:0/277525044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66e0106660 0x7f66e0106a80 unknown :-1 s=CLOSED pgs=22 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:51.141 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.251+0000 7f66e778b700 1 -- 172.21.15.86:0/277525044 >> 172.21.15.86:0/277525044 conn(0x7f66e0101fb0 msgr2=0x7f66e01043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:51.141 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.251+0000 7f66e778b700 1 -- 172.21.15.86:0/277525044 shutdown_connections 2024-04-21T08:28:51.141 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.251+0000 7f66e778b700 1 -- 172.21.15.86:0/277525044 wait complete. 2024-04-21T08:28:51.141 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.252+0000 7f66e778b700 1 Processor -- start 2024-04-21T08:28:51.141 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.252+0000 7f66e778b700 1 -- start start 2024-04-21T08:28:51.141 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.252+0000 7f66e778b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66e0106660 0x7f66e0108e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:51.141 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.252+0000 7f66e778b700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f66e010c9a0 con 0x7f66e0106660 2024-04-21T08:28:51.141 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.252+0000 7f66e5527700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66e0106660 0x7f66e0108e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:51.141 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.252+0000 7f66e5527700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66e0106660 0x7f66e0108e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49718/0 (socket says 172.21.15.86:49718) 2024-04-21T08:28:51.141 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.252+0000 7f66e5527700 1 -- 172.21.15.86:0/168784971 learned_addr learned my addr 172.21.15.86:0/168784971 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:28:51.141 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.252+0000 7f66e5527700 1 -- 172.21.15.86:0/168784971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e01093b0 con 0x7f66e0106660 2024-04-21T08:28:51.142 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.253+0000 7f66e5527700 1 --2- 172.21.15.86:0/168784971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66e0106660 0x7f66e0108e70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f66d0003f40 tx=0x7f66d0011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:51.142 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.253+0000 7f66d67fc700 1 -- 172.21.15.86:0/168784971 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f66d00038b0 con 0x7f66e0106660 2024-04-21T08:28:51.142 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.253+0000 7f66d67fc700 1 -- 172.21.15.86:0/168784971 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f66d0003a50 con 0x7f66e0106660 2024-04-21T08:28:51.142 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.253+0000 7f66e778b700 1 -- 172.21.15.86:0/168784971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66e01075a0 con 0x7f66e0106660 2024-04-21T08:28:51.142 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.253+0000 7f66d67fc700 1 -- 172.21.15.86:0/168784971 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f66d001a5c0 con 0x7f66e0106660 2024-04-21T08:28:51.142 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.253+0000 7f66e778b700 1 -- 172.21.15.86:0/168784971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66e0107ae0 con 0x7f66e0106660 2024-04-21T08:28:51.142 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.254+0000 7f66e778b700 1 -- 172.21.15.86:0/168784971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66e00640a0 con 0x7f66e0106660 2024-04-21T08:28:51.142 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.254+0000 7f66d67fc700 1 -- 172.21.15.86:0/168784971 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 4) ==== 46547+0+0 (secure 0 0 0) 0x7f66d0024430 con 0x7f66e0106660 2024-04-21T08:28:51.142 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.254+0000 7f66d67fc700 1 --2- 172.21.15.86:0/168784971 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7f66cc0395d0 0x7f66cc03ba90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:51.142 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.254+0000 7f66d67fc700 1 -- 172.21.15.86:0/168784971 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f66d004da80 con 0x7f66e0106660 2024-04-21T08:28:51.142 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.254+0000 7f66e4d26700 1 --2- 172.21.15.86:0/168784971 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7f66cc0395d0 0x7f66cc03ba90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:51.142 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.254+0000 7f66e4d26700 1 --2- 172.21.15.86:0/168784971 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7f66cc0395d0 0x7f66cc03ba90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f66dc006fd0 tx=0x7f66dc006e40 comp rx=0 tx=0).ready entity=mgr.14100 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:51.142 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.259+0000 7f66d67fc700 1 -- 172.21.15.86:0/168784971 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f66d004fe20 con 0x7f66e0106660 2024-04-21T08:28:51.142 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.510+0000 7f66e778b700 1 -- 172.21.15.86:0/168784971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "cephadm"} v 0) -- 0x7f66e0107d60 con 0x7f66e0106660 2024-04-21T08:28:51.143 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.562+0000 7f66d67fc700 1 -- 172.21.15.86:0/168784971 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "cephadm"}]=0 v5) ==== 86+0+0 (secure 0 0 0) 0x7f66d004d4a0 con 0x7f66e0106660 2024-04-21T08:28:51.143 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.563+0000 7f66d67fc700 1 -- 172.21.15.86:0/168784971 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f66d002d910 con 0x7f66e0106660 2024-04-21T08:28:51.143 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.564+0000 7f66e778b700 1 -- 172.21.15.86:0/168784971 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7f66cc0395d0 msgr2=0x7f66cc03ba90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:51.143 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.564+0000 7f66e778b700 1 --2- 172.21.15.86:0/168784971 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7f66cc0395d0 0x7f66cc03ba90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f66dc006fd0 tx=0x7f66dc006e40 comp rx=0 tx=0).stop 2024-04-21T08:28:51.143 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.564+0000 7f66e778b700 1 -- 172.21.15.86:0/168784971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66e0106660 msgr2=0x7f66e0108e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:51.143 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.564+0000 7f66e778b700 1 --2- 172.21.15.86:0/168784971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66e0106660 0x7f66e0108e70 secure :-1 s=READY pgs=23 cs=0 l=1 rev1=1 crypto rx=0x7f66d0003f40 tx=0x7f66d0011890 comp rx=0 tx=0).stop 2024-04-21T08:28:51.143 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.564+0000 7f66e778b700 1 -- 172.21.15.86:0/168784971 shutdown_connections 2024-04-21T08:28:51.143 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.564+0000 7f66e778b700 1 --2- 172.21.15.86:0/168784971 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7f66cc0395d0 0x7f66cc03ba90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:51.143 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.564+0000 7f66e778b700 1 --2- 172.21.15.86:0/168784971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66e0106660 0x7f66e0108e70 unknown :-1 s=CLOSED pgs=23 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:51.143 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.564+0000 7f66e778b700 1 -- 172.21.15.86:0/168784971 >> 172.21.15.86:0/168784971 conn(0x7f66e0101fb0 msgr2=0x7f66e01029f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:51.143 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.564+0000 7f66e778b700 1 -- 172.21.15.86:0/168784971 shutdown_connections 2024-04-21T08:28:51.143 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:50.564+0000 7f66e778b700 1 -- 172.21.15.86:0/168784971 wait complete. 2024-04-21T08:28:51.414 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:51 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:51.327+0000 7f4936bc3200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-21T08:28:51.415 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:51 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:51.415+0000 7f4936bc3200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T08:28:51.793 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:51 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/168784971' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "cephadm"}]': finished 2024-04-21T08:28:51.793 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:51 smithi086 ceph-mon[27939]: mgrmap e5: x(active, since 6s) 2024-04-21T08:28:52.398 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-04-21T08:28:52.398 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 5, 2024-04-21T08:28:52.398 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-21T08:28:52.398 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-21T08:28:52.398 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-21T08:28:52.398 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-04-21T08:28:52.398 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.588+0000 7fc5b18f1700 1 Processor -- start 2024-04-21T08:28:52.398 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.588+0000 7fc5b18f1700 1 -- start start 2024-04-21T08:28:52.399 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.588+0000 7fc5b18f1700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5ac108440 0x7fc5ac108860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:52.399 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.588+0000 7fc5b18f1700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac108e30 con 0x7fc5ac108440 2024-04-21T08:28:52.399 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.588+0000 7fc5aaffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5ac108440 0x7fc5ac108860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:52.399 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.588+0000 7fc5aaffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5ac108440 0x7fc5ac108860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49744/0 (socket says 172.21.15.86:49744) 2024-04-21T08:28:52.399 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.588+0000 7fc5aaffd700 1 -- 172.21.15.86:0/3549301506 learned_addr learned my addr 172.21.15.86:0/3549301506 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:28:52.399 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.589+0000 7fc5aaffd700 1 -- 172.21.15.86:0/3549301506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5ac1095f0 con 0x7fc5ac108440 2024-04-21T08:28:52.399 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.589+0000 7fc5aaffd700 1 --2- 172.21.15.86:0/3549301506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5ac108440 0x7fc5ac108860 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc59c0098a0 tx=0x7fc59c009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2aac3ec67a950 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:52.399 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.589+0000 7fc5a9ffb700 1 -- 172.21.15.86:0/3549301506 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc59c004160 con 0x7fc5ac108440 2024-04-21T08:28:52.399 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.589+0000 7fc5a9ffb700 1 -- 172.21.15.86:0/3549301506 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc59c004300 con 0x7fc5ac108440 2024-04-21T08:28:52.399 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.589+0000 7fc5a9ffb700 1 -- 172.21.15.86:0/3549301506 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc59c011af0 con 0x7fc5ac108440 2024-04-21T08:28:52.399 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.590+0000 7fc5b18f1700 1 -- 172.21.15.86:0/3549301506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5ac108440 msgr2=0x7fc5ac108860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:52.399 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.590+0000 7fc5b18f1700 1 --2- 172.21.15.86:0/3549301506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5ac108440 0x7fc5ac108860 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7fc59c0098a0 tx=0x7fc59c009bb0 comp rx=0 tx=0).stop 2024-04-21T08:28:52.399 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.590+0000 7fc5b18f1700 1 -- 172.21.15.86:0/3549301506 shutdown_connections 2024-04-21T08:28:52.399 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.590+0000 7fc5b18f1700 1 --2- 172.21.15.86:0/3549301506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5ac108440 0x7fc5ac108860 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:52.400 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.590+0000 7fc5b18f1700 1 -- 172.21.15.86:0/3549301506 >> 172.21.15.86:0/3549301506 conn(0x7fc5ac103a60 msgr2=0x7fc5ac105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:52.400 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.590+0000 7fc5b18f1700 1 -- 172.21.15.86:0/3549301506 shutdown_connections 2024-04-21T08:28:52.400 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.590+0000 7fc5b18f1700 1 -- 172.21.15.86:0/3549301506 wait complete. 2024-04-21T08:28:52.400 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.590+0000 7fc5b18f1700 1 Processor -- start 2024-04-21T08:28:52.400 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.591+0000 7fc5b18f1700 1 -- start start 2024-04-21T08:28:52.400 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.591+0000 7fc5b18f1700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5ac108440 0x7fc5ac19c290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:52.400 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.591+0000 7fc5b18f1700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc5ac10c820 con 0x7fc5ac108440 2024-04-21T08:28:52.400 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.591+0000 7fc5aaffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5ac108440 0x7fc5ac19c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:52.400 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.591+0000 7fc5aaffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5ac108440 0x7fc5ac19c290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49754/0 (socket says 172.21.15.86:49754) 2024-04-21T08:28:52.400 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.591+0000 7fc5aaffd700 1 -- 172.21.15.86:0/4120508968 learned_addr learned my addr 172.21.15.86:0/4120508968 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:28:52.400 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.591+0000 7fc5aaffd700 1 -- 172.21.15.86:0/4120508968 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5ac19c7d0 con 0x7fc5ac108440 2024-04-21T08:28:52.400 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.591+0000 7fc5aaffd700 1 --2- 172.21.15.86:0/4120508968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5ac108440 0x7fc5ac19c290 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc59c0117b0 tx=0x7fc59c011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:52.401 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.591+0000 7fc5b08ef700 1 -- 172.21.15.86:0/4120508968 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc59c003af0 con 0x7fc5ac108440 2024-04-21T08:28:52.401 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.591+0000 7fc5b08ef700 1 -- 172.21.15.86:0/4120508968 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fc59c020070 con 0x7fc5ac108440 2024-04-21T08:28:52.401 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.591+0000 7fc5b18f1700 1 -- 172.21.15.86:0/4120508968 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5ac19ca60 con 0x7fc5ac108440 2024-04-21T08:28:52.401 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.591+0000 7fc5b08ef700 1 -- 172.21.15.86:0/4120508968 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fc59c019630 con 0x7fc5ac108440 2024-04-21T08:28:52.401 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.592+0000 7fc5b18f1700 1 -- 172.21.15.86:0/4120508968 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5ac19f750 con 0x7fc5ac108440 2024-04-21T08:28:52.401 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.592+0000 7fc5b18f1700 1 -- 172.21.15.86:0/4120508968 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5ac10c9a0 con 0x7fc5ac108440 2024-04-21T08:28:52.401 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.593+0000 7fc5b08ef700 1 -- 172.21.15.86:0/4120508968 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7fc59c0198f0 con 0x7fc5ac108440 2024-04-21T08:28:52.402 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.593+0000 7fc5b08ef700 1 --2- 172.21.15.86:0/4120508968 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7fc594039670 0x7fc59403bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:52.402 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.593+0000 7fc5b08ef700 1 -- 172.21.15.86:0/4120508968 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7fc59c04dd80 con 0x7fc5ac108440 2024-04-21T08:28:52.402 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.593+0000 7fc5aa7fc700 1 -- 172.21.15.86:0/4120508968 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7fc594039670 msgr2=0x7fc59403bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2386990413 2024-04-21T08:28:52.402 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.593+0000 7fc5aa7fc700 1 --2- 172.21.15.86:0/4120508968 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7fc594039670 0x7fc59403bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:28:52.402 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.598+0000 7fc5b08ef700 1 -- 172.21.15.86:0/4120508968 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc59c04d7a0 con 0x7fc5ac108440 2024-04-21T08:28:52.403 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.794+0000 7fc5aa7fc700 1 -- 172.21.15.86:0/4120508968 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7fc594039670 msgr2=0x7fc59403bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2386990413 2024-04-21T08:28:52.403 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.794+0000 7fc5aa7fc700 1 --2- 172.21.15.86:0/4120508968 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7fc594039670 0x7fc59403bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T08:28:52.403 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.894+0000 7fc5b18f1700 1 -- 172.21.15.86:0/4120508968 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fc5ac19fda0 con 0x7fc5ac108440 2024-04-21T08:28:52.403 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.895+0000 7fc5b08ef700 1 -- 172.21.15.86:0/4120508968 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v5) ==== 56+0+88 (secure 0 0 0) 0x7fc59c02bd00 con 0x7fc5ac108440 2024-04-21T08:28:52.403 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.897+0000 7fc5b18f1700 1 -- 172.21.15.86:0/4120508968 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7fc594039670 msgr2=0x7fc59403bb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:28:52.403 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.897+0000 7fc5b18f1700 1 --2- 172.21.15.86:0/4120508968 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7fc594039670 0x7fc59403bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:52.403 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.897+0000 7fc5b18f1700 1 -- 172.21.15.86:0/4120508968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5ac108440 msgr2=0x7fc5ac19c290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:52.404 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.897+0000 7fc5b18f1700 1 --2- 172.21.15.86:0/4120508968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5ac108440 0x7fc5ac19c290 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7fc59c0117b0 tx=0x7fc59c011b70 comp rx=0 tx=0).stop 2024-04-21T08:28:52.404 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.897+0000 7fc5b18f1700 1 -- 172.21.15.86:0/4120508968 shutdown_connections 2024-04-21T08:28:52.404 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.897+0000 7fc5b18f1700 1 --2- 172.21.15.86:0/4120508968 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7fc594039670 0x7fc59403bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:52.404 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.897+0000 7fc5b18f1700 1 --2- 172.21.15.86:0/4120508968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5ac108440 0x7fc5ac19c290 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:52.404 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.897+0000 7fc5b18f1700 1 -- 172.21.15.86:0/4120508968 >> 172.21.15.86:0/4120508968 conn(0x7fc5ac103a60 msgr2=0x7fc5ac104530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:52.404 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.897+0000 7fc5b18f1700 1 -- 172.21.15.86:0/4120508968 shutdown_connections 2024-04-21T08:28:52.404 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:51.897+0000 7fc5b18f1700 1 -- 172.21.15.86:0/4120508968 wait complete. 2024-04-21T08:28:52.404 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for the mgr to restart... 2024-04-21T08:28:52.404 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mgr epoch 5... 2024-04-21T08:28:52.797 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:52 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/4120508968' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-21T08:28:52.797 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:52 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:52.625+0000 7f4936bc3200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T08:28:52.797 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:52 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:52.797+0000 7f4936bc3200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T08:28:53.116 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:52 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:52.967+0000 7f4936bc3200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T08:28:53.535 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:53 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:53.249+0000 7f4936bc3200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T08:28:53.535 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:53 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:53.339+0000 7f4936bc3200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T08:28:53.535 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:53 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:53.536+0000 7f4936bc3200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T08:28:53.866 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:53 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:53.725+0000 7f4936bc3200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:28:54.616 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:54 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:54.310+0000 7f4936bc3200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T08:28:55.027 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:54 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:54.730+0000 7f4936bc3200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T08:28:55.027 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:54 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:54.828+0000 7f4936bc3200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T08:28:55.028 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:55 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:55.028+0000 7f4936bc3200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T08:28:55.842 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:55 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:55.566+0000 7f4936bc3200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T08:28:55.842 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:55 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:55.658+0000 7f4936bc3200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T08:28:56.116 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:55 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:55.901+0000 7f4936bc3200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T08:28:56.116 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:56 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:56.034+0000 7f4936bc3200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T08:28:56.572 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:56 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:56.132+0000 7f4936bc3200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T08:28:56.572 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:56 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:56.248+0000 7f4936bc3200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T08:28:56.572 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:56 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:56.572+0000 7f4936bc3200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T08:28:57.434 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:57 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:57.166+0000 7f4936bc3200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T08:28:57.434 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:28:57 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:28:57.382+0000 7f4936bc3200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T08:28:57.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:57 smithi086 ceph-mon[27939]: Active manager daemon x restarted 2024-04-21T08:28:57.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:57 smithi086 ceph-mon[27939]: Activating manager daemon x 2024-04-21T08:28:57.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:57 smithi086 ceph-mon[27939]: osdmap e2: 0 total, 0 up, 0 in 2024-04-21T08:28:57.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:57 smithi086 ceph-mon[27939]: mgrmap e6: x(active, starting, since 0.00335409s) 2024-04-21T08:28:57.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:57 smithi086 ceph-mon[27939]: from='mgr.14120 172.21.15.86:0/1976532381' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:28:57.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:57 smithi086 ceph-mon[27939]: from='mgr.14120 172.21.15.86:0/1976532381' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-21T08:28:57.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:57 smithi086 ceph-mon[27939]: from='mgr.14120 172.21.15.86:0/1976532381' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-21T08:28:57.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:57 smithi086 ceph-mon[27939]: from='mgr.14120 172.21.15.86:0/1976532381' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-21T08:28:57.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:57 smithi086 ceph-mon[27939]: from='mgr.14120 172.21.15.86:0/1976532381' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-21T08:28:57.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:57 smithi086 ceph-mon[27939]: Manager daemon x is now available 2024-04-21T08:28:57.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:57 smithi086 ceph-mon[27939]: from='mgr.14120 172.21.15.86:0/1976532381' entity='mgr.x' 2024-04-21T08:28:57.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:57 smithi086 ceph-mon[27939]: from='mgr.14120 172.21.15.86:0/1976532381' entity='mgr.x' 2024-04-21T08:28:57.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:57 smithi086 ceph-mon[27939]: from='mgr.14120 172.21.15.86:0/1976532381' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:28:57.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:57 smithi086 ceph-mon[27939]: from='mgr.14120 172.21.15.86:0/1976532381' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:28:57.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:57 smithi086 ceph-mon[27939]: from='mgr.14120 172.21.15.86:0/1976532381' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-21T08:28:57.868 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:57 smithi086 ceph-mon[27939]: from='mgr.14120 172.21.15.86:0/1976532381' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-21T08:28:58.694 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:58 smithi086 ceph-mon[27939]: Found migration_current of "None". Setting to last migration. 2024-04-21T08:28:58.694 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:58 smithi086 ceph-mon[27939]: from='mgr.14120 172.21.15.86:0/1976532381' entity='mgr.x' 2024-04-21T08:28:58.694 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:58 smithi086 ceph-mon[27939]: from='mgr.14120 172.21.15.86:0/1976532381' entity='mgr.x' 2024-04-21T08:28:58.694 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:58 smithi086 ceph-mon[27939]: from='mgr.14120 172.21.15.86:0/1976532381' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:28:58.694 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:58 smithi086 ceph-mon[27939]: mgrmap e7: x(active, since 1.00654s) 2024-04-21T08:28:58.972 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-04-21T08:28:58.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 7, 2024-04-21T08:28:58.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-21T08:28:58.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-04-21T08:28:58.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.834+0000 7f006f6ee700 1 Processor -- start 2024-04-21T08:28:58.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.834+0000 7f006f6ee700 1 -- start start 2024-04-21T08:28:58.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.834+0000 7f006f6ee700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0068108e00 0x7f006810b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:58.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.834+0000 7f006f6ee700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f006807be80 con 0x7f0068108e00 2024-04-21T08:28:58.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.835+0000 7f006d48a700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0068108e00 0x7f006810b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:58.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.835+0000 7f006d48a700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0068108e00 0x7f006810b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49766/0 (socket says 172.21.15.86:49766) 2024-04-21T08:28:58.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.835+0000 7f006d48a700 1 -- 172.21.15.86:0/962655053 learned_addr learned my addr 172.21.15.86:0/962655053 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:28:58.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.835+0000 7f006d48a700 1 -- 172.21.15.86:0/962655053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f006810b730 con 0x7f0068108e00 2024-04-21T08:28:58.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.835+0000 7f006d48a700 1 --2- 172.21.15.86:0/962655053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0068108e00 0x7f006810b1f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f00580098a0 tx=0x7f0058009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa3e6863a577d314 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:58.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.835+0000 7f005ffff700 1 -- 172.21.15.86:0/962655053 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0058004160 con 0x7f0068108e00 2024-04-21T08:28:58.973 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.835+0000 7f005ffff700 1 -- 172.21.15.86:0/962655053 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0058004300 con 0x7f0068108e00 2024-04-21T08:28:58.974 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.835+0000 7f005ffff700 1 -- 172.21.15.86:0/962655053 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0058011af0 con 0x7f0068108e00 2024-04-21T08:28:58.974 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.836+0000 7f006f6ee700 1 -- 172.21.15.86:0/962655053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0068108e00 msgr2=0x7f006810b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:58.974 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.836+0000 7f006f6ee700 1 --2- 172.21.15.86:0/962655053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0068108e00 0x7f006810b1f0 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f00580098a0 tx=0x7f0058009bb0 comp rx=0 tx=0).stop 2024-04-21T08:28:58.974 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.836+0000 7f006f6ee700 1 -- 172.21.15.86:0/962655053 shutdown_connections 2024-04-21T08:28:58.974 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.836+0000 7f006f6ee700 1 --2- 172.21.15.86:0/962655053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0068108e00 0x7f006810b1f0 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:58.974 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.836+0000 7f006f6ee700 1 -- 172.21.15.86:0/962655053 >> 172.21.15.86:0/962655053 conn(0x7f00681027d0 msgr2=0x7f0068104c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:58.974 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.836+0000 7f006f6ee700 1 -- 172.21.15.86:0/962655053 shutdown_connections 2024-04-21T08:28:58.974 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.836+0000 7f006f6ee700 1 -- 172.21.15.86:0/962655053 wait complete. 2024-04-21T08:28:58.974 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.837+0000 7f006f6ee700 1 Processor -- start 2024-04-21T08:28:58.974 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.837+0000 7f006f6ee700 1 -- start start 2024-04-21T08:28:58.974 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.837+0000 7f006f6ee700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0068108e00 0x7f006819c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:58.974 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.837+0000 7f006f6ee700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f006810c680 con 0x7f0068108e00 2024-04-21T08:28:58.975 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.837+0000 7f006d48a700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0068108e00 0x7f006819c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:58.975 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.837+0000 7f006d48a700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0068108e00 0x7f006819c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49768/0 (socket says 172.21.15.86:49768) 2024-04-21T08:28:58.975 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.837+0000 7f006d48a700 1 -- 172.21.15.86:0/2519177089 learned_addr learned my addr 172.21.15.86:0/2519177089 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:28:58.975 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.838+0000 7f006d48a700 1 -- 172.21.15.86:0/2519177089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f006819c760 con 0x7f0068108e00 2024-04-21T08:28:58.975 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.838+0000 7f006d48a700 1 --2- 172.21.15.86:0/2519177089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0068108e00 0x7f006819c220 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0058009fb0 tx=0x7f0058011940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:58.975 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.838+0000 7f005e7fc700 1 -- 172.21.15.86:0/2519177089 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f00580036a0 con 0x7f0068108e00 2024-04-21T08:28:58.975 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.838+0000 7f005e7fc700 1 -- 172.21.15.86:0/2519177089 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f005801a430 con 0x7f0068108e00 2024-04-21T08:28:58.975 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.838+0000 7f006f6ee700 1 -- 172.21.15.86:0/2519177089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f006819ca50 con 0x7f0068108e00 2024-04-21T08:28:58.975 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.838+0000 7f005e7fc700 1 -- 172.21.15.86:0/2519177089 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0058019550 con 0x7f0068108e00 2024-04-21T08:28:58.975 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.838+0000 7f006f6ee700 1 -- 172.21.15.86:0/2519177089 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f006819f6e0 con 0x7f0068108e00 2024-04-21T08:28:58.975 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.839+0000 7f005e7fc700 1 -- 172.21.15.86:0/2519177089 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 5) ==== 46558+0+0 (secure 0 0 0) 0x7f0058019850 con 0x7f0068108e00 2024-04-21T08:28:58.976 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.839+0000 7f005e7fc700 1 --2- 172.21.15.86:0/2519177089 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7f0054039620 0x7f005403bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:58.976 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.839+0000 7f005e7fc700 1 -- 172.21.15.86:0/2519177089 --> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f005403c1f0 con 0x7f0054039620 2024-04-21T08:28:58.976 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.839+0000 7f005e7fc700 1 -- 172.21.15.86:0/2519177089 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(1..1 src has 1..1) ==== 725+0+0 (secure 0 0 0) 0x7f005804e850 con 0x7f0068108e00 2024-04-21T08:28:58.976 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.839+0000 7f006cc89700 1 -- 172.21.15.86:0/2519177089 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7f0054039620 msgr2=0x7f005403bae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2386990413 2024-04-21T08:28:58.976 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:52.839+0000 7f006cc89700 1 --2- 172.21.15.86:0/2519177089 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7f0054039620 0x7f005403bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:28:58.976 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.040+0000 7f006cc89700 1 -- 172.21.15.86:0/2519177089 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7f0054039620 msgr2=0x7f005403bae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2386990413 2024-04-21T08:28:58.976 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.040+0000 7f006cc89700 1 --2- 172.21.15.86:0/2519177089 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7f0054039620 0x7f005403bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T08:28:58.976 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.441+0000 7f006cc89700 1 -- 172.21.15.86:0/2519177089 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7f0054039620 msgr2=0x7f005403bae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2386990413 2024-04-21T08:28:58.976 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:53.441+0000 7f006cc89700 1 --2- 172.21.15.86:0/2519177089 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7f0054039620 0x7f005403bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-21T08:28:58.976 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.242+0000 7f006cc89700 1 -- 172.21.15.86:0/2519177089 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7f0054039620 msgr2=0x7f005403bae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2386990413 2024-04-21T08:28:58.976 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:54.242+0000 7f006cc89700 1 --2- 172.21.15.86:0/2519177089 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7f0054039620 0x7f005403bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-21T08:28:58.976 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:55.843+0000 7f006cc89700 1 -- 172.21.15.86:0/2519177089 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7f0054039620 msgr2=0x7f005403bae0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2386990413 2024-04-21T08:28:58.977 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:55.843+0000 7f006cc89700 1 --2- 172.21.15.86:0/2519177089 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7f0054039620 0x7f005403bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-21T08:28:58.977 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:57.385+0000 7f005e7fc700 1 -- 172.21.15.86:0/2519177089 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mgrmap(e 6) ==== 46325+0+0 (secure 0 0 0) 0x7f0058019b90 con 0x7f0068108e00 2024-04-21T08:28:58.977 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:57.386+0000 7f005e7fc700 1 -- 172.21.15.86:0/2519177089 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7f0054039620 msgr2=0x7f005403bae0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:28:58.977 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:57.386+0000 7f005e7fc700 1 --2- 172.21.15.86:0/2519177089 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7f0054039620 0x7f005403bae0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:58.977 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:58.387+0000 7f005e7fc700 1 -- 172.21.15.86:0/2519177089 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f005802cc50 con 0x7f0068108e00 2024-04-21T08:28:58.977 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:58.387+0000 7f005e7fc700 1 --2- 172.21.15.86:0/2519177089 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f005403cf10 0x7f005403f300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:28:58.977 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:58.387+0000 7f005e7fc700 1 -- 172.21.15.86:0/2519177089 --> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f0058047980 con 0x7f005403cf10 2024-04-21T08:28:58.977 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:58.388+0000 7f006cc89700 1 --2- 172.21.15.86:0/2519177089 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f005403cf10 0x7f005403f300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:28:58.977 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:58.388+0000 7f006cc89700 1 --2- 172.21.15.86:0/2519177089 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f005403cf10 0x7f005403f300 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0064003a10 tx=0x7f0064009290 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:28:58.977 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:58.389+0000 7f005e7fc700 1 -- 172.21.15.86:0/2519177089 <== mgr.14120 v2:172.21.15.86:6800/2143206409 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f0058047980 con 0x7f005403cf10 2024-04-21T08:28:58.977 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:58.396+0000 7f006f6ee700 1 -- 172.21.15.86:0/2519177089 --> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f0068069b50 con 0x7f005403cf10 2024-04-21T08:28:58.977 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:58.396+0000 7f005e7fc700 1 -- 172.21.15.86:0/2519177089 <== mgr.14120 v2:172.21.15.86:6800/2143206409 2 ==== command_reply(tid 1: 0 ) ==== 8+0+51 (secure 0 0 0) 0x7f0068069b50 con 0x7f005403cf10 2024-04-21T08:28:58.977 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:58.397+0000 7f006f6ee700 1 -- 172.21.15.86:0/2519177089 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f005403cf10 msgr2=0x7f005403f300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:58.978 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:58.397+0000 7f006f6ee700 1 --2- 172.21.15.86:0/2519177089 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f005403cf10 0x7f005403f300 secure :-1 s=READY pgs=1 cs=0 l=1 rev1=1 crypto rx=0x7f0064003a10 tx=0x7f0064009290 comp rx=0 tx=0).stop 2024-04-21T08:28:58.978 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:58.397+0000 7f006f6ee700 1 -- 172.21.15.86:0/2519177089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0068108e00 msgr2=0x7f006819c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:28:58.978 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:58.397+0000 7f006f6ee700 1 --2- 172.21.15.86:0/2519177089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0068108e00 0x7f006819c220 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7f0058009fb0 tx=0x7f0058011940 comp rx=0 tx=0).stop 2024-04-21T08:28:58.978 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:58.397+0000 7f006f6ee700 1 -- 172.21.15.86:0/2519177089 shutdown_connections 2024-04-21T08:28:58.978 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:58.397+0000 7f006f6ee700 1 --2- 172.21.15.86:0/2519177089 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f005403cf10 0x7f005403f300 unknown :-1 s=CLOSED pgs=1 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:58.978 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:58.397+0000 7f006f6ee700 1 --2- 172.21.15.86:0/2519177089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0068108e00 0x7f006819c220 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:58.978 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:58.397+0000 7f006f6ee700 1 --2- 172.21.15.86:0/2519177089 >> [v2:172.21.15.86:6800/2386990413,v1:172.21.15.86:6801/2386990413] conn(0x7f0054039620 0x7f005403bae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:28:58.978 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:58.397+0000 7f006f6ee700 1 -- 172.21.15.86:0/2519177089 >> 172.21.15.86:0/2519177089 conn(0x7f00681027d0 msgr2=0x7f0068103300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:28:58.978 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:58.397+0000 7f006f6ee700 1 -- 172.21.15.86:0/2519177089 shutdown_connections 2024-04-21T08:28:58.978 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:58.397+0000 7f006f6ee700 1 -- 172.21.15.86:0/2519177089 wait complete. 2024-04-21T08:28:58.978 INFO:teuthology.orchestra.run.smithi086.stdout:mgr epoch 5 is available 2024-04-21T08:28:58.978 INFO:teuthology.orchestra.run.smithi086.stdout:Setting orchestrator backend to cephadm... 2024-04-21T08:28:59.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:59 smithi086 ceph-mon[27939]: [21/Apr/2024:08:28:57] ENGINE Bus STARTING 2024-04-21T08:28:59.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:59 smithi086 ceph-mon[27939]: [21/Apr/2024:08:28:57] ENGINE Serving on http://172.21.15.86:8765 2024-04-21T08:28:59.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:59 smithi086 ceph-mon[27939]: [21/Apr/2024:08:28:58] ENGINE Serving on https://172.21.15.86:7150 2024-04-21T08:28:59.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:59 smithi086 ceph-mon[27939]: [21/Apr/2024:08:28:58] ENGINE Bus STARTED 2024-04-21T08:28:59.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:59 smithi086 ceph-mon[27939]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-21T08:28:59.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:28:59 smithi086 ceph-mon[27939]: from='client.14124 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-21T08:29:00.150 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.389+0000 7f24aadd6700 1 Processor -- start 2024-04-21T08:29:00.151 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.389+0000 7f24aadd6700 1 -- start start 2024-04-21T08:29:00.151 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.389+0000 7f24aadd6700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24a4108420 0x7f24a4108840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:00.151 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.389+0000 7f24aadd6700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f24a4108e10 con 0x7f24a4108420 2024-04-21T08:29:00.151 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.389+0000 7f24a8b72700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24a4108420 0x7f24a4108840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:00.151 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.389+0000 7f24a8b72700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24a4108420 0x7f24a4108840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49830/0 (socket says 172.21.15.86:49830) 2024-04-21T08:29:00.151 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.389+0000 7f24a8b72700 1 -- 172.21.15.86:0/3040328195 learned_addr learned my addr 172.21.15.86:0/3040328195 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:00.151 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.389+0000 7f24a8b72700 1 -- 172.21.15.86:0/3040328195 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a41095d0 con 0x7f24a4108420 2024-04-21T08:29:00.151 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.390+0000 7f24a8b72700 1 --2- 172.21.15.86:0/3040328195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24a4108420 0x7f24a4108840 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f24980098a0 tx=0x7f2498009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c14ac3b3234738 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:00.151 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.390+0000 7f24a37fe700 1 -- 172.21.15.86:0/3040328195 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2498004160 con 0x7f24a4108420 2024-04-21T08:29:00.151 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.390+0000 7f24a37fe700 1 -- 172.21.15.86:0/3040328195 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2498004300 con 0x7f24a4108420 2024-04-21T08:29:00.151 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.390+0000 7f24a37fe700 1 -- 172.21.15.86:0/3040328195 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2498011af0 con 0x7f24a4108420 2024-04-21T08:29:00.151 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.390+0000 7f24aadd6700 1 -- 172.21.15.86:0/3040328195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24a4108420 msgr2=0x7f24a4108840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:00.152 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.390+0000 7f24aadd6700 1 --2- 172.21.15.86:0/3040328195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24a4108420 0x7f24a4108840 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f24980098a0 tx=0x7f2498009bb0 comp rx=0 tx=0).stop 2024-04-21T08:29:00.152 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.390+0000 7f24aadd6700 1 -- 172.21.15.86:0/3040328195 shutdown_connections 2024-04-21T08:29:00.152 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.390+0000 7f24aadd6700 1 --2- 172.21.15.86:0/3040328195 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24a4108420 0x7f24a4108840 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:00.152 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.390+0000 7f24aadd6700 1 -- 172.21.15.86:0/3040328195 >> 172.21.15.86:0/3040328195 conn(0x7f24a41039e0 msgr2=0x7f24a4105e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:00.152 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.391+0000 7f24aadd6700 1 -- 172.21.15.86:0/3040328195 shutdown_connections 2024-04-21T08:29:00.152 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.391+0000 7f24aadd6700 1 -- 172.21.15.86:0/3040328195 wait complete. 2024-04-21T08:29:00.152 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.391+0000 7f24aadd6700 1 Processor -- start 2024-04-21T08:29:00.152 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.391+0000 7f24aadd6700 1 -- start start 2024-04-21T08:29:00.152 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.391+0000 7f24aadd6700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24a4108420 0x7f24a419c1c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:00.152 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.391+0000 7f24aadd6700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f24a410c800 con 0x7f24a4108420 2024-04-21T08:29:00.152 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.391+0000 7f24a8b72700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24a4108420 0x7f24a419c1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:00.152 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.391+0000 7f24a8b72700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24a4108420 0x7f24a419c1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49844/0 (socket says 172.21.15.86:49844) 2024-04-21T08:29:00.152 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.391+0000 7f24a8b72700 1 -- 172.21.15.86:0/583973087 learned_addr learned my addr 172.21.15.86:0/583973087 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:00.152 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.392+0000 7f24a8b72700 1 -- 172.21.15.86:0/583973087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24a419c700 con 0x7f24a4108420 2024-04-21T08:29:00.152 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.392+0000 7f24a8b72700 1 --2- 172.21.15.86:0/583973087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24a4108420 0x7f24a419c1c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2498003f40 tx=0x7f2498011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:00.153 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.392+0000 7f24a1ffb700 1 -- 172.21.15.86:0/583973087 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f24980038b0 con 0x7f24a4108420 2024-04-21T08:29:00.153 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.392+0000 7f24a1ffb700 1 -- 172.21.15.86:0/583973087 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2498003a50 con 0x7f24a4108420 2024-04-21T08:29:00.153 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.392+0000 7f24a1ffb700 1 -- 172.21.15.86:0/583973087 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f249801a5c0 con 0x7f24a4108420 2024-04-21T08:29:00.153 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.392+0000 7f24aadd6700 1 -- 172.21.15.86:0/583973087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24a419c990 con 0x7f24a4108420 2024-04-21T08:29:00.153 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.392+0000 7f24aadd6700 1 -- 172.21.15.86:0/583973087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24a419f680 con 0x7f24a4108420 2024-04-21T08:29:00.153 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.393+0000 7f24a1ffb700 1 -- 172.21.15.86:0/583973087 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 7) ==== 46452+0+0 (secure 0 0 0) 0x7f2498024430 con 0x7f24a4108420 2024-04-21T08:29:00.153 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.393+0000 7f24a1ffb700 1 --2- 172.21.15.86:0/583973087 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f2494039510 0x7f249403b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:00.153 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.393+0000 7f24aadd6700 1 -- 172.21.15.86:0/583973087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24a410c980 con 0x7f24a4108420 2024-04-21T08:29:00.154 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.393+0000 7f24a1ffb700 1 -- 172.21.15.86:0/583973087 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f249804d920 con 0x7f24a4108420 2024-04-21T08:29:00.154 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.393+0000 7f24a3fff700 1 --2- 172.21.15.86:0/583973087 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f2494039510 0x7f249403b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:00.154 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.394+0000 7f24a3fff700 1 --2- 172.21.15.86:0/583973087 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f2494039510 0x7f249403b9d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2490006fd0 tx=0x7f2490006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:00.154 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.399+0000 7f24a1ffb700 1 -- 172.21.15.86:0/583973087 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f249802d910 con 0x7f24a4108420 2024-04-21T08:29:00.154 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.547+0000 7f24a1ffb700 1 -- 172.21.15.86:0/583973087 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f249802d3f0 con 0x7f24a4108420 2024-04-21T08:29:00.154 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.616+0000 7f24aadd6700 1 -- 172.21.15.86:0/583973087 --> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] -- mgr_command(tid 0: {"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}) -- 0x7f24a4104af0 con 0x7f2494039510 2024-04-21T08:29:00.154 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.620+0000 7f24a1ffb700 1 -- 172.21.15.86:0/583973087 <== mgr.14120 v2:172.21.15.86:6800/2143206409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f24a4104af0 con 0x7f2494039510 2024-04-21T08:29:00.154 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.623+0000 7f24aadd6700 1 -- 172.21.15.86:0/583973087 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f2494039510 msgr2=0x7f249403b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:00.154 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.623+0000 7f24aadd6700 1 --2- 172.21.15.86:0/583973087 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f2494039510 0x7f249403b9d0 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f2490006fd0 tx=0x7f2490006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:00.154 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.623+0000 7f24aadd6700 1 -- 172.21.15.86:0/583973087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24a4108420 msgr2=0x7f24a419c1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:00.154 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.623+0000 7f24aadd6700 1 --2- 172.21.15.86:0/583973087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24a4108420 0x7f24a419c1c0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f2498003f40 tx=0x7f2498011890 comp rx=0 tx=0).stop 2024-04-21T08:29:00.154 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.623+0000 7f24aadd6700 1 -- 172.21.15.86:0/583973087 shutdown_connections 2024-04-21T08:29:00.154 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.623+0000 7f24aadd6700 1 --2- 172.21.15.86:0/583973087 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f2494039510 0x7f249403b9d0 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:00.154 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.623+0000 7f24aadd6700 1 --2- 172.21.15.86:0/583973087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f24a4108420 0x7f24a419c1c0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:00.155 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.623+0000 7f24aadd6700 1 -- 172.21.15.86:0/583973087 >> 172.21.15.86:0/583973087 conn(0x7f24a41039e0 msgr2=0x7f24a41043e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:00.155 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.623+0000 7f24aadd6700 1 -- 172.21.15.86:0/583973087 shutdown_connections 2024-04-21T08:29:00.155 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:28:59.623+0000 7f24aadd6700 1 -- 172.21.15.86:0/583973087 wait complete. 2024-04-21T08:29:00.806 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:00 smithi086 ceph-mon[27939]: mgrmap e8: x(active, since 2s) 2024-04-21T08:29:00.806 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:00 smithi086 ceph-mon[27939]: from='mgr.14120 172.21.15.86:0/1976532381' entity='mgr.x' 2024-04-21T08:29:00.806 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:00 smithi086 ceph-mon[27939]: from='mgr.14120 172.21.15.86:0/1976532381' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:29:01.373 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout value unchanged 2024-04-21T08:29:01.373 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.585+0000 7f0fd4cef700 1 Processor -- start 2024-04-21T08:29:01.373 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.585+0000 7f0fd4cef700 1 -- start start 2024-04-21T08:29:01.373 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.586+0000 7f0fd4cef700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fd0107310 0x7f0fd0107730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:01.373 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.586+0000 7f0fd4cef700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd0107d00 con 0x7f0fd0107310 2024-04-21T08:29:01.373 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.586+0000 7f0fce59c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fd0107310 0x7f0fd0107730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:01.373 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.586+0000 7f0fce59c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fd0107310 0x7f0fd0107730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44010/0 (socket says 172.21.15.86:44010) 2024-04-21T08:29:01.374 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.586+0000 7f0fce59c700 1 -- 172.21.15.86:0/449873718 learned_addr learned my addr 172.21.15.86:0/449873718 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:01.374 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.586+0000 7f0fce59c700 1 -- 172.21.15.86:0/449873718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd0108510 con 0x7f0fd0107310 2024-04-21T08:29:01.374 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.586+0000 7f0fce59c700 1 --2- 172.21.15.86:0/449873718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fd0107310 0x7f0fd0107730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0fc40098a0 tx=0x7f0fc4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dc873bec72c4ca8f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:01.374 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.586+0000 7f0fcd59a700 1 -- 172.21.15.86:0/449873718 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fc4004160 con 0x7f0fd0107310 2024-04-21T08:29:01.374 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.586+0000 7f0fcd59a700 1 -- 172.21.15.86:0/449873718 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0fc4004300 con 0x7f0fd0107310 2024-04-21T08:29:01.374 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.587+0000 7f0fcd59a700 1 -- 172.21.15.86:0/449873718 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fc4011af0 con 0x7f0fd0107310 2024-04-21T08:29:01.374 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.587+0000 7f0fd4cef700 1 -- 172.21.15.86:0/449873718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fd0107310 msgr2=0x7f0fd0107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:01.374 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.587+0000 7f0fd4cef700 1 --2- 172.21.15.86:0/449873718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fd0107310 0x7f0fd0107730 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7f0fc40098a0 tx=0x7f0fc4009bb0 comp rx=0 tx=0).stop 2024-04-21T08:29:01.374 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.587+0000 7f0fd4cef700 1 -- 172.21.15.86:0/449873718 shutdown_connections 2024-04-21T08:29:01.374 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.587+0000 7f0fd4cef700 1 --2- 172.21.15.86:0/449873718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fd0107310 0x7f0fd0107730 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:01.374 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.587+0000 7f0fd4cef700 1 -- 172.21.15.86:0/449873718 >> 172.21.15.86:0/449873718 conn(0x7f0fd01028b0 msgr2=0x7f0fd0104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:01.374 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.587+0000 7f0fd4cef700 1 -- 172.21.15.86:0/449873718 shutdown_connections 2024-04-21T08:29:01.374 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.587+0000 7f0fd4cef700 1 -- 172.21.15.86:0/449873718 wait complete. 2024-04-21T08:29:01.375 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.588+0000 7f0fd4cef700 1 Processor -- start 2024-04-21T08:29:01.375 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.588+0000 7f0fd4cef700 1 -- start start 2024-04-21T08:29:01.375 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.588+0000 7f0fd4cef700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fd0107310 0x7f0fd007baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:01.375 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.588+0000 7f0fd4cef700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0fd010b740 con 0x7f0fd0107310 2024-04-21T08:29:01.375 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.588+0000 7f0fce59c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fd0107310 0x7f0fd007baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:01.375 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.588+0000 7f0fce59c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fd0107310 0x7f0fd007baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44016/0 (socket says 172.21.15.86:44016) 2024-04-21T08:29:01.375 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.588+0000 7f0fce59c700 1 -- 172.21.15.86:0/220837740 learned_addr learned my addr 172.21.15.86:0/220837740 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:01.375 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.588+0000 7f0fce59c700 1 -- 172.21.15.86:0/220837740 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fd019ef00 con 0x7f0fd0107310 2024-04-21T08:29:01.375 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.589+0000 7f0fce59c700 1 --2- 172.21.15.86:0/220837740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fd0107310 0x7f0fd007baa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4000c00 tx=0x7f0fc4011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:01.375 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.589+0000 7f0fbf7fe700 1 -- 172.21.15.86:0/220837740 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fc4003b20 con 0x7f0fd0107310 2024-04-21T08:29:01.375 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.589+0000 7f0fbf7fe700 1 -- 172.21.15.86:0/220837740 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f0fc401a8c0 con 0x7f0fd0107310 2024-04-21T08:29:01.375 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.589+0000 7f0fd4cef700 1 -- 172.21.15.86:0/220837740 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fd019f0d0 con 0x7f0fd0107310 2024-04-21T08:29:01.375 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.589+0000 7f0fbf7fe700 1 -- 172.21.15.86:0/220837740 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f0fc4024940 con 0x7f0fd0107310 2024-04-21T08:29:01.375 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.589+0000 7f0fd4cef700 1 -- 172.21.15.86:0/220837740 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fd019f4f0 con 0x7f0fd0107310 2024-04-21T08:29:01.376 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.590+0000 7f0fbf7fe700 1 -- 172.21.15.86:0/220837740 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f0fc4022020 con 0x7f0fd0107310 2024-04-21T08:29:01.376 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.590+0000 7f0fd4cef700 1 -- 172.21.15.86:0/220837740 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0fd010b8c0 con 0x7f0fd0107310 2024-04-21T08:29:01.376 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.590+0000 7f0fbf7fe700 1 --2- 172.21.15.86:0/220837740 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f0fb8039670 0x7f0fb803bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:01.376 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.590+0000 7f0fbf7fe700 1 -- 172.21.15.86:0/220837740 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f0fc404d810 con 0x7f0fd0107310 2024-04-21T08:29:01.376 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.590+0000 7f0fcdd9b700 1 --2- 172.21.15.86:0/220837740 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f0fb8039670 0x7f0fb803bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:01.376 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.590+0000 7f0fcdd9b700 1 --2- 172.21.15.86:0/220837740 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f0fb8039670 0x7f0fb803bb30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0006fd0 tx=0x7f0fc0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:01.376 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.595+0000 7f0fbf7fe700 1 -- 172.21.15.86:0/220837740 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0fc4021d80 con 0x7f0fd0107310 2024-04-21T08:29:01.376 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.794+0000 7f0fd4cef700 1 -- 172.21.15.86:0/220837740 --> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] -- mgr_command(tid 0: {"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}) -- 0x7f0fd0103a20 con 0x7f0fb8039670 2024-04-21T08:29:01.376 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.794+0000 7f0fbf7fe700 1 -- 172.21.15.86:0/220837740 <== mgr.14120 v2:172.21.15.86:6800/2143206409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+16 (secure 0 0 0) 0x7f0fd0103a20 con 0x7f0fb8039670 2024-04-21T08:29:01.376 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.796+0000 7f0fd4cef700 1 -- 172.21.15.86:0/220837740 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f0fb8039670 msgr2=0x7f0fb803bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:01.376 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.796+0000 7f0fd4cef700 1 --2- 172.21.15.86:0/220837740 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f0fb8039670 0x7f0fb803bb30 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f0fc0006fd0 tx=0x7f0fc0006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:01.376 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.796+0000 7f0fd4cef700 1 -- 172.21.15.86:0/220837740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fd0107310 msgr2=0x7f0fd007baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:01.376 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.796+0000 7f0fd4cef700 1 --2- 172.21.15.86:0/220837740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fd0107310 0x7f0fd007baa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f0fc4000c00 tx=0x7f0fc4011b70 comp rx=0 tx=0).stop 2024-04-21T08:29:01.377 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.796+0000 7f0fd4cef700 1 -- 172.21.15.86:0/220837740 shutdown_connections 2024-04-21T08:29:01.377 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.796+0000 7f0fd4cef700 1 --2- 172.21.15.86:0/220837740 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f0fb8039670 0x7f0fb803bb30 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:01.377 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.796+0000 7f0fd4cef700 1 --2- 172.21.15.86:0/220837740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fd0107310 0x7f0fd007baa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:01.377 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.796+0000 7f0fd4cef700 1 -- 172.21.15.86:0/220837740 >> 172.21.15.86:0/220837740 conn(0x7f0fd01028b0 msgr2=0x7f0fd0103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:01.377 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.796+0000 7f0fd4cef700 1 -- 172.21.15.86:0/220837740 shutdown_connections 2024-04-21T08:29:01.377 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:00.796+0000 7f0fd4cef700 1 -- 172.21.15.86:0/220837740 wait complete. 2024-04-21T08:29:01.377 INFO:teuthology.orchestra.run.smithi086.stdout:Generating ssh key... 2024-04-21T08:29:01.783 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:01 smithi086 ceph-mon[27939]: from='client.14132 -' entity='client.admin' cmd=[{"prefix": "orch set backend", "module_name": "cephadm", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:29:02.528 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:02 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: Generating public/private rsa key pair. 2024-04-21T08:29:02.529 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:02 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: Your identification has been saved in /tmp/tmp14o2f2_4/key. 2024-04-21T08:29:02.529 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:02 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: Your public key has been saved in /tmp/tmp14o2f2_4/key.pub. 2024-04-21T08:29:02.529 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:02 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: The key fingerprint is: 2024-04-21T08:29:02.529 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:02 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: SHA256:PqzQtYadWref2IRsBQj+znVMad6/wOqKxwcOjhwCNqE ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:29:02.529 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:02 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: The key's randomart image is: 2024-04-21T08:29:02.529 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:02 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: +---[RSA 3072]----+ 2024-04-21T08:29:02.529 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:02 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: | . | 2024-04-21T08:29:02.529 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:02 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: | . . . . . | 2024-04-21T08:29:02.529 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:02 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: |. . . . . + | 2024-04-21T08:29:02.529 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:02 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: |E+ . * . | 2024-04-21T08:29:02.529 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:02 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: |. o S . = . | 2024-04-21T08:29:02.529 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:02 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: | . o X * +. . | 2024-04-21T08:29:02.529 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:02 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: | + * / * .o .| 2024-04-21T08:29:02.529 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:02 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: | + *.B *... .| 2024-04-21T08:29:02.530 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:02 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: | o...*++ . | 2024-04-21T08:29:02.530 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:02 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: +----[SHA256]-----+ 2024-04-21T08:29:02.795 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:02 smithi086 ceph-mon[27939]: from='client.14134 -' entity='client.admin' cmd=[{"prefix": "cephadm set-user", "user": "root", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:29:02.796 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:02 smithi086 ceph-mon[27939]: from='mgr.14120 172.21.15.86:0/1976532381' entity='mgr.x' 2024-04-21T08:29:02.796 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:02 smithi086 ceph-mon[27939]: from='mgr.14120 172.21.15.86:0/1976532381' entity='mgr.x' 2024-04-21T08:29:02.803 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.807+0000 7f62a03aa700 1 Processor -- start 2024-04-21T08:29:02.803 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.808+0000 7f62a03aa700 1 -- start start 2024-04-21T08:29:02.804 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.808+0000 7f62a03aa700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6298106660 0x7f6298106a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:02.804 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.808+0000 7f62a03aa700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6298107050 con 0x7f6298106660 2024-04-21T08:29:02.804 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.808+0000 7f629e146700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6298106660 0x7f6298106a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:02.804 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.808+0000 7f629e146700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6298106660 0x7f6298106a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44028/0 (socket says 172.21.15.86:44028) 2024-04-21T08:29:02.804 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.808+0000 7f629e146700 1 -- 172.21.15.86:0/4032867963 learned_addr learned my addr 172.21.15.86:0/4032867963 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:02.804 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.808+0000 7f629e146700 1 -- 172.21.15.86:0/4032867963 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62981095f0 con 0x7f6298106660 2024-04-21T08:29:02.804 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.808+0000 7f629e146700 1 --2- 172.21.15.86:0/4032867963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6298106660 0x7f6298106a80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f62940098a0 tx=0x7f6294009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=186f616b0f0c44a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:02.804 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.809+0000 7f629d144700 1 -- 172.21.15.86:0/4032867963 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6294004160 con 0x7f6298106660 2024-04-21T08:29:02.804 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.809+0000 7f629d144700 1 -- 172.21.15.86:0/4032867963 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f6294004300 con 0x7f6298106660 2024-04-21T08:29:02.804 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.809+0000 7f629d144700 1 -- 172.21.15.86:0/4032867963 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6294011af0 con 0x7f6298106660 2024-04-21T08:29:02.804 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.809+0000 7f62a03aa700 1 -- 172.21.15.86:0/4032867963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6298106660 msgr2=0x7f6298106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:02.804 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.809+0000 7f62a03aa700 1 --2- 172.21.15.86:0/4032867963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6298106660 0x7f6298106a80 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f62940098a0 tx=0x7f6294009bb0 comp rx=0 tx=0).stop 2024-04-21T08:29:02.804 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.809+0000 7f62a03aa700 1 -- 172.21.15.86:0/4032867963 shutdown_connections 2024-04-21T08:29:02.805 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.809+0000 7f62a03aa700 1 --2- 172.21.15.86:0/4032867963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6298106660 0x7f6298106a80 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:02.805 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.809+0000 7f62a03aa700 1 -- 172.21.15.86:0/4032867963 >> 172.21.15.86:0/4032867963 conn(0x7f6298101fb0 msgr2=0x7f62981043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:02.805 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.809+0000 7f62a03aa700 1 -- 172.21.15.86:0/4032867963 shutdown_connections 2024-04-21T08:29:02.805 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.809+0000 7f62a03aa700 1 -- 172.21.15.86:0/4032867963 wait complete. 2024-04-21T08:29:02.805 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.810+0000 7f62a03aa700 1 Processor -- start 2024-04-21T08:29:02.805 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.810+0000 7f62a03aa700 1 -- start start 2024-04-21T08:29:02.805 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.810+0000 7f62a03aa700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6298106660 0x7f629819c210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:02.805 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.810+0000 7f62a03aa700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f629810c9a0 con 0x7f6298106660 2024-04-21T08:29:02.805 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.810+0000 7f629e146700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6298106660 0x7f629819c210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:02.805 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.810+0000 7f629e146700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6298106660 0x7f629819c210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44044/0 (socket says 172.21.15.86:44044) 2024-04-21T08:29:02.805 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.810+0000 7f629e146700 1 -- 172.21.15.86:0/613807472 learned_addr learned my addr 172.21.15.86:0/613807472 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:02.805 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.810+0000 7f629e146700 1 -- 172.21.15.86:0/613807472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f629819c750 con 0x7f6298106660 2024-04-21T08:29:02.805 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.811+0000 7f629e146700 1 --2- 172.21.15.86:0/613807472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6298106660 0x7f629819c210 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6294000c00 tx=0x7f6294011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:02.805 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.811+0000 7f628f7fe700 1 -- 172.21.15.86:0/613807472 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6294003b20 con 0x7f6298106660 2024-04-21T08:29:02.805 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.811+0000 7f628f7fe700 1 -- 172.21.15.86:0/613807472 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f629401a8c0 con 0x7f6298106660 2024-04-21T08:29:02.806 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.811+0000 7f628f7fe700 1 -- 172.21.15.86:0/613807472 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6294024940 con 0x7f6298106660 2024-04-21T08:29:02.806 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.811+0000 7f62a03aa700 1 -- 172.21.15.86:0/613807472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f629819c9e0 con 0x7f6298106660 2024-04-21T08:29:02.806 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.811+0000 7f62a03aa700 1 -- 172.21.15.86:0/613807472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f629819f6d0 con 0x7f6298106660 2024-04-21T08:29:02.806 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.812+0000 7f628f7fe700 1 -- 172.21.15.86:0/613807472 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f6294022020 con 0x7f6298106660 2024-04-21T08:29:02.806 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.812+0000 7f628f7fe700 1 --2- 172.21.15.86:0/613807472 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f62840396c0 0x7f628403bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:02.806 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.812+0000 7f62a03aa700 1 -- 172.21.15.86:0/613807472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f62980640a0 con 0x7f6298106660 2024-04-21T08:29:02.806 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.812+0000 7f628f7fe700 1 -- 172.21.15.86:0/613807472 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f629404d860 con 0x7f6298106660 2024-04-21T08:29:02.806 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.812+0000 7f629d945700 1 --2- 172.21.15.86:0/613807472 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f62840396c0 0x7f628403bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:02.806 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.812+0000 7f629d945700 1 --2- 172.21.15.86:0/613807472 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f62840396c0 0x7f628403bb80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6288006fd0 tx=0x7f6288006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:02.806 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:01.817+0000 7f628f7fe700 1 -- 172.21.15.86:0/613807472 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f629400be50 con 0x7f6298106660 2024-04-21T08:29:02.806 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.017+0000 7f62a03aa700 1 -- 172.21.15.86:0/613807472 --> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] -- mgr_command(tid 0: {"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}) -- 0x7f62981030f0 con 0x7f62840396c0 2024-04-21T08:29:02.806 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.218+0000 7f628f7fe700 1 -- 172.21.15.86:0/613807472 <== mgr.14120 v2:172.21.15.86:6800/2143206409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f62981030f0 con 0x7f62840396c0 2024-04-21T08:29:02.807 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.220+0000 7f62a03aa700 1 -- 172.21.15.86:0/613807472 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f62840396c0 msgr2=0x7f628403bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:02.807 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.220+0000 7f62a03aa700 1 --2- 172.21.15.86:0/613807472 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f62840396c0 0x7f628403bb80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7f6288006fd0 tx=0x7f6288006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:02.807 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.220+0000 7f62a03aa700 1 -- 172.21.15.86:0/613807472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6298106660 msgr2=0x7f629819c210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:02.807 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.220+0000 7f62a03aa700 1 --2- 172.21.15.86:0/613807472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6298106660 0x7f629819c210 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f6294000c00 tx=0x7f6294011b70 comp rx=0 tx=0).stop 2024-04-21T08:29:02.807 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.220+0000 7f62a03aa700 1 -- 172.21.15.86:0/613807472 shutdown_connections 2024-04-21T08:29:02.807 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.220+0000 7f62a03aa700 1 --2- 172.21.15.86:0/613807472 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f62840396c0 0x7f628403bb80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:02.807 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.220+0000 7f62a03aa700 1 --2- 172.21.15.86:0/613807472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6298106660 0x7f629819c210 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:02.807 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.220+0000 7f62a03aa700 1 -- 172.21.15.86:0/613807472 >> 172.21.15.86:0/613807472 conn(0x7f6298101fb0 msgr2=0x7f62981029e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:02.807 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.220+0000 7f62a03aa700 1 -- 172.21.15.86:0/613807472 shutdown_connections 2024-04-21T08:29:02.807 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:02.220+0000 7f62a03aa700 1 -- 172.21.15.86:0/613807472 wait complete. 2024-04-21T08:29:03.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:03 smithi086 ceph-mon[27939]: from='client.14136 -' entity='client.admin' cmd=[{"prefix": "cephadm generate-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:29:03.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:03 smithi086 ceph-mon[27939]: Generating ssh key... 2024-04-21T08:29:03.984 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6kglyLLRmoLt8PcpxPgppEi4yJspZMBDoafJN50b/Kv1iDtR4x6WeIhI7YYuOnpcUKh5vBpkoRuF2oLVQ4ia5tZV6VXy2eOKB61WtSG2C2so8O/4xKbvTovqS9C5fCU6JBjknroUSIUQjr+wNMF9tbXiI07evecXfR6Vlrp1L/cDgzfvDfCvgRXiUjlCUnL0hCFBtAZ4L07pSyvMfp/LqlSPhcoe2oMgxHvryc6jXy1Iyta9+WLRRt3eqQFnx7fQINBGNtb0ZRtlPAvmIoJwDl0vSFw4yYsvzrH39shxthSQ4D0ps2tcUNk9LCPuxWx9yloixVpg3+sCDN9eDHaFVU3iCcEhuuz5Rc5Iufcy0i8SDpSC40Hs5+m6xHYPj3zbdZkseYmjMXk12Zg3wIcRLO5SLiscVHUtmEDbUCpTvXYqMbMm1KJgkNaqs9rwz8yl0+E+vcqz3itBsxr8192eVh8H7NJfiTxTM4kyPp9ebNReo9kZ5+O84zuLEJl61g20= ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:29:03.984 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.243+0000 7f289a2c6700 1 Processor -- start 2024-04-21T08:29:03.984 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.243+0000 7f289a2c6700 1 -- start start 2024-04-21T08:29:03.984 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.244+0000 7f289a2c6700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f289407de10 0x7f289407c310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:03.984 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.244+0000 7f289a2c6700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f289407c850 con 0x7f289407de10 2024-04-21T08:29:03.985 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.244+0000 7f2893fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f289407de10 0x7f289407c310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:03.985 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.244+0000 7f2893fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f289407de10 0x7f289407c310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44060/0 (socket says 172.21.15.86:44060) 2024-04-21T08:29:03.985 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.244+0000 7f2893fff700 1 -- 172.21.15.86:0/1184722791 learned_addr learned my addr 172.21.15.86:0/1184722791 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:03.985 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.244+0000 7f2893fff700 1 -- 172.21.15.86:0/1184722791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f289407c9d0 con 0x7f289407de10 2024-04-21T08:29:03.985 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.244+0000 7f2893fff700 1 --2- 172.21.15.86:0/1184722791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f289407de10 0x7f289407c310 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f28880098a0 tx=0x7f2888009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f8896c682a3cc86 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:03.985 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.244+0000 7f2892ffd700 1 -- 172.21.15.86:0/1184722791 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2888004160 con 0x7f289407de10 2024-04-21T08:29:03.985 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.244+0000 7f2892ffd700 1 -- 172.21.15.86:0/1184722791 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2888004300 con 0x7f289407de10 2024-04-21T08:29:03.985 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.244+0000 7f2892ffd700 1 -- 172.21.15.86:0/1184722791 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f2888011af0 con 0x7f289407de10 2024-04-21T08:29:03.985 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.245+0000 7f289a2c6700 1 -- 172.21.15.86:0/1184722791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f289407de10 msgr2=0x7f289407c310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:03.985 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.245+0000 7f289a2c6700 1 --2- 172.21.15.86:0/1184722791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f289407de10 0x7f289407c310 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f28880098a0 tx=0x7f2888009bb0 comp rx=0 tx=0).stop 2024-04-21T08:29:03.985 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.245+0000 7f289a2c6700 1 -- 172.21.15.86:0/1184722791 shutdown_connections 2024-04-21T08:29:03.985 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.245+0000 7f289a2c6700 1 --2- 172.21.15.86:0/1184722791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f289407de10 0x7f289407c310 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:03.985 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.245+0000 7f289a2c6700 1 -- 172.21.15.86:0/1184722791 >> 172.21.15.86:0/1184722791 conn(0x7f28941039c0 msgr2=0x7f2894105e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:03.985 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.245+0000 7f289a2c6700 1 -- 172.21.15.86:0/1184722791 shutdown_connections 2024-04-21T08:29:03.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.245+0000 7f289a2c6700 1 -- 172.21.15.86:0/1184722791 wait complete. 2024-04-21T08:29:03.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.246+0000 7f289a2c6700 1 Processor -- start 2024-04-21T08:29:03.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.246+0000 7f289a2c6700 1 -- start start 2024-04-21T08:29:03.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.246+0000 7f289a2c6700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f289407de10 0x7f289419e370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:03.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.246+0000 7f289a2c6700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f289410e8d0 con 0x7f289407de10 2024-04-21T08:29:03.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.246+0000 7f2893fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f289407de10 0x7f289419e370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:03.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.246+0000 7f2893fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f289407de10 0x7f289419e370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44074/0 (socket says 172.21.15.86:44074) 2024-04-21T08:29:03.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.246+0000 7f2893fff700 1 -- 172.21.15.86:0/3541846477 learned_addr learned my addr 172.21.15.86:0/3541846477 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:03.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.246+0000 7f2893fff700 1 -- 172.21.15.86:0/3541846477 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f289419e8b0 con 0x7f289407de10 2024-04-21T08:29:03.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.246+0000 7f2893fff700 1 --2- 172.21.15.86:0/3541846477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f289407de10 0x7f289419e370 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2888003f40 tx=0x7f2888011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:03.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.247+0000 7f28917fa700 1 -- 172.21.15.86:0/3541846477 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f28880038b0 con 0x7f289407de10 2024-04-21T08:29:03.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.247+0000 7f28917fa700 1 -- 172.21.15.86:0/3541846477 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7f2888003a50 con 0x7f289407de10 2024-04-21T08:29:03.986 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.247+0000 7f28917fa700 1 -- 172.21.15.86:0/3541846477 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f288801a5c0 con 0x7f289407de10 2024-04-21T08:29:03.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.247+0000 7f289a2c6700 1 -- 172.21.15.86:0/3541846477 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f289419eb40 con 0x7f289407de10 2024-04-21T08:29:03.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.247+0000 7f289a2c6700 1 -- 172.21.15.86:0/3541846477 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28941a1830 con 0x7f289407de10 2024-04-21T08:29:03.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.248+0000 7f28917fa700 1 -- 172.21.15.86:0/3541846477 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f2888024430 con 0x7f289407de10 2024-04-21T08:29:03.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.248+0000 7f289a2c6700 1 -- 172.21.15.86:0/3541846477 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28940640a0 con 0x7f289407de10 2024-04-21T08:29:03.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.248+0000 7f28917fa700 1 --2- 172.21.15.86:0/3541846477 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f287c039670 0x7f287c03bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:03.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.248+0000 7f28917fa700 1 -- 172.21.15.86:0/3541846477 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f288804da70 con 0x7f289407de10 2024-04-21T08:29:03.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.248+0000 7f28937fe700 1 --2- 172.21.15.86:0/3541846477 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f287c039670 0x7f287c03bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:03.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.248+0000 7f28937fe700 1 --2- 172.21.15.86:0/3541846477 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f287c039670 0x7f287c03bb30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2884006fd0 tx=0x7f2884006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:03.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.253+0000 7f28917fa700 1 -- 172.21.15.86:0/3541846477 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f288802d910 con 0x7f289407de10 2024-04-21T08:29:03.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.450+0000 7f289a2c6700 1 -- 172.21.15.86:0/3541846477 --> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] -- mgr_command(tid 0: {"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}) -- 0x7f2894104b30 con 0x7f287c039670 2024-04-21T08:29:03.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.450+0000 7f28917fa700 1 -- 172.21.15.86:0/3541846477 <== mgr.14120 v2:172.21.15.86:6800/2143206409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+595 (secure 0 0 0) 0x7f2894104b30 con 0x7f287c039670 2024-04-21T08:29:03.987 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.452+0000 7f289a2c6700 1 -- 172.21.15.86:0/3541846477 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f287c039670 msgr2=0x7f287c03bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:03.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.452+0000 7f289a2c6700 1 --2- 172.21.15.86:0/3541846477 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f287c039670 0x7f287c03bb30 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f2884006fd0 tx=0x7f2884006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:03.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.452+0000 7f289a2c6700 1 -- 172.21.15.86:0/3541846477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f289407de10 msgr2=0x7f289419e370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:03.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.452+0000 7f289a2c6700 1 --2- 172.21.15.86:0/3541846477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f289407de10 0x7f289419e370 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f2888003f40 tx=0x7f2888011890 comp rx=0 tx=0).stop 2024-04-21T08:29:03.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.453+0000 7f289a2c6700 1 -- 172.21.15.86:0/3541846477 shutdown_connections 2024-04-21T08:29:03.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.453+0000 7f289a2c6700 1 --2- 172.21.15.86:0/3541846477 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f287c039670 0x7f287c03bb30 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:03.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.453+0000 7f289a2c6700 1 --2- 172.21.15.86:0/3541846477 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f289407de10 0x7f289419e370 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:03.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.453+0000 7f289a2c6700 1 -- 172.21.15.86:0/3541846477 >> 172.21.15.86:0/3541846477 conn(0x7f28941039c0 msgr2=0x7f2894104420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:03.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.453+0000 7f289a2c6700 1 -- 172.21.15.86:0/3541846477 shutdown_connections 2024-04-21T08:29:03.988 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:03.453+0000 7f289a2c6700 1 -- 172.21.15.86:0/3541846477 wait complete. 2024-04-21T08:29:03.988 INFO:teuthology.orchestra.run.smithi086.stdout:Wrote public SSH key to /home/ubuntu/cephtest/ceph.pub 2024-04-21T08:29:03.988 INFO:teuthology.orchestra.run.smithi086.stdout:Adding key to root@localhost authorized_keys... 2024-04-21T08:29:03.988 INFO:teuthology.orchestra.run.smithi086.stdout:Adding host smithi086... 2024-04-21T08:29:04.754 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:04 smithi086 ceph-mon[27939]: from='client.14138 -' entity='client.admin' cmd=[{"prefix": "cephadm get-pub-key", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:29:05.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:05 smithi086 ceph-mon[27939]: from='client.14140 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi086", "addr": "172.21.15.86", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:29:05.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:05 smithi086 ceph-mon[27939]: Deploying cephadm binary to smithi086 2024-04-21T08:29:09.308 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout Added host 'smithi086' with addr '172.21.15.86' 2024-04-21T08:29:09.308 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.484+0000 7fe0d9592700 1 Processor -- start 2024-04-21T08:29:09.308 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.484+0000 7fe0d9592700 1 -- start start 2024-04-21T08:29:09.308 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.485+0000 7fe0d9592700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0d41084b0 0x7fe0d41088d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:09.308 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.485+0000 7fe0d9592700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d4108ea0 con 0x7fe0d41084b0 2024-04-21T08:29:09.308 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.485+0000 7fe0d2ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0d41084b0 0x7fe0d41088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:09.308 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.485+0000 7fe0d2ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0d41084b0 0x7fe0d41088d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44090/0 (socket says 172.21.15.86:44090) 2024-04-21T08:29:09.308 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.485+0000 7fe0d2ffd700 1 -- 172.21.15.86:0/3037879306 learned_addr learned my addr 172.21.15.86:0/3037879306 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:09.308 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.485+0000 7fe0d2ffd700 1 -- 172.21.15.86:0/3037879306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0d4109660 con 0x7fe0d41084b0 2024-04-21T08:29:09.308 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.485+0000 7fe0d2ffd700 1 --2- 172.21.15.86:0/3037879306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0d41084b0 0x7fe0d41088d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe0c40098a0 tx=0x7fe0c4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a2ec02efdde0448 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:09.309 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.486+0000 7fe0d1ffb700 1 -- 172.21.15.86:0/3037879306 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0c4004160 con 0x7fe0d41084b0 2024-04-21T08:29:09.309 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.486+0000 7fe0d1ffb700 1 -- 172.21.15.86:0/3037879306 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe0c4004300 con 0x7fe0d41084b0 2024-04-21T08:29:09.309 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.486+0000 7fe0d1ffb700 1 -- 172.21.15.86:0/3037879306 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0c4011af0 con 0x7fe0d41084b0 2024-04-21T08:29:09.309 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.486+0000 7fe0d9592700 1 -- 172.21.15.86:0/3037879306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0d41084b0 msgr2=0x7fe0d41088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:09.309 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.486+0000 7fe0d9592700 1 --2- 172.21.15.86:0/3037879306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0d41084b0 0x7fe0d41088d0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7fe0c40098a0 tx=0x7fe0c4009bb0 comp rx=0 tx=0).stop 2024-04-21T08:29:09.309 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.486+0000 7fe0d9592700 1 -- 172.21.15.86:0/3037879306 shutdown_connections 2024-04-21T08:29:09.309 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.486+0000 7fe0d9592700 1 --2- 172.21.15.86:0/3037879306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0d41084b0 0x7fe0d41088d0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:09.309 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.486+0000 7fe0d9592700 1 -- 172.21.15.86:0/3037879306 >> 172.21.15.86:0/3037879306 conn(0x7fe0d4103af0 msgr2=0x7fe0d4105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:09.309 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.486+0000 7fe0d9592700 1 -- 172.21.15.86:0/3037879306 shutdown_connections 2024-04-21T08:29:09.309 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.486+0000 7fe0d9592700 1 -- 172.21.15.86:0/3037879306 wait complete. 2024-04-21T08:29:09.309 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.487+0000 7fe0d9592700 1 Processor -- start 2024-04-21T08:29:09.309 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.487+0000 7fe0d9592700 1 -- start start 2024-04-21T08:29:09.310 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.487+0000 7fe0d9592700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0d41084b0 0x7fe0d419c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:09.310 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.487+0000 7fe0d9592700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe0d410c890 con 0x7fe0d41084b0 2024-04-21T08:29:09.310 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.487+0000 7fe0d2ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0d41084b0 0x7fe0d419c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:09.310 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.487+0000 7fe0d2ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0d41084b0 0x7fe0d419c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44100/0 (socket says 172.21.15.86:44100) 2024-04-21T08:29:09.310 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.487+0000 7fe0d2ffd700 1 -- 172.21.15.86:0/3768643449 learned_addr learned my addr 172.21.15.86:0/3768643449 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:09.310 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.487+0000 7fe0d2ffd700 1 -- 172.21.15.86:0/3768643449 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe0d419c740 con 0x7fe0d41084b0 2024-04-21T08:29:09.310 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.488+0000 7fe0d2ffd700 1 --2- 172.21.15.86:0/3768643449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0d41084b0 0x7fe0d419c200 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4003f40 tx=0x7fe0c4011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:09.310 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.488+0000 7fe0bbfff700 1 -- 172.21.15.86:0/3768643449 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0c40038b0 con 0x7fe0d41084b0 2024-04-21T08:29:09.310 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.488+0000 7fe0bbfff700 1 -- 172.21.15.86:0/3768643449 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fe0c4003a50 con 0x7fe0d41084b0 2024-04-21T08:29:09.310 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.488+0000 7fe0d9592700 1 -- 172.21.15.86:0/3768643449 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe0d419c9d0 con 0x7fe0d41084b0 2024-04-21T08:29:09.310 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.488+0000 7fe0bbfff700 1 -- 172.21.15.86:0/3768643449 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe0c401a5c0 con 0x7fe0d41084b0 2024-04-21T08:29:09.311 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.488+0000 7fe0d9592700 1 -- 172.21.15.86:0/3768643449 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe0d419f6c0 con 0x7fe0d41084b0 2024-04-21T08:29:09.311 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.489+0000 7fe0bbfff700 1 -- 172.21.15.86:0/3768643449 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fe0c401a8c0 con 0x7fe0d41084b0 2024-04-21T08:29:09.311 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.489+0000 7fe0d9592700 1 -- 172.21.15.86:0/3768643449 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe0d40640a0 con 0x7fe0d41084b0 2024-04-21T08:29:09.311 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.489+0000 7fe0bbfff700 1 --2- 172.21.15.86:0/3768643449 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7fe0bc039620 0x7fe0bc03bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:09.311 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.489+0000 7fe0bbfff700 1 -- 172.21.15.86:0/3768643449 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe0c404dad0 con 0x7fe0d41084b0 2024-04-21T08:29:09.311 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.489+0000 7fe0d27fc700 1 --2- 172.21.15.86:0/3768643449 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7fe0bc039620 0x7fe0bc03bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:09.311 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.489+0000 7fe0d27fc700 1 --2- 172.21.15.86:0/3768643449 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7fe0bc039620 0x7fe0bc03bae0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8006fd0 tx=0x7fe0c8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:09.311 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.494+0000 7fe0bbfff700 1 -- 172.21.15.86:0/3768643449 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe0c402d3f0 con 0x7fe0d41084b0 2024-04-21T08:29:09.311 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:04.703+0000 7fe0d9592700 1 -- 172.21.15.86:0/3768643449 --> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi086", "addr": "172.21.15.86", "target": ["mon-mgr", ""]}) -- 0x7fe0d4104c50 con 0x7fe0bc039620 2024-04-21T08:29:09.311 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:08.769+0000 7fe0bbfff700 1 -- 172.21.15.86:0/3768643449 <== mgr.14120 v2:172.21.15.86:6800/2143206409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7fe0d4104c50 con 0x7fe0bc039620 2024-04-21T08:29:09.311 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:08.772+0000 7fe0d9592700 1 -- 172.21.15.86:0/3768643449 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7fe0bc039620 msgr2=0x7fe0bc03bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:09.311 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:08.772+0000 7fe0d9592700 1 --2- 172.21.15.86:0/3768643449 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7fe0bc039620 0x7fe0bc03bae0 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7fe0c8006fd0 tx=0x7fe0c8006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:09.312 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:08.772+0000 7fe0d9592700 1 -- 172.21.15.86:0/3768643449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0d41084b0 msgr2=0x7fe0d419c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:09.312 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:08.772+0000 7fe0d9592700 1 --2- 172.21.15.86:0/3768643449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0d41084b0 0x7fe0d419c200 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fe0c4003f40 tx=0x7fe0c4011890 comp rx=0 tx=0).stop 2024-04-21T08:29:09.312 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:08.772+0000 7fe0d9592700 1 -- 172.21.15.86:0/3768643449 shutdown_connections 2024-04-21T08:29:09.312 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:08.772+0000 7fe0d9592700 1 --2- 172.21.15.86:0/3768643449 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7fe0bc039620 0x7fe0bc03bae0 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:09.312 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:08.772+0000 7fe0d9592700 1 --2- 172.21.15.86:0/3768643449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe0d41084b0 0x7fe0d419c200 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:09.312 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:08.772+0000 7fe0d9592700 1 -- 172.21.15.86:0/3768643449 >> 172.21.15.86:0/3768643449 conn(0x7fe0d4103af0 msgr2=0x7fe0d4104540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:09.312 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:08.772+0000 7fe0d9592700 1 -- 172.21.15.86:0/3768643449 shutdown_connections 2024-04-21T08:29:09.312 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:08.772+0000 7fe0d9592700 1 -- 172.21.15.86:0/3768643449 wait complete. 2024-04-21T08:29:09.312 INFO:teuthology.orchestra.run.smithi086.stdout:Deploying unmanaged mon service... 2024-04-21T08:29:09.805 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:09 smithi086 ceph-mon[27939]: from='mgr.14120 172.21.15.86:0/1976532381' entity='mgr.x' 2024-04-21T08:29:09.805 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:09 smithi086 ceph-mon[27939]: Added host smithi086 2024-04-21T08:29:09.806 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:09 smithi086 ceph-mon[27939]: from='mgr.14120 172.21.15.86:0/1976532381' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:29:11.095 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout Scheduled mon update... 2024-04-21T08:29:11.095 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.806+0000 7fdece51b700 1 Processor -- start 2024-04-21T08:29:11.095 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.806+0000 7fdece51b700 1 -- start start 2024-04-21T08:29:11.095 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.806+0000 7fdece51b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdec81084b0 0x7fdec81088d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:11.095 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.806+0000 7fdece51b700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdec8108ea0 con 0x7fdec81084b0 2024-04-21T08:29:11.095 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.807+0000 7fdec7fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdec81084b0 0x7fdec81088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:11.095 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.807+0000 7fdec7fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdec81084b0 0x7fdec81088d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54148/0 (socket says 172.21.15.86:54148) 2024-04-21T08:29:11.095 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.807+0000 7fdec7fff700 1 -- 172.21.15.86:0/1299888267 learned_addr learned my addr 172.21.15.86:0/1299888267 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:11.096 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.807+0000 7fdec7fff700 1 -- 172.21.15.86:0/1299888267 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec8109660 con 0x7fdec81084b0 2024-04-21T08:29:11.096 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.807+0000 7fdec7fff700 1 --2- 172.21.15.86:0/1299888267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdec81084b0 0x7fdec81088d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdeb80098a0 tx=0x7fdeb8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1e8041b672eaa40 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:11.096 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.807+0000 7fdec6ffd700 1 -- 172.21.15.86:0/1299888267 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdeb8004160 con 0x7fdec81084b0 2024-04-21T08:29:11.096 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.807+0000 7fdec6ffd700 1 -- 172.21.15.86:0/1299888267 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fdeb8004300 con 0x7fdec81084b0 2024-04-21T08:29:11.096 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.807+0000 7fdec6ffd700 1 -- 172.21.15.86:0/1299888267 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdeb8011af0 con 0x7fdec81084b0 2024-04-21T08:29:11.096 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.808+0000 7fdece51b700 1 -- 172.21.15.86:0/1299888267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdec81084b0 msgr2=0x7fdec81088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:11.096 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.808+0000 7fdece51b700 1 --2- 172.21.15.86:0/1299888267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdec81084b0 0x7fdec81088d0 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7fdeb80098a0 tx=0x7fdeb8009bb0 comp rx=0 tx=0).stop 2024-04-21T08:29:11.096 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.808+0000 7fdece51b700 1 -- 172.21.15.86:0/1299888267 shutdown_connections 2024-04-21T08:29:11.096 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.808+0000 7fdece51b700 1 --2- 172.21.15.86:0/1299888267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdec81084b0 0x7fdec81088d0 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:11.096 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.808+0000 7fdece51b700 1 -- 172.21.15.86:0/1299888267 >> 172.21.15.86:0/1299888267 conn(0x7fdec8103af0 msgr2=0x7fdec8105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:11.096 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.808+0000 7fdece51b700 1 -- 172.21.15.86:0/1299888267 shutdown_connections 2024-04-21T08:29:11.096 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.808+0000 7fdece51b700 1 -- 172.21.15.86:0/1299888267 wait complete. 2024-04-21T08:29:11.096 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.808+0000 7fdece51b700 1 Processor -- start 2024-04-21T08:29:11.097 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.809+0000 7fdece51b700 1 -- start start 2024-04-21T08:29:11.097 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.809+0000 7fdece51b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdec81084b0 0x7fdec819c200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:11.097 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.809+0000 7fdece51b700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdec810c890 con 0x7fdec81084b0 2024-04-21T08:29:11.097 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.809+0000 7fdec7fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdec81084b0 0x7fdec819c200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:11.097 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.809+0000 7fdec7fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdec81084b0 0x7fdec819c200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54162/0 (socket says 172.21.15.86:54162) 2024-04-21T08:29:11.097 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.809+0000 7fdec7fff700 1 -- 172.21.15.86:0/3156797839 learned_addr learned my addr 172.21.15.86:0/3156797839 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:11.097 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.809+0000 7fdec7fff700 1 -- 172.21.15.86:0/3156797839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdec819c740 con 0x7fdec81084b0 2024-04-21T08:29:11.097 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.809+0000 7fdec7fff700 1 --2- 172.21.15.86:0/3156797839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdec81084b0 0x7fdec819c200 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8003f40 tx=0x7fdeb8011890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:11.098 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.809+0000 7fdec57fa700 1 -- 172.21.15.86:0/3156797839 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdeb80038b0 con 0x7fdec81084b0 2024-04-21T08:29:11.098 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.809+0000 7fdec57fa700 1 -- 172.21.15.86:0/3156797839 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fdeb8003a50 con 0x7fdec81084b0 2024-04-21T08:29:11.098 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.809+0000 7fdece51b700 1 -- 172.21.15.86:0/3156797839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdec819c9d0 con 0x7fdec81084b0 2024-04-21T08:29:11.098 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.809+0000 7fdec57fa700 1 -- 172.21.15.86:0/3156797839 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fdeb801a5c0 con 0x7fdec81084b0 2024-04-21T08:29:11.098 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.809+0000 7fdece51b700 1 -- 172.21.15.86:0/3156797839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdec819f6c0 con 0x7fdec81084b0 2024-04-21T08:29:11.098 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.810+0000 7fdec57fa700 1 -- 172.21.15.86:0/3156797839 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fdeb801a8c0 con 0x7fdec81084b0 2024-04-21T08:29:11.098 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.810+0000 7fdece51b700 1 -- 172.21.15.86:0/3156797839 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdec80640a0 con 0x7fdec81084b0 2024-04-21T08:29:11.098 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.810+0000 7fdec57fa700 1 --2- 172.21.15.86:0/3156797839 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7fdeb0039620 0x7fdeb003bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:11.098 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.810+0000 7fdec57fa700 1 -- 172.21.15.86:0/3156797839 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fdeb804dc70 con 0x7fdec81084b0 2024-04-21T08:29:11.099 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.811+0000 7fdec77fe700 1 --2- 172.21.15.86:0/3156797839 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7fdeb0039620 0x7fdeb003bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:11.099 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.811+0000 7fdec77fe700 1 --2- 172.21.15.86:0/3156797839 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7fdeb0039620 0x7fdeb003bae0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdebc006fd0 tx=0x7fdebc006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:11.099 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:09.816+0000 7fdec57fa700 1 -- 172.21.15.86:0/3156797839 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdeb802d910 con 0x7fdec81084b0 2024-04-21T08:29:11.099 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:10.046+0000 7fdece51b700 1 -- 172.21.15.86:0/3156797839 --> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fdec8104c50 con 0x7fdeb0039620 2024-04-21T08:29:11.099 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:10.054+0000 7fdec57fa700 1 -- 172.21.15.86:0/3156797839 <== mgr.14120 v2:172.21.15.86:6800/2143206409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fdec8104c50 con 0x7fdeb0039620 2024-04-21T08:29:11.099 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:10.056+0000 7fdece51b700 1 -- 172.21.15.86:0/3156797839 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7fdeb0039620 msgr2=0x7fdeb003bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:11.099 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:10.056+0000 7fdece51b700 1 --2- 172.21.15.86:0/3156797839 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7fdeb0039620 0x7fdeb003bae0 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fdebc006fd0 tx=0x7fdebc006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:11.099 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:10.056+0000 7fdece51b700 1 -- 172.21.15.86:0/3156797839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdec81084b0 msgr2=0x7fdec819c200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:11.099 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:10.056+0000 7fdece51b700 1 --2- 172.21.15.86:0/3156797839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdec81084b0 0x7fdec819c200 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7fdeb8003f40 tx=0x7fdeb8011890 comp rx=0 tx=0).stop 2024-04-21T08:29:11.099 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:10.056+0000 7fdece51b700 1 -- 172.21.15.86:0/3156797839 shutdown_connections 2024-04-21T08:29:11.099 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:10.056+0000 7fdece51b700 1 --2- 172.21.15.86:0/3156797839 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7fdeb0039620 0x7fdeb003bae0 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:11.099 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:10.056+0000 7fdece51b700 1 --2- 172.21.15.86:0/3156797839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdec81084b0 0x7fdec819c200 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:11.100 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:10.056+0000 7fdece51b700 1 -- 172.21.15.86:0/3156797839 >> 172.21.15.86:0/3156797839 conn(0x7fdec8103af0 msgr2=0x7fdec8104540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:11.100 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:10.056+0000 7fdece51b700 1 -- 172.21.15.86:0/3156797839 shutdown_connections 2024-04-21T08:29:11.100 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:10.056+0000 7fdece51b700 1 -- 172.21.15.86:0/3156797839 wait complete. 2024-04-21T08:29:11.100 INFO:teuthology.orchestra.run.smithi086.stdout:Deploying unmanaged mgr service... 2024-04-21T08:29:11.353 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:11 smithi086 ceph-mon[27939]: from='client.14142 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:29:11.353 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:11 smithi086 ceph-mon[27939]: Saving service mon spec with placement count:5 2024-04-21T08:29:11.353 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:11 smithi086 ceph-mon[27939]: from='mgr.14120 172.21.15.86:0/1976532381' entity='mgr.x' 2024-04-21T08:29:12.403 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout Scheduled mgr update... 2024-04-21T08:29:12.403 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.566+0000 7fd640a36700 1 Processor -- start 2024-04-21T08:29:12.403 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.567+0000 7fd640a36700 1 -- start start 2024-04-21T08:29:12.403 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.567+0000 7fd640a36700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd63c1084b0 0x7fd63c1088d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:12.403 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.567+0000 7fd640a36700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c108ea0 con 0x7fd63c1084b0 2024-04-21T08:29:12.403 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.567+0000 7fd63a59c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd63c1084b0 0x7fd63c1088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:12.403 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.567+0000 7fd63a59c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd63c1084b0 0x7fd63c1088d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54178/0 (socket says 172.21.15.86:54178) 2024-04-21T08:29:12.403 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.567+0000 7fd63a59c700 1 -- 172.21.15.86:0/1491435047 learned_addr learned my addr 172.21.15.86:0/1491435047 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:12.403 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.567+0000 7fd63a59c700 1 -- 172.21.15.86:0/1491435047 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63c109660 con 0x7fd63c1084b0 2024-04-21T08:29:12.403 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.567+0000 7fd63a59c700 1 --2- 172.21.15.86:0/1491435047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd63c1084b0 0x7fd63c1088d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd6240098a0 tx=0x7fd624009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=731b97bde4a1d53d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:12.404 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.568+0000 7fd63959a700 1 -- 172.21.15.86:0/1491435047 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd624004160 con 0x7fd63c1084b0 2024-04-21T08:29:12.404 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.568+0000 7fd63959a700 1 -- 172.21.15.86:0/1491435047 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd624004300 con 0x7fd63c1084b0 2024-04-21T08:29:12.404 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.568+0000 7fd63959a700 1 -- 172.21.15.86:0/1491435047 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd624011af0 con 0x7fd63c1084b0 2024-04-21T08:29:12.404 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.568+0000 7fd640a36700 1 -- 172.21.15.86:0/1491435047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd63c1084b0 msgr2=0x7fd63c1088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:12.404 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.568+0000 7fd640a36700 1 --2- 172.21.15.86:0/1491435047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd63c1084b0 0x7fd63c1088d0 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7fd6240098a0 tx=0x7fd624009bb0 comp rx=0 tx=0).stop 2024-04-21T08:29:12.404 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.568+0000 7fd640a36700 1 -- 172.21.15.86:0/1491435047 shutdown_connections 2024-04-21T08:29:12.404 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.568+0000 7fd640a36700 1 --2- 172.21.15.86:0/1491435047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd63c1084b0 0x7fd63c1088d0 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:12.404 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.568+0000 7fd640a36700 1 -- 172.21.15.86:0/1491435047 >> 172.21.15.86:0/1491435047 conn(0x7fd63c103af0 msgr2=0x7fd63c105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:12.404 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.568+0000 7fd640a36700 1 -- 172.21.15.86:0/1491435047 shutdown_connections 2024-04-21T08:29:12.404 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.568+0000 7fd640a36700 1 -- 172.21.15.86:0/1491435047 wait complete. 2024-04-21T08:29:12.404 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.569+0000 7fd640a36700 1 Processor -- start 2024-04-21T08:29:12.404 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.569+0000 7fd640a36700 1 -- start start 2024-04-21T08:29:12.404 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.569+0000 7fd640a36700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd63c1084b0 0x7fd63c07db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:12.404 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.569+0000 7fd640a36700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd63c10c890 con 0x7fd63c1084b0 2024-04-21T08:29:12.405 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.569+0000 7fd63a59c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd63c1084b0 0x7fd63c07db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:12.405 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.569+0000 7fd63a59c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd63c1084b0 0x7fd63c07db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54188/0 (socket says 172.21.15.86:54188) 2024-04-21T08:29:12.405 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.569+0000 7fd63a59c700 1 -- 172.21.15.86:0/3576303954 learned_addr learned my addr 172.21.15.86:0/3576303954 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:12.405 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.570+0000 7fd63a59c700 1 -- 172.21.15.86:0/3576303954 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd63c07e0a0 con 0x7fd63c1084b0 2024-04-21T08:29:12.405 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.570+0000 7fd63a59c700 1 --2- 172.21.15.86:0/3576303954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd63c1084b0 0x7fd63c07db60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd624000c00 tx=0x7fd624011b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:12.405 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.570+0000 7fd6337fe700 1 -- 172.21.15.86:0/3576303954 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd624003b20 con 0x7fd63c1084b0 2024-04-21T08:29:12.405 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.570+0000 7fd6337fe700 1 -- 172.21.15.86:0/3576303954 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1070+0+0 (secure 0 0 0) 0x7fd62401a8c0 con 0x7fd63c1084b0 2024-04-21T08:29:12.405 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.570+0000 7fd640a36700 1 -- 172.21.15.86:0/3576303954 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd63c07c290 con 0x7fd63c1084b0 2024-04-21T08:29:12.405 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.570+0000 7fd6337fe700 1 -- 172.21.15.86:0/3576303954 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd624024940 con 0x7fd63c1084b0 2024-04-21T08:29:12.405 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.570+0000 7fd640a36700 1 -- 172.21.15.86:0/3576303954 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd63c07c7d0 con 0x7fd63c1084b0 2024-04-21T08:29:12.405 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.571+0000 7fd6337fe700 1 -- 172.21.15.86:0/3576303954 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7fd624022020 con 0x7fd63c1084b0 2024-04-21T08:29:12.405 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.571+0000 7fd640a36700 1 -- 172.21.15.86:0/3576303954 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd63c0640a0 con 0x7fd63c1084b0 2024-04-21T08:29:12.405 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.571+0000 7fd6337fe700 1 --2- 172.21.15.86:0/3576303954 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7fd628039670 0x7fd62803bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:12.405 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.571+0000 7fd6337fe700 1 -- 172.21.15.86:0/3576303954 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fd62404d7f0 con 0x7fd63c1084b0 2024-04-21T08:29:12.406 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.571+0000 7fd639d9b700 1 --2- 172.21.15.86:0/3576303954 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7fd628039670 0x7fd62803bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:12.406 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.571+0000 7fd639d9b700 1 --2- 172.21.15.86:0/3576303954 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7fd628039670 0x7fd62803bb30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd62c006fd0 tx=0x7fd62c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:12.406 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.576+0000 7fd6337fe700 1 -- 172.21.15.86:0/3576303954 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd624018d80 con 0x7fd63c1084b0 2024-04-21T08:29:12.406 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.791+0000 7fd640a36700 1 -- 172.21.15.86:0/3576303954 --> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}) -- 0x7fd63c104c50 con 0x7fd628039670 2024-04-21T08:29:12.406 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.795+0000 7fd6337fe700 1 -- 172.21.15.86:0/3576303954 <== mgr.14120 v2:172.21.15.86:6800/2143206409 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fd63c104c50 con 0x7fd628039670 2024-04-21T08:29:12.406 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.797+0000 7fd640a36700 1 -- 172.21.15.86:0/3576303954 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7fd628039670 msgr2=0x7fd62803bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:12.406 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.797+0000 7fd640a36700 1 --2- 172.21.15.86:0/3576303954 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7fd628039670 0x7fd62803bb30 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7fd62c006fd0 tx=0x7fd62c006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:12.406 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.797+0000 7fd640a36700 1 -- 172.21.15.86:0/3576303954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd63c1084b0 msgr2=0x7fd63c07db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:12.406 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.797+0000 7fd640a36700 1 --2- 172.21.15.86:0/3576303954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd63c1084b0 0x7fd63c07db60 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fd624000c00 tx=0x7fd624011b70 comp rx=0 tx=0).stop 2024-04-21T08:29:12.406 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.797+0000 7fd640a36700 1 -- 172.21.15.86:0/3576303954 shutdown_connections 2024-04-21T08:29:12.406 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.797+0000 7fd640a36700 1 --2- 172.21.15.86:0/3576303954 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7fd628039670 0x7fd62803bb30 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:12.406 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.797+0000 7fd640a36700 1 --2- 172.21.15.86:0/3576303954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd63c1084b0 0x7fd63c07db60 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:12.406 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.797+0000 7fd640a36700 1 -- 172.21.15.86:0/3576303954 >> 172.21.15.86:0/3576303954 conn(0x7fd63c103af0 msgr2=0x7fd63c104540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:12.406 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.797+0000 7fd640a36700 1 -- 172.21.15.86:0/3576303954 shutdown_connections 2024-04-21T08:29:12.407 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:11.797+0000 7fd640a36700 1 -- 172.21.15.86:0/3576303954 wait complete. 2024-04-21T08:29:13.083 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:12 smithi086 ceph-mon[27939]: from='client.14144 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "unmanaged": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:29:13.084 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:12 smithi086 ceph-mon[27939]: Saving service mgr spec with placement count:2 2024-04-21T08:29:13.084 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:12 smithi086 ceph-mon[27939]: from='mgr.14120 172.21.15.86:0/1976532381' entity='mgr.x' 2024-04-21T08:29:13.084 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:12 smithi086 ceph-mon[27939]: from='mgr.14120 172.21.15.86:0/1976532381' entity='mgr.x' 2024-04-21T08:29:13.084 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:12 smithi086 ceph-mon[27939]: from='mgr.14120 172.21.15.86:0/1976532381' entity='mgr.x' 2024-04-21T08:29:13.693 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.854+0000 7f6c76d87700 1 Processor -- start 2024-04-21T08:29:13.693 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.854+0000 7f6c76d87700 1 -- start start 2024-04-21T08:29:13.693 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.855+0000 7f6c76d87700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c70106640 0x7f6c70106a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:13.693 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.855+0000 7f6c76d87700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6c70107030 con 0x7f6c70106640 2024-04-21T08:29:13.693 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.855+0000 7f6c74b23700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c70106640 0x7f6c70106a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:13.694 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.855+0000 7f6c74b23700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c70106640 0x7f6c70106a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54192/0 (socket says 172.21.15.86:54192) 2024-04-21T08:29:13.694 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.855+0000 7f6c74b23700 1 -- 172.21.15.86:0/3164935796 learned_addr learned my addr 172.21.15.86:0/3164935796 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:13.694 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.855+0000 7f6c74b23700 1 -- 172.21.15.86:0/3164935796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c70109670 con 0x7f6c70106640 2024-04-21T08:29:13.694 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.855+0000 7f6c74b23700 1 --2- 172.21.15.86:0/3164935796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c70106640 0x7f6c70106a60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6c640098a0 tx=0x7f6c64009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81a4546d10b67f08 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:13.694 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.856+0000 7f6c6f7fe700 1 -- 172.21.15.86:0/3164935796 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c64004160 con 0x7f6c70106640 2024-04-21T08:29:13.694 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.856+0000 7f6c6f7fe700 1 -- 172.21.15.86:0/3164935796 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c64004300 con 0x7f6c70106640 2024-04-21T08:29:13.694 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.856+0000 7f6c6f7fe700 1 -- 172.21.15.86:0/3164935796 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c64011a20 con 0x7f6c70106640 2024-04-21T08:29:13.694 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.856+0000 7f6c76d87700 1 -- 172.21.15.86:0/3164935796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c70106640 msgr2=0x7f6c70106a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:13.694 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.856+0000 7f6c76d87700 1 --2- 172.21.15.86:0/3164935796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c70106640 0x7f6c70106a60 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7f6c640098a0 tx=0x7f6c64009bb0 comp rx=0 tx=0).stop 2024-04-21T08:29:13.695 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.856+0000 7f6c76d87700 1 -- 172.21.15.86:0/3164935796 shutdown_connections 2024-04-21T08:29:13.695 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.856+0000 7f6c76d87700 1 --2- 172.21.15.86:0/3164935796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c70106640 0x7f6c70106a60 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:13.695 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.856+0000 7f6c76d87700 1 -- 172.21.15.86:0/3164935796 >> 172.21.15.86:0/3164935796 conn(0x7f6c70101f70 msgr2=0x7f6c701043b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:13.695 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.856+0000 7f6c76d87700 1 -- 172.21.15.86:0/3164935796 shutdown_connections 2024-04-21T08:29:13.695 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.856+0000 7f6c76d87700 1 -- 172.21.15.86:0/3164935796 wait complete. 2024-04-21T08:29:13.695 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.857+0000 7f6c76d87700 1 Processor -- start 2024-04-21T08:29:13.695 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.857+0000 7f6c76d87700 1 -- start start 2024-04-21T08:29:13.695 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.857+0000 7f6c76d87700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c70106640 0x7f6c7019e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:13.696 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.857+0000 7f6c76d87700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6c7010c8a0 con 0x7f6c70106640 2024-04-21T08:29:13.696 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.857+0000 7f6c74b23700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c70106640 0x7f6c7019e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:13.696 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.857+0000 7f6c74b23700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c70106640 0x7f6c7019e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54200/0 (socket says 172.21.15.86:54200) 2024-04-21T08:29:13.696 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.857+0000 7f6c74b23700 1 -- 172.21.15.86:0/350176102 learned_addr learned my addr 172.21.15.86:0/350176102 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:13.696 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.857+0000 7f6c74b23700 1 -- 172.21.15.86:0/350176102 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c7019e940 con 0x7f6c70106640 2024-04-21T08:29:13.696 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.858+0000 7f6c74b23700 1 --2- 172.21.15.86:0/350176102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c70106640 0x7f6c7019e400 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6c640116e0 tx=0x7f6c64011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:13.696 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.858+0000 7f6c6dffb700 1 -- 172.21.15.86:0/350176102 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c640039e0 con 0x7f6c70106640 2024-04-21T08:29:13.696 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.858+0000 7f6c6dffb700 1 -- 172.21.15.86:0/350176102 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c6401b8f0 con 0x7f6c70106640 2024-04-21T08:29:13.697 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.858+0000 7f6c76d87700 1 -- 172.21.15.86:0/350176102 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c7019ebd0 con 0x7f6c70106640 2024-04-21T08:29:13.697 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.858+0000 7f6c6dffb700 1 -- 172.21.15.86:0/350176102 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6c64023760 con 0x7f6c70106640 2024-04-21T08:29:13.697 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.858+0000 7f6c76d87700 1 -- 172.21.15.86:0/350176102 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c701a18c0 con 0x7f6c70106640 2024-04-21T08:29:13.697 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.859+0000 7f6c6dffb700 1 -- 172.21.15.86:0/350176102 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f6c640239f0 con 0x7f6c70106640 2024-04-21T08:29:13.697 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.859+0000 7f6c76d87700 1 -- 172.21.15.86:0/350176102 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6c700640a0 con 0x7f6c70106640 2024-04-21T08:29:13.697 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.859+0000 7f6c6dffb700 1 --2- 172.21.15.86:0/350176102 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f6c60039620 0x7f6c6003bae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:13.697 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.859+0000 7f6c6dffb700 1 -- 172.21.15.86:0/350176102 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6c6404e250 con 0x7f6c70106640 2024-04-21T08:29:13.698 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.859+0000 7f6c6ffff700 1 --2- 172.21.15.86:0/350176102 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f6c60039620 0x7f6c6003bae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:13.698 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.860+0000 7f6c6ffff700 1 --2- 172.21.15.86:0/350176102 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f6c60039620 0x7f6c6003bae0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c006fd0 tx=0x7f6c5c006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:13.698 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:12.864+0000 7f6c6dffb700 1 -- 172.21.15.86:0/350176102 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6c6401f030 con 0x7f6c70106640 2024-04-21T08:29:13.698 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:13.084+0000 7f6c76d87700 1 -- 172.21.15.86:0/350176102 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/container_init}] v 0) -- 0x7f6c701a1e90 con 0x7f6c70106640 2024-04-21T08:29:13.698 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:13.089+0000 7f6c6dffb700 1 -- 172.21.15.86:0/350176102 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/container_init}]=0 v7)=0 v7) ==== 142+0+0 (secure 0 0 0) 0x7f6c64016260 con 0x7f6c70106640 2024-04-21T08:29:13.698 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:13.092+0000 7f6c76d87700 1 -- 172.21.15.86:0/350176102 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f6c60039620 msgr2=0x7f6c6003bae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:13.698 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:13.092+0000 7f6c76d87700 1 --2- 172.21.15.86:0/350176102 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f6c60039620 0x7f6c6003bae0 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f6c5c006fd0 tx=0x7f6c5c006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:13.698 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:13.092+0000 7f6c76d87700 1 -- 172.21.15.86:0/350176102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c70106640 msgr2=0x7f6c7019e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:13.698 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:13.092+0000 7f6c76d87700 1 --2- 172.21.15.86:0/350176102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c70106640 0x7f6c7019e400 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f6c640116e0 tx=0x7f6c64011aa0 comp rx=0 tx=0).stop 2024-04-21T08:29:13.699 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:13.092+0000 7f6c76d87700 1 -- 172.21.15.86:0/350176102 shutdown_connections 2024-04-21T08:29:13.699 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:13.092+0000 7f6c76d87700 1 --2- 172.21.15.86:0/350176102 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f6c60039620 0x7f6c6003bae0 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:13.699 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:13.092+0000 7f6c76d87700 1 --2- 172.21.15.86:0/350176102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c70106640 0x7f6c7019e400 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:13.699 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:13.092+0000 7f6c76d87700 1 -- 172.21.15.86:0/350176102 >> 172.21.15.86:0/350176102 conn(0x7f6c70101f70 msgr2=0x7f6c70102930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:13.699 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:13.092+0000 7f6c76d87700 1 -- 172.21.15.86:0/350176102 shutdown_connections 2024-04-21T08:29:13.699 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:13.092+0000 7f6c76d87700 1 -- 172.21.15.86:0/350176102 wait complete. 2024-04-21T08:29:14.297 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:14 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/350176102' entity='client.admin' 2024-04-21T08:29:14.991 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.132+0000 7f5dc33fe700 1 Processor -- start 2024-04-21T08:29:14.991 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.132+0000 7f5dc33fe700 1 -- start start 2024-04-21T08:29:14.991 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.132+0000 7f5dc33fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dbc106660 0x7f5dbc106a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:14.991 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.132+0000 7f5dc33fe700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc107050 con 0x7f5dbc106660 2024-04-21T08:29:14.992 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.132+0000 7f5dc119a700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dbc106660 0x7f5dbc106a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:14.992 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.132+0000 7f5dc119a700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dbc106660 0x7f5dbc106a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54216/0 (socket says 172.21.15.86:54216) 2024-04-21T08:29:14.992 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.132+0000 7f5dc119a700 1 -- 172.21.15.86:0/2237535735 learned_addr learned my addr 172.21.15.86:0/2237535735 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:14.992 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.133+0000 7f5dc119a700 1 -- 172.21.15.86:0/2237535735 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dbc1095f0 con 0x7f5dbc106660 2024-04-21T08:29:14.992 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.133+0000 7f5dc119a700 1 --2- 172.21.15.86:0/2237535735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dbc106660 0x7f5dbc106a80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5dac0098a0 tx=0x7f5dac009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3849ff7a23498f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:14.992 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.133+0000 7f5db3fff700 1 -- 172.21.15.86:0/2237535735 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5dac004160 con 0x7f5dbc106660 2024-04-21T08:29:14.992 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.133+0000 7f5db3fff700 1 -- 172.21.15.86:0/2237535735 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dac004300 con 0x7f5dbc106660 2024-04-21T08:29:14.992 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.133+0000 7f5db3fff700 1 -- 172.21.15.86:0/2237535735 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5dac011a20 con 0x7f5dbc106660 2024-04-21T08:29:14.992 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.133+0000 7f5dc33fe700 1 -- 172.21.15.86:0/2237535735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dbc106660 msgr2=0x7f5dbc106a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:14.992 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.133+0000 7f5dc33fe700 1 --2- 172.21.15.86:0/2237535735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dbc106660 0x7f5dbc106a80 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f5dac0098a0 tx=0x7f5dac009bb0 comp rx=0 tx=0).stop 2024-04-21T08:29:14.992 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.133+0000 7f5dc33fe700 1 -- 172.21.15.86:0/2237535735 shutdown_connections 2024-04-21T08:29:14.992 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.133+0000 7f5dc33fe700 1 --2- 172.21.15.86:0/2237535735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dbc106660 0x7f5dbc106a80 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:14.992 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.134+0000 7f5dc33fe700 1 -- 172.21.15.86:0/2237535735 >> 172.21.15.86:0/2237535735 conn(0x7f5dbc101fb0 msgr2=0x7f5dbc1043d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:14.992 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.134+0000 7f5dc33fe700 1 -- 172.21.15.86:0/2237535735 shutdown_connections 2024-04-21T08:29:14.993 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.134+0000 7f5dc33fe700 1 -- 172.21.15.86:0/2237535735 wait complete. 2024-04-21T08:29:14.993 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.134+0000 7f5dc33fe700 1 Processor -- start 2024-04-21T08:29:14.993 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.134+0000 7f5dc33fe700 1 -- start start 2024-04-21T08:29:14.993 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.134+0000 7f5dc33fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dbc106660 0x7f5dbc19c180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:14.993 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.134+0000 7f5dc33fe700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5dbc10c9a0 con 0x7f5dbc106660 2024-04-21T08:29:14.993 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.134+0000 7f5dc119a700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dbc106660 0x7f5dbc19c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:14.993 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.134+0000 7f5dc119a700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dbc106660 0x7f5dbc19c180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54224/0 (socket says 172.21.15.86:54224) 2024-04-21T08:29:14.993 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.134+0000 7f5dc119a700 1 -- 172.21.15.86:0/1504869621 learned_addr learned my addr 172.21.15.86:0/1504869621 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:14.993 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.135+0000 7f5dc119a700 1 -- 172.21.15.86:0/1504869621 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dbc19c6c0 con 0x7f5dbc106660 2024-04-21T08:29:14.993 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.135+0000 7f5dc119a700 1 --2- 172.21.15.86:0/1504869621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dbc106660 0x7f5dbc19c180 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5dac0116e0 tx=0x7f5dac011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:14.993 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.135+0000 7f5db27fc700 1 -- 172.21.15.86:0/1504869621 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5dac0039e0 con 0x7f5dbc106660 2024-04-21T08:29:14.993 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.135+0000 7f5db27fc700 1 -- 172.21.15.86:0/1504869621 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dac01b8f0 con 0x7f5dbc106660 2024-04-21T08:29:14.993 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.135+0000 7f5dc33fe700 1 -- 172.21.15.86:0/1504869621 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dbc19c950 con 0x7f5dbc106660 2024-04-21T08:29:14.993 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.135+0000 7f5db27fc700 1 -- 172.21.15.86:0/1504869621 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f5dac023760 con 0x7f5dbc106660 2024-04-21T08:29:14.993 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.135+0000 7f5dc33fe700 1 -- 172.21.15.86:0/1504869621 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dbc19f640 con 0x7f5dbc106660 2024-04-21T08:29:14.994 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.136+0000 7f5db27fc700 1 -- 172.21.15.86:0/1504869621 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f5dac023a00 con 0x7f5dbc106660 2024-04-21T08:29:14.994 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.136+0000 7f5dc33fe700 1 -- 172.21.15.86:0/1504869621 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dbc0640a0 con 0x7f5dbc106660 2024-04-21T08:29:14.994 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.136+0000 7f5db27fc700 1 --2- 172.21.15.86:0/1504869621 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f5da8039670 0x7f5da803bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:14.994 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.136+0000 7f5db27fc700 1 -- 172.21.15.86:0/1504869621 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f5dac04e050 con 0x7f5dbc106660 2024-04-21T08:29:14.994 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.136+0000 7f5dc0999700 1 --2- 172.21.15.86:0/1504869621 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f5da8039670 0x7f5da803bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:14.994 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.137+0000 7f5dc0999700 1 --2- 172.21.15.86:0/1504869621 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f5da8039670 0x7f5da803bb30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5db8006fd0 tx=0x7f5db8006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:14.994 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.142+0000 7f5db27fc700 1 -- 172.21.15.86:0/1504869621 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5dac01f030 con 0x7f5dbc106660 2024-04-21T08:29:14.994 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.341+0000 7f5dc33fe700 1 -- 172.21.15.86:0/1504869621 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mgr/dashboard/ssl_server_port}] v 0) -- 0x7f5dbc19f810 con 0x7f5dbc106660 2024-04-21T08:29:14.994 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.345+0000 7f5db27fc700 1 -- 172.21.15.86:0/1504869621 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/dashboard/ssl_server_port}]=0 v8)=0 v8) ==== 130+0+0 (secure 0 0 0) 0x7f5dac018e30 con 0x7f5dbc106660 2024-04-21T08:29:14.994 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.347+0000 7f5dc33fe700 1 -- 172.21.15.86:0/1504869621 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f5da8039670 msgr2=0x7f5da803bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:14.994 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.347+0000 7f5dc33fe700 1 --2- 172.21.15.86:0/1504869621 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f5da8039670 0x7f5da803bb30 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f5db8006fd0 tx=0x7f5db8006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:14.994 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.347+0000 7f5dc33fe700 1 -- 172.21.15.86:0/1504869621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dbc106660 msgr2=0x7f5dbc19c180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:14.994 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.347+0000 7f5dc33fe700 1 --2- 172.21.15.86:0/1504869621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dbc106660 0x7f5dbc19c180 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7f5dac0116e0 tx=0x7f5dac011aa0 comp rx=0 tx=0).stop 2024-04-21T08:29:14.995 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.347+0000 7f5dc33fe700 1 -- 172.21.15.86:0/1504869621 shutdown_connections 2024-04-21T08:29:14.995 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.347+0000 7f5dc33fe700 1 --2- 172.21.15.86:0/1504869621 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f5da8039670 0x7f5da803bb30 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:14.995 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.347+0000 7f5dc33fe700 1 --2- 172.21.15.86:0/1504869621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dbc106660 0x7f5dbc19c180 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:14.995 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.347+0000 7f5dc33fe700 1 -- 172.21.15.86:0/1504869621 >> 172.21.15.86:0/1504869621 conn(0x7f5dbc101fb0 msgr2=0x7f5dbc102900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:14.995 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.347+0000 7f5dc33fe700 1 -- 172.21.15.86:0/1504869621 shutdown_connections 2024-04-21T08:29:14.995 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:14.347+0000 7f5dc33fe700 1 -- 172.21.15.86:0/1504869621 wait complete. 2024-04-21T08:29:14.995 INFO:teuthology.orchestra.run.smithi086.stdout:Enabling the dashboard module... 2024-04-21T08:29:15.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:15 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1504869621' entity='client.admin' 2024-04-21T08:29:15.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:15 smithi086 ceph-mon[27939]: from='mgr.14120 172.21.15.86:0/1976532381' entity='mgr.x' 2024-04-21T08:29:16.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:16 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2621635936' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "dashboard"}]: dispatch 2024-04-21T08:29:16.616 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:16 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: ignoring --setuser ceph since I am not root 2024-04-21T08:29:16.617 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:16 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: ignoring --setgroup ceph since I am not root 2024-04-21T08:29:16.617 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:16 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:29:16.528+0000 7fb0a0286200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-21T08:29:16.933 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:16 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:29:16.719+0000 7fb0a0286200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:29:16.933 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:16 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:29:16.821+0000 7fb0a0286200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-21T08:29:16.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.775+0000 7f4fd27cf700 1 Processor -- start 2024-04-21T08:29:16.941 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.775+0000 7f4fd27cf700 1 -- start start 2024-04-21T08:29:16.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.775+0000 7f4fd27cf700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fcc108360 0x7f4fcc108780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:16.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.775+0000 7f4fd27cf700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc108d50 con 0x7f4fcc108360 2024-04-21T08:29:16.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.775+0000 7f4fcbfff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fcc108360 0x7f4fcc108780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:16.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.776+0000 7f4fcbfff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fcc108360 0x7f4fcc108780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54238/0 (socket says 172.21.15.86:54238) 2024-04-21T08:29:16.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.776+0000 7f4fcbfff700 1 -- 172.21.15.86:0/2735281767 learned_addr learned my addr 172.21.15.86:0/2735281767 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:16.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.776+0000 7f4fcbfff700 1 -- 172.21.15.86:0/2735281767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fcc109560 con 0x7f4fcc108360 2024-04-21T08:29:16.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.776+0000 7f4fcbfff700 1 --2- 172.21.15.86:0/2735281767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fcc108360 0x7f4fcc108780 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc0098a0 tx=0x7f4fbc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77214bde79748c42 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:16.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.776+0000 7f4fcaffd700 1 -- 172.21.15.86:0/2735281767 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fbc004160 con 0x7f4fcc108360 2024-04-21T08:29:16.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.776+0000 7f4fcaffd700 1 -- 172.21.15.86:0/2735281767 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fbc004300 con 0x7f4fcc108360 2024-04-21T08:29:16.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.776+0000 7f4fcaffd700 1 -- 172.21.15.86:0/2735281767 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fbc011a20 con 0x7f4fcc108360 2024-04-21T08:29:16.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.777+0000 7f4fd27cf700 1 -- 172.21.15.86:0/2735281767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fcc108360 msgr2=0x7f4fcc108780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:16.942 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.777+0000 7f4fd27cf700 1 --2- 172.21.15.86:0/2735281767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fcc108360 0x7f4fcc108780 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc0098a0 tx=0x7f4fbc009bb0 comp rx=0 tx=0).stop 2024-04-21T08:29:16.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.777+0000 7f4fd27cf700 1 -- 172.21.15.86:0/2735281767 shutdown_connections 2024-04-21T08:29:16.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.777+0000 7f4fd27cf700 1 --2- 172.21.15.86:0/2735281767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fcc108360 0x7f4fcc108780 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:16.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.777+0000 7f4fd27cf700 1 -- 172.21.15.86:0/2735281767 >> 172.21.15.86:0/2735281767 conn(0x7f4fcc1038e0 msgr2=0x7f4fcc105d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:16.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.777+0000 7f4fd27cf700 1 -- 172.21.15.86:0/2735281767 shutdown_connections 2024-04-21T08:29:16.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.777+0000 7f4fd27cf700 1 -- 172.21.15.86:0/2735281767 wait complete. 2024-04-21T08:29:16.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.778+0000 7f4fd27cf700 1 Processor -- start 2024-04-21T08:29:16.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.778+0000 7f4fd27cf700 1 -- start start 2024-04-21T08:29:16.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.778+0000 7f4fd27cf700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fcc108360 0x7f4fcc19c1a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:16.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.778+0000 7f4fd27cf700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4fcc10c910 con 0x7f4fcc108360 2024-04-21T08:29:16.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.778+0000 7f4fcbfff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fcc108360 0x7f4fcc19c1a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:16.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.778+0000 7f4fcbfff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fcc108360 0x7f4fcc19c1a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54246/0 (socket says 172.21.15.86:54246) 2024-04-21T08:29:16.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.778+0000 7f4fcbfff700 1 -- 172.21.15.86:0/2621635936 learned_addr learned my addr 172.21.15.86:0/2621635936 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:16.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.778+0000 7f4fcbfff700 1 -- 172.21.15.86:0/2621635936 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4fcc19c6e0 con 0x7f4fcc108360 2024-04-21T08:29:16.943 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.778+0000 7f4fcbfff700 1 --2- 172.21.15.86:0/2621635936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fcc108360 0x7f4fcc19c1a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc009520 tx=0x7f4fbc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:16.944 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.779+0000 7f4fc97fa700 1 -- 172.21.15.86:0/2621635936 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fbc0039e0 con 0x7f4fcc108360 2024-04-21T08:29:16.944 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.779+0000 7f4fc97fa700 1 -- 172.21.15.86:0/2621635936 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fbc01b8f0 con 0x7f4fcc108360 2024-04-21T08:29:16.944 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.779+0000 7f4fd27cf700 1 -- 172.21.15.86:0/2621635936 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4fcc19c970 con 0x7f4fcc108360 2024-04-21T08:29:16.944 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.779+0000 7f4fc97fa700 1 -- 172.21.15.86:0/2621635936 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f4fbc019770 con 0x7f4fcc108360 2024-04-21T08:29:16.944 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.779+0000 7f4fd27cf700 1 -- 172.21.15.86:0/2621635936 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4fcc19f660 con 0x7f4fcc108360 2024-04-21T08:29:16.944 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.780+0000 7f4fc97fa700 1 -- 172.21.15.86:0/2621635936 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 8) ==== 46558+0+0 (secure 0 0 0) 0x7f4fbc019a70 con 0x7f4fcc108360 2024-04-21T08:29:16.944 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.780+0000 7f4fd27cf700 1 -- 172.21.15.86:0/2621635936 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4fcc0640a0 con 0x7f4fcc108360 2024-04-21T08:29:16.944 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.780+0000 7f4fc97fa700 1 --2- 172.21.15.86:0/2621635936 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f4fb4039670 0x7f4fb403bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:16.944 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.780+0000 7f4fc97fa700 1 -- 172.21.15.86:0/2621635936 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f4fbc04ded0 con 0x7f4fcc108360 2024-04-21T08:29:16.944 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.780+0000 7f4fcb7fe700 1 --2- 172.21.15.86:0/2621635936 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f4fb4039670 0x7f4fb403bb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:16.945 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.780+0000 7f4fcb7fe700 1 --2- 172.21.15.86:0/2621635936 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f4fb4039670 0x7f4fb403bb30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0006fd0 tx=0x7f4fc0006e40 comp rx=0 tx=0).ready entity=mgr.14120 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:16.945 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:15.785+0000 7f4fc97fa700 1 -- 172.21.15.86:0/2621635936 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4fbc01f030 con 0x7f4fcc108360 2024-04-21T08:29:16.945 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.078+0000 7f4fd27cf700 1 -- 172.21.15.86:0/2621635936 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "dashboard"} v 0) -- 0x7f4fcc19fc50 con 0x7f4fcc108360 2024-04-21T08:29:16.945 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.348+0000 7f4fc97fa700 1 -- 172.21.15.86:0/2621635936 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "dashboard"}]=0 v9) ==== 88+0+0 (secure 0 0 0) 0x7f4fbc0172b0 con 0x7f4fcc108360 2024-04-21T08:29:16.945 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.349+0000 7f4fc97fa700 1 -- 172.21.15.86:0/2621635936 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f4fbc01a8f0 con 0x7f4fcc108360 2024-04-21T08:29:16.945 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.351+0000 7f4fd27cf700 1 -- 172.21.15.86:0/2621635936 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f4fb4039670 msgr2=0x7f4fb403bb30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:16.946 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.351+0000 7f4fd27cf700 1 --2- 172.21.15.86:0/2621635936 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f4fb4039670 0x7f4fb403bb30 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f4fc0006fd0 tx=0x7f4fc0006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:16.946 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.351+0000 7f4fd27cf700 1 -- 172.21.15.86:0/2621635936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fcc108360 msgr2=0x7f4fcc19c1a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:16.946 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.351+0000 7f4fd27cf700 1 --2- 172.21.15.86:0/2621635936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fcc108360 0x7f4fcc19c1a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f4fbc009520 tx=0x7f4fbc011aa0 comp rx=0 tx=0).stop 2024-04-21T08:29:16.946 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.351+0000 7f4fd27cf700 1 -- 172.21.15.86:0/2621635936 shutdown_connections 2024-04-21T08:29:16.946 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.351+0000 7f4fd27cf700 1 --2- 172.21.15.86:0/2621635936 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f4fb4039670 0x7f4fb403bb30 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:16.946 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.351+0000 7f4fd27cf700 1 --2- 172.21.15.86:0/2621635936 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4fcc108360 0x7f4fcc19c1a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:16.947 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.351+0000 7f4fd27cf700 1 -- 172.21.15.86:0/2621635936 >> 172.21.15.86:0/2621635936 conn(0x7f4fcc1038e0 msgr2=0x7f4fcc1042a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:16.947 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.352+0000 7f4fd27cf700 1 -- 172.21.15.86:0/2621635936 shutdown_connections 2024-04-21T08:29:16.947 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:16.352+0000 7f4fd27cf700 1 -- 172.21.15.86:0/2621635936 wait complete. 2024-04-21T08:29:17.191 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:17 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:29:17.135+0000 7fb0a0286200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-21T08:29:17.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:17 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2621635936' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "dashboard"}]': finished 2024-04-21T08:29:17.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:17 smithi086 ceph-mon[27939]: mgrmap e9: x(active, since 18s) 2024-04-21T08:29:17.617 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:17 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:29:17.224+0000 7fb0a0286200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T08:29:18.315 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-04-21T08:29:18.315 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "epoch": 9, 2024-04-21T08:29:18.315 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "available": true, 2024-04-21T08:29:18.315 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "active_name": "x", 2024-04-21T08:29:18.315 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "num_standby": 0 2024-04-21T08:29:18.315 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-04-21T08:29:18.315 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.415+0000 7fe3a5445700 1 Processor -- start 2024-04-21T08:29:18.315 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.415+0000 7fe3a5445700 1 -- start start 2024-04-21T08:29:18.315 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.415+0000 7fe3a5445700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3a007de80 0x7fe3a007c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:18.315 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.415+0000 7fe3a5445700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a007c900 con 0x7fe3a007de80 2024-04-21T08:29:18.316 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.416+0000 7fe39effd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3a007de80 0x7fe3a007c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:18.316 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.416+0000 7fe39effd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3a007de80 0x7fe3a007c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54282/0 (socket says 172.21.15.86:54282) 2024-04-21T08:29:18.316 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.416+0000 7fe39effd700 1 -- 172.21.15.86:0/3448593663 learned_addr learned my addr 172.21.15.86:0/3448593663 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:18.316 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.416+0000 7fe39effd700 1 -- 172.21.15.86:0/3448593663 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a007ca80 con 0x7fe3a007de80 2024-04-21T08:29:18.316 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.416+0000 7fe39effd700 1 --2- 172.21.15.86:0/3448593663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3a007de80 0x7fe3a007c330 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe3940098a0 tx=0x7fe394009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=102430a0af58821a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:18.316 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.416+0000 7fe39dffb700 1 -- 172.21.15.86:0/3448593663 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe394004160 con 0x7fe3a007de80 2024-04-21T08:29:18.316 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.416+0000 7fe39dffb700 1 -- 172.21.15.86:0/3448593663 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe394004300 con 0x7fe3a007de80 2024-04-21T08:29:18.316 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.416+0000 7fe39dffb700 1 -- 172.21.15.86:0/3448593663 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe394011a20 con 0x7fe3a007de80 2024-04-21T08:29:18.316 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.417+0000 7fe3a5445700 1 -- 172.21.15.86:0/3448593663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3a007de80 msgr2=0x7fe3a007c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:18.316 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.417+0000 7fe3a5445700 1 --2- 172.21.15.86:0/3448593663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3a007de80 0x7fe3a007c330 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fe3940098a0 tx=0x7fe394009bb0 comp rx=0 tx=0).stop 2024-04-21T08:29:18.316 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.417+0000 7fe3a5445700 1 -- 172.21.15.86:0/3448593663 shutdown_connections 2024-04-21T08:29:18.316 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.417+0000 7fe3a5445700 1 --2- 172.21.15.86:0/3448593663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3a007de80 0x7fe3a007c330 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:18.316 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.417+0000 7fe3a5445700 1 -- 172.21.15.86:0/3448593663 >> 172.21.15.86:0/3448593663 conn(0x7fe3a0103a60 msgr2=0x7fe3a0105e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:18.316 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.417+0000 7fe3a5445700 1 -- 172.21.15.86:0/3448593663 shutdown_connections 2024-04-21T08:29:18.317 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.417+0000 7fe3a5445700 1 -- 172.21.15.86:0/3448593663 wait complete. 2024-04-21T08:29:18.317 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.417+0000 7fe3a5445700 1 Processor -- start 2024-04-21T08:29:18.317 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.418+0000 7fe3a5445700 1 -- start start 2024-04-21T08:29:18.317 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.418+0000 7fe3a5445700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3a007de80 0x7fe3a019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:18.317 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.418+0000 7fe3a5445700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe3a010ea00 con 0x7fe3a007de80 2024-04-21T08:29:18.317 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.418+0000 7fe39effd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3a007de80 0x7fe3a019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:18.317 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.418+0000 7fe39effd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3a007de80 0x7fe3a019e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54298/0 (socket says 172.21.15.86:54298) 2024-04-21T08:29:18.317 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.418+0000 7fe39effd700 1 -- 172.21.15.86:0/3878403273 learned_addr learned my addr 172.21.15.86:0/3878403273 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:18.317 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.418+0000 7fe39effd700 1 -- 172.21.15.86:0/3878403273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3a019e920 con 0x7fe3a007de80 2024-04-21T08:29:18.317 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.418+0000 7fe39effd700 1 --2- 172.21.15.86:0/3878403273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3a007de80 0x7fe3a019e3e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe3940116e0 tx=0x7fe394011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:18.317 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.419+0000 7fe387fff700 1 -- 172.21.15.86:0/3878403273 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe394003b20 con 0x7fe3a007de80 2024-04-21T08:29:18.317 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.419+0000 7fe387fff700 1 -- 172.21.15.86:0/3878403273 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe39401b8f0 con 0x7fe3a007de80 2024-04-21T08:29:18.317 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.419+0000 7fe3a5445700 1 -- 172.21.15.86:0/3878403273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3a019ebb0 con 0x7fe3a007de80 2024-04-21T08:29:18.317 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.419+0000 7fe387fff700 1 -- 172.21.15.86:0/3878403273 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe3940238c0 con 0x7fe3a007de80 2024-04-21T08:29:18.318 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.419+0000 7fe3a5445700 1 -- 172.21.15.86:0/3878403273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3a01a18a0 con 0x7fe3a007de80 2024-04-21T08:29:18.318 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.419+0000 7fe3a5445700 1 -- 172.21.15.86:0/3878403273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe3a010eb80 con 0x7fe3a007de80 2024-04-21T08:29:18.318 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.420+0000 7fe387fff700 1 -- 172.21.15.86:0/3878403273 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7fe39401baa0 con 0x7fe3a007de80 2024-04-21T08:29:18.318 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.420+0000 7fe387fff700 1 --2- 172.21.15.86:0/3878403273 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7fe388039670 0x7fe38803bb30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:18.318 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.420+0000 7fe387fff700 1 -- 172.21.15.86:0/3878403273 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7fe39404e090 con 0x7fe3a007de80 2024-04-21T08:29:18.318 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.420+0000 7fe39e7fc700 1 -- 172.21.15.86:0/3878403273 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7fe388039670 msgr2=0x7fe38803bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2143206409 2024-04-21T08:29:18.318 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.420+0000 7fe39e7fc700 1 --2- 172.21.15.86:0/3878403273 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7fe388039670 0x7fe38803bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:29:18.318 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.425+0000 7fe387fff700 1 -- 172.21.15.86:0/3878403273 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe39401f030 con 0x7fe3a007de80 2024-04-21T08:29:18.318 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.621+0000 7fe39e7fc700 1 -- 172.21.15.86:0/3878403273 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7fe388039670 msgr2=0x7fe38803bb30 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2143206409 2024-04-21T08:29:18.318 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.621+0000 7fe39e7fc700 1 --2- 172.21.15.86:0/3878403273 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7fe388039670 0x7fe38803bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T08:29:18.318 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.722+0000 7fe3a5445700 1 -- 172.21.15.86:0/3878403273 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr stat"} v 0) -- 0x7fe3a01a1a70 con 0x7fe3a007de80 2024-04-21T08:29:18.318 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.722+0000 7fe387fff700 1 -- 172.21.15.86:0/3878403273 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr stat"}]=0 v9) ==== 56+0+88 (secure 0 0 0) 0x7fe39400be50 con 0x7fe3a007de80 2024-04-21T08:29:18.319 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.724+0000 7fe3a5445700 1 -- 172.21.15.86:0/3878403273 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7fe388039670 msgr2=0x7fe38803bb30 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:29:18.319 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.724+0000 7fe3a5445700 1 --2- 172.21.15.86:0/3878403273 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7fe388039670 0x7fe38803bb30 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:18.319 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.724+0000 7fe3a5445700 1 -- 172.21.15.86:0/3878403273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3a007de80 msgr2=0x7fe3a019e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:18.319 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.724+0000 7fe3a5445700 1 --2- 172.21.15.86:0/3878403273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3a007de80 0x7fe3a019e3e0 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe3940116e0 tx=0x7fe394011aa0 comp rx=0 tx=0).stop 2024-04-21T08:29:18.319 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.724+0000 7fe3a5445700 1 -- 172.21.15.86:0/3878403273 shutdown_connections 2024-04-21T08:29:18.319 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.724+0000 7fe3a5445700 1 --2- 172.21.15.86:0/3878403273 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7fe388039670 0x7fe38803bb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:18.319 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.724+0000 7fe3a5445700 1 --2- 172.21.15.86:0/3878403273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe3a007de80 0x7fe3a019e3e0 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:18.319 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.724+0000 7fe3a5445700 1 -- 172.21.15.86:0/3878403273 >> 172.21.15.86:0/3878403273 conn(0x7fe3a0103a60 msgr2=0x7fe3a0104470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:18.319 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.724+0000 7fe3a5445700 1 -- 172.21.15.86:0/3878403273 shutdown_connections 2024-04-21T08:29:18.319 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:17.725+0000 7fe3a5445700 1 -- 172.21.15.86:0/3878403273 wait complete. 2024-04-21T08:29:18.319 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for the mgr to restart... 2024-04-21T08:29:18.319 INFO:teuthology.orchestra.run.smithi086.stdout:Waiting for mgr epoch 9... 2024-04-21T08:29:18.559 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:18 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3878403273' entity='client.admin' cmd=[{"prefix": "mgr stat"}]: dispatch 2024-04-21T08:29:18.560 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:18 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:29:18.428+0000 7fb0a0286200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T08:29:18.866 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:18 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:29:18.605+0000 7fb0a0286200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T08:29:18.866 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:18 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:29:18.782+0000 7fb0a0286200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T08:29:19.343 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:19 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:29:19.070+0000 7fb0a0286200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T08:29:19.343 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:19 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:29:19.163+0000 7fb0a0286200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T08:29:19.616 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:19 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:29:19.364+0000 7fb0a0286200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T08:29:19.616 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:19 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:29:19.556+0000 7fb0a0286200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:29:20.568 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:20 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:29:20.148+0000 7fb0a0286200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T08:29:20.866 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:20 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:29:20.569+0000 7fb0a0286200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T08:29:20.866 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:20 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:29:20.665+0000 7fb0a0286200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T08:29:21.323 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:20 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:29:20.872+0000 7fb0a0286200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T08:29:21.616 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:21 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:29:21.394+0000 7fb0a0286200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T08:29:21.616 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:21 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:29:21.486+0000 7fb0a0286200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T08:29:22.071 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:21 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:29:21.728+0000 7fb0a0286200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T08:29:22.071 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:21 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:29:21.859+0000 7fb0a0286200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T08:29:22.071 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:21 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:29:21.953+0000 7fb0a0286200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T08:29:22.071 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:22 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:29:22.070+0000 7fb0a0286200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T08:29:22.866 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:22 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:29:22.372+0000 7fb0a0286200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T08:29:23.240 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:22 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:29:22.970+0000 7fb0a0286200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T08:29:23.240 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:29:23 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:29:23.186+0000 7fb0a0286200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T08:29:23.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:23 smithi086 ceph-mon[27939]: Active manager daemon x restarted 2024-04-21T08:29:23.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:23 smithi086 ceph-mon[27939]: Activating manager daemon x 2024-04-21T08:29:23.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:23 smithi086 ceph-mon[27939]: osdmap e3: 0 total, 0 up, 0 in 2024-04-21T08:29:23.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:23 smithi086 ceph-mon[27939]: mgrmap e10: x(active, starting, since 0.00335403s) 2024-04-21T08:29:23.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:23 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:29:23.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:23 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-21T08:29:23.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:23 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-21T08:29:23.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:23 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-21T08:29:23.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:23 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-21T08:29:23.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:23 smithi086 ceph-mon[27939]: Manager daemon x is now available 2024-04-21T08:29:23.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:23 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:29:23.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:23 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-21T08:29:24.512 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:24 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-21T08:29:24.513 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:24 smithi086 ceph-mon[27939]: mgrmap e11: x(active, since 1.00589s) 2024-04-21T08:29:24.807 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout { 2024-04-21T08:29:24.807 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "mgrmap_epoch": 11, 2024-04-21T08:29:24.807 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout "initialized": true 2024-04-21T08:29:24.807 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout } 2024-04-21T08:29:24.807 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.737+0000 7f6d053fd700 1 Processor -- start 2024-04-21T08:29:24.808 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.737+0000 7f6d053fd700 1 -- start start 2024-04-21T08:29:24.808 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.737+0000 7f6d053fd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d001072f0 0x7f6d00107710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:24.808 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.737+0000 7f6d053fd700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6d00107ce0 con 0x7f6d001072f0 2024-04-21T08:29:24.808 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.737+0000 7f6cfeffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d001072f0 0x7f6d00107710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:24.808 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.737+0000 7f6cfeffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d001072f0 0x7f6d00107710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54308/0 (socket says 172.21.15.86:54308) 2024-04-21T08:29:24.808 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.737+0000 7f6cfeffd700 1 -- 172.21.15.86:0/1861330036 learned_addr learned my addr 172.21.15.86:0/1861330036 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:24.808 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.738+0000 7f6cfeffd700 1 -- 172.21.15.86:0/1861330036 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d001084f0 con 0x7f6d001072f0 2024-04-21T08:29:24.808 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.738+0000 7f6cfeffd700 1 --2- 172.21.15.86:0/1861330036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d001072f0 0x7f6d00107710 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6cf40098a0 tx=0x7f6cf4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da3c11cc527c3af5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:24.808 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.738+0000 7f6cfdffb700 1 -- 172.21.15.86:0/1861330036 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6cf4004160 con 0x7f6d001072f0 2024-04-21T08:29:24.808 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.738+0000 7f6cfdffb700 1 -- 172.21.15.86:0/1861330036 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cf4004300 con 0x7f6d001072f0 2024-04-21T08:29:24.808 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.738+0000 7f6cfdffb700 1 -- 172.21.15.86:0/1861330036 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6cf4011a20 con 0x7f6d001072f0 2024-04-21T08:29:24.808 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.739+0000 7f6d053fd700 1 -- 172.21.15.86:0/1861330036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d001072f0 msgr2=0x7f6d00107710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:24.808 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.739+0000 7f6d053fd700 1 --2- 172.21.15.86:0/1861330036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d001072f0 0x7f6d00107710 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f6cf40098a0 tx=0x7f6cf4009bb0 comp rx=0 tx=0).stop 2024-04-21T08:29:24.809 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.739+0000 7f6d053fd700 1 -- 172.21.15.86:0/1861330036 shutdown_connections 2024-04-21T08:29:24.809 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.739+0000 7f6d053fd700 1 --2- 172.21.15.86:0/1861330036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d001072f0 0x7f6d00107710 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:24.809 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.739+0000 7f6d053fd700 1 -- 172.21.15.86:0/1861330036 >> 172.21.15.86:0/1861330036 conn(0x7f6d00102890 msgr2=0x7f6d00104cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:24.809 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.739+0000 7f6d053fd700 1 -- 172.21.15.86:0/1861330036 shutdown_connections 2024-04-21T08:29:24.809 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.739+0000 7f6d053fd700 1 -- 172.21.15.86:0/1861330036 wait complete. 2024-04-21T08:29:24.809 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.740+0000 7f6d053fd700 1 Processor -- start 2024-04-21T08:29:24.809 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.740+0000 7f6d053fd700 1 -- start start 2024-04-21T08:29:24.809 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.740+0000 7f6d053fd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d001072f0 0x7f6d0007baa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:24.809 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.740+0000 7f6d053fd700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6d0010b8a0 con 0x7f6d001072f0 2024-04-21T08:29:24.809 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.740+0000 7f6cfeffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d001072f0 0x7f6d0007baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:24.809 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.740+0000 7f6cfeffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d001072f0 0x7f6d0007baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54322/0 (socket says 172.21.15.86:54322) 2024-04-21T08:29:24.809 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.740+0000 7f6cfeffd700 1 -- 172.21.15.86:0/15074422 learned_addr learned my addr 172.21.15.86:0/15074422 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:24.810 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.740+0000 7f6cfeffd700 1 -- 172.21.15.86:0/15074422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d0019eee0 con 0x7f6d001072f0 2024-04-21T08:29:24.810 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.741+0000 7f6cfeffd700 1 --2- 172.21.15.86:0/15074422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d001072f0 0x7f6d0007baa0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4009520 tx=0x7f6cf4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:24.810 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.741+0000 7f6ce7fff700 1 -- 172.21.15.86:0/15074422 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6cf40039e0 con 0x7f6d001072f0 2024-04-21T08:29:24.810 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.741+0000 7f6ce7fff700 1 -- 172.21.15.86:0/15074422 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6cf401b8f0 con 0x7f6d001072f0 2024-04-21T08:29:24.810 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.741+0000 7f6d053fd700 1 -- 172.21.15.86:0/15074422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d0019f0b0 con 0x7f6d001072f0 2024-04-21T08:29:24.810 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.741+0000 7f6ce7fff700 1 -- 172.21.15.86:0/15074422 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f6cf4019770 con 0x7f6d001072f0 2024-04-21T08:29:24.810 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.741+0000 7f6d053fd700 1 -- 172.21.15.86:0/15074422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d0019f4d0 con 0x7f6d001072f0 2024-04-21T08:29:24.810 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.742+0000 7f6ce7fff700 1 -- 172.21.15.86:0/15074422 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 9) ==== 46571+0+0 (secure 0 0 0) 0x7f6cf4019a70 con 0x7f6d001072f0 2024-04-21T08:29:24.810 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.742+0000 7f6ce7fff700 1 --2- 172.21.15.86:0/15074422 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f6ce80396c0 0x7f6ce803bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:24.810 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.742+0000 7f6d053fd700 1 -- 172.21.15.86:0/15074422 --> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6d0010ba20 con 0x7f6ce80396c0 2024-04-21T08:29:24.810 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.742+0000 7f6ce7fff700 1 -- 172.21.15.86:0/15074422 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(2..2 src has 1..2) ==== 940+0+0 (secure 0 0 0) 0x7f6cf404eeb0 con 0x7f6d001072f0 2024-04-21T08:29:24.811 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.742+0000 7f6cfe7fc700 1 -- 172.21.15.86:0/15074422 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f6ce80396c0 msgr2=0x7f6ce803bb80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2143206409 2024-04-21T08:29:24.811 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.742+0000 7f6cfe7fc700 1 --2- 172.21.15.86:0/15074422 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f6ce80396c0 0x7f6ce803bb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:29:24.811 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.943+0000 7f6cfe7fc700 1 -- 172.21.15.86:0/15074422 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f6ce80396c0 msgr2=0x7f6ce803bb80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2143206409 2024-04-21T08:29:24.811 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:18.943+0000 7f6cfe7fc700 1 --2- 172.21.15.86:0/15074422 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f6ce80396c0 0x7f6ce803bb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T08:29:24.811 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.344+0000 7f6cfe7fc700 1 -- 172.21.15.86:0/15074422 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f6ce80396c0 msgr2=0x7f6ce803bb80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2143206409 2024-04-21T08:29:24.811 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:19.344+0000 7f6cfe7fc700 1 --2- 172.21.15.86:0/15074422 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f6ce80396c0 0x7f6ce803bb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.800000 2024-04-21T08:29:24.811 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:20.145+0000 7f6cfe7fc700 1 -- 172.21.15.86:0/15074422 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f6ce80396c0 msgr2=0x7f6ce803bb80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2143206409 2024-04-21T08:29:24.811 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:20.145+0000 7f6cfe7fc700 1 --2- 172.21.15.86:0/15074422 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f6ce80396c0 0x7f6ce803bb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 1.600000 2024-04-21T08:29:24.811 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.745+0000 7f6cfe7fc700 1 -- 172.21.15.86:0/15074422 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f6ce80396c0 msgr2=0x7f6ce803bb80 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/2143206409 2024-04-21T08:29:24.811 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:21.745+0000 7f6cfe7fc700 1 --2- 172.21.15.86:0/15074422 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f6ce80396c0 0x7f6ce803bb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 3.200000 2024-04-21T08:29:24.812 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:23.190+0000 7f6ce7fff700 1 -- 172.21.15.86:0/15074422 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mgrmap(e 10) ==== 46338+0+0 (secure 0 0 0) 0x7f6cf401ac70 con 0x7f6d001072f0 2024-04-21T08:29:24.812 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:23.190+0000 7f6ce7fff700 1 -- 172.21.15.86:0/15074422 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f6ce80396c0 msgr2=0x7f6ce803bb80 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:29:24.812 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:23.190+0000 7f6ce7fff700 1 --2- 172.21.15.86:0/15074422 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f6ce80396c0 0x7f6ce803bb80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:24.812 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:24.192+0000 7f6ce7fff700 1 -- 172.21.15.86:0/15074422 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f6cf4024d50 con 0x7f6d001072f0 2024-04-21T08:29:24.812 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:24.192+0000 7f6ce7fff700 1 --2- 172.21.15.86:0/15074422 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6ce803cfa0 0x7f6ce803f390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:24.812 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:24.192+0000 7f6ce7fff700 1 -- 172.21.15.86:0/15074422 --> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7f6d0010ba20 con 0x7f6ce803cfa0 2024-04-21T08:29:24.812 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:24.192+0000 7f6cfe7fc700 1 --2- 172.21.15.86:0/15074422 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6ce803cfa0 0x7f6ce803f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:24.812 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:24.193+0000 7f6cfe7fc700 1 --2- 172.21.15.86:0/15074422 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6ce803cfa0 0x7f6ce803f390 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0003a10 tx=0x7f6cf00092b0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:24.812 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:24.193+0000 7f6ce7fff700 1 -- 172.21.15.86:0/15074422 <== mgr.14152 v2:172.21.15.86:6800/1364156965 1 ==== command_reply(tid 0: 0 ) ==== 8+0+7774 (secure 0 0 0) 0x7f6d0010ba20 con 0x7f6ce803cfa0 2024-04-21T08:29:24.812 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:24.200+0000 7f6d053fd700 1 -- 172.21.15.86:0/15074422 --> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] -- command(tid 1: {"prefix": "mgr_status"}) -- 0x7f6d00069b50 con 0x7f6ce803cfa0 2024-04-21T08:29:24.812 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:24.200+0000 7f6ce7fff700 1 -- 172.21.15.86:0/15074422 <== mgr.14152 v2:172.21.15.86:6800/1364156965 2 ==== command_reply(tid 1: 0 ) ==== 8+0+52 (secure 0 0 0) 0x7f6d00069b50 con 0x7f6ce803cfa0 2024-04-21T08:29:24.812 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:24.200+0000 7f6d053fd700 1 -- 172.21.15.86:0/15074422 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6ce803cfa0 msgr2=0x7f6ce803f390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:24.813 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:24.200+0000 7f6d053fd700 1 --2- 172.21.15.86:0/15074422 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6ce803cfa0 0x7f6ce803f390 secure :-1 s=READY pgs=2 cs=0 l=1 rev1=1 crypto rx=0x7f6cf0003a10 tx=0x7f6cf00092b0 comp rx=0 tx=0).stop 2024-04-21T08:29:24.813 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:24.200+0000 7f6d053fd700 1 -- 172.21.15.86:0/15074422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d001072f0 msgr2=0x7f6d0007baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:24.813 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:24.200+0000 7f6d053fd700 1 --2- 172.21.15.86:0/15074422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d001072f0 0x7f6d0007baa0 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f6cf4009520 tx=0x7f6cf4011aa0 comp rx=0 tx=0).stop 2024-04-21T08:29:24.813 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:24.200+0000 7f6d053fd700 1 -- 172.21.15.86:0/15074422 shutdown_connections 2024-04-21T08:29:24.813 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:24.200+0000 7f6d053fd700 1 --2- 172.21.15.86:0/15074422 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6ce803cfa0 0x7f6ce803f390 unknown :-1 s=CLOSED pgs=2 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:24.813 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:24.200+0000 7f6d053fd700 1 --2- 172.21.15.86:0/15074422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d001072f0 0x7f6d0007baa0 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:24.813 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:24.200+0000 7f6d053fd700 1 --2- 172.21.15.86:0/15074422 >> [v2:172.21.15.86:6800/2143206409,v1:172.21.15.86:6801/2143206409] conn(0x7f6ce80396c0 0x7f6ce803bb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:24.813 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:24.200+0000 7f6d053fd700 1 -- 172.21.15.86:0/15074422 >> 172.21.15.86:0/15074422 conn(0x7f6d00102890 msgr2=0x7f6d00103280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:24.813 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:24.201+0000 7f6d053fd700 1 -- 172.21.15.86:0/15074422 shutdown_connections 2024-04-21T08:29:24.813 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:24.201+0000 7f6d053fd700 1 -- 172.21.15.86:0/15074422 wait complete. 2024-04-21T08:29:24.813 INFO:teuthology.orchestra.run.smithi086.stdout:mgr epoch 9 is available 2024-04-21T08:29:24.813 INFO:teuthology.orchestra.run.smithi086.stdout:Generating a dashboard self-signed certificate... 2024-04-21T08:29:25.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:25 smithi086 ceph-mon[27939]: [21/Apr/2024:08:29:24] ENGINE Bus STARTING 2024-04-21T08:29:25.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:25 smithi086 ceph-mon[27939]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "get_command_descriptions"}]: dispatch 2024-04-21T08:29:25.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:25 smithi086 ceph-mon[27939]: from='client.14156 -' entity='client.admin' cmd=[{"prefix": "mgr_status"}]: dispatch 2024-04-21T08:29:25.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:25 smithi086 ceph-mon[27939]: [21/Apr/2024:08:29:24] ENGINE Serving on http://172.21.15.86:8765 2024-04-21T08:29:25.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:25 smithi086 ceph-mon[27939]: [21/Apr/2024:08:29:24] ENGINE Serving on https://172.21.15.86:7150 2024-04-21T08:29:25.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:25 smithi086 ceph-mon[27939]: [21/Apr/2024:08:29:24] ENGINE Bus STARTED 2024-04-21T08:29:26.253 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout Self-signed certificate created 2024-04-21T08:29:26.253 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.243+0000 7f52fd313700 1 Processor -- start 2024-04-21T08:29:26.253 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.243+0000 7f52fd313700 1 -- start start 2024-04-21T08:29:26.253 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.243+0000 7f52fd313700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52f81083f0 0x7f52f8108810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:26.253 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.243+0000 7f52fd313700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f52f8108de0 con 0x7f52f81083f0 2024-04-21T08:29:26.253 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.243+0000 7f52f6ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52f81083f0 0x7f52f8108810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:26.253 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.244+0000 7f52f6ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52f81083f0 0x7f52f8108810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50802/0 (socket says 172.21.15.86:50802) 2024-04-21T08:29:26.253 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.244+0000 7f52f6ffd700 1 -- 172.21.15.86:0/126384247 learned_addr learned my addr 172.21.15.86:0/126384247 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:26.254 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.244+0000 7f52f6ffd700 1 -- 172.21.15.86:0/126384247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52f81095f0 con 0x7f52f81083f0 2024-04-21T08:29:26.254 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.244+0000 7f52f6ffd700 1 --2- 172.21.15.86:0/126384247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52f81083f0 0x7f52f8108810 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f52e80098a0 tx=0x7f52e8009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=412adb6bc4acb904 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:26.254 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.248+0000 7f52f5ffb700 1 -- 172.21.15.86:0/126384247 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52e8004160 con 0x7f52f81083f0 2024-04-21T08:29:26.254 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.248+0000 7f52f5ffb700 1 -- 172.21.15.86:0/126384247 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52e8004300 con 0x7f52f81083f0 2024-04-21T08:29:26.254 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.248+0000 7f52f5ffb700 1 -- 172.21.15.86:0/126384247 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52e8011a20 con 0x7f52f81083f0 2024-04-21T08:29:26.254 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.249+0000 7f52fd313700 1 -- 172.21.15.86:0/126384247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52f81083f0 msgr2=0x7f52f8108810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:26.254 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.249+0000 7f52fd313700 1 --2- 172.21.15.86:0/126384247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52f81083f0 0x7f52f8108810 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f52e80098a0 tx=0x7f52e8009bb0 comp rx=0 tx=0).stop 2024-04-21T08:29:26.254 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.249+0000 7f52fd313700 1 -- 172.21.15.86:0/126384247 shutdown_connections 2024-04-21T08:29:26.254 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.249+0000 7f52fd313700 1 --2- 172.21.15.86:0/126384247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52f81083f0 0x7f52f8108810 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:26.254 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.249+0000 7f52fd313700 1 -- 172.21.15.86:0/126384247 >> 172.21.15.86:0/126384247 conn(0x7f52f81039f0 msgr2=0x7f52f8105e10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:26.254 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.249+0000 7f52fd313700 1 -- 172.21.15.86:0/126384247 shutdown_connections 2024-04-21T08:29:26.254 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.249+0000 7f52fd313700 1 -- 172.21.15.86:0/126384247 wait complete. 2024-04-21T08:29:26.254 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.249+0000 7f52fd313700 1 Processor -- start 2024-04-21T08:29:26.254 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.250+0000 7f52fd313700 1 -- start start 2024-04-21T08:29:26.255 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.250+0000 7f52fd313700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52f81083f0 0x7f52f819c220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:26.255 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.250+0000 7f52fd313700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f52f810c9a0 con 0x7f52f81083f0 2024-04-21T08:29:26.255 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.250+0000 7f52f6ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52f81083f0 0x7f52f819c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:26.255 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.250+0000 7f52f6ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52f81083f0 0x7f52f819c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50808/0 (socket says 172.21.15.86:50808) 2024-04-21T08:29:26.255 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.250+0000 7f52f6ffd700 1 -- 172.21.15.86:0/3977016187 learned_addr learned my addr 172.21.15.86:0/3977016187 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:26.255 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.250+0000 7f52f6ffd700 1 -- 172.21.15.86:0/3977016187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f52f819c760 con 0x7f52f81083f0 2024-04-21T08:29:26.255 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.250+0000 7f52f6ffd700 1 --2- 172.21.15.86:0/3977016187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52f81083f0 0x7f52f819c220 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f52e8000c00 tx=0x7f52e8011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:26.255 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.250+0000 7f52dffff700 1 -- 172.21.15.86:0/3977016187 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52e8003aa0 con 0x7f52f81083f0 2024-04-21T08:29:26.255 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.251+0000 7f52fd313700 1 -- 172.21.15.86:0/3977016187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f52f819c9f0 con 0x7f52f81083f0 2024-04-21T08:29:26.255 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.251+0000 7f52dffff700 1 -- 172.21.15.86:0/3977016187 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f52e801b8f0 con 0x7f52f81083f0 2024-04-21T08:29:26.255 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.251+0000 7f52dffff700 1 -- 172.21.15.86:0/3977016187 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f52e8023890 con 0x7f52f81083f0 2024-04-21T08:29:26.255 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.251+0000 7f52fd313700 1 -- 172.21.15.86:0/3977016187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52f819f6e0 con 0x7f52f81083f0 2024-04-21T08:29:26.255 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.251+0000 7f52dffff700 1 -- 172.21.15.86:0/3977016187 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 11) ==== 46465+0+0 (secure 0 0 0) 0x7f52e802c3f0 con 0x7f52f81083f0 2024-04-21T08:29:26.255 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.251+0000 7f52dffff700 1 --2- 172.21.15.86:0/3977016187 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f52e00395a0 0x7f52e003ba60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:26.256 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.252+0000 7f52dffff700 1 -- 172.21.15.86:0/3977016187 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f52e804ddf0 con 0x7f52f81083f0 2024-04-21T08:29:26.256 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.252+0000 7f52f67fc700 1 --2- 172.21.15.86:0/3977016187 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f52e00395a0 0x7f52e003ba60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:26.256 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.252+0000 7f52fd313700 1 -- 172.21.15.86:0/3977016187 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f52f810cb20 con 0x7f52f81083f0 2024-04-21T08:29:26.256 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.252+0000 7f52f67fc700 1 --2- 172.21.15.86:0/3977016187 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f52e00395a0 0x7f52e003ba60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f52ec006fd0 tx=0x7f52ec006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:26.256 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.253+0000 7f52dffff700 1 -- 172.21.15.86:0/3977016187 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f52e802c780 con 0x7f52f81083f0 2024-04-21T08:29:26.256 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.259+0000 7f52dffff700 1 -- 172.21.15.86:0/3977016187 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f52e8048080 con 0x7f52f81083f0 2024-04-21T08:29:26.256 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.468+0000 7f52fd313700 1 -- 172.21.15.86:0/3977016187 --> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] -- mgr_command(tid 0: {"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}) -- 0x7f52f8104b70 con 0x7f52e00395a0 2024-04-21T08:29:26.256 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.580+0000 7f52dffff700 1 -- 172.21.15.86:0/3977016187 <== mgr.14152 v2:172.21.15.86:6800/1364156965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+32 (secure 0 0 0) 0x7f52f8104b70 con 0x7f52e00395a0 2024-04-21T08:29:26.256 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.582+0000 7f52fd313700 1 -- 172.21.15.86:0/3977016187 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f52e00395a0 msgr2=0x7f52e003ba60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:26.256 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.582+0000 7f52fd313700 1 --2- 172.21.15.86:0/3977016187 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f52e00395a0 0x7f52e003ba60 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f52ec006fd0 tx=0x7f52ec006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:26.256 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.582+0000 7f52fd313700 1 -- 172.21.15.86:0/3977016187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52f81083f0 msgr2=0x7f52f819c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:26.256 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.582+0000 7f52fd313700 1 --2- 172.21.15.86:0/3977016187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52f81083f0 0x7f52f819c220 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7f52e8000c00 tx=0x7f52e8011aa0 comp rx=0 tx=0).stop 2024-04-21T08:29:26.256 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.582+0000 7f52fd313700 1 -- 172.21.15.86:0/3977016187 shutdown_connections 2024-04-21T08:29:26.256 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.582+0000 7f52fd313700 1 --2- 172.21.15.86:0/3977016187 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f52e00395a0 0x7f52e003ba60 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:26.257 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.582+0000 7f52fd313700 1 --2- 172.21.15.86:0/3977016187 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52f81083f0 0x7f52f819c220 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:26.257 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.582+0000 7f52fd313700 1 -- 172.21.15.86:0/3977016187 >> 172.21.15.86:0/3977016187 conn(0x7f52f81039f0 msgr2=0x7f52f8104460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:26.257 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.582+0000 7f52fd313700 1 -- 172.21.15.86:0/3977016187 shutdown_connections 2024-04-21T08:29:26.257 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:25.582+0000 7f52fd313700 1 -- 172.21.15.86:0/3977016187 wait complete. 2024-04-21T08:29:26.257 INFO:teuthology.orchestra.run.smithi086.stdout:Creating initial admin user... 2024-04-21T08:29:26.387 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:26 smithi086 ceph-mon[27939]: mgrmap e12: x(active, since 2s) 2024-04-21T08:29:26.387 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:26 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:29:26.387 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:26 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:29:26.388 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:26 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:29:27.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:27 smithi086 ceph-mon[27939]: from='client.14164 -' entity='client.admin' cmd=[{"prefix": "dashboard create-self-signed-cert", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:29:27.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:27 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:29:27.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:27 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:29:27.709 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout {"username": "admin", "password": "$2b$12$IasBbx30MOHV.whOaX60o.kJDjAV3nsspDnsP0NORB5ThFDnG0GIu", "roles": ["administrator"], "name": null, "email": null, "lastUpdate": 1713688167, "enabled": true, "pwdExpirationDate": null, "pwdUpdateRequired": true} 2024-04-21T08:29:27.709 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.700+0000 7f7b6e552700 1 Processor -- start 2024-04-21T08:29:27.709 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.700+0000 7f7b6e552700 1 -- start start 2024-04-21T08:29:27.709 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.700+0000 7f7b6e552700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b68108eb0 0x7f7b6810b2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:27.710 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.700+0000 7f7b6e552700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6807be60 con 0x7f7b68108eb0 2024-04-21T08:29:27.710 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.700+0000 7f7b67fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b68108eb0 0x7f7b6810b2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:27.710 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.700+0000 7f7b67fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b68108eb0 0x7f7b6810b2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50816/0 (socket says 172.21.15.86:50816) 2024-04-21T08:29:27.710 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.700+0000 7f7b67fff700 1 -- 172.21.15.86:0/2395425890 learned_addr learned my addr 172.21.15.86:0/2395425890 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:27.710 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.701+0000 7f7b67fff700 1 -- 172.21.15.86:0/2395425890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b6810b7e0 con 0x7f7b68108eb0 2024-04-21T08:29:27.710 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.701+0000 7f7b67fff700 1 --2- 172.21.15.86:0/2395425890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b68108eb0 0x7f7b6810b2a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7b580098a0 tx=0x7f7b58009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c69b02febb62ebb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:27.710 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.701+0000 7f7b66ffd700 1 -- 172.21.15.86:0/2395425890 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b58004160 con 0x7f7b68108eb0 2024-04-21T08:29:27.710 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.701+0000 7f7b66ffd700 1 -- 172.21.15.86:0/2395425890 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b58004300 con 0x7f7b68108eb0 2024-04-21T08:29:27.710 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.701+0000 7f7b66ffd700 1 -- 172.21.15.86:0/2395425890 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b58011a20 con 0x7f7b68108eb0 2024-04-21T08:29:27.713 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.702+0000 7f7b6e552700 1 -- 172.21.15.86:0/2395425890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b68108eb0 msgr2=0x7f7b6810b2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:27.713 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.702+0000 7f7b6e552700 1 --2- 172.21.15.86:0/2395425890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b68108eb0 0x7f7b6810b2a0 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f7b580098a0 tx=0x7f7b58009bb0 comp rx=0 tx=0).stop 2024-04-21T08:29:27.713 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.702+0000 7f7b6e552700 1 -- 172.21.15.86:0/2395425890 shutdown_connections 2024-04-21T08:29:27.713 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.702+0000 7f7b6e552700 1 --2- 172.21.15.86:0/2395425890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b68108eb0 0x7f7b6810b2a0 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:27.714 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.702+0000 7f7b6e552700 1 -- 172.21.15.86:0/2395425890 >> 172.21.15.86:0/2395425890 conn(0x7f7b681028c0 msgr2=0x7f7b68104ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:27.714 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.702+0000 7f7b6e552700 1 -- 172.21.15.86:0/2395425890 shutdown_connections 2024-04-21T08:29:27.714 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.702+0000 7f7b6e552700 1 -- 172.21.15.86:0/2395425890 wait complete. 2024-04-21T08:29:27.714 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.702+0000 7f7b6e552700 1 Processor -- start 2024-04-21T08:29:27.714 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.702+0000 7f7b6e552700 1 -- start start 2024-04-21T08:29:27.714 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.703+0000 7f7b6e552700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b68108eb0 0x7f7b68108730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:27.714 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.703+0000 7f7b6e552700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7b6810c7e0 con 0x7f7b68108eb0 2024-04-21T08:29:27.714 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.703+0000 7f7b67fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b68108eb0 0x7f7b68108730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:27.714 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.703+0000 7f7b67fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b68108eb0 0x7f7b68108730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50820/0 (socket says 172.21.15.86:50820) 2024-04-21T08:29:27.714 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.703+0000 7f7b67fff700 1 -- 172.21.15.86:0/158257504 learned_addr learned my addr 172.21.15.86:0/158257504 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:27.714 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.703+0000 7f7b67fff700 1 -- 172.21.15.86:0/158257504 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7b68108c70 con 0x7f7b68108eb0 2024-04-21T08:29:27.715 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.703+0000 7f7b67fff700 1 --2- 172.21.15.86:0/158257504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b68108eb0 0x7f7b68108730 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7b580116e0 tx=0x7f7b58011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:27.715 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.704+0000 7f7b657fa700 1 -- 172.21.15.86:0/158257504 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b58003b20 con 0x7f7b68108eb0 2024-04-21T08:29:27.715 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.704+0000 7f7b657fa700 1 -- 172.21.15.86:0/158257504 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7b5801b8f0 con 0x7f7b68108eb0 2024-04-21T08:29:27.715 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.704+0000 7f7b6e552700 1 -- 172.21.15.86:0/158257504 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7b68106e60 con 0x7f7b68108eb0 2024-04-21T08:29:27.715 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.704+0000 7f7b657fa700 1 -- 172.21.15.86:0/158257504 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f7b580238c0 con 0x7f7b68108eb0 2024-04-21T08:29:27.715 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.704+0000 7f7b6e552700 1 -- 172.21.15.86:0/158257504 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7b681073a0 con 0x7f7b68108eb0 2024-04-21T08:29:27.715 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.705+0000 7f7b657fa700 1 -- 172.21.15.86:0/158257504 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7f7b58023a60 con 0x7f7b68108eb0 2024-04-21T08:29:27.715 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.705+0000 7f7b657fa700 1 --2- 172.21.15.86:0/158257504 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f7b500396d0 0x7f7b5003bb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:27.715 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.705+0000 7f7b657fa700 1 -- 172.21.15.86:0/158257504 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f7b5804efe0 con 0x7f7b68108eb0 2024-04-21T08:29:27.716 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.705+0000 7f7b677fe700 1 --2- 172.21.15.86:0/158257504 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f7b500396d0 0x7f7b5003bb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:27.716 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.705+0000 7f7b6e552700 1 -- 172.21.15.86:0/158257504 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7b680640a0 con 0x7f7b68108eb0 2024-04-21T08:29:27.716 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.705+0000 7f7b677fe700 1 --2- 172.21.15.86:0/158257504 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f7b500396d0 0x7f7b5003bb90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c006fd0 tx=0x7f7b5c006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:27.716 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.712+0000 7f7b657fa700 1 -- 172.21.15.86:0/158257504 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7b5801f030 con 0x7f7b68108eb0 2024-04-21T08:29:27.716 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:26.933+0000 7f7b6e552700 1 -- 172.21.15.86:0/158257504 --> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] -- mgr_command(tid 0: {"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}) -- 0x7f7b68107a20 con 0x7f7b500396d0 2024-04-21T08:29:27.716 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:27.158+0000 7f7b657fa700 1 -- 172.21.15.86:0/158257504 <== mgr.14152 v2:172.21.15.86:6800/1364156965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+252 (secure 0 0 0) 0x7f7b68107a20 con 0x7f7b500396d0 2024-04-21T08:29:27.716 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:27.160+0000 7f7b6e552700 1 -- 172.21.15.86:0/158257504 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f7b500396d0 msgr2=0x7f7b5003bb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:27.716 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:27.160+0000 7f7b6e552700 1 --2- 172.21.15.86:0/158257504 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f7b500396d0 0x7f7b5003bb90 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7f7b5c006fd0 tx=0x7f7b5c006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:27.716 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:27.160+0000 7f7b6e552700 1 -- 172.21.15.86:0/158257504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b68108eb0 msgr2=0x7f7b68108730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:27.716 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:27.160+0000 7f7b6e552700 1 --2- 172.21.15.86:0/158257504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b68108eb0 0x7f7b68108730 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7b580116e0 tx=0x7f7b58011aa0 comp rx=0 tx=0).stop 2024-04-21T08:29:27.716 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:27.160+0000 7f7b6e552700 1 -- 172.21.15.86:0/158257504 shutdown_connections 2024-04-21T08:29:27.716 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:27.160+0000 7f7b6e552700 1 --2- 172.21.15.86:0/158257504 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f7b500396d0 0x7f7b5003bb90 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:27.716 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:27.160+0000 7f7b6e552700 1 --2- 172.21.15.86:0/158257504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7b68108eb0 0x7f7b68108730 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:27.717 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:27.160+0000 7f7b6e552700 1 -- 172.21.15.86:0/158257504 >> 172.21.15.86:0/158257504 conn(0x7f7b681028c0 msgr2=0x7f7b68103310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:27.717 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:27.160+0000 7f7b6e552700 1 -- 172.21.15.86:0/158257504 shutdown_connections 2024-04-21T08:29:27.717 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:27.160+0000 7f7b6e552700 1 -- 172.21.15.86:0/158257504 wait complete. 2024-04-21T08:29:27.717 INFO:teuthology.orchestra.run.smithi086.stdout:Fetching dashboard port number... 2024-04-21T08:29:28.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:28 smithi086 ceph-mon[27939]: from='client.14166 -' entity='client.admin' cmd=[{"prefix": "dashboard ac-user-create", "username": "admin", "rolename": "administrator", "force_password": true, "pwd_update_required": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:29:28.893 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stdout 8443 2024-04-21T08:29:28.893 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.078+0000 7fd1c8d56700 1 Processor -- start 2024-04-21T08:29:28.894 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.078+0000 7fd1c8d56700 1 -- start start 2024-04-21T08:29:28.894 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.078+0000 7fd1c8d56700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1c4108e00 0x7fd1c410b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:28.894 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.078+0000 7fd1c8d56700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c407be60 con 0x7fd1c4108e00 2024-04-21T08:29:28.894 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.078+0000 7fd1c259c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1c4108e00 0x7fd1c410b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:28.894 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.079+0000 7fd1c259c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1c4108e00 0x7fd1c410b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50822/0 (socket says 172.21.15.86:50822) 2024-04-21T08:29:28.894 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.079+0000 7fd1c259c700 1 -- 172.21.15.86:0/2291116186 learned_addr learned my addr 172.21.15.86:0/2291116186 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:28.894 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.079+0000 7fd1c259c700 1 -- 172.21.15.86:0/2291116186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1c410b730 con 0x7fd1c4108e00 2024-04-21T08:29:28.894 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.079+0000 7fd1c259c700 1 --2- 172.21.15.86:0/2291116186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1c4108e00 0x7fd1c410b1f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd1b40098a0 tx=0x7fd1b4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41b87694e4018e6f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:28.894 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.079+0000 7fd1c1d9b700 1 -- 172.21.15.86:0/2291116186 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1b4004160 con 0x7fd1c4108e00 2024-04-21T08:29:28.895 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.079+0000 7fd1c1d9b700 1 -- 172.21.15.86:0/2291116186 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1b4004300 con 0x7fd1c4108e00 2024-04-21T08:29:28.895 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.079+0000 7fd1c1d9b700 1 -- 172.21.15.86:0/2291116186 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1b4011a20 con 0x7fd1c4108e00 2024-04-21T08:29:28.895 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.080+0000 7fd1c8d56700 1 -- 172.21.15.86:0/2291116186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1c4108e00 msgr2=0x7fd1c410b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:28.895 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.080+0000 7fd1c8d56700 1 --2- 172.21.15.86:0/2291116186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1c4108e00 0x7fd1c410b1f0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7fd1b40098a0 tx=0x7fd1b4009bb0 comp rx=0 tx=0).stop 2024-04-21T08:29:28.895 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.080+0000 7fd1c8d56700 1 -- 172.21.15.86:0/2291116186 shutdown_connections 2024-04-21T08:29:28.895 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.080+0000 7fd1c8d56700 1 --2- 172.21.15.86:0/2291116186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1c4108e00 0x7fd1c410b1f0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:28.895 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.080+0000 7fd1c8d56700 1 -- 172.21.15.86:0/2291116186 >> 172.21.15.86:0/2291116186 conn(0x7fd1c41027d0 msgr2=0x7fd1c4104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:28.895 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.080+0000 7fd1c8d56700 1 -- 172.21.15.86:0/2291116186 shutdown_connections 2024-04-21T08:29:28.895 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.080+0000 7fd1c8d56700 1 -- 172.21.15.86:0/2291116186 wait complete. 2024-04-21T08:29:28.895 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.081+0000 7fd1c8d56700 1 Processor -- start 2024-04-21T08:29:28.895 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.081+0000 7fd1c8d56700 1 -- start start 2024-04-21T08:29:28.896 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.081+0000 7fd1c8d56700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1c4108e00 0x7fd1c419c170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:28.896 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.081+0000 7fd1c8d56700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd1c410c730 con 0x7fd1c4108e00 2024-04-21T08:29:28.896 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.081+0000 7fd1c259c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1c4108e00 0x7fd1c419c170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:28.896 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.081+0000 7fd1c259c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1c4108e00 0x7fd1c419c170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50832/0 (socket says 172.21.15.86:50832) 2024-04-21T08:29:28.896 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.081+0000 7fd1c259c700 1 -- 172.21.15.86:0/1378619458 learned_addr learned my addr 172.21.15.86:0/1378619458 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:28.896 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.082+0000 7fd1c259c700 1 -- 172.21.15.86:0/1378619458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1c419c6b0 con 0x7fd1c4108e00 2024-04-21T08:29:28.896 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.082+0000 7fd1c259c700 1 --2- 172.21.15.86:0/1378619458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1c4108e00 0x7fd1c419c170 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4000c00 tx=0x7fd1b4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:28.896 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.082+0000 7fd1bbfff700 1 -- 172.21.15.86:0/1378619458 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1b4003aa0 con 0x7fd1c4108e00 2024-04-21T08:29:28.896 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.082+0000 7fd1bbfff700 1 -- 172.21.15.86:0/1378619458 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1b401b8f0 con 0x7fd1c4108e00 2024-04-21T08:29:28.896 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.082+0000 7fd1c8d56700 1 -- 172.21.15.86:0/1378619458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1c419c940 con 0x7fd1c4108e00 2024-04-21T08:29:28.896 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.082+0000 7fd1bbfff700 1 -- 172.21.15.86:0/1378619458 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd1b4023870 con 0x7fd1c4108e00 2024-04-21T08:29:28.896 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.082+0000 7fd1c8d56700 1 -- 172.21.15.86:0/1378619458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1c419f630 con 0x7fd1c4108e00 2024-04-21T08:29:28.897 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.083+0000 7fd1bbfff700 1 -- 172.21.15.86:0/1378619458 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 12) ==== 46571+0+0 (secure 0 0 0) 0x7fd1b4023b70 con 0x7fd1c4108e00 2024-04-21T08:29:28.897 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.083+0000 7fd1bbfff700 1 --2- 172.21.15.86:0/1378619458 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fd1b00396c0 0x7fd1b003bb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:28.897 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.083+0000 7fd1bbfff700 1 -- 172.21.15.86:0/1378619458 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd1b404ef20 con 0x7fd1c4108e00 2024-04-21T08:29:28.897 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.083+0000 7fd1b9d9b700 1 --2- 172.21.15.86:0/1378619458 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fd1b00396c0 0x7fd1b003bb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:28.897 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.083+0000 7fd1c8d56700 1 -- 172.21.15.86:0/1378619458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1c40640a0 con 0x7fd1c4108e00 2024-04-21T08:29:28.897 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.084+0000 7fd1b9d9b700 1 --2- 172.21.15.86:0/1378619458 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fd1b00396c0 0x7fd1b003bb80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac006fd0 tx=0x7fd1ac006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:28.897 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.088+0000 7fd1bbfff700 1 -- 172.21.15.86:0/1378619458 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd1b401f030 con 0x7fd1c4108e00 2024-04-21T08:29:28.897 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.298+0000 7fd1c8d56700 1 -- 172.21.15.86:0/1378619458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"} v 0) -- 0x7fd1c419f800 con 0x7fd1c4108e00 2024-04-21T08:29:28.897 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.298+0000 7fd1bbfff700 1 -- 172.21.15.86:0/1378619458 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]=0 v8) ==== 112+0+5 (secure 0 0 0) 0x7fd1b4018e00 con 0x7fd1c4108e00 2024-04-21T08:29:28.897 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.300+0000 7fd1c8d56700 1 -- 172.21.15.86:0/1378619458 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fd1b00396c0 msgr2=0x7fd1b003bb80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:28.897 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.300+0000 7fd1c8d56700 1 --2- 172.21.15.86:0/1378619458 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fd1b00396c0 0x7fd1b003bb80 secure :-1 s=READY pgs=9 cs=0 l=1 rev1=1 crypto rx=0x7fd1ac006fd0 tx=0x7fd1ac006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:28.897 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.300+0000 7fd1c8d56700 1 -- 172.21.15.86:0/1378619458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1c4108e00 msgr2=0x7fd1c419c170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:28.897 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.300+0000 7fd1c8d56700 1 --2- 172.21.15.86:0/1378619458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1c4108e00 0x7fd1c419c170 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fd1b4000c00 tx=0x7fd1b4011aa0 comp rx=0 tx=0).stop 2024-04-21T08:29:28.898 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.300+0000 7fd1c8d56700 1 -- 172.21.15.86:0/1378619458 shutdown_connections 2024-04-21T08:29:28.898 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.300+0000 7fd1c8d56700 1 --2- 172.21.15.86:0/1378619458 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fd1b00396c0 0x7fd1b003bb80 unknown :-1 s=CLOSED pgs=9 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:28.898 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.300+0000 7fd1c8d56700 1 --2- 172.21.15.86:0/1378619458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1c4108e00 0x7fd1c419c170 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:28.898 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.300+0000 7fd1c8d56700 1 -- 172.21.15.86:0/1378619458 >> 172.21.15.86:0/1378619458 conn(0x7fd1c41027d0 msgr2=0x7fd1c41032a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:28.898 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.301+0000 7fd1c8d56700 1 -- 172.21.15.86:0/1378619458 shutdown_connections 2024-04-21T08:29:28.898 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:28.301+0000 7fd1c8d56700 1 -- 172.21.15.86:0/1378619458 wait complete. 2024-04-21T08:29:28.899 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 1 from systemctl is-enabled firewalld.service 2024-04-21T08:29:28.899 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout disabled 2024-04-21T08:29:28.905 INFO:teuthology.orchestra.run.smithi086.stdout:Non-zero exit code 3 from systemctl is-active firewalld.service 2024-04-21T08:29:28.905 INFO:teuthology.orchestra.run.smithi086.stdout:systemctl: stdout inactive 2024-04-21T08:29:28.905 INFO:teuthology.orchestra.run.smithi086.stdout:firewalld.service is not enabled 2024-04-21T08:29:28.905 INFO:teuthology.orchestra.run.smithi086.stdout:Not possible to open ports <[8443]>. firewalld.service is not available 2024-04-21T08:29:28.906 INFO:teuthology.orchestra.run.smithi086.stdout:Ceph Dashboard is now available at: 2024-04-21T08:29:28.906 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:29:28.906 INFO:teuthology.orchestra.run.smithi086.stdout: URL: https://smithi086:8443/ 2024-04-21T08:29:28.906 INFO:teuthology.orchestra.run.smithi086.stdout: User: admin 2024-04-21T08:29:28.906 INFO:teuthology.orchestra.run.smithi086.stdout: Password: yoqyp2n5ik 2024-04-21T08:29:28.906 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:29:28.907 INFO:teuthology.orchestra.run.smithi086.stdout:Saving cluster configuration to /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config directory 2024-04-21T08:29:28.907 INFO:teuthology.orchestra.run.smithi086.stdout:Enabling autotune for osd_memory_target 2024-04-21T08:29:29.574 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:29 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1378619458' entity='client.admin' cmd=[{"prefix": "config get", "who": "mgr", "key": "mgr/dashboard/ssl_server_port"}]: dispatch 2024-04-21T08:29:30.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:30 smithi086 ceph-mon[27939]: mgrmap e13: x(active, since 6s) 2024-04-21T08:29:30.693 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.771+0000 7fd90eae1700 1 Processor -- start 2024-04-21T08:29:30.693 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.771+0000 7fd90eae1700 1 -- start start 2024-04-21T08:29:30.693 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.771+0000 7fd90eae1700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd908108ee0 0x7fd90810b2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:30.693 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.771+0000 7fd90eae1700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd90807be40 con 0x7fd908108ee0 2024-04-21T08:29:30.693 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.771+0000 7fd90c87d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd908108ee0 0x7fd90810b2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:30.693 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.771+0000 7fd90c87d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd908108ee0 0x7fd90810b2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36946/0 (socket says 172.21.15.86:36946) 2024-04-21T08:29:30.693 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.771+0000 7fd90c87d700 1 -- 172.21.15.86:0/320757044 learned_addr learned my addr 172.21.15.86:0/320757044 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:30.693 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.772+0000 7fd90c87d700 1 -- 172.21.15.86:0/320757044 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd90810b810 con 0x7fd908108ee0 2024-04-21T08:29:30.694 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.772+0000 7fd90c87d700 1 --2- 172.21.15.86:0/320757044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd908108ee0 0x7fd90810b2d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc0098a0 tx=0x7fd8fc009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57e403c415d7c368 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:30.694 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.772+0000 7fd9077fe700 1 -- 172.21.15.86:0/320757044 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8fc004160 con 0x7fd908108ee0 2024-04-21T08:29:30.694 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.772+0000 7fd9077fe700 1 -- 172.21.15.86:0/320757044 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8fc004300 con 0x7fd908108ee0 2024-04-21T08:29:30.694 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.772+0000 7fd9077fe700 1 -- 172.21.15.86:0/320757044 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8fc011a20 con 0x7fd908108ee0 2024-04-21T08:29:30.694 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.772+0000 7fd90eae1700 1 -- 172.21.15.86:0/320757044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd908108ee0 msgr2=0x7fd90810b2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:30.694 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.772+0000 7fd90eae1700 1 --2- 172.21.15.86:0/320757044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd908108ee0 0x7fd90810b2d0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc0098a0 tx=0x7fd8fc009bb0 comp rx=0 tx=0).stop 2024-04-21T08:29:30.694 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.773+0000 7fd90eae1700 1 -- 172.21.15.86:0/320757044 shutdown_connections 2024-04-21T08:29:30.694 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.773+0000 7fd90eae1700 1 --2- 172.21.15.86:0/320757044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd908108ee0 0x7fd90810b2d0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:30.694 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.773+0000 7fd90eae1700 1 -- 172.21.15.86:0/320757044 >> 172.21.15.86:0/320757044 conn(0x7fd9081028b0 msgr2=0x7fd908104cf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:30.694 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.773+0000 7fd90eae1700 1 -- 172.21.15.86:0/320757044 shutdown_connections 2024-04-21T08:29:30.694 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.773+0000 7fd90eae1700 1 -- 172.21.15.86:0/320757044 wait complete. 2024-04-21T08:29:30.694 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.773+0000 7fd90eae1700 1 Processor -- start 2024-04-21T08:29:30.694 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.773+0000 7fd90eae1700 1 -- start start 2024-04-21T08:29:30.695 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.773+0000 7fd90eae1700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd908108ee0 0x7fd90819c1d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:30.695 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.773+0000 7fd90eae1700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd90810c7c0 con 0x7fd908108ee0 2024-04-21T08:29:30.695 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.773+0000 7fd90c87d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd908108ee0 0x7fd90819c1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:30.695 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.773+0000 7fd90c87d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd908108ee0 0x7fd90819c1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36956/0 (socket says 172.21.15.86:36956) 2024-04-21T08:29:30.695 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.773+0000 7fd90c87d700 1 -- 172.21.15.86:0/3817149083 learned_addr learned my addr 172.21.15.86:0/3817149083 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:30.695 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.774+0000 7fd90c87d700 1 -- 172.21.15.86:0/3817149083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd90819c710 con 0x7fd908108ee0 2024-04-21T08:29:30.695 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.774+0000 7fd90c87d700 1 --2- 172.21.15.86:0/3817149083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd908108ee0 0x7fd90819c1d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc004660 tx=0x7fd8fc011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:30.695 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.774+0000 7fd905ffb700 1 -- 172.21.15.86:0/3817149083 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8fc003aa0 con 0x7fd908108ee0 2024-04-21T08:29:30.695 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.774+0000 7fd90eae1700 1 -- 172.21.15.86:0/3817149083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd90819c9a0 con 0x7fd908108ee0 2024-04-21T08:29:30.695 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.774+0000 7fd905ffb700 1 -- 172.21.15.86:0/3817149083 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8fc01b8f0 con 0x7fd908108ee0 2024-04-21T08:29:30.695 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.774+0000 7fd905ffb700 1 -- 172.21.15.86:0/3817149083 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fd8fc023900 con 0x7fd908108ee0 2024-04-21T08:29:30.695 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.774+0000 7fd90eae1700 1 -- 172.21.15.86:0/3817149083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd90819f690 con 0x7fd908108ee0 2024-04-21T08:29:30.695 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.775+0000 7fd905ffb700 1 -- 172.21.15.86:0/3817149083 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fd8fc023bc0 con 0x7fd908108ee0 2024-04-21T08:29:30.695 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.775+0000 7fd90eae1700 1 -- 172.21.15.86:0/3817149083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd9080640a0 con 0x7fd908108ee0 2024-04-21T08:29:30.696 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.775+0000 7fd905ffb700 1 --2- 172.21.15.86:0/3817149083 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fd8f8039760 0x7fd8f803bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:30.696 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.775+0000 7fd905ffb700 1 -- 172.21.15.86:0/3817149083 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fd8fc04e260 con 0x7fd908108ee0 2024-04-21T08:29:30.696 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.776+0000 7fd907fff700 1 --2- 172.21.15.86:0/3817149083 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fd8f8039760 0x7fd8f803bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:30.696 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.776+0000 7fd907fff700 1 --2- 172.21.15.86:0/3817149083 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fd8f8039760 0x7fd8f803bc20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4006fd0 tx=0x7fd8f4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:30.696 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.780+0000 7fd905ffb700 1 -- 172.21.15.86:0/3817149083 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd8fc01f030 con 0x7fd908108ee0 2024-04-21T08:29:30.696 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.983+0000 7fd90eae1700 1 -- 172.21.15.86:0/3817149083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=osd_memory_target_autotune}] v 0) -- 0x7fd90819fcf0 con 0x7fd908108ee0 2024-04-21T08:29:30.696 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.984+0000 7fd905ffb700 1 -- 172.21.15.86:0/3817149083 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=osd_memory_target_autotune}]=0 v8)=0 v8) ==== 127+0+0 (secure 0 0 0) 0x7fd8fc010380 con 0x7fd908108ee0 2024-04-21T08:29:30.696 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.986+0000 7fd90eae1700 1 -- 172.21.15.86:0/3817149083 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fd8f8039760 msgr2=0x7fd8f803bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:30.696 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.986+0000 7fd90eae1700 1 --2- 172.21.15.86:0/3817149083 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fd8f8039760 0x7fd8f803bc20 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4006fd0 tx=0x7fd8f4006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:30.696 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.986+0000 7fd90eae1700 1 -- 172.21.15.86:0/3817149083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd908108ee0 msgr2=0x7fd90819c1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:30.696 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.986+0000 7fd90eae1700 1 --2- 172.21.15.86:0/3817149083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd908108ee0 0x7fd90819c1d0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7fd8fc004660 tx=0x7fd8fc011aa0 comp rx=0 tx=0).stop 2024-04-21T08:29:30.696 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.986+0000 7fd90eae1700 1 -- 172.21.15.86:0/3817149083 shutdown_connections 2024-04-21T08:29:30.696 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.986+0000 7fd90eae1700 1 --2- 172.21.15.86:0/3817149083 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fd8f8039760 0x7fd8f803bc20 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:30.696 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.986+0000 7fd90eae1700 1 --2- 172.21.15.86:0/3817149083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd908108ee0 0x7fd90819c1d0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:30.697 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.986+0000 7fd90eae1700 1 -- 172.21.15.86:0/3817149083 >> 172.21.15.86:0/3817149083 conn(0x7fd9081028b0 msgr2=0x7fd908103300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:30.697 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.986+0000 7fd90eae1700 1 -- 172.21.15.86:0/3817149083 shutdown_connections 2024-04-21T08:29:30.697 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:29.986+0000 7fd90eae1700 1 -- 172.21.15.86:0/3817149083 wait complete. 2024-04-21T08:29:32.016 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.113+0000 7f1fe8981700 1 Processor -- start 2024-04-21T08:29:32.016 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.113+0000 7f1fe8981700 1 -- start start 2024-04-21T08:29:32.016 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.113+0000 7f1fe8981700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe41084b0 0x7f1fe41088d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:32.016 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.113+0000 7f1fe8981700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe4108ea0 con 0x7f1fe41084b0 2024-04-21T08:29:32.016 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.113+0000 7f1fe259c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe41084b0 0x7f1fe41088d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:32.016 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.113+0000 7f1fe259c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe41084b0 0x7f1fe41088d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36968/0 (socket says 172.21.15.86:36968) 2024-04-21T08:29:32.016 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.113+0000 7f1fe259c700 1 -- 172.21.15.86:0/4033135267 learned_addr learned my addr 172.21.15.86:0/4033135267 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:32.016 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.114+0000 7f1fe259c700 1 -- 172.21.15.86:0/4033135267 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe4109660 con 0x7f1fe41084b0 2024-04-21T08:29:32.017 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.114+0000 7f1fe259c700 1 --2- 172.21.15.86:0/4033135267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe41084b0 0x7f1fe41088d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1fd40098a0 tx=0x7f1fd4009bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac82add53cbcfcb6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:32.017 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.114+0000 7f1fe159a700 1 -- 172.21.15.86:0/4033135267 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fd4004160 con 0x7f1fe41084b0 2024-04-21T08:29:32.017 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.114+0000 7f1fe159a700 1 -- 172.21.15.86:0/4033135267 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fd4004300 con 0x7f1fe41084b0 2024-04-21T08:29:32.017 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.114+0000 7f1fe159a700 1 -- 172.21.15.86:0/4033135267 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fd4011a20 con 0x7f1fe41084b0 2024-04-21T08:29:32.017 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.114+0000 7f1fe8981700 1 -- 172.21.15.86:0/4033135267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe41084b0 msgr2=0x7f1fe41088d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:32.017 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.114+0000 7f1fe8981700 1 --2- 172.21.15.86:0/4033135267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe41084b0 0x7f1fe41088d0 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f1fd40098a0 tx=0x7f1fd4009bb0 comp rx=0 tx=0).stop 2024-04-21T08:29:32.017 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.115+0000 7f1fe8981700 1 -- 172.21.15.86:0/4033135267 shutdown_connections 2024-04-21T08:29:32.017 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.115+0000 7f1fe8981700 1 --2- 172.21.15.86:0/4033135267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe41084b0 0x7f1fe41088d0 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:32.017 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.115+0000 7f1fe8981700 1 -- 172.21.15.86:0/4033135267 >> 172.21.15.86:0/4033135267 conn(0x7f1fe4103af0 msgr2=0x7f1fe4105f10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:32.017 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.115+0000 7f1fe8981700 1 -- 172.21.15.86:0/4033135267 shutdown_connections 2024-04-21T08:29:32.017 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.115+0000 7f1fe8981700 1 -- 172.21.15.86:0/4033135267 wait complete. 2024-04-21T08:29:32.017 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.115+0000 7f1fe8981700 1 Processor -- start 2024-04-21T08:29:32.017 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.115+0000 7f1fe8981700 1 -- start start 2024-04-21T08:29:32.017 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.116+0000 7f1fe8981700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe41084b0 0x7f1fe407db60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:32.017 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.116+0000 7f1fe8981700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe410ca10 con 0x7f1fe41084b0 2024-04-21T08:29:32.018 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.116+0000 7f1fe259c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe41084b0 0x7f1fe407db60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:32.018 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.116+0000 7f1fe259c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe41084b0 0x7f1fe407db60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36976/0 (socket says 172.21.15.86:36976) 2024-04-21T08:29:32.018 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.116+0000 7f1fe259c700 1 -- 172.21.15.86:0/3411168484 learned_addr learned my addr 172.21.15.86:0/3411168484 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:32.018 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.116+0000 7f1fe259c700 1 -- 172.21.15.86:0/3411168484 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe407e0a0 con 0x7f1fe41084b0 2024-04-21T08:29:32.018 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.116+0000 7f1fe259c700 1 --2- 172.21.15.86:0/3411168484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe41084b0 0x7f1fe407db60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4009520 tx=0x7f1fd4011aa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:32.018 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.116+0000 7f1fd37fe700 1 -- 172.21.15.86:0/3411168484 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fd40039e0 con 0x7f1fe41084b0 2024-04-21T08:29:32.018 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.116+0000 7f1fe8981700 1 -- 172.21.15.86:0/3411168484 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe407c290 con 0x7f1fe41084b0 2024-04-21T08:29:32.018 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.116+0000 7f1fd37fe700 1 -- 172.21.15.86:0/3411168484 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fd401b8f0 con 0x7f1fe41084b0 2024-04-21T08:29:32.018 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.116+0000 7f1fd37fe700 1 -- 172.21.15.86:0/3411168484 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1fd4019770 con 0x7f1fe41084b0 2024-04-21T08:29:32.018 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.116+0000 7f1fe8981700 1 -- 172.21.15.86:0/3411168484 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe407c7d0 con 0x7f1fe41084b0 2024-04-21T08:29:32.018 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.117+0000 7f1fd37fe700 1 -- 172.21.15.86:0/3411168484 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f1fd4019a70 con 0x7f1fe41084b0 2024-04-21T08:29:32.018 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.117+0000 7f1fe8981700 1 -- 172.21.15.86:0/3411168484 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1fe40640a0 con 0x7f1fe41084b0 2024-04-21T08:29:32.018 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.117+0000 7f1fd37fe700 1 --2- 172.21.15.86:0/3411168484 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f1fcc039760 0x7f1fcc03bc20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:32.018 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.117+0000 7f1fd37fe700 1 -- 172.21.15.86:0/3411168484 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f1fd404e280 con 0x7f1fe41084b0 2024-04-21T08:29:32.019 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.117+0000 7f1fe1d9b700 1 --2- 172.21.15.86:0/3411168484 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f1fcc039760 0x7f1fcc03bc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:32.019 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.118+0000 7f1fe1d9b700 1 --2- 172.21.15.86:0/3411168484 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f1fcc039760 0x7f1fcc03bc20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8006fd0 tx=0x7f1fd8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:32.019 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.122+0000 7f1fd37fe700 1 -- 172.21.15.86:0/3411168484 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1fd401f030 con 0x7f1fe41084b0 2024-04-21T08:29:32.019 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.433+0000 7f1fe8981700 1 -- 172.21.15.86:0/3411168484 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config-key set, key=mgr/dashboard/cluster/status}] v 0) -- 0x7f1fe407ca50 con 0x7f1fe41084b0 2024-04-21T08:29:32.019 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.435+0000 7f1fd37fe700 1 -- 172.21.15.86:0/3411168484 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config-key set, key=mgr/dashboard/cluster/status}]=0 set mgr/dashboard/cluster/status v27)=0 set mgr/dashboard/cluster/status v27) ==== 153+0+0 (secure 0 0 0) 0x7f1fd4010a10 con 0x7f1fe41084b0 2024-04-21T08:29:32.019 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.437+0000 7f1fe8981700 1 -- 172.21.15.86:0/3411168484 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f1fcc039760 msgr2=0x7f1fcc03bc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:32.019 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.437+0000 7f1fe8981700 1 --2- 172.21.15.86:0/3411168484 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f1fcc039760 0x7f1fcc03bc20 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8006fd0 tx=0x7f1fd8006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:32.019 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.437+0000 7f1fe8981700 1 -- 172.21.15.86:0/3411168484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe41084b0 msgr2=0x7f1fe407db60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:32.019 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.437+0000 7f1fe8981700 1 --2- 172.21.15.86:0/3411168484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe41084b0 0x7f1fe407db60 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4009520 tx=0x7f1fd4011aa0 comp rx=0 tx=0).stop 2024-04-21T08:29:32.019 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.437+0000 7f1fe8981700 1 -- 172.21.15.86:0/3411168484 shutdown_connections 2024-04-21T08:29:32.019 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.437+0000 7f1fe8981700 1 --2- 172.21.15.86:0/3411168484 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f1fcc039760 0x7f1fcc03bc20 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:32.019 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.437+0000 7f1fe8981700 1 --2- 172.21.15.86:0/3411168484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe41084b0 0x7f1fe407db60 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:32.019 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.437+0000 7f1fe8981700 1 -- 172.21.15.86:0/3411168484 >> 172.21.15.86:0/3411168484 conn(0x7f1fe4103af0 msgr2=0x7f1fe41044a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:32.020 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.438+0000 7f1fe8981700 1 -- 172.21.15.86:0/3411168484 shutdown_connections 2024-04-21T08:29:32.020 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr 2024-04-21T08:29:31.438+0000 7f1fe8981700 1 -- 172.21.15.86:0/3411168484 wait complete. 2024-04-21T08:29:32.020 INFO:teuthology.orchestra.run.smithi086.stdout:/usr/bin/ceph: stderr set mgr/dashboard/cluster/status 2024-04-21T08:29:32.020 INFO:teuthology.orchestra.run.smithi086.stdout:You can access the Ceph CLI as following in case of multi-cluster or non-default config: 2024-04-21T08:29:32.020 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:29:32.020 INFO:teuthology.orchestra.run.smithi086.stdout: sudo /home/ubuntu/cephtest/cephadm shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring 2024-04-21T08:29:32.020 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:29:32.020 INFO:teuthology.orchestra.run.smithi086.stdout:Or, if you are only running a single cluster on this host: 2024-04-21T08:29:32.020 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:29:32.020 INFO:teuthology.orchestra.run.smithi086.stdout: sudo /home/ubuntu/cephtest/cephadm shell 2024-04-21T08:29:32.020 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:29:32.020 INFO:teuthology.orchestra.run.smithi086.stdout:Please consider enabling telemetry to help improve Ceph: 2024-04-21T08:29:32.020 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:29:32.020 INFO:teuthology.orchestra.run.smithi086.stdout: ceph telemetry on 2024-04-21T08:29:32.020 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:29:32.021 INFO:teuthology.orchestra.run.smithi086.stdout:For more information see: 2024-04-21T08:29:32.021 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:29:32.021 INFO:teuthology.orchestra.run.smithi086.stdout: https://docs.ceph.com/en/latest/mgr/telemetry/ 2024-04-21T08:29:32.021 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:29:32.021 INFO:teuthology.orchestra.run.smithi086.stdout:Bootstrap complete. 2024-04-21T08:29:32.053 INFO:tasks.cephadm:Fetching config... 2024-04-21T08:29:32.054 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-21T08:29:32.054 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/etc/ceph/ceph.conf of=/dev/stdout 2024-04-21T08:29:32.069 INFO:tasks.cephadm:Fetching client.admin keyring... 2024-04-21T08:29:32.069 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-21T08:29:32.070 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/etc/ceph/ceph.client.admin.keyring of=/dev/stdout 2024-04-21T08:29:32.125 INFO:tasks.cephadm:Fetching mon keyring... 2024-04-21T08:29:32.125 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-21T08:29:32.125 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/keyring of=/dev/stdout 2024-04-21T08:29:32.188 INFO:tasks.cephadm:Fetching pub ssh key... 2024-04-21T08:29:32.188 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-21T08:29:32.189 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/home/ubuntu/cephtest/ceph.pub of=/dev/stdout 2024-04-21T08:29:32.243 INFO:tasks.cephadm:Installing pub ssh key for root users... 2024-04-21T08:29:32.243 DEBUG:teuthology.orchestra.run.smithi086:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6kglyLLRmoLt8PcpxPgppEi4yJspZMBDoafJN50b/Kv1iDtR4x6WeIhI7YYuOnpcUKh5vBpkoRuF2oLVQ4ia5tZV6VXy2eOKB61WtSG2C2so8O/4xKbvTovqS9C5fCU6JBjknroUSIUQjr+wNMF9tbXiI07evecXfR6Vlrp1L/cDgzfvDfCvgRXiUjlCUnL0hCFBtAZ4L07pSyvMfp/LqlSPhcoe2oMgxHvryc6jXy1Iyta9+WLRRt3eqQFnx7fQINBGNtb0ZRtlPAvmIoJwDl0vSFw4yYsvzrH39shxthSQ4D0ps2tcUNk9LCPuxWx9yloixVpg3+sCDN9eDHaFVU3iCcEhuuz5Rc5Iufcy0i8SDpSC40Hs5+m6xHYPj3zbdZkseYmjMXk12Zg3wIcRLO5SLiscVHUtmEDbUCpTvXYqMbMm1KJgkNaqs9rwz8yl0+E+vcqz3itBsxr8192eVh8H7NJfiTxTM4kyPp9ebNReo9kZ5+O84zuLEJl61g20= ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-21T08:29:32.314 INFO:teuthology.orchestra.run.smithi086.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6kglyLLRmoLt8PcpxPgppEi4yJspZMBDoafJN50b/Kv1iDtR4x6WeIhI7YYuOnpcUKh5vBpkoRuF2oLVQ4ia5tZV6VXy2eOKB61WtSG2C2so8O/4xKbvTovqS9C5fCU6JBjknroUSIUQjr+wNMF9tbXiI07evecXfR6Vlrp1L/cDgzfvDfCvgRXiUjlCUnL0hCFBtAZ4L07pSyvMfp/LqlSPhcoe2oMgxHvryc6jXy1Iyta9+WLRRt3eqQFnx7fQINBGNtb0ZRtlPAvmIoJwDl0vSFw4yYsvzrH39shxthSQ4D0ps2tcUNk9LCPuxWx9yloixVpg3+sCDN9eDHaFVU3iCcEhuuz5Rc5Iufcy0i8SDpSC40Hs5+m6xHYPj3zbdZkseYmjMXk12Zg3wIcRLO5SLiscVHUtmEDbUCpTvXYqMbMm1KJgkNaqs9rwz8yl0+E+vcqz3itBsxr8192eVh8H7NJfiTxTM4kyPp9ebNReo9kZ5+O84zuLEJl61g20= ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:29:32.325 DEBUG:teuthology.orchestra.run.smithi088:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6kglyLLRmoLt8PcpxPgppEi4yJspZMBDoafJN50b/Kv1iDtR4x6WeIhI7YYuOnpcUKh5vBpkoRuF2oLVQ4ia5tZV6VXy2eOKB61WtSG2C2so8O/4xKbvTovqS9C5fCU6JBjknroUSIUQjr+wNMF9tbXiI07evecXfR6Vlrp1L/cDgzfvDfCvgRXiUjlCUnL0hCFBtAZ4L07pSyvMfp/LqlSPhcoe2oMgxHvryc6jXy1Iyta9+WLRRt3eqQFnx7fQINBGNtb0ZRtlPAvmIoJwDl0vSFw4yYsvzrH39shxthSQ4D0ps2tcUNk9LCPuxWx9yloixVpg3+sCDN9eDHaFVU3iCcEhuuz5Rc5Iufcy0i8SDpSC40Hs5+m6xHYPj3zbdZkseYmjMXk12Zg3wIcRLO5SLiscVHUtmEDbUCpTvXYqMbMm1KJgkNaqs9rwz8yl0+E+vcqz3itBsxr8192eVh8H7NJfiTxTM4kyPp9ebNReo9kZ5+O84zuLEJl61g20= ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-21T08:29:32.364 INFO:teuthology.orchestra.run.smithi088.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6kglyLLRmoLt8PcpxPgppEi4yJspZMBDoafJN50b/Kv1iDtR4x6WeIhI7YYuOnpcUKh5vBpkoRuF2oLVQ4ia5tZV6VXy2eOKB61WtSG2C2so8O/4xKbvTovqS9C5fCU6JBjknroUSIUQjr+wNMF9tbXiI07evecXfR6Vlrp1L/cDgzfvDfCvgRXiUjlCUnL0hCFBtAZ4L07pSyvMfp/LqlSPhcoe2oMgxHvryc6jXy1Iyta9+WLRRt3eqQFnx7fQINBGNtb0ZRtlPAvmIoJwDl0vSFw4yYsvzrH39shxthSQ4D0ps2tcUNk9LCPuxWx9yloixVpg3+sCDN9eDHaFVU3iCcEhuuz5Rc5Iufcy0i8SDpSC40Hs5+m6xHYPj3zbdZkseYmjMXk12Zg3wIcRLO5SLiscVHUtmEDbUCpTvXYqMbMm1KJgkNaqs9rwz8yl0+E+vcqz3itBsxr8192eVh8H7NJfiTxTM4kyPp9ebNReo9kZ5+O84zuLEJl61g20= ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:29:32.376 DEBUG:teuthology.orchestra.run.smithi179:> sudo install -d -m 0700 /root/.ssh && echo 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6kglyLLRmoLt8PcpxPgppEi4yJspZMBDoafJN50b/Kv1iDtR4x6WeIhI7YYuOnpcUKh5vBpkoRuF2oLVQ4ia5tZV6VXy2eOKB61WtSG2C2so8O/4xKbvTovqS9C5fCU6JBjknroUSIUQjr+wNMF9tbXiI07evecXfR6Vlrp1L/cDgzfvDfCvgRXiUjlCUnL0hCFBtAZ4L07pSyvMfp/LqlSPhcoe2oMgxHvryc6jXy1Iyta9+WLRRt3eqQFnx7fQINBGNtb0ZRtlPAvmIoJwDl0vSFw4yYsvzrH39shxthSQ4D0ps2tcUNk9LCPuxWx9yloixVpg3+sCDN9eDHaFVU3iCcEhuuz5Rc5Iufcy0i8SDpSC40Hs5+m6xHYPj3zbdZkseYmjMXk12Zg3wIcRLO5SLiscVHUtmEDbUCpTvXYqMbMm1KJgkNaqs9rwz8yl0+E+vcqz3itBsxr8192eVh8H7NJfiTxTM4kyPp9ebNReo9kZ5+O84zuLEJl61g20= ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968' | sudo tee -a /root/.ssh/authorized_keys && sudo chmod 0600 /root/.ssh/authorized_keys 2024-04-21T08:29:32.415 INFO:teuthology.orchestra.run.smithi179.stdout:ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQC6kglyLLRmoLt8PcpxPgppEi4yJspZMBDoafJN50b/Kv1iDtR4x6WeIhI7YYuOnpcUKh5vBpkoRuF2oLVQ4ia5tZV6VXy2eOKB61WtSG2C2so8O/4xKbvTovqS9C5fCU6JBjknroUSIUQjr+wNMF9tbXiI07evecXfR6Vlrp1L/cDgzfvDfCvgRXiUjlCUnL0hCFBtAZ4L07pSyvMfp/LqlSPhcoe2oMgxHvryc6jXy1Iyta9+WLRRt3eqQFnx7fQINBGNtb0ZRtlPAvmIoJwDl0vSFw4yYsvzrH39shxthSQ4D0ps2tcUNk9LCPuxWx9yloixVpg3+sCDN9eDHaFVU3iCcEhuuz5Rc5Iufcy0i8SDpSC40Hs5+m6xHYPj3zbdZkseYmjMXk12Zg3wIcRLO5SLiscVHUtmEDbUCpTvXYqMbMm1KJgkNaqs9rwz8yl0+E+vcqz3itBsxr8192eVh8H7NJfiTxTM4kyPp9ebNReo9kZ5+O84zuLEJl61g20= ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:29:32.427 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph config set mgr mgr/cephadm/allow_ptrace true 2024-04-21T08:29:32.561 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:32 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3411168484' entity='client.admin' 2024-04-21T08:29:32.677 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:29:34.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:34.911+0000 7fe081512700 1 -- 172.21.15.86:0/1044889183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe07c102f30 msgr2=0x7fe07c105320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:34.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:34.911+0000 7fe081512700 1 --2- 172.21.15.86:0/1044889183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe07c102f30 0x7fe07c105320 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7fe06c009910 tx=0x7fe06c009c20 comp rx=0 tx=0).stop 2024-04-21T08:29:34.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:34.911+0000 7fe081512700 1 -- 172.21.15.86:0/1044889183 shutdown_connections 2024-04-21T08:29:34.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:34.911+0000 7fe081512700 1 --2- 172.21.15.86:0/1044889183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe07c102f30 0x7fe07c105320 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:34.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:34.911+0000 7fe081512700 1 -- 172.21.15.86:0/1044889183 >> 172.21.15.86:0/1044889183 conn(0x7fe07c0fc940 msgr2=0x7fe07c0fed80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:34.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:34.911+0000 7fe081512700 1 -- 172.21.15.86:0/1044889183 shutdown_connections 2024-04-21T08:29:34.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:34.911+0000 7fe081512700 1 -- 172.21.15.86:0/1044889183 wait complete. 2024-04-21T08:29:34.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:34.911+0000 7fe081512700 1 Processor -- start 2024-04-21T08:29:34.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:34.912+0000 7fe081512700 1 -- start start 2024-04-21T08:29:34.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:34.912+0000 7fe081512700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe07c102f30 0x7fe07c197a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:34.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:34.912+0000 7fe081512700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe07c108380 con 0x7fe07c102f30 2024-04-21T08:29:34.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:34.912+0000 7fe07affd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe07c102f30 0x7fe07c197a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:34.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:34.912+0000 7fe07affd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe07c102f30 0x7fe07c197a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36994/0 (socket says 172.21.15.86:36994) 2024-04-21T08:29:34.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:34.912+0000 7fe07affd700 1 -- 172.21.15.86:0/4230601215 learned_addr learned my addr 172.21.15.86:0/4230601215 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:34.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:34.912+0000 7fe07affd700 1 -- 172.21.15.86:0/4230601215 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe07c197f70 con 0x7fe07c102f30 2024-04-21T08:29:34.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:34.912+0000 7fe07affd700 1 --2- 172.21.15.86:0/4230601215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe07c102f30 0x7fe07c197a30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe06c005190 tx=0x7fe06c004f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:34.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:34.912+0000 7fe078ff9700 1 -- 172.21.15.86:0/4230601215 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe06c027070 con 0x7fe07c102f30 2024-04-21T08:29:34.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:34.913+0000 7fe081512700 1 -- 172.21.15.86:0/4230601215 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe07c198260 con 0x7fe07c102f30 2024-04-21T08:29:34.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:34.913+0000 7fe078ff9700 1 -- 172.21.15.86:0/4230601215 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe06c022470 con 0x7fe07c102f30 2024-04-21T08:29:34.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:34.913+0000 7fe078ff9700 1 -- 172.21.15.86:0/4230601215 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fe06c00f5f0 con 0x7fe07c102f30 2024-04-21T08:29:34.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:34.913+0000 7fe081512700 1 -- 172.21.15.86:0/4230601215 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe07c19aef0 con 0x7fe07c102f30 2024-04-21T08:29:34.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:34.913+0000 7fe078ff9700 1 -- 172.21.15.86:0/4230601215 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fe06c02a070 con 0x7fe07c102f30 2024-04-21T08:29:34.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:34.913+0000 7fe078ff9700 1 --2- 172.21.15.86:0/4230601215 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe068039650 0x7fe06803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:34.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:34.914+0000 7fe078ff9700 1 -- 172.21.15.86:0/4230601215 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fe06c01e030 con 0x7fe07c102f30 2024-04-21T08:29:34.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:34.914+0000 7fe0727fc700 1 --2- 172.21.15.86:0/4230601215 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe068039650 0x7fe06803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:34.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:34.914+0000 7fe081512700 1 -- 172.21.15.86:0/4230601215 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe07c108b00 con 0x7fe07c102f30 2024-04-21T08:29:34.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:34.914+0000 7fe0727fc700 1 --2- 172.21.15.86:0/4230601215 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe068039650 0x7fe06803bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe064006fd0 tx=0x7fe064006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:34.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:34.920+0000 7fe078ff9700 1 -- 172.21.15.86:0/4230601215 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe06c031070 con 0x7fe07c102f30 2024-04-21T08:29:35.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:35.115+0000 7fe081512700 1 -- 172.21.15.86:0/4230601215 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mgr/cephadm/allow_ptrace}] v 0) -- 0x7fe07c02df80 con 0x7fe07c102f30 2024-04-21T08:29:35.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:35.117+0000 7fe078ff9700 1 -- 172.21.15.86:0/4230601215 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/cephadm/allow_ptrace}]=0 v9)=0 v9) ==== 125+0+0 (secure 0 0 0) 0x7fe06c013070 con 0x7fe07c102f30 2024-04-21T08:29:35.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:35.119+0000 7fe081512700 1 -- 172.21.15.86:0/4230601215 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe068039650 msgr2=0x7fe06803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:35.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:35.119+0000 7fe081512700 1 --2- 172.21.15.86:0/4230601215 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe068039650 0x7fe06803bb10 secure :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0x7fe064006fd0 tx=0x7fe064006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:35.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:35.119+0000 7fe081512700 1 -- 172.21.15.86:0/4230601215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe07c102f30 msgr2=0x7fe07c197a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:35.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:35.119+0000 7fe081512700 1 --2- 172.21.15.86:0/4230601215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe07c102f30 0x7fe07c197a30 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7fe06c005190 tx=0x7fe06c004f80 comp rx=0 tx=0).stop 2024-04-21T08:29:35.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:35.119+0000 7fe081512700 1 -- 172.21.15.86:0/4230601215 shutdown_connections 2024-04-21T08:29:35.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:35.119+0000 7fe081512700 1 --2- 172.21.15.86:0/4230601215 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe068039650 0x7fe06803bb10 unknown :-1 s=CLOSED pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:35.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:35.119+0000 7fe081512700 1 --2- 172.21.15.86:0/4230601215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe07c102f30 0x7fe07c197a30 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:35.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:35.119+0000 7fe081512700 1 -- 172.21.15.86:0/4230601215 >> 172.21.15.86:0/4230601215 conn(0x7fe07c0fc940 msgr2=0x7fe07c0fd3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:35.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:35.119+0000 7fe081512700 1 -- 172.21.15.86:0/4230601215 shutdown_connections 2024-04-21T08:29:35.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:35.119+0000 7fe081512700 1 -- 172.21.15.86:0/4230601215 wait complete. 2024-04-21T08:29:35.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:35 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:29:35.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:35 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:29:35.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:35 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi086", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:29:35.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:35 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:29:35.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:35 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:29:35.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:35 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:29:35.678 INFO:tasks.cephadm:Distributing conf and client.admin keyring to all hosts + 0755 2024-04-21T08:29:35.678 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph orch client-keyring set client.admin '*' --mode 0755 2024-04-21T08:29:35.910 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:29:36.346 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:36 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/4230601215' entity='client.admin' 2024-04-21T08:29:36.346 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:36 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:29:36.346 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:36 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:29:37.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.431+0000 7f5432747700 1 -- 172.21.15.86:0/1370138176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f542c103b90 msgr2=0x7f542c103fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:37.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.431+0000 7f5432747700 1 --2- 172.21.15.86:0/1370138176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f542c103b90 0x7f542c103fb0 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f541c009910 tx=0x7f541c009c20 comp rx=0 tx=0).stop 2024-04-21T08:29:37.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.431+0000 7f5432747700 1 -- 172.21.15.86:0/1370138176 shutdown_connections 2024-04-21T08:29:37.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.431+0000 7f5432747700 1 --2- 172.21.15.86:0/1370138176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f542c103b90 0x7f542c103fb0 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:37.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.431+0000 7f5432747700 1 -- 172.21.15.86:0/1370138176 >> 172.21.15.86:0/1370138176 conn(0x7f542c0fda70 msgr2=0x7f542c0ffe90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:37.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.431+0000 7f5432747700 1 -- 172.21.15.86:0/1370138176 shutdown_connections 2024-04-21T08:29:37.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.431+0000 7f5432747700 1 -- 172.21.15.86:0/1370138176 wait complete. 2024-04-21T08:29:37.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.432+0000 7f5432747700 1 Processor -- start 2024-04-21T08:29:37.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.432+0000 7f5432747700 1 -- start start 2024-04-21T08:29:37.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.432+0000 7f5432747700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f542c103b90 0x7f542c076700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:37.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.432+0000 7f5432747700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f542c108500 con 0x7f542c103b90 2024-04-21T08:29:37.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.432+0000 7f542bfff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f542c103b90 0x7f542c076700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:37.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.432+0000 7f542bfff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f542c103b90 0x7f542c076700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37020/0 (socket says 172.21.15.86:37020) 2024-04-21T08:29:37.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.432+0000 7f542bfff700 1 -- 172.21.15.86:0/751019347 learned_addr learned my addr 172.21.15.86:0/751019347 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:37.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.432+0000 7f542bfff700 1 -- 172.21.15.86:0/751019347 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f542c076c40 con 0x7f542c103b90 2024-04-21T08:29:37.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.432+0000 7f542bfff700 1 --2- 172.21.15.86:0/751019347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f542c103b90 0x7f542c076700 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f541c005800 tx=0x7f541c009f60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:37.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.433+0000 7f54297fa700 1 -- 172.21.15.86:0/751019347 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f541c027070 con 0x7f542c103b90 2024-04-21T08:29:37.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.433+0000 7f5432747700 1 -- 172.21.15.86:0/751019347 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f542c074e30 con 0x7f542c103b90 2024-04-21T08:29:37.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.433+0000 7f54297fa700 1 -- 172.21.15.86:0/751019347 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f541c00be10 con 0x7f542c103b90 2024-04-21T08:29:37.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.433+0000 7f5432747700 1 -- 172.21.15.86:0/751019347 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f542c075370 con 0x7f542c103b90 2024-04-21T08:29:37.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.433+0000 7f54297fa700 1 -- 172.21.15.86:0/751019347 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f541c00f460 con 0x7f542c103b90 2024-04-21T08:29:37.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.433+0000 7f54297fa700 1 -- 172.21.15.86:0/751019347 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f541c02a070 con 0x7f542c103b90 2024-04-21T08:29:37.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.433+0000 7f5432747700 1 -- 172.21.15.86:0/751019347 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f542c108c80 con 0x7f542c103b90 2024-04-21T08:29:37.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.434+0000 7f54297fa700 1 --2- 172.21.15.86:0/751019347 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f54140396b0 0x7f541403bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:37.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.434+0000 7f54297fa700 1 -- 172.21.15.86:0/751019347 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f541c01e030 con 0x7f542c103b90 2024-04-21T08:29:37.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.434+0000 7f542b7fe700 1 --2- 172.21.15.86:0/751019347 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f54140396b0 0x7f541403bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:37.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.434+0000 7f542b7fe700 1 --2- 172.21.15.86:0/751019347 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f54140396b0 0x7f541403bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5420006fd0 tx=0x7f5420006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:37.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.439+0000 7f54297fa700 1 -- 172.21.15.86:0/751019347 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f541c013070 con 0x7f542c103b90 2024-04-21T08:29:37.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.639+0000 7f5432747700 1 -- 172.21.15.86:0/751019347 --> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] -- mgr_command(tid 0: {"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}) -- 0x7f542c190e70 con 0x7f54140396b0 2024-04-21T08:29:37.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.643+0000 7f54297fa700 1 -- 172.21.15.86:0/751019347 <== mgr.14152 v2:172.21.15.86:6800/1364156965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f542c190e70 con 0x7f54140396b0 2024-04-21T08:29:37.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.645+0000 7f5432747700 1 -- 172.21.15.86:0/751019347 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f54140396b0 msgr2=0x7f541403bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:37.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.645+0000 7f5432747700 1 --2- 172.21.15.86:0/751019347 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f54140396b0 0x7f541403bb70 secure :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0x7f5420006fd0 tx=0x7f5420006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:37.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.645+0000 7f5432747700 1 -- 172.21.15.86:0/751019347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f542c103b90 msgr2=0x7f542c076700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:37.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.645+0000 7f5432747700 1 --2- 172.21.15.86:0/751019347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f542c103b90 0x7f542c076700 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f541c005800 tx=0x7f541c009f60 comp rx=0 tx=0).stop 2024-04-21T08:29:37.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.645+0000 7f5432747700 1 -- 172.21.15.86:0/751019347 shutdown_connections 2024-04-21T08:29:37.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.645+0000 7f5432747700 1 --2- 172.21.15.86:0/751019347 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f54140396b0 0x7f541403bb70 unknown :-1 s=CLOSED pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:37.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.645+0000 7f5432747700 1 --2- 172.21.15.86:0/751019347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f542c103b90 0x7f542c076700 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:37.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.645+0000 7f5432747700 1 -- 172.21.15.86:0/751019347 >> 172.21.15.86:0/751019347 conn(0x7f542c0fda70 msgr2=0x7f542c0fe590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:37.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.645+0000 7f5432747700 1 -- 172.21.15.86:0/751019347 shutdown_connections 2024-04-21T08:29:37.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:37.645+0000 7f5432747700 1 -- 172.21.15.86:0/751019347 wait complete. 2024-04-21T08:29:38.404 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi088 2024-04-21T08:29:38.404 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-21T08:29:38.404 DEBUG:teuthology.orchestra.run.smithi088:> dd of=/etc/ceph/ceph.conf 2024-04-21T08:29:38.423 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-21T08:29:38.423 DEBUG:teuthology.orchestra.run.smithi088:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:29:38.481 INFO:tasks.cephadm:Adding host smithi088 to orchestrator... 2024-04-21T08:29:38.481 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph orch host add smithi088 2024-04-21T08:29:38.716 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:29:38.916 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:38 smithi086 ceph-mon[27939]: from='client.14176 -' entity='client.admin' cmd=[{"prefix": "orch client-keyring set", "entity": "client.admin", "placement": "*", "mode": "0755", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:29:38.916 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:38 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:29:38.916 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:38 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:29:38.916 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:38 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:29:38.916 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:38 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:29:38.917 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:38 smithi086 ceph-mon[27939]: Updating smithi086:/etc/ceph/ceph.conf 2024-04-21T08:29:38.917 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:38 smithi086 ceph-mon[27939]: Updating smithi086:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:29:38.917 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:38 smithi086 ceph-mon[27939]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:29:38.917 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:38 smithi086 ceph-mon[27939]: Updating smithi086:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:29:38.917 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:38 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:29:38.917 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:38 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:29:38.917 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:38 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:29:40.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:40.195+0000 7f46def5f700 1 -- 172.21.15.86:0/3542000384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46d81084e0 msgr2=0x7f46d81088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:40.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:40.195+0000 7f46def5f700 1 --2- 172.21.15.86:0/3542000384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46d81084e0 0x7f46d81088c0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7f46cc009910 tx=0x7f46cc009c20 comp rx=0 tx=0).stop 2024-04-21T08:29:40.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:40.195+0000 7f46def5f700 1 -- 172.21.15.86:0/3542000384 shutdown_connections 2024-04-21T08:29:40.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:40.195+0000 7f46def5f700 1 --2- 172.21.15.86:0/3542000384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46d81084e0 0x7f46d81088c0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:40.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:40.195+0000 7f46def5f700 1 -- 172.21.15.86:0/3542000384 >> 172.21.15.86:0/3542000384 conn(0x7f46d80fdf80 msgr2=0x7f46d81003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:40.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:40.195+0000 7f46def5f700 1 -- 172.21.15.86:0/3542000384 shutdown_connections 2024-04-21T08:29:40.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:40.195+0000 7f46def5f700 1 -- 172.21.15.86:0/3542000384 wait complete. 2024-04-21T08:29:40.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:40.195+0000 7f46def5f700 1 Processor -- start 2024-04-21T08:29:40.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:40.196+0000 7f46def5f700 1 -- start start 2024-04-21T08:29:40.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:40.196+0000 7f46def5f700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46d81084e0 0x7f46d8197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:40.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:40.196+0000 7f46def5f700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f46d810c940 con 0x7f46d81084e0 2024-04-21T08:29:40.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:40.196+0000 7f46dccfb700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46d81084e0 0x7f46d8197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:40.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:40.196+0000 7f46dccfb700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46d81084e0 0x7f46d8197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60206/0 (socket says 172.21.15.86:60206) 2024-04-21T08:29:40.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:40.196+0000 7f46dccfb700 1 -- 172.21.15.86:0/1838333562 learned_addr learned my addr 172.21.15.86:0/1838333562 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:40.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:40.196+0000 7f46dccfb700 1 -- 172.21.15.86:0/1838333562 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f46d81983f0 con 0x7f46d81084e0 2024-04-21T08:29:40.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:40.196+0000 7f46dccfb700 1 --2- 172.21.15.86:0/1838333562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46d81084e0 0x7f46d8197eb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f46cc006010 tx=0x7f46cc005780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:40.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:40.196+0000 7f46d5ffb700 1 -- 172.21.15.86:0/1838333562 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46cc026070 con 0x7f46d81084e0 2024-04-21T08:29:40.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:40.196+0000 7f46d5ffb700 1 -- 172.21.15.86:0/1838333562 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f46cc00bba0 con 0x7f46d81084e0 2024-04-21T08:29:40.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:40.196+0000 7f46d5ffb700 1 -- 172.21.15.86:0/1838333562 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f46cc003910 con 0x7f46d81084e0 2024-04-21T08:29:40.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:40.196+0000 7f46def5f700 1 -- 172.21.15.86:0/1838333562 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46d81986e0 con 0x7f46d81084e0 2024-04-21T08:29:40.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:40.197+0000 7f46def5f700 1 -- 172.21.15.86:0/1838333562 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46d819c780 con 0x7f46d81084e0 2024-04-21T08:29:40.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:40.197+0000 7f46d5ffb700 1 -- 172.21.15.86:0/1838333562 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f46cc029070 con 0x7f46d81084e0 2024-04-21T08:29:40.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:40.197+0000 7f46def5f700 1 -- 172.21.15.86:0/1838333562 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f46d8103b90 con 0x7f46d81084e0 2024-04-21T08:29:40.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:40.198+0000 7f46d5ffb700 1 --2- 172.21.15.86:0/1838333562 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f46c8039650 0x7f46c803bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:40.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:40.198+0000 7f46d5ffb700 1 -- 172.21.15.86:0/1838333562 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f46cc01e070 con 0x7f46d81084e0 2024-04-21T08:29:40.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:40.198+0000 7f46d7fff700 1 --2- 172.21.15.86:0/1838333562 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f46c8039650 0x7f46c803bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:40.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:40.198+0000 7f46d7fff700 1 --2- 172.21.15.86:0/1838333562 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f46c8039650 0x7f46c803bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f46c4006fd0 tx=0x7f46c4006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:40.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:40.203+0000 7f46d5ffb700 1 -- 172.21.15.86:0/1838333562 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f46cc013070 con 0x7f46d81084e0 2024-04-21T08:29:40.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:40.407+0000 7f46def5f700 1 -- 172.21.15.86:0/1838333562 --> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi088", "target": ["mon-mgr", ""]}) -- 0x7f46d8199210 con 0x7f46c8039650 2024-04-21T08:29:40.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:40 smithi086 ceph-mon[27939]: from='client.14178 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi088", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:29:41.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:41 smithi086 ceph-mon[27939]: Deploying cephadm binary to smithi088 2024-04-21T08:29:44.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:44.503+0000 7f46d5ffb700 1 -- 172.21.15.86:0/1838333562 <== mgr.14152 v2:172.21.15.86:6800/1364156965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+48 (secure 0 0 0) 0x7f46d8199210 con 0x7f46c8039650 2024-04-21T08:29:44.503 INFO:teuthology.orchestra.run.smithi086.stdout:Added host 'smithi088' with addr '172.21.15.88' 2024-04-21T08:29:44.503 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:44 smithi086 ceph-mon[27939]: pgmap v3: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:29:44.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:44.506+0000 7f46def5f700 1 -- 172.21.15.86:0/1838333562 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f46c8039650 msgr2=0x7f46c803bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:44.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:44.506+0000 7f46def5f700 1 --2- 172.21.15.86:0/1838333562 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f46c8039650 0x7f46c803bb10 secure :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0x7f46c4006fd0 tx=0x7f46c4006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:44.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:44.506+0000 7f46def5f700 1 -- 172.21.15.86:0/1838333562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46d81084e0 msgr2=0x7f46d8197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:44.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:44.506+0000 7f46def5f700 1 --2- 172.21.15.86:0/1838333562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46d81084e0 0x7f46d8197eb0 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7f46cc006010 tx=0x7f46cc005780 comp rx=0 tx=0).stop 2024-04-21T08:29:44.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:44.506+0000 7f46def5f700 1 -- 172.21.15.86:0/1838333562 shutdown_connections 2024-04-21T08:29:44.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:44.506+0000 7f46def5f700 1 --2- 172.21.15.86:0/1838333562 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f46c8039650 0x7f46c803bb10 unknown :-1 s=CLOSED pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:44.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:44.506+0000 7f46def5f700 1 --2- 172.21.15.86:0/1838333562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f46d81084e0 0x7f46d8197eb0 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:44.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:44.506+0000 7f46def5f700 1 -- 172.21.15.86:0/1838333562 >> 172.21.15.86:0/1838333562 conn(0x7f46d80fdf80 msgr2=0x7f46d810a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:44.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:44.506+0000 7f46def5f700 1 -- 172.21.15.86:0/1838333562 shutdown_connections 2024-04-21T08:29:44.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:44.506+0000 7f46def5f700 1 -- 172.21.15.86:0/1838333562 wait complete. 2024-04-21T08:29:45.145 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-21T08:29:45.380 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:29:45.783 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:45 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:29:45.783 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:45 smithi086 ceph-mon[27939]: Added host smithi088 2024-04-21T08:29:45.783 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:45 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:29:45.783 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:45 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:29:45.783 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:45 smithi086 ceph-mon[27939]: pgmap v4: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:29:46.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:46.862+0000 7f142c269700 1 -- 172.21.15.86:0/2035989490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14241084e0 msgr2=0x7f14241088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:46.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:46.862+0000 7f142c269700 1 --2- 172.21.15.86:0/2035989490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14241084e0 0x7f14241088c0 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f1420009910 tx=0x7f1420009c20 comp rx=0 tx=0).stop 2024-04-21T08:29:46.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:46.862+0000 7f142c269700 1 -- 172.21.15.86:0/2035989490 shutdown_connections 2024-04-21T08:29:46.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:46.862+0000 7f142c269700 1 --2- 172.21.15.86:0/2035989490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14241084e0 0x7f14241088c0 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:46.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:46.862+0000 7f142c269700 1 -- 172.21.15.86:0/2035989490 >> 172.21.15.86:0/2035989490 conn(0x7f1424076ea0 msgr2=0x7f14240772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:46.862 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:46 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:29:46.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:46.862+0000 7f142c269700 1 -- 172.21.15.86:0/2035989490 shutdown_connections 2024-04-21T08:29:46.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:46.862+0000 7f142c269700 1 -- 172.21.15.86:0/2035989490 wait complete. 2024-04-21T08:29:46.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:46.863+0000 7f142c269700 1 Processor -- start 2024-04-21T08:29:46.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:46.863+0000 7f142c269700 1 -- start start 2024-04-21T08:29:46.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:46.863+0000 7f142c269700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14241084e0 0x7f1424101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:46.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:46.863+0000 7f142c269700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f142410c920 con 0x7f14241084e0 2024-04-21T08:29:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:46.863+0000 7f142a005700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14241084e0 0x7f1424101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:46.863+0000 7f142a005700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14241084e0 0x7f1424101460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60220/0 (socket says 172.21.15.86:60220) 2024-04-21T08:29:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:46.863+0000 7f142a005700 1 -- 172.21.15.86:0/554352880 learned_addr learned my addr 172.21.15.86:0/554352880 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:46.864+0000 7f142a005700 1 -- 172.21.15.86:0/554352880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1424103350 con 0x7f14241084e0 2024-04-21T08:29:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:46.864+0000 7f142a005700 1 --2- 172.21.15.86:0/554352880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14241084e0 0x7f1424101460 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f14200056d0 tx=0x7f14200057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:46.864+0000 7f141b7fe700 1 -- 172.21.15.86:0/554352880 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1420026070 con 0x7f14241084e0 2024-04-21T08:29:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:46.864+0000 7f141b7fe700 1 -- 172.21.15.86:0/554352880 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f142000bc00 con 0x7f14241084e0 2024-04-21T08:29:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:46.864+0000 7f142c269700 1 -- 172.21.15.86:0/554352880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1424101a00 con 0x7f14241084e0 2024-04-21T08:29:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:46.864+0000 7f141b7fe700 1 -- 172.21.15.86:0/554352880 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1420003920 con 0x7f14241084e0 2024-04-21T08:29:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:46.864+0000 7f142c269700 1 -- 172.21.15.86:0/554352880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1424101ec0 con 0x7f14241084e0 2024-04-21T08:29:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:46.865+0000 7f141b7fe700 1 -- 172.21.15.86:0/554352880 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f1420029070 con 0x7f14241084e0 2024-04-21T08:29:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:46.865+0000 7f141b7fe700 1 --2- 172.21.15.86:0/554352880 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f1410039650 0x7f141003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:46.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:46.865+0000 7f141b7fe700 1 -- 172.21.15.86:0/554352880 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f142001e070 con 0x7f14241084e0 2024-04-21T08:29:46.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:46.865+0000 7f142c269700 1 -- 172.21.15.86:0/554352880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1424103b90 con 0x7f14241084e0 2024-04-21T08:29:46.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:46.865+0000 7f1429804700 1 --2- 172.21.15.86:0/554352880 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f1410039650 0x7f141003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:46.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:46.866+0000 7f1429804700 1 --2- 172.21.15.86:0/554352880 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f1410039650 0x7f141003bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1414006fd0 tx=0x7f1414006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:46.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:46.870+0000 7f141b7fe700 1 -- 172.21.15.86:0/554352880 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1420030070 con 0x7f14241084e0 2024-04-21T08:29:47.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:47.074+0000 7f142c269700 1 -- 172.21.15.86:0/554352880 --> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f1424102a00 con 0x7f1410039650 2024-04-21T08:29:47.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:47.075+0000 7f141b7fe700 1 -- 172.21.15.86:0/554352880 <== mgr.14152 v2:172.21.15.86:6800/1364156965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+159 (secure 0 0 0) 0x7f1424102a00 con 0x7f1410039650 2024-04-21T08:29:47.074 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:29:47.074 INFO:teuthology.orchestra.run.smithi086.stdout:[{"addr": "172.21.15.86", "hostname": "smithi086", "labels": [], "status": ""}, {"addr": "172.21.15.88", "hostname": "smithi088", "labels": [], "status": ""}] 2024-04-21T08:29:47.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:47.077+0000 7f142c269700 1 -- 172.21.15.86:0/554352880 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f1410039650 msgr2=0x7f141003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:47.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:47.077+0000 7f142c269700 1 --2- 172.21.15.86:0/554352880 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f1410039650 0x7f141003bb10 secure :-1 s=READY pgs=15 cs=0 l=1 rev1=1 crypto rx=0x7f1414006fd0 tx=0x7f1414006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:47.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:47.077+0000 7f142c269700 1 -- 172.21.15.86:0/554352880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14241084e0 msgr2=0x7f1424101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:47.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:47.077+0000 7f142c269700 1 --2- 172.21.15.86:0/554352880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14241084e0 0x7f1424101460 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f14200056d0 tx=0x7f14200057b0 comp rx=0 tx=0).stop 2024-04-21T08:29:47.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:47.077+0000 7f142c269700 1 -- 172.21.15.86:0/554352880 shutdown_connections 2024-04-21T08:29:47.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:47.077+0000 7f142c269700 1 --2- 172.21.15.86:0/554352880 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f1410039650 0x7f141003bb10 unknown :-1 s=CLOSED pgs=15 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:47.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:47.077+0000 7f142c269700 1 --2- 172.21.15.86:0/554352880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14241084e0 0x7f1424101460 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:47.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:47.077+0000 7f142c269700 1 -- 172.21.15.86:0/554352880 >> 172.21.15.86:0/554352880 conn(0x7f1424076ea0 msgr2=0x7f14240fe900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:47.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:47.077+0000 7f142c269700 1 -- 172.21.15.86:0/554352880 shutdown_connections 2024-04-21T08:29:47.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:47.077+0000 7f142c269700 1 -- 172.21.15.86:0/554352880 wait complete. 2024-04-21T08:29:47.669 INFO:tasks.cephadm:Writing (initial) conf and keyring to smithi179 2024-04-21T08:29:47.669 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-21T08:29:47.669 DEBUG:teuthology.orchestra.run.smithi179:> dd of=/etc/ceph/ceph.conf 2024-04-21T08:29:47.689 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-21T08:29:47.689 DEBUG:teuthology.orchestra.run.smithi179:> dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:29:47.746 INFO:tasks.cephadm:Adding host smithi179 to orchestrator... 2024-04-21T08:29:47.746 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph orch host add smithi179 2024-04-21T08:29:47.974 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:29:48.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:48 smithi086 ceph-mon[27939]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:29:48.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:48 smithi086 ceph-mon[27939]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:29:48.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:48 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:29:49.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:49.453+0000 7f92aaff7700 1 -- 172.21.15.86:0/1669265019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92a41084e0 msgr2=0x7f92a41088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:49.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:49.453+0000 7f92aaff7700 1 --2- 172.21.15.86:0/1669265019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92a41084e0 0x7f92a41088c0 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7f9298009910 tx=0x7f9298009c20 comp rx=0 tx=0).stop 2024-04-21T08:29:49.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:49.453+0000 7f92aaff7700 1 -- 172.21.15.86:0/1669265019 shutdown_connections 2024-04-21T08:29:49.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:49.453+0000 7f92aaff7700 1 --2- 172.21.15.86:0/1669265019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92a41084e0 0x7f92a41088c0 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:49.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:49.453+0000 7f92aaff7700 1 -- 172.21.15.86:0/1669265019 >> 172.21.15.86:0/1669265019 conn(0x7f92a40fdf80 msgr2=0x7f92a41003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:49.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:49.453+0000 7f92aaff7700 1 -- 172.21.15.86:0/1669265019 shutdown_connections 2024-04-21T08:29:49.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:49.453+0000 7f92aaff7700 1 -- 172.21.15.86:0/1669265019 wait complete. 2024-04-21T08:29:49.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:49.454+0000 7f92aaff7700 1 Processor -- start 2024-04-21T08:29:49.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:49.454+0000 7f92aaff7700 1 -- start start 2024-04-21T08:29:49.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:49.454+0000 7f92aaff7700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92a41084e0 0x7f92a4197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:49.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:49.454+0000 7f92aaff7700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f92a410c940 con 0x7f92a41084e0 2024-04-21T08:29:49.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:49.454+0000 7f92a8d93700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92a41084e0 0x7f92a4197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:49.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:49.454+0000 7f92a8d93700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92a41084e0 0x7f92a4197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60240/0 (socket says 172.21.15.86:60240) 2024-04-21T08:29:49.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:49.454+0000 7f92a8d93700 1 -- 172.21.15.86:0/1117759370 learned_addr learned my addr 172.21.15.86:0/1117759370 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:49.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:49.454+0000 7f92a8d93700 1 -- 172.21.15.86:0/1117759370 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f92a41983f0 con 0x7f92a41084e0 2024-04-21T08:29:49.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:49.454+0000 7f92a8d93700 1 --2- 172.21.15.86:0/1117759370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92a41084e0 0x7f92a4197eb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f92980056d0 tx=0x7f92980057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:49.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:49.454+0000 7f92a1ffb700 1 -- 172.21.15.86:0/1117759370 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9298026070 con 0x7f92a41084e0 2024-04-21T08:29:49.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:49.454+0000 7f92a1ffb700 1 -- 172.21.15.86:0/1117759370 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f929800bba0 con 0x7f92a41084e0 2024-04-21T08:29:49.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:49.455+0000 7f92a1ffb700 1 -- 172.21.15.86:0/1117759370 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f9298003920 con 0x7f92a41084e0 2024-04-21T08:29:49.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:49.455+0000 7f92aaff7700 1 -- 172.21.15.86:0/1117759370 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f92a41986e0 con 0x7f92a41084e0 2024-04-21T08:29:49.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:49.455+0000 7f92aaff7700 1 -- 172.21.15.86:0/1117759370 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92a419c780 con 0x7f92a41084e0 2024-04-21T08:29:49.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:49.456+0000 7f92a1ffb700 1 -- 172.21.15.86:0/1117759370 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f9298029070 con 0x7f92a41084e0 2024-04-21T08:29:49.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:49.456+0000 7f92a1ffb700 1 --2- 172.21.15.86:0/1117759370 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f9294039650 0x7f929403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:49.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:49.456+0000 7f92aaff7700 1 -- 172.21.15.86:0/1117759370 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f92a4103b90 con 0x7f92a41084e0 2024-04-21T08:29:49.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:49.456+0000 7f92a1ffb700 1 -- 172.21.15.86:0/1117759370 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7f929801e070 con 0x7f92a41084e0 2024-04-21T08:29:49.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:49.456+0000 7f92a3fff700 1 --2- 172.21.15.86:0/1117759370 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f9294039650 0x7f929403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:49.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:49.456+0000 7f92a3fff700 1 --2- 172.21.15.86:0/1117759370 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f9294039650 0x7f929403bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9290006fd0 tx=0x7f9290006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:49.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:49.461+0000 7f92a1ffb700 1 -- 172.21.15.86:0/1117759370 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f929804a020 con 0x7f92a41084e0 2024-04-21T08:29:49.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:49.662+0000 7f92aaff7700 1 -- 172.21.15.86:0/1117759370 --> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] -- mgr_command(tid 0: {"prefix": "orch host add", "hostname": "smithi179", "target": ["mon-mgr", ""]}) -- 0x7f92a4199210 con 0x7f9294039650 2024-04-21T08:29:50.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:50 smithi086 ceph-mon[27939]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:29:51.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:51 smithi086 ceph-mon[27939]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi179", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:29:51.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:51 smithi086 ceph-mon[27939]: Deploying cephadm binary to smithi179 2024-04-21T08:29:51.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:51 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:29:51.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:51 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:29:51.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:51 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:29:51.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:51 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:29:51.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:51 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi088", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:29:51.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:51 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:29:51.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:51 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:29:51.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:51 smithi086 ceph-mon[27939]: Updating smithi088:/etc/ceph/ceph.conf 2024-04-21T08:29:52.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:52 smithi086 ceph-mon[27939]: Updating smithi088:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:29:52.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:52 smithi086 ceph-mon[27939]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:29:53.748 INFO:teuthology.orchestra.run.smithi086.stdout:Added host 'smithi179' with addr '172.21.15.179' 2024-04-21T08:29:53.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:53.749+0000 7f92a1ffb700 1 -- 172.21.15.86:0/1117759370 <== mgr.14152 v2:172.21.15.86:6800/1364156965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+49 (secure 0 0 0) 0x7f92a4199210 con 0x7f9294039650 2024-04-21T08:29:53.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:53.751+0000 7f92aaff7700 1 -- 172.21.15.86:0/1117759370 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f9294039650 msgr2=0x7f929403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:53.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:53.751+0000 7f92aaff7700 1 --2- 172.21.15.86:0/1117759370 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f9294039650 0x7f929403bb10 secure :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0x7f9290006fd0 tx=0x7f9290006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:53.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:53.751+0000 7f92aaff7700 1 -- 172.21.15.86:0/1117759370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92a41084e0 msgr2=0x7f92a4197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:53.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:53.751+0000 7f92aaff7700 1 --2- 172.21.15.86:0/1117759370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92a41084e0 0x7f92a4197eb0 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f92980056d0 tx=0x7f92980057b0 comp rx=0 tx=0).stop 2024-04-21T08:29:53.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:53.752+0000 7f92aaff7700 1 -- 172.21.15.86:0/1117759370 shutdown_connections 2024-04-21T08:29:53.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:53.752+0000 7f92aaff7700 1 --2- 172.21.15.86:0/1117759370 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f9294039650 0x7f929403bb10 unknown :-1 s=CLOSED pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:53.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:53.752+0000 7f92aaff7700 1 --2- 172.21.15.86:0/1117759370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f92a41084e0 0x7f92a4197eb0 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:53.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:53.752+0000 7f92aaff7700 1 -- 172.21.15.86:0/1117759370 >> 172.21.15.86:0/1117759370 conn(0x7f92a40fdf80 msgr2=0x7f92a410a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:53.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:53.752+0000 7f92aaff7700 1 -- 172.21.15.86:0/1117759370 shutdown_connections 2024-04-21T08:29:53.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:53.752+0000 7f92aaff7700 1 -- 172.21.15.86:0/1117759370 wait complete. 2024-04-21T08:29:54.359 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph orch host ls --format=json 2024-04-21T08:29:54.589 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:29:54.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:54 smithi086 ceph-mon[27939]: Updating smithi088:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:29:54.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:54 smithi086 ceph-mon[27939]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:29:54.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:54 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:29:54.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:54 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:29:54.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:54 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:29:54.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:54 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:29:54.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:54 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:29:54.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:54 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:29:55.461 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:55 smithi086 ceph-mon[27939]: Updating smithi088:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:29:55.461 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:55 smithi086 ceph-mon[27939]: Added host smithi179 2024-04-21T08:29:55.461 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:55 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:29:55.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:55.933+0000 7efd284f6700 1 -- 172.21.15.86:0/1835481764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd201084e0 msgr2=0x7efd201088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:55.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:55.933+0000 7efd284f6700 1 --2- 172.21.15.86:0/1835481764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd201084e0 0x7efd201088c0 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7efd1c009910 tx=0x7efd1c009c20 comp rx=0 tx=0).stop 2024-04-21T08:29:55.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:55.933+0000 7efd284f6700 1 -- 172.21.15.86:0/1835481764 shutdown_connections 2024-04-21T08:29:55.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:55.933+0000 7efd284f6700 1 --2- 172.21.15.86:0/1835481764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd201084e0 0x7efd201088c0 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:55.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:55.933+0000 7efd284f6700 1 -- 172.21.15.86:0/1835481764 >> 172.21.15.86:0/1835481764 conn(0x7efd200fdf80 msgr2=0x7efd201003a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:55.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:55.933+0000 7efd284f6700 1 -- 172.21.15.86:0/1835481764 shutdown_connections 2024-04-21T08:29:55.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:55.933+0000 7efd284f6700 1 -- 172.21.15.86:0/1835481764 wait complete. 2024-04-21T08:29:55.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:55.934+0000 7efd284f6700 1 Processor -- start 2024-04-21T08:29:55.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:55.934+0000 7efd284f6700 1 -- start start 2024-04-21T08:29:55.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:55.934+0000 7efd284f6700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd201084e0 0x7efd20197eb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:55.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:55.934+0000 7efd284f6700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efd2010c940 con 0x7efd201084e0 2024-04-21T08:29:55.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:55.934+0000 7efd26292700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd201084e0 0x7efd20197eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:55.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:55.934+0000 7efd26292700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd201084e0 0x7efd20197eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55718/0 (socket says 172.21.15.86:55718) 2024-04-21T08:29:55.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:55.934+0000 7efd26292700 1 -- 172.21.15.86:0/2869570969 learned_addr learned my addr 172.21.15.86:0/2869570969 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:55.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:55.934+0000 7efd26292700 1 -- 172.21.15.86:0/2869570969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd201983f0 con 0x7efd201084e0 2024-04-21T08:29:55.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:55.934+0000 7efd26292700 1 --2- 172.21.15.86:0/2869570969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd201084e0 0x7efd20197eb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efd1c0056d0 tx=0x7efd1c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:55.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:55.934+0000 7efd177fe700 1 -- 172.21.15.86:0/2869570969 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd1c026070 con 0x7efd201084e0 2024-04-21T08:29:55.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:55.935+0000 7efd284f6700 1 -- 172.21.15.86:0/2869570969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd201986e0 con 0x7efd201084e0 2024-04-21T08:29:55.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:55.935+0000 7efd177fe700 1 -- 172.21.15.86:0/2869570969 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd1c00bba0 con 0x7efd201084e0 2024-04-21T08:29:55.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:55.935+0000 7efd177fe700 1 -- 172.21.15.86:0/2869570969 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7efd1c003920 con 0x7efd201084e0 2024-04-21T08:29:55.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:55.935+0000 7efd284f6700 1 -- 172.21.15.86:0/2869570969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd2019c780 con 0x7efd201084e0 2024-04-21T08:29:55.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:55.935+0000 7efd177fe700 1 -- 172.21.15.86:0/2869570969 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7efd1c029070 con 0x7efd201084e0 2024-04-21T08:29:55.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:55.935+0000 7efd284f6700 1 -- 172.21.15.86:0/2869570969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd20103b90 con 0x7efd201084e0 2024-04-21T08:29:55.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:55.936+0000 7efd177fe700 1 --2- 172.21.15.86:0/2869570969 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7efd0c0396b0 0x7efd0c03bb70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:55.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:55.936+0000 7efd177fe700 1 -- 172.21.15.86:0/2869570969 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7efd1c01e070 con 0x7efd201084e0 2024-04-21T08:29:55.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:55.936+0000 7efd25a91700 1 --2- 172.21.15.86:0/2869570969 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7efd0c0396b0 0x7efd0c03bb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:55.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:55.936+0000 7efd25a91700 1 --2- 172.21.15.86:0/2869570969 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7efd0c0396b0 0x7efd0c03bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7efd10006fd0 tx=0x7efd10006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:55.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:55.941+0000 7efd177fe700 1 -- 172.21.15.86:0/2869570969 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efd1c013070 con 0x7efd201084e0 2024-04-21T08:29:56.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:56.147+0000 7efd284f6700 1 -- 172.21.15.86:0/2869570969 --> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7efd20199210 con 0x7efd0c0396b0 2024-04-21T08:29:56.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:56.148+0000 7efd177fe700 1 -- 172.21.15.86:0/2869570969 <== mgr.14152 v2:172.21.15.86:6800/1364156965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+239 (secure 0 0 0) 0x7efd20199210 con 0x7efd0c0396b0 2024-04-21T08:29:56.147 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:29:56.147 INFO:teuthology.orchestra.run.smithi086.stdout:[{"addr": "172.21.15.86", "hostname": "smithi086", "labels": [], "status": ""}, {"addr": "172.21.15.88", "hostname": "smithi088", "labels": [], "status": ""}, {"addr": "172.21.15.179", "hostname": "smithi179", "labels": [], "status": ""}] 2024-04-21T08:29:56.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:56.150+0000 7efd284f6700 1 -- 172.21.15.86:0/2869570969 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7efd0c0396b0 msgr2=0x7efd0c03bb70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:56.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:56.150+0000 7efd284f6700 1 --2- 172.21.15.86:0/2869570969 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7efd0c0396b0 0x7efd0c03bb70 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7efd10006fd0 tx=0x7efd10006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:56.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:56.150+0000 7efd284f6700 1 -- 172.21.15.86:0/2869570969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd201084e0 msgr2=0x7efd20197eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:56.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:56.150+0000 7efd284f6700 1 --2- 172.21.15.86:0/2869570969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd201084e0 0x7efd20197eb0 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7efd1c0056d0 tx=0x7efd1c0057b0 comp rx=0 tx=0).stop 2024-04-21T08:29:56.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:56.150+0000 7efd284f6700 1 -- 172.21.15.86:0/2869570969 shutdown_connections 2024-04-21T08:29:56.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:56.150+0000 7efd284f6700 1 --2- 172.21.15.86:0/2869570969 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7efd0c0396b0 0x7efd0c03bb70 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:56.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:56.150+0000 7efd284f6700 1 --2- 172.21.15.86:0/2869570969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd201084e0 0x7efd20197eb0 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:56.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:56.150+0000 7efd284f6700 1 -- 172.21.15.86:0/2869570969 >> 172.21.15.86:0/2869570969 conn(0x7efd200fdf80 msgr2=0x7efd2010a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:56.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:56.151+0000 7efd284f6700 1 -- 172.21.15.86:0/2869570969 shutdown_connections 2024-04-21T08:29:56.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:56.151+0000 7efd284f6700 1 -- 172.21.15.86:0/2869570969 wait complete. 2024-04-21T08:29:56.734 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:56 smithi086 ceph-mon[27939]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:29:56.734 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:56 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:29:56.760 INFO:tasks.cephadm:Setting crush tunables to default 2024-04-21T08:29:56.761 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph osd crush tunables default 2024-04-21T08:29:56.991 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:29:57.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:57 smithi086 ceph-mon[27939]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:29:58.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:58.438+0000 7fcb0d62c700 1 -- 172.21.15.86:0/3885838121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb081015b0 msgr2=0x7fcb08101990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:58.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:58.438+0000 7fcb0d62c700 1 --2- 172.21.15.86:0/3885838121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb081015b0 0x7fcb08101990 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7fcafc009910 tx=0x7fcafc009c20 comp rx=0 tx=0).stop 2024-04-21T08:29:58.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:58.438+0000 7fcb0d62c700 1 -- 172.21.15.86:0/3885838121 shutdown_connections 2024-04-21T08:29:58.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:58.438+0000 7fcb0d62c700 1 --2- 172.21.15.86:0/3885838121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb081015b0 0x7fcb08101990 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:58.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:58.438+0000 7fcb0d62c700 1 -- 172.21.15.86:0/3885838121 >> 172.21.15.86:0/3885838121 conn(0x7fcb080fce40 msgr2=0x7fcb080ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:58.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:58.438+0000 7fcb0d62c700 1 -- 172.21.15.86:0/3885838121 shutdown_connections 2024-04-21T08:29:58.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:58.438+0000 7fcb0d62c700 1 -- 172.21.15.86:0/3885838121 wait complete. 2024-04-21T08:29:58.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:58.439+0000 7fcb0d62c700 1 Processor -- start 2024-04-21T08:29:58.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:58.439+0000 7fcb0d62c700 1 -- start start 2024-04-21T08:29:58.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:58.439+0000 7fcb0d62c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb081015b0 0x7fcb08198d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:58.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:58.439+0000 7fcb0d62c700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcb0810b830 con 0x7fcb081015b0 2024-04-21T08:29:58.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:58.439+0000 7fcb06ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb081015b0 0x7fcb08198d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:58.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:58.439+0000 7fcb06ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb081015b0 0x7fcb08198d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55742/0 (socket says 172.21.15.86:55742) 2024-04-21T08:29:58.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:58.439+0000 7fcb06ffd700 1 -- 172.21.15.86:0/3489651478 learned_addr learned my addr 172.21.15.86:0/3489651478 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:29:58.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:58.440+0000 7fcb06ffd700 1 -- 172.21.15.86:0/3489651478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcb08199240 con 0x7fcb081015b0 2024-04-21T08:29:58.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:58.440+0000 7fcb06ffd700 1 --2- 172.21.15.86:0/3489651478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb081015b0 0x7fcb08198d00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcafc0056d0 tx=0x7fcafc0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:58.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:58.440+0000 7fcaeffff700 1 -- 172.21.15.86:0/3489651478 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcafc026070 con 0x7fcb081015b0 2024-04-21T08:29:58.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:58.440+0000 7fcaeffff700 1 -- 172.21.15.86:0/3489651478 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcafc00bba0 con 0x7fcb081015b0 2024-04-21T08:29:58.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:58.440+0000 7fcb0d62c700 1 -- 172.21.15.86:0/3489651478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcb08199530 con 0x7fcb081015b0 2024-04-21T08:29:58.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:58.440+0000 7fcaeffff700 1 -- 172.21.15.86:0/3489651478 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7fcafc003920 con 0x7fcb081015b0 2024-04-21T08:29:58.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:58.440+0000 7fcb0d62c700 1 -- 172.21.15.86:0/3489651478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcb081999f0 con 0x7fcb081015b0 2024-04-21T08:29:58.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:58.441+0000 7fcaeffff700 1 -- 172.21.15.86:0/3489651478 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fcafc029070 con 0x7fcb081015b0 2024-04-21T08:29:58.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:58.441+0000 7fcb0d62c700 1 -- 172.21.15.86:0/3489651478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcb08102a90 con 0x7fcb081015b0 2024-04-21T08:29:58.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:58.441+0000 7fcaeffff700 1 --2- 172.21.15.86:0/3489651478 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fcaf0039650 0x7fcaf003bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:29:58.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:58.441+0000 7fcaeffff700 1 -- 172.21.15.86:0/3489651478 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(3..3 src has 1..3) ==== 1155+0+0 (secure 0 0 0) 0x7fcafc01e070 con 0x7fcb081015b0 2024-04-21T08:29:58.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:58.441+0000 7fcb067fc700 1 --2- 172.21.15.86:0/3489651478 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fcaf0039650 0x7fcaf003bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:29:58.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:58.441+0000 7fcb067fc700 1 --2- 172.21.15.86:0/3489651478 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fcaf0039650 0x7fcaf003bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8006fd0 tx=0x7fcaf8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:29:58.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:58.446+0000 7fcaeffff700 1 -- 172.21.15.86:0/3489651478 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcafc013070 con 0x7fcb081015b0 2024-04-21T08:29:58.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:58 smithi086 ceph-mon[27939]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:29:58.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:58.647+0000 7fcb0d62c700 1 -- 172.21.15.86:0/3489651478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd crush tunables", "profile": "default"} v 0) -- 0x7fcb0806aea0 con 0x7fcb081015b0 2024-04-21T08:29:59.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:59.413+0000 7fcaeffff700 1 -- 172.21.15.86:0/3489651478 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd crush tunables", "profile": "default"}]=0 adjusted tunables profile to default v4) ==== 124+0+0 (secure 0 0 0) 0x7fcafc030090 con 0x7fcb081015b0 2024-04-21T08:29:59.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:59.416+0000 7fcb0d62c700 1 -- 172.21.15.86:0/3489651478 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fcaf0039650 msgr2=0x7fcaf003bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:59.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:59.416+0000 7fcb0d62c700 1 --2- 172.21.15.86:0/3489651478 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fcaf0039650 0x7fcaf003bb10 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fcaf8006fd0 tx=0x7fcaf8006e40 comp rx=0 tx=0).stop 2024-04-21T08:29:59.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:59.416+0000 7fcb0d62c700 1 -- 172.21.15.86:0/3489651478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb081015b0 msgr2=0x7fcb08198d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:29:59.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:59.416+0000 7fcb0d62c700 1 --2- 172.21.15.86:0/3489651478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb081015b0 0x7fcb08198d00 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7fcafc0056d0 tx=0x7fcafc0057b0 comp rx=0 tx=0).stop 2024-04-21T08:29:59.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:59.416+0000 7fcb0d62c700 1 -- 172.21.15.86:0/3489651478 shutdown_connections 2024-04-21T08:29:59.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:59.416+0000 7fcb0d62c700 1 --2- 172.21.15.86:0/3489651478 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fcaf0039650 0x7fcaf003bb10 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:59.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:59.416+0000 7fcb0d62c700 1 --2- 172.21.15.86:0/3489651478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcb081015b0 0x7fcb08198d00 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:29:59.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:59.416+0000 7fcb0d62c700 1 -- 172.21.15.86:0/3489651478 >> 172.21.15.86:0/3489651478 conn(0x7fcb080fce40 msgr2=0x7fcb08102f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:29:59.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:59.416+0000 7fcb0d62c700 1 -- 172.21.15.86:0/3489651478 shutdown_connections 2024-04-21T08:29:59.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:29:59.416+0000 7fcb0d62c700 1 -- 172.21.15.86:0/3489651478 wait complete. 2024-04-21T08:29:59.416 INFO:teuthology.orchestra.run.smithi086.stderr:adjusted tunables profile to default 2024-04-21T08:29:59.729 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:29:59 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3489651478' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-21T08:30:00.159 INFO:tasks.cephadm:Adding mon.a on smithi086 2024-04-21T08:30:00.159 INFO:tasks.cephadm:Adding mon.b on smithi088 2024-04-21T08:30:00.159 INFO:tasks.cephadm:Adding mon.c on smithi179 2024-04-21T08:30:00.160 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph orch apply mon '3;smithi086:172.21.15.86=a;smithi088:172.21.15.88=b;smithi179:172.21.15.179=c' 2024-04-21T08:30:00.410 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:30:00.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:00 smithi086 ceph-mon[27939]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:00.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:00 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3489651478' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-21T08:30:00.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:00 smithi086 ceph-mon[27939]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T08:30:00.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:00 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:00.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:00 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:00.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:00 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:00.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:00 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:00.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:00 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi179", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:30:00.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:00 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:00.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:00 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:01.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:01 smithi086 ceph-mon[27939]: Updating smithi179:/etc/ceph/ceph.conf 2024-04-21T08:30:01.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:01 smithi086 ceph-mon[27939]: Updating smithi179:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:30:01.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:01 smithi086 ceph-mon[27939]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:30:01.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:01 smithi086 ceph-mon[27939]: Updating smithi179:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:30:01.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:01 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:01.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:01 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:01.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:01 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:01.883 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:01.882+0000 7f20bf4ad700 1 -- 172.21.15.179:0/850202329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20b81084e0 msgr2=0x7f20b81088c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:01.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:01.882+0000 7f20bf4ad700 1 --2- 172.21.15.179:0/850202329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20b81084e0 0x7f20b81088c0 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f20b400b0a0 tx=0x7f20b400b3b0 comp rx=0 tx=0).stop 2024-04-21T08:30:01.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:01.882+0000 7f20bf4ad700 1 -- 172.21.15.179:0/850202329 shutdown_connections 2024-04-21T08:30:01.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:01.882+0000 7f20bf4ad700 1 --2- 172.21.15.179:0/850202329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20b81084e0 0x7f20b81088c0 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:01.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:01.882+0000 7f20bf4ad700 1 -- 172.21.15.179:0/850202329 >> 172.21.15.179:0/850202329 conn(0x7f20b8076ea0 msgr2=0x7f20b80772b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:30:01.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:01.883+0000 7f20bf4ad700 1 -- 172.21.15.179:0/850202329 shutdown_connections 2024-04-21T08:30:01.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:01.883+0000 7f20bf4ad700 1 -- 172.21.15.179:0/850202329 wait complete. 2024-04-21T08:30:01.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:01.883+0000 7f20bf4ad700 1 Processor -- start 2024-04-21T08:30:01.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:01.883+0000 7f20bf4ad700 1 -- start start 2024-04-21T08:30:01.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:01.883+0000 7f20bf4ad700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20b81084e0 0x7f20b8101460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:01.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:01.883+0000 7f20bf4ad700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f20b810c920 con 0x7f20b81084e0 2024-04-21T08:30:01.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:01.883+0000 7f20bd249700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20b81084e0 0x7f20b8101460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:01.884 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:01.883+0000 7f20bd249700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20b81084e0 0x7f20b8101460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.179:57428/0 (socket says 172.21.15.179:57428) 2024-04-21T08:30:01.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:01.883+0000 7f20bd249700 1 -- 172.21.15.179:0/860996657 learned_addr learned my addr 172.21.15.179:0/860996657 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-21T08:30:01.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:01.884+0000 7f20bd249700 1 -- 172.21.15.179:0/860996657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20b8103300 con 0x7f20b81084e0 2024-04-21T08:30:01.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:01.884+0000 7f20bd249700 1 --2- 172.21.15.179:0/860996657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20b81084e0 0x7f20b8101460 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f20b4006130 tx=0x7f20b40097d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:30:01.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:01.884+0000 7f20ae7fc700 1 -- 172.21.15.179:0/860996657 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20b4021020 con 0x7f20b81084e0 2024-04-21T08:30:01.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:01.884+0000 7f20bf4ad700 1 -- 172.21.15.179:0/860996657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20b81019a0 con 0x7f20b81084e0 2024-04-21T08:30:01.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:01.884+0000 7f20ae7fc700 1 -- 172.21.15.179:0/860996657 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20b40079b0 con 0x7f20b81084e0 2024-04-21T08:30:01.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:01.884+0000 7f20ae7fc700 1 -- 172.21.15.179:0/860996657 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f20b4017040 con 0x7f20b81084e0 2024-04-21T08:30:01.885 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:01.884+0000 7f20bf4ad700 1 -- 172.21.15.179:0/860996657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20b8101e60 con 0x7f20b81084e0 2024-04-21T08:30:01.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:01.885+0000 7f20bf4ad700 1 -- 172.21.15.179:0/860996657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20b8103b90 con 0x7f20b81084e0 2024-04-21T08:30:01.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:01.885+0000 7f20ae7fc700 1 -- 172.21.15.179:0/860996657 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f20b4024020 con 0x7f20b81084e0 2024-04-21T08:30:01.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:01.885+0000 7f20ae7fc700 1 --2- 172.21.15.179:0/860996657 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f20a4039650 0x7f20a403bb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:01.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:01.885+0000 7f20ae7fc700 1 -- 172.21.15.179:0/860996657 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f20b400f050 con 0x7f20b81084e0 2024-04-21T08:30:01.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:01.885+0000 7f20bca48700 1 --2- 172.21.15.179:0/860996657 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f20a4039650 0x7f20a403bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:01.886 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:01.886+0000 7f20bca48700 1 --2- 172.21.15.179:0/860996657 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f20a4039650 0x7f20a403bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f20a8006fd0 tx=0x7f20a8006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:30:01.890 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:01.891+0000 7f20ae7fc700 1 -- 172.21.15.179:0/860996657 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f20b402b020 con 0x7f20b81084e0 2024-04-21T08:30:02.099 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:02.100+0000 7f20bf4ad700 1 -- 172.21.15.179:0/860996657 --> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi086:172.21.15.86=a;smithi088:172.21.15.88=b;smithi179:172.21.15.179=c", "target": ["mon-mgr", ""]}) -- 0x7f20b8102940 con 0x7f20a4039650 2024-04-21T08:30:02.106 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:02.107+0000 7f20ae7fc700 1 -- 172.21.15.179:0/860996657 <== mgr.14152 v2:172.21.15.86:6800/1364156965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f20b8102940 con 0x7f20a4039650 2024-04-21T08:30:02.106 INFO:teuthology.orchestra.run.smithi179.stdout:Scheduled mon update... 2024-04-21T08:30:02.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:02.109+0000 7f20bf4ad700 1 -- 172.21.15.179:0/860996657 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f20a4039650 msgr2=0x7f20a403bb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:02.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:02.109+0000 7f20bf4ad700 1 --2- 172.21.15.179:0/860996657 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f20a4039650 0x7f20a403bb10 secure :-1 s=READY pgs=19 cs=0 l=1 rev1=1 crypto rx=0x7f20a8006fd0 tx=0x7f20a8006e40 comp rx=0 tx=0).stop 2024-04-21T08:30:02.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:02.109+0000 7f20bf4ad700 1 -- 172.21.15.179:0/860996657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20b81084e0 msgr2=0x7f20b8101460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:02.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:02.109+0000 7f20bf4ad700 1 --2- 172.21.15.179:0/860996657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20b81084e0 0x7f20b8101460 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f20b4006130 tx=0x7f20b40097d0 comp rx=0 tx=0).stop 2024-04-21T08:30:02.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:02.109+0000 7f20bf4ad700 1 -- 172.21.15.179:0/860996657 shutdown_connections 2024-04-21T08:30:02.108 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:02.109+0000 7f20bf4ad700 1 --2- 172.21.15.179:0/860996657 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f20a4039650 0x7f20a403bb10 unknown :-1 s=CLOSED pgs=19 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:02.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:02.109+0000 7f20bf4ad700 1 --2- 172.21.15.179:0/860996657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20b81084e0 0x7f20b8101460 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:02.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:02.109+0000 7f20bf4ad700 1 -- 172.21.15.179:0/860996657 >> 172.21.15.179:0/860996657 conn(0x7f20b8076ea0 msgr2=0x7f20b80fe960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:30:02.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:02.109+0000 7f20bf4ad700 1 -- 172.21.15.179:0/860996657 shutdown_connections 2024-04-21T08:30:02.109 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:02.109+0000 7f20bf4ad700 1 -- 172.21.15.179:0/860996657 wait complete. 2024-04-21T08:30:02.779 DEBUG:teuthology.orchestra.run.smithi088:mon.b> sudo journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mon.b.service 2024-04-21T08:30:02.782 DEBUG:teuthology.orchestra.run.smithi179:mon.c> sudo journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mon.c.service 2024-04-21T08:30:02.784 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-21T08:30:02.784 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-21T08:30:02.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:02 smithi086 ceph-mon[27939]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:02.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:02 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:02.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:02 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:02.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:02 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:02.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:02 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:02.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:02 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:02.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:02 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:30:02.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:02 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:03.501 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:30:03.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:03 smithi086 ceph-mon[27939]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi086:172.21.15.86=a;smithi088:172.21.15.88=b;smithi179:172.21.15.179=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:30:03.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:03 smithi086 ceph-mon[27939]: Saving service mon spec with placement smithi086:172.21.15.86=a;smithi088:172.21.15.88=b;smithi179:172.21.15.179=c;count:3 2024-04-21T08:30:03.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:03 smithi086 ceph-mon[27939]: Deploying daemon mon.c on smithi179 2024-04-21T08:30:04.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:04 smithi086 ceph-mon[27939]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:05.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:05 smithi086 ceph-mon[27939]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:06.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.362+0000 7f1d873b5700 1 -- 172.21.15.179:0/2041544560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d80101590 msgr2=0x7f1d80101970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:06.363 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.362+0000 7f1d873b5700 1 --2- 172.21.15.179:0/2041544560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d80101590 0x7f1d80101970 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c009910 tx=0x7f1d7c009c20 comp rx=0 tx=0).stop 2024-04-21T08:30:06.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.362+0000 7f1d873b5700 1 -- 172.21.15.179:0/2041544560 shutdown_connections 2024-04-21T08:30:06.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.362+0000 7f1d873b5700 1 --2- 172.21.15.179:0/2041544560 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d80101590 0x7f1d80101970 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:06.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.362+0000 7f1d873b5700 1 -- 172.21.15.179:0/2041544560 >> 172.21.15.179:0/2041544560 conn(0x7f1d800fce40 msgr2=0x7f1d800ff260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:30:06.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.362+0000 7f1d873b5700 1 -- 172.21.15.179:0/2041544560 shutdown_connections 2024-04-21T08:30:06.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.362+0000 7f1d873b5700 1 -- 172.21.15.179:0/2041544560 wait complete. 2024-04-21T08:30:06.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.363+0000 7f1d873b5700 1 Processor -- start 2024-04-21T08:30:06.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.363+0000 7f1d873b5700 1 -- start start 2024-04-21T08:30:06.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.363+0000 7f1d873b5700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d80101590 0x7f1d80196d80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:06.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.363+0000 7f1d873b5700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1d8010b810 con 0x7f1d80101590 2024-04-21T08:30:06.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.363+0000 7f1d85151700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d80101590 0x7f1d80196d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:06.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.363+0000 7f1d85151700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d80101590 0x7f1d80196d80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.179:57440/0 (socket says 172.21.15.179:57440) 2024-04-21T08:30:06.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.363+0000 7f1d85151700 1 -- 172.21.15.179:0/1551897969 learned_addr learned my addr 172.21.15.179:0/1551897969 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-21T08:30:06.364 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.363+0000 7f1d85151700 1 -- 172.21.15.179:0/1551897969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1d801972c0 con 0x7f1d80101590 2024-04-21T08:30:06.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.364+0000 7f1d85151700 1 --2- 172.21.15.179:0/1551897969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d80101590 0x7f1d80196d80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c0056d0 tx=0x7f1d7c0057b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:30:06.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.364+0000 7f1d767fc700 1 -- 172.21.15.179:0/1551897969 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d7c026070 con 0x7f1d80101590 2024-04-21T08:30:06.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.364+0000 7f1d873b5700 1 -- 172.21.15.179:0/1551897969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1d801975b0 con 0x7f1d80101590 2024-04-21T08:30:06.365 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.364+0000 7f1d873b5700 1 -- 172.21.15.179:0/1551897969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1d8019b650 con 0x7f1d80101590 2024-04-21T08:30:06.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.364+0000 7f1d767fc700 1 -- 172.21.15.179:0/1551897969 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1d7c00b6f0 con 0x7f1d80101590 2024-04-21T08:30:06.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.364+0000 7f1d767fc700 1 -- 172.21.15.179:0/1551897969 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 205+0+0 (secure 0 0 0) 0x7f1d7c030060 con 0x7f1d80101590 2024-04-21T08:30:06.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.365+0000 7f1d767fc700 1 -- 172.21.15.179:0/1551897969 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f1d7c029070 con 0x7f1d80101590 2024-04-21T08:30:06.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.365+0000 7f1d767fc700 1 --2- 172.21.15.179:0/1551897969 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f1d6c039630 0x7f1d6c03baf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:06.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.365+0000 7f1d767fc700 1 -- 172.21.15.179:0/1551897969 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f1d7c01e070 con 0x7f1d80101590 2024-04-21T08:30:06.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.365+0000 7f1d873b5700 1 -- 172.21.15.179:0/1551897969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1d64005330 con 0x7f1d80101590 2024-04-21T08:30:06.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.365+0000 7f1d84950700 1 --2- 172.21.15.179:0/1551897969 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f1d6c039630 0x7f1d6c03baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:06.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.366+0000 7f1d84950700 1 --2- 172.21.15.179:0/1551897969 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f1d6c039630 0x7f1d6c03baf0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1d70006fd0 tx=0x7f1d70006e40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:30:06.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.371+0000 7f1d767fc700 1 -- 172.21.15.179:0/1551897969 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1d7c013070 con 0x7f1d80101590 2024-04-21T08:30:06.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.680+0000 7f1d873b5700 1 -- 172.21.15.179:0/1551897969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f1d64005620 con 0x7f1d80101590 2024-04-21T08:30:06.680 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.680+0000 7f1d767fc700 1 -- 172.21.15.179:0/1551897969 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 1 v1) ==== 95+0+745 (secure 0 0 0) 0x7f1d7c015350 con 0x7f1d80101590 2024-04-21T08:30:06.680 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-21T08:30:06.681 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":1,"fsid":"f8cec7cc-ffb8-11ee-bc93-c7b262605968","modified":"2024-04-21T08:28:19.891280Z","created":"2024-04-21T08:28:19.891280Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:3300","nonce":0},{"type":"v1","addr":"172.21.15.86:6789","nonce":0}]},"addr":"172.21.15.86:6789/0","public_addr":"172.21.15.86:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0]} 2024-04-21T08:30:06.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.683+0000 7f1d873b5700 1 -- 172.21.15.179:0/1551897969 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f1d6c039630 msgr2=0x7f1d6c03baf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:06.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.683+0000 7f1d873b5700 1 --2- 172.21.15.179:0/1551897969 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f1d6c039630 0x7f1d6c03baf0 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f1d70006fd0 tx=0x7f1d70006e40 comp rx=0 tx=0).stop 2024-04-21T08:30:06.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.683+0000 7f1d873b5700 1 -- 172.21.15.179:0/1551897969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d80101590 msgr2=0x7f1d80196d80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:06.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.683+0000 7f1d873b5700 1 --2- 172.21.15.179:0/1551897969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d80101590 0x7f1d80196d80 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f1d7c0056d0 tx=0x7f1d7c0057b0 comp rx=0 tx=0).stop 2024-04-21T08:30:06.682 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.683+0000 7f1d873b5700 1 -- 172.21.15.179:0/1551897969 shutdown_connections 2024-04-21T08:30:06.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.683+0000 7f1d873b5700 1 --2- 172.21.15.179:0/1551897969 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f1d6c039630 0x7f1d6c03baf0 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:06.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.683+0000 7f1d873b5700 1 --2- 172.21.15.179:0/1551897969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1d80101590 0x7f1d80196d80 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:06.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.683+0000 7f1d873b5700 1 -- 172.21.15.179:0/1551897969 >> 172.21.15.179:0/1551897969 conn(0x7f1d800fce40 msgr2=0x7f1d80102f60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:30:06.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.683+0000 7f1d873b5700 1 -- 172.21.15.179:0/1551897969 shutdown_connections 2024-04-21T08:30:06.683 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:06.683+0000 7f1d873b5700 1 -- 172.21.15.179:0/1551897969 wait complete. 2024-04-21T08:30:06.683 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 1 2024-04-21T08:30:06.789 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:06 smithi179 podman[31796]: 2024-04-21 08:30:06.791098395 +0000 UTC m=+0.140372547 container create e2e289b587bcba803498048168715310bb280631cd7eb827a7877873c61eec2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mon-c, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-21T08:30:07.044 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:06 smithi179 podman[31796]: 2024-04-21 08:30:06.993351841 +0000 UTC m=+0.342625951 container init e2e289b587bcba803498048168715310bb280631cd7eb827a7877873c61eec2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mon-c, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-21T08:30:07.044 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:06 smithi179 podman[31796]: 2024-04-21 08:30:06.996668562 +0000 UTC m=+0.345942673 container start e2e289b587bcba803498048168715310bb280631cd7eb827a7877873c61eec2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mon-c, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-21T08:30:07.044 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: set uid:gid to 167:167 (ceph:ceph) 2024-04-21T08:30:07.044 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev), process ceph-mon, pid 2 2024-04-21T08:30:07.044 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: pidfile_write: ignore empty --pid-file 2024-04-21T08:30:07.044 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: load: jerasure load: lrc 2024-04-21T08:30:07.045 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: RocksDB version: 7.9.2 2024-04-21T08:30:07.045 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Git sha 0 2024-04-21T08:30:07.045 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Compile date 2024-04-19 14:19:22 2024-04-21T08:30:07.045 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: DB SUMMARY 2024-04-21T08:30:07.045 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: DB Session ID: AZWD2YLB7IDF53CXMPLJ 2024-04-21T08:30:07.045 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: CURRENT file: CURRENT 2024-04-21T08:30:07.045 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: IDENTITY file: IDENTITY 2024-04-21T08:30:07.045 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: MANIFEST file: MANIFEST-000005 size: 59 Bytes 2024-04-21T08:30:07.045 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: SST files in /var/lib/ceph/mon/ceph-c/store.db dir, Total Num: 0, files: 2024-04-21T08:30:07.045 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Write Ahead Log file in /var/lib/ceph/mon/ceph-c/store.db: 000004.log size: 511 ; 2024-04-21T08:30:07.045 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.error_if_exists: 0 2024-04-21T08:30:07.045 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.create_if_missing: 0 2024-04-21T08:30:07.045 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.paranoid_checks: 1 2024-04-21T08:30:07.045 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.flush_verify_memtable_count: 1 2024-04-21T08:30:07.046 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.track_and_verify_wals_in_manifest: 0 2024-04-21T08:30:07.046 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.verify_sst_unique_id_in_manifest: 1 2024-04-21T08:30:07.046 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.env: 0x562a0a06ccc0 2024-04-21T08:30:07.046 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.fs: PosixFileSystem 2024-04-21T08:30:07.046 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.info_log: 0x562a0c69dbe0 2024-04-21T08:30:07.046 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.max_file_opening_threads: 16 2024-04-21T08:30:07.048 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.statistics: (nil) 2024-04-21T08:30:07.048 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.use_fsync: 0 2024-04-21T08:30:07.048 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.max_log_file_size: 0 2024-04-21T08:30:07.048 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.max_manifest_file_size: 1073741824 2024-04-21T08:30:07.048 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.log_file_time_to_roll: 0 2024-04-21T08:30:07.049 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.keep_log_file_num: 1000 2024-04-21T08:30:07.049 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.recycle_log_file_num: 0 2024-04-21T08:30:07.049 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.allow_fallocate: 1 2024-04-21T08:30:07.049 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.allow_mmap_reads: 0 2024-04-21T08:30:07.049 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.allow_mmap_writes: 0 2024-04-21T08:30:07.049 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.use_direct_reads: 0 2024-04-21T08:30:07.049 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.use_direct_io_for_flush_and_compaction: 0 2024-04-21T08:30:07.050 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.create_missing_column_families: 0 2024-04-21T08:30:07.050 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.db_log_dir: 2024-04-21T08:30:07.050 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.wal_dir: 2024-04-21T08:30:07.050 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.table_cache_numshardbits: 6 2024-04-21T08:30:07.050 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.WAL_ttl_seconds: 0 2024-04-21T08:30:07.050 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.WAL_size_limit_MB: 0 2024-04-21T08:30:07.050 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.max_write_batch_group_size_bytes: 1048576 2024-04-21T08:30:07.050 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.manifest_preallocation_size: 4194304 2024-04-21T08:30:07.050 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.is_fd_close_on_exec: 1 2024-04-21T08:30:07.050 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.advise_random_on_open: 1 2024-04-21T08:30:07.050 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.db_write_buffer_size: 0 2024-04-21T08:30:07.050 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.write_buffer_manager: 0x562a0c6310e0 2024-04-21T08:30:07.050 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.access_hint_on_compaction_start: 1 2024-04-21T08:30:07.051 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.random_access_max_buffer_size: 1048576 2024-04-21T08:30:07.051 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.use_adaptive_mutex: 0 2024-04-21T08:30:07.051 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.rate_limiter: (nil) 2024-04-21T08:30:07.051 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.sst_file_manager.rate_bytes_per_sec: 0 2024-04-21T08:30:07.051 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.wal_recovery_mode: 2 2024-04-21T08:30:07.051 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.enable_thread_tracking: 0 2024-04-21T08:30:07.051 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.enable_pipelined_write: 0 2024-04-21T08:30:07.051 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.unordered_write: 0 2024-04-21T08:30:07.051 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.allow_concurrent_memtable_write: 1 2024-04-21T08:30:07.051 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.enable_write_thread_adaptive_yield: 1 2024-04-21T08:30:07.051 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.write_thread_max_yield_usec: 100 2024-04-21T08:30:07.051 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.write_thread_slow_yield_usec: 3 2024-04-21T08:30:07.051 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.row_cache: None 2024-04-21T08:30:07.051 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.wal_filter: None 2024-04-21T08:30:07.052 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.avoid_flush_during_recovery: 0 2024-04-21T08:30:07.052 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.allow_ingest_behind: 0 2024-04-21T08:30:07.052 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.two_write_queues: 0 2024-04-21T08:30:07.052 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.manual_wal_flush: 0 2024-04-21T08:30:07.052 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.wal_compression: 0 2024-04-21T08:30:07.052 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.atomic_flush: 0 2024-04-21T08:30:07.052 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.avoid_unnecessary_blocking_io: 0 2024-04-21T08:30:07.052 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.persist_stats_to_disk: 0 2024-04-21T08:30:07.052 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.write_dbid_to_manifest: 0 2024-04-21T08:30:07.052 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.log_readahead_size: 0 2024-04-21T08:30:07.052 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.file_checksum_gen_factory: Unknown 2024-04-21T08:30:07.052 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.best_efforts_recovery: 0 2024-04-21T08:30:07.052 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.max_bgerror_resume_count: 2147483647 2024-04-21T08:30:07.052 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.bgerror_resume_retry_interval: 1000000 2024-04-21T08:30:07.052 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.allow_data_in_errors: 0 2024-04-21T08:30:07.053 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.db_host_id: __hostname__ 2024-04-21T08:30:07.053 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.enforce_single_del_contracts: true 2024-04-21T08:30:07.053 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.max_background_jobs: 2 2024-04-21T08:30:07.053 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.max_background_compactions: -1 2024-04-21T08:30:07.053 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.max_subcompactions: 1 2024-04-21T08:30:07.053 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.avoid_flush_during_shutdown: 0 2024-04-21T08:30:07.053 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.writable_file_max_buffer_size: 1048576 2024-04-21T08:30:07.053 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.delayed_write_rate : 16777216 2024-04-21T08:30:07.053 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.max_total_wal_size: 0 2024-04-21T08:30:07.053 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.delete_obsolete_files_period_micros: 21600000000 2024-04-21T08:30:07.053 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.stats_dump_period_sec: 600 2024-04-21T08:30:07.053 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.stats_persist_period_sec: 600 2024-04-21T08:30:07.053 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.stats_history_buffer_size: 1048576 2024-04-21T08:30:07.053 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.max_open_files: -1 2024-04-21T08:30:07.053 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.bytes_per_sync: 0 2024-04-21T08:30:07.054 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.wal_bytes_per_sync: 0 2024-04-21T08:30:07.054 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.strict_bytes_per_sync: 0 2024-04-21T08:30:07.054 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.compaction_readahead_size: 0 2024-04-21T08:30:07.054 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.max_background_flushes: -1 2024-04-21T08:30:07.054 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Compression algorithms supported: 2024-04-21T08:30:07.054 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: kZSTDNotFinalCompression supported: 0 2024-04-21T08:30:07.054 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: kZSTD supported: 0 2024-04-21T08:30:07.054 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: kXpressCompression supported: 0 2024-04-21T08:30:07.054 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: kLZ4HCCompression supported: 1 2024-04-21T08:30:07.054 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: kZlibCompression supported: 1 2024-04-21T08:30:07.054 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: kSnappyCompression supported: 1 2024-04-21T08:30:07.054 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: kLZ4Compression supported: 1 2024-04-21T08:30:07.054 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: kBZip2Compression supported: 0 2024-04-21T08:30:07.055 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Fast CRC32 supported: Supported on x86 2024-04-21T08:30:07.055 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: DMutex implementation: pthread_mutex_t 2024-04-21T08:30:07.055 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: [db/version_set.cc:5527] Recovering from manifest file: /var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 2024-04-21T08:30:07.055 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: [db/column_family.cc:630] --------------- Options for column family [default]: 2024-04-21T08:30:07.055 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.comparator: leveldb.BytewiseComparator 2024-04-21T08:30:07.055 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.merge_operator: 2024-04-21T08:30:07.055 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.compaction_filter: None 2024-04-21T08:30:07.055 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.compaction_filter_factory: None 2024-04-21T08:30:07.055 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.sst_partitioner_factory: None 2024-04-21T08:30:07.055 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.memtable_factory: SkipListFactory 2024-04-21T08:30:07.055 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.table_factory: BlockBasedTable 2024-04-21T08:30:07.055 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: table_factory options: flush_block_policy_factory: FlushBlockBySizePolicyFactory (0x562a0c69ddc0) 2024-04-21T08:30:07.055 INFO:journalctl@ceph.mon.c.smithi179.stdout: cache_index_and_filter_blocks: 1 2024-04-21T08:30:07.055 INFO:journalctl@ceph.mon.c.smithi179.stdout: cache_index_and_filter_blocks_with_high_priority: 0 2024-04-21T08:30:07.056 INFO:journalctl@ceph.mon.c.smithi179.stdout: pin_l0_filter_and_index_blocks_in_cache: 0 2024-04-21T08:30:07.056 INFO:journalctl@ceph.mon.c.smithi179.stdout: pin_top_level_index_and_filter: 1 2024-04-21T08:30:07.056 INFO:journalctl@ceph.mon.c.smithi179.stdout: index_type: 0 2024-04-21T08:30:07.056 INFO:journalctl@ceph.mon.c.smithi179.stdout: data_block_index_type: 0 2024-04-21T08:30:07.056 INFO:journalctl@ceph.mon.c.smithi179.stdout: index_shortening: 1 2024-04-21T08:30:07.056 INFO:journalctl@ceph.mon.c.smithi179.stdout: data_block_hash_table_util_ratio: 0.750000 2024-04-21T08:30:07.056 INFO:journalctl@ceph.mon.c.smithi179.stdout: checksum: 4 2024-04-21T08:30:07.056 INFO:journalctl@ceph.mon.c.smithi179.stdout: no_block_cache: 0 2024-04-21T08:30:07.056 INFO:journalctl@ceph.mon.c.smithi179.stdout: block_cache: 0x562a0c6631f0 2024-04-21T08:30:07.056 INFO:journalctl@ceph.mon.c.smithi179.stdout: block_cache_name: BinnedLRUCache 2024-04-21T08:30:07.056 INFO:journalctl@ceph.mon.c.smithi179.stdout: block_cache_options: 2024-04-21T08:30:07.056 INFO:journalctl@ceph.mon.c.smithi179.stdout: capacity : 536870912 2024-04-21T08:30:07.056 INFO:journalctl@ceph.mon.c.smithi179.stdout: num_shard_bits : 4 2024-04-21T08:30:07.056 INFO:journalctl@ceph.mon.c.smithi179.stdout: strict_capacity_limit : 0 2024-04-21T08:30:07.057 INFO:journalctl@ceph.mon.c.smithi179.stdout: high_pri_pool_ratio: 0.000 2024-04-21T08:30:07.057 INFO:journalctl@ceph.mon.c.smithi179.stdout: block_cache_compressed: (nil) 2024-04-21T08:30:07.057 INFO:journalctl@ceph.mon.c.smithi179.stdout: persistent_cache: (nil) 2024-04-21T08:30:07.057 INFO:journalctl@ceph.mon.c.smithi179.stdout: block_size: 4096 2024-04-21T08:30:07.057 INFO:journalctl@ceph.mon.c.smithi179.stdout: block_size_deviation: 10 2024-04-21T08:30:07.057 INFO:journalctl@ceph.mon.c.smithi179.stdout: block_restart_interval: 16 2024-04-21T08:30:07.057 INFO:journalctl@ceph.mon.c.smithi179.stdout: index_block_restart_interval: 1 2024-04-21T08:30:07.057 INFO:journalctl@ceph.mon.c.smithi179.stdout: metadata_block_size: 4096 2024-04-21T08:30:07.057 INFO:journalctl@ceph.mon.c.smithi179.stdout: partition_filters: 0 2024-04-21T08:30:07.057 INFO:journalctl@ceph.mon.c.smithi179.stdout: use_delta_encoding: 1 2024-04-21T08:30:07.057 INFO:journalctl@ceph.mon.c.smithi179.stdout: filter_policy: bloomfilter 2024-04-21T08:30:07.057 INFO:journalctl@ceph.mon.c.smithi179.stdout: whole_key_filtering: 1 2024-04-21T08:30:07.057 INFO:journalctl@ceph.mon.c.smithi179.stdout: verify_compression: 0 2024-04-21T08:30:07.057 INFO:journalctl@ceph.mon.c.smithi179.stdout: read_amp_bytes_per_bit: 0 2024-04-21T08:30:07.057 INFO:journalctl@ceph.mon.c.smithi179.stdout: format_version: 5 2024-04-21T08:30:07.058 INFO:journalctl@ceph.mon.c.smithi179.stdout: enable_index_compression: 1 2024-04-21T08:30:07.058 INFO:journalctl@ceph.mon.c.smithi179.stdout: block_align: 0 2024-04-21T08:30:07.058 INFO:journalctl@ceph.mon.c.smithi179.stdout: max_auto_readahead_size: 262144 2024-04-21T08:30:07.058 INFO:journalctl@ceph.mon.c.smithi179.stdout: prepopulate_block_cache: 0 2024-04-21T08:30:07.058 INFO:journalctl@ceph.mon.c.smithi179.stdout: initial_auto_readahead_size: 8192 2024-04-21T08:30:07.058 INFO:journalctl@ceph.mon.c.smithi179.stdout: num_file_reads_for_auto_readahead: 2 2024-04-21T08:30:07.058 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.write_buffer_size: 33554432 2024-04-21T08:30:07.058 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.max_write_buffer_number: 2 2024-04-21T08:30:07.058 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.compression: NoCompression 2024-04-21T08:30:07.058 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.bottommost_compression: Disabled 2024-04-21T08:30:07.058 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.prefix_extractor: nullptr 2024-04-21T08:30:07.058 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.memtable_insert_with_hint_prefix_extractor: nullptr 2024-04-21T08:30:07.058 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.num_levels: 7 2024-04-21T08:30:07.058 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.min_write_buffer_number_to_merge: 1 2024-04-21T08:30:07.059 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.max_write_buffer_number_to_maintain: 0 2024-04-21T08:30:07.059 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.max_write_buffer_size_to_maintain: 0 2024-04-21T08:30:07.059 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.bottommost_compression_opts.window_bits: -14 2024-04-21T08:30:07.059 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.bottommost_compression_opts.level: 32767 2024-04-21T08:30:07.059 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.bottommost_compression_opts.strategy: 0 2024-04-21T08:30:07.059 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.bottommost_compression_opts.max_dict_bytes: 0 2024-04-21T08:30:07.059 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.bottommost_compression_opts.zstd_max_train_bytes: 0 2024-04-21T08:30:07.059 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.bottommost_compression_opts.parallel_threads: 1 2024-04-21T08:30:07.059 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.bottommost_compression_opts.enabled: false 2024-04-21T08:30:07.059 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.bottommost_compression_opts.max_dict_buffer_bytes: 0 2024-04-21T08:30:07.059 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.bottommost_compression_opts.use_zstd_dict_trainer: true 2024-04-21T08:30:07.059 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.compression_opts.window_bits: -14 2024-04-21T08:30:07.059 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.compression_opts.level: 32767 2024-04-21T08:30:07.059 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.compression_opts.strategy: 0 2024-04-21T08:30:07.059 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.compression_opts.max_dict_bytes: 0 2024-04-21T08:30:07.060 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.compression_opts.zstd_max_train_bytes: 0 2024-04-21T08:30:07.060 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.compression_opts.use_zstd_dict_trainer: true 2024-04-21T08:30:07.060 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.compression_opts.parallel_threads: 1 2024-04-21T08:30:07.060 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.compression_opts.enabled: false 2024-04-21T08:30:07.060 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.compression_opts.max_dict_buffer_bytes: 0 2024-04-21T08:30:07.060 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.level0_file_num_compaction_trigger: 4 2024-04-21T08:30:07.060 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.level0_slowdown_writes_trigger: 20 2024-04-21T08:30:07.060 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.level0_stop_writes_trigger: 36 2024-04-21T08:30:07.060 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.target_file_size_base: 67108864 2024-04-21T08:30:07.060 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.target_file_size_multiplier: 1 2024-04-21T08:30:07.060 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.max_bytes_for_level_base: 268435456 2024-04-21T08:30:07.060 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.level_compaction_dynamic_level_bytes: 1 2024-04-21T08:30:07.060 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.max_bytes_for_level_multiplier: 10.000000 2024-04-21T08:30:07.060 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[0]: 1 2024-04-21T08:30:07.061 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[1]: 1 2024-04-21T08:30:07.061 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[2]: 1 2024-04-21T08:30:07.061 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[3]: 1 2024-04-21T08:30:07.061 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[4]: 1 2024-04-21T08:30:07.061 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[5]: 1 2024-04-21T08:30:07.061 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.max_bytes_for_level_multiplier_addtl[6]: 1 2024-04-21T08:30:07.061 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.max_sequential_skip_in_iterations: 8 2024-04-21T08:30:07.061 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.max_compaction_bytes: 1677721600 2024-04-21T08:30:07.061 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.ignore_max_compaction_bytes_for_input: true 2024-04-21T08:30:07.061 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.arena_block_size: 1048576 2024-04-21T08:30:07.061 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.soft_pending_compaction_bytes_limit: 68719476736 2024-04-21T08:30:07.061 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.hard_pending_compaction_bytes_limit: 274877906944 2024-04-21T08:30:07.061 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.disable_auto_compactions: 0 2024-04-21T08:30:07.061 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.compaction_style: kCompactionStyleLevel 2024-04-21T08:30:07.061 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.compaction_pri: kMinOverlappingRatio 2024-04-21T08:30:07.062 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.compaction_options_universal.size_ratio: 1 2024-04-21T08:30:07.062 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.compaction_options_universal.min_merge_width: 2 2024-04-21T08:30:07.062 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.compaction_options_universal.max_merge_width: 4294967295 2024-04-21T08:30:07.062 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.compaction_options_universal.max_size_amplification_percent: 200 2024-04-21T08:30:07.062 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.compaction_options_universal.compression_size_percent: -1 2024-04-21T08:30:07.062 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.compaction_options_universal.stop_style: kCompactionStopStyleTotalSize 2024-04-21T08:30:07.062 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.compaction_options_fifo.max_table_files_size: 1073741824 2024-04-21T08:30:07.062 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.compaction_options_fifo.allow_compaction: 0 2024-04-21T08:30:07.062 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.table_properties_collectors: CompactOnDeletionCollector (Sliding window size = 32768 Deletion trigger = 16384 Deletion ratio = 0); 2024-04-21T08:30:07.062 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.inplace_update_support: 0 2024-04-21T08:30:07.065 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.inplace_update_num_locks: 10000 2024-04-21T08:30:07.065 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.memtable_prefix_bloom_size_ratio: 0.000000 2024-04-21T08:30:07.065 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.memtable_whole_key_filtering: 0 2024-04-21T08:30:07.065 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.memtable_huge_page_size: 0 2024-04-21T08:30:07.066 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.bloom_locality: 0 2024-04-21T08:30:07.066 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.max_successive_merges: 0 2024-04-21T08:30:07.066 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.optimize_filters_for_hits: 0 2024-04-21T08:30:07.066 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.paranoid_file_checks: 0 2024-04-21T08:30:07.066 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.force_consistency_checks: 1 2024-04-21T08:30:07.066 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.report_bg_io_stats: 0 2024-04-21T08:30:07.066 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.ttl: 2592000 2024-04-21T08:30:07.066 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.periodic_compaction_seconds: 0 2024-04-21T08:30:07.066 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.preclude_last_level_data_seconds: 0 2024-04-21T08:30:07.066 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.preserve_internal_time_seconds: 0 2024-04-21T08:30:07.066 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.enable_blob_files: false 2024-04-21T08:30:07.066 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.min_blob_size: 0 2024-04-21T08:30:07.066 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.blob_file_size: 268435456 2024-04-21T08:30:07.066 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.blob_compression_type: NoCompression 2024-04-21T08:30:07.067 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.enable_blob_garbage_collection: false 2024-04-21T08:30:07.067 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.blob_garbage_collection_age_cutoff: 0.250000 2024-04-21T08:30:07.067 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.blob_garbage_collection_force_threshold: 1.000000 2024-04-21T08:30:07.067 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.blob_compaction_readahead_size: 0 2024-04-21T08:30:07.067 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.blob_file_starting_level: 0 2024-04-21T08:30:07.067 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: Options.experimental_mempurge_threshold: 0.000000 2024-04-21T08:30:07.067 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: [db/version_set.cc:5566] Recovered from manifest file:/var/lib/ceph/mon/ceph-c/store.db/MANIFEST-000005 succeeded,manifest_file_number is 5, next_file_number is 7, last_sequence is 0, log_number is 0,prev_log_number is 0,max_column_family is 0,min_log_number_to_keep is 0 2024-04-21T08:30:07.067 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: [db/version_set.cc:5581] Column family [default] (ID 0), log number is 0 2024-04-21T08:30:07.067 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: [db/db_impl/db_impl_open.cc:539] DB ID: f6983e23-ad4e-4dc4-b974-009f848da5b5 2024-04-21T08:30:07.067 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713688207026464, "job": 1, "event": "recovery_started", "wal_files": [4]} 2024-04-21T08:30:07.067 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: [db/db_impl/db_impl_open.cc:1043] Recovering log #4 mode 2 2024-04-21T08:30:07.067 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713688207026875, "cf_name": "default", "job": 1, "event": "table_file_creation", "file_number": 8, "file_size": 1648, "file_checksum": "", "file_checksum_func_name": "Unknown", "smallest_seqno": 1, "largest_seqno": 5, "table_properties": {"data_size": 523, "index_size": 31, "index_partitions": 0, "top_level_index_size": 0, "index_key_is_user_key": 1, "index_value_is_delta_encoded": 1, "filter_size": 69, "raw_key_size": 115, "raw_average_key_size": 23, "raw_value_size": 401, "raw_average_value_size": 80, "num_data_blocks": 1, "num_entries": 5, "num_filter_entries": 5, "num_deletions": 0, "num_merge_operands": 0, "num_range_deletions": 0, "format_version": 0, "fixed_key_len": 0, "filter_policy": "bloomfilter", "column_family_name": "default", "column_family_id": 0, "comparator": "leveldb.BytewiseComparator", "merge_operator": "", "prefix_extractor_name": "nullptr", "property_collectors": "[CompactOnDeletionCollector]", "compression": "NoCompression", "compression_options": "window_bits=-14; level=32767; strategy=0; max_dict_bytes=0; zstd_max_train_bytes=0; enabled=0; max_dict_buffer_bytes=0; use_zstd_dict_trainer=1; ", "creation_time": 1713688207, "oldest_key_time": 0, "file_creation_time": 0, "slow_compression_estimated_data_size": 0, "fast_compression_estimated_data_size": 0, "db_id": "f6983e23-ad4e-4dc4-b974-009f848da5b5", "db_session_id": "AZWD2YLB7IDF53CXMPLJ", "orig_file_number": 8, "seqno_to_time_mapping": "N/A"}} 2024-04-21T08:30:07.067 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: EVENT_LOG_v1 {"time_micros": 1713688207026953, "job": 1, "event": "recovery_finished"} 2024-04-21T08:30:07.067 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: [db/version_set.cc:5047] Creating manifest 10 2024-04-21T08:30:07.068 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: [file/delete_scheduler.cc:74] Deleted file /var/lib/ceph/mon/ceph-c/store.db/000004.log immediately, rate_bytes_per_sec 0, total_trash_size 0 max_trash_db_ratio 0.250000 2024-04-21T08:30:07.068 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: [db/db_impl/db_impl_open.cc:1987] SstFileManager instance 0x562a0c778000 2024-04-21T08:30:07.068 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: DB pointer 0x562a0c762000 2024-04-21T08:30:07.068 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: [db/db_impl/db_impl.cc:1109] ------- DUMPING STATS ------- 2024-04-21T08:30:07.068 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: rocksdb: [db/db_impl/db_impl.cc:1111] 2024-04-21T08:30:07.068 INFO:journalctl@ceph.mon.c.smithi179.stdout: ** DB Stats ** 2024-04-21T08:30:07.068 INFO:journalctl@ceph.mon.c.smithi179.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-21T08:30:07.068 INFO:journalctl@ceph.mon.c.smithi179.stdout: Cumulative writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 GB, 0.00 MB/s 2024-04-21T08:30:07.068 INFO:journalctl@ceph.mon.c.smithi179.stdout: Cumulative WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-21T08:30:07.068 INFO:journalctl@ceph.mon.c.smithi179.stdout: Cumulative stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-21T08:30:07.068 INFO:journalctl@ceph.mon.c.smithi179.stdout: Interval writes: 0 writes, 0 keys, 0 commit groups, 0.0 writes per commit group, ingest: 0.00 MB, 0.00 MB/s 2024-04-21T08:30:07.068 INFO:journalctl@ceph.mon.c.smithi179.stdout: Interval WAL: 0 writes, 0 syncs, 0.00 writes per sync, written: 0.00 GB, 0.00 MB/s 2024-04-21T08:30:07.068 INFO:journalctl@ceph.mon.c.smithi179.stdout: Interval stall: 00:00:0.000 H:M:S, 0.0 percent 2024-04-21T08:30:07.068 INFO:journalctl@ceph.mon.c.smithi179.stdout: 2024-04-21T08:30:07.069 INFO:journalctl@ceph.mon.c.smithi179.stdout: ** Compaction Stats [default] ** 2024-04-21T08:30:07.069 INFO:journalctl@ceph.mon.c.smithi179.stdout: Level Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-21T08:30:07.069 INFO:journalctl@ceph.mon.c.smithi179.stdout: ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ 2024-04-21T08:30:07.069 INFO:journalctl@ceph.mon.c.smithi179.stdout: L0 1/0 1.61 KB 0.2 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-21T08:30:07.069 INFO:journalctl@ceph.mon.c.smithi179.stdout: Sum 1/0 1.61 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-21T08:30:07.069 INFO:journalctl@ceph.mon.c.smithi179.stdout: Int 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 1.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-21T08:30:07.069 INFO:journalctl@ceph.mon.c.smithi179.stdout: 2024-04-21T08:30:07.069 INFO:journalctl@ceph.mon.c.smithi179.stdout: ** Compaction Stats [default] ** 2024-04-21T08:30:07.069 INFO:journalctl@ceph.mon.c.smithi179.stdout: Priority Files Size Score Read(GB) Rn(GB) Rnp1(GB) Write(GB) Wnew(GB) Moved(GB) W-Amp Rd(MB/s) Wr(MB/s) Comp(sec) CompMergeCPU(sec) Comp(cnt) Avg(sec) KeyIn KeyDrop Rblob(GB) Wblob(GB) 2024-04-21T08:30:07.069 INFO:journalctl@ceph.mon.c.smithi179.stdout: --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 2024-04-21T08:30:07.069 INFO:journalctl@ceph.mon.c.smithi179.stdout: User 0/0 0.00 KB 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 0.0 4.0 0.00 0.00 1 0.000 0 0 0.0 0.0 2024-04-21T08:30:07.069 INFO:journalctl@ceph.mon.c.smithi179.stdout: 2024-04-21T08:30:07.069 INFO:journalctl@ceph.mon.c.smithi179.stdout: Blob file count: 0, total size: 0.0 GB, garbage size: 0.0 GB, space amp: 0.0 2024-04-21T08:30:07.069 INFO:journalctl@ceph.mon.c.smithi179.stdout: 2024-04-21T08:30:07.070 INFO:journalctl@ceph.mon.c.smithi179.stdout: Uptime(secs): 0.0 total, 0.0 interval 2024-04-21T08:30:07.070 INFO:journalctl@ceph.mon.c.smithi179.stdout: Flush(GB): cumulative 0.000, interval 0.000 2024-04-21T08:30:07.070 INFO:journalctl@ceph.mon.c.smithi179.stdout: AddFile(GB): cumulative 0.000, interval 0.000 2024-04-21T08:30:07.070 INFO:journalctl@ceph.mon.c.smithi179.stdout: AddFile(Total Files): cumulative 0, interval 0 2024-04-21T08:30:07.070 INFO:journalctl@ceph.mon.c.smithi179.stdout: AddFile(L0 Files): cumulative 0, interval 0 2024-04-21T08:30:07.070 INFO:journalctl@ceph.mon.c.smithi179.stdout: AddFile(Keys): cumulative 0, interval 0 2024-04-21T08:30:07.070 INFO:journalctl@ceph.mon.c.smithi179.stdout: Cumulative compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-21T08:30:07.070 INFO:journalctl@ceph.mon.c.smithi179.stdout: Interval compaction: 0.00 GB write, 0.67 MB/s write, 0.00 GB read, 0.00 MB/s read, 0.0 seconds 2024-04-21T08:30:07.070 INFO:journalctl@ceph.mon.c.smithi179.stdout: Stalls(count): 0 level0_slowdown, 0 level0_slowdown_with_compaction, 0 level0_numfiles, 0 level0_numfiles_with_compaction, 0 stop for pending_compaction_bytes, 0 slowdown for pending_compaction_bytes, 0 memtable_compaction, 0 memtable_slowdown, interval 0 total count 2024-04-21T08:30:07.070 INFO:journalctl@ceph.mon.c.smithi179.stdout: Block cache BinnedLRUCache@0x562a0c6631f0#2 capacity: 512.00 MB usage: 0.22 KB table_size: 0 occupancy: 18446744073709551615 collections: 1 last_copies: 0 last_secs: 8e-06 secs_since: 0 2024-04-21T08:30:07.070 INFO:journalctl@ceph.mon.c.smithi179.stdout: Block cache entry stats(count,size,portion): FilterBlock(1,0.11 KB,2.08616e-05%) IndexBlock(1,0.11 KB,2.08616e-05%) Misc(1,0.00 KB,0%) 2024-04-21T08:30:07.070 INFO:journalctl@ceph.mon.c.smithi179.stdout: 2024-04-21T08:30:07.070 INFO:journalctl@ceph.mon.c.smithi179.stdout: ** File Read Latency Histogram By Level [default] ** 2024-04-21T08:30:07.070 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: mon.c does not exist in monmap, will attempt to join an existing cluster 2024-04-21T08:30:07.070 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: using public_addr v2:172.21.15.179:0/0 -> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-21T08:30:07.071 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: starting mon.c rank -1 at public addrs [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] at bind addrs [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon_data /var/lib/ceph/mon/ceph-c fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:30:07.071 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: mon.c@-1(???) e0 preinit fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:30:07.071 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: mon.c@-1(synchronizing).mds e1 new map 2024-04-21T08:30:07.071 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: mon.c@-1(synchronizing).mds e1 print_map 2024-04-21T08:30:07.071 INFO:journalctl@ceph.mon.c.smithi179.stdout: e1 2024-04-21T08:30:07.071 INFO:journalctl@ceph.mon.c.smithi179.stdout: enable_multiple, ever_enabled_multiple: 1,1 2024-04-21T08:30:07.071 INFO:journalctl@ceph.mon.c.smithi179.stdout: default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-21T08:30:07.071 INFO:journalctl@ceph.mon.c.smithi179.stdout: legacy client fscid: -1 2024-04-21T08:30:07.071 INFO:journalctl@ceph.mon.c.smithi179.stdout: 2024-04-21T08:30:07.071 INFO:journalctl@ceph.mon.c.smithi179.stdout: No filesystems configured 2024-04-21T08:30:07.071 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: mon.c@-1(synchronizing).osd e0 _set_cache_ratios kv ratio 0.25 inc ratio 0.375 full ratio 0.375 2024-04-21T08:30:07.071 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: mon.c@-1(synchronizing).osd e0 register_cache_with_pcm pcm target: 2147483648 pcm max: 1020054732 pcm min: 134217728 inc_osd_cache size: 1 2024-04-21T08:30:07.071 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: mon.c@-1(synchronizing).osd e1 e1: 0 total, 0 up, 0 in 2024-04-21T08:30:07.071 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: mon.c@-1(synchronizing).osd e2 e2: 0 total, 0 up, 0 in 2024-04-21T08:30:07.072 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: mon.c@-1(synchronizing).osd e3 e3: 0 total, 0 up, 0 in 2024-04-21T08:30:07.072 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: mon.c@-1(synchronizing).osd e4 e4: 0 total, 0 up, 0 in 2024-04-21T08:30:07.072 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: mon.c@-1(synchronizing).osd e4 crush map has features 3314932999778484224, adjusting msgr requires 2024-04-21T08:30:07.072 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-21T08:30:07.072 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-21T08:30:07.072 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: mon.c@-1(synchronizing).osd e4 crush map has features 288514050185494528, adjusting msgr requires 2024-04-21T08:30:07.072 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='client.14180 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:30:07.072 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: pgmap v5: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:07.072 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:07.072 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: pgmap v6: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:07.072 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='client.14182 -' entity='client.admin' cmd=[{"prefix": "orch host add", "hostname": "smithi179", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:30:07.072 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: Deploying cephadm binary to smithi179 2024-04-21T08:30:07.073 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:07.073 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:07.073 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:07.073 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:07.073 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi088", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:30:07.073 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:07.073 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:07.074 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: Updating smithi088:/etc/ceph/ceph.conf 2024-04-21T08:30:07.074 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: Updating smithi088:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:30:07.074 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: pgmap v7: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:07.074 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: Updating smithi088:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:30:07.074 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: pgmap v8: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:07.075 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:07.075 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:07.075 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:07.075 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:07.075 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:07.075 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:07.075 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: Updating smithi088:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:30:07.075 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: Added host smithi179 2024-04-21T08:30:07.075 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:07.075 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: pgmap v9: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:07.075 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:07.075 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='client.14184 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:30:07.075 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: pgmap v10: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:07.076 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3489651478' entity='client.admin' cmd=[{"prefix": "osd crush tunables", "profile": "default"}]: dispatch 2024-04-21T08:30:07.076 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: pgmap v11: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:07.076 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3489651478' entity='client.admin' cmd='[{"prefix": "osd crush tunables", "profile": "default"}]': finished 2024-04-21T08:30:07.076 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T08:30:07.076 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:07.076 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:07.076 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:07.076 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:07.076 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd/host:smithi179", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:30:07.076 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:07.076 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:07.076 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: Updating smithi179:/etc/ceph/ceph.conf 2024-04-21T08:30:07.076 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: Updating smithi179:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:30:07.076 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:30:07.076 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: Updating smithi179:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:30:07.077 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:07.077 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:07.077 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:07.077 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: pgmap v13: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:07.077 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:07.077 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:07.077 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:07.077 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:07.077 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:07.077 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:30:07.077 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:07.077 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='client.14188 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mon", "placement": "3;smithi086:172.21.15.86=a;smithi088:172.21.15.88=b;smithi179:172.21.15.179=c", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:30:07.077 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: Saving service mon spec with placement smithi086:172.21.15.86=a;smithi088:172.21.15.88=b;smithi179:172.21.15.179=c;count:3 2024-04-21T08:30:07.077 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: Deploying daemon mon.c on smithi179 2024-04-21T08:30:07.077 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: pgmap v14: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:07.078 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: pgmap v15: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:07.078 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: from='client.? 172.21.15.179:0/1551897969' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-21T08:30:07.078 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 ceph-mon[31829]: mon.c@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-21T08:30:07.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:06 smithi086 ceph-mon[27939]: from='client.? 172.21.15.179:0/1551897969' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-21T08:30:07.403 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 bash[31796]: e2e289b587bcba803498048168715310bb280631cd7eb827a7877873c61eec2f 2024-04-21T08:30:07.403 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:07 smithi179 systemd[1]: Started Ceph mon.c for f8cec7cc-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:30:08.631 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-21T08:30:08.631 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-21T08:30:08.876 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:30:12.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:12 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:30:12.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:12 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:30:12.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:12 smithi179 ceph-mon[31829]: mon.a calling monitor election 2024-04-21T08:30:12.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:12 smithi179 ceph-mon[31829]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:12.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:12 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:30:12.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:12 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:30:12.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:12 smithi179 ceph-mon[31829]: mon.c calling monitor election 2024-04-21T08:30:12.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:12 smithi179 ceph-mon[31829]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:12.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:12 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:30:12.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:12 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:30:12.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:12 smithi179 ceph-mon[31829]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:12.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:12 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:30:12.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:12 smithi179 ceph-mon[31829]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-21T08:30:12.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:12 smithi179 ceph-mon[31829]: monmap epoch 2 2024-04-21T08:30:12.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:12 smithi179 ceph-mon[31829]: fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:30:12.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:12 smithi179 ceph-mon[31829]: last_changed 2024-04-21T08:30:07.051236+0000 2024-04-21T08:30:12.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:12 smithi179 ceph-mon[31829]: created 2024-04-21T08:28:19.891280+0000 2024-04-21T08:30:12.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:12 smithi179 ceph-mon[31829]: min_mon_release 19 (squid) 2024-04-21T08:30:12.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:12 smithi179 ceph-mon[31829]: election_strategy: 1 2024-04-21T08:30:12.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:12 smithi179 ceph-mon[31829]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-21T08:30:12.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:12 smithi179 ceph-mon[31829]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.c 2024-04-21T08:30:12.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:12 smithi179 ceph-mon[31829]: fsmap 2024-04-21T08:30:12.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:12 smithi179 ceph-mon[31829]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T08:30:12.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:12 smithi179 ceph-mon[31829]: mgrmap e13: x(active, since 48s) 2024-04-21T08:30:12.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:12 smithi179 ceph-mon[31829]: overall HEALTH_OK 2024-04-21T08:30:12.499 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:12 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:12.499 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:12 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:12.499 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:12 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:12.499 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:12 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:30:12.499 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:12 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:12.522 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:12 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:30:12.522 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:12 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:30:12.522 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:12 smithi086 ceph-mon[27939]: mon.a calling monitor election 2024-04-21T08:30:12.522 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:12 smithi086 ceph-mon[27939]: pgmap v16: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:12.522 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:12 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:30:12.522 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:12 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:30:12.523 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:12 smithi086 ceph-mon[27939]: mon.c calling monitor election 2024-04-21T08:30:12.523 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:12 smithi086 ceph-mon[27939]: pgmap v17: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:12.523 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:12 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:30:12.523 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:12 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:30:12.523 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:12 smithi086 ceph-mon[27939]: pgmap v18: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:12.523 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:12 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:30:12.523 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:12 smithi086 ceph-mon[27939]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-21T08:30:12.523 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:12 smithi086 ceph-mon[27939]: monmap epoch 2 2024-04-21T08:30:12.523 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:12 smithi086 ceph-mon[27939]: fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:30:12.523 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:12 smithi086 ceph-mon[27939]: last_changed 2024-04-21T08:30:07.051236+0000 2024-04-21T08:30:12.523 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:12 smithi086 ceph-mon[27939]: created 2024-04-21T08:28:19.891280+0000 2024-04-21T08:30:12.523 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:12 smithi086 ceph-mon[27939]: min_mon_release 19 (squid) 2024-04-21T08:30:12.523 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:12 smithi086 ceph-mon[27939]: election_strategy: 1 2024-04-21T08:30:12.523 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:12 smithi086 ceph-mon[27939]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-21T08:30:12.523 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:12 smithi086 ceph-mon[27939]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.c 2024-04-21T08:30:12.524 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:12 smithi086 ceph-mon[27939]: fsmap 2024-04-21T08:30:12.524 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:12 smithi086 ceph-mon[27939]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T08:30:12.524 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:12 smithi086 ceph-mon[27939]: mgrmap e13: x(active, since 48s) 2024-04-21T08:30:12.524 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:12 smithi086 ceph-mon[27939]: overall HEALTH_OK 2024-04-21T08:30:12.524 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:12 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:12.524 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:12 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:12.524 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:12 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:12.524 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:12 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:30:12.524 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:12 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:13.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.366+0000 7f0a14433700 1 -- 172.21.15.179:0/410217132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09f80035c0 msgr2=0x7f09f8005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:13.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.366+0000 7f0a14433700 1 --2- 172.21.15.179:0/410217132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09f80035c0 0x7f09f8005a50 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f0a0c06ad30 tx=0x7f0a0800feb0 comp rx=0 tx=0).stop 2024-04-21T08:30:13.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.366+0000 7f0a14433700 1 -- 172.21.15.179:0/410217132 shutdown_connections 2024-04-21T08:30:13.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.366+0000 7f0a14433700 1 --2- 172.21.15.179:0/410217132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f09f80035c0 0x7f09f8005a50 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:13.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.366+0000 7f0a14433700 1 -- 172.21.15.179:0/410217132 >> 172.21.15.179:0/410217132 conn(0x7f0a0c0fcf40 msgr2=0x7f0a0c0ff360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:30:13.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.366+0000 7f0a14433700 1 -- 172.21.15.179:0/410217132 shutdown_connections 2024-04-21T08:30:13.366 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.366+0000 7f0a14433700 1 -- 172.21.15.179:0/410217132 wait complete. 2024-04-21T08:30:13.368 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.367+0000 7f0a14433700 1 Processor -- start 2024-04-21T08:30:13.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.367+0000 7f0a14433700 1 -- start start 2024-04-21T08:30:13.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.367+0000 7f0a14433700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09f80035c0 0x7f0a0c197ff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:13.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.367+0000 7f0a14433700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a0c198530 0x7f0a0c19c8c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:13.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.367+0000 7f0a14433700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0c10a4a0 con 0x7f0a0c198530 2024-04-21T08:30:13.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.367+0000 7f0a14433700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0a0c10a320 con 0x7f09f80035c0 2024-04-21T08:30:13.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.368+0000 7f0a121cf700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09f80035c0 0x7f0a0c197ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:13.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.368+0000 7f0a121cf700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09f80035c0 0x7f0a0c197ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54198/0 (socket says 172.21.15.179:54198) 2024-04-21T08:30:13.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.368+0000 7f0a121cf700 1 -- 172.21.15.179:0/3470994933 learned_addr learned my addr 172.21.15.179:0/3470994933 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-21T08:30:13.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.368+0000 7f0a119ce700 1 --2- 172.21.15.179:0/3470994933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a0c198530 0x7f0a0c19c8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:13.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.368+0000 7f0a121cf700 1 -- 172.21.15.179:0/3470994933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09f80035c0 msgr2=0x7f0a0c197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-21T08:30:13.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.368+0000 7f0a121cf700 1 -- 172.21.15.179:0/3470994933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09f80035c0 msgr2=0x7f0a0c197ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-21T08:30:13.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.368+0000 7f0a121cf700 1 --2- 172.21.15.179:0/3470994933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09f80035c0 0x7f0a0c197ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-21T08:30:13.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.368+0000 7f0a121cf700 1 --2- 172.21.15.179:0/3470994933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09f80035c0 0x7f0a0c197ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:30:13.369 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.368+0000 7f0a119ce700 1 -- 172.21.15.179:0/3470994933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09f80035c0 msgr2=0x7f0a0c197ff0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:30:13.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.368+0000 7f0a119ce700 1 --2- 172.21.15.179:0/3470994933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09f80035c0 0x7f0a0c197ff0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:13.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.368+0000 7f0a119ce700 1 -- 172.21.15.179:0/3470994933 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a0c198b70 con 0x7f0a0c198530 2024-04-21T08:30:13.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.368+0000 7f0a119ce700 1 --2- 172.21.15.179:0/3470994933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a0c198530 0x7f0a0c19c8c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00b4d0 tx=0x7f09fc00b890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:30:13.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.368+0000 7f0a037fe700 1 -- 172.21.15.179:0/3470994933 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f09fc011630 con 0x7f0a0c198530 2024-04-21T08:30:13.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.368+0000 7f0a037fe700 1 -- 172.21.15.179:0/3470994933 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09fc011c80 con 0x7f0a0c198530 2024-04-21T08:30:13.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.368+0000 7f0a037fe700 1 -- 172.21.15.179:0/3470994933 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 309+0+0 (secure 0 0 0) 0x7f09fc010da0 con 0x7f0a0c198530 2024-04-21T08:30:13.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.369+0000 7f0a14433700 1 -- 172.21.15.179:0/3470994933 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a0c19ce00 con 0x7f0a0c198530 2024-04-21T08:30:13.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.369+0000 7f0a14433700 1 -- 172.21.15.179:0/3470994933 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a0c19d360 con 0x7f0a0c198530 2024-04-21T08:30:13.370 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.369+0000 7f0a037fe700 1 -- 172.21.15.179:0/3470994933 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f09fc018020 con 0x7f0a0c198530 2024-04-21T08:30:13.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.370+0000 7f0a037fe700 1 --2- 172.21.15.179:0/3470994933 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f09f0039440 0x7f09f003b900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:13.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.370+0000 7f0a037fe700 1 -- 172.21.15.179:0/3470994933 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f09fc04c210 con 0x7f0a0c198530 2024-04-21T08:30:13.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.370+0000 7f0a14433700 1 -- 172.21.15.179:0/3470994933 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0a0c06b530 con 0x7f0a0c198530 2024-04-21T08:30:13.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.370+0000 7f0a121cf700 1 --2- 172.21.15.179:0/3470994933 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f09f0039440 0x7f09f003b900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:13.371 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.370+0000 7f0a121cf700 1 --2- 172.21.15.179:0/3470994933 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f09f0039440 0x7f09f003b900 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0a0800fa60 tx=0x7f0a08005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:30:13.374 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.375+0000 7f0a037fe700 1 -- 172.21.15.179:0/3470994933 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f09fc01f020 con 0x7f0a0c198530 2024-04-21T08:30:13.467 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:13 smithi179 ceph-mon[31829]: Deploying daemon mon.b on smithi088 2024-04-21T08:30:13.467 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:13 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:30:13.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:13 smithi086 ceph-mon[27939]: Deploying daemon mon.b on smithi088 2024-04-21T08:30:13.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:13 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:30:13.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.663+0000 7f0a14433700 1 -- 172.21.15.179:0/3470994933 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f0a0c19d670 con 0x7f0a0c198530 2024-04-21T08:30:13.663 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.664+0000 7f0a037fe700 1 -- 172.21.15.179:0/3470994933 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 2 v2) ==== 95+0+1015 (secure 0 0 0) 0x7f09fc018570 con 0x7f0a0c198530 2024-04-21T08:30:13.663 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-21T08:30:13.663 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":2,"fsid":"f8cec7cc-ffb8-11ee-bc93-c7b262605968","modified":"2024-04-21T08:30:07.051236Z","created":"2024-04-21T08:28:19.891280Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:3300","nonce":0},{"type":"v1","addr":"172.21.15.86:6789","nonce":0}]},"addr":"172.21.15.86:6789/0","public_addr":"172.21.15.86:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-21T08:30:13.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.666+0000 7f0a14433700 1 -- 172.21.15.179:0/3470994933 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f09f0039440 msgr2=0x7f09f003b900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:13.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.666+0000 7f0a14433700 1 --2- 172.21.15.179:0/3470994933 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f09f0039440 0x7f09f003b900 secure :-1 s=READY pgs=28 cs=0 l=1 rev1=1 crypto rx=0x7f0a0800fa60 tx=0x7f0a08005960 comp rx=0 tx=0).stop 2024-04-21T08:30:13.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.666+0000 7f0a14433700 1 -- 172.21.15.179:0/3470994933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a0c198530 msgr2=0x7f0a0c19c8c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:13.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.666+0000 7f0a14433700 1 --2- 172.21.15.179:0/3470994933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a0c198530 0x7f0a0c19c8c0 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7f09fc00b4d0 tx=0x7f09fc00b890 comp rx=0 tx=0).stop 2024-04-21T08:30:13.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.666+0000 7f0a14433700 1 -- 172.21.15.179:0/3470994933 shutdown_connections 2024-04-21T08:30:13.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.666+0000 7f0a14433700 1 --2- 172.21.15.179:0/3470994933 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f09f0039440 0x7f09f003b900 unknown :-1 s=CLOSED pgs=28 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:13.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.666+0000 7f0a14433700 1 --2- 172.21.15.179:0/3470994933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f09f80035c0 0x7f0a0c197ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:13.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.666+0000 7f0a14433700 1 --2- 172.21.15.179:0/3470994933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a0c198530 0x7f0a0c19c8c0 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:13.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.666+0000 7f0a14433700 1 -- 172.21.15.179:0/3470994933 >> 172.21.15.179:0/3470994933 conn(0x7f0a0c0fcf40 msgr2=0x7f0a0c0fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:30:13.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.666+0000 7f0a14433700 1 -- 172.21.15.179:0/3470994933 shutdown_connections 2024-04-21T08:30:13.666 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:13.666+0000 7f0a14433700 1 -- 172.21.15.179:0/3470994933 wait complete. 2024-04-21T08:30:13.667 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 2 2024-04-21T08:30:14.399 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:14 smithi179 ceph-mon[31829]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:14.400 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:14 smithi179 ceph-mon[31829]: from='client.? 172.21.15.179:0/3470994933' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-21T08:30:14.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:14 smithi086 ceph-mon[27939]: pgmap v19: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:14.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:14 smithi086 ceph-mon[27939]: from='client.? 172.21.15.179:0/3470994933' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-21T08:30:15.429 INFO:tasks.cephadm:Waiting for 3 mons in monmap... 2024-04-21T08:30:15.430 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph mon dump -f json 2024-04-21T08:30:15.673 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:30:16.200 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:16 smithi088 ceph-mon[31259]: mon.b@-1(synchronizing).paxosservice(auth 1..3) refresh upgraded, format 0 -> 3 2024-04-21T08:30:16.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:16 smithi088 bash[31240]: 2386b428da3ad25c981b7b09b63b5f6c3723d6ceb7033ddf1a08130eaa8016f1 2024-04-21T08:30:16.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:16 smithi088 systemd[1]: Started Ceph mon.b for f8cec7cc-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:30:21.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.466+0000 7f5edd08a700 1 -- 172.21.15.179:0/819506351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ec00035c0 msgr2=0x7f5ec0005a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:21.471 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.466+0000 7f5edd08a700 1 --2- 172.21.15.179:0/819506351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ec00035c0 0x7f5ec0005a50 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f5ed8101360 tx=0x7f5ecc00fe60 comp rx=0 tx=0).stop 2024-04-21T08:30:21.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.467+0000 7f5edd08a700 1 -- 172.21.15.179:0/819506351 shutdown_connections 2024-04-21T08:30:21.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.467+0000 7f5edd08a700 1 --2- 172.21.15.179:0/819506351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ec00035c0 0x7f5ec0005a50 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:21.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.467+0000 7f5edd08a700 1 -- 172.21.15.179:0/819506351 >> 172.21.15.179:0/819506351 conn(0x7f5ed80fcf20 msgr2=0x7f5ed80ff340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:30:21.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.467+0000 7f5edd08a700 1 -- 172.21.15.179:0/819506351 shutdown_connections 2024-04-21T08:30:21.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.467+0000 7f5edd08a700 1 -- 172.21.15.179:0/819506351 wait complete. 2024-04-21T08:30:21.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.468+0000 7f5edd08a700 1 Processor -- start 2024-04-21T08:30:21.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.468+0000 7f5edd08a700 1 -- start start 2024-04-21T08:30:21.472 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.468+0000 7f5edd08a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ec00035c0 0x7f5ed8103090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:21.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.468+0000 7f5edd08a700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ed81035d0 0x7f5ed8103a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:21.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.468+0000 7f5edd08a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ed81a42a0 0x7f5ed81a6690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:21.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.468+0000 7f5edd08a700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed8109340 con 0x7f5ed81035d0 2024-04-21T08:30:21.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.468+0000 7f5edd08a700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed81091c0 con 0x7f5ec00035c0 2024-04-21T08:30:21.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.468+0000 7f5edd08a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5ed81094c0 con 0x7f5ed81a42a0 2024-04-21T08:30:21.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.468+0000 7f5ed759e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ed81a42a0 0x7f5ed81a6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:21.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.468+0000 7f5ed759e700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ed81a42a0 0x7f5ed81a6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:54202/0 (socket says 172.21.15.179:54202) 2024-04-21T08:30:21.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.468+0000 7f5ed659c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ed81035d0 0x7f5ed8103a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:21.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.468+0000 7f5ed759e700 1 -- 172.21.15.179:0/3151204746 learned_addr learned my addr 172.21.15.179:0/3151204746 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-21T08:30:21.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.469+0000 7f5ed6d9d700 1 --2- 172.21.15.179:0/3151204746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ec00035c0 0x7f5ed8103090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:21.473 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.469+0000 7f5ed759e700 1 -- 172.21.15.179:0/3151204746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ed81a42a0 msgr2=0x7f5ed81a6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 14 2024-04-21T08:30:21.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.469+0000 7f5ed759e700 1 -- 172.21.15.179:0/3151204746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ed81a42a0 msgr2=0x7f5ed81a6690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-21T08:30:21.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.469+0000 7f5ed759e700 1 --2- 172.21.15.179:0/3151204746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ed81a42a0 0x7f5ed81a6690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-21T08:30:21.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.469+0000 7f5ed759e700 1 --2- 172.21.15.179:0/3151204746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ed81a42a0 0x7f5ed81a6690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:30:21.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.469+0000 7f5ed6d9d700 1 -- 172.21.15.179:0/3151204746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ec00035c0 msgr2=0x7f5ed8103090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_bulk peer close file descriptor 12 2024-04-21T08:30:21.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.469+0000 7f5ed6d9d700 1 -- 172.21.15.179:0/3151204746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ec00035c0 msgr2=0x7f5ed8103090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until read failed 2024-04-21T08:30:21.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.469+0000 7f5ed6d9d700 1 --2- 172.21.15.179:0/3151204746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ec00035c0 0x7f5ed8103090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_read_frame_preamble_main read frame preamble failed r=-1 2024-04-21T08:30:21.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.469+0000 7f5ed6d9d700 1 --2- 172.21.15.179:0/3151204746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ec00035c0 0x7f5ed8103090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:30:21.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.469+0000 7f5ed659c700 1 -- 172.21.15.179:0/3151204746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ed81a42a0 msgr2=0x7f5ed81a6690 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:30:21.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.469+0000 7f5ed659c700 1 --2- 172.21.15.179:0/3151204746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ed81a42a0 0x7f5ed81a6690 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:21.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.469+0000 7f5ed659c700 1 -- 172.21.15.179:0/3151204746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ec00035c0 msgr2=0x7f5ed8103090 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:30:21.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.469+0000 7f5ed659c700 1 --2- 172.21.15.179:0/3151204746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ec00035c0 0x7f5ed8103090 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:21.474 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.469+0000 7f5ed659c700 1 -- 172.21.15.179:0/3151204746 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ed81a6dc0 con 0x7f5ed81035d0 2024-04-21T08:30:21.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.469+0000 7f5ed659c700 1 --2- 172.21.15.179:0/3151204746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ed81035d0 0x7f5ed8103a50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5ec800b550 tx=0x7f5ec800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:30:21.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.469+0000 7f5ebffff700 1 -- 172.21.15.179:0/3151204746 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec8004230 con 0x7f5ed81035d0 2024-04-21T08:30:21.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.469+0000 7f5ebffff700 1 -- 172.21.15.179:0/3151204746 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ec80043d0 con 0x7f5ed81035d0 2024-04-21T08:30:21.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.469+0000 7f5edd08a700 1 -- 172.21.15.179:0/3151204746 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ed81a70b0 con 0x7f5ed81035d0 2024-04-21T08:30:21.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.469+0000 7f5ebffff700 1 -- 172.21.15.179:0/3151204746 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ec8011b90 con 0x7f5ed81035d0 2024-04-21T08:30:21.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.470+0000 7f5edd08a700 1 -- 172.21.15.179:0/3151204746 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ed81a7640 con 0x7f5ed81035d0 2024-04-21T08:30:21.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.470+0000 7f5ebffff700 1 -- 172.21.15.179:0/3151204746 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f5ec801a400 con 0x7f5ed81035d0 2024-04-21T08:30:21.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.471+0000 7f5edd08a700 1 -- 172.21.15.179:0/3151204746 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ed80640a0 con 0x7f5ed81035d0 2024-04-21T08:30:21.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.471+0000 7f5ebffff700 1 --2- 172.21.15.179:0/3151204746 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f5ec4039ae0 0x7f5ec403bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:21.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.471+0000 7f5ebffff700 1 -- 172.21.15.179:0/3151204746 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f5ec804c350 con 0x7f5ed81035d0 2024-04-21T08:30:21.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.471+0000 7f5ed6d9d700 1 --2- 172.21.15.179:0/3151204746 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f5ec4039ae0 0x7f5ec403bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:21.475 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.471+0000 7f5ed6d9d700 1 --2- 172.21.15.179:0/3151204746 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f5ec4039ae0 0x7f5ec403bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc00fa10 tx=0x7f5ecc005a90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:30:21.476 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.476+0000 7f5ebffff700 1 -- 172.21.15.179:0/3151204746 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5ec80151c0 con 0x7f5ed81035d0 2024-04-21T08:30:21.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:21 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:30:21.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:21 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:21.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:21 smithi179 ceph-mon[31829]: mon.a calling monitor election 2024-04-21T08:30:21.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:21 smithi179 ceph-mon[31829]: mon.c calling monitor election 2024-04-21T08:30:21.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:21 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:30:21.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:21 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:21.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:21 smithi179 ceph-mon[31829]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:21.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:21 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:21.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:21 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:21.499 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:21 smithi179 ceph-mon[31829]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:21.499 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:21 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:21.499 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:21 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:21.499 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:21 smithi179 ceph-mon[31829]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-21T08:30:21.499 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:21 smithi179 ceph-mon[31829]: monmap epoch 3 2024-04-21T08:30:21.499 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:21 smithi179 ceph-mon[31829]: fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:30:21.499 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:21 smithi179 ceph-mon[31829]: last_changed 2024-04-21T08:30:16.180543+0000 2024-04-21T08:30:21.500 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:21 smithi179 ceph-mon[31829]: created 2024-04-21T08:28:19.891280+0000 2024-04-21T08:30:21.500 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:21 smithi179 ceph-mon[31829]: min_mon_release 19 (squid) 2024-04-21T08:30:21.500 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:21 smithi179 ceph-mon[31829]: election_strategy: 1 2024-04-21T08:30:21.500 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:21 smithi179 ceph-mon[31829]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-21T08:30:21.500 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:21 smithi179 ceph-mon[31829]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.c 2024-04-21T08:30:21.500 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:21 smithi179 ceph-mon[31829]: 2: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.b 2024-04-21T08:30:21.500 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:21 smithi179 ceph-mon[31829]: fsmap 2024-04-21T08:30:21.500 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:21 smithi179 ceph-mon[31829]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T08:30:21.501 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:21 smithi179 ceph-mon[31829]: mgrmap e13: x(active, since 57s) 2024-04-21T08:30:21.501 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:21 smithi179 ceph-mon[31829]: overall HEALTH_OK 2024-04-21T08:30:21.501 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:21 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:21.501 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:21 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:21.501 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:21 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:21.501 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:21 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:21.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:21 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:30:21.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:21 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:21.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:21 smithi086 ceph-mon[27939]: mon.a calling monitor election 2024-04-21T08:30:21.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:21 smithi086 ceph-mon[27939]: mon.c calling monitor election 2024-04-21T08:30:21.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:21 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:30:21.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:21 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:21.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:21 smithi086 ceph-mon[27939]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:21.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:21 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:21.619 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:21 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:21.619 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:21 smithi086 ceph-mon[27939]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:21.619 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:21 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:21.619 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:21 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:21.619 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:21 smithi086 ceph-mon[27939]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-21T08:30:21.620 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:21 smithi086 ceph-mon[27939]: monmap epoch 3 2024-04-21T08:30:21.620 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:21 smithi086 ceph-mon[27939]: fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:30:21.620 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:21 smithi086 ceph-mon[27939]: last_changed 2024-04-21T08:30:16.180543+0000 2024-04-21T08:30:21.620 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:21 smithi086 ceph-mon[27939]: created 2024-04-21T08:28:19.891280+0000 2024-04-21T08:30:21.620 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:21 smithi086 ceph-mon[27939]: min_mon_release 19 (squid) 2024-04-21T08:30:21.620 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:21 smithi086 ceph-mon[27939]: election_strategy: 1 2024-04-21T08:30:21.621 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:21 smithi086 ceph-mon[27939]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-21T08:30:21.621 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:21 smithi086 ceph-mon[27939]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.c 2024-04-21T08:30:21.621 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:21 smithi086 ceph-mon[27939]: 2: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.b 2024-04-21T08:30:21.621 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:21 smithi086 ceph-mon[27939]: fsmap 2024-04-21T08:30:21.621 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:21 smithi086 ceph-mon[27939]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T08:30:21.621 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:21 smithi086 ceph-mon[27939]: mgrmap e13: x(active, since 57s) 2024-04-21T08:30:21.622 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:21 smithi086 ceph-mon[27939]: overall HEALTH_OK 2024-04-21T08:30:21.622 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:21 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:21.622 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:21 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:21.622 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:21 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:21.622 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:21 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:21.766 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.767+0000 7f5edd08a700 1 -- 172.21.15.179:0/3151204746 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mon dump", "format": "json"} v 0) -- 0x7f5ed81a7a40 con 0x7f5ed81035d0 2024-04-21T08:30:21.767 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.767+0000 7f5ebffff700 1 -- 172.21.15.179:0/3151204746 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mon dump", "format": "json"}]=0 dumped monmap epoch 3 v3) ==== 95+0+1279 (secure 0 0 0) 0x7f5ec801a840 con 0x7f5ed81035d0 2024-04-21T08:30:21.767 INFO:teuthology.orchestra.run.smithi179.stdout: 2024-04-21T08:30:21.767 INFO:teuthology.orchestra.run.smithi179.stdout:{"epoch":3,"fsid":"f8cec7cc-ffb8-11ee-bc93-c7b262605968","modified":"2024-04-21T08:30:16.180543Z","created":"2024-04-21T08:28:19.891280Z","min_mon_release":19,"min_mon_release_name":"squid","election_strategy":1,"disallowed_leaders: ":"","stretch_mode":false,"tiebreaker_mon":"","removed_ranks: ":"","features":{"persistent":["kraken","luminous","mimic","osdmap-prune","nautilus","octopus","pacific","elector-pinging","quincy","reef","squid"],"optional":[]},"mons":[{"rank":0,"name":"a","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:3300","nonce":0},{"type":"v1","addr":"172.21.15.86:6789","nonce":0}]},"addr":"172.21.15.86:6789/0","public_addr":"172.21.15.86:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":1,"name":"c","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:3300","nonce":0},{"type":"v1","addr":"172.21.15.179:6789","nonce":0}]},"addr":"172.21.15.179:6789/0","public_addr":"172.21.15.179:6789/0","priority":0,"weight":0,"crush_location":"{}"},{"rank":2,"name":"b","public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:3300","nonce":0},{"type":"v1","addr":"172.21.15.88:6789","nonce":0}]},"addr":"172.21.15.88:6789/0","public_addr":"172.21.15.88:6789/0","priority":0,"weight":0,"crush_location":"{}"}],"quorum":[0,1]} 2024-04-21T08:30:21.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.770+0000 7f5edd08a700 1 -- 172.21.15.179:0/3151204746 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f5ec4039ae0 msgr2=0x7f5ec403bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:21.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.770+0000 7f5edd08a700 1 --2- 172.21.15.179:0/3151204746 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f5ec4039ae0 0x7f5ec403bfa0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f5ecc00fa10 tx=0x7f5ecc005a90 comp rx=0 tx=0).stop 2024-04-21T08:30:21.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.770+0000 7f5edd08a700 1 -- 172.21.15.179:0/3151204746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ed81035d0 msgr2=0x7f5ed8103a50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:21.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.770+0000 7f5edd08a700 1 --2- 172.21.15.179:0/3151204746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ed81035d0 0x7f5ed8103a50 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f5ec800b550 tx=0x7f5ec800b910 comp rx=0 tx=0).stop 2024-04-21T08:30:21.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.770+0000 7f5edd08a700 1 -- 172.21.15.179:0/3151204746 shutdown_connections 2024-04-21T08:30:21.769 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.770+0000 7f5edd08a700 1 --2- 172.21.15.179:0/3151204746 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f5ec4039ae0 0x7f5ec403bfa0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:21.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.770+0000 7f5edd08a700 1 --2- 172.21.15.179:0/3151204746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ed81a42a0 0x7f5ed81a6690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:21.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.770+0000 7f5edd08a700 1 --2- 172.21.15.179:0/3151204746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ec00035c0 0x7f5ed8103090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:21.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.770+0000 7f5edd08a700 1 --2- 172.21.15.179:0/3151204746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ed81035d0 0x7f5ed8103a50 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:21.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.770+0000 7f5edd08a700 1 -- 172.21.15.179:0/3151204746 >> 172.21.15.179:0/3151204746 conn(0x7f5ed80fcf20 msgr2=0x7f5ed81051d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:30:21.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.770+0000 7f5edd08a700 1 -- 172.21.15.179:0/3151204746 shutdown_connections 2024-04-21T08:30:21.770 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:21.770+0000 7f5edd08a700 1 -- 172.21.15.179:0/3151204746 wait complete. 2024-04-21T08:30:21.770 INFO:teuthology.orchestra.run.smithi179.stderr:dumped monmap epoch 3 2024-04-21T08:30:22.365 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:22 smithi179 ceph-mon[31829]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:22.365 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:22 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:22.365 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:22 smithi179 ceph-mon[31829]: from='client.? 172.21.15.179:0/3151204746' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-21T08:30:22.365 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:22 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:22.400 INFO:tasks.cephadm:Generating final ceph.conf file... 2024-04-21T08:30:22.401 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph config generate-minimal-conf 2024-04-21T08:30:22.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:22 smithi086 ceph-mon[27939]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:22.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:22 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:22.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:22 smithi086 ceph-mon[27939]: from='client.? 172.21.15.179:0/3151204746' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-21T08:30:22.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:22 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:22.647 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:30:23.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:23 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:30:23.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:23 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:23.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:23 smithi088 ceph-mon[31259]: mon.a calling monitor election 2024-04-21T08:30:23.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:23 smithi088 ceph-mon[31259]: mon.c calling monitor election 2024-04-21T08:30:23.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:23 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:30:23.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:23 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:23.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:23 smithi088 ceph-mon[31259]: pgmap v21: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:23.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:23 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:23.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:23 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:23.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:23 smithi088 ceph-mon[31259]: pgmap v22: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:23.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:23 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:23.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:23 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:23.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:23 smithi088 ceph-mon[31259]: mon.a is new leader, mons a,c in quorum (ranks 0,1) 2024-04-21T08:30:23.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:23 smithi088 ceph-mon[31259]: monmap epoch 3 2024-04-21T08:30:23.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:23 smithi088 ceph-mon[31259]: fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:30:23.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:23 smithi088 ceph-mon[31259]: last_changed 2024-04-21T08:30:16.180543+0000 2024-04-21T08:30:23.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:23 smithi088 ceph-mon[31259]: created 2024-04-21T08:28:19.891280+0000 2024-04-21T08:30:23.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:23 smithi088 ceph-mon[31259]: min_mon_release 19 (squid) 2024-04-21T08:30:23.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:23 smithi088 ceph-mon[31259]: election_strategy: 1 2024-04-21T08:30:23.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:23 smithi088 ceph-mon[31259]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-21T08:30:23.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:23 smithi088 ceph-mon[31259]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.c 2024-04-21T08:30:23.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:23 smithi088 ceph-mon[31259]: 2: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.b 2024-04-21T08:30:23.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:23 smithi088 ceph-mon[31259]: fsmap 2024-04-21T08:30:23.466 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:23 smithi088 ceph-mon[31259]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T08:30:23.466 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:23 smithi088 ceph-mon[31259]: mgrmap e13: x(active, since 57s) 2024-04-21T08:30:23.466 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:23 smithi088 ceph-mon[31259]: overall HEALTH_OK 2024-04-21T08:30:23.466 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:23 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:23.466 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:23 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:23.466 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:23 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:23.466 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:23 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:23.466 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:23 smithi088 ceph-mon[31259]: pgmap v23: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:23.466 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:23 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:23.466 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:23 smithi088 ceph-mon[31259]: from='client.? 172.21.15.179:0/3151204746' entity='client.admin' cmd=[{"prefix": "mon dump", "format": "json"}]: dispatch 2024-04-21T08:30:23.466 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:23 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:24.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.266+0000 7fcf33cd3700 1 -- 172.21.15.86:0/277450639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf2c1015c0 msgr2=0x7fcf2c1019a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:24.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.266+0000 7fcf33cd3700 1 --2- 172.21.15.86:0/277450639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf2c1015c0 0x7fcf2c1019a0 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7fcf28009910 tx=0x7fcf28009c20 comp rx=0 tx=0).stop 2024-04-21T08:30:24.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.266+0000 7fcf33cd3700 1 -- 172.21.15.86:0/277450639 shutdown_connections 2024-04-21T08:30:24.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.266+0000 7fcf33cd3700 1 --2- 172.21.15.86:0/277450639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf2c1015c0 0x7fcf2c1019a0 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:24.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.266+0000 7fcf33cd3700 1 -- 172.21.15.86:0/277450639 >> 172.21.15.86:0/277450639 conn(0x7fcf2c0fce90 msgr2=0x7fcf2c0ff2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:30:24.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.266+0000 7fcf33cd3700 1 -- 172.21.15.86:0/277450639 shutdown_connections 2024-04-21T08:30:24.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.266+0000 7fcf33cd3700 1 -- 172.21.15.86:0/277450639 wait complete. 2024-04-21T08:30:24.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.267+0000 7fcf33cd3700 1 Processor -- start 2024-04-21T08:30:24.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.267+0000 7fcf33cd3700 1 -- start start 2024-04-21T08:30:24.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.267+0000 7fcf33cd3700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf2c1015c0 0x7fcf2c19a330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:24.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.267+0000 7fcf33cd3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf2c19a870 0x7fcf2c19ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:24.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.267+0000 7fcf33cd3700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf2c19f0f0 0x7fcf2c19f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:24.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.267+0000 7fcf33cd3700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c10b840 con 0x7fcf2c1015c0 2024-04-21T08:30:24.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.267+0000 7fcf33cd3700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c10b6c0 con 0x7fcf2c19f0f0 2024-04-21T08:30:24.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.267+0000 7fcf33cd3700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fcf2c10b9c0 con 0x7fcf2c19a870 2024-04-21T08:30:24.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.267+0000 7fcf31a6f700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf2c1015c0 0x7fcf2c19a330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:24.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.267+0000 7fcf31a6f700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf2c1015c0 0x7fcf2c19a330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46532/0 (socket says 172.21.15.86:46532) 2024-04-21T08:30:24.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.267+0000 7fcf31a6f700 1 -- 172.21.15.86:0/1496337058 learned_addr learned my addr 172.21.15.86:0/1496337058 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:30:24.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.267+0000 7fcf3126e700 1 --2- 172.21.15.86:0/1496337058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf2c19a870 0x7fcf2c19ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:24.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.267+0000 7fcf32270700 1 --2- 172.21.15.86:0/1496337058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf2c19f0f0 0x7fcf2c19f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:24.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.268+0000 7fcf31a6f700 1 -- 172.21.15.86:0/1496337058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf2c19a870 msgr2=0x7fcf2c19ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:24.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.268+0000 7fcf31a6f700 1 --2- 172.21.15.86:0/1496337058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf2c19a870 0x7fcf2c19ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:24.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.268+0000 7fcf31a6f700 1 -- 172.21.15.86:0/1496337058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf2c19f0f0 msgr2=0x7fcf2c19f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:24.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.268+0000 7fcf31a6f700 1 --2- 172.21.15.86:0/1496337058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf2c19f0f0 0x7fcf2c19f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:24.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.268+0000 7fcf31a6f700 1 -- 172.21.15.86:0/1496337058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf2c1a6e60 con 0x7fcf2c1015c0 2024-04-21T08:30:24.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.268+0000 7fcf32270700 1 --2- 172.21.15.86:0/1496337058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf2c19f0f0 0x7fcf2c19f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:30:24.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.268+0000 7fcf31a6f700 1 --2- 172.21.15.86:0/1496337058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf2c1015c0 0x7fcf2c19a330 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcf28004f80 tx=0x7fcf28004fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:30:24.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.268+0000 7fcf22ffd700 1 -- 172.21.15.86:0/1496337058 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf28026070 con 0x7fcf2c1015c0 2024-04-21T08:30:24.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.268+0000 7fcf33cd3700 1 -- 172.21.15.86:0/1496337058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf2c1a7090 con 0x7fcf2c1015c0 2024-04-21T08:30:24.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.268+0000 7fcf22ffd700 1 -- 172.21.15.86:0/1496337058 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf2800b6f0 con 0x7fcf2c1015c0 2024-04-21T08:30:24.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.268+0000 7fcf22ffd700 1 -- 172.21.15.86:0/1496337058 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf280038e0 con 0x7fcf2c1015c0 2024-04-21T08:30:24.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.268+0000 7fcf33cd3700 1 -- 172.21.15.86:0/1496337058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf2c1a7620 con 0x7fcf2c1015c0 2024-04-21T08:30:24.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.269+0000 7fcf22ffd700 1 -- 172.21.15.86:0/1496337058 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7fcf28029070 con 0x7fcf2c1015c0 2024-04-21T08:30:24.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.269+0000 7fcf22ffd700 1 --2- 172.21.15.86:0/1496337058 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fcf18039ae0 0x7fcf1803bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:24.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.269+0000 7fcf22ffd700 1 -- 172.21.15.86:0/1496337058 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7fcf2804e3f0 con 0x7fcf2c1015c0 2024-04-21T08:30:24.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.269+0000 7fcf3126e700 1 --2- 172.21.15.86:0/1496337058 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fcf18039ae0 0x7fcf1803bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:24.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.269+0000 7fcf33cd3700 1 -- 172.21.15.86:0/1496337058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf2c0640a0 con 0x7fcf2c1015c0 2024-04-21T08:30:24.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.270+0000 7fcf3126e700 1 --2- 172.21.15.86:0/1496337058 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fcf18039ae0 0x7fcf1803bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c009ad0 tx=0x7fcf1c009210 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:30:24.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.275+0000 7fcf22ffd700 1 -- 172.21.15.86:0/1496337058 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcf28030070 con 0x7fcf2c1015c0 2024-04-21T08:30:24.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: mon.b calling monitor election 2024-04-21T08:30:24.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: Updating smithi086:/etc/ceph/ceph.conf 2024-04-21T08:30:24.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: Updating smithi088:/etc/ceph/ceph.conf 2024-04-21T08:30:24.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: Updating smithi179:/etc/ceph/ceph.conf 2024-04-21T08:30:24.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: mon.b calling monitor election 2024-04-21T08:30:24.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: mon.c calling monitor election 2024-04-21T08:30:24.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: mon.a calling monitor election 2024-04-21T08:30:24.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-21T08:30:24.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: monmap epoch 3 2024-04-21T08:30:24.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:30:24.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: last_changed 2024-04-21T08:30:16.180543+0000 2024-04-21T08:30:24.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: created 2024-04-21T08:28:19.891280+0000 2024-04-21T08:30:24.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: min_mon_release 19 (squid) 2024-04-21T08:30:24.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: election_strategy: 1 2024-04-21T08:30:24.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-21T08:30:24.369 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.c 2024-04-21T08:30:24.369 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: 2: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.b 2024-04-21T08:30:24.369 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: fsmap 2024-04-21T08:30:24.369 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:24.369 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T08:30:24.369 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: mgrmap e13: x(active, since 60s) 2024-04-21T08:30:24.369 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: overall HEALTH_OK 2024-04-21T08:30:24.369 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:24.369 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: Updating smithi086:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:30:24.369 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: Updating smithi179:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:30:24.369 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: Updating smithi088:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:30:24.369 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:24.369 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:24.370 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:24.370 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:24.370 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:24.370 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:24.370 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:24.370 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:24.370 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:24.370 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:24.370 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:24.370 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:30:24.370 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T08:30:24.370 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:24 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:24.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: mon.b calling monitor election 2024-04-21T08:30:24.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: Updating smithi086:/etc/ceph/ceph.conf 2024-04-21T08:30:24.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: Updating smithi088:/etc/ceph/ceph.conf 2024-04-21T08:30:24.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: Updating smithi179:/etc/ceph/ceph.conf 2024-04-21T08:30:24.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: mon.b calling monitor election 2024-04-21T08:30:24.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: mon.c calling monitor election 2024-04-21T08:30:24.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: mon.a calling monitor election 2024-04-21T08:30:24.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-21T08:30:24.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: monmap epoch 3 2024-04-21T08:30:24.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:30:24.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: last_changed 2024-04-21T08:30:16.180543+0000 2024-04-21T08:30:24.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: created 2024-04-21T08:28:19.891280+0000 2024-04-21T08:30:24.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: min_mon_release 19 (squid) 2024-04-21T08:30:24.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: election_strategy: 1 2024-04-21T08:30:24.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-21T08:30:24.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.c 2024-04-21T08:30:24.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: 2: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.b 2024-04-21T08:30:24.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: fsmap 2024-04-21T08:30:24.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:24.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T08:30:24.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: mgrmap e13: x(active, since 60s) 2024-04-21T08:30:24.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: overall HEALTH_OK 2024-04-21T08:30:24.466 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:24.466 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: Updating smithi086:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:30:24.466 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: Updating smithi179:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:30:24.466 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: Updating smithi088:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:30:24.466 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:24.466 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:24.466 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:24.466 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:24.466 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:24.466 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:24.466 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:24.466 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:24.466 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:24.467 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:24.467 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:24.467 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:30:24.467 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T08:30:24.467 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:24 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:24.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.481+0000 7fcf33cd3700 1 -- 172.21.15.86:0/1496337058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "config generate-minimal-conf"} v 0) -- 0x7fcf2c1a7880 con 0x7fcf2c1015c0 2024-04-21T08:30:24.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.481+0000 7fcf22ffd700 1 -- 172.21.15.86:0/1496337058 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "config generate-minimal-conf"}]=0 v9) ==== 76+0+273 (secure 0 0 0) 0x7fcf28013070 con 0x7fcf2c1015c0 2024-04-21T08:30:24.481 INFO:teuthology.orchestra.run.smithi086.stdout:# minimal ceph.conf for f8cec7cc-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:30:24.481 INFO:teuthology.orchestra.run.smithi086.stdout:[global] 2024-04-21T08:30:24.481 INFO:teuthology.orchestra.run.smithi086.stdout: fsid = f8cec7cc-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:30:24.481 INFO:teuthology.orchestra.run.smithi086.stdout: mon_host = [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] 2024-04-21T08:30:24.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.483+0000 7fcf33cd3700 1 -- 172.21.15.86:0/1496337058 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fcf18039ae0 msgr2=0x7fcf1803bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:24.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.483+0000 7fcf33cd3700 1 --2- 172.21.15.86:0/1496337058 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fcf18039ae0 0x7fcf1803bfa0 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7fcf1c009ad0 tx=0x7fcf1c009210 comp rx=0 tx=0).stop 2024-04-21T08:30:24.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.483+0000 7fcf33cd3700 1 -- 172.21.15.86:0/1496337058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf2c1015c0 msgr2=0x7fcf2c19a330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:24.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.483+0000 7fcf33cd3700 1 --2- 172.21.15.86:0/1496337058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf2c1015c0 0x7fcf2c19a330 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7fcf28004f80 tx=0x7fcf28004fb0 comp rx=0 tx=0).stop 2024-04-21T08:30:24.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.483+0000 7fcf33cd3700 1 -- 172.21.15.86:0/1496337058 shutdown_connections 2024-04-21T08:30:24.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.483+0000 7fcf33cd3700 1 --2- 172.21.15.86:0/1496337058 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fcf18039ae0 0x7fcf1803bfa0 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:24.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.483+0000 7fcf33cd3700 1 --2- 172.21.15.86:0/1496337058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf2c19f0f0 0x7fcf2c19f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:24.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.483+0000 7fcf33cd3700 1 --2- 172.21.15.86:0/1496337058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf2c1015c0 0x7fcf2c19a330 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:24.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.483+0000 7fcf33cd3700 1 --2- 172.21.15.86:0/1496337058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf2c19a870 0x7fcf2c19ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:24.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.483+0000 7fcf33cd3700 1 -- 172.21.15.86:0/1496337058 >> 172.21.15.86:0/1496337058 conn(0x7fcf2c0fce90 msgr2=0x7fcf2c104fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:30:24.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.483+0000 7fcf33cd3700 1 -- 172.21.15.86:0/1496337058 shutdown_connections 2024-04-21T08:30:24.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:24.484+0000 7fcf33cd3700 1 -- 172.21.15.86:0/1496337058 wait complete. 2024-04-21T08:30:24.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: mon.b calling monitor election 2024-04-21T08:30:24.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: Updating smithi086:/etc/ceph/ceph.conf 2024-04-21T08:30:24.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: Updating smithi088:/etc/ceph/ceph.conf 2024-04-21T08:30:24.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: Updating smithi179:/etc/ceph/ceph.conf 2024-04-21T08:30:24.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: mon.b calling monitor election 2024-04-21T08:30:24.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: mon.c calling monitor election 2024-04-21T08:30:24.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: mon.a calling monitor election 2024-04-21T08:30:24.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: mon.a is new leader, mons a,c,b in quorum (ranks 0,1,2) 2024-04-21T08:30:24.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: monmap epoch 3 2024-04-21T08:30:24.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:30:24.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: last_changed 2024-04-21T08:30:16.180543+0000 2024-04-21T08:30:24.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: created 2024-04-21T08:28:19.891280+0000 2024-04-21T08:30:24.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: min_mon_release 19 (squid) 2024-04-21T08:30:24.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: election_strategy: 1 2024-04-21T08:30:24.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: 0: [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] mon.a 2024-04-21T08:30:24.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: 1: [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] mon.c 2024-04-21T08:30:24.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: 2: [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] mon.b 2024-04-21T08:30:24.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: fsmap 2024-04-21T08:30:24.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: pgmap v24: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:24.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: osdmap e4: 0 total, 0 up, 0 in 2024-04-21T08:30:24.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: mgrmap e13: x(active, since 60s) 2024-04-21T08:30:24.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: overall HEALTH_OK 2024-04-21T08:30:24.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:24.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: Updating smithi086:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:30:24.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: Updating smithi179:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:30:24.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: Updating smithi088:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:30:24.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:24.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:24.499 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:24.499 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:24.499 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:24.499 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:24.499 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:24.499 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:24.499 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:24.499 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:24.499 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:24.499 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:30:24.499 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T08:30:24.499 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:24 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:25.116 INFO:tasks.cephadm:Distributing (final) config and client.admin keyring... 2024-04-21T08:30:25.116 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-21T08:30:25.116 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.conf 2024-04-21T08:30:25.142 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-21T08:30:25.143 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:30:25.206 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-21T08:30:25.206 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/ceph/ceph.conf 2024-04-21T08:30:25.232 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-21T08:30:25.232 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:30:25.296 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-21T08:30:25.296 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/ceph/ceph.conf 2024-04-21T08:30:25.327 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-21T08:30:25.327 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd of=/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:30:25.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:25 smithi086 ceph-mon[27939]: Reconfiguring mon.a (unknown last config time)... 2024-04-21T08:30:25.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:25 smithi086 ceph-mon[27939]: Reconfiguring daemon mon.a on smithi086 2024-04-21T08:30:25.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:25 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:25.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:25 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1496337058' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:25.390 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:25 smithi179 ceph-mon[31829]: Reconfiguring mon.a (unknown last config time)... 2024-04-21T08:30:25.390 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:25 smithi179 ceph-mon[31829]: Reconfiguring daemon mon.a on smithi086 2024-04-21T08:30:25.390 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:25 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:25.390 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:25 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1496337058' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:25.394 INFO:tasks.cephadm:Adding mgr.x on smithi086 2024-04-21T08:30:25.394 INFO:tasks.cephadm:Adding mgr.y on smithi088 2024-04-21T08:30:25.394 INFO:tasks.cephadm:Adding mgr.z on smithi179 2024-04-21T08:30:25.394 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph orch apply mgr '3;smithi086=x;smithi088=y;smithi179=z' 2024-04-21T08:30:25.463 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:25 smithi088 ceph-mon[31259]: Reconfiguring mon.a (unknown last config time)... 2024-04-21T08:30:25.463 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:25 smithi088 ceph-mon[31259]: Reconfiguring daemon mon.a on smithi086 2024-04-21T08:30:25.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:25 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:30:25.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:25 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1496337058' entity='client.admin' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:25.668 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:30:26.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:26 smithi086 ceph-mon[27939]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:26.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:26 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:26.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:26 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:26.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:26 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:30:26.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:26 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T08:30:26.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:26 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:26.463 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:26 smithi088 ceph-mon[31259]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:26.463 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:26 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:26.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:26 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:26.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:26 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:30:26.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:26 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T08:30:26.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:26 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:26.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:26 smithi179 ceph-mon[31829]: pgmap v25: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:26.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:26 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:26.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:26 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:26.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:26 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:30:26.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:26 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T08:30:26.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:26 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:27.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.172+0000 7f54a84fa700 1 -- 172.21.15.179:0/3676802803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54a01014e0 msgr2=0x7f54a0101900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:27.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.172+0000 7f54a84fa700 1 --2- 172.21.15.179:0/3676802803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54a01014e0 0x7f54a0101900 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7f5490009910 tx=0x7f5490009c20 comp rx=0 tx=0).stop 2024-04-21T08:30:27.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.173+0000 7f54a84fa700 1 -- 172.21.15.179:0/3676802803 shutdown_connections 2024-04-21T08:30:27.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.173+0000 7f54a84fa700 1 --2- 172.21.15.179:0/3676802803 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54a01014e0 0x7f54a0101900 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:27.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.173+0000 7f54a84fa700 1 -- 172.21.15.179:0/3676802803 >> 172.21.15.179:0/3676802803 conn(0x7f54a00fca80 msgr2=0x7f54a00feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:30:27.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.173+0000 7f54a84fa700 1 -- 172.21.15.179:0/3676802803 shutdown_connections 2024-04-21T08:30:27.174 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.173+0000 7f54a84fa700 1 -- 172.21.15.179:0/3676802803 wait complete. 2024-04-21T08:30:27.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.173+0000 7f54a84fa700 1 Processor -- start 2024-04-21T08:30:27.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.173+0000 7f54a84fa700 1 -- start start 2024-04-21T08:30:27.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.173+0000 7f54a84fa700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54a01014e0 0x7f54a0197d90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:27.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.173+0000 7f54a84fa700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54a01982d0 0x7f54a019d300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:27.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.173+0000 7f54a84fa700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54a0198750 0x7f54a019f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:27.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.173+0000 7f54a84fa700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f54a0107420 con 0x7f54a01014e0 2024-04-21T08:30:27.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.173+0000 7f54a84fa700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f54a01072a0 con 0x7f54a0198750 2024-04-21T08:30:27.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.173+0000 7f54a84fa700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f54a01075a0 con 0x7f54a01982d0 2024-04-21T08:30:27.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.173+0000 7f54a5a95700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54a01982d0 0x7f54a019d300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:27.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.173+0000 7f54a6296700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54a01014e0 0x7f54a0197d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:27.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.174+0000 7f54a5a95700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54a01982d0 0x7f54a019d300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:37188/0 (socket says 172.21.15.179:37188) 2024-04-21T08:30:27.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.174+0000 7f54a5a95700 1 -- 172.21.15.179:0/685431751 learned_addr learned my addr 172.21.15.179:0/685431751 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-21T08:30:27.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.174+0000 7f54a6a97700 1 --2- 172.21.15.179:0/685431751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54a0198750 0x7f54a019f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:27.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.174+0000 7f54a5a95700 1 -- 172.21.15.179:0/685431751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54a0198750 msgr2=0x7f54a019f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:27.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.174+0000 7f54a5a95700 1 --2- 172.21.15.179:0/685431751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54a0198750 0x7f54a019f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:27.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.174+0000 7f54a5a95700 1 -- 172.21.15.179:0/685431751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54a01014e0 msgr2=0x7f54a0197d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:27.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.174+0000 7f54a5a95700 1 --2- 172.21.15.179:0/685431751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54a01014e0 0x7f54a0197d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:27.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.174+0000 7f54a5a95700 1 -- 172.21.15.179:0/685431751 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54a01a01b0 con 0x7f54a01982d0 2024-04-21T08:30:27.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.174+0000 7f54a6296700 1 --2- 172.21.15.179:0/685431751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54a01014e0 0x7f54a0197d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:30:27.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.174+0000 7f54a6a97700 1 --2- 172.21.15.179:0/685431751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54a0198750 0x7f54a019f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:30:27.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.174+0000 7f54a5a95700 1 --2- 172.21.15.179:0/685431751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54a01982d0 0x7f54a019d300 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f549c00b780 tx=0x7f549c00ba90 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:30:27.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.174+0000 7f54977fe700 1 -- 172.21.15.179:0/685431751 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c004230 con 0x7f54a01982d0 2024-04-21T08:30:27.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.174+0000 7f54977fe700 1 -- 172.21.15.179:0/685431751 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f549c0043d0 con 0x7f54a01982d0 2024-04-21T08:30:27.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.174+0000 7f54a84fa700 1 -- 172.21.15.179:0/685431751 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54a01a04a0 con 0x7f54a01982d0 2024-04-21T08:30:27.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.174+0000 7f54977fe700 1 -- 172.21.15.179:0/685431751 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f549c011b20 con 0x7f54a01982d0 2024-04-21T08:30:27.176 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.174+0000 7f54a84fa700 1 -- 172.21.15.179:0/685431751 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54a01a0a60 con 0x7f54a01982d0 2024-04-21T08:30:27.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.175+0000 7f54977fe700 1 -- 172.21.15.179:0/685431751 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7f549c0036a0 con 0x7f54a01982d0 2024-04-21T08:30:27.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.175+0000 7f54a84fa700 1 -- 172.21.15.179:0/685431751 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54a00640a0 con 0x7f54a01982d0 2024-04-21T08:30:27.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.175+0000 7f54977fe700 1 --2- 172.21.15.179:0/685431751 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f548c039ae0 0x7f548c03bfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:27.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.175+0000 7f54977fe700 1 -- 172.21.15.179:0/685431751 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7f549c04c3b0 con 0x7f54a01982d0 2024-04-21T08:30:27.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.175+0000 7f54a6296700 1 --2- 172.21.15.179:0/685431751 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f548c039ae0 0x7f548c03bfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:27.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.176+0000 7f54a6296700 1 --2- 172.21.15.179:0/685431751 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f548c039ae0 0x7f548c03bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5490006010 tx=0x7f549001a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:30:27.180 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.181+0000 7f54977fe700 1 -- 172.21.15.179:0/685431751 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f549c020910 con 0x7f54a01982d0 2024-04-21T08:30:27.180 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:27 smithi179 ceph-mon[31829]: Reconfiguring mon.b (monmap changed)... 2024-04-21T08:30:27.180 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:27 smithi179 ceph-mon[31829]: Reconfiguring daemon mon.b on smithi088 2024-04-21T08:30:27.390 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.390+0000 7f54a84fa700 1 -- 172.21.15.179:0/685431751 --> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] -- mgr_command(tid 0: {"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi086=x;smithi088=y;smithi179=z", "target": ["mon-mgr", ""]}) -- 0x7f54a0062f10 con 0x7f548c039ae0 2024-04-21T08:30:27.396 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.397+0000 7f54977fe700 1 -- 172.21.15.179:0/685431751 <== mgr.14152 v2:172.21.15.86:6800/1364156965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7f54a0062f10 con 0x7f548c039ae0 2024-04-21T08:30:27.396 INFO:teuthology.orchestra.run.smithi179.stdout:Scheduled mgr update... 2024-04-21T08:30:27.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.399+0000 7f54a84fa700 1 -- 172.21.15.179:0/685431751 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f548c039ae0 msgr2=0x7f548c03bfa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:27.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.399+0000 7f54a84fa700 1 --2- 172.21.15.179:0/685431751 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f548c039ae0 0x7f548c03bfa0 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f5490006010 tx=0x7f549001a040 comp rx=0 tx=0).stop 2024-04-21T08:30:27.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.399+0000 7f54a84fa700 1 -- 172.21.15.179:0/685431751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54a01982d0 msgr2=0x7f54a019d300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:27.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.399+0000 7f54a84fa700 1 --2- 172.21.15.179:0/685431751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54a01982d0 0x7f54a019d300 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7f549c00b780 tx=0x7f549c00ba90 comp rx=0 tx=0).stop 2024-04-21T08:30:27.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.399+0000 7f54a84fa700 1 -- 172.21.15.179:0/685431751 shutdown_connections 2024-04-21T08:30:27.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.399+0000 7f54a84fa700 1 --2- 172.21.15.179:0/685431751 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f548c039ae0 0x7f548c03bfa0 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:27.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.399+0000 7f54a84fa700 1 --2- 172.21.15.179:0/685431751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54a0198750 0x7f54a019f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:27.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.399+0000 7f54a84fa700 1 --2- 172.21.15.179:0/685431751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54a01014e0 0x7f54a0197d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:27.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.399+0000 7f54a84fa700 1 --2- 172.21.15.179:0/685431751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54a01982d0 0x7f54a019d300 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:27.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.399+0000 7f54a84fa700 1 -- 172.21.15.179:0/685431751 >> 172.21.15.179:0/685431751 conn(0x7f54a00fca80 msgr2=0x7f54a018ead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:30:27.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.399+0000 7f54a84fa700 1 -- 172.21.15.179:0/685431751 shutdown_connections 2024-04-21T08:30:27.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:30:27.399+0000 7f54a84fa700 1 -- 172.21.15.179:0/685431751 wait complete. 2024-04-21T08:30:27.427 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:27 smithi088 ceph-mon[31259]: Reconfiguring mon.b (monmap changed)... 2024-04-21T08:30:27.427 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:27 smithi088 ceph-mon[31259]: Reconfiguring daemon mon.b on smithi088 2024-04-21T08:30:27.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:27 smithi086 ceph-mon[27939]: Reconfiguring mon.b (monmap changed)... 2024-04-21T08:30:27.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:27 smithi086 ceph-mon[27939]: Reconfiguring daemon mon.b on smithi088 2024-04-21T08:30:27.952 DEBUG:teuthology.orchestra.run.smithi088:mgr.y> sudo journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mgr.y.service 2024-04-21T08:30:27.955 DEBUG:teuthology.orchestra.run.smithi179:mgr.z> sudo journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mgr.z.service 2024-04-21T08:30:27.957 INFO:tasks.cephadm:Deploying OSDs... 2024-04-21T08:30:27.957 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-21T08:30:27.957 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/scratch_devs of=/dev/stdout 2024-04-21T08:30:27.976 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-21T08:30:27.976 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_1 2024-04-21T08:30:28.035 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-21T08:30:28.035 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:30:28.035 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 664 Links: 1 2024-04-21T08:30:28.035 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:30:28.035 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:30:28.035 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-04-21 08:29:33.065750668 +0000 2024-04-21T08:30:28.035 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-04-21 08:24:38.621387103 +0000 2024-04-21T08:30:28.036 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-04-21 08:24:38.621387103 +0000 2024-04-21T08:30:28.036 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-04-21 08:24:38.621387103 +0000 2024-04-21T08:30:28.036 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-21T08:30:28.103 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-21T08:30:28.103 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-21T08:30:28.103 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000243014 s, 2.1 MB/s 2024-04-21T08:30:28.104 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-21T08:30:28.163 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_2 2024-04-21T08:30:28.220 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-21T08:30:28.220 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:30:28.220 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-21T08:30:28.220 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:30:28.220 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:30:28.221 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-04-21 08:29:33.098749975 +0000 2024-04-21T08:30:28.221 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-04-21 08:24:38.981379443 +0000 2024-04-21T08:30:28.221 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-04-21 08:24:38.981379443 +0000 2024-04-21T08:30:28.221 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-04-21 08:24:38.981379443 +0000 2024-04-21T08:30:28.221 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-21T08:30:28.286 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-21T08:30:28.286 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-21T08:30:28.286 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000239236 s, 2.1 MB/s 2024-04-21T08:30:28.287 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-21T08:30:28.344 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_3 2024-04-21T08:30:28.402 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-21T08:30:28.402 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:30:28.402 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 702 Links: 1 2024-04-21T08:30:28.402 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:30:28.402 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:30:28.402 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-04-21 08:29:33.132749261 +0000 2024-04-21T08:30:28.402 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-04-21 08:24:39.320372229 +0000 2024-04-21T08:30:28.402 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-04-21 08:24:39.320372229 +0000 2024-04-21T08:30:28.402 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-04-21 08:24:39.320372229 +0000 2024-04-21T08:30:28.403 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-21T08:30:28.465 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-21T08:30:28.465 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-21T08:30:28.465 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000206887 s, 2.5 MB/s 2024-04-21T08:30:28.466 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-21T08:30:28.523 DEBUG:teuthology.orchestra.run.smithi086:> stat /dev/vg_nvme/lv_4 2024-04-21T08:30:28.577 INFO:teuthology.orchestra.run.smithi086.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-21T08:30:28.578 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:30:28.578 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 5h/5d Inode: 722 Links: 1 2024-04-21T08:30:28.578 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:30:28.578 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:30:28.578 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-04-21 08:29:33.166748547 +0000 2024-04-21T08:30:28.578 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-04-21 08:24:39.671364760 +0000 2024-04-21T08:30:28.578 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-04-21 08:24:39.671364760 +0000 2024-04-21T08:30:28.578 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-04-21 08:24:39.671364760 +0000 2024-04-21T08:30:28.578 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-21T08:30:28.640 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records in 2024-04-21T08:30:28.640 INFO:teuthology.orchestra.run.smithi086.stderr:1+0 records out 2024-04-21T08:30:28.640 INFO:teuthology.orchestra.run.smithi086.stderr:512 bytes copied, 0.000203375 s, 2.5 MB/s 2024-04-21T08:30:28.642 DEBUG:teuthology.orchestra.run.smithi086:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-21T08:30:28.697 DEBUG:teuthology.orchestra.run.smithi088:> set -ex 2024-04-21T08:30:28.697 DEBUG:teuthology.orchestra.run.smithi088:> dd if=/scratch_devs of=/dev/stdout 2024-04-21T08:30:28.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:28 smithi088 ceph-mon[31259]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:28.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:28 smithi088 ceph-mon[31259]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi086=x;smithi088=y;smithi179=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:30:28.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:28 smithi088 ceph-mon[31259]: Saving service mgr spec with placement smithi086=x;smithi088=y;smithi179=z;count:3 2024-04-21T08:30:28.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:28 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:28.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:28 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:28.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:28 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:28.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:28 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:30:28.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:28 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T08:30:28.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:28 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:28.716 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-21T08:30:28.716 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_1 2024-04-21T08:30:28.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:28 smithi179 ceph-mon[31829]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:28.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:28 smithi179 ceph-mon[31829]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi086=x;smithi088=y;smithi179=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:30:28.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:28 smithi179 ceph-mon[31829]: Saving service mgr spec with placement smithi086=x;smithi088=y;smithi179=z;count:3 2024-04-21T08:30:28.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:28 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:28.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:28 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:28.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:28 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:28.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:28 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:30:28.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:28 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T08:30:28.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:28 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:28.774 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-21T08:30:28.774 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:30:28.774 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 5h/5d Inode: 663 Links: 1 2024-04-21T08:30:28.774 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:30:28.775 INFO:teuthology.orchestra.run.smithi088.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:30:28.775 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2024-04-21 08:30:15.658790740 +0000 2024-04-21T08:30:28.775 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2024-04-21 08:25:00.303367076 +0000 2024-04-21T08:30:28.775 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2024-04-21 08:25:00.303367076 +0000 2024-04-21T08:30:28.775 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: 2024-04-21 08:25:00.303367076 +0000 2024-04-21T08:30:28.775 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-21T08:30:28.840 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-04-21T08:30:28.840 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-04-21T08:30:28.840 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.000287919 s, 1.8 MB/s 2024-04-21T08:30:28.842 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-21T08:30:28.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:28 smithi086 ceph-mon[27939]: pgmap v26: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:28.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:28 smithi086 ceph-mon[27939]: from='client.24103 -' entity='client.admin' cmd=[{"prefix": "orch apply", "service_type": "mgr", "placement": "3;smithi086=x;smithi088=y;smithi179=z", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:30:28.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:28 smithi086 ceph-mon[27939]: Saving service mgr spec with placement smithi086=x;smithi088=y;smithi179=z;count:3 2024-04-21T08:30:28.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:28 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:28.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:28 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:28.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:28 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:28.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:28 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "mon."}]: dispatch 2024-04-21T08:30:28.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:28 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config get", "who": "mon", "key": "public_network"}]: dispatch 2024-04-21T08:30:28.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:28 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:28.900 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_2 2024-04-21T08:30:28.957 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-21T08:30:28.957 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:30:28.957 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 5h/5d Inode: 683 Links: 1 2024-04-21T08:30:28.957 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:30:28.957 INFO:teuthology.orchestra.run.smithi088.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:30:28.957 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2024-04-21 08:30:15.658790740 +0000 2024-04-21T08:30:28.957 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2024-04-21 08:25:00.667359454 +0000 2024-04-21T08:30:28.957 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2024-04-21 08:25:00.667359454 +0000 2024-04-21T08:30:28.957 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: 2024-04-21 08:25:00.667359454 +0000 2024-04-21T08:30:28.958 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-21T08:30:29.022 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-04-21T08:30:29.022 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-04-21T08:30:29.022 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.000301668 s, 1.7 MB/s 2024-04-21T08:30:29.023 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-21T08:30:29.080 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_3 2024-04-21T08:30:29.137 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-21T08:30:29.137 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:30:29.137 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 5h/5d Inode: 704 Links: 1 2024-04-21T08:30:29.137 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:30:29.137 INFO:teuthology.orchestra.run.smithi088.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:30:29.138 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2024-04-21 08:30:15.658790740 +0000 2024-04-21T08:30:29.138 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2024-04-21 08:25:01.010352273 +0000 2024-04-21T08:30:29.138 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2024-04-21 08:25:01.010352273 +0000 2024-04-21T08:30:29.138 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: 2024-04-21 08:25:01.010352273 +0000 2024-04-21T08:30:29.138 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-21T08:30:29.200 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-04-21T08:30:29.200 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-04-21T08:30:29.200 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.000201457 s, 2.5 MB/s 2024-04-21T08:30:29.202 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-21T08:30:29.258 DEBUG:teuthology.orchestra.run.smithi088:> stat /dev/vg_nvme/lv_4 2024-04-21T08:30:29.313 INFO:teuthology.orchestra.run.smithi088.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-21T08:30:29.313 INFO:teuthology.orchestra.run.smithi088.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:30:29.313 INFO:teuthology.orchestra.run.smithi088.stdout:Device: 5h/5d Inode: 721 Links: 1 2024-04-21T08:30:29.313 INFO:teuthology.orchestra.run.smithi088.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:30:29.314 INFO:teuthology.orchestra.run.smithi088.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:30:29.314 INFO:teuthology.orchestra.run.smithi088.stdout:Access: 2024-04-21 08:30:15.659790718 +0000 2024-04-21T08:30:29.314 INFO:teuthology.orchestra.run.smithi088.stdout:Modify: 2024-04-21 08:25:01.341345343 +0000 2024-04-21T08:30:29.314 INFO:teuthology.orchestra.run.smithi088.stdout:Change: 2024-04-21 08:25:01.341345343 +0000 2024-04-21T08:30:29.314 INFO:teuthology.orchestra.run.smithi088.stdout: Birth: 2024-04-21 08:25:01.341345343 +0000 2024-04-21T08:30:29.314 DEBUG:teuthology.orchestra.run.smithi088:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-21T08:30:29.375 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records in 2024-04-21T08:30:29.375 INFO:teuthology.orchestra.run.smithi088.stderr:1+0 records out 2024-04-21T08:30:29.376 INFO:teuthology.orchestra.run.smithi088.stderr:512 bytes copied, 0.00021807 s, 2.3 MB/s 2024-04-21T08:30:29.377 DEBUG:teuthology.orchestra.run.smithi088:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-21T08:30:29.434 DEBUG:teuthology.orchestra.run.smithi179:> set -ex 2024-04-21T08:30:29.434 DEBUG:teuthology.orchestra.run.smithi179:> dd if=/scratch_devs of=/dev/stdout 2024-04-21T08:30:29.451 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2024-04-21T08:30:29.451 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_1 2024-04-21T08:30:29.463 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:29 smithi088 ceph-mon[31259]: Reconfiguring mon.c (monmap changed)... 2024-04-21T08:30:29.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:29 smithi088 ceph-mon[31259]: Reconfiguring daemon mon.c on smithi179 2024-04-21T08:30:29.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:29 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:29.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:29 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:29.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:29 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:29.506 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2024-04-21T08:30:29.506 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:30:29.507 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 5h/5d Inode: 667 Links: 1 2024-04-21T08:30:29.507 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:30:29.507 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:30:29.507 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-04-21 08:30:06.470888376 +0000 2024-04-21T08:30:29.507 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-04-21 08:25:03.954159202 +0000 2024-04-21T08:30:29.507 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-04-21 08:25:03.954159202 +0000 2024-04-21T08:30:29.507 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-04-21 08:25:03.954159202 +0000 2024-04-21T08:30:29.507 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2024-04-21T08:30:29.570 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-04-21T08:30:29.571 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-04-21T08:30:29.571 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000269163 s, 1.9 MB/s 2024-04-21T08:30:29.573 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2024-04-21T08:30:29.629 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_2 2024-04-21T08:30:29.684 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2024-04-21T08:30:29.684 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:30:29.685 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 5h/5d Inode: 686 Links: 1 2024-04-21T08:30:29.685 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:30:29.685 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:30:29.685 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-04-21 08:30:06.470888376 +0000 2024-04-21T08:30:29.685 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-04-21 08:25:04.300151897 +0000 2024-04-21T08:30:29.685 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-04-21 08:25:04.300151897 +0000 2024-04-21T08:30:29.685 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-04-21 08:25:04.300151897 +0000 2024-04-21T08:30:29.685 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2024-04-21T08:30:29.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:29 smithi179 ceph-mon[31829]: Reconfiguring mon.c (monmap changed)... 2024-04-21T08:30:29.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:29 smithi179 ceph-mon[31829]: Reconfiguring daemon mon.c on smithi179 2024-04-21T08:30:29.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:29 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:29.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:29 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:29.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:29 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:29.748 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-04-21T08:30:29.748 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-04-21T08:30:29.748 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000191449 s, 2.7 MB/s 2024-04-21T08:30:29.749 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2024-04-21T08:30:29.805 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_3 2024-04-21T08:30:29.860 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2024-04-21T08:30:29.860 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:30:29.860 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 5h/5d Inode: 703 Links: 1 2024-04-21T08:30:29.860 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:30:29.860 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:30:29.860 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-04-21 08:30:06.470888376 +0000 2024-04-21T08:30:29.860 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-04-21 08:25:04.637144783 +0000 2024-04-21T08:30:29.861 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-04-21 08:25:04.637144783 +0000 2024-04-21T08:30:29.861 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-04-21 08:25:04.637144783 +0000 2024-04-21T08:30:29.861 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2024-04-21T08:30:29.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:29 smithi086 ceph-mon[27939]: Reconfiguring mon.c (monmap changed)... 2024-04-21T08:30:29.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:29 smithi086 ceph-mon[27939]: Reconfiguring daemon mon.c on smithi179 2024-04-21T08:30:29.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:29 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:29.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:29 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:29.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:29 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:29.922 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-04-21T08:30:29.923 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-04-21T08:30:29.923 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000244408 s, 2.1 MB/s 2024-04-21T08:30:29.924 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2024-04-21T08:30:29.980 DEBUG:teuthology.orchestra.run.smithi179:> stat /dev/vg_nvme/lv_4 2024-04-21T08:30:30.035 INFO:teuthology.orchestra.run.smithi179.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2024-04-21T08:30:30.036 INFO:teuthology.orchestra.run.smithi179.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2024-04-21T08:30:30.036 INFO:teuthology.orchestra.run.smithi179.stdout:Device: 5h/5d Inode: 723 Links: 1 2024-04-21T08:30:30.036 INFO:teuthology.orchestra.run.smithi179.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:30:30.036 INFO:teuthology.orchestra.run.smithi179.stdout:Context: system_u:object_r:device_t:s0 2024-04-21T08:30:30.036 INFO:teuthology.orchestra.run.smithi179.stdout:Access: 2024-04-21 08:30:06.471888355 +0000 2024-04-21T08:30:30.036 INFO:teuthology.orchestra.run.smithi179.stdout:Modify: 2024-04-21 08:25:04.980137542 +0000 2024-04-21T08:30:30.036 INFO:teuthology.orchestra.run.smithi179.stdout:Change: 2024-04-21 08:25:04.980137542 +0000 2024-04-21T08:30:30.036 INFO:teuthology.orchestra.run.smithi179.stdout: Birth: 2024-04-21 08:25:04.980137542 +0000 2024-04-21T08:30:30.036 DEBUG:teuthology.orchestra.run.smithi179:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2024-04-21T08:30:30.099 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records in 2024-04-21T08:30:30.099 INFO:teuthology.orchestra.run.smithi179.stderr:1+0 records out 2024-04-21T08:30:30.099 INFO:teuthology.orchestra.run.smithi179.stderr:512 bytes copied, 0.000191712 s, 2.7 MB/s 2024-04-21T08:30:30.100 DEBUG:teuthology.orchestra.run.smithi179:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2024-04-21T08:30:30.156 INFO:tasks.cephadm:Deploying osd.0 on smithi086 with /dev/vg_nvme/lv_4... 2024-04-21T08:30:30.156 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-21T08:30:30.383 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:30:30.505 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:30 smithi086 ceph-mon[27939]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:30.505 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:30 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:30.505 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:30 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:30.505 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:30 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:30.506 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:30 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T08:30:30.506 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:30 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-21T08:30:30.506 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:30 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T08:30:30.506 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:30 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:30.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:30 smithi088 ceph-mon[31259]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:30.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:30 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:30.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:30 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:30.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:30 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:30.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:30 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T08:30:30.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:30 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-21T08:30:30.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:30 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T08:30:30.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:30 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:30.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:30 smithi179 ceph-mon[31829]: pgmap v27: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:30.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:30 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:30.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:30 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:30.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:30 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:30.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:30 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T08:30:30.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:30 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.z", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-21T08:30:30.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:30 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T08:30:30.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:30 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:31.676 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:31 smithi086 ceph-mon[27939]: Deploying daemon mgr.z on smithi179 2024-04-21T08:30:31.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:31 smithi088 ceph-mon[31259]: Deploying daemon mgr.z on smithi179 2024-04-21T08:30:31.720 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:31 smithi179 ceph-mon[31829]: Deploying daemon mgr.z on smithi179 2024-04-21T08:30:31.720 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:30:31 smithi179 systemd[1]: Starting Ceph mgr.z for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:30:31.989 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:30:31 smithi179 podman[33666]: 2024-04-21 08:30:31.721541842 +0000 UTC m=+0.108293458 container create b71731ad80c807de8d3724df59b92561f5c6ce317d6ad3ebd51923bb1569ee5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2) 2024-04-21T08:30:31.989 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:30:31 smithi179 podman[33666]: 2024-04-21 08:30:31.631168403 +0000 UTC m=+0.017920041 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:30:31.989 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:30:31 smithi179 podman[33666]: 2024-04-21 08:30:31.915307507 +0000 UTC m=+0.302059123 container init b71731ad80c807de8d3724df59b92561f5c6ce317d6ad3ebd51923bb1569ee5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-21T08:30:31.989 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:30:31 smithi179 podman[33666]: 2024-04-21 08:30:31.918075467 +0000 UTC m=+0.304827091 container start b71731ad80c807de8d3724df59b92561f5c6ce317d6ad3ebd51923bb1569ee5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:30:31.989 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:30:31 smithi179 ceph-mgr[33687]: -- 172.21.15.179:0/3496397454 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5569fe6c05a0 con 0x5569fe695000 2024-04-21T08:30:31.989 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:30:31 smithi179 bash[33666]: b71731ad80c807de8d3724df59b92561f5c6ce317d6ad3ebd51923bb1569ee5e 2024-04-21T08:30:32.213 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-21T08:30:32.213 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-21T08:30:32.213 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-04-21T08:30:32.213 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-04-21T08:30:32.213 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0115414 s, 909 MB/s 2024-04-21T08:30:32.213 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-04-21T08:30:32.246 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:30:31 smithi179 systemd[1]: Started Ceph mgr.z for f8cec7cc-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:30:32.246 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:30:32 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:30:32.071+0000 7fd1aaf81200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-21T08:30:32.539 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:32 smithi086 ceph-mon[27939]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:32.539 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:32 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:32.539 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:32 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:32.539 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:32 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:32.539 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:32 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T08:30:32.539 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:32 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-21T08:30:32.539 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:32 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T08:30:32.539 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:32 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:32.657 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:32 smithi179 ceph-mon[31829]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:32.658 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:32 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:32.658 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:32 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:32.658 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:32 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:32.658 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:32 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T08:30:32.658 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:32 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-21T08:30:32.658 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:32 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T08:30:32.658 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:32 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:32.658 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:30:32 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:30:32.253+0000 7fd1aaf81200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:30:32.658 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:30:32 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:30:32.354+0000 7fd1aaf81200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-21T08:30:32.658 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:30:32 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:30:32.658+0000 7fd1aaf81200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-21T08:30:32.700 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:32 smithi088 ceph-mon[31259]: pgmap v28: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:32.700 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:32 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:32.700 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:32 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:32.700 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:32 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:32.700 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:32 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T08:30:32.700 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:32 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mgr.y", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]': finished 2024-04-21T08:30:32.700 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:32 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T08:30:32.700 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:32 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:32.757 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi086:vg_nvme/lv_4 2024-04-21T08:30:32.990 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:30:32.996 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:30:32 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:30:32.741+0000 7fd1aaf81200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T08:30:33.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:33 smithi086 ceph-mon[27939]: Deploying daemon mgr.y on smithi088 2024-04-21T08:30:33.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:33 smithi088 ceph-mon[31259]: Deploying daemon mgr.y on smithi088 2024-04-21T08:30:33.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:33 smithi179 ceph-mon[31829]: Deploying daemon mgr.y on smithi088 2024-04-21T08:30:34.246 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:30:33 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:30:33.961+0000 7fd1aaf81200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T08:30:34.246 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:30:34 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:30:34.135+0000 7fd1aaf81200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T08:30:34.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.428+0000 7ff443646700 1 -- 172.21.15.86:0/1341607582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff43c102ac0 msgr2=0x7ff43c102ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:34.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.428+0000 7ff443646700 1 --2- 172.21.15.86:0/1341607582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff43c102ac0 0x7ff43c102ee0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff4300099c0 tx=0x7ff430009cd0 comp rx=0 tx=0).stop 2024-04-21T08:30:34.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.428+0000 7ff443646700 1 -- 172.21.15.86:0/1341607582 shutdown_connections 2024-04-21T08:30:34.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.428+0000 7ff443646700 1 --2- 172.21.15.86:0/1341607582 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff43c104660 0x7ff43c10af10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:34.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.428+0000 7ff443646700 1 --2- 172.21.15.86:0/1341607582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff43c102ac0 0x7ff43c102ee0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:34.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.428+0000 7ff443646700 1 --2- 172.21.15.86:0/1341607582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff43c103cc0 0x7ff43c104120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:34.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.428+0000 7ff443646700 1 -- 172.21.15.86:0/1341607582 >> 172.21.15.86:0/1341607582 conn(0x7ff43c0fe060 msgr2=0x7ff43c1004a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:30:34.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.428+0000 7ff443646700 1 -- 172.21.15.86:0/1341607582 shutdown_connections 2024-04-21T08:30:34.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.428+0000 7ff443646700 1 -- 172.21.15.86:0/1341607582 wait complete. 2024-04-21T08:30:34.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.429+0000 7ff443646700 1 Processor -- start 2024-04-21T08:30:34.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.429+0000 7ff443646700 1 -- start start 2024-04-21T08:30:34.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.429+0000 7ff443646700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff43c102ac0 0x7ff43c0767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:34.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.429+0000 7ff443646700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff43c103cc0 0x7ff43c074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:34.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.429+0000 7ff443646700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff43c104660 0x7ff43c075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:34.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.429+0000 7ff443646700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c10e580 con 0x7ff43c103cc0 2024-04-21T08:30:34.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.429+0000 7ff443646700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c10e400 con 0x7ff43c102ac0 2024-04-21T08:30:34.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.429+0000 7ff443646700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff43c10e700 con 0x7ff43c104660 2024-04-21T08:30:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.429+0000 7ff440be1700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff43c103cc0 0x7ff43c074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.429+0000 7ff440be1700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff43c103cc0 0x7ff43c074e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36828/0 (socket says 172.21.15.86:36828) 2024-04-21T08:30:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.429+0000 7ff440be1700 1 -- 172.21.15.86:0/1773151293 learned_addr learned my addr 172.21.15.86:0/1773151293 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:30:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.429+0000 7ff441be3700 1 --2- 172.21.15.86:0/1773151293 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff43c104660 0x7ff43c075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.429+0000 7ff4413e2700 1 --2- 172.21.15.86:0/1773151293 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff43c102ac0 0x7ff43c0767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.430+0000 7ff440be1700 1 -- 172.21.15.86:0/1773151293 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff43c104660 msgr2=0x7ff43c075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.430+0000 7ff440be1700 1 --2- 172.21.15.86:0/1773151293 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff43c104660 0x7ff43c075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.430+0000 7ff440be1700 1 -- 172.21.15.86:0/1773151293 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff43c102ac0 msgr2=0x7ff43c0767e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.430+0000 7ff440be1700 1 --2- 172.21.15.86:0/1773151293 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff43c102ac0 0x7ff43c0767e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.430+0000 7ff440be1700 1 -- 172.21.15.86:0/1773151293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff43c075c20 con 0x7ff43c103cc0 2024-04-21T08:30:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.430+0000 7ff440be1700 1 --2- 172.21.15.86:0/1773151293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff43c103cc0 0x7ff43c074e30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff43800d980 tx=0x7ff43800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:30:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.430+0000 7ff42e7fc700 1 -- 172.21.15.86:0/1773151293 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff438016070 con 0x7ff43c103cc0 2024-04-21T08:30:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.430+0000 7ff42e7fc700 1 -- 172.21.15.86:0/1773151293 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff438004830 con 0x7ff43c103cc0 2024-04-21T08:30:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.430+0000 7ff443646700 1 -- 172.21.15.86:0/1773151293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff43c1a2de0 con 0x7ff43c103cc0 2024-04-21T08:30:34.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.430+0000 7ff42e7fc700 1 -- 172.21.15.86:0/1773151293 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff438005540 con 0x7ff43c103cc0 2024-04-21T08:30:34.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.430+0000 7ff443646700 1 -- 172.21.15.86:0/1773151293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff43c1a3290 con 0x7ff43c103cc0 2024-04-21T08:30:34.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.431+0000 7ff42e7fc700 1 -- 172.21.15.86:0/1773151293 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 13) ==== 46614+0+0 (secure 0 0 0) 0x7ff43800f690 con 0x7ff43c103cc0 2024-04-21T08:30:34.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.431+0000 7ff42e7fc700 1 --2- 172.21.15.86:0/1773151293 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ff428039890 0x7ff42803bd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:34.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.431+0000 7ff443646700 1 -- 172.21.15.86:0/1773151293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff43c069730 con 0x7ff43c103cc0 2024-04-21T08:30:34.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.431+0000 7ff42e7fc700 1 -- 172.21.15.86:0/1773151293 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(4..4 src has 1..4) ==== 1155+0+0 (secure 0 0 0) 0x7ff43804c720 con 0x7ff43c103cc0 2024-04-21T08:30:34.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.431+0000 7ff4413e2700 1 --2- 172.21.15.86:0/1773151293 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ff428039890 0x7ff42803bd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:34.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.431+0000 7ff4413e2700 1 --2- 172.21.15.86:0/1773151293 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ff428039890 0x7ff42803bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff430006010 tx=0x7ff43000b540 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:30:34.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.436+0000 7ff42e7fc700 1 -- 172.21.15.86:0/1773151293 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff43801f020 con 0x7ff43c103cc0 2024-04-21T08:30:34.600 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:34 smithi179 ceph-mon[31829]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:34.600 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:30:34 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:30:34.312+0000 7fd1aaf81200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T08:30:34.601 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:30:34 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:30:34.600+0000 7fd1aaf81200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T08:30:34.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:34.643+0000 7ff443646700 1 -- 172.21.15.86:0/1773151293 --> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7ff43c062e80 con 0x7ff428039890 2024-04-21T08:30:34.642 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:34 smithi086 ceph-mon[27939]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:34.670 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:34 smithi088 ceph-mon[31259]: pgmap v29: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:34.671 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:30:34 smithi088 ceph-mgr[32655]: -- 172.21.15.88:0/51724638 <== mon.2 v2:172.21.15.88:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55681f1da5a0 con 0x55681f1aec00 2024-04-21T08:30:34.671 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:30:34 smithi088 bash[32628]: d163c7a7cdb535423088004942a4c61a4181eb9d02ce0f121e96091379305ad0 2024-04-21T08:30:34.898 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:30:34 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:30:34.692+0000 7fd1aaf81200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T08:30:34.898 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:30:34 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:30:34.899+0000 7fd1aaf81200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T08:30:34.963 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:30:34 smithi088 systemd[1]: Started Ceph mgr.y for f8cec7cc-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:30:34.963 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:30:34 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:30:34.702+0000 7f219e836200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-21T08:30:34.964 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:30:34 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:30:34.894+0000 7f219e836200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:30:35.246 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:30:35 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:30:35.100+0000 7fd1aaf81200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:30:35.306 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:30:35 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:30:35.003+0000 7f219e836200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-21T08:30:35.534 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:35 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:30:35.535 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:35 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:30:35.535 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:35 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:35.535 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:35 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:35.535 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:35 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:35.535 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:35 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:35.535 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:35 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:35.535 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:35 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:35.684 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:30:35 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:30:35.330+0000 7f219e836200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-21T08:30:35.684 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:30:35 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:30:35.432+0000 7f219e836200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T08:30:35.684 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:35 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:30:35.684 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:35 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:30:35.684 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:35 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:35.684 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:35 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:35.684 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:35 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:35.684 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:35 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:35.684 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:35 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:35.685 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:35 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:35.733 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:35 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:30:35.734 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:35 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:30:35.734 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:35 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:35.734 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:35 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:35.734 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:35 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:35.734 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:35 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:35.734 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:35 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:35.734 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:35 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:35.996 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:30:35 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:30:35.733+0000 7fd1aaf81200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T08:30:36.434 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:36 smithi086 ceph-mon[27939]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:30:36.434 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:36 smithi086 ceph-mon[27939]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:36.434 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:36 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:36.434 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:36 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:36.435 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:36 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:36.435 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:36 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:36.435 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:36 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:36.435 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:36 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:36.435 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:36 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:36.435 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:36 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T08:30:36.435 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:36 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T08:30:36.435 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:36 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:36.465 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:30:36 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:30:36.167+0000 7fd1aaf81200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T08:30:36.465 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:30:36 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:30:36.264+0000 7fd1aaf81200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T08:30:36.465 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:30:36 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:30:36.465+0000 7fd1aaf81200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T08:30:36.465 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:36 smithi179 ceph-mon[31829]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:30:36.465 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:36 smithi179 ceph-mon[31829]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:36.465 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:36 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:36.465 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:36 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:36.465 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:36 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:36.465 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:36 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:36.465 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:36 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:36.466 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:36 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:36.466 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:36 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:36.466 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:36 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T08:30:36.466 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:36 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T08:30:36.466 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:36 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:36.713 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:30:36 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:30:36.658+0000 7f219e836200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T08:30:36.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:36 smithi088 ceph-mon[31259]: from='client.14220 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:30:36.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:36 smithi088 ceph-mon[31259]: pgmap v30: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:36.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:36 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:36.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:36 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:36.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:36 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:36.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:36 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:36.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:36 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:36.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:36 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:36.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:36 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:36.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:36 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mgr.x", "caps": ["mon", "profile mgr", "osd", "allow *", "mds", "allow *"]}]: dispatch 2024-04-21T08:30:36.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:36 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mgr services"}]: dispatch 2024-04-21T08:30:36.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:36 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:37.213 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:30:36 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:30:36.827+0000 7f219e836200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T08:30:37.214 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:30:36 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:30:36.995+0000 7f219e836200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T08:30:37.246 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:30:36 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:30:36.982+0000 7fd1aaf81200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T08:30:37.246 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:30:37 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:30:37.073+0000 7fd1aaf81200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T08:30:37.576 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:30:37 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:30:37.283+0000 7f219e836200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T08:30:37.576 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:30:37 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:30:37.374+0000 7f219e836200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T08:30:37.576 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:30:37 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:30:37.576+0000 7f219e836200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T08:30:37.576 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:37 smithi088 ceph-mon[31259]: Reconfiguring mgr.x (unknown last config time)... 2024-04-21T08:30:37.576 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:37 smithi088 ceph-mon[31259]: Reconfiguring daemon mgr.x on smithi086 2024-04-21T08:30:37.577 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:37 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1270917712' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "89b7601f-0620-4db2-90f1-b45cea173738"}]: dispatch 2024-04-21T08:30:37.577 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:37 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1270917712' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "89b7601f-0620-4db2-90f1-b45cea173738"}]': finished 2024-04-21T08:30:37.577 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:37 smithi088 ceph-mon[31259]: osdmap e5: 1 total, 0 up, 1 in 2024-04-21T08:30:37.577 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:37 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:30:37.647 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:37 smithi179 ceph-mon[31829]: Reconfiguring mgr.x (unknown last config time)... 2024-04-21T08:30:37.647 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:37 smithi179 ceph-mon[31829]: Reconfiguring daemon mgr.x on smithi086 2024-04-21T08:30:37.647 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:37 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1270917712' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "89b7601f-0620-4db2-90f1-b45cea173738"}]: dispatch 2024-04-21T08:30:37.647 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:37 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1270917712' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "89b7601f-0620-4db2-90f1-b45cea173738"}]': finished 2024-04-21T08:30:37.647 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:37 smithi179 ceph-mon[31829]: osdmap e5: 1 total, 0 up, 1 in 2024-04-21T08:30:37.647 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:37 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:30:37.648 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:30:37 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:30:37.310+0000 7fd1aaf81200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T08:30:37.648 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:30:37 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:30:37.442+0000 7fd1aaf81200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T08:30:37.648 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:30:37 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:30:37.534+0000 7fd1aaf81200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T08:30:37.648 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:30:37 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:30:37.647+0000 7fd1aaf81200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T08:30:37.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:37 smithi086 ceph-mon[27939]: Reconfiguring mgr.x (unknown last config time)... 2024-04-21T08:30:37.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:37 smithi086 ceph-mon[27939]: Reconfiguring daemon mgr.x on smithi086 2024-04-21T08:30:37.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:37 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1270917712' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "89b7601f-0620-4db2-90f1-b45cea173738"}]: dispatch 2024-04-21T08:30:37.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:37 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1270917712' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "89b7601f-0620-4db2-90f1-b45cea173738"}]': finished 2024-04-21T08:30:37.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:37 smithi086 ceph-mon[27939]: osdmap e5: 1 total, 0 up, 1 in 2024-04-21T08:30:37.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:37 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:30:37.963 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:30:37 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:30:37.766+0000 7f219e836200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:30:38.246 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:30:37 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:30:37.948+0000 7fd1aaf81200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T08:30:38.713 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:30:38 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:30:38.360+0000 7f219e836200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T08:30:38.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:38 smithi088 ceph-mon[31259]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:38.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:38 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/988551414' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:30:38.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:38 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:38.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:38 smithi179 ceph-mon[31829]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:38.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:38 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/988551414' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:30:38.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:38 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:38.746 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:30:38 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:30:38.539+0000 7fd1aaf81200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T08:30:38.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:38 smithi086 ceph-mon[27939]: pgmap v31: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:38.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:38 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/988551414' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:30:38.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:38 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:39.068 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:30:38 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:30:38.776+0000 7f219e836200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T08:30:39.068 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:30:38 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:30:38.870+0000 7f219e836200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T08:30:39.068 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:30:39 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:30:39.068+0000 7f219e836200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T08:30:39.246 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:30:38 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:30:38.755+0000 7fd1aaf81200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T08:30:39.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:39.431+0000 7ff42e7fc700 1 -- 172.21.15.86:0/1773151293 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 14) ==== 92602+0+0 (secure 0 0 0) 0x7ff438003a00 con 0x7ff43c103cc0 2024-04-21T08:30:39.682 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:30:39 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:30:39.591+0000 7f219e836200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T08:30:39.682 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:30:39 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:30:39.682+0000 7f219e836200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T08:30:39.682 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:39 smithi088 ceph-mon[31259]: Standby manager daemon z started 2024-04-21T08:30:39.682 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:39 smithi088 ceph-mon[31259]: from='mgr.? 172.21.15.179:0/3713305417' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-21T08:30:39.682 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:39 smithi088 ceph-mon[31259]: from='mgr.? 172.21.15.179:0/3713305417' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:30:39.682 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:39 smithi088 ceph-mon[31259]: from='mgr.? 172.21.15.179:0/3713305417' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-21T08:30:39.682 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:39 smithi088 ceph-mon[31259]: from='mgr.? 172.21.15.179:0/3713305417' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:30:39.682 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:39 smithi088 ceph-mon[31259]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:39.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:39 smithi179 ceph-mon[31829]: Standby manager daemon z started 2024-04-21T08:30:39.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:39 smithi179 ceph-mon[31829]: from='mgr.? 172.21.15.179:0/3713305417' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-21T08:30:39.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:39 smithi179 ceph-mon[31829]: from='mgr.? 172.21.15.179:0/3713305417' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:30:39.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:39 smithi179 ceph-mon[31829]: from='mgr.? 172.21.15.179:0/3713305417' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-21T08:30:39.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:39 smithi179 ceph-mon[31829]: from='mgr.? 172.21.15.179:0/3713305417' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:30:39.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:39 smithi179 ceph-mon[31829]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:39.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:39 smithi086 ceph-mon[27939]: Standby manager daemon z started 2024-04-21T08:30:39.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:39 smithi086 ceph-mon[27939]: from='mgr.? 172.21.15.179:0/3713305417' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-21T08:30:39.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:39 smithi086 ceph-mon[27939]: from='mgr.? 172.21.15.179:0/3713305417' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:30:39.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:39 smithi086 ceph-mon[27939]: from='mgr.? 172.21.15.179:0/3713305417' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-21T08:30:39.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:39 smithi086 ceph-mon[27939]: from='mgr.? 172.21.15.179:0/3713305417' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:30:39.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:39 smithi086 ceph-mon[27939]: pgmap v33: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:39.963 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:30:39 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:30:39.920+0000 7f219e836200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T08:30:40.429 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:30:40 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:30:40.052+0000 7f219e836200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T08:30:40.429 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:30:40 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:30:40.145+0000 7f219e836200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T08:30:40.429 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:30:40 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:30:40.258+0000 7f219e836200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T08:30:40.713 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:30:40 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:30:40.557+0000 7f219e836200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T08:30:40.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:40 smithi088 ceph-mon[31259]: mgrmap e14: x(active, since 76s), standbys: z 2024-04-21T08:30:40.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:40 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-21T08:30:40.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:40 smithi179 ceph-mon[31829]: mgrmap e14: x(active, since 76s), standbys: z 2024-04-21T08:30:40.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:40 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-21T08:30:40.848 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:40 smithi086 ceph-mon[27939]: mgrmap e14: x(active, since 76s), standbys: z 2024-04-21T08:30:40.848 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:40 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-21T08:30:41.434 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:30:41 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:30:41.152+0000 7f219e836200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T08:30:41.435 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:30:41 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:30:41.364+0000 7f219e836200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T08:30:41.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:41.440+0000 7ff42e7fc700 1 -- 172.21.15.86:0/1773151293 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7ff43800bc90 con 0x7ff43c103cc0 2024-04-21T08:30:41.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:41 smithi088 ceph-mon[31259]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:41.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:41 smithi088 ceph-mon[31259]: Standby manager daemon y started 2024-04-21T08:30:41.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:41 smithi088 ceph-mon[31259]: from='mgr.? 172.21.15.88:0/537302167' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-21T08:30:41.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:41 smithi088 ceph-mon[31259]: from='mgr.? 172.21.15.88:0/537302167' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:30:41.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:41 smithi088 ceph-mon[31259]: from='mgr.? 172.21.15.88:0/537302167' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-21T08:30:41.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:41 smithi088 ceph-mon[31259]: from='mgr.? 172.21.15.88:0/537302167' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:30:41.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:41 smithi179 ceph-mon[31829]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:41.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:41 smithi179 ceph-mon[31829]: Standby manager daemon y started 2024-04-21T08:30:41.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:41 smithi179 ceph-mon[31829]: from='mgr.? 172.21.15.88:0/537302167' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-21T08:30:41.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:41 smithi179 ceph-mon[31829]: from='mgr.? 172.21.15.88:0/537302167' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:30:41.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:41 smithi179 ceph-mon[31829]: from='mgr.? 172.21.15.88:0/537302167' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-21T08:30:41.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:41 smithi179 ceph-mon[31829]: from='mgr.? 172.21.15.88:0/537302167' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:30:41.784 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:41 smithi086 ceph-mon[27939]: pgmap v34: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:41.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:41 smithi086 ceph-mon[27939]: Standby manager daemon y started 2024-04-21T08:30:41.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:41 smithi086 ceph-mon[27939]: from='mgr.? 172.21.15.88:0/537302167' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-21T08:30:41.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:41 smithi086 ceph-mon[27939]: from='mgr.? 172.21.15.88:0/537302167' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:30:41.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:41 smithi086 ceph-mon[27939]: from='mgr.? 172.21.15.88:0/537302167' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-21T08:30:41.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:41 smithi086 ceph-mon[27939]: from='mgr.? 172.21.15.88:0/537302167' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:30:42.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:42 smithi088 ceph-mon[31259]: mgrmap e15: x(active, since 78s), standbys: z, y 2024-04-21T08:30:42.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:42 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-21T08:30:42.730 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:42 smithi086 ceph-mon[27939]: mgrmap e15: x(active, since 78s), standbys: z, y 2024-04-21T08:30:42.730 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:42 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-21T08:30:42.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:42 smithi179 ceph-mon[31829]: mgrmap e15: x(active, since 78s), standbys: z, y 2024-04-21T08:30:42.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:42 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-21T08:30:44.087 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:43 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:44.087 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:43 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:44.087 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:43 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:44.087 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:43 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:44.087 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:43 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:44.087 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:43 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:44.087 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:43 smithi086 ceph-mon[27939]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:44.213 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:43 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:44.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:43 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:44.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:43 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:44.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:43 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:44.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:43 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:44.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:43 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:44.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:43 smithi088 ceph-mon[31259]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:44.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:43 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:44.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:43 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:44.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:43 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:44.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:43 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:44.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:43 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:44.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:43 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:44.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:43 smithi179 ceph-mon[31829]: pgmap v35: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:45.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:45 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-21T08:30:45.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:45 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:45.384 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:45 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-21T08:30:45.384 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:45 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:45.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:45 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.0"}]: dispatch 2024-04-21T08:30:45.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:45 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:46.287 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:46 smithi086 ceph-mon[27939]: Deploying daemon osd.0 on smithi086 2024-04-21T08:30:46.288 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:46 smithi086 ceph-mon[27939]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:46.463 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:46 smithi088 ceph-mon[31259]: Deploying daemon osd.0 on smithi086 2024-04-21T08:30:46.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:46 smithi088 ceph-mon[31259]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:46.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:46 smithi179 ceph-mon[31829]: Deploying daemon osd.0 on smithi086 2024-04-21T08:30:46.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:46 smithi179 ceph-mon[31829]: pgmap v36: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:48.515 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:48 smithi086 ceph-mon[27939]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:48.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:48 smithi088 ceph-mon[31259]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:48.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:48 smithi179 ceph-mon[31829]: pgmap v37: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:50.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:50 smithi086 ceph-mon[27939]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:50.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:50 smithi088 ceph-mon[31259]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:50.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:50 smithi179 ceph-mon[31829]: pgmap v38: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:51.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:51 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:51.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:51 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:51.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:51 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:51.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:51 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:51.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:51 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:51.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:51 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:51.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:51 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:51.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:51 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:51.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:51 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:52.611 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:52 smithi086 ceph-mon[27939]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:52.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:52 smithi088 ceph-mon[31259]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:52.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:52 smithi179 ceph-mon[31829]: pgmap v39: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:54.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:54.098+0000 7ff42e7fc700 1 -- 172.21.15.86:0/1773151293 <== mgr.14152 v2:172.21.15.86:6800/1364156965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff43c062e80 con 0x7ff428039890 2024-04-21T08:30:54.097 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 0 on host 'smithi086' 2024-04-21T08:30:54.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:54.100+0000 7ff443646700 1 -- 172.21.15.86:0/1773151293 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ff428039890 msgr2=0x7ff42803bd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:54.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:54.100+0000 7ff443646700 1 --2- 172.21.15.86:0/1773151293 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ff428039890 0x7ff42803bd50 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7ff430006010 tx=0x7ff43000b540 comp rx=0 tx=0).stop 2024-04-21T08:30:54.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:54.100+0000 7ff443646700 1 -- 172.21.15.86:0/1773151293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff43c103cc0 msgr2=0x7ff43c074e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:54.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:54.100+0000 7ff443646700 1 --2- 172.21.15.86:0/1773151293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff43c103cc0 0x7ff43c074e30 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ff43800d980 tx=0x7ff43800dc90 comp rx=0 tx=0).stop 2024-04-21T08:30:54.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:54.100+0000 7ff443646700 1 -- 172.21.15.86:0/1773151293 shutdown_connections 2024-04-21T08:30:54.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:54.100+0000 7ff443646700 1 --2- 172.21.15.86:0/1773151293 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ff428039890 0x7ff42803bd50 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:54.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:54.100+0000 7ff443646700 1 --2- 172.21.15.86:0/1773151293 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff43c104660 0x7ff43c075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:54.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:54.100+0000 7ff443646700 1 --2- 172.21.15.86:0/1773151293 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff43c102ac0 0x7ff43c0767e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:54.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:54.100+0000 7ff443646700 1 --2- 172.21.15.86:0/1773151293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff43c103cc0 0x7ff43c074e30 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:54.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:54.100+0000 7ff443646700 1 -- 172.21.15.86:0/1773151293 >> 172.21.15.86:0/1773151293 conn(0x7ff43c0fe060 msgr2=0x7ff43c1070e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:30:54.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:54.100+0000 7ff443646700 1 -- 172.21.15.86:0/1773151293 shutdown_connections 2024-04-21T08:30:54.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:30:54.100+0000 7ff443646700 1 -- 172.21.15.86:0/1773151293 wait complete. 2024-04-21T08:30:54.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:54 smithi086 ceph-mon[27939]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:54.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:54 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:54.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:54 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:54.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:54 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:54.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:54 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:54.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:54 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:54.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:54 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:54.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:54 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:54.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:54 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:54.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:54 smithi086 ceph-mon[27939]: from='osd.0 [v2:172.21.15.86:6802/2571516897,v1:172.21.15.86:6803/2571516897]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-21T08:30:54.700 DEBUG:teuthology.orchestra.run.smithi086:osd.0> sudo journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@osd.0.service 2024-04-21T08:30:54.703 INFO:tasks.cephadm:Deploying osd.1 on smithi088 with /dev/vg_nvme/lv_4... 2024-04-21T08:30:54.704 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-21T08:30:54.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:54 smithi088 ceph-mon[31259]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:54.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:54 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:54.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:54 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:54.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:54 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:54.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:54 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:54.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:54 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:54.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:54 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:54.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:54 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:54.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:54 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:54.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:54 smithi088 ceph-mon[31259]: from='osd.0 [v2:172.21.15.86:6802/2571516897,v1:172.21.15.86:6803/2571516897]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-21T08:30:54.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:54 smithi179 ceph-mon[31829]: pgmap v40: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:54.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:54 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:54.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:54 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:54.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:54 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:54.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:54 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:54.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:54 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:54.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:54 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:54.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:54 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:54.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:54 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:30:54.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:54 smithi179 ceph-mon[31829]: from='osd.0 [v2:172.21.15.86:6802/2571516897,v1:172.21.15.86:6803/2571516897]' entity='osd.0' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]: dispatch 2024-04-21T08:30:54.949 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T08:30:56.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:56 smithi086 ceph-mon[27939]: from='osd.0 [v2:172.21.15.86:6802/2571516897,v1:172.21.15.86:6803/2571516897]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-21T08:30:56.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:56 smithi086 ceph-mon[27939]: osdmap e6: 1 total, 0 up, 1 in 2024-04-21T08:30:56.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:56 smithi086 ceph-mon[27939]: from='osd.0 [v2:172.21.15.86:6802/2571516897,v1:172.21.15.86:6803/2571516897]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-21T08:30:56.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:56 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:30:56.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:56 smithi086 ceph-mon[27939]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:56.117 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 21 08:30:56 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-0[40513]: 2024-04-21T08:30:56.091+0000 7f99d3428700 -1 osd.0 0 waiting for initial osdmap 2024-04-21T08:30:56.117 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 21 08:30:56 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-0[40513]: 2024-04-21T08:30:56.098+0000 7f99cf21b700 -1 osd.0 7 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T08:30:56.197 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:56 smithi088 ceph-mon[31259]: from='osd.0 [v2:172.21.15.86:6802/2571516897,v1:172.21.15.86:6803/2571516897]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-21T08:30:56.197 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:56 smithi088 ceph-mon[31259]: osdmap e6: 1 total, 0 up, 1 in 2024-04-21T08:30:56.197 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:56 smithi088 ceph-mon[31259]: from='osd.0 [v2:172.21.15.86:6802/2571516897,v1:172.21.15.86:6803/2571516897]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-21T08:30:56.197 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:56 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:30:56.197 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:56 smithi088 ceph-mon[31259]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:56.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:56 smithi179 ceph-mon[31829]: from='osd.0 [v2:172.21.15.86:6802/2571516897,v1:172.21.15.86:6803/2571516897]' entity='osd.0' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["0"]}]': finished 2024-04-21T08:30:56.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:56 smithi179 ceph-mon[31829]: osdmap e6: 1 total, 0 up, 1 in 2024-04-21T08:30:56.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:56 smithi179 ceph-mon[31829]: from='osd.0 [v2:172.21.15.86:6802/2571516897,v1:172.21.15.86:6803/2571516897]' entity='osd.0' cmd=[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-21T08:30:56.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:56 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:30:56.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:56 smithi179 ceph-mon[31829]: pgmap v42: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:56.777 INFO:teuthology.orchestra.run.smithi088.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-21T08:30:56.777 INFO:teuthology.orchestra.run.smithi088.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-21T08:30:56.777 INFO:teuthology.orchestra.run.smithi088.stderr: stderr: 10+0 records in 2024-04-21T08:30:56.778 INFO:teuthology.orchestra.run.smithi088.stderr:10+0 records out 2024-04-21T08:30:56.778 INFO:teuthology.orchestra.run.smithi088.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0124806 s, 840 MB/s 2024-04-21T08:30:56.778 INFO:teuthology.orchestra.run.smithi088.stderr:--> Zapping successful for: 2024-04-21T08:30:57.270 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:57 smithi086 ceph-mon[27939]: from='osd.0 [v2:172.21.15.86:6802/2571516897,v1:172.21.15.86:6803/2571516897]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-21T08:30:57.270 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:57 smithi086 ceph-mon[27939]: osdmap e7: 1 total, 0 up, 1 in 2024-04-21T08:30:57.271 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:57 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:30:57.271 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:57 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:30:57.353 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:57 smithi088 ceph-mon[31259]: from='osd.0 [v2:172.21.15.86:6802/2571516897,v1:172.21.15.86:6803/2571516897]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-21T08:30:57.353 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:57 smithi088 ceph-mon[31259]: osdmap e7: 1 total, 0 up, 1 in 2024-04-21T08:30:57.353 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:57 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:30:57.353 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:57 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:30:57.380 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi088:vg_nvme/lv_4 2024-04-21T08:30:57.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:57 smithi179 ceph-mon[31829]: from='osd.0 [v2:172.21.15.86:6802/2571516897,v1:172.21.15.86:6803/2571516897]' entity='osd.0' cmd='[{"prefix": "osd crush create-or-move", "id": 0, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-21T08:30:57.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:57 smithi179 ceph-mon[31829]: osdmap e7: 1 total, 0 up, 1 in 2024-04-21T08:30:57.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:57 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:30:57.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:57 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:30:57.663 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T08:30:58.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:58 smithi088 ceph-mon[31259]: purged_snaps scrub starts 2024-04-21T08:30:58.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:58 smithi088 ceph-mon[31259]: purged_snaps scrub ok 2024-04-21T08:30:58.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:58 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:30:58.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:58 smithi088 ceph-mon[31259]: osd.0 [v2:172.21.15.86:6802/2571516897,v1:172.21.15.86:6803/2571516897] boot 2024-04-21T08:30:58.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:58 smithi088 ceph-mon[31259]: osdmap e8: 1 total, 1 up, 1 in 2024-04-21T08:30:58.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:58 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:30:58.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:58 smithi088 ceph-mon[31259]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:58.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:58 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:58.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:58 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:58.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:58 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:30:58.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:58 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:58.215 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:58 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:58.215 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:58 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:58.215 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:58 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:58.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:58 smithi086 ceph-mon[27939]: purged_snaps scrub starts 2024-04-21T08:30:58.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:58 smithi086 ceph-mon[27939]: purged_snaps scrub ok 2024-04-21T08:30:58.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:58 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:30:58.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:58 smithi086 ceph-mon[27939]: osd.0 [v2:172.21.15.86:6802/2571516897,v1:172.21.15.86:6803/2571516897] boot 2024-04-21T08:30:58.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:58 smithi086 ceph-mon[27939]: osdmap e8: 1 total, 1 up, 1 in 2024-04-21T08:30:58.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:58 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:30:58.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:58 smithi086 ceph-mon[27939]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:58.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:58 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:58.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:58 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:58.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:58 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:30:58.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:58 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:58.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:58 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:58.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:58 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:58.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:58 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:58.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:58 smithi179 ceph-mon[31829]: purged_snaps scrub starts 2024-04-21T08:30:58.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:58 smithi179 ceph-mon[31829]: purged_snaps scrub ok 2024-04-21T08:30:58.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:58 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:30:58.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:58 smithi179 ceph-mon[31829]: osd.0 [v2:172.21.15.86:6802/2571516897,v1:172.21.15.86:6803/2571516897] boot 2024-04-21T08:30:58.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:58 smithi179 ceph-mon[31829]: osdmap e8: 1 total, 1 up, 1 in 2024-04-21T08:30:58.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:58 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:30:58.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:58 smithi179 ceph-mon[31829]: pgmap v45: 0 pgs: ; 0 B data, 0 B used, 0 B / 0 B avail 2024-04-21T08:30:58.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:58 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:58.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:58 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:58.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:58 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:30:58.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:58 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:58.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:58 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:30:58.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:58 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:30:58.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:58 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:30:59.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.091+0000 7feeaf552700 1 -- 172.21.15.88:0/2010916918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feea8108800 msgr2=0x7feea810ac10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:59.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.091+0000 7feeaf552700 1 --2- 172.21.15.88:0/2010916918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feea8108800 0x7feea810ac10 secure :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0x7feea000b0a0 tx=0x7feea000b3b0 comp rx=0 tx=0).stop 2024-04-21T08:30:59.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.091+0000 7feeaf552700 1 -- 172.21.15.88:0/2010916918 shutdown_connections 2024-04-21T08:30:59.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.091+0000 7feeaf552700 1 --2- 172.21.15.88:0/2010916918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feea8108800 0x7feea810ac10 unknown :-1 s=CLOSED pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:59.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.091+0000 7feeaf552700 1 --2- 172.21.15.88:0/2010916918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feea81035a0 0x7feea8105990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:59.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.092+0000 7feeaf552700 1 --2- 172.21.15.88:0/2010916918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feea8105ed0 0x7feea81082c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:59.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.092+0000 7feeaf552700 1 -- 172.21.15.88:0/2010916918 >> 172.21.15.88:0/2010916918 conn(0x7feea80fcf70 msgr2=0x7feea80ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:30:59.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.092+0000 7feeaf552700 1 -- 172.21.15.88:0/2010916918 shutdown_connections 2024-04-21T08:30:59.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.092+0000 7feeaf552700 1 -- 172.21.15.88:0/2010916918 wait complete. 2024-04-21T08:30:59.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.092+0000 7feeaf552700 1 Processor -- start 2024-04-21T08:30:59.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.092+0000 7feeaf552700 1 -- start start 2024-04-21T08:30:59.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.092+0000 7feeaf552700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feea81035a0 0x7feea8198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:59.094 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.092+0000 7feeaf552700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feea8105ed0 0x7feea8199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:59.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.092+0000 7feeaf552700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feea8108800 0x7feea81a0240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:59.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.092+0000 7feeaf552700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feea810e590 con 0x7feea81035a0 2024-04-21T08:30:59.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.092+0000 7feeaf552700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7feea810e410 con 0x7feea8105ed0 2024-04-21T08:30:59.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.092+0000 7feeaf552700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7feea810e710 con 0x7feea8108800 2024-04-21T08:30:59.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.092+0000 7feeacaed700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feea8105ed0 0x7feea8199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:59.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.093+0000 7feeacaed700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feea8105ed0 0x7feea8199220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:55210/0 (socket says 172.21.15.88:55210) 2024-04-21T08:30:59.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.093+0000 7feeacaed700 1 -- 172.21.15.88:0/952615076 learned_addr learned my addr 172.21.15.88:0/952615076 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-21T08:30:59.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.093+0000 7feead2ee700 1 --2- 172.21.15.88:0/952615076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feea81035a0 0x7feea8198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:59.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.093+0000 7feeadaef700 1 --2- 172.21.15.88:0/952615076 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feea8108800 0x7feea81a0240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:59.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.093+0000 7feeacaed700 1 -- 172.21.15.88:0/952615076 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feea8108800 msgr2=0x7feea81a0240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:59.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.093+0000 7feeacaed700 1 --2- 172.21.15.88:0/952615076 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feea8108800 0x7feea81a0240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:59.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.093+0000 7feeacaed700 1 -- 172.21.15.88:0/952615076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feea81035a0 msgr2=0x7feea8198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:30:59.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.093+0000 7feeacaed700 1 --2- 172.21.15.88:0/952615076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feea81035a0 0x7feea8198ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:30:59.095 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.093+0000 7feeacaed700 1 -- 172.21.15.88:0/952615076 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feea81a0910 con 0x7feea8105ed0 2024-04-21T08:30:59.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.093+0000 7feeadaef700 1 --2- 172.21.15.88:0/952615076 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feea8108800 0x7feea81a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:30:59.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.093+0000 7feeacaed700 1 --2- 172.21.15.88:0/952615076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feea8105ed0 0x7feea8199220 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fee9800b530 tx=0x7fee9800b840 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:30:59.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.093+0000 7fee9e7fc700 1 -- 172.21.15.88:0/952615076 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee98015020 con 0x7feea8105ed0 2024-04-21T08:30:59.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.093+0000 7feeaf552700 1 -- 172.21.15.88:0/952615076 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feea81a0c00 con 0x7feea8105ed0 2024-04-21T08:30:59.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.093+0000 7fee9e7fc700 1 -- 172.21.15.88:0/952615076 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee98011ce0 con 0x7feea8105ed0 2024-04-21T08:30:59.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.093+0000 7fee9e7fc700 1 -- 172.21.15.88:0/952615076 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee98010e40 con 0x7feea8105ed0 2024-04-21T08:30:59.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.093+0000 7feeaf552700 1 -- 172.21.15.88:0/952615076 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feea81a1140 con 0x7feea8105ed0 2024-04-21T08:30:59.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.094+0000 7feeaf552700 1 -- 172.21.15.88:0/952615076 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feea8069730 con 0x7feea8105ed0 2024-04-21T08:30:59.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.095+0000 7fee9e7fc700 1 -- 172.21.15.88:0/952615076 <== mon.2 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7fee98018020 con 0x7feea8105ed0 2024-04-21T08:30:59.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.096+0000 7fee9e7fc700 1 --2- 172.21.15.88:0/952615076 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fee940a3bb0 0x7fee940a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:30:59.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.096+0000 7fee9e7fc700 1 -- 172.21.15.88:0/952615076 <== mon.2 v2:172.21.15.88:3300/0 5 ==== osd_map(9..9 src has 1..9) ==== 1767+0+0 (secure 0 0 0) 0x7fee980ce1e0 con 0x7feea8105ed0 2024-04-21T08:30:59.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.096+0000 7feead2ee700 1 --2- 172.21.15.88:0/952615076 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fee940a3bb0 0x7fee940a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:30:59.097 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.096+0000 7feead2ee700 1 --2- 172.21.15.88:0/952615076 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fee940a3bb0 0x7fee940a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7feea810b120 tx=0x7feea400b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:30:59.100 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.100+0000 7fee9e7fc700 1 -- 172.21.15.88:0/952615076 <== mon.2 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fee9801f020 con 0x7feea8105ed0 2024-04-21T08:30:59.305 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:30:59.305+0000 7feeaf552700 1 -- 172.21.15.88:0/952615076 --> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7feea8062e80 con 0x7fee940a3bb0 2024-04-21T08:30:59.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:59 smithi086 ceph-mon[27939]: Detected new or changed devices on smithi086 2024-04-21T08:30:59.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:59 smithi086 ceph-mon[27939]: Adjusting osd_memory_target on smithi086 to 17088M 2024-04-21T08:30:59.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:30:59 smithi086 ceph-mon[27939]: osdmap e9: 1 total, 1 up, 1 in 2024-04-21T08:30:59.463 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:59 smithi088 ceph-mon[31259]: Detected new or changed devices on smithi086 2024-04-21T08:30:59.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:59 smithi088 ceph-mon[31259]: Adjusting osd_memory_target on smithi086 to 17088M 2024-04-21T08:30:59.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:30:59 smithi088 ceph-mon[31259]: osdmap e9: 1 total, 1 up, 1 in 2024-04-21T08:30:59.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:59 smithi179 ceph-mon[31829]: Detected new or changed devices on smithi086 2024-04-21T08:30:59.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:59 smithi179 ceph-mon[31829]: Adjusting osd_memory_target on smithi086 to 17088M 2024-04-21T08:30:59.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:30:59 smithi179 ceph-mon[31829]: osdmap e9: 1 total, 1 up, 1 in 2024-04-21T08:31:00.213 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:00 smithi088 ceph-mon[31259]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:00.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:00 smithi088 ceph-mon[31259]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:31:00.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:00 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:31:00.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:00 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:31:00.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:00 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:00.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:00 smithi086 ceph-mon[27939]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:00.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:00 smithi086 ceph-mon[27939]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:31:00.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:00 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:31:00.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:00 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:31:00.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:00 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:00.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:00 smithi179 ceph-mon[31829]: pgmap v47: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:00.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:00 smithi179 ceph-mon[31829]: from='client.24143 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:31:00.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:00 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:31:00.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:00 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:31:00.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:00 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:02.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:02 smithi086 ceph-mon[27939]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:02.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:02 smithi086 ceph-mon[27939]: from='client.? 172.21.15.88:0/3255021059' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4891031c-f98c-42bc-bb3b-a420855ecc06"}]: dispatch 2024-04-21T08:31:02.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:02 smithi086 ceph-mon[27939]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4891031c-f98c-42bc-bb3b-a420855ecc06"}]: dispatch 2024-04-21T08:31:02.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:02 smithi086 ceph-mon[27939]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4891031c-f98c-42bc-bb3b-a420855ecc06"}]': finished 2024-04-21T08:31:02.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:02 smithi086 ceph-mon[27939]: osdmap e10: 2 total, 1 up, 2 in 2024-04-21T08:31:02.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:02 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:31:02.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:02 smithi088 ceph-mon[31259]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:02.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:02 smithi088 ceph-mon[31259]: from='client.? 172.21.15.88:0/3255021059' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4891031c-f98c-42bc-bb3b-a420855ecc06"}]: dispatch 2024-04-21T08:31:02.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:02 smithi088 ceph-mon[31259]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4891031c-f98c-42bc-bb3b-a420855ecc06"}]: dispatch 2024-04-21T08:31:02.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:02 smithi088 ceph-mon[31259]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4891031c-f98c-42bc-bb3b-a420855ecc06"}]': finished 2024-04-21T08:31:02.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:02 smithi088 ceph-mon[31259]: osdmap e10: 2 total, 1 up, 2 in 2024-04-21T08:31:02.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:02 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:31:02.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:02 smithi179 ceph-mon[31829]: pgmap v48: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:02.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:02 smithi179 ceph-mon[31829]: from='client.? 172.21.15.88:0/3255021059' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4891031c-f98c-42bc-bb3b-a420855ecc06"}]: dispatch 2024-04-21T08:31:02.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:02 smithi179 ceph-mon[31829]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "4891031c-f98c-42bc-bb3b-a420855ecc06"}]: dispatch 2024-04-21T08:31:02.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:02 smithi179 ceph-mon[31829]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "4891031c-f98c-42bc-bb3b-a420855ecc06"}]': finished 2024-04-21T08:31:02.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:02 smithi179 ceph-mon[31829]: osdmap e10: 2 total, 1 up, 2 in 2024-04-21T08:31:02.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:02 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:31:03.614 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:03 smithi088 ceph-mon[31259]: from='client.? 172.21.15.88:0/518003121' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:31:03.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:03 smithi086 ceph-mon[27939]: from='client.? 172.21.15.88:0/518003121' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:31:03.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:03 smithi179 ceph-mon[31829]: from='client.? 172.21.15.88:0/518003121' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:31:04.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:04 smithi086 ceph-mon[27939]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:04.626 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:04 smithi088 ceph-mon[31259]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:04.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:04 smithi179 ceph-mon[31829]: pgmap v50: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:06.565 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:06 smithi088 ceph-mon[31259]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:06.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:06 smithi086 ceph-mon[27939]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:06.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:06 smithi179 ceph-mon[31829]: pgmap v51: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:08.538 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:08 smithi088 ceph-mon[31259]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:08.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:08 smithi086 ceph-mon[27939]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:08.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:08 smithi179 ceph-mon[31829]: pgmap v52: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:09.463 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:09 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-21T08:31:09.463 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:09 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:09.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:09 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-21T08:31:09.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:09 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:09.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:09 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.1"}]: dispatch 2024-04-21T08:31:09.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:09 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:10.563 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:10 smithi088 ceph-mon[31259]: Deploying daemon osd.1 on smithi088 2024-04-21T08:31:10.563 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:10 smithi088 ceph-mon[31259]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:10.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:10 smithi086 ceph-mon[27939]: Deploying daemon osd.1 on smithi088 2024-04-21T08:31:10.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:10 smithi086 ceph-mon[27939]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:10.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:10 smithi179 ceph-mon[31829]: Deploying daemon osd.1 on smithi088 2024-04-21T08:31:10.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:10 smithi179 ceph-mon[31829]: pgmap v53: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:12.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:12 smithi088 ceph-mon[31259]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:12.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:12 smithi086 ceph-mon[27939]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:12.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:12 smithi179 ceph-mon[31829]: pgmap v54: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:14.454 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:14 smithi088 ceph-mon[31259]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:14.455 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:14 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:14.455 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:14 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:31:14.455 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:14 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:14.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:14 smithi086 ceph-mon[27939]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:14.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:14 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:14.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:14 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:31:14.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:14 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:14.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:14 smithi179 ceph-mon[31829]: pgmap v55: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:14.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:14 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:14.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:14 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:31:14.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:14 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:16.463 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:16 smithi088 ceph-mon[31259]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:16.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:16 smithi086 ceph-mon[27939]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:16.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:16 smithi179 ceph-mon[31829]: pgmap v56: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:17.217 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:31:17.217+0000 7fee9e7fc700 1 -- 172.21.15.88:0/952615076 <== mgr.14152 v2:172.21.15.86:6800/1364156965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7feea8062e80 con 0x7fee940a3bb0 2024-04-21T08:31:17.218 INFO:teuthology.orchestra.run.smithi088.stdout:Created osd(s) 1 on host 'smithi088' 2024-04-21T08:31:17.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:31:17.220+0000 7feeaf552700 1 -- 172.21.15.88:0/952615076 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fee940a3bb0 msgr2=0x7fee940a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:31:17.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:31:17.220+0000 7feeaf552700 1 --2- 172.21.15.88:0/952615076 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fee940a3bb0 0x7fee940a6070 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7feea810b120 tx=0x7feea400b410 comp rx=0 tx=0).stop 2024-04-21T08:31:17.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:31:17.220+0000 7feeaf552700 1 -- 172.21.15.88:0/952615076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feea8105ed0 msgr2=0x7feea8199220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:31:17.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:31:17.220+0000 7feeaf552700 1 --2- 172.21.15.88:0/952615076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feea8105ed0 0x7feea8199220 secure :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0x7fee9800b530 tx=0x7fee9800b840 comp rx=0 tx=0).stop 2024-04-21T08:31:17.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:31:17.220+0000 7feeaf552700 1 -- 172.21.15.88:0/952615076 shutdown_connections 2024-04-21T08:31:17.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:31:17.220+0000 7feeaf552700 1 --2- 172.21.15.88:0/952615076 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fee940a3bb0 0x7fee940a6070 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:31:17.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:31:17.220+0000 7feeaf552700 1 --2- 172.21.15.88:0/952615076 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feea8108800 0x7feea81a0240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:31:17.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:31:17.220+0000 7feeaf552700 1 --2- 172.21.15.88:0/952615076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feea81035a0 0x7feea8198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:31:17.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:31:17.220+0000 7feeaf552700 1 --2- 172.21.15.88:0/952615076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feea8105ed0 0x7feea8199220 unknown :-1 s=CLOSED pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:31:17.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:31:17.220+0000 7feeaf552700 1 -- 172.21.15.88:0/952615076 >> 172.21.15.88:0/952615076 conn(0x7feea80fcf70 msgr2=0x7feea8103e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:31:17.220 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:31:17.220+0000 7feeaf552700 1 -- 172.21.15.88:0/952615076 shutdown_connections 2024-04-21T08:31:17.221 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:31:17.220+0000 7feeaf552700 1 -- 172.21.15.88:0/952615076 wait complete. 2024-04-21T08:31:17.811 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:17 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:17.812 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:17 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:17.812 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:17 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:17.812 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:17 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:31:17.812 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:17 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:17.812 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:17 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:17.812 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:17 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:17.812 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:17 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:31:17.812 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:17 smithi088 ceph-mon[31259]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:17.812 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:17 smithi088 ceph-mon[31259]: from='osd.1 [v2:172.21.15.88:6800/74067182,v1:172.21.15.88:6801/74067182]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-21T08:31:17.812 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:17 smithi088 ceph-mon[31259]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-21T08:31:17.842 DEBUG:teuthology.orchestra.run.smithi088:osd.1> sudo journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@osd.1.service 2024-04-21T08:31:17.845 INFO:tasks.cephadm:Deploying osd.2 on smithi179 with /dev/vg_nvme/lv_4... 2024-04-21T08:31:17.845 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_4 2024-04-21T08:31:18.085 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:31:18.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:17 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:18.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:17 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:18.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:17 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:18.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:17 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:31:18.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:17 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:18.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:17 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:18.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:17 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:18.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:17 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:31:18.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:17 smithi086 ceph-mon[27939]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:18.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:17 smithi086 ceph-mon[27939]: from='osd.1 [v2:172.21.15.88:6800/74067182,v1:172.21.15.88:6801/74067182]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-21T08:31:18.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:17 smithi086 ceph-mon[27939]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-21T08:31:18.207 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:17 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:18.208 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:17 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:18.208 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:17 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:18.208 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:17 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:31:18.208 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:17 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:18.208 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:17 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:18.208 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:17 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:18.208 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:17 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:31:18.208 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:17 smithi179 ceph-mon[31829]: pgmap v57: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:18.208 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:17 smithi179 ceph-mon[31829]: from='osd.1 [v2:172.21.15.88:6800/74067182,v1:172.21.15.88:6801/74067182]' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-21T08:31:18.208 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:17 smithi179 ceph-mon[31829]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]: dispatch 2024-04-21T08:31:19.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:19 smithi088 ceph-mon[31259]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-21T08:31:19.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:19 smithi088 ceph-mon[31259]: from='osd.1 [v2:172.21.15.88:6800/74067182,v1:172.21.15.88:6801/74067182]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-21T08:31:19.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:19 smithi088 ceph-mon[31259]: osdmap e11: 2 total, 1 up, 2 in 2024-04-21T08:31:19.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:19 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:31:19.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:19 smithi088 ceph-mon[31259]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-21T08:31:19.482 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:19 smithi179 ceph-mon[31829]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-21T08:31:19.482 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:19 smithi179 ceph-mon[31829]: from='osd.1 [v2:172.21.15.88:6800/74067182,v1:172.21.15.88:6801/74067182]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-21T08:31:19.482 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:19 smithi179 ceph-mon[31829]: osdmap e11: 2 total, 1 up, 2 in 2024-04-21T08:31:19.482 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:19 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:31:19.482 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:19 smithi179 ceph-mon[31829]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-21T08:31:19.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:19 smithi086 ceph-mon[27939]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["1"]}]': finished 2024-04-21T08:31:19.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:19 smithi086 ceph-mon[27939]: from='osd.1 [v2:172.21.15.88:6800/74067182,v1:172.21.15.88:6801/74067182]' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-21T08:31:19.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:19 smithi086 ceph-mon[27939]: osdmap e11: 2 total, 1 up, 2 in 2024-04-21T08:31:19.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:19 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:31:19.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:19 smithi086 ceph-mon[27939]: from='osd.1 ' entity='osd.1' cmd=[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-21T08:31:20.021 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping: /dev/vg_nvme/lv_4 2024-04-21T08:31:20.021 INFO:teuthology.orchestra.run.smithi179.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_4 bs=1M count=10 conv=fsync 2024-04-21T08:31:20.021 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10+0 records in 2024-04-21T08:31:20.021 INFO:teuthology.orchestra.run.smithi179.stderr:10+0 records out 2024-04-21T08:31:20.021 INFO:teuthology.orchestra.run.smithi179.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0114236 s, 918 MB/s 2024-04-21T08:31:20.022 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping successful for: 2024-04-21T08:31:20.320 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:20 smithi179 ceph-mon[31829]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]': finished 2024-04-21T08:31:20.320 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:20 smithi179 ceph-mon[31829]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:20.320 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:20 smithi179 ceph-mon[31829]: osdmap e12: 2 total, 1 up, 2 in 2024-04-21T08:31:20.321 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:20 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:31:20.321 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:20 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:31:20.321 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:20 smithi179 ceph-mon[31829]: from='osd.1 ' entity='osd.1' 2024-04-21T08:31:20.321 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:20 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:31:20.458 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:20 smithi088 ceph-mon[31259]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]': finished 2024-04-21T08:31:20.458 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:20 smithi088 ceph-mon[31259]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:20.458 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:20 smithi088 ceph-mon[31259]: osdmap e12: 2 total, 1 up, 2 in 2024-04-21T08:31:20.458 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:20 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:31:20.458 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:20 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:31:20.458 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:20 smithi088 ceph-mon[31259]: from='osd.1 ' entity='osd.1' 2024-04-21T08:31:20.458 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:20 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:31:20.458 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 21 08:31:20 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-1[35842]: 2024-04-21T08:31:20.033+0000 7fe4d25bf700 -1 osd.1 0 waiting for initial osdmap 2024-04-21T08:31:20.458 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 21 08:31:20 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-1[35842]: 2024-04-21T08:31:20.037+0000 7fe4c9ba9700 -1 osd.1 12 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T08:31:20.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:20 smithi086 ceph-mon[27939]: from='osd.1 ' entity='osd.1' cmd='[{"prefix": "osd crush create-or-move", "id": 1, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]': finished 2024-04-21T08:31:20.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:20 smithi086 ceph-mon[27939]: pgmap v59: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:20.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:20 smithi086 ceph-mon[27939]: osdmap e12: 2 total, 1 up, 2 in 2024-04-21T08:31:20.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:20 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:31:20.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:20 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:31:20.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:20 smithi086 ceph-mon[27939]: from='osd.1 ' entity='osd.1' 2024-04-21T08:31:20.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:20 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:31:20.653 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi179:vg_nvme/lv_4 2024-04-21T08:31:20.908 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:31:21.296 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:21 smithi179 ceph-mon[31829]: purged_snaps scrub starts 2024-04-21T08:31:21.296 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:21 smithi179 ceph-mon[31829]: purged_snaps scrub ok 2024-04-21T08:31:21.296 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:21 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:21.296 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:21 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:21.296 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:21 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:31:21.296 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:21 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:21.296 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:21 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:21.296 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:21 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:31:21.296 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:21 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:21.296 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:21 smithi179 ceph-mon[31829]: osd.1 [v2:172.21.15.88:6800/74067182,v1:172.21.15.88:6801/74067182] boot 2024-04-21T08:31:21.296 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:21 smithi179 ceph-mon[31829]: osdmap e13: 2 total, 2 up, 2 in 2024-04-21T08:31:21.296 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:21 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:31:21.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:21 smithi086 ceph-mon[27939]: purged_snaps scrub starts 2024-04-21T08:31:21.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:21 smithi086 ceph-mon[27939]: purged_snaps scrub ok 2024-04-21T08:31:21.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:21 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:21.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:21 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:21.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:21 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:31:21.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:21 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:21.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:21 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:21.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:21 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:31:21.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:21 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:21.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:21 smithi086 ceph-mon[27939]: osd.1 [v2:172.21.15.88:6800/74067182,v1:172.21.15.88:6801/74067182] boot 2024-04-21T08:31:21.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:21 smithi086 ceph-mon[27939]: osdmap e13: 2 total, 2 up, 2 in 2024-04-21T08:31:21.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:21 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:31:21.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:21 smithi088 ceph-mon[31259]: purged_snaps scrub starts 2024-04-21T08:31:21.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:21 smithi088 ceph-mon[31259]: purged_snaps scrub ok 2024-04-21T08:31:21.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:21 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:21.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:21 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:21.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:21 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:31:21.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:21 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:21.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:21 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:21.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:21 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:31:21.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:21 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:21.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:21 smithi088 ceph-mon[31259]: osd.1 [v2:172.21.15.88:6800/74067182,v1:172.21.15.88:6801/74067182] boot 2024-04-21T08:31:21.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:21 smithi088 ceph-mon[31259]: osdmap e13: 2 total, 2 up, 2 in 2024-04-21T08:31:21.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:21 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:31:22.292 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:22 smithi179 ceph-mon[31829]: Detected new or changed devices on smithi088 2024-04-21T08:31:22.292 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:22 smithi179 ceph-mon[31829]: Adjusting osd_memory_target on smithi088 to 17088M 2024-04-21T08:31:22.293 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:22 smithi179 ceph-mon[31829]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:22.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.531+0000 7f88ee676700 1 -- 172.21.15.179:0/3216156212 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88e8102f90 msgr2=0x7f88e81033b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:31:22.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.531+0000 7f88ee676700 1 --2- 172.21.15.179:0/3216156212 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88e8102f90 0x7f88e81033b0 secure :-1 s=READY pgs=10 cs=0 l=1 rev1=1 crypto rx=0x7f88d8009990 tx=0x7f88d8009ca0 comp rx=0 tx=0).stop 2024-04-21T08:31:22.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.531+0000 7f88ee676700 1 -- 172.21.15.179:0/3216156212 shutdown_connections 2024-04-21T08:31:22.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.531+0000 7f88ee676700 1 --2- 172.21.15.179:0/3216156212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88e8102600 0x7f88e8109ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:31:22.533 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.531+0000 7f88ee676700 1 --2- 172.21.15.179:0/3216156212 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88e8102f90 0x7f88e81033b0 unknown :-1 s=CLOSED pgs=10 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:31:22.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.531+0000 7f88ee676700 1 --2- 172.21.15.179:0/3216156212 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88e8101c40 0x7f88e81020c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:31:22.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.531+0000 7f88ee676700 1 -- 172.21.15.179:0/3216156212 >> 172.21.15.179:0/3216156212 conn(0x7f88e80fc6d0 msgr2=0x7f88e80feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:31:22.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.531+0000 7f88ee676700 1 -- 172.21.15.179:0/3216156212 shutdown_connections 2024-04-21T08:31:22.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.531+0000 7f88ee676700 1 -- 172.21.15.179:0/3216156212 wait complete. 2024-04-21T08:31:22.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.532+0000 7f88ee676700 1 Processor -- start 2024-04-21T08:31:22.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.532+0000 7f88ee676700 1 -- start start 2024-04-21T08:31:22.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.532+0000 7f88ee676700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88e8101c40 0x7f88e81016b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:31:22.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.532+0000 7f88ee676700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88e8102600 0x7f88e819fed0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:31:22.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.532+0000 7f88ee676700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88e8102f90 0x7f88e81a2420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:31:22.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.532+0000 7f88ee676700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f88e810d590 con 0x7f88e8102600 2024-04-21T08:31:22.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.532+0000 7f88ee676700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f88e810d410 con 0x7f88e8101c40 2024-04-21T08:31:22.534 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.532+0000 7f88ee676700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f88e810d710 con 0x7f88e8102f90 2024-04-21T08:31:22.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.532+0000 7f88ecc13700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88e8102f90 0x7f88e81a2420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:31:22.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.532+0000 7f88ecc13700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88e8102f90 0x7f88e81a2420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:43312/0 (socket says 172.21.15.179:43312) 2024-04-21T08:31:22.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.532+0000 7f88ecc13700 1 -- 172.21.15.179:0/1092167952 learned_addr learned my addr 172.21.15.179:0/1092167952 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-21T08:31:22.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.533+0000 7f88e77fe700 1 --2- 172.21.15.179:0/1092167952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88e8102600 0x7f88e819fed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:31:22.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.533+0000 7f88e7fff700 1 --2- 172.21.15.179:0/1092167952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88e8101c40 0x7f88e81016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:31:22.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.533+0000 7f88ecc13700 1 -- 172.21.15.179:0/1092167952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88e8101c40 msgr2=0x7f88e81016b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:31:22.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.533+0000 7f88ecc13700 1 --2- 172.21.15.179:0/1092167952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88e8101c40 0x7f88e81016b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:31:22.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.533+0000 7f88ecc13700 1 -- 172.21.15.179:0/1092167952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88e8102600 msgr2=0x7f88e819fed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:31:22.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.533+0000 7f88ecc13700 1 --2- 172.21.15.179:0/1092167952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88e8102600 0x7f88e819fed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:31:22.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.533+0000 7f88ecc13700 1 -- 172.21.15.179:0/1092167952 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f88e81a2c00 con 0x7f88e8102f90 2024-04-21T08:31:22.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.533+0000 7f88e7fff700 1 --2- 172.21.15.179:0/1092167952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88e8101c40 0x7f88e81016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:31:22.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.533+0000 7f88ecc13700 1 --2- 172.21.15.179:0/1092167952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88e8102f90 0x7f88e81a2420 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f88e000b970 tx=0x7f88e000bd30 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:31:22.535 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.533+0000 7f88e57fa700 1 -- 172.21.15.179:0/1092167952 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e0019070 con 0x7f88e8102f90 2024-04-21T08:31:22.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.533+0000 7f88e57fa700 1 -- 172.21.15.179:0/1092167952 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f88e0014440 con 0x7f88e8102f90 2024-04-21T08:31:22.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.533+0000 7f88ee676700 1 -- 172.21.15.179:0/1092167952 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f88e81a2ef0 con 0x7f88e8102f90 2024-04-21T08:31:22.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.533+0000 7f88e57fa700 1 -- 172.21.15.179:0/1092167952 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f88e000ce60 con 0x7f88e8102f90 2024-04-21T08:31:22.536 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.533+0000 7f88ee676700 1 -- 172.21.15.179:0/1092167952 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f88e81a3420 con 0x7f88e8102f90 2024-04-21T08:31:22.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.534+0000 7f88ee676700 1 -- 172.21.15.179:0/1092167952 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f88e8069730 con 0x7f88e8102f90 2024-04-21T08:31:22.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.535+0000 7f88e57fa700 1 -- 172.21.15.179:0/1092167952 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7f88e001c070 con 0x7f88e8102f90 2024-04-21T08:31:22.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.536+0000 7f88e57fa700 1 --2- 172.21.15.179:0/1092167952 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f88d00a3c30 0x7f88d00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:31:22.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.536+0000 7f88e57fa700 1 -- 172.21.15.179:0/1092167952 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(14..14 src has 1..14) ==== 2309+0+0 (secure 0 0 0) 0x7f88e00ce3d0 con 0x7f88e8102f90 2024-04-21T08:31:22.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.536+0000 7f88e7fff700 1 --2- 172.21.15.179:0/1092167952 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f88d00a3c30 0x7f88d00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:31:22.537 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.536+0000 7f88e7fff700 1 --2- 172.21.15.179:0/1092167952 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f88d00a3c30 0x7f88d00a60f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f88d8006010 tx=0x7f88d8005c40 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:31:22.539 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.539+0000 7f88e57fa700 1 -- 172.21.15.179:0/1092167952 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f88e009c600 con 0x7f88e8102f90 2024-04-21T08:31:22.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:22 smithi086 ceph-mon[27939]: Detected new or changed devices on smithi088 2024-04-21T08:31:22.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:22 smithi086 ceph-mon[27939]: Adjusting osd_memory_target on smithi088 to 17088M 2024-04-21T08:31:22.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:22 smithi086 ceph-mon[27939]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:22.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:22 smithi088 ceph-mon[31259]: Detected new or changed devices on smithi088 2024-04-21T08:31:22.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:22 smithi088 ceph-mon[31259]: Adjusting osd_memory_target on smithi088 to 17088M 2024-04-21T08:31:22.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:22 smithi088 ceph-mon[31259]: pgmap v62: 0 pgs: ; 0 B data, 26 MiB used, 89 GiB / 89 GiB avail 2024-04-21T08:31:22.743 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:22.744+0000 7f88ee676700 1 -- 172.21.15.179:0/1092167952 --> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_4", "target": ["mon-mgr", ""]}) -- 0x7f88e8062e80 con 0x7f88d00a3c30 2024-04-21T08:31:23.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:23 smithi179 ceph-mon[31829]: osdmap e14: 2 total, 2 up, 2 in 2024-04-21T08:31:23.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:23 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:31:23.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:23 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:31:23.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:23 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:23.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:23 smithi086 ceph-mon[27939]: osdmap e14: 2 total, 2 up, 2 in 2024-04-21T08:31:23.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:23 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:31:23.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:23 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:31:23.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:23 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:23.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:23 smithi088 ceph-mon[31259]: osdmap e14: 2 total, 2 up, 2 in 2024-04-21T08:31:23.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:23 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:31:23.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:23 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:31:23.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:23 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:24.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:24 smithi179 ceph-mon[31829]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:31:24.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:24 smithi179 ceph-mon[31829]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:24.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:24 smithi086 ceph-mon[27939]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:31:24.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:24 smithi086 ceph-mon[27939]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:24.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:24 smithi088 ceph-mon[31259]: from='client.24157 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_4", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:31:24.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:24 smithi088 ceph-mon[31259]: pgmap v64: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:26.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:26 smithi179 ceph-mon[31829]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:26.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:26 smithi179 ceph-mon[31829]: from='client.? 172.21.15.179:0/3117974624' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60d12aa4-9312-4fa2-8d0b-2b3f42caad00"}]: dispatch 2024-04-21T08:31:26.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:26 smithi179 ceph-mon[31829]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60d12aa4-9312-4fa2-8d0b-2b3f42caad00"}]: dispatch 2024-04-21T08:31:26.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:26 smithi179 ceph-mon[31829]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "60d12aa4-9312-4fa2-8d0b-2b3f42caad00"}]': finished 2024-04-21T08:31:26.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:26 smithi179 ceph-mon[31829]: osdmap e15: 3 total, 2 up, 3 in 2024-04-21T08:31:26.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:26 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:31:26.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:26 smithi179 ceph-mon[31829]: from='client.? 172.21.15.179:0/1641390293' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:31:26.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:26 smithi086 ceph-mon[27939]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:26.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:26 smithi086 ceph-mon[27939]: from='client.? 172.21.15.179:0/3117974624' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60d12aa4-9312-4fa2-8d0b-2b3f42caad00"}]: dispatch 2024-04-21T08:31:26.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:26 smithi086 ceph-mon[27939]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60d12aa4-9312-4fa2-8d0b-2b3f42caad00"}]: dispatch 2024-04-21T08:31:26.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:26 smithi086 ceph-mon[27939]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "60d12aa4-9312-4fa2-8d0b-2b3f42caad00"}]': finished 2024-04-21T08:31:26.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:26 smithi086 ceph-mon[27939]: osdmap e15: 3 total, 2 up, 3 in 2024-04-21T08:31:26.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:26 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:31:26.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:26 smithi086 ceph-mon[27939]: from='client.? 172.21.15.179:0/1641390293' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:31:26.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:26 smithi088 ceph-mon[31259]: pgmap v65: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:26.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:26 smithi088 ceph-mon[31259]: from='client.? 172.21.15.179:0/3117974624' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60d12aa4-9312-4fa2-8d0b-2b3f42caad00"}]: dispatch 2024-04-21T08:31:26.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:26 smithi088 ceph-mon[31259]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "60d12aa4-9312-4fa2-8d0b-2b3f42caad00"}]: dispatch 2024-04-21T08:31:26.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:26 smithi088 ceph-mon[31259]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "60d12aa4-9312-4fa2-8d0b-2b3f42caad00"}]': finished 2024-04-21T08:31:26.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:26 smithi088 ceph-mon[31259]: osdmap e15: 3 total, 2 up, 3 in 2024-04-21T08:31:26.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:26 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:31:26.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:26 smithi088 ceph-mon[31259]: from='client.? 172.21.15.179:0/1641390293' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:31:28.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:28 smithi179 ceph-mon[31829]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:28.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:28 smithi086 ceph-mon[27939]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:28.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:28 smithi088 ceph-mon[31259]: pgmap v67: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:30.485 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:30 smithi179 ceph-mon[31829]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:30.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:30 smithi086 ceph-mon[27939]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:30.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:30 smithi088 ceph-mon[31259]: pgmap v68: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:32.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:32 smithi086 ceph-mon[27939]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:32.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:32 smithi088 ceph-mon[31259]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:32.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:32 smithi179 ceph-mon[31829]: pgmap v69: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:33.519 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:33 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-21T08:31:33.520 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:33 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:33.520 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:33 smithi179 ceph-mon[31829]: Deploying daemon osd.2 on smithi179 2024-04-21T08:31:33.615 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:33 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-21T08:31:33.615 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:33 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:33.615 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:33 smithi088 ceph-mon[31259]: Deploying daemon osd.2 on smithi179 2024-04-21T08:31:33.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:33 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.2"}]: dispatch 2024-04-21T08:31:33.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:33 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:33.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:33 smithi086 ceph-mon[27939]: Deploying daemon osd.2 on smithi179 2024-04-21T08:31:34.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:34 smithi179 ceph-mon[31829]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:34.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:34 smithi086 ceph-mon[27939]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:34.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:34 smithi088 ceph-mon[31259]: pgmap v70: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:36.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:36 smithi179 ceph-mon[31829]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:36.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:36 smithi086 ceph-mon[27939]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:36.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:36 smithi088 ceph-mon[31259]: pgmap v71: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:38.438 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:38 smithi179 ceph-mon[31829]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:38.439 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:38 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:38.439 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:38 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:38.439 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:38 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:31:38.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:38 smithi086 ceph-mon[27939]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:38.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:38 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:38.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:38 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:38.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:38 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:31:38.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:38 smithi088 ceph-mon[31259]: pgmap v72: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:38.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:38 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:38.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:38 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:38.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:38 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:31:40.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:40 smithi086 ceph-mon[27939]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:40.631 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:40 smithi179 ceph-mon[31829]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:40.711 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:40.712+0000 7f88e57fa700 1 -- 172.21.15.179:0/1092167952 <== mgr.14152 v2:172.21.15.86:6800/1364156965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f88e8062e80 con 0x7f88d00a3c30 2024-04-21T08:31:40.711 INFO:teuthology.orchestra.run.smithi179.stdout:Created osd(s) 2 on host 'smithi179' 2024-04-21T08:31:40.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:40 smithi088 ceph-mon[31259]: pgmap v73: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:40.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:40.714+0000 7f88ee676700 1 -- 172.21.15.179:0/1092167952 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f88d00a3c30 msgr2=0x7f88d00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:31:40.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:40.714+0000 7f88ee676700 1 --2- 172.21.15.179:0/1092167952 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f88d00a3c30 0x7f88d00a60f0 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f88d8006010 tx=0x7f88d8005c40 comp rx=0 tx=0).stop 2024-04-21T08:31:40.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:40.714+0000 7f88ee676700 1 -- 172.21.15.179:0/1092167952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88e8102f90 msgr2=0x7f88e81a2420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:31:40.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:40.714+0000 7f88ee676700 1 --2- 172.21.15.179:0/1092167952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88e8102f90 0x7f88e81a2420 secure :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0x7f88e000b970 tx=0x7f88e000bd30 comp rx=0 tx=0).stop 2024-04-21T08:31:40.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:40.714+0000 7f88ee676700 1 -- 172.21.15.179:0/1092167952 shutdown_connections 2024-04-21T08:31:40.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:40.714+0000 7f88ee676700 1 --2- 172.21.15.179:0/1092167952 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f88d00a3c30 0x7f88d00a60f0 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:31:40.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:40.714+0000 7f88ee676700 1 --2- 172.21.15.179:0/1092167952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f88e8102f90 0x7f88e81a2420 unknown :-1 s=CLOSED pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:31:40.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:40.714+0000 7f88ee676700 1 --2- 172.21.15.179:0/1092167952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f88e8101c40 0x7f88e81016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:31:40.714 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:40.714+0000 7f88ee676700 1 --2- 172.21.15.179:0/1092167952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f88e8102600 0x7f88e819fed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:31:40.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:40.714+0000 7f88ee676700 1 -- 172.21.15.179:0/1092167952 >> 172.21.15.179:0/1092167952 conn(0x7f88e80fc6d0 msgr2=0x7f88e8106020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:31:40.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:40.714+0000 7f88ee676700 1 -- 172.21.15.179:0/1092167952 shutdown_connections 2024-04-21T08:31:40.715 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:31:40.714+0000 7f88ee676700 1 -- 172.21.15.179:0/1092167952 wait complete. 2024-04-21T08:31:41.604 DEBUG:teuthology.orchestra.run.smithi179:osd.2> sudo journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@osd.2.service 2024-04-21T08:31:41.606 INFO:tasks.cephadm:Deploying osd.3 on smithi086 with /dev/vg_nvme/lv_3... 2024-04-21T08:31:41.606 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-21T08:31:41.861 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:31:41.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:41 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:41.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:41 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:41.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:41 smithi088 ceph-mon[31259]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:41.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:41 smithi088 ceph-mon[31259]: from='osd.2 [v2:172.21.15.179:6800/4137659051,v1:172.21.15.179:6801/4137659051]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-21T08:31:41.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:41 smithi088 ceph-mon[31259]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-21T08:31:41.983 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:41 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:41.983 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:41 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:41.983 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:41 smithi086 ceph-mon[27939]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:41.983 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:41 smithi086 ceph-mon[27939]: from='osd.2 [v2:172.21.15.179:6800/4137659051,v1:172.21.15.179:6801/4137659051]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-21T08:31:41.983 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:41 smithi086 ceph-mon[27939]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-21T08:31:41.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:41 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:41.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:41 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:41.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:41 smithi179 ceph-mon[31829]: pgmap v74: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:41.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:41 smithi179 ceph-mon[31829]: from='osd.2 [v2:172.21.15.179:6800/4137659051,v1:172.21.15.179:6801/4137659051]' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-21T08:31:41.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:41 smithi179 ceph-mon[31829]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]: dispatch 2024-04-21T08:31:42.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:42 smithi088 ceph-mon[31259]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-21T08:31:42.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:42 smithi088 ceph-mon[31259]: osdmap e16: 3 total, 2 up, 3 in 2024-04-21T08:31:42.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:42 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:31:42.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:42 smithi088 ceph-mon[31259]: from='osd.2 [v2:172.21.15.179:6800/4137659051,v1:172.21.15.179:6801/4137659051]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-21T08:31:42.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:42 smithi088 ceph-mon[31259]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-21T08:31:42.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:42 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:42.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:42 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:42.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:42 smithi179 ceph-mon[31829]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-21T08:31:42.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:42 smithi179 ceph-mon[31829]: osdmap e16: 3 total, 2 up, 3 in 2024-04-21T08:31:42.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:42 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:31:42.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:42 smithi179 ceph-mon[31829]: from='osd.2 [v2:172.21.15.179:6800/4137659051,v1:172.21.15.179:6801/4137659051]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-21T08:31:42.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:42 smithi179 ceph-mon[31829]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-21T08:31:42.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:42 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:42.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:42 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:43.050 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:42 smithi086 ceph-mon[27939]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["2"]}]': finished 2024-04-21T08:31:43.051 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:42 smithi086 ceph-mon[27939]: osdmap e16: 3 total, 2 up, 3 in 2024-04-21T08:31:43.051 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:42 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:31:43.051 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:42 smithi086 ceph-mon[27939]: from='osd.2 [v2:172.21.15.179:6800/4137659051,v1:172.21.15.179:6801/4137659051]' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-21T08:31:43.051 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:42 smithi086 ceph-mon[27939]: from='osd.2 ' entity='osd.2' cmd=[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-21T08:31:43.051 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:42 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:43.051 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:42 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:43.568 INFO:journalctl@ceph.osd.2.smithi179.stdout:Apr 21 08:31:43 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-2[36968]: 2024-04-21T08:31:43.354+0000 7f776b8cb700 -1 osd.2 0 waiting for initial osdmap 2024-04-21T08:31:43.568 INFO:journalctl@ceph.osd.2.smithi179.stdout:Apr 21 08:31:43 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-2[36968]: 2024-04-21T08:31:43.358+0000 7f77636b6700 -1 osd.2 17 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T08:31:43.743 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-21T08:31:43.743 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-21T08:31:43.744 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-04-21T08:31:43.744 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-04-21T08:31:43.744 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0114452 s, 916 MB/s 2024-04-21T08:31:43.744 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-04-21T08:31:43.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:43 smithi088 ceph-mon[31259]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-04-21T08:31:43.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:43 smithi088 ceph-mon[31259]: osdmap e17: 3 total, 2 up, 3 in 2024-04-21T08:31:43.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:43 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:31:43.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:43 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:31:43.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:43 smithi088 ceph-mon[31259]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:43.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:43 smithi088 ceph-mon[31259]: from='osd.2 ' entity='osd.2' 2024-04-21T08:31:43.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:43 smithi179 ceph-mon[31829]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-04-21T08:31:43.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:43 smithi179 ceph-mon[31829]: osdmap e17: 3 total, 2 up, 3 in 2024-04-21T08:31:43.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:43 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:31:43.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:43 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:31:43.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:43 smithi179 ceph-mon[31829]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:43.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:43 smithi179 ceph-mon[31829]: from='osd.2 ' entity='osd.2' 2024-04-21T08:31:44.068 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:43 smithi086 ceph-mon[27939]: from='osd.2 ' entity='osd.2' cmd='[{"prefix": "osd crush create-or-move", "id": 2, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-04-21T08:31:44.068 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:43 smithi086 ceph-mon[27939]: osdmap e17: 3 total, 2 up, 3 in 2024-04-21T08:31:44.068 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:43 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:31:44.068 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:43 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:31:44.068 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:43 smithi086 ceph-mon[27939]: pgmap v77: 0 pgs: ; 0 B data, 61 MiB used, 179 GiB / 179 GiB avail 2024-04-21T08:31:44.068 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:43 smithi086 ceph-mon[27939]: from='osd.2 ' entity='osd.2' 2024-04-21T08:31:44.369 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi086:vg_nvme/lv_3 2024-04-21T08:31:44.609 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:31:44.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:44 smithi088 ceph-mon[31259]: purged_snaps scrub starts 2024-04-21T08:31:44.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:44 smithi088 ceph-mon[31259]: purged_snaps scrub ok 2024-04-21T08:31:44.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:44 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:31:44.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:44 smithi088 ceph-mon[31259]: osd.2 [v2:172.21.15.179:6800/4137659051,v1:172.21.15.179:6801/4137659051] boot 2024-04-21T08:31:44.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:44 smithi088 ceph-mon[31259]: osdmap e18: 3 total, 3 up, 3 in 2024-04-21T08:31:44.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:44 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:31:44.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:44 smithi086 ceph-mon[27939]: purged_snaps scrub starts 2024-04-21T08:31:44.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:44 smithi086 ceph-mon[27939]: purged_snaps scrub ok 2024-04-21T08:31:44.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:44 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:31:44.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:44 smithi086 ceph-mon[27939]: osd.2 [v2:172.21.15.179:6800/4137659051,v1:172.21.15.179:6801/4137659051] boot 2024-04-21T08:31:44.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:44 smithi086 ceph-mon[27939]: osdmap e18: 3 total, 3 up, 3 in 2024-04-21T08:31:44.967 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:44 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:31:44.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:44 smithi179 ceph-mon[31829]: purged_snaps scrub starts 2024-04-21T08:31:44.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:44 smithi179 ceph-mon[31829]: purged_snaps scrub ok 2024-04-21T08:31:44.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:44 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:31:44.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:44 smithi179 ceph-mon[31829]: osd.2 [v2:172.21.15.179:6800/4137659051,v1:172.21.15.179:6801/4137659051] boot 2024-04-21T08:31:44.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:44 smithi179 ceph-mon[31829]: osdmap e18: 3 total, 3 up, 3 in 2024-04-21T08:31:44.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:44 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:31:46.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.110+0000 7ff0524f9700 1 -- 172.21.15.86:0/3222280669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff04c103540 msgr2=0x7ff04c109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:31:46.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.110+0000 7ff0524f9700 1 --2- 172.21.15.86:0/3222280669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff04c103540 0x7ff04c109df0 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7ff04400b0d0 tx=0x7ff04400b3e0 comp rx=0 tx=0).stop 2024-04-21T08:31:46.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.110+0000 7ff0524f9700 1 -- 172.21.15.86:0/3222280669 shutdown_connections 2024-04-21T08:31:46.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.110+0000 7ff0524f9700 1 --2- 172.21.15.86:0/3222280669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff04c103540 0x7ff04c109df0 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:31:46.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.110+0000 7ff0524f9700 1 --2- 172.21.15.86:0/3222280669 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff04c1019a0 0x7ff04c101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:31:46.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.110+0000 7ff0524f9700 1 --2- 172.21.15.86:0/3222280669 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff04c102ba0 0x7ff04c103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:31:46.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.110+0000 7ff0524f9700 1 -- 172.21.15.86:0/3222280669 >> 172.21.15.86:0/3222280669 conn(0x7ff04c0fcf40 msgr2=0x7ff04c0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:31:46.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.110+0000 7ff0524f9700 1 -- 172.21.15.86:0/3222280669 shutdown_connections 2024-04-21T08:31:46.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.110+0000 7ff0524f9700 1 -- 172.21.15.86:0/3222280669 wait complete. 2024-04-21T08:31:46.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.111+0000 7ff0524f9700 1 Processor -- start 2024-04-21T08:31:46.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.111+0000 7ff0524f9700 1 -- start start 2024-04-21T08:31:46.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.111+0000 7ff0524f9700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff04c1019a0 0x7ff04c06bd30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:31:46.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.111+0000 7ff0524f9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff04c102ba0 0x7ff04c19da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:31:46.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.111+0000 7ff0524f9700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff04c103540 0x7ff04c1a0090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:31:46.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.111+0000 7ff0524f9700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff04c10d530 con 0x7ff04c1019a0 2024-04-21T08:31:46.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.111+0000 7ff0524f9700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff04c10d3b0 con 0x7ff04c103540 2024-04-21T08:31:46.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.111+0000 7ff0524f9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff04c10d6b0 con 0x7ff04c102ba0 2024-04-21T08:31:46.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.111+0000 7ff04bfff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff04c1019a0 0x7ff04c06bd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:31:46.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.111+0000 7ff04bfff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff04c1019a0 0x7ff04c06bd30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56002/0 (socket says 172.21.15.86:56002) 2024-04-21T08:31:46.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.111+0000 7ff04bfff700 1 -- 172.21.15.86:0/3942256537 learned_addr learned my addr 172.21.15.86:0/3942256537 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:31:46.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.111+0000 7ff050a96700 1 --2- 172.21.15.86:0/3942256537 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff04c103540 0x7ff04c1a0090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:31:46.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.111+0000 7ff04b7fe700 1 --2- 172.21.15.86:0/3942256537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff04c102ba0 0x7ff04c19da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:31:46.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.112+0000 7ff04bfff700 1 -- 172.21.15.86:0/3942256537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff04c102ba0 msgr2=0x7ff04c19da00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:31:46.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.112+0000 7ff04bfff700 1 --2- 172.21.15.86:0/3942256537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff04c102ba0 0x7ff04c19da00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:31:46.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.112+0000 7ff04bfff700 1 -- 172.21.15.86:0/3942256537 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff04c103540 msgr2=0x7ff04c1a0090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:31:46.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.112+0000 7ff04bfff700 1 --2- 172.21.15.86:0/3942256537 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff04c103540 0x7ff04c1a0090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:31:46.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.112+0000 7ff04bfff700 1 -- 172.21.15.86:0/3942256537 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff04c1a0960 con 0x7ff04c1019a0 2024-04-21T08:31:46.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.112+0000 7ff04b7fe700 1 --2- 172.21.15.86:0/3942256537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff04c102ba0 0x7ff04c19da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:31:46.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.112+0000 7ff04bfff700 1 --2- 172.21.15.86:0/3942256537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff04c1019a0 0x7ff04c06bd30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff03c00b500 tx=0x7ff03c00b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:31:46.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.112+0000 7ff0497fa700 1 -- 172.21.15.86:0/3942256537 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03c015020 con 0x7ff04c1019a0 2024-04-21T08:31:46.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.112+0000 7ff0497fa700 1 -- 172.21.15.86:0/3942256537 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff03c011ce0 con 0x7ff04c1019a0 2024-04-21T08:31:46.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.112+0000 7ff0524f9700 1 -- 172.21.15.86:0/3942256537 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff04c1a0c50 con 0x7ff04c1019a0 2024-04-21T08:31:46.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.112+0000 7ff0497fa700 1 -- 172.21.15.86:0/3942256537 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff03c010d20 con 0x7ff04c1019a0 2024-04-21T08:31:46.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.112+0000 7ff0524f9700 1 -- 172.21.15.86:0/3942256537 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff04c1a1190 con 0x7ff04c1019a0 2024-04-21T08:31:46.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.113+0000 7ff0524f9700 1 -- 172.21.15.86:0/3942256537 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff04c069730 con 0x7ff04c1019a0 2024-04-21T08:31:46.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.114+0000 7ff0497fa700 1 -- 172.21.15.86:0/3942256537 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 15) ==== 138590+0+0 (secure 0 0 0) 0x7ff03c018020 con 0x7ff04c1019a0 2024-04-21T08:31:46.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.114+0000 7ff0497fa700 1 --2- 172.21.15.86:0/3942256537 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ff0340a3bb0 0x7ff0340a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:31:46.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.115+0000 7ff04b7fe700 1 --2- 172.21.15.86:0/3942256537 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ff0340a3bb0 0x7ff0340a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:31:46.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.115+0000 7ff0497fa700 1 -- 172.21.15.86:0/3942256537 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(19..19 src has 1..19) ==== 3226+0+0 (secure 0 0 0) 0x7ff03c0cebc0 con 0x7ff04c1019a0 2024-04-21T08:31:46.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.115+0000 7ff04b7fe700 1 --2- 172.21.15.86:0/3942256537 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ff0340a3bb0 0x7ff0340a6070 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff0400099d0 tx=0x7ff040008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:31:46.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.118+0000 7ff0497fa700 1 -- 172.21.15.86:0/3942256537 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff03c01f020 con 0x7ff04c1019a0 2024-04-21T08:31:46.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:46.328+0000 7ff0524f9700 1 -- 172.21.15.86:0/3942256537 --> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7ff04c062e80 con 0x7ff0340a3bb0 2024-04-21T08:31:46.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:46 smithi086 ceph-mon[27939]: Detected new or changed devices on smithi179 2024-04-21T08:31:46.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:46 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:46.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:46 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:46.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:46 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:31:46.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:46 smithi086 ceph-mon[27939]: Adjusting osd_memory_target on smithi179 to 17088M 2024-04-21T08:31:46.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:46 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:46.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:46 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:46.329 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:46 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:31:46.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:46 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:46.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:46 smithi086 ceph-mon[27939]: pgmap v79: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:46.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:46 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:31:46.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:46 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:46.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:46 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:31:46.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:46 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:46.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:46 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-21T08:31:46.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:46 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-21T08:31:46.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:46 smithi086 ceph-mon[27939]: osdmap e19: 3 total, 3 up, 3 in 2024-04-21T08:31:46.330 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:46 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-21T08:31:46.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:46 smithi088 ceph-mon[31259]: Detected new or changed devices on smithi179 2024-04-21T08:31:46.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:46 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:46.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:46 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:46.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:46 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:31:46.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:46 smithi088 ceph-mon[31259]: Adjusting osd_memory_target on smithi179 to 17088M 2024-04-21T08:31:46.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:46 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:46.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:46 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:46.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:46 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:31:46.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:46 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:46.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:46 smithi088 ceph-mon[31259]: pgmap v79: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:46.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:46 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:31:46.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:46 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:46.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:46 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:31:46.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:46 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:46.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:46 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-21T08:31:46.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:46 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-21T08:31:46.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:46 smithi088 ceph-mon[31259]: osdmap e19: 3 total, 3 up, 3 in 2024-04-21T08:31:46.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:46 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-21T08:31:46.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:46 smithi179 ceph-mon[31829]: Detected new or changed devices on smithi179 2024-04-21T08:31:46.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:46 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:46.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:46 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:46.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:46 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:31:46.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:46 smithi179 ceph-mon[31829]: Adjusting osd_memory_target on smithi179 to 17088M 2024-04-21T08:31:46.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:46 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:46.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:46 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:46.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:46 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:31:46.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:46 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:46.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:46 smithi179 ceph-mon[31829]: pgmap v79: 0 pgs: ; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:46.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:46 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:31:46.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:46 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:46.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:46 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:31:46.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:46 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:46.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:46 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]: dispatch 2024-04-21T08:31:46.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:46 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pool create", "format": "json", "pool": ".mgr", "pg_num": 1, "pg_num_min": 1, "pg_num_max": 32, "yes_i_really_mean_it": true}]': finished 2024-04-21T08:31:46.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:46 smithi179 ceph-mon[31829]: osdmap e19: 3 total, 3 up, 3 in 2024-04-21T08:31:46.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:46 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]: dispatch 2024-04-21T08:31:46.616 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 21 08:31:46 smithi086 sudo[45150]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-21T08:31:46.616 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 21 08:31:46 smithi086 sudo[45150]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T08:31:46.616 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 21 08:31:46 smithi086 sudo[45150]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T08:31:46.617 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 21 08:31:46 smithi086 sudo[45150]: pam_unix(sudo:session): session closed for user root 2024-04-21T08:31:46.617 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 21 08:31:46 smithi086 sudo[45153]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-21T08:31:46.617 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 21 08:31:46 smithi086 sudo[45153]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T08:31:46.617 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 21 08:31:46 smithi086 sudo[45153]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T08:31:46.617 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 21 08:31:46 smithi086 sudo[45153]: pam_unix(sudo:session): session closed for user root 2024-04-21T08:31:46.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:46 smithi088 sudo[39702]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-21T08:31:46.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:46 smithi088 sudo[39702]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T08:31:46.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:46 smithi088 sudo[39702]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T08:31:46.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:46 smithi088 sudo[39702]: pam_unix(sudo:session): session closed for user root 2024-04-21T08:31:46.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:46 smithi088 sudo[39705]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-21T08:31:46.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:46 smithi088 sudo[39705]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T08:31:46.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:46 smithi088 sudo[39705]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T08:31:46.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:46 smithi088 sudo[39705]: pam_unix(sudo:session): session closed for user root 2024-04-21T08:31:46.964 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 21 08:31:46 smithi088 sudo[39696]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-21T08:31:46.964 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 21 08:31:46 smithi088 sudo[39696]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T08:31:46.965 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 21 08:31:46 smithi088 sudo[39696]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T08:31:46.965 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 21 08:31:46 smithi088 sudo[39696]: pam_unix(sudo:session): session closed for user root 2024-04-21T08:31:46.965 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 21 08:31:46 smithi088 sudo[39699]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-21T08:31:46.965 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 21 08:31:46 smithi088 sudo[39699]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T08:31:46.965 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 21 08:31:46 smithi088 sudo[39699]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T08:31:46.965 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 21 08:31:46 smithi088 sudo[39699]: pam_unix(sudo:session): session closed for user root 2024-04-21T08:31:46.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:46 smithi086 sudo[45156]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-21T08:31:46.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:46 smithi086 sudo[45156]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T08:31:46.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:46 smithi086 sudo[45156]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T08:31:46.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:46 smithi086 sudo[45156]: pam_unix(sudo:session): session closed for user root 2024-04-21T08:31:46.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:46 smithi086 sudo[45159]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-21T08:31:46.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:46 smithi086 sudo[45159]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T08:31:46.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:46 smithi086 sudo[45159]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T08:31:46.968 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:46 smithi086 sudo[45159]: pam_unix(sudo:session): session closed for user root 2024-04-21T08:31:46.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:46 smithi179 sudo[40590]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-21T08:31:46.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:46 smithi179 sudo[40590]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T08:31:46.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:46 smithi179 sudo[40590]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T08:31:46.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:46 smithi179 sudo[40590]: pam_unix(sudo:session): session closed for user root 2024-04-21T08:31:46.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:46 smithi179 sudo[40593]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-21T08:31:46.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:46 smithi179 sudo[40593]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T08:31:46.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:46 smithi179 sudo[40593]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T08:31:46.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:46 smithi179 sudo[40593]: pam_unix(sudo:session): session closed for user root 2024-04-21T08:31:46.997 INFO:journalctl@ceph.osd.2.smithi179.stdout:Apr 21 08:31:46 smithi179 sudo[40584]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/smartctl -x --json=o /dev/nvme0n1 2024-04-21T08:31:46.997 INFO:journalctl@ceph.osd.2.smithi179.stdout:Apr 21 08:31:46 smithi179 sudo[40584]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T08:31:46.997 INFO:journalctl@ceph.osd.2.smithi179.stdout:Apr 21 08:31:46 smithi179 sudo[40584]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T08:31:46.997 INFO:journalctl@ceph.osd.2.smithi179.stdout:Apr 21 08:31:46 smithi179 sudo[40584]: pam_unix(sudo:session): session closed for user root 2024-04-21T08:31:46.997 INFO:journalctl@ceph.osd.2.smithi179.stdout:Apr 21 08:31:46 smithi179 sudo[40587]: ceph : PWD=/ ; USER=root ; COMMAND=/usr/sbin/nvme intel smart-log-add --json /dev/nvme0n1 2024-04-21T08:31:46.997 INFO:journalctl@ceph.osd.2.smithi179.stdout:Apr 21 08:31:46 smithi179 sudo[40587]: pam_systemd(sudo:session): Failed to connect to system bus: No such file or directory 2024-04-21T08:31:46.997 INFO:journalctl@ceph.osd.2.smithi179.stdout:Apr 21 08:31:46 smithi179 sudo[40587]: pam_unix(sudo:session): session opened for user root by (uid=167) 2024-04-21T08:31:46.997 INFO:journalctl@ceph.osd.2.smithi179.stdout:Apr 21 08:31:46 smithi179 sudo[40587]: pam_unix(sudo:session): session closed for user root 2024-04-21T08:31:47.286 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:47 smithi086 ceph-mon[27939]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:31:47.287 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:47 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:31:47.287 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:47 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:31:47.287 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:47 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:47.287 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:47 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-21T08:31:47.287 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:47 smithi086 ceph-mon[27939]: osdmap e20: 3 total, 3 up, 3 in 2024-04-21T08:31:47.287 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:47 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:47.287 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:47 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:47.287 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:47 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:47.287 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:47 smithi086 ceph-mon[27939]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T08:31:47.287 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:47 smithi086 ceph-mon[27939]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T08:31:47.287 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:47 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:31:47.287 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:47 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:31:47.287 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:47 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:31:47.287 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:47 smithi086 ceph-mon[27939]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T08:31:47.287 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:47 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:31:47.288 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:47 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:31:47.288 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:47 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:31:47.288 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:47 smithi086 ceph-mon[27939]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T08:31:47.288 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:47 smithi086 ceph-mon[27939]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T08:31:47.288 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:47 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:31:47.288 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:47 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:31:47.288 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:47 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:31:47.288 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:47 smithi086 ceph-mon[27939]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T08:31:47.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:47 smithi088 ceph-mon[31259]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:31:47.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:47 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:31:47.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:47 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:31:47.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:47 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:47.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:47 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-21T08:31:47.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:47 smithi088 ceph-mon[31259]: osdmap e20: 3 total, 3 up, 3 in 2024-04-21T08:31:47.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:47 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:47.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:47 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:47.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:47 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:47.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:47 smithi088 ceph-mon[31259]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T08:31:47.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:47 smithi088 ceph-mon[31259]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T08:31:47.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:47 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:31:47.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:47 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:31:47.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:47 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:31:47.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:47 smithi088 ceph-mon[31259]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T08:31:47.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:47 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:31:47.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:47 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:31:47.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:47 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:31:47.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:47 smithi088 ceph-mon[31259]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T08:31:47.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:47 smithi088 ceph-mon[31259]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T08:31:47.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:47 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:31:47.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:47 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:31:47.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:47 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:31:47.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:47 smithi088 ceph-mon[31259]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T08:31:47.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:47 smithi179 ceph-mon[31829]: from='client.14286 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:31:47.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:47 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:31:47.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:47 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:31:47.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:47 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:47.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:47 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pool application enable", "format": "json", "pool": ".mgr", "app": "mgr", "yes_i_really_mean_it": true}]': finished 2024-04-21T08:31:47.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:47 smithi179 ceph-mon[31829]: osdmap e20: 3 total, 3 up, 3 in 2024-04-21T08:31:47.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:47 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:47.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:47 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:47.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:47 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:31:47.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:47 smithi179 ceph-mon[31829]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T08:31:47.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:47 smithi179 ceph-mon[31829]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T08:31:47.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:47 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:31:47.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:47 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:31:47.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:47 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:31:47.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:47 smithi179 ceph-mon[31829]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T08:31:47.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:47 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:31:47.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:47 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:31:47.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:47 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:31:47.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:47 smithi179 ceph-mon[31829]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T08:31:47.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:47 smithi179 ceph-mon[31829]: from='admin socket' entity='admin socket' cmd='smart' args=[json]: dispatch 2024-04-21T08:31:47.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:47 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:31:47.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:47 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:31:47.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:47 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:31:47.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:47 smithi179 ceph-mon[31829]: from='admin socket' entity='admin socket' cmd=smart args=[json]: finished 2024-04-21T08:31:48.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:31:48.107+0000 7ff0497fa700 1 -- 172.21.15.86:0/3942256537 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff03c010380 con 0x7ff04c1019a0 2024-04-21T08:31:48.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:48 smithi086 ceph-mon[27939]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:48.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:48 smithi086 ceph-mon[27939]: osdmap e21: 3 total, 3 up, 3 in 2024-04-21T08:31:48.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:48 smithi086 ceph-mon[27939]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-21T08:31:48.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:48 smithi088 ceph-mon[31259]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:48.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:48 smithi088 ceph-mon[31259]: osdmap e21: 3 total, 3 up, 3 in 2024-04-21T08:31:48.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:48 smithi088 ceph-mon[31259]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-21T08:31:48.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:48 smithi179 ceph-mon[31829]: pgmap v82: 1 pgs: 1 creating+peering; 0 B data, 92 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:48.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:48 smithi179 ceph-mon[31829]: osdmap e21: 3 total, 3 up, 3 in 2024-04-21T08:31:48.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:48 smithi179 ceph-mon[31829]: mgrmap e16: x(active, since 2m), standbys: z, y 2024-04-21T08:31:49.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:49 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3770083078' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "404c3158-8cb0-4c02-b48a-f9ee5ff7b784"}]: dispatch 2024-04-21T08:31:49.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:49 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3770083078' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "404c3158-8cb0-4c02-b48a-f9ee5ff7b784"}]': finished 2024-04-21T08:31:49.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:49 smithi086 ceph-mon[27939]: osdmap e22: 4 total, 3 up, 4 in 2024-04-21T08:31:49.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:49 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:31:49.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:49 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3770083078' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "404c3158-8cb0-4c02-b48a-f9ee5ff7b784"}]: dispatch 2024-04-21T08:31:49.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:49 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3770083078' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "404c3158-8cb0-4c02-b48a-f9ee5ff7b784"}]': finished 2024-04-21T08:31:49.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:49 smithi088 ceph-mon[31259]: osdmap e22: 4 total, 3 up, 4 in 2024-04-21T08:31:49.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:49 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:31:49.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:49 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3770083078' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "404c3158-8cb0-4c02-b48a-f9ee5ff7b784"}]: dispatch 2024-04-21T08:31:49.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:49 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3770083078' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "404c3158-8cb0-4c02-b48a-f9ee5ff7b784"}]': finished 2024-04-21T08:31:49.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:49 smithi179 ceph-mon[31829]: osdmap e22: 4 total, 3 up, 4 in 2024-04-21T08:31:49.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:49 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:31:50.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:50 smithi088 ceph-mon[31259]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:50.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:50 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2406436053' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:31:50.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:50 smithi179 ceph-mon[31829]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:50.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:50 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2406436053' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:31:50.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:50 smithi086 ceph-mon[27939]: pgmap v85: 1 pgs: 1 creating+peering; 0 B data, 93 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:50.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:50 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2406436053' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:31:52.620 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:52 smithi086 ceph-mon[27939]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:52.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:52 smithi088 ceph-mon[31259]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:52.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:52 smithi179 ceph-mon[31829]: pgmap v86: 1 pgs: 1 creating+peering; 0 B data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:54.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:54 smithi086 ceph-mon[27939]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:54.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:54 smithi088 ceph-mon[31259]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:54.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:54 smithi179 ceph-mon[31829]: pgmap v87: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:56.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:56 smithi086 ceph-mon[27939]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:56.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:56 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-21T08:31:56.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:56 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:56.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:56 smithi088 ceph-mon[31259]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:56.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:56 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-21T08:31:56.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:56 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:56.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:56 smithi179 ceph-mon[31829]: pgmap v88: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:56.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:56 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.3"}]: dispatch 2024-04-21T08:31:56.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:56 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:31:57.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:57 smithi086 ceph-mon[27939]: Deploying daemon osd.3 on smithi086 2024-04-21T08:31:57.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:57 smithi088 ceph-mon[31259]: Deploying daemon osd.3 on smithi086 2024-04-21T08:31:57.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:57 smithi179 ceph-mon[31829]: Deploying daemon osd.3 on smithi086 2024-04-21T08:31:58.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:31:58 smithi088 ceph-mon[31259]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:58.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:31:58 smithi179 ceph-mon[31829]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:31:58.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:31:58 smithi086 ceph-mon[27939]: pgmap v89: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:32:00.662 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:00 smithi086 ceph-mon[27939]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:32:00.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:00 smithi088 ceph-mon[31259]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:32:00.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:00 smithi179 ceph-mon[31829]: pgmap v90: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:32:02.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:02 smithi086 ceph-mon[27939]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:32:02.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:02 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:02.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:02 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:02.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:02 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:32:02.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:02 smithi088 ceph-mon[31259]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:32:02.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:02 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:02.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:02 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:02.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:02 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:32:02.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:02 smithi179 ceph-mon[31829]: pgmap v91: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:32:02.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:02 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:02.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:02 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:02.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:02 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:32:04.581 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:04 smithi086 ceph-mon[27939]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:32:04.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:04 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:04.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:04 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:04.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:04 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:04.582 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:04 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:32:04.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:04 smithi088 ceph-mon[31259]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:32:04.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:04 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:04.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:04 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:04.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:04 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:04.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:04 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:32:04.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:04 smithi179 ceph-mon[31829]: pgmap v92: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:32:04.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:04 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:04.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:04 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:04.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:04 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:04.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:04 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:32:04.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:04.955+0000 7ff0497fa700 1 -- 172.21.15.86:0/3942256537 <== mgr.14152 v2:172.21.15.86:6800/1364156965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7ff04c062e80 con 0x7ff0340a3bb0 2024-04-21T08:32:04.955 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 3 on host 'smithi086' 2024-04-21T08:32:04.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:04.958+0000 7ff0524f9700 1 -- 172.21.15.86:0/3942256537 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ff0340a3bb0 msgr2=0x7ff0340a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:32:04.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:04.958+0000 7ff0524f9700 1 --2- 172.21.15.86:0/3942256537 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ff0340a3bb0 0x7ff0340a6070 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ff0400099d0 tx=0x7ff040008040 comp rx=0 tx=0).stop 2024-04-21T08:32:04.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:04.958+0000 7ff0524f9700 1 -- 172.21.15.86:0/3942256537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff04c1019a0 msgr2=0x7ff04c06bd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:32:04.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:04.958+0000 7ff0524f9700 1 --2- 172.21.15.86:0/3942256537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff04c1019a0 0x7ff04c06bd30 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7ff03c00b500 tx=0x7ff03c00b810 comp rx=0 tx=0).stop 2024-04-21T08:32:04.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:04.958+0000 7ff0524f9700 1 -- 172.21.15.86:0/3942256537 shutdown_connections 2024-04-21T08:32:04.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:04.958+0000 7ff0524f9700 1 --2- 172.21.15.86:0/3942256537 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ff0340a3bb0 0x7ff0340a6070 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:04.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:04.958+0000 7ff0524f9700 1 --2- 172.21.15.86:0/3942256537 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff04c103540 0x7ff04c1a0090 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:04.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:04.958+0000 7ff0524f9700 1 --2- 172.21.15.86:0/3942256537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff04c1019a0 0x7ff04c06bd30 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:04.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:04.958+0000 7ff0524f9700 1 --2- 172.21.15.86:0/3942256537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff04c102ba0 0x7ff04c19da00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:04.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:04.958+0000 7ff0524f9700 1 -- 172.21.15.86:0/3942256537 >> 172.21.15.86:0/3942256537 conn(0x7ff04c0fcf40 msgr2=0x7ff04c105ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:32:04.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:04.958+0000 7ff0524f9700 1 -- 172.21.15.86:0/3942256537 shutdown_connections 2024-04-21T08:32:04.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:04.958+0000 7ff0524f9700 1 -- 172.21.15.86:0/3942256537 wait complete. 2024-04-21T08:32:05.545 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:05 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:05.545 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:05 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:05.545 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:05 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:05.545 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:05 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:32:05.572 DEBUG:teuthology.orchestra.run.smithi086:osd.3> sudo journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@osd.3.service 2024-04-21T08:32:05.575 INFO:tasks.cephadm:Deploying osd.4 on smithi088 with /dev/vg_nvme/lv_3... 2024-04-21T08:32:05.575 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-21T08:32:05.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:05 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:05.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:05 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:05.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:05 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:05.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:05 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:32:05.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:05 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:05.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:05 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:05.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:05 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:05.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:05 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:32:05.818 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T08:32:06.463 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:06 smithi088 ceph-mon[31259]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:32:06.463 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:06 smithi088 ceph-mon[31259]: from='osd.3 [v2:172.21.15.86:6810/2309603148,v1:172.21.15.86:6811/2309603148]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-21T08:32:06.743 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:06 smithi086 ceph-mon[27939]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:32:06.743 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:06 smithi086 ceph-mon[27939]: from='osd.3 [v2:172.21.15.86:6810/2309603148,v1:172.21.15.86:6811/2309603148]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-21T08:32:06.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:06 smithi179 ceph-mon[31829]: pgmap v93: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:32:06.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:06 smithi179 ceph-mon[31829]: from='osd.3 [v2:172.21.15.86:6810/2309603148,v1:172.21.15.86:6811/2309603148]' entity='osd.3' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]: dispatch 2024-04-21T08:32:07.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:07 smithi088 ceph-mon[31259]: from='osd.3 [v2:172.21.15.86:6810/2309603148,v1:172.21.15.86:6811/2309603148]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-21T08:32:07.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:07 smithi088 ceph-mon[31259]: osdmap e23: 4 total, 3 up, 4 in 2024-04-21T08:32:07.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:07 smithi088 ceph-mon[31259]: from='osd.3 [v2:172.21.15.86:6810/2309603148,v1:172.21.15.86:6811/2309603148]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-21T08:32:07.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:07 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:32:07.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:07 smithi088 ceph-mon[31259]: from='osd.3 [v2:172.21.15.86:6810/2309603148,v1:172.21.15.86:6811/2309603148]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-21T08:32:07.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:07 smithi179 ceph-mon[31829]: from='osd.3 [v2:172.21.15.86:6810/2309603148,v1:172.21.15.86:6811/2309603148]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-21T08:32:07.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:07 smithi179 ceph-mon[31829]: osdmap e23: 4 total, 3 up, 4 in 2024-04-21T08:32:07.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:07 smithi179 ceph-mon[31829]: from='osd.3 [v2:172.21.15.86:6810/2309603148,v1:172.21.15.86:6811/2309603148]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-21T08:32:07.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:07 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:32:07.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:07 smithi179 ceph-mon[31829]: from='osd.3 [v2:172.21.15.86:6810/2309603148,v1:172.21.15.86:6811/2309603148]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-21T08:32:07.796 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:07 smithi086 ceph-mon[27939]: from='osd.3 [v2:172.21.15.86:6810/2309603148,v1:172.21.15.86:6811/2309603148]' entity='osd.3' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["3"]}]': finished 2024-04-21T08:32:07.796 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:07 smithi086 ceph-mon[27939]: osdmap e23: 4 total, 3 up, 4 in 2024-04-21T08:32:07.796 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:07 smithi086 ceph-mon[27939]: from='osd.3 [v2:172.21.15.86:6810/2309603148,v1:172.21.15.86:6811/2309603148]' entity='osd.3' cmd=[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-21T08:32:07.797 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:07 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:32:07.797 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:07 smithi086 ceph-mon[27939]: from='osd.3 [v2:172.21.15.86:6810/2309603148,v1:172.21.15.86:6811/2309603148]' entity='osd.3' cmd='[{"prefix": "osd crush create-or-move", "id": 3, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-21T08:32:07.797 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 21 08:32:07 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-3[47346]: 2024-04-21T08:32:07.407+0000 7f1ad9110700 -1 osd.3 0 waiting for initial osdmap 2024-04-21T08:32:07.797 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 21 08:32:07 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-3[47346]: 2024-04-21T08:32:07.414+0000 7f1ad4f03700 -1 osd.3 24 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T08:32:07.816 INFO:teuthology.orchestra.run.smithi088.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-21T08:32:07.816 INFO:teuthology.orchestra.run.smithi088.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-21T08:32:07.816 INFO:teuthology.orchestra.run.smithi088.stderr: stderr: 10+0 records in 2024-04-21T08:32:07.816 INFO:teuthology.orchestra.run.smithi088.stderr:10+0 records out 2024-04-21T08:32:07.816 INFO:teuthology.orchestra.run.smithi088.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0111013 s, 945 MB/s 2024-04-21T08:32:07.816 INFO:teuthology.orchestra.run.smithi088.stderr:--> Zapping successful for: 2024-04-21T08:32:08.475 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi088:vg_nvme/lv_3 2024-04-21T08:32:08.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:08 smithi088 ceph-mon[31259]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:32:08.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:08 smithi088 ceph-mon[31259]: osdmap e24: 4 total, 3 up, 4 in 2024-04-21T08:32:08.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:08 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:32:08.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:08 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:32:08.722 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T08:32:08.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:08 smithi179 ceph-mon[31829]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:32:08.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:08 smithi179 ceph-mon[31829]: osdmap e24: 4 total, 3 up, 4 in 2024-04-21T08:32:08.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:08 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:32:08.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:08 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:32:08.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:08 smithi086 ceph-mon[27939]: pgmap v95: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:32:08.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:08 smithi086 ceph-mon[27939]: osdmap e24: 4 total, 3 up, 4 in 2024-04-21T08:32:08.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:08 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:32:08.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:08 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:32:09.704 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:09 smithi088 ceph-mon[31259]: purged_snaps scrub starts 2024-04-21T08:32:09.704 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:09 smithi088 ceph-mon[31259]: purged_snaps scrub ok 2024-04-21T08:32:09.705 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:09 smithi088 ceph-mon[31259]: osd.3 [v2:172.21.15.86:6810/2309603148,v1:172.21.15.86:6811/2309603148] boot 2024-04-21T08:32:09.705 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:09 smithi088 ceph-mon[31259]: osdmap e25: 4 total, 4 up, 4 in 2024-04-21T08:32:09.705 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:09 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:32:09.705 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:09 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:09.705 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:09 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:09.705 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:09 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:32:09.705 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:09 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:32:09.705 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:09 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:09.705 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:09 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:09.705 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:09 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:32:09.705 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:09 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:09.705 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:09 smithi088 ceph-mon[31259]: osdmap e26: 4 total, 4 up, 4 in 2024-04-21T08:32:09.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:09 smithi179 ceph-mon[31829]: purged_snaps scrub starts 2024-04-21T08:32:09.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:09 smithi179 ceph-mon[31829]: purged_snaps scrub ok 2024-04-21T08:32:09.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:09 smithi179 ceph-mon[31829]: osd.3 [v2:172.21.15.86:6810/2309603148,v1:172.21.15.86:6811/2309603148] boot 2024-04-21T08:32:09.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:09 smithi179 ceph-mon[31829]: osdmap e25: 4 total, 4 up, 4 in 2024-04-21T08:32:09.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:09 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:32:09.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:09 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:09.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:09 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:09.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:09 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:32:09.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:09 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:32:09.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:09 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:09.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:09 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:09.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:09 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:32:09.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:09 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:09.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:09 smithi179 ceph-mon[31829]: osdmap e26: 4 total, 4 up, 4 in 2024-04-21T08:32:09.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:09 smithi086 ceph-mon[27939]: purged_snaps scrub starts 2024-04-21T08:32:09.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:09 smithi086 ceph-mon[27939]: purged_snaps scrub ok 2024-04-21T08:32:09.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:09 smithi086 ceph-mon[27939]: osd.3 [v2:172.21.15.86:6810/2309603148,v1:172.21.15.86:6811/2309603148] boot 2024-04-21T08:32:09.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:09 smithi086 ceph-mon[27939]: osdmap e25: 4 total, 4 up, 4 in 2024-04-21T08:32:09.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:09 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:32:09.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:09 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:09.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:09 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:09.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:09 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:32:09.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:09 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:32:09.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:09 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:09.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:09 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:09.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:09 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:32:09.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:09 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:09.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:09 smithi086 ceph-mon[27939]: osdmap e26: 4 total, 4 up, 4 in 2024-04-21T08:32:10.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.304+0000 7fbf7c96f700 1 -- 172.21.15.88:0/2318145461 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf78102ba0 msgr2=0x7fbf78103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:32:10.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.304+0000 7fbf7c96f700 1 --2- 172.21.15.88:0/2318145461 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf78102ba0 0x7fbf78103000 secure :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0x7fbf600099c0 tx=0x7fbf60009cd0 comp rx=0 tx=0).stop 2024-04-21T08:32:10.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.304+0000 7fbf7c96f700 1 -- 172.21.15.88:0/2318145461 shutdown_connections 2024-04-21T08:32:10.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.304+0000 7fbf7c96f700 1 --2- 172.21.15.88:0/2318145461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf78103540 0x7fbf78109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:10.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.304+0000 7fbf7c96f700 1 --2- 172.21.15.88:0/2318145461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf781019a0 0x7fbf78101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:10.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.304+0000 7fbf7c96f700 1 --2- 172.21.15.88:0/2318145461 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf78102ba0 0x7fbf78103000 unknown :-1 s=CLOSED pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:10.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.304+0000 7fbf7c96f700 1 -- 172.21.15.88:0/2318145461 >> 172.21.15.88:0/2318145461 conn(0x7fbf780fcf60 msgr2=0x7fbf780ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:32:10.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.304+0000 7fbf7c96f700 1 -- 172.21.15.88:0/2318145461 shutdown_connections 2024-04-21T08:32:10.304 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.304+0000 7fbf7c96f700 1 -- 172.21.15.88:0/2318145461 wait complete. 2024-04-21T08:32:10.306 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.305+0000 7fbf7c96f700 1 Processor -- start 2024-04-21T08:32:10.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.305+0000 7fbf7c96f700 1 -- start start 2024-04-21T08:32:10.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.305+0000 7fbf7c96f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf781019a0 0x7fbf78198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:32:10.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.305+0000 7fbf7c96f700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf78102ba0 0x7fbf78199160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:32:10.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.305+0000 7fbf7c96f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf78103540 0x7fbf781a01a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:32:10.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.305+0000 7fbf7c96f700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7810d440 con 0x7fbf78102ba0 2024-04-21T08:32:10.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.305+0000 7fbf7c96f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7810d2c0 con 0x7fbf781019a0 2024-04-21T08:32:10.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.305+0000 7fbf7c96f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbf7810d5c0 con 0x7fbf78103540 2024-04-21T08:32:10.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.305+0000 7fbf7659c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf781019a0 0x7fbf78198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:32:10.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.305+0000 7fbf7659c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf781019a0 0x7fbf78198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:35082/0 (socket says 172.21.15.88:35082) 2024-04-21T08:32:10.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.305+0000 7fbf7659c700 1 -- 172.21.15.88:0/3622896977 learned_addr learned my addr 172.21.15.88:0/3622896977 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-21T08:32:10.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.305+0000 7fbf75d9b700 1 --2- 172.21.15.88:0/3622896977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf78102ba0 0x7fbf78199160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:32:10.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.305+0000 7fbf76d9d700 1 --2- 172.21.15.88:0/3622896977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf78103540 0x7fbf781a01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:32:10.307 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.305+0000 7fbf7659c700 1 -- 172.21.15.88:0/3622896977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf78103540 msgr2=0x7fbf781a01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:32:10.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.305+0000 7fbf7659c700 1 --2- 172.21.15.88:0/3622896977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf78103540 0x7fbf781a01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:10.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.305+0000 7fbf7659c700 1 -- 172.21.15.88:0/3622896977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf78102ba0 msgr2=0x7fbf78199160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:32:10.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.305+0000 7fbf7659c700 1 --2- 172.21.15.88:0/3622896977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf78102ba0 0x7fbf78199160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:10.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.305+0000 7fbf7659c700 1 -- 172.21.15.88:0/3622896977 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf781a0870 con 0x7fbf781019a0 2024-04-21T08:32:10.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.305+0000 7fbf7659c700 1 --2- 172.21.15.88:0/3622896977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf781019a0 0x7fbf78198c20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbf68007f10 tx=0x7fbf6800d3b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:32:10.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.306+0000 7fbf6f7fe700 1 -- 172.21.15.88:0/3622896977 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf68003c00 con 0x7fbf781019a0 2024-04-21T08:32:10.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.306+0000 7fbf6f7fe700 1 -- 172.21.15.88:0/3622896977 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf68003da0 con 0x7fbf781019a0 2024-04-21T08:32:10.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.306+0000 7fbf7c96f700 1 -- 172.21.15.88:0/3622896977 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf78103e20 con 0x7fbf781019a0 2024-04-21T08:32:10.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.306+0000 7fbf6f7fe700 1 -- 172.21.15.88:0/3622896977 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf68012550 con 0x7fbf781019a0 2024-04-21T08:32:10.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.306+0000 7fbf7c96f700 1 -- 172.21.15.88:0/3622896977 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf78104360 con 0x7fbf781019a0 2024-04-21T08:32:10.308 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.307+0000 7fbf7c96f700 1 -- 172.21.15.88:0/3622896977 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf78069730 con 0x7fbf781019a0 2024-04-21T08:32:10.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.308+0000 7fbf6f7fe700 1 -- 172.21.15.88:0/3622896977 <== mon.2 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbf680040b0 con 0x7fbf781019a0 2024-04-21T08:32:10.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.308+0000 7fbf6f7fe700 1 --2- 172.21.15.88:0/3622896977 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fbf640a3c40 0x7fbf640a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:32:10.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.308+0000 7fbf6f7fe700 1 -- 172.21.15.88:0/3622896977 <== mon.2 v2:172.21.15.88:3300/0 5 ==== osd_map(26..26 src has 1..26) ==== 3698+0+0 (secure 0 0 0) 0x7fbf680d3170 con 0x7fbf781019a0 2024-04-21T08:32:10.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.309+0000 7fbf75d9b700 1 --2- 172.21.15.88:0/3622896977 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fbf640a3c40 0x7fbf640a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:32:10.309 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.309+0000 7fbf75d9b700 1 --2- 172.21.15.88:0/3622896977 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fbf640a3c40 0x7fbf640a6100 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbf60009990 tx=0x7fbf6000b870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:32:10.312 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.312+0000 7fbf6f7fe700 1 -- 172.21.15.88:0/3622896977 <== mon.2 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbf680a0d50 con 0x7fbf781019a0 2024-04-21T08:32:10.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:10.518+0000 7fbf7c96f700 1 -- 172.21.15.88:0/3622896977 --> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7fbf7806b820 con 0x7fbf640a3c40 2024-04-21T08:32:10.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:10 smithi088 ceph-mon[31259]: Detected new or changed devices on smithi086 2024-04-21T08:32:10.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:10 smithi088 ceph-mon[31259]: Adjusting osd_memory_target on smithi086 to 8544M 2024-04-21T08:32:10.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:10 smithi088 ceph-mon[31259]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:32:10.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:10 smithi179 ceph-mon[31829]: Detected new or changed devices on smithi086 2024-04-21T08:32:10.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:10 smithi179 ceph-mon[31829]: Adjusting osd_memory_target on smithi086 to 8544M 2024-04-21T08:32:10.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:10 smithi179 ceph-mon[31829]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:32:10.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:10 smithi086 ceph-mon[27939]: Detected new or changed devices on smithi086 2024-04-21T08:32:10.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:10 smithi086 ceph-mon[27939]: Adjusting osd_memory_target on smithi086 to 8544M 2024-04-21T08:32:10.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:10 smithi086 ceph-mon[27939]: pgmap v98: 1 pgs: 1 active+clean; 577 KiB data, 94 MiB used, 268 GiB / 268 GiB avail 2024-04-21T08:32:11.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:11 smithi088 ceph-mon[31259]: osdmap e27: 4 total, 4 up, 4 in 2024-04-21T08:32:11.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:11 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:32:11.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:11 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:32:11.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:11 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:11.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:11 smithi179 ceph-mon[31829]: osdmap e27: 4 total, 4 up, 4 in 2024-04-21T08:32:11.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:11 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:32:11.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:11 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:32:11.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:11 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:11.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:11 smithi086 ceph-mon[27939]: osdmap e27: 4 total, 4 up, 4 in 2024-04-21T08:32:11.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:11 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:32:11.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:11 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:32:11.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:11 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:12.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:12 smithi088 ceph-mon[31259]: from='client.24197 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:32:12.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:12 smithi088 ceph-mon[31259]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-21T08:32:12.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:12 smithi088 ceph-mon[31259]: osdmap e28: 4 total, 4 up, 4 in 2024-04-21T08:32:12.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:12 smithi179 ceph-mon[31829]: from='client.24197 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:32:12.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:12 smithi179 ceph-mon[31829]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-21T08:32:12.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:12 smithi179 ceph-mon[31829]: osdmap e28: 4 total, 4 up, 4 in 2024-04-21T08:32:12.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:12 smithi086 ceph-mon[27939]: from='client.24197 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:32:12.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:12 smithi086 ceph-mon[27939]: pgmap v101: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-21T08:32:12.768 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:12 smithi086 ceph-mon[27939]: osdmap e28: 4 total, 4 up, 4 in 2024-04-21T08:32:13.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:13 smithi088 ceph-mon[31259]: osdmap e29: 4 total, 4 up, 4 in 2024-04-21T08:32:13.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:13 smithi088 ceph-mon[31259]: from='client.? 172.21.15.88:0/3566662980' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "458ff202-d781-4ec7-ac93-0b49f835b503"}]: dispatch 2024-04-21T08:32:13.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:13 smithi088 ceph-mon[31259]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "458ff202-d781-4ec7-ac93-0b49f835b503"}]: dispatch 2024-04-21T08:32:13.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:13 smithi088 ceph-mon[31259]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "458ff202-d781-4ec7-ac93-0b49f835b503"}]': finished 2024-04-21T08:32:13.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:13 smithi088 ceph-mon[31259]: osdmap e30: 5 total, 4 up, 5 in 2024-04-21T08:32:13.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:13 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:32:13.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:13 smithi179 ceph-mon[31829]: osdmap e29: 4 total, 4 up, 4 in 2024-04-21T08:32:13.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:13 smithi179 ceph-mon[31829]: from='client.? 172.21.15.88:0/3566662980' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "458ff202-d781-4ec7-ac93-0b49f835b503"}]: dispatch 2024-04-21T08:32:13.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:13 smithi179 ceph-mon[31829]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "458ff202-d781-4ec7-ac93-0b49f835b503"}]: dispatch 2024-04-21T08:32:13.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:13 smithi179 ceph-mon[31829]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "458ff202-d781-4ec7-ac93-0b49f835b503"}]': finished 2024-04-21T08:32:13.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:13 smithi179 ceph-mon[31829]: osdmap e30: 5 total, 4 up, 5 in 2024-04-21T08:32:13.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:13 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:32:13.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:13 smithi086 ceph-mon[27939]: osdmap e29: 4 total, 4 up, 4 in 2024-04-21T08:32:13.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:13 smithi086 ceph-mon[27939]: from='client.? 172.21.15.88:0/3566662980' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "458ff202-d781-4ec7-ac93-0b49f835b503"}]: dispatch 2024-04-21T08:32:13.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:13 smithi086 ceph-mon[27939]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "458ff202-d781-4ec7-ac93-0b49f835b503"}]: dispatch 2024-04-21T08:32:13.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:13 smithi086 ceph-mon[27939]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "458ff202-d781-4ec7-ac93-0b49f835b503"}]': finished 2024-04-21T08:32:13.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:13 smithi086 ceph-mon[27939]: osdmap e30: 5 total, 4 up, 5 in 2024-04-21T08:32:13.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:13 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:32:14.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:14 smithi088 ceph-mon[31259]: pgmap v104: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-21T08:32:14.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:14 smithi088 ceph-mon[31259]: from='client.? 172.21.15.88:0/3622175123' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:32:14.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:14 smithi179 ceph-mon[31829]: pgmap v104: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-21T08:32:14.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:14 smithi179 ceph-mon[31829]: from='client.? 172.21.15.88:0/3622175123' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:32:14.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:14 smithi086 ceph-mon[27939]: pgmap v104: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-21T08:32:14.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:14 smithi086 ceph-mon[27939]: from='client.? 172.21.15.88:0/3622175123' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:32:15.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:15 smithi088 ceph-mon[31259]: pgmap v106: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-04-21T08:32:15.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:15 smithi179 ceph-mon[31829]: pgmap v106: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-04-21T08:32:15.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:15 smithi086 ceph-mon[27939]: pgmap v106: 1 pgs: 1 active+undersized+remapped; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 120 KiB/s, 0 objects/s recovering 2024-04-21T08:32:18.417 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:18 smithi088 ceph-mon[31259]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-21T08:32:18.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:18 smithi086 ceph-mon[27939]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-21T08:32:18.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:18 smithi179 ceph-mon[31829]: pgmap v107: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-21T08:32:20.427 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:20 smithi088 ceph-mon[31259]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-21T08:32:20.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:20 smithi086 ceph-mon[27939]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-21T08:32:20.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:20 smithi179 ceph-mon[31829]: pgmap v108: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-21T08:32:21.389 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:21 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-21T08:32:21.389 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:21 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:21.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:21 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-21T08:32:21.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:21 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:21.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:21 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.4"}]: dispatch 2024-04-21T08:32:21.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:21 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:22.536 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:22 smithi088 ceph-mon[31259]: Deploying daemon osd.4 on smithi088 2024-04-21T08:32:22.536 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:22 smithi088 ceph-mon[31259]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-21T08:32:22.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:22 smithi086 ceph-mon[27939]: Deploying daemon osd.4 on smithi088 2024-04-21T08:32:22.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:22 smithi086 ceph-mon[27939]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-21T08:32:22.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:22 smithi179 ceph-mon[31829]: Deploying daemon osd.4 on smithi088 2024-04-21T08:32:22.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:22 smithi179 ceph-mon[31829]: pgmap v109: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-21T08:32:24.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:24 smithi086 ceph-mon[27939]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-21T08:32:24.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:24 smithi088 ceph-mon[31259]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-21T08:32:24.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:24 smithi179 ceph-mon[31829]: pgmap v110: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-21T08:32:26.593 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:26 smithi088 ceph-mon[31259]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-21T08:32:26.594 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:26 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:26.594 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:26 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:26.594 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:26 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:32:26.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:26 smithi086 ceph-mon[27939]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-21T08:32:26.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:26 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:26.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:26 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:26.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:26 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:32:26.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:26 smithi179 ceph-mon[31829]: pgmap v111: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-21T08:32:26.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:26 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:26.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:26 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:26.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:26 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:32:28.463 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:28 smithi088 ceph-mon[31259]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-21T08:32:28.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:28 smithi086 ceph-mon[27939]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-21T08:32:28.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:28 smithi179 ceph-mon[31829]: pgmap v112: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-21T08:32:29.585 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:29.585+0000 7fbf6f7fe700 1 -- 172.21.15.88:0/3622896977 <== mgr.14152 v2:172.21.15.86:6800/1364156965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbf7806b820 con 0x7fbf640a3c40 2024-04-21T08:32:29.586 INFO:teuthology.orchestra.run.smithi088.stdout:Created osd(s) 4 on host 'smithi088' 2024-04-21T08:32:29.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:29.587+0000 7fbf7c96f700 1 -- 172.21.15.88:0/3622896977 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fbf640a3c40 msgr2=0x7fbf640a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:32:29.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:29.587+0000 7fbf7c96f700 1 --2- 172.21.15.88:0/3622896977 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fbf640a3c40 0x7fbf640a6100 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7fbf60009990 tx=0x7fbf6000b870 comp rx=0 tx=0).stop 2024-04-21T08:32:29.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:29.587+0000 7fbf7c96f700 1 -- 172.21.15.88:0/3622896977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf781019a0 msgr2=0x7fbf78198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:32:29.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:29.587+0000 7fbf7c96f700 1 --2- 172.21.15.88:0/3622896977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf781019a0 0x7fbf78198c20 secure :-1 s=READY pgs=18 cs=0 l=1 rev1=1 crypto rx=0x7fbf68007f10 tx=0x7fbf6800d3b0 comp rx=0 tx=0).stop 2024-04-21T08:32:29.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:29.587+0000 7fbf7c96f700 1 -- 172.21.15.88:0/3622896977 shutdown_connections 2024-04-21T08:32:29.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:29.587+0000 7fbf7c96f700 1 --2- 172.21.15.88:0/3622896977 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fbf640a3c40 0x7fbf640a6100 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:29.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:29.587+0000 7fbf7c96f700 1 --2- 172.21.15.88:0/3622896977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf78103540 0x7fbf781a01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:29.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:29.587+0000 7fbf7c96f700 1 --2- 172.21.15.88:0/3622896977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf781019a0 0x7fbf78198c20 unknown :-1 s=CLOSED pgs=18 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:29.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:29.587+0000 7fbf7c96f700 1 --2- 172.21.15.88:0/3622896977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf78102ba0 0x7fbf78199160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:29.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:29.587+0000 7fbf7c96f700 1 -- 172.21.15.88:0/3622896977 >> 172.21.15.88:0/3622896977 conn(0x7fbf780fcf60 msgr2=0x7fbf7806b110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:32:29.587 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:29.587+0000 7fbf7c96f700 1 -- 172.21.15.88:0/3622896977 shutdown_connections 2024-04-21T08:32:29.588 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:32:29.587+0000 7fbf7c96f700 1 -- 172.21.15.88:0/3622896977 wait complete. 2024-04-21T08:32:29.828 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:29 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:29.828 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:29 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:29.828 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:29 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:29.828 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:29 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:32:29.828 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:29 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:29.828 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:29 smithi088 ceph-mon[31259]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-21T08:32:29.828 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:29 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:29.829 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:29 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:29.829 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:29 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:32:29.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:29 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:29.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:29 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:29.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:29 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:29.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:29 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:32:29.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:29 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:29.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:29 smithi179 ceph-mon[31829]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-21T08:32:29.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:29 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:29.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:29 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:29.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:29 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:32:30.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:29 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:30.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:29 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:30.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:29 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:30.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:29 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:32:30.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:29 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:30.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:29 smithi086 ceph-mon[27939]: pgmap v113: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-21T08:32:30.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:29 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:30.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:29 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:30.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:29 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:32:30.237 DEBUG:teuthology.orchestra.run.smithi088:osd.4> sudo journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@osd.4.service 2024-04-21T08:32:30.240 INFO:tasks.cephadm:Deploying osd.5 on smithi179 with /dev/vg_nvme/lv_3... 2024-04-21T08:32:30.240 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_3 2024-04-21T08:32:30.485 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:32:30.876 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:30 smithi179 ceph-mon[31829]: from='osd.4 [v2:172.21.15.88:6808/1818714217,v1:172.21.15.88:6809/1818714217]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-21T08:32:30.876 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:30 smithi179 ceph-mon[31829]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-21T08:32:30.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:30 smithi088 ceph-mon[31259]: from='osd.4 [v2:172.21.15.88:6808/1818714217,v1:172.21.15.88:6809/1818714217]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-21T08:32:30.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:30 smithi088 ceph-mon[31259]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-21T08:32:31.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:30 smithi086 ceph-mon[27939]: from='osd.4 [v2:172.21.15.88:6808/1818714217,v1:172.21.15.88:6809/1818714217]' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-21T08:32:31.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:30 smithi086 ceph-mon[27939]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]: dispatch 2024-04-21T08:32:31.823 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:31 smithi179 ceph-mon[31829]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-21T08:32:31.823 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:31 smithi179 ceph-mon[31829]: osdmap e31: 5 total, 4 up, 5 in 2024-04-21T08:32:31.824 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:31 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:32:31.824 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:31 smithi179 ceph-mon[31829]: from='osd.4 [v2:172.21.15.88:6808/1818714217,v1:172.21.15.88:6809/1818714217]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-21T08:32:31.824 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:31 smithi179 ceph-mon[31829]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-21T08:32:31.824 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:31 smithi179 ceph-mon[31829]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-21T08:32:31.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:31 smithi088 ceph-mon[31259]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-21T08:32:31.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:31 smithi088 ceph-mon[31259]: osdmap e31: 5 total, 4 up, 5 in 2024-04-21T08:32:31.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:31 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:32:31.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:31 smithi088 ceph-mon[31259]: from='osd.4 [v2:172.21.15.88:6808/1818714217,v1:172.21.15.88:6809/1818714217]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-21T08:32:31.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:31 smithi088 ceph-mon[31259]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-21T08:32:31.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:31 smithi088 ceph-mon[31259]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-21T08:32:32.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:31 smithi086 ceph-mon[27939]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["4"]}]': finished 2024-04-21T08:32:32.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:31 smithi086 ceph-mon[27939]: osdmap e31: 5 total, 4 up, 5 in 2024-04-21T08:32:32.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:31 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:32:32.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:31 smithi086 ceph-mon[27939]: from='osd.4 [v2:172.21.15.88:6808/1818714217,v1:172.21.15.88:6809/1818714217]' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-21T08:32:32.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:31 smithi086 ceph-mon[27939]: from='osd.4 ' entity='osd.4' cmd=[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-21T08:32:32.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:31 smithi086 ceph-mon[27939]: pgmap v115: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-21T08:32:32.383 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping: /dev/vg_nvme/lv_3 2024-04-21T08:32:32.383 INFO:teuthology.orchestra.run.smithi179.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_3 bs=1M count=10 conv=fsync 2024-04-21T08:32:32.383 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10+0 records in 2024-04-21T08:32:32.383 INFO:teuthology.orchestra.run.smithi179.stderr:10+0 records out 2024-04-21T08:32:32.383 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0113777 s, 922 MB/s 2024-04-21T08:32:32.383 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping successful for: 2024-04-21T08:32:32.730 INFO:journalctl@ceph.osd.4.smithi088.stdout:Apr 21 08:32:32 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-4[42528]: 2024-04-21T08:32:32.464+0000 7f608d223700 -1 osd.4 0 waiting for initial osdmap 2024-04-21T08:32:32.730 INFO:journalctl@ceph.osd.4.smithi088.stdout:Apr 21 08:32:32 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-4[42528]: 2024-04-21T08:32:32.471+0000 7f608500e700 -1 osd.4 32 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T08:32:32.987 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:32 smithi179 ceph-mon[31829]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]': finished 2024-04-21T08:32:32.987 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:32 smithi179 ceph-mon[31829]: osdmap e32: 5 total, 4 up, 5 in 2024-04-21T08:32:32.987 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:32 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:32:32.987 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:32 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:32:32.987 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:32 smithi179 ceph-mon[31829]: from='osd.4 ' entity='osd.4' 2024-04-21T08:32:33.015 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi179:vg_nvme/lv_3 2024-04-21T08:32:33.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:32 smithi086 ceph-mon[27939]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]': finished 2024-04-21T08:32:33.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:32 smithi086 ceph-mon[27939]: osdmap e32: 5 total, 4 up, 5 in 2024-04-21T08:32:33.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:32 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:32:33.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:32 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:32:33.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:32 smithi086 ceph-mon[27939]: from='osd.4 ' entity='osd.4' 2024-04-21T08:32:33.147 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:32 smithi088 ceph-mon[31259]: from='osd.4 ' entity='osd.4' cmd='[{"prefix": "osd crush create-or-move", "id": 4, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]': finished 2024-04-21T08:32:33.147 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:32 smithi088 ceph-mon[31259]: osdmap e32: 5 total, 4 up, 5 in 2024-04-21T08:32:33.147 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:32 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:32:33.147 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:32 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:32:33.147 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:32 smithi088 ceph-mon[31259]: from='osd.4 ' entity='osd.4' 2024-04-21T08:32:33.255 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:32:33.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:33 smithi179 ceph-mon[31829]: purged_snaps scrub starts 2024-04-21T08:32:33.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:33 smithi179 ceph-mon[31829]: purged_snaps scrub ok 2024-04-21T08:32:33.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:33 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:32:33.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:33 smithi179 ceph-mon[31829]: Detected new or changed devices on smithi088 2024-04-21T08:32:33.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:33 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:33.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:33 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:33.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:33 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:32:33.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:33 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:32:33.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:33 smithi179 ceph-mon[31829]: Adjusting osd_memory_target on smithi088 to 8544M 2024-04-21T08:32:33.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:33 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:33.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:33 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:33.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:33 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:32:33.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:33 smithi179 ceph-mon[31829]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-21T08:32:33.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:33 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:33.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:33 smithi179 ceph-mon[31829]: osd.4 [v2:172.21.15.88:6808/1818714217,v1:172.21.15.88:6809/1818714217] boot 2024-04-21T08:32:33.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:33 smithi179 ceph-mon[31829]: osdmap e33: 5 total, 5 up, 5 in 2024-04-21T08:32:33.998 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:33 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:32:34.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:33 smithi086 ceph-mon[27939]: purged_snaps scrub starts 2024-04-21T08:32:34.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:33 smithi086 ceph-mon[27939]: purged_snaps scrub ok 2024-04-21T08:32:34.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:33 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:32:34.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:33 smithi086 ceph-mon[27939]: Detected new or changed devices on smithi088 2024-04-21T08:32:34.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:33 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:34.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:33 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:34.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:33 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:32:34.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:33 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:32:34.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:33 smithi086 ceph-mon[27939]: Adjusting osd_memory_target on smithi088 to 8544M 2024-04-21T08:32:34.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:33 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:34.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:33 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:34.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:33 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:32:34.118 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:33 smithi086 ceph-mon[27939]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-21T08:32:34.118 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:33 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:34.118 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:33 smithi086 ceph-mon[27939]: osd.4 [v2:172.21.15.88:6808/1818714217,v1:172.21.15.88:6809/1818714217] boot 2024-04-21T08:32:34.118 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:33 smithi086 ceph-mon[27939]: osdmap e33: 5 total, 5 up, 5 in 2024-04-21T08:32:34.118 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:33 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:32:34.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:33 smithi088 ceph-mon[31259]: purged_snaps scrub starts 2024-04-21T08:32:34.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:33 smithi088 ceph-mon[31259]: purged_snaps scrub ok 2024-04-21T08:32:34.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:33 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:32:34.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:33 smithi088 ceph-mon[31259]: Detected new or changed devices on smithi088 2024-04-21T08:32:34.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:33 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:34.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:33 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:34.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:33 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:32:34.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:33 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:32:34.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:33 smithi088 ceph-mon[31259]: Adjusting osd_memory_target on smithi088 to 8544M 2024-04-21T08:32:34.215 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:33 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:34.215 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:33 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:34.215 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:33 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:32:34.215 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:33 smithi088 ceph-mon[31259]: pgmap v117: 1 pgs: 1 active+clean; 577 KiB data, 120 MiB used, 357 GiB / 358 GiB avail 2024-04-21T08:32:34.215 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:33 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:34.215 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:33 smithi088 ceph-mon[31259]: osd.4 [v2:172.21.15.88:6808/1818714217,v1:172.21.15.88:6809/1818714217] boot 2024-04-21T08:32:34.215 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:33 smithi088 ceph-mon[31259]: osdmap e33: 5 total, 5 up, 5 in 2024-04-21T08:32:34.215 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:33 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:32:34.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.891+0000 7f8546683700 1 -- 172.21.15.179:0/2706706005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8540102ba0 msgr2=0x7f8540103000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:32:34.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.891+0000 7f8546683700 1 --2- 172.21.15.179:0/2706706005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8540102ba0 0x7f8540103000 secure :-1 s=READY pgs=20 cs=0 l=1 rev1=1 crypto rx=0x7f8534009990 tx=0x7f8534009ca0 comp rx=0 tx=0).stop 2024-04-21T08:32:34.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.891+0000 7f8546683700 1 -- 172.21.15.179:0/2706706005 shutdown_connections 2024-04-21T08:32:34.891 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.891+0000 7f8546683700 1 --2- 172.21.15.179:0/2706706005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8540103540 0x7f8540109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:34.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.891+0000 7f8546683700 1 --2- 172.21.15.179:0/2706706005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85401019a0 0x7f8540101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:34.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.891+0000 7f8546683700 1 --2- 172.21.15.179:0/2706706005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8540102ba0 0x7f8540103000 unknown :-1 s=CLOSED pgs=20 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:34.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.891+0000 7f8546683700 1 -- 172.21.15.179:0/2706706005 >> 172.21.15.179:0/2706706005 conn(0x7f85400fcf40 msgr2=0x7f85400ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:32:34.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.891+0000 7f8546683700 1 -- 172.21.15.179:0/2706706005 shutdown_connections 2024-04-21T08:32:34.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.891+0000 7f8546683700 1 -- 172.21.15.179:0/2706706005 wait complete. 2024-04-21T08:32:34.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.892+0000 7f8546683700 1 Processor -- start 2024-04-21T08:32:34.892 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.892+0000 7f8546683700 1 -- start start 2024-04-21T08:32:34.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.892+0000 7f8546683700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85401019a0 0x7f8540198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:32:34.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.892+0000 7f8546683700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8540102ba0 0x7f85401991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:32:34.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.892+0000 7f8546683700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8540103540 0x7f85401a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:32:34.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.892+0000 7f8546683700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f854010d410 con 0x7f85401019a0 2024-04-21T08:32:34.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.892+0000 7f8546683700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f854010d290 con 0x7f8540102ba0 2024-04-21T08:32:34.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.892+0000 7f8546683700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f854010d590 con 0x7f8540103540 2024-04-21T08:32:34.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.892+0000 7f8544c20700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8540103540 0x7f85401a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:32:34.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.892+0000 7f8544c20700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8540103540 0x7f85401a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:41040/0 (socket says 172.21.15.179:41040) 2024-04-21T08:32:34.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.892+0000 7f8544c20700 1 -- 172.21.15.179:0/3567349127 learned_addr learned my addr 172.21.15.179:0/3567349127 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-21T08:32:34.894 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.892+0000 7f853f7fe700 1 --2- 172.21.15.179:0/3567349127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8540102ba0 0x7f85401991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:32:34.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.893+0000 7f853ffff700 1 --2- 172.21.15.179:0/3567349127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85401019a0 0x7f8540198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:32:34.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.893+0000 7f8544c20700 1 -- 172.21.15.179:0/3567349127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8540102ba0 msgr2=0x7f85401991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:32:34.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.893+0000 7f8544c20700 1 --2- 172.21.15.179:0/3567349127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8540102ba0 0x7f85401991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:34.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.893+0000 7f8544c20700 1 -- 172.21.15.179:0/3567349127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85401019a0 msgr2=0x7f8540198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:32:34.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.893+0000 7f8544c20700 1 --2- 172.21.15.179:0/3567349127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85401019a0 0x7f8540198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:34.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.893+0000 7f8544c20700 1 -- 172.21.15.179:0/3567349127 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85401a08c0 con 0x7f8540103540 2024-04-21T08:32:34.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.893+0000 7f8544c20700 1 --2- 172.21.15.179:0/3567349127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8540103540 0x7f85401a01f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8538007ee0 tx=0x7f853800d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:32:34.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.893+0000 7f853d7fa700 1 -- 172.21.15.179:0/3567349127 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f853800ddd0 con 0x7f8540103540 2024-04-21T08:32:34.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.893+0000 7f8546683700 1 -- 172.21.15.179:0/3567349127 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85401a0bb0 con 0x7f8540103540 2024-04-21T08:32:34.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.893+0000 7f853d7fa700 1 -- 172.21.15.179:0/3567349127 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8538004020 con 0x7f8540103540 2024-04-21T08:32:34.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.893+0000 7f853d7fa700 1 -- 172.21.15.179:0/3567349127 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8538015660 con 0x7f8540103540 2024-04-21T08:32:34.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.893+0000 7f8546683700 1 -- 172.21.15.179:0/3567349127 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85401a10f0 con 0x7f8540103540 2024-04-21T08:32:34.895 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.894+0000 7f8546683700 1 -- 172.21.15.179:0/3567349127 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8540069730 con 0x7f8540103540 2024-04-21T08:32:34.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.895+0000 7f853d7fa700 1 -- 172.21.15.179:0/3567349127 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f85380041d0 con 0x7f8540103540 2024-04-21T08:32:34.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.896+0000 7f853d7fa700 1 --2- 172.21.15.179:0/3567349127 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f85280a3c40 0x7f85280a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:32:34.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.896+0000 7f853d7fa700 1 -- 172.21.15.179:0/3567349127 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(34..34 src has 1..34) ==== 4101+0+0 (secure 0 0 0) 0x7f85380ced10 con 0x7f8540103540 2024-04-21T08:32:34.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.896+0000 7f853ffff700 1 --2- 172.21.15.179:0/3567349127 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f85280a3c40 0x7f85280a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:32:34.896 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.897+0000 7f853ffff700 1 --2- 172.21.15.179:0/3567349127 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f85280a3c40 0x7f85280a6100 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f854019a150 tx=0x7f853000b5d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:32:34.900 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:34.901+0000 7f853d7fa700 1 -- 172.21.15.179:0/3567349127 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8538010070 con 0x7f8540103540 2024-04-21T08:32:35.110 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:35.110+0000 7f8546683700 1 -- 172.21.15.179:0/3567349127 --> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_3", "target": ["mon-mgr", ""]}) -- 0x7f85400ff180 con 0x7f85280a3c40 2024-04-21T08:32:35.983 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:35 smithi179 ceph-mon[31829]: osdmap e34: 5 total, 5 up, 5 in 2024-04-21T08:32:35.983 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:35 smithi179 ceph-mon[31829]: from='client.24217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:32:35.983 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:35 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:32:35.983 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:35 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:32:35.983 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:35 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:35.983 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:35 smithi179 ceph-mon[31829]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:36.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:35 smithi086 ceph-mon[27939]: osdmap e34: 5 total, 5 up, 5 in 2024-04-21T08:32:36.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:35 smithi086 ceph-mon[27939]: from='client.24217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:32:36.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:35 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:32:36.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:35 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:32:36.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:35 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:36.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:35 smithi086 ceph-mon[27939]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:36.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:35 smithi088 ceph-mon[31259]: osdmap e34: 5 total, 5 up, 5 in 2024-04-21T08:32:36.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:35 smithi088 ceph-mon[31259]: from='client.24217 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_3", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:32:36.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:35 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:32:36.215 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:35 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:32:36.215 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:35 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:36.215 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:35 smithi088 ceph-mon[31259]: pgmap v120: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:38.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:38 smithi179 ceph-mon[31829]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:38.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:38 smithi179 ceph-mon[31829]: from='client.? 172.21.15.179:0/1601826773' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8123cc90-bd8b-4204-bf31-46b34e9f589d"}]: dispatch 2024-04-21T08:32:38.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:38 smithi179 ceph-mon[31829]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8123cc90-bd8b-4204-bf31-46b34e9f589d"}]: dispatch 2024-04-21T08:32:38.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:38 smithi179 ceph-mon[31829]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8123cc90-bd8b-4204-bf31-46b34e9f589d"}]': finished 2024-04-21T08:32:38.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:38 smithi179 ceph-mon[31829]: osdmap e35: 6 total, 5 up, 6 in 2024-04-21T08:32:38.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:38 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:32:38.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:38 smithi086 ceph-mon[27939]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:38.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:38 smithi086 ceph-mon[27939]: from='client.? 172.21.15.179:0/1601826773' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8123cc90-bd8b-4204-bf31-46b34e9f589d"}]: dispatch 2024-04-21T08:32:38.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:38 smithi086 ceph-mon[27939]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8123cc90-bd8b-4204-bf31-46b34e9f589d"}]: dispatch 2024-04-21T08:32:38.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:38 smithi086 ceph-mon[27939]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8123cc90-bd8b-4204-bf31-46b34e9f589d"}]': finished 2024-04-21T08:32:38.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:38 smithi086 ceph-mon[27939]: osdmap e35: 6 total, 5 up, 6 in 2024-04-21T08:32:38.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:38 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:32:38.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:38 smithi088 ceph-mon[31259]: pgmap v121: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:38.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:38 smithi088 ceph-mon[31259]: from='client.? 172.21.15.179:0/1601826773' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8123cc90-bd8b-4204-bf31-46b34e9f589d"}]: dispatch 2024-04-21T08:32:38.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:38 smithi088 ceph-mon[31259]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "8123cc90-bd8b-4204-bf31-46b34e9f589d"}]: dispatch 2024-04-21T08:32:38.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:38 smithi088 ceph-mon[31259]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "8123cc90-bd8b-4204-bf31-46b34e9f589d"}]': finished 2024-04-21T08:32:38.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:38 smithi088 ceph-mon[31259]: osdmap e35: 6 total, 5 up, 6 in 2024-04-21T08:32:38.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:38 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:32:39.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:39 smithi086 ceph-mon[27939]: from='client.? 172.21.15.179:0/283696638' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:32:39.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:39 smithi088 ceph-mon[31259]: from='client.? 172.21.15.179:0/283696638' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:32:39.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:39 smithi179 ceph-mon[31829]: from='client.? 172.21.15.179:0/283696638' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:32:40.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:40 smithi086 ceph-mon[27939]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:40.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:40 smithi088 ceph-mon[31259]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:40.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:40 smithi179 ceph-mon[31829]: pgmap v123: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:42.597 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:42 smithi179 ceph-mon[31829]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:42.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:42 smithi086 ceph-mon[27939]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:42.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:42 smithi088 ceph-mon[31259]: pgmap v124: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:44.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:44 smithi179 ceph-mon[31829]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:44.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:44 smithi086 ceph-mon[27939]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:44.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:44 smithi088 ceph-mon[31259]: pgmap v125: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:45.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:45 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-21T08:32:45.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:45 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:45.640 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:45 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-21T08:32:45.640 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:45 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:45.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:45 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.5"}]: dispatch 2024-04-21T08:32:45.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:45 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:46.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:46 smithi179 ceph-mon[31829]: Deploying daemon osd.5 on smithi179 2024-04-21T08:32:46.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:46 smithi179 ceph-mon[31829]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:46.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:46 smithi086 ceph-mon[27939]: Deploying daemon osd.5 on smithi179 2024-04-21T08:32:46.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:46 smithi086 ceph-mon[27939]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:46.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:46 smithi088 ceph-mon[31259]: Deploying daemon osd.5 on smithi179 2024-04-21T08:32:46.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:46 smithi088 ceph-mon[31259]: pgmap v126: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:48.562 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:48 smithi179 ceph-mon[31829]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:48.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:48 smithi086 ceph-mon[27939]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:48.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:48 smithi088 ceph-mon[31259]: pgmap v127: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:50.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:50 smithi086 ceph-mon[27939]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:50.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:50 smithi088 ceph-mon[31259]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:50.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:50 smithi179 ceph-mon[31829]: pgmap v128: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:51.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:51 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:51.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:51 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:51.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:51 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:32:51.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:51 smithi179 ceph-mon[31829]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:51.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:51 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:51.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:51 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:51.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:51 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:32:51.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:51 smithi086 ceph-mon[27939]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:51.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:51 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:51.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:51 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:51.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:51 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:32:51.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:51 smithi088 ceph-mon[31259]: pgmap v129: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:54.224 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:54.224+0000 7f853d7fa700 1 -- 172.21.15.179:0/3567349127 <== mgr.14152 v2:172.21.15.86:6800/1364156965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f85400ff180 con 0x7f85280a3c40 2024-04-21T08:32:54.224 INFO:teuthology.orchestra.run.smithi179.stdout:Created osd(s) 5 on host 'smithi179' 2024-04-21T08:32:54.226 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:54.227+0000 7f8546683700 1 -- 172.21.15.179:0/3567349127 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f85280a3c40 msgr2=0x7f85280a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:32:54.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:54.227+0000 7f8546683700 1 --2- 172.21.15.179:0/3567349127 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f85280a3c40 0x7f85280a6100 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7f854019a150 tx=0x7f853000b5d0 comp rx=0 tx=0).stop 2024-04-21T08:32:54.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:54.227+0000 7f8546683700 1 -- 172.21.15.179:0/3567349127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8540103540 msgr2=0x7f85401a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:32:54.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:54.227+0000 7f8546683700 1 --2- 172.21.15.179:0/3567349127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8540103540 0x7f85401a01f0 secure :-1 s=READY pgs=21 cs=0 l=1 rev1=1 crypto rx=0x7f8538007ee0 tx=0x7f853800d3b0 comp rx=0 tx=0).stop 2024-04-21T08:32:54.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:54.227+0000 7f8546683700 1 -- 172.21.15.179:0/3567349127 shutdown_connections 2024-04-21T08:32:54.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:54.227+0000 7f8546683700 1 --2- 172.21.15.179:0/3567349127 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f85280a3c40 0x7f85280a6100 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:54.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:54.227+0000 7f8546683700 1 --2- 172.21.15.179:0/3567349127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8540103540 0x7f85401a01f0 unknown :-1 s=CLOSED pgs=21 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:54.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:54.227+0000 7f8546683700 1 --2- 172.21.15.179:0/3567349127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85401019a0 0x7f8540198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:54.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:54.227+0000 7f8546683700 1 --2- 172.21.15.179:0/3567349127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8540102ba0 0x7f85401991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:54.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:54.227+0000 7f8546683700 1 -- 172.21.15.179:0/3567349127 >> 172.21.15.179:0/3567349127 conn(0x7f85400fcf40 msgr2=0x7f85400fea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:32:54.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:54.227+0000 7f8546683700 1 -- 172.21.15.179:0/3567349127 shutdown_connections 2024-04-21T08:32:54.227 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:32:54.227+0000 7f8546683700 1 -- 172.21.15.179:0/3567349127 wait complete. 2024-04-21T08:32:54.421 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:54 smithi179 ceph-mon[31829]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:54.422 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:54 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:54.422 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:54 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:54.422 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:54 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:54.422 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:54 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:32:54.422 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:54 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:54.422 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:54 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:54.422 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:54 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:54.422 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:54 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:32:54.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:54 smithi086 ceph-mon[27939]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:54.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:54 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:54.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:54 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:54.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:54 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:54.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:54 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:32:54.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:54 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:54.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:54 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:54.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:54 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:54.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:54 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:32:54.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:54 smithi088 ceph-mon[31259]: pgmap v130: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:54.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:54 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:54.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:54 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:54.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:54 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:54.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:54 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:32:54.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:54 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:54.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:54 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:54.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:54 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:54.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:54 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:32:54.932 DEBUG:teuthology.orchestra.run.smithi179:osd.5> sudo journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@osd.5.service 2024-04-21T08:32:54.934 INFO:tasks.cephadm:Deploying osd.6 on smithi086 with /dev/vg_nvme/lv_2... 2024-04-21T08:32:54.935 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-21T08:32:55.178 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:32:55.441 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:55 smithi179 ceph-mon[31829]: from='osd.5 [v2:172.21.15.179:6808/863840990,v1:172.21.15.179:6809/863840990]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-21T08:32:55.441 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:55 smithi179 ceph-mon[31829]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-21T08:32:55.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:55 smithi086 ceph-mon[27939]: from='osd.5 [v2:172.21.15.179:6808/863840990,v1:172.21.15.179:6809/863840990]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-21T08:32:55.570 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:55 smithi086 ceph-mon[27939]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-21T08:32:55.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:55 smithi088 ceph-mon[31259]: from='osd.5 [v2:172.21.15.179:6808/863840990,v1:172.21.15.179:6809/863840990]' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-21T08:32:55.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:55 smithi088 ceph-mon[31259]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]: dispatch 2024-04-21T08:32:56.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:56 smithi086 ceph-mon[27939]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:56.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:56 smithi086 ceph-mon[27939]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-21T08:32:56.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:56 smithi086 ceph-mon[27939]: from='osd.5 [v2:172.21.15.179:6808/863840990,v1:172.21.15.179:6809/863840990]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-21T08:32:56.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:56 smithi086 ceph-mon[27939]: osdmap e36: 6 total, 5 up, 6 in 2024-04-21T08:32:56.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:56 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:32:56.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:56 smithi086 ceph-mon[27939]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-21T08:32:56.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:56 smithi088 ceph-mon[31259]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:56.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:56 smithi088 ceph-mon[31259]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-21T08:32:56.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:56 smithi088 ceph-mon[31259]: from='osd.5 [v2:172.21.15.179:6808/863840990,v1:172.21.15.179:6809/863840990]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-21T08:32:56.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:56 smithi088 ceph-mon[31259]: osdmap e36: 6 total, 5 up, 6 in 2024-04-21T08:32:56.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:56 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:32:56.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:56 smithi088 ceph-mon[31259]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-21T08:32:56.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:56 smithi179 ceph-mon[31829]: pgmap v131: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:56.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:56 smithi179 ceph-mon[31829]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["5"]}]': finished 2024-04-21T08:32:56.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:56 smithi179 ceph-mon[31829]: from='osd.5 [v2:172.21.15.179:6808/863840990,v1:172.21.15.179:6809/863840990]' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-21T08:32:56.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:56 smithi179 ceph-mon[31829]: osdmap e36: 6 total, 5 up, 6 in 2024-04-21T08:32:56.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:56 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:32:56.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:56 smithi179 ceph-mon[31829]: from='osd.5 ' entity='osd.5' cmd=[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-21T08:32:57.206 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-21T08:32:57.206 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-21T08:32:57.206 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-04-21T08:32:57.206 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-04-21T08:32:57.206 INFO:teuthology.orchestra.run.smithi086.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0112869 s, 929 MB/s 2024-04-21T08:32:57.206 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-04-21T08:32:57.318 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:32:57 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-5[43475]: 2024-04-21T08:32:57.046+0000 7f8dbf46d700 -1 osd.5 0 waiting for initial osdmap 2024-04-21T08:32:57.318 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:32:57 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-5[43475]: 2024-04-21T08:32:57.054+0000 7f8db6a57700 -1 osd.5 37 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T08:32:57.644 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:57 smithi179 ceph-mon[31829]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-04-21T08:32:57.644 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:57 smithi179 ceph-mon[31829]: osdmap e37: 6 total, 5 up, 6 in 2024-04-21T08:32:57.644 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:57 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:32:57.645 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:57 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:32:57.645 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:57 smithi179 ceph-mon[31829]: from='osd.5 ' entity='osd.5' 2024-04-21T08:32:57.645 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:57 smithi179 ceph-mon[31829]: osd.5 [v2:172.21.15.179:6808/863840990,v1:172.21.15.179:6809/863840990] boot 2024-04-21T08:32:57.645 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:57 smithi179 ceph-mon[31829]: osdmap e38: 6 total, 6 up, 6 in 2024-04-21T08:32:57.645 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:57 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:32:57.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:57 smithi088 ceph-mon[31259]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-04-21T08:32:57.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:57 smithi088 ceph-mon[31259]: osdmap e37: 6 total, 5 up, 6 in 2024-04-21T08:32:57.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:57 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:32:57.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:57 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:32:57.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:57 smithi088 ceph-mon[31259]: from='osd.5 ' entity='osd.5' 2024-04-21T08:32:57.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:57 smithi088 ceph-mon[31259]: osd.5 [v2:172.21.15.179:6808/863840990,v1:172.21.15.179:6809/863840990] boot 2024-04-21T08:32:57.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:57 smithi088 ceph-mon[31259]: osdmap e38: 6 total, 6 up, 6 in 2024-04-21T08:32:57.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:57 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:32:57.822 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:57 smithi086 ceph-mon[27939]: from='osd.5 ' entity='osd.5' cmd='[{"prefix": "osd crush create-or-move", "id": 5, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-04-21T08:32:57.823 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:57 smithi086 ceph-mon[27939]: osdmap e37: 6 total, 5 up, 6 in 2024-04-21T08:32:57.823 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:57 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:32:57.823 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:57 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:32:57.823 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:57 smithi086 ceph-mon[27939]: from='osd.5 ' entity='osd.5' 2024-04-21T08:32:57.823 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:57 smithi086 ceph-mon[27939]: osd.5 [v2:172.21.15.179:6808/863840990,v1:172.21.15.179:6809/863840990] boot 2024-04-21T08:32:57.823 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:57 smithi086 ceph-mon[27939]: osdmap e38: 6 total, 6 up, 6 in 2024-04-21T08:32:57.823 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:57 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:32:57.850 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi086:vg_nvme/lv_2 2024-04-21T08:32:58.088 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:32:58.526 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:58 smithi086 ceph-mon[27939]: purged_snaps scrub starts 2024-04-21T08:32:58.526 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:58 smithi086 ceph-mon[27939]: purged_snaps scrub ok 2024-04-21T08:32:58.526 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:58 smithi086 ceph-mon[27939]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:58.526 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:58 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:58.527 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:58 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:58.527 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:58 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:32:58.527 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:58 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:32:58.527 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:58 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:58.527 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:58 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:58.527 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:58 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:32:58.527 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:58 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:58.527 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:58 smithi086 ceph-mon[27939]: osdmap e39: 6 total, 6 up, 6 in 2024-04-21T08:32:58.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:58 smithi088 ceph-mon[31259]: purged_snaps scrub starts 2024-04-21T08:32:58.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:58 smithi088 ceph-mon[31259]: purged_snaps scrub ok 2024-04-21T08:32:58.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:58 smithi088 ceph-mon[31259]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:58.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:58 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:58.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:58 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:58.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:58 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:32:58.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:58 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:32:58.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:58 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:58.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:58 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:58.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:58 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:32:58.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:58 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:58.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:58 smithi088 ceph-mon[31259]: osdmap e39: 6 total, 6 up, 6 in 2024-04-21T08:32:58.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:58 smithi179 ceph-mon[31829]: purged_snaps scrub starts 2024-04-21T08:32:58.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:58 smithi179 ceph-mon[31829]: purged_snaps scrub ok 2024-04-21T08:32:58.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:58 smithi179 ceph-mon[31829]: pgmap v134: 1 pgs: 1 active+clean; 577 KiB data, 151 MiB used, 447 GiB / 447 GiB avail 2024-04-21T08:32:58.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:58 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:58.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:58 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:58.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:58 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:32:58.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:58 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:32:58.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:58 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:58.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:58 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:32:58.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:58 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:32:58.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:58 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:32:58.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:58 smithi179 ceph-mon[31829]: osdmap e39: 6 total, 6 up, 6 in 2024-04-21T08:32:59.459 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:59 smithi086 ceph-mon[27939]: Detected new or changed devices on smithi179 2024-04-21T08:32:59.459 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:59 smithi086 ceph-mon[27939]: Adjusting osd_memory_target on smithi179 to 8544M 2024-04-21T08:32:59.459 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:32:59 smithi086 ceph-mon[27939]: osdmap e40: 6 total, 6 up, 6 in 2024-04-21T08:32:59.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.679+0000 7f6ff5bdd700 1 -- 172.21.15.86:0/2678302745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ff0103530 msgr2=0x7f6ff0105920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:32:59.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.679+0000 7f6ff5bdd700 1 --2- 172.21.15.86:0/2678302745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ff0103530 0x7f6ff0105920 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f6fe0009990 tx=0x7f6fe0009ca0 comp rx=0 tx=0).stop 2024-04-21T08:32:59.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.679+0000 7f6ff5bdd700 1 -- 172.21.15.86:0/2678302745 shutdown_connections 2024-04-21T08:32:59.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.679+0000 7f6ff5bdd700 1 --2- 172.21.15.86:0/2678302745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ff0108790 0x7f6ff010ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:59.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.679+0000 7f6ff5bdd700 1 --2- 172.21.15.86:0/2678302745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ff0103530 0x7f6ff0105920 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:59.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.679+0000 7f6ff5bdd700 1 --2- 172.21.15.86:0/2678302745 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ff0105e60 0x7f6ff0108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:59.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.679+0000 7f6ff5bdd700 1 -- 172.21.15.86:0/2678302745 >> 172.21.15.86:0/2678302745 conn(0x7f6ff00fcf40 msgr2=0x7f6ff00ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:32:59.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.679+0000 7f6ff5bdd700 1 -- 172.21.15.86:0/2678302745 shutdown_connections 2024-04-21T08:32:59.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.679+0000 7f6ff5bdd700 1 -- 172.21.15.86:0/2678302745 wait complete. 2024-04-21T08:32:59.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.680+0000 7f6ff5bdd700 1 Processor -- start 2024-04-21T08:32:59.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.680+0000 7f6ff5bdd700 1 -- start start 2024-04-21T08:32:59.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.680+0000 7f6ff5bdd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ff0103530 0x7f6ff0198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:32:59.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.680+0000 7f6ff5bdd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ff0105e60 0x7f6ff0199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:32:59.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.680+0000 7f6ff5bdd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ff0108790 0x7f6ff01a01d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:32:59.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.680+0000 7f6ff5bdd700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff010e5a0 con 0x7f6ff0108790 2024-04-21T08:32:59.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.680+0000 7f6ff5bdd700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff010e420 con 0x7f6ff0105e60 2024-04-21T08:32:59.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.680+0000 7f6ff5bdd700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6ff010e720 con 0x7f6ff0103530 2024-04-21T08:32:59.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.680+0000 7f6feffff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ff0108790 0x7f6ff01a01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:32:59.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.680+0000 7f6feffff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ff0108790 0x7f6ff01a01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45106/0 (socket says 172.21.15.86:45106) 2024-04-21T08:32:59.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.680+0000 7f6feffff700 1 -- 172.21.15.86:0/170525549 learned_addr learned my addr 172.21.15.86:0/170525549 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:32:59.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.680+0000 7f6fe6ffd700 1 --2- 172.21.15.86:0/170525549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ff0105e60 0x7f6ff0199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:32:59.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.680+0000 7f6fef7fe700 1 --2- 172.21.15.86:0/170525549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ff0103530 0x7f6ff0198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:32:59.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.681+0000 7f6feffff700 1 -- 172.21.15.86:0/170525549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ff0103530 msgr2=0x7f6ff0198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:32:59.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.681+0000 7f6feffff700 1 --2- 172.21.15.86:0/170525549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ff0103530 0x7f6ff0198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:59.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.681+0000 7f6feffff700 1 -- 172.21.15.86:0/170525549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ff0105e60 msgr2=0x7f6ff0199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:32:59.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.681+0000 7f6feffff700 1 --2- 172.21.15.86:0/170525549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ff0105e60 0x7f6ff0199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:32:59.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.681+0000 7f6feffff700 1 -- 172.21.15.86:0/170525549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6ff01a08a0 con 0x7f6ff0108790 2024-04-21T08:32:59.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.681+0000 7f6feffff700 1 --2- 172.21.15.86:0/170525549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ff0108790 0x7f6ff01a01d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8007d40 tx=0x7f6fe800a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:32:59.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.681+0000 7f6fed7fa700 1 -- 172.21.15.86:0/170525549 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe8016020 con 0x7f6ff0108790 2024-04-21T08:32:59.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.681+0000 7f6fed7fa700 1 -- 172.21.15.86:0/170525549 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6fe800ad40 con 0x7f6ff0108790 2024-04-21T08:32:59.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.681+0000 7f6fed7fa700 1 -- 172.21.15.86:0/170525549 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6fe80115e0 con 0x7f6ff0108790 2024-04-21T08:32:59.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.681+0000 7f6ff5bdd700 1 -- 172.21.15.86:0/170525549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6ff01a0b90 con 0x7f6ff0108790 2024-04-21T08:32:59.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.681+0000 7f6ff5bdd700 1 -- 172.21.15.86:0/170525549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6ff01a10d0 con 0x7f6ff0108790 2024-04-21T08:32:59.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.682+0000 7f6ff5bdd700 1 -- 172.21.15.86:0/170525549 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6ff0069730 con 0x7f6ff0108790 2024-04-21T08:32:59.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.683+0000 7f6fed7fa700 1 -- 172.21.15.86:0/170525549 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6fe8019020 con 0x7f6ff0108790 2024-04-21T08:32:59.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.684+0000 7f6fed7fa700 1 --2- 172.21.15.86:0/170525549 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6fdc0a3c40 0x7f6fdc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:32:59.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.684+0000 7f6fef7fe700 1 --2- 172.21.15.86:0/170525549 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6fdc0a3c40 0x7f6fdc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:32:59.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.684+0000 7f6fed7fa700 1 -- 172.21.15.86:0/170525549 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(40..40 src has 1..40) ==== 4562+0+0 (secure 0 0 0) 0x7f6fe80cf870 con 0x7f6ff0108790 2024-04-21T08:32:59.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.684+0000 7f6fef7fe700 1 --2- 172.21.15.86:0/170525549 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6fdc0a3c40 0x7f6fdc0a6100 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6fe000b5c0 tx=0x7f6fe0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:32:59.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.687+0000 7f6fed7fa700 1 -- 172.21.15.86:0/170525549 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6fe8020020 con 0x7f6ff0108790 2024-04-21T08:32:59.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:59 smithi088 ceph-mon[31259]: Detected new or changed devices on smithi179 2024-04-21T08:32:59.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:59 smithi088 ceph-mon[31259]: Adjusting osd_memory_target on smithi179 to 8544M 2024-04-21T08:32:59.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:32:59 smithi088 ceph-mon[31259]: osdmap e40: 6 total, 6 up, 6 in 2024-04-21T08:32:59.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:59 smithi179 ceph-mon[31829]: Detected new or changed devices on smithi179 2024-04-21T08:32:59.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:59 smithi179 ceph-mon[31829]: Adjusting osd_memory_target on smithi179 to 8544M 2024-04-21T08:32:59.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:32:59 smithi179 ceph-mon[31829]: osdmap e40: 6 total, 6 up, 6 in 2024-04-21T08:32:59.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:32:59.897+0000 7f6ff5bdd700 1 -- 172.21.15.86:0/170525549 --> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f6ff0062e80 con 0x7f6fdc0a3c40 2024-04-21T08:33:00.641 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:00 smithi086 ceph-mon[27939]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:00.642 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:00 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:33:00.642 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:00 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:33:00.642 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:00 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:00.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:00 smithi088 ceph-mon[31259]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:00.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:00 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:33:00.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:00 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:33:00.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:00 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:00.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:00 smithi179 ceph-mon[31829]: pgmap v137: 1 pgs: 1 active+clean; 577 KiB data, 294 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:00.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:00 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:33:00.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:00 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:33:00.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:00 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:01.710 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:01 smithi086 ceph-mon[27939]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:33:01.710 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:01 smithi086 ceph-mon[27939]: osdmap e41: 6 total, 6 up, 6 in 2024-04-21T08:33:01.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:01 smithi088 ceph-mon[31259]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:33:01.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:01 smithi088 ceph-mon[31259]: osdmap e41: 6 total, 6 up, 6 in 2024-04-21T08:33:01.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:01 smithi179 ceph-mon[31829]: from='client.14367 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:33:01.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:01 smithi179 ceph-mon[31829]: osdmap e41: 6 total, 6 up, 6 in 2024-04-21T08:33:02.647 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:02 smithi179 ceph-mon[31829]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 294 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-21T08:33:02.648 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:02 smithi179 ceph-mon[31829]: osdmap e42: 6 total, 6 up, 6 in 2024-04-21T08:33:02.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:02 smithi088 ceph-mon[31259]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 294 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-21T08:33:02.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:02 smithi088 ceph-mon[31259]: osdmap e42: 6 total, 6 up, 6 in 2024-04-21T08:33:02.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:02 smithi086 ceph-mon[27939]: pgmap v140: 1 pgs: 1 peering; 577 KiB data, 294 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-21T08:33:02.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:02 smithi086 ceph-mon[27939]: osdmap e42: 6 total, 6 up, 6 in 2024-04-21T08:33:03.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:03 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3002335005' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aac9d7c7-659a-4110-903a-a871c086066f"}]: dispatch 2024-04-21T08:33:03.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:03 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3002335005' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aac9d7c7-659a-4110-903a-a871c086066f"}]': finished 2024-04-21T08:33:03.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:03 smithi088 ceph-mon[31259]: osdmap e43: 7 total, 6 up, 7 in 2024-04-21T08:33:03.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:03 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:33:03.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:03 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3550435522' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:33:03.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:03 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3002335005' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aac9d7c7-659a-4110-903a-a871c086066f"}]: dispatch 2024-04-21T08:33:03.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:03 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3002335005' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aac9d7c7-659a-4110-903a-a871c086066f"}]': finished 2024-04-21T08:33:03.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:03 smithi179 ceph-mon[31829]: osdmap e43: 7 total, 6 up, 7 in 2024-04-21T08:33:03.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:03 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:33:03.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:03 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3550435522' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:33:03.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:03 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3002335005' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "aac9d7c7-659a-4110-903a-a871c086066f"}]: dispatch 2024-04-21T08:33:03.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:03 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3002335005' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "aac9d7c7-659a-4110-903a-a871c086066f"}]': finished 2024-04-21T08:33:03.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:03 smithi086 ceph-mon[27939]: osdmap e43: 7 total, 6 up, 7 in 2024-04-21T08:33:03.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:03 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:33:03.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:03 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3550435522' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:33:04.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:04 smithi088 ceph-mon[31259]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-21T08:33:04.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:04 smithi179 ceph-mon[31829]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-21T08:33:04.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:04 smithi086 ceph-mon[27939]: pgmap v143: 1 pgs: 1 peering; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-21T08:33:06.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:06 smithi086 ceph-mon[27939]: pgmap v144: 1 pgs: 1 peering; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-04-21T08:33:06.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:06 smithi088 ceph-mon[31259]: pgmap v144: 1 pgs: 1 peering; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-04-21T08:33:06.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:06 smithi179 ceph-mon[31829]: pgmap v144: 1 pgs: 1 peering; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 98 KiB/s, 0 objects/s recovering 2024-04-21T08:33:08.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:08 smithi088 ceph-mon[31259]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-21T08:33:08.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:08 smithi179 ceph-mon[31829]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-21T08:33:08.756 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:08 smithi086 ceph-mon[27939]: pgmap v145: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail; 84 KiB/s, 0 objects/s recovering 2024-04-21T08:33:10.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:10 smithi088 ceph-mon[31259]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:10.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:10 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-21T08:33:10.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:10 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:10.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:10 smithi179 ceph-mon[31829]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:10.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:10 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-21T08:33:10.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:10 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:10.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:10 smithi086 ceph-mon[27939]: pgmap v146: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:10.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:10 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.6"}]: dispatch 2024-04-21T08:33:10.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:10 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:11.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:11 smithi088 ceph-mon[31259]: Deploying daemon osd.6 on smithi086 2024-04-21T08:33:11.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:11 smithi179 ceph-mon[31829]: Deploying daemon osd.6 on smithi086 2024-04-21T08:33:11.747 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:11 smithi086 ceph-mon[27939]: Deploying daemon osd.6 on smithi086 2024-04-21T08:33:12.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:12 smithi086 ceph-mon[27939]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:12.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:12 smithi088 ceph-mon[31259]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:12.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:12 smithi179 ceph-mon[31829]: pgmap v147: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:14.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:14 smithi088 ceph-mon[31259]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:14.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:14 smithi179 ceph-mon[31829]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:14.821 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:14 smithi086 ceph-mon[27939]: pgmap v148: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:16.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:16 smithi088 ceph-mon[31259]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:16.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:16 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:16.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:16 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:16.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:16 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:33:16.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:16 smithi179 ceph-mon[31829]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:16.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:16 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:16.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:16 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:16.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:16 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:33:16.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:16 smithi086 ceph-mon[27939]: pgmap v149: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:16.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:16 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:16.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:16 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:16.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:16 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:33:18.588 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:18 smithi086 ceph-mon[27939]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:18.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:18 smithi088 ceph-mon[31259]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:18.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:18 smithi179 ceph-mon[31829]: pgmap v150: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:19.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:33:19.668+0000 7f6fed7fa700 1 -- 172.21.15.86:0/170525549 <== mgr.14152 v2:172.21.15.86:6800/1364156965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f6ff0062e80 con 0x7f6fdc0a3c40 2024-04-21T08:33:19.668 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 6 on host 'smithi086' 2024-04-21T08:33:19.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:33:19.670+0000 7f6ff5bdd700 1 -- 172.21.15.86:0/170525549 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6fdc0a3c40 msgr2=0x7f6fdc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:33:19.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:33:19.670+0000 7f6ff5bdd700 1 --2- 172.21.15.86:0/170525549 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6fdc0a3c40 0x7f6fdc0a6100 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f6fe000b5c0 tx=0x7f6fe0005fb0 comp rx=0 tx=0).stop 2024-04-21T08:33:19.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:33:19.670+0000 7f6ff5bdd700 1 -- 172.21.15.86:0/170525549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ff0108790 msgr2=0x7f6ff01a01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:33:19.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:33:19.670+0000 7f6ff5bdd700 1 --2- 172.21.15.86:0/170525549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ff0108790 0x7f6ff01a01d0 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7f6fe8007d40 tx=0x7f6fe800a340 comp rx=0 tx=0).stop 2024-04-21T08:33:19.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:33:19.671+0000 7f6ff5bdd700 1 -- 172.21.15.86:0/170525549 shutdown_connections 2024-04-21T08:33:19.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:33:19.671+0000 7f6ff5bdd700 1 --2- 172.21.15.86:0/170525549 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6fdc0a3c40 0x7f6fdc0a6100 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:33:19.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:33:19.671+0000 7f6ff5bdd700 1 --2- 172.21.15.86:0/170525549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6ff0108790 0x7f6ff01a01d0 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:33:19.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:33:19.671+0000 7f6ff5bdd700 1 --2- 172.21.15.86:0/170525549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6ff0103530 0x7f6ff0198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:33:19.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:33:19.671+0000 7f6ff5bdd700 1 --2- 172.21.15.86:0/170525549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6ff0105e60 0x7f6ff0199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:33:19.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:33:19.671+0000 7f6ff5bdd700 1 -- 172.21.15.86:0/170525549 >> 172.21.15.86:0/170525549 conn(0x7f6ff00fcf40 msgr2=0x7f6ff0103df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:33:19.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:33:19.671+0000 7f6ff5bdd700 1 -- 172.21.15.86:0/170525549 shutdown_connections 2024-04-21T08:33:19.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:33:19.671+0000 7f6ff5bdd700 1 -- 172.21.15.86:0/170525549 wait complete. 2024-04-21T08:33:19.990 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:19 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:19.991 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:19 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:19.991 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:19 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:19.991 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:19 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:33:19.991 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:19 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:19.991 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:19 smithi086 ceph-mon[27939]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:19.991 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:19 smithi086 ceph-mon[27939]: from='osd.6 [v2:172.21.15.86:6818/1224231118,v1:172.21.15.86:6819/1224231118]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-21T08:33:19.991 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:19 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:19.991 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:19 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:19.991 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:19 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:33:20.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:19 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:20.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:19 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:20.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:19 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:20.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:19 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:33:20.215 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:19 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:20.215 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:19 smithi088 ceph-mon[31259]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:20.215 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:19 smithi088 ceph-mon[31259]: from='osd.6 [v2:172.21.15.86:6818/1224231118,v1:172.21.15.86:6819/1224231118]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-21T08:33:20.215 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:19 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:20.215 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:19 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:20.215 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:19 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:33:20.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:19 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:20.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:19 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:20.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:19 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:20.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:19 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:33:20.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:19 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:20.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:19 smithi179 ceph-mon[31829]: pgmap v151: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:20.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:19 smithi179 ceph-mon[31829]: from='osd.6 [v2:172.21.15.86:6818/1224231118,v1:172.21.15.86:6819/1224231118]' entity='osd.6' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]: dispatch 2024-04-21T08:33:20.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:19 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:20.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:19 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:20.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:19 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:33:20.445 DEBUG:teuthology.orchestra.run.smithi086:osd.6> sudo journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@osd.6.service 2024-04-21T08:33:20.447 INFO:tasks.cephadm:Deploying osd.7 on smithi088 with /dev/vg_nvme/lv_2... 2024-04-21T08:33:20.447 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-21T08:33:20.690 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T08:33:21.070 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:20 smithi088 ceph-mon[31259]: from='osd.6 [v2:172.21.15.86:6818/1224231118,v1:172.21.15.86:6819/1224231118]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-21T08:33:21.070 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:20 smithi088 ceph-mon[31259]: osdmap e44: 7 total, 6 up, 7 in 2024-04-21T08:33:21.070 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:20 smithi088 ceph-mon[31259]: from='osd.6 [v2:172.21.15.86:6818/1224231118,v1:172.21.15.86:6819/1224231118]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-21T08:33:21.070 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:20 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:33:21.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:20 smithi086 ceph-mon[27939]: from='osd.6 [v2:172.21.15.86:6818/1224231118,v1:172.21.15.86:6819/1224231118]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-21T08:33:21.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:20 smithi086 ceph-mon[27939]: osdmap e44: 7 total, 6 up, 7 in 2024-04-21T08:33:21.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:20 smithi086 ceph-mon[27939]: from='osd.6 [v2:172.21.15.86:6818/1224231118,v1:172.21.15.86:6819/1224231118]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-21T08:33:21.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:20 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:33:21.117 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:33:20 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:33:20.839+0000 7f654d9cf700 -1 osd.6 0 waiting for initial osdmap 2024-04-21T08:33:21.117 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:33:20 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:33:20.848+0000 7f65497c2700 -1 osd.6 45 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T08:33:21.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:20 smithi179 ceph-mon[31829]: from='osd.6 [v2:172.21.15.86:6818/1224231118,v1:172.21.15.86:6819/1224231118]' entity='osd.6' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["6"]}]': finished 2024-04-21T08:33:21.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:20 smithi179 ceph-mon[31829]: osdmap e44: 7 total, 6 up, 7 in 2024-04-21T08:33:21.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:20 smithi179 ceph-mon[31829]: from='osd.6 [v2:172.21.15.86:6818/1224231118,v1:172.21.15.86:6819/1224231118]' entity='osd.6' cmd=[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-21T08:33:21.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:20 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:33:22.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:21 smithi086 ceph-mon[27939]: from='osd.6 [v2:172.21.15.86:6818/1224231118,v1:172.21.15.86:6819/1224231118]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-21T08:33:22.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:21 smithi086 ceph-mon[27939]: osdmap e45: 7 total, 6 up, 7 in 2024-04-21T08:33:22.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:21 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:33:22.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:21 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:33:22.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:21 smithi086 ceph-mon[27939]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:22.197 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:21 smithi088 ceph-mon[31259]: from='osd.6 [v2:172.21.15.86:6818/1224231118,v1:172.21.15.86:6819/1224231118]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-21T08:33:22.198 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:21 smithi088 ceph-mon[31259]: osdmap e45: 7 total, 6 up, 7 in 2024-04-21T08:33:22.198 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:21 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:33:22.198 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:21 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:33:22.198 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:21 smithi088 ceph-mon[31259]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:22.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:21 smithi179 ceph-mon[31829]: from='osd.6 [v2:172.21.15.86:6818/1224231118,v1:172.21.15.86:6819/1224231118]' entity='osd.6' cmd='[{"prefix": "osd crush create-or-move", "id": 6, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-21T08:33:22.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:21 smithi179 ceph-mon[31829]: osdmap e45: 7 total, 6 up, 7 in 2024-04-21T08:33:22.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:21 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:33:22.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:21 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:33:22.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:21 smithi179 ceph-mon[31829]: pgmap v154: 1 pgs: 1 active+clean; 577 KiB data, 183 MiB used, 536 GiB / 536 GiB avail 2024-04-21T08:33:22.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:22 smithi086 ceph-mon[27939]: purged_snaps scrub starts 2024-04-21T08:33:22.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:22 smithi086 ceph-mon[27939]: purged_snaps scrub ok 2024-04-21T08:33:22.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:22 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:33:22.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:22 smithi086 ceph-mon[27939]: osd.6 [v2:172.21.15.86:6818/1224231118,v1:172.21.15.86:6819/1224231118] boot 2024-04-21T08:33:22.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:22 smithi086 ceph-mon[27939]: osdmap e46: 7 total, 7 up, 7 in 2024-04-21T08:33:22.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:22 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:33:22.920 INFO:teuthology.orchestra.run.smithi088.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-21T08:33:22.920 INFO:teuthology.orchestra.run.smithi088.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-21T08:33:22.921 INFO:teuthology.orchestra.run.smithi088.stderr: stderr: 10+0 records in 2024-04-21T08:33:22.921 INFO:teuthology.orchestra.run.smithi088.stderr:10+0 records out 2024-04-21T08:33:22.921 INFO:teuthology.orchestra.run.smithi088.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0128843 s, 814 MB/s 2024-04-21T08:33:22.921 INFO:teuthology.orchestra.run.smithi088.stderr:--> Zapping successful for: 2024-04-21T08:33:23.213 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:22 smithi088 ceph-mon[31259]: purged_snaps scrub starts 2024-04-21T08:33:23.213 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:22 smithi088 ceph-mon[31259]: purged_snaps scrub ok 2024-04-21T08:33:23.213 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:22 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:33:23.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:22 smithi088 ceph-mon[31259]: osd.6 [v2:172.21.15.86:6818/1224231118,v1:172.21.15.86:6819/1224231118] boot 2024-04-21T08:33:23.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:22 smithi088 ceph-mon[31259]: osdmap e46: 7 total, 7 up, 7 in 2024-04-21T08:33:23.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:22 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:33:23.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:22 smithi179 ceph-mon[31829]: purged_snaps scrub starts 2024-04-21T08:33:23.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:22 smithi179 ceph-mon[31829]: purged_snaps scrub ok 2024-04-21T08:33:23.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:22 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:33:23.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:22 smithi179 ceph-mon[31829]: osd.6 [v2:172.21.15.86:6818/1224231118,v1:172.21.15.86:6819/1224231118] boot 2024-04-21T08:33:23.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:22 smithi179 ceph-mon[31829]: osdmap e46: 7 total, 7 up, 7 in 2024-04-21T08:33:23.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:22 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:33:23.600 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi088:vg_nvme/lv_2 2024-04-21T08:33:23.831 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T08:33:24.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:23 smithi086 ceph-mon[27939]: osdmap e47: 7 total, 7 up, 7 in 2024-04-21T08:33:24.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:23 smithi086 ceph-mon[27939]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:24.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:23 smithi086 ceph-mon[27939]: Detected new or changed devices on smithi086 2024-04-21T08:33:24.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:23 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:24.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:23 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:24.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:23 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:33:24.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:23 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:33:24.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:23 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:33:24.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:23 smithi086 ceph-mon[27939]: Adjusting osd_memory_target on smithi086 to 5696M 2024-04-21T08:33:24.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:23 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:24.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:23 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:24.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:23 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:33:24.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:23 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:24.143 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:23 smithi088 ceph-mon[31259]: osdmap e47: 7 total, 7 up, 7 in 2024-04-21T08:33:24.144 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:23 smithi088 ceph-mon[31259]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:24.144 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:23 smithi088 ceph-mon[31259]: Detected new or changed devices on smithi086 2024-04-21T08:33:24.144 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:23 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:24.144 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:23 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:24.144 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:23 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:33:24.144 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:23 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:33:24.144 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:23 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:33:24.144 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:23 smithi088 ceph-mon[31259]: Adjusting osd_memory_target on smithi086 to 5696M 2024-04-21T08:33:24.144 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:23 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:24.144 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:23 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:24.144 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:23 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:33:24.144 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:23 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:24.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:23 smithi179 ceph-mon[31829]: osdmap e47: 7 total, 7 up, 7 in 2024-04-21T08:33:24.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:23 smithi179 ceph-mon[31829]: pgmap v157: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:24.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:23 smithi179 ceph-mon[31829]: Detected new or changed devices on smithi086 2024-04-21T08:33:24.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:23 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:24.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:23 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:24.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:23 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:33:24.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:23 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:33:24.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:23 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:33:24.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:23 smithi179 ceph-mon[31829]: Adjusting osd_memory_target on smithi086 to 5696M 2024-04-21T08:33:24.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:23 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:24.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:23 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:24.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:23 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:33:24.248 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:23 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:25.348 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.347+0000 7f81a3b8d700 1 -- 172.21.15.88:0/330997238 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f819c102f60 msgr2=0x7f819c103380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:33:25.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.347+0000 7f81a3b8d700 1 --2- 172.21.15.88:0/330997238 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f819c102f60 0x7f819c103380 secure :-1 s=READY pgs=26 cs=0 l=1 rev1=1 crypto rx=0x7f8198009990 tx=0x7f8198009ca0 comp rx=0 tx=0).stop 2024-04-21T08:33:25.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.347+0000 7f81a3b8d700 1 -- 172.21.15.88:0/330997238 shutdown_connections 2024-04-21T08:33:25.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.347+0000 7f81a3b8d700 1 --2- 172.21.15.88:0/330997238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f819c102640 0x7f819c109f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:33:25.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.347+0000 7f81a3b8d700 1 --2- 172.21.15.88:0/330997238 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f819c102f60 0x7f819c103380 unknown :-1 s=CLOSED pgs=26 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:33:25.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.347+0000 7f81a3b8d700 1 --2- 172.21.15.88:0/330997238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f819c101ca0 0x7f819c102100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:33:25.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.347+0000 7f81a3b8d700 1 -- 172.21.15.88:0/330997238 >> 172.21.15.88:0/330997238 conn(0x7f819c0fc6f0 msgr2=0x7f819c0feb30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:33:25.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.347+0000 7f81a3b8d700 1 -- 172.21.15.88:0/330997238 shutdown_connections 2024-04-21T08:33:25.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.347+0000 7f81a3b8d700 1 -- 172.21.15.88:0/330997238 wait complete. 2024-04-21T08:33:25.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.347+0000 7f81a3b8d700 1 Processor -- start 2024-04-21T08:33:25.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.347+0000 7f81a3b8d700 1 -- start start 2024-04-21T08:33:25.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.348+0000 7f81a3b8d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f819c101ca0 0x7f819c101710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:33:25.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.348+0000 7f81a3b8d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f819c102640 0x7f819c19d940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:33:25.349 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.348+0000 7f81a3b8d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f819c102f60 0x7f819c19ff90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:33:25.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.348+0000 7f81a3b8d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f819c10d4c0 con 0x7f819c102f60 2024-04-21T08:33:25.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.348+0000 7f81a3b8d700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f819c10d340 con 0x7f819c102640 2024-04-21T08:33:25.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.348+0000 7f81a3b8d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f819c10d640 con 0x7f819c101ca0 2024-04-21T08:33:25.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.348+0000 7f81a1128700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f819c102640 0x7f819c19d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:33:25.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.348+0000 7f81a1128700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f819c102640 0x7f819c19d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:60906/0 (socket says 172.21.15.88:60906) 2024-04-21T08:33:25.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.348+0000 7f81a1128700 1 -- 172.21.15.88:0/2380706549 learned_addr learned my addr 172.21.15.88:0/2380706549 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-21T08:33:25.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.348+0000 7f81a212a700 1 --2- 172.21.15.88:0/2380706549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f819c102f60 0x7f819c19ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:33:25.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.348+0000 7f81a1929700 1 --2- 172.21.15.88:0/2380706549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f819c101ca0 0x7f819c101710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:33:25.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.348+0000 7f81a1128700 1 -- 172.21.15.88:0/2380706549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f819c101ca0 msgr2=0x7f819c101710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:33:25.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.348+0000 7f81a1128700 1 --2- 172.21.15.88:0/2380706549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f819c101ca0 0x7f819c101710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:33:25.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.348+0000 7f81a1128700 1 -- 172.21.15.88:0/2380706549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f819c102f60 msgr2=0x7f819c19ff90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:33:25.350 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.348+0000 7f81a1128700 1 --2- 172.21.15.88:0/2380706549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f819c102f60 0x7f819c19ff90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:33:25.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.348+0000 7f81a1128700 1 -- 172.21.15.88:0/2380706549 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f819c1a0860 con 0x7f819c102640 2024-04-21T08:33:25.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.348+0000 7f81a1929700 1 --2- 172.21.15.88:0/2380706549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f819c101ca0 0x7f819c101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:33:25.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.348+0000 7f81a1128700 1 --2- 172.21.15.88:0/2380706549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f819c102640 0x7f819c19d940 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f818c00d950 tx=0x7f818c00dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:33:25.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.348+0000 7f8192ffd700 1 -- 172.21.15.88:0/2380706549 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818c016070 con 0x7f819c102640 2024-04-21T08:33:25.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.348+0000 7f81a3b8d700 1 -- 172.21.15.88:0/2380706549 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f819c1a0b50 con 0x7f819c102640 2024-04-21T08:33:25.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.348+0000 7f8192ffd700 1 -- 172.21.15.88:0/2380706549 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f818c009df0 con 0x7f819c102640 2024-04-21T08:33:25.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.348+0000 7f8192ffd700 1 -- 172.21.15.88:0/2380706549 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f818c00f3c0 con 0x7f819c102640 2024-04-21T08:33:25.351 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.349+0000 7f81a3b8d700 1 -- 172.21.15.88:0/2380706549 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f819c1a10e0 con 0x7f819c102640 2024-04-21T08:33:25.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.349+0000 7f81a3b8d700 1 -- 172.21.15.88:0/2380706549 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f819c069730 con 0x7f819c102640 2024-04-21T08:33:25.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.350+0000 7f8192ffd700 1 -- 172.21.15.88:0/2380706549 <== mon.2 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f818c019070 con 0x7f819c102640 2024-04-21T08:33:25.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.351+0000 7f8192ffd700 1 --2- 172.21.15.88:0/2380706549 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f81880a3c40 0x7f81880a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:33:25.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.351+0000 7f8192ffd700 1 -- 172.21.15.88:0/2380706549 <== mon.2 v2:172.21.15.88:3300/0 5 ==== osd_map(47..47 src has 1..47) ==== 4965+0+0 (secure 0 0 0) 0x7f818c0cf2c0 con 0x7f819c102640 2024-04-21T08:33:25.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.351+0000 7f81a1929700 1 --2- 172.21.15.88:0/2380706549 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f81880a3c40 0x7f81880a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:33:25.352 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.352+0000 7f81a1929700 1 --2- 172.21.15.88:0/2380706549 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f81880a3c40 0x7f81880a6100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f81980038a0 tx=0x7f8198005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:33:25.355 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.356+0000 7f8192ffd700 1 -- 172.21.15.88:0/2380706549 <== mon.2 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f818c09c9b0 con 0x7f819c102640 2024-04-21T08:33:25.560 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:25.560+0000 7f81a3b8d700 1 -- 172.21.15.88:0/2380706549 --> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7f819c062e80 con 0x7f81880a3c40 2024-04-21T08:33:26.463 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:26 smithi088 ceph-mon[31259]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:26.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:26 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:33:26.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:26 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:33:26.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:26 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:26.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:26 smithi086 ceph-mon[27939]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:26.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:26 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:33:26.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:26 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:33:26.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:26 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:26.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:26 smithi179 ceph-mon[31829]: pgmap v158: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:26.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:26 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:33:26.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:26 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:33:26.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:26 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:27.435 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:27 smithi088 ceph-mon[31259]: from='client.24251 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:33:27.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:27 smithi086 ceph-mon[27939]: from='client.24251 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:33:27.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:27 smithi179 ceph-mon[31829]: from='client.24251 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:33:28.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:28 smithi086 ceph-mon[27939]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:28.652 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:28 smithi088 ceph-mon[31259]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:28.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:28 smithi179 ceph-mon[31829]: pgmap v159: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:29.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:29 smithi086 ceph-mon[27939]: from='client.? 172.21.15.88:0/1999228225' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5"}]: dispatch 2024-04-21T08:33:29.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:29 smithi086 ceph-mon[27939]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5"}]: dispatch 2024-04-21T08:33:29.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:29 smithi086 ceph-mon[27939]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5"}]': finished 2024-04-21T08:33:29.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:29 smithi086 ceph-mon[27939]: osdmap e48: 8 total, 7 up, 8 in 2024-04-21T08:33:29.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:29 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:33:29.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:29 smithi086 ceph-mon[27939]: from='client.? 172.21.15.88:0/3268720544' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:33:29.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:29 smithi088 ceph-mon[31259]: from='client.? 172.21.15.88:0/1999228225' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5"}]: dispatch 2024-04-21T08:33:29.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:29 smithi088 ceph-mon[31259]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5"}]: dispatch 2024-04-21T08:33:29.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:29 smithi088 ceph-mon[31259]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5"}]': finished 2024-04-21T08:33:29.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:29 smithi088 ceph-mon[31259]: osdmap e48: 8 total, 7 up, 8 in 2024-04-21T08:33:29.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:29 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:33:29.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:29 smithi088 ceph-mon[31259]: from='client.? 172.21.15.88:0/3268720544' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:33:29.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:29 smithi179 ceph-mon[31829]: from='client.? 172.21.15.88:0/1999228225' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5"}]: dispatch 2024-04-21T08:33:29.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:29 smithi179 ceph-mon[31829]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5"}]: dispatch 2024-04-21T08:33:29.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:29 smithi179 ceph-mon[31829]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5"}]': finished 2024-04-21T08:33:29.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:29 smithi179 ceph-mon[31829]: osdmap e48: 8 total, 7 up, 8 in 2024-04-21T08:33:29.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:29 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:33:29.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:29 smithi179 ceph-mon[31829]: from='client.? 172.21.15.88:0/3268720544' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:33:30.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:30 smithi086 ceph-mon[27939]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:30.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:30 smithi088 ceph-mon[31259]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:30.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:30 smithi179 ceph-mon[31829]: pgmap v161: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:32.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:32 smithi086 ceph-mon[27939]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:32.660 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:32 smithi088 ceph-mon[31259]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:32.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:32 smithi179 ceph-mon[31829]: pgmap v162: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:34.537 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:34 smithi088 ceph-mon[31259]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:34.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:34 smithi086 ceph-mon[27939]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:34.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:34 smithi179 ceph-mon[31829]: pgmap v163: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:36.479 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:36 smithi088 ceph-mon[31259]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:36.479 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:36 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-21T08:33:36.479 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:36 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:36.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:36 smithi086 ceph-mon[27939]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:36.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:36 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-21T08:33:36.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:36 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:36.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:36 smithi179 ceph-mon[31829]: pgmap v164: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:36.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:36 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.7"}]: dispatch 2024-04-21T08:33:36.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:36 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:37.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:37 smithi086 ceph-mon[27939]: Deploying daemon osd.7 on smithi088 2024-04-21T08:33:37.620 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:37 smithi088 ceph-mon[31259]: Deploying daemon osd.7 on smithi088 2024-04-21T08:33:37.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:37 smithi179 ceph-mon[31829]: Deploying daemon osd.7 on smithi088 2024-04-21T08:33:38.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:38 smithi086 ceph-mon[27939]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:38.633 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:38 smithi088 ceph-mon[31259]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:38.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:38 smithi179 ceph-mon[31829]: pgmap v165: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:40.442 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:40 smithi088 ceph-mon[31259]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:40.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:40 smithi086 ceph-mon[27939]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:40.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:40 smithi179 ceph-mon[31829]: pgmap v166: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:42.463 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:42 smithi088 ceph-mon[31259]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:42.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:42 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:42.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:42 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:42.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:42 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:33:42.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:42 smithi086 ceph-mon[27939]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:42.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:42 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:42.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:42 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:42.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:42 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:33:42.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:42 smithi179 ceph-mon[31829]: pgmap v167: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:42.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:42 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:42.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:42 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:42.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:42 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:33:44.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:44 smithi086 ceph-mon[27939]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:44.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:44 smithi088 ceph-mon[31259]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:44.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:44 smithi179 ceph-mon[31829]: pgmap v168: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:44.825 INFO:teuthology.orchestra.run.smithi088.stdout:Created osd(s) 7 on host 'smithi088' 2024-04-21T08:33:44.825 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:44.825+0000 7f8192ffd700 1 -- 172.21.15.88:0/2380706549 <== mgr.14152 v2:172.21.15.86:6800/1364156965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f819c062e80 con 0x7f81880a3c40 2024-04-21T08:33:44.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:44.827+0000 7f81a3b8d700 1 -- 172.21.15.88:0/2380706549 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f81880a3c40 msgr2=0x7f81880a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:33:44.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:44.827+0000 7f81a3b8d700 1 --2- 172.21.15.88:0/2380706549 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f81880a3c40 0x7f81880a6100 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7f81980038a0 tx=0x7f8198005fd0 comp rx=0 tx=0).stop 2024-04-21T08:33:44.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:44.827+0000 7f81a3b8d700 1 -- 172.21.15.88:0/2380706549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f819c102640 msgr2=0x7f819c19d940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:33:44.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:44.827+0000 7f81a3b8d700 1 --2- 172.21.15.88:0/2380706549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f819c102640 0x7f819c19d940 secure :-1 s=READY pgs=27 cs=0 l=1 rev1=1 crypto rx=0x7f818c00d950 tx=0x7f818c00dc60 comp rx=0 tx=0).stop 2024-04-21T08:33:44.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:44.827+0000 7f81a3b8d700 1 -- 172.21.15.88:0/2380706549 shutdown_connections 2024-04-21T08:33:44.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:44.827+0000 7f81a3b8d700 1 --2- 172.21.15.88:0/2380706549 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f81880a3c40 0x7f81880a6100 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:33:44.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:44.827+0000 7f81a3b8d700 1 --2- 172.21.15.88:0/2380706549 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f819c102f60 0x7f819c19ff90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:33:44.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:44.827+0000 7f81a3b8d700 1 --2- 172.21.15.88:0/2380706549 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f819c101ca0 0x7f819c101710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:33:44.827 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:44.827+0000 7f81a3b8d700 1 --2- 172.21.15.88:0/2380706549 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f819c102640 0x7f819c19d940 unknown :-1 s=CLOSED pgs=27 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:33:44.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:44.827+0000 7f81a3b8d700 1 -- 172.21.15.88:0/2380706549 >> 172.21.15.88:0/2380706549 conn(0x7f819c0fc6f0 msgr2=0x7f819c105fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:33:44.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:44.827+0000 7f81a3b8d700 1 -- 172.21.15.88:0/2380706549 shutdown_connections 2024-04-21T08:33:44.828 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:33:44.827+0000 7f81a3b8d700 1 -- 172.21.15.88:0/2380706549 wait complete. 2024-04-21T08:33:45.466 DEBUG:teuthology.orchestra.run.smithi088:osd.7> sudo journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@osd.7.service 2024-04-21T08:33:45.468 INFO:tasks.cephadm:Deploying osd.8 on smithi179 with /dev/vg_nvme/lv_2... 2024-04-21T08:33:45.468 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_2 2024-04-21T08:33:45.715 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:33:45.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:45 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:45.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:45 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:45.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:45 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:45.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:45 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:45.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:45 smithi088 ceph-mon[31259]: from='osd.7 [v2:172.21.15.88:6816/136629878,v1:172.21.15.88:6817/136629878]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-21T08:33:45.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:45 smithi088 ceph-mon[31259]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-21T08:33:45.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:45 smithi088 ceph-mon[31259]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:46.094 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:45 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:46.094 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:45 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:46.094 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:45 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:46.095 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:45 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:46.095 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:45 smithi179 ceph-mon[31829]: from='osd.7 [v2:172.21.15.88:6816/136629878,v1:172.21.15.88:6817/136629878]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-21T08:33:46.095 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:45 smithi179 ceph-mon[31829]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-21T08:33:46.095 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:45 smithi179 ceph-mon[31829]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:46.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:45 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:46.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:45 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:46.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:45 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:46.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:45 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:46.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:45 smithi086 ceph-mon[27939]: from='osd.7 [v2:172.21.15.88:6816/136629878,v1:172.21.15.88:6817/136629878]' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-21T08:33:46.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:45 smithi086 ceph-mon[27939]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]: dispatch 2024-04-21T08:33:46.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:45 smithi086 ceph-mon[27939]: pgmap v169: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:47.211 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:47 smithi179 ceph-mon[31829]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-21T08:33:47.211 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:47 smithi179 ceph-mon[31829]: from='osd.7 [v2:172.21.15.88:6816/136629878,v1:172.21.15.88:6817/136629878]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-21T08:33:47.211 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:47 smithi179 ceph-mon[31829]: osdmap e49: 8 total, 7 up, 8 in 2024-04-21T08:33:47.211 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:47 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:33:47.212 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:47 smithi179 ceph-mon[31829]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-21T08:33:47.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:47 smithi086 ceph-mon[27939]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-21T08:33:47.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:47 smithi086 ceph-mon[27939]: from='osd.7 [v2:172.21.15.88:6816/136629878,v1:172.21.15.88:6817/136629878]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-21T08:33:47.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:47 smithi086 ceph-mon[27939]: osdmap e49: 8 total, 7 up, 8 in 2024-04-21T08:33:47.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:47 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:33:47.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:47 smithi086 ceph-mon[27939]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-21T08:33:47.463 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:47 smithi088 ceph-mon[31259]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["7"]}]': finished 2024-04-21T08:33:47.463 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:47 smithi088 ceph-mon[31259]: from='osd.7 [v2:172.21.15.88:6816/136629878,v1:172.21.15.88:6817/136629878]' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-21T08:33:47.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:47 smithi088 ceph-mon[31259]: osdmap e49: 8 total, 7 up, 8 in 2024-04-21T08:33:47.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:47 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:33:47.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:47 smithi088 ceph-mon[31259]: from='osd.7 ' entity='osd.7' cmd=[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-21T08:33:47.464 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:33:47 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:33:47.098+0000 7f3ac99a9700 -1 osd.7 0 waiting for initial osdmap 2024-04-21T08:33:47.464 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:33:47 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:33:47.107+0000 7f3ac579c700 -1 osd.7 50 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T08:33:47.791 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping: /dev/vg_nvme/lv_2 2024-04-21T08:33:47.791 INFO:teuthology.orchestra.run.smithi179.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_2 bs=1M count=10 conv=fsync 2024-04-21T08:33:47.791 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10+0 records in 2024-04-21T08:33:47.791 INFO:teuthology.orchestra.run.smithi179.stderr:10+0 records out 2024-04-21T08:33:47.791 INFO:teuthology.orchestra.run.smithi179.stderr:10485760 bytes (10 MB, 10 MiB) copied, 0.0127514 s, 822 MB/s 2024-04-21T08:33:47.791 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping successful for: 2024-04-21T08:33:48.362 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:48 smithi088 ceph-mon[31259]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]': finished 2024-04-21T08:33:48.362 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:48 smithi088 ceph-mon[31259]: osdmap e50: 8 total, 7 up, 8 in 2024-04-21T08:33:48.362 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:48 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:33:48.362 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:48 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:33:48.362 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:48 smithi088 ceph-mon[31259]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:48.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:48 smithi086 ceph-mon[27939]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]': finished 2024-04-21T08:33:48.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:48 smithi086 ceph-mon[27939]: osdmap e50: 8 total, 7 up, 8 in 2024-04-21T08:33:48.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:48 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:33:48.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:48 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:33:48.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:48 smithi086 ceph-mon[27939]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:48.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:48 smithi179 ceph-mon[31829]: from='osd.7 ' entity='osd.7' cmd='[{"prefix": "osd crush create-or-move", "id": 7, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]': finished 2024-04-21T08:33:48.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:48 smithi179 ceph-mon[31829]: osdmap e50: 8 total, 7 up, 8 in 2024-04-21T08:33:48.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:48 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:33:48.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:48 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:33:48.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:48 smithi179 ceph-mon[31829]: pgmap v172: 1 pgs: 1 active+clean; 577 KiB data, 210 MiB used, 626 GiB / 626 GiB avail 2024-04-21T08:33:48.553 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi179:vg_nvme/lv_2 2024-04-21T08:33:48.798 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:33:49.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:49 smithi086 ceph-mon[27939]: purged_snaps scrub starts 2024-04-21T08:33:49.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:49 smithi086 ceph-mon[27939]: purged_snaps scrub ok 2024-04-21T08:33:49.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:49 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:33:49.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:49 smithi086 ceph-mon[27939]: osd.7 [v2:172.21.15.88:6816/136629878,v1:172.21.15.88:6817/136629878] boot 2024-04-21T08:33:49.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:49 smithi086 ceph-mon[27939]: osdmap e51: 8 total, 8 up, 8 in 2024-04-21T08:33:49.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:49 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:33:49.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:49 smithi086 ceph-mon[27939]: Detected new or changed devices on smithi088 2024-04-21T08:33:49.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:49 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:49.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:49 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:49.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:49 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:33:49.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:49 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:33:49.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:49 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:33:49.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:49 smithi086 ceph-mon[27939]: Adjusting osd_memory_target on smithi088 to 5696M 2024-04-21T08:33:49.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:49 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:49.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:49 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:49.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:49 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:33:49.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:49 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:49.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:49 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:33:49.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:49 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:49.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:49 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:33:49.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:49 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:49.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:49 smithi088 ceph-mon[31259]: purged_snaps scrub starts 2024-04-21T08:33:49.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:49 smithi088 ceph-mon[31259]: purged_snaps scrub ok 2024-04-21T08:33:49.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:49 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:33:49.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:49 smithi088 ceph-mon[31259]: osd.7 [v2:172.21.15.88:6816/136629878,v1:172.21.15.88:6817/136629878] boot 2024-04-21T08:33:49.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:49 smithi088 ceph-mon[31259]: osdmap e51: 8 total, 8 up, 8 in 2024-04-21T08:33:49.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:49 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:33:49.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:49 smithi088 ceph-mon[31259]: Detected new or changed devices on smithi088 2024-04-21T08:33:49.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:49 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:49.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:49 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:49.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:49 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:33:49.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:49 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:33:49.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:49 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:33:49.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:49 smithi088 ceph-mon[31259]: Adjusting osd_memory_target on smithi088 to 5696M 2024-04-21T08:33:49.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:49 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:49.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:49 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:49.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:49 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:33:49.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:49 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:49.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:49 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:33:49.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:49 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:49.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:49 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:33:49.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:49 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:49.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:49 smithi179 ceph-mon[31829]: purged_snaps scrub starts 2024-04-21T08:33:49.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:49 smithi179 ceph-mon[31829]: purged_snaps scrub ok 2024-04-21T08:33:49.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:49 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:33:49.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:49 smithi179 ceph-mon[31829]: osd.7 [v2:172.21.15.88:6816/136629878,v1:172.21.15.88:6817/136629878] boot 2024-04-21T08:33:49.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:49 smithi179 ceph-mon[31829]: osdmap e51: 8 total, 8 up, 8 in 2024-04-21T08:33:49.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:49 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:33:49.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:49 smithi179 ceph-mon[31829]: Detected new or changed devices on smithi088 2024-04-21T08:33:49.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:49 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:49.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:49 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:49.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:49 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:33:49.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:49 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:33:49.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:49 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:33:49.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:49 smithi179 ceph-mon[31829]: Adjusting osd_memory_target on smithi088 to 5696M 2024-04-21T08:33:49.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:49 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:49.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:49 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:49.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:49 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:33:49.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:49 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:49.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:49 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:33:49.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:49 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:49.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:49 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:33:49.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:49 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:33:50.157 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:50 smithi179 ceph-mon[31829]: osdmap e52: 8 total, 8 up, 8 in 2024-04-21T08:33:50.157 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:50 smithi179 ceph-mon[31829]: pgmap v175: 1 pgs: 1 unknown; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:33:50.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.396+0000 7fbf5b898700 1 -- 172.21.15.179:0/3439108688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf54108790 msgr2=0x7fbf5410ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:33:50.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.396+0000 7fbf5b898700 1 --2- 172.21.15.179:0/3439108688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf54108790 0x7fbf5410ac00 secure :-1 s=READY pgs=29 cs=0 l=1 rev1=1 crypto rx=0x7fbf4c00b0a0 tx=0x7fbf4c00b3b0 comp rx=0 tx=0).stop 2024-04-21T08:33:50.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.396+0000 7fbf5b898700 1 -- 172.21.15.179:0/3439108688 shutdown_connections 2024-04-21T08:33:50.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.396+0000 7fbf5b898700 1 --2- 172.21.15.179:0/3439108688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf54108790 0x7fbf5410ac00 unknown :-1 s=CLOSED pgs=29 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:33:50.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.396+0000 7fbf5b898700 1 --2- 172.21.15.179:0/3439108688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf54103530 0x7fbf54105920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:33:50.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.396+0000 7fbf5b898700 1 --2- 172.21.15.179:0/3439108688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf54105e60 0x7fbf54108250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:33:50.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.396+0000 7fbf5b898700 1 -- 172.21.15.179:0/3439108688 >> 172.21.15.179:0/3439108688 conn(0x7fbf540fcf40 msgr2=0x7fbf540ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:33:50.397 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.396+0000 7fbf5b898700 1 -- 172.21.15.179:0/3439108688 shutdown_connections 2024-04-21T08:33:50.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.396+0000 7fbf5b898700 1 -- 172.21.15.179:0/3439108688 wait complete. 2024-04-21T08:33:50.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.396+0000 7fbf5b898700 1 Processor -- start 2024-04-21T08:33:50.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.396+0000 7fbf5b898700 1 -- start start 2024-04-21T08:33:50.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.397+0000 7fbf5b898700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf54103530 0x7fbf54198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:33:50.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.397+0000 7fbf5b898700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf54105e60 0x7fbf54199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:33:50.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.397+0000 7fbf5b898700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf54108790 0x7fbf541a0190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:33:50.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.397+0000 7fbf5b898700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5410e520 con 0x7fbf54105e60 2024-04-21T08:33:50.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.397+0000 7fbf5b898700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5410e3a0 con 0x7fbf54108790 2024-04-21T08:33:50.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.397+0000 7fbf5b898700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbf5410e6a0 con 0x7fbf54103530 2024-04-21T08:33:50.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.397+0000 7fbf59634700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf54103530 0x7fbf54198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:33:50.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.397+0000 7fbf59634700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf54103530 0x7fbf54198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:49140/0 (socket says 172.21.15.179:49140) 2024-04-21T08:33:50.398 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.397+0000 7fbf59634700 1 -- 172.21.15.179:0/733739347 learned_addr learned my addr 172.21.15.179:0/733739347 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-21T08:33:50.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.397+0000 7fbf59e35700 1 --2- 172.21.15.179:0/733739347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf54108790 0x7fbf541a0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:33:50.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.397+0000 7fbf58e33700 1 --2- 172.21.15.179:0/733739347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf54105e60 0x7fbf54199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:33:50.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.397+0000 7fbf59634700 1 -- 172.21.15.179:0/733739347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf54108790 msgr2=0x7fbf541a0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:33:50.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.397+0000 7fbf59634700 1 --2- 172.21.15.179:0/733739347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf54108790 0x7fbf541a0190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:33:50.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.397+0000 7fbf59634700 1 -- 172.21.15.179:0/733739347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf54105e60 msgr2=0x7fbf54199150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:33:50.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.397+0000 7fbf59634700 1 --2- 172.21.15.179:0/733739347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf54105e60 0x7fbf54199150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:33:50.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.397+0000 7fbf59634700 1 -- 172.21.15.179:0/733739347 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf541a0860 con 0x7fbf54103530 2024-04-21T08:33:50.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.397+0000 7fbf59634700 1 --2- 172.21.15.179:0/733739347 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf54103530 0x7fbf54198c10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbf4400e9c0 tx=0x7fbf4400ecd0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:33:50.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.397+0000 7fbf4a7fc700 1 -- 172.21.15.179:0/733739347 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf4400cbd0 con 0x7fbf54103530 2024-04-21T08:33:50.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.397+0000 7fbf4a7fc700 1 -- 172.21.15.179:0/733739347 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbf4400cd70 con 0x7fbf54103530 2024-04-21T08:33:50.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.397+0000 7fbf4a7fc700 1 -- 172.21.15.179:0/733739347 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbf44005610 con 0x7fbf54103530 2024-04-21T08:33:50.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.397+0000 7fbf5b898700 1 -- 172.21.15.179:0/733739347 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf541a0b50 con 0x7fbf54103530 2024-04-21T08:33:50.399 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.398+0000 7fbf5b898700 1 -- 172.21.15.179:0/733739347 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf541a1040 con 0x7fbf54103530 2024-04-21T08:33:50.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.398+0000 7fbf5b898700 1 -- 172.21.15.179:0/733739347 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbf54069730 con 0x7fbf54103530 2024-04-21T08:33:50.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.399+0000 7fbf4a7fc700 1 -- 172.21.15.179:0/733739347 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fbf44004750 con 0x7fbf54103530 2024-04-21T08:33:50.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.400+0000 7fbf4a7fc700 1 --2- 172.21.15.179:0/733739347 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fbf400a3c40 0x7fbf400a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:33:50.400 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.400+0000 7fbf4a7fc700 1 -- 172.21.15.179:0/733739347 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(54..54 src has 1..54) ==== 5397+0+0 (secure 0 0 0) 0x7fbf440cfbf0 con 0x7fbf54103530 2024-04-21T08:33:50.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.400+0000 7fbf58e33700 1 --2- 172.21.15.179:0/733739347 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fbf400a3c40 0x7fbf400a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:33:50.401 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.401+0000 7fbf58e33700 1 --2- 172.21.15.179:0/733739347 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fbf400a3c40 0x7fbf400a6100 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbf50009bf0 tx=0x7fbf50009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:33:50.404 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.405+0000 7fbf4a7fc700 1 -- 172.21.15.179:0/733739347 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbf4409d130 con 0x7fbf54103530 2024-04-21T08:33:50.463 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:50 smithi088 ceph-mon[31259]: osdmap e52: 8 total, 8 up, 8 in 2024-04-21T08:33:50.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:50 smithi088 ceph-mon[31259]: pgmap v175: 1 pgs: 1 unknown; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:33:50.610 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:33:50.610+0000 7fbf5b898700 1 -- 172.21.15.179:0/733739347 --> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_2", "target": ["mon-mgr", ""]}) -- 0x7fbf54062e80 con 0x7fbf400a3c40 2024-04-21T08:33:50.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:50 smithi086 ceph-mon[27939]: osdmap e52: 8 total, 8 up, 8 in 2024-04-21T08:33:50.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:50 smithi086 ceph-mon[27939]: pgmap v175: 1 pgs: 1 unknown; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:33:51.463 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:51 smithi088 ceph-mon[31259]: osdmap e53: 8 total, 8 up, 8 in 2024-04-21T08:33:51.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:51 smithi088 ceph-mon[31259]: osdmap e54: 8 total, 8 up, 8 in 2024-04-21T08:33:51.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:51 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:33:51.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:51 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:33:51.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:51 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:51.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:51 smithi179 ceph-mon[31829]: osdmap e53: 8 total, 8 up, 8 in 2024-04-21T08:33:51.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:51 smithi179 ceph-mon[31829]: osdmap e54: 8 total, 8 up, 8 in 2024-04-21T08:33:51.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:51 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:33:51.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:51 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:33:51.478 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:51 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:51.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:51 smithi086 ceph-mon[27939]: osdmap e53: 8 total, 8 up, 8 in 2024-04-21T08:33:51.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:51 smithi086 ceph-mon[27939]: osdmap e54: 8 total, 8 up, 8 in 2024-04-21T08:33:51.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:51 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:33:51.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:51 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:33:51.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:51 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:33:52.473 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:52 smithi179 ceph-mon[31829]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:33:52.473 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:52 smithi179 ceph-mon[31829]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:33:52.473 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:52 smithi179 ceph-mon[31829]: osdmap e55: 8 total, 8 up, 8 in 2024-04-21T08:33:52.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:52 smithi088 ceph-mon[31259]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:33:52.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:52 smithi088 ceph-mon[31259]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:33:52.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:52 smithi088 ceph-mon[31259]: osdmap e55: 8 total, 8 up, 8 in 2024-04-21T08:33:52.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:52 smithi086 ceph-mon[27939]: from='client.24277 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_2", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:33:52.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:52 smithi086 ceph-mon[27939]: pgmap v178: 1 pgs: 1 unknown; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:33:52.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:52 smithi086 ceph-mon[27939]: osdmap e55: 8 total, 8 up, 8 in 2024-04-21T08:33:53.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:53 smithi179 ceph-mon[31829]: from='client.? 172.21.15.179:0/2171145000' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63e59f4b-0426-46db-92f7-c55dae61b5ee"}]: dispatch 2024-04-21T08:33:53.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:53 smithi179 ceph-mon[31829]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63e59f4b-0426-46db-92f7-c55dae61b5ee"}]: dispatch 2024-04-21T08:33:53.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:53 smithi179 ceph-mon[31829]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63e59f4b-0426-46db-92f7-c55dae61b5ee"}]': finished 2024-04-21T08:33:53.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:53 smithi179 ceph-mon[31829]: osdmap e56: 9 total, 8 up, 9 in 2024-04-21T08:33:53.687 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:53 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:33:53.692 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:53 smithi086 ceph-mon[27939]: from='client.? 172.21.15.179:0/2171145000' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63e59f4b-0426-46db-92f7-c55dae61b5ee"}]: dispatch 2024-04-21T08:33:53.692 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:53 smithi086 ceph-mon[27939]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63e59f4b-0426-46db-92f7-c55dae61b5ee"}]: dispatch 2024-04-21T08:33:53.692 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:53 smithi086 ceph-mon[27939]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63e59f4b-0426-46db-92f7-c55dae61b5ee"}]': finished 2024-04-21T08:33:53.693 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:53 smithi086 ceph-mon[27939]: osdmap e56: 9 total, 8 up, 9 in 2024-04-21T08:33:53.693 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:53 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:33:53.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:53 smithi088 ceph-mon[31259]: from='client.? 172.21.15.179:0/2171145000' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63e59f4b-0426-46db-92f7-c55dae61b5ee"}]: dispatch 2024-04-21T08:33:53.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:53 smithi088 ceph-mon[31259]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "63e59f4b-0426-46db-92f7-c55dae61b5ee"}]: dispatch 2024-04-21T08:33:53.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:53 smithi088 ceph-mon[31259]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "63e59f4b-0426-46db-92f7-c55dae61b5ee"}]': finished 2024-04-21T08:33:53.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:53 smithi088 ceph-mon[31259]: osdmap e56: 9 total, 8 up, 9 in 2024-04-21T08:33:53.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:53 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:33:54.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:54 smithi088 ceph-mon[31259]: pgmap v180: 1 pgs: 1 unknown; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:33:54.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:54 smithi088 ceph-mon[31259]: from='client.? 172.21.15.179:0/4175063876' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:33:54.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:54 smithi179 ceph-mon[31829]: pgmap v180: 1 pgs: 1 unknown; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:33:54.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:54 smithi179 ceph-mon[31829]: from='client.? 172.21.15.179:0/4175063876' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:33:54.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:54 smithi086 ceph-mon[27939]: pgmap v180: 1 pgs: 1 unknown; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:33:54.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:54 smithi086 ceph-mon[27939]: from='client.? 172.21.15.179:0/4175063876' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:33:56.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:56 smithi088 ceph-mon[31259]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:33:56.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:56 smithi179 ceph-mon[31829]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:33:56.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:56 smithi086 ceph-mon[27939]: pgmap v182: 1 pgs: 1 active+clean; 577 KiB data, 238 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:33:58.636 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:33:58 smithi179 ceph-mon[31829]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:33:58.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:33:58 smithi088 ceph-mon[31259]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:33:58.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:33:58 smithi086 ceph-mon[27939]: pgmap v183: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:00.623 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:00 smithi179 ceph-mon[31829]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:00.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:00 smithi088 ceph-mon[31259]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:00.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:00 smithi086 ceph-mon[27939]: pgmap v184: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:01.601 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:01 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-21T08:34:01.601 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:01 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:01.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:01 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-21T08:34:01.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:01 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:01.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:01 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.8"}]: dispatch 2024-04-21T08:34:01.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:01 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:02.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:02 smithi179 ceph-mon[31829]: Deploying daemon osd.8 on smithi179 2024-04-21T08:34:02.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:02 smithi179 ceph-mon[31829]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:02.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:02 smithi088 ceph-mon[31259]: Deploying daemon osd.8 on smithi179 2024-04-21T08:34:02.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:02 smithi088 ceph-mon[31259]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:02.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:02 smithi086 ceph-mon[27939]: Deploying daemon osd.8 on smithi179 2024-04-21T08:34:02.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:02 smithi086 ceph-mon[27939]: pgmap v185: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:04.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:04 smithi088 ceph-mon[31259]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:04.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:04 smithi179 ceph-mon[31829]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:04.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:04 smithi086 ceph-mon[27939]: pgmap v186: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:06.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:06 smithi088 ceph-mon[31259]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:06.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:06 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:06.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:06 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:06.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:06 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:34:06.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:06 smithi179 ceph-mon[31829]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:06.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:06 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:06.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:06 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:06.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:06 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:34:06.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:06 smithi086 ceph-mon[27939]: pgmap v187: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:06.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:06 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:06.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:06 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:06.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:06 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:34:08.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:08 smithi088 ceph-mon[31259]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:08.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:08 smithi179 ceph-mon[31829]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:08.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:08 smithi086 ceph-mon[27939]: pgmap v188: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:10.175 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:34:10.175+0000 7fbf4a7fc700 1 -- 172.21.15.179:0/733739347 <== mgr.14152 v2:172.21.15.86:6800/1364156965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7fbf54062e80 con 0x7fbf400a3c40 2024-04-21T08:34:10.175 INFO:teuthology.orchestra.run.smithi179.stdout:Created osd(s) 8 on host 'smithi179' 2024-04-21T08:34:10.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:34:10.178+0000 7fbf5b898700 1 -- 172.21.15.179:0/733739347 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fbf400a3c40 msgr2=0x7fbf400a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:34:10.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:34:10.178+0000 7fbf5b898700 1 --2- 172.21.15.179:0/733739347 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fbf400a3c40 0x7fbf400a6100 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7fbf50009bf0 tx=0x7fbf50009260 comp rx=0 tx=0).stop 2024-04-21T08:34:10.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:34:10.178+0000 7fbf5b898700 1 -- 172.21.15.179:0/733739347 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf54103530 msgr2=0x7fbf54198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:34:10.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:34:10.178+0000 7fbf5b898700 1 --2- 172.21.15.179:0/733739347 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf54103530 0x7fbf54198c10 secure :-1 s=READY pgs=30 cs=0 l=1 rev1=1 crypto rx=0x7fbf4400e9c0 tx=0x7fbf4400ecd0 comp rx=0 tx=0).stop 2024-04-21T08:34:10.177 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:34:10.178+0000 7fbf5b898700 1 -- 172.21.15.179:0/733739347 shutdown_connections 2024-04-21T08:34:10.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:34:10.178+0000 7fbf5b898700 1 --2- 172.21.15.179:0/733739347 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fbf400a3c40 0x7fbf400a6100 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:34:10.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:34:10.178+0000 7fbf5b898700 1 --2- 172.21.15.179:0/733739347 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf54108790 0x7fbf541a0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:34:10.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:34:10.178+0000 7fbf5b898700 1 --2- 172.21.15.179:0/733739347 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf54103530 0x7fbf54198c10 unknown :-1 s=CLOSED pgs=30 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:34:10.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:34:10.178+0000 7fbf5b898700 1 --2- 172.21.15.179:0/733739347 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf54105e60 0x7fbf54199150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:34:10.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:34:10.178+0000 7fbf5b898700 1 -- 172.21.15.179:0/733739347 >> 172.21.15.179:0/733739347 conn(0x7fbf540fcf40 msgr2=0x7fbf54103f50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:34:10.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:34:10.178+0000 7fbf5b898700 1 -- 172.21.15.179:0/733739347 shutdown_connections 2024-04-21T08:34:10.178 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:34:10.178+0000 7fbf5b898700 1 -- 172.21.15.179:0/733739347 wait complete. 2024-04-21T08:34:10.481 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:10 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:10.481 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:10 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:10.481 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:10 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:10.481 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:10 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:34:10.481 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:10 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:10.481 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:10 smithi179 ceph-mon[31829]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:10.481 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:10 smithi179 ceph-mon[31829]: from='osd.8 [v2:172.21.15.179:6816/1770418957,v1:172.21.15.179:6817/1770418957]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-21T08:34:10.481 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:10 smithi179 ceph-mon[31829]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-21T08:34:10.481 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:10 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:10.481 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:10 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:34:10.481 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:10 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:10.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:10 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:10.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:10 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:10.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:10 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:10.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:10 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:34:10.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:10 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:10.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:10 smithi086 ceph-mon[27939]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:10.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:10 smithi086 ceph-mon[27939]: from='osd.8 [v2:172.21.15.179:6816/1770418957,v1:172.21.15.179:6817/1770418957]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-21T08:34:10.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:10 smithi086 ceph-mon[27939]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-21T08:34:10.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:10 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:10.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:10 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:34:10.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:10 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:10.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:10 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:10.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:10 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:10.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:10 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:10.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:10 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:34:10.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:10 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:10.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:10 smithi088 ceph-mon[31259]: pgmap v189: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:10.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:10 smithi088 ceph-mon[31259]: from='osd.8 [v2:172.21.15.179:6816/1770418957,v1:172.21.15.179:6817/1770418957]' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-21T08:34:10.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:10 smithi088 ceph-mon[31259]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]: dispatch 2024-04-21T08:34:10.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:10 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:10.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:10 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:34:10.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:10 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:10.828 DEBUG:teuthology.orchestra.run.smithi179:osd.8> sudo journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@osd.8.service 2024-04-21T08:34:10.830 INFO:tasks.cephadm:Deploying osd.9 on smithi086 with /dev/vg_nvme/lv_1... 2024-04-21T08:34:10.831 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-21T08:34:11.075 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:34:11.463 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:11 smithi086 ceph-mon[27939]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-21T08:34:11.463 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:11 smithi086 ceph-mon[27939]: osdmap e57: 9 total, 8 up, 9 in 2024-04-21T08:34:11.463 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:11 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:34:11.463 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:11 smithi086 ceph-mon[27939]: from='osd.8 [v2:172.21.15.179:6816/1770418957,v1:172.21.15.179:6817/1770418957]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-21T08:34:11.463 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:11 smithi086 ceph-mon[27939]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-21T08:34:11.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:11 smithi179 ceph-mon[31829]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-21T08:34:11.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:11 smithi179 ceph-mon[31829]: osdmap e57: 9 total, 8 up, 9 in 2024-04-21T08:34:11.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:11 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:34:11.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:11 smithi179 ceph-mon[31829]: from='osd.8 [v2:172.21.15.179:6816/1770418957,v1:172.21.15.179:6817/1770418957]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-21T08:34:11.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:11 smithi179 ceph-mon[31829]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-21T08:34:11.497 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:34:11 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:34:11.260+0000 7f2201566700 -1 osd.8 0 waiting for initial osdmap 2024-04-21T08:34:11.497 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:34:11 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:34:11.270+0000 7f21fd359700 -1 osd.8 58 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T08:34:11.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:11 smithi088 ceph-mon[31259]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["8"]}]': finished 2024-04-21T08:34:11.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:11 smithi088 ceph-mon[31259]: osdmap e57: 9 total, 8 up, 9 in 2024-04-21T08:34:11.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:11 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:34:11.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:11 smithi088 ceph-mon[31259]: from='osd.8 [v2:172.21.15.179:6816/1770418957,v1:172.21.15.179:6817/1770418957]' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-21T08:34:11.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:11 smithi088 ceph-mon[31259]: from='osd.8 ' entity='osd.8' cmd=[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-21T08:34:12.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:12 smithi086 ceph-mon[27939]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:12.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:12 smithi086 ceph-mon[27939]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-04-21T08:34:12.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:12 smithi086 ceph-mon[27939]: osdmap e58: 9 total, 8 up, 9 in 2024-04-21T08:34:12.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:12 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:34:12.468 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:12 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:34:12.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:12 smithi179 ceph-mon[31829]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:12.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:12 smithi179 ceph-mon[31829]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-04-21T08:34:12.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:12 smithi179 ceph-mon[31829]: osdmap e58: 9 total, 8 up, 9 in 2024-04-21T08:34:12.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:12 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:34:12.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:12 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:34:12.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:12 smithi088 ceph-mon[31259]: pgmap v191: 1 pgs: 1 active+clean; 577 KiB data, 237 MiB used, 715 GiB / 715 GiB avail 2024-04-21T08:34:12.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:12 smithi088 ceph-mon[31259]: from='osd.8 ' entity='osd.8' cmd='[{"prefix": "osd crush create-or-move", "id": 8, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-04-21T08:34:12.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:12 smithi088 ceph-mon[31259]: osdmap e58: 9 total, 8 up, 9 in 2024-04-21T08:34:12.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:12 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:34:12.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:12 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:34:13.173 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-21T08:34:13.173 INFO:teuthology.orchestra.run.smithi086.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-21T08:34:13.174 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10+0 records in 2024-04-21T08:34:13.174 INFO:teuthology.orchestra.run.smithi086.stderr:10+0 records out 2024-04-21T08:34:13.174 INFO:teuthology.orchestra.run.smithi086.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0129219 s, 811 MB/s 2024-04-21T08:34:13.174 INFO:teuthology.orchestra.run.smithi086.stderr:--> Zapping successful for: 2024-04-21T08:34:13.379 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:13 smithi179 ceph-mon[31829]: purged_snaps scrub starts 2024-04-21T08:34:13.379 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:13 smithi179 ceph-mon[31829]: purged_snaps scrub ok 2024-04-21T08:34:13.379 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:13 smithi179 ceph-mon[31829]: osd.8 [v2:172.21.15.179:6816/1770418957,v1:172.21.15.179:6817/1770418957] boot 2024-04-21T08:34:13.380 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:13 smithi179 ceph-mon[31829]: osdmap e59: 9 total, 9 up, 9 in 2024-04-21T08:34:13.380 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:13 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:34:13.478 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:13 smithi086 ceph-mon[27939]: purged_snaps scrub starts 2024-04-21T08:34:13.479 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:13 smithi086 ceph-mon[27939]: purged_snaps scrub ok 2024-04-21T08:34:13.479 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:13 smithi086 ceph-mon[27939]: osd.8 [v2:172.21.15.179:6816/1770418957,v1:172.21.15.179:6817/1770418957] boot 2024-04-21T08:34:13.479 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:13 smithi086 ceph-mon[27939]: osdmap e59: 9 total, 9 up, 9 in 2024-04-21T08:34:13.479 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:13 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:34:13.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:13 smithi088 ceph-mon[31259]: purged_snaps scrub starts 2024-04-21T08:34:13.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:13 smithi088 ceph-mon[31259]: purged_snaps scrub ok 2024-04-21T08:34:13.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:13 smithi088 ceph-mon[31259]: osd.8 [v2:172.21.15.179:6816/1770418957,v1:172.21.15.179:6817/1770418957] boot 2024-04-21T08:34:13.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:13 smithi088 ceph-mon[31259]: osdmap e59: 9 total, 9 up, 9 in 2024-04-21T08:34:13.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:13 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:34:13.870 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi086:vg_nvme/lv_1 2024-04-21T08:34:14.110 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:34:14.562 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:14 smithi086 ceph-mon[27939]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:14.562 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:14 smithi086 ceph-mon[27939]: osdmap e60: 9 total, 9 up, 9 in 2024-04-21T08:34:14.562 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:14 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:14.562 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:14 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:14.562 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:14 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:14.563 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:14 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:14.563 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:14 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:14.563 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:14 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:14.563 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:14 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:14.563 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:14 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:34:14.563 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:14 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:14.585 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:14 smithi179 ceph-mon[31829]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:14.585 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:14 smithi179 ceph-mon[31829]: osdmap e60: 9 total, 9 up, 9 in 2024-04-21T08:34:14.585 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:14 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:14.585 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:14 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:14.585 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:14 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:14.585 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:14 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:14.585 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:14 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:14.585 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:14 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:14.585 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:14 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:14.586 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:14 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:34:14.586 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:14 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:14.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:14 smithi088 ceph-mon[31259]: pgmap v194: 1 pgs: 1 active+clean; 577 KiB data, 264 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:14.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:14 smithi088 ceph-mon[31259]: osdmap e60: 9 total, 9 up, 9 in 2024-04-21T08:34:14.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:14 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:14.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:14 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:14.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:14 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:14.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:14 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:14.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:14 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:14.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:14 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:14.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:14 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:14.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:14 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:34:14.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:14 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:15.463 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:15 smithi088 ceph-mon[31259]: Detected new or changed devices on smithi179 2024-04-21T08:34:15.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:15 smithi088 ceph-mon[31259]: Adjusting osd_memory_target on smithi179 to 5696M 2024-04-21T08:34:15.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:15 smithi088 ceph-mon[31259]: osdmap e61: 9 total, 9 up, 9 in 2024-04-21T08:34:15.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:15 smithi179 ceph-mon[31829]: Detected new or changed devices on smithi179 2024-04-21T08:34:15.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:15 smithi179 ceph-mon[31829]: Adjusting osd_memory_target on smithi179 to 5696M 2024-04-21T08:34:15.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:15 smithi179 ceph-mon[31829]: osdmap e61: 9 total, 9 up, 9 in 2024-04-21T08:34:15.578 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:15 smithi086 ceph-mon[27939]: Detected new or changed devices on smithi179 2024-04-21T08:34:15.578 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:15 smithi086 ceph-mon[27939]: Adjusting osd_memory_target on smithi179 to 5696M 2024-04-21T08:34:15.578 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:15 smithi086 ceph-mon[27939]: osdmap e61: 9 total, 9 up, 9 in 2024-04-21T08:34:15.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.791+0000 7f8fbeb2c700 1 -- 172.21.15.86:0/3087606708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fb8103540 msgr2=0x7f8fb8109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:34:15.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.791+0000 7f8fbeb2c700 1 --2- 172.21.15.86:0/3087606708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fb8103540 0x7f8fb8109df0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f8fb000b0d0 tx=0x7f8fb000b3e0 comp rx=0 tx=0).stop 2024-04-21T08:34:15.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.791+0000 7f8fbeb2c700 1 -- 172.21.15.86:0/3087606708 shutdown_connections 2024-04-21T08:34:15.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.791+0000 7f8fbeb2c700 1 --2- 172.21.15.86:0/3087606708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fb8103540 0x7f8fb8109df0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:34:15.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.791+0000 7f8fbeb2c700 1 --2- 172.21.15.86:0/3087606708 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8fb81019a0 0x7f8fb8101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:34:15.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.791+0000 7f8fbeb2c700 1 --2- 172.21.15.86:0/3087606708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fb8102ba0 0x7f8fb8103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:34:15.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.791+0000 7f8fbeb2c700 1 -- 172.21.15.86:0/3087606708 >> 172.21.15.86:0/3087606708 conn(0x7f8fb80fcf40 msgr2=0x7f8fb80ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:34:15.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.791+0000 7f8fbeb2c700 1 -- 172.21.15.86:0/3087606708 shutdown_connections 2024-04-21T08:34:15.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.791+0000 7f8fbeb2c700 1 -- 172.21.15.86:0/3087606708 wait complete. 2024-04-21T08:34:15.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.791+0000 7f8fbeb2c700 1 Processor -- start 2024-04-21T08:34:15.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.791+0000 7f8fbeb2c700 1 -- start start 2024-04-21T08:34:15.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.792+0000 7f8fbeb2c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fb81019a0 0x7f8fb806b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:34:15.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.792+0000 7f8fbeb2c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fb8102ba0 0x7f8fb819d930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:34:15.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.792+0000 7f8fbeb2c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8fb8103540 0x7f8fb819ffc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:34:15.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.792+0000 7f8fbeb2c700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb810d490 con 0x7f8fb81019a0 2024-04-21T08:34:15.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.792+0000 7f8fbeb2c700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb810d310 con 0x7f8fb8103540 2024-04-21T08:34:15.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.792+0000 7f8fbeb2c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8fb810d610 con 0x7f8fb8102ba0 2024-04-21T08:34:15.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.792+0000 7f8fbc8c8700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fb81019a0 0x7f8fb806b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:34:15.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.792+0000 7f8fbc8c8700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fb81019a0 0x7f8fb806b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42192/0 (socket says 172.21.15.86:42192) 2024-04-21T08:34:15.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.792+0000 7f8fbc8c8700 1 -- 172.21.15.86:0/3669555373 learned_addr learned my addr 172.21.15.86:0/3669555373 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:34:15.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.792+0000 7f8fb7fff700 1 --2- 172.21.15.86:0/3669555373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fb8102ba0 0x7f8fb819d930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:34:15.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.792+0000 7f8fbd0c9700 1 --2- 172.21.15.86:0/3669555373 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8fb8103540 0x7f8fb819ffc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:34:15.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.792+0000 7f8fbc8c8700 1 -- 172.21.15.86:0/3669555373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fb8102ba0 msgr2=0x7f8fb819d930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:34:15.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.792+0000 7f8fbc8c8700 1 --2- 172.21.15.86:0/3669555373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fb8102ba0 0x7f8fb819d930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:34:15.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.792+0000 7f8fbc8c8700 1 -- 172.21.15.86:0/3669555373 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8fb8103540 msgr2=0x7f8fb819ffc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:34:15.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.792+0000 7f8fbc8c8700 1 --2- 172.21.15.86:0/3669555373 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8fb8103540 0x7f8fb819ffc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:34:15.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.792+0000 7f8fbc8c8700 1 -- 172.21.15.86:0/3669555373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8fb81a0890 con 0x7f8fb81019a0 2024-04-21T08:34:15.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.792+0000 7f8fb7fff700 1 --2- 172.21.15.86:0/3669555373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fb8102ba0 0x7f8fb819d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:34:15.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.792+0000 7f8fbc8c8700 1 --2- 172.21.15.86:0/3669555373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fb81019a0 0x7f8fb806b2e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8fac00d980 tx=0x7f8fac00dd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:34:15.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.792+0000 7f8fb5ffb700 1 -- 172.21.15.86:0/3669555373 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac016070 con 0x7f8fb81019a0 2024-04-21T08:34:15.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.792+0000 7f8fb5ffb700 1 -- 172.21.15.86:0/3669555373 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8fac00bb60 con 0x7f8fb81019a0 2024-04-21T08:34:15.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.792+0000 7f8fbeb2c700 1 -- 172.21.15.86:0/3669555373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8fb81a0b80 con 0x7f8fb81019a0 2024-04-21T08:34:15.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.792+0000 7f8fb5ffb700 1 -- 172.21.15.86:0/3669555373 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8fac00f3c0 con 0x7f8fb81019a0 2024-04-21T08:34:15.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.793+0000 7f8fbeb2c700 1 -- 172.21.15.86:0/3669555373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8fb81a1110 con 0x7f8fb81019a0 2024-04-21T08:34:15.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.793+0000 7f8fbeb2c700 1 -- 172.21.15.86:0/3669555373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8fb8069730 con 0x7f8fb81019a0 2024-04-21T08:34:15.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.795+0000 7f8fb5ffb700 1 -- 172.21.15.86:0/3669555373 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8fac019070 con 0x7f8fb81019a0 2024-04-21T08:34:15.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.795+0000 7f8fb5ffb700 1 --2- 172.21.15.86:0/3669555373 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f8fa80a3cc0 0x7f8fa80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:34:15.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.795+0000 7f8fb7fff700 1 --2- 172.21.15.86:0/3669555373 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f8fa80a3cc0 0x7f8fa80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:34:15.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.795+0000 7f8fb5ffb700 1 -- 172.21.15.86:0/3669555373 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(62..62 src has 1..62) ==== 5829+0+0 (secure 0 0 0) 0x7f8fac0cf730 con 0x7f8fb81019a0 2024-04-21T08:34:15.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.795+0000 7f8fb7fff700 1 --2- 172.21.15.86:0/3669555373 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f8fa80a3cc0 0x7f8fa80a6180 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4005e10 tx=0x7f8fa4005d80 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:34:15.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:15.800+0000 7f8fb5ffb700 1 -- 172.21.15.86:0/3669555373 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8fac020030 con 0x7f8fb81019a0 2024-04-21T08:34:16.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:16.007+0000 7f8fbeb2c700 1 -- 172.21.15.86:0/3669555373 --> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f8fb8062e80 con 0x7f8fa80a3cc0 2024-04-21T08:34:16.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:16 smithi086 ceph-mon[27939]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:16.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:16 smithi086 ceph-mon[27939]: osdmap e62: 9 total, 9 up, 9 in 2024-04-21T08:34:16.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:16 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:34:16.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:16 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:34:16.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:16 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:16.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:16 smithi088 ceph-mon[31259]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:16.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:16 smithi088 ceph-mon[31259]: osdmap e62: 9 total, 9 up, 9 in 2024-04-21T08:34:16.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:16 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:34:16.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:16 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:34:16.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:16 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:16.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:16 smithi179 ceph-mon[31829]: pgmap v197: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:16.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:16 smithi179 ceph-mon[31829]: osdmap e62: 9 total, 9 up, 9 in 2024-04-21T08:34:16.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:16 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:34:16.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:16 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:34:16.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:16 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:17.568 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:17 smithi086 ceph-mon[27939]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:34:17.568 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:17 smithi086 ceph-mon[27939]: osdmap e63: 9 total, 9 up, 9 in 2024-04-21T08:34:17.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:17 smithi088 ceph-mon[31259]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:34:17.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:17 smithi088 ceph-mon[31259]: osdmap e63: 9 total, 9 up, 9 in 2024-04-21T08:34:17.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:17 smithi179 ceph-mon[31829]: from='client.14436 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi086:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:34:17.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:17 smithi179 ceph-mon[31829]: osdmap e63: 9 total, 9 up, 9 in 2024-04-21T08:34:18.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:18 smithi086 ceph-mon[27939]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:18.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:18 smithi088 ceph-mon[31259]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:18.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:18 smithi179 ceph-mon[31829]: pgmap v200: 1 pgs: 1 remapped+peering; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:19.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:19 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1563478782' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5841bb36-ed6b-45d8-b857-eeab139c44e6"}]: dispatch 2024-04-21T08:34:19.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:19 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1563478782' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5841bb36-ed6b-45d8-b857-eeab139c44e6"}]': finished 2024-04-21T08:34:19.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:19 smithi086 ceph-mon[27939]: osdmap e64: 10 total, 9 up, 10 in 2024-04-21T08:34:19.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:19 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:34:19.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:19 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1563478782' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5841bb36-ed6b-45d8-b857-eeab139c44e6"}]: dispatch 2024-04-21T08:34:19.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:19 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1563478782' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5841bb36-ed6b-45d8-b857-eeab139c44e6"}]': finished 2024-04-21T08:34:19.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:19 smithi088 ceph-mon[31259]: osdmap e64: 10 total, 9 up, 10 in 2024-04-21T08:34:19.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:19 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:34:19.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:19 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1563478782' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "5841bb36-ed6b-45d8-b857-eeab139c44e6"}]: dispatch 2024-04-21T08:34:19.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:19 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1563478782' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "5841bb36-ed6b-45d8-b857-eeab139c44e6"}]': finished 2024-04-21T08:34:19.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:19 smithi179 ceph-mon[31829]: osdmap e64: 10 total, 9 up, 10 in 2024-04-21T08:34:19.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:19 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:34:20.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:20 smithi086 ceph-mon[27939]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-04-21T08:34:20.672 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:20 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3258935435' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:34:20.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:20 smithi088 ceph-mon[31259]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-04-21T08:34:20.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:20 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3258935435' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:34:20.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:20 smithi179 ceph-mon[31829]: pgmap v202: 1 pgs: 1 active+clean; 577 KiB data, 270 MiB used, 804 GiB / 805 GiB avail; 116 KiB/s, 0 objects/s recovering 2024-04-21T08:34:20.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:20 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3258935435' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:34:22.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:22 smithi086 ceph-mon[27939]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-21T08:34:22.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:22 smithi088 ceph-mon[31259]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-21T08:34:22.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:22 smithi179 ceph-mon[31829]: pgmap v203: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-21T08:34:24.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:24 smithi086 ceph-mon[27939]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-21T08:34:24.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:24 smithi088 ceph-mon[31259]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-21T08:34:24.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:24 smithi179 ceph-mon[31829]: pgmap v204: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 72 KiB/s, 0 objects/s recovering 2024-04-21T08:34:26.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:26 smithi086 ceph-mon[27939]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-21T08:34:26.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:26 smithi088 ceph-mon[31259]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-21T08:34:26.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:26 smithi179 ceph-mon[31829]: pgmap v205: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 64 KiB/s, 0 objects/s recovering 2024-04-21T08:34:27.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:27 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-21T08:34:27.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:27 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:27.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:27 smithi086 ceph-mon[27939]: Deploying daemon osd.9 on smithi086 2024-04-21T08:34:27.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:27 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-21T08:34:27.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:27 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:27.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:27 smithi088 ceph-mon[31259]: Deploying daemon osd.9 on smithi086 2024-04-21T08:34:27.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:27 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.9"}]: dispatch 2024-04-21T08:34:27.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:27 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:27.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:27 smithi179 ceph-mon[31829]: Deploying daemon osd.9 on smithi086 2024-04-21T08:34:28.588 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:28 smithi086 ceph-mon[27939]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-21T08:34:28.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:28 smithi088 ceph-mon[31259]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-21T08:34:28.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:28 smithi179 ceph-mon[31829]: pgmap v206: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 58 KiB/s, 0 objects/s recovering 2024-04-21T08:34:30.507 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:30 smithi086 ceph-mon[27939]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-04-21T08:34:30.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:30 smithi088 ceph-mon[31259]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-04-21T08:34:30.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:30 smithi179 ceph-mon[31829]: pgmap v207: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail; 55 KiB/s, 0 objects/s recovering 2024-04-21T08:34:32.571 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:32 smithi179 ceph-mon[31829]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:32.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:32 smithi086 ceph-mon[27939]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:32.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:32 smithi088 ceph-mon[31259]: pgmap v208: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:33.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:33 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:33.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:33 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:33.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:33 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:34:33.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:33 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:33.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:33 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:33.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:33 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:34:33.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:33 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:33.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:33 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:33.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:33 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:34:34.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:34 smithi088 ceph-mon[31259]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:34.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:34 smithi179 ceph-mon[31829]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:34.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:34 smithi086 ceph-mon[27939]: pgmap v209: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:36.442 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:36 smithi086 ceph-mon[27939]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:36.443 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:36 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:36.443 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:36 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:36.443 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:36 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:36.443 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:36 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:34:36.443 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:36 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:36.443 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:36 smithi086 ceph-mon[27939]: from='osd.9 [v2:172.21.15.86:6826/1108500274,v1:172.21.15.86:6827/1108500274]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-21T08:34:36.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:36.629+0000 7f8fb5ffb700 1 -- 172.21.15.86:0/3669555373 <== mgr.14152 v2:172.21.15.86:6800/1364156965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+37 (secure 0 0 0) 0x7f8fb8062e80 con 0x7f8fa80a3cc0 2024-04-21T08:34:36.629 INFO:teuthology.orchestra.run.smithi086.stdout:Created osd(s) 9 on host 'smithi086' 2024-04-21T08:34:36.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:36.632+0000 7f8fbeb2c700 1 -- 172.21.15.86:0/3669555373 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f8fa80a3cc0 msgr2=0x7f8fa80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:34:36.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:36.632+0000 7f8fbeb2c700 1 --2- 172.21.15.86:0/3669555373 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f8fa80a3cc0 0x7f8fa80a6180 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f8fa4005e10 tx=0x7f8fa4005d80 comp rx=0 tx=0).stop 2024-04-21T08:34:36.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:36.632+0000 7f8fbeb2c700 1 -- 172.21.15.86:0/3669555373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fb81019a0 msgr2=0x7f8fb806b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:34:36.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:36.632+0000 7f8fbeb2c700 1 --2- 172.21.15.86:0/3669555373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fb81019a0 0x7f8fb806b2e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f8fac00d980 tx=0x7f8fac00dd40 comp rx=0 tx=0).stop 2024-04-21T08:34:36.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:36.632+0000 7f8fbeb2c700 1 -- 172.21.15.86:0/3669555373 shutdown_connections 2024-04-21T08:34:36.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:36.632+0000 7f8fbeb2c700 1 --2- 172.21.15.86:0/3669555373 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f8fa80a3cc0 0x7f8fa80a6180 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:34:36.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:36.632+0000 7f8fbeb2c700 1 --2- 172.21.15.86:0/3669555373 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8fb8103540 0x7f8fb819ffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:34:36.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:36.632+0000 7f8fbeb2c700 1 --2- 172.21.15.86:0/3669555373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8fb81019a0 0x7f8fb806b2e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:34:36.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:36.632+0000 7f8fbeb2c700 1 --2- 172.21.15.86:0/3669555373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8fb8102ba0 0x7f8fb819d930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:34:36.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:36.632+0000 7f8fbeb2c700 1 -- 172.21.15.86:0/3669555373 >> 172.21.15.86:0/3669555373 conn(0x7f8fb80fcf40 msgr2=0x7f8fb8105fc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:34:36.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:36.632+0000 7f8fbeb2c700 1 -- 172.21.15.86:0/3669555373 shutdown_connections 2024-04-21T08:34:36.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:34:36.632+0000 7f8fbeb2c700 1 -- 172.21.15.86:0/3669555373 wait complete. 2024-04-21T08:34:36.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:36 smithi088 ceph-mon[31259]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:36.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:36 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:36.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:36 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:36.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:36 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:36.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:36 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:34:36.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:36 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:36.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:36 smithi088 ceph-mon[31259]: from='osd.9 [v2:172.21.15.86:6826/1108500274,v1:172.21.15.86:6827/1108500274]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-21T08:34:36.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:36 smithi179 ceph-mon[31829]: pgmap v210: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:36.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:36 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:36.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:36 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:36.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:36 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:36.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:36 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:34:36.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:36 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:36.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:36 smithi179 ceph-mon[31829]: from='osd.9 [v2:172.21.15.86:6826/1108500274,v1:172.21.15.86:6827/1108500274]' entity='osd.9' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]: dispatch 2024-04-21T08:34:37.400 DEBUG:teuthology.orchestra.run.smithi086:osd.9> sudo journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@osd.9.service 2024-04-21T08:34:37.402 INFO:tasks.cephadm:Deploying osd.10 on smithi088 with /dev/vg_nvme/lv_1... 2024-04-21T08:34:37.402 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-21T08:34:37.650 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T08:34:37.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:37 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:37.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:37 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:37.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:37 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:34:37.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:37 smithi088 ceph-mon[31259]: from='osd.9 [v2:172.21.15.86:6826/1108500274,v1:172.21.15.86:6827/1108500274]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-21T08:34:37.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:37 smithi088 ceph-mon[31259]: osdmap e65: 10 total, 9 up, 10 in 2024-04-21T08:34:37.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:37 smithi088 ceph-mon[31259]: from='osd.9 [v2:172.21.15.86:6826/1108500274,v1:172.21.15.86:6827/1108500274]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-21T08:34:37.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:37 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:34:37.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:37 smithi088 ceph-mon[31259]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:37.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:37 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:37.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:37 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:37.717 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:37 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:34:37.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:37 smithi086 ceph-mon[27939]: from='osd.9 [v2:172.21.15.86:6826/1108500274,v1:172.21.15.86:6827/1108500274]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-21T08:34:37.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:37 smithi086 ceph-mon[27939]: osdmap e65: 10 total, 9 up, 10 in 2024-04-21T08:34:37.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:37 smithi086 ceph-mon[27939]: from='osd.9 [v2:172.21.15.86:6826/1108500274,v1:172.21.15.86:6827/1108500274]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-21T08:34:37.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:37 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:34:37.718 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:37 smithi086 ceph-mon[27939]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:37.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:37 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:37.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:37 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:37.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:37 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:34:37.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:37 smithi179 ceph-mon[31829]: from='osd.9 [v2:172.21.15.86:6826/1108500274,v1:172.21.15.86:6827/1108500274]' entity='osd.9' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["9"]}]': finished 2024-04-21T08:34:37.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:37 smithi179 ceph-mon[31829]: osdmap e65: 10 total, 9 up, 10 in 2024-04-21T08:34:37.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:37 smithi179 ceph-mon[31829]: from='osd.9 [v2:172.21.15.86:6826/1108500274,v1:172.21.15.86:6827/1108500274]' entity='osd.9' cmd=[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]: dispatch 2024-04-21T08:34:37.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:37 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:34:37.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:37 smithi179 ceph-mon[31829]: pgmap v212: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:38.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:34:38 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:34:38.371+0000 7f3574651700 -1 osd.9 0 waiting for initial osdmap 2024-04-21T08:34:38.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:34:38 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:34:38.382+0000 7f356c43c700 -1 osd.9 66 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T08:34:38.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:38 smithi088 ceph-mon[31259]: from='osd.9 [v2:172.21.15.86:6826/1108500274,v1:172.21.15.86:6827/1108500274]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-21T08:34:38.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:38 smithi088 ceph-mon[31259]: osdmap e66: 10 total, 9 up, 10 in 2024-04-21T08:34:38.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:38 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:34:38.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:38 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:34:38.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:38 smithi088 ceph-mon[31259]: from='osd.9 [v2:172.21.15.86:6826/1108500274,v1:172.21.15.86:6827/1108500274]' entity='osd.9' 2024-04-21T08:34:38.985 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:38 smithi086 ceph-mon[27939]: from='osd.9 [v2:172.21.15.86:6826/1108500274,v1:172.21.15.86:6827/1108500274]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-21T08:34:38.985 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:38 smithi086 ceph-mon[27939]: osdmap e66: 10 total, 9 up, 10 in 2024-04-21T08:34:38.985 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:38 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:34:38.985 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:38 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:34:38.985 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:38 smithi086 ceph-mon[27939]: from='osd.9 [v2:172.21.15.86:6826/1108500274,v1:172.21.15.86:6827/1108500274]' entity='osd.9' 2024-04-21T08:34:38.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:38 smithi179 ceph-mon[31829]: from='osd.9 [v2:172.21.15.86:6826/1108500274,v1:172.21.15.86:6827/1108500274]' entity='osd.9' cmd='[{"prefix": "osd crush create-or-move", "id": 9, "weight":0.0873, "args": ["host=smithi086", "root=default"]}]': finished 2024-04-21T08:34:38.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:38 smithi179 ceph-mon[31829]: osdmap e66: 10 total, 9 up, 10 in 2024-04-21T08:34:38.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:38 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:34:38.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:38 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:34:38.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:38 smithi179 ceph-mon[31829]: from='osd.9 [v2:172.21.15.86:6826/1108500274,v1:172.21.15.86:6827/1108500274]' entity='osd.9' 2024-04-21T08:34:39.573 INFO:teuthology.orchestra.run.smithi088.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-21T08:34:39.573 INFO:teuthology.orchestra.run.smithi088.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-21T08:34:39.573 INFO:teuthology.orchestra.run.smithi088.stderr: stderr: 10+0 records in 2024-04-21T08:34:39.573 INFO:teuthology.orchestra.run.smithi088.stderr:10+0 records out 2024-04-21T08:34:39.573 INFO:teuthology.orchestra.run.smithi088.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.0110732 s, 947 MB/s 2024-04-21T08:34:39.573 INFO:teuthology.orchestra.run.smithi088.stderr:--> Zapping successful for: 2024-04-21T08:34:39.896 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:39 smithi088 ceph-mon[31259]: purged_snaps scrub starts 2024-04-21T08:34:39.896 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:39 smithi088 ceph-mon[31259]: purged_snaps scrub ok 2024-04-21T08:34:39.896 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:39 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:34:39.896 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:39 smithi088 ceph-mon[31259]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:39.896 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:39 smithi088 ceph-mon[31259]: osd.9 [v2:172.21.15.86:6826/1108500274,v1:172.21.15.86:6827/1108500274] boot 2024-04-21T08:34:39.896 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:39 smithi088 ceph-mon[31259]: osdmap e67: 10 total, 10 up, 10 in 2024-04-21T08:34:39.896 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:39 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:34:39.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:39 smithi179 ceph-mon[31829]: purged_snaps scrub starts 2024-04-21T08:34:39.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:39 smithi179 ceph-mon[31829]: purged_snaps scrub ok 2024-04-21T08:34:39.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:39 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:34:39.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:39 smithi179 ceph-mon[31829]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:39.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:39 smithi179 ceph-mon[31829]: osd.9 [v2:172.21.15.86:6826/1108500274,v1:172.21.15.86:6827/1108500274] boot 2024-04-21T08:34:39.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:39 smithi179 ceph-mon[31829]: osdmap e67: 10 total, 10 up, 10 in 2024-04-21T08:34:39.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:39 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:34:40.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:39 smithi086 ceph-mon[27939]: purged_snaps scrub starts 2024-04-21T08:34:40.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:39 smithi086 ceph-mon[27939]: purged_snaps scrub ok 2024-04-21T08:34:40.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:39 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:34:40.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:39 smithi086 ceph-mon[27939]: pgmap v214: 1 pgs: 1 active+clean; 577 KiB data, 269 MiB used, 804 GiB / 805 GiB avail 2024-04-21T08:34:40.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:39 smithi086 ceph-mon[27939]: osd.9 [v2:172.21.15.86:6826/1108500274,v1:172.21.15.86:6827/1108500274] boot 2024-04-21T08:34:40.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:39 smithi086 ceph-mon[27939]: osdmap e67: 10 total, 10 up, 10 in 2024-04-21T08:34:40.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:39 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:34:40.210 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi088:vg_nvme/lv_1 2024-04-21T08:34:40.452 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T08:34:41.860 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:41 smithi088 ceph-mon[31259]: Detected new or changed devices on smithi086 2024-04-21T08:34:41.860 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:41 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:41.860 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:41 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:41.860 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:41 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:41.860 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:41 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:41.860 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:41 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:41.860 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:41 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:41.860 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:41 smithi088 ceph-mon[31259]: Adjusting osd_memory_target on smithi086 to 4272M 2024-04-21T08:34:41.860 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:41 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:41.860 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:41 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:41.860 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:41 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:34:41.861 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:41 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:41.861 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:41 smithi088 ceph-mon[31259]: osdmap e68: 10 total, 10 up, 10 in 2024-04-21T08:34:41.861 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:41 smithi088 ceph-mon[31259]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:41.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:41 smithi179 ceph-mon[31829]: Detected new or changed devices on smithi086 2024-04-21T08:34:41.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:41 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:41.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:41 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:41.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:41 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:41.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:41 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:41.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:41 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:41.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:41 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:41.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:41 smithi179 ceph-mon[31829]: Adjusting osd_memory_target on smithi086 to 4272M 2024-04-21T08:34:41.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:41 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:41.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:41 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:41.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:41 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:34:41.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:41 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:41.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:41 smithi179 ceph-mon[31829]: osdmap e68: 10 total, 10 up, 10 in 2024-04-21T08:34:41.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:41 smithi179 ceph-mon[31829]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:42.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.087+0000 7f3151d82700 1 -- 172.21.15.88:0/1242406831 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f314c103cb0 msgr2=0x7f314c104110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:34:42.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.087+0000 7f3151d82700 1 --2- 172.21.15.88:0/1242406831 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f314c103cb0 0x7f314c104110 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7f313c0099c0 tx=0x7f313c009cd0 comp rx=0 tx=0).stop 2024-04-21T08:34:42.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.087+0000 7f3151d82700 1 -- 172.21.15.88:0/1242406831 shutdown_connections 2024-04-21T08:34:42.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.087+0000 7f3151d82700 1 --2- 172.21.15.88:0/1242406831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f314c104650 0x7f314c10af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:34:42.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.087+0000 7f3151d82700 1 --2- 172.21.15.88:0/1242406831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f314c102ab0 0x7f314c102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:34:42.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.087+0000 7f3151d82700 1 --2- 172.21.15.88:0/1242406831 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f314c103cb0 0x7f314c104110 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:34:42.089 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.087+0000 7f3151d82700 1 -- 172.21.15.88:0/1242406831 >> 172.21.15.88:0/1242406831 conn(0x7f314c0fe010 msgr2=0x7f314c100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:34:42.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.087+0000 7f3151d82700 1 -- 172.21.15.88:0/1242406831 shutdown_connections 2024-04-21T08:34:42.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.087+0000 7f3151d82700 1 -- 172.21.15.88:0/1242406831 wait complete. 2024-04-21T08:34:42.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.088+0000 7f3151d82700 1 Processor -- start 2024-04-21T08:34:42.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.088+0000 7f3151d82700 1 -- start start 2024-04-21T08:34:42.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.088+0000 7f3151d82700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f314c102ab0 0x7f314c0767e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:34:42.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.088+0000 7f3151d82700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f314c103cb0 0x7f314c074e30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:34:42.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.088+0000 7f3151d82700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f314c104650 0x7f314c075370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:34:42.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.088+0000 7f3151d82700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f314c10e5a0 con 0x7f314c103cb0 2024-04-21T08:34:42.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.088+0000 7f3151d82700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f314c10e420 con 0x7f314c102ab0 2024-04-21T08:34:42.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.088+0000 7f3151d82700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f314c10e720 con 0x7f314c104650 2024-04-21T08:34:42.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.088+0000 7f314b7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f314c102ab0 0x7f314c0767e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:34:42.090 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.088+0000 7f314b7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f314c102ab0 0x7f314c0767e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.88:40222/0 (socket says 172.21.15.88:40222) 2024-04-21T08:34:42.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.088+0000 7f314b7fe700 1 -- 172.21.15.88:0/599173660 learned_addr learned my addr 172.21.15.88:0/599173660 (peer_addr_for_me v2:172.21.15.88:0/0) 2024-04-21T08:34:42.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.088+0000 7f314affd700 1 --2- 172.21.15.88:0/599173660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f314c103cb0 0x7f314c074e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:34:42.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.088+0000 7f314bfff700 1 --2- 172.21.15.88:0/599173660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f314c104650 0x7f314c075370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:34:42.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.089+0000 7f314b7fe700 1 -- 172.21.15.88:0/599173660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f314c104650 msgr2=0x7f314c075370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:34:42.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.089+0000 7f314b7fe700 1 --2- 172.21.15.88:0/599173660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f314c104650 0x7f314c075370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:34:42.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.089+0000 7f314b7fe700 1 -- 172.21.15.88:0/599173660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f314c103cb0 msgr2=0x7f314c074e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:34:42.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.089+0000 7f314b7fe700 1 --2- 172.21.15.88:0/599173660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f314c103cb0 0x7f314c074e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:34:42.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.089+0000 7f314b7fe700 1 -- 172.21.15.88:0/599173660 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f314c075c20 con 0x7f314c102ab0 2024-04-21T08:34:42.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.089+0000 7f314bfff700 1 --2- 172.21.15.88:0/599173660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f314c104650 0x7f314c075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:34:42.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.089+0000 7f314b7fe700 1 --2- 172.21.15.88:0/599173660 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f314c102ab0 0x7f314c0767e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f313400d980 tx=0x7f313400dc90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:34:42.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.089+0000 7f3148ff9700 1 -- 172.21.15.88:0/599173660 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3134017030 con 0x7f314c102ab0 2024-04-21T08:34:42.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.089+0000 7f3148ff9700 1 -- 172.21.15.88:0/599173660 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31340044d0 con 0x7f314c102ab0 2024-04-21T08:34:42.091 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.089+0000 7f3148ff9700 1 -- 172.21.15.88:0/599173660 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31340039e0 con 0x7f314c102ab0 2024-04-21T08:34:42.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.089+0000 7f3151d82700 1 -- 172.21.15.88:0/599173660 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f314c1a2d80 con 0x7f314c102ab0 2024-04-21T08:34:42.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.089+0000 7f3151d82700 1 -- 172.21.15.88:0/599173660 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f314c1a3200 con 0x7f314c102ab0 2024-04-21T08:34:42.092 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.090+0000 7f3151d82700 1 -- 172.21.15.88:0/599173660 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f314c069730 con 0x7f314c102ab0 2024-04-21T08:34:42.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.091+0000 7f3148ff9700 1 -- 172.21.15.88:0/599173660 <== mon.2 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f313401a070 con 0x7f314c102ab0 2024-04-21T08:34:42.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.091+0000 7f3148ff9700 1 --2- 172.21.15.88:0/599173660 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f31380a3cc0 0x7f31380a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:34:42.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.092+0000 7f3148ff9700 1 -- 172.21.15.88:0/599173660 <== mon.2 v2:172.21.15.88:3300/0 5 ==== osd_map(68..68 src has 1..68) ==== 6261+0+0 (secure 0 0 0) 0x7f31340cf4c0 con 0x7f314c102ab0 2024-04-21T08:34:42.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.092+0000 7f314affd700 1 --2- 172.21.15.88:0/599173660 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f31380a3cc0 0x7f31380a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:34:42.093 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.092+0000 7f314affd700 1 --2- 172.21.15.88:0/599173660 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f31380a3cc0 0x7f31380a6180 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f313c009990 tx=0x7f313c00b870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:34:42.096 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.096+0000 7f3148ff9700 1 -- 172.21.15.88:0/599173660 <== mon.2 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f313409c6a0 con 0x7f314c102ab0 2024-04-21T08:34:42.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:41 smithi086 ceph-mon[27939]: Detected new or changed devices on smithi086 2024-04-21T08:34:42.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:41 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:42.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:41 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:42.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:41 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:42.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:41 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:42.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:41 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:42.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:41 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:34:42.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:41 smithi086 ceph-mon[27939]: Adjusting osd_memory_target on smithi086 to 4272M 2024-04-21T08:34:42.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:41 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:42.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:41 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:42.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:41 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:34:42.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:41 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:42.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:41 smithi086 ceph-mon[27939]: osdmap e68: 10 total, 10 up, 10 in 2024-04-21T08:34:42.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:41 smithi086 ceph-mon[27939]: pgmap v217: 1 pgs: 1 active+clean; 577 KiB data, 300 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:42.303 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:34:42.303+0000 7f3151d82700 1 -- 172.21.15.88:0/599173660 --> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7f314c062e80 con 0x7f31380a3cc0 2024-04-21T08:34:42.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:42 smithi088 ceph-mon[31259]: from='client.24314 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:34:42.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:42 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:34:42.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:42 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:34:42.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:42 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:42.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:42 smithi179 ceph-mon[31829]: from='client.24314 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:34:42.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:42 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:34:42.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:42 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:34:42.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:42 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:43.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:42 smithi086 ceph-mon[27939]: from='client.24314 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi088:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:34:43.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:42 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:34:43.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:42 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:34:43.017 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:42 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:43.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:43 smithi088 ceph-mon[31259]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:43.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:43 smithi179 ceph-mon[31829]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:44.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:43 smithi086 ceph-mon[27939]: pgmap v218: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:45.337 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:45 smithi088 ceph-mon[31259]: from='client.? 172.21.15.88:0/115860797' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c"}]: dispatch 2024-04-21T08:34:45.337 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:45 smithi088 ceph-mon[31259]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c"}]: dispatch 2024-04-21T08:34:45.338 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:45 smithi088 ceph-mon[31259]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c"}]': finished 2024-04-21T08:34:45.338 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:45 smithi088 ceph-mon[31259]: osdmap e69: 11 total, 10 up, 11 in 2024-04-21T08:34:45.338 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:45 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:34:45.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:45 smithi086 ceph-mon[27939]: from='client.? 172.21.15.88:0/115860797' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c"}]: dispatch 2024-04-21T08:34:45.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:45 smithi086 ceph-mon[27939]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c"}]: dispatch 2024-04-21T08:34:45.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:45 smithi086 ceph-mon[27939]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c"}]': finished 2024-04-21T08:34:45.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:45 smithi086 ceph-mon[27939]: osdmap e69: 11 total, 10 up, 11 in 2024-04-21T08:34:45.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:45 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:34:45.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:45 smithi179 ceph-mon[31829]: from='client.? 172.21.15.88:0/115860797' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c"}]: dispatch 2024-04-21T08:34:45.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:45 smithi179 ceph-mon[31829]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c"}]: dispatch 2024-04-21T08:34:45.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:45 smithi179 ceph-mon[31829]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c"}]': finished 2024-04-21T08:34:45.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:45 smithi179 ceph-mon[31829]: osdmap e69: 11 total, 10 up, 11 in 2024-04-21T08:34:45.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:45 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:34:46.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:46 smithi086 ceph-mon[27939]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:46.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:46 smithi086 ceph-mon[27939]: from='client.? 172.21.15.88:0/4279865051' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:34:46.463 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:46 smithi088 ceph-mon[31259]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:46.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:46 smithi088 ceph-mon[31259]: from='client.? 172.21.15.88:0/4279865051' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:34:46.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:46 smithi179 ceph-mon[31829]: pgmap v220: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:46.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:46 smithi179 ceph-mon[31829]: from='client.? 172.21.15.88:0/4279865051' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:34:48.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:48 smithi086 ceph-mon[27939]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:48.617 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:48 smithi088 ceph-mon[31259]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:48.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:48 smithi179 ceph-mon[31829]: pgmap v221: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:50.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:50 smithi086 ceph-mon[27939]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:50.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:50 smithi088 ceph-mon[31259]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:50.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:50 smithi179 ceph-mon[31829]: pgmap v222: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:52.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:52 smithi086 ceph-mon[27939]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:52.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:52 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-21T08:34:52.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:52 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:52.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:52 smithi088 ceph-mon[31259]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:52.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:52 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-21T08:34:52.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:52 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:52.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:52 smithi179 ceph-mon[31829]: pgmap v223: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:52.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:52 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.10"}]: dispatch 2024-04-21T08:34:52.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:52 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:34:53.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:53 smithi088 ceph-mon[31259]: Deploying daemon osd.10 on smithi088 2024-04-21T08:34:53.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:53 smithi086 ceph-mon[27939]: Deploying daemon osd.10 on smithi088 2024-04-21T08:34:53.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:53 smithi179 ceph-mon[31829]: Deploying daemon osd.10 on smithi088 2024-04-21T08:34:54.463 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:54 smithi088 ceph-mon[31259]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:54.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:54 smithi086 ceph-mon[27939]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:54.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:54 smithi179 ceph-mon[31829]: pgmap v224: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:56.463 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:56 smithi088 ceph-mon[31259]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:56.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:56 smithi086 ceph-mon[27939]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:56.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:56 smithi179 ceph-mon[31829]: pgmap v225: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:58.390 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:58 smithi088 ceph-mon[31259]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:58.390 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:58 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:58.390 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:58 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:58.390 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:34:58 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:34:58.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:58 smithi086 ceph-mon[27939]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:58.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:58 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:58.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:58 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:58.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:34:58 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:34:58.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:58 smithi179 ceph-mon[31829]: pgmap v226: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:34:58.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:58 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:58.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:58 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:34:58.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:34:58 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:35:00.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:00 smithi088 ceph-mon[31259]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:35:00.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:00 smithi086 ceph-mon[27939]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:35:00.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:00 smithi179 ceph-mon[31829]: pgmap v227: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:35:01.518 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:35:01.518+0000 7f3148ff9700 1 -- 172.21.15.88:0/599173660 <== mgr.14152 v2:172.21.15.86:6800/1364156965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7f314c062e80 con 0x7f31380a3cc0 2024-04-21T08:35:01.518 INFO:teuthology.orchestra.run.smithi088.stdout:Created osd(s) 10 on host 'smithi088' 2024-04-21T08:35:01.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:35:01.520+0000 7f3151d82700 1 -- 172.21.15.88:0/599173660 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f31380a3cc0 msgr2=0x7f31380a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:01.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:35:01.520+0000 7f3151d82700 1 --2- 172.21.15.88:0/599173660 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f31380a3cc0 0x7f31380a6180 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f313c009990 tx=0x7f313c00b870 comp rx=0 tx=0).stop 2024-04-21T08:35:01.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:35:01.520+0000 7f3151d82700 1 -- 172.21.15.88:0/599173660 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f314c102ab0 msgr2=0x7f314c0767e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:01.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:35:01.520+0000 7f3151d82700 1 --2- 172.21.15.88:0/599173660 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f314c102ab0 0x7f314c0767e0 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7f313400d980 tx=0x7f313400dc90 comp rx=0 tx=0).stop 2024-04-21T08:35:01.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:35:01.520+0000 7f3151d82700 1 -- 172.21.15.88:0/599173660 shutdown_connections 2024-04-21T08:35:01.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:35:01.520+0000 7f3151d82700 1 --2- 172.21.15.88:0/599173660 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f31380a3cc0 0x7f31380a6180 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:01.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:35:01.520+0000 7f3151d82700 1 --2- 172.21.15.88:0/599173660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f314c104650 0x7f314c075370 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:01.520 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:35:01.520+0000 7f3151d82700 1 --2- 172.21.15.88:0/599173660 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f314c102ab0 0x7f314c0767e0 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:01.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:35:01.520+0000 7f3151d82700 1 --2- 172.21.15.88:0/599173660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f314c103cb0 0x7f314c074e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:01.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:35:01.520+0000 7f3151d82700 1 -- 172.21.15.88:0/599173660 >> 172.21.15.88:0/599173660 conn(0x7f314c0fe010 msgr2=0x7f314c106fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:01.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:35:01.520+0000 7f3151d82700 1 -- 172.21.15.88:0/599173660 shutdown_connections 2024-04-21T08:35:01.521 INFO:teuthology.orchestra.run.smithi088.stderr:2024-04-21T08:35:01.520+0000 7f3151d82700 1 -- 172.21.15.88:0/599173660 wait complete. 2024-04-21T08:35:01.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:01 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:01.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:01 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:01.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:01 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:01.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:01 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:35:01.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:01 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:01.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:01 smithi088 ceph-mon[31259]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:35:01.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:01 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:01.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:01 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:01.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:01 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:35:01.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:01 smithi088 ceph-mon[31259]: from='osd.10 [v2:172.21.15.88:6824/1294275353,v1:172.21.15.88:6825/1294275353]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-21T08:35:01.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:01 smithi088 ceph-mon[31259]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-21T08:35:01.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:01 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:01.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:01 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:01.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:01 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:01.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:01 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:35:01.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:01 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:01.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:01 smithi179 ceph-mon[31829]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:35:01.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:01 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:01.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:01 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:01.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:01 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:35:01.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:01 smithi179 ceph-mon[31829]: from='osd.10 [v2:172.21.15.88:6824/1294275353,v1:172.21.15.88:6825/1294275353]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-21T08:35:01.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:01 smithi179 ceph-mon[31829]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-21T08:35:02.114 DEBUG:teuthology.orchestra.run.smithi088:osd.10> sudo journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@osd.10.service 2024-04-21T08:35:02.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:01 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:02.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:01 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:02.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:01 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:02.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:01 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:35:02.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:01 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:02.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:01 smithi086 ceph-mon[27939]: pgmap v228: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:35:02.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:01 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:02.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:01 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:02.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:01 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:35:02.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:01 smithi086 ceph-mon[27939]: from='osd.10 [v2:172.21.15.88:6824/1294275353,v1:172.21.15.88:6825/1294275353]' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-21T08:35:02.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:01 smithi086 ceph-mon[27939]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]: dispatch 2024-04-21T08:35:02.118 INFO:tasks.cephadm:Deploying osd.11 on smithi179 with /dev/vg_nvme/lv_1... 2024-04-21T08:35:02.118 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph-volume lvm zap /dev/vg_nvme/lv_1 2024-04-21T08:35:02.362 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:35:03.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:03 smithi086 ceph-mon[27939]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-21T08:35:03.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:03 smithi086 ceph-mon[27939]: from='osd.10 [v2:172.21.15.88:6824/1294275353,v1:172.21.15.88:6825/1294275353]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-21T08:35:03.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:03 smithi086 ceph-mon[27939]: osdmap e70: 11 total, 10 up, 11 in 2024-04-21T08:35:03.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:03 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:35:03.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:03 smithi086 ceph-mon[27939]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-21T08:35:03.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:03 smithi086 ceph-mon[27939]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:35:03.881 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:03 smithi179 ceph-mon[31829]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-21T08:35:03.881 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:03 smithi179 ceph-mon[31829]: from='osd.10 [v2:172.21.15.88:6824/1294275353,v1:172.21.15.88:6825/1294275353]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-21T08:35:03.881 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:03 smithi179 ceph-mon[31829]: osdmap e70: 11 total, 10 up, 11 in 2024-04-21T08:35:03.881 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:03 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:35:03.881 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:03 smithi179 ceph-mon[31829]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-21T08:35:03.881 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:03 smithi179 ceph-mon[31829]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:35:03.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:03 smithi088 ceph-mon[31259]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["10"]}]': finished 2024-04-21T08:35:03.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:03 smithi088 ceph-mon[31259]: from='osd.10 [v2:172.21.15.88:6824/1294275353,v1:172.21.15.88:6825/1294275353]' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-21T08:35:03.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:03 smithi088 ceph-mon[31259]: osdmap e70: 11 total, 10 up, 11 in 2024-04-21T08:35:03.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:03 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:35:03.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:03 smithi088 ceph-mon[31259]: from='osd.10 ' entity='osd.10' cmd=[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]: dispatch 2024-04-21T08:35:03.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:03 smithi088 ceph-mon[31259]: pgmap v230: 1 pgs: 1 active+clean; 577 KiB data, 301 MiB used, 894 GiB / 894 GiB avail 2024-04-21T08:35:03.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:35:03 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:35:03.517+0000 7fd48066d700 -1 osd.10 0 waiting for initial osdmap 2024-04-21T08:35:03.965 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:35:03 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:35:03.529+0000 7fd47c460700 -1 osd.10 71 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T08:35:04.449 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping: /dev/vg_nvme/lv_1 2024-04-21T08:35:04.450 INFO:teuthology.orchestra.run.smithi179.stderr:Running command: /usr/bin/dd if=/dev/zero of=/dev/vg_nvme/lv_1 bs=1M count=10 conv=fsync 2024-04-21T08:35:04.450 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10+0 records in 2024-04-21T08:35:04.450 INFO:teuthology.orchestra.run.smithi179.stderr:10+0 records out 2024-04-21T08:35:04.450 INFO:teuthology.orchestra.run.smithi179.stderr: stderr: 10485760 bytes (10 MB, 10 MiB) copied, 0.011406 s, 919 MB/s 2024-04-21T08:35:04.450 INFO:teuthology.orchestra.run.smithi179.stderr:--> Zapping successful for: 2024-04-21T08:35:04.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:04 smithi086 ceph-mon[27939]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]': finished 2024-04-21T08:35:04.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:04 smithi086 ceph-mon[27939]: osdmap e71: 11 total, 10 up, 11 in 2024-04-21T08:35:04.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:04 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:35:04.875 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:04 smithi088 ceph-mon[31259]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]': finished 2024-04-21T08:35:04.875 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:04 smithi088 ceph-mon[31259]: osdmap e71: 11 total, 10 up, 11 in 2024-04-21T08:35:04.875 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:04 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:35:04.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:04 smithi179 ceph-mon[31829]: from='osd.10 ' entity='osd.10' cmd='[{"prefix": "osd crush create-or-move", "id": 10, "weight":0.0873, "args": ["host=smithi088", "root=default"]}]': finished 2024-04-21T08:35:04.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:04 smithi179 ceph-mon[31829]: osdmap e71: 11 total, 10 up, 11 in 2024-04-21T08:35:04.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:04 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:35:05.164 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph orch daemon add osd smithi179:vg_nvme/lv_1 2024-04-21T08:35:05.399 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:35:05.858 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:05 smithi179 ceph-mon[31829]: purged_snaps scrub starts 2024-04-21T08:35:05.858 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:05 smithi179 ceph-mon[31829]: purged_snaps scrub ok 2024-04-21T08:35:05.858 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:05 smithi179 ceph-mon[31829]: osd.10 [v2:172.21.15.88:6824/1294275353,v1:172.21.15.88:6825/1294275353] boot 2024-04-21T08:35:05.858 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:05 smithi179 ceph-mon[31829]: osdmap e72: 11 total, 11 up, 11 in 2024-04-21T08:35:05.858 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:05 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:35:05.858 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:05 smithi179 ceph-mon[31829]: Detected new or changed devices on smithi088 2024-04-21T08:35:05.858 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:05 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:05.858 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:05 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:05.858 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:05 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:05.858 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:05 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:05.859 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:05 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:05.859 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:05 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:05.859 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:05 smithi179 ceph-mon[31829]: Adjusting osd_memory_target on smithi088 to 4272M 2024-04-21T08:35:05.859 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:05 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:05.859 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:05 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:05.859 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:05 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:35:05.859 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:05 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:05.859 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:05 smithi179 ceph-mon[31829]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:05.859 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:05 smithi179 ceph-mon[31829]: osdmap e73: 11 total, 11 up, 11 in 2024-04-21T08:35:05.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:05 smithi086 ceph-mon[27939]: purged_snaps scrub starts 2024-04-21T08:35:05.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:05 smithi086 ceph-mon[27939]: purged_snaps scrub ok 2024-04-21T08:35:05.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:05 smithi086 ceph-mon[27939]: osd.10 [v2:172.21.15.88:6824/1294275353,v1:172.21.15.88:6825/1294275353] boot 2024-04-21T08:35:05.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:05 smithi086 ceph-mon[27939]: osdmap e72: 11 total, 11 up, 11 in 2024-04-21T08:35:05.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:05 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:35:05.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:05 smithi086 ceph-mon[27939]: Detected new or changed devices on smithi088 2024-04-21T08:35:05.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:05 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:05.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:05 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:05.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:05 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:05.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:05 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:05.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:05 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:05.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:05 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:05.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:05 smithi086 ceph-mon[27939]: Adjusting osd_memory_target on smithi088 to 4272M 2024-04-21T08:35:05.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:05 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:05.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:05 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:05.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:05 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:35:05.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:05 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:05.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:05 smithi086 ceph-mon[27939]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:05.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:05 smithi086 ceph-mon[27939]: osdmap e73: 11 total, 11 up, 11 in 2024-04-21T08:35:05.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:05 smithi088 ceph-mon[31259]: purged_snaps scrub starts 2024-04-21T08:35:05.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:05 smithi088 ceph-mon[31259]: purged_snaps scrub ok 2024-04-21T08:35:05.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:05 smithi088 ceph-mon[31259]: osd.10 [v2:172.21.15.88:6824/1294275353,v1:172.21.15.88:6825/1294275353] boot 2024-04-21T08:35:05.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:05 smithi088 ceph-mon[31259]: osdmap e72: 11 total, 11 up, 11 in 2024-04-21T08:35:05.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:05 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:35:05.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:05 smithi088 ceph-mon[31259]: Detected new or changed devices on smithi088 2024-04-21T08:35:05.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:05 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:05.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:05 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:05.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:05 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:05.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:05 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:05.965 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:05 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:05.965 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:05 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:05.965 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:05 smithi088 ceph-mon[31259]: Adjusting osd_memory_target on smithi088 to 4272M 2024-04-21T08:35:05.965 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:05 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:05.965 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:05 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:05.965 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:05 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:35:05.965 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:05 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:05.965 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:05 smithi088 ceph-mon[31259]: pgmap v233: 1 pgs: 1 peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:05.965 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:05 smithi088 ceph-mon[31259]: osdmap e73: 11 total, 11 up, 11 in 2024-04-21T08:35:07.027 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.025+0000 7fc4207da700 1 -- 172.21.15.179:0/2638833812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4181019c0 msgr2=0x7fc418101de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:07.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.025+0000 7fc4207da700 1 --2- 172.21.15.179:0/2638833812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4181019c0 0x7fc418101de0 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7fc414009990 tx=0x7fc414009ca0 comp rx=0 tx=0).stop 2024-04-21T08:35:07.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.026+0000 7fc4207da700 1 -- 172.21.15.179:0/2638833812 shutdown_connections 2024-04-21T08:35:07.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.026+0000 7fc4207da700 1 --2- 172.21.15.179:0/2638833812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc418103560 0x7fc418109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:07.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.026+0000 7fc4207da700 1 --2- 172.21.15.179:0/2638833812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4181019c0 0x7fc418101de0 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:07.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.026+0000 7fc4207da700 1 --2- 172.21.15.179:0/2638833812 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc418102bc0 0x7fc418103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:07.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.026+0000 7fc4207da700 1 -- 172.21.15.179:0/2638833812 >> 172.21.15.179:0/2638833812 conn(0x7fc4180fcfa0 msgr2=0x7fc4180ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:07.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.026+0000 7fc4207da700 1 -- 172.21.15.179:0/2638833812 shutdown_connections 2024-04-21T08:35:07.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.026+0000 7fc4207da700 1 -- 172.21.15.179:0/2638833812 wait complete. 2024-04-21T08:35:07.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.026+0000 7fc4207da700 1 Processor -- start 2024-04-21T08:35:07.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.026+0000 7fc4207da700 1 -- start start 2024-04-21T08:35:07.028 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.026+0000 7fc4207da700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4181019c0 0x7fc418197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:07.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.026+0000 7fc4207da700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc418102bc0 0x7fc4181980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:07.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.026+0000 7fc4207da700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc418103560 0x7fc41819f150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:07.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.026+0000 7fc4207da700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc41810d4a0 con 0x7fc4181019c0 2024-04-21T08:35:07.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.026+0000 7fc4207da700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc41810d320 con 0x7fc418103560 2024-04-21T08:35:07.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.026+0000 7fc4207da700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc41810d620 con 0x7fc418102bc0 2024-04-21T08:35:07.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.026+0000 7fc41dd75700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc418102bc0 0x7fc4181980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:07.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.027+0000 7fc41dd75700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc418102bc0 0x7fc4181980d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:42210/0 (socket says 172.21.15.179:42210) 2024-04-21T08:35:07.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.027+0000 7fc41dd75700 1 -- 172.21.15.179:0/958143830 learned_addr learned my addr 172.21.15.179:0/958143830 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-21T08:35:07.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.027+0000 7fc41e576700 1 --2- 172.21.15.179:0/958143830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4181019c0 0x7fc418197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:07.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.027+0000 7fc41ed77700 1 --2- 172.21.15.179:0/958143830 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc418103560 0x7fc41819f150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:07.029 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.027+0000 7fc41dd75700 1 -- 172.21.15.179:0/958143830 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc418103560 msgr2=0x7fc41819f150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:07.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.027+0000 7fc41dd75700 1 --2- 172.21.15.179:0/958143830 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc418103560 0x7fc41819f150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:07.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.027+0000 7fc41dd75700 1 -- 172.21.15.179:0/958143830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4181019c0 msgr2=0x7fc418197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:07.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.027+0000 7fc41dd75700 1 --2- 172.21.15.179:0/958143830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4181019c0 0x7fc418197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:07.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.027+0000 7fc41dd75700 1 -- 172.21.15.179:0/958143830 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc41819f7f0 con 0x7fc418102bc0 2024-04-21T08:35:07.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.027+0000 7fc41dd75700 1 --2- 172.21.15.179:0/958143830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc418102bc0 0x7fc4181980d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc40800d9b0 tx=0x7fc40800dcc0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:07.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.027+0000 7fc40f7fe700 1 -- 172.21.15.179:0/958143830 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc408017070 con 0x7fc418102bc0 2024-04-21T08:35:07.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.027+0000 7fc40f7fe700 1 -- 172.21.15.179:0/958143830 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc408005020 con 0x7fc418102bc0 2024-04-21T08:35:07.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.027+0000 7fc40f7fe700 1 -- 172.21.15.179:0/958143830 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc408009a20 con 0x7fc418102bc0 2024-04-21T08:35:07.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.027+0000 7fc4207da700 1 -- 172.21.15.179:0/958143830 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc41819fae0 con 0x7fc418102bc0 2024-04-21T08:35:07.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.027+0000 7fc4207da700 1 -- 172.21.15.179:0/958143830 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc41819ffd0 con 0x7fc418102bc0 2024-04-21T08:35:07.030 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.028+0000 7fc4207da700 1 -- 172.21.15.179:0/958143830 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc418069730 con 0x7fc418102bc0 2024-04-21T08:35:07.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.029+0000 7fc40f7fe700 1 -- 172.21.15.179:0/958143830 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc4080051d0 con 0x7fc418102bc0 2024-04-21T08:35:07.031 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.030+0000 7fc40f7fe700 1 --2- 172.21.15.179:0/958143830 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fc4040a3cc0 0x7fc4040a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:07.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.030+0000 7fc40f7fe700 1 -- 172.21.15.179:0/958143830 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(74..74 src has 1..74) ==== 6723+0+0 (secure 0 0 0) 0x7fc4080cfb50 con 0x7fc418102bc0 2024-04-21T08:35:07.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.030+0000 7fc41e576700 1 --2- 172.21.15.179:0/958143830 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fc4040a3cc0 0x7fc4040a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:07.032 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.031+0000 7fc41e576700 1 --2- 172.21.15.179:0/958143830 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fc4040a3cc0 0x7fc4040a6180 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc4140038a0 tx=0x7fc414005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:07.034 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.034+0000 7fc40f7fe700 1 -- 172.21.15.179:0/958143830 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc408020020 con 0x7fc418102bc0 2024-04-21T08:35:07.242 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:07.243+0000 7fc4207da700 1 -- 172.21.15.179:0/958143830 --> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] -- mgr_command(tid 0: {"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_1", "target": ["mon-mgr", ""]}) -- 0x7fc418062e80 con 0x7fc4040a3cc0 2024-04-21T08:35:07.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:07 smithi179 ceph-mon[31829]: osdmap e74: 11 total, 11 up, 11 in 2024-04-21T08:35:07.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:07 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:35:07.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:07 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:35:07.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:07 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:07.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:07 smithi088 ceph-mon[31259]: osdmap e74: 11 total, 11 up, 11 in 2024-04-21T08:35:07.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:07 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:35:07.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:07 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:35:07.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:07 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:07.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:07 smithi086 ceph-mon[27939]: osdmap e74: 11 total, 11 up, 11 in 2024-04-21T08:35:07.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:07 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd tree", "states": ["destroyed"], "format": "json"}]: dispatch 2024-04-21T08:35:07.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:07 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.bootstrap-osd"}]: dispatch 2024-04-21T08:35:07.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:07 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:08.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:08 smithi088 ceph-mon[31259]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:35:08.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:08 smithi088 ceph-mon[31259]: pgmap v236: 1 pgs: 1 peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:08.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:08 smithi088 ceph-mon[31259]: osdmap e75: 11 total, 11 up, 11 in 2024-04-21T08:35:08.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:08 smithi179 ceph-mon[31829]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:35:08.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:08 smithi179 ceph-mon[31829]: pgmap v236: 1 pgs: 1 peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:08.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:08 smithi179 ceph-mon[31829]: osdmap e75: 11 total, 11 up, 11 in 2024-04-21T08:35:08.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:08 smithi086 ceph-mon[27939]: from='client.24334 -' entity='client.admin' cmd=[{"prefix": "orch daemon add osd", "svc_arg": "smithi179:vg_nvme/lv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:35:08.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:08 smithi086 ceph-mon[27939]: pgmap v236: 1 pgs: 1 peering; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:08.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:08 smithi086 ceph-mon[27939]: osdmap e75: 11 total, 11 up, 11 in 2024-04-21T08:35:09.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:09 smithi088 ceph-mon[31259]: osdmap e76: 11 total, 11 up, 11 in 2024-04-21T08:35:09.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:09 smithi179 ceph-mon[31829]: osdmap e76: 11 total, 11 up, 11 in 2024-04-21T08:35:09.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:09 smithi086 ceph-mon[27939]: osdmap e76: 11 total, 11 up, 11 in 2024-04-21T08:35:10.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:10 smithi088 ceph-mon[31259]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-21T08:35:10.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:10 smithi088 ceph-mon[31259]: from='client.? 172.21.15.179:0/1093202276' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0d4cc76-e7a6-4a6b-909c-95488ccd5a36"}]: dispatch 2024-04-21T08:35:10.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:10 smithi088 ceph-mon[31259]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0d4cc76-e7a6-4a6b-909c-95488ccd5a36"}]: dispatch 2024-04-21T08:35:10.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:10 smithi088 ceph-mon[31259]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0d4cc76-e7a6-4a6b-909c-95488ccd5a36"}]': finished 2024-04-21T08:35:10.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:10 smithi088 ceph-mon[31259]: osdmap e77: 12 total, 11 up, 12 in 2024-04-21T08:35:10.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:10 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:35:10.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:10 smithi179 ceph-mon[31829]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-21T08:35:10.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:10 smithi179 ceph-mon[31829]: from='client.? 172.21.15.179:0/1093202276' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0d4cc76-e7a6-4a6b-909c-95488ccd5a36"}]: dispatch 2024-04-21T08:35:10.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:10 smithi179 ceph-mon[31829]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0d4cc76-e7a6-4a6b-909c-95488ccd5a36"}]: dispatch 2024-04-21T08:35:10.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:10 smithi179 ceph-mon[31829]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0d4cc76-e7a6-4a6b-909c-95488ccd5a36"}]': finished 2024-04-21T08:35:10.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:10 smithi179 ceph-mon[31829]: osdmap e77: 12 total, 11 up, 12 in 2024-04-21T08:35:10.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:10 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:35:10.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:10 smithi086 ceph-mon[27939]: pgmap v239: 1 pgs: 1 active+clean; 577 KiB data, 328 MiB used, 983 GiB / 983 GiB avail; 144 KiB/s, 0 objects/s recovering 2024-04-21T08:35:10.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:10 smithi086 ceph-mon[27939]: from='client.? 172.21.15.179:0/1093202276' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0d4cc76-e7a6-4a6b-909c-95488ccd5a36"}]: dispatch 2024-04-21T08:35:10.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:10 smithi086 ceph-mon[27939]: from='client.? ' entity='client.bootstrap-osd' cmd=[{"prefix": "osd new", "uuid": "e0d4cc76-e7a6-4a6b-909c-95488ccd5a36"}]: dispatch 2024-04-21T08:35:10.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:10 smithi086 ceph-mon[27939]: from='client.? ' entity='client.bootstrap-osd' cmd='[{"prefix": "osd new", "uuid": "e0d4cc76-e7a6-4a6b-909c-95488ccd5a36"}]': finished 2024-04-21T08:35:10.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:10 smithi086 ceph-mon[27939]: osdmap e77: 12 total, 11 up, 12 in 2024-04-21T08:35:10.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:10 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:35:11.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:11 smithi088 ceph-mon[31259]: from='client.? 172.21.15.179:0/3039527384' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:35:11.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:11 smithi179 ceph-mon[31829]: from='client.? 172.21.15.179:0/3039527384' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:35:11.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:11 smithi086 ceph-mon[27939]: from='client.? 172.21.15.179:0/3039527384' entity='client.bootstrap-osd' cmd=[{"prefix": "mon getmap"}]: dispatch 2024-04-21T08:35:12.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:12 smithi088 ceph-mon[31259]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-04-21T08:35:12.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:12 smithi179 ceph-mon[31829]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-04-21T08:35:12.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:12 smithi086 ceph-mon[27939]: pgmap v241: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 119 KiB/s, 0 objects/s recovering 2024-04-21T08:35:14.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:14 smithi088 ceph-mon[31259]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-21T08:35:14.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:14 smithi179 ceph-mon[31829]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-21T08:35:14.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:14 smithi086 ceph-mon[27939]: pgmap v242: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 96 KiB/s, 0 objects/s recovering 2024-04-21T08:35:16.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:16 smithi088 ceph-mon[31259]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-21T08:35:16.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:16 smithi179 ceph-mon[31829]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-21T08:35:16.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:16 smithi086 ceph-mon[27939]: pgmap v243: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 74 KiB/s, 0 objects/s recovering 2024-04-21T08:35:18.457 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:18 smithi179 ceph-mon[31829]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-21T08:35:18.458 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:18 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-21T08:35:18.458 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:18 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:18.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:18 smithi088 ceph-mon[31259]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-21T08:35:18.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:18 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-21T08:35:18.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:18 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:18.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:18 smithi086 ceph-mon[27939]: pgmap v244: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail; 65 KiB/s, 0 objects/s recovering 2024-04-21T08:35:18.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:18 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "osd.11"}]: dispatch 2024-04-21T08:35:18.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:18 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:19.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:19 smithi179 ceph-mon[31829]: Deploying daemon osd.11 on smithi179 2024-04-21T08:35:19.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:19 smithi086 ceph-mon[27939]: Deploying daemon osd.11 on smithi179 2024-04-21T08:35:19.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:19 smithi088 ceph-mon[31259]: Deploying daemon osd.11 on smithi179 2024-04-21T08:35:20.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:20 smithi179 ceph-mon[31829]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:20.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:20 smithi086 ceph-mon[27939]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:20.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:20 smithi088 ceph-mon[31259]: pgmap v245: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:21.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:21 smithi179 ceph-mon[31829]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:21.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:21 smithi086 ceph-mon[27939]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:21.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:21 smithi088 ceph-mon[31259]: pgmap v246: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:24.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:24 smithi179 ceph-mon[31829]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:24.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:24 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:24.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:24 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:24.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:24 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:35:24.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:24 smithi086 ceph-mon[27939]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:24.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:24 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:24.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:24 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:24.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:24 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:35:24.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:24 smithi088 ceph-mon[31259]: pgmap v247: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:24.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:24 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:24.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:24 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:24.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:24 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:35:26.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:26 smithi086 ceph-mon[27939]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:26.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:26 smithi088 ceph-mon[31259]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:26.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:26 smithi179 ceph-mon[31829]: pgmap v248: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:27.265 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:27.266+0000 7fc40f7fe700 1 -- 172.21.15.179:0/958143830 <== mgr.14152 v2:172.21.15.86:6800/1364156965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+38 (secure 0 0 0) 0x7fc418062e80 con 0x7fc4040a3cc0 2024-04-21T08:35:27.266 INFO:teuthology.orchestra.run.smithi179.stdout:Created osd(s) 11 on host 'smithi179' 2024-04-21T08:35:27.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:27.268+0000 7fc4207da700 1 -- 172.21.15.179:0/958143830 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fc4040a3cc0 msgr2=0x7fc4040a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:27.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:27.268+0000 7fc4207da700 1 --2- 172.21.15.179:0/958143830 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fc4040a3cc0 0x7fc4040a6180 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7fc4140038a0 tx=0x7fc414005fb0 comp rx=0 tx=0).stop 2024-04-21T08:35:27.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:27.268+0000 7fc4207da700 1 -- 172.21.15.179:0/958143830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc418102bc0 msgr2=0x7fc4181980d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:27.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:27.268+0000 7fc4207da700 1 --2- 172.21.15.179:0/958143830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc418102bc0 0x7fc4181980d0 secure :-1 s=READY pgs=39 cs=0 l=1 rev1=1 crypto rx=0x7fc40800d9b0 tx=0x7fc40800dcc0 comp rx=0 tx=0).stop 2024-04-21T08:35:27.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:27.268+0000 7fc4207da700 1 -- 172.21.15.179:0/958143830 shutdown_connections 2024-04-21T08:35:27.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:27.268+0000 7fc4207da700 1 --2- 172.21.15.179:0/958143830 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fc4040a3cc0 0x7fc4040a6180 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:27.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:27.268+0000 7fc4207da700 1 --2- 172.21.15.179:0/958143830 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc418103560 0x7fc41819f150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:27.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:27.268+0000 7fc4207da700 1 --2- 172.21.15.179:0/958143830 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4181019c0 0x7fc418197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:27.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:27.268+0000 7fc4207da700 1 --2- 172.21.15.179:0/958143830 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc418102bc0 0x7fc4181980d0 unknown :-1 s=CLOSED pgs=39 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:27.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:27.268+0000 7fc4207da700 1 -- 172.21.15.179:0/958143830 >> 172.21.15.179:0/958143830 conn(0x7fc4180fcfa0 msgr2=0x7fc418105fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:27.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:27.268+0000 7fc4207da700 1 -- 172.21.15.179:0/958143830 shutdown_connections 2024-04-21T08:35:27.268 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:27.269+0000 7fc4207da700 1 -- 172.21.15.179:0/958143830 wait complete. 2024-04-21T08:35:27.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:27 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:27.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:27 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:27.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:27 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:27.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:27 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:35:27.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:27 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:27.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:27 smithi179 ceph-mon[31829]: from='osd.11 [v2:172.21.15.179:6824/4007070233,v1:172.21.15.179:6825/4007070233]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-21T08:35:27.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:27 smithi179 ceph-mon[31829]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-21T08:35:27.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:27 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:27.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:27 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:27.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:27 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:35:27.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:27 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:27.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:27 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:27.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:27 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:27.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:27 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:35:27.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:27 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:27.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:27 smithi086 ceph-mon[27939]: from='osd.11 [v2:172.21.15.179:6824/4007070233,v1:172.21.15.179:6825/4007070233]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-21T08:35:27.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:27 smithi086 ceph-mon[27939]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-21T08:35:27.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:27 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:27.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:27 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:27.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:27 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:35:27.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:27 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:27.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:27 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:27.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:27 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:27.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:27 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:35:27.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:27 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:27.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:27 smithi088 ceph-mon[31259]: from='osd.11 [v2:172.21.15.179:6824/4007070233,v1:172.21.15.179:6825/4007070233]' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-21T08:35:27.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:27 smithi088 ceph-mon[31259]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]: dispatch 2024-04-21T08:35:27.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:27 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:27.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:27 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:27.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:27 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:35:28.162 DEBUG:teuthology.orchestra.run.smithi179:osd.11> sudo journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@osd.11.service 2024-04-21T08:35:28.165 INFO:tasks.cephadm:Waiting for 12 OSDs to come up... 2024-04-21T08:35:28.165 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph osd stat -f json 2024-04-21T08:35:28.402 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:28 smithi179 ceph-mon[31829]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:28.403 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:28 smithi179 ceph-mon[31829]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-21T08:35:28.403 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:28 smithi179 ceph-mon[31829]: osdmap e78: 12 total, 11 up, 12 in 2024-04-21T08:35:28.403 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:28 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:35:28.403 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:28 smithi179 ceph-mon[31829]: from='osd.11 [v2:172.21.15.179:6824/4007070233,v1:172.21.15.179:6825/4007070233]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-21T08:35:28.403 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:28 smithi179 ceph-mon[31829]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-21T08:35:28.414 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:35:28.536 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:28 smithi086 ceph-mon[27939]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:28.536 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:28 smithi086 ceph-mon[27939]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-21T08:35:28.536 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:28 smithi086 ceph-mon[27939]: osdmap e78: 12 total, 11 up, 12 in 2024-04-21T08:35:28.537 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:28 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:35:28.537 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:28 smithi086 ceph-mon[27939]: from='osd.11 [v2:172.21.15.179:6824/4007070233,v1:172.21.15.179:6825/4007070233]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-21T08:35:28.537 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:28 smithi086 ceph-mon[27939]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-21T08:35:28.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:28 smithi088 ceph-mon[31259]: pgmap v249: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:28.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:28 smithi088 ceph-mon[31259]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush set-device-class", "class": "ssd", "ids": ["11"]}]': finished 2024-04-21T08:35:28.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:28 smithi088 ceph-mon[31259]: osdmap e78: 12 total, 11 up, 12 in 2024-04-21T08:35:28.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:28 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:35:28.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:28 smithi088 ceph-mon[31259]: from='osd.11 [v2:172.21.15.179:6824/4007070233,v1:172.21.15.179:6825/4007070233]' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-21T08:35:28.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:28 smithi088 ceph-mon[31259]: from='osd.11 ' entity='osd.11' cmd=[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]: dispatch 2024-04-21T08:35:29.368 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:35:29 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:35:29.045+0000 7f4a20f20700 -1 osd.11 0 waiting for initial osdmap 2024-04-21T08:35:29.369 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:35:29 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:35:29.061+0000 7f4a1850a700 -1 osd.11 79 set_numa_affinity unable to identify public interface '' numa node: (2) No such file or directory 2024-04-21T08:35:29.619 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:29 smithi086 ceph-mon[27939]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-04-21T08:35:29.619 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:29 smithi086 ceph-mon[27939]: osdmap e79: 12 total, 11 up, 12 in 2024-04-21T08:35:29.620 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:29 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:35:29.620 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:29 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:35:29.620 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:29 smithi086 ceph-mon[27939]: from='osd.11 ' entity='osd.11' 2024-04-21T08:35:29.667 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:29 smithi179 ceph-mon[31829]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-04-21T08:35:29.667 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:29 smithi179 ceph-mon[31829]: osdmap e79: 12 total, 11 up, 12 in 2024-04-21T08:35:29.667 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:29 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:35:29.667 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:29 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:35:29.667 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:29 smithi179 ceph-mon[31829]: from='osd.11 ' entity='osd.11' 2024-04-21T08:35:29.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:29 smithi088 ceph-mon[31259]: from='osd.11 ' entity='osd.11' cmd='[{"prefix": "osd crush create-or-move", "id": 11, "weight":0.0873, "args": ["host=smithi179", "root=default"]}]': finished 2024-04-21T08:35:29.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:29 smithi088 ceph-mon[31259]: osdmap e79: 12 total, 11 up, 12 in 2024-04-21T08:35:29.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:29 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:35:29.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:29 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:35:29.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:29 smithi088 ceph-mon[31259]: from='osd.11 ' entity='osd.11' 2024-04-21T08:35:30.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.381+0000 7f3eb7af3700 1 -- 172.21.15.86:0/2702465535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3eb0102ad0 msgr2=0x7f3eb0102ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:30.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.381+0000 7f3eb7af3700 1 --2- 172.21.15.86:0/2702465535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3eb0102ad0 0x7f3eb0102ef0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f3eac0099c0 tx=0x7f3eac009cd0 comp rx=0 tx=0).stop 2024-04-21T08:35:30.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.381+0000 7f3eb7af3700 1 -- 172.21.15.86:0/2702465535 shutdown_connections 2024-04-21T08:35:30.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.381+0000 7f3eb7af3700 1 --2- 172.21.15.86:0/2702465535 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3eb0104670 0x7f3eb010af20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:30.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.381+0000 7f3eb7af3700 1 --2- 172.21.15.86:0/2702465535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3eb0102ad0 0x7f3eb0102ef0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:30.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.381+0000 7f3eb7af3700 1 --2- 172.21.15.86:0/2702465535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3eb0103cd0 0x7f3eb0104130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:30.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.381+0000 7f3eb7af3700 1 -- 172.21.15.86:0/2702465535 >> 172.21.15.86:0/2702465535 conn(0x7f3eb00fe050 msgr2=0x7f3eb01004b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:30.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.381+0000 7f3eb7af3700 1 -- 172.21.15.86:0/2702465535 shutdown_connections 2024-04-21T08:35:30.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.381+0000 7f3eb7af3700 1 -- 172.21.15.86:0/2702465535 wait complete. 2024-04-21T08:35:30.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.382+0000 7f3eb7af3700 1 Processor -- start 2024-04-21T08:35:30.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.382+0000 7f3eb7af3700 1 -- start start 2024-04-21T08:35:30.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.382+0000 7f3eb7af3700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3eb0102ad0 0x7f3eb0198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:30.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.382+0000 7f3eb7af3700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3eb0103cd0 0x7f3eb01990c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:30.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.382+0000 7f3eb7af3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3eb0104670 0x7f3eb01a0140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:30.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.382+0000 7f3eb7af3700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb010e540 con 0x7f3eb0103cd0 2024-04-21T08:35:30.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.382+0000 7f3eb7af3700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb010e3c0 con 0x7f3eb0102ad0 2024-04-21T08:35:30.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.382+0000 7f3eb7af3700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3eb010e6c0 con 0x7f3eb0104670 2024-04-21T08:35:30.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.382+0000 7f3eb508e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3eb0103cd0 0x7f3eb01990c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:30.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.382+0000 7f3eb508e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3eb0103cd0 0x7f3eb01990c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40788/0 (socket says 172.21.15.86:40788) 2024-04-21T08:35:30.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.382+0000 7f3eb508e700 1 -- 172.21.15.86:0/3829902225 learned_addr learned my addr 172.21.15.86:0/3829902225 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:35:30.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.382+0000 7f3eb6090700 1 --2- 172.21.15.86:0/3829902225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3eb0104670 0x7f3eb01a0140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:30.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.382+0000 7f3eb588f700 1 --2- 172.21.15.86:0/3829902225 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3eb0102ad0 0x7f3eb0198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:30.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.382+0000 7f3eb508e700 1 -- 172.21.15.86:0/3829902225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3eb0104670 msgr2=0x7f3eb01a0140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:30.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.382+0000 7f3eb508e700 1 --2- 172.21.15.86:0/3829902225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3eb0104670 0x7f3eb01a0140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:30.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.382+0000 7f3eb508e700 1 -- 172.21.15.86:0/3829902225 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3eb0102ad0 msgr2=0x7f3eb0198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:30.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.382+0000 7f3eb508e700 1 --2- 172.21.15.86:0/3829902225 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3eb0102ad0 0x7f3eb0198b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:30.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.382+0000 7f3eb508e700 1 -- 172.21.15.86:0/3829902225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3eb01a07e0 con 0x7f3eb0103cd0 2024-04-21T08:35:30.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.383+0000 7f3eb508e700 1 --2- 172.21.15.86:0/3829902225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3eb0103cd0 0x7f3eb01990c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000d980 tx=0x7f3ea000dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:30.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.383+0000 7f3ea6ffd700 1 -- 172.21.15.86:0/3829902225 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea0016070 con 0x7f3eb0103cd0 2024-04-21T08:35:30.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.383+0000 7f3eb7af3700 1 -- 172.21.15.86:0/3829902225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3eb01a0ad0 con 0x7f3eb0103cd0 2024-04-21T08:35:30.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.383+0000 7f3ea6ffd700 1 -- 172.21.15.86:0/3829902225 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3ea000de70 con 0x7f3eb0103cd0 2024-04-21T08:35:30.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.383+0000 7f3ea6ffd700 1 -- 172.21.15.86:0/3829902225 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3ea000f3c0 con 0x7f3eb0103cd0 2024-04-21T08:35:30.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.383+0000 7f3eb7af3700 1 -- 172.21.15.86:0/3829902225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3eb01a1010 con 0x7f3eb0103cd0 2024-04-21T08:35:30.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.385+0000 7f3eb7af3700 1 -- 172.21.15.86:0/3829902225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3eb0069730 con 0x7f3eb0103cd0 2024-04-21T08:35:30.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.385+0000 7f3ea6ffd700 1 -- 172.21.15.86:0/3829902225 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3ea0019070 con 0x7f3eb0103cd0 2024-04-21T08:35:30.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.385+0000 7f3ea6ffd700 1 --2- 172.21.15.86:0/3829902225 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f3e9c0a3c40 0x7f3e9c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:30.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.386+0000 7f3eb588f700 1 --2- 172.21.15.86:0/3829902225 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f3e9c0a3c40 0x7f3e9c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:30.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.386+0000 7f3ea6ffd700 1 -- 172.21.15.86:0/3829902225 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(80..80 src has 1..80) ==== 7127+0+0 (secure 0 0 0) 0x7f3ea00cfdf0 con 0x7f3eb0103cd0 2024-04-21T08:35:30.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.386+0000 7f3eb588f700 1 --2- 172.21.15.86:0/3829902225 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f3e9c0a3c40 0x7f3e9c0a6100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3eac00b580 tx=0x7f3eac005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:30.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.390+0000 7f3ea6ffd700 1 -- 172.21.15.86:0/3829902225 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3ea009cc70 con 0x7f3eb0103cd0 2024-04-21T08:35:30.549 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:30 smithi086 ceph-mon[27939]: purged_snaps scrub starts 2024-04-21T08:35:30.549 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:30 smithi086 ceph-mon[27939]: purged_snaps scrub ok 2024-04-21T08:35:30.549 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:30 smithi086 ceph-mon[27939]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:30.549 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:30 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:35:30.549 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:30 smithi086 ceph-mon[27939]: osd.11 [v2:172.21.15.179:6824/4007070233,v1:172.21.15.179:6825/4007070233] boot 2024-04-21T08:35:30.549 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:30 smithi086 ceph-mon[27939]: osdmap e80: 12 total, 12 up, 12 in 2024-04-21T08:35:30.550 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:30 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:35:30.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.596+0000 7f3eb7af3700 1 -- 172.21.15.86:0/3829902225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd stat", "format": "json"} v 0) -- 0x7f3eb0063650 con 0x7f3eb0103cd0 2024-04-21T08:35:30.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.596+0000 7f3ea6ffd700 1 -- 172.21.15.86:0/3829902225 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd stat", "format": "json"}]=0 v80) ==== 74+0+133 (secure 0 0 0) 0x7f3ea009c570 con 0x7f3eb0103cd0 2024-04-21T08:35:30.596 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:35:30.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.599+0000 7f3eb7af3700 1 -- 172.21.15.86:0/3829902225 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f3e9c0a3c40 msgr2=0x7f3e9c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:30.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.599+0000 7f3eb7af3700 1 --2- 172.21.15.86:0/3829902225 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f3e9c0a3c40 0x7f3e9c0a6100 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f3eac00b580 tx=0x7f3eac005fb0 comp rx=0 tx=0).stop 2024-04-21T08:35:30.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.599+0000 7f3eb7af3700 1 -- 172.21.15.86:0/3829902225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3eb0103cd0 msgr2=0x7f3eb01990c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:30.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.599+0000 7f3eb7af3700 1 --2- 172.21.15.86:0/3829902225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3eb0103cd0 0x7f3eb01990c0 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f3ea000d980 tx=0x7f3ea000dc90 comp rx=0 tx=0).stop 2024-04-21T08:35:30.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.599+0000 7f3eb7af3700 1 -- 172.21.15.86:0/3829902225 shutdown_connections 2024-04-21T08:35:30.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.599+0000 7f3eb7af3700 1 --2- 172.21.15.86:0/3829902225 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f3e9c0a3c40 0x7f3e9c0a6100 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:30.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.599+0000 7f3eb7af3700 1 --2- 172.21.15.86:0/3829902225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3eb0104670 0x7f3eb01a0140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:30.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.599+0000 7f3eb7af3700 1 --2- 172.21.15.86:0/3829902225 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3eb0102ad0 0x7f3eb0198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:30.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.599+0000 7f3eb7af3700 1 --2- 172.21.15.86:0/3829902225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3eb0103cd0 0x7f3eb01990c0 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:30.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.599+0000 7f3eb7af3700 1 -- 172.21.15.86:0/3829902225 >> 172.21.15.86:0/3829902225 conn(0x7f3eb00fe050 msgr2=0x7f3eb0106fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:30.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.599+0000 7f3eb7af3700 1 -- 172.21.15.86:0/3829902225 shutdown_connections 2024-04-21T08:35:30.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:30.599+0000 7f3eb7af3700 1 -- 172.21.15.86:0/3829902225 wait complete. 2024-04-21T08:35:30.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:30 smithi088 ceph-mon[31259]: purged_snaps scrub starts 2024-04-21T08:35:30.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:30 smithi088 ceph-mon[31259]: purged_snaps scrub ok 2024-04-21T08:35:30.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:30 smithi088 ceph-mon[31259]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:30.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:30 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:35:30.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:30 smithi088 ceph-mon[31259]: osd.11 [v2:172.21.15.179:6824/4007070233,v1:172.21.15.179:6825/4007070233] boot 2024-04-21T08:35:30.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:30 smithi088 ceph-mon[31259]: osdmap e80: 12 total, 12 up, 12 in 2024-04-21T08:35:30.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:30 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:35:30.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:30 smithi179 ceph-mon[31829]: purged_snaps scrub starts 2024-04-21T08:35:30.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:30 smithi179 ceph-mon[31829]: purged_snaps scrub ok 2024-04-21T08:35:30.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:30 smithi179 ceph-mon[31829]: pgmap v252: 1 pgs: 1 active+clean; 577 KiB data, 329 MiB used, 983 GiB / 983 GiB avail 2024-04-21T08:35:30.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:30 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:35:30.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:30 smithi179 ceph-mon[31829]: osd.11 [v2:172.21.15.179:6824/4007070233,v1:172.21.15.179:6825/4007070233] boot 2024-04-21T08:35:30.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:30 smithi179 ceph-mon[31829]: osdmap e80: 12 total, 12 up, 12 in 2024-04-21T08:35:30.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:30 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:35:31.371 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":80,"num_osds":12,"num_up_osds":12,"osd_up_since":1713688530,"num_in_osds":12,"osd_in_since":1713688510,"num_remapped_pgs":0} 2024-04-21T08:35:31.372 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-21T08:35:31.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:31 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3829902225' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-21T08:35:31.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:31 smithi086 ceph-mon[27939]: osdmap e81: 12 total, 12 up, 12 in 2024-04-21T08:35:31.622 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:35:31.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:31 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3829902225' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-21T08:35:31.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:31 smithi088 ceph-mon[31259]: osdmap e81: 12 total, 12 up, 12 in 2024-04-21T08:35:31.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:31 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3829902225' entity='client.admin' cmd=[{"prefix": "osd stat", "format": "json"}]: dispatch 2024-04-21T08:35:31.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:31 smithi179 ceph-mon[31829]: osdmap e81: 12 total, 12 up, 12 in 2024-04-21T08:35:32.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:32 smithi088 ceph-mon[31259]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:32.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:32 smithi088 ceph-mon[31259]: Detected new or changed devices on smithi179 2024-04-21T08:35:32.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:32 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:32.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:32 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:32.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:32 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:32.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:32 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:32.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:32 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:32.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:32 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:32.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:32 smithi088 ceph-mon[31259]: Adjusting osd_memory_target on smithi179 to 4272M 2024-04-21T08:35:32.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:32 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:32.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:32 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:32.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:32 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:35:32.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:32 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:32.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:32 smithi179 ceph-mon[31829]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:32.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:32 smithi179 ceph-mon[31829]: Detected new or changed devices on smithi179 2024-04-21T08:35:32.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:32 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:32.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:32 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:32.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:32 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:32.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:32 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:32.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:32 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:32.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:32 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:32.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:32 smithi179 ceph-mon[31829]: Adjusting osd_memory_target on smithi179 to 4272M 2024-04-21T08:35:32.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:32 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:32.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:32 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:32.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:32 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:35:32.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:32 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:32.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:32 smithi086 ceph-mon[27939]: pgmap v255: 1 pgs: 1 active+clean; 577 KiB data, 360 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:32.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:32 smithi086 ceph-mon[27939]: Detected new or changed devices on smithi179 2024-04-21T08:35:32.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:32 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:32.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:32 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:32.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:32 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:32.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:32 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:32.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:32 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:32.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:32 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:35:32.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:32 smithi086 ceph-mon[27939]: Adjusting osd_memory_target on smithi179 to 4272M 2024-04-21T08:35:32.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:32 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:32.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:32 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:32.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:32 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:35:32.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:32 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:33.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.504+0000 7f8a3708b700 1 -- 172.21.15.86:0/3243705516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a30101a30 msgr2=0x7f8a301056d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:33.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.504+0000 7f8a3708b700 1 --2- 172.21.15.86:0/3243705516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a30101a30 0x7f8a301056d0 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f8a2800b0a0 tx=0x7f8a2800b3b0 comp rx=0 tx=0).stop 2024-04-21T08:35:33.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.504+0000 7f8a3708b700 1 -- 172.21.15.86:0/3243705516 shutdown_connections 2024-04-21T08:35:33.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.504+0000 7f8a3708b700 1 --2- 172.21.15.86:0/3243705516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a30101a30 0x7f8a301056d0 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:33.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.504+0000 7f8a3708b700 1 --2- 172.21.15.86:0/3243705516 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a30107bc0 0x7f8a30107fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:33.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.504+0000 7f8a3708b700 1 --2- 172.21.15.86:0/3243705516 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a30100f90 0x7f8a30101410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:33.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.504+0000 7f8a3708b700 1 -- 172.21.15.86:0/3243705516 >> 172.21.15.86:0/3243705516 conn(0x7f8a30077f40 msgr2=0x7f8a300ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:33.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.504+0000 7f8a3708b700 1 -- 172.21.15.86:0/3243705516 shutdown_connections 2024-04-21T08:35:33.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.504+0000 7f8a3708b700 1 -- 172.21.15.86:0/3243705516 wait complete. 2024-04-21T08:35:33.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.504+0000 7f8a3708b700 1 Processor -- start 2024-04-21T08:35:33.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.504+0000 7f8a3708b700 1 -- start start 2024-04-21T08:35:33.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.504+0000 7f8a3708b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a30100f90 0x7f8a301a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:33.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.504+0000 7f8a3708b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a30101a30 0x7f8a301a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:33.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.504+0000 7f8a3708b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a30107bc0 0x7f8a3019f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:33.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.504+0000 7f8a3708b700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8a301194c0 con 0x7f8a30100f90 2024-04-21T08:35:33.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.504+0000 7f8a3708b700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8a30119340 con 0x7f8a30107bc0 2024-04-21T08:35:33.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.504+0000 7f8a3708b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8a30119640 con 0x7f8a30101a30 2024-04-21T08:35:33.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.505+0000 7f8a34e27700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a30100f90 0x7f8a301a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:33.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.505+0000 7f8a34e27700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a30100f90 0x7f8a301a55a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40792/0 (socket says 172.21.15.86:40792) 2024-04-21T08:35:33.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.505+0000 7f8a34e27700 1 -- 172.21.15.86:0/1074955625 learned_addr learned my addr 172.21.15.86:0/1074955625 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:35:33.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.505+0000 7f8a2ffff700 1 --2- 172.21.15.86:0/1074955625 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a30101a30 0x7f8a301a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:33.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.505+0000 7f8a34e27700 1 -- 172.21.15.86:0/1074955625 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a30101a30 msgr2=0x7f8a301a5ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:33.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.505+0000 7f8a34e27700 1 --2- 172.21.15.86:0/1074955625 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a30101a30 0x7f8a301a5ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:33.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.505+0000 7f8a34e27700 1 -- 172.21.15.86:0/1074955625 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a30107bc0 msgr2=0x7f8a3019f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:33.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.505+0000 7f8a34e27700 1 --2- 172.21.15.86:0/1074955625 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a30107bc0 0x7f8a3019f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:33.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.505+0000 7f8a34e27700 1 -- 172.21.15.86:0/1074955625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a3019ffa0 con 0x7f8a30100f90 2024-04-21T08:35:33.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.505+0000 7f8a34e27700 1 --2- 172.21.15.86:0/1074955625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a30100f90 0x7f8a301a55a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8a2400c9b0 tx=0x7f8a2400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:33.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.505+0000 7f8a2dffb700 1 -- 172.21.15.86:0/1074955625 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a24007b00 con 0x7f8a30100f90 2024-04-21T08:35:33.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.505+0000 7f8a3708b700 1 -- 172.21.15.86:0/1074955625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a301a0290 con 0x7f8a30100f90 2024-04-21T08:35:33.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.505+0000 7f8a2dffb700 1 -- 172.21.15.86:0/1074955625 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a24007ca0 con 0x7f8a30100f90 2024-04-21T08:35:33.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.505+0000 7f8a2dffb700 1 -- 172.21.15.86:0/1074955625 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a24018660 con 0x7f8a30100f90 2024-04-21T08:35:33.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.505+0000 7f8a3708b700 1 -- 172.21.15.86:0/1074955625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a301ac520 con 0x7f8a30100f90 2024-04-21T08:35:33.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.506+0000 7f8a3708b700 1 -- 172.21.15.86:0/1074955625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8a30102240 con 0x7f8a30100f90 2024-04-21T08:35:33.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.507+0000 7f8a2dffb700 1 -- 172.21.15.86:0/1074955625 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8a2400f450 con 0x7f8a30100f90 2024-04-21T08:35:33.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.508+0000 7f8a2dffb700 1 --2- 172.21.15.86:0/1074955625 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f8a200a3c40 0x7f8a200a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:33.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.508+0000 7f8a2ffff700 1 --2- 172.21.15.86:0/1074955625 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f8a200a3c40 0x7f8a200a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:33.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.508+0000 7f8a2dffb700 1 -- 172.21.15.86:0/1074955625 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f8a240cff20 con 0x7f8a30100f90 2024-04-21T08:35:33.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.508+0000 7f8a2ffff700 1 --2- 172.21.15.86:0/1074955625 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f8a200a3c40 0x7f8a200a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8a301a0e30 tx=0x7f8a1c009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:33.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.512+0000 7f8a2dffb700 1 -- 172.21.15.86:0/1074955625 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8a24014030 con 0x7f8a30100f90 2024-04-21T08:35:33.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.707+0000 7f8a3708b700 1 -- 172.21.15.86:0/1074955625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f8a301ac6f0 con 0x7f8a30100f90 2024-04-21T08:35:33.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.708+0000 7f8a2dffb700 1 -- 172.21.15.86:0/1074955625 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18398 (secure 0 0 0) 0x7f8a240b5e20 con 0x7f8a30100f90 2024-04-21T08:35:33.707 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:35:33.708 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":81,"fsid":"f8cec7cc-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:22.687329+0000","modified":"2024-04-21T08:35:31.056297+0000","last_up_change":"2024-04-21T08:35:30.048315+0000","last_in_change":"2024-04-21T08:35:10.099496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:45.279109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"89b7601f-0620-4db2-90f1-b45cea173738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6803","nonce":2571516897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6805","nonce":2571516897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6809","nonce":2571516897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6807","nonce":2571516897}]},"public_addr":"172.21.15.86:6803/2571516897","cluster_addr":"172.21.15.86:6805/2571516897","heartbeat_back_addr":"172.21.15.86:6809/2571516897","heartbeat_front_addr":"172.21.15.86:6807/2571516897","state":["exists","up"]},{"osd":1,"uuid":"4891031c-f98c-42bc-bb3b-a420855ecc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6801","nonce":74067182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6803","nonce":74067182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6807","nonce":74067182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6805","nonce":74067182}]},"public_addr":"172.21.15.88:6801/74067182","cluster_addr":"172.21.15.88:6803/74067182","heartbeat_back_addr":"172.21.15.88:6807/74067182","heartbeat_front_addr":"172.21.15.88:6805/74067182","state":["exists","up"]},{"osd":2,"uuid":"60d12aa4-9312-4fa2-8d0b-2b3f42caad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6801","nonce":4137659051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6803","nonce":4137659051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6807","nonce":4137659051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6805","nonce":4137659051}]},"public_addr":"172.21.15.179:6801/4137659051","cluster_addr":"172.21.15.179:6803/4137659051","heartbeat_back_addr":"172.21.15.179:6807/4137659051","heartbeat_front_addr":"172.21.15.179:6805/4137659051","state":["exists","up"]},{"osd":3,"uuid":"404c3158-8cb0-4c02-b48a-f9ee5ff7b784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6811","nonce":2309603148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6813","nonce":2309603148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6817","nonce":2309603148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6815","nonce":2309603148}]},"public_addr":"172.21.15.86:6811/2309603148","cluster_addr":"172.21.15.86:6813/2309603148","heartbeat_back_addr":"172.21.15.86:6817/2309603148","heartbeat_front_addr":"172.21.15.86:6815/2309603148","state":["exists","up"]},{"osd":4,"uuid":"458ff202-d781-4ec7-ac93-0b49f835b503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6809","nonce":1818714217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6811","nonce":1818714217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6815","nonce":1818714217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6813","nonce":1818714217}]},"public_addr":"172.21.15.88:6809/1818714217","cluster_addr":"172.21.15.88:6811/1818714217","heartbeat_back_addr":"172.21.15.88:6815/1818714217","heartbeat_front_addr":"172.21.15.88:6813/1818714217","state":["exists","up"]},{"osd":5,"uuid":"8123cc90-bd8b-4204-bf31-46b34e9f589d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6809","nonce":863840990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6811","nonce":863840990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6815","nonce":863840990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6813","nonce":863840990}]},"public_addr":"172.21.15.179:6809/863840990","cluster_addr":"172.21.15.179:6811/863840990","heartbeat_back_addr":"172.21.15.179:6815/863840990","heartbeat_front_addr":"172.21.15.179:6813/863840990","state":["exists","up"]},{"osd":6,"uuid":"aac9d7c7-659a-4110-903a-a871c086066f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6819","nonce":1224231118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6821","nonce":1224231118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6825","nonce":1224231118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6823","nonce":1224231118}]},"public_addr":"172.21.15.86:6819/1224231118","cluster_addr":"172.21.15.86:6821/1224231118","heartbeat_back_addr":"172.21.15.86:6825/1224231118","heartbeat_front_addr":"172.21.15.86:6823/1224231118","state":["exists","up"]},{"osd":7,"uuid":"df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6817","nonce":136629878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6819","nonce":136629878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6823","nonce":136629878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6821","nonce":136629878}]},"public_addr":"172.21.15.88:6817/136629878","cluster_addr":"172.21.15.88:6819/136629878","heartbeat_back_addr":"172.21.15.88:6823/136629878","heartbeat_front_addr":"172.21.15.88:6821/136629878","state":["exists","up"]},{"osd":8,"uuid":"63e59f4b-0426-46db-92f7-c55dae61b5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6817","nonce":1770418957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6819","nonce":1770418957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6823","nonce":1770418957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6821","nonce":1770418957}]},"public_addr":"172.21.15.179:6817/1770418957","cluster_addr":"172.21.15.179:6819/1770418957","heartbeat_back_addr":"172.21.15.179:6823/1770418957","heartbeat_front_addr":"172.21.15.179:6821/1770418957","state":["exists","up"]},{"osd":9,"uuid":"5841bb36-ed6b-45d8-b857-eeab139c44e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6827","nonce":1108500274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6829","nonce":1108500274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6833","nonce":1108500274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6831","nonce":1108500274}]},"public_addr":"172.21.15.86:6827/1108500274","cluster_addr":"172.21.15.86:6829/1108500274","heartbeat_back_addr":"172.21.15.86:6833/1108500274","heartbeat_front_addr":"172.21.15.86:6831/1108500274","state":["exists","up"]},{"osd":10,"uuid":"f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6825","nonce":1294275353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6827","nonce":1294275353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6831","nonce":1294275353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6829","nonce":1294275353}]},"public_addr":"172.21.15.88:6825/1294275353","cluster_addr":"172.21.15.88:6827/1294275353","heartbeat_back_addr":"172.21.15.88:6831/1294275353","heartbeat_front_addr":"172.21.15.88:6829/1294275353","state":["exists","up"]},{"osd":11,"uuid":"e0d4cc76-e7a6-4a6b-909c-95488ccd5a36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6825","nonce":4007070233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6827","nonce":4007070233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6831","nonce":4007070233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6829","nonce":4007070233}]},"public_addr":"172.21.15.179:6825/4007070233","cluster_addr":"172.21.15.179:6827/4007070233","heartbeat_back_addr":"172.21.15.179:6831/4007070233","heartbeat_front_addr":"172.21.15.179:6829/4007070233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:55.210605+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:18.723090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:42.547979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:06.421622+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:30.858315+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:55.455313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.623369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:46.185111+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.103627+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:37.297545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:02.637431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:28.199292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6800/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/1412393225":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6800/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/1687852404":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2020707404":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2568130933":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/3952607648":"2024-04-22T08:28:57.384403+0000","172.21.15.86:6801/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/2756903426":"2024-04-22T08:29:23.190001+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:35:33.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.710+0000 7f8a3708b700 1 -- 172.21.15.86:0/1074955625 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f8a200a3c40 msgr2=0x7f8a200a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:33.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.710+0000 7f8a3708b700 1 --2- 172.21.15.86:0/1074955625 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f8a200a3c40 0x7f8a200a6100 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7f8a301a0e30 tx=0x7f8a1c009190 comp rx=0 tx=0).stop 2024-04-21T08:35:33.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.710+0000 7f8a3708b700 1 -- 172.21.15.86:0/1074955625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a30100f90 msgr2=0x7f8a301a55a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:33.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.710+0000 7f8a3708b700 1 --2- 172.21.15.86:0/1074955625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a30100f90 0x7f8a301a55a0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7f8a2400c9b0 tx=0x7f8a2400ccc0 comp rx=0 tx=0).stop 2024-04-21T08:35:33.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.710+0000 7f8a3708b700 1 -- 172.21.15.86:0/1074955625 shutdown_connections 2024-04-21T08:35:33.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.710+0000 7f8a3708b700 1 --2- 172.21.15.86:0/1074955625 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f8a200a3c40 0x7f8a200a6100 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:33.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.710+0000 7f8a3708b700 1 --2- 172.21.15.86:0/1074955625 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a30107bc0 0x7f8a3019f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:33.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.710+0000 7f8a3708b700 1 --2- 172.21.15.86:0/1074955625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a30100f90 0x7f8a301a55a0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:33.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.710+0000 7f8a3708b700 1 --2- 172.21.15.86:0/1074955625 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a30101a30 0x7f8a301a5ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:33.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.710+0000 7f8a3708b700 1 -- 172.21.15.86:0/1074955625 >> 172.21.15.86:0/1074955625 conn(0x7f8a30077f40 msgr2=0x7f8a30103530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:33.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.710+0000 7f8a3708b700 1 -- 172.21.15.86:0/1074955625 shutdown_connections 2024-04-21T08:35:33.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:33.710+0000 7f8a3708b700 1 -- 172.21.15.86:0/1074955625 wait complete. 2024-04-21T08:35:34.551 INFO:tasks.cephadm.ceph_manager.ceph:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:31:45.279109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-21T08:35:34.552 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph osd pool get .mgr pg_num 2024-04-21T08:35:34.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:34 smithi088 ceph-mon[31259]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:34.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:34 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1074955625' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:35:34.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:34 smithi179 ceph-mon[31829]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:34.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:34 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1074955625' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:35:34.789 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:35:34.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:34 smithi086 ceph-mon[27939]: pgmap v256: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:34.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:34 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1074955625' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:35:36.646 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:36 smithi086 ceph-mon[27939]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:36.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.646+0000 7f4740e29700 1 -- 172.21.15.86:0/100691197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f473c108a50 msgr2=0x7f473c108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:36.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.646+0000 7f4740e29700 1 --2- 172.21.15.86:0/100691197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f473c108a50 0x7f473c108e30 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f472c009990 tx=0x7f472c009ca0 comp rx=0 tx=0).stop 2024-04-21T08:35:36.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.647+0000 7f4740e29700 1 -- 172.21.15.86:0/100691197 shutdown_connections 2024-04-21T08:35:36.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.647+0000 7f4740e29700 1 --2- 172.21.15.86:0/100691197 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f473c103450 0x7f473c10f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:36.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.647+0000 7f4740e29700 1 --2- 172.21.15.86:0/100691197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f473c108a50 0x7f473c108e30 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:36.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.647+0000 7f4740e29700 1 --2- 172.21.15.86:0/100691197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f473c102a90 0x7f473c102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:36.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.647+0000 7f4740e29700 1 -- 172.21.15.86:0/100691197 >> 172.21.15.86:0/100691197 conn(0x7f473c079ba0 msgr2=0x7f473c079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:36.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.647+0000 7f4740e29700 1 -- 172.21.15.86:0/100691197 shutdown_connections 2024-04-21T08:35:36.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.647+0000 7f4740e29700 1 -- 172.21.15.86:0/100691197 wait complete. 2024-04-21T08:35:36.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.647+0000 7f4740e29700 1 Processor -- start 2024-04-21T08:35:36.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.648+0000 7f4740e29700 1 -- start start 2024-04-21T08:35:36.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.648+0000 7f4740e29700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f473c102a90 0x7f473c199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:36.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.648+0000 7f4740e29700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f473c103450 0x7f473c199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:36.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.648+0000 7f4740e29700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f473c108a50 0x7f473c19d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:36.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.648+0000 7f4740e29700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f473c112c00 con 0x7f473c108a50 2024-04-21T08:35:36.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.648+0000 7f4740e29700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f473c112a80 con 0x7f473c103450 2024-04-21T08:35:36.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.648+0000 7f4740e29700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f473c112d80 con 0x7f473c102a90 2024-04-21T08:35:36.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.648+0000 7f473ad9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f473c108a50 0x7f473c19d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:36.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.648+0000 7f4739d9b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f473c103450 0x7f473c199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:36.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.648+0000 7f473ad9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f473c108a50 0x7f473c19d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40798/0 (socket says 172.21.15.86:40798) 2024-04-21T08:35:36.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.648+0000 7f473ad9d700 1 -- 172.21.15.86:0/1707969653 learned_addr learned my addr 172.21.15.86:0/1707969653 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:35:36.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.648+0000 7f473a59c700 1 --2- 172.21.15.86:0/1707969653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f473c102a90 0x7f473c199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:36.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.648+0000 7f473ad9d700 1 -- 172.21.15.86:0/1707969653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f473c102a90 msgr2=0x7f473c199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:36.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.648+0000 7f473ad9d700 1 --2- 172.21.15.86:0/1707969653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f473c102a90 0x7f473c199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:36.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.648+0000 7f473ad9d700 1 -- 172.21.15.86:0/1707969653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f473c103450 msgr2=0x7f473c199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:36.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.648+0000 7f473ad9d700 1 --2- 172.21.15.86:0/1707969653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f473c103450 0x7f473c199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:36.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.648+0000 7f473ad9d700 1 -- 172.21.15.86:0/1707969653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f473c19e120 con 0x7f473c108a50 2024-04-21T08:35:36.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.648+0000 7f4739d9b700 1 --2- 172.21.15.86:0/1707969653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f473c103450 0x7f473c199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:35:36.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.648+0000 7f473ad9d700 1 --2- 172.21.15.86:0/1707969653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f473c108a50 0x7f473c19d9e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f473400ef30 tx=0x7f473400c3a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:36.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.648+0000 7f472b7fe700 1 -- 172.21.15.86:0/1707969653 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47340092e0 con 0x7f473c108a50 2024-04-21T08:35:36.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.648+0000 7f4740e29700 1 -- 172.21.15.86:0/1707969653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f473c19e3b0 con 0x7f473c108a50 2024-04-21T08:35:36.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.649+0000 7f472b7fe700 1 -- 172.21.15.86:0/1707969653 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4734004bb0 con 0x7f473c108a50 2024-04-21T08:35:36.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.649+0000 7f472b7fe700 1 -- 172.21.15.86:0/1707969653 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4734004020 con 0x7f473c108a50 2024-04-21T08:35:36.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.649+0000 7f4740e29700 1 -- 172.21.15.86:0/1707969653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f473c1a5c90 con 0x7f473c108a50 2024-04-21T08:35:36.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.650+0000 7f4740e29700 1 -- 172.21.15.86:0/1707969653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f473c069730 con 0x7f473c108a50 2024-04-21T08:35:36.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.651+0000 7f472b7fe700 1 -- 172.21.15.86:0/1707969653 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f473401b030 con 0x7f473c108a50 2024-04-21T08:35:36.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.651+0000 7f472b7fe700 1 --2- 172.21.15.86:0/1707969653 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f47240a3c40 0x7f47240a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:36.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.651+0000 7f473a59c700 1 --2- 172.21.15.86:0/1707969653 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f47240a3c40 0x7f47240a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:36.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.651+0000 7f472b7fe700 1 -- 172.21.15.86:0/1707969653 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f47340d0340 con 0x7f473c108a50 2024-04-21T08:35:36.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.652+0000 7f473a59c700 1 --2- 172.21.15.86:0/1707969653 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f47240a3c40 0x7f47240a6100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f472c00b2e0 tx=0x7f472c005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:36.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.656+0000 7f472b7fe700 1 -- 172.21.15.86:0/1707969653 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4734022020 con 0x7f473c108a50 2024-04-21T08:35:36.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:36 smithi088 ceph-mon[31259]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:36.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:36 smithi179 ceph-mon[31829]: pgmap v257: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:36.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.850+0000 7f4740e29700 1 -- 172.21.15.86:0/1707969653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f473c02df80 con 0x7f473c108a50 2024-04-21T08:35:36.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.851+0000 7f472b7fe700 1 -- 172.21.15.86:0/1707969653 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f4734010040 con 0x7f473c108a50 2024-04-21T08:35:36.850 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-21T08:35:36.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.853+0000 7f4740e29700 1 -- 172.21.15.86:0/1707969653 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f47240a3c40 msgr2=0x7f47240a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:36.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.853+0000 7f4740e29700 1 --2- 172.21.15.86:0/1707969653 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f47240a3c40 0x7f47240a6100 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f472c00b2e0 tx=0x7f472c005960 comp rx=0 tx=0).stop 2024-04-21T08:35:36.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.853+0000 7f4740e29700 1 -- 172.21.15.86:0/1707969653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f473c108a50 msgr2=0x7f473c19d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:36.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.853+0000 7f4740e29700 1 --2- 172.21.15.86:0/1707969653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f473c108a50 0x7f473c19d9e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f473400ef30 tx=0x7f473400c3a0 comp rx=0 tx=0).stop 2024-04-21T08:35:36.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.853+0000 7f4740e29700 1 -- 172.21.15.86:0/1707969653 shutdown_connections 2024-04-21T08:35:36.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.853+0000 7f4740e29700 1 --2- 172.21.15.86:0/1707969653 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f47240a3c40 0x7f47240a6100 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:36.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.853+0000 7f4740e29700 1 --2- 172.21.15.86:0/1707969653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f473c108a50 0x7f473c19d9e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:36.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.853+0000 7f4740e29700 1 --2- 172.21.15.86:0/1707969653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f473c102a90 0x7f473c199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:36.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.853+0000 7f4740e29700 1 --2- 172.21.15.86:0/1707969653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f473c103450 0x7f473c199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:36.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.853+0000 7f4740e29700 1 -- 172.21.15.86:0/1707969653 >> 172.21.15.86:0/1707969653 conn(0x7f473c079ba0 msgr2=0x7f473c101930 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:36.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.853+0000 7f4740e29700 1 -- 172.21.15.86:0/1707969653 shutdown_connections 2024-04-21T08:35:36.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:36.853+0000 7f4740e29700 1 -- 172.21.15.86:0/1707969653 wait complete. 2024-04-21T08:35:37.559 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:37 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1707969653' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:35:37.586 INFO:tasks.cephadm:Adding mds.a on smithi086 2024-04-21T08:35:37.586 INFO:tasks.cephadm:Adding mds.d on smithi086 2024-04-21T08:35:37.586 INFO:tasks.cephadm:Adding mds.g on smithi086 2024-04-21T08:35:37.586 INFO:tasks.cephadm:Adding mds.j on smithi086 2024-04-21T08:35:37.586 INFO:tasks.cephadm:Adding mds.b on smithi088 2024-04-21T08:35:37.586 INFO:tasks.cephadm:Adding mds.e on smithi088 2024-04-21T08:35:37.586 INFO:tasks.cephadm:Adding mds.h on smithi088 2024-04-21T08:35:37.586 INFO:tasks.cephadm:Adding mds.k on smithi088 2024-04-21T08:35:37.587 INFO:tasks.cephadm:Adding mds.c on smithi179 2024-04-21T08:35:37.587 INFO:tasks.cephadm:Adding mds.f on smithi179 2024-04-21T08:35:37.587 INFO:tasks.cephadm:Adding mds.i on smithi179 2024-04-21T08:35:37.587 INFO:tasks.cephadm:Adding mds.l on smithi179 2024-04-21T08:35:37.587 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph orch apply mds all '12;smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi088=b;smithi088=e;smithi088=h;smithi088=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l' 2024-04-21T08:35:37.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:37 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1707969653' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:35:37.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:37 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1707969653' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:35:37.837 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:35:38.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:38 smithi179 ceph-mon[31829]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:38.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:38 smithi088 ceph-mon[31259]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:38.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:38 smithi086 ceph-mon[27939]: pgmap v258: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:39.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.520+0000 7f8c91cb3700 1 -- 172.21.15.179:0/3625373457 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c8c103570 msgr2=0x7f8c8c105960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:39.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.520+0000 7f8c91cb3700 1 --2- 172.21.15.179:0/3625373457 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c8c103570 0x7f8c8c105960 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c009990 tx=0x7f8c7c009ca0 comp rx=0 tx=0).stop 2024-04-21T08:35:39.521 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.520+0000 7f8c91cb3700 1 -- 172.21.15.179:0/3625373457 shutdown_connections 2024-04-21T08:35:39.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.520+0000 7f8c91cb3700 1 --2- 172.21.15.179:0/3625373457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c8c1087d0 0x7f8c8c10ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:39.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.520+0000 7f8c91cb3700 1 --2- 172.21.15.179:0/3625373457 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c8c103570 0x7f8c8c105960 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:39.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.520+0000 7f8c91cb3700 1 --2- 172.21.15.179:0/3625373457 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c8c105ea0 0x7f8c8c108290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:39.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.520+0000 7f8c91cb3700 1 -- 172.21.15.179:0/3625373457 >> 172.21.15.179:0/3625373457 conn(0x7f8c8c0fcfa0 msgr2=0x7f8c8c0ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:39.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.520+0000 7f8c91cb3700 1 -- 172.21.15.179:0/3625373457 shutdown_connections 2024-04-21T08:35:39.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.520+0000 7f8c91cb3700 1 -- 172.21.15.179:0/3625373457 wait complete. 2024-04-21T08:35:39.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.520+0000 7f8c91cb3700 1 Processor -- start 2024-04-21T08:35:39.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.521+0000 7f8c91cb3700 1 -- start start 2024-04-21T08:35:39.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.521+0000 7f8c91cb3700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c8c103570 0x7f8c8c198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:39.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.521+0000 7f8c91cb3700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c8c105ea0 0x7f8c8c1991b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:39.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.521+0000 7f8c91cb3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c8c1087d0 0x7f8c8c1a01f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:39.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.521+0000 7f8c91cb3700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8c10e580 con 0x7f8c8c105ea0 2024-04-21T08:35:39.522 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.521+0000 7f8c91cb3700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8c10e400 con 0x7f8c8c103570 2024-04-21T08:35:39.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.521+0000 7f8c91cb3700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8c8c10e700 con 0x7f8c8c1087d0 2024-04-21T08:35:39.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.521+0000 7f8c8bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c8c1087d0 0x7f8c8c1a01f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:39.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.521+0000 7f8c8bfff700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c8c1087d0 0x7f8c8c1a01f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.179:41232/0 (socket says 172.21.15.179:41232) 2024-04-21T08:35:39.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.521+0000 7f8c8bfff700 1 -- 172.21.15.179:0/2118584052 learned_addr learned my addr 172.21.15.179:0/2118584052 (peer_addr_for_me v2:172.21.15.179:0/0) 2024-04-21T08:35:39.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.521+0000 7f8c8affd700 1 --2- 172.21.15.179:0/2118584052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c8c105ea0 0x7f8c8c1991b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:39.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.521+0000 7f8c8b7fe700 1 --2- 172.21.15.179:0/2118584052 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c8c103570 0x7f8c8c198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:39.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.521+0000 7f8c8bfff700 1 -- 172.21.15.179:0/2118584052 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c8c103570 msgr2=0x7f8c8c198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:39.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.521+0000 7f8c8bfff700 1 --2- 172.21.15.179:0/2118584052 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c8c103570 0x7f8c8c198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:39.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.521+0000 7f8c8bfff700 1 -- 172.21.15.179:0/2118584052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c8c105ea0 msgr2=0x7f8c8c1991b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:39.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.521+0000 7f8c8bfff700 1 --2- 172.21.15.179:0/2118584052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c8c105ea0 0x7f8c8c1991b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:39.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.521+0000 7f8c8bfff700 1 -- 172.21.15.179:0/2118584052 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c8c1a08c0 con 0x7f8c8c1087d0 2024-04-21T08:35:39.523 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.521+0000 7f8c8bfff700 1 --2- 172.21.15.179:0/2118584052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c8c1087d0 0x7f8c8c1a01f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8c840094f0 tx=0x7f8c8400d3b0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:39.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.521+0000 7f8c88ff9700 1 -- 172.21.15.179:0/2118584052 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c84004db0 con 0x7f8c8c1087d0 2024-04-21T08:35:39.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.521+0000 7f8c91cb3700 1 -- 172.21.15.179:0/2118584052 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c8c1a0bb0 con 0x7f8c8c1087d0 2024-04-21T08:35:39.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.521+0000 7f8c88ff9700 1 -- 172.21.15.179:0/2118584052 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8c84009e20 con 0x7f8c8c1087d0 2024-04-21T08:35:39.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.521+0000 7f8c88ff9700 1 -- 172.21.15.179:0/2118584052 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8c84015400 con 0x7f8c8c1087d0 2024-04-21T08:35:39.524 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.522+0000 7f8c91cb3700 1 -- 172.21.15.179:0/2118584052 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c8c1a10f0 con 0x7f8c8c1087d0 2024-04-21T08:35:39.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.523+0000 7f8c91cb3700 1 -- 172.21.15.179:0/2118584052 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c8c069730 con 0x7f8c8c1087d0 2024-04-21T08:35:39.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.524+0000 7f8c88ff9700 1 -- 172.21.15.179:0/2118584052 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8c84005260 con 0x7f8c8c1087d0 2024-04-21T08:35:39.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.524+0000 7f8c88ff9700 1 --2- 172.21.15.179:0/2118584052 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f8c740a3c40 0x7f8c740a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:39.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.524+0000 7f8c88ff9700 1 -- 172.21.15.179:0/2118584052 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f8c840cf970 con 0x7f8c8c1087d0 2024-04-21T08:35:39.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.524+0000 7f8c8b7fe700 1 --2- 172.21.15.179:0/2118584052 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f8c740a3c40 0x7f8c740a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:39.525 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.525+0000 7f8c8b7fe700 1 --2- 172.21.15.179:0/2118584052 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f8c740a3c40 0x7f8c740a6100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c00b5c0 tx=0x7f8c7c005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:39.528 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.529+0000 7f8c88ff9700 1 -- 172.21.15.179:0/2118584052 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8c84010070 con 0x7f8c8c1087d0 2024-04-21T08:35:39.735 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.736+0000 7f8c91cb3700 1 -- 172.21.15.179:0/2118584052 --> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] -- mgr_command(tid 0: {"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi088=b;smithi088=e;smithi088=h;smithi088=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l", "target": ["mon-mgr", ""]}) -- 0x7f8c8c063140 con 0x7f8c740a3c40 2024-04-21T08:35:39.744 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.744+0000 7f8c88ff9700 1 -- 172.21.15.179:0/2118584052 <== mgr.14152 v2:172.21.15.86:6800/1364156965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+28 (secure 0 0 0) 0x7f8c8c063140 con 0x7f8c740a3c40 2024-04-21T08:35:39.744 INFO:teuthology.orchestra.run.smithi179.stdout:Scheduled mds.all update... 2024-04-21T08:35:39.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.746+0000 7f8c91cb3700 1 -- 172.21.15.179:0/2118584052 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f8c740a3c40 msgr2=0x7f8c740a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:39.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.746+0000 7f8c91cb3700 1 --2- 172.21.15.179:0/2118584052 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f8c740a3c40 0x7f8c740a6100 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7f8c7c00b5c0 tx=0x7f8c7c005fb0 comp rx=0 tx=0).stop 2024-04-21T08:35:39.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.746+0000 7f8c91cb3700 1 -- 172.21.15.179:0/2118584052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c8c1087d0 msgr2=0x7f8c8c1a01f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:39.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.746+0000 7f8c91cb3700 1 --2- 172.21.15.179:0/2118584052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c8c1087d0 0x7f8c8c1a01f0 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f8c840094f0 tx=0x7f8c8400d3b0 comp rx=0 tx=0).stop 2024-04-21T08:35:39.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.746+0000 7f8c91cb3700 1 -- 172.21.15.179:0/2118584052 shutdown_connections 2024-04-21T08:35:39.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.746+0000 7f8c91cb3700 1 --2- 172.21.15.179:0/2118584052 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f8c740a3c40 0x7f8c740a6100 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:39.746 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.746+0000 7f8c91cb3700 1 --2- 172.21.15.179:0/2118584052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c8c1087d0 0x7f8c8c1a01f0 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:39.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.746+0000 7f8c91cb3700 1 --2- 172.21.15.179:0/2118584052 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c8c103570 0x7f8c8c198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:39.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.746+0000 7f8c91cb3700 1 --2- 172.21.15.179:0/2118584052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c8c105ea0 0x7f8c8c1991b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:39.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.746+0000 7f8c91cb3700 1 -- 172.21.15.179:0/2118584052 >> 172.21.15.179:0/2118584052 conn(0x7f8c8c0fcfa0 msgr2=0x7f8c8c103ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:39.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.747+0000 7f8c91cb3700 1 -- 172.21.15.179:0/2118584052 shutdown_connections 2024-04-21T08:35:39.747 INFO:teuthology.orchestra.run.smithi179.stderr:2024-04-21T08:35:39.747+0000 7f8c91cb3700 1 -- 172.21.15.179:0/2118584052 wait complete. 2024-04-21T08:35:40.392 DEBUG:teuthology.orchestra.run.smithi086:mds.a> sudo journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mds.a.service 2024-04-21T08:35:40.395 DEBUG:teuthology.orchestra.run.smithi086:mds.d> sudo journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mds.d.service 2024-04-21T08:35:40.438 DEBUG:teuthology.orchestra.run.smithi086:mds.g> sudo journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mds.g.service 2024-04-21T08:35:40.480 DEBUG:teuthology.orchestra.run.smithi086:mds.j> sudo journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mds.j.service 2024-04-21T08:35:40.524 DEBUG:teuthology.orchestra.run.smithi088:mds.b> sudo journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mds.b.service 2024-04-21T08:35:40.526 DEBUG:teuthology.orchestra.run.smithi088:mds.e> sudo journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mds.e.service 2024-04-21T08:35:40.569 DEBUG:teuthology.orchestra.run.smithi088:mds.h> sudo journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mds.h.service 2024-04-21T08:35:40.612 DEBUG:teuthology.orchestra.run.smithi088:mds.k> sudo journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mds.k.service 2024-04-21T08:35:40.655 DEBUG:teuthology.orchestra.run.smithi179:mds.c> sudo journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mds.c.service 2024-04-21T08:35:40.657 DEBUG:teuthology.orchestra.run.smithi179:mds.f> sudo journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mds.f.service 2024-04-21T08:35:40.676 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:40 smithi179 ceph-mon[31829]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:40.676 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:40 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:40.676 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:40 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:35:40.676 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:40 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:40.677 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:40 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:35:40.677 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:40 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:40.677 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:40 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:35:40.677 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:40 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:35:40.677 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:40 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:40.677 DEBUG:teuthology.orchestra.run.smithi179:mds.i> sudo journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mds.i.service 2024-04-21T08:35:40.720 DEBUG:teuthology.orchestra.run.smithi179:mds.l> sudo journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mds.l.service 2024-04-21T08:35:40.763 INFO:tasks.cephadm:Setting up CephFS filesystem(s)... 2024-04-21T08:35:40.764 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:35:40.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:40 smithi086 ceph-mon[27939]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:40.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:40 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:40.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:40 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:35:40.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:40 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:40.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:40 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:35:40.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:40 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:40.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:40 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:35:40.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:40 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:35:40.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:40 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:40.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.872+0000 7f377f152640 1 -- 172.21.15.86:0/840990750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3778102990 msgr2=0x7f37781092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:40.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.872+0000 7f377f152640 1 --2- 172.21.15.86:0/840990750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3778102990 0x7f37781092b0 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f3770009ca0 tx=0x7f377002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:35:40.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.872+0000 7f377f152640 1 -- 172.21.15.86:0/840990750 shutdown_connections 2024-04-21T08:35:40.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.872+0000 7f377f152640 1 --2- 172.21.15.86:0/840990750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3778102990 0x7f37781092b0 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:40.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.872+0000 7f377f152640 1 --2- 172.21.15.86:0/840990750 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3778101fd0 0x7f3778102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:40.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.872+0000 7f377f152640 1 --2- 172.21.15.86:0/840990750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3778100e80 0x7f37781012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:40.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.872+0000 7f377f152640 1 -- 172.21.15.86:0/840990750 >> 172.21.15.86:0/840990750 conn(0x7f37780fc410 msgr2=0x7f37780fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:40.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.872+0000 7f377f152640 1 -- 172.21.15.86:0/840990750 shutdown_connections 2024-04-21T08:35:40.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.872+0000 7f377f152640 1 -- 172.21.15.86:0/840990750 wait complete. 2024-04-21T08:35:40.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.873+0000 7f377f152640 1 Processor -- start 2024-04-21T08:35:40.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.873+0000 7f377f152640 1 -- start start 2024-04-21T08:35:40.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.873+0000 7f377f152640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3778100e80 0x7f377806b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:40.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.873+0000 7f377f152640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3778101fd0 0x7f377819ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:40.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.873+0000 7f377f152640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3778102990 0x7f377819f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:40.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.873+0000 7f377f152640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f377810c9c0 con 0x7f3778101fd0 2024-04-21T08:35:40.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.873+0000 7f377f152640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f377810c840 con 0x7f3778102990 2024-04-21T08:35:40.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.873+0000 7f377f152640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f377810cb40 con 0x7f3778100e80 2024-04-21T08:35:40.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.873+0000 7f3777fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3778101fd0 0x7f377819ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:40.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.873+0000 7f3777fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3778101fd0 0x7f377819ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55412/0 (socket says 172.21.15.86:55412) 2024-04-21T08:35:40.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.873+0000 7f3777fff640 1 -- 172.21.15.86:0/1801236344 learned_addr learned my addr 172.21.15.86:0/1801236344 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:35:40.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.873+0000 7f377ceee640 1 --2- 172.21.15.86:0/1801236344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3778100e80 0x7f377806b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:40.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.873+0000 7f377d6ef640 1 --2- 172.21.15.86:0/1801236344 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3778102990 0x7f377819f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:40.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.873+0000 7f3777fff640 1 -- 172.21.15.86:0/1801236344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3778100e80 msgr2=0x7f377806b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:40.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.873+0000 7f3777fff640 1 --2- 172.21.15.86:0/1801236344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3778100e80 0x7f377806b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:40.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.873+0000 7f3777fff640 1 -- 172.21.15.86:0/1801236344 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3778102990 msgr2=0x7f377819f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:40.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.873+0000 7f3777fff640 1 --2- 172.21.15.86:0/1801236344 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3778102990 0x7f377819f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:40.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.873+0000 7f3777fff640 1 -- 172.21.15.86:0/1801236344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f377819fd90 con 0x7f3778101fd0 2024-04-21T08:35:40.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.874+0000 7f3777fff640 1 --2- 172.21.15.86:0/1801236344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3778101fd0 0x7f377819ce60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f376400d950 tx=0x7f376400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:40.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.874+0000 7f3775ffb640 1 -- 172.21.15.86:0/1801236344 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3764017020 con 0x7f3778101fd0 2024-04-21T08:35:40.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.874+0000 7f3775ffb640 1 -- 172.21.15.86:0/1801236344 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3764009c30 con 0x7f3778101fd0 2024-04-21T08:35:40.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.874+0000 7f377f152640 1 -- 172.21.15.86:0/1801236344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f377819ffc0 con 0x7f3778101fd0 2024-04-21T08:35:40.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.874+0000 7f3775ffb640 1 -- 172.21.15.86:0/1801236344 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f376400f6b0 con 0x7f3778101fd0 2024-04-21T08:35:40.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.874+0000 7f377f152640 1 -- 172.21.15.86:0/1801236344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37781a0530 con 0x7f3778101fd0 2024-04-21T08:35:40.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.875+0000 7f377f152640 1 -- 172.21.15.86:0/1801236344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3778108190 con 0x7f3778101fd0 2024-04-21T08:35:40.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.876+0000 7f3775ffb640 1 -- 172.21.15.86:0/1801236344 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f376401a020 con 0x7f3778101fd0 2024-04-21T08:35:40.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.876+0000 7f3775ffb640 1 --2- 172.21.15.86:0/1801236344 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f37680a3c30 0x7f37680a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:40.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.876+0000 7f377ceee640 1 --2- 172.21.15.86:0/1801236344 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f37680a3c30 0x7f37680a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:40.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.877+0000 7f3775ffb640 1 -- 172.21.15.86:0/1801236344 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f37640cfaf0 con 0x7f3778101fd0 2024-04-21T08:35:40.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.877+0000 7f377ceee640 1 --2- 172.21.15.86:0/1801236344 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f37680a3c30 0x7f37680a60f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f376c00bbe0 tx=0x7f376c00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:40.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:40.881+0000 7f3775ffb640 1 -- 172.21.15.86:0/1801236344 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3764009de0 con 0x7f3778101fd0 2024-04-21T08:35:40.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:40 smithi088 ceph-mon[31259]: pgmap v259: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:40.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:40 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:40.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:40 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:35:40.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:40 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:40.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:40 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:35:40.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:40 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:40.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:40 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:35:40.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:40 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.l", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:35:40.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:40 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:41.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.063+0000 7f377f152640 1 -- 172.21.15.86:0/1801236344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f377802d830 con 0x7f3778101fd0 2024-04-21T08:35:41.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.064+0000 7f3775ffb640 1 -- 172.21.15.86:0/1801236344 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18398 (secure 0 0 0) 0x7f376409c970 con 0x7f3778101fd0 2024-04-21T08:35:41.063 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:35:41.063 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":81,"fsid":"f8cec7cc-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:22.687329+0000","modified":"2024-04-21T08:35:31.056297+0000","last_up_change":"2024-04-21T08:35:30.048315+0000","last_in_change":"2024-04-21T08:35:10.099496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:45.279109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"89b7601f-0620-4db2-90f1-b45cea173738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6803","nonce":2571516897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6805","nonce":2571516897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6809","nonce":2571516897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6807","nonce":2571516897}]},"public_addr":"172.21.15.86:6803/2571516897","cluster_addr":"172.21.15.86:6805/2571516897","heartbeat_back_addr":"172.21.15.86:6809/2571516897","heartbeat_front_addr":"172.21.15.86:6807/2571516897","state":["exists","up"]},{"osd":1,"uuid":"4891031c-f98c-42bc-bb3b-a420855ecc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6801","nonce":74067182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6803","nonce":74067182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6807","nonce":74067182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6805","nonce":74067182}]},"public_addr":"172.21.15.88:6801/74067182","cluster_addr":"172.21.15.88:6803/74067182","heartbeat_back_addr":"172.21.15.88:6807/74067182","heartbeat_front_addr":"172.21.15.88:6805/74067182","state":["exists","up"]},{"osd":2,"uuid":"60d12aa4-9312-4fa2-8d0b-2b3f42caad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6801","nonce":4137659051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6803","nonce":4137659051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6807","nonce":4137659051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6805","nonce":4137659051}]},"public_addr":"172.21.15.179:6801/4137659051","cluster_addr":"172.21.15.179:6803/4137659051","heartbeat_back_addr":"172.21.15.179:6807/4137659051","heartbeat_front_addr":"172.21.15.179:6805/4137659051","state":["exists","up"]},{"osd":3,"uuid":"404c3158-8cb0-4c02-b48a-f9ee5ff7b784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6811","nonce":2309603148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6813","nonce":2309603148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6817","nonce":2309603148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6815","nonce":2309603148}]},"public_addr":"172.21.15.86:6811/2309603148","cluster_addr":"172.21.15.86:6813/2309603148","heartbeat_back_addr":"172.21.15.86:6817/2309603148","heartbeat_front_addr":"172.21.15.86:6815/2309603148","state":["exists","up"]},{"osd":4,"uuid":"458ff202-d781-4ec7-ac93-0b49f835b503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6809","nonce":1818714217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6811","nonce":1818714217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6815","nonce":1818714217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6813","nonce":1818714217}]},"public_addr":"172.21.15.88:6809/1818714217","cluster_addr":"172.21.15.88:6811/1818714217","heartbeat_back_addr":"172.21.15.88:6815/1818714217","heartbeat_front_addr":"172.21.15.88:6813/1818714217","state":["exists","up"]},{"osd":5,"uuid":"8123cc90-bd8b-4204-bf31-46b34e9f589d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6809","nonce":863840990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6811","nonce":863840990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6815","nonce":863840990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6813","nonce":863840990}]},"public_addr":"172.21.15.179:6809/863840990","cluster_addr":"172.21.15.179:6811/863840990","heartbeat_back_addr":"172.21.15.179:6815/863840990","heartbeat_front_addr":"172.21.15.179:6813/863840990","state":["exists","up"]},{"osd":6,"uuid":"aac9d7c7-659a-4110-903a-a871c086066f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6819","nonce":1224231118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6821","nonce":1224231118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6825","nonce":1224231118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6823","nonce":1224231118}]},"public_addr":"172.21.15.86:6819/1224231118","cluster_addr":"172.21.15.86:6821/1224231118","heartbeat_back_addr":"172.21.15.86:6825/1224231118","heartbeat_front_addr":"172.21.15.86:6823/1224231118","state":["exists","up"]},{"osd":7,"uuid":"df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6817","nonce":136629878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6819","nonce":136629878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6823","nonce":136629878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6821","nonce":136629878}]},"public_addr":"172.21.15.88:6817/136629878","cluster_addr":"172.21.15.88:6819/136629878","heartbeat_back_addr":"172.21.15.88:6823/136629878","heartbeat_front_addr":"172.21.15.88:6821/136629878","state":["exists","up"]},{"osd":8,"uuid":"63e59f4b-0426-46db-92f7-c55dae61b5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6817","nonce":1770418957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6819","nonce":1770418957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6823","nonce":1770418957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6821","nonce":1770418957}]},"public_addr":"172.21.15.179:6817/1770418957","cluster_addr":"172.21.15.179:6819/1770418957","heartbeat_back_addr":"172.21.15.179:6823/1770418957","heartbeat_front_addr":"172.21.15.179:6821/1770418957","state":["exists","up"]},{"osd":9,"uuid":"5841bb36-ed6b-45d8-b857-eeab139c44e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6827","nonce":1108500274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6829","nonce":1108500274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6833","nonce":1108500274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6831","nonce":1108500274}]},"public_addr":"172.21.15.86:6827/1108500274","cluster_addr":"172.21.15.86:6829/1108500274","heartbeat_back_addr":"172.21.15.86:6833/1108500274","heartbeat_front_addr":"172.21.15.86:6831/1108500274","state":["exists","up"]},{"osd":10,"uuid":"f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6825","nonce":1294275353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6827","nonce":1294275353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6831","nonce":1294275353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6829","nonce":1294275353}]},"public_addr":"172.21.15.88:6825/1294275353","cluster_addr":"172.21.15.88:6827/1294275353","heartbeat_back_addr":"172.21.15.88:6831/1294275353","heartbeat_front_addr":"172.21.15.88:6829/1294275353","state":["exists","up"]},{"osd":11,"uuid":"e0d4cc76-e7a6-4a6b-909c-95488ccd5a36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6825","nonce":4007070233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6827","nonce":4007070233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6831","nonce":4007070233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6829","nonce":4007070233}]},"public_addr":"172.21.15.179:6825/4007070233","cluster_addr":"172.21.15.179:6827/4007070233","heartbeat_back_addr":"172.21.15.179:6831/4007070233","heartbeat_front_addr":"172.21.15.179:6829/4007070233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:55.210605+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:18.723090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:42.547979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:06.421622+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:30.858315+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:55.455313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.623369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:46.185111+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.103627+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:37.297545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:02.637431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:28.199292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6800/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/1412393225":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6800/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/1687852404":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2020707404":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2568130933":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/3952607648":"2024-04-22T08:28:57.384403+0000","172.21.15.86:6801/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/2756903426":"2024-04-22T08:29:23.190001+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:35:41.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.066+0000 7f377f152640 1 -- 172.21.15.86:0/1801236344 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f37680a3c30 msgr2=0x7f37680a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:41.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.066+0000 7f377f152640 1 --2- 172.21.15.86:0/1801236344 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f37680a3c30 0x7f37680a60f0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7f376c00bbe0 tx=0x7f376c00b390 comp rx=0 tx=0).stop 2024-04-21T08:35:41.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.066+0000 7f377f152640 1 -- 172.21.15.86:0/1801236344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3778101fd0 msgr2=0x7f377819ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:41.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.066+0000 7f377f152640 1 --2- 172.21.15.86:0/1801236344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3778101fd0 0x7f377819ce60 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f376400d950 tx=0x7f376400de10 comp rx=0 tx=0).stop 2024-04-21T08:35:41.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.066+0000 7f377f152640 1 -- 172.21.15.86:0/1801236344 shutdown_connections 2024-04-21T08:35:41.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.066+0000 7f377f152640 1 --2- 172.21.15.86:0/1801236344 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f37680a3c30 0x7f37680a60f0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:41.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.066+0000 7f377f152640 1 --2- 172.21.15.86:0/1801236344 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3778102990 0x7f377819f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:41.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.066+0000 7f377f152640 1 --2- 172.21.15.86:0/1801236344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3778101fd0 0x7f377819ce60 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:41.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.066+0000 7f377f152640 1 --2- 172.21.15.86:0/1801236344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3778100e80 0x7f377806b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:41.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.066+0000 7f377f152640 1 -- 172.21.15.86:0/1801236344 >> 172.21.15.86:0/1801236344 conn(0x7f37780fc410 msgr2=0x7f3778105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:41.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.066+0000 7f377f152640 1 -- 172.21.15.86:0/1801236344 shutdown_connections 2024-04-21T08:35:41.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.066+0000 7f377f152640 1 -- 172.21.15.86:0/1801236344 wait complete. 2024-04-21T08:35:41.076 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:31:45.279109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-21T08:35:41.077 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:35:41.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.218+0000 7f21b57a4640 1 -- 172.21.15.86:0/4250942863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21b0102040 msgr2=0x7f21b01024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:41.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.218+0000 7f21b57a4640 1 --2- 172.21.15.86:0/4250942863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21b0102040 0x7f21b01024c0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f2198009900 tx=0x7f219802bc30 comp rx=0 tx=0).stop 2024-04-21T08:35:41.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.218+0000 7f21b57a4640 1 -- 172.21.15.86:0/4250942863 shutdown_connections 2024-04-21T08:35:41.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.218+0000 7f21b57a4640 1 --2- 172.21.15.86:0/4250942863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21b0102a00 0x7f21b0109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:41.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.218+0000 7f21b57a4640 1 --2- 172.21.15.86:0/4250942863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21b0102040 0x7f21b01024c0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:41.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.218+0000 7f21b57a4640 1 --2- 172.21.15.86:0/4250942863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21b0100e60 0x7f21b0101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:41.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.218+0000 7f21b57a4640 1 -- 172.21.15.86:0/4250942863 >> 172.21.15.86:0/4250942863 conn(0x7f21b00fc420 msgr2=0x7f21b00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:41.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.218+0000 7f21b57a4640 1 -- 172.21.15.86:0/4250942863 shutdown_connections 2024-04-21T08:35:41.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.218+0000 7f21b57a4640 1 -- 172.21.15.86:0/4250942863 wait complete. 2024-04-21T08:35:41.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.219+0000 7f21b57a4640 1 Processor -- start 2024-04-21T08:35:41.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.219+0000 7f21b57a4640 1 -- start start 2024-04-21T08:35:41.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.219+0000 7f21b57a4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21b0100e60 0x7f21b0197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:41.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.219+0000 7f21b57a4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21b0102040 0x7f21b0197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:41.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.219+0000 7f21b57a4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21b0102a00 0x7f21b019e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:41.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.219+0000 7f21b57a4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f21b010ca10 con 0x7f21b0102a00 2024-04-21T08:35:41.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.219+0000 7f21b57a4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f21b010c890 con 0x7f21b0100e60 2024-04-21T08:35:41.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.219+0000 7f21b57a4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f21b010cb90 con 0x7f21b0102040 2024-04-21T08:35:41.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.219+0000 7f21af7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21b0102a00 0x7f21b019e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:41.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.219+0000 7f21af7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21b0102a00 0x7f21b019e6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55434/0 (socket says 172.21.15.86:55434) 2024-04-21T08:35:41.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.219+0000 7f21af7fe640 1 -- 172.21.15.86:0/731335731 learned_addr learned my addr 172.21.15.86:0/731335731 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:35:41.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.219+0000 7f21a7fff640 1 --2- 172.21.15.86:0/731335731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21b0102040 0x7f21b0197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:41.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.219+0000 7f21aeffd640 1 --2- 172.21.15.86:0/731335731 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21b0100e60 0x7f21b0197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:41.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.219+0000 7f21af7fe640 1 -- 172.21.15.86:0/731335731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21b0102040 msgr2=0x7f21b0197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:41.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.219+0000 7f21af7fe640 1 --2- 172.21.15.86:0/731335731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21b0102040 0x7f21b0197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:41.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.219+0000 7f21af7fe640 1 -- 172.21.15.86:0/731335731 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21b0100e60 msgr2=0x7f21b0197140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:41.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.219+0000 7f21af7fe640 1 --2- 172.21.15.86:0/731335731 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21b0100e60 0x7f21b0197140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:41.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.219+0000 7f21af7fe640 1 -- 172.21.15.86:0/731335731 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21b019ed60 con 0x7f21b0102a00 2024-04-21T08:35:41.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.219+0000 7f21af7fe640 1 --2- 172.21.15.86:0/731335731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21b0102a00 0x7f21b019e6f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f21a8009fd0 tx=0x7f21a8007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:41.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.219+0000 7f21acff9640 1 -- 172.21.15.86:0/731335731 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a800de10 con 0x7f21b0102a00 2024-04-21T08:35:41.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.219+0000 7f21acff9640 1 -- 172.21.15.86:0/731335731 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21a8003e20 con 0x7f21b0102a00 2024-04-21T08:35:41.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.219+0000 7f21b57a4640 1 -- 172.21.15.86:0/731335731 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21b006a5b0 con 0x7f21b0102a00 2024-04-21T08:35:41.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.219+0000 7f21acff9640 1 -- 172.21.15.86:0/731335731 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21a8004810 con 0x7f21b0102a00 2024-04-21T08:35:41.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.219+0000 7f21b57a4640 1 -- 172.21.15.86:0/731335731 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21b006aa80 con 0x7f21b0102a00 2024-04-21T08:35:41.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.220+0000 7f21b57a4640 1 -- 172.21.15.86:0/731335731 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21b00691c0 con 0x7f21b0102a00 2024-04-21T08:35:41.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.222+0000 7f21acff9640 1 -- 172.21.15.86:0/731335731 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f21a8003970 con 0x7f21b0102a00 2024-04-21T08:35:41.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.222+0000 7f21acff9640 1 --2- 172.21.15.86:0/731335731 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f219c0a3d50 0x7f219c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:41.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.222+0000 7f21aeffd640 1 --2- 172.21.15.86:0/731335731 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f219c0a3d50 0x7f219c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:41.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.222+0000 7f21acff9640 1 -- 172.21.15.86:0/731335731 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f21a80d1ad0 con 0x7f21b0102a00 2024-04-21T08:35:41.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.223+0000 7f21aeffd640 1 --2- 172.21.15.86:0/731335731 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f219c0a3d50 0x7f219c0a6210 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f21a0005fd0 tx=0x7f21a000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:41.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.226+0000 7f21acff9640 1 -- 172.21.15.86:0/731335731 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f21a809e300 con 0x7f21b0102a00 2024-04-21T08:35:41.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.406+0000 7f21b57a4640 1 -- 172.21.15.86:0/731335731 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f21b0107ec0 con 0x7f21b0102a00 2024-04-21T08:35:41.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.407+0000 7f21acff9640 1 -- 172.21.15.86:0/731335731 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f21a80a2600 con 0x7f21b0102a00 2024-04-21T08:35:41.406 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-21T08:35:41.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.409+0000 7f21b57a4640 1 -- 172.21.15.86:0/731335731 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f219c0a3d50 msgr2=0x7f219c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:41.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.409+0000 7f21b57a4640 1 --2- 172.21.15.86:0/731335731 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f219c0a3d50 0x7f219c0a6210 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7f21a0005fd0 tx=0x7f21a000b390 comp rx=0 tx=0).stop 2024-04-21T08:35:41.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.409+0000 7f21b57a4640 1 -- 172.21.15.86:0/731335731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21b0102a00 msgr2=0x7f21b019e6f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:41.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.409+0000 7f21b57a4640 1 --2- 172.21.15.86:0/731335731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21b0102a00 0x7f21b019e6f0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f21a8009fd0 tx=0x7f21a8007590 comp rx=0 tx=0).stop 2024-04-21T08:35:41.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.409+0000 7f21b57a4640 1 -- 172.21.15.86:0/731335731 shutdown_connections 2024-04-21T08:35:41.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.409+0000 7f21b57a4640 1 --2- 172.21.15.86:0/731335731 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f219c0a3d50 0x7f219c0a6210 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:41.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.409+0000 7f21b57a4640 1 --2- 172.21.15.86:0/731335731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21b0102a00 0x7f21b019e6f0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:41.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.409+0000 7f21b57a4640 1 --2- 172.21.15.86:0/731335731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21b0102040 0x7f21b0197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:41.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.409+0000 7f21b57a4640 1 --2- 172.21.15.86:0/731335731 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21b0100e60 0x7f21b0197140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:41.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.409+0000 7f21b57a4640 1 -- 172.21.15.86:0/731335731 >> 172.21.15.86:0/731335731 conn(0x7f21b00fc420 msgr2=0x7f21b0105440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:41.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.409+0000 7f21b57a4640 1 -- 172.21.15.86:0/731335731 shutdown_connections 2024-04-21T08:35:41.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.409+0000 7f21b57a4640 1 -- 172.21.15.86:0/731335731 wait complete. 2024-04-21T08:35:41.419 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:35:41.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.517+0000 7fb62be74640 1 -- 172.21.15.86:0/3850811128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb624101fa0 msgr2=0x7fb6241023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:41.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.517+0000 7fb62be74640 1 --2- 172.21.15.86:0/3850811128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb624101fa0 0x7fb6241023c0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7fb614009900 tx=0x7fb61402bc30 comp rx=0 tx=0).stop 2024-04-21T08:35:41.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.517+0000 7fb62be74640 1 -- 172.21.15.86:0/3850811128 shutdown_connections 2024-04-21T08:35:41.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.517+0000 7fb62be74640 1 --2- 172.21.15.86:0/3850811128 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb624103b50 0x7fb62410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:41.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.517+0000 7fb62be74640 1 --2- 172.21.15.86:0/3850811128 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb624103190 0x7fb624103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:41.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.517+0000 7fb62be74640 1 --2- 172.21.15.86:0/3850811128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb624101fa0 0x7fb6241023c0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:41.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.517+0000 7fb62be74640 1 -- 172.21.15.86:0/3850811128 >> 172.21.15.86:0/3850811128 conn(0x7fb6240fd530 msgr2=0x7fb6240ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:41.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.518+0000 7fb62be74640 1 -- 172.21.15.86:0/3850811128 shutdown_connections 2024-04-21T08:35:41.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.518+0000 7fb62be74640 1 -- 172.21.15.86:0/3850811128 wait complete. 2024-04-21T08:35:41.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.518+0000 7fb62be74640 1 Processor -- start 2024-04-21T08:35:41.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.518+0000 7fb62be74640 1 -- start start 2024-04-21T08:35:41.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.518+0000 7fb62be74640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb624101fa0 0x7fb624198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:41.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.518+0000 7fb62be74640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb624103190 0x7fb6241987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:41.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.518+0000 7fb62be74640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb624103b50 0x7fb62419f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:41.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.518+0000 7fb62be74640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb62410db40 con 0x7fb624103190 2024-04-21T08:35:41.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.518+0000 7fb62be74640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb62410d9c0 con 0x7fb624103b50 2024-04-21T08:35:41.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.518+0000 7fb62be74640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb62410dcc0 con 0x7fb624101fa0 2024-04-21T08:35:41.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.518+0000 7fb62940f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb624103190 0x7fb6241987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:41.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.518+0000 7fb62940f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb624103190 0x7fb6241987a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55446/0 (socket says 172.21.15.86:55446) 2024-04-21T08:35:41.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.518+0000 7fb62940f640 1 -- 172.21.15.86:0/642038948 learned_addr learned my addr 172.21.15.86:0/642038948 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:35:41.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.519+0000 7fb62940f640 1 -- 172.21.15.86:0/642038948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb624101fa0 msgr2=0x7fb624198260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:41.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.519+0000 7fb62940f640 1 --2- 172.21.15.86:0/642038948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb624101fa0 0x7fb624198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:41.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.519+0000 7fb62940f640 1 -- 172.21.15.86:0/642038948 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb624103b50 msgr2=0x7fb62419f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:41.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.519+0000 7fb62940f640 1 --2- 172.21.15.86:0/642038948 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb624103b50 0x7fb62419f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:41.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.519+0000 7fb62940f640 1 -- 172.21.15.86:0/642038948 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb62419fe80 con 0x7fb624103190 2024-04-21T08:35:41.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.519+0000 7fb62940f640 1 --2- 172.21.15.86:0/642038948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb624103190 0x7fb6241987a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb62000d770 tx=0x7fb62000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:41.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.519+0000 7fb61affd640 1 -- 172.21.15.86:0/642038948 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb620017070 con 0x7fb624103190 2024-04-21T08:35:41.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.519+0000 7fb61affd640 1 -- 172.21.15.86:0/642038948 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6200054d0 con 0x7fb624103190 2024-04-21T08:35:41.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.519+0000 7fb62be74640 1 -- 172.21.15.86:0/642038948 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6241a0170 con 0x7fb624103190 2024-04-21T08:35:41.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.519+0000 7fb61affd640 1 -- 172.21.15.86:0/642038948 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb620004c30 con 0x7fb624103190 2024-04-21T08:35:41.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.519+0000 7fb62be74640 1 -- 172.21.15.86:0/642038948 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6241a06f0 con 0x7fb624103190 2024-04-21T08:35:41.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.520+0000 7fb62be74640 1 -- 172.21.15.86:0/642038948 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb624109390 con 0x7fb624103190 2024-04-21T08:35:41.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.521+0000 7fb61affd640 1 -- 172.21.15.86:0/642038948 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb620004e10 con 0x7fb624103190 2024-04-21T08:35:41.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.522+0000 7fb61affd640 1 --2- 172.21.15.86:0/642038948 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb6100a3bb0 0x7fb6100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:41.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.522+0000 7fb629c10640 1 --2- 172.21.15.86:0/642038948 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb6100a3bb0 0x7fb6100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:41.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.522+0000 7fb61affd640 1 -- 172.21.15.86:0/642038948 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb6200cfad0 con 0x7fb624103190 2024-04-21T08:35:41.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.522+0000 7fb629c10640 1 --2- 172.21.15.86:0/642038948 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb6100a3bb0 0x7fb6100a6070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb614036040 tx=0x7fb614000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:41.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.526+0000 7fb61affd640 1 -- 172.21.15.86:0/642038948 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb62009c950 con 0x7fb624103190 2024-04-21T08:35:41.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.745+0000 7fb62be74640 1 -- 172.21.15.86:0/642038948 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb62402d800 con 0x7fb624103190 2024-04-21T08:35:41.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.746+0000 7fb61affd640 1 -- 172.21.15.86:0/642038948 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 1 v1) ==== 93+0+569 (secure 0 0 0) 0x7fb62009c250 con 0x7fb624103190 2024-04-21T08:35:41.746 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:35:41.746 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":1,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[],"filesystems":[]} 2024-04-21T08:35:41.746 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 1 2024-04-21T08:35:41.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:41 smithi179 ceph-mon[31829]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi088=b;smithi088=e;smithi088=h;smithi088=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:35:41.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:41 smithi179 ceph-mon[31829]: Saving service mds.all spec with placement smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi088=b;smithi088=e;smithi088=h;smithi088=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l;count:12 2024-04-21T08:35:41.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:41 smithi179 ceph-mon[31829]: Deploying daemon mds.l on smithi179 2024-04-21T08:35:41.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:41 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1801236344' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:35:41.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:41 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/731335731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:35:41.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.748+0000 7fb62be74640 1 -- 172.21.15.86:0/642038948 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb6100a3bb0 msgr2=0x7fb6100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:41.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.748+0000 7fb62be74640 1 --2- 172.21.15.86:0/642038948 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb6100a3bb0 0x7fb6100a6070 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7fb614036040 tx=0x7fb614000bc0 comp rx=0 tx=0).stop 2024-04-21T08:35:41.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.748+0000 7fb62be74640 1 -- 172.21.15.86:0/642038948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb624103190 msgr2=0x7fb6241987a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:41.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.748+0000 7fb62be74640 1 --2- 172.21.15.86:0/642038948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb624103190 0x7fb6241987a0 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fb62000d770 tx=0x7fb62000dc30 comp rx=0 tx=0).stop 2024-04-21T08:35:41.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.748+0000 7fb62be74640 1 -- 172.21.15.86:0/642038948 shutdown_connections 2024-04-21T08:35:41.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.748+0000 7fb62be74640 1 --2- 172.21.15.86:0/642038948 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb6100a3bb0 0x7fb6100a6070 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:41.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.748+0000 7fb62be74640 1 --2- 172.21.15.86:0/642038948 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb624103b50 0x7fb62419f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:41.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.748+0000 7fb62be74640 1 --2- 172.21.15.86:0/642038948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb624103190 0x7fb6241987a0 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:41.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.748+0000 7fb62be74640 1 --2- 172.21.15.86:0/642038948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb624101fa0 0x7fb624198260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:41.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.748+0000 7fb62be74640 1 -- 172.21.15.86:0/642038948 >> 172.21.15.86:0/642038948 conn(0x7fb6240fd530 msgr2=0x7fb624106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:41.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.748+0000 7fb62be74640 1 -- 172.21.15.86:0/642038948 shutdown_connections 2024-04-21T08:35:41.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:41.748+0000 7fb62be74640 1 -- 172.21.15.86:0/642038948 wait complete. 2024-04-21T08:35:41.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:41 smithi086 ceph-mon[27939]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi088=b;smithi088=e;smithi088=h;smithi088=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:35:41.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:41 smithi086 ceph-mon[27939]: Saving service mds.all spec with placement smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi088=b;smithi088=e;smithi088=h;smithi088=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l;count:12 2024-04-21T08:35:41.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:41 smithi086 ceph-mon[27939]: Deploying daemon mds.l on smithi179 2024-04-21T08:35:41.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:41 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1801236344' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:35:41.758 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:41 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/731335731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:35:41.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:41 smithi088 ceph-mon[31259]: from='client.24373 -' entity='client.admin' cmd=[{"prefix": "orch apply mds", "fs_name": "all", "placement": "12;smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi088=b;smithi088=e;smithi088=h;smithi088=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:35:41.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:41 smithi088 ceph-mon[31259]: Saving service mds.all spec with placement smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi088=b;smithi088=e;smithi088=h;smithi088=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l;count:12 2024-04-21T08:35:41.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:41 smithi088 ceph-mon[31259]: Deploying daemon mds.l on smithi179 2024-04-21T08:35:41.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:41 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1801236344' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:35:41.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:41 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/731335731' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:35:42.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:42 smithi179 ceph-mon[31829]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:42.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:42 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/642038948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:42.747 INFO:journalctl@ceph.mds.l.smithi179.stdout:Apr 21 08:35:42 smithi179 systemd[1]: Starting Ceph mds.l for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:35:42.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:42 smithi086 ceph-mon[27939]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:42.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:42 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/642038948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:42.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:42 smithi088 ceph-mon[31259]: pgmap v260: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:42.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:42 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/642038948' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:43.186 INFO:journalctl@ceph.mds.l.smithi179.stdout:Apr 21 08:35:42 smithi179 podman[60879]: 2024-04-21 08:35:42.81511962 +0000 UTC m=+0.017488995 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:35:43.186 INFO:journalctl@ceph.mds.l.smithi179.stdout:Apr 21 08:35:42 smithi179 podman[60879]: 2024-04-21 08:35:42.942851036 +0000 UTC m=+0.145220447 container create c381a381724c219ebda0071c7b02a48e9d1acde2dafa308bc690793a470ec97d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:35:43.187 INFO:journalctl@ceph.mds.l.smithi179.stdout:Apr 21 08:35:43 smithi179 podman[60879]: 2024-04-21 08:35:43.187619356 +0000 UTC m=+0.389988728 container init c381a381724c219ebda0071c7b02a48e9d1acde2dafa308bc690793a470ec97d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-l, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, ceph=True, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_CLEAN=True) 2024-04-21T08:35:43.496 INFO:journalctl@ceph.mds.l.smithi179.stdout:Apr 21 08:35:43 smithi179 podman[60879]: 2024-04-21 08:35:43.190192452 +0000 UTC m=+0.392561827 container start c381a381724c219ebda0071c7b02a48e9d1acde2dafa308bc690793a470ec97d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-l, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:35:43.497 INFO:journalctl@ceph.mds.l.smithi179.stdout:Apr 21 08:35:43 smithi179 ceph-mds[60899]: -- 172.21.15.179:0/2126914131 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b6abd11a40 con 0x55b6abce5000 2024-04-21T08:35:43.497 INFO:journalctl@ceph.mds.l.smithi179.stdout:Apr 21 08:35:43 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-l[60895]: starting mds.l at 2024-04-21T08:35:43.497 INFO:journalctl@ceph.mds.l.smithi179.stdout:Apr 21 08:35:43 smithi179 bash[60879]: c381a381724c219ebda0071c7b02a48e9d1acde2dafa308bc690793a470ec97d 2024-04-21T08:35:43.497 INFO:journalctl@ceph.mds.l.smithi179.stdout:Apr 21 08:35:43 smithi179 systemd[1]: Started Ceph mds.l for f8cec7cc-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:35:43.760 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:35:43.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.868+0000 7f60f77bd640 1 -- 172.21.15.86:0/2902670862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60f0102990 msgr2=0x7f60f01092b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:43.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.868+0000 7f60f77bd640 1 --2- 172.21.15.86:0/2902670862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60f0102990 0x7f60f01092b0 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f60e8009ca0 tx=0x7f60e802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:35:43.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.868+0000 7f60f77bd640 1 -- 172.21.15.86:0/2902670862 shutdown_connections 2024-04-21T08:35:43.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.868+0000 7f60f77bd640 1 --2- 172.21.15.86:0/2902670862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60f0102990 0x7f60f01092b0 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:43.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.868+0000 7f60f77bd640 1 --2- 172.21.15.86:0/2902670862 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60f0101fd0 0x7f60f0102450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:43.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.868+0000 7f60f77bd640 1 --2- 172.21.15.86:0/2902670862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f60f0100e80 0x7f60f01012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:43.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.868+0000 7f60f77bd640 1 -- 172.21.15.86:0/2902670862 >> 172.21.15.86:0/2902670862 conn(0x7f60f00fc410 msgr2=0x7f60f00fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:43.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.869+0000 7f60f77bd640 1 -- 172.21.15.86:0/2902670862 shutdown_connections 2024-04-21T08:35:43.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.869+0000 7f60f77bd640 1 -- 172.21.15.86:0/2902670862 wait complete. 2024-04-21T08:35:43.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.869+0000 7f60f77bd640 1 Processor -- start 2024-04-21T08:35:43.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.869+0000 7f60f77bd640 1 -- start start 2024-04-21T08:35:43.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.869+0000 7f60f77bd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60f0100e80 0x7f60f006b210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:43.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.869+0000 7f60f77bd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60f0101fd0 0x7f60f019ce80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:43.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.869+0000 7f60f77bd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f60f0102990 0x7f60f019f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:43.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.869+0000 7f60f77bd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60f010ca70 con 0x7f60f0100e80 2024-04-21T08:35:43.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.869+0000 7f60f77bd640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f60f010c8f0 con 0x7f60f0101fd0 2024-04-21T08:35:43.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.869+0000 7f60f77bd640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f60f010cbf0 con 0x7f60f0102990 2024-04-21T08:35:43.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.869+0000 7f60f5559640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60f0100e80 0x7f60f006b210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:43.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.869+0000 7f60f5559640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60f0100e80 0x7f60f006b210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55452/0 (socket says 172.21.15.86:55452) 2024-04-21T08:35:43.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.869+0000 7f60f5559640 1 -- 172.21.15.86:0/1231721457 learned_addr learned my addr 172.21.15.86:0/1231721457 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:35:43.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.869+0000 7f60f4d58640 1 --2- 172.21.15.86:0/1231721457 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60f0101fd0 0x7f60f019ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:43.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.870+0000 7f60f5d5a640 1 --2- 172.21.15.86:0/1231721457 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f60f0102990 0x7f60f019f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:43.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.870+0000 7f60f5559640 1 -- 172.21.15.86:0/1231721457 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f60f0102990 msgr2=0x7f60f019f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:43.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.870+0000 7f60f5559640 1 --2- 172.21.15.86:0/1231721457 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f60f0102990 0x7f60f019f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:43.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.870+0000 7f60f5559640 1 -- 172.21.15.86:0/1231721457 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60f0101fd0 msgr2=0x7f60f019ce80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:43.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.870+0000 7f60f5559640 1 --2- 172.21.15.86:0/1231721457 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60f0101fd0 0x7f60f019ce80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:43.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.870+0000 7f60f5559640 1 -- 172.21.15.86:0/1231721457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60f019fdb0 con 0x7f60f0100e80 2024-04-21T08:35:43.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.870+0000 7f60f5559640 1 --2- 172.21.15.86:0/1231721457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60f0100e80 0x7f60f006b210 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00e9f0 tx=0x7f60ec00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:43.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.870+0000 7f60e67fc640 1 -- 172.21.15.86:0/1231721457 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec00ccc0 con 0x7f60f0100e80 2024-04-21T08:35:43.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.870+0000 7f60e67fc640 1 -- 172.21.15.86:0/1231721457 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60ec00ce60 con 0x7f60f0100e80 2024-04-21T08:35:43.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.870+0000 7f60f77bd640 1 -- 172.21.15.86:0/1231721457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60f01a00a0 con 0x7f60f0100e80 2024-04-21T08:35:43.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.870+0000 7f60e67fc640 1 -- 172.21.15.86:0/1231721457 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60ec010430 con 0x7f60f0100e80 2024-04-21T08:35:43.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.870+0000 7f60f77bd640 1 -- 172.21.15.86:0/1231721457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60f01a05a0 con 0x7f60f0100e80 2024-04-21T08:35:43.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.871+0000 7f60f77bd640 1 -- 172.21.15.86:0/1231721457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f60f0108190 con 0x7f60f0100e80 2024-04-21T08:35:43.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.872+0000 7f60e67fc640 1 -- 172.21.15.86:0/1231721457 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f60ec0046c0 con 0x7f60f0100e80 2024-04-21T08:35:43.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.873+0000 7f60e67fc640 1 --2- 172.21.15.86:0/1231721457 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f60dc0a3bb0 0x7f60dc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:43.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.873+0000 7f60f4d58640 1 --2- 172.21.15.86:0/1231721457 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f60dc0a3bb0 0x7f60dc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:43.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.873+0000 7f60e67fc640 1 -- 172.21.15.86:0/1231721457 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f60ec0d0330 con 0x7f60f0100e80 2024-04-21T08:35:43.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.873+0000 7f60f4d58640 1 --2- 172.21.15.86:0/1231721457 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f60dc0a3bb0 0x7f60dc0a6070 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f60f006af80 tx=0x7f60e0009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:43.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:43.877+0000 7f60e67fc640 1 -- 172.21.15.86:0/1231721457 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f60ec09d1b0 con 0x7f60f0100e80 2024-04-21T08:35:44.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:44.094+0000 7f60f77bd640 1 -- 172.21.15.86:0/1231721457 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f60f002d5c0 con 0x7f60f0100e80 2024-04-21T08:35:44.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:44.094+0000 7f60e67fc640 1 -- 172.21.15.86:0/1231721457 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 2 v2) ==== 93+0+1396 (secure 0 0 0) 0x7f60ec09cab0 con 0x7f60f0100e80 2024-04-21T08:35:44.095 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:35:44.095 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":2,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2}],"filesystems":[]} 2024-04-21T08:35:44.095 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 2 2024-04-21T08:35:44.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:44.096+0000 7f60f77bd640 1 -- 172.21.15.86:0/1231721457 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f60dc0a3bb0 msgr2=0x7f60dc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:44.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:44.096+0000 7f60f77bd640 1 --2- 172.21.15.86:0/1231721457 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f60dc0a3bb0 0x7f60dc0a6070 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7f60f006af80 tx=0x7f60e0009110 comp rx=0 tx=0).stop 2024-04-21T08:35:44.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:44.096+0000 7f60f77bd640 1 -- 172.21.15.86:0/1231721457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60f0100e80 msgr2=0x7f60f006b210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:44.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:44.096+0000 7f60f77bd640 1 --2- 172.21.15.86:0/1231721457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60f0100e80 0x7f60f006b210 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f60ec00e9f0 tx=0x7f60ec00edf0 comp rx=0 tx=0).stop 2024-04-21T08:35:44.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:44.096+0000 7f60f77bd640 1 -- 172.21.15.86:0/1231721457 shutdown_connections 2024-04-21T08:35:44.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:44.096+0000 7f60f77bd640 1 --2- 172.21.15.86:0/1231721457 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f60dc0a3bb0 0x7f60dc0a6070 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:44.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:44.096+0000 7f60f77bd640 1 --2- 172.21.15.86:0/1231721457 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f60f0102990 0x7f60f019f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:44.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:44.097+0000 7f60f77bd640 1 --2- 172.21.15.86:0/1231721457 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f60f0101fd0 0x7f60f019ce80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:44.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:44.097+0000 7f60f77bd640 1 --2- 172.21.15.86:0/1231721457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f60f0100e80 0x7f60f006b210 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:44.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:44.097+0000 7f60f77bd640 1 -- 172.21.15.86:0/1231721457 >> 172.21.15.86:0/1231721457 conn(0x7f60f00fc410 msgr2=0x7f60f0105360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:44.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:44.097+0000 7f60f77bd640 1 -- 172.21.15.86:0/1231721457 shutdown_connections 2024-04-21T08:35:44.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:44.097+0000 7f60f77bd640 1 -- 172.21.15.86:0/1231721457 wait complete. 2024-04-21T08:35:44.249 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:44 smithi088 ceph-mon[31259]: mds.? [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] up:boot 2024-04-21T08:35:44.249 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:44 smithi088 ceph-mon[31259]: fsmap 1 up:standby 2024-04-21T08:35:44.249 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:44 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-21T08:35:44.249 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:44 smithi088 ceph-mon[31259]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:44.249 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:44 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:44.249 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:44 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:44.249 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:44 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:44.249 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:44 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:35:44.249 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:44 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:35:44.249 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:44 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:44.250 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:44 smithi088 ceph-mon[31259]: Deploying daemon mds.k on smithi088 2024-04-21T08:35:44.250 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:44 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1231721457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:44.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:44 smithi086 ceph-mon[27939]: mds.? [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] up:boot 2024-04-21T08:35:44.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:44 smithi086 ceph-mon[27939]: fsmap 1 up:standby 2024-04-21T08:35:44.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:44 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-21T08:35:44.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:44 smithi086 ceph-mon[27939]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:44.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:44 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:44.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:44 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:44.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:44 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:44.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:44 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:35:44.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:44 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:35:44.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:44 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:44.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:44 smithi086 ceph-mon[27939]: Deploying daemon mds.k on smithi088 2024-04-21T08:35:44.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:44 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1231721457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:44.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:44 smithi179 ceph-mon[31829]: mds.? [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] up:boot 2024-04-21T08:35:44.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:44 smithi179 ceph-mon[31829]: fsmap 1 up:standby 2024-04-21T08:35:44.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:44 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-21T08:35:44.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:44 smithi179 ceph-mon[31829]: pgmap v261: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:44.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:44 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:44.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:44 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:44.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:44 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:44.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:44 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:35:44.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:44 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.k", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:35:44.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:44 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:44.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:44 smithi179 ceph-mon[31829]: Deploying daemon mds.k on smithi088 2024-04-21T08:35:44.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:44 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1231721457' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:45.714 INFO:journalctl@ceph.mds.k.smithi088.stdout:Apr 21 08:35:45 smithi088 systemd[1]: Starting Ceph mds.k for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:35:46.017 INFO:journalctl@ceph.mds.k.smithi088.stdout:Apr 21 08:35:45 smithi088 podman[60680]: 2024-04-21 08:35:45.765751427 +0000 UTC m=+0.113333184 container create e43b96f6ea27643ff10ecd0b4d7487265e4e5ec1b09d63703fd3c1e255ce02ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-k, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, ceph=True) 2024-04-21T08:35:46.017 INFO:journalctl@ceph.mds.k.smithi088.stdout:Apr 21 08:35:45 smithi088 podman[60680]: 2024-04-21 08:35:45.670797608 +0000 UTC m=+0.018379389 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:35:46.017 INFO:journalctl@ceph.mds.k.smithi088.stdout:Apr 21 08:35:46 smithi088 podman[60680]: 2024-04-21 08:35:46.01828661 +0000 UTC m=+0.365868369 container init e43b96f6ea27643ff10ecd0b4d7487265e4e5ec1b09d63703fd3c1e255ce02ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-k, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD) 2024-04-21T08:35:46.109 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:35:46.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.226+0000 7f67d797e640 1 -- 172.21.15.86:0/877974486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67d0102980 msgr2=0x7f67d01092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:46.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.226+0000 7f67d797e640 1 --2- 172.21.15.86:0/877974486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67d0102980 0x7f67d01092a0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f67c8009ca0 tx=0x7f67c802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:35:46.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.226+0000 7f67d797e640 1 -- 172.21.15.86:0/877974486 shutdown_connections 2024-04-21T08:35:46.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.226+0000 7f67d797e640 1 --2- 172.21.15.86:0/877974486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67d0102980 0x7f67d01092a0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:46.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.226+0000 7f67d797e640 1 --2- 172.21.15.86:0/877974486 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67d0101fc0 0x7f67d0102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:46.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.226+0000 7f67d797e640 1 --2- 172.21.15.86:0/877974486 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67d0100e70 0x7f67d0101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:46.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.226+0000 7f67d797e640 1 -- 172.21.15.86:0/877974486 >> 172.21.15.86:0/877974486 conn(0x7f67d00fc3e0 msgr2=0x7f67d00fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:46.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.226+0000 7f67d797e640 1 -- 172.21.15.86:0/877974486 shutdown_connections 2024-04-21T08:35:46.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.226+0000 7f67d797e640 1 -- 172.21.15.86:0/877974486 wait complete. 2024-04-21T08:35:46.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.227+0000 7f67d797e640 1 Processor -- start 2024-04-21T08:35:46.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.227+0000 7f67d797e640 1 -- start start 2024-04-21T08:35:46.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.227+0000 7f67d797e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67d0100e70 0x7f67d0069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:46.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.227+0000 7f67d797e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67d0101fc0 0x7f67d019ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:46.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.227+0000 7f67d797e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67d0102980 0x7f67d019f480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:46.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.227+0000 7f67d797e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f67d010c9c0 con 0x7f67d0101fc0 2024-04-21T08:35:46.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.227+0000 7f67d797e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f67d010c840 con 0x7f67d0100e70 2024-04-21T08:35:46.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.227+0000 7f67d797e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f67d010cb40 con 0x7f67d0102980 2024-04-21T08:35:46.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.227+0000 7f67d4f19640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67d0101fc0 0x7f67d019ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:46.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.227+0000 7f67d4f19640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67d0101fc0 0x7f67d019ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55474/0 (socket says 172.21.15.86:55474) 2024-04-21T08:35:46.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.227+0000 7f67d4f19640 1 -- 172.21.15.86:0/2074142615 learned_addr learned my addr 172.21.15.86:0/2074142615 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:35:46.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.227+0000 7f67d571a640 1 --2- 172.21.15.86:0/2074142615 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67d0100e70 0x7f67d0069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:46.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.228+0000 7f67d5f1b640 1 --2- 172.21.15.86:0/2074142615 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67d0102980 0x7f67d019f480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:46.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.228+0000 7f67d4f19640 1 -- 172.21.15.86:0/2074142615 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67d0102980 msgr2=0x7f67d019f480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:46.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.228+0000 7f67d4f19640 1 --2- 172.21.15.86:0/2074142615 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67d0102980 0x7f67d019f480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:46.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.228+0000 7f67d4f19640 1 -- 172.21.15.86:0/2074142615 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67d0100e70 msgr2=0x7f67d0069c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:46.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.228+0000 7f67d4f19640 1 --2- 172.21.15.86:0/2074142615 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67d0100e70 0x7f67d0069c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:46.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.228+0000 7f67d4f19640 1 -- 172.21.15.86:0/2074142615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67d019fd90 con 0x7f67d0101fc0 2024-04-21T08:35:46.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.228+0000 7f67d571a640 1 --2- 172.21.15.86:0/2074142615 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67d0100e70 0x7f67d0069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:35:46.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.228+0000 7f67d5f1b640 1 --2- 172.21.15.86:0/2074142615 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67d0102980 0x7f67d019f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:35:46.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.228+0000 7f67d4f19640 1 --2- 172.21.15.86:0/2074142615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67d0101fc0 0x7f67d019ce60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f67c000e9c0 tx=0x7f67c000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:46.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.228+0000 7f67c67fc640 1 -- 172.21.15.86:0/2074142615 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c000cd80 con 0x7f67d0101fc0 2024-04-21T08:35:46.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.228+0000 7f67d797e640 1 -- 172.21.15.86:0/2074142615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67d019ffc0 con 0x7f67d0101fc0 2024-04-21T08:35:46.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.228+0000 7f67c67fc640 1 -- 172.21.15.86:0/2074142615 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67c00035f0 con 0x7f67d0101fc0 2024-04-21T08:35:46.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.228+0000 7f67c67fc640 1 -- 172.21.15.86:0/2074142615 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67c0010690 con 0x7f67d0101fc0 2024-04-21T08:35:46.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.228+0000 7f67d797e640 1 -- 172.21.15.86:0/2074142615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67d01a0530 con 0x7f67d0101fc0 2024-04-21T08:35:46.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.229+0000 7f67d797e640 1 -- 172.21.15.86:0/2074142615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f67d0108180 con 0x7f67d0101fc0 2024-04-21T08:35:46.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.230+0000 7f67c67fc640 1 -- 172.21.15.86:0/2074142615 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f67c00037a0 con 0x7f67d0101fc0 2024-04-21T08:35:46.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.231+0000 7f67c67fc640 1 --2- 172.21.15.86:0/2074142615 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f67bc0a3c30 0x7f67bc0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:46.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.231+0000 7f67d571a640 1 --2- 172.21.15.86:0/2074142615 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f67bc0a3c30 0x7f67bc0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:46.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.231+0000 7f67c67fc640 1 -- 172.21.15.86:0/2074142615 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f67c00d02f0 con 0x7f67d0101fc0 2024-04-21T08:35:46.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.231+0000 7f67d571a640 1 --2- 172.21.15.86:0/2074142615 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f67bc0a3c30 0x7f67bc0a60f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f67cc005fd0 tx=0x7f67cc00b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:46.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.235+0000 7f67c67fc640 1 -- 172.21.15.86:0/2074142615 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f67c009d170 con 0x7f67d0101fc0 2024-04-21T08:35:46.313 INFO:journalctl@ceph.mds.k.smithi088.stdout:Apr 21 08:35:46 smithi088 podman[60680]: 2024-04-21 08:35:46.021017113 +0000 UTC m=+0.368598877 container start e43b96f6ea27643ff10ecd0b4d7487265e4e5ec1b09d63703fd3c1e255ce02ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-k, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0) 2024-04-21T08:35:46.313 INFO:journalctl@ceph.mds.k.smithi088.stdout:Apr 21 08:35:46 smithi088 ceph-mds[60699]: -- 172.21.15.88:0/423826200 <== mon.2 v2:172.21.15.88:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55af3d669a40 con 0x55af3d63d000 2024-04-21T08:35:46.313 INFO:journalctl@ceph.mds.k.smithi088.stdout:Apr 21 08:35:46 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-k[60695]: starting mds.k at 2024-04-21T08:35:46.313 INFO:journalctl@ceph.mds.k.smithi088.stdout:Apr 21 08:35:46 smithi088 bash[60680]: e43b96f6ea27643ff10ecd0b4d7487265e4e5ec1b09d63703fd3c1e255ce02ad 2024-04-21T08:35:46.313 INFO:journalctl@ceph.mds.k.smithi088.stdout:Apr 21 08:35:46 smithi088 systemd[1]: Started Ceph mds.k for f8cec7cc-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:35:46.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.468+0000 7f67d797e640 1 -- 172.21.15.86:0/2074142615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f67d002d810 con 0x7f67d0101fc0 2024-04-21T08:35:46.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.468+0000 7f67c67fc640 1 -- 172.21.15.86:0/2074142615 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7f67c009ca70 con 0x7f67d0101fc0 2024-04-21T08:35:46.469 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:35:46.469 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-21T08:35:46.469 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 3 2024-04-21T08:35:46.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.471+0000 7f67d797e640 1 -- 172.21.15.86:0/2074142615 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f67bc0a3c30 msgr2=0x7f67bc0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:46.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.471+0000 7f67d797e640 1 --2- 172.21.15.86:0/2074142615 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f67bc0a3c30 0x7f67bc0a60f0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f67cc005fd0 tx=0x7f67cc00b390 comp rx=0 tx=0).stop 2024-04-21T08:35:46.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.471+0000 7f67d797e640 1 -- 172.21.15.86:0/2074142615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67d0101fc0 msgr2=0x7f67d019ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:46.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.471+0000 7f67d797e640 1 --2- 172.21.15.86:0/2074142615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67d0101fc0 0x7f67d019ce60 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f67c000e9c0 tx=0x7f67c000ee80 comp rx=0 tx=0).stop 2024-04-21T08:35:46.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.471+0000 7f67d797e640 1 -- 172.21.15.86:0/2074142615 shutdown_connections 2024-04-21T08:35:46.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.471+0000 7f67d797e640 1 --2- 172.21.15.86:0/2074142615 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f67bc0a3c30 0x7f67bc0a60f0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:46.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.471+0000 7f67d797e640 1 --2- 172.21.15.86:0/2074142615 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67d0102980 0x7f67d019f480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:46.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.471+0000 7f67d797e640 1 --2- 172.21.15.86:0/2074142615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67d0101fc0 0x7f67d019ce60 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:46.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.471+0000 7f67d797e640 1 --2- 172.21.15.86:0/2074142615 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67d0100e70 0x7f67d0069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:46.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.471+0000 7f67d797e640 1 -- 172.21.15.86:0/2074142615 >> 172.21.15.86:0/2074142615 conn(0x7f67d00fc3e0 msgr2=0x7f67d0105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:46.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.471+0000 7f67d797e640 1 -- 172.21.15.86:0/2074142615 shutdown_connections 2024-04-21T08:35:46.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:46.471+0000 7f67d797e640 1 -- 172.21.15.86:0/2074142615 wait complete. 2024-04-21T08:35:46.481 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:46 smithi086 ceph-mon[27939]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:46.481 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:46 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:46.481 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:46 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:46.481 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:46 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:46.481 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:46 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:35:46.481 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:46 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:35:46.481 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:46 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:46.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:46 smithi088 ceph-mon[31259]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:46.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:46 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:46.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:46 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:46.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:46 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:46.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:46 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:35:46.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:46 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:35:46.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:46 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:46.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:46 smithi179 ceph-mon[31829]: pgmap v262: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:46.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:46 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:46.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:46 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:46.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:46 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:46.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:46 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:35:46.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:46 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.j", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:35:46.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:46 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:47.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:47 smithi086 ceph-mon[27939]: Deploying daemon mds.j on smithi086 2024-04-21T08:35:47.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:47 smithi086 ceph-mon[27939]: mds.? [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] up:boot 2024-04-21T08:35:47.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:47 smithi086 ceph-mon[27939]: fsmap 2 up:standby 2024-04-21T08:35:47.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:47 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-21T08:35:47.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:47 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2074142615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:47.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:47 smithi088 ceph-mon[31259]: Deploying daemon mds.j on smithi086 2024-04-21T08:35:47.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:47 smithi088 ceph-mon[31259]: mds.? [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] up:boot 2024-04-21T08:35:47.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:47 smithi088 ceph-mon[31259]: fsmap 2 up:standby 2024-04-21T08:35:47.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:47 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-21T08:35:47.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:47 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2074142615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:47.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:47 smithi179 ceph-mon[31829]: Deploying daemon mds.j on smithi086 2024-04-21T08:35:47.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:47 smithi179 ceph-mon[31829]: mds.? [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] up:boot 2024-04-21T08:35:47.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:47 smithi179 ceph-mon[31829]: fsmap 2 up:standby 2024-04-21T08:35:47.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:47 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-21T08:35:47.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:47 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2074142615' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:48.483 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:35:48.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.594+0000 7fe74b9e6640 1 -- 172.21.15.86:0/3442893797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe744103b30 msgr2=0x7fe74410a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:48.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.594+0000 7fe74b9e6640 1 --2- 172.21.15.86:0/3442893797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe744103b30 0x7fe74410a3c0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fe73c009ca0 tx=0x7fe73c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:35:48.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.595+0000 7fe74b9e6640 1 -- 172.21.15.86:0/3442893797 shutdown_connections 2024-04-21T08:35:48.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.595+0000 7fe74b9e6640 1 --2- 172.21.15.86:0/3442893797 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe744103b30 0x7fe74410a3c0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:48.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.595+0000 7fe74b9e6640 1 --2- 172.21.15.86:0/3442893797 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe744103170 0x7fe7441035f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:48.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.595+0000 7fe74b9e6640 1 --2- 172.21.15.86:0/3442893797 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe744101f80 0x7fe7441023a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:48.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.595+0000 7fe74b9e6640 1 -- 172.21.15.86:0/3442893797 >> 172.21.15.86:0/3442893797 conn(0x7fe7440fd530 msgr2=0x7fe7440ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:48.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.595+0000 7fe74b9e6640 1 -- 172.21.15.86:0/3442893797 shutdown_connections 2024-04-21T08:35:48.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.595+0000 7fe74b9e6640 1 -- 172.21.15.86:0/3442893797 wait complete. 2024-04-21T08:35:48.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.595+0000 7fe74b9e6640 1 Processor -- start 2024-04-21T08:35:48.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.595+0000 7fe74b9e6640 1 -- start start 2024-04-21T08:35:48.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.595+0000 7fe74b9e6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe744101f80 0x7fe744198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:48.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.595+0000 7fe74b9e6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe744103170 0x7fe744198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:48.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.595+0000 7fe74b9e6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe744103b30 0x7fe74419f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:48.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.595+0000 7fe74b9e6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe74410dac0 con 0x7fe744103170 2024-04-21T08:35:48.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.595+0000 7fe74b9e6640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe74410d940 con 0x7fe744103b30 2024-04-21T08:35:48.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.595+0000 7fe74b9e6640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe74410dc40 con 0x7fe744101f80 2024-04-21T08:35:48.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.596+0000 7fe748f81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe744103170 0x7fe744198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:48.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.596+0000 7fe748f81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe744103170 0x7fe744198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55500/0 (socket says 172.21.15.86:55500) 2024-04-21T08:35:48.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.596+0000 7fe749782640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe744101f80 0x7fe744198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:48.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.596+0000 7fe748f81640 1 -- 172.21.15.86:0/974544789 learned_addr learned my addr 172.21.15.86:0/974544789 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:35:48.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.596+0000 7fe749f83640 1 --2- 172.21.15.86:0/974544789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe744103b30 0x7fe74419f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:48.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.596+0000 7fe748f81640 1 -- 172.21.15.86:0/974544789 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe744101f80 msgr2=0x7fe744198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:48.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.596+0000 7fe748f81640 1 --2- 172.21.15.86:0/974544789 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe744101f80 0x7fe744198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:48.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.596+0000 7fe748f81640 1 -- 172.21.15.86:0/974544789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe744103b30 msgr2=0x7fe74419f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:48.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.596+0000 7fe748f81640 1 --2- 172.21.15.86:0/974544789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe744103b30 0x7fe74419f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:48.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.596+0000 7fe748f81640 1 -- 172.21.15.86:0/974544789 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe74419fd30 con 0x7fe744103170 2024-04-21T08:35:48.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.596+0000 7fe749f83640 1 --2- 172.21.15.86:0/974544789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe744103b30 0x7fe74419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:35:48.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.596+0000 7fe748f81640 1 --2- 172.21.15.86:0/974544789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe744103170 0x7fe744198650 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe73400e9f0 tx=0x7fe73400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:48.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.596+0000 7fe73a7fc640 1 -- 172.21.15.86:0/974544789 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe73400cd80 con 0x7fe744103170 2024-04-21T08:35:48.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.596+0000 7fe73a7fc640 1 -- 172.21.15.86:0/974544789 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7340108e0 con 0x7fe744103170 2024-04-21T08:35:48.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.596+0000 7fe74b9e6640 1 -- 172.21.15.86:0/974544789 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe74419ffc0 con 0x7fe744103170 2024-04-21T08:35:48.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.596+0000 7fe73a7fc640 1 -- 172.21.15.86:0/974544789 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe734018980 con 0x7fe744103170 2024-04-21T08:35:48.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.596+0000 7fe74b9e6640 1 -- 172.21.15.86:0/974544789 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7441a04e0 con 0x7fe744103170 2024-04-21T08:35:48.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.598+0000 7fe74b9e6640 1 -- 172.21.15.86:0/974544789 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe7441092f0 con 0x7fe744103170 2024-04-21T08:35:48.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.598+0000 7fe73a7fc640 1 -- 172.21.15.86:0/974544789 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe734010430 con 0x7fe744103170 2024-04-21T08:35:48.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.599+0000 7fe73a7fc640 1 --2- 172.21.15.86:0/974544789 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe7300a3bb0 0x7fe7300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:48.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.599+0000 7fe73a7fc640 1 -- 172.21.15.86:0/974544789 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe7340d0210 con 0x7fe744103170 2024-04-21T08:35:48.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.599+0000 7fe749782640 1 --2- 172.21.15.86:0/974544789 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe7300a3bb0 0x7fe7300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:48.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.600+0000 7fe749782640 1 --2- 172.21.15.86:0/974544789 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe7300a3bb0 0x7fe7300a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe740005fd0 tx=0x7fe74000b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:48.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.604+0000 7fe73a7fc640 1 -- 172.21.15.86:0/974544789 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe73409ca40 con 0x7fe744103170 2024-04-21T08:35:48.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:48 smithi086 ceph-mon[27939]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:48.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:48 smithi088 ceph-mon[31259]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:48.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:48 smithi179 ceph-mon[31829]: pgmap v263: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:48.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.826+0000 7fe74b9e6640 1 -- 172.21.15.86:0/974544789 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe74402d810 con 0x7fe744103170 2024-04-21T08:35:48.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.827+0000 7fe73a7fc640 1 -- 172.21.15.86:0/974544789 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 3 v3) ==== 93+0+2221 (secure 0 0 0) 0x7fe7340a0d40 con 0x7fe744103170 2024-04-21T08:35:48.827 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:35:48.827 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":3,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-21T08:35:48.828 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 3 2024-04-21T08:35:48.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.829+0000 7fe74b9e6640 1 -- 172.21.15.86:0/974544789 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe7300a3bb0 msgr2=0x7fe7300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:48.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.829+0000 7fe74b9e6640 1 --2- 172.21.15.86:0/974544789 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe7300a3bb0 0x7fe7300a6070 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7fe740005fd0 tx=0x7fe74000b390 comp rx=0 tx=0).stop 2024-04-21T08:35:48.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.829+0000 7fe74b9e6640 1 -- 172.21.15.86:0/974544789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe744103170 msgr2=0x7fe744198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:48.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.829+0000 7fe74b9e6640 1 --2- 172.21.15.86:0/974544789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe744103170 0x7fe744198650 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7fe73400e9f0 tx=0x7fe73400eeb0 comp rx=0 tx=0).stop 2024-04-21T08:35:48.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.829+0000 7fe74b9e6640 1 -- 172.21.15.86:0/974544789 shutdown_connections 2024-04-21T08:35:48.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.829+0000 7fe74b9e6640 1 --2- 172.21.15.86:0/974544789 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe7300a3bb0 0x7fe7300a6070 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:48.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.829+0000 7fe74b9e6640 1 --2- 172.21.15.86:0/974544789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe744103b30 0x7fe74419f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:48.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.829+0000 7fe74b9e6640 1 --2- 172.21.15.86:0/974544789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe744103170 0x7fe744198650 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:48.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.829+0000 7fe74b9e6640 1 --2- 172.21.15.86:0/974544789 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe744101f80 0x7fe744198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:48.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.829+0000 7fe74b9e6640 1 -- 172.21.15.86:0/974544789 >> 172.21.15.86:0/974544789 conn(0x7fe7440fd530 msgr2=0x7fe7441064c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:48.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.829+0000 7fe74b9e6640 1 -- 172.21.15.86:0/974544789 shutdown_connections 2024-04-21T08:35:48.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:48.829+0000 7fe74b9e6640 1 -- 172.21.15.86:0/974544789 wait complete. 2024-04-21T08:35:49.341 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 21 08:35:49 smithi086 systemd[1]: Started Ceph mds.j for f8cec7cc-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:35:49.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:49 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/974544789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:49.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:49 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:49.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:49 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:49.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:49 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:49.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:49 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:35:49.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:49 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:35:49.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:49 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:49.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:49 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/974544789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:49.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:49 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:49.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:49 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:49.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:49 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:49.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:49 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:35:49.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:49 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:35:49.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:49 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:49.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:49 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/974544789' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:49.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:49 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:49.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:49 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:49.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:49 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:49.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:49 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:35:49.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:49 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.d", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:35:49.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:49 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:50.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:50 smithi086 ceph-mon[27939]: Deploying daemon mds.d on smithi086 2024-04-21T08:35:50.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:50 smithi086 ceph-mon[27939]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:50.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:50 smithi086 ceph-mon[27939]: mds.? [v2:172.21.15.86:6834/4288306918,v1:172.21.15.86:6835/4288306918] up:boot 2024-04-21T08:35:50.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:50 smithi086 ceph-mon[27939]: fsmap 3 up:standby 2024-04-21T08:35:50.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:50 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-21T08:35:50.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:50 smithi088 ceph-mon[31259]: Deploying daemon mds.d on smithi086 2024-04-21T08:35:50.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:50 smithi088 ceph-mon[31259]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:50.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:50 smithi088 ceph-mon[31259]: mds.? [v2:172.21.15.86:6834/4288306918,v1:172.21.15.86:6835/4288306918] up:boot 2024-04-21T08:35:50.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:50 smithi088 ceph-mon[31259]: fsmap 3 up:standby 2024-04-21T08:35:50.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:50 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-21T08:35:50.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:50 smithi179 ceph-mon[31829]: Deploying daemon mds.d on smithi086 2024-04-21T08:35:50.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:50 smithi179 ceph-mon[31829]: pgmap v264: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:50.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:50 smithi179 ceph-mon[31829]: mds.? [v2:172.21.15.86:6834/4288306918,v1:172.21.15.86:6835/4288306918] up:boot 2024-04-21T08:35:50.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:50 smithi179 ceph-mon[31829]: fsmap 3 up:standby 2024-04-21T08:35:50.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:50 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-21T08:35:50.842 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:35:50.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.949+0000 7f485973d640 1 -- 172.21.15.86:0/1754918940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4854100e70 msgr2=0x7f4854101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:50.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.949+0000 7f485973d640 1 --2- 172.21.15.86:0/1754918940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4854100e70 0x7f4854101290 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f4844009900 tx=0x7f484402bc30 comp rx=0 tx=0).stop 2024-04-21T08:35:50.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.949+0000 7f485973d640 1 -- 172.21.15.86:0/1754918940 shutdown_connections 2024-04-21T08:35:50.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.949+0000 7f485973d640 1 --2- 172.21.15.86:0/1754918940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4854102980 0x7f48541092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:50.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.949+0000 7f485973d640 1 --2- 172.21.15.86:0/1754918940 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4854101fc0 0x7f4854102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:50.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.949+0000 7f485973d640 1 --2- 172.21.15.86:0/1754918940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4854100e70 0x7f4854101290 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:50.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.949+0000 7f485973d640 1 -- 172.21.15.86:0/1754918940 >> 172.21.15.86:0/1754918940 conn(0x7f48540fc3e0 msgr2=0x7f48540fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:50.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.949+0000 7f485973d640 1 -- 172.21.15.86:0/1754918940 shutdown_connections 2024-04-21T08:35:50.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.949+0000 7f485973d640 1 -- 172.21.15.86:0/1754918940 wait complete. 2024-04-21T08:35:50.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.949+0000 7f485973d640 1 Processor -- start 2024-04-21T08:35:50.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.950+0000 7f485973d640 1 -- start start 2024-04-21T08:35:50.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.950+0000 7f485973d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4854100e70 0x7f4854198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:50.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.950+0000 7f485973d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4854101fc0 0x7f4854198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:50.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.950+0000 7f485973d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4854102980 0x7f485419f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:50.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.950+0000 7f485973d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f485410ca20 con 0x7f4854100e70 2024-04-21T08:35:50.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.950+0000 7f485973d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f485410c8a0 con 0x7f4854101fc0 2024-04-21T08:35:50.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.950+0000 7f485973d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f485410cba0 con 0x7f4854102980 2024-04-21T08:35:50.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.950+0000 7f4852ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4854100e70 0x7f4854198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:50.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.950+0000 7f4852ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4854100e70 0x7f4854198240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39366/0 (socket says 172.21.15.86:39366) 2024-04-21T08:35:50.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.950+0000 7f4852ffd640 1 -- 172.21.15.86:0/3234951497 learned_addr learned my addr 172.21.15.86:0/3234951497 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:35:50.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.950+0000 7f48537fe640 1 --2- 172.21.15.86:0/3234951497 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4854102980 0x7f485419f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:50.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.950+0000 7f48527fc640 1 --2- 172.21.15.86:0/3234951497 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4854101fc0 0x7f4854198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:50.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.950+0000 7f4852ffd640 1 -- 172.21.15.86:0/3234951497 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4854102980 msgr2=0x7f485419f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:50.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.950+0000 7f4852ffd640 1 --2- 172.21.15.86:0/3234951497 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4854102980 0x7f485419f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:50.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.950+0000 7f4852ffd640 1 -- 172.21.15.86:0/3234951497 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4854101fc0 msgr2=0x7f4854198780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:50.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.950+0000 7f4852ffd640 1 --2- 172.21.15.86:0/3234951497 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4854101fc0 0x7f4854198780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:50.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.950+0000 7f4852ffd640 1 -- 172.21.15.86:0/3234951497 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f485419fe60 con 0x7f4854100e70 2024-04-21T08:35:50.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.950+0000 7f4852ffd640 1 --2- 172.21.15.86:0/3234951497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4854100e70 0x7f4854198240 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4844006010 tx=0x7f48440042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:50.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.950+0000 7f483bfff640 1 -- 172.21.15.86:0/3234951497 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4844043020 con 0x7f4854100e70 2024-04-21T08:35:50.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.951+0000 7f485973d640 1 -- 172.21.15.86:0/3234951497 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f48541a00f0 con 0x7f4854100e70 2024-04-21T08:35:50.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.951+0000 7f483bfff640 1 -- 172.21.15.86:0/3234951497 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4844004aa0 con 0x7f4854100e70 2024-04-21T08:35:50.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.951+0000 7f483bfff640 1 -- 172.21.15.86:0/3234951497 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4844003c70 con 0x7f4854100e70 2024-04-21T08:35:50.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.951+0000 7f485973d640 1 -- 172.21.15.86:0/3234951497 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f48541a0710 con 0x7f4854100e70 2024-04-21T08:35:50.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.952+0000 7f485973d640 1 -- 172.21.15.86:0/3234951497 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f48541a0a80 con 0x7f4854100e70 2024-04-21T08:35:50.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.953+0000 7f483bfff640 1 -- 172.21.15.86:0/3234951497 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4844046020 con 0x7f4854100e70 2024-04-21T08:35:50.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.953+0000 7f483bfff640 1 --2- 172.21.15.86:0/3234951497 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f483c0a3c30 0x7f483c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:50.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.953+0000 7f48527fc640 1 --2- 172.21.15.86:0/3234951497 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f483c0a3c30 0x7f483c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:50.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.953+0000 7f483bfff640 1 -- 172.21.15.86:0/3234951497 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f48440f22c0 con 0x7f4854100e70 2024-04-21T08:35:50.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.954+0000 7f48527fc640 1 --2- 172.21.15.86:0/3234951497 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f483c0a3c30 0x7f483c0a60f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4848005fd0 tx=0x7f4848005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:50.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:50.958+0000 7f483bfff640 1 -- 172.21.15.86:0/3234951497 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f484404d020 con 0x7f4854100e70 2024-04-21T08:35:51.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:51.197+0000 7f485973d640 1 -- 172.21.15.86:0/3234951497 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f48541a0c50 con 0x7f4854100e70 2024-04-21T08:35:51.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:51.197+0000 7f483bfff640 1 -- 172.21.15.86:0/3234951497 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 4 v4) ==== 93+0+3046 (secure 0 0 0) 0x7f48440bf140 con 0x7f4854100e70 2024-04-21T08:35:51.198 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:35:51.198 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":4,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-21T08:35:51.198 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 4 2024-04-21T08:35:51.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:51.199+0000 7f485973d640 1 -- 172.21.15.86:0/3234951497 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f483c0a3c30 msgr2=0x7f483c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:51.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:51.199+0000 7f485973d640 1 --2- 172.21.15.86:0/3234951497 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f483c0a3c30 0x7f483c0a60f0 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f4848005fd0 tx=0x7f4848005850 comp rx=0 tx=0).stop 2024-04-21T08:35:51.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:51.199+0000 7f485973d640 1 -- 172.21.15.86:0/3234951497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4854100e70 msgr2=0x7f4854198240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:51.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:51.199+0000 7f485973d640 1 --2- 172.21.15.86:0/3234951497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4854100e70 0x7f4854198240 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7f4844006010 tx=0x7f48440042f0 comp rx=0 tx=0).stop 2024-04-21T08:35:51.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:51.199+0000 7f485973d640 1 -- 172.21.15.86:0/3234951497 shutdown_connections 2024-04-21T08:35:51.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:51.199+0000 7f485973d640 1 --2- 172.21.15.86:0/3234951497 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f483c0a3c30 0x7f483c0a60f0 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:51.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:51.199+0000 7f485973d640 1 --2- 172.21.15.86:0/3234951497 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4854102980 0x7f485419f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:51.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:51.200+0000 7f485973d640 1 --2- 172.21.15.86:0/3234951497 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4854101fc0 0x7f4854198780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:51.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:51.200+0000 7f485973d640 1 --2- 172.21.15.86:0/3234951497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4854100e70 0x7f4854198240 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:51.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:51.200+0000 7f485973d640 1 -- 172.21.15.86:0/3234951497 >> 172.21.15.86:0/3234951497 conn(0x7f48540fc3e0 msgr2=0x7f48540fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:51.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:51.200+0000 7f485973d640 1 -- 172.21.15.86:0/3234951497 shutdown_connections 2024-04-21T08:35:51.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:51.200+0000 7f485973d640 1 -- 172.21.15.86:0/3234951497 wait complete. 2024-04-21T08:35:51.549 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:51 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3234951497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:51.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:51 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3234951497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:51.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:51 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3234951497' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:52.116 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 21 08:35:51 smithi086 systemd[1]: Started Ceph mds.d for f8cec7cc-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:35:52.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:52 smithi086 ceph-mon[27939]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:52.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:52 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:52.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:52 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:52.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:52 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:52.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:52 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:35:52.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:52 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:35:52.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:52 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:52.644 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:52 smithi088 ceph-mon[31259]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:52.644 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:52 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:52.644 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:52 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:52.644 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:52 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:52.644 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:52 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:35:52.644 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:52 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:35:52.644 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:52 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:52.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:52 smithi179 ceph-mon[31829]: pgmap v265: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:52.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:52 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:52.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:52 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:52.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:52 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:52.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:52 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:35:52.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:52 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.e", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:35:52.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:52 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:53.211 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:35:53.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.325+0000 7f5dd759e640 1 -- 172.21.15.86:0/436747193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dd8101db0 msgr2=0x7f5dd81021d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:53.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.325+0000 7f5dd759e640 1 --2- 172.21.15.86:0/436747193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dd8101db0 0x7f5dd81021d0 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8009900 tx=0x7f5dc802bc30 comp rx=0 tx=0).stop 2024-04-21T08:35:53.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.325+0000 7f5dd759e640 1 -- 172.21.15.86:0/436747193 shutdown_connections 2024-04-21T08:35:53.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.325+0000 7f5dd759e640 1 --2- 172.21.15.86:0/436747193 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5dd8103960 0x7f5dd810a1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:53.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.325+0000 7f5dd759e640 1 --2- 172.21.15.86:0/436747193 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5dd8102fa0 0x7f5dd8103420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:53.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.325+0000 7f5dd759e640 1 --2- 172.21.15.86:0/436747193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dd8101db0 0x7f5dd81021d0 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:53.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.326+0000 7f5dd759e640 1 -- 172.21.15.86:0/436747193 >> 172.21.15.86:0/436747193 conn(0x7f5dd80fd320 msgr2=0x7f5dd80ff760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:53.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.326+0000 7f5dd759e640 1 -- 172.21.15.86:0/436747193 shutdown_connections 2024-04-21T08:35:53.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.326+0000 7f5dd759e640 1 -- 172.21.15.86:0/436747193 wait complete. 2024-04-21T08:35:53.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.326+0000 7f5dd759e640 1 Processor -- start 2024-04-21T08:35:53.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.326+0000 7f5dd759e640 1 -- start start 2024-04-21T08:35:53.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.326+0000 7f5dd759e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dd8101db0 0x7f5dd8198020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:53.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.326+0000 7f5dd759e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5dd8102fa0 0x7f5dd8198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:53.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.326+0000 7f5dd759e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5dd8103960 0x7f5dd819f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:53.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.327+0000 7f5dd759e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd810d8e0 con 0x7f5dd8101db0 2024-04-21T08:35:53.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.327+0000 7f5dd759e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd810d760 con 0x7f5dd8102fa0 2024-04-21T08:35:53.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.327+0000 7f5dd759e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5dd810da60 con 0x7f5dd8103960 2024-04-21T08:35:53.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.327+0000 7f5dd659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dd8101db0 0x7f5dd8198020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:53.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.327+0000 7f5dd659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dd8101db0 0x7f5dd8198020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39392/0 (socket says 172.21.15.86:39392) 2024-04-21T08:35:53.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.327+0000 7f5dd659c640 1 -- 172.21.15.86:0/4189546104 learned_addr learned my addr 172.21.15.86:0/4189546104 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:35:53.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.327+0000 7f5dd5d9b640 1 --2- 172.21.15.86:0/4189546104 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5dd8102fa0 0x7f5dd8198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:53.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.327+0000 7f5dd6d9d640 1 --2- 172.21.15.86:0/4189546104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5dd8103960 0x7f5dd819f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:53.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.327+0000 7f5dd659c640 1 -- 172.21.15.86:0/4189546104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5dd8103960 msgr2=0x7f5dd819f5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:53.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.327+0000 7f5dd659c640 1 --2- 172.21.15.86:0/4189546104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5dd8103960 0x7f5dd819f5d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:53.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.327+0000 7f5dd659c640 1 -- 172.21.15.86:0/4189546104 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5dd8102fa0 msgr2=0x7f5dd8198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:53.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.327+0000 7f5dd659c640 1 --2- 172.21.15.86:0/4189546104 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5dd8102fa0 0x7f5dd8198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:53.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.327+0000 7f5dd659c640 1 -- 172.21.15.86:0/4189546104 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5dd819fc40 con 0x7f5dd8101db0 2024-04-21T08:35:53.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.327+0000 7f5dd659c640 1 --2- 172.21.15.86:0/4189546104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dd8101db0 0x7f5dd8198020 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8000c00 tx=0x7f5dc8004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:53.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.327+0000 7f5dc77fe640 1 -- 172.21.15.86:0/4189546104 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc8043020 con 0x7f5dd8101db0 2024-04-21T08:35:53.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.327+0000 7f5dc77fe640 1 -- 172.21.15.86:0/4189546104 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dc8004440 con 0x7f5dd8101db0 2024-04-21T08:35:53.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.327+0000 7f5dd759e640 1 -- 172.21.15.86:0/4189546104 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5dd819fed0 con 0x7f5dd8101db0 2024-04-21T08:35:53.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.327+0000 7f5dc77fe640 1 -- 172.21.15.86:0/4189546104 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dc8003c90 con 0x7f5dd8101db0 2024-04-21T08:35:53.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.327+0000 7f5dd759e640 1 -- 172.21.15.86:0/4189546104 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5dd81a0330 con 0x7f5dd8101db0 2024-04-21T08:35:53.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.328+0000 7f5dd759e640 1 -- 172.21.15.86:0/4189546104 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5dd80690e0 con 0x7f5dd8101db0 2024-04-21T08:35:53.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.329+0000 7f5dc77fe640 1 -- 172.21.15.86:0/4189546104 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5dc8046020 con 0x7f5dd8101db0 2024-04-21T08:35:53.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.330+0000 7f5dc77fe640 1 --2- 172.21.15.86:0/4189546104 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f5dc00a3bb0 0x7f5dc00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:53.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.330+0000 7f5dd5d9b640 1 --2- 172.21.15.86:0/4189546104 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f5dc00a3bb0 0x7f5dc00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:53.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.330+0000 7f5dc77fe640 1 -- 172.21.15.86:0/4189546104 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f5dc80f21b0 con 0x7f5dd8101db0 2024-04-21T08:35:53.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.330+0000 7f5dd5d9b640 1 --2- 172.21.15.86:0/4189546104 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f5dc00a3bb0 0x7f5dc00a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8199530 tx=0x7f5dcc005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:53.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.334+0000 7f5dc77fe640 1 -- 172.21.15.86:0/4189546104 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5dc804d020 con 0x7f5dd8101db0 2024-04-21T08:35:53.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.549+0000 7f5dd759e640 1 -- 172.21.15.86:0/4189546104 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f5dd8109140 con 0x7f5dd8101db0 2024-04-21T08:35:53.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.550+0000 7f5dc77fe640 1 -- 172.21.15.86:0/4189546104 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 5 v5) ==== 93+0+3871 (secure 0 0 0) 0x7f5dc80bf030 con 0x7f5dd8101db0 2024-04-21T08:35:53.550 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:35:53.551 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":5,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3}],"filesystems":[]} 2024-04-21T08:35:53.551 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 5 2024-04-21T08:35:53.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.552+0000 7f5dd759e640 1 -- 172.21.15.86:0/4189546104 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f5dc00a3bb0 msgr2=0x7f5dc00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:53.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.552+0000 7f5dd759e640 1 --2- 172.21.15.86:0/4189546104 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f5dc00a3bb0 0x7f5dc00a6070 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5dd8199530 tx=0x7f5dcc005f90 comp rx=0 tx=0).stop 2024-04-21T08:35:53.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.552+0000 7f5dd759e640 1 -- 172.21.15.86:0/4189546104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dd8101db0 msgr2=0x7f5dd8198020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:53.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.552+0000 7f5dd759e640 1 --2- 172.21.15.86:0/4189546104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dd8101db0 0x7f5dd8198020 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f5dc8000c00 tx=0x7f5dc8004040 comp rx=0 tx=0).stop 2024-04-21T08:35:53.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.552+0000 7f5dd759e640 1 -- 172.21.15.86:0/4189546104 shutdown_connections 2024-04-21T08:35:53.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.552+0000 7f5dd759e640 1 --2- 172.21.15.86:0/4189546104 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f5dc00a3bb0 0x7f5dc00a6070 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:53.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.552+0000 7f5dd759e640 1 --2- 172.21.15.86:0/4189546104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5dd8103960 0x7f5dd819f5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:53.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.552+0000 7f5dd759e640 1 --2- 172.21.15.86:0/4189546104 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5dd8102fa0 0x7f5dd8198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:53.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.552+0000 7f5dd759e640 1 --2- 172.21.15.86:0/4189546104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5dd8101db0 0x7f5dd8198020 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:53.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.552+0000 7f5dd759e640 1 -- 172.21.15.86:0/4189546104 >> 172.21.15.86:0/4189546104 conn(0x7f5dd80fd320 msgr2=0x7f5dd8106310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:53.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.552+0000 7f5dd759e640 1 -- 172.21.15.86:0/4189546104 shutdown_connections 2024-04-21T08:35:53.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:53.552+0000 7f5dd759e640 1 -- 172.21.15.86:0/4189546104 wait complete. 2024-04-21T08:35:53.562 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:53 smithi086 ceph-mon[27939]: Deploying daemon mds.e on smithi088 2024-04-21T08:35:53.562 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:53 smithi086 ceph-mon[27939]: mds.? [v2:172.21.15.86:6836/1247248764,v1:172.21.15.86:6837/1247248764] up:boot 2024-04-21T08:35:53.562 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:53 smithi086 ceph-mon[27939]: fsmap 4 up:standby 2024-04-21T08:35:53.562 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:53 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-21T08:35:53.666 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:53 smithi088 ceph-mon[31259]: Deploying daemon mds.e on smithi088 2024-04-21T08:35:53.667 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:53 smithi088 ceph-mon[31259]: mds.? [v2:172.21.15.86:6836/1247248764,v1:172.21.15.86:6837/1247248764] up:boot 2024-04-21T08:35:53.667 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:53 smithi088 ceph-mon[31259]: fsmap 4 up:standby 2024-04-21T08:35:53.667 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:53 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-21T08:35:53.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:53 smithi179 ceph-mon[31829]: Deploying daemon mds.e on smithi088 2024-04-21T08:35:53.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:53 smithi179 ceph-mon[31829]: mds.? [v2:172.21.15.86:6836/1247248764,v1:172.21.15.86:6837/1247248764] up:boot 2024-04-21T08:35:53.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:53 smithi179 ceph-mon[31829]: fsmap 4 up:standby 2024-04-21T08:35:53.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:53 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-21T08:35:54.344 INFO:journalctl@ceph.mds.e.smithi088.stdout:Apr 21 08:35:54 smithi088 systemd[1]: Starting Ceph mds.e for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:35:54.602 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:54 smithi088 ceph-mon[31259]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:54.602 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:54 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/4189546104' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:54.603 INFO:journalctl@ceph.mds.e.smithi088.stdout:Apr 21 08:35:54 smithi088 podman[60998]: 2024-04-21 08:35:54.345174367 +0000 UTC m=+0.088204933 container create 110f8ccf6dd1697ef6a5d6fa68e2f0261c340962f8ffd1dd29a3a0f794e181df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-e, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416) 2024-04-21T08:35:54.603 INFO:journalctl@ceph.mds.e.smithi088.stdout:Apr 21 08:35:54 smithi088 podman[60998]: 2024-04-21 08:35:54.27553898 +0000 UTC m=+0.018569519 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:35:54.603 INFO:journalctl@ceph.mds.e.smithi088.stdout:Apr 21 08:35:54 smithi088 podman[60998]: 2024-04-21 08:35:54.578030704 +0000 UTC m=+0.321061228 container init 110f8ccf6dd1697ef6a5d6fa68e2f0261c340962f8ffd1dd29a3a0f794e181df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-e, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, GIT_BRANCH=HEAD) 2024-04-21T08:35:54.603 INFO:journalctl@ceph.mds.e.smithi088.stdout:Apr 21 08:35:54 smithi088 podman[60998]: 2024-04-21 08:35:54.580733439 +0000 UTC m=+0.323763964 container start 110f8ccf6dd1697ef6a5d6fa68e2f0261c340962f8ffd1dd29a3a0f794e181df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-e, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-21T08:35:54.603 INFO:journalctl@ceph.mds.e.smithi088.stdout:Apr 21 08:35:54 smithi088 ceph-mds[61016]: -- 172.21.15.88:0/2727444027 <== mon.2 v2:172.21.15.88:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x562dd2a33a40 con 0x562dd2a06c00 2024-04-21T08:35:54.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:54 smithi179 ceph-mon[31829]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:54.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:54 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/4189546104' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:54.800 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:54 smithi086 ceph-mon[27939]: pgmap v266: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:54.800 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:54 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/4189546104' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:54.963 INFO:journalctl@ceph.mds.e.smithi088.stdout:Apr 21 08:35:54 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-e[61012]: starting mds.e at 2024-04-21T08:35:54.964 INFO:journalctl@ceph.mds.e.smithi088.stdout:Apr 21 08:35:54 smithi088 bash[60998]: 110f8ccf6dd1697ef6a5d6fa68e2f0261c340962f8ffd1dd29a3a0f794e181df 2024-04-21T08:35:54.964 INFO:journalctl@ceph.mds.e.smithi088.stdout:Apr 21 08:35:54 smithi088 systemd[1]: Started Ceph mds.e for f8cec7cc-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:35:55.564 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:35:55.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.670+0000 7fb7dd3b6640 1 -- 172.21.15.86:0/2334940146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7d81030f0 msgr2=0x7fb7d8103570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:55.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.670+0000 7fb7dd3b6640 1 --2- 172.21.15.86:0/2334940146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7d81030f0 0x7fb7d8103570 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fb7c0009900 tx=0x7fb7c002bc30 comp rx=0 tx=0).stop 2024-04-21T08:35:55.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.670+0000 7fb7dd3b6640 1 -- 172.21.15.86:0/2334940146 shutdown_connections 2024-04-21T08:35:55.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.670+0000 7fb7dd3b6640 1 --2- 172.21.15.86:0/2334940146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7d8103ab0 0x7fb7d810a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:55.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.670+0000 7fb7dd3b6640 1 --2- 172.21.15.86:0/2334940146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7d81030f0 0x7fb7d8103570 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:55.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.670+0000 7fb7dd3b6640 1 --2- 172.21.15.86:0/2334940146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7d8101f20 0x7fb7d8102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:55.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.670+0000 7fb7dd3b6640 1 -- 172.21.15.86:0/2334940146 >> 172.21.15.86:0/2334940146 conn(0x7fb7d80fd500 msgr2=0x7fb7d80ff920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:55.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.670+0000 7fb7dd3b6640 1 -- 172.21.15.86:0/2334940146 shutdown_connections 2024-04-21T08:35:55.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.670+0000 7fb7dd3b6640 1 -- 172.21.15.86:0/2334940146 wait complete. 2024-04-21T08:35:55.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.671+0000 7fb7dd3b6640 1 Processor -- start 2024-04-21T08:35:55.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.671+0000 7fb7dd3b6640 1 -- start start 2024-04-21T08:35:55.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.671+0000 7fb7dd3b6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7d8101f20 0x7fb7d81981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:55.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.671+0000 7fb7dd3b6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7d81030f0 0x7fb7d81986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:55.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.671+0000 7fb7dd3b6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7d8103ab0 0x7fb7d819f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:55.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.671+0000 7fb7dd3b6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d810daa0 con 0x7fb7d8103ab0 2024-04-21T08:35:55.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.671+0000 7fb7dd3b6640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d810d920 con 0x7fb7d8101f20 2024-04-21T08:35:55.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.671+0000 7fb7dd3b6640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb7d810dc20 con 0x7fb7d81030f0 2024-04-21T08:35:55.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.671+0000 7fb7d77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7d8103ab0 0x7fb7d819f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:55.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.671+0000 7fb7d77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7d8103ab0 0x7fb7d819f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39422/0 (socket says 172.21.15.86:39422) 2024-04-21T08:35:55.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.671+0000 7fb7d77fe640 1 -- 172.21.15.86:0/2236931907 learned_addr learned my addr 172.21.15.86:0/2236931907 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:35:55.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.671+0000 7fb7cffff640 1 --2- 172.21.15.86:0/2236931907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7d81030f0 0x7fb7d81986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:55.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.671+0000 7fb7d6ffd640 1 --2- 172.21.15.86:0/2236931907 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7d8101f20 0x7fb7d81981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:55.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.672+0000 7fb7d77fe640 1 -- 172.21.15.86:0/2236931907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7d81030f0 msgr2=0x7fb7d81986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:55.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.672+0000 7fb7d77fe640 1 --2- 172.21.15.86:0/2236931907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7d81030f0 0x7fb7d81986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:55.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.672+0000 7fb7d77fe640 1 -- 172.21.15.86:0/2236931907 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7d8101f20 msgr2=0x7fb7d81981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:55.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.672+0000 7fb7d77fe640 1 --2- 172.21.15.86:0/2236931907 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7d8101f20 0x7fb7d81981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:55.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.672+0000 7fb7d77fe640 1 -- 172.21.15.86:0/2236931907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb7d819fdc0 con 0x7fb7d8103ab0 2024-04-21T08:35:55.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.672+0000 7fb7d77fe640 1 --2- 172.21.15.86:0/2236931907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7d8103ab0 0x7fb7d819f750 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb7d000eaa0 tx=0x7fb7d000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:55.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.672+0000 7fb7d4ff9640 1 -- 172.21.15.86:0/2236931907 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d000cdb0 con 0x7fb7d8103ab0 2024-04-21T08:35:55.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.672+0000 7fb7d4ff9640 1 -- 172.21.15.86:0/2236931907 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb7d0005020 con 0x7fb7d8103ab0 2024-04-21T08:35:55.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.672+0000 7fb7dd3b6640 1 -- 172.21.15.86:0/2236931907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb7d81a00b0 con 0x7fb7d8103ab0 2024-04-21T08:35:55.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.672+0000 7fb7d4ff9640 1 -- 172.21.15.86:0/2236931907 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7d00116d0 con 0x7fb7d8103ab0 2024-04-21T08:35:55.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.672+0000 7fb7dd3b6640 1 -- 172.21.15.86:0/2236931907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7d81a0620 con 0x7fb7d8103ab0 2024-04-21T08:35:55.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.673+0000 7fb7dd3b6640 1 -- 172.21.15.86:0/2236931907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb7d81093d0 con 0x7fb7d8103ab0 2024-04-21T08:35:55.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.674+0000 7fb7d4ff9640 1 -- 172.21.15.86:0/2236931907 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb7d00051d0 con 0x7fb7d8103ab0 2024-04-21T08:35:55.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.675+0000 7fb7d4ff9640 1 --2- 172.21.15.86:0/2236931907 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb7c40a3c30 0x7fb7c40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:55.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.675+0000 7fb7d6ffd640 1 --2- 172.21.15.86:0/2236931907 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb7c40a3c30 0x7fb7c40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:55.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.675+0000 7fb7d4ff9640 1 -- 172.21.15.86:0/2236931907 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fb7d00d0390 con 0x7fb7d8103ab0 2024-04-21T08:35:55.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.675+0000 7fb7d6ffd640 1 --2- 172.21.15.86:0/2236931907 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb7c40a3c30 0x7fb7c40a60f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8101c40 tx=0x7fb7c8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:55.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.678+0000 7fb7d4ff9640 1 -- 172.21.15.86:0/2236931907 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb7d0016030 con 0x7fb7d8103ab0 2024-04-21T08:35:55.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.897+0000 7fb7dd3b6640 1 -- 172.21.15.86:0/2236931907 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb7d802d5c0 con 0x7fb7d8103ab0 2024-04-21T08:35:55.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.898+0000 7fb7d4ff9640 1 -- 172.21.15.86:0/2236931907 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 6 v6) ==== 93+0+4693 (secure 0 0 0) 0x7fb7d009cc40 con 0x7fb7d8103ab0 2024-04-21T08:35:55.897 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:35:55.898 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":6,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6}],"filesystems":[]} 2024-04-21T08:35:55.898 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 6 2024-04-21T08:35:55.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.900+0000 7fb7dd3b6640 1 -- 172.21.15.86:0/2236931907 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb7c40a3c30 msgr2=0x7fb7c40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:55.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.900+0000 7fb7dd3b6640 1 --2- 172.21.15.86:0/2236931907 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb7c40a3c30 0x7fb7c40a60f0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7fb7d8101c40 tx=0x7fb7c8005f90 comp rx=0 tx=0).stop 2024-04-21T08:35:55.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.900+0000 7fb7dd3b6640 1 -- 172.21.15.86:0/2236931907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7d8103ab0 msgr2=0x7fb7d819f750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:55.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.900+0000 7fb7dd3b6640 1 --2- 172.21.15.86:0/2236931907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7d8103ab0 0x7fb7d819f750 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb7d000eaa0 tx=0x7fb7d000ef60 comp rx=0 tx=0).stop 2024-04-21T08:35:55.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.900+0000 7fb7dd3b6640 1 -- 172.21.15.86:0/2236931907 shutdown_connections 2024-04-21T08:35:55.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.900+0000 7fb7dd3b6640 1 --2- 172.21.15.86:0/2236931907 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb7c40a3c30 0x7fb7c40a60f0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:55.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.900+0000 7fb7dd3b6640 1 --2- 172.21.15.86:0/2236931907 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb7d8103ab0 0x7fb7d819f750 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:55.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.900+0000 7fb7dd3b6640 1 --2- 172.21.15.86:0/2236931907 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb7d81030f0 0x7fb7d81986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:55.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.900+0000 7fb7dd3b6640 1 --2- 172.21.15.86:0/2236931907 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb7d8101f20 0x7fb7d81981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:55.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.900+0000 7fb7dd3b6640 1 -- 172.21.15.86:0/2236931907 >> 172.21.15.86:0/2236931907 conn(0x7fb7d80fd500 msgr2=0x7fb7d81065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:55.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.900+0000 7fb7dd3b6640 1 -- 172.21.15.86:0/2236931907 shutdown_connections 2024-04-21T08:35:55.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:55.900+0000 7fb7dd3b6640 1 -- 172.21.15.86:0/2236931907 wait complete. 2024-04-21T08:35:55.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:55 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:55.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:55 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:55.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:55 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:55.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:55 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:35:55.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:55 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:35:55.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:55 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:55.965 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:55 smithi088 ceph-mon[31259]: Deploying daemon mds.f on smithi179 2024-04-21T08:35:55.965 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:55 smithi088 ceph-mon[31259]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:55.965 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:55 smithi088 ceph-mon[31259]: mds.? [v2:172.21.15.88:6834/454792798,v1:172.21.15.88:6835/454792798] up:boot 2024-04-21T08:35:55.966 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:55 smithi088 ceph-mon[31259]: fsmap 5 up:standby 2024-04-21T08:35:55.966 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:55 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T08:35:55.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:55 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:55.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:55 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:55.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:55 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:55.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:55 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:35:55.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:55 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:35:55.998 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:55 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:55.998 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:55 smithi179 ceph-mon[31829]: Deploying daemon mds.f on smithi179 2024-04-21T08:35:55.998 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:55 smithi179 ceph-mon[31829]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:55.998 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:55 smithi179 ceph-mon[31829]: mds.? [v2:172.21.15.88:6834/454792798,v1:172.21.15.88:6835/454792798] up:boot 2024-04-21T08:35:55.998 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:55 smithi179 ceph-mon[31829]: fsmap 5 up:standby 2024-04-21T08:35:55.998 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:55 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T08:35:56.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:55 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:56.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:55 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:56.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:55 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:56.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:55 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:35:56.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:55 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.f", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:35:56.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:55 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:56.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:55 smithi086 ceph-mon[27939]: Deploying daemon mds.f on smithi179 2024-04-21T08:35:56.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:55 smithi086 ceph-mon[27939]: pgmap v267: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:56.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:55 smithi086 ceph-mon[27939]: mds.? [v2:172.21.15.88:6834/454792798,v1:172.21.15.88:6835/454792798] up:boot 2024-04-21T08:35:56.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:55 smithi086 ceph-mon[27939]: fsmap 5 up:standby 2024-04-21T08:35:56.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:55 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T08:35:56.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:56 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2236931907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:56.997 INFO:journalctl@ceph.mds.f.smithi179.stdout:Apr 21 08:35:56 smithi179 systemd[1]: Starting Ceph mds.f for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:35:57.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:56 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2236931907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:57.213 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:56 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2236931907' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:57.419 INFO:journalctl@ceph.mds.f.smithi179.stdout:Apr 21 08:35:57 smithi179 podman[61189]: 2024-04-21 08:35:56.950468327 +0000 UTC m=+0.018106172 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:35:57.419 INFO:journalctl@ceph.mds.f.smithi179.stdout:Apr 21 08:35:57 smithi179 podman[61189]: 2024-04-21 08:35:57.075736304 +0000 UTC m=+0.143374181 container create 09df6c287e69ceb971ba0df1c95ef1af6ad29207bc52e979b7e28541e439b270 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-f, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True) 2024-04-21T08:35:57.419 INFO:journalctl@ceph.mds.f.smithi179.stdout:Apr 21 08:35:57 smithi179 podman[61189]: 2024-04-21 08:35:57.420334805 +0000 UTC m=+0.487972659 container init 09df6c287e69ceb971ba0df1c95ef1af6ad29207bc52e979b7e28541e439b270 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-f, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, ceph=True, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T08:35:57.746 INFO:journalctl@ceph.mds.f.smithi179.stdout:Apr 21 08:35:57 smithi179 podman[61189]: 2024-04-21 08:35:57.422938427 +0000 UTC m=+0.490576272 container start 09df6c287e69ceb971ba0df1c95ef1af6ad29207bc52e979b7e28541e439b270 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-f, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=) 2024-04-21T08:35:57.747 INFO:journalctl@ceph.mds.f.smithi179.stdout:Apr 21 08:35:57 smithi179 ceph-mds[61208]: -- 172.21.15.179:0/2408321269 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5620d1a29a40 con 0x5620d19fd000 2024-04-21T08:35:57.747 INFO:journalctl@ceph.mds.f.smithi179.stdout:Apr 21 08:35:57 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-f[61204]: starting mds.f at 2024-04-21T08:35:57.747 INFO:journalctl@ceph.mds.f.smithi179.stdout:Apr 21 08:35:57 smithi179 bash[61189]: 09df6c287e69ceb971ba0df1c95ef1af6ad29207bc52e979b7e28541e439b270 2024-04-21T08:35:57.747 INFO:journalctl@ceph.mds.f.smithi179.stdout:Apr 21 08:35:57 smithi179 systemd[1]: Started Ceph mds.f for f8cec7cc-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:35:57.911 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:35:58.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.022+0000 7f0912421640 1 -- 172.21.15.86:0/3304642558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f090c100ea0 msgr2=0x7f090c1012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:58.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.022+0000 7f0912421640 1 --2- 172.21.15.86:0/3304642558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f090c100ea0 0x7f090c1012c0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f0900009900 tx=0x7f090002bc30 comp rx=0 tx=0).stop 2024-04-21T08:35:58.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.022+0000 7f0912421640 1 -- 172.21.15.86:0/3304642558 shutdown_connections 2024-04-21T08:35:58.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.022+0000 7f0912421640 1 --2- 172.21.15.86:0/3304642558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f090c1029b0 0x7f090c1092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:58.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.022+0000 7f0912421640 1 --2- 172.21.15.86:0/3304642558 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f090c101ff0 0x7f090c102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:58.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.022+0000 7f0912421640 1 --2- 172.21.15.86:0/3304642558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f090c100ea0 0x7f090c1012c0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:58.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.022+0000 7f0912421640 1 -- 172.21.15.86:0/3304642558 >> 172.21.15.86:0/3304642558 conn(0x7f090c0fc410 msgr2=0x7f090c0fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:58.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.022+0000 7f0912421640 1 -- 172.21.15.86:0/3304642558 shutdown_connections 2024-04-21T08:35:58.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.022+0000 7f0912421640 1 -- 172.21.15.86:0/3304642558 wait complete. 2024-04-21T08:35:58.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.022+0000 7f0912421640 1 Processor -- start 2024-04-21T08:35:58.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.022+0000 7f0912421640 1 -- start start 2024-04-21T08:35:58.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.023+0000 7f0912421640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f090c100ea0 0x7f090c1981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:58.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.023+0000 7f0912421640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f090c101ff0 0x7f090c198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:58.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.023+0000 7f0912421640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f090c1029b0 0x7f090c19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:58.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.023+0000 7f0912421640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f090c10dba0 con 0x7f090c101ff0 2024-04-21T08:35:58.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.023+0000 7f0912421640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f090c10da20 con 0x7f090c1029b0 2024-04-21T08:35:58.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.023+0000 7f0912421640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f090c10dd20 con 0x7f090c100ea0 2024-04-21T08:35:58.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.023+0000 7f090b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f090c101ff0 0x7f090c198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:58.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.023+0000 7f090b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f090c101ff0 0x7f090c198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39454/0 (socket says 172.21.15.86:39454) 2024-04-21T08:35:58.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.023+0000 7f090b7fe640 1 -- 172.21.15.86:0/1721429065 learned_addr learned my addr 172.21.15.86:0/1721429065 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:35:58.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.023+0000 7f090bfff640 1 --2- 172.21.15.86:0/1721429065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f090c100ea0 0x7f090c1981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:58.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.023+0000 7f09109be640 1 --2- 172.21.15.86:0/1721429065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f090c1029b0 0x7f090c19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:58.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.023+0000 7f090b7fe640 1 -- 172.21.15.86:0/1721429065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f090c100ea0 msgr2=0x7f090c1981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:58.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.023+0000 7f090b7fe640 1 --2- 172.21.15.86:0/1721429065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f090c100ea0 0x7f090c1981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:58.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.023+0000 7f090b7fe640 1 -- 172.21.15.86:0/1721429065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f090c1029b0 msgr2=0x7f090c19f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:58.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.023+0000 7f090b7fe640 1 --2- 172.21.15.86:0/1721429065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f090c1029b0 0x7f090c19f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:58.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.023+0000 7f090b7fe640 1 -- 172.21.15.86:0/1721429065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f090c19fde0 con 0x7f090c101ff0 2024-04-21T08:35:58.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.023+0000 7f090b7fe640 1 --2- 172.21.15.86:0/1721429065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f090c101ff0 0x7f090c198700 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f08fc00e9c0 tx=0x7f08fc00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:58.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.023+0000 7f09097fa640 1 -- 172.21.15.86:0/1721429065 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08fc00cd80 con 0x7f090c101ff0 2024-04-21T08:35:58.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.023+0000 7f0912421640 1 -- 172.21.15.86:0/1721429065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f090c1a00d0 con 0x7f090c101ff0 2024-04-21T08:35:58.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.023+0000 7f09097fa640 1 -- 172.21.15.86:0/1721429065 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f08fc0035f0 con 0x7f090c101ff0 2024-04-21T08:35:58.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.023+0000 7f09097fa640 1 -- 172.21.15.86:0/1721429065 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08fc010690 con 0x7f090c101ff0 2024-04-21T08:35:58.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.023+0000 7f0912421640 1 -- 172.21.15.86:0/1721429065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f090c1a0620 con 0x7f090c101ff0 2024-04-21T08:35:58.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.024+0000 7f0912421640 1 -- 172.21.15.86:0/1721429065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f090c108230 con 0x7f090c101ff0 2024-04-21T08:35:58.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.025+0000 7f09097fa640 1 -- 172.21.15.86:0/1721429065 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f08fc004000 con 0x7f090c101ff0 2024-04-21T08:35:58.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.026+0000 7f09097fa640 1 --2- 172.21.15.86:0/1721429065 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f08f40a3bb0 0x7f08f40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:35:58.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.026+0000 7f090bfff640 1 --2- 172.21.15.86:0/1721429065 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f08f40a3bb0 0x7f08f40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:35:58.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.026+0000 7f09097fa640 1 -- 172.21.15.86:0/1721429065 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f08fc0d0450 con 0x7f090c101ff0 2024-04-21T08:35:58.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.026+0000 7f090bfff640 1 --2- 172.21.15.86:0/1721429065 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f08f40a3bb0 0x7f08f40a6070 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0900000c00 tx=0x7f090002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:35:58.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.031+0000 7f09097fa640 1 -- 172.21.15.86:0/1721429065 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f08fc09d2d0 con 0x7f090c101ff0 2024-04-21T08:35:58.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:57 smithi086 ceph-mon[27939]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:58.141 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:57 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:58.141 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:57 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:58.141 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:57 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:58.141 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:57 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:35:58.141 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:57 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:35:58.141 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:57 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:58.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:57 smithi088 ceph-mon[31259]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:58.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:57 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:58.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:57 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:58.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:57 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:58.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:57 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:35:58.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:57 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:35:58.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:57 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:58.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:57 smithi179 ceph-mon[31829]: pgmap v268: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:35:58.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:57 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:58.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:57 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:58.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:57 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:35:58.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:57 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:35:58.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:57 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.c", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:35:58.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:57 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:35:58.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.251+0000 7f0912421640 1 -- 172.21.15.86:0/1721429065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f090c02d7c0 con 0x7f090c101ff0 2024-04-21T08:35:58.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.251+0000 7f09097fa640 1 -- 172.21.15.86:0/1721429065 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5521 (secure 0 0 0) 0x7f08fc09cbd0 con 0x7f090c101ff0 2024-04-21T08:35:58.250 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:35:58.250 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-21T08:35:58.254 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 7 2024-04-21T08:35:58.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.253+0000 7f0912421640 1 -- 172.21.15.86:0/1721429065 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f08f40a3bb0 msgr2=0x7f08f40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:58.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.253+0000 7f0912421640 1 --2- 172.21.15.86:0/1721429065 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f08f40a3bb0 0x7f08f40a6070 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7f0900000c00 tx=0x7f090002c7e0 comp rx=0 tx=0).stop 2024-04-21T08:35:58.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.253+0000 7f0912421640 1 -- 172.21.15.86:0/1721429065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f090c101ff0 msgr2=0x7f090c198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:35:58.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.253+0000 7f0912421640 1 --2- 172.21.15.86:0/1721429065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f090c101ff0 0x7f090c198700 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7f08fc00e9c0 tx=0x7f08fc00ee80 comp rx=0 tx=0).stop 2024-04-21T08:35:58.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.253+0000 7f0912421640 1 -- 172.21.15.86:0/1721429065 shutdown_connections 2024-04-21T08:35:58.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.253+0000 7f0912421640 1 --2- 172.21.15.86:0/1721429065 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f08f40a3bb0 0x7f08f40a6070 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:58.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.253+0000 7f0912421640 1 --2- 172.21.15.86:0/1721429065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f090c1029b0 0x7f090c19f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:58.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.253+0000 7f0912421640 1 --2- 172.21.15.86:0/1721429065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f090c101ff0 0x7f090c198700 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:58.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.253+0000 7f0912421640 1 --2- 172.21.15.86:0/1721429065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f090c100ea0 0x7f090c1981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:35:58.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.253+0000 7f0912421640 1 -- 172.21.15.86:0/1721429065 >> 172.21.15.86:0/1721429065 conn(0x7f090c0fc410 msgr2=0x7f090c105400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:35:58.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.253+0000 7f0912421640 1 -- 172.21.15.86:0/1721429065 shutdown_connections 2024-04-21T08:35:58.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:35:58.254+0000 7f0912421640 1 -- 172.21.15.86:0/1721429065 wait complete. 2024-04-21T08:35:58.961 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:58 smithi179 ceph-mon[31829]: Deploying daemon mds.c on smithi179 2024-04-21T08:35:58.961 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:58 smithi179 ceph-mon[31829]: mds.? [v2:172.21.15.179:6834/3343050063,v1:172.21.15.179:6835/3343050063] up:boot 2024-04-21T08:35:58.961 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:58 smithi179 ceph-mon[31829]: fsmap 6 up:standby 2024-04-21T08:35:58.961 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:58 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T08:35:58.961 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:58 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1721429065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:59.039 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:58 smithi088 ceph-mon[31259]: Deploying daemon mds.c on smithi179 2024-04-21T08:35:59.039 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:58 smithi088 ceph-mon[31259]: mds.? [v2:172.21.15.179:6834/3343050063,v1:172.21.15.179:6835/3343050063] up:boot 2024-04-21T08:35:59.040 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:58 smithi088 ceph-mon[31259]: fsmap 6 up:standby 2024-04-21T08:35:59.040 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:58 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T08:35:59.040 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:58 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1721429065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:59.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:58 smithi086 ceph-mon[27939]: Deploying daemon mds.c on smithi179 2024-04-21T08:35:59.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:58 smithi086 ceph-mon[27939]: mds.? [v2:172.21.15.179:6834/3343050063,v1:172.21.15.179:6835/3343050063] up:boot 2024-04-21T08:35:59.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:58 smithi086 ceph-mon[27939]: fsmap 6 up:standby 2024-04-21T08:35:59.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:58 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T08:35:59.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:58 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1721429065' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:35:59.946 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:35:59 smithi179 ceph-mon[31829]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:00.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:35:59 smithi086 ceph-mon[27939]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:00.213 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:35:59 smithi088 ceph-mon[31259]: pgmap v269: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:00.264 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:36:00.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.377+0000 7f646633d640 1 -- 172.21.15.86:0/1787669761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6460102040 msgr2=0x7f64601024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:00.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.377+0000 7f646633d640 1 --2- 172.21.15.86:0/1787669761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6460102040 0x7f64601024c0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f6454009900 tx=0x7f645402bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:00.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.377+0000 7f646633d640 1 -- 172.21.15.86:0/1787669761 shutdown_connections 2024-04-21T08:36:00.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.377+0000 7f646633d640 1 --2- 172.21.15.86:0/1787669761 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6460102a00 0x7f6460109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:00.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.377+0000 7f646633d640 1 --2- 172.21.15.86:0/1787669761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6460102040 0x7f64601024c0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:00.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.377+0000 7f646633d640 1 --2- 172.21.15.86:0/1787669761 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6460100e60 0x7f6460101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:00.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.377+0000 7f646633d640 1 -- 172.21.15.86:0/1787669761 >> 172.21.15.86:0/1787669761 conn(0x7f64600fc420 msgr2=0x7f64600fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:00.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.377+0000 7f646633d640 1 -- 172.21.15.86:0/1787669761 shutdown_connections 2024-04-21T08:36:00.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.377+0000 7f646633d640 1 -- 172.21.15.86:0/1787669761 wait complete. 2024-04-21T08:36:00.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.377+0000 7f646633d640 1 Processor -- start 2024-04-21T08:36:00.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.377+0000 7f646633d640 1 -- start start 2024-04-21T08:36:00.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.378+0000 7f646633d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6460100e60 0x7f64601970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:00.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.378+0000 7f646633d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6460102040 0x7f64601975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:00.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.378+0000 7f646633d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6460102a00 0x7f646019e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:00.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.378+0000 7f646633d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f646010ca10 con 0x7f6460100e60 2024-04-21T08:36:00.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.378+0000 7f646633d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f646010c890 con 0x7f6460102040 2024-04-21T08:36:00.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.378+0000 7f646633d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f646010cb90 con 0x7f6460102a00 2024-04-21T08:36:00.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.378+0000 7f645ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6460100e60 0x7f64601970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:00.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.378+0000 7f645ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6460100e60 0x7f64601970b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38192/0 (socket says 172.21.15.86:38192) 2024-04-21T08:36:00.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.378+0000 7f645ffff640 1 -- 172.21.15.86:0/1667820392 learned_addr learned my addr 172.21.15.86:0/1667820392 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:00.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.378+0000 7f64648da640 1 --2- 172.21.15.86:0/1667820392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6460102a00 0x7f646019e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:00.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.378+0000 7f645f7fe640 1 --2- 172.21.15.86:0/1667820392 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6460102040 0x7f64601975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:00.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.378+0000 7f645ffff640 1 -- 172.21.15.86:0/1667820392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6460102a00 msgr2=0x7f646019e660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:00.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.378+0000 7f645ffff640 1 --2- 172.21.15.86:0/1667820392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6460102a00 0x7f646019e660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:00.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.378+0000 7f645ffff640 1 -- 172.21.15.86:0/1667820392 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6460102040 msgr2=0x7f64601975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:00.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.378+0000 7f645ffff640 1 --2- 172.21.15.86:0/1667820392 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6460102040 0x7f64601975f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:00.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.378+0000 7f645ffff640 1 -- 172.21.15.86:0/1667820392 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f646019ecd0 con 0x7f6460100e60 2024-04-21T08:36:00.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.378+0000 7f645ffff640 1 --2- 172.21.15.86:0/1667820392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6460100e60 0x7f64601970b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f645000e9c0 tx=0x7f645000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:00.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.378+0000 7f645d7fa640 1 -- 172.21.15.86:0/1667820392 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f645000cd80 con 0x7f6460100e60 2024-04-21T08:36:00.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.378+0000 7f645d7fa640 1 -- 172.21.15.86:0/1667820392 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64500035f0 con 0x7f6460100e60 2024-04-21T08:36:00.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.378+0000 7f646633d640 1 -- 172.21.15.86:0/1667820392 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f646019efc0 con 0x7f6460100e60 2024-04-21T08:36:00.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.378+0000 7f645d7fa640 1 -- 172.21.15.86:0/1667820392 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6450010690 con 0x7f6460100e60 2024-04-21T08:36:00.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.378+0000 7f646633d640 1 -- 172.21.15.86:0/1667820392 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f646019f420 con 0x7f6460100e60 2024-04-21T08:36:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.380+0000 7f646633d640 1 -- 172.21.15.86:0/1667820392 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6460108200 con 0x7f6460100e60 2024-04-21T08:36:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.381+0000 7f645d7fa640 1 -- 172.21.15.86:0/1667820392 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6450004000 con 0x7f6460100e60 2024-04-21T08:36:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.381+0000 7f645d7fa640 1 --2- 172.21.15.86:0/1667820392 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f64480a3c30 0x7f64480a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:00.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.381+0000 7f645f7fe640 1 --2- 172.21.15.86:0/1667820392 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f64480a3c30 0x7f64480a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:00.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.381+0000 7f645d7fa640 1 -- 172.21.15.86:0/1667820392 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f64500d0470 con 0x7f6460100e60 2024-04-21T08:36:00.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.382+0000 7f645f7fe640 1 --2- 172.21.15.86:0/1667820392 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f64480a3c30 0x7f64480a60f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6454036040 tx=0x7f6454000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:00.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.386+0000 7f645d7fa640 1 -- 172.21.15.86:0/1667820392 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f645009d2f0 con 0x7f6460100e60 2024-04-21T08:36:00.496 INFO:journalctl@ceph.mds.c.smithi179.stdout:Apr 21 08:36:00 smithi179 systemd[1]: Started Ceph mds.c for f8cec7cc-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:36:00.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.604+0000 7f646633d640 1 -- 172.21.15.86:0/1667820392 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f646002d5c0 con 0x7f6460100e60 2024-04-21T08:36:00.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.605+0000 7f645d7fa640 1 -- 172.21.15.86:0/1667820392 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 7 v7) ==== 93+0+5521 (secure 0 0 0) 0x7f645009cbf0 con 0x7f6460100e60 2024-04-21T08:36:00.604 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:36:00.604 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":7,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7}],"filesystems":[]} 2024-04-21T08:36:00.605 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 7 2024-04-21T08:36:00.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.607+0000 7f646633d640 1 -- 172.21.15.86:0/1667820392 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f64480a3c30 msgr2=0x7f64480a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:00.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.607+0000 7f646633d640 1 --2- 172.21.15.86:0/1667820392 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f64480a3c30 0x7f64480a60f0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6454036040 tx=0x7f6454000bc0 comp rx=0 tx=0).stop 2024-04-21T08:36:00.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.607+0000 7f646633d640 1 -- 172.21.15.86:0/1667820392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6460100e60 msgr2=0x7f64601970b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:00.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.607+0000 7f646633d640 1 --2- 172.21.15.86:0/1667820392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6460100e60 0x7f64601970b0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7f645000e9c0 tx=0x7f645000ee80 comp rx=0 tx=0).stop 2024-04-21T08:36:00.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.607+0000 7f646633d640 1 -- 172.21.15.86:0/1667820392 shutdown_connections 2024-04-21T08:36:00.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.607+0000 7f646633d640 1 --2- 172.21.15.86:0/1667820392 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f64480a3c30 0x7f64480a60f0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:00.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.607+0000 7f646633d640 1 --2- 172.21.15.86:0/1667820392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6460102a00 0x7f646019e660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:00.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.607+0000 7f646633d640 1 --2- 172.21.15.86:0/1667820392 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6460102040 0x7f64601975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:00.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.607+0000 7f646633d640 1 --2- 172.21.15.86:0/1667820392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6460100e60 0x7f64601970b0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:00.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.607+0000 7f646633d640 1 -- 172.21.15.86:0/1667820392 >> 172.21.15.86:0/1667820392 conn(0x7f64600fc420 msgr2=0x7f64601053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:00.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.607+0000 7f646633d640 1 -- 172.21.15.86:0/1667820392 shutdown_connections 2024-04-21T08:36:00.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:00.607+0000 7f646633d640 1 -- 172.21.15.86:0/1667820392 wait complete. 2024-04-21T08:36:01.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:01 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:01.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:01 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:01.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:01 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:01.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:01 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:36:01.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:01 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:36:01.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:01 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:01.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:01 smithi088 ceph-mon[31259]: Deploying daemon mds.h on smithi088 2024-04-21T08:36:01.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:01 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1667820392' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:01.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:01 smithi088 ceph-mon[31259]: mds.? [v2:172.21.15.179:6836/1280529894,v1:172.21.15.179:6837/1280529894] up:boot 2024-04-21T08:36:01.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:01 smithi088 ceph-mon[31259]: fsmap 7 up:standby 2024-04-21T08:36:01.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:01 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T08:36:01.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:01 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:01.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:01 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:01.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:01 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:01.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:01 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:36:01.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:01 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:36:01.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:01 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:01.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:01 smithi179 ceph-mon[31829]: Deploying daemon mds.h on smithi088 2024-04-21T08:36:01.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:01 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1667820392' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:01.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:01 smithi179 ceph-mon[31829]: mds.? [v2:172.21.15.179:6836/1280529894,v1:172.21.15.179:6837/1280529894] up:boot 2024-04-21T08:36:01.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:01 smithi179 ceph-mon[31829]: fsmap 7 up:standby 2024-04-21T08:36:01.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:01 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T08:36:01.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:01 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:01.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:01 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:01.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:01 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:01.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:01 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:36:01.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:01 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.h", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:36:01.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:01 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:01.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:01 smithi086 ceph-mon[27939]: Deploying daemon mds.h on smithi088 2024-04-21T08:36:01.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:01 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1667820392' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:01.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:01 smithi086 ceph-mon[27939]: mds.? [v2:172.21.15.179:6836/1280529894,v1:172.21.15.179:6837/1280529894] up:boot 2024-04-21T08:36:01.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:01 smithi086 ceph-mon[27939]: fsmap 7 up:standby 2024-04-21T08:36:01.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:01 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T08:36:02.451 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:02 smithi088 ceph-mon[31259]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:02.618 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:36:02.644 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:02 smithi086 ceph-mon[27939]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:02.647 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:02 smithi179 ceph-mon[31829]: pgmap v270: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:02.713 INFO:journalctl@ceph.mds.h.smithi088.stdout:Apr 21 08:36:02 smithi088 systemd[1]: Starting Ceph mds.h for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:36:02.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.729+0000 7fa190596640 1 -- 172.21.15.86:0/29524009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa188109e20 msgr2=0x7fa18810c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:02.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.729+0000 7fa190596640 1 --2- 172.21.15.86:0/29524009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa188109e20 0x7fa18810c2c0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa180009ca0 tx=0x7fa18002c180 comp rx=0 tx=0).stop 2024-04-21T08:36:02.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.729+0000 7fa190596640 1 -- 172.21.15.86:0/29524009 shutdown_connections 2024-04-21T08:36:02.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.729+0000 7fa190596640 1 --2- 172.21.15.86:0/29524009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa188109e20 0x7fa18810c2c0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:02.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.729+0000 7fa190596640 1 --2- 172.21.15.86:0/29524009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa188074950 0x7fa188074db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:02.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.729+0000 7fa190596640 1 --2- 172.21.15.86:0/29524009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa188075f30 0x7fa188074380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:02.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.729+0000 7fa190596640 1 -- 172.21.15.86:0/29524009 >> 172.21.15.86:0/29524009 conn(0x7fa1880fd500 msgr2=0x7fa1880ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:02.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.729+0000 7fa190596640 1 -- 172.21.15.86:0/29524009 shutdown_connections 2024-04-21T08:36:02.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.729+0000 7fa190596640 1 -- 172.21.15.86:0/29524009 wait complete. 2024-04-21T08:36:02.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.730+0000 7fa190596640 1 Processor -- start 2024-04-21T08:36:02.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.730+0000 7fa190596640 1 -- start start 2024-04-21T08:36:02.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.730+0000 7fa190596640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa188074950 0x7fa18819a2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:02.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.730+0000 7fa190596640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa188075f30 0x7fa18819a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:02.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.730+0000 7fa190596640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa188109e20 0x7fa1881a1850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:02.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.730+0000 7fa190596640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa18810fcd0 con 0x7fa188074950 2024-04-21T08:36:02.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.730+0000 7fa190596640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa18810fb50 con 0x7fa188075f30 2024-04-21T08:36:02.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.730+0000 7fa190596640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa18810fe50 con 0x7fa188109e20 2024-04-21T08:36:02.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.730+0000 7fa18e332640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa188074950 0x7fa18819a2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:02.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.730+0000 7fa18e332640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa188074950 0x7fa18819a2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38206/0 (socket says 172.21.15.86:38206) 2024-04-21T08:36:02.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.730+0000 7fa18db31640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa188075f30 0x7fa18819a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:02.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.730+0000 7fa18e332640 1 -- 172.21.15.86:0/3863971902 learned_addr learned my addr 172.21.15.86:0/3863971902 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:02.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.730+0000 7fa18eb33640 1 --2- 172.21.15.86:0/3863971902 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa188109e20 0x7fa1881a1850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:02.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.730+0000 7fa18e332640 1 -- 172.21.15.86:0/3863971902 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa188109e20 msgr2=0x7fa1881a1850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:02.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.730+0000 7fa18e332640 1 --2- 172.21.15.86:0/3863971902 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa188109e20 0x7fa1881a1850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:02.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.730+0000 7fa18e332640 1 -- 172.21.15.86:0/3863971902 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa188075f30 msgr2=0x7fa18819a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:02.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.730+0000 7fa18e332640 1 --2- 172.21.15.86:0/3863971902 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa188075f30 0x7fa18819a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:02.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.731+0000 7fa18e332640 1 -- 172.21.15.86:0/3863971902 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1881a1ec0 con 0x7fa188074950 2024-04-21T08:36:02.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.731+0000 7fa18e332640 1 --2- 172.21.15.86:0/3863971902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa188074950 0x7fa18819a2a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa18400c920 tx=0x7fa18400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:02.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.731+0000 7fa17f7fe640 1 -- 172.21.15.86:0/3863971902 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa184007c70 con 0x7fa188074950 2024-04-21T08:36:02.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.731+0000 7fa17f7fe640 1 -- 172.21.15.86:0/3863971902 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa184007e10 con 0x7fa188074950 2024-04-21T08:36:02.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.731+0000 7fa17f7fe640 1 -- 172.21.15.86:0/3863971902 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa184018820 con 0x7fa188074950 2024-04-21T08:36:02.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.731+0000 7fa190596640 1 -- 172.21.15.86:0/3863971902 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1881a21b0 con 0x7fa188074950 2024-04-21T08:36:02.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.731+0000 7fa190596640 1 -- 172.21.15.86:0/3863971902 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1881a2700 con 0x7fa188074950 2024-04-21T08:36:02.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.732+0000 7fa190596640 1 -- 172.21.15.86:0/3863971902 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa188108be0 con 0x7fa188074950 2024-04-21T08:36:02.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.733+0000 7fa17f7fe640 1 -- 172.21.15.86:0/3863971902 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa18400f450 con 0x7fa188074950 2024-04-21T08:36:02.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.734+0000 7fa17f7fe640 1 --2- 172.21.15.86:0/3863971902 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa1740a3bb0 0x7fa1740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:02.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.734+0000 7fa18db31640 1 --2- 172.21.15.86:0/3863971902 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa1740a3bb0 0x7fa1740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:02.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.734+0000 7fa17f7fe640 1 -- 172.21.15.86:0/3863971902 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fa1840d0070 con 0x7fa188074950 2024-04-21T08:36:02.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.734+0000 7fa18db31640 1 --2- 172.21.15.86:0/3863971902 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa1740a3bb0 0x7fa1740a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa18819b7b0 tx=0x7fa17800b390 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:02.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.739+0000 7fa17f7fe640 1 -- 172.21.15.86:0/3863971902 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa184014030 con 0x7fa188074950 2024-04-21T08:36:02.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.954+0000 7fa190596640 1 -- 172.21.15.86:0/3863971902 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa18802d5c0 con 0x7fa188074950 2024-04-21T08:36:02.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.955+0000 7fa17f7fe640 1 -- 172.21.15.86:0/3863971902 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 8 v8) ==== 93+0+6349 (secure 0 0 0) 0x7fa18409cef0 con 0x7fa188074950 2024-04-21T08:36:02.954 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:36:02.954 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":8,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8}],"filesystems":[]} 2024-04-21T08:36:02.954 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 8 2024-04-21T08:36:02.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.957+0000 7fa190596640 1 -- 172.21.15.86:0/3863971902 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa1740a3bb0 msgr2=0x7fa1740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:02.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.957+0000 7fa190596640 1 --2- 172.21.15.86:0/3863971902 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa1740a3bb0 0x7fa1740a6070 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7fa18819b7b0 tx=0x7fa17800b390 comp rx=0 tx=0).stop 2024-04-21T08:36:02.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.957+0000 7fa190596640 1 -- 172.21.15.86:0/3863971902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa188074950 msgr2=0x7fa18819a2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:02.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.957+0000 7fa190596640 1 --2- 172.21.15.86:0/3863971902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa188074950 0x7fa18819a2a0 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7fa18400c920 tx=0x7fa18400cde0 comp rx=0 tx=0).stop 2024-04-21T08:36:02.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.957+0000 7fa190596640 1 -- 172.21.15.86:0/3863971902 shutdown_connections 2024-04-21T08:36:02.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.957+0000 7fa190596640 1 --2- 172.21.15.86:0/3863971902 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa1740a3bb0 0x7fa1740a6070 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:02.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.957+0000 7fa190596640 1 --2- 172.21.15.86:0/3863971902 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa188109e20 0x7fa1881a1850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:02.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.957+0000 7fa190596640 1 --2- 172.21.15.86:0/3863971902 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa188075f30 0x7fa18819a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:02.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.957+0000 7fa190596640 1 --2- 172.21.15.86:0/3863971902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa188074950 0x7fa18819a2a0 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:02.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.957+0000 7fa190596640 1 -- 172.21.15.86:0/3863971902 >> 172.21.15.86:0/3863971902 conn(0x7fa1880fd500 msgr2=0x7fa188105db0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:02.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.957+0000 7fa190596640 1 -- 172.21.15.86:0/3863971902 shutdown_connections 2024-04-21T08:36:02.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:02.957+0000 7fa190596640 1 -- 172.21.15.86:0/3863971902 wait complete. 2024-04-21T08:36:03.114 INFO:journalctl@ceph.mds.h.smithi088.stdout:Apr 21 08:36:02 smithi088 podman[61313]: 2024-04-21 08:36:02.756401713 +0000 UTC m=+0.018004555 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:36:03.114 INFO:journalctl@ceph.mds.h.smithi088.stdout:Apr 21 08:36:02 smithi088 podman[61313]: 2024-04-21 08:36:02.879889939 +0000 UTC m=+0.141492770 container create 708fa1f6099be6c56bdec537b258d5537c11fe9947cdbaba5bb5481d491f843b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-h, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:36:03.115 INFO:journalctl@ceph.mds.h.smithi088.stdout:Apr 21 08:36:03 smithi088 podman[61313]: 2024-04-21 08:36:03.116025997 +0000 UTC m=+0.377628828 container init 708fa1f6099be6c56bdec537b258d5537c11fe9947cdbaba5bb5481d491f843b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-h, GIT_BRANCH=HEAD, GIT_CLEAN=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux ) 2024-04-21T08:36:03.374 INFO:journalctl@ceph.mds.h.smithi088.stdout:Apr 21 08:36:03 smithi088 podman[61313]: 2024-04-21 08:36:03.118676659 +0000 UTC m=+0.380279499 container start 708fa1f6099be6c56bdec537b258d5537c11fe9947cdbaba5bb5481d491f843b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-h, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=) 2024-04-21T08:36:03.374 INFO:journalctl@ceph.mds.h.smithi088.stdout:Apr 21 08:36:03 smithi088 ceph-mds[61333]: -- 172.21.15.88:0/2687725014 <== mon.2 v2:172.21.15.88:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55a9d7f0da40 con 0x55a9d7ee0c00 2024-04-21T08:36:03.375 INFO:journalctl@ceph.mds.h.smithi088.stdout:Apr 21 08:36:03 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-h[61329]: starting mds.h at 2024-04-21T08:36:03.375 INFO:journalctl@ceph.mds.h.smithi088.stdout:Apr 21 08:36:03 smithi088 bash[61313]: 708fa1f6099be6c56bdec537b258d5537c11fe9947cdbaba5bb5481d491f843b 2024-04-21T08:36:03.375 INFO:journalctl@ceph.mds.h.smithi088.stdout:Apr 21 08:36:03 smithi088 systemd[1]: Started Ceph mds.h for f8cec7cc-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:36:03.574 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:03 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3863971902' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:03.574 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:03 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:03.574 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:03 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:03.574 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:03 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:03.574 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:03 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:36:03.575 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:03 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:36:03.575 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:03 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:03.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:03 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3863971902' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:03.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:03 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:03.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:03 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:03.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:03 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:03.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:03 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:36:03.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:03 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:36:03.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:03 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:03.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:03 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3863971902' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:03.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:03 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:03.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:03 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:03.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:03 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:03.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:03 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:36:03.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:03 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.i", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:36:03.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:03 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:04.534 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:04 smithi179 ceph-mon[31829]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:04.535 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:04 smithi179 ceph-mon[31829]: Deploying daemon mds.i on smithi179 2024-04-21T08:36:04.535 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:04 smithi179 ceph-mon[31829]: mds.? [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] up:boot 2024-04-21T08:36:04.535 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:04 smithi179 ceph-mon[31829]: fsmap 8 up:standby 2024-04-21T08:36:04.535 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:04 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T08:36:04.629 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:04 smithi088 ceph-mon[31259]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:04.629 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:04 smithi088 ceph-mon[31259]: Deploying daemon mds.i on smithi179 2024-04-21T08:36:04.629 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:04 smithi088 ceph-mon[31259]: mds.? [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] up:boot 2024-04-21T08:36:04.629 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:04 smithi088 ceph-mon[31259]: fsmap 8 up:standby 2024-04-21T08:36:04.630 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:04 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T08:36:04.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:04 smithi086 ceph-mon[27939]: pgmap v271: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:04.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:04 smithi086 ceph-mon[27939]: Deploying daemon mds.i on smithi179 2024-04-21T08:36:04.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:04 smithi086 ceph-mon[27939]: mds.? [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] up:boot 2024-04-21T08:36:04.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:04 smithi086 ceph-mon[27939]: fsmap 8 up:standby 2024-04-21T08:36:04.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:04 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T08:36:04.968 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:36:05.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.078+0000 7f43ceff4640 1 -- 172.21.15.86:0/3880936976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43c8103b40 msgr2=0x7f43c810a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:05.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.078+0000 7f43ceff4640 1 --2- 172.21.15.86:0/3880936976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43c8103b40 0x7f43c810a3d0 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f43c0009ca0 tx=0x7f43c002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:36:05.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.078+0000 7f43ceff4640 1 -- 172.21.15.86:0/3880936976 shutdown_connections 2024-04-21T08:36:05.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.078+0000 7f43ceff4640 1 --2- 172.21.15.86:0/3880936976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43c8103b40 0x7f43c810a3d0 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:05.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.078+0000 7f43ceff4640 1 --2- 172.21.15.86:0/3880936976 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43c8103180 0x7f43c8103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:05.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.078+0000 7f43ceff4640 1 --2- 172.21.15.86:0/3880936976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43c8101f90 0x7f43c81023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:05.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.078+0000 7f43ceff4640 1 -- 172.21.15.86:0/3880936976 >> 172.21.15.86:0/3880936976 conn(0x7f43c80fd500 msgr2=0x7f43c80ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:05.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.079+0000 7f43ceff4640 1 -- 172.21.15.86:0/3880936976 shutdown_connections 2024-04-21T08:36:05.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.079+0000 7f43ceff4640 1 -- 172.21.15.86:0/3880936976 wait complete. 2024-04-21T08:36:05.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.079+0000 7f43ceff4640 1 Processor -- start 2024-04-21T08:36:05.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.079+0000 7f43ceff4640 1 -- start start 2024-04-21T08:36:05.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.079+0000 7f43ceff4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43c8101f90 0x7f43c8198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:05.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.079+0000 7f43ceff4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43c8103180 0x7f43c8198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:05.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.079+0000 7f43ceff4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43c8103b40 0x7f43c819f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:05.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.079+0000 7f43ceff4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f43c810dad0 con 0x7f43c8103b40 2024-04-21T08:36:05.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.079+0000 7f43ceff4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f43c810d950 con 0x7f43c8103180 2024-04-21T08:36:05.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.079+0000 7f43ceff4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f43c810dc50 con 0x7f43c8101f90 2024-04-21T08:36:05.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.079+0000 7f43cd591640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43c8103b40 0x7f43c819f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:05.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.079+0000 7f43cd591640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43c8103b40 0x7f43c819f6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38238/0 (socket says 172.21.15.86:38238) 2024-04-21T08:36:05.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.079+0000 7f43cd591640 1 -- 172.21.15.86:0/314985700 learned_addr learned my addr 172.21.15.86:0/314985700 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:05.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.079+0000 7f43c7fff640 1 --2- 172.21.15.86:0/314985700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43c8103180 0x7f43c8198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:05.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.079+0000 7f43ccd90640 1 --2- 172.21.15.86:0/314985700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43c8101f90 0x7f43c8198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:05.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.080+0000 7f43cd591640 1 -- 172.21.15.86:0/314985700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43c8101f90 msgr2=0x7f43c8198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:05.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.080+0000 7f43cd591640 1 --2- 172.21.15.86:0/314985700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43c8101f90 0x7f43c8198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:05.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.080+0000 7f43cd591640 1 -- 172.21.15.86:0/314985700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43c8103180 msgr2=0x7f43c8198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:05.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.080+0000 7f43cd591640 1 --2- 172.21.15.86:0/314985700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43c8103180 0x7f43c8198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:05.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.080+0000 7f43cd591640 1 -- 172.21.15.86:0/314985700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43c819fd50 con 0x7f43c8103b40 2024-04-21T08:36:05.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.080+0000 7f43cd591640 1 --2- 172.21.15.86:0/314985700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43c8103b40 0x7f43c819f6e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f43c002c700 tx=0x7f43c0034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:05.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.080+0000 7f43c5ffb640 1 -- 172.21.15.86:0/314985700 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c0039040 con 0x7f43c8103b40 2024-04-21T08:36:05.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.080+0000 7f43ceff4640 1 -- 172.21.15.86:0/314985700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43c819ffe0 con 0x7f43c8103b40 2024-04-21T08:36:05.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.080+0000 7f43c5ffb640 1 -- 172.21.15.86:0/314985700 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43c0044070 con 0x7f43c8103b40 2024-04-21T08:36:05.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.080+0000 7f43c5ffb640 1 -- 172.21.15.86:0/314985700 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43c00042a0 con 0x7f43c8103b40 2024-04-21T08:36:05.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.080+0000 7f43ceff4640 1 -- 172.21.15.86:0/314985700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43c81a0440 con 0x7f43c8103b40 2024-04-21T08:36:05.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.081+0000 7f43ceff4640 1 -- 172.21.15.86:0/314985700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f43c81092b0 con 0x7f43c8103b40 2024-04-21T08:36:05.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.082+0000 7f43c5ffb640 1 -- 172.21.15.86:0/314985700 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f43c0003970 con 0x7f43c8103b40 2024-04-21T08:36:05.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.083+0000 7f43c5ffb640 1 --2- 172.21.15.86:0/314985700 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f43b00a3bb0 0x7f43b00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:05.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.083+0000 7f43c5ffb640 1 -- 172.21.15.86:0/314985700 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f43c00f2e10 con 0x7f43c8103b40 2024-04-21T08:36:05.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.083+0000 7f43ccd90640 1 --2- 172.21.15.86:0/314985700 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f43b00a3bb0 0x7f43b00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:05.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.083+0000 7f43ccd90640 1 --2- 172.21.15.86:0/314985700 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f43b00a3bb0 0x7f43b00a6070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f43b8005fd0 tx=0x7f43b8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:05.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.086+0000 7f43c5ffb640 1 -- 172.21.15.86:0/314985700 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f43c00bfdb0 con 0x7f43c8103b40 2024-04-21T08:36:05.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.303+0000 7f43ceff4640 1 -- 172.21.15.86:0/314985700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f43c802d5c0 con 0x7f43c8103b40 2024-04-21T08:36:05.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.304+0000 7f43c5ffb640 1 -- 172.21.15.86:0/314985700 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 9 v9) ==== 93+0+7174 (secure 0 0 0) 0x7f43c00bf6b0 con 0x7f43c8103b40 2024-04-21T08:36:05.303 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:36:05.304 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":9,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9}],"filesystems":[]} 2024-04-21T08:36:05.304 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 9 2024-04-21T08:36:05.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.306+0000 7f43ceff4640 1 -- 172.21.15.86:0/314985700 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f43b00a3bb0 msgr2=0x7f43b00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:05.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.306+0000 7f43ceff4640 1 --2- 172.21.15.86:0/314985700 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f43b00a3bb0 0x7f43b00a6070 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f43b8005fd0 tx=0x7f43b8005850 comp rx=0 tx=0).stop 2024-04-21T08:36:05.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.306+0000 7f43ceff4640 1 -- 172.21.15.86:0/314985700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43c8103b40 msgr2=0x7f43c819f6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:05.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.306+0000 7f43ceff4640 1 --2- 172.21.15.86:0/314985700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43c8103b40 0x7f43c819f6e0 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f43c002c700 tx=0x7f43c0034760 comp rx=0 tx=0).stop 2024-04-21T08:36:05.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.306+0000 7f43ceff4640 1 -- 172.21.15.86:0/314985700 shutdown_connections 2024-04-21T08:36:05.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.306+0000 7f43ceff4640 1 --2- 172.21.15.86:0/314985700 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f43b00a3bb0 0x7f43b00a6070 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:05.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.306+0000 7f43ceff4640 1 --2- 172.21.15.86:0/314985700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43c8103b40 0x7f43c819f6e0 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:05.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.306+0000 7f43ceff4640 1 --2- 172.21.15.86:0/314985700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43c8103180 0x7f43c8198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:05.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.306+0000 7f43ceff4640 1 --2- 172.21.15.86:0/314985700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43c8101f90 0x7f43c8198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:05.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.306+0000 7f43ceff4640 1 -- 172.21.15.86:0/314985700 >> 172.21.15.86:0/314985700 conn(0x7f43c80fd500 msgr2=0x7f43c8106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:05.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.306+0000 7f43ceff4640 1 -- 172.21.15.86:0/314985700 shutdown_connections 2024-04-21T08:36:05.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:05.306+0000 7f43ceff4640 1 -- 172.21.15.86:0/314985700 wait complete. 2024-04-21T08:36:05.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:05 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/314985700' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:05.497 INFO:journalctl@ceph.mds.i.smithi179.stdout:Apr 21 08:36:05 smithi179 systemd[1]: Starting Ceph mds.i for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:36:05.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:05 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/314985700' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:05.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:05 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/314985700' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:05.902 INFO:journalctl@ceph.mds.i.smithi179.stdout:Apr 21 08:36:05 smithi179 podman[61810]: 2024-04-21 08:36:05.543281689 +0000 UTC m=+0.018367847 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:36:05.902 INFO:journalctl@ceph.mds.i.smithi179.stdout:Apr 21 08:36:05 smithi179 podman[61810]: 2024-04-21 08:36:05.650959772 +0000 UTC m=+0.126045917 container create 52eb9c4c876e38889f661b5004d26dfbc43101596a38f9fb78c738e7b59e00f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-i, org.label-schema.vendor=CentOS, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-21T08:36:05.902 INFO:journalctl@ceph.mds.i.smithi179.stdout:Apr 21 08:36:05 smithi179 podman[61810]: 2024-04-21 08:36:05.904070214 +0000 UTC m=+0.379156366 container init 52eb9c4c876e38889f661b5004d26dfbc43101596a38f9fb78c738e7b59e00f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-i, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, ceph=True, io.buildah.version=1.33.5) 2024-04-21T08:36:06.246 INFO:journalctl@ceph.mds.i.smithi179.stdout:Apr 21 08:36:05 smithi179 podman[61810]: 2024-04-21 08:36:05.906793494 +0000 UTC m=+0.381879649 container start 52eb9c4c876e38889f661b5004d26dfbc43101596a38f9fb78c738e7b59e00f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-i, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD) 2024-04-21T08:36:06.246 INFO:journalctl@ceph.mds.i.smithi179.stdout:Apr 21 08:36:05 smithi179 ceph-mds[61830]: -- 172.21.15.179:0/3869807203 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5611cc341a40 con 0x5611cc314800 2024-04-21T08:36:06.247 INFO:journalctl@ceph.mds.i.smithi179.stdout:Apr 21 08:36:05 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-i[61826]: starting mds.i at 2024-04-21T08:36:06.247 INFO:journalctl@ceph.mds.i.smithi179.stdout:Apr 21 08:36:06 smithi179 bash[61810]: 52eb9c4c876e38889f661b5004d26dfbc43101596a38f9fb78c738e7b59e00f5 2024-04-21T08:36:06.247 INFO:journalctl@ceph.mds.i.smithi179.stdout:Apr 21 08:36:06 smithi179 systemd[1]: Started Ceph mds.i for f8cec7cc-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:36:06.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:06 smithi088 ceph-mon[31259]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:06.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:06 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:06.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:06 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:06.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:06 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:06.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:06 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:36:06.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:06 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:36:06.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:06 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:06.734 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:06 smithi086 ceph-mon[27939]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:06.734 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:06 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:06.734 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:06 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:06.734 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:06 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:06.734 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:06 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:36:06.734 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:06 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:36:06.734 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:06 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:06.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:06 smithi179 ceph-mon[31829]: pgmap v272: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:06.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:06 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:06.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:06 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:06.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:06 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:06.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:06 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:36:06.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:06 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.g", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:36:06.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:06 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:07.317 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:36:07.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.425+0000 7f228b89a640 1 -- 172.21.15.86:0/2346101208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2284101fc0 msgr2=0x7f2284102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:07.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.425+0000 7f228b89a640 1 --2- 172.21.15.86:0/2346101208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2284101fc0 0x7f2284102440 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f2274009900 tx=0x7f227402bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:07.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.425+0000 7f228b89a640 1 -- 172.21.15.86:0/2346101208 shutdown_connections 2024-04-21T08:36:07.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.425+0000 7f228b89a640 1 --2- 172.21.15.86:0/2346101208 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2284102980 0x7f22841092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:07.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.425+0000 7f228b89a640 1 --2- 172.21.15.86:0/2346101208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2284101fc0 0x7f2284102440 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:07.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.425+0000 7f228b89a640 1 --2- 172.21.15.86:0/2346101208 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2284100e70 0x7f2284101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:07.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.425+0000 7f228b89a640 1 -- 172.21.15.86:0/2346101208 >> 172.21.15.86:0/2346101208 conn(0x7f22840fc3e0 msgr2=0x7f22840fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:07.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.425+0000 7f228b89a640 1 -- 172.21.15.86:0/2346101208 shutdown_connections 2024-04-21T08:36:07.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.425+0000 7f228b89a640 1 -- 172.21.15.86:0/2346101208 wait complete. 2024-04-21T08:36:07.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.425+0000 7f228b89a640 1 Processor -- start 2024-04-21T08:36:07.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.425+0000 7f228b89a640 1 -- start start 2024-04-21T08:36:07.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.426+0000 7f228b89a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2284100e70 0x7f2284198390 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:07.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.426+0000 7f228b89a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2284101fc0 0x7f22841988d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:07.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.426+0000 7f228b89a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2284102980 0x7f228419f8f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:07.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.426+0000 7f228b89a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f228410caa0 con 0x7f2284102980 2024-04-21T08:36:07.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.426+0000 7f228b89a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f228410c920 con 0x7f2284101fc0 2024-04-21T08:36:07.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.426+0000 7f228b89a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f228410cc20 con 0x7f2284100e70 2024-04-21T08:36:07.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.426+0000 7f2289e37640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2284102980 0x7f228419f8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:07.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.426+0000 7f2289e37640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2284102980 0x7f228419f8f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38256/0 (socket says 172.21.15.86:38256) 2024-04-21T08:36:07.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.426+0000 7f2289e37640 1 -- 172.21.15.86:0/3601783821 learned_addr learned my addr 172.21.15.86:0/3601783821 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:07.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.426+0000 7f2289636640 1 --2- 172.21.15.86:0/3601783821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2284100e70 0x7f2284198390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:07.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.426+0000 7f2288e35640 1 --2- 172.21.15.86:0/3601783821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2284101fc0 0x7f22841988d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:07.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.426+0000 7f2289e37640 1 -- 172.21.15.86:0/3601783821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2284100e70 msgr2=0x7f2284198390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:07.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.426+0000 7f2289e37640 1 --2- 172.21.15.86:0/3601783821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2284100e70 0x7f2284198390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:07.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.426+0000 7f2289e37640 1 -- 172.21.15.86:0/3601783821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2284101fc0 msgr2=0x7f22841988d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:07.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.426+0000 7f2289e37640 1 --2- 172.21.15.86:0/3601783821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2284101fc0 0x7f22841988d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:07.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.426+0000 7f2289e37640 1 -- 172.21.15.86:0/3601783821 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f228419ff60 con 0x7f2284102980 2024-04-21T08:36:07.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.426+0000 7f2289636640 1 --2- 172.21.15.86:0/3601783821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2284100e70 0x7f2284198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:36:07.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.426+0000 7f2289e37640 1 --2- 172.21.15.86:0/3601783821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2284102980 0x7f228419f8f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f228000b970 tx=0x7f228000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:07.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.426+0000 7f227a7fc640 1 -- 172.21.15.86:0/3601783821 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2280019070 con 0x7f2284102980 2024-04-21T08:36:07.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.426+0000 7f228b89a640 1 -- 172.21.15.86:0/3601783821 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22841a0250 con 0x7f2284102980 2024-04-21T08:36:07.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.426+0000 7f227a7fc640 1 -- 172.21.15.86:0/3601783821 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2280014440 con 0x7f2284102980 2024-04-21T08:36:07.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.426+0000 7f227a7fc640 1 -- 172.21.15.86:0/3601783821 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22800075d0 con 0x7f2284102980 2024-04-21T08:36:07.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.426+0000 7f228b89a640 1 -- 172.21.15.86:0/3601783821 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22841a07d0 con 0x7f2284102980 2024-04-21T08:36:07.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.427+0000 7f228b89a640 1 -- 172.21.15.86:0/3601783821 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f22841a0a90 con 0x7f2284102980 2024-04-21T08:36:07.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.428+0000 7f227a7fc640 1 -- 172.21.15.86:0/3601783821 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f228001c070 con 0x7f2284102980 2024-04-21T08:36:07.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.429+0000 7f227a7fc640 1 --2- 172.21.15.86:0/3601783821 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f22700a3bb0 0x7f22700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:07.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.429+0000 7f2289636640 1 --2- 172.21.15.86:0/3601783821 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f22700a3bb0 0x7f22700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:07.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.429+0000 7f227a7fc640 1 -- 172.21.15.86:0/3601783821 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f22800cfad0 con 0x7f2284102980 2024-04-21T08:36:07.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.429+0000 7f2289636640 1 --2- 172.21.15.86:0/3601783821 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f22700a3bb0 0x7f22700a6070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f227c006fd0 tx=0x7f227c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:07.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.432+0000 7f227a7fc640 1 -- 172.21.15.86:0/3601783821 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f228009c950 con 0x7f2284102980 2024-04-21T08:36:07.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:07 smithi086 ceph-mon[27939]: Deploying daemon mds.g on smithi086 2024-04-21T08:36:07.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:07 smithi086 ceph-mon[27939]: mds.? [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] up:boot 2024-04-21T08:36:07.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:07 smithi086 ceph-mon[27939]: fsmap 9 up:standby 2024-04-21T08:36:07.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:07 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T08:36:07.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.651+0000 7f228b89a640 1 -- 172.21.15.86:0/3601783821 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f22841a0c60 con 0x7f2284102980 2024-04-21T08:36:07.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.651+0000 7f227a7fc640 1 -- 172.21.15.86:0/3601783821 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 10 v10) ==== 94+0+8004 (secure 0 0 0) 0x7f228009c250 con 0x7f2284102980 2024-04-21T08:36:07.651 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:36:07.651 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":10,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-21T08:36:07.651 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 10 2024-04-21T08:36:07.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.654+0000 7f228b89a640 1 -- 172.21.15.86:0/3601783821 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f22700a3bb0 msgr2=0x7f22700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:07.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.654+0000 7f228b89a640 1 --2- 172.21.15.86:0/3601783821 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f22700a3bb0 0x7f22700a6070 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f227c006fd0 tx=0x7f227c008040 comp rx=0 tx=0).stop 2024-04-21T08:36:07.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.654+0000 7f228b89a640 1 -- 172.21.15.86:0/3601783821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2284102980 msgr2=0x7f228419f8f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:07.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.654+0000 7f228b89a640 1 --2- 172.21.15.86:0/3601783821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2284102980 0x7f228419f8f0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f228000b970 tx=0x7f228000be30 comp rx=0 tx=0).stop 2024-04-21T08:36:07.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.654+0000 7f228b89a640 1 -- 172.21.15.86:0/3601783821 shutdown_connections 2024-04-21T08:36:07.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.654+0000 7f228b89a640 1 --2- 172.21.15.86:0/3601783821 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f22700a3bb0 0x7f22700a6070 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:07.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.654+0000 7f228b89a640 1 --2- 172.21.15.86:0/3601783821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2284102980 0x7f228419f8f0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:07.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.654+0000 7f228b89a640 1 --2- 172.21.15.86:0/3601783821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2284101fc0 0x7f22841988d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:07.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.654+0000 7f228b89a640 1 --2- 172.21.15.86:0/3601783821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2284100e70 0x7f2284198390 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:07.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.654+0000 7f228b89a640 1 -- 172.21.15.86:0/3601783821 >> 172.21.15.86:0/3601783821 conn(0x7f22840fc3e0 msgr2=0x7f22840690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:07.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.654+0000 7f228b89a640 1 -- 172.21.15.86:0/3601783821 shutdown_connections 2024-04-21T08:36:07.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:07.654+0000 7f228b89a640 1 -- 172.21.15.86:0/3601783821 wait complete. 2024-04-21T08:36:07.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:07 smithi088 ceph-mon[31259]: Deploying daemon mds.g on smithi086 2024-04-21T08:36:07.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:07 smithi088 ceph-mon[31259]: mds.? [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] up:boot 2024-04-21T08:36:07.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:07 smithi088 ceph-mon[31259]: fsmap 9 up:standby 2024-04-21T08:36:07.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:07 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T08:36:07.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:07 smithi179 ceph-mon[31829]: Deploying daemon mds.g on smithi086 2024-04-21T08:36:07.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:07 smithi179 ceph-mon[31829]: mds.? [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] up:boot 2024-04-21T08:36:07.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:07 smithi179 ceph-mon[31829]: fsmap 9 up:standby 2024-04-21T08:36:07.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:07 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T08:36:08.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:08 smithi086 ceph-mon[27939]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:08.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:08 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3601783821' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:08.617 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 21 08:36:08 smithi086 podman[66803]: 2024-04-21 08:36:08.54492601 +0000 UTC m=+0.122114676 container create 6d381c165a0f6fe94ee0bde8fc729ef3a4a385dc93c419bf64372ca94c2654da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-g, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.license=GPLv2, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T08:36:08.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:08 smithi088 ceph-mon[31259]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:08.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:08 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3601783821' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:08.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:08 smithi179 ceph-mon[31829]: pgmap v273: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:08.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:08 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3601783821' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:09.117 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 21 08:36:08 smithi086 podman[66803]: 2024-04-21 08:36:08.814299947 +0000 UTC m=+0.391488619 container init 6d381c165a0f6fe94ee0bde8fc729ef3a4a385dc93c419bf64372ca94c2654da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-g, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2) 2024-04-21T08:36:09.117 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 21 08:36:08 smithi086 podman[66803]: 2024-04-21 08:36:08.816864264 +0000 UTC m=+0.394052923 container start 6d381c165a0f6fe94ee0bde8fc729ef3a4a385dc93c419bf64372ca94c2654da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-g, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:36:09.117 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 21 08:36:08 smithi086 ceph-mds[66823]: -- 172.21.15.86:0/2573705486 <== mon.0 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55fad7d59a40 con 0x55fad7d2cc00 2024-04-21T08:36:09.117 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 21 08:36:08 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-g[66819]: starting mds.g at 2024-04-21T08:36:09.117 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 21 08:36:08 smithi086 bash[66803]: 6d381c165a0f6fe94ee0bde8fc729ef3a4a385dc93c419bf64372ca94c2654da 2024-04-21T08:36:09.117 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 21 08:36:08 smithi086 systemd[1]: Started Ceph mds.g for f8cec7cc-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:36:09.664 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:36:09.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.769+0000 7f7f21967640 1 -- 172.21.15.86:0/2853433341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f1c1029f0 msgr2=0x7f7f1c104de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:09.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.769+0000 7f7f21967640 1 --2- 172.21.15.86:0/2853433341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f1c1029f0 0x7f7f1c104de0 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c009900 tx=0x7f7f0c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:09.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.769+0000 7f7f21967640 1 -- 172.21.15.86:0/2853433341 shutdown_connections 2024-04-21T08:36:09.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.769+0000 7f7f21967640 1 --2- 172.21.15.86:0/2853433341 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f1c107c50 0x7f7f1c10a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:09.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.769+0000 7f7f21967640 1 --2- 172.21.15.86:0/2853433341 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f1c105320 0x7f7f1c107710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:09.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.769+0000 7f7f21967640 1 --2- 172.21.15.86:0/2853433341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f1c1029f0 0x7f7f1c104de0 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:09.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.769+0000 7f7f21967640 1 -- 172.21.15.86:0/2853433341 >> 172.21.15.86:0/2853433341 conn(0x7f7f1c0fc3e0 msgr2=0x7f7f1c0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:09.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.770+0000 7f7f21967640 1 -- 172.21.15.86:0/2853433341 shutdown_connections 2024-04-21T08:36:09.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.770+0000 7f7f21967640 1 -- 172.21.15.86:0/2853433341 wait complete. 2024-04-21T08:36:09.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.770+0000 7f7f21967640 1 Processor -- start 2024-04-21T08:36:09.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.770+0000 7f7f21967640 1 -- start start 2024-04-21T08:36:09.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.770+0000 7f7f21967640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f1c1029f0 0x7f7f1c198140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:09.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.770+0000 7f7f21967640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f1c105320 0x7f7f1c198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:09.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.770+0000 7f7f21967640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f1c107c50 0x7f7f1c19f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:09.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.771+0000 7f7f21967640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1c10daf0 con 0x7f7f1c1029f0 2024-04-21T08:36:09.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.771+0000 7f7f21967640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1c10d970 con 0x7f7f1c107c50 2024-04-21T08:36:09.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.771+0000 7f7f21967640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7f1c10dc70 con 0x7f7f1c105320 2024-04-21T08:36:09.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.771+0000 7f7f1affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f1c1029f0 0x7f7f1c198140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:09.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.771+0000 7f7f1affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f1c1029f0 0x7f7f1c198140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50694/0 (socket says 172.21.15.86:50694) 2024-04-21T08:36:09.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.771+0000 7f7f1affd640 1 -- 172.21.15.86:0/1501344174 learned_addr learned my addr 172.21.15.86:0/1501344174 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:09.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.771+0000 7f7f1a7fc640 1 --2- 172.21.15.86:0/1501344174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f1c105320 0x7f7f1c198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:09.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.771+0000 7f7f1affd640 1 -- 172.21.15.86:0/1501344174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f1c105320 msgr2=0x7f7f1c198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:09.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.771+0000 7f7f1affd640 1 --2- 172.21.15.86:0/1501344174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f1c105320 0x7f7f1c198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:09.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.771+0000 7f7f1affd640 1 -- 172.21.15.86:0/1501344174 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f1c107c50 msgr2=0x7f7f1c19f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:09.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.771+0000 7f7f1affd640 1 --2- 172.21.15.86:0/1501344174 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f1c107c50 0x7f7f1c19f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:09.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.771+0000 7f7f1affd640 1 -- 172.21.15.86:0/1501344174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7f1c19fd60 con 0x7f7f1c1029f0 2024-04-21T08:36:09.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.771+0000 7f7f1affd640 1 --2- 172.21.15.86:0/1501344174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f1c1029f0 0x7f7f1c198140 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c000c00 tx=0x7f7f0c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:09.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.771+0000 7f7f20965640 1 -- 172.21.15.86:0/1501344174 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c043020 con 0x7f7f1c1029f0 2024-04-21T08:36:09.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.771+0000 7f7f20965640 1 -- 172.21.15.86:0/1501344174 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7f0c0052b0 con 0x7f7f1c1029f0 2024-04-21T08:36:09.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.771+0000 7f7f20965640 1 -- 172.21.15.86:0/1501344174 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7f0c003cc0 con 0x7f7f1c1029f0 2024-04-21T08:36:09.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.771+0000 7f7f21967640 1 -- 172.21.15.86:0/1501344174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7f1c19fff0 con 0x7f7f1c1029f0 2024-04-21T08:36:09.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.771+0000 7f7f21967640 1 -- 172.21.15.86:0/1501344174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7f1c1a0460 con 0x7f7f1c1029f0 2024-04-21T08:36:09.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.772+0000 7f7f21967640 1 -- 172.21.15.86:0/1501344174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7f1c0fded0 con 0x7f7f1c1029f0 2024-04-21T08:36:09.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.773+0000 7f7f20965640 1 -- 172.21.15.86:0/1501344174 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f7f0c046020 con 0x7f7f1c1029f0 2024-04-21T08:36:09.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.774+0000 7f7f20965640 1 --2- 172.21.15.86:0/1501344174 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f7f040a3c00 0x7f7f040a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:09.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.774+0000 7f7f1a7fc640 1 --2- 172.21.15.86:0/1501344174 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f7f040a3c00 0x7f7f040a60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:09.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.774+0000 7f7f20965640 1 -- 172.21.15.86:0/1501344174 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f7f0c0f2480 con 0x7f7f1c1029f0 2024-04-21T08:36:09.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.774+0000 7f7f1a7fc640 1 --2- 172.21.15.86:0/1501344174 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f7f040a3c00 0x7f7f040a60c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7f10005e10 tx=0x7f7f10005da0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:09.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.778+0000 7f7f20965640 1 -- 172.21.15.86:0/1501344174 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7f0c04d020 con 0x7f7f1c1029f0 2024-04-21T08:36:09.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.998+0000 7f7f21967640 1 -- 172.21.15.86:0/1501344174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7f1c02d790 con 0x7f7f1c1029f0 2024-04-21T08:36:09.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:09.999+0000 7f7f20965640 1 -- 172.21.15.86:0/1501344174 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 11 v11) ==== 94+0+8830 (secure 0 0 0) 0x7f7f0c0bf300 con 0x7f7f1c1029f0 2024-04-21T08:36:09.998 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:36:09.998 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":11,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-21T08:36:09.999 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 11 2024-04-21T08:36:10.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:10.001+0000 7f7f21967640 1 -- 172.21.15.86:0/1501344174 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f7f040a3c00 msgr2=0x7f7f040a60c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:10.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:10.001+0000 7f7f21967640 1 --2- 172.21.15.86:0/1501344174 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f7f040a3c00 0x7f7f040a60c0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7f7f10005e10 tx=0x7f7f10005da0 comp rx=0 tx=0).stop 2024-04-21T08:36:10.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:10.001+0000 7f7f21967640 1 -- 172.21.15.86:0/1501344174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f1c1029f0 msgr2=0x7f7f1c198140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:10.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:10.001+0000 7f7f21967640 1 --2- 172.21.15.86:0/1501344174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f1c1029f0 0x7f7f1c198140 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f7f0c000c00 tx=0x7f7f0c004270 comp rx=0 tx=0).stop 2024-04-21T08:36:10.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:10.001+0000 7f7f21967640 1 -- 172.21.15.86:0/1501344174 shutdown_connections 2024-04-21T08:36:10.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:10.001+0000 7f7f21967640 1 --2- 172.21.15.86:0/1501344174 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f7f040a3c00 0x7f7f040a60c0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:10.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:10.001+0000 7f7f21967640 1 --2- 172.21.15.86:0/1501344174 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7f1c107c50 0x7f7f1c19f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:10.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:10.001+0000 7f7f21967640 1 --2- 172.21.15.86:0/1501344174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7f1c105320 0x7f7f1c198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:10.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:10.001+0000 7f7f21967640 1 --2- 172.21.15.86:0/1501344174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7f1c1029f0 0x7f7f1c198140 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:10.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:10.001+0000 7f7f21967640 1 -- 172.21.15.86:0/1501344174 >> 172.21.15.86:0/1501344174 conn(0x7f7f1c0fc3e0 msgr2=0x7f7f1c103430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:10.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:10.001+0000 7f7f21967640 1 -- 172.21.15.86:0/1501344174 shutdown_connections 2024-04-21T08:36:10.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:10.001+0000 7f7f21967640 1 -- 172.21.15.86:0/1501344174 wait complete. 2024-04-21T08:36:10.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:09 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:10.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:09 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:10.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:09 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:10.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:09 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:36:10.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:09 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:36:10.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:09 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:10.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:09 smithi088 ceph-mon[31259]: Deploying daemon mds.a on smithi086 2024-04-21T08:36:10.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:09 smithi088 ceph-mon[31259]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:10.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:09 smithi088 ceph-mon[31259]: mds.? [v2:172.21.15.86:6838/3163123023,v1:172.21.15.86:6839/3163123023] up:boot 2024-04-21T08:36:10.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:09 smithi088 ceph-mon[31259]: fsmap 10 up:standby 2024-04-21T08:36:10.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:09 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-21T08:36:10.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:09 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:10.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:09 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:10.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:09 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:10.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:09 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:36:10.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:09 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:36:10.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:09 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:10.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:09 smithi179 ceph-mon[31829]: Deploying daemon mds.a on smithi086 2024-04-21T08:36:10.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:09 smithi179 ceph-mon[31829]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:10.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:09 smithi179 ceph-mon[31829]: mds.? [v2:172.21.15.86:6838/3163123023,v1:172.21.15.86:6839/3163123023] up:boot 2024-04-21T08:36:10.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:09 smithi179 ceph-mon[31829]: fsmap 10 up:standby 2024-04-21T08:36:10.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:09 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-21T08:36:10.275 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:09 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:10.275 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:09 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:10.275 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:09 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:10.275 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:09 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:36:10.275 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:09 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.a", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:36:10.275 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:09 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:10.275 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:09 smithi086 ceph-mon[27939]: Deploying daemon mds.a on smithi086 2024-04-21T08:36:10.275 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:09 smithi086 ceph-mon[27939]: pgmap v274: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:10.275 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:09 smithi086 ceph-mon[27939]: mds.? [v2:172.21.15.86:6838/3163123023,v1:172.21.15.86:6839/3163123023] up:boot 2024-04-21T08:36:10.276 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:09 smithi086 ceph-mon[27939]: fsmap 10 up:standby 2024-04-21T08:36:10.276 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:09 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-21T08:36:10.996 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:10 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1501344174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:11.213 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:10 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1501344174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:11.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:10 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1501344174' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:11.617 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 21 08:36:11 smithi086 podman[67163]: 2024-04-21 08:36:11.47570974 +0000 UTC m=+0.141632499 container create f9fbd33c53f616b8450d58512bcd779ef660061774504dcff95fba8739070fa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-a, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T08:36:11.969 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 21 08:36:11 smithi086 podman[67163]: 2024-04-21 08:36:11.761602111 +0000 UTC m=+0.427524872 container init f9fbd33c53f616b8450d58512bcd779ef660061774504dcff95fba8739070fa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-a, ceph=True, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-21T08:36:11.970 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 21 08:36:11 smithi086 podman[67163]: 2024-04-21 08:36:11.764153036 +0000 UTC m=+0.430075798 container start f9fbd33c53f616b8450d58512bcd779ef660061774504dcff95fba8739070fa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-a, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-21T08:36:11.970 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 21 08:36:11 smithi086 ceph-mds[67182]: -- 172.21.15.86:0/3509720681 <== mon.0 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5600b3b63a40 con 0x5600b3b36c00 2024-04-21T08:36:11.970 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 21 08:36:11 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-a[67178]: starting mds.a at 2024-04-21T08:36:11.970 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 21 08:36:11 smithi086 bash[67163]: f9fbd33c53f616b8450d58512bcd779ef660061774504dcff95fba8739070fa1 2024-04-21T08:36:11.970 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 21 08:36:11 smithi086 systemd[1]: Started Ceph mds.a for f8cec7cc-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:36:12.012 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:36:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.114+0000 7ff603ef4640 1 -- 172.21.15.86:0/3086312900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5fc102a10 msgr2=0x7ff5fc109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.114+0000 7ff603ef4640 1 --2- 172.21.15.86:0/3086312900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5fc102a10 0x7ff5fc109330 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7ff5f4009ca0 tx=0x7ff5f402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:36:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.114+0000 7ff603ef4640 1 -- 172.21.15.86:0/3086312900 shutdown_connections 2024-04-21T08:36:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.114+0000 7ff603ef4640 1 --2- 172.21.15.86:0/3086312900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5fc102a10 0x7ff5fc109330 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.114+0000 7ff603ef4640 1 --2- 172.21.15.86:0/3086312900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5fc102050 0x7ff5fc1024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.114+0000 7ff603ef4640 1 --2- 172.21.15.86:0/3086312900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5fc100e70 0x7ff5fc101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.114+0000 7ff603ef4640 1 -- 172.21.15.86:0/3086312900 >> 172.21.15.86:0/3086312900 conn(0x7ff5fc0fc470 msgr2=0x7ff5fc0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.114+0000 7ff603ef4640 1 -- 172.21.15.86:0/3086312900 shutdown_connections 2024-04-21T08:36:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.115+0000 7ff603ef4640 1 -- 172.21.15.86:0/3086312900 wait complete. 2024-04-21T08:36:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.115+0000 7ff603ef4640 1 Processor -- start 2024-04-21T08:36:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.115+0000 7ff603ef4640 1 -- start start 2024-04-21T08:36:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.115+0000 7ff603ef4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5fc100e70 0x7ff5fc06b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.115+0000 7ff603ef4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5fc102050 0x7ff5fc19ce60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.115+0000 7ff603ef4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5fc102a10 0x7ff5fc19f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.115+0000 7ff603ef4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc10ca10 con 0x7ff5fc102050 2024-04-21T08:36:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.115+0000 7ff603ef4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc10c890 con 0x7ff5fc100e70 2024-04-21T08:36:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.115+0000 7ff603ef4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff5fc10cb90 con 0x7ff5fc102a10 2024-04-21T08:36:12.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.115+0000 7ff60148f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5fc102050 0x7ff5fc19ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:12.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.115+0000 7ff60148f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5fc102050 0x7ff5fc19ce60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50726/0 (socket says 172.21.15.86:50726) 2024-04-21T08:36:12.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.115+0000 7ff60148f640 1 -- 172.21.15.86:0/137007659 learned_addr learned my addr 172.21.15.86:0/137007659 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:12.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.115+0000 7ff601c90640 1 --2- 172.21.15.86:0/137007659 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5fc100e70 0x7ff5fc06b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:12.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.115+0000 7ff602491640 1 --2- 172.21.15.86:0/137007659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5fc102a10 0x7ff5fc19f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:12.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.115+0000 7ff60148f640 1 -- 172.21.15.86:0/137007659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5fc102a10 msgr2=0x7ff5fc19f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:12.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.115+0000 7ff60148f640 1 --2- 172.21.15.86:0/137007659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5fc102a10 0x7ff5fc19f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:12.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.115+0000 7ff60148f640 1 -- 172.21.15.86:0/137007659 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5fc100e70 msgr2=0x7ff5fc06b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:12.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.116+0000 7ff60148f640 1 --2- 172.21.15.86:0/137007659 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5fc100e70 0x7ff5fc06b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:12.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.116+0000 7ff60148f640 1 -- 172.21.15.86:0/137007659 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff5fc19fd90 con 0x7ff5fc102050 2024-04-21T08:36:12.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.116+0000 7ff60148f640 1 --2- 172.21.15.86:0/137007659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5fc102050 0x7ff5fc19ce60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec00d770 tx=0x7ff5ec00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.116+0000 7ff5f2ffd640 1 -- 172.21.15.86:0/137007659 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec017070 con 0x7ff5fc102050 2024-04-21T08:36:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.116+0000 7ff5f2ffd640 1 -- 172.21.15.86:0/137007659 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff5ec0054d0 con 0x7ff5fc102050 2024-04-21T08:36:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.116+0000 7ff603ef4640 1 -- 172.21.15.86:0/137007659 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff5fc1a0080 con 0x7ff5fc102050 2024-04-21T08:36:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.116+0000 7ff5f2ffd640 1 -- 172.21.15.86:0/137007659 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff5ec004c30 con 0x7ff5fc102050 2024-04-21T08:36:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.116+0000 7ff603ef4640 1 -- 172.21.15.86:0/137007659 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff5fc1a0580 con 0x7ff5fc102050 2024-04-21T08:36:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.117+0000 7ff603ef4640 1 -- 172.21.15.86:0/137007659 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff5fc1081b0 con 0x7ff5fc102050 2024-04-21T08:36:12.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.118+0000 7ff5f2ffd640 1 -- 172.21.15.86:0/137007659 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff5ec004dd0 con 0x7ff5fc102050 2024-04-21T08:36:12.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.119+0000 7ff5f2ffd640 1 --2- 172.21.15.86:0/137007659 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ff5e80a3bb0 0x7ff5e80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:12.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.119+0000 7ff601c90640 1 --2- 172.21.15.86:0/137007659 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ff5e80a3bb0 0x7ff5e80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:12.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.119+0000 7ff5f2ffd640 1 -- 172.21.15.86:0/137007659 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7ff5ec0cff00 con 0x7ff5fc102050 2024-04-21T08:36:12.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.119+0000 7ff601c90640 1 --2- 172.21.15.86:0/137007659 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ff5e80a3bb0 0x7ff5e80a6070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc101bc0 tx=0x7ff5f8005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:12.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.123+0000 7ff5f2ffd640 1 -- 172.21.15.86:0/137007659 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff5ec09cce0 con 0x7ff5fc102050 2024-04-21T08:36:12.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:11 smithi179 ceph-mon[31829]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:12.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:11 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:12.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:11 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:12.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:11 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:12.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:11 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:36:12.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:11 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:36:12.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:11 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:12.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.338+0000 7ff603ef4640 1 -- 172.21.15.86:0/137007659 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff5fc02d5c0 con 0x7ff5fc102050 2024-04-21T08:36:12.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.338+0000 7ff5f2ffd640 1 -- 172.21.15.86:0/137007659 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9656 (secure 0 0 0) 0x7ff5ec09c5e0 con 0x7ff5fc102050 2024-04-21T08:36:12.338 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:36:12.338 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-21T08:36:12.339 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 12 2024-04-21T08:36:12.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.340+0000 7ff603ef4640 1 -- 172.21.15.86:0/137007659 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ff5e80a3bb0 msgr2=0x7ff5e80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:12.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.340+0000 7ff603ef4640 1 --2- 172.21.15.86:0/137007659 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ff5e80a3bb0 0x7ff5e80a6070 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7ff5fc101bc0 tx=0x7ff5f8005f90 comp rx=0 tx=0).stop 2024-04-21T08:36:12.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.340+0000 7ff603ef4640 1 -- 172.21.15.86:0/137007659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5fc102050 msgr2=0x7ff5fc19ce60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:12.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.340+0000 7ff603ef4640 1 --2- 172.21.15.86:0/137007659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5fc102050 0x7ff5fc19ce60 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7ff5ec00d770 tx=0x7ff5ec00dc30 comp rx=0 tx=0).stop 2024-04-21T08:36:12.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.340+0000 7ff603ef4640 1 -- 172.21.15.86:0/137007659 shutdown_connections 2024-04-21T08:36:12.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.340+0000 7ff603ef4640 1 --2- 172.21.15.86:0/137007659 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ff5e80a3bb0 0x7ff5e80a6070 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:12.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.340+0000 7ff603ef4640 1 --2- 172.21.15.86:0/137007659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff5fc102a10 0x7ff5fc19f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:12.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.340+0000 7ff603ef4640 1 --2- 172.21.15.86:0/137007659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff5fc102050 0x7ff5fc19ce60 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:12.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.340+0000 7ff603ef4640 1 --2- 172.21.15.86:0/137007659 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff5fc100e70 0x7ff5fc06b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:12.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.340+0000 7ff603ef4640 1 -- 172.21.15.86:0/137007659 >> 172.21.15.86:0/137007659 conn(0x7ff5fc0fc470 msgr2=0x7ff5fc105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:12.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.341+0000 7ff603ef4640 1 -- 172.21.15.86:0/137007659 shutdown_connections 2024-04-21T08:36:12.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:12.341+0000 7ff603ef4640 1 -- 172.21.15.86:0/137007659 wait complete. 2024-04-21T08:36:12.350 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:11 smithi086 ceph-mon[27939]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:12.351 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:11 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:12.351 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:11 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:12.351 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:11 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:12.351 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:11 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:36:12.351 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:11 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:36:12.352 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:11 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:12.463 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:11 smithi088 ceph-mon[31259]: pgmap v275: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:12.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:11 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:12.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:11 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:12.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:11 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:12.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:11 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]: dispatch 2024-04-21T08:36:12.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:11 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "auth get-or-create", "entity": "mds.b", "caps": ["mon", "profile mds", "osd", "allow rw tag cephfs *=*", "mds", "allow"]}]': finished 2024-04-21T08:36:12.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:11 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:13.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:12 smithi088 ceph-mon[31259]: Deploying daemon mds.b on smithi088 2024-04-21T08:36:13.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:12 smithi088 ceph-mon[31259]: mds.? [v2:172.21.15.86:6840/3537972023,v1:172.21.15.86:6841/3537972023] up:boot 2024-04-21T08:36:13.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:12 smithi088 ceph-mon[31259]: fsmap 11 up:standby 2024-04-21T08:36:13.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:12 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-21T08:36:13.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:12 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/137007659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:13.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:12 smithi179 ceph-mon[31829]: Deploying daemon mds.b on smithi088 2024-04-21T08:36:13.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:12 smithi179 ceph-mon[31829]: mds.? [v2:172.21.15.86:6840/3537972023,v1:172.21.15.86:6841/3537972023] up:boot 2024-04-21T08:36:13.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:12 smithi179 ceph-mon[31829]: fsmap 11 up:standby 2024-04-21T08:36:13.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:12 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-21T08:36:13.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:12 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/137007659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:13.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:12 smithi086 ceph-mon[27939]: Deploying daemon mds.b on smithi088 2024-04-21T08:36:13.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:12 smithi086 ceph-mon[27939]: mds.? [v2:172.21.15.86:6840/3537972023,v1:172.21.15.86:6841/3537972023] up:boot 2024-04-21T08:36:13.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:12 smithi086 ceph-mon[27939]: fsmap 11 up:standby 2024-04-21T08:36:13.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:12 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-21T08:36:13.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:12 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/137007659' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:14.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:13 smithi088 ceph-mon[31259]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:14.214 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 21 08:36:14 smithi088 systemd[1]: Starting Ceph mds.b for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:36:14.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:13 smithi179 ceph-mon[31829]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:14.268 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:13 smithi086 ceph-mon[27939]: pgmap v276: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:14.354 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:36:14.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.465+0000 7f6e38a81640 1 -- 172.21.15.86:0/4234929165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e34101fd0 msgr2=0x7f6e34102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:14.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.465+0000 7f6e38a81640 1 --2- 172.21.15.86:0/4234929165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e34101fd0 0x7f6e34102450 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f6e24009900 tx=0x7f6e2402bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:14.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.465+0000 7f6e38a81640 1 -- 172.21.15.86:0/4234929165 shutdown_connections 2024-04-21T08:36:14.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.465+0000 7f6e38a81640 1 --2- 172.21.15.86:0/4234929165 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e34102990 0x7f6e341092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:14.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.465+0000 7f6e38a81640 1 --2- 172.21.15.86:0/4234929165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e34101fd0 0x7f6e34102450 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:14.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.465+0000 7f6e38a81640 1 --2- 172.21.15.86:0/4234929165 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e34100e80 0x7f6e341012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:14.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.465+0000 7f6e38a81640 1 -- 172.21.15.86:0/4234929165 >> 172.21.15.86:0/4234929165 conn(0x7f6e340fc410 msgr2=0x7f6e340fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:14.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.465+0000 7f6e38a81640 1 -- 172.21.15.86:0/4234929165 shutdown_connections 2024-04-21T08:36:14.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.465+0000 7f6e38a81640 1 -- 172.21.15.86:0/4234929165 wait complete. 2024-04-21T08:36:14.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.466+0000 7f6e38a81640 1 Processor -- start 2024-04-21T08:36:14.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.466+0000 7f6e38a81640 1 -- start start 2024-04-21T08:36:14.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.466+0000 7f6e38a81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e34100e80 0x7f6e3406b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:14.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.466+0000 7f6e38a81640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e34101fd0 0x7f6e3419cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:14.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.466+0000 7f6e38a81640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e34102990 0x7f6e3419f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:14.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.466+0000 7f6e38a81640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3410ca50 con 0x7f6e34100e80 2024-04-21T08:36:14.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.466+0000 7f6e38a81640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3410c8d0 con 0x7f6e34101fd0 2024-04-21T08:36:14.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.466+0000 7f6e38a81640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6e3410cbd0 con 0x7f6e34102990 2024-04-21T08:36:14.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.466+0000 7f6e3259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e34100e80 0x7f6e3406b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:14.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.466+0000 7f6e3259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e34100e80 0x7f6e3406b1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50738/0 (socket says 172.21.15.86:50738) 2024-04-21T08:36:14.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.466+0000 7f6e3259c640 1 -- 172.21.15.86:0/2465580648 learned_addr learned my addr 172.21.15.86:0/2465580648 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:14.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.466+0000 7f6e32d9d640 1 --2- 172.21.15.86:0/2465580648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e34102990 0x7f6e3419f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:14.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.466+0000 7f6e3259c640 1 -- 172.21.15.86:0/2465580648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e34102990 msgr2=0x7f6e3419f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:14.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.466+0000 7f6e3259c640 1 --2- 172.21.15.86:0/2465580648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e34102990 0x7f6e3419f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:14.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.466+0000 7f6e3259c640 1 -- 172.21.15.86:0/2465580648 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e34101fd0 msgr2=0x7f6e3419cf80 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T08:36:14.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.466+0000 7f6e3259c640 1 --2- 172.21.15.86:0/2465580648 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e34101fd0 0x7f6e3419cf80 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:14.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.466+0000 7f6e3259c640 1 -- 172.21.15.86:0/2465580648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6e3419feb0 con 0x7f6e34100e80 2024-04-21T08:36:14.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.466+0000 7f6e3259c640 1 --2- 172.21.15.86:0/2465580648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e34100e80 0x7f6e3406b1f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6e2800b4a0 tx=0x7f6e2800b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:14.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.467+0000 7f6e237fe640 1 -- 172.21.15.86:0/2465580648 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e28014070 con 0x7f6e34100e80 2024-04-21T08:36:14.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.467+0000 7f6e237fe640 1 -- 172.21.15.86:0/2465580648 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6e28010ac0 con 0x7f6e34100e80 2024-04-21T08:36:14.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.467+0000 7f6e38a81640 1 -- 172.21.15.86:0/2465580648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6e341a0140 con 0x7f6e34100e80 2024-04-21T08:36:14.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.467+0000 7f6e237fe640 1 -- 172.21.15.86:0/2465580648 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6e28017c30 con 0x7f6e34100e80 2024-04-21T08:36:14.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.467+0000 7f6e38a81640 1 -- 172.21.15.86:0/2465580648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6e341a0710 con 0x7f6e34100e80 2024-04-21T08:36:14.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.468+0000 7f6e38a81640 1 -- 172.21.15.86:0/2465580648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6e34108210 con 0x7f6e34100e80 2024-04-21T08:36:14.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.469+0000 7f6e237fe640 1 -- 172.21.15.86:0/2465580648 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f6e28017070 con 0x7f6e34100e80 2024-04-21T08:36:14.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.469+0000 7f6e237fe640 1 --2- 172.21.15.86:0/2465580648 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6e1c0a3bb0 0x7f6e1c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:14.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.470+0000 7f6e31d9b640 1 --2- 172.21.15.86:0/2465580648 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6e1c0a3bb0 0x7f6e1c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:14.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.470+0000 7f6e237fe640 1 -- 172.21.15.86:0/2465580648 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f6e280cf6f0 con 0x7f6e34100e80 2024-04-21T08:36:14.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.470+0000 7f6e31d9b640 1 --2- 172.21.15.86:0/2465580648 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6e1c0a3bb0 0x7f6e1c0a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6e24036040 tx=0x7f6e24005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:14.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.474+0000 7f6e237fe640 1 -- 172.21.15.86:0/2465580648 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6e2809c570 con 0x7f6e34100e80 2024-04-21T08:36:14.527 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 21 08:36:14 smithi088 podman[61635]: 2024-04-21 08:36:14.178992005 +0000 UTC m=+0.017634352 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:36:14.528 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 21 08:36:14 smithi088 podman[61635]: 2024-04-21 08:36:14.301682275 +0000 UTC m=+0.140324628 container create 6ae9477515466e140501ecf80fc9d1184594bec1ac44afbad2aee80dcc7b2f92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-b, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD) 2024-04-21T08:36:14.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.701+0000 7f6e38a81640 1 -- 172.21.15.86:0/2465580648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6e3402d810 con 0x7f6e34100e80 2024-04-21T08:36:14.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.701+0000 7f6e237fe640 1 -- 172.21.15.86:0/2465580648 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 12 v12) ==== 94+0+9656 (secure 0 0 0) 0x7f6e2809be70 con 0x7f6e34100e80 2024-04-21T08:36:14.701 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:36:14.701 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":12,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[]} 2024-04-21T08:36:14.701 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 12 2024-04-21T08:36:14.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.703+0000 7f6e38a81640 1 -- 172.21.15.86:0/2465580648 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6e1c0a3bb0 msgr2=0x7f6e1c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:14.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.703+0000 7f6e38a81640 1 --2- 172.21.15.86:0/2465580648 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6e1c0a3bb0 0x7f6e1c0a6070 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7f6e24036040 tx=0x7f6e24005fd0 comp rx=0 tx=0).stop 2024-04-21T08:36:14.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.703+0000 7f6e38a81640 1 -- 172.21.15.86:0/2465580648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e34100e80 msgr2=0x7f6e3406b1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:14.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.703+0000 7f6e38a81640 1 --2- 172.21.15.86:0/2465580648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e34100e80 0x7f6e3406b1f0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f6e2800b4a0 tx=0x7f6e2800b960 comp rx=0 tx=0).stop 2024-04-21T08:36:14.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.704+0000 7f6e38a81640 1 -- 172.21.15.86:0/2465580648 shutdown_connections 2024-04-21T08:36:14.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.704+0000 7f6e38a81640 1 --2- 172.21.15.86:0/2465580648 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6e1c0a3bb0 0x7f6e1c0a6070 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:14.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.704+0000 7f6e38a81640 1 --2- 172.21.15.86:0/2465580648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6e34102990 0x7f6e3419f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:14.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.704+0000 7f6e38a81640 1 --2- 172.21.15.86:0/2465580648 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6e34101fd0 0x7f6e3419cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:14.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.704+0000 7f6e38a81640 1 --2- 172.21.15.86:0/2465580648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6e34100e80 0x7f6e3406b1f0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:14.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.704+0000 7f6e38a81640 1 -- 172.21.15.86:0/2465580648 >> 172.21.15.86:0/2465580648 conn(0x7f6e340fc410 msgr2=0x7f6e341053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:14.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.704+0000 7f6e38a81640 1 -- 172.21.15.86:0/2465580648 shutdown_connections 2024-04-21T08:36:14.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:14.704+0000 7f6e38a81640 1 -- 172.21.15.86:0/2465580648 wait complete. 2024-04-21T08:36:14.964 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 21 08:36:14 smithi088 podman[61635]: 2024-04-21 08:36:14.52936877 +0000 UTC m=+0.368011119 container init 6ae9477515466e140501ecf80fc9d1184594bec1ac44afbad2aee80dcc7b2f92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-b, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-21T08:36:14.964 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 21 08:36:14 smithi088 podman[61635]: 2024-04-21 08:36:14.532041086 +0000 UTC m=+0.370683439 container start 6ae9477515466e140501ecf80fc9d1184594bec1ac44afbad2aee80dcc7b2f92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-b, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:36:14.964 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 21 08:36:14 smithi088 ceph-mds[61655]: -- 172.21.15.88:0/1917590334 <== mon.2 v2:172.21.15.88:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5654a4deba40 con 0x5654a4dbec00 2024-04-21T08:36:14.964 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 21 08:36:14 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-b[61651]: starting mds.b at 2024-04-21T08:36:14.964 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 21 08:36:14 smithi088 bash[61635]: 6ae9477515466e140501ecf80fc9d1184594bec1ac44afbad2aee80dcc7b2f92 2024-04-21T08:36:14.964 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 21 08:36:14 smithi088 systemd[1]: Started Ceph mds.b for f8cec7cc-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:36:15.866 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:15 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:15.866 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:15 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:15.866 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:15 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:15.867 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:15 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:15.867 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:15 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:15.867 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:15 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2465580648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:15.867 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:15 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:36:15.867 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:15 smithi088 ceph-mon[31259]: mds.? [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] up:boot 2024-04-21T08:36:15.867 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:15 smithi088 ceph-mon[31259]: fsmap 12 up:standby 2024-04-21T08:36:15.867 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:15 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T08:36:15.867 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:15 smithi088 ceph-mon[31259]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:15.868 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:15 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:15.868 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:15 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:15.868 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:15 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:15.868 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:15 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:15.868 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:15 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:15.868 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:15 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2465580648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:15.868 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:15 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:36:15.868 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:15 smithi086 ceph-mon[27939]: mds.? [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] up:boot 2024-04-21T08:36:15.868 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:15 smithi086 ceph-mon[27939]: fsmap 12 up:standby 2024-04-21T08:36:15.868 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:15 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T08:36:15.868 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:15 smithi086 ceph-mon[27939]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:15.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:15 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:15.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:15 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:15.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:15 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:15.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:15 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:15.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:15 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:15.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:15 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2465580648' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:15.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:15 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:36:15.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:15 smithi179 ceph-mon[31829]: mds.? [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] up:boot 2024-04-21T08:36:15.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:15 smithi179 ceph-mon[31829]: fsmap 12 up:standby 2024-04-21T08:36:15.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:15 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T08:36:15.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:15 smithi179 ceph-mon[31829]: pgmap v277: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:16.715 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:36:16.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.816+0000 7f59ebbe6640 1 -- 172.21.15.86:0/2599267986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59e4101fd0 msgr2=0x7f59e4102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:16.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.816+0000 7f59ebbe6640 1 --2- 172.21.15.86:0/2599267986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59e4101fd0 0x7f59e4102450 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f59d4009900 tx=0x7f59d402bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:16.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.816+0000 7f59ebbe6640 1 -- 172.21.15.86:0/2599267986 shutdown_connections 2024-04-21T08:36:16.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.816+0000 7f59ebbe6640 1 --2- 172.21.15.86:0/2599267986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59e4102990 0x7f59e41092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:16.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.816+0000 7f59ebbe6640 1 --2- 172.21.15.86:0/2599267986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59e4101fd0 0x7f59e4102450 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:16.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.816+0000 7f59ebbe6640 1 --2- 172.21.15.86:0/2599267986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59e4100e80 0x7f59e41012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:16.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.816+0000 7f59ebbe6640 1 -- 172.21.15.86:0/2599267986 >> 172.21.15.86:0/2599267986 conn(0x7f59e40fc410 msgr2=0x7f59e40fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:16.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.816+0000 7f59ebbe6640 1 -- 172.21.15.86:0/2599267986 shutdown_connections 2024-04-21T08:36:16.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.816+0000 7f59ebbe6640 1 -- 172.21.15.86:0/2599267986 wait complete. 2024-04-21T08:36:16.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.817+0000 7f59ebbe6640 1 Processor -- start 2024-04-21T08:36:16.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.817+0000 7f59ebbe6640 1 -- start start 2024-04-21T08:36:16.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.817+0000 7f59ebbe6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59e4100e80 0x7f59e41981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:16.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.817+0000 7f59ebbe6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59e4101fd0 0x7f59e41986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:16.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.817+0000 7f59ebbe6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59e4102990 0x7f59e419f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:16.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.817+0000 7f59ebbe6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f59e410c9e0 con 0x7f59e4100e80 2024-04-21T08:36:16.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.817+0000 7f59ebbe6640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f59e410c860 con 0x7f59e4101fd0 2024-04-21T08:36:16.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.817+0000 7f59ebbe6640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f59e410cb60 con 0x7f59e4102990 2024-04-21T08:36:16.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.817+0000 7f59e9982640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59e4100e80 0x7f59e41981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:16.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.817+0000 7f59e9982640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59e4100e80 0x7f59e41981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50758/0 (socket says 172.21.15.86:50758) 2024-04-21T08:36:16.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.817+0000 7f59e9982640 1 -- 172.21.15.86:0/3267513351 learned_addr learned my addr 172.21.15.86:0/3267513351 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:16.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.817+0000 7f59ea183640 1 --2- 172.21.15.86:0/3267513351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59e4102990 0x7f59e419f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:16.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.817+0000 7f59e9181640 1 --2- 172.21.15.86:0/3267513351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59e4101fd0 0x7f59e41986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:16.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.818+0000 7f59e9982640 1 -- 172.21.15.86:0/3267513351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59e4102990 msgr2=0x7f59e419f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:16.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.818+0000 7f59e9982640 1 --2- 172.21.15.86:0/3267513351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59e4102990 0x7f59e419f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:16.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.818+0000 7f59e9982640 1 -- 172.21.15.86:0/3267513351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59e4101fd0 msgr2=0x7f59e41986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:16.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.818+0000 7f59e9982640 1 --2- 172.21.15.86:0/3267513351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59e4101fd0 0x7f59e41986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:16.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.818+0000 7f59e9982640 1 -- 172.21.15.86:0/3267513351 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59e419fdd0 con 0x7f59e4100e80 2024-04-21T08:36:16.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.818+0000 7f59ea183640 1 --2- 172.21.15.86:0/3267513351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59e4102990 0x7f59e419f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:36:16.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.818+0000 7f59e9982640 1 --2- 172.21.15.86:0/3267513351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59e4100e80 0x7f59e41981b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f59e000d770 tx=0x7f59e000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:16.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.818+0000 7f59daffd640 1 -- 172.21.15.86:0/3267513351 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e0017070 con 0x7f59e4100e80 2024-04-21T08:36:16.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.818+0000 7f59daffd640 1 -- 172.21.15.86:0/3267513351 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59e00054d0 con 0x7f59e4100e80 2024-04-21T08:36:16.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.818+0000 7f59ebbe6640 1 -- 172.21.15.86:0/3267513351 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59e41a00c0 con 0x7f59e4100e80 2024-04-21T08:36:16.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.818+0000 7f59daffd640 1 -- 172.21.15.86:0/3267513351 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59e0004c30 con 0x7f59e4100e80 2024-04-21T08:36:16.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.818+0000 7f59ebbe6640 1 -- 172.21.15.86:0/3267513351 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59e41a0590 con 0x7f59e4100e80 2024-04-21T08:36:16.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.819+0000 7f59ebbe6640 1 -- 172.21.15.86:0/3267513351 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59e41a07f0 con 0x7f59e4100e80 2024-04-21T08:36:16.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.820+0000 7f59daffd640 1 -- 172.21.15.86:0/3267513351 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f59e0004e10 con 0x7f59e4100e80 2024-04-21T08:36:16.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.820+0000 7f59daffd640 1 --2- 172.21.15.86:0/3267513351 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f59d00a3bb0 0x7f59d00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:16.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.820+0000 7f59e9181640 1 --2- 172.21.15.86:0/3267513351 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f59d00a3bb0 0x7f59d00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:16.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.821+0000 7f59daffd640 1 -- 172.21.15.86:0/3267513351 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f59e00cfa20 con 0x7f59e4100e80 2024-04-21T08:36:16.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.821+0000 7f59e9181640 1 --2- 172.21.15.86:0/3267513351 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f59d00a3bb0 0x7f59d00a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f59d4005ab0 tx=0x7f59d4005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:16.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:16.825+0000 7f59daffd640 1 -- 172.21.15.86:0/3267513351 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f59e009c8a0 con 0x7f59e4100e80 2024-04-21T08:36:17.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.039+0000 7f59ebbe6640 1 -- 172.21.15.86:0/3267513351 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f59e41a09c0 con 0x7f59e4100e80 2024-04-21T08:36:17.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.039+0000 7f59daffd640 1 -- 172.21.15.86:0/3267513351 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 13 v13) ==== 94+0+10482 (secure 0 0 0) 0x7f59e009c1a0 con 0x7f59e4100e80 2024-04-21T08:36:17.039 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:36:17.039 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":13,"default_fscid":-1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10},{"gid":24482,"name":"b","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":13}],"filesystems":[]} 2024-04-21T08:36:17.039 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 13 2024-04-21T08:36:17.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.041+0000 7f59ebbe6640 1 -- 172.21.15.86:0/3267513351 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f59d00a3bb0 msgr2=0x7f59d00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:17.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.041+0000 7f59ebbe6640 1 --2- 172.21.15.86:0/3267513351 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f59d00a3bb0 0x7f59d00a6070 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f59d4005ab0 tx=0x7f59d4005fd0 comp rx=0 tx=0).stop 2024-04-21T08:36:17.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.041+0000 7f59ebbe6640 1 -- 172.21.15.86:0/3267513351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59e4100e80 msgr2=0x7f59e41981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:17.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.041+0000 7f59ebbe6640 1 --2- 172.21.15.86:0/3267513351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59e4100e80 0x7f59e41981b0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f59e000d770 tx=0x7f59e000dc30 comp rx=0 tx=0).stop 2024-04-21T08:36:17.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.041+0000 7f59ebbe6640 1 -- 172.21.15.86:0/3267513351 shutdown_connections 2024-04-21T08:36:17.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.041+0000 7f59ebbe6640 1 --2- 172.21.15.86:0/3267513351 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f59d00a3bb0 0x7f59d00a6070 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:17.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.041+0000 7f59ebbe6640 1 --2- 172.21.15.86:0/3267513351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59e4102990 0x7f59e419f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:17.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.041+0000 7f59ebbe6640 1 --2- 172.21.15.86:0/3267513351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59e4101fd0 0x7f59e41986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:17.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.041+0000 7f59ebbe6640 1 --2- 172.21.15.86:0/3267513351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59e4100e80 0x7f59e41981b0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:17.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.041+0000 7f59ebbe6640 1 -- 172.21.15.86:0/3267513351 >> 172.21.15.86:0/3267513351 conn(0x7f59e40fc410 msgr2=0x7f59e40fdef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:17.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.041+0000 7f59ebbe6640 1 -- 172.21.15.86:0/3267513351 shutdown_connections 2024-04-21T08:36:17.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.041+0000 7f59ebbe6640 1 -- 172.21.15.86:0/3267513351 wait complete. 2024-04-21T08:36:17.056 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:36:17.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.157+0000 7fe6bb37e640 1 -- 172.21.15.86:0/2205169020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6b4101fc0 msgr2=0x7fe6b4102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.157+0000 7fe6bb37e640 1 --2- 172.21.15.86:0/2205169020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6b4101fc0 0x7fe6b4102440 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fe6a4009900 tx=0x7fe6a402bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.157+0000 7fe6bb37e640 1 -- 172.21.15.86:0/2205169020 shutdown_connections 2024-04-21T08:36:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.157+0000 7fe6bb37e640 1 --2- 172.21.15.86:0/2205169020 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6b4102980 0x7fe6b41092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.157+0000 7fe6bb37e640 1 --2- 172.21.15.86:0/2205169020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6b4101fc0 0x7fe6b4102440 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.157+0000 7fe6bb37e640 1 --2- 172.21.15.86:0/2205169020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6b4100e70 0x7fe6b4101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.157+0000 7fe6bb37e640 1 -- 172.21.15.86:0/2205169020 >> 172.21.15.86:0/2205169020 conn(0x7fe6b40fc3e0 msgr2=0x7fe6b40fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.157+0000 7fe6bb37e640 1 -- 172.21.15.86:0/2205169020 shutdown_connections 2024-04-21T08:36:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.157+0000 7fe6bb37e640 1 -- 172.21.15.86:0/2205169020 wait complete. 2024-04-21T08:36:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.157+0000 7fe6bb37e640 1 Processor -- start 2024-04-21T08:36:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.157+0000 7fe6bb37e640 1 -- start start 2024-04-21T08:36:17.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.158+0000 7fe6bb37e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6b4100e70 0x7fe6b4198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.158+0000 7fe6bb37e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6b4101fc0 0x7fe6b41986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.158+0000 7fe6bb37e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6b4102980 0x7fe6b419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.158+0000 7fe6bb37e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b410c9a0 con 0x7fe6b4102980 2024-04-21T08:36:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.158+0000 7fe6bb37e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b410c820 con 0x7fe6b4100e70 2024-04-21T08:36:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.158+0000 7fe6bb37e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe6b410cb20 con 0x7fe6b4101fc0 2024-04-21T08:36:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.158+0000 7fe6b991b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6b4102980 0x7fe6b419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.158+0000 7fe6b991b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6b4102980 0x7fe6b419f730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50772/0 (socket says 172.21.15.86:50772) 2024-04-21T08:36:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.158+0000 7fe6b991b640 1 -- 172.21.15.86:0/682843403 learned_addr learned my addr 172.21.15.86:0/682843403 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.158+0000 7fe6b911a640 1 --2- 172.21.15.86:0/682843403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6b4100e70 0x7fe6b4198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.158+0000 7fe6b8919640 1 --2- 172.21.15.86:0/682843403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6b4101fc0 0x7fe6b41986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.158+0000 7fe6b991b640 1 -- 172.21.15.86:0/682843403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6b4101fc0 msgr2=0x7fe6b41986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.158+0000 7fe6b991b640 1 --2- 172.21.15.86:0/682843403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6b4101fc0 0x7fe6b41986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:17.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.158+0000 7fe6b991b640 1 -- 172.21.15.86:0/682843403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6b4100e70 msgr2=0x7fe6b4198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.158+0000 7fe6b991b640 1 --2- 172.21.15.86:0/682843403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6b4100e70 0x7fe6b4198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.158+0000 7fe6b991b640 1 -- 172.21.15.86:0/682843403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe6b419fda0 con 0x7fe6b4102980 2024-04-21T08:36:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.158+0000 7fe6b8919640 1 --2- 172.21.15.86:0/682843403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6b4101fc0 0x7fe6b41986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:36:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.158+0000 7fe6b991b640 1 --2- 172.21.15.86:0/682843403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6b4102980 0x7fe6b419f730 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac00eb10 tx=0x7fe6ac00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.158+0000 7fe6aa7fc640 1 -- 172.21.15.86:0/682843403 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac004b20 con 0x7fe6b4102980 2024-04-21T08:36:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.158+0000 7fe6aa7fc640 1 -- 172.21.15.86:0/682843403 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6ac004cc0 con 0x7fe6b4102980 2024-04-21T08:36:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.158+0000 7fe6bb37e640 1 -- 172.21.15.86:0/682843403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe6b41a0090 con 0x7fe6b4102980 2024-04-21T08:36:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.158+0000 7fe6aa7fc640 1 -- 172.21.15.86:0/682843403 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6ac010670 con 0x7fe6b4102980 2024-04-21T08:36:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.158+0000 7fe6bb37e640 1 -- 172.21.15.86:0/682843403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6b41a05b0 con 0x7fe6b4102980 2024-04-21T08:36:17.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.159+0000 7fe6bb37e640 1 -- 172.21.15.86:0/682843403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe6b41a0870 con 0x7fe6b4102980 2024-04-21T08:36:17.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.160+0000 7fe6aa7fc640 1 -- 172.21.15.86:0/682843403 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe6ac005020 con 0x7fe6b4102980 2024-04-21T08:36:17.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.161+0000 7fe6aa7fc640 1 --2- 172.21.15.86:0/682843403 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe6a00a3c30 0x7fe6a00a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:17.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.161+0000 7fe6b911a640 1 --2- 172.21.15.86:0/682843403 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe6a00a3c30 0x7fe6a00a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:17.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.161+0000 7fe6aa7fc640 1 -- 172.21.15.86:0/682843403 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fe6ac0d0540 con 0x7fe6b4102980 2024-04-21T08:36:17.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.162+0000 7fe6b911a640 1 --2- 172.21.15.86:0/682843403 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe6a00a3c30 0x7fe6a00a60f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0009bd0 tx=0x7fe6b0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:17.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.166+0000 7fe6aa7fc640 1 -- 172.21.15.86:0/682843403 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe6ac09d3c0 con 0x7fe6b4102980 2024-04-21T08:36:17.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.350+0000 7fe6bb37e640 1 -- 172.21.15.86:0/682843403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fe6b41a0b60 con 0x7fe6b4102980 2024-04-21T08:36:17.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.351+0000 7fe6aa7fc640 1 -- 172.21.15.86:0/682843403 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v81) ==== 74+0+18398 (secure 0 0 0) 0x7fe6ac09ccc0 con 0x7fe6b4102980 2024-04-21T08:36:17.351 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:36:17.351 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":81,"fsid":"f8cec7cc-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:22.687329+0000","modified":"2024-04-21T08:35:31.056297+0000","last_up_change":"2024-04-21T08:35:30.048315+0000","last_in_change":"2024-04-21T08:35:10.099496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":26,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:45.279109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"89b7601f-0620-4db2-90f1-b45cea173738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6803","nonce":2571516897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6805","nonce":2571516897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6809","nonce":2571516897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6807","nonce":2571516897}]},"public_addr":"172.21.15.86:6803/2571516897","cluster_addr":"172.21.15.86:6805/2571516897","heartbeat_back_addr":"172.21.15.86:6809/2571516897","heartbeat_front_addr":"172.21.15.86:6807/2571516897","state":["exists","up"]},{"osd":1,"uuid":"4891031c-f98c-42bc-bb3b-a420855ecc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":41,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6801","nonce":74067182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6803","nonce":74067182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6807","nonce":74067182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6805","nonce":74067182}]},"public_addr":"172.21.15.88:6801/74067182","cluster_addr":"172.21.15.88:6803/74067182","heartbeat_back_addr":"172.21.15.88:6807/74067182","heartbeat_front_addr":"172.21.15.88:6805/74067182","state":["exists","up"]},{"osd":2,"uuid":"60d12aa4-9312-4fa2-8d0b-2b3f42caad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6801","nonce":4137659051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6803","nonce":4137659051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6807","nonce":4137659051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6805","nonce":4137659051}]},"public_addr":"172.21.15.179:6801/4137659051","cluster_addr":"172.21.15.179:6803/4137659051","heartbeat_back_addr":"172.21.15.179:6807/4137659051","heartbeat_front_addr":"172.21.15.179:6805/4137659051","state":["exists","up"]},{"osd":3,"uuid":"404c3158-8cb0-4c02-b48a-f9ee5ff7b784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6811","nonce":2309603148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6813","nonce":2309603148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6817","nonce":2309603148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6815","nonce":2309603148}]},"public_addr":"172.21.15.86:6811/2309603148","cluster_addr":"172.21.15.86:6813/2309603148","heartbeat_back_addr":"172.21.15.86:6817/2309603148","heartbeat_front_addr":"172.21.15.86:6815/2309603148","state":["exists","up"]},{"osd":4,"uuid":"458ff202-d781-4ec7-ac93-0b49f835b503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6809","nonce":1818714217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6811","nonce":1818714217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6815","nonce":1818714217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6813","nonce":1818714217}]},"public_addr":"172.21.15.88:6809/1818714217","cluster_addr":"172.21.15.88:6811/1818714217","heartbeat_back_addr":"172.21.15.88:6815/1818714217","heartbeat_front_addr":"172.21.15.88:6813/1818714217","state":["exists","up"]},{"osd":5,"uuid":"8123cc90-bd8b-4204-bf31-46b34e9f589d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":53,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6809","nonce":863840990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6811","nonce":863840990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6815","nonce":863840990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6813","nonce":863840990}]},"public_addr":"172.21.15.179:6809/863840990","cluster_addr":"172.21.15.179:6811/863840990","heartbeat_back_addr":"172.21.15.179:6815/863840990","heartbeat_front_addr":"172.21.15.179:6813/863840990","state":["exists","up"]},{"osd":6,"uuid":"aac9d7c7-659a-4110-903a-a871c086066f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6819","nonce":1224231118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6821","nonce":1224231118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6825","nonce":1224231118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6823","nonce":1224231118}]},"public_addr":"172.21.15.86:6819/1224231118","cluster_addr":"172.21.15.86:6821/1224231118","heartbeat_back_addr":"172.21.15.86:6825/1224231118","heartbeat_front_addr":"172.21.15.86:6823/1224231118","state":["exists","up"]},{"osd":7,"uuid":"df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":75,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6817","nonce":136629878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6819","nonce":136629878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6823","nonce":136629878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6821","nonce":136629878}]},"public_addr":"172.21.15.88:6817/136629878","cluster_addr":"172.21.15.88:6819/136629878","heartbeat_back_addr":"172.21.15.88:6823/136629878","heartbeat_front_addr":"172.21.15.88:6821/136629878","state":["exists","up"]},{"osd":8,"uuid":"63e59f4b-0426-46db-92f7-c55dae61b5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6817","nonce":1770418957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6819","nonce":1770418957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6823","nonce":1770418957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6821","nonce":1770418957}]},"public_addr":"172.21.15.179:6817/1770418957","cluster_addr":"172.21.15.179:6819/1770418957","heartbeat_back_addr":"172.21.15.179:6823/1770418957","heartbeat_front_addr":"172.21.15.179:6821/1770418957","state":["exists","up"]},{"osd":9,"uuid":"5841bb36-ed6b-45d8-b857-eeab139c44e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6827","nonce":1108500274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6829","nonce":1108500274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6833","nonce":1108500274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6831","nonce":1108500274}]},"public_addr":"172.21.15.86:6827/1108500274","cluster_addr":"172.21.15.86:6829/1108500274","heartbeat_back_addr":"172.21.15.86:6833/1108500274","heartbeat_front_addr":"172.21.15.86:6831/1108500274","state":["exists","up"]},{"osd":10,"uuid":"f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6825","nonce":1294275353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6827","nonce":1294275353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6831","nonce":1294275353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6829","nonce":1294275353}]},"public_addr":"172.21.15.88:6825/1294275353","cluster_addr":"172.21.15.88:6827/1294275353","heartbeat_back_addr":"172.21.15.88:6831/1294275353","heartbeat_front_addr":"172.21.15.88:6829/1294275353","state":["exists","up"]},{"osd":11,"uuid":"e0d4cc76-e7a6-4a6b-909c-95488ccd5a36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6825","nonce":4007070233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6827","nonce":4007070233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6831","nonce":4007070233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6829","nonce":4007070233}]},"public_addr":"172.21.15.179:6825/4007070233","cluster_addr":"172.21.15.179:6827/4007070233","heartbeat_back_addr":"172.21.15.179:6831/4007070233","heartbeat_front_addr":"172.21.15.179:6829/4007070233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:55.210605+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:18.723090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:42.547979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:06.421622+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:30.858315+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:55.455313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.623369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:46.185111+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.103627+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:37.297545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:02.637431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:28.199292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6800/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/1412393225":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6800/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/1687852404":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2020707404":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2568130933":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/3952607648":"2024-04-22T08:28:57.384403+0000","172.21.15.86:6801/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/2756903426":"2024-04-22T08:29:23.190001+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:36:17.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.353+0000 7fe6bb37e640 1 -- 172.21.15.86:0/682843403 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe6a00a3c30 msgr2=0x7fe6a00a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:17.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.353+0000 7fe6bb37e640 1 --2- 172.21.15.86:0/682843403 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe6a00a3c30 0x7fe6a00a60f0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fe6b0009bd0 tx=0x7fe6b0005f90 comp rx=0 tx=0).stop 2024-04-21T08:36:17.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.353+0000 7fe6bb37e640 1 -- 172.21.15.86:0/682843403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6b4102980 msgr2=0x7fe6b419f730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:17.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.353+0000 7fe6bb37e640 1 --2- 172.21.15.86:0/682843403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6b4102980 0x7fe6b419f730 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7fe6ac00eb10 tx=0x7fe6ac00c2d0 comp rx=0 tx=0).stop 2024-04-21T08:36:17.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.353+0000 7fe6bb37e640 1 -- 172.21.15.86:0/682843403 shutdown_connections 2024-04-21T08:36:17.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.353+0000 7fe6bb37e640 1 --2- 172.21.15.86:0/682843403 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe6a00a3c30 0x7fe6a00a60f0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:17.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.353+0000 7fe6bb37e640 1 --2- 172.21.15.86:0/682843403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe6b4102980 0x7fe6b419f730 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:17.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.353+0000 7fe6bb37e640 1 --2- 172.21.15.86:0/682843403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe6b4101fc0 0x7fe6b41986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:17.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.353+0000 7fe6bb37e640 1 --2- 172.21.15.86:0/682843403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe6b4100e70 0x7fe6b4198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:17.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.353+0000 7fe6bb37e640 1 -- 172.21.15.86:0/682843403 >> 172.21.15.86:0/682843403 conn(0x7fe6b40fc3e0 msgr2=0x7fe6b40690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:17.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.354+0000 7fe6bb37e640 1 -- 172.21.15.86:0/682843403 shutdown_connections 2024-04-21T08:36:17.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.354+0000 7fe6bb37e640 1 -- 172.21.15.86:0/682843403 wait complete. 2024-04-21T08:36:17.364 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:31:45.279109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}] 2024-04-21T08:36:17.365 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:36:17.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.468+0000 7f993ed3d640 1 -- 172.21.15.86:0/3324830170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9938101ff0 msgr2=0x7f9938102470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:17.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.468+0000 7f993ed3d640 1 --2- 172.21.15.86:0/3324830170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9938101ff0 0x7f9938102470 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f9924009900 tx=0x7f992402bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:17.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.468+0000 7f993ed3d640 1 -- 172.21.15.86:0/3324830170 shutdown_connections 2024-04-21T08:36:17.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.468+0000 7f993ed3d640 1 --2- 172.21.15.86:0/3324830170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99381029b0 0x7f99381092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:17.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.468+0000 7f993ed3d640 1 --2- 172.21.15.86:0/3324830170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9938101ff0 0x7f9938102470 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:17.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.468+0000 7f993ed3d640 1 --2- 172.21.15.86:0/3324830170 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9938100ea0 0x7f99381012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:17.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.468+0000 7f993ed3d640 1 -- 172.21.15.86:0/3324830170 >> 172.21.15.86:0/3324830170 conn(0x7f99380fc410 msgr2=0x7f99380fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:17.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.468+0000 7f993ed3d640 1 -- 172.21.15.86:0/3324830170 shutdown_connections 2024-04-21T08:36:17.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.468+0000 7f993ed3d640 1 -- 172.21.15.86:0/3324830170 wait complete. 2024-04-21T08:36:17.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.469+0000 7f993ed3d640 1 Processor -- start 2024-04-21T08:36:17.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.469+0000 7f993ed3d640 1 -- start start 2024-04-21T08:36:17.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.469+0000 7f993ed3d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9938100ea0 0x7f993806b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:17.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.469+0000 7f993ed3d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9938101ff0 0x7f993819cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:17.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.469+0000 7f993ed3d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99381029b0 0x7f993819f580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:17.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.469+0000 7f993ed3d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f993810c9f0 con 0x7f9938100ea0 2024-04-21T08:36:17.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.469+0000 7f993ed3d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f993810c870 con 0x7f9938101ff0 2024-04-21T08:36:17.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.469+0000 7f993ed3d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f993810cb70 con 0x7f99381029b0 2024-04-21T08:36:17.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.469+0000 7f993cad9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9938100ea0 0x7f993806b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:17.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.469+0000 7f993cad9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9938100ea0 0x7f993806b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50784/0 (socket says 172.21.15.86:50784) 2024-04-21T08:36:17.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.469+0000 7f993cad9640 1 -- 172.21.15.86:0/672130526 learned_addr learned my addr 172.21.15.86:0/672130526 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:17.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.469+0000 7f9937fff640 1 --2- 172.21.15.86:0/672130526 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9938101ff0 0x7f993819cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:17.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.469+0000 7f993d2da640 1 --2- 172.21.15.86:0/672130526 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99381029b0 0x7f993819f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:17.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.469+0000 7f993cad9640 1 -- 172.21.15.86:0/672130526 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99381029b0 msgr2=0x7f993819f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:17.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.469+0000 7f993cad9640 1 --2- 172.21.15.86:0/672130526 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99381029b0 0x7f993819f580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:17.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.469+0000 7f993cad9640 1 -- 172.21.15.86:0/672130526 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9938101ff0 msgr2=0x7f993819cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:17.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.469+0000 7f993cad9640 1 --2- 172.21.15.86:0/672130526 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9938101ff0 0x7f993819cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:17.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.469+0000 7f993cad9640 1 -- 172.21.15.86:0/672130526 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f993819fe40 con 0x7f9938100ea0 2024-04-21T08:36:17.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.469+0000 7f993cad9640 1 --2- 172.21.15.86:0/672130526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9938100ea0 0x7f993806b0c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f992c00d950 tx=0x7f992c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:17.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.469+0000 7f9935ffb640 1 -- 172.21.15.86:0/672130526 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c017030 con 0x7f9938100ea0 2024-04-21T08:36:17.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.469+0000 7f993ed3d640 1 -- 172.21.15.86:0/672130526 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f99381a0130 con 0x7f9938100ea0 2024-04-21T08:36:17.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.470+0000 7f9935ffb640 1 -- 172.21.15.86:0/672130526 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f992c004020 con 0x7f9938100ea0 2024-04-21T08:36:17.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.470+0000 7f9935ffb640 1 -- 172.21.15.86:0/672130526 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f992c0047a0 con 0x7f9938100ea0 2024-04-21T08:36:17.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.470+0000 7f993ed3d640 1 -- 172.21.15.86:0/672130526 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f99381a0650 con 0x7f9938100ea0 2024-04-21T08:36:17.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.471+0000 7f993ed3d640 1 -- 172.21.15.86:0/672130526 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9938108280 con 0x7f9938100ea0 2024-04-21T08:36:17.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.472+0000 7f9935ffb640 1 -- 172.21.15.86:0/672130526 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f992c009a30 con 0x7f9938100ea0 2024-04-21T08:36:17.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.472+0000 7f9935ffb640 1 --2- 172.21.15.86:0/672130526 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f99280a3bb0 0x7f99280a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:17.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.472+0000 7f9937fff640 1 --2- 172.21.15.86:0/672130526 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f99280a3bb0 0x7f99280a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:17.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.472+0000 7f9935ffb640 1 -- 172.21.15.86:0/672130526 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f992c0cf970 con 0x7f9938100ea0 2024-04-21T08:36:17.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.473+0000 7f9937fff640 1 --2- 172.21.15.86:0/672130526 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f99280a3bb0 0x7f99280a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9924005ab0 tx=0x7f9924005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:17.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.477+0000 7f9935ffb640 1 -- 172.21.15.86:0/672130526 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f992c09c7f0 con 0x7f9938100ea0 2024-04-21T08:36:17.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:17 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:17.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:17 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:17.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:17 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:17.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:17 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:17.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:17 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:17.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:17 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:17.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:17 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3267513351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:17.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:17 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:17.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:17 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:36:17.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:17 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:17.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:17 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:36:17.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:17 smithi086 ceph-mon[27939]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:17.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:17 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/682843403' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:17.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.663+0000 7f993ed3d640 1 -- 172.21.15.86:0/672130526 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9938062b50 con 0x7f9938100ea0 2024-04-21T08:36:17.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.663+0000 7f9935ffb640 1 -- 172.21.15.86:0/672130526 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v81) ==== 93+0+10 (secure 0 0 0) 0x7f992c09c0f0 con 0x7f9938100ea0 2024-04-21T08:36:17.666 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-21T08:36:17.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.665+0000 7f993ed3d640 1 -- 172.21.15.86:0/672130526 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f99280a3bb0 msgr2=0x7f99280a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:17.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.665+0000 7f993ed3d640 1 --2- 172.21.15.86:0/672130526 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f99280a3bb0 0x7f99280a6070 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7f9924005ab0 tx=0x7f9924005fd0 comp rx=0 tx=0).stop 2024-04-21T08:36:17.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.665+0000 7f993ed3d640 1 -- 172.21.15.86:0/672130526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9938100ea0 msgr2=0x7f993806b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:17.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.665+0000 7f993ed3d640 1 --2- 172.21.15.86:0/672130526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9938100ea0 0x7f993806b0c0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f992c00d950 tx=0x7f992c00de10 comp rx=0 tx=0).stop 2024-04-21T08:36:17.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.665+0000 7f993ed3d640 1 -- 172.21.15.86:0/672130526 shutdown_connections 2024-04-21T08:36:17.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.665+0000 7f993ed3d640 1 --2- 172.21.15.86:0/672130526 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f99280a3bb0 0x7f99280a6070 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:17.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.665+0000 7f993ed3d640 1 --2- 172.21.15.86:0/672130526 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f99381029b0 0x7f993819f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:17.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.665+0000 7f993ed3d640 1 --2- 172.21.15.86:0/672130526 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9938101ff0 0x7f993819cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:17.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.665+0000 7f993ed3d640 1 --2- 172.21.15.86:0/672130526 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9938100ea0 0x7f993806b0c0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:17.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.665+0000 7f993ed3d640 1 -- 172.21.15.86:0/672130526 >> 172.21.15.86:0/672130526 conn(0x7f99380fc410 msgr2=0x7f9938105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:17.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.665+0000 7f993ed3d640 1 -- 172.21.15.86:0/672130526 shutdown_connections 2024-04-21T08:36:17.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.665+0000 7f993ed3d640 1 -- 172.21.15.86:0/672130526 wait complete. 2024-04-21T08:36:17.676 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph --format=json-pretty osd lspools 2024-04-21T08:36:17.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.788+0000 7f835dc25640 1 -- 172.21.15.86:0/129337662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8358101fe0 msgr2=0x7f8358102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:17.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.788+0000 7f835dc25640 1 --2- 172.21.15.86:0/129337662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8358101fe0 0x7f8358102460 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7f834c009900 tx=0x7f834c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:17.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.788+0000 7f835dc25640 1 -- 172.21.15.86:0/129337662 shutdown_connections 2024-04-21T08:36:17.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.788+0000 7f835dc25640 1 --2- 172.21.15.86:0/129337662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83581029a0 0x7f83581092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:17.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.788+0000 7f835dc25640 1 --2- 172.21.15.86:0/129337662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8358101fe0 0x7f8358102460 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:17.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.788+0000 7f835dc25640 1 --2- 172.21.15.86:0/129337662 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8358100e90 0x7f83581012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:17.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.788+0000 7f835dc25640 1 -- 172.21.15.86:0/129337662 >> 172.21.15.86:0/129337662 conn(0x7f83580fc400 msgr2=0x7f83580fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:17.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.788+0000 7f835dc25640 1 -- 172.21.15.86:0/129337662 shutdown_connections 2024-04-21T08:36:17.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.788+0000 7f835dc25640 1 -- 172.21.15.86:0/129337662 wait complete. 2024-04-21T08:36:17.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.788+0000 7f835dc25640 1 Processor -- start 2024-04-21T08:36:17.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.789+0000 7f835dc25640 1 -- start start 2024-04-21T08:36:17.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.789+0000 7f835dc25640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8358100e90 0x7f8358069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:17.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.789+0000 7f835dc25640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8358101fe0 0x7f835819cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:17.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.789+0000 7f835dc25640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83581029a0 0x7f835819f500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:17.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.789+0000 7f835dc25640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f835810c9e0 con 0x7f8358100e90 2024-04-21T08:36:17.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.789+0000 7f835dc25640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f835810c860 con 0x7f83581029a0 2024-04-21T08:36:17.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.789+0000 7f835dc25640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f835810cb60 con 0x7f8358101fe0 2024-04-21T08:36:17.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.789+0000 7f83577fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8358100e90 0x7f8358069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:17.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.789+0000 7f83577fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8358100e90 0x7f8358069c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50802/0 (socket says 172.21.15.86:50802) 2024-04-21T08:36:17.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.789+0000 7f83577fe640 1 -- 172.21.15.86:0/2649630551 learned_addr learned my addr 172.21.15.86:0/2649630551 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:17.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.789+0000 7f8356ffd640 1 --2- 172.21.15.86:0/2649630551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8358101fe0 0x7f835819cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:17.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.789+0000 7f8357fff640 1 --2- 172.21.15.86:0/2649630551 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83581029a0 0x7f835819f500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:17.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.789+0000 7f83577fe640 1 -- 172.21.15.86:0/2649630551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8358101fe0 msgr2=0x7f835819cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:17.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.789+0000 7f83577fe640 1 --2- 172.21.15.86:0/2649630551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8358101fe0 0x7f835819cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:17.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.789+0000 7f83577fe640 1 -- 172.21.15.86:0/2649630551 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83581029a0 msgr2=0x7f835819f500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:17.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.789+0000 7f83577fe640 1 --2- 172.21.15.86:0/2649630551 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83581029a0 0x7f835819f500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:17.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.789+0000 7f83577fe640 1 -- 172.21.15.86:0/2649630551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f835819fe10 con 0x7f8358100e90 2024-04-21T08:36:17.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.789+0000 7f83577fe640 1 --2- 172.21.15.86:0/2649630551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8358100e90 0x7f8358069c50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f834800e9f0 tx=0x7f834800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:17.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.790+0000 7f8354ff9640 1 -- 172.21.15.86:0/2649630551 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f834800cd80 con 0x7f8358100e90 2024-04-21T08:36:17.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.790+0000 7f8354ff9640 1 -- 172.21.15.86:0/2649630551 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f83480108e0 con 0x7f8358100e90 2024-04-21T08:36:17.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.790+0000 7f835dc25640 1 -- 172.21.15.86:0/2649630551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83581a00a0 con 0x7f8358100e90 2024-04-21T08:36:17.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.790+0000 7f8354ff9640 1 -- 172.21.15.86:0/2649630551 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8348018980 con 0x7f8358100e90 2024-04-21T08:36:17.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.790+0000 7f835dc25640 1 -- 172.21.15.86:0/2649630551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83581a05c0 con 0x7f8358100e90 2024-04-21T08:36:17.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.791+0000 7f835dc25640 1 -- 172.21.15.86:0/2649630551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8358108220 con 0x7f8358100e90 2024-04-21T08:36:17.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.792+0000 7f8354ff9640 1 -- 172.21.15.86:0/2649630551 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8348010430 con 0x7f8358100e90 2024-04-21T08:36:17.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.792+0000 7f8354ff9640 1 --2- 172.21.15.86:0/2649630551 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f83400a3bb0 0x7f83400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:17.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.793+0000 7f8356ffd640 1 --2- 172.21.15.86:0/2649630551 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f83400a3bb0 0x7f83400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:17.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.793+0000 7f8354ff9640 1 -- 172.21.15.86:0/2649630551 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7f83480d0230 con 0x7f8358100e90 2024-04-21T08:36:17.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.793+0000 7f8356ffd640 1 --2- 172.21.15.86:0/2649630551 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f83400a3bb0 0x7f83400a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f834c005ab0 tx=0x7f834c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:17.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.797+0000 7f8354ff9640 1 -- 172.21.15.86:0/2649630551 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f834809ca60 con 0x7f8358100e90 2024-04-21T08:36:17.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:17 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:17.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:17 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:17.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:17 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:17.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:17 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:17.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:17 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:17.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:17 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:17.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:17 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3267513351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:17.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:17 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:17.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:17 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:36:17.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:17 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:17.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:17 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:36:17.965 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:17 smithi088 ceph-mon[31259]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:17.965 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:17 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/682843403' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:17.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.985+0000 7f835dc25640 1 -- 172.21.15.86:0/2649630551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd lspools", "format": "json-pretty"} v 0) -- 0x7f8358062b50 con 0x7f8358100e90 2024-04-21T08:36:17.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.985+0000 7f8354ff9640 1 -- 172.21.15.86:0/2649630551 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd lspools", "format": "json-pretty"}]=0 v81) ==== 84+0+65 (secure 0 0 0) 0x7f83480a0d60 con 0x7f8358100e90 2024-04-21T08:36:17.984 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:36:17.985 INFO:teuthology.orchestra.run.smithi086.stdout:[ 2024-04-21T08:36:17.985 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:36:17.985 INFO:teuthology.orchestra.run.smithi086.stdout: "poolnum": 1, 2024-04-21T08:36:17.985 INFO:teuthology.orchestra.run.smithi086.stdout: "poolname": ".mgr" 2024-04-21T08:36:17.985 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:36:17.985 INFO:teuthology.orchestra.run.smithi086.stdout:] 2024-04-21T08:36:17.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.987+0000 7f835dc25640 1 -- 172.21.15.86:0/2649630551 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f83400a3bb0 msgr2=0x7f83400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:17.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.987+0000 7f835dc25640 1 --2- 172.21.15.86:0/2649630551 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f83400a3bb0 0x7f83400a6070 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f834c005ab0 tx=0x7f834c005fd0 comp rx=0 tx=0).stop 2024-04-21T08:36:17.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.987+0000 7f835dc25640 1 -- 172.21.15.86:0/2649630551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8358100e90 msgr2=0x7f8358069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:17.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.987+0000 7f835dc25640 1 --2- 172.21.15.86:0/2649630551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8358100e90 0x7f8358069c50 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f834800e9f0 tx=0x7f834800eeb0 comp rx=0 tx=0).stop 2024-04-21T08:36:17.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.987+0000 7f835dc25640 1 -- 172.21.15.86:0/2649630551 shutdown_connections 2024-04-21T08:36:17.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.987+0000 7f835dc25640 1 --2- 172.21.15.86:0/2649630551 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f83400a3bb0 0x7f83400a6070 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:17.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.987+0000 7f835dc25640 1 --2- 172.21.15.86:0/2649630551 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83581029a0 0x7f835819f500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:17.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.987+0000 7f835dc25640 1 --2- 172.21.15.86:0/2649630551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8358101fe0 0x7f835819cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:17.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.987+0000 7f835dc25640 1 --2- 172.21.15.86:0/2649630551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8358100e90 0x7f8358069c50 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:17.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.987+0000 7f835dc25640 1 -- 172.21.15.86:0/2649630551 >> 172.21.15.86:0/2649630551 conn(0x7f83580fc400 msgr2=0x7f83581053f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:17.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.987+0000 7f835dc25640 1 -- 172.21.15.86:0/2649630551 shutdown_connections 2024-04-21T08:36:17.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:17.987+0000 7f835dc25640 1 -- 172.21.15.86:0/2649630551 wait complete. 2024-04-21T08:36:17.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:17 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:17.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:17 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:17.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:17 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:17.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:17 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:17.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:17 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:17.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:17 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:17.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:17 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3267513351' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:17.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:17 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:17.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:17 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:36:17.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:17 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:17.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:17 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:36:17.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:17 smithi179 ceph-mon[31829]: pgmap v278: 1 pgs: 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:17.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:17 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/682843403' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:17.998 DEBUG:tasks.cephfs.filesystem:Creating filesystem 'cephfs' 2024-04-21T08:36:17.998 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_metadata --pg_num_min 64 2024-04-21T08:36:18.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.102+0000 7fec7cc06640 1 -- 172.21.15.86:0/3262350724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec781053e0 msgr2=0x7fec781077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:18.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.102+0000 7fec7cc06640 1 --2- 172.21.15.86:0/3262350724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec781053e0 0x7fec781077d0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fec6c009900 tx=0x7fec6c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:18.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.102+0000 7fec7cc06640 1 -- 172.21.15.86:0/3262350724 shutdown_connections 2024-04-21T08:36:18.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.102+0000 7fec7cc06640 1 --2- 172.21.15.86:0/3262350724 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec78107d10 0x7fec7810a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:18.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.102+0000 7fec7cc06640 1 --2- 172.21.15.86:0/3262350724 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec781053e0 0x7fec781077d0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:18.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.102+0000 7fec7cc06640 1 --2- 172.21.15.86:0/3262350724 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec78102ab0 0x7fec78104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:18.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.102+0000 7fec7cc06640 1 -- 172.21.15.86:0/3262350724 >> 172.21.15.86:0/3262350724 conn(0x7fec780fc440 msgr2=0x7fec780fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:18.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.102+0000 7fec7cc06640 1 -- 172.21.15.86:0/3262350724 shutdown_connections 2024-04-21T08:36:18.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.102+0000 7fec7cc06640 1 -- 172.21.15.86:0/3262350724 wait complete. 2024-04-21T08:36:18.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.103+0000 7fec7cc06640 1 Processor -- start 2024-04-21T08:36:18.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.103+0000 7fec7cc06640 1 -- start start 2024-04-21T08:36:18.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.103+0000 7fec7cc06640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec78102ab0 0x7fec78198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:18.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.103+0000 7fec7cc06640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec781053e0 0x7fec781987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:18.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.103+0000 7fec7cc06640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec78107d10 0x7fec7819f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:18.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.103+0000 7fec7cc06640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fec7810daf0 con 0x7fec78107d10 2024-04-21T08:36:18.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.103+0000 7fec7cc06640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fec7810d970 con 0x7fec781053e0 2024-04-21T08:36:18.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.103+0000 7fec7cc06640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fec7810dc70 con 0x7fec78102ab0 2024-04-21T08:36:18.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.103+0000 7fec76d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec78107d10 0x7fec7819f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:18.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.103+0000 7fec7659c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec78102ab0 0x7fec78198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:18.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.103+0000 7fec76d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec78107d10 0x7fec7819f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50822/0 (socket says 172.21.15.86:50822) 2024-04-21T08:36:18.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.103+0000 7fec76d9d640 1 -- 172.21.15.86:0/3981340307 learned_addr learned my addr 172.21.15.86:0/3981340307 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:18.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.103+0000 7fec75d9b640 1 --2- 172.21.15.86:0/3981340307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec781053e0 0x7fec781987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:18.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.103+0000 7fec76d9d640 1 -- 172.21.15.86:0/3981340307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec78102ab0 msgr2=0x7fec78198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:18.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.103+0000 7fec76d9d640 1 --2- 172.21.15.86:0/3981340307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec78102ab0 0x7fec78198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:18.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.103+0000 7fec76d9d640 1 -- 172.21.15.86:0/3981340307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec781053e0 msgr2=0x7fec781987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:18.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.103+0000 7fec76d9d640 1 --2- 172.21.15.86:0/3981340307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec781053e0 0x7fec781987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:18.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.103+0000 7fec76d9d640 1 -- 172.21.15.86:0/3981340307 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec7819feb0 con 0x7fec78107d10 2024-04-21T08:36:18.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.103+0000 7fec75d9b640 1 --2- 172.21.15.86:0/3981340307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec781053e0 0x7fec781987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:36:18.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.103+0000 7fec7659c640 1 --2- 172.21.15.86:0/3981340307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec78102ab0 0x7fec78198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:36:18.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.103+0000 7fec76d9d640 1 --2- 172.21.15.86:0/3981340307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec78107d10 0x7fec7819f840 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fec7000eb10 tx=0x7fec7000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:18.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.104+0000 7fec677fe640 1 -- 172.21.15.86:0/3981340307 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec70004b20 con 0x7fec78107d10 2024-04-21T08:36:18.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.104+0000 7fec7cc06640 1 -- 172.21.15.86:0/3981340307 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec781a01a0 con 0x7fec78107d10 2024-04-21T08:36:18.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.104+0000 7fec677fe640 1 -- 172.21.15.86:0/3981340307 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec70004cc0 con 0x7fec78107d10 2024-04-21T08:36:18.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.104+0000 7fec677fe640 1 -- 172.21.15.86:0/3981340307 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec70010690 con 0x7fec78107d10 2024-04-21T08:36:18.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.104+0000 7fec7cc06640 1 -- 172.21.15.86:0/3981340307 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec781a0710 con 0x7fec78107d10 2024-04-21T08:36:18.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.105+0000 7fec7cc06640 1 -- 172.21.15.86:0/3981340307 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fec780fdf30 con 0x7fec78107d10 2024-04-21T08:36:18.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.106+0000 7fec677fe640 1 -- 172.21.15.86:0/3981340307 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fec70005000 con 0x7fec78107d10 2024-04-21T08:36:18.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.106+0000 7fec677fe640 1 --2- 172.21.15.86:0/3981340307 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fec600a3bb0 0x7fec600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:18.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.107+0000 7fec7659c640 1 --2- 172.21.15.86:0/3981340307 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fec600a3bb0 0x7fec600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:18.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.107+0000 7fec677fe640 1 -- 172.21.15.86:0/3981340307 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(81..81 src has 1..81) ==== 7127+0+0 (secure 0 0 0) 0x7fec700d06c0 con 0x7fec78107d10 2024-04-21T08:36:18.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.107+0000 7fec7659c640 1 --2- 172.21.15.86:0/3981340307 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fec600a3bb0 0x7fec600a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fec68009fc0 tx=0x7fec68005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:18.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.110+0000 7fec677fe640 1 -- 172.21.15.86:0/3981340307 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fec70053ff0 con 0x7fec78107d10 2024-04-21T08:36:18.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:18.296+0000 7fec7cc06640 1 -- 172.21.15.86:0/3981340307 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64} v 0) -- 0x7fec7802d5c0 con 0x7fec78107d10 2024-04-21T08:36:18.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:18 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/672130526' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:36:18.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:18 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2649630551' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-21T08:36:18.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:18 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:18.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:18 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3981340307' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-21T08:36:18.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:18 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/672130526' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:36:18.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:18 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2649630551' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-21T08:36:18.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:18 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:18.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:18 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3981340307' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-21T08:36:18.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:18 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/672130526' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:36:18.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:18 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2649630551' entity='client.admin' cmd=[{"prefix": "osd lspools", "format": "json-pretty"}]: dispatch 2024-04-21T08:36:18.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:18 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:18.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:18 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3981340307' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]: dispatch 2024-04-21T08:36:19.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.260+0000 7fec677fe640 1 -- 172.21.15.86:0/3981340307 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]=0 pool 'cephfs_metadata' created v82) ==== 138+0+0 (secure 0 0 0) 0x7fec7009d540 con 0x7fec78107d10 2024-04-21T08:36:19.259 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' created 2024-04-21T08:36:19.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.263+0000 7fec7cc06640 1 -- 172.21.15.86:0/3981340307 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fec600a3bb0 msgr2=0x7fec600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:19.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.263+0000 7fec7cc06640 1 --2- 172.21.15.86:0/3981340307 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fec600a3bb0 0x7fec600a6070 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7fec68009fc0 tx=0x7fec68005f90 comp rx=0 tx=0).stop 2024-04-21T08:36:19.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.263+0000 7fec7cc06640 1 -- 172.21.15.86:0/3981340307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec78107d10 msgr2=0x7fec7819f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:19.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.263+0000 7fec7cc06640 1 --2- 172.21.15.86:0/3981340307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec78107d10 0x7fec7819f840 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fec7000eb10 tx=0x7fec7000c2d0 comp rx=0 tx=0).stop 2024-04-21T08:36:19.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.263+0000 7fec7cc06640 1 -- 172.21.15.86:0/3981340307 shutdown_connections 2024-04-21T08:36:19.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.263+0000 7fec7cc06640 1 --2- 172.21.15.86:0/3981340307 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fec600a3bb0 0x7fec600a6070 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:19.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.263+0000 7fec7cc06640 1 --2- 172.21.15.86:0/3981340307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec78107d10 0x7fec7819f840 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:19.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.263+0000 7fec7cc06640 1 --2- 172.21.15.86:0/3981340307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec781053e0 0x7fec781987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:19.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.263+0000 7fec7cc06640 1 --2- 172.21.15.86:0/3981340307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec78102ab0 0x7fec78198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:19.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.263+0000 7fec7cc06640 1 -- 172.21.15.86:0/3981340307 >> 172.21.15.86:0/3981340307 conn(0x7fec780fc440 msgr2=0x7fec78104950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:19.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.263+0000 7fec7cc06640 1 -- 172.21.15.86:0/3981340307 shutdown_connections 2024-04-21T08:36:19.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.263+0000 7fec7cc06640 1 -- 172.21.15.86:0/3981340307 wait complete. 2024-04-21T08:36:19.276 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data 64 --pg_num_min 64 --target_size_ratio 0.05 2024-04-21T08:36:19.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.385+0000 7f9fdf3a4640 1 -- 172.21.15.86:0/2697598999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fd8100e60 msgr2=0x7f9fd8101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:19.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.385+0000 7f9fdf3a4640 1 --2- 172.21.15.86:0/2697598999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fd8100e60 0x7f9fd8101280 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4009900 tx=0x7f9fd402bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:19.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.385+0000 7f9fdf3a4640 1 -- 172.21.15.86:0/2697598999 shutdown_connections 2024-04-21T08:36:19.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.385+0000 7f9fdf3a4640 1 --2- 172.21.15.86:0/2697598999 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9fd8102a00 0x7f9fd8109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:19.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.385+0000 7f9fdf3a4640 1 --2- 172.21.15.86:0/2697598999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fd8102040 0x7f9fd81024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:19.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.385+0000 7f9fdf3a4640 1 --2- 172.21.15.86:0/2697598999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fd8100e60 0x7f9fd8101280 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:19.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.385+0000 7f9fdf3a4640 1 -- 172.21.15.86:0/2697598999 >> 172.21.15.86:0/2697598999 conn(0x7f9fd80fc440 msgr2=0x7f9fd80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:19.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.385+0000 7f9fdf3a4640 1 -- 172.21.15.86:0/2697598999 shutdown_connections 2024-04-21T08:36:19.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.385+0000 7f9fdf3a4640 1 -- 172.21.15.86:0/2697598999 wait complete. 2024-04-21T08:36:19.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.385+0000 7f9fdf3a4640 1 Processor -- start 2024-04-21T08:36:19.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.385+0000 7f9fdf3a4640 1 -- start start 2024-04-21T08:36:19.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.385+0000 7f9fdf3a4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fd8100e60 0x7f9fd81982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:19.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.385+0000 7f9fdf3a4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9fd8102040 0x7f9fd81987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:19.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.385+0000 7f9fdf3a4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fd8102a00 0x7f9fd819f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:19.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.386+0000 7f9fdf3a4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd810db30 con 0x7f9fd8100e60 2024-04-21T08:36:19.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.386+0000 7f9fdf3a4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd810d9b0 con 0x7f9fd8102040 2024-04-21T08:36:19.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.386+0000 7f9fdf3a4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9fd810dcb0 con 0x7f9fd8102a00 2024-04-21T08:36:19.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.386+0000 7f9fdd140640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fd8100e60 0x7f9fd81982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:19.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.386+0000 7f9fdd140640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fd8100e60 0x7f9fd81982a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50832/0 (socket says 172.21.15.86:50832) 2024-04-21T08:36:19.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.386+0000 7f9fdd140640 1 -- 172.21.15.86:0/2703171206 learned_addr learned my addr 172.21.15.86:0/2703171206 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:19.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.386+0000 7f9fdd941640 1 --2- 172.21.15.86:0/2703171206 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fd8102a00 0x7f9fd819f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:19.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.386+0000 7f9fdc93f640 1 --2- 172.21.15.86:0/2703171206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9fd8102040 0x7f9fd81987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:19.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.386+0000 7f9fdd140640 1 -- 172.21.15.86:0/2703171206 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fd8102a00 msgr2=0x7f9fd819f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:19.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.386+0000 7f9fdd140640 1 --2- 172.21.15.86:0/2703171206 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fd8102a00 0x7f9fd819f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:19.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.386+0000 7f9fdd140640 1 -- 172.21.15.86:0/2703171206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9fd8102040 msgr2=0x7f9fd81987e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:19.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.386+0000 7f9fdd140640 1 --2- 172.21.15.86:0/2703171206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9fd8102040 0x7f9fd81987e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:19.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.386+0000 7f9fdd140640 1 -- 172.21.15.86:0/2703171206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9fd819fe70 con 0x7f9fd8100e60 2024-04-21T08:36:19.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.386+0000 7f9fdd941640 1 --2- 172.21.15.86:0/2703171206 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fd8102a00 0x7f9fd819f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:36:19.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.386+0000 7f9fdd140640 1 --2- 172.21.15.86:0/2703171206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fd8100e60 0x7f9fd81982a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4000c00 tx=0x7f9fd4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:19.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.386+0000 7f9fce7fc640 1 -- 172.21.15.86:0/2703171206 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4043020 con 0x7f9fd8100e60 2024-04-21T08:36:19.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.386+0000 7f9fdf3a4640 1 -- 172.21.15.86:0/2703171206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9fd81a0100 con 0x7f9fd8100e60 2024-04-21T08:36:19.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.386+0000 7f9fce7fc640 1 -- 172.21.15.86:0/2703171206 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9fd4004aa0 con 0x7f9fd8100e60 2024-04-21T08:36:19.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.386+0000 7f9fce7fc640 1 -- 172.21.15.86:0/2703171206 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9fd4003d00 con 0x7f9fd8100e60 2024-04-21T08:36:19.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.386+0000 7f9fdf3a4640 1 -- 172.21.15.86:0/2703171206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9fd81a0650 con 0x7f9fd8100e60 2024-04-21T08:36:19.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.387+0000 7f9fdf3a4640 1 -- 172.21.15.86:0/2703171206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9fd8108370 con 0x7f9fd8100e60 2024-04-21T08:36:19.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.388+0000 7f9fce7fc640 1 -- 172.21.15.86:0/2703171206 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f9fd4046020 con 0x7f9fd8100e60 2024-04-21T08:36:19.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.389+0000 7f9fce7fc640 1 --2- 172.21.15.86:0/2703171206 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f9fc40a3bb0 0x7f9fc40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:19.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.389+0000 7f9fdc93f640 1 --2- 172.21.15.86:0/2703171206 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f9fc40a3bb0 0x7f9fc40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:19.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.389+0000 7f9fce7fc640 1 -- 172.21.15.86:0/2703171206 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(82..82 src has 1..82) ==== 7497+0+0 (secure 0 0 0) 0x7f9fd40f24f0 con 0x7f9fd8100e60 2024-04-21T08:36:19.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.389+0000 7f9fdc93f640 1 --2- 172.21.15.86:0/2703171206 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f9fc40a3bb0 0x7f9fc40a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8005fd0 tx=0x7f9fc8005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:19.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.393+0000 7f9fce7fc640 1 -- 172.21.15.86:0/2703171206 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9fd404d020 con 0x7f9fd8100e60 2024-04-21T08:36:19.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:19.570+0000 7f9fdf3a4640 1 -- 172.21.15.86:0/2703171206 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05} v 0) -- 0x7f9fd802d5c0 con 0x7f9fd8100e60 2024-04-21T08:36:20.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:19 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:20.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:19 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:20.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:19 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:20.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:19 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:20.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:19 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:20.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:19 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:20.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:19 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:20.215 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:19 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:36:20.215 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:19 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:20.215 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:19 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3981340307' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-21T08:36:20.215 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:19 smithi088 ceph-mon[31259]: osdmap e82: 12 total, 12 up, 12 in 2024-04-21T08:36:20.215 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:19 smithi088 ceph-mon[31259]: pgmap v280: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:20.215 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:19 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2703171206' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-21T08:36:20.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:19 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:20.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:19 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:20.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:19 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:20.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:19 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:20.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:19 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:20.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:19 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:20.248 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:19 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:20.248 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:19 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:36:20.248 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:19 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:20.248 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:19 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3981340307' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-21T08:36:20.248 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:19 smithi179 ceph-mon[31829]: osdmap e82: 12 total, 12 up, 12 in 2024-04-21T08:36:20.248 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:19 smithi179 ceph-mon[31829]: pgmap v280: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:20.249 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:19 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2703171206' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-21T08:36:20.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.265+0000 7f9fce7fc640 1 -- 172.21.15.86:0/2703171206 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]=0 pool 'cephfs_data' created v83) ==== 171+0+0 (secure 0 0 0) 0x7f9fd40bf200 con 0x7f9fd8100e60 2024-04-21T08:36:20.265 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' created 2024-04-21T08:36:20.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.268+0000 7f9fdf3a4640 1 -- 172.21.15.86:0/2703171206 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f9fc40a3bb0 msgr2=0x7f9fc40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:20.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.268+0000 7f9fdf3a4640 1 --2- 172.21.15.86:0/2703171206 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f9fc40a3bb0 0x7f9fc40a6070 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f9fc8005fd0 tx=0x7f9fc8005850 comp rx=0 tx=0).stop 2024-04-21T08:36:20.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.268+0000 7f9fdf3a4640 1 -- 172.21.15.86:0/2703171206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fd8100e60 msgr2=0x7f9fd81982a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:20.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.268+0000 7f9fdf3a4640 1 --2- 172.21.15.86:0/2703171206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fd8100e60 0x7f9fd81982a0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f9fd4000c00 tx=0x7f9fd4004270 comp rx=0 tx=0).stop 2024-04-21T08:36:20.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.269+0000 7f9fdf3a4640 1 -- 172.21.15.86:0/2703171206 shutdown_connections 2024-04-21T08:36:20.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.269+0000 7f9fdf3a4640 1 --2- 172.21.15.86:0/2703171206 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f9fc40a3bb0 0x7f9fc40a6070 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:20.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.269+0000 7f9fdf3a4640 1 --2- 172.21.15.86:0/2703171206 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9fd8102a00 0x7f9fd819f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:20.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.269+0000 7f9fdf3a4640 1 --2- 172.21.15.86:0/2703171206 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9fd8102040 0x7f9fd81987e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:20.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.269+0000 7f9fdf3a4640 1 --2- 172.21.15.86:0/2703171206 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9fd8100e60 0x7f9fd81982a0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:20.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.269+0000 7f9fdf3a4640 1 -- 172.21.15.86:0/2703171206 >> 172.21.15.86:0/2703171206 conn(0x7f9fd80fc440 msgr2=0x7f9fd8105540 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:20.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.269+0000 7f9fdf3a4640 1 -- 172.21.15.86:0/2703171206 shutdown_connections 2024-04-21T08:36:20.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.269+0000 7f9fdf3a4640 1 -- 172.21.15.86:0/2703171206 wait complete. 2024-04-21T08:36:20.292 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs new cephfs cephfs_metadata cephfs_data 2024-04-21T08:36:20.292 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:19 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:20.293 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:19 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:20.293 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:19 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:20.293 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:19 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:20.293 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:19 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:20.293 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:19 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:20.294 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:19 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:36:20.294 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:19 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:36:20.294 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:19 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:36:20.294 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:19 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3981340307' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_metadata", "pg_num_min": 64}]': finished 2024-04-21T08:36:20.294 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:19 smithi086 ceph-mon[27939]: osdmap e82: 12 total, 12 up, 12 in 2024-04-21T08:36:20.294 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:19 smithi086 ceph-mon[27939]: pgmap v280: 33 pgs: 32 unknown, 1 active+clean; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:20.295 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:19 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2703171206' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]: dispatch 2024-04-21T08:36:20.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.399+0000 7f55699ef640 1 -- 172.21.15.86:0/4012800201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5564102a00 msgr2=0x7f5564109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:20.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.399+0000 7f55699ef640 1 --2- 172.21.15.86:0/4012800201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5564102a00 0x7f5564109320 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f555c009ca0 tx=0x7f555c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:36:20.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.400+0000 7f55699ef640 1 -- 172.21.15.86:0/4012800201 shutdown_connections 2024-04-21T08:36:20.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.400+0000 7f55699ef640 1 --2- 172.21.15.86:0/4012800201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5564102a00 0x7f5564109320 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:20.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.400+0000 7f55699ef640 1 --2- 172.21.15.86:0/4012800201 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5564102040 0x7f55641024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:20.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.400+0000 7f55699ef640 1 --2- 172.21.15.86:0/4012800201 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5564100e60 0x7f5564101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:20.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.400+0000 7f55699ef640 1 -- 172.21.15.86:0/4012800201 >> 172.21.15.86:0/4012800201 conn(0x7f55640fc420 msgr2=0x7f55640fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:20.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.400+0000 7f55699ef640 1 -- 172.21.15.86:0/4012800201 shutdown_connections 2024-04-21T08:36:20.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.400+0000 7f55699ef640 1 -- 172.21.15.86:0/4012800201 wait complete. 2024-04-21T08:36:20.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.400+0000 7f55699ef640 1 Processor -- start 2024-04-21T08:36:20.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.400+0000 7f55699ef640 1 -- start start 2024-04-21T08:36:20.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.400+0000 7f55699ef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5564100e60 0x7f556406b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:20.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.400+0000 7f55699ef640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5564102040 0x7f556419cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:20.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.400+0000 7f55699ef640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5564102a00 0x7f556419f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:20.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.400+0000 7f55699ef640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f556410ca90 con 0x7f5564100e60 2024-04-21T08:36:20.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.400+0000 7f55699ef640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f556410c910 con 0x7f5564102040 2024-04-21T08:36:20.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.400+0000 7f55699ef640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f556410cc10 con 0x7f5564102a00 2024-04-21T08:36:20.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.401+0000 7f5562ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5564100e60 0x7f556406b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:20.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.401+0000 7f5562ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5564100e60 0x7f556406b0c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35904/0 (socket says 172.21.15.86:35904) 2024-04-21T08:36:20.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.401+0000 7f5562ffd640 1 -- 172.21.15.86:0/3728152986 learned_addr learned my addr 172.21.15.86:0/3728152986 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:20.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.401+0000 7f5562ffd640 1 -- 172.21.15.86:0/3728152986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5564102a00 msgr2=0x7f556419f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:20.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.401+0000 7f5562ffd640 1 --2- 172.21.15.86:0/3728152986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5564102a00 0x7f556419f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:20.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.401+0000 7f5562ffd640 1 -- 172.21.15.86:0/3728152986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5564102040 msgr2=0x7f556419cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:20.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.401+0000 7f5562ffd640 1 --2- 172.21.15.86:0/3728152986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5564102040 0x7f556419cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:20.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.401+0000 7f5562ffd640 1 -- 172.21.15.86:0/3728152986 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f556419feb0 con 0x7f5564100e60 2024-04-21T08:36:20.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.401+0000 7f5562ffd640 1 --2- 172.21.15.86:0/3728152986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5564100e60 0x7f556406b0c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f555800b450 tx=0x7f555800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:20.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.401+0000 7f55689ed640 1 -- 172.21.15.86:0/3728152986 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5558015020 con 0x7f5564100e60 2024-04-21T08:36:20.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.401+0000 7f55689ed640 1 -- 172.21.15.86:0/3728152986 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5558011d40 con 0x7f5564100e60 2024-04-21T08:36:20.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.401+0000 7f55699ef640 1 -- 172.21.15.86:0/3728152986 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f55641a0140 con 0x7f5564100e60 2024-04-21T08:36:20.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.401+0000 7f55689ed640 1 -- 172.21.15.86:0/3728152986 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f555800f340 con 0x7f5564100e60 2024-04-21T08:36:20.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.401+0000 7f55699ef640 1 -- 172.21.15.86:0/3728152986 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55641a06e0 con 0x7f5564100e60 2024-04-21T08:36:20.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.402+0000 7f55699ef640 1 -- 172.21.15.86:0/3728152986 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5564108280 con 0x7f5564100e60 2024-04-21T08:36:20.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.403+0000 7f55689ed640 1 -- 172.21.15.86:0/3728152986 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f5558018020 con 0x7f5564100e60 2024-04-21T08:36:20.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.404+0000 7f55689ed640 1 --2- 172.21.15.86:0/3728152986 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f554c0a3c30 0x7f554c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:20.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.404+0000 7f55627fc640 1 --2- 172.21.15.86:0/3728152986 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f554c0a3c30 0x7f554c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:20.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.404+0000 7f55689ed640 1 -- 172.21.15.86:0/3728152986 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(83..83 src has 1..83) ==== 7879+0+0 (secure 0 0 0) 0x7f55580cfd10 con 0x7f5564100e60 2024-04-21T08:36:20.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.404+0000 7f55627fc640 1 --2- 172.21.15.86:0/3728152986 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f554c0a3c30 0x7f554c0a60f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5554005fd0 tx=0x7f5554005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:20.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.408+0000 7f55689ed640 1 -- 172.21.15.86:0/3728152986 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f555801f020 con 0x7f5564100e60 2024-04-21T08:36:20.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:20.625+0000 7f55699ef640 1 -- 172.21.15.86:0/3728152986 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"} v 0) -- 0x7f5564062c60 con 0x7f5564100e60 2024-04-21T08:36:21.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:20 smithi088 ceph-mon[31259]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-21T08:36:21.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:20 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2703171206' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-21T08:36:21.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:20 smithi088 ceph-mon[31259]: osdmap e83: 12 total, 12 up, 12 in 2024-04-21T08:36:21.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:20 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3728152986' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-21T08:36:21.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:20 smithi088 ceph-mon[31259]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-21T08:36:21.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:20 smithi179 ceph-mon[31829]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-21T08:36:21.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:20 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2703171206' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-21T08:36:21.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:20 smithi179 ceph-mon[31829]: osdmap e83: 12 total, 12 up, 12 in 2024-04-21T08:36:21.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:20 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3728152986' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-21T08:36:21.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:20 smithi179 ceph-mon[31829]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-21T08:36:21.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:20 smithi086 ceph-mon[27939]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-21T08:36:21.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:20 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2703171206' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data", "pg_num": 64, "pg_num_min": 64, "target_size_ratio": 0.05}]': finished 2024-04-21T08:36:21.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:20 smithi086 ceph-mon[27939]: osdmap e83: 12 total, 12 up, 12 in 2024-04-21T08:36:21.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:20 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3728152986' entity='client.admin' cmd=[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]: dispatch 2024-04-21T08:36:21.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:20 smithi086 ceph-mon[27939]: daemon mds.b assigned to filesystem cephfs as rank 0 2024-04-21T08:36:21.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.425+0000 7f55689ed640 1 -- 172.21.15.86:0/3728152986 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]=0 new fs with metadata pool 2 and data pool 3 v14) ==== 172+0+0 (secure 0 0 0) 0x7f555800d040 con 0x7f5564100e60 2024-04-21T08:36:21.425 INFO:teuthology.orchestra.run.smithi086.stderr:new fs with metadata pool 2 and data pool 3 2024-04-21T08:36:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.428+0000 7f55699ef640 1 -- 172.21.15.86:0/3728152986 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f554c0a3c30 msgr2=0x7f554c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:21.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.428+0000 7f55699ef640 1 --2- 172.21.15.86:0/3728152986 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f554c0a3c30 0x7f554c0a60f0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f5554005fd0 tx=0x7f5554005850 comp rx=0 tx=0).stop 2024-04-21T08:36:21.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.428+0000 7f55699ef640 1 -- 172.21.15.86:0/3728152986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5564100e60 msgr2=0x7f556406b0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:21.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.428+0000 7f55699ef640 1 --2- 172.21.15.86:0/3728152986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5564100e60 0x7f556406b0c0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f555800b450 tx=0x7f555800b910 comp rx=0 tx=0).stop 2024-04-21T08:36:21.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.428+0000 7f55699ef640 1 -- 172.21.15.86:0/3728152986 shutdown_connections 2024-04-21T08:36:21.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.428+0000 7f55699ef640 1 --2- 172.21.15.86:0/3728152986 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f554c0a3c30 0x7f554c0a60f0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.428+0000 7f55699ef640 1 --2- 172.21.15.86:0/3728152986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5564102a00 0x7f556419f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.428+0000 7f55699ef640 1 --2- 172.21.15.86:0/3728152986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5564102040 0x7f556419cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.428+0000 7f55699ef640 1 --2- 172.21.15.86:0/3728152986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5564100e60 0x7f556406b0c0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.428+0000 7f55699ef640 1 -- 172.21.15.86:0/3728152986 >> 172.21.15.86:0/3728152986 conn(0x7f55640fc420 msgr2=0x7f5564105450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:21.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.428+0000 7f55699ef640 1 -- 172.21.15.86:0/3728152986 shutdown_connections 2024-04-21T08:36:21.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.428+0000 7f55699ef640 1 -- 172.21.15.86:0/3728152986 wait complete. 2024-04-21T08:36:21.443 DEBUG:tasks.cephfs.filesystem:EC profile is ['m=2', 'k=2', 'crush-failure-domain=osd'] 2024-04-21T08:36:21.443 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd erasure-code-profile set cephfs_data_ec m=2 k=2 crush-failure-domain=osd 2024-04-21T08:36:21.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.550+0000 7f76c4f9f640 1 -- 172.21.15.86:0/3936212088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76c0102040 msgr2=0x7f76c01024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:21.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.550+0000 7f76c4f9f640 1 --2- 172.21.15.86:0/3936212088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76c0102040 0x7f76c01024c0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7f76b4009900 tx=0x7f76b402bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:21.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.550+0000 7f76c4f9f640 1 -- 172.21.15.86:0/3936212088 shutdown_connections 2024-04-21T08:36:21.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.550+0000 7f76c4f9f640 1 --2- 172.21.15.86:0/3936212088 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76c0102a00 0x7f76c0109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.550+0000 7f76c4f9f640 1 --2- 172.21.15.86:0/3936212088 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76c0102040 0x7f76c01024c0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.550+0000 7f76c4f9f640 1 --2- 172.21.15.86:0/3936212088 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76c0100e60 0x7f76c0101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.550+0000 7f76c4f9f640 1 -- 172.21.15.86:0/3936212088 >> 172.21.15.86:0/3936212088 conn(0x7f76c00fc420 msgr2=0x7f76c00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:21.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.551+0000 7f76c4f9f640 1 -- 172.21.15.86:0/3936212088 shutdown_connections 2024-04-21T08:36:21.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.551+0000 7f76c4f9f640 1 -- 172.21.15.86:0/3936212088 wait complete. 2024-04-21T08:36:21.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.551+0000 7f76c4f9f640 1 Processor -- start 2024-04-21T08:36:21.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.551+0000 7f76c4f9f640 1 -- start start 2024-04-21T08:36:21.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.551+0000 7f76c4f9f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76c0100e60 0x7f76c0197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:21.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.551+0000 7f76c4f9f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76c0102040 0x7f76c0197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:21.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.551+0000 7f76c4f9f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76c0102a00 0x7f76c019e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:21.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.551+0000 7f76c4f9f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f76c010ca70 con 0x7f76c0102040 2024-04-21T08:36:21.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.551+0000 7f76c4f9f640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f76c010c8f0 con 0x7f76c0100e60 2024-04-21T08:36:21.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.551+0000 7f76c4f9f640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f76c010cbf0 con 0x7f76c0102a00 2024-04-21T08:36:21.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.551+0000 7f76bdd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76c0102040 0x7f76c0197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:21.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.552+0000 7f76bdd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76c0102040 0x7f76c0197650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35922/0 (socket says 172.21.15.86:35922) 2024-04-21T08:36:21.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.552+0000 7f76bdd9b640 1 -- 172.21.15.86:0/1486031264 learned_addr learned my addr 172.21.15.86:0/1486031264 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:21.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.552+0000 7f76bdd9b640 1 -- 172.21.15.86:0/1486031264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76c0102a00 msgr2=0x7f76c019e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:21.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.552+0000 7f76bdd9b640 1 --2- 172.21.15.86:0/1486031264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76c0102a00 0x7f76c019e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.552+0000 7f76bdd9b640 1 -- 172.21.15.86:0/1486031264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76c0100e60 msgr2=0x7f76c0197110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:21.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.552+0000 7f76be59c640 1 --2- 172.21.15.86:0/1486031264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76c0100e60 0x7f76c0197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:21.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.552+0000 7f76bdd9b640 1 --2- 172.21.15.86:0/1486031264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76c0100e60 0x7f76c0197110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.552+0000 7f76bdd9b640 1 -- 172.21.15.86:0/1486031264 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f76c019ed30 con 0x7f76c0102040 2024-04-21T08:36:21.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.552+0000 7f76bdd9b640 1 --2- 172.21.15.86:0/1486031264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76c0102040 0x7f76c0197650 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f76b4036040 tx=0x7f76b40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:21.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.552+0000 7f76af7fe640 1 -- 172.21.15.86:0/1486031264 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b4043020 con 0x7f76c0102040 2024-04-21T08:36:21.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.552+0000 7f76c4f9f640 1 -- 172.21.15.86:0/1486031264 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f76c019efc0 con 0x7f76c0102040 2024-04-21T08:36:21.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.552+0000 7f76af7fe640 1 -- 172.21.15.86:0/1486031264 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f76b40054c0 con 0x7f76c0102040 2024-04-21T08:36:21.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.552+0000 7f76af7fe640 1 -- 172.21.15.86:0/1486031264 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f76b4003db0 con 0x7f76c0102040 2024-04-21T08:36:21.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.552+0000 7f76c4f9f640 1 -- 172.21.15.86:0/1486031264 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f76c019f500 con 0x7f76c0102040 2024-04-21T08:36:21.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.553+0000 7f76c4f9f640 1 -- 172.21.15.86:0/1486031264 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f76c0108200 con 0x7f76c0102040 2024-04-21T08:36:21.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.554+0000 7f76af7fe640 1 -- 172.21.15.86:0/1486031264 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f76b4046020 con 0x7f76c0102040 2024-04-21T08:36:21.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.555+0000 7f76af7fe640 1 --2- 172.21.15.86:0/1486031264 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f76a80a3bb0 0x7f76a80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:21.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.555+0000 7f76be59c640 1 --2- 172.21.15.86:0/1486031264 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f76a80a3bb0 0x7f76a80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:21.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.555+0000 7f76af7fe640 1 -- 172.21.15.86:0/1486031264 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(84..84 src has 1..84) ==== 7979+0+0 (secure 0 0 0) 0x7f76b40f27a0 con 0x7f76c0102040 2024-04-21T08:36:21.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.555+0000 7f76be59c640 1 --2- 172.21.15.86:0/1486031264 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f76a80a3bb0 0x7f76a80a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f76c0101bb0 tx=0x7f76b0008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:21.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.560+0000 7f76af7fe640 1 -- 172.21.15.86:0/1486031264 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f76b404d020 con 0x7f76c0102040 2024-04-21T08:36:21.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.747+0000 7f76c4f9f640 1 -- 172.21.15.86:0/1486031264 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]} v 0) -- 0x7f76c002d810 con 0x7f76c0102040 2024-04-21T08:36:21.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.890+0000 7f76af7fe640 1 -- 172.21.15.86:0/1486031264 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]=0 v85) ==== 157+0+0 (secure 0 0 0) 0x7f76b40bf2d0 con 0x7f76c0102040 2024-04-21T08:36:21.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.893+0000 7f76c4f9f640 1 -- 172.21.15.86:0/1486031264 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f76a80a3bb0 msgr2=0x7f76a80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:21.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.893+0000 7f76c4f9f640 1 --2- 172.21.15.86:0/1486031264 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f76a80a3bb0 0x7f76a80a6070 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7f76c0101bb0 tx=0x7f76b0008040 comp rx=0 tx=0).stop 2024-04-21T08:36:21.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.893+0000 7f76c4f9f640 1 -- 172.21.15.86:0/1486031264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76c0102040 msgr2=0x7f76c0197650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:21.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.893+0000 7f76c4f9f640 1 --2- 172.21.15.86:0/1486031264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76c0102040 0x7f76c0197650 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f76b4036040 tx=0x7f76b40042f0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.893+0000 7f76c4f9f640 1 -- 172.21.15.86:0/1486031264 shutdown_connections 2024-04-21T08:36:21.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.893+0000 7f76c4f9f640 1 --2- 172.21.15.86:0/1486031264 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f76a80a3bb0 0x7f76a80a6070 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.893+0000 7f76c4f9f640 1 --2- 172.21.15.86:0/1486031264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f76c0102a00 0x7f76c019e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.893+0000 7f76c4f9f640 1 --2- 172.21.15.86:0/1486031264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f76c0102040 0x7f76c0197650 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.893+0000 7f76c4f9f640 1 --2- 172.21.15.86:0/1486031264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f76c0100e60 0x7f76c0197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:21.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.893+0000 7f76c4f9f640 1 -- 172.21.15.86:0/1486031264 >> 172.21.15.86:0/1486031264 conn(0x7f76c00fc420 msgr2=0x7f76c01053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:21.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.893+0000 7f76c4f9f640 1 -- 172.21.15.86:0/1486031264 shutdown_connections 2024-04-21T08:36:21.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:21.893+0000 7f76c4f9f640 1 -- 172.21.15.86:0/1486031264 wait complete. 2024-04-21T08:36:21.903 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create cephfs_data_ec erasure cephfs_data_ec --pg_num_min 64 --target_size_ratio 0.9 2024-04-21T08:36:22.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.011+0000 7f2e4f5b6640 1 -- 172.21.15.86:0/807817384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e48102a00 msgr2=0x7f2e48109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:22.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.011+0000 7f2e4f5b6640 1 --2- 172.21.15.86:0/807817384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e48102a00 0x7f2e48109320 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f2e40009ca0 tx=0x7f2e4002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:36:22.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.011+0000 7f2e4f5b6640 1 -- 172.21.15.86:0/807817384 shutdown_connections 2024-04-21T08:36:22.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.011+0000 7f2e4f5b6640 1 --2- 172.21.15.86:0/807817384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e48102a00 0x7f2e48109320 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:22.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.011+0000 7f2e4f5b6640 1 --2- 172.21.15.86:0/807817384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e48102040 0x7f2e481024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:22.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.011+0000 7f2e4f5b6640 1 --2- 172.21.15.86:0/807817384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e48100e60 0x7f2e48101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:22.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.011+0000 7f2e4f5b6640 1 -- 172.21.15.86:0/807817384 >> 172.21.15.86:0/807817384 conn(0x7f2e480fc440 msgr2=0x7f2e480fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:22.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.011+0000 7f2e4f5b6640 1 -- 172.21.15.86:0/807817384 shutdown_connections 2024-04-21T08:36:22.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.011+0000 7f2e4f5b6640 1 -- 172.21.15.86:0/807817384 wait complete. 2024-04-21T08:36:22.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.012+0000 7f2e4f5b6640 1 Processor -- start 2024-04-21T08:36:22.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.012+0000 7f2e4f5b6640 1 -- start start 2024-04-21T08:36:22.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.012+0000 7f2e4f5b6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e48100e60 0x7f2e48198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:22.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.012+0000 7f2e4f5b6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e48102040 0x7f2e481986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:22.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.012+0000 7f2e4f5b6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e48102a00 0x7f2e4819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:22.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.012+0000 7f2e4f5b6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4810ca00 con 0x7f2e48102040 2024-04-21T08:36:22.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.012+0000 7f2e4f5b6640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4810c880 con 0x7f2e48100e60 2024-04-21T08:36:22.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.012+0000 7f2e4f5b6640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2e4810cb80 con 0x7f2e48102a00 2024-04-21T08:36:22.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.012+0000 7f2e4cb51640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e48102040 0x7f2e481986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:22.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.012+0000 7f2e4cb51640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e48102040 0x7f2e481986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35926/0 (socket says 172.21.15.86:35926) 2024-04-21T08:36:22.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.012+0000 7f2e4cb51640 1 -- 172.21.15.86:0/2884863572 learned_addr learned my addr 172.21.15.86:0/2884863572 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:22.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.012+0000 7f2e4cb51640 1 -- 172.21.15.86:0/2884863572 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e48102a00 msgr2=0x7f2e4819f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:22.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.012+0000 7f2e4cb51640 1 --2- 172.21.15.86:0/2884863572 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e48102a00 0x7f2e4819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:22.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.012+0000 7f2e4cb51640 1 -- 172.21.15.86:0/2884863572 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e48100e60 msgr2=0x7f2e48198160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:22.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.012+0000 7f2e4d352640 1 --2- 172.21.15.86:0/2884863572 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e48100e60 0x7f2e48198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:22.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.012+0000 7f2e4cb51640 1 --2- 172.21.15.86:0/2884863572 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e48100e60 0x7f2e48198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:22.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.012+0000 7f2e4cb51640 1 -- 172.21.15.86:0/2884863572 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2e4819fd80 con 0x7f2e48102040 2024-04-21T08:36:22.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.012+0000 7f2e4cb51640 1 --2- 172.21.15.86:0/2884863572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e48102040 0x7f2e481986a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2e3800d770 tx=0x7f2e3800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:22.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.012+0000 7f2e3e7fc640 1 -- 172.21.15.86:0/2884863572 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38017070 con 0x7f2e48102040 2024-04-21T08:36:22.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.013+0000 7f2e3e7fc640 1 -- 172.21.15.86:0/2884863572 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2e380054d0 con 0x7f2e48102040 2024-04-21T08:36:22.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.013+0000 7f2e4f5b6640 1 -- 172.21.15.86:0/2884863572 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2e481a0070 con 0x7f2e48102040 2024-04-21T08:36:22.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.013+0000 7f2e3e7fc640 1 -- 172.21.15.86:0/2884863572 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2e38004c30 con 0x7f2e48102040 2024-04-21T08:36:22.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.013+0000 7f2e4f5b6640 1 -- 172.21.15.86:0/2884863572 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2e481a05c0 con 0x7f2e48102040 2024-04-21T08:36:22.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.014+0000 7f2e4f5b6640 1 -- 172.21.15.86:0/2884863572 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2e481a0820 con 0x7f2e48102040 2024-04-21T08:36:22.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.015+0000 7f2e3e7fc640 1 -- 172.21.15.86:0/2884863572 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2e38004dd0 con 0x7f2e48102040 2024-04-21T08:36:22.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.015+0000 7f2e3e7fc640 1 --2- 172.21.15.86:0/2884863572 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f2e340a3c30 0x7f2e340a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:22.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.015+0000 7f2e3e7fc640 1 -- 172.21.15.86:0/2884863572 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(85..85 src has 1..85) ==== 8277+0+0 (secure 0 0 0) 0x7f2e380d0350 con 0x7f2e48102040 2024-04-21T08:36:22.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.015+0000 7f2e4d352640 1 --2- 172.21.15.86:0/2884863572 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f2e340a3c30 0x7f2e340a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:22.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.016+0000 7f2e4d352640 1 --2- 172.21.15.86:0/2884863572 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f2e340a3c30 0x7f2e340a60f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2e48101bb0 tx=0x7f2e44005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:22.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.019+0000 7f2e3e7fc640 1 -- 172.21.15.86:0/2884863572 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2e3809cd80 con 0x7f2e48102040 2024-04-21T08:36:22.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:22.196+0000 7f2e4f5b6640 1 -- 172.21.15.86:0/2884863572 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9} v 0) -- 0x7f2e481a1020 con 0x7f2e48102040 2024-04-21T08:36:22.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:21 smithi088 ceph-mon[31259]: osdmap e84: 12 total, 12 up, 12 in 2024-04-21T08:36:22.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:21 smithi088 ceph-mon[31259]: pgmap v283: 97 pgs: 18 creating+peering, 8 active+clean, 71 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:22.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:21 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3728152986' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-21T08:36:22.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:21 smithi088 ceph-mon[31259]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-21T08:36:22.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:21 smithi088 ceph-mon[31259]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-21T08:36:22.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:21 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1486031264' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-21T08:36:22.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:21 smithi179 ceph-mon[31829]: osdmap e84: 12 total, 12 up, 12 in 2024-04-21T08:36:22.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:21 smithi179 ceph-mon[31829]: pgmap v283: 97 pgs: 18 creating+peering, 8 active+clean, 71 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:22.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:21 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3728152986' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-21T08:36:22.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:21 smithi179 ceph-mon[31829]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-21T08:36:22.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:21 smithi179 ceph-mon[31829]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-21T08:36:22.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:21 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1486031264' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-21T08:36:22.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:21 smithi086 ceph-mon[27939]: osdmap e84: 12 total, 12 up, 12 in 2024-04-21T08:36:22.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:21 smithi086 ceph-mon[27939]: pgmap v283: 97 pgs: 18 creating+peering, 8 active+clean, 71 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:22.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:21 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3728152986' entity='client.admin' cmd='[{"prefix": "fs new", "fs_name": "cephfs", "metadata": "cephfs_metadata", "data": "cephfs_data"}]': finished 2024-04-21T08:36:22.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:21 smithi086 ceph-mon[27939]: fsmap cephfs:1 {0=b=up:creating} 11 up:standby 2024-04-21T08:36:22.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:21 smithi086 ceph-mon[27939]: daemon mds.b is now active in filesystem cephfs as rank 0 2024-04-21T08:36:22.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:21 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1486031264' entity='client.admin' cmd=[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]: dispatch 2024-04-21T08:36:23.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:22 smithi088 ceph-mon[31259]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-21T08:36:23.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:22 smithi088 ceph-mon[31259]: Cluster is now healthy 2024-04-21T08:36:23.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:22 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1486031264' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-21T08:36:23.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:22 smithi088 ceph-mon[31259]: osdmap e85: 12 total, 12 up, 12 in 2024-04-21T08:36:23.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:22 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2884863572' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-21T08:36:23.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:22 smithi088 ceph-mon[31259]: mds.? [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] up:active 2024-04-21T08:36:23.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:22 smithi088 ceph-mon[31259]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T08:36:23.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:22 smithi088 ceph-mon[31259]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T08:36:23.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:22 smithi179 ceph-mon[31829]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-21T08:36:23.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:22 smithi179 ceph-mon[31829]: Cluster is now healthy 2024-04-21T08:36:23.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:22 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1486031264' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-21T08:36:23.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:22 smithi179 ceph-mon[31829]: osdmap e85: 12 total, 12 up, 12 in 2024-04-21T08:36:23.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:22 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2884863572' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-21T08:36:23.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:22 smithi179 ceph-mon[31829]: mds.? [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] up:active 2024-04-21T08:36:23.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:22 smithi179 ceph-mon[31829]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T08:36:23.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:22 smithi179 ceph-mon[31829]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T08:36:23.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:22 smithi086 ceph-mon[27939]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-21T08:36:23.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:22 smithi086 ceph-mon[27939]: Cluster is now healthy 2024-04-21T08:36:23.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:22 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1486031264' entity='client.admin' cmd='[{"prefix": "osd erasure-code-profile set", "name": "cephfs_data_ec", "profile": ["m=2", "k=2", "crush-failure-domain=osd"]}]': finished 2024-04-21T08:36:23.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:22 smithi086 ceph-mon[27939]: osdmap e85: 12 total, 12 up, 12 in 2024-04-21T08:36:23.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:22 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2884863572' entity='client.admin' cmd=[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]: dispatch 2024-04-21T08:36:23.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:22 smithi086 ceph-mon[27939]: mds.? [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] up:active 2024-04-21T08:36:23.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:22 smithi086 ceph-mon[27939]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T08:36:23.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:22 smithi086 ceph-mon[27939]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T08:36:23.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:23.896+0000 7f2e3e7fc640 1 -- 172.21.15.86:0/2884863572 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]=0 pool 'cephfs_data_ec' created v87) ==== 228+0+0 (secure 0 0 0) 0x7f2e3809c680 con 0x7f2e48102040 2024-04-21T08:36:23.896 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data_ec' created 2024-04-21T08:36:23.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:23.899+0000 7f2e4f5b6640 1 -- 172.21.15.86:0/2884863572 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f2e340a3c30 msgr2=0x7f2e340a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:23.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:23.899+0000 7f2e4f5b6640 1 --2- 172.21.15.86:0/2884863572 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f2e340a3c30 0x7f2e340a60f0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f2e48101bb0 tx=0x7f2e44005c60 comp rx=0 tx=0).stop 2024-04-21T08:36:23.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:23.899+0000 7f2e4f5b6640 1 -- 172.21.15.86:0/2884863572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e48102040 msgr2=0x7f2e481986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:23.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:23.899+0000 7f2e4f5b6640 1 --2- 172.21.15.86:0/2884863572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e48102040 0x7f2e481986a0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f2e3800d770 tx=0x7f2e3800dc30 comp rx=0 tx=0).stop 2024-04-21T08:36:23.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:23.899+0000 7f2e4f5b6640 1 -- 172.21.15.86:0/2884863572 shutdown_connections 2024-04-21T08:36:23.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:23.899+0000 7f2e4f5b6640 1 --2- 172.21.15.86:0/2884863572 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f2e340a3c30 0x7f2e340a60f0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:23.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:23.899+0000 7f2e4f5b6640 1 --2- 172.21.15.86:0/2884863572 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2e48102a00 0x7f2e4819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:23.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:23.899+0000 7f2e4f5b6640 1 --2- 172.21.15.86:0/2884863572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2e48102040 0x7f2e481986a0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:23.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:23.899+0000 7f2e4f5b6640 1 --2- 172.21.15.86:0/2884863572 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2e48100e60 0x7f2e48198160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:23.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:23.899+0000 7f2e4f5b6640 1 -- 172.21.15.86:0/2884863572 >> 172.21.15.86:0/2884863572 conn(0x7f2e480fc440 msgr2=0x7f2e480fdec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:23.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:23.899+0000 7f2e4f5b6640 1 -- 172.21.15.86:0/2884863572 shutdown_connections 2024-04-21T08:36:23.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:23.899+0000 7f2e4f5b6640 1 -- 172.21.15.86:0/2884863572 wait complete. 2024-04-21T08:36:23.917 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool set cephfs_data_ec allow_ec_overwrites true 2024-04-21T08:36:24.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.026+0000 7fe961f96640 1 -- 172.21.15.86:0/38354847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe95c102a30 msgr2=0x7fe95c104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:24.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.026+0000 7fe961f96640 1 --2- 172.21.15.86:0/38354847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe95c102a30 0x7fe95c104e20 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fe94c009900 tx=0x7fe94c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:24.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.026+0000 7fe961f96640 1 -- 172.21.15.86:0/38354847 shutdown_connections 2024-04-21T08:36:24.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.026+0000 7fe961f96640 1 --2- 172.21.15.86:0/38354847 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe95c107c90 0x7fe95c10a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:24.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.026+0000 7fe961f96640 1 --2- 172.21.15.86:0/38354847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe95c105360 0x7fe95c107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:24.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.026+0000 7fe961f96640 1 --2- 172.21.15.86:0/38354847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe95c102a30 0x7fe95c104e20 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:24.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.026+0000 7fe961f96640 1 -- 172.21.15.86:0/38354847 >> 172.21.15.86:0/38354847 conn(0x7fe95c0fc440 msgr2=0x7fe95c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:24.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.026+0000 7fe961f96640 1 -- 172.21.15.86:0/38354847 shutdown_connections 2024-04-21T08:36:24.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.026+0000 7fe961f96640 1 -- 172.21.15.86:0/38354847 wait complete. 2024-04-21T08:36:24.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.026+0000 7fe961f96640 1 Processor -- start 2024-04-21T08:36:24.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.027+0000 7fe961f96640 1 -- start start 2024-04-21T08:36:24.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.027+0000 7fe961f96640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe95c102a30 0x7fe95c198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:24.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.027+0000 7fe961f96640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe95c105360 0x7fe95c198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:24.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.027+0000 7fe961f96640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe95c107c90 0x7fe95c19f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:24.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.027+0000 7fe961f96640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe95c10daf0 con 0x7fe95c107c90 2024-04-21T08:36:24.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.027+0000 7fe961f96640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe95c10d970 con 0x7fe95c102a30 2024-04-21T08:36:24.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.027+0000 7fe961f96640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe95c10dc70 con 0x7fe95c105360 2024-04-21T08:36:24.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.027+0000 7fe95bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe95c107c90 0x7fe95c19f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:24.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.027+0000 7fe95bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe95c107c90 0x7fe95c19f7b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35942/0 (socket says 172.21.15.86:35942) 2024-04-21T08:36:24.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.027+0000 7fe95bfff640 1 -- 172.21.15.86:0/2129325108 learned_addr learned my addr 172.21.15.86:0/2129325108 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:24.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.027+0000 7fe95b7fe640 1 --2- 172.21.15.86:0/2129325108 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe95c102a30 0x7fe95c198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:24.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.027+0000 7fe95affd640 1 --2- 172.21.15.86:0/2129325108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe95c105360 0x7fe95c198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:24.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.027+0000 7fe95bfff640 1 -- 172.21.15.86:0/2129325108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe95c105360 msgr2=0x7fe95c198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:24.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.027+0000 7fe95bfff640 1 --2- 172.21.15.86:0/2129325108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe95c105360 0x7fe95c198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:24.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.027+0000 7fe95bfff640 1 -- 172.21.15.86:0/2129325108 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe95c102a30 msgr2=0x7fe95c198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:24.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.027+0000 7fe95bfff640 1 --2- 172.21.15.86:0/2129325108 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe95c102a30 0x7fe95c198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:24.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.027+0000 7fe95bfff640 1 -- 172.21.15.86:0/2129325108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe95c19fe20 con 0x7fe95c107c90 2024-04-21T08:36:24.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.027+0000 7fe95bfff640 1 --2- 172.21.15.86:0/2129325108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe95c107c90 0x7fe95c19f7b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe95400eaa0 tx=0x7fe95400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:24.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.028+0000 7fe958ff9640 1 -- 172.21.15.86:0/2129325108 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe95400cdb0 con 0x7fe95c107c90 2024-04-21T08:36:24.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.028+0000 7fe961f96640 1 -- 172.21.15.86:0/2129325108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe95c1a0110 con 0x7fe95c107c90 2024-04-21T08:36:24.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.028+0000 7fe958ff9640 1 -- 172.21.15.86:0/2129325108 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe954005020 con 0x7fe95c107c90 2024-04-21T08:36:24.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.028+0000 7fe958ff9640 1 -- 172.21.15.86:0/2129325108 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe9540116d0 con 0x7fe95c107c90 2024-04-21T08:36:24.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.028+0000 7fe961f96640 1 -- 172.21.15.86:0/2129325108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe95c1a06b0 con 0x7fe95c107c90 2024-04-21T08:36:24.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.029+0000 7fe961f96640 1 -- 172.21.15.86:0/2129325108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe95c0fdf30 con 0x7fe95c107c90 2024-04-21T08:36:24.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.030+0000 7fe958ff9640 1 -- 172.21.15.86:0/2129325108 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe9540054a0 con 0x7fe95c107c90 2024-04-21T08:36:24.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.030+0000 7fe958ff9640 1 --2- 172.21.15.86:0/2129325108 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe9440a3bb0 0x7fe9440a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:24.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.031+0000 7fe95b7fe640 1 --2- 172.21.15.86:0/2129325108 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe9440a3bb0 0x7fe9440a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:24.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.031+0000 7fe958ff9640 1 -- 172.21.15.86:0/2129325108 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(87..87 src has 1..87) ==== 8770+0+0 (secure 0 0 0) 0x7fe9540d0b40 con 0x7fe95c107c90 2024-04-21T08:36:24.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.031+0000 7fe95b7fe640 1 --2- 172.21.15.86:0/2129325108 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe9440a3bb0 0x7fe9440a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe94c009230 tx=0x7fe94c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:24.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.034+0000 7fe958ff9640 1 -- 172.21.15.86:0/2129325108 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe954016030 con 0x7fe95c107c90 2024-04-21T08:36:24.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.213+0000 7fe961f96640 1 -- 172.21.15.86:0/2129325108 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"} v 0) -- 0x7fe95c062bc0 con 0x7fe95c107c90 2024-04-21T08:36:24.213 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:23 smithi088 ceph-mon[31259]: osdmap e86: 12 total, 12 up, 12 in 2024-04-21T08:36:24.213 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:23 smithi088 ceph-mon[31259]: pgmap v286: 97 pgs: 18 creating+peering, 45 active+clean, 34 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 6 op/s 2024-04-21T08:36:24.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:23 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2884863572' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-21T08:36:24.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:23 smithi088 ceph-mon[31259]: osdmap e87: 12 total, 12 up, 12 in 2024-04-21T08:36:24.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:23 smithi179 ceph-mon[31829]: osdmap e86: 12 total, 12 up, 12 in 2024-04-21T08:36:24.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:23 smithi179 ceph-mon[31829]: pgmap v286: 97 pgs: 18 creating+peering, 45 active+clean, 34 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 6 op/s 2024-04-21T08:36:24.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:23 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2884863572' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-21T08:36:24.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:23 smithi179 ceph-mon[31829]: osdmap e87: 12 total, 12 up, 12 in 2024-04-21T08:36:24.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:23 smithi086 ceph-mon[27939]: osdmap e86: 12 total, 12 up, 12 in 2024-04-21T08:36:24.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:23 smithi086 ceph-mon[27939]: pgmap v286: 97 pgs: 18 creating+peering, 45 active+clean, 34 unknown; 577 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 767 B/s wr, 6 op/s 2024-04-21T08:36:24.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:23 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2884863572' entity='client.admin' cmd='[{"prefix": "osd pool create", "pool": "cephfs_data_ec", "pool_type": "erasure", "erasure_code_profile": "cephfs_data_ec", "pg_num_min": 64, "target_size_ratio": 0.9}]': finished 2024-04-21T08:36:24.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:23 smithi086 ceph-mon[27939]: osdmap e87: 12 total, 12 up, 12 in 2024-04-21T08:36:24.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.898+0000 7fe958ff9640 1 -- 172.21.15.86:0/2129325108 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]=0 set pool 4 allow_ec_overwrites to true v88) ==== 169+0+0 (secure 0 0 0) 0x7fe95409d300 con 0x7fe95c107c90 2024-04-21T08:36:24.897 INFO:teuthology.orchestra.run.smithi086.stderr:set pool 4 allow_ec_overwrites to true 2024-04-21T08:36:24.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.900+0000 7fe961f96640 1 -- 172.21.15.86:0/2129325108 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe9440a3bb0 msgr2=0x7fe9440a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:24.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.900+0000 7fe961f96640 1 --2- 172.21.15.86:0/2129325108 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe9440a3bb0 0x7fe9440a6070 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fe94c009230 tx=0x7fe94c005fd0 comp rx=0 tx=0).stop 2024-04-21T08:36:24.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.900+0000 7fe961f96640 1 -- 172.21.15.86:0/2129325108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe95c107c90 msgr2=0x7fe95c19f7b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:24.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.900+0000 7fe961f96640 1 --2- 172.21.15.86:0/2129325108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe95c107c90 0x7fe95c19f7b0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7fe95400eaa0 tx=0x7fe95400ef60 comp rx=0 tx=0).stop 2024-04-21T08:36:24.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.900+0000 7fe961f96640 1 -- 172.21.15.86:0/2129325108 shutdown_connections 2024-04-21T08:36:24.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.900+0000 7fe961f96640 1 --2- 172.21.15.86:0/2129325108 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe9440a3bb0 0x7fe9440a6070 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:24.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.900+0000 7fe961f96640 1 --2- 172.21.15.86:0/2129325108 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe95c107c90 0x7fe95c19f7b0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:24.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.900+0000 7fe961f96640 1 --2- 172.21.15.86:0/2129325108 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe95c105360 0x7fe95c198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:24.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.900+0000 7fe961f96640 1 --2- 172.21.15.86:0/2129325108 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe95c102a30 0x7fe95c198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:24.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.900+0000 7fe961f96640 1 -- 172.21.15.86:0/2129325108 >> 172.21.15.86:0/2129325108 conn(0x7fe95c0fc440 msgr2=0x7fe95c103470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:24.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.900+0000 7fe961f96640 1 -- 172.21.15.86:0/2129325108 shutdown_connections 2024-04-21T08:36:24.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:24.900+0000 7fe961f96640 1 -- 172.21.15.86:0/2129325108 wait complete. 2024-04-21T08:36:24.921 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs add_data_pool cephfs cephfs_data_ec 2024-04-21T08:36:25.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.024+0000 7f439df96640 1 -- 172.21.15.86:0/1419689145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4398100e90 msgr2=0x7f43981012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:25.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.024+0000 7f439df96640 1 --2- 172.21.15.86:0/1419689145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4398100e90 0x7f43981012b0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f4388009900 tx=0x7f438802bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:25.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.024+0000 7f439df96640 1 -- 172.21.15.86:0/1419689145 shutdown_connections 2024-04-21T08:36:25.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.024+0000 7f439df96640 1 --2- 172.21.15.86:0/1419689145 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43981029a0 0x7f43981092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:25.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.024+0000 7f439df96640 1 --2- 172.21.15.86:0/1419689145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4398101fe0 0x7f4398102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:25.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.024+0000 7f439df96640 1 --2- 172.21.15.86:0/1419689145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4398100e90 0x7f43981012b0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:25.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.024+0000 7f439df96640 1 -- 172.21.15.86:0/1419689145 >> 172.21.15.86:0/1419689145 conn(0x7f43980fc440 msgr2=0x7f43980fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:25.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.024+0000 7f439df96640 1 -- 172.21.15.86:0/1419689145 shutdown_connections 2024-04-21T08:36:25.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.024+0000 7f439df96640 1 -- 172.21.15.86:0/1419689145 wait complete. 2024-04-21T08:36:25.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.024+0000 7f439df96640 1 Processor -- start 2024-04-21T08:36:25.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.025+0000 7f439df96640 1 -- start start 2024-04-21T08:36:25.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.025+0000 7f439df96640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4398100e90 0x7f439806b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:25.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.025+0000 7f439df96640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4398101fe0 0x7f439819cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:25.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.025+0000 7f439df96640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43981029a0 0x7f439819f5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:25.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.025+0000 7f439df96640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f439810c9e0 con 0x7f4398100e90 2024-04-21T08:36:25.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.025+0000 7f439df96640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f439810c860 con 0x7f4398101fe0 2024-04-21T08:36:25.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.025+0000 7f439df96640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f439810cb60 con 0x7f43981029a0 2024-04-21T08:36:25.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.025+0000 7f43977fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4398100e90 0x7f439806b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:25.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.025+0000 7f43977fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4398100e90 0x7f439806b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35960/0 (socket says 172.21.15.86:35960) 2024-04-21T08:36:25.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.025+0000 7f43977fe640 1 -- 172.21.15.86:0/1149502879 learned_addr learned my addr 172.21.15.86:0/1149502879 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:25.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.025+0000 7f438ffff640 1 --2- 172.21.15.86:0/1149502879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4398101fe0 0x7f439819cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:25.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.025+0000 7f4397fff640 1 --2- 172.21.15.86:0/1149502879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43981029a0 0x7f439819f5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:25.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.025+0000 7f43977fe640 1 -- 172.21.15.86:0/1149502879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43981029a0 msgr2=0x7f439819f5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:25.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.025+0000 7f43977fe640 1 --2- 172.21.15.86:0/1149502879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43981029a0 0x7f439819f5b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:25.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.025+0000 7f43977fe640 1 -- 172.21.15.86:0/1149502879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4398101fe0 msgr2=0x7f439819cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:25.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.025+0000 7f43977fe640 1 --2- 172.21.15.86:0/1149502879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4398101fe0 0x7f439819cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:25.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.025+0000 7f43977fe640 1 -- 172.21.15.86:0/1149502879 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f439819fe70 con 0x7f4398100e90 2024-04-21T08:36:25.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.025+0000 7f4397fff640 1 --2- 172.21.15.86:0/1149502879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43981029a0 0x7f439819f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:36:25.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.025+0000 7f43977fe640 1 --2- 172.21.15.86:0/1149502879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4398100e90 0x7f439806b120 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4388036040 tx=0x7f43880042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:25.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.025+0000 7f43957fa640 1 -- 172.21.15.86:0/1149502879 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4388043020 con 0x7f4398100e90 2024-04-21T08:36:25.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.025+0000 7f439df96640 1 -- 172.21.15.86:0/1149502879 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43981a0100 con 0x7f4398100e90 2024-04-21T08:36:25.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.025+0000 7f43957fa640 1 -- 172.21.15.86:0/1149502879 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43880054c0 con 0x7f4398100e90 2024-04-21T08:36:25.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.025+0000 7f43957fa640 1 -- 172.21.15.86:0/1149502879 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4388003db0 con 0x7f4398100e90 2024-04-21T08:36:25.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.025+0000 7f439df96640 1 -- 172.21.15.86:0/1149502879 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43981a06a0 con 0x7f4398100e90 2024-04-21T08:36:25.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.026+0000 7f439df96640 1 -- 172.21.15.86:0/1149502879 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4398108290 con 0x7f4398100e90 2024-04-21T08:36:25.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.027+0000 7f43957fa640 1 -- 172.21.15.86:0/1149502879 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4388046020 con 0x7f4398100e90 2024-04-21T08:36:25.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.028+0000 7f43957fa640 1 --2- 172.21.15.86:0/1149502879 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f43840a3bb0 0x7f43840a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:25.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.028+0000 7f438ffff640 1 --2- 172.21.15.86:0/1149502879 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f43840a3bb0 0x7f43840a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:25.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.028+0000 7f43957fa640 1 -- 172.21.15.86:0/1149502879 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(88..88 src has 1..88) ==== 8770+0+0 (secure 0 0 0) 0x7f43880f2bc0 con 0x7f4398100e90 2024-04-21T08:36:25.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.028+0000 7f438ffff640 1 --2- 172.21.15.86:0/1149502879 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f43840a3bb0 0x7f43840a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f439806ae90 tx=0x7f438000a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:25.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.032+0000 7f43957fa640 1 -- 172.21.15.86:0/1149502879 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f438804d020 con 0x7f4398100e90 2024-04-21T08:36:25.213 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:24 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2129325108' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-21T08:36:25.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:24 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2129325108' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-21T08:36:25.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:24 smithi088 ceph-mon[31259]: osdmap e88: 12 total, 12 up, 12 in 2024-04-21T08:36:25.214 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 21 08:36:24 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-b[61651]: 2024-04-21T08:36:24.985+0000 7fa7417e1700 -1 mds.pinger is_rank_lagging: rank=0 was never sent ping request. 2024-04-21T08:36:25.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:24 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2129325108' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-21T08:36:25.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:24 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2129325108' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-21T08:36:25.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:24 smithi179 ceph-mon[31829]: osdmap e88: 12 total, 12 up, 12 in 2024-04-21T08:36:25.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.250+0000 7f439df96640 1 -- 172.21.15.86:0/1149502879 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"} v 0) -- 0x7f4398062b50 con 0x7f4398100e90 2024-04-21T08:36:25.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:24 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2129325108' entity='client.admin' cmd=[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]: dispatch 2024-04-21T08:36:25.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:24 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2129325108' entity='client.admin' cmd='[{"prefix": "osd pool set", "pool": "cephfs_data_ec", "var": "allow_ec_overwrites", "val": "true"}]': finished 2024-04-21T08:36:25.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:24 smithi086 ceph-mon[27939]: osdmap e88: 12 total, 12 up, 12 in 2024-04-21T08:36:25.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.925+0000 7f43957fa640 1 -- 172.21.15.86:0/1149502879 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]=0 added data pool 4 to fsmap v17) ==== 137+0+0 (secure 0 0 0) 0x7f43880bf380 con 0x7f4398100e90 2024-04-21T08:36:25.924 INFO:teuthology.orchestra.run.smithi086.stderr:added data pool 4 to fsmap 2024-04-21T08:36:25.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.927+0000 7f439df96640 1 -- 172.21.15.86:0/1149502879 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f43840a3bb0 msgr2=0x7f43840a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:25.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.927+0000 7f439df96640 1 --2- 172.21.15.86:0/1149502879 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f43840a3bb0 0x7f43840a6070 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7f439806ae90 tx=0x7f438000a4c0 comp rx=0 tx=0).stop 2024-04-21T08:36:25.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.927+0000 7f439df96640 1 -- 172.21.15.86:0/1149502879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4398100e90 msgr2=0x7f439806b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:25.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.927+0000 7f439df96640 1 --2- 172.21.15.86:0/1149502879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4398100e90 0x7f439806b120 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f4388036040 tx=0x7f43880042f0 comp rx=0 tx=0).stop 2024-04-21T08:36:25.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.927+0000 7f439df96640 1 -- 172.21.15.86:0/1149502879 shutdown_connections 2024-04-21T08:36:25.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.927+0000 7f439df96640 1 --2- 172.21.15.86:0/1149502879 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f43840a3bb0 0x7f43840a6070 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:25.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.927+0000 7f439df96640 1 --2- 172.21.15.86:0/1149502879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43981029a0 0x7f439819f5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:25.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.927+0000 7f439df96640 1 --2- 172.21.15.86:0/1149502879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4398101fe0 0x7f439819cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:25.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.927+0000 7f439df96640 1 --2- 172.21.15.86:0/1149502879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4398100e90 0x7f439806b120 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:25.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.927+0000 7f439df96640 1 -- 172.21.15.86:0/1149502879 >> 172.21.15.86:0/1149502879 conn(0x7f43980fc440 msgr2=0x7f4398105460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:25.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.927+0000 7f439df96640 1 -- 172.21.15.86:0/1149502879 shutdown_connections 2024-04-21T08:36:25.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:25.927+0000 7f439df96640 1 -- 172.21.15.86:0/1149502879 wait complete. 2024-04-21T08:36:25.938 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:36:26.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.039+0000 7f26b5f2e640 1 -- 172.21.15.86:0/59987740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26b0101fd0 msgr2=0x7f26b0102450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:26.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.039+0000 7f26b5f2e640 1 --2- 172.21.15.86:0/59987740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26b0101fd0 0x7f26b0102450 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f26a4009900 tx=0x7f26a402bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:26.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.039+0000 7f26b5f2e640 1 -- 172.21.15.86:0/59987740 shutdown_connections 2024-04-21T08:36:26.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.039+0000 7f26b5f2e640 1 --2- 172.21.15.86:0/59987740 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26b0102990 0x7f26b01092b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.039+0000 7f26b5f2e640 1 --2- 172.21.15.86:0/59987740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26b0101fd0 0x7f26b0102450 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.039+0000 7f26b5f2e640 1 --2- 172.21.15.86:0/59987740 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26b0100e80 0x7f26b01012a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.039+0000 7f26b5f2e640 1 -- 172.21.15.86:0/59987740 >> 172.21.15.86:0/59987740 conn(0x7f26b00fc410 msgr2=0x7f26b00fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:26.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.039+0000 7f26b5f2e640 1 -- 172.21.15.86:0/59987740 shutdown_connections 2024-04-21T08:36:26.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.039+0000 7f26b5f2e640 1 -- 172.21.15.86:0/59987740 wait complete. 2024-04-21T08:36:26.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.039+0000 7f26b5f2e640 1 Processor -- start 2024-04-21T08:36:26.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.040+0000 7f26b5f2e640 1 -- start start 2024-04-21T08:36:26.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.040+0000 7f26b5f2e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26b0100e80 0x7f26b0198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:26.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.040+0000 7f26b5f2e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26b0101fd0 0x7f26b0198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:26.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.040+0000 7f26b5f2e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26b0102990 0x7f26b019f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:26.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.040+0000 7f26b5f2e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f26b010c9c0 con 0x7f26b0101fd0 2024-04-21T08:36:26.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.040+0000 7f26b5f2e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f26b010c840 con 0x7f26b0102990 2024-04-21T08:36:26.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.040+0000 7f26b5f2e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f26b010cb40 con 0x7f26b0100e80 2024-04-21T08:36:26.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.040+0000 7f26aeffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26b0101fd0 0x7f26b0198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:26.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.040+0000 7f26aeffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26b0101fd0 0x7f26b0198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35982/0 (socket says 172.21.15.86:35982) 2024-04-21T08:36:26.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.040+0000 7f26aeffd640 1 -- 172.21.15.86:0/68670884 learned_addr learned my addr 172.21.15.86:0/68670884 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:26.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.040+0000 7f26af7fe640 1 --2- 172.21.15.86:0/68670884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26b0100e80 0x7f26b0198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:26.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.040+0000 7f26aeffd640 1 -- 172.21.15.86:0/68670884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26b0100e80 msgr2=0x7f26b0198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:26.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.040+0000 7f26aeffd640 1 --2- 172.21.15.86:0/68670884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26b0100e80 0x7f26b0198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.040+0000 7f26aeffd640 1 -- 172.21.15.86:0/68670884 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26b0102990 msgr2=0x7f26b019f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:26.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.040+0000 7f26aeffd640 1 --2- 172.21.15.86:0/68670884 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26b0102990 0x7f26b019f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.040+0000 7f26aeffd640 1 -- 172.21.15.86:0/68670884 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f26b019fd70 con 0x7f26b0101fd0 2024-04-21T08:36:26.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.040+0000 7f26aeffd640 1 --2- 172.21.15.86:0/68670884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26b0101fd0 0x7f26b0198690 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f26a4000c00 tx=0x7f26a4004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:26.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.040+0000 7f26acff9640 1 -- 172.21.15.86:0/68670884 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a4043020 con 0x7f26b0101fd0 2024-04-21T08:36:26.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.040+0000 7f26b5f2e640 1 -- 172.21.15.86:0/68670884 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26b01a0000 con 0x7f26b0101fd0 2024-04-21T08:36:26.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.040+0000 7f26acff9640 1 -- 172.21.15.86:0/68670884 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f26a4004aa0 con 0x7f26b0101fd0 2024-04-21T08:36:26.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.040+0000 7f26acff9640 1 -- 172.21.15.86:0/68670884 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f26a4003bf0 con 0x7f26b0101fd0 2024-04-21T08:36:26.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.040+0000 7f26b5f2e640 1 -- 172.21.15.86:0/68670884 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26b01a05a0 con 0x7f26b0101fd0 2024-04-21T08:36:26.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.041+0000 7f26b5f2e640 1 -- 172.21.15.86:0/68670884 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26b01a07a0 con 0x7f26b0101fd0 2024-04-21T08:36:26.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.042+0000 7f26acff9640 1 -- 172.21.15.86:0/68670884 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f26a4046020 con 0x7f26b0101fd0 2024-04-21T08:36:26.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.043+0000 7f26acff9640 1 --2- 172.21.15.86:0/68670884 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f26980a3bb0 0x7f26980a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:26.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.043+0000 7f26af7fe640 1 --2- 172.21.15.86:0/68670884 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f26980a3bb0 0x7f26980a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:26.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.043+0000 7f26acff9640 1 -- 172.21.15.86:0/68670884 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f26a40f2f90 con 0x7f26b0101fd0 2024-04-21T08:36:26.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.043+0000 7f26af7fe640 1 --2- 172.21.15.86:0/68670884 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f26980a3bb0 0x7f26980a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f26a000abe0 tx=0x7f26a000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:26.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.047+0000 7f26acff9640 1 -- 172.21.15.86:0/68670884 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f26a404d020 con 0x7f26b0101fd0 2024-04-21T08:36:26.213 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:25 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1149502879' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-21T08:36:26.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:25 smithi088 ceph-mon[31259]: pgmap v289: 129 pgs: 18 creating+peering, 90 active+clean, 21 unknown; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 11 op/s 2024-04-21T08:36:26.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:25 smithi088 ceph-mon[31259]: osdmap e89: 12 total, 12 up, 12 in 2024-04-21T08:36:26.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:25 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1149502879' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-21T08:36:26.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:25 smithi179 ceph-mon[31829]: pgmap v289: 129 pgs: 18 creating+peering, 90 active+clean, 21 unknown; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 11 op/s 2024-04-21T08:36:26.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:25 smithi179 ceph-mon[31829]: osdmap e89: 12 total, 12 up, 12 in 2024-04-21T08:36:26.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.265+0000 7f26b5f2e640 1 -- 172.21.15.86:0/68670884 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f26b01a0970 con 0x7f26b0101fd0 2024-04-21T08:36:26.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.266+0000 7f26acff9640 1 -- 172.21.15.86:0/68670884 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11800 (secure 0 0 0) 0x7f26a40bf730 con 0x7f26b0101fd0 2024-04-21T08:36:26.265 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:36:26.265 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:36:25.251833+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-04-21T08:36:26.266 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 17 2024-04-21T08:36:26.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.268+0000 7f26b5f2e640 1 -- 172.21.15.86:0/68670884 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f26980a3bb0 msgr2=0x7f26980a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:26.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.268+0000 7f26b5f2e640 1 --2- 172.21.15.86:0/68670884 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f26980a3bb0 0x7f26980a6070 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f26a000abe0 tx=0x7f26a000a2d0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.268+0000 7f26b5f2e640 1 -- 172.21.15.86:0/68670884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26b0101fd0 msgr2=0x7f26b0198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:26.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.268+0000 7f26b5f2e640 1 --2- 172.21.15.86:0/68670884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26b0101fd0 0x7f26b0198690 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f26a4000c00 tx=0x7f26a4004270 comp rx=0 tx=0).stop 2024-04-21T08:36:26.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.268+0000 7f26b5f2e640 1 -- 172.21.15.86:0/68670884 shutdown_connections 2024-04-21T08:36:26.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.268+0000 7f26b5f2e640 1 --2- 172.21.15.86:0/68670884 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f26980a3bb0 0x7f26980a6070 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.268+0000 7f26b5f2e640 1 --2- 172.21.15.86:0/68670884 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26b0102990 0x7f26b019f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.268+0000 7f26b5f2e640 1 --2- 172.21.15.86:0/68670884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f26b0101fd0 0x7f26b0198690 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.268+0000 7f26b5f2e640 1 --2- 172.21.15.86:0/68670884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f26b0100e80 0x7f26b0198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.268+0000 7f26b5f2e640 1 -- 172.21.15.86:0/68670884 >> 172.21.15.86:0/68670884 conn(0x7f26b00fc410 msgr2=0x7f26b00fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:26.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.268+0000 7f26b5f2e640 1 -- 172.21.15.86:0/68670884 shutdown_connections 2024-04-21T08:36:26.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.268+0000 7f26b5f2e640 1 -- 172.21.15.86:0/68670884 wait complete. 2024-04-21T08:36:26.277 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:25 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1149502879' entity='client.admin' cmd=[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]: dispatch 2024-04-21T08:36:26.277 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:25 smithi086 ceph-mon[27939]: pgmap v289: 129 pgs: 18 creating+peering, 90 active+clean, 21 unknown; 579 KiB data, 361 MiB used, 1.0 TiB / 1.0 TiB avail; 3.5 KiB/s wr, 11 op/s 2024-04-21T08:36:26.277 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:25 smithi086 ceph-mon[27939]: osdmap e89: 12 total, 12 up, 12 in 2024-04-21T08:36:26.278 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:36:26.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.378+0000 7f144ca6a640 1 -- 172.21.15.86:0/4034926488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1448100620 msgr2=0x7f1448100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:26.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.378+0000 7f144ca6a640 1 --2- 172.21.15.86:0/4034926488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1448100620 0x7f1448100a40 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f1438009ca0 tx=0x7f143802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.378+0000 7f144ca6a640 1 -- 172.21.15.86:0/4034926488 shutdown_connections 2024-04-21T08:36:26.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.378+0000 7f144ca6a640 1 --2- 172.21.15.86:0/4034926488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14481021d0 0x7f1448108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.378+0000 7f144ca6a640 1 --2- 172.21.15.86:0/4034926488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1448101810 0x7f1448101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.378+0000 7f144ca6a640 1 --2- 172.21.15.86:0/4034926488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1448100620 0x7f1448100a40 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.378+0000 7f144ca6a640 1 -- 172.21.15.86:0/4034926488 >> 172.21.15.86:0/4034926488 conn(0x7f14480fbb90 msgr2=0x7f14480fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:26.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.378+0000 7f144ca6a640 1 -- 172.21.15.86:0/4034926488 shutdown_connections 2024-04-21T08:36:26.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.378+0000 7f144ca6a640 1 -- 172.21.15.86:0/4034926488 wait complete. 2024-04-21T08:36:26.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.379+0000 7f144ca6a640 1 Processor -- start 2024-04-21T08:36:26.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.379+0000 7f144ca6a640 1 -- start start 2024-04-21T08:36:26.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.379+0000 7f144ca6a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1448100620 0x7f14481970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:26.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.379+0000 7f144ca6a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1448101810 0x7f1448197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:26.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.379+0000 7f144ca6a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14481021d0 0x7f144819e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:26.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.379+0000 7f144ca6a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f144810ca20 con 0x7f14481021d0 2024-04-21T08:36:26.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.379+0000 7f144ca6a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f144810c8a0 con 0x7f1448100620 2024-04-21T08:36:26.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.379+0000 7f144ca6a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f144810cba0 con 0x7f1448101810 2024-04-21T08:36:26.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.379+0000 7f1446d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14481021d0 0x7f144819e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:26.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.379+0000 7f1446d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14481021d0 0x7f144819e6a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36006/0 (socket says 172.21.15.86:36006) 2024-04-21T08:36:26.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.379+0000 7f1446d9d640 1 -- 172.21.15.86:0/1475010588 learned_addr learned my addr 172.21.15.86:0/1475010588 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:26.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.379+0000 7f1445d9b640 1 --2- 172.21.15.86:0/1475010588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1448101810 0x7f1448197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:26.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.379+0000 7f144659c640 1 --2- 172.21.15.86:0/1475010588 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1448100620 0x7f14481970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:26.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.379+0000 7f1446d9d640 1 -- 172.21.15.86:0/1475010588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1448101810 msgr2=0x7f1448197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:26.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.379+0000 7f1446d9d640 1 --2- 172.21.15.86:0/1475010588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1448101810 0x7f1448197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.379+0000 7f1446d9d640 1 -- 172.21.15.86:0/1475010588 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1448100620 msgr2=0x7f14481970f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:26.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.379+0000 7f1446d9d640 1 --2- 172.21.15.86:0/1475010588 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1448100620 0x7f14481970f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.379+0000 7f1446d9d640 1 -- 172.21.15.86:0/1475010588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f144819ed10 con 0x7f14481021d0 2024-04-21T08:36:26.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.379+0000 7f144659c640 1 --2- 172.21.15.86:0/1475010588 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1448100620 0x7f14481970f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:36:26.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.379+0000 7f1446d9d640 1 --2- 172.21.15.86:0/1475010588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14481021d0 0x7f144819e6a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1440009fd0 tx=0x7f144000b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:26.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.380+0000 7f14377fe640 1 -- 172.21.15.86:0/1475010588 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1440011830 con 0x7f14481021d0 2024-04-21T08:36:26.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.380+0000 7f14377fe640 1 -- 172.21.15.86:0/1475010588 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1440016070 con 0x7f14481021d0 2024-04-21T08:36:26.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.380+0000 7f144ca6a640 1 -- 172.21.15.86:0/1475010588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f144819efa0 con 0x7f14481021d0 2024-04-21T08:36:26.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.380+0000 7f14377fe640 1 -- 172.21.15.86:0/1475010588 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1440010de0 con 0x7f14481021d0 2024-04-21T08:36:26.382 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.380+0000 7f144ca6a640 1 -- 172.21.15.86:0/1475010588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f144819f4c0 con 0x7f14481021d0 2024-04-21T08:36:26.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.381+0000 7f144ca6a640 1 -- 172.21.15.86:0/1475010588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1448107940 con 0x7f14481021d0 2024-04-21T08:36:26.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.382+0000 7f14377fe640 1 -- 172.21.15.86:0/1475010588 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f14400119d0 con 0x7f14481021d0 2024-04-21T08:36:26.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.382+0000 7f14377fe640 1 --2- 172.21.15.86:0/1475010588 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f14300a3c30 0x7f14300a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:26.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.383+0000 7f144659c640 1 --2- 172.21.15.86:0/1475010588 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f14300a3c30 0x7f14300a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:26.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.383+0000 7f14377fe640 1 -- 172.21.15.86:0/1475010588 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f14400cfcb0 con 0x7f14481021d0 2024-04-21T08:36:26.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.383+0000 7f144659c640 1 --2- 172.21.15.86:0/1475010588 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f14300a3c30 0x7f14300a60f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f143802c700 tx=0x7f14380324c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:26.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.387+0000 7f14377fe640 1 -- 172.21.15.86:0/1475010588 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f144009c450 con 0x7f14481021d0 2024-04-21T08:36:26.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.565+0000 7f144ca6a640 1 -- 172.21.15.86:0/1475010588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1448062b50 con 0x7f14481021d0 2024-04-21T08:36:26.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.566+0000 7f14377fe640 1 -- 172.21.15.86:0/1475010588 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24070 (secure 0 0 0) 0x7f144009bd50 con 0x7f14481021d0 2024-04-21T08:36:26.566 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:36:26.566 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":89,"fsid":"f8cec7cc-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:22.687329+0000","modified":"2024-04-21T08:36:25.251657+0000","last_up_change":"2024-04-21T08:35:30.048315+0000","last_in_change":"2024-04-21T08:35:10.099496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:45.279109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:18.300782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:19.574636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:22.905632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89b7601f-0620-4db2-90f1-b45cea173738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6803","nonce":2571516897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6805","nonce":2571516897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6809","nonce":2571516897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6807","nonce":2571516897}]},"public_addr":"172.21.15.86:6803/2571516897","cluster_addr":"172.21.15.86:6805/2571516897","heartbeat_back_addr":"172.21.15.86:6809/2571516897","heartbeat_front_addr":"172.21.15.86:6807/2571516897","state":["exists","up"]},{"osd":1,"uuid":"4891031c-f98c-42bc-bb3b-a420855ecc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6801","nonce":74067182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6803","nonce":74067182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6807","nonce":74067182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6805","nonce":74067182}]},"public_addr":"172.21.15.88:6801/74067182","cluster_addr":"172.21.15.88:6803/74067182","heartbeat_back_addr":"172.21.15.88:6807/74067182","heartbeat_front_addr":"172.21.15.88:6805/74067182","state":["exists","up"]},{"osd":2,"uuid":"60d12aa4-9312-4fa2-8d0b-2b3f42caad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6801","nonce":4137659051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6803","nonce":4137659051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6807","nonce":4137659051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6805","nonce":4137659051}]},"public_addr":"172.21.15.179:6801/4137659051","cluster_addr":"172.21.15.179:6803/4137659051","heartbeat_back_addr":"172.21.15.179:6807/4137659051","heartbeat_front_addr":"172.21.15.179:6805/4137659051","state":["exists","up"]},{"osd":3,"uuid":"404c3158-8cb0-4c02-b48a-f9ee5ff7b784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6811","nonce":2309603148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6813","nonce":2309603148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6817","nonce":2309603148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6815","nonce":2309603148}]},"public_addr":"172.21.15.86:6811/2309603148","cluster_addr":"172.21.15.86:6813/2309603148","heartbeat_back_addr":"172.21.15.86:6817/2309603148","heartbeat_front_addr":"172.21.15.86:6815/2309603148","state":["exists","up"]},{"osd":4,"uuid":"458ff202-d781-4ec7-ac93-0b49f835b503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6809","nonce":1818714217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6811","nonce":1818714217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6815","nonce":1818714217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6813","nonce":1818714217}]},"public_addr":"172.21.15.88:6809/1818714217","cluster_addr":"172.21.15.88:6811/1818714217","heartbeat_back_addr":"172.21.15.88:6815/1818714217","heartbeat_front_addr":"172.21.15.88:6813/1818714217","state":["exists","up"]},{"osd":5,"uuid":"8123cc90-bd8b-4204-bf31-46b34e9f589d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6809","nonce":863840990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6811","nonce":863840990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6815","nonce":863840990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6813","nonce":863840990}]},"public_addr":"172.21.15.179:6809/863840990","cluster_addr":"172.21.15.179:6811/863840990","heartbeat_back_addr":"172.21.15.179:6815/863840990","heartbeat_front_addr":"172.21.15.179:6813/863840990","state":["exists","up"]},{"osd":6,"uuid":"aac9d7c7-659a-4110-903a-a871c086066f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6819","nonce":1224231118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6821","nonce":1224231118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6825","nonce":1224231118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6823","nonce":1224231118}]},"public_addr":"172.21.15.86:6819/1224231118","cluster_addr":"172.21.15.86:6821/1224231118","heartbeat_back_addr":"172.21.15.86:6825/1224231118","heartbeat_front_addr":"172.21.15.86:6823/1224231118","state":["exists","up"]},{"osd":7,"uuid":"df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6817","nonce":136629878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6819","nonce":136629878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6823","nonce":136629878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6821","nonce":136629878}]},"public_addr":"172.21.15.88:6817/136629878","cluster_addr":"172.21.15.88:6819/136629878","heartbeat_back_addr":"172.21.15.88:6823/136629878","heartbeat_front_addr":"172.21.15.88:6821/136629878","state":["exists","up"]},{"osd":8,"uuid":"63e59f4b-0426-46db-92f7-c55dae61b5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6817","nonce":1770418957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6819","nonce":1770418957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6823","nonce":1770418957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6821","nonce":1770418957}]},"public_addr":"172.21.15.179:6817/1770418957","cluster_addr":"172.21.15.179:6819/1770418957","heartbeat_back_addr":"172.21.15.179:6823/1770418957","heartbeat_front_addr":"172.21.15.179:6821/1770418957","state":["exists","up"]},{"osd":9,"uuid":"5841bb36-ed6b-45d8-b857-eeab139c44e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6827","nonce":1108500274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6829","nonce":1108500274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6833","nonce":1108500274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6831","nonce":1108500274}]},"public_addr":"172.21.15.86:6827/1108500274","cluster_addr":"172.21.15.86:6829/1108500274","heartbeat_back_addr":"172.21.15.86:6833/1108500274","heartbeat_front_addr":"172.21.15.86:6831/1108500274","state":["exists","up"]},{"osd":10,"uuid":"f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6825","nonce":1294275353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6827","nonce":1294275353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6831","nonce":1294275353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6829","nonce":1294275353}]},"public_addr":"172.21.15.88:6825/1294275353","cluster_addr":"172.21.15.88:6827/1294275353","heartbeat_back_addr":"172.21.15.88:6831/1294275353","heartbeat_front_addr":"172.21.15.88:6829/1294275353","state":["exists","up"]},{"osd":11,"uuid":"e0d4cc76-e7a6-4a6b-909c-95488ccd5a36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6825","nonce":4007070233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6827","nonce":4007070233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6831","nonce":4007070233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6829","nonce":4007070233}]},"public_addr":"172.21.15.179:6825/4007070233","cluster_addr":"172.21.15.179:6827/4007070233","heartbeat_back_addr":"172.21.15.179:6831/4007070233","heartbeat_front_addr":"172.21.15.179:6829/4007070233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:55.210605+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:18.723090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:42.547979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:06.421622+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:30.858315+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:55.455313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.623369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:46.185111+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.103627+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:37.297545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:02.637431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:28.199292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6800/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/1412393225":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6800/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/1687852404":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2020707404":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2568130933":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/3952607648":"2024-04-22T08:28:57.384403+0000","172.21.15.86:6801/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/2756903426":"2024-04-22T08:29:23.190001+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:36:26.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.568+0000 7f144ca6a640 1 -- 172.21.15.86:0/1475010588 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f14300a3c30 msgr2=0x7f14300a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:26.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.568+0000 7f144ca6a640 1 --2- 172.21.15.86:0/1475010588 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f14300a3c30 0x7f14300a60f0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f143802c700 tx=0x7f14380324c0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.568+0000 7f144ca6a640 1 -- 172.21.15.86:0/1475010588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14481021d0 msgr2=0x7f144819e6a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:26.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.568+0000 7f144ca6a640 1 --2- 172.21.15.86:0/1475010588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14481021d0 0x7f144819e6a0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f1440009fd0 tx=0x7f144000b8e0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.568+0000 7f144ca6a640 1 -- 172.21.15.86:0/1475010588 shutdown_connections 2024-04-21T08:36:26.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.568+0000 7f144ca6a640 1 --2- 172.21.15.86:0/1475010588 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f14300a3c30 0x7f14300a60f0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.568+0000 7f144ca6a640 1 --2- 172.21.15.86:0/1475010588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14481021d0 0x7f144819e6a0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.568+0000 7f144ca6a640 1 --2- 172.21.15.86:0/1475010588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1448101810 0x7f1448197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.568+0000 7f144ca6a640 1 --2- 172.21.15.86:0/1475010588 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1448100620 0x7f14481970f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.568+0000 7f144ca6a640 1 -- 172.21.15.86:0/1475010588 >> 172.21.15.86:0/1475010588 conn(0x7f14480fbb90 msgr2=0x7f1448104b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:26.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.569+0000 7f144ca6a640 1 -- 172.21.15.86:0/1475010588 shutdown_connections 2024-04-21T08:36:26.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.569+0000 7f144ca6a640 1 -- 172.21.15.86:0/1475010588 wait complete. 2024-04-21T08:36:26.579 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:36:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.677+0000 7ff01b2e0640 1 -- 172.21.15.86:0/2806732572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff014102a10 msgr2=0x7ff014109330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.677+0000 7ff01b2e0640 1 --2- 172.21.15.86:0/2806732572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff014102a10 0x7ff014109330 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7ff00c009ca0 tx=0x7ff00c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.677+0000 7ff01b2e0640 1 -- 172.21.15.86:0/2806732572 shutdown_connections 2024-04-21T08:36:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.677+0000 7ff01b2e0640 1 --2- 172.21.15.86:0/2806732572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff014102a10 0x7ff014109330 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.677+0000 7ff01b2e0640 1 --2- 172.21.15.86:0/2806732572 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff014102050 0x7ff0141024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.677+0000 7ff01b2e0640 1 --2- 172.21.15.86:0/2806732572 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff014100e70 0x7ff014101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.677+0000 7ff01b2e0640 1 -- 172.21.15.86:0/2806732572 >> 172.21.15.86:0/2806732572 conn(0x7ff0140fc470 msgr2=0x7ff0140fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.677+0000 7ff01b2e0640 1 -- 172.21.15.86:0/2806732572 shutdown_connections 2024-04-21T08:36:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.677+0000 7ff01b2e0640 1 -- 172.21.15.86:0/2806732572 wait complete. 2024-04-21T08:36:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.678+0000 7ff01b2e0640 1 Processor -- start 2024-04-21T08:36:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.678+0000 7ff01b2e0640 1 -- start start 2024-04-21T08:36:26.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.678+0000 7ff01b2e0640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff014100e70 0x7ff014198130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:26.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.678+0000 7ff01b2e0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff014102050 0x7ff014198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:26.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.678+0000 7ff01b2e0640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff014102a10 0x7ff01419f6e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:26.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.678+0000 7ff01b2e0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff01410ca10 con 0x7ff014102050 2024-04-21T08:36:26.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.678+0000 7ff01b2e0640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff01410c890 con 0x7ff014102a10 2024-04-21T08:36:26.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.678+0000 7ff01b2e0640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff01410cb90 con 0x7ff014100e70 2024-04-21T08:36:26.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.678+0000 7ff01887b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff014102050 0x7ff014198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:26.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.678+0000 7ff01887b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff014102050 0x7ff014198670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36020/0 (socket says 172.21.15.86:36020) 2024-04-21T08:36:26.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.678+0000 7ff01887b640 1 -- 172.21.15.86:0/245969189 learned_addr learned my addr 172.21.15.86:0/245969189 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:26.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.678+0000 7ff01987d640 1 --2- 172.21.15.86:0/245969189 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff014102a10 0x7ff01419f6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:26.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.678+0000 7ff01907c640 1 --2- 172.21.15.86:0/245969189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff014100e70 0x7ff014198130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:26.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.678+0000 7ff01887b640 1 -- 172.21.15.86:0/245969189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff014100e70 msgr2=0x7ff014198130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:26.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.678+0000 7ff01887b640 1 --2- 172.21.15.86:0/245969189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff014100e70 0x7ff014198130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.678+0000 7ff01887b640 1 -- 172.21.15.86:0/245969189 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff014102a10 msgr2=0x7ff01419f6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:26.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.678+0000 7ff01887b640 1 --2- 172.21.15.86:0/245969189 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff014102a10 0x7ff01419f6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.678+0000 7ff01887b640 1 -- 172.21.15.86:0/245969189 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff01419fd50 con 0x7ff014102050 2024-04-21T08:36:26.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.678+0000 7ff01987d640 1 --2- 172.21.15.86:0/245969189 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff014102a10 0x7ff01419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:36:26.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.678+0000 7ff01887b640 1 --2- 172.21.15.86:0/245969189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff014102050 0x7ff014198670 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff00400c970 tx=0x7ff00400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:26.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.678+0000 7ff00a7fc640 1 -- 172.21.15.86:0/245969189 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff004007bb0 con 0x7ff014102050 2024-04-21T08:36:26.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.679+0000 7ff01b2e0640 1 -- 172.21.15.86:0/245969189 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff01419ffe0 con 0x7ff014102050 2024-04-21T08:36:26.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.679+0000 7ff00a7fc640 1 -- 172.21.15.86:0/245969189 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff004007d50 con 0x7ff014102050 2024-04-21T08:36:26.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.679+0000 7ff00a7fc640 1 -- 172.21.15.86:0/245969189 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff00400f450 con 0x7ff014102050 2024-04-21T08:36:26.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.679+0000 7ff01b2e0640 1 -- 172.21.15.86:0/245969189 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0141a04b0 con 0x7ff014102050 2024-04-21T08:36:26.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.679+0000 7ff01b2e0640 1 -- 172.21.15.86:0/245969189 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0141a0740 con 0x7ff014102050 2024-04-21T08:36:26.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.681+0000 7ff00a7fc640 1 -- 172.21.15.86:0/245969189 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff004004f00 con 0x7ff014102050 2024-04-21T08:36:26.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.682+0000 7ff00a7fc640 1 --2- 172.21.15.86:0/245969189 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ff0000a3bb0 0x7ff0000a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:26.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.682+0000 7ff01907c640 1 --2- 172.21.15.86:0/245969189 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ff0000a3bb0 0x7ff0000a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:26.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.682+0000 7ff00a7fc640 1 -- 172.21.15.86:0/245969189 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7ff0040d0750 con 0x7ff014102050 2024-04-21T08:36:26.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.682+0000 7ff01907c640 1 --2- 172.21.15.86:0/245969189 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ff0000a3bb0 0x7ff0000a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff010005d00 tx=0x7ff010005c50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:26.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.685+0000 7ff00a7fc640 1 -- 172.21.15.86:0/245969189 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff004014030 con 0x7ff014102050 2024-04-21T08:36:26.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.862+0000 7ff01b2e0640 1 -- 172.21.15.86:0/245969189 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff0141a0a30 con 0x7ff014102050 2024-04-21T08:36:26.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.864+0000 7ff00a7fc640 1 -- 172.21.15.86:0/245969189 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24070 (secure 0 0 0) 0x7ff00409cef0 con 0x7ff014102050 2024-04-21T08:36:26.863 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:36:26.863 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":89,"fsid":"f8cec7cc-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:22.687329+0000","modified":"2024-04-21T08:36:25.251657+0000","last_up_change":"2024-04-21T08:35:30.048315+0000","last_in_change":"2024-04-21T08:35:10.099496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:45.279109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:18.300782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:19.574636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:22.905632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89b7601f-0620-4db2-90f1-b45cea173738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6803","nonce":2571516897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6805","nonce":2571516897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6809","nonce":2571516897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6807","nonce":2571516897}]},"public_addr":"172.21.15.86:6803/2571516897","cluster_addr":"172.21.15.86:6805/2571516897","heartbeat_back_addr":"172.21.15.86:6809/2571516897","heartbeat_front_addr":"172.21.15.86:6807/2571516897","state":["exists","up"]},{"osd":1,"uuid":"4891031c-f98c-42bc-bb3b-a420855ecc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6801","nonce":74067182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6803","nonce":74067182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6807","nonce":74067182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6805","nonce":74067182}]},"public_addr":"172.21.15.88:6801/74067182","cluster_addr":"172.21.15.88:6803/74067182","heartbeat_back_addr":"172.21.15.88:6807/74067182","heartbeat_front_addr":"172.21.15.88:6805/74067182","state":["exists","up"]},{"osd":2,"uuid":"60d12aa4-9312-4fa2-8d0b-2b3f42caad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6801","nonce":4137659051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6803","nonce":4137659051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6807","nonce":4137659051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6805","nonce":4137659051}]},"public_addr":"172.21.15.179:6801/4137659051","cluster_addr":"172.21.15.179:6803/4137659051","heartbeat_back_addr":"172.21.15.179:6807/4137659051","heartbeat_front_addr":"172.21.15.179:6805/4137659051","state":["exists","up"]},{"osd":3,"uuid":"404c3158-8cb0-4c02-b48a-f9ee5ff7b784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6811","nonce":2309603148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6813","nonce":2309603148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6817","nonce":2309603148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6815","nonce":2309603148}]},"public_addr":"172.21.15.86:6811/2309603148","cluster_addr":"172.21.15.86:6813/2309603148","heartbeat_back_addr":"172.21.15.86:6817/2309603148","heartbeat_front_addr":"172.21.15.86:6815/2309603148","state":["exists","up"]},{"osd":4,"uuid":"458ff202-d781-4ec7-ac93-0b49f835b503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6809","nonce":1818714217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6811","nonce":1818714217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6815","nonce":1818714217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6813","nonce":1818714217}]},"public_addr":"172.21.15.88:6809/1818714217","cluster_addr":"172.21.15.88:6811/1818714217","heartbeat_back_addr":"172.21.15.88:6815/1818714217","heartbeat_front_addr":"172.21.15.88:6813/1818714217","state":["exists","up"]},{"osd":5,"uuid":"8123cc90-bd8b-4204-bf31-46b34e9f589d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6809","nonce":863840990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6811","nonce":863840990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6815","nonce":863840990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6813","nonce":863840990}]},"public_addr":"172.21.15.179:6809/863840990","cluster_addr":"172.21.15.179:6811/863840990","heartbeat_back_addr":"172.21.15.179:6815/863840990","heartbeat_front_addr":"172.21.15.179:6813/863840990","state":["exists","up"]},{"osd":6,"uuid":"aac9d7c7-659a-4110-903a-a871c086066f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6819","nonce":1224231118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6821","nonce":1224231118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6825","nonce":1224231118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6823","nonce":1224231118}]},"public_addr":"172.21.15.86:6819/1224231118","cluster_addr":"172.21.15.86:6821/1224231118","heartbeat_back_addr":"172.21.15.86:6825/1224231118","heartbeat_front_addr":"172.21.15.86:6823/1224231118","state":["exists","up"]},{"osd":7,"uuid":"df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6817","nonce":136629878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6819","nonce":136629878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6823","nonce":136629878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6821","nonce":136629878}]},"public_addr":"172.21.15.88:6817/136629878","cluster_addr":"172.21.15.88:6819/136629878","heartbeat_back_addr":"172.21.15.88:6823/136629878","heartbeat_front_addr":"172.21.15.88:6821/136629878","state":["exists","up"]},{"osd":8,"uuid":"63e59f4b-0426-46db-92f7-c55dae61b5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6817","nonce":1770418957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6819","nonce":1770418957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6823","nonce":1770418957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6821","nonce":1770418957}]},"public_addr":"172.21.15.179:6817/1770418957","cluster_addr":"172.21.15.179:6819/1770418957","heartbeat_back_addr":"172.21.15.179:6823/1770418957","heartbeat_front_addr":"172.21.15.179:6821/1770418957","state":["exists","up"]},{"osd":9,"uuid":"5841bb36-ed6b-45d8-b857-eeab139c44e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6827","nonce":1108500274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6829","nonce":1108500274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6833","nonce":1108500274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6831","nonce":1108500274}]},"public_addr":"172.21.15.86:6827/1108500274","cluster_addr":"172.21.15.86:6829/1108500274","heartbeat_back_addr":"172.21.15.86:6833/1108500274","heartbeat_front_addr":"172.21.15.86:6831/1108500274","state":["exists","up"]},{"osd":10,"uuid":"f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6825","nonce":1294275353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6827","nonce":1294275353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6831","nonce":1294275353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6829","nonce":1294275353}]},"public_addr":"172.21.15.88:6825/1294275353","cluster_addr":"172.21.15.88:6827/1294275353","heartbeat_back_addr":"172.21.15.88:6831/1294275353","heartbeat_front_addr":"172.21.15.88:6829/1294275353","state":["exists","up"]},{"osd":11,"uuid":"e0d4cc76-e7a6-4a6b-909c-95488ccd5a36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6825","nonce":4007070233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6827","nonce":4007070233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6831","nonce":4007070233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6829","nonce":4007070233}]},"public_addr":"172.21.15.179:6825/4007070233","cluster_addr":"172.21.15.179:6827/4007070233","heartbeat_back_addr":"172.21.15.179:6831/4007070233","heartbeat_front_addr":"172.21.15.179:6829/4007070233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:55.210605+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:18.723090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:42.547979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:06.421622+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:30.858315+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:55.455313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.623369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:46.185111+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.103627+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:37.297545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:02.637431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:28.199292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6800/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/1412393225":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6800/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/1687852404":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2020707404":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2568130933":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/3952607648":"2024-04-22T08:28:57.384403+0000","172.21.15.86:6801/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/2756903426":"2024-04-22T08:29:23.190001+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:36:26.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.866+0000 7ff01b2e0640 1 -- 172.21.15.86:0/245969189 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ff0000a3bb0 msgr2=0x7ff0000a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:26.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.866+0000 7ff01b2e0640 1 --2- 172.21.15.86:0/245969189 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ff0000a3bb0 0x7ff0000a6070 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7ff010005d00 tx=0x7ff010005c50 comp rx=0 tx=0).stop 2024-04-21T08:36:26.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.866+0000 7ff01b2e0640 1 -- 172.21.15.86:0/245969189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff014102050 msgr2=0x7ff014198670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:26.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.866+0000 7ff01b2e0640 1 --2- 172.21.15.86:0/245969189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff014102050 0x7ff014198670 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7ff00400c970 tx=0x7ff00400ce30 comp rx=0 tx=0).stop 2024-04-21T08:36:26.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.866+0000 7ff01b2e0640 1 -- 172.21.15.86:0/245969189 shutdown_connections 2024-04-21T08:36:26.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.866+0000 7ff01b2e0640 1 --2- 172.21.15.86:0/245969189 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ff0000a3bb0 0x7ff0000a6070 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.866+0000 7ff01b2e0640 1 --2- 172.21.15.86:0/245969189 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff014102a10 0x7ff01419f6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.866+0000 7ff01b2e0640 1 --2- 172.21.15.86:0/245969189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff014102050 0x7ff014198670 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.866+0000 7ff01b2e0640 1 --2- 172.21.15.86:0/245969189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff014100e70 0x7ff014198130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.866+0000 7ff01b2e0640 1 -- 172.21.15.86:0/245969189 >> 172.21.15.86:0/245969189 conn(0x7ff0140fc470 msgr2=0x7ff0140fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:26.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.866+0000 7ff01b2e0640 1 -- 172.21.15.86:0/245969189 shutdown_connections 2024-04-21T08:36:26.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.866+0000 7ff01b2e0640 1 -- 172.21.15.86:0/245969189 wait complete. 2024-04-21T08:36:26.895 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:36:26.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.994+0000 7fb79050e640 1 -- 172.21.15.86:0/2225053747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb788102980 msgr2=0x7fb7881092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:26.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.994+0000 7fb79050e640 1 --2- 172.21.15.86:0/2225053747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb788102980 0x7fb7881092a0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7fb780009ca0 tx=0x7fb78002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.994+0000 7fb79050e640 1 -- 172.21.15.86:0/2225053747 shutdown_connections 2024-04-21T08:36:26.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.994+0000 7fb79050e640 1 --2- 172.21.15.86:0/2225053747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb788102980 0x7fb7881092a0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.994+0000 7fb79050e640 1 --2- 172.21.15.86:0/2225053747 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb788101fc0 0x7fb788102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.994+0000 7fb79050e640 1 --2- 172.21.15.86:0/2225053747 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb788100e70 0x7fb788101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.994+0000 7fb79050e640 1 -- 172.21.15.86:0/2225053747 >> 172.21.15.86:0/2225053747 conn(0x7fb7880fc3e0 msgr2=0x7fb7880fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:26.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.994+0000 7fb79050e640 1 -- 172.21.15.86:0/2225053747 shutdown_connections 2024-04-21T08:36:26.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.994+0000 7fb79050e640 1 -- 172.21.15.86:0/2225053747 wait complete. 2024-04-21T08:36:26.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.994+0000 7fb79050e640 1 Processor -- start 2024-04-21T08:36:26.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.994+0000 7fb79050e640 1 -- start start 2024-04-21T08:36:26.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.994+0000 7fb79050e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb788100e70 0x7fb788069c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:26.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.994+0000 7fb79050e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb788101fc0 0x7fb78819ce50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:26.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.994+0000 7fb79050e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb788102980 0x7fb78819f490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:26.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.994+0000 7fb79050e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb78810c9d0 con 0x7fb788102980 2024-04-21T08:36:26.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.994+0000 7fb79050e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb78810c850 con 0x7fb788101fc0 2024-04-21T08:36:26.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.994+0000 7fb79050e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb78810cb50 con 0x7fb788100e70 2024-04-21T08:36:26.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.995+0000 7fb78eaab640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb788102980 0x7fb78819f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:26.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.995+0000 7fb78eaab640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb788102980 0x7fb78819f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36040/0 (socket says 172.21.15.86:36040) 2024-04-21T08:36:26.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.995+0000 7fb78eaab640 1 -- 172.21.15.86:0/1915534602 learned_addr learned my addr 172.21.15.86:0/1915534602 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:26.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.995+0000 7fb78daa9640 1 --2- 172.21.15.86:0/1915534602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb788101fc0 0x7fb78819ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:26.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.995+0000 7fb78e2aa640 1 --2- 172.21.15.86:0/1915534602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb788100e70 0x7fb788069c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:26.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.995+0000 7fb78eaab640 1 -- 172.21.15.86:0/1915534602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb788100e70 msgr2=0x7fb788069c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:26.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.995+0000 7fb78eaab640 1 --2- 172.21.15.86:0/1915534602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb788100e70 0x7fb788069c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.995+0000 7fb78eaab640 1 -- 172.21.15.86:0/1915534602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb788101fc0 msgr2=0x7fb78819ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:26.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.995+0000 7fb78eaab640 1 --2- 172.21.15.86:0/1915534602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb788101fc0 0x7fb78819ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:26.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.995+0000 7fb78eaab640 1 -- 172.21.15.86:0/1915534602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb78819fd50 con 0x7fb788102980 2024-04-21T08:36:26.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.995+0000 7fb78daa9640 1 --2- 172.21.15.86:0/1915534602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb788101fc0 0x7fb78819ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:36:26.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.995+0000 7fb78eaab640 1 --2- 172.21.15.86:0/1915534602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb788102980 0x7fb78819f490 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb78002c700 tx=0x7fb7800093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:26.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.995+0000 7fb77f7fe640 1 -- 172.21.15.86:0/1915534602 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb780041020 con 0x7fb788102980 2024-04-21T08:36:26.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.995+0000 7fb77f7fe640 1 -- 172.21.15.86:0/1915534602 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb780034d60 con 0x7fb788102980 2024-04-21T08:36:26.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.995+0000 7fb79050e640 1 -- 172.21.15.86:0/1915534602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb78819ffe0 con 0x7fb788102980 2024-04-21T08:36:26.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.995+0000 7fb77f7fe640 1 -- 172.21.15.86:0/1915534602 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb780039040 con 0x7fb788102980 2024-04-21T08:36:26.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.995+0000 7fb79050e640 1 -- 172.21.15.86:0/1915534602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb7881a04f0 con 0x7fb788102980 2024-04-21T08:36:26.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.996+0000 7fb79050e640 1 -- 172.21.15.86:0/1915534602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb788108180 con 0x7fb788102980 2024-04-21T08:36:26.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.997+0000 7fb77f7fe640 1 -- 172.21.15.86:0/1915534602 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fb780044020 con 0x7fb788102980 2024-04-21T08:36:26.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.998+0000 7fb77f7fe640 1 --2- 172.21.15.86:0/1915534602 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb7740a3c30 0x7fb7740a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:26.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.998+0000 7fb78e2aa640 1 --2- 172.21.15.86:0/1915534602 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb7740a3c30 0x7fb7740a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:26.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.998+0000 7fb77f7fe640 1 -- 172.21.15.86:0/1915534602 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fb7800f29c0 con 0x7fb788102980 2024-04-21T08:36:26.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:26.998+0000 7fb78e2aa640 1 --2- 172.21.15.86:0/1915534602 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb7740a3c30 0x7fb7740a60f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb7780058c0 tx=0x7fb77800a4c0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:27.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.001+0000 7fb77f7fe640 1 -- 172.21.15.86:0/1915534602 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb78004b020 con 0x7fb788102980 2024-04-21T08:36:27.213 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:26 smithi088 ceph-mon[31259]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-21T08:36:27.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:26 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1149502879' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-21T08:36:27.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:26 smithi088 ceph-mon[31259]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T08:36:27.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:26 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/68670884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:27.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:26 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1475010588' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:27.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:26 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/245969189' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:27.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.216+0000 7fb79050e640 1 -- 172.21.15.86:0/1915534602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb78802d5c0 con 0x7fb788102980 2024-04-21T08:36:27.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.216+0000 7fb77f7fe640 1 -- 172.21.15.86:0/1915534602 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 17 v17) ==== 94+0+11800 (secure 0 0 0) 0x7fb7800bf160 con 0x7fb788102980 2024-04-21T08:36:27.216 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:36:27.216 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":17,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5},{"gid":14640,"name":"g","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":11},{"gid":14652,"name":"a","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":12},{"gid":24394,"name":"l","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":2},{"gid":24398,"name":"k","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":3},{"gid":24428,"name":"e","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":6},{"gid":24436,"name":"f","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":7},{"gid":24448,"name":"c","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":8},{"gid":24455,"name":"h","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":9},{"gid":24472,"name":"i","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":10}],"filesystems":[{"mdsmap":{"epoch":17,"flags":18,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":false,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":0,"explicitly_allowed_features":0,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:36:25.251833+0000","tableserver":0,"root":0,"session_timeout":60,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":1,"in":[0],"up":{"mds_0":24482},"failed":[],"damaged":[],"stopped":[],"info":{"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":1,"qdb_leader":24482,"qdb_cluster":[24482]},"id":1}]} 2024-04-21T08:36:27.216 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 17 2024-04-21T08:36:27.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.218+0000 7fb79050e640 1 -- 172.21.15.86:0/1915534602 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb7740a3c30 msgr2=0x7fb7740a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:27.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.218+0000 7fb79050e640 1 --2- 172.21.15.86:0/1915534602 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb7740a3c30 0x7fb7740a60f0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7fb7780058c0 tx=0x7fb77800a4c0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.218+0000 7fb79050e640 1 -- 172.21.15.86:0/1915534602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb788102980 msgr2=0x7fb78819f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:27.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.218+0000 7fb79050e640 1 --2- 172.21.15.86:0/1915534602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb788102980 0x7fb78819f490 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fb78002c700 tx=0x7fb7800093f0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.218+0000 7fb79050e640 1 -- 172.21.15.86:0/1915534602 shutdown_connections 2024-04-21T08:36:27.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.218+0000 7fb79050e640 1 --2- 172.21.15.86:0/1915534602 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb7740a3c30 0x7fb7740a60f0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.218+0000 7fb79050e640 1 --2- 172.21.15.86:0/1915534602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb788102980 0x7fb78819f490 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.218+0000 7fb79050e640 1 --2- 172.21.15.86:0/1915534602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb788101fc0 0x7fb78819ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.218+0000 7fb79050e640 1 --2- 172.21.15.86:0/1915534602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb788100e70 0x7fb788069c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.218+0000 7fb79050e640 1 -- 172.21.15.86:0/1915534602 >> 172.21.15.86:0/1915534602 conn(0x7fb7880fc3e0 msgr2=0x7fb788105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:27.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.219+0000 7fb79050e640 1 -- 172.21.15.86:0/1915534602 shutdown_connections 2024-04-21T08:36:27.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.219+0000 7fb79050e640 1 -- 172.21.15.86:0/1915534602 wait complete. 2024-04-21T08:36:27.227 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:26 smithi086 ceph-mon[27939]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-21T08:36:27.228 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:26 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1149502879' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-21T08:36:27.228 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:26 smithi086 ceph-mon[27939]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T08:36:27.228 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:26 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/68670884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:27.228 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:26 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1475010588' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:27.228 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:26 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/245969189' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:27.229 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:36:27.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:26 smithi179 ceph-mon[31829]: Health check failed: 1 pool(s) do not have an application enabled (POOL_APP_NOT_ENABLED) 2024-04-21T08:36:27.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:26 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1149502879' entity='client.admin' cmd='[{"prefix": "fs add_data_pool", "fs_name": "cephfs", "pool": "cephfs_data_ec"}]': finished 2024-04-21T08:36:27.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:26 smithi179 ceph-mon[31829]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T08:36:27.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:26 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/68670884' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:27.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:26 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1475010588' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:27.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:26 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/245969189' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:27.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.327+0000 7ff0f02f2640 1 -- 172.21.15.86:0/1498337597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0e8101fc0 msgr2=0x7ff0e8102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:27.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.327+0000 7ff0f02f2640 1 --2- 172.21.15.86:0/1498337597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0e8101fc0 0x7ff0e8102440 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4009900 tx=0x7ff0e402bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:27.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.327+0000 7ff0f02f2640 1 -- 172.21.15.86:0/1498337597 shutdown_connections 2024-04-21T08:36:27.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.327+0000 7ff0f02f2640 1 --2- 172.21.15.86:0/1498337597 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0e8102980 0x7ff0e81092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.327+0000 7ff0f02f2640 1 --2- 172.21.15.86:0/1498337597 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0e8101fc0 0x7ff0e8102440 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.327+0000 7ff0f02f2640 1 --2- 172.21.15.86:0/1498337597 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0e8100e70 0x7ff0e8101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.327+0000 7ff0f02f2640 1 -- 172.21.15.86:0/1498337597 >> 172.21.15.86:0/1498337597 conn(0x7ff0e80fc3e0 msgr2=0x7ff0e80fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:27.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.327+0000 7ff0f02f2640 1 -- 172.21.15.86:0/1498337597 shutdown_connections 2024-04-21T08:36:27.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.327+0000 7ff0f02f2640 1 -- 172.21.15.86:0/1498337597 wait complete. 2024-04-21T08:36:27.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.327+0000 7ff0f02f2640 1 Processor -- start 2024-04-21T08:36:27.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.327+0000 7ff0f02f2640 1 -- start start 2024-04-21T08:36:27.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.328+0000 7ff0f02f2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0e8100e70 0x7ff0e8198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:27.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.328+0000 7ff0f02f2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0e8101fc0 0x7ff0e8198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:27.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.328+0000 7ff0f02f2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0e8102980 0x7ff0e819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:27.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.328+0000 7ff0f02f2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e810db70 con 0x7ff0e8101fc0 2024-04-21T08:36:27.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.328+0000 7ff0f02f2640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e810d9f0 con 0x7ff0e8102980 2024-04-21T08:36:27.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.328+0000 7ff0f02f2640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff0e810dcf0 con 0x7ff0e8100e70 2024-04-21T08:36:27.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.328+0000 7ff0ed88d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0e8101fc0 0x7ff0e8198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:27.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.328+0000 7ff0ed88d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0e8101fc0 0x7ff0e8198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36068/0 (socket says 172.21.15.86:36068) 2024-04-21T08:36:27.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.328+0000 7ff0ed88d640 1 -- 172.21.15.86:0/390978677 learned_addr learned my addr 172.21.15.86:0/390978677 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:27.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.328+0000 7ff0ee08e640 1 --2- 172.21.15.86:0/390978677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0e8100e70 0x7ff0e8198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:27.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.328+0000 7ff0ee88f640 1 --2- 172.21.15.86:0/390978677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0e8102980 0x7ff0e819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:27.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.328+0000 7ff0ed88d640 1 -- 172.21.15.86:0/390978677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0e8100e70 msgr2=0x7ff0e8198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:27.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.328+0000 7ff0ed88d640 1 --2- 172.21.15.86:0/390978677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0e8100e70 0x7ff0e8198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.328+0000 7ff0ed88d640 1 -- 172.21.15.86:0/390978677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0e8102980 msgr2=0x7ff0e819f7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:27.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.328+0000 7ff0ed88d640 1 --2- 172.21.15.86:0/390978677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0e8102980 0x7ff0e819f7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.328+0000 7ff0ed88d640 1 -- 172.21.15.86:0/390978677 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0e819fe30 con 0x7ff0e8101fc0 2024-04-21T08:36:27.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.328+0000 7ff0ee88f640 1 --2- 172.21.15.86:0/390978677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0e8102980 0x7ff0e819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:36:27.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.328+0000 7ff0ed88d640 1 --2- 172.21.15.86:0/390978677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0e8101fc0 0x7ff0e8198750 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4005ab0 tx=0x7ff0e40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:27.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.329+0000 7ff0db7fe640 1 -- 172.21.15.86:0/390978677 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4043020 con 0x7ff0e8101fc0 2024-04-21T08:36:27.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.329+0000 7ff0f02f2640 1 -- 172.21.15.86:0/390978677 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0e81a00c0 con 0x7ff0e8101fc0 2024-04-21T08:36:27.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.329+0000 7ff0db7fe640 1 -- 172.21.15.86:0/390978677 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0e40054c0 con 0x7ff0e8101fc0 2024-04-21T08:36:27.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.329+0000 7ff0db7fe640 1 -- 172.21.15.86:0/390978677 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0e4003de0 con 0x7ff0e8101fc0 2024-04-21T08:36:27.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.329+0000 7ff0f02f2640 1 -- 172.21.15.86:0/390978677 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0e81a06b0 con 0x7ff0e8101fc0 2024-04-21T08:36:27.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.330+0000 7ff0f02f2640 1 -- 172.21.15.86:0/390978677 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff0e8068de0 con 0x7ff0e8101fc0 2024-04-21T08:36:27.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.331+0000 7ff0db7fe640 1 -- 172.21.15.86:0/390978677 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7ff0e4046020 con 0x7ff0e8101fc0 2024-04-21T08:36:27.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.331+0000 7ff0db7fe640 1 --2- 172.21.15.86:0/390978677 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ff0d40a3bb0 0x7ff0d40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:27.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.331+0000 7ff0ee08e640 1 --2- 172.21.15.86:0/390978677 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ff0d40a3bb0 0x7ff0d40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:27.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.331+0000 7ff0db7fe640 1 -- 172.21.15.86:0/390978677 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7ff0e40f2ab0 con 0x7ff0e8101fc0 2024-04-21T08:36:27.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.332+0000 7ff0ee08e640 1 --2- 172.21.15.86:0/390978677 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ff0d40a3bb0 0x7ff0d40a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc0058c0 tx=0x7ff0dc00a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:27.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.336+0000 7ff0db7fe640 1 -- 172.21.15.86:0/390978677 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff0e404d020 con 0x7ff0e8101fc0 2024-04-21T08:36:27.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.515+0000 7ff0f02f2640 1 -- 172.21.15.86:0/390978677 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7ff0e8062b50 con 0x7ff0e8101fc0 2024-04-21T08:36:27.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.517+0000 7ff0db7fe640 1 -- 172.21.15.86:0/390978677 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24070 (secure 0 0 0) 0x7ff0e40bf370 con 0x7ff0e8101fc0 2024-04-21T08:36:27.516 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:36:27.517 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":89,"fsid":"f8cec7cc-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:22.687329+0000","modified":"2024-04-21T08:36:25.251657+0000","last_up_change":"2024-04-21T08:35:30.048315+0000","last_in_change":"2024-04-21T08:35:10.099496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:45.279109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:18.300782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:19.574636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:22.905632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89b7601f-0620-4db2-90f1-b45cea173738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6803","nonce":2571516897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6805","nonce":2571516897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6809","nonce":2571516897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6807","nonce":2571516897}]},"public_addr":"172.21.15.86:6803/2571516897","cluster_addr":"172.21.15.86:6805/2571516897","heartbeat_back_addr":"172.21.15.86:6809/2571516897","heartbeat_front_addr":"172.21.15.86:6807/2571516897","state":["exists","up"]},{"osd":1,"uuid":"4891031c-f98c-42bc-bb3b-a420855ecc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6801","nonce":74067182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6803","nonce":74067182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6807","nonce":74067182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6805","nonce":74067182}]},"public_addr":"172.21.15.88:6801/74067182","cluster_addr":"172.21.15.88:6803/74067182","heartbeat_back_addr":"172.21.15.88:6807/74067182","heartbeat_front_addr":"172.21.15.88:6805/74067182","state":["exists","up"]},{"osd":2,"uuid":"60d12aa4-9312-4fa2-8d0b-2b3f42caad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6801","nonce":4137659051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6803","nonce":4137659051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6807","nonce":4137659051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6805","nonce":4137659051}]},"public_addr":"172.21.15.179:6801/4137659051","cluster_addr":"172.21.15.179:6803/4137659051","heartbeat_back_addr":"172.21.15.179:6807/4137659051","heartbeat_front_addr":"172.21.15.179:6805/4137659051","state":["exists","up"]},{"osd":3,"uuid":"404c3158-8cb0-4c02-b48a-f9ee5ff7b784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6811","nonce":2309603148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6813","nonce":2309603148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6817","nonce":2309603148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6815","nonce":2309603148}]},"public_addr":"172.21.15.86:6811/2309603148","cluster_addr":"172.21.15.86:6813/2309603148","heartbeat_back_addr":"172.21.15.86:6817/2309603148","heartbeat_front_addr":"172.21.15.86:6815/2309603148","state":["exists","up"]},{"osd":4,"uuid":"458ff202-d781-4ec7-ac93-0b49f835b503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6809","nonce":1818714217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6811","nonce":1818714217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6815","nonce":1818714217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6813","nonce":1818714217}]},"public_addr":"172.21.15.88:6809/1818714217","cluster_addr":"172.21.15.88:6811/1818714217","heartbeat_back_addr":"172.21.15.88:6815/1818714217","heartbeat_front_addr":"172.21.15.88:6813/1818714217","state":["exists","up"]},{"osd":5,"uuid":"8123cc90-bd8b-4204-bf31-46b34e9f589d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6809","nonce":863840990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6811","nonce":863840990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6815","nonce":863840990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6813","nonce":863840990}]},"public_addr":"172.21.15.179:6809/863840990","cluster_addr":"172.21.15.179:6811/863840990","heartbeat_back_addr":"172.21.15.179:6815/863840990","heartbeat_front_addr":"172.21.15.179:6813/863840990","state":["exists","up"]},{"osd":6,"uuid":"aac9d7c7-659a-4110-903a-a871c086066f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6819","nonce":1224231118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6821","nonce":1224231118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6825","nonce":1224231118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6823","nonce":1224231118}]},"public_addr":"172.21.15.86:6819/1224231118","cluster_addr":"172.21.15.86:6821/1224231118","heartbeat_back_addr":"172.21.15.86:6825/1224231118","heartbeat_front_addr":"172.21.15.86:6823/1224231118","state":["exists","up"]},{"osd":7,"uuid":"df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6817","nonce":136629878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6819","nonce":136629878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6823","nonce":136629878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6821","nonce":136629878}]},"public_addr":"172.21.15.88:6817/136629878","cluster_addr":"172.21.15.88:6819/136629878","heartbeat_back_addr":"172.21.15.88:6823/136629878","heartbeat_front_addr":"172.21.15.88:6821/136629878","state":["exists","up"]},{"osd":8,"uuid":"63e59f4b-0426-46db-92f7-c55dae61b5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6817","nonce":1770418957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6819","nonce":1770418957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6823","nonce":1770418957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6821","nonce":1770418957}]},"public_addr":"172.21.15.179:6817/1770418957","cluster_addr":"172.21.15.179:6819/1770418957","heartbeat_back_addr":"172.21.15.179:6823/1770418957","heartbeat_front_addr":"172.21.15.179:6821/1770418957","state":["exists","up"]},{"osd":9,"uuid":"5841bb36-ed6b-45d8-b857-eeab139c44e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6827","nonce":1108500274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6829","nonce":1108500274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6833","nonce":1108500274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6831","nonce":1108500274}]},"public_addr":"172.21.15.86:6827/1108500274","cluster_addr":"172.21.15.86:6829/1108500274","heartbeat_back_addr":"172.21.15.86:6833/1108500274","heartbeat_front_addr":"172.21.15.86:6831/1108500274","state":["exists","up"]},{"osd":10,"uuid":"f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6825","nonce":1294275353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6827","nonce":1294275353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6831","nonce":1294275353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6829","nonce":1294275353}]},"public_addr":"172.21.15.88:6825/1294275353","cluster_addr":"172.21.15.88:6827/1294275353","heartbeat_back_addr":"172.21.15.88:6831/1294275353","heartbeat_front_addr":"172.21.15.88:6829/1294275353","state":["exists","up"]},{"osd":11,"uuid":"e0d4cc76-e7a6-4a6b-909c-95488ccd5a36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6825","nonce":4007070233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6827","nonce":4007070233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6831","nonce":4007070233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6829","nonce":4007070233}]},"public_addr":"172.21.15.179:6825/4007070233","cluster_addr":"172.21.15.179:6827/4007070233","heartbeat_back_addr":"172.21.15.179:6831/4007070233","heartbeat_front_addr":"172.21.15.179:6829/4007070233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:55.210605+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:18.723090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:42.547979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:06.421622+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:30.858315+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:55.455313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.623369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:46.185111+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.103627+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:37.297545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:02.637431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:28.199292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6800/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/1412393225":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6800/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/1687852404":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2020707404":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2568130933":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/3952607648":"2024-04-22T08:28:57.384403+0000","172.21.15.86:6801/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/2756903426":"2024-04-22T08:29:23.190001+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:36:27.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.519+0000 7ff0f02f2640 1 -- 172.21.15.86:0/390978677 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ff0d40a3bb0 msgr2=0x7ff0d40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:27.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.519+0000 7ff0f02f2640 1 --2- 172.21.15.86:0/390978677 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ff0d40a3bb0 0x7ff0d40a6070 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7ff0dc0058c0 tx=0x7ff0dc00a2d0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.519+0000 7ff0f02f2640 1 -- 172.21.15.86:0/390978677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0e8101fc0 msgr2=0x7ff0e8198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:27.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.519+0000 7ff0f02f2640 1 --2- 172.21.15.86:0/390978677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0e8101fc0 0x7ff0e8198750 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7ff0e4005ab0 tx=0x7ff0e40042f0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.519+0000 7ff0f02f2640 1 -- 172.21.15.86:0/390978677 shutdown_connections 2024-04-21T08:36:27.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.519+0000 7ff0f02f2640 1 --2- 172.21.15.86:0/390978677 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ff0d40a3bb0 0x7ff0d40a6070 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.519+0000 7ff0f02f2640 1 --2- 172.21.15.86:0/390978677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0e8102980 0x7ff0e819f7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.519+0000 7ff0f02f2640 1 --2- 172.21.15.86:0/390978677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0e8101fc0 0x7ff0e8198750 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.519+0000 7ff0f02f2640 1 --2- 172.21.15.86:0/390978677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0e8100e70 0x7ff0e8198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.519+0000 7ff0f02f2640 1 -- 172.21.15.86:0/390978677 >> 172.21.15.86:0/390978677 conn(0x7ff0e80fc3e0 msgr2=0x7ff0e8105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:27.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.519+0000 7ff0f02f2640 1 -- 172.21.15.86:0/390978677 shutdown_connections 2024-04-21T08:36:27.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.519+0000 7ff0f02f2640 1 -- 172.21.15.86:0/390978677 wait complete. 2024-04-21T08:36:27.530 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 17, 'flags': 18, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': False, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 0, 'explicitly_allowed_features': 0, 'created': '2024-04-21T08:36:20.626894+0000', 'modified': '2024-04-21T08:36:25.251833+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 60, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 1, 'in': [0], 'up': {'mds_0': 24482}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.88:6839/3969608142', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6838', 'nonce': 3969608142}, {'type': 'v1', 'addr': '172.21.15.88:6839', 'nonce': 3969608142}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 1, 'qdb_leader': 24482, 'qdb_cluster': [24482]} 2024-04-21T08:36:27.530 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 1/1 2024-04-21T08:36:27.530 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-21T08:36:27.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.629+0000 7fde13c8c640 1 -- 172.21.15.86:0/1920792832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde0c1029a0 msgr2=0x7fde0c1092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:27.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.629+0000 7fde13c8c640 1 --2- 172.21.15.86:0/1920792832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde0c1029a0 0x7fde0c1092c0 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7fde04009ca0 tx=0x7fde0402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.629+0000 7fde13c8c640 1 -- 172.21.15.86:0/1920792832 shutdown_connections 2024-04-21T08:36:27.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.629+0000 7fde13c8c640 1 --2- 172.21.15.86:0/1920792832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde0c1029a0 0x7fde0c1092c0 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.630+0000 7fde13c8c640 1 --2- 172.21.15.86:0/1920792832 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde0c101fe0 0x7fde0c102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.630+0000 7fde13c8c640 1 --2- 172.21.15.86:0/1920792832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde0c100e90 0x7fde0c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.630+0000 7fde13c8c640 1 -- 172.21.15.86:0/1920792832 >> 172.21.15.86:0/1920792832 conn(0x7fde0c0fc400 msgr2=0x7fde0c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:27.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.630+0000 7fde13c8c640 1 -- 172.21.15.86:0/1920792832 shutdown_connections 2024-04-21T08:36:27.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.630+0000 7fde13c8c640 1 -- 172.21.15.86:0/1920792832 wait complete. 2024-04-21T08:36:27.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.630+0000 7fde13c8c640 1 Processor -- start 2024-04-21T08:36:27.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.630+0000 7fde13c8c640 1 -- start start 2024-04-21T08:36:27.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.630+0000 7fde13c8c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde0c100e90 0x7fde0c198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:27.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.630+0000 7fde13c8c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde0c101fe0 0x7fde0c198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:27.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.630+0000 7fde13c8c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde0c1029a0 0x7fde0c19f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:27.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.630+0000 7fde13c8c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c10db50 con 0x7fde0c1029a0 2024-04-21T08:36:27.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.630+0000 7fde13c8c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c10d9d0 con 0x7fde0c101fe0 2024-04-21T08:36:27.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.630+0000 7fde13c8c640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fde0c10dcd0 con 0x7fde0c100e90 2024-04-21T08:36:27.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.630+0000 7fde12229640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde0c1029a0 0x7fde0c19f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:27.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.630+0000 7fde12229640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde0c1029a0 0x7fde0c19f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36090/0 (socket says 172.21.15.86:36090) 2024-04-21T08:36:27.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.630+0000 7fde12229640 1 -- 172.21.15.86:0/616318246 learned_addr learned my addr 172.21.15.86:0/616318246 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:27.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.631+0000 7fde12229640 1 -- 172.21.15.86:0/616318246 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde0c100e90 msgr2=0x7fde0c198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:27.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.631+0000 7fde12229640 1 --2- 172.21.15.86:0/616318246 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde0c100e90 0x7fde0c198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.631+0000 7fde12229640 1 -- 172.21.15.86:0/616318246 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde0c101fe0 msgr2=0x7fde0c198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:27.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.631+0000 7fde12229640 1 --2- 172.21.15.86:0/616318246 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde0c101fe0 0x7fde0c198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.631+0000 7fde12229640 1 -- 172.21.15.86:0/616318246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde0c19fe30 con 0x7fde0c1029a0 2024-04-21T08:36:27.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.631+0000 7fde12229640 1 --2- 172.21.15.86:0/616318246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde0c1029a0 0x7fde0c19f7c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fde0402c700 tx=0x7fde04035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:27.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.631+0000 7fde02ffd640 1 -- 172.21.15.86:0/616318246 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde04039040 con 0x7fde0c1029a0 2024-04-21T08:36:27.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.631+0000 7fde02ffd640 1 -- 172.21.15.86:0/616318246 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde04044070 con 0x7fde0c1029a0 2024-04-21T08:36:27.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.631+0000 7fde13c8c640 1 -- 172.21.15.86:0/616318246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde0c1a00c0 con 0x7fde0c1029a0 2024-04-21T08:36:27.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.631+0000 7fde02ffd640 1 -- 172.21.15.86:0/616318246 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde04004440 con 0x7fde0c1029a0 2024-04-21T08:36:27.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.631+0000 7fde13c8c640 1 -- 172.21.15.86:0/616318246 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde0c1a0600 con 0x7fde0c1029a0 2024-04-21T08:36:27.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.633+0000 7fde02ffd640 1 -- 172.21.15.86:0/616318246 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fde040045e0 con 0x7fde0c1029a0 2024-04-21T08:36:27.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.633+0000 7fde16881740 1 Processor -- start 2024-04-21T08:36:27.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.634+0000 7fde16881740 1 -- start start 2024-04-21T08:36:27.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.634+0000 7fde16881740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55606a573e40 0x55606a574260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:27.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.634+0000 7fde16881740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55606a574830 0x55606a574cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:27.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.634+0000 7fde16881740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55606a5752d0 0x55606a57beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:27.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.634+0000 7fde16881740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55606a4d8b10 con 0x55606a5752d0 2024-04-21T08:36:27.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.634+0000 7fde16881740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55606a476240 con 0x55606a573e40 2024-04-21T08:36:27.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.634+0000 7fde16881740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55606a4d4ad0 con 0x55606a574830 2024-04-21T08:36:27.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.634+0000 7fde02ffd640 1 --2- 172.21.15.86:0/616318246 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fddf80a3c30 0x7fddf80a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:27.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.634+0000 7fde11227640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55606a5752d0 0x55606a57beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:27.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.634+0000 7fde11227640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55606a5752d0 0x55606a57beb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36102/0 (socket says 172.21.15.86:36102) 2024-04-21T08:36:27.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.634+0000 7fde11227640 1 -- 172.21.15.86:0/792739219 learned_addr learned my addr 172.21.15.86:0/792739219 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:27.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.634+0000 7fde11a28640 1 --2- 172.21.15.86:0/792739219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55606a574830 0x55606a574cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:27.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.634+0000 7fde12229640 1 --2- 172.21.15.86:0/616318246 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fddf80a3c30 0x7fddf80a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:27.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.634+0000 7fde11227640 1 --2- 172.21.15.86:0/792739219 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55606a573e40 0x55606a574260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:27.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.634+0000 7fde02ffd640 1 -- 172.21.15.86:0/616318246 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fde040f2d50 con 0x7fde0c1029a0 2024-04-21T08:36:27.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.634+0000 7fde11227640 1 -- 172.21.15.86:0/792739219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55606a574830 msgr2=0x55606a574cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:36:27.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.634+0000 7fde11227640 1 --2- 172.21.15.86:0/792739219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55606a574830 0x55606a574cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.634+0000 7fde11227640 1 -- 172.21.15.86:0/792739219 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55606a573e40 msgr2=0x55606a574260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:36:27.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.634+0000 7fde11227640 1 --2- 172.21.15.86:0/792739219 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55606a573e40 0x55606a574260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.634+0000 7fde11227640 1 -- 172.21.15.86:0/792739219 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55606a1eb2d0 con 0x55606a5752d0 2024-04-21T08:36:27.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.634+0000 7fde11a28640 1 --2- 172.21.15.86:0/792739219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55606a574830 0x55606a574cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:36:27.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.634+0000 7fde12229640 1 --2- 172.21.15.86:0/616318246 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fddf80a3c30 0x7fddf80a60f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fde04032380 tx=0x7fde04089440 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:27.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.635+0000 7fde11227640 1 --2- 172.21.15.86:0/792739219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55606a5752d0 0x55606a57beb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fde08005e40 tx=0x7fde0800c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81fcee2780fd5ecc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:27.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.635+0000 7fddf7fff640 1 -- 172.21.15.86:0/792739219 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0800ed00 con 0x55606a5752d0 2024-04-21T08:36:27.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.635+0000 7fddf7fff640 1 -- 172.21.15.86:0/792739219 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde080035f0 con 0x55606a5752d0 2024-04-21T08:36:27.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.635+0000 7fddf7fff640 1 -- 172.21.15.86:0/792739219 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde080126e0 con 0x55606a5752d0 2024-04-21T08:36:27.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.635+0000 7fde16881740 1 -- 172.21.15.86:0/792739219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55606a5752d0 msgr2=0x55606a57beb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:27.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.635+0000 7fde16881740 1 --2- 172.21.15.86:0/792739219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55606a5752d0 0x55606a57beb0 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7fde08005e40 tx=0x7fde0800c450 comp rx=0 tx=0).stop 2024-04-21T08:36:27.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.635+0000 7fde16881740 1 -- 172.21.15.86:0/792739219 shutdown_connections 2024-04-21T08:36:27.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.635+0000 7fde16881740 1 --2- 172.21.15.86:0/792739219 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55606a5752d0 0x55606a57beb0 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.635+0000 7fde16881740 1 --2- 172.21.15.86:0/792739219 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55606a574830 0x55606a574cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.635+0000 7fde16881740 1 --2- 172.21.15.86:0/792739219 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55606a573e40 0x55606a574260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.635+0000 7fde16881740 1 -- 172.21.15.86:0/792739219 >> 172.21.15.86:0/792739219 conn(0x55606a56ff40 msgr2=0x55606a56e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:27.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.636+0000 7fde16881740 1 -- 172.21.15.86:0/792739219 shutdown_connections 2024-04-21T08:36:27.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.636+0000 7fde16881740 1 -- 172.21.15.86:0/792739219 wait complete. 2024-04-21T08:36:27.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.636+0000 7fde16881740 1 -- start start 2024-04-21T08:36:27.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.636+0000 7fde16881740 1 Processor -- start 2024-04-21T08:36:27.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.636+0000 7fde16881740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55606a573e40 0x55606a57dee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:27.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.636+0000 7fde16881740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55606a574830 0x55606a57e420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:27.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.636+0000 7fde16881740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55606a5752d0 0x55606a585460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:27.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.636+0000 7fde16881740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55606a57c4b0 con 0x55606a5752d0 2024-04-21T08:36:27.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.636+0000 7fde16881740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55606a461190 con 0x55606a573e40 2024-04-21T08:36:27.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.636+0000 7fde16881740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55606a57c630 con 0x55606a574830 2024-04-21T08:36:27.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.636+0000 7fde16881740 10 client.0 _ll_register_callbacks cb 0x55606a45c8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:36:27.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.636+0000 7fde11227640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55606a5752d0 0x55606a585460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:27.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.636+0000 7fde11227640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55606a5752d0 0x55606a585460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36104/0 (socket says 172.21.15.86:36104) 2024-04-21T08:36:27.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.636+0000 7fde11227640 1 -- 172.21.15.86:0/1406718135 learned_addr learned my addr 172.21.15.86:0/1406718135 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:27.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.637+0000 7fde11a28640 1 --2- 172.21.15.86:0/1406718135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55606a574830 0x55606a57e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:27.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.637+0000 7fde11227640 1 -- 172.21.15.86:0/1406718135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55606a574830 msgr2=0x55606a57e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:36:27.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.637+0000 7fde11227640 1 --2- 172.21.15.86:0/1406718135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55606a574830 0x55606a57e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.637+0000 7fde11227640 1 -- 172.21.15.86:0/1406718135 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55606a573e40 msgr2=0x55606a57dee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:36:27.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.637+0000 7fde11227640 1 --2- 172.21.15.86:0/1406718135 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55606a573e40 0x55606a57dee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.637+0000 7fde11227640 1 -- 172.21.15.86:0/1406718135 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55606a2b56b0 con 0x55606a5752d0 2024-04-21T08:36:27.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.637+0000 7fde11227640 1 --2- 172.21.15.86:0/1406718135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55606a5752d0 0x55606a585460 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fde08004700 tx=0x7fde08003d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55e2f9b7f966b5e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:27.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.637+0000 7fddf57fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:36:27.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.637+0000 7fddf57fa640 1 -- 172.21.15.86:0/1406718135 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde08004000 con 0x55606a5752d0 2024-04-21T08:36:27.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.637+0000 7fddf57fa640 1 -- 172.21.15.86:0/1406718135 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fde080041a0 con 0x55606a5752d0 2024-04-21T08:36:27.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.637+0000 7fde16881740 1 -- 172.21.15.86:0/1406718135 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde080035f0 con 0x55606a5752d0 2024-04-21T08:36:27.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.637+0000 7fddf57fa640 1 -- 172.21.15.86:0/1406718135 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fde080043a0 con 0x55606a5752d0 2024-04-21T08:36:27.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.637+0000 7fde16881740 10 client.14778 fetch_fsmap learned FSMap version 17 2024-04-21T08:36:27.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.637+0000 7fde16881740 1 -- 172.21.15.86:0/1406718135 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55606a5887d0 con 0x55606a5752d0 2024-04-21T08:36:27.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.637+0000 7fddf57fa640 1 -- 172.21.15.86:0/1406718135 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde080035f0 con 0x55606a5752d0 2024-04-21T08:36:27.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.637+0000 7fddf57fa640 1 -- 172.21.15.86:0/1406718135 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7fde08023030 con 0x55606a5752d0 2024-04-21T08:36:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.637+0000 7fde16881740 10 client.14778 fetch_fsmap finished waiting for FSMap version 17 2024-04-21T08:36:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.638+0000 7fde16881740 10 client.14778 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-21T08:36:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.638+0000 7fde16881740 20 client.14778 populate_metadata read hostname 'smithi086' 2024-04-21T08:36:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.638+0000 7fde16881740 1 --2- 172.21.15.86:0/1406718135 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55606a58ac00 0x55606a58c070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.638+0000 7fde16881740 4 client.14778 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:36:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.638+0000 7fde16881740 1 -- 172.21.15.86:0/1406718135 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55606a4e6530 con 0x55606a58ac00 2024-04-21T08:36:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.638+0000 7fde11a28640 1 --2- 172.21.15.86:0/1406718135 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55606a58ac00 0x55606a58c070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.638+0000 7fde11a28640 1 --2- 172.21.15.86:0/1406718135 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55606a58ac00 0x55606a58c070 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3aec21f9ddda53a server_cookie=4ccf535dcb9846e9 in_seq=0 out_seq=0 2024-04-21T08:36:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.638+0000 7fddf57fa640 10 client.14778 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:36:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.639+0000 7fddf57fa640 1 -- 172.21.15.86:0/1406718135 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55606a4e6530 con 0x55606a58ac00 2024-04-21T08:36:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.639+0000 7fddf57fa640 10 client.14778 handle_command_reply: tid=0 2024-04-21T08:36:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.639+0000 7fde16881740 1 client.14778 shutdown 2024-04-21T08:36:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.639+0000 7fde16881740 1 -- 172.21.15.86:0/1406718135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55606a5752d0 msgr2=0x55606a585460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:27.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.639+0000 7fde16881740 1 --2- 172.21.15.86:0/1406718135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55606a5752d0 0x55606a585460 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fde08004700 tx=0x7fde08003d70 comp rx=0 tx=0).stop 2024-04-21T08:36:27.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.639+0000 7fde16881740 1 -- 172.21.15.86:0/1406718135 shutdown_connections 2024-04-21T08:36:27.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.639+0000 7fde16881740 1 --2- 172.21.15.86:0/1406718135 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55606a58ac00 0x55606a58c070 crc :-1 s=READY pgs=13 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.639+0000 7fde16881740 1 --2- 172.21.15.86:0/1406718135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55606a5752d0 0x55606a585460 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.639+0000 7fddf57fa640 0 client.14778 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:36:27.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.639+0000 7fde16881740 1 --2- 172.21.15.86:0/1406718135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55606a574830 0x55606a57e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.639+0000 7fde16881740 1 --2- 172.21.15.86:0/1406718135 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55606a573e40 0x55606a57dee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.639+0000 7fde16881740 1 -- 172.21.15.86:0/1406718135 >> 172.21.15.86:0/1406718135 conn(0x55606a56ff40 msgr2=0x55606a56da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:27.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.639+0000 7fde16881740 1 -- 172.21.15.86:0/1406718135 shutdown_connections 2024-04-21T08:36:27.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.639+0000 7fde16881740 1 -- 172.21.15.86:0/1406718135 wait complete. 2024-04-21T08:36:27.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.639+0000 7fde16881740 20 client.14778 trim_cache size 0 max 16384 2024-04-21T08:36:27.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.657+0000 7fde16881740 1 Processor -- start 2024-04-21T08:36:27.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.657+0000 7fde16881740 1 -- start start 2024-04-21T08:36:27.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.657+0000 7fde16881740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55606a573e40 0x55606a589c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:27.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.657+0000 7fde16881740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55606a574830 0x55606a585420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:27.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.657+0000 7fde16881740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55606a5752d0 0x55606a578840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:27.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.657+0000 7fde16881740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55606a58a3b0 con 0x55606a5752d0 2024-04-21T08:36:27.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.657+0000 7fde16881740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55606a58aa80 con 0x55606a574830 2024-04-21T08:36:27.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.657+0000 7fde16881740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55606a58b150 con 0x55606a573e40 2024-04-21T08:36:27.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.657+0000 7fde11227640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55606a5752d0 0x55606a578840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:27.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.658+0000 7fde11227640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55606a5752d0 0x55606a578840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36108/0 (socket says 172.21.15.86:36108) 2024-04-21T08:36:27.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.658+0000 7fde11227640 1 -- 172.21.15.86:0/2315455580 learned_addr learned my addr 172.21.15.86:0/2315455580 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:27.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.658+0000 7fde11a28640 1 --2- 172.21.15.86:0/2315455580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55606a574830 0x55606a585420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:27.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.658+0000 7fde11227640 1 --2- 172.21.15.86:0/2315455580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55606a573e40 0x55606a589c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:27.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.658+0000 7fde11227640 1 -- 172.21.15.86:0/2315455580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55606a573e40 msgr2=0x55606a589c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:36:27.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.658+0000 7fde11227640 1 --2- 172.21.15.86:0/2315455580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55606a573e40 0x55606a589c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.658+0000 7fde11227640 1 -- 172.21.15.86:0/2315455580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55606a574830 msgr2=0x55606a585420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:36:27.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.658+0000 7fde11227640 1 --2- 172.21.15.86:0/2315455580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55606a574830 0x55606a585420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.658+0000 7fde11227640 1 -- 172.21.15.86:0/2315455580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55606a58c3a0 con 0x55606a5752d0 2024-04-21T08:36:27.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.658+0000 7fde11227640 1 --2- 172.21.15.86:0/2315455580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55606a5752d0 0x55606a578840 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fde0801aa40 tx=0x7fde08004540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bab7f18219e5fbf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:27.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.658+0000 7fddf77fe640 1 -- 172.21.15.86:0/2315455580 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde08005b90 con 0x55606a5752d0 2024-04-21T08:36:27.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.658+0000 7fddf77fe640 1 -- 172.21.15.86:0/2315455580 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde0801a640 con 0x55606a5752d0 2024-04-21T08:36:27.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.658+0000 7fddf77fe640 1 -- 172.21.15.86:0/2315455580 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde0802b480 con 0x55606a5752d0 2024-04-21T08:36:27.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.659+0000 7fde16881740 1 -- 172.21.15.86:0/2315455580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55606a5752d0 msgr2=0x55606a578840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:27.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.659+0000 7fde16881740 1 --2- 172.21.15.86:0/2315455580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55606a5752d0 0x55606a578840 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fde0801aa40 tx=0x7fde08004540 comp rx=0 tx=0).stop 2024-04-21T08:36:27.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.659+0000 7fde16881740 1 -- 172.21.15.86:0/2315455580 shutdown_connections 2024-04-21T08:36:27.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.659+0000 7fde16881740 1 --2- 172.21.15.86:0/2315455580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55606a5752d0 0x55606a578840 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.659+0000 7fde16881740 1 --2- 172.21.15.86:0/2315455580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55606a574830 0x55606a585420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.659+0000 7fde16881740 1 --2- 172.21.15.86:0/2315455580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55606a573e40 0x55606a589c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.659+0000 7fde16881740 1 -- 172.21.15.86:0/2315455580 >> 172.21.15.86:0/2315455580 conn(0x55606a56ff40 msgr2=0x55606a57ca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:27.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.659+0000 7fde16881740 1 -- 172.21.15.86:0/2315455580 shutdown_connections 2024-04-21T08:36:27.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.659+0000 7fde16881740 1 -- 172.21.15.86:0/2315455580 wait complete. 2024-04-21T08:36:27.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.659+0000 7fde16881740 1 -- start start 2024-04-21T08:36:27.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.659+0000 7fde16881740 1 Processor -- start 2024-04-21T08:36:27.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.659+0000 7fde16881740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55606a573e40 0x55606a581400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:27.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.660+0000 7fde16881740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55606a574830 0x55606a583950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:27.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.660+0000 7fde16881740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55606a5752d0 0x55606a583e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:27.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.660+0000 7fde16881740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55606a58bef0 con 0x55606a574830 2024-04-21T08:36:27.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.660+0000 7fde16881740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55606a58b820 con 0x55606a5752d0 2024-04-21T08:36:27.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.660+0000 7fde16881740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55606a58c5c0 con 0x55606a573e40 2024-04-21T08:36:27.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.660+0000 7fde16881740 10 client.0 _ll_register_callbacks cb 0x7fddfc006f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:36:27.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.660+0000 7fde11a28640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55606a574830 0x55606a583950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:27.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.660+0000 7fde11a28640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55606a574830 0x55606a583950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36124/0 (socket says 172.21.15.86:36124) 2024-04-21T08:36:27.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.660+0000 7fde11a28640 1 -- 172.21.15.86:0/2239775818 learned_addr learned my addr 172.21.15.86:0/2239775818 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:27.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.660+0000 7fde11227640 1 --2- 172.21.15.86:0/2239775818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55606a5752d0 0x55606a583e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:27.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.660+0000 7fde11227640 1 --2- 172.21.15.86:0/2239775818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55606a573e40 0x55606a581400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:27.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.660+0000 7fde11a28640 1 -- 172.21.15.86:0/2239775818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55606a573e40 msgr2=0x55606a581400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:36:27.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.660+0000 7fde11a28640 1 --2- 172.21.15.86:0/2239775818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55606a573e40 0x55606a581400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.660+0000 7fde11a28640 1 -- 172.21.15.86:0/2239775818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55606a5752d0 msgr2=0x55606a583e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:36:27.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.660+0000 7fde11a28640 1 --2- 172.21.15.86:0/2239775818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55606a5752d0 0x55606a583e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.660+0000 7fde11a28640 1 -- 172.21.15.86:0/2239775818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55606a584620 con 0x55606a574830 2024-04-21T08:36:27.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.660+0000 7fde11227640 1 --2- 172.21.15.86:0/2239775818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55606a573e40 0x55606a581400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:36:27.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.660+0000 7fde11a28640 1 --2- 172.21.15.86:0/2239775818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55606a574830 0x55606a583950 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fddfc009950 tx=0x7fddfc009f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4356ae7d5259b2cd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:27.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.660+0000 7fddf57fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:36:27.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.660+0000 7fddf57fa640 1 -- 172.21.15.86:0/2239775818 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fddfc015d80 con 0x55606a574830 2024-04-21T08:36:27.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.660+0000 7fde16881740 1 -- 172.21.15.86:0/2239775818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde0801a640 con 0x55606a574830 2024-04-21T08:36:27.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.660+0000 7fddf57fa640 1 -- 172.21.15.86:0/2239775818 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fddfc024480 con 0x55606a574830 2024-04-21T08:36:27.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.660+0000 7fddf57fa640 1 -- 172.21.15.86:0/2239775818 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=17) ==== 24+0+0 (secure 0 0 0) 0x7fddfc024680 con 0x55606a574830 2024-04-21T08:36:27.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.660+0000 7fde16881740 10 client.14784 fetch_fsmap learned FSMap version 17 2024-04-21T08:36:27.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.660+0000 7fde16881740 1 -- 172.21.15.86:0/2239775818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=17,monmap=4+}) -- 0x55606a5f0bb0 con 0x55606a574830 2024-04-21T08:36:27.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.660+0000 7fddf57fa640 1 -- 172.21.15.86:0/2239775818 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde0801a640 con 0x55606a574830 2024-04-21T08:36:27.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.661+0000 7fddf57fa640 1 -- 172.21.15.86:0/2239775818 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 17) ==== 8342+0+0 (secure 0 0 0) 0x7fddfc02b970 con 0x55606a574830 2024-04-21T08:36:27.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.661+0000 7fde16881740 10 client.14784 fetch_fsmap finished waiting for FSMap version 17 2024-04-21T08:36:27.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.661+0000 7fde16881740 10 client.14784 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-21T08:36:27.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.661+0000 7fde16881740 20 client.14784 populate_metadata read hostname 'smithi086' 2024-04-21T08:36:27.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.661+0000 7fde16881740 1 --2- 172.21.15.86:0/2239775818 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55606a5f5740 0x55606a5f4450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:27.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.661+0000 7fde16881740 4 client.14784 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:36:27.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.661+0000 7fde16881740 1 -- 172.21.15.86:0/2239775818 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "status"}) -- 0x55606a4e03a0 con 0x55606a5f5740 2024-04-21T08:36:27.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.661+0000 7fde11227640 1 --2- 172.21.15.86:0/2239775818 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55606a5f5740 0x55606a5f4450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:27.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.661+0000 7fddf57fa640 10 client.14784 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:36:27.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.661+0000 7fde11227640 1 --2- 172.21.15.86:0/2239775818 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55606a5f5740 0x55606a5f4450 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c372b70832d55948 server_cookie=321514c4c3b9869e in_seq=0 out_seq=0 2024-04-21T08:36:27.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.661+0000 7fddf57fa640 1 -- 172.21.15.86:0/2239775818 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x55606a4e03a0 con 0x55606a5f5740 2024-04-21T08:36:27.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.661+0000 7fddf57fa640 10 client.14784 handle_command_reply: tid=0 2024-04-21T08:36:27.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.661+0000 7fde16881740 1 client.14784 shutdown 2024-04-21T08:36:27.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.662+0000 7fde16881740 1 -- 172.21.15.86:0/2239775818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55606a574830 msgr2=0x55606a583950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:27.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.662+0000 7fde16881740 1 --2- 172.21.15.86:0/2239775818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55606a574830 0x55606a583950 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fddfc009950 tx=0x7fddfc009f70 comp rx=0 tx=0).stop 2024-04-21T08:36:27.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.662+0000 7fde16881740 1 -- 172.21.15.86:0/2239775818 shutdown_connections 2024-04-21T08:36:27.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.662+0000 7fde16881740 1 --2- 172.21.15.86:0/2239775818 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55606a5f5740 0x55606a5f4450 crc :-1 s=READY pgs=14 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.662+0000 7fde16881740 1 --2- 172.21.15.86:0/2239775818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55606a5752d0 0x55606a583e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.662+0000 7fddf57fa640 0 client.14784 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:36:27.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.662+0000 7fde16881740 1 --2- 172.21.15.86:0/2239775818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55606a574830 0x55606a583950 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.662+0000 7fde16881740 1 --2- 172.21.15.86:0/2239775818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55606a573e40 0x55606a581400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.662+0000 7fde16881740 1 -- 172.21.15.86:0/2239775818 >> 172.21.15.86:0/2239775818 conn(0x55606a56ff40 msgr2=0x55606a589c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:27.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.662+0000 7fde16881740 1 -- 172.21.15.86:0/2239775818 shutdown_connections 2024-04-21T08:36:27.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.662+0000 7fde16881740 1 -- 172.21.15.86:0/2239775818 wait complete. 2024-04-21T08:36:27.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.662+0000 7fde16881740 20 client.14784 trim_cache size 0 max 16384 2024-04-21T08:36:27.668 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:36:27.669 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "f8cec7cc-ffb8-11ee-bc93-c7b262605968", 2024-04-21T08:36:27.669 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-21T08:36:27.669 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24482, 2024-04-21T08:36:27.669 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-21T08:36:27.669 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-21T08:36:27.669 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-21T08:36:27.669 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 6.2346927689999996, 2024-04-21T08:36:27.669 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 17, 2024-04-21T08:36:27.669 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 89, 2024-04-21T08:36:27.669 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 89, 2024-04-21T08:36:27.669 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 13.105252775 2024-04-21T08:36:27.669 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:36:27.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.662+0000 7fde13c8c640 1 -- 172.21.15.86:0/616318246 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fddf80a3c30 msgr2=0x7fddf80a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:27.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.662+0000 7fde13c8c640 1 --2- 172.21.15.86:0/616318246 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fddf80a3c30 0x7fddf80a60f0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fde04032380 tx=0x7fde04089440 comp rx=0 tx=0).stop 2024-04-21T08:36:27.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.662+0000 7fde13c8c640 1 -- 172.21.15.86:0/616318246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde0c1029a0 msgr2=0x7fde0c19f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:27.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.662+0000 7fde13c8c640 1 --2- 172.21.15.86:0/616318246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde0c1029a0 0x7fde0c19f7c0 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7fde0402c700 tx=0x7fde04035d60 comp rx=0 tx=0).stop 2024-04-21T08:36:27.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.662+0000 7fde13c8c640 1 -- 172.21.15.86:0/616318246 shutdown_connections 2024-04-21T08:36:27.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.662+0000 7fde13c8c640 1 --2- 172.21.15.86:0/616318246 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fddf80a3c30 0x7fddf80a60f0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.662+0000 7fde13c8c640 1 --2- 172.21.15.86:0/616318246 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde0c1029a0 0x7fde0c19f7c0 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.662+0000 7fde13c8c640 1 --2- 172.21.15.86:0/616318246 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde0c101fe0 0x7fde0c198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.662+0000 7fde13c8c640 1 --2- 172.21.15.86:0/616318246 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde0c100e90 0x7fde0c198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:27.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.662+0000 7fde13c8c640 1 -- 172.21.15.86:0/616318246 >> 172.21.15.86:0/616318246 conn(0x7fde0c0fc400 msgr2=0x7fde0c105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:27.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.662+0000 7fde13c8c640 1 -- 172.21.15.86:0/616318246 shutdown_connections 2024-04-21T08:36:27.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:27.662+0000 7fde13c8c640 1 -- 172.21.15.86:0/616318246 wait complete. 2024-04-21T08:36:27.672 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-21T08:36:27.672 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-21T08:36:27.673 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph fs ls 2024-04-21T08:36:27.904 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:36:28.213 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:27 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1915534602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:28.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:27 smithi088 ceph-mon[31259]: pgmap v291: 129 pgs: 11 creating+peering, 101 active+clean, 17 unknown; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 3 op/s 2024-04-21T08:36:28.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:27 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/390978677' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:28.240 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:27 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1915534602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:28.240 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:27 smithi086 ceph-mon[27939]: pgmap v291: 129 pgs: 11 creating+peering, 101 active+clean, 17 unknown; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 3 op/s 2024-04-21T08:36:28.241 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:27 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/390978677' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:28.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:27 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1915534602' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:28.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:27 smithi179 ceph-mon[31829]: pgmap v291: 129 pgs: 11 creating+peering, 101 active+clean, 17 unknown; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 2.5 KiB/s wr, 3 op/s 2024-04-21T08:36:28.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:27 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/390978677' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:29.202 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:28 smithi086 ceph-mon[27939]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-21T08:36:29.202 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:28 smithi086 ceph-mon[27939]: Cluster is now healthy 2024-04-21T08:36:29.213 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:28 smithi088 ceph-mon[31259]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-21T08:36:29.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:28 smithi088 ceph-mon[31259]: Cluster is now healthy 2024-04-21T08:36:29.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:28 smithi179 ceph-mon[31829]: Health check cleared: POOL_APP_NOT_ENABLED (was: 1 pool(s) do not have an application enabled) 2024-04-21T08:36:29.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:28 smithi179 ceph-mon[31829]: Cluster is now healthy 2024-04-21T08:36:29.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.632+0000 7f97faa29700 1 -- 172.21.15.86:0/70646669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97f406b190 msgr2=0x7f97f406b5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:29.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.632+0000 7f97faa29700 1 --2- 172.21.15.86:0/70646669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97f406b190 0x7f97f406b5f0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f97e8009990 tx=0x7f97e8009ca0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.632+0000 7f97faa29700 1 -- 172.21.15.86:0/70646669 shutdown_connections 2024-04-21T08:36:29.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.632+0000 7f97faa29700 1 --2- 172.21.15.86:0/70646669 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97f406bb30 0x7f97f410f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.632+0000 7f97faa29700 1 --2- 172.21.15.86:0/70646669 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97f4108a30 0x7f97f4108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.632+0000 7f97faa29700 1 --2- 172.21.15.86:0/70646669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97f406b190 0x7f97f406b5f0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.632+0000 7f97faa29700 1 -- 172.21.15.86:0/70646669 >> 172.21.15.86:0/70646669 conn(0x7f97f4078910 msgr2=0x7f97f4078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:29.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.632+0000 7f97faa29700 1 -- 172.21.15.86:0/70646669 shutdown_connections 2024-04-21T08:36:29.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.632+0000 7f97faa29700 1 -- 172.21.15.86:0/70646669 wait complete. 2024-04-21T08:36:29.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.633+0000 7f97faa29700 1 Processor -- start 2024-04-21T08:36:29.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.633+0000 7f97faa29700 1 -- start start 2024-04-21T08:36:29.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.633+0000 7f97faa29700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97f406b190 0x7f97f41991d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:29.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.633+0000 7f97faa29700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97f406bb30 0x7f97f4199710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:29.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.633+0000 7f97faa29700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97f4108a30 0x7f97f419daa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:29.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.633+0000 7f97faa29700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f97f4112cc0 con 0x7f97f4108a30 2024-04-21T08:36:29.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.633+0000 7f97faa29700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f97f4112b40 con 0x7f97f406b190 2024-04-21T08:36:29.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.633+0000 7f97faa29700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f97f4112e40 con 0x7f97f406bb30 2024-04-21T08:36:29.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.634+0000 7f97f8fc6700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97f4108a30 0x7f97f419daa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:29.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.634+0000 7f97f8fc6700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97f4108a30 0x7f97f419daa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36280/0 (socket says 172.21.15.86:36280) 2024-04-21T08:36:29.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.634+0000 7f97f8fc6700 1 -- 172.21.15.86:0/1246700639 learned_addr learned my addr 172.21.15.86:0/1246700639 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:29.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.634+0000 7f97f37fe700 1 --2- 172.21.15.86:0/1246700639 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97f406bb30 0x7f97f4199710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:29.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.634+0000 7f97f3fff700 1 --2- 172.21.15.86:0/1246700639 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97f406b190 0x7f97f41991d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:29.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.634+0000 7f97f8fc6700 1 -- 172.21.15.86:0/1246700639 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97f406bb30 msgr2=0x7f97f4199710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:29.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.634+0000 7f97f8fc6700 1 --2- 172.21.15.86:0/1246700639 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97f406bb30 0x7f97f4199710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.634+0000 7f97f8fc6700 1 -- 172.21.15.86:0/1246700639 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97f406b190 msgr2=0x7f97f41991d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:29.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.634+0000 7f97f8fc6700 1 --2- 172.21.15.86:0/1246700639 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97f406b190 0x7f97f41991d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.634+0000 7f97f8fc6700 1 -- 172.21.15.86:0/1246700639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97f419e1e0 con 0x7f97f4108a30 2024-04-21T08:36:29.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.634+0000 7f97f8fc6700 1 --2- 172.21.15.86:0/1246700639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97f4108a30 0x7f97f419daa0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f97ec00b970 tx=0x7f97ec00bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:29.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.634+0000 7f97f17fa700 1 -- 172.21.15.86:0/1246700639 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ec019070 con 0x7f97f4108a30 2024-04-21T08:36:29.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.634+0000 7f97faa29700 1 -- 172.21.15.86:0/1246700639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97f419e4d0 con 0x7f97f4108a30 2024-04-21T08:36:29.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.634+0000 7f97f17fa700 1 -- 172.21.15.86:0/1246700639 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97ec00cde0 con 0x7f97f4108a30 2024-04-21T08:36:29.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.634+0000 7f97f17fa700 1 -- 172.21.15.86:0/1246700639 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97ec007370 con 0x7f97f4108a30 2024-04-21T08:36:29.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.634+0000 7f97faa29700 1 -- 172.21.15.86:0/1246700639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97f41a5d50 con 0x7f97f4108a30 2024-04-21T08:36:29.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.635+0000 7f97faa29700 1 -- 172.21.15.86:0/1246700639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f97f4069730 con 0x7f97f4108a30 2024-04-21T08:36:29.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.636+0000 7f97f17fa700 1 -- 172.21.15.86:0/1246700639 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f97ec01c070 con 0x7f97f4108a30 2024-04-21T08:36:29.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.637+0000 7f97f17fa700 1 --2- 172.21.15.86:0/1246700639 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f97dc0a3c40 0x7f97dc0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:29.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.637+0000 7f97f3fff700 1 --2- 172.21.15.86:0/1246700639 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f97dc0a3c40 0x7f97dc0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:29.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.637+0000 7f97f17fa700 1 -- 172.21.15.86:0/1246700639 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f97ec0cfb10 con 0x7f97f4108a30 2024-04-21T08:36:29.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.637+0000 7f97f3fff700 1 --2- 172.21.15.86:0/1246700639 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f97dc0a3c40 0x7f97dc0a6100 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f97e4005950 tx=0x7f97e4005eb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:29.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.641+0000 7f97f17fa700 1 -- 172.21.15.86:0/1246700639 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f97ec09c2b0 con 0x7f97f4108a30 2024-04-21T08:36:29.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.884+0000 7f97faa29700 1 -- 172.21.15.86:0/1246700639 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7f97f402df80 con 0x7f97f4108a30 2024-04-21T08:36:29.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.885+0000 7f97f17fa700 1 -- 172.21.15.86:0/1246700639 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v17) ==== 53+0+88 (secure 0 0 0) 0x7f97ec09bbb0 con 0x7f97f4108a30 2024-04-21T08:36:29.885 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-21T08:36:29.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.887+0000 7f97faa29700 1 -- 172.21.15.86:0/1246700639 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f97dc0a3c40 msgr2=0x7f97dc0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:29.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.887+0000 7f97faa29700 1 --2- 172.21.15.86:0/1246700639 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f97dc0a3c40 0x7f97dc0a6100 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7f97e4005950 tx=0x7f97e4005eb0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.887+0000 7f97faa29700 1 -- 172.21.15.86:0/1246700639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97f4108a30 msgr2=0x7f97f419daa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:29.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.887+0000 7f97faa29700 1 --2- 172.21.15.86:0/1246700639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97f4108a30 0x7f97f419daa0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f97ec00b970 tx=0x7f97ec00bc80 comp rx=0 tx=0).stop 2024-04-21T08:36:29.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.887+0000 7f97faa29700 1 -- 172.21.15.86:0/1246700639 shutdown_connections 2024-04-21T08:36:29.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.887+0000 7f97faa29700 1 --2- 172.21.15.86:0/1246700639 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f97dc0a3c40 0x7f97dc0a6100 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.887+0000 7f97faa29700 1 --2- 172.21.15.86:0/1246700639 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97f4108a30 0x7f97f419daa0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.887+0000 7f97faa29700 1 --2- 172.21.15.86:0/1246700639 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97f406b190 0x7f97f41991d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.887+0000 7f97faa29700 1 --2- 172.21.15.86:0/1246700639 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97f406bb30 0x7f97f4199710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:29.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.887+0000 7f97faa29700 1 -- 172.21.15.86:0/1246700639 >> 172.21.15.86:0/1246700639 conn(0x7f97f4078910 msgr2=0x7f97f40ffa80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:29.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.887+0000 7f97faa29700 1 -- 172.21.15.86:0/1246700639 shutdown_connections 2024-04-21T08:36:29.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:29.887+0000 7f97faa29700 1 -- 172.21.15.86:0/1246700639 wait complete. 2024-04-21T08:36:30.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:29 smithi088 ceph-mon[31259]: pgmap v292: 129 pgs: 10 creating+peering, 119 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 2 op/s 2024-04-21T08:36:30.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:29 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1246700639' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-21T08:36:30.244 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:29 smithi086 ceph-mon[27939]: pgmap v292: 129 pgs: 10 creating+peering, 119 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 2 op/s 2024-04-21T08:36:30.245 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:29 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1246700639' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-21T08:36:30.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:29 smithi179 ceph-mon[31829]: pgmap v292: 129 pgs: 10 creating+peering, 119 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.8 KiB/s wr, 2 op/s 2024-04-21T08:36:30.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:29 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1246700639' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-21T08:36:30.614 INFO:tasks.cephfs.mount:Mounting Ceph FS cephfs; just confirmed its presence on cluster 2024-04-21T08:36:30.614 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-21T08:36:30.614 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-21T08:36:30.614 INFO:tasks.cephfs.mount:self.client.name = client.admin 2024-04-21T08:36:30.614 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.admin 2024-04-21T08:36:30.614 INFO:tasks.cephfs.mount:self.cephfs_name = cephfs 2024-04-21T08:36:30.614 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-21T08:36:30.615 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-21T08:36:30.615 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.admin' 2024-04-21T08:36:30.615 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:36:30.615 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-21T08:36:30.630 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-21T08:36:30.631 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-21T08:36:30.631 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-21T08:36:30.631 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-21T08:36:30.631 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-21T08:36:30.631 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-21T08:36:30.631 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-21T08:36:30.631 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-21T08:36:30.631 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-21T08:36:30.631 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-21T08:36:30.631 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-21T08:36:30.631 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-21T08:36:30.631 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-21T08:36:30.631 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-21T08:36:30.631 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-21T08:36:30.631 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-21T08:36:30.632 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-21T08:36:30.632 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42211sec preferred_lft 42211sec 2024-04-21T08:36:30.632 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ecad:5405:e8c2:c812/64 scope link noprefixroute 2024-04-21T08:36:30.632 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-21T08:36:30.632 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-21T08:36:30.632 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T08:36:30.632 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-21T08:36:30.633 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add name ceph-brx type bridge 2024-04-21T08:36:30.633 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr flush dev ceph-brx 2024-04-21T08:36:30.633 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set ceph-brx up 2024-04-21T08:36:30.633 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-21T08:36:30.633 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-21T08:36:30.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T08:36:30.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:30 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:36:30.812 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:36:30.812 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-21T08:36:30.837 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-21T08:36:30.839 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:36:30.839 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-21T08:36:30.903 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-21T08:36:30.904 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-21T08:36:30.904 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-21T08:36:30.904 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-21T08:36:30.904 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-21T08:36:30.904 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T08:36:30.904 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-21T08:36:30.905 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-21T08:36:30.905 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-21T08:36:30.905 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-21T08:36:30.905 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-21T08:36:30.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:30 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T08:36:31.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:36:31.203 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:36:31.203 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-21T08:36:31.218 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:36:31.218 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list-id 2024-04-21T08:36:31.273 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T08:36:31.274 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-21T08:36:31.274 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-21T08:36:31.274 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.admin 0 2024-04-21T08:36:31.274 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-21T08:36:31.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T08:36:31.383 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:36:31.389 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.admin' with 192.168.0.1/16 2024-04-21T08:36:31.390 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T08:36:31.390 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-21T08:36:31.390 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.admin type veth peer name brx.0 2024-04-21T08:36:31.390 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-21T08:36:31.390 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set veth0 up 2024-04-21T08:36:31.390 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip link set lo up 2024-04-21T08:36:31.390 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.admin ip route add default via 192.168.255.254 2024-04-21T08:36:31.390 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-21T08:36:31.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T08:36:31.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:36:31.509 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T08:36:31.509 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-21T08:36:31.509 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-21T08:36:31.509 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set dev brx.0 master ceph-brx 2024-04-21T08:36:31.509 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-21T08:36:31.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:31 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T08:36:31.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:31 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:36:31.584 INFO:tasks.cephfs.fuse_mount:Client client.admin config is {} 2024-04-21T08:36:31.584 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-21T08:36:31.584 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.admin 2024-04-21T08:36:31.599 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.admin' 2024-04-21T08:36:31.599 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-21T08:36:31.599 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.admin 2024-04-21T08:36:31.653 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-21T08:36:31.718 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-21T08:36:31.773 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-21T08:36:31.773 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-21T08:36:31.774 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-21T08:36:31.774 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-21T08:36:31.774 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-21T08:36:31.774 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-21T08:36:31.774 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-21T08:36:31.774 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-21T08:36:31.774 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-21T08:36:31.774 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-21T08:36:31.774 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-21T08:36:31.774 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-21T08:36:31.774 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-21T08:36:31.774 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-21T08:36:31.775 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-21T08:36:31.775 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-21T08:36:31.775 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-21T08:36:31.775 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-21T08:36:31.775 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2024-04-21T08:36:31.775 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-21T08:36:31.775 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-21T08:36:31.775 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-21T08:36:31.775 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-21T08:36:31.775 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-21T08:36:31.775 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-21T08:36:31.775 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-21T08:36:31.775 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay 2024-04-21T08:36:31.775 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/e00e735beb8948002023d075774cf7ac1f99ec712d86b4a39c92a687d927bf6e/merged 2024-04-21T08:36:31.776 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/34b2fa3cee96150a369891b0e4dd1059fe0cf229b9a42cf92c6156f1b46c4680/merged 2024-04-21T08:36:31.776 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/0 2024-04-21T08:36:31.776 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-21T08:36:31.776 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/4391a3955d3a1e334c3a1bf82bd5f7a3d87409842a8b6f60be9a5b6e2ce5786d/merged 2024-04-21T08:36:31.776 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/0894c811b296cb4ab5cb5a82f4a9b4119c08624dae243f55bdd5bf30f8eaa241/merged 2024-04-21T08:36:31.776 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/496800ad7894f09324f6d30b2466678790c8e9ce9563cfd2ac1ae2a7ef9a6390/merged 2024-04-21T08:36:31.776 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/f6e7e96b7a10576c6101c9cb66a2ae7f87b312db016d17e4a44e64470e7c4f90/merged 2024-04-21T08:36:31.776 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/cc97f79ea9026d54bca6a6e998bcbbcfe6aedc00e04b9ca15d16d22c3ad5a0a9/merged 2024-04-21T08:36:31.776 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/95616b28bf77279c93c2b1a6fa4347ea4613430a4189d2323a4c61ffe512afbe/merged 2024-04-21T08:36:31.776 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/bd7da644af577f438f424da955fba8b6e073f28bd2952920a24e8f69a81f5bab/merged 2024-04-21T08:36:31.776 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/fded4a0ffeb938e056b2a3db157b569417c20245771918b428b6f1fc91a79ce4/merged 2024-04-21T08:36:31.776 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-21T08:36:31.776 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-21T08:36:31.776 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-21T08:36:31.777 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:36:31.777 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-21T08:36:31.828 INFO:tasks.cephfs.fuse_mount:Pre-mount connections: [] 2024-04-21T08:36:31.829 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-fuse -f --admin-socket '/var/run/ceph/$cluster-$name.$pid.asok' /home/ubuntu/cephtest/mnt.admin --id admin --client_fs=cephfs) 2024-04-21T08:36:31.871 DEBUG:teuthology.orchestra.run.smithi086:> sudo modprobe fuse 2024-04-21T08:36:31.894 DEBUG:teuthology.orchestra.run.smithi086:> cat /proc/self/mounts | awk '{print $2}' 2024-04-21T08:36:31.949 INFO:teuthology.orchestra.run.smithi086.stdout:/proc 2024-04-21T08:36:31.949 INFO:teuthology.orchestra.run.smithi086.stdout:/sys 2024-04-21T08:36:31.949 INFO:teuthology.orchestra.run.smithi086.stdout:/dev 2024-04-21T08:36:31.949 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/security 2024-04-21T08:36:31.949 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/shm 2024-04-21T08:36:31.949 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/pts 2024-04-21T08:36:31.950 INFO:teuthology.orchestra.run.smithi086.stdout:/run 2024-04-21T08:36:31.950 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/cgroup 2024-04-21T08:36:31.950 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/pstore 2024-04-21T08:36:31.950 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/bpf 2024-04-21T08:36:31.950 INFO:teuthology.orchestra.run.smithi086.stdout:/ 2024-04-21T08:36:31.950 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/selinux 2024-04-21T08:36:31.950 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-21T08:36:31.950 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/hugepages 2024-04-21T08:36:31.950 INFO:teuthology.orchestra.run.smithi086.stdout:/dev/mqueue 2024-04-21T08:36:31.950 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/debug 2024-04-21T08:36:31.950 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/tracing 2024-04-21T08:36:31.950 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/kernel/config 2024-04-21T08:36:31.950 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysusers.service 2024-04-21T08:36:31.951 INFO:teuthology.orchestra.run.smithi086.stdout:/sys/fs/fuse/connections 2024-04-21T08:36:31.951 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-sysctl.service 2024-04-21T08:36:31.951 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup-dev.service 2024-04-21T08:36:31.951 INFO:teuthology.orchestra.run.smithi086.stdout:/run/credentials/systemd-tmpfiles-setup.service 2024-04-21T08:36:31.951 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/nfs/rpc_pipefs 2024-04-21T08:36:31.951 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/1000 2024-04-21T08:36:31.951 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/ceph 2024-04-21T08:36:31.951 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay 2024-04-21T08:36:31.951 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/e00e735beb8948002023d075774cf7ac1f99ec712d86b4a39c92a687d927bf6e/merged 2024-04-21T08:36:31.951 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/34b2fa3cee96150a369891b0e4dd1059fe0cf229b9a42cf92c6156f1b46c4680/merged 2024-04-21T08:36:31.951 INFO:teuthology.orchestra.run.smithi086.stdout:/run/user/0 2024-04-21T08:36:31.951 INFO:teuthology.orchestra.run.smithi086.stdout:/proc/sys/fs/binfmt_misc 2024-04-21T08:36:31.951 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/4391a3955d3a1e334c3a1bf82bd5f7a3d87409842a8b6f60be9a5b6e2ce5786d/merged 2024-04-21T08:36:31.951 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/0894c811b296cb4ab5cb5a82f4a9b4119c08624dae243f55bdd5bf30f8eaa241/merged 2024-04-21T08:36:31.952 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/496800ad7894f09324f6d30b2466678790c8e9ce9563cfd2ac1ae2a7ef9a6390/merged 2024-04-21T08:36:31.952 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/f6e7e96b7a10576c6101c9cb66a2ae7f87b312db016d17e4a44e64470e7c4f90/merged 2024-04-21T08:36:31.952 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/cc97f79ea9026d54bca6a6e998bcbbcfe6aedc00e04b9ca15d16d22c3ad5a0a9/merged 2024-04-21T08:36:31.952 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/95616b28bf77279c93c2b1a6fa4347ea4613430a4189d2323a4c61ffe512afbe/merged 2024-04-21T08:36:31.952 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/bd7da644af577f438f424da955fba8b6e073f28bd2952920a24e8f69a81f5bab/merged 2024-04-21T08:36:31.952 INFO:teuthology.orchestra.run.smithi086.stdout:/var/lib/containers/storage/overlay/fded4a0ffeb938e056b2a3db157b569417c20245771918b428b6f1fc91a79ce4/merged 2024-04-21T08:36:31.952 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns 2024-04-21T08:36:31.952 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-21T08:36:31.952 INFO:teuthology.orchestra.run.smithi086.stdout:/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-21T08:36:31.952 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:36:31.952 DEBUG:teuthology.orchestra.run.smithi086:> ls /sys/fs/fuse/connections 2024-04-21T08:36:31.958 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi086.stderr:2024-04-21T08:36:31.958+0000 7fd271846480 -1 init, newargv = 0x559c61e23fa0 newargc=15 2024-04-21T08:36:31.958 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi086.stderr:ceph-fuse[69339]: starting ceph client 2024-04-21T08:36:31.964 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi086.stderr:ceph-fuse[69339]: starting fuse 2024-04-21T08:36:31.973 INFO:teuthology.orchestra.run.smithi086.stdout:115 2024-04-21T08:36:31.973 INFO:tasks.cephfs.fuse_mount:Post-mount connections: [115] 2024-04-21T08:36:31.974 DEBUG:teuthology.orchestra.run.smithi086:> sudo stdin-killer -- python3 -c ' 2024-04-21T08:36:31.974 DEBUG:teuthology.orchestra.run.smithi086:> import glob 2024-04-21T08:36:31.974 DEBUG:teuthology.orchestra.run.smithi086:> import re 2024-04-21T08:36:31.974 DEBUG:teuthology.orchestra.run.smithi086:> import os 2024-04-21T08:36:31.974 DEBUG:teuthology.orchestra.run.smithi086:> import subprocess 2024-04-21T08:36:31.974 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-21T08:36:31.974 DEBUG:teuthology.orchestra.run.smithi086:> def _find_admin_socket(client_name): 2024-04-21T08:36:31.974 DEBUG:teuthology.orchestra.run.smithi086:> asok_path = "/var/run/ceph/ceph-client.admin.*.asok" 2024-04-21T08:36:31.974 DEBUG:teuthology.orchestra.run.smithi086:> files = glob.glob(asok_path) 2024-04-21T08:36:31.974 DEBUG:teuthology.orchestra.run.smithi086:> mountpoint = "/home/ubuntu/cephtest/mnt.admin" 2024-04-21T08:36:31.975 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-21T08:36:31.975 DEBUG:teuthology.orchestra.run.smithi086:> # Given a non-glob path, it better be there 2024-04-21T08:36:31.975 DEBUG:teuthology.orchestra.run.smithi086:> if "*" not in asok_path: 2024-04-21T08:36:31.975 DEBUG:teuthology.orchestra.run.smithi086:> assert(len(files) == 1) 2024-04-21T08:36:31.975 DEBUG:teuthology.orchestra.run.smithi086:> return files[0] 2024-04-21T08:36:31.975 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-21T08:36:31.975 DEBUG:teuthology.orchestra.run.smithi086:> for f in files: 2024-04-21T08:36:31.975 DEBUG:teuthology.orchestra.run.smithi086:> pid = re.match(".*\.(\d+)\.asok$", f).group(1) 2024-04-21T08:36:31.975 DEBUG:teuthology.orchestra.run.smithi086:> if os.path.exists("/proc/{0}".format(pid)): 2024-04-21T08:36:31.975 DEBUG:teuthology.orchestra.run.smithi086:> with open("/proc/{0}/cmdline".format(pid), '"'"'r'"'"') as proc_f: 2024-04-21T08:36:31.975 DEBUG:teuthology.orchestra.run.smithi086:> contents = proc_f.read() 2024-04-21T08:36:31.975 DEBUG:teuthology.orchestra.run.smithi086:> if mountpoint in contents: 2024-04-21T08:36:31.975 DEBUG:teuthology.orchestra.run.smithi086:> return f 2024-04-21T08:36:31.976 DEBUG:teuthology.orchestra.run.smithi086:> raise RuntimeError("Client socket {0} not found".format(client_name)) 2024-04-21T08:36:31.976 DEBUG:teuthology.orchestra.run.smithi086:> 2024-04-21T08:36:31.976 DEBUG:teuthology.orchestra.run.smithi086:> print(_find_admin_socket("client.admin")) 2024-04-21T08:36:31.976 DEBUG:teuthology.orchestra.run.smithi086:> ' 2024-04-21T08:36:32.090 INFO:teuthology.orchestra.run.smithi086.stdout:/var/run/ceph/ceph-client.admin.69339.asok 2024-04-21T08:36:32.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:36:32.100 INFO:tasks.cephfs.fuse_mount:Found client admin socket at /var/run/ceph/ceph-client.admin.69339.asok 2024-04-21T08:36:32.100 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:36:32.100 DEBUG:teuthology.orchestra.run.smithi086:> sudo ceph --admin-daemon /var/run/ceph/ceph-client.admin.69339.asok status 2024-04-21T08:36:32.232 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:36:32.232 INFO:teuthology.orchestra.run.smithi086.stdout: "metadata": { 2024-04-21T08:36:32.232 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_sha1": "a630765c1f93c6e61d9b40c4b76437611d5ef29e", 2024-04-21T08:36:32.232 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_version": "ceph version 19.0.0-3154-ga630765c (a630765c1f93c6e61d9b40c4b76437611d5ef29e) squid (dev)", 2024-04-21T08:36:32.232 INFO:teuthology.orchestra.run.smithi086.stdout: "entity_id": "admin", 2024-04-21T08:36:32.232 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": "smithi086", 2024-04-21T08:36:32.232 INFO:teuthology.orchestra.run.smithi086.stdout: "mount_point": "/home/ubuntu/cephtest/mnt.admin", 2024-04-21T08:36:32.232 INFO:teuthology.orchestra.run.smithi086.stdout: "pid": "69339", 2024-04-21T08:36:32.232 INFO:teuthology.orchestra.run.smithi086.stdout: "root": "/" 2024-04-21T08:36:32.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:36:32.232 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_count": 0, 2024-04-21T08:36:32.233 INFO:teuthology.orchestra.run.smithi086.stdout: "dentry_pinned_count": 0, 2024-04-21T08:36:32.233 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 14796, 2024-04-21T08:36:32.233 INFO:teuthology.orchestra.run.smithi086.stdout: "inst": { 2024-04-21T08:36:32.233 INFO:teuthology.orchestra.run.smithi086.stdout: "name": { 2024-04-21T08:36:32.233 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "client", 2024-04-21T08:36:32.233 INFO:teuthology.orchestra.run.smithi086.stdout: "num": 14796 2024-04-21T08:36:32.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:36:32.233 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-21T08:36:32.233 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-21T08:36:32.233 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-21T08:36:32.233 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 3569277034 2024-04-21T08:36:32.233 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:36:32.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:36:32.233 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": { 2024-04-21T08:36:32.234 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-21T08:36:32.234 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "192.168.0.1:0", 2024-04-21T08:36:32.234 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 3569277034 2024-04-21T08:36:32.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:36:32.234 INFO:teuthology.orchestra.run.smithi086.stdout: "inst_str": "client.14796 192.168.0.1:0/3569277034", 2024-04-21T08:36:32.234 INFO:teuthology.orchestra.run.smithi086.stdout: "addr_str": "192.168.0.1:0/3569277034", 2024-04-21T08:36:32.234 INFO:teuthology.orchestra.run.smithi086.stdout: "inode_count": 1, 2024-04-21T08:36:32.234 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_epoch": 17, 2024-04-21T08:36:32.234 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch": 89, 2024-04-21T08:36:32.234 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_epoch_barrier": 0, 2024-04-21T08:36:32.234 INFO:teuthology.orchestra.run.smithi086.stdout: "blocklisted": false, 2024-04-21T08:36:32.234 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs" 2024-04-21T08:36:32.234 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:36:32.240 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:36:32.240 DEBUG:teuthology.orchestra.run.smithi086:> stat --file-system '--printf=%T 2024-04-21T08:36:32.240 DEBUG:teuthology.orchestra.run.smithi086:> ' -- /home/ubuntu/cephtest/mnt.admin 2024-04-21T08:36:32.255 INFO:teuthology.orchestra.run.smithi086.stdout:fuseblk 2024-04-21T08:36:32.256 INFO:tasks.cephfs.fuse_mount:ceph-fuse is mounted on /home/ubuntu/cephtest/mnt.admin 2024-04-21T08:36:32.256 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:36:32.256 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.admin 2024-04-21T08:36:32.320 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.admin && exec stdin-killer --timeout=900 -- bash -c 'setfattr -n ceph.dir.layout.pool -v cephfs_data_ec . && getfattr -n ceph.dir.layout .') 2024-04-21T08:36:32.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-21T08:36:32.427 INFO:teuthology.orchestra.run.smithi086.stdout:# file: . 2024-04-21T08:36:32.428 INFO:teuthology.orchestra.run.smithi086.stdout:ceph.dir.layout="stripe_unit=4194304 stripe_count=1 object_size=4194304 pool=cephfs_data_ec" 2024-04-21T08:36:32.428 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:36:32.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:36:32.433 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-21T08:36:32.433 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-21T08:36:32.450 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:36:32.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.597+0000 7fed481fd640 1 -- 172.21.15.86:0/887801804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed401053d0 msgr2=0x7fed401077c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:32.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.597+0000 7fed481fd640 1 --2- 172.21.15.86:0/887801804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed401053d0 0x7fed401077c0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fed3c009900 tx=0x7fed3c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:32.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.597+0000 7fed481fd640 1 -- 172.21.15.86:0/887801804 shutdown_connections 2024-04-21T08:36:32.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.597+0000 7fed481fd640 1 --2- 172.21.15.86:0/887801804 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed40107d00 0x7fed4010a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:32.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.597+0000 7fed481fd640 1 --2- 172.21.15.86:0/887801804 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed401053d0 0x7fed401077c0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:32.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.597+0000 7fed481fd640 1 --2- 172.21.15.86:0/887801804 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed40102aa0 0x7fed40104e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:32.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.597+0000 7fed481fd640 1 -- 172.21.15.86:0/887801804 >> 172.21.15.86:0/887801804 conn(0x7fed400fc470 msgr2=0x7fed400fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:32.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.597+0000 7fed481fd640 1 -- 172.21.15.86:0/887801804 shutdown_connections 2024-04-21T08:36:32.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.597+0000 7fed481fd640 1 -- 172.21.15.86:0/887801804 wait complete. 2024-04-21T08:36:32.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.597+0000 7fed481fd640 1 Processor -- start 2024-04-21T08:36:32.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.597+0000 7fed481fd640 1 -- start start 2024-04-21T08:36:32.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.597+0000 7fed481fd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed40102aa0 0x7fed4019a460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:32.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.597+0000 7fed481fd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed401053d0 0x7fed4019a9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:32.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.597+0000 7fed481fd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed40107d00 0x7fed401a1a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:32.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.597+0000 7fed481fd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fed4010db40 con 0x7fed40107d00 2024-04-21T08:36:32.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.597+0000 7fed481fd640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fed4010d9c0 con 0x7fed40102aa0 2024-04-21T08:36:32.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.597+0000 7fed481fd640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fed4010dcc0 con 0x7fed401053d0 2024-04-21T08:36:32.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.598+0000 7fed4679a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed40107d00 0x7fed401a1a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:32.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.598+0000 7fed4679a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed40107d00 0x7fed401a1a10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36296/0 (socket says 172.21.15.86:36296) 2024-04-21T08:36:32.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.598+0000 7fed4679a640 1 -- 172.21.15.86:0/1338139594 learned_addr learned my addr 172.21.15.86:0/1338139594 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:32.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.598+0000 7fed45f99640 1 --2- 172.21.15.86:0/1338139594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed40102aa0 0x7fed4019a460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:32.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.598+0000 7fed4679a640 1 -- 172.21.15.86:0/1338139594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed401053d0 msgr2=0x7fed4019a9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:32.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.598+0000 7fed4679a640 1 --2- 172.21.15.86:0/1338139594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed401053d0 0x7fed4019a9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:32.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.598+0000 7fed4679a640 1 -- 172.21.15.86:0/1338139594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed40102aa0 msgr2=0x7fed4019a460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:32.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.598+0000 7fed4679a640 1 --2- 172.21.15.86:0/1338139594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed40102aa0 0x7fed4019a460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:32.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.598+0000 7fed4679a640 1 -- 172.21.15.86:0/1338139594 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed401a2080 con 0x7fed40107d00 2024-04-21T08:36:32.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.598+0000 7fed4679a640 1 --2- 172.21.15.86:0/1338139594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed40107d00 0x7fed401a1a10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fed3800b970 tx=0x7fed3800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:32.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.598+0000 7fed36ffd640 1 -- 172.21.15.86:0/1338139594 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed38019070 con 0x7fed40107d00 2024-04-21T08:36:32.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.598+0000 7fed36ffd640 1 -- 172.21.15.86:0/1338139594 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed38014440 con 0x7fed40107d00 2024-04-21T08:36:32.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.598+0000 7fed481fd640 1 -- 172.21.15.86:0/1338139594 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed401a2370 con 0x7fed40107d00 2024-04-21T08:36:32.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.598+0000 7fed36ffd640 1 -- 172.21.15.86:0/1338139594 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed380075d0 con 0x7fed40107d00 2024-04-21T08:36:32.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.598+0000 7fed481fd640 1 -- 172.21.15.86:0/1338139594 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed401a2940 con 0x7fed40107d00 2024-04-21T08:36:32.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.599+0000 7fed481fd640 1 -- 172.21.15.86:0/1338139594 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fed401097f0 con 0x7fed40107d00 2024-04-21T08:36:32.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.600+0000 7fed36ffd640 1 -- 172.21.15.86:0/1338139594 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fed3801c070 con 0x7fed40107d00 2024-04-21T08:36:32.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.601+0000 7fed36ffd640 1 --2- 172.21.15.86:0/1338139594 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fed2c0a3bb0 0x7fed2c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:32.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.601+0000 7fed45f99640 1 --2- 172.21.15.86:0/1338139594 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fed2c0a3bb0 0x7fed2c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:32.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.601+0000 7fed36ffd640 1 -- 172.21.15.86:0/1338139594 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fed380cfb90 con 0x7fed40107d00 2024-04-21T08:36:32.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.601+0000 7fed45f99640 1 --2- 172.21.15.86:0/1338139594 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fed2c0a3bb0 0x7fed2c0a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fed300058c0 tx=0x7fed3000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:32.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.605+0000 7fed36ffd640 1 -- 172.21.15.86:0/1338139594 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fed3809bce0 con 0x7fed40107d00 2024-04-21T08:36:32.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:32 smithi086 ceph-mon[27939]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 1 op/s 2024-04-21T08:36:32.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:32 smithi088 ceph-mon[31259]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 1 op/s 2024-04-21T08:36:32.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:32 smithi179 ceph-mon[31829]: pgmap v293: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.5 KiB/s wr, 1 op/s 2024-04-21T08:36:32.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.787+0000 7fed481fd640 1 -- 172.21.15.86:0/1338139594 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fed4002d5c0 con 0x7fed40107d00 2024-04-21T08:36:32.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.789+0000 7fed36ffd640 1 -- 172.21.15.86:0/1338139594 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24070 (secure 0 0 0) 0x7fed3809ffe0 con 0x7fed40107d00 2024-04-21T08:36:32.789 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:36:32.789 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":89,"fsid":"f8cec7cc-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:22.687329+0000","modified":"2024-04-21T08:36:25.251657+0000","last_up_change":"2024-04-21T08:35:30.048315+0000","last_in_change":"2024-04-21T08:35:10.099496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:45.279109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:18.300782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:19.574636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:22.905632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89b7601f-0620-4db2-90f1-b45cea173738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6803","nonce":2571516897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6805","nonce":2571516897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6809","nonce":2571516897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6807","nonce":2571516897}]},"public_addr":"172.21.15.86:6803/2571516897","cluster_addr":"172.21.15.86:6805/2571516897","heartbeat_back_addr":"172.21.15.86:6809/2571516897","heartbeat_front_addr":"172.21.15.86:6807/2571516897","state":["exists","up"]},{"osd":1,"uuid":"4891031c-f98c-42bc-bb3b-a420855ecc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6801","nonce":74067182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6803","nonce":74067182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6807","nonce":74067182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6805","nonce":74067182}]},"public_addr":"172.21.15.88:6801/74067182","cluster_addr":"172.21.15.88:6803/74067182","heartbeat_back_addr":"172.21.15.88:6807/74067182","heartbeat_front_addr":"172.21.15.88:6805/74067182","state":["exists","up"]},{"osd":2,"uuid":"60d12aa4-9312-4fa2-8d0b-2b3f42caad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6801","nonce":4137659051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6803","nonce":4137659051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6807","nonce":4137659051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6805","nonce":4137659051}]},"public_addr":"172.21.15.179:6801/4137659051","cluster_addr":"172.21.15.179:6803/4137659051","heartbeat_back_addr":"172.21.15.179:6807/4137659051","heartbeat_front_addr":"172.21.15.179:6805/4137659051","state":["exists","up"]},{"osd":3,"uuid":"404c3158-8cb0-4c02-b48a-f9ee5ff7b784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6811","nonce":2309603148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6813","nonce":2309603148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6817","nonce":2309603148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6815","nonce":2309603148}]},"public_addr":"172.21.15.86:6811/2309603148","cluster_addr":"172.21.15.86:6813/2309603148","heartbeat_back_addr":"172.21.15.86:6817/2309603148","heartbeat_front_addr":"172.21.15.86:6815/2309603148","state":["exists","up"]},{"osd":4,"uuid":"458ff202-d781-4ec7-ac93-0b49f835b503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6809","nonce":1818714217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6811","nonce":1818714217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6815","nonce":1818714217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6813","nonce":1818714217}]},"public_addr":"172.21.15.88:6809/1818714217","cluster_addr":"172.21.15.88:6811/1818714217","heartbeat_back_addr":"172.21.15.88:6815/1818714217","heartbeat_front_addr":"172.21.15.88:6813/1818714217","state":["exists","up"]},{"osd":5,"uuid":"8123cc90-bd8b-4204-bf31-46b34e9f589d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6809","nonce":863840990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6811","nonce":863840990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6815","nonce":863840990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6813","nonce":863840990}]},"public_addr":"172.21.15.179:6809/863840990","cluster_addr":"172.21.15.179:6811/863840990","heartbeat_back_addr":"172.21.15.179:6815/863840990","heartbeat_front_addr":"172.21.15.179:6813/863840990","state":["exists","up"]},{"osd":6,"uuid":"aac9d7c7-659a-4110-903a-a871c086066f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6819","nonce":1224231118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6821","nonce":1224231118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6825","nonce":1224231118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6823","nonce":1224231118}]},"public_addr":"172.21.15.86:6819/1224231118","cluster_addr":"172.21.15.86:6821/1224231118","heartbeat_back_addr":"172.21.15.86:6825/1224231118","heartbeat_front_addr":"172.21.15.86:6823/1224231118","state":["exists","up"]},{"osd":7,"uuid":"df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6817","nonce":136629878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6819","nonce":136629878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6823","nonce":136629878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6821","nonce":136629878}]},"public_addr":"172.21.15.88:6817/136629878","cluster_addr":"172.21.15.88:6819/136629878","heartbeat_back_addr":"172.21.15.88:6823/136629878","heartbeat_front_addr":"172.21.15.88:6821/136629878","state":["exists","up"]},{"osd":8,"uuid":"63e59f4b-0426-46db-92f7-c55dae61b5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6817","nonce":1770418957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6819","nonce":1770418957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6823","nonce":1770418957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6821","nonce":1770418957}]},"public_addr":"172.21.15.179:6817/1770418957","cluster_addr":"172.21.15.179:6819/1770418957","heartbeat_back_addr":"172.21.15.179:6823/1770418957","heartbeat_front_addr":"172.21.15.179:6821/1770418957","state":["exists","up"]},{"osd":9,"uuid":"5841bb36-ed6b-45d8-b857-eeab139c44e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6827","nonce":1108500274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6829","nonce":1108500274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6833","nonce":1108500274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6831","nonce":1108500274}]},"public_addr":"172.21.15.86:6827/1108500274","cluster_addr":"172.21.15.86:6829/1108500274","heartbeat_back_addr":"172.21.15.86:6833/1108500274","heartbeat_front_addr":"172.21.15.86:6831/1108500274","state":["exists","up"]},{"osd":10,"uuid":"f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6825","nonce":1294275353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6827","nonce":1294275353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6831","nonce":1294275353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6829","nonce":1294275353}]},"public_addr":"172.21.15.88:6825/1294275353","cluster_addr":"172.21.15.88:6827/1294275353","heartbeat_back_addr":"172.21.15.88:6831/1294275353","heartbeat_front_addr":"172.21.15.88:6829/1294275353","state":["exists","up"]},{"osd":11,"uuid":"e0d4cc76-e7a6-4a6b-909c-95488ccd5a36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6825","nonce":4007070233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6827","nonce":4007070233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6831","nonce":4007070233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6829","nonce":4007070233}]},"public_addr":"172.21.15.179:6825/4007070233","cluster_addr":"172.21.15.179:6827/4007070233","heartbeat_back_addr":"172.21.15.179:6831/4007070233","heartbeat_front_addr":"172.21.15.179:6829/4007070233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:55.210605+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:18.723090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:42.547979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:06.421622+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:30.858315+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:55.455313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.623369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:46.185111+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.103627+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:37.297545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:02.637431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:28.199292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6800/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/1412393225":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6800/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/1687852404":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2020707404":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2568130933":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/3952607648":"2024-04-22T08:28:57.384403+0000","172.21.15.86:6801/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/2756903426":"2024-04-22T08:29:23.190001+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:36:32.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.791+0000 7fed481fd640 1 -- 172.21.15.86:0/1338139594 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fed2c0a3bb0 msgr2=0x7fed2c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:32.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.791+0000 7fed481fd640 1 --2- 172.21.15.86:0/1338139594 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fed2c0a3bb0 0x7fed2c0a6070 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7fed300058c0 tx=0x7fed3000a2d0 comp rx=0 tx=0).stop 2024-04-21T08:36:32.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.791+0000 7fed481fd640 1 -- 172.21.15.86:0/1338139594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed40107d00 msgr2=0x7fed401a1a10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:32.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.791+0000 7fed481fd640 1 --2- 172.21.15.86:0/1338139594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed40107d00 0x7fed401a1a10 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fed3800b970 tx=0x7fed3800be30 comp rx=0 tx=0).stop 2024-04-21T08:36:32.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.791+0000 7fed481fd640 1 -- 172.21.15.86:0/1338139594 shutdown_connections 2024-04-21T08:36:32.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.791+0000 7fed481fd640 1 --2- 172.21.15.86:0/1338139594 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fed2c0a3bb0 0x7fed2c0a6070 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:32.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.791+0000 7fed481fd640 1 --2- 172.21.15.86:0/1338139594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed40107d00 0x7fed401a1a10 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:32.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.791+0000 7fed481fd640 1 --2- 172.21.15.86:0/1338139594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed401053d0 0x7fed4019a9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:32.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.791+0000 7fed481fd640 1 --2- 172.21.15.86:0/1338139594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed40102aa0 0x7fed4019a460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:32.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.791+0000 7fed481fd640 1 -- 172.21.15.86:0/1338139594 >> 172.21.15.86:0/1338139594 conn(0x7fed400fc470 msgr2=0x7fed40102210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:32.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.791+0000 7fed481fd640 1 -- 172.21.15.86:0/1338139594 shutdown_connections 2024-04-21T08:36:32.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.791+0000 7fed481fd640 1 -- 172.21.15.86:0/1338139594 wait complete. 2024-04-21T08:36:32.802 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:31:45.279109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:36:18.300782+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:36:19.574636+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:36:22.905632+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:36:32.803 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:36:32.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.903+0000 7fdf2d10e640 1 -- 172.21.15.86:0/677793066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf28107c70 msgr2=0x7fdf2810a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:32.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.903+0000 7fdf2d10e640 1 --2- 172.21.15.86:0/677793066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf28107c70 0x7fdf2810a0c0 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7fdf20009ca0 tx=0x7fdf2002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:36:32.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.903+0000 7fdf2d10e640 1 -- 172.21.15.86:0/677793066 shutdown_connections 2024-04-21T08:36:32.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.903+0000 7fdf2d10e640 1 --2- 172.21.15.86:0/677793066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf28107c70 0x7fdf2810a0c0 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:32.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.903+0000 7fdf2d10e640 1 --2- 172.21.15.86:0/677793066 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf28105340 0x7fdf28107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:32.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.903+0000 7fdf2d10e640 1 --2- 172.21.15.86:0/677793066 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf28102a10 0x7fdf28104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:32.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.903+0000 7fdf2d10e640 1 -- 172.21.15.86:0/677793066 >> 172.21.15.86:0/677793066 conn(0x7fdf280fc400 msgr2=0x7fdf280fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:32.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.903+0000 7fdf2d10e640 1 -- 172.21.15.86:0/677793066 shutdown_connections 2024-04-21T08:36:32.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.903+0000 7fdf2d10e640 1 -- 172.21.15.86:0/677793066 wait complete. 2024-04-21T08:36:32.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.904+0000 7fdf2d10e640 1 Processor -- start 2024-04-21T08:36:32.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.904+0000 7fdf2d10e640 1 -- start start 2024-04-21T08:36:32.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.904+0000 7fdf2d10e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf28102a10 0x7fdf281981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:32.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.904+0000 7fdf2d10e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf28105340 0x7fdf281986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:32.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.904+0000 7fdf2d10e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf28107c70 0x7fdf2819f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:32.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.904+0000 7fdf2d10e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2810dae0 con 0x7fdf28105340 2024-04-21T08:36:32.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.904+0000 7fdf2d10e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2810d960 con 0x7fdf28107c70 2024-04-21T08:36:32.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.904+0000 7fdf2d10e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdf2810dc60 con 0x7fdf28102a10 2024-04-21T08:36:32.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.905+0000 7fdf2659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf28105340 0x7fdf281986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:32.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.905+0000 7fdf2659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf28105340 0x7fdf281986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36314/0 (socket says 172.21.15.86:36314) 2024-04-21T08:36:32.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.905+0000 7fdf2659c640 1 -- 172.21.15.86:0/940629868 learned_addr learned my addr 172.21.15.86:0/940629868 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:32.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.905+0000 7fdf26d9d640 1 --2- 172.21.15.86:0/940629868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf28102a10 0x7fdf281981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:32.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.905+0000 7fdf2759e640 1 --2- 172.21.15.86:0/940629868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf28107c70 0x7fdf2819f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:32.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.905+0000 7fdf2659c640 1 -- 172.21.15.86:0/940629868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf28102a10 msgr2=0x7fdf281981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:32.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.905+0000 7fdf2659c640 1 --2- 172.21.15.86:0/940629868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf28102a10 0x7fdf281981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:32.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.905+0000 7fdf2659c640 1 -- 172.21.15.86:0/940629868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf28107c70 msgr2=0x7fdf2819f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:32.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.905+0000 7fdf2659c640 1 --2- 172.21.15.86:0/940629868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf28107c70 0x7fdf2819f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:32.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.905+0000 7fdf2659c640 1 -- 172.21.15.86:0/940629868 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf2819fdd0 con 0x7fdf28105340 2024-04-21T08:36:32.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.905+0000 7fdf2759e640 1 --2- 172.21.15.86:0/940629868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf28107c70 0x7fdf2819f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:36:32.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.905+0000 7fdf2659c640 1 --2- 172.21.15.86:0/940629868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf28105340 0x7fdf281986f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800e9c0 tx=0x7fdf1800ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:32.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.905+0000 7fdf0ffff640 1 -- 172.21.15.86:0/940629868 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf1800cd80 con 0x7fdf28105340 2024-04-21T08:36:32.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.905+0000 7fdf2d10e640 1 -- 172.21.15.86:0/940629868 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf281a00c0 con 0x7fdf28105340 2024-04-21T08:36:32.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.905+0000 7fdf0ffff640 1 -- 172.21.15.86:0/940629868 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf180054d0 con 0x7fdf28105340 2024-04-21T08:36:32.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.905+0000 7fdf0ffff640 1 -- 172.21.15.86:0/940629868 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf18010430 con 0x7fdf28105340 2024-04-21T08:36:32.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.905+0000 7fdf2d10e640 1 -- 172.21.15.86:0/940629868 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf281a0660 con 0x7fdf28105340 2024-04-21T08:36:32.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.906+0000 7fdf2d10e640 1 -- 172.21.15.86:0/940629868 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdf280fdef0 con 0x7fdf28105340 2024-04-21T08:36:32.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.907+0000 7fdf0ffff640 1 -- 172.21.15.86:0/940629868 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdf18004fb0 con 0x7fdf28105340 2024-04-21T08:36:32.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.908+0000 7fdf0ffff640 1 --2- 172.21.15.86:0/940629868 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fdf100a3bb0 0x7fdf100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:32.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.908+0000 7fdf0ffff640 1 -- 172.21.15.86:0/940629868 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fdf180d0b00 con 0x7fdf28105340 2024-04-21T08:36:32.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.908+0000 7fdf26d9d640 1 --2- 172.21.15.86:0/940629868 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fdf100a3bb0 0x7fdf100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:32.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.908+0000 7fdf26d9d640 1 --2- 172.21.15.86:0/940629868 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fdf100a3bb0 0x7fdf100a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009bd0 tx=0x7fdf1c005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:32.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:32.913+0000 7fdf0ffff640 1 -- 172.21.15.86:0/940629868 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdf1809d2a0 con 0x7fdf28105340 2024-04-21T08:36:33.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.092+0000 7fdf2d10e640 1 -- 172.21.15.86:0/940629868 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fdf28062b50 con 0x7fdf28105340 2024-04-21T08:36:33.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.092+0000 7fdf0ffff640 1 -- 172.21.15.86:0/940629868 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7fdf1809cba0 con 0x7fdf28105340 2024-04-21T08:36:33.093 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-21T08:36:33.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.094+0000 7fdf2d10e640 1 -- 172.21.15.86:0/940629868 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fdf100a3bb0 msgr2=0x7fdf100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:33.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.094+0000 7fdf2d10e640 1 --2- 172.21.15.86:0/940629868 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fdf100a3bb0 0x7fdf100a6070 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fdf1c009bd0 tx=0x7fdf1c005f90 comp rx=0 tx=0).stop 2024-04-21T08:36:33.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.094+0000 7fdf2d10e640 1 -- 172.21.15.86:0/940629868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf28105340 msgr2=0x7fdf281986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:33.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.094+0000 7fdf2d10e640 1 --2- 172.21.15.86:0/940629868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf28105340 0x7fdf281986f0 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fdf1800e9c0 tx=0x7fdf1800ee80 comp rx=0 tx=0).stop 2024-04-21T08:36:33.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.094+0000 7fdf2d10e640 1 -- 172.21.15.86:0/940629868 shutdown_connections 2024-04-21T08:36:33.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.094+0000 7fdf2d10e640 1 --2- 172.21.15.86:0/940629868 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fdf100a3bb0 0x7fdf100a6070 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.094+0000 7fdf2d10e640 1 --2- 172.21.15.86:0/940629868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf28107c70 0x7fdf2819f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.094+0000 7fdf2d10e640 1 --2- 172.21.15.86:0/940629868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf28105340 0x7fdf281986f0 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.094+0000 7fdf2d10e640 1 --2- 172.21.15.86:0/940629868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf28102a10 0x7fdf281981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.094+0000 7fdf2d10e640 1 -- 172.21.15.86:0/940629868 >> 172.21.15.86:0/940629868 conn(0x7fdf280fc400 msgr2=0x7fdf281032b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:33.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.095+0000 7fdf2d10e640 1 -- 172.21.15.86:0/940629868 shutdown_connections 2024-04-21T08:36:33.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.095+0000 7fdf2d10e640 1 -- 172.21.15.86:0/940629868 wait complete. 2024-04-21T08:36:33.106 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:36:33.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.206+0000 7f1b42fb2640 1 -- 172.21.15.86:0/1922600217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b3c101fe0 msgr2=0x7f1b3c102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:33.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.206+0000 7f1b42fb2640 1 --2- 172.21.15.86:0/1922600217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b3c101fe0 0x7f1b3c102460 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f1b28009900 tx=0x7f1b2802bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:33.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.206+0000 7f1b42fb2640 1 -- 172.21.15.86:0/1922600217 shutdown_connections 2024-04-21T08:36:33.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.206+0000 7f1b42fb2640 1 --2- 172.21.15.86:0/1922600217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b3c1029a0 0x7f1b3c1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.206+0000 7f1b42fb2640 1 --2- 172.21.15.86:0/1922600217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b3c101fe0 0x7f1b3c102460 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.206+0000 7f1b42fb2640 1 --2- 172.21.15.86:0/1922600217 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b3c100e90 0x7f1b3c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.207+0000 7f1b42fb2640 1 -- 172.21.15.86:0/1922600217 >> 172.21.15.86:0/1922600217 conn(0x7f1b3c0fc400 msgr2=0x7f1b3c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:33.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.207+0000 7f1b42fb2640 1 -- 172.21.15.86:0/1922600217 shutdown_connections 2024-04-21T08:36:33.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.207+0000 7f1b42fb2640 1 -- 172.21.15.86:0/1922600217 wait complete. 2024-04-21T08:36:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.207+0000 7f1b42fb2640 1 Processor -- start 2024-04-21T08:36:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.207+0000 7f1b42fb2640 1 -- start start 2024-04-21T08:36:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.207+0000 7f1b42fb2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b3c100e90 0x7f1b3c198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.207+0000 7f1b42fb2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b3c101fe0 0x7f1b3c1987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.207+0000 7f1b42fb2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b3c1029a0 0x7f1b3c19f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.207+0000 7f1b42fb2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c10c9f0 con 0x7f1b3c1029a0 2024-04-21T08:36:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.207+0000 7f1b42fb2640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c10c870 con 0x7f1b3c101fe0 2024-04-21T08:36:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.207+0000 7f1b42fb2640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1b3c10cb70 con 0x7f1b3c100e90 2024-04-21T08:36:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.207+0000 7f1b4154f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b3c1029a0 0x7f1b3c19f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.207+0000 7f1b4154f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b3c1029a0 0x7f1b3c19f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36334/0 (socket says 172.21.15.86:36334) 2024-04-21T08:36:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.207+0000 7f1b4154f640 1 -- 172.21.15.86:0/2376979340 learned_addr learned my addr 172.21.15.86:0/2376979340 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.207+0000 7f1b40d4e640 1 --2- 172.21.15.86:0/2376979340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b3c100e90 0x7f1b3c198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:33.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.207+0000 7f1b3bfff640 1 --2- 172.21.15.86:0/2376979340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b3c101fe0 0x7f1b3c1987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:33.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.208+0000 7f1b4154f640 1 -- 172.21.15.86:0/2376979340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b3c100e90 msgr2=0x7f1b3c198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:33.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.208+0000 7f1b4154f640 1 --2- 172.21.15.86:0/2376979340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b3c100e90 0x7f1b3c198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.208+0000 7f1b4154f640 1 -- 172.21.15.86:0/2376979340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b3c101fe0 msgr2=0x7f1b3c1987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:33.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.208+0000 7f1b4154f640 1 --2- 172.21.15.86:0/2376979340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b3c101fe0 0x7f1b3c1987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.208+0000 7f1b4154f640 1 -- 172.21.15.86:0/2376979340 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b3c19fea0 con 0x7f1b3c1029a0 2024-04-21T08:36:33.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.208+0000 7f1b4154f640 1 --2- 172.21.15.86:0/2376979340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b3c1029a0 0x7f1b3c19f830 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1b3400cad0 tx=0x7f1b3400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:33.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.208+0000 7f1b39ffb640 1 -- 172.21.15.86:0/2376979340 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3400ae20 con 0x7f1b3c1029a0 2024-04-21T08:36:33.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.208+0000 7f1b39ffb640 1 -- 172.21.15.86:0/2376979340 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b34005020 con 0x7f1b3c1029a0 2024-04-21T08:36:33.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.208+0000 7f1b42fb2640 1 -- 172.21.15.86:0/2376979340 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b3c1a0190 con 0x7f1b3c1029a0 2024-04-21T08:36:33.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.208+0000 7f1b39ffb640 1 -- 172.21.15.86:0/2376979340 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b3400f6b0 con 0x7f1b3c1029a0 2024-04-21T08:36:33.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.208+0000 7f1b42fb2640 1 -- 172.21.15.86:0/2376979340 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b3c1a06b0 con 0x7f1b3c1029a0 2024-04-21T08:36:33.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.209+0000 7f1b42fb2640 1 -- 172.21.15.86:0/2376979340 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1b3c1a0970 con 0x7f1b3c1029a0 2024-04-21T08:36:33.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.210+0000 7f1b39ffb640 1 -- 172.21.15.86:0/2376979340 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f1b340051d0 con 0x7f1b3c1029a0 2024-04-21T08:36:33.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.211+0000 7f1b39ffb640 1 --2- 172.21.15.86:0/2376979340 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f1b2c0a3c30 0x7f1b2c0a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:33.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.211+0000 7f1b40d4e640 1 --2- 172.21.15.86:0/2376979340 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f1b2c0a3c30 0x7f1b2c0a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:33.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.211+0000 7f1b39ffb640 1 -- 172.21.15.86:0/2376979340 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f1b340d05d0 con 0x7f1b3c1029a0 2024-04-21T08:36:33.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.211+0000 7f1b40d4e640 1 --2- 172.21.15.86:0/2376979340 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f1b2c0a3c30 0x7f1b2c0a60f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1b30009bd0 tx=0x7f1b30005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:33.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.215+0000 7f1b39ffb640 1 -- 172.21.15.86:0/2376979340 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1b34014030 con 0x7f1b3c1029a0 2024-04-21T08:36:33.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:33 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1338139594' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:33.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:33 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/940629868' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:36:33.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.396+0000 7f1b42fb2640 1 -- 172.21.15.86:0/2376979340 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f1b3c1a0be0 con 0x7f1b3c1029a0 2024-04-21T08:36:33.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.396+0000 7f1b39ffb640 1 -- 172.21.15.86:0/2376979340 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f1b3409c720 con 0x7f1b3c1029a0 2024-04-21T08:36:33.396 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-21T08:36:33.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.398+0000 7f1b42fb2640 1 -- 172.21.15.86:0/2376979340 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f1b2c0a3c30 msgr2=0x7f1b2c0a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:33.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.398+0000 7f1b42fb2640 1 --2- 172.21.15.86:0/2376979340 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f1b2c0a3c30 0x7f1b2c0a60f0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7f1b30009bd0 tx=0x7f1b30005f90 comp rx=0 tx=0).stop 2024-04-21T08:36:33.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.398+0000 7f1b42fb2640 1 -- 172.21.15.86:0/2376979340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b3c1029a0 msgr2=0x7f1b3c19f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:33.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.398+0000 7f1b42fb2640 1 --2- 172.21.15.86:0/2376979340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b3c1029a0 0x7f1b3c19f830 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f1b3400cad0 tx=0x7f1b3400cf90 comp rx=0 tx=0).stop 2024-04-21T08:36:33.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.398+0000 7f1b42fb2640 1 -- 172.21.15.86:0/2376979340 shutdown_connections 2024-04-21T08:36:33.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.398+0000 7f1b42fb2640 1 --2- 172.21.15.86:0/2376979340 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f1b2c0a3c30 0x7f1b2c0a60f0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.398+0000 7f1b42fb2640 1 --2- 172.21.15.86:0/2376979340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b3c1029a0 0x7f1b3c19f830 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.398+0000 7f1b42fb2640 1 --2- 172.21.15.86:0/2376979340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b3c101fe0 0x7f1b3c1987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.398+0000 7f1b42fb2640 1 --2- 172.21.15.86:0/2376979340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b3c100e90 0x7f1b3c198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.398+0000 7f1b42fb2640 1 -- 172.21.15.86:0/2376979340 >> 172.21.15.86:0/2376979340 conn(0x7f1b3c0fc400 msgr2=0x7f1b3c0690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:33.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.398+0000 7f1b42fb2640 1 -- 172.21.15.86:0/2376979340 shutdown_connections 2024-04-21T08:36:33.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.398+0000 7f1b42fb2640 1 -- 172.21.15.86:0/2376979340 wait complete. 2024-04-21T08:36:33.409 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:36:33.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.511+0000 7f63506e4640 1 -- 172.21.15.86:0/3143255919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6348103230 msgr2=0x7f63481036b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:33.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.511+0000 7f63506e4640 1 --2- 172.21.15.86:0/3143255919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6348103230 0x7f63481036b0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f633c009cd0 tx=0x7f633c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.511+0000 7f63506e4640 1 -- 172.21.15.86:0/3143255919 shutdown_connections 2024-04-21T08:36:33.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.511+0000 7f63506e4640 1 --2- 172.21.15.86:0/3143255919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6348103bf0 0x7f634810a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.511+0000 7f63506e4640 1 --2- 172.21.15.86:0/3143255919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6348103230 0x7f63481036b0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.511+0000 7f63506e4640 1 --2- 172.21.15.86:0/3143255919 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6348100260 0x7f6348100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.511+0000 7f63506e4640 1 -- 172.21.15.86:0/3143255919 >> 172.21.15.86:0/3143255919 conn(0x7f63480fbb90 msgr2=0x7f63480fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:33.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.511+0000 7f63506e4640 1 -- 172.21.15.86:0/3143255919 shutdown_connections 2024-04-21T08:36:33.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.511+0000 7f63506e4640 1 -- 172.21.15.86:0/3143255919 wait complete. 2024-04-21T08:36:33.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.512+0000 7f63506e4640 1 Processor -- start 2024-04-21T08:36:33.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.512+0000 7f63506e4640 1 -- start start 2024-04-21T08:36:33.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.512+0000 7f63506e4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6348100260 0x7f63481981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:33.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.512+0000 7f63506e4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6348103230 0x7f6348198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:33.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.512+0000 7f63506e4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6348103bf0 0x7f634819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:33.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.512+0000 7f63506e4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f634810db00 con 0x7f6348103bf0 2024-04-21T08:36:33.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.512+0000 7f63506e4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f634810d980 con 0x7f6348103230 2024-04-21T08:36:33.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.512+0000 7f63506e4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f634810dc80 con 0x7f6348100260 2024-04-21T08:36:33.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.512+0000 7f634ec81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6348103bf0 0x7f634819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:33.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.512+0000 7f634ec81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6348103bf0 0x7f634819f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36342/0 (socket says 172.21.15.86:36342) 2024-04-21T08:36:33.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.512+0000 7f634ec81640 1 -- 172.21.15.86:0/3402987492 learned_addr learned my addr 172.21.15.86:0/3402987492 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:33.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.512+0000 7f634dc7f640 1 --2- 172.21.15.86:0/3402987492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6348103230 0x7f6348198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:33.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.512+0000 7f634e480640 1 --2- 172.21.15.86:0/3402987492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6348100260 0x7f63481981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:33.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.512+0000 7f634ec81640 1 -- 172.21.15.86:0/3402987492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6348100260 msgr2=0x7f63481981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:33.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.512+0000 7f634ec81640 1 --2- 172.21.15.86:0/3402987492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6348100260 0x7f63481981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.512+0000 7f634ec81640 1 -- 172.21.15.86:0/3402987492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6348103230 msgr2=0x7f6348198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:33.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.512+0000 7f634ec81640 1 --2- 172.21.15.86:0/3402987492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6348103230 0x7f6348198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.512+0000 7f634ec81640 1 -- 172.21.15.86:0/3402987492 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f634819fe10 con 0x7f6348103bf0 2024-04-21T08:36:33.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.513+0000 7f634ec81640 1 --2- 172.21.15.86:0/3402987492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6348103bf0 0x7f634819f7a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f634000d950 tx=0x7f634000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:33.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.513+0000 7f63477fe640 1 -- 172.21.15.86:0/3402987492 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6340017030 con 0x7f6348103bf0 2024-04-21T08:36:33.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.513+0000 7f63477fe640 1 -- 172.21.15.86:0/3402987492 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6340004020 con 0x7f6348103bf0 2024-04-21T08:36:33.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.513+0000 7f63506e4640 1 -- 172.21.15.86:0/3402987492 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f63481a0100 con 0x7f6348103bf0 2024-04-21T08:36:33.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.513+0000 7f63477fe640 1 -- 172.21.15.86:0/3402987492 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6340003b50 con 0x7f6348103bf0 2024-04-21T08:36:33.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.513+0000 7f63506e4640 1 -- 172.21.15.86:0/3402987492 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f63481a05d0 con 0x7f6348103bf0 2024-04-21T08:36:33.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.514+0000 7f63506e4640 1 -- 172.21.15.86:0/3402987492 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f63481093d0 con 0x7f6348103bf0 2024-04-21T08:36:33.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.515+0000 7f63477fe640 1 -- 172.21.15.86:0/3402987492 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f634001a070 con 0x7f6348103bf0 2024-04-21T08:36:33.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.515+0000 7f63477fe640 1 --2- 172.21.15.86:0/3402987492 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f63300a3bb0 0x7f63300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:33.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.516+0000 7f634e480640 1 --2- 172.21.15.86:0/3402987492 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f63300a3bb0 0x7f63300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:33.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.516+0000 7f63477fe640 1 -- 172.21.15.86:0/3402987492 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f63400d0010 con 0x7f6348103bf0 2024-04-21T08:36:33.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.516+0000 7f634e480640 1 --2- 172.21.15.86:0/3402987492 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f63300a3bb0 0x7f63300a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6338005fd0 tx=0x7f633800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:33.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.519+0000 7f63477fe640 1 -- 172.21.15.86:0/3402987492 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6340021070 con 0x7f6348103bf0 2024-04-21T08:36:33.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.696+0000 7f63506e4640 1 -- 172.21.15.86:0/3402987492 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f6348062b50 con 0x7f6348103bf0 2024-04-21T08:36:33.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.696+0000 7f63477fe640 1 -- 172.21.15.86:0/3402987492 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7f634009c7b0 con 0x7f6348103bf0 2024-04-21T08:36:33.696 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-21T08:36:33.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.698+0000 7f63506e4640 1 -- 172.21.15.86:0/3402987492 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f63300a3bb0 msgr2=0x7f63300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:33.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.698+0000 7f63506e4640 1 --2- 172.21.15.86:0/3402987492 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f63300a3bb0 0x7f63300a6070 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f6338005fd0 tx=0x7f633800b410 comp rx=0 tx=0).stop 2024-04-21T08:36:33.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.698+0000 7f63506e4640 1 -- 172.21.15.86:0/3402987492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6348103bf0 msgr2=0x7f634819f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:33.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.698+0000 7f63506e4640 1 --2- 172.21.15.86:0/3402987492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6348103bf0 0x7f634819f7a0 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7f634000d950 tx=0x7f634000de10 comp rx=0 tx=0).stop 2024-04-21T08:36:33.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.698+0000 7f63506e4640 1 -- 172.21.15.86:0/3402987492 shutdown_connections 2024-04-21T08:36:33.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.698+0000 7f63506e4640 1 --2- 172.21.15.86:0/3402987492 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f63300a3bb0 0x7f63300a6070 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.698+0000 7f63506e4640 1 --2- 172.21.15.86:0/3402987492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6348103bf0 0x7f634819f7a0 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.698+0000 7f63506e4640 1 --2- 172.21.15.86:0/3402987492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6348103230 0x7f6348198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.698+0000 7f63506e4640 1 --2- 172.21.15.86:0/3402987492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6348100260 0x7f63481981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.698+0000 7f63506e4640 1 -- 172.21.15.86:0/3402987492 >> 172.21.15.86:0/3402987492 conn(0x7f63480fbb90 msgr2=0x7f63481065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:33.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.698+0000 7f63506e4640 1 -- 172.21.15.86:0/3402987492 shutdown_connections 2024-04-21T08:36:33.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.698+0000 7f63506e4640 1 -- 172.21.15.86:0/3402987492 wait complete. 2024-04-21T08:36:33.709 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:36:33.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:33 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1338139594' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:33.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:33 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/940629868' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:36:33.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:33 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1338139594' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:33.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:33 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/940629868' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:36:33.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.809+0000 7feee5043640 1 -- 172.21.15.86:0/1571581713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feee0107c70 msgr2=0x7feee010a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:33.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.809+0000 7feee5043640 1 --2- 172.21.15.86:0/1571581713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feee0107c70 0x7feee010a0c0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7feed8009ca0 tx=0x7feed802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.809+0000 7feee5043640 1 -- 172.21.15.86:0/1571581713 shutdown_connections 2024-04-21T08:36:33.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.809+0000 7feee5043640 1 --2- 172.21.15.86:0/1571581713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feee0107c70 0x7feee010a0c0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.809+0000 7feee5043640 1 --2- 172.21.15.86:0/1571581713 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feee0105340 0x7feee0107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.809+0000 7feee5043640 1 --2- 172.21.15.86:0/1571581713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feee0102a10 0x7feee0104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.809+0000 7feee5043640 1 -- 172.21.15.86:0/1571581713 >> 172.21.15.86:0/1571581713 conn(0x7feee00fc400 msgr2=0x7feee00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:33.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.809+0000 7feee5043640 1 -- 172.21.15.86:0/1571581713 shutdown_connections 2024-04-21T08:36:33.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.809+0000 7feee5043640 1 -- 172.21.15.86:0/1571581713 wait complete. 2024-04-21T08:36:33.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.810+0000 7feee5043640 1 Processor -- start 2024-04-21T08:36:33.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.810+0000 7feee5043640 1 -- start start 2024-04-21T08:36:33.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.810+0000 7feee5043640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feee0102a10 0x7feee01981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:33.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.810+0000 7feee5043640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feee0105340 0x7feee01986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:33.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.810+0000 7feee5043640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feee0107c70 0x7feee019f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:33.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.811+0000 7feee5043640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feee010dae0 con 0x7feee0102a10 2024-04-21T08:36:33.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.811+0000 7feee5043640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7feee010d960 con 0x7feee0107c70 2024-04-21T08:36:33.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.811+0000 7feee5043640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7feee010dc60 con 0x7feee0105340 2024-04-21T08:36:33.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.811+0000 7feeded9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feee0102a10 0x7feee01981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:33.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.811+0000 7feeded9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feee0102a10 0x7feee01981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36360/0 (socket says 172.21.15.86:36360) 2024-04-21T08:36:33.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.811+0000 7feeded9d640 1 -- 172.21.15.86:0/586493694 learned_addr learned my addr 172.21.15.86:0/586493694 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:33.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.811+0000 7feede59c640 1 --2- 172.21.15.86:0/586493694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feee0105340 0x7feee01986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:33.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.811+0000 7feedf59e640 1 --2- 172.21.15.86:0/586493694 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feee0107c70 0x7feee019f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:33.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.811+0000 7feeded9d640 1 -- 172.21.15.86:0/586493694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feee0105340 msgr2=0x7feee01986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:33.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.811+0000 7feeded9d640 1 --2- 172.21.15.86:0/586493694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feee0105340 0x7feee01986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.811+0000 7feeded9d640 1 -- 172.21.15.86:0/586493694 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feee0107c70 msgr2=0x7feee019f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:33.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.811+0000 7feeded9d640 1 --2- 172.21.15.86:0/586493694 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feee0107c70 0x7feee019f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.811+0000 7feeded9d640 1 -- 172.21.15.86:0/586493694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feee019fdd0 con 0x7feee0102a10 2024-04-21T08:36:33.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.811+0000 7feedf59e640 1 --2- 172.21.15.86:0/586493694 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feee0107c70 0x7feee019f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:36:33.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.811+0000 7feeded9d640 1 --2- 172.21.15.86:0/586493694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feee0102a10 0x7feee01981b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7feed400e9f0 tx=0x7feed400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:33.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.811+0000 7feec7fff640 1 -- 172.21.15.86:0/586493694 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed400cd80 con 0x7feee0102a10 2024-04-21T08:36:33.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.811+0000 7feee5043640 1 -- 172.21.15.86:0/586493694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feee01a00c0 con 0x7feee0102a10 2024-04-21T08:36:33.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.811+0000 7feec7fff640 1 -- 172.21.15.86:0/586493694 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feed40054d0 con 0x7feee0102a10 2024-04-21T08:36:33.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.811+0000 7feee5043640 1 -- 172.21.15.86:0/586493694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feee01a0610 con 0x7feee0102a10 2024-04-21T08:36:33.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.811+0000 7feec7fff640 1 -- 172.21.15.86:0/586493694 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feed4010690 con 0x7feee0102a10 2024-04-21T08:36:33.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.812+0000 7feee5043640 1 -- 172.21.15.86:0/586493694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feee00fdef0 con 0x7feee0102a10 2024-04-21T08:36:33.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.813+0000 7feec7fff640 1 -- 172.21.15.86:0/586493694 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7feed4004e50 con 0x7feee0102a10 2024-04-21T08:36:33.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.814+0000 7feec7fff640 1 --2- 172.21.15.86:0/586493694 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7feec80a3bb0 0x7feec80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:33.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.814+0000 7feede59c640 1 --2- 172.21.15.86:0/586493694 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7feec80a3bb0 0x7feec80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:33.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.814+0000 7feec7fff640 1 -- 172.21.15.86:0/586493694 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7feed40d0d10 con 0x7feee0102a10 2024-04-21T08:36:33.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.815+0000 7feede59c640 1 --2- 172.21.15.86:0/586493694 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7feec80a3bb0 0x7feec80a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7feee01996c0 tx=0x7feed000a2d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:33.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.818+0000 7feec7fff640 1 -- 172.21.15.86:0/586493694 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feed4053ff0 con 0x7feee0102a10 2024-04-21T08:36:33.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.995+0000 7feee5043640 1 -- 172.21.15.86:0/586493694 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7feee002d850 con 0x7feee0102a10 2024-04-21T08:36:33.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.995+0000 7feec7fff640 1 -- 172.21.15.86:0/586493694 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) ==== 103+0+11 (secure 0 0 0) 0x7feed409d4b0 con 0x7feee0102a10 2024-04-21T08:36:33.996 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-21T08:36:33.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.997+0000 7feee5043640 1 -- 172.21.15.86:0/586493694 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7feec80a3bb0 msgr2=0x7feec80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:33.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.997+0000 7feee5043640 1 --2- 172.21.15.86:0/586493694 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7feec80a3bb0 0x7feec80a6070 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7feee01996c0 tx=0x7feed000a2d0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.997+0000 7feee5043640 1 -- 172.21.15.86:0/586493694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feee0102a10 msgr2=0x7feee01981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:33.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.997+0000 7feee5043640 1 --2- 172.21.15.86:0/586493694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feee0102a10 0x7feee01981b0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7feed400e9f0 tx=0x7feed400edf0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.997+0000 7feee5043640 1 -- 172.21.15.86:0/586493694 shutdown_connections 2024-04-21T08:36:33.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.997+0000 7feee5043640 1 --2- 172.21.15.86:0/586493694 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7feec80a3bb0 0x7feec80a6070 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.997+0000 7feee5043640 1 --2- 172.21.15.86:0/586493694 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feee0107c70 0x7feee019f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.997+0000 7feee5043640 1 --2- 172.21.15.86:0/586493694 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feee0105340 0x7feee01986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.997+0000 7feee5043640 1 --2- 172.21.15.86:0/586493694 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feee0102a10 0x7feee01981b0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:33.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.997+0000 7feee5043640 1 -- 172.21.15.86:0/586493694 >> 172.21.15.86:0/586493694 conn(0x7feee00fc400 msgr2=0x7feee01032b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:33.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.998+0000 7feee5043640 1 -- 172.21.15.86:0/586493694 shutdown_connections 2024-04-21T08:36:33.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:33.998+0000 7feee5043640 1 -- 172.21.15.86:0/586493694 wait complete. 2024-04-21T08:36:34.009 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-21T08:36:34.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.108+0000 7f2d12ce0640 1 -- 172.21.15.86:0/480596898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d0c101fe0 msgr2=0x7f2d0c102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:34.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.108+0000 7f2d12ce0640 1 --2- 172.21.15.86:0/480596898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d0c101fe0 0x7f2d0c102460 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f2d00009900 tx=0x7f2d0002bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:34.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.108+0000 7f2d12ce0640 1 -- 172.21.15.86:0/480596898 shutdown_connections 2024-04-21T08:36:34.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.108+0000 7f2d12ce0640 1 --2- 172.21.15.86:0/480596898 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d0c1029a0 0x7f2d0c1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:34.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.108+0000 7f2d12ce0640 1 --2- 172.21.15.86:0/480596898 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d0c101fe0 0x7f2d0c102460 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:34.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.108+0000 7f2d12ce0640 1 --2- 172.21.15.86:0/480596898 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d0c100e90 0x7f2d0c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:34.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.108+0000 7f2d12ce0640 1 -- 172.21.15.86:0/480596898 >> 172.21.15.86:0/480596898 conn(0x7f2d0c0fc400 msgr2=0x7f2d0c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:34.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.108+0000 7f2d12ce0640 1 -- 172.21.15.86:0/480596898 shutdown_connections 2024-04-21T08:36:34.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.109+0000 7f2d12ce0640 1 -- 172.21.15.86:0/480596898 wait complete. 2024-04-21T08:36:34.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.109+0000 7f2d12ce0640 1 Processor -- start 2024-04-21T08:36:34.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.109+0000 7f2d12ce0640 1 -- start start 2024-04-21T08:36:34.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.109+0000 7f2d12ce0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d0c100e90 0x7f2d0c198260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:34.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.109+0000 7f2d12ce0640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d0c101fe0 0x7f2d0c1987a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:34.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.109+0000 7f2d12ce0640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d0c1029a0 0x7f2d0c19f810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:34.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.109+0000 7f2d12ce0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0c10db30 con 0x7f2d0c100e90 2024-04-21T08:36:34.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.109+0000 7f2d12ce0640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0c10d9b0 con 0x7f2d0c1029a0 2024-04-21T08:36:34.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.109+0000 7f2d12ce0640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2d0c10dcb0 con 0x7f2d0c101fe0 2024-04-21T08:36:34.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.109+0000 7f2d10a7c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d0c100e90 0x7f2d0c198260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:34.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.109+0000 7f2d10a7c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d0c100e90 0x7f2d0c198260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36378/0 (socket says 172.21.15.86:36378) 2024-04-21T08:36:34.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.109+0000 7f2d10a7c640 1 -- 172.21.15.86:0/437266760 learned_addr learned my addr 172.21.15.86:0/437266760 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:34.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.109+0000 7f2d0bfff640 1 --2- 172.21.15.86:0/437266760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d0c101fe0 0x7f2d0c1987a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.109+0000 7f2d1127d640 1 --2- 172.21.15.86:0/437266760 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d0c1029a0 0x7f2d0c19f810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.110+0000 7f2d10a7c640 1 -- 172.21.15.86:0/437266760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d0c101fe0 msgr2=0x7f2d0c1987a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.110+0000 7f2d10a7c640 1 --2- 172.21.15.86:0/437266760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d0c101fe0 0x7f2d0c1987a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.110+0000 7f2d10a7c640 1 -- 172.21.15.86:0/437266760 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d0c1029a0 msgr2=0x7f2d0c19f810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.110+0000 7f2d10a7c640 1 --2- 172.21.15.86:0/437266760 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d0c1029a0 0x7f2d0c19f810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.110+0000 7f2d10a7c640 1 -- 172.21.15.86:0/437266760 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d0c19fe80 con 0x7f2d0c100e90 2024-04-21T08:36:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.110+0000 7f2d0bfff640 1 --2- 172.21.15.86:0/437266760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d0c101fe0 0x7f2d0c1987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:36:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.110+0000 7f2d1127d640 1 --2- 172.21.15.86:0/437266760 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d0c1029a0 0x7f2d0c19f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:36:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.110+0000 7f2d10a7c640 1 --2- 172.21.15.86:0/437266760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d0c100e90 0x7f2d0c198260 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc00d950 tx=0x7f2cfc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.110+0000 7f2d09ffb640 1 -- 172.21.15.86:0/437266760 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc017070 con 0x7f2d0c100e90 2024-04-21T08:36:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.110+0000 7f2d12ce0640 1 -- 172.21.15.86:0/437266760 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d0c1a0170 con 0x7f2d0c100e90 2024-04-21T08:36:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.110+0000 7f2d09ffb640 1 -- 172.21.15.86:0/437266760 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2cfc009c30 con 0x7f2d0c100e90 2024-04-21T08:36:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.110+0000 7f2d09ffb640 1 -- 172.21.15.86:0/437266760 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2cfc00f620 con 0x7f2d0c100e90 2024-04-21T08:36:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.110+0000 7f2d12ce0640 1 -- 172.21.15.86:0/437266760 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d0c1a0710 con 0x7f2d0c100e90 2024-04-21T08:36:34.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.111+0000 7f2d12ce0640 1 -- 172.21.15.86:0/437266760 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2d0c108310 con 0x7f2d0c100e90 2024-04-21T08:36:34.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.112+0000 7f2d09ffb640 1 -- 172.21.15.86:0/437266760 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f2cfc010460 con 0x7f2d0c100e90 2024-04-21T08:36:34.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.112+0000 7f2d09ffb640 1 --2- 172.21.15.86:0/437266760 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f2cf40a3bb0 0x7f2cf40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:34.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.113+0000 7f2d09ffb640 1 -- 172.21.15.86:0/437266760 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f2cfc0d0320 con 0x7f2d0c100e90 2024-04-21T08:36:34.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.113+0000 7f2d0bfff640 1 --2- 172.21.15.86:0/437266760 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f2cf40a3bb0 0x7f2cf40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:34.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.113+0000 7f2d0bfff640 1 --2- 172.21.15.86:0/437266760 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f2cf40a3bb0 0x7f2cf40a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2d00036040 tx=0x7f2d00000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:34.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.117+0000 7f2d09ffb640 1 -- 172.21.15.86:0/437266760 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2cfc09cac0 con 0x7f2d0c100e90 2024-04-21T08:36:34.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.309+0000 7f2d12ce0640 1 -- 172.21.15.86:0/437266760 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7f2d0c02d980 con 0x7f2d0c100e90 2024-04-21T08:36:34.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.309+0000 7f2d09ffb640 1 -- 172.21.15.86:0/437266760 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+592 (secure 0 0 0) 0x7f2cfc009de0 con 0x7f2d0c100e90 2024-04-21T08:36:34.309 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/2143206409 2024-04-22T08:29:23.190001+0000 2024-04-21T08:36:34.309 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1412393225 2024-04-22T08:29:23.190001+0000 2024-04-21T08:36:34.310 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/2386990413 2024-04-22T08:28:57.384403+0000 2024-04-21T08:36:34.310 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1687852404 2024-04-22T08:29:23.190001+0000 2024-04-21T08:36:34.310 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/2386990413 2024-04-22T08:28:57.384403+0000 2024-04-21T08:36:34.310 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2020707404 2024-04-22T08:28:57.384403+0000 2024-04-21T08:36:34.310 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2568130933 2024-04-22T08:28:57.384403+0000 2024-04-21T08:36:34.310 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3952607648 2024-04-22T08:28:57.384403+0000 2024-04-21T08:36:34.310 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/2143206409 2024-04-22T08:29:23.190001+0000 2024-04-21T08:36:34.310 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2756903426 2024-04-22T08:29:23.190001+0000 2024-04-21T08:36:34.310 INFO:teuthology.orchestra.run.smithi086.stderr:listed 10 entries 2024-04-21T08:36:34.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.311+0000 7f2d12ce0640 1 -- 172.21.15.86:0/437266760 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f2cf40a3bb0 msgr2=0x7f2cf40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:34.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.311+0000 7f2d12ce0640 1 --2- 172.21.15.86:0/437266760 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f2cf40a3bb0 0x7f2cf40a6070 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f2d00036040 tx=0x7f2d00000bc0 comp rx=0 tx=0).stop 2024-04-21T08:36:34.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.311+0000 7f2d12ce0640 1 -- 172.21.15.86:0/437266760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d0c100e90 msgr2=0x7f2d0c198260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:34.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.311+0000 7f2d12ce0640 1 --2- 172.21.15.86:0/437266760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d0c100e90 0x7f2d0c198260 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f2cfc00d950 tx=0x7f2cfc00de10 comp rx=0 tx=0).stop 2024-04-21T08:36:34.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.311+0000 7f2d12ce0640 1 -- 172.21.15.86:0/437266760 shutdown_connections 2024-04-21T08:36:34.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.311+0000 7f2d12ce0640 1 --2- 172.21.15.86:0/437266760 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f2cf40a3bb0 0x7f2cf40a6070 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:34.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.311+0000 7f2d12ce0640 1 --2- 172.21.15.86:0/437266760 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d0c1029a0 0x7f2d0c19f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:34.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.311+0000 7f2d12ce0640 1 --2- 172.21.15.86:0/437266760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d0c101fe0 0x7f2d0c1987a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:34.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.311+0000 7f2d12ce0640 1 --2- 172.21.15.86:0/437266760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d0c100e90 0x7f2d0c198260 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:34.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.311+0000 7f2d12ce0640 1 -- 172.21.15.86:0/437266760 >> 172.21.15.86:0/437266760 conn(0x7f2d0c0fc400 msgr2=0x7f2d0c1054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:34.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.311+0000 7f2d12ce0640 1 -- 172.21.15.86:0/437266760 shutdown_connections 2024-04-21T08:36:34.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.311+0000 7f2d12ce0640 1 -- 172.21.15.86:0/437266760 wait complete. 2024-04-21T08:36:34.322 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-21T08:36:34.323 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-21T08:36:34.338 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:36:34.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.477+0000 7fc4c2700640 1 -- 172.21.15.86:0/240217876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4bc107ca0 msgr2=0x7fc4bc10a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:34.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.477+0000 7fc4c2700640 1 --2- 172.21.15.86:0/240217876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4bc107ca0 0x7fc4bc10a0f0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fc4b4009ca0 tx=0x7fc4b402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:36:34.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.477+0000 7fc4c2700640 1 -- 172.21.15.86:0/240217876 shutdown_connections 2024-04-21T08:36:34.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.477+0000 7fc4c2700640 1 --2- 172.21.15.86:0/240217876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4bc107ca0 0x7fc4bc10a0f0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:34.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.477+0000 7fc4c2700640 1 --2- 172.21.15.86:0/240217876 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4bc105370 0x7fc4bc107760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:34.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.477+0000 7fc4c2700640 1 --2- 172.21.15.86:0/240217876 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4bc102a40 0x7fc4bc104e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:34.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.477+0000 7fc4c2700640 1 -- 172.21.15.86:0/240217876 >> 172.21.15.86:0/240217876 conn(0x7fc4bc0fc470 msgr2=0x7fc4bc0fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:34.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.477+0000 7fc4c2700640 1 -- 172.21.15.86:0/240217876 shutdown_connections 2024-04-21T08:36:34.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.477+0000 7fc4c2700640 1 -- 172.21.15.86:0/240217876 wait complete. 2024-04-21T08:36:34.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.478+0000 7fc4c2700640 1 Processor -- start 2024-04-21T08:36:34.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.478+0000 7fc4c2700640 1 -- start start 2024-04-21T08:36:34.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.478+0000 7fc4c2700640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4bc102a40 0x7fc4bc1023b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:34.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.478+0000 7fc4c2700640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4bc105370 0x7fc4bc1009f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:34.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.478+0000 7fc4c2700640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4bc107ca0 0x7fc4bc100f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:34.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.478+0000 7fc4c2700640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc4bc10db00 con 0x7fc4bc102a40 2024-04-21T08:36:34.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.478+0000 7fc4c2700640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc4bc10d980 con 0x7fc4bc107ca0 2024-04-21T08:36:34.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.478+0000 7fc4c2700640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc4bc10dc80 con 0x7fc4bc105370 2024-04-21T08:36:34.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.478+0000 7fc4bbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4bc102a40 0x7fc4bc1023b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.478+0000 7fc4bbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4bc102a40 0x7fc4bc1023b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36394/0 (socket says 172.21.15.86:36394) 2024-04-21T08:36:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.478+0000 7fc4bbfff640 1 -- 172.21.15.86:0/125315564 learned_addr learned my addr 172.21.15.86:0/125315564 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.478+0000 7fc4bb7fe640 1 --2- 172.21.15.86:0/125315564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4bc105370 0x7fc4bc1009f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.478+0000 7fc4c0c9d640 1 --2- 172.21.15.86:0/125315564 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4bc107ca0 0x7fc4bc100f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.478+0000 7fc4bbfff640 1 -- 172.21.15.86:0/125315564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4bc105370 msgr2=0x7fc4bc1009f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.478+0000 7fc4bbfff640 1 --2- 172.21.15.86:0/125315564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4bc105370 0x7fc4bc1009f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.478+0000 7fc4bbfff640 1 -- 172.21.15.86:0/125315564 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4bc107ca0 msgr2=0x7fc4bc100f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.478+0000 7fc4bbfff640 1 --2- 172.21.15.86:0/125315564 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4bc107ca0 0x7fc4bc100f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.478+0000 7fc4bbfff640 1 -- 172.21.15.86:0/125315564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4bc1017c0 con 0x7fc4bc102a40 2024-04-21T08:36:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.478+0000 7fc4bb7fe640 1 --2- 172.21.15.86:0/125315564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4bc105370 0x7fc4bc1009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:36:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.478+0000 7fc4bbfff640 1 --2- 172.21.15.86:0/125315564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4bc102a40 0x7fc4bc1023b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac00e9f0 tx=0x7fc4ac00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.479+0000 7fc4b97fa640 1 -- 172.21.15.86:0/125315564 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac00cd80 con 0x7fc4bc102a40 2024-04-21T08:36:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.479+0000 7fc4b97fa640 1 -- 172.21.15.86:0/125315564 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4ac0108e0 con 0x7fc4bc102a40 2024-04-21T08:36:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.479+0000 7fc4c2700640 1 -- 172.21.15.86:0/125315564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4bc1a2250 con 0x7fc4bc102a40 2024-04-21T08:36:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.479+0000 7fc4b97fa640 1 -- 172.21.15.86:0/125315564 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4ac018980 con 0x7fc4bc102a40 2024-04-21T08:36:34.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.479+0000 7fc4c2700640 1 -- 172.21.15.86:0/125315564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4bc1a27f0 con 0x7fc4bc102a40 2024-04-21T08:36:34.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.480+0000 7fc4c2700640 1 -- 172.21.15.86:0/125315564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc4bc0fdf60 con 0x7fc4bc102a40 2024-04-21T08:36:34.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.481+0000 7fc4b97fa640 1 -- 172.21.15.86:0/125315564 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fc4ac010430 con 0x7fc4bc102a40 2024-04-21T08:36:34.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.481+0000 7fc4b97fa640 1 --2- 172.21.15.86:0/125315564 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fc4a40a3c30 0x7fc4a40a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:34.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.482+0000 7fc4bb7fe640 1 --2- 172.21.15.86:0/125315564 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fc4a40a3c30 0x7fc4a40a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:34.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.482+0000 7fc4b97fa640 1 -- 172.21.15.86:0/125315564 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fc4ac0d0ed0 con 0x7fc4bc102a40 2024-04-21T08:36:34.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.482+0000 7fc4bb7fe640 1 --2- 172.21.15.86:0/125315564 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fc4a40a3c30 0x7fc4a40a60f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc102120 tx=0x7fc4b000b5f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:34.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.485+0000 7fc4b97fa640 1 -- 172.21.15.86:0/125315564 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc4ac09d670 con 0x7fc4bc102a40 2024-04-21T08:36:34.615 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:34 smithi088 ceph-mon[31259]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:34.616 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:34 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2376979340' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:36:34.616 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:34 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3402987492' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:36:34.616 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:34 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/586493694' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:36:34.616 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:34 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/437266760' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-21T08:36:34.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:34 smithi086 ceph-mon[27939]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:34.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:34 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2376979340' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:36:34.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:34 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3402987492' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:36:34.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:34 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/586493694' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:36:34.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:34 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/437266760' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-21T08:36:34.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.665+0000 7fc4c2700640 1 -- 172.21.15.86:0/125315564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc4bc02d870 con 0x7fc4bc102a40 2024-04-21T08:36:34.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.666+0000 7fc4b97fa640 1 -- 172.21.15.86:0/125315564 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24070 (secure 0 0 0) 0x7fc4ac09cf70 con 0x7fc4bc102a40 2024-04-21T08:36:34.667 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:36:34.667 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":89,"fsid":"f8cec7cc-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:22.687329+0000","modified":"2024-04-21T08:36:25.251657+0000","last_up_change":"2024-04-21T08:35:30.048315+0000","last_in_change":"2024-04-21T08:35:10.099496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:45.279109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:18.300782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:19.574636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:22.905632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89b7601f-0620-4db2-90f1-b45cea173738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6803","nonce":2571516897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6805","nonce":2571516897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6809","nonce":2571516897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6807","nonce":2571516897}]},"public_addr":"172.21.15.86:6803/2571516897","cluster_addr":"172.21.15.86:6805/2571516897","heartbeat_back_addr":"172.21.15.86:6809/2571516897","heartbeat_front_addr":"172.21.15.86:6807/2571516897","state":["exists","up"]},{"osd":1,"uuid":"4891031c-f98c-42bc-bb3b-a420855ecc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6801","nonce":74067182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6803","nonce":74067182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6807","nonce":74067182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6805","nonce":74067182}]},"public_addr":"172.21.15.88:6801/74067182","cluster_addr":"172.21.15.88:6803/74067182","heartbeat_back_addr":"172.21.15.88:6807/74067182","heartbeat_front_addr":"172.21.15.88:6805/74067182","state":["exists","up"]},{"osd":2,"uuid":"60d12aa4-9312-4fa2-8d0b-2b3f42caad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6801","nonce":4137659051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6803","nonce":4137659051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6807","nonce":4137659051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6805","nonce":4137659051}]},"public_addr":"172.21.15.179:6801/4137659051","cluster_addr":"172.21.15.179:6803/4137659051","heartbeat_back_addr":"172.21.15.179:6807/4137659051","heartbeat_front_addr":"172.21.15.179:6805/4137659051","state":["exists","up"]},{"osd":3,"uuid":"404c3158-8cb0-4c02-b48a-f9ee5ff7b784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6811","nonce":2309603148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6813","nonce":2309603148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6817","nonce":2309603148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6815","nonce":2309603148}]},"public_addr":"172.21.15.86:6811/2309603148","cluster_addr":"172.21.15.86:6813/2309603148","heartbeat_back_addr":"172.21.15.86:6817/2309603148","heartbeat_front_addr":"172.21.15.86:6815/2309603148","state":["exists","up"]},{"osd":4,"uuid":"458ff202-d781-4ec7-ac93-0b49f835b503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6809","nonce":1818714217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6811","nonce":1818714217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6815","nonce":1818714217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6813","nonce":1818714217}]},"public_addr":"172.21.15.88:6809/1818714217","cluster_addr":"172.21.15.88:6811/1818714217","heartbeat_back_addr":"172.21.15.88:6815/1818714217","heartbeat_front_addr":"172.21.15.88:6813/1818714217","state":["exists","up"]},{"osd":5,"uuid":"8123cc90-bd8b-4204-bf31-46b34e9f589d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6809","nonce":863840990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6811","nonce":863840990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6815","nonce":863840990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6813","nonce":863840990}]},"public_addr":"172.21.15.179:6809/863840990","cluster_addr":"172.21.15.179:6811/863840990","heartbeat_back_addr":"172.21.15.179:6815/863840990","heartbeat_front_addr":"172.21.15.179:6813/863840990","state":["exists","up"]},{"osd":6,"uuid":"aac9d7c7-659a-4110-903a-a871c086066f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6819","nonce":1224231118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6821","nonce":1224231118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6825","nonce":1224231118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6823","nonce":1224231118}]},"public_addr":"172.21.15.86:6819/1224231118","cluster_addr":"172.21.15.86:6821/1224231118","heartbeat_back_addr":"172.21.15.86:6825/1224231118","heartbeat_front_addr":"172.21.15.86:6823/1224231118","state":["exists","up"]},{"osd":7,"uuid":"df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6817","nonce":136629878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6819","nonce":136629878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6823","nonce":136629878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6821","nonce":136629878}]},"public_addr":"172.21.15.88:6817/136629878","cluster_addr":"172.21.15.88:6819/136629878","heartbeat_back_addr":"172.21.15.88:6823/136629878","heartbeat_front_addr":"172.21.15.88:6821/136629878","state":["exists","up"]},{"osd":8,"uuid":"63e59f4b-0426-46db-92f7-c55dae61b5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6817","nonce":1770418957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6819","nonce":1770418957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6823","nonce":1770418957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6821","nonce":1770418957}]},"public_addr":"172.21.15.179:6817/1770418957","cluster_addr":"172.21.15.179:6819/1770418957","heartbeat_back_addr":"172.21.15.179:6823/1770418957","heartbeat_front_addr":"172.21.15.179:6821/1770418957","state":["exists","up"]},{"osd":9,"uuid":"5841bb36-ed6b-45d8-b857-eeab139c44e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6827","nonce":1108500274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6829","nonce":1108500274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6833","nonce":1108500274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6831","nonce":1108500274}]},"public_addr":"172.21.15.86:6827/1108500274","cluster_addr":"172.21.15.86:6829/1108500274","heartbeat_back_addr":"172.21.15.86:6833/1108500274","heartbeat_front_addr":"172.21.15.86:6831/1108500274","state":["exists","up"]},{"osd":10,"uuid":"f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6825","nonce":1294275353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6827","nonce":1294275353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6831","nonce":1294275353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6829","nonce":1294275353}]},"public_addr":"172.21.15.88:6825/1294275353","cluster_addr":"172.21.15.88:6827/1294275353","heartbeat_back_addr":"172.21.15.88:6831/1294275353","heartbeat_front_addr":"172.21.15.88:6829/1294275353","state":["exists","up"]},{"osd":11,"uuid":"e0d4cc76-e7a6-4a6b-909c-95488ccd5a36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6825","nonce":4007070233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6827","nonce":4007070233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6831","nonce":4007070233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6829","nonce":4007070233}]},"public_addr":"172.21.15.179:6825/4007070233","cluster_addr":"172.21.15.179:6827/4007070233","heartbeat_back_addr":"172.21.15.179:6831/4007070233","heartbeat_front_addr":"172.21.15.179:6829/4007070233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:55.210605+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:18.723090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:42.547979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:06.421622+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:30.858315+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:55.455313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.623369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:46.185111+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.103627+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:37.297545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:02.637431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:28.199292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6800/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/1412393225":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6800/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/1687852404":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2020707404":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2568130933":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/3952607648":"2024-04-22T08:28:57.384403+0000","172.21.15.86:6801/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/2756903426":"2024-04-22T08:29:23.190001+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:36:34.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.668+0000 7fc4c2700640 1 -- 172.21.15.86:0/125315564 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fc4a40a3c30 msgr2=0x7fc4a40a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:34.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.668+0000 7fc4c2700640 1 --2- 172.21.15.86:0/125315564 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fc4a40a3c30 0x7fc4a40a60f0 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7fc4bc102120 tx=0x7fc4b000b5f0 comp rx=0 tx=0).stop 2024-04-21T08:36:34.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.668+0000 7fc4c2700640 1 -- 172.21.15.86:0/125315564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4bc102a40 msgr2=0x7fc4bc1023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:34.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.668+0000 7fc4c2700640 1 --2- 172.21.15.86:0/125315564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4bc102a40 0x7fc4bc1023b0 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7fc4ac00e9f0 tx=0x7fc4ac00eeb0 comp rx=0 tx=0).stop 2024-04-21T08:36:34.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.668+0000 7fc4c2700640 1 -- 172.21.15.86:0/125315564 shutdown_connections 2024-04-21T08:36:34.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.668+0000 7fc4c2700640 1 --2- 172.21.15.86:0/125315564 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fc4a40a3c30 0x7fc4a40a60f0 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:34.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.668+0000 7fc4c2700640 1 --2- 172.21.15.86:0/125315564 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc4bc107ca0 0x7fc4bc100f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:34.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.668+0000 7fc4c2700640 1 --2- 172.21.15.86:0/125315564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4bc105370 0x7fc4bc1009f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:34.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.668+0000 7fc4c2700640 1 --2- 172.21.15.86:0/125315564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4bc102a40 0x7fc4bc1023b0 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:34.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.668+0000 7fc4c2700640 1 -- 172.21.15.86:0/125315564 >> 172.21.15.86:0/125315564 conn(0x7fc4bc0fc470 msgr2=0x7fc4bc103380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:34.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.668+0000 7fc4c2700640 1 -- 172.21.15.86:0/125315564 shutdown_connections 2024-04-21T08:36:34.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.668+0000 7fc4c2700640 1 -- 172.21.15.86:0/125315564 wait complete. 2024-04-21T08:36:34.679 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:31:45.279109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:36:18.300782+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 2.25, 'score_stable': 2.25, 'optimal_score': 1, 'raw_score_acting': 2.25, 'raw_score_stable': 2.25, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:36:19.574636+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:36:22.905632+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:36:34.680 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:36:34.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:34 smithi179 ceph-mon[31829]: pgmap v294: 129 pgs: 129 active+clean; 579 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:36:34.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:34 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2376979340' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:36:34.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:34 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3402987492' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:36:34.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:34 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/586493694' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:36:34.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:34 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/437266760' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-21T08:36:34.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.781+0000 7f041a1b6640 1 -- 172.21.15.86:0/1767537789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0414101fa0 msgr2=0x7f04141023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:34.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.781+0000 7f041a1b6640 1 --2- 172.21.15.86:0/1767537789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0414101fa0 0x7f04141023c0 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f0404009900 tx=0x7f040402bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:34.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.781+0000 7f041a1b6640 1 -- 172.21.15.86:0/1767537789 shutdown_connections 2024-04-21T08:36:34.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.781+0000 7f041a1b6640 1 --2- 172.21.15.86:0/1767537789 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0414103b50 0x7f041410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:34.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.781+0000 7f041a1b6640 1 --2- 172.21.15.86:0/1767537789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0414103190 0x7f0414103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:34.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.781+0000 7f041a1b6640 1 --2- 172.21.15.86:0/1767537789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0414101fa0 0x7f04141023c0 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:34.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.781+0000 7f041a1b6640 1 -- 172.21.15.86:0/1767537789 >> 172.21.15.86:0/1767537789 conn(0x7f04140fd530 msgr2=0x7f04140ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:34.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.781+0000 7f041a1b6640 1 -- 172.21.15.86:0/1767537789 shutdown_connections 2024-04-21T08:36:34.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.781+0000 7f041a1b6640 1 -- 172.21.15.86:0/1767537789 wait complete. 2024-04-21T08:36:34.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.782+0000 7f041a1b6640 1 Processor -- start 2024-04-21T08:36:34.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.782+0000 7f041a1b6640 1 -- start start 2024-04-21T08:36:34.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.782+0000 7f041a1b6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0414101fa0 0x7f0414075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:34.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.782+0000 7f041a1b6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0414103190 0x7f0414074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:34.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.782+0000 7f041a1b6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0414103b50 0x7f0414074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:34.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.782+0000 7f041a1b6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f041410dae0 con 0x7f0414103b50 2024-04-21T08:36:34.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.782+0000 7f041a1b6640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f041410d960 con 0x7f0414101fa0 2024-04-21T08:36:34.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.782+0000 7f041a1b6640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f041410dc60 con 0x7f0414103190 2024-04-21T08:36:34.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.782+0000 7f0413fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0414103b50 0x7f0414074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:34.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.782+0000 7f0413fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0414103b50 0x7f0414074880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36408/0 (socket says 172.21.15.86:36408) 2024-04-21T08:36:34.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.782+0000 7f0413fff640 1 -- 172.21.15.86:0/2657689500 learned_addr learned my addr 172.21.15.86:0/2657689500 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:34.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.782+0000 7f040bfff640 1 --2- 172.21.15.86:0/2657689500 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0414103190 0x7f0414074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:34.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.782+0000 7f04137fe640 1 --2- 172.21.15.86:0/2657689500 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0414101fa0 0x7f0414075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:34.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.782+0000 7f0413fff640 1 -- 172.21.15.86:0/2657689500 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0414103190 msgr2=0x7f0414074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:34.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.782+0000 7f0413fff640 1 --2- 172.21.15.86:0/2657689500 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0414103190 0x7f0414074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:34.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.782+0000 7f0413fff640 1 -- 172.21.15.86:0/2657689500 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0414101fa0 msgr2=0x7f0414075d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:34.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.782+0000 7f0413fff640 1 --2- 172.21.15.86:0/2657689500 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0414101fa0 0x7f0414075d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:34.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.782+0000 7f0413fff640 1 -- 172.21.15.86:0/2657689500 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0414075110 con 0x7f0414103b50 2024-04-21T08:36:34.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.782+0000 7f040bfff640 1 --2- 172.21.15.86:0/2657689500 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0414103190 0x7f0414074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:36:34.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.782+0000 7f0413fff640 1 --2- 172.21.15.86:0/2657689500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0414103b50 0x7f0414074880 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f040c00eb10 tx=0x7f040c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:34.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.782+0000 7f04117fa640 1 -- 172.21.15.86:0/2657689500 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c004b20 con 0x7f0414103b50 2024-04-21T08:36:34.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.783+0000 7f04117fa640 1 -- 172.21.15.86:0/2657689500 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f040c004cc0 con 0x7f0414103b50 2024-04-21T08:36:34.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.783+0000 7f041a1b6640 1 -- 172.21.15.86:0/2657689500 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04141a2340 con 0x7f0414103b50 2024-04-21T08:36:34.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.783+0000 7f04117fa640 1 -- 172.21.15.86:0/2657689500 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f040c010670 con 0x7f0414103b50 2024-04-21T08:36:34.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.783+0000 7f041a1b6640 1 -- 172.21.15.86:0/2657689500 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04141a2890 con 0x7f0414103b50 2024-04-21T08:36:34.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.784+0000 7f041a1b6640 1 -- 172.21.15.86:0/2657689500 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0414109380 con 0x7f0414103b50 2024-04-21T08:36:34.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.785+0000 7f04117fa640 1 -- 172.21.15.86:0/2657689500 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f040c005000 con 0x7f0414103b50 2024-04-21T08:36:34.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.785+0000 7f04117fa640 1 --2- 172.21.15.86:0/2657689500 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f04000a3c30 0x7f04000a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:34.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.785+0000 7f04137fe640 1 --2- 172.21.15.86:0/2657689500 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f04000a3c30 0x7f04000a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:34.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.785+0000 7f04117fa640 1 -- 172.21.15.86:0/2657689500 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f040c0d0e80 con 0x7f0414103b50 2024-04-21T08:36:34.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.786+0000 7f04137fe640 1 --2- 172.21.15.86:0/2657689500 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f04000a3c30 0x7f04000a60f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0404006010 tx=0x7f0404005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:34.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.789+0000 7f04117fa640 1 -- 172.21.15.86:0/2657689500 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f040c053ff0 con 0x7f0414103b50 2024-04-21T08:36:34.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.967+0000 7f041a1b6640 1 -- 172.21.15.86:0/2657689500 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f041402d5c0 con 0x7f0414103b50 2024-04-21T08:36:34.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.967+0000 7f04117fa640 1 -- 172.21.15.86:0/2657689500 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v89) ==== 93+0+10 (secure 0 0 0) 0x7f040c09d620 con 0x7f0414103b50 2024-04-21T08:36:34.967 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-21T08:36:34.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.969+0000 7f041a1b6640 1 -- 172.21.15.86:0/2657689500 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f04000a3c30 msgr2=0x7f04000a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:34.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.969+0000 7f041a1b6640 1 --2- 172.21.15.86:0/2657689500 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f04000a3c30 0x7f04000a60f0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7f0404006010 tx=0x7f0404005be0 comp rx=0 tx=0).stop 2024-04-21T08:36:34.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.969+0000 7f041a1b6640 1 -- 172.21.15.86:0/2657689500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0414103b50 msgr2=0x7f0414074880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:34.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.969+0000 7f041a1b6640 1 --2- 172.21.15.86:0/2657689500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0414103b50 0x7f0414074880 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f040c00eb10 tx=0x7f040c00c2d0 comp rx=0 tx=0).stop 2024-04-21T08:36:34.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.969+0000 7f041a1b6640 1 -- 172.21.15.86:0/2657689500 shutdown_connections 2024-04-21T08:36:34.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.969+0000 7f041a1b6640 1 --2- 172.21.15.86:0/2657689500 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f04000a3c30 0x7f04000a60f0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:34.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.969+0000 7f041a1b6640 1 --2- 172.21.15.86:0/2657689500 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0414103b50 0x7f0414074880 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:34.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.969+0000 7f041a1b6640 1 --2- 172.21.15.86:0/2657689500 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0414103190 0x7f0414074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:34.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.969+0000 7f041a1b6640 1 --2- 172.21.15.86:0/2657689500 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0414101fa0 0x7f0414075d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:34.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.969+0000 7f041a1b6640 1 -- 172.21.15.86:0/2657689500 >> 172.21.15.86:0/2657689500 conn(0x7f04140fd530 msgr2=0x7f0414106550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:34.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.969+0000 7f041a1b6640 1 -- 172.21.15.86:0/2657689500 shutdown_connections 2024-04-21T08:36:34.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:34.969+0000 7f041a1b6640 1 -- 172.21.15.86:0/2657689500 wait complete. 2024-04-21T08:36:34.980 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:36:35.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.081+0000 7f20cb932640 1 -- 172.21.15.86:0/4070118518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20c4102a30 msgr2=0x7f20c4104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:35.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.081+0000 7f20cb932640 1 --2- 172.21.15.86:0/4070118518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20c4102a30 0x7f20c4104e20 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f20c0009ca0 tx=0x7f20c002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.081+0000 7f20cb932640 1 -- 172.21.15.86:0/4070118518 shutdown_connections 2024-04-21T08:36:35.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.081+0000 7f20cb932640 1 --2- 172.21.15.86:0/4070118518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20c4107c90 0x7f20c410a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.081+0000 7f20cb932640 1 --2- 172.21.15.86:0/4070118518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20c4105360 0x7f20c4107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.081+0000 7f20cb932640 1 --2- 172.21.15.86:0/4070118518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20c4102a30 0x7f20c4104e20 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.081+0000 7f20cb932640 1 -- 172.21.15.86:0/4070118518 >> 172.21.15.86:0/4070118518 conn(0x7f20c40fc440 msgr2=0x7f20c40fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:35.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.081+0000 7f20cb932640 1 -- 172.21.15.86:0/4070118518 shutdown_connections 2024-04-21T08:36:35.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.081+0000 7f20cb932640 1 -- 172.21.15.86:0/4070118518 wait complete. 2024-04-21T08:36:35.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.082+0000 7f20cb932640 1 Processor -- start 2024-04-21T08:36:35.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.082+0000 7f20cb932640 1 -- start start 2024-04-21T08:36:35.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.082+0000 7f20cb932640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20c4102a30 0x7f20c41023a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:35.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.082+0000 7f20cb932640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20c4105360 0x7f20c41009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:35.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.082+0000 7f20cb932640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20c4107c90 0x7f20c4100f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:35.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.082+0000 7f20cb932640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f20c410daf0 con 0x7f20c4102a30 2024-04-21T08:36:35.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.082+0000 7f20cb932640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f20c410d970 con 0x7f20c4105360 2024-04-21T08:36:35.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.082+0000 7f20cb932640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f20c410dc70 con 0x7f20c4107c90 2024-04-21T08:36:35.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.082+0000 7f20c96ce640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20c4102a30 0x7f20c41023a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:35.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.082+0000 7f20c96ce640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20c4102a30 0x7f20c41023a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36412/0 (socket says 172.21.15.86:36412) 2024-04-21T08:36:35.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.082+0000 7f20c96ce640 1 -- 172.21.15.86:0/3980796030 learned_addr learned my addr 172.21.15.86:0/3980796030 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:35.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.082+0000 7f20c96ce640 1 -- 172.21.15.86:0/3980796030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20c4107c90 msgr2=0x7f20c4100f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:35.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.082+0000 7f20c9ecf640 1 --2- 172.21.15.86:0/3980796030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20c4107c90 0x7f20c4100f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:35.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.082+0000 7f20c8ecd640 1 --2- 172.21.15.86:0/3980796030 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20c4105360 0x7f20c41009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:35.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.083+0000 7f20c96ce640 1 --2- 172.21.15.86:0/3980796030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20c4107c90 0x7f20c4100f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.083+0000 7f20c96ce640 1 -- 172.21.15.86:0/3980796030 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20c4105360 msgr2=0x7f20c41009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:35.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.083+0000 7f20c96ce640 1 --2- 172.21.15.86:0/3980796030 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20c4105360 0x7f20c41009e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.083+0000 7f20c96ce640 1 -- 172.21.15.86:0/3980796030 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20c41017b0 con 0x7f20c4102a30 2024-04-21T08:36:35.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.083+0000 7f20c9ecf640 1 --2- 172.21.15.86:0/3980796030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20c4107c90 0x7f20c4100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:36:35.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.083+0000 7f20c96ce640 1 --2- 172.21.15.86:0/3980796030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20c4102a30 0x7f20c41023a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f20c0005e30 tx=0x7f20c0034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:35.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.083+0000 7f20ba7fc640 1 -- 172.21.15.86:0/3980796030 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c0039040 con 0x7f20c4102a30 2024-04-21T08:36:35.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.083+0000 7f20cb932640 1 -- 172.21.15.86:0/3980796030 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20c41a2320 con 0x7f20c4102a30 2024-04-21T08:36:35.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.083+0000 7f20ba7fc640 1 -- 172.21.15.86:0/3980796030 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20c00325c0 con 0x7f20c4102a30 2024-04-21T08:36:35.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.083+0000 7f20ba7fc640 1 -- 172.21.15.86:0/3980796030 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20c00042a0 con 0x7f20c4102a30 2024-04-21T08:36:35.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.083+0000 7f20cb932640 1 -- 172.21.15.86:0/3980796030 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20c41a2840 con 0x7f20c4102a30 2024-04-21T08:36:35.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.084+0000 7f20cb932640 1 -- 172.21.15.86:0/3980796030 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20c40691c0 con 0x7f20c4102a30 2024-04-21T08:36:35.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.085+0000 7f20ba7fc640 1 -- 172.21.15.86:0/3980796030 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f20c0046030 con 0x7f20c4102a30 2024-04-21T08:36:35.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.085+0000 7f20ba7fc640 1 --2- 172.21.15.86:0/3980796030 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f20b00a3bb0 0x7f20b00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:35.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.086+0000 7f20c8ecd640 1 --2- 172.21.15.86:0/3980796030 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f20b00a3bb0 0x7f20b00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:35.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.086+0000 7f20ba7fc640 1 -- 172.21.15.86:0/3980796030 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f20c00f25c0 con 0x7f20c4102a30 2024-04-21T08:36:35.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.086+0000 7f20c8ecd640 1 --2- 172.21.15.86:0/3980796030 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f20b00a3bb0 0x7f20b00a6070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f20c4102110 tx=0x7f20b4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:35.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.089+0000 7f20ba7fc640 1 -- 172.21.15.86:0/3980796030 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f20c004d060 con 0x7f20c4102a30 2024-04-21T08:36:35.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.268+0000 7f20cb932640 1 -- 172.21.15.86:0/3980796030 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f20c4062b50 con 0x7f20c4102a30 2024-04-21T08:36:35.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.269+0000 7f20ba7fc640 1 -- 172.21.15.86:0/3980796030 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v89) ==== 104+0+11 (secure 0 0 0) 0x7f20c00bedf0 con 0x7f20c4102a30 2024-04-21T08:36:35.269 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-21T08:36:35.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.271+0000 7f20cb932640 1 -- 172.21.15.86:0/3980796030 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f20b00a3bb0 msgr2=0x7f20b00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:35.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.271+0000 7f20cb932640 1 --2- 172.21.15.86:0/3980796030 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f20b00a3bb0 0x7f20b00a6070 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f20c4102110 tx=0x7f20b4005f90 comp rx=0 tx=0).stop 2024-04-21T08:36:35.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.271+0000 7f20cb932640 1 -- 172.21.15.86:0/3980796030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20c4102a30 msgr2=0x7f20c41023a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:35.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.271+0000 7f20cb932640 1 --2- 172.21.15.86:0/3980796030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20c4102a30 0x7f20c41023a0 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f20c0005e30 tx=0x7f20c0034760 comp rx=0 tx=0).stop 2024-04-21T08:36:35.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.271+0000 7f20cb932640 1 -- 172.21.15.86:0/3980796030 shutdown_connections 2024-04-21T08:36:35.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.271+0000 7f20cb932640 1 --2- 172.21.15.86:0/3980796030 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f20b00a3bb0 0x7f20b00a6070 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.271+0000 7f20cb932640 1 --2- 172.21.15.86:0/3980796030 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20c4107c90 0x7f20c4100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.271+0000 7f20cb932640 1 --2- 172.21.15.86:0/3980796030 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20c4105360 0x7f20c41009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.271+0000 7f20cb932640 1 --2- 172.21.15.86:0/3980796030 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20c4102a30 0x7f20c41023a0 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.271+0000 7f20cb932640 1 -- 172.21.15.86:0/3980796030 >> 172.21.15.86:0/3980796030 conn(0x7f20c40fc440 msgr2=0x7f20c41034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:35.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.271+0000 7f20cb932640 1 -- 172.21.15.86:0/3980796030 shutdown_connections 2024-04-21T08:36:35.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.271+0000 7f20cb932640 1 -- 172.21.15.86:0/3980796030 wait complete. 2024-04-21T08:36:35.282 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:36:35.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.384+0000 7fa778eba640 1 -- 172.21.15.86:0/3270421028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa774101100 msgr2=0x7fa774101580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:35.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.384+0000 7fa778eba640 1 --2- 172.21.15.86:0/3270421028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa774101100 0x7fa774101580 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7fa75c009900 tx=0x7fa75c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:35.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.384+0000 7fa778eba640 1 -- 172.21.15.86:0/3270421028 shutdown_connections 2024-04-21T08:36:35.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.384+0000 7fa778eba640 1 --2- 172.21.15.86:0/3270421028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa774101ac0 0x7fa774109360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.384+0000 7fa778eba640 1 --2- 172.21.15.86:0/3270421028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa774101100 0x7fa774101580 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.384+0000 7fa778eba640 1 --2- 172.21.15.86:0/3270421028 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7741024b0 0x7fa7741028d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.384+0000 7fa778eba640 1 -- 172.21.15.86:0/3270421028 >> 172.21.15.86:0/3270421028 conn(0x7fa7740fbb90 msgr2=0x7fa7740fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:35.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.385+0000 7fa778eba640 1 -- 172.21.15.86:0/3270421028 shutdown_connections 2024-04-21T08:36:35.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.385+0000 7fa778eba640 1 -- 172.21.15.86:0/3270421028 wait complete. 2024-04-21T08:36:35.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.385+0000 7fa778eba640 1 Processor -- start 2024-04-21T08:36:35.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.385+0000 7fa778eba640 1 -- start start 2024-04-21T08:36:35.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.385+0000 7fa778eba640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa774101100 0x7fa774197290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:35.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.385+0000 7fa778eba640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa774101ac0 0x7fa7741977d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:35.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.385+0000 7fa778eba640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7741024b0 0x7fa77419e840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:35.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.385+0000 7fa778eba640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa77410cb30 con 0x7fa774101ac0 2024-04-21T08:36:35.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.385+0000 7fa778eba640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa77410c9b0 con 0x7fa7741024b0 2024-04-21T08:36:35.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.385+0000 7fa778eba640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa77410ccb0 con 0x7fa774101100 2024-04-21T08:36:35.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.386+0000 7fa76bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa774101ac0 0x7fa7741977d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:35.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.386+0000 7fa76bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa774101ac0 0x7fa7741977d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36438/0 (socket says 172.21.15.86:36438) 2024-04-21T08:36:35.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.386+0000 7fa76bfff640 1 -- 172.21.15.86:0/41443359 learned_addr learned my addr 172.21.15.86:0/41443359 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:35.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.386+0000 7fa77259c640 1 --2- 172.21.15.86:0/41443359 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa774101100 0x7fa774197290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:35.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.386+0000 7fa76bfff640 1 -- 172.21.15.86:0/41443359 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa774101100 msgr2=0x7fa774197290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:35.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.386+0000 7fa76bfff640 1 --2- 172.21.15.86:0/41443359 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa774101100 0x7fa774197290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.386+0000 7fa772d9d640 1 --2- 172.21.15.86:0/41443359 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7741024b0 0x7fa77419e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:35.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.386+0000 7fa76bfff640 1 -- 172.21.15.86:0/41443359 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7741024b0 msgr2=0x7fa77419e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:35.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.386+0000 7fa76bfff640 1 --2- 172.21.15.86:0/41443359 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7741024b0 0x7fa77419e840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.386+0000 7fa76bfff640 1 -- 172.21.15.86:0/41443359 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa77419eeb0 con 0x7fa774101ac0 2024-04-21T08:36:35.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.386+0000 7fa76bfff640 1 --2- 172.21.15.86:0/41443359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa774101ac0 0x7fa7741977d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa75c005ab0 tx=0x7fa75c0042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:35.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.386+0000 7fa76b7fe640 1 -- 172.21.15.86:0/41443359 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75c0044e0 con 0x7fa774101ac0 2024-04-21T08:36:35.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.386+0000 7fa76b7fe640 1 -- 172.21.15.86:0/41443359 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa75c0035f0 con 0x7fa774101ac0 2024-04-21T08:36:35.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.386+0000 7fa778eba640 1 -- 172.21.15.86:0/41443359 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7740ffff0 con 0x7fa774101ac0 2024-04-21T08:36:35.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.386+0000 7fa76b7fe640 1 -- 172.21.15.86:0/41443359 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa75c0476e0 con 0x7fa774101ac0 2024-04-21T08:36:35.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.386+0000 7fa778eba640 1 -- 172.21.15.86:0/41443359 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa774100590 con 0x7fa774101ac0 2024-04-21T08:36:35.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.387+0000 7fa778eba640 1 -- 172.21.15.86:0/41443359 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7740691c0 con 0x7fa774101ac0 2024-04-21T08:36:35.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.389+0000 7fa76b7fe640 1 -- 172.21.15.86:0/41443359 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa75c0037a0 con 0x7fa774101ac0 2024-04-21T08:36:35.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.389+0000 7fa76b7fe640 1 --2- 172.21.15.86:0/41443359 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa7600a3bb0 0x7fa7600a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:35.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.390+0000 7fa77259c640 1 --2- 172.21.15.86:0/41443359 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa7600a3bb0 0x7fa7600a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:35.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.390+0000 7fa76b7fe640 1 -- 172.21.15.86:0/41443359 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fa75c0f4e20 con 0x7fa774101ac0 2024-04-21T08:36:35.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.390+0000 7fa77259c640 1 --2- 172.21.15.86:0/41443359 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa7600a3bb0 0x7fa7600a6070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa77406a350 tx=0x7fa764009110 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:35.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.393+0000 7fa76b7fe640 1 -- 172.21.15.86:0/41443359 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa75c0c15c0 con 0x7fa774101ac0 2024-04-21T08:36:35.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.572+0000 7fa778eba640 1 -- 172.21.15.86:0/41443359 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa774108030 con 0x7fa774101ac0 2024-04-21T08:36:35.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.572+0000 7fa76b7fe640 1 -- 172.21.15.86:0/41443359 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v89) ==== 100+0+11 (secure 0 0 0) 0x7fa75c0c0ec0 con 0x7fa774101ac0 2024-04-21T08:36:35.573 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-21T08:36:35.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.574+0000 7fa778eba640 1 -- 172.21.15.86:0/41443359 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa7600a3bb0 msgr2=0x7fa7600a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:35.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.574+0000 7fa778eba640 1 --2- 172.21.15.86:0/41443359 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa7600a3bb0 0x7fa7600a6070 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fa77406a350 tx=0x7fa764009110 comp rx=0 tx=0).stop 2024-04-21T08:36:35.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.574+0000 7fa778eba640 1 -- 172.21.15.86:0/41443359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa774101ac0 msgr2=0x7fa7741977d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:35.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.574+0000 7fa778eba640 1 --2- 172.21.15.86:0/41443359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa774101ac0 0x7fa7741977d0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fa75c005ab0 tx=0x7fa75c0042f0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.575+0000 7fa778eba640 1 -- 172.21.15.86:0/41443359 shutdown_connections 2024-04-21T08:36:35.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.575+0000 7fa778eba640 1 --2- 172.21.15.86:0/41443359 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa7600a3bb0 0x7fa7600a6070 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.575+0000 7fa778eba640 1 --2- 172.21.15.86:0/41443359 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7741024b0 0x7fa77419e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.575+0000 7fa778eba640 1 --2- 172.21.15.86:0/41443359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa774101ac0 0x7fa7741977d0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.575+0000 7fa778eba640 1 --2- 172.21.15.86:0/41443359 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa774101100 0x7fa774197290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.575+0000 7fa778eba640 1 -- 172.21.15.86:0/41443359 >> 172.21.15.86:0/41443359 conn(0x7fa7740fbb90 msgr2=0x7fa774105580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:35.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.575+0000 7fa778eba640 1 -- 172.21.15.86:0/41443359 shutdown_connections 2024-04-21T08:36:35.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.575+0000 7fa778eba640 1 -- 172.21.15.86:0/41443359 wait complete. 2024-04-21T08:36:35.585 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:35 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/125315564' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:35.585 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:35 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2657689500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:36:35.585 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:35 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3980796030' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:36:35.586 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:36:35.631 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:35 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/125315564' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:35.631 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:35 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2657689500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:36:35.631 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:35 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3980796030' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:36:35.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.685+0000 7fe4668d2640 1 -- 172.21.15.86:0/357075920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe460107c70 msgr2=0x7fe46010a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:35.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.685+0000 7fe4668d2640 1 --2- 172.21.15.86:0/357075920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe460107c70 0x7fe46010a0c0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fe458009ca0 tx=0x7fe45802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.685+0000 7fe4668d2640 1 -- 172.21.15.86:0/357075920 shutdown_connections 2024-04-21T08:36:35.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.685+0000 7fe4668d2640 1 --2- 172.21.15.86:0/357075920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe460107c70 0x7fe46010a0c0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.685+0000 7fe4668d2640 1 --2- 172.21.15.86:0/357075920 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe460105340 0x7fe460107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.685+0000 7fe4668d2640 1 --2- 172.21.15.86:0/357075920 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe460102a10 0x7fe460104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.685+0000 7fe4668d2640 1 -- 172.21.15.86:0/357075920 >> 172.21.15.86:0/357075920 conn(0x7fe4600fc400 msgr2=0x7fe4600fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:35.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.685+0000 7fe4668d2640 1 -- 172.21.15.86:0/357075920 shutdown_connections 2024-04-21T08:36:35.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.685+0000 7fe4668d2640 1 -- 172.21.15.86:0/357075920 wait complete. 2024-04-21T08:36:35.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.686+0000 7fe4668d2640 1 Processor -- start 2024-04-21T08:36:35.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.686+0000 7fe4668d2640 1 -- start start 2024-04-21T08:36:35.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.686+0000 7fe4668d2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe460102a10 0x7fe4601981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:35.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.686+0000 7fe4668d2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe460105340 0x7fe4601986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:35.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.686+0000 7fe4668d2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe460107c70 0x7fe46019f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:35.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.686+0000 7fe4668d2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe46010dae0 con 0x7fe460107c70 2024-04-21T08:36:35.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.686+0000 7fe4668d2640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe46010d960 con 0x7fe460105340 2024-04-21T08:36:35.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.686+0000 7fe4668d2640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe46010dc60 con 0x7fe460102a10 2024-04-21T08:36:35.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.686+0000 7fe464e6f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe460107c70 0x7fe46019f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:35.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.686+0000 7fe464e6f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe460107c70 0x7fe46019f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36456/0 (socket says 172.21.15.86:36456) 2024-04-21T08:36:35.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.686+0000 7fe464e6f640 1 -- 172.21.15.86:0/1721554538 learned_addr learned my addr 172.21.15.86:0/1721554538 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:35.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.687+0000 7fe45f7fe640 1 --2- 172.21.15.86:0/1721554538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe460105340 0x7fe4601986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:35.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.687+0000 7fe45ffff640 1 --2- 172.21.15.86:0/1721554538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe460102a10 0x7fe4601981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:35.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.687+0000 7fe464e6f640 1 -- 172.21.15.86:0/1721554538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe460102a10 msgr2=0x7fe4601981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:35.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.687+0000 7fe464e6f640 1 --2- 172.21.15.86:0/1721554538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe460102a10 0x7fe4601981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.687+0000 7fe464e6f640 1 -- 172.21.15.86:0/1721554538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe460105340 msgr2=0x7fe4601986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:35.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.687+0000 7fe464e6f640 1 --2- 172.21.15.86:0/1721554538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe460105340 0x7fe4601986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.687+0000 7fe464e6f640 1 -- 172.21.15.86:0/1721554538 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe46019fdd0 con 0x7fe460107c70 2024-04-21T08:36:35.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.687+0000 7fe45f7fe640 1 --2- 172.21.15.86:0/1721554538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe460105340 0x7fe4601986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:36:35.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.687+0000 7fe464e6f640 1 --2- 172.21.15.86:0/1721554538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe460107c70 0x7fe46019f760 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe45802c700 tx=0x7fe4580093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:35.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.687+0000 7fe45d7fa640 1 -- 172.21.15.86:0/1721554538 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe458041020 con 0x7fe460107c70 2024-04-21T08:36:35.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.687+0000 7fe45d7fa640 1 -- 172.21.15.86:0/1721554538 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe458034d60 con 0x7fe460107c70 2024-04-21T08:36:35.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.687+0000 7fe45d7fa640 1 -- 172.21.15.86:0/1721554538 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe458039040 con 0x7fe460107c70 2024-04-21T08:36:35.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.687+0000 7fe4668d2640 1 -- 172.21.15.86:0/1721554538 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe4601a0060 con 0x7fe460107c70 2024-04-21T08:36:35.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.687+0000 7fe4668d2640 1 -- 172.21.15.86:0/1721554538 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe4601a0550 con 0x7fe460107c70 2024-04-21T08:36:35.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.688+0000 7fe4668d2640 1 -- 172.21.15.86:0/1721554538 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe4600fdef0 con 0x7fe460107c70 2024-04-21T08:36:35.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.689+0000 7fe45d7fa640 1 -- 172.21.15.86:0/1721554538 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fe458044020 con 0x7fe460107c70 2024-04-21T08:36:35.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.690+0000 7fe45d7fa640 1 --2- 172.21.15.86:0/1721554538 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe4480a3bb0 0x7fe4480a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:35.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.690+0000 7fe45d7fa640 1 -- 172.21.15.86:0/1721554538 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fe4580f2640 con 0x7fe460107c70 2024-04-21T08:36:35.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.690+0000 7fe45ffff640 1 --2- 172.21.15.86:0/1721554538 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe4480a3bb0 0x7fe4480a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:35.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.690+0000 7fe45ffff640 1 --2- 172.21.15.86:0/1721554538 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe4480a3bb0 0x7fe4480a6070 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe450005fd0 tx=0x7fe450005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:35.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.694+0000 7fe45d7fa640 1 -- 172.21.15.86:0/1721554538 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe45804b020 con 0x7fe460107c70 2024-04-21T08:36:35.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:35 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/125315564' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:35.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:35 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2657689500' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:36:35.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:35 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3980796030' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:36:35.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.869+0000 7fe4668d2640 1 -- 172.21.15.86:0/1721554538 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fe460062b50 con 0x7fe460107c70 2024-04-21T08:36:35.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.870+0000 7fe45d7fa640 1 -- 172.21.15.86:0/1721554538 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v89) ==== 103+0+11 (secure 0 0 0) 0x7fe4580bede0 con 0x7fe460107c70 2024-04-21T08:36:35.870 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-21T08:36:35.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.872+0000 7fe4668d2640 1 -- 172.21.15.86:0/1721554538 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe4480a3bb0 msgr2=0x7fe4480a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:35.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.872+0000 7fe4668d2640 1 --2- 172.21.15.86:0/1721554538 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe4480a3bb0 0x7fe4480a6070 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7fe450005fd0 tx=0x7fe450005850 comp rx=0 tx=0).stop 2024-04-21T08:36:35.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.872+0000 7fe4668d2640 1 -- 172.21.15.86:0/1721554538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe460107c70 msgr2=0x7fe46019f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:35.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.872+0000 7fe4668d2640 1 --2- 172.21.15.86:0/1721554538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe460107c70 0x7fe46019f760 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7fe45802c700 tx=0x7fe4580093f0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.872+0000 7fe4668d2640 1 -- 172.21.15.86:0/1721554538 shutdown_connections 2024-04-21T08:36:35.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.872+0000 7fe4668d2640 1 --2- 172.21.15.86:0/1721554538 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe4480a3bb0 0x7fe4480a6070 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.872+0000 7fe4668d2640 1 --2- 172.21.15.86:0/1721554538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe460107c70 0x7fe46019f760 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.872+0000 7fe4668d2640 1 --2- 172.21.15.86:0/1721554538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe460105340 0x7fe4601986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.872+0000 7fe4668d2640 1 --2- 172.21.15.86:0/1721554538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe460102a10 0x7fe4601981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.872+0000 7fe4668d2640 1 -- 172.21.15.86:0/1721554538 >> 172.21.15.86:0/1721554538 conn(0x7fe4600fc400 msgr2=0x7fe4601032b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:35.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.872+0000 7fe4668d2640 1 -- 172.21.15.86:0/1721554538 shutdown_connections 2024-04-21T08:36:35.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.872+0000 7fe4668d2640 1 -- 172.21.15.86:0/1721554538 wait complete. 2024-04-21T08:36:35.883 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-21T08:36:35.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.982+0000 7fdb8d4fd640 1 -- 172.21.15.86:0/3674240899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb88107c70 msgr2=0x7fdb8810a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:35.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.982+0000 7fdb8d4fd640 1 --2- 172.21.15.86:0/3674240899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb88107c70 0x7fdb8810a0c0 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7fdb80009900 tx=0x7fdb8002bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:35.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.982+0000 7fdb8d4fd640 1 -- 172.21.15.86:0/3674240899 shutdown_connections 2024-04-21T08:36:35.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.982+0000 7fdb8d4fd640 1 --2- 172.21.15.86:0/3674240899 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb88107c70 0x7fdb8810a0c0 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.982+0000 7fdb8d4fd640 1 --2- 172.21.15.86:0/3674240899 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb88105340 0x7fdb88107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.982+0000 7fdb8d4fd640 1 --2- 172.21.15.86:0/3674240899 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb88102a10 0x7fdb88104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.982+0000 7fdb8d4fd640 1 -- 172.21.15.86:0/3674240899 >> 172.21.15.86:0/3674240899 conn(0x7fdb880fc400 msgr2=0x7fdb880fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:35.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.982+0000 7fdb8d4fd640 1 -- 172.21.15.86:0/3674240899 shutdown_connections 2024-04-21T08:36:35.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.982+0000 7fdb8d4fd640 1 -- 172.21.15.86:0/3674240899 wait complete. 2024-04-21T08:36:35.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.983+0000 7fdb8d4fd640 1 Processor -- start 2024-04-21T08:36:35.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.983+0000 7fdb8d4fd640 1 -- start start 2024-04-21T08:36:35.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.983+0000 7fdb8d4fd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb88102a10 0x7fdb881981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:35.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.983+0000 7fdb8d4fd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb88105340 0x7fdb88198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:35.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.983+0000 7fdb8d4fd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb88107c70 0x7fdb8819f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:35.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.983+0000 7fdb8d4fd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8810dac0 con 0x7fdb88107c70 2024-04-21T08:36:35.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.983+0000 7fdb8d4fd640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8810d940 con 0x7fdb88105340 2024-04-21T08:36:35.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.983+0000 7fdb8d4fd640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdb8810dc40 con 0x7fdb88102a10 2024-04-21T08:36:35.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.983+0000 7fdb877fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb88107c70 0x7fdb8819f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:35.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.983+0000 7fdb867fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb88105340 0x7fdb88198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:35.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.983+0000 7fdb877fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb88107c70 0x7fdb8819f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36466/0 (socket says 172.21.15.86:36466) 2024-04-21T08:36:35.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.983+0000 7fdb877fe640 1 -- 172.21.15.86:0/3755376302 learned_addr learned my addr 172.21.15.86:0/3755376302 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:35.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.984+0000 7fdb86ffd640 1 --2- 172.21.15.86:0/3755376302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb88102a10 0x7fdb881981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:35.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.984+0000 7fdb867fc640 1 -- 172.21.15.86:0/3755376302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb88102a10 msgr2=0x7fdb881981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:35.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.984+0000 7fdb867fc640 1 --2- 172.21.15.86:0/3755376302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb88102a10 0x7fdb881981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.984+0000 7fdb867fc640 1 -- 172.21.15.86:0/3755376302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb88107c70 msgr2=0x7fdb8819f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:35.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.984+0000 7fdb867fc640 1 --2- 172.21.15.86:0/3755376302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb88107c70 0x7fdb8819f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:35.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.984+0000 7fdb867fc640 1 -- 172.21.15.86:0/3755376302 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb8819fde0 con 0x7fdb88105340 2024-04-21T08:36:35.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.984+0000 7fdb877fe640 1 --2- 172.21.15.86:0/3755376302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb88107c70 0x7fdb8819f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:36:35.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.984+0000 7fdb867fc640 1 --2- 172.21.15.86:0/3755376302 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb88105340 0x7fdb88198700 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c00e9c0 tx=0x7fdb7c00ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:35.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.984+0000 7fdb6ffff640 1 -- 172.21.15.86:0/3755376302 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c00ccc0 con 0x7fdb88105340 2024-04-21T08:36:35.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.984+0000 7fdb8d4fd640 1 -- 172.21.15.86:0/3755376302 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb881a00d0 con 0x7fdb88105340 2024-04-21T08:36:35.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.984+0000 7fdb6ffff640 1 -- 172.21.15.86:0/3755376302 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb7c0108e0 con 0x7fdb88105340 2024-04-21T08:36:35.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.984+0000 7fdb6ffff640 1 -- 172.21.15.86:0/3755376302 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb7c0187c0 con 0x7fdb88105340 2024-04-21T08:36:35.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.984+0000 7fdb8d4fd640 1 -- 172.21.15.86:0/3755376302 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb881a0620 con 0x7fdb88105340 2024-04-21T08:36:35.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.985+0000 7fdb8d4fd640 1 -- 172.21.15.86:0/3755376302 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb880fdef0 con 0x7fdb88105340 2024-04-21T08:36:35.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.986+0000 7fdb6ffff640 1 -- 172.21.15.86:0/3755376302 <== mon.2 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fdb7c010430 con 0x7fdb88105340 2024-04-21T08:36:35.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.987+0000 7fdb6ffff640 1 --2- 172.21.15.86:0/3755376302 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fdb700a3bb0 0x7fdb700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:35.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.987+0000 7fdb86ffd640 1 --2- 172.21.15.86:0/3755376302 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fdb700a3bb0 0x7fdb700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:35.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.987+0000 7fdb6ffff640 1 -- 172.21.15.86:0/3755376302 <== mon.2 v2:172.21.15.88:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fdb7c0d0e20 con 0x7fdb88105340 2024-04-21T08:36:35.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.987+0000 7fdb86ffd640 1 --2- 172.21.15.86:0/3755376302 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fdb700a3bb0 0x7fdb700a6070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdb78005fd0 tx=0x7fdb78009520 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:35.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:35.991+0000 7fdb6ffff640 1 -- 172.21.15.86:0/3755376302 <== mon.2 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdb7c09d5c0 con 0x7fdb88105340 2024-04-21T08:36:36.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:36.180+0000 7fdb8d4fd640 1 -- 172.21.15.86:0/3755376302 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fdb8802d5c0 con 0x7fdb88105340 2024-04-21T08:36:36.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:36.181+0000 7fdb6ffff640 1 -- 172.21.15.86:0/3755376302 <== mon.2 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 10 entries v89) ==== 81+0+592 (secure 0 0 0) 0x7fdb7c09cec0 con 0x7fdb88105340 2024-04-21T08:36:36.181 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/2143206409 2024-04-22T08:29:23.190001+0000 2024-04-21T08:36:36.181 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1412393225 2024-04-22T08:29:23.190001+0000 2024-04-21T08:36:36.181 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/2386990413 2024-04-22T08:28:57.384403+0000 2024-04-21T08:36:36.181 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1687852404 2024-04-22T08:29:23.190001+0000 2024-04-21T08:36:36.181 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/2386990413 2024-04-22T08:28:57.384403+0000 2024-04-21T08:36:36.181 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2020707404 2024-04-22T08:28:57.384403+0000 2024-04-21T08:36:36.181 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2568130933 2024-04-22T08:28:57.384403+0000 2024-04-21T08:36:36.182 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3952607648 2024-04-22T08:28:57.384403+0000 2024-04-21T08:36:36.182 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/2143206409 2024-04-22T08:29:23.190001+0000 2024-04-21T08:36:36.182 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2756903426 2024-04-22T08:29:23.190001+0000 2024-04-21T08:36:36.182 INFO:teuthology.orchestra.run.smithi086.stderr:listed 10 entries 2024-04-21T08:36:36.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:36.183+0000 7fdb8d4fd640 1 -- 172.21.15.86:0/3755376302 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fdb700a3bb0 msgr2=0x7fdb700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:36.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:36.183+0000 7fdb8d4fd640 1 --2- 172.21.15.86:0/3755376302 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fdb700a3bb0 0x7fdb700a6070 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fdb78005fd0 tx=0x7fdb78009520 comp rx=0 tx=0).stop 2024-04-21T08:36:36.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:36.183+0000 7fdb8d4fd640 1 -- 172.21.15.86:0/3755376302 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb88105340 msgr2=0x7fdb88198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:36.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:36.183+0000 7fdb8d4fd640 1 --2- 172.21.15.86:0/3755376302 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb88105340 0x7fdb88198700 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7fdb7c00e9c0 tx=0x7fdb7c00ee80 comp rx=0 tx=0).stop 2024-04-21T08:36:36.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:36.183+0000 7fdb8d4fd640 1 -- 172.21.15.86:0/3755376302 shutdown_connections 2024-04-21T08:36:36.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:36.183+0000 7fdb8d4fd640 1 --2- 172.21.15.86:0/3755376302 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fdb700a3bb0 0x7fdb700a6070 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:36.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:36.183+0000 7fdb8d4fd640 1 --2- 172.21.15.86:0/3755376302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb88107c70 0x7fdb8819f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:36.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:36.183+0000 7fdb8d4fd640 1 --2- 172.21.15.86:0/3755376302 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb88105340 0x7fdb88198700 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:36.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:36.183+0000 7fdb8d4fd640 1 --2- 172.21.15.86:0/3755376302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb88102a10 0x7fdb881981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:36.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:36.183+0000 7fdb8d4fd640 1 -- 172.21.15.86:0/3755376302 >> 172.21.15.86:0/3755376302 conn(0x7fdb880fc400 msgr2=0x7fdb88103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:36.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:36.183+0000 7fdb8d4fd640 1 -- 172.21.15.86:0/3755376302 shutdown_connections 2024-04-21T08:36:36.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:36.183+0000 7fdb8d4fd640 1 -- 172.21.15.86:0/3755376302 wait complete. 2024-04-21T08:36:36.194 INFO:tasks.cephfs.fuse_mount:Running fusermount -u on ubuntu@smithi086.front.sepia.ceph.com... 2024-04-21T08:36:36.194 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:36:36.194 DEBUG:teuthology.orchestra.run.smithi086:> sudo fusermount -u /home/ubuntu/cephtest/mnt.admin 2024-04-21T08:36:36.216 INFO:tasks.cephfs.fuse_mount.ceph-fuse.admin.smithi086.stderr:ceph-fuse[69339]: fuse finished with error 0 and tester_r 0 2024-04-21T08:36:36.217 INFO:teuthology.orchestra.run:waiting for 300 2024-04-21T08:36:36.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:36 smithi086 ceph-mon[27939]: pgmap v295: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 307 B/s wr, 0 op/s 2024-04-21T08:36:36.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:36 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/41443359' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:36:36.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:36 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1721554538' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:36:36.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:36 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3755376302' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-21T08:36:36.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:36 smithi088 ceph-mon[31259]: pgmap v295: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 307 B/s wr, 0 op/s 2024-04-21T08:36:36.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:36 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/41443359' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:36:36.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:36 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1721554538' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:36:36.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:36 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3755376302' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-21T08:36:36.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:36 smithi179 ceph-mon[31829]: pgmap v295: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 307 B/s wr, 0 op/s 2024-04-21T08:36:36.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:36 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/41443359' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:36:36.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:36 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1721554538' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:36:36.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:36 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3755376302' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-21T08:36:38.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:38 smithi086 ceph-mon[27939]: pgmap v296: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 259 B/s wr, 0 op/s 2024-04-21T08:36:38.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:38 smithi088 ceph-mon[31259]: pgmap v296: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 259 B/s wr, 0 op/s 2024-04-21T08:36:38.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:38 smithi179 ceph-mon[31829]: pgmap v296: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 259 B/s wr, 0 op/s 2024-04-21T08:36:40.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:40 smithi086 ceph-mon[27939]: pgmap v297: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-04-21T08:36:40.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:40 smithi088 ceph-mon[31259]: pgmap v297: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-04-21T08:36:40.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:40 smithi179 ceph-mon[31829]: pgmap v297: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 0 op/s 2024-04-21T08:36:42.320 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-21T08:36:42.320 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:36:42.321 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.admin) 2024-04-21T08:36:42.354 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:36:42.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.505+0000 7fa576324640 1 -- 172.21.15.86:0/1573997917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa570105330 msgr2=0x7fa570107720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:42.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.505+0000 7fa576324640 1 --2- 172.21.15.86:0/1573997917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa570105330 0x7fa570107720 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fa564009900 tx=0x7fa56402bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:42.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.505+0000 7fa576324640 1 -- 172.21.15.86:0/1573997917 shutdown_connections 2024-04-21T08:36:42.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.505+0000 7fa576324640 1 --2- 172.21.15.86:0/1573997917 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa570107c60 0x7fa57010a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:42.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.505+0000 7fa576324640 1 --2- 172.21.15.86:0/1573997917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa570105330 0x7fa570107720 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:42.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.505+0000 7fa576324640 1 --2- 172.21.15.86:0/1573997917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa570102a00 0x7fa570104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:42.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.505+0000 7fa576324640 1 -- 172.21.15.86:0/1573997917 >> 172.21.15.86:0/1573997917 conn(0x7fa5700fc410 msgr2=0x7fa5700fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:42.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.505+0000 7fa576324640 1 -- 172.21.15.86:0/1573997917 shutdown_connections 2024-04-21T08:36:42.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.505+0000 7fa576324640 1 -- 172.21.15.86:0/1573997917 wait complete. 2024-04-21T08:36:42.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.505+0000 7fa576324640 1 Processor -- start 2024-04-21T08:36:42.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.505+0000 7fa576324640 1 -- start start 2024-04-21T08:36:42.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.506+0000 7fa576324640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa570102a00 0x7fa5701981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:42.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.506+0000 7fa576324640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa570105330 0x7fa5701986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:42.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.506+0000 7fa576324640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa570107c60 0x7fa57019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:42.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.506+0000 7fa576324640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa57010daa0 con 0x7fa570105330 2024-04-21T08:36:42.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.506+0000 7fa576324640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa57010d920 con 0x7fa570102a00 2024-04-21T08:36:42.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.506+0000 7fa576324640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa57010dc20 con 0x7fa570107c60 2024-04-21T08:36:42.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.506+0000 7fa56f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa570105330 0x7fa5701986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:42.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.506+0000 7fa56f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa570105330 0x7fa5701986e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35036/0 (socket says 172.21.15.86:35036) 2024-04-21T08:36:42.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.506+0000 7fa56f7fe640 1 -- 172.21.15.86:0/3121347832 learned_addr learned my addr 172.21.15.86:0/3121347832 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:42.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.506+0000 7fa56ffff640 1 --2- 172.21.15.86:0/3121347832 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa570102a00 0x7fa5701981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:42.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.506+0000 7fa5748c1640 1 --2- 172.21.15.86:0/3121347832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa570107c60 0x7fa57019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:42.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.506+0000 7fa56f7fe640 1 -- 172.21.15.86:0/3121347832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa570107c60 msgr2=0x7fa57019f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:42.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.506+0000 7fa56f7fe640 1 --2- 172.21.15.86:0/3121347832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa570107c60 0x7fa57019f750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:42.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.506+0000 7fa56f7fe640 1 -- 172.21.15.86:0/3121347832 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa570102a00 msgr2=0x7fa5701981a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:42.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.506+0000 7fa56f7fe640 1 --2- 172.21.15.86:0/3121347832 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa570102a00 0x7fa5701981a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:42.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.506+0000 7fa56f7fe640 1 -- 172.21.15.86:0/3121347832 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa57019fdc0 con 0x7fa570105330 2024-04-21T08:36:42.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.506+0000 7fa56f7fe640 1 --2- 172.21.15.86:0/3121347832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa570105330 0x7fa5701986e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa564000c00 tx=0x7fa564004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:42.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.506+0000 7fa56d7fa640 1 -- 172.21.15.86:0/3121347832 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa564043020 con 0x7fa570105330 2024-04-21T08:36:42.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.506+0000 7fa56d7fa640 1 -- 172.21.15.86:0/3121347832 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa564004aa0 con 0x7fa570105330 2024-04-21T08:36:42.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.506+0000 7fa576324640 1 -- 172.21.15.86:0/3121347832 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5701a0050 con 0x7fa570105330 2024-04-21T08:36:42.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.506+0000 7fa56d7fa640 1 -- 172.21.15.86:0/3121347832 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa564003d00 con 0x7fa570105330 2024-04-21T08:36:42.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.506+0000 7fa576324640 1 -- 172.21.15.86:0/3121347832 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5701a05a0 con 0x7fa570105330 2024-04-21T08:36:42.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.507+0000 7fa576324640 1 -- 172.21.15.86:0/3121347832 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa5700fdf00 con 0x7fa570105330 2024-04-21T08:36:42.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.508+0000 7fa56d7fa640 1 -- 172.21.15.86:0/3121347832 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fa564046020 con 0x7fa570105330 2024-04-21T08:36:42.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.508+0000 7fa56d7fa640 1 --2- 172.21.15.86:0/3121347832 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa5580a3bb0 0x7fa5580a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:42.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.508+0000 7fa56ffff640 1 --2- 172.21.15.86:0/3121347832 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa5580a3bb0 0x7fa5580a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:42.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.508+0000 7fa56d7fa640 1 -- 172.21.15.86:0/3121347832 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fa5640f2c80 con 0x7fa570105330 2024-04-21T08:36:42.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.510+0000 7fa56ffff640 1 --2- 172.21.15.86:0/3121347832 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa5580a3bb0 0x7fa5580a6070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa560005fd0 tx=0x7fa560005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:42.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.514+0000 7fa56d7fa640 1 -- 172.21.15.86:0/3121347832 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa56404d020 con 0x7fa570105330 2024-04-21T08:36:42.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:42 smithi086 ceph-mon[27939]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-21T08:36:42.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.692+0000 7fa576324640 1 -- 172.21.15.86:0/3121347832 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fa570062b50 con 0x7fa570105330 2024-04-21T08:36:42.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.694+0000 7fa56d7fa640 1 -- 172.21.15.86:0/3121347832 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24070 (secure 0 0 0) 0x7fa5640bf420 con 0x7fa570105330 2024-04-21T08:36:42.694 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:36:42.694 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":89,"fsid":"f8cec7cc-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:22.687329+0000","modified":"2024-04-21T08:36:25.251657+0000","last_up_change":"2024-04-21T08:35:30.048315+0000","last_in_change":"2024-04-21T08:35:10.099496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:45.279109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:18.300782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:19.574636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:22.905632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89b7601f-0620-4db2-90f1-b45cea173738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6803","nonce":2571516897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6805","nonce":2571516897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6809","nonce":2571516897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6807","nonce":2571516897}]},"public_addr":"172.21.15.86:6803/2571516897","cluster_addr":"172.21.15.86:6805/2571516897","heartbeat_back_addr":"172.21.15.86:6809/2571516897","heartbeat_front_addr":"172.21.15.86:6807/2571516897","state":["exists","up"]},{"osd":1,"uuid":"4891031c-f98c-42bc-bb3b-a420855ecc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6801","nonce":74067182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6803","nonce":74067182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6807","nonce":74067182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6805","nonce":74067182}]},"public_addr":"172.21.15.88:6801/74067182","cluster_addr":"172.21.15.88:6803/74067182","heartbeat_back_addr":"172.21.15.88:6807/74067182","heartbeat_front_addr":"172.21.15.88:6805/74067182","state":["exists","up"]},{"osd":2,"uuid":"60d12aa4-9312-4fa2-8d0b-2b3f42caad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6801","nonce":4137659051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6803","nonce":4137659051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6807","nonce":4137659051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6805","nonce":4137659051}]},"public_addr":"172.21.15.179:6801/4137659051","cluster_addr":"172.21.15.179:6803/4137659051","heartbeat_back_addr":"172.21.15.179:6807/4137659051","heartbeat_front_addr":"172.21.15.179:6805/4137659051","state":["exists","up"]},{"osd":3,"uuid":"404c3158-8cb0-4c02-b48a-f9ee5ff7b784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6811","nonce":2309603148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6813","nonce":2309603148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6817","nonce":2309603148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6815","nonce":2309603148}]},"public_addr":"172.21.15.86:6811/2309603148","cluster_addr":"172.21.15.86:6813/2309603148","heartbeat_back_addr":"172.21.15.86:6817/2309603148","heartbeat_front_addr":"172.21.15.86:6815/2309603148","state":["exists","up"]},{"osd":4,"uuid":"458ff202-d781-4ec7-ac93-0b49f835b503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6809","nonce":1818714217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6811","nonce":1818714217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6815","nonce":1818714217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6813","nonce":1818714217}]},"public_addr":"172.21.15.88:6809/1818714217","cluster_addr":"172.21.15.88:6811/1818714217","heartbeat_back_addr":"172.21.15.88:6815/1818714217","heartbeat_front_addr":"172.21.15.88:6813/1818714217","state":["exists","up"]},{"osd":5,"uuid":"8123cc90-bd8b-4204-bf31-46b34e9f589d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6809","nonce":863840990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6811","nonce":863840990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6815","nonce":863840990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6813","nonce":863840990}]},"public_addr":"172.21.15.179:6809/863840990","cluster_addr":"172.21.15.179:6811/863840990","heartbeat_back_addr":"172.21.15.179:6815/863840990","heartbeat_front_addr":"172.21.15.179:6813/863840990","state":["exists","up"]},{"osd":6,"uuid":"aac9d7c7-659a-4110-903a-a871c086066f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6819","nonce":1224231118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6821","nonce":1224231118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6825","nonce":1224231118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6823","nonce":1224231118}]},"public_addr":"172.21.15.86:6819/1224231118","cluster_addr":"172.21.15.86:6821/1224231118","heartbeat_back_addr":"172.21.15.86:6825/1224231118","heartbeat_front_addr":"172.21.15.86:6823/1224231118","state":["exists","up"]},{"osd":7,"uuid":"df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6817","nonce":136629878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6819","nonce":136629878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6823","nonce":136629878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6821","nonce":136629878}]},"public_addr":"172.21.15.88:6817/136629878","cluster_addr":"172.21.15.88:6819/136629878","heartbeat_back_addr":"172.21.15.88:6823/136629878","heartbeat_front_addr":"172.21.15.88:6821/136629878","state":["exists","up"]},{"osd":8,"uuid":"63e59f4b-0426-46db-92f7-c55dae61b5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6817","nonce":1770418957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6819","nonce":1770418957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6823","nonce":1770418957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6821","nonce":1770418957}]},"public_addr":"172.21.15.179:6817/1770418957","cluster_addr":"172.21.15.179:6819/1770418957","heartbeat_back_addr":"172.21.15.179:6823/1770418957","heartbeat_front_addr":"172.21.15.179:6821/1770418957","state":["exists","up"]},{"osd":9,"uuid":"5841bb36-ed6b-45d8-b857-eeab139c44e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6827","nonce":1108500274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6829","nonce":1108500274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6833","nonce":1108500274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6831","nonce":1108500274}]},"public_addr":"172.21.15.86:6827/1108500274","cluster_addr":"172.21.15.86:6829/1108500274","heartbeat_back_addr":"172.21.15.86:6833/1108500274","heartbeat_front_addr":"172.21.15.86:6831/1108500274","state":["exists","up"]},{"osd":10,"uuid":"f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6825","nonce":1294275353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6827","nonce":1294275353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6831","nonce":1294275353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6829","nonce":1294275353}]},"public_addr":"172.21.15.88:6825/1294275353","cluster_addr":"172.21.15.88:6827/1294275353","heartbeat_back_addr":"172.21.15.88:6831/1294275353","heartbeat_front_addr":"172.21.15.88:6829/1294275353","state":["exists","up"]},{"osd":11,"uuid":"e0d4cc76-e7a6-4a6b-909c-95488ccd5a36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6825","nonce":4007070233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6827","nonce":4007070233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6831","nonce":4007070233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6829","nonce":4007070233}]},"public_addr":"172.21.15.179:6825/4007070233","cluster_addr":"172.21.15.179:6827/4007070233","heartbeat_back_addr":"172.21.15.179:6831/4007070233","heartbeat_front_addr":"172.21.15.179:6829/4007070233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:55.210605+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:18.723090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:42.547979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:06.421622+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:30.858315+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:55.455313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.623369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:46.185111+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.103627+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:37.297545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:02.637431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:28.199292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6800/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/1412393225":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6800/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/1687852404":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2020707404":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2568130933":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/3952607648":"2024-04-22T08:28:57.384403+0000","172.21.15.86:6801/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/2756903426":"2024-04-22T08:29:23.190001+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:36:42.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.696+0000 7fa576324640 1 -- 172.21.15.86:0/3121347832 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa5580a3bb0 msgr2=0x7fa5580a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:42.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.696+0000 7fa576324640 1 --2- 172.21.15.86:0/3121347832 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa5580a3bb0 0x7fa5580a6070 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fa560005fd0 tx=0x7fa560005850 comp rx=0 tx=0).stop 2024-04-21T08:36:42.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.696+0000 7fa576324640 1 -- 172.21.15.86:0/3121347832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa570105330 msgr2=0x7fa5701986e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:42.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.696+0000 7fa576324640 1 --2- 172.21.15.86:0/3121347832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa570105330 0x7fa5701986e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fa564000c00 tx=0x7fa564004270 comp rx=0 tx=0).stop 2024-04-21T08:36:42.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.696+0000 7fa576324640 1 -- 172.21.15.86:0/3121347832 shutdown_connections 2024-04-21T08:36:42.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.696+0000 7fa576324640 1 --2- 172.21.15.86:0/3121347832 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa5580a3bb0 0x7fa5580a6070 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:42.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.696+0000 7fa576324640 1 --2- 172.21.15.86:0/3121347832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa570107c60 0x7fa57019f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:42.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.696+0000 7fa576324640 1 --2- 172.21.15.86:0/3121347832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa570105330 0x7fa5701986e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:42.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.696+0000 7fa576324640 1 --2- 172.21.15.86:0/3121347832 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa570102a00 0x7fa5701981a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:42.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.696+0000 7fa576324640 1 -- 172.21.15.86:0/3121347832 >> 172.21.15.86:0/3121347832 conn(0x7fa5700fc410 msgr2=0x7fa5701034a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:42.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.696+0000 7fa576324640 1 -- 172.21.15.86:0/3121347832 shutdown_connections 2024-04-21T08:36:42.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.696+0000 7fa576324640 1 -- 172.21.15.86:0/3121347832 wait complete. 2024-04-21T08:36:42.707 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:36:42.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:42 smithi088 ceph-mon[31259]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-21T08:36:42.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:42 smithi179 ceph-mon[31829]: pgmap v298: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-21T08:36:42.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.806+0000 7f62602c8640 1 -- 172.21.15.86:0/924907126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6258074930 msgr2=0x7f6258074d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:42.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.806+0000 7f62602c8640 1 --2- 172.21.15.86:0/924907126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6258074930 0x7f6258074d90 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f6248009900 tx=0x7f624802bcd0 comp rx=0 tx=0).stop 2024-04-21T08:36:42.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.806+0000 7f62602c8640 1 -- 172.21.15.86:0/924907126 shutdown_connections 2024-04-21T08:36:42.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.806+0000 7f62602c8640 1 --2- 172.21.15.86:0/924907126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6258109e40 0x7f625810c2d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:42.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.806+0000 7f62602c8640 1 --2- 172.21.15.86:0/924907126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6258074930 0x7f6258074d90 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:42.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.806+0000 7f62602c8640 1 --2- 172.21.15.86:0/924907126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6258075f10 0x7f6258074360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:42.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.806+0000 7f62602c8640 1 -- 172.21.15.86:0/924907126 >> 172.21.15.86:0/924907126 conn(0x7f62580fd500 msgr2=0x7f62580ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:42.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.806+0000 7f62602c8640 1 -- 172.21.15.86:0/924907126 shutdown_connections 2024-04-21T08:36:42.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.806+0000 7f62602c8640 1 -- 172.21.15.86:0/924907126 wait complete. 2024-04-21T08:36:42.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.807+0000 7f62602c8640 1 Processor -- start 2024-04-21T08:36:42.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.807+0000 7f62602c8640 1 -- start start 2024-04-21T08:36:42.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.807+0000 7f62602c8640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6258074930 0x7f625819a3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:42.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.807+0000 7f62602c8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6258075f10 0x7f625819a930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:42.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.807+0000 7f62602c8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6258109e40 0x7f62581a19a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:42.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.807+0000 7f62602c8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f625810fd20 con 0x7f6258109e40 2024-04-21T08:36:42.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.807+0000 7f62602c8640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f625810fba0 con 0x7f6258075f10 2024-04-21T08:36:42.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.807+0000 7f62602c8640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f625810fea0 con 0x7f6258074930 2024-04-21T08:36:42.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.807+0000 7f625e865640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6258109e40 0x7f62581a19a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:42.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.807+0000 7f625e865640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6258109e40 0x7f62581a19a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35050/0 (socket says 172.21.15.86:35050) 2024-04-21T08:36:42.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.807+0000 7f625e865640 1 -- 172.21.15.86:0/876345700 learned_addr learned my addr 172.21.15.86:0/876345700 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:42.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.807+0000 7f625d863640 1 --2- 172.21.15.86:0/876345700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6258075f10 0x7f625819a930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:42.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.807+0000 7f625e064640 1 --2- 172.21.15.86:0/876345700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6258074930 0x7f625819a3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:42.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.808+0000 7f625e865640 1 -- 172.21.15.86:0/876345700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6258074930 msgr2=0x7f625819a3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:42.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.808+0000 7f625e865640 1 --2- 172.21.15.86:0/876345700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6258074930 0x7f625819a3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:42.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.808+0000 7f625e865640 1 -- 172.21.15.86:0/876345700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6258075f10 msgr2=0x7f625819a930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:42.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.808+0000 7f625e865640 1 --2- 172.21.15.86:0/876345700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6258075f10 0x7f625819a930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:42.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.808+0000 7f625e865640 1 -- 172.21.15.86:0/876345700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f62581a2010 con 0x7f6258109e40 2024-04-21T08:36:42.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.808+0000 7f625e865640 1 --2- 172.21.15.86:0/876345700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6258109e40 0x7f62581a19a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f625000b970 tx=0x7f625000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:42.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.808+0000 7f624f7fe640 1 -- 172.21.15.86:0/876345700 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6250019070 con 0x7f6258109e40 2024-04-21T08:36:42.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.808+0000 7f624f7fe640 1 -- 172.21.15.86:0/876345700 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6250014440 con 0x7f6258109e40 2024-04-21T08:36:42.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.808+0000 7f62602c8640 1 -- 172.21.15.86:0/876345700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f62581a2300 con 0x7f6258109e40 2024-04-21T08:36:42.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.808+0000 7f624f7fe640 1 -- 172.21.15.86:0/876345700 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f62500075d0 con 0x7f6258109e40 2024-04-21T08:36:42.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.808+0000 7f62602c8640 1 -- 172.21.15.86:0/876345700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f62581a2850 con 0x7f6258109e40 2024-04-21T08:36:42.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.809+0000 7f62602c8640 1 -- 172.21.15.86:0/876345700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6258108c00 con 0x7f6258109e40 2024-04-21T08:36:42.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.810+0000 7f624f7fe640 1 -- 172.21.15.86:0/876345700 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f625001c070 con 0x7f6258109e40 2024-04-21T08:36:42.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.810+0000 7f624f7fe640 1 --2- 172.21.15.86:0/876345700 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f62440a3bb0 0x7f62440a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:42.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.811+0000 7f625e064640 1 --2- 172.21.15.86:0/876345700 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f62440a3bb0 0x7f62440a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:42.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.811+0000 7f624f7fe640 1 -- 172.21.15.86:0/876345700 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f62500cfbc0 con 0x7f6258109e40 2024-04-21T08:36:42.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.811+0000 7f625e064640 1 --2- 172.21.15.86:0/876345700 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f62440a3bb0 0x7f62440a6070 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6254005fd0 tx=0x7f6254005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:42.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.815+0000 7f624f7fe640 1 -- 172.21.15.86:0/876345700 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f625009bd10 con 0x7f6258109e40 2024-04-21T08:36:42.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.994+0000 7f62602c8640 1 -- 172.21.15.86:0/876345700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f6258062b50 con 0x7f6258109e40 2024-04-21T08:36:42.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.995+0000 7f624f7fe640 1 -- 172.21.15.86:0/876345700 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24070 (secure 0 0 0) 0x7f62500bae20 con 0x7f6258109e40 2024-04-21T08:36:42.995 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:36:42.995 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":89,"fsid":"f8cec7cc-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:22.687329+0000","modified":"2024-04-21T08:36:25.251657+0000","last_up_change":"2024-04-21T08:35:30.048315+0000","last_in_change":"2024-04-21T08:35:10.099496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:45.279109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:18.300782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:19.574636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:22.905632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89b7601f-0620-4db2-90f1-b45cea173738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6803","nonce":2571516897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6805","nonce":2571516897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6809","nonce":2571516897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6807","nonce":2571516897}]},"public_addr":"172.21.15.86:6803/2571516897","cluster_addr":"172.21.15.86:6805/2571516897","heartbeat_back_addr":"172.21.15.86:6809/2571516897","heartbeat_front_addr":"172.21.15.86:6807/2571516897","state":["exists","up"]},{"osd":1,"uuid":"4891031c-f98c-42bc-bb3b-a420855ecc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6801","nonce":74067182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6803","nonce":74067182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6807","nonce":74067182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6805","nonce":74067182}]},"public_addr":"172.21.15.88:6801/74067182","cluster_addr":"172.21.15.88:6803/74067182","heartbeat_back_addr":"172.21.15.88:6807/74067182","heartbeat_front_addr":"172.21.15.88:6805/74067182","state":["exists","up"]},{"osd":2,"uuid":"60d12aa4-9312-4fa2-8d0b-2b3f42caad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6801","nonce":4137659051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6803","nonce":4137659051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6807","nonce":4137659051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6805","nonce":4137659051}]},"public_addr":"172.21.15.179:6801/4137659051","cluster_addr":"172.21.15.179:6803/4137659051","heartbeat_back_addr":"172.21.15.179:6807/4137659051","heartbeat_front_addr":"172.21.15.179:6805/4137659051","state":["exists","up"]},{"osd":3,"uuid":"404c3158-8cb0-4c02-b48a-f9ee5ff7b784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6811","nonce":2309603148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6813","nonce":2309603148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6817","nonce":2309603148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6815","nonce":2309603148}]},"public_addr":"172.21.15.86:6811/2309603148","cluster_addr":"172.21.15.86:6813/2309603148","heartbeat_back_addr":"172.21.15.86:6817/2309603148","heartbeat_front_addr":"172.21.15.86:6815/2309603148","state":["exists","up"]},{"osd":4,"uuid":"458ff202-d781-4ec7-ac93-0b49f835b503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6809","nonce":1818714217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6811","nonce":1818714217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6815","nonce":1818714217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6813","nonce":1818714217}]},"public_addr":"172.21.15.88:6809/1818714217","cluster_addr":"172.21.15.88:6811/1818714217","heartbeat_back_addr":"172.21.15.88:6815/1818714217","heartbeat_front_addr":"172.21.15.88:6813/1818714217","state":["exists","up"]},{"osd":5,"uuid":"8123cc90-bd8b-4204-bf31-46b34e9f589d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6809","nonce":863840990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6811","nonce":863840990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6815","nonce":863840990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6813","nonce":863840990}]},"public_addr":"172.21.15.179:6809/863840990","cluster_addr":"172.21.15.179:6811/863840990","heartbeat_back_addr":"172.21.15.179:6815/863840990","heartbeat_front_addr":"172.21.15.179:6813/863840990","state":["exists","up"]},{"osd":6,"uuid":"aac9d7c7-659a-4110-903a-a871c086066f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6819","nonce":1224231118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6821","nonce":1224231118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6825","nonce":1224231118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6823","nonce":1224231118}]},"public_addr":"172.21.15.86:6819/1224231118","cluster_addr":"172.21.15.86:6821/1224231118","heartbeat_back_addr":"172.21.15.86:6825/1224231118","heartbeat_front_addr":"172.21.15.86:6823/1224231118","state":["exists","up"]},{"osd":7,"uuid":"df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6817","nonce":136629878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6819","nonce":136629878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6823","nonce":136629878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6821","nonce":136629878}]},"public_addr":"172.21.15.88:6817/136629878","cluster_addr":"172.21.15.88:6819/136629878","heartbeat_back_addr":"172.21.15.88:6823/136629878","heartbeat_front_addr":"172.21.15.88:6821/136629878","state":["exists","up"]},{"osd":8,"uuid":"63e59f4b-0426-46db-92f7-c55dae61b5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6817","nonce":1770418957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6819","nonce":1770418957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6823","nonce":1770418957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6821","nonce":1770418957}]},"public_addr":"172.21.15.179:6817/1770418957","cluster_addr":"172.21.15.179:6819/1770418957","heartbeat_back_addr":"172.21.15.179:6823/1770418957","heartbeat_front_addr":"172.21.15.179:6821/1770418957","state":["exists","up"]},{"osd":9,"uuid":"5841bb36-ed6b-45d8-b857-eeab139c44e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6827","nonce":1108500274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6829","nonce":1108500274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6833","nonce":1108500274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6831","nonce":1108500274}]},"public_addr":"172.21.15.86:6827/1108500274","cluster_addr":"172.21.15.86:6829/1108500274","heartbeat_back_addr":"172.21.15.86:6833/1108500274","heartbeat_front_addr":"172.21.15.86:6831/1108500274","state":["exists","up"]},{"osd":10,"uuid":"f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6825","nonce":1294275353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6827","nonce":1294275353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6831","nonce":1294275353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6829","nonce":1294275353}]},"public_addr":"172.21.15.88:6825/1294275353","cluster_addr":"172.21.15.88:6827/1294275353","heartbeat_back_addr":"172.21.15.88:6831/1294275353","heartbeat_front_addr":"172.21.15.88:6829/1294275353","state":["exists","up"]},{"osd":11,"uuid":"e0d4cc76-e7a6-4a6b-909c-95488ccd5a36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6825","nonce":4007070233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6827","nonce":4007070233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6831","nonce":4007070233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6829","nonce":4007070233}]},"public_addr":"172.21.15.179:6825/4007070233","cluster_addr":"172.21.15.179:6827/4007070233","heartbeat_back_addr":"172.21.15.179:6831/4007070233","heartbeat_front_addr":"172.21.15.179:6829/4007070233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:55.210605+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:18.723090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:42.547979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:06.421622+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:30.858315+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:55.455313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.623369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:46.185111+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.103627+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:37.297545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:02.637431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:28.199292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6800/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/1412393225":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6800/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/1687852404":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2020707404":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2568130933":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/3952607648":"2024-04-22T08:28:57.384403+0000","172.21.15.86:6801/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/2756903426":"2024-04-22T08:29:23.190001+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:36:42.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.997+0000 7f62602c8640 1 -- 172.21.15.86:0/876345700 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f62440a3bb0 msgr2=0x7f62440a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:42.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.997+0000 7f62602c8640 1 --2- 172.21.15.86:0/876345700 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f62440a3bb0 0x7f62440a6070 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f6254005fd0 tx=0x7f6254005850 comp rx=0 tx=0).stop 2024-04-21T08:36:42.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.997+0000 7f62602c8640 1 -- 172.21.15.86:0/876345700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6258109e40 msgr2=0x7f62581a19a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:42.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.997+0000 7f62602c8640 1 --2- 172.21.15.86:0/876345700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6258109e40 0x7f62581a19a0 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f625000b970 tx=0x7f625000be30 comp rx=0 tx=0).stop 2024-04-21T08:36:42.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.997+0000 7f62602c8640 1 -- 172.21.15.86:0/876345700 shutdown_connections 2024-04-21T08:36:42.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.997+0000 7f62602c8640 1 --2- 172.21.15.86:0/876345700 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f62440a3bb0 0x7f62440a6070 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:42.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.997+0000 7f62602c8640 1 --2- 172.21.15.86:0/876345700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6258109e40 0x7f62581a19a0 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:42.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.997+0000 7f62602c8640 1 --2- 172.21.15.86:0/876345700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6258075f10 0x7f625819a930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:42.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.997+0000 7f62602c8640 1 --2- 172.21.15.86:0/876345700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6258074930 0x7f625819a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:42.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.997+0000 7f62602c8640 1 -- 172.21.15.86:0/876345700 >> 172.21.15.86:0/876345700 conn(0x7f62580fd500 msgr2=0x7f6258105dd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:42.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.998+0000 7f62602c8640 1 -- 172.21.15.86:0/876345700 shutdown_connections 2024-04-21T08:36:42.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:42.998+0000 7f62602c8640 1 -- 172.21.15.86:0/876345700 wait complete. 2024-04-21T08:36:43.008 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs standby_count_wanted 0 2024-04-21T08:36:43.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.110+0000 7f3d44192640 1 -- 172.21.15.86:0/2437344012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d3c101fb0 msgr2=0x7f3d3c1023d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:43.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.110+0000 7f3d44192640 1 --2- 172.21.15.86:0/2437344012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d3c101fb0 0x7f3d3c1023d0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c009900 tx=0x7f3d2c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:43.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.110+0000 7f3d44192640 1 -- 172.21.15.86:0/2437344012 shutdown_connections 2024-04-21T08:36:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.110+0000 7f3d44192640 1 --2- 172.21.15.86:0/2437344012 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d3c103b60 0x7f3d3c10a3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.110+0000 7f3d44192640 1 --2- 172.21.15.86:0/2437344012 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d3c1031a0 0x7f3d3c103620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.110+0000 7f3d44192640 1 --2- 172.21.15.86:0/2437344012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d3c101fb0 0x7f3d3c1023d0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.110+0000 7f3d44192640 1 -- 172.21.15.86:0/2437344012 >> 172.21.15.86:0/2437344012 conn(0x7f3d3c0fd520 msgr2=0x7f3d3c0ff960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.110+0000 7f3d44192640 1 -- 172.21.15.86:0/2437344012 shutdown_connections 2024-04-21T08:36:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.110+0000 7f3d44192640 1 -- 172.21.15.86:0/2437344012 wait complete. 2024-04-21T08:36:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.110+0000 7f3d44192640 1 Processor -- start 2024-04-21T08:36:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.111+0000 7f3d44192640 1 -- start start 2024-04-21T08:36:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.111+0000 7f3d44192640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d3c101fb0 0x7f3d3c198270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.111+0000 7f3d44192640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d3c1031a0 0x7f3d3c1987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.111+0000 7f3d44192640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d3c103b60 0x7f3d3c19f820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.111+0000 7f3d44192640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3c10db30 con 0x7f3d3c103b60 2024-04-21T08:36:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.111+0000 7f3d44192640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3c10d9b0 con 0x7f3d3c1031a0 2024-04-21T08:36:43.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.111+0000 7f3d44192640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3d3c10dcb0 con 0x7f3d3c101fb0 2024-04-21T08:36:43.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.111+0000 7f3d4272f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d3c103b60 0x7f3d3c19f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:43.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.111+0000 7f3d4272f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d3c103b60 0x7f3d3c19f820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35076/0 (socket says 172.21.15.86:35076) 2024-04-21T08:36:43.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.111+0000 7f3d4272f640 1 -- 172.21.15.86:0/2048190447 learned_addr learned my addr 172.21.15.86:0/2048190447 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:43.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.111+0000 7f3d41f2e640 1 --2- 172.21.15.86:0/2048190447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d3c101fb0 0x7f3d3c198270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:43.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.111+0000 7f3d4172d640 1 --2- 172.21.15.86:0/2048190447 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d3c1031a0 0x7f3d3c1987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:43.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.111+0000 7f3d4272f640 1 -- 172.21.15.86:0/2048190447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d3c101fb0 msgr2=0x7f3d3c198270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:43.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.111+0000 7f3d4272f640 1 --2- 172.21.15.86:0/2048190447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d3c101fb0 0x7f3d3c198270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:43.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.111+0000 7f3d4272f640 1 -- 172.21.15.86:0/2048190447 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d3c1031a0 msgr2=0x7f3d3c1987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:43.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.111+0000 7f3d4272f640 1 --2- 172.21.15.86:0/2048190447 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d3c1031a0 0x7f3d3c1987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:43.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.111+0000 7f3d4272f640 1 -- 172.21.15.86:0/2048190447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3d3c19fe90 con 0x7f3d3c103b60 2024-04-21T08:36:43.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.111+0000 7f3d4272f640 1 --2- 172.21.15.86:0/2048190447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d3c103b60 0x7f3d3c19f820 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3d3400b970 tx=0x7f3d3400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:43.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.111+0000 7f3d32ffd640 1 -- 172.21.15.86:0/2048190447 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d34019070 con 0x7f3d3c103b60 2024-04-21T08:36:43.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.111+0000 7f3d32ffd640 1 -- 172.21.15.86:0/2048190447 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3d34014440 con 0x7f3d3c103b60 2024-04-21T08:36:43.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.111+0000 7f3d44192640 1 -- 172.21.15.86:0/2048190447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3d3c1a0180 con 0x7f3d3c103b60 2024-04-21T08:36:43.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.111+0000 7f3d44192640 1 -- 172.21.15.86:0/2048190447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3d3c1a0750 con 0x7f3d3c103b60 2024-04-21T08:36:43.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.112+0000 7f3d32ffd640 1 -- 172.21.15.86:0/2048190447 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3d34023060 con 0x7f3d3c103b60 2024-04-21T08:36:43.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.112+0000 7f3d44192640 1 -- 172.21.15.86:0/2048190447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3d3c109340 con 0x7f3d3c103b60 2024-04-21T08:36:43.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.114+0000 7f3d32ffd640 1 -- 172.21.15.86:0/2048190447 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f3d3401c070 con 0x7f3d3c103b60 2024-04-21T08:36:43.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.114+0000 7f3d32ffd640 1 --2- 172.21.15.86:0/2048190447 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f3d280a3bb0 0x7f3d280a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:43.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.115+0000 7f3d41f2e640 1 --2- 172.21.15.86:0/2048190447 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f3d280a3bb0 0x7f3d280a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:43.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.115+0000 7f3d32ffd640 1 -- 172.21.15.86:0/2048190447 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f3d340d0480 con 0x7f3d3c103b60 2024-04-21T08:36:43.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.115+0000 7f3d41f2e640 1 --2- 172.21.15.86:0/2048190447 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f3d280a3bb0 0x7f3d280a6070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c006010 tx=0x7f3d2c02c800 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:43.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.118+0000 7f3d32ffd640 1 -- 172.21.15.86:0/2048190447 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3d3409cc20 con 0x7f3d3c103b60 2024-04-21T08:36:43.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.339+0000 7f3d44192640 1 -- 172.21.15.86:0/2048190447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"} v 0) -- 0x7f3d3c02d5c0 con 0x7f3d3c103b60 2024-04-21T08:36:43.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.361+0000 7f3d32ffd640 1 -- 172.21.15.86:0/2048190447 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]=0 v18) ==== 118+0+0 (secure 0 0 0) 0x7f3d3409c520 con 0x7f3d3c103b60 2024-04-21T08:36:43.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.364+0000 7f3d44192640 1 -- 172.21.15.86:0/2048190447 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f3d280a3bb0 msgr2=0x7f3d280a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:43.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.364+0000 7f3d44192640 1 --2- 172.21.15.86:0/2048190447 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f3d280a3bb0 0x7f3d280a6070 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7f3d2c006010 tx=0x7f3d2c02c800 comp rx=0 tx=0).stop 2024-04-21T08:36:43.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.364+0000 7f3d44192640 1 -- 172.21.15.86:0/2048190447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d3c103b60 msgr2=0x7f3d3c19f820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:43.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.364+0000 7f3d44192640 1 --2- 172.21.15.86:0/2048190447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d3c103b60 0x7f3d3c19f820 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7f3d3400b970 tx=0x7f3d3400be30 comp rx=0 tx=0).stop 2024-04-21T08:36:43.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.364+0000 7f3d44192640 1 -- 172.21.15.86:0/2048190447 shutdown_connections 2024-04-21T08:36:43.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.364+0000 7f3d44192640 1 --2- 172.21.15.86:0/2048190447 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f3d280a3bb0 0x7f3d280a6070 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:43.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.364+0000 7f3d44192640 1 --2- 172.21.15.86:0/2048190447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3d3c103b60 0x7f3d3c19f820 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:43.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.364+0000 7f3d44192640 1 --2- 172.21.15.86:0/2048190447 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3d3c1031a0 0x7f3d3c1987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:43.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.364+0000 7f3d44192640 1 --2- 172.21.15.86:0/2048190447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3d3c101fb0 0x7f3d3c198270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:43.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.364+0000 7f3d44192640 1 -- 172.21.15.86:0/2048190447 >> 172.21.15.86:0/2048190447 conn(0x7f3d3c0fd520 msgr2=0x7f3d3c106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:43.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.364+0000 7f3d44192640 1 -- 172.21.15.86:0/2048190447 shutdown_connections 2024-04-21T08:36:43.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.364+0000 7f3d44192640 1 -- 172.21.15.86:0/2048190447 wait complete. 2024-04-21T08:36:43.374 DEBUG:tasks.cephfs.filesystem:fs_config: {'ec_profile': ['m=2', 'k=2', 'crush-failure-domain=osd'], 'max_mds': 5, 'session_timeout': 300, 'standby_replay': True, 'subvols': {'create': 2, 'subvol_options': '--size 25000000000'}} 2024-04-21T08:36:43.374 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs max_mds 5 2024-04-21T08:36:43.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.475+0000 7f0801750640 1 -- 172.21.15.86:0/2054568418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07fc107c70 msgr2=0x7f07fc10a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:43.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.475+0000 7f0801750640 1 --2- 172.21.15.86:0/2054568418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07fc107c70 0x7f07fc10a0c0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f07f4009900 tx=0x7f07f402bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:43.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.475+0000 7f0801750640 1 -- 172.21.15.86:0/2054568418 shutdown_connections 2024-04-21T08:36:43.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.475+0000 7f0801750640 1 --2- 172.21.15.86:0/2054568418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07fc107c70 0x7f07fc10a0c0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:43.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.475+0000 7f0801750640 1 --2- 172.21.15.86:0/2054568418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07fc105340 0x7f07fc107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:43.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.475+0000 7f0801750640 1 --2- 172.21.15.86:0/2054568418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07fc102a10 0x7f07fc104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:43.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.475+0000 7f0801750640 1 -- 172.21.15.86:0/2054568418 >> 172.21.15.86:0/2054568418 conn(0x7f07fc0fc400 msgr2=0x7f07fc0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:43.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.475+0000 7f0801750640 1 -- 172.21.15.86:0/2054568418 shutdown_connections 2024-04-21T08:36:43.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.475+0000 7f0801750640 1 -- 172.21.15.86:0/2054568418 wait complete. 2024-04-21T08:36:43.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.476+0000 7f0801750640 1 Processor -- start 2024-04-21T08:36:43.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.476+0000 7f0801750640 1 -- start start 2024-04-21T08:36:43.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.476+0000 7f0801750640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07fc102a10 0x7f07fc1981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:43.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.476+0000 7f0801750640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07fc105340 0x7f07fc198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:43.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.476+0000 7f0801750640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07fc107c70 0x7f07fc19f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:43.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.476+0000 7f0801750640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f07fc10dac0 con 0x7f07fc107c70 2024-04-21T08:36:43.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.476+0000 7f0801750640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f07fc10d940 con 0x7f07fc105340 2024-04-21T08:36:43.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.476+0000 7f0801750640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f07fc10dc40 con 0x7f07fc102a10 2024-04-21T08:36:43.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.476+0000 7f07fb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07fc107c70 0x7f07fc19f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:43.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.476+0000 7f07fb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07fc107c70 0x7f07fc19f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35090/0 (socket says 172.21.15.86:35090) 2024-04-21T08:36:43.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.476+0000 7f07fb7fe640 1 -- 172.21.15.86:0/2502661234 learned_addr learned my addr 172.21.15.86:0/2502661234 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:43.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.476+0000 7f07fa7fc640 1 --2- 172.21.15.86:0/2502661234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07fc105340 0x7f07fc198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:43.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.477+0000 7f07fb7fe640 1 -- 172.21.15.86:0/2502661234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07fc102a10 msgr2=0x7f07fc1981c0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T08:36:43.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.477+0000 7f07fb7fe640 1 --2- 172.21.15.86:0/2502661234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07fc102a10 0x7f07fc1981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:43.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.477+0000 7f07fb7fe640 1 -- 172.21.15.86:0/2502661234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07fc105340 msgr2=0x7f07fc198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:43.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.477+0000 7f07fb7fe640 1 --2- 172.21.15.86:0/2502661234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07fc105340 0x7f07fc198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:43.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.477+0000 7f07fb7fe640 1 -- 172.21.15.86:0/2502661234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f07fc19fde0 con 0x7f07fc107c70 2024-04-21T08:36:43.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.477+0000 7f07fa7fc640 1 --2- 172.21.15.86:0/2502661234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07fc105340 0x7f07fc198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:36:43.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.477+0000 7f07fb7fe640 1 --2- 172.21.15.86:0/2502661234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07fc107c70 0x7f07fc19f770 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f07f4009230 tx=0x7f07f40042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:43.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.477+0000 7f07e3fff640 1 -- 172.21.15.86:0/2502661234 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f4043020 con 0x7f07fc107c70 2024-04-21T08:36:43.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.477+0000 7f07e3fff640 1 -- 172.21.15.86:0/2502661234 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07f4004aa0 con 0x7f07fc107c70 2024-04-21T08:36:43.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.477+0000 7f0801750640 1 -- 172.21.15.86:0/2502661234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f07fc1a0070 con 0x7f07fc107c70 2024-04-21T08:36:43.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.477+0000 7f07e3fff640 1 -- 172.21.15.86:0/2502661234 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07f4003c70 con 0x7f07fc107c70 2024-04-21T08:36:43.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.477+0000 7f0801750640 1 -- 172.21.15.86:0/2502661234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07fc1a05c0 con 0x7f07fc107c70 2024-04-21T08:36:43.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.478+0000 7f0801750640 1 -- 172.21.15.86:0/2502661234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f07fc0fdef0 con 0x7f07fc107c70 2024-04-21T08:36:43.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.479+0000 7f07e3fff640 1 -- 172.21.15.86:0/2502661234 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f07f4046020 con 0x7f07fc107c70 2024-04-21T08:36:43.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.479+0000 7f07e3fff640 1 --2- 172.21.15.86:0/2502661234 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f07e40a3bb0 0x7f07e40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:43.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.480+0000 7f07faffd640 1 --2- 172.21.15.86:0/2502661234 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f07e40a3bb0 0x7f07e40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:43.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.480+0000 7f07e3fff640 1 -- 172.21.15.86:0/2502661234 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f07f40f2af0 con 0x7f07fc107c70 2024-04-21T08:36:43.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.480+0000 7f07faffd640 1 --2- 172.21.15.86:0/2502661234 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f07e40a3bb0 0x7f07e40a6070 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f07ec005fd0 tx=0x7f07ec009710 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:43.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.484+0000 7f07e3fff640 1 -- 172.21.15.86:0/2502661234 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f07f404d020 con 0x7f07fc107c70 2024-04-21T08:36:43.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:43 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3121347832' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:43.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:43 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/876345700' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:43.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:43 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2048190447' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-21T08:36:43.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:43.702+0000 7f0801750640 1 -- 172.21.15.86:0/2502661234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"} v 0) -- 0x7f07fc02d5c0 con 0x7f07fc107c70 2024-04-21T08:36:43.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:43 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3121347832' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:43.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:43 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/876345700' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:43.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:43 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2048190447' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-21T08:36:43.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:43 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3121347832' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:43.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:43 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/876345700' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:43.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:43 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2048190447' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]: dispatch 2024-04-21T08:36:44.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.372+0000 7f07e3fff640 1 -- 172.21.15.86:0/2502661234 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]=0 v19) ==== 105+0+0 (secure 0 0 0) 0x7f07fc02d5c0 con 0x7f07fc107c70 2024-04-21T08:36:44.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.376+0000 7f0801750640 1 -- 172.21.15.86:0/2502661234 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f07e40a3bb0 msgr2=0x7f07e40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:44.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.376+0000 7f0801750640 1 --2- 172.21.15.86:0/2502661234 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f07e40a3bb0 0x7f07e40a6070 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f07ec005fd0 tx=0x7f07ec009710 comp rx=0 tx=0).stop 2024-04-21T08:36:44.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.376+0000 7f0801750640 1 -- 172.21.15.86:0/2502661234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07fc107c70 msgr2=0x7f07fc19f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:44.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.376+0000 7f0801750640 1 --2- 172.21.15.86:0/2502661234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07fc107c70 0x7f07fc19f770 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f07f4009230 tx=0x7f07f40042f0 comp rx=0 tx=0).stop 2024-04-21T08:36:44.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.376+0000 7f0801750640 1 -- 172.21.15.86:0/2502661234 shutdown_connections 2024-04-21T08:36:44.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.376+0000 7f0801750640 1 --2- 172.21.15.86:0/2502661234 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f07e40a3bb0 0x7f07e40a6070 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:44.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.376+0000 7f0801750640 1 --2- 172.21.15.86:0/2502661234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f07fc107c70 0x7f07fc19f770 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:44.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.376+0000 7f0801750640 1 --2- 172.21.15.86:0/2502661234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07fc105340 0x7f07fc198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:44.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.376+0000 7f0801750640 1 --2- 172.21.15.86:0/2502661234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07fc102a10 0x7f07fc1981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:44.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.376+0000 7f0801750640 1 -- 172.21.15.86:0/2502661234 >> 172.21.15.86:0/2502661234 conn(0x7f07fc0fc400 msgr2=0x7f07fc1033f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:44.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.376+0000 7f0801750640 1 -- 172.21.15.86:0/2502661234 shutdown_connections 2024-04-21T08:36:44.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.376+0000 7f0801750640 1 -- 172.21.15.86:0/2502661234 wait complete. 2024-04-21T08:36:44.391 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs allow_standby_replay true 2024-04-21T08:36:44.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.498+0000 7f842da06640 1 -- 172.21.15.86:0/242219576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84281009c0 msgr2=0x7f842810a570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:44.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.498+0000 7f842da06640 1 --2- 172.21.15.86:0/242219576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84281009c0 0x7f842810a570 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f8420009ca0 tx=0x7f842002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:36:44.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.499+0000 7f842da06640 1 -- 172.21.15.86:0/242219576 shutdown_connections 2024-04-21T08:36:44.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.499+0000 7f842da06640 1 --2- 172.21.15.86:0/242219576 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84281009c0 0x7f842810a570 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:44.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.499+0000 7f842da06640 1 --2- 172.21.15.86:0/242219576 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8428100000 0x7f8428100480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:44.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.499+0000 7f842da06640 1 --2- 172.21.15.86:0/242219576 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8428103590 0x7f84281039b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:44.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.499+0000 7f842da06640 1 -- 172.21.15.86:0/242219576 >> 172.21.15.86:0/242219576 conn(0x7f84280fbb80 msgr2=0x7f84280fdfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:44.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.499+0000 7f842da06640 1 -- 172.21.15.86:0/242219576 shutdown_connections 2024-04-21T08:36:44.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.499+0000 7f842da06640 1 -- 172.21.15.86:0/242219576 wait complete. 2024-04-21T08:36:44.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.499+0000 7f842da06640 1 Processor -- start 2024-04-21T08:36:44.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.499+0000 7f842da06640 1 -- start start 2024-04-21T08:36:44.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.499+0000 7f842da06640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8428100000 0x7f8428198290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:44.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.499+0000 7f842da06640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84281009c0 0x7f84281987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:44.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.499+0000 7f842da06640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8428103590 0x7f842819f840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:44.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.499+0000 7f842da06640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f842810dbe0 con 0x7f8428103590 2024-04-21T08:36:44.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.499+0000 7f842da06640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f842810da60 con 0x7f8428100000 2024-04-21T08:36:44.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.499+0000 7f842da06640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f842810dd60 con 0x7f84281009c0 2024-04-21T08:36:44.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.500+0000 7f84277fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8428103590 0x7f842819f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:44.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.500+0000 7f84277fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8428103590 0x7f842819f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35106/0 (socket says 172.21.15.86:35106) 2024-04-21T08:36:44.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.500+0000 7f84277fe640 1 -- 172.21.15.86:0/2693403403 learned_addr learned my addr 172.21.15.86:0/2693403403 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:44.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.500+0000 7f8426ffd640 1 --2- 172.21.15.86:0/2693403403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8428100000 0x7f8428198290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:44.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.500+0000 7f84277fe640 1 -- 172.21.15.86:0/2693403403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84281009c0 msgr2=0x7f84281987d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:44.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.500+0000 7f84267fc640 1 --2- 172.21.15.86:0/2693403403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84281009c0 0x7f84281987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:44.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.500+0000 7f84277fe640 1 --2- 172.21.15.86:0/2693403403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84281009c0 0x7f84281987d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:44.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.500+0000 7f84277fe640 1 -- 172.21.15.86:0/2693403403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8428100000 msgr2=0x7f8428198290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:44.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.500+0000 7f84277fe640 1 --2- 172.21.15.86:0/2693403403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8428100000 0x7f8428198290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:44.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.500+0000 7f84277fe640 1 -- 172.21.15.86:0/2693403403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f842819feb0 con 0x7f8428103590 2024-04-21T08:36:44.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.500+0000 7f8426ffd640 1 --2- 172.21.15.86:0/2693403403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8428100000 0x7f8428198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:36:44.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.500+0000 7f84267fc640 1 --2- 172.21.15.86:0/2693403403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84281009c0 0x7f84281987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:36:44.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.500+0000 7f84277fe640 1 --2- 172.21.15.86:0/2693403403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8428103590 0x7f842819f840 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8420009dc0 tx=0x7f8420032870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:44.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.500+0000 7f842ca04640 1 -- 172.21.15.86:0/2693403403 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8420041020 con 0x7f8428103590 2024-04-21T08:36:44.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.500+0000 7f842da06640 1 -- 172.21.15.86:0/2693403403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84281a0140 con 0x7f8428103590 2024-04-21T08:36:44.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.500+0000 7f842da06640 1 -- 172.21.15.86:0/2693403403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84281a0700 con 0x7f8428103590 2024-04-21T08:36:44.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.500+0000 7f842ca04640 1 -- 172.21.15.86:0/2693403403 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8420003e20 con 0x7f8428103590 2024-04-21T08:36:44.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.500+0000 7f842ca04640 1 -- 172.21.15.86:0/2693403403 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8420039040 con 0x7f8428103590 2024-04-21T08:36:44.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.501+0000 7f842da06640 1 -- 172.21.15.86:0/2693403403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84140052c0 con 0x7f8428103590 2024-04-21T08:36:44.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.503+0000 7f842ca04640 1 -- 172.21.15.86:0/2693403403 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f8420044020 con 0x7f8428103590 2024-04-21T08:36:44.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.504+0000 7f842ca04640 1 --2- 172.21.15.86:0/2693403403 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f84100a3bb0 0x7f84100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:44.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.504+0000 7f8426ffd640 1 --2- 172.21.15.86:0/2693403403 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f84100a3bb0 0x7f84100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:44.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.504+0000 7f842ca04640 1 -- 172.21.15.86:0/2693403403 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f84200f2d70 con 0x7f8428103590 2024-04-21T08:36:44.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.504+0000 7f8426ffd640 1 --2- 172.21.15.86:0/2693403403 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f84100a3bb0 0x7f84100a6070 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f842806a270 tx=0x7f8418008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:44.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.507+0000 7f842ca04640 1 -- 172.21.15.86:0/2693403403 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f842004b020 con 0x7f8428103590 2024-04-21T08:36:44.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:44 smithi086 ceph-mon[27939]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-21T08:36:44.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:44 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2048190447' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-21T08:36:44.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:44 smithi086 ceph-mon[27939]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T08:36:44.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:44 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2502661234' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-21T08:36:44.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:44 smithi088 ceph-mon[31259]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-21T08:36:44.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:44 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2048190447' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-21T08:36:44.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:44 smithi088 ceph-mon[31259]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T08:36:44.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:44 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2502661234' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-21T08:36:44.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:44.731+0000 7f842da06640 1 -- 172.21.15.86:0/2693403403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"} v 0) -- 0x7f84140055b0 con 0x7f8428103590 2024-04-21T08:36:44.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:44 smithi179 ceph-mon[31829]: pgmap v299: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 341 B/s wr, 0 op/s 2024-04-21T08:36:44.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:44 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2048190447' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "standby_count_wanted", "val": "0"}]': finished 2024-04-21T08:36:44.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:44 smithi179 ceph-mon[31829]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T08:36:44.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:44 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2502661234' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]: dispatch 2024-04-21T08:36:45.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.385+0000 7f842ca04640 1 -- 172.21.15.86:0/2693403403 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]=0 v21) ==== 121+0+0 (secure 0 0 0) 0x7f84200bf510 con 0x7f8428103590 2024-04-21T08:36:45.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.388+0000 7f842da06640 1 -- 172.21.15.86:0/2693403403 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f84100a3bb0 msgr2=0x7f84100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:45.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.388+0000 7f842da06640 1 --2- 172.21.15.86:0/2693403403 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f84100a3bb0 0x7f84100a6070 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7f842806a270 tx=0x7f8418008040 comp rx=0 tx=0).stop 2024-04-21T08:36:45.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.388+0000 7f842da06640 1 -- 172.21.15.86:0/2693403403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8428103590 msgr2=0x7f842819f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:45.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.388+0000 7f842da06640 1 --2- 172.21.15.86:0/2693403403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8428103590 0x7f842819f840 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f8420009dc0 tx=0x7f8420032870 comp rx=0 tx=0).stop 2024-04-21T08:36:45.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.388+0000 7f842da06640 1 -- 172.21.15.86:0/2693403403 shutdown_connections 2024-04-21T08:36:45.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.388+0000 7f842da06640 1 --2- 172.21.15.86:0/2693403403 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f84100a3bb0 0x7f84100a6070 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:45.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.388+0000 7f842da06640 1 --2- 172.21.15.86:0/2693403403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8428103590 0x7f842819f840 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:45.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.388+0000 7f842da06640 1 --2- 172.21.15.86:0/2693403403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84281009c0 0x7f84281987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:45.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.388+0000 7f842da06640 1 --2- 172.21.15.86:0/2693403403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8428100000 0x7f8428198290 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:45.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.388+0000 7f842da06640 1 -- 172.21.15.86:0/2693403403 >> 172.21.15.86:0/2693403403 conn(0x7f84280fbb80 msgr2=0x7f8428106640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:45.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.389+0000 7f842da06640 1 -- 172.21.15.86:0/2693403403 shutdown_connections 2024-04-21T08:36:45.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.389+0000 7f842da06640 1 -- 172.21.15.86:0/2693403403 wait complete. 2024-04-21T08:36:45.402 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs set cephfs session_timeout 300 2024-04-21T08:36:45.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.511+0000 7f4e3f625640 1 -- 172.21.15.86:0/2004585260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e38100e90 msgr2=0x7f4e381012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:45.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.511+0000 7f4e3f625640 1 --2- 172.21.15.86:0/2004585260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e38100e90 0x7f4e381012b0 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7f4e28009900 tx=0x7f4e2802bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:45.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.512+0000 7f4e3f625640 1 -- 172.21.15.86:0/2004585260 shutdown_connections 2024-04-21T08:36:45.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.512+0000 7f4e3f625640 1 --2- 172.21.15.86:0/2004585260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4e381029a0 0x7f4e381092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:45.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.512+0000 7f4e3f625640 1 --2- 172.21.15.86:0/2004585260 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e38101fe0 0x7f4e38102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:45.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.512+0000 7f4e3f625640 1 --2- 172.21.15.86:0/2004585260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e38100e90 0x7f4e381012b0 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:45.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.512+0000 7f4e3f625640 1 -- 172.21.15.86:0/2004585260 >> 172.21.15.86:0/2004585260 conn(0x7f4e380fc400 msgr2=0x7f4e380fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:45.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.512+0000 7f4e3f625640 1 -- 172.21.15.86:0/2004585260 shutdown_connections 2024-04-21T08:36:45.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.512+0000 7f4e3f625640 1 -- 172.21.15.86:0/2004585260 wait complete. 2024-04-21T08:36:45.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.512+0000 7f4e3f625640 1 Processor -- start 2024-04-21T08:36:45.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.512+0000 7f4e3f625640 1 -- start start 2024-04-21T08:36:45.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.512+0000 7f4e3f625640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4e38100e90 0x7f4e38198300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:45.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.512+0000 7f4e3f625640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e38101fe0 0x7f4e38198840 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:45.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.512+0000 7f4e3f625640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e381029a0 0x7f4e3819f860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:45.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.512+0000 7f4e3f625640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3810ca40 con 0x7f4e381029a0 2024-04-21T08:36:45.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.512+0000 7f4e3f625640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3810c8c0 con 0x7f4e38101fe0 2024-04-21T08:36:45.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.512+0000 7f4e3f625640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4e3810cbc0 con 0x7f4e38100e90 2024-04-21T08:36:45.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.512+0000 7f4e3dbc2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e381029a0 0x7f4e3819f860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:45.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.512+0000 7f4e3dbc2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e381029a0 0x7f4e3819f860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35124/0 (socket says 172.21.15.86:35124) 2024-04-21T08:36:45.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.512+0000 7f4e3dbc2640 1 -- 172.21.15.86:0/1585924865 learned_addr learned my addr 172.21.15.86:0/1585924865 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:45.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.512+0000 7f4e3cbc0640 1 --2- 172.21.15.86:0/1585924865 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e38101fe0 0x7f4e38198840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:45.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.513+0000 7f4e3d3c1640 1 --2- 172.21.15.86:0/1585924865 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4e38100e90 0x7f4e38198300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:45.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.513+0000 7f4e3dbc2640 1 -- 172.21.15.86:0/1585924865 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4e38100e90 msgr2=0x7f4e38198300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:45.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.513+0000 7f4e3dbc2640 1 --2- 172.21.15.86:0/1585924865 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4e38100e90 0x7f4e38198300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:45.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.513+0000 7f4e3dbc2640 1 -- 172.21.15.86:0/1585924865 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e38101fe0 msgr2=0x7f4e38198840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:45.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.513+0000 7f4e3dbc2640 1 --2- 172.21.15.86:0/1585924865 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e38101fe0 0x7f4e38198840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:45.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.513+0000 7f4e3dbc2640 1 -- 172.21.15.86:0/1585924865 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e3819fed0 con 0x7f4e381029a0 2024-04-21T08:36:45.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.513+0000 7f4e3cbc0640 1 --2- 172.21.15.86:0/1585924865 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e38101fe0 0x7f4e38198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:36:45.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.513+0000 7f4e3d3c1640 1 --2- 172.21.15.86:0/1585924865 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4e38100e90 0x7f4e38198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:36:45.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.513+0000 7f4e3dbc2640 1 --2- 172.21.15.86:0/1585924865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e381029a0 0x7f4e3819f860 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4e3000b970 tx=0x7f4e3000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:45.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.513+0000 7f4e2e7fc640 1 -- 172.21.15.86:0/1585924865 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e30019070 con 0x7f4e381029a0 2024-04-21T08:36:45.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.513+0000 7f4e2e7fc640 1 -- 172.21.15.86:0/1585924865 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e30014440 con 0x7f4e381029a0 2024-04-21T08:36:45.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.513+0000 7f4e3f625640 1 -- 172.21.15.86:0/1585924865 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e381a01c0 con 0x7f4e381029a0 2024-04-21T08:36:45.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.513+0000 7f4e2e7fc640 1 -- 172.21.15.86:0/1585924865 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e300075d0 con 0x7f4e381029a0 2024-04-21T08:36:45.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.513+0000 7f4e3f625640 1 -- 172.21.15.86:0/1585924865 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e381a0790 con 0x7f4e381029a0 2024-04-21T08:36:45.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.514+0000 7f4e3f625640 1 -- 172.21.15.86:0/1585924865 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e381a0a50 con 0x7f4e381029a0 2024-04-21T08:36:45.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.515+0000 7f4e2e7fc640 1 -- 172.21.15.86:0/1585924865 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f4e3001c070 con 0x7f4e381029a0 2024-04-21T08:36:45.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.516+0000 7f4e2e7fc640 1 --2- 172.21.15.86:0/1585924865 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f4e240a3bb0 0x7f4e240a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:45.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.516+0000 7f4e3d3c1640 1 --2- 172.21.15.86:0/1585924865 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f4e240a3bb0 0x7f4e240a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:45.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.516+0000 7f4e2e7fc640 1 -- 172.21.15.86:0/1585924865 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f4e300cfff0 con 0x7f4e381029a0 2024-04-21T08:36:45.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.517+0000 7f4e3d3c1640 1 --2- 172.21.15.86:0/1585924865 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f4e240a3bb0 0x7f4e240a6070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4e280096c0 tx=0x7f4e28000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:45.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.520+0000 7f4e2e7fc640 1 -- 172.21.15.86:0/1585924865 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4e3009c140 con 0x7f4e381029a0 2024-04-21T08:36:45.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:45 smithi086 ceph-mon[27939]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-21T08:36:45.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:45 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2502661234' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-21T08:36:45.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:45 smithi086 ceph-mon[27939]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-21T08:36:45.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:45 smithi086 ceph-mon[27939]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T08:36:45.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:45 smithi086 ceph-mon[27939]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-21T08:36:45.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:45 smithi086 ceph-mon[27939]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-21T08:36:45.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:45 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2693403403' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-21T08:36:45.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:45 smithi088 ceph-mon[31259]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-21T08:36:45.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:45 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2502661234' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-21T08:36:45.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:45 smithi088 ceph-mon[31259]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-21T08:36:45.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:45 smithi088 ceph-mon[31259]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T08:36:45.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:45 smithi088 ceph-mon[31259]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-21T08:36:45.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:45 smithi088 ceph-mon[31259]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-21T08:36:45.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:45 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2693403403' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-21T08:36:45.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:45.743+0000 7f4e3f625640 1 -- 172.21.15.86:0/1585924865 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"} v 0) -- 0x7f4e381a0c20 con 0x7f4e381029a0 2024-04-21T08:36:45.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:45 smithi179 ceph-mon[31829]: Health check failed: 1 filesystem is online with fewer MDS than max_mds (MDS_UP_LESS_THAN_MAX) 2024-04-21T08:36:45.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:45 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2502661234' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "max_mds", "val": "5"}]': finished 2024-04-21T08:36:45.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:45 smithi179 ceph-mon[31829]: daemon mds.i assigned to filesystem cephfs as rank 1 (now has 2 ranks) 2024-04-21T08:36:45.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:45 smithi179 ceph-mon[31829]: fsmap cephfs:1 {0=b=up:active} 11 up:standby 2024-04-21T08:36:45.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:45 smithi179 ceph-mon[31829]: fsmap cephfs:2 {0=b=up:active,1=i=up:creating} 10 up:standby 2024-04-21T08:36:45.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:45 smithi179 ceph-mon[31829]: daemon mds.i is now active in filesystem cephfs as rank 1 2024-04-21T08:36:45.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:45 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2693403403' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]: dispatch 2024-04-21T08:36:46.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.435+0000 7f4e2e7fc640 1 -- 172.21.15.86:0/1585924865 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]=0 v25) ==== 115+0+0 (secure 0 0 0) 0x7f4e300a0440 con 0x7f4e381029a0 2024-04-21T08:36:46.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.438+0000 7f4e3f625640 1 -- 172.21.15.86:0/1585924865 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f4e240a3bb0 msgr2=0x7f4e240a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:46.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.438+0000 7f4e3f625640 1 --2- 172.21.15.86:0/1585924865 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f4e240a3bb0 0x7f4e240a6070 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7f4e280096c0 tx=0x7f4e28000bc0 comp rx=0 tx=0).stop 2024-04-21T08:36:46.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.438+0000 7f4e3f625640 1 -- 172.21.15.86:0/1585924865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e381029a0 msgr2=0x7f4e3819f860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:46.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.438+0000 7f4e3f625640 1 --2- 172.21.15.86:0/1585924865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e381029a0 0x7f4e3819f860 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f4e3000b970 tx=0x7f4e3000be30 comp rx=0 tx=0).stop 2024-04-21T08:36:46.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.438+0000 7f4e3f625640 1 -- 172.21.15.86:0/1585924865 shutdown_connections 2024-04-21T08:36:46.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.438+0000 7f4e3f625640 1 --2- 172.21.15.86:0/1585924865 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f4e240a3bb0 0x7f4e240a6070 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:46.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.438+0000 7f4e3f625640 1 --2- 172.21.15.86:0/1585924865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e381029a0 0x7f4e3819f860 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:46.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.438+0000 7f4e3f625640 1 --2- 172.21.15.86:0/1585924865 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e38101fe0 0x7f4e38198840 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:46.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.438+0000 7f4e3f625640 1 --2- 172.21.15.86:0/1585924865 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4e38100e90 0x7f4e38198300 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:46.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.438+0000 7f4e3f625640 1 -- 172.21.15.86:0/1585924865 >> 172.21.15.86:0/1585924865 conn(0x7f4e380fc400 msgr2=0x7f4e380691d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:46.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.438+0000 7f4e3f625640 1 -- 172.21.15.86:0/1585924865 shutdown_connections 2024-04-21T08:36:46.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.438+0000 7f4e3f625640 1 -- 172.21.15.86:0/1585924865 wait complete. 2024-04-21T08:36:46.453 DEBUG:tasks.cephfs.filesystem:Creating {'create': 2, 'subvol_options': '--size 25000000000'} subvols for filesystem 'cephfs' 2024-04-21T08:36:46.454 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolumegroup create cephfs qa 2024-04-21T08:36:46.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.561+0000 7f9131306640 1 -- 172.21.15.86:0/3687265771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f912c1053e0 msgr2=0x7f912c1077d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:46.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.561+0000 7f9131306640 1 --2- 172.21.15.86:0/3687265771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f912c1053e0 0x7f912c1077d0 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f9120009900 tx=0x7f912002bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:46.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.561+0000 7f9131306640 1 -- 172.21.15.86:0/3687265771 shutdown_connections 2024-04-21T08:36:46.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.561+0000 7f9131306640 1 --2- 172.21.15.86:0/3687265771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f912c107d10 0x7f912c10a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:46.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.561+0000 7f9131306640 1 --2- 172.21.15.86:0/3687265771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f912c1053e0 0x7f912c1077d0 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:46.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.561+0000 7f9131306640 1 --2- 172.21.15.86:0/3687265771 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f912c102ab0 0x7f912c104ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:46.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.561+0000 7f9131306640 1 -- 172.21.15.86:0/3687265771 >> 172.21.15.86:0/3687265771 conn(0x7f912c0fc400 msgr2=0x7f912c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:46.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.561+0000 7f9131306640 1 -- 172.21.15.86:0/3687265771 shutdown_connections 2024-04-21T08:36:46.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.561+0000 7f9131306640 1 -- 172.21.15.86:0/3687265771 wait complete. 2024-04-21T08:36:46.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.562+0000 7f9131306640 1 Processor -- start 2024-04-21T08:36:46.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.562+0000 7f9131306640 1 -- start start 2024-04-21T08:36:46.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.562+0000 7f9131306640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f912c102ab0 0x7f912c198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:46.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.562+0000 7f9131306640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f912c1053e0 0x7f912c1987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:46.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.562+0000 7f9131306640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f912c107d10 0x7f912c19f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.562+0000 7f9131306640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f912c10db10 con 0x7f912c107d10 2024-04-21T08:36:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.562+0000 7f9131306640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f912c10d990 con 0x7f912c1053e0 2024-04-21T08:36:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.562+0000 7f9131306640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f912c10dc90 con 0x7f912c102ab0 2024-04-21T08:36:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.562+0000 7f912b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f912c107d10 0x7f912c19f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.562+0000 7f912b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f912c107d10 0x7f912c19f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35140/0 (socket says 172.21.15.86:35140) 2024-04-21T08:36:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.562+0000 7f912b7fe640 1 -- 172.21.15.86:0/3968353813 learned_addr learned my addr 172.21.15.86:0/3968353813 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.562+0000 7f912a7fc640 1 --2- 172.21.15.86:0/3968353813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f912c1053e0 0x7f912c1987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.563+0000 7f912b7fe640 1 -- 172.21.15.86:0/3968353813 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f912c102ab0 msgr2=0x7f912c198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.563+0000 7f912b7fe640 1 --2- 172.21.15.86:0/3968353813 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f912c102ab0 0x7f912c198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.563+0000 7f912b7fe640 1 -- 172.21.15.86:0/3968353813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f912c1053e0 msgr2=0x7f912c1987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.563+0000 7f912b7fe640 1 --2- 172.21.15.86:0/3968353813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f912c1053e0 0x7f912c1987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.563+0000 7f912b7fe640 1 -- 172.21.15.86:0/3968353813 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f912c19fea0 con 0x7f912c107d10 2024-04-21T08:36:46.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.563+0000 7f912b7fe640 1 --2- 172.21.15.86:0/3968353813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f912c107d10 0x7f912c19f830 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f912400b970 tx=0x7f912400be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:46.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.563+0000 7f9113fff640 1 -- 172.21.15.86:0/3968353813 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9124019070 con 0x7f912c107d10 2024-04-21T08:36:46.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.563+0000 7f9131306640 1 -- 172.21.15.86:0/3968353813 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f912c1a0190 con 0x7f912c107d10 2024-04-21T08:36:46.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.563+0000 7f9113fff640 1 -- 172.21.15.86:0/3968353813 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9124014440 con 0x7f912c107d10 2024-04-21T08:36:46.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.563+0000 7f9113fff640 1 -- 172.21.15.86:0/3968353813 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f91240075d0 con 0x7f912c107d10 2024-04-21T08:36:46.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.563+0000 7f9131306640 1 -- 172.21.15.86:0/3968353813 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f912c1a0730 con 0x7f912c107d10 2024-04-21T08:36:46.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.564+0000 7f9131306640 1 -- 172.21.15.86:0/3968353813 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f912c0fdef0 con 0x7f912c107d10 2024-04-21T08:36:46.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.565+0000 7f9113fff640 1 -- 172.21.15.86:0/3968353813 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f912401c070 con 0x7f912c107d10 2024-04-21T08:36:46.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.565+0000 7f9113fff640 1 --2- 172.21.15.86:0/3968353813 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f91140a3bb0 0x7f91140a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:46.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.566+0000 7f912affd640 1 --2- 172.21.15.86:0/3968353813 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f91140a3bb0 0x7f91140a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:46.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.566+0000 7f9113fff640 1 -- 172.21.15.86:0/3968353813 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f91240cfbd0 con 0x7f912c107d10 2024-04-21T08:36:46.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.566+0000 7f912affd640 1 --2- 172.21.15.86:0/3968353813 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f91140a3bb0 0x7f91140a6070 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f911c0058c0 tx=0x7f911c005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:46.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.570+0000 7f9113fff640 1 -- 172.21.15.86:0/3968353813 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f912409bd20 con 0x7f912c107d10 2024-04-21T08:36:46.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:46 smithi088 ceph-mon[31259]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-04-21T08:36:46.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:46 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2693403403' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-21T08:36:46.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:46 smithi088 ceph-mon[31259]: mds.? [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] up:active 2024-04-21T08:36:46.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:46 smithi088 ceph-mon[31259]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-21T08:36:46.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:46 smithi088 ceph-mon[31259]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-21T08:36:46.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:46 smithi088 ceph-mon[31259]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-04-21T08:36:46.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:46 smithi088 ceph-mon[31259]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-04-21T08:36:46.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:46 smithi088 ceph-mon[31259]: mds.? [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] up:active 2024-04-21T08:36:46.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:46 smithi088 ceph-mon[31259]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-21T08:36:46.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:46 smithi088 ceph-mon[31259]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-21T08:36:46.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:46 smithi088 ceph-mon[31259]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=k=up:creating} 3 up:standby-replay 5 up:standby 2024-04-21T08:36:46.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:46 smithi088 ceph-mon[31259]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-04-21T08:36:46.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:46 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1585924865' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-21T08:36:46.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:46 smithi179 ceph-mon[31829]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-04-21T08:36:46.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:46 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2693403403' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-21T08:36:46.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:46 smithi179 ceph-mon[31829]: mds.? [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] up:active 2024-04-21T08:36:46.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:46 smithi179 ceph-mon[31829]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-21T08:36:46.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:46 smithi179 ceph-mon[31829]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-21T08:36:46.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:46 smithi179 ceph-mon[31829]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-04-21T08:36:46.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:46 smithi179 ceph-mon[31829]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-04-21T08:36:46.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:46 smithi179 ceph-mon[31829]: mds.? [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] up:active 2024-04-21T08:36:46.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:46 smithi179 ceph-mon[31829]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-21T08:36:46.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:46 smithi179 ceph-mon[31829]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-21T08:36:46.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:46 smithi179 ceph-mon[31829]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=k=up:creating} 3 up:standby-replay 5 up:standby 2024-04-21T08:36:46.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:46 smithi179 ceph-mon[31829]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-04-21T08:36:46.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:46 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1585924865' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-21T08:36:46.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.772+0000 7f9131306640 1 -- 172.21.15.86:0/3968353813 --> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] -- mgr_command(tid 0: {"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f912c062390 con 0x7f91140a3bb0 2024-04-21T08:36:46.777 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:46 smithi086 ceph-mon[27939]: pgmap v300: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 511 B/s wr, 1 op/s 2024-04-21T08:36:46.777 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:46 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2693403403' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "allow_standby_replay", "val": "true"}]': finished 2024-04-21T08:36:46.778 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:46 smithi086 ceph-mon[27939]: mds.? [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] up:active 2024-04-21T08:36:46.778 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:46 smithi086 ceph-mon[27939]: daemon mds.h assigned to filesystem cephfs as rank 2 (now has 3 ranks) 2024-04-21T08:36:46.778 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:46 smithi086 ceph-mon[27939]: fsmap cephfs:2 {0=b=up:active,1=i=up:active} 10 up:standby 2024-04-21T08:36:46.778 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:46 smithi086 ceph-mon[27939]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:creating} 9 up:standby 2024-04-21T08:36:46.778 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:46 smithi086 ceph-mon[27939]: daemon mds.h is now active in filesystem cephfs as rank 2 2024-04-21T08:36:46.778 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:46 smithi086 ceph-mon[27939]: mds.? [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] up:active 2024-04-21T08:36:46.778 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:46 smithi086 ceph-mon[27939]: daemon mds.k assigned to filesystem cephfs as rank 3 (now has 4 ranks) 2024-04-21T08:36:46.778 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:46 smithi086 ceph-mon[27939]: fsmap cephfs:3 {0=b=up:active,1=i=up:active,2=h=up:active} 3 up:standby-replay 6 up:standby 2024-04-21T08:36:46.778 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:46 smithi086 ceph-mon[27939]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=k=up:creating} 3 up:standby-replay 5 up:standby 2024-04-21T08:36:46.778 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:46 smithi086 ceph-mon[27939]: daemon mds.k is now active in filesystem cephfs as rank 3 2024-04-21T08:36:46.778 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:46 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1585924865' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]: dispatch 2024-04-21T08:36:46.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.801+0000 7f9113fff640 1 -- 172.21.15.86:0/3968353813 <== mgr.14152 v2:172.21.15.86:6800/1364156965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f912c062390 con 0x7f91140a3bb0 2024-04-21T08:36:46.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.803+0000 7f9131306640 1 -- 172.21.15.86:0/3968353813 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f91140a3bb0 msgr2=0x7f91140a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:46.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.803+0000 7f9131306640 1 --2- 172.21.15.86:0/3968353813 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f91140a3bb0 0x7f91140a6070 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f911c0058c0 tx=0x7f911c005850 comp rx=0 tx=0).stop 2024-04-21T08:36:46.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.803+0000 7f9131306640 1 -- 172.21.15.86:0/3968353813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f912c107d10 msgr2=0x7f912c19f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:46.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.803+0000 7f9131306640 1 --2- 172.21.15.86:0/3968353813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f912c107d10 0x7f912c19f830 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f912400b970 tx=0x7f912400be30 comp rx=0 tx=0).stop 2024-04-21T08:36:46.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.804+0000 7f9131306640 1 -- 172.21.15.86:0/3968353813 shutdown_connections 2024-04-21T08:36:46.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.804+0000 7f9131306640 1 --2- 172.21.15.86:0/3968353813 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f91140a3bb0 0x7f91140a6070 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:46.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.804+0000 7f9131306640 1 --2- 172.21.15.86:0/3968353813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f912c107d10 0x7f912c19f830 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:46.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.804+0000 7f9131306640 1 --2- 172.21.15.86:0/3968353813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f912c1053e0 0x7f912c1987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:46.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.804+0000 7f9131306640 1 --2- 172.21.15.86:0/3968353813 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f912c102ab0 0x7f912c198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:46.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.804+0000 7f9131306640 1 -- 172.21.15.86:0/3968353813 >> 172.21.15.86:0/3968353813 conn(0x7f912c0fc400 msgr2=0x7f912c1035b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:46.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.804+0000 7f9131306640 1 -- 172.21.15.86:0/3968353813 shutdown_connections 2024-04-21T08:36:46.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.804+0000 7f9131306640 1 -- 172.21.15.86:0/3968353813 wait complete. 2024-04-21T08:36:46.814 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_0 --group_name qa 2024-04-21T08:36:46.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.919+0000 7f54e626a640 1 -- 172.21.15.86:0/806126677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54e0102a70 msgr2=0x7f54e0104e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:46.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.919+0000 7f54e626a640 1 --2- 172.21.15.86:0/806126677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54e0102a70 0x7f54e0104e60 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7f54d0009900 tx=0x7f54d002bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:46.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.919+0000 7f54e626a640 1 -- 172.21.15.86:0/806126677 shutdown_connections 2024-04-21T08:36:46.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.919+0000 7f54e626a640 1 --2- 172.21.15.86:0/806126677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54e0107cd0 0x7f54e010a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:46.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.919+0000 7f54e626a640 1 --2- 172.21.15.86:0/806126677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54e01053a0 0x7f54e0107790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:46.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.919+0000 7f54e626a640 1 --2- 172.21.15.86:0/806126677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54e0102a70 0x7f54e0104e60 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:46.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.919+0000 7f54e626a640 1 -- 172.21.15.86:0/806126677 >> 172.21.15.86:0/806126677 conn(0x7f54e00fc400 msgr2=0x7f54e00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:46.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.919+0000 7f54e626a640 1 -- 172.21.15.86:0/806126677 shutdown_connections 2024-04-21T08:36:46.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.919+0000 7f54e626a640 1 -- 172.21.15.86:0/806126677 wait complete. 2024-04-21T08:36:46.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.920+0000 7f54e626a640 1 Processor -- start 2024-04-21T08:36:46.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.920+0000 7f54e626a640 1 -- start start 2024-04-21T08:36:46.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.920+0000 7f54e626a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54e0102a70 0x7f54e0198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:46.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.920+0000 7f54e626a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54e01053a0 0x7f54e01987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:46.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.920+0000 7f54e626a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54e0107cd0 0x7f54e019f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:46.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.920+0000 7f54e626a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f54e010dae0 con 0x7f54e0107cd0 2024-04-21T08:36:46.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.920+0000 7f54e626a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f54e010d960 con 0x7f54e01053a0 2024-04-21T08:36:46.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.920+0000 7f54e626a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f54e010dc60 con 0x7f54e0102a70 2024-04-21T08:36:46.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.920+0000 7f54e4807640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54e0107cd0 0x7f54e019f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:46.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.920+0000 7f54e4807640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54e0107cd0 0x7f54e019f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35168/0 (socket says 172.21.15.86:35168) 2024-04-21T08:36:46.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.920+0000 7f54e4807640 1 -- 172.21.15.86:0/366181720 learned_addr learned my addr 172.21.15.86:0/366181720 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:46.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.920+0000 7f54df7fe640 1 --2- 172.21.15.86:0/366181720 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54e01053a0 0x7f54e01987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:46.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.920+0000 7f54dffff640 1 --2- 172.21.15.86:0/366181720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54e0102a70 0x7f54e0198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:46.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.920+0000 7f54e4807640 1 -- 172.21.15.86:0/366181720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54e0102a70 msgr2=0x7f54e0198280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:46.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.920+0000 7f54e4807640 1 --2- 172.21.15.86:0/366181720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54e0102a70 0x7f54e0198280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:46.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.920+0000 7f54e4807640 1 -- 172.21.15.86:0/366181720 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54e01053a0 msgr2=0x7f54e01987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:46.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.920+0000 7f54e4807640 1 --2- 172.21.15.86:0/366181720 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54e01053a0 0x7f54e01987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:46.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.920+0000 7f54e4807640 1 -- 172.21.15.86:0/366181720 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f54e019fea0 con 0x7f54e0107cd0 2024-04-21T08:36:46.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.920+0000 7f54df7fe640 1 --2- 172.21.15.86:0/366181720 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54e01053a0 0x7f54e01987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:36:46.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.921+0000 7f54e4807640 1 --2- 172.21.15.86:0/366181720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54e0107cd0 0x7f54e019f830 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f54d800cad0 tx=0x7f54d800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:46.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.921+0000 7f54dd7fa640 1 -- 172.21.15.86:0/366181720 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d800ae20 con 0x7f54e0107cd0 2024-04-21T08:36:46.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.921+0000 7f54e626a640 1 -- 172.21.15.86:0/366181720 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f54e01a0190 con 0x7f54e0107cd0 2024-04-21T08:36:46.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.921+0000 7f54dd7fa640 1 -- 172.21.15.86:0/366181720 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f54d8005020 con 0x7f54e0107cd0 2024-04-21T08:36:46.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.921+0000 7f54dd7fa640 1 -- 172.21.15.86:0/366181720 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f54d800f6b0 con 0x7f54e0107cd0 2024-04-21T08:36:46.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.921+0000 7f54e626a640 1 -- 172.21.15.86:0/366181720 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f54e01a06e0 con 0x7f54e0107cd0 2024-04-21T08:36:46.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.922+0000 7f54e626a640 1 -- 172.21.15.86:0/366181720 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f54e00fdef0 con 0x7f54e0107cd0 2024-04-21T08:36:46.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.923+0000 7f54dd7fa640 1 -- 172.21.15.86:0/366181720 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7f54d80051d0 con 0x7f54e0107cd0 2024-04-21T08:36:46.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.923+0000 7f54dd7fa640 1 --2- 172.21.15.86:0/366181720 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f54c80a3bb0 0x7f54c80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:46.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.924+0000 7f54dffff640 1 --2- 172.21.15.86:0/366181720 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f54c80a3bb0 0x7f54c80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:46.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.924+0000 7f54dd7fa640 1 -- 172.21.15.86:0/366181720 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f54d80d0650 con 0x7f54e0107cd0 2024-04-21T08:36:46.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.924+0000 7f54dffff640 1 --2- 172.21.15.86:0/366181720 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f54c80a3bb0 0x7f54c80a6070 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f54d0000c00 tx=0x7f54d0005be0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:46.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:46.928+0000 7f54dd7fa640 1 -- 172.21.15.86:0/366181720 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f54d8014030 con 0x7f54e0107cd0 2024-04-21T08:36:47.116 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:36:46 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:36:46.777+0000 7fb00267f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:36:47.116 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:36:46 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:36:46.777+0000 7fb00267f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:36:47.117 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:36:46 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:36:46.777+0000 7fb00267f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:36:47.117 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:36:46 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:36:46.777+0000 7fb00267f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:36:47.117 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:36:46 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:36:46.777+0000 7fb00267f700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:36:47.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.134+0000 7f54e626a640 1 -- 172.21.15.86:0/366181720 --> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7f54e0062390 con 0x7f54c80a3bb0 2024-04-21T08:36:47.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.161+0000 7f54dd7fa640 1 -- 172.21.15.86:0/366181720 <== mgr.14152 v2:172.21.15.86:6800/1364156965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7f54e0062390 con 0x7f54c80a3bb0 2024-04-21T08:36:47.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.163+0000 7f54e626a640 1 -- 172.21.15.86:0/366181720 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f54c80a3bb0 msgr2=0x7f54c80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:47.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.163+0000 7f54e626a640 1 --2- 172.21.15.86:0/366181720 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f54c80a3bb0 0x7f54c80a6070 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7f54d0000c00 tx=0x7f54d0005be0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.163+0000 7f54e626a640 1 -- 172.21.15.86:0/366181720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54e0107cd0 msgr2=0x7f54e019f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:47.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.163+0000 7f54e626a640 1 --2- 172.21.15.86:0/366181720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54e0107cd0 0x7f54e019f830 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f54d800cad0 tx=0x7f54d800cf90 comp rx=0 tx=0).stop 2024-04-21T08:36:47.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.163+0000 7f54e626a640 1 -- 172.21.15.86:0/366181720 shutdown_connections 2024-04-21T08:36:47.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.163+0000 7f54e626a640 1 --2- 172.21.15.86:0/366181720 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f54c80a3bb0 0x7f54c80a6070 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.163+0000 7f54e626a640 1 --2- 172.21.15.86:0/366181720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f54e0107cd0 0x7f54e019f830 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.163+0000 7f54e626a640 1 --2- 172.21.15.86:0/366181720 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f54e01053a0 0x7f54e01987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.163+0000 7f54e626a640 1 --2- 172.21.15.86:0/366181720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f54e0102a70 0x7f54e0198280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.163+0000 7f54e626a640 1 -- 172.21.15.86:0/366181720 >> 172.21.15.86:0/366181720 conn(0x7f54e00fc400 msgr2=0x7f54e0103470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:47.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.163+0000 7f54e626a640 1 -- 172.21.15.86:0/366181720 shutdown_connections 2024-04-21T08:36:47.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.163+0000 7f54e626a640 1 -- 172.21.15.86:0/366181720 wait complete. 2024-04-21T08:36:47.173 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_0 2024-04-21T08:36:47.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.278+0000 7fea906a0640 1 -- 172.21.15.86:0/1179372658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea88101fc0 msgr2=0x7fea88102440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:47.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.278+0000 7fea906a0640 1 --2- 172.21.15.86:0/1179372658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea88101fc0 0x7fea88102440 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7fea78009900 tx=0x7fea7802bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:47.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.278+0000 7fea906a0640 1 -- 172.21.15.86:0/1179372658 shutdown_connections 2024-04-21T08:36:47.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.278+0000 7fea906a0640 1 --2- 172.21.15.86:0/1179372658 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea88102980 0x7fea881092a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.278+0000 7fea906a0640 1 --2- 172.21.15.86:0/1179372658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea88101fc0 0x7fea88102440 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.278+0000 7fea906a0640 1 --2- 172.21.15.86:0/1179372658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea88100e70 0x7fea88101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.278+0000 7fea906a0640 1 -- 172.21.15.86:0/1179372658 >> 172.21.15.86:0/1179372658 conn(0x7fea880fc3e0 msgr2=0x7fea880fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:47.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.278+0000 7fea906a0640 1 -- 172.21.15.86:0/1179372658 shutdown_connections 2024-04-21T08:36:47.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.278+0000 7fea906a0640 1 -- 172.21.15.86:0/1179372658 wait complete. 2024-04-21T08:36:47.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.279+0000 7fea906a0640 1 Processor -- start 2024-04-21T08:36:47.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.279+0000 7fea906a0640 1 -- start start 2024-04-21T08:36:47.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.279+0000 7fea906a0640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea88100e70 0x7fea88197110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:47.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.279+0000 7fea906a0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea88101fc0 0x7fea88197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:47.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.279+0000 7fea906a0640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea88102980 0x7fea8819e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:47.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.279+0000 7fea906a0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea8810c9d0 con 0x7fea88101fc0 2024-04-21T08:36:47.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.279+0000 7fea906a0640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fea8810c850 con 0x7fea88100e70 2024-04-21T08:36:47.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.279+0000 7fea906a0640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fea8810cb50 con 0x7fea88102980 2024-04-21T08:36:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.279+0000 7fea8dc3b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea88101fc0 0x7fea88197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.279+0000 7fea8e43c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea88100e70 0x7fea88197110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.279+0000 7fea8dc3b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea88101fc0 0x7fea88197650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35184/0 (socket says 172.21.15.86:35184) 2024-04-21T08:36:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.279+0000 7fea8dc3b640 1 -- 172.21.15.86:0/4060029357 learned_addr learned my addr 172.21.15.86:0/4060029357 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.279+0000 7fea8ec3d640 1 --2- 172.21.15.86:0/4060029357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea88102980 0x7fea8819e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.279+0000 7fea8dc3b640 1 -- 172.21.15.86:0/4060029357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea88102980 msgr2=0x7fea8819e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.279+0000 7fea8dc3b640 1 --2- 172.21.15.86:0/4060029357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea88102980 0x7fea8819e6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.279+0000 7fea8dc3b640 1 -- 172.21.15.86:0/4060029357 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea88100e70 msgr2=0x7fea88197110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.279+0000 7fea8dc3b640 1 --2- 172.21.15.86:0/4060029357 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea88100e70 0x7fea88197110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.279+0000 7fea8dc3b640 1 -- 172.21.15.86:0/4060029357 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea8819ed30 con 0x7fea88101fc0 2024-04-21T08:36:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.279+0000 7fea8e43c640 1 --2- 172.21.15.86:0/4060029357 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea88100e70 0x7fea88197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:36:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.279+0000 7fea8dc3b640 1 --2- 172.21.15.86:0/4060029357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea88101fc0 0x7fea88197650 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fea78000c00 tx=0x7fea78004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.280+0000 7fea7f7fe640 1 -- 172.21.15.86:0/4060029357 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea78043020 con 0x7fea88101fc0 2024-04-21T08:36:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.280+0000 7fea7f7fe640 1 -- 172.21.15.86:0/4060029357 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea78004440 con 0x7fea88101fc0 2024-04-21T08:36:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.280+0000 7fea906a0640 1 -- 172.21.15.86:0/4060029357 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea8819efc0 con 0x7fea88101fc0 2024-04-21T08:36:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.280+0000 7fea7f7fe640 1 -- 172.21.15.86:0/4060029357 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea78003c90 con 0x7fea88101fc0 2024-04-21T08:36:47.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.280+0000 7fea906a0640 1 -- 172.21.15.86:0/4060029357 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea8819f420 con 0x7fea88101fc0 2024-04-21T08:36:47.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.281+0000 7fea906a0640 1 -- 172.21.15.86:0/4060029357 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fea88108310 con 0x7fea88101fc0 2024-04-21T08:36:47.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.282+0000 7fea7f7fe640 1 -- 172.21.15.86:0/4060029357 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 16) ==== 138646+0+0 (secure 0 0 0) 0x7fea78046020 con 0x7fea88101fc0 2024-04-21T08:36:47.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.282+0000 7fea7f7fe640 1 --2- 172.21.15.86:0/4060029357 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fea740a3bb0 0x7fea740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:47.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.282+0000 7fea8e43c640 1 --2- 172.21.15.86:0/4060029357 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fea740a3bb0 0x7fea740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:47.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.282+0000 7fea7f7fe640 1 -- 172.21.15.86:0/4060029357 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fea780f2890 con 0x7fea88101fc0 2024-04-21T08:36:47.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.283+0000 7fea8e43c640 1 --2- 172.21.15.86:0/4060029357 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fea740a3bb0 0x7fea740a6070 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fea84009fc0 tx=0x7fea84005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:47.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.286+0000 7fea7f7fe640 1 -- 172.21.15.86:0/4060029357 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fea7804d020 con 0x7fea88101fc0 2024-04-21T08:36:47.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.456+0000 7fea7f7fe640 1 -- 172.21.15.86:0/4060029357 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fea780bafa0 con 0x7fea88101fc0 2024-04-21T08:36:47.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.500+0000 7fea906a0640 1 -- 172.21.15.86:0/4060029357 --> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}) -- 0x7fea88062420 con 0x7fea740a3bb0 2024-04-21T08:36:47.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.505+0000 7fea7f7fe640 1 -- 172.21.15.86:0/4060029357 <== mgr.14152 v2:172.21.15.86:6800/1364156965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7fea88062420 con 0x7fea740a3bb0 2024-04-21T08:36:47.504 INFO:teuthology.orchestra.run.smithi086.stdout:/volumes/qa/sv_0/5c3ad362-b0f1-4c5b-b549-5b82e399d53b 2024-04-21T08:36:47.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.507+0000 7fea906a0640 1 -- 172.21.15.86:0/4060029357 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fea740a3bb0 msgr2=0x7fea740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:47.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.507+0000 7fea906a0640 1 --2- 172.21.15.86:0/4060029357 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fea740a3bb0 0x7fea740a6070 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7fea84009fc0 tx=0x7fea84005f90 comp rx=0 tx=0).stop 2024-04-21T08:36:47.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.507+0000 7fea906a0640 1 -- 172.21.15.86:0/4060029357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea88101fc0 msgr2=0x7fea88197650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:47.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.507+0000 7fea906a0640 1 --2- 172.21.15.86:0/4060029357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea88101fc0 0x7fea88197650 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fea78000c00 tx=0x7fea78004040 comp rx=0 tx=0).stop 2024-04-21T08:36:47.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.507+0000 7fea906a0640 1 -- 172.21.15.86:0/4060029357 shutdown_connections 2024-04-21T08:36:47.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.507+0000 7fea906a0640 1 --2- 172.21.15.86:0/4060029357 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fea740a3bb0 0x7fea740a6070 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.507+0000 7fea906a0640 1 --2- 172.21.15.86:0/4060029357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea88102980 0x7fea8819e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.507+0000 7fea906a0640 1 --2- 172.21.15.86:0/4060029357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea88101fc0 0x7fea88197650 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.507+0000 7fea906a0640 1 --2- 172.21.15.86:0/4060029357 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea88100e70 0x7fea88197110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.507+0000 7fea906a0640 1 -- 172.21.15.86:0/4060029357 >> 172.21.15.86:0/4060029357 conn(0x7fea880fc3e0 msgr2=0x7fea881054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:47.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.507+0000 7fea906a0640 1 -- 172.21.15.86:0/4060029357 shutdown_connections 2024-04-21T08:36:47.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.507+0000 7fea906a0640 1 -- 172.21.15.86:0/4060029357 wait complete. 2024-04-21T08:36:47.517 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume create cephfs sv_1 --group_name qa 2024-04-21T08:36:47.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.621+0000 7fde8ce1f640 1 -- 172.21.15.86:0/2346057755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde88102af0 msgr2=0x7fde88104ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:47.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.621+0000 7fde8ce1f640 1 --2- 172.21.15.86:0/2346057755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde88102af0 0x7fde88104ee0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7fde78009900 tx=0x7fde7802bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:47.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.621+0000 7fde8ce1f640 1 -- 172.21.15.86:0/2346057755 shutdown_connections 2024-04-21T08:36:47.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.621+0000 7fde8ce1f640 1 --2- 172.21.15.86:0/2346057755 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde88107d50 0x7fde8810a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.621+0000 7fde8ce1f640 1 --2- 172.21.15.86:0/2346057755 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde88105420 0x7fde88107810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.621+0000 7fde8ce1f640 1 --2- 172.21.15.86:0/2346057755 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde88102af0 0x7fde88104ee0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.621+0000 7fde8ce1f640 1 -- 172.21.15.86:0/2346057755 >> 172.21.15.86:0/2346057755 conn(0x7fde880fc420 msgr2=0x7fde880fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:47.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.622+0000 7fde8ce1f640 1 -- 172.21.15.86:0/2346057755 shutdown_connections 2024-04-21T08:36:47.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.622+0000 7fde8ce1f640 1 -- 172.21.15.86:0/2346057755 wait complete. 2024-04-21T08:36:47.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.622+0000 7fde8ce1f640 1 Processor -- start 2024-04-21T08:36:47.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.622+0000 7fde8ce1f640 1 -- start start 2024-04-21T08:36:47.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.622+0000 7fde8ce1f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde88102af0 0x7fde881982a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:47.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.622+0000 7fde8ce1f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde88105420 0x7fde881987e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:47.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.622+0000 7fde8ce1f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde88107d50 0x7fde8819f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:47.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.622+0000 7fde8ce1f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fde8810db00 con 0x7fde88105420 2024-04-21T08:36:47.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.622+0000 7fde8ce1f640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fde8810d980 con 0x7fde88107d50 2024-04-21T08:36:47.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.622+0000 7fde8ce1f640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fde8810dc80 con 0x7fde88102af0 2024-04-21T08:36:47.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.623+0000 7fde85d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde88105420 0x7fde881987e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:47.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.623+0000 7fde85d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde88105420 0x7fde881987e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35198/0 (socket says 172.21.15.86:35198) 2024-04-21T08:36:47.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.623+0000 7fde85d9b640 1 -- 172.21.15.86:0/689985199 learned_addr learned my addr 172.21.15.86:0/689985199 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:47.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.623+0000 7fde86d9d640 1 --2- 172.21.15.86:0/689985199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde88107d50 0x7fde8819f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:47.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.623+0000 7fde8659c640 1 --2- 172.21.15.86:0/689985199 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde88102af0 0x7fde881982a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:47.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.623+0000 7fde85d9b640 1 -- 172.21.15.86:0/689985199 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde88102af0 msgr2=0x7fde881982a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:47.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.623+0000 7fde85d9b640 1 --2- 172.21.15.86:0/689985199 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde88102af0 0x7fde881982a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.623+0000 7fde85d9b640 1 -- 172.21.15.86:0/689985199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde88107d50 msgr2=0x7fde8819f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:47.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.623+0000 7fde85d9b640 1 --2- 172.21.15.86:0/689985199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde88107d50 0x7fde8819f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.623+0000 7fde85d9b640 1 -- 172.21.15.86:0/689985199 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde8819fe70 con 0x7fde88105420 2024-04-21T08:36:47.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.623+0000 7fde8659c640 1 --2- 172.21.15.86:0/689985199 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde88102af0 0x7fde881982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:36:47.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.623+0000 7fde85d9b640 1 --2- 172.21.15.86:0/689985199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde88105420 0x7fde881987e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fde7c00e9a0 tx=0x7fde7c00ee60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:47.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.623+0000 7fde777fe640 1 -- 172.21.15.86:0/689985199 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7c00cd80 con 0x7fde88105420 2024-04-21T08:36:47.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.623+0000 7fde777fe640 1 -- 172.21.15.86:0/689985199 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde7c0035f0 con 0x7fde88105420 2024-04-21T08:36:47.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.623+0000 7fde8ce1f640 1 -- 172.21.15.86:0/689985199 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde881a0160 con 0x7fde88105420 2024-04-21T08:36:47.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.623+0000 7fde777fe640 1 -- 172.21.15.86:0/689985199 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde7c010690 con 0x7fde88105420 2024-04-21T08:36:47.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.623+0000 7fde8ce1f640 1 -- 172.21.15.86:0/689985199 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde881a0700 con 0x7fde88105420 2024-04-21T08:36:47.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.624+0000 7fde8ce1f640 1 -- 172.21.15.86:0/689985199 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fde880fdf10 con 0x7fde88105420 2024-04-21T08:36:47.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.625+0000 7fde777fe640 1 -- 172.21.15.86:0/689985199 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fde7c004000 con 0x7fde88105420 2024-04-21T08:36:47.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.626+0000 7fde777fe640 1 --2- 172.21.15.86:0/689985199 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fde700a3cc0 0x7fde700a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:47.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.626+0000 7fde777fe640 1 -- 172.21.15.86:0/689985199 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fde7c0d0a50 con 0x7fde88105420 2024-04-21T08:36:47.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.626+0000 7fde8659c640 1 --2- 172.21.15.86:0/689985199 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fde700a3cc0 0x7fde700a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:47.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.626+0000 7fde8659c640 1 --2- 172.21.15.86:0/689985199 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fde700a3cc0 0x7fde700a6180 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fde780096c0 tx=0x7fde78000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:47.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.630+0000 7fde777fe640 1 -- 172.21.15.86:0/689985199 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fde7c0042f0 con 0x7fde88105420 2024-04-21T08:36:47.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:47 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1585924865' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-21T08:36:47.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:47 smithi088 ceph-mon[31259]: mds.? [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] up:active 2024-04-21T08:36:47.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:47 smithi088 ceph-mon[31259]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-21T08:36:47.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:47 smithi088 ceph-mon[31259]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-21T08:36:47.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:47 smithi088 ceph-mon[31259]: Cluster is now healthy 2024-04-21T08:36:47.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:47 smithi088 ceph-mon[31259]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=k=up:active} 3 up:standby-replay 5 up:standby 2024-04-21T08:36:47.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:47 smithi088 ceph-mon[31259]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-04-21T08:36:47.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:47 smithi088 ceph-mon[31259]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-04-21T08:36:47.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:47 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1585924865' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-21T08:36:47.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:47 smithi179 ceph-mon[31829]: mds.? [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] up:active 2024-04-21T08:36:47.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:47 smithi179 ceph-mon[31829]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-21T08:36:47.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:47 smithi179 ceph-mon[31829]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-21T08:36:47.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:47 smithi179 ceph-mon[31829]: Cluster is now healthy 2024-04-21T08:36:47.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:47 smithi179 ceph-mon[31829]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=k=up:active} 3 up:standby-replay 5 up:standby 2024-04-21T08:36:47.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:47 smithi179 ceph-mon[31829]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-04-21T08:36:47.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:47 smithi179 ceph-mon[31829]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-04-21T08:36:47.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.836+0000 7fde8ce1f640 1 -- 172.21.15.86:0/689985199 --> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] -- mgr_command(tid 0: {"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}) -- 0x7fde88062390 con 0x7fde700a3cc0 2024-04-21T08:36:47.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.861+0000 7fde777fe640 1 -- 172.21.15.86:0/689985199 <== mgr.14152 v2:172.21.15.86:6800/1364156965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+0 (secure 0 0 0) 0x7fde88062390 con 0x7fde700a3cc0 2024-04-21T08:36:47.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.863+0000 7fde8ce1f640 1 -- 172.21.15.86:0/689985199 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fde700a3cc0 msgr2=0x7fde700a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:47.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.863+0000 7fde8ce1f640 1 --2- 172.21.15.86:0/689985199 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fde700a3cc0 0x7fde700a6180 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7fde780096c0 tx=0x7fde78000bc0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.863+0000 7fde8ce1f640 1 -- 172.21.15.86:0/689985199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde88105420 msgr2=0x7fde881987e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:47.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.863+0000 7fde8ce1f640 1 --2- 172.21.15.86:0/689985199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde88105420 0x7fde881987e0 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7fde7c00e9a0 tx=0x7fde7c00ee60 comp rx=0 tx=0).stop 2024-04-21T08:36:47.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.863+0000 7fde8ce1f640 1 -- 172.21.15.86:0/689985199 shutdown_connections 2024-04-21T08:36:47.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.863+0000 7fde8ce1f640 1 --2- 172.21.15.86:0/689985199 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fde700a3cc0 0x7fde700a6180 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.863+0000 7fde8ce1f640 1 --2- 172.21.15.86:0/689985199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde88107d50 0x7fde8819f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.863+0000 7fde8ce1f640 1 --2- 172.21.15.86:0/689985199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde88105420 0x7fde881987e0 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.863+0000 7fde8ce1f640 1 --2- 172.21.15.86:0/689985199 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde88102af0 0x7fde881982a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.863+0000 7fde8ce1f640 1 -- 172.21.15.86:0/689985199 >> 172.21.15.86:0/689985199 conn(0x7fde880fc420 msgr2=0x7fde88103610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:47.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.863+0000 7fde8ce1f640 1 -- 172.21.15.86:0/689985199 shutdown_connections 2024-04-21T08:36:47.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.863+0000 7fde8ce1f640 1 -- 172.21.15.86:0/689985199 wait complete. 2024-04-21T08:36:47.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:47 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1585924865' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "session_timeout", "val": "300"}]': finished 2024-04-21T08:36:47.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:47 smithi086 ceph-mon[27939]: mds.? [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] up:active 2024-04-21T08:36:47.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:47 smithi086 ceph-mon[27939]: daemon mds.l assigned to filesystem cephfs as rank 4 (now has 5 ranks) 2024-04-21T08:36:47.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:47 smithi086 ceph-mon[27939]: Health check cleared: MDS_UP_LESS_THAN_MAX (was: 1 filesystem is online with fewer MDS than max_mds) 2024-04-21T08:36:47.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:47 smithi086 ceph-mon[27939]: Cluster is now healthy 2024-04-21T08:36:47.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:47 smithi086 ceph-mon[27939]: fsmap cephfs:4 {0=b=up:active,1=i=up:active,2=h=up:active,3=k=up:active} 3 up:standby-replay 5 up:standby 2024-04-21T08:36:47.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:47 smithi086 ceph-mon[27939]: fsmap cephfs:5 3 up:standby-replay 1 up:creating 4 up:standby 4 up:active 2024-04-21T08:36:47.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:47 smithi086 ceph-mon[27939]: daemon mds.l is now active in filesystem cephfs as rank 4 2024-04-21T08:36:47.873 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs subvolume getpath cephfs --group_name qa sv_1 2024-04-21T08:36:47.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.978+0000 7f14866ba640 1 -- 172.21.15.86:0/687010611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1480107c90 msgr2=0x7f148010a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:47.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.978+0000 7f14866ba640 1 --2- 172.21.15.86:0/687010611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1480107c90 0x7f148010a0e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7f1478009ca0 tx=0x7f147802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.978+0000 7f14866ba640 1 -- 172.21.15.86:0/687010611 shutdown_connections 2024-04-21T08:36:47.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.978+0000 7f14866ba640 1 --2- 172.21.15.86:0/687010611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1480107c90 0x7f148010a0e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.978+0000 7f14866ba640 1 --2- 172.21.15.86:0/687010611 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1480105360 0x7f1480107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.978+0000 7f14866ba640 1 --2- 172.21.15.86:0/687010611 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1480102a30 0x7f1480104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.978+0000 7f14866ba640 1 -- 172.21.15.86:0/687010611 >> 172.21.15.86:0/687010611 conn(0x7f14800fc440 msgr2=0x7f14800fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:47.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.978+0000 7f14866ba640 1 -- 172.21.15.86:0/687010611 shutdown_connections 2024-04-21T08:36:47.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.978+0000 7f14866ba640 1 -- 172.21.15.86:0/687010611 wait complete. 2024-04-21T08:36:47.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.978+0000 7f14866ba640 1 Processor -- start 2024-04-21T08:36:47.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.978+0000 7f14866ba640 1 -- start start 2024-04-21T08:36:47.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.978+0000 7f14866ba640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1480102a30 0x7f1480198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:47.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.978+0000 7f14866ba640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1480105360 0x7f14801986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:47.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.979+0000 7f14866ba640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1480107c90 0x7f148019f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:47.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.979+0000 7f14866ba640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f148010daf0 con 0x7f1480102a30 2024-04-21T08:36:47.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.979+0000 7f14866ba640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f148010d970 con 0x7f1480105360 2024-04-21T08:36:47.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.979+0000 7f14866ba640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f148010dc70 con 0x7f1480107c90 2024-04-21T08:36:47.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.979+0000 7f147ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1480102a30 0x7f1480198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:47.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.979+0000 7f147ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1480102a30 0x7f1480198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35214/0 (socket says 172.21.15.86:35214) 2024-04-21T08:36:47.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.979+0000 7f147ffff640 1 -- 172.21.15.86:0/3360491927 learned_addr learned my addr 172.21.15.86:0/3360491927 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:47.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.979+0000 7f147f7fe640 1 --2- 172.21.15.86:0/3360491927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1480105360 0x7f14801986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:47.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.979+0000 7f147ffff640 1 -- 172.21.15.86:0/3360491927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1480107c90 msgr2=0x7f148019f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:47.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.979+0000 7f147ffff640 1 --2- 172.21.15.86:0/3360491927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1480107c90 0x7f148019f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.979+0000 7f147ffff640 1 -- 172.21.15.86:0/3360491927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1480105360 msgr2=0x7f14801986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:47.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.979+0000 7f147ffff640 1 --2- 172.21.15.86:0/3360491927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1480105360 0x7f14801986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:47.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.979+0000 7f147ffff640 1 -- 172.21.15.86:0/3360491927 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f148019fd90 con 0x7f1480102a30 2024-04-21T08:36:47.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.979+0000 7f147ffff640 1 --2- 172.21.15.86:0/3360491927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1480102a30 0x7f1480198170 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f147000c970 tx=0x7f147000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:47.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.979+0000 7f147d7fa640 1 -- 172.21.15.86:0/3360491927 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1470007bb0 con 0x7f1480102a30 2024-04-21T08:36:47.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.979+0000 7f147d7fa640 1 -- 172.21.15.86:0/3360491927 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1470007d50 con 0x7f1480102a30 2024-04-21T08:36:47.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.979+0000 7f147d7fa640 1 -- 172.21.15.86:0/3360491927 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f147000f450 con 0x7f1480102a30 2024-04-21T08:36:47.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.979+0000 7f14866ba640 1 -- 172.21.15.86:0/3360491927 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14801a0080 con 0x7f1480102a30 2024-04-21T08:36:47.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.979+0000 7f14866ba640 1 -- 172.21.15.86:0/3360491927 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14801a05d0 con 0x7f1480102a30 2024-04-21T08:36:47.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.980+0000 7f14866ba640 1 -- 172.21.15.86:0/3360491927 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f14800fdf30 con 0x7f1480102a30 2024-04-21T08:36:47.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.981+0000 7f147d7fa640 1 -- 172.21.15.86:0/3360491927 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f1470004f20 con 0x7f1480102a30 2024-04-21T08:36:47.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.982+0000 7f147d7fa640 1 --2- 172.21.15.86:0/3360491927 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f14680a3cc0 0x7f14680a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:47.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.982+0000 7f147f7fe640 1 --2- 172.21.15.86:0/3360491927 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f14680a3cc0 0x7f14680a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:47.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.982+0000 7f147d7fa640 1 -- 172.21.15.86:0/3360491927 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f14700d04b0 con 0x7f1480102a30 2024-04-21T08:36:47.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.982+0000 7f147f7fe640 1 --2- 172.21.15.86:0/3360491927 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f14680a3cc0 0x7f14680a6180 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1480199680 tx=0x7f1474005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:47.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:47.986+0000 7f147d7fa640 1 -- 172.21.15.86:0/3360491927 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1470014030 con 0x7f1480102a30 2024-04-21T08:36:48.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.193+0000 7f14866ba640 1 -- 172.21.15.86:0/3360491927 --> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] -- mgr_command(tid 0: {"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}) -- 0x7f1480062390 con 0x7f14680a3cc0 2024-04-21T08:36:48.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.198+0000 7f147d7fa640 1 -- 172.21.15.86:0/3360491927 <== mgr.14152 v2:172.21.15.86:6800/1364156965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+54 (secure 0 0 0) 0x7f1480062390 con 0x7f14680a3cc0 2024-04-21T08:36:48.197 INFO:teuthology.orchestra.run.smithi086.stdout:/volumes/qa/sv_1/3e019f7c-7e70-4c51-be9e-eaa2c75bd456 2024-04-21T08:36:48.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.200+0000 7f14866ba640 1 -- 172.21.15.86:0/3360491927 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f14680a3cc0 msgr2=0x7f14680a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:48.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.200+0000 7f14866ba640 1 --2- 172.21.15.86:0/3360491927 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f14680a3cc0 0x7f14680a6180 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f1480199680 tx=0x7f1474005f90 comp rx=0 tx=0).stop 2024-04-21T08:36:48.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.200+0000 7f14866ba640 1 -- 172.21.15.86:0/3360491927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1480102a30 msgr2=0x7f1480198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:48.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.200+0000 7f14866ba640 1 --2- 172.21.15.86:0/3360491927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1480102a30 0x7f1480198170 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7f147000c970 tx=0x7f147000ce30 comp rx=0 tx=0).stop 2024-04-21T08:36:48.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.200+0000 7f14866ba640 1 -- 172.21.15.86:0/3360491927 shutdown_connections 2024-04-21T08:36:48.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.200+0000 7f14866ba640 1 --2- 172.21.15.86:0/3360491927 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f14680a3cc0 0x7f14680a6180 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:48.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.200+0000 7f14866ba640 1 --2- 172.21.15.86:0/3360491927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1480107c90 0x7f148019f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:48.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.200+0000 7f14866ba640 1 --2- 172.21.15.86:0/3360491927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1480105360 0x7f14801986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:48.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.200+0000 7f14866ba640 1 --2- 172.21.15.86:0/3360491927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1480102a30 0x7f1480198170 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:48.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.200+0000 7f14866ba640 1 -- 172.21.15.86:0/3360491927 >> 172.21.15.86:0/3360491927 conn(0x7f14800fc440 msgr2=0x7f1480103370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:48.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.200+0000 7f14866ba640 1 -- 172.21.15.86:0/3360491927 shutdown_connections 2024-04-21T08:36:48.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.200+0000 7f14866ba640 1 -- 172.21.15.86:0/3360491927 wait complete. 2024-04-21T08:36:48.210 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:36:48.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.309+0000 7fbac5c93640 1 -- 172.21.15.86:0/1723359849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbac0100e70 msgr2=0x7fbac0101290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:48.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.309+0000 7fbac5c93640 1 --2- 172.21.15.86:0/1723359849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbac0100e70 0x7fbac0101290 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fbab0009900 tx=0x7fbab002bc50 comp rx=0 tx=0).stop 2024-04-21T08:36:48.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.309+0000 7fbac5c93640 1 -- 172.21.15.86:0/1723359849 shutdown_connections 2024-04-21T08:36:48.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.309+0000 7fbac5c93640 1 --2- 172.21.15.86:0/1723359849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbac0102a10 0x7fbac0109330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:48.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.309+0000 7fbac5c93640 1 --2- 172.21.15.86:0/1723359849 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbac0102050 0x7fbac01024d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:48.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.309+0000 7fbac5c93640 1 --2- 172.21.15.86:0/1723359849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbac0100e70 0x7fbac0101290 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:48.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.309+0000 7fbac5c93640 1 -- 172.21.15.86:0/1723359849 >> 172.21.15.86:0/1723359849 conn(0x7fbac00fc470 msgr2=0x7fbac00fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:48.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.309+0000 7fbac5c93640 1 -- 172.21.15.86:0/1723359849 shutdown_connections 2024-04-21T08:36:48.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.309+0000 7fbac5c93640 1 -- 172.21.15.86:0/1723359849 wait complete. 2024-04-21T08:36:48.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.309+0000 7fbac5c93640 1 Processor -- start 2024-04-21T08:36:48.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.310+0000 7fbac5c93640 1 -- start start 2024-04-21T08:36:48.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.310+0000 7fbac5c93640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbac0100e70 0x7fbac006b190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:48.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.310+0000 7fbac5c93640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbac0102050 0x7fbac019cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:48.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.310+0000 7fbac5c93640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbac0102a10 0x7fbac019f590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:48.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.310+0000 7fbac5c93640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbac010ca80 con 0x7fbac0100e70 2024-04-21T08:36:48.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.310+0000 7fbac5c93640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbac010c900 con 0x7fbac0102050 2024-04-21T08:36:48.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.310+0000 7fbac5c93640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbac010cc00 con 0x7fbac0102a10 2024-04-21T08:36:48.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.310+0000 7fbabf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbac0100e70 0x7fbac006b190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:48.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.310+0000 7fbabf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbac0100e70 0x7fbac006b190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35230/0 (socket says 172.21.15.86:35230) 2024-04-21T08:36:48.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.310+0000 7fbabf7fe640 1 -- 172.21.15.86:0/1347651182 learned_addr learned my addr 172.21.15.86:0/1347651182 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:48.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.310+0000 7fbabeffd640 1 --2- 172.21.15.86:0/1347651182 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbac0102050 0x7fbac019cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:48.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.310+0000 7fbabffff640 1 --2- 172.21.15.86:0/1347651182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbac0102a10 0x7fbac019f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:48.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.310+0000 7fbabf7fe640 1 -- 172.21.15.86:0/1347651182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbac0102a10 msgr2=0x7fbac019f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:48.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.310+0000 7fbabf7fe640 1 --2- 172.21.15.86:0/1347651182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbac0102a10 0x7fbac019f590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:48.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.310+0000 7fbabf7fe640 1 -- 172.21.15.86:0/1347651182 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbac0102050 msgr2=0x7fbac019cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:48.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.310+0000 7fbabf7fe640 1 --2- 172.21.15.86:0/1347651182 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbac0102050 0x7fbac019cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:48.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.310+0000 7fbabf7fe640 1 -- 172.21.15.86:0/1347651182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbac019fea0 con 0x7fbac0100e70 2024-04-21T08:36:48.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.310+0000 7fbabffff640 1 --2- 172.21.15.86:0/1347651182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbac0102a10 0x7fbac019f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:36:48.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.310+0000 7fbabf7fe640 1 --2- 172.21.15.86:0/1347651182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbac0100e70 0x7fbac006b190 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbab0000c00 tx=0x7fbab0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:48.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.311+0000 7fbabcff9640 1 -- 172.21.15.86:0/1347651182 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab0043020 con 0x7fbac0100e70 2024-04-21T08:36:48.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.311+0000 7fbac5c93640 1 -- 172.21.15.86:0/1347651182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbac01a00d0 con 0x7fbac0100e70 2024-04-21T08:36:48.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.311+0000 7fbabcff9640 1 -- 172.21.15.86:0/1347651182 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbab00052b0 con 0x7fbac0100e70 2024-04-21T08:36:48.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.311+0000 7fbabcff9640 1 -- 172.21.15.86:0/1347651182 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbab0003ca0 con 0x7fbac0100e70 2024-04-21T08:36:48.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.311+0000 7fbac5c93640 1 -- 172.21.15.86:0/1347651182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbac01a06a0 con 0x7fbac0100e70 2024-04-21T08:36:48.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.312+0000 7fbac5c93640 1 -- 172.21.15.86:0/1347651182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbac0108210 con 0x7fbac0100e70 2024-04-21T08:36:48.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.313+0000 7fbabcff9640 1 -- 172.21.15.86:0/1347651182 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbab0046020 con 0x7fbac0100e70 2024-04-21T08:36:48.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.313+0000 7fbabcff9640 1 --2- 172.21.15.86:0/1347651182 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fbaa80a3c40 0x7fbaa80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:48.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.313+0000 7fbabeffd640 1 --2- 172.21.15.86:0/1347651182 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fbaa80a3c40 0x7fbaa80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:48.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.313+0000 7fbabcff9640 1 -- 172.21.15.86:0/1347651182 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fbab00f2990 con 0x7fbac0100e70 2024-04-21T08:36:48.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.314+0000 7fbabeffd640 1 --2- 172.21.15.86:0/1347651182 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fbaa80a3c40 0x7fbaa80a6100 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbab40095f0 tx=0x7fbab4005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:48.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.318+0000 7fbabcff9640 1 -- 172.21.15.86:0/1347651182 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbab004d020 con 0x7fbac0100e70 2024-04-21T08:36:48.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.538+0000 7fbac5c93640 1 -- 172.21.15.86:0/1347651182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbac002d5c0 con 0x7fbac0100e70 2024-04-21T08:36:48.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.539+0000 7fbabcff9640 1 -- 172.21.15.86:0/1347651182 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7fbab00bf0e0 con 0x7fbac0100e70 2024-04-21T08:36:48.538 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:36:48.538 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:36:47.458637+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24455,24472,24394,24398,24482]},"id":1}]} 2024-04-21T08:36:48.539 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-21T08:36:48.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.541+0000 7fbac5c93640 1 -- 172.21.15.86:0/1347651182 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fbaa80a3c40 msgr2=0x7fbaa80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:48.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.541+0000 7fbac5c93640 1 --2- 172.21.15.86:0/1347651182 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fbaa80a3c40 0x7fbaa80a6100 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7fbab40095f0 tx=0x7fbab4005f90 comp rx=0 tx=0).stop 2024-04-21T08:36:48.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.541+0000 7fbac5c93640 1 -- 172.21.15.86:0/1347651182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbac0100e70 msgr2=0x7fbac006b190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:48.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.541+0000 7fbac5c93640 1 --2- 172.21.15.86:0/1347651182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbac0100e70 0x7fbac006b190 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7fbab0000c00 tx=0x7fbab0004040 comp rx=0 tx=0).stop 2024-04-21T08:36:48.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.541+0000 7fbac5c93640 1 -- 172.21.15.86:0/1347651182 shutdown_connections 2024-04-21T08:36:48.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.541+0000 7fbac5c93640 1 --2- 172.21.15.86:0/1347651182 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fbaa80a3c40 0x7fbaa80a6100 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:48.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.541+0000 7fbac5c93640 1 --2- 172.21.15.86:0/1347651182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbac0102a10 0x7fbac019f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:48.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.541+0000 7fbac5c93640 1 --2- 172.21.15.86:0/1347651182 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbac0102050 0x7fbac019cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:48.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.541+0000 7fbac5c93640 1 --2- 172.21.15.86:0/1347651182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbac0100e70 0x7fbac006b190 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:48.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.541+0000 7fbac5c93640 1 -- 172.21.15.86:0/1347651182 >> 172.21.15.86:0/1347651182 conn(0x7fbac00fc470 msgr2=0x7fbac01053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:48.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.541+0000 7fbac5c93640 1 -- 172.21.15.86:0/1347651182 shutdown_connections 2024-04-21T08:36:48.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.541+0000 7fbac5c93640 1 -- 172.21.15.86:0/1347651182 wait complete. 2024-04-21T08:36:48.550 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:48 smithi086 ceph-mon[27939]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:36:48.550 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:48 smithi086 ceph-mon[27939]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:36:48.550 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:48 smithi086 ceph-mon[27939]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2024-04-21T08:36:48.550 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:48 smithi086 ceph-mon[27939]: mds.? [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] up:active 2024-04-21T08:36:48.550 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:48 smithi086 ceph-mon[27939]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-04-21T08:36:48.551 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:48 smithi086 ceph-mon[27939]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-21T08:36:48.551 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:48 smithi086 ceph-mon[27939]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:36:48.552 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:36:48.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.650+0000 7f261afbb640 1 -- 172.21.15.86:0/3580773711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2614100ea0 msgr2=0x7f26141012c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:48.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.650+0000 7f261afbb640 1 --2- 172.21.15.86:0/3580773711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2614100ea0 0x7f26141012c0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7f2608009900 tx=0x7f260802bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:48.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.650+0000 7f261afbb640 1 -- 172.21.15.86:0/3580773711 shutdown_connections 2024-04-21T08:36:48.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.650+0000 7f261afbb640 1 --2- 172.21.15.86:0/3580773711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26141029b0 0x7f26141092d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:48.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.650+0000 7f261afbb640 1 --2- 172.21.15.86:0/3580773711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2614101ff0 0x7f2614102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:48.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.650+0000 7f261afbb640 1 --2- 172.21.15.86:0/3580773711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2614100ea0 0x7f26141012c0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:48.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.650+0000 7f261afbb640 1 -- 172.21.15.86:0/3580773711 >> 172.21.15.86:0/3580773711 conn(0x7f26140fc410 msgr2=0x7f26140fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:48.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.650+0000 7f261afbb640 1 -- 172.21.15.86:0/3580773711 shutdown_connections 2024-04-21T08:36:48.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.650+0000 7f261afbb640 1 -- 172.21.15.86:0/3580773711 wait complete. 2024-04-21T08:36:48.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.651+0000 7f261afbb640 1 Processor -- start 2024-04-21T08:36:48.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.651+0000 7f261afbb640 1 -- start start 2024-04-21T08:36:48.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.651+0000 7f261afbb640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2614100ea0 0x7f2614197070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:48.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.651+0000 7f261afbb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2614101ff0 0x7f26141975b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:48.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.651+0000 7f261afbb640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26141029b0 0x7f261419e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:48.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.651+0000 7f261afbb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f261410c9d0 con 0x7f2614101ff0 2024-04-21T08:36:48.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.651+0000 7f261afbb640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f261410c850 con 0x7f26141029b0 2024-04-21T08:36:48.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.651+0000 7f261afbb640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f261410cb50 con 0x7f2614100ea0 2024-04-21T08:36:48.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.651+0000 7f2613fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2614101ff0 0x7f26141975b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:48.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.651+0000 7f2613fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2614101ff0 0x7f26141975b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35246/0 (socket says 172.21.15.86:35246) 2024-04-21T08:36:48.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.651+0000 7f2613fff640 1 -- 172.21.15.86:0/2258307924 learned_addr learned my addr 172.21.15.86:0/2258307924 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:48.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.651+0000 7f2618d57640 1 --2- 172.21.15.86:0/2258307924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2614100ea0 0x7f2614197070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:48.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.651+0000 7f2619558640 1 --2- 172.21.15.86:0/2258307924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26141029b0 0x7f261419e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:48.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.651+0000 7f2613fff640 1 -- 172.21.15.86:0/2258307924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2614100ea0 msgr2=0x7f2614197070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:48.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.651+0000 7f2613fff640 1 --2- 172.21.15.86:0/2258307924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2614100ea0 0x7f2614197070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:48.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.651+0000 7f2613fff640 1 -- 172.21.15.86:0/2258307924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26141029b0 msgr2=0x7f261419e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:48.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.651+0000 7f2613fff640 1 --2- 172.21.15.86:0/2258307924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26141029b0 0x7f261419e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:48.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.651+0000 7f2613fff640 1 -- 172.21.15.86:0/2258307924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f261419ec90 con 0x7f2614101ff0 2024-04-21T08:36:48.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.651+0000 7f2619558640 1 --2- 172.21.15.86:0/2258307924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26141029b0 0x7f261419e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:36:48.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.651+0000 7f2613fff640 1 --2- 172.21.15.86:0/2258307924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2614101ff0 0x7f26141975b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f260000d950 tx=0x7f260000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:48.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.652+0000 7f2611ffb640 1 -- 172.21.15.86:0/2258307924 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2600017020 con 0x7f2614101ff0 2024-04-21T08:36:48.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.652+0000 7f2611ffb640 1 -- 172.21.15.86:0/2258307924 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2600009c30 con 0x7f2614101ff0 2024-04-21T08:36:48.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.652+0000 7f261afbb640 1 -- 172.21.15.86:0/2258307924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f261419ef80 con 0x7f2614101ff0 2024-04-21T08:36:48.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.652+0000 7f2611ffb640 1 -- 172.21.15.86:0/2258307924 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f260000f690 con 0x7f2614101ff0 2024-04-21T08:36:48.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.652+0000 7f261afbb640 1 -- 172.21.15.86:0/2258307924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f261419f490 con 0x7f2614101ff0 2024-04-21T08:36:48.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.653+0000 7f261afbb640 1 -- 172.21.15.86:0/2258307924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f26141081b0 con 0x7f2614101ff0 2024-04-21T08:36:48.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.654+0000 7f2611ffb640 1 -- 172.21.15.86:0/2258307924 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f260001a020 con 0x7f2614101ff0 2024-04-21T08:36:48.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.654+0000 7f2611ffb640 1 --2- 172.21.15.86:0/2258307924 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f26040a3cc0 0x7f26040a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:48.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.654+0000 7f2618d57640 1 --2- 172.21.15.86:0/2258307924 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f26040a3cc0 0x7f26040a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:48.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.654+0000 7f2611ffb640 1 -- 172.21.15.86:0/2258307924 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f26000d0650 con 0x7f2614101ff0 2024-04-21T08:36:48.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.655+0000 7f2618d57640 1 --2- 172.21.15.86:0/2258307924 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f26040a3cc0 0x7f26040a6180 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2608009230 tx=0x7f2608005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:48.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.659+0000 7f2611ffb640 1 -- 172.21.15.86:0/2258307924 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2600021020 con 0x7f2614101ff0 2024-04-21T08:36:48.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:48 smithi088 ceph-mon[31259]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:36:48.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:48 smithi088 ceph-mon[31259]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:36:48.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:48 smithi088 ceph-mon[31259]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2024-04-21T08:36:48.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:48 smithi088 ceph-mon[31259]: mds.? [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] up:active 2024-04-21T08:36:48.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:48 smithi088 ceph-mon[31259]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-04-21T08:36:48.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:48 smithi088 ceph-mon[31259]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-21T08:36:48.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:48 smithi088 ceph-mon[31259]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:36:48.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:48 smithi179 ceph-mon[31829]: from='client.14910 -' entity='client.admin' cmd=[{"prefix": "fs subvolumegroup create", "vol_name": "cephfs", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:36:48.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:48 smithi179 ceph-mon[31829]: from='client.14922 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_0", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:36:48.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:48 smithi179 ceph-mon[31829]: pgmap v301: 129 pgs: 129 active+clean; 581 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 255 B/s wr, 1 op/s 2024-04-21T08:36:48.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:48 smithi179 ceph-mon[31829]: mds.? [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] up:active 2024-04-21T08:36:48.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:48 smithi179 ceph-mon[31829]: fsmap cephfs:5 3 up:standby-replay 4 up:standby 5 up:active 2024-04-21T08:36:48.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:48 smithi179 ceph-mon[31829]: mgrmap e17: x(active, since 7m), standbys: z, y 2024-04-21T08:36:48.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:48 smithi179 ceph-mon[31829]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:36:48.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.836+0000 7f261afbb640 1 -- 172.21.15.86:0/2258307924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f261402d770 con 0x7f2614101ff0 2024-04-21T08:36:48.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.838+0000 7f2611ffb640 1 -- 172.21.15.86:0/2258307924 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24070 (secure 0 0 0) 0x7f2600009de0 con 0x7f2614101ff0 2024-04-21T08:36:48.837 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:36:48.838 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":89,"fsid":"f8cec7cc-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:22.687329+0000","modified":"2024-04-21T08:36:25.251657+0000","last_up_change":"2024-04-21T08:35:30.048315+0000","last_in_change":"2024-04-21T08:35:10.099496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:45.279109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:18.300782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:19.574636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:22.905632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89b7601f-0620-4db2-90f1-b45cea173738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6803","nonce":2571516897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6805","nonce":2571516897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6809","nonce":2571516897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6807","nonce":2571516897}]},"public_addr":"172.21.15.86:6803/2571516897","cluster_addr":"172.21.15.86:6805/2571516897","heartbeat_back_addr":"172.21.15.86:6809/2571516897","heartbeat_front_addr":"172.21.15.86:6807/2571516897","state":["exists","up"]},{"osd":1,"uuid":"4891031c-f98c-42bc-bb3b-a420855ecc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6801","nonce":74067182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6803","nonce":74067182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6807","nonce":74067182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6805","nonce":74067182}]},"public_addr":"172.21.15.88:6801/74067182","cluster_addr":"172.21.15.88:6803/74067182","heartbeat_back_addr":"172.21.15.88:6807/74067182","heartbeat_front_addr":"172.21.15.88:6805/74067182","state":["exists","up"]},{"osd":2,"uuid":"60d12aa4-9312-4fa2-8d0b-2b3f42caad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6801","nonce":4137659051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6803","nonce":4137659051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6807","nonce":4137659051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6805","nonce":4137659051}]},"public_addr":"172.21.15.179:6801/4137659051","cluster_addr":"172.21.15.179:6803/4137659051","heartbeat_back_addr":"172.21.15.179:6807/4137659051","heartbeat_front_addr":"172.21.15.179:6805/4137659051","state":["exists","up"]},{"osd":3,"uuid":"404c3158-8cb0-4c02-b48a-f9ee5ff7b784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6811","nonce":2309603148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6813","nonce":2309603148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6817","nonce":2309603148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6815","nonce":2309603148}]},"public_addr":"172.21.15.86:6811/2309603148","cluster_addr":"172.21.15.86:6813/2309603148","heartbeat_back_addr":"172.21.15.86:6817/2309603148","heartbeat_front_addr":"172.21.15.86:6815/2309603148","state":["exists","up"]},{"osd":4,"uuid":"458ff202-d781-4ec7-ac93-0b49f835b503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6809","nonce":1818714217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6811","nonce":1818714217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6815","nonce":1818714217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6813","nonce":1818714217}]},"public_addr":"172.21.15.88:6809/1818714217","cluster_addr":"172.21.15.88:6811/1818714217","heartbeat_back_addr":"172.21.15.88:6815/1818714217","heartbeat_front_addr":"172.21.15.88:6813/1818714217","state":["exists","up"]},{"osd":5,"uuid":"8123cc90-bd8b-4204-bf31-46b34e9f589d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6809","nonce":863840990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6811","nonce":863840990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6815","nonce":863840990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6813","nonce":863840990}]},"public_addr":"172.21.15.179:6809/863840990","cluster_addr":"172.21.15.179:6811/863840990","heartbeat_back_addr":"172.21.15.179:6815/863840990","heartbeat_front_addr":"172.21.15.179:6813/863840990","state":["exists","up"]},{"osd":6,"uuid":"aac9d7c7-659a-4110-903a-a871c086066f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6819","nonce":1224231118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6821","nonce":1224231118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6825","nonce":1224231118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6823","nonce":1224231118}]},"public_addr":"172.21.15.86:6819/1224231118","cluster_addr":"172.21.15.86:6821/1224231118","heartbeat_back_addr":"172.21.15.86:6825/1224231118","heartbeat_front_addr":"172.21.15.86:6823/1224231118","state":["exists","up"]},{"osd":7,"uuid":"df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6817","nonce":136629878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6819","nonce":136629878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6823","nonce":136629878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6821","nonce":136629878}]},"public_addr":"172.21.15.88:6817/136629878","cluster_addr":"172.21.15.88:6819/136629878","heartbeat_back_addr":"172.21.15.88:6823/136629878","heartbeat_front_addr":"172.21.15.88:6821/136629878","state":["exists","up"]},{"osd":8,"uuid":"63e59f4b-0426-46db-92f7-c55dae61b5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6817","nonce":1770418957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6819","nonce":1770418957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6823","nonce":1770418957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6821","nonce":1770418957}]},"public_addr":"172.21.15.179:6817/1770418957","cluster_addr":"172.21.15.179:6819/1770418957","heartbeat_back_addr":"172.21.15.179:6823/1770418957","heartbeat_front_addr":"172.21.15.179:6821/1770418957","state":["exists","up"]},{"osd":9,"uuid":"5841bb36-ed6b-45d8-b857-eeab139c44e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6827","nonce":1108500274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6829","nonce":1108500274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6833","nonce":1108500274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6831","nonce":1108500274}]},"public_addr":"172.21.15.86:6827/1108500274","cluster_addr":"172.21.15.86:6829/1108500274","heartbeat_back_addr":"172.21.15.86:6833/1108500274","heartbeat_front_addr":"172.21.15.86:6831/1108500274","state":["exists","up"]},{"osd":10,"uuid":"f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6825","nonce":1294275353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6827","nonce":1294275353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6831","nonce":1294275353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6829","nonce":1294275353}]},"public_addr":"172.21.15.88:6825/1294275353","cluster_addr":"172.21.15.88:6827/1294275353","heartbeat_back_addr":"172.21.15.88:6831/1294275353","heartbeat_front_addr":"172.21.15.88:6829/1294275353","state":["exists","up"]},{"osd":11,"uuid":"e0d4cc76-e7a6-4a6b-909c-95488ccd5a36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6825","nonce":4007070233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6827","nonce":4007070233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6831","nonce":4007070233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6829","nonce":4007070233}]},"public_addr":"172.21.15.179:6825/4007070233","cluster_addr":"172.21.15.179:6827/4007070233","heartbeat_back_addr":"172.21.15.179:6831/4007070233","heartbeat_front_addr":"172.21.15.179:6829/4007070233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:55.210605+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:18.723090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:42.547979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:06.421622+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:30.858315+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:55.455313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.623369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:46.185111+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.103627+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:37.297545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:02.637431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:28.199292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6800/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/1412393225":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6800/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/1687852404":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2020707404":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2568130933":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/3952607648":"2024-04-22T08:28:57.384403+0000","172.21.15.86:6801/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/2756903426":"2024-04-22T08:29:23.190001+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:36:48.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.840+0000 7f261afbb640 1 -- 172.21.15.86:0/2258307924 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f26040a3cc0 msgr2=0x7f26040a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:48.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.840+0000 7f261afbb640 1 --2- 172.21.15.86:0/2258307924 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f26040a3cc0 0x7f26040a6180 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f2608009230 tx=0x7f2608005fd0 comp rx=0 tx=0).stop 2024-04-21T08:36:48.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.840+0000 7f261afbb640 1 -- 172.21.15.86:0/2258307924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2614101ff0 msgr2=0x7f26141975b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:48.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.840+0000 7f261afbb640 1 --2- 172.21.15.86:0/2258307924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2614101ff0 0x7f26141975b0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f260000d950 tx=0x7f260000de10 comp rx=0 tx=0).stop 2024-04-21T08:36:48.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.840+0000 7f261afbb640 1 -- 172.21.15.86:0/2258307924 shutdown_connections 2024-04-21T08:36:48.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.840+0000 7f261afbb640 1 --2- 172.21.15.86:0/2258307924 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f26040a3cc0 0x7f26040a6180 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:48.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.840+0000 7f261afbb640 1 --2- 172.21.15.86:0/2258307924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f26141029b0 0x7f261419e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:48.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.840+0000 7f261afbb640 1 --2- 172.21.15.86:0/2258307924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2614101ff0 0x7f26141975b0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:48.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.840+0000 7f261afbb640 1 --2- 172.21.15.86:0/2258307924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2614100ea0 0x7f2614197070 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:48.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.840+0000 7f261afbb640 1 -- 172.21.15.86:0/2258307924 >> 172.21.15.86:0/2258307924 conn(0x7f26140fc410 msgr2=0x7f2614105380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:48.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.840+0000 7f261afbb640 1 -- 172.21.15.86:0/2258307924 shutdown_connections 2024-04-21T08:36:48.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.840+0000 7f261afbb640 1 -- 172.21.15.86:0/2258307924 wait complete. 2024-04-21T08:36:48.851 INFO:tasks.cephfs.filesystem.ceph_manager:waiting for clean 2024-04-21T08:36:48.851 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-21T08:36:48.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.952+0000 7f67577b2640 1 -- 172.21.15.86:0/3694994538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6750100270 msgr2=0x7f6750100690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:48.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.952+0000 7f67577b2640 1 --2- 172.21.15.86:0/3694994538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6750100270 0x7f6750100690 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7f674c009930 tx=0x7f674c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:36:48.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.952+0000 7f67577b2640 1 -- 172.21.15.86:0/3694994538 shutdown_connections 2024-04-21T08:36:48.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.952+0000 7f67577b2640 1 --2- 172.21.15.86:0/3694994538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6750103ca0 0x7f675010a530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:48.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.952+0000 7f67577b2640 1 --2- 172.21.15.86:0/3694994538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67501032e0 0x7f6750103760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:48.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.952+0000 7f67577b2640 1 --2- 172.21.15.86:0/3694994538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6750100270 0x7f6750100690 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:48.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.952+0000 7f67577b2640 1 -- 172.21.15.86:0/3694994538 >> 172.21.15.86:0/3694994538 conn(0x7f67500fbb80 msgr2=0x7f67500fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:48.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.952+0000 7f67577b2640 1 -- 172.21.15.86:0/3694994538 shutdown_connections 2024-04-21T08:36:48.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.952+0000 7f67577b2640 1 -- 172.21.15.86:0/3694994538 wait complete. 2024-04-21T08:36:48.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.953+0000 7f67577b2640 1 Processor -- start 2024-04-21T08:36:48.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.953+0000 7f67577b2640 1 -- start start 2024-04-21T08:36:48.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.953+0000 7f67577b2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6750100270 0x7f67501981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:48.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.953+0000 7f67577b2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67501032e0 0x7f6750198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:48.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.953+0000 7f67577b2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6750103ca0 0x7f675019f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:48.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.953+0000 7f67577b2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f675010dae0 con 0x7f6750103ca0 2024-04-21T08:36:48.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.953+0000 7f67577b2640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f675010d960 con 0x7f6750100270 2024-04-21T08:36:48.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.953+0000 7f67577b2640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f675010dc60 con 0x7f67501032e0 2024-04-21T08:36:48.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.953+0000 7f6755d4f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6750103ca0 0x7f675019f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:48.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.953+0000 7f6755d4f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6750103ca0 0x7f675019f790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35270/0 (socket says 172.21.15.86:35270) 2024-04-21T08:36:48.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.953+0000 7f6755d4f640 1 -- 172.21.15.86:0/2801806442 learned_addr learned my addr 172.21.15.86:0/2801806442 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:48.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.953+0000 7f675554e640 1 --2- 172.21.15.86:0/2801806442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6750100270 0x7f67501981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:48.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.953+0000 7f6755d4f640 1 -- 172.21.15.86:0/2801806442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67501032e0 msgr2=0x7f6750198720 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T08:36:48.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.953+0000 7f6755d4f640 1 --2- 172.21.15.86:0/2801806442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67501032e0 0x7f6750198720 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:48.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.953+0000 7f6755d4f640 1 -- 172.21.15.86:0/2801806442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6750100270 msgr2=0x7f67501981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:48.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.953+0000 7f6755d4f640 1 --2- 172.21.15.86:0/2801806442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6750100270 0x7f67501981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:48.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.953+0000 7f6755d4f640 1 -- 172.21.15.86:0/2801806442 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f675019fe00 con 0x7f6750103ca0 2024-04-21T08:36:48.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.953+0000 7f6755d4f640 1 --2- 172.21.15.86:0/2801806442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6750103ca0 0x7f675019f790 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f674800eaa0 tx=0x7f674800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:48.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.954+0000 7f67467fc640 1 -- 172.21.15.86:0/2801806442 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f674800cdb0 con 0x7f6750103ca0 2024-04-21T08:36:48.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.954+0000 7f67467fc640 1 -- 172.21.15.86:0/2801806442 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6748005020 con 0x7f6750103ca0 2024-04-21T08:36:48.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.954+0000 7f67577b2640 1 -- 172.21.15.86:0/2801806442 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67501a00f0 con 0x7f6750103ca0 2024-04-21T08:36:48.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.954+0000 7f67467fc640 1 -- 172.21.15.86:0/2801806442 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67480116d0 con 0x7f6750103ca0 2024-04-21T08:36:48.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.954+0000 7f67577b2640 1 -- 172.21.15.86:0/2801806442 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67501a05f0 con 0x7f6750103ca0 2024-04-21T08:36:48.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.955+0000 7f67577b2640 1 -- 172.21.15.86:0/2801806442 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6750109530 con 0x7f6750103ca0 2024-04-21T08:36:48.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.955+0000 7f67467fc640 1 -- 172.21.15.86:0/2801806442 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f67480051d0 con 0x7f6750103ca0 2024-04-21T08:36:48.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.956+0000 7f67467fc640 1 --2- 172.21.15.86:0/2801806442 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f673c0a3c40 0x7f673c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:48.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.956+0000 7f675554e640 1 --2- 172.21.15.86:0/2801806442 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f673c0a3c40 0x7f673c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:48.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.956+0000 7f67467fc640 1 -- 172.21.15.86:0/2801806442 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f67480d0a10 con 0x7f6750103ca0 2024-04-21T08:36:48.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.957+0000 7f675554e640 1 --2- 172.21.15.86:0/2801806442 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f673c0a3c40 0x7f673c0a6100 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f674c009900 tx=0x7f674c005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:48.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:48.961+0000 7f67467fc640 1 -- 172.21.15.86:0/2801806442 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6748016030 con 0x7f6750103ca0 2024-04-21T08:36:49.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.136+0000 7f67577b2640 1 -- 172.21.15.86:0/2801806442 --> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6750062390 con 0x7f673c0a3c40 2024-04-21T08:36:49.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.146+0000 7f67467fc640 1 -- 172.21.15.86:0/2801806442 <== mgr.14152 v2:172.21.15.86:6800/1364156965 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340228 (secure 0 0 0) 0x7f6750062390 con 0x7f673c0a3c40 2024-04-21T08:36:49.150 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:36:49.152 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-21T08:36:49.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.150+0000 7f67577b2640 1 -- 172.21.15.86:0/2801806442 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f673c0a3c40 msgr2=0x7f673c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:49.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.150+0000 7f67577b2640 1 --2- 172.21.15.86:0/2801806442 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f673c0a3c40 0x7f673c0a6100 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f674c009900 tx=0x7f674c005fd0 comp rx=0 tx=0).stop 2024-04-21T08:36:49.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.150+0000 7f67577b2640 1 -- 172.21.15.86:0/2801806442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6750103ca0 msgr2=0x7f675019f790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:49.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.150+0000 7f67577b2640 1 --2- 172.21.15.86:0/2801806442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6750103ca0 0x7f675019f790 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f674800eaa0 tx=0x7f674800ef60 comp rx=0 tx=0).stop 2024-04-21T08:36:49.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.150+0000 7f67577b2640 1 -- 172.21.15.86:0/2801806442 shutdown_connections 2024-04-21T08:36:49.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.150+0000 7f67577b2640 1 --2- 172.21.15.86:0/2801806442 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f673c0a3c40 0x7f673c0a6100 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:49.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.150+0000 7f67577b2640 1 --2- 172.21.15.86:0/2801806442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6750103ca0 0x7f675019f790 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:49.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.150+0000 7f67577b2640 1 --2- 172.21.15.86:0/2801806442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67501032e0 0x7f6750198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:49.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.150+0000 7f67577b2640 1 --2- 172.21.15.86:0/2801806442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6750100270 0x7f67501981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:49.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.150+0000 7f67577b2640 1 -- 172.21.15.86:0/2801806442 >> 172.21.15.86:0/2801806442 conn(0x7f67500fbb80 msgr2=0x7f6750106700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:49.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.150+0000 7f67577b2640 1 -- 172.21.15.86:0/2801806442 shutdown_connections 2024-04-21T08:36:49.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.150+0000 7f67577b2640 1 -- 172.21.15.86:0/2801806442 wait complete. 2024-04-21T08:36:49.160 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":301,"stamp":"2024-04-21T08:36:47.275264+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":595188,"num_objects":31,"num_object_clones":0,"num_object_copies":93,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":31,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":246,"num_write_kb":2958,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":191,"ondisk_log_size":191,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":370924,"kb_used_data":13392,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124570900,"statfs":{"total":1151940427776,"available":1151560601600,"internally_reserved":0,"allocated":13713408,"data_stored":7513772,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":365933884},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":321,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":18,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":5,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003599"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.112095+0000","last_change":"2024-04-21T08:36:20.913421+0000","last_active":"2024-04-21T08:36:26.112095+0000","last_peered":"2024-04-21T08:36:26.112095+0000","last_clean":"2024-04-21T08:36:26.112095+0000","last_became_active":"2024-04-21T08:36:20.913017+0000","last_became_peered":"2024-04-21T08:36:20.913017+0000","last_unstale":"2024-04-21T08:36:26.112095+0000","last_undegraded":"2024-04-21T08:36:26.112095+0000","last_fullsized":"2024-04-21T08:36:26.112095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:16:29.885916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.348243+0000","last_change":"2024-04-21T08:36:20.911675+0000","last_active":"2024-04-21T08:36:26.348243+0000","last_peered":"2024-04-21T08:36:26.348243+0000","last_clean":"2024-04-21T08:36:26.348243+0000","last_became_active":"2024-04-21T08:36:20.910591+0000","last_became_peered":"2024-04-21T08:36:20.910591+0000","last_unstale":"2024-04-21T08:36:26.348243+0000","last_undegraded":"2024-04-21T08:36:26.348243+0000","last_fullsized":"2024-04-21T08:36:26.348243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:01:40.735376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.348481+0000","last_change":"2024-04-21T08:36:20.911978+0000","last_active":"2024-04-21T08:36:26.348481+0000","last_peered":"2024-04-21T08:36:26.348481+0000","last_clean":"2024-04-21T08:36:26.348481+0000","last_became_active":"2024-04-21T08:36:20.911641+0000","last_became_peered":"2024-04-21T08:36:20.911641+0000","last_unstale":"2024-04-21T08:36:26.348481+0000","last_undegraded":"2024-04-21T08:36:26.348481+0000","last_fullsized":"2024-04-21T08:36:26.348481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:14:23.789790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111254+0000","last_change":"2024-04-21T08:36:20.919027+0000","last_active":"2024-04-21T08:36:26.111254+0000","last_peered":"2024-04-21T08:36:26.111254+0000","last_clean":"2024-04-21T08:36:26.111254+0000","last_became_active":"2024-04-21T08:36:20.918740+0000","last_became_peered":"2024-04-21T08:36:20.918740+0000","last_unstale":"2024-04-21T08:36:26.111254+0000","last_undegraded":"2024-04-21T08:36:26.111254+0000","last_fullsized":"2024-04-21T08:36:26.111254+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:00:31.066938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110759+0000","last_change":"2024-04-21T08:36:20.907175+0000","last_active":"2024-04-21T08:36:26.110759+0000","last_peered":"2024-04-21T08:36:26.110759+0000","last_clean":"2024-04-21T08:36:26.110759+0000","last_became_active":"2024-04-21T08:36:20.906818+0000","last_became_peered":"2024-04-21T08:36:20.906818+0000","last_unstale":"2024-04-21T08:36:26.110759+0000","last_undegraded":"2024-04-21T08:36:26.110759+0000","last_fullsized":"2024-04-21T08:36:26.110759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:41:03.332231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111378+0000","last_change":"2024-04-21T08:36:20.905968+0000","last_active":"2024-04-21T08:36:26.111378+0000","last_peered":"2024-04-21T08:36:26.111378+0000","last_clean":"2024-04-21T08:36:26.111378+0000","last_became_active":"2024-04-21T08:36:20.904840+0000","last_became_peered":"2024-04-21T08:36:20.904840+0000","last_unstale":"2024-04-21T08:36:26.111378+0000","last_undegraded":"2024-04-21T08:36:26.111378+0000","last_fullsized":"2024-04-21T08:36:26.111378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:23:46.886241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.428415+0000","last_change":"2024-04-21T08:36:20.912045+0000","last_active":"2024-04-21T08:36:25.428415+0000","last_peered":"2024-04-21T08:36:25.428415+0000","last_clean":"2024-04-21T08:36:25.428415+0000","last_became_active":"2024-04-21T08:36:20.911690+0000","last_became_peered":"2024-04-21T08:36:20.911690+0000","last_unstale":"2024-04-21T08:36:25.428415+0000","last_undegraded":"2024-04-21T08:36:25.428415+0000","last_fullsized":"2024-04-21T08:36:25.428415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:06:28.229307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110591+0000","last_change":"2024-04-21T08:36:20.904874+0000","last_active":"2024-04-21T08:36:26.110591+0000","last_peered":"2024-04-21T08:36:26.110591+0000","last_clean":"2024-04-21T08:36:26.110591+0000","last_became_active":"2024-04-21T08:36:20.904217+0000","last_became_peered":"2024-04-21T08:36:20.904217+0000","last_unstale":"2024-04-21T08:36:26.110591+0000","last_undegraded":"2024-04-21T08:36:26.110591+0000","last_fullsized":"2024-04-21T08:36:26.110591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:37:04.423812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.025560+0000","last_change":"2024-04-21T08:36:20.912874+0000","last_active":"2024-04-21T08:36:26.025560+0000","last_peered":"2024-04-21T08:36:26.025560+0000","last_clean":"2024-04-21T08:36:26.025560+0000","last_became_active":"2024-04-21T08:36:20.912569+0000","last_became_peered":"2024-04-21T08:36:20.912569+0000","last_unstale":"2024-04-21T08:36:26.025560+0000","last_undegraded":"2024-04-21T08:36:26.025560+0000","last_fullsized":"2024-04-21T08:36:26.025560+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:55:20.763157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111846+0000","last_change":"2024-04-21T08:36:20.911125+0000","last_active":"2024-04-21T08:36:26.111846+0000","last_peered":"2024-04-21T08:36:26.111846+0000","last_clean":"2024-04-21T08:36:26.111846+0000","last_became_active":"2024-04-21T08:36:20.910614+0000","last_became_peered":"2024-04-21T08:36:20.910614+0000","last_unstale":"2024-04-21T08:36:26.111846+0000","last_undegraded":"2024-04-21T08:36:26.111846+0000","last_fullsized":"2024-04-21T08:36:26.111846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:20:16.448824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.428995+0000","last_change":"2024-04-21T08:36:20.909094+0000","last_active":"2024-04-21T08:36:25.428995+0000","last_peered":"2024-04-21T08:36:25.428995+0000","last_clean":"2024-04-21T08:36:25.428995+0000","last_became_active":"2024-04-21T08:36:20.908634+0000","last_became_peered":"2024-04-21T08:36:20.908634+0000","last_unstale":"2024-04-21T08:36:25.428995+0000","last_undegraded":"2024-04-21T08:36:25.428995+0000","last_fullsized":"2024-04-21T08:36:25.428995+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:28:38.450621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110958+0000","last_change":"2024-04-21T08:36:20.909429+0000","last_active":"2024-04-21T08:36:26.110958+0000","last_peered":"2024-04-21T08:36:26.110958+0000","last_clean":"2024-04-21T08:36:26.110958+0000","last_became_active":"2024-04-21T08:36:20.909081+0000","last_became_peered":"2024-04-21T08:36:20.909081+0000","last_unstale":"2024-04-21T08:36:26.110958+0000","last_undegraded":"2024-04-21T08:36:26.110958+0000","last_fullsized":"2024-04-21T08:36:26.110958+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:45:46.939125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110964+0000","last_change":"2024-04-21T08:36:20.895894+0000","last_active":"2024-04-21T08:36:26.110964+0000","last_peered":"2024-04-21T08:36:26.110964+0000","last_clean":"2024-04-21T08:36:26.110964+0000","last_became_active":"2024-04-21T08:36:20.895572+0000","last_became_peered":"2024-04-21T08:36:20.895572+0000","last_unstale":"2024-04-21T08:36:26.110964+0000","last_undegraded":"2024-04-21T08:36:26.110964+0000","last_fullsized":"2024-04-21T08:36:26.110964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:12:48.750555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.348127+0000","last_change":"2024-04-21T08:36:20.905356+0000","last_active":"2024-04-21T08:36:26.348127+0000","last_peered":"2024-04-21T08:36:26.348127+0000","last_clean":"2024-04-21T08:36:26.348127+0000","last_became_active":"2024-04-21T08:36:20.904954+0000","last_became_peered":"2024-04-21T08:36:20.904954+0000","last_unstale":"2024-04-21T08:36:26.348127+0000","last_undegraded":"2024-04-21T08:36:26.348127+0000","last_fullsized":"2024-04-21T08:36:26.348127+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:45:49.116482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.348172+0000","last_change":"2024-04-21T08:36:20.911550+0000","last_active":"2024-04-21T08:36:26.348172+0000","last_peered":"2024-04-21T08:36:26.348172+0000","last_clean":"2024-04-21T08:36:26.348172+0000","last_became_active":"2024-04-21T08:36:20.910557+0000","last_became_peered":"2024-04-21T08:36:20.910557+0000","last_unstale":"2024-04-21T08:36:26.348172+0000","last_undegraded":"2024-04-21T08:36:26.348172+0000","last_fullsized":"2024-04-21T08:36:26.348172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:09:49.422729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.024946+0000","last_change":"2024-04-21T08:36:20.903954+0000","last_active":"2024-04-21T08:36:26.024946+0000","last_peered":"2024-04-21T08:36:26.024946+0000","last_clean":"2024-04-21T08:36:26.024946+0000","last_became_active":"2024-04-21T08:36:20.903641+0000","last_became_peered":"2024-04-21T08:36:20.903641+0000","last_unstale":"2024-04-21T08:36:26.024946+0000","last_undegraded":"2024-04-21T08:36:26.024946+0000","last_fullsized":"2024-04-21T08:36:26.024946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:08:55.678940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.112308+0000","last_change":"2024-04-21T08:36:20.914256+0000","last_active":"2024-04-21T08:36:26.112308+0000","last_peered":"2024-04-21T08:36:26.112308+0000","last_clean":"2024-04-21T08:36:26.112308+0000","last_became_active":"2024-04-21T08:36:20.913858+0000","last_became_peered":"2024-04-21T08:36:20.913858+0000","last_unstale":"2024-04-21T08:36:26.112308+0000","last_undegraded":"2024-04-21T08:36:26.112308+0000","last_fullsized":"2024-04-21T08:36:26.112308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:01:10.276763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.427500+0000","last_change":"2024-04-21T08:36:20.904138+0000","last_active":"2024-04-21T08:36:25.427500+0000","last_peered":"2024-04-21T08:36:25.427500+0000","last_clean":"2024-04-21T08:36:25.427500+0000","last_became_active":"2024-04-21T08:36:20.903670+0000","last_became_peered":"2024-04-21T08:36:20.903670+0000","last_unstale":"2024-04-21T08:36:25.427500+0000","last_undegraded":"2024-04-21T08:36:25.427500+0000","last_fullsized":"2024-04-21T08:36:25.427500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:38:50.160867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.429013+0000","last_change":"2024-04-21T08:36:20.910798+0000","last_active":"2024-04-21T08:36:25.429013+0000","last_peered":"2024-04-21T08:36:25.429013+0000","last_clean":"2024-04-21T08:36:25.429013+0000","last_became_active":"2024-04-21T08:36:20.910411+0000","last_became_peered":"2024-04-21T08:36:20.910411+0000","last_unstale":"2024-04-21T08:36:25.429013+0000","last_undegraded":"2024-04-21T08:36:25.429013+0000","last_fullsized":"2024-04-21T08:36:25.429013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:13:12.109818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.428444+0000","last_change":"2024-04-21T08:36:24.924831+0000","last_active":"2024-04-21T08:36:25.428444+0000","last_peered":"2024-04-21T08:36:25.428444+0000","last_clean":"2024-04-21T08:36:25.428444+0000","last_became_active":"2024-04-21T08:36:24.924435+0000","last_became_peered":"2024-04-21T08:36:24.924435+0000","last_unstale":"2024-04-21T08:36:25.428444+0000","last_undegraded":"2024-04-21T08:36:25.428444+0000","last_fullsized":"2024-04-21T08:36:25.428444+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:23.447079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.427560+0000","last_change":"2024-04-21T08:36:20.286270+0000","last_active":"2024-04-21T08:36:25.427560+0000","last_peered":"2024-04-21T08:36:25.427560+0000","last_clean":"2024-04-21T08:36:25.427560+0000","last_became_active":"2024-04-21T08:36:20.285959+0000","last_became_peered":"2024-04-21T08:36:20.285959+0000","last_unstale":"2024-04-21T08:36:25.427560+0000","last_undegraded":"2024-04-21T08:36:25.427560+0000","last_fullsized":"2024-04-21T08:36:25.427560+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:15:49.116420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111318+0000","last_change":"2024-04-21T08:36:20.912851+0000","last_active":"2024-04-21T08:36:26.111318+0000","last_peered":"2024-04-21T08:36:26.111318+0000","last_clean":"2024-04-21T08:36:26.111318+0000","last_became_active":"2024-04-21T08:36:20.912407+0000","last_became_peered":"2024-04-21T08:36:20.912407+0000","last_unstale":"2024-04-21T08:36:26.111318+0000","last_undegraded":"2024-04-21T08:36:26.111318+0000","last_fullsized":"2024-04-21T08:36:26.111318+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:45:18.026940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110547+0000","last_change":"2024-04-21T08:36:24.924052+0000","last_active":"2024-04-21T08:36:26.110547+0000","last_peered":"2024-04-21T08:36:26.110547+0000","last_clean":"2024-04-21T08:36:26.110547+0000","last_became_active":"2024-04-21T08:36:24.923618+0000","last_became_peered":"2024-04-21T08:36:24.923618+0000","last_unstale":"2024-04-21T08:36:26.110547+0000","last_undegraded":"2024-04-21T08:36:26.110547+0000","last_fullsized":"2024-04-21T08:36:26.110547+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:08:18.182386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111336+0000","last_change":"2024-04-21T08:36:20.285291+0000","last_active":"2024-04-21T08:36:26.111336+0000","last_peered":"2024-04-21T08:36:26.111336+0000","last_clean":"2024-04-21T08:36:26.111336+0000","last_became_active":"2024-04-21T08:36:20.284969+0000","last_became_peered":"2024-04-21T08:36:20.284969+0000","last_unstale":"2024-04-21T08:36:26.111336+0000","last_undegraded":"2024-04-21T08:36:26.111336+0000","last_fullsized":"2024-04-21T08:36:26.111336+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:55:29.484439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.348589+0000","last_change":"2024-04-21T08:36:20.902819+0000","last_active":"2024-04-21T08:36:26.348589+0000","last_peered":"2024-04-21T08:36:26.348589+0000","last_clean":"2024-04-21T08:36:26.348589+0000","last_became_active":"2024-04-21T08:36:20.902466+0000","last_became_peered":"2024-04-21T08:36:20.902466+0000","last_unstale":"2024-04-21T08:36:26.348589+0000","last_undegraded":"2024-04-21T08:36:26.348589+0000","last_fullsized":"2024-04-21T08:36:26.348589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:29:40.454026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110899+0000","last_change":"2024-04-21T08:36:24.926480+0000","last_active":"2024-04-21T08:36:26.110899+0000","last_peered":"2024-04-21T08:36:26.110899+0000","last_clean":"2024-04-21T08:36:26.110899+0000","last_became_active":"2024-04-21T08:36:24.926116+0000","last_became_peered":"2024-04-21T08:36:24.926116+0000","last_unstale":"2024-04-21T08:36:26.110899+0000","last_undegraded":"2024-04-21T08:36:26.110899+0000","last_fullsized":"2024-04-21T08:36:26.110899+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:43:18.839396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.112046+0000","last_change":"2024-04-21T08:36:20.293086+0000","last_active":"2024-04-21T08:36:26.112046+0000","last_peered":"2024-04-21T08:36:26.112046+0000","last_clean":"2024-04-21T08:36:26.112046+0000","last_became_active":"2024-04-21T08:36:20.291178+0000","last_became_peered":"2024-04-21T08:36:20.291178+0000","last_unstale":"2024-04-21T08:36:26.112046+0000","last_undegraded":"2024-04-21T08:36:26.112046+0000","last_fullsized":"2024-04-21T08:36:26.112046+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:02:32.935930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.025226+0000","last_change":"2024-04-21T08:36:20.909374+0000","last_active":"2024-04-21T08:36:26.025226+0000","last_peered":"2024-04-21T08:36:26.025226+0000","last_clean":"2024-04-21T08:36:26.025226+0000","last_became_active":"2024-04-21T08:36:20.909043+0000","last_became_peered":"2024-04-21T08:36:20.909043+0000","last_unstale":"2024-04-21T08:36:26.025226+0000","last_undegraded":"2024-04-21T08:36:26.025226+0000","last_fullsized":"2024-04-21T08:36:26.025226+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:45:45.719942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110695+0000","last_change":"2024-04-21T08:36:24.923989+0000","last_active":"2024-04-21T08:36:26.110695+0000","last_peered":"2024-04-21T08:36:26.110695+0000","last_clean":"2024-04-21T08:36:26.110695+0000","last_became_active":"2024-04-21T08:36:24.923542+0000","last_became_peered":"2024-04-21T08:36:24.923542+0000","last_unstale":"2024-04-21T08:36:26.110695+0000","last_undegraded":"2024-04-21T08:36:26.110695+0000","last_fullsized":"2024-04-21T08:36:26.110695+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:14:31.282802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111404+0000","last_change":"2024-04-21T08:36:20.288644+0000","last_active":"2024-04-21T08:36:26.111404+0000","last_peered":"2024-04-21T08:36:26.111404+0000","last_clean":"2024-04-21T08:36:26.111404+0000","last_became_active":"2024-04-21T08:36:20.288069+0000","last_became_peered":"2024-04-21T08:36:20.288069+0000","last_unstale":"2024-04-21T08:36:26.111404+0000","last_undegraded":"2024-04-21T08:36:26.111404+0000","last_fullsized":"2024-04-21T08:36:26.111404+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:37:36.522713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.112531+0000","last_change":"2024-04-21T08:36:20.908505+0000","last_active":"2024-04-21T08:36:26.112531+0000","last_peered":"2024-04-21T08:36:26.112531+0000","last_clean":"2024-04-21T08:36:26.112531+0000","last_became_active":"2024-04-21T08:36:20.908163+0000","last_became_peered":"2024-04-21T08:36:20.908163+0000","last_unstale":"2024-04-21T08:36:26.112531+0000","last_undegraded":"2024-04-21T08:36:26.112531+0000","last_fullsized":"2024-04-21T08:36:26.112531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:54:18.530203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.429083+0000","last_change":"2024-04-21T08:36:24.919428+0000","last_active":"2024-04-21T08:36:25.429083+0000","last_peered":"2024-04-21T08:36:25.429083+0000","last_clean":"2024-04-21T08:36:25.429083+0000","last_became_active":"2024-04-21T08:36:24.918722+0000","last_became_peered":"2024-04-21T08:36:24.918722+0000","last_unstale":"2024-04-21T08:36:25.429083+0000","last_undegraded":"2024-04-21T08:36:25.429083+0000","last_fullsized":"2024-04-21T08:36:25.429083+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:30:32.369551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"84'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.112273+0000","last_change":"2024-04-21T08:36:20.287642+0000","last_active":"2024-04-21T08:36:26.112273+0000","last_peered":"2024-04-21T08:36:26.112273+0000","last_clean":"2024-04-21T08:36:26.112273+0000","last_became_active":"2024-04-21T08:36:20.287296+0000","last_became_peered":"2024-04-21T08:36:20.287296+0000","last_unstale":"2024-04-21T08:36:26.112273+0000","last_undegraded":"2024-04-21T08:36:26.112273+0000","last_fullsized":"2024-04-21T08:36:26.112273+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:59:28.579020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111165+0000","last_change":"2024-04-21T08:36:20.916971+0000","last_active":"2024-04-21T08:36:26.111165+0000","last_peered":"2024-04-21T08:36:26.111165+0000","last_clean":"2024-04-21T08:36:26.111165+0000","last_became_active":"2024-04-21T08:36:20.916622+0000","last_became_peered":"2024-04-21T08:36:20.916622+0000","last_unstale":"2024-04-21T08:36:26.111165+0000","last_undegraded":"2024-04-21T08:36:26.111165+0000","last_fullsized":"2024-04-21T08:36:26.111165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:49:38.858772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.428697+0000","last_change":"2024-04-21T08:36:24.924170+0000","last_active":"2024-04-21T08:36:25.428697+0000","last_peered":"2024-04-21T08:36:25.428697+0000","last_clean":"2024-04-21T08:36:25.428697+0000","last_became_active":"2024-04-21T08:36:24.923744+0000","last_became_peered":"2024-04-21T08:36:24.923744+0000","last_unstale":"2024-04-21T08:36:25.428697+0000","last_undegraded":"2024-04-21T08:36:25.428697+0000","last_fullsized":"2024-04-21T08:36:25.428697+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:23:24.605021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111283+0000","last_change":"2024-04-21T08:36:20.286545+0000","last_active":"2024-04-21T08:36:26.111283+0000","last_peered":"2024-04-21T08:36:26.111283+0000","last_clean":"2024-04-21T08:36:26.111283+0000","last_became_active":"2024-04-21T08:36:20.286197+0000","last_became_peered":"2024-04-21T08:36:20.286197+0000","last_unstale":"2024-04-21T08:36:26.111283+0000","last_undegraded":"2024-04-21T08:36:26.111283+0000","last_fullsized":"2024-04-21T08:36:26.111283+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:27:55.430428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.112478+0000","last_change":"2024-04-21T08:36:20.907743+0000","last_active":"2024-04-21T08:36:26.112478+0000","last_peered":"2024-04-21T08:36:26.112478+0000","last_clean":"2024-04-21T08:36:26.112478+0000","last_became_active":"2024-04-21T08:36:20.907163+0000","last_became_peered":"2024-04-21T08:36:20.907163+0000","last_unstale":"2024-04-21T08:36:26.112478+0000","last_undegraded":"2024-04-21T08:36:26.112478+0000","last_fullsized":"2024-04-21T08:36:26.112478+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:41:00.172353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111965+0000","last_change":"2024-04-21T08:36:24.915823+0000","last_active":"2024-04-21T08:36:26.111965+0000","last_peered":"2024-04-21T08:36:26.111965+0000","last_clean":"2024-04-21T08:36:26.111965+0000","last_became_active":"2024-04-21T08:36:24.915392+0000","last_became_peered":"2024-04-21T08:36:24.915392+0000","last_unstale":"2024-04-21T08:36:26.111965+0000","last_undegraded":"2024-04-21T08:36:26.111965+0000","last_fullsized":"2024-04-21T08:36:26.111965+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:24:27.226163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110387+0000","last_change":"2024-04-21T08:36:20.283143+0000","last_active":"2024-04-21T08:36:26.110387+0000","last_peered":"2024-04-21T08:36:26.110387+0000","last_clean":"2024-04-21T08:36:26.110387+0000","last_became_active":"2024-04-21T08:36:20.282585+0000","last_became_peered":"2024-04-21T08:36:20.282585+0000","last_unstale":"2024-04-21T08:36:26.110387+0000","last_undegraded":"2024-04-21T08:36:26.110387+0000","last_fullsized":"2024-04-21T08:36:26.110387+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:12:37.588764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110504+0000","last_change":"2024-04-21T08:36:20.912591+0000","last_active":"2024-04-21T08:36:26.110504+0000","last_peered":"2024-04-21T08:36:26.110504+0000","last_clean":"2024-04-21T08:36:26.110504+0000","last_became_active":"2024-04-21T08:36:20.912174+0000","last_became_peered":"2024-04-21T08:36:20.912174+0000","last_unstale":"2024-04-21T08:36:26.110504+0000","last_undegraded":"2024-04-21T08:36:26.110504+0000","last_fullsized":"2024-04-21T08:36:26.110504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:33:58.949506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.348352+0000","last_change":"2024-04-21T08:36:24.915998+0000","last_active":"2024-04-21T08:36:26.348352+0000","last_peered":"2024-04-21T08:36:26.348352+0000","last_clean":"2024-04-21T08:36:26.348352+0000","last_became_active":"2024-04-21T08:36:24.915621+0000","last_became_peered":"2024-04-21T08:36:24.915621+0000","last_unstale":"2024-04-21T08:36:26.348352+0000","last_undegraded":"2024-04-21T08:36:26.348352+0000","last_fullsized":"2024-04-21T08:36:26.348352+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:43:05.117181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.112486+0000","last_change":"2024-04-21T08:36:20.286308+0000","last_active":"2024-04-21T08:36:26.112486+0000","last_peered":"2024-04-21T08:36:26.112486+0000","last_clean":"2024-04-21T08:36:26.112486+0000","last_became_active":"2024-04-21T08:36:20.285975+0000","last_became_peered":"2024-04-21T08:36:20.285975+0000","last_unstale":"2024-04-21T08:36:26.112486+0000","last_undegraded":"2024-04-21T08:36:26.112486+0000","last_fullsized":"2024-04-21T08:36:26.112486+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:55:10.658349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.427085+0000","last_change":"2024-04-21T08:36:20.900502+0000","last_active":"2024-04-21T08:36:25.427085+0000","last_peered":"2024-04-21T08:36:25.427085+0000","last_clean":"2024-04-21T08:36:25.427085+0000","last_became_active":"2024-04-21T08:36:20.900199+0000","last_became_peered":"2024-04-21T08:36:20.900199+0000","last_unstale":"2024-04-21T08:36:25.427085+0000","last_undegraded":"2024-04-21T08:36:25.427085+0000","last_fullsized":"2024-04-21T08:36:25.427085+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:17:00.720918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110709+0000","last_change":"2024-04-21T08:36:24.920991+0000","last_active":"2024-04-21T08:36:26.110709+0000","last_peered":"2024-04-21T08:36:26.110709+0000","last_clean":"2024-04-21T08:36:26.110709+0000","last_became_active":"2024-04-21T08:36:24.916611+0000","last_became_peered":"2024-04-21T08:36:24.916611+0000","last_unstale":"2024-04-21T08:36:26.110709+0000","last_undegraded":"2024-04-21T08:36:26.110709+0000","last_fullsized":"2024-04-21T08:36:26.110709+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:01:01.412127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:44.391195+0000","last_change":"2024-04-21T08:36:20.284966+0000","last_active":"2024-04-21T08:36:44.391195+0000","last_peered":"2024-04-21T08:36:44.391195+0000","last_clean":"2024-04-21T08:36:44.391195+0000","last_became_active":"2024-04-21T08:36:20.284612+0000","last_became_peered":"2024-04-21T08:36:20.284612+0000","last_unstale":"2024-04-21T08:36:44.391195+0000","last_undegraded":"2024-04-21T08:36:44.391195+0000","last_fullsized":"2024-04-21T08:36:44.391195+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:54:50.095013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.025508+0000","last_change":"2024-04-21T08:36:20.903493+0000","last_active":"2024-04-21T08:36:26.025508+0000","last_peered":"2024-04-21T08:36:26.025508+0000","last_clean":"2024-04-21T08:36:26.025508+0000","last_became_active":"2024-04-21T08:36:20.902752+0000","last_became_peered":"2024-04-21T08:36:20.902752+0000","last_unstale":"2024-04-21T08:36:26.025508+0000","last_undegraded":"2024-04-21T08:36:26.025508+0000","last_fullsized":"2024-04-21T08:36:26.025508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:40.684264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.427040+0000","last_change":"2024-04-21T08:36:24.920505+0000","last_active":"2024-04-21T08:36:25.427040+0000","last_peered":"2024-04-21T08:36:25.427040+0000","last_clean":"2024-04-21T08:36:25.427040+0000","last_became_active":"2024-04-21T08:36:24.919966+0000","last_became_peered":"2024-04-21T08:36:24.919966+0000","last_unstale":"2024-04-21T08:36:25.427040+0000","last_undegraded":"2024-04-21T08:36:25.427040+0000","last_fullsized":"2024-04-21T08:36:25.427040+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:58:05.131815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.428715+0000","last_change":"2024-04-21T08:36:20.281720+0000","last_active":"2024-04-21T08:36:25.428715+0000","last_peered":"2024-04-21T08:36:25.428715+0000","last_clean":"2024-04-21T08:36:25.428715+0000","last_became_active":"2024-04-21T08:36:20.279862+0000","last_became_peered":"2024-04-21T08:36:20.279862+0000","last_unstale":"2024-04-21T08:36:25.428715+0000","last_undegraded":"2024-04-21T08:36:25.428715+0000","last_fullsized":"2024-04-21T08:36:25.428715+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:18:58.106532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.348682+0000","last_change":"2024-04-21T08:36:20.908520+0000","last_active":"2024-04-21T08:36:26.348682+0000","last_peered":"2024-04-21T08:36:26.348682+0000","last_clean":"2024-04-21T08:36:26.348682+0000","last_became_active":"2024-04-21T08:36:20.908184+0000","last_became_peered":"2024-04-21T08:36:20.908184+0000","last_unstale":"2024-04-21T08:36:26.348682+0000","last_undegraded":"2024-04-21T08:36:26.348682+0000","last_fullsized":"2024-04-21T08:36:26.348682+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:14:33.895864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111335+0000","last_change":"2024-04-21T08:36:24.920697+0000","last_active":"2024-04-21T08:36:26.111335+0000","last_peered":"2024-04-21T08:36:26.111335+0000","last_clean":"2024-04-21T08:36:26.111335+0000","last_became_active":"2024-04-21T08:36:24.919976+0000","last_became_peered":"2024-04-21T08:36:24.919976+0000","last_unstale":"2024-04-21T08:36:26.111335+0000","last_undegraded":"2024-04-21T08:36:26.111335+0000","last_fullsized":"2024-04-21T08:36:26.111335+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:40:09.238887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"84'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.112393+0000","last_change":"2024-04-21T08:36:20.283531+0000","last_active":"2024-04-21T08:36:26.112393+0000","last_peered":"2024-04-21T08:36:26.112393+0000","last_clean":"2024-04-21T08:36:26.112393+0000","last_became_active":"2024-04-21T08:36:20.283159+0000","last_became_peered":"2024-04-21T08:36:20.283159+0000","last_unstale":"2024-04-21T08:36:26.112393+0000","last_undegraded":"2024-04-21T08:36:26.112393+0000","last_fullsized":"2024-04-21T08:36:26.112393+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:57:02.917537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.025352+0000","last_change":"2024-04-21T08:36:20.910113+0000","last_active":"2024-04-21T08:36:26.025352+0000","last_peered":"2024-04-21T08:36:26.025352+0000","last_clean":"2024-04-21T08:36:26.025352+0000","last_became_active":"2024-04-21T08:36:20.909744+0000","last_became_peered":"2024-04-21T08:36:20.909744+0000","last_unstale":"2024-04-21T08:36:26.025352+0000","last_undegraded":"2024-04-21T08:36:26.025352+0000","last_fullsized":"2024-04-21T08:36:26.025352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:16:14.568318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.427041+0000","last_change":"2024-04-21T08:36:24.922232+0000","last_active":"2024-04-21T08:36:25.427041+0000","last_peered":"2024-04-21T08:36:25.427041+0000","last_clean":"2024-04-21T08:36:25.427041+0000","last_became_active":"2024-04-21T08:36:24.921690+0000","last_became_peered":"2024-04-21T08:36:24.921690+0000","last_unstale":"2024-04-21T08:36:25.427041+0000","last_undegraded":"2024-04-21T08:36:25.427041+0000","last_fullsized":"2024-04-21T08:36:25.427041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:21:42.580203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'11","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:44.390799+0000","last_change":"2024-04-21T08:36:20.283574+0000","last_active":"2024-04-21T08:36:44.390799+0000","last_peered":"2024-04-21T08:36:44.390799+0000","last_clean":"2024-04-21T08:36:44.390799+0000","last_became_active":"2024-04-21T08:36:20.283220+0000","last_became_peered":"2024-04-21T08:36:20.283220+0000","last_unstale":"2024-04-21T08:36:44.390799+0000","last_undegraded":"2024-04-21T08:36:44.390799+0000","last_fullsized":"2024-04-21T08:36:44.390799+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:55:24.233237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111763+0000","last_change":"2024-04-21T08:36:20.919264+0000","last_active":"2024-04-21T08:36:26.111763+0000","last_peered":"2024-04-21T08:36:26.111763+0000","last_clean":"2024-04-21T08:36:26.111763+0000","last_became_active":"2024-04-21T08:36:20.918878+0000","last_became_peered":"2024-04-21T08:36:20.918878+0000","last_unstale":"2024-04-21T08:36:26.111763+0000","last_undegraded":"2024-04-21T08:36:26.111763+0000","last_fullsized":"2024-04-21T08:36:26.111763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:24:06.460758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.112790+0000","last_change":"2024-04-21T08:36:24.926878+0000","last_active":"2024-04-21T08:36:26.112790+0000","last_peered":"2024-04-21T08:36:26.112790+0000","last_clean":"2024-04-21T08:36:26.112790+0000","last_became_active":"2024-04-21T08:36:24.926431+0000","last_became_peered":"2024-04-21T08:36:24.926431+0000","last_unstale":"2024-04-21T08:36:26.112790+0000","last_undegraded":"2024-04-21T08:36:26.112790+0000","last_fullsized":"2024-04-21T08:36:26.112790+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:51:29.065962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:44.391809+0000","last_change":"2024-04-21T08:36:20.284841+0000","last_active":"2024-04-21T08:36:44.391809+0000","last_peered":"2024-04-21T08:36:44.391809+0000","last_clean":"2024-04-21T08:36:44.391809+0000","last_became_active":"2024-04-21T08:36:20.283888+0000","last_became_peered":"2024-04-21T08:36:20.283888+0000","last_unstale":"2024-04-21T08:36:44.391809+0000","last_undegraded":"2024-04-21T08:36:44.391809+0000","last_fullsized":"2024-04-21T08:36:44.391809+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:29:42.659608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110561+0000","last_change":"2024-04-21T08:36:20.905140+0000","last_active":"2024-04-21T08:36:26.110561+0000","last_peered":"2024-04-21T08:36:26.110561+0000","last_clean":"2024-04-21T08:36:26.110561+0000","last_became_active":"2024-04-21T08:36:20.904821+0000","last_became_peered":"2024-04-21T08:36:20.904821+0000","last_unstale":"2024-04-21T08:36:26.110561+0000","last_undegraded":"2024-04-21T08:36:26.110561+0000","last_fullsized":"2024-04-21T08:36:26.110561+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:09:11.962877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111735+0000","last_change":"2024-04-21T08:36:24.912062+0000","last_active":"2024-04-21T08:36:26.111735+0000","last_peered":"2024-04-21T08:36:26.111735+0000","last_clean":"2024-04-21T08:36:26.111735+0000","last_became_active":"2024-04-21T08:36:24.911600+0000","last_became_peered":"2024-04-21T08:36:24.911600+0000","last_unstale":"2024-04-21T08:36:26.111735+0000","last_undegraded":"2024-04-21T08:36:26.111735+0000","last_fullsized":"2024-04-21T08:36:26.111735+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:31:59.617161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111800+0000","last_change":"2024-04-21T08:36:20.290289+0000","last_active":"2024-04-21T08:36:26.111800+0000","last_peered":"2024-04-21T08:36:26.111800+0000","last_clean":"2024-04-21T08:36:26.111800+0000","last_became_active":"2024-04-21T08:36:20.289922+0000","last_became_peered":"2024-04-21T08:36:20.289922+0000","last_unstale":"2024-04-21T08:36:26.111800+0000","last_undegraded":"2024-04-21T08:36:26.111800+0000","last_fullsized":"2024-04-21T08:36:26.111800+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:10:00.926023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.429020+0000","last_change":"2024-04-21T08:36:20.906950+0000","last_active":"2024-04-21T08:36:25.429020+0000","last_peered":"2024-04-21T08:36:25.429020+0000","last_clean":"2024-04-21T08:36:25.429020+0000","last_became_active":"2024-04-21T08:36:20.906599+0000","last_became_peered":"2024-04-21T08:36:20.906599+0000","last_unstale":"2024-04-21T08:36:25.429020+0000","last_undegraded":"2024-04-21T08:36:25.429020+0000","last_fullsized":"2024-04-21T08:36:25.429020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:55:27.595997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.427452+0000","last_change":"2024-04-21T08:36:24.924349+0000","last_active":"2024-04-21T08:36:25.427452+0000","last_peered":"2024-04-21T08:36:25.427452+0000","last_clean":"2024-04-21T08:36:25.427452+0000","last_became_active":"2024-04-21T08:36:24.923982+0000","last_became_peered":"2024-04-21T08:36:24.923982+0000","last_unstale":"2024-04-21T08:36:25.427452+0000","last_undegraded":"2024-04-21T08:36:25.427452+0000","last_fullsized":"2024-04-21T08:36:25.427452+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:20:29.600759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110496+0000","last_change":"2024-04-21T08:36:20.285629+0000","last_active":"2024-04-21T08:36:26.110496+0000","last_peered":"2024-04-21T08:36:26.110496+0000","last_clean":"2024-04-21T08:36:26.110496+0000","last_became_active":"2024-04-21T08:36:20.285234+0000","last_became_peered":"2024-04-21T08:36:20.285234+0000","last_unstale":"2024-04-21T08:36:26.110496+0000","last_undegraded":"2024-04-21T08:36:26.110496+0000","last_fullsized":"2024-04-21T08:36:26.110496+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:19:32.966146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.428721+0000","last_change":"2024-04-21T08:36:20.906819+0000","last_active":"2024-04-21T08:36:25.428721+0000","last_peered":"2024-04-21T08:36:25.428721+0000","last_clean":"2024-04-21T08:36:25.428721+0000","last_became_active":"2024-04-21T08:36:20.906459+0000","last_became_peered":"2024-04-21T08:36:20.906459+0000","last_unstale":"2024-04-21T08:36:25.428721+0000","last_undegraded":"2024-04-21T08:36:25.428721+0000","last_fullsized":"2024-04-21T08:36:25.428721+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:12:39.312857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110833+0000","last_change":"2024-04-21T08:36:24.929452+0000","last_active":"2024-04-21T08:36:26.110833+0000","last_peered":"2024-04-21T08:36:26.110833+0000","last_clean":"2024-04-21T08:36:26.110833+0000","last_became_active":"2024-04-21T08:36:24.929039+0000","last_became_peered":"2024-04-21T08:36:24.929039+0000","last_unstale":"2024-04-21T08:36:26.110833+0000","last_undegraded":"2024-04-21T08:36:26.110833+0000","last_fullsized":"2024-04-21T08:36:26.110833+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:16:05.432024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"84'8","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110731+0000","last_change":"2024-04-21T08:36:20.293011+0000","last_active":"2024-04-21T08:36:26.110731+0000","last_peered":"2024-04-21T08:36:26.110731+0000","last_clean":"2024-04-21T08:36:26.110731+0000","last_became_active":"2024-04-21T08:36:20.291505+0000","last_became_peered":"2024-04-21T08:36:20.291505+0000","last_unstale":"2024-04-21T08:36:26.110731+0000","last_undegraded":"2024-04-21T08:36:26.110731+0000","last_fullsized":"2024-04-21T08:36:26.110731+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:33:22.665809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111789+0000","last_change":"2024-04-21T08:36:20.909076+0000","last_active":"2024-04-21T08:36:26.111789+0000","last_peered":"2024-04-21T08:36:26.111789+0000","last_clean":"2024-04-21T08:36:26.111789+0000","last_became_active":"2024-04-21T08:36:20.908739+0000","last_became_peered":"2024-04-21T08:36:20.908739+0000","last_unstale":"2024-04-21T08:36:26.111789+0000","last_undegraded":"2024-04-21T08:36:26.111789+0000","last_fullsized":"2024-04-21T08:36:26.111789+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:03:56.870477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111724+0000","last_change":"2024-04-21T08:36:24.926871+0000","last_active":"2024-04-21T08:36:26.111724+0000","last_peered":"2024-04-21T08:36:26.111724+0000","last_clean":"2024-04-21T08:36:26.111724+0000","last_became_active":"2024-04-21T08:36:24.926438+0000","last_became_peered":"2024-04-21T08:36:24.926438+0000","last_unstale":"2024-04-21T08:36:26.111724+0000","last_undegraded":"2024-04-21T08:36:26.111724+0000","last_fullsized":"2024-04-21T08:36:26.111724+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:24:48.984906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111468+0000","last_change":"2024-04-21T08:36:20.284686+0000","last_active":"2024-04-21T08:36:26.111468+0000","last_peered":"2024-04-21T08:36:26.111468+0000","last_clean":"2024-04-21T08:36:26.111468+0000","last_became_active":"2024-04-21T08:36:20.284366+0000","last_became_peered":"2024-04-21T08:36:20.284366+0000","last_unstale":"2024-04-21T08:36:26.111468+0000","last_undegraded":"2024-04-21T08:36:26.111468+0000","last_fullsized":"2024-04-21T08:36:26.111468+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:01:38.557724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.112120+0000","last_change":"2024-04-21T08:36:20.909532+0000","last_active":"2024-04-21T08:36:26.112120+0000","last_peered":"2024-04-21T08:36:26.112120+0000","last_clean":"2024-04-21T08:36:26.112120+0000","last_became_active":"2024-04-21T08:36:20.909013+0000","last_became_peered":"2024-04-21T08:36:20.909013+0000","last_unstale":"2024-04-21T08:36:26.112120+0000","last_undegraded":"2024-04-21T08:36:26.112120+0000","last_fullsized":"2024-04-21T08:36:26.112120+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:52:05.349237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111121+0000","last_change":"2024-04-21T08:36:24.929420+0000","last_active":"2024-04-21T08:36:26.111121+0000","last_peered":"2024-04-21T08:36:26.111121+0000","last_clean":"2024-04-21T08:36:26.111121+0000","last_became_active":"2024-04-21T08:36:24.928981+0000","last_became_peered":"2024-04-21T08:36:24.928981+0000","last_unstale":"2024-04-21T08:36:26.111121+0000","last_undegraded":"2024-04-21T08:36:26.111121+0000","last_fullsized":"2024-04-21T08:36:26.111121+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:38:00.040456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.427240+0000","last_change":"2024-04-21T08:36:20.279910+0000","last_active":"2024-04-21T08:36:25.427240+0000","last_peered":"2024-04-21T08:36:25.427240+0000","last_clean":"2024-04-21T08:36:25.427240+0000","last_became_active":"2024-04-21T08:36:20.278904+0000","last_became_peered":"2024-04-21T08:36:20.278904+0000","last_unstale":"2024-04-21T08:36:25.427240+0000","last_undegraded":"2024-04-21T08:36:25.427240+0000","last_fullsized":"2024-04-21T08:36:25.427240+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:40:19.209213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111602+0000","last_change":"2024-04-21T08:36:20.895816+0000","last_active":"2024-04-21T08:36:26.111602+0000","last_peered":"2024-04-21T08:36:26.111602+0000","last_clean":"2024-04-21T08:36:26.111602+0000","last_became_active":"2024-04-21T08:36:20.895487+0000","last_became_peered":"2024-04-21T08:36:20.895487+0000","last_unstale":"2024-04-21T08:36:26.111602+0000","last_undegraded":"2024-04-21T08:36:26.111602+0000","last_fullsized":"2024-04-21T08:36:26.111602+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:53:52.153683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111291+0000","last_change":"2024-04-21T08:36:24.917637+0000","last_active":"2024-04-21T08:36:26.111291+0000","last_peered":"2024-04-21T08:36:26.111291+0000","last_clean":"2024-04-21T08:36:26.111291+0000","last_became_active":"2024-04-21T08:36:24.917192+0000","last_became_peered":"2024-04-21T08:36:24.917192+0000","last_unstale":"2024-04-21T08:36:26.111291+0000","last_undegraded":"2024-04-21T08:36:26.111291+0000","last_fullsized":"2024-04-21T08:36:26.111291+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:02:59.843903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.428531+0000","last_change":"2024-04-21T08:36:20.286424+0000","last_active":"2024-04-21T08:36:25.428531+0000","last_peered":"2024-04-21T08:36:25.428531+0000","last_clean":"2024-04-21T08:36:25.428531+0000","last_became_active":"2024-04-21T08:36:20.285767+0000","last_became_peered":"2024-04-21T08:36:20.285767+0000","last_unstale":"2024-04-21T08:36:25.428531+0000","last_undegraded":"2024-04-21T08:36:25.428531+0000","last_fullsized":"2024-04-21T08:36:25.428531+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:53:33.202734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111388+0000","last_change":"2024-04-21T08:36:20.906785+0000","last_active":"2024-04-21T08:36:26.111388+0000","last_peered":"2024-04-21T08:36:26.111388+0000","last_clean":"2024-04-21T08:36:26.111388+0000","last_became_active":"2024-04-21T08:36:20.906485+0000","last_became_peered":"2024-04-21T08:36:20.906485+0000","last_unstale":"2024-04-21T08:36:26.111388+0000","last_undegraded":"2024-04-21T08:36:26.111388+0000","last_fullsized":"2024-04-21T08:36:26.111388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:54:34.664898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110796+0000","last_change":"2024-04-21T08:36:24.919669+0000","last_active":"2024-04-21T08:36:26.110796+0000","last_peered":"2024-04-21T08:36:26.110796+0000","last_clean":"2024-04-21T08:36:26.110796+0000","last_became_active":"2024-04-21T08:36:24.919281+0000","last_became_peered":"2024-04-21T08:36:24.919281+0000","last_unstale":"2024-04-21T08:36:26.110796+0000","last_undegraded":"2024-04-21T08:36:26.110796+0000","last_fullsized":"2024-04-21T08:36:26.110796+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:44:27.480367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.112602+0000","last_change":"2024-04-21T08:36:20.909272+0000","last_active":"2024-04-21T08:36:26.112602+0000","last_peered":"2024-04-21T08:36:26.112602+0000","last_clean":"2024-04-21T08:36:26.112602+0000","last_became_active":"2024-04-21T08:36:20.908928+0000","last_became_peered":"2024-04-21T08:36:20.908928+0000","last_unstale":"2024-04-21T08:36:26.112602+0000","last_undegraded":"2024-04-21T08:36:26.112602+0000","last_fullsized":"2024-04-21T08:36:26.112602+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:28:35.822293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110287+0000","last_change":"2024-04-21T08:36:20.291033+0000","last_active":"2024-04-21T08:36:26.110287+0000","last_peered":"2024-04-21T08:36:26.110287+0000","last_clean":"2024-04-21T08:36:26.110287+0000","last_became_active":"2024-04-21T08:36:20.290697+0000","last_became_peered":"2024-04-21T08:36:20.290697+0000","last_unstale":"2024-04-21T08:36:26.110287+0000","last_undegraded":"2024-04-21T08:36:26.110287+0000","last_fullsized":"2024-04-21T08:36:26.110287+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:22:54.593595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.428956+0000","last_change":"2024-04-21T08:36:24.917490+0000","last_active":"2024-04-21T08:36:25.428956+0000","last_peered":"2024-04-21T08:36:25.428956+0000","last_clean":"2024-04-21T08:36:25.428956+0000","last_became_active":"2024-04-21T08:36:24.917076+0000","last_became_peered":"2024-04-21T08:36:24.917076+0000","last_unstale":"2024-04-21T08:36:25.428956+0000","last_undegraded":"2024-04-21T08:36:25.428956+0000","last_fullsized":"2024-04-21T08:36:25.428956+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:20:43.085350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110425+0000","last_change":"2024-04-21T08:36:20.904197+0000","last_active":"2024-04-21T08:36:26.110425+0000","last_peered":"2024-04-21T08:36:26.110425+0000","last_clean":"2024-04-21T08:36:26.110425+0000","last_became_active":"2024-04-21T08:36:20.903099+0000","last_became_peered":"2024-04-21T08:36:20.903099+0000","last_unstale":"2024-04-21T08:36:26.110425+0000","last_undegraded":"2024-04-21T08:36:26.110425+0000","last_fullsized":"2024-04-21T08:36:26.110425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:16:08.914335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110801+0000","last_change":"2024-04-21T08:36:20.280608+0000","last_active":"2024-04-21T08:36:26.110801+0000","last_peered":"2024-04-21T08:36:26.110801+0000","last_clean":"2024-04-21T08:36:26.110801+0000","last_became_active":"2024-04-21T08:36:20.280172+0000","last_became_peered":"2024-04-21T08:36:20.280172+0000","last_unstale":"2024-04-21T08:36:26.110801+0000","last_undegraded":"2024-04-21T08:36:26.110801+0000","last_fullsized":"2024-04-21T08:36:26.110801+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:51:12.284463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":355,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.428352+0000","last_change":"2024-04-21T08:35:08.423545+0000","last_active":"2024-04-21T08:36:25.428352+0000","last_peered":"2024-04-21T08:36:25.428352+0000","last_clean":"2024-04-21T08:36:25.428352+0000","last_became_active":"2024-04-21T08:35:08.423027+0000","last_became_peered":"2024-04-21T08:35:08.423027+0000","last_unstale":"2024-04-21T08:36:25.428352+0000","last_undegraded":"2024-04-21T08:36:25.428352+0000","last_fullsized":"2024-04-21T08:36:25.428352+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:31:45.359253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:31:45.359253+0000","last_clean_scrub_stamp":"2024-04-21T08:31:45.359253+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:53:24.310824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.112115+0000","last_change":"2024-04-21T08:36:24.922422+0000","last_active":"2024-04-21T08:36:26.112115+0000","last_peered":"2024-04-21T08:36:26.112115+0000","last_clean":"2024-04-21T08:36:26.112115+0000","last_became_active":"2024-04-21T08:36:24.922006+0000","last_became_peered":"2024-04-21T08:36:24.922006+0000","last_unstale":"2024-04-21T08:36:26.112115+0000","last_undegraded":"2024-04-21T08:36:26.112115+0000","last_fullsized":"2024-04-21T08:36:26.112115+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:46:22.840802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110164+0000","last_change":"2024-04-21T08:36:20.911222+0000","last_active":"2024-04-21T08:36:26.110164+0000","last_peered":"2024-04-21T08:36:26.110164+0000","last_clean":"2024-04-21T08:36:26.110164+0000","last_became_active":"2024-04-21T08:36:20.910933+0000","last_became_peered":"2024-04-21T08:36:20.910933+0000","last_unstale":"2024-04-21T08:36:26.110164+0000","last_undegraded":"2024-04-21T08:36:26.110164+0000","last_fullsized":"2024-04-21T08:36:26.110164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:41:25.621266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":22,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.428632+0000","last_change":"2024-04-21T08:36:20.288409+0000","last_active":"2024-04-21T08:36:25.428632+0000","last_peered":"2024-04-21T08:36:25.428632+0000","last_clean":"2024-04-21T08:36:25.428632+0000","last_became_active":"2024-04-21T08:36:20.287916+0000","last_became_peered":"2024-04-21T08:36:20.287916+0000","last_unstale":"2024-04-21T08:36:25.428632+0000","last_undegraded":"2024-04-21T08:36:25.428632+0000","last_fullsized":"2024-04-21T08:36:25.428632+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:04:31.068352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111671+0000","last_change":"2024-04-21T08:36:24.918166+0000","last_active":"2024-04-21T08:36:26.111671+0000","last_peered":"2024-04-21T08:36:26.111671+0000","last_clean":"2024-04-21T08:36:26.111671+0000","last_became_active":"2024-04-21T08:36:24.917745+0000","last_became_peered":"2024-04-21T08:36:24.917745+0000","last_unstale":"2024-04-21T08:36:26.111671+0000","last_undegraded":"2024-04-21T08:36:26.111671+0000","last_fullsized":"2024-04-21T08:36:26.111671+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:55:41.169681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111866+0000","last_change":"2024-04-21T08:36:20.291079+0000","last_active":"2024-04-21T08:36:26.111866+0000","last_peered":"2024-04-21T08:36:26.111866+0000","last_clean":"2024-04-21T08:36:26.111866+0000","last_became_active":"2024-04-21T08:36:20.290720+0000","last_became_peered":"2024-04-21T08:36:20.290720+0000","last_unstale":"2024-04-21T08:36:26.111866+0000","last_undegraded":"2024-04-21T08:36:26.111866+0000","last_fullsized":"2024-04-21T08:36:26.111866+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:41.269029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111091+0000","last_change":"2024-04-21T08:36:20.907416+0000","last_active":"2024-04-21T08:36:26.111091+0000","last_peered":"2024-04-21T08:36:26.111091+0000","last_clean":"2024-04-21T08:36:26.111091+0000","last_became_active":"2024-04-21T08:36:20.907005+0000","last_became_peered":"2024-04-21T08:36:20.907005+0000","last_unstale":"2024-04-21T08:36:26.111091+0000","last_undegraded":"2024-04-21T08:36:26.111091+0000","last_fullsized":"2024-04-21T08:36:26.111091+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:58:17.979947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110396+0000","last_change":"2024-04-21T08:36:24.921475+0000","last_active":"2024-04-21T08:36:26.110396+0000","last_peered":"2024-04-21T08:36:26.110396+0000","last_clean":"2024-04-21T08:36:26.110396+0000","last_became_active":"2024-04-21T08:36:24.921091+0000","last_became_peered":"2024-04-21T08:36:24.921091+0000","last_unstale":"2024-04-21T08:36:26.110396+0000","last_undegraded":"2024-04-21T08:36:26.110396+0000","last_fullsized":"2024-04-21T08:36:26.110396+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:28:14.224501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.427518+0000","last_change":"2024-04-21T08:36:20.288916+0000","last_active":"2024-04-21T08:36:25.427518+0000","last_peered":"2024-04-21T08:36:25.427518+0000","last_clean":"2024-04-21T08:36:25.427518+0000","last_became_active":"2024-04-21T08:36:20.288575+0000","last_became_peered":"2024-04-21T08:36:20.288575+0000","last_unstale":"2024-04-21T08:36:25.427518+0000","last_undegraded":"2024-04-21T08:36:25.427518+0000","last_fullsized":"2024-04-21T08:36:25.427518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:52:05.795554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111360+0000","last_change":"2024-04-21T08:36:20.912053+0000","last_active":"2024-04-21T08:36:26.111360+0000","last_peered":"2024-04-21T08:36:26.111360+0000","last_clean":"2024-04-21T08:36:26.111360+0000","last_became_active":"2024-04-21T08:36:20.911584+0000","last_became_peered":"2024-04-21T08:36:20.911584+0000","last_unstale":"2024-04-21T08:36:26.111360+0000","last_undegraded":"2024-04-21T08:36:26.111360+0000","last_fullsized":"2024-04-21T08:36:26.111360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:48:36.369954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.428405+0000","last_change":"2024-04-21T08:36:24.919212+0000","last_active":"2024-04-21T08:36:25.428405+0000","last_peered":"2024-04-21T08:36:25.428405+0000","last_clean":"2024-04-21T08:36:25.428405+0000","last_became_active":"2024-04-21T08:36:24.918807+0000","last_became_peered":"2024-04-21T08:36:24.918807+0000","last_unstale":"2024-04-21T08:36:25.428405+0000","last_undegraded":"2024-04-21T08:36:25.428405+0000","last_fullsized":"2024-04-21T08:36:25.428405+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:25:50.816775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"84'2","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111113+0000","last_change":"2024-04-21T08:36:20.282536+0000","last_active":"2024-04-21T08:36:26.111113+0000","last_peered":"2024-04-21T08:36:26.111113+0000","last_clean":"2024-04-21T08:36:26.111113+0000","last_became_active":"2024-04-21T08:36:20.282172+0000","last_became_peered":"2024-04-21T08:36:20.282172+0000","last_unstale":"2024-04-21T08:36:26.111113+0000","last_undegraded":"2024-04-21T08:36:26.111113+0000","last_fullsized":"2024-04-21T08:36:26.111113+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:15:38.992441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.348351+0000","last_change":"2024-04-21T08:36:20.908416+0000","last_active":"2024-04-21T08:36:26.348351+0000","last_peered":"2024-04-21T08:36:26.348351+0000","last_clean":"2024-04-21T08:36:26.348351+0000","last_became_active":"2024-04-21T08:36:20.908054+0000","last_became_peered":"2024-04-21T08:36:20.908054+0000","last_unstale":"2024-04-21T08:36:26.348351+0000","last_undegraded":"2024-04-21T08:36:26.348351+0000","last_fullsized":"2024-04-21T08:36:26.348351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:23:16.562060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111261+0000","last_change":"2024-04-21T08:36:24.920637+0000","last_active":"2024-04-21T08:36:26.111261+0000","last_peered":"2024-04-21T08:36:26.111261+0000","last_clean":"2024-04-21T08:36:26.111261+0000","last_became_active":"2024-04-21T08:36:24.919921+0000","last_became_peered":"2024-04-21T08:36:24.919921+0000","last_unstale":"2024-04-21T08:36:26.111261+0000","last_undegraded":"2024-04-21T08:36:26.111261+0000","last_fullsized":"2024-04-21T08:36:26.111261+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:20:38.230087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:44.390346+0000","last_change":"2024-04-21T08:36:20.279332+0000","last_active":"2024-04-21T08:36:44.390346+0000","last_peered":"2024-04-21T08:36:44.390346+0000","last_clean":"2024-04-21T08:36:44.390346+0000","last_became_active":"2024-04-21T08:36:20.278970+0000","last_became_peered":"2024-04-21T08:36:20.278970+0000","last_unstale":"2024-04-21T08:36:44.390346+0000","last_undegraded":"2024-04-21T08:36:44.390346+0000","last_fullsized":"2024-04-21T08:36:44.390346+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:36:16.041610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110784+0000","last_change":"2024-04-21T08:36:20.919621+0000","last_active":"2024-04-21T08:36:26.110784+0000","last_peered":"2024-04-21T08:36:26.110784+0000","last_clean":"2024-04-21T08:36:26.110784+0000","last_became_active":"2024-04-21T08:36:20.919246+0000","last_became_peered":"2024-04-21T08:36:20.919246+0000","last_unstale":"2024-04-21T08:36:26.110784+0000","last_undegraded":"2024-04-21T08:36:26.110784+0000","last_fullsized":"2024-04-21T08:36:26.110784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:31:59.001501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.427567+0000","last_change":"2024-04-21T08:36:24.920459+0000","last_active":"2024-04-21T08:36:25.427567+0000","last_peered":"2024-04-21T08:36:25.427567+0000","last_clean":"2024-04-21T08:36:25.427567+0000","last_became_active":"2024-04-21T08:36:24.919691+0000","last_became_peered":"2024-04-21T08:36:24.919691+0000","last_unstale":"2024-04-21T08:36:25.427567+0000","last_undegraded":"2024-04-21T08:36:25.427567+0000","last_fullsized":"2024-04-21T08:36:25.427567+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:38:26.655452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:44.392193+0000","last_change":"2024-04-21T08:36:20.284286+0000","last_active":"2024-04-21T08:36:44.392193+0000","last_peered":"2024-04-21T08:36:44.392193+0000","last_clean":"2024-04-21T08:36:44.392193+0000","last_became_active":"2024-04-21T08:36:20.283903+0000","last_became_peered":"2024-04-21T08:36:20.283903+0000","last_unstale":"2024-04-21T08:36:44.392193+0000","last_undegraded":"2024-04-21T08:36:44.392193+0000","last_fullsized":"2024-04-21T08:36:44.392193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:51:52.409988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111041+0000","last_change":"2024-04-21T08:36:20.911892+0000","last_active":"2024-04-21T08:36:26.111041+0000","last_peered":"2024-04-21T08:36:26.111041+0000","last_clean":"2024-04-21T08:36:26.111041+0000","last_became_active":"2024-04-21T08:36:20.911513+0000","last_became_peered":"2024-04-21T08:36:20.911513+0000","last_unstale":"2024-04-21T08:36:26.111041+0000","last_undegraded":"2024-04-21T08:36:26.111041+0000","last_fullsized":"2024-04-21T08:36:26.111041+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:51:13.733534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111028+0000","last_change":"2024-04-21T08:36:24.920347+0000","last_active":"2024-04-21T08:36:26.111028+0000","last_peered":"2024-04-21T08:36:26.111028+0000","last_clean":"2024-04-21T08:36:26.111028+0000","last_became_active":"2024-04-21T08:36:24.919831+0000","last_became_peered":"2024-04-21T08:36:24.919831+0000","last_unstale":"2024-04-21T08:36:26.111028+0000","last_undegraded":"2024-04-21T08:36:26.111028+0000","last_fullsized":"2024-04-21T08:36:26.111028+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:14:27.020476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.428598+0000","last_change":"2024-04-21T08:36:20.286786+0000","last_active":"2024-04-21T08:36:25.428598+0000","last_peered":"2024-04-21T08:36:25.428598+0000","last_clean":"2024-04-21T08:36:25.428598+0000","last_became_active":"2024-04-21T08:36:20.286429+0000","last_became_peered":"2024-04-21T08:36:20.286429+0000","last_unstale":"2024-04-21T08:36:25.428598+0000","last_undegraded":"2024-04-21T08:36:25.428598+0000","last_fullsized":"2024-04-21T08:36:25.428598+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:40:57.093370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111061+0000","last_change":"2024-04-21T08:36:20.904767+0000","last_active":"2024-04-21T08:36:26.111061+0000","last_peered":"2024-04-21T08:36:26.111061+0000","last_clean":"2024-04-21T08:36:26.111061+0000","last_became_active":"2024-04-21T08:36:20.904406+0000","last_became_peered":"2024-04-21T08:36:20.904406+0000","last_unstale":"2024-04-21T08:36:26.111061+0000","last_undegraded":"2024-04-21T08:36:26.111061+0000","last_fullsized":"2024-04-21T08:36:26.111061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:11:41.644549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111147+0000","last_change":"2024-04-21T08:36:24.918031+0000","last_active":"2024-04-21T08:36:26.111147+0000","last_peered":"2024-04-21T08:36:26.111147+0000","last_clean":"2024-04-21T08:36:26.111147+0000","last_became_active":"2024-04-21T08:36:24.917618+0000","last_became_peered":"2024-04-21T08:36:24.917618+0000","last_unstale":"2024-04-21T08:36:26.111147+0000","last_undegraded":"2024-04-21T08:36:26.111147+0000","last_fullsized":"2024-04-21T08:36:26.111147+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:50:35.797911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110932+0000","last_change":"2024-04-21T08:36:20.286212+0000","last_active":"2024-04-21T08:36:26.110932+0000","last_peered":"2024-04-21T08:36:26.110932+0000","last_clean":"2024-04-21T08:36:26.110932+0000","last_became_active":"2024-04-21T08:36:20.285878+0000","last_became_peered":"2024-04-21T08:36:20.285878+0000","last_unstale":"2024-04-21T08:36:26.110932+0000","last_undegraded":"2024-04-21T08:36:26.110932+0000","last_fullsized":"2024-04-21T08:36:26.110932+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:19.375455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.428818+0000","last_change":"2024-04-21T08:36:20.908335+0000","last_active":"2024-04-21T08:36:25.428818+0000","last_peered":"2024-04-21T08:36:25.428818+0000","last_clean":"2024-04-21T08:36:25.428818+0000","last_became_active":"2024-04-21T08:36:20.907945+0000","last_became_peered":"2024-04-21T08:36:20.907945+0000","last_unstale":"2024-04-21T08:36:25.428818+0000","last_undegraded":"2024-04-21T08:36:25.428818+0000","last_fullsized":"2024-04-21T08:36:25.428818+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:47:17.841511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.025123+0000","last_change":"2024-04-21T08:36:24.915553+0000","last_active":"2024-04-21T08:36:26.025123+0000","last_peered":"2024-04-21T08:36:26.025123+0000","last_clean":"2024-04-21T08:36:26.025123+0000","last_became_active":"2024-04-21T08:36:24.915028+0000","last_became_peered":"2024-04-21T08:36:24.915028+0000","last_unstale":"2024-04-21T08:36:26.025123+0000","last_undegraded":"2024-04-21T08:36:26.025123+0000","last_fullsized":"2024-04-21T08:36:26.025123+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:17:21.192946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.428927+0000","last_change":"2024-04-21T08:36:20.279649+0000","last_active":"2024-04-21T08:36:25.428927+0000","last_peered":"2024-04-21T08:36:25.428927+0000","last_clean":"2024-04-21T08:36:25.428927+0000","last_became_active":"2024-04-21T08:36:20.279111+0000","last_became_peered":"2024-04-21T08:36:20.279111+0000","last_unstale":"2024-04-21T08:36:25.428927+0000","last_undegraded":"2024-04-21T08:36:25.428927+0000","last_fullsized":"2024-04-21T08:36:25.428927+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:19:57.380495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.429094+0000","last_change":"2024-04-21T08:36:20.911489+0000","last_active":"2024-04-21T08:36:25.429094+0000","last_peered":"2024-04-21T08:36:25.429094+0000","last_clean":"2024-04-21T08:36:25.429094+0000","last_became_active":"2024-04-21T08:36:20.911168+0000","last_became_peered":"2024-04-21T08:36:20.911168+0000","last_unstale":"2024-04-21T08:36:25.429094+0000","last_undegraded":"2024-04-21T08:36:25.429094+0000","last_fullsized":"2024-04-21T08:36:25.429094+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:40:41.585841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.348262+0000","last_change":"2024-04-21T08:36:24.921175+0000","last_active":"2024-04-21T08:36:26.348262+0000","last_peered":"2024-04-21T08:36:26.348262+0000","last_clean":"2024-04-21T08:36:26.348262+0000","last_became_active":"2024-04-21T08:36:24.920802+0000","last_became_peered":"2024-04-21T08:36:24.920802+0000","last_unstale":"2024-04-21T08:36:26.348262+0000","last_undegraded":"2024-04-21T08:36:26.348262+0000","last_fullsized":"2024-04-21T08:36:26.348262+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:43:12.140016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"84'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.112541+0000","last_change":"2024-04-21T08:36:20.290202+0000","last_active":"2024-04-21T08:36:26.112541+0000","last_peered":"2024-04-21T08:36:26.112541+0000","last_clean":"2024-04-21T08:36:26.112541+0000","last_became_active":"2024-04-21T08:36:20.289868+0000","last_became_peered":"2024-04-21T08:36:20.289868+0000","last_unstale":"2024-04-21T08:36:26.112541+0000","last_undegraded":"2024-04-21T08:36:26.112541+0000","last_fullsized":"2024-04-21T08:36:26.112541+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:07:45.525980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.428608+0000","last_change":"2024-04-21T08:36:20.899590+0000","last_active":"2024-04-21T08:36:25.428608+0000","last_peered":"2024-04-21T08:36:25.428608+0000","last_clean":"2024-04-21T08:36:25.428608+0000","last_became_active":"2024-04-21T08:36:20.899254+0000","last_became_peered":"2024-04-21T08:36:20.899254+0000","last_unstale":"2024-04-21T08:36:25.428608+0000","last_undegraded":"2024-04-21T08:36:25.428608+0000","last_fullsized":"2024-04-21T08:36:25.428608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:34:25.409694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111016+0000","last_change":"2024-04-21T08:36:24.918310+0000","last_active":"2024-04-21T08:36:26.111016+0000","last_peered":"2024-04-21T08:36:26.111016+0000","last_clean":"2024-04-21T08:36:26.111016+0000","last_became_active":"2024-04-21T08:36:24.917899+0000","last_became_peered":"2024-04-21T08:36:24.917899+0000","last_unstale":"2024-04-21T08:36:26.111016+0000","last_undegraded":"2024-04-21T08:36:26.111016+0000","last_fullsized":"2024-04-21T08:36:26.111016+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:09:13.006884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"84'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111023+0000","last_change":"2024-04-21T08:36:20.289465+0000","last_active":"2024-04-21T08:36:26.111023+0000","last_peered":"2024-04-21T08:36:26.111023+0000","last_clean":"2024-04-21T08:36:26.111023+0000","last_became_active":"2024-04-21T08:36:20.289190+0000","last_became_peered":"2024-04-21T08:36:20.289190+0000","last_unstale":"2024-04-21T08:36:26.111023+0000","last_undegraded":"2024-04-21T08:36:26.111023+0000","last_fullsized":"2024-04-21T08:36:26.111023+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:01.267400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.429006+0000","last_change":"2024-04-21T08:36:20.912368+0000","last_active":"2024-04-21T08:36:25.429006+0000","last_peered":"2024-04-21T08:36:25.429006+0000","last_clean":"2024-04-21T08:36:25.429006+0000","last_became_active":"2024-04-21T08:36:20.912009+0000","last_became_peered":"2024-04-21T08:36:20.912009+0000","last_unstale":"2024-04-21T08:36:25.429006+0000","last_undegraded":"2024-04-21T08:36:25.429006+0000","last_fullsized":"2024-04-21T08:36:25.429006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:35:40.700314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111165+0000","last_change":"2024-04-21T08:36:20.913472+0000","last_active":"2024-04-21T08:36:26.111165+0000","last_peered":"2024-04-21T08:36:26.111165+0000","last_clean":"2024-04-21T08:36:26.111165+0000","last_became_active":"2024-04-21T08:36:20.913118+0000","last_became_peered":"2024-04-21T08:36:20.913118+0000","last_unstale":"2024-04-21T08:36:26.111165+0000","last_undegraded":"2024-04-21T08:36:26.111165+0000","last_fullsized":"2024-04-21T08:36:26.111165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:31:53.957044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.428667+0000","last_change":"2024-04-21T08:36:20.913677+0000","last_active":"2024-04-21T08:36:25.428667+0000","last_peered":"2024-04-21T08:36:25.428667+0000","last_clean":"2024-04-21T08:36:25.428667+0000","last_became_active":"2024-04-21T08:36:20.913097+0000","last_became_peered":"2024-04-21T08:36:20.913097+0000","last_unstale":"2024-04-21T08:36:25.428667+0000","last_undegraded":"2024-04-21T08:36:25.428667+0000","last_fullsized":"2024-04-21T08:36:25.428667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:08:40.388370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.428415+0000","last_change":"2024-04-21T08:36:20.906624+0000","last_active":"2024-04-21T08:36:25.428415+0000","last_peered":"2024-04-21T08:36:25.428415+0000","last_clean":"2024-04-21T08:36:25.428415+0000","last_became_active":"2024-04-21T08:36:20.906247+0000","last_became_peered":"2024-04-21T08:36:20.906247+0000","last_unstale":"2024-04-21T08:36:25.428415+0000","last_undegraded":"2024-04-21T08:36:25.428415+0000","last_fullsized":"2024-04-21T08:36:25.428415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:44:11.855183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.348364+0000","last_change":"2024-04-21T08:36:20.906576+0000","last_active":"2024-04-21T08:36:26.348364+0000","last_peered":"2024-04-21T08:36:26.348364+0000","last_clean":"2024-04-21T08:36:26.348364+0000","last_became_active":"2024-04-21T08:36:20.906231+0000","last_became_peered":"2024-04-21T08:36:20.906231+0000","last_unstale":"2024-04-21T08:36:26.348364+0000","last_undegraded":"2024-04-21T08:36:26.348364+0000","last_fullsized":"2024-04-21T08:36:26.348364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:42:48.156614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110877+0000","last_change":"2024-04-21T08:36:20.902551+0000","last_active":"2024-04-21T08:36:26.110877+0000","last_peered":"2024-04-21T08:36:26.110877+0000","last_clean":"2024-04-21T08:36:26.110877+0000","last_became_active":"2024-04-21T08:36:20.902208+0000","last_became_peered":"2024-04-21T08:36:20.902208+0000","last_unstale":"2024-04-21T08:36:26.110877+0000","last_undegraded":"2024-04-21T08:36:26.110877+0000","last_fullsized":"2024-04-21T08:36:26.110877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:36:30.016256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111890+0000","last_change":"2024-04-21T08:36:20.918588+0000","last_active":"2024-04-21T08:36:26.111890+0000","last_peered":"2024-04-21T08:36:26.111890+0000","last_clean":"2024-04-21T08:36:26.111890+0000","last_became_active":"2024-04-21T08:36:20.918263+0000","last_became_peered":"2024-04-21T08:36:20.918263+0000","last_unstale":"2024-04-21T08:36:26.111890+0000","last_undegraded":"2024-04-21T08:36:26.111890+0000","last_fullsized":"2024-04-21T08:36:26.111890+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:51:30.340336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.429140+0000","last_change":"2024-04-21T08:36:20.911063+0000","last_active":"2024-04-21T08:36:25.429140+0000","last_peered":"2024-04-21T08:36:25.429140+0000","last_clean":"2024-04-21T08:36:25.429140+0000","last_became_active":"2024-04-21T08:36:20.910647+0000","last_became_peered":"2024-04-21T08:36:20.910647+0000","last_unstale":"2024-04-21T08:36:25.429140+0000","last_undegraded":"2024-04-21T08:36:25.429140+0000","last_fullsized":"2024-04-21T08:36:25.429140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:10:31.079782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111443+0000","last_change":"2024-04-21T08:36:20.914079+0000","last_active":"2024-04-21T08:36:26.111443+0000","last_peered":"2024-04-21T08:36:26.111443+0000","last_clean":"2024-04-21T08:36:26.111443+0000","last_became_active":"2024-04-21T08:36:20.912851+0000","last_became_peered":"2024-04-21T08:36:20.912851+0000","last_unstale":"2024-04-21T08:36:26.111443+0000","last_undegraded":"2024-04-21T08:36:26.111443+0000","last_fullsized":"2024-04-21T08:36:26.111443+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:53:35.811281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.112496+0000","last_change":"2024-04-21T08:36:20.900551+0000","last_active":"2024-04-21T08:36:26.112496+0000","last_peered":"2024-04-21T08:36:26.112496+0000","last_clean":"2024-04-21T08:36:26.112496+0000","last_became_active":"2024-04-21T08:36:20.900173+0000","last_became_peered":"2024-04-21T08:36:20.900173+0000","last_unstale":"2024-04-21T08:36:26.112496+0000","last_undegraded":"2024-04-21T08:36:26.112496+0000","last_fullsized":"2024-04-21T08:36:26.112496+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:21.389498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110424+0000","last_change":"2024-04-21T08:36:20.912953+0000","last_active":"2024-04-21T08:36:26.110424+0000","last_peered":"2024-04-21T08:36:26.110424+0000","last_clean":"2024-04-21T08:36:26.110424+0000","last_became_active":"2024-04-21T08:36:20.912223+0000","last_became_peered":"2024-04-21T08:36:20.912223+0000","last_unstale":"2024-04-21T08:36:26.110424+0000","last_undegraded":"2024-04-21T08:36:26.110424+0000","last_fullsized":"2024-04-21T08:36:26.110424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:43:14.277179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.025233+0000","last_change":"2024-04-21T08:36:20.906164+0000","last_active":"2024-04-21T08:36:26.025233+0000","last_peered":"2024-04-21T08:36:26.025233+0000","last_clean":"2024-04-21T08:36:26.025233+0000","last_became_active":"2024-04-21T08:36:20.905749+0000","last_became_peered":"2024-04-21T08:36:20.905749+0000","last_unstale":"2024-04-21T08:36:26.025233+0000","last_undegraded":"2024-04-21T08:36:26.025233+0000","last_fullsized":"2024-04-21T08:36:26.025233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:02:10.569087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110554+0000","last_change":"2024-04-21T08:36:20.900314+0000","last_active":"2024-04-21T08:36:26.110554+0000","last_peered":"2024-04-21T08:36:26.110554+0000","last_clean":"2024-04-21T08:36:26.110554+0000","last_became_active":"2024-04-21T08:36:20.900009+0000","last_became_peered":"2024-04-21T08:36:20.900009+0000","last_unstale":"2024-04-21T08:36:26.110554+0000","last_undegraded":"2024-04-21T08:36:26.110554+0000","last_fullsized":"2024-04-21T08:36:26.110554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:19:59.997302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111776+0000","last_change":"2024-04-21T08:36:20.920725+0000","last_active":"2024-04-21T08:36:26.111776+0000","last_peered":"2024-04-21T08:36:26.111776+0000","last_clean":"2024-04-21T08:36:26.111776+0000","last_became_active":"2024-04-21T08:36:20.920288+0000","last_became_peered":"2024-04-21T08:36:20.920288+0000","last_unstale":"2024-04-21T08:36:26.111776+0000","last_undegraded":"2024-04-21T08:36:26.111776+0000","last_fullsized":"2024-04-21T08:36:26.111776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:36:18.736388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":4820,"num_objects":29,"num_object_clones":0,"num_object_copies":87,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":29,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":65,"num_write_kb":20,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":19,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":139264,"data_stored":16178,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383698,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":983040,"data_stored":477215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645334,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28592,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716560,"statfs":{"total":95995035648,"available":95965757440,"internally_reserved":0,"allocated":1613824,"data_stored":1072381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808860,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":477999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070496,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":991232,"data_stored":477819,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332134,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28568,"kb_used_data":1544,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716584,"statfs":{"total":95995035648,"available":95965782016,"internally_reserved":0,"allocated":1581056,"data_stored":1068165,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495659,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27988,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717164,"statfs":{"total":95995035648,"available":95966375936,"internally_reserved":0,"allocated":983040,"data_stored":477215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":481639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738440,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27984,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717168,"statfs":{"total":95995035648,"available":95966380032,"internally_reserved":0,"allocated":987136,"data_stored":477305,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1007616,"data_stored":481713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182457,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":477319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920821,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":964,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":987136,"data_stored":477237,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757295,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32732,"kb_used_data":1552,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712420,"statfs":{"total":95995035648,"available":95961518080,"internally_reserved":0,"allocated":1589248,"data_stored":1067765,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":90,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":22,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":182,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":582,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":604,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T08:36:49.167 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-21T08:36:49.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.300+0000 7feb50055640 1 -- 172.21.15.86:0/3694257189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb48102980 msgr2=0x7feb481092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:49.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.301+0000 7feb50055640 1 --2- 172.21.15.86:0/3694257189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb48102980 0x7feb481092a0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7feb40009ca0 tx=0x7feb4002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:36:49.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.301+0000 7feb50055640 1 -- 172.21.15.86:0/3694257189 shutdown_connections 2024-04-21T08:36:49.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.301+0000 7feb50055640 1 --2- 172.21.15.86:0/3694257189 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb48102980 0x7feb481092a0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:49.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.301+0000 7feb50055640 1 --2- 172.21.15.86:0/3694257189 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb48101fc0 0x7feb48102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:49.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.301+0000 7feb50055640 1 --2- 172.21.15.86:0/3694257189 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb48100e70 0x7feb48101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:49.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.301+0000 7feb50055640 1 -- 172.21.15.86:0/3694257189 >> 172.21.15.86:0/3694257189 conn(0x7feb480fc3e0 msgr2=0x7feb480fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:49.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.301+0000 7feb50055640 1 -- 172.21.15.86:0/3694257189 shutdown_connections 2024-04-21T08:36:49.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.301+0000 7feb50055640 1 -- 172.21.15.86:0/3694257189 wait complete. 2024-04-21T08:36:49.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.301+0000 7feb50055640 1 Processor -- start 2024-04-21T08:36:49.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.301+0000 7feb50055640 1 -- start start 2024-04-21T08:36:49.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.301+0000 7feb50055640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb48100e70 0x7feb48198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:49.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.301+0000 7feb50055640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb48101fc0 0x7feb481986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:49.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.301+0000 7feb50055640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb48102980 0x7feb4819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:49.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.301+0000 7feb50055640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7feb4810db30 con 0x7feb48102980 2024-04-21T08:36:49.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.301+0000 7feb50055640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7feb4810d9b0 con 0x7feb48101fc0 2024-04-21T08:36:49.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.301+0000 7feb50055640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7feb4810dcb0 con 0x7feb48100e70 2024-04-21T08:36:49.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.301+0000 7feb4e5f2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb48102980 0x7feb4819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:49.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.301+0000 7feb4e5f2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb48102980 0x7feb4819f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35292/0 (socket says 172.21.15.86:35292) 2024-04-21T08:36:49.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.301+0000 7feb4e5f2640 1 -- 172.21.15.86:0/2216269326 learned_addr learned my addr 172.21.15.86:0/2216269326 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:49.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.302+0000 7feb4e5f2640 1 -- 172.21.15.86:0/2216269326 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb48100e70 msgr2=0x7feb48198170 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T08:36:49.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.302+0000 7feb4e5f2640 1 --2- 172.21.15.86:0/2216269326 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb48100e70 0x7feb48198170 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:49.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.302+0000 7feb4e5f2640 1 -- 172.21.15.86:0/2216269326 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb48101fc0 msgr2=0x7feb481986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:49.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.302+0000 7feb4e5f2640 1 --2- 172.21.15.86:0/2216269326 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb48101fc0 0x7feb481986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:49.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.302+0000 7feb4e5f2640 1 -- 172.21.15.86:0/2216269326 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7feb4819fd90 con 0x7feb48102980 2024-04-21T08:36:49.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.302+0000 7feb4e5f2640 1 --2- 172.21.15.86:0/2216269326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb48102980 0x7feb4819f720 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7feb4002c700 tx=0x7feb40034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:49.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.302+0000 7feb3effd640 1 -- 172.21.15.86:0/2216269326 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb40039040 con 0x7feb48102980 2024-04-21T08:36:49.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.302+0000 7feb3effd640 1 -- 172.21.15.86:0/2216269326 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feb40044070 con 0x7feb48102980 2024-04-21T08:36:49.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.302+0000 7feb50055640 1 -- 172.21.15.86:0/2216269326 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7feb481a0020 con 0x7feb48102980 2024-04-21T08:36:49.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.302+0000 7feb3effd640 1 -- 172.21.15.86:0/2216269326 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feb400042a0 con 0x7feb48102980 2024-04-21T08:36:49.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.302+0000 7feb50055640 1 -- 172.21.15.86:0/2216269326 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7feb481a0560 con 0x7feb48102980 2024-04-21T08:36:49.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.303+0000 7feb50055640 1 -- 172.21.15.86:0/2216269326 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7feb48108180 con 0x7feb48102980 2024-04-21T08:36:49.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.304+0000 7feb3effd640 1 -- 172.21.15.86:0/2216269326 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7feb40003970 con 0x7feb48102980 2024-04-21T08:36:49.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.305+0000 7feb3effd640 1 --2- 172.21.15.86:0/2216269326 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7feb340a3cc0 0x7feb340a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:49.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.305+0000 7feb4ddf1640 1 --2- 172.21.15.86:0/2216269326 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7feb340a3cc0 0x7feb340a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:49.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.305+0000 7feb3effd640 1 -- 172.21.15.86:0/2216269326 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7feb400f2b30 con 0x7feb48102980 2024-04-21T08:36:49.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.305+0000 7feb4ddf1640 1 --2- 172.21.15.86:0/2216269326 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7feb340a3cc0 0x7feb340a6180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7feb440096b0 tx=0x7feb44006c20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:49.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.309+0000 7feb3effd640 1 -- 172.21.15.86:0/2216269326 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feb400bf280 con 0x7feb48102980 2024-04-21T08:36:49.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.486+0000 7feb50055640 1 -- 172.21.15.86:0/2216269326 --> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7feb48062390 con 0x7feb340a3cc0 2024-04-21T08:36:49.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.496+0000 7feb3effd640 1 -- 172.21.15.86:0/2216269326 <== mgr.14152 v2:172.21.15.86:6800/1364156965 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340354 (secure 0 0 0) 0x7feb48062390 con 0x7feb340a3cc0 2024-04-21T08:36:49.497 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:36:49.501 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-21T08:36:49.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.499+0000 7feb50055640 1 -- 172.21.15.86:0/2216269326 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7feb340a3cc0 msgr2=0x7feb340a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:49.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.499+0000 7feb50055640 1 --2- 172.21.15.86:0/2216269326 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7feb340a3cc0 0x7feb340a6180 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7feb440096b0 tx=0x7feb44006c20 comp rx=0 tx=0).stop 2024-04-21T08:36:49.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.499+0000 7feb50055640 1 -- 172.21.15.86:0/2216269326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb48102980 msgr2=0x7feb4819f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:49.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.499+0000 7feb50055640 1 --2- 172.21.15.86:0/2216269326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb48102980 0x7feb4819f720 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7feb4002c700 tx=0x7feb40034760 comp rx=0 tx=0).stop 2024-04-21T08:36:49.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.499+0000 7feb50055640 1 -- 172.21.15.86:0/2216269326 shutdown_connections 2024-04-21T08:36:49.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.499+0000 7feb50055640 1 --2- 172.21.15.86:0/2216269326 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7feb340a3cc0 0x7feb340a6180 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:49.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.499+0000 7feb50055640 1 --2- 172.21.15.86:0/2216269326 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7feb48102980 0x7feb4819f720 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:49.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.499+0000 7feb50055640 1 --2- 172.21.15.86:0/2216269326 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7feb48101fc0 0x7feb481986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:49.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.499+0000 7feb50055640 1 --2- 172.21.15.86:0/2216269326 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7feb48100e70 0x7feb48198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:49.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.499+0000 7feb50055640 1 -- 172.21.15.86:0/2216269326 >> 172.21.15.86:0/2216269326 conn(0x7feb480fc3e0 msgr2=0x7feb48105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:49.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.499+0000 7feb50055640 1 -- 172.21.15.86:0/2216269326 shutdown_connections 2024-04-21T08:36:49.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:49.499+0000 7feb50055640 1 -- 172.21.15.86:0/2216269326 wait complete. 2024-04-21T08:36:49.509 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":302,"stamp":"2024-04-21T08:36:49.275670+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":600264,"num_objects":79,"num_object_clones":0,"num_object_copies":239,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":79,"num_whiteouts":0,"num_read":167,"num_read_kb":142,"num_write":340,"num_write_kb":3003,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":279,"ondisk_log_size":279,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":371052,"kb_used_data":13560,"kb_used_omap":18,"kb_used_meta":357357,"kb_avail":1124570772,"statfs":{"total":1151940427776,"available":1151560470528,"internally_reserved":0,"allocated":13885440,"data_stored":7541693,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":365933884},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":3,"apply_latency_ms":3,"commit_latency_ns":3000000,"apply_latency_ns":3000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":5397,"num_objects":55,"num_object_clones":0,"num_object_copies":167,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":55,"num_whiteouts":0,"num_read":31,"num_read_kb":23,"num_write":112,"num_write_kb":48,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":34,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003384"},"pg_stats":[{"pgid":"3.3f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.112095+0000","last_change":"2024-04-21T08:36:20.913421+0000","last_active":"2024-04-21T08:36:26.112095+0000","last_peered":"2024-04-21T08:36:26.112095+0000","last_clean":"2024-04-21T08:36:26.112095+0000","last_became_active":"2024-04-21T08:36:20.913017+0000","last_became_peered":"2024-04-21T08:36:20.913017+0000","last_unstale":"2024-04-21T08:36:26.112095+0000","last_undegraded":"2024-04-21T08:36:26.112095+0000","last_fullsized":"2024-04-21T08:36:26.112095+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:16:29.885916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.348243+0000","last_change":"2024-04-21T08:36:20.911675+0000","last_active":"2024-04-21T08:36:26.348243+0000","last_peered":"2024-04-21T08:36:26.348243+0000","last_clean":"2024-04-21T08:36:26.348243+0000","last_became_active":"2024-04-21T08:36:20.910591+0000","last_became_peered":"2024-04-21T08:36:20.910591+0000","last_unstale":"2024-04-21T08:36:26.348243+0000","last_undegraded":"2024-04-21T08:36:26.348243+0000","last_fullsized":"2024-04-21T08:36:26.348243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:01:40.735376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.348481+0000","last_change":"2024-04-21T08:36:20.911978+0000","last_active":"2024-04-21T08:36:26.348481+0000","last_peered":"2024-04-21T08:36:26.348481+0000","last_clean":"2024-04-21T08:36:26.348481+0000","last_became_active":"2024-04-21T08:36:20.911641+0000","last_became_peered":"2024-04-21T08:36:20.911641+0000","last_unstale":"2024-04-21T08:36:26.348481+0000","last_undegraded":"2024-04-21T08:36:26.348481+0000","last_fullsized":"2024-04-21T08:36:26.348481+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:14:23.789790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111254+0000","last_change":"2024-04-21T08:36:20.919027+0000","last_active":"2024-04-21T08:36:26.111254+0000","last_peered":"2024-04-21T08:36:26.111254+0000","last_clean":"2024-04-21T08:36:26.111254+0000","last_became_active":"2024-04-21T08:36:20.918740+0000","last_became_peered":"2024-04-21T08:36:20.918740+0000","last_unstale":"2024-04-21T08:36:26.111254+0000","last_undegraded":"2024-04-21T08:36:26.111254+0000","last_fullsized":"2024-04-21T08:36:26.111254+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:00:31.066938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110759+0000","last_change":"2024-04-21T08:36:20.907175+0000","last_active":"2024-04-21T08:36:26.110759+0000","last_peered":"2024-04-21T08:36:26.110759+0000","last_clean":"2024-04-21T08:36:26.110759+0000","last_became_active":"2024-04-21T08:36:20.906818+0000","last_became_peered":"2024-04-21T08:36:20.906818+0000","last_unstale":"2024-04-21T08:36:26.110759+0000","last_undegraded":"2024-04-21T08:36:26.110759+0000","last_fullsized":"2024-04-21T08:36:26.110759+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:41:03.332231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111378+0000","last_change":"2024-04-21T08:36:20.905968+0000","last_active":"2024-04-21T08:36:26.111378+0000","last_peered":"2024-04-21T08:36:26.111378+0000","last_clean":"2024-04-21T08:36:26.111378+0000","last_became_active":"2024-04-21T08:36:20.904840+0000","last_became_peered":"2024-04-21T08:36:20.904840+0000","last_unstale":"2024-04-21T08:36:26.111378+0000","last_undegraded":"2024-04-21T08:36:26.111378+0000","last_fullsized":"2024-04-21T08:36:26.111378+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:23:46.886241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.428415+0000","last_change":"2024-04-21T08:36:20.912045+0000","last_active":"2024-04-21T08:36:25.428415+0000","last_peered":"2024-04-21T08:36:25.428415+0000","last_clean":"2024-04-21T08:36:25.428415+0000","last_became_active":"2024-04-21T08:36:20.911690+0000","last_became_peered":"2024-04-21T08:36:20.911690+0000","last_unstale":"2024-04-21T08:36:25.428415+0000","last_undegraded":"2024-04-21T08:36:25.428415+0000","last_fullsized":"2024-04-21T08:36:25.428415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:06:28.229307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110591+0000","last_change":"2024-04-21T08:36:20.904874+0000","last_active":"2024-04-21T08:36:26.110591+0000","last_peered":"2024-04-21T08:36:26.110591+0000","last_clean":"2024-04-21T08:36:26.110591+0000","last_became_active":"2024-04-21T08:36:20.904217+0000","last_became_peered":"2024-04-21T08:36:20.904217+0000","last_unstale":"2024-04-21T08:36:26.110591+0000","last_undegraded":"2024-04-21T08:36:26.110591+0000","last_fullsized":"2024-04-21T08:36:26.110591+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:37:04.423812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.025560+0000","last_change":"2024-04-21T08:36:20.912874+0000","last_active":"2024-04-21T08:36:26.025560+0000","last_peered":"2024-04-21T08:36:26.025560+0000","last_clean":"2024-04-21T08:36:26.025560+0000","last_became_active":"2024-04-21T08:36:20.912569+0000","last_became_peered":"2024-04-21T08:36:20.912569+0000","last_unstale":"2024-04-21T08:36:26.025560+0000","last_undegraded":"2024-04-21T08:36:26.025560+0000","last_fullsized":"2024-04-21T08:36:26.025560+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:55:20.763157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111846+0000","last_change":"2024-04-21T08:36:20.911125+0000","last_active":"2024-04-21T08:36:26.111846+0000","last_peered":"2024-04-21T08:36:26.111846+0000","last_clean":"2024-04-21T08:36:26.111846+0000","last_became_active":"2024-04-21T08:36:20.910614+0000","last_became_peered":"2024-04-21T08:36:20.910614+0000","last_unstale":"2024-04-21T08:36:26.111846+0000","last_undegraded":"2024-04-21T08:36:26.111846+0000","last_fullsized":"2024-04-21T08:36:26.111846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:20:16.448824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,4],"acting":[10,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.428995+0000","last_change":"2024-04-21T08:36:20.909094+0000","last_active":"2024-04-21T08:36:25.428995+0000","last_peered":"2024-04-21T08:36:25.428995+0000","last_clean":"2024-04-21T08:36:25.428995+0000","last_became_active":"2024-04-21T08:36:20.908634+0000","last_became_peered":"2024-04-21T08:36:20.908634+0000","last_unstale":"2024-04-21T08:36:25.428995+0000","last_undegraded":"2024-04-21T08:36:25.428995+0000","last_fullsized":"2024-04-21T08:36:25.428995+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:28:38.450621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110958+0000","last_change":"2024-04-21T08:36:20.909429+0000","last_active":"2024-04-21T08:36:26.110958+0000","last_peered":"2024-04-21T08:36:26.110958+0000","last_clean":"2024-04-21T08:36:26.110958+0000","last_became_active":"2024-04-21T08:36:20.909081+0000","last_became_peered":"2024-04-21T08:36:20.909081+0000","last_unstale":"2024-04-21T08:36:26.110958+0000","last_undegraded":"2024-04-21T08:36:26.110958+0000","last_fullsized":"2024-04-21T08:36:26.110958+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:45:46.939125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110964+0000","last_change":"2024-04-21T08:36:20.895894+0000","last_active":"2024-04-21T08:36:26.110964+0000","last_peered":"2024-04-21T08:36:26.110964+0000","last_clean":"2024-04-21T08:36:26.110964+0000","last_became_active":"2024-04-21T08:36:20.895572+0000","last_became_peered":"2024-04-21T08:36:20.895572+0000","last_unstale":"2024-04-21T08:36:26.110964+0000","last_undegraded":"2024-04-21T08:36:26.110964+0000","last_fullsized":"2024-04-21T08:36:26.110964+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:12:48.750555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.348127+0000","last_change":"2024-04-21T08:36:20.905356+0000","last_active":"2024-04-21T08:36:26.348127+0000","last_peered":"2024-04-21T08:36:26.348127+0000","last_clean":"2024-04-21T08:36:26.348127+0000","last_became_active":"2024-04-21T08:36:20.904954+0000","last_became_peered":"2024-04-21T08:36:20.904954+0000","last_unstale":"2024-04-21T08:36:26.348127+0000","last_undegraded":"2024-04-21T08:36:26.348127+0000","last_fullsized":"2024-04-21T08:36:26.348127+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:45:49.116482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.348172+0000","last_change":"2024-04-21T08:36:20.911550+0000","last_active":"2024-04-21T08:36:26.348172+0000","last_peered":"2024-04-21T08:36:26.348172+0000","last_clean":"2024-04-21T08:36:26.348172+0000","last_became_active":"2024-04-21T08:36:20.910557+0000","last_became_peered":"2024-04-21T08:36:20.910557+0000","last_unstale":"2024-04-21T08:36:26.348172+0000","last_undegraded":"2024-04-21T08:36:26.348172+0000","last_fullsized":"2024-04-21T08:36:26.348172+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:09:49.422729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.024946+0000","last_change":"2024-04-21T08:36:20.903954+0000","last_active":"2024-04-21T08:36:26.024946+0000","last_peered":"2024-04-21T08:36:26.024946+0000","last_clean":"2024-04-21T08:36:26.024946+0000","last_became_active":"2024-04-21T08:36:20.903641+0000","last_became_peered":"2024-04-21T08:36:20.903641+0000","last_unstale":"2024-04-21T08:36:26.024946+0000","last_undegraded":"2024-04-21T08:36:26.024946+0000","last_fullsized":"2024-04-21T08:36:26.024946+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:08:55.678940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.112308+0000","last_change":"2024-04-21T08:36:20.914256+0000","last_active":"2024-04-21T08:36:26.112308+0000","last_peered":"2024-04-21T08:36:26.112308+0000","last_clean":"2024-04-21T08:36:26.112308+0000","last_became_active":"2024-04-21T08:36:20.913858+0000","last_became_peered":"2024-04-21T08:36:20.913858+0000","last_unstale":"2024-04-21T08:36:26.112308+0000","last_undegraded":"2024-04-21T08:36:26.112308+0000","last_fullsized":"2024-04-21T08:36:26.112308+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:01:10.276763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.427500+0000","last_change":"2024-04-21T08:36:20.904138+0000","last_active":"2024-04-21T08:36:25.427500+0000","last_peered":"2024-04-21T08:36:25.427500+0000","last_clean":"2024-04-21T08:36:25.427500+0000","last_became_active":"2024-04-21T08:36:20.903670+0000","last_became_peered":"2024-04-21T08:36:20.903670+0000","last_unstale":"2024-04-21T08:36:25.427500+0000","last_undegraded":"2024-04-21T08:36:25.427500+0000","last_fullsized":"2024-04-21T08:36:25.427500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:38:50.160867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.429013+0000","last_change":"2024-04-21T08:36:20.910798+0000","last_active":"2024-04-21T08:36:25.429013+0000","last_peered":"2024-04-21T08:36:25.429013+0000","last_clean":"2024-04-21T08:36:25.429013+0000","last_became_active":"2024-04-21T08:36:20.910411+0000","last_became_peered":"2024-04-21T08:36:20.910411+0000","last_unstale":"2024-04-21T08:36:25.429013+0000","last_undegraded":"2024-04-21T08:36:25.429013+0000","last_fullsized":"2024-04-21T08:36:25.429013+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:13:12.109818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.428444+0000","last_change":"2024-04-21T08:36:24.924831+0000","last_active":"2024-04-21T08:36:25.428444+0000","last_peered":"2024-04-21T08:36:25.428444+0000","last_clean":"2024-04-21T08:36:25.428444+0000","last_became_active":"2024-04-21T08:36:24.924435+0000","last_became_peered":"2024-04-21T08:36:24.924435+0000","last_unstale":"2024-04-21T08:36:25.428444+0000","last_undegraded":"2024-04-21T08:36:25.428444+0000","last_fullsized":"2024-04-21T08:36:25.428444+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:23.447079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"84'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.427560+0000","last_change":"2024-04-21T08:36:20.286270+0000","last_active":"2024-04-21T08:36:25.427560+0000","last_peered":"2024-04-21T08:36:25.427560+0000","last_clean":"2024-04-21T08:36:25.427560+0000","last_became_active":"2024-04-21T08:36:20.285959+0000","last_became_peered":"2024-04-21T08:36:20.285959+0000","last_unstale":"2024-04-21T08:36:25.427560+0000","last_undegraded":"2024-04-21T08:36:25.427560+0000","last_fullsized":"2024-04-21T08:36:25.427560+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:15:49.116420+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,8],"acting":[9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111318+0000","last_change":"2024-04-21T08:36:20.912851+0000","last_active":"2024-04-21T08:36:26.111318+0000","last_peered":"2024-04-21T08:36:26.111318+0000","last_clean":"2024-04-21T08:36:26.111318+0000","last_became_active":"2024-04-21T08:36:20.912407+0000","last_became_peered":"2024-04-21T08:36:20.912407+0000","last_unstale":"2024-04-21T08:36:26.111318+0000","last_undegraded":"2024-04-21T08:36:26.111318+0000","last_fullsized":"2024-04-21T08:36:26.111318+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:45:18.026940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110547+0000","last_change":"2024-04-21T08:36:24.924052+0000","last_active":"2024-04-21T08:36:26.110547+0000","last_peered":"2024-04-21T08:36:26.110547+0000","last_clean":"2024-04-21T08:36:26.110547+0000","last_became_active":"2024-04-21T08:36:24.923618+0000","last_became_peered":"2024-04-21T08:36:24.923618+0000","last_unstale":"2024-04-21T08:36:26.110547+0000","last_undegraded":"2024-04-21T08:36:26.110547+0000","last_fullsized":"2024-04-21T08:36:26.110547+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:08:18.182386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"84'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111336+0000","last_change":"2024-04-21T08:36:20.285291+0000","last_active":"2024-04-21T08:36:26.111336+0000","last_peered":"2024-04-21T08:36:26.111336+0000","last_clean":"2024-04-21T08:36:26.111336+0000","last_became_active":"2024-04-21T08:36:20.284969+0000","last_became_peered":"2024-04-21T08:36:20.284969+0000","last_unstale":"2024-04-21T08:36:26.111336+0000","last_undegraded":"2024-04-21T08:36:26.111336+0000","last_fullsized":"2024-04-21T08:36:26.111336+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:55:29.484439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.348589+0000","last_change":"2024-04-21T08:36:20.902819+0000","last_active":"2024-04-21T08:36:26.348589+0000","last_peered":"2024-04-21T08:36:26.348589+0000","last_clean":"2024-04-21T08:36:26.348589+0000","last_became_active":"2024-04-21T08:36:20.902466+0000","last_became_peered":"2024-04-21T08:36:20.902466+0000","last_unstale":"2024-04-21T08:36:26.348589+0000","last_undegraded":"2024-04-21T08:36:26.348589+0000","last_fullsized":"2024-04-21T08:36:26.348589+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:29:40.454026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110899+0000","last_change":"2024-04-21T08:36:24.926480+0000","last_active":"2024-04-21T08:36:26.110899+0000","last_peered":"2024-04-21T08:36:26.110899+0000","last_clean":"2024-04-21T08:36:26.110899+0000","last_became_active":"2024-04-21T08:36:24.926116+0000","last_became_peered":"2024-04-21T08:36:24.926116+0000","last_unstale":"2024-04-21T08:36:26.110899+0000","last_undegraded":"2024-04-21T08:36:26.110899+0000","last_fullsized":"2024-04-21T08:36:26.110899+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:43:18.839396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'11","reported_seq":34,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:46.454485+0000","last_change":"2024-04-21T08:36:20.293086+0000","last_active":"2024-04-21T08:36:46.454485+0000","last_peered":"2024-04-21T08:36:46.454485+0000","last_clean":"2024-04-21T08:36:46.454485+0000","last_became_active":"2024-04-21T08:36:20.291178+0000","last_became_peered":"2024-04-21T08:36:20.291178+0000","last_unstale":"2024-04-21T08:36:46.454485+0000","last_undegraded":"2024-04-21T08:36:46.454485+0000","last_fullsized":"2024-04-21T08:36:46.454485+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:02:32.935930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":15,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.025226+0000","last_change":"2024-04-21T08:36:20.909374+0000","last_active":"2024-04-21T08:36:26.025226+0000","last_peered":"2024-04-21T08:36:26.025226+0000","last_clean":"2024-04-21T08:36:26.025226+0000","last_became_active":"2024-04-21T08:36:20.909043+0000","last_became_peered":"2024-04-21T08:36:20.909043+0000","last_unstale":"2024-04-21T08:36:26.025226+0000","last_undegraded":"2024-04-21T08:36:26.025226+0000","last_fullsized":"2024-04-21T08:36:26.025226+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:45:45.719942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110695+0000","last_change":"2024-04-21T08:36:24.923989+0000","last_active":"2024-04-21T08:36:26.110695+0000","last_peered":"2024-04-21T08:36:26.110695+0000","last_clean":"2024-04-21T08:36:26.110695+0000","last_became_active":"2024-04-21T08:36:24.923542+0000","last_became_peered":"2024-04-21T08:36:24.923542+0000","last_unstale":"2024-04-21T08:36:26.110695+0000","last_undegraded":"2024-04-21T08:36:26.110695+0000","last_fullsized":"2024-04-21T08:36:26.110695+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:14:31.282802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:46.453045+0000","last_change":"2024-04-21T08:36:20.288644+0000","last_active":"2024-04-21T08:36:46.453045+0000","last_peered":"2024-04-21T08:36:46.453045+0000","last_clean":"2024-04-21T08:36:46.453045+0000","last_became_active":"2024-04-21T08:36:20.288069+0000","last_became_peered":"2024-04-21T08:36:20.288069+0000","last_unstale":"2024-04-21T08:36:46.453045+0000","last_undegraded":"2024-04-21T08:36:46.453045+0000","last_fullsized":"2024-04-21T08:36:46.453045+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:37:36.522713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.112531+0000","last_change":"2024-04-21T08:36:20.908505+0000","last_active":"2024-04-21T08:36:26.112531+0000","last_peered":"2024-04-21T08:36:26.112531+0000","last_clean":"2024-04-21T08:36:26.112531+0000","last_became_active":"2024-04-21T08:36:20.908163+0000","last_became_peered":"2024-04-21T08:36:20.908163+0000","last_unstale":"2024-04-21T08:36:26.112531+0000","last_undegraded":"2024-04-21T08:36:26.112531+0000","last_fullsized":"2024-04-21T08:36:26.112531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:54:18.530203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.429083+0000","last_change":"2024-04-21T08:36:24.919428+0000","last_active":"2024-04-21T08:36:25.429083+0000","last_peered":"2024-04-21T08:36:25.429083+0000","last_clean":"2024-04-21T08:36:25.429083+0000","last_became_active":"2024-04-21T08:36:24.918722+0000","last_became_peered":"2024-04-21T08:36:24.918722+0000","last_unstale":"2024-04-21T08:36:25.429083+0000","last_undegraded":"2024-04-21T08:36:25.429083+0000","last_fullsized":"2024-04-21T08:36:25.429083+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:30:32.369551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"89'6","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:46.441268+0000","last_change":"2024-04-21T08:36:20.287642+0000","last_active":"2024-04-21T08:36:46.441268+0000","last_peered":"2024-04-21T08:36:46.441268+0000","last_clean":"2024-04-21T08:36:46.441268+0000","last_became_active":"2024-04-21T08:36:20.287296+0000","last_became_peered":"2024-04-21T08:36:20.287296+0000","last_unstale":"2024-04-21T08:36:46.441268+0000","last_undegraded":"2024-04-21T08:36:46.441268+0000","last_fullsized":"2024-04-21T08:36:46.441268+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:59:28.579020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1013,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":4,"num_read_kb":2,"num_write":3,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111165+0000","last_change":"2024-04-21T08:36:20.916971+0000","last_active":"2024-04-21T08:36:26.111165+0000","last_peered":"2024-04-21T08:36:26.111165+0000","last_clean":"2024-04-21T08:36:26.111165+0000","last_became_active":"2024-04-21T08:36:20.916622+0000","last_became_peered":"2024-04-21T08:36:20.916622+0000","last_unstale":"2024-04-21T08:36:26.111165+0000","last_undegraded":"2024-04-21T08:36:26.111165+0000","last_fullsized":"2024-04-21T08:36:26.111165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:49:38.858772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.428697+0000","last_change":"2024-04-21T08:36:24.924170+0000","last_active":"2024-04-21T08:36:25.428697+0000","last_peered":"2024-04-21T08:36:25.428697+0000","last_clean":"2024-04-21T08:36:25.428697+0000","last_became_active":"2024-04-21T08:36:24.923744+0000","last_became_peered":"2024-04-21T08:36:24.923744+0000","last_unstale":"2024-04-21T08:36:25.428697+0000","last_undegraded":"2024-04-21T08:36:25.428697+0000","last_fullsized":"2024-04-21T08:36:25.428697+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:23:24.605021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"84'1","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111283+0000","last_change":"2024-04-21T08:36:20.286545+0000","last_active":"2024-04-21T08:36:26.111283+0000","last_peered":"2024-04-21T08:36:26.111283+0000","last_clean":"2024-04-21T08:36:26.111283+0000","last_became_active":"2024-04-21T08:36:20.286197+0000","last_became_peered":"2024-04-21T08:36:20.286197+0000","last_unstale":"2024-04-21T08:36:26.111283+0000","last_undegraded":"2024-04-21T08:36:26.111283+0000","last_fullsized":"2024-04-21T08:36:26.111283+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:27:55.430428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.112478+0000","last_change":"2024-04-21T08:36:20.907743+0000","last_active":"2024-04-21T08:36:26.112478+0000","last_peered":"2024-04-21T08:36:26.112478+0000","last_clean":"2024-04-21T08:36:26.112478+0000","last_became_active":"2024-04-21T08:36:20.907163+0000","last_became_peered":"2024-04-21T08:36:20.907163+0000","last_unstale":"2024-04-21T08:36:26.112478+0000","last_undegraded":"2024-04-21T08:36:26.112478+0000","last_fullsized":"2024-04-21T08:36:26.112478+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:41:00.172353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111965+0000","last_change":"2024-04-21T08:36:24.915823+0000","last_active":"2024-04-21T08:36:26.111965+0000","last_peered":"2024-04-21T08:36:26.111965+0000","last_clean":"2024-04-21T08:36:26.111965+0000","last_became_active":"2024-04-21T08:36:24.915392+0000","last_became_peered":"2024-04-21T08:36:24.915392+0000","last_unstale":"2024-04-21T08:36:26.111965+0000","last_undegraded":"2024-04-21T08:36:26.111965+0000","last_fullsized":"2024-04-21T08:36:26.111965+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:24:27.226163+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,9,0],"acting":[11,4,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:45.402094+0000","last_change":"2024-04-21T08:36:20.283143+0000","last_active":"2024-04-21T08:36:45.402094+0000","last_peered":"2024-04-21T08:36:45.402094+0000","last_clean":"2024-04-21T08:36:45.402094+0000","last_became_active":"2024-04-21T08:36:20.282585+0000","last_became_peered":"2024-04-21T08:36:20.282585+0000","last_unstale":"2024-04-21T08:36:45.402094+0000","last_undegraded":"2024-04-21T08:36:45.402094+0000","last_fullsized":"2024-04-21T08:36:45.402094+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:12:37.588764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110504+0000","last_change":"2024-04-21T08:36:20.912591+0000","last_active":"2024-04-21T08:36:26.110504+0000","last_peered":"2024-04-21T08:36:26.110504+0000","last_clean":"2024-04-21T08:36:26.110504+0000","last_became_active":"2024-04-21T08:36:20.912174+0000","last_became_peered":"2024-04-21T08:36:20.912174+0000","last_unstale":"2024-04-21T08:36:26.110504+0000","last_undegraded":"2024-04-21T08:36:26.110504+0000","last_fullsized":"2024-04-21T08:36:26.110504+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:33:58.949506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.348352+0000","last_change":"2024-04-21T08:36:24.915998+0000","last_active":"2024-04-21T08:36:26.348352+0000","last_peered":"2024-04-21T08:36:26.348352+0000","last_clean":"2024-04-21T08:36:26.348352+0000","last_became_active":"2024-04-21T08:36:24.915621+0000","last_became_peered":"2024-04-21T08:36:24.915621+0000","last_unstale":"2024-04-21T08:36:26.348352+0000","last_undegraded":"2024-04-21T08:36:26.348352+0000","last_fullsized":"2024-04-21T08:36:26.348352+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:43:05.117181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:45.446923+0000","last_change":"2024-04-21T08:36:20.286308+0000","last_active":"2024-04-21T08:36:45.446923+0000","last_peered":"2024-04-21T08:36:45.446923+0000","last_clean":"2024-04-21T08:36:45.446923+0000","last_became_active":"2024-04-21T08:36:20.285975+0000","last_became_peered":"2024-04-21T08:36:20.285975+0000","last_unstale":"2024-04-21T08:36:45.446923+0000","last_undegraded":"2024-04-21T08:36:45.446923+0000","last_fullsized":"2024-04-21T08:36:45.446923+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:55:10.658349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.427085+0000","last_change":"2024-04-21T08:36:20.900502+0000","last_active":"2024-04-21T08:36:25.427085+0000","last_peered":"2024-04-21T08:36:25.427085+0000","last_clean":"2024-04-21T08:36:25.427085+0000","last_became_active":"2024-04-21T08:36:20.900199+0000","last_became_peered":"2024-04-21T08:36:20.900199+0000","last_unstale":"2024-04-21T08:36:25.427085+0000","last_undegraded":"2024-04-21T08:36:25.427085+0000","last_fullsized":"2024-04-21T08:36:25.427085+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:17:00.720918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110709+0000","last_change":"2024-04-21T08:36:24.920991+0000","last_active":"2024-04-21T08:36:26.110709+0000","last_peered":"2024-04-21T08:36:26.110709+0000","last_clean":"2024-04-21T08:36:26.110709+0000","last_became_active":"2024-04-21T08:36:24.916611+0000","last_became_peered":"2024-04-21T08:36:24.916611+0000","last_unstale":"2024-04-21T08:36:26.110709+0000","last_undegraded":"2024-04-21T08:36:26.110709+0000","last_fullsized":"2024-04-21T08:36:26.110709+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:01:01.412127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"89'3","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:44.391195+0000","last_change":"2024-04-21T08:36:20.284966+0000","last_active":"2024-04-21T08:36:44.391195+0000","last_peered":"2024-04-21T08:36:44.391195+0000","last_clean":"2024-04-21T08:36:44.391195+0000","last_became_active":"2024-04-21T08:36:20.284612+0000","last_became_peered":"2024-04-21T08:36:20.284612+0000","last_unstale":"2024-04-21T08:36:44.391195+0000","last_undegraded":"2024-04-21T08:36:44.391195+0000","last_fullsized":"2024-04-21T08:36:44.391195+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:54:50.095013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.025508+0000","last_change":"2024-04-21T08:36:20.903493+0000","last_active":"2024-04-21T08:36:26.025508+0000","last_peered":"2024-04-21T08:36:26.025508+0000","last_clean":"2024-04-21T08:36:26.025508+0000","last_became_active":"2024-04-21T08:36:20.902752+0000","last_became_peered":"2024-04-21T08:36:20.902752+0000","last_unstale":"2024-04-21T08:36:26.025508+0000","last_undegraded":"2024-04-21T08:36:26.025508+0000","last_fullsized":"2024-04-21T08:36:26.025508+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:40.684264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.427040+0000","last_change":"2024-04-21T08:36:24.920505+0000","last_active":"2024-04-21T08:36:25.427040+0000","last_peered":"2024-04-21T08:36:25.427040+0000","last_clean":"2024-04-21T08:36:25.427040+0000","last_became_active":"2024-04-21T08:36:24.919966+0000","last_became_peered":"2024-04-21T08:36:24.919966+0000","last_unstale":"2024-04-21T08:36:25.427040+0000","last_undegraded":"2024-04-21T08:36:25.427040+0000","last_fullsized":"2024-04-21T08:36:25.427040+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:58:05.131815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:46.451604+0000","last_change":"2024-04-21T08:36:20.281720+0000","last_active":"2024-04-21T08:36:46.451604+0000","last_peered":"2024-04-21T08:36:46.451604+0000","last_clean":"2024-04-21T08:36:46.451604+0000","last_became_active":"2024-04-21T08:36:20.279862+0000","last_became_peered":"2024-04-21T08:36:20.279862+0000","last_unstale":"2024-04-21T08:36:46.451604+0000","last_undegraded":"2024-04-21T08:36:46.451604+0000","last_fullsized":"2024-04-21T08:36:46.451604+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:18:58.106532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.348682+0000","last_change":"2024-04-21T08:36:20.908520+0000","last_active":"2024-04-21T08:36:26.348682+0000","last_peered":"2024-04-21T08:36:26.348682+0000","last_clean":"2024-04-21T08:36:26.348682+0000","last_became_active":"2024-04-21T08:36:20.908184+0000","last_became_peered":"2024-04-21T08:36:20.908184+0000","last_unstale":"2024-04-21T08:36:26.348682+0000","last_undegraded":"2024-04-21T08:36:26.348682+0000","last_fullsized":"2024-04-21T08:36:26.348682+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:14:33.895864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111335+0000","last_change":"2024-04-21T08:36:24.920697+0000","last_active":"2024-04-21T08:36:26.111335+0000","last_peered":"2024-04-21T08:36:26.111335+0000","last_clean":"2024-04-21T08:36:26.111335+0000","last_became_active":"2024-04-21T08:36:24.919976+0000","last_became_peered":"2024-04-21T08:36:24.919976+0000","last_unstale":"2024-04-21T08:36:26.111335+0000","last_undegraded":"2024-04-21T08:36:26.111335+0000","last_fullsized":"2024-04-21T08:36:26.111335+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:40:09.238887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:45.439216+0000","last_change":"2024-04-21T08:36:20.283531+0000","last_active":"2024-04-21T08:36:45.439216+0000","last_peered":"2024-04-21T08:36:45.439216+0000","last_clean":"2024-04-21T08:36:45.439216+0000","last_became_active":"2024-04-21T08:36:20.283159+0000","last_became_peered":"2024-04-21T08:36:20.283159+0000","last_unstale":"2024-04-21T08:36:45.439216+0000","last_undegraded":"2024-04-21T08:36:45.439216+0000","last_fullsized":"2024-04-21T08:36:45.439216+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:57:02.917537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.025352+0000","last_change":"2024-04-21T08:36:20.910113+0000","last_active":"2024-04-21T08:36:26.025352+0000","last_peered":"2024-04-21T08:36:26.025352+0000","last_clean":"2024-04-21T08:36:26.025352+0000","last_became_active":"2024-04-21T08:36:20.909744+0000","last_became_peered":"2024-04-21T08:36:20.909744+0000","last_unstale":"2024-04-21T08:36:26.025352+0000","last_undegraded":"2024-04-21T08:36:26.025352+0000","last_fullsized":"2024-04-21T08:36:26.025352+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:16:14.568318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.427041+0000","last_change":"2024-04-21T08:36:24.922232+0000","last_active":"2024-04-21T08:36:25.427041+0000","last_peered":"2024-04-21T08:36:25.427041+0000","last_clean":"2024-04-21T08:36:25.427041+0000","last_became_active":"2024-04-21T08:36:24.921690+0000","last_became_peered":"2024-04-21T08:36:24.921690+0000","last_unstale":"2024-04-21T08:36:25.427041+0000","last_undegraded":"2024-04-21T08:36:25.427041+0000","last_fullsized":"2024-04-21T08:36:25.427041+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:21:42.580203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"89'11","reported_seq":33,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:44.390799+0000","last_change":"2024-04-21T08:36:20.283574+0000","last_active":"2024-04-21T08:36:44.390799+0000","last_peered":"2024-04-21T08:36:44.390799+0000","last_clean":"2024-04-21T08:36:44.390799+0000","last_became_active":"2024-04-21T08:36:20.283220+0000","last_became_peered":"2024-04-21T08:36:20.283220+0000","last_unstale":"2024-04-21T08:36:44.390799+0000","last_undegraded":"2024-04-21T08:36:44.390799+0000","last_fullsized":"2024-04-21T08:36:44.390799+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:55:24.233237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":3842,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":12,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111763+0000","last_change":"2024-04-21T08:36:20.919264+0000","last_active":"2024-04-21T08:36:26.111763+0000","last_peered":"2024-04-21T08:36:26.111763+0000","last_clean":"2024-04-21T08:36:26.111763+0000","last_became_active":"2024-04-21T08:36:20.918878+0000","last_became_peered":"2024-04-21T08:36:20.918878+0000","last_unstale":"2024-04-21T08:36:26.111763+0000","last_undegraded":"2024-04-21T08:36:26.111763+0000","last_fullsized":"2024-04-21T08:36:26.111763+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:24:06.460758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.112790+0000","last_change":"2024-04-21T08:36:24.926878+0000","last_active":"2024-04-21T08:36:26.112790+0000","last_peered":"2024-04-21T08:36:26.112790+0000","last_clean":"2024-04-21T08:36:26.112790+0000","last_became_active":"2024-04-21T08:36:24.926431+0000","last_became_peered":"2024-04-21T08:36:24.926431+0000","last_unstale":"2024-04-21T08:36:26.112790+0000","last_undegraded":"2024-04-21T08:36:26.112790+0000","last_fullsized":"2024-04-21T08:36:26.112790+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:51:29.065962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'4","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:44.391809+0000","last_change":"2024-04-21T08:36:20.284841+0000","last_active":"2024-04-21T08:36:44.391809+0000","last_peered":"2024-04-21T08:36:44.391809+0000","last_clean":"2024-04-21T08:36:44.391809+0000","last_became_active":"2024-04-21T08:36:20.283888+0000","last_became_peered":"2024-04-21T08:36:20.283888+0000","last_unstale":"2024-04-21T08:36:44.391809+0000","last_undegraded":"2024-04-21T08:36:44.391809+0000","last_fullsized":"2024-04-21T08:36:44.391809+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:29:42.659608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":34,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":7,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110561+0000","last_change":"2024-04-21T08:36:20.905140+0000","last_active":"2024-04-21T08:36:26.110561+0000","last_peered":"2024-04-21T08:36:26.110561+0000","last_clean":"2024-04-21T08:36:26.110561+0000","last_became_active":"2024-04-21T08:36:20.904821+0000","last_became_peered":"2024-04-21T08:36:20.904821+0000","last_unstale":"2024-04-21T08:36:26.110561+0000","last_undegraded":"2024-04-21T08:36:26.110561+0000","last_fullsized":"2024-04-21T08:36:26.110561+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:09:11.962877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111735+0000","last_change":"2024-04-21T08:36:24.912062+0000","last_active":"2024-04-21T08:36:26.111735+0000","last_peered":"2024-04-21T08:36:26.111735+0000","last_clean":"2024-04-21T08:36:26.111735+0000","last_became_active":"2024-04-21T08:36:24.911600+0000","last_became_peered":"2024-04-21T08:36:24.911600+0000","last_unstale":"2024-04-21T08:36:26.111735+0000","last_undegraded":"2024-04-21T08:36:26.111735+0000","last_fullsized":"2024-04-21T08:36:26.111735+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:31:59.617161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":27,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:45.447682+0000","last_change":"2024-04-21T08:36:20.290289+0000","last_active":"2024-04-21T08:36:45.447682+0000","last_peered":"2024-04-21T08:36:45.447682+0000","last_clean":"2024-04-21T08:36:45.447682+0000","last_became_active":"2024-04-21T08:36:20.289922+0000","last_became_peered":"2024-04-21T08:36:20.289922+0000","last_unstale":"2024-04-21T08:36:45.447682+0000","last_undegraded":"2024-04-21T08:36:45.447682+0000","last_fullsized":"2024-04-21T08:36:45.447682+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:10:00.926023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.429020+0000","last_change":"2024-04-21T08:36:20.906950+0000","last_active":"2024-04-21T08:36:25.429020+0000","last_peered":"2024-04-21T08:36:25.429020+0000","last_clean":"2024-04-21T08:36:25.429020+0000","last_became_active":"2024-04-21T08:36:20.906599+0000","last_became_peered":"2024-04-21T08:36:20.906599+0000","last_unstale":"2024-04-21T08:36:25.429020+0000","last_undegraded":"2024-04-21T08:36:25.429020+0000","last_fullsized":"2024-04-21T08:36:25.429020+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:55:27.595997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.427452+0000","last_change":"2024-04-21T08:36:24.924349+0000","last_active":"2024-04-21T08:36:25.427452+0000","last_peered":"2024-04-21T08:36:25.427452+0000","last_clean":"2024-04-21T08:36:25.427452+0000","last_became_active":"2024-04-21T08:36:24.923982+0000","last_became_peered":"2024-04-21T08:36:24.923982+0000","last_unstale":"2024-04-21T08:36:25.427452+0000","last_undegraded":"2024-04-21T08:36:25.427452+0000","last_fullsized":"2024-04-21T08:36:25.427452+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:20:29.600759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"89'6","reported_seq":35,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:47.444257+0000","last_change":"2024-04-21T08:36:20.285629+0000","last_active":"2024-04-21T08:36:47.444257+0000","last_peered":"2024-04-21T08:36:47.444257+0000","last_clean":"2024-04-21T08:36:47.444257+0000","last_became_active":"2024-04-21T08:36:20.285234+0000","last_became_peered":"2024-04-21T08:36:20.285234+0000","last_unstale":"2024-04-21T08:36:47.444257+0000","last_undegraded":"2024-04-21T08:36:47.444257+0000","last_fullsized":"2024-04-21T08:36:47.444257+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:19:32.966146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":979,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":8,"num_read_kb":4,"num_write":6,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.428721+0000","last_change":"2024-04-21T08:36:20.906819+0000","last_active":"2024-04-21T08:36:25.428721+0000","last_peered":"2024-04-21T08:36:25.428721+0000","last_clean":"2024-04-21T08:36:25.428721+0000","last_became_active":"2024-04-21T08:36:20.906459+0000","last_became_peered":"2024-04-21T08:36:20.906459+0000","last_unstale":"2024-04-21T08:36:25.428721+0000","last_undegraded":"2024-04-21T08:36:25.428721+0000","last_fullsized":"2024-04-21T08:36:25.428721+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:12:39.312857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110833+0000","last_change":"2024-04-21T08:36:24.929452+0000","last_active":"2024-04-21T08:36:26.110833+0000","last_peered":"2024-04-21T08:36:26.110833+0000","last_clean":"2024-04-21T08:36:26.110833+0000","last_became_active":"2024-04-21T08:36:24.929039+0000","last_became_peered":"2024-04-21T08:36:24.929039+0000","last_unstale":"2024-04-21T08:36:26.110833+0000","last_undegraded":"2024-04-21T08:36:26.110833+0000","last_fullsized":"2024-04-21T08:36:26.110833+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:16:05.432024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":41,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:45.449767+0000","last_change":"2024-04-21T08:36:20.293011+0000","last_active":"2024-04-21T08:36:45.449767+0000","last_peered":"2024-04-21T08:36:45.449767+0000","last_clean":"2024-04-21T08:36:45.449767+0000","last_became_active":"2024-04-21T08:36:20.291505+0000","last_became_peered":"2024-04-21T08:36:20.291505+0000","last_unstale":"2024-04-21T08:36:45.449767+0000","last_undegraded":"2024-04-21T08:36:45.449767+0000","last_fullsized":"2024-04-21T08:36:45.449767+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:33:22.665809+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,2],"acting":[8,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111789+0000","last_change":"2024-04-21T08:36:20.909076+0000","last_active":"2024-04-21T08:36:26.111789+0000","last_peered":"2024-04-21T08:36:26.111789+0000","last_clean":"2024-04-21T08:36:26.111789+0000","last_became_active":"2024-04-21T08:36:20.908739+0000","last_became_peered":"2024-04-21T08:36:20.908739+0000","last_unstale":"2024-04-21T08:36:26.111789+0000","last_undegraded":"2024-04-21T08:36:26.111789+0000","last_fullsized":"2024-04-21T08:36:26.111789+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:03:56.870477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111724+0000","last_change":"2024-04-21T08:36:24.926871+0000","last_active":"2024-04-21T08:36:26.111724+0000","last_peered":"2024-04-21T08:36:26.111724+0000","last_clean":"2024-04-21T08:36:26.111724+0000","last_became_active":"2024-04-21T08:36:24.926438+0000","last_became_peered":"2024-04-21T08:36:24.926438+0000","last_unstale":"2024-04-21T08:36:26.111724+0000","last_undegraded":"2024-04-21T08:36:26.111724+0000","last_fullsized":"2024-04-21T08:36:26.111724+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:24:48.984906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:45.436287+0000","last_change":"2024-04-21T08:36:20.284686+0000","last_active":"2024-04-21T08:36:45.436287+0000","last_peered":"2024-04-21T08:36:45.436287+0000","last_clean":"2024-04-21T08:36:45.436287+0000","last_became_active":"2024-04-21T08:36:20.284366+0000","last_became_peered":"2024-04-21T08:36:20.284366+0000","last_unstale":"2024-04-21T08:36:45.436287+0000","last_undegraded":"2024-04-21T08:36:45.436287+0000","last_fullsized":"2024-04-21T08:36:45.436287+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:01:38.557724+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,5,4],"acting":[8,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.e","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.112120+0000","last_change":"2024-04-21T08:36:20.909532+0000","last_active":"2024-04-21T08:36:26.112120+0000","last_peered":"2024-04-21T08:36:26.112120+0000","last_clean":"2024-04-21T08:36:26.112120+0000","last_became_active":"2024-04-21T08:36:20.909013+0000","last_became_peered":"2024-04-21T08:36:20.909013+0000","last_unstale":"2024-04-21T08:36:26.112120+0000","last_undegraded":"2024-04-21T08:36:26.112120+0000","last_fullsized":"2024-04-21T08:36:26.112120+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:52:05.349237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111121+0000","last_change":"2024-04-21T08:36:24.929420+0000","last_active":"2024-04-21T08:36:26.111121+0000","last_peered":"2024-04-21T08:36:26.111121+0000","last_clean":"2024-04-21T08:36:26.111121+0000","last_became_active":"2024-04-21T08:36:24.928981+0000","last_became_peered":"2024-04-21T08:36:24.928981+0000","last_unstale":"2024-04-21T08:36:26.111121+0000","last_undegraded":"2024-04-21T08:36:26.111121+0000","last_fullsized":"2024-04-21T08:36:26.111121+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:38:00.040456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.427240+0000","last_change":"2024-04-21T08:36:20.279910+0000","last_active":"2024-04-21T08:36:25.427240+0000","last_peered":"2024-04-21T08:36:25.427240+0000","last_clean":"2024-04-21T08:36:25.427240+0000","last_became_active":"2024-04-21T08:36:20.278904+0000","last_became_peered":"2024-04-21T08:36:20.278904+0000","last_unstale":"2024-04-21T08:36:25.427240+0000","last_undegraded":"2024-04-21T08:36:25.427240+0000","last_fullsized":"2024-04-21T08:36:25.427240+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:40:19.209213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111602+0000","last_change":"2024-04-21T08:36:20.895816+0000","last_active":"2024-04-21T08:36:26.111602+0000","last_peered":"2024-04-21T08:36:26.111602+0000","last_clean":"2024-04-21T08:36:26.111602+0000","last_became_active":"2024-04-21T08:36:20.895487+0000","last_became_peered":"2024-04-21T08:36:20.895487+0000","last_unstale":"2024-04-21T08:36:26.111602+0000","last_undegraded":"2024-04-21T08:36:26.111602+0000","last_fullsized":"2024-04-21T08:36:26.111602+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:53:52.153683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111291+0000","last_change":"2024-04-21T08:36:24.917637+0000","last_active":"2024-04-21T08:36:26.111291+0000","last_peered":"2024-04-21T08:36:26.111291+0000","last_clean":"2024-04-21T08:36:26.111291+0000","last_became_active":"2024-04-21T08:36:24.917192+0000","last_became_peered":"2024-04-21T08:36:24.917192+0000","last_unstale":"2024-04-21T08:36:26.111291+0000","last_undegraded":"2024-04-21T08:36:26.111291+0000","last_fullsized":"2024-04-21T08:36:26.111291+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:02:59.843903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:47.468020+0000","last_change":"2024-04-21T08:36:20.286424+0000","last_active":"2024-04-21T08:36:47.468020+0000","last_peered":"2024-04-21T08:36:47.468020+0000","last_clean":"2024-04-21T08:36:47.468020+0000","last_became_active":"2024-04-21T08:36:20.285767+0000","last_became_peered":"2024-04-21T08:36:20.285767+0000","last_unstale":"2024-04-21T08:36:47.468020+0000","last_undegraded":"2024-04-21T08:36:47.468020+0000","last_fullsized":"2024-04-21T08:36:47.468020+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:53:33.202734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":3,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111388+0000","last_change":"2024-04-21T08:36:20.906785+0000","last_active":"2024-04-21T08:36:26.111388+0000","last_peered":"2024-04-21T08:36:26.111388+0000","last_clean":"2024-04-21T08:36:26.111388+0000","last_became_active":"2024-04-21T08:36:20.906485+0000","last_became_peered":"2024-04-21T08:36:20.906485+0000","last_unstale":"2024-04-21T08:36:26.111388+0000","last_undegraded":"2024-04-21T08:36:26.111388+0000","last_fullsized":"2024-04-21T08:36:26.111388+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:54:34.664898+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,4,0],"acting":[10,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110796+0000","last_change":"2024-04-21T08:36:24.919669+0000","last_active":"2024-04-21T08:36:26.110796+0000","last_peered":"2024-04-21T08:36:26.110796+0000","last_clean":"2024-04-21T08:36:26.110796+0000","last_became_active":"2024-04-21T08:36:24.919281+0000","last_became_peered":"2024-04-21T08:36:24.919281+0000","last_unstale":"2024-04-21T08:36:26.110796+0000","last_undegraded":"2024-04-21T08:36:26.110796+0000","last_fullsized":"2024-04-21T08:36:26.110796+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:44:27.480367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.112602+0000","last_change":"2024-04-21T08:36:20.909272+0000","last_active":"2024-04-21T08:36:26.112602+0000","last_peered":"2024-04-21T08:36:26.112602+0000","last_clean":"2024-04-21T08:36:26.112602+0000","last_became_active":"2024-04-21T08:36:20.908928+0000","last_became_peered":"2024-04-21T08:36:20.908928+0000","last_unstale":"2024-04-21T08:36:26.112602+0000","last_undegraded":"2024-04-21T08:36:26.112602+0000","last_fullsized":"2024-04-21T08:36:26.112602+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:28:35.822293+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,4],"acting":[5,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"89'5","reported_seq":32,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:47.443433+0000","last_change":"2024-04-21T08:36:20.291033+0000","last_active":"2024-04-21T08:36:47.443433+0000","last_peered":"2024-04-21T08:36:47.443433+0000","last_clean":"2024-04-21T08:36:47.443433+0000","last_became_active":"2024-04-21T08:36:20.290697+0000","last_became_peered":"2024-04-21T08:36:20.290697+0000","last_unstale":"2024-04-21T08:36:47.443433+0000","last_undegraded":"2024-04-21T08:36:47.443433+0000","last_fullsized":"2024-04-21T08:36:47.443433+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:22:54.593595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.429044+0000","last_change":"2024-04-21T08:36:24.917490+0000","last_active":"2024-04-21T08:36:25.429044+0000","last_peered":"2024-04-21T08:36:25.429044+0000","last_clean":"2024-04-21T08:36:25.429044+0000","last_became_active":"2024-04-21T08:36:24.917076+0000","last_became_peered":"2024-04-21T08:36:24.917076+0000","last_unstale":"2024-04-21T08:36:25.429044+0000","last_undegraded":"2024-04-21T08:36:25.429044+0000","last_fullsized":"2024-04-21T08:36:25.429044+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:20:43.085350+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,9],"acting":[7,5,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110425+0000","last_change":"2024-04-21T08:36:20.904197+0000","last_active":"2024-04-21T08:36:26.110425+0000","last_peered":"2024-04-21T08:36:26.110425+0000","last_clean":"2024-04-21T08:36:26.110425+0000","last_became_active":"2024-04-21T08:36:20.903099+0000","last_became_peered":"2024-04-21T08:36:20.903099+0000","last_unstale":"2024-04-21T08:36:26.110425+0000","last_undegraded":"2024-04-21T08:36:26.110425+0000","last_fullsized":"2024-04-21T08:36:26.110425+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:16:08.914335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":31,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:46.452582+0000","last_change":"2024-04-21T08:36:20.280608+0000","last_active":"2024-04-21T08:36:46.452582+0000","last_peered":"2024-04-21T08:36:46.452582+0000","last_clean":"2024-04-21T08:36:46.452582+0000","last_became_active":"2024-04-21T08:36:20.280172+0000","last_became_peered":"2024-04-21T08:36:20.280172+0000","last_unstale":"2024-04-21T08:36:46.452582+0000","last_undegraded":"2024-04-21T08:36:46.452582+0000","last_fullsized":"2024-04-21T08:36:46.452582+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:51:12.284463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":355,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.428352+0000","last_change":"2024-04-21T08:35:08.423545+0000","last_active":"2024-04-21T08:36:25.428352+0000","last_peered":"2024-04-21T08:36:25.428352+0000","last_clean":"2024-04-21T08:36:25.428352+0000","last_became_active":"2024-04-21T08:35:08.423027+0000","last_became_peered":"2024-04-21T08:35:08.423027+0000","last_unstale":"2024-04-21T08:36:25.428352+0000","last_undegraded":"2024-04-21T08:36:25.428352+0000","last_fullsized":"2024-04-21T08:36:25.428352+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:31:45.359253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:31:45.359253+0000","last_clean_scrub_stamp":"2024-04-21T08:31:45.359253+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:53:24.310824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.112115+0000","last_change":"2024-04-21T08:36:24.922422+0000","last_active":"2024-04-21T08:36:26.112115+0000","last_peered":"2024-04-21T08:36:26.112115+0000","last_clean":"2024-04-21T08:36:26.112115+0000","last_became_active":"2024-04-21T08:36:24.922006+0000","last_became_peered":"2024-04-21T08:36:24.922006+0000","last_unstale":"2024-04-21T08:36:26.112115+0000","last_undegraded":"2024-04-21T08:36:26.112115+0000","last_fullsized":"2024-04-21T08:36:26.112115+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:46:22.840802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110164+0000","last_change":"2024-04-21T08:36:20.911222+0000","last_active":"2024-04-21T08:36:26.110164+0000","last_peered":"2024-04-21T08:36:26.110164+0000","last_clean":"2024-04-21T08:36:26.110164+0000","last_became_active":"2024-04-21T08:36:20.910933+0000","last_became_peered":"2024-04-21T08:36:20.910933+0000","last_unstale":"2024-04-21T08:36:26.110164+0000","last_undegraded":"2024-04-21T08:36:26.110164+0000","last_fullsized":"2024-04-21T08:36:26.110164+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:41:25.621266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:45.398912+0000","last_change":"2024-04-21T08:36:20.288409+0000","last_active":"2024-04-21T08:36:45.398912+0000","last_peered":"2024-04-21T08:36:45.398912+0000","last_clean":"2024-04-21T08:36:45.398912+0000","last_became_active":"2024-04-21T08:36:20.287916+0000","last_became_peered":"2024-04-21T08:36:20.287916+0000","last_unstale":"2024-04-21T08:36:45.398912+0000","last_undegraded":"2024-04-21T08:36:45.398912+0000","last_fullsized":"2024-04-21T08:36:45.398912+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:04:31.068352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111671+0000","last_change":"2024-04-21T08:36:24.918166+0000","last_active":"2024-04-21T08:36:26.111671+0000","last_peered":"2024-04-21T08:36:26.111671+0000","last_clean":"2024-04-21T08:36:26.111671+0000","last_became_active":"2024-04-21T08:36:24.917745+0000","last_became_peered":"2024-04-21T08:36:24.917745+0000","last_unstale":"2024-04-21T08:36:26.111671+0000","last_undegraded":"2024-04-21T08:36:26.111671+0000","last_fullsized":"2024-04-21T08:36:26.111671+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:55:41.169681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:45.450149+0000","last_change":"2024-04-21T08:36:20.291079+0000","last_active":"2024-04-21T08:36:45.450149+0000","last_peered":"2024-04-21T08:36:45.450149+0000","last_clean":"2024-04-21T08:36:45.450149+0000","last_became_active":"2024-04-21T08:36:20.290720+0000","last_became_peered":"2024-04-21T08:36:20.290720+0000","last_unstale":"2024-04-21T08:36:45.450149+0000","last_undegraded":"2024-04-21T08:36:45.450149+0000","last_fullsized":"2024-04-21T08:36:45.450149+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:41.269029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111091+0000","last_change":"2024-04-21T08:36:20.907416+0000","last_active":"2024-04-21T08:36:26.111091+0000","last_peered":"2024-04-21T08:36:26.111091+0000","last_clean":"2024-04-21T08:36:26.111091+0000","last_became_active":"2024-04-21T08:36:20.907005+0000","last_became_peered":"2024-04-21T08:36:20.907005+0000","last_unstale":"2024-04-21T08:36:26.111091+0000","last_undegraded":"2024-04-21T08:36:26.111091+0000","last_fullsized":"2024-04-21T08:36:26.111091+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:58:17.979947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110396+0000","last_change":"2024-04-21T08:36:24.921475+0000","last_active":"2024-04-21T08:36:26.110396+0000","last_peered":"2024-04-21T08:36:26.110396+0000","last_clean":"2024-04-21T08:36:26.110396+0000","last_became_active":"2024-04-21T08:36:24.921091+0000","last_became_peered":"2024-04-21T08:36:24.921091+0000","last_unstale":"2024-04-21T08:36:26.110396+0000","last_undegraded":"2024-04-21T08:36:26.110396+0000","last_fullsized":"2024-04-21T08:36:26.110396+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:28:14.224501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"85'1","reported_seq":24,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.427518+0000","last_change":"2024-04-21T08:36:20.288916+0000","last_active":"2024-04-21T08:36:25.427518+0000","last_peered":"2024-04-21T08:36:25.427518+0000","last_clean":"2024-04-21T08:36:25.427518+0000","last_became_active":"2024-04-21T08:36:20.288575+0000","last_became_peered":"2024-04-21T08:36:20.288575+0000","last_unstale":"2024-04-21T08:36:25.427518+0000","last_undegraded":"2024-04-21T08:36:25.427518+0000","last_fullsized":"2024-04-21T08:36:25.427518+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:52:05.795554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111360+0000","last_change":"2024-04-21T08:36:20.912053+0000","last_active":"2024-04-21T08:36:26.111360+0000","last_peered":"2024-04-21T08:36:26.111360+0000","last_clean":"2024-04-21T08:36:26.111360+0000","last_became_active":"2024-04-21T08:36:20.911584+0000","last_became_peered":"2024-04-21T08:36:20.911584+0000","last_unstale":"2024-04-21T08:36:26.111360+0000","last_undegraded":"2024-04-21T08:36:26.111360+0000","last_fullsized":"2024-04-21T08:36:26.111360+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:48:36.369954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.428405+0000","last_change":"2024-04-21T08:36:24.919212+0000","last_active":"2024-04-21T08:36:25.428405+0000","last_peered":"2024-04-21T08:36:25.428405+0000","last_clean":"2024-04-21T08:36:25.428405+0000","last_became_active":"2024-04-21T08:36:24.918807+0000","last_became_peered":"2024-04-21T08:36:24.918807+0000","last_unstale":"2024-04-21T08:36:25.428405+0000","last_undegraded":"2024-04-21T08:36:25.428405+0000","last_fullsized":"2024-04-21T08:36:25.428405+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:25:50.816775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":30,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:46.452689+0000","last_change":"2024-04-21T08:36:20.282536+0000","last_active":"2024-04-21T08:36:46.452689+0000","last_peered":"2024-04-21T08:36:46.452689+0000","last_clean":"2024-04-21T08:36:46.452689+0000","last_became_active":"2024-04-21T08:36:20.282172+0000","last_became_peered":"2024-04-21T08:36:20.282172+0000","last_unstale":"2024-04-21T08:36:46.452689+0000","last_undegraded":"2024-04-21T08:36:46.452689+0000","last_fullsized":"2024-04-21T08:36:46.452689+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:15:38.992441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.348351+0000","last_change":"2024-04-21T08:36:20.908416+0000","last_active":"2024-04-21T08:36:26.348351+0000","last_peered":"2024-04-21T08:36:26.348351+0000","last_clean":"2024-04-21T08:36:26.348351+0000","last_became_active":"2024-04-21T08:36:20.908054+0000","last_became_peered":"2024-04-21T08:36:20.908054+0000","last_unstale":"2024-04-21T08:36:26.348351+0000","last_undegraded":"2024-04-21T08:36:26.348351+0000","last_fullsized":"2024-04-21T08:36:26.348351+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:23:16.562060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111261+0000","last_change":"2024-04-21T08:36:24.920637+0000","last_active":"2024-04-21T08:36:26.111261+0000","last_peered":"2024-04-21T08:36:26.111261+0000","last_clean":"2024-04-21T08:36:26.111261+0000","last_became_active":"2024-04-21T08:36:24.919921+0000","last_became_peered":"2024-04-21T08:36:24.919921+0000","last_unstale":"2024-04-21T08:36:26.111261+0000","last_undegraded":"2024-04-21T08:36:26.111261+0000","last_fullsized":"2024-04-21T08:36:26.111261+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:20:38.230087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":25,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:44.390346+0000","last_change":"2024-04-21T08:36:20.279332+0000","last_active":"2024-04-21T08:36:44.390346+0000","last_peered":"2024-04-21T08:36:44.390346+0000","last_clean":"2024-04-21T08:36:44.390346+0000","last_became_active":"2024-04-21T08:36:20.278970+0000","last_became_peered":"2024-04-21T08:36:20.278970+0000","last_unstale":"2024-04-21T08:36:44.390346+0000","last_undegraded":"2024-04-21T08:36:44.390346+0000","last_fullsized":"2024-04-21T08:36:44.390346+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:36:16.041610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110784+0000","last_change":"2024-04-21T08:36:20.919621+0000","last_active":"2024-04-21T08:36:26.110784+0000","last_peered":"2024-04-21T08:36:26.110784+0000","last_clean":"2024-04-21T08:36:26.110784+0000","last_became_active":"2024-04-21T08:36:20.919246+0000","last_became_peered":"2024-04-21T08:36:20.919246+0000","last_unstale":"2024-04-21T08:36:26.110784+0000","last_undegraded":"2024-04-21T08:36:26.110784+0000","last_fullsized":"2024-04-21T08:36:26.110784+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:31:59.001501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.427567+0000","last_change":"2024-04-21T08:36:24.920459+0000","last_active":"2024-04-21T08:36:25.427567+0000","last_peered":"2024-04-21T08:36:25.427567+0000","last_clean":"2024-04-21T08:36:25.427567+0000","last_became_active":"2024-04-21T08:36:24.919691+0000","last_became_peered":"2024-04-21T08:36:24.919691+0000","last_unstale":"2024-04-21T08:36:25.427567+0000","last_undegraded":"2024-04-21T08:36:25.427567+0000","last_fullsized":"2024-04-21T08:36:25.427567+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:38:26.655452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:44.392193+0000","last_change":"2024-04-21T08:36:20.284286+0000","last_active":"2024-04-21T08:36:44.392193+0000","last_peered":"2024-04-21T08:36:44.392193+0000","last_clean":"2024-04-21T08:36:44.392193+0000","last_became_active":"2024-04-21T08:36:20.283903+0000","last_became_peered":"2024-04-21T08:36:20.283903+0000","last_unstale":"2024-04-21T08:36:44.392193+0000","last_undegraded":"2024-04-21T08:36:44.392193+0000","last_fullsized":"2024-04-21T08:36:44.392193+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:51:52.409988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111041+0000","last_change":"2024-04-21T08:36:20.911892+0000","last_active":"2024-04-21T08:36:26.111041+0000","last_peered":"2024-04-21T08:36:26.111041+0000","last_clean":"2024-04-21T08:36:26.111041+0000","last_became_active":"2024-04-21T08:36:20.911513+0000","last_became_peered":"2024-04-21T08:36:20.911513+0000","last_unstale":"2024-04-21T08:36:26.111041+0000","last_undegraded":"2024-04-21T08:36:26.111041+0000","last_fullsized":"2024-04-21T08:36:26.111041+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:51:13.733534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":15,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:28.757014+0000","last_change":"2024-04-21T08:36:24.920347+0000","last_active":"2024-04-21T08:36:28.757014+0000","last_peered":"2024-04-21T08:36:28.757014+0000","last_clean":"2024-04-21T08:36:28.757014+0000","last_became_active":"2024-04-21T08:36:24.919831+0000","last_became_peered":"2024-04-21T08:36:24.919831+0000","last_unstale":"2024-04-21T08:36:28.757014+0000","last_undegraded":"2024-04-21T08:36:28.757014+0000","last_fullsized":"2024-04-21T08:36:28.757014+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:14:27.020476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.428598+0000","last_change":"2024-04-21T08:36:20.286786+0000","last_active":"2024-04-21T08:36:25.428598+0000","last_peered":"2024-04-21T08:36:25.428598+0000","last_clean":"2024-04-21T08:36:25.428598+0000","last_became_active":"2024-04-21T08:36:20.286429+0000","last_became_peered":"2024-04-21T08:36:20.286429+0000","last_unstale":"2024-04-21T08:36:25.428598+0000","last_undegraded":"2024-04-21T08:36:25.428598+0000","last_fullsized":"2024-04-21T08:36:25.428598+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:40:57.093370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111061+0000","last_change":"2024-04-21T08:36:20.904767+0000","last_active":"2024-04-21T08:36:26.111061+0000","last_peered":"2024-04-21T08:36:26.111061+0000","last_clean":"2024-04-21T08:36:26.111061+0000","last_became_active":"2024-04-21T08:36:20.904406+0000","last_became_peered":"2024-04-21T08:36:20.904406+0000","last_unstale":"2024-04-21T08:36:26.111061+0000","last_undegraded":"2024-04-21T08:36:26.111061+0000","last_fullsized":"2024-04-21T08:36:26.111061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:11:41.644549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111147+0000","last_change":"2024-04-21T08:36:24.918031+0000","last_active":"2024-04-21T08:36:26.111147+0000","last_peered":"2024-04-21T08:36:26.111147+0000","last_clean":"2024-04-21T08:36:26.111147+0000","last_became_active":"2024-04-21T08:36:24.917618+0000","last_became_peered":"2024-04-21T08:36:24.917618+0000","last_unstale":"2024-04-21T08:36:26.111147+0000","last_undegraded":"2024-04-21T08:36:26.111147+0000","last_fullsized":"2024-04-21T08:36:26.111147+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:50:35.797911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:44.391721+0000","last_change":"2024-04-21T08:36:20.286212+0000","last_active":"2024-04-21T08:36:44.391721+0000","last_peered":"2024-04-21T08:36:44.391721+0000","last_clean":"2024-04-21T08:36:44.391721+0000","last_became_active":"2024-04-21T08:36:20.285878+0000","last_became_peered":"2024-04-21T08:36:20.285878+0000","last_unstale":"2024-04-21T08:36:44.391721+0000","last_undegraded":"2024-04-21T08:36:44.391721+0000","last_fullsized":"2024-04-21T08:36:44.391721+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:19.375455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.428818+0000","last_change":"2024-04-21T08:36:20.908335+0000","last_active":"2024-04-21T08:36:25.428818+0000","last_peered":"2024-04-21T08:36:25.428818+0000","last_clean":"2024-04-21T08:36:25.428818+0000","last_became_active":"2024-04-21T08:36:20.907945+0000","last_became_peered":"2024-04-21T08:36:20.907945+0000","last_unstale":"2024-04-21T08:36:25.428818+0000","last_undegraded":"2024-04-21T08:36:25.428818+0000","last_fullsized":"2024-04-21T08:36:25.428818+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:47:17.841511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.025123+0000","last_change":"2024-04-21T08:36:24.915553+0000","last_active":"2024-04-21T08:36:26.025123+0000","last_peered":"2024-04-21T08:36:26.025123+0000","last_clean":"2024-04-21T08:36:26.025123+0000","last_became_active":"2024-04-21T08:36:24.915028+0000","last_became_peered":"2024-04-21T08:36:24.915028+0000","last_unstale":"2024-04-21T08:36:26.025123+0000","last_undegraded":"2024-04-21T08:36:26.025123+0000","last_fullsized":"2024-04-21T08:36:26.025123+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:17:21.192946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"0'0","reported_seq":23,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.428927+0000","last_change":"2024-04-21T08:36:20.279649+0000","last_active":"2024-04-21T08:36:25.428927+0000","last_peered":"2024-04-21T08:36:25.428927+0000","last_clean":"2024-04-21T08:36:25.428927+0000","last_became_active":"2024-04-21T08:36:20.279111+0000","last_became_peered":"2024-04-21T08:36:20.279111+0000","last_unstale":"2024-04-21T08:36:25.428927+0000","last_undegraded":"2024-04-21T08:36:25.428927+0000","last_fullsized":"2024-04-21T08:36:25.428927+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:19:57.380495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.429094+0000","last_change":"2024-04-21T08:36:20.911489+0000","last_active":"2024-04-21T08:36:25.429094+0000","last_peered":"2024-04-21T08:36:25.429094+0000","last_clean":"2024-04-21T08:36:25.429094+0000","last_became_active":"2024-04-21T08:36:20.911168+0000","last_became_peered":"2024-04-21T08:36:20.911168+0000","last_unstale":"2024-04-21T08:36:25.429094+0000","last_undegraded":"2024-04-21T08:36:25.429094+0000","last_fullsized":"2024-04-21T08:36:25.429094+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:40:41.585841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.348262+0000","last_change":"2024-04-21T08:36:24.921175+0000","last_active":"2024-04-21T08:36:26.348262+0000","last_peered":"2024-04-21T08:36:26.348262+0000","last_clean":"2024-04-21T08:36:26.348262+0000","last_became_active":"2024-04-21T08:36:24.920802+0000","last_became_peered":"2024-04-21T08:36:24.920802+0000","last_unstale":"2024-04-21T08:36:26.348262+0000","last_undegraded":"2024-04-21T08:36:26.348262+0000","last_fullsized":"2024-04-21T08:36:26.348262+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:43:12.140016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":26,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:45.447489+0000","last_change":"2024-04-21T08:36:20.290202+0000","last_active":"2024-04-21T08:36:45.447489+0000","last_peered":"2024-04-21T08:36:45.447489+0000","last_clean":"2024-04-21T08:36:45.447489+0000","last_became_active":"2024-04-21T08:36:20.289868+0000","last_became_peered":"2024-04-21T08:36:20.289868+0000","last_unstale":"2024-04-21T08:36:45.447489+0000","last_undegraded":"2024-04-21T08:36:45.447489+0000","last_fullsized":"2024-04-21T08:36:45.447489+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:07:45.525980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.428608+0000","last_change":"2024-04-21T08:36:20.899590+0000","last_active":"2024-04-21T08:36:25.428608+0000","last_peered":"2024-04-21T08:36:25.428608+0000","last_clean":"2024-04-21T08:36:25.428608+0000","last_became_active":"2024-04-21T08:36:20.899254+0000","last_became_peered":"2024-04-21T08:36:20.899254+0000","last_unstale":"2024-04-21T08:36:25.428608+0000","last_undegraded":"2024-04-21T08:36:25.428608+0000","last_fullsized":"2024-04-21T08:36:25.428608+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:34:25.409694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":14,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111016+0000","last_change":"2024-04-21T08:36:24.918310+0000","last_active":"2024-04-21T08:36:26.111016+0000","last_peered":"2024-04-21T08:36:26.111016+0000","last_clean":"2024-04-21T08:36:26.111016+0000","last_became_active":"2024-04-21T08:36:24.917899+0000","last_became_peered":"2024-04-21T08:36:24.917899+0000","last_unstale":"2024-04-21T08:36:26.111016+0000","last_undegraded":"2024-04-21T08:36:26.111016+0000","last_fullsized":"2024-04-21T08:36:26.111016+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:09:13.006884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":28,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:46.453981+0000","last_change":"2024-04-21T08:36:20.289465+0000","last_active":"2024-04-21T08:36:46.453981+0000","last_peered":"2024-04-21T08:36:46.453981+0000","last_clean":"2024-04-21T08:36:46.453981+0000","last_became_active":"2024-04-21T08:36:20.289190+0000","last_became_peered":"2024-04-21T08:36:20.289190+0000","last_unstale":"2024-04-21T08:36:46.453981+0000","last_undegraded":"2024-04-21T08:36:46.453981+0000","last_fullsized":"2024-04-21T08:36:46.453981+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:33:01.267400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,2],"acting":[6,3,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.429006+0000","last_change":"2024-04-21T08:36:20.912368+0000","last_active":"2024-04-21T08:36:25.429006+0000","last_peered":"2024-04-21T08:36:25.429006+0000","last_clean":"2024-04-21T08:36:25.429006+0000","last_became_active":"2024-04-21T08:36:20.912009+0000","last_became_peered":"2024-04-21T08:36:20.912009+0000","last_unstale":"2024-04-21T08:36:25.429006+0000","last_undegraded":"2024-04-21T08:36:25.429006+0000","last_fullsized":"2024-04-21T08:36:25.429006+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:35:40.700314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111165+0000","last_change":"2024-04-21T08:36:20.913472+0000","last_active":"2024-04-21T08:36:26.111165+0000","last_peered":"2024-04-21T08:36:26.111165+0000","last_clean":"2024-04-21T08:36:26.111165+0000","last_became_active":"2024-04-21T08:36:20.913118+0000","last_became_peered":"2024-04-21T08:36:20.913118+0000","last_unstale":"2024-04-21T08:36:26.111165+0000","last_undegraded":"2024-04-21T08:36:26.111165+0000","last_fullsized":"2024-04-21T08:36:26.111165+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:31:53.957044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.428667+0000","last_change":"2024-04-21T08:36:20.913677+0000","last_active":"2024-04-21T08:36:25.428667+0000","last_peered":"2024-04-21T08:36:25.428667+0000","last_clean":"2024-04-21T08:36:25.428667+0000","last_became_active":"2024-04-21T08:36:20.913097+0000","last_became_peered":"2024-04-21T08:36:20.913097+0000","last_unstale":"2024-04-21T08:36:25.428667+0000","last_undegraded":"2024-04-21T08:36:25.428667+0000","last_fullsized":"2024-04-21T08:36:25.428667+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:08:40.388370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.428415+0000","last_change":"2024-04-21T08:36:20.906624+0000","last_active":"2024-04-21T08:36:25.428415+0000","last_peered":"2024-04-21T08:36:25.428415+0000","last_clean":"2024-04-21T08:36:25.428415+0000","last_became_active":"2024-04-21T08:36:20.906247+0000","last_became_peered":"2024-04-21T08:36:20.906247+0000","last_unstale":"2024-04-21T08:36:25.428415+0000","last_undegraded":"2024-04-21T08:36:25.428415+0000","last_fullsized":"2024-04-21T08:36:25.428415+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:44:11.855183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.348364+0000","last_change":"2024-04-21T08:36:20.906576+0000","last_active":"2024-04-21T08:36:26.348364+0000","last_peered":"2024-04-21T08:36:26.348364+0000","last_clean":"2024-04-21T08:36:26.348364+0000","last_became_active":"2024-04-21T08:36:20.906231+0000","last_became_peered":"2024-04-21T08:36:20.906231+0000","last_unstale":"2024-04-21T08:36:26.348364+0000","last_undegraded":"2024-04-21T08:36:26.348364+0000","last_fullsized":"2024-04-21T08:36:26.348364+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:42:48.156614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110877+0000","last_change":"2024-04-21T08:36:20.902551+0000","last_active":"2024-04-21T08:36:26.110877+0000","last_peered":"2024-04-21T08:36:26.110877+0000","last_clean":"2024-04-21T08:36:26.110877+0000","last_became_active":"2024-04-21T08:36:20.902208+0000","last_became_peered":"2024-04-21T08:36:20.902208+0000","last_unstale":"2024-04-21T08:36:26.110877+0000","last_undegraded":"2024-04-21T08:36:26.110877+0000","last_fullsized":"2024-04-21T08:36:26.110877+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:36:30.016256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111890+0000","last_change":"2024-04-21T08:36:20.918588+0000","last_active":"2024-04-21T08:36:26.111890+0000","last_peered":"2024-04-21T08:36:26.111890+0000","last_clean":"2024-04-21T08:36:26.111890+0000","last_became_active":"2024-04-21T08:36:20.918263+0000","last_became_peered":"2024-04-21T08:36:20.918263+0000","last_unstale":"2024-04-21T08:36:26.111890+0000","last_undegraded":"2024-04-21T08:36:26.111890+0000","last_fullsized":"2024-04-21T08:36:26.111890+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:51:30.340336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:25.429140+0000","last_change":"2024-04-21T08:36:20.911063+0000","last_active":"2024-04-21T08:36:25.429140+0000","last_peered":"2024-04-21T08:36:25.429140+0000","last_clean":"2024-04-21T08:36:25.429140+0000","last_became_active":"2024-04-21T08:36:20.910647+0000","last_became_peered":"2024-04-21T08:36:20.910647+0000","last_unstale":"2024-04-21T08:36:25.429140+0000","last_undegraded":"2024-04-21T08:36:25.429140+0000","last_fullsized":"2024-04-21T08:36:25.429140+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:10:31.079782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111443+0000","last_change":"2024-04-21T08:36:20.914079+0000","last_active":"2024-04-21T08:36:26.111443+0000","last_peered":"2024-04-21T08:36:26.111443+0000","last_clean":"2024-04-21T08:36:26.111443+0000","last_became_active":"2024-04-21T08:36:20.912851+0000","last_became_peered":"2024-04-21T08:36:20.912851+0000","last_unstale":"2024-04-21T08:36:26.111443+0000","last_undegraded":"2024-04-21T08:36:26.111443+0000","last_fullsized":"2024-04-21T08:36:26.111443+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:53:35.811281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"0'0","reported_seq":20,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.112496+0000","last_change":"2024-04-21T08:36:20.900551+0000","last_active":"2024-04-21T08:36:26.112496+0000","last_peered":"2024-04-21T08:36:26.112496+0000","last_clean":"2024-04-21T08:36:26.112496+0000","last_became_active":"2024-04-21T08:36:20.900173+0000","last_became_peered":"2024-04-21T08:36:20.900173+0000","last_unstale":"2024-04-21T08:36:26.112496+0000","last_undegraded":"2024-04-21T08:36:26.112496+0000","last_fullsized":"2024-04-21T08:36:26.112496+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:21.389498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110424+0000","last_change":"2024-04-21T08:36:20.912953+0000","last_active":"2024-04-21T08:36:26.110424+0000","last_peered":"2024-04-21T08:36:26.110424+0000","last_clean":"2024-04-21T08:36:26.110424+0000","last_became_active":"2024-04-21T08:36:20.912223+0000","last_became_peered":"2024-04-21T08:36:20.912223+0000","last_unstale":"2024-04-21T08:36:26.110424+0000","last_undegraded":"2024-04-21T08:36:26.110424+0000","last_fullsized":"2024-04-21T08:36:26.110424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:43:14.277179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.025233+0000","last_change":"2024-04-21T08:36:20.906164+0000","last_active":"2024-04-21T08:36:26.025233+0000","last_peered":"2024-04-21T08:36:26.025233+0000","last_clean":"2024-04-21T08:36:26.025233+0000","last_became_active":"2024-04-21T08:36:20.905749+0000","last_became_peered":"2024-04-21T08:36:20.905749+0000","last_unstale":"2024-04-21T08:36:26.025233+0000","last_undegraded":"2024-04-21T08:36:26.025233+0000","last_fullsized":"2024-04-21T08:36:26.025233+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:02:10.569087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.110554+0000","last_change":"2024-04-21T08:36:20.900314+0000","last_active":"2024-04-21T08:36:26.110554+0000","last_peered":"2024-04-21T08:36:26.110554+0000","last_clean":"2024-04-21T08:36:26.110554+0000","last_became_active":"2024-04-21T08:36:20.900009+0000","last_became_peered":"2024-04-21T08:36:20.900009+0000","last_unstale":"2024-04-21T08:36:26.110554+0000","last_undegraded":"2024-04-21T08:36:26.110554+0000","last_fullsized":"2024-04-21T08:36:26.110554+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:19:59.997302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"0'0","reported_seq":21,"reported_epoch":89,"state":"active+clean","last_fresh":"2024-04-21T08:36:26.111776+0000","last_change":"2024-04-21T08:36:20.920725+0000","last_active":"2024-04-21T08:36:26.111776+0000","last_peered":"2024-04-21T08:36:26.111776+0000","last_clean":"2024-04-21T08:36:26.111776+0000","last_became_active":"2024-04-21T08:36:20.920288+0000","last_became_peered":"2024-04-21T08:36:20.920288+0000","last_unstale":"2024-04-21T08:36:26.111776+0000","last_undegraded":"2024-04-21T08:36:26.111776+0000","last_fullsized":"2024-04-21T08:36:26.111776+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:36:18.736388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":16384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":9662,"num_objects":75,"num_object_clones":0,"num_object_copies":225,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":75,"num_whiteouts":0,"num_read":31,"num_read_kb":23,"num_write":157,"num_write_kb":63,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":48,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":294912,"data_stored":27715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":140,"ondisk_log_size":140,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383698,"num_pgs":26,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32148,"kb_used_data":960,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713004,"statfs":{"total":95995035648,"available":95962116096,"internally_reserved":0,"allocated":983040,"data_stored":477215,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":31914436},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645334,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28592,"kb_used_data":1576,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716560,"statfs":{"total":95995035648,"available":95965757440,"internally_reserved":0,"allocated":1613824,"data_stored":1072381,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808860,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32152,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93713000,"statfs":{"total":95995035648,"available":95962112000,"internally_reserved":0,"allocated":999424,"data_stored":477999,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":31914432},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070497,"num_pgs":29,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28008,"kb_used_data":992,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717144,"statfs":{"total":95995035648,"available":95966355456,"internally_reserved":0,"allocated":1015808,"data_stored":478715,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332135,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28596,"kb_used_data":1580,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716556,"statfs":{"total":95995035648,"available":95965753344,"internally_reserved":0,"allocated":1617920,"data_stored":1079699,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495660,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":999424,"data_stored":477937,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574916,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36312,"kb_used_data":976,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708840,"statfs":{"total":95995035648,"available":95957852160,"internally_reserved":0,"allocated":999424,"data_stored":481639,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738441,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28016,"kb_used_data":1000,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717136,"statfs":{"total":95995035648,"available":95966347264,"internally_reserved":0,"allocated":1024000,"data_stored":479784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411391,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36320,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708832,"statfs":{"total":95995035648,"available":95957843968,"internally_reserved":0,"allocated":1007616,"data_stored":481713,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182457,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":27992,"kb_used_data":968,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717160,"statfs":{"total":95995035648,"available":95966371840,"internally_reserved":0,"allocated":991232,"data_stored":477319,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920822,"num_pgs":37,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32160,"kb_used_data":984,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712992,"statfs":{"total":95995035648,"available":95962103808,"internally_reserved":0,"allocated":1007616,"data_stored":482402,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":31914438},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757296,"num_pgs":46,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32764,"kb_used_data":1588,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712388,"statfs":{"total":95995035648,"available":95961485312,"internally_reserved":0,"allocated":1626112,"data_stored":1074890,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2569,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":4424,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4498,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":12288,"data_stored":194,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1091,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3211,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":3924,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":1500,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":4798,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T08:36:49.514 INFO:tasks.cephfs.filesystem.ceph_manager:clean! 2024-04-21T08:36:49.515 INFO:tasks.cephadm:Setting up client nodes... 2024-04-21T08:36:49.515 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph auth get-or-create client.0 mon 'allow *' osd 'allow *' mds 'allow *' mgr 'allow *' 2024-04-21T08:36:49.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:49 smithi088 ceph-mon[31259]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:36:49.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:49 smithi088 ceph-mon[31259]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:36:49.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:49 smithi088 ceph-mon[31259]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:36:49.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:49 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1347651182' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:49.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:49 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2258307924' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:49.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:49 smithi179 ceph-mon[31829]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:36:49.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:49 smithi179 ceph-mon[31829]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:36:49.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:49 smithi179 ceph-mon[31829]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:36:49.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:49 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1347651182' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:49.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:49 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2258307924' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:49.787 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:36:49.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:49 smithi086 ceph-mon[27939]: from='client.14928 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_0", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:36:49.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:49 smithi086 ceph-mon[27939]: from='client.14934 -' entity='client.admin' cmd=[{"prefix": "fs subvolume create", "vol_name": "cephfs", "sub_name": "sv_1", "group_name": "qa", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:36:49.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:49 smithi086 ceph-mon[27939]: from='client.14940 -' entity='client.admin' cmd=[{"prefix": "fs subvolume getpath", "vol_name": "cephfs", "group_name": "qa", "sub_name": "sv_1", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:36:49.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:49 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1347651182' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:36:49.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:49 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2258307924' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:50.459 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 21 08:36:49 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-b[61651]: 2024-04-21T08:36:49.986+0000 7fa7417e1700 -1 mds.pinger is_rank_lagging: rank=4 was never sent ping request. 2024-04-21T08:36:50.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:50 smithi088 ceph-mon[31259]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:36:50.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:50 smithi088 ceph-mon[31259]: pgmap v302: 129 pgs: 129 active+clean; 586 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4.0 KiB/s wr, 11 op/s 2024-04-21T08:36:50.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:50 smithi179 ceph-mon[31829]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:36:50.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:50 smithi179 ceph-mon[31829]: pgmap v302: 129 pgs: 129 active+clean; 586 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4.0 KiB/s wr, 11 op/s 2024-04-21T08:36:50.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:50 smithi086 ceph-mon[27939]: from='client.14958 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:36:50.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:50 smithi086 ceph-mon[27939]: pgmap v302: 129 pgs: 129 active+clean; 586 KiB data, 362 MiB used, 1.0 TiB / 1.0 TiB avail; 1.9 KiB/s rd, 4.0 KiB/s wr, 11 op/s 2024-04-21T08:36:51.561 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:51 smithi086 ceph-mon[27939]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:36:51.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.572+0000 7f5001730700 1 -- 172.21.15.86:0/3221606772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ffc102bc0 msgr2=0x7f4ffc103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:51.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.572+0000 7f5001730700 1 --2- 172.21.15.86:0/3221606772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ffc102bc0 0x7f4ffc103020 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f4fec0099c0 tx=0x7f4fec009cd0 comp rx=0 tx=0).stop 2024-04-21T08:36:51.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.572+0000 7f5001730700 1 -- 172.21.15.86:0/3221606772 shutdown_connections 2024-04-21T08:36:51.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.572+0000 7f5001730700 1 --2- 172.21.15.86:0/3221606772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ffc103560 0x7f4ffc109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:51.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.572+0000 7f5001730700 1 --2- 172.21.15.86:0/3221606772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ffc1019c0 0x7f4ffc101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:51.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.572+0000 7f5001730700 1 --2- 172.21.15.86:0/3221606772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ffc102bc0 0x7f4ffc103020 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:51.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.572+0000 7f5001730700 1 -- 172.21.15.86:0/3221606772 >> 172.21.15.86:0/3221606772 conn(0x7f4ffc0fcf60 msgr2=0x7f4ffc0ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:51.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.572+0000 7f5001730700 1 -- 172.21.15.86:0/3221606772 shutdown_connections 2024-04-21T08:36:51.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.572+0000 7f5001730700 1 -- 172.21.15.86:0/3221606772 wait complete. 2024-04-21T08:36:51.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.573+0000 7f5001730700 1 Processor -- start 2024-04-21T08:36:51.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.573+0000 7f5001730700 1 -- start start 2024-04-21T08:36:51.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.573+0000 7f5001730700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ffc1019c0 0x7f4ffc06b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:51.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.573+0000 7f5001730700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ffc102bc0 0x7f4ffc19fba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:51.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.574+0000 7f5001730700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ffc103560 0x7f4ffc1a20f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:51.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.574+0000 7f5001730700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4ffc10d4b0 con 0x7f4ffc1019c0 2024-04-21T08:36:51.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.574+0000 7f5001730700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4ffc10d330 con 0x7f4ffc103560 2024-04-21T08:36:51.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.574+0000 7f5001730700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4ffc10d630 con 0x7f4ffc102bc0 2024-04-21T08:36:51.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.574+0000 7f4ffaffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ffc1019c0 0x7f4ffc06b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:51.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.574+0000 7f4ffaffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ffc1019c0 0x7f4ffc06b2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59650/0 (socket says 172.21.15.86:59650) 2024-04-21T08:36:51.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.574+0000 7f4ffaffd700 1 -- 172.21.15.86:0/1451580807 learned_addr learned my addr 172.21.15.86:0/1451580807 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:51.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.574+0000 7f4ffb7fe700 1 --2- 172.21.15.86:0/1451580807 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ffc103560 0x7f4ffc1a20f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:51.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.574+0000 7f4ffaffd700 1 -- 172.21.15.86:0/1451580807 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ffc102bc0 msgr2=0x7f4ffc19fba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:51.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.574+0000 7f4ffaffd700 1 --2- 172.21.15.86:0/1451580807 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ffc102bc0 0x7f4ffc19fba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:51.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.574+0000 7f4ffaffd700 1 -- 172.21.15.86:0/1451580807 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ffc103560 msgr2=0x7f4ffc1a20f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:51.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.574+0000 7f4ffaffd700 1 --2- 172.21.15.86:0/1451580807 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ffc103560 0x7f4ffc1a20f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:51.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.574+0000 7f4ffaffd700 1 -- 172.21.15.86:0/1451580807 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ffc1a2900 con 0x7f4ffc1019c0 2024-04-21T08:36:51.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.574+0000 7f4ffb7fe700 1 --2- 172.21.15.86:0/1451580807 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ffc103560 0x7f4ffc1a20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:36:51.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.574+0000 7f4ffaffd700 1 --2- 172.21.15.86:0/1451580807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ffc1019c0 0x7f4ffc06b2e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4fe400c990 tx=0x7f4fe400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:51.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.574+0000 7f4ff3fff700 1 -- 172.21.15.86:0/1451580807 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4007b00 con 0x7f4ffc1019c0 2024-04-21T08:36:51.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.574+0000 7f5001730700 1 -- 172.21.15.86:0/1451580807 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ffc1a2bf0 con 0x7f4ffc1019c0 2024-04-21T08:36:51.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.574+0000 7f4ff3fff700 1 -- 172.21.15.86:0/1451580807 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4fe4007ca0 con 0x7f4ffc1019c0 2024-04-21T08:36:51.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.574+0000 7f4ff3fff700 1 -- 172.21.15.86:0/1451580807 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4fe4018660 con 0x7f4ffc1019c0 2024-04-21T08:36:51.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.574+0000 7f5001730700 1 -- 172.21.15.86:0/1451580807 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ffc1a3180 con 0x7f4ffc1019c0 2024-04-21T08:36:51.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.576+0000 7f5001730700 1 -- 172.21.15.86:0/1451580807 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ffc050f80 con 0x7f4ffc1019c0 2024-04-21T08:36:51.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.576+0000 7f4ff3fff700 1 -- 172.21.15.86:0/1451580807 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4fe400f450 con 0x7f4ffc1019c0 2024-04-21T08:36:51.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.577+0000 7f4ff3fff700 1 --2- 172.21.15.86:0/1451580807 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f4fe80a3e00 0x7f4fe80a62c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:51.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.577+0000 7f4ffa7fc700 1 --2- 172.21.15.86:0/1451580807 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f4fe80a3e00 0x7f4fe80a62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:51.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.577+0000 7f4ff3fff700 1 -- 172.21.15.86:0/1451580807 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f4fe40d0770 con 0x7f4ffc1019c0 2024-04-21T08:36:51.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.578+0000 7f4ffa7fc700 1 --2- 172.21.15.86:0/1451580807 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f4fe80a3e00 0x7f4fe80a62c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4fec0098e0 tx=0x7f4fec009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:51.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.582+0000 7f4ff3fff700 1 -- 172.21.15.86:0/1451580807 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4fe4014030 con 0x7f4ffc1019c0 2024-04-21T08:36:51.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:51 smithi179 ceph-mon[31829]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:36:51.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.858+0000 7f5001730700 1 -- 172.21.15.86:0/1451580807 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]} v 0) -- 0x7f4ffc063650 con 0x7f4ffc1019c0 2024-04-21T08:36:51.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.861+0000 7f4ff3fff700 1 -- 172.21.15.86:0/1451580807 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]=0 v32) ==== 170+0+59 (secure 0 0 0) 0x7f4fe409cdb0 con 0x7f4ffc1019c0 2024-04-21T08:36:51.860 INFO:teuthology.orchestra.run.smithi086.stdout:[client.0] 2024-04-21T08:36:51.860 INFO:teuthology.orchestra.run.smithi086.stdout: key = AQAj0CRm7P9CMxAAvj7TDK++qoi9zYD7lYDOfg== 2024-04-21T08:36:51.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.863+0000 7f5001730700 1 -- 172.21.15.86:0/1451580807 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f4fe80a3e00 msgr2=0x7f4fe80a62c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:51.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.863+0000 7f5001730700 1 --2- 172.21.15.86:0/1451580807 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f4fe80a3e00 0x7f4fe80a62c0 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7f4fec0098e0 tx=0x7f4fec009f10 comp rx=0 tx=0).stop 2024-04-21T08:36:51.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.863+0000 7f5001730700 1 -- 172.21.15.86:0/1451580807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ffc1019c0 msgr2=0x7f4ffc06b2e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:51.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.863+0000 7f5001730700 1 --2- 172.21.15.86:0/1451580807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ffc1019c0 0x7f4ffc06b2e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7f4fe400c990 tx=0x7f4fe400cd50 comp rx=0 tx=0).stop 2024-04-21T08:36:51.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.863+0000 7f5001730700 1 -- 172.21.15.86:0/1451580807 shutdown_connections 2024-04-21T08:36:51.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.863+0000 7f5001730700 1 --2- 172.21.15.86:0/1451580807 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f4fe80a3e00 0x7f4fe80a62c0 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:51.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.863+0000 7f5001730700 1 --2- 172.21.15.86:0/1451580807 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ffc103560 0x7f4ffc1a20f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:51.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.863+0000 7f5001730700 1 --2- 172.21.15.86:0/1451580807 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ffc1019c0 0x7f4ffc06b2e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:51.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.863+0000 7f5001730700 1 --2- 172.21.15.86:0/1451580807 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ffc102bc0 0x7f4ffc19fba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:51.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.863+0000 7f5001730700 1 -- 172.21.15.86:0/1451580807 >> 172.21.15.86:0/1451580807 conn(0x7f4ffc0fcf60 msgr2=0x7f4ffc105df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:51.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.864+0000 7f5001730700 1 -- 172.21.15.86:0/1451580807 shutdown_connections 2024-04-21T08:36:51.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:51.864+0000 7f5001730700 1 -- 172.21.15.86:0/1451580807 wait complete. 2024-04-21T08:36:51.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:51 smithi088 ceph-mon[31259]: from='client.14964 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:36:52.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:52 smithi086 ceph-mon[27939]: pgmap v303: 129 pgs: 129 active+clean; 624 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 8.1 KiB/s wr, 20 op/s 2024-04-21T08:36:52.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:52 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1451580807' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-21T08:36:52.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:52 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1451580807' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-21T08:36:52.702 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-21T08:36:52.702 DEBUG:teuthology.orchestra.run.smithi086:> sudo dd of=/etc/ceph/ceph.client.0.keyring 2024-04-21T08:36:52.702 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2024-04-21T08:36:52.740 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2024-04-21T08:36:52.740 INFO:tasks.cephadm.ceph_manager.ceph:waiting for mgr available 2024-04-21T08:36:52.740 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph mgr dump --format=json 2024-04-21T08:36:52.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:52 smithi179 ceph-mon[31829]: pgmap v303: 129 pgs: 129 active+clean; 624 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 8.1 KiB/s wr, 20 op/s 2024-04-21T08:36:52.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:52 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1451580807' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-21T08:36:52.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:52 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1451580807' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-21T08:36:52.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:52 smithi088 ceph-mon[31259]: pgmap v303: 129 pgs: 129 active+clean; 624 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 8.1 KiB/s wr, 20 op/s 2024-04-21T08:36:52.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:52 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1451580807' entity='client.admin' cmd=[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]: dispatch 2024-04-21T08:36:52.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:52 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1451580807' entity='client.admin' cmd='[{"prefix": "auth get-or-create", "entity": "client.0", "caps": ["mon", "allow *", "osd", "allow *", "mds", "allow *", "mgr", "allow *"]}]': finished 2024-04-21T08:36:53.021 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:36:54.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:54 smithi179 ceph-mon[31829]: pgmap v304: 129 pgs: 129 active+clean; 624 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.0 KiB/s wr, 21 op/s 2024-04-21T08:36:54.770 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:54 smithi086 ceph-mon[27939]: pgmap v304: 129 pgs: 129 active+clean; 624 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.0 KiB/s wr, 21 op/s 2024-04-21T08:36:54.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.770+0000 7fbbd2c6d700 1 -- 172.21.15.86:0/510590460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbcc101400 msgr2=0x7fbbcc101880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:54.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.770+0000 7fbbd2c6d700 1 --2- 172.21.15.86:0/510590460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbcc101400 0x7fbbcc101880 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7fbbb8009990 tx=0x7fbbb8009ca0 comp rx=0 tx=0).stop 2024-04-21T08:36:54.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.770+0000 7fbbd2c6d700 1 -- 172.21.15.86:0/510590460 shutdown_connections 2024-04-21T08:36:54.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.770+0000 7fbbd2c6d700 1 --2- 172.21.15.86:0/510590460 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbcc101dc0 0x7fbbcc10f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:54.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.770+0000 7fbbd2c6d700 1 --2- 172.21.15.86:0/510590460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbcc107b20 0x7fbbcc107f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:54.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.770+0000 7fbbd2c6d700 1 --2- 172.21.15.86:0/510590460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbcc101400 0x7fbbcc101880 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:54.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.770+0000 7fbbd2c6d700 1 -- 172.21.15.86:0/510590460 >> 172.21.15.86:0/510590460 conn(0x7fbbcc076c60 msgr2=0x7fbbcc077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:54.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.771+0000 7fbbd2c6d700 1 -- 172.21.15.86:0/510590460 shutdown_connections 2024-04-21T08:36:54.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.771+0000 7fbbd2c6d700 1 -- 172.21.15.86:0/510590460 wait complete. 2024-04-21T08:36:54.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.771+0000 7fbbd2c6d700 1 Processor -- start 2024-04-21T08:36:54.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.771+0000 7fbbd2c6d700 1 -- start start 2024-04-21T08:36:54.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.771+0000 7fbbd2c6d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbcc101400 0x7fbbcc1991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:54.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.771+0000 7fbbd2c6d700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbcc101dc0 0x7fbbcc199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:54.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.771+0000 7fbbd2c6d700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbcc107b20 0x7fbbcc19dab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:54.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.771+0000 7fbbd2c6d700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc112cb0 con 0x7fbbcc101400 2024-04-21T08:36:54.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.771+0000 7fbbd2c6d700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc112b30 con 0x7fbbcc101dc0 2024-04-21T08:36:54.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.771+0000 7fbbd2c6d700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbbcc112e30 con 0x7fbbcc107b20 2024-04-21T08:36:54.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.771+0000 7fbbd0a09700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbcc101400 0x7fbbcc1991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:54.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.771+0000 7fbbd0a09700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbcc101400 0x7fbbcc1991e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59668/0 (socket says 172.21.15.86:59668) 2024-04-21T08:36:54.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.772+0000 7fbbd0a09700 1 -- 172.21.15.86:0/2657644859 learned_addr learned my addr 172.21.15.86:0/2657644859 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:54.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.772+0000 7fbbcbfff700 1 --2- 172.21.15.86:0/2657644859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbcc101dc0 0x7fbbcc199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:54.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.772+0000 7fbbd0a09700 1 -- 172.21.15.86:0/2657644859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbcc107b20 msgr2=0x7fbbcc19dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:54.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.772+0000 7fbbd0a09700 1 --2- 172.21.15.86:0/2657644859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbcc107b20 0x7fbbcc19dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:54.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.772+0000 7fbbd0a09700 1 -- 172.21.15.86:0/2657644859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbcc101dc0 msgr2=0x7fbbcc199720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:54.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.772+0000 7fbbd0a09700 1 --2- 172.21.15.86:0/2657644859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbcc101dc0 0x7fbbcc199720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:54.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.772+0000 7fbbd0a09700 1 -- 172.21.15.86:0/2657644859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbcc19e1f0 con 0x7fbbcc101400 2024-04-21T08:36:54.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.772+0000 7fbbcbfff700 1 --2- 172.21.15.86:0/2657644859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbcc101dc0 0x7fbbcc199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:36:54.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.772+0000 7fbbd0a09700 1 --2- 172.21.15.86:0/2657644859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbcc101400 0x7fbbcc1991e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbbc000b500 tx=0x7fbbc000b8c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:54.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.772+0000 7fbbc9ffb700 1 -- 172.21.15.86:0/2657644859 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0015020 con 0x7fbbcc101400 2024-04-21T08:36:54.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.772+0000 7fbbc9ffb700 1 -- 172.21.15.86:0/2657644859 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbc0011ce0 con 0x7fbbcc101400 2024-04-21T08:36:54.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.772+0000 7fbbc9ffb700 1 -- 172.21.15.86:0/2657644859 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbc0010d60 con 0x7fbbcc101400 2024-04-21T08:36:54.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.772+0000 7fbbd2c6d700 1 -- 172.21.15.86:0/2657644859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbcc19e4e0 con 0x7fbbcc101400 2024-04-21T08:36:54.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.772+0000 7fbbd2c6d700 1 -- 172.21.15.86:0/2657644859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbcc102510 con 0x7fbbcc101400 2024-04-21T08:36:54.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.773+0000 7fbbd2c6d700 1 -- 172.21.15.86:0/2657644859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbcc069730 con 0x7fbbcc101400 2024-04-21T08:36:54.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.774+0000 7fbbc9ffb700 1 -- 172.21.15.86:0/2657644859 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbbc0018020 con 0x7fbbcc101400 2024-04-21T08:36:54.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.775+0000 7fbbc9ffb700 1 --2- 172.21.15.86:0/2657644859 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fbbbc0a3d50 0x7fbbbc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:54.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.775+0000 7fbbcbfff700 1 --2- 172.21.15.86:0/2657644859 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fbbbc0a3d50 0x7fbbbc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:54.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.775+0000 7fbbc9ffb700 1 -- 172.21.15.86:0/2657644859 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fbbc00cfe90 con 0x7fbbcc101400 2024-04-21T08:36:54.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.775+0000 7fbbcbfff700 1 --2- 172.21.15.86:0/2657644859 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fbbbc0a3d50 0x7fbbbc0a6210 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbbb800b5c0 tx=0x7fbbb8005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:54.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:54.779+0000 7fbbc9ffb700 1 -- 172.21.15.86:0/2657644859 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbbc001f020 con 0x7fbbcc101400 2024-04-21T08:36:54.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:54 smithi088 ceph-mon[31259]: pgmap v304: 129 pgs: 129 active+clean; 624 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 8.0 KiB/s wr, 21 op/s 2024-04-21T08:36:55.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:55.035+0000 7fbbd2c6d700 1 -- 172.21.15.86:0/2657644859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json"} v 0) -- 0x7fbbcc02df80 con 0x7fbbcc101400 2024-04-21T08:36:55.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:55.040+0000 7fbbc9ffb700 1 -- 172.21.15.86:0/2657644859 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json"}]=0 v17) ==== 74+0+264997 (secure 0 0 0) 0x7fbbc009c5e0 con 0x7fbbcc101400 2024-04-21T08:36:55.042 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:36:55.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:55.043+0000 7fbbd2c6d700 1 -- 172.21.15.86:0/2657644859 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fbbbc0a3d50 msgr2=0x7fbbbc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:55.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:55.043+0000 7fbbd2c6d700 1 --2- 172.21.15.86:0/2657644859 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fbbbc0a3d50 0x7fbbbc0a6210 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7fbbb800b5c0 tx=0x7fbbb8005fb0 comp rx=0 tx=0).stop 2024-04-21T08:36:55.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:55.043+0000 7fbbd2c6d700 1 -- 172.21.15.86:0/2657644859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbcc101400 msgr2=0x7fbbcc1991e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:55.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:55.043+0000 7fbbd2c6d700 1 --2- 172.21.15.86:0/2657644859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbcc101400 0x7fbbcc1991e0 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7fbbc000b500 tx=0x7fbbc000b8c0 comp rx=0 tx=0).stop 2024-04-21T08:36:55.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:55.043+0000 7fbbd2c6d700 1 -- 172.21.15.86:0/2657644859 shutdown_connections 2024-04-21T08:36:55.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:55.043+0000 7fbbd2c6d700 1 --2- 172.21.15.86:0/2657644859 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fbbbc0a3d50 0x7fbbbc0a6210 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:55.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:55.043+0000 7fbbd2c6d700 1 --2- 172.21.15.86:0/2657644859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbcc107b20 0x7fbbcc19dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:55.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:55.043+0000 7fbbd2c6d700 1 --2- 172.21.15.86:0/2657644859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbcc101400 0x7fbbcc1991e0 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:55.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:55.043+0000 7fbbd2c6d700 1 --2- 172.21.15.86:0/2657644859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbcc101dc0 0x7fbbcc199720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:55.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:55.043+0000 7fbbd2c6d700 1 -- 172.21.15.86:0/2657644859 >> 172.21.15.86:0/2657644859 conn(0x7fbbcc076c60 msgr2=0x7fbbcc100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:55.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:55.043+0000 7fbbd2c6d700 1 -- 172.21.15.86:0/2657644859 shutdown_connections 2024-04-21T08:36:55.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:55.043+0000 7fbbd2c6d700 1 -- 172.21.15.86:0/2657644859 wait complete. 2024-04-21T08:36:55.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:55 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2657644859' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-21T08:36:55.808 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:55 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2657644859' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-21T08:36:55.834 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":17,"flags":0,"active_gid":14152,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6800","nonce":1364156965},{"type":"v1","addr":"172.21.15.86:6801","nonce":1364156965}]},"active_addr":"172.21.15.86:6801/1364156965","active_change":"2024-04-21T08:29:23.190113+0000","active_mgr_features":4540701547738038271,"available":true,"standbys":[{"gid":24109,"name":"z","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]},{"gid":24119,"name":"y","mgr_features":4540701547738038271,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["cephadm","dashboard","iostat","nfs","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often agent on each host will try to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"ratio of total system memory to divide amongst autotuned daemons","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command's persistent logging","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable the cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.25.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:9.4.12","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"HAproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.5.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.2.1","min":"","max":"","enum_allowed":[],"desc":"Nvme-of container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v2.43.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:2.4.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba/SMB container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"SNMP Gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"docker.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"location of dashboards to include in grafana deployments","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Deploy hw monitoring daemon on every host.","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on the hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of daemons per service per host","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"max number of osds that will be drained simultaneously when osds are removed","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default address for RedFish API (oob management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"location of alerts to include in prometheus deployments","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry url for login purposes. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest. Make sure all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"restful","can_run":true,"error_string":"","module_options":{"enable_auth":{"name":"enable_auth","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]}}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"uint","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{"dashboard":"https://172.21.15.86:8443/"},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"last_failure_osd_epoch":3,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":4273877219}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":4060044103}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":1068089552}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":2707594358}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"172.21.15.86:0","nonce":801506269}]}]} 2024-04-21T08:36:55.839 INFO:tasks.cephadm.ceph_manager.ceph:mgr available! 2024-04-21T08:36:55.839 INFO:tasks.cephadm.ceph_manager.ceph:waiting for all up 2024-04-21T08:36:55.839 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-21T08:36:55.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:55 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2657644859' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json"}]: dispatch 2024-04-21T08:36:56.074 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:36:56.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:56 smithi179 ceph-mon[31829]: pgmap v305: 129 pgs: 129 active+clean; 629 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.6 KiB/s wr, 25 op/s 2024-04-21T08:36:56.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:56 smithi086 ceph-mon[27939]: pgmap v305: 129 pgs: 129 active+clean; 629 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.6 KiB/s wr, 25 op/s 2024-04-21T08:36:56.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:56 smithi088 ceph-mon[31259]: pgmap v305: 129 pgs: 129 active+clean; 629 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.6 KiB/s wr, 25 op/s 2024-04-21T08:36:57.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:57.879+0000 7f14075da700 1 -- 172.21.15.86:0/3819372494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1400101970 msgr2=0x7f1400101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:57.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:57.879+0000 7f14075da700 1 --2- 172.21.15.86:0/3819372494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1400101970 0x7f1400101df0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7f13f0009990 tx=0x7f13f0009ca0 comp rx=0 tx=0).stop 2024-04-21T08:36:57.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:57.880+0000 7f14075da700 1 -- 172.21.15.86:0/3819372494 shutdown_connections 2024-04-21T08:36:57.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:57.880+0000 7f14075da700 1 --2- 172.21.15.86:0/3819372494 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1400102330 0x7f140010e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:57.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:57.880+0000 7f14075da700 1 --2- 172.21.15.86:0/3819372494 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1400107920 0x7f1400107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:57.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:57.880+0000 7f14075da700 1 --2- 172.21.15.86:0/3819372494 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1400101970 0x7f1400101df0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:57.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:57.880+0000 7f14075da700 1 -- 172.21.15.86:0/3819372494 >> 172.21.15.86:0/3819372494 conn(0x7f1400078910 msgr2=0x7f1400078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:57.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:57.880+0000 7f14075da700 1 -- 172.21.15.86:0/3819372494 shutdown_connections 2024-04-21T08:36:57.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:57.880+0000 7f14075da700 1 -- 172.21.15.86:0/3819372494 wait complete. 2024-04-21T08:36:57.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:57.880+0000 7f14075da700 1 Processor -- start 2024-04-21T08:36:57.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:57.880+0000 7f14075da700 1 -- start start 2024-04-21T08:36:57.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:57.881+0000 7f14075da700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1400101970 0x7f1400199f30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:57.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:57.881+0000 7f14075da700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1400102330 0x7f140019a470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:57.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:57.881+0000 7f14075da700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1400107920 0x7f14001a4fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:57.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:57.881+0000 7f14075da700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1400111b30 con 0x7f1400107920 2024-04-21T08:36:57.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:57.881+0000 7f14075da700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f14001119b0 con 0x7f1400101970 2024-04-21T08:36:57.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:57.881+0000 7f14075da700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1400111cb0 con 0x7f1400102330 2024-04-21T08:36:57.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:57.881+0000 7f1405b77700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1400107920 0x7f14001a4fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:57.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:57.881+0000 7f1405b77700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1400107920 0x7f14001a4fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59680/0 (socket says 172.21.15.86:59680) 2024-04-21T08:36:57.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:57.881+0000 7f1405b77700 1 -- 172.21.15.86:0/3532399390 learned_addr learned my addr 172.21.15.86:0/3532399390 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:36:57.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:57.881+0000 7f1405376700 1 --2- 172.21.15.86:0/3532399390 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1400101970 0x7f1400199f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:57.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:57.881+0000 7f1405b77700 1 -- 172.21.15.86:0/3532399390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1400102330 msgr2=0x7f140019a470 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T08:36:57.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:57.881+0000 7f1405b77700 1 --2- 172.21.15.86:0/3532399390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1400102330 0x7f140019a470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:57.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:57.881+0000 7f1405b77700 1 -- 172.21.15.86:0/3532399390 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1400101970 msgr2=0x7f1400199f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:57.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:57.881+0000 7f1405b77700 1 --2- 172.21.15.86:0/3532399390 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1400101970 0x7f1400199f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:57.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:57.881+0000 7f1405b77700 1 -- 172.21.15.86:0/3532399390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14001a54f0 con 0x7f1400107920 2024-04-21T08:36:57.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:57.881+0000 7f1405b77700 1 --2- 172.21.15.86:0/3532399390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1400107920 0x7f14001a4fb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f13f8007ee0 tx=0x7f13f800d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:57.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:57.881+0000 7f13f67fc700 1 -- 172.21.15.86:0/3532399390 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f800dbd0 con 0x7f1400107920 2024-04-21T08:36:57.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:57.881+0000 7f13f67fc700 1 -- 172.21.15.86:0/3532399390 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13f800dd70 con 0x7f1400107920 2024-04-21T08:36:57.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:57.881+0000 7f14075da700 1 -- 172.21.15.86:0/3532399390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14001a5780 con 0x7f1400107920 2024-04-21T08:36:57.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:57.881+0000 7f13f67fc700 1 -- 172.21.15.86:0/3532399390 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13f8005990 con 0x7f1400107920 2024-04-21T08:36:57.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:57.881+0000 7f14075da700 1 -- 172.21.15.86:0/3532399390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14001a5c90 con 0x7f1400107920 2024-04-21T08:36:57.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:57.882+0000 7f14075da700 1 -- 172.21.15.86:0/3532399390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1400103060 con 0x7f1400107920 2024-04-21T08:36:57.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:57.883+0000 7f13f67fc700 1 -- 172.21.15.86:0/3532399390 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f13f801a020 con 0x7f1400107920 2024-04-21T08:36:57.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:57.884+0000 7f13f67fc700 1 --2- 172.21.15.86:0/3532399390 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f13ec0a3d50 0x7f13ec0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:36:57.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:57.884+0000 7f1405376700 1 --2- 172.21.15.86:0/3532399390 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f13ec0a3d50 0x7f13ec0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:36:57.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:57.884+0000 7f13f67fc700 1 -- 172.21.15.86:0/3532399390 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f13f80d05e0 con 0x7f1400107920 2024-04-21T08:36:57.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:57.884+0000 7f1405376700 1 --2- 172.21.15.86:0/3532399390 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f13ec0a3d50 0x7f13ec0a6210 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f13fc005fd0 tx=0x7f13fc005ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:36:57.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:57.888+0000 7f13f67fc700 1 -- 172.21.15.86:0/3532399390 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f13f8010070 con 0x7f1400107920 2024-04-21T08:36:58.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:58.084+0000 7f14075da700 1 -- 172.21.15.86:0/3532399390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f140002df80 con 0x7f1400107920 2024-04-21T08:36:58.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:58.085+0000 7f13f67fc700 1 -- 172.21.15.86:0/3532399390 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24070 (secure 0 0 0) 0x7f13f809cc20 con 0x7f1400107920 2024-04-21T08:36:58.085 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:36:58.085 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":89,"fsid":"f8cec7cc-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:22.687329+0000","modified":"2024-04-21T08:36:25.251657+0000","last_up_change":"2024-04-21T08:35:30.048315+0000","last_in_change":"2024-04-21T08:35:10.099496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:45.279109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:18.300782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:19.574636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:22.905632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89b7601f-0620-4db2-90f1-b45cea173738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6803","nonce":2571516897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6805","nonce":2571516897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6809","nonce":2571516897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6807","nonce":2571516897}]},"public_addr":"172.21.15.86:6803/2571516897","cluster_addr":"172.21.15.86:6805/2571516897","heartbeat_back_addr":"172.21.15.86:6809/2571516897","heartbeat_front_addr":"172.21.15.86:6807/2571516897","state":["exists","up"]},{"osd":1,"uuid":"4891031c-f98c-42bc-bb3b-a420855ecc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6801","nonce":74067182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6803","nonce":74067182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6807","nonce":74067182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6805","nonce":74067182}]},"public_addr":"172.21.15.88:6801/74067182","cluster_addr":"172.21.15.88:6803/74067182","heartbeat_back_addr":"172.21.15.88:6807/74067182","heartbeat_front_addr":"172.21.15.88:6805/74067182","state":["exists","up"]},{"osd":2,"uuid":"60d12aa4-9312-4fa2-8d0b-2b3f42caad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6801","nonce":4137659051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6803","nonce":4137659051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6807","nonce":4137659051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6805","nonce":4137659051}]},"public_addr":"172.21.15.179:6801/4137659051","cluster_addr":"172.21.15.179:6803/4137659051","heartbeat_back_addr":"172.21.15.179:6807/4137659051","heartbeat_front_addr":"172.21.15.179:6805/4137659051","state":["exists","up"]},{"osd":3,"uuid":"404c3158-8cb0-4c02-b48a-f9ee5ff7b784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6811","nonce":2309603148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6813","nonce":2309603148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6817","nonce":2309603148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6815","nonce":2309603148}]},"public_addr":"172.21.15.86:6811/2309603148","cluster_addr":"172.21.15.86:6813/2309603148","heartbeat_back_addr":"172.21.15.86:6817/2309603148","heartbeat_front_addr":"172.21.15.86:6815/2309603148","state":["exists","up"]},{"osd":4,"uuid":"458ff202-d781-4ec7-ac93-0b49f835b503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6809","nonce":1818714217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6811","nonce":1818714217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6815","nonce":1818714217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6813","nonce":1818714217}]},"public_addr":"172.21.15.88:6809/1818714217","cluster_addr":"172.21.15.88:6811/1818714217","heartbeat_back_addr":"172.21.15.88:6815/1818714217","heartbeat_front_addr":"172.21.15.88:6813/1818714217","state":["exists","up"]},{"osd":5,"uuid":"8123cc90-bd8b-4204-bf31-46b34e9f589d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6809","nonce":863840990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6811","nonce":863840990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6815","nonce":863840990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6813","nonce":863840990}]},"public_addr":"172.21.15.179:6809/863840990","cluster_addr":"172.21.15.179:6811/863840990","heartbeat_back_addr":"172.21.15.179:6815/863840990","heartbeat_front_addr":"172.21.15.179:6813/863840990","state":["exists","up"]},{"osd":6,"uuid":"aac9d7c7-659a-4110-903a-a871c086066f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6819","nonce":1224231118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6821","nonce":1224231118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6825","nonce":1224231118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6823","nonce":1224231118}]},"public_addr":"172.21.15.86:6819/1224231118","cluster_addr":"172.21.15.86:6821/1224231118","heartbeat_back_addr":"172.21.15.86:6825/1224231118","heartbeat_front_addr":"172.21.15.86:6823/1224231118","state":["exists","up"]},{"osd":7,"uuid":"df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6817","nonce":136629878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6819","nonce":136629878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6823","nonce":136629878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6821","nonce":136629878}]},"public_addr":"172.21.15.88:6817/136629878","cluster_addr":"172.21.15.88:6819/136629878","heartbeat_back_addr":"172.21.15.88:6823/136629878","heartbeat_front_addr":"172.21.15.88:6821/136629878","state":["exists","up"]},{"osd":8,"uuid":"63e59f4b-0426-46db-92f7-c55dae61b5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6817","nonce":1770418957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6819","nonce":1770418957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6823","nonce":1770418957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6821","nonce":1770418957}]},"public_addr":"172.21.15.179:6817/1770418957","cluster_addr":"172.21.15.179:6819/1770418957","heartbeat_back_addr":"172.21.15.179:6823/1770418957","heartbeat_front_addr":"172.21.15.179:6821/1770418957","state":["exists","up"]},{"osd":9,"uuid":"5841bb36-ed6b-45d8-b857-eeab139c44e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6827","nonce":1108500274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6829","nonce":1108500274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6833","nonce":1108500274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6831","nonce":1108500274}]},"public_addr":"172.21.15.86:6827/1108500274","cluster_addr":"172.21.15.86:6829/1108500274","heartbeat_back_addr":"172.21.15.86:6833/1108500274","heartbeat_front_addr":"172.21.15.86:6831/1108500274","state":["exists","up"]},{"osd":10,"uuid":"f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6825","nonce":1294275353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6827","nonce":1294275353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6831","nonce":1294275353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6829","nonce":1294275353}]},"public_addr":"172.21.15.88:6825/1294275353","cluster_addr":"172.21.15.88:6827/1294275353","heartbeat_back_addr":"172.21.15.88:6831/1294275353","heartbeat_front_addr":"172.21.15.88:6829/1294275353","state":["exists","up"]},{"osd":11,"uuid":"e0d4cc76-e7a6-4a6b-909c-95488ccd5a36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6825","nonce":4007070233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6827","nonce":4007070233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6831","nonce":4007070233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6829","nonce":4007070233}]},"public_addr":"172.21.15.179:6825/4007070233","cluster_addr":"172.21.15.179:6827/4007070233","heartbeat_back_addr":"172.21.15.179:6831/4007070233","heartbeat_front_addr":"172.21.15.179:6829/4007070233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:55.210605+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:18.723090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:42.547979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:06.421622+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:30.858315+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:55.455313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.623369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:46.185111+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.103627+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:37.297545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:02.637431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:28.199292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6800/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/1412393225":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6800/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/1687852404":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2020707404":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2568130933":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/3952607648":"2024-04-22T08:28:57.384403+0000","172.21.15.86:6801/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/2756903426":"2024-04-22T08:29:23.190001+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:36:58.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:58.087+0000 7f14075da700 1 -- 172.21.15.86:0/3532399390 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f13ec0a3d50 msgr2=0x7f13ec0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:58.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:58.087+0000 7f14075da700 1 --2- 172.21.15.86:0/3532399390 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f13ec0a3d50 0x7f13ec0a6210 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f13fc005fd0 tx=0x7f13fc005ee0 comp rx=0 tx=0).stop 2024-04-21T08:36:58.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:58.087+0000 7f14075da700 1 -- 172.21.15.86:0/3532399390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1400107920 msgr2=0x7f14001a4fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:36:58.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:58.087+0000 7f14075da700 1 --2- 172.21.15.86:0/3532399390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1400107920 0x7f14001a4fb0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f13f8007ee0 tx=0x7f13f800d3b0 comp rx=0 tx=0).stop 2024-04-21T08:36:58.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:58.087+0000 7f14075da700 1 -- 172.21.15.86:0/3532399390 shutdown_connections 2024-04-21T08:36:58.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:58.087+0000 7f14075da700 1 --2- 172.21.15.86:0/3532399390 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f13ec0a3d50 0x7f13ec0a6210 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:58.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:58.087+0000 7f14075da700 1 --2- 172.21.15.86:0/3532399390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1400107920 0x7f14001a4fb0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:58.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:58.087+0000 7f14075da700 1 --2- 172.21.15.86:0/3532399390 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1400101970 0x7f1400199f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:58.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:58.087+0000 7f14075da700 1 --2- 172.21.15.86:0/3532399390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1400102330 0x7f140019a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:36:58.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:58.087+0000 7f14075da700 1 -- 172.21.15.86:0/3532399390 >> 172.21.15.86:0/3532399390 conn(0x7f1400078910 msgr2=0x7f14000ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:36:58.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:58.088+0000 7f14075da700 1 -- 172.21.15.86:0/3532399390 shutdown_connections 2024-04-21T08:36:58.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:36:58.088+0000 7f14075da700 1 -- 172.21.15.86:0/3532399390 wait complete. 2024-04-21T08:36:58.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:58 smithi179 ceph-mon[31829]: pgmap v306: 129 pgs: 129 active+clean; 629 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.4 KiB/s wr, 24 op/s 2024-04-21T08:36:58.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:58 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3532399390' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:58.775 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:58 smithi086 ceph-mon[27939]: pgmap v306: 129 pgs: 129 active+clean; 629 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.4 KiB/s wr, 24 op/s 2024-04-21T08:36:58.775 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:58 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3532399390' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:58.807 INFO:tasks.cephadm.ceph_manager.ceph:all up! 2024-04-21T08:36:58.808 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph osd dump --format=json 2024-04-21T08:36:58.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:58 smithi088 ceph-mon[31259]: pgmap v306: 129 pgs: 129 active+clean; 629 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 8.4 KiB/s wr, 24 op/s 2024-04-21T08:36:58.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:58 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3532399390' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:36:59.051 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:36:59.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:36:59 smithi086 ceph-mon[27939]: pgmap v307: 129 pgs: 129 active+clean; 629 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 KiB/s wr, 26 op/s 2024-04-21T08:36:59.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:36:59 smithi088 ceph-mon[31259]: pgmap v307: 129 pgs: 129 active+clean; 629 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 KiB/s wr, 26 op/s 2024-04-21T08:36:59.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:36:59 smithi179 ceph-mon[31829]: pgmap v307: 129 pgs: 129 active+clean; 629 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 8.4 KiB/s wr, 26 op/s 2024-04-21T08:37:00.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.671+0000 7fd90b3c3700 1 -- 172.21.15.86:0/2980408046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9041024f0 msgr2=0x7fd904102970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:00.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.671+0000 7fd90b3c3700 1 --2- 172.21.15.86:0/2980408046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9041024f0 0x7fd904102970 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fd8f40099c0 tx=0x7fd8f4009cd0 comp rx=0 tx=0).stop 2024-04-21T08:37:00.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.671+0000 7fd90b3c3700 1 -- 172.21.15.86:0/2980408046 shutdown_connections 2024-04-21T08:37:00.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.671+0000 7fd90b3c3700 1 --2- 172.21.15.86:0/2980408046 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd904102eb0 0x7fd90410e8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:00.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.671+0000 7fd90b3c3700 1 --2- 172.21.15.86:0/2980408046 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd904107c10 0x7fd904107ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:00.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.671+0000 7fd90b3c3700 1 --2- 172.21.15.86:0/2980408046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9041024f0 0x7fd904102970 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:00.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.671+0000 7fd90b3c3700 1 -- 172.21.15.86:0/2980408046 >> 172.21.15.86:0/2980408046 conn(0x7fd904076c60 msgr2=0x7fd904077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:00.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.672+0000 7fd90b3c3700 1 -- 172.21.15.86:0/2980408046 shutdown_connections 2024-04-21T08:37:00.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.672+0000 7fd90b3c3700 1 -- 172.21.15.86:0/2980408046 wait complete. 2024-04-21T08:37:00.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.672+0000 7fd90b3c3700 1 Processor -- start 2024-04-21T08:37:00.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.672+0000 7fd90b3c3700 1 -- start start 2024-04-21T08:37:00.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.672+0000 7fd90b3c3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9041024f0 0x7fd90419a020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:00.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.672+0000 7fd90b3c3700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd904102eb0 0x7fd90419a560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:00.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.672+0000 7fd90b3c3700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd904107c10 0x7fd9041a50a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:00.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.672+0000 7fd90b3c3700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd904111c60 con 0x7fd904102eb0 2024-04-21T08:37:00.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.672+0000 7fd90b3c3700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd904111ae0 con 0x7fd904107c10 2024-04-21T08:37:00.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.672+0000 7fd90b3c3700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd904111de0 con 0x7fd9041024f0 2024-04-21T08:37:00.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.673+0000 7fd90895e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd904102eb0 0x7fd90419a560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:00.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.673+0000 7fd90895e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd904102eb0 0x7fd90419a560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59404/0 (socket says 172.21.15.86:59404) 2024-04-21T08:37:00.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.673+0000 7fd90895e700 1 -- 172.21.15.86:0/2870901054 learned_addr learned my addr 172.21.15.86:0/2870901054 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:37:00.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.673+0000 7fd90915f700 1 --2- 172.21.15.86:0/2870901054 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9041024f0 0x7fd90419a020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:00.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.673+0000 7fd909960700 1 --2- 172.21.15.86:0/2870901054 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd904107c10 0x7fd9041a50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:00.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.673+0000 7fd90895e700 1 -- 172.21.15.86:0/2870901054 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9041024f0 msgr2=0x7fd90419a020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:00.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.673+0000 7fd90895e700 1 --2- 172.21.15.86:0/2870901054 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9041024f0 0x7fd90419a020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:00.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.673+0000 7fd90895e700 1 -- 172.21.15.86:0/2870901054 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd904107c10 msgr2=0x7fd9041a50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:00.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.673+0000 7fd90895e700 1 --2- 172.21.15.86:0/2870901054 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd904107c10 0x7fd9041a50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:00.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.673+0000 7fd90895e700 1 -- 172.21.15.86:0/2870901054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9041a55e0 con 0x7fd904102eb0 2024-04-21T08:37:00.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.673+0000 7fd90895e700 1 --2- 172.21.15.86:0/2870901054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd904102eb0 0x7fd90419a560 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4006010 tx=0x7fd8f40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:00.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.673+0000 7fd8fa7fc700 1 -- 172.21.15.86:0/2870901054 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4026070 con 0x7fd904102eb0 2024-04-21T08:37:00.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.673+0000 7fd90b3c3700 1 -- 172.21.15.86:0/2870901054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9041a57b0 con 0x7fd904102eb0 2024-04-21T08:37:00.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.673+0000 7fd8fa7fc700 1 -- 172.21.15.86:0/2870901054 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd8f400f460 con 0x7fd904102eb0 2024-04-21T08:37:00.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.673+0000 7fd8fa7fc700 1 -- 172.21.15.86:0/2870901054 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd8f4009e60 con 0x7fd904102eb0 2024-04-21T08:37:00.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.673+0000 7fd90b3c3700 1 -- 172.21.15.86:0/2870901054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9041a5c60 con 0x7fd904102eb0 2024-04-21T08:37:00.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.674+0000 7fd90b3c3700 1 -- 172.21.15.86:0/2870901054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd904069730 con 0x7fd904102eb0 2024-04-21T08:37:00.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.675+0000 7fd8fa7fc700 1 -- 172.21.15.86:0/2870901054 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fd8f4029070 con 0x7fd904102eb0 2024-04-21T08:37:00.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.676+0000 7fd8fa7fc700 1 --2- 172.21.15.86:0/2870901054 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fd8f00a3cd0 0x7fd8f00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:00.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.676+0000 7fd90915f700 1 --2- 172.21.15.86:0/2870901054 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fd8f00a3cd0 0x7fd8f00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:00.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.676+0000 7fd8fa7fc700 1 -- 172.21.15.86:0/2870901054 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fd8f40d1030 con 0x7fd904102eb0 2024-04-21T08:37:00.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.676+0000 7fd90915f700 1 --2- 172.21.15.86:0/2870901054 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fd8f00a3cd0 0x7fd8f00a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd900007b60 tx=0x7fd9000073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:00.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.681+0000 7fd8fa7fc700 1 -- 172.21.15.86:0/2870901054 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd8f405dff0 con 0x7fd904102eb0 2024-04-21T08:37:00.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.880+0000 7fd90b3c3700 1 -- 172.21.15.86:0/2870901054 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fd90402df80 con 0x7fd904102eb0 2024-04-21T08:37:00.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.882+0000 7fd8fa7fc700 1 -- 172.21.15.86:0/2870901054 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v89) ==== 74+0+24070 (secure 0 0 0) 0x7fd8f4013070 con 0x7fd904102eb0 2024-04-21T08:37:00.881 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:37:00.881 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":89,"fsid":"f8cec7cc-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:22.687329+0000","modified":"2024-04-21T08:36:25.251657+0000","last_up_change":"2024-04-21T08:35:30.048315+0000","last_in_change":"2024-04-21T08:35:10.099496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:45.279109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:18.300782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":2.25,"score_stable":2.25,"optimal_score":1,"raw_score_acting":2.25,"raw_score_stable":2.25,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:19.574636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:22.905632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89b7601f-0620-4db2-90f1-b45cea173738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6803","nonce":2571516897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6805","nonce":2571516897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6809","nonce":2571516897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6807","nonce":2571516897}]},"public_addr":"172.21.15.86:6803/2571516897","cluster_addr":"172.21.15.86:6805/2571516897","heartbeat_back_addr":"172.21.15.86:6809/2571516897","heartbeat_front_addr":"172.21.15.86:6807/2571516897","state":["exists","up"]},{"osd":1,"uuid":"4891031c-f98c-42bc-bb3b-a420855ecc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6801","nonce":74067182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6803","nonce":74067182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6807","nonce":74067182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6805","nonce":74067182}]},"public_addr":"172.21.15.88:6801/74067182","cluster_addr":"172.21.15.88:6803/74067182","heartbeat_back_addr":"172.21.15.88:6807/74067182","heartbeat_front_addr":"172.21.15.88:6805/74067182","state":["exists","up"]},{"osd":2,"uuid":"60d12aa4-9312-4fa2-8d0b-2b3f42caad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6801","nonce":4137659051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6803","nonce":4137659051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6807","nonce":4137659051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6805","nonce":4137659051}]},"public_addr":"172.21.15.179:6801/4137659051","cluster_addr":"172.21.15.179:6803/4137659051","heartbeat_back_addr":"172.21.15.179:6807/4137659051","heartbeat_front_addr":"172.21.15.179:6805/4137659051","state":["exists","up"]},{"osd":3,"uuid":"404c3158-8cb0-4c02-b48a-f9ee5ff7b784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6811","nonce":2309603148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6813","nonce":2309603148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6817","nonce":2309603148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6815","nonce":2309603148}]},"public_addr":"172.21.15.86:6811/2309603148","cluster_addr":"172.21.15.86:6813/2309603148","heartbeat_back_addr":"172.21.15.86:6817/2309603148","heartbeat_front_addr":"172.21.15.86:6815/2309603148","state":["exists","up"]},{"osd":4,"uuid":"458ff202-d781-4ec7-ac93-0b49f835b503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6809","nonce":1818714217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6811","nonce":1818714217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6815","nonce":1818714217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6813","nonce":1818714217}]},"public_addr":"172.21.15.88:6809/1818714217","cluster_addr":"172.21.15.88:6811/1818714217","heartbeat_back_addr":"172.21.15.88:6815/1818714217","heartbeat_front_addr":"172.21.15.88:6813/1818714217","state":["exists","up"]},{"osd":5,"uuid":"8123cc90-bd8b-4204-bf31-46b34e9f589d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6809","nonce":863840990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6811","nonce":863840990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6815","nonce":863840990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6813","nonce":863840990}]},"public_addr":"172.21.15.179:6809/863840990","cluster_addr":"172.21.15.179:6811/863840990","heartbeat_back_addr":"172.21.15.179:6815/863840990","heartbeat_front_addr":"172.21.15.179:6813/863840990","state":["exists","up"]},{"osd":6,"uuid":"aac9d7c7-659a-4110-903a-a871c086066f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6819","nonce":1224231118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6821","nonce":1224231118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6825","nonce":1224231118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6823","nonce":1224231118}]},"public_addr":"172.21.15.86:6819/1224231118","cluster_addr":"172.21.15.86:6821/1224231118","heartbeat_back_addr":"172.21.15.86:6825/1224231118","heartbeat_front_addr":"172.21.15.86:6823/1224231118","state":["exists","up"]},{"osd":7,"uuid":"df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6817","nonce":136629878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6819","nonce":136629878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6823","nonce":136629878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6821","nonce":136629878}]},"public_addr":"172.21.15.88:6817/136629878","cluster_addr":"172.21.15.88:6819/136629878","heartbeat_back_addr":"172.21.15.88:6823/136629878","heartbeat_front_addr":"172.21.15.88:6821/136629878","state":["exists","up"]},{"osd":8,"uuid":"63e59f4b-0426-46db-92f7-c55dae61b5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6817","nonce":1770418957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6819","nonce":1770418957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6823","nonce":1770418957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6821","nonce":1770418957}]},"public_addr":"172.21.15.179:6817/1770418957","cluster_addr":"172.21.15.179:6819/1770418957","heartbeat_back_addr":"172.21.15.179:6823/1770418957","heartbeat_front_addr":"172.21.15.179:6821/1770418957","state":["exists","up"]},{"osd":9,"uuid":"5841bb36-ed6b-45d8-b857-eeab139c44e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6827","nonce":1108500274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6829","nonce":1108500274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6833","nonce":1108500274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6831","nonce":1108500274}]},"public_addr":"172.21.15.86:6827/1108500274","cluster_addr":"172.21.15.86:6829/1108500274","heartbeat_back_addr":"172.21.15.86:6833/1108500274","heartbeat_front_addr":"172.21.15.86:6831/1108500274","state":["exists","up"]},{"osd":10,"uuid":"f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":83,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6825","nonce":1294275353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6827","nonce":1294275353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6831","nonce":1294275353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6829","nonce":1294275353}]},"public_addr":"172.21.15.88:6825/1294275353","cluster_addr":"172.21.15.88:6827/1294275353","heartbeat_back_addr":"172.21.15.88:6831/1294275353","heartbeat_front_addr":"172.21.15.88:6829/1294275353","state":["exists","up"]},{"osd":11,"uuid":"e0d4cc76-e7a6-4a6b-909c-95488ccd5a36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6825","nonce":4007070233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6827","nonce":4007070233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6831","nonce":4007070233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6829","nonce":4007070233}]},"public_addr":"172.21.15.179:6825/4007070233","cluster_addr":"172.21.15.179:6827/4007070233","heartbeat_back_addr":"172.21.15.179:6831/4007070233","heartbeat_front_addr":"172.21.15.179:6829/4007070233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:55.210605+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:18.723090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:42.547979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:06.421622+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:30.858315+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:55.455313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.623369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:46.185111+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.103627+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:37.297545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:02.637431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:28.199292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6800/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/1412393225":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6800/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/1687852404":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2020707404":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2568130933":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/3952607648":"2024-04-22T08:28:57.384403+0000","172.21.15.86:6801/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/2756903426":"2024-04-22T08:29:23.190001+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:37:00.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.884+0000 7fd90b3c3700 1 -- 172.21.15.86:0/2870901054 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fd8f00a3cd0 msgr2=0x7fd8f00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:00.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.884+0000 7fd90b3c3700 1 --2- 172.21.15.86:0/2870901054 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fd8f00a3cd0 0x7fd8f00a6190 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7fd900007b60 tx=0x7fd9000073d0 comp rx=0 tx=0).stop 2024-04-21T08:37:00.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.884+0000 7fd90b3c3700 1 -- 172.21.15.86:0/2870901054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd904102eb0 msgr2=0x7fd90419a560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:00.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.884+0000 7fd90b3c3700 1 --2- 172.21.15.86:0/2870901054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd904102eb0 0x7fd90419a560 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7fd8f4006010 tx=0x7fd8f40045e0 comp rx=0 tx=0).stop 2024-04-21T08:37:00.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.884+0000 7fd90b3c3700 1 -- 172.21.15.86:0/2870901054 shutdown_connections 2024-04-21T08:37:00.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.884+0000 7fd90b3c3700 1 --2- 172.21.15.86:0/2870901054 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fd8f00a3cd0 0x7fd8f00a6190 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:00.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.884+0000 7fd90b3c3700 1 --2- 172.21.15.86:0/2870901054 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd904107c10 0x7fd9041a50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:00.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.884+0000 7fd90b3c3700 1 --2- 172.21.15.86:0/2870901054 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9041024f0 0x7fd90419a020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:00.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.884+0000 7fd90b3c3700 1 --2- 172.21.15.86:0/2870901054 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd904102eb0 0x7fd90419a560 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:00.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.884+0000 7fd90b3c3700 1 -- 172.21.15.86:0/2870901054 >> 172.21.15.86:0/2870901054 conn(0x7fd904076c60 msgr2=0x7fd9040ff390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:00.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.885+0000 7fd90b3c3700 1 -- 172.21.15.86:0/2870901054 shutdown_connections 2024-04-21T08:37:00.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:00.885+0000 7fd90b3c3700 1 -- 172.21.15.86:0/2870901054 wait complete. 2024-04-21T08:37:01.213 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:00 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2870901054' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:37:01.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:00 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2870901054' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:37:01.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:00 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2870901054' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:37:01.705 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph tell osd.0 flush_pg_stats 2024-04-21T08:37:01.706 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph tell osd.1 flush_pg_stats 2024-04-21T08:37:01.706 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph tell osd.2 flush_pg_stats 2024-04-21T08:37:01.707 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph tell osd.3 flush_pg_stats 2024-04-21T08:37:01.707 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph tell osd.4 flush_pg_stats 2024-04-21T08:37:01.707 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph tell osd.5 flush_pg_stats 2024-04-21T08:37:01.708 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph tell osd.6 flush_pg_stats 2024-04-21T08:37:01.708 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph tell osd.7 flush_pg_stats 2024-04-21T08:37:01.709 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph tell osd.8 flush_pg_stats 2024-04-21T08:37:01.709 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph tell osd.9 flush_pg_stats 2024-04-21T08:37:01.709 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph tell osd.10 flush_pg_stats 2024-04-21T08:37:01.710 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph tell osd.11 flush_pg_stats 2024-04-21T08:37:02.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:01 smithi088 ceph-mon[31259]: pgmap v308: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 KiB/s wr, 18 op/s 2024-04-21T08:37:02.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:01 smithi179 ceph-mon[31829]: pgmap v308: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 KiB/s wr, 18 op/s 2024-04-21T08:37:02.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:01 smithi086 ceph-mon[27939]: pgmap v308: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 5.9 KiB/s wr, 18 op/s 2024-04-21T08:37:02.372 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:02.379 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:02.388 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:02.481 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:02.740 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:02.741 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:02.745 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:02.745 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:02.745 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:02.990 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:02.990 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:02.990 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:04.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:04 smithi086 ceph-mon[27939]: pgmap v309: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-04-21T08:37:04.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:04 smithi088 ceph-mon[31259]: pgmap v309: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-04-21T08:37:04.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:04 smithi179 ceph-mon[31829]: pgmap v309: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 1.8 KiB/s wr, 10 op/s 2024-04-21T08:37:06.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:06 smithi086 ceph-mon[27939]: pgmap v310: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-21T08:37:06.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:06 smithi088 ceph-mon[31259]: pgmap v310: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-21T08:37:06.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:06 smithi179 ceph-mon[31829]: pgmap v310: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.4 KiB/s rd, 1.8 KiB/s wr, 12 op/s 2024-04-21T08:37:08.482 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:08 smithi086 ceph-mon[27939]: pgmap v311: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-21T08:37:08.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:08 smithi088 ceph-mon[31259]: pgmap v311: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-21T08:37:08.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:08 smithi179 ceph-mon[31829]: pgmap v311: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-21T08:37:10.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:10 smithi086 ceph-mon[27939]: pgmap v312: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-21T08:37:10.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:10 smithi088 ceph-mon[31259]: pgmap v312: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-21T08:37:10.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:10 smithi179 ceph-mon[31829]: pgmap v312: 129 pgs: 129 active+clean; 641 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 1.2 KiB/s wr, 10 op/s 2024-04-21T08:37:11.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.494+0000 7f699065f700 1 -- 172.21.15.86:0/337579107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69881034b0 msgr2=0x7f698810f920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:11.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.494+0000 7f699065f700 1 --2- 172.21.15.86:0/337579107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69881034b0 0x7f698810f920 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f698000b0d0 tx=0x7f698000b3e0 comp rx=0 tx=0).stop 2024-04-21T08:37:11.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.494+0000 7f699065f700 1 -- 172.21.15.86:0/337579107 shutdown_connections 2024-04-21T08:37:11.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.494+0000 7f699065f700 1 --2- 172.21.15.86:0/337579107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69881034b0 0x7f698810f920 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:11.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.494+0000 7f699065f700 1 --2- 172.21.15.86:0/337579107 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6988107940 0x7f6988107d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:11.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.494+0000 7f699065f700 1 --2- 172.21.15.86:0/337579107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6988102af0 0x7f6988102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:11.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.494+0000 7f699065f700 1 -- 172.21.15.86:0/337579107 >> 172.21.15.86:0/337579107 conn(0x7f6988079ba0 msgr2=0x7f6988079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:11.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.494+0000 7f699065f700 1 -- 172.21.15.86:0/337579107 shutdown_connections 2024-04-21T08:37:11.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.494+0000 7f699065f700 1 -- 172.21.15.86:0/337579107 wait complete. 2024-04-21T08:37:11.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.494+0000 7f699065f700 1 Processor -- start 2024-04-21T08:37:11.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.494+0000 7f699065f700 1 -- start start 2024-04-21T08:37:11.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.495+0000 7f699065f700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6988102af0 0x7f6988074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:11.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.495+0000 7f699065f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69881034b0 0x7f6988075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:11.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.495+0000 7f699065f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6988107940 0x7f69880757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:11.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.495+0000 7f699065f700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6988112d30 con 0x7f6988102af0 2024-04-21T08:37:11.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.495+0000 7f699065f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6988112bb0 con 0x7f69881034b0 2024-04-21T08:37:11.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.495+0000 7f699065f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6988112eb0 con 0x7f6988107940 2024-04-21T08:37:11.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.495+0000 7f698e3fb700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6988102af0 0x7f6988074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:11.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.495+0000 7f698e3fb700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6988102af0 0x7f6988074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48788/0 (socket says 172.21.15.86:48788) 2024-04-21T08:37:11.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.495+0000 7f698e3fb700 1 -- 172.21.15.86:0/937923616 learned_addr learned my addr 172.21.15.86:0/937923616 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:37:11.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.495+0000 7f698dbfa700 1 --2- 172.21.15.86:0/937923616 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69881034b0 0x7f6988075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:11.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.495+0000 7f698ebfc700 1 --2- 172.21.15.86:0/937923616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6988107940 0x7f69880757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:11.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.495+0000 7f698e3fb700 1 -- 172.21.15.86:0/937923616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6988107940 msgr2=0x7f69880757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:11.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.495+0000 7f698e3fb700 1 --2- 172.21.15.86:0/937923616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6988107940 0x7f69880757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:11.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.495+0000 7f698e3fb700 1 -- 172.21.15.86:0/937923616 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69881034b0 msgr2=0x7f6988075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:11.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.495+0000 7f698e3fb700 1 --2- 172.21.15.86:0/937923616 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69881034b0 0x7f6988075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:11.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.495+0000 7f698e3fb700 1 -- 172.21.15.86:0/937923616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69881a7740 con 0x7f6988102af0 2024-04-21T08:37:11.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.495+0000 7f698ebfc700 1 --2- 172.21.15.86:0/937923616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6988107940 0x7f69880757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:37:11.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.495+0000 7f698dbfa700 1 --2- 172.21.15.86:0/937923616 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69881034b0 0x7f6988075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:37:11.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.495+0000 7f698e3fb700 1 --2- 172.21.15.86:0/937923616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6988102af0 0x7f6988074d40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f698400b530 tx=0x7f698400b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:11.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.495+0000 7f697f7fe700 1 -- 172.21.15.86:0/937923616 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6984015020 con 0x7f6988102af0 2024-04-21T08:37:11.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.495+0000 7f697f7fe700 1 -- 172.21.15.86:0/937923616 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6984011ce0 con 0x7f6988102af0 2024-04-21T08:37:11.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.495+0000 7f699065f700 1 -- 172.21.15.86:0/937923616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69881a7a30 con 0x7f6988102af0 2024-04-21T08:37:11.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.495+0000 7f697f7fe700 1 -- 172.21.15.86:0/937923616 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6984010d20 con 0x7f6988102af0 2024-04-21T08:37:11.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.496+0000 7f699065f700 1 -- 172.21.15.86:0/937923616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69881a7f70 con 0x7f6988102af0 2024-04-21T08:37:11.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.496+0000 7f699065f700 1 -- 172.21.15.86:0/937923616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f69880630d0 con 0x7f6988102af0 2024-04-21T08:37:11.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.498+0000 7f697f7fe700 1 -- 172.21.15.86:0/937923616 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f6984018020 con 0x7f6988102af0 2024-04-21T08:37:11.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.498+0000 7f697f7fe700 1 --2- 172.21.15.86:0/937923616 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f69740a3cd0 0x7f69740a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:11.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.498+0000 7f698dbfa700 1 --2- 172.21.15.86:0/937923616 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f69740a3cd0 0x7f69740a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:11.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.498+0000 7f697f7fe700 1 -- 172.21.15.86:0/937923616 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f69840d05d0 con 0x7f6988102af0 2024-04-21T08:37:11.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.498+0000 7f697f7fe700 1 --2- 172.21.15.86:0/937923616 >> [v2:172.21.15.86:6818/1224231118,v1:172.21.15.86:6819/1224231118] conn(0x7f69740ae990 0x7f69740b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:11.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.498+0000 7f697f7fe700 1 -- 172.21.15.86:0/937923616 --> [v2:172.21.15.86:6818/1224231118,v1:172.21.15.86:6819/1224231118] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f698401deb0 con 0x7f69740ae990 2024-04-21T08:37:11.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.498+0000 7f697f7fe700 1 -- 172.21.15.86:0/937923616 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f69840d0a20 con 0x7f6988102af0 2024-04-21T08:37:11.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.499+0000 7f698ebfc700 1 --2- 172.21.15.86:0/937923616 >> [v2:172.21.15.86:6818/1224231118,v1:172.21.15.86:6819/1224231118] conn(0x7f69740ae990 0x7f69740b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:11.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.499+0000 7f698dbfa700 1 --2- 172.21.15.86:0/937923616 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f69740a3cd0 0x7f69740a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f698807a1f0 tx=0x7f6978008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:11.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.499+0000 7f698ebfc700 1 --2- 172.21.15.86:0/937923616 >> [v2:172.21.15.86:6818/1224231118,v1:172.21.15.86:6819/1224231118] conn(0x7f69740ae990 0x7f69740b0dd0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.6 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:11.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.500+0000 7f697f7fe700 1 -- 172.21.15.86:0/937923616 <== osd.6 v2:172.21.15.86:6818/1224231118 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f698401deb0 con 0x7f69740ae990 2024-04-21T08:37:11.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.516+0000 7f699065f700 1 -- 172.21.15.86:0/937923616 --> [v2:172.21.15.86:6818/1224231118,v1:172.21.15.86:6819/1224231118] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f6988063550 con 0x7f69740ae990 2024-04-21T08:37:11.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.517+0000 7f697f7fe700 1 -- 172.21.15.86:0/937923616 <== osd.6 v2:172.21.15.86:6818/1224231118 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f6988063550 con 0x7f69740ae990 2024-04-21T08:37:11.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.517+0000 7f699065f700 1 -- 172.21.15.86:0/937923616 >> [v2:172.21.15.86:6818/1224231118,v1:172.21.15.86:6819/1224231118] conn(0x7f69740ae990 msgr2=0x7f69740b0dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:11.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.517+0000 7f699065f700 1 --2- 172.21.15.86:0/937923616 >> [v2:172.21.15.86:6818/1224231118,v1:172.21.15.86:6819/1224231118] conn(0x7f69740ae990 0x7f69740b0dd0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:11.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.517+0000 7f699065f700 1 -- 172.21.15.86:0/937923616 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f69740a3cd0 msgr2=0x7f69740a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:11.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.517+0000 7f699065f700 1 --2- 172.21.15.86:0/937923616 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f69740a3cd0 0x7f69740a6190 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f698807a1f0 tx=0x7f6978008040 comp rx=0 tx=0).stop 2024-04-21T08:37:11.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.517+0000 7f699065f700 1 -- 172.21.15.86:0/937923616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6988102af0 msgr2=0x7f6988074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:11.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.517+0000 7f699065f700 1 --2- 172.21.15.86:0/937923616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6988102af0 0x7f6988074d40 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f698400b530 tx=0x7f698400b8f0 comp rx=0 tx=0).stop 2024-04-21T08:37:11.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.517+0000 7f698ebfc700 1 -- 172.21.15.86:0/937923616 reap_dead start 2024-04-21T08:37:11.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.517+0000 7f699065f700 1 -- 172.21.15.86:0/937923616 shutdown_connections 2024-04-21T08:37:11.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.517+0000 7f699065f700 1 -- 172.21.15.86:0/937923616 >> 172.21.15.86:0/937923616 conn(0x7f6988079ba0 msgr2=0x7f6988100be0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:11.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.517+0000 7f699065f700 1 -- 172.21.15.86:0/937923616 shutdown_connections 2024-04-21T08:37:11.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:11.517+0000 7f699065f700 1 -- 172.21.15.86:0/937923616 wait complete. 2024-04-21T08:37:12.598 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:12 smithi086 ceph-mon[27939]: pgmap v313: 129 pgs: 129 active+clean; 644 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-21T08:37:12.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:12 smithi088 ceph-mon[31259]: pgmap v313: 129 pgs: 129 active+clean; 644 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-21T08:37:12.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:12 smithi179 ceph-mon[31829]: pgmap v313: 129 pgs: 129 active+clean; 644 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 1.7 KiB/s wr, 11 op/s 2024-04-21T08:37:13.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.501+0000 7ffb7f071700 1 -- 172.21.15.86:0/1770186665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb7806a750 msgr2=0x7ffb7806abb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:13.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.501+0000 7ffb7f071700 1 --2- 172.21.15.86:0/1770186665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb7806a750 0x7ffb7806abb0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7ffb64009990 tx=0x7ffb64009ca0 comp rx=0 tx=0).stop 2024-04-21T08:37:13.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.501+0000 7ffb7f071700 1 -- 172.21.15.86:0/1770186665 shutdown_connections 2024-04-21T08:37:13.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.501+0000 7ffb7f071700 1 --2- 172.21.15.86:0/1770186665 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb7806b0f0 0x7ffb7810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:13.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.501+0000 7ffb7f071700 1 --2- 172.21.15.86:0/1770186665 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb78108a30 0x7ffb78108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:13.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.501+0000 7ffb7f071700 1 --2- 172.21.15.86:0/1770186665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb7806a750 0x7ffb7806abb0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:13.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.501+0000 7ffb7f071700 1 -- 172.21.15.86:0/1770186665 >> 172.21.15.86:0/1770186665 conn(0x7ffb780788f0 msgr2=0x7ffb78078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:13.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.501+0000 7ffb7f071700 1 -- 172.21.15.86:0/1770186665 shutdown_connections 2024-04-21T08:37:13.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.501+0000 7ffb7f071700 1 -- 172.21.15.86:0/1770186665 wait complete. 2024-04-21T08:37:13.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.502+0000 7ffb7f071700 1 Processor -- start 2024-04-21T08:37:13.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.502+0000 7ffb7f071700 1 -- start start 2024-04-21T08:37:13.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.502+0000 7ffb7f071700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb7806a750 0x7ffb781991e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:13.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.502+0000 7ffb7f071700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb7806b0f0 0x7ffb78199720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:13.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.502+0000 7ffb7f071700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb78108a30 0x7ffb7819da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:13.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.502+0000 7ffb7f071700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffb78112c30 con 0x7ffb7806b0f0 2024-04-21T08:37:13.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.502+0000 7ffb7f071700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ffb78112ab0 con 0x7ffb7806a750 2024-04-21T08:37:13.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.502+0000 7ffb7f071700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ffb78112db0 con 0x7ffb78108a30 2024-04-21T08:37:13.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.502+0000 7ffb77fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb7806b0f0 0x7ffb78199720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:13.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.502+0000 7ffb77fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb7806b0f0 0x7ffb78199720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48808/0 (socket says 172.21.15.86:48808) 2024-04-21T08:37:13.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.502+0000 7ffb77fff700 1 -- 172.21.15.86:0/1662585791 learned_addr learned my addr 172.21.15.86:0/1662585791 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:37:13.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.502+0000 7ffb7ce0d700 1 --2- 172.21.15.86:0/1662585791 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb7806a750 0x7ffb781991e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:13.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.502+0000 7ffb7d60e700 1 --2- 172.21.15.86:0/1662585791 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb78108a30 0x7ffb7819da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:13.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.502+0000 7ffb77fff700 1 -- 172.21.15.86:0/1662585791 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb78108a30 msgr2=0x7ffb7819da60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:13.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.502+0000 7ffb77fff700 1 --2- 172.21.15.86:0/1662585791 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb78108a30 0x7ffb7819da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:13.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.502+0000 7ffb77fff700 1 -- 172.21.15.86:0/1662585791 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb7806a750 msgr2=0x7ffb781991e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:13.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.502+0000 7ffb77fff700 1 --2- 172.21.15.86:0/1662585791 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb7806a750 0x7ffb781991e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:13.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.502+0000 7ffb77fff700 1 -- 172.21.15.86:0/1662585791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb7819e1a0 con 0x7ffb7806b0f0 2024-04-21T08:37:13.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.502+0000 7ffb7ce0d700 1 --2- 172.21.15.86:0/1662585791 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb7806a750 0x7ffb781991e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:37:13.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.502+0000 7ffb77fff700 1 --2- 172.21.15.86:0/1662585791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb7806b0f0 0x7ffb78199720 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ffb64000c00 tx=0x7ffb640047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:13.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.503+0000 7ffb75ffb700 1 -- 172.21.15.86:0/1662585791 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb64027070 con 0x7ffb7806b0f0 2024-04-21T08:37:13.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.503+0000 7ffb7f071700 1 -- 172.21.15.86:0/1662585791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb7819e430 con 0x7ffb7806b0f0 2024-04-21T08:37:13.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.503+0000 7ffb75ffb700 1 -- 172.21.15.86:0/1662585791 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb64003fa0 con 0x7ffb7806b0f0 2024-04-21T08:37:13.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.503+0000 7ffb75ffb700 1 -- 172.21.15.86:0/1662585791 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb64022db0 con 0x7ffb7806b0f0 2024-04-21T08:37:13.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.503+0000 7ffb7f071700 1 -- 172.21.15.86:0/1662585791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb781a5d10 con 0x7ffb7806b0f0 2024-04-21T08:37:13.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.504+0000 7ffb7f071700 1 -- 172.21.15.86:0/1662585791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffb780631d0 con 0x7ffb7806b0f0 2024-04-21T08:37:13.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.505+0000 7ffb75ffb700 1 -- 172.21.15.86:0/1662585791 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ffb6402a070 con 0x7ffb7806b0f0 2024-04-21T08:37:13.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.505+0000 7ffb75ffb700 1 --2- 172.21.15.86:0/1662585791 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ffb680a3d50 0x7ffb680a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:13.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.506+0000 7ffb7ce0d700 1 --2- 172.21.15.86:0/1662585791 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ffb680a3d50 0x7ffb680a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:13.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.506+0000 7ffb75ffb700 1 -- 172.21.15.86:0/1662585791 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7ffb640d1fa0 con 0x7ffb7806b0f0 2024-04-21T08:37:13.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.506+0000 7ffb75ffb700 1 --2- 172.21.15.86:0/1662585791 >> [v2:172.21.15.86:6826/1108500274,v1:172.21.15.86:6827/1108500274] conn(0x7ffb680aea10 0x7ffb680b0e50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:13.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.506+0000 7ffb75ffb700 1 -- 172.21.15.86:0/1662585791 --> [v2:172.21.15.86:6826/1108500274,v1:172.21.15.86:6827/1108500274] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffb640043d0 con 0x7ffb680aea10 2024-04-21T08:37:13.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.506+0000 7ffb75ffb700 1 -- 172.21.15.86:0/1662585791 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7ffb640d23f0 con 0x7ffb7806b0f0 2024-04-21T08:37:13.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.506+0000 7ffb7d60e700 1 --2- 172.21.15.86:0/1662585791 >> [v2:172.21.15.86:6826/1108500274,v1:172.21.15.86:6827/1108500274] conn(0x7ffb680aea10 0x7ffb680b0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:13.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.506+0000 7ffb7ce0d700 1 --2- 172.21.15.86:0/1662585791 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ffb680a3d50 0x7ffb680a6210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c007900 tx=0x7ffb6c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:13.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.506+0000 7ffb7d60e700 1 --2- 172.21.15.86:0/1662585791 >> [v2:172.21.15.86:6826/1108500274,v1:172.21.15.86:6827/1108500274] conn(0x7ffb680aea10 0x7ffb680b0e50 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.9 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:13.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.507+0000 7ffb75ffb700 1 -- 172.21.15.86:0/1662585791 <== osd.9 v2:172.21.15.86:6826/1108500274 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffb640043d0 con 0x7ffb680aea10 2024-04-21T08:37:13.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.523+0000 7ffb7f071700 1 -- 172.21.15.86:0/1662585791 --> [v2:172.21.15.86:6826/1108500274,v1:172.21.15.86:6827/1108500274] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffb780635d0 con 0x7ffb680aea10 2024-04-21T08:37:13.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.523+0000 7ffb75ffb700 1 -- 172.21.15.86:0/1662585791 <== osd.9 v2:172.21.15.86:6826/1108500274 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ffb780635d0 con 0x7ffb680aea10 2024-04-21T08:37:13.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.524+0000 7ffb7f071700 1 -- 172.21.15.86:0/1662585791 >> [v2:172.21.15.86:6826/1108500274,v1:172.21.15.86:6827/1108500274] conn(0x7ffb680aea10 msgr2=0x7ffb680b0e50 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:13.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.524+0000 7ffb7f071700 1 --2- 172.21.15.86:0/1662585791 >> [v2:172.21.15.86:6826/1108500274,v1:172.21.15.86:6827/1108500274] conn(0x7ffb680aea10 0x7ffb680b0e50 crc :-1 s=READY pgs=13 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:13.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.524+0000 7ffb7f071700 1 -- 172.21.15.86:0/1662585791 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ffb680a3d50 msgr2=0x7ffb680a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:13.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.524+0000 7ffb7f071700 1 --2- 172.21.15.86:0/1662585791 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ffb680a3d50 0x7ffb680a6210 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7ffb6c007900 tx=0x7ffb6c008040 comp rx=0 tx=0).stop 2024-04-21T08:37:13.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.524+0000 7ffb7f071700 1 -- 172.21.15.86:0/1662585791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb7806b0f0 msgr2=0x7ffb78199720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:13.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.524+0000 7ffb7f071700 1 --2- 172.21.15.86:0/1662585791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb7806b0f0 0x7ffb78199720 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7ffb64000c00 tx=0x7ffb640047b0 comp rx=0 tx=0).stop 2024-04-21T08:37:13.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.524+0000 7ffb7d60e700 1 -- 172.21.15.86:0/1662585791 reap_dead start 2024-04-21T08:37:13.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.524+0000 7ffb7f071700 1 -- 172.21.15.86:0/1662585791 shutdown_connections 2024-04-21T08:37:13.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.524+0000 7ffb7f071700 1 -- 172.21.15.86:0/1662585791 >> 172.21.15.86:0/1662585791 conn(0x7ffb780788f0 msgr2=0x7ffb780ffa30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:13.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.524+0000 7ffb7f071700 1 -- 172.21.15.86:0/1662585791 shutdown_connections 2024-04-21T08:37:13.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.524+0000 7ffb7f071700 1 -- 172.21.15.86:0/1662585791 wait complete. 2024-04-21T08:37:13.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.770+0000 7f9ef5af8700 1 -- 172.21.15.86:0/2141314399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ef006b0f0 msgr2=0x7f9ef010f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:13.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.770+0000 7f9ef5af8700 1 --2- 172.21.15.86:0/2141314399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ef006b0f0 0x7f9ef010f900 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f9ee800b0a0 tx=0x7f9ee800b3b0 comp rx=0 tx=0).stop 2024-04-21T08:37:13.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.770+0000 7f9ef5af8700 1 -- 172.21.15.86:0/2141314399 shutdown_connections 2024-04-21T08:37:13.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.770+0000 7f9ef5af8700 1 --2- 172.21.15.86:0/2141314399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ef006b0f0 0x7f9ef010f900 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:13.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.770+0000 7f9ef5af8700 1 --2- 172.21.15.86:0/2141314399 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ef0108a30 0x7f9ef0108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:13.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.770+0000 7f9ef5af8700 1 --2- 172.21.15.86:0/2141314399 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ef006a750 0x7f9ef006abb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:13.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.770+0000 7f9ef5af8700 1 -- 172.21.15.86:0/2141314399 >> 172.21.15.86:0/2141314399 conn(0x7f9ef00788f0 msgr2=0x7f9ef0078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:13.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.770+0000 7f9ef5af8700 1 -- 172.21.15.86:0/2141314399 shutdown_connections 2024-04-21T08:37:13.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.770+0000 7f9ef5af8700 1 -- 172.21.15.86:0/2141314399 wait complete. 2024-04-21T08:37:13.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.771+0000 7f9ef5af8700 1 Processor -- start 2024-04-21T08:37:13.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.771+0000 7f9ef5af8700 1 -- start start 2024-04-21T08:37:13.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.771+0000 7f9ef5af8700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ef006a750 0x7f9ef0101a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:13.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.771+0000 7f9ef5af8700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ef006b0f0 0x7f9ef0101f50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:13.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.771+0000 7f9ef5af8700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ef0108a30 0x7f9ef0102490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:13.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.771+0000 7f9ef5af8700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef0112bd0 con 0x7f9ef006b0f0 2024-04-21T08:37:13.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.771+0000 7f9ef5af8700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef0112a50 con 0x7f9ef0108a30 2024-04-21T08:37:13.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.771+0000 7f9ef5af8700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9ef0112d50 con 0x7f9ef006a750 2024-04-21T08:37:13.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.771+0000 7f9ee6ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ef006b0f0 0x7f9ef0101f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:13.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.771+0000 7f9ee6ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ef006b0f0 0x7f9ef0101f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48830/0 (socket says 172.21.15.86:48830) 2024-04-21T08:37:13.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.771+0000 7f9ee6ffd700 1 -- 172.21.15.86:0/3862860140 learned_addr learned my addr 172.21.15.86:0/3862860140 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:37:13.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.771+0000 7f9eef7fe700 1 --2- 172.21.15.86:0/3862860140 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ef006a750 0x7f9ef0101a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:13.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.771+0000 7f9eeffff700 1 --2- 172.21.15.86:0/3862860140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ef0108a30 0x7f9ef0102490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:13.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.771+0000 7f9ee6ffd700 1 -- 172.21.15.86:0/3862860140 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ef006a750 msgr2=0x7f9ef0101a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:13.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.771+0000 7f9ee6ffd700 1 --2- 172.21.15.86:0/3862860140 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ef006a750 0x7f9ef0101a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:13.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.772+0000 7f9ee6ffd700 1 -- 172.21.15.86:0/3862860140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ef0108a30 msgr2=0x7f9ef0102490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:13.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.772+0000 7f9ee6ffd700 1 --2- 172.21.15.86:0/3862860140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ef0108a30 0x7f9ef0102490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:13.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.772+0000 7f9ee6ffd700 1 -- 172.21.15.86:0/3862860140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ef01a7610 con 0x7f9ef006b0f0 2024-04-21T08:37:13.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.772+0000 7f9ee6ffd700 1 --2- 172.21.15.86:0/3862860140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ef006b0f0 0x7f9ef0101f50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9ed800d980 tx=0x7f9ed800dc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:13.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.772+0000 7f9eed7fa700 1 -- 172.21.15.86:0/3862860140 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed8016070 con 0x7f9ef006b0f0 2024-04-21T08:37:13.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.772+0000 7f9eed7fa700 1 -- 172.21.15.86:0/3862860140 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ed8004830 con 0x7f9ef006b0f0 2024-04-21T08:37:13.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.772+0000 7f9ef5af8700 1 -- 172.21.15.86:0/3862860140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ef01a7900 con 0x7f9ef006b0f0 2024-04-21T08:37:13.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.772+0000 7f9eed7fa700 1 -- 172.21.15.86:0/3862860140 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ed8005540 con 0x7f9ef006b0f0 2024-04-21T08:37:13.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.772+0000 7f9ef5af8700 1 -- 172.21.15.86:0/3862860140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ef01a7e40 con 0x7f9ef006b0f0 2024-04-21T08:37:13.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.773+0000 7f9ef5af8700 1 -- 172.21.15.86:0/3862860140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f9ef0063110 con 0x7f9ef006b0f0 2024-04-21T08:37:13.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.774+0000 7f9eed7fa700 1 -- 172.21.15.86:0/3862860140 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9ed8004220 con 0x7f9ef006b0f0 2024-04-21T08:37:13.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.775+0000 7f9eed7fa700 1 --2- 172.21.15.86:0/3862860140 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f9edc0a3cd0 0x7f9edc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:13.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.775+0000 7f9eef7fe700 1 --2- 172.21.15.86:0/3862860140 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f9edc0a3cd0 0x7f9edc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:13.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.775+0000 7f9eed7fa700 1 -- 172.21.15.86:0/3862860140 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f9ed80d0ba0 con 0x7f9ef006b0f0 2024-04-21T08:37:13.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.775+0000 7f9eed7fa700 1 --2- 172.21.15.86:0/3862860140 >> [v2:172.21.15.88:6800/74067182,v1:172.21.15.88:6801/74067182] conn(0x7f9edc0ae990 0x7f9edc0b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:13.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.775+0000 7f9eed7fa700 1 -- 172.21.15.86:0/3862860140 --> [v2:172.21.15.88:6800/74067182,v1:172.21.15.88:6801/74067182] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f9edc0b14a0 con 0x7f9edc0ae990 2024-04-21T08:37:13.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.775+0000 7f9eed7fa700 1 -- 172.21.15.86:0/3862860140 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f9ed80d0ff0 con 0x7f9ef006b0f0 2024-04-21T08:37:13.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.775+0000 7f9eef7fe700 1 --2- 172.21.15.86:0/3862860140 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f9edc0a3cd0 0x7f9edc0a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0009be0 tx=0x7f9ee0009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:13.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.775+0000 7f9eeffff700 1 --2- 172.21.15.86:0/3862860140 >> [v2:172.21.15.88:6800/74067182,v1:172.21.15.88:6801/74067182] conn(0x7f9edc0ae990 0x7f9edc0b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:13.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.776+0000 7f9eeffff700 1 --2- 172.21.15.86:0/3862860140 >> [v2:172.21.15.88:6800/74067182,v1:172.21.15.88:6801/74067182] conn(0x7f9edc0ae990 0x7f9edc0b0dd0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:13.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.777+0000 7f9eed7fa700 1 -- 172.21.15.86:0/3862860140 <== osd.1 v2:172.21.15.88:6800/74067182 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f9edc0b14a0 con 0x7f9edc0ae990 2024-04-21T08:37:13.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.793+0000 7f9ef5af8700 1 -- 172.21.15.86:0/3862860140 --> [v2:172.21.15.88:6800/74067182,v1:172.21.15.88:6801/74067182] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f9ef0063590 con 0x7f9edc0ae990 2024-04-21T08:37:13.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.794+0000 7f9eed7fa700 1 -- 172.21.15.86:0/3862860140 <== osd.1 v2:172.21.15.88:6800/74067182 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7f9ef0063590 con 0x7f9edc0ae990 2024-04-21T08:37:13.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.794+0000 7f9ef5af8700 1 -- 172.21.15.86:0/3862860140 >> [v2:172.21.15.88:6800/74067182,v1:172.21.15.88:6801/74067182] conn(0x7f9edc0ae990 msgr2=0x7f9edc0b0dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:13.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.794+0000 7f9ef5af8700 1 --2- 172.21.15.86:0/3862860140 >> [v2:172.21.15.88:6800/74067182,v1:172.21.15.88:6801/74067182] conn(0x7f9edc0ae990 0x7f9edc0b0dd0 crc :-1 s=READY pgs=11 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:13.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.794+0000 7f9ef5af8700 1 -- 172.21.15.86:0/3862860140 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f9edc0a3cd0 msgr2=0x7f9edc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:13.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.794+0000 7f9ef5af8700 1 --2- 172.21.15.86:0/3862860140 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f9edc0a3cd0 0x7f9edc0a6190 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f9ee0009be0 tx=0x7f9ee0009260 comp rx=0 tx=0).stop 2024-04-21T08:37:13.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.794+0000 7f9ef5af8700 1 -- 172.21.15.86:0/3862860140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ef006b0f0 msgr2=0x7f9ef0101f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:13.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.794+0000 7f9ef5af8700 1 --2- 172.21.15.86:0/3862860140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ef006b0f0 0x7f9ef0101f50 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7f9ed800d980 tx=0x7f9ed800dc90 comp rx=0 tx=0).stop 2024-04-21T08:37:13.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.794+0000 7f9eeffff700 1 -- 172.21.15.86:0/3862860140 reap_dead start 2024-04-21T08:37:13.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.794+0000 7f9ef5af8700 1 -- 172.21.15.86:0/3862860140 shutdown_connections 2024-04-21T08:37:13.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.794+0000 7f9ef5af8700 1 -- 172.21.15.86:0/3862860140 >> 172.21.15.86:0/3862860140 conn(0x7f9ef00788f0 msgr2=0x7f9ef00ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:13.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.795+0000 7f9ef5af8700 1 -- 172.21.15.86:0/3862860140 shutdown_connections 2024-04-21T08:37:13.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:13.795+0000 7f9ef5af8700 1 -- 172.21.15.86:0/3862860140 wait complete. 2024-04-21T08:37:14.511 INFO:teuthology.orchestra.run.smithi086.stdout:197568495665 2024-04-21T08:37:14.511 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.6 2024-04-21T08:37:14.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:14 smithi086 ceph-mon[27939]: pgmap v314: 129 pgs: 129 active+clean; 644 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-21T08:37:14.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:14 smithi088 ceph-mon[31259]: pgmap v314: 129 pgs: 129 active+clean; 644 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-21T08:37:14.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:14 smithi179 ceph-mon[31829]: pgmap v314: 129 pgs: 129 active+clean; 644 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 9 op/s 2024-04-21T08:37:15.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.030+0000 7fcd205eb700 1 -- 172.21.15.86:0/1249512920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd18101940 msgr2=0x7fcd18101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:15.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.030+0000 7fcd205eb700 1 --2- 172.21.15.86:0/1249512920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd18101940 0x7fcd18101dc0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7fcd08009990 tx=0x7fcd08009ca0 comp rx=0 tx=0).stop 2024-04-21T08:37:15.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.030+0000 7fcd205eb700 1 -- 172.21.15.86:0/1249512920 shutdown_connections 2024-04-21T08:37:15.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.030+0000 7fcd205eb700 1 --2- 172.21.15.86:0/1249512920 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd18102300 0x7fcd1810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:15.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.030+0000 7fcd205eb700 1 --2- 172.21.15.86:0/1249512920 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd18107990 0x7fcd18107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:15.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.030+0000 7fcd205eb700 1 --2- 172.21.15.86:0/1249512920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd18101940 0x7fcd18101dc0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:15.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.030+0000 7fcd205eb700 1 -- 172.21.15.86:0/1249512920 >> 172.21.15.86:0/1249512920 conn(0x7fcd180788f0 msgr2=0x7fcd18078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:15.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.030+0000 7fcd205eb700 1 -- 172.21.15.86:0/1249512920 shutdown_connections 2024-04-21T08:37:15.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.030+0000 7fcd205eb700 1 -- 172.21.15.86:0/1249512920 wait complete. 2024-04-21T08:37:15.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.031+0000 7fcd205eb700 1 Processor -- start 2024-04-21T08:37:15.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.031+0000 7fcd205eb700 1 -- start start 2024-04-21T08:37:15.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.031+0000 7fcd205eb700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd18101940 0x7fcd18198090 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:15.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.031+0000 7fcd205eb700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd18102300 0x7fcd181985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:15.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.031+0000 7fcd205eb700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd18107990 0x7fcd1819c960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:15.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.031+0000 7fcd205eb700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcd18111b60 con 0x7fcd18102300 2024-04-21T08:37:15.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.031+0000 7fcd205eb700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcd181119e0 con 0x7fcd18107990 2024-04-21T08:37:15.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.031+0000 7fcd205eb700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fcd18111ce0 con 0x7fcd18101940 2024-04-21T08:37:15.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.031+0000 7fcd1db86700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd18102300 0x7fcd181985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:15.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.031+0000 7fcd1db86700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd18102300 0x7fcd181985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48840/0 (socket says 172.21.15.86:48840) 2024-04-21T08:37:15.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.031+0000 7fcd1db86700 1 -- 172.21.15.86:0/2511217872 learned_addr learned my addr 172.21.15.86:0/2511217872 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:37:15.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.031+0000 7fcd1eb88700 1 --2- 172.21.15.86:0/2511217872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd18107990 0x7fcd1819c960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:15.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.031+0000 7fcd1e387700 1 --2- 172.21.15.86:0/2511217872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd18101940 0x7fcd18198090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:15.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.031+0000 7fcd1db86700 1 -- 172.21.15.86:0/2511217872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd18101940 msgr2=0x7fcd18198090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:15.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.031+0000 7fcd1db86700 1 --2- 172.21.15.86:0/2511217872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcd18101940 0x7fcd18198090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:15.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.031+0000 7fcd1db86700 1 -- 172.21.15.86:0/2511217872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd18107990 msgr2=0x7fcd1819c960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:15.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.032+0000 7fcd1db86700 1 --2- 172.21.15.86:0/2511217872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcd18107990 0x7fcd1819c960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:15.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.032+0000 7fcd1db86700 1 -- 172.21.15.86:0/2511217872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcd1819d0a0 con 0x7fcd18102300 2024-04-21T08:37:15.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.032+0000 7fcd1db86700 1 --2- 172.21.15.86:0/2511217872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd18102300 0x7fcd181985d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fcd08000c00 tx=0x7fcd080047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:15.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.032+0000 7fcd0f7fe700 1 -- 172.21.15.86:0/2511217872 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd08027070 con 0x7fcd18102300 2024-04-21T08:37:15.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.032+0000 7fcd205eb700 1 -- 172.21.15.86:0/2511217872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcd1819d330 con 0x7fcd18102300 2024-04-21T08:37:15.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.032+0000 7fcd0f7fe700 1 -- 172.21.15.86:0/2511217872 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcd08003f20 con 0x7fcd18102300 2024-04-21T08:37:15.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.032+0000 7fcd0f7fe700 1 -- 172.21.15.86:0/2511217872 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcd08022cf0 con 0x7fcd18102300 2024-04-21T08:37:15.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.032+0000 7fcd205eb700 1 -- 172.21.15.86:0/2511217872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcd181a4c10 con 0x7fcd18102300 2024-04-21T08:37:15.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.033+0000 7fcd0d7fa700 1 -- 172.21.15.86:0/2511217872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fcd00001030 con 0x7fcd18102300 2024-04-21T08:37:15.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.034+0000 7fcd0f7fe700 1 -- 172.21.15.86:0/2511217872 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fcd0802a070 con 0x7fcd18102300 2024-04-21T08:37:15.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.034+0000 7fcd0f7fe700 1 --2- 172.21.15.86:0/2511217872 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fcd040a3cd0 0x7fcd040a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:15.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.035+0000 7fcd0f7fe700 1 -- 172.21.15.86:0/2511217872 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fcd080d1b70 con 0x7fcd18102300 2024-04-21T08:37:15.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.035+0000 7fcd1e387700 1 --2- 172.21.15.86:0/2511217872 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fcd040a3cd0 0x7fcd040a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:15.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.035+0000 7fcd0f7fe700 1 --2- 172.21.15.86:0/2511217872 >> [v2:172.21.15.86:6802/2571516897,v1:172.21.15.86:6803/2571516897] conn(0x7fcd040ae990 0x7fcd040b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:15.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.035+0000 7fcd0f7fe700 1 -- 172.21.15.86:0/2511217872 --> [v2:172.21.15.86:6802/2571516897,v1:172.21.15.86:6803/2571516897] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fcd080095d0 con 0x7fcd040ae990 2024-04-21T08:37:15.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.035+0000 7fcd0f7fe700 1 -- 172.21.15.86:0/2511217872 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fcd080d1fc0 con 0x7fcd18102300 2024-04-21T08:37:15.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.035+0000 7fcd1eb88700 1 --2- 172.21.15.86:0/2511217872 >> [v2:172.21.15.86:6802/2571516897,v1:172.21.15.86:6803/2571516897] conn(0x7fcd040ae990 0x7fcd040b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:15.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.035+0000 7fcd1e387700 1 --2- 172.21.15.86:0/2511217872 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fcd040a3cd0 0x7fcd040a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcd14006fd0 tx=0x7fcd14008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:15.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.035+0000 7fcd1eb88700 1 --2- 172.21.15.86:0/2511217872 >> [v2:172.21.15.86:6802/2571516897,v1:172.21.15.86:6803/2571516897] conn(0x7fcd040ae990 0x7fcd040b0dd0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:15.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.036+0000 7fcd0f7fe700 1 -- 172.21.15.86:0/2511217872 <== osd.0 v2:172.21.15.86:6802/2571516897 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fcd080095d0 con 0x7fcd040ae990 2024-04-21T08:37:15.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.053+0000 7fcd0d7fa700 1 -- 172.21.15.86:0/2511217872 --> [v2:172.21.15.86:6802/2571516897,v1:172.21.15.86:6803/2571516897] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fcd00002e60 con 0x7fcd040ae990 2024-04-21T08:37:15.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.053+0000 7fcd0f7fe700 1 -- 172.21.15.86:0/2511217872 <== osd.0 v2:172.21.15.86:6802/2571516897 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fcd00002e60 con 0x7fcd040ae990 2024-04-21T08:37:15.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.053+0000 7fcd0d7fa700 1 -- 172.21.15.86:0/2511217872 >> [v2:172.21.15.86:6802/2571516897,v1:172.21.15.86:6803/2571516897] conn(0x7fcd040ae990 msgr2=0x7fcd040b0dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:15.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.053+0000 7fcd0d7fa700 1 --2- 172.21.15.86:0/2511217872 >> [v2:172.21.15.86:6802/2571516897,v1:172.21.15.86:6803/2571516897] conn(0x7fcd040ae990 0x7fcd040b0dd0 crc :-1 s=READY pgs=8 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:15.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.053+0000 7fcd0d7fa700 1 -- 172.21.15.86:0/2511217872 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fcd040a3cd0 msgr2=0x7fcd040a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:15.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.053+0000 7fcd0d7fa700 1 --2- 172.21.15.86:0/2511217872 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fcd040a3cd0 0x7fcd040a6190 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x7fcd14006fd0 tx=0x7fcd14008040 comp rx=0 tx=0).stop 2024-04-21T08:37:15.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.053+0000 7fcd0d7fa700 1 -- 172.21.15.86:0/2511217872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd18102300 msgr2=0x7fcd181985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:15.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.053+0000 7fcd0d7fa700 1 --2- 172.21.15.86:0/2511217872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcd18102300 0x7fcd181985d0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7fcd08000c00 tx=0x7fcd080047b0 comp rx=0 tx=0).stop 2024-04-21T08:37:15.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.053+0000 7fcd1eb88700 1 -- 172.21.15.86:0/2511217872 reap_dead start 2024-04-21T08:37:15.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.054+0000 7fcd0d7fa700 1 -- 172.21.15.86:0/2511217872 shutdown_connections 2024-04-21T08:37:15.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.054+0000 7fcd0d7fa700 1 -- 172.21.15.86:0/2511217872 >> 172.21.15.86:0/2511217872 conn(0x7fcd180788f0 msgr2=0x7fcd180ffa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:15.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.054+0000 7fcd0d7fa700 1 -- 172.21.15.86:0/2511217872 shutdown_connections 2024-04-21T08:37:15.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:15.054+0000 7fcd0d7fa700 1 -- 172.21.15.86:0/2511217872 wait complete. 2024-04-21T08:37:16.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:16 smithi086 ceph-mon[27939]: pgmap v315: 129 pgs: 129 active+clean; 644 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-21T08:37:16.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:16 smithi088 ceph-mon[31259]: pgmap v315: 129 pgs: 129 active+clean; 644 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-21T08:37:16.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:16 smithi179 ceph-mon[31829]: pgmap v315: 129 pgs: 129 active+clean; 644 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 426 B/s wr, 11 op/s 2024-04-21T08:37:17.068 INFO:teuthology.orchestra.run.smithi086.stdout:55834574922 2024-04-21T08:37:17.069 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.1 2024-04-21T08:37:18.245 INFO:teuthology.orchestra.run.smithi086.stdout:34359738447 2024-04-21T08:37:18.245 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.0 2024-04-21T08:37:18.547 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:18 smithi086 ceph-mon[27939]: pgmap v316: 129 pgs: 129 active+clean; 644 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-21T08:37:18.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:18 smithi088 ceph-mon[31259]: pgmap v316: 129 pgs: 129 active+clean; 644 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-21T08:37:18.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:18 smithi179 ceph-mon[31829]: pgmap v316: 129 pgs: 129 active+clean; 644 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 426 B/s wr, 8 op/s 2024-04-21T08:37:18.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.774+0000 7f222ec96700 1 -- 172.21.15.86:0/1606093839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2228107980 msgr2=0x7f2228107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:18.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.774+0000 7f222ec96700 1 --2- 172.21.15.86:0/1606093839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2228107980 0x7f2228107d60 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f221c009990 tx=0x7f221c009ca0 comp rx=0 tx=0).stop 2024-04-21T08:37:18.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.774+0000 7f222ec96700 1 -- 172.21.15.86:0/1606093839 shutdown_connections 2024-04-21T08:37:18.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.774+0000 7f222ec96700 1 --2- 172.21.15.86:0/1606093839 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f22281022f0 0x7f222810e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:18.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.774+0000 7f222ec96700 1 --2- 172.21.15.86:0/1606093839 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2228107980 0x7f2228107d60 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:18.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.774+0000 7f222ec96700 1 --2- 172.21.15.86:0/1606093839 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2228101930 0x7f2228101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:18.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.774+0000 7f222ec96700 1 -- 172.21.15.86:0/1606093839 >> 172.21.15.86:0/1606093839 conn(0x7f22280788b0 msgr2=0x7f2228078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:18.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.774+0000 7f222ec96700 1 -- 172.21.15.86:0/1606093839 shutdown_connections 2024-04-21T08:37:18.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.774+0000 7f222ec96700 1 -- 172.21.15.86:0/1606093839 wait complete. 2024-04-21T08:37:18.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.775+0000 7f222ec96700 1 Processor -- start 2024-04-21T08:37:18.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.775+0000 7f222ec96700 1 -- start start 2024-04-21T08:37:18.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.775+0000 7f222ec96700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2228101930 0x7f2228199fb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:18.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.775+0000 7f222ec96700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22281022f0 0x7f222819a4f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:18.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.775+0000 7f222ec96700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2228107980 0x7f22281a5030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:18.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.775+0000 7f222ec96700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2228111b50 con 0x7f22281022f0 2024-04-21T08:37:18.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.775+0000 7f222ec96700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f22281119d0 con 0x7f2228107980 2024-04-21T08:37:18.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.775+0000 7f222ec96700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2228111cd0 con 0x7f2228101930 2024-04-21T08:37:18.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.775+0000 7f2227fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22281022f0 0x7f222819a4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:18.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.775+0000 7f2227fff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22281022f0 0x7f222819a4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48862/0 (socket says 172.21.15.86:48862) 2024-04-21T08:37:18.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.775+0000 7f2227fff700 1 -- 172.21.15.86:0/2918372279 learned_addr learned my addr 172.21.15.86:0/2918372279 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:37:18.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.776+0000 7f222d233700 1 --2- 172.21.15.86:0/2918372279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2228107980 0x7f22281a5030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:18.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.776+0000 7f222ca32700 1 --2- 172.21.15.86:0/2918372279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2228101930 0x7f2228199fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:18.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.776+0000 7f2227fff700 1 -- 172.21.15.86:0/2918372279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2228101930 msgr2=0x7f2228199fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:18.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.776+0000 7f2227fff700 1 --2- 172.21.15.86:0/2918372279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2228101930 0x7f2228199fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:18.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.776+0000 7f2227fff700 1 -- 172.21.15.86:0/2918372279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2228107980 msgr2=0x7f22281a5030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:18.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.776+0000 7f2227fff700 1 --2- 172.21.15.86:0/2918372279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2228107980 0x7f22281a5030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:18.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.776+0000 7f2227fff700 1 -- 172.21.15.86:0/2918372279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f22281a5570 con 0x7f22281022f0 2024-04-21T08:37:18.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.776+0000 7f2227fff700 1 --2- 172.21.15.86:0/2918372279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22281022f0 0x7f222819a4f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f221400c920 tx=0x7f221400cc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:18.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.776+0000 7f2225ffb700 1 -- 172.21.15.86:0/2918372279 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2214007b00 con 0x7f22281022f0 2024-04-21T08:37:18.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.776+0000 7f222ec96700 1 -- 172.21.15.86:0/2918372279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f22281a5800 con 0x7f22281022f0 2024-04-21T08:37:18.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.776+0000 7f2225ffb700 1 -- 172.21.15.86:0/2918372279 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2214004ce0 con 0x7f22281022f0 2024-04-21T08:37:18.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.776+0000 7f2225ffb700 1 -- 172.21.15.86:0/2918372279 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2214005500 con 0x7f22281022f0 2024-04-21T08:37:18.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.776+0000 7f222ec96700 1 -- 172.21.15.86:0/2918372279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f22281a5d60 con 0x7f22281022f0 2024-04-21T08:37:18.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.777+0000 7f222ec96700 1 -- 172.21.15.86:0/2918372279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f22280630d0 con 0x7f22281022f0 2024-04-21T08:37:18.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.778+0000 7f2225ffb700 1 -- 172.21.15.86:0/2918372279 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f2214007ca0 con 0x7f22281022f0 2024-04-21T08:37:18.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.779+0000 7f2225ffb700 1 --2- 172.21.15.86:0/2918372279 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f22180a3cd0 0x7f22180a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:18.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.779+0000 7f222ca32700 1 --2- 172.21.15.86:0/2918372279 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f22180a3cd0 0x7f22180a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:18.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.779+0000 7f2225ffb700 1 -- 172.21.15.86:0/2918372279 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7f22140d0360 con 0x7f22281022f0 2024-04-21T08:37:18.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.779+0000 7f2225ffb700 1 --2- 172.21.15.86:0/2918372279 >> [v2:172.21.15.88:6808/1818714217,v1:172.21.15.88:6809/1818714217] conn(0x7f22180ae990 0x7f22180b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:18.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.779+0000 7f2225ffb700 1 -- 172.21.15.86:0/2918372279 --> [v2:172.21.15.88:6808/1818714217,v1:172.21.15.88:6809/1818714217] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f221400eec0 con 0x7f22180ae990 2024-04-21T08:37:18.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.779+0000 7f2225ffb700 1 -- 172.21.15.86:0/2918372279 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7f22140d7050 con 0x7f22281022f0 2024-04-21T08:37:18.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.779+0000 7f222ca32700 1 --2- 172.21.15.86:0/2918372279 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f22180a3cd0 0x7f22180a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f221c006010 tx=0x7f221c01a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:18.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.779+0000 7f222d233700 1 --2- 172.21.15.86:0/2918372279 >> [v2:172.21.15.88:6808/1818714217,v1:172.21.15.88:6809/1818714217] conn(0x7f22180ae990 0x7f22180b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:18.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.780+0000 7f222d233700 1 --2- 172.21.15.86:0/2918372279 >> [v2:172.21.15.88:6808/1818714217,v1:172.21.15.88:6809/1818714217] conn(0x7f22180ae990 0x7f22180b0dd0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.4 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:18.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.781+0000 7f2225ffb700 1 -- 172.21.15.86:0/2918372279 <== osd.4 v2:172.21.15.88:6808/1818714217 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f221400eec0 con 0x7f22180ae990 2024-04-21T08:37:18.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.797+0000 7f222ec96700 1 -- 172.21.15.86:0/2918372279 --> [v2:172.21.15.88:6808/1818714217,v1:172.21.15.88:6809/1818714217] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f2228063550 con 0x7f22180ae990 2024-04-21T08:37:18.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.797+0000 7f2225ffb700 1 -- 172.21.15.86:0/2918372279 <== osd.4 v2:172.21.15.88:6808/1818714217 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f2228063550 con 0x7f22180ae990 2024-04-21T08:37:18.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.797+0000 7f222ec96700 1 -- 172.21.15.86:0/2918372279 >> [v2:172.21.15.88:6808/1818714217,v1:172.21.15.88:6809/1818714217] conn(0x7f22180ae990 msgr2=0x7f22180b0dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:18.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.797+0000 7f222ec96700 1 --2- 172.21.15.86:0/2918372279 >> [v2:172.21.15.88:6808/1818714217,v1:172.21.15.88:6809/1818714217] conn(0x7f22180ae990 0x7f22180b0dd0 crc :-1 s=READY pgs=17 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:18.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.797+0000 7f222ec96700 1 -- 172.21.15.86:0/2918372279 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f22180a3cd0 msgr2=0x7f22180a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:18.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.797+0000 7f222ec96700 1 --2- 172.21.15.86:0/2918372279 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f22180a3cd0 0x7f22180a6190 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f221c006010 tx=0x7f221c01a040 comp rx=0 tx=0).stop 2024-04-21T08:37:18.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.797+0000 7f222ec96700 1 -- 172.21.15.86:0/2918372279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22281022f0 msgr2=0x7f222819a4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:18.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.797+0000 7f222ec96700 1 --2- 172.21.15.86:0/2918372279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f22281022f0 0x7f222819a4f0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f221400c920 tx=0x7f221400cc30 comp rx=0 tx=0).stop 2024-04-21T08:37:18.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.797+0000 7f222d233700 1 -- 172.21.15.86:0/2918372279 reap_dead start 2024-04-21T08:37:18.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.798+0000 7f222ec96700 1 -- 172.21.15.86:0/2918372279 shutdown_connections 2024-04-21T08:37:18.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.798+0000 7f222ec96700 1 -- 172.21.15.86:0/2918372279 >> 172.21.15.86:0/2918372279 conn(0x7f22280788b0 msgr2=0x7f22281007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:18.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.798+0000 7f222ec96700 1 -- 172.21.15.86:0/2918372279 shutdown_connections 2024-04-21T08:37:18.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:18.798+0000 7f222ec96700 1 -- 172.21.15.86:0/2918372279 wait complete. 2024-04-21T08:37:19.454 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:19 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:37:19.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:19 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:37:19.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:19 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:37:20.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:20 smithi086 ceph-mon[27939]: pgmap v317: 129 pgs: 129 active+clean; 644 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-21T08:37:20.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:20 smithi088 ceph-mon[31259]: pgmap v317: 129 pgs: 129 active+clean; 644 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-21T08:37:20.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:20 smithi179 ceph-mon[31829]: pgmap v317: 129 pgs: 129 active+clean; 644 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 4.9 KiB/s rd, 426 B/s wr, 10 op/s 2024-04-21T08:37:21.604 INFO:teuthology.orchestra.run.smithi086.stdout:287762808866 2024-04-21T08:37:21.605 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.9 2024-04-21T08:37:21.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:21 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:37:21.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:21 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:37:21.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:21 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:37:21.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:21 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:37:21.738 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:21 smithi086 ceph-mon[27939]: pgmap v318: 129 pgs: 129 active+clean; 646 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-21T08:37:21.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:21 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:37:21.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:21 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:37:21.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:21 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:37:21.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:21 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:37:21.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:21 smithi088 ceph-mon[31259]: pgmap v318: 129 pgs: 129 active+clean; 646 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-21T08:37:21.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:21 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:37:21.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:21 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:37:21.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:21 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:37:21.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:21 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:37:21.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:21 smithi179 ceph-mon[31829]: pgmap v318: 129 pgs: 129 active+clean; 646 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 597 B/s wr, 11 op/s 2024-04-21T08:37:22.371 INFO:teuthology.orchestra.run.smithi086.stdout:141733920829 2024-04-21T08:37:22.371 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.4 2024-04-21T08:37:23.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.800+0000 7fa9ee242700 1 -- 172.21.15.86:0/2857048358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9e8102a90 msgr2=0x7fa9e8102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:23.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.800+0000 7fa9ee242700 1 --2- 172.21.15.86:0/2857048358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9e8102a90 0x7fa9e8102f10 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc009990 tx=0x7fa9dc009ca0 comp rx=0 tx=0).stop 2024-04-21T08:37:23.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.800+0000 7fa9ee242700 1 -- 172.21.15.86:0/2857048358 shutdown_connections 2024-04-21T08:37:23.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.800+0000 7fa9ee242700 1 --2- 172.21.15.86:0/2857048358 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9e8103450 0x7fa9e810f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:23.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.800+0000 7fa9ee242700 1 --2- 172.21.15.86:0/2857048358 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9e8108a50 0x7fa9e8108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:23.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.800+0000 7fa9ee242700 1 --2- 172.21.15.86:0/2857048358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9e8102a90 0x7fa9e8102f10 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:23.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.800+0000 7fa9ee242700 1 -- 172.21.15.86:0/2857048358 >> 172.21.15.86:0/2857048358 conn(0x7fa9e8079ba0 msgr2=0x7fa9e8079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:23.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.801+0000 7fa9ee242700 1 -- 172.21.15.86:0/2857048358 shutdown_connections 2024-04-21T08:37:23.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.801+0000 7fa9ee242700 1 -- 172.21.15.86:0/2857048358 wait complete. 2024-04-21T08:37:23.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.801+0000 7fa9ee242700 1 Processor -- start 2024-04-21T08:37:23.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.801+0000 7fa9ee242700 1 -- start start 2024-04-21T08:37:23.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.802+0000 7fa9ee242700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9e8102a90 0x7fa9e8199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:23.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.802+0000 7fa9ee242700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9e8103450 0x7fa9e8199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:23.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.802+0000 7fa9ee242700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9e8108a50 0x7fa9e819d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:23.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.802+0000 7fa9ee242700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e8112be0 con 0x7fa9e8103450 2024-04-21T08:37:23.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.802+0000 7fa9ee242700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e8112a60 con 0x7fa9e8108a50 2024-04-21T08:37:23.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.802+0000 7fa9ee242700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa9e8112d60 con 0x7fa9e8102a90 2024-04-21T08:37:23.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.802+0000 7fa9e6ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9e8103450 0x7fa9e8199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:23.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.802+0000 7fa9e6ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9e8103450 0x7fa9e8199570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57452/0 (socket says 172.21.15.86:57452) 2024-04-21T08:37:23.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.802+0000 7fa9e6ffd700 1 -- 172.21.15.86:0/3306585170 learned_addr learned my addr 172.21.15.86:0/3306585170 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:37:23.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.802+0000 7fa9e7fff700 1 --2- 172.21.15.86:0/3306585170 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9e8108a50 0x7fa9e819d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:23.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.802+0000 7fa9e77fe700 1 --2- 172.21.15.86:0/3306585170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9e8102a90 0x7fa9e8199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:23.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.802+0000 7fa9e6ffd700 1 -- 172.21.15.86:0/3306585170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9e8102a90 msgr2=0x7fa9e8199030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:23.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.802+0000 7fa9e6ffd700 1 --2- 172.21.15.86:0/3306585170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa9e8102a90 0x7fa9e8199030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:23.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.802+0000 7fa9e6ffd700 1 -- 172.21.15.86:0/3306585170 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9e8108a50 msgr2=0x7fa9e819d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:23.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.802+0000 7fa9e6ffd700 1 --2- 172.21.15.86:0/3306585170 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa9e8108a50 0x7fa9e819d900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:23.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.802+0000 7fa9e6ffd700 1 -- 172.21.15.86:0/3306585170 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa9e819e040 con 0x7fa9e8103450 2024-04-21T08:37:23.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.802+0000 7fa9e6ffd700 1 --2- 172.21.15.86:0/3306585170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9e8103450 0x7fa9e8199570 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc0057f0 tx=0x7fa9dc003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:23.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.802+0000 7fa9e4ff9700 1 -- 172.21.15.86:0/3306585170 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc027070 con 0x7fa9e8103450 2024-04-21T08:37:23.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.802+0000 7fa9e4ff9700 1 -- 172.21.15.86:0/3306585170 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9dc0040a0 con 0x7fa9e8103450 2024-04-21T08:37:23.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.802+0000 7fa9ee242700 1 -- 172.21.15.86:0/3306585170 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa9e819e2d0 con 0x7fa9e8103450 2024-04-21T08:37:23.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.802+0000 7fa9e4ff9700 1 -- 172.21.15.86:0/3306585170 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9dc022e20 con 0x7fa9e8103450 2024-04-21T08:37:23.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.802+0000 7fa9ee242700 1 -- 172.21.15.86:0/3306585170 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa9e81a5bb0 con 0x7fa9e8103450 2024-04-21T08:37:23.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.803+0000 7fa9ee242700 1 -- 172.21.15.86:0/3306585170 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa9e80631d0 con 0x7fa9e8103450 2024-04-21T08:37:23.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.804+0000 7fa9e4ff9700 1 -- 172.21.15.86:0/3306585170 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa9dc02a070 con 0x7fa9e8103450 2024-04-21T08:37:23.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.805+0000 7fa9e4ff9700 1 --2- 172.21.15.86:0/3306585170 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa9d00a3cd0 0x7fa9d00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:23.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.805+0000 7fa9e77fe700 1 --2- 172.21.15.86:0/3306585170 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa9d00a3cd0 0x7fa9d00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:23.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.805+0000 7fa9e4ff9700 1 -- 172.21.15.86:0/3306585170 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fa9dc0d1ef0 con 0x7fa9e8103450 2024-04-21T08:37:23.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.805+0000 7fa9e4ff9700 1 --2- 172.21.15.86:0/3306585170 >> [v2:172.21.15.179:6800/4137659051,v1:172.21.15.179:6801/4137659051] conn(0x7fa9d00ae990 0x7fa9d00b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:23.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.805+0000 7fa9e4ff9700 1 -- 172.21.15.86:0/3306585170 --> [v2:172.21.15.179:6800/4137659051,v1:172.21.15.179:6801/4137659051] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa9d00b14a0 con 0x7fa9d00ae990 2024-04-21T08:37:23.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.805+0000 7fa9e4ff9700 1 -- 172.21.15.86:0/3306585170 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fa9dc0d2340 con 0x7fa9e8103450 2024-04-21T08:37:23.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.805+0000 7fa9e77fe700 1 --2- 172.21.15.86:0/3306585170 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa9d00a3cd0 0x7fa9d00a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8005fd0 tx=0x7fa9d800af20 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:23.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.805+0000 7fa9e7fff700 1 --2- 172.21.15.86:0/3306585170 >> [v2:172.21.15.179:6800/4137659051,v1:172.21.15.179:6801/4137659051] conn(0x7fa9d00ae990 0x7fa9d00b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:23.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.806+0000 7fa9e7fff700 1 --2- 172.21.15.86:0/3306585170 >> [v2:172.21.15.179:6800/4137659051,v1:172.21.15.179:6801/4137659051] conn(0x7fa9d00ae990 0x7fa9d00b0dd0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:23.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.807+0000 7fa9e4ff9700 1 -- 172.21.15.86:0/3306585170 <== osd.2 v2:172.21.15.179:6800/4137659051 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa9d00b14a0 con 0x7fa9d00ae990 2024-04-21T08:37:23.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.824+0000 7fa9ee242700 1 -- 172.21.15.86:0/3306585170 --> [v2:172.21.15.179:6800/4137659051,v1:172.21.15.179:6801/4137659051] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa9e8063650 con 0x7fa9d00ae990 2024-04-21T08:37:23.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.825+0000 7fa9e4ff9700 1 -- 172.21.15.86:0/3306585170 <== osd.2 v2:172.21.15.179:6800/4137659051 2 ==== command_reply(tid 2: 0 ) ==== 8+0+11 (crc 0 0 0) 0x7fa9e8063650 con 0x7fa9d00ae990 2024-04-21T08:37:23.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.825+0000 7fa9ee242700 1 -- 172.21.15.86:0/3306585170 >> [v2:172.21.15.179:6800/4137659051,v1:172.21.15.179:6801/4137659051] conn(0x7fa9d00ae990 msgr2=0x7fa9d00b0dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:23.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.825+0000 7fa9ee242700 1 --2- 172.21.15.86:0/3306585170 >> [v2:172.21.15.179:6800/4137659051,v1:172.21.15.179:6801/4137659051] conn(0x7fa9d00ae990 0x7fa9d00b0dd0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:23.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.825+0000 7fa9ee242700 1 -- 172.21.15.86:0/3306585170 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa9d00a3cd0 msgr2=0x7fa9d00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:23.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.825+0000 7fa9ee242700 1 --2- 172.21.15.86:0/3306585170 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa9d00a3cd0 0x7fa9d00a6190 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7fa9d8005fd0 tx=0x7fa9d800af20 comp rx=0 tx=0).stop 2024-04-21T08:37:23.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.825+0000 7fa9ee242700 1 -- 172.21.15.86:0/3306585170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9e8103450 msgr2=0x7fa9e8199570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:23.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.825+0000 7fa9ee242700 1 --2- 172.21.15.86:0/3306585170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa9e8103450 0x7fa9e8199570 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7fa9dc0057f0 tx=0x7fa9dc003680 comp rx=0 tx=0).stop 2024-04-21T08:37:23.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.825+0000 7fa9e7fff700 1 -- 172.21.15.86:0/3306585170 reap_dead start 2024-04-21T08:37:23.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.825+0000 7fa9ee242700 1 -- 172.21.15.86:0/3306585170 shutdown_connections 2024-04-21T08:37:23.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.825+0000 7fa9ee242700 1 -- 172.21.15.86:0/3306585170 >> 172.21.15.86:0/3306585170 conn(0x7fa9e8079ba0 msgr2=0x7fa9e8101610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:23.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.825+0000 7fa9ee242700 1 -- 172.21.15.86:0/3306585170 shutdown_connections 2024-04-21T08:37:23.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:22.825+0000 7fa9ee242700 1 -- 172.21.15.86:0/3306585170 wait complete. 2024-04-21T08:37:23.513 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:23 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-04-21T08:37:23.513 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:23 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-04-21T08:37:23.513 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:23 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]: dispatch 2024-04-21T08:37:23.513 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:23 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2024-04-21T08:37:23.513 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:23 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-04-21T08:37:23.513 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:23 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-21T08:37:23.513 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:23 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-04-21T08:37:23.513 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:23 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-04-21T08:37:23.514 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:23 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-04-21T08:37:23.514 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:23 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-04-21T08:37:23.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.616+0000 7fa3c7c1a700 1 -- 172.21.15.86:0/1416405109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3c0102a90 msgr2=0x7fa3c0102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:23.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.616+0000 7fa3c7c1a700 1 --2- 172.21.15.86:0/1416405109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3c0102a90 0x7fa3c0102f10 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7fa3b0009990 tx=0x7fa3b0009ca0 comp rx=0 tx=0).stop 2024-04-21T08:37:23.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.616+0000 7fa3c7c1a700 1 -- 172.21.15.86:0/1416405109 shutdown_connections 2024-04-21T08:37:23.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.616+0000 7fa3c7c1a700 1 --2- 172.21.15.86:0/1416405109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3c0103450 0x7fa3c010f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:23.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.616+0000 7fa3c7c1a700 1 --2- 172.21.15.86:0/1416405109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3c0108a50 0x7fa3c0108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:23.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.616+0000 7fa3c7c1a700 1 --2- 172.21.15.86:0/1416405109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3c0102a90 0x7fa3c0102f10 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:23.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.616+0000 7fa3c7c1a700 1 -- 172.21.15.86:0/1416405109 >> 172.21.15.86:0/1416405109 conn(0x7fa3c0079ba0 msgr2=0x7fa3c0079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:23.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.616+0000 7fa3c7c1a700 1 -- 172.21.15.86:0/1416405109 shutdown_connections 2024-04-21T08:37:23.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.616+0000 7fa3c7c1a700 1 -- 172.21.15.86:0/1416405109 wait complete. 2024-04-21T08:37:23.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.616+0000 7fa3c7c1a700 1 Processor -- start 2024-04-21T08:37:23.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.616+0000 7fa3c7c1a700 1 -- start start 2024-04-21T08:37:23.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.617+0000 7fa3c7c1a700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3c0102a90 0x7fa3c0199150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:23.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.617+0000 7fa3c7c1a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3c0103450 0x7fa3c0199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:23.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.617+0000 7fa3c7c1a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3c0108a50 0x7fa3c019da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:23.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.617+0000 7fa3c7c1a700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c0112be0 con 0x7fa3c0102a90 2024-04-21T08:37:23.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.617+0000 7fa3c7c1a700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c0112a60 con 0x7fa3c0103450 2024-04-21T08:37:23.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.617+0000 7fa3c7c1a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c0112d60 con 0x7fa3c0108a50 2024-04-21T08:37:23.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.617+0000 7fa3c59b6700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3c0102a90 0x7fa3c0199150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:23.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.617+0000 7fa3c59b6700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3c0102a90 0x7fa3c0199150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57472/0 (socket says 172.21.15.86:57472) 2024-04-21T08:37:23.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.617+0000 7fa3c59b6700 1 -- 172.21.15.86:0/929812354 learned_addr learned my addr 172.21.15.86:0/929812354 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:37:23.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.617+0000 7fa3c61b7700 1 --2- 172.21.15.86:0/929812354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3c0108a50 0x7fa3c019da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:23.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.617+0000 7fa3c51b5700 1 --2- 172.21.15.86:0/929812354 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3c0103450 0x7fa3c0199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:23.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.617+0000 7fa3c59b6700 1 -- 172.21.15.86:0/929812354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3c0108a50 msgr2=0x7fa3c019da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:23.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.617+0000 7fa3c59b6700 1 --2- 172.21.15.86:0/929812354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3c0108a50 0x7fa3c019da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:23.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.617+0000 7fa3c59b6700 1 -- 172.21.15.86:0/929812354 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3c0103450 msgr2=0x7fa3c0199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:23.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.617+0000 7fa3c59b6700 1 --2- 172.21.15.86:0/929812354 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3c0103450 0x7fa3c0199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:23.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.617+0000 7fa3c59b6700 1 -- 172.21.15.86:0/929812354 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c019e160 con 0x7fa3c0102a90 2024-04-21T08:37:23.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.617+0000 7fa3c59b6700 1 --2- 172.21.15.86:0/929812354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3c0102a90 0x7fa3c0199150 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa3b800e9a0 tx=0x7fa3b800ecb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:23.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.618+0000 7fa3b6ffd700 1 -- 172.21.15.86:0/929812354 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b800ca10 con 0x7fa3c0102a90 2024-04-21T08:37:23.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.618+0000 7fa3b6ffd700 1 -- 172.21.15.86:0/929812354 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3b800cbb0 con 0x7fa3c0102a90 2024-04-21T08:37:23.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.618+0000 7fa3c7c1a700 1 -- 172.21.15.86:0/929812354 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c019e450 con 0x7fa3c0102a90 2024-04-21T08:37:23.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.618+0000 7fa3b6ffd700 1 -- 172.21.15.86:0/929812354 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b8018680 con 0x7fa3c0102a90 2024-04-21T08:37:23.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.618+0000 7fa3c7c1a700 1 -- 172.21.15.86:0/929812354 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c01a5cd0 con 0x7fa3c0102a90 2024-04-21T08:37:23.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.619+0000 7fa3c7c1a700 1 -- 172.21.15.86:0/929812354 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fa3c0063190 con 0x7fa3c0102a90 2024-04-21T08:37:23.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.620+0000 7fa3b6ffd700 1 -- 172.21.15.86:0/929812354 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fa3b8010430 con 0x7fa3c0102a90 2024-04-21T08:37:23.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.621+0000 7fa3b6ffd700 1 --2- 172.21.15.86:0/929812354 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa3ac0a3cd0 0x7fa3ac0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:23.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.621+0000 7fa3b6ffd700 1 -- 172.21.15.86:0/929812354 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(90..90 src has 1..90) ==== 9092+0+0 (secure 0 0 0) 0x7fa3b80d0f50 con 0x7fa3c0102a90 2024-04-21T08:37:23.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.621+0000 7fa3c51b5700 1 --2- 172.21.15.86:0/929812354 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa3ac0a3cd0 0x7fa3ac0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:23.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.621+0000 7fa3b6ffd700 1 --2- 172.21.15.86:0/929812354 >> [v2:172.21.15.179:6816/1770418957,v1:172.21.15.179:6817/1770418957] conn(0x7fa3ac0aee30 0x7fa3ac0b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:23.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.621+0000 7fa3b6ffd700 1 -- 172.21.15.86:0/929812354 --> [v2:172.21.15.179:6816/1770418957,v1:172.21.15.179:6817/1770418957] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fa3b8019ec0 con 0x7fa3ac0aee30 2024-04-21T08:37:23.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.621+0000 7fa3b6ffd700 1 -- 172.21.15.86:0/929812354 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=90) ==== 24+0+0 (secure 0 0 0) 0x7fa3b80d13a0 con 0x7fa3c0102a90 2024-04-21T08:37:23.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.621+0000 7fa3c51b5700 1 --2- 172.21.15.86:0/929812354 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa3ac0a3cd0 0x7fa3ac0a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa3b000b2e0 tx=0x7fa3b0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:23.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.621+0000 7fa3c61b7700 1 --2- 172.21.15.86:0/929812354 >> [v2:172.21.15.179:6816/1770418957,v1:172.21.15.179:6817/1770418957] conn(0x7fa3ac0aee30 0x7fa3ac0b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:23.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.622+0000 7fa3c61b7700 1 --2- 172.21.15.86:0/929812354 >> [v2:172.21.15.179:6816/1770418957,v1:172.21.15.179:6817/1770418957] conn(0x7fa3ac0aee30 0x7fa3ac0b12d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.8 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:23.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.623+0000 7fa3b6ffd700 1 -- 172.21.15.86:0/929812354 <== osd.8 v2:172.21.15.179:6816/1770418957 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fa3b8019ec0 con 0x7fa3ac0aee30 2024-04-21T08:37:23.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.641+0000 7fa3c7c1a700 1 -- 172.21.15.86:0/929812354 --> [v2:172.21.15.179:6816/1770418957,v1:172.21.15.179:6817/1770418957] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fa3c0063610 con 0x7fa3ac0aee30 2024-04-21T08:37:23.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.642+0000 7fa3b6ffd700 1 -- 172.21.15.86:0/929812354 <== osd.8 v2:172.21.15.179:6816/1770418957 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fa3c0063610 con 0x7fa3ac0aee30 2024-04-21T08:37:23.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.642+0000 7fa3c7c1a700 1 -- 172.21.15.86:0/929812354 >> [v2:172.21.15.179:6816/1770418957,v1:172.21.15.179:6817/1770418957] conn(0x7fa3ac0aee30 msgr2=0x7fa3ac0b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:23.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.642+0000 7fa3c7c1a700 1 --2- 172.21.15.86:0/929812354 >> [v2:172.21.15.179:6816/1770418957,v1:172.21.15.179:6817/1770418957] conn(0x7fa3ac0aee30 0x7fa3ac0b12d0 crc :-1 s=READY pgs=16 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:23.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.642+0000 7fa3c7c1a700 1 -- 172.21.15.86:0/929812354 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa3ac0a3cd0 msgr2=0x7fa3ac0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:23.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.642+0000 7fa3c7c1a700 1 --2- 172.21.15.86:0/929812354 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa3ac0a3cd0 0x7fa3ac0a6190 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7fa3b000b2e0 tx=0x7fa3b0005960 comp rx=0 tx=0).stop 2024-04-21T08:37:23.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.642+0000 7fa3c7c1a700 1 -- 172.21.15.86:0/929812354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3c0102a90 msgr2=0x7fa3c0199150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:23.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.642+0000 7fa3c7c1a700 1 --2- 172.21.15.86:0/929812354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3c0102a90 0x7fa3c0199150 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7fa3b800e9a0 tx=0x7fa3b800ecb0 comp rx=0 tx=0).stop 2024-04-21T08:37:23.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.642+0000 7fa3c61b7700 1 -- 172.21.15.86:0/929812354 reap_dead start 2024-04-21T08:37:23.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.642+0000 7fa3c7c1a700 1 -- 172.21.15.86:0/929812354 shutdown_connections 2024-04-21T08:37:23.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.642+0000 7fa3c7c1a700 1 -- 172.21.15.86:0/929812354 >> 172.21.15.86:0/929812354 conn(0x7fa3c0079ba0 msgr2=0x7fa3c0101970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:23.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.642+0000 7fa3c7c1a700 1 -- 172.21.15.86:0/929812354 shutdown_connections 2024-04-21T08:37:23.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.642+0000 7fa3c7c1a700 1 -- 172.21.15.86:0/929812354 wait complete. 2024-04-21T08:37:23.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:23 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-04-21T08:37:23.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:23 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-04-21T08:37:23.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:23 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]: dispatch 2024-04-21T08:37:23.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:23 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2024-04-21T08:37:23.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:23 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-04-21T08:37:23.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:23 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-21T08:37:23.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:23 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-04-21T08:37:23.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:23 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-04-21T08:37:23.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:23 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-04-21T08:37:23.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:23 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-04-21T08:37:23.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:23 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]: dispatch 2024-04-21T08:37:23.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:23 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]: dispatch 2024-04-21T08:37:23.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:23 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]: dispatch 2024-04-21T08:37:23.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:23 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]: dispatch 2024-04-21T08:37:23.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:23 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]: dispatch 2024-04-21T08:37:23.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:23 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]: dispatch 2024-04-21T08:37:23.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:23 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]: dispatch 2024-04-21T08:37:23.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:23 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]: dispatch 2024-04-21T08:37:23.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:23 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]: dispatch 2024-04-21T08:37:23.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:23 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]: dispatch 2024-04-21T08:37:24.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.236+0000 7fc164190700 1 -- 172.21.15.86:0/411115217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc15c101d20 msgr2=0x7fc15c1021a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:24.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.236+0000 7fc164190700 1 --2- 172.21.15.86:0/411115217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc15c101d20 0x7fc15c1021a0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7fc14c009990 tx=0x7fc14c009ca0 comp rx=0 tx=0).stop 2024-04-21T08:37:24.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.236+0000 7fc164190700 1 -- 172.21.15.86:0/411115217 shutdown_connections 2024-04-21T08:37:24.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.236+0000 7fc164190700 1 --2- 172.21.15.86:0/411115217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc15c1026e0 0x7fc15c10e770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:24.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.236+0000 7fc164190700 1 --2- 172.21.15.86:0/411115217 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc15c106990 0x7fc15c106d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:24.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.236+0000 7fc164190700 1 --2- 172.21.15.86:0/411115217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc15c101d20 0x7fc15c1021a0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:24.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.236+0000 7fc164190700 1 -- 172.21.15.86:0/411115217 >> 172.21.15.86:0/411115217 conn(0x7fc15c0788f0 msgr2=0x7fc15c078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:24.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.236+0000 7fc164190700 1 -- 172.21.15.86:0/411115217 shutdown_connections 2024-04-21T08:37:24.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.236+0000 7fc164190700 1 -- 172.21.15.86:0/411115217 wait complete. 2024-04-21T08:37:24.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.237+0000 7fc164190700 1 Processor -- start 2024-04-21T08:37:24.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.237+0000 7fc164190700 1 -- start start 2024-04-21T08:37:24.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.237+0000 7fc164190700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc15c101d20 0x7fc15c199f00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:24.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.237+0000 7fc164190700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc15c1026e0 0x7fc15c19a440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:24.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.237+0000 7fc164190700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc15c106990 0x7fc15c1a4f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:24.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.237+0000 7fc164190700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc15c111ae0 con 0x7fc15c106990 2024-04-21T08:37:24.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.237+0000 7fc164190700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc15c111960 con 0x7fc15c101d20 2024-04-21T08:37:24.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.237+0000 7fc164190700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc15c111c60 con 0x7fc15c1026e0 2024-04-21T08:37:24.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.237+0000 7fc16272d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc15c106990 0x7fc15c1a4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:24.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.237+0000 7fc161f2c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc15c101d20 0x7fc15c199f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:24.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.237+0000 7fc16272d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc15c106990 0x7fc15c1a4f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57424/0 (socket says 172.21.15.86:57424) 2024-04-21T08:37:24.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.237+0000 7fc16272d700 1 -- 172.21.15.86:0/690271014 learned_addr learned my addr 172.21.15.86:0/690271014 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:37:24.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.237+0000 7fc16172b700 1 --2- 172.21.15.86:0/690271014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc15c1026e0 0x7fc15c19a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:24.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.238+0000 7fc16272d700 1 -- 172.21.15.86:0/690271014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc15c1026e0 msgr2=0x7fc15c19a440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:24.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.238+0000 7fc16272d700 1 --2- 172.21.15.86:0/690271014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc15c1026e0 0x7fc15c19a440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:24.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.238+0000 7fc16272d700 1 -- 172.21.15.86:0/690271014 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc15c101d20 msgr2=0x7fc15c199f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:24.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.238+0000 7fc16272d700 1 --2- 172.21.15.86:0/690271014 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc15c101d20 0x7fc15c199f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:24.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.238+0000 7fc16272d700 1 -- 172.21.15.86:0/690271014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc15c1a54c0 con 0x7fc15c106990 2024-04-21T08:37:24.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.238+0000 7fc161f2c700 1 --2- 172.21.15.86:0/690271014 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc15c101d20 0x7fc15c199f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:37:24.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.238+0000 7fc16272d700 1 --2- 172.21.15.86:0/690271014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc15c106990 0x7fc15c1a4f80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc154007d70 tx=0x7fc15400a340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:24.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.238+0000 7fc152ffd700 1 -- 172.21.15.86:0/690271014 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc154016020 con 0x7fc15c106990 2024-04-21T08:37:24.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.238+0000 7fc152ffd700 1 -- 172.21.15.86:0/690271014 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc15400ac90 con 0x7fc15c106990 2024-04-21T08:37:24.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.238+0000 7fc164190700 1 -- 172.21.15.86:0/690271014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc15c1a5690 con 0x7fc15c106990 2024-04-21T08:37:24.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.238+0000 7fc152ffd700 1 -- 172.21.15.86:0/690271014 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc1540115e0 con 0x7fc15c106990 2024-04-21T08:37:24.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.238+0000 7fc164190700 1 -- 172.21.15.86:0/690271014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc15c1a5b50 con 0x7fc15c106990 2024-04-21T08:37:24.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.239+0000 7fc164190700 1 -- 172.21.15.86:0/690271014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fc15c0631d0 con 0x7fc15c106990 2024-04-21T08:37:24.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.240+0000 7fc152ffd700 1 -- 172.21.15.86:0/690271014 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fc154019020 con 0x7fc15c106990 2024-04-21T08:37:24.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.240+0000 7fc152ffd700 1 --2- 172.21.15.86:0/690271014 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fc1480a3cd0 0x7fc1480a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:24.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.241+0000 7fc161f2c700 1 --2- 172.21.15.86:0/690271014 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fc1480a3cd0 0x7fc1480a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:24.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.241+0000 7fc152ffd700 1 -- 172.21.15.86:0/690271014 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7fc1540d0b20 con 0x7fc15c106990 2024-04-21T08:37:24.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.241+0000 7fc152ffd700 1 --2- 172.21.15.86:0/690271014 >> [v2:172.21.15.88:6816/136629878,v1:172.21.15.88:6817/136629878] conn(0x7fc1480ae990 0x7fc1480b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:24.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.241+0000 7fc152ffd700 1 -- 172.21.15.86:0/690271014 --> [v2:172.21.15.88:6816/136629878,v1:172.21.15.88:6817/136629878] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fc154024e50 con 0x7fc1480ae990 2024-04-21T08:37:24.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.241+0000 7fc152ffd700 1 -- 172.21.15.86:0/690271014 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7fc1540d0f70 con 0x7fc15c106990 2024-04-21T08:37:24.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.241+0000 7fc161f2c700 1 --2- 172.21.15.86:0/690271014 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fc1480a3cd0 0x7fc1480a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc158005950 tx=0x7fc15800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:24.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.241+0000 7fc16172b700 1 --2- 172.21.15.86:0/690271014 >> [v2:172.21.15.88:6816/136629878,v1:172.21.15.88:6817/136629878] conn(0x7fc1480ae990 0x7fc1480b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:24.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.241+0000 7fc16172b700 1 --2- 172.21.15.86:0/690271014 >> [v2:172.21.15.88:6816/136629878,v1:172.21.15.88:6817/136629878] conn(0x7fc1480ae990 0x7fc1480b0dd0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.7 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:24.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.242+0000 7fc152ffd700 1 -- 172.21.15.86:0/690271014 <== osd.7 v2:172.21.15.88:6816/136629878 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fc154024e50 con 0x7fc1480ae990 2024-04-21T08:37:24.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.258+0000 7fc164190700 1 -- 172.21.15.86:0/690271014 --> [v2:172.21.15.88:6816/136629878,v1:172.21.15.88:6817/136629878] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fc15c063680 con 0x7fc1480ae990 2024-04-21T08:37:24.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.259+0000 7fc152ffd700 1 -- 172.21.15.86:0/690271014 <== osd.7 v2:172.21.15.88:6816/136629878 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fc15c063680 con 0x7fc1480ae990 2024-04-21T08:37:24.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.259+0000 7fc164190700 1 -- 172.21.15.86:0/690271014 >> [v2:172.21.15.88:6816/136629878,v1:172.21.15.88:6817/136629878] conn(0x7fc1480ae990 msgr2=0x7fc1480b0dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:24.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.259+0000 7fc164190700 1 --2- 172.21.15.86:0/690271014 >> [v2:172.21.15.88:6816/136629878,v1:172.21.15.88:6817/136629878] conn(0x7fc1480ae990 0x7fc1480b0dd0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:24.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.259+0000 7fc164190700 1 -- 172.21.15.86:0/690271014 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fc1480a3cd0 msgr2=0x7fc1480a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:24.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.259+0000 7fc164190700 1 --2- 172.21.15.86:0/690271014 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fc1480a3cd0 0x7fc1480a6190 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fc158005950 tx=0x7fc15800b410 comp rx=0 tx=0).stop 2024-04-21T08:37:24.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.259+0000 7fc164190700 1 -- 172.21.15.86:0/690271014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc15c106990 msgr2=0x7fc15c1a4f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:24.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.259+0000 7fc164190700 1 --2- 172.21.15.86:0/690271014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc15c106990 0x7fc15c1a4f80 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fc154007d70 tx=0x7fc15400a340 comp rx=0 tx=0).stop 2024-04-21T08:37:24.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.259+0000 7fc16272d700 1 -- 172.21.15.86:0/690271014 reap_dead start 2024-04-21T08:37:24.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.259+0000 7fc164190700 1 -- 172.21.15.86:0/690271014 shutdown_connections 2024-04-21T08:37:24.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.259+0000 7fc164190700 1 -- 172.21.15.86:0/690271014 >> 172.21.15.86:0/690271014 conn(0x7fc15c0788f0 msgr2=0x7fc15c0ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:24.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.259+0000 7fc164190700 1 -- 172.21.15.86:0/690271014 shutdown_connections 2024-04-21T08:37:24.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:21.259+0000 7fc164190700 1 -- 172.21.15.86:0/690271014 wait complete. 2024-04-21T08:37:24.371 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:24 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-04-21T08:37:24.371 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:24 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-04-21T08:37:24.371 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:24 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-04-21T08:37:24.371 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:24 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-04-21T08:37:24.371 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:24 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-04-21T08:37:24.371 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:24 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-21T08:37:24.371 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:24 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-04-21T08:37:24.371 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:24 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-04-21T08:37:24.372 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:24 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-04-21T08:37:24.372 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:24 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-04-21T08:37:24.372 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:24 smithi086 ceph-mon[27939]: osdmap e90: 12 total, 12 up, 12 in 2024-04-21T08:37:24.372 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:24 smithi086 ceph-mon[27939]: pgmap v320: 129 pgs: 129 active+clean; 646 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-21T08:37:24.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:24 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-04-21T08:37:24.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:24 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-04-21T08:37:24.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:24 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-04-21T08:37:24.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:24 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-04-21T08:37:24.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:24 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-04-21T08:37:24.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:24 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-21T08:37:24.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:24 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-04-21T08:37:24.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:24 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-04-21T08:37:24.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:24 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-04-21T08:37:24.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:24 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-04-21T08:37:24.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:24 smithi088 ceph-mon[31259]: osdmap e90: 12 total, 12 up, 12 in 2024-04-21T08:37:24.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:24 smithi088 ceph-mon[31259]: pgmap v320: 129 pgs: 129 active+clean; 646 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-21T08:37:24.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:24 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.e", "id": [2, 4]}]': finished 2024-04-21T08:37:24.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:24 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.f", "id": [8, 11]}]': finished 2024-04-21T08:37:24.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:24 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.10", "id": [2, 1]}]': finished 2024-04-21T08:37:24.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:24 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "2.1f", "id": [7, 11]}]': finished 2024-04-21T08:37:24.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:24 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.0", "id": [4, 8]}]': finished 2024-04-21T08:37:24.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:24 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.3", "id": [4, 9]}]': finished 2024-04-21T08:37:24.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:24 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.e", "id": [5, 9]}]': finished 2024-04-21T08:37:24.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:24 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "3.36", "id": [4, 9]}]': finished 2024-04-21T08:37:24.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:24 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.5", "id": [9, 10]}]': finished 2024-04-21T08:37:24.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:24 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1f", "id": [9, 8]}]': finished 2024-04-21T08:37:24.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:24 smithi179 ceph-mon[31829]: osdmap e90: 12 total, 12 up, 12 in 2024-04-21T08:37:24.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:24 smithi179 ceph-mon[31829]: pgmap v320: 129 pgs: 129 active+clean; 646 KiB data, 363 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 204 B/s wr, 10 op/s 2024-04-21T08:37:24.949 INFO:teuthology.orchestra.run.smithi086.stdout:253403070505 2024-04-21T08:37:24.949 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.8 2024-04-21T08:37:25.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.244+0000 7ffaab72b700 1 -- 172.21.15.86:0/305811396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffaa4107960 msgr2=0x7ffaa4107d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:25.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.244+0000 7ffaab72b700 1 --2- 172.21.15.86:0/305811396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffaa4107960 0x7ffaa4107d40 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0009990 tx=0x7ffaa0009ca0 comp rx=0 tx=0).stop 2024-04-21T08:37:25.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.245+0000 7ffaab72b700 1 -- 172.21.15.86:0/305811396 shutdown_connections 2024-04-21T08:37:25.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.245+0000 7ffaab72b700 1 --2- 172.21.15.86:0/305811396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffaa4102320 0x7ffaa410e830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:25.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.245+0000 7ffaab72b700 1 --2- 172.21.15.86:0/305811396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffaa4107960 0x7ffaa4107d40 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:25.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.245+0000 7ffaab72b700 1 --2- 172.21.15.86:0/305811396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffaa4101960 0x7ffaa4101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:25.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.245+0000 7ffaab72b700 1 -- 172.21.15.86:0/305811396 >> 172.21.15.86:0/305811396 conn(0x7ffaa40789e0 msgr2=0x7ffaa4078df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:25.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.245+0000 7ffaab72b700 1 -- 172.21.15.86:0/305811396 shutdown_connections 2024-04-21T08:37:25.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.245+0000 7ffaab72b700 1 -- 172.21.15.86:0/305811396 wait complete. 2024-04-21T08:37:25.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.245+0000 7ffaab72b700 1 Processor -- start 2024-04-21T08:37:25.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.245+0000 7ffaab72b700 1 -- start start 2024-04-21T08:37:25.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.246+0000 7ffaab72b700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffaa4101960 0x7ffaa419b2b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:25.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.246+0000 7ffaab72b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffaa4102320 0x7ffaa419b7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:25.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.246+0000 7ffaab72b700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffaa4107960 0x7ffaa419fb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:25.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.246+0000 7ffaab72b700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa4111b40 con 0x7ffaa4102320 2024-04-21T08:37:25.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.246+0000 7ffaab72b700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa41119c0 con 0x7ffaa4101960 2024-04-21T08:37:25.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.246+0000 7ffaab72b700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ffaa4111cc0 con 0x7ffaa4107960 2024-04-21T08:37:25.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.246+0000 7ffaa8cc6700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffaa4102320 0x7ffaa419b7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:25.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.246+0000 7ffaa8cc6700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffaa4102320 0x7ffaa419b7f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57456/0 (socket says 172.21.15.86:57456) 2024-04-21T08:37:25.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.246+0000 7ffaa8cc6700 1 -- 172.21.15.86:0/1395622949 learned_addr learned my addr 172.21.15.86:0/1395622949 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:37:25.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.246+0000 7ffaa94c7700 1 --2- 172.21.15.86:0/1395622949 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffaa4101960 0x7ffaa419b2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:25.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.246+0000 7ffaa9cc8700 1 --2- 172.21.15.86:0/1395622949 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffaa4107960 0x7ffaa419fb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:25.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.246+0000 7ffaa8cc6700 1 -- 172.21.15.86:0/1395622949 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffaa4107960 msgr2=0x7ffaa419fb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:25.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.246+0000 7ffaa8cc6700 1 --2- 172.21.15.86:0/1395622949 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffaa4107960 0x7ffaa419fb80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:25.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.246+0000 7ffaa8cc6700 1 -- 172.21.15.86:0/1395622949 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffaa4101960 msgr2=0x7ffaa419b2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:25.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.246+0000 7ffaa8cc6700 1 --2- 172.21.15.86:0/1395622949 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffaa4101960 0x7ffaa419b2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:25.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.246+0000 7ffaa8cc6700 1 -- 172.21.15.86:0/1395622949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffaa41a02c0 con 0x7ffaa4102320 2024-04-21T08:37:25.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.246+0000 7ffaa94c7700 1 --2- 172.21.15.86:0/1395622949 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffaa4101960 0x7ffaa419b2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:37:25.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.246+0000 7ffaa9cc8700 1 --2- 172.21.15.86:0/1395622949 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffaa4107960 0x7ffaa419fb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:37:25.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.246+0000 7ffaa8cc6700 1 --2- 172.21.15.86:0/1395622949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffaa4102320 0x7ffaa419b7f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ffa9400e9a0 tx=0x7ffa9400ed60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:25.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.246+0000 7ffa9a7fc700 1 -- 172.21.15.86:0/1395622949 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa9400cbd0 con 0x7ffaa4102320 2024-04-21T08:37:25.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.246+0000 7ffaab72b700 1 -- 172.21.15.86:0/1395622949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffaa41a05b0 con 0x7ffaa4102320 2024-04-21T08:37:25.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.247+0000 7ffa9a7fc700 1 -- 172.21.15.86:0/1395622949 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa9400cd70 con 0x7ffaa4102320 2024-04-21T08:37:25.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.247+0000 7ffa9a7fc700 1 -- 172.21.15.86:0/1395622949 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa94018680 con 0x7ffaa4102320 2024-04-21T08:37:25.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.247+0000 7ffaab72b700 1 -- 172.21.15.86:0/1395622949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffaa41a7e30 con 0x7ffaa4102320 2024-04-21T08:37:25.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.247+0000 7ffaab72b700 1 -- 172.21.15.86:0/1395622949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7ffaa406bbd0 con 0x7ffaa4102320 2024-04-21T08:37:25.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.249+0000 7ffa9a7fc700 1 -- 172.21.15.86:0/1395622949 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7ffa94010970 con 0x7ffaa4102320 2024-04-21T08:37:25.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.249+0000 7ffa9a7fc700 1 --2- 172.21.15.86:0/1395622949 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ffa900a3cd0 0x7ffa900a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:25.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.249+0000 7ffaa94c7700 1 --2- 172.21.15.86:0/1395622949 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ffa900a3cd0 0x7ffa900a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:25.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.249+0000 7ffa9a7fc700 1 -- 172.21.15.86:0/1395622949 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(89..89 src has 1..89) ==== 8802+0+0 (secure 0 0 0) 0x7ffa940d1330 con 0x7ffaa4102320 2024-04-21T08:37:25.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.250+0000 7ffa9a7fc700 1 --2- 172.21.15.86:0/1395622949 >> [v2:172.21.15.86:6810/2309603148,v1:172.21.15.86:6811/2309603148] conn(0x7ffa900ae990 0x7ffa900b0dd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:25.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.250+0000 7ffa9a7fc700 1 -- 172.21.15.86:0/1395622949 --> [v2:172.21.15.86:6810/2309603148,v1:172.21.15.86:6811/2309603148] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7ffa94027e50 con 0x7ffa900ae990 2024-04-21T08:37:25.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.250+0000 7ffa9a7fc700 1 -- 172.21.15.86:0/1395622949 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=89) ==== 24+0+0 (secure 0 0 0) 0x7ffa940d1780 con 0x7ffaa4102320 2024-04-21T08:37:25.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.250+0000 7ffaa9cc8700 1 --2- 172.21.15.86:0/1395622949 >> [v2:172.21.15.86:6810/2309603148,v1:172.21.15.86:6811/2309603148] conn(0x7ffa900ae990 0x7ffa900b0dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:25.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.250+0000 7ffaa94c7700 1 --2- 172.21.15.86:0/1395622949 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ffa900a3cd0 0x7ffa900a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0009750 tx=0x7ffaa0005fb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:25.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.250+0000 7ffaa9cc8700 1 --2- 172.21.15.86:0/1395622949 >> [v2:172.21.15.86:6810/2309603148,v1:172.21.15.86:6811/2309603148] conn(0x7ffa900ae990 0x7ffa900b0dd0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.3 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:25.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.251+0000 7ffa9a7fc700 1 -- 172.21.15.86:0/1395622949 <== osd.3 v2:172.21.15.86:6810/2309603148 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7ffa94027e50 con 0x7ffa900ae990 2024-04-21T08:37:25.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.267+0000 7ffaab72b700 1 -- 172.21.15.86:0/1395622949 --> [v2:172.21.15.86:6810/2309603148,v1:172.21.15.86:6811/2309603148] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7ffaa41030a0 con 0x7ffa900ae990 2024-04-21T08:37:25.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.267+0000 7ffa9a7fc700 1 -- 172.21.15.86:0/1395622949 <== osd.3 v2:172.21.15.86:6810/2309603148 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7ffaa41030a0 con 0x7ffa900ae990 2024-04-21T08:37:25.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.268+0000 7ffaab72b700 1 -- 172.21.15.86:0/1395622949 >> [v2:172.21.15.86:6810/2309603148,v1:172.21.15.86:6811/2309603148] conn(0x7ffa900ae990 msgr2=0x7ffa900b0dd0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:25.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.268+0000 7ffaab72b700 1 --2- 172.21.15.86:0/1395622949 >> [v2:172.21.15.86:6810/2309603148,v1:172.21.15.86:6811/2309603148] conn(0x7ffa900ae990 0x7ffa900b0dd0 crc :-1 s=READY pgs=12 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:25.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.268+0000 7ffaab72b700 1 -- 172.21.15.86:0/1395622949 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ffa900a3cd0 msgr2=0x7ffa900a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:25.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.268+0000 7ffaab72b700 1 --2- 172.21.15.86:0/1395622949 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ffa900a3cd0 0x7ffa900a6190 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7ffaa0009750 tx=0x7ffaa0005fb0 comp rx=0 tx=0).stop 2024-04-21T08:37:25.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.268+0000 7ffaab72b700 1 -- 172.21.15.86:0/1395622949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffaa4102320 msgr2=0x7ffaa419b7f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:25.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.268+0000 7ffaab72b700 1 --2- 172.21.15.86:0/1395622949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffaa4102320 0x7ffaa419b7f0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7ffa9400e9a0 tx=0x7ffa9400ed60 comp rx=0 tx=0).stop 2024-04-21T08:37:25.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.268+0000 7ffaa9cc8700 1 -- 172.21.15.86:0/1395622949 reap_dead start 2024-04-21T08:37:25.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.268+0000 7ffaab72b700 1 -- 172.21.15.86:0/1395622949 shutdown_connections 2024-04-21T08:37:25.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.268+0000 7ffaab72b700 1 -- 172.21.15.86:0/1395622949 >> 172.21.15.86:0/1395622949 conn(0x7ffaa40789e0 msgr2=0x7ffaa406b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:25.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.268+0000 7ffaab72b700 1 -- 172.21.15.86:0/1395622949 shutdown_connections 2024-04-21T08:37:25.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:23.268+0000 7ffaab72b700 1 -- 172.21.15.86:0/1395622949 wait complete. 2024-04-21T08:37:25.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:25 smithi086 ceph-mon[27939]: osdmap e91: 12 total, 12 up, 12 in 2024-04-21T08:37:25.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:25 smithi088 ceph-mon[31259]: osdmap e91: 12 total, 12 up, 12 in 2024-04-21T08:37:25.734 INFO:teuthology.orchestra.run.smithi086.stdout:107374182465 2024-04-21T08:37:25.734 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.3 2024-04-21T08:37:25.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:25 smithi179 ceph-mon[31829]: osdmap e91: 12 total, 12 up, 12 in 2024-04-21T08:37:26.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:26 smithi086 ceph-mon[27939]: pgmap v322: 129 pgs: 2 peering, 127 active+clean; 646 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s; 83 B/s, 0 objects/s recovering 2024-04-21T08:37:26.654 INFO:teuthology.orchestra.run.smithi086.stdout:77309411399 2024-04-21T08:37:26.654 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.2 2024-04-21T08:37:26.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:26 smithi088 ceph-mon[31259]: pgmap v322: 129 pgs: 2 peering, 127 active+clean; 646 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s; 83 B/s, 0 objects/s recovering 2024-04-21T08:37:26.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:26 smithi179 ceph-mon[31829]: pgmap v322: 129 pgs: 2 peering, 127 active+clean; 646 KiB data, 367 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 255 B/s wr, 11 op/s; 83 B/s, 0 objects/s recovering 2024-04-21T08:37:26.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.849+0000 7f8e210dd700 1 -- 172.21.15.86:0/1064249519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e1c101970 msgr2=0x7f8e1c101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:26.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.849+0000 7f8e210dd700 1 --2- 172.21.15.86:0/1064249519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e1c101970 0x7f8e1c101df0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7f8e10009990 tx=0x7f8e10009ca0 comp rx=0 tx=0).stop 2024-04-21T08:37:26.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.849+0000 7f8e210dd700 1 -- 172.21.15.86:0/1064249519 shutdown_connections 2024-04-21T08:37:26.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.849+0000 7f8e210dd700 1 --2- 172.21.15.86:0/1064249519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e1c102330 0x7f8e1c10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:26.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.849+0000 7f8e210dd700 1 --2- 172.21.15.86:0/1064249519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8e1c107920 0x7f8e1c107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:26.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.849+0000 7f8e210dd700 1 --2- 172.21.15.86:0/1064249519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e1c101970 0x7f8e1c101df0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:26.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.849+0000 7f8e210dd700 1 -- 172.21.15.86:0/1064249519 >> 172.21.15.86:0/1064249519 conn(0x7f8e1c078910 msgr2=0x7f8e1c078d20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:26.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.850+0000 7f8e210dd700 1 -- 172.21.15.86:0/1064249519 shutdown_connections 2024-04-21T08:37:26.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.850+0000 7f8e210dd700 1 -- 172.21.15.86:0/1064249519 wait complete. 2024-04-21T08:37:26.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.850+0000 7f8e210dd700 1 Processor -- start 2024-04-21T08:37:26.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.851+0000 7f8e210dd700 1 -- start start 2024-04-21T08:37:26.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.851+0000 7f8e210dd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e1c101970 0x7f8e1c1034f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:26.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.851+0000 7f8e210dd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8e1c102330 0x7f8e1c103a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:26.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.851+0000 7f8e210dd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e1c107920 0x7f8e1c103f70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:26.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.851+0000 7f8e210dd700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c111b30 con 0x7f8e1c107920 2024-04-21T08:37:26.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.851+0000 7f8e210dd700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c1119b0 con 0x7f8e1c102330 2024-04-21T08:37:26.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.851+0000 7f8e210dd700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8e1c111cb0 con 0x7f8e1c101970 2024-04-21T08:37:26.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.851+0000 7f8e1b59e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e1c107920 0x7f8e1c103f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:26.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.851+0000 7f8e1b59e700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e1c107920 0x7f8e1c103f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57482/0 (socket says 172.21.15.86:57482) 2024-04-21T08:37:26.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.851+0000 7f8e1b59e700 1 -- 172.21.15.86:0/881900412 learned_addr learned my addr 172.21.15.86:0/881900412 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:37:26.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.851+0000 7f8e1a59c700 1 --2- 172.21.15.86:0/881900412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8e1c102330 0x7f8e1c103a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:26.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.851+0000 7f8e1ad9d700 1 --2- 172.21.15.86:0/881900412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e1c101970 0x7f8e1c1034f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:26.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.851+0000 7f8e1b59e700 1 -- 172.21.15.86:0/881900412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e1c101970 msgr2=0x7f8e1c1034f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:26.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.851+0000 7f8e1b59e700 1 --2- 172.21.15.86:0/881900412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e1c101970 0x7f8e1c1034f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:26.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.851+0000 7f8e1b59e700 1 -- 172.21.15.86:0/881900412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8e1c102330 msgr2=0x7f8e1c103a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:26.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.851+0000 7f8e1b59e700 1 --2- 172.21.15.86:0/881900412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8e1c102330 0x7f8e1c103a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:26.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.851+0000 7f8e1b59e700 1 -- 172.21.15.86:0/881900412 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e1c1a7630 con 0x7f8e1c107920 2024-04-21T08:37:26.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.852+0000 7f8e1b59e700 1 --2- 172.21.15.86:0/881900412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e1c107920 0x7f8e1c103f70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8e14007ee0 tx=0x7f8e1400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:26.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.852+0000 7f8e03fff700 1 -- 172.21.15.86:0/881900412 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e14004bb0 con 0x7f8e1c107920 2024-04-21T08:37:26.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.852+0000 7f8e210dd700 1 -- 172.21.15.86:0/881900412 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e1c1a7920 con 0x7f8e1c107920 2024-04-21T08:37:26.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.852+0000 7f8e03fff700 1 -- 172.21.15.86:0/881900412 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e14004d50 con 0x7f8e1c107920 2024-04-21T08:37:26.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.852+0000 7f8e03fff700 1 -- 172.21.15.86:0/881900412 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e14005920 con 0x7f8e1c107920 2024-04-21T08:37:26.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.852+0000 7f8e210dd700 1 -- 172.21.15.86:0/881900412 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e1c1a7e80 con 0x7f8e1c107920 2024-04-21T08:37:26.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.853+0000 7f8e210dd700 1 -- 172.21.15.86:0/881900412 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f8e1c06b3e0 con 0x7f8e1c107920 2024-04-21T08:37:26.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.855+0000 7f8e03fff700 1 -- 172.21.15.86:0/881900412 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f8e1401a020 con 0x7f8e1c107920 2024-04-21T08:37:26.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.855+0000 7f8e03fff700 1 --2- 172.21.15.86:0/881900412 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f8e040a3cd0 0x7f8e040a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:26.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.855+0000 7f8e03fff700 1 -- 172.21.15.86:0/881900412 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f8e140d0630 con 0x7f8e1c107920 2024-04-21T08:37:26.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.855+0000 7f8e1ad9d700 1 --2- 172.21.15.86:0/881900412 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f8e040a3cd0 0x7f8e040a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:26.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.856+0000 7f8e03fff700 1 --2- 172.21.15.86:0/881900412 >> [v2:172.21.15.179:6808/863840990,v1:172.21.15.179:6809/863840990] conn(0x7f8e040aee30 0x7f8e040b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:26.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.856+0000 7f8e03fff700 1 -- 172.21.15.86:0/881900412 --> [v2:172.21.15.179:6808/863840990,v1:172.21.15.179:6809/863840990] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f8e040b19a0 con 0x7f8e040aee30 2024-04-21T08:37:26.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.856+0000 7f8e03fff700 1 -- 172.21.15.86:0/881900412 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f8e140d0a80 con 0x7f8e1c107920 2024-04-21T08:37:26.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.856+0000 7f8e1a59c700 1 --2- 172.21.15.86:0/881900412 >> [v2:172.21.15.179:6808/863840990,v1:172.21.15.179:6809/863840990] conn(0x7f8e040aee30 0x7f8e040b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:26.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.856+0000 7f8e1ad9d700 1 --2- 172.21.15.86:0/881900412 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f8e040a3cd0 0x7f8e040a6190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c005950 tx=0x7f8e0c00b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:26.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.857+0000 7f8e1a59c700 1 --2- 172.21.15.86:0/881900412 >> [v2:172.21.15.179:6808/863840990,v1:172.21.15.179:6809/863840990] conn(0x7f8e040aee30 0x7f8e040b12d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.5 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:26.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.857+0000 7f8e03fff700 1 -- 172.21.15.86:0/881900412 <== osd.5 v2:172.21.15.179:6808/863840990 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f8e040b19a0 con 0x7f8e040aee30 2024-04-21T08:37:26.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.875+0000 7f8e210dd700 1 -- 172.21.15.86:0/881900412 --> [v2:172.21.15.179:6808/863840990,v1:172.21.15.179:6809/863840990] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f8e1c103060 con 0x7f8e040aee30 2024-04-21T08:37:26.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.875+0000 7f8e03fff700 1 -- 172.21.15.86:0/881900412 <== osd.5 v2:172.21.15.179:6808/863840990 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f8e1c103060 con 0x7f8e040aee30 2024-04-21T08:37:26.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.876+0000 7f8e210dd700 1 -- 172.21.15.86:0/881900412 >> [v2:172.21.15.179:6808/863840990,v1:172.21.15.179:6809/863840990] conn(0x7f8e040aee30 msgr2=0x7f8e040b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:26.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.876+0000 7f8e210dd700 1 --2- 172.21.15.86:0/881900412 >> [v2:172.21.15.179:6808/863840990,v1:172.21.15.179:6809/863840990] conn(0x7f8e040aee30 0x7f8e040b12d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:26.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.876+0000 7f8e210dd700 1 -- 172.21.15.86:0/881900412 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f8e040a3cd0 msgr2=0x7f8e040a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:26.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.876+0000 7f8e210dd700 1 --2- 172.21.15.86:0/881900412 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f8e040a3cd0 0x7f8e040a6190 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f8e0c005950 tx=0x7f8e0c00b410 comp rx=0 tx=0).stop 2024-04-21T08:37:26.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.876+0000 7f8e210dd700 1 -- 172.21.15.86:0/881900412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e1c107920 msgr2=0x7f8e1c103f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:26.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.876+0000 7f8e210dd700 1 --2- 172.21.15.86:0/881900412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e1c107920 0x7f8e1c103f70 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x7f8e14007ee0 tx=0x7f8e1400d3b0 comp rx=0 tx=0).stop 2024-04-21T08:37:26.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.876+0000 7f8e1b59e700 1 -- 172.21.15.86:0/881900412 reap_dead start 2024-04-21T08:37:26.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.876+0000 7f8e210dd700 1 -- 172.21.15.86:0/881900412 shutdown_connections 2024-04-21T08:37:26.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.876+0000 7f8e210dd700 1 -- 172.21.15.86:0/881900412 >> 172.21.15.86:0/881900412 conn(0x7f8e1c078910 msgr2=0x7f8e1c105770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:26.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.876+0000 7f8e210dd700 1 -- 172.21.15.86:0/881900412 shutdown_connections 2024-04-21T08:37:26.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:26.876+0000 7f8e210dd700 1 -- 172.21.15.86:0/881900412 wait complete. 2024-04-21T08:37:27.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.090+0000 7f5d7fba4700 1 -- 172.21.15.86:0/4160614625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d78107980 msgr2=0x7f5d78107d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:27.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.090+0000 7f5d7fba4700 1 --2- 172.21.15.86:0/4160614625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d78107980 0x7f5d78107d60 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f5d74009990 tx=0x7f5d74009ca0 comp rx=0 tx=0).stop 2024-04-21T08:37:27.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.090+0000 7f5d7fba4700 1 -- 172.21.15.86:0/4160614625 shutdown_connections 2024-04-21T08:37:27.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.090+0000 7f5d7fba4700 1 --2- 172.21.15.86:0/4160614625 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d781022f0 0x7f5d7810e7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:27.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.090+0000 7f5d7fba4700 1 --2- 172.21.15.86:0/4160614625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d78107980 0x7f5d78107d60 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:27.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.090+0000 7f5d7fba4700 1 --2- 172.21.15.86:0/4160614625 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d78101930 0x7f5d78101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:27.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.090+0000 7f5d7fba4700 1 -- 172.21.15.86:0/4160614625 >> 172.21.15.86:0/4160614625 conn(0x7f5d780788b0 msgr2=0x7f5d78078cc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:27.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.090+0000 7f5d7fba4700 1 -- 172.21.15.86:0/4160614625 shutdown_connections 2024-04-21T08:37:27.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.090+0000 7f5d7fba4700 1 -- 172.21.15.86:0/4160614625 wait complete. 2024-04-21T08:37:27.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.091+0000 7f5d7fba4700 1 Processor -- start 2024-04-21T08:37:27.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.091+0000 7f5d7fba4700 1 -- start start 2024-04-21T08:37:27.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.091+0000 7f5d7fba4700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d78101930 0x7f5d781980a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:27.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.091+0000 7f5d7fba4700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d781022f0 0x7f5d781985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:27.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.091+0000 7f5d7fba4700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d78107980 0x7f5d7819c970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:27.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.091+0000 7f5d7fba4700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5d78111b50 con 0x7f5d78101930 2024-04-21T08:37:27.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.091+0000 7f5d7fba4700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5d781119d0 con 0x7f5d781022f0 2024-04-21T08:37:27.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.091+0000 7f5d7fba4700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5d78111cd0 con 0x7f5d78107980 2024-04-21T08:37:27.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.091+0000 7f5d7d940700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d78101930 0x7f5d781980a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:27.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.091+0000 7f5d7d940700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d78101930 0x7f5d781980a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57492/0 (socket says 172.21.15.86:57492) 2024-04-21T08:37:27.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.091+0000 7f5d7d940700 1 -- 172.21.15.86:0/3664487265 learned_addr learned my addr 172.21.15.86:0/3664487265 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:37:27.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.091+0000 7f5d7d13f700 1 --2- 172.21.15.86:0/3664487265 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d781022f0 0x7f5d781985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:27.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.092+0000 7f5d7d940700 1 -- 172.21.15.86:0/3664487265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d78107980 msgr2=0x7f5d7819c970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:27.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.092+0000 7f5d7e141700 1 --2- 172.21.15.86:0/3664487265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d78107980 0x7f5d7819c970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:27.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.092+0000 7f5d7d940700 1 --2- 172.21.15.86:0/3664487265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5d78107980 0x7f5d7819c970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:27.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.092+0000 7f5d7d940700 1 -- 172.21.15.86:0/3664487265 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d781022f0 msgr2=0x7f5d781985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:27.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.092+0000 7f5d7d940700 1 --2- 172.21.15.86:0/3664487265 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5d781022f0 0x7f5d781985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:27.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.092+0000 7f5d7d940700 1 -- 172.21.15.86:0/3664487265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5d7819d0b0 con 0x7f5d78101930 2024-04-21T08:37:27.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.092+0000 7f5d7d940700 1 --2- 172.21.15.86:0/3664487265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d78101930 0x7f5d781980a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5d74000c00 tx=0x7f5d740047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:27.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.092+0000 7f5d6effd700 1 -- 172.21.15.86:0/3664487265 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d74027070 con 0x7f5d78101930 2024-04-21T08:37:27.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.092+0000 7f5d7fba4700 1 -- 172.21.15.86:0/3664487265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5d7819d340 con 0x7f5d78101930 2024-04-21T08:37:27.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.092+0000 7f5d6effd700 1 -- 172.21.15.86:0/3664487265 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5d740040e0 con 0x7f5d78101930 2024-04-21T08:37:27.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.092+0000 7f5d6effd700 1 -- 172.21.15.86:0/3664487265 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5d7400f460 con 0x7f5d78101930 2024-04-21T08:37:27.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.092+0000 7f5d7fba4700 1 -- 172.21.15.86:0/3664487265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5d781a4c20 con 0x7f5d78101930 2024-04-21T08:37:27.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.093+0000 7f5d7fba4700 1 -- 172.21.15.86:0/3664487265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7f5d780631d0 con 0x7f5d78101930 2024-04-21T08:37:27.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.094+0000 7f5d6effd700 1 -- 172.21.15.86:0/3664487265 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f5d7402a070 con 0x7f5d78101930 2024-04-21T08:37:27.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.094+0000 7f5d6effd700 1 --2- 172.21.15.86:0/3664487265 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f5d640a3cd0 0x7f5d640a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:27.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.095+0000 7f5d7d13f700 1 --2- 172.21.15.86:0/3664487265 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f5d640a3cd0 0x7f5d640a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:27.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.095+0000 7f5d6effd700 1 -- 172.21.15.86:0/3664487265 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f5d740d1f90 con 0x7f5d78101930 2024-04-21T08:37:27.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.095+0000 7f5d6effd700 1 --2- 172.21.15.86:0/3664487265 >> [v2:172.21.15.179:6824/4007070233,v1:172.21.15.179:6825/4007070233] conn(0x7f5d640aee30 0x7f5d640b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:27.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.095+0000 7f5d6effd700 1 -- 172.21.15.86:0/3664487265 --> [v2:172.21.15.179:6824/4007070233,v1:172.21.15.179:6825/4007070233] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7f5d74011ec0 con 0x7f5d640aee30 2024-04-21T08:37:27.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.095+0000 7f5d6effd700 1 -- 172.21.15.86:0/3664487265 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7f5d740d23e0 con 0x7f5d78101930 2024-04-21T08:37:27.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.095+0000 7f5d7d13f700 1 --2- 172.21.15.86:0/3664487265 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f5d640a3cd0 0x7f5d640a6190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5d78078f90 tx=0x7f5d6800a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:27.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.095+0000 7f5d7e141700 1 --2- 172.21.15.86:0/3664487265 >> [v2:172.21.15.179:6824/4007070233,v1:172.21.15.179:6825/4007070233] conn(0x7f5d640aee30 0x7f5d640b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:27.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.095+0000 7f5d7e141700 1 --2- 172.21.15.86:0/3664487265 >> [v2:172.21.15.179:6824/4007070233,v1:172.21.15.179:6825/4007070233] conn(0x7f5d640aee30 0x7f5d640b12d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.11 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:27.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.096+0000 7f5d6effd700 1 -- 172.21.15.86:0/3664487265 <== osd.11 v2:172.21.15.179:6824/4007070233 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7f5d74011ec0 con 0x7f5d640aee30 2024-04-21T08:37:27.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.112+0000 7f5d7fba4700 1 -- 172.21.15.86:0/3664487265 --> [v2:172.21.15.179:6824/4007070233,v1:172.21.15.179:6825/4007070233] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7f5d78063650 con 0x7f5d640aee30 2024-04-21T08:37:27.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.113+0000 7f5d6effd700 1 -- 172.21.15.86:0/3664487265 <== osd.11 v2:172.21.15.179:6824/4007070233 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7f5d78063650 con 0x7f5d640aee30 2024-04-21T08:37:27.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.113+0000 7f5d7fba4700 1 -- 172.21.15.86:0/3664487265 >> [v2:172.21.15.179:6824/4007070233,v1:172.21.15.179:6825/4007070233] conn(0x7f5d640aee30 msgr2=0x7f5d640b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:27.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.113+0000 7f5d7fba4700 1 --2- 172.21.15.86:0/3664487265 >> [v2:172.21.15.179:6824/4007070233,v1:172.21.15.179:6825/4007070233] conn(0x7f5d640aee30 0x7f5d640b12d0 crc :-1 s=READY pgs=7 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:27.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.113+0000 7f5d7fba4700 1 -- 172.21.15.86:0/3664487265 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f5d640a3cd0 msgr2=0x7f5d640a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:27.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.113+0000 7f5d7fba4700 1 --2- 172.21.15.86:0/3664487265 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f5d640a3cd0 0x7f5d640a6190 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f5d78078f90 tx=0x7f5d6800a300 comp rx=0 tx=0).stop 2024-04-21T08:37:27.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.113+0000 7f5d7fba4700 1 -- 172.21.15.86:0/3664487265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d78101930 msgr2=0x7f5d781980a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:27.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.113+0000 7f5d7fba4700 1 --2- 172.21.15.86:0/3664487265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5d78101930 0x7f5d781980a0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7f5d74000c00 tx=0x7f5d740047b0 comp rx=0 tx=0).stop 2024-04-21T08:37:27.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.113+0000 7f5d7e141700 1 -- 172.21.15.86:0/3664487265 reap_dead start 2024-04-21T08:37:27.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.113+0000 7f5d7fba4700 1 -- 172.21.15.86:0/3664487265 shutdown_connections 2024-04-21T08:37:27.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.113+0000 7f5d7fba4700 1 -- 172.21.15.86:0/3664487265 >> 172.21.15.86:0/3664487265 conn(0x7f5d780788b0 msgr2=0x7f5d781007d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:27.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.113+0000 7f5d7fba4700 1 -- 172.21.15.86:0/3664487265 shutdown_connections 2024-04-21T08:37:27.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:27.113+0000 7f5d7fba4700 1 -- 172.21.15.86:0/3664487265 wait complete. 2024-04-21T08:37:28.103 INFO:teuthology.orchestra.run.smithi086.stdout:343597383707 2024-04-21T08:37:28.104 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.11 2024-04-21T08:37:28.557 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:28 smithi086 ceph-mon[27939]: pgmap v323: 129 pgs: 2 peering, 127 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s; 86 B/s, 0 objects/s recovering 2024-04-21T08:37:28.583 INFO:teuthology.orchestra.run.smithi086.stdout:219043332142 2024-04-21T08:37:28.584 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.7 2024-04-21T08:37:28.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:28 smithi088 ceph-mon[31259]: pgmap v323: 129 pgs: 2 peering, 127 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s; 86 B/s, 0 objects/s recovering 2024-04-21T08:37:28.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:28 smithi179 ceph-mon[31829]: pgmap v323: 129 pgs: 2 peering, 127 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s; 86 B/s, 0 objects/s recovering 2024-04-21T08:37:29.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.039+0000 7fb39ddd6700 1 -- 172.21.15.86:0/111821646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3981026e0 msgr2=0x7fb39810e770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:29.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.039+0000 7fb39ddd6700 1 --2- 172.21.15.86:0/111821646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3981026e0 0x7fb39810e770 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb39000b0a0 tx=0x7fb39000b3b0 comp rx=0 tx=0).stop 2024-04-21T08:37:29.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.039+0000 7fb39ddd6700 1 -- 172.21.15.86:0/111821646 shutdown_connections 2024-04-21T08:37:29.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.039+0000 7fb39ddd6700 1 --2- 172.21.15.86:0/111821646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3981026e0 0x7fb39810e770 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:29.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.039+0000 7fb39ddd6700 1 --2- 172.21.15.86:0/111821646 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb398106990 0x7fb398106d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:29.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.039+0000 7fb39ddd6700 1 --2- 172.21.15.86:0/111821646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb398101d20 0x7fb3981021a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:29.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.039+0000 7fb39ddd6700 1 -- 172.21.15.86:0/111821646 >> 172.21.15.86:0/111821646 conn(0x7fb3980788f0 msgr2=0x7fb398078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:29.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.039+0000 7fb39ddd6700 1 -- 172.21.15.86:0/111821646 shutdown_connections 2024-04-21T08:37:29.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.039+0000 7fb39ddd6700 1 -- 172.21.15.86:0/111821646 wait complete. 2024-04-21T08:37:29.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.039+0000 7fb39ddd6700 1 Processor -- start 2024-04-21T08:37:29.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.040+0000 7fb39ddd6700 1 -- start start 2024-04-21T08:37:29.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.040+0000 7fb39ddd6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb398101d20 0x7fb3981990a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:29.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.040+0000 7fb39ddd6700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3981026e0 0x7fb3981995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:29.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.040+0000 7fb39ddd6700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb398106990 0x7fb39819d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:29.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.040+0000 7fb39ddd6700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb398111b50 con 0x7fb3981026e0 2024-04-21T08:37:29.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.040+0000 7fb39ddd6700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb3981119d0 con 0x7fb398106990 2024-04-21T08:37:29.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.040+0000 7fb39ddd6700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb398111cd0 con 0x7fb398101d20 2024-04-21T08:37:29.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.040+0000 7fb396ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3981026e0 0x7fb3981995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:29.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.040+0000 7fb396ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3981026e0 0x7fb3981995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57504/0 (socket says 172.21.15.86:57504) 2024-04-21T08:37:29.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.040+0000 7fb396ffd700 1 -- 172.21.15.86:0/2359233336 learned_addr learned my addr 172.21.15.86:0/2359233336 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:37:29.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.040+0000 7fb3977fe700 1 --2- 172.21.15.86:0/2359233336 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb398101d20 0x7fb3981990a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:29.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.040+0000 7fb397fff700 1 --2- 172.21.15.86:0/2359233336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb398106990 0x7fb39819d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:29.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.040+0000 7fb396ffd700 1 -- 172.21.15.86:0/2359233336 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb398101d20 msgr2=0x7fb3981990a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:29.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.040+0000 7fb396ffd700 1 --2- 172.21.15.86:0/2359233336 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb398101d20 0x7fb3981990a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:29.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.040+0000 7fb396ffd700 1 -- 172.21.15.86:0/2359233336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb398106990 msgr2=0x7fb39819d970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:29.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.040+0000 7fb396ffd700 1 --2- 172.21.15.86:0/2359233336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb398106990 0x7fb39819d970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:29.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.040+0000 7fb396ffd700 1 -- 172.21.15.86:0/2359233336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb39819e0b0 con 0x7fb3981026e0 2024-04-21T08:37:29.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.040+0000 7fb397fff700 1 --2- 172.21.15.86:0/2359233336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb398106990 0x7fb39819d970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:37:29.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.040+0000 7fb396ffd700 1 --2- 172.21.15.86:0/2359233336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3981026e0 0x7fb3981995e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb38c00c970 tx=0x7fb38c00cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:29.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.041+0000 7fb394ff9700 1 -- 172.21.15.86:0/2359233336 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38c007aa0 con 0x7fb3981026e0 2024-04-21T08:37:29.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.041+0000 7fb394ff9700 1 -- 172.21.15.86:0/2359233336 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb38c004ce0 con 0x7fb3981026e0 2024-04-21T08:37:29.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.041+0000 7fb39ddd6700 1 -- 172.21.15.86:0/2359233336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb39819e3a0 con 0x7fb3981026e0 2024-04-21T08:37:29.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.041+0000 7fb394ff9700 1 -- 172.21.15.86:0/2359233336 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb38c005510 con 0x7fb3981026e0 2024-04-21T08:37:29.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.041+0000 7fb39ddd6700 1 -- 172.21.15.86:0/2359233336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3981a5c20 con 0x7fb3981026e0 2024-04-21T08:37:29.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.041+0000 7fb39ddd6700 1 -- 172.21.15.86:0/2359233336 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=osdmap handle=1) -- 0x7fb398063190 con 0x7fb3981026e0 2024-04-21T08:37:29.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.043+0000 7fb394ff9700 1 -- 172.21.15.86:0/2359233336 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb38c007c40 con 0x7fb3981026e0 2024-04-21T08:37:29.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.043+0000 7fb394ff9700 1 --2- 172.21.15.86:0/2359233336 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb3800a3cd0 0x7fb3800a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:29.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.043+0000 7fb3977fe700 1 --2- 172.21.15.86:0/2359233336 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb3800a3cd0 0x7fb3800a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:29.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.043+0000 7fb394ff9700 1 -- 172.21.15.86:0/2359233336 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb38c0d0ae0 con 0x7fb3981026e0 2024-04-21T08:37:29.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.044+0000 7fb394ff9700 1 --2- 172.21.15.86:0/2359233336 >> [v2:172.21.15.88:6824/1294275353,v1:172.21.15.88:6825/1294275353] conn(0x7fb3800aee30 0x7fb3800b12d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:29.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.044+0000 7fb394ff9700 1 -- 172.21.15.86:0/2359233336 --> [v2:172.21.15.88:6824/1294275353,v1:172.21.15.88:6825/1294275353] -- command(tid 1: {"prefix": "get_command_descriptions"}) -- 0x7fb38c00eec0 con 0x7fb3800aee30 2024-04-21T08:37:29.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.044+0000 7fb394ff9700 1 -- 172.21.15.86:0/2359233336 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_get_version_reply(handle=1 version=91) ==== 24+0+0 (secure 0 0 0) 0x7fb38c0d0f30 con 0x7fb3981026e0 2024-04-21T08:37:29.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.044+0000 7fb3977fe700 1 --2- 172.21.15.86:0/2359233336 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb3800a3cd0 0x7fb3800a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb388009a10 tx=0x7fb388009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:29.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.044+0000 7fb397fff700 1 --2- 172.21.15.86:0/2359233336 >> [v2:172.21.15.88:6824/1294275353,v1:172.21.15.88:6825/1294275353] conn(0x7fb3800aee30 0x7fb3800b12d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:29.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.044+0000 7fb397fff700 1 --2- 172.21.15.86:0/2359233336 >> [v2:172.21.15.88:6824/1294275353,v1:172.21.15.88:6825/1294275353] conn(0x7fb3800aee30 0x7fb3800b12d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=osd.10 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:29.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.045+0000 7fb394ff9700 1 -- 172.21.15.86:0/2359233336 <== osd.10 v2:172.21.15.88:6824/1294275353 1 ==== command_reply(tid 1: 0 ) ==== 8+0+27193 (crc 0 0 0) 0x7fb38c00eec0 con 0x7fb3800aee30 2024-04-21T08:37:29.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.061+0000 7fb39ddd6700 1 -- 172.21.15.86:0/2359233336 --> [v2:172.21.15.88:6824/1294275353,v1:172.21.15.88:6825/1294275353] -- command(tid 2: {"prefix": "flush_pg_stats"}) -- 0x7fb398063610 con 0x7fb3800aee30 2024-04-21T08:37:29.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.062+0000 7fb394ff9700 1 -- 172.21.15.86:0/2359233336 <== osd.10 v2:172.21.15.88:6824/1294275353 2 ==== command_reply(tid 2: 0 ) ==== 8+0+12 (crc 0 0 0) 0x7fb398063610 con 0x7fb3800aee30 2024-04-21T08:37:29.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.062+0000 7fb39ddd6700 1 -- 172.21.15.86:0/2359233336 >> [v2:172.21.15.88:6824/1294275353,v1:172.21.15.88:6825/1294275353] conn(0x7fb3800aee30 msgr2=0x7fb3800b12d0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:29.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.062+0000 7fb39ddd6700 1 --2- 172.21.15.86:0/2359233336 >> [v2:172.21.15.88:6824/1294275353,v1:172.21.15.88:6825/1294275353] conn(0x7fb3800aee30 0x7fb3800b12d0 crc :-1 s=READY pgs=14 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:29.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.062+0000 7fb39ddd6700 1 -- 172.21.15.86:0/2359233336 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb3800a3cd0 msgr2=0x7fb3800a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:29.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.062+0000 7fb39ddd6700 1 --2- 172.21.15.86:0/2359233336 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb3800a3cd0 0x7fb3800a6190 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fb388009a10 tx=0x7fb388009190 comp rx=0 tx=0).stop 2024-04-21T08:37:29.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.062+0000 7fb39ddd6700 1 -- 172.21.15.86:0/2359233336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3981026e0 msgr2=0x7fb3981995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:29.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.062+0000 7fb39ddd6700 1 --2- 172.21.15.86:0/2359233336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3981026e0 0x7fb3981995e0 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fb38c00c970 tx=0x7fb38c00cc80 comp rx=0 tx=0).stop 2024-04-21T08:37:29.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.062+0000 7fb397fff700 1 -- 172.21.15.86:0/2359233336 reap_dead start 2024-04-21T08:37:29.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.063+0000 7fb39ddd6700 1 -- 172.21.15.86:0/2359233336 shutdown_connections 2024-04-21T08:37:29.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.063+0000 7fb39ddd6700 1 -- 172.21.15.86:0/2359233336 >> 172.21.15.86:0/2359233336 conn(0x7fb3980788f0 msgr2=0x7fb39806b3f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:29.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.063+0000 7fb39ddd6700 1 -- 172.21.15.86:0/2359233336 shutdown_connections 2024-04-21T08:37:29.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:29.063+0000 7fb39ddd6700 1 -- 172.21.15.86:0/2359233336 wait complete. 2024-04-21T08:37:29.484 INFO:teuthology.orchestra.run.smithi086.stdout:163208757304 2024-04-21T08:37:29.484 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.5 2024-04-21T08:37:30.164 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:30.167 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:30.167 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:30.234 INFO:teuthology.orchestra.run.smithi086.stdout:309237645343 2024-04-21T08:37:30.234 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph osd last-stat-seq osd.10 2024-04-21T08:37:30.531 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:30 smithi086 ceph-mon[27939]: pgmap v324: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s; 115 B/s, 2 objects/s recovering 2024-04-21T08:37:30.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:30 smithi088 ceph-mon[31259]: pgmap v324: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s; 115 B/s, 2 objects/s recovering 2024-04-21T08:37:30.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:30 smithi179 ceph-mon[31829]: pgmap v324: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 9 op/s; 115 B/s, 2 objects/s recovering 2024-04-21T08:37:32.605 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:32 smithi086 ceph-mon[27939]: pgmap v325: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 115 B/s, 2 objects/s recovering 2024-04-21T08:37:32.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:32 smithi088 ceph-mon[31259]: pgmap v325: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 115 B/s, 2 objects/s recovering 2024-04-21T08:37:32.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:32 smithi179 ceph-mon[31829]: pgmap v325: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s; 115 B/s, 2 objects/s recovering 2024-04-21T08:37:32.955 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:32.955 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:32.956 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:33.800 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:34.106 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:34.107 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:34.107 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:34.108 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:34.488 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:34 smithi086 ceph-mon[27939]: pgmap v326: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 92 B/s, 1 objects/s recovering 2024-04-21T08:37:34.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:34 smithi088 ceph-mon[31259]: pgmap v326: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 92 B/s, 1 objects/s recovering 2024-04-21T08:37:34.737 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:34 smithi179 ceph-mon[31829]: pgmap v326: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 92 B/s, 1 objects/s recovering 2024-04-21T08:37:35.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.632+0000 7f9ac34b2700 1 -- 172.21.15.86:0/468077293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9abc101970 msgr2=0x7f9abc101df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:35.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.632+0000 7f9ac34b2700 1 --2- 172.21.15.86:0/468077293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9abc101970 0x7f9abc101df0 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7f9aac009990 tx=0x7f9aac009ca0 comp rx=0 tx=0).stop 2024-04-21T08:37:35.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.632+0000 7f9ac34b2700 1 -- 172.21.15.86:0/468077293 shutdown_connections 2024-04-21T08:37:35.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.632+0000 7f9ac34b2700 1 --2- 172.21.15.86:0/468077293 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9abc102330 0x7f9abc10e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:35.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.632+0000 7f9ac34b2700 1 --2- 172.21.15.86:0/468077293 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9abc107920 0x7f9abc107d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:35.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.632+0000 7f9ac34b2700 1 --2- 172.21.15.86:0/468077293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9abc101970 0x7f9abc101df0 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:35.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.632+0000 7f9ac34b2700 1 -- 172.21.15.86:0/468077293 >> 172.21.15.86:0/468077293 conn(0x7f9abc0788f0 msgr2=0x7f9abc078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:35.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.632+0000 7f9ac34b2700 1 -- 172.21.15.86:0/468077293 shutdown_connections 2024-04-21T08:37:35.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.632+0000 7f9ac34b2700 1 -- 172.21.15.86:0/468077293 wait complete. 2024-04-21T08:37:35.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.633+0000 7f9ac34b2700 1 Processor -- start 2024-04-21T08:37:35.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.633+0000 7f9ac34b2700 1 -- start start 2024-04-21T08:37:35.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.633+0000 7f9ac34b2700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9abc101970 0x7f9abc199190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:35.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.633+0000 7f9ac34b2700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9abc102330 0x7f9abc1996d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:35.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.633+0000 7f9ac34b2700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9abc107920 0x7f9abc19da60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:35.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.633+0000 7f9ac34b2700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc111b30 con 0x7f9abc107920 2024-04-21T08:37:35.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.633+0000 7f9ac34b2700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc1119b0 con 0x7f9abc102330 2024-04-21T08:37:35.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.633+0000 7f9ac34b2700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9abc111cb0 con 0x7f9abc101970 2024-04-21T08:37:35.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.633+0000 7f9ac1a4f700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9abc107920 0x7f9abc19da60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:35.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.633+0000 7f9ac1a4f700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9abc107920 0x7f9abc19da60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37124/0 (socket says 172.21.15.86:37124) 2024-04-21T08:37:35.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.633+0000 7f9ac1a4f700 1 -- 172.21.15.86:0/2696835827 learned_addr learned my addr 172.21.15.86:0/2696835827 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:37:35.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.633+0000 7f9ac0a4d700 1 --2- 172.21.15.86:0/2696835827 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9abc102330 0x7f9abc1996d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:35.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.634+0000 7f9ac124e700 1 --2- 172.21.15.86:0/2696835827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9abc101970 0x7f9abc199190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:35.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.634+0000 7f9ac1a4f700 1 -- 172.21.15.86:0/2696835827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9abc101970 msgr2=0x7f9abc199190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:35.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.634+0000 7f9ac1a4f700 1 --2- 172.21.15.86:0/2696835827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9abc101970 0x7f9abc199190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:35.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.634+0000 7f9ac1a4f700 1 -- 172.21.15.86:0/2696835827 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9abc102330 msgr2=0x7f9abc1996d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:35.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.634+0000 7f9ac1a4f700 1 --2- 172.21.15.86:0/2696835827 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9abc102330 0x7f9abc1996d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:35.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.634+0000 7f9ac1a4f700 1 -- 172.21.15.86:0/2696835827 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9abc19e1a0 con 0x7f9abc107920 2024-04-21T08:37:35.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.634+0000 7f9ac124e700 1 --2- 172.21.15.86:0/2696835827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9abc101970 0x7f9abc199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:37:35.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.634+0000 7f9ac0a4d700 1 --2- 172.21.15.86:0/2696835827 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9abc102330 0x7f9abc1996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:37:35.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.634+0000 7f9ac1a4f700 1 --2- 172.21.15.86:0/2696835827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9abc107920 0x7f9abc19da60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4007ee0 tx=0x7f9ab400d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:35.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.634+0000 7f9ab27fc700 1 -- 172.21.15.86:0/2696835827 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab400dbd0 con 0x7f9abc107920 2024-04-21T08:37:35.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.634+0000 7f9ab27fc700 1 -- 172.21.15.86:0/2696835827 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9ab400dd70 con 0x7f9abc107920 2024-04-21T08:37:35.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.634+0000 7f9ac34b2700 1 -- 172.21.15.86:0/2696835827 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9abc19e490 con 0x7f9abc107920 2024-04-21T08:37:35.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.634+0000 7f9ab27fc700 1 -- 172.21.15.86:0/2696835827 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9ab4005990 con 0x7f9abc107920 2024-04-21T08:37:35.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.634+0000 7f9ac34b2700 1 -- 172.21.15.86:0/2696835827 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9abc1a5d10 con 0x7f9abc107920 2024-04-21T08:37:35.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.635+0000 7f9ac34b2700 1 -- 172.21.15.86:0/2696835827 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9abc069730 con 0x7f9abc107920 2024-04-21T08:37:35.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.636+0000 7f9ab27fc700 1 -- 172.21.15.86:0/2696835827 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f9ab401a020 con 0x7f9abc107920 2024-04-21T08:37:35.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.637+0000 7f9ab27fc700 1 --2- 172.21.15.86:0/2696835827 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f9aa80a3cd0 0x7f9aa80a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:35.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.637+0000 7f9ac124e700 1 --2- 172.21.15.86:0/2696835827 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f9aa80a3cd0 0x7f9aa80a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:35.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.637+0000 7f9ab27fc700 1 -- 172.21.15.86:0/2696835827 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f9ab40d06d0 con 0x7f9abc107920 2024-04-21T08:37:35.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.637+0000 7f9ac124e700 1 --2- 172.21.15.86:0/2696835827 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f9aa80a3cd0 0x7f9aa80a6190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8009c90 tx=0x7f9ab8009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:35.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.641+0000 7f9ab27fc700 1 -- 172.21.15.86:0/2696835827 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9ab4010070 con 0x7f9abc107920 2024-04-21T08:37:35.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.855+0000 7f9ac34b2700 1 -- 172.21.15.86:0/2696835827 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 0} v 0) -- 0x7f9abc02df80 con 0x7f9abc107920 2024-04-21T08:37:35.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.855+0000 7f9ab27fc700 1 -- 172.21.15.86:0/2696835827 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 0}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f9ab409cbf0 con 0x7f9abc107920 2024-04-21T08:37:35.854 INFO:teuthology.orchestra.run.smithi086.stdout:34359738451 2024-04-21T08:37:35.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.857+0000 7f9ac34b2700 1 -- 172.21.15.86:0/2696835827 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f9aa80a3cd0 msgr2=0x7f9aa80a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:35.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.857+0000 7f9ac34b2700 1 --2- 172.21.15.86:0/2696835827 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f9aa80a3cd0 0x7f9aa80a6190 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7f9ab8009c90 tx=0x7f9ab8009260 comp rx=0 tx=0).stop 2024-04-21T08:37:35.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.857+0000 7f9ac34b2700 1 -- 172.21.15.86:0/2696835827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9abc107920 msgr2=0x7f9abc19da60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:35.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.857+0000 7f9ac34b2700 1 --2- 172.21.15.86:0/2696835827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9abc107920 0x7f9abc19da60 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7f9ab4007ee0 tx=0x7f9ab400d3b0 comp rx=0 tx=0).stop 2024-04-21T08:37:35.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.857+0000 7f9ac34b2700 1 -- 172.21.15.86:0/2696835827 shutdown_connections 2024-04-21T08:37:35.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.857+0000 7f9ac34b2700 1 --2- 172.21.15.86:0/2696835827 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f9aa80a3cd0 0x7f9aa80a6190 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:35.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.857+0000 7f9ac34b2700 1 --2- 172.21.15.86:0/2696835827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9abc107920 0x7f9abc19da60 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:35.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.857+0000 7f9ac34b2700 1 --2- 172.21.15.86:0/2696835827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9abc101970 0x7f9abc199190 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:35.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.857+0000 7f9ac34b2700 1 --2- 172.21.15.86:0/2696835827 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9abc102330 0x7f9abc1996d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:35.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.857+0000 7f9ac34b2700 1 -- 172.21.15.86:0/2696835827 >> 172.21.15.86:0/2696835827 conn(0x7f9abc0788f0 msgr2=0x7f9abc06b510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:35.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.857+0000 7f9ac34b2700 1 -- 172.21.15.86:0/2696835827 shutdown_connections 2024-04-21T08:37:35.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:35.857+0000 7f9ac34b2700 1 -- 172.21.15.86:0/2696835827 wait complete. 2024-04-21T08:37:36.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:36 smithi086 ceph-mon[27939]: pgmap v327: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s; 83 B/s, 1 objects/s recovering 2024-04-21T08:37:36.467 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:36 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2696835827' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-21T08:37:36.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:36 smithi088 ceph-mon[31259]: pgmap v327: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s; 83 B/s, 1 objects/s recovering 2024-04-21T08:37:36.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:36 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2696835827' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-21T08:37:36.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:36 smithi179 ceph-mon[31829]: pgmap v327: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.5 KiB/s rd, 10 op/s; 83 B/s, 1 objects/s recovering 2024-04-21T08:37:36.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:36 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2696835827' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 0}]: dispatch 2024-04-21T08:37:38.579 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:38 smithi086 ceph-mon[27939]: pgmap v328: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 20 B/s, 1 objects/s recovering 2024-04-21T08:37:38.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:38 smithi088 ceph-mon[31259]: pgmap v328: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 20 B/s, 1 objects/s recovering 2024-04-21T08:37:38.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:38 smithi179 ceph-mon[31829]: pgmap v328: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s; 20 B/s, 1 objects/s recovering 2024-04-21T08:37:39.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.223+0000 7f796e20c700 1 -- 172.21.15.86:0/2578963171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7968108a50 msgr2=0x7f7968108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:39.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.223+0000 7f796e20c700 1 --2- 172.21.15.86:0/2578963171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7968108a50 0x7f7968108e30 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7f7958009990 tx=0x7f7958009ca0 comp rx=0 tx=0).stop 2024-04-21T08:37:39.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.223+0000 7f796e20c700 1 -- 172.21.15.86:0/2578963171 shutdown_connections 2024-04-21T08:37:39.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.223+0000 7f796e20c700 1 --2- 172.21.15.86:0/2578963171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7968103450 0x7f796810f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:39.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.223+0000 7f796e20c700 1 --2- 172.21.15.86:0/2578963171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7968108a50 0x7f7968108e30 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:39.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.223+0000 7f796e20c700 1 --2- 172.21.15.86:0/2578963171 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7968102a90 0x7f7968102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:39.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.223+0000 7f796e20c700 1 -- 172.21.15.86:0/2578963171 >> 172.21.15.86:0/2578963171 conn(0x7f7968079ba0 msgr2=0x7f7968079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:39.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.223+0000 7f796e20c700 1 -- 172.21.15.86:0/2578963171 shutdown_connections 2024-04-21T08:37:39.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.223+0000 7f796e20c700 1 -- 172.21.15.86:0/2578963171 wait complete. 2024-04-21T08:37:39.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.224+0000 7f796e20c700 1 Processor -- start 2024-04-21T08:37:39.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.224+0000 7f796e20c700 1 -- start start 2024-04-21T08:37:39.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.224+0000 7f796e20c700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7968102a90 0x7f7968074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:39.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.224+0000 7f796e20c700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7968103450 0x7f7968075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:39.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.224+0000 7f796e20c700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7968108a50 0x7f79680757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:39.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.224+0000 7f796e20c700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7968112be0 con 0x7f7968102a90 2024-04-21T08:37:39.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.224+0000 7f796e20c700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7968112a60 con 0x7f7968103450 2024-04-21T08:37:39.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.224+0000 7f796e20c700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7968112d60 con 0x7f7968108a50 2024-04-21T08:37:39.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.224+0000 7f79677fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7968102a90 0x7f7968074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:39.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.224+0000 7f79677fe700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7968102a90 0x7f7968074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37142/0 (socket says 172.21.15.86:37142) 2024-04-21T08:37:39.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.224+0000 7f79677fe700 1 -- 172.21.15.86:0/1568002976 learned_addr learned my addr 172.21.15.86:0/1568002976 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:37:39.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.224+0000 7f7967fff700 1 --2- 172.21.15.86:0/1568002976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7968108a50 0x7f79680757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:39.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.224+0000 7f795ffff700 1 --2- 172.21.15.86:0/1568002976 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7968103450 0x7f7968075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:39.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.224+0000 7f79677fe700 1 -- 172.21.15.86:0/1568002976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7968108a50 msgr2=0x7f79680757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:39.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.224+0000 7f79677fe700 1 --2- 172.21.15.86:0/1568002976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7968108a50 0x7f79680757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:39.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.224+0000 7f79677fe700 1 -- 172.21.15.86:0/1568002976 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7968103450 msgr2=0x7f7968075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:39.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.224+0000 7f79677fe700 1 --2- 172.21.15.86:0/1568002976 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7968103450 0x7f7968075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:39.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.224+0000 7f79677fe700 1 -- 172.21.15.86:0/1568002976 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79681a7620 con 0x7f7968102a90 2024-04-21T08:37:39.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.225+0000 7f79677fe700 1 --2- 172.21.15.86:0/1568002976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7968102a90 0x7f7968074d40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7958000c00 tx=0x7f79580047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:39.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.225+0000 7f79657fa700 1 -- 172.21.15.86:0/1568002976 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7958027070 con 0x7f7968102a90 2024-04-21T08:37:39.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.225+0000 7f79657fa700 1 -- 172.21.15.86:0/1568002976 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7958003f20 con 0x7f7968102a90 2024-04-21T08:37:39.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.225+0000 7f796e20c700 1 -- 172.21.15.86:0/1568002976 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79681a78b0 con 0x7f7968102a90 2024-04-21T08:37:39.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.225+0000 7f79657fa700 1 -- 172.21.15.86:0/1568002976 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7958022ca0 con 0x7f7968102a90 2024-04-21T08:37:39.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.225+0000 7f796e20c700 1 -- 172.21.15.86:0/1568002976 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79681a7de0 con 0x7f7968102a90 2024-04-21T08:37:39.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.226+0000 7f796e20c700 1 -- 172.21.15.86:0/1568002976 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7968069730 con 0x7f7968102a90 2024-04-21T08:37:39.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.227+0000 7f79657fa700 1 -- 172.21.15.86:0/1568002976 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f795802a070 con 0x7f7968102a90 2024-04-21T08:37:39.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.228+0000 7f79657fa700 1 --2- 172.21.15.86:0/1568002976 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f79540a3d50 0x7f79540a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:39.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.228+0000 7f795ffff700 1 --2- 172.21.15.86:0/1568002976 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f79540a3d50 0x7f79540a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:39.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.228+0000 7f79657fa700 1 -- 172.21.15.86:0/1568002976 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f79580d1660 con 0x7f7968102a90 2024-04-21T08:37:39.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.228+0000 7f795ffff700 1 --2- 172.21.15.86:0/1568002976 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f79540a3d50 0x7f79540a6210 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f796807a260 tx=0x7f795000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:39.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.232+0000 7f79657fa700 1 -- 172.21.15.86:0/1568002976 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f795805eff0 con 0x7f7968102a90 2024-04-21T08:37:39.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.441+0000 7f796e20c700 1 -- 172.21.15.86:0/1568002976 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 6} v 0) -- 0x7f796802df80 con 0x7f7968102a90 2024-04-21T08:37:39.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.442+0000 7f79657fa700 1 -- 172.21.15.86:0/1568002976 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 6}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f7958013070 con 0x7f7968102a90 2024-04-21T08:37:39.441 INFO:teuthology.orchestra.run.smithi086.stdout:197568495671 2024-04-21T08:37:39.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.444+0000 7f796e20c700 1 -- 172.21.15.86:0/1568002976 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f79540a3d50 msgr2=0x7f79540a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:39.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.444+0000 7f796e20c700 1 --2- 172.21.15.86:0/1568002976 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f79540a3d50 0x7f79540a6210 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f796807a260 tx=0x7f795000a380 comp rx=0 tx=0).stop 2024-04-21T08:37:39.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.444+0000 7f796e20c700 1 -- 172.21.15.86:0/1568002976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7968102a90 msgr2=0x7f7968074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:39.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.444+0000 7f796e20c700 1 --2- 172.21.15.86:0/1568002976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7968102a90 0x7f7968074d40 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f7958000c00 tx=0x7f79580047b0 comp rx=0 tx=0).stop 2024-04-21T08:37:39.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.444+0000 7f796e20c700 1 -- 172.21.15.86:0/1568002976 shutdown_connections 2024-04-21T08:37:39.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.444+0000 7f796e20c700 1 --2- 172.21.15.86:0/1568002976 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f79540a3d50 0x7f79540a6210 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:39.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.444+0000 7f796e20c700 1 --2- 172.21.15.86:0/1568002976 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7968108a50 0x7f79680757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:39.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.444+0000 7f796e20c700 1 --2- 172.21.15.86:0/1568002976 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7968102a90 0x7f7968074d40 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:39.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.444+0000 7f796e20c700 1 --2- 172.21.15.86:0/1568002976 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7968103450 0x7f7968075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:39.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.444+0000 7f796e20c700 1 -- 172.21.15.86:0/1568002976 >> 172.21.15.86:0/1568002976 conn(0x7f7968079ba0 msgr2=0x7f79681017d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:39.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.444+0000 7f796e20c700 1 -- 172.21.15.86:0/1568002976 shutdown_connections 2024-04-21T08:37:39.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:39.444+0000 7f796e20c700 1 -- 172.21.15.86:0/1568002976 wait complete. 2024-04-21T08:37:40.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:40 smithi086 ceph-mon[27939]: pgmap v329: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 19 B/s, 0 objects/s recovering 2024-04-21T08:37:40.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:40 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1568002976' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-21T08:37:40.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:40 smithi088 ceph-mon[31259]: pgmap v329: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 19 B/s, 0 objects/s recovering 2024-04-21T08:37:40.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:40 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1568002976' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-21T08:37:40.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:40 smithi179 ceph-mon[31829]: pgmap v329: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s; 19 B/s, 0 objects/s recovering 2024-04-21T08:37:40.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:40 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1568002976' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 6}]: dispatch 2024-04-21T08:37:41.689 INFO:tasks.cephadm.ceph_manager.ceph:need seq 34359738447 got 34359738451 for osd.0 2024-04-21T08:37:41.689 DEBUG:teuthology.parallel:result is None 2024-04-21T08:37:42.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.453+0000 7f30d2774700 1 -- 172.21.15.86:0/2684816934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30cc107b00 msgr2=0x7f30cc107ee0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.453+0000 7f30d2774700 1 --2- 172.21.15.86:0/2684816934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30cc107b00 0x7f30cc107ee0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7f30bc009990 tx=0x7f30bc009ca0 comp rx=0 tx=0).stop 2024-04-21T08:37:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.453+0000 7f30d2774700 1 -- 172.21.15.86:0/2684816934 shutdown_connections 2024-04-21T08:37:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.453+0000 7f30d2774700 1 --2- 172.21.15.86:0/2684816934 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30cc101da0 0x7f30cc10f810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.453+0000 7f30d2774700 1 --2- 172.21.15.86:0/2684816934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30cc107b00 0x7f30cc107ee0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.453+0000 7f30d2774700 1 --2- 172.21.15.86:0/2684816934 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30cc1013e0 0x7f30cc101860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.453+0000 7f30d2774700 1 -- 172.21.15.86:0/2684816934 >> 172.21.15.86:0/2684816934 conn(0x7f30cc076c60 msgr2=0x7f30cc077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.453+0000 7f30d2774700 1 -- 172.21.15.86:0/2684816934 shutdown_connections 2024-04-21T08:37:42.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.453+0000 7f30d2774700 1 -- 172.21.15.86:0/2684816934 wait complete. 2024-04-21T08:37:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.454+0000 7f30d2774700 1 Processor -- start 2024-04-21T08:37:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.454+0000 7f30d2774700 1 -- start start 2024-04-21T08:37:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.454+0000 7f30d2774700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30cc1013e0 0x7f30cc199140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.454+0000 7f30d2774700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30cc101da0 0x7f30cc199680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.454+0000 7f30d2774700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30cc107b00 0x7f30cc19da10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.454+0000 7f30d2774700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc112c90 con 0x7f30cc107b00 2024-04-21T08:37:42.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.454+0000 7f30d2774700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc112b10 con 0x7f30cc101da0 2024-04-21T08:37:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.454+0000 7f30d2774700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f30cc112e10 con 0x7f30cc1013e0 2024-04-21T08:37:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.454+0000 7f30d0d11700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30cc107b00 0x7f30cc19da10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.455+0000 7f30d0d11700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30cc107b00 0x7f30cc19da10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43932/0 (socket says 172.21.15.86:43932) 2024-04-21T08:37:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.455+0000 7f30d0d11700 1 -- 172.21.15.86:0/1477440483 learned_addr learned my addr 172.21.15.86:0/1477440483 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:37:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.455+0000 7f30cbfff700 1 --2- 172.21.15.86:0/1477440483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30cc1013e0 0x7f30cc199140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.455+0000 7f30cb7fe700 1 --2- 172.21.15.86:0/1477440483 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30cc101da0 0x7f30cc199680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.455+0000 7f30d0d11700 1 -- 172.21.15.86:0/1477440483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30cc1013e0 msgr2=0x7f30cc199140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.455+0000 7f30d0d11700 1 --2- 172.21.15.86:0/1477440483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30cc1013e0 0x7f30cc199140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.455+0000 7f30d0d11700 1 -- 172.21.15.86:0/1477440483 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30cc101da0 msgr2=0x7f30cc199680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.455+0000 7f30d0d11700 1 --2- 172.21.15.86:0/1477440483 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30cc101da0 0x7f30cc199680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.455+0000 7f30d0d11700 1 -- 172.21.15.86:0/1477440483 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f30cc19e150 con 0x7f30cc107b00 2024-04-21T08:37:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.455+0000 7f30d0d11700 1 --2- 172.21.15.86:0/1477440483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30cc107b00 0x7f30cc19da10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f30c400b970 tx=0x7f30c400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:42.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.455+0000 7f30c97fa700 1 -- 172.21.15.86:0/1477440483 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c400d860 con 0x7f30cc107b00 2024-04-21T08:37:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.455+0000 7f30c97fa700 1 -- 172.21.15.86:0/1477440483 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f30c400e430 con 0x7f30cc107b00 2024-04-21T08:37:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.455+0000 7f30d2774700 1 -- 172.21.15.86:0/1477440483 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f30cc19e440 con 0x7f30cc107b00 2024-04-21T08:37:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.455+0000 7f30c97fa700 1 -- 172.21.15.86:0/1477440483 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f30c4017660 con 0x7f30cc107b00 2024-04-21T08:37:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.455+0000 7f30d2774700 1 -- 172.21.15.86:0/1477440483 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f30cc1a5cc0 con 0x7f30cc107b00 2024-04-21T08:37:42.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.456+0000 7f30d2774700 1 -- 172.21.15.86:0/1477440483 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f30cc069730 con 0x7f30cc107b00 2024-04-21T08:37:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.457+0000 7f30c97fa700 1 -- 172.21.15.86:0/1477440483 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f30c400da00 con 0x7f30cc107b00 2024-04-21T08:37:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.458+0000 7f30c97fa700 1 --2- 172.21.15.86:0/1477440483 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f30b40a3d50 0x7f30b40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.458+0000 7f30c97fa700 1 -- 172.21.15.86:0/1477440483 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f30c40d05b0 con 0x7f30cc107b00 2024-04-21T08:37:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.458+0000 7f30cbfff700 1 --2- 172.21.15.86:0/1477440483 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f30b40a3d50 0x7f30b40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:42.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.458+0000 7f30cbfff700 1 --2- 172.21.15.86:0/1477440483 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f30b40a3d50 0x7f30b40a6210 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f30bc006010 tx=0x7f30bc005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:42.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.462+0000 7f30c97fa700 1 -- 172.21.15.86:0/1477440483 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f30c4013030 con 0x7f30cc107b00 2024-04-21T08:37:42.562 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:42 smithi086 ceph-mon[27939]: pgmap v330: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-21T08:37:42.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.673+0000 7f30d2774700 1 -- 172.21.15.86:0/1477440483 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 9} v 0) -- 0x7f30cc02df80 con 0x7f30cc107b00 2024-04-21T08:37:42.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.673+0000 7f30c97fa700 1 -- 172.21.15.86:0/1477440483 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 9}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f30c409cb50 con 0x7f30cc107b00 2024-04-21T08:37:42.673 INFO:teuthology.orchestra.run.smithi086.stdout:287762808872 2024-04-21T08:37:42.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.675+0000 7f30d2774700 1 -- 172.21.15.86:0/1477440483 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f30b40a3d50 msgr2=0x7f30b40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:42.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.675+0000 7f30d2774700 1 --2- 172.21.15.86:0/1477440483 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f30b40a3d50 0x7f30b40a6210 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7f30bc006010 tx=0x7f30bc005960 comp rx=0 tx=0).stop 2024-04-21T08:37:42.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.675+0000 7f30d2774700 1 -- 172.21.15.86:0/1477440483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30cc107b00 msgr2=0x7f30cc19da10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:42.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.675+0000 7f30d2774700 1 --2- 172.21.15.86:0/1477440483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30cc107b00 0x7f30cc19da10 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7f30c400b970 tx=0x7f30c400bc80 comp rx=0 tx=0).stop 2024-04-21T08:37:42.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.676+0000 7f30d2774700 1 -- 172.21.15.86:0/1477440483 shutdown_connections 2024-04-21T08:37:42.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.676+0000 7f30d2774700 1 --2- 172.21.15.86:0/1477440483 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f30b40a3d50 0x7f30b40a6210 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:42.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.676+0000 7f30d2774700 1 --2- 172.21.15.86:0/1477440483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f30cc107b00 0x7f30cc19da10 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:42.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.676+0000 7f30d2774700 1 --2- 172.21.15.86:0/1477440483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f30cc1013e0 0x7f30cc199140 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:42.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.676+0000 7f30d2774700 1 --2- 172.21.15.86:0/1477440483 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f30cc101da0 0x7f30cc199680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:42.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.676+0000 7f30d2774700 1 -- 172.21.15.86:0/1477440483 >> 172.21.15.86:0/1477440483 conn(0x7f30cc076c60 msgr2=0x7f30cc0fffc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:42.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.676+0000 7f30d2774700 1 -- 172.21.15.86:0/1477440483 shutdown_connections 2024-04-21T08:37:42.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:42.676+0000 7f30d2774700 1 -- 172.21.15.86:0/1477440483 wait complete. 2024-04-21T08:37:42.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:42 smithi088 ceph-mon[31259]: pgmap v330: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-21T08:37:42.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:42 smithi179 ceph-mon[31829]: pgmap v330: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 9 op/s 2024-04-21T08:37:43.568 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:43 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1477440483' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-21T08:37:43.600 INFO:tasks.cephadm.ceph_manager.ceph:need seq 287762808866 got 287762808872 for osd.9 2024-04-21T08:37:43.600 DEBUG:teuthology.parallel:result is None 2024-04-21T08:37:43.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:43 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1477440483' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-21T08:37:43.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:43 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1477440483' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 9}]: dispatch 2024-04-21T08:37:43.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.788+0000 7fbd36256700 1 -- 172.21.15.86:0/690581286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd3006bb10 msgr2=0x7fbd3010f900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:43.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.788+0000 7fbd36256700 1 --2- 172.21.15.86:0/690581286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd3006bb10 0x7fbd3010f900 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7fbd2800b0a0 tx=0x7fbd2800b3b0 comp rx=0 tx=0).stop 2024-04-21T08:37:43.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.788+0000 7fbd36256700 1 -- 172.21.15.86:0/690581286 shutdown_connections 2024-04-21T08:37:43.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.788+0000 7fbd36256700 1 --2- 172.21.15.86:0/690581286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd3006bb10 0x7fbd3010f900 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:43.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.788+0000 7fbd36256700 1 --2- 172.21.15.86:0/690581286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd30108a30 0x7fbd30108e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:43.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.788+0000 7fbd36256700 1 --2- 172.21.15.86:0/690581286 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd3006b170 0x7fbd3006b5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:43.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.788+0000 7fbd36256700 1 -- 172.21.15.86:0/690581286 >> 172.21.15.86:0/690581286 conn(0x7fbd300788f0 msgr2=0x7fbd30078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:43.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.788+0000 7fbd36256700 1 -- 172.21.15.86:0/690581286 shutdown_connections 2024-04-21T08:37:43.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.788+0000 7fbd36256700 1 -- 172.21.15.86:0/690581286 wait complete. 2024-04-21T08:37:43.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.789+0000 7fbd36256700 1 Processor -- start 2024-04-21T08:37:43.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.789+0000 7fbd36256700 1 -- start start 2024-04-21T08:37:43.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.789+0000 7fbd36256700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd3006b170 0x7fbd3019b240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:43.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.789+0000 7fbd36256700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd3006bb10 0x7fbd3019b780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:43.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.789+0000 7fbd36256700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd30108a30 0x7fbd3019fb10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:43.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.789+0000 7fbd36256700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbd30112bd0 con 0x7fbd30108a30 2024-04-21T08:37:43.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.789+0000 7fbd36256700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbd30112a50 con 0x7fbd3006b170 2024-04-21T08:37:43.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.789+0000 7fbd36256700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbd30112d50 con 0x7fbd3006bb10 2024-04-21T08:37:43.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.789+0000 7fbd2ffff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd30108a30 0x7fbd3019fb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:43.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.789+0000 7fbd2f7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd3006b170 0x7fbd3019b240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:43.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.789+0000 7fbd2ffff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd30108a30 0x7fbd3019fb10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43962/0 (socket says 172.21.15.86:43962) 2024-04-21T08:37:43.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.789+0000 7fbd2f7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd3006b170 0x7fbd3019b240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.86:37998/0 (socket says 172.21.15.86:37998) 2024-04-21T08:37:43.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.789+0000 7fbd2ffff700 1 -- 172.21.15.86:0/3433138622 learned_addr learned my addr 172.21.15.86:0/3433138622 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:37:43.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.789+0000 7fbd2effd700 1 --2- 172.21.15.86:0/3433138622 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd3006bb10 0x7fbd3019b780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:43.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.790+0000 7fbd2ffff700 1 -- 172.21.15.86:0/3433138622 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd3006bb10 msgr2=0x7fbd3019b780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:43.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.790+0000 7fbd2ffff700 1 --2- 172.21.15.86:0/3433138622 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd3006bb10 0x7fbd3019b780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:43.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.790+0000 7fbd2ffff700 1 -- 172.21.15.86:0/3433138622 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd3006b170 msgr2=0x7fbd3019b240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:43.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.790+0000 7fbd2ffff700 1 --2- 172.21.15.86:0/3433138622 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd3006b170 0x7fbd3019b240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:43.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.790+0000 7fbd2ffff700 1 -- 172.21.15.86:0/3433138622 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbd301a0250 con 0x7fbd30108a30 2024-04-21T08:37:43.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.790+0000 7fbd2effd700 1 --2- 172.21.15.86:0/3433138622 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd3006bb10 0x7fbd3019b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:37:43.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.790+0000 7fbd2ffff700 1 --2- 172.21.15.86:0/3433138622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd30108a30 0x7fbd3019fb10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbd280062a0 tx=0x7fbd28004e20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:43.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.790+0000 7fbd2cff9700 1 -- 172.21.15.86:0/3433138622 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd28021070 con 0x7fbd30108a30 2024-04-21T08:37:43.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.790+0000 7fbd2cff9700 1 -- 172.21.15.86:0/3433138622 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbd28007c80 con 0x7fbd30108a30 2024-04-21T08:37:43.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.790+0000 7fbd36256700 1 -- 172.21.15.86:0/3433138622 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbd301a04e0 con 0x7fbd30108a30 2024-04-21T08:37:43.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.790+0000 7fbd2cff9700 1 -- 172.21.15.86:0/3433138622 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbd2801d450 con 0x7fbd30108a30 2024-04-21T08:37:43.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.790+0000 7fbd36256700 1 -- 172.21.15.86:0/3433138622 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbd301a7dc0 con 0x7fbd30108a30 2024-04-21T08:37:43.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.791+0000 7fbd36256700 1 -- 172.21.15.86:0/3433138622 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbd30069730 con 0x7fbd30108a30 2024-04-21T08:37:43.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.792+0000 7fbd2cff9700 1 -- 172.21.15.86:0/3433138622 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fbd28004000 con 0x7fbd30108a30 2024-04-21T08:37:43.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.793+0000 7fbd2cff9700 1 --2- 172.21.15.86:0/3433138622 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fbd180a3cd0 0x7fbd180a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:43.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.793+0000 7fbd2cff9700 1 -- 172.21.15.86:0/3433138622 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fbd2800f020 con 0x7fbd30108a30 2024-04-21T08:37:43.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.793+0000 7fbd2f7fe700 1 --2- 172.21.15.86:0/3433138622 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fbd180a3cd0 0x7fbd180a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:43.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.794+0000 7fbd2f7fe700 1 --2- 172.21.15.86:0/3433138622 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fbd180a3cd0 0x7fbd180a6190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbd2000abd0 tx=0x7fbd2000a380 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:43.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:43.799+0000 7fbd2cff9700 1 -- 172.21.15.86:0/3433138622 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbd2802a020 con 0x7fbd30108a30 2024-04-21T08:37:44.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:44.010+0000 7fbd36256700 1 -- 172.21.15.86:0/3433138622 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 8} v 0) -- 0x7fbd3002df80 con 0x7fbd30108a30 2024-04-21T08:37:44.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:44.011+0000 7fbd2cff9700 1 -- 172.21.15.86:0/3433138622 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 8}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fbd28098e20 con 0x7fbd30108a30 2024-04-21T08:37:44.010 INFO:teuthology.orchestra.run.smithi086.stdout:253403070509 2024-04-21T08:37:44.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:44.013+0000 7fbd36256700 1 -- 172.21.15.86:0/3433138622 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fbd180a3cd0 msgr2=0x7fbd180a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:44.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:44.013+0000 7fbd36256700 1 --2- 172.21.15.86:0/3433138622 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fbd180a3cd0 0x7fbd180a6190 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fbd2000abd0 tx=0x7fbd2000a380 comp rx=0 tx=0).stop 2024-04-21T08:37:44.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:44.013+0000 7fbd36256700 1 -- 172.21.15.86:0/3433138622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd30108a30 msgr2=0x7fbd3019fb10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:44.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:44.013+0000 7fbd36256700 1 --2- 172.21.15.86:0/3433138622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd30108a30 0x7fbd3019fb10 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7fbd280062a0 tx=0x7fbd28004e20 comp rx=0 tx=0).stop 2024-04-21T08:37:44.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:44.013+0000 7fbd36256700 1 -- 172.21.15.86:0/3433138622 shutdown_connections 2024-04-21T08:37:44.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:44.013+0000 7fbd36256700 1 --2- 172.21.15.86:0/3433138622 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fbd180a3cd0 0x7fbd180a6190 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:44.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:44.013+0000 7fbd36256700 1 --2- 172.21.15.86:0/3433138622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbd30108a30 0x7fbd3019fb10 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:44.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:44.013+0000 7fbd36256700 1 --2- 172.21.15.86:0/3433138622 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbd3006b170 0x7fbd3019b240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:44.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:44.013+0000 7fbd36256700 1 --2- 172.21.15.86:0/3433138622 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbd3006bb10 0x7fbd3019b780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:44.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:44.013+0000 7fbd36256700 1 -- 172.21.15.86:0/3433138622 >> 172.21.15.86:0/3433138622 conn(0x7fbd300788f0 msgr2=0x7fbd3010ddf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:44.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:44.013+0000 7fbd36256700 1 -- 172.21.15.86:0/3433138622 shutdown_connections 2024-04-21T08:37:44.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:44.013+0000 7fbd36256700 1 -- 172.21.15.86:0/3433138622 wait complete. 2024-04-21T08:37:44.476 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:44 smithi086 ceph-mon[27939]: pgmap v331: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-21T08:37:44.476 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:44 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3433138622' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-21T08:37:44.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:44 smithi179 ceph-mon[31829]: pgmap v331: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-21T08:37:44.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:44 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3433138622' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-21T08:37:44.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:44 smithi088 ceph-mon[31259]: pgmap v331: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-21T08:37:44.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:44 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3433138622' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 8}]: dispatch 2024-04-21T08:37:45.207 INFO:tasks.cephadm.ceph_manager.ceph:need seq 253403070505 got 253403070509 for osd.8 2024-04-21T08:37:45.207 DEBUG:teuthology.parallel:result is None 2024-04-21T08:37:45.615 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:45.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.822+0000 7f4e3652a700 1 -- 172.21.15.86:0/3258143253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e30103450 msgr2=0x7f4e3010f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:45.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.822+0000 7f4e3652a700 1 --2- 172.21.15.86:0/3258143253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e30103450 0x7f4e3010f8b0 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7f4e2800b0a0 tx=0x7f4e2800b3b0 comp rx=0 tx=0).stop 2024-04-21T08:37:45.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.822+0000 7f4e3652a700 1 -- 172.21.15.86:0/3258143253 shutdown_connections 2024-04-21T08:37:45.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.822+0000 7f4e3652a700 1 --2- 172.21.15.86:0/3258143253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e30103450 0x7f4e3010f8b0 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:45.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.822+0000 7f4e3652a700 1 --2- 172.21.15.86:0/3258143253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e30108a50 0x7f4e30108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:45.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.822+0000 7f4e3652a700 1 --2- 172.21.15.86:0/3258143253 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4e30102a90 0x7f4e30102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:45.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.822+0000 7f4e3652a700 1 -- 172.21.15.86:0/3258143253 >> 172.21.15.86:0/3258143253 conn(0x7f4e30079ba0 msgr2=0x7f4e30079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:45.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.822+0000 7f4e3652a700 1 -- 172.21.15.86:0/3258143253 shutdown_connections 2024-04-21T08:37:45.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.822+0000 7f4e3652a700 1 -- 172.21.15.86:0/3258143253 wait complete. 2024-04-21T08:37:45.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.823+0000 7f4e3652a700 1 Processor -- start 2024-04-21T08:37:45.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.823+0000 7f4e3652a700 1 -- start start 2024-04-21T08:37:45.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.823+0000 7f4e3652a700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e30102a90 0x7f4e30074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:45.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.823+0000 7f4e3652a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e30103450 0x7f4e30075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:45.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.823+0000 7f4e3652a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4e30108a50 0x7f4e300757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:45.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.823+0000 7f4e3652a700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4e30112be0 con 0x7f4e30102a90 2024-04-21T08:37:45.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.823+0000 7f4e3652a700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4e30112a60 con 0x7f4e30103450 2024-04-21T08:37:45.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.823+0000 7f4e3652a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4e30112d60 con 0x7f4e30108a50 2024-04-21T08:37:45.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.823+0000 7f4e2ffff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e30102a90 0x7f4e30074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:45.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.823+0000 7f4e2ffff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e30102a90 0x7f4e30074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43970/0 (socket says 172.21.15.86:43970) 2024-04-21T08:37:45.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.823+0000 7f4e2ffff700 1 -- 172.21.15.86:0/3998967072 learned_addr learned my addr 172.21.15.86:0/3998967072 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:37:45.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.823+0000 7f4e2f7fe700 1 --2- 172.21.15.86:0/3998967072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e30103450 0x7f4e30075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:45.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.823+0000 7f4e2ffff700 1 -- 172.21.15.86:0/3998967072 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4e30108a50 msgr2=0x7f4e300757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:45.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.823+0000 7f4e34ac7700 1 --2- 172.21.15.86:0/3998967072 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4e30108a50 0x7f4e300757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:45.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.823+0000 7f4e2ffff700 1 --2- 172.21.15.86:0/3998967072 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4e30108a50 0x7f4e300757c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:45.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.823+0000 7f4e2ffff700 1 -- 172.21.15.86:0/3998967072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e30103450 msgr2=0x7f4e30075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:45.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.823+0000 7f4e2ffff700 1 --2- 172.21.15.86:0/3998967072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e30103450 0x7f4e30075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:45.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.823+0000 7f4e2ffff700 1 -- 172.21.15.86:0/3998967072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4e301a7580 con 0x7f4e30102a90 2024-04-21T08:37:45.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.823+0000 7f4e2f7fe700 1 --2- 172.21.15.86:0/3998967072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e30103450 0x7f4e30075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:37:45.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.824+0000 7f4e2ffff700 1 --2- 172.21.15.86:0/3998967072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e30102a90 0x7f4e30074d40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4e1800c9b0 tx=0x7f4e1800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:45.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.824+0000 7f4e2d7fa700 1 -- 172.21.15.86:0/3998967072 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e18007b00 con 0x7f4e30102a90 2024-04-21T08:37:45.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.824+0000 7f4e2d7fa700 1 -- 172.21.15.86:0/3998967072 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4e18007ca0 con 0x7f4e30102a90 2024-04-21T08:37:45.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.824+0000 7f4e3652a700 1 -- 172.21.15.86:0/3998967072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4e301a7870 con 0x7f4e30102a90 2024-04-21T08:37:45.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.824+0000 7f4e2d7fa700 1 -- 172.21.15.86:0/3998967072 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4e18018730 con 0x7f4e30102a90 2024-04-21T08:37:45.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.824+0000 7f4e3652a700 1 -- 172.21.15.86:0/3998967072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4e301a7db0 con 0x7f4e30102a90 2024-04-21T08:37:45.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.825+0000 7f4e3652a700 1 -- 172.21.15.86:0/3998967072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4e30069730 con 0x7f4e30102a90 2024-04-21T08:37:45.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.826+0000 7f4e2d7fa700 1 -- 172.21.15.86:0/3998967072 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f4e1800f450 con 0x7f4e30102a90 2024-04-21T08:37:45.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.826+0000 7f4e2d7fa700 1 --2- 172.21.15.86:0/3998967072 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f4e1c0a3cd0 0x7f4e1c0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:45.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.826+0000 7f4e2f7fe700 1 --2- 172.21.15.86:0/3998967072 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f4e1c0a3cd0 0x7f4e1c0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:45.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.826+0000 7f4e2d7fa700 1 -- 172.21.15.86:0/3998967072 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f4e180d08e0 con 0x7f4e30102a90 2024-04-21T08:37:45.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.827+0000 7f4e2f7fe700 1 --2- 172.21.15.86:0/3998967072 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f4e1c0a3cd0 0x7f4e1c0a6190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4e3007a180 tx=0x7f4e20005f50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:45.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:45.831+0000 7f4e2d7fa700 1 -- 172.21.15.86:0/3998967072 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4e18014030 con 0x7f4e30102a90 2024-04-21T08:37:46.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:46.038+0000 7f4e3652a700 1 -- 172.21.15.86:0/3998967072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 4} v 0) -- 0x7f4e3002df80 con 0x7f4e30102a90 2024-04-21T08:37:46.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:46.038+0000 7f4e2d7fa700 1 -- 172.21.15.86:0/3998967072 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 4}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f4e1809ce00 con 0x7f4e30102a90 2024-04-21T08:37:46.038 INFO:teuthology.orchestra.run.smithi086.stdout:141733920834 2024-04-21T08:37:46.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:46.040+0000 7f4e3652a700 1 -- 172.21.15.86:0/3998967072 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f4e1c0a3cd0 msgr2=0x7f4e1c0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:46.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:46.040+0000 7f4e3652a700 1 --2- 172.21.15.86:0/3998967072 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f4e1c0a3cd0 0x7f4e1c0a6190 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4e3007a180 tx=0x7f4e20005f50 comp rx=0 tx=0).stop 2024-04-21T08:37:46.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:46.040+0000 7f4e3652a700 1 -- 172.21.15.86:0/3998967072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e30102a90 msgr2=0x7f4e30074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:46.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:46.040+0000 7f4e3652a700 1 --2- 172.21.15.86:0/3998967072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e30102a90 0x7f4e30074d40 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f4e1800c9b0 tx=0x7f4e1800cd70 comp rx=0 tx=0).stop 2024-04-21T08:37:46.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:46.041+0000 7f4e3652a700 1 -- 172.21.15.86:0/3998967072 shutdown_connections 2024-04-21T08:37:46.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:46.041+0000 7f4e3652a700 1 --2- 172.21.15.86:0/3998967072 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f4e1c0a3cd0 0x7f4e1c0a6190 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:46.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:46.041+0000 7f4e3652a700 1 --2- 172.21.15.86:0/3998967072 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4e30108a50 0x7f4e300757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:46.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:46.041+0000 7f4e3652a700 1 --2- 172.21.15.86:0/3998967072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4e30102a90 0x7f4e30074d40 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:46.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:46.041+0000 7f4e3652a700 1 --2- 172.21.15.86:0/3998967072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4e30103450 0x7f4e30075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:46.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:46.041+0000 7f4e3652a700 1 -- 172.21.15.86:0/3998967072 >> 172.21.15.86:0/3998967072 conn(0x7f4e30079ba0 msgr2=0x7f4e301016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:46.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:46.041+0000 7f4e3652a700 1 -- 172.21.15.86:0/3998967072 shutdown_connections 2024-04-21T08:37:46.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:46.041+0000 7f4e3652a700 1 -- 172.21.15.86:0/3998967072 wait complete. 2024-04-21T08:37:46.422 INFO:tasks.cephadm.ceph_manager.ceph:need seq 197568495665 got 197568495671 for osd.6 2024-04-21T08:37:46.422 DEBUG:teuthology.parallel:result is None 2024-04-21T08:37:46.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:46 smithi086 ceph-mon[27939]: pgmap v332: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-21T08:37:46.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:46 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3998967072' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-21T08:37:46.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:46 smithi088 ceph-mon[31259]: pgmap v332: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-21T08:37:46.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:46 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3998967072' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-21T08:37:46.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:46 smithi179 ceph-mon[31829]: pgmap v332: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 11 op/s 2024-04-21T08:37:46.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:46 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3998967072' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 4}]: dispatch 2024-04-21T08:37:47.839 INFO:tasks.cephadm.ceph_manager.ceph:need seq 141733920829 got 141733920834 for osd.4 2024-04-21T08:37:47.839 DEBUG:teuthology.parallel:result is None 2024-04-21T08:37:48.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.044+0000 7f95d7764700 1 -- 172.21.15.86:0/2613801287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95d0102b00 msgr2=0x7f95d0102f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:48.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.044+0000 7f95d7764700 1 --2- 172.21.15.86:0/2613801287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95d0102b00 0x7f95d0102f80 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f95c0009990 tx=0x7f95c0009ca0 comp rx=0 tx=0).stop 2024-04-21T08:37:48.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.044+0000 7f95d7764700 1 -- 172.21.15.86:0/2613801287 shutdown_connections 2024-04-21T08:37:48.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.044+0000 7f95d7764700 1 --2- 172.21.15.86:0/2613801287 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95d01034c0 0x7f95d010f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:48.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.044+0000 7f95d7764700 1 --2- 172.21.15.86:0/2613801287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95d0108a10 0x7f95d0108df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:48.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.044+0000 7f95d7764700 1 --2- 172.21.15.86:0/2613801287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95d0102b00 0x7f95d0102f80 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:48.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.044+0000 7f95d7764700 1 -- 172.21.15.86:0/2613801287 >> 172.21.15.86:0/2613801287 conn(0x7f95d0079c00 msgr2=0x7f95d007a010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:48.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.044+0000 7f95d7764700 1 -- 172.21.15.86:0/2613801287 shutdown_connections 2024-04-21T08:37:48.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.044+0000 7f95d7764700 1 -- 172.21.15.86:0/2613801287 wait complete. 2024-04-21T08:37:48.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.045+0000 7f95d7764700 1 Processor -- start 2024-04-21T08:37:48.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.045+0000 7f95d7764700 1 -- start start 2024-04-21T08:37:48.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.045+0000 7f95d7764700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95d0102b00 0x7f95d0199110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:48.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.045+0000 7f95d7764700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95d01034c0 0x7f95d0199650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:48.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.045+0000 7f95d7764700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95d0108a10 0x7f95d019d9e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:48.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.045+0000 7f95d7764700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f95d0112ca0 con 0x7f95d0108a10 2024-04-21T08:37:48.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.045+0000 7f95d7764700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f95d0112b20 con 0x7f95d0102b00 2024-04-21T08:37:48.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.045+0000 7f95d7764700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f95d0112e20 con 0x7f95d01034c0 2024-04-21T08:37:48.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.045+0000 7f95d5d01700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95d0108a10 0x7f95d019d9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:48.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.045+0000 7f95d5d01700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95d0108a10 0x7f95d019d9e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:43982/0 (socket says 172.21.15.86:43982) 2024-04-21T08:37:48.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.045+0000 7f95d5d01700 1 -- 172.21.15.86:0/608484521 learned_addr learned my addr 172.21.15.86:0/608484521 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:37:48.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.045+0000 7f95d4cff700 1 --2- 172.21.15.86:0/608484521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95d01034c0 0x7f95d0199650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:48.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.045+0000 7f95d5500700 1 --2- 172.21.15.86:0/608484521 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95d0102b00 0x7f95d0199110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:48.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.046+0000 7f95d5d01700 1 -- 172.21.15.86:0/608484521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95d01034c0 msgr2=0x7f95d0199650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:48.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.046+0000 7f95d5d01700 1 --2- 172.21.15.86:0/608484521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95d01034c0 0x7f95d0199650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:48.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.046+0000 7f95d5d01700 1 -- 172.21.15.86:0/608484521 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95d0102b00 msgr2=0x7f95d0199110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:48.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.046+0000 7f95d5d01700 1 --2- 172.21.15.86:0/608484521 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95d0102b00 0x7f95d0199110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:48.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.046+0000 7f95d5d01700 1 -- 172.21.15.86:0/608484521 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f95d019e120 con 0x7f95d0108a10 2024-04-21T08:37:48.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.046+0000 7f95d5d01700 1 --2- 172.21.15.86:0/608484521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95d0108a10 0x7f95d019d9e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f95c800b970 tx=0x7f95c800bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:48.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.046+0000 7f95c67fc700 1 -- 172.21.15.86:0/608484521 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c8019070 con 0x7f95d0108a10 2024-04-21T08:37:48.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.046+0000 7f95c67fc700 1 -- 172.21.15.86:0/608484521 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f95c800cde0 con 0x7f95d0108a10 2024-04-21T08:37:48.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.046+0000 7f95d7764700 1 -- 172.21.15.86:0/608484521 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f95d019e410 con 0x7f95d0108a10 2024-04-21T08:37:48.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.046+0000 7f95c67fc700 1 -- 172.21.15.86:0/608484521 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f95c8007370 con 0x7f95d0108a10 2024-04-21T08:37:48.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.046+0000 7f95d7764700 1 -- 172.21.15.86:0/608484521 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f95d01a5c90 con 0x7f95d0108a10 2024-04-21T08:37:48.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.047+0000 7f95d7764700 1 -- 172.21.15.86:0/608484521 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f95d0069730 con 0x7f95d0108a10 2024-04-21T08:37:48.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.048+0000 7f95c67fc700 1 -- 172.21.15.86:0/608484521 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f95c801c070 con 0x7f95d0108a10 2024-04-21T08:37:48.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.049+0000 7f95c67fc700 1 --2- 172.21.15.86:0/608484521 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f95bc0a3cd0 0x7f95bc0a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:48.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.049+0000 7f95d5500700 1 --2- 172.21.15.86:0/608484521 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f95bc0a3cd0 0x7f95bc0a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:48.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.049+0000 7f95c67fc700 1 -- 172.21.15.86:0/608484521 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f95c80d0420 con 0x7f95d0108a10 2024-04-21T08:37:48.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.049+0000 7f95d5500700 1 --2- 172.21.15.86:0/608484521 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f95bc0a3cd0 0x7f95bc0a6190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f95cc005ea0 tx=0x7f95cc005e30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:48.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.052+0000 7f95c67fc700 1 -- 172.21.15.86:0/608484521 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f95c8023070 con 0x7f95d0108a10 2024-04-21T08:37:48.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.257+0000 7f95d7764700 1 -- 172.21.15.86:0/608484521 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 2} v 0) -- 0x7f95d002df80 con 0x7f95d0108a10 2024-04-21T08:37:48.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.258+0000 7f95c67fc700 1 -- 172.21.15.86:0/608484521 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 2}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f95c809ca50 con 0x7f95d0108a10 2024-04-21T08:37:48.257 INFO:teuthology.orchestra.run.smithi086.stdout:77309411404 2024-04-21T08:37:48.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.260+0000 7f95d7764700 1 -- 172.21.15.86:0/608484521 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f95bc0a3cd0 msgr2=0x7f95bc0a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:48.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.260+0000 7f95d7764700 1 --2- 172.21.15.86:0/608484521 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f95bc0a3cd0 0x7f95bc0a6190 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7f95cc005ea0 tx=0x7f95cc005e30 comp rx=0 tx=0).stop 2024-04-21T08:37:48.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.260+0000 7f95d7764700 1 -- 172.21.15.86:0/608484521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95d0108a10 msgr2=0x7f95d019d9e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:48.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.260+0000 7f95d7764700 1 --2- 172.21.15.86:0/608484521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95d0108a10 0x7f95d019d9e0 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7f95c800b970 tx=0x7f95c800bc80 comp rx=0 tx=0).stop 2024-04-21T08:37:48.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.260+0000 7f95d7764700 1 -- 172.21.15.86:0/608484521 shutdown_connections 2024-04-21T08:37:48.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.260+0000 7f95d7764700 1 --2- 172.21.15.86:0/608484521 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f95bc0a3cd0 0x7f95bc0a6190 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:48.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.260+0000 7f95d7764700 1 --2- 172.21.15.86:0/608484521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f95d0108a10 0x7f95d019d9e0 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:48.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.260+0000 7f95d7764700 1 --2- 172.21.15.86:0/608484521 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f95d0102b00 0x7f95d0199110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:48.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.260+0000 7f95d7764700 1 --2- 172.21.15.86:0/608484521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f95d01034c0 0x7f95d0199650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:48.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.260+0000 7f95d7764700 1 -- 172.21.15.86:0/608484521 >> 172.21.15.86:0/608484521 conn(0x7f95d0079c00 msgr2=0x7f95d0100b10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:48.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.260+0000 7f95d7764700 1 -- 172.21.15.86:0/608484521 shutdown_connections 2024-04-21T08:37:48.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.260+0000 7f95d7764700 1 -- 172.21.15.86:0/608484521 wait complete. 2024-04-21T08:37:48.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.285+0000 7fb476117700 1 -- 172.21.15.86:0/3493935000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb470103460 msgr2=0x7fb47010f940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:48.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.285+0000 7fb476117700 1 --2- 172.21.15.86:0/3493935000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb470103460 0x7fb47010f940 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7fb46800b0a0 tx=0x7fb46800b3b0 comp rx=0 tx=0).stop 2024-04-21T08:37:48.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.285+0000 7fb476117700 1 -- 172.21.15.86:0/3493935000 shutdown_connections 2024-04-21T08:37:48.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.285+0000 7fb476117700 1 --2- 172.21.15.86:0/3493935000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb470103460 0x7fb47010f940 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:48.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.285+0000 7fb476117700 1 --2- 172.21.15.86:0/3493935000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb470108a70 0x7fb470108e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:48.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.285+0000 7fb476117700 1 --2- 172.21.15.86:0/3493935000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb470102ac0 0x7fb470102f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:48.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.285+0000 7fb476117700 1 -- 172.21.15.86:0/3493935000 >> 172.21.15.86:0/3493935000 conn(0x7fb470079b10 msgr2=0x7fb470079f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:48.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.285+0000 7fb476117700 1 -- 172.21.15.86:0/3493935000 shutdown_connections 2024-04-21T08:37:48.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.285+0000 7fb476117700 1 -- 172.21.15.86:0/3493935000 wait complete. 2024-04-21T08:37:48.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.286+0000 7fb476117700 1 Processor -- start 2024-04-21T08:37:48.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.286+0000 7fb476117700 1 -- start start 2024-04-21T08:37:48.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.287+0000 7fb476117700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb470102ac0 0x7fb470074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:48.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.287+0000 7fb476117700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb470103460 0x7fb470075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:48.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.287+0000 7fb476117700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb470108a70 0x7fb4700757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:48.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.287+0000 7fb476117700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb470112c50 con 0x7fb470103460 2024-04-21T08:37:48.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.287+0000 7fb476117700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb470112ad0 con 0x7fb470102ac0 2024-04-21T08:37:48.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.287+0000 7fb476117700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb470112dd0 con 0x7fb470108a70 2024-04-21T08:37:48.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.287+0000 7fb46effd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb470103460 0x7fb470075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:48.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.287+0000 7fb46f7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb470102ac0 0x7fb470074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:48.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.287+0000 7fb46effd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb470103460 0x7fb470075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44006/0 (socket says 172.21.15.86:44006) 2024-04-21T08:37:48.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.287+0000 7fb46f7fe700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb470102ac0 0x7fb470074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.86:38048/0 (socket says 172.21.15.86:38048) 2024-04-21T08:37:48.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.287+0000 7fb46effd700 1 -- 172.21.15.86:0/2975445043 learned_addr learned my addr 172.21.15.86:0/2975445043 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:37:48.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.287+0000 7fb46ffff700 1 --2- 172.21.15.86:0/2975445043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb470108a70 0x7fb4700757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:48.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.287+0000 7fb46f7fe700 1 -- 172.21.15.86:0/2975445043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb470108a70 msgr2=0x7fb4700757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:48.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.287+0000 7fb46f7fe700 1 --2- 172.21.15.86:0/2975445043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb470108a70 0x7fb4700757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:48.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.287+0000 7fb46f7fe700 1 -- 172.21.15.86:0/2975445043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb470103460 msgr2=0x7fb470075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:48.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.287+0000 7fb46f7fe700 1 --2- 172.21.15.86:0/2975445043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb470103460 0x7fb470075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:48.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.287+0000 7fb46f7fe700 1 -- 172.21.15.86:0/2975445043 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4701a7730 con 0x7fb470102ac0 2024-04-21T08:37:48.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.287+0000 7fb46effd700 1 --2- 172.21.15.86:0/2975445043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb470103460 0x7fb470075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:37:48.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.288+0000 7fb46f7fe700 1 --2- 172.21.15.86:0/2975445043 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb470102ac0 0x7fb470074d40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb46000d950 tx=0x7fb46000dc60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:48.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.288+0000 7fb46cff9700 1 -- 172.21.15.86:0/2975445043 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb460016070 con 0x7fb470102ac0 2024-04-21T08:37:48.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.288+0000 7fb476117700 1 -- 172.21.15.86:0/2975445043 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4701a7a20 con 0x7fb470102ac0 2024-04-21T08:37:48.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.288+0000 7fb476117700 1 -- 172.21.15.86:0/2975445043 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4701a7fb0 con 0x7fb470102ac0 2024-04-21T08:37:48.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.288+0000 7fb46cff9700 1 -- 172.21.15.86:0/2975445043 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb46000fd40 con 0x7fb470102ac0 2024-04-21T08:37:48.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.288+0000 7fb46cff9700 1 -- 172.21.15.86:0/2975445043 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb460020060 con 0x7fb470102ac0 2024-04-21T08:37:48.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.289+0000 7fb476117700 1 -- 172.21.15.86:0/2975445043 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb45c005330 con 0x7fb470102ac0 2024-04-21T08:37:48.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.290+0000 7fb46cff9700 1 -- 172.21.15.86:0/2975445043 <== mon.2 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7fb460019070 con 0x7fb470102ac0 2024-04-21T08:37:48.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.291+0000 7fb46cff9700 1 --2- 172.21.15.86:0/2975445043 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb4580a3d50 0x7fb4580a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:48.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.291+0000 7fb46cff9700 1 -- 172.21.15.86:0/2975445043 <== mon.2 v2:172.21.15.88:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb4600d06d0 con 0x7fb470102ac0 2024-04-21T08:37:48.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.291+0000 7fb46effd700 1 --2- 172.21.15.86:0/2975445043 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb4580a3d50 0x7fb4580a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:48.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.292+0000 7fb46effd700 1 --2- 172.21.15.86:0/2975445043 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb4580a3d50 0x7fb4580a6210 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb47007a110 tx=0x7fb464009260 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:48.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.295+0000 7fb46cff9700 1 -- 172.21.15.86:0/2975445043 <== mon.2 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb46009cb90 con 0x7fb470102ac0 2024-04-21T08:37:48.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.502+0000 7fb476117700 1 -- 172.21.15.86:0/2975445043 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 7} v 0) -- 0x7fb45c0058f0 con 0x7fb470102ac0 2024-04-21T08:37:48.502 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:48 smithi086 ceph-mon[27939]: pgmap v333: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-21T08:37:48.502 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:48 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/608484521' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-21T08:37:48.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.503+0000 7fb46cff9700 1 -- 172.21.15.86:0/2975445043 <== mon.2 v2:172.21.15.88:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 7}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7fb46009c490 con 0x7fb470102ac0 2024-04-21T08:37:48.502 INFO:teuthology.orchestra.run.smithi086.stdout:219043332147 2024-04-21T08:37:48.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.505+0000 7fb476117700 1 -- 172.21.15.86:0/2975445043 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb4580a3d50 msgr2=0x7fb4580a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:48.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.505+0000 7fb476117700 1 --2- 172.21.15.86:0/2975445043 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb4580a3d50 0x7fb4580a6210 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7fb47007a110 tx=0x7fb464009260 comp rx=0 tx=0).stop 2024-04-21T08:37:48.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.505+0000 7fb476117700 1 -- 172.21.15.86:0/2975445043 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb470102ac0 msgr2=0x7fb470074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:48.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.505+0000 7fb476117700 1 --2- 172.21.15.86:0/2975445043 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb470102ac0 0x7fb470074d40 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7fb46000d950 tx=0x7fb46000dc60 comp rx=0 tx=0).stop 2024-04-21T08:37:48.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.505+0000 7fb476117700 1 -- 172.21.15.86:0/2975445043 shutdown_connections 2024-04-21T08:37:48.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.505+0000 7fb476117700 1 --2- 172.21.15.86:0/2975445043 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb4580a3d50 0x7fb4580a6210 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:48.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.505+0000 7fb476117700 1 --2- 172.21.15.86:0/2975445043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb470108a70 0x7fb4700757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:48.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.505+0000 7fb476117700 1 --2- 172.21.15.86:0/2975445043 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb470102ac0 0x7fb470074d40 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:48.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.505+0000 7fb476117700 1 --2- 172.21.15.86:0/2975445043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb470103460 0x7fb470075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:48.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.505+0000 7fb476117700 1 -- 172.21.15.86:0/2975445043 >> 172.21.15.86:0/2975445043 conn(0x7fb470079b10 msgr2=0x7fb470100c10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:48.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.505+0000 7fb476117700 1 -- 172.21.15.86:0/2975445043 shutdown_connections 2024-04-21T08:37:48.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:48.505+0000 7fb476117700 1 -- 172.21.15.86:0/2975445043 wait complete. 2024-04-21T08:37:48.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:48 smithi088 ceph-mon[31259]: pgmap v333: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-21T08:37:48.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:48 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/608484521' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-21T08:37:48.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:48 smithi179 ceph-mon[31829]: pgmap v333: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.1 KiB/s rd, 8 op/s 2024-04-21T08:37:48.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:48 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/608484521' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 2}]: dispatch 2024-04-21T08:37:49.558 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:49 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2975445043' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-21T08:37:49.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:49 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2975445043' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-21T08:37:49.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:49 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2975445043' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 7}]: dispatch 2024-04-21T08:37:50.059 INFO:tasks.cephadm.ceph_manager.ceph:need seq 219043332142 got 219043332147 for osd.7 2024-04-21T08:37:50.059 DEBUG:teuthology.parallel:result is None 2024-04-21T08:37:50.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:50 smithi086 ceph-mon[27939]: pgmap v334: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-21T08:37:50.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:50 smithi088 ceph-mon[31259]: pgmap v334: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-21T08:37:50.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:50 smithi179 ceph-mon[31829]: pgmap v334: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 9 op/s 2024-04-21T08:37:51.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.685+0000 7f21eb34f700 1 -- 172.21.15.86:0/2443642919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21e4101940 msgr2=0x7f21e4101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:51.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.685+0000 7f21eb34f700 1 --2- 172.21.15.86:0/2443642919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21e4101940 0x7f21e4101dc0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f21e0009990 tx=0x7f21e0009ca0 comp rx=0 tx=0).stop 2024-04-21T08:37:51.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.685+0000 7f21eb34f700 1 -- 172.21.15.86:0/2443642919 shutdown_connections 2024-04-21T08:37:51.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.685+0000 7f21eb34f700 1 --2- 172.21.15.86:0/2443642919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21e4102300 0x7f21e410e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:51.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.685+0000 7f21eb34f700 1 --2- 172.21.15.86:0/2443642919 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21e4107990 0x7f21e4107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:51.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.685+0000 7f21eb34f700 1 --2- 172.21.15.86:0/2443642919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21e4101940 0x7f21e4101dc0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:51.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.685+0000 7f21eb34f700 1 -- 172.21.15.86:0/2443642919 >> 172.21.15.86:0/2443642919 conn(0x7f21e40788f0 msgr2=0x7f21e4078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:51.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.685+0000 7f21eb34f700 1 -- 172.21.15.86:0/2443642919 shutdown_connections 2024-04-21T08:37:51.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.685+0000 7f21eb34f700 1 -- 172.21.15.86:0/2443642919 wait complete. 2024-04-21T08:37:51.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.686+0000 7f21eb34f700 1 Processor -- start 2024-04-21T08:37:51.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.686+0000 7f21eb34f700 1 -- start start 2024-04-21T08:37:51.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.686+0000 7f21eb34f700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21e4101940 0x7f21e41990e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:51.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.686+0000 7f21eb34f700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21e4102300 0x7f21e4199620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:51.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.686+0000 7f21eb34f700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21e4107990 0x7f21e419d960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:51.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.686+0000 7f21eb34f700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f21e4111af0 con 0x7f21e4101940 2024-04-21T08:37:51.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.686+0000 7f21eb34f700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f21e4111970 con 0x7f21e4107990 2024-04-21T08:37:51.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.686+0000 7f21eb34f700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f21e4111c70 con 0x7f21e4102300 2024-04-21T08:37:51.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.686+0000 7f21e90eb700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21e4101940 0x7f21e41990e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:51.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.686+0000 7f21e90eb700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21e4101940 0x7f21e41990e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44650/0 (socket says 172.21.15.86:44650) 2024-04-21T08:37:51.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.686+0000 7f21e90eb700 1 -- 172.21.15.86:0/1876450853 learned_addr learned my addr 172.21.15.86:0/1876450853 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:37:51.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.686+0000 7f21e98ec700 1 --2- 172.21.15.86:0/1876450853 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21e4107990 0x7f21e419d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:51.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.686+0000 7f21e88ea700 1 --2- 172.21.15.86:0/1876450853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21e4102300 0x7f21e4199620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:51.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.687+0000 7f21e90eb700 1 -- 172.21.15.86:0/1876450853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21e4102300 msgr2=0x7f21e4199620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:51.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.687+0000 7f21e90eb700 1 --2- 172.21.15.86:0/1876450853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21e4102300 0x7f21e4199620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:51.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.687+0000 7f21e90eb700 1 -- 172.21.15.86:0/1876450853 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21e4107990 msgr2=0x7f21e419d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:51.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.687+0000 7f21e90eb700 1 --2- 172.21.15.86:0/1876450853 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21e4107990 0x7f21e419d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:51.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.687+0000 7f21e90eb700 1 -- 172.21.15.86:0/1876450853 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21e419e0a0 con 0x7f21e4101940 2024-04-21T08:37:51.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.687+0000 7f21e90eb700 1 --2- 172.21.15.86:0/1876450853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21e4101940 0x7f21e41990e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f21d400c9b0 tx=0x7f21d400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:51.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.687+0000 7f21da7fc700 1 -- 172.21.15.86:0/1876450853 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d4007b00 con 0x7f21e4101940 2024-04-21T08:37:51.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.687+0000 7f21da7fc700 1 -- 172.21.15.86:0/1876450853 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21d400ce50 con 0x7f21e4101940 2024-04-21T08:37:51.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.687+0000 7f21eb34f700 1 -- 172.21.15.86:0/1876450853 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21e419e390 con 0x7f21e4101940 2024-04-21T08:37:51.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.687+0000 7f21da7fc700 1 -- 172.21.15.86:0/1876450853 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d4018660 con 0x7f21e4101940 2024-04-21T08:37:51.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.687+0000 7f21eb34f700 1 -- 172.21.15.86:0/1876450853 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21e41a5c10 con 0x7f21e4101940 2024-04-21T08:37:51.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.688+0000 7f21eb34f700 1 -- 172.21.15.86:0/1876450853 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f21e4103060 con 0x7f21e4101940 2024-04-21T08:37:51.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.689+0000 7f21da7fc700 1 -- 172.21.15.86:0/1876450853 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f21d400f450 con 0x7f21e4101940 2024-04-21T08:37:51.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.690+0000 7f21da7fc700 1 --2- 172.21.15.86:0/1876450853 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f21d00a3cd0 0x7f21d00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:51.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.690+0000 7f21e88ea700 1 --2- 172.21.15.86:0/1876450853 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f21d00a3cd0 0x7f21d00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:51.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.690+0000 7f21da7fc700 1 -- 172.21.15.86:0/1876450853 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f21d40d0790 con 0x7f21e4101940 2024-04-21T08:37:51.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.690+0000 7f21e88ea700 1 --2- 172.21.15.86:0/1876450853 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f21d00a3cd0 0x7f21d00a6190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f21e000b5c0 tx=0x7f21e0005960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:51.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.694+0000 7f21da7fc700 1 -- 172.21.15.86:0/1876450853 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f21d4014030 con 0x7f21e4101940 2024-04-21T08:37:51.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.901+0000 7f21eb34f700 1 -- 172.21.15.86:0/1876450853 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 1} v 0) -- 0x7f21e419a3f0 con 0x7f21e4101940 2024-04-21T08:37:51.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.901+0000 7f21da7fc700 1 -- 172.21.15.86:0/1876450853 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 1}]=0 v0) ==== 74+0+12 (secure 0 0 0) 0x7f21d409ccb0 con 0x7f21e4101940 2024-04-21T08:37:51.900 INFO:teuthology.orchestra.run.smithi086.stdout:55834574930 2024-04-21T08:37:51.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.903+0000 7f21eb34f700 1 -- 172.21.15.86:0/1876450853 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f21d00a3cd0 msgr2=0x7f21d00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:51.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.904+0000 7f21eb34f700 1 --2- 172.21.15.86:0/1876450853 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f21d00a3cd0 0x7f21d00a6190 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f21e000b5c0 tx=0x7f21e0005960 comp rx=0 tx=0).stop 2024-04-21T08:37:51.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.904+0000 7f21eb34f700 1 -- 172.21.15.86:0/1876450853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21e4101940 msgr2=0x7f21e41990e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:51.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.904+0000 7f21eb34f700 1 --2- 172.21.15.86:0/1876450853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21e4101940 0x7f21e41990e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f21d400c9b0 tx=0x7f21d400ccc0 comp rx=0 tx=0).stop 2024-04-21T08:37:51.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.904+0000 7f21eb34f700 1 -- 172.21.15.86:0/1876450853 shutdown_connections 2024-04-21T08:37:51.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.904+0000 7f21eb34f700 1 --2- 172.21.15.86:0/1876450853 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f21d00a3cd0 0x7f21d00a6190 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:51.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.904+0000 7f21eb34f700 1 --2- 172.21.15.86:0/1876450853 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21e4107990 0x7f21e419d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:51.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.904+0000 7f21eb34f700 1 --2- 172.21.15.86:0/1876450853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21e4101940 0x7f21e41990e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:51.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.904+0000 7f21eb34f700 1 --2- 172.21.15.86:0/1876450853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21e4102300 0x7f21e4199620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:51.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.904+0000 7f21eb34f700 1 -- 172.21.15.86:0/1876450853 >> 172.21.15.86:0/1876450853 conn(0x7f21e40788f0 msgr2=0x7f21e406aaa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:51.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.904+0000 7f21eb34f700 1 -- 172.21.15.86:0/1876450853 shutdown_connections 2024-04-21T08:37:51.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.904+0000 7f21eb34f700 1 -- 172.21.15.86:0/1876450853 wait complete. 2024-04-21T08:37:52.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:52 smithi086 ceph-mon[27939]: pgmap v335: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-21T08:37:52.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:52 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1876450853' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-21T08:37:52.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:52 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/890101506' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-21T08:37:52.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:52 smithi088 ceph-mon[31259]: pgmap v335: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-21T08:37:52.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:52 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1876450853' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-21T08:37:52.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:52 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/890101506' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-21T08:37:52.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.903+0000 7f28bcd09700 1 -- 172.21.15.86:0/2501429669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28b8101940 msgr2=0x7f28b8101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:52.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.903+0000 7f28bcd09700 1 --2- 172.21.15.86:0/2501429669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28b8101940 0x7f28b8101dc0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7f28ac009990 tx=0x7f28ac009ca0 comp rx=0 tx=0).stop 2024-04-21T08:37:52.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.903+0000 7f28bcd09700 1 -- 172.21.15.86:0/2501429669 shutdown_connections 2024-04-21T08:37:52.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.903+0000 7f28bcd09700 1 --2- 172.21.15.86:0/2501429669 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28b8102300 0x7f28b810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:52.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.903+0000 7f28bcd09700 1 --2- 172.21.15.86:0/2501429669 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28b8107990 0x7f28b8107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:52.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.903+0000 7f28bcd09700 1 --2- 172.21.15.86:0/2501429669 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28b8101940 0x7f28b8101dc0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:52.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.903+0000 7f28bcd09700 1 -- 172.21.15.86:0/2501429669 >> 172.21.15.86:0/2501429669 conn(0x7f28b80788f0 msgr2=0x7f28b8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:52.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.903+0000 7f28bcd09700 1 -- 172.21.15.86:0/2501429669 shutdown_connections 2024-04-21T08:37:52.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.903+0000 7f28bcd09700 1 -- 172.21.15.86:0/2501429669 wait complete. 2024-04-21T08:37:52.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.904+0000 7f28bcd09700 1 Processor -- start 2024-04-21T08:37:52.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.904+0000 7f28bcd09700 1 -- start start 2024-04-21T08:37:52.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.905+0000 7f28bcd09700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28b8101940 0x7f28b8199f40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:52.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.905+0000 7f28bcd09700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28b8102300 0x7f28b819a480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:52.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.905+0000 7f28bcd09700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28b8107990 0x7f28b81a4fc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:52.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.905+0000 7f28bcd09700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f28b8111ae0 con 0x7f28b8107990 2024-04-21T08:37:52.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.905+0000 7f28bcd09700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f28b8111960 con 0x7f28b8102300 2024-04-21T08:37:52.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.905+0000 7f28bcd09700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f28b8111c60 con 0x7f28b8101940 2024-04-21T08:37:52.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.905+0000 7f28b6d9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28b8107990 0x7f28b81a4fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:52.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.905+0000 7f28b6d9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28b8107990 0x7f28b81a4fc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44676/0 (socket says 172.21.15.86:44676) 2024-04-21T08:37:52.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.905+0000 7f28b6d9d700 1 -- 172.21.15.86:0/890101506 learned_addr learned my addr 172.21.15.86:0/890101506 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:37:52.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.905+0000 7f28b659c700 1 --2- 172.21.15.86:0/890101506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28b8101940 0x7f28b8199f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:52.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.905+0000 7f28b5d9b700 1 --2- 172.21.15.86:0/890101506 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28b8102300 0x7f28b819a480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:52.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.905+0000 7f28b6d9d700 1 -- 172.21.15.86:0/890101506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28b8101940 msgr2=0x7f28b8199f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:52.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.905+0000 7f28b6d9d700 1 --2- 172.21.15.86:0/890101506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28b8101940 0x7f28b8199f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:52.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.905+0000 7f28b6d9d700 1 -- 172.21.15.86:0/890101506 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28b8102300 msgr2=0x7f28b819a480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:52.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.905+0000 7f28b6d9d700 1 --2- 172.21.15.86:0/890101506 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28b8102300 0x7f28b819a480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:52.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.905+0000 7f28b6d9d700 1 -- 172.21.15.86:0/890101506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28b81a5500 con 0x7f28b8107990 2024-04-21T08:37:52.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.905+0000 7f28b5d9b700 1 --2- 172.21.15.86:0/890101506 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28b8102300 0x7f28b819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:37:52.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.905+0000 7f28b6d9d700 1 --2- 172.21.15.86:0/890101506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28b8107990 0x7f28b81a4fc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f28b0007ee0 tx=0x7f28b000d3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:52.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.905+0000 7f28a77fe700 1 -- 172.21.15.86:0/890101506 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b000ddd0 con 0x7f28b8107990 2024-04-21T08:37:52.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.905+0000 7f28a77fe700 1 -- 172.21.15.86:0/890101506 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28b0004020 con 0x7f28b8107990 2024-04-21T08:37:52.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.905+0000 7f28bcd09700 1 -- 172.21.15.86:0/890101506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28b81a5730 con 0x7f28b8107990 2024-04-21T08:37:52.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.905+0000 7f28a77fe700 1 -- 172.21.15.86:0/890101506 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b0015660 con 0x7f28b8107990 2024-04-21T08:37:52.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.906+0000 7f28bcd09700 1 -- 172.21.15.86:0/890101506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28b81a5bf0 con 0x7f28b8107990 2024-04-21T08:37:52.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.906+0000 7f28bcd09700 1 -- 172.21.15.86:0/890101506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f28b8103060 con 0x7f28b8107990 2024-04-21T08:37:52.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.908+0000 7f28a77fe700 1 -- 172.21.15.86:0/890101506 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 17) ==== 138697+0+0 (secure 0 0 0) 0x7f28b00041d0 con 0x7f28b8107990 2024-04-21T08:37:52.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.908+0000 7f28a77fe700 1 --2- 172.21.15.86:0/890101506 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f28a00a3cd0 0x7f28a00a6190 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:52.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.908+0000 7f28a77fe700 1 -- 172.21.15.86:0/890101506 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f28b00d0b20 con 0x7f28b8107990 2024-04-21T08:37:52.736 INFO:teuthology.orchestra.run.smithi086.stdout:163208757309 2024-04-21T08:37:52.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.908+0000 7f28b659c700 1 --2- 172.21.15.86:0/890101506 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f28a00a3cd0 0x7f28a00a6190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:52.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.909+0000 7f28b659c700 1 --2- 172.21.15.86:0/890101506 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f28a00a3cd0 0x7f28a00a6190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f28a8005950 tx=0x7f28a800b410 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:52.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:51.912+0000 7f28a77fe700 1 -- 172.21.15.86:0/890101506 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f28b0010070 con 0x7f28b8107990 2024-04-21T08:37:52.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:52.117+0000 7f28bcd09700 1 -- 172.21.15.86:0/890101506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 5} v 0) -- 0x7f28b802df80 con 0x7f28b8107990 2024-04-21T08:37:52.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:52.117+0000 7f28a77fe700 1 -- 172.21.15.86:0/890101506 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 5}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f28b009d040 con 0x7f28b8107990 2024-04-21T08:37:52.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:52.120+0000 7f28bcd09700 1 -- 172.21.15.86:0/890101506 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f28a00a3cd0 msgr2=0x7f28a00a6190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:52.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:52.120+0000 7f28bcd09700 1 --2- 172.21.15.86:0/890101506 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f28a00a3cd0 0x7f28a00a6190 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f28a8005950 tx=0x7f28a800b410 comp rx=0 tx=0).stop 2024-04-21T08:37:52.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:52.120+0000 7f28bcd09700 1 -- 172.21.15.86:0/890101506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28b8107990 msgr2=0x7f28b81a4fc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:52.736 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:52.120+0000 7f28bcd09700 1 --2- 172.21.15.86:0/890101506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28b8107990 0x7f28b81a4fc0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f28b0007ee0 tx=0x7f28b000d3b0 comp rx=0 tx=0).stop 2024-04-21T08:37:52.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:52.120+0000 7f28bcd09700 1 -- 172.21.15.86:0/890101506 shutdown_connections 2024-04-21T08:37:52.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:52.120+0000 7f28bcd09700 1 --2- 172.21.15.86:0/890101506 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f28a00a3cd0 0x7f28a00a6190 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:52.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:52.120+0000 7f28bcd09700 1 --2- 172.21.15.86:0/890101506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28b8107990 0x7f28b81a4fc0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:52.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:52.120+0000 7f28bcd09700 1 --2- 172.21.15.86:0/890101506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28b8101940 0x7f28b8199f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:52.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:52.120+0000 7f28bcd09700 1 --2- 172.21.15.86:0/890101506 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28b8102300 0x7f28b819a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:52.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:52.120+0000 7f28bcd09700 1 -- 172.21.15.86:0/890101506 >> 172.21.15.86:0/890101506 conn(0x7f28b80788f0 msgr2=0x7f28b80ffaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:52.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:52.120+0000 7f28bcd09700 1 -- 172.21.15.86:0/890101506 shutdown_connections 2024-04-21T08:37:52.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:52.120+0000 7f28bcd09700 1 -- 172.21.15.86:0/890101506 wait complete. 2024-04-21T08:37:52.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:52 smithi179 ceph-mon[31829]: pgmap v335: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.7 KiB/s rd, 10 op/s 2024-04-21T08:37:52.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:52 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1876450853' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 1}]: dispatch 2024-04-21T08:37:52.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:52 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/890101506' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 5}]: dispatch 2024-04-21T08:37:53.297 INFO:tasks.cephadm.ceph_manager.ceph:need seq 77309411399 got 77309411404 for osd.2 2024-04-21T08:37:53.297 DEBUG:teuthology.parallel:result is None 2024-04-21T08:37:53.602 INFO:tasks.cephadm.ceph_manager.ceph:need seq 55834574922 got 55834574930 for osd.1 2024-04-21T08:37:53.602 DEBUG:teuthology.parallel:result is None 2024-04-21T08:37:54.060 INFO:tasks.cephadm.ceph_manager.ceph:need seq 163208757304 got 163208757309 for osd.5 2024-04-21T08:37:54.060 DEBUG:teuthology.parallel:result is None 2024-04-21T08:37:54.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:54 smithi086 ceph-mon[27939]: pgmap v336: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-21T08:37:54.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:54 smithi086 ceph-mon[27939]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-21T08:37:54.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:54 smithi088 ceph-mon[31259]: pgmap v336: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-21T08:37:54.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:54 smithi088 ceph-mon[31259]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-21T08:37:54.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:54 smithi179 ceph-mon[31829]: pgmap v336: 129 pgs: 129 active+clean; 646 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.5 KiB/s rd, 8 op/s 2024-04-21T08:37:54.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:54 smithi179 ceph-mon[31829]: mgrmap e18: x(active, since 8m), standbys: z, y 2024-04-21T08:37:54.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:54.790+0000 7f6d36509700 1 -- 172.21.15.86:0/908670996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d30103450 msgr2=0x7f6d3010f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:54.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:54.790+0000 7f6d36509700 1 --2- 172.21.15.86:0/908670996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d30103450 0x7f6d3010f8b0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f6d2800b0a0 tx=0x7f6d2800b3b0 comp rx=0 tx=0).stop 2024-04-21T08:37:54.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:54.790+0000 7f6d36509700 1 -- 172.21.15.86:0/908670996 shutdown_connections 2024-04-21T08:37:54.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:54.790+0000 7f6d36509700 1 --2- 172.21.15.86:0/908670996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d30103450 0x7f6d3010f8b0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:54.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:54.790+0000 7f6d36509700 1 --2- 172.21.15.86:0/908670996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d30108a50 0x7f6d30108e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:54.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:54.790+0000 7f6d36509700 1 --2- 172.21.15.86:0/908670996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d30102a90 0x7f6d30102f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:54.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:54.790+0000 7f6d36509700 1 -- 172.21.15.86:0/908670996 >> 172.21.15.86:0/908670996 conn(0x7f6d30079ba0 msgr2=0x7f6d30079fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:54.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:54.790+0000 7f6d36509700 1 -- 172.21.15.86:0/908670996 shutdown_connections 2024-04-21T08:37:54.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:54.790+0000 7f6d36509700 1 -- 172.21.15.86:0/908670996 wait complete. 2024-04-21T08:37:54.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:54.790+0000 7f6d36509700 1 Processor -- start 2024-04-21T08:37:54.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:54.790+0000 7f6d36509700 1 -- start start 2024-04-21T08:37:54.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:54.791+0000 7f6d36509700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d30102a90 0x7f6d30199030 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:54.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:54.791+0000 7f6d36509700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d30103450 0x7f6d30199570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:54.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:54.791+0000 7f6d36509700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d30108a50 0x7f6d3019d900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:54.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:54.791+0000 7f6d36509700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6d30112be0 con 0x7f6d30102a90 2024-04-21T08:37:54.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:54.791+0000 7f6d36509700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6d30112a60 con 0x7f6d30103450 2024-04-21T08:37:54.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:54.791+0000 7f6d36509700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6d30112d60 con 0x7f6d30108a50 2024-04-21T08:37:54.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:54.791+0000 7f6d2ffff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d30102a90 0x7f6d30199030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:54.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:54.791+0000 7f6d2ffff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d30102a90 0x7f6d30199030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44690/0 (socket says 172.21.15.86:44690) 2024-04-21T08:37:54.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:54.791+0000 7f6d2ffff700 1 -- 172.21.15.86:0/409420131 learned_addr learned my addr 172.21.15.86:0/409420131 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:37:54.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:54.791+0000 7f6d34aa6700 1 --2- 172.21.15.86:0/409420131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d30108a50 0x7f6d3019d900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:54.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:54.791+0000 7f6d2ffff700 1 -- 172.21.15.86:0/409420131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d30108a50 msgr2=0x7f6d3019d900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:54.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:54.791+0000 7f6d2ffff700 1 --2- 172.21.15.86:0/409420131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d30108a50 0x7f6d3019d900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:54.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:54.791+0000 7f6d2ffff700 1 -- 172.21.15.86:0/409420131 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d30103450 msgr2=0x7f6d30199570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:54.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:54.791+0000 7f6d2ffff700 1 --2- 172.21.15.86:0/409420131 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d30103450 0x7f6d30199570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:54.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:54.791+0000 7f6d2ffff700 1 -- 172.21.15.86:0/409420131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d3019e040 con 0x7f6d30102a90 2024-04-21T08:37:54.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:54.791+0000 7f6d2ffff700 1 --2- 172.21.15.86:0/409420131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d30102a90 0x7f6d30199030 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6d2000d950 tx=0x7f6d2000dd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:54.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:54.791+0000 7f6d2d7fa700 1 -- 172.21.15.86:0/409420131 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d20016070 con 0x7f6d30102a90 2024-04-21T08:37:54.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:54.791+0000 7f6d2d7fa700 1 -- 172.21.15.86:0/409420131 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d20010460 con 0x7f6d30102a90 2024-04-21T08:37:54.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:54.791+0000 7f6d36509700 1 -- 172.21.15.86:0/409420131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d3019e330 con 0x7f6d30102a90 2024-04-21T08:37:54.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:54.791+0000 7f6d2d7fa700 1 -- 172.21.15.86:0/409420131 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d2000f620 con 0x7f6d30102a90 2024-04-21T08:37:54.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:54.792+0000 7f6d36509700 1 -- 172.21.15.86:0/409420131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d301a5bb0 con 0x7f6d30102a90 2024-04-21T08:37:54.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:54.793+0000 7f6d36509700 1 -- 172.21.15.86:0/409420131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d30069730 con 0x7f6d30102a90 2024-04-21T08:37:54.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:54.794+0000 7f6d2d7fa700 1 -- 172.21.15.86:0/409420131 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6d20019070 con 0x7f6d30102a90 2024-04-21T08:37:54.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:54.794+0000 7f6d2d7fa700 1 --2- 172.21.15.86:0/409420131 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6d180a3c40 0x7f6d180a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:54.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:54.794+0000 7f6d2f7fe700 1 --2- 172.21.15.86:0/409420131 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6d180a3c40 0x7f6d180a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:54.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:54.794+0000 7f6d2d7fa700 1 -- 172.21.15.86:0/409420131 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f6d200d0350 con 0x7f6d30102a90 2024-04-21T08:37:54.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:54.795+0000 7f6d2f7fe700 1 --2- 172.21.15.86:0/409420131 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6d180a3c40 0x7f6d180a6100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6d24005fd0 tx=0x7f6d24005d00 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:54.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:54.799+0000 7f6d2d7fa700 1 -- 172.21.15.86:0/409420131 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6d2009c9d0 con 0x7f6d30102a90 2024-04-21T08:37:55.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.008+0000 7f6d36509700 1 -- 172.21.15.86:0/409420131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 11} v 0) -- 0x7f6d3002e140 con 0x7f6d30102a90 2024-04-21T08:37:55.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.009+0000 7f6d2d7fa700 1 -- 172.21.15.86:0/409420131 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 11}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f6d2009c2d0 con 0x7f6d30102a90 2024-04-21T08:37:55.008 INFO:teuthology.orchestra.run.smithi086.stdout:343597383712 2024-04-21T08:37:55.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.011+0000 7f6d36509700 1 -- 172.21.15.86:0/409420131 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6d180a3c40 msgr2=0x7f6d180a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:55.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.011+0000 7f6d36509700 1 --2- 172.21.15.86:0/409420131 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6d180a3c40 0x7f6d180a6100 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7f6d24005fd0 tx=0x7f6d24005d00 comp rx=0 tx=0).stop 2024-04-21T08:37:55.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.011+0000 7f6d36509700 1 -- 172.21.15.86:0/409420131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d30102a90 msgr2=0x7f6d30199030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:55.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.011+0000 7f6d36509700 1 --2- 172.21.15.86:0/409420131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d30102a90 0x7f6d30199030 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f6d2000d950 tx=0x7f6d2000dd10 comp rx=0 tx=0).stop 2024-04-21T08:37:55.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.011+0000 7f6d36509700 1 -- 172.21.15.86:0/409420131 shutdown_connections 2024-04-21T08:37:55.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.011+0000 7f6d36509700 1 --2- 172.21.15.86:0/409420131 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6d180a3c40 0x7f6d180a6100 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:55.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.011+0000 7f6d36509700 1 --2- 172.21.15.86:0/409420131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d30108a50 0x7f6d3019d900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:55.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.011+0000 7f6d36509700 1 --2- 172.21.15.86:0/409420131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d30102a90 0x7f6d30199030 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:55.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.011+0000 7f6d36509700 1 --2- 172.21.15.86:0/409420131 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d30103450 0x7f6d30199570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:55.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.011+0000 7f6d36509700 1 -- 172.21.15.86:0/409420131 >> 172.21.15.86:0/409420131 conn(0x7f6d30079ba0 msgr2=0x7f6d301016f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:55.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.011+0000 7f6d36509700 1 -- 172.21.15.86:0/409420131 shutdown_connections 2024-04-21T08:37:55.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.011+0000 7f6d36509700 1 -- 172.21.15.86:0/409420131 wait complete. 2024-04-21T08:37:55.614 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:55 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/409420131' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-21T08:37:55.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:55 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/409420131' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-21T08:37:55.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.716+0000 7f6d0c126700 1 -- 172.21.15.86:0/4284080262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d04101640 msgr2=0x7f6d04101a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:55.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.716+0000 7f6d0c126700 1 --2- 172.21.15.86:0/4284080262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d04101640 0x7f6d04101a20 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8009990 tx=0x7f6cf8009ca0 comp rx=0 tx=0).stop 2024-04-21T08:37:55.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.716+0000 7f6d0c126700 1 -- 172.21.15.86:0/4284080262 shutdown_connections 2024-04-21T08:37:55.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.716+0000 7f6d0c126700 1 --2- 172.21.15.86:0/4284080262 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d041064e0 0x7f6d0410e020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:55.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.716+0000 7f6d0c126700 1 --2- 172.21.15.86:0/4284080262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d04101640 0x7f6d04101a20 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:55.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.716+0000 7f6d0c126700 1 --2- 172.21.15.86:0/4284080262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d04101ff0 0x7f6d04105db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:55.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.716+0000 7f6d0c126700 1 -- 172.21.15.86:0/4284080262 >> 172.21.15.86:0/4284080262 conn(0x7f6d04076c60 msgr2=0x7f6d04077070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:55.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.716+0000 7f6d0c126700 1 -- 172.21.15.86:0/4284080262 shutdown_connections 2024-04-21T08:37:55.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.716+0000 7f6d0c126700 1 -- 172.21.15.86:0/4284080262 wait complete. 2024-04-21T08:37:55.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.716+0000 7f6d0c126700 1 Processor -- start 2024-04-21T08:37:55.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.716+0000 7f6d0c126700 1 -- start start 2024-04-21T08:37:55.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.717+0000 7f6d0c126700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d04101640 0x7f6d04197f80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:55.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.717+0000 7f6d0c126700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d04101ff0 0x7f6d041984c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:55.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.717+0000 7f6d0c126700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d041064e0 0x7f6d0419c850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:55.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.717+0000 7f6d0c126700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6d041112c0 con 0x7f6d04101ff0 2024-04-21T08:37:55.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.717+0000 7f6d0c126700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6d04111140 con 0x7f6d041064e0 2024-04-21T08:37:55.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.717+0000 7f6d0c126700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6d04111440 con 0x7f6d04101640 2024-04-21T08:37:55.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.717+0000 7f6d096c1700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d04101ff0 0x7f6d041984c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:55.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.717+0000 7f6d096c1700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d04101ff0 0x7f6d041984c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44706/0 (socket says 172.21.15.86:44706) 2024-04-21T08:37:55.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.717+0000 7f6d096c1700 1 -- 172.21.15.86:0/2982114137 learned_addr learned my addr 172.21.15.86:0/2982114137 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:37:55.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.717+0000 7f6d0a6c3700 1 --2- 172.21.15.86:0/2982114137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d041064e0 0x7f6d0419c850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:55.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.717+0000 7f6d09ec2700 1 --2- 172.21.15.86:0/2982114137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d04101640 0x7f6d04197f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:55.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.717+0000 7f6d096c1700 1 -- 172.21.15.86:0/2982114137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d04101640 msgr2=0x7f6d04197f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:55.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.717+0000 7f6d096c1700 1 --2- 172.21.15.86:0/2982114137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d04101640 0x7f6d04197f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:55.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.717+0000 7f6d096c1700 1 -- 172.21.15.86:0/2982114137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d041064e0 msgr2=0x7f6d0419c850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:55.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.717+0000 7f6d096c1700 1 --2- 172.21.15.86:0/2982114137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d041064e0 0x7f6d0419c850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:55.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.717+0000 7f6d096c1700 1 -- 172.21.15.86:0/2982114137 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6d0419cf90 con 0x7f6d04101ff0 2024-04-21T08:37:55.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.717+0000 7f6d096c1700 1 --2- 172.21.15.86:0/2982114137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d04101ff0 0x7f6d041984c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6d0000c9b0 tx=0x7f6d0000ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:55.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.717+0000 7f6cf6ffd700 1 -- 172.21.15.86:0/2982114137 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d00007aa0 con 0x7f6d04101ff0 2024-04-21T08:37:55.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.718+0000 7f6cf6ffd700 1 -- 172.21.15.86:0/2982114137 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6d00007c40 con 0x7f6d04101ff0 2024-04-21T08:37:55.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.718+0000 7f6d0c126700 1 -- 172.21.15.86:0/2982114137 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6d0419d280 con 0x7f6d04101ff0 2024-04-21T08:37:55.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.718+0000 7f6cf6ffd700 1 -- 172.21.15.86:0/2982114137 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6d00018660 con 0x7f6d04101ff0 2024-04-21T08:37:55.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.718+0000 7f6d0c126700 1 -- 172.21.15.86:0/2982114137 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6d041a4b00 con 0x7f6d04101ff0 2024-04-21T08:37:55.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.719+0000 7f6d0c126700 1 -- 172.21.15.86:0/2982114137 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6d04069730 con 0x7f6d04101ff0 2024-04-21T08:37:55.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.720+0000 7f6cf6ffd700 1 -- 172.21.15.86:0/2982114137 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6d0000f450 con 0x7f6d04101ff0 2024-04-21T08:37:55.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.720+0000 7f6cf6ffd700 1 --2- 172.21.15.86:0/2982114137 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6cf00a3cc0 0x7f6cf00a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:55.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.720+0000 7f6d09ec2700 1 --2- 172.21.15.86:0/2982114137 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6cf00a3cc0 0x7f6cf00a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:55.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.720+0000 7f6cf6ffd700 1 -- 172.21.15.86:0/2982114137 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f6d000d09f0 con 0x7f6d04101ff0 2024-04-21T08:37:55.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.721+0000 7f6d09ec2700 1 --2- 172.21.15.86:0/2982114137 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6cf00a3cc0 0x7f6cf00a6180 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8006010 tx=0x7f6cf801a040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:55.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.724+0000 7f6cf6ffd700 1 -- 172.21.15.86:0/2982114137 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6d00014030 con 0x7f6d04101ff0 2024-04-21T08:37:55.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:55 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/409420131' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 11}]: dispatch 2024-04-21T08:37:55.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.926+0000 7f6d0c126700 1 -- 172.21.15.86:0/2982114137 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 3} v 0) -- 0x7f6d041992e0 con 0x7f6d04101ff0 2024-04-21T08:37:55.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.927+0000 7f6cf6ffd700 1 -- 172.21.15.86:0/2982114137 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 3}]=0 v0) ==== 74+0+13 (secure 0 0 0) 0x7f6d041992e0 con 0x7f6d04101ff0 2024-04-21T08:37:55.926 INFO:teuthology.orchestra.run.smithi086.stdout:107374182472 2024-04-21T08:37:55.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.929+0000 7f6d0c126700 1 -- 172.21.15.86:0/2982114137 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6cf00a3cc0 msgr2=0x7f6cf00a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:55.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.929+0000 7f6d0c126700 1 --2- 172.21.15.86:0/2982114137 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6cf00a3cc0 0x7f6cf00a6180 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7f6cf8006010 tx=0x7f6cf801a040 comp rx=0 tx=0).stop 2024-04-21T08:37:55.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.929+0000 7f6d0c126700 1 -- 172.21.15.86:0/2982114137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d04101ff0 msgr2=0x7f6d041984c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:55.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.929+0000 7f6d0c126700 1 --2- 172.21.15.86:0/2982114137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d04101ff0 0x7f6d041984c0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6d0000c9b0 tx=0x7f6d0000ccc0 comp rx=0 tx=0).stop 2024-04-21T08:37:55.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.929+0000 7f6d0c126700 1 -- 172.21.15.86:0/2982114137 shutdown_connections 2024-04-21T08:37:55.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.929+0000 7f6d0c126700 1 --2- 172.21.15.86:0/2982114137 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6cf00a3cc0 0x7f6cf00a6180 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:55.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.929+0000 7f6d0c126700 1 --2- 172.21.15.86:0/2982114137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6d041064e0 0x7f6d0419c850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:55.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.929+0000 7f6d0c126700 1 --2- 172.21.15.86:0/2982114137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6d04101640 0x7f6d04197f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:55.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.929+0000 7f6d0c126700 1 --2- 172.21.15.86:0/2982114137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6d04101ff0 0x7f6d041984c0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:55.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.929+0000 7f6d0c126700 1 -- 172.21.15.86:0/2982114137 >> 172.21.15.86:0/2982114137 conn(0x7f6d04076c60 msgr2=0x7f6d041003b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:55.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.929+0000 7f6d0c126700 1 -- 172.21.15.86:0/2982114137 shutdown_connections 2024-04-21T08:37:55.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.929+0000 7f6d0c126700 1 -- 172.21.15.86:0/2982114137 wait complete. 2024-04-21T08:37:55.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.956+0000 7f7fe44fd700 1 -- 172.21.15.86:0/3590747345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fdc1034b0 msgr2=0x7f7fdc10f8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:55.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.956+0000 7f7fe44fd700 1 --2- 172.21.15.86:0/3590747345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fdc1034b0 0x7f7fdc10f8e0 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7f7fd400b0a0 tx=0x7f7fd400b3b0 comp rx=0 tx=0).stop 2024-04-21T08:37:55.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.956+0000 7f7fe44fd700 1 -- 172.21.15.86:0/3590747345 shutdown_connections 2024-04-21T08:37:55.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.956+0000 7f7fe44fd700 1 --2- 172.21.15.86:0/3590747345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fdc1034b0 0x7f7fdc10f8e0 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:55.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.956+0000 7f7fe44fd700 1 --2- 172.21.15.86:0/3590747345 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fdc108a80 0x7f7fdc108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:55.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.956+0000 7f7fe44fd700 1 --2- 172.21.15.86:0/3590747345 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7fdc102af0 0x7f7fdc102f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:55.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.956+0000 7f7fe44fd700 1 -- 172.21.15.86:0/3590747345 >> 172.21.15.86:0/3590747345 conn(0x7f7fdc079ad0 msgr2=0x7f7fdc079ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:55.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.956+0000 7f7fe44fd700 1 -- 172.21.15.86:0/3590747345 shutdown_connections 2024-04-21T08:37:55.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.956+0000 7f7fe44fd700 1 -- 172.21.15.86:0/3590747345 wait complete. 2024-04-21T08:37:55.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.957+0000 7f7fe44fd700 1 Processor -- start 2024-04-21T08:37:55.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.957+0000 7f7fe44fd700 1 -- start start 2024-04-21T08:37:55.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.957+0000 7f7fe44fd700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fdc102af0 0x7f7fdc199260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:55.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.957+0000 7f7fe44fd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fdc1034b0 0x7f7fdc1997a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:55.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.957+0000 7f7fe44fd700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7fdc108a80 0x7f7fdc19db30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:55.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.957+0000 7f7fe44fd700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7fdc112d10 con 0x7f7fdc1034b0 2024-04-21T08:37:55.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.957+0000 7f7fe44fd700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7fdc112b90 con 0x7f7fdc102af0 2024-04-21T08:37:55.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.957+0000 7f7fe44fd700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7fdc112e90 con 0x7f7fdc108a80 2024-04-21T08:37:55.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.957+0000 7f7fe1a98700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fdc1034b0 0x7f7fdc1997a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:55.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.957+0000 7f7fe1a98700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fdc1034b0 0x7f7fdc1997a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44718/0 (socket says 172.21.15.86:44718) 2024-04-21T08:37:55.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.957+0000 7f7fe1a98700 1 -- 172.21.15.86:0/2685316383 learned_addr learned my addr 172.21.15.86:0/2685316383 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:37:55.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.957+0000 7f7fe2a9a700 1 --2- 172.21.15.86:0/2685316383 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7fdc108a80 0x7f7fdc19db30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:55.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.957+0000 7f7fe2299700 1 --2- 172.21.15.86:0/2685316383 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fdc102af0 0x7f7fdc199260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:55.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.958+0000 7f7fe1a98700 1 -- 172.21.15.86:0/2685316383 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7fdc108a80 msgr2=0x7f7fdc19db30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:55.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.958+0000 7f7fe1a98700 1 --2- 172.21.15.86:0/2685316383 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7fdc108a80 0x7f7fdc19db30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:55.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.958+0000 7f7fe1a98700 1 -- 172.21.15.86:0/2685316383 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fdc102af0 msgr2=0x7f7fdc199260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:55.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.958+0000 7f7fe1a98700 1 --2- 172.21.15.86:0/2685316383 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fdc102af0 0x7f7fdc199260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:55.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.958+0000 7f7fe1a98700 1 -- 172.21.15.86:0/2685316383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fdc19e270 con 0x7f7fdc1034b0 2024-04-21T08:37:55.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.958+0000 7f7fe1a98700 1 --2- 172.21.15.86:0/2685316383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fdc1034b0 0x7f7fdc1997a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc00b530 tx=0x7f7fcc00b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:55.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.958+0000 7f7fd37fe700 1 -- 172.21.15.86:0/2685316383 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc015020 con 0x7f7fdc1034b0 2024-04-21T08:37:55.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.958+0000 7f7fe44fd700 1 -- 172.21.15.86:0/2685316383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fdc19e560 con 0x7f7fdc1034b0 2024-04-21T08:37:55.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.958+0000 7f7fd37fe700 1 -- 172.21.15.86:0/2685316383 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fcc011ce0 con 0x7f7fdc1034b0 2024-04-21T08:37:55.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.958+0000 7f7fd37fe700 1 -- 172.21.15.86:0/2685316383 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fcc010dd0 con 0x7f7fdc1034b0 2024-04-21T08:37:55.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.958+0000 7f7fe44fd700 1 -- 172.21.15.86:0/2685316383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fdc074c70 con 0x7f7fdc1034b0 2024-04-21T08:37:55.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.959+0000 7f7fe44fd700 1 -- 172.21.15.86:0/2685316383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7fdc069730 con 0x7f7fdc1034b0 2024-04-21T08:37:55.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.961+0000 7f7fd37fe700 1 -- 172.21.15.86:0/2685316383 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7fcc018020 con 0x7f7fdc1034b0 2024-04-21T08:37:55.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.962+0000 7f7fd37fe700 1 --2- 172.21.15.86:0/2685316383 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f7fc80a3c40 0x7f7fc80a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:37:55.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.962+0000 7f7fd37fe700 1 -- 172.21.15.86:0/2685316383 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f7fcc0d0330 con 0x7f7fdc1034b0 2024-04-21T08:37:55.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.962+0000 7f7fe2299700 1 --2- 172.21.15.86:0/2685316383 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f7fc80a3c40 0x7f7fc80a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:37:55.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.963+0000 7f7fe2299700 1 --2- 172.21.15.86:0/2685316383 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f7fc80a3c40 0x7f7fc80a6100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8009a10 tx=0x7f7fd8009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:37:55.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:55.965+0000 7f7fd37fe700 1 -- 172.21.15.86:0/2685316383 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7fcc01f020 con 0x7f7fdc1034b0 2024-04-21T08:37:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:56.171+0000 7f7fe44fd700 1 -- 172.21.15.86:0/2685316383 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd last-stat-seq", "id": 10} v 0) -- 0x7f7fdc19a5c0 con 0x7f7fdc1034b0 2024-04-21T08:37:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:56.171+0000 7f7fd37fe700 1 -- 172.21.15.86:0/2685316383 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd last-stat-seq", "id": 10}]=0 v0) ==== 75+0+13 (secure 0 0 0) 0x7f7fcc09ca50 con 0x7f7fdc1034b0 2024-04-21T08:37:56.170 INFO:teuthology.orchestra.run.smithi086.stdout:309237645349 2024-04-21T08:37:56.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:56.173+0000 7f7fe44fd700 1 -- 172.21.15.86:0/2685316383 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f7fc80a3c40 msgr2=0x7f7fc80a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:56.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:56.173+0000 7f7fe44fd700 1 --2- 172.21.15.86:0/2685316383 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f7fc80a3c40 0x7f7fc80a6100 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7f7fd8009a10 tx=0x7f7fd8009190 comp rx=0 tx=0).stop 2024-04-21T08:37:56.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:56.173+0000 7f7fe44fd700 1 -- 172.21.15.86:0/2685316383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fdc1034b0 msgr2=0x7f7fdc1997a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:37:56.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:56.173+0000 7f7fe44fd700 1 --2- 172.21.15.86:0/2685316383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fdc1034b0 0x7f7fdc1997a0 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f7fcc00b530 tx=0x7f7fcc00b840 comp rx=0 tx=0).stop 2024-04-21T08:37:56.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:56.173+0000 7f7fe44fd700 1 -- 172.21.15.86:0/2685316383 shutdown_connections 2024-04-21T08:37:56.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:56.173+0000 7f7fe44fd700 1 --2- 172.21.15.86:0/2685316383 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f7fc80a3c40 0x7f7fc80a6100 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:56.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:56.173+0000 7f7fe44fd700 1 --2- 172.21.15.86:0/2685316383 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7fdc108a80 0x7f7fdc19db30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:56.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:56.173+0000 7f7fe44fd700 1 --2- 172.21.15.86:0/2685316383 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fdc102af0 0x7f7fdc199260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:56.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:56.173+0000 7f7fe44fd700 1 --2- 172.21.15.86:0/2685316383 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fdc1034b0 0x7f7fdc1997a0 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:37:56.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:56.173+0000 7f7fe44fd700 1 -- 172.21.15.86:0/2685316383 >> 172.21.15.86:0/2685316383 conn(0x7f7fdc079ad0 msgr2=0x7f7fdc101860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:37:56.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:56.173+0000 7f7fe44fd700 1 -- 172.21.15.86:0/2685316383 shutdown_connections 2024-04-21T08:37:56.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:37:56.173+0000 7f7fe44fd700 1 -- 172.21.15.86:0/2685316383 wait complete. 2024-04-21T08:37:56.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:56 smithi086 ceph-mon[27939]: pgmap v337: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 12 op/s 2024-04-21T08:37:56.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:56 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2982114137' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-21T08:37:56.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:56 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2685316383' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-21T08:37:56.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:56 smithi088 ceph-mon[31259]: pgmap v337: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 12 op/s 2024-04-21T08:37:56.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:56 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2982114137' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-21T08:37:56.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:56 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2685316383' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-21T08:37:56.740 INFO:tasks.cephadm.ceph_manager.ceph:need seq 107374182465 got 107374182472 for osd.3 2024-04-21T08:37:56.740 DEBUG:teuthology.parallel:result is None 2024-04-21T08:37:56.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:56 smithi179 ceph-mon[31829]: pgmap v337: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 5.9 KiB/s rd, 597 B/s wr, 12 op/s 2024-04-21T08:37:56.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:56 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2982114137' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 3}]: dispatch 2024-04-21T08:37:56.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:56 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2685316383' entity='client.admin' cmd=[{"prefix": "osd last-stat-seq", "id": 10}]: dispatch 2024-04-21T08:37:57.465 INFO:tasks.cephadm.ceph_manager.ceph:need seq 343597383707 got 343597383712 for osd.11 2024-04-21T08:37:57.465 DEBUG:teuthology.parallel:result is None 2024-04-21T08:37:57.905 INFO:tasks.cephadm.ceph_manager.ceph:need seq 309237645343 got 309237645349 for osd.10 2024-04-21T08:37:57.906 DEBUG:teuthology.parallel:result is None 2024-04-21T08:37:57.906 INFO:tasks.cephadm.ceph_manager.ceph:waiting for clean 2024-04-21T08:37:57.906 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-21T08:37:58.191 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:37:58.583 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:58 smithi086 ceph-mon[27939]: pgmap v338: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-21T08:37:58.584 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:58 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:37:58.584 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:58 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:37:58.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:58 smithi088 ceph-mon[31259]: pgmap v338: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-21T08:37:58.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:58 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:37:58.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:58 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:37:58.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:58 smithi179 ceph-mon[31829]: pgmap v338: 129 pgs: 129 active+clean; 647 KiB data, 371 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 597 B/s wr, 9 op/s 2024-04-21T08:37:58.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:58 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:37:58.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:58 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:37:59.647 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:59 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:37:59.647 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:59 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:37:59.647 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:37:59 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:37:59.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:59 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:37:59.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:59 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:37:59.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:37:59 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:37:59.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:59 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:37:59.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:59 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:37:59.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:37:59 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' 2024-04-21T08:38:00.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.047+0000 7fd2adb58700 1 -- 172.21.15.86:0/2238740708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2a8101940 msgr2=0x7fd2a8101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:00.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.047+0000 7fd2adb58700 1 --2- 172.21.15.86:0/2238740708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2a8101940 0x7fd2a8101dc0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7fd29c009990 tx=0x7fd29c009ca0 comp rx=0 tx=0).stop 2024-04-21T08:38:00.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.047+0000 7fd2adb58700 1 -- 172.21.15.86:0/2238740708 shutdown_connections 2024-04-21T08:38:00.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.047+0000 7fd2adb58700 1 --2- 172.21.15.86:0/2238740708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd2a8102300 0x7fd2a810e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:00.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.047+0000 7fd2adb58700 1 --2- 172.21.15.86:0/2238740708 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2a8107990 0x7fd2a8107d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:00.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.047+0000 7fd2adb58700 1 --2- 172.21.15.86:0/2238740708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2a8101940 0x7fd2a8101dc0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:00.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.047+0000 7fd2adb58700 1 -- 172.21.15.86:0/2238740708 >> 172.21.15.86:0/2238740708 conn(0x7fd2a80788f0 msgr2=0x7fd2a8078d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:00.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.047+0000 7fd2adb58700 1 -- 172.21.15.86:0/2238740708 shutdown_connections 2024-04-21T08:38:00.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.047+0000 7fd2adb58700 1 -- 172.21.15.86:0/2238740708 wait complete. 2024-04-21T08:38:00.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.047+0000 7fd2adb58700 1 Processor -- start 2024-04-21T08:38:00.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.048+0000 7fd2adb58700 1 -- start start 2024-04-21T08:38:00.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.048+0000 7fd2adb58700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd2a8101940 0x7fd2a819b360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:00.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.048+0000 7fd2adb58700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2a8102300 0x7fd2a819b8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:00.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.048+0000 7fd2adb58700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2a8107990 0x7fd2a819fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:00.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.048+0000 7fd2adb58700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a8111af0 con 0x7fd2a8102300 2024-04-21T08:38:00.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.048+0000 7fd2adb58700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a8111970 con 0x7fd2a8107990 2024-04-21T08:38:00.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.048+0000 7fd2adb58700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd2a8111c70 con 0x7fd2a8101940 2024-04-21T08:38:00.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.048+0000 7fd2a6ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2a8102300 0x7fd2a819b8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:00.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.048+0000 7fd2a6ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2a8102300 0x7fd2a819b8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51082/0 (socket says 172.21.15.86:51082) 2024-04-21T08:38:00.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.048+0000 7fd2a6ffd700 1 -- 172.21.15.86:0/1482278138 learned_addr learned my addr 172.21.15.86:0/1482278138 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:00.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.048+0000 7fd2a77fe700 1 --2- 172.21.15.86:0/1482278138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd2a8101940 0x7fd2a819b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:00.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.048+0000 7fd2a7fff700 1 --2- 172.21.15.86:0/1482278138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2a8107990 0x7fd2a819fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:00.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.048+0000 7fd2a6ffd700 1 -- 172.21.15.86:0/1482278138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd2a8101940 msgr2=0x7fd2a819b360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:00.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.048+0000 7fd2a6ffd700 1 --2- 172.21.15.86:0/1482278138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd2a8101940 0x7fd2a819b360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:00.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.048+0000 7fd2a6ffd700 1 -- 172.21.15.86:0/1482278138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2a8107990 msgr2=0x7fd2a819fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:00.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.048+0000 7fd2a6ffd700 1 --2- 172.21.15.86:0/1482278138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2a8107990 0x7fd2a819fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:00.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.048+0000 7fd2a6ffd700 1 -- 172.21.15.86:0/1482278138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2a81a0370 con 0x7fd2a8102300 2024-04-21T08:38:00.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.048+0000 7fd2a6ffd700 1 --2- 172.21.15.86:0/1482278138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2a8102300 0x7fd2a819b8a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd29c000c00 tx=0x7fd29c0047b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:00.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.049+0000 7fd2a4ff9700 1 -- 172.21.15.86:0/1482278138 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29c027070 con 0x7fd2a8102300 2024-04-21T08:38:00.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.049+0000 7fd2adb58700 1 -- 172.21.15.86:0/1482278138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2a81a0600 con 0x7fd2a8102300 2024-04-21T08:38:00.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.049+0000 7fd2a4ff9700 1 -- 172.21.15.86:0/1482278138 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd29c003f20 con 0x7fd2a8102300 2024-04-21T08:38:00.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.049+0000 7fd2a4ff9700 1 -- 172.21.15.86:0/1482278138 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29c022ca0 con 0x7fd2a8102300 2024-04-21T08:38:00.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.049+0000 7fd2adb58700 1 -- 172.21.15.86:0/1482278138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2a81a7ee0 con 0x7fd2a8102300 2024-04-21T08:38:00.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.050+0000 7fd2adb58700 1 -- 172.21.15.86:0/1482278138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd2a8103060 con 0x7fd2a8102300 2024-04-21T08:38:00.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.051+0000 7fd2a4ff9700 1 -- 172.21.15.86:0/1482278138 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd29c02a070 con 0x7fd2a8102300 2024-04-21T08:38:00.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.051+0000 7fd2a4ff9700 1 --2- 172.21.15.86:0/1482278138 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fd2900a3c40 0x7fd2900a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:00.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.051+0000 7fd2a77fe700 1 --2- 172.21.15.86:0/1482278138 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fd2900a3c40 0x7fd2900a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:00.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.051+0000 7fd2a4ff9700 1 -- 172.21.15.86:0/1482278138 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fd29c0d15c0 con 0x7fd2a8102300 2024-04-21T08:38:00.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.052+0000 7fd2a77fe700 1 --2- 172.21.15.86:0/1482278138 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fd2900a3c40 0x7fd2900a6100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd298007b60 tx=0x7fd2980073d0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:00.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.056+0000 7fd2a4ff9700 1 -- 172.21.15.86:0/1482278138 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd29c05eff0 con 0x7fd2a8102300 2024-04-21T08:38:00.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.250+0000 7fd2adb58700 1 -- 172.21.15.86:0/1482278138 --> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fd2a819c6c0 con 0x7fd2900a3c40 2024-04-21T08:38:00.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.260+0000 7fd2a4ff9700 1 -- 172.21.15.86:0/1482278138 <== mgr.14152 v2:172.21.15.86:6800/1364156965 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340655 (secure 0 0 0) 0x7fd2a819c6c0 con 0x7fd2900a3c40 2024-04-21T08:38:00.264 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:38:00.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.263+0000 7fd2adb58700 1 -- 172.21.15.86:0/1482278138 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fd2900a3c40 msgr2=0x7fd2900a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:00.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.263+0000 7fd2adb58700 1 --2- 172.21.15.86:0/1482278138 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fd2900a3c40 0x7fd2900a6100 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fd298007b60 tx=0x7fd2980073d0 comp rx=0 tx=0).stop 2024-04-21T08:38:00.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.263+0000 7fd2adb58700 1 -- 172.21.15.86:0/1482278138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2a8102300 msgr2=0x7fd2a819b8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:00.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.263+0000 7fd2adb58700 1 --2- 172.21.15.86:0/1482278138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2a8102300 0x7fd2a819b8a0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7fd29c000c00 tx=0x7fd29c0047b0 comp rx=0 tx=0).stop 2024-04-21T08:38:00.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.263+0000 7fd2adb58700 1 -- 172.21.15.86:0/1482278138 shutdown_connections 2024-04-21T08:38:00.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.263+0000 7fd2adb58700 1 --2- 172.21.15.86:0/1482278138 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fd2900a3c40 0x7fd2900a6100 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:00.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.263+0000 7fd2adb58700 1 --2- 172.21.15.86:0/1482278138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2a8107990 0x7fd2a819fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:00.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.263+0000 7fd2adb58700 1 --2- 172.21.15.86:0/1482278138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd2a8101940 0x7fd2a819b360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:00.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.263+0000 7fd2adb58700 1 --2- 172.21.15.86:0/1482278138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2a8102300 0x7fd2a819b8a0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:00.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.263+0000 7fd2adb58700 1 -- 172.21.15.86:0/1482278138 >> 172.21.15.86:0/1482278138 conn(0x7fd2a80788f0 msgr2=0x7fd2a806a920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:00.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.263+0000 7fd2adb58700 1 -- 172.21.15.86:0/1482278138 shutdown_connections 2024-04-21T08:38:00.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:00.263+0000 7fd2adb58700 1 -- 172.21.15.86:0/1482278138 wait complete. 2024-04-21T08:38:00.267 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-21T08:38:00.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:00 smithi086 ceph-mon[27939]: pgmap v339: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2024-04-21T08:38:00.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:00 smithi088 ceph-mon[31259]: pgmap v339: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2024-04-21T08:38:00.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:00 smithi179 ceph-mon[31829]: pgmap v339: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 10 op/s 2024-04-21T08:38:01.099 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":339,"stamp":"2024-04-21T08:37:59.296219+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662225,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":871,"num_read_kb":558,"num_write":436,"num_write_kb":3090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":1181659,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":860,"ondisk_log_size":860,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":384192,"kb_used_data":14240,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124557632,"statfs":{"total":1151940427776,"available":1151547015168,"internally_reserved":0,"allocated":14581760,"data_stored":7972481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":378713404},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":28,"apply_latency_ms":28,"commit_latency_ns":28000000,"apply_latency_ns":28000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":672,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":121,"num_read_kb":57,"num_write":11,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003636"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.306918+0000","last_change":"2024-04-21T08:36:20.913421+0000","last_active":"2024-04-21T08:37:53.306918+0000","last_peered":"2024-04-21T08:37:53.306918+0000","last_clean":"2024-04-21T08:37:53.306918+0000","last_became_active":"2024-04-21T08:36:20.913017+0000","last_became_peered":"2024-04-21T08:36:20.913017+0000","last_unstale":"2024-04-21T08:37:53.306918+0000","last_undegraded":"2024-04-21T08:37:53.306918+0000","last_fullsized":"2024-04-21T08:37:53.306918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:16:29.885916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.298871+0000","last_change":"2024-04-21T08:36:20.911675+0000","last_active":"2024-04-21T08:37:53.298871+0000","last_peered":"2024-04-21T08:37:53.298871+0000","last_clean":"2024-04-21T08:37:53.298871+0000","last_became_active":"2024-04-21T08:36:20.910591+0000","last_became_peered":"2024-04-21T08:36:20.910591+0000","last_unstale":"2024-04-21T08:37:53.298871+0000","last_undegraded":"2024-04-21T08:37:53.298871+0000","last_fullsized":"2024-04-21T08:37:53.298871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:01:40.735376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.310261+0000","last_change":"2024-04-21T08:36:20.911978+0000","last_active":"2024-04-21T08:37:53.310261+0000","last_peered":"2024-04-21T08:37:53.310261+0000","last_clean":"2024-04-21T08:37:53.310261+0000","last_became_active":"2024-04-21T08:36:20.911641+0000","last_became_peered":"2024-04-21T08:36:20.911641+0000","last_unstale":"2024-04-21T08:37:53.310261+0000","last_undegraded":"2024-04-21T08:37:53.310261+0000","last_fullsized":"2024-04-21T08:37:53.310261+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:14:23.789790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.287047+0000","last_change":"2024-04-21T08:36:20.919027+0000","last_active":"2024-04-21T08:37:53.287047+0000","last_peered":"2024-04-21T08:37:53.287047+0000","last_clean":"2024-04-21T08:37:53.287047+0000","last_became_active":"2024-04-21T08:36:20.918740+0000","last_became_peered":"2024-04-21T08:36:20.918740+0000","last_unstale":"2024-04-21T08:37:53.287047+0000","last_undegraded":"2024-04-21T08:37:53.287047+0000","last_fullsized":"2024-04-21T08:37:53.287047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:00:31.066938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.287750+0000","last_change":"2024-04-21T08:36:20.907175+0000","last_active":"2024-04-21T08:37:53.287750+0000","last_peered":"2024-04-21T08:37:53.287750+0000","last_clean":"2024-04-21T08:37:53.287750+0000","last_became_active":"2024-04-21T08:36:20.906818+0000","last_became_peered":"2024-04-21T08:36:20.906818+0000","last_unstale":"2024-04-21T08:37:53.287750+0000","last_undegraded":"2024-04-21T08:37:53.287750+0000","last_fullsized":"2024-04-21T08:37:53.287750+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:41:03.332231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.289860+0000","last_change":"2024-04-21T08:36:20.905968+0000","last_active":"2024-04-21T08:37:53.289860+0000","last_peered":"2024-04-21T08:37:53.289860+0000","last_clean":"2024-04-21T08:37:53.289860+0000","last_became_active":"2024-04-21T08:36:20.904840+0000","last_became_peered":"2024-04-21T08:36:20.904840+0000","last_unstale":"2024-04-21T08:37:53.289860+0000","last_undegraded":"2024-04-21T08:37:53.289860+0000","last_fullsized":"2024-04-21T08:37:53.289860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:23:46.886241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.289034+0000","last_change":"2024-04-21T08:36:20.912045+0000","last_active":"2024-04-21T08:37:53.289034+0000","last_peered":"2024-04-21T08:37:53.289034+0000","last_clean":"2024-04-21T08:37:53.289034+0000","last_became_active":"2024-04-21T08:36:20.911690+0000","last_became_peered":"2024-04-21T08:36:20.911690+0000","last_unstale":"2024-04-21T08:37:53.289034+0000","last_undegraded":"2024-04-21T08:37:53.289034+0000","last_fullsized":"2024-04-21T08:37:53.289034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:06:28.229307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.305439+0000","last_change":"2024-04-21T08:36:20.904874+0000","last_active":"2024-04-21T08:37:53.305439+0000","last_peered":"2024-04-21T08:37:53.305439+0000","last_clean":"2024-04-21T08:37:53.305439+0000","last_became_active":"2024-04-21T08:36:20.904217+0000","last_became_peered":"2024-04-21T08:36:20.904217+0000","last_unstale":"2024-04-21T08:37:53.305439+0000","last_undegraded":"2024-04-21T08:37:53.305439+0000","last_fullsized":"2024-04-21T08:37:53.305439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:37:04.423812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.310061+0000","last_change":"2024-04-21T08:36:20.912874+0000","last_active":"2024-04-21T08:37:53.310061+0000","last_peered":"2024-04-21T08:37:53.310061+0000","last_clean":"2024-04-21T08:37:53.310061+0000","last_became_active":"2024-04-21T08:36:20.912569+0000","last_became_peered":"2024-04-21T08:36:20.912569+0000","last_unstale":"2024-04-21T08:37:53.310061+0000","last_undegraded":"2024-04-21T08:37:53.310061+0000","last_fullsized":"2024-04-21T08:37:53.310061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:55:20.763157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.297432+0000","last_change":"2024-04-21T08:37:24.296916+0000","last_active":"2024-04-21T08:37:53.297432+0000","last_peered":"2024-04-21T08:37:53.297432+0000","last_clean":"2024-04-21T08:37:53.297432+0000","last_became_active":"2024-04-21T08:37:24.296547+0000","last_became_peered":"2024-04-21T08:37:24.296547+0000","last_unstale":"2024-04-21T08:37:53.297432+0000","last_undegraded":"2024-04-21T08:37:53.297432+0000","last_fullsized":"2024-04-21T08:37:53.297432+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:47:56.683793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.307187+0000","last_change":"2024-04-21T08:36:20.909094+0000","last_active":"2024-04-21T08:37:53.307187+0000","last_peered":"2024-04-21T08:37:53.307187+0000","last_clean":"2024-04-21T08:37:53.307187+0000","last_became_active":"2024-04-21T08:36:20.908634+0000","last_became_peered":"2024-04-21T08:36:20.908634+0000","last_unstale":"2024-04-21T08:37:53.307187+0000","last_undegraded":"2024-04-21T08:37:53.307187+0000","last_fullsized":"2024-04-21T08:37:53.307187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:28:38.450621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.277892+0000","last_change":"2024-04-21T08:36:20.909429+0000","last_active":"2024-04-21T08:37:53.277892+0000","last_peered":"2024-04-21T08:37:53.277892+0000","last_clean":"2024-04-21T08:37:53.277892+0000","last_became_active":"2024-04-21T08:36:20.909081+0000","last_became_peered":"2024-04-21T08:36:20.909081+0000","last_unstale":"2024-04-21T08:37:53.277892+0000","last_undegraded":"2024-04-21T08:37:53.277892+0000","last_fullsized":"2024-04-21T08:37:53.277892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:45:46.939125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.300525+0000","last_change":"2024-04-21T08:36:20.895894+0000","last_active":"2024-04-21T08:37:53.300525+0000","last_peered":"2024-04-21T08:37:53.300525+0000","last_clean":"2024-04-21T08:37:53.300525+0000","last_became_active":"2024-04-21T08:36:20.895572+0000","last_became_peered":"2024-04-21T08:36:20.895572+0000","last_unstale":"2024-04-21T08:37:53.300525+0000","last_undegraded":"2024-04-21T08:37:53.300525+0000","last_fullsized":"2024-04-21T08:37:53.300525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:12:48.750555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.298339+0000","last_change":"2024-04-21T08:36:20.905356+0000","last_active":"2024-04-21T08:37:53.298339+0000","last_peered":"2024-04-21T08:37:53.298339+0000","last_clean":"2024-04-21T08:37:53.298339+0000","last_became_active":"2024-04-21T08:36:20.904954+0000","last_became_peered":"2024-04-21T08:36:20.904954+0000","last_unstale":"2024-04-21T08:37:53.298339+0000","last_undegraded":"2024-04-21T08:37:53.298339+0000","last_fullsized":"2024-04-21T08:37:53.298339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:45:49.116482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.306501+0000","last_change":"2024-04-21T08:36:20.911550+0000","last_active":"2024-04-21T08:37:53.306501+0000","last_peered":"2024-04-21T08:37:53.306501+0000","last_clean":"2024-04-21T08:37:53.306501+0000","last_became_active":"2024-04-21T08:36:20.910557+0000","last_became_peered":"2024-04-21T08:36:20.910557+0000","last_unstale":"2024-04-21T08:37:53.306501+0000","last_undegraded":"2024-04-21T08:37:53.306501+0000","last_fullsized":"2024-04-21T08:37:53.306501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:09:49.422729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.307606+0000","last_change":"2024-04-21T08:36:20.903954+0000","last_active":"2024-04-21T08:37:53.307606+0000","last_peered":"2024-04-21T08:37:53.307606+0000","last_clean":"2024-04-21T08:37:53.307606+0000","last_became_active":"2024-04-21T08:36:20.903641+0000","last_became_peered":"2024-04-21T08:36:20.903641+0000","last_unstale":"2024-04-21T08:37:53.307606+0000","last_undegraded":"2024-04-21T08:37:53.307606+0000","last_fullsized":"2024-04-21T08:37:53.307606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:08:55.678940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.307105+0000","last_change":"2024-04-21T08:36:20.914256+0000","last_active":"2024-04-21T08:37:53.307105+0000","last_peered":"2024-04-21T08:37:53.307105+0000","last_clean":"2024-04-21T08:37:53.307105+0000","last_became_active":"2024-04-21T08:36:20.913858+0000","last_became_peered":"2024-04-21T08:36:20.913858+0000","last_unstale":"2024-04-21T08:37:53.307105+0000","last_undegraded":"2024-04-21T08:37:53.307105+0000","last_fullsized":"2024-04-21T08:37:53.307105+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:01:10.276763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.295256+0000","last_change":"2024-04-21T08:36:20.904138+0000","last_active":"2024-04-21T08:37:53.295256+0000","last_peered":"2024-04-21T08:37:53.295256+0000","last_clean":"2024-04-21T08:37:53.295256+0000","last_became_active":"2024-04-21T08:36:20.903670+0000","last_became_peered":"2024-04-21T08:36:20.903670+0000","last_unstale":"2024-04-21T08:37:53.295256+0000","last_undegraded":"2024-04-21T08:37:53.295256+0000","last_fullsized":"2024-04-21T08:37:53.295256+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:38:50.160867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.302234+0000","last_change":"2024-04-21T08:36:20.910798+0000","last_active":"2024-04-21T08:37:53.302234+0000","last_peered":"2024-04-21T08:37:53.302234+0000","last_clean":"2024-04-21T08:37:53.302234+0000","last_became_active":"2024-04-21T08:36:20.910411+0000","last_became_peered":"2024-04-21T08:36:20.910411+0000","last_unstale":"2024-04-21T08:37:53.302234+0000","last_undegraded":"2024-04-21T08:37:53.302234+0000","last_fullsized":"2024-04-21T08:37:53.302234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:13:12.109818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.288019+0000","last_change":"2024-04-21T08:36:24.924831+0000","last_active":"2024-04-21T08:37:24.288019+0000","last_peered":"2024-04-21T08:37:24.288019+0000","last_clean":"2024-04-21T08:37:24.288019+0000","last_became_active":"2024-04-21T08:36:24.924435+0000","last_became_peered":"2024-04-21T08:36:24.924435+0000","last_unstale":"2024-04-21T08:37:24.288019+0000","last_undegraded":"2024-04-21T08:37:24.288019+0000","last_fullsized":"2024-04-21T08:37:24.288019+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:23.447079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":128,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.334209+0000","last_change":"2024-04-21T08:37:24.308024+0000","last_active":"2024-04-21T08:37:53.334209+0000","last_peered":"2024-04-21T08:37:53.334209+0000","last_clean":"2024-04-21T08:37:53.334209+0000","last_became_active":"2024-04-21T08:37:24.297491+0000","last_became_peered":"2024-04-21T08:37:24.297491+0000","last_unstale":"2024-04-21T08:37:53.334209+0000","last_undegraded":"2024-04-21T08:37:53.334209+0000","last_fullsized":"2024-04-21T08:37:53.334209+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:48:10.039545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":61,"num_read_kb":61,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.305004+0000","last_change":"2024-04-21T08:36:20.912851+0000","last_active":"2024-04-21T08:37:53.305004+0000","last_peered":"2024-04-21T08:37:53.305004+0000","last_clean":"2024-04-21T08:37:53.305004+0000","last_became_active":"2024-04-21T08:36:20.912407+0000","last_became_peered":"2024-04-21T08:36:20.912407+0000","last_unstale":"2024-04-21T08:37:53.305004+0000","last_undegraded":"2024-04-21T08:37:53.305004+0000","last_fullsized":"2024-04-21T08:37:53.305004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:45:18.026940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.290917+0000","last_change":"2024-04-21T08:36:24.924052+0000","last_active":"2024-04-21T08:37:24.290917+0000","last_peered":"2024-04-21T08:37:24.290917+0000","last_clean":"2024-04-21T08:37:24.290917+0000","last_became_active":"2024-04-21T08:36:24.923618+0000","last_became_peered":"2024-04-21T08:36:24.923618+0000","last_unstale":"2024-04-21T08:37:24.290917+0000","last_undegraded":"2024-04-21T08:37:24.290917+0000","last_fullsized":"2024-04-21T08:37:24.290917+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:08:18.182386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.293039+0000","last_change":"2024-04-21T08:36:20.285291+0000","last_active":"2024-04-21T08:37:24.293039+0000","last_peered":"2024-04-21T08:37:24.293039+0000","last_clean":"2024-04-21T08:37:24.293039+0000","last_became_active":"2024-04-21T08:36:20.284969+0000","last_became_peered":"2024-04-21T08:36:20.284969+0000","last_unstale":"2024-04-21T08:37:24.293039+0000","last_undegraded":"2024-04-21T08:37:24.293039+0000","last_fullsized":"2024-04-21T08:37:24.293039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:55:29.484439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.310149+0000","last_change":"2024-04-21T08:36:20.902819+0000","last_active":"2024-04-21T08:37:53.310149+0000","last_peered":"2024-04-21T08:37:53.310149+0000","last_clean":"2024-04-21T08:37:53.310149+0000","last_became_active":"2024-04-21T08:36:20.902466+0000","last_became_peered":"2024-04-21T08:36:20.902466+0000","last_unstale":"2024-04-21T08:37:53.310149+0000","last_undegraded":"2024-04-21T08:37:53.310149+0000","last_fullsized":"2024-04-21T08:37:53.310149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:29:40.454026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.291673+0000","last_change":"2024-04-21T08:36:24.926480+0000","last_active":"2024-04-21T08:37:24.291673+0000","last_peered":"2024-04-21T08:37:24.291673+0000","last_clean":"2024-04-21T08:37:24.291673+0000","last_became_active":"2024-04-21T08:36:24.926116+0000","last_became_peered":"2024-04-21T08:36:24.926116+0000","last_unstale":"2024-04-21T08:37:24.291673+0000","last_undegraded":"2024-04-21T08:37:24.291673+0000","last_fullsized":"2024-04-21T08:37:24.291673+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:43:18.839396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.292631+0000","last_change":"2024-04-21T08:36:20.293086+0000","last_active":"2024-04-21T08:37:24.292631+0000","last_peered":"2024-04-21T08:37:24.292631+0000","last_clean":"2024-04-21T08:37:24.292631+0000","last_became_active":"2024-04-21T08:36:20.291178+0000","last_became_peered":"2024-04-21T08:36:20.291178+0000","last_unstale":"2024-04-21T08:37:24.292631+0000","last_undegraded":"2024-04-21T08:37:24.292631+0000","last_fullsized":"2024-04-21T08:37:24.292631+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:02:32.935930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.310382+0000","last_change":"2024-04-21T08:36:20.909374+0000","last_active":"2024-04-21T08:37:53.310382+0000","last_peered":"2024-04-21T08:37:53.310382+0000","last_clean":"2024-04-21T08:37:53.310382+0000","last_became_active":"2024-04-21T08:36:20.909043+0000","last_became_peered":"2024-04-21T08:36:20.909043+0000","last_unstale":"2024-04-21T08:37:53.310382+0000","last_undegraded":"2024-04-21T08:37:53.310382+0000","last_fullsized":"2024-04-21T08:37:53.310382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:45:45.719942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.291273+0000","last_change":"2024-04-21T08:36:24.923989+0000","last_active":"2024-04-21T08:37:24.291273+0000","last_peered":"2024-04-21T08:37:24.291273+0000","last_clean":"2024-04-21T08:37:24.291273+0000","last_became_active":"2024-04-21T08:36:24.923542+0000","last_became_peered":"2024-04-21T08:36:24.923542+0000","last_unstale":"2024-04-21T08:37:24.291273+0000","last_undegraded":"2024-04-21T08:37:24.291273+0000","last_fullsized":"2024-04-21T08:37:24.291273+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:14:31.282802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.291247+0000","last_change":"2024-04-21T08:36:20.288644+0000","last_active":"2024-04-21T08:37:24.291247+0000","last_peered":"2024-04-21T08:37:24.291247+0000","last_clean":"2024-04-21T08:37:24.291247+0000","last_became_active":"2024-04-21T08:36:20.288069+0000","last_became_peered":"2024-04-21T08:36:20.288069+0000","last_unstale":"2024-04-21T08:37:24.291247+0000","last_undegraded":"2024-04-21T08:37:24.291247+0000","last_fullsized":"2024-04-21T08:37:24.291247+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:37:36.522713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.309870+0000","last_change":"2024-04-21T08:36:20.908505+0000","last_active":"2024-04-21T08:37:53.309870+0000","last_peered":"2024-04-21T08:37:53.309870+0000","last_clean":"2024-04-21T08:37:53.309870+0000","last_became_active":"2024-04-21T08:36:20.908163+0000","last_became_peered":"2024-04-21T08:36:20.908163+0000","last_unstale":"2024-04-21T08:37:53.309870+0000","last_undegraded":"2024-04-21T08:37:53.309870+0000","last_fullsized":"2024-04-21T08:37:53.309870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:54:18.530203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.288322+0000","last_change":"2024-04-21T08:36:24.919428+0000","last_active":"2024-04-21T08:37:24.288322+0000","last_peered":"2024-04-21T08:37:24.288322+0000","last_clean":"2024-04-21T08:37:24.288322+0000","last_became_active":"2024-04-21T08:36:24.918722+0000","last_became_peered":"2024-04-21T08:36:24.918722+0000","last_unstale":"2024-04-21T08:37:24.288322+0000","last_undegraded":"2024-04-21T08:37:24.288322+0000","last_fullsized":"2024-04-21T08:37:24.288322+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:30:32.369551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":109,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:55.546132+0000","last_change":"2024-04-21T08:36:20.287642+0000","last_active":"2024-04-21T08:37:55.546132+0000","last_peered":"2024-04-21T08:37:55.546132+0000","last_clean":"2024-04-21T08:37:55.546132+0000","last_became_active":"2024-04-21T08:36:20.287296+0000","last_became_peered":"2024-04-21T08:36:20.287296+0000","last_unstale":"2024-04-21T08:37:55.546132+0000","last_undegraded":"2024-04-21T08:37:55.546132+0000","last_fullsized":"2024-04-21T08:37:55.546132+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:59:28.579020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":76,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.289873+0000","last_change":"2024-04-21T08:36:20.916971+0000","last_active":"2024-04-21T08:37:53.289873+0000","last_peered":"2024-04-21T08:37:53.289873+0000","last_clean":"2024-04-21T08:37:53.289873+0000","last_became_active":"2024-04-21T08:36:20.916622+0000","last_became_peered":"2024-04-21T08:36:20.916622+0000","last_unstale":"2024-04-21T08:37:53.289873+0000","last_undegraded":"2024-04-21T08:37:53.289873+0000","last_fullsized":"2024-04-21T08:37:53.289873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:49:38.858772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.288279+0000","last_change":"2024-04-21T08:36:24.924170+0000","last_active":"2024-04-21T08:37:24.288279+0000","last_peered":"2024-04-21T08:37:24.288279+0000","last_clean":"2024-04-21T08:37:24.288279+0000","last_became_active":"2024-04-21T08:36:24.923744+0000","last_became_peered":"2024-04-21T08:36:24.923744+0000","last_unstale":"2024-04-21T08:37:24.288279+0000","last_undegraded":"2024-04-21T08:37:24.288279+0000","last_fullsized":"2024-04-21T08:37:24.288279+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:23:24.605021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.290957+0000","last_change":"2024-04-21T08:36:20.286545+0000","last_active":"2024-04-21T08:37:24.290957+0000","last_peered":"2024-04-21T08:37:24.290957+0000","last_clean":"2024-04-21T08:37:24.290957+0000","last_became_active":"2024-04-21T08:36:20.286197+0000","last_became_peered":"2024-04-21T08:36:20.286197+0000","last_unstale":"2024-04-21T08:37:24.290957+0000","last_undegraded":"2024-04-21T08:37:24.290957+0000","last_fullsized":"2024-04-21T08:37:24.290957+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:27:55.430428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.301472+0000","last_change":"2024-04-21T08:36:20.907743+0000","last_active":"2024-04-21T08:37:53.301472+0000","last_peered":"2024-04-21T08:37:53.301472+0000","last_clean":"2024-04-21T08:37:53.301472+0000","last_became_active":"2024-04-21T08:36:20.907163+0000","last_became_peered":"2024-04-21T08:36:20.907163+0000","last_unstale":"2024-04-21T08:37:53.301472+0000","last_undegraded":"2024-04-21T08:37:53.301472+0000","last_fullsized":"2024-04-21T08:37:53.301472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:41:00.172353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.301529+0000","last_change":"2024-04-21T08:37:24.301529+0000","last_active":"2024-04-21T08:37:24.301529+0000","last_peered":"2024-04-21T08:37:24.301529+0000","last_clean":"2024-04-21T08:37:24.301529+0000","last_became_active":"2024-04-21T08:37:24.301014+0000","last_became_peered":"2024-04-21T08:37:24.301014+0000","last_unstale":"2024-04-21T08:37:24.301529+0000","last_undegraded":"2024-04-21T08:37:24.301529+0000","last_fullsized":"2024-04-21T08:37:24.301529+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:58:20.088501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.290429+0000","last_change":"2024-04-21T08:36:20.283143+0000","last_active":"2024-04-21T08:37:24.290429+0000","last_peered":"2024-04-21T08:37:24.290429+0000","last_clean":"2024-04-21T08:37:24.290429+0000","last_became_active":"2024-04-21T08:36:20.282585+0000","last_became_peered":"2024-04-21T08:36:20.282585+0000","last_unstale":"2024-04-21T08:37:24.290429+0000","last_undegraded":"2024-04-21T08:37:24.290429+0000","last_fullsized":"2024-04-21T08:37:24.290429+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:12:37.588764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.304009+0000","last_change":"2024-04-21T08:36:20.912591+0000","last_active":"2024-04-21T08:37:53.304009+0000","last_peered":"2024-04-21T08:37:53.304009+0000","last_clean":"2024-04-21T08:37:53.304009+0000","last_became_active":"2024-04-21T08:36:20.912174+0000","last_became_peered":"2024-04-21T08:36:20.912174+0000","last_unstale":"2024-04-21T08:37:53.304009+0000","last_undegraded":"2024-04-21T08:37:53.304009+0000","last_fullsized":"2024-04-21T08:37:53.304009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:33:58.949506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.294603+0000","last_change":"2024-04-21T08:36:24.915998+0000","last_active":"2024-04-21T08:37:24.294603+0000","last_peered":"2024-04-21T08:37:24.294603+0000","last_clean":"2024-04-21T08:37:24.294603+0000","last_became_active":"2024-04-21T08:36:24.915621+0000","last_became_peered":"2024-04-21T08:36:24.915621+0000","last_unstale":"2024-04-21T08:37:24.294603+0000","last_undegraded":"2024-04-21T08:37:24.294603+0000","last_fullsized":"2024-04-21T08:37:24.294603+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:43:05.117181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.290949+0000","last_change":"2024-04-21T08:36:20.286308+0000","last_active":"2024-04-21T08:37:24.290949+0000","last_peered":"2024-04-21T08:37:24.290949+0000","last_clean":"2024-04-21T08:37:24.290949+0000","last_became_active":"2024-04-21T08:36:20.285975+0000","last_became_peered":"2024-04-21T08:36:20.285975+0000","last_unstale":"2024-04-21T08:37:24.290949+0000","last_undegraded":"2024-04-21T08:37:24.290949+0000","last_fullsized":"2024-04-21T08:37:24.290949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:55:10.658349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.301022+0000","last_change":"2024-04-21T08:36:20.900502+0000","last_active":"2024-04-21T08:37:53.301022+0000","last_peered":"2024-04-21T08:37:53.301022+0000","last_clean":"2024-04-21T08:37:53.301022+0000","last_became_active":"2024-04-21T08:36:20.900199+0000","last_became_peered":"2024-04-21T08:36:20.900199+0000","last_unstale":"2024-04-21T08:37:53.301022+0000","last_undegraded":"2024-04-21T08:37:53.301022+0000","last_fullsized":"2024-04-21T08:37:53.301022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:17:00.720918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.293324+0000","last_change":"2024-04-21T08:36:24.920991+0000","last_active":"2024-04-21T08:37:24.293324+0000","last_peered":"2024-04-21T08:37:24.293324+0000","last_clean":"2024-04-21T08:37:24.293324+0000","last_became_active":"2024-04-21T08:36:24.916611+0000","last_became_peered":"2024-04-21T08:36:24.916611+0000","last_unstale":"2024-04-21T08:37:24.293324+0000","last_undegraded":"2024-04-21T08:37:24.293324+0000","last_fullsized":"2024-04-21T08:37:24.293324+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:01:01.412127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'6","reported_seq":168,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.541716+0000","last_change":"2024-04-21T08:36:20.284966+0000","last_active":"2024-04-21T08:37:53.541716+0000","last_peered":"2024-04-21T08:37:53.541716+0000","last_clean":"2024-04-21T08:37:53.541716+0000","last_became_active":"2024-04-21T08:36:20.284612+0000","last_became_peered":"2024-04-21T08:36:20.284612+0000","last_unstale":"2024-04-21T08:37:53.541716+0000","last_undegraded":"2024-04-21T08:37:53.541716+0000","last_fullsized":"2024-04-21T08:37:53.541716+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:54:50.095013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":136,"num_read_kb":136,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.309944+0000","last_change":"2024-04-21T08:36:20.903493+0000","last_active":"2024-04-21T08:37:53.309944+0000","last_peered":"2024-04-21T08:37:53.309944+0000","last_clean":"2024-04-21T08:37:53.309944+0000","last_became_active":"2024-04-21T08:36:20.902752+0000","last_became_peered":"2024-04-21T08:36:20.902752+0000","last_unstale":"2024-04-21T08:37:53.309944+0000","last_undegraded":"2024-04-21T08:37:53.309944+0000","last_fullsized":"2024-04-21T08:37:53.309944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:40.684264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.289103+0000","last_change":"2024-04-21T08:36:24.920505+0000","last_active":"2024-04-21T08:37:24.289103+0000","last_peered":"2024-04-21T08:37:24.289103+0000","last_clean":"2024-04-21T08:37:24.289103+0000","last_became_active":"2024-04-21T08:36:24.919966+0000","last_became_peered":"2024-04-21T08:36:24.919966+0000","last_unstale":"2024-04-21T08:37:24.289103+0000","last_undegraded":"2024-04-21T08:37:24.289103+0000","last_fullsized":"2024-04-21T08:37:24.289103+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:58:05.131815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.292920+0000","last_change":"2024-04-21T08:36:20.281720+0000","last_active":"2024-04-21T08:37:24.292920+0000","last_peered":"2024-04-21T08:37:24.292920+0000","last_clean":"2024-04-21T08:37:24.292920+0000","last_became_active":"2024-04-21T08:36:20.279862+0000","last_became_peered":"2024-04-21T08:36:20.279862+0000","last_unstale":"2024-04-21T08:37:24.292920+0000","last_undegraded":"2024-04-21T08:37:24.292920+0000","last_fullsized":"2024-04-21T08:37:24.292920+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:18:58.106532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.307190+0000","last_change":"2024-04-21T08:36:20.908520+0000","last_active":"2024-04-21T08:37:53.307190+0000","last_peered":"2024-04-21T08:37:53.307190+0000","last_clean":"2024-04-21T08:37:53.307190+0000","last_became_active":"2024-04-21T08:36:20.908184+0000","last_became_peered":"2024-04-21T08:36:20.908184+0000","last_unstale":"2024-04-21T08:37:53.307190+0000","last_undegraded":"2024-04-21T08:37:53.307190+0000","last_fullsized":"2024-04-21T08:37:53.307190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:14:33.895864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.291901+0000","last_change":"2024-04-21T08:36:24.920697+0000","last_active":"2024-04-21T08:37:24.291901+0000","last_peered":"2024-04-21T08:37:24.291901+0000","last_clean":"2024-04-21T08:37:24.291901+0000","last_became_active":"2024-04-21T08:36:24.919976+0000","last_became_peered":"2024-04-21T08:36:24.919976+0000","last_unstale":"2024-04-21T08:37:24.291901+0000","last_undegraded":"2024-04-21T08:37:24.291901+0000","last_fullsized":"2024-04-21T08:37:24.291901+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:40:09.238887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.293783+0000","last_change":"2024-04-21T08:36:20.283531+0000","last_active":"2024-04-21T08:37:24.293783+0000","last_peered":"2024-04-21T08:37:24.293783+0000","last_clean":"2024-04-21T08:37:24.293783+0000","last_became_active":"2024-04-21T08:36:20.283159+0000","last_became_peered":"2024-04-21T08:36:20.283159+0000","last_unstale":"2024-04-21T08:37:24.293783+0000","last_undegraded":"2024-04-21T08:37:24.293783+0000","last_fullsized":"2024-04-21T08:37:24.293783+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:57:02.917537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.305531+0000","last_change":"2024-04-21T08:36:20.910113+0000","last_active":"2024-04-21T08:37:53.305531+0000","last_peered":"2024-04-21T08:37:53.305531+0000","last_clean":"2024-04-21T08:37:53.305531+0000","last_became_active":"2024-04-21T08:36:20.909744+0000","last_became_peered":"2024-04-21T08:36:20.909744+0000","last_unstale":"2024-04-21T08:37:53.305531+0000","last_undegraded":"2024-04-21T08:37:53.305531+0000","last_fullsized":"2024-04-21T08:37:53.305531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:16:14.568318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.289215+0000","last_change":"2024-04-21T08:36:24.922232+0000","last_active":"2024-04-21T08:37:24.289215+0000","last_peered":"2024-04-21T08:37:24.289215+0000","last_clean":"2024-04-21T08:37:24.289215+0000","last_became_active":"2024-04-21T08:36:24.921690+0000","last_became_peered":"2024-04-21T08:36:24.921690+0000","last_unstale":"2024-04-21T08:37:24.289215+0000","last_undegraded":"2024-04-21T08:37:24.289215+0000","last_fullsized":"2024-04-21T08:37:24.289215+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:21:42.580203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'37","reported_seq":210,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.551914+0000","last_change":"2024-04-21T08:36:20.283574+0000","last_active":"2024-04-21T08:37:53.551914+0000","last_peered":"2024-04-21T08:37:53.551914+0000","last_clean":"2024-04-21T08:37:53.551914+0000","last_became_active":"2024-04-21T08:36:20.283220+0000","last_became_peered":"2024-04-21T08:36:20.283220+0000","last_unstale":"2024-04-21T08:37:53.551914+0000","last_undegraded":"2024-04-21T08:37:53.551914+0000","last_fullsized":"2024-04-21T08:37:53.551914+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:55:24.233237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63605,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":147,"num_read_kb":136,"num_write":39,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.293969+0000","last_change":"2024-04-21T08:36:20.919264+0000","last_active":"2024-04-21T08:37:53.293969+0000","last_peered":"2024-04-21T08:37:53.293969+0000","last_clean":"2024-04-21T08:37:53.293969+0000","last_became_active":"2024-04-21T08:36:20.918878+0000","last_became_peered":"2024-04-21T08:36:20.918878+0000","last_unstale":"2024-04-21T08:37:53.293969+0000","last_undegraded":"2024-04-21T08:37:53.293969+0000","last_fullsized":"2024-04-21T08:37:53.293969+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:24:06.460758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.293974+0000","last_change":"2024-04-21T08:36:24.926878+0000","last_active":"2024-04-21T08:37:24.293974+0000","last_peered":"2024-04-21T08:37:24.293974+0000","last_clean":"2024-04-21T08:37:24.293974+0000","last_became_active":"2024-04-21T08:36:24.926431+0000","last_became_peered":"2024-04-21T08:36:24.926431+0000","last_unstale":"2024-04-21T08:37:24.293974+0000","last_undegraded":"2024-04-21T08:37:24.293974+0000","last_fullsized":"2024-04-21T08:37:24.293974+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:51:29.065962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.288373+0000","last_change":"2024-04-21T08:36:20.284841+0000","last_active":"2024-04-21T08:37:24.288373+0000","last_peered":"2024-04-21T08:37:24.288373+0000","last_clean":"2024-04-21T08:37:24.288373+0000","last_became_active":"2024-04-21T08:36:20.283888+0000","last_became_peered":"2024-04-21T08:36:20.283888+0000","last_unstale":"2024-04-21T08:37:24.288373+0000","last_undegraded":"2024-04-21T08:37:24.288373+0000","last_fullsized":"2024-04-21T08:37:24.288373+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:29:42.659608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.302269+0000","last_change":"2024-04-21T08:36:20.905140+0000","last_active":"2024-04-21T08:37:53.302269+0000","last_peered":"2024-04-21T08:37:53.302269+0000","last_clean":"2024-04-21T08:37:53.302269+0000","last_became_active":"2024-04-21T08:36:20.904821+0000","last_became_peered":"2024-04-21T08:36:20.904821+0000","last_unstale":"2024-04-21T08:37:53.302269+0000","last_undegraded":"2024-04-21T08:37:53.302269+0000","last_fullsized":"2024-04-21T08:37:53.302269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:09:11.962877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.292823+0000","last_change":"2024-04-21T08:36:24.912062+0000","last_active":"2024-04-21T08:37:24.292823+0000","last_peered":"2024-04-21T08:37:24.292823+0000","last_clean":"2024-04-21T08:37:24.292823+0000","last_became_active":"2024-04-21T08:36:24.911600+0000","last_became_peered":"2024-04-21T08:36:24.911600+0000","last_unstale":"2024-04-21T08:37:24.292823+0000","last_undegraded":"2024-04-21T08:37:24.292823+0000","last_fullsized":"2024-04-21T08:37:24.292823+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:31:59.617161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.291066+0000","last_change":"2024-04-21T08:36:20.290289+0000","last_active":"2024-04-21T08:37:24.291066+0000","last_peered":"2024-04-21T08:37:24.291066+0000","last_clean":"2024-04-21T08:37:24.291066+0000","last_became_active":"2024-04-21T08:36:20.289922+0000","last_became_peered":"2024-04-21T08:36:20.289922+0000","last_unstale":"2024-04-21T08:37:24.291066+0000","last_undegraded":"2024-04-21T08:37:24.291066+0000","last_fullsized":"2024-04-21T08:37:24.291066+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:10:00.926023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.303846+0000","last_change":"2024-04-21T08:36:20.906950+0000","last_active":"2024-04-21T08:37:53.303846+0000","last_peered":"2024-04-21T08:37:53.303846+0000","last_clean":"2024-04-21T08:37:53.303846+0000","last_became_active":"2024-04-21T08:36:20.906599+0000","last_became_peered":"2024-04-21T08:36:20.906599+0000","last_unstale":"2024-04-21T08:37:53.303846+0000","last_undegraded":"2024-04-21T08:37:53.303846+0000","last_fullsized":"2024-04-21T08:37:53.303846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:55:27.595997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.289665+0000","last_change":"2024-04-21T08:36:24.924349+0000","last_active":"2024-04-21T08:37:24.289665+0000","last_peered":"2024-04-21T08:37:24.289665+0000","last_clean":"2024-04-21T08:37:24.289665+0000","last_became_active":"2024-04-21T08:36:24.923982+0000","last_became_peered":"2024-04-21T08:36:24.923982+0000","last_unstale":"2024-04-21T08:37:24.289665+0000","last_undegraded":"2024-04-21T08:37:24.289665+0000","last_fullsized":"2024-04-21T08:37:24.289665+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:20:29.600759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'7","reported_seq":111,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:57.563256+0000","last_change":"2024-04-21T08:36:20.285629+0000","last_active":"2024-04-21T08:37:57.563256+0000","last_peered":"2024-04-21T08:37:57.563256+0000","last_clean":"2024-04-21T08:37:57.563256+0000","last_became_active":"2024-04-21T08:36:20.285234+0000","last_became_peered":"2024-04-21T08:36:20.285234+0000","last_unstale":"2024-04-21T08:37:57.563256+0000","last_undegraded":"2024-04-21T08:37:57.563256+0000","last_fullsized":"2024-04-21T08:37:57.563256+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:19:32.966146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.294729+0000","last_change":"2024-04-21T08:36:20.906819+0000","last_active":"2024-04-21T08:37:53.294729+0000","last_peered":"2024-04-21T08:37:53.294729+0000","last_clean":"2024-04-21T08:37:53.294729+0000","last_became_active":"2024-04-21T08:36:20.906459+0000","last_became_peered":"2024-04-21T08:36:20.906459+0000","last_unstale":"2024-04-21T08:37:53.294729+0000","last_undegraded":"2024-04-21T08:37:53.294729+0000","last_fullsized":"2024-04-21T08:37:53.294729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:12:39.312857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.291312+0000","last_change":"2024-04-21T08:36:24.929452+0000","last_active":"2024-04-21T08:37:24.291312+0000","last_peered":"2024-04-21T08:37:24.291312+0000","last_clean":"2024-04-21T08:37:24.291312+0000","last_became_active":"2024-04-21T08:36:24.929039+0000","last_became_peered":"2024-04-21T08:36:24.929039+0000","last_unstale":"2024-04-21T08:37:24.291312+0000","last_undegraded":"2024-04-21T08:37:24.291312+0000","last_fullsized":"2024-04-21T08:37:24.291312+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:16:05.432024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":69,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.719766+0000","last_change":"2024-04-21T08:37:24.719766+0000","last_active":"2024-04-21T08:37:24.719766+0000","last_peered":"2024-04-21T08:37:24.719766+0000","last_clean":"2024-04-21T08:37:24.719766+0000","last_became_active":"2024-04-21T08:37:24.708201+0000","last_became_peered":"2024-04-21T08:37:24.708201+0000","last_unstale":"2024-04-21T08:37:24.719766+0000","last_undegraded":"2024-04-21T08:37:24.719766+0000","last_fullsized":"2024-04-21T08:37:24.719766+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:10:51.158061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.308405+0000","last_change":"2024-04-21T08:36:20.909076+0000","last_active":"2024-04-21T08:37:53.308405+0000","last_peered":"2024-04-21T08:37:53.308405+0000","last_clean":"2024-04-21T08:37:53.308405+0000","last_became_active":"2024-04-21T08:36:20.908739+0000","last_became_peered":"2024-04-21T08:36:20.908739+0000","last_unstale":"2024-04-21T08:37:53.308405+0000","last_undegraded":"2024-04-21T08:37:53.308405+0000","last_fullsized":"2024-04-21T08:37:53.308405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:03:56.870477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.293591+0000","last_change":"2024-04-21T08:36:24.926871+0000","last_active":"2024-04-21T08:37:24.293591+0000","last_peered":"2024-04-21T08:37:24.293591+0000","last_clean":"2024-04-21T08:37:24.293591+0000","last_became_active":"2024-04-21T08:36:24.926438+0000","last_became_peered":"2024-04-21T08:36:24.926438+0000","last_unstale":"2024-04-21T08:37:24.293591+0000","last_undegraded":"2024-04-21T08:37:24.293591+0000","last_fullsized":"2024-04-21T08:37:24.293591+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:24:48.984906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.308569+0000","last_change":"2024-04-21T08:37:24.308038+0000","last_active":"2024-04-21T08:37:24.308569+0000","last_peered":"2024-04-21T08:37:24.308569+0000","last_clean":"2024-04-21T08:37:24.308569+0000","last_became_active":"2024-04-21T08:37:24.301034+0000","last_became_peered":"2024-04-21T08:37:24.301034+0000","last_unstale":"2024-04-21T08:37:24.308569+0000","last_undegraded":"2024-04-21T08:37:24.308569+0000","last_fullsized":"2024-04-21T08:37:24.308569+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:37:06.598400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.284965+0000","last_change":"2024-04-21T08:37:24.299260+0000","last_active":"2024-04-21T08:37:53.284965+0000","last_peered":"2024-04-21T08:37:53.284965+0000","last_clean":"2024-04-21T08:37:53.284965+0000","last_became_active":"2024-04-21T08:37:24.298831+0000","last_became_peered":"2024-04-21T08:37:24.298831+0000","last_unstale":"2024-04-21T08:37:53.284965+0000","last_undegraded":"2024-04-21T08:37:53.284965+0000","last_fullsized":"2024-04-21T08:37:53.284965+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:58:29.368027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.290736+0000","last_change":"2024-04-21T08:36:24.929420+0000","last_active":"2024-04-21T08:37:24.290736+0000","last_peered":"2024-04-21T08:37:24.290736+0000","last_clean":"2024-04-21T08:37:24.290736+0000","last_became_active":"2024-04-21T08:36:24.928981+0000","last_became_peered":"2024-04-21T08:36:24.928981+0000","last_unstale":"2024-04-21T08:37:24.290736+0000","last_undegraded":"2024-04-21T08:37:24.290736+0000","last_fullsized":"2024-04-21T08:37:24.290736+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:38:00.040456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.289571+0000","last_change":"2024-04-21T08:36:20.279910+0000","last_active":"2024-04-21T08:37:24.289571+0000","last_peered":"2024-04-21T08:37:24.289571+0000","last_clean":"2024-04-21T08:37:24.289571+0000","last_became_active":"2024-04-21T08:36:20.278904+0000","last_became_peered":"2024-04-21T08:36:20.278904+0000","last_unstale":"2024-04-21T08:37:24.289571+0000","last_undegraded":"2024-04-21T08:37:24.289571+0000","last_fullsized":"2024-04-21T08:37:24.289571+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:40:19.209213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.305424+0000","last_change":"2024-04-21T08:36:20.895816+0000","last_active":"2024-04-21T08:37:53.305424+0000","last_peered":"2024-04-21T08:37:53.305424+0000","last_clean":"2024-04-21T08:37:53.305424+0000","last_became_active":"2024-04-21T08:36:20.895487+0000","last_became_peered":"2024-04-21T08:36:20.895487+0000","last_unstale":"2024-04-21T08:37:53.305424+0000","last_undegraded":"2024-04-21T08:37:53.305424+0000","last_fullsized":"2024-04-21T08:37:53.305424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:53:52.153683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.291156+0000","last_change":"2024-04-21T08:36:24.917637+0000","last_active":"2024-04-21T08:37:24.291156+0000","last_peered":"2024-04-21T08:37:24.291156+0000","last_clean":"2024-04-21T08:37:24.291156+0000","last_became_active":"2024-04-21T08:36:24.917192+0000","last_became_peered":"2024-04-21T08:36:24.917192+0000","last_unstale":"2024-04-21T08:37:24.291156+0000","last_undegraded":"2024-04-21T08:37:24.291156+0000","last_fullsized":"2024-04-21T08:37:24.291156+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:02:59.843903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":103,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:57.340400+0000","last_change":"2024-04-21T08:36:20.286424+0000","last_active":"2024-04-21T08:37:57.340400+0000","last_peered":"2024-04-21T08:37:57.340400+0000","last_clean":"2024-04-21T08:37:57.340400+0000","last_became_active":"2024-04-21T08:36:20.285767+0000","last_became_peered":"2024-04-21T08:36:20.285767+0000","last_unstale":"2024-04-21T08:37:57.340400+0000","last_undegraded":"2024-04-21T08:37:57.340400+0000","last_fullsized":"2024-04-21T08:37:57.340400+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:53:33.202734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.301527+0000","last_change":"2024-04-21T08:37:24.297650+0000","last_active":"2024-04-21T08:37:53.301527+0000","last_peered":"2024-04-21T08:37:53.301527+0000","last_clean":"2024-04-21T08:37:53.301527+0000","last_became_active":"2024-04-21T08:37:24.297250+0000","last_became_peered":"2024-04-21T08:37:24.297250+0000","last_unstale":"2024-04-21T08:37:53.301527+0000","last_undegraded":"2024-04-21T08:37:53.301527+0000","last_fullsized":"2024-04-21T08:37:53.301527+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:21:21.419556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.291071+0000","last_change":"2024-04-21T08:36:24.919669+0000","last_active":"2024-04-21T08:37:24.291071+0000","last_peered":"2024-04-21T08:37:24.291071+0000","last_clean":"2024-04-21T08:37:24.291071+0000","last_became_active":"2024-04-21T08:36:24.919281+0000","last_became_peered":"2024-04-21T08:36:24.919281+0000","last_unstale":"2024-04-21T08:37:24.291071+0000","last_undegraded":"2024-04-21T08:37:24.291071+0000","last_fullsized":"2024-04-21T08:37:24.291071+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:44:27.480367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.292747+0000","last_change":"2024-04-21T08:37:24.297826+0000","last_active":"2024-04-21T08:37:53.292747+0000","last_peered":"2024-04-21T08:37:53.292747+0000","last_clean":"2024-04-21T08:37:53.292747+0000","last_became_active":"2024-04-21T08:37:24.297397+0000","last_became_peered":"2024-04-21T08:37:24.297397+0000","last_unstale":"2024-04-21T08:37:53.292747+0000","last_undegraded":"2024-04-21T08:37:53.292747+0000","last_fullsized":"2024-04-21T08:37:53.292747+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:52:17.693725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":107,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:56.561050+0000","last_change":"2024-04-21T08:36:20.291033+0000","last_active":"2024-04-21T08:37:56.561050+0000","last_peered":"2024-04-21T08:37:56.561050+0000","last_clean":"2024-04-21T08:37:56.561050+0000","last_became_active":"2024-04-21T08:36:20.290697+0000","last_became_peered":"2024-04-21T08:36:20.290697+0000","last_unstale":"2024-04-21T08:37:56.561050+0000","last_undegraded":"2024-04-21T08:37:56.561050+0000","last_fullsized":"2024-04-21T08:37:56.561050+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:22:54.593595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":74,"num_read_kb":74,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.310936+0000","last_change":"2024-04-21T08:37:24.310936+0000","last_active":"2024-04-21T08:37:24.310936+0000","last_peered":"2024-04-21T08:37:24.310936+0000","last_clean":"2024-04-21T08:37:24.310936+0000","last_became_active":"2024-04-21T08:37:24.297349+0000","last_became_peered":"2024-04-21T08:37:24.297349+0000","last_unstale":"2024-04-21T08:37:24.310936+0000","last_undegraded":"2024-04-21T08:37:24.310936+0000","last_fullsized":"2024-04-21T08:37:24.310936+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:31:39.884609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.310460+0000","last_change":"2024-04-21T08:36:20.904197+0000","last_active":"2024-04-21T08:37:53.310460+0000","last_peered":"2024-04-21T08:37:53.310460+0000","last_clean":"2024-04-21T08:37:53.310460+0000","last_became_active":"2024-04-21T08:36:20.903099+0000","last_became_peered":"2024-04-21T08:36:20.903099+0000","last_unstale":"2024-04-21T08:37:53.310460+0000","last_undegraded":"2024-04-21T08:37:53.310460+0000","last_fullsized":"2024-04-21T08:37:53.310460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:16:08.914335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.291788+0000","last_change":"2024-04-21T08:36:20.280608+0000","last_active":"2024-04-21T08:37:24.291788+0000","last_peered":"2024-04-21T08:37:24.291788+0000","last_clean":"2024-04-21T08:37:24.291788+0000","last_became_active":"2024-04-21T08:36:20.280172+0000","last_became_peered":"2024-04-21T08:36:20.280172+0000","last_unstale":"2024-04-21T08:37:24.291788+0000","last_undegraded":"2024-04-21T08:37:24.291788+0000","last_fullsized":"2024-04-21T08:37:24.291788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:51:12.284463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":360,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.292472+0000","last_change":"2024-04-21T08:35:08.423545+0000","last_active":"2024-04-21T08:37:24.292472+0000","last_peered":"2024-04-21T08:37:24.292472+0000","last_clean":"2024-04-21T08:37:24.292472+0000","last_became_active":"2024-04-21T08:35:08.423027+0000","last_became_peered":"2024-04-21T08:35:08.423027+0000","last_unstale":"2024-04-21T08:37:24.292472+0000","last_undegraded":"2024-04-21T08:37:24.292472+0000","last_fullsized":"2024-04-21T08:37:24.292472+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:31:45.359253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:31:45.359253+0000","last_clean_scrub_stamp":"2024-04-21T08:31:45.359253+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:53:24.310824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.293980+0000","last_change":"2024-04-21T08:36:24.922422+0000","last_active":"2024-04-21T08:37:24.293980+0000","last_peered":"2024-04-21T08:37:24.293980+0000","last_clean":"2024-04-21T08:37:24.293980+0000","last_became_active":"2024-04-21T08:36:24.922006+0000","last_became_peered":"2024-04-21T08:36:24.922006+0000","last_unstale":"2024-04-21T08:37:24.293980+0000","last_undegraded":"2024-04-21T08:37:24.293980+0000","last_fullsized":"2024-04-21T08:37:24.293980+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:46:22.840802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.297239+0000","last_change":"2024-04-21T08:36:20.911222+0000","last_active":"2024-04-21T08:37:53.297239+0000","last_peered":"2024-04-21T08:37:53.297239+0000","last_clean":"2024-04-21T08:37:53.297239+0000","last_became_active":"2024-04-21T08:36:20.910933+0000","last_became_peered":"2024-04-21T08:36:20.910933+0000","last_unstale":"2024-04-21T08:37:53.297239+0000","last_undegraded":"2024-04-21T08:37:53.297239+0000","last_fullsized":"2024-04-21T08:37:53.297239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:41:25.621266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.292533+0000","last_change":"2024-04-21T08:36:20.288409+0000","last_active":"2024-04-21T08:37:24.292533+0000","last_peered":"2024-04-21T08:37:24.292533+0000","last_clean":"2024-04-21T08:37:24.292533+0000","last_became_active":"2024-04-21T08:36:20.287916+0000","last_became_peered":"2024-04-21T08:36:20.287916+0000","last_unstale":"2024-04-21T08:37:24.292533+0000","last_undegraded":"2024-04-21T08:37:24.292533+0000","last_fullsized":"2024-04-21T08:37:24.292533+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:04:31.068352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.292720+0000","last_change":"2024-04-21T08:36:24.918166+0000","last_active":"2024-04-21T08:37:24.292720+0000","last_peered":"2024-04-21T08:37:24.292720+0000","last_clean":"2024-04-21T08:37:24.292720+0000","last_became_active":"2024-04-21T08:36:24.917745+0000","last_became_peered":"2024-04-21T08:36:24.917745+0000","last_unstale":"2024-04-21T08:37:24.292720+0000","last_undegraded":"2024-04-21T08:37:24.292720+0000","last_fullsized":"2024-04-21T08:37:24.292720+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:55:41.169681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.291577+0000","last_change":"2024-04-21T08:36:20.291079+0000","last_active":"2024-04-21T08:37:24.291577+0000","last_peered":"2024-04-21T08:37:24.291577+0000","last_clean":"2024-04-21T08:37:24.291577+0000","last_became_active":"2024-04-21T08:36:20.290720+0000","last_became_peered":"2024-04-21T08:36:20.290720+0000","last_unstale":"2024-04-21T08:37:24.291577+0000","last_undegraded":"2024-04-21T08:37:24.291577+0000","last_fullsized":"2024-04-21T08:37:24.291577+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:41.269029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.293500+0000","last_change":"2024-04-21T08:36:20.907416+0000","last_active":"2024-04-21T08:37:53.293500+0000","last_peered":"2024-04-21T08:37:53.293500+0000","last_clean":"2024-04-21T08:37:53.293500+0000","last_became_active":"2024-04-21T08:36:20.907005+0000","last_became_peered":"2024-04-21T08:36:20.907005+0000","last_unstale":"2024-04-21T08:37:53.293500+0000","last_undegraded":"2024-04-21T08:37:53.293500+0000","last_fullsized":"2024-04-21T08:37:53.293500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:58:17.979947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.292450+0000","last_change":"2024-04-21T08:36:24.921475+0000","last_active":"2024-04-21T08:37:24.292450+0000","last_peered":"2024-04-21T08:37:24.292450+0000","last_clean":"2024-04-21T08:37:24.292450+0000","last_became_active":"2024-04-21T08:36:24.921091+0000","last_became_peered":"2024-04-21T08:36:24.921091+0000","last_unstale":"2024-04-21T08:37:24.292450+0000","last_undegraded":"2024-04-21T08:37:24.292450+0000","last_fullsized":"2024-04-21T08:37:24.292450+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:28:14.224501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'13","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.289769+0000","last_change":"2024-04-21T08:36:20.288916+0000","last_active":"2024-04-21T08:37:24.289769+0000","last_peered":"2024-04-21T08:37:24.289769+0000","last_clean":"2024-04-21T08:37:24.289769+0000","last_became_active":"2024-04-21T08:36:20.288575+0000","last_became_peered":"2024-04-21T08:36:20.288575+0000","last_unstale":"2024-04-21T08:37:24.289769+0000","last_undegraded":"2024-04-21T08:37:24.289769+0000","last_fullsized":"2024-04-21T08:37:24.289769+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:52:05.795554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.303522+0000","last_change":"2024-04-21T08:36:20.912053+0000","last_active":"2024-04-21T08:37:53.303522+0000","last_peered":"2024-04-21T08:37:53.303522+0000","last_clean":"2024-04-21T08:37:53.303522+0000","last_became_active":"2024-04-21T08:36:20.911584+0000","last_became_peered":"2024-04-21T08:36:20.911584+0000","last_unstale":"2024-04-21T08:37:53.303522+0000","last_undegraded":"2024-04-21T08:37:53.303522+0000","last_fullsized":"2024-04-21T08:37:53.303522+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:48:36.369954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.287880+0000","last_change":"2024-04-21T08:36:24.919212+0000","last_active":"2024-04-21T08:37:24.287880+0000","last_peered":"2024-04-21T08:37:24.287880+0000","last_clean":"2024-04-21T08:37:24.287880+0000","last_became_active":"2024-04-21T08:36:24.918807+0000","last_became_peered":"2024-04-21T08:36:24.918807+0000","last_unstale":"2024-04-21T08:37:24.287880+0000","last_undegraded":"2024-04-21T08:37:24.287880+0000","last_fullsized":"2024-04-21T08:37:24.287880+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:25:50.816775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.294049+0000","last_change":"2024-04-21T08:36:20.282536+0000","last_active":"2024-04-21T08:37:24.294049+0000","last_peered":"2024-04-21T08:37:24.294049+0000","last_clean":"2024-04-21T08:37:24.294049+0000","last_became_active":"2024-04-21T08:36:20.282172+0000","last_became_peered":"2024-04-21T08:36:20.282172+0000","last_unstale":"2024-04-21T08:37:24.294049+0000","last_undegraded":"2024-04-21T08:37:24.294049+0000","last_fullsized":"2024-04-21T08:37:24.294049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:15:38.992441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.306910+0000","last_change":"2024-04-21T08:36:20.908416+0000","last_active":"2024-04-21T08:37:53.306910+0000","last_peered":"2024-04-21T08:37:53.306910+0000","last_clean":"2024-04-21T08:37:53.306910+0000","last_became_active":"2024-04-21T08:36:20.908054+0000","last_became_peered":"2024-04-21T08:36:20.908054+0000","last_unstale":"2024-04-21T08:37:53.306910+0000","last_undegraded":"2024-04-21T08:37:53.306910+0000","last_fullsized":"2024-04-21T08:37:53.306910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:23:16.562060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.292421+0000","last_change":"2024-04-21T08:36:24.920637+0000","last_active":"2024-04-21T08:37:24.292421+0000","last_peered":"2024-04-21T08:37:24.292421+0000","last_clean":"2024-04-21T08:37:24.292421+0000","last_became_active":"2024-04-21T08:36:24.919921+0000","last_became_peered":"2024-04-21T08:36:24.919921+0000","last_unstale":"2024-04-21T08:37:24.292421+0000","last_undegraded":"2024-04-21T08:37:24.292421+0000","last_fullsized":"2024-04-21T08:37:24.292421+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:20:38.230087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.707218+0000","last_change":"2024-04-21T08:36:20.279332+0000","last_active":"2024-04-21T08:37:24.707218+0000","last_peered":"2024-04-21T08:37:24.707218+0000","last_clean":"2024-04-21T08:37:24.707218+0000","last_became_active":"2024-04-21T08:36:20.278970+0000","last_became_peered":"2024-04-21T08:36:20.278970+0000","last_unstale":"2024-04-21T08:37:24.707218+0000","last_undegraded":"2024-04-21T08:37:24.707218+0000","last_fullsized":"2024-04-21T08:37:24.707218+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:36:16.041610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.259529+0000","last_change":"2024-04-21T08:36:20.919621+0000","last_active":"2024-04-21T08:37:53.259529+0000","last_peered":"2024-04-21T08:37:53.259529+0000","last_clean":"2024-04-21T08:37:53.259529+0000","last_became_active":"2024-04-21T08:36:20.919246+0000","last_became_peered":"2024-04-21T08:36:20.919246+0000","last_unstale":"2024-04-21T08:37:53.259529+0000","last_undegraded":"2024-04-21T08:37:53.259529+0000","last_fullsized":"2024-04-21T08:37:53.259529+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:31:59.001501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.289794+0000","last_change":"2024-04-21T08:36:24.920459+0000","last_active":"2024-04-21T08:37:24.289794+0000","last_peered":"2024-04-21T08:37:24.289794+0000","last_clean":"2024-04-21T08:37:24.289794+0000","last_became_active":"2024-04-21T08:36:24.919691+0000","last_became_peered":"2024-04-21T08:36:24.919691+0000","last_unstale":"2024-04-21T08:37:24.289794+0000","last_undegraded":"2024-04-21T08:37:24.289794+0000","last_fullsized":"2024-04-21T08:37:24.289794+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:38:26.655452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.293040+0000","last_change":"2024-04-21T08:36:20.284286+0000","last_active":"2024-04-21T08:37:24.293040+0000","last_peered":"2024-04-21T08:37:24.293040+0000","last_clean":"2024-04-21T08:37:24.293040+0000","last_became_active":"2024-04-21T08:36:20.283903+0000","last_became_peered":"2024-04-21T08:36:20.283903+0000","last_unstale":"2024-04-21T08:37:24.293040+0000","last_undegraded":"2024-04-21T08:37:24.293040+0000","last_fullsized":"2024-04-21T08:37:24.293040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:51:52.409988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.299685+0000","last_change":"2024-04-21T08:36:20.911892+0000","last_active":"2024-04-21T08:37:53.299685+0000","last_peered":"2024-04-21T08:37:53.299685+0000","last_clean":"2024-04-21T08:37:53.299685+0000","last_became_active":"2024-04-21T08:36:20.911513+0000","last_became_peered":"2024-04-21T08:36:20.911513+0000","last_unstale":"2024-04-21T08:37:53.299685+0000","last_undegraded":"2024-04-21T08:37:53.299685+0000","last_fullsized":"2024-04-21T08:37:53.299685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:51:13.733534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.291554+0000","last_change":"2024-04-21T08:36:24.920347+0000","last_active":"2024-04-21T08:37:24.291554+0000","last_peered":"2024-04-21T08:37:24.291554+0000","last_clean":"2024-04-21T08:37:24.291554+0000","last_became_active":"2024-04-21T08:36:24.919831+0000","last_became_peered":"2024-04-21T08:36:24.919831+0000","last_unstale":"2024-04-21T08:37:24.291554+0000","last_undegraded":"2024-04-21T08:37:24.291554+0000","last_fullsized":"2024-04-21T08:37:24.291554+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:14:27.020476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.288651+0000","last_change":"2024-04-21T08:36:20.286786+0000","last_active":"2024-04-21T08:37:24.288651+0000","last_peered":"2024-04-21T08:37:24.288651+0000","last_clean":"2024-04-21T08:37:24.288651+0000","last_became_active":"2024-04-21T08:36:20.286429+0000","last_became_peered":"2024-04-21T08:36:20.286429+0000","last_unstale":"2024-04-21T08:37:24.288651+0000","last_undegraded":"2024-04-21T08:37:24.288651+0000","last_fullsized":"2024-04-21T08:37:24.288651+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:40:57.093370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.305474+0000","last_change":"2024-04-21T08:36:20.904767+0000","last_active":"2024-04-21T08:37:53.305474+0000","last_peered":"2024-04-21T08:37:53.305474+0000","last_clean":"2024-04-21T08:37:53.305474+0000","last_became_active":"2024-04-21T08:36:20.904406+0000","last_became_peered":"2024-04-21T08:36:20.904406+0000","last_unstale":"2024-04-21T08:37:53.305474+0000","last_undegraded":"2024-04-21T08:37:53.305474+0000","last_fullsized":"2024-04-21T08:37:53.305474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:11:41.644549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.294150+0000","last_change":"2024-04-21T08:36:24.918031+0000","last_active":"2024-04-21T08:37:24.294150+0000","last_peered":"2024-04-21T08:37:24.294150+0000","last_clean":"2024-04-21T08:37:24.294150+0000","last_became_active":"2024-04-21T08:36:24.917618+0000","last_became_peered":"2024-04-21T08:36:24.917618+0000","last_unstale":"2024-04-21T08:37:24.294150+0000","last_undegraded":"2024-04-21T08:37:24.294150+0000","last_fullsized":"2024-04-21T08:37:24.294150+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:50:35.797911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.290778+0000","last_change":"2024-04-21T08:36:20.286212+0000","last_active":"2024-04-21T08:37:24.290778+0000","last_peered":"2024-04-21T08:37:24.290778+0000","last_clean":"2024-04-21T08:37:24.290778+0000","last_became_active":"2024-04-21T08:36:20.285878+0000","last_became_peered":"2024-04-21T08:36:20.285878+0000","last_unstale":"2024-04-21T08:37:24.290778+0000","last_undegraded":"2024-04-21T08:37:24.290778+0000","last_fullsized":"2024-04-21T08:37:24.290778+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:19.375455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.306985+0000","last_change":"2024-04-21T08:36:20.908335+0000","last_active":"2024-04-21T08:37:53.306985+0000","last_peered":"2024-04-21T08:37:53.306985+0000","last_clean":"2024-04-21T08:37:53.306985+0000","last_became_active":"2024-04-21T08:36:20.907945+0000","last_became_peered":"2024-04-21T08:36:20.907945+0000","last_unstale":"2024-04-21T08:37:53.306985+0000","last_undegraded":"2024-04-21T08:37:53.306985+0000","last_fullsized":"2024-04-21T08:37:53.306985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:47:17.841511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.706560+0000","last_change":"2024-04-21T08:36:24.915553+0000","last_active":"2024-04-21T08:37:24.706560+0000","last_peered":"2024-04-21T08:37:24.706560+0000","last_clean":"2024-04-21T08:37:24.706560+0000","last_became_active":"2024-04-21T08:36:24.915028+0000","last_became_peered":"2024-04-21T08:36:24.915028+0000","last_unstale":"2024-04-21T08:37:24.706560+0000","last_undegraded":"2024-04-21T08:37:24.706560+0000","last_fullsized":"2024-04-21T08:37:24.706560+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:17:21.192946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'7","reported_seq":105,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.567960+0000","last_change":"2024-04-21T08:36:20.279649+0000","last_active":"2024-04-21T08:37:53.567960+0000","last_peered":"2024-04-21T08:37:53.567960+0000","last_clean":"2024-04-21T08:37:53.567960+0000","last_became_active":"2024-04-21T08:36:20.279111+0000","last_became_peered":"2024-04-21T08:36:20.279111+0000","last_unstale":"2024-04-21T08:37:53.567960+0000","last_undegraded":"2024-04-21T08:37:53.567960+0000","last_fullsized":"2024-04-21T08:37:53.567960+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:19:57.380495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":71,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.307325+0000","last_change":"2024-04-21T08:36:20.911489+0000","last_active":"2024-04-21T08:37:53.307325+0000","last_peered":"2024-04-21T08:37:53.307325+0000","last_clean":"2024-04-21T08:37:53.307325+0000","last_became_active":"2024-04-21T08:36:20.911168+0000","last_became_peered":"2024-04-21T08:36:20.911168+0000","last_unstale":"2024-04-21T08:37:53.307325+0000","last_undegraded":"2024-04-21T08:37:53.307325+0000","last_fullsized":"2024-04-21T08:37:53.307325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:40:41.585841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.293211+0000","last_change":"2024-04-21T08:36:24.921175+0000","last_active":"2024-04-21T08:37:24.293211+0000","last_peered":"2024-04-21T08:37:24.293211+0000","last_clean":"2024-04-21T08:37:24.293211+0000","last_became_active":"2024-04-21T08:36:24.920802+0000","last_became_peered":"2024-04-21T08:36:24.920802+0000","last_unstale":"2024-04-21T08:37:24.293211+0000","last_undegraded":"2024-04-21T08:37:24.293211+0000","last_fullsized":"2024-04-21T08:37:24.293211+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:43:12.140016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.291285+0000","last_change":"2024-04-21T08:36:20.290202+0000","last_active":"2024-04-21T08:37:24.291285+0000","last_peered":"2024-04-21T08:37:24.291285+0000","last_clean":"2024-04-21T08:37:24.291285+0000","last_became_active":"2024-04-21T08:36:20.289868+0000","last_became_peered":"2024-04-21T08:36:20.289868+0000","last_unstale":"2024-04-21T08:37:24.291285+0000","last_undegraded":"2024-04-21T08:37:24.291285+0000","last_fullsized":"2024-04-21T08:37:24.291285+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:07:45.525980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.301690+0000","last_change":"2024-04-21T08:36:20.899590+0000","last_active":"2024-04-21T08:37:53.301690+0000","last_peered":"2024-04-21T08:37:53.301690+0000","last_clean":"2024-04-21T08:37:53.301690+0000","last_became_active":"2024-04-21T08:36:20.899254+0000","last_became_peered":"2024-04-21T08:36:20.899254+0000","last_unstale":"2024-04-21T08:37:53.301690+0000","last_undegraded":"2024-04-21T08:37:53.301690+0000","last_fullsized":"2024-04-21T08:37:53.301690+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:34:25.409694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.290801+0000","last_change":"2024-04-21T08:36:24.918310+0000","last_active":"2024-04-21T08:37:24.290801+0000","last_peered":"2024-04-21T08:37:24.290801+0000","last_clean":"2024-04-21T08:37:24.290801+0000","last_became_active":"2024-04-21T08:36:24.917899+0000","last_became_peered":"2024-04-21T08:36:24.917899+0000","last_unstale":"2024-04-21T08:37:24.290801+0000","last_undegraded":"2024-04-21T08:37:24.290801+0000","last_fullsized":"2024-04-21T08:37:24.290801+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:09:13.006884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.304544+0000","last_change":"2024-04-21T08:37:24.304544+0000","last_active":"2024-04-21T08:37:24.304544+0000","last_peered":"2024-04-21T08:37:24.304544+0000","last_clean":"2024-04-21T08:37:24.304544+0000","last_became_active":"2024-04-21T08:37:24.295808+0000","last_became_peered":"2024-04-21T08:37:24.295808+0000","last_unstale":"2024-04-21T08:37:24.304544+0000","last_undegraded":"2024-04-21T08:37:24.304544+0000","last_fullsized":"2024-04-21T08:37:24.304544+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:07:15.797133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.296557+0000","last_change":"2024-04-21T08:36:20.912368+0000","last_active":"2024-04-21T08:37:53.296557+0000","last_peered":"2024-04-21T08:37:53.296557+0000","last_clean":"2024-04-21T08:37:53.296557+0000","last_became_active":"2024-04-21T08:36:20.912009+0000","last_became_peered":"2024-04-21T08:36:20.912009+0000","last_unstale":"2024-04-21T08:37:53.296557+0000","last_undegraded":"2024-04-21T08:37:53.296557+0000","last_fullsized":"2024-04-21T08:37:53.296557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:35:40.700314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.292751+0000","last_change":"2024-04-21T08:36:20.913472+0000","last_active":"2024-04-21T08:37:53.292751+0000","last_peered":"2024-04-21T08:37:53.292751+0000","last_clean":"2024-04-21T08:37:53.292751+0000","last_became_active":"2024-04-21T08:36:20.913118+0000","last_became_peered":"2024-04-21T08:36:20.913118+0000","last_unstale":"2024-04-21T08:37:53.292751+0000","last_undegraded":"2024-04-21T08:37:53.292751+0000","last_fullsized":"2024-04-21T08:37:53.292751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:31:53.957044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.298054+0000","last_change":"2024-04-21T08:36:20.913677+0000","last_active":"2024-04-21T08:37:53.298054+0000","last_peered":"2024-04-21T08:37:53.298054+0000","last_clean":"2024-04-21T08:37:53.298054+0000","last_became_active":"2024-04-21T08:36:20.913097+0000","last_became_peered":"2024-04-21T08:36:20.913097+0000","last_unstale":"2024-04-21T08:37:53.298054+0000","last_undegraded":"2024-04-21T08:37:53.298054+0000","last_fullsized":"2024-04-21T08:37:53.298054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:08:40.388370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.301817+0000","last_change":"2024-04-21T08:36:20.906624+0000","last_active":"2024-04-21T08:37:53.301817+0000","last_peered":"2024-04-21T08:37:53.301817+0000","last_clean":"2024-04-21T08:37:53.301817+0000","last_became_active":"2024-04-21T08:36:20.906247+0000","last_became_peered":"2024-04-21T08:36:20.906247+0000","last_unstale":"2024-04-21T08:37:53.301817+0000","last_undegraded":"2024-04-21T08:37:53.301817+0000","last_fullsized":"2024-04-21T08:37:53.301817+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:44:11.855183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.306768+0000","last_change":"2024-04-21T08:36:20.906576+0000","last_active":"2024-04-21T08:37:53.306768+0000","last_peered":"2024-04-21T08:37:53.306768+0000","last_clean":"2024-04-21T08:37:53.306768+0000","last_became_active":"2024-04-21T08:36:20.906231+0000","last_became_peered":"2024-04-21T08:36:20.906231+0000","last_unstale":"2024-04-21T08:37:53.306768+0000","last_undegraded":"2024-04-21T08:37:53.306768+0000","last_fullsized":"2024-04-21T08:37:53.306768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:42:48.156614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.304203+0000","last_change":"2024-04-21T08:36:20.902551+0000","last_active":"2024-04-21T08:37:53.304203+0000","last_peered":"2024-04-21T08:37:53.304203+0000","last_clean":"2024-04-21T08:37:53.304203+0000","last_became_active":"2024-04-21T08:36:20.902208+0000","last_became_peered":"2024-04-21T08:36:20.902208+0000","last_unstale":"2024-04-21T08:37:53.304203+0000","last_undegraded":"2024-04-21T08:37:53.304203+0000","last_fullsized":"2024-04-21T08:37:53.304203+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:36:30.016256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.290914+0000","last_change":"2024-04-21T08:36:20.918588+0000","last_active":"2024-04-21T08:37:53.290914+0000","last_peered":"2024-04-21T08:37:53.290914+0000","last_clean":"2024-04-21T08:37:53.290914+0000","last_became_active":"2024-04-21T08:36:20.918263+0000","last_became_peered":"2024-04-21T08:36:20.918263+0000","last_unstale":"2024-04-21T08:37:53.290914+0000","last_undegraded":"2024-04-21T08:37:53.290914+0000","last_fullsized":"2024-04-21T08:37:53.290914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:51:30.340336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.300582+0000","last_change":"2024-04-21T08:36:20.911063+0000","last_active":"2024-04-21T08:37:53.300582+0000","last_peered":"2024-04-21T08:37:53.300582+0000","last_clean":"2024-04-21T08:37:53.300582+0000","last_became_active":"2024-04-21T08:36:20.910647+0000","last_became_peered":"2024-04-21T08:36:20.910647+0000","last_unstale":"2024-04-21T08:37:53.300582+0000","last_undegraded":"2024-04-21T08:37:53.300582+0000","last_fullsized":"2024-04-21T08:37:53.300582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:10:31.079782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.304377+0000","last_change":"2024-04-21T08:36:20.914079+0000","last_active":"2024-04-21T08:37:53.304377+0000","last_peered":"2024-04-21T08:37:53.304377+0000","last_clean":"2024-04-21T08:37:53.304377+0000","last_became_active":"2024-04-21T08:36:20.912851+0000","last_became_peered":"2024-04-21T08:36:20.912851+0000","last_unstale":"2024-04-21T08:37:53.304377+0000","last_undegraded":"2024-04-21T08:37:53.304377+0000","last_fullsized":"2024-04-21T08:37:53.304377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:53:35.811281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.300652+0000","last_change":"2024-04-21T08:36:20.900551+0000","last_active":"2024-04-21T08:37:53.300652+0000","last_peered":"2024-04-21T08:37:53.300652+0000","last_clean":"2024-04-21T08:37:53.300652+0000","last_became_active":"2024-04-21T08:36:20.900173+0000","last_became_peered":"2024-04-21T08:36:20.900173+0000","last_unstale":"2024-04-21T08:37:53.300652+0000","last_undegraded":"2024-04-21T08:37:53.300652+0000","last_fullsized":"2024-04-21T08:37:53.300652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:21.389498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.304808+0000","last_change":"2024-04-21T08:36:20.912953+0000","last_active":"2024-04-21T08:37:53.304808+0000","last_peered":"2024-04-21T08:37:53.304808+0000","last_clean":"2024-04-21T08:37:53.304808+0000","last_became_active":"2024-04-21T08:36:20.912223+0000","last_became_peered":"2024-04-21T08:36:20.912223+0000","last_unstale":"2024-04-21T08:37:53.304808+0000","last_undegraded":"2024-04-21T08:37:53.304808+0000","last_fullsized":"2024-04-21T08:37:53.304808+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:43:14.277179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.309674+0000","last_change":"2024-04-21T08:36:20.906164+0000","last_active":"2024-04-21T08:37:53.309674+0000","last_peered":"2024-04-21T08:37:53.309674+0000","last_clean":"2024-04-21T08:37:53.309674+0000","last_became_active":"2024-04-21T08:36:20.905749+0000","last_became_peered":"2024-04-21T08:36:20.905749+0000","last_unstale":"2024-04-21T08:37:53.309674+0000","last_undegraded":"2024-04-21T08:37:53.309674+0000","last_fullsized":"2024-04-21T08:37:53.309674+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:02:10.569087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.304742+0000","last_change":"2024-04-21T08:36:20.900314+0000","last_active":"2024-04-21T08:37:53.304742+0000","last_peered":"2024-04-21T08:37:53.304742+0000","last_clean":"2024-04-21T08:37:53.304742+0000","last_became_active":"2024-04-21T08:36:20.900009+0000","last_became_peered":"2024-04-21T08:36:20.900009+0000","last_unstale":"2024-04-21T08:37:53.304742+0000","last_undegraded":"2024-04-21T08:37:53.304742+0000","last_fullsized":"2024-04-21T08:37:53.304742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:19:59.997302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.300407+0000","last_change":"2024-04-21T08:36:20.920725+0000","last_active":"2024-04-21T08:37:53.300407+0000","last_peered":"2024-04-21T08:37:53.300407+0000","last_clean":"2024-04-21T08:37:53.300407+0000","last_became_active":"2024-04-21T08:36:20.920288+0000","last_became_peered":"2024-04-21T08:36:20.920288+0000","last_unstale":"2024-04-21T08:37:53.300407+0000","last_undegraded":"2024-04-21T08:37:53.300407+0000","last_fullsized":"2024-04-21T08:37:53.300407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:36:18.736388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71623,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":735,"num_read_kb":439,"num_write":253,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":806,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":214869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":221,"ondisk_log_size":221,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383713,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1044480,"data_stored":502659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645349,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28708,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716444,"statfs":{"total":95995035648,"available":95965638656,"internally_reserved":0,"allocated":1732608,"data_stored":1156482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808875,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1056768,"data_stored":501363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070512,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":501719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332150,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1642496,"data_stored":1098239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495675,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":496867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574931,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1110016,"data_stored":561614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738456,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":498855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":6,"apply_latency_ms":6,"commit_latency_ns":6000000,"apply_latency_ns":6000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411406,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1114112,"data_stored":562674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182472,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":496575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":4,"apply_latency_ms":4,"commit_latency_ns":4000000,"apply_latency_ns":4000000},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920837,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1040384,"data_stored":501473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757311,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32796,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712356,"statfs":{"total":95995035648,"available":95961452544,"internally_reserved":0,"allocated":1658880,"data_stored":1093961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":2,"apply_latency_ms":2,"commit_latency_ns":2000000,"apply_latency_ns":2000000},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":65469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":66529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T08:38:01.105 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph pg dump --format=json 2024-04-21T08:38:01.347 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:38:01.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:01 smithi088 ceph-mon[31259]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:01.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:01 smithi179 ceph-mon[31829]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:01.786 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:01 smithi086 ceph-mon[27939]: from='client.15138 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:02.566 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:02 smithi086 ceph-mon[27939]: pgmap v340: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 11 op/s 2024-04-21T08:38:02.648 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:02 smithi179 ceph-mon[31829]: pgmap v340: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 11 op/s 2024-04-21T08:38:02.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:02 smithi088 ceph-mon[31259]: pgmap v340: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.0 KiB/s rd, 767 B/s wr, 11 op/s 2024-04-21T08:38:03.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.145+0000 7f6bcd8c9700 1 -- 172.21.15.86:0/658197518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bc8106ec0 msgr2=0x7f6bc810ea90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:03.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.145+0000 7f6bcd8c9700 1 --2- 172.21.15.86:0/658197518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bc8106ec0 0x7f6bc810ea90 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f6bc000b0d0 tx=0x7f6bc000b3e0 comp rx=0 tx=0).stop 2024-04-21T08:38:03.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.145+0000 7f6bcd8c9700 1 -- 172.21.15.86:0/658197518 shutdown_connections 2024-04-21T08:38:03.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.145+0000 7f6bcd8c9700 1 --2- 172.21.15.86:0/658197518 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bc8106ec0 0x7f6bc810ea90 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:03.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.145+0000 7f6bcd8c9700 1 --2- 172.21.15.86:0/658197518 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6bc8102300 0x7f6bc81026e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:03.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.145+0000 7f6bcd8c9700 1 --2- 172.21.15.86:0/658197518 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6bc8102cb0 0x7f6bc81067e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:03.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.145+0000 7f6bcd8c9700 1 -- 172.21.15.86:0/658197518 >> 172.21.15.86:0/658197518 conn(0x7f6bc8077f40 msgr2=0x7f6bc80ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:03.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.145+0000 7f6bcd8c9700 1 -- 172.21.15.86:0/658197518 shutdown_connections 2024-04-21T08:38:03.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.146+0000 7f6bcd8c9700 1 -- 172.21.15.86:0/658197518 wait complete. 2024-04-21T08:38:03.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.146+0000 7f6bcd8c9700 1 Processor -- start 2024-04-21T08:38:03.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.146+0000 7f6bcd8c9700 1 -- start start 2024-04-21T08:38:03.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.146+0000 7f6bcd8c9700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bc8102300 0x7f6bc81992f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:03.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.146+0000 7f6bcd8c9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6bc8102cb0 0x7f6bc8199830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:03.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.146+0000 7f6bcd8c9700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6bc8106ec0 0x7f6bc819dbc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:03.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.146+0000 7f6bcd8c9700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc8111d70 con 0x7f6bc8102300 2024-04-21T08:38:03.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.146+0000 7f6bcd8c9700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc8111bf0 con 0x7f6bc8106ec0 2024-04-21T08:38:03.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.146+0000 7f6bcd8c9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6bc8111ef0 con 0x7f6bc8102cb0 2024-04-21T08:38:03.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.147+0000 7f6bc6ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bc8102300 0x7f6bc81992f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:03.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.147+0000 7f6bc6ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bc8102300 0x7f6bc81992f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51090/0 (socket says 172.21.15.86:51090) 2024-04-21T08:38:03.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.147+0000 7f6bc6ffd700 1 -- 172.21.15.86:0/4081842137 learned_addr learned my addr 172.21.15.86:0/4081842137 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:03.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.147+0000 7f6bc67fc700 1 --2- 172.21.15.86:0/4081842137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6bc8102cb0 0x7f6bc8199830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:03.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.147+0000 7f6bc77fe700 1 --2- 172.21.15.86:0/4081842137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6bc8106ec0 0x7f6bc819dbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:03.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.147+0000 7f6bc6ffd700 1 -- 172.21.15.86:0/4081842137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6bc8102cb0 msgr2=0x7f6bc8199830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:03.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.147+0000 7f6bc6ffd700 1 --2- 172.21.15.86:0/4081842137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6bc8102cb0 0x7f6bc8199830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:03.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.147+0000 7f6bc6ffd700 1 -- 172.21.15.86:0/4081842137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6bc8106ec0 msgr2=0x7f6bc819dbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:03.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.147+0000 7f6bc6ffd700 1 --2- 172.21.15.86:0/4081842137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6bc8106ec0 0x7f6bc819dbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:03.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.147+0000 7f6bc6ffd700 1 -- 172.21.15.86:0/4081842137 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6bc819e300 con 0x7f6bc8102300 2024-04-21T08:38:03.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.147+0000 7f6bc6ffd700 1 --2- 172.21.15.86:0/4081842137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bc8102300 0x7f6bc81992f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6bb800c970 tx=0x7f6bb800cc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:03.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.147+0000 7f6bcc8c7700 1 -- 172.21.15.86:0/4081842137 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb8007aa0 con 0x7f6bc8102300 2024-04-21T08:38:03.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.147+0000 7f6bcd8c9700 1 -- 172.21.15.86:0/4081842137 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6bc819e5f0 con 0x7f6bc8102300 2024-04-21T08:38:03.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.147+0000 7f6bcc8c7700 1 -- 172.21.15.86:0/4081842137 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6bb8004ce0 con 0x7f6bc8102300 2024-04-21T08:38:03.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.147+0000 7f6bcc8c7700 1 -- 172.21.15.86:0/4081842137 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6bb8005510 con 0x7f6bc8102300 2024-04-21T08:38:03.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.147+0000 7f6bcd8c9700 1 -- 172.21.15.86:0/4081842137 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6bc81a5e70 con 0x7f6bc8102300 2024-04-21T08:38:03.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.148+0000 7f6bcd8c9700 1 -- 172.21.15.86:0/4081842137 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6bc8069730 con 0x7f6bc8102300 2024-04-21T08:38:03.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.149+0000 7f6bcc8c7700 1 -- 172.21.15.86:0/4081842137 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6bb8007c40 con 0x7f6bc8102300 2024-04-21T08:38:03.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.150+0000 7f6bcc8c7700 1 --2- 172.21.15.86:0/4081842137 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6bb00a3c40 0x7f6bb00a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:03.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.150+0000 7f6bc67fc700 1 --2- 172.21.15.86:0/4081842137 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6bb00a3c40 0x7f6bb00a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:03.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.150+0000 7f6bcc8c7700 1 -- 172.21.15.86:0/4081842137 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f6bb80d0530 con 0x7f6bc8102300 2024-04-21T08:38:03.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.150+0000 7f6bc67fc700 1 --2- 172.21.15.86:0/4081842137 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6bb00a3c40 0x7f6bb00a6100 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8108440 tx=0x7f6bbc00a300 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:03.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.155+0000 7f6bcc8c7700 1 -- 172.21.15.86:0/4081842137 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6bb809cc40 con 0x7f6bc8102300 2024-04-21T08:38:03.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.345+0000 7f6bcd8c9700 1 -- 172.21.15.86:0/4081842137 --> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f6bc806af90 con 0x7f6bb00a3c40 2024-04-21T08:38:03.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.357+0000 7f6bcc8c7700 1 -- 172.21.15.86:0/4081842137 <== mgr.14152 v2:172.21.15.86:6800/1364156965 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+340531 (secure 0 0 0) 0x7f6bc806af90 con 0x7f6bb00a3c40 2024-04-21T08:38:03.360 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:38:03.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.360+0000 7f6bcd8c9700 1 -- 172.21.15.86:0/4081842137 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6bb00a3c40 msgr2=0x7f6bb00a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:03.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.360+0000 7f6bcd8c9700 1 --2- 172.21.15.86:0/4081842137 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6bb00a3c40 0x7f6bb00a6100 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f6bc8108440 tx=0x7f6bbc00a300 comp rx=0 tx=0).stop 2024-04-21T08:38:03.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.360+0000 7f6bcd8c9700 1 -- 172.21.15.86:0/4081842137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bc8102300 msgr2=0x7f6bc81992f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:03.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.360+0000 7f6bcd8c9700 1 --2- 172.21.15.86:0/4081842137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bc8102300 0x7f6bc81992f0 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f6bb800c970 tx=0x7f6bb800cc80 comp rx=0 tx=0).stop 2024-04-21T08:38:03.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.360+0000 7f6bcd8c9700 1 -- 172.21.15.86:0/4081842137 shutdown_connections 2024-04-21T08:38:03.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.360+0000 7f6bcd8c9700 1 --2- 172.21.15.86:0/4081842137 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6bb00a3c40 0x7f6bb00a6100 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:03.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.360+0000 7f6bcd8c9700 1 --2- 172.21.15.86:0/4081842137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6bc8106ec0 0x7f6bc819dbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:03.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.360+0000 7f6bcd8c9700 1 --2- 172.21.15.86:0/4081842137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6bc8102300 0x7f6bc81992f0 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:03.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.360+0000 7f6bcd8c9700 1 --2- 172.21.15.86:0/4081842137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6bc8102cb0 0x7f6bc8199830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:03.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.360+0000 7f6bcd8c9700 1 -- 172.21.15.86:0/4081842137 >> 172.21.15.86:0/4081842137 conn(0x7f6bc8077f40 msgr2=0x7f6bc81011d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:03.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.361+0000 7f6bcd8c9700 1 -- 172.21.15.86:0/4081842137 shutdown_connections 2024-04-21T08:38:03.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:03.361+0000 7f6bcd8c9700 1 -- 172.21.15.86:0/4081842137 wait complete. 2024-04-21T08:38:03.364 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-21T08:38:04.078 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":341,"stamp":"2024-04-21T08:38:03.297404+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":662225,"num_objects":102,"num_object_clones":0,"num_object_copies":308,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":102,"num_whiteouts":0,"num_read":911,"num_read_kb":583,"num_write":436,"num_write_kb":3090,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":1181659,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":860,"ondisk_log_size":860,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":384192,"kb_used_data":14240,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124557632,"statfs":{"total":1151940427776,"available":1151547015168,"internally_reserved":0,"allocated":14581760,"data_stored":7972481,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":378713404},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":672,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":113,"num_read_kb":58,"num_write":11,"num_write_kb":9,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.003784"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.306918+0000","last_change":"2024-04-21T08:36:20.913421+0000","last_active":"2024-04-21T08:37:53.306918+0000","last_peered":"2024-04-21T08:37:53.306918+0000","last_clean":"2024-04-21T08:37:53.306918+0000","last_became_active":"2024-04-21T08:36:20.913017+0000","last_became_peered":"2024-04-21T08:36:20.913017+0000","last_unstale":"2024-04-21T08:37:53.306918+0000","last_undegraded":"2024-04-21T08:37:53.306918+0000","last_fullsized":"2024-04-21T08:37:53.306918+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:16:29.885916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.298871+0000","last_change":"2024-04-21T08:36:20.911675+0000","last_active":"2024-04-21T08:37:53.298871+0000","last_peered":"2024-04-21T08:37:53.298871+0000","last_clean":"2024-04-21T08:37:53.298871+0000","last_became_active":"2024-04-21T08:36:20.910591+0000","last_became_peered":"2024-04-21T08:36:20.910591+0000","last_unstale":"2024-04-21T08:37:53.298871+0000","last_undegraded":"2024-04-21T08:37:53.298871+0000","last_fullsized":"2024-04-21T08:37:53.298871+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:01:40.735376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.310261+0000","last_change":"2024-04-21T08:36:20.911978+0000","last_active":"2024-04-21T08:37:53.310261+0000","last_peered":"2024-04-21T08:37:53.310261+0000","last_clean":"2024-04-21T08:37:53.310261+0000","last_became_active":"2024-04-21T08:36:20.911641+0000","last_became_peered":"2024-04-21T08:36:20.911641+0000","last_unstale":"2024-04-21T08:37:53.310261+0000","last_undegraded":"2024-04-21T08:37:53.310261+0000","last_fullsized":"2024-04-21T08:37:53.310261+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:14:23.789790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.287047+0000","last_change":"2024-04-21T08:36:20.919027+0000","last_active":"2024-04-21T08:37:53.287047+0000","last_peered":"2024-04-21T08:37:53.287047+0000","last_clean":"2024-04-21T08:37:53.287047+0000","last_became_active":"2024-04-21T08:36:20.918740+0000","last_became_peered":"2024-04-21T08:36:20.918740+0000","last_unstale":"2024-04-21T08:37:53.287047+0000","last_undegraded":"2024-04-21T08:37:53.287047+0000","last_fullsized":"2024-04-21T08:37:53.287047+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:00:31.066938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.287750+0000","last_change":"2024-04-21T08:36:20.907175+0000","last_active":"2024-04-21T08:37:53.287750+0000","last_peered":"2024-04-21T08:37:53.287750+0000","last_clean":"2024-04-21T08:37:53.287750+0000","last_became_active":"2024-04-21T08:36:20.906818+0000","last_became_peered":"2024-04-21T08:36:20.906818+0000","last_unstale":"2024-04-21T08:37:53.287750+0000","last_undegraded":"2024-04-21T08:37:53.287750+0000","last_fullsized":"2024-04-21T08:37:53.287750+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:41:03.332231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.289860+0000","last_change":"2024-04-21T08:36:20.905968+0000","last_active":"2024-04-21T08:37:53.289860+0000","last_peered":"2024-04-21T08:37:53.289860+0000","last_clean":"2024-04-21T08:37:53.289860+0000","last_became_active":"2024-04-21T08:36:20.904840+0000","last_became_peered":"2024-04-21T08:36:20.904840+0000","last_unstale":"2024-04-21T08:37:53.289860+0000","last_undegraded":"2024-04-21T08:37:53.289860+0000","last_fullsized":"2024-04-21T08:37:53.289860+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:23:46.886241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.289034+0000","last_change":"2024-04-21T08:36:20.912045+0000","last_active":"2024-04-21T08:37:53.289034+0000","last_peered":"2024-04-21T08:37:53.289034+0000","last_clean":"2024-04-21T08:37:53.289034+0000","last_became_active":"2024-04-21T08:36:20.911690+0000","last_became_peered":"2024-04-21T08:36:20.911690+0000","last_unstale":"2024-04-21T08:37:53.289034+0000","last_undegraded":"2024-04-21T08:37:53.289034+0000","last_fullsized":"2024-04-21T08:37:53.289034+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:06:28.229307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.305439+0000","last_change":"2024-04-21T08:36:20.904874+0000","last_active":"2024-04-21T08:37:53.305439+0000","last_peered":"2024-04-21T08:37:53.305439+0000","last_clean":"2024-04-21T08:37:53.305439+0000","last_became_active":"2024-04-21T08:36:20.904217+0000","last_became_peered":"2024-04-21T08:36:20.904217+0000","last_unstale":"2024-04-21T08:37:53.305439+0000","last_undegraded":"2024-04-21T08:37:53.305439+0000","last_fullsized":"2024-04-21T08:37:53.305439+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:37:04.423812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.310061+0000","last_change":"2024-04-21T08:36:20.912874+0000","last_active":"2024-04-21T08:37:53.310061+0000","last_peered":"2024-04-21T08:37:53.310061+0000","last_clean":"2024-04-21T08:37:53.310061+0000","last_became_active":"2024-04-21T08:36:20.912569+0000","last_became_peered":"2024-04-21T08:36:20.912569+0000","last_unstale":"2024-04-21T08:37:53.310061+0000","last_undegraded":"2024-04-21T08:37:53.310061+0000","last_fullsized":"2024-04-21T08:37:53.310061+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:55:20.763157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":46,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.297432+0000","last_change":"2024-04-21T08:37:24.296916+0000","last_active":"2024-04-21T08:37:53.297432+0000","last_peered":"2024-04-21T08:37:53.297432+0000","last_clean":"2024-04-21T08:37:53.297432+0000","last_became_active":"2024-04-21T08:37:24.296547+0000","last_became_peered":"2024-04-21T08:37:24.296547+0000","last_unstale":"2024-04-21T08:37:53.297432+0000","last_undegraded":"2024-04-21T08:37:53.297432+0000","last_fullsized":"2024-04-21T08:37:53.297432+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:47:56.683793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.307187+0000","last_change":"2024-04-21T08:36:20.909094+0000","last_active":"2024-04-21T08:37:53.307187+0000","last_peered":"2024-04-21T08:37:53.307187+0000","last_clean":"2024-04-21T08:37:53.307187+0000","last_became_active":"2024-04-21T08:36:20.908634+0000","last_became_peered":"2024-04-21T08:36:20.908634+0000","last_unstale":"2024-04-21T08:37:53.307187+0000","last_undegraded":"2024-04-21T08:37:53.307187+0000","last_fullsized":"2024-04-21T08:37:53.307187+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:28:38.450621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.277892+0000","last_change":"2024-04-21T08:36:20.909429+0000","last_active":"2024-04-21T08:37:53.277892+0000","last_peered":"2024-04-21T08:37:53.277892+0000","last_clean":"2024-04-21T08:37:53.277892+0000","last_became_active":"2024-04-21T08:36:20.909081+0000","last_became_peered":"2024-04-21T08:36:20.909081+0000","last_unstale":"2024-04-21T08:37:53.277892+0000","last_undegraded":"2024-04-21T08:37:53.277892+0000","last_fullsized":"2024-04-21T08:37:53.277892+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:45:46.939125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.300525+0000","last_change":"2024-04-21T08:36:20.895894+0000","last_active":"2024-04-21T08:37:53.300525+0000","last_peered":"2024-04-21T08:37:53.300525+0000","last_clean":"2024-04-21T08:37:53.300525+0000","last_became_active":"2024-04-21T08:36:20.895572+0000","last_became_peered":"2024-04-21T08:36:20.895572+0000","last_unstale":"2024-04-21T08:37:53.300525+0000","last_undegraded":"2024-04-21T08:37:53.300525+0000","last_fullsized":"2024-04-21T08:37:53.300525+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:12:48.750555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.298339+0000","last_change":"2024-04-21T08:36:20.905356+0000","last_active":"2024-04-21T08:37:53.298339+0000","last_peered":"2024-04-21T08:37:53.298339+0000","last_clean":"2024-04-21T08:37:53.298339+0000","last_became_active":"2024-04-21T08:36:20.904954+0000","last_became_peered":"2024-04-21T08:36:20.904954+0000","last_unstale":"2024-04-21T08:37:53.298339+0000","last_undegraded":"2024-04-21T08:37:53.298339+0000","last_fullsized":"2024-04-21T08:37:53.298339+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:45:49.116482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.306501+0000","last_change":"2024-04-21T08:36:20.911550+0000","last_active":"2024-04-21T08:37:53.306501+0000","last_peered":"2024-04-21T08:37:53.306501+0000","last_clean":"2024-04-21T08:37:53.306501+0000","last_became_active":"2024-04-21T08:36:20.910557+0000","last_became_peered":"2024-04-21T08:36:20.910557+0000","last_unstale":"2024-04-21T08:37:53.306501+0000","last_undegraded":"2024-04-21T08:37:53.306501+0000","last_fullsized":"2024-04-21T08:37:53.306501+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:09:49.422729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.307606+0000","last_change":"2024-04-21T08:36:20.903954+0000","last_active":"2024-04-21T08:37:53.307606+0000","last_peered":"2024-04-21T08:37:53.307606+0000","last_clean":"2024-04-21T08:37:53.307606+0000","last_became_active":"2024-04-21T08:36:20.903641+0000","last_became_peered":"2024-04-21T08:36:20.903641+0000","last_unstale":"2024-04-21T08:37:53.307606+0000","last_undegraded":"2024-04-21T08:37:53.307606+0000","last_fullsized":"2024-04-21T08:37:53.307606+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:08:55.678940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.307105+0000","last_change":"2024-04-21T08:36:20.914256+0000","last_active":"2024-04-21T08:37:53.307105+0000","last_peered":"2024-04-21T08:37:53.307105+0000","last_clean":"2024-04-21T08:37:53.307105+0000","last_became_active":"2024-04-21T08:36:20.913858+0000","last_became_peered":"2024-04-21T08:36:20.913858+0000","last_unstale":"2024-04-21T08:37:53.307105+0000","last_undegraded":"2024-04-21T08:37:53.307105+0000","last_fullsized":"2024-04-21T08:37:53.307105+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:01:10.276763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.295256+0000","last_change":"2024-04-21T08:36:20.904138+0000","last_active":"2024-04-21T08:37:53.295256+0000","last_peered":"2024-04-21T08:37:53.295256+0000","last_clean":"2024-04-21T08:37:53.295256+0000","last_became_active":"2024-04-21T08:36:20.903670+0000","last_became_peered":"2024-04-21T08:36:20.903670+0000","last_unstale":"2024-04-21T08:37:53.295256+0000","last_undegraded":"2024-04-21T08:37:53.295256+0000","last_fullsized":"2024-04-21T08:37:53.295256+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:38:50.160867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.302234+0000","last_change":"2024-04-21T08:36:20.910798+0000","last_active":"2024-04-21T08:37:53.302234+0000","last_peered":"2024-04-21T08:37:53.302234+0000","last_clean":"2024-04-21T08:37:53.302234+0000","last_became_active":"2024-04-21T08:36:20.910411+0000","last_became_peered":"2024-04-21T08:36:20.910411+0000","last_unstale":"2024-04-21T08:37:53.302234+0000","last_undegraded":"2024-04-21T08:37:53.302234+0000","last_fullsized":"2024-04-21T08:37:53.302234+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:13:12.109818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.288019+0000","last_change":"2024-04-21T08:36:24.924831+0000","last_active":"2024-04-21T08:37:24.288019+0000","last_peered":"2024-04-21T08:37:24.288019+0000","last_clean":"2024-04-21T08:37:24.288019+0000","last_became_active":"2024-04-21T08:36:24.924435+0000","last_became_peered":"2024-04-21T08:36:24.924435+0000","last_unstale":"2024-04-21T08:37:24.288019+0000","last_undegraded":"2024-04-21T08:37:24.288019+0000","last_fullsized":"2024-04-21T08:37:24.288019+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:23.447079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":133,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:58.341143+0000","last_change":"2024-04-21T08:37:24.308024+0000","last_active":"2024-04-21T08:37:58.341143+0000","last_peered":"2024-04-21T08:37:58.341143+0000","last_clean":"2024-04-21T08:37:58.341143+0000","last_became_active":"2024-04-21T08:37:24.297491+0000","last_became_peered":"2024-04-21T08:37:24.297491+0000","last_unstale":"2024-04-21T08:37:58.341143+0000","last_undegraded":"2024-04-21T08:37:58.341143+0000","last_fullsized":"2024-04-21T08:37:58.341143+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:48:10.039545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":66,"num_read_kb":66,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.305004+0000","last_change":"2024-04-21T08:36:20.912851+0000","last_active":"2024-04-21T08:37:53.305004+0000","last_peered":"2024-04-21T08:37:53.305004+0000","last_clean":"2024-04-21T08:37:53.305004+0000","last_became_active":"2024-04-21T08:36:20.912407+0000","last_became_peered":"2024-04-21T08:36:20.912407+0000","last_unstale":"2024-04-21T08:37:53.305004+0000","last_undegraded":"2024-04-21T08:37:53.305004+0000","last_fullsized":"2024-04-21T08:37:53.305004+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:45:18.026940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.290917+0000","last_change":"2024-04-21T08:36:24.924052+0000","last_active":"2024-04-21T08:37:24.290917+0000","last_peered":"2024-04-21T08:37:24.290917+0000","last_clean":"2024-04-21T08:37:24.290917+0000","last_became_active":"2024-04-21T08:36:24.923618+0000","last_became_peered":"2024-04-21T08:36:24.923618+0000","last_unstale":"2024-04-21T08:37:24.290917+0000","last_undegraded":"2024-04-21T08:37:24.290917+0000","last_fullsized":"2024-04-21T08:37:24.290917+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:08:18.182386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.293039+0000","last_change":"2024-04-21T08:36:20.285291+0000","last_active":"2024-04-21T08:37:24.293039+0000","last_peered":"2024-04-21T08:37:24.293039+0000","last_clean":"2024-04-21T08:37:24.293039+0000","last_became_active":"2024-04-21T08:36:20.284969+0000","last_became_peered":"2024-04-21T08:36:20.284969+0000","last_unstale":"2024-04-21T08:37:24.293039+0000","last_undegraded":"2024-04-21T08:37:24.293039+0000","last_fullsized":"2024-04-21T08:37:24.293039+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:55:29.484439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.310149+0000","last_change":"2024-04-21T08:36:20.902819+0000","last_active":"2024-04-21T08:37:53.310149+0000","last_peered":"2024-04-21T08:37:53.310149+0000","last_clean":"2024-04-21T08:37:53.310149+0000","last_became_active":"2024-04-21T08:36:20.902466+0000","last_became_peered":"2024-04-21T08:36:20.902466+0000","last_unstale":"2024-04-21T08:37:53.310149+0000","last_undegraded":"2024-04-21T08:37:53.310149+0000","last_fullsized":"2024-04-21T08:37:53.310149+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:29:40.454026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.291673+0000","last_change":"2024-04-21T08:36:24.926480+0000","last_active":"2024-04-21T08:37:24.291673+0000","last_peered":"2024-04-21T08:37:24.291673+0000","last_clean":"2024-04-21T08:37:24.291673+0000","last_became_active":"2024-04-21T08:36:24.926116+0000","last_became_peered":"2024-04-21T08:36:24.926116+0000","last_unstale":"2024-04-21T08:37:24.291673+0000","last_undegraded":"2024-04-21T08:37:24.291673+0000","last_fullsized":"2024-04-21T08:37:24.291673+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:43:18.839396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.292631+0000","last_change":"2024-04-21T08:36:20.293086+0000","last_active":"2024-04-21T08:37:24.292631+0000","last_peered":"2024-04-21T08:37:24.292631+0000","last_clean":"2024-04-21T08:37:24.292631+0000","last_became_active":"2024-04-21T08:36:20.291178+0000","last_became_peered":"2024-04-21T08:36:20.291178+0000","last_unstale":"2024-04-21T08:37:24.292631+0000","last_undegraded":"2024-04-21T08:37:24.292631+0000","last_fullsized":"2024-04-21T08:37:24.292631+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:02:32.935930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.310382+0000","last_change":"2024-04-21T08:36:20.909374+0000","last_active":"2024-04-21T08:37:53.310382+0000","last_peered":"2024-04-21T08:37:53.310382+0000","last_clean":"2024-04-21T08:37:53.310382+0000","last_became_active":"2024-04-21T08:36:20.909043+0000","last_became_peered":"2024-04-21T08:36:20.909043+0000","last_unstale":"2024-04-21T08:37:53.310382+0000","last_undegraded":"2024-04-21T08:37:53.310382+0000","last_fullsized":"2024-04-21T08:37:53.310382+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:45:45.719942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.291273+0000","last_change":"2024-04-21T08:36:24.923989+0000","last_active":"2024-04-21T08:37:24.291273+0000","last_peered":"2024-04-21T08:37:24.291273+0000","last_clean":"2024-04-21T08:37:24.291273+0000","last_became_active":"2024-04-21T08:36:24.923542+0000","last_became_peered":"2024-04-21T08:36:24.923542+0000","last_unstale":"2024-04-21T08:37:24.291273+0000","last_undegraded":"2024-04-21T08:37:24.291273+0000","last_fullsized":"2024-04-21T08:37:24.291273+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:14:31.282802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,5,3],"acting":[4,6,5,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.291247+0000","last_change":"2024-04-21T08:36:20.288644+0000","last_active":"2024-04-21T08:37:24.291247+0000","last_peered":"2024-04-21T08:37:24.291247+0000","last_clean":"2024-04-21T08:37:24.291247+0000","last_became_active":"2024-04-21T08:36:20.288069+0000","last_became_peered":"2024-04-21T08:36:20.288069+0000","last_unstale":"2024-04-21T08:37:24.291247+0000","last_undegraded":"2024-04-21T08:37:24.291247+0000","last_fullsized":"2024-04-21T08:37:24.291247+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:37:36.522713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.309870+0000","last_change":"2024-04-21T08:36:20.908505+0000","last_active":"2024-04-21T08:37:53.309870+0000","last_peered":"2024-04-21T08:37:53.309870+0000","last_clean":"2024-04-21T08:37:53.309870+0000","last_became_active":"2024-04-21T08:36:20.908163+0000","last_became_peered":"2024-04-21T08:36:20.908163+0000","last_unstale":"2024-04-21T08:37:53.309870+0000","last_undegraded":"2024-04-21T08:37:53.309870+0000","last_fullsized":"2024-04-21T08:37:53.309870+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:54:18.530203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.288322+0000","last_change":"2024-04-21T08:36:24.919428+0000","last_active":"2024-04-21T08:37:24.288322+0000","last_peered":"2024-04-21T08:37:24.288322+0000","last_clean":"2024-04-21T08:37:24.288322+0000","last_became_active":"2024-04-21T08:36:24.918722+0000","last_became_peered":"2024-04-21T08:36:24.918722+0000","last_unstale":"2024-04-21T08:37:24.288322+0000","last_undegraded":"2024-04-21T08:37:24.288322+0000","last_fullsized":"2024-04-21T08:37:24.288322+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:30:32.369551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"91'7","reported_seq":114,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:38:00.554747+0000","last_change":"2024-04-21T08:36:20.287642+0000","last_active":"2024-04-21T08:38:00.554747+0000","last_peered":"2024-04-21T08:38:00.554747+0000","last_clean":"2024-04-21T08:38:00.554747+0000","last_became_active":"2024-04-21T08:36:20.287296+0000","last_became_peered":"2024-04-21T08:36:20.287296+0000","last_unstale":"2024-04-21T08:38:00.554747+0000","last_undegraded":"2024-04-21T08:38:00.554747+0000","last_fullsized":"2024-04-21T08:38:00.554747+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:59:28.579020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1154,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":81,"num_read_kb":4,"num_write":4,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.289873+0000","last_change":"2024-04-21T08:36:20.916971+0000","last_active":"2024-04-21T08:37:53.289873+0000","last_peered":"2024-04-21T08:37:53.289873+0000","last_clean":"2024-04-21T08:37:53.289873+0000","last_became_active":"2024-04-21T08:36:20.916622+0000","last_became_peered":"2024-04-21T08:36:20.916622+0000","last_unstale":"2024-04-21T08:37:53.289873+0000","last_undegraded":"2024-04-21T08:37:53.289873+0000","last_fullsized":"2024-04-21T08:37:53.289873+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:49:38.858772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.288279+0000","last_change":"2024-04-21T08:36:24.924170+0000","last_active":"2024-04-21T08:37:24.288279+0000","last_peered":"2024-04-21T08:37:24.288279+0000","last_clean":"2024-04-21T08:37:24.288279+0000","last_became_active":"2024-04-21T08:36:24.923744+0000","last_became_peered":"2024-04-21T08:36:24.923744+0000","last_unstale":"2024-04-21T08:37:24.288279+0000","last_undegraded":"2024-04-21T08:37:24.288279+0000","last_fullsized":"2024-04-21T08:37:24.288279+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:23:24.605021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.290957+0000","last_change":"2024-04-21T08:36:20.286545+0000","last_active":"2024-04-21T08:37:24.290957+0000","last_peered":"2024-04-21T08:37:24.290957+0000","last_clean":"2024-04-21T08:37:24.290957+0000","last_became_active":"2024-04-21T08:36:20.286197+0000","last_became_peered":"2024-04-21T08:36:20.286197+0000","last_unstale":"2024-04-21T08:37:24.290957+0000","last_undegraded":"2024-04-21T08:37:24.290957+0000","last_fullsized":"2024-04-21T08:37:24.290957+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:27:55.430428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.301472+0000","last_change":"2024-04-21T08:36:20.907743+0000","last_active":"2024-04-21T08:37:53.301472+0000","last_peered":"2024-04-21T08:37:53.301472+0000","last_clean":"2024-04-21T08:37:53.301472+0000","last_became_active":"2024-04-21T08:36:20.907163+0000","last_became_peered":"2024-04-21T08:36:20.907163+0000","last_unstale":"2024-04-21T08:37:53.301472+0000","last_undegraded":"2024-04-21T08:37:53.301472+0000","last_fullsized":"2024-04-21T08:37:53.301472+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:41:00.172353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":25,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.301529+0000","last_change":"2024-04-21T08:37:24.301529+0000","last_active":"2024-04-21T08:37:24.301529+0000","last_peered":"2024-04-21T08:37:24.301529+0000","last_clean":"2024-04-21T08:37:24.301529+0000","last_became_active":"2024-04-21T08:37:24.301014+0000","last_became_peered":"2024-04-21T08:37:24.301014+0000","last_unstale":"2024-04-21T08:37:24.301529+0000","last_undegraded":"2024-04-21T08:37:24.301529+0000","last_fullsized":"2024-04-21T08:37:24.301529+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:58:20.088501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.290429+0000","last_change":"2024-04-21T08:36:20.283143+0000","last_active":"2024-04-21T08:37:24.290429+0000","last_peered":"2024-04-21T08:37:24.290429+0000","last_clean":"2024-04-21T08:37:24.290429+0000","last_became_active":"2024-04-21T08:36:20.282585+0000","last_became_peered":"2024-04-21T08:36:20.282585+0000","last_unstale":"2024-04-21T08:37:24.290429+0000","last_undegraded":"2024-04-21T08:37:24.290429+0000","last_fullsized":"2024-04-21T08:37:24.290429+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:12:37.588764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.304009+0000","last_change":"2024-04-21T08:36:20.912591+0000","last_active":"2024-04-21T08:37:53.304009+0000","last_peered":"2024-04-21T08:37:53.304009+0000","last_clean":"2024-04-21T08:37:53.304009+0000","last_became_active":"2024-04-21T08:36:20.912174+0000","last_became_peered":"2024-04-21T08:36:20.912174+0000","last_unstale":"2024-04-21T08:37:53.304009+0000","last_undegraded":"2024-04-21T08:37:53.304009+0000","last_fullsized":"2024-04-21T08:37:53.304009+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:33:58.949506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.294603+0000","last_change":"2024-04-21T08:36:24.915998+0000","last_active":"2024-04-21T08:37:24.294603+0000","last_peered":"2024-04-21T08:37:24.294603+0000","last_clean":"2024-04-21T08:37:24.294603+0000","last_became_active":"2024-04-21T08:36:24.915621+0000","last_became_peered":"2024-04-21T08:36:24.915621+0000","last_unstale":"2024-04-21T08:37:24.294603+0000","last_undegraded":"2024-04-21T08:37:24.294603+0000","last_fullsized":"2024-04-21T08:37:24.294603+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:43:05.117181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.290949+0000","last_change":"2024-04-21T08:36:20.286308+0000","last_active":"2024-04-21T08:37:24.290949+0000","last_peered":"2024-04-21T08:37:24.290949+0000","last_clean":"2024-04-21T08:37:24.290949+0000","last_became_active":"2024-04-21T08:36:20.285975+0000","last_became_peered":"2024-04-21T08:36:20.285975+0000","last_unstale":"2024-04-21T08:37:24.290949+0000","last_undegraded":"2024-04-21T08:37:24.290949+0000","last_fullsized":"2024-04-21T08:37:24.290949+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:55:10.658349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.301022+0000","last_change":"2024-04-21T08:36:20.900502+0000","last_active":"2024-04-21T08:37:53.301022+0000","last_peered":"2024-04-21T08:37:53.301022+0000","last_clean":"2024-04-21T08:37:53.301022+0000","last_became_active":"2024-04-21T08:36:20.900199+0000","last_became_peered":"2024-04-21T08:36:20.900199+0000","last_unstale":"2024-04-21T08:37:53.301022+0000","last_undegraded":"2024-04-21T08:37:53.301022+0000","last_fullsized":"2024-04-21T08:37:53.301022+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:17:00.720918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.293324+0000","last_change":"2024-04-21T08:36:24.920991+0000","last_active":"2024-04-21T08:37:24.293324+0000","last_peered":"2024-04-21T08:37:24.293324+0000","last_clean":"2024-04-21T08:37:24.293324+0000","last_became_active":"2024-04-21T08:36:24.916611+0000","last_became_peered":"2024-04-21T08:36:24.916611+0000","last_unstale":"2024-04-21T08:37:24.293324+0000","last_undegraded":"2024-04-21T08:37:24.293324+0000","last_fullsized":"2024-04-21T08:37:24.293324+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:01:01.412127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"91'6","reported_seq":178,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:58.550667+0000","last_change":"2024-04-21T08:36:20.284966+0000","last_active":"2024-04-21T08:37:58.550667+0000","last_peered":"2024-04-21T08:37:58.550667+0000","last_clean":"2024-04-21T08:37:58.550667+0000","last_became_active":"2024-04-21T08:36:20.284612+0000","last_became_peered":"2024-04-21T08:36:20.284612+0000","last_unstale":"2024-04-21T08:37:58.550667+0000","last_undegraded":"2024-04-21T08:37:58.550667+0000","last_fullsized":"2024-04-21T08:37:58.550667+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:54:50.095013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":146,"num_read_kb":146,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.309944+0000","last_change":"2024-04-21T08:36:20.903493+0000","last_active":"2024-04-21T08:37:53.309944+0000","last_peered":"2024-04-21T08:37:53.309944+0000","last_clean":"2024-04-21T08:37:53.309944+0000","last_became_active":"2024-04-21T08:36:20.902752+0000","last_became_peered":"2024-04-21T08:36:20.902752+0000","last_unstale":"2024-04-21T08:37:53.309944+0000","last_undegraded":"2024-04-21T08:37:53.309944+0000","last_fullsized":"2024-04-21T08:37:53.309944+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:40.684264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.289103+0000","last_change":"2024-04-21T08:36:24.920505+0000","last_active":"2024-04-21T08:37:24.289103+0000","last_peered":"2024-04-21T08:37:24.289103+0000","last_clean":"2024-04-21T08:37:24.289103+0000","last_became_active":"2024-04-21T08:36:24.919966+0000","last_became_peered":"2024-04-21T08:36:24.919966+0000","last_unstale":"2024-04-21T08:37:24.289103+0000","last_undegraded":"2024-04-21T08:37:24.289103+0000","last_fullsized":"2024-04-21T08:37:24.289103+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:58:05.131815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.292920+0000","last_change":"2024-04-21T08:36:20.281720+0000","last_active":"2024-04-21T08:37:24.292920+0000","last_peered":"2024-04-21T08:37:24.292920+0000","last_clean":"2024-04-21T08:37:24.292920+0000","last_became_active":"2024-04-21T08:36:20.279862+0000","last_became_peered":"2024-04-21T08:36:20.279862+0000","last_unstale":"2024-04-21T08:37:24.292920+0000","last_undegraded":"2024-04-21T08:37:24.292920+0000","last_fullsized":"2024-04-21T08:37:24.292920+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:18:58.106532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.307190+0000","last_change":"2024-04-21T08:36:20.908520+0000","last_active":"2024-04-21T08:37:53.307190+0000","last_peered":"2024-04-21T08:37:53.307190+0000","last_clean":"2024-04-21T08:37:53.307190+0000","last_became_active":"2024-04-21T08:36:20.908184+0000","last_became_peered":"2024-04-21T08:36:20.908184+0000","last_unstale":"2024-04-21T08:37:53.307190+0000","last_undegraded":"2024-04-21T08:37:53.307190+0000","last_fullsized":"2024-04-21T08:37:53.307190+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:14:33.895864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.291901+0000","last_change":"2024-04-21T08:36:24.920697+0000","last_active":"2024-04-21T08:37:24.291901+0000","last_peered":"2024-04-21T08:37:24.291901+0000","last_clean":"2024-04-21T08:37:24.291901+0000","last_became_active":"2024-04-21T08:36:24.919976+0000","last_became_peered":"2024-04-21T08:36:24.919976+0000","last_unstale":"2024-04-21T08:37:24.291901+0000","last_undegraded":"2024-04-21T08:37:24.291901+0000","last_fullsized":"2024-04-21T08:37:24.291901+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:40:09.238887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.293783+0000","last_change":"2024-04-21T08:36:20.283531+0000","last_active":"2024-04-21T08:37:24.293783+0000","last_peered":"2024-04-21T08:37:24.293783+0000","last_clean":"2024-04-21T08:37:24.293783+0000","last_became_active":"2024-04-21T08:36:20.283159+0000","last_became_peered":"2024-04-21T08:36:20.283159+0000","last_unstale":"2024-04-21T08:37:24.293783+0000","last_undegraded":"2024-04-21T08:37:24.293783+0000","last_fullsized":"2024-04-21T08:37:24.293783+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:57:02.917537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.305531+0000","last_change":"2024-04-21T08:36:20.910113+0000","last_active":"2024-04-21T08:37:53.305531+0000","last_peered":"2024-04-21T08:37:53.305531+0000","last_clean":"2024-04-21T08:37:53.305531+0000","last_became_active":"2024-04-21T08:36:20.909744+0000","last_became_peered":"2024-04-21T08:36:20.909744+0000","last_unstale":"2024-04-21T08:37:53.305531+0000","last_undegraded":"2024-04-21T08:37:53.305531+0000","last_fullsized":"2024-04-21T08:37:53.305531+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:16:14.568318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.289215+0000","last_change":"2024-04-21T08:36:24.922232+0000","last_active":"2024-04-21T08:37:24.289215+0000","last_peered":"2024-04-21T08:37:24.289215+0000","last_clean":"2024-04-21T08:37:24.289215+0000","last_became_active":"2024-04-21T08:36:24.921690+0000","last_became_peered":"2024-04-21T08:36:24.921690+0000","last_unstale":"2024-04-21T08:37:24.289215+0000","last_undegraded":"2024-04-21T08:37:24.289215+0000","last_fullsized":"2024-04-21T08:37:24.289215+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:21:42.580203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"91'37","reported_seq":220,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:58.558782+0000","last_change":"2024-04-21T08:36:20.283574+0000","last_active":"2024-04-21T08:37:58.558782+0000","last_peered":"2024-04-21T08:37:58.558782+0000","last_clean":"2024-04-21T08:37:58.558782+0000","last_became_active":"2024-04-21T08:36:20.283220+0000","last_became_peered":"2024-04-21T08:36:20.283220+0000","last_unstale":"2024-04-21T08:37:58.558782+0000","last_undegraded":"2024-04-21T08:37:58.558782+0000","last_fullsized":"2024-04-21T08:37:58.558782+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":37,"log_dups_size":0,"ondisk_log_size":37,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:55:24.233237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":63605,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":157,"num_read_kb":141,"num_write":39,"num_write_kb":79,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.293969+0000","last_change":"2024-04-21T08:36:20.919264+0000","last_active":"2024-04-21T08:37:53.293969+0000","last_peered":"2024-04-21T08:37:53.293969+0000","last_clean":"2024-04-21T08:37:53.293969+0000","last_became_active":"2024-04-21T08:36:20.918878+0000","last_became_peered":"2024-04-21T08:36:20.918878+0000","last_unstale":"2024-04-21T08:37:53.293969+0000","last_undegraded":"2024-04-21T08:37:53.293969+0000","last_fullsized":"2024-04-21T08:37:53.293969+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:24:06.460758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.293974+0000","last_change":"2024-04-21T08:36:24.926878+0000","last_active":"2024-04-21T08:37:24.293974+0000","last_peered":"2024-04-21T08:37:24.293974+0000","last_clean":"2024-04-21T08:37:24.293974+0000","last_became_active":"2024-04-21T08:36:24.926431+0000","last_became_peered":"2024-04-21T08:36:24.926431+0000","last_unstale":"2024-04-21T08:37:24.293974+0000","last_undegraded":"2024-04-21T08:37:24.293974+0000","last_fullsized":"2024-04-21T08:37:24.293974+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:51:29.065962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.288373+0000","last_change":"2024-04-21T08:36:20.284841+0000","last_active":"2024-04-21T08:37:24.288373+0000","last_peered":"2024-04-21T08:37:24.288373+0000","last_clean":"2024-04-21T08:37:24.288373+0000","last_became_active":"2024-04-21T08:36:20.283888+0000","last_became_peered":"2024-04-21T08:36:20.283888+0000","last_unstale":"2024-04-21T08:37:24.288373+0000","last_undegraded":"2024-04-21T08:37:24.288373+0000","last_fullsized":"2024-04-21T08:37:24.288373+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:29:42.659608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.302269+0000","last_change":"2024-04-21T08:36:20.905140+0000","last_active":"2024-04-21T08:37:53.302269+0000","last_peered":"2024-04-21T08:37:53.302269+0000","last_clean":"2024-04-21T08:37:53.302269+0000","last_became_active":"2024-04-21T08:36:20.904821+0000","last_became_peered":"2024-04-21T08:36:20.904821+0000","last_unstale":"2024-04-21T08:37:53.302269+0000","last_undegraded":"2024-04-21T08:37:53.302269+0000","last_fullsized":"2024-04-21T08:37:53.302269+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:09:11.962877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.292823+0000","last_change":"2024-04-21T08:36:24.912062+0000","last_active":"2024-04-21T08:37:24.292823+0000","last_peered":"2024-04-21T08:37:24.292823+0000","last_clean":"2024-04-21T08:37:24.292823+0000","last_became_active":"2024-04-21T08:36:24.911600+0000","last_became_peered":"2024-04-21T08:36:24.911600+0000","last_unstale":"2024-04-21T08:37:24.292823+0000","last_undegraded":"2024-04-21T08:37:24.292823+0000","last_fullsized":"2024-04-21T08:37:24.292823+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:31:59.617161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,7,6],"acting":[11,9,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.291066+0000","last_change":"2024-04-21T08:36:20.290289+0000","last_active":"2024-04-21T08:37:24.291066+0000","last_peered":"2024-04-21T08:37:24.291066+0000","last_clean":"2024-04-21T08:37:24.291066+0000","last_became_active":"2024-04-21T08:36:20.289922+0000","last_became_peered":"2024-04-21T08:36:20.289922+0000","last_unstale":"2024-04-21T08:37:24.291066+0000","last_undegraded":"2024-04-21T08:37:24.291066+0000","last_fullsized":"2024-04-21T08:37:24.291066+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:10:00.926023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":39,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.303846+0000","last_change":"2024-04-21T08:36:20.906950+0000","last_active":"2024-04-21T08:37:53.303846+0000","last_peered":"2024-04-21T08:37:53.303846+0000","last_clean":"2024-04-21T08:37:53.303846+0000","last_became_active":"2024-04-21T08:36:20.906599+0000","last_became_peered":"2024-04-21T08:36:20.906599+0000","last_unstale":"2024-04-21T08:37:53.303846+0000","last_undegraded":"2024-04-21T08:37:53.303846+0000","last_fullsized":"2024-04-21T08:37:53.303846+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:55:27.595997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.289665+0000","last_change":"2024-04-21T08:36:24.924349+0000","last_active":"2024-04-21T08:37:24.289665+0000","last_peered":"2024-04-21T08:37:24.289665+0000","last_clean":"2024-04-21T08:37:24.289665+0000","last_became_active":"2024-04-21T08:36:24.923982+0000","last_became_peered":"2024-04-21T08:36:24.923982+0000","last_unstale":"2024-04-21T08:37:24.289665+0000","last_undegraded":"2024-04-21T08:37:24.289665+0000","last_fullsized":"2024-04-21T08:37:24.289665+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:20:29.600759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"91'7","reported_seq":111,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:57.563256+0000","last_change":"2024-04-21T08:36:20.285629+0000","last_active":"2024-04-21T08:37:57.563256+0000","last_peered":"2024-04-21T08:37:57.563256+0000","last_clean":"2024-04-21T08:37:57.563256+0000","last_became_active":"2024-04-21T08:36:20.285234+0000","last_became_peered":"2024-04-21T08:36:20.285234+0000","last_unstale":"2024-04-21T08:37:57.563256+0000","last_undegraded":"2024-04-21T08:37:57.563256+0000","last_fullsized":"2024-04-21T08:37:57.563256+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:19:32.966146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1120,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":5,"num_write":7,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.294729+0000","last_change":"2024-04-21T08:36:20.906819+0000","last_active":"2024-04-21T08:37:53.294729+0000","last_peered":"2024-04-21T08:37:53.294729+0000","last_clean":"2024-04-21T08:37:53.294729+0000","last_became_active":"2024-04-21T08:36:20.906459+0000","last_became_peered":"2024-04-21T08:36:20.906459+0000","last_unstale":"2024-04-21T08:37:53.294729+0000","last_undegraded":"2024-04-21T08:37:53.294729+0000","last_fullsized":"2024-04-21T08:37:53.294729+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:12:39.312857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.291312+0000","last_change":"2024-04-21T08:36:24.929452+0000","last_active":"2024-04-21T08:37:24.291312+0000","last_peered":"2024-04-21T08:37:24.291312+0000","last_clean":"2024-04-21T08:37:24.291312+0000","last_became_active":"2024-04-21T08:36:24.929039+0000","last_became_peered":"2024-04-21T08:36:24.929039+0000","last_unstale":"2024-04-21T08:37:24.291312+0000","last_undegraded":"2024-04-21T08:37:24.291312+0000","last_fullsized":"2024-04-21T08:37:24.291312+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:16:05.432024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":69,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.719766+0000","last_change":"2024-04-21T08:37:24.719766+0000","last_active":"2024-04-21T08:37:24.719766+0000","last_peered":"2024-04-21T08:37:24.719766+0000","last_clean":"2024-04-21T08:37:24.719766+0000","last_became_active":"2024-04-21T08:37:24.708201+0000","last_became_peered":"2024-04-21T08:37:24.708201+0000","last_unstale":"2024-04-21T08:37:24.719766+0000","last_undegraded":"2024-04-21T08:37:24.719766+0000","last_fullsized":"2024-04-21T08:37:24.719766+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:10:51.158061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.308405+0000","last_change":"2024-04-21T08:36:20.909076+0000","last_active":"2024-04-21T08:37:53.308405+0000","last_peered":"2024-04-21T08:37:53.308405+0000","last_clean":"2024-04-21T08:37:53.308405+0000","last_became_active":"2024-04-21T08:36:20.908739+0000","last_became_peered":"2024-04-21T08:36:20.908739+0000","last_unstale":"2024-04-21T08:37:53.308405+0000","last_undegraded":"2024-04-21T08:37:53.308405+0000","last_fullsized":"2024-04-21T08:37:53.308405+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:03:56.870477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.293591+0000","last_change":"2024-04-21T08:36:24.926871+0000","last_active":"2024-04-21T08:37:24.293591+0000","last_peered":"2024-04-21T08:37:24.293591+0000","last_clean":"2024-04-21T08:37:24.293591+0000","last_became_active":"2024-04-21T08:36:24.926438+0000","last_became_peered":"2024-04-21T08:36:24.926438+0000","last_unstale":"2024-04-21T08:37:24.293591+0000","last_undegraded":"2024-04-21T08:37:24.293591+0000","last_fullsized":"2024-04-21T08:37:24.293591+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:24:48.984906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"89'2","reported_seq":17,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.308569+0000","last_change":"2024-04-21T08:37:24.308038+0000","last_active":"2024-04-21T08:37:24.308569+0000","last_peered":"2024-04-21T08:37:24.308569+0000","last_clean":"2024-04-21T08:37:24.308569+0000","last_became_active":"2024-04-21T08:37:24.301034+0000","last_became_peered":"2024-04-21T08:37:24.301034+0000","last_unstale":"2024-04-21T08:37:24.308569+0000","last_undegraded":"2024-04-21T08:37:24.308569+0000","last_fullsized":"2024-04-21T08:37:24.308569+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:37:06.598400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.284965+0000","last_change":"2024-04-21T08:37:24.299260+0000","last_active":"2024-04-21T08:37:53.284965+0000","last_peered":"2024-04-21T08:37:53.284965+0000","last_clean":"2024-04-21T08:37:53.284965+0000","last_became_active":"2024-04-21T08:37:24.298831+0000","last_became_peered":"2024-04-21T08:37:24.298831+0000","last_unstale":"2024-04-21T08:37:53.284965+0000","last_undegraded":"2024-04-21T08:37:53.284965+0000","last_fullsized":"2024-04-21T08:37:53.284965+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:58:29.368027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.290736+0000","last_change":"2024-04-21T08:36:24.929420+0000","last_active":"2024-04-21T08:37:24.290736+0000","last_peered":"2024-04-21T08:37:24.290736+0000","last_clean":"2024-04-21T08:37:24.290736+0000","last_became_active":"2024-04-21T08:36:24.928981+0000","last_became_peered":"2024-04-21T08:36:24.928981+0000","last_unstale":"2024-04-21T08:37:24.290736+0000","last_undegraded":"2024-04-21T08:37:24.290736+0000","last_fullsized":"2024-04-21T08:37:24.290736+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:38:00.040456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.289571+0000","last_change":"2024-04-21T08:36:20.279910+0000","last_active":"2024-04-21T08:37:24.289571+0000","last_peered":"2024-04-21T08:37:24.289571+0000","last_clean":"2024-04-21T08:37:24.289571+0000","last_became_active":"2024-04-21T08:36:20.278904+0000","last_became_peered":"2024-04-21T08:36:20.278904+0000","last_unstale":"2024-04-21T08:37:24.289571+0000","last_undegraded":"2024-04-21T08:37:24.289571+0000","last_fullsized":"2024-04-21T08:37:24.289571+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:40:19.209213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":28,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.305424+0000","last_change":"2024-04-21T08:36:20.895816+0000","last_active":"2024-04-21T08:37:53.305424+0000","last_peered":"2024-04-21T08:37:53.305424+0000","last_clean":"2024-04-21T08:37:53.305424+0000","last_became_active":"2024-04-21T08:36:20.895487+0000","last_became_peered":"2024-04-21T08:36:20.895487+0000","last_unstale":"2024-04-21T08:37:53.305424+0000","last_undegraded":"2024-04-21T08:37:53.305424+0000","last_fullsized":"2024-04-21T08:37:53.305424+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:53:52.153683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.291156+0000","last_change":"2024-04-21T08:36:24.917637+0000","last_active":"2024-04-21T08:37:24.291156+0000","last_peered":"2024-04-21T08:37:24.291156+0000","last_clean":"2024-04-21T08:37:24.291156+0000","last_became_active":"2024-04-21T08:36:24.917192+0000","last_became_peered":"2024-04-21T08:36:24.917192+0000","last_unstale":"2024-04-21T08:37:24.291156+0000","last_undegraded":"2024-04-21T08:37:24.291156+0000","last_fullsized":"2024-04-21T08:37:24.291156+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:02:59.843903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":103,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:57.340400+0000","last_change":"2024-04-21T08:36:20.286424+0000","last_active":"2024-04-21T08:37:57.340400+0000","last_peered":"2024-04-21T08:37:57.340400+0000","last_clean":"2024-04-21T08:37:57.340400+0000","last_became_active":"2024-04-21T08:36:20.285767+0000","last_became_peered":"2024-04-21T08:36:20.285767+0000","last_unstale":"2024-04-21T08:37:57.340400+0000","last_undegraded":"2024-04-21T08:37:57.340400+0000","last_fullsized":"2024-04-21T08:37:57.340400+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:53:33.202734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":72,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.301527+0000","last_change":"2024-04-21T08:37:24.297650+0000","last_active":"2024-04-21T08:37:53.301527+0000","last_peered":"2024-04-21T08:37:53.301527+0000","last_clean":"2024-04-21T08:37:53.301527+0000","last_became_active":"2024-04-21T08:37:24.297250+0000","last_became_peered":"2024-04-21T08:37:24.297250+0000","last_unstale":"2024-04-21T08:37:53.301527+0000","last_undegraded":"2024-04-21T08:37:53.301527+0000","last_fullsized":"2024-04-21T08:37:53.301527+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:21:21.419556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.291071+0000","last_change":"2024-04-21T08:36:24.919669+0000","last_active":"2024-04-21T08:37:24.291071+0000","last_peered":"2024-04-21T08:37:24.291071+0000","last_clean":"2024-04-21T08:37:24.291071+0000","last_became_active":"2024-04-21T08:36:24.919281+0000","last_became_peered":"2024-04-21T08:36:24.919281+0000","last_unstale":"2024-04-21T08:37:24.291071+0000","last_undegraded":"2024-04-21T08:37:24.291071+0000","last_fullsized":"2024-04-21T08:37:24.291071+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:44:27.480367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":38,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.292747+0000","last_change":"2024-04-21T08:37:24.297826+0000","last_active":"2024-04-21T08:37:53.292747+0000","last_peered":"2024-04-21T08:37:53.292747+0000","last_clean":"2024-04-21T08:37:53.292747+0000","last_became_active":"2024-04-21T08:37:24.297397+0000","last_became_peered":"2024-04-21T08:37:24.297397+0000","last_unstale":"2024-04-21T08:37:53.292747+0000","last_undegraded":"2024-04-21T08:37:53.292747+0000","last_fullsized":"2024-04-21T08:37:53.292747+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:52:17.693725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"91'6","reported_seq":112,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:38:01.568016+0000","last_change":"2024-04-21T08:36:20.291033+0000","last_active":"2024-04-21T08:38:01.568016+0000","last_peered":"2024-04-21T08:38:01.568016+0000","last_clean":"2024-04-21T08:38:01.568016+0000","last_became_active":"2024-04-21T08:36:20.290697+0000","last_became_peered":"2024-04-21T08:36:20.290697+0000","last_unstale":"2024-04-21T08:38:01.568016+0000","last_undegraded":"2024-04-21T08:38:01.568016+0000","last_fullsized":"2024-04-21T08:38:01.568016+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:22:54.593595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":79,"num_read_kb":79,"num_write":6,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.310936+0000","last_change":"2024-04-21T08:37:24.310936+0000","last_active":"2024-04-21T08:37:24.310936+0000","last_peered":"2024-04-21T08:37:24.310936+0000","last_clean":"2024-04-21T08:37:24.310936+0000","last_became_active":"2024-04-21T08:37:24.297349+0000","last_became_peered":"2024-04-21T08:37:24.297349+0000","last_unstale":"2024-04-21T08:37:24.310936+0000","last_undegraded":"2024-04-21T08:37:24.310936+0000","last_fullsized":"2024-04-21T08:37:24.310936+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:31:39.884609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.310460+0000","last_change":"2024-04-21T08:36:20.904197+0000","last_active":"2024-04-21T08:37:53.310460+0000","last_peered":"2024-04-21T08:37:53.310460+0000","last_clean":"2024-04-21T08:37:53.310460+0000","last_became_active":"2024-04-21T08:36:20.903099+0000","last_became_peered":"2024-04-21T08:36:20.903099+0000","last_unstale":"2024-04-21T08:37:53.310460+0000","last_undegraded":"2024-04-21T08:37:53.310460+0000","last_fullsized":"2024-04-21T08:37:53.310460+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:16:08.914335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.291788+0000","last_change":"2024-04-21T08:36:20.280608+0000","last_active":"2024-04-21T08:37:24.291788+0000","last_peered":"2024-04-21T08:37:24.291788+0000","last_clean":"2024-04-21T08:37:24.291788+0000","last_became_active":"2024-04-21T08:36:20.280172+0000","last_became_peered":"2024-04-21T08:36:20.280172+0000","last_unstale":"2024-04-21T08:37:24.291788+0000","last_undegraded":"2024-04-21T08:37:24.291788+0000","last_fullsized":"2024-04-21T08:37:24.291788+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:51:12.284463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":360,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.292472+0000","last_change":"2024-04-21T08:35:08.423545+0000","last_active":"2024-04-21T08:37:24.292472+0000","last_peered":"2024-04-21T08:37:24.292472+0000","last_clean":"2024-04-21T08:37:24.292472+0000","last_became_active":"2024-04-21T08:35:08.423027+0000","last_became_peered":"2024-04-21T08:35:08.423027+0000","last_unstale":"2024-04-21T08:37:24.292472+0000","last_undegraded":"2024-04-21T08:37:24.292472+0000","last_fullsized":"2024-04-21T08:37:24.292472+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:31:45.359253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:31:45.359253+0000","last_clean_scrub_stamp":"2024-04-21T08:31:45.359253+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:53:24.310824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.293980+0000","last_change":"2024-04-21T08:36:24.922422+0000","last_active":"2024-04-21T08:37:24.293980+0000","last_peered":"2024-04-21T08:37:24.293980+0000","last_clean":"2024-04-21T08:37:24.293980+0000","last_became_active":"2024-04-21T08:36:24.922006+0000","last_became_peered":"2024-04-21T08:36:24.922006+0000","last_unstale":"2024-04-21T08:37:24.293980+0000","last_undegraded":"2024-04-21T08:37:24.293980+0000","last_fullsized":"2024-04-21T08:37:24.293980+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:46:22.840802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.297239+0000","last_change":"2024-04-21T08:36:20.911222+0000","last_active":"2024-04-21T08:37:53.297239+0000","last_peered":"2024-04-21T08:37:53.297239+0000","last_clean":"2024-04-21T08:37:53.297239+0000","last_became_active":"2024-04-21T08:36:20.910933+0000","last_became_peered":"2024-04-21T08:36:20.910933+0000","last_unstale":"2024-04-21T08:37:53.297239+0000","last_undegraded":"2024-04-21T08:37:53.297239+0000","last_fullsized":"2024-04-21T08:37:53.297239+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:41:25.621266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.292533+0000","last_change":"2024-04-21T08:36:20.288409+0000","last_active":"2024-04-21T08:37:24.292533+0000","last_peered":"2024-04-21T08:37:24.292533+0000","last_clean":"2024-04-21T08:37:24.292533+0000","last_became_active":"2024-04-21T08:36:20.287916+0000","last_became_peered":"2024-04-21T08:36:20.287916+0000","last_unstale":"2024-04-21T08:37:24.292533+0000","last_undegraded":"2024-04-21T08:37:24.292533+0000","last_fullsized":"2024-04-21T08:37:24.292533+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:04:31.068352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.292720+0000","last_change":"2024-04-21T08:36:24.918166+0000","last_active":"2024-04-21T08:37:24.292720+0000","last_peered":"2024-04-21T08:37:24.292720+0000","last_clean":"2024-04-21T08:37:24.292720+0000","last_became_active":"2024-04-21T08:36:24.917745+0000","last_became_peered":"2024-04-21T08:36:24.917745+0000","last_unstale":"2024-04-21T08:37:24.292720+0000","last_undegraded":"2024-04-21T08:37:24.292720+0000","last_fullsized":"2024-04-21T08:37:24.292720+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:55:41.169681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.291577+0000","last_change":"2024-04-21T08:36:20.291079+0000","last_active":"2024-04-21T08:37:24.291577+0000","last_peered":"2024-04-21T08:37:24.291577+0000","last_clean":"2024-04-21T08:37:24.291577+0000","last_became_active":"2024-04-21T08:36:20.290720+0000","last_became_peered":"2024-04-21T08:36:20.290720+0000","last_unstale":"2024-04-21T08:37:24.291577+0000","last_undegraded":"2024-04-21T08:37:24.291577+0000","last_fullsized":"2024-04-21T08:37:24.291577+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:41.269029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.293500+0000","last_change":"2024-04-21T08:36:20.907416+0000","last_active":"2024-04-21T08:37:53.293500+0000","last_peered":"2024-04-21T08:37:53.293500+0000","last_clean":"2024-04-21T08:37:53.293500+0000","last_became_active":"2024-04-21T08:36:20.907005+0000","last_became_peered":"2024-04-21T08:36:20.907005+0000","last_unstale":"2024-04-21T08:37:53.293500+0000","last_undegraded":"2024-04-21T08:37:53.293500+0000","last_fullsized":"2024-04-21T08:37:53.293500+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:58:17.979947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.292450+0000","last_change":"2024-04-21T08:36:24.921475+0000","last_active":"2024-04-21T08:37:24.292450+0000","last_peered":"2024-04-21T08:37:24.292450+0000","last_clean":"2024-04-21T08:37:24.292450+0000","last_became_active":"2024-04-21T08:36:24.921091+0000","last_became_peered":"2024-04-21T08:36:24.921091+0000","last_unstale":"2024-04-21T08:37:24.292450+0000","last_undegraded":"2024-04-21T08:37:24.292450+0000","last_fullsized":"2024-04-21T08:37:24.292450+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:28:14.224501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"91'13","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.289769+0000","last_change":"2024-04-21T08:36:20.288916+0000","last_active":"2024-04-21T08:37:24.289769+0000","last_peered":"2024-04-21T08:37:24.289769+0000","last_clean":"2024-04-21T08:37:24.289769+0000","last_became_active":"2024-04-21T08:36:20.288575+0000","last_became_peered":"2024-04-21T08:36:20.288575+0000","last_unstale":"2024-04-21T08:37:24.289769+0000","last_undegraded":"2024-04-21T08:37:24.289769+0000","last_fullsized":"2024-04-21T08:37:24.289769+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:52:05.795554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":22,"num_write_kb":3,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.303522+0000","last_change":"2024-04-21T08:36:20.912053+0000","last_active":"2024-04-21T08:37:53.303522+0000","last_peered":"2024-04-21T08:37:53.303522+0000","last_clean":"2024-04-21T08:37:53.303522+0000","last_became_active":"2024-04-21T08:36:20.911584+0000","last_became_peered":"2024-04-21T08:36:20.911584+0000","last_unstale":"2024-04-21T08:37:53.303522+0000","last_undegraded":"2024-04-21T08:37:53.303522+0000","last_fullsized":"2024-04-21T08:37:53.303522+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:48:36.369954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.287880+0000","last_change":"2024-04-21T08:36:24.919212+0000","last_active":"2024-04-21T08:37:24.287880+0000","last_peered":"2024-04-21T08:37:24.287880+0000","last_clean":"2024-04-21T08:37:24.287880+0000","last_became_active":"2024-04-21T08:36:24.918807+0000","last_became_peered":"2024-04-21T08:36:24.918807+0000","last_unstale":"2024-04-21T08:37:24.287880+0000","last_undegraded":"2024-04-21T08:37:24.287880+0000","last_fullsized":"2024-04-21T08:37:24.287880+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:25:50.816775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.294049+0000","last_change":"2024-04-21T08:36:20.282536+0000","last_active":"2024-04-21T08:37:24.294049+0000","last_peered":"2024-04-21T08:37:24.294049+0000","last_clean":"2024-04-21T08:37:24.294049+0000","last_became_active":"2024-04-21T08:36:20.282172+0000","last_became_peered":"2024-04-21T08:36:20.282172+0000","last_unstale":"2024-04-21T08:37:24.294049+0000","last_undegraded":"2024-04-21T08:37:24.294049+0000","last_fullsized":"2024-04-21T08:37:24.294049+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:15:38.992441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.306910+0000","last_change":"2024-04-21T08:36:20.908416+0000","last_active":"2024-04-21T08:37:53.306910+0000","last_peered":"2024-04-21T08:37:53.306910+0000","last_clean":"2024-04-21T08:37:53.306910+0000","last_became_active":"2024-04-21T08:36:20.908054+0000","last_became_peered":"2024-04-21T08:36:20.908054+0000","last_unstale":"2024-04-21T08:37:53.306910+0000","last_undegraded":"2024-04-21T08:37:53.306910+0000","last_fullsized":"2024-04-21T08:37:53.306910+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:23:16.562060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.292421+0000","last_change":"2024-04-21T08:36:24.920637+0000","last_active":"2024-04-21T08:37:24.292421+0000","last_peered":"2024-04-21T08:37:24.292421+0000","last_clean":"2024-04-21T08:37:24.292421+0000","last_became_active":"2024-04-21T08:36:24.919921+0000","last_became_peered":"2024-04-21T08:36:24.919921+0000","last_unstale":"2024-04-21T08:37:24.292421+0000","last_undegraded":"2024-04-21T08:37:24.292421+0000","last_fullsized":"2024-04-21T08:37:24.292421+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:20:38.230087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.707218+0000","last_change":"2024-04-21T08:36:20.279332+0000","last_active":"2024-04-21T08:37:24.707218+0000","last_peered":"2024-04-21T08:37:24.707218+0000","last_clean":"2024-04-21T08:37:24.707218+0000","last_became_active":"2024-04-21T08:36:20.278970+0000","last_became_peered":"2024-04-21T08:36:20.278970+0000","last_unstale":"2024-04-21T08:37:24.707218+0000","last_undegraded":"2024-04-21T08:37:24.707218+0000","last_fullsized":"2024-04-21T08:37:24.707218+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:36:16.041610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.259529+0000","last_change":"2024-04-21T08:36:20.919621+0000","last_active":"2024-04-21T08:37:53.259529+0000","last_peered":"2024-04-21T08:37:53.259529+0000","last_clean":"2024-04-21T08:37:53.259529+0000","last_became_active":"2024-04-21T08:36:20.919246+0000","last_became_peered":"2024-04-21T08:36:20.919246+0000","last_unstale":"2024-04-21T08:37:53.259529+0000","last_undegraded":"2024-04-21T08:37:53.259529+0000","last_fullsized":"2024-04-21T08:37:53.259529+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:31:59.001501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.289794+0000","last_change":"2024-04-21T08:36:24.920459+0000","last_active":"2024-04-21T08:37:24.289794+0000","last_peered":"2024-04-21T08:37:24.289794+0000","last_clean":"2024-04-21T08:37:24.289794+0000","last_became_active":"2024-04-21T08:36:24.919691+0000","last_became_peered":"2024-04-21T08:36:24.919691+0000","last_unstale":"2024-04-21T08:37:24.289794+0000","last_undegraded":"2024-04-21T08:37:24.289794+0000","last_fullsized":"2024-04-21T08:37:24.289794+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:38:26.655452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.293040+0000","last_change":"2024-04-21T08:36:20.284286+0000","last_active":"2024-04-21T08:37:24.293040+0000","last_peered":"2024-04-21T08:37:24.293040+0000","last_clean":"2024-04-21T08:37:24.293040+0000","last_became_active":"2024-04-21T08:36:20.283903+0000","last_became_peered":"2024-04-21T08:36:20.283903+0000","last_unstale":"2024-04-21T08:37:24.293040+0000","last_undegraded":"2024-04-21T08:37:24.293040+0000","last_fullsized":"2024-04-21T08:37:24.293040+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:51:52.409988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.299685+0000","last_change":"2024-04-21T08:36:20.911892+0000","last_active":"2024-04-21T08:37:53.299685+0000","last_peered":"2024-04-21T08:37:53.299685+0000","last_clean":"2024-04-21T08:37:53.299685+0000","last_became_active":"2024-04-21T08:36:20.911513+0000","last_became_peered":"2024-04-21T08:36:20.911513+0000","last_unstale":"2024-04-21T08:37:53.299685+0000","last_undegraded":"2024-04-21T08:37:53.299685+0000","last_fullsized":"2024-04-21T08:37:53.299685+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:51:13.733534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":19,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.291554+0000","last_change":"2024-04-21T08:36:24.920347+0000","last_active":"2024-04-21T08:37:24.291554+0000","last_peered":"2024-04-21T08:37:24.291554+0000","last_clean":"2024-04-21T08:37:24.291554+0000","last_became_active":"2024-04-21T08:36:24.919831+0000","last_became_peered":"2024-04-21T08:36:24.919831+0000","last_unstale":"2024-04-21T08:37:24.291554+0000","last_undegraded":"2024-04-21T08:37:24.291554+0000","last_fullsized":"2024-04-21T08:37:24.291554+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:14:27.020476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.288651+0000","last_change":"2024-04-21T08:36:20.286786+0000","last_active":"2024-04-21T08:37:24.288651+0000","last_peered":"2024-04-21T08:37:24.288651+0000","last_clean":"2024-04-21T08:37:24.288651+0000","last_became_active":"2024-04-21T08:36:20.286429+0000","last_became_peered":"2024-04-21T08:36:20.286429+0000","last_unstale":"2024-04-21T08:37:24.288651+0000","last_undegraded":"2024-04-21T08:37:24.288651+0000","last_fullsized":"2024-04-21T08:37:24.288651+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:40:57.093370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.305474+0000","last_change":"2024-04-21T08:36:20.904767+0000","last_active":"2024-04-21T08:37:53.305474+0000","last_peered":"2024-04-21T08:37:53.305474+0000","last_clean":"2024-04-21T08:37:53.305474+0000","last_became_active":"2024-04-21T08:36:20.904406+0000","last_became_peered":"2024-04-21T08:36:20.904406+0000","last_unstale":"2024-04-21T08:37:53.305474+0000","last_undegraded":"2024-04-21T08:37:53.305474+0000","last_fullsized":"2024-04-21T08:37:53.305474+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:11:41.644549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.294150+0000","last_change":"2024-04-21T08:36:24.918031+0000","last_active":"2024-04-21T08:37:24.294150+0000","last_peered":"2024-04-21T08:37:24.294150+0000","last_clean":"2024-04-21T08:37:24.294150+0000","last_became_active":"2024-04-21T08:36:24.917618+0000","last_became_peered":"2024-04-21T08:36:24.917618+0000","last_unstale":"2024-04-21T08:37:24.294150+0000","last_undegraded":"2024-04-21T08:37:24.294150+0000","last_fullsized":"2024-04-21T08:37:24.294150+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:50:35.797911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.290778+0000","last_change":"2024-04-21T08:36:20.286212+0000","last_active":"2024-04-21T08:37:24.290778+0000","last_peered":"2024-04-21T08:37:24.290778+0000","last_clean":"2024-04-21T08:37:24.290778+0000","last_became_active":"2024-04-21T08:36:20.285878+0000","last_became_peered":"2024-04-21T08:36:20.285878+0000","last_unstale":"2024-04-21T08:37:24.290778+0000","last_undegraded":"2024-04-21T08:37:24.290778+0000","last_fullsized":"2024-04-21T08:37:24.290778+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:19.375455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.306985+0000","last_change":"2024-04-21T08:36:20.908335+0000","last_active":"2024-04-21T08:37:53.306985+0000","last_peered":"2024-04-21T08:37:53.306985+0000","last_clean":"2024-04-21T08:37:53.306985+0000","last_became_active":"2024-04-21T08:36:20.907945+0000","last_became_peered":"2024-04-21T08:36:20.907945+0000","last_unstale":"2024-04-21T08:37:53.306985+0000","last_undegraded":"2024-04-21T08:37:53.306985+0000","last_fullsized":"2024-04-21T08:37:53.306985+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:47:17.841511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.706560+0000","last_change":"2024-04-21T08:36:24.915553+0000","last_active":"2024-04-21T08:37:24.706560+0000","last_peered":"2024-04-21T08:37:24.706560+0000","last_clean":"2024-04-21T08:37:24.706560+0000","last_became_active":"2024-04-21T08:36:24.915028+0000","last_became_peered":"2024-04-21T08:36:24.915028+0000","last_unstale":"2024-04-21T08:37:24.706560+0000","last_undegraded":"2024-04-21T08:37:24.706560+0000","last_fullsized":"2024-04-21T08:37:24.706560+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:17:21.192946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'7","reported_seq":110,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:58.575620+0000","last_change":"2024-04-21T08:36:20.279649+0000","last_active":"2024-04-21T08:37:58.575620+0000","last_peered":"2024-04-21T08:37:58.575620+0000","last_clean":"2024-04-21T08:37:58.575620+0000","last_became_active":"2024-04-21T08:36:20.279111+0000","last_became_peered":"2024-04-21T08:36:20.279111+0000","last_unstale":"2024-04-21T08:37:58.575620+0000","last_undegraded":"2024-04-21T08:37:58.575620+0000","last_fullsized":"2024-04-21T08:37:58.575620+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:19:57.380495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":76,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.307325+0000","last_change":"2024-04-21T08:36:20.911489+0000","last_active":"2024-04-21T08:37:53.307325+0000","last_peered":"2024-04-21T08:37:53.307325+0000","last_clean":"2024-04-21T08:37:53.307325+0000","last_became_active":"2024-04-21T08:36:20.911168+0000","last_became_peered":"2024-04-21T08:36:20.911168+0000","last_unstale":"2024-04-21T08:37:53.307325+0000","last_undegraded":"2024-04-21T08:37:53.307325+0000","last_fullsized":"2024-04-21T08:37:53.307325+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:40:41.585841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.293211+0000","last_change":"2024-04-21T08:36:24.921175+0000","last_active":"2024-04-21T08:37:24.293211+0000","last_peered":"2024-04-21T08:37:24.293211+0000","last_clean":"2024-04-21T08:37:24.293211+0000","last_became_active":"2024-04-21T08:36:24.920802+0000","last_became_peered":"2024-04-21T08:36:24.920802+0000","last_unstale":"2024-04-21T08:37:24.293211+0000","last_undegraded":"2024-04-21T08:37:24.293211+0000","last_fullsized":"2024-04-21T08:37:24.293211+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:43:12.140016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"89'3","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.291285+0000","last_change":"2024-04-21T08:36:20.290202+0000","last_active":"2024-04-21T08:37:24.291285+0000","last_peered":"2024-04-21T08:37:24.291285+0000","last_clean":"2024-04-21T08:37:24.291285+0000","last_became_active":"2024-04-21T08:36:20.289868+0000","last_became_peered":"2024-04-21T08:36:20.289868+0000","last_unstale":"2024-04-21T08:37:24.291285+0000","last_undegraded":"2024-04-21T08:37:24.291285+0000","last_fullsized":"2024-04-21T08:37:24.291285+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:07:45.525980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.301690+0000","last_change":"2024-04-21T08:36:20.899590+0000","last_active":"2024-04-21T08:37:53.301690+0000","last_peered":"2024-04-21T08:37:53.301690+0000","last_clean":"2024-04-21T08:37:53.301690+0000","last_became_active":"2024-04-21T08:36:20.899254+0000","last_became_peered":"2024-04-21T08:36:20.899254+0000","last_unstale":"2024-04-21T08:37:53.301690+0000","last_undegraded":"2024-04-21T08:37:53.301690+0000","last_fullsized":"2024-04-21T08:37:53.301690+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:34:25.409694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":18,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.290801+0000","last_change":"2024-04-21T08:36:24.918310+0000","last_active":"2024-04-21T08:37:24.290801+0000","last_peered":"2024-04-21T08:37:24.290801+0000","last_clean":"2024-04-21T08:37:24.290801+0000","last_became_active":"2024-04-21T08:36:24.917899+0000","last_became_peered":"2024-04-21T08:36:24.917899+0000","last_unstale":"2024-04-21T08:37:24.290801+0000","last_undegraded":"2024-04-21T08:37:24.290801+0000","last_fullsized":"2024-04-21T08:37:24.290801+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:09:13.006884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":44,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:24.304544+0000","last_change":"2024-04-21T08:37:24.304544+0000","last_active":"2024-04-21T08:37:24.304544+0000","last_peered":"2024-04-21T08:37:24.304544+0000","last_clean":"2024-04-21T08:37:24.304544+0000","last_became_active":"2024-04-21T08:37:24.295808+0000","last_became_peered":"2024-04-21T08:37:24.295808+0000","last_unstale":"2024-04-21T08:37:24.304544+0000","last_undegraded":"2024-04-21T08:37:24.304544+0000","last_fullsized":"2024-04-21T08:37:24.304544+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:07:15.797133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.296557+0000","last_change":"2024-04-21T08:36:20.912368+0000","last_active":"2024-04-21T08:37:53.296557+0000","last_peered":"2024-04-21T08:37:53.296557+0000","last_clean":"2024-04-21T08:37:53.296557+0000","last_became_active":"2024-04-21T08:36:20.912009+0000","last_became_peered":"2024-04-21T08:36:20.912009+0000","last_unstale":"2024-04-21T08:37:53.296557+0000","last_undegraded":"2024-04-21T08:37:53.296557+0000","last_fullsized":"2024-04-21T08:37:53.296557+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:35:40.700314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":34,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.292751+0000","last_change":"2024-04-21T08:36:20.913472+0000","last_active":"2024-04-21T08:37:53.292751+0000","last_peered":"2024-04-21T08:37:53.292751+0000","last_clean":"2024-04-21T08:37:53.292751+0000","last_became_active":"2024-04-21T08:36:20.913118+0000","last_became_peered":"2024-04-21T08:36:20.913118+0000","last_unstale":"2024-04-21T08:37:53.292751+0000","last_undegraded":"2024-04-21T08:37:53.292751+0000","last_fullsized":"2024-04-21T08:37:53.292751+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:31:53.957044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":37,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.298054+0000","last_change":"2024-04-21T08:36:20.913677+0000","last_active":"2024-04-21T08:37:53.298054+0000","last_peered":"2024-04-21T08:37:53.298054+0000","last_clean":"2024-04-21T08:37:53.298054+0000","last_became_active":"2024-04-21T08:36:20.913097+0000","last_became_peered":"2024-04-21T08:36:20.913097+0000","last_unstale":"2024-04-21T08:37:53.298054+0000","last_undegraded":"2024-04-21T08:37:53.298054+0000","last_fullsized":"2024-04-21T08:37:53.298054+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:08:40.388370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.301817+0000","last_change":"2024-04-21T08:36:20.906624+0000","last_active":"2024-04-21T08:37:53.301817+0000","last_peered":"2024-04-21T08:37:53.301817+0000","last_clean":"2024-04-21T08:37:53.301817+0000","last_became_active":"2024-04-21T08:36:20.906247+0000","last_became_peered":"2024-04-21T08:36:20.906247+0000","last_unstale":"2024-04-21T08:37:53.301817+0000","last_undegraded":"2024-04-21T08:37:53.301817+0000","last_fullsized":"2024-04-21T08:37:53.301817+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:44:11.855183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":32,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.306768+0000","last_change":"2024-04-21T08:36:20.906576+0000","last_active":"2024-04-21T08:37:53.306768+0000","last_peered":"2024-04-21T08:37:53.306768+0000","last_clean":"2024-04-21T08:37:53.306768+0000","last_became_active":"2024-04-21T08:36:20.906231+0000","last_became_peered":"2024-04-21T08:36:20.906231+0000","last_unstale":"2024-04-21T08:37:53.306768+0000","last_undegraded":"2024-04-21T08:37:53.306768+0000","last_fullsized":"2024-04-21T08:37:53.306768+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:42:48.156614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":29,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.304203+0000","last_change":"2024-04-21T08:36:20.902551+0000","last_active":"2024-04-21T08:37:53.304203+0000","last_peered":"2024-04-21T08:37:53.304203+0000","last_clean":"2024-04-21T08:37:53.304203+0000","last_became_active":"2024-04-21T08:36:20.902208+0000","last_became_peered":"2024-04-21T08:36:20.902208+0000","last_unstale":"2024-04-21T08:37:53.304203+0000","last_undegraded":"2024-04-21T08:37:53.304203+0000","last_fullsized":"2024-04-21T08:37:53.304203+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:36:30.016256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":31,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.290914+0000","last_change":"2024-04-21T08:36:20.918588+0000","last_active":"2024-04-21T08:37:53.290914+0000","last_peered":"2024-04-21T08:37:53.290914+0000","last_clean":"2024-04-21T08:37:53.290914+0000","last_became_active":"2024-04-21T08:36:20.918263+0000","last_became_peered":"2024-04-21T08:36:20.918263+0000","last_unstale":"2024-04-21T08:37:53.290914+0000","last_undegraded":"2024-04-21T08:37:53.290914+0000","last_fullsized":"2024-04-21T08:37:53.290914+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:51:30.340336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":40,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.300582+0000","last_change":"2024-04-21T08:36:20.911063+0000","last_active":"2024-04-21T08:37:53.300582+0000","last_peered":"2024-04-21T08:37:53.300582+0000","last_clean":"2024-04-21T08:37:53.300582+0000","last_became_active":"2024-04-21T08:36:20.910647+0000","last_became_peered":"2024-04-21T08:36:20.910647+0000","last_unstale":"2024-04-21T08:37:53.300582+0000","last_undegraded":"2024-04-21T08:37:53.300582+0000","last_fullsized":"2024-04-21T08:37:53.300582+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:10:31.079782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":27,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.304377+0000","last_change":"2024-04-21T08:36:20.914079+0000","last_active":"2024-04-21T08:37:53.304377+0000","last_peered":"2024-04-21T08:37:53.304377+0000","last_clean":"2024-04-21T08:37:53.304377+0000","last_became_active":"2024-04-21T08:36:20.912851+0000","last_became_peered":"2024-04-21T08:36:20.912851+0000","last_unstale":"2024-04-21T08:37:53.304377+0000","last_undegraded":"2024-04-21T08:37:53.304377+0000","last_fullsized":"2024-04-21T08:37:53.304377+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:53:35.811281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":30,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.300652+0000","last_change":"2024-04-21T08:36:20.900551+0000","last_active":"2024-04-21T08:37:53.300652+0000","last_peered":"2024-04-21T08:37:53.300652+0000","last_clean":"2024-04-21T08:37:53.300652+0000","last_became_active":"2024-04-21T08:36:20.900173+0000","last_became_peered":"2024-04-21T08:36:20.900173+0000","last_unstale":"2024-04-21T08:37:53.300652+0000","last_undegraded":"2024-04-21T08:37:53.300652+0000","last_fullsized":"2024-04-21T08:37:53.300652+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:21.389498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.304808+0000","last_change":"2024-04-21T08:36:20.912953+0000","last_active":"2024-04-21T08:37:53.304808+0000","last_peered":"2024-04-21T08:37:53.304808+0000","last_clean":"2024-04-21T08:37:53.304808+0000","last_became_active":"2024-04-21T08:36:20.912223+0000","last_became_peered":"2024-04-21T08:36:20.912223+0000","last_unstale":"2024-04-21T08:37:53.304808+0000","last_undegraded":"2024-04-21T08:37:53.304808+0000","last_fullsized":"2024-04-21T08:37:53.304808+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:43:14.277179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":35,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.309674+0000","last_change":"2024-04-21T08:36:20.906164+0000","last_active":"2024-04-21T08:37:53.309674+0000","last_peered":"2024-04-21T08:37:53.309674+0000","last_clean":"2024-04-21T08:37:53.309674+0000","last_became_active":"2024-04-21T08:36:20.905749+0000","last_became_peered":"2024-04-21T08:36:20.905749+0000","last_unstale":"2024-04-21T08:37:53.309674+0000","last_undegraded":"2024-04-21T08:37:53.309674+0000","last_fullsized":"2024-04-21T08:37:53.309674+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:02:10.569087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":36,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.304742+0000","last_change":"2024-04-21T08:36:20.900314+0000","last_active":"2024-04-21T08:37:53.304742+0000","last_peered":"2024-04-21T08:37:53.304742+0000","last_clean":"2024-04-21T08:37:53.304742+0000","last_became_active":"2024-04-21T08:36:20.900009+0000","last_became_peered":"2024-04-21T08:36:20.900009+0000","last_unstale":"2024-04-21T08:37:53.304742+0000","last_undegraded":"2024-04-21T08:37:53.304742+0000","last_fullsized":"2024-04-21T08:37:53.304742+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:19:59.997302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":33,"reported_epoch":91,"state":"active+clean","last_fresh":"2024-04-21T08:37:53.300407+0000","last_change":"2024-04-21T08:36:20.920725+0000","last_active":"2024-04-21T08:37:53.300407+0000","last_peered":"2024-04-21T08:37:53.300407+0000","last_clean":"2024-04-21T08:37:53.300407+0000","last_became_active":"2024-04-21T08:36:20.920288+0000","last_became_peered":"2024-04-21T08:36:20.920288+0000","last_unstale":"2024-04-21T08:37:53.300407+0000","last_undegraded":"2024-04-21T08:37:53.300407+0000","last_fullsized":"2024-04-21T08:37:53.300407+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:36:18.736388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":4157440,"data_stored":4132576,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":8},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":71623,"num_objects":98,"num_object_clones":0,"num_object_copies":294,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":98,"num_whiteouts":0,"num_read":775,"num_read_kb":464,"num_write":253,"num_write_kb":150,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":806,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":577536,"data_stored":214869,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":221,"ondisk_log_size":221,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383714,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36356,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708796,"statfs":{"total":95995035648,"available":95957807104,"internally_reserved":0,"allocated":1044480,"data_stored":502659,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645350,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28708,"kb_used_data":1692,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716444,"statfs":{"total":95995035648,"available":95965638656,"internally_reserved":0,"allocated":1732608,"data_stored":1156482,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808876,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36368,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708784,"statfs":{"total":95995035648,"available":95957794816,"internally_reserved":0,"allocated":1056768,"data_stored":501363,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070513,"num_pgs":30,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28040,"kb_used_data":1024,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717112,"statfs":{"total":95995035648,"available":95966322688,"internally_reserved":0,"allocated":1048576,"data_stored":501719,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332150,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28620,"kb_used_data":1604,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716532,"statfs":{"total":95995035648,"available":95965728768,"internally_reserved":0,"allocated":1642496,"data_stored":1098239,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495676,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28024,"kb_used_data":1008,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717128,"statfs":{"total":95995035648,"available":95966339072,"internally_reserved":0,"allocated":1032192,"data_stored":496867,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574932,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1084,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1110016,"data_stored":561614,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738457,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28048,"kb_used_data":1032,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717104,"statfs":{"total":95995035648,"available":95966314496,"internally_reserved":0,"allocated":1056768,"data_stored":498855,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411407,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36424,"kb_used_data":1088,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708728,"statfs":{"total":95995035648,"available":95957737472,"internally_reserved":0,"allocated":1114112,"data_stored":562674,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182473,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28036,"kb_used_data":1020,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717116,"statfs":{"total":95995035648,"available":95966326784,"internally_reserved":0,"allocated":1044480,"data_stored":496575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920837,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36352,"kb_used_data":1016,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708800,"statfs":{"total":95995035648,"available":95957811200,"internally_reserved":0,"allocated":1040384,"data_stored":501473,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757312,"num_pgs":45,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":32796,"kb_used_data":1620,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93712356,"statfs":{"total":95995035648,"available":95961452544,"internally_reserved":0,"allocated":1658880,"data_stored":1093961,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":94208,"data_stored":65469,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":66529,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":430,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":1232,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":45056,"data_stored":3352,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":3534,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":28672,"data_stored":1478,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":36864,"data_stored":1122,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":118784,"data_stored":65873,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T08:38:04.083 INFO:tasks.cephadm.ceph_manager.ceph:clean! 2024-04-21T08:38:04.083 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2024-04-21T08:38:04.084 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy 2024-04-21T08:38:04.084 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph health --format=json 2024-04-21T08:38:04.331 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:38:04.703 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:04 smithi086 ceph-mon[27939]: pgmap v341: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 10 op/s 2024-04-21T08:38:04.703 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:04 smithi086 ceph-mon[27939]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:04.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:04 smithi088 ceph-mon[31259]: pgmap v341: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 10 op/s 2024-04-21T08:38:04.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:04 smithi088 ceph-mon[31259]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:04.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:04 smithi179 ceph-mon[31829]: pgmap v341: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.8 KiB/s rd, 767 B/s wr, 10 op/s 2024-04-21T08:38:04.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:04 smithi179 ceph-mon[31829]: from='client.15144 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:06.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.009+0000 7ffbcf6ce700 1 -- 172.21.15.86:0/2928414029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbc8108a50 msgr2=0x7ffbc8108e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:06.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.009+0000 7ffbcf6ce700 1 --2- 172.21.15.86:0/2928414029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbc8108a50 0x7ffbc8108e30 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7ffbc40099c0 tx=0x7ffbc4009cd0 comp rx=0 tx=0).stop 2024-04-21T08:38:06.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.009+0000 7ffbcf6ce700 1 -- 172.21.15.86:0/2928414029 shutdown_connections 2024-04-21T08:38:06.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.009+0000 7ffbcf6ce700 1 --2- 172.21.15.86:0/2928414029 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffbc806bb70 0x7ffbc810f900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:06.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.009+0000 7ffbcf6ce700 1 --2- 172.21.15.86:0/2928414029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbc8108a50 0x7ffbc8108e30 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:06.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.009+0000 7ffbcf6ce700 1 --2- 172.21.15.86:0/2928414029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffbc806b1b0 0x7ffbc806b630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:06.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.009+0000 7ffbcf6ce700 1 -- 172.21.15.86:0/2928414029 >> 172.21.15.86:0/2928414029 conn(0x7ffbc8077990 msgr2=0x7ffbc8077da0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:06.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.010+0000 7ffbcf6ce700 1 -- 172.21.15.86:0/2928414029 shutdown_connections 2024-04-21T08:38:06.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.010+0000 7ffbcf6ce700 1 -- 172.21.15.86:0/2928414029 wait complete. 2024-04-21T08:38:06.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.010+0000 7ffbcf6ce700 1 Processor -- start 2024-04-21T08:38:06.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.010+0000 7ffbcf6ce700 1 -- start start 2024-04-21T08:38:06.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.010+0000 7ffbcf6ce700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffbc806b1b0 0x7ffbc8199180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:06.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.010+0000 7ffbcf6ce700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbc806bb70 0x7ffbc81996c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:06.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.010+0000 7ffbcf6ce700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffbc8108a50 0x7ffbc819da50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:06.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.010+0000 7ffbcf6ce700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc8112c90 con 0x7ffbc806bb70 2024-04-21T08:38:06.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.010+0000 7ffbcf6ce700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc8112b10 con 0x7ffbc8108a50 2024-04-21T08:38:06.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.010+0000 7ffbcf6ce700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ffbc8112e10 con 0x7ffbc806b1b0 2024-04-21T08:38:06.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.010+0000 7ffbccc69700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbc806bb70 0x7ffbc81996c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:06.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.010+0000 7ffbccc69700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbc806bb70 0x7ffbc81996c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51106/0 (socket says 172.21.15.86:51106) 2024-04-21T08:38:06.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.010+0000 7ffbccc69700 1 -- 172.21.15.86:0/745063354 learned_addr learned my addr 172.21.15.86:0/745063354 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:06.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.011+0000 7ffbcdc6b700 1 --2- 172.21.15.86:0/745063354 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffbc8108a50 0x7ffbc819da50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:06.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.011+0000 7ffbcd46a700 1 --2- 172.21.15.86:0/745063354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffbc806b1b0 0x7ffbc8199180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:06.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.011+0000 7ffbccc69700 1 -- 172.21.15.86:0/745063354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffbc806b1b0 msgr2=0x7ffbc8199180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:06.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.011+0000 7ffbccc69700 1 --2- 172.21.15.86:0/745063354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffbc806b1b0 0x7ffbc8199180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:06.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.011+0000 7ffbccc69700 1 -- 172.21.15.86:0/745063354 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffbc8108a50 msgr2=0x7ffbc819da50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:06.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.011+0000 7ffbccc69700 1 --2- 172.21.15.86:0/745063354 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffbc8108a50 0x7ffbc819da50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:06.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.011+0000 7ffbccc69700 1 -- 172.21.15.86:0/745063354 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffbc819e190 con 0x7ffbc806bb70 2024-04-21T08:38:06.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.011+0000 7ffbcdc6b700 1 --2- 172.21.15.86:0/745063354 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffbc8108a50 0x7ffbc819da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:06.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.011+0000 7ffbccc69700 1 --2- 172.21.15.86:0/745063354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbc806bb70 0x7ffbc81996c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ffbb800b530 tx=0x7ffbb800b8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:06.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.011+0000 7ffbbe7fc700 1 -- 172.21.15.86:0/745063354 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb8015020 con 0x7ffbc806bb70 2024-04-21T08:38:06.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.011+0000 7ffbcf6ce700 1 -- 172.21.15.86:0/745063354 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffbc819e480 con 0x7ffbc806bb70 2024-04-21T08:38:06.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.011+0000 7ffbbe7fc700 1 -- 172.21.15.86:0/745063354 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffbb8011ce0 con 0x7ffbc806bb70 2024-04-21T08:38:06.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.011+0000 7ffbbe7fc700 1 -- 172.21.15.86:0/745063354 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffbb8010d20 con 0x7ffbc806bb70 2024-04-21T08:38:06.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.011+0000 7ffbcf6ce700 1 -- 172.21.15.86:0/745063354 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffbc81a5d00 con 0x7ffbc806bb70 2024-04-21T08:38:06.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.012+0000 7ffbcf6ce700 1 -- 172.21.15.86:0/745063354 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffbc8069730 con 0x7ffbc806bb70 2024-04-21T08:38:06.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.014+0000 7ffbbe7fc700 1 -- 172.21.15.86:0/745063354 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7ffbb8018020 con 0x7ffbc806bb70 2024-04-21T08:38:06.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.014+0000 7ffbbe7fc700 1 --2- 172.21.15.86:0/745063354 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ffbb40a3c40 0x7ffbb40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:06.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.015+0000 7ffbcd46a700 1 --2- 172.21.15.86:0/745063354 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ffbb40a3c40 0x7ffbb40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:06.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.015+0000 7ffbbe7fc700 1 -- 172.21.15.86:0/745063354 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7ffbb80d04e0 con 0x7ffbc806bb70 2024-04-21T08:38:06.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.015+0000 7ffbcd46a700 1 --2- 172.21.15.86:0/745063354 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ffbb40a3c40 0x7ffbb40a6100 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4003800 tx=0x7ffbc4009f10 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:06.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.018+0000 7ffbbe7fc700 1 -- 172.21.15.86:0/745063354 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffbb801f020 con 0x7ffbc806bb70 2024-04-21T08:38:06.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.261+0000 7ffbcf6ce700 1 -- 172.21.15.86:0/745063354 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "health", "format": "json"} v 0) -- 0x7ffbc819a4e0 con 0x7ffbc806bb70 2024-04-21T08:38:06.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.261+0000 7ffbbe7fc700 1 -- 172.21.15.86:0/745063354 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "health", "format": "json"}]=0 v0) ==== 72+0+46 (secure 0 0 0) 0x7ffbb809cb60 con 0x7ffbc806bb70 2024-04-21T08:38:06.262 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:38:06.262 INFO:teuthology.orchestra.run.smithi086.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2024-04-21T08:38:06.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.263+0000 7ffbcf6ce700 1 -- 172.21.15.86:0/745063354 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ffbb40a3c40 msgr2=0x7ffbb40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:06.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.263+0000 7ffbcf6ce700 1 --2- 172.21.15.86:0/745063354 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ffbb40a3c40 0x7ffbb40a6100 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7ffbc4003800 tx=0x7ffbc4009f10 comp rx=0 tx=0).stop 2024-04-21T08:38:06.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.263+0000 7ffbcf6ce700 1 -- 172.21.15.86:0/745063354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbc806bb70 msgr2=0x7ffbc81996c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:06.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.263+0000 7ffbcf6ce700 1 --2- 172.21.15.86:0/745063354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbc806bb70 0x7ffbc81996c0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7ffbb800b530 tx=0x7ffbb800b8f0 comp rx=0 tx=0).stop 2024-04-21T08:38:06.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.264+0000 7ffbcf6ce700 1 -- 172.21.15.86:0/745063354 shutdown_connections 2024-04-21T08:38:06.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.264+0000 7ffbcf6ce700 1 --2- 172.21.15.86:0/745063354 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7ffbb40a3c40 0x7ffbb40a6100 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:06.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.264+0000 7ffbcf6ce700 1 --2- 172.21.15.86:0/745063354 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffbc8108a50 0x7ffbc819da50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:06.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.264+0000 7ffbcf6ce700 1 --2- 172.21.15.86:0/745063354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffbc806b1b0 0x7ffbc8199180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:06.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.264+0000 7ffbcf6ce700 1 --2- 172.21.15.86:0/745063354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffbc806bb70 0x7ffbc81996c0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:06.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.264+0000 7ffbcf6ce700 1 -- 172.21.15.86:0/745063354 >> 172.21.15.86:0/745063354 conn(0x7ffbc8077990 msgr2=0x7ffbc8100650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:06.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.264+0000 7ffbcf6ce700 1 -- 172.21.15.86:0/745063354 shutdown_connections 2024-04-21T08:38:06.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:06.264+0000 7ffbcf6ce700 1 -- 172.21.15.86:0/745063354 wait complete. 2024-04-21T08:38:06.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:06 smithi086 ceph-mon[27939]: pgmap v342: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 767 B/s wr, 12 op/s 2024-04-21T08:38:06.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:06 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/745063354' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-21T08:38:06.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:06 smithi088 ceph-mon[31259]: pgmap v342: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 767 B/s wr, 12 op/s 2024-04-21T08:38:06.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:06 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/745063354' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-21T08:38:06.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:06 smithi179 ceph-mon[31829]: pgmap v342: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.1 KiB/s rd, 767 B/s wr, 12 op/s 2024-04-21T08:38:06.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:06 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/745063354' entity='client.admin' cmd=[{"prefix": "health", "format": "json"}]: dispatch 2024-04-21T08:38:07.090 INFO:tasks.cephadm.ceph_manager.ceph:wait_until_healthy done 2024-04-21T08:38:07.090 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:38:07.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.194+0000 7f7c18fc7640 1 -- 172.21.15.86:0/34140904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c141029f0 msgr2=0x7f7c14104de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:07.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.194+0000 7f7c18fc7640 1 --2- 172.21.15.86:0/34140904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c141029f0 0x7f7c14104de0 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f7c04009900 tx=0x7f7c0402bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:07.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.194+0000 7f7c18fc7640 1 -- 172.21.15.86:0/34140904 shutdown_connections 2024-04-21T08:38:07.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.194+0000 7f7c18fc7640 1 --2- 172.21.15.86:0/34140904 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c14107c50 0x7f7c1410a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.194+0000 7f7c18fc7640 1 --2- 172.21.15.86:0/34140904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c14105320 0x7f7c14107710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.194+0000 7f7c18fc7640 1 --2- 172.21.15.86:0/34140904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c141029f0 0x7f7c14104de0 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.194+0000 7f7c18fc7640 1 -- 172.21.15.86:0/34140904 >> 172.21.15.86:0/34140904 conn(0x7f7c140fc3e0 msgr2=0x7f7c140fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:07.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.194+0000 7f7c18fc7640 1 -- 172.21.15.86:0/34140904 shutdown_connections 2024-04-21T08:38:07.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.194+0000 7f7c18fc7640 1 -- 172.21.15.86:0/34140904 wait complete. 2024-04-21T08:38:07.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.195+0000 7f7c18fc7640 1 Processor -- start 2024-04-21T08:38:07.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.195+0000 7f7c18fc7640 1 -- start start 2024-04-21T08:38:07.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.195+0000 7f7c18fc7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c141029f0 0x7f7c14198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:07.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.195+0000 7f7c18fc7640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c14105320 0x7f7c14198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:07.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.195+0000 7f7c18fc7640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c14107c50 0x7f7c1419f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:07.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.195+0000 7f7c18fc7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1410daa0 con 0x7f7c141029f0 2024-04-21T08:38:07.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.195+0000 7f7c18fc7640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1410d920 con 0x7f7c14105320 2024-04-21T08:38:07.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.195+0000 7f7c18fc7640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7c1410dc20 con 0x7f7c14107c50 2024-04-21T08:38:07.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.195+0000 7f7c1259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c141029f0 0x7f7c14198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:07.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.195+0000 7f7c1259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c141029f0 0x7f7c14198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51116/0 (socket says 172.21.15.86:51116) 2024-04-21T08:38:07.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.195+0000 7f7c1259c640 1 -- 172.21.15.86:0/1954705450 learned_addr learned my addr 172.21.15.86:0/1954705450 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:07.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.195+0000 7f7c12d9d640 1 --2- 172.21.15.86:0/1954705450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c14107c50 0x7f7c1419f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:07.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.195+0000 7f7c11d9b640 1 --2- 172.21.15.86:0/1954705450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c14105320 0x7f7c14198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:07.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.195+0000 7f7c1259c640 1 -- 172.21.15.86:0/1954705450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c14107c50 msgr2=0x7f7c1419f780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:07.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.195+0000 7f7c1259c640 1 --2- 172.21.15.86:0/1954705450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c14107c50 0x7f7c1419f780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.195+0000 7f7c1259c640 1 -- 172.21.15.86:0/1954705450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c14105320 msgr2=0x7f7c14198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:07.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.195+0000 7f7c1259c640 1 --2- 172.21.15.86:0/1954705450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c14105320 0x7f7c14198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.195+0000 7f7c1259c640 1 -- 172.21.15.86:0/1954705450 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c1419fdf0 con 0x7f7c141029f0 2024-04-21T08:38:07.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.195+0000 7f7c1259c640 1 --2- 172.21.15.86:0/1954705450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c141029f0 0x7f7c14198220 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7c04006010 tx=0x7f7c040042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:07.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.196+0000 7f7c037fe640 1 -- 172.21.15.86:0/1954705450 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c04043020 con 0x7f7c141029f0 2024-04-21T08:38:07.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.196+0000 7f7c037fe640 1 -- 172.21.15.86:0/1954705450 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c04004aa0 con 0x7f7c141029f0 2024-04-21T08:38:07.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.196+0000 7f7c18fc7640 1 -- 172.21.15.86:0/1954705450 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c141a0080 con 0x7f7c141029f0 2024-04-21T08:38:07.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.196+0000 7f7c037fe640 1 -- 172.21.15.86:0/1954705450 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c04003c70 con 0x7f7c141029f0 2024-04-21T08:38:07.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.196+0000 7f7c18fc7640 1 -- 172.21.15.86:0/1954705450 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c141a05d0 con 0x7f7c141029f0 2024-04-21T08:38:07.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.197+0000 7f7c18fc7640 1 -- 172.21.15.86:0/1954705450 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7c140fded0 con 0x7f7c141029f0 2024-04-21T08:38:07.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.198+0000 7f7c037fe640 1 -- 172.21.15.86:0/1954705450 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7c04046020 con 0x7f7c141029f0 2024-04-21T08:38:07.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.198+0000 7f7c037fe640 1 --2- 172.21.15.86:0/1954705450 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f7bfc0a3bb0 0x7f7bfc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:07.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.198+0000 7f7c11d9b640 1 --2- 172.21.15.86:0/1954705450 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f7bfc0a3bb0 0x7f7bfc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:07.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.198+0000 7f7c037fe640 1 -- 172.21.15.86:0/1954705450 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f7c040f2be0 con 0x7f7c141029f0 2024-04-21T08:38:07.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.199+0000 7f7c11d9b640 1 --2- 172.21.15.86:0/1954705450 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f7bfc0a3bb0 0x7f7bfc0a6070 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7c08005fd0 tx=0x7f7c08005850 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:07.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.203+0000 7f7c037fe640 1 -- 172.21.15.86:0/1954705450 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7c0404d020 con 0x7f7c141029f0 2024-04-21T08:38:07.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.386+0000 7f7c18fc7640 1 -- 172.21.15.86:0/1954705450 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7c14062b50 con 0x7f7c141029f0 2024-04-21T08:38:07.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.388+0000 7f7c037fe640 1 -- 172.21.15.86:0/1954705450 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24595 (secure 0 0 0) 0x7f7c040bf260 con 0x7f7c141029f0 2024-04-21T08:38:07.389 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:38:07.389 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":91,"fsid":"f8cec7cc-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:22.687329+0000","modified":"2024-04-21T08:37:24.279025+0000","last_up_change":"2024-04-21T08:35:30.048315+0000","last_in_change":"2024-04-21T08:35:10.099496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:45.279109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:18.300782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:19.574636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:22.905632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89b7601f-0620-4db2-90f1-b45cea173738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6803","nonce":2571516897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6805","nonce":2571516897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6809","nonce":2571516897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6807","nonce":2571516897}]},"public_addr":"172.21.15.86:6803/2571516897","cluster_addr":"172.21.15.86:6805/2571516897","heartbeat_back_addr":"172.21.15.86:6809/2571516897","heartbeat_front_addr":"172.21.15.86:6807/2571516897","state":["exists","up"]},{"osd":1,"uuid":"4891031c-f98c-42bc-bb3b-a420855ecc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6801","nonce":74067182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6803","nonce":74067182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6807","nonce":74067182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6805","nonce":74067182}]},"public_addr":"172.21.15.88:6801/74067182","cluster_addr":"172.21.15.88:6803/74067182","heartbeat_back_addr":"172.21.15.88:6807/74067182","heartbeat_front_addr":"172.21.15.88:6805/74067182","state":["exists","up"]},{"osd":2,"uuid":"60d12aa4-9312-4fa2-8d0b-2b3f42caad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6801","nonce":4137659051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6803","nonce":4137659051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6807","nonce":4137659051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6805","nonce":4137659051}]},"public_addr":"172.21.15.179:6801/4137659051","cluster_addr":"172.21.15.179:6803/4137659051","heartbeat_back_addr":"172.21.15.179:6807/4137659051","heartbeat_front_addr":"172.21.15.179:6805/4137659051","state":["exists","up"]},{"osd":3,"uuid":"404c3158-8cb0-4c02-b48a-f9ee5ff7b784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6811","nonce":2309603148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6813","nonce":2309603148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6817","nonce":2309603148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6815","nonce":2309603148}]},"public_addr":"172.21.15.86:6811/2309603148","cluster_addr":"172.21.15.86:6813/2309603148","heartbeat_back_addr":"172.21.15.86:6817/2309603148","heartbeat_front_addr":"172.21.15.86:6815/2309603148","state":["exists","up"]},{"osd":4,"uuid":"458ff202-d781-4ec7-ac93-0b49f835b503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6809","nonce":1818714217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6811","nonce":1818714217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6815","nonce":1818714217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6813","nonce":1818714217}]},"public_addr":"172.21.15.88:6809/1818714217","cluster_addr":"172.21.15.88:6811/1818714217","heartbeat_back_addr":"172.21.15.88:6815/1818714217","heartbeat_front_addr":"172.21.15.88:6813/1818714217","state":["exists","up"]},{"osd":5,"uuid":"8123cc90-bd8b-4204-bf31-46b34e9f589d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6809","nonce":863840990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6811","nonce":863840990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6815","nonce":863840990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6813","nonce":863840990}]},"public_addr":"172.21.15.179:6809/863840990","cluster_addr":"172.21.15.179:6811/863840990","heartbeat_back_addr":"172.21.15.179:6815/863840990","heartbeat_front_addr":"172.21.15.179:6813/863840990","state":["exists","up"]},{"osd":6,"uuid":"aac9d7c7-659a-4110-903a-a871c086066f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6819","nonce":1224231118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6821","nonce":1224231118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6825","nonce":1224231118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6823","nonce":1224231118}]},"public_addr":"172.21.15.86:6819/1224231118","cluster_addr":"172.21.15.86:6821/1224231118","heartbeat_back_addr":"172.21.15.86:6825/1224231118","heartbeat_front_addr":"172.21.15.86:6823/1224231118","state":["exists","up"]},{"osd":7,"uuid":"df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6817","nonce":136629878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6819","nonce":136629878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6823","nonce":136629878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6821","nonce":136629878}]},"public_addr":"172.21.15.88:6817/136629878","cluster_addr":"172.21.15.88:6819/136629878","heartbeat_back_addr":"172.21.15.88:6823/136629878","heartbeat_front_addr":"172.21.15.88:6821/136629878","state":["exists","up"]},{"osd":8,"uuid":"63e59f4b-0426-46db-92f7-c55dae61b5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6817","nonce":1770418957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6819","nonce":1770418957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6823","nonce":1770418957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6821","nonce":1770418957}]},"public_addr":"172.21.15.179:6817/1770418957","cluster_addr":"172.21.15.179:6819/1770418957","heartbeat_back_addr":"172.21.15.179:6823/1770418957","heartbeat_front_addr":"172.21.15.179:6821/1770418957","state":["exists","up"]},{"osd":9,"uuid":"5841bb36-ed6b-45d8-b857-eeab139c44e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6827","nonce":1108500274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6829","nonce":1108500274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6833","nonce":1108500274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6831","nonce":1108500274}]},"public_addr":"172.21.15.86:6827/1108500274","cluster_addr":"172.21.15.86:6829/1108500274","heartbeat_back_addr":"172.21.15.86:6833/1108500274","heartbeat_front_addr":"172.21.15.86:6831/1108500274","state":["exists","up"]},{"osd":10,"uuid":"f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6825","nonce":1294275353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6827","nonce":1294275353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6831","nonce":1294275353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6829","nonce":1294275353}]},"public_addr":"172.21.15.88:6825/1294275353","cluster_addr":"172.21.15.88:6827/1294275353","heartbeat_back_addr":"172.21.15.88:6831/1294275353","heartbeat_front_addr":"172.21.15.88:6829/1294275353","state":["exists","up"]},{"osd":11,"uuid":"e0d4cc76-e7a6-4a6b-909c-95488ccd5a36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6825","nonce":4007070233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6827","nonce":4007070233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6831","nonce":4007070233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6829","nonce":4007070233}]},"public_addr":"172.21.15.179:6825/4007070233","cluster_addr":"172.21.15.179:6827/4007070233","heartbeat_back_addr":"172.21.15.179:6831/4007070233","heartbeat_front_addr":"172.21.15.179:6829/4007070233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:55.210605+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:18.723090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:42.547979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:06.421622+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:30.858315+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:55.455313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.623369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:46.185111+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.103627+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:37.297545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:02.637431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:28.199292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6800/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/1412393225":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6800/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/1687852404":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2020707404":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2568130933":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/3952607648":"2024-04-22T08:28:57.384403+0000","172.21.15.86:6801/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/2756903426":"2024-04-22T08:29:23.190001+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:38:07.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.390+0000 7f7c18fc7640 1 -- 172.21.15.86:0/1954705450 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f7bfc0a3bb0 msgr2=0x7f7bfc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:07.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.390+0000 7f7c18fc7640 1 --2- 172.21.15.86:0/1954705450 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f7bfc0a3bb0 0x7f7bfc0a6070 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f7c08005fd0 tx=0x7f7c08005850 comp rx=0 tx=0).stop 2024-04-21T08:38:07.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.390+0000 7f7c18fc7640 1 -- 172.21.15.86:0/1954705450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c141029f0 msgr2=0x7f7c14198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:07.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.390+0000 7f7c18fc7640 1 --2- 172.21.15.86:0/1954705450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c141029f0 0x7f7c14198220 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f7c04006010 tx=0x7f7c040042f0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.391+0000 7f7c18fc7640 1 -- 172.21.15.86:0/1954705450 shutdown_connections 2024-04-21T08:38:07.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.391+0000 7f7c18fc7640 1 --2- 172.21.15.86:0/1954705450 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f7bfc0a3bb0 0x7f7bfc0a6070 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.391+0000 7f7c18fc7640 1 --2- 172.21.15.86:0/1954705450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c14107c50 0x7f7c1419f780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.391+0000 7f7c18fc7640 1 --2- 172.21.15.86:0/1954705450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c14105320 0x7f7c14198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.391+0000 7f7c18fc7640 1 --2- 172.21.15.86:0/1954705450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c141029f0 0x7f7c14198220 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.391+0000 7f7c18fc7640 1 -- 172.21.15.86:0/1954705450 >> 172.21.15.86:0/1954705450 conn(0x7f7c140fc3e0 msgr2=0x7f7c14103430 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:07.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.391+0000 7f7c18fc7640 1 -- 172.21.15.86:0/1954705450 shutdown_connections 2024-04-21T08:38:07.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.391+0000 7f7c18fc7640 1 -- 172.21.15.86:0/1954705450 wait complete. 2024-04-21T08:38:07.402 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:31:45.279109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:36:18.300782+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:36:19.574636+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:36:22.905632+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:38:07.403 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:38:07.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.504+0000 7f378ddc7640 1 -- 172.21.15.86:0/1368810565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3788103bf0 msgr2=0x7f378810a480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:07.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.504+0000 7f378ddc7640 1 --2- 172.21.15.86:0/1368810565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3788103bf0 0x7f378810a480 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f3780009cd0 tx=0x7f378002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.504+0000 7f378ddc7640 1 -- 172.21.15.86:0/1368810565 shutdown_connections 2024-04-21T08:38:07.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.504+0000 7f378ddc7640 1 --2- 172.21.15.86:0/1368810565 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3788103bf0 0x7f378810a480 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.504+0000 7f378ddc7640 1 --2- 172.21.15.86:0/1368810565 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3788103230 0x7f37881036b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.504+0000 7f378ddc7640 1 --2- 172.21.15.86:0/1368810565 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3788100260 0x7f3788100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.504+0000 7f378ddc7640 1 -- 172.21.15.86:0/1368810565 >> 172.21.15.86:0/1368810565 conn(0x7f37880fbb90 msgr2=0x7f37880fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:07.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.504+0000 7f378ddc7640 1 -- 172.21.15.86:0/1368810565 shutdown_connections 2024-04-21T08:38:07.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.504+0000 7f378ddc7640 1 -- 172.21.15.86:0/1368810565 wait complete. 2024-04-21T08:38:07.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.504+0000 7f378ddc7640 1 Processor -- start 2024-04-21T08:38:07.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.504+0000 7f378ddc7640 1 -- start start 2024-04-21T08:38:07.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.504+0000 7f378ddc7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3788100260 0x7f37881982e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:07.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.504+0000 7f378ddc7640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3788103230 0x7f3788198820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:07.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.504+0000 7f378ddc7640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3788103bf0 0x7f378819f890 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:07.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.505+0000 7f378ddc7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f378810dbe0 con 0x7f3788100260 2024-04-21T08:38:07.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.505+0000 7f378ddc7640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f378810da60 con 0x7f3788103230 2024-04-21T08:38:07.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.505+0000 7f378ddc7640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f378810dd60 con 0x7f3788103bf0 2024-04-21T08:38:07.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.505+0000 7f37877fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3788100260 0x7f37881982e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:07.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.505+0000 7f37877fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3788100260 0x7f37881982e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51138/0 (socket says 172.21.15.86:51138) 2024-04-21T08:38:07.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.505+0000 7f37877fe640 1 -- 172.21.15.86:0/933591141 learned_addr learned my addr 172.21.15.86:0/933591141 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:07.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.505+0000 7f37877fe640 1 -- 172.21.15.86:0/933591141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3788103bf0 msgr2=0x7f378819f890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:07.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.505+0000 7f37877fe640 1 --2- 172.21.15.86:0/933591141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3788103bf0 0x7f378819f890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.505+0000 7f37877fe640 1 -- 172.21.15.86:0/933591141 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3788103230 msgr2=0x7f3788198820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:07.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.505+0000 7f37877fe640 1 --2- 172.21.15.86:0/933591141 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3788103230 0x7f3788198820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.505+0000 7f37877fe640 1 -- 172.21.15.86:0/933591141 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f378819ff00 con 0x7f3788100260 2024-04-21T08:38:07.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.505+0000 7f37877fe640 1 --2- 172.21.15.86:0/933591141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3788100260 0x7f37881982e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f377800b450 tx=0x7f377800b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:07.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.505+0000 7f3784ff9640 1 -- 172.21.15.86:0/933591141 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3778015020 con 0x7f3788100260 2024-04-21T08:38:07.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.505+0000 7f3784ff9640 1 -- 172.21.15.86:0/933591141 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3778011d40 con 0x7f3788100260 2024-04-21T08:38:07.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.505+0000 7f3784ff9640 1 -- 172.21.15.86:0/933591141 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f377800f340 con 0x7f3788100260 2024-04-21T08:38:07.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.505+0000 7f378ddc7640 1 -- 172.21.15.86:0/933591141 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37881a01f0 con 0x7f3788100260 2024-04-21T08:38:07.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.505+0000 7f378ddc7640 1 -- 172.21.15.86:0/933591141 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37881a0770 con 0x7f3788100260 2024-04-21T08:38:07.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.506+0000 7f378ddc7640 1 -- 172.21.15.86:0/933591141 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3788109430 con 0x7f3788100260 2024-04-21T08:38:07.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.507+0000 7f3784ff9640 1 -- 172.21.15.86:0/933591141 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f3778018020 con 0x7f3788100260 2024-04-21T08:38:07.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.508+0000 7f3784ff9640 1 --2- 172.21.15.86:0/933591141 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f37700a3bb0 0x7f37700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:07.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.508+0000 7f3786ffd640 1 --2- 172.21.15.86:0/933591141 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f37700a3bb0 0x7f37700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:07.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.508+0000 7f3784ff9640 1 -- 172.21.15.86:0/933591141 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f37780d02a0 con 0x7f3788100260 2024-04-21T08:38:07.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.508+0000 7f3786ffd640 1 --2- 172.21.15.86:0/933591141 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f37700a3bb0 0x7f37700a6070 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f37881997f0 tx=0x7f377c006c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:07.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.512+0000 7f3784ff9640 1 -- 172.21.15.86:0/933591141 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f377801f020 con 0x7f3788100260 2024-04-21T08:38:07.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.692+0000 7f378ddc7640 1 -- 172.21.15.86:0/933591141 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f3788062c50 con 0x7f3788100260 2024-04-21T08:38:07.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.692+0000 7f3784ff9640 1 -- 172.21.15.86:0/933591141 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7f377800d040 con 0x7f3788100260 2024-04-21T08:38:07.692 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-21T08:38:07.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.694+0000 7f378ddc7640 1 -- 172.21.15.86:0/933591141 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f37700a3bb0 msgr2=0x7f37700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:07.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.694+0000 7f378ddc7640 1 --2- 172.21.15.86:0/933591141 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f37700a3bb0 0x7f37700a6070 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f37881997f0 tx=0x7f377c006c90 comp rx=0 tx=0).stop 2024-04-21T08:38:07.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.694+0000 7f378ddc7640 1 -- 172.21.15.86:0/933591141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3788100260 msgr2=0x7f37881982e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:07.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.694+0000 7f378ddc7640 1 --2- 172.21.15.86:0/933591141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3788100260 0x7f37881982e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7f377800b450 tx=0x7f377800b910 comp rx=0 tx=0).stop 2024-04-21T08:38:07.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.694+0000 7f378ddc7640 1 -- 172.21.15.86:0/933591141 shutdown_connections 2024-04-21T08:38:07.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.694+0000 7f378ddc7640 1 --2- 172.21.15.86:0/933591141 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f37700a3bb0 0x7f37700a6070 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.694+0000 7f378ddc7640 1 --2- 172.21.15.86:0/933591141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3788103bf0 0x7f378819f890 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.694+0000 7f378ddc7640 1 --2- 172.21.15.86:0/933591141 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3788103230 0x7f3788198820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.694+0000 7f378ddc7640 1 --2- 172.21.15.86:0/933591141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3788100260 0x7f37881982e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.694+0000 7f378ddc7640 1 -- 172.21.15.86:0/933591141 >> 172.21.15.86:0/933591141 conn(0x7f37880fbb90 msgr2=0x7f3788106600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:07.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.694+0000 7f378ddc7640 1 -- 172.21.15.86:0/933591141 shutdown_connections 2024-04-21T08:38:07.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.694+0000 7f378ddc7640 1 -- 172.21.15.86:0/933591141 wait complete. 2024-04-21T08:38:07.706 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:38:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.806+0000 7f6af87b6640 1 -- 172.21.15.86:0/1697836721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6af0101fa0 msgr2=0x7f6af01023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.806+0000 7f6af87b6640 1 --2- 172.21.15.86:0/1697836721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6af0101fa0 0x7f6af01023c0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7f6aec009900 tx=0x7f6aec02bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.807+0000 7f6af87b6640 1 -- 172.21.15.86:0/1697836721 shutdown_connections 2024-04-21T08:38:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.807+0000 7f6af87b6640 1 --2- 172.21.15.86:0/1697836721 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6af0103b50 0x7f6af010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.807+0000 7f6af87b6640 1 --2- 172.21.15.86:0/1697836721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6af0103190 0x7f6af0103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.807+0000 7f6af87b6640 1 --2- 172.21.15.86:0/1697836721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6af0101fa0 0x7f6af01023c0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.807+0000 7f6af87b6640 1 -- 172.21.15.86:0/1697836721 >> 172.21.15.86:0/1697836721 conn(0x7f6af00fd530 msgr2=0x7f6af00ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.807+0000 7f6af87b6640 1 -- 172.21.15.86:0/1697836721 shutdown_connections 2024-04-21T08:38:07.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.807+0000 7f6af87b6640 1 -- 172.21.15.86:0/1697836721 wait complete. 2024-04-21T08:38:07.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.807+0000 7f6af87b6640 1 Processor -- start 2024-04-21T08:38:07.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.807+0000 7f6af87b6640 1 -- start start 2024-04-21T08:38:07.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.807+0000 7f6af87b6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6af0101fa0 0x7f6af0198200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:07.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.807+0000 7f6af87b6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6af0103190 0x7f6af0198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:07.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.807+0000 7f6af87b6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6af0103b50 0x7f6af019f7b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:07.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.807+0000 7f6af87b6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6af010dae0 con 0x7f6af0103190 2024-04-21T08:38:07.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.807+0000 7f6af87b6640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6af010d960 con 0x7f6af0101fa0 2024-04-21T08:38:07.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.807+0000 7f6af87b6640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6af010dc60 con 0x7f6af0103b50 2024-04-21T08:38:07.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.807+0000 7f6af5d51640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6af0103190 0x7f6af0198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:07.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.807+0000 7f6af5d51640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6af0103190 0x7f6af0198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51154/0 (socket says 172.21.15.86:51154) 2024-04-21T08:38:07.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.807+0000 7f6af5d51640 1 -- 172.21.15.86:0/58603177 learned_addr learned my addr 172.21.15.86:0/58603177 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:07.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.808+0000 7f6af6552640 1 --2- 172.21.15.86:0/58603177 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6af0101fa0 0x7f6af0198200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:07.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.808+0000 7f6af5d51640 1 -- 172.21.15.86:0/58603177 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6af0103b50 msgr2=0x7f6af019f7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:07.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.808+0000 7f6af5d51640 1 --2- 172.21.15.86:0/58603177 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6af0103b50 0x7f6af019f7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.808+0000 7f6af5d51640 1 -- 172.21.15.86:0/58603177 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6af0101fa0 msgr2=0x7f6af0198200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:07.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.808+0000 7f6af5d51640 1 --2- 172.21.15.86:0/58603177 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6af0101fa0 0x7f6af0198200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.808+0000 7f6af5d51640 1 -- 172.21.15.86:0/58603177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6af019fe20 con 0x7f6af0103190 2024-04-21T08:38:07.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.808+0000 7f6af5d51640 1 --2- 172.21.15.86:0/58603177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6af0103190 0x7f6af0198740 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6ae000c970 tx=0x7f6ae000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:07.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.808+0000 7f6ae77fe640 1 -- 172.21.15.86:0/58603177 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae0007bb0 con 0x7f6af0103190 2024-04-21T08:38:07.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.808+0000 7f6ae77fe640 1 -- 172.21.15.86:0/58603177 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6ae0007d50 con 0x7f6af0103190 2024-04-21T08:38:07.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.808+0000 7f6af87b6640 1 -- 172.21.15.86:0/58603177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6af01a0110 con 0x7f6af0103190 2024-04-21T08:38:07.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.808+0000 7f6ae77fe640 1 -- 172.21.15.86:0/58603177 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6ae000f450 con 0x7f6af0103190 2024-04-21T08:38:07.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.808+0000 7f6af87b6640 1 -- 172.21.15.86:0/58603177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6af01a0610 con 0x7f6af0103190 2024-04-21T08:38:07.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.809+0000 7f6af87b6640 1 -- 172.21.15.86:0/58603177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6af01093e0 con 0x7f6af0103190 2024-04-21T08:38:07.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.810+0000 7f6ae77fe640 1 -- 172.21.15.86:0/58603177 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f6ae0004f20 con 0x7f6af0103190 2024-04-21T08:38:07.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.811+0000 7f6ae77fe640 1 --2- 172.21.15.86:0/58603177 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6adc0a3bb0 0x7f6adc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:07.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.811+0000 7f6af6552640 1 --2- 172.21.15.86:0/58603177 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6adc0a3bb0 0x7f6adc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:07.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.811+0000 7f6ae77fe640 1 -- 172.21.15.86:0/58603177 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f6ae00d04d0 con 0x7f6af0103190 2024-04-21T08:38:07.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.811+0000 7f6af6552640 1 --2- 172.21.15.86:0/58603177 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6adc0a3bb0 0x7f6adc0a6070 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6aec005ab0 tx=0x7f6aec005fd0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:07.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.815+0000 7f6ae77fe640 1 -- 172.21.15.86:0/58603177 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6ae0014030 con 0x7f6af0103190 2024-04-21T08:38:07.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.992+0000 7f6af87b6640 1 -- 172.21.15.86:0/58603177 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6af0062b80 con 0x7f6af0103190 2024-04-21T08:38:07.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.993+0000 7f6ae77fe640 1 -- 172.21.15.86:0/58603177 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f6ae009cb50 con 0x7f6af0103190 2024-04-21T08:38:07.993 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-21T08:38:07.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.995+0000 7f6af87b6640 1 -- 172.21.15.86:0/58603177 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6adc0a3bb0 msgr2=0x7f6adc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:07.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.995+0000 7f6af87b6640 1 --2- 172.21.15.86:0/58603177 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6adc0a3bb0 0x7f6adc0a6070 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6aec005ab0 tx=0x7f6aec005fd0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.995+0000 7f6af87b6640 1 -- 172.21.15.86:0/58603177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6af0103190 msgr2=0x7f6af0198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:07.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.995+0000 7f6af87b6640 1 --2- 172.21.15.86:0/58603177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6af0103190 0x7f6af0198740 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f6ae000c970 tx=0x7f6ae000ce30 comp rx=0 tx=0).stop 2024-04-21T08:38:07.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.995+0000 7f6af87b6640 1 -- 172.21.15.86:0/58603177 shutdown_connections 2024-04-21T08:38:07.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.995+0000 7f6af87b6640 1 --2- 172.21.15.86:0/58603177 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f6adc0a3bb0 0x7f6adc0a6070 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.995+0000 7f6af87b6640 1 --2- 172.21.15.86:0/58603177 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6af0103b50 0x7f6af019f7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.995+0000 7f6af87b6640 1 --2- 172.21.15.86:0/58603177 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6af0103190 0x7f6af0198740 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.995+0000 7f6af87b6640 1 --2- 172.21.15.86:0/58603177 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6af0101fa0 0x7f6af0198200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:07.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.995+0000 7f6af87b6640 1 -- 172.21.15.86:0/58603177 >> 172.21.15.86:0/58603177 conn(0x7f6af00fd530 msgr2=0x7f6af01065b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:07.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.995+0000 7f6af87b6640 1 -- 172.21.15.86:0/58603177 shutdown_connections 2024-04-21T08:38:07.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:07.995+0000 7f6af87b6640 1 -- 172.21.15.86:0/58603177 wait complete. 2024-04-21T08:38:08.006 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:38:08.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.107+0000 7f20f5fce640 1 -- 172.21.15.86:0/1454564855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20f0100e90 msgr2=0x7f20f01012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:08.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.107+0000 7f20f5fce640 1 --2- 172.21.15.86:0/1454564855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20f0100e90 0x7f20f01012b0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f20e0009900 tx=0x7f20e002bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:08.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.107+0000 7f20f5fce640 1 -- 172.21.15.86:0/1454564855 shutdown_connections 2024-04-21T08:38:08.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.107+0000 7f20f5fce640 1 --2- 172.21.15.86:0/1454564855 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20f01029a0 0x7f20f01092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:08.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.107+0000 7f20f5fce640 1 --2- 172.21.15.86:0/1454564855 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20f0101fe0 0x7f20f0102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:08.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.107+0000 7f20f5fce640 1 --2- 172.21.15.86:0/1454564855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20f0100e90 0x7f20f01012b0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:08.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.107+0000 7f20f5fce640 1 -- 172.21.15.86:0/1454564855 >> 172.21.15.86:0/1454564855 conn(0x7f20f00fc400 msgr2=0x7f20f00fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:08.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.107+0000 7f20f5fce640 1 -- 172.21.15.86:0/1454564855 shutdown_connections 2024-04-21T08:38:08.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.107+0000 7f20f5fce640 1 -- 172.21.15.86:0/1454564855 wait complete. 2024-04-21T08:38:08.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.107+0000 7f20f5fce640 1 Processor -- start 2024-04-21T08:38:08.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.107+0000 7f20f5fce640 1 -- start start 2024-04-21T08:38:08.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.107+0000 7f20f5fce640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20f0100e90 0x7f20f0104e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:08.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.107+0000 7f20f5fce640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20f0101fe0 0x7f20f0103450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:08.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.107+0000 7f20f5fce640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20f01029a0 0x7f20f0103990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:08.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.107+0000 7f20f5fce640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f20f010c9e0 con 0x7f20f0101fe0 2024-04-21T08:38:08.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.107+0000 7f20f5fce640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f20f010c860 con 0x7f20f0100e90 2024-04-21T08:38:08.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.107+0000 7f20f5fce640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f20f010cb60 con 0x7f20f01029a0 2024-04-21T08:38:08.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.108+0000 7f20e7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20f0101fe0 0x7f20f0103450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:08.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.108+0000 7f20e7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20f0101fe0 0x7f20f0103450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51174/0 (socket says 172.21.15.86:51174) 2024-04-21T08:38:08.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.108+0000 7f20e7fff640 1 -- 172.21.15.86:0/3016433152 learned_addr learned my addr 172.21.15.86:0/3016433152 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:08.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.108+0000 7f20ef7fe640 1 --2- 172.21.15.86:0/3016433152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20f0100e90 0x7f20f0104e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:08.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.108+0000 7f20effff640 1 --2- 172.21.15.86:0/3016433152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20f01029a0 0x7f20f0103990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:08.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.108+0000 7f20e7fff640 1 -- 172.21.15.86:0/3016433152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20f01029a0 msgr2=0x7f20f0103990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:08.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.108+0000 7f20e7fff640 1 --2- 172.21.15.86:0/3016433152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20f01029a0 0x7f20f0103990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:08.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.108+0000 7f20e7fff640 1 -- 172.21.15.86:0/3016433152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20f0100e90 msgr2=0x7f20f0104e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:08.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.108+0000 7f20e7fff640 1 --2- 172.21.15.86:0/3016433152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20f0100e90 0x7f20f0104e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:08.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.108+0000 7f20e7fff640 1 -- 172.21.15.86:0/3016433152 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20f0104220 con 0x7f20f0101fe0 2024-04-21T08:38:08.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.108+0000 7f20e7fff640 1 --2- 172.21.15.86:0/3016433152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20f0101fe0 0x7f20f0103450 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f20d800c970 tx=0x7f20d800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:08.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.108+0000 7f20ed7fa640 1 -- 172.21.15.86:0/3016433152 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d8004220 con 0x7f20f0101fe0 2024-04-21T08:38:08.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.108+0000 7f20ed7fa640 1 -- 172.21.15.86:0/3016433152 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20d80043c0 con 0x7f20f0101fe0 2024-04-21T08:38:08.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.108+0000 7f20ed7fa640 1 -- 172.21.15.86:0/3016433152 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20d8004a00 con 0x7f20f0101fe0 2024-04-21T08:38:08.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.108+0000 7f20f5fce640 1 -- 172.21.15.86:0/3016433152 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20f01a2390 con 0x7f20f0101fe0 2024-04-21T08:38:08.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.108+0000 7f20f5fce640 1 -- 172.21.15.86:0/3016433152 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20f01a2890 con 0x7f20f0101fe0 2024-04-21T08:38:08.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.109+0000 7f20f5fce640 1 -- 172.21.15.86:0/3016433152 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20f01a2bb0 con 0x7f20f0101fe0 2024-04-21T08:38:08.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.110+0000 7f20ed7fa640 1 -- 172.21.15.86:0/3016433152 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f20d80079b0 con 0x7f20f0101fe0 2024-04-21T08:38:08.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.111+0000 7f20ed7fa640 1 --2- 172.21.15.86:0/3016433152 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f20dc0a3bb0 0x7f20dc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:08.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.111+0000 7f20ef7fe640 1 --2- 172.21.15.86:0/3016433152 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f20dc0a3bb0 0x7f20dc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:08.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.111+0000 7f20ed7fa640 1 -- 172.21.15.86:0/3016433152 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f20d80d0520 con 0x7f20f0101fe0 2024-04-21T08:38:08.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.111+0000 7f20ef7fe640 1 --2- 172.21.15.86:0/3016433152 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f20dc0a3bb0 0x7f20dc0a6070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f20e0006010 tx=0x7f20e00095a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:08.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.115+0000 7f20ed7fa640 1 -- 172.21.15.86:0/3016433152 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f20d8014030 con 0x7f20f0101fe0 2024-04-21T08:38:08.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.292+0000 7f20f5fce640 1 -- 172.21.15.86:0/3016433152 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f20f01a2d80 con 0x7f20f0101fe0 2024-04-21T08:38:08.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.293+0000 7f20ed7fa640 1 -- 172.21.15.86:0/3016433152 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7f20d809cba0 con 0x7f20f0101fe0 2024-04-21T08:38:08.293 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-21T08:38:08.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.294+0000 7f20f5fce640 1 -- 172.21.15.86:0/3016433152 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f20dc0a3bb0 msgr2=0x7f20dc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:08.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.294+0000 7f20f5fce640 1 --2- 172.21.15.86:0/3016433152 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f20dc0a3bb0 0x7f20dc0a6070 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f20e0006010 tx=0x7f20e00095a0 comp rx=0 tx=0).stop 2024-04-21T08:38:08.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.294+0000 7f20f5fce640 1 -- 172.21.15.86:0/3016433152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20f0101fe0 msgr2=0x7f20f0103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:08.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.294+0000 7f20f5fce640 1 --2- 172.21.15.86:0/3016433152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20f0101fe0 0x7f20f0103450 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f20d800c970 tx=0x7f20d800ce30 comp rx=0 tx=0).stop 2024-04-21T08:38:08.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.295+0000 7f20f5fce640 1 -- 172.21.15.86:0/3016433152 shutdown_connections 2024-04-21T08:38:08.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.295+0000 7f20f5fce640 1 --2- 172.21.15.86:0/3016433152 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f20dc0a3bb0 0x7f20dc0a6070 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:08.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.295+0000 7f20f5fce640 1 --2- 172.21.15.86:0/3016433152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20f01029a0 0x7f20f0103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:08.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.295+0000 7f20f5fce640 1 --2- 172.21.15.86:0/3016433152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20f0101fe0 0x7f20f0103450 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:08.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.295+0000 7f20f5fce640 1 --2- 172.21.15.86:0/3016433152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20f0100e90 0x7f20f0104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:08.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.295+0000 7f20f5fce640 1 -- 172.21.15.86:0/3016433152 >> 172.21.15.86:0/3016433152 conn(0x7f20f00fc400 msgr2=0x7f20f0069150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:08.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.295+0000 7f20f5fce640 1 -- 172.21.15.86:0/3016433152 shutdown_connections 2024-04-21T08:38:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.295+0000 7f20f5fce640 1 -- 172.21.15.86:0/3016433152 wait complete. 2024-04-21T08:38:08.306 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:38:08.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.407+0000 7faa4e932640 1 -- 172.21.15.86:0/3961833020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa481029a0 msgr2=0x7faa481092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:08.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.407+0000 7faa4e932640 1 --2- 172.21.15.86:0/3961833020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa481029a0 0x7faa481092c0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7faa40009ca0 tx=0x7faa4002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:08.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.407+0000 7faa4e932640 1 -- 172.21.15.86:0/3961833020 shutdown_connections 2024-04-21T08:38:08.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.407+0000 7faa4e932640 1 --2- 172.21.15.86:0/3961833020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa481029a0 0x7faa481092c0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:08.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.407+0000 7faa4e932640 1 --2- 172.21.15.86:0/3961833020 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa48101fe0 0x7faa48102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:08.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.407+0000 7faa4e932640 1 --2- 172.21.15.86:0/3961833020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa48100e90 0x7faa481012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:08.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.407+0000 7faa4e932640 1 -- 172.21.15.86:0/3961833020 >> 172.21.15.86:0/3961833020 conn(0x7faa480fc440 msgr2=0x7faa480fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:08.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.407+0000 7faa4e932640 1 -- 172.21.15.86:0/3961833020 shutdown_connections 2024-04-21T08:38:08.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.407+0000 7faa4e932640 1 -- 172.21.15.86:0/3961833020 wait complete. 2024-04-21T08:38:08.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.408+0000 7faa4e932640 1 Processor -- start 2024-04-21T08:38:08.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.408+0000 7faa4e932640 1 -- start start 2024-04-21T08:38:08.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.408+0000 7faa4e932640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa48100e90 0x7faa481970e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:08.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.408+0000 7faa4e932640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa48101fe0 0x7faa48197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:08.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.408+0000 7faa4e932640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa481029a0 0x7faa4819e690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:08.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.408+0000 7faa4e932640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa4810ca40 con 0x7faa48101fe0 2024-04-21T08:38:08.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.408+0000 7faa4e932640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faa4810c8c0 con 0x7faa48100e90 2024-04-21T08:38:08.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.408+0000 7faa4e932640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faa4810cbc0 con 0x7faa481029a0 2024-04-21T08:38:08.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.408+0000 7faa477fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa48101fe0 0x7faa48197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:08.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.408+0000 7faa477fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa48101fe0 0x7faa48197620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51192/0 (socket says 172.21.15.86:51192) 2024-04-21T08:38:08.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.408+0000 7faa477fe640 1 -- 172.21.15.86:0/2777279963 learned_addr learned my addr 172.21.15.86:0/2777279963 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:08.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.408+0000 7faa47fff640 1 --2- 172.21.15.86:0/2777279963 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa48100e90 0x7faa481970e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:08.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.408+0000 7faa477fe640 1 -- 172.21.15.86:0/2777279963 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa481029a0 msgr2=0x7faa4819e690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:08.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.408+0000 7faa4cecf640 1 --2- 172.21.15.86:0/2777279963 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa481029a0 0x7faa4819e690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:08.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.408+0000 7faa477fe640 1 --2- 172.21.15.86:0/2777279963 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa481029a0 0x7faa4819e690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:08.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.408+0000 7faa477fe640 1 -- 172.21.15.86:0/2777279963 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa48100e90 msgr2=0x7faa481970e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:08.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.408+0000 7faa477fe640 1 --2- 172.21.15.86:0/2777279963 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa48100e90 0x7faa481970e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:08.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.408+0000 7faa477fe640 1 -- 172.21.15.86:0/2777279963 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa4819ed00 con 0x7faa48101fe0 2024-04-21T08:38:08.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.409+0000 7faa4cecf640 1 --2- 172.21.15.86:0/2777279963 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa481029a0 0x7faa4819e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:08.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.409+0000 7faa47fff640 1 --2- 172.21.15.86:0/2777279963 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa48100e90 0x7faa481970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:08.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.409+0000 7faa477fe640 1 --2- 172.21.15.86:0/2777279963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa48101fe0 0x7faa48197620 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7faa3c00d770 tx=0x7faa3c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:08.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.409+0000 7faa457fa640 1 -- 172.21.15.86:0/2777279963 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3c017070 con 0x7faa48101fe0 2024-04-21T08:38:08.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.409+0000 7faa457fa640 1 -- 172.21.15.86:0/2777279963 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa3c009c30 con 0x7faa48101fe0 2024-04-21T08:38:08.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.409+0000 7faa4e932640 1 -- 172.21.15.86:0/2777279963 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa4819eff0 con 0x7faa48101fe0 2024-04-21T08:38:08.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.409+0000 7faa457fa640 1 -- 172.21.15.86:0/2777279963 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa3c003cc0 con 0x7faa48101fe0 2024-04-21T08:38:08.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.409+0000 7faa4e932640 1 -- 172.21.15.86:0/2777279963 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa4819f4b0 con 0x7faa48101fe0 2024-04-21T08:38:08.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.410+0000 7faa4e932640 1 -- 172.21.15.86:0/2777279963 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faa481081a0 con 0x7faa48101fe0 2024-04-21T08:38:08.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.411+0000 7faa457fa640 1 -- 172.21.15.86:0/2777279963 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7faa3c003e60 con 0x7faa48101fe0 2024-04-21T08:38:08.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.412+0000 7faa457fa640 1 --2- 172.21.15.86:0/2777279963 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7faa300a3bb0 0x7faa300a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:08.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.412+0000 7faa47fff640 1 --2- 172.21.15.86:0/2777279963 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7faa300a3bb0 0x7faa300a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:08.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.412+0000 7faa457fa640 1 -- 172.21.15.86:0/2777279963 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7faa3c0d05b0 con 0x7faa48101fe0 2024-04-21T08:38:08.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.412+0000 7faa47fff640 1 --2- 172.21.15.86:0/2777279963 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7faa300a3bb0 0x7faa300a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7faa38009930 tx=0x7faa38008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:08.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.415+0000 7faa457fa640 1 -- 172.21.15.86:0/2777279963 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faa3c009de0 con 0x7faa48101fe0 2024-04-21T08:38:08.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.591+0000 7faa4e932640 1 -- 172.21.15.86:0/2777279963 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7faa4802d810 con 0x7faa48101fe0 2024-04-21T08:38:08.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.591+0000 7faa457fa640 1 -- 172.21.15.86:0/2777279963 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) ==== 103+0+11 (secure 0 0 0) 0x7faa3c09cc30 con 0x7faa48101fe0 2024-04-21T08:38:08.592 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-21T08:38:08.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.593+0000 7faa4e932640 1 -- 172.21.15.86:0/2777279963 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7faa300a3bb0 msgr2=0x7faa300a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:08.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.593+0000 7faa4e932640 1 --2- 172.21.15.86:0/2777279963 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7faa300a3bb0 0x7faa300a6070 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7faa38009930 tx=0x7faa38008040 comp rx=0 tx=0).stop 2024-04-21T08:38:08.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.593+0000 7faa4e932640 1 -- 172.21.15.86:0/2777279963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa48101fe0 msgr2=0x7faa48197620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:08.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.593+0000 7faa4e932640 1 --2- 172.21.15.86:0/2777279963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa48101fe0 0x7faa48197620 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7faa3c00d770 tx=0x7faa3c00dc30 comp rx=0 tx=0).stop 2024-04-21T08:38:08.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.594+0000 7faa4e932640 1 -- 172.21.15.86:0/2777279963 shutdown_connections 2024-04-21T08:38:08.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.594+0000 7faa4e932640 1 --2- 172.21.15.86:0/2777279963 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7faa300a3bb0 0x7faa300a6070 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:08.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.594+0000 7faa4e932640 1 --2- 172.21.15.86:0/2777279963 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa481029a0 0x7faa4819e690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:08.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.594+0000 7faa4e932640 1 --2- 172.21.15.86:0/2777279963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa48101fe0 0x7faa48197620 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:08.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.594+0000 7faa4e932640 1 --2- 172.21.15.86:0/2777279963 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa48100e90 0x7faa481970e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:08.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.594+0000 7faa4e932640 1 -- 172.21.15.86:0/2777279963 >> 172.21.15.86:0/2777279963 conn(0x7faa480fc440 msgr2=0x7faa48105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:08.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.594+0000 7faa4e932640 1 -- 172.21.15.86:0/2777279963 shutdown_connections 2024-04-21T08:38:08.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.594+0000 7faa4e932640 1 -- 172.21.15.86:0/2777279963 wait complete. 2024-04-21T08:38:08.604 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:08 smithi086 ceph-mon[27939]: pgmap v343: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-21T08:38:08.604 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:08 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1954705450' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:08.604 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:08 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/933591141' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:08.604 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:08 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/58603177' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:08.604 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:08 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3016433152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:08.605 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:38:08.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.702+0000 7fb196367640 1 -- 172.21.15.86:0/1701513940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb190102a50 msgr2=0x7fb190104e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:08.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.702+0000 7fb196367640 1 --2- 172.21.15.86:0/1701513940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb190102a50 0x7fb190104e40 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7fb180009900 tx=0x7fb18002bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:08.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.703+0000 7fb196367640 1 -- 172.21.15.86:0/1701513940 shutdown_connections 2024-04-21T08:38:08.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.703+0000 7fb196367640 1 --2- 172.21.15.86:0/1701513940 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb190107cb0 0x7fb19010a0a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:08.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.703+0000 7fb196367640 1 --2- 172.21.15.86:0/1701513940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb190105380 0x7fb190107770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:08.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.703+0000 7fb196367640 1 --2- 172.21.15.86:0/1701513940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb190102a50 0x7fb190104e40 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:08.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.703+0000 7fb196367640 1 -- 172.21.15.86:0/1701513940 >> 172.21.15.86:0/1701513940 conn(0x7fb1900fc3e0 msgr2=0x7fb1900fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:08.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.703+0000 7fb196367640 1 -- 172.21.15.86:0/1701513940 shutdown_connections 2024-04-21T08:38:08.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.703+0000 7fb196367640 1 -- 172.21.15.86:0/1701513940 wait complete. 2024-04-21T08:38:08.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.703+0000 7fb196367640 1 Processor -- start 2024-04-21T08:38:08.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.703+0000 7fb196367640 1 -- start start 2024-04-21T08:38:08.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.703+0000 7fb196367640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb190102a50 0x7fb190198330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:08.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.703+0000 7fb196367640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb190105380 0x7fb190198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:08.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.703+0000 7fb196367640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb190107cb0 0x7fb19019f8e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:08.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.703+0000 7fb196367640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb19010db90 con 0x7fb190102a50 2024-04-21T08:38:08.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.703+0000 7fb196367640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb19010da10 con 0x7fb190107cb0 2024-04-21T08:38:08.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.703+0000 7fb196367640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb19010dd10 con 0x7fb190105380 2024-04-21T08:38:08.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.704+0000 7fb18ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb190102a50 0x7fb190198330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:08.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.704+0000 7fb18ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb190102a50 0x7fb190198330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51204/0 (socket says 172.21.15.86:51204) 2024-04-21T08:38:08.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.704+0000 7fb18ffff640 1 -- 172.21.15.86:0/2785258834 learned_addr learned my addr 172.21.15.86:0/2785258834 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:08.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.704+0000 7fb194904640 1 --2- 172.21.15.86:0/2785258834 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb190107cb0 0x7fb19019f8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:08.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.704+0000 7fb18ffff640 1 -- 172.21.15.86:0/2785258834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb190105380 msgr2=0x7fb190198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:08.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.704+0000 7fb18ffff640 1 --2- 172.21.15.86:0/2785258834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb190105380 0x7fb190198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:08.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.704+0000 7fb18ffff640 1 -- 172.21.15.86:0/2785258834 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb190107cb0 msgr2=0x7fb19019f8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:08.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.704+0000 7fb18ffff640 1 --2- 172.21.15.86:0/2785258834 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb190107cb0 0x7fb19019f8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:08.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.704+0000 7fb18ffff640 1 -- 172.21.15.86:0/2785258834 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb19019ff50 con 0x7fb190102a50 2024-04-21T08:38:08.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.704+0000 7fb194904640 1 --2- 172.21.15.86:0/2785258834 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb190107cb0 0x7fb19019f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:08.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.704+0000 7fb18ffff640 1 --2- 172.21.15.86:0/2785258834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb190102a50 0x7fb190198330 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb180036040 tx=0x7fb1800042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:08.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.704+0000 7fb18d7fa640 1 -- 172.21.15.86:0/2785258834 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb180043020 con 0x7fb190102a50 2024-04-21T08:38:08.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.704+0000 7fb196367640 1 -- 172.21.15.86:0/2785258834 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb1901a01e0 con 0x7fb190102a50 2024-04-21T08:38:08.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.704+0000 7fb18d7fa640 1 -- 172.21.15.86:0/2785258834 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1800054c0 con 0x7fb190102a50 2024-04-21T08:38:08.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.704+0000 7fb18d7fa640 1 -- 172.21.15.86:0/2785258834 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb180003db0 con 0x7fb190102a50 2024-04-21T08:38:08.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.704+0000 7fb196367640 1 -- 172.21.15.86:0/2785258834 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1901a07b0 con 0x7fb190102a50 2024-04-21T08:38:08.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.705+0000 7fb196367640 1 -- 172.21.15.86:0/2785258834 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb1900fded0 con 0x7fb190102a50 2024-04-21T08:38:08.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.706+0000 7fb18d7fa640 1 -- 172.21.15.86:0/2785258834 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb180046020 con 0x7fb190102a50 2024-04-21T08:38:08.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.707+0000 7fb18d7fa640 1 --2- 172.21.15.86:0/2785258834 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb1780a3bb0 0x7fb1780a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:08.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.707+0000 7fb18f7fe640 1 --2- 172.21.15.86:0/2785258834 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb1780a3bb0 0x7fb1780a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:08.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.707+0000 7fb18d7fa640 1 -- 172.21.15.86:0/2785258834 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb1800f2d00 con 0x7fb190102a50 2024-04-21T08:38:08.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.707+0000 7fb18f7fe640 1 --2- 172.21.15.86:0/2785258834 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb1780a3bb0 0x7fb1780a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb190199840 tx=0x7fb184006c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:08.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.712+0000 7fb18d7fa640 1 -- 172.21.15.86:0/2785258834 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb18004d020 con 0x7fb190102a50 2024-04-21T08:38:08.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:08 smithi088 ceph-mon[31259]: pgmap v343: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-21T08:38:08.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:08 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1954705450' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:08.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:08 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/933591141' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:08.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:08 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/58603177' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:08.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:08 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3016433152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:08.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:08 smithi179 ceph-mon[31829]: pgmap v343: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 170 B/s wr, 8 op/s 2024-04-21T08:38:08.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:08 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1954705450' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:08.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:08 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/933591141' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:08.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:08 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/58603177' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:08.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:08 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3016433152' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:08.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.931+0000 7fb196367640 1 -- 172.21.15.86:0/2785258834 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb190062b80 con 0x7fb190102a50 2024-04-21T08:38:08.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.932+0000 7fb18d7fa640 1 -- 172.21.15.86:0/2785258834 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7fb1800bf380 con 0x7fb190102a50 2024-04-21T08:38:08.932 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:38:08.932 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:36:47.458637+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24455,24472,24394,24398,24482]},"id":1}]} 2024-04-21T08:38:08.932 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-21T08:38:08.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.934+0000 7fb196367640 1 -- 172.21.15.86:0/2785258834 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb1780a3bb0 msgr2=0x7fb1780a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:08.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.934+0000 7fb196367640 1 --2- 172.21.15.86:0/2785258834 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb1780a3bb0 0x7fb1780a6070 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7fb190199840 tx=0x7fb184006c90 comp rx=0 tx=0).stop 2024-04-21T08:38:08.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.934+0000 7fb196367640 1 -- 172.21.15.86:0/2785258834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb190102a50 msgr2=0x7fb190198330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:08.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.934+0000 7fb196367640 1 --2- 172.21.15.86:0/2785258834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb190102a50 0x7fb190198330 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fb180036040 tx=0x7fb1800042f0 comp rx=0 tx=0).stop 2024-04-21T08:38:08.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.934+0000 7fb196367640 1 -- 172.21.15.86:0/2785258834 shutdown_connections 2024-04-21T08:38:08.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.934+0000 7fb196367640 1 --2- 172.21.15.86:0/2785258834 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb1780a3bb0 0x7fb1780a6070 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:08.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.934+0000 7fb196367640 1 --2- 172.21.15.86:0/2785258834 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb190107cb0 0x7fb19019f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:08.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.934+0000 7fb196367640 1 --2- 172.21.15.86:0/2785258834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb190105380 0x7fb190198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:08.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.934+0000 7fb196367640 1 --2- 172.21.15.86:0/2785258834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb190102a50 0x7fb190198330 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:08.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.934+0000 7fb196367640 1 -- 172.21.15.86:0/2785258834 >> 172.21.15.86:0/2785258834 conn(0x7fb1900fc3e0 msgr2=0x7fb1901034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:08.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.934+0000 7fb196367640 1 -- 172.21.15.86:0/2785258834 shutdown_connections 2024-04-21T08:38:08.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:08.934+0000 7fb196367640 1 -- 172.21.15.86:0/2785258834 wait complete. 2024-04-21T08:38:08.945 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:38:09.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.043+0000 7fabc1bf4640 1 -- 172.21.15.86:0/987490536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabbc102980 msgr2=0x7fabbc1092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:09.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.043+0000 7fabc1bf4640 1 --2- 172.21.15.86:0/987490536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabbc102980 0x7fabbc1092a0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fabb4009ca0 tx=0x7fabb402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:09.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.043+0000 7fabc1bf4640 1 -- 172.21.15.86:0/987490536 shutdown_connections 2024-04-21T08:38:09.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.043+0000 7fabc1bf4640 1 --2- 172.21.15.86:0/987490536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabbc102980 0x7fabbc1092a0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:09.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.043+0000 7fabc1bf4640 1 --2- 172.21.15.86:0/987490536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabbc101fc0 0x7fabbc102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:09.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.043+0000 7fabc1bf4640 1 --2- 172.21.15.86:0/987490536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fabbc100e70 0x7fabbc101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:09.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.043+0000 7fabc1bf4640 1 -- 172.21.15.86:0/987490536 >> 172.21.15.86:0/987490536 conn(0x7fabbc0fc3e0 msgr2=0x7fabbc0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:09.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.044+0000 7fabc1bf4640 1 -- 172.21.15.86:0/987490536 shutdown_connections 2024-04-21T08:38:09.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.044+0000 7fabc1bf4640 1 -- 172.21.15.86:0/987490536 wait complete. 2024-04-21T08:38:09.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.044+0000 7fabc1bf4640 1 Processor -- start 2024-04-21T08:38:09.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.044+0000 7fabc1bf4640 1 -- start start 2024-04-21T08:38:09.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.044+0000 7fabc1bf4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabbc100e70 0x7fabbc069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:09.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.044+0000 7fabc1bf4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabbc101fc0 0x7fabbc19cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:09.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.044+0000 7fabc1bf4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fabbc102980 0x7fabbc19f530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:09.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.044+0000 7fabc1bf4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc10ca40 con 0x7fabbc101fc0 2024-04-21T08:38:09.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.044+0000 7fabc1bf4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc10c8c0 con 0x7fabbc102980 2024-04-21T08:38:09.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.044+0000 7fabc1bf4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fabbc10cbc0 con 0x7fabbc100e70 2024-04-21T08:38:09.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.045+0000 7fabbaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabbc101fc0 0x7fabbc19cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:09.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.045+0000 7fabbaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabbc101fc0 0x7fabbc19cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51222/0 (socket says 172.21.15.86:51222) 2024-04-21T08:38:09.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.045+0000 7fabbaffd640 1 -- 172.21.15.86:0/182952996 learned_addr learned my addr 172.21.15.86:0/182952996 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:09.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.045+0000 7fabbb7fe640 1 --2- 172.21.15.86:0/182952996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabbc100e70 0x7fabbc069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:09.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.045+0000 7fabbbfff640 1 --2- 172.21.15.86:0/182952996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fabbc102980 0x7fabbc19f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:09.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.045+0000 7fabbaffd640 1 -- 172.21.15.86:0/182952996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabbc100e70 msgr2=0x7fabbc069c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:09.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.045+0000 7fabbaffd640 1 --2- 172.21.15.86:0/182952996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabbc100e70 0x7fabbc069c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:09.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.045+0000 7fabbaffd640 1 -- 172.21.15.86:0/182952996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fabbc102980 msgr2=0x7fabbc19f530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:09.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.045+0000 7fabbaffd640 1 --2- 172.21.15.86:0/182952996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fabbc102980 0x7fabbc19f530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:09.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.045+0000 7fabbaffd640 1 -- 172.21.15.86:0/182952996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabbc19fe40 con 0x7fabbc101fc0 2024-04-21T08:38:09.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.045+0000 7fabbb7fe640 1 --2- 172.21.15.86:0/182952996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabbc100e70 0x7fabbc069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:09.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.045+0000 7fabbbfff640 1 --2- 172.21.15.86:0/182952996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fabbc102980 0x7fabbc19f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:09.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.045+0000 7fabbaffd640 1 --2- 172.21.15.86:0/182952996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabbc101fc0 0x7fabbc19cf10 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7faba400b450 tx=0x7faba400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:09.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.045+0000 7fabb8ff9640 1 -- 172.21.15.86:0/182952996 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba4015020 con 0x7fabbc101fc0 2024-04-21T08:38:09.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.045+0000 7fabc1bf4640 1 -- 172.21.15.86:0/182952996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabbc1a00d0 con 0x7fabbc101fc0 2024-04-21T08:38:09.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.045+0000 7fabb8ff9640 1 -- 172.21.15.86:0/182952996 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faba4011d40 con 0x7fabbc101fc0 2024-04-21T08:38:09.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.045+0000 7fabb8ff9640 1 -- 172.21.15.86:0/182952996 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba400f340 con 0x7fabbc101fc0 2024-04-21T08:38:09.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.045+0000 7fabc1bf4640 1 -- 172.21.15.86:0/182952996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabbc1a05d0 con 0x7fabbc101fc0 2024-04-21T08:38:09.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.046+0000 7fabc1bf4640 1 -- 172.21.15.86:0/182952996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fabbc108180 con 0x7fabbc101fc0 2024-04-21T08:38:09.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.047+0000 7fabb8ff9640 1 -- 172.21.15.86:0/182952996 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7faba4018020 con 0x7fabbc101fc0 2024-04-21T08:38:09.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.048+0000 7fabb8ff9640 1 --2- 172.21.15.86:0/182952996 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7faba80a3bb0 0x7faba80a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:09.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.048+0000 7fabbb7fe640 1 --2- 172.21.15.86:0/182952996 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7faba80a3bb0 0x7faba80a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:09.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.048+0000 7fabb8ff9640 1 -- 172.21.15.86:0/182952996 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7faba40d08c0 con 0x7fabbc101fc0 2024-04-21T08:38:09.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.048+0000 7fabbb7fe640 1 --2- 172.21.15.86:0/182952996 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7faba80a3bb0 0x7faba80a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fabac0058c0 tx=0x7fabac005d30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:09.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.051+0000 7fabb8ff9640 1 -- 172.21.15.86:0/182952996 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faba401f020 con 0x7fabbc101fc0 2024-04-21T08:38:09.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.228+0000 7fabc1bf4640 1 -- 172.21.15.86:0/182952996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fabbc02d850 con 0x7fabbc101fc0 2024-04-21T08:38:09.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.230+0000 7fabb8ff9640 1 -- 172.21.15.86:0/182952996 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24595 (secure 0 0 0) 0x7faba400d040 con 0x7fabbc101fc0 2024-04-21T08:38:09.230 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:38:09.231 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":91,"fsid":"f8cec7cc-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:22.687329+0000","modified":"2024-04-21T08:37:24.279025+0000","last_up_change":"2024-04-21T08:35:30.048315+0000","last_in_change":"2024-04-21T08:35:10.099496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:45.279109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:18.300782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:19.574636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:22.905632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89b7601f-0620-4db2-90f1-b45cea173738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6803","nonce":2571516897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6805","nonce":2571516897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6809","nonce":2571516897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6807","nonce":2571516897}]},"public_addr":"172.21.15.86:6803/2571516897","cluster_addr":"172.21.15.86:6805/2571516897","heartbeat_back_addr":"172.21.15.86:6809/2571516897","heartbeat_front_addr":"172.21.15.86:6807/2571516897","state":["exists","up"]},{"osd":1,"uuid":"4891031c-f98c-42bc-bb3b-a420855ecc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6801","nonce":74067182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6803","nonce":74067182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6807","nonce":74067182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6805","nonce":74067182}]},"public_addr":"172.21.15.88:6801/74067182","cluster_addr":"172.21.15.88:6803/74067182","heartbeat_back_addr":"172.21.15.88:6807/74067182","heartbeat_front_addr":"172.21.15.88:6805/74067182","state":["exists","up"]},{"osd":2,"uuid":"60d12aa4-9312-4fa2-8d0b-2b3f42caad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6801","nonce":4137659051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6803","nonce":4137659051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6807","nonce":4137659051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6805","nonce":4137659051}]},"public_addr":"172.21.15.179:6801/4137659051","cluster_addr":"172.21.15.179:6803/4137659051","heartbeat_back_addr":"172.21.15.179:6807/4137659051","heartbeat_front_addr":"172.21.15.179:6805/4137659051","state":["exists","up"]},{"osd":3,"uuid":"404c3158-8cb0-4c02-b48a-f9ee5ff7b784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6811","nonce":2309603148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6813","nonce":2309603148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6817","nonce":2309603148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6815","nonce":2309603148}]},"public_addr":"172.21.15.86:6811/2309603148","cluster_addr":"172.21.15.86:6813/2309603148","heartbeat_back_addr":"172.21.15.86:6817/2309603148","heartbeat_front_addr":"172.21.15.86:6815/2309603148","state":["exists","up"]},{"osd":4,"uuid":"458ff202-d781-4ec7-ac93-0b49f835b503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6809","nonce":1818714217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6811","nonce":1818714217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6815","nonce":1818714217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6813","nonce":1818714217}]},"public_addr":"172.21.15.88:6809/1818714217","cluster_addr":"172.21.15.88:6811/1818714217","heartbeat_back_addr":"172.21.15.88:6815/1818714217","heartbeat_front_addr":"172.21.15.88:6813/1818714217","state":["exists","up"]},{"osd":5,"uuid":"8123cc90-bd8b-4204-bf31-46b34e9f589d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6809","nonce":863840990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6811","nonce":863840990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6815","nonce":863840990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6813","nonce":863840990}]},"public_addr":"172.21.15.179:6809/863840990","cluster_addr":"172.21.15.179:6811/863840990","heartbeat_back_addr":"172.21.15.179:6815/863840990","heartbeat_front_addr":"172.21.15.179:6813/863840990","state":["exists","up"]},{"osd":6,"uuid":"aac9d7c7-659a-4110-903a-a871c086066f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6819","nonce":1224231118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6821","nonce":1224231118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6825","nonce":1224231118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6823","nonce":1224231118}]},"public_addr":"172.21.15.86:6819/1224231118","cluster_addr":"172.21.15.86:6821/1224231118","heartbeat_back_addr":"172.21.15.86:6825/1224231118","heartbeat_front_addr":"172.21.15.86:6823/1224231118","state":["exists","up"]},{"osd":7,"uuid":"df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6817","nonce":136629878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6819","nonce":136629878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6823","nonce":136629878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6821","nonce":136629878}]},"public_addr":"172.21.15.88:6817/136629878","cluster_addr":"172.21.15.88:6819/136629878","heartbeat_back_addr":"172.21.15.88:6823/136629878","heartbeat_front_addr":"172.21.15.88:6821/136629878","state":["exists","up"]},{"osd":8,"uuid":"63e59f4b-0426-46db-92f7-c55dae61b5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6817","nonce":1770418957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6819","nonce":1770418957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6823","nonce":1770418957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6821","nonce":1770418957}]},"public_addr":"172.21.15.179:6817/1770418957","cluster_addr":"172.21.15.179:6819/1770418957","heartbeat_back_addr":"172.21.15.179:6823/1770418957","heartbeat_front_addr":"172.21.15.179:6821/1770418957","state":["exists","up"]},{"osd":9,"uuid":"5841bb36-ed6b-45d8-b857-eeab139c44e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6827","nonce":1108500274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6829","nonce":1108500274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6833","nonce":1108500274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6831","nonce":1108500274}]},"public_addr":"172.21.15.86:6827/1108500274","cluster_addr":"172.21.15.86:6829/1108500274","heartbeat_back_addr":"172.21.15.86:6833/1108500274","heartbeat_front_addr":"172.21.15.86:6831/1108500274","state":["exists","up"]},{"osd":10,"uuid":"f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6825","nonce":1294275353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6827","nonce":1294275353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6831","nonce":1294275353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6829","nonce":1294275353}]},"public_addr":"172.21.15.88:6825/1294275353","cluster_addr":"172.21.15.88:6827/1294275353","heartbeat_back_addr":"172.21.15.88:6831/1294275353","heartbeat_front_addr":"172.21.15.88:6829/1294275353","state":["exists","up"]},{"osd":11,"uuid":"e0d4cc76-e7a6-4a6b-909c-95488ccd5a36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6825","nonce":4007070233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6827","nonce":4007070233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6831","nonce":4007070233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6829","nonce":4007070233}]},"public_addr":"172.21.15.179:6825/4007070233","cluster_addr":"172.21.15.179:6827/4007070233","heartbeat_back_addr":"172.21.15.179:6831/4007070233","heartbeat_front_addr":"172.21.15.179:6829/4007070233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:55.210605+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:18.723090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:42.547979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:06.421622+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:30.858315+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:55.455313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.623369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:46.185111+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.103627+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:37.297545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:02.637431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:28.199292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6800/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/1412393225":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6800/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/1687852404":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2020707404":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2568130933":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/3952607648":"2024-04-22T08:28:57.384403+0000","172.21.15.86:6801/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/2756903426":"2024-04-22T08:29:23.190001+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:38:09.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.232+0000 7fabc1bf4640 1 -- 172.21.15.86:0/182952996 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7faba80a3bb0 msgr2=0x7faba80a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:09.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.232+0000 7fabc1bf4640 1 --2- 172.21.15.86:0/182952996 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7faba80a3bb0 0x7faba80a6070 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7fabac0058c0 tx=0x7fabac005d30 comp rx=0 tx=0).stop 2024-04-21T08:38:09.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.232+0000 7fabc1bf4640 1 -- 172.21.15.86:0/182952996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabbc101fc0 msgr2=0x7fabbc19cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:09.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.232+0000 7fabc1bf4640 1 --2- 172.21.15.86:0/182952996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabbc101fc0 0x7fabbc19cf10 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7faba400b450 tx=0x7faba400b910 comp rx=0 tx=0).stop 2024-04-21T08:38:09.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.232+0000 7fabc1bf4640 1 -- 172.21.15.86:0/182952996 shutdown_connections 2024-04-21T08:38:09.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.232+0000 7fabc1bf4640 1 --2- 172.21.15.86:0/182952996 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7faba80a3bb0 0x7faba80a6070 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:09.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.232+0000 7fabc1bf4640 1 --2- 172.21.15.86:0/182952996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fabbc102980 0x7fabbc19f530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:09.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.232+0000 7fabc1bf4640 1 --2- 172.21.15.86:0/182952996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabbc101fc0 0x7fabbc19cf10 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:09.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.232+0000 7fabc1bf4640 1 --2- 172.21.15.86:0/182952996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabbc100e70 0x7fabbc069c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:09.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.232+0000 7fabc1bf4640 1 -- 172.21.15.86:0/182952996 >> 172.21.15.86:0/182952996 conn(0x7fabbc0fc3e0 msgr2=0x7fabbc105350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:09.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.232+0000 7fabc1bf4640 1 -- 172.21.15.86:0/182952996 shutdown_connections 2024-04-21T08:38:09.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.232+0000 7fabc1bf4640 1 -- 172.21.15.86:0/182952996 wait complete. 2024-04-21T08:38:09.244 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:31:45.279109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:36:18.300782+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:36:19.574636+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:36:22.905632+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:38:09.244 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:38:09.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.345+0000 7fb9546a8640 1 -- 172.21.15.86:0/2115808473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb94c107c70 msgr2=0x7fb94c10a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:09.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.345+0000 7fb9546a8640 1 --2- 172.21.15.86:0/2115808473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb94c107c70 0x7fb94c10a0c0 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7fb944009900 tx=0x7fb94402bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:09.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.346+0000 7fb9546a8640 1 -- 172.21.15.86:0/2115808473 shutdown_connections 2024-04-21T08:38:09.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.346+0000 7fb9546a8640 1 --2- 172.21.15.86:0/2115808473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb94c107c70 0x7fb94c10a0c0 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:09.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.346+0000 7fb9546a8640 1 --2- 172.21.15.86:0/2115808473 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb94c105340 0x7fb94c107730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:09.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.346+0000 7fb9546a8640 1 --2- 172.21.15.86:0/2115808473 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb94c102a10 0x7fb94c104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:09.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.346+0000 7fb9546a8640 1 -- 172.21.15.86:0/2115808473 >> 172.21.15.86:0/2115808473 conn(0x7fb94c0fc400 msgr2=0x7fb94c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:09.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.346+0000 7fb9546a8640 1 -- 172.21.15.86:0/2115808473 shutdown_connections 2024-04-21T08:38:09.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.346+0000 7fb9546a8640 1 -- 172.21.15.86:0/2115808473 wait complete. 2024-04-21T08:38:09.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.346+0000 7fb9546a8640 1 Processor -- start 2024-04-21T08:38:09.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.346+0000 7fb9546a8640 1 -- start start 2024-04-21T08:38:09.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.346+0000 7fb9546a8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb94c102a10 0x7fb94c1981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:09.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.346+0000 7fb9546a8640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb94c105340 0x7fb94c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:09.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.346+0000 7fb9546a8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb94c107c70 0x7fb94c19f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:09.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.346+0000 7fb9546a8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb94c10dae0 con 0x7fb94c102a10 2024-04-21T08:38:09.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.346+0000 7fb9546a8640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb94c10d960 con 0x7fb94c107c70 2024-04-21T08:38:09.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.346+0000 7fb9546a8640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb94c10dc60 con 0x7fb94c105340 2024-04-21T08:38:09.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.346+0000 7fb952444640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb94c102a10 0x7fb94c1981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:09.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.346+0000 7fb952444640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb94c102a10 0x7fb94c1981b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51250/0 (socket says 172.21.15.86:51250) 2024-04-21T08:38:09.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.346+0000 7fb952444640 1 -- 172.21.15.86:0/1406406385 learned_addr learned my addr 172.21.15.86:0/1406406385 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:09.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.347+0000 7fb952c45640 1 --2- 172.21.15.86:0/1406406385 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb94c107c70 0x7fb94c19f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:09.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.347+0000 7fb952444640 1 -- 172.21.15.86:0/1406406385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb94c105340 msgr2=0x7fb94c1986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:09.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.347+0000 7fb952444640 1 --2- 172.21.15.86:0/1406406385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb94c105340 0x7fb94c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:09.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.347+0000 7fb952444640 1 -- 172.21.15.86:0/1406406385 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb94c107c70 msgr2=0x7fb94c19f760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:09.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.347+0000 7fb952444640 1 --2- 172.21.15.86:0/1406406385 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb94c107c70 0x7fb94c19f760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:09.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.347+0000 7fb952444640 1 -- 172.21.15.86:0/1406406385 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb94c19fdd0 con 0x7fb94c102a10 2024-04-21T08:38:09.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.347+0000 7fb952c45640 1 --2- 172.21.15.86:0/1406406385 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb94c107c70 0x7fb94c19f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:09.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.347+0000 7fb952444640 1 --2- 172.21.15.86:0/1406406385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb94c102a10 0x7fb94c1981b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb93c00ea80 tx=0x7fb93c00ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:09.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.347+0000 7fb94b7fe640 1 -- 172.21.15.86:0/1406406385 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93c00ce40 con 0x7fb94c102a10 2024-04-21T08:38:09.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.347+0000 7fb94b7fe640 1 -- 172.21.15.86:0/1406406385 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb93c005020 con 0x7fb94c102a10 2024-04-21T08:38:09.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.347+0000 7fb94b7fe640 1 -- 172.21.15.86:0/1406406385 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb93c0116d0 con 0x7fb94c102a10 2024-04-21T08:38:09.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.347+0000 7fb9546a8640 1 -- 172.21.15.86:0/1406406385 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb94c1a00c0 con 0x7fb94c102a10 2024-04-21T08:38:09.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.347+0000 7fb9546a8640 1 -- 172.21.15.86:0/1406406385 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb94c1a0610 con 0x7fb94c102a10 2024-04-21T08:38:09.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.348+0000 7fb9546a8640 1 -- 172.21.15.86:0/1406406385 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb94c0fdef0 con 0x7fb94c102a10 2024-04-21T08:38:09.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.349+0000 7fb94b7fe640 1 -- 172.21.15.86:0/1406406385 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb93c0054d0 con 0x7fb94c102a10 2024-04-21T08:38:09.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.350+0000 7fb94b7fe640 1 --2- 172.21.15.86:0/1406406385 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb9340a3c30 0x7fb9340a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:09.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.350+0000 7fb951c43640 1 --2- 172.21.15.86:0/1406406385 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb9340a3c30 0x7fb9340a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:09.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.350+0000 7fb94b7fe640 1 -- 172.21.15.86:0/1406406385 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb93c0d0a60 con 0x7fb94c102a10 2024-04-21T08:38:09.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.350+0000 7fb951c43640 1 --2- 172.21.15.86:0/1406406385 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb9340a3c30 0x7fb9340a60f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb94c1996c0 tx=0x7fb940005bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:09.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.354+0000 7fb94b7fe640 1 -- 172.21.15.86:0/1406406385 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb93c016030 con 0x7fb94c102a10 2024-04-21T08:38:09.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.532+0000 7fb9546a8640 1 -- 172.21.15.86:0/1406406385 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb94c062b80 con 0x7fb94c102a10 2024-04-21T08:38:09.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.532+0000 7fb94b7fe640 1 -- 172.21.15.86:0/1406406385 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v91) ==== 93+0+10 (secure 0 0 0) 0x7fb93c09d0e0 con 0x7fb94c102a10 2024-04-21T08:38:09.533 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-21T08:38:09.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.534+0000 7fb9546a8640 1 -- 172.21.15.86:0/1406406385 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb9340a3c30 msgr2=0x7fb9340a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:09.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.534+0000 7fb9546a8640 1 --2- 172.21.15.86:0/1406406385 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb9340a3c30 0x7fb9340a60f0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7fb94c1996c0 tx=0x7fb940005bf0 comp rx=0 tx=0).stop 2024-04-21T08:38:09.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.534+0000 7fb9546a8640 1 -- 172.21.15.86:0/1406406385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb94c102a10 msgr2=0x7fb94c1981b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:09.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.534+0000 7fb9546a8640 1 --2- 172.21.15.86:0/1406406385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb94c102a10 0x7fb94c1981b0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7fb93c00ea80 tx=0x7fb93c00ef40 comp rx=0 tx=0).stop 2024-04-21T08:38:09.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.534+0000 7fb9546a8640 1 -- 172.21.15.86:0/1406406385 shutdown_connections 2024-04-21T08:38:09.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.534+0000 7fb9546a8640 1 --2- 172.21.15.86:0/1406406385 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb9340a3c30 0x7fb9340a60f0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:09.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.534+0000 7fb9546a8640 1 --2- 172.21.15.86:0/1406406385 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb94c107c70 0x7fb94c19f760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:09.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.534+0000 7fb9546a8640 1 --2- 172.21.15.86:0/1406406385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb94c105340 0x7fb94c1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:09.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.534+0000 7fb9546a8640 1 --2- 172.21.15.86:0/1406406385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb94c102a10 0x7fb94c1981b0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:09.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.534+0000 7fb9546a8640 1 -- 172.21.15.86:0/1406406385 >> 172.21.15.86:0/1406406385 conn(0x7fb94c0fc400 msgr2=0x7fb94c1032b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:09.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.535+0000 7fb9546a8640 1 -- 172.21.15.86:0/1406406385 shutdown_connections 2024-04-21T08:38:09.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.535+0000 7fb9546a8640 1 -- 172.21.15.86:0/1406406385 wait complete. 2024-04-21T08:38:09.545 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:09 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2777279963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:09.545 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:09 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2785258834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:09.545 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:09 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/182952996' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:09.546 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:38:09.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.647+0000 7f984933d640 1 -- 172.21.15.86:0/181083646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9844101fe0 msgr2=0x7f9844102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:09.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.647+0000 7f984933d640 1 --2- 172.21.15.86:0/181083646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9844101fe0 0x7f9844102460 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7f9838009900 tx=0x7f983802bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:09.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.647+0000 7f984933d640 1 -- 172.21.15.86:0/181083646 shutdown_connections 2024-04-21T08:38:09.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.647+0000 7f984933d640 1 --2- 172.21.15.86:0/181083646 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98441029a0 0x7f98441092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:09.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.647+0000 7f984933d640 1 --2- 172.21.15.86:0/181083646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9844101fe0 0x7f9844102460 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:09.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.647+0000 7f984933d640 1 --2- 172.21.15.86:0/181083646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9844100e90 0x7f98441012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:09.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.647+0000 7f984933d640 1 -- 172.21.15.86:0/181083646 >> 172.21.15.86:0/181083646 conn(0x7f98440fc400 msgr2=0x7f98440fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:09.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.647+0000 7f984933d640 1 -- 172.21.15.86:0/181083646 shutdown_connections 2024-04-21T08:38:09.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.648+0000 7f984933d640 1 -- 172.21.15.86:0/181083646 wait complete. 2024-04-21T08:38:09.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.648+0000 7f984933d640 1 Processor -- start 2024-04-21T08:38:09.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.648+0000 7f984933d640 1 -- start start 2024-04-21T08:38:09.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.648+0000 7f984933d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9844100e90 0x7f9844198240 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:09.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.648+0000 7f984933d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9844101fe0 0x7f9844198780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:09.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.648+0000 7f984933d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98441029a0 0x7f984419f7f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:09.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.648+0000 7f984933d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f984410c9d0 con 0x7f9844101fe0 2024-04-21T08:38:09.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.648+0000 7f984933d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f984410c850 con 0x7f98441029a0 2024-04-21T08:38:09.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.648+0000 7f984933d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f984410cb50 con 0x7f9844100e90 2024-04-21T08:38:09.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.648+0000 7f98427fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9844101fe0 0x7f9844198780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:09.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.648+0000 7f98427fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9844101fe0 0x7f9844198780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36260/0 (socket says 172.21.15.86:36260) 2024-04-21T08:38:09.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.648+0000 7f98427fc640 1 -- 172.21.15.86:0/53768860 learned_addr learned my addr 172.21.15.86:0/53768860 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:09.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.649+0000 7f9842ffd640 1 --2- 172.21.15.86:0/53768860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9844100e90 0x7f9844198240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:09.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.649+0000 7f98437fe640 1 --2- 172.21.15.86:0/53768860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98441029a0 0x7f984419f7f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:09.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.649+0000 7f98427fc640 1 -- 172.21.15.86:0/53768860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9844100e90 msgr2=0x7f9844198240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:09.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.649+0000 7f98427fc640 1 --2- 172.21.15.86:0/53768860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9844100e90 0x7f9844198240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:09.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.649+0000 7f98427fc640 1 -- 172.21.15.86:0/53768860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98441029a0 msgr2=0x7f984419f7f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:09.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.649+0000 7f98427fc640 1 --2- 172.21.15.86:0/53768860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98441029a0 0x7f984419f7f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:09.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.649+0000 7f98427fc640 1 -- 172.21.15.86:0/53768860 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f984419fe60 con 0x7f9844101fe0 2024-04-21T08:38:09.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.649+0000 7f98437fe640 1 --2- 172.21.15.86:0/53768860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98441029a0 0x7f984419f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:09.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.649+0000 7f98427fc640 1 --2- 172.21.15.86:0/53768860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9844101fe0 0x7f9844198780 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9838000c00 tx=0x7f9838004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:09.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.649+0000 7f982bfff640 1 -- 172.21.15.86:0/53768860 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838043020 con 0x7f9844101fe0 2024-04-21T08:38:09.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.649+0000 7f982bfff640 1 -- 172.21.15.86:0/53768860 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9838004aa0 con 0x7f9844101fe0 2024-04-21T08:38:09.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.649+0000 7f984933d640 1 -- 172.21.15.86:0/53768860 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f98441a00f0 con 0x7f9844101fe0 2024-04-21T08:38:09.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.649+0000 7f982bfff640 1 -- 172.21.15.86:0/53768860 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9838003d00 con 0x7f9844101fe0 2024-04-21T08:38:09.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.649+0000 7f984933d640 1 -- 172.21.15.86:0/53768860 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f98441a0640 con 0x7f9844101fe0 2024-04-21T08:38:09.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.650+0000 7f984933d640 1 -- 172.21.15.86:0/53768860 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f98441a0960 con 0x7f9844101fe0 2024-04-21T08:38:09.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.651+0000 7f982bfff640 1 -- 172.21.15.86:0/53768860 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9838046020 con 0x7f9844101fe0 2024-04-21T08:38:09.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.652+0000 7f982bfff640 1 --2- 172.21.15.86:0/53768860 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f982c0a3bb0 0x7f982c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:09.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.652+0000 7f9842ffd640 1 --2- 172.21.15.86:0/53768860 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f982c0a3bb0 0x7f982c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:09.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.652+0000 7f982bfff640 1 -- 172.21.15.86:0/53768860 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f98380f2b20 con 0x7f9844101fe0 2024-04-21T08:38:09.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.652+0000 7f9842ffd640 1 --2- 172.21.15.86:0/53768860 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f982c0a3bb0 0x7f982c0a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9834005c40 tx=0x7f983400a590 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:09.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.656+0000 7f982bfff640 1 -- 172.21.15.86:0/53768860 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f983804d020 con 0x7f9844101fe0 2024-04-21T08:38:09.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:09 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2777279963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:09.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:09 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2785258834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:09.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:09 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/182952996' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:09.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:09 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2777279963' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:09.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:09 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2785258834' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:09.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:09 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/182952996' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:09.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.833+0000 7f984933d640 1 -- 172.21.15.86:0/53768860 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f98441a0b30 con 0x7f9844101fe0 2024-04-21T08:38:09.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.833+0000 7f982bfff640 1 -- 172.21.15.86:0/53768860 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v91) ==== 104+0+11 (secure 0 0 0) 0x7f98380bf1a0 con 0x7f9844101fe0 2024-04-21T08:38:09.834 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-21T08:38:09.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.835+0000 7f984933d640 1 -- 172.21.15.86:0/53768860 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f982c0a3bb0 msgr2=0x7f982c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:09.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.835+0000 7f984933d640 1 --2- 172.21.15.86:0/53768860 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f982c0a3bb0 0x7f982c0a6070 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f9834005c40 tx=0x7f983400a590 comp rx=0 tx=0).stop 2024-04-21T08:38:09.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.835+0000 7f984933d640 1 -- 172.21.15.86:0/53768860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9844101fe0 msgr2=0x7f9844198780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:09.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.835+0000 7f984933d640 1 --2- 172.21.15.86:0/53768860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9844101fe0 0x7f9844198780 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7f9838000c00 tx=0x7f9838004270 comp rx=0 tx=0).stop 2024-04-21T08:38:09.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.835+0000 7f984933d640 1 -- 172.21.15.86:0/53768860 shutdown_connections 2024-04-21T08:38:09.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.835+0000 7f984933d640 1 --2- 172.21.15.86:0/53768860 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f982c0a3bb0 0x7f982c0a6070 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:09.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.835+0000 7f984933d640 1 --2- 172.21.15.86:0/53768860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f98441029a0 0x7f984419f7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:09.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.835+0000 7f984933d640 1 --2- 172.21.15.86:0/53768860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9844101fe0 0x7f9844198780 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:09.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.835+0000 7f984933d640 1 --2- 172.21.15.86:0/53768860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9844100e90 0x7f9844198240 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:09.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.835+0000 7f984933d640 1 -- 172.21.15.86:0/53768860 >> 172.21.15.86:0/53768860 conn(0x7f98440fc400 msgr2=0x7f98440690e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:09.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.835+0000 7f984933d640 1 -- 172.21.15.86:0/53768860 shutdown_connections 2024-04-21T08:38:09.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.835+0000 7f984933d640 1 -- 172.21.15.86:0/53768860 wait complete. 2024-04-21T08:38:09.847 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:38:09.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.948+0000 7eff6e070640 1 -- 172.21.15.86:0/668191419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff681029b0 msgr2=0x7eff681092d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:09.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.948+0000 7eff6e070640 1 --2- 172.21.15.86:0/668191419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff681029b0 0x7eff681092d0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7eff60009ca0 tx=0x7eff6002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:09.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.948+0000 7eff6e070640 1 -- 172.21.15.86:0/668191419 shutdown_connections 2024-04-21T08:38:09.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.948+0000 7eff6e070640 1 --2- 172.21.15.86:0/668191419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff681029b0 0x7eff681092d0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:09.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.948+0000 7eff6e070640 1 --2- 172.21.15.86:0/668191419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff68101ff0 0x7eff68102470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:09.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.948+0000 7eff6e070640 1 --2- 172.21.15.86:0/668191419 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff68100ea0 0x7eff681012c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:09.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.948+0000 7eff6e070640 1 -- 172.21.15.86:0/668191419 >> 172.21.15.86:0/668191419 conn(0x7eff680fc410 msgr2=0x7eff680fe850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:09.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.948+0000 7eff6e070640 1 -- 172.21.15.86:0/668191419 shutdown_connections 2024-04-21T08:38:09.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.948+0000 7eff6e070640 1 -- 172.21.15.86:0/668191419 wait complete. 2024-04-21T08:38:09.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.948+0000 7eff6e070640 1 Processor -- start 2024-04-21T08:38:09.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.948+0000 7eff6e070640 1 -- start start 2024-04-21T08:38:09.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.949+0000 7eff6e070640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff68100ea0 0x7eff68198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:09.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.949+0000 7eff6e070640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff68101ff0 0x7eff681986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:09.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.949+0000 7eff6e070640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff681029b0 0x7eff6819f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:09.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.949+0000 7eff6e070640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7eff6810ca00 con 0x7eff68100ea0 2024-04-21T08:38:09.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.949+0000 7eff6e070640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7eff6810c880 con 0x7eff681029b0 2024-04-21T08:38:09.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.949+0000 7eff6e070640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7eff6810cb80 con 0x7eff68101ff0 2024-04-21T08:38:09.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.949+0000 7eff677fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff68100ea0 0x7eff68198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:09.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.949+0000 7eff677fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff68100ea0 0x7eff68198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36280/0 (socket says 172.21.15.86:36280) 2024-04-21T08:38:09.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.949+0000 7eff677fe640 1 -- 172.21.15.86:0/3524028294 learned_addr learned my addr 172.21.15.86:0/3524028294 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:09.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.949+0000 7eff67fff640 1 --2- 172.21.15.86:0/3524028294 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff681029b0 0x7eff6819f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:09.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.949+0000 7eff677fe640 1 -- 172.21.15.86:0/3524028294 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff68101ff0 msgr2=0x7eff681986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:09.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.949+0000 7eff66ffd640 1 --2- 172.21.15.86:0/3524028294 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff68101ff0 0x7eff681986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:09.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.949+0000 7eff677fe640 1 --2- 172.21.15.86:0/3524028294 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff68101ff0 0x7eff681986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:09.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.949+0000 7eff677fe640 1 -- 172.21.15.86:0/3524028294 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff681029b0 msgr2=0x7eff6819f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:09.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.949+0000 7eff677fe640 1 --2- 172.21.15.86:0/3524028294 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff681029b0 0x7eff6819f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:09.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.949+0000 7eff677fe640 1 -- 172.21.15.86:0/3524028294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7eff6819fd90 con 0x7eff68100ea0 2024-04-21T08:38:09.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.949+0000 7eff67fff640 1 --2- 172.21.15.86:0/3524028294 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff681029b0 0x7eff6819f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:09.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.949+0000 7eff677fe640 1 --2- 172.21.15.86:0/3524028294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff68100ea0 0x7eff68198170 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7eff5800d770 tx=0x7eff5800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:09.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.949+0000 7eff64ff9640 1 -- 172.21.15.86:0/3524028294 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff58017070 con 0x7eff68100ea0 2024-04-21T08:38:09.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.949+0000 7eff64ff9640 1 -- 172.21.15.86:0/3524028294 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7eff580054d0 con 0x7eff68100ea0 2024-04-21T08:38:09.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.949+0000 7eff6e070640 1 -- 172.21.15.86:0/3524028294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7eff681a0080 con 0x7eff68100ea0 2024-04-21T08:38:09.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.949+0000 7eff64ff9640 1 -- 172.21.15.86:0/3524028294 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7eff58004c30 con 0x7eff68100ea0 2024-04-21T08:38:09.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.949+0000 7eff6e070640 1 -- 172.21.15.86:0/3524028294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7eff681a0580 con 0x7eff68100ea0 2024-04-21T08:38:09.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.950+0000 7eff6e070640 1 -- 172.21.15.86:0/3524028294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7eff681a07e0 con 0x7eff68100ea0 2024-04-21T08:38:09.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.951+0000 7eff64ff9640 1 -- 172.21.15.86:0/3524028294 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7eff58004dd0 con 0x7eff68100ea0 2024-04-21T08:38:09.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.952+0000 7eff64ff9640 1 --2- 172.21.15.86:0/3524028294 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7eff500a3bb0 0x7eff500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:09.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.952+0000 7eff66ffd640 1 --2- 172.21.15.86:0/3524028294 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7eff500a3bb0 0x7eff500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:09.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.952+0000 7eff64ff9640 1 -- 172.21.15.86:0/3524028294 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7eff580d0710 con 0x7eff68100ea0 2024-04-21T08:38:09.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.952+0000 7eff66ffd640 1 --2- 172.21.15.86:0/3524028294 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7eff500a3bb0 0x7eff500a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7eff68199680 tx=0x7eff5c005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:09.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:09.956+0000 7eff64ff9640 1 -- 172.21.15.86:0/3524028294 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7eff5809cd90 con 0x7eff68100ea0 2024-04-21T08:38:10.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.133+0000 7eff6e070640 1 -- 172.21.15.86:0/3524028294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7eff681a09b0 con 0x7eff68100ea0 2024-04-21T08:38:10.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.133+0000 7eff64ff9640 1 -- 172.21.15.86:0/3524028294 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v91) ==== 100+0+11 (secure 0 0 0) 0x7eff5809c690 con 0x7eff68100ea0 2024-04-21T08:38:10.134 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-21T08:38:10.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.135+0000 7eff6e070640 1 -- 172.21.15.86:0/3524028294 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7eff500a3bb0 msgr2=0x7eff500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:10.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.135+0000 7eff6e070640 1 --2- 172.21.15.86:0/3524028294 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7eff500a3bb0 0x7eff500a6070 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7eff68199680 tx=0x7eff5c005c60 comp rx=0 tx=0).stop 2024-04-21T08:38:10.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.135+0000 7eff6e070640 1 -- 172.21.15.86:0/3524028294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff68100ea0 msgr2=0x7eff68198170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:10.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.135+0000 7eff6e070640 1 --2- 172.21.15.86:0/3524028294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff68100ea0 0x7eff68198170 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7eff5800d770 tx=0x7eff5800dc30 comp rx=0 tx=0).stop 2024-04-21T08:38:10.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.135+0000 7eff6e070640 1 -- 172.21.15.86:0/3524028294 shutdown_connections 2024-04-21T08:38:10.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.135+0000 7eff6e070640 1 --2- 172.21.15.86:0/3524028294 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7eff500a3bb0 0x7eff500a6070 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:10.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.135+0000 7eff6e070640 1 --2- 172.21.15.86:0/3524028294 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7eff681029b0 0x7eff6819f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:10.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.135+0000 7eff6e070640 1 --2- 172.21.15.86:0/3524028294 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7eff68101ff0 0x7eff681986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:10.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.135+0000 7eff6e070640 1 --2- 172.21.15.86:0/3524028294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7eff68100ea0 0x7eff68198170 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:10.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.135+0000 7eff6e070640 1 -- 172.21.15.86:0/3524028294 >> 172.21.15.86:0/3524028294 conn(0x7eff680fc410 msgr2=0x7eff6806a570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:10.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.135+0000 7eff6e070640 1 -- 172.21.15.86:0/3524028294 shutdown_connections 2024-04-21T08:38:10.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.136+0000 7eff6e070640 1 -- 172.21.15.86:0/3524028294 wait complete. 2024-04-21T08:38:10.147 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:38:10.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.248+0000 7fab2c146640 1 -- 172.21.15.86:0/2590374677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab24100e90 msgr2=0x7fab241012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:10.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.248+0000 7fab2c146640 1 --2- 172.21.15.86:0/2590374677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab24100e90 0x7fab241012b0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fab14009900 tx=0x7fab1402bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:10.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.248+0000 7fab2c146640 1 -- 172.21.15.86:0/2590374677 shutdown_connections 2024-04-21T08:38:10.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.248+0000 7fab2c146640 1 --2- 172.21.15.86:0/2590374677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab241029a0 0x7fab241092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:10.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.248+0000 7fab2c146640 1 --2- 172.21.15.86:0/2590374677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab24101fe0 0x7fab24102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:10.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.248+0000 7fab2c146640 1 --2- 172.21.15.86:0/2590374677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab24100e90 0x7fab241012b0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:10.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.248+0000 7fab2c146640 1 -- 172.21.15.86:0/2590374677 >> 172.21.15.86:0/2590374677 conn(0x7fab240fc400 msgr2=0x7fab240fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:10.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.248+0000 7fab2c146640 1 -- 172.21.15.86:0/2590374677 shutdown_connections 2024-04-21T08:38:10.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.248+0000 7fab2c146640 1 -- 172.21.15.86:0/2590374677 wait complete. 2024-04-21T08:38:10.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.248+0000 7fab2c146640 1 Processor -- start 2024-04-21T08:38:10.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.248+0000 7fab2c146640 1 -- start start 2024-04-21T08:38:10.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.249+0000 7fab2c146640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab24100e90 0x7fab24198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:10.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.249+0000 7fab2c146640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab24101fe0 0x7fab24198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:10.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.249+0000 7fab2c146640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab241029a0 0x7fab2419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:10.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.249+0000 7fab2c146640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fab2410db30 con 0x7fab241029a0 2024-04-21T08:38:10.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.249+0000 7fab2c146640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fab2410d9b0 con 0x7fab24100e90 2024-04-21T08:38:10.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.249+0000 7fab2c146640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fab2410dcb0 con 0x7fab24101fe0 2024-04-21T08:38:10.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.249+0000 7fab2a6e3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab241029a0 0x7fab2419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:10.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.249+0000 7fab2a6e3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab241029a0 0x7fab2419f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36290/0 (socket says 172.21.15.86:36290) 2024-04-21T08:38:10.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.249+0000 7fab2a6e3640 1 -- 172.21.15.86:0/4143416052 learned_addr learned my addr 172.21.15.86:0/4143416052 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:10.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.249+0000 7fab296e1640 1 --2- 172.21.15.86:0/4143416052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab24101fe0 0x7fab24198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:10.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.249+0000 7fab2a6e3640 1 -- 172.21.15.86:0/4143416052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab24101fe0 msgr2=0x7fab24198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:10.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.249+0000 7fab2a6e3640 1 --2- 172.21.15.86:0/4143416052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab24101fe0 0x7fab24198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:10.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.249+0000 7fab2a6e3640 1 -- 172.21.15.86:0/4143416052 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab24100e90 msgr2=0x7fab24198220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:10.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.249+0000 7fab29ee2640 1 --2- 172.21.15.86:0/4143416052 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab24100e90 0x7fab24198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:10.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.249+0000 7fab2a6e3640 1 --2- 172.21.15.86:0/4143416052 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab24100e90 0x7fab24198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:10.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.249+0000 7fab2a6e3640 1 -- 172.21.15.86:0/4143416052 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fab2419fe40 con 0x7fab241029a0 2024-04-21T08:38:10.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.249+0000 7fab2a6e3640 1 --2- 172.21.15.86:0/4143416052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab241029a0 0x7fab2419f7d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00eaa0 tx=0x7fab1c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:10.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.249+0000 7fab1affd640 1 -- 172.21.15.86:0/4143416052 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c00cdb0 con 0x7fab241029a0 2024-04-21T08:38:10.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.249+0000 7fab2c146640 1 -- 172.21.15.86:0/4143416052 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fab241a0130 con 0x7fab241029a0 2024-04-21T08:38:10.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.249+0000 7fab1affd640 1 -- 172.21.15.86:0/4143416052 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab1c005020 con 0x7fab241029a0 2024-04-21T08:38:10.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.249+0000 7fab1affd640 1 -- 172.21.15.86:0/4143416052 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab1c0116d0 con 0x7fab241029a0 2024-04-21T08:38:10.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.249+0000 7fab2c146640 1 -- 172.21.15.86:0/4143416052 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fab241a0680 con 0x7fab241029a0 2024-04-21T08:38:10.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.250+0000 7fab2c146640 1 -- 172.21.15.86:0/4143416052 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fab24108210 con 0x7fab241029a0 2024-04-21T08:38:10.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.251+0000 7fab1affd640 1 -- 172.21.15.86:0/4143416052 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fab1c0051d0 con 0x7fab241029a0 2024-04-21T08:38:10.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.252+0000 7fab1affd640 1 --2- 172.21.15.86:0/4143416052 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fab100a3bb0 0x7fab100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:10.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.252+0000 7fab29ee2640 1 --2- 172.21.15.86:0/4143416052 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fab100a3bb0 0x7fab100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:10.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.252+0000 7fab1affd640 1 -- 172.21.15.86:0/4143416052 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fab1c0d09c0 con 0x7fab241029a0 2024-04-21T08:38:10.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.252+0000 7fab29ee2640 1 --2- 172.21.15.86:0/4143416052 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fab100a3bb0 0x7fab100a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fab140096c0 tx=0x7fab14000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:10.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.257+0000 7fab1affd640 1 -- 172.21.15.86:0/4143416052 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fab1c0d2e20 con 0x7fab241029a0 2024-04-21T08:38:10.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.436+0000 7fab2c146640 1 -- 172.21.15.86:0/4143416052 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fab24062b50 con 0x7fab241029a0 2024-04-21T08:38:10.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.436+0000 7fab1affd640 1 -- 172.21.15.86:0/4143416052 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v91) ==== 103+0+11 (secure 0 0 0) 0x7fab1c016030 con 0x7fab241029a0 2024-04-21T08:38:10.436 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-21T08:38:10.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.438+0000 7fab2c146640 1 -- 172.21.15.86:0/4143416052 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fab100a3bb0 msgr2=0x7fab100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:10.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.438+0000 7fab2c146640 1 --2- 172.21.15.86:0/4143416052 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fab100a3bb0 0x7fab100a6070 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fab140096c0 tx=0x7fab14000bc0 comp rx=0 tx=0).stop 2024-04-21T08:38:10.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.438+0000 7fab2c146640 1 -- 172.21.15.86:0/4143416052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab241029a0 msgr2=0x7fab2419f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:10.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.438+0000 7fab2c146640 1 --2- 172.21.15.86:0/4143416052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab241029a0 0x7fab2419f7d0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7fab1c00eaa0 tx=0x7fab1c00ef60 comp rx=0 tx=0).stop 2024-04-21T08:38:10.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.438+0000 7fab2c146640 1 -- 172.21.15.86:0/4143416052 shutdown_connections 2024-04-21T08:38:10.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.438+0000 7fab2c146640 1 --2- 172.21.15.86:0/4143416052 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fab100a3bb0 0x7fab100a6070 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:10.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.438+0000 7fab2c146640 1 --2- 172.21.15.86:0/4143416052 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fab241029a0 0x7fab2419f7d0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:10.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.438+0000 7fab2c146640 1 --2- 172.21.15.86:0/4143416052 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fab24101fe0 0x7fab24198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:10.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.438+0000 7fab2c146640 1 --2- 172.21.15.86:0/4143416052 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fab24100e90 0x7fab24198220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:10.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.438+0000 7fab2c146640 1 -- 172.21.15.86:0/4143416052 >> 172.21.15.86:0/4143416052 conn(0x7fab240fc400 msgr2=0x7fab241053e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:10.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.438+0000 7fab2c146640 1 -- 172.21.15.86:0/4143416052 shutdown_connections 2024-04-21T08:38:10.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.438+0000 7fab2c146640 1 -- 172.21.15.86:0/4143416052 wait complete. 2024-04-21T08:38:10.450 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:38:10.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.548+0000 7f4f5b504640 1 -- 172.21.15.86:0/1413611607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f54107cc0 msgr2=0x7f4f5410a0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:10.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.548+0000 7f4f5b504640 1 --2- 172.21.15.86:0/1413611607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f54107cc0 0x7f4f5410a0b0 secure :-1 s=READY pgs=376 cs=0 l=1 rev1=1 crypto rx=0x7f4f4c009ca0 tx=0x7f4f4c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:10.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.548+0000 7f4f5b504640 1 -- 172.21.15.86:0/1413611607 shutdown_connections 2024-04-21T08:38:10.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.548+0000 7f4f5b504640 1 --2- 172.21.15.86:0/1413611607 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f54107cc0 0x7f4f5410a0b0 unknown :-1 s=CLOSED pgs=376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:10.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.548+0000 7f4f5b504640 1 --2- 172.21.15.86:0/1413611607 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f54105390 0x7f4f54107780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:10.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.548+0000 7f4f5b504640 1 --2- 172.21.15.86:0/1413611607 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f54102a60 0x7f4f54104e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:10.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.548+0000 7f4f5b504640 1 -- 172.21.15.86:0/1413611607 >> 172.21.15.86:0/1413611607 conn(0x7f4f540fc410 msgr2=0x7f4f540fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:10.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.548+0000 7f4f5b504640 1 -- 172.21.15.86:0/1413611607 shutdown_connections 2024-04-21T08:38:10.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.548+0000 7f4f5b504640 1 -- 172.21.15.86:0/1413611607 wait complete. 2024-04-21T08:38:10.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.548+0000 7f4f5b504640 1 Processor -- start 2024-04-21T08:38:10.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.549+0000 7f4f5b504640 1 -- start start 2024-04-21T08:38:10.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.549+0000 7f4f5b504640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f54102a60 0x7f4f541981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:10.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.549+0000 7f4f5b504640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f54105390 0x7f4f54198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:10.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.549+0000 7f4f5b504640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f54107cc0 0x7f4f5419f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:10.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.549+0000 7f4f5b504640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5410db30 con 0x7f4f54105390 2024-04-21T08:38:10.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.549+0000 7f4f5b504640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5410d9b0 con 0x7f4f54102a60 2024-04-21T08:38:10.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.549+0000 7f4f5b504640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4f5410dcb0 con 0x7f4f54107cc0 2024-04-21T08:38:10.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.549+0000 7f4f58a9f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f54105390 0x7f4f54198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:10.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.549+0000 7f4f58a9f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f54105390 0x7f4f54198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36302/0 (socket says 172.21.15.86:36302) 2024-04-21T08:38:10.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.549+0000 7f4f58a9f640 1 -- 172.21.15.86:0/450586032 learned_addr learned my addr 172.21.15.86:0/450586032 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:10.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.549+0000 7f4f59aa1640 1 --2- 172.21.15.86:0/450586032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f54107cc0 0x7f4f5419f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:10.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.549+0000 7f4f58a9f640 1 -- 172.21.15.86:0/450586032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f54107cc0 msgr2=0x7f4f5419f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:10.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.549+0000 7f4f58a9f640 1 --2- 172.21.15.86:0/450586032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f54107cc0 0x7f4f5419f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:10.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.549+0000 7f4f58a9f640 1 -- 172.21.15.86:0/450586032 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f54102a60 msgr2=0x7f4f541981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:10.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.549+0000 7f4f58a9f640 1 --2- 172.21.15.86:0/450586032 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f54102a60 0x7f4f541981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:10.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.549+0000 7f4f58a9f640 1 -- 172.21.15.86:0/450586032 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f5419fe00 con 0x7f4f54105390 2024-04-21T08:38:10.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.549+0000 7f4f59aa1640 1 --2- 172.21.15.86:0/450586032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f54107cc0 0x7f4f5419f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:10.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.549+0000 7f4f58a9f640 1 --2- 172.21.15.86:0/450586032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f54105390 0x7f4f54198720 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4f5000c970 tx=0x7f4f5000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:10.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.549+0000 7f4f4a7fc640 1 -- 172.21.15.86:0/450586032 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f50007bb0 con 0x7f4f54105390 2024-04-21T08:38:10.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.549+0000 7f4f4a7fc640 1 -- 172.21.15.86:0/450586032 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f50007d50 con 0x7f4f54105390 2024-04-21T08:38:10.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.549+0000 7f4f5b504640 1 -- 172.21.15.86:0/450586032 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f541a00f0 con 0x7f4f54105390 2024-04-21T08:38:10.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.549+0000 7f4f4a7fc640 1 -- 172.21.15.86:0/450586032 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f5000f450 con 0x7f4f54105390 2024-04-21T08:38:10.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.549+0000 7f4f5b504640 1 -- 172.21.15.86:0/450586032 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f541a0640 con 0x7f4f54105390 2024-04-21T08:38:10.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.550+0000 7f4f5b504640 1 -- 172.21.15.86:0/450586032 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4f540fdf00 con 0x7f4f54105390 2024-04-21T08:38:10.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.551+0000 7f4f4a7fc640 1 -- 172.21.15.86:0/450586032 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4f50004f20 con 0x7f4f54105390 2024-04-21T08:38:10.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.552+0000 7f4f4a7fc640 1 --2- 172.21.15.86:0/450586032 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f4f400a3bb0 0x7f4f400a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:10.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.552+0000 7f4f592a0640 1 --2- 172.21.15.86:0/450586032 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f4f400a3bb0 0x7f4f400a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:10.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.552+0000 7f4f4a7fc640 1 -- 172.21.15.86:0/450586032 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f4f500d0830 con 0x7f4f54105390 2024-04-21T08:38:10.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.552+0000 7f4f592a0640 1 --2- 172.21.15.86:0/450586032 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f4f400a3bb0 0x7f4f400a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4f44009710 tx=0x7f4f44006c90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:10.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.557+0000 7f4f4a7fc640 1 -- 172.21.15.86:0/450586032 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4f50014030 con 0x7f4f54105390 2024-04-21T08:38:10.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:10 smithi088 ceph-mon[31259]: pgmap v344: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-21T08:38:10.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:10 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1406406385' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:10.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:10 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/53768860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:10.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:10 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3524028294' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:10.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:10 smithi179 ceph-mon[31829]: pgmap v344: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-21T08:38:10.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:10 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1406406385' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:10.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:10 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/53768860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:10.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:10 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3524028294' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:10.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.774+0000 7f4f5b504640 1 -- 172.21.15.86:0/450586032 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4f5402d5c0 con 0x7f4f54105390 2024-04-21T08:38:10.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.775+0000 7f4f4a7fc640 1 -- 172.21.15.86:0/450586032 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7f4f5009ceb0 con 0x7f4f54105390 2024-04-21T08:38:10.775 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:38:10.775 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:36:47.458637+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24455,24472,24394,24398,24482]},"id":1}]} 2024-04-21T08:38:10.775 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-21T08:38:10.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.777+0000 7f4f5b504640 1 -- 172.21.15.86:0/450586032 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f4f400a3bb0 msgr2=0x7f4f400a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:10.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.777+0000 7f4f5b504640 1 --2- 172.21.15.86:0/450586032 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f4f400a3bb0 0x7f4f400a6070 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7f4f44009710 tx=0x7f4f44006c90 comp rx=0 tx=0).stop 2024-04-21T08:38:10.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.777+0000 7f4f5b504640 1 -- 172.21.15.86:0/450586032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f54105390 msgr2=0x7f4f54198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:10.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.777+0000 7f4f5b504640 1 --2- 172.21.15.86:0/450586032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f54105390 0x7f4f54198720 secure :-1 s=READY pgs=377 cs=0 l=1 rev1=1 crypto rx=0x7f4f5000c970 tx=0x7f4f5000ce30 comp rx=0 tx=0).stop 2024-04-21T08:38:10.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.777+0000 7f4f5b504640 1 -- 172.21.15.86:0/450586032 shutdown_connections 2024-04-21T08:38:10.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.777+0000 7f4f5b504640 1 --2- 172.21.15.86:0/450586032 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f4f400a3bb0 0x7f4f400a6070 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:10.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.777+0000 7f4f5b504640 1 --2- 172.21.15.86:0/450586032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f54107cc0 0x7f4f5419f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:10.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.777+0000 7f4f5b504640 1 --2- 172.21.15.86:0/450586032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f54105390 0x7f4f54198720 unknown :-1 s=CLOSED pgs=377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:10.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.777+0000 7f4f5b504640 1 --2- 172.21.15.86:0/450586032 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f54102a60 0x7f4f541981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:10.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.777+0000 7f4f5b504640 1 -- 172.21.15.86:0/450586032 >> 172.21.15.86:0/450586032 conn(0x7f4f540fc410 msgr2=0x7f4f54103300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:10.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.777+0000 7f4f5b504640 1 -- 172.21.15.86:0/450586032 shutdown_connections 2024-04-21T08:38:10.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.777+0000 7f4f5b504640 1 -- 172.21.15.86:0/450586032 wait complete. 2024-04-21T08:38:10.787 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:10 smithi086 ceph-mon[27939]: pgmap v344: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 170 B/s wr, 10 op/s 2024-04-21T08:38:10.787 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:10 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1406406385' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:10.787 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:10 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/53768860' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:10.787 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:10 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3524028294' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:10.788 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:38:10.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.885+0000 7fc499d06640 1 -- 172.21.15.86:0/2693469553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc494101b50 msgr2=0x7fc494101fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:10.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.885+0000 7fc499d06640 1 --2- 172.21.15.86:0/2693469553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc494101b50 0x7fc494101fd0 secure :-1 s=READY pgs=378 cs=0 l=1 rev1=1 crypto rx=0x7fc48c009900 tx=0x7fc48c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:10.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.885+0000 7fc499d06640 1 -- 172.21.15.86:0/2693469553 shutdown_connections 2024-04-21T08:38:10.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.885+0000 7fc499d06640 1 --2- 172.21.15.86:0/2693469553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc494101b50 0x7fc494101fd0 unknown :-1 s=CLOSED pgs=378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:10.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.885+0000 7fc499d06640 1 --2- 172.21.15.86:0/2693469553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4941011b0 0x7fc494101610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:10.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.885+0000 7fc499d06640 1 --2- 172.21.15.86:0/2693469553 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc494100850 0x7fc494100c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:10.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.886+0000 7fc499d06640 1 -- 172.21.15.86:0/2693469553 >> 172.21.15.86:0/2693469553 conn(0x7fc4940fc410 msgr2=0x7fc4940fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:10.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.886+0000 7fc499d06640 1 -- 172.21.15.86:0/2693469553 shutdown_connections 2024-04-21T08:38:10.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.886+0000 7fc499d06640 1 -- 172.21.15.86:0/2693469553 wait complete. 2024-04-21T08:38:10.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.886+0000 7fc499d06640 1 Processor -- start 2024-04-21T08:38:10.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.886+0000 7fc499d06640 1 -- start start 2024-04-21T08:38:10.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.886+0000 7fc499d06640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc494100850 0x7fc49419a360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:10.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.886+0000 7fc499d06640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4941011b0 0x7fc49419a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:10.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.886+0000 7fc499d06640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc494101b50 0x7fc4941a1910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:10.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.886+0000 7fc499d06640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc49410fcd0 con 0x7fc4941011b0 2024-04-21T08:38:10.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.886+0000 7fc499d06640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc49410fb50 con 0x7fc494100850 2024-04-21T08:38:10.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.886+0000 7fc499d06640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc49410fe50 con 0x7fc494101b50 2024-04-21T08:38:10.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.887+0000 7fc492ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4941011b0 0x7fc49419a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:10.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.887+0000 7fc492ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4941011b0 0x7fc49419a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36324/0 (socket says 172.21.15.86:36324) 2024-04-21T08:38:10.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.887+0000 7fc492ffd640 1 -- 172.21.15.86:0/4092817116 learned_addr learned my addr 172.21.15.86:0/4092817116 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:10.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.887+0000 7fc493fff640 1 --2- 172.21.15.86:0/4092817116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc494101b50 0x7fc4941a1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:10.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.887+0000 7fc4937fe640 1 --2- 172.21.15.86:0/4092817116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc494100850 0x7fc49419a360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:10.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.887+0000 7fc492ffd640 1 -- 172.21.15.86:0/4092817116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc494101b50 msgr2=0x7fc4941a1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:10.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.887+0000 7fc492ffd640 1 --2- 172.21.15.86:0/4092817116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc494101b50 0x7fc4941a1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:10.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.887+0000 7fc492ffd640 1 -- 172.21.15.86:0/4092817116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc494100850 msgr2=0x7fc49419a360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:10.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.887+0000 7fc492ffd640 1 --2- 172.21.15.86:0/4092817116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc494100850 0x7fc49419a360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:10.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.887+0000 7fc492ffd640 1 -- 172.21.15.86:0/4092817116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc4941a1f80 con 0x7fc4941011b0 2024-04-21T08:38:10.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.887+0000 7fc492ffd640 1 --2- 172.21.15.86:0/4092817116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4941011b0 0x7fc49419a8a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc48800d770 tx=0x7fc48800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:10.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.887+0000 7fc490ff9640 1 -- 172.21.15.86:0/4092817116 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc488017070 con 0x7fc4941011b0 2024-04-21T08:38:10.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.887+0000 7fc499d06640 1 -- 172.21.15.86:0/4092817116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc4941a2270 con 0x7fc4941011b0 2024-04-21T08:38:10.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.887+0000 7fc490ff9640 1 -- 172.21.15.86:0/4092817116 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc4880054d0 con 0x7fc4941011b0 2024-04-21T08:38:10.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.887+0000 7fc490ff9640 1 -- 172.21.15.86:0/4092817116 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc488004c30 con 0x7fc4941011b0 2024-04-21T08:38:10.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.887+0000 7fc499d06640 1 -- 172.21.15.86:0/4092817116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4941a2770 con 0x7fc4941011b0 2024-04-21T08:38:10.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.888+0000 7fc499d06640 1 -- 172.21.15.86:0/4092817116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc49410b7a0 con 0x7fc4941011b0 2024-04-21T08:38:10.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.889+0000 7fc490ff9640 1 -- 172.21.15.86:0/4092817116 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc488004e10 con 0x7fc4941011b0 2024-04-21T08:38:10.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.890+0000 7fc490ff9640 1 --2- 172.21.15.86:0/4092817116 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fc47c0a3bb0 0x7fc47c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:10.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.890+0000 7fc490ff9640 1 -- 172.21.15.86:0/4092817116 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fc4880d0220 con 0x7fc4941011b0 2024-04-21T08:38:10.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.890+0000 7fc4937fe640 1 --2- 172.21.15.86:0/4092817116 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fc47c0a3bb0 0x7fc47c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:10.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.890+0000 7fc4937fe640 1 --2- 172.21.15.86:0/4092817116 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fc47c0a3bb0 0x7fc47c0a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc484005fd0 tx=0x7fc484009870 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:10.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:10.894+0000 7fc490ff9640 1 -- 172.21.15.86:0/4092817116 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc48809c8a0 con 0x7fc4941011b0 2024-04-21T08:38:11.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.073+0000 7fc499d06640 1 -- 172.21.15.86:0/4092817116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fc494062b50 con 0x7fc4941011b0 2024-04-21T08:38:11.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.075+0000 7fc490ff9640 1 -- 172.21.15.86:0/4092817116 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v91) ==== 74+0+24595 (secure 0 0 0) 0x7fc48809c1a0 con 0x7fc4941011b0 2024-04-21T08:38:11.075 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:38:11.075 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":91,"fsid":"f8cec7cc-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:22.687329+0000","modified":"2024-04-21T08:37:24.279025+0000","last_up_change":"2024-04-21T08:35:30.048315+0000","last_in_change":"2024-04-21T08:35:10.099496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:45.279109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:18.300782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:19.574636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:22.905632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89b7601f-0620-4db2-90f1-b45cea173738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6803","nonce":2571516897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6805","nonce":2571516897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6809","nonce":2571516897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6807","nonce":2571516897}]},"public_addr":"172.21.15.86:6803/2571516897","cluster_addr":"172.21.15.86:6805/2571516897","heartbeat_back_addr":"172.21.15.86:6809/2571516897","heartbeat_front_addr":"172.21.15.86:6807/2571516897","state":["exists","up"]},{"osd":1,"uuid":"4891031c-f98c-42bc-bb3b-a420855ecc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6801","nonce":74067182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6803","nonce":74067182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6807","nonce":74067182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6805","nonce":74067182}]},"public_addr":"172.21.15.88:6801/74067182","cluster_addr":"172.21.15.88:6803/74067182","heartbeat_back_addr":"172.21.15.88:6807/74067182","heartbeat_front_addr":"172.21.15.88:6805/74067182","state":["exists","up"]},{"osd":2,"uuid":"60d12aa4-9312-4fa2-8d0b-2b3f42caad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6801","nonce":4137659051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6803","nonce":4137659051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6807","nonce":4137659051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6805","nonce":4137659051}]},"public_addr":"172.21.15.179:6801/4137659051","cluster_addr":"172.21.15.179:6803/4137659051","heartbeat_back_addr":"172.21.15.179:6807/4137659051","heartbeat_front_addr":"172.21.15.179:6805/4137659051","state":["exists","up"]},{"osd":3,"uuid":"404c3158-8cb0-4c02-b48a-f9ee5ff7b784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6811","nonce":2309603148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6813","nonce":2309603148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6817","nonce":2309603148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6815","nonce":2309603148}]},"public_addr":"172.21.15.86:6811/2309603148","cluster_addr":"172.21.15.86:6813/2309603148","heartbeat_back_addr":"172.21.15.86:6817/2309603148","heartbeat_front_addr":"172.21.15.86:6815/2309603148","state":["exists","up"]},{"osd":4,"uuid":"458ff202-d781-4ec7-ac93-0b49f835b503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6809","nonce":1818714217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6811","nonce":1818714217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6815","nonce":1818714217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6813","nonce":1818714217}]},"public_addr":"172.21.15.88:6809/1818714217","cluster_addr":"172.21.15.88:6811/1818714217","heartbeat_back_addr":"172.21.15.88:6815/1818714217","heartbeat_front_addr":"172.21.15.88:6813/1818714217","state":["exists","up"]},{"osd":5,"uuid":"8123cc90-bd8b-4204-bf31-46b34e9f589d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6809","nonce":863840990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6811","nonce":863840990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6815","nonce":863840990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6813","nonce":863840990}]},"public_addr":"172.21.15.179:6809/863840990","cluster_addr":"172.21.15.179:6811/863840990","heartbeat_back_addr":"172.21.15.179:6815/863840990","heartbeat_front_addr":"172.21.15.179:6813/863840990","state":["exists","up"]},{"osd":6,"uuid":"aac9d7c7-659a-4110-903a-a871c086066f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6819","nonce":1224231118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6821","nonce":1224231118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6825","nonce":1224231118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6823","nonce":1224231118}]},"public_addr":"172.21.15.86:6819/1224231118","cluster_addr":"172.21.15.86:6821/1224231118","heartbeat_back_addr":"172.21.15.86:6825/1224231118","heartbeat_front_addr":"172.21.15.86:6823/1224231118","state":["exists","up"]},{"osd":7,"uuid":"df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6817","nonce":136629878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6819","nonce":136629878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6823","nonce":136629878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6821","nonce":136629878}]},"public_addr":"172.21.15.88:6817/136629878","cluster_addr":"172.21.15.88:6819/136629878","heartbeat_back_addr":"172.21.15.88:6823/136629878","heartbeat_front_addr":"172.21.15.88:6821/136629878","state":["exists","up"]},{"osd":8,"uuid":"63e59f4b-0426-46db-92f7-c55dae61b5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6817","nonce":1770418957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6819","nonce":1770418957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6823","nonce":1770418957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6821","nonce":1770418957}]},"public_addr":"172.21.15.179:6817/1770418957","cluster_addr":"172.21.15.179:6819/1770418957","heartbeat_back_addr":"172.21.15.179:6823/1770418957","heartbeat_front_addr":"172.21.15.179:6821/1770418957","state":["exists","up"]},{"osd":9,"uuid":"5841bb36-ed6b-45d8-b857-eeab139c44e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6827","nonce":1108500274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6829","nonce":1108500274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6833","nonce":1108500274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6831","nonce":1108500274}]},"public_addr":"172.21.15.86:6827/1108500274","cluster_addr":"172.21.15.86:6829/1108500274","heartbeat_back_addr":"172.21.15.86:6833/1108500274","heartbeat_front_addr":"172.21.15.86:6831/1108500274","state":["exists","up"]},{"osd":10,"uuid":"f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6825","nonce":1294275353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6827","nonce":1294275353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6831","nonce":1294275353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6829","nonce":1294275353}]},"public_addr":"172.21.15.88:6825/1294275353","cluster_addr":"172.21.15.88:6827/1294275353","heartbeat_back_addr":"172.21.15.88:6831/1294275353","heartbeat_front_addr":"172.21.15.88:6829/1294275353","state":["exists","up"]},{"osd":11,"uuid":"e0d4cc76-e7a6-4a6b-909c-95488ccd5a36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6825","nonce":4007070233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6827","nonce":4007070233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6831","nonce":4007070233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6829","nonce":4007070233}]},"public_addr":"172.21.15.179:6825/4007070233","cluster_addr":"172.21.15.179:6827/4007070233","heartbeat_back_addr":"172.21.15.179:6831/4007070233","heartbeat_front_addr":"172.21.15.179:6829/4007070233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:55.210605+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:18.723090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:42.547979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:06.421622+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:30.858315+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:55.455313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.623369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:46.185111+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.103627+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:37.297545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:02.637431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:28.199292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6800/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/1412393225":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6800/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/1687852404":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2020707404":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2568130933":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/3952607648":"2024-04-22T08:28:57.384403+0000","172.21.15.86:6801/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/2756903426":"2024-04-22T08:29:23.190001+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:38:11.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.077+0000 7fc499d06640 1 -- 172.21.15.86:0/4092817116 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fc47c0a3bb0 msgr2=0x7fc47c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.077+0000 7fc499d06640 1 --2- 172.21.15.86:0/4092817116 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fc47c0a3bb0 0x7fc47c0a6070 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fc484005fd0 tx=0x7fc484009870 comp rx=0 tx=0).stop 2024-04-21T08:38:11.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.077+0000 7fc499d06640 1 -- 172.21.15.86:0/4092817116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4941011b0 msgr2=0x7fc49419a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.077+0000 7fc499d06640 1 --2- 172.21.15.86:0/4092817116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4941011b0 0x7fc49419a8a0 secure :-1 s=READY pgs=379 cs=0 l=1 rev1=1 crypto rx=0x7fc48800d770 tx=0x7fc48800dc30 comp rx=0 tx=0).stop 2024-04-21T08:38:11.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.077+0000 7fc499d06640 1 -- 172.21.15.86:0/4092817116 shutdown_connections 2024-04-21T08:38:11.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.077+0000 7fc499d06640 1 --2- 172.21.15.86:0/4092817116 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fc47c0a3bb0 0x7fc47c0a6070 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.077+0000 7fc499d06640 1 --2- 172.21.15.86:0/4092817116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc494101b50 0x7fc4941a1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.077+0000 7fc499d06640 1 --2- 172.21.15.86:0/4092817116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc4941011b0 0x7fc49419a8a0 unknown :-1 s=CLOSED pgs=379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.077+0000 7fc499d06640 1 --2- 172.21.15.86:0/4092817116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc494100850 0x7fc49419a360 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.077+0000 7fc499d06640 1 -- 172.21.15.86:0/4092817116 >> 172.21.15.86:0/4092817116 conn(0x7fc4940fc410 msgr2=0x7fc4940fdb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:11.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.077+0000 7fc499d06640 1 -- 172.21.15.86:0/4092817116 shutdown_connections 2024-04-21T08:38:11.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.077+0000 7fc499d06640 1 -- 172.21.15.86:0/4092817116 wait complete. 2024-04-21T08:38:11.088 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:38:11.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.185+0000 7fb348952640 1 -- 172.21.15.86:0/3662391170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3441053a0 msgr2=0x7fb344105820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.185+0000 7fb348952640 1 --2- 172.21.15.86:0/3662391170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3441053a0 0x7fb344105820 secure :-1 s=READY pgs=380 cs=0 l=1 rev1=1 crypto rx=0x7fb338009900 tx=0x7fb33802bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:11.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.185+0000 7fb348952640 1 -- 172.21.15.86:0/3662391170 shutdown_connections 2024-04-21T08:38:11.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.185+0000 7fb348952640 1 --2- 172.21.15.86:0/3662391170 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb344105d60 0x7fb34410c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.185+0000 7fb348952640 1 --2- 172.21.15.86:0/3662391170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3441053a0 0x7fb344105820 unknown :-1 s=CLOSED pgs=380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.185+0000 7fb348952640 1 --2- 172.21.15.86:0/3662391170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3441041b0 0x7fb3441045d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.185+0000 7fb348952640 1 -- 172.21.15.86:0/3662391170 >> 172.21.15.86:0/3662391170 conn(0x7fb3440ff740 msgr2=0x7fb344101b60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:11.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.185+0000 7fb348952640 1 -- 172.21.15.86:0/3662391170 shutdown_connections 2024-04-21T08:38:11.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.185+0000 7fb348952640 1 -- 172.21.15.86:0/3662391170 wait complete. 2024-04-21T08:38:11.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.186+0000 7fb348952640 1 Processor -- start 2024-04-21T08:38:11.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.186+0000 7fb348952640 1 -- start start 2024-04-21T08:38:11.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.186+0000 7fb348952640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3441041b0 0x7fb34419a320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.186+0000 7fb348952640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3441053a0 0x7fb34419a860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.186+0000 7fb348952640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb344105d60 0x7fb3441a18d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.186+0000 7fb348952640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb34410fcf0 con 0x7fb344105d60 2024-04-21T08:38:11.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.186+0000 7fb348952640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb34410fb70 con 0x7fb3441053a0 2024-04-21T08:38:11.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.186+0000 7fb348952640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb34410fe70 con 0x7fb3441041b0 2024-04-21T08:38:11.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.186+0000 7fb342d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb344105d60 0x7fb3441a18d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.186+0000 7fb342d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb344105d60 0x7fb3441a18d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36338/0 (socket says 172.21.15.86:36338) 2024-04-21T08:38:11.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.186+0000 7fb342d9d640 1 -- 172.21.15.86:0/1456025873 learned_addr learned my addr 172.21.15.86:0/1456025873 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:11.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.187+0000 7fb341d9b640 1 --2- 172.21.15.86:0/1456025873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3441053a0 0x7fb34419a860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.187+0000 7fb34259c640 1 --2- 172.21.15.86:0/1456025873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3441041b0 0x7fb34419a320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.187+0000 7fb342d9d640 1 -- 172.21.15.86:0/1456025873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3441041b0 msgr2=0x7fb34419a320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.187+0000 7fb342d9d640 1 --2- 172.21.15.86:0/1456025873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3441041b0 0x7fb34419a320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.187+0000 7fb342d9d640 1 -- 172.21.15.86:0/1456025873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3441053a0 msgr2=0x7fb34419a860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.187+0000 7fb342d9d640 1 --2- 172.21.15.86:0/1456025873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3441053a0 0x7fb34419a860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.187+0000 7fb342d9d640 1 -- 172.21.15.86:0/1456025873 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3441a1f40 con 0x7fb344105d60 2024-04-21T08:38:11.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.187+0000 7fb34259c640 1 --2- 172.21.15.86:0/1456025873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3441041b0 0x7fb34419a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:11.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.187+0000 7fb342d9d640 1 --2- 172.21.15.86:0/1456025873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb344105d60 0x7fb3441a18d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb33400c970 tx=0x7fb33400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:11.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.187+0000 7fb3337fe640 1 -- 172.21.15.86:0/1456025873 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb334007bb0 con 0x7fb344105d60 2024-04-21T08:38:11.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.187+0000 7fb348952640 1 -- 172.21.15.86:0/1456025873 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3441a2230 con 0x7fb344105d60 2024-04-21T08:38:11.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.187+0000 7fb3337fe640 1 -- 172.21.15.86:0/1456025873 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb334007d50 con 0x7fb344105d60 2024-04-21T08:38:11.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.187+0000 7fb3337fe640 1 -- 172.21.15.86:0/1456025873 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb334018770 con 0x7fb344105d60 2024-04-21T08:38:11.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.187+0000 7fb348952640 1 -- 172.21.15.86:0/1456025873 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3441a2780 con 0x7fb344105d60 2024-04-21T08:38:11.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.188+0000 7fb348952640 1 -- 172.21.15.86:0/1456025873 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb34410b4d0 con 0x7fb344105d60 2024-04-21T08:38:11.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.189+0000 7fb3337fe640 1 -- 172.21.15.86:0/1456025873 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb33400f990 con 0x7fb344105d60 2024-04-21T08:38:11.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.190+0000 7fb3337fe640 1 --2- 172.21.15.86:0/1456025873 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb32c0a3bb0 0x7fb32c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.190+0000 7fb34259c640 1 --2- 172.21.15.86:0/1456025873 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb32c0a3bb0 0x7fb32c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.190+0000 7fb3337fe640 1 -- 172.21.15.86:0/1456025873 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fb3340d08f0 con 0x7fb344105d60 2024-04-21T08:38:11.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.190+0000 7fb34259c640 1 --2- 172.21.15.86:0/1456025873 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb32c0a3bb0 0x7fb32c0a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb344105200 tx=0x7fb33c009400 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:11.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.194+0000 7fb3337fe640 1 -- 172.21.15.86:0/1456025873 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb334014030 con 0x7fb344105d60 2024-04-21T08:38:11.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.415+0000 7fb348952640 1 -- 172.21.15.86:0/1456025873 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb344062b50 con 0x7fb344105d60 2024-04-21T08:38:11.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.416+0000 7fb3337fe640 1 -- 172.21.15.86:0/1456025873 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7fb33409cf70 con 0x7fb344105d60 2024-04-21T08:38:11.416 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:38:11.416 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:36:47.458637+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24455,24472,24394,24398,24482]},"id":1}]} 2024-04-21T08:38:11.416 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-21T08:38:11.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.418+0000 7fb348952640 1 -- 172.21.15.86:0/1456025873 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb32c0a3bb0 msgr2=0x7fb32c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.418+0000 7fb348952640 1 --2- 172.21.15.86:0/1456025873 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb32c0a3bb0 0x7fb32c0a6070 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7fb344105200 tx=0x7fb33c009400 comp rx=0 tx=0).stop 2024-04-21T08:38:11.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.418+0000 7fb348952640 1 -- 172.21.15.86:0/1456025873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb344105d60 msgr2=0x7fb3441a18d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.418+0000 7fb348952640 1 --2- 172.21.15.86:0/1456025873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb344105d60 0x7fb3441a18d0 secure :-1 s=READY pgs=381 cs=0 l=1 rev1=1 crypto rx=0x7fb33400c970 tx=0x7fb33400ce30 comp rx=0 tx=0).stop 2024-04-21T08:38:11.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.418+0000 7fb348952640 1 -- 172.21.15.86:0/1456025873 shutdown_connections 2024-04-21T08:38:11.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.418+0000 7fb348952640 1 --2- 172.21.15.86:0/1456025873 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb32c0a3bb0 0x7fb32c0a6070 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.418+0000 7fb348952640 1 --2- 172.21.15.86:0/1456025873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb344105d60 0x7fb3441a18d0 unknown :-1 s=CLOSED pgs=381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.418+0000 7fb348952640 1 --2- 172.21.15.86:0/1456025873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3441053a0 0x7fb34419a860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.418+0000 7fb348952640 1 --2- 172.21.15.86:0/1456025873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3441041b0 0x7fb34419a320 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.418+0000 7fb348952640 1 -- 172.21.15.86:0/1456025873 >> 172.21.15.86:0/1456025873 conn(0x7fb3440ff740 msgr2=0x7fb3441086a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:11.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.418+0000 7fb348952640 1 -- 172.21.15.86:0/1456025873 shutdown_connections 2024-04-21T08:38:11.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.418+0000 7fb348952640 1 -- 172.21.15.86:0/1456025873 wait complete. 2024-04-21T08:38:11.429 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-21T08:36:20.626894+0000', 'modified': '2024-04-21T08:36:47.458637+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24482, 'mds_1': 24472, 'mds_2': 24455, 'mds_3': 24398, 'mds_4': 24394}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14640': {'gid': 14640, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6839/3163123023', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6838', 'nonce': 3163123023}, {'type': 'v1', 'addr': '172.21.15.86:6839', 'nonce': 3163123023}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14652': {'gid': 14652, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6841/3537972023', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6840', 'nonce': 3537972023}, {'type': 'v1', 'addr': '172.21.15.86:6841', 'nonce': 3537972023}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24394': {'gid': 24394, 'name': 'l', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.179:6833/3606421532', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6832', 'nonce': 3606421532}, {'type': 'v1', 'addr': '172.21.15.179:6833', 'nonce': 3606421532}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24398': {'gid': 24398, 'name': 'k', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.88:6833/1872744608', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6832', 'nonce': 1872744608}, {'type': 'v1', 'addr': '172.21.15.88:6833', 'nonce': 1872744608}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.88:6835/454792798', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6834', 'nonce': 454792798}, {'type': 'v1', 'addr': '172.21.15.88:6835', 'nonce': 454792798}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.179:6835/3343050063', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6834', 'nonce': 3343050063}, {'type': 'v1', 'addr': '172.21.15.179:6835', 'nonce': 3343050063}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.179:6837/1280529894', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6836', 'nonce': 1280529894}, {'type': 'v1', 'addr': '172.21.15.179:6837', 'nonce': 1280529894}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.88:6837/1950568471', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6836', 'nonce': 1950568471}, {'type': 'v1', 'addr': '172.21.15.88:6837', 'nonce': 1950568471}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.179:6839/3029310820', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6838', 'nonce': 3029310820}, {'type': 'v1', 'addr': '172.21.15.179:6839', 'nonce': 3029310820}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.88:6839/3969608142', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6838', 'nonce': 3969608142}, {'type': 'v1', 'addr': '172.21.15.88:6839', 'nonce': 3969608142}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24482, 'qdb_cluster': [24455, 24472, 24394, 24398, 24482]} 2024-04-21T08:38:11.430 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-21T08:38:11.430 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-04-21T08:38:11.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.529+0000 7f94ecd0e640 1 -- 172.21.15.86:0/4240089550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94e8100e60 msgr2=0x7f94e8101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.529+0000 7f94ecd0e640 1 --2- 172.21.15.86:0/4240089550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94e8100e60 0x7f94e8101280 secure :-1 s=READY pgs=382 cs=0 l=1 rev1=1 crypto rx=0x7f94d8009900 tx=0x7f94d802bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:11.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.529+0000 7f94ecd0e640 1 -- 172.21.15.86:0/4240089550 shutdown_connections 2024-04-21T08:38:11.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.529+0000 7f94ecd0e640 1 --2- 172.21.15.86:0/4240089550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94e8102a00 0x7f94e8109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.529+0000 7f94ecd0e640 1 --2- 172.21.15.86:0/4240089550 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94e8102040 0x7f94e81024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.529+0000 7f94ecd0e640 1 --2- 172.21.15.86:0/4240089550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94e8100e60 0x7f94e8101280 unknown :-1 s=CLOSED pgs=382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.529+0000 7f94ecd0e640 1 -- 172.21.15.86:0/4240089550 >> 172.21.15.86:0/4240089550 conn(0x7f94e80fc420 msgr2=0x7f94e80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:11.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.529+0000 7f94ecd0e640 1 -- 172.21.15.86:0/4240089550 shutdown_connections 2024-04-21T08:38:11.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.529+0000 7f94ecd0e640 1 -- 172.21.15.86:0/4240089550 wait complete. 2024-04-21T08:38:11.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.530+0000 7f94ecd0e640 1 Processor -- start 2024-04-21T08:38:11.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.530+0000 7f94ecd0e640 1 -- start start 2024-04-21T08:38:11.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.530+0000 7f94ecd0e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94e8100e60 0x7f94e81983b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.530+0000 7f94ecd0e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94e8102040 0x7f94e81988f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.530+0000 7f94ecd0e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94e8102a00 0x7f94e819f960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.530+0000 7f94ecd0e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f94e810dbf0 con 0x7f94e8102a00 2024-04-21T08:38:11.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.530+0000 7f94ecd0e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f94e810da70 con 0x7f94e8102040 2024-04-21T08:38:11.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.530+0000 7f94ecd0e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f94e810dd70 con 0x7f94e8100e60 2024-04-21T08:38:11.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.530+0000 7f94e6d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94e8102a00 0x7f94e819f960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.530+0000 7f94e6d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94e8102a00 0x7f94e819f960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36366/0 (socket says 172.21.15.86:36366) 2024-04-21T08:38:11.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.530+0000 7f94e6d9d640 1 -- 172.21.15.86:0/1113901651 learned_addr learned my addr 172.21.15.86:0/1113901651 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:11.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.530+0000 7f94e5d9b640 1 --2- 172.21.15.86:0/1113901651 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94e8102040 0x7f94e81988f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.530+0000 7f94e659c640 1 --2- 172.21.15.86:0/1113901651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94e8100e60 0x7f94e81983b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.530+0000 7f94e6d9d640 1 -- 172.21.15.86:0/1113901651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94e8100e60 msgr2=0x7f94e81983b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.530+0000 7f94e6d9d640 1 --2- 172.21.15.86:0/1113901651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94e8100e60 0x7f94e81983b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.530+0000 7f94e6d9d640 1 -- 172.21.15.86:0/1113901651 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94e8102040 msgr2=0x7f94e81988f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.530+0000 7f94e6d9d640 1 --2- 172.21.15.86:0/1113901651 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94e8102040 0x7f94e81988f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.530+0000 7f94e6d9d640 1 -- 172.21.15.86:0/1113901651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f94e819ffd0 con 0x7f94e8102a00 2024-04-21T08:38:11.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.531+0000 7f94e659c640 1 --2- 172.21.15.86:0/1113901651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94e8100e60 0x7f94e81983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:11.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.531+0000 7f94e6d9d640 1 --2- 172.21.15.86:0/1113901651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94e8102a00 0x7f94e819f960 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f94e000b970 tx=0x7f94e000bd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:11.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.531+0000 7f94d77fe640 1 -- 172.21.15.86:0/1113901651 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e0019070 con 0x7f94e8102a00 2024-04-21T08:38:11.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.531+0000 7f94d77fe640 1 -- 172.21.15.86:0/1113901651 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94e0014440 con 0x7f94e8102a00 2024-04-21T08:38:11.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.531+0000 7f94ecd0e640 1 -- 172.21.15.86:0/1113901651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f94e81a02c0 con 0x7f94e8102a00 2024-04-21T08:38:11.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.531+0000 7f94d77fe640 1 -- 172.21.15.86:0/1113901651 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94e00075d0 con 0x7f94e8102a00 2024-04-21T08:38:11.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.531+0000 7f94ecd0e640 1 -- 172.21.15.86:0/1113901651 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f94e81a0890 con 0x7f94e8102a00 2024-04-21T08:38:11.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.533+0000 7f94d77fe640 1 -- 172.21.15.86:0/1113901651 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f94e001c070 con 0x7f94e8102a00 2024-04-21T08:38:11.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.533+0000 7f94ef9dc740 1 Processor -- start 2024-04-21T08:38:11.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.534+0000 7f94ef9dc740 1 -- start start 2024-04-21T08:38:11.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.534+0000 7f94ef9dc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555817216e40 0x555817217260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.534+0000 7f94ef9dc740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555817217830 0x555817217cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.534+0000 7f94ef9dc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5558172182d0 0x55581721eeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.534+0000 7f94ef9dc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55581717bb10 con 0x555817216e40 2024-04-21T08:38:11.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.534+0000 7f94ef9dc740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x555817119240 con 0x555817217830 2024-04-21T08:38:11.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.534+0000 7f94ef9dc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555817177ad0 con 0x5558172182d0 2024-04-21T08:38:11.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.534+0000 7f94d77fe640 1 --2- 172.21.15.86:0/1113901651 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f94d00a3bb0 0x7f94d00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.534+0000 7f94e5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555817216e40 0x555817217260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.534+0000 7f94e5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555817216e40 0x555817217260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36378/0 (socket says 172.21.15.86:36378) 2024-04-21T08:38:11.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.534+0000 7f94e5d9b640 1 -- 172.21.15.86:0/2800787957 learned_addr learned my addr 172.21.15.86:0/2800787957 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:11.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.534+0000 7f94e6d9d640 1 --2- 172.21.15.86:0/1113901651 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f94d00a3bb0 0x7f94d00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.534+0000 7f94e5d9b640 1 --2- 172.21.15.86:0/2800787957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5558172182d0 0x55581721eeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.534+0000 7f94e659c640 1 --2- 172.21.15.86:0/2800787957 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555817217830 0x555817217cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.534+0000 7f94e5d9b640 1 -- 172.21.15.86:0/2800787957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5558172182d0 msgr2=0x55581721eeb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:11.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.534+0000 7f94e5d9b640 1 --2- 172.21.15.86:0/2800787957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5558172182d0 0x55581721eeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.534+0000 7f94e5d9b640 1 -- 172.21.15.86:0/2800787957 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555817217830 msgr2=0x555817217cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:11.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.534+0000 7f94e5d9b640 1 --2- 172.21.15.86:0/2800787957 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555817217830 0x555817217cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.534+0000 7f94d77fe640 1 -- 172.21.15.86:0/1113901651 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f94e00cfc60 con 0x7f94e8102a00 2024-04-21T08:38:11.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.534+0000 7f94e5d9b640 1 -- 172.21.15.86:0/2800787957 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555816e8e2d0 con 0x555817216e40 2024-04-21T08:38:11.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.534+0000 7f94e5d9b640 1 --2- 172.21.15.86:0/2800787957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555817216e40 0x555817217260 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f94dc006aa0 tx=0x7f94dc0095d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e6116f0d6d05241 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:11.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.534+0000 7f94e6d9d640 1 --2- 172.21.15.86:0/1113901651 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f94d00a3bb0 0x7f94d00a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f94e00a4320 tx=0x7f94e00c6cf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:11.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.534+0000 7f94d4ff9640 1 -- 172.21.15.86:0/2800787957 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc015620 con 0x555817216e40 2024-04-21T08:38:11.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.535+0000 7f94d4ff9640 1 -- 172.21.15.86:0/2800787957 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94dc01a070 con 0x555817216e40 2024-04-21T08:38:11.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.535+0000 7f94d4ff9640 1 -- 172.21.15.86:0/2800787957 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc0147b0 con 0x555817216e40 2024-04-21T08:38:11.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.535+0000 7f94ef9dc740 1 -- 172.21.15.86:0/2800787957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555817216e40 msgr2=0x555817217260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.535+0000 7f94ef9dc740 1 --2- 172.21.15.86:0/2800787957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555817216e40 0x555817217260 secure :-1 s=READY pgs=384 cs=0 l=1 rev1=1 crypto rx=0x7f94dc006aa0 tx=0x7f94dc0095d0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.535+0000 7f94ef9dc740 1 -- 172.21.15.86:0/2800787957 shutdown_connections 2024-04-21T08:38:11.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.535+0000 7f94ef9dc740 1 --2- 172.21.15.86:0/2800787957 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5558172182d0 0x55581721eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.535+0000 7f94ef9dc740 1 --2- 172.21.15.86:0/2800787957 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555817217830 0x555817217cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.535+0000 7f94ef9dc740 1 --2- 172.21.15.86:0/2800787957 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555817216e40 0x555817217260 unknown :-1 s=CLOSED pgs=384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.535+0000 7f94ef9dc740 1 -- 172.21.15.86:0/2800787957 >> 172.21.15.86:0/2800787957 conn(0x555817212f40 msgr2=0x555817211490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:11.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.535+0000 7f94ef9dc740 1 -- 172.21.15.86:0/2800787957 shutdown_connections 2024-04-21T08:38:11.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.535+0000 7f94ef9dc740 1 -- 172.21.15.86:0/2800787957 wait complete. 2024-04-21T08:38:11.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.536+0000 7f94ef9dc740 1 -- start start 2024-04-21T08:38:11.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.536+0000 7f94ef9dc740 1 Processor -- start 2024-04-21T08:38:11.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.536+0000 7f94ef9dc740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555817216e40 0x555817220e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.536+0000 7f94ef9dc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555817217830 0x5558172213c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.536+0000 7f94ef9dc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5558172182d0 0x555817228390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.536+0000 7f94ef9dc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55581721f450 con 0x555817217830 2024-04-21T08:38:11.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.536+0000 7f94ef9dc740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x555817104190 con 0x555817216e40 2024-04-21T08:38:11.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.536+0000 7f94ef9dc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55581721f5d0 con 0x5558172182d0 2024-04-21T08:38:11.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.536+0000 7f94ef9dc740 10 client.0 _ll_register_callbacks cb 0x5558170ff8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:11.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.536+0000 7f94e659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555817217830 0x5558172213c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.536+0000 7f94e659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555817217830 0x5558172213c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36384/0 (socket says 172.21.15.86:36384) 2024-04-21T08:38:11.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.536+0000 7f94e659c640 1 -- 172.21.15.86:0/1882473532 learned_addr learned my addr 172.21.15.86:0/1882473532 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:11.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.536+0000 7f94e5d9b640 1 --2- 172.21.15.86:0/1882473532 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555817216e40 0x555817220e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.536+0000 7f94e5d9b640 1 --2- 172.21.15.86:0/1882473532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5558172182d0 0x555817228390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.536+0000 7f94e659c640 1 -- 172.21.15.86:0/1882473532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5558172182d0 msgr2=0x555817228390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:11.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.536+0000 7f94e659c640 1 --2- 172.21.15.86:0/1882473532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5558172182d0 0x555817228390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.536+0000 7f94e659c640 1 -- 172.21.15.86:0/1882473532 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555817216e40 msgr2=0x555817220e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:11.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.536+0000 7f94e659c640 1 --2- 172.21.15.86:0/1882473532 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555817216e40 0x555817220e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.536+0000 7f94e659c640 1 -- 172.21.15.86:0/1882473532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555816f586b0 con 0x555817217830 2024-04-21T08:38:11.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.537+0000 7f94e659c640 1 --2- 172.21.15.86:0/1882473532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555817217830 0x5558172213c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f94d8036040 tx=0x7f94d8004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c63ad453e7abb81b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:11.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.537+0000 7f94cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:38:11.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.537+0000 7f94cdffb640 1 -- 172.21.15.86:0/1882473532 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94d8005d50 con 0x555817217830 2024-04-21T08:38:11.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.537+0000 7f94cdffb640 1 -- 172.21.15.86:0/1882473532 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f94d8005230 con 0x555817217830 2024-04-21T08:38:11.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.537+0000 7f94ef9dc740 1 -- 172.21.15.86:0/1882473532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94dc01a070 con 0x555817217830 2024-04-21T08:38:11.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.537+0000 7f94cdffb640 1 -- 172.21.15.86:0/1882473532 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f94d804d050 con 0x555817217830 2024-04-21T08:38:11.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.537+0000 7f94ef9dc740 10 client.15246 fetch_fsmap learned FSMap version 28 2024-04-21T08:38:11.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.537+0000 7f94ef9dc740 1 -- 172.21.15.86:0/1882473532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55581722b770 con 0x555817217830 2024-04-21T08:38:11.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.537+0000 7f94cdffb640 1 -- 172.21.15.86:0/1882473532 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94dc01a070 con 0x555817217830 2024-04-21T08:38:11.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.537+0000 7f94cdffb640 1 -- 172.21.15.86:0/1882473532 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f94d8003870 con 0x555817217830 2024-04-21T08:38:11.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.537+0000 7f94ef9dc740 10 client.15246 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:38:11.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.537+0000 7f94ef9dc740 10 client.15246 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-21T08:38:11.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.537+0000 7f94ef9dc740 20 client.15246 populate_metadata read hostname 'smithi086' 2024-04-21T08:38:11.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.537+0000 7f94ef9dc740 1 --2- 172.21.15.86:0/1882473532 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55581722de50 0x55581722f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.537+0000 7f94ef9dc740 4 client.15246 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:38:11.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.537+0000 7f94ef9dc740 1 -- 172.21.15.86:0/1882473532 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555817189530 con 0x55581722de50 2024-04-21T08:38:11.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.538+0000 7f94e5d9b640 1 --2- 172.21.15.86:0/1882473532 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55581722de50 0x55581722f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.538+0000 7f94e5d9b640 1 --2- 172.21.15.86:0/1882473532 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55581722de50 0x55581722f2c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=504e00f021086e58 server_cookie=409ffb708bea88e8 in_seq=0 out_seq=0 2024-04-21T08:38:11.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.538+0000 7f94cdffb640 10 client.15246 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:38:11.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.539+0000 7f94cdffb640 1 -- 172.21.15.86:0/1882473532 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555817189530 con 0x55581722de50 2024-04-21T08:38:11.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.539+0000 7f94cdffb640 10 client.15246 handle_command_reply: tid=0 2024-04-21T08:38:11.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.539+0000 7f94ef9dc740 1 client.15246 shutdown 2024-04-21T08:38:11.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.539+0000 7f94ef9dc740 1 -- 172.21.15.86:0/1882473532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555817217830 msgr2=0x5558172213c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.539+0000 7f94ef9dc740 1 --2- 172.21.15.86:0/1882473532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555817217830 0x5558172213c0 secure :-1 s=READY pgs=385 cs=0 l=1 rev1=1 crypto rx=0x7f94d8036040 tx=0x7f94d8004270 comp rx=0 tx=0).stop 2024-04-21T08:38:11.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.539+0000 7f94ef9dc740 1 -- 172.21.15.86:0/1882473532 shutdown_connections 2024-04-21T08:38:11.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.539+0000 7f94ef9dc740 1 --2- 172.21.15.86:0/1882473532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5558172182d0 0x555817228390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.539+0000 7f94ef9dc740 1 --2- 172.21.15.86:0/1882473532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555817217830 0x5558172213c0 unknown :-1 s=CLOSED pgs=385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.539+0000 7f94ef9dc740 1 --2- 172.21.15.86:0/1882473532 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55581722de50 0x55581722f2c0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.539+0000 7f94ef9dc740 1 --2- 172.21.15.86:0/1882473532 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555817216e40 0x555817220e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.539+0000 7f94cdffb640 0 client.15246 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:38:11.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.539+0000 7f94ef9dc740 1 -- 172.21.15.86:0/1882473532 >> 172.21.15.86:0/1882473532 conn(0x555817212f40 msgr2=0x555817210a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:11.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.539+0000 7f94ef9dc740 1 -- 172.21.15.86:0/1882473532 shutdown_connections 2024-04-21T08:38:11.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.539+0000 7f94ef9dc740 1 -- 172.21.15.86:0/1882473532 wait complete. 2024-04-21T08:38:11.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.539+0000 7f94ef9dc740 20 client.15246 trim_cache size 0 max 16384 2024-04-21T08:38:11.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.557+0000 7f94ef9dc740 1 Processor -- start 2024-04-21T08:38:11.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.557+0000 7f94ef9dc740 1 -- start start 2024-04-21T08:38:11.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.557+0000 7f94ef9dc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555817216e40 0x55581721fa00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.557+0000 7f94ef9dc740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555817217830 0x555817228350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.557+0000 7f94ef9dc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5558172182d0 0x55581721b840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.557+0000 7f94ef9dc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55581722d600 con 0x555817216e40 2024-04-21T08:38:11.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.557+0000 7f94ef9dc740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55581722dcd0 con 0x555817217830 2024-04-21T08:38:11.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.557+0000 7f94ef9dc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55581722e3a0 con 0x5558172182d0 2024-04-21T08:38:11.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.558+0000 7f94e5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555817216e40 0x55581721fa00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.558+0000 7f94e5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555817216e40 0x55581721fa00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36390/0 (socket says 172.21.15.86:36390) 2024-04-21T08:38:11.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.558+0000 7f94e5d9b640 1 -- 172.21.15.86:0/2918051578 learned_addr learned my addr 172.21.15.86:0/2918051578 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:11.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.558+0000 7f94e659c640 1 --2- 172.21.15.86:0/2918051578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555817217830 0x555817228350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.558+0000 7f94e5d9b640 1 -- 172.21.15.86:0/2918051578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5558172182d0 msgr2=0x55581721b840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:11.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.558+0000 7f94e5d9b640 1 --2- 172.21.15.86:0/2918051578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5558172182d0 0x55581721b840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.558+0000 7f94e5d9b640 1 -- 172.21.15.86:0/2918051578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555817217830 msgr2=0x555817228350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:11.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.558+0000 7f94e5d9b640 1 --2- 172.21.15.86:0/2918051578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555817217830 0x555817228350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.558+0000 7f94e5d9b640 1 -- 172.21.15.86:0/2918051578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55581721eac0 con 0x555817216e40 2024-04-21T08:38:11.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.558+0000 7f94e659c640 1 --2- 172.21.15.86:0/2918051578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555817217830 0x555817228350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:11.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.558+0000 7f94e5d9b640 1 --2- 172.21.15.86:0/2918051578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555817216e40 0x55581721fa00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x555817212200 tx=0x7f94dc0047d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=350d3240968841dc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:11.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.558+0000 7f94cffff640 1 -- 172.21.15.86:0/2918051578 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc030020 con 0x555817216e40 2024-04-21T08:38:11.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.558+0000 7f94cffff640 1 -- 172.21.15.86:0/2918051578 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94dc01a050 con 0x555817216e40 2024-04-21T08:38:11.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.558+0000 7f94cffff640 1 -- 172.21.15.86:0/2918051578 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc0158c0 con 0x555817216e40 2024-04-21T08:38:11.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.559+0000 7f94ef9dc740 1 -- 172.21.15.86:0/2918051578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555817216e40 msgr2=0x55581721fa00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.559+0000 7f94ef9dc740 1 --2- 172.21.15.86:0/2918051578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555817216e40 0x55581721fa00 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x555817212200 tx=0x7f94dc0047d0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.559+0000 7f94ef9dc740 1 -- 172.21.15.86:0/2918051578 shutdown_connections 2024-04-21T08:38:11.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.559+0000 7f94ef9dc740 1 --2- 172.21.15.86:0/2918051578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5558172182d0 0x55581721b840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.559+0000 7f94ef9dc740 1 --2- 172.21.15.86:0/2918051578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555817217830 0x555817228350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.559+0000 7f94ef9dc740 1 --2- 172.21.15.86:0/2918051578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555817216e40 0x55581721fa00 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.559+0000 7f94ef9dc740 1 -- 172.21.15.86:0/2918051578 >> 172.21.15.86:0/2918051578 conn(0x555817212f40 msgr2=0x55581720c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:11.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.559+0000 7f94ef9dc740 1 -- 172.21.15.86:0/2918051578 shutdown_connections 2024-04-21T08:38:11.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.559+0000 7f94ef9dc740 1 -- 172.21.15.86:0/2918051578 wait complete. 2024-04-21T08:38:11.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.559+0000 7f94ef9dc740 1 -- start start 2024-04-21T08:38:11.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.559+0000 7f94ef9dc740 1 Processor -- start 2024-04-21T08:38:11.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.559+0000 7f94ef9dc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555817216e40 0x555817224330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.559+0000 7f94ef9dc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555817217830 0x555817226880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.559+0000 7f94ef9dc740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558172182d0 0x555817226dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.559+0000 7f94ef9dc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55581722f140 con 0x555817216e40 2024-04-21T08:38:11.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.559+0000 7f94ef9dc740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55581722ea70 con 0x5558172182d0 2024-04-21T08:38:11.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.559+0000 7f94ef9dc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55581722f810 con 0x555817217830 2024-04-21T08:38:11.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.559+0000 7f94ef9dc740 10 client.0 _ll_register_callbacks cb 0x7f94d8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:11.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.559+0000 7f94e5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555817216e40 0x555817224330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.559+0000 7f94e5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555817216e40 0x555817224330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36406/0 (socket says 172.21.15.86:36406) 2024-04-21T08:38:11.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.559+0000 7f94e5d9b640 1 -- 172.21.15.86:0/949907832 learned_addr learned my addr 172.21.15.86:0/949907832 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:11.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.559+0000 7f94e659c640 1 --2- 172.21.15.86:0/949907832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555817217830 0x555817226880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.560+0000 7f94e5d9b640 1 --2- 172.21.15.86:0/949907832 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558172182d0 0x555817226dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.560+0000 7f94e5d9b640 1 -- 172.21.15.86:0/949907832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555817217830 msgr2=0x555817226880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:11.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.560+0000 7f94e5d9b640 1 --2- 172.21.15.86:0/949907832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555817217830 0x555817226880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.560+0000 7f94e5d9b640 1 -- 172.21.15.86:0/949907832 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558172182d0 msgr2=0x555817226dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:11.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.560+0000 7f94e5d9b640 1 --2- 172.21.15.86:0/949907832 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558172182d0 0x555817226dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.560+0000 7f94e5d9b640 1 -- 172.21.15.86:0/949907832 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555817227300 con 0x555817216e40 2024-04-21T08:38:11.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.560+0000 7f94e5d9b640 1 --2- 172.21.15.86:0/949907832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555817216e40 0x555817224330 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f94dc009780 tx=0x7f94dc029040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1241c4756e8b3ed8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:11.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.560+0000 7f94cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:38:11.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.560+0000 7f94cdffb640 1 -- 172.21.15.86:0/949907832 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94dc030050 con 0x555817216e40 2024-04-21T08:38:11.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.560+0000 7f94cdffb640 1 -- 172.21.15.86:0/949907832 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f94dc015640 con 0x555817216e40 2024-04-21T08:38:11.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.560+0000 7f94ef9dc740 1 -- 172.21.15.86:0/949907832 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f94dc01a050 con 0x555817216e40 2024-04-21T08:38:11.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.560+0000 7f94cdffb640 1 -- 172.21.15.86:0/949907832 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f94dc015840 con 0x555817216e40 2024-04-21T08:38:11.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.560+0000 7f94ef9dc740 10 client.15252 fetch_fsmap learned FSMap version 28 2024-04-21T08:38:11.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.560+0000 7f94ef9dc740 1 -- 172.21.15.86:0/949907832 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x555817228570 con 0x555817216e40 2024-04-21T08:38:11.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.560+0000 7f94cdffb640 1 -- 172.21.15.86:0/949907832 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f94dc01a050 con 0x555817216e40 2024-04-21T08:38:11.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.560+0000 7f94cdffb640 1 -- 172.21.15.86:0/949907832 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f94dc0062f0 con 0x555817216e40 2024-04-21T08:38:11.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.560+0000 7f94ef9dc740 10 client.15252 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:38:11.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.560+0000 7f94ef9dc740 10 client.15252 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-21T08:38:11.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.560+0000 7f94ef9dc740 20 client.15252 populate_metadata read hostname 'smithi086' 2024-04-21T08:38:11.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.560+0000 7f94ef9dc740 1 --2- 172.21.15.86:0/949907832 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x555817297fe0 0x555817296cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.560+0000 7f94ef9dc740 4 client.15252 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:38:11.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.560+0000 7f94ef9dc740 1 -- 172.21.15.86:0/949907832 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "status"}) -- 0x5558171833a0 con 0x555817297fe0 2024-04-21T08:38:11.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.561+0000 7f94e659c640 1 --2- 172.21.15.86:0/949907832 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x555817297fe0 0x555817296cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.561+0000 7f94e659c640 1 --2- 172.21.15.86:0/949907832 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x555817297fe0 0x555817296cf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8ff15e7b1f209fbf server_cookie=db552117f68e4a68 in_seq=0 out_seq=0 2024-04-21T08:38:11.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.561+0000 7f94cdffb640 10 client.15252 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:38:11.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.561+0000 7f94cdffb640 1 -- 172.21.15.86:0/949907832 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x5558171833a0 con 0x555817297fe0 2024-04-21T08:38:11.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.561+0000 7f94cdffb640 10 client.15252 handle_command_reply: tid=0 2024-04-21T08:38:11.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.561+0000 7f94ef9dc740 1 client.15252 shutdown 2024-04-21T08:38:11.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.562+0000 7f94ef9dc740 1 -- 172.21.15.86:0/949907832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555817216e40 msgr2=0x555817224330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.562+0000 7f94ef9dc740 1 --2- 172.21.15.86:0/949907832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555817216e40 0x555817224330 secure :-1 s=READY pgs=387 cs=0 l=1 rev1=1 crypto rx=0x7f94dc009780 tx=0x7f94dc029040 comp rx=0 tx=0).stop 2024-04-21T08:38:11.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.562+0000 7f94ef9dc740 1 -- 172.21.15.86:0/949907832 shutdown_connections 2024-04-21T08:38:11.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.562+0000 7f94ef9dc740 1 --2- 172.21.15.86:0/949907832 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x555817297fe0 0x555817296cf0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.562+0000 7f94ef9dc740 1 --2- 172.21.15.86:0/949907832 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5558172182d0 0x555817226dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.562+0000 7f94ef9dc740 1 --2- 172.21.15.86:0/949907832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555817217830 0x555817226880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.562+0000 7f94cdffb640 0 client.15252 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:38:11.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.562+0000 7f94ef9dc740 1 --2- 172.21.15.86:0/949907832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555817216e40 0x555817224330 unknown :-1 s=CLOSED pgs=387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.562+0000 7f94ef9dc740 1 -- 172.21.15.86:0/949907832 >> 172.21.15.86:0/949907832 conn(0x555817212f40 msgr2=0x55581721b840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:11.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.562+0000 7f94ef9dc740 1 -- 172.21.15.86:0/949907832 shutdown_connections 2024-04-21T08:38:11.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.562+0000 7f94ef9dc740 1 -- 172.21.15.86:0/949907832 wait complete. 2024-04-21T08:38:11.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.562+0000 7f94ef9dc740 20 client.15252 trim_cache size 0 max 16384 2024-04-21T08:38:11.569 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:38:11.569 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "f8cec7cc-ffb8-11ee-bc93-c7b262605968", 2024-04-21T08:38:11.570 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 4, 2024-04-21T08:38:11.570 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24394, 2024-04-21T08:38:11.570 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-21T08:38:11.570 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-21T08:38:11.570 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-21T08:38:11.570 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 85.119484858999996, 2024-04-21T08:38:11.570 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 28, 2024-04-21T08:38:11.570 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 91, 2024-04-21T08:38:11.570 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 89, 2024-04-21T08:38:11.570 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 148.34786556099999 2024-04-21T08:38:11.570 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:38:11.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.562+0000 7f94ecd0e640 1 -- 172.21.15.86:0/1113901651 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f94d00a3bb0 msgr2=0x7f94d00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.562+0000 7f94ecd0e640 1 --2- 172.21.15.86:0/1113901651 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f94d00a3bb0 0x7f94d00a6070 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f94e00a4320 tx=0x7f94e00c6cf0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.562+0000 7f94ecd0e640 1 -- 172.21.15.86:0/1113901651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94e8102a00 msgr2=0x7f94e819f960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.562+0000 7f94ecd0e640 1 --2- 172.21.15.86:0/1113901651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94e8102a00 0x7f94e819f960 secure :-1 s=READY pgs=383 cs=0 l=1 rev1=1 crypto rx=0x7f94e000b970 tx=0x7f94e000bd70 comp rx=0 tx=0).stop 2024-04-21T08:38:11.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.562+0000 7f94ecd0e640 1 -- 172.21.15.86:0/1113901651 shutdown_connections 2024-04-21T08:38:11.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.562+0000 7f94ecd0e640 1 --2- 172.21.15.86:0/1113901651 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f94d00a3bb0 0x7f94d00a6070 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.562+0000 7f94ecd0e640 1 --2- 172.21.15.86:0/1113901651 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f94e8102a00 0x7f94e819f960 unknown :-1 s=CLOSED pgs=383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.562+0000 7f94ecd0e640 1 --2- 172.21.15.86:0/1113901651 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f94e8102040 0x7f94e81988f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.562+0000 7f94ecd0e640 1 --2- 172.21.15.86:0/1113901651 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f94e8100e60 0x7f94e81983b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.562+0000 7f94ecd0e640 1 -- 172.21.15.86:0/1113901651 >> 172.21.15.86:0/1113901651 conn(0x7f94e80fc420 msgr2=0x7f94e8105560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:11.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.562+0000 7f94ecd0e640 1 -- 172.21.15.86:0/1113901651 shutdown_connections 2024-04-21T08:38:11.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.562+0000 7f94ecd0e640 1 -- 172.21.15.86:0/1113901651 wait complete. 2024-04-21T08:38:11.573 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-04-21T08:38:11.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.671+0000 7fd03a567640 1 -- 172.21.15.86:0/4039455691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd034105340 msgr2=0x7fd034107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.671+0000 7fd03a567640 1 --2- 172.21.15.86:0/4039455691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd034105340 0x7fd034107730 secure :-1 s=READY pgs=388 cs=0 l=1 rev1=1 crypto rx=0x7fd028009900 tx=0x7fd02802bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:11.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.671+0000 7fd03a567640 1 -- 172.21.15.86:0/4039455691 shutdown_connections 2024-04-21T08:38:11.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.671+0000 7fd03a567640 1 --2- 172.21.15.86:0/4039455691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd034107c70 0x7fd03410a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.671+0000 7fd03a567640 1 --2- 172.21.15.86:0/4039455691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd034105340 0x7fd034107730 unknown :-1 s=CLOSED pgs=388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.671+0000 7fd03a567640 1 --2- 172.21.15.86:0/4039455691 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd034102a10 0x7fd034104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.671+0000 7fd03a567640 1 -- 172.21.15.86:0/4039455691 >> 172.21.15.86:0/4039455691 conn(0x7fd0340fc440 msgr2=0x7fd0340fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:11.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.671+0000 7fd03a567640 1 -- 172.21.15.86:0/4039455691 shutdown_connections 2024-04-21T08:38:11.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.671+0000 7fd03a567640 1 -- 172.21.15.86:0/4039455691 wait complete. 2024-04-21T08:38:11.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.672+0000 7fd03a567640 1 Processor -- start 2024-04-21T08:38:11.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.672+0000 7fd03a567640 1 -- start start 2024-04-21T08:38:11.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.672+0000 7fd03a567640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd034102a10 0x7fd034198220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.672+0000 7fd03a567640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd034105340 0x7fd034198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.672+0000 7fd03a567640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd034107c70 0x7fd03419f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.672+0000 7fd03a567640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd03410dab0 con 0x7fd034102a10 2024-04-21T08:38:11.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.672+0000 7fd03a567640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd03410d930 con 0x7fd034107c70 2024-04-21T08:38:11.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.672+0000 7fd03a567640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd03410dc30 con 0x7fd034105340 2024-04-21T08:38:11.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.672+0000 7fd033fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd034102a10 0x7fd034198220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.672+0000 7fd033fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd034102a10 0x7fd034198220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36422/0 (socket says 172.21.15.86:36422) 2024-04-21T08:38:11.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.672+0000 7fd033fff640 1 -- 172.21.15.86:0/3438933770 learned_addr learned my addr 172.21.15.86:0/3438933770 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:11.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.672+0000 7fd038b04640 1 --2- 172.21.15.86:0/3438933770 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd034107c70 0x7fd03419f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.672+0000 7fd0337fe640 1 --2- 172.21.15.86:0/3438933770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd034105340 0x7fd034198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.672+0000 7fd033fff640 1 -- 172.21.15.86:0/3438933770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd034105340 msgr2=0x7fd034198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.672+0000 7fd033fff640 1 --2- 172.21.15.86:0/3438933770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd034105340 0x7fd034198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.672+0000 7fd033fff640 1 -- 172.21.15.86:0/3438933770 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd034107c70 msgr2=0x7fd03419f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.672+0000 7fd033fff640 1 --2- 172.21.15.86:0/3438933770 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd034107c70 0x7fd03419f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.672+0000 7fd033fff640 1 -- 172.21.15.86:0/3438933770 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd03419fe40 con 0x7fd034102a10 2024-04-21T08:38:11.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.672+0000 7fd038b04640 1 --2- 172.21.15.86:0/3438933770 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd034107c70 0x7fd03419f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:11.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.672+0000 7fd0337fe640 1 --2- 172.21.15.86:0/3438933770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd034105340 0x7fd034198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:11.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.673+0000 7fd033fff640 1 --2- 172.21.15.86:0/3438933770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd034102a10 0x7fd034198220 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd02400c890 tx=0x7fd02400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:11.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.673+0000 7fd0317fa640 1 -- 172.21.15.86:0/3438933770 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd024004220 con 0x7fd034102a10 2024-04-21T08:38:11.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.673+0000 7fd03a567640 1 -- 172.21.15.86:0/3438933770 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd0341a0130 con 0x7fd034102a10 2024-04-21T08:38:11.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.673+0000 7fd0317fa640 1 -- 172.21.15.86:0/3438933770 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd0240043c0 con 0x7fd034102a10 2024-04-21T08:38:11.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.673+0000 7fd0317fa640 1 -- 172.21.15.86:0/3438933770 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd024004a00 con 0x7fd034102a10 2024-04-21T08:38:11.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.673+0000 7fd03a567640 1 -- 172.21.15.86:0/3438933770 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd0341a06d0 con 0x7fd034102a10 2024-04-21T08:38:11.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.675+0000 7fd0317fa640 1 -- 172.21.15.86:0/3438933770 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fd0240079b0 con 0x7fd034102a10 2024-04-21T08:38:11.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.675+0000 7fd03d238740 1 Processor -- start 2024-04-21T08:38:11.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.675+0000 7fd03d238740 1 -- start start 2024-04-21T08:38:11.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.675+0000 7fd03d238740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568f4413e40 0x5568f4414260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.675+0000 7fd03d238740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5568f4414830 0x5568f4414cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.675+0000 7fd03d238740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f44152d0 0x5568f441beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.675+0000 7fd03d238740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5568f4378b10 con 0x5568f4413e40 2024-04-21T08:38:11.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.675+0000 7fd03d238740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5568f4316240 con 0x5568f4414830 2024-04-21T08:38:11.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.675+0000 7fd03d238740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5568f4374ad0 con 0x5568f44152d0 2024-04-21T08:38:11.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.675+0000 7fd038b04640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568f4413e40 0x5568f4414260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.676+0000 7fd038b04640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568f4413e40 0x5568f4414260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36432/0 (socket says 172.21.15.86:36432) 2024-04-21T08:38:11.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.676+0000 7fd038b04640 1 -- 172.21.15.86:0/2123407553 learned_addr learned my addr 172.21.15.86:0/2123407553 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:11.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.676+0000 7fd0317fa640 1 --2- 172.21.15.86:0/3438933770 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fd01c0a3bb0 0x7fd01c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.676+0000 7fd033fff640 1 --2- 172.21.15.86:0/2123407553 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5568f4414830 0x5568f4414cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.676+0000 7fd038b04640 1 -- 172.21.15.86:0/2123407553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f44152d0 msgr2=0x5568f441beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:11.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.676+0000 7fd038b04640 1 --2- 172.21.15.86:0/2123407553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f44152d0 0x5568f441beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.676+0000 7fd038b04640 1 -- 172.21.15.86:0/2123407553 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5568f4414830 msgr2=0x5568f4414cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:11.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.676+0000 7fd038b04640 1 --2- 172.21.15.86:0/2123407553 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5568f4414830 0x5568f4414cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.676+0000 7fd038b04640 1 -- 172.21.15.86:0/2123407553 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568f408b2d0 con 0x5568f4413e40 2024-04-21T08:38:11.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.676+0000 7fd033fff640 1 --2- 172.21.15.86:0/2123407553 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5568f4414830 0x5568f4414cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:11.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.676+0000 7fd038b04640 1 --2- 172.21.15.86:0/3438933770 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fd01c0a3bb0 0x7fd01c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.676+0000 7fd0317fa640 1 -- 172.21.15.86:0/3438933770 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fd0240d05b0 con 0x7fd034102a10 2024-04-21T08:38:11.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.676+0000 7fd038b04640 1 --2- 172.21.15.86:0/2123407553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568f4413e40 0x5568f4414260 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd02c008f50 tx=0x7fd02c00d7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99e5728865f15e5c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:11.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.676+0000 7fd01a7fc640 1 -- 172.21.15.86:0/2123407553 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c01a8e0 con 0x5568f4413e40 2024-04-21T08:38:11.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.676+0000 7fd01a7fc640 1 -- 172.21.15.86:0/2123407553 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd02c019410 con 0x5568f4413e40 2024-04-21T08:38:11.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.676+0000 7fd038b04640 1 --2- 172.21.15.86:0/3438933770 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fd01c0a3bb0 0x7fd01c0a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd02c01e000 tx=0x7fd02c01f000 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:11.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.676+0000 7fd01a7fc640 1 -- 172.21.15.86:0/2123407553 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c00e610 con 0x5568f4413e40 2024-04-21T08:38:11.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.677+0000 7fd03d238740 1 -- 172.21.15.86:0/2123407553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568f4413e40 msgr2=0x5568f4414260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.677+0000 7fd03d238740 1 --2- 172.21.15.86:0/2123407553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568f4413e40 0x5568f4414260 secure :-1 s=READY pgs=390 cs=0 l=1 rev1=1 crypto rx=0x7fd02c008f50 tx=0x7fd02c00d7d0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.677+0000 7fd03d238740 1 -- 172.21.15.86:0/2123407553 shutdown_connections 2024-04-21T08:38:11.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.677+0000 7fd03d238740 1 --2- 172.21.15.86:0/2123407553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f44152d0 0x5568f441beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.677+0000 7fd03d238740 1 --2- 172.21.15.86:0/2123407553 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5568f4414830 0x5568f4414cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.677+0000 7fd03d238740 1 --2- 172.21.15.86:0/2123407553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568f4413e40 0x5568f4414260 unknown :-1 s=CLOSED pgs=390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.677+0000 7fd03d238740 1 -- 172.21.15.86:0/2123407553 >> 172.21.15.86:0/2123407553 conn(0x5568f440ff40 msgr2=0x5568f440e490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:11.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.677+0000 7fd03d238740 1 -- 172.21.15.86:0/2123407553 shutdown_connections 2024-04-21T08:38:11.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.677+0000 7fd03d238740 1 -- 172.21.15.86:0/2123407553 wait complete. 2024-04-21T08:38:11.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.677+0000 7fd03d238740 1 -- start start 2024-04-21T08:38:11.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.677+0000 7fd03d238740 1 Processor -- start 2024-04-21T08:38:11.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.678+0000 7fd03d238740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568f4413e40 0x5568f441deb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.678+0000 7fd03d238740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5568f4414830 0x5568f441e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.678+0000 7fd03d238740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f44152d0 0x5568f4425430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.678+0000 7fd03d238740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5568f441c4b0 con 0x5568f4413e40 2024-04-21T08:38:11.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.678+0000 7fd03d238740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5568f4301190 con 0x5568f4414830 2024-04-21T08:38:11.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.678+0000 7fd03d238740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5568f441c630 con 0x5568f44152d0 2024-04-21T08:38:11.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.678+0000 7fd03d238740 10 client.0 _ll_register_callbacks cb 0x5568f42fc8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:11.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.678+0000 7fd033fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568f4413e40 0x5568f441deb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.678+0000 7fd033fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568f4413e40 0x5568f441deb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36446/0 (socket says 172.21.15.86:36446) 2024-04-21T08:38:11.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.678+0000 7fd033fff640 1 -- 172.21.15.86:0/4084196633 learned_addr learned my addr 172.21.15.86:0/4084196633 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:11.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.678+0000 7fd038b04640 1 --2- 172.21.15.86:0/4084196633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f44152d0 0x5568f4425430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.678+0000 7fd0337fe640 1 --2- 172.21.15.86:0/4084196633 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5568f4414830 0x5568f441e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.678+0000 7fd033fff640 1 -- 172.21.15.86:0/4084196633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f44152d0 msgr2=0x5568f4425430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:11.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.678+0000 7fd033fff640 1 --2- 172.21.15.86:0/4084196633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f44152d0 0x5568f4425430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.678+0000 7fd033fff640 1 -- 172.21.15.86:0/4084196633 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5568f4414830 msgr2=0x5568f441e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:11.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.678+0000 7fd033fff640 1 --2- 172.21.15.86:0/4084196633 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5568f4414830 0x5568f441e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.678+0000 7fd033fff640 1 -- 172.21.15.86:0/4084196633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568f41556b0 con 0x5568f4413e40 2024-04-21T08:38:11.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.678+0000 7fd033fff640 1 --2- 172.21.15.86:0/4084196633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568f4413e40 0x5568f441deb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd024005fd0 tx=0x7fd0240053d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f06c6d1ffb32b400 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:11.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.678+0000 7fd0137fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:38:11.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.678+0000 7fd0137fe640 1 -- 172.21.15.86:0/4084196633 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02401d040 con 0x5568f4413e40 2024-04-21T08:38:11.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.678+0000 7fd0137fe640 1 -- 172.21.15.86:0/4084196633 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fd024007ca0 con 0x5568f4413e40 2024-04-21T08:38:11.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.678+0000 7fd03d238740 1 -- 172.21.15.86:0/4084196633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd02c019410 con 0x5568f4413e40 2024-04-21T08:38:11.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.679+0000 7fd0137fe640 1 -- 172.21.15.86:0/4084196633 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd0240a5140 con 0x5568f4413e40 2024-04-21T08:38:11.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.679+0000 7fd03d238740 10 client.15264 fetch_fsmap learned FSMap version 28 2024-04-21T08:38:11.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.679+0000 7fd03d238740 1 -- 172.21.15.86:0/4084196633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5568f4428830 con 0x5568f4413e40 2024-04-21T08:38:11.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.679+0000 7fd0137fe640 1 -- 172.21.15.86:0/4084196633 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd02c019410 con 0x5568f4413e40 2024-04-21T08:38:11.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.679+0000 7fd0137fe640 1 -- 172.21.15.86:0/4084196633 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd02408cc70 con 0x5568f4413e40 2024-04-21T08:38:11.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.679+0000 7fd03d238740 10 client.15264 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:38:11.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.679+0000 7fd03d238740 10 client.15264 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-21T08:38:11.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.679+0000 7fd03d238740 20 client.15264 populate_metadata read hostname 'smithi086' 2024-04-21T08:38:11.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.679+0000 7fd03d238740 1 --2- 172.21.15.86:0/4084196633 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x5568f442ae30 0x5568f442c2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.679+0000 7fd03d238740 4 client.15264 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:38:11.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.679+0000 7fd03d238740 1 -- 172.21.15.86:0/4084196633 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5568f4386530 con 0x5568f442ae30 2024-04-21T08:38:11.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.679+0000 7fd0337fe640 1 --2- 172.21.15.86:0/4084196633 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x5568f442ae30 0x5568f442c2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.679+0000 7fd0337fe640 1 --2- 172.21.15.86:0/4084196633 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x5568f442ae30 0x5568f442c2a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6c7e3df3f634411 server_cookie=e139075749415d4e in_seq=0 out_seq=0 2024-04-21T08:38:11.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.679+0000 7fd0137fe640 10 client.15264 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:38:11.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.680+0000 7fd0137fe640 1 -- 172.21.15.86:0/4084196633 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5568f4386530 con 0x5568f442ae30 2024-04-21T08:38:11.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.680+0000 7fd0137fe640 10 client.15264 handle_command_reply: tid=0 2024-04-21T08:38:11.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.680+0000 7fd03d238740 1 client.15264 shutdown 2024-04-21T08:38:11.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.680+0000 7fd03d238740 1 -- 172.21.15.86:0/4084196633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568f4413e40 msgr2=0x5568f441deb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.680+0000 7fd03d238740 1 --2- 172.21.15.86:0/4084196633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568f4413e40 0x5568f441deb0 secure :-1 s=READY pgs=391 cs=0 l=1 rev1=1 crypto rx=0x7fd024005fd0 tx=0x7fd0240053d0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.680+0000 7fd03d238740 1 -- 172.21.15.86:0/4084196633 shutdown_connections 2024-04-21T08:38:11.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.680+0000 7fd03d238740 1 --2- 172.21.15.86:0/4084196633 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x5568f442ae30 0x5568f442c2a0 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.680+0000 7fd03d238740 1 --2- 172.21.15.86:0/4084196633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f44152d0 0x5568f4425430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.680+0000 7fd03d238740 1 --2- 172.21.15.86:0/4084196633 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5568f4414830 0x5568f441e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.680+0000 7fd03d238740 1 --2- 172.21.15.86:0/4084196633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568f4413e40 0x5568f441deb0 unknown :-1 s=CLOSED pgs=391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.680+0000 7fd0137fe640 0 client.15264 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:38:11.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.680+0000 7fd03d238740 1 -- 172.21.15.86:0/4084196633 >> 172.21.15.86:0/4084196633 conn(0x5568f440ff40 msgr2=0x5568f440da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:11.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.681+0000 7fd03d238740 1 -- 172.21.15.86:0/4084196633 shutdown_connections 2024-04-21T08:38:11.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.681+0000 7fd03d238740 1 -- 172.21.15.86:0/4084196633 wait complete. 2024-04-21T08:38:11.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.681+0000 7fd03d238740 20 client.15264 trim_cache size 0 max 16384 2024-04-21T08:38:11.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.698+0000 7fd03d238740 1 Processor -- start 2024-04-21T08:38:11.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.698+0000 7fd03d238740 1 -- start start 2024-04-21T08:38:11.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.699+0000 7fd03d238740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f4413e40 0x5568f44187d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.699+0000 7fd03d238740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5568f4414830 0x5568f4418d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.699+0000 7fd03d238740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568f44152d0 0x5568f44253c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.699+0000 7fd03d238740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5568f442a5e0 con 0x5568f44152d0 2024-04-21T08:38:11.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.699+0000 7fd03d238740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5568f442acb0 con 0x5568f4414830 2024-04-21T08:38:11.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.699+0000 7fd03d238740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5568f442b380 con 0x5568f4413e40 2024-04-21T08:38:11.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.699+0000 7fd038b04640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568f44152d0 0x5568f44253c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.699+0000 7fd038b04640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568f44152d0 0x5568f44253c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36462/0 (socket says 172.21.15.86:36462) 2024-04-21T08:38:11.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.699+0000 7fd038b04640 1 -- 172.21.15.86:0/2320193343 learned_addr learned my addr 172.21.15.86:0/2320193343 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:11.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.699+0000 7fd033fff640 1 --2- 172.21.15.86:0/2320193343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f4413e40 0x5568f44187d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.699+0000 7fd0337fe640 1 --2- 172.21.15.86:0/2320193343 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5568f4414830 0x5568f4418d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.699+0000 7fd038b04640 1 -- 172.21.15.86:0/2320193343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f4413e40 msgr2=0x5568f44187d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:11.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.699+0000 7fd038b04640 1 --2- 172.21.15.86:0/2320193343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f4413e40 0x5568f44187d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.699+0000 7fd038b04640 1 -- 172.21.15.86:0/2320193343 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5568f4414830 msgr2=0x5568f4418d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:11.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.699+0000 7fd038b04640 1 --2- 172.21.15.86:0/2320193343 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5568f4414830 0x5568f4418d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.699+0000 7fd038b04640 1 -- 172.21.15.86:0/2320193343 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568f440f230 con 0x5568f44152d0 2024-04-21T08:38:11.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.699+0000 7fd033fff640 1 --2- 172.21.15.86:0/2320193343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f4413e40 0x5568f44187d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:11.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.699+0000 7fd0337fe640 1 --2- 172.21.15.86:0/2320193343 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5568f4414830 0x5568f4418d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:11.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.699+0000 7fd038b04640 1 --2- 172.21.15.86:0/2320193343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568f44152d0 0x5568f44253c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd02c004ab0 tx=0x7fd02c005f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d28339fe809f677d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:11.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.699+0000 7fd019ffb640 1 -- 172.21.15.86:0/2320193343 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c034070 con 0x5568f44152d0 2024-04-21T08:38:11.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.699+0000 7fd019ffb640 1 -- 172.21.15.86:0/2320193343 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd02c011440 con 0x5568f44152d0 2024-04-21T08:38:11.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.699+0000 7fd019ffb640 1 -- 172.21.15.86:0/2320193343 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02c01aaa0 con 0x5568f44152d0 2024-04-21T08:38:11.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.700+0000 7fd03d238740 1 -- 172.21.15.86:0/2320193343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568f44152d0 msgr2=0x5568f44253c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.700+0000 7fd03d238740 1 --2- 172.21.15.86:0/2320193343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568f44152d0 0x5568f44253c0 secure :-1 s=READY pgs=392 cs=0 l=1 rev1=1 crypto rx=0x7fd02c004ab0 tx=0x7fd02c005f50 comp rx=0 tx=0).stop 2024-04-21T08:38:11.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.700+0000 7fd03d238740 1 -- 172.21.15.86:0/2320193343 shutdown_connections 2024-04-21T08:38:11.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.700+0000 7fd03d238740 1 --2- 172.21.15.86:0/2320193343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568f44152d0 0x5568f44253c0 unknown :-1 s=CLOSED pgs=392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.700+0000 7fd03d238740 1 --2- 172.21.15.86:0/2320193343 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5568f4414830 0x5568f4418d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.700+0000 7fd03d238740 1 --2- 172.21.15.86:0/2320193343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f4413e40 0x5568f44187d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.700+0000 7fd03d238740 1 -- 172.21.15.86:0/2320193343 >> 172.21.15.86:0/2320193343 conn(0x5568f440ff40 msgr2=0x5568f4409800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:11.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.700+0000 7fd03d238740 1 -- 172.21.15.86:0/2320193343 shutdown_connections 2024-04-21T08:38:11.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.700+0000 7fd03d238740 1 -- 172.21.15.86:0/2320193343 wait complete. 2024-04-21T08:38:11.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.700+0000 7fd03d238740 1 -- start start 2024-04-21T08:38:11.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.700+0000 7fd03d238740 1 Processor -- start 2024-04-21T08:38:11.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.701+0000 7fd03d238740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568f4413e40 0x5568f44209f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.701+0000 7fd03d238740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f4414830 0x5568f4420f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.701+0000 7fd03d238740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5568f44152d0 0x5568f4423f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.701+0000 7fd03d238740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5568f442c120 con 0x5568f4413e40 2024-04-21T08:38:11.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.701+0000 7fd03d238740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5568f442ba50 con 0x5568f44152d0 2024-04-21T08:38:11.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.701+0000 7fd03d238740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5568f442c7f0 con 0x5568f4414830 2024-04-21T08:38:11.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.701+0000 7fd03d238740 10 client.0 _ll_register_callbacks cb 0x7fd024005760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:11.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.701+0000 7fd033fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568f4413e40 0x5568f44209f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.701+0000 7fd033fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568f4413e40 0x5568f44209f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36470/0 (socket says 172.21.15.86:36470) 2024-04-21T08:38:11.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.701+0000 7fd033fff640 1 -- 172.21.15.86:0/3967153475 learned_addr learned my addr 172.21.15.86:0/3967153475 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:11.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.701+0000 7fd033fff640 1 -- 172.21.15.86:0/3967153475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f4414830 msgr2=0x5568f4420f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:11.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.701+0000 7fd033fff640 1 --2- 172.21.15.86:0/3967153475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f4414830 0x5568f4420f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.701+0000 7fd033fff640 1 -- 172.21.15.86:0/3967153475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5568f44152d0 msgr2=0x5568f4423f50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:11.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.701+0000 7fd033fff640 1 --2- 172.21.15.86:0/3967153475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5568f44152d0 0x5568f4423f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.701+0000 7fd033fff640 1 -- 172.21.15.86:0/3967153475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5568f4424540 con 0x5568f4413e40 2024-04-21T08:38:11.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.701+0000 7fd033fff640 1 --2- 172.21.15.86:0/3967153475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568f4413e40 0x5568f44209f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd024098ea0 tx=0x7fd02409c010 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a2d2ac7edf47b61 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:11.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.701+0000 7fd0137fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:38:11.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.701+0000 7fd0137fe640 1 -- 172.21.15.86:0/3967153475 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd02401d040 con 0x5568f4413e40 2024-04-21T08:38:11.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.701+0000 7fd0137fe640 1 -- 172.21.15.86:0/3967153475 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fd02409bb20 con 0x5568f4413e40 2024-04-21T08:38:11.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.701+0000 7fd03d238740 1 -- 172.21.15.86:0/3967153475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd02c011440 con 0x5568f4413e40 2024-04-21T08:38:11.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.701+0000 7fd0137fe640 1 -- 172.21.15.86:0/3967153475 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fd02409be30 con 0x5568f4413e40 2024-04-21T08:38:11.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.701+0000 7fd03d238740 10 client.15270 fetch_fsmap learned FSMap version 28 2024-04-21T08:38:11.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.701+0000 7fd03d238740 1 -- 172.21.15.86:0/3967153475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5568f4425820 con 0x5568f4413e40 2024-04-21T08:38:11.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.702+0000 7fd0137fe640 1 -- 172.21.15.86:0/3967153475 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd02c011440 con 0x5568f4413e40 2024-04-21T08:38:11.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.702+0000 7fd0137fe640 1 -- 172.21.15.86:0/3967153475 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fd0240bb2b0 con 0x5568f4413e40 2024-04-21T08:38:11.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.702+0000 7fd03d238740 10 client.15270 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:38:11.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.702+0000 7fd03d238740 10 client.15270 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-21T08:38:11.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.702+0000 7fd03d238740 20 client.15270 populate_metadata read hostname 'smithi086' 2024-04-21T08:38:11.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.702+0000 7fd03d238740 1 --2- 172.21.15.86:0/3967153475 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x5568f44956c0 0x5568f44943d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.702+0000 7fd03d238740 4 client.15270 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:38:11.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.702+0000 7fd03d238740 1 -- 172.21.15.86:0/3967153475 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "status"}) -- 0x5568f43803a0 con 0x5568f44956c0 2024-04-21T08:38:11.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.702+0000 7fd0337fe640 1 --2- 172.21.15.86:0/3967153475 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x5568f44956c0 0x5568f44943d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.702+0000 7fd0337fe640 1 --2- 172.21.15.86:0/3967153475 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x5568f44956c0 0x5568f44943d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=866e7846e37d2d50 server_cookie=d5eb231528ef90a2 in_seq=0 out_seq=0 2024-04-21T08:38:11.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.702+0000 7fd0137fe640 10 client.15270 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:38:11.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.703+0000 7fd0137fe640 1 -- 172.21.15.86:0/3967153475 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5568f43803a0 con 0x5568f44956c0 2024-04-21T08:38:11.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.703+0000 7fd0137fe640 10 client.15270 handle_command_reply: tid=0 2024-04-21T08:38:11.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.703+0000 7fd03d238740 1 client.15270 shutdown 2024-04-21T08:38:11.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.703+0000 7fd03d238740 1 -- 172.21.15.86:0/3967153475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568f4413e40 msgr2=0x5568f44209f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.703+0000 7fd03d238740 1 --2- 172.21.15.86:0/3967153475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568f4413e40 0x5568f44209f0 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7fd024098ea0 tx=0x7fd02409c010 comp rx=0 tx=0).stop 2024-04-21T08:38:11.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.703+0000 7fd03d238740 1 -- 172.21.15.86:0/3967153475 shutdown_connections 2024-04-21T08:38:11.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.703+0000 7fd03d238740 1 --2- 172.21.15.86:0/3967153475 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x5568f44956c0 0x5568f44943d0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.703+0000 7fd03d238740 1 --2- 172.21.15.86:0/3967153475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5568f44152d0 0x5568f4423f50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.703+0000 7fd0137fe640 0 client.15270 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:38:11.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.703+0000 7fd03d238740 1 --2- 172.21.15.86:0/3967153475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5568f4414830 0x5568f4420f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.703+0000 7fd03d238740 1 --2- 172.21.15.86:0/3967153475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5568f4413e40 0x5568f44209f0 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.703+0000 7fd03d238740 1 -- 172.21.15.86:0/3967153475 >> 172.21.15.86:0/3967153475 conn(0x5568f440ff40 msgr2=0x5568f44187d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:11.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.703+0000 7fd03d238740 1 -- 172.21.15.86:0/3967153475 shutdown_connections 2024-04-21T08:38:11.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.703+0000 7fd03d238740 1 -- 172.21.15.86:0/3967153475 wait complete. 2024-04-21T08:38:11.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.703+0000 7fd03d238740 20 client.15270 trim_cache size 0 max 16384 2024-04-21T08:38:11.713 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:38:11.713 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "f8cec7cc-ffb8-11ee-bc93-c7b262605968", 2024-04-21T08:38:11.713 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 3, 2024-04-21T08:38:11.713 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24398, 2024-04-21T08:38:11.713 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-21T08:38:11.713 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-21T08:38:11.714 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-21T08:38:11.714 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 86.267169541000001, 2024-04-21T08:38:11.714 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 28, 2024-04-21T08:38:11.714 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 91, 2024-04-21T08:38:11.714 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 89, 2024-04-21T08:38:11.714 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 145.658103506 2024-04-21T08:38:11.714 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:38:11.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.703+0000 7fd03a567640 1 -- 172.21.15.86:0/3438933770 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fd01c0a3bb0 msgr2=0x7fd01c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.703+0000 7fd03a567640 1 --2- 172.21.15.86:0/3438933770 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fd01c0a3bb0 0x7fd01c0a6070 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fd02c01e000 tx=0x7fd02c01f000 comp rx=0 tx=0).stop 2024-04-21T08:38:11.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.703+0000 7fd03a567640 1 -- 172.21.15.86:0/3438933770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd034102a10 msgr2=0x7fd034198220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.703+0000 7fd03a567640 1 --2- 172.21.15.86:0/3438933770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd034102a10 0x7fd034198220 secure :-1 s=READY pgs=389 cs=0 l=1 rev1=1 crypto rx=0x7fd02400c890 tx=0x7fd02400cd50 comp rx=0 tx=0).stop 2024-04-21T08:38:11.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.703+0000 7fd03a567640 1 -- 172.21.15.86:0/3438933770 shutdown_connections 2024-04-21T08:38:11.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.703+0000 7fd03a567640 1 --2- 172.21.15.86:0/3438933770 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fd01c0a3bb0 0x7fd01c0a6070 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.703+0000 7fd03a567640 1 --2- 172.21.15.86:0/3438933770 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd034107c70 0x7fd03419f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.703+0000 7fd03a567640 1 --2- 172.21.15.86:0/3438933770 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd034105340 0x7fd034198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.703+0000 7fd03a567640 1 --2- 172.21.15.86:0/3438933770 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd034102a10 0x7fd034198220 unknown :-1 s=CLOSED pgs=389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.703+0000 7fd03a567640 1 -- 172.21.15.86:0/3438933770 >> 172.21.15.86:0/3438933770 conn(0x7fd0340fc440 msgr2=0x7fd0341034b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:11.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.704+0000 7fd03a567640 1 -- 172.21.15.86:0/3438933770 shutdown_connections 2024-04-21T08:38:11.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.704+0000 7fd03a567640 1 -- 172.21.15.86:0/3438933770 wait complete. 2024-04-21T08:38:11.716 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:11 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/4143416052' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:11.716 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:11 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/450586032' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:11.716 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:11 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/4092817116' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:11.717 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:11 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/4143416052' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:11.717 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:11 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/450586032' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:11.717 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:11 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/4092817116' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:11.718 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-21T08:38:11.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:11 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/4143416052' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:11.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:11 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/450586032' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:11.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:11 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/4092817116' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:11.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.818+0000 7f29a491e640 1 -- 172.21.15.86:0/714488661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a01032f0 msgr2=0x7f29a0103770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.818+0000 7f29a491e640 1 --2- 172.21.15.86:0/714488661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a01032f0 0x7f29a0103770 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7f2988009930 tx=0x7f298802bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:11.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.818+0000 7f29a491e640 1 -- 172.21.15.86:0/714488661 shutdown_connections 2024-04-21T08:38:11.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.818+0000 7f29a491e640 1 --2- 172.21.15.86:0/714488661 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29a0103cb0 0x7f29a010a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.818+0000 7f29a491e640 1 --2- 172.21.15.86:0/714488661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a01032f0 0x7f29a0103770 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.818+0000 7f29a491e640 1 --2- 172.21.15.86:0/714488661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a0100280 0x7f29a01006a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.818+0000 7f29a491e640 1 -- 172.21.15.86:0/714488661 >> 172.21.15.86:0/714488661 conn(0x7f29a00fbb90 msgr2=0x7f29a00fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:11.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.818+0000 7f29a491e640 1 -- 172.21.15.86:0/714488661 shutdown_connections 2024-04-21T08:38:11.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.818+0000 7f29a491e640 1 -- 172.21.15.86:0/714488661 wait complete. 2024-04-21T08:38:11.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.818+0000 7f29a491e640 1 Processor -- start 2024-04-21T08:38:11.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.819+0000 7f29a491e640 1 -- start start 2024-04-21T08:38:11.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.819+0000 7f29a491e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a0100280 0x7f29a01982d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.819+0000 7f29a491e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29a01032f0 0x7f29a0198810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.819+0000 7f29a491e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a0103cb0 0x7f29a019f880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.819+0000 7f29a491e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29a010db50 con 0x7f29a0103cb0 2024-04-21T08:38:11.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.819+0000 7f29a491e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f29a010d9d0 con 0x7f29a01032f0 2024-04-21T08:38:11.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.819+0000 7f29a491e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f29a010dcd0 con 0x7f29a0100280 2024-04-21T08:38:11.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.819+0000 7f299ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a0103cb0 0x7f29a019f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.819+0000 7f299ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a0103cb0 0x7f29a019f880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36480/0 (socket says 172.21.15.86:36480) 2024-04-21T08:38:11.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.819+0000 7f299ed9d640 1 -- 172.21.15.86:0/689617917 learned_addr learned my addr 172.21.15.86:0/689617917 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:11.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.819+0000 7f2997fff640 1 --2- 172.21.15.86:0/689617917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29a01032f0 0x7f29a0198810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.819+0000 7f299e59c640 1 --2- 172.21.15.86:0/689617917 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a0100280 0x7f29a01982d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.819+0000 7f299ed9d640 1 -- 172.21.15.86:0/689617917 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a0100280 msgr2=0x7f29a01982d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.819+0000 7f299ed9d640 1 --2- 172.21.15.86:0/689617917 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a0100280 0x7f29a01982d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.819+0000 7f299ed9d640 1 -- 172.21.15.86:0/689617917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29a01032f0 msgr2=0x7f29a0198810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.819+0000 7f299ed9d640 1 --2- 172.21.15.86:0/689617917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29a01032f0 0x7f29a0198810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.819+0000 7f299ed9d640 1 -- 172.21.15.86:0/689617917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a019fef0 con 0x7f29a0103cb0 2024-04-21T08:38:11.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.819+0000 7f299e59c640 1 --2- 172.21.15.86:0/689617917 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a0100280 0x7f29a01982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:11.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.819+0000 7f299ed9d640 1 --2- 172.21.15.86:0/689617917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a0103cb0 0x7f29a019f880 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f299800cad0 tx=0x7f299800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:11.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.819+0000 7f29977fe640 1 -- 172.21.15.86:0/689617917 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299800ae20 con 0x7f29a0103cb0 2024-04-21T08:38:11.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.819+0000 7f29a491e640 1 -- 172.21.15.86:0/689617917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29a01a01e0 con 0x7f29a0103cb0 2024-04-21T08:38:11.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.819+0000 7f29977fe640 1 -- 172.21.15.86:0/689617917 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2998005020 con 0x7f29a0103cb0 2024-04-21T08:38:11.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.819+0000 7f29977fe640 1 -- 172.21.15.86:0/689617917 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299800f6b0 con 0x7f29a0103cb0 2024-04-21T08:38:11.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.820+0000 7f29a491e640 1 -- 172.21.15.86:0/689617917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29a01a0780 con 0x7f29a0103cb0 2024-04-21T08:38:11.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.822+0000 7f29977fe640 1 -- 172.21.15.86:0/689617917 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f29980051d0 con 0x7f29a0103cb0 2024-04-21T08:38:11.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.822+0000 7f29a75df740 1 Processor -- start 2024-04-21T08:38:11.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.822+0000 7f29a75df740 1 -- start start 2024-04-21T08:38:11.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.822+0000 7f29a75df740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b83d9cde40 0x55b83d9ce260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.822+0000 7f29a75df740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b83d9ce830 0x55b83d9cecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.823+0000 7f29a75df740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83d9cf2d0 0x55b83d9d5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.823+0000 7f29a75df740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b83d932b10 con 0x55b83d9cf2d0 2024-04-21T08:38:11.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.823+0000 7f29a75df740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b83d8d0240 con 0x55b83d9cde40 2024-04-21T08:38:11.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.823+0000 7f29a75df740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b83d92ead0 con 0x55b83d9ce830 2024-04-21T08:38:11.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.823+0000 7f29977fe640 1 --2- 172.21.15.86:0/689617917 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f298c0a3bb0 0x7f298c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.823+0000 7f2997fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83d9cf2d0 0x55b83d9d5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.823+0000 7f2997fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83d9cf2d0 0x55b83d9d5eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36496/0 (socket says 172.21.15.86:36496) 2024-04-21T08:38:11.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.823+0000 7f2997fff640 1 -- 172.21.15.86:0/606918296 learned_addr learned my addr 172.21.15.86:0/606918296 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:11.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.823+0000 7f299ed9d640 1 --2- 172.21.15.86:0/689617917 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f298c0a3bb0 0x7f298c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.823+0000 7f2997fff640 1 --2- 172.21.15.86:0/606918296 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b83d9cde40 0x55b83d9ce260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.823+0000 7f299e59c640 1 --2- 172.21.15.86:0/606918296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b83d9ce830 0x55b83d9cecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.823+0000 7f2997fff640 1 -- 172.21.15.86:0/606918296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b83d9ce830 msgr2=0x55b83d9cecb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:11.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.823+0000 7f2997fff640 1 --2- 172.21.15.86:0/606918296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b83d9ce830 0x55b83d9cecb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.823+0000 7f29977fe640 1 -- 172.21.15.86:0/689617917 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f29980d07e0 con 0x7f29a0103cb0 2024-04-21T08:38:11.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.823+0000 7f2997fff640 1 -- 172.21.15.86:0/606918296 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b83d9cde40 msgr2=0x55b83d9ce260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:11.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.823+0000 7f2997fff640 1 --2- 172.21.15.86:0/606918296 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b83d9cde40 0x55b83d9ce260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.823+0000 7f2997fff640 1 -- 172.21.15.86:0/606918296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b83d6452d0 con 0x55b83d9cf2d0 2024-04-21T08:38:11.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.823+0000 7f2997fff640 1 --2- 172.21.15.86:0/606918296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83d9cf2d0 0x55b83d9d5eb0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2988005ec0 tx=0x7f2988009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17427be13e9c8d32 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:11.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.823+0000 7f2994ff9640 1 -- 172.21.15.86:0/606918296 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2988047020 con 0x55b83d9cf2d0 2024-04-21T08:38:11.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.823+0000 7f2994ff9640 1 -- 172.21.15.86:0/606918296 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2988005040 con 0x55b83d9cf2d0 2024-04-21T08:38:11.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.823+0000 7f299ed9d640 1 --2- 172.21.15.86:0/689617917 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f298c0a3bb0 0x7f298c0a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f299800a6e0 tx=0x7f299806f2a0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:11.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.823+0000 7f2994ff9640 1 -- 172.21.15.86:0/606918296 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2988004a30 con 0x55b83d9cf2d0 2024-04-21T08:38:11.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.824+0000 7f29a75df740 1 -- 172.21.15.86:0/606918296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83d9cf2d0 msgr2=0x55b83d9d5eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.824+0000 7f29a75df740 1 --2- 172.21.15.86:0/606918296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83d9cf2d0 0x55b83d9d5eb0 secure :-1 s=READY pgs=396 cs=0 l=1 rev1=1 crypto rx=0x7f2988005ec0 tx=0x7f2988009610 comp rx=0 tx=0).stop 2024-04-21T08:38:11.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.824+0000 7f29a75df740 1 -- 172.21.15.86:0/606918296 shutdown_connections 2024-04-21T08:38:11.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.824+0000 7f29a75df740 1 --2- 172.21.15.86:0/606918296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83d9cf2d0 0x55b83d9d5eb0 unknown :-1 s=CLOSED pgs=396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.824+0000 7f29a75df740 1 --2- 172.21.15.86:0/606918296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b83d9ce830 0x55b83d9cecb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.824+0000 7f29a75df740 1 --2- 172.21.15.86:0/606918296 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b83d9cde40 0x55b83d9ce260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.824+0000 7f29a75df740 1 -- 172.21.15.86:0/606918296 >> 172.21.15.86:0/606918296 conn(0x55b83d9c9f40 msgr2=0x55b83d9c8490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:11.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.824+0000 7f29a75df740 1 -- 172.21.15.86:0/606918296 shutdown_connections 2024-04-21T08:38:11.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.824+0000 7f29a75df740 1 -- 172.21.15.86:0/606918296 wait complete. 2024-04-21T08:38:11.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.824+0000 7f29a75df740 1 -- start start 2024-04-21T08:38:11.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.825+0000 7f29a75df740 1 Processor -- start 2024-04-21T08:38:11.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.825+0000 7f29a75df740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b83d9cde40 0x55b83d9d7e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.825+0000 7f29a75df740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83d9ce830 0x55b83d9d83d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.825+0000 7f29a75df740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b83d9cf2d0 0x55b83d9df410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.825+0000 7f29a75df740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b83d9d6450 con 0x55b83d9ce830 2024-04-21T08:38:11.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.825+0000 7f29a75df740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b83d8bb190 con 0x55b83d9cf2d0 2024-04-21T08:38:11.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.825+0000 7f29a75df740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b83d9d65d0 con 0x55b83d9cde40 2024-04-21T08:38:11.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.825+0000 7f29a75df740 10 client.0 _ll_register_callbacks cb 0x55b83d8b68b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:11.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.825+0000 7f299e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83d9ce830 0x55b83d9d83d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.825+0000 7f299e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83d9ce830 0x55b83d9d83d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36510/0 (socket says 172.21.15.86:36510) 2024-04-21T08:38:11.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.825+0000 7f299e59c640 1 -- 172.21.15.86:0/83939390 learned_addr learned my addr 172.21.15.86:0/83939390 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:11.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.825+0000 7f2997fff640 1 --2- 172.21.15.86:0/83939390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b83d9cde40 0x55b83d9d7e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.825+0000 7f2997fff640 1 --2- 172.21.15.86:0/83939390 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b83d9cf2d0 0x55b83d9df410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.825+0000 7f299e59c640 1 -- 172.21.15.86:0/83939390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b83d9cde40 msgr2=0x55b83d9d7e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:11.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.825+0000 7f299e59c640 1 --2- 172.21.15.86:0/83939390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b83d9cde40 0x55b83d9d7e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.825+0000 7f299e59c640 1 -- 172.21.15.86:0/83939390 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b83d9cf2d0 msgr2=0x55b83d9df410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:11.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.825+0000 7f299e59c640 1 --2- 172.21.15.86:0/83939390 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b83d9cf2d0 0x55b83d9df410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.825+0000 7f299e59c640 1 -- 172.21.15.86:0/83939390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b83d70f6b0 con 0x55b83d9ce830 2024-04-21T08:38:11.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.825+0000 7f2997fff640 1 --2- 172.21.15.86:0/83939390 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b83d9cf2d0 0x55b83d9df410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:11.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.825+0000 7f299e59c640 1 --2- 172.21.15.86:0/83939390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83d9ce830 0x55b83d9d83d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f299000bf80 tx=0x7f299000f670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=483385a8bc4b2cdd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:11.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.825+0000 7f2985ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:38:11.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.825+0000 7f2985ffb640 1 -- 172.21.15.86:0/83939390 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2990018020 con 0x55b83d9ce830 2024-04-21T08:38:11.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.825+0000 7f2985ffb640 1 -- 172.21.15.86:0/83939390 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f2990014670 con 0x55b83d9ce830 2024-04-21T08:38:11.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.825+0000 7f29a75df740 1 -- 172.21.15.86:0/83939390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2988005040 con 0x55b83d9ce830 2024-04-21T08:38:11.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.826+0000 7f2985ffb640 1 -- 172.21.15.86:0/83939390 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2990023050 con 0x55b83d9ce830 2024-04-21T08:38:11.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.826+0000 7f29a75df740 10 client.15282 fetch_fsmap learned FSMap version 28 2024-04-21T08:38:11.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.826+0000 7f29a75df740 1 -- 172.21.15.86:0/83939390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b83d9e28a0 con 0x55b83d9ce830 2024-04-21T08:38:11.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.826+0000 7f2985ffb640 1 -- 172.21.15.86:0/83939390 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2988005040 con 0x55b83d9ce830 2024-04-21T08:38:11.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.826+0000 7f2985ffb640 1 -- 172.21.15.86:0/83939390 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2990013a10 con 0x55b83d9ce830 2024-04-21T08:38:11.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.826+0000 7f29a75df740 10 client.15282 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:38:11.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.826+0000 7f29a75df740 10 client.15282 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-21T08:38:11.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.826+0000 7f29a75df740 20 client.15282 populate_metadata read hostname 'smithi086' 2024-04-21T08:38:11.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.826+0000 7f29a75df740 1 --2- 172.21.15.86:0/83939390 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55b83d9e4fa0 0x55b83d9e6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.826+0000 7f29a75df740 4 client.15282 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:38:11.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.826+0000 7f29a75df740 1 -- 172.21.15.86:0/83939390 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b83d940530 con 0x55b83d9e4fa0 2024-04-21T08:38:11.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.826+0000 7f2997fff640 1 --2- 172.21.15.86:0/83939390 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55b83d9e4fa0 0x55b83d9e6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.827+0000 7f2985ffb640 10 client.15282 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:38:11.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.827+0000 7f2997fff640 1 --2- 172.21.15.86:0/83939390 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55b83d9e4fa0 0x55b83d9e6410 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c2b817eafd778327 server_cookie=98fb5c0cd8c569a in_seq=0 out_seq=0 2024-04-21T08:38:11.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.828+0000 7f2985ffb640 1 -- 172.21.15.86:0/83939390 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b83d940530 con 0x55b83d9e4fa0 2024-04-21T08:38:11.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.828+0000 7f2985ffb640 10 client.15282 handle_command_reply: tid=0 2024-04-21T08:38:11.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.828+0000 7f29a75df740 1 client.15282 shutdown 2024-04-21T08:38:11.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.828+0000 7f29a75df740 1 -- 172.21.15.86:0/83939390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83d9ce830 msgr2=0x55b83d9d83d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.828+0000 7f29a75df740 1 --2- 172.21.15.86:0/83939390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83d9ce830 0x55b83d9d83d0 secure :-1 s=READY pgs=397 cs=0 l=1 rev1=1 crypto rx=0x7f299000bf80 tx=0x7f299000f670 comp rx=0 tx=0).stop 2024-04-21T08:38:11.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.828+0000 7f29a75df740 1 -- 172.21.15.86:0/83939390 shutdown_connections 2024-04-21T08:38:11.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.828+0000 7f29a75df740 1 --2- 172.21.15.86:0/83939390 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55b83d9e4fa0 0x55b83d9e6410 crc :-1 s=READY pgs=15 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.828+0000 7f29a75df740 1 --2- 172.21.15.86:0/83939390 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b83d9cf2d0 0x55b83d9df410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.828+0000 7f29a75df740 1 --2- 172.21.15.86:0/83939390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83d9ce830 0x55b83d9d83d0 unknown :-1 s=CLOSED pgs=397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.828+0000 7f2985ffb640 0 client.15282 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:38:11.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.828+0000 7f29a75df740 1 --2- 172.21.15.86:0/83939390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b83d9cde40 0x55b83d9d7e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.828+0000 7f29a75df740 1 -- 172.21.15.86:0/83939390 >> 172.21.15.86:0/83939390 conn(0x55b83d9c9f40 msgr2=0x55b83d9c7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:11.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.828+0000 7f29a75df740 1 -- 172.21.15.86:0/83939390 shutdown_connections 2024-04-21T08:38:11.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.828+0000 7f29a75df740 1 -- 172.21.15.86:0/83939390 wait complete. 2024-04-21T08:38:11.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.828+0000 7f29a75df740 20 client.15282 trim_cache size 0 max 16384 2024-04-21T08:38:11.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.845+0000 7f29a75df740 1 Processor -- start 2024-04-21T08:38:11.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.845+0000 7f29a75df740 1 -- start start 2024-04-21T08:38:11.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.846+0000 7f29a75df740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b83d9cde40 0x55b83d9d2840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.846+0000 7f29a75df740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b83d9ce830 0x55b83d9d2d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.846+0000 7f29a75df740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83d9cf2d0 0x55b83d9df3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.846+0000 7f29a75df740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b83d9e4750 con 0x55b83d9cf2d0 2024-04-21T08:38:11.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.846+0000 7f29a75df740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b83d9e4e20 con 0x55b83d9cde40 2024-04-21T08:38:11.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.846+0000 7f29a75df740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b83d9e54f0 con 0x55b83d9ce830 2024-04-21T08:38:11.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.846+0000 7f2997fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83d9cf2d0 0x55b83d9df3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.846+0000 7f2997fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83d9cf2d0 0x55b83d9df3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36516/0 (socket says 172.21.15.86:36516) 2024-04-21T08:38:11.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.846+0000 7f2997fff640 1 -- 172.21.15.86:0/150751699 learned_addr learned my addr 172.21.15.86:0/150751699 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:11.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.846+0000 7f2997fff640 1 -- 172.21.15.86:0/150751699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b83d9ce830 msgr2=0x55b83d9d2d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:11.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.846+0000 7f2997fff640 1 --2- 172.21.15.86:0/150751699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b83d9ce830 0x55b83d9d2d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.846+0000 7f2997fff640 1 -- 172.21.15.86:0/150751699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b83d9cde40 msgr2=0x55b83d9d2840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:11.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.846+0000 7f2997fff640 1 --2- 172.21.15.86:0/150751699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b83d9cde40 0x55b83d9d2840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.846+0000 7f2997fff640 1 -- 172.21.15.86:0/150751699 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b83d9c9230 con 0x55b83d9cf2d0 2024-04-21T08:38:11.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.846+0000 7f2997fff640 1 --2- 172.21.15.86:0/150751699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83d9cf2d0 0x55b83d9df3d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2988009930 tx=0x7f2988009610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=395f4cc2d9d4b784 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:11.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.846+0000 7f2987fff640 1 -- 172.21.15.86:0/150751699 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298805a070 con 0x55b83d9cf2d0 2024-04-21T08:38:11.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.846+0000 7f2987fff640 1 -- 172.21.15.86:0/150751699 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2988005500 con 0x55b83d9cf2d0 2024-04-21T08:38:11.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.846+0000 7f2987fff640 1 -- 172.21.15.86:0/150751699 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2988047030 con 0x55b83d9cf2d0 2024-04-21T08:38:11.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.847+0000 7f29a75df740 1 -- 172.21.15.86:0/150751699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83d9cf2d0 msgr2=0x55b83d9df3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.847+0000 7f29a75df740 1 --2- 172.21.15.86:0/150751699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83d9cf2d0 0x55b83d9df3d0 secure :-1 s=READY pgs=398 cs=0 l=1 rev1=1 crypto rx=0x7f2988009930 tx=0x7f2988009610 comp rx=0 tx=0).stop 2024-04-21T08:38:11.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.847+0000 7f29a75df740 1 -- 172.21.15.86:0/150751699 shutdown_connections 2024-04-21T08:38:11.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.847+0000 7f29a75df740 1 --2- 172.21.15.86:0/150751699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83d9cf2d0 0x55b83d9df3d0 unknown :-1 s=CLOSED pgs=398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.847+0000 7f29a75df740 1 --2- 172.21.15.86:0/150751699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b83d9ce830 0x55b83d9d2d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.847+0000 7f29a75df740 1 --2- 172.21.15.86:0/150751699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b83d9cde40 0x55b83d9d2840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.847+0000 7f29a75df740 1 -- 172.21.15.86:0/150751699 >> 172.21.15.86:0/150751699 conn(0x55b83d9c9f40 msgr2=0x55b83d9d5e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:11.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.847+0000 7f29a75df740 1 -- 172.21.15.86:0/150751699 shutdown_connections 2024-04-21T08:38:11.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.847+0000 7f29a75df740 1 -- 172.21.15.86:0/150751699 wait complete. 2024-04-21T08:38:11.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.847+0000 7f29a75df740 1 -- start start 2024-04-21T08:38:11.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.847+0000 7f29a75df740 1 Processor -- start 2024-04-21T08:38:11.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.847+0000 7f29a75df740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83d9cde40 0x55b83d9db3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.847+0000 7f29a75df740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b83d9ce830 0x55b83d9dd900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.847+0000 7f29a75df740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b83d9cf2d0 0x55b83d9dde40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.847+0000 7f29a75df740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b83d9e6290 con 0x55b83d9cde40 2024-04-21T08:38:11.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.847+0000 7f29a75df740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b83d9e5bc0 con 0x55b83d9cf2d0 2024-04-21T08:38:11.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.847+0000 7f29a75df740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b83d9e6960 con 0x55b83d9ce830 2024-04-21T08:38:11.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.847+0000 7f29a75df740 10 client.0 _ll_register_callbacks cb 0x7f299000b2a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:11.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.847+0000 7f2997fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83d9cde40 0x55b83d9db3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.847+0000 7f2997fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83d9cde40 0x55b83d9db3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36524/0 (socket says 172.21.15.86:36524) 2024-04-21T08:38:11.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.848+0000 7f2997fff640 1 -- 172.21.15.86:0/679285713 learned_addr learned my addr 172.21.15.86:0/679285713 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:11.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.848+0000 7f2997fff640 1 -- 172.21.15.86:0/679285713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b83d9ce830 msgr2=0x55b83d9dd900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:11.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.848+0000 7f2997fff640 1 --2- 172.21.15.86:0/679285713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b83d9ce830 0x55b83d9dd900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.848+0000 7f2997fff640 1 -- 172.21.15.86:0/679285713 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b83d9cf2d0 msgr2=0x55b83d9dde40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:11.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.848+0000 7f2997fff640 1 --2- 172.21.15.86:0/679285713 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b83d9cf2d0 0x55b83d9dde40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.848+0000 7f2997fff640 1 -- 172.21.15.86:0/679285713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b83d9de380 con 0x55b83d9cde40 2024-04-21T08:38:11.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.848+0000 7f2997fff640 1 --2- 172.21.15.86:0/679285713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83d9cde40 0x55b83d9db3b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2988058040 tx=0x7f2988005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82facc6e331db3b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:11.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.848+0000 7f2985ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:38:11.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.848+0000 7f2985ffb640 1 -- 172.21.15.86:0/679285713 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2988047030 con 0x55b83d9cde40 2024-04-21T08:38:11.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.848+0000 7f2985ffb640 1 -- 172.21.15.86:0/679285713 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f2988055280 con 0x55b83d9cde40 2024-04-21T08:38:11.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.848+0000 7f29a75df740 1 -- 172.21.15.86:0/679285713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2988005500 con 0x55b83d9cde40 2024-04-21T08:38:11.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.848+0000 7f2985ffb640 1 -- 172.21.15.86:0/679285713 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f298805a070 con 0x55b83d9cde40 2024-04-21T08:38:11.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.848+0000 7f29a75df740 10 client.15288 fetch_fsmap learned FSMap version 28 2024-04-21T08:38:11.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.848+0000 7f29a75df740 1 -- 172.21.15.86:0/679285713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b83d9df680 con 0x55b83d9cde40 2024-04-21T08:38:11.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.848+0000 7f2985ffb640 1 -- 172.21.15.86:0/679285713 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2988005500 con 0x55b83d9cde40 2024-04-21T08:38:11.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.848+0000 7f2985ffb640 1 -- 172.21.15.86:0/679285713 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2988040040 con 0x55b83d9cde40 2024-04-21T08:38:11.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.848+0000 7f29a75df740 10 client.15288 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:38:11.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.848+0000 7f29a75df740 10 client.15288 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-21T08:38:11.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.848+0000 7f29a75df740 20 client.15288 populate_metadata read hostname 'smithi086' 2024-04-21T08:38:11.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.849+0000 7f29a75df740 1 --2- 172.21.15.86:0/679285713 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55b83da4f060 0x55b83da4dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.849+0000 7f29a75df740 4 client.15288 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:38:11.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.849+0000 7f29a75df740 1 -- 172.21.15.86:0/679285713 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "status"}) -- 0x55b83d93a3a0 con 0x55b83da4f060 2024-04-21T08:38:11.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.849+0000 7f299e59c640 1 --2- 172.21.15.86:0/679285713 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55b83da4f060 0x55b83da4dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.849+0000 7f299e59c640 1 --2- 172.21.15.86:0/679285713 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55b83da4f060 0x55b83da4dd70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=44ea9aecf9195ccc server_cookie=7d1f3d4527640f2e in_seq=0 out_seq=0 2024-04-21T08:38:11.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.849+0000 7f2985ffb640 10 client.15288 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:38:11.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.849+0000 7f2985ffb640 1 -- 172.21.15.86:0/679285713 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55b83d93a3a0 con 0x55b83da4f060 2024-04-21T08:38:11.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.849+0000 7f2985ffb640 10 client.15288 handle_command_reply: tid=0 2024-04-21T08:38:11.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.849+0000 7f29a75df740 1 client.15288 shutdown 2024-04-21T08:38:11.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.849+0000 7f29a75df740 1 -- 172.21.15.86:0/679285713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83d9cde40 msgr2=0x55b83d9db3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.849+0000 7f29a75df740 1 --2- 172.21.15.86:0/679285713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83d9cde40 0x55b83d9db3b0 secure :-1 s=READY pgs=399 cs=0 l=1 rev1=1 crypto rx=0x7f2988058040 tx=0x7f2988005be0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.849+0000 7f29a75df740 1 -- 172.21.15.86:0/679285713 shutdown_connections 2024-04-21T08:38:11.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.849+0000 7f29a75df740 1 --2- 172.21.15.86:0/679285713 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55b83da4f060 0x55b83da4dd70 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.849+0000 7f29a75df740 1 --2- 172.21.15.86:0/679285713 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b83d9cf2d0 0x55b83d9dde40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.849+0000 7f2985ffb640 0 client.15288 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:38:11.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.849+0000 7f29a75df740 1 --2- 172.21.15.86:0/679285713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b83d9ce830 0x55b83d9dd900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.849+0000 7f29a75df740 1 --2- 172.21.15.86:0/679285713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b83d9cde40 0x55b83d9db3b0 unknown :-1 s=CLOSED pgs=399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.849+0000 7f29a75df740 1 -- 172.21.15.86:0/679285713 >> 172.21.15.86:0/679285713 conn(0x55b83d9c9f40 msgr2=0x55b83d9d5a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:11.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.850+0000 7f29a75df740 1 -- 172.21.15.86:0/679285713 shutdown_connections 2024-04-21T08:38:11.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.850+0000 7f29a75df740 1 -- 172.21.15.86:0/679285713 wait complete. 2024-04-21T08:38:11.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.850+0000 7f29a75df740 20 client.15288 trim_cache size 0 max 16384 2024-04-21T08:38:11.857 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:38:11.857 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "f8cec7cc-ffb8-11ee-bc93-c7b262605968", 2024-04-21T08:38:11.857 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 2, 2024-04-21T08:38:11.857 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24455, 2024-04-21T08:38:11.857 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-21T08:38:11.857 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-21T08:38:11.858 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-21T08:38:11.858 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 86.459494289999995, 2024-04-21T08:38:11.858 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 28, 2024-04-21T08:38:11.858 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 91, 2024-04-21T08:38:11.858 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 89, 2024-04-21T08:38:11.858 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 128.70668668299999 2024-04-21T08:38:11.858 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:38:11.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.850+0000 7f29a491e640 1 -- 172.21.15.86:0/689617917 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f298c0a3bb0 msgr2=0x7f298c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.850+0000 7f29a491e640 1 --2- 172.21.15.86:0/689617917 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f298c0a3bb0 0x7f298c0a6070 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f299800a6e0 tx=0x7f299806f2a0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.850+0000 7f29a491e640 1 -- 172.21.15.86:0/689617917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a0103cb0 msgr2=0x7f29a019f880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.850+0000 7f29a491e640 1 --2- 172.21.15.86:0/689617917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a0103cb0 0x7f29a019f880 secure :-1 s=READY pgs=395 cs=0 l=1 rev1=1 crypto rx=0x7f299800cad0 tx=0x7f299800cf90 comp rx=0 tx=0).stop 2024-04-21T08:38:11.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.850+0000 7f29a491e640 1 -- 172.21.15.86:0/689617917 shutdown_connections 2024-04-21T08:38:11.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.850+0000 7f29a491e640 1 --2- 172.21.15.86:0/689617917 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f298c0a3bb0 0x7f298c0a6070 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.850+0000 7f29a491e640 1 --2- 172.21.15.86:0/689617917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a0103cb0 0x7f29a019f880 unknown :-1 s=CLOSED pgs=395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.850+0000 7f29a491e640 1 --2- 172.21.15.86:0/689617917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29a01032f0 0x7f29a0198810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.850+0000 7f29a491e640 1 --2- 172.21.15.86:0/689617917 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a0100280 0x7f29a01982d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.850+0000 7f29a491e640 1 -- 172.21.15.86:0/689617917 >> 172.21.15.86:0/689617917 conn(0x7f29a00fbb90 msgr2=0x7f29a0106780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:11.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.850+0000 7f29a491e640 1 -- 172.21.15.86:0/689617917 shutdown_connections 2024-04-21T08:38:11.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.850+0000 7f29a491e640 1 -- 172.21.15.86:0/689617917 wait complete. 2024-04-21T08:38:11.861 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-21T08:38:11.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.959+0000 7f2d28ac0640 1 -- 172.21.15.86:0/1193467691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d24101fa0 msgr2=0x7f2d241023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.959+0000 7f2d28ac0640 1 --2- 172.21.15.86:0/1193467691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d24101fa0 0x7f2d241023c0 secure :-1 s=READY pgs=400 cs=0 l=1 rev1=1 crypto rx=0x7f2d14009900 tx=0x7f2d1402bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:11.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.960+0000 7f2d28ac0640 1 -- 172.21.15.86:0/1193467691 shutdown_connections 2024-04-21T08:38:11.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.960+0000 7f2d28ac0640 1 --2- 172.21.15.86:0/1193467691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d24103b50 0x7f2d2410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.960+0000 7f2d28ac0640 1 --2- 172.21.15.86:0/1193467691 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d24103190 0x7f2d24103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.960+0000 7f2d28ac0640 1 --2- 172.21.15.86:0/1193467691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d24101fa0 0x7f2d241023c0 unknown :-1 s=CLOSED pgs=400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.960+0000 7f2d28ac0640 1 -- 172.21.15.86:0/1193467691 >> 172.21.15.86:0/1193467691 conn(0x7f2d240fd530 msgr2=0x7f2d240ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:11.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.960+0000 7f2d28ac0640 1 -- 172.21.15.86:0/1193467691 shutdown_connections 2024-04-21T08:38:11.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.960+0000 7f2d28ac0640 1 -- 172.21.15.86:0/1193467691 wait complete. 2024-04-21T08:38:11.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.960+0000 7f2d28ac0640 1 Processor -- start 2024-04-21T08:38:11.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.960+0000 7f2d28ac0640 1 -- start start 2024-04-21T08:38:11.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.960+0000 7f2d28ac0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d24101fa0 0x7f2d24075d00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.960+0000 7f2d28ac0640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d24103190 0x7f2d24074340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.960+0000 7f2d28ac0640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d24103b50 0x7f2d24074880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.960+0000 7f2d28ac0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2410dae0 con 0x7f2d24101fa0 2024-04-21T08:38:11.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.960+0000 7f2d28ac0640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2410d960 con 0x7f2d24103190 2024-04-21T08:38:11.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.960+0000 7f2d28ac0640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2d2410dc60 con 0x7f2d24103b50 2024-04-21T08:38:11.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.961+0000 7f2d2259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d24101fa0 0x7f2d24075d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.961+0000 7f2d2259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d24101fa0 0x7f2d24075d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36542/0 (socket says 172.21.15.86:36542) 2024-04-21T08:38:11.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.961+0000 7f2d2259c640 1 -- 172.21.15.86:0/782861233 learned_addr learned my addr 172.21.15.86:0/782861233 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:11.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.961+0000 7f2d22d9d640 1 --2- 172.21.15.86:0/782861233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d24103b50 0x7f2d24074880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.961+0000 7f2d1bfff640 1 --2- 172.21.15.86:0/782861233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d24103190 0x7f2d24074340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.961+0000 7f2d2259c640 1 -- 172.21.15.86:0/782861233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d24103b50 msgr2=0x7f2d24074880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.961+0000 7f2d2259c640 1 --2- 172.21.15.86:0/782861233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d24103b50 0x7f2d24074880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.961+0000 7f2d2259c640 1 -- 172.21.15.86:0/782861233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d24103190 msgr2=0x7f2d24074340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.961+0000 7f2d2259c640 1 --2- 172.21.15.86:0/782861233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d24103190 0x7f2d24074340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.961+0000 7f2d2259c640 1 -- 172.21.15.86:0/782861233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d24075110 con 0x7f2d24101fa0 2024-04-21T08:38:11.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.961+0000 7f2d1bfff640 1 --2- 172.21.15.86:0/782861233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d24103190 0x7f2d24074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:11.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.961+0000 7f2d2259c640 1 --2- 172.21.15.86:0/782861233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d24101fa0 0x7f2d24075d00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2d14036040 tx=0x7f2d140042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:11.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.961+0000 7f2d1b7fe640 1 -- 172.21.15.86:0/782861233 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14043020 con 0x7f2d24101fa0 2024-04-21T08:38:11.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.961+0000 7f2d1b7fe640 1 -- 172.21.15.86:0/782861233 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d14004aa0 con 0x7f2d24101fa0 2024-04-21T08:38:11.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.961+0000 7f2d28ac0640 1 -- 172.21.15.86:0/782861233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d241a22c0 con 0x7f2d24101fa0 2024-04-21T08:38:11.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.961+0000 7f2d1b7fe640 1 -- 172.21.15.86:0/782861233 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d14003de0 con 0x7f2d24101fa0 2024-04-21T08:38:11.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.961+0000 7f2d28ac0640 1 -- 172.21.15.86:0/782861233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d241a2810 con 0x7f2d24101fa0 2024-04-21T08:38:11.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.964+0000 7f2d1b7fe640 1 -- 172.21.15.86:0/782861233 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2d14046020 con 0x7f2d24101fa0 2024-04-21T08:38:11.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.964+0000 7f2d2b781740 1 Processor -- start 2024-04-21T08:38:11.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.964+0000 7f2d2b781740 1 -- start start 2024-04-21T08:38:11.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.964+0000 7f2d2b781740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d7cc18e40 0x563d7cc19260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.964+0000 7f2d2b781740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d7cc19830 0x563d7cc19cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.964+0000 7f2d2b781740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d7cc1a2d0 0x563d7cc20eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.964+0000 7f2d1b7fe640 1 --2- 172.21.15.86:0/782861233 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f2d100a3bb0 0x7f2d100a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.964+0000 7f2d2b781740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563d7cb7db10 con 0x563d7cc1a2d0 2024-04-21T08:38:11.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.964+0000 7f2d2b781740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x563d7cb1b240 con 0x563d7cc19830 2024-04-21T08:38:11.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.964+0000 7f2d2b781740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563d7cb79ad0 con 0x563d7cc18e40 2024-04-21T08:38:11.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.964+0000 7f2d1bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d7cc1a2d0 0x563d7cc20eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.964+0000 7f2d22d9d640 1 --2- 172.21.15.86:0/782861233 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f2d100a3bb0 0x7f2d100a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.965+0000 7f2d1bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d7cc1a2d0 0x563d7cc20eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36558/0 (socket says 172.21.15.86:36558) 2024-04-21T08:38:11.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.965+0000 7f2d1bfff640 1 -- 172.21.15.86:0/3720848014 learned_addr learned my addr 172.21.15.86:0/3720848014 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:11.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.965+0000 7f2d22d9d640 1 --2- 172.21.15.86:0/3720848014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d7cc18e40 0x563d7cc19260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.965+0000 7f2d1b7fe640 1 -- 172.21.15.86:0/782861233 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f2d140f2c50 con 0x7f2d24101fa0 2024-04-21T08:38:11.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.965+0000 7f2d2259c640 1 --2- 172.21.15.86:0/3720848014 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d7cc19830 0x563d7cc19cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.965+0000 7f2d1bfff640 1 -- 172.21.15.86:0/3720848014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d7cc18e40 msgr2=0x563d7cc19260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:11.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.965+0000 7f2d1bfff640 1 --2- 172.21.15.86:0/3720848014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d7cc18e40 0x563d7cc19260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.965+0000 7f2d1bfff640 1 -- 172.21.15.86:0/3720848014 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d7cc19830 msgr2=0x563d7cc19cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:11.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.965+0000 7f2d1bfff640 1 --2- 172.21.15.86:0/3720848014 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d7cc19830 0x563d7cc19cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.965+0000 7f2d1bfff640 1 -- 172.21.15.86:0/3720848014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d7c8902d0 con 0x563d7cc1a2d0 2024-04-21T08:38:11.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.965+0000 7f2d2259c640 1 --2- 172.21.15.86:0/3720848014 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d7cc19830 0x563d7cc19cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:11.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.965+0000 7f2d22d9d640 1 --2- 172.21.15.86:0/782861233 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f2d100a3bb0 0x7f2d100a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c013a70 tx=0x7f2d1c00ac50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:11.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.965+0000 7f2d1bfff640 1 --2- 172.21.15.86:0/3720848014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d7cc1a2d0 0x563d7cc20eb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c00bbe0 tx=0x7f2d0c00d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c74cc89966dbe5cb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:11.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.965+0000 7f2d18ff9640 1 -- 172.21.15.86:0/3720848014 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c017620 con 0x563d7cc1a2d0 2024-04-21T08:38:11.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.965+0000 7f2d18ff9640 1 -- 172.21.15.86:0/3720848014 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d0c017c70 con 0x563d7cc1a2d0 2024-04-21T08:38:11.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.965+0000 7f2d18ff9640 1 -- 172.21.15.86:0/3720848014 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c00fc10 con 0x563d7cc1a2d0 2024-04-21T08:38:11.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.966+0000 7f2d2b781740 1 -- 172.21.15.86:0/3720848014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d7cc1a2d0 msgr2=0x563d7cc20eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.966+0000 7f2d2b781740 1 --2- 172.21.15.86:0/3720848014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d7cc1a2d0 0x563d7cc20eb0 secure :-1 s=READY pgs=402 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c00bbe0 tx=0x7f2d0c00d710 comp rx=0 tx=0).stop 2024-04-21T08:38:11.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.966+0000 7f2d2b781740 1 -- 172.21.15.86:0/3720848014 shutdown_connections 2024-04-21T08:38:11.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.966+0000 7f2d2b781740 1 --2- 172.21.15.86:0/3720848014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d7cc1a2d0 0x563d7cc20eb0 unknown :-1 s=CLOSED pgs=402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.966+0000 7f2d2b781740 1 --2- 172.21.15.86:0/3720848014 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d7cc19830 0x563d7cc19cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.966+0000 7f2d2b781740 1 --2- 172.21.15.86:0/3720848014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d7cc18e40 0x563d7cc19260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.966+0000 7f2d2b781740 1 -- 172.21.15.86:0/3720848014 >> 172.21.15.86:0/3720848014 conn(0x563d7cc14f40 msgr2=0x563d7cc13490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:11.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.966+0000 7f2d2b781740 1 -- 172.21.15.86:0/3720848014 shutdown_connections 2024-04-21T08:38:11.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.966+0000 7f2d2b781740 1 -- 172.21.15.86:0/3720848014 wait complete. 2024-04-21T08:38:11.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.966+0000 7f2d2b781740 1 -- start start 2024-04-21T08:38:11.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.966+0000 7f2d2b781740 1 Processor -- start 2024-04-21T08:38:11.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.966+0000 7f2d2b781740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d7cc18e40 0x563d7cc22e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.966+0000 7f2d2b781740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d7cc19830 0x563d7cc23390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.967+0000 7f2d2b781740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d7cc1a2d0 0x563d7cc2a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.967+0000 7f2d2b781740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563d7cc214b0 con 0x563d7cc1a2d0 2024-04-21T08:38:11.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.967+0000 7f2d2b781740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x563d7cb06190 con 0x563d7cc19830 2024-04-21T08:38:11.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.967+0000 7f2d2b781740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563d7cc21630 con 0x563d7cc18e40 2024-04-21T08:38:11.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.967+0000 7f2d2b781740 10 client.0 _ll_register_callbacks cb 0x563d7cb018b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:11.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.967+0000 7f2d22d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d7cc1a2d0 0x563d7cc2a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.967+0000 7f2d22d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d7cc1a2d0 0x563d7cc2a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36570/0 (socket says 172.21.15.86:36570) 2024-04-21T08:38:11.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.967+0000 7f2d22d9d640 1 -- 172.21.15.86:0/253977296 learned_addr learned my addr 172.21.15.86:0/253977296 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:11.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.967+0000 7f2d2259c640 1 --2- 172.21.15.86:0/253977296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d7cc18e40 0x563d7cc22e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.967+0000 7f2d1bfff640 1 --2- 172.21.15.86:0/253977296 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d7cc19830 0x563d7cc23390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.967+0000 7f2d22d9d640 1 -- 172.21.15.86:0/253977296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d7cc18e40 msgr2=0x563d7cc22e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:11.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.967+0000 7f2d22d9d640 1 --2- 172.21.15.86:0/253977296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d7cc18e40 0x563d7cc22e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.967+0000 7f2d22d9d640 1 -- 172.21.15.86:0/253977296 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d7cc19830 msgr2=0x563d7cc23390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:11.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.967+0000 7f2d22d9d640 1 --2- 172.21.15.86:0/253977296 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d7cc19830 0x563d7cc23390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.967+0000 7f2d22d9d640 1 -- 172.21.15.86:0/253977296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d7c95a6b0 con 0x563d7cc1a2d0 2024-04-21T08:38:11.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.967+0000 7f2d1bfff640 1 --2- 172.21.15.86:0/253977296 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d7cc19830 0x563d7cc23390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:11.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.967+0000 7f2d2259c640 1 --2- 172.21.15.86:0/253977296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d7cc18e40 0x563d7cc22e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:11.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.967+0000 7f2d22d9d640 1 --2- 172.21.15.86:0/253977296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d7cc1a2d0 0x563d7cc2a3d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c010dc0 tx=0x7f2d1c005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2dc9734152615b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:11.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.967+0000 7f2d09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:38:11.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.967+0000 7f2d09ffb640 1 -- 172.21.15.86:0/253977296 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c024070 con 0x563d7cc1a2d0 2024-04-21T08:38:11.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.967+0000 7f2d09ffb640 1 -- 172.21.15.86:0/253977296 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f2d1c004d20 con 0x563d7cc1a2d0 2024-04-21T08:38:11.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.967+0000 7f2d2b781740 1 -- 172.21.15.86:0/253977296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d0c017c70 con 0x563d7cc1a2d0 2024-04-21T08:38:11.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.967+0000 7f2d09ffb640 1 -- 172.21.15.86:0/253977296 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2d1c031050 con 0x563d7cc1a2d0 2024-04-21T08:38:11.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.967+0000 7f2d2b781740 10 client.15300 fetch_fsmap learned FSMap version 28 2024-04-21T08:38:11.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.967+0000 7f2d2b781740 1 -- 172.21.15.86:0/253977296 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563d7cc2d720 con 0x563d7cc1a2d0 2024-04-21T08:38:11.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.968+0000 7f2d09ffb640 1 -- 172.21.15.86:0/253977296 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d0c017c70 con 0x563d7cc1a2d0 2024-04-21T08:38:11.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.968+0000 7f2d09ffb640 1 -- 172.21.15.86:0/253977296 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2d1c0203f0 con 0x563d7cc1a2d0 2024-04-21T08:38:11.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.968+0000 7f2d2b781740 10 client.15300 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:38:11.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.968+0000 7f2d2b781740 10 client.15300 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-21T08:38:11.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.968+0000 7f2d2b781740 20 client.15300 populate_metadata read hostname 'smithi086' 2024-04-21T08:38:11.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.968+0000 7f2d2b781740 1 --2- 172.21.15.86:0/253977296 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x563d7cc2fe70 0x563d7cc312e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.968+0000 7f2d2b781740 4 client.15300 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:38:11.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.968+0000 7f2d2b781740 1 -- 172.21.15.86:0/253977296 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d7cb8b530 con 0x563d7cc2fe70 2024-04-21T08:38:11.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.968+0000 7f2d2259c640 1 --2- 172.21.15.86:0/253977296 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x563d7cc2fe70 0x563d7cc312e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.969+0000 7f2d2259c640 1 --2- 172.21.15.86:0/253977296 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x563d7cc2fe70 0x563d7cc312e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=14db6993ce7397d4 server_cookie=a857aceb06d6f6cc in_seq=0 out_seq=0 2024-04-21T08:38:11.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.969+0000 7f2d09ffb640 10 client.15300 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:38:11.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.969+0000 7f2d09ffb640 1 -- 172.21.15.86:0/253977296 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563d7cb8b530 con 0x563d7cc2fe70 2024-04-21T08:38:11.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.969+0000 7f2d09ffb640 10 client.15300 handle_command_reply: tid=0 2024-04-21T08:38:11.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.969+0000 7f2d2b781740 1 client.15300 shutdown 2024-04-21T08:38:11.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.970+0000 7f2d2b781740 1 -- 172.21.15.86:0/253977296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d7cc1a2d0 msgr2=0x563d7cc2a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.970+0000 7f2d2b781740 1 --2- 172.21.15.86:0/253977296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d7cc1a2d0 0x563d7cc2a3d0 secure :-1 s=READY pgs=403 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c010dc0 tx=0x7f2d1c005bd0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.970+0000 7f2d2b781740 1 -- 172.21.15.86:0/253977296 shutdown_connections 2024-04-21T08:38:11.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.970+0000 7f2d2b781740 1 --2- 172.21.15.86:0/253977296 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x563d7cc2fe70 0x563d7cc312e0 crc :-1 s=READY pgs=16 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.970+0000 7f2d2b781740 1 --2- 172.21.15.86:0/253977296 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d7cc1a2d0 0x563d7cc2a3d0 unknown :-1 s=CLOSED pgs=403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.970+0000 7f2d2b781740 1 --2- 172.21.15.86:0/253977296 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d7cc19830 0x563d7cc23390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.970+0000 7f2d2b781740 1 --2- 172.21.15.86:0/253977296 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d7cc18e40 0x563d7cc22e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.970+0000 7f2d09ffb640 0 client.15300 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:38:11.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.970+0000 7f2d2b781740 1 -- 172.21.15.86:0/253977296 >> 172.21.15.86:0/253977296 conn(0x563d7cc14f40 msgr2=0x563d7cc12a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:11.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.970+0000 7f2d2b781740 1 -- 172.21.15.86:0/253977296 shutdown_connections 2024-04-21T08:38:11.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.970+0000 7f2d2b781740 1 -- 172.21.15.86:0/253977296 wait complete. 2024-04-21T08:38:11.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.970+0000 7f2d2b781740 20 client.15300 trim_cache size 0 max 16384 2024-04-21T08:38:11.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.988+0000 7f2d2b781740 1 Processor -- start 2024-04-21T08:38:11.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.988+0000 7f2d2b781740 1 -- start start 2024-04-21T08:38:11.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.988+0000 7f2d2b781740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d7cc18e40 0x563d7cc0e800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.988+0000 7f2d2b781740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d7cc19830 0x563d7cc2a390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.988+0000 7f2d2b781740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d7cc1a2d0 0x563d7cc1d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.988+0000 7f2d2b781740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563d7cc2f620 con 0x563d7cc19830 2024-04-21T08:38:11.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.988+0000 7f2d2b781740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x563d7cc2fcf0 con 0x563d7cc18e40 2024-04-21T08:38:11.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.988+0000 7f2d2b781740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563d7cc303c0 con 0x563d7cc1a2d0 2024-04-21T08:38:11.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.988+0000 7f2d1bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d7cc19830 0x563d7cc2a390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.988+0000 7f2d1bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d7cc19830 0x563d7cc2a390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36572/0 (socket says 172.21.15.86:36572) 2024-04-21T08:38:11.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.988+0000 7f2d1bfff640 1 -- 172.21.15.86:0/1766446046 learned_addr learned my addr 172.21.15.86:0/1766446046 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:11.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.988+0000 7f2d1bfff640 1 -- 172.21.15.86:0/1766446046 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d7cc1a2d0 msgr2=0x563d7cc1d770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:11.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.988+0000 7f2d1bfff640 1 --2- 172.21.15.86:0/1766446046 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d7cc1a2d0 0x563d7cc1d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.988+0000 7f2d1bfff640 1 -- 172.21.15.86:0/1766446046 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d7cc18e40 msgr2=0x563d7cc0e800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:11.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.988+0000 7f2d1bfff640 1 --2- 172.21.15.86:0/1766446046 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d7cc18e40 0x563d7cc0e800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.988+0000 7f2d1bfff640 1 -- 172.21.15.86:0/1766446046 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d7cc20a40 con 0x563d7cc19830 2024-04-21T08:38:11.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.988+0000 7f2d1bfff640 1 --2- 172.21.15.86:0/1766446046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d7cc19830 0x563d7cc2a390 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c019040 tx=0x7f2d0c00b370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed51557538415862 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:11.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.988+0000 7f2d0bfff640 1 -- 172.21.15.86:0/1766446046 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c025020 con 0x563d7cc19830 2024-04-21T08:38:11.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.988+0000 7f2d0bfff640 1 -- 172.21.15.86:0/1766446046 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d0c00b750 con 0x563d7cc19830 2024-04-21T08:38:11.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.988+0000 7f2d0bfff640 1 -- 172.21.15.86:0/1766446046 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d0c004d20 con 0x563d7cc19830 2024-04-21T08:38:11.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.989+0000 7f2d2b781740 1 -- 172.21.15.86:0/1766446046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d7cc19830 msgr2=0x563d7cc2a390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.989+0000 7f2d2b781740 1 --2- 172.21.15.86:0/1766446046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d7cc19830 0x563d7cc2a390 secure :-1 s=READY pgs=404 cs=0 l=1 rev1=1 crypto rx=0x7f2d0c019040 tx=0x7f2d0c00b370 comp rx=0 tx=0).stop 2024-04-21T08:38:11.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.989+0000 7f2d2b781740 1 -- 172.21.15.86:0/1766446046 shutdown_connections 2024-04-21T08:38:11.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.989+0000 7f2d2b781740 1 --2- 172.21.15.86:0/1766446046 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d7cc1a2d0 0x563d7cc1d770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.989+0000 7f2d2b781740 1 --2- 172.21.15.86:0/1766446046 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d7cc19830 0x563d7cc2a390 unknown :-1 s=CLOSED pgs=404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.989+0000 7f2d2b781740 1 --2- 172.21.15.86:0/1766446046 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d7cc18e40 0x563d7cc0e800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.989+0000 7f2d2b781740 1 -- 172.21.15.86:0/1766446046 >> 172.21.15.86:0/1766446046 conn(0x563d7cc14f40 msgr2=0x563d7cc22e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:11.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.989+0000 7f2d2b781740 1 -- 172.21.15.86:0/1766446046 shutdown_connections 2024-04-21T08:38:11.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.989+0000 7f2d2b781740 1 -- 172.21.15.86:0/1766446046 wait complete. 2024-04-21T08:38:11.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.989+0000 7f2d2b781740 1 -- start start 2024-04-21T08:38:11.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.989+0000 7f2d2b781740 1 Processor -- start 2024-04-21T08:38:11.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.989+0000 7f2d2b781740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d7cc18e40 0x563d7cc260b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.990+0000 7f2d2b781740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d7cc19830 0x563d7cc265f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.990+0000 7f2d2b781740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d7cc1a2d0 0x563d7cc29570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.990+0000 7f2d2b781740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563d7cc31160 con 0x563d7cc1a2d0 2024-04-21T08:38:11.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.990+0000 7f2d2b781740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x563d7cc30a90 con 0x563d7cc18e40 2024-04-21T08:38:11.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.990+0000 7f2d2b781740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563d7cc31830 con 0x563d7cc19830 2024-04-21T08:38:11.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.990+0000 7f2d2b781740 10 client.0 _ll_register_callbacks cb 0x7f2d1c013f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:11.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.990+0000 7f2d22d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d7cc1a2d0 0x563d7cc29570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.990+0000 7f2d22d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d7cc1a2d0 0x563d7cc29570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36582/0 (socket says 172.21.15.86:36582) 2024-04-21T08:38:11.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.990+0000 7f2d22d9d640 1 -- 172.21.15.86:0/990326087 learned_addr learned my addr 172.21.15.86:0/990326087 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:11.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.990+0000 7f2d2259c640 1 --2- 172.21.15.86:0/990326087 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d7cc18e40 0x563d7cc260b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.990+0000 7f2d1bfff640 1 --2- 172.21.15.86:0/990326087 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d7cc19830 0x563d7cc265f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.990+0000 7f2d22d9d640 1 -- 172.21.15.86:0/990326087 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d7cc19830 msgr2=0x563d7cc265f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:11.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.990+0000 7f2d22d9d640 1 --2- 172.21.15.86:0/990326087 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d7cc19830 0x563d7cc265f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.990+0000 7f2d22d9d640 1 -- 172.21.15.86:0/990326087 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d7cc18e40 msgr2=0x563d7cc260b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:11.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.990+0000 7f2d22d9d640 1 --2- 172.21.15.86:0/990326087 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d7cc18e40 0x563d7cc260b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.990+0000 7f2d22d9d640 1 -- 172.21.15.86:0/990326087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d7cc29b60 con 0x563d7cc1a2d0 2024-04-21T08:38:11.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.990+0000 7f2d2259c640 1 --2- 172.21.15.86:0/990326087 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d7cc18e40 0x563d7cc260b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:11.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.990+0000 7f2d22d9d640 1 --2- 172.21.15.86:0/990326087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d7cc1a2d0 0x563d7cc29570 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c004a80 tx=0x7f2d1c00aed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46b343c52b80b510 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:11.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.990+0000 7f2d09ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:38:11.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.990+0000 7f2d09ffb640 1 -- 172.21.15.86:0/990326087 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d1c024050 con 0x563d7cc1a2d0 2024-04-21T08:38:11.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.990+0000 7f2d09ffb640 1 -- 172.21.15.86:0/990326087 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f2d1c035630 con 0x563d7cc1a2d0 2024-04-21T08:38:11.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.990+0000 7f2d2b781740 1 -- 172.21.15.86:0/990326087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d0c00b750 con 0x563d7cc1a2d0 2024-04-21T08:38:11.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.990+0000 7f2d09ffb640 1 -- 172.21.15.86:0/990326087 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2d1c043050 con 0x563d7cc1a2d0 2024-04-21T08:38:11.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.990+0000 7f2d2b781740 10 client.15306 fetch_fsmap learned FSMap version 28 2024-04-21T08:38:11.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.990+0000 7f2d2b781740 1 -- 172.21.15.86:0/990326087 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x563d7cc963e0 con 0x563d7cc1a2d0 2024-04-21T08:38:11.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.991+0000 7f2d09ffb640 1 -- 172.21.15.86:0/990326087 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d0c00b750 con 0x563d7cc1a2d0 2024-04-21T08:38:11.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.991+0000 7f2d09ffb640 1 -- 172.21.15.86:0/990326087 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2d1c034070 con 0x563d7cc1a2d0 2024-04-21T08:38:11.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.991+0000 7f2d2b781740 10 client.15306 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:38:11.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.991+0000 7f2d2b781740 10 client.15306 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-21T08:38:11.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.991+0000 7f2d2b781740 20 client.15306 populate_metadata read hostname 'smithi086' 2024-04-21T08:38:11.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.991+0000 7f2d2b781740 1 --2- 172.21.15.86:0/990326087 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x563d7cc9b000 0x563d7cc99d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:11.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.991+0000 7f2d2b781740 4 client.15306 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:38:11.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.991+0000 7f2d2b781740 1 -- 172.21.15.86:0/990326087 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "status"}) -- 0x563d7cb853a0 con 0x563d7cc9b000 2024-04-21T08:38:11.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.991+0000 7f2d2259c640 1 --2- 172.21.15.86:0/990326087 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x563d7cc9b000 0x563d7cc99d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:11.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.991+0000 7f2d09ffb640 10 client.15306 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:38:11.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.991+0000 7f2d2259c640 1 --2- 172.21.15.86:0/990326087 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x563d7cc9b000 0x563d7cc99d10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=91d641e64baa6144 server_cookie=d88e7cf10322df49 in_seq=0 out_seq=0 2024-04-21T08:38:11.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.992+0000 7f2d09ffb640 1 -- 172.21.15.86:0/990326087 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+326 (crc 0 0 0) 0x563d7cb853a0 con 0x563d7cc9b000 2024-04-21T08:38:11.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.992+0000 7f2d09ffb640 10 client.15306 handle_command_reply: tid=0 2024-04-21T08:38:11.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.992+0000 7f2d2b781740 1 client.15306 shutdown 2024-04-21T08:38:11.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.992+0000 7f2d2b781740 1 -- 172.21.15.86:0/990326087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d7cc1a2d0 msgr2=0x563d7cc29570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:11.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.992+0000 7f2d2b781740 1 --2- 172.21.15.86:0/990326087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d7cc1a2d0 0x563d7cc29570 secure :-1 s=READY pgs=405 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c004a80 tx=0x7f2d1c00aed0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.992+0000 7f2d2b781740 1 -- 172.21.15.86:0/990326087 shutdown_connections 2024-04-21T08:38:11.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.992+0000 7f2d2b781740 1 --2- 172.21.15.86:0/990326087 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x563d7cc9b000 0x563d7cc99d10 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.992+0000 7f2d2b781740 1 --2- 172.21.15.86:0/990326087 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d7cc1a2d0 0x563d7cc29570 unknown :-1 s=CLOSED pgs=405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.992+0000 7f2d2b781740 1 --2- 172.21.15.86:0/990326087 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d7cc19830 0x563d7cc265f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.992+0000 7f2d2b781740 1 --2- 172.21.15.86:0/990326087 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d7cc18e40 0x563d7cc260b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:11.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.992+0000 7f2d09ffb640 0 client.15306 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:38:11.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.992+0000 7f2d2b781740 1 -- 172.21.15.86:0/990326087 >> 172.21.15.86:0/990326087 conn(0x563d7cc14f40 msgr2=0x563d7cc1d770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:11.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.992+0000 7f2d2b781740 1 -- 172.21.15.86:0/990326087 shutdown_connections 2024-04-21T08:38:11.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.992+0000 7f2d2b781740 1 -- 172.21.15.86:0/990326087 wait complete. 2024-04-21T08:38:11.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.992+0000 7f2d2b781740 20 client.15306 trim_cache size 0 max 16384 2024-04-21T08:38:11.999 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:38:11.999 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "f8cec7cc-ffb8-11ee-bc93-c7b262605968", 2024-04-21T08:38:11.999 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 1, 2024-04-21T08:38:11.999 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24472, 2024-04-21T08:38:12.000 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-21T08:38:12.000 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-21T08:38:12.000 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-21T08:38:12.000 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 87.611676041999999, 2024-04-21T08:38:12.000 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 28, 2024-04-21T08:38:12.000 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 91, 2024-04-21T08:38:12.000 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 89, 2024-04-21T08:38:12.000 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 126.06128678 2024-04-21T08:38:12.000 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:38:12.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.992+0000 7f2d28ac0640 1 -- 172.21.15.86:0/782861233 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f2d100a3bb0 msgr2=0x7f2d100a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:12.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.992+0000 7f2d28ac0640 1 --2- 172.21.15.86:0/782861233 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f2d100a3bb0 0x7f2d100a6070 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f2d1c013a70 tx=0x7f2d1c00ac50 comp rx=0 tx=0).stop 2024-04-21T08:38:12.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.992+0000 7f2d28ac0640 1 -- 172.21.15.86:0/782861233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d24101fa0 msgr2=0x7f2d24075d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:12.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.992+0000 7f2d28ac0640 1 --2- 172.21.15.86:0/782861233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d24101fa0 0x7f2d24075d00 secure :-1 s=READY pgs=401 cs=0 l=1 rev1=1 crypto rx=0x7f2d14036040 tx=0x7f2d140042f0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.992+0000 7f2d28ac0640 1 -- 172.21.15.86:0/782861233 shutdown_connections 2024-04-21T08:38:12.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.992+0000 7f2d28ac0640 1 --2- 172.21.15.86:0/782861233 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f2d100a3bb0 0x7f2d100a6070 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.992+0000 7f2d28ac0640 1 --2- 172.21.15.86:0/782861233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d24103b50 0x7f2d24074880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.992+0000 7f2d28ac0640 1 --2- 172.21.15.86:0/782861233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d24103190 0x7f2d24074340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.992+0000 7f2d28ac0640 1 --2- 172.21.15.86:0/782861233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d24101fa0 0x7f2d24075d00 unknown :-1 s=CLOSED pgs=401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.992+0000 7f2d28ac0640 1 -- 172.21.15.86:0/782861233 >> 172.21.15.86:0/782861233 conn(0x7f2d240fd530 msgr2=0x7f2d24106510 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:12.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.992+0000 7f2d28ac0640 1 -- 172.21.15.86:0/782861233 shutdown_connections 2024-04-21T08:38:12.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:11.992+0000 7f2d28ac0640 1 -- 172.21.15.86:0/782861233 wait complete. 2024-04-21T08:38:12.003 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-21T08:38:12.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.102+0000 7f2a83a43640 1 -- 172.21.15.86:0/1498745142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a7c101fa0 msgr2=0x7f2a7c1023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:12.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.102+0000 7f2a83a43640 1 --2- 172.21.15.86:0/1498745142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a7c101fa0 0x7f2a7c1023c0 secure :-1 s=READY pgs=406 cs=0 l=1 rev1=1 crypto rx=0x7f2a74009ca0 tx=0x7f2a7402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.102+0000 7f2a83a43640 1 -- 172.21.15.86:0/1498745142 shutdown_connections 2024-04-21T08:38:12.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.102+0000 7f2a83a43640 1 --2- 172.21.15.86:0/1498745142 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a7c103b50 0x7f2a7c10a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.102+0000 7f2a83a43640 1 --2- 172.21.15.86:0/1498745142 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a7c103190 0x7f2a7c103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.102+0000 7f2a83a43640 1 --2- 172.21.15.86:0/1498745142 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a7c101fa0 0x7f2a7c1023c0 unknown :-1 s=CLOSED pgs=406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.102+0000 7f2a83a43640 1 -- 172.21.15.86:0/1498745142 >> 172.21.15.86:0/1498745142 conn(0x7f2a7c0fd530 msgr2=0x7f2a7c0ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:12.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.102+0000 7f2a83a43640 1 -- 172.21.15.86:0/1498745142 shutdown_connections 2024-04-21T08:38:12.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.102+0000 7f2a83a43640 1 -- 172.21.15.86:0/1498745142 wait complete. 2024-04-21T08:38:12.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.103+0000 7f2a83a43640 1 Processor -- start 2024-04-21T08:38:12.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.103+0000 7f2a83a43640 1 -- start start 2024-04-21T08:38:12.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.103+0000 7f2a83a43640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a7c101fa0 0x7f2a7c198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:12.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.103+0000 7f2a83a43640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a7c103190 0x7f2a7c198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:12.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.103+0000 7f2a83a43640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a7c103b50 0x7f2a7c19f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:12.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.103+0000 7f2a83a43640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7c10dae0 con 0x7f2a7c103b50 2024-04-21T08:38:12.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.103+0000 7f2a83a43640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7c10d960 con 0x7f2a7c101fa0 2024-04-21T08:38:12.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.103+0000 7f2a83a43640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2a7c10dc60 con 0x7f2a7c103190 2024-04-21T08:38:12.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.103+0000 7f2a81fe0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a7c103b50 0x7f2a7c19f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:12.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.103+0000 7f2a81fe0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a7c103b50 0x7f2a7c19f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36602/0 (socket says 172.21.15.86:36602) 2024-04-21T08:38:12.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.103+0000 7f2a81fe0640 1 -- 172.21.15.86:0/233437757 learned_addr learned my addr 172.21.15.86:0/233437757 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:12.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.103+0000 7f2a817df640 1 --2- 172.21.15.86:0/233437757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a7c101fa0 0x7f2a7c198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:12.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.103+0000 7f2a80fde640 1 --2- 172.21.15.86:0/233437757 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a7c103190 0x7f2a7c198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:12.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.103+0000 7f2a81fe0640 1 -- 172.21.15.86:0/233437757 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a7c103190 msgr2=0x7f2a7c198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:12.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.103+0000 7f2a81fe0640 1 --2- 172.21.15.86:0/233437757 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a7c103190 0x7f2a7c198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.103+0000 7f2a81fe0640 1 -- 172.21.15.86:0/233437757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a7c101fa0 msgr2=0x7f2a7c198250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:12.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.103+0000 7f2a81fe0640 1 --2- 172.21.15.86:0/233437757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a7c101fa0 0x7f2a7c198250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.103+0000 7f2a81fe0640 1 -- 172.21.15.86:0/233437757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2a7c19fe70 con 0x7f2a7c103b50 2024-04-21T08:38:12.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.103+0000 7f2a817df640 1 --2- 172.21.15.86:0/233437757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a7c101fa0 0x7f2a7c198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:12.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.104+0000 7f2a81fe0640 1 --2- 172.21.15.86:0/233437757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a7c103b50 0x7f2a7c19f800 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800c970 tx=0x7f2a7800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:12.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.104+0000 7f2a727fc640 1 -- 172.21.15.86:0/233437757 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a78007bb0 con 0x7f2a7c103b50 2024-04-21T08:38:12.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.104+0000 7f2a83a43640 1 -- 172.21.15.86:0/233437757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2a7c1a0160 con 0x7f2a7c103b50 2024-04-21T08:38:12.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.104+0000 7f2a727fc640 1 -- 172.21.15.86:0/233437757 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a78007d50 con 0x7f2a7c103b50 2024-04-21T08:38:12.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.104+0000 7f2a83a43640 1 -- 172.21.15.86:0/233437757 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2a7c1a06b0 con 0x7f2a7c103b50 2024-04-21T08:38:12.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.104+0000 7f2a727fc640 1 -- 172.21.15.86:0/233437757 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a78004e60 con 0x7f2a7c103b50 2024-04-21T08:38:12.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.106+0000 7f2a727fc640 1 -- 172.21.15.86:0/233437757 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f2a78005020 con 0x7f2a7c103b50 2024-04-21T08:38:12.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.106+0000 7f2a8663c740 1 Processor -- start 2024-04-21T08:38:12.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.106+0000 7f2a8663c740 1 -- start start 2024-04-21T08:38:12.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.106+0000 7f2a8663c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caff55ee40 0x55caff55f260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:12.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.106+0000 7f2a8663c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caff55f830 0x55caff55fcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:12.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.106+0000 7f2a8663c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55caff5602d0 0x55caff566eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:12.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.106+0000 7f2a8663c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55caff4c3b10 con 0x55caff55f830 2024-04-21T08:38:12.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.106+0000 7f2a8663c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55caff461240 con 0x55caff5602d0 2024-04-21T08:38:12.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.106+0000 7f2a8663c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55caff4bfad0 con 0x55caff55ee40 2024-04-21T08:38:12.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.107+0000 7f2a727fc640 1 --2- 172.21.15.86:0/233437757 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f2a680a3bb0 0x7f2a680a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:12.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.107+0000 7f2a817df640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caff55f830 0x55caff55fcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:12.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.107+0000 7f2a817df640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caff55f830 0x55caff55fcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36614/0 (socket says 172.21.15.86:36614) 2024-04-21T08:38:12.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.107+0000 7f2a817df640 1 -- 172.21.15.86:0/302190160 learned_addr learned my addr 172.21.15.86:0/302190160 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:12.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.107+0000 7f2a81fe0640 1 --2- 172.21.15.86:0/233437757 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f2a680a3bb0 0x7f2a680a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:12.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.107+0000 7f2a80fde640 1 --2- 172.21.15.86:0/302190160 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caff55ee40 0x55caff55f260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:12.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.107+0000 7f2a80fde640 1 --2- 172.21.15.86:0/302190160 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55caff5602d0 0x55caff566eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:12.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.107+0000 7f2a727fc640 1 -- 172.21.15.86:0/233437757 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f2a780d04f0 con 0x7f2a7c103b50 2024-04-21T08:38:12.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.107+0000 7f2a817df640 1 -- 172.21.15.86:0/302190160 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caff55ee40 msgr2=0x55caff55f260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:12.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.107+0000 7f2a817df640 1 --2- 172.21.15.86:0/302190160 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caff55ee40 0x55caff55f260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.107+0000 7f2a817df640 1 -- 172.21.15.86:0/302190160 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55caff5602d0 msgr2=0x55caff566eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:12.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.107+0000 7f2a817df640 1 --2- 172.21.15.86:0/302190160 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55caff5602d0 0x55caff566eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.107+0000 7f2a817df640 1 -- 172.21.15.86:0/302190160 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55caff1d62d0 con 0x55caff55f830 2024-04-21T08:38:12.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.107+0000 7f2a817df640 1 --2- 172.21.15.86:0/302190160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caff55f830 0x55caff55fcb0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2a74032380 tx=0x7f2a74009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae92c4d79144db server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:12.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.107+0000 7f2a81fe0640 1 --2- 172.21.15.86:0/233437757 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f2a680a3bb0 0x7f2a680a6070 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2a7809c400 tx=0x7f2a7806efb0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:12.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.107+0000 7f2a677fe640 1 -- 172.21.15.86:0/302190160 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a74043020 con 0x55caff55f830 2024-04-21T08:38:12.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.107+0000 7f2a677fe640 1 -- 172.21.15.86:0/302190160 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a74032dc0 con 0x55caff55f830 2024-04-21T08:38:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.107+0000 7f2a677fe640 1 -- 172.21.15.86:0/302190160 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a74039040 con 0x55caff55f830 2024-04-21T08:38:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.108+0000 7f2a8663c740 1 -- 172.21.15.86:0/302190160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caff55f830 msgr2=0x55caff55fcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.108+0000 7f2a8663c740 1 --2- 172.21.15.86:0/302190160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caff55f830 0x55caff55fcb0 secure :-1 s=READY pgs=408 cs=0 l=1 rev1=1 crypto rx=0x7f2a74032380 tx=0x7f2a74009a00 comp rx=0 tx=0).stop 2024-04-21T08:38:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.108+0000 7f2a8663c740 1 -- 172.21.15.86:0/302190160 shutdown_connections 2024-04-21T08:38:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.108+0000 7f2a8663c740 1 --2- 172.21.15.86:0/302190160 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55caff5602d0 0x55caff566eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.108+0000 7f2a8663c740 1 --2- 172.21.15.86:0/302190160 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caff55f830 0x55caff55fcb0 unknown :-1 s=CLOSED pgs=408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.108+0000 7f2a8663c740 1 --2- 172.21.15.86:0/302190160 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caff55ee40 0x55caff55f260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.108+0000 7f2a8663c740 1 -- 172.21.15.86:0/302190160 >> 172.21.15.86:0/302190160 conn(0x55caff55af40 msgr2=0x55caff559490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.108+0000 7f2a8663c740 1 -- 172.21.15.86:0/302190160 shutdown_connections 2024-04-21T08:38:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.108+0000 7f2a8663c740 1 -- 172.21.15.86:0/302190160 wait complete. 2024-04-21T08:38:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.108+0000 7f2a8663c740 1 -- start start 2024-04-21T08:38:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.108+0000 7f2a8663c740 1 Processor -- start 2024-04-21T08:38:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.109+0000 7f2a8663c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caff55ee40 0x55caff568e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.109+0000 7f2a8663c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caff55f830 0x55caff5693a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:12.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.109+0000 7f2a8663c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55caff5602d0 0x55caff5703e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.109+0000 7f2a8663c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55caff567450 con 0x55caff55ee40 2024-04-21T08:38:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.109+0000 7f2a8663c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55caff44c190 con 0x55caff5602d0 2024-04-21T08:38:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.109+0000 7f2a8663c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55caff5675d0 con 0x55caff55f830 2024-04-21T08:38:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.109+0000 7f2a8663c740 10 client.0 _ll_register_callbacks cb 0x55caff4478b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.109+0000 7f2a80fde640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caff55ee40 0x55caff568e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.109+0000 7f2a80fde640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caff55ee40 0x55caff568e60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36624/0 (socket says 172.21.15.86:36624) 2024-04-21T08:38:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.109+0000 7f2a80fde640 1 -- 172.21.15.86:0/3505754454 learned_addr learned my addr 172.21.15.86:0/3505754454 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.109+0000 7f2a817df640 1 --2- 172.21.15.86:0/3505754454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caff55f830 0x55caff5693a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.109+0000 7f2a80fde640 1 -- 172.21.15.86:0/3505754454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caff55f830 msgr2=0x55caff5693a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.109+0000 7f2a80fde640 1 --2- 172.21.15.86:0/3505754454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caff55f830 0x55caff5693a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.109+0000 7f2a80fde640 1 -- 172.21.15.86:0/3505754454 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55caff5602d0 msgr2=0x55caff5703e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.109+0000 7f2a80fde640 1 --2- 172.21.15.86:0/3505754454 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55caff5602d0 0x55caff5703e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.109+0000 7f2a80fde640 1 -- 172.21.15.86:0/3505754454 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55caff2a06b0 con 0x55caff55ee40 2024-04-21T08:38:12.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.109+0000 7f2a80fde640 1 --2- 172.21.15.86:0/3505754454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caff55ee40 0x55caff568e60 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c0058c0 tx=0x7f2a6c0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a71c1757ae2c1e5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:12.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.109+0000 7f2a64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:38:12.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.110+0000 7f2a64ff9640 1 -- 172.21.15.86:0/3505754454 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c005220 con 0x55caff55ee40 2024-04-21T08:38:12.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.110+0000 7f2a64ff9640 1 -- 172.21.15.86:0/3505754454 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f2a6c0053c0 con 0x55caff55ee40 2024-04-21T08:38:12.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.110+0000 7f2a8663c740 1 -- 172.21.15.86:0/3505754454 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a74032dc0 con 0x55caff55ee40 2024-04-21T08:38:12.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.110+0000 7f2a64ff9640 1 -- 172.21.15.86:0/3505754454 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2a6c010e90 con 0x55caff55ee40 2024-04-21T08:38:12.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.110+0000 7f2a8663c740 10 client.15318 fetch_fsmap learned FSMap version 28 2024-04-21T08:38:12.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.110+0000 7f2a8663c740 1 -- 172.21.15.86:0/3505754454 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55caff573820 con 0x55caff55ee40 2024-04-21T08:38:12.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.110+0000 7f2a64ff9640 1 -- 172.21.15.86:0/3505754454 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a74032dc0 con 0x55caff55ee40 2024-04-21T08:38:12.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.110+0000 7f2a64ff9640 1 -- 172.21.15.86:0/3505754454 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2a6c01b070 con 0x55caff55ee40 2024-04-21T08:38:12.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.110+0000 7f2a8663c740 10 client.15318 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:38:12.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.110+0000 7f2a8663c740 10 client.15318 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-21T08:38:12.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.110+0000 7f2a8663c740 20 client.15318 populate_metadata read hostname 'smithi086' 2024-04-21T08:38:12.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.110+0000 7f2a8663c740 1 --2- 172.21.15.86:0/3505754454 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55caff575f00 0x55caff577370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:12.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.110+0000 7f2a8663c740 4 client.15318 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:38:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.110+0000 7f2a8663c740 1 -- 172.21.15.86:0/3505754454 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55caff4d1530 con 0x55caff575f00 2024-04-21T08:38:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.110+0000 7f2a817df640 1 --2- 172.21.15.86:0/3505754454 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55caff575f00 0x55caff577370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.111+0000 7f2a64ff9640 10 client.15318 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:38:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.111+0000 7f2a817df640 1 --2- 172.21.15.86:0/3505754454 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55caff575f00 0x55caff577370 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be7c600cb51d3b57 server_cookie=7c7c99303e69e175 in_seq=0 out_seq=0 2024-04-21T08:38:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.112+0000 7f2a64ff9640 1 -- 172.21.15.86:0/3505754454 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55caff4d1530 con 0x55caff575f00 2024-04-21T08:38:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.112+0000 7f2a64ff9640 10 client.15318 handle_command_reply: tid=0 2024-04-21T08:38:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.112+0000 7f2a8663c740 1 client.15318 shutdown 2024-04-21T08:38:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.112+0000 7f2a8663c740 1 -- 172.21.15.86:0/3505754454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caff55ee40 msgr2=0x55caff568e60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.112+0000 7f2a8663c740 1 --2- 172.21.15.86:0/3505754454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caff55ee40 0x55caff568e60 secure :-1 s=READY pgs=409 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c0058c0 tx=0x7f2a6c0107a0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.112+0000 7f2a8663c740 1 -- 172.21.15.86:0/3505754454 shutdown_connections 2024-04-21T08:38:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.112+0000 7f2a8663c740 1 --2- 172.21.15.86:0/3505754454 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55caff575f00 0x55caff577370 crc :-1 s=READY pgs=29 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.112+0000 7f2a8663c740 1 --2- 172.21.15.86:0/3505754454 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55caff5602d0 0x55caff5703e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.112+0000 7f2a8663c740 1 --2- 172.21.15.86:0/3505754454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caff55f830 0x55caff5693a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.112+0000 7f2a64ff9640 0 client.15318 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:38:12.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.112+0000 7f2a8663c740 1 --2- 172.21.15.86:0/3505754454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caff55ee40 0x55caff568e60 unknown :-1 s=CLOSED pgs=409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.112+0000 7f2a8663c740 1 -- 172.21.15.86:0/3505754454 >> 172.21.15.86:0/3505754454 conn(0x55caff55af40 msgr2=0x55caff558a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:12.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.112+0000 7f2a8663c740 1 -- 172.21.15.86:0/3505754454 shutdown_connections 2024-04-21T08:38:12.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.112+0000 7f2a8663c740 1 -- 172.21.15.86:0/3505754454 wait complete. 2024-04-21T08:38:12.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.112+0000 7f2a8663c740 20 client.15318 trim_cache size 0 max 16384 2024-04-21T08:38:12.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.130+0000 7f2a8663c740 1 Processor -- start 2024-04-21T08:38:12.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.130+0000 7f2a8663c740 1 -- start start 2024-04-21T08:38:12.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.130+0000 7f2a8663c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caff55ee40 0x55caff568a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:12.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.130+0000 7f2a8663c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55caff55f830 0x55caff568fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:12.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.130+0000 7f2a8663c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caff5602d0 0x55caff558170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:12.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.130+0000 7f2a8663c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55caff5756b0 con 0x55caff5602d0 2024-04-21T08:38:12.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.130+0000 7f2a8663c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55caff575d80 con 0x55caff55f830 2024-04-21T08:38:12.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.130+0000 7f2a8663c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55caff576450 con 0x55caff55ee40 2024-04-21T08:38:12.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.130+0000 7f2a80fde640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caff5602d0 0x55caff558170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:12.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.130+0000 7f2a80fde640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caff5602d0 0x55caff558170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36640/0 (socket says 172.21.15.86:36640) 2024-04-21T08:38:12.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.130+0000 7f2a80fde640 1 -- 172.21.15.86:0/1040513443 learned_addr learned my addr 172.21.15.86:0/1040513443 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:12.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.130+0000 7f2a80fde640 1 --2- 172.21.15.86:0/1040513443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caff55ee40 0x55caff568a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:12.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.130+0000 7f2a817df640 1 --2- 172.21.15.86:0/1040513443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55caff55f830 0x55caff568fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:12.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.130+0000 7f2a80fde640 1 -- 172.21.15.86:0/1040513443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caff55ee40 msgr2=0x55caff568a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:12.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.130+0000 7f2a80fde640 1 --2- 172.21.15.86:0/1040513443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caff55ee40 0x55caff568a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.130+0000 7f2a80fde640 1 -- 172.21.15.86:0/1040513443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55caff55f830 msgr2=0x55caff568fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:12.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.130+0000 7f2a80fde640 1 --2- 172.21.15.86:0/1040513443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55caff55f830 0x55caff568fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.130+0000 7f2a80fde640 1 -- 172.21.15.86:0/1040513443 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55caff55a630 con 0x55caff5602d0 2024-04-21T08:38:12.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.130+0000 7f2a817df640 1 --2- 172.21.15.86:0/1040513443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55caff55f830 0x55caff568fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:12.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.131+0000 7f2a80fde640 1 --2- 172.21.15.86:0/1040513443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caff5602d0 0x55caff558170 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c011b30 tx=0x7f2a6c00efd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=54612e061ea5bcf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:12.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.131+0000 7f2a66ffd640 1 -- 172.21.15.86:0/1040513443 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c01cbb0 con 0x55caff5602d0 2024-04-21T08:38:12.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.131+0000 7f2a66ffd640 1 -- 172.21.15.86:0/1040513443 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a6c0053c0 con 0x55caff5602d0 2024-04-21T08:38:12.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.131+0000 7f2a66ffd640 1 -- 172.21.15.86:0/1040513443 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c003a90 con 0x55caff5602d0 2024-04-21T08:38:12.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.131+0000 7f2a8663c740 1 -- 172.21.15.86:0/1040513443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caff5602d0 msgr2=0x55caff558170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:12.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.131+0000 7f2a8663c740 1 --2- 172.21.15.86:0/1040513443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caff5602d0 0x55caff558170 secure :-1 s=READY pgs=410 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c011b30 tx=0x7f2a6c00efd0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.131+0000 7f2a8663c740 1 -- 172.21.15.86:0/1040513443 shutdown_connections 2024-04-21T08:38:12.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.131+0000 7f2a8663c740 1 --2- 172.21.15.86:0/1040513443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caff5602d0 0x55caff558170 unknown :-1 s=CLOSED pgs=410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.131+0000 7f2a8663c740 1 --2- 172.21.15.86:0/1040513443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55caff55f830 0x55caff568fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.131+0000 7f2a8663c740 1 --2- 172.21.15.86:0/1040513443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caff55ee40 0x55caff568a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.131+0000 7f2a8663c740 1 -- 172.21.15.86:0/1040513443 >> 172.21.15.86:0/1040513443 conn(0x55caff55af40 msgr2=0x55caff559400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:12.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.131+0000 7f2a8663c740 1 -- 172.21.15.86:0/1040513443 shutdown_connections 2024-04-21T08:38:12.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.131+0000 7f2a8663c740 1 -- 172.21.15.86:0/1040513443 wait complete. 2024-04-21T08:38:12.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.132+0000 7f2a8663c740 1 -- start start 2024-04-21T08:38:12.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.132+0000 7f2a8663c740 1 Processor -- start 2024-04-21T08:38:12.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.132+0000 7f2a8663c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55caff55ee40 0x55caff561d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:12.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.132+0000 7f2a8663c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caff55f830 0x55caff5622d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:12.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.132+0000 7f2a8663c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caff5602d0 0x55caff56a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:12.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.132+0000 7f2a8663c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55caff5771f0 con 0x55caff5602d0 2024-04-21T08:38:12.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.132+0000 7f2a8663c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55caff576b20 con 0x55caff55ee40 2024-04-21T08:38:12.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.132+0000 7f2a8663c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55caff5778c0 con 0x55caff55f830 2024-04-21T08:38:12.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.132+0000 7f2a8663c740 10 client.0 _ll_register_callbacks cb 0x7f2a6c00ed10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:12.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.132+0000 7f2a80fde640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caff5602d0 0x55caff56a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:12.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.132+0000 7f2a80fde640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caff5602d0 0x55caff56a950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36654/0 (socket says 172.21.15.86:36654) 2024-04-21T08:38:12.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.132+0000 7f2a80fde640 1 -- 172.21.15.86:0/1692139695 learned_addr learned my addr 172.21.15.86:0/1692139695 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:12.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.132+0000 7f2a817df640 1 --2- 172.21.15.86:0/1692139695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caff55f830 0x55caff5622d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:12.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.132+0000 7f2a80fde640 1 --2- 172.21.15.86:0/1692139695 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55caff55ee40 0x55caff561d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:12.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.133+0000 7f2a80fde640 1 -- 172.21.15.86:0/1692139695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caff55f830 msgr2=0x55caff5622d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:12.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.133+0000 7f2a80fde640 1 --2- 172.21.15.86:0/1692139695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caff55f830 0x55caff5622d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.133+0000 7f2a80fde640 1 -- 172.21.15.86:0/1692139695 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55caff55ee40 msgr2=0x55caff561d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:12.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.133+0000 7f2a80fde640 1 --2- 172.21.15.86:0/1692139695 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55caff55ee40 0x55caff561d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.133+0000 7f2a80fde640 1 -- 172.21.15.86:0/1692139695 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55caff56af40 con 0x55caff5602d0 2024-04-21T08:38:12.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.133+0000 7f2a80fde640 1 --2- 172.21.15.86:0/1692139695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caff5602d0 0x55caff56a950 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c00eab0 tx=0x7f2a6c013040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=187cc1e9eb4b2669 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:12.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.133+0000 7f2a64ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:38:12.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.133+0000 7f2a64ff9640 1 -- 172.21.15.86:0/1692139695 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a6c003a50 con 0x55caff5602d0 2024-04-21T08:38:12.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.133+0000 7f2a64ff9640 1 -- 172.21.15.86:0/1692139695 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7f2a6c003bf0 con 0x55caff5602d0 2024-04-21T08:38:12.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.133+0000 7f2a8663c740 1 -- 172.21.15.86:0/1692139695 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a6c0053c0 con 0x55caff5602d0 2024-04-21T08:38:12.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.133+0000 7f2a64ff9640 1 -- 172.21.15.86:0/1692139695 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f2a6c004460 con 0x55caff5602d0 2024-04-21T08:38:12.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.133+0000 7f2a8663c740 10 client.15324 fetch_fsmap learned FSMap version 28 2024-04-21T08:38:12.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.133+0000 7f2a8663c740 1 -- 172.21.15.86:0/1692139695 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55caff5dfed0 con 0x55caff5602d0 2024-04-21T08:38:12.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.133+0000 7f2a64ff9640 1 -- 172.21.15.86:0/1692139695 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a6c0053c0 con 0x55caff5602d0 2024-04-21T08:38:12.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.133+0000 7f2a64ff9640 1 -- 172.21.15.86:0/1692139695 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f2a6c01b070 con 0x55caff5602d0 2024-04-21T08:38:12.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.133+0000 7f2a8663c740 10 client.15324 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:38:12.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.133+0000 7f2a8663c740 10 client.15324 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-21T08:38:12.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.133+0000 7f2a8663c740 20 client.15324 populate_metadata read hostname 'smithi086' 2024-04-21T08:38:12.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.133+0000 7f2a8663c740 1 --2- 172.21.15.86:0/1692139695 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55caff5e4650 0x55caff5e1160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:12.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.133+0000 7f2a8663c740 4 client.15324 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:38:12.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.133+0000 7f2a8663c740 1 -- 172.21.15.86:0/1692139695 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "status"}) -- 0x55caff4cb3a0 con 0x55caff5e4650 2024-04-21T08:38:12.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.134+0000 7f2a817df640 1 --2- 172.21.15.86:0/1692139695 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55caff5e4650 0x55caff5e1160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:12.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.134+0000 7f2a64ff9640 10 client.15324 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:38:12.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.134+0000 7f2a817df640 1 --2- 172.21.15.86:0/1692139695 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55caff5e4650 0x55caff5e1160 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=54d53cc9a3c25b4 server_cookie=a9decc6f9d06e22a in_seq=0 out_seq=0 2024-04-21T08:38:12.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.134+0000 7f2a64ff9640 1 -- 172.21.15.86:0/1692139695 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x55caff4cb3a0 con 0x55caff5e4650 2024-04-21T08:38:12.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.134+0000 7f2a64ff9640 10 client.15324 handle_command_reply: tid=0 2024-04-21T08:38:12.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.134+0000 7f2a8663c740 1 client.15324 shutdown 2024-04-21T08:38:12.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.134+0000 7f2a8663c740 1 -- 172.21.15.86:0/1692139695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caff5602d0 msgr2=0x55caff56a950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:12.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.134+0000 7f2a8663c740 1 --2- 172.21.15.86:0/1692139695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caff5602d0 0x55caff56a950 secure :-1 s=READY pgs=411 cs=0 l=1 rev1=1 crypto rx=0x7f2a6c00eab0 tx=0x7f2a6c013040 comp rx=0 tx=0).stop 2024-04-21T08:38:12.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.134+0000 7f2a8663c740 1 -- 172.21.15.86:0/1692139695 shutdown_connections 2024-04-21T08:38:12.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.134+0000 7f2a8663c740 1 --2- 172.21.15.86:0/1692139695 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55caff5e4650 0x55caff5e1160 crc :-1 s=READY pgs=30 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.134+0000 7f2a8663c740 1 --2- 172.21.15.86:0/1692139695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55caff5602d0 0x55caff56a950 unknown :-1 s=CLOSED pgs=411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.134+0000 7f2a8663c740 1 --2- 172.21.15.86:0/1692139695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55caff55f830 0x55caff5622d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.134+0000 7f2a8663c740 1 --2- 172.21.15.86:0/1692139695 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55caff55ee40 0x55caff561d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.134+0000 7f2a8663c740 1 -- 172.21.15.86:0/1692139695 >> 172.21.15.86:0/1692139695 conn(0x55caff55af40 msgr2=0x55caff568a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:12.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.134+0000 7f2a64ff9640 0 client.15324 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:38:12.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.134+0000 7f2a8663c740 1 -- 172.21.15.86:0/1692139695 shutdown_connections 2024-04-21T08:38:12.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.134+0000 7f2a8663c740 1 -- 172.21.15.86:0/1692139695 wait complete. 2024-04-21T08:38:12.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.135+0000 7f2a8663c740 20 client.15324 trim_cache size 0 max 16384 2024-04-21T08:38:12.143 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:38:12.143 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "f8cec7cc-ffb8-11ee-bc93-c7b262605968", 2024-04-21T08:38:12.143 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-21T08:38:12.143 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24482, 2024-04-21T08:38:12.143 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-21T08:38:12.143 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-21T08:38:12.143 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-21T08:38:12.143 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 110.708364211, 2024-04-21T08:38:12.143 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 28, 2024-04-21T08:38:12.143 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 91, 2024-04-21T08:38:12.143 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 89, 2024-04-21T08:38:12.144 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 117.578924298 2024-04-21T08:38:12.144 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:38:12.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.135+0000 7f2a83a43640 1 -- 172.21.15.86:0/233437757 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f2a680a3bb0 msgr2=0x7f2a680a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:12.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.135+0000 7f2a83a43640 1 --2- 172.21.15.86:0/233437757 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f2a680a3bb0 0x7f2a680a6070 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7f2a7809c400 tx=0x7f2a7806efb0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.135+0000 7f2a83a43640 1 -- 172.21.15.86:0/233437757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a7c103b50 msgr2=0x7f2a7c19f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:12.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.135+0000 7f2a83a43640 1 --2- 172.21.15.86:0/233437757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a7c103b50 0x7f2a7c19f800 secure :-1 s=READY pgs=407 cs=0 l=1 rev1=1 crypto rx=0x7f2a7800c970 tx=0x7f2a7800ce30 comp rx=0 tx=0).stop 2024-04-21T08:38:12.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.135+0000 7f2a83a43640 1 -- 172.21.15.86:0/233437757 shutdown_connections 2024-04-21T08:38:12.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.135+0000 7f2a83a43640 1 --2- 172.21.15.86:0/233437757 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f2a680a3bb0 0x7f2a680a6070 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.135+0000 7f2a83a43640 1 --2- 172.21.15.86:0/233437757 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2a7c103b50 0x7f2a7c19f800 unknown :-1 s=CLOSED pgs=407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.135+0000 7f2a83a43640 1 --2- 172.21.15.86:0/233437757 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2a7c103190 0x7f2a7c198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.135+0000 7f2a83a43640 1 --2- 172.21.15.86:0/233437757 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2a7c101fa0 0x7f2a7c198250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:12.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.135+0000 7f2a83a43640 1 -- 172.21.15.86:0/233437757 >> 172.21.15.86:0/233437757 conn(0x7f2a7c0fd530 msgr2=0x7f2a7c1065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:12.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.135+0000 7f2a83a43640 1 -- 172.21.15.86:0/233437757 shutdown_connections 2024-04-21T08:38:12.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:12.135+0000 7f2a83a43640 1 -- 172.21.15.86:0/233437757 wait complete. 2024-04-21T08:38:12.147 INFO:tasks.cephadm:Setup complete, yielding 2024-04-21T08:38:12.147 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-21T08:38:12.157 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi086.front.sepia.ceph.com 2024-04-21T08:38:12.157 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- bash -c 'ceph orch status' 2024-04-21T08:38:12.395 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:38:12.653 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:12 smithi086 ceph-mon[27939]: pgmap v345: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-21T08:38:12.653 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:12 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1456025873' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:12.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:12 smithi088 ceph-mon[31259]: pgmap v345: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-21T08:38:12.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:12 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1456025873' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:12.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:12 smithi179 ceph-mon[31829]: pgmap v345: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-21T08:38:12.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:12 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1456025873' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:14.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.186+0000 7fef068d3700 1 -- 172.21.15.86:0/2947809825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef001025a0 msgr2=0x7fef00109e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:14.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.186+0000 7fef068d3700 1 --2- 172.21.15.86:0/2947809825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef001025a0 0x7fef00109e40 secure :-1 s=READY pgs=412 cs=0 l=1 rev1=1 crypto rx=0x7feef800b0a0 tx=0x7feef800b3b0 comp rx=0 tx=0).stop 2024-04-21T08:38:14.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.186+0000 7fef068d3700 1 -- 172.21.15.86:0/2947809825 shutdown_connections 2024-04-21T08:38:14.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.186+0000 7fef068d3700 1 --2- 172.21.15.86:0/2947809825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef001025a0 0x7fef00109e40 unknown :-1 s=CLOSED pgs=412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:14.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.186+0000 7fef068d3700 1 --2- 172.21.15.86:0/2947809825 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef00103010 0x7fef00103430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:14.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.186+0000 7fef068d3700 1 --2- 172.21.15.86:0/2947809825 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef00101be0 0x7fef00102060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:14.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.186+0000 7fef068d3700 1 -- 172.21.15.86:0/2947809825 >> 172.21.15.86:0/2947809825 conn(0x7fef000fc6d0 msgr2=0x7fef000feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:14.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.186+0000 7fef068d3700 1 -- 172.21.15.86:0/2947809825 shutdown_connections 2024-04-21T08:38:14.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.186+0000 7fef068d3700 1 -- 172.21.15.86:0/2947809825 wait complete. 2024-04-21T08:38:14.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.186+0000 7fef068d3700 1 Processor -- start 2024-04-21T08:38:14.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.186+0000 7fef068d3700 1 -- start start 2024-04-21T08:38:14.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.187+0000 7fef068d3700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef00101be0 0x7fef00101630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:14.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.187+0000 7fef068d3700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef001025a0 0x7fef0019fc30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:14.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.187+0000 7fef068d3700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef00103010 0x7fef001a2180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:14.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.187+0000 7fef068d3700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fef0010d480 con 0x7fef00101be0 2024-04-21T08:38:14.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.187+0000 7fef068d3700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fef0010d300 con 0x7fef00103010 2024-04-21T08:38:14.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.187+0000 7fef068d3700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fef0010d600 con 0x7fef001025a0 2024-04-21T08:38:14.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.187+0000 7feefffff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef00101be0 0x7fef00101630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:14.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.187+0000 7feefffff700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef00101be0 0x7fef00101630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36676/0 (socket says 172.21.15.86:36676) 2024-04-21T08:38:14.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.187+0000 7feefffff700 1 -- 172.21.15.86:0/691475882 learned_addr learned my addr 172.21.15.86:0/691475882 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:14.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.187+0000 7feeff7fe700 1 --2- 172.21.15.86:0/691475882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef001025a0 0x7fef0019fc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:14.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.187+0000 7feefffff700 1 -- 172.21.15.86:0/691475882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef001025a0 msgr2=0x7fef0019fc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:14.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.187+0000 7feefffff700 1 --2- 172.21.15.86:0/691475882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef001025a0 0x7fef0019fc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:14.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.187+0000 7feefffff700 1 -- 172.21.15.86:0/691475882 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef00103010 msgr2=0x7fef001a2180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:14.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.187+0000 7feefffff700 1 --2- 172.21.15.86:0/691475882 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef00103010 0x7fef001a2180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:14.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.187+0000 7feefffff700 1 -- 172.21.15.86:0/691475882 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fef001a2960 con 0x7fef00101be0 2024-04-21T08:38:14.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.187+0000 7feefffff700 1 --2- 172.21.15.86:0/691475882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef00101be0 0x7fef00101630 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7feef400c9b0 tx=0x7feef400ccc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:14.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.187+0000 7feefd7fa700 1 -- 172.21.15.86:0/691475882 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef4007b00 con 0x7fef00101be0 2024-04-21T08:38:14.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.187+0000 7fef068d3700 1 -- 172.21.15.86:0/691475882 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fef001a2c50 con 0x7fef00101be0 2024-04-21T08:38:14.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.187+0000 7feefd7fa700 1 -- 172.21.15.86:0/691475882 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7feef4007ca0 con 0x7fef00101be0 2024-04-21T08:38:14.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.187+0000 7feefd7fa700 1 -- 172.21.15.86:0/691475882 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7feef4018660 con 0x7fef00101be0 2024-04-21T08:38:14.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.187+0000 7fef068d3700 1 -- 172.21.15.86:0/691475882 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fef001a3210 con 0x7fef00101be0 2024-04-21T08:38:14.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.189+0000 7fef068d3700 1 -- 172.21.15.86:0/691475882 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fef00069730 con 0x7fef00101be0 2024-04-21T08:38:14.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.190+0000 7feefd7fa700 1 -- 172.21.15.86:0/691475882 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7feef400f450 con 0x7fef00101be0 2024-04-21T08:38:14.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.191+0000 7feefd7fa700 1 --2- 172.21.15.86:0/691475882 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7feee80a3cc0 0x7feee80a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:14.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.191+0000 7feefd7fa700 1 -- 172.21.15.86:0/691475882 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7feef40d07d0 con 0x7fef00101be0 2024-04-21T08:38:14.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.191+0000 7feeff7fe700 1 --2- 172.21.15.86:0/691475882 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7feee80a3cc0 0x7feee80a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:14.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.191+0000 7feeff7fe700 1 --2- 172.21.15.86:0/691475882 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7feee80a3cc0 0x7feee80a6180 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fef00101bb0 tx=0x7feef0009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:14.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.195+0000 7feefd7fa700 1 -- 172.21.15.86:0/691475882 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7feef4014030 con 0x7fef00101be0 2024-04-21T08:38:14.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.462+0000 7fef068d3700 1 -- 172.21.15.86:0/691475882 --> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] -- mgr_command(tid 0: {"prefix": "orch status", "target": ["mon-mgr", ""]}) -- 0x7fef00062e80 con 0x7feee80a3cc0 2024-04-21T08:38:14.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.463+0000 7feefd7fa700 1 -- 172.21.15.86:0/691475882 <== mgr.14152 v2:172.21.15.86:6800/1364156965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+43 (secure 0 0 0) 0x7fef00062e80 con 0x7feee80a3cc0 2024-04-21T08:38:14.463 INFO:teuthology.orchestra.run.smithi086.stdout:Backend: cephadm 2024-04-21T08:38:14.463 INFO:teuthology.orchestra.run.smithi086.stdout:Available: Yes 2024-04-21T08:38:14.463 INFO:teuthology.orchestra.run.smithi086.stdout:Paused: No 2024-04-21T08:38:14.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.465+0000 7fef068d3700 1 -- 172.21.15.86:0/691475882 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7feee80a3cc0 msgr2=0x7feee80a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:14.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.465+0000 7fef068d3700 1 --2- 172.21.15.86:0/691475882 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7feee80a3cc0 0x7feee80a6180 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fef00101bb0 tx=0x7feef0009190 comp rx=0 tx=0).stop 2024-04-21T08:38:14.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.465+0000 7fef068d3700 1 -- 172.21.15.86:0/691475882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef00101be0 msgr2=0x7fef00101630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:14.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.465+0000 7fef068d3700 1 --2- 172.21.15.86:0/691475882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef00101be0 0x7fef00101630 secure :-1 s=READY pgs=413 cs=0 l=1 rev1=1 crypto rx=0x7feef400c9b0 tx=0x7feef400ccc0 comp rx=0 tx=0).stop 2024-04-21T08:38:14.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.466+0000 7fef068d3700 1 -- 172.21.15.86:0/691475882 shutdown_connections 2024-04-21T08:38:14.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.466+0000 7fef068d3700 1 --2- 172.21.15.86:0/691475882 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7feee80a3cc0 0x7feee80a6180 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:14.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.466+0000 7fef068d3700 1 --2- 172.21.15.86:0/691475882 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fef00103010 0x7fef001a2180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:14.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.466+0000 7fef068d3700 1 --2- 172.21.15.86:0/691475882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fef00101be0 0x7fef00101630 unknown :-1 s=CLOSED pgs=413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:14.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.466+0000 7fef068d3700 1 --2- 172.21.15.86:0/691475882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fef001025a0 0x7fef0019fc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:14.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.466+0000 7fef068d3700 1 -- 172.21.15.86:0/691475882 >> 172.21.15.86:0/691475882 conn(0x7fef000fc6d0 msgr2=0x7fef00105e90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:14.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.466+0000 7fef068d3700 1 -- 172.21.15.86:0/691475882 shutdown_connections 2024-04-21T08:38:14.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:14.466+0000 7fef068d3700 1 -- 172.21.15.86:0/691475882 wait complete. 2024-04-21T08:38:14.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:14 smithi088 ceph-mon[31259]: pgmap v346: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:38:14.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:14 smithi179 ceph-mon[31829]: pgmap v346: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:38:14.785 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:14 smithi086 ceph-mon[27939]: pgmap v346: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:38:15.238 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- bash -c 'ceph orch ps' 2024-04-21T08:38:15.485 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:38:15.607 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:15 smithi086 ceph-mon[27939]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:15.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:15 smithi088 ceph-mon[31259]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:15.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:15 smithi179 ceph-mon[31829]: from='client.15330 -' entity='client.admin' cmd=[{"prefix": "orch status", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:16.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:16 smithi086 ceph-mon[27939]: pgmap v347: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-21T08:38:16.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:16 smithi088 ceph-mon[31259]: pgmap v347: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-21T08:38:16.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:16 smithi179 ceph-mon[31829]: pgmap v347: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 11 op/s 2024-04-21T08:38:17.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.326+0000 7fcc64219700 1 -- 172.21.15.86:0/1866584990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc5c103540 msgr2=0x7fcc5c109df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:17.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.326+0000 7fcc64219700 1 --2- 172.21.15.86:0/1866584990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc5c103540 0x7fcc5c109df0 secure :-1 s=READY pgs=414 cs=0 l=1 rev1=1 crypto rx=0x7fcc540099c0 tx=0x7fcc54009cd0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.326+0000 7fcc64219700 1 -- 172.21.15.86:0/1866584990 shutdown_connections 2024-04-21T08:38:17.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.326+0000 7fcc64219700 1 --2- 172.21.15.86:0/1866584990 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc5c103540 0x7fcc5c109df0 unknown :-1 s=CLOSED pgs=414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.326+0000 7fcc64219700 1 --2- 172.21.15.86:0/1866584990 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc5c1019a0 0x7fcc5c101dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.326+0000 7fcc64219700 1 --2- 172.21.15.86:0/1866584990 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc5c102ba0 0x7fcc5c103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.326+0000 7fcc64219700 1 -- 172.21.15.86:0/1866584990 >> 172.21.15.86:0/1866584990 conn(0x7fcc5c0fcf40 msgr2=0x7fcc5c0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:17.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.326+0000 7fcc64219700 1 -- 172.21.15.86:0/1866584990 shutdown_connections 2024-04-21T08:38:17.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.326+0000 7fcc64219700 1 -- 172.21.15.86:0/1866584990 wait complete. 2024-04-21T08:38:17.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.327+0000 7fcc64219700 1 Processor -- start 2024-04-21T08:38:17.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.327+0000 7fcc64219700 1 -- start start 2024-04-21T08:38:17.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.327+0000 7fcc64219700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc5c1019a0 0x7fcc5c06b280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:17.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.327+0000 7fcc64219700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc5c102ba0 0x7fcc5c19da20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:17.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.327+0000 7fcc64219700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc5c103540 0x7fcc5c1a0060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:17.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.327+0000 7fcc64219700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c10d530 con 0x7fcc5c1019a0 2024-04-21T08:38:17.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.327+0000 7fcc64219700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c10d3b0 con 0x7fcc5c102ba0 2024-04-21T08:38:17.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.327+0000 7fcc64219700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fcc5c10d6b0 con 0x7fcc5c103540 2024-04-21T08:38:17.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.327+0000 7fcc61fb5700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc5c1019a0 0x7fcc5c06b280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:17.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.327+0000 7fcc61fb5700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc5c1019a0 0x7fcc5c06b280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36702/0 (socket says 172.21.15.86:36702) 2024-04-21T08:38:17.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.327+0000 7fcc61fb5700 1 -- 172.21.15.86:0/2089319009 learned_addr learned my addr 172.21.15.86:0/2089319009 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:17.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.327+0000 7fcc617b4700 1 --2- 172.21.15.86:0/2089319009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc5c102ba0 0x7fcc5c19da20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:17.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.327+0000 7fcc627b6700 1 --2- 172.21.15.86:0/2089319009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc5c103540 0x7fcc5c1a0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:17.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.327+0000 7fcc61fb5700 1 -- 172.21.15.86:0/2089319009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc5c103540 msgr2=0x7fcc5c1a0060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:17.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.327+0000 7fcc61fb5700 1 --2- 172.21.15.86:0/2089319009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc5c103540 0x7fcc5c1a0060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.327+0000 7fcc61fb5700 1 -- 172.21.15.86:0/2089319009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc5c102ba0 msgr2=0x7fcc5c19da20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:17.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.327+0000 7fcc61fb5700 1 --2- 172.21.15.86:0/2089319009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc5c102ba0 0x7fcc5c19da20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.327+0000 7fcc61fb5700 1 -- 172.21.15.86:0/2089319009 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc5c1a0930 con 0x7fcc5c1019a0 2024-04-21T08:38:17.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.327+0000 7fcc617b4700 1 --2- 172.21.15.86:0/2089319009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc5c102ba0 0x7fcc5c19da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:17.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.327+0000 7fcc61fb5700 1 --2- 172.21.15.86:0/2089319009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc5c1019a0 0x7fcc5c06b280 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fcc5800b9c0 tx=0x7fcc5800bd80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:17.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.327+0000 7fcc52ffd700 1 -- 172.21.15.86:0/2089319009 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc58019070 con 0x7fcc5c1019a0 2024-04-21T08:38:17.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.328+0000 7fcc64219700 1 -- 172.21.15.86:0/2089319009 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc5c1a0c20 con 0x7fcc5c1019a0 2024-04-21T08:38:17.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.328+0000 7fcc52ffd700 1 -- 172.21.15.86:0/2089319009 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc58014440 con 0x7fcc5c1019a0 2024-04-21T08:38:17.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.328+0000 7fcc52ffd700 1 -- 172.21.15.86:0/2089319009 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc580125a0 con 0x7fcc5c1019a0 2024-04-21T08:38:17.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.328+0000 7fcc64219700 1 -- 172.21.15.86:0/2089319009 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc5c1a11b0 con 0x7fcc5c1019a0 2024-04-21T08:38:17.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.328+0000 7fcc64219700 1 -- 172.21.15.86:0/2089319009 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcc5c069730 con 0x7fcc5c1019a0 2024-04-21T08:38:17.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.330+0000 7fcc52ffd700 1 -- 172.21.15.86:0/2089319009 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fcc5801c070 con 0x7fcc5c1019a0 2024-04-21T08:38:17.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.330+0000 7fcc52ffd700 1 --2- 172.21.15.86:0/2089319009 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fcc480a3c40 0x7fcc480a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:17.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.330+0000 7fcc617b4700 1 --2- 172.21.15.86:0/2089319009 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fcc480a3c40 0x7fcc480a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:17.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.330+0000 7fcc52ffd700 1 -- 172.21.15.86:0/2089319009 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fcc580d0760 con 0x7fcc5c1019a0 2024-04-21T08:38:17.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.331+0000 7fcc617b4700 1 --2- 172.21.15.86:0/2089319009 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fcc480a3c40 0x7fcc480a6100 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c06b800 tx=0x7fcc4c008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:17.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.334+0000 7fcc52ffd700 1 -- 172.21.15.86:0/2089319009 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcc58023030 con 0x7fcc5c1019a0 2024-04-21T08:38:17.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.545+0000 7fcc64219700 1 -- 172.21.15.86:0/2089319009 --> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] -- mgr_command(tid 0: {"prefix": "orch ps", "target": ["mon-mgr", ""]}) -- 0x7fcc5c062e80 con 0x7fcc480a3c40 2024-04-21T08:38:17.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.558+0000 7fcc52ffd700 1 -- 172.21.15.86:0/2089319009 <== mgr.14152 v2:172.21.15.86:6800/1364156965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+4092 (secure 0 0 0) 0x7fcc5c062e80 con 0x7fcc480a3c40 2024-04-21T08:38:17.557 INFO:teuthology.orchestra.run.smithi086.stdout:NAME HOST PORTS STATUS REFRESHED AGE MEM USE MEM LIM VERSION IMAGE ID CONTAINER ID 2024-04-21T08:38:17.557 INFO:teuthology.orchestra.run.smithi086.stdout:mds.a smithi086 running (2m) 19s ago 2m 16.2M - 19.0.0-3154-ga630765c b45a994450e2 f9fbd33c53f6 2024-04-21T08:38:17.558 INFO:teuthology.orchestra.run.smithi086.stdout:mds.b smithi088 running (2m) 56s ago 2m 28.6M - 19.0.0-3154-ga630765c b45a994450e2 6ae947751546 2024-04-21T08:38:17.558 INFO:teuthology.orchestra.run.smithi086.stdout:mds.c smithi179 running (2m) 56s ago 2m 19.3M - 19.0.0-3154-ga630765c b45a994450e2 0cbbb963e350 2024-04-21T08:38:17.558 INFO:teuthology.orchestra.run.smithi086.stdout:mds.d smithi086 running (2m) 19s ago 2m 16.4M - 19.0.0-3154-ga630765c b45a994450e2 ce45a36ff107 2024-04-21T08:38:17.558 INFO:teuthology.orchestra.run.smithi086.stdout:mds.e smithi088 running (2m) 56s ago 2m 16.0M - 19.0.0-3154-ga630765c b45a994450e2 110f8ccf6dd1 2024-04-21T08:38:17.558 INFO:teuthology.orchestra.run.smithi086.stdout:mds.f smithi179 running (2m) 56s ago 2m 18.7M - 19.0.0-3154-ga630765c b45a994450e2 09df6c287e69 2024-04-21T08:38:17.558 INFO:teuthology.orchestra.run.smithi086.stdout:mds.g smithi086 running (2m) 19s ago 2m 16.2M - 19.0.0-3154-ga630765c b45a994450e2 6d381c165a0f 2024-04-21T08:38:17.558 INFO:teuthology.orchestra.run.smithi086.stdout:mds.h smithi088 running (2m) 56s ago 2m 16.9M - 19.0.0-3154-ga630765c b45a994450e2 708fa1f6099b 2024-04-21T08:38:17.558 INFO:teuthology.orchestra.run.smithi086.stdout:mds.i smithi179 running (2m) 56s ago 2m 17.8M - 19.0.0-3154-ga630765c b45a994450e2 52eb9c4c876e 2024-04-21T08:38:17.558 INFO:teuthology.orchestra.run.smithi086.stdout:mds.j smithi086 running (2m) 19s ago 2m 14.0M - 19.0.0-3154-ga630765c b45a994450e2 845db0fd9413 2024-04-21T08:38:17.558 INFO:teuthology.orchestra.run.smithi086.stdout:mds.k smithi088 running (2m) 56s ago 2m 17.3M - 19.0.0-3154-ga630765c b45a994450e2 e43b96f6ea27 2024-04-21T08:38:17.558 INFO:teuthology.orchestra.run.smithi086.stdout:mds.l smithi179 running (2m) 56s ago 2m 16.3M - 19.0.0-3154-ga630765c b45a994450e2 c381a381724c 2024-04-21T08:38:17.558 INFO:teuthology.orchestra.run.smithi086.stdout:mgr.x smithi086 *:9283,8765 running (9m) 19s ago 9m 518M - 19.0.0-3154-ga630765c b45a994450e2 bbba55e9c9c1 2024-04-21T08:38:17.558 INFO:teuthology.orchestra.run.smithi086.stdout:mgr.y smithi088 *:8443,8765 running (7m) 56s ago 7m 454M - 19.0.0-3154-ga630765c b45a994450e2 d163c7a7cdb5 2024-04-21T08:38:17.559 INFO:teuthology.orchestra.run.smithi086.stdout:mgr.z smithi179 *:8443,8765 running (7m) 56s ago 7m 455M - 19.0.0-3154-ga630765c b45a994450e2 b71731ad80c8 2024-04-21T08:38:17.559 INFO:teuthology.orchestra.run.smithi086.stdout:mon.a smithi086 running (9m) 19s ago 9m 51.0M 2048M 19.0.0-3154-ga630765c b45a994450e2 f70ad6da7b69 2024-04-21T08:38:17.559 INFO:teuthology.orchestra.run.smithi086.stdout:mon.b smithi088 running (8m) 56s ago 8m 39.6M 2048M 19.0.0-3154-ga630765c b45a994450e2 2386b428da3a 2024-04-21T08:38:17.559 INFO:teuthology.orchestra.run.smithi086.stdout:mon.c smithi179 running (8m) 56s ago 8m 42.2M 2048M 19.0.0-3154-ga630765c b45a994450e2 e2e289b587bc 2024-04-21T08:38:17.559 INFO:teuthology.orchestra.run.smithi086.stdout:osd.0 smithi086 running (7m) 19s ago 7m 45.6M 4272M 19.0.0-3154-ga630765c b45a994450e2 6dbe1fac2a65 2024-04-21T08:38:17.559 INFO:teuthology.orchestra.run.smithi086.stdout:osd.1 smithi088 running (7m) 56s ago 7m 54.5M 4272M 19.0.0-3154-ga630765c b45a994450e2 f8ea0f5e9ef0 2024-04-21T08:38:17.559 INFO:teuthology.orchestra.run.smithi086.stdout:osd.2 smithi179 running (6m) 56s ago 6m 55.4M 4272M 19.0.0-3154-ga630765c b45a994450e2 fae7dd6aa9f7 2024-04-21T08:38:17.559 INFO:teuthology.orchestra.run.smithi086.stdout:osd.3 smithi086 running (6m) 19s ago 6m 44.5M 4272M 19.0.0-3154-ga630765c b45a994450e2 ecb378610ce2 2024-04-21T08:38:17.559 INFO:teuthology.orchestra.run.smithi086.stdout:osd.4 smithi088 running (5m) 56s ago 5m 51.5M 4272M 19.0.0-3154-ga630765c b45a994450e2 f41c09f675bf 2024-04-21T08:38:17.559 INFO:teuthology.orchestra.run.smithi086.stdout:osd.5 smithi179 running (5m) 56s ago 5m 53.3M 4272M 19.0.0-3154-ga630765c b45a994450e2 2c39de4ecbad 2024-04-21T08:38:17.559 INFO:teuthology.orchestra.run.smithi086.stdout:osd.6 smithi086 running (5m) 19s ago 5m 41.8M 4272M 19.0.0-3154-ga630765c b45a994450e2 b1aae3596c7d 2024-04-21T08:38:17.559 INFO:teuthology.orchestra.run.smithi086.stdout:osd.7 smithi088 running (4m) 56s ago 4m 46.6M 4272M 19.0.0-3154-ga630765c b45a994450e2 b2654b41ae16 2024-04-21T08:38:17.559 INFO:teuthology.orchestra.run.smithi086.stdout:osd.8 smithi179 running (4m) 56s ago 4m 45.9M 4272M 19.0.0-3154-ga630765c b45a994450e2 7239adc6ea73 2024-04-21T08:38:17.559 INFO:teuthology.orchestra.run.smithi086.stdout:osd.9 smithi086 running (3m) 19s ago 3m 48.4M 4272M 19.0.0-3154-ga630765c b45a994450e2 7d5034b82b7d 2024-04-21T08:38:17.560 INFO:teuthology.orchestra.run.smithi086.stdout:osd.10 smithi088 running (3m) 56s ago 3m 44.3M 4272M 19.0.0-3154-ga630765c b45a994450e2 df5e727d553d 2024-04-21T08:38:17.560 INFO:teuthology.orchestra.run.smithi086.stdout:osd.11 smithi179 running (2m) 56s ago 2m 48.7M 4272M 19.0.0-3154-ga630765c b45a994450e2 77c37a049b52 2024-04-21T08:38:17.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.561+0000 7fcc64219700 1 -- 172.21.15.86:0/2089319009 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fcc480a3c40 msgr2=0x7fcc480a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:17.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.561+0000 7fcc64219700 1 --2- 172.21.15.86:0/2089319009 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fcc480a3c40 0x7fcc480a6100 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fcc5c06b800 tx=0x7fcc4c008040 comp rx=0 tx=0).stop 2024-04-21T08:38:17.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.561+0000 7fcc64219700 1 -- 172.21.15.86:0/2089319009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc5c1019a0 msgr2=0x7fcc5c06b280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:17.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.561+0000 7fcc64219700 1 --2- 172.21.15.86:0/2089319009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc5c1019a0 0x7fcc5c06b280 secure :-1 s=READY pgs=415 cs=0 l=1 rev1=1 crypto rx=0x7fcc5800b9c0 tx=0x7fcc5800bd80 comp rx=0 tx=0).stop 2024-04-21T08:38:17.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.561+0000 7fcc64219700 1 -- 172.21.15.86:0/2089319009 shutdown_connections 2024-04-21T08:38:17.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.561+0000 7fcc64219700 1 --2- 172.21.15.86:0/2089319009 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fcc480a3c40 0x7fcc480a6100 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.561+0000 7fcc64219700 1 --2- 172.21.15.86:0/2089319009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc5c103540 0x7fcc5c1a0060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.561+0000 7fcc64219700 1 --2- 172.21.15.86:0/2089319009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc5c1019a0 0x7fcc5c06b280 unknown :-1 s=CLOSED pgs=415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.561+0000 7fcc64219700 1 --2- 172.21.15.86:0/2089319009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc5c102ba0 0x7fcc5c19da20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:17.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.561+0000 7fcc64219700 1 -- 172.21.15.86:0/2089319009 >> 172.21.15.86:0/2089319009 conn(0x7fcc5c0fcf40 msgr2=0x7fcc5c105f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:17.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.561+0000 7fcc64219700 1 -- 172.21.15.86:0/2089319009 shutdown_connections 2024-04-21T08:38:17.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:17.561+0000 7fcc64219700 1 -- 172.21.15.86:0/2089319009 wait complete. 2024-04-21T08:38:18.426 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- bash -c 'ceph orch ls' 2024-04-21T08:38:18.669 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:38:18.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:18 smithi088 ceph-mon[31259]: pgmap v348: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-21T08:38:18.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:18 smithi179 ceph-mon[31829]: pgmap v348: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-21T08:38:18.791 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:18 smithi086 ceph-mon[27939]: pgmap v348: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-21T08:38:19.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:19 smithi088 ceph-mon[31259]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:19.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:19 smithi179 ceph-mon[31829]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:19.817 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:19 smithi086 ceph-mon[27939]: from='client.15336 -' entity='client.admin' cmd=[{"prefix": "orch ps", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:20.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.536+0000 7fca08f57700 1 -- 172.21.15.86:0/2726673471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca04102bc0 msgr2=0x7fca04103020 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:20.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.536+0000 7fca08f57700 1 --2- 172.21.15.86:0/2726673471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca04102bc0 0x7fca04103020 secure :-1 s=READY pgs=416 cs=0 l=1 rev1=1 crypto rx=0x7fc9f80099c0 tx=0x7fc9f8009cd0 comp rx=0 tx=0).stop 2024-04-21T08:38:20.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.537+0000 7fca08f57700 1 -- 172.21.15.86:0/2726673471 shutdown_connections 2024-04-21T08:38:20.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.537+0000 7fca08f57700 1 --2- 172.21.15.86:0/2726673471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca04103560 0x7fca04109e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:20.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.537+0000 7fca08f57700 1 --2- 172.21.15.86:0/2726673471 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca041019c0 0x7fca04101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:20.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.537+0000 7fca08f57700 1 --2- 172.21.15.86:0/2726673471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca04102bc0 0x7fca04103020 unknown :-1 s=CLOSED pgs=416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:20.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.537+0000 7fca08f57700 1 -- 172.21.15.86:0/2726673471 >> 172.21.15.86:0/2726673471 conn(0x7fca040fcfa0 msgr2=0x7fca040ff3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:20.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.537+0000 7fca08f57700 1 -- 172.21.15.86:0/2726673471 shutdown_connections 2024-04-21T08:38:20.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.537+0000 7fca08f57700 1 -- 172.21.15.86:0/2726673471 wait complete. 2024-04-21T08:38:20.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.537+0000 7fca08f57700 1 Processor -- start 2024-04-21T08:38:20.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.537+0000 7fca08f57700 1 -- start start 2024-04-21T08:38:20.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.537+0000 7fca08f57700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca041019c0 0x7fca0406bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:20.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.538+0000 7fca08f57700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca04102bc0 0x7fca0419da00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:20.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.538+0000 7fca08f57700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca04103560 0x7fca041a0040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:20.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.538+0000 7fca08f57700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fca0410d4d0 con 0x7fca04102bc0 2024-04-21T08:38:20.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.538+0000 7fca08f57700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fca0410d350 con 0x7fca04103560 2024-04-21T08:38:20.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.538+0000 7fca08f57700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fca0410d650 con 0x7fca041019c0 2024-04-21T08:38:20.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.538+0000 7fca01d9b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca04102bc0 0x7fca0419da00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:20.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.538+0000 7fca01d9b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca04102bc0 0x7fca0419da00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54214/0 (socket says 172.21.15.86:54214) 2024-04-21T08:38:20.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.538+0000 7fca01d9b700 1 -- 172.21.15.86:0/2714303476 learned_addr learned my addr 172.21.15.86:0/2714303476 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:20.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.538+0000 7fca0259c700 1 --2- 172.21.15.86:0/2714303476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca041019c0 0x7fca0406bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:20.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.538+0000 7fca02d9d700 1 --2- 172.21.15.86:0/2714303476 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca04103560 0x7fca041a0040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:20.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.538+0000 7fca01d9b700 1 -- 172.21.15.86:0/2714303476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca041019c0 msgr2=0x7fca0406bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:20.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.538+0000 7fca01d9b700 1 --2- 172.21.15.86:0/2714303476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca041019c0 0x7fca0406bd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:20.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.538+0000 7fca01d9b700 1 -- 172.21.15.86:0/2714303476 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca04103560 msgr2=0x7fca041a0040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:20.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.538+0000 7fca01d9b700 1 --2- 172.21.15.86:0/2714303476 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca04103560 0x7fca041a0040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:20.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.538+0000 7fca01d9b700 1 -- 172.21.15.86:0/2714303476 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca041a0910 con 0x7fca04102bc0 2024-04-21T08:38:20.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.538+0000 7fca01d9b700 1 --2- 172.21.15.86:0/2714303476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca04102bc0 0x7fca0419da00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8003790 tx=0x7fc9f8003a10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:20.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.538+0000 7fc9f37fe700 1 -- 172.21.15.86:0/2714303476 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f8027070 con 0x7fca04102bc0 2024-04-21T08:38:20.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.538+0000 7fc9f37fe700 1 -- 172.21.15.86:0/2714303476 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9f8003f80 con 0x7fca04102bc0 2024-04-21T08:38:20.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.538+0000 7fca08f57700 1 -- 172.21.15.86:0/2714303476 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca041a0ba0 con 0x7fca04102bc0 2024-04-21T08:38:20.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.538+0000 7fc9f37fe700 1 -- 172.21.15.86:0/2714303476 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9f800f460 con 0x7fca04102bc0 2024-04-21T08:38:20.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.538+0000 7fca08f57700 1 -- 172.21.15.86:0/2714303476 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca041a1080 con 0x7fca04102bc0 2024-04-21T08:38:20.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.539+0000 7fca08f57700 1 -- 172.21.15.86:0/2714303476 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca04069730 con 0x7fca04102bc0 2024-04-21T08:38:20.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.541+0000 7fc9f37fe700 1 -- 172.21.15.86:0/2714303476 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fc9f802a070 con 0x7fca04102bc0 2024-04-21T08:38:20.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.541+0000 7fc9f37fe700 1 --2- 172.21.15.86:0/2714303476 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fc9ec0a3c40 0x7fc9ec0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:20.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.542+0000 7fca0259c700 1 --2- 172.21.15.86:0/2714303476 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fc9ec0a3c40 0x7fc9ec0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:20.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.542+0000 7fc9f37fe700 1 -- 172.21.15.86:0/2714303476 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(91..91 src has 1..91) ==== 9092+0+0 (secure 0 0 0) 0x7fc9f80d1bb0 con 0x7fca04102bc0 2024-04-21T08:38:20.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.542+0000 7fca0259c700 1 --2- 172.21.15.86:0/2714303476 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fc9ec0a3c40 0x7fc9ec0a6100 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4006fd0 tx=0x7fc9f4008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:20.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.545+0000 7fc9f37fe700 1 -- 172.21.15.86:0/2714303476 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc9f8013070 con 0x7fca04102bc0 2024-04-21T08:38:20.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:20 smithi088 ceph-mon[31259]: pgmap v349: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:38:20.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:20 smithi179 ceph-mon[31829]: pgmap v349: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:38:20.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.755+0000 7fca08f57700 1 -- 172.21.15.86:0/2714303476 --> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] -- mgr_command(tid 0: {"prefix": "orch ls", "target": ["mon-mgr", ""]}) -- 0x7fca04062e80 con 0x7fc9ec0a3c40 2024-04-21T08:38:20.755 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:20 smithi086 ceph-mon[27939]: pgmap v349: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:38:20.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.761+0000 7fc9f37fe700 1 -- 172.21.15.86:0/2714303476 <== mgr.14152 v2:172.21.15.86:6800/1364156965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+980 (secure 0 0 0) 0x7fca04062e80 con 0x7fc9ec0a3c40 2024-04-21T08:38:20.761 INFO:teuthology.orchestra.run.smithi086.stdout:NAME PORTS RUNNING REFRESHED AGE PLACEMENT 2024-04-21T08:38:20.761 INFO:teuthology.orchestra.run.smithi086.stdout:mds.all 12/12 60s ago 2m smithi086=a;smithi086=d;smithi086=g;smithi086=j;smithi088=b;smithi088=e;smithi088=h;smithi088=k;smithi179=c;smithi179=f;smithi179=i;smithi179=l;count:12 2024-04-21T08:38:20.761 INFO:teuthology.orchestra.run.smithi086.stdout:mgr 3/3 60s ago 7m smithi086=x;smithi088=y;smithi179=z;count:3 2024-04-21T08:38:20.761 INFO:teuthology.orchestra.run.smithi086.stdout:mon 3/3 60s ago 8m smithi086:172.21.15.86=a;smithi088:172.21.15.88=b;smithi179:172.21.15.179=c;count:3 2024-04-21T08:38:20.761 INFO:teuthology.orchestra.run.smithi086.stdout:osd 12 60s ago - 2024-04-21T08:38:20.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.764+0000 7fca08f57700 1 -- 172.21.15.86:0/2714303476 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fc9ec0a3c40 msgr2=0x7fc9ec0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:20.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.764+0000 7fca08f57700 1 --2- 172.21.15.86:0/2714303476 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fc9ec0a3c40 0x7fc9ec0a6100 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fc9f4006fd0 tx=0x7fc9f4008040 comp rx=0 tx=0).stop 2024-04-21T08:38:20.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.764+0000 7fca08f57700 1 -- 172.21.15.86:0/2714303476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca04102bc0 msgr2=0x7fca0419da00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:20.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.764+0000 7fca08f57700 1 --2- 172.21.15.86:0/2714303476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca04102bc0 0x7fca0419da00 secure :-1 s=READY pgs=417 cs=0 l=1 rev1=1 crypto rx=0x7fc9f8003790 tx=0x7fc9f8003a10 comp rx=0 tx=0).stop 2024-04-21T08:38:20.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.764+0000 7fca08f57700 1 -- 172.21.15.86:0/2714303476 shutdown_connections 2024-04-21T08:38:20.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.764+0000 7fca08f57700 1 --2- 172.21.15.86:0/2714303476 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fc9ec0a3c40 0x7fc9ec0a6100 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:20.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.764+0000 7fca08f57700 1 --2- 172.21.15.86:0/2714303476 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca04103560 0x7fca041a0040 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:20.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.764+0000 7fca08f57700 1 --2- 172.21.15.86:0/2714303476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca041019c0 0x7fca0406bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:20.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.764+0000 7fca08f57700 1 --2- 172.21.15.86:0/2714303476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca04102bc0 0x7fca0419da00 unknown :-1 s=CLOSED pgs=417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:20.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.764+0000 7fca08f57700 1 -- 172.21.15.86:0/2714303476 >> 172.21.15.86:0/2714303476 conn(0x7fca040fcfa0 msgr2=0x7fca04105fe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:20.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.764+0000 7fca08f57700 1 -- 172.21.15.86:0/2714303476 shutdown_connections 2024-04-21T08:38:20.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:20.764+0000 7fca08f57700 1 -- 172.21.15.86:0/2714303476 wait complete. 2024-04-21T08:38:21.525 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- bash -c 'ceph orch host ls' 2024-04-21T08:38:21.765 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:38:22.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:22 smithi086 ceph-mon[27939]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:22.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:22 smithi086 ceph-mon[27939]: pgmap v350: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-21T08:38:22.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:22 smithi088 ceph-mon[31259]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:22.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:22 smithi088 ceph-mon[31259]: pgmap v350: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-21T08:38:22.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:22 smithi179 ceph-mon[31829]: from='client.15342 -' entity='client.admin' cmd=[{"prefix": "orch ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:22.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:22 smithi179 ceph-mon[31829]: pgmap v350: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.8 KiB/s rd, 10 op/s 2024-04-21T08:38:23.557 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:23 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-04-21T08:38:23.557 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:23 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]: dispatch 2024-04-21T08:38:23.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.660+0000 7f8607bb9700 1 -- 172.21.15.86:0/4129845701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8600103560 msgr2=0x7f8600109e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:23.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.660+0000 7f8607bb9700 1 --2- 172.21.15.86:0/4129845701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8600103560 0x7f8600109e10 secure :-1 s=READY pgs=418 cs=0 l=1 rev1=1 crypto rx=0x7f85f800b0d0 tx=0x7f85f800b3e0 comp rx=0 tx=0).stop 2024-04-21T08:38:23.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.660+0000 7f8607bb9700 1 -- 172.21.15.86:0/4129845701 shutdown_connections 2024-04-21T08:38:23.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.660+0000 7f8607bb9700 1 --2- 172.21.15.86:0/4129845701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8600103560 0x7f8600109e10 unknown :-1 s=CLOSED pgs=418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:23.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.660+0000 7f8607bb9700 1 --2- 172.21.15.86:0/4129845701 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86001019c0 0x7f8600101de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:23.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.660+0000 7f8607bb9700 1 --2- 172.21.15.86:0/4129845701 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8600102bc0 0x7f8600103020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:23.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.660+0000 7f8607bb9700 1 -- 172.21.15.86:0/4129845701 >> 172.21.15.86:0/4129845701 conn(0x7f86000fcf60 msgr2=0x7f86000ff3a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:23.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.661+0000 7f8607bb9700 1 -- 172.21.15.86:0/4129845701 shutdown_connections 2024-04-21T08:38:23.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.661+0000 7f8607bb9700 1 -- 172.21.15.86:0/4129845701 wait complete. 2024-04-21T08:38:23.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.661+0000 7f8607bb9700 1 Processor -- start 2024-04-21T08:38:23.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.661+0000 7f8607bb9700 1 -- start start 2024-04-21T08:38:23.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.661+0000 7f8607bb9700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86001019c0 0x7f860006bd10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:23.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.661+0000 7f8607bb9700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8600102bc0 0x7f860019d970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:23.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.661+0000 7f8607bb9700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8600103560 0x7f860019ffb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:23.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.661+0000 7f8607bb9700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f860010d4d0 con 0x7f8600102bc0 2024-04-21T08:38:23.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.661+0000 7f8607bb9700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f860010d350 con 0x7f8600103560 2024-04-21T08:38:23.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.661+0000 7f8607bb9700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f860010d650 con 0x7f86001019c0 2024-04-21T08:38:23.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.661+0000 7f8605154700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8600102bc0 0x7f860019d970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:23.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.661+0000 7f8605154700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8600102bc0 0x7f860019d970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54228/0 (socket says 172.21.15.86:54228) 2024-04-21T08:38:23.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.661+0000 7f8605154700 1 -- 172.21.15.86:0/543446280 learned_addr learned my addr 172.21.15.86:0/543446280 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:23.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.662+0000 7f8606156700 1 --2- 172.21.15.86:0/543446280 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8600103560 0x7f860019ffb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:23.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.662+0000 7f8605955700 1 --2- 172.21.15.86:0/543446280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86001019c0 0x7f860006bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:23.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.662+0000 7f8605154700 1 -- 172.21.15.86:0/543446280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86001019c0 msgr2=0x7f860006bd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:23.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.662+0000 7f8605154700 1 --2- 172.21.15.86:0/543446280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86001019c0 0x7f860006bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:23.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.662+0000 7f8605154700 1 -- 172.21.15.86:0/543446280 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8600103560 msgr2=0x7f860019ffb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:23.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.662+0000 7f8605154700 1 --2- 172.21.15.86:0/543446280 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8600103560 0x7f860019ffb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:23.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.662+0000 7f8605154700 1 -- 172.21.15.86:0/543446280 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f86001a0880 con 0x7f8600102bc0 2024-04-21T08:38:23.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.662+0000 7f8605154700 1 --2- 172.21.15.86:0/543446280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8600102bc0 0x7f860019d970 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f85f000d950 tx=0x7f85f000dc60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:23.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.662+0000 7f85f6ffd700 1 -- 172.21.15.86:0/543446280 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f0016070 con 0x7f8600102bc0 2024-04-21T08:38:23.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.662+0000 7f85f6ffd700 1 -- 172.21.15.86:0/543446280 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85f0009df0 con 0x7f8600102bc0 2024-04-21T08:38:23.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.662+0000 7f8607bb9700 1 -- 172.21.15.86:0/543446280 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f86001a0b70 con 0x7f8600102bc0 2024-04-21T08:38:23.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.662+0000 7f85f6ffd700 1 -- 172.21.15.86:0/543446280 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85f000f3c0 con 0x7f8600102bc0 2024-04-21T08:38:23.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.662+0000 7f8607bb9700 1 -- 172.21.15.86:0/543446280 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f86001a1100 con 0x7f8600102bc0 2024-04-21T08:38:23.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.663+0000 7f8607bb9700 1 -- 172.21.15.86:0/543446280 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8600069730 con 0x7f8600102bc0 2024-04-21T08:38:23.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.664+0000 7f85f6ffd700 1 -- 172.21.15.86:0/543446280 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f85f0019070 con 0x7f8600102bc0 2024-04-21T08:38:23.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.665+0000 7f85f6ffd700 1 --2- 172.21.15.86:0/543446280 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f85ec0a3cc0 0x7f85ec0a6180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:23.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.665+0000 7f8605955700 1 --2- 172.21.15.86:0/543446280 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f85ec0a3cc0 0x7f85ec0a6180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:23.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.665+0000 7f85f6ffd700 1 -- 172.21.15.86:0/543446280 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(92..92 src has 1..92) ==== 9150+0+0 (secure 0 0 0) 0x7f85f00d02c0 con 0x7f8600102bc0 2024-04-21T08:38:23.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.665+0000 7f8605955700 1 --2- 172.21.15.86:0/543446280 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f85ec0a3cc0 0x7f85ec0a6180 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f85fc006fd0 tx=0x7f85fc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:23.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.668+0000 7f85f6ffd700 1 -- 172.21.15.86:0/543446280 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f85f009c910 con 0x7f8600102bc0 2024-04-21T08:38:23.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:23 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-04-21T08:38:23.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:23 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]: dispatch 2024-04-21T08:38:23.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.876+0000 7f8607bb9700 1 -- 172.21.15.86:0/543446280 --> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] -- mgr_command(tid 0: {"prefix": "orch host ls", "target": ["mon-mgr", ""]}) -- 0x7f8600062e80 con 0x7f85ec0a3cc0 2024-04-21T08:38:23.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.877+0000 7f85f6ffd700 1 -- 172.21.15.86:0/543446280 <== mgr.14152 v2:172.21.15.86:6800/1364156965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+191 (secure 0 0 0) 0x7f8600062e80 con 0x7f85ec0a3cc0 2024-04-21T08:38:23.877 INFO:teuthology.orchestra.run.smithi086.stdout:HOST ADDR LABELS STATUS 2024-04-21T08:38:23.877 INFO:teuthology.orchestra.run.smithi086.stdout:smithi086 172.21.15.86 2024-04-21T08:38:23.877 INFO:teuthology.orchestra.run.smithi086.stdout:smithi088 172.21.15.88 2024-04-21T08:38:23.877 INFO:teuthology.orchestra.run.smithi086.stdout:smithi179 172.21.15.179 2024-04-21T08:38:23.877 INFO:teuthology.orchestra.run.smithi086.stdout:3 hosts in cluster 2024-04-21T08:38:23.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.879+0000 7f8607bb9700 1 -- 172.21.15.86:0/543446280 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f85ec0a3cc0 msgr2=0x7f85ec0a6180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:23.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.879+0000 7f8607bb9700 1 --2- 172.21.15.86:0/543446280 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f85ec0a3cc0 0x7f85ec0a6180 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f85fc006fd0 tx=0x7f85fc008040 comp rx=0 tx=0).stop 2024-04-21T08:38:23.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.879+0000 7f8607bb9700 1 -- 172.21.15.86:0/543446280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8600102bc0 msgr2=0x7f860019d970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:23.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.879+0000 7f8607bb9700 1 --2- 172.21.15.86:0/543446280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8600102bc0 0x7f860019d970 secure :-1 s=READY pgs=419 cs=0 l=1 rev1=1 crypto rx=0x7f85f000d950 tx=0x7f85f000dc60 comp rx=0 tx=0).stop 2024-04-21T08:38:23.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.880+0000 7f8607bb9700 1 -- 172.21.15.86:0/543446280 shutdown_connections 2024-04-21T08:38:23.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.880+0000 7f8607bb9700 1 --2- 172.21.15.86:0/543446280 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f85ec0a3cc0 0x7f85ec0a6180 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:23.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.880+0000 7f8607bb9700 1 --2- 172.21.15.86:0/543446280 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8600103560 0x7f860019ffb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:23.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.880+0000 7f8607bb9700 1 --2- 172.21.15.86:0/543446280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f86001019c0 0x7f860006bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:23.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.880+0000 7f8607bb9700 1 --2- 172.21.15.86:0/543446280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8600102bc0 0x7f860019d970 unknown :-1 s=CLOSED pgs=419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:23.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.880+0000 7f8607bb9700 1 -- 172.21.15.86:0/543446280 >> 172.21.15.86:0/543446280 conn(0x7f86000fcf60 msgr2=0x7f8600105f40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:23.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.880+0000 7f8607bb9700 1 -- 172.21.15.86:0/543446280 shutdown_connections 2024-04-21T08:38:23.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:23.880+0000 7f8607bb9700 1 -- 172.21.15.86:0/543446280 wait complete. 2024-04-21T08:38:23.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:23 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]: dispatch 2024-04-21T08:38:23.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:23 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd=[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]: dispatch 2024-04-21T08:38:24.562 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:24 smithi086 ceph-mon[27939]: pgmap v351: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:38:24.562 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:24 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-04-21T08:38:24.562 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:24 smithi086 ceph-mon[27939]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]': finished 2024-04-21T08:38:24.562 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:24 smithi086 ceph-mon[27939]: osdmap e92: 12 total, 12 up, 12 in 2024-04-21T08:38:24.591 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- bash -c 'ceph orch device ls' 2024-04-21T08:38:24.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:24 smithi179 ceph-mon[31829]: pgmap v351: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:38:24.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:24 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-04-21T08:38:24.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:24 smithi179 ceph-mon[31829]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]': finished 2024-04-21T08:38:24.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:24 smithi179 ceph-mon[31829]: osdmap e92: 12 total, 12 up, 12 in 2024-04-21T08:38:24.832 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:38:24.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:24 smithi088 ceph-mon[31259]: pgmap v351: 129 pgs: 129 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 9 op/s 2024-04-21T08:38:24.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:24 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.14", "id": [9, 8]}]': finished 2024-04-21T08:38:24.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:24 smithi088 ceph-mon[31259]: from='mgr.14152 172.21.15.86:0/3964553679' entity='mgr.x' cmd='[{"prefix": "osd pg-upmap-items", "format": "json", "pgid": "4.1a", "id": [5, 8]}]': finished 2024-04-21T08:38:24.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:24 smithi088 ceph-mon[31259]: osdmap e92: 12 total, 12 up, 12 in 2024-04-21T08:38:25.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:25 smithi179 ceph-mon[31829]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:25.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:25 smithi179 ceph-mon[31829]: osdmap e93: 12 total, 12 up, 12 in 2024-04-21T08:38:25.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:25 smithi179 ceph-mon[31829]: pgmap v354: 129 pgs: 1 peering, 128 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-21T08:38:25.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:25 smithi086 ceph-mon[27939]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:25.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:25 smithi086 ceph-mon[27939]: osdmap e93: 12 total, 12 up, 12 in 2024-04-21T08:38:25.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:25 smithi086 ceph-mon[27939]: pgmap v354: 129 pgs: 1 peering, 128 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-21T08:38:25.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:25 smithi088 ceph-mon[31259]: from='client.15348 -' entity='client.admin' cmd=[{"prefix": "orch host ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:25.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:25 smithi088 ceph-mon[31259]: osdmap e93: 12 total, 12 up, 12 in 2024-04-21T08:38:25.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:25 smithi088 ceph-mon[31259]: pgmap v354: 129 pgs: 1 peering, 128 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-21T08:38:26.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.554+0000 7f04361be700 1 -- 172.21.15.86:0/2413090061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0430103cb0 msgr2=0x7f0430104110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:26.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.554+0000 7f04361be700 1 --2- 172.21.15.86:0/2413090061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0430103cb0 0x7f0430104110 secure :-1 s=READY pgs=420 cs=0 l=1 rev1=1 crypto rx=0x7f04180099c0 tx=0x7f0418009cd0 comp rx=0 tx=0).stop 2024-04-21T08:38:26.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.554+0000 7f04361be700 1 -- 172.21.15.86:0/2413090061 shutdown_connections 2024-04-21T08:38:26.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.554+0000 7f04361be700 1 --2- 172.21.15.86:0/2413090061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0430104650 0x7f043010af00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:26.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.554+0000 7f04361be700 1 --2- 172.21.15.86:0/2413090061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0430102ab0 0x7f0430102ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:26.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.554+0000 7f04361be700 1 --2- 172.21.15.86:0/2413090061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0430103cb0 0x7f0430104110 unknown :-1 s=CLOSED pgs=420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:26.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.554+0000 7f04361be700 1 -- 172.21.15.86:0/2413090061 >> 172.21.15.86:0/2413090061 conn(0x7f04300fe050 msgr2=0x7f0430100490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:26.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.554+0000 7f04361be700 1 -- 172.21.15.86:0/2413090061 shutdown_connections 2024-04-21T08:38:26.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.554+0000 7f04361be700 1 -- 172.21.15.86:0/2413090061 wait complete. 2024-04-21T08:38:26.555 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:26 smithi086 ceph-mon[27939]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-21T08:38:26.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.555+0000 7f04361be700 1 Processor -- start 2024-04-21T08:38:26.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.555+0000 7f04361be700 1 -- start start 2024-04-21T08:38:26.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.555+0000 7f04361be700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0430102ab0 0x7f0430198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:26.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.555+0000 7f04361be700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0430103cb0 0x7f04301991f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:26.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.555+0000 7f04361be700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0430104650 0x7f04301a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:26.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.555+0000 7f04361be700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f043010e640 con 0x7f0430103cb0 2024-04-21T08:38:26.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.555+0000 7f04361be700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f043010e4c0 con 0x7f0430104650 2024-04-21T08:38:26.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.555+0000 7f04361be700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f043010e7c0 con 0x7f0430102ab0 2024-04-21T08:38:26.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.555+0000 7f0426ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0430103cb0 0x7f04301991f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:26.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.555+0000 7f0426ffd700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0430103cb0 0x7f04301991f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54248/0 (socket says 172.21.15.86:54248) 2024-04-21T08:38:26.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.555+0000 7f0426ffd700 1 -- 172.21.15.86:0/3636672414 learned_addr learned my addr 172.21.15.86:0/3636672414 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:26.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.555+0000 7f042f7fe700 1 --2- 172.21.15.86:0/3636672414 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0430102ab0 0x7f0430198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:26.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.555+0000 7f042ffff700 1 --2- 172.21.15.86:0/3636672414 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0430104650 0x7f04301a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:26.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.556+0000 7f0426ffd700 1 -- 172.21.15.86:0/3636672414 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0430102ab0 msgr2=0x7f0430198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:26.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.556+0000 7f0426ffd700 1 --2- 172.21.15.86:0/3636672414 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0430102ab0 0x7f0430198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:26.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.556+0000 7f0426ffd700 1 -- 172.21.15.86:0/3636672414 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0430104650 msgr2=0x7f04301a0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:26.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.556+0000 7f0426ffd700 1 --2- 172.21.15.86:0/3636672414 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0430104650 0x7f04301a0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:26.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.556+0000 7f0426ffd700 1 -- 172.21.15.86:0/3636672414 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f04301a0930 con 0x7f0430103cb0 2024-04-21T08:38:26.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.556+0000 7f0426ffd700 1 --2- 172.21.15.86:0/3636672414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0430103cb0 0x7f04301991f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0418003790 tx=0x7f0418005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:26.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.556+0000 7f042d7fa700 1 -- 172.21.15.86:0/3636672414 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0418027070 con 0x7f0430103cb0 2024-04-21T08:38:26.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.556+0000 7f042d7fa700 1 -- 172.21.15.86:0/3636672414 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0418022470 con 0x7f0430103cb0 2024-04-21T08:38:26.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.556+0000 7f042d7fa700 1 -- 172.21.15.86:0/3636672414 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f041800f6e0 con 0x7f0430103cb0 2024-04-21T08:38:26.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.556+0000 7f04361be700 1 -- 172.21.15.86:0/3636672414 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f04301a0bc0 con 0x7f0430103cb0 2024-04-21T08:38:26.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.556+0000 7f04361be700 1 -- 172.21.15.86:0/3636672414 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f04301a10a0 con 0x7f0430103cb0 2024-04-21T08:38:26.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.557+0000 7f04361be700 1 -- 172.21.15.86:0/3636672414 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0430069730 con 0x7f0430103cb0 2024-04-21T08:38:26.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.558+0000 7f042d7fa700 1 -- 172.21.15.86:0/3636672414 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f041802a070 con 0x7f0430103cb0 2024-04-21T08:38:26.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.559+0000 7f042d7fa700 1 --2- 172.21.15.86:0/3636672414 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f041c0a3c40 0x7f041c0a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:26.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.559+0000 7f042f7fe700 1 --2- 172.21.15.86:0/3636672414 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f041c0a3c40 0x7f041c0a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:26.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.559+0000 7f042d7fa700 1 -- 172.21.15.86:0/3636672414 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f04180d2090 con 0x7f0430103cb0 2024-04-21T08:38:26.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.559+0000 7f042f7fe700 1 --2- 172.21.15.86:0/3636672414 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f041c0a3c40 0x7f041c0a6100 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0430199b90 tx=0x7f0420009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:26.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.563+0000 7f042d7fa700 1 -- 172.21.15.86:0/3636672414 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0418031030 con 0x7f0430103cb0 2024-04-21T08:38:26.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.770+0000 7f04361be700 1 -- 172.21.15.86:0/3636672414 --> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] -- mgr_command(tid 0: {"prefix": "orch device ls", "target": ["mon-mgr", ""]}) -- 0x7f0430062e80 con 0x7f041c0a3c40 2024-04-21T08:38:26.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.772+0000 7f042d7fa700 1 -- 172.21.15.86:0/3636672414 <== mgr.14152 v2:172.21.15.86:6800/1364156965 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+532 (secure 0 0 0) 0x7f0430062e80 con 0x7f041c0a3c40 2024-04-21T08:38:26.771 INFO:teuthology.orchestra.run.smithi086.stdout:HOST PATH TYPE DEVICE ID SIZE AVAILABLE REFRESHED REJECT REASONS 2024-04-21T08:38:26.771 INFO:teuthology.orchestra.run.smithi086.stdout:smithi086 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400UL400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-21T08:38:26.771 INFO:teuthology.orchestra.run.smithi086.stdout:smithi088 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_PHFT620400XD400BGN 372G No 3m ago Has a FileSystem, LVM detected 2024-04-21T08:38:26.772 INFO:teuthology.orchestra.run.smithi086.stdout:smithi179 /dev/nvme0n1 ssd INTEL_SSDPEDMD400G4_CVFT6303009J400BGN 372G No 2m ago Has a FileSystem, LVM detected 2024-04-21T08:38:26.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.774+0000 7f04361be700 1 -- 172.21.15.86:0/3636672414 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f041c0a3c40 msgr2=0x7f041c0a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:26.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.774+0000 7f04361be700 1 --2- 172.21.15.86:0/3636672414 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f041c0a3c40 0x7f041c0a6100 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f0430199b90 tx=0x7f0420009190 comp rx=0 tx=0).stop 2024-04-21T08:38:26.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.774+0000 7f04361be700 1 -- 172.21.15.86:0/3636672414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0430103cb0 msgr2=0x7f04301991f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:26.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.774+0000 7f04361be700 1 --2- 172.21.15.86:0/3636672414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0430103cb0 0x7f04301991f0 secure :-1 s=READY pgs=421 cs=0 l=1 rev1=1 crypto rx=0x7f0418003790 tx=0x7f0418005ef0 comp rx=0 tx=0).stop 2024-04-21T08:38:26.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.775+0000 7f04361be700 1 -- 172.21.15.86:0/3636672414 shutdown_connections 2024-04-21T08:38:26.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.775+0000 7f04361be700 1 --2- 172.21.15.86:0/3636672414 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f041c0a3c40 0x7f041c0a6100 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:26.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.775+0000 7f04361be700 1 --2- 172.21.15.86:0/3636672414 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0430104650 0x7f04301a0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:26.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.775+0000 7f04361be700 1 --2- 172.21.15.86:0/3636672414 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0430102ab0 0x7f0430198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:26.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.775+0000 7f04361be700 1 --2- 172.21.15.86:0/3636672414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0430103cb0 0x7f04301991f0 unknown :-1 s=CLOSED pgs=421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:26.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.775+0000 7f04361be700 1 -- 172.21.15.86:0/3636672414 >> 172.21.15.86:0/3636672414 conn(0x7f04300fe050 msgr2=0x7f0430106fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:26.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.775+0000 7f04361be700 1 -- 172.21.15.86:0/3636672414 shutdown_connections 2024-04-21T08:38:26.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:26.775+0000 7f04361be700 1 -- 172.21.15.86:0/3636672414 wait complete. 2024-04-21T08:38:26.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:26 smithi088 ceph-mon[31259]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-21T08:38:26.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:26 smithi179 ceph-mon[31829]: Health check failed: Reduced data availability: 1 pg peering (PG_AVAILABILITY) 2024-04-21T08:38:27.479 INFO:teuthology.run_tasks:Running task cephadm.shell... 2024-04-21T08:38:27.492 INFO:tasks.cephadm:Running commands on role mon.a host ubuntu@smithi086.front.sepia.ceph.com 2024-04-21T08:38:27.492 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- bash -c 'ceph fs dump' 2024-04-21T08:38:27.730 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:38:27.852 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:27 smithi086 ceph-mon[27939]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:27.852 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:27 smithi086 ceph-mon[27939]: pgmap v355: 129 pgs: 1 peering, 128 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-04-21T08:38:27.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:27 smithi088 ceph-mon[31259]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:27.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:27 smithi088 ceph-mon[31259]: pgmap v355: 129 pgs: 1 peering, 128 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-04-21T08:38:27.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:27 smithi179 ceph-mon[31829]: from='client.15354 -' entity='client.admin' cmd=[{"prefix": "orch device ls", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:27.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:27 smithi179 ceph-mon[31829]: pgmap v355: 129 pgs: 1 peering, 128 active+clean; 647 KiB data, 375 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-04-21T08:38:29.515 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:29 smithi086 ceph-mon[27939]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-21T08:38:29.515 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:29 smithi086 ceph-mon[27939]: Cluster is now healthy 2024-04-21T08:38:29.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.522+0000 7f7850f96700 1 -- 172.21.15.86:0/1440431692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f784c1019a0 msgr2=0x7f784c101dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:29.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.522+0000 7f7850f96700 1 --2- 172.21.15.86:0/1440431692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f784c1019a0 0x7f784c101dc0 secure :-1 s=READY pgs=422 cs=0 l=1 rev1=1 crypto rx=0x7f783c0099c0 tx=0x7f783c009cd0 comp rx=0 tx=0).stop 2024-04-21T08:38:29.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.522+0000 7f7850f96700 1 -- 172.21.15.86:0/1440431692 shutdown_connections 2024-04-21T08:38:29.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.522+0000 7f7850f96700 1 --2- 172.21.15.86:0/1440431692 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f784c103540 0x7f784c109df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:29.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.522+0000 7f7850f96700 1 --2- 172.21.15.86:0/1440431692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f784c1019a0 0x7f784c101dc0 unknown :-1 s=CLOSED pgs=422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:29.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.522+0000 7f7850f96700 1 --2- 172.21.15.86:0/1440431692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f784c102ba0 0x7f784c103000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:29.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.522+0000 7f7850f96700 1 -- 172.21.15.86:0/1440431692 >> 172.21.15.86:0/1440431692 conn(0x7f784c0fcf40 msgr2=0x7f784c0ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:29.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.522+0000 7f7850f96700 1 -- 172.21.15.86:0/1440431692 shutdown_connections 2024-04-21T08:38:29.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.522+0000 7f7850f96700 1 -- 172.21.15.86:0/1440431692 wait complete. 2024-04-21T08:38:29.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.523+0000 7f7850f96700 1 Processor -- start 2024-04-21T08:38:29.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.523+0000 7f7850f96700 1 -- start start 2024-04-21T08:38:29.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.523+0000 7f7850f96700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f784c1019a0 0x7f784c105b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:29.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.523+0000 7f7850f96700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f784c102ba0 0x7f784c1041e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:29.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.523+0000 7f7850f96700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f784c103540 0x7f784c104720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:29.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.523+0000 7f7850f96700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f784c10d510 con 0x7f784c102ba0 2024-04-21T08:38:29.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.523+0000 7f7850f96700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f784c10d390 con 0x7f784c1019a0 2024-04-21T08:38:29.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.523+0000 7f7850f96700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f784c10d690 con 0x7f784c103540 2024-04-21T08:38:29.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.523+0000 7f7849d9b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f784c102ba0 0x7f784c1041e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:29.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.523+0000 7f7849d9b700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f784c102ba0 0x7f784c1041e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60024/0 (socket says 172.21.15.86:60024) 2024-04-21T08:38:29.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.523+0000 7f7849d9b700 1 -- 172.21.15.86:0/2528755721 learned_addr learned my addr 172.21.15.86:0/2528755721 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:29.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.523+0000 7f784a59c700 1 --2- 172.21.15.86:0/2528755721 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f784c1019a0 0x7f784c105b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:29.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.523+0000 7f784ad9d700 1 --2- 172.21.15.86:0/2528755721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f784c103540 0x7f784c104720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:29.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.523+0000 7f7849d9b700 1 -- 172.21.15.86:0/2528755721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f784c103540 msgr2=0x7f784c104720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:29.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.523+0000 7f7849d9b700 1 --2- 172.21.15.86:0/2528755721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f784c103540 0x7f784c104720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:29.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.523+0000 7f7849d9b700 1 -- 172.21.15.86:0/2528755721 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f784c1019a0 msgr2=0x7f784c105b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:29.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.523+0000 7f7849d9b700 1 --2- 172.21.15.86:0/2528755721 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f784c1019a0 0x7f784c105b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:29.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.524+0000 7f7849d9b700 1 -- 172.21.15.86:0/2528755721 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f784c104ff0 con 0x7f784c102ba0 2024-04-21T08:38:29.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.524+0000 7f784ad9d700 1 --2- 172.21.15.86:0/2528755721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f784c103540 0x7f784c104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:29.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.524+0000 7f7849d9b700 1 --2- 172.21.15.86:0/2528755721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f784c102ba0 0x7f784c1041e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f784000b530 tx=0x7f784000b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:29.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.524+0000 7f783b7fe700 1 -- 172.21.15.86:0/2528755721 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7840015020 con 0x7f784c102ba0 2024-04-21T08:38:29.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.524+0000 7f7850f96700 1 -- 172.21.15.86:0/2528755721 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f784c1a2f10 con 0x7f784c102ba0 2024-04-21T08:38:29.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.524+0000 7f783b7fe700 1 -- 172.21.15.86:0/2528755721 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7840011ce0 con 0x7f784c102ba0 2024-04-21T08:38:29.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.524+0000 7f783b7fe700 1 -- 172.21.15.86:0/2528755721 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7840010e40 con 0x7f784c102ba0 2024-04-21T08:38:29.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.524+0000 7f7850f96700 1 -- 172.21.15.86:0/2528755721 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f784c1a33e0 con 0x7f784c102ba0 2024-04-21T08:38:29.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.525+0000 7f7850f96700 1 -- 172.21.15.86:0/2528755721 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f784c1a3670 con 0x7f784c102ba0 2024-04-21T08:38:29.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.526+0000 7f783b7fe700 1 -- 172.21.15.86:0/2528755721 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7840018020 con 0x7f784c102ba0 2024-04-21T08:38:29.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.527+0000 7f783b7fe700 1 --2- 172.21.15.86:0/2528755721 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f78340a3c40 0x7f78340a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:29.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.527+0000 7f784a59c700 1 --2- 172.21.15.86:0/2528755721 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f78340a3c40 0x7f78340a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:29.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.527+0000 7f783b7fe700 1 -- 172.21.15.86:0/2528755721 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f78400d05e0 con 0x7f784c102ba0 2024-04-21T08:38:29.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.527+0000 7f784a59c700 1 --2- 172.21.15.86:0/2528755721 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f78340a3c40 0x7f78340a6100 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f783c006010 tx=0x7f783c003960 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:29.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.531+0000 7f783b7fe700 1 -- 172.21.15.86:0/2528755721 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f784001f020 con 0x7f784c102ba0 2024-04-21T08:38:29.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:29 smithi088 ceph-mon[31259]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-21T08:38:29.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:29 smithi088 ceph-mon[31259]: Cluster is now healthy 2024-04-21T08:38:29.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:29 smithi179 ceph-mon[31829]: Health check cleared: PG_AVAILABILITY (was: Reduced data availability: 1 pg peering) 2024-04-21T08:38:29.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:29 smithi179 ceph-mon[31829]: Cluster is now healthy 2024-04-21T08:38:29.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.774+0000 7f7850f96700 1 -- 172.21.15.86:0/2528755721 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump"} v 0) -- 0x7f784c1a3840 con 0x7f784c102ba0 2024-04-21T08:38:29.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.775+0000 7f783b7fe700 1 -- 172.21.15.86:0/2528755721 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump"}]=0 dumped fsmap epoch 28 v28) ==== 76+0+3094 (secure 0 0 0) 0x7f78400cfc30 con 0x7f784c102ba0 2024-04-21T08:38:29.774 INFO:teuthology.orchestra.run.smithi086.stdout:e28 2024-04-21T08:38:29.774 INFO:teuthology.orchestra.run.smithi086.stdout:enable_multiple, ever_enabled_multiple: 1,1 2024-04-21T08:38:29.775 INFO:teuthology.orchestra.run.smithi086.stdout:default compat: compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-21T08:38:29.775 INFO:teuthology.orchestra.run.smithi086.stdout:legacy client fscid: 1 2024-04-21T08:38:29.775 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:38:29.775 INFO:teuthology.orchestra.run.smithi086.stdout:Filesystem 'cephfs' (1) 2024-04-21T08:38:29.775 INFO:teuthology.orchestra.run.smithi086.stdout:fs_name cephfs 2024-04-21T08:38:29.775 INFO:teuthology.orchestra.run.smithi086.stdout:epoch 28 2024-04-21T08:38:29.775 INFO:teuthology.orchestra.run.smithi086.stdout:flags 32 joinable allow_snaps allow_multimds_snaps allow_standby_replay 2024-04-21T08:38:29.775 INFO:teuthology.orchestra.run.smithi086.stdout:created 2024-04-21T08:36:20.626894+0000 2024-04-21T08:38:29.775 INFO:teuthology.orchestra.run.smithi086.stdout:modified 2024-04-21T08:36:47.458637+0000 2024-04-21T08:38:29.775 INFO:teuthology.orchestra.run.smithi086.stdout:tableserver 0 2024-04-21T08:38:29.775 INFO:teuthology.orchestra.run.smithi086.stdout:root 0 2024-04-21T08:38:29.775 INFO:teuthology.orchestra.run.smithi086.stdout:session_timeout 300 2024-04-21T08:38:29.775 INFO:teuthology.orchestra.run.smithi086.stdout:session_autoclose 300 2024-04-21T08:38:29.775 INFO:teuthology.orchestra.run.smithi086.stdout:max_file_size 1099511627776 2024-04-21T08:38:29.776 INFO:teuthology.orchestra.run.smithi086.stdout:max_xattr_size 65536 2024-04-21T08:38:29.776 INFO:teuthology.orchestra.run.smithi086.stdout:required_client_features {} 2024-04-21T08:38:29.776 INFO:teuthology.orchestra.run.smithi086.stdout:last_failure 0 2024-04-21T08:38:29.776 INFO:teuthology.orchestra.run.smithi086.stdout:last_failure_osd_epoch 0 2024-04-21T08:38:29.776 INFO:teuthology.orchestra.run.smithi086.stdout:compat compat={},rocompat={},incompat={1=base v0.20,2=client writeable ranges,3=default file layouts on dirs,4=dir inode in separate object,5=mds uses versioned encoding,6=dirfrag is stored in omap,7=mds uses inline data,8=no anchor table,9=file layout v2,10=snaprealm v2,11=minor log segments,12=quiesce subvolumes} 2024-04-21T08:38:29.776 INFO:teuthology.orchestra.run.smithi086.stdout:max_mds 5 2024-04-21T08:38:29.776 INFO:teuthology.orchestra.run.smithi086.stdout:in 0,1,2,3,4 2024-04-21T08:38:29.776 INFO:teuthology.orchestra.run.smithi086.stdout:up {0=24482,1=24472,2=24455,3=24398,4=24394} 2024-04-21T08:38:29.776 INFO:teuthology.orchestra.run.smithi086.stdout:failed 2024-04-21T08:38:29.776 INFO:teuthology.orchestra.run.smithi086.stdout:damaged 2024-04-21T08:38:29.776 INFO:teuthology.orchestra.run.smithi086.stdout:stopped 2024-04-21T08:38:29.776 INFO:teuthology.orchestra.run.smithi086.stdout:data_pools [3,4] 2024-04-21T08:38:29.776 INFO:teuthology.orchestra.run.smithi086.stdout:metadata_pool 2 2024-04-21T08:38:29.776 INFO:teuthology.orchestra.run.smithi086.stdout:inline_data disabled 2024-04-21T08:38:29.777 INFO:teuthology.orchestra.run.smithi086.stdout:balancer 2024-04-21T08:38:29.777 INFO:teuthology.orchestra.run.smithi086.stdout:bal_rank_mask -1 2024-04-21T08:38:29.777 INFO:teuthology.orchestra.run.smithi086.stdout:standby_count_wanted 0 2024-04-21T08:38:29.777 INFO:teuthology.orchestra.run.smithi086.stdout:qdb_cluster leader: 24482 members: 24455,24472,24394,24398,24482 2024-04-21T08:38:29.777 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.b{0:24482} state up:active seq 3 addr [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T08:38:29.777 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.c{0:24448} state up:standby-replay seq 1 addr [v2:172.21.15.179:6836/1280529894,v1:172.21.15.179:6837/1280529894] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T08:38:29.777 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.i{1:24472} state up:active seq 11 addr [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T08:38:29.777 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.f{1:24436} state up:standby-replay seq 1 addr [v2:172.21.15.179:6834/3343050063,v1:172.21.15.179:6835/3343050063] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T08:38:29.777 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.h{2:24455} state up:active seq 12 addr [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T08:38:29.777 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.e{2:24428} state up:standby-replay seq 1 addr [v2:172.21.15.88:6834/454792798,v1:172.21.15.88:6835/454792798] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T08:38:29.777 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.k{3:24398} state up:active seq 16 addr [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T08:38:29.777 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.a{3:14652} state up:standby-replay seq 1 addr [v2:172.21.15.86:6840/3537972023,v1:172.21.15.86:6841/3537972023] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T08:38:29.777 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.l{4:24394} state up:active seq 17 addr [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T08:38:29.777 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.g{4:14640} state up:standby-replay seq 1 addr [v2:172.21.15.86:6838/3163123023,v1:172.21.15.86:6839/3163123023] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T08:38:29.777 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:38:29.777 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:38:29.778 INFO:teuthology.orchestra.run.smithi086.stdout:Standby daemons: 2024-04-21T08:38:29.778 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:38:29.778 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.j{-1:14571} state up:standby seq 1 addr [v2:172.21.15.86:6834/4288306918,v1:172.21.15.86:6835/4288306918] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T08:38:29.778 INFO:teuthology.orchestra.run.smithi086.stdout:[mds.d{-1:14583} state up:standby seq 1 addr [v2:172.21.15.86:6836/1247248764,v1:172.21.15.86:6837/1247248764] compat {c=[1],r=[1],i=[1fff]}] 2024-04-21T08:38:29.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.777+0000 7f7850f96700 1 -- 172.21.15.86:0/2528755721 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f78340a3c40 msgr2=0x7f78340a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:29.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.777+0000 7f7850f96700 1 --2- 172.21.15.86:0/2528755721 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f78340a3c40 0x7f78340a6100 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7f783c006010 tx=0x7f783c003960 comp rx=0 tx=0).stop 2024-04-21T08:38:29.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.777+0000 7f7850f96700 1 -- 172.21.15.86:0/2528755721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f784c102ba0 msgr2=0x7f784c1041e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:29.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.777+0000 7f7850f96700 1 --2- 172.21.15.86:0/2528755721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f784c102ba0 0x7f784c1041e0 secure :-1 s=READY pgs=423 cs=0 l=1 rev1=1 crypto rx=0x7f784000b530 tx=0x7f784000b840 comp rx=0 tx=0).stop 2024-04-21T08:38:29.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.778+0000 7f7850f96700 1 -- 172.21.15.86:0/2528755721 shutdown_connections 2024-04-21T08:38:29.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.778+0000 7f7850f96700 1 --2- 172.21.15.86:0/2528755721 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f78340a3c40 0x7f78340a6100 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:29.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.778+0000 7f7850f96700 1 --2- 172.21.15.86:0/2528755721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f784c103540 0x7f784c104720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:29.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.778+0000 7f7850f96700 1 --2- 172.21.15.86:0/2528755721 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f784c1019a0 0x7f784c105b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:29.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.778+0000 7f7850f96700 1 --2- 172.21.15.86:0/2528755721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f784c102ba0 0x7f784c1041e0 unknown :-1 s=CLOSED pgs=423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:29.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.778+0000 7f7850f96700 1 -- 172.21.15.86:0/2528755721 >> 172.21.15.86:0/2528755721 conn(0x7f784c0fcf40 msgr2=0x7f784c06a720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:29.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.778+0000 7f7850f96700 1 -- 172.21.15.86:0/2528755721 shutdown_connections 2024-04-21T08:38:29.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:29.778+0000 7f7850f96700 1 -- 172.21.15.86:0/2528755721 wait complete. 2024-04-21T08:38:29.779 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-21T08:38:30.470 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:30 smithi086 ceph-mon[27939]: pgmap v356: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-21T08:38:30.470 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:30 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2528755721' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-21T08:38:30.498 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell -c /etc/ceph/ceph.conf -k /etc/ceph/ceph.client.admin.keyring --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- bash -c 'ceph osd dump' 2024-04-21T08:38:30.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:30 smithi088 ceph-mon[31259]: pgmap v356: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-21T08:38:30.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:30 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2528755721' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-21T08:38:30.737 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:38:30.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:30 smithi179 ceph-mon[31829]: pgmap v356: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 3.7 KiB/s rd, 8 op/s 2024-04-21T08:38:30.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:30 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2528755721' entity='client.admin' cmd=[{"prefix": "fs dump"}]: dispatch 2024-04-21T08:38:32.532 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:32 smithi086 ceph-mon[27939]: pgmap v357: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-04-21T08:38:32.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.536+0000 7f5c10966700 1 -- 172.21.15.86:0/741445521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c0c102b90 msgr2=0x7f5c0c102ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:32.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.536+0000 7f5c10966700 1 --2- 172.21.15.86:0/741445521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c0c102b90 0x7f5c0c102ff0 secure :-1 s=READY pgs=424 cs=0 l=1 rev1=1 crypto rx=0x7f5c000099c0 tx=0x7f5c00009cd0 comp rx=0 tx=0).stop 2024-04-21T08:38:32.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.536+0000 7f5c10966700 1 -- 172.21.15.86:0/741445521 shutdown_connections 2024-04-21T08:38:32.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.536+0000 7f5c10966700 1 --2- 172.21.15.86:0/741445521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c0c103530 0x7f5c0c109de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:32.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.536+0000 7f5c10966700 1 --2- 172.21.15.86:0/741445521 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c0c101990 0x7f5c0c101db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:32.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.536+0000 7f5c10966700 1 --2- 172.21.15.86:0/741445521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c0c102b90 0x7f5c0c102ff0 unknown :-1 s=CLOSED pgs=424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:32.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.536+0000 7f5c10966700 1 -- 172.21.15.86:0/741445521 >> 172.21.15.86:0/741445521 conn(0x7f5c0c0fcf30 msgr2=0x7f5c0c0ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:32.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.536+0000 7f5c10966700 1 -- 172.21.15.86:0/741445521 shutdown_connections 2024-04-21T08:38:32.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.536+0000 7f5c10966700 1 -- 172.21.15.86:0/741445521 wait complete. 2024-04-21T08:38:32.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.537+0000 7f5c10966700 1 Processor -- start 2024-04-21T08:38:32.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.537+0000 7f5c10966700 1 -- start start 2024-04-21T08:38:32.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.537+0000 7f5c10966700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c0c101990 0x7f5c0c198ce0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:32.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.537+0000 7f5c10966700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c0c102b90 0x7f5c0c199220 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:32.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.537+0000 7f5c10966700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c0c103530 0x7f5c0c1a0260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.537+0000 7f5c10966700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0c10d520 con 0x7f5c0c103530 2024-04-21T08:38:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.537+0000 7f5c10966700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0c10d3a0 con 0x7f5c0c102b90 2024-04-21T08:38:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.537+0000 7f5c10966700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5c0c10d6a0 con 0x7f5c0c101990 2024-04-21T08:38:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.538+0000 7f5c0ad9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c0c103530 0x7f5c0c1a0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.538+0000 7f5c0ad9d700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c0c103530 0x7f5c0c1a0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60034/0 (socket says 172.21.15.86:60034) 2024-04-21T08:38:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.538+0000 7f5c0ad9d700 1 -- 172.21.15.86:0/2114887812 learned_addr learned my addr 172.21.15.86:0/2114887812 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.538+0000 7f5c09d9b700 1 --2- 172.21.15.86:0/2114887812 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c0c102b90 0x7f5c0c199220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.538+0000 7f5c0a59c700 1 --2- 172.21.15.86:0/2114887812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c0c101990 0x7f5c0c198ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.538+0000 7f5c0ad9d700 1 -- 172.21.15.86:0/2114887812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c0c101990 msgr2=0x7f5c0c198ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.538+0000 7f5c0ad9d700 1 --2- 172.21.15.86:0/2114887812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c0c101990 0x7f5c0c198ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.538+0000 7f5c0ad9d700 1 -- 172.21.15.86:0/2114887812 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c0c102b90 msgr2=0x7f5c0c199220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.538+0000 7f5c0ad9d700 1 --2- 172.21.15.86:0/2114887812 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c0c102b90 0x7f5c0c199220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:32.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.538+0000 7f5c0ad9d700 1 -- 172.21.15.86:0/2114887812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c0c1a0930 con 0x7f5c0c103530 2024-04-21T08:38:32.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.538+0000 7f5c0ad9d700 1 --2- 172.21.15.86:0/2114887812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c0c103530 0x7f5c0c1a0260 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5c0400b970 tx=0x7f5c0400bc80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:32.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.538+0000 7f5bfb7fe700 1 -- 172.21.15.86:0/2114887812 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c0400d860 con 0x7f5c0c103530 2024-04-21T08:38:32.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.538+0000 7f5c10966700 1 -- 172.21.15.86:0/2114887812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c0c1a0c20 con 0x7f5c0c103530 2024-04-21T08:38:32.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.538+0000 7f5bfb7fe700 1 -- 172.21.15.86:0/2114887812 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c0400e430 con 0x7f5c0c103530 2024-04-21T08:38:32.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.538+0000 7f5bfb7fe700 1 -- 172.21.15.86:0/2114887812 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c04017660 con 0x7f5c0c103530 2024-04-21T08:38:32.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.538+0000 7f5c10966700 1 -- 172.21.15.86:0/2114887812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c0c1a1160 con 0x7f5c0c103530 2024-04-21T08:38:32.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.539+0000 7f5c10966700 1 -- 172.21.15.86:0/2114887812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5c0c069760 con 0x7f5c0c103530 2024-04-21T08:38:32.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.540+0000 7f5bfb7fe700 1 -- 172.21.15.86:0/2114887812 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f5c0400da00 con 0x7f5c0c103530 2024-04-21T08:38:32.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.541+0000 7f5bfb7fe700 1 --2- 172.21.15.86:0/2114887812 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f5bf40a3c40 0x7f5bf40a6100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:32.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.541+0000 7f5c0a59c700 1 --2- 172.21.15.86:0/2114887812 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f5bf40a3c40 0x7f5bf40a6100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:32.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.541+0000 7f5bfb7fe700 1 -- 172.21.15.86:0/2114887812 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f5c040d0730 con 0x7f5c0c103530 2024-04-21T08:38:32.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.541+0000 7f5c0a59c700 1 --2- 172.21.15.86:0/2114887812 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f5bf40a3c40 0x7f5bf40a6100 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c19a1c0 tx=0x7f5bfc009190 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:32.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.545+0000 7f5bfb7fe700 1 -- 172.21.15.86:0/2114887812 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5c0c069760 con 0x7f5c0c103530 2024-04-21T08:38:32.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:32 smithi088 ceph-mon[31259]: pgmap v357: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-04-21T08:38:32.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.743+0000 7f5c10966700 1 -- 172.21.15.86:0/2114887812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump"} v 0) -- 0x7f5c0c1a13e0 con 0x7f5c0c103530 2024-04-21T08:38:32.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.744+0000 7f5bfb7fe700 1 -- 172.21.15.86:0/2114887812 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump"}]=0 v93) ==== 56+0+5634 (secure 0 0 0) 0x7f5c04013030 con 0x7f5c0c103530 2024-04-21T08:38:32.744 INFO:teuthology.orchestra.run.smithi086.stdout:epoch 93 2024-04-21T08:38:32.744 INFO:teuthology.orchestra.run.smithi086.stdout:fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:38:32.744 INFO:teuthology.orchestra.run.smithi086.stdout:created 2024-04-21T08:28:22.687329+0000 2024-04-21T08:38:32.744 INFO:teuthology.orchestra.run.smithi086.stdout:modified 2024-04-21T08:38:24.484832+0000 2024-04-21T08:38:32.744 INFO:teuthology.orchestra.run.smithi086.stdout:flags sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit 2024-04-21T08:38:32.744 INFO:teuthology.orchestra.run.smithi086.stdout:crush_version 27 2024-04-21T08:38:32.744 INFO:teuthology.orchestra.run.smithi086.stdout:full_ratio 0.95 2024-04-21T08:38:32.744 INFO:teuthology.orchestra.run.smithi086.stdout:backfillfull_ratio 0.9 2024-04-21T08:38:32.744 INFO:teuthology.orchestra.run.smithi086.stdout:nearfull_ratio 0.85 2024-04-21T08:38:32.744 INFO:teuthology.orchestra.run.smithi086.stdout:require_min_compat_client luminous 2024-04-21T08:38:32.744 INFO:teuthology.orchestra.run.smithi086.stdout:min_compat_client luminous 2024-04-21T08:38:32.744 INFO:teuthology.orchestra.run.smithi086.stdout:require_osd_release squid 2024-04-21T08:38:32.744 INFO:teuthology.orchestra.run.smithi086.stdout:stretch_mode_enabled false 2024-04-21T08:38:32.745 INFO:teuthology.orchestra.run.smithi086.stdout:pool 1 '.mgr' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 1 pgp_num 1 autoscale_mode off last_change 21 flags hashpspool stripe_width 0 pg_num_max 32 pg_num_min 1 application mgr read_balance_score 12.00 2024-04-21T08:38:32.745 INFO:teuthology.orchestra.run.smithi086.stdout:pool 2 'cephfs_metadata' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 84 flags hashpspool stripe_width 0 pg_autoscale_bias 4 pg_num_min 16 recovery_priority 5 application cephfs read_balance_score 1.88 2024-04-21T08:38:32.745 INFO:teuthology.orchestra.run.smithi086.stdout:pool 3 'cephfs_data' replicated size 3 min_size 2 crush_rule 0 object_hash rjenkins pg_num 64 pgp_num 64 autoscale_mode off last_change 85 flags hashpspool stripe_width 0 pg_num_min 64 target_size_ratio 0.05 application cephfs read_balance_score 1.50 2024-04-21T08:38:32.745 INFO:teuthology.orchestra.run.smithi086.stdout:pool 4 'cephfs_data_ec' erasure profile cephfs_data_ec size 4 min_size 3 crush_rule 1 object_hash rjenkins pg_num 32 pgp_num 32 autoscale_mode off last_change 89 flags hashpspool,ec_overwrites stripe_width 8192 pg_num_min 64 target_size_ratio 0.9 application cephfs 2024-04-21T08:38:32.745 INFO:teuthology.orchestra.run.smithi086.stdout:max_osd 12 2024-04-21T08:38:32.745 INFO:teuthology.orchestra.run.smithi086.stdout:osd.0 up in weight 1 up_from 8 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6802/2571516897,v1:172.21.15.86:6803/2571516897] [v2:172.21.15.86:6804/2571516897,v1:172.21.15.86:6805/2571516897] exists,up 89b7601f-0620-4db2-90f1-b45cea173738 2024-04-21T08:38:32.745 INFO:teuthology.orchestra.run.smithi086.stdout:osd.1 up in weight 1 up_from 13 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.88:6800/74067182,v1:172.21.15.88:6801/74067182] [v2:172.21.15.88:6802/74067182,v1:172.21.15.88:6803/74067182] exists,up 4891031c-f98c-42bc-bb3b-a420855ecc06 2024-04-21T08:38:32.745 INFO:teuthology.orchestra.run.smithi086.stdout:osd.2 up in weight 1 up_from 18 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.179:6800/4137659051,v1:172.21.15.179:6801/4137659051] [v2:172.21.15.179:6802/4137659051,v1:172.21.15.179:6803/4137659051] exists,up 60d12aa4-9312-4fa2-8d0b-2b3f42caad00 2024-04-21T08:38:32.745 INFO:teuthology.orchestra.run.smithi086.stdout:osd.3 up in weight 1 up_from 25 up_thru 87 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6810/2309603148,v1:172.21.15.86:6811/2309603148] [v2:172.21.15.86:6812/2309603148,v1:172.21.15.86:6813/2309603148] exists,up 404c3158-8cb0-4c02-b48a-f9ee5ff7b784 2024-04-21T08:38:32.745 INFO:teuthology.orchestra.run.smithi086.stdout:osd.4 up in weight 1 up_from 33 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.88:6808/1818714217,v1:172.21.15.88:6809/1818714217] [v2:172.21.15.88:6810/1818714217,v1:172.21.15.88:6811/1818714217] exists,up 458ff202-d781-4ec7-ac93-0b49f835b503 2024-04-21T08:38:32.745 INFO:teuthology.orchestra.run.smithi086.stdout:osd.5 up in weight 1 up_from 38 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.179:6808/863840990,v1:172.21.15.179:6809/863840990] [v2:172.21.15.179:6810/863840990,v1:172.21.15.179:6811/863840990] exists,up 8123cc90-bd8b-4204-bf31-46b34e9f589d 2024-04-21T08:38:32.745 INFO:teuthology.orchestra.run.smithi086.stdout:osd.6 up in weight 1 up_from 46 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6818/1224231118,v1:172.21.15.86:6819/1224231118] [v2:172.21.15.86:6820/1224231118,v1:172.21.15.86:6821/1224231118] exists,up aac9d7c7-659a-4110-903a-a871c086066f 2024-04-21T08:38:32.745 INFO:teuthology.orchestra.run.smithi086.stdout:osd.7 up in weight 1 up_from 51 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.88:6816/136629878,v1:172.21.15.88:6817/136629878] [v2:172.21.15.88:6818/136629878,v1:172.21.15.88:6819/136629878] exists,up df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5 2024-04-21T08:38:32.746 INFO:teuthology.orchestra.run.smithi086.stdout:osd.8 up in weight 1 up_from 59 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.179:6816/1770418957,v1:172.21.15.179:6817/1770418957] [v2:172.21.15.179:6818/1770418957,v1:172.21.15.179:6819/1770418957] exists,up 63e59f4b-0426-46db-92f7-c55dae61b5ee 2024-04-21T08:38:32.746 INFO:teuthology.orchestra.run.smithi086.stdout:osd.9 up in weight 1 up_from 67 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.86:6826/1108500274,v1:172.21.15.86:6827/1108500274] [v2:172.21.15.86:6828/1108500274,v1:172.21.15.86:6829/1108500274] exists,up 5841bb36-ed6b-45d8-b857-eeab139c44e6 2024-04-21T08:38:32.746 INFO:teuthology.orchestra.run.smithi086.stdout:osd.10 up in weight 1 up_from 72 up_thru 90 down_at 0 last_clean_interval [0,0) [v2:172.21.15.88:6824/1294275353,v1:172.21.15.88:6825/1294275353] [v2:172.21.15.88:6826/1294275353,v1:172.21.15.88:6827/1294275353] exists,up f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c 2024-04-21T08:38:32.746 INFO:teuthology.orchestra.run.smithi086.stdout:osd.11 up in weight 1 up_from 80 up_thru 92 down_at 0 last_clean_interval [0,0) [v2:172.21.15.179:6824/4007070233,v1:172.21.15.179:6825/4007070233] [v2:172.21.15.179:6826/4007070233,v1:172.21.15.179:6827/4007070233] exists,up e0d4cc76-e7a6-4a6b-909c-95488ccd5a36 2024-04-21T08:38:32.746 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 2.e [2,4] 2024-04-21T08:38:32.746 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 2.f [8,11] 2024-04-21T08:38:32.746 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 2.10 [2,1] 2024-04-21T08:38:32.746 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 2.1f [7,11] 2024-04-21T08:38:32.746 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 3.0 [4,8] 2024-04-21T08:38:32.746 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 3.3 [4,9] 2024-04-21T08:38:32.746 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 3.e [5,9] 2024-04-21T08:38:32.746 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 3.36 [4,9] 2024-04-21T08:38:32.746 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 4.5 [9,10] 2024-04-21T08:38:32.746 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 4.14 [9,8] 2024-04-21T08:38:32.747 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 4.1a [5,8] 2024-04-21T08:38:32.747 INFO:teuthology.orchestra.run.smithi086.stdout:pg_upmap_items 4.1f [9,8] 2024-04-21T08:38:32.747 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:6800/2143206409 expires 2024-04-22T08:29:23.190001+0000 2024-04-21T08:38:32.747 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/1412393225 expires 2024-04-22T08:29:23.190001+0000 2024-04-21T08:38:32.747 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:6800/2386990413 expires 2024-04-22T08:28:57.384403+0000 2024-04-21T08:38:32.747 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/1687852404 expires 2024-04-22T08:29:23.190001+0000 2024-04-21T08:38:32.747 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:6801/2386990413 expires 2024-04-22T08:28:57.384403+0000 2024-04-21T08:38:32.747 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/2020707404 expires 2024-04-22T08:28:57.384403+0000 2024-04-21T08:38:32.747 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/2568130933 expires 2024-04-22T08:28:57.384403+0000 2024-04-21T08:38:32.747 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/3952607648 expires 2024-04-22T08:28:57.384403+0000 2024-04-21T08:38:32.747 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:6801/2143206409 expires 2024-04-22T08:29:23.190001+0000 2024-04-21T08:38:32.747 INFO:teuthology.orchestra.run.smithi086.stdout:blocklist 172.21.15.86:0/2756903426 expires 2024-04-22T08:29:23.190001+0000 2024-04-21T08:38:32.748 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:32 smithi179 ceph-mon[31829]: pgmap v357: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 11 op/s 2024-04-21T08:38:32.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.746+0000 7f5c10966700 1 -- 172.21.15.86:0/2114887812 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f5bf40a3c40 msgr2=0x7f5bf40a6100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:32.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.746+0000 7f5c10966700 1 --2- 172.21.15.86:0/2114887812 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f5bf40a3c40 0x7f5bf40a6100 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7f5c0c19a1c0 tx=0x7f5bfc009190 comp rx=0 tx=0).stop 2024-04-21T08:38:32.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.746+0000 7f5c10966700 1 -- 172.21.15.86:0/2114887812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c0c103530 msgr2=0x7f5c0c1a0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:32.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.746+0000 7f5c10966700 1 --2- 172.21.15.86:0/2114887812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c0c103530 0x7f5c0c1a0260 secure :-1 s=READY pgs=425 cs=0 l=1 rev1=1 crypto rx=0x7f5c0400b970 tx=0x7f5c0400bc80 comp rx=0 tx=0).stop 2024-04-21T08:38:32.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.746+0000 7f5c10966700 1 -- 172.21.15.86:0/2114887812 shutdown_connections 2024-04-21T08:38:32.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.746+0000 7f5c10966700 1 --2- 172.21.15.86:0/2114887812 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f5bf40a3c40 0x7f5bf40a6100 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:32.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.746+0000 7f5c10966700 1 --2- 172.21.15.86:0/2114887812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c0c103530 0x7f5c0c1a0260 unknown :-1 s=CLOSED pgs=425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:32.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.746+0000 7f5c10966700 1 --2- 172.21.15.86:0/2114887812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c0c101990 0x7f5c0c198ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:32.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.746+0000 7f5c10966700 1 --2- 172.21.15.86:0/2114887812 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c0c102b90 0x7f5c0c199220 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:32.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.746+0000 7f5c10966700 1 -- 172.21.15.86:0/2114887812 >> 172.21.15.86:0/2114887812 conn(0x7f5c0c0fcf30 msgr2=0x7f5c0c06afb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:32.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.747+0000 7f5c10966700 1 -- 172.21.15.86:0/2114887812 shutdown_connections 2024-04-21T08:38:32.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:32.747+0000 7f5c10966700 1 -- 172.21.15.86:0/2114887812 wait complete. 2024-04-21T08:38:33.457 INFO:teuthology.run_tasks:Running task fs.ready... 2024-04-21T08:38:33.469 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:38:33.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.572+0000 7f7ffabbd640 1 -- 172.21.15.86:0/3802184161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ff4103180 msgr2=0x7f7ff4103600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:33.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.572+0000 7f7ffabbd640 1 --2- 172.21.15.86:0/3802184161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ff4103180 0x7f7ff4103600 secure :-1 s=READY pgs=426 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0009900 tx=0x7f7fe002bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:33.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.572+0000 7f7ffabbd640 1 -- 172.21.15.86:0/3802184161 shutdown_connections 2024-04-21T08:38:33.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.572+0000 7f7ffabbd640 1 --2- 172.21.15.86:0/3802184161 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ff4103b40 0x7f7ff410a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:33.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.572+0000 7f7ffabbd640 1 --2- 172.21.15.86:0/3802184161 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ff4103180 0x7f7ff4103600 unknown :-1 s=CLOSED pgs=426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:33.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.572+0000 7f7ffabbd640 1 --2- 172.21.15.86:0/3802184161 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ff4101f90 0x7f7ff41023b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:33.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.572+0000 7f7ffabbd640 1 -- 172.21.15.86:0/3802184161 >> 172.21.15.86:0/3802184161 conn(0x7f7ff40fd500 msgr2=0x7f7ff40ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:33.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.572+0000 7f7ffabbd640 1 -- 172.21.15.86:0/3802184161 shutdown_connections 2024-04-21T08:38:33.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.572+0000 7f7ffabbd640 1 -- 172.21.15.86:0/3802184161 wait complete. 2024-04-21T08:38:33.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.572+0000 7f7ffabbd640 1 Processor -- start 2024-04-21T08:38:33.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.572+0000 7f7ffabbd640 1 -- start start 2024-04-21T08:38:33.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.573+0000 7f7ffabbd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ff4101f90 0x7f7ff4198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:33.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.573+0000 7f7ffabbd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ff4103180 0x7f7ff41986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:33.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.573+0000 7f7ffabbd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ff4103b40 0x7f7ff419f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:33.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.573+0000 7f7ffabbd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff410dad0 con 0x7f7ff4101f90 2024-04-21T08:38:33.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.573+0000 7f7ffabbd640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff410d950 con 0x7f7ff4103b40 2024-04-21T08:38:33.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.573+0000 7f7ffabbd640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7ff410dc50 con 0x7f7ff4103180 2024-04-21T08:38:33.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.573+0000 7f7ff8959640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ff4101f90 0x7f7ff4198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:33.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.573+0000 7f7ff8959640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ff4101f90 0x7f7ff4198180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60062/0 (socket says 172.21.15.86:60062) 2024-04-21T08:38:33.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.573+0000 7f7ff8959640 1 -- 172.21.15.86:0/3234210115 learned_addr learned my addr 172.21.15.86:0/3234210115 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:33.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.573+0000 7f7ff915a640 1 --2- 172.21.15.86:0/3234210115 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ff4103b40 0x7f7ff419f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:33.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.573+0000 7f7ff8959640 1 -- 172.21.15.86:0/3234210115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ff4103180 msgr2=0x7f7ff41986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:33.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.573+0000 7f7ff8959640 1 --2- 172.21.15.86:0/3234210115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ff4103180 0x7f7ff41986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:33.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.573+0000 7f7ff8959640 1 -- 172.21.15.86:0/3234210115 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ff4103b40 msgr2=0x7f7ff419f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:33.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.573+0000 7f7ff8959640 1 --2- 172.21.15.86:0/3234210115 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ff4103b40 0x7f7ff419f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:33.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.573+0000 7f7ff8959640 1 -- 172.21.15.86:0/3234210115 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ff419fda0 con 0x7f7ff4101f90 2024-04-21T08:38:33.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.573+0000 7f7ff915a640 1 --2- 172.21.15.86:0/3234210115 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ff4103b40 0x7f7ff419f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:33.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.573+0000 7f7ff8959640 1 --2- 172.21.15.86:0/3234210115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ff4101f90 0x7f7ff4198180 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7fe800d950 tx=0x7f7fe800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:33.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.573+0000 7f7ff1ffb640 1 -- 172.21.15.86:0/3234210115 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe8017020 con 0x7f7ff4101f90 2024-04-21T08:38:33.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.573+0000 7f7ff1ffb640 1 -- 172.21.15.86:0/3234210115 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fe8009c30 con 0x7f7ff4101f90 2024-04-21T08:38:33.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.573+0000 7f7ffabbd640 1 -- 172.21.15.86:0/3234210115 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ff41a0090 con 0x7f7ff4101f90 2024-04-21T08:38:33.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.573+0000 7f7ff1ffb640 1 -- 172.21.15.86:0/3234210115 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fe800f6b0 con 0x7f7ff4101f90 2024-04-21T08:38:33.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.574+0000 7f7ffabbd640 1 -- 172.21.15.86:0/3234210115 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ff41a05e0 con 0x7f7ff4101f90 2024-04-21T08:38:33.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.574+0000 7f7ffabbd640 1 -- 172.21.15.86:0/3234210115 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ff41092b0 con 0x7f7ff4101f90 2024-04-21T08:38:33.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.575+0000 7f7ff1ffb640 1 -- 172.21.15.86:0/3234210115 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f7fe801a020 con 0x7f7ff4101f90 2024-04-21T08:38:33.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.576+0000 7f7ff1ffb640 1 --2- 172.21.15.86:0/3234210115 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f7fe40a3bb0 0x7f7fe40a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:33.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.576+0000 7f7ff3fff640 1 --2- 172.21.15.86:0/3234210115 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f7fe40a3bb0 0x7f7fe40a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:33.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.576+0000 7f7ff1ffb640 1 -- 172.21.15.86:0/3234210115 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f7fe80d0510 con 0x7f7ff4101f90 2024-04-21T08:38:33.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.576+0000 7f7ff3fff640 1 --2- 172.21.15.86:0/3234210115 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f7fe40a3bb0 0x7f7fe40a6070 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0036040 tx=0x7f7fe0000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:33.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.580+0000 7f7ff1ffb640 1 -- 172.21.15.86:0/3234210115 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7fe8009de0 con 0x7f7ff4101f90 2024-04-21T08:38:33.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:33 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2114887812' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-21T08:38:33.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:33 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2114887812' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-21T08:38:33.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.760+0000 7f7ffabbd640 1 -- 172.21.15.86:0/3234210115 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7ff402d5c0 con 0x7f7ff4101f90 2024-04-21T08:38:33.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.762+0000 7f7ff1ffb640 1 -- 172.21.15.86:0/3234210115 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24689 (secure 0 0 0) 0x7f7fe809cb60 con 0x7f7ff4101f90 2024-04-21T08:38:33.761 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:38:33.762 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":93,"fsid":"f8cec7cc-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:22.687329+0000","modified":"2024-04-21T08:38:24.484832+0000","last_up_change":"2024-04-21T08:35:30.048315+0000","last_in_change":"2024-04-21T08:35:10.099496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:45.279109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:18.300782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:19.574636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:22.905632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89b7601f-0620-4db2-90f1-b45cea173738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6803","nonce":2571516897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6805","nonce":2571516897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6809","nonce":2571516897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6807","nonce":2571516897}]},"public_addr":"172.21.15.86:6803/2571516897","cluster_addr":"172.21.15.86:6805/2571516897","heartbeat_back_addr":"172.21.15.86:6809/2571516897","heartbeat_front_addr":"172.21.15.86:6807/2571516897","state":["exists","up"]},{"osd":1,"uuid":"4891031c-f98c-42bc-bb3b-a420855ecc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6801","nonce":74067182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6803","nonce":74067182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6807","nonce":74067182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6805","nonce":74067182}]},"public_addr":"172.21.15.88:6801/74067182","cluster_addr":"172.21.15.88:6803/74067182","heartbeat_back_addr":"172.21.15.88:6807/74067182","heartbeat_front_addr":"172.21.15.88:6805/74067182","state":["exists","up"]},{"osd":2,"uuid":"60d12aa4-9312-4fa2-8d0b-2b3f42caad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6801","nonce":4137659051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6803","nonce":4137659051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6807","nonce":4137659051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6805","nonce":4137659051}]},"public_addr":"172.21.15.179:6801/4137659051","cluster_addr":"172.21.15.179:6803/4137659051","heartbeat_back_addr":"172.21.15.179:6807/4137659051","heartbeat_front_addr":"172.21.15.179:6805/4137659051","state":["exists","up"]},{"osd":3,"uuid":"404c3158-8cb0-4c02-b48a-f9ee5ff7b784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6811","nonce":2309603148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6813","nonce":2309603148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6817","nonce":2309603148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6815","nonce":2309603148}]},"public_addr":"172.21.15.86:6811/2309603148","cluster_addr":"172.21.15.86:6813/2309603148","heartbeat_back_addr":"172.21.15.86:6817/2309603148","heartbeat_front_addr":"172.21.15.86:6815/2309603148","state":["exists","up"]},{"osd":4,"uuid":"458ff202-d781-4ec7-ac93-0b49f835b503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6809","nonce":1818714217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6811","nonce":1818714217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6815","nonce":1818714217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6813","nonce":1818714217}]},"public_addr":"172.21.15.88:6809/1818714217","cluster_addr":"172.21.15.88:6811/1818714217","heartbeat_back_addr":"172.21.15.88:6815/1818714217","heartbeat_front_addr":"172.21.15.88:6813/1818714217","state":["exists","up"]},{"osd":5,"uuid":"8123cc90-bd8b-4204-bf31-46b34e9f589d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6809","nonce":863840990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6811","nonce":863840990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6815","nonce":863840990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6813","nonce":863840990}]},"public_addr":"172.21.15.179:6809/863840990","cluster_addr":"172.21.15.179:6811/863840990","heartbeat_back_addr":"172.21.15.179:6815/863840990","heartbeat_front_addr":"172.21.15.179:6813/863840990","state":["exists","up"]},{"osd":6,"uuid":"aac9d7c7-659a-4110-903a-a871c086066f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6819","nonce":1224231118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6821","nonce":1224231118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6825","nonce":1224231118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6823","nonce":1224231118}]},"public_addr":"172.21.15.86:6819/1224231118","cluster_addr":"172.21.15.86:6821/1224231118","heartbeat_back_addr":"172.21.15.86:6825/1224231118","heartbeat_front_addr":"172.21.15.86:6823/1224231118","state":["exists","up"]},{"osd":7,"uuid":"df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6817","nonce":136629878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6819","nonce":136629878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6823","nonce":136629878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6821","nonce":136629878}]},"public_addr":"172.21.15.88:6817/136629878","cluster_addr":"172.21.15.88:6819/136629878","heartbeat_back_addr":"172.21.15.88:6823/136629878","heartbeat_front_addr":"172.21.15.88:6821/136629878","state":["exists","up"]},{"osd":8,"uuid":"63e59f4b-0426-46db-92f7-c55dae61b5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6817","nonce":1770418957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6819","nonce":1770418957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6823","nonce":1770418957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6821","nonce":1770418957}]},"public_addr":"172.21.15.179:6817/1770418957","cluster_addr":"172.21.15.179:6819/1770418957","heartbeat_back_addr":"172.21.15.179:6823/1770418957","heartbeat_front_addr":"172.21.15.179:6821/1770418957","state":["exists","up"]},{"osd":9,"uuid":"5841bb36-ed6b-45d8-b857-eeab139c44e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6827","nonce":1108500274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6829","nonce":1108500274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6833","nonce":1108500274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6831","nonce":1108500274}]},"public_addr":"172.21.15.86:6827/1108500274","cluster_addr":"172.21.15.86:6829/1108500274","heartbeat_back_addr":"172.21.15.86:6833/1108500274","heartbeat_front_addr":"172.21.15.86:6831/1108500274","state":["exists","up"]},{"osd":10,"uuid":"f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6825","nonce":1294275353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6827","nonce":1294275353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6831","nonce":1294275353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6829","nonce":1294275353}]},"public_addr":"172.21.15.88:6825/1294275353","cluster_addr":"172.21.15.88:6827/1294275353","heartbeat_back_addr":"172.21.15.88:6831/1294275353","heartbeat_front_addr":"172.21.15.88:6829/1294275353","state":["exists","up"]},{"osd":11,"uuid":"e0d4cc76-e7a6-4a6b-909c-95488ccd5a36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6825","nonce":4007070233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6827","nonce":4007070233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6831","nonce":4007070233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6829","nonce":4007070233}]},"public_addr":"172.21.15.179:6825/4007070233","cluster_addr":"172.21.15.179:6827/4007070233","heartbeat_back_addr":"172.21.15.179:6831/4007070233","heartbeat_front_addr":"172.21.15.179:6829/4007070233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:55.210605+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:18.723090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:42.547979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:06.421622+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:30.858315+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:55.455313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.623369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:46.185111+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.103627+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:37.297545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:02.637431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:28.199292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6800/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/1412393225":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6800/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/1687852404":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2020707404":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2568130933":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/3952607648":"2024-04-22T08:28:57.384403+0000","172.21.15.86:6801/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/2756903426":"2024-04-22T08:29:23.190001+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:38:33.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.764+0000 7f7ffabbd640 1 -- 172.21.15.86:0/3234210115 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f7fe40a3bb0 msgr2=0x7f7fe40a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:33.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.764+0000 7f7ffabbd640 1 --2- 172.21.15.86:0/3234210115 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f7fe40a3bb0 0x7f7fe40a6070 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7fe0036040 tx=0x7f7fe0000bc0 comp rx=0 tx=0).stop 2024-04-21T08:38:33.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.764+0000 7f7ffabbd640 1 -- 172.21.15.86:0/3234210115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ff4101f90 msgr2=0x7f7ff4198180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:33.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.764+0000 7f7ffabbd640 1 --2- 172.21.15.86:0/3234210115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ff4101f90 0x7f7ff4198180 secure :-1 s=READY pgs=427 cs=0 l=1 rev1=1 crypto rx=0x7f7fe800d950 tx=0x7f7fe800de10 comp rx=0 tx=0).stop 2024-04-21T08:38:33.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.764+0000 7f7ffabbd640 1 -- 172.21.15.86:0/3234210115 shutdown_connections 2024-04-21T08:38:33.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.764+0000 7f7ffabbd640 1 --2- 172.21.15.86:0/3234210115 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f7fe40a3bb0 0x7f7fe40a6070 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:33.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.764+0000 7f7ffabbd640 1 --2- 172.21.15.86:0/3234210115 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ff4103b40 0x7f7ff419f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:33.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.764+0000 7f7ffabbd640 1 --2- 172.21.15.86:0/3234210115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ff4103180 0x7f7ff41986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:33.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.764+0000 7f7ffabbd640 1 --2- 172.21.15.86:0/3234210115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ff4101f90 0x7f7ff4198180 unknown :-1 s=CLOSED pgs=427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:33.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.764+0000 7f7ffabbd640 1 -- 172.21.15.86:0/3234210115 >> 172.21.15.86:0/3234210115 conn(0x7f7ff40fd500 msgr2=0x7f7ff4106480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:33.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.764+0000 7f7ffabbd640 1 -- 172.21.15.86:0/3234210115 shutdown_connections 2024-04-21T08:38:33.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.764+0000 7f7ffabbd640 1 -- 172.21.15.86:0/3234210115 wait complete. 2024-04-21T08:38:33.774 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:33 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2114887812' entity='client.admin' cmd=[{"prefix": "osd dump"}]: dispatch 2024-04-21T08:38:33.775 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:31:45.279109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:36:18.300782+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:36:19.574636+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:36:22.905632+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:38:33.776 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:38:33.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.878+0000 7fe561a6d640 1 -- 172.21.15.86:0/4009464223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe55c100e90 msgr2=0x7fe55c1012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:33.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.878+0000 7fe561a6d640 1 --2- 172.21.15.86:0/4009464223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe55c100e90 0x7fe55c1012b0 secure :-1 s=READY pgs=428 cs=0 l=1 rev1=1 crypto rx=0x7fe54c009900 tx=0x7fe54c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:33.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.878+0000 7fe561a6d640 1 -- 172.21.15.86:0/4009464223 shutdown_connections 2024-04-21T08:38:33.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.878+0000 7fe561a6d640 1 --2- 172.21.15.86:0/4009464223 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe55c1029a0 0x7fe55c1092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:33.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.878+0000 7fe561a6d640 1 --2- 172.21.15.86:0/4009464223 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe55c101fe0 0x7fe55c102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:33.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.878+0000 7fe561a6d640 1 --2- 172.21.15.86:0/4009464223 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe55c100e90 0x7fe55c1012b0 unknown :-1 s=CLOSED pgs=428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:33.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.878+0000 7fe561a6d640 1 -- 172.21.15.86:0/4009464223 >> 172.21.15.86:0/4009464223 conn(0x7fe55c0fc440 msgr2=0x7fe55c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:33.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.878+0000 7fe561a6d640 1 -- 172.21.15.86:0/4009464223 shutdown_connections 2024-04-21T08:38:33.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.878+0000 7fe561a6d640 1 -- 172.21.15.86:0/4009464223 wait complete. 2024-04-21T08:38:33.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.878+0000 7fe561a6d640 1 Processor -- start 2024-04-21T08:38:33.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.879+0000 7fe561a6d640 1 -- start start 2024-04-21T08:38:33.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.879+0000 7fe561a6d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe55c101fe0 0x7fe55c104e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:33.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.879+0000 7fe561a6d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe55c1029a0 0x7fe55c103450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:33.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.879+0000 7fe561a6d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe55c103990 0x7fe55c103e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:33.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.879+0000 7fe561a6d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c10c9e0 con 0x7fe55c101fe0 2024-04-21T08:38:33.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.879+0000 7fe561a6d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c10c860 con 0x7fe55c1029a0 2024-04-21T08:38:33.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.879+0000 7fe561a6d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe55c10cb60 con 0x7fe55c103990 2024-04-21T08:38:33.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.879+0000 7fe55b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe55c101fe0 0x7fe55c104e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:33.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.879+0000 7fe55b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe55c101fe0 0x7fe55c104e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60068/0 (socket says 172.21.15.86:60068) 2024-04-21T08:38:33.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.879+0000 7fe55b7fe640 1 -- 172.21.15.86:0/1059397350 learned_addr learned my addr 172.21.15.86:0/1059397350 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:33.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.879+0000 7fe55bfff640 1 --2- 172.21.15.86:0/1059397350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe55c103990 0x7fe55c103e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:33.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.879+0000 7fe55affd640 1 --2- 172.21.15.86:0/1059397350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe55c1029a0 0x7fe55c103450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:33.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.879+0000 7fe55b7fe640 1 -- 172.21.15.86:0/1059397350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe55c103990 msgr2=0x7fe55c103e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:33.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.879+0000 7fe55b7fe640 1 --2- 172.21.15.86:0/1059397350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe55c103990 0x7fe55c103e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:33.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.879+0000 7fe55b7fe640 1 -- 172.21.15.86:0/1059397350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe55c1029a0 msgr2=0x7fe55c103450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:33.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.879+0000 7fe55b7fe640 1 --2- 172.21.15.86:0/1059397350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe55c1029a0 0x7fe55c103450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:33.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.879+0000 7fe55b7fe640 1 -- 172.21.15.86:0/1059397350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe55c1a2690 con 0x7fe55c101fe0 2024-04-21T08:38:33.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.879+0000 7fe55b7fe640 1 --2- 172.21.15.86:0/1059397350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe55c101fe0 0x7fe55c104e10 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe54c000c00 tx=0x7fe54c004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:33.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.879+0000 7fe558ff9640 1 -- 172.21.15.86:0/1059397350 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c043020 con 0x7fe55c101fe0 2024-04-21T08:38:33.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.880+0000 7fe558ff9640 1 -- 172.21.15.86:0/1059397350 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe54c0052b0 con 0x7fe55c101fe0 2024-04-21T08:38:33.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.880+0000 7fe561a6d640 1 -- 172.21.15.86:0/1059397350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe55c1a2920 con 0x7fe55c101fe0 2024-04-21T08:38:33.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.880+0000 7fe558ff9640 1 -- 172.21.15.86:0/1059397350 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe54c003d10 con 0x7fe55c101fe0 2024-04-21T08:38:33.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.880+0000 7fe561a6d640 1 -- 172.21.15.86:0/1059397350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe55c1a2e20 con 0x7fe55c101fe0 2024-04-21T08:38:33.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.881+0000 7fe561a6d640 1 -- 172.21.15.86:0/1059397350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe55c1a2ff0 con 0x7fe55c101fe0 2024-04-21T08:38:33.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.882+0000 7fe558ff9640 1 -- 172.21.15.86:0/1059397350 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe54c046020 con 0x7fe55c101fe0 2024-04-21T08:38:33.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.882+0000 7fe558ff9640 1 --2- 172.21.15.86:0/1059397350 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe5440a3bb0 0x7fe5440a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:33.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.882+0000 7fe55affd640 1 --2- 172.21.15.86:0/1059397350 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe5440a3bb0 0x7fe5440a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:33.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.882+0000 7fe558ff9640 1 -- 172.21.15.86:0/1059397350 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fe54c0f3550 con 0x7fe55c101fe0 2024-04-21T08:38:33.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.883+0000 7fe55affd640 1 --2- 172.21.15.86:0/1059397350 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe5440a3bb0 0x7fe5440a6070 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe55c104b80 tx=0x7fe550005c60 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:33.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:33.886+0000 7fe558ff9640 1 -- 172.21.15.86:0/1059397350 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe54c04d030 con 0x7fe55c101fe0 2024-04-21T08:38:34.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.060+0000 7fe561a6d640 1 -- 172.21.15.86:0/1059397350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fe55c1a32e0 con 0x7fe55c101fe0 2024-04-21T08:38:34.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.060+0000 7fe558ff9640 1 -- 172.21.15.86:0/1059397350 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7fe54c0bfba0 con 0x7fe55c101fe0 2024-04-21T08:38:34.060 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-21T08:38:34.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.062+0000 7fe561a6d640 1 -- 172.21.15.86:0/1059397350 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe5440a3bb0 msgr2=0x7fe5440a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:34.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.062+0000 7fe561a6d640 1 --2- 172.21.15.86:0/1059397350 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe5440a3bb0 0x7fe5440a6070 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7fe55c104b80 tx=0x7fe550005c60 comp rx=0 tx=0).stop 2024-04-21T08:38:34.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.062+0000 7fe561a6d640 1 -- 172.21.15.86:0/1059397350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe55c101fe0 msgr2=0x7fe55c104e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:34.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.062+0000 7fe561a6d640 1 --2- 172.21.15.86:0/1059397350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe55c101fe0 0x7fe55c104e10 secure :-1 s=READY pgs=429 cs=0 l=1 rev1=1 crypto rx=0x7fe54c000c00 tx=0x7fe54c004270 comp rx=0 tx=0).stop 2024-04-21T08:38:34.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.062+0000 7fe561a6d640 1 -- 172.21.15.86:0/1059397350 shutdown_connections 2024-04-21T08:38:34.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.062+0000 7fe561a6d640 1 --2- 172.21.15.86:0/1059397350 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe5440a3bb0 0x7fe5440a6070 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:34.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.062+0000 7fe561a6d640 1 --2- 172.21.15.86:0/1059397350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe55c103990 0x7fe55c103e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:34.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.062+0000 7fe561a6d640 1 --2- 172.21.15.86:0/1059397350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe55c1029a0 0x7fe55c103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:34.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.062+0000 7fe561a6d640 1 --2- 172.21.15.86:0/1059397350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe55c101fe0 0x7fe55c104e10 unknown :-1 s=CLOSED pgs=429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:34.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.062+0000 7fe561a6d640 1 -- 172.21.15.86:0/1059397350 >> 172.21.15.86:0/1059397350 conn(0x7fe55c0fc440 msgr2=0x7fe55c06a5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:34.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.063+0000 7fe561a6d640 1 -- 172.21.15.86:0/1059397350 shutdown_connections 2024-04-21T08:38:34.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.063+0000 7fe561a6d640 1 -- 172.21.15.86:0/1059397350 wait complete. 2024-04-21T08:38:34.073 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:38:34.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.175+0000 7f606e396640 1 -- 172.21.15.86:0/1486255894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6068101fa0 msgr2=0x7f60681023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:34.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.175+0000 7f606e396640 1 --2- 172.21.15.86:0/1486255894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6068101fa0 0x7f60681023c0 secure :-1 s=READY pgs=430 cs=0 l=1 rev1=1 crypto rx=0x7f6058009900 tx=0x7f605802bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:34.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.175+0000 7f606e396640 1 -- 172.21.15.86:0/1486255894 shutdown_connections 2024-04-21T08:38:34.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.175+0000 7f606e396640 1 --2- 172.21.15.86:0/1486255894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6068103b50 0x7f606810a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:34.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.175+0000 7f606e396640 1 --2- 172.21.15.86:0/1486255894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6068103190 0x7f6068103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:34.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.175+0000 7f606e396640 1 --2- 172.21.15.86:0/1486255894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6068101fa0 0x7f60681023c0 unknown :-1 s=CLOSED pgs=430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:34.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.175+0000 7f606e396640 1 -- 172.21.15.86:0/1486255894 >> 172.21.15.86:0/1486255894 conn(0x7f60680fd530 msgr2=0x7f60680ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:34.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.175+0000 7f606e396640 1 -- 172.21.15.86:0/1486255894 shutdown_connections 2024-04-21T08:38:34.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.175+0000 7f606e396640 1 -- 172.21.15.86:0/1486255894 wait complete. 2024-04-21T08:38:34.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.176+0000 7f606e396640 1 Processor -- start 2024-04-21T08:38:34.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.176+0000 7f606e396640 1 -- start start 2024-04-21T08:38:34.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.176+0000 7f606e396640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6068101fa0 0x7f6068198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:34.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.176+0000 7f606e396640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6068103190 0x7f6068198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:34.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.176+0000 7f606e396640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6068103b50 0x7f606819f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:34.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.176+0000 7f606e396640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f606810dae0 con 0x7f6068103b50 2024-04-21T08:38:34.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.176+0000 7f606e396640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f606810d960 con 0x7f6068103190 2024-04-21T08:38:34.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.176+0000 7f606e396640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f606810dc60 con 0x7f6068101fa0 2024-04-21T08:38:34.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.176+0000 7f606c933640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6068103b50 0x7f606819f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:34.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.176+0000 7f606c933640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6068103b50 0x7f606819f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60084/0 (socket says 172.21.15.86:60084) 2024-04-21T08:38:34.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.176+0000 7f606c933640 1 -- 172.21.15.86:0/2397676700 learned_addr learned my addr 172.21.15.86:0/2397676700 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:34.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.176+0000 7f60677fe640 1 --2- 172.21.15.86:0/2397676700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6068103190 0x7f6068198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:34.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.176+0000 7f6067fff640 1 --2- 172.21.15.86:0/2397676700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6068101fa0 0x7f6068198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:34.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.176+0000 7f606c933640 1 -- 172.21.15.86:0/2397676700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6068101fa0 msgr2=0x7f6068198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:34.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.177+0000 7f606c933640 1 --2- 172.21.15.86:0/2397676700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6068101fa0 0x7f6068198230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:34.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.177+0000 7f606c933640 1 -- 172.21.15.86:0/2397676700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6068103190 msgr2=0x7f6068198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:34.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.177+0000 7f606c933640 1 --2- 172.21.15.86:0/2397676700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6068103190 0x7f6068198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:34.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.177+0000 7f606c933640 1 -- 172.21.15.86:0/2397676700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f606819fe50 con 0x7f6068103b50 2024-04-21T08:38:34.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.177+0000 7f606c933640 1 --2- 172.21.15.86:0/2397676700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6068103b50 0x7f606819f7e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f606000cad0 tx=0x7f606000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:34.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.177+0000 7f60657fa640 1 -- 172.21.15.86:0/2397676700 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606000ae20 con 0x7f6068103b50 2024-04-21T08:38:34.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.177+0000 7f606e396640 1 -- 172.21.15.86:0/2397676700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60681a0140 con 0x7f6068103b50 2024-04-21T08:38:34.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.177+0000 7f60657fa640 1 -- 172.21.15.86:0/2397676700 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6060005020 con 0x7f6068103b50 2024-04-21T08:38:34.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.177+0000 7f60657fa640 1 -- 172.21.15.86:0/2397676700 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f606000f6b0 con 0x7f6068103b50 2024-04-21T08:38:34.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.177+0000 7f606e396640 1 -- 172.21.15.86:0/2397676700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60681a0640 con 0x7f6068103b50 2024-04-21T08:38:34.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.178+0000 7f606e396640 1 -- 172.21.15.86:0/2397676700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6068109480 con 0x7f6068103b50 2024-04-21T08:38:34.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.179+0000 7f60657fa640 1 -- 172.21.15.86:0/2397676700 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f60600051d0 con 0x7f6068103b50 2024-04-21T08:38:34.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.179+0000 7f60657fa640 1 --2- 172.21.15.86:0/2397676700 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f60500a3bb0 0x7f60500a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:34.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.180+0000 7f6067fff640 1 --2- 172.21.15.86:0/2397676700 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f60500a3bb0 0x7f60500a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:34.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.180+0000 7f60657fa640 1 -- 172.21.15.86:0/2397676700 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f60600d0900 con 0x7f6068103b50 2024-04-21T08:38:34.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.180+0000 7f6067fff640 1 --2- 172.21.15.86:0/2397676700 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f60500a3bb0 0x7f60500a6070 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6058036040 tx=0x7f6058000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:34.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.184+0000 7f60657fa640 1 -- 172.21.15.86:0/2397676700 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6060014030 con 0x7f6068103b50 2024-04-21T08:38:34.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.362+0000 7f606e396640 1 -- 172.21.15.86:0/2397676700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f6068062b50 con 0x7f6068103b50 2024-04-21T08:38:34.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.362+0000 7f60657fa640 1 -- 172.21.15.86:0/2397676700 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f606009cf50 con 0x7f6068103b50 2024-04-21T08:38:34.362 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-21T08:38:34.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.365+0000 7f606e396640 1 -- 172.21.15.86:0/2397676700 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f60500a3bb0 msgr2=0x7f60500a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:34.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.365+0000 7f606e396640 1 --2- 172.21.15.86:0/2397676700 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f60500a3bb0 0x7f60500a6070 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6058036040 tx=0x7f6058000bc0 comp rx=0 tx=0).stop 2024-04-21T08:38:34.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.365+0000 7f606e396640 1 -- 172.21.15.86:0/2397676700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6068103b50 msgr2=0x7f606819f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:34.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.365+0000 7f606e396640 1 --2- 172.21.15.86:0/2397676700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6068103b50 0x7f606819f7e0 secure :-1 s=READY pgs=431 cs=0 l=1 rev1=1 crypto rx=0x7f606000cad0 tx=0x7f606000cf90 comp rx=0 tx=0).stop 2024-04-21T08:38:34.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.365+0000 7f606e396640 1 -- 172.21.15.86:0/2397676700 shutdown_connections 2024-04-21T08:38:34.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.365+0000 7f606e396640 1 --2- 172.21.15.86:0/2397676700 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f60500a3bb0 0x7f60500a6070 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:34.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.365+0000 7f606e396640 1 --2- 172.21.15.86:0/2397676700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6068103b50 0x7f606819f7e0 unknown :-1 s=CLOSED pgs=431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:34.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.365+0000 7f606e396640 1 --2- 172.21.15.86:0/2397676700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6068103190 0x7f6068198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:34.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.365+0000 7f606e396640 1 --2- 172.21.15.86:0/2397676700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6068101fa0 0x7f6068198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:34.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.365+0000 7f606e396640 1 -- 172.21.15.86:0/2397676700 >> 172.21.15.86:0/2397676700 conn(0x7f60680fd530 msgr2=0x7f6068106650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:34.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.365+0000 7f606e396640 1 -- 172.21.15.86:0/2397676700 shutdown_connections 2024-04-21T08:38:34.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.365+0000 7f606e396640 1 -- 172.21.15.86:0/2397676700 wait complete. 2024-04-21T08:38:34.375 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:38:34.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.478+0000 7f0c3b4ba640 1 -- 172.21.15.86:0/1229588760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c34102040 msgr2=0x7f0c341024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:34.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.478+0000 7f0c3b4ba640 1 --2- 172.21.15.86:0/1229588760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c34102040 0x7f0c341024c0 secure :-1 s=READY pgs=432 cs=0 l=1 rev1=1 crypto rx=0x7f0c24009900 tx=0x7f0c2402bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:34.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.478+0000 7f0c3b4ba640 1 -- 172.21.15.86:0/1229588760 shutdown_connections 2024-04-21T08:38:34.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.478+0000 7f0c3b4ba640 1 --2- 172.21.15.86:0/1229588760 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c34102a00 0x7f0c34109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:34.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.478+0000 7f0c3b4ba640 1 --2- 172.21.15.86:0/1229588760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c34102040 0x7f0c341024c0 unknown :-1 s=CLOSED pgs=432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:34.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.478+0000 7f0c3b4ba640 1 --2- 172.21.15.86:0/1229588760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c34100e60 0x7f0c34101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:34.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.478+0000 7f0c3b4ba640 1 -- 172.21.15.86:0/1229588760 >> 172.21.15.86:0/1229588760 conn(0x7f0c340fc420 msgr2=0x7f0c340fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:34.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.478+0000 7f0c3b4ba640 1 -- 172.21.15.86:0/1229588760 shutdown_connections 2024-04-21T08:38:34.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.478+0000 7f0c3b4ba640 1 -- 172.21.15.86:0/1229588760 wait complete. 2024-04-21T08:38:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.479+0000 7f0c3b4ba640 1 Processor -- start 2024-04-21T08:38:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.479+0000 7f0c3b4ba640 1 -- start start 2024-04-21T08:38:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.479+0000 7f0c3b4ba640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c34100e60 0x7f0c341981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.479+0000 7f0c3b4ba640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c34102040 0x7f0c34198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.479+0000 7f0c3b4ba640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c34102a00 0x7f0c3419f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.479+0000 7f0c3b4ba640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3410daf0 con 0x7f0c34102a00 2024-04-21T08:38:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.479+0000 7f0c3b4ba640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3410d970 con 0x7f0c34102040 2024-04-21T08:38:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.479+0000 7f0c3b4ba640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0c3410dc70 con 0x7f0c34100e60 2024-04-21T08:38:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.479+0000 7f0c39a57640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c34102a00 0x7f0c3419f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.479+0000 7f0c39a57640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c34102a00 0x7f0c3419f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60106/0 (socket says 172.21.15.86:60106) 2024-04-21T08:38:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.479+0000 7f0c39a57640 1 -- 172.21.15.86:0/3556191574 learned_addr learned my addr 172.21.15.86:0/3556191574 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.479+0000 7f0c39256640 1 --2- 172.21.15.86:0/3556191574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c34100e60 0x7f0c341981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.479+0000 7f0c38a55640 1 --2- 172.21.15.86:0/3556191574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c34102040 0x7f0c34198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:34.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.479+0000 7f0c39a57640 1 -- 172.21.15.86:0/3556191574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c34100e60 msgr2=0x7f0c341981c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:34.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.479+0000 7f0c39a57640 1 --2- 172.21.15.86:0/3556191574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c34100e60 0x7f0c341981c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:34.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.480+0000 7f0c39a57640 1 -- 172.21.15.86:0/3556191574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c34102040 msgr2=0x7f0c34198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:34.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.480+0000 7f0c39a57640 1 --2- 172.21.15.86:0/3556191574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c34102040 0x7f0c34198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:34.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.480+0000 7f0c39a57640 1 -- 172.21.15.86:0/3556191574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0c3419fde0 con 0x7f0c34102a00 2024-04-21T08:38:34.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.480+0000 7f0c38a55640 1 --2- 172.21.15.86:0/3556191574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c34102040 0x7f0c34198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:34.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.480+0000 7f0c39a57640 1 --2- 172.21.15.86:0/3556191574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c34102a00 0x7f0c3419f770 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00c970 tx=0x7f0c2c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:34.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.480+0000 7f0c2a7fc640 1 -- 172.21.15.86:0/3556191574 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c007bb0 con 0x7f0c34102a00 2024-04-21T08:38:34.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.480+0000 7f0c3b4ba640 1 -- 172.21.15.86:0/3556191574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0c341a00d0 con 0x7f0c34102a00 2024-04-21T08:38:34.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.480+0000 7f0c2a7fc640 1 -- 172.21.15.86:0/3556191574 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0c2c007d50 con 0x7f0c34102a00 2024-04-21T08:38:34.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.480+0000 7f0c2a7fc640 1 -- 172.21.15.86:0/3556191574 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0c2c004e60 con 0x7f0c34102a00 2024-04-21T08:38:34.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.480+0000 7f0c3b4ba640 1 -- 172.21.15.86:0/3556191574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0c341a05d0 con 0x7f0c34102a00 2024-04-21T08:38:34.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.481+0000 7f0c3b4ba640 1 -- 172.21.15.86:0/3556191574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0c341082a0 con 0x7f0c34102a00 2024-04-21T08:38:34.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.482+0000 7f0c2a7fc640 1 -- 172.21.15.86:0/3556191574 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0c2c005020 con 0x7f0c34102a00 2024-04-21T08:38:34.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.482+0000 7f0c2a7fc640 1 --2- 172.21.15.86:0/3556191574 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f0c200a3bb0 0x7f0c200a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:34.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.483+0000 7f0c39256640 1 --2- 172.21.15.86:0/3556191574 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f0c200a3bb0 0x7f0c200a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:34.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.483+0000 7f0c2a7fc640 1 -- 172.21.15.86:0/3556191574 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f0c2c0d0380 con 0x7f0c34102a00 2024-04-21T08:38:34.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.483+0000 7f0c39256640 1 --2- 172.21.15.86:0/3556191574 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f0c200a3bb0 0x7f0c200a6070 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0c34101bb0 tx=0x7f0c30007600 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:34.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.487+0000 7f0c2a7fc640 1 -- 172.21.15.86:0/3556191574 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f0c2c014030 con 0x7f0c34102a00 2024-04-21T08:38:34.630 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:34 smithi086 ceph-mon[27939]: pgmap v358: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-04-21T08:38:34.630 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:34 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3234210115' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:34.631 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:34 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1059397350' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:34.631 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:34 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2397676700' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:34.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.692+0000 7f0c3b4ba640 1 -- 172.21.15.86:0/3556191574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f0c34062b50 con 0x7f0c34102a00 2024-04-21T08:38:34.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.693+0000 7f0c2a7fc640 1 -- 172.21.15.86:0/3556191574 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f0c2c09ca70 con 0x7f0c34102a00 2024-04-21T08:38:34.692 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-21T08:38:34.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.695+0000 7f0c3b4ba640 1 -- 172.21.15.86:0/3556191574 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f0c200a3bb0 msgr2=0x7f0c200a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:34.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.695+0000 7f0c3b4ba640 1 --2- 172.21.15.86:0/3556191574 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f0c200a3bb0 0x7f0c200a6070 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7f0c34101bb0 tx=0x7f0c30007600 comp rx=0 tx=0).stop 2024-04-21T08:38:34.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.695+0000 7f0c3b4ba640 1 -- 172.21.15.86:0/3556191574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c34102a00 msgr2=0x7f0c3419f770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:34.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.695+0000 7f0c3b4ba640 1 --2- 172.21.15.86:0/3556191574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c34102a00 0x7f0c3419f770 secure :-1 s=READY pgs=433 cs=0 l=1 rev1=1 crypto rx=0x7f0c2c00c970 tx=0x7f0c2c00ce30 comp rx=0 tx=0).stop 2024-04-21T08:38:34.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.695+0000 7f0c3b4ba640 1 -- 172.21.15.86:0/3556191574 shutdown_connections 2024-04-21T08:38:34.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.695+0000 7f0c3b4ba640 1 --2- 172.21.15.86:0/3556191574 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f0c200a3bb0 0x7f0c200a6070 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:34.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.695+0000 7f0c3b4ba640 1 --2- 172.21.15.86:0/3556191574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0c34102a00 0x7f0c3419f770 unknown :-1 s=CLOSED pgs=433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:34.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.695+0000 7f0c3b4ba640 1 --2- 172.21.15.86:0/3556191574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0c34102040 0x7f0c34198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:34.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.695+0000 7f0c3b4ba640 1 --2- 172.21.15.86:0/3556191574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0c34100e60 0x7f0c341981c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:34.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.695+0000 7f0c3b4ba640 1 -- 172.21.15.86:0/3556191574 >> 172.21.15.86:0/3556191574 conn(0x7f0c340fc420 msgr2=0x7f0c34105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:34.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.695+0000 7f0c3b4ba640 1 -- 172.21.15.86:0/3556191574 shutdown_connections 2024-04-21T08:38:34.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.695+0000 7f0c3b4ba640 1 -- 172.21.15.86:0/3556191574 wait complete. 2024-04-21T08:38:34.707 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:38:34.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:34 smithi088 ceph-mon[31259]: pgmap v358: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-04-21T08:38:34.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:34 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3234210115' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:34.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:34 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1059397350' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:34.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:34 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2397676700' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:34.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:34 smithi179 ceph-mon[31829]: pgmap v358: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 5.1 KiB/s rd, 10 op/s 2024-04-21T08:38:34.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:34 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3234210115' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:34.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:34 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1059397350' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:34.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:34 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2397676700' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:34.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.810+0000 7f6544a4a640 1 -- 172.21.15.86:0/336544015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6540103230 msgr2=0x7f65401036b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:34.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.810+0000 7f6544a4a640 1 --2- 172.21.15.86:0/336544015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6540103230 0x7f65401036b0 secure :-1 s=READY pgs=434 cs=0 l=1 rev1=1 crypto rx=0x7f6530009930 tx=0x7f653002bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:34.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.810+0000 7f6544a4a640 1 -- 172.21.15.86:0/336544015 shutdown_connections 2024-04-21T08:38:34.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.810+0000 7f6544a4a640 1 --2- 172.21.15.86:0/336544015 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6540103bf0 0x7f654010a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:34.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.810+0000 7f6544a4a640 1 --2- 172.21.15.86:0/336544015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6540103230 0x7f65401036b0 unknown :-1 s=CLOSED pgs=434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:34.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.810+0000 7f6544a4a640 1 --2- 172.21.15.86:0/336544015 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6540100260 0x7f6540100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:34.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.810+0000 7f6544a4a640 1 -- 172.21.15.86:0/336544015 >> 172.21.15.86:0/336544015 conn(0x7f65400fbb90 msgr2=0x7f65400fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:34.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.810+0000 7f6544a4a640 1 -- 172.21.15.86:0/336544015 shutdown_connections 2024-04-21T08:38:34.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.810+0000 7f6544a4a640 1 -- 172.21.15.86:0/336544015 wait complete. 2024-04-21T08:38:34.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.811+0000 7f6544a4a640 1 Processor -- start 2024-04-21T08:38:34.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.811+0000 7f6544a4a640 1 -- start start 2024-04-21T08:38:34.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.811+0000 7f6544a4a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6540100260 0x7f6540198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:34.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.811+0000 7f6544a4a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6540103230 0x7f6540198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:34.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.811+0000 7f6544a4a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6540103bf0 0x7f654019f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:34.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.811+0000 7f6544a4a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f654010db50 con 0x7f6540100260 2024-04-21T08:38:34.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.811+0000 7f6544a4a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f654010d9d0 con 0x7f6540103bf0 2024-04-21T08:38:34.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.811+0000 7f6544a4a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f654010dcd0 con 0x7f6540103230 2024-04-21T08:38:34.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.811+0000 7f653e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6540100260 0x7f6540198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:34.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.811+0000 7f653e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6540100260 0x7f6540198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60128/0 (socket says 172.21.15.86:60128) 2024-04-21T08:38:34.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.811+0000 7f653e59c640 1 -- 172.21.15.86:0/137473126 learned_addr learned my addr 172.21.15.86:0/137473126 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:34.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.811+0000 7f653dd9b640 1 --2- 172.21.15.86:0/137473126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6540103230 0x7f6540198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:34.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.811+0000 7f653e59c640 1 -- 172.21.15.86:0/137473126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6540103230 msgr2=0x7f6540198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:34.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.811+0000 7f653e59c640 1 --2- 172.21.15.86:0/137473126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6540103230 0x7f6540198790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:34.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.811+0000 7f653e59c640 1 -- 172.21.15.86:0/137473126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6540103bf0 msgr2=0x7f654019f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:34.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.811+0000 7f653ed9d640 1 --2- 172.21.15.86:0/137473126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6540103bf0 0x7f654019f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:34.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.811+0000 7f653e59c640 1 --2- 172.21.15.86:0/137473126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6540103bf0 0x7f654019f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:34.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.811+0000 7f653e59c640 1 -- 172.21.15.86:0/137473126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f654019fe70 con 0x7f6540100260 2024-04-21T08:38:34.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.811+0000 7f653e59c640 1 --2- 172.21.15.86:0/137473126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6540100260 0x7f6540198250 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f653400d950 tx=0x7f653400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:34.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.812+0000 7f652f7fe640 1 -- 172.21.15.86:0/137473126 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6534017020 con 0x7f6540100260 2024-04-21T08:38:34.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.812+0000 7f652f7fe640 1 -- 172.21.15.86:0/137473126 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6534009c30 con 0x7f6540100260 2024-04-21T08:38:34.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.812+0000 7f6544a4a640 1 -- 172.21.15.86:0/137473126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f65401a0160 con 0x7f6540100260 2024-04-21T08:38:34.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.812+0000 7f652f7fe640 1 -- 172.21.15.86:0/137473126 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f653400f6b0 con 0x7f6540100260 2024-04-21T08:38:34.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.812+0000 7f6544a4a640 1 -- 172.21.15.86:0/137473126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f65401a0680 con 0x7f6540100260 2024-04-21T08:38:34.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.813+0000 7f6544a4a640 1 -- 172.21.15.86:0/137473126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f65401093d0 con 0x7f6540100260 2024-04-21T08:38:34.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.814+0000 7f652f7fe640 1 -- 172.21.15.86:0/137473126 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f653401a020 con 0x7f6540100260 2024-04-21T08:38:34.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.815+0000 7f652f7fe640 1 --2- 172.21.15.86:0/137473126 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f65280a3bb0 0x7f65280a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:34.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.815+0000 7f653dd9b640 1 --2- 172.21.15.86:0/137473126 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f65280a3bb0 0x7f65280a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:34.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.815+0000 7f652f7fe640 1 -- 172.21.15.86:0/137473126 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f65340d0380 con 0x7f6540100260 2024-04-21T08:38:34.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.815+0000 7f653dd9b640 1 --2- 172.21.15.86:0/137473126 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f65280a3bb0 0x7f65280a6070 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6530009900 tx=0x7f653002c7e0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:34.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.819+0000 7f652f7fe640 1 -- 172.21.15.86:0/137473126 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6534009de0 con 0x7f6540100260 2024-04-21T08:38:34.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:34.999+0000 7f6544a4a640 1 -- 172.21.15.86:0/137473126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f6540062b80 con 0x7f6540100260 2024-04-21T08:38:34.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.000+0000 7f652f7fe640 1 -- 172.21.15.86:0/137473126 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7f653409ca70 con 0x7f6540100260 2024-04-21T08:38:34.999 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-21T08:38:35.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.002+0000 7f6544a4a640 1 -- 172.21.15.86:0/137473126 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f65280a3bb0 msgr2=0x7f65280a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:35.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.002+0000 7f6544a4a640 1 --2- 172.21.15.86:0/137473126 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f65280a3bb0 0x7f65280a6070 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f6530009900 tx=0x7f653002c7e0 comp rx=0 tx=0).stop 2024-04-21T08:38:35.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.002+0000 7f6544a4a640 1 -- 172.21.15.86:0/137473126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6540100260 msgr2=0x7f6540198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:35.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.002+0000 7f6544a4a640 1 --2- 172.21.15.86:0/137473126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6540100260 0x7f6540198250 secure :-1 s=READY pgs=435 cs=0 l=1 rev1=1 crypto rx=0x7f653400d950 tx=0x7f653400de10 comp rx=0 tx=0).stop 2024-04-21T08:38:35.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.002+0000 7f6544a4a640 1 -- 172.21.15.86:0/137473126 shutdown_connections 2024-04-21T08:38:35.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.002+0000 7f6544a4a640 1 --2- 172.21.15.86:0/137473126 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f65280a3bb0 0x7f65280a6070 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:35.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.002+0000 7f6544a4a640 1 --2- 172.21.15.86:0/137473126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6540103bf0 0x7f654019f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:35.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.002+0000 7f6544a4a640 1 --2- 172.21.15.86:0/137473126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6540103230 0x7f6540198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:35.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.002+0000 7f6544a4a640 1 --2- 172.21.15.86:0/137473126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6540100260 0x7f6540198250 unknown :-1 s=CLOSED pgs=435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:35.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.002+0000 7f6544a4a640 1 -- 172.21.15.86:0/137473126 >> 172.21.15.86:0/137473126 conn(0x7f65400fbb90 msgr2=0x7f65401065a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:35.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.002+0000 7f6544a4a640 1 -- 172.21.15.86:0/137473126 shutdown_connections 2024-04-21T08:38:35.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.002+0000 7f6544a4a640 1 -- 172.21.15.86:0/137473126 wait complete. 2024-04-21T08:38:35.012 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:38:35.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.111+0000 7fb5e5985640 1 -- 172.21.15.86:0/959442965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5e0102a60 msgr2=0x7fb5e0104e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:35.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.111+0000 7fb5e5985640 1 --2- 172.21.15.86:0/959442965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5e0102a60 0x7fb5e0104e50 secure :-1 s=READY pgs=436 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0009900 tx=0x7fb5d002bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:35.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.111+0000 7fb5e5985640 1 -- 172.21.15.86:0/959442965 shutdown_connections 2024-04-21T08:38:35.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.111+0000 7fb5e5985640 1 --2- 172.21.15.86:0/959442965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5e0107cc0 0x7fb5e010a0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:35.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.111+0000 7fb5e5985640 1 --2- 172.21.15.86:0/959442965 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5e0105390 0x7fb5e0107780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:35.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.111+0000 7fb5e5985640 1 --2- 172.21.15.86:0/959442965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5e0102a60 0x7fb5e0104e50 unknown :-1 s=CLOSED pgs=436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:35.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.111+0000 7fb5e5985640 1 -- 172.21.15.86:0/959442965 >> 172.21.15.86:0/959442965 conn(0x7fb5e00fc410 msgr2=0x7fb5e00fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:35.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.111+0000 7fb5e5985640 1 -- 172.21.15.86:0/959442965 shutdown_connections 2024-04-21T08:38:35.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.111+0000 7fb5e5985640 1 -- 172.21.15.86:0/959442965 wait complete. 2024-04-21T08:38:35.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.111+0000 7fb5e5985640 1 Processor -- start 2024-04-21T08:38:35.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.111+0000 7fb5e5985640 1 -- start start 2024-04-21T08:38:35.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.112+0000 7fb5e5985640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5e0102a60 0x7fb5e01981a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:35.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.112+0000 7fb5e5985640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5e0105390 0x7fb5e01986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:35.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.112+0000 7fb5e5985640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5e0107cc0 0x7fb5e019f750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:35.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.112+0000 7fb5e5985640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e010dab0 con 0x7fb5e0102a60 2024-04-21T08:38:35.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.112+0000 7fb5e5985640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e010d930 con 0x7fb5e0105390 2024-04-21T08:38:35.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.112+0000 7fb5e5985640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb5e010dc30 con 0x7fb5e0107cc0 2024-04-21T08:38:35.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.112+0000 7fb5deffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5e0102a60 0x7fb5e01981a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:35.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.112+0000 7fb5deffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5e0102a60 0x7fb5e01981a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60148/0 (socket says 172.21.15.86:60148) 2024-04-21T08:38:35.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.112+0000 7fb5deffd640 1 -- 172.21.15.86:0/2449269031 learned_addr learned my addr 172.21.15.86:0/2449269031 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:35.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.112+0000 7fb5deffd640 1 -- 172.21.15.86:0/2449269031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5e0107cc0 msgr2=0x7fb5e019f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:35.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.112+0000 7fb5deffd640 1 --2- 172.21.15.86:0/2449269031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5e0107cc0 0x7fb5e019f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:35.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.112+0000 7fb5deffd640 1 -- 172.21.15.86:0/2449269031 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5e0105390 msgr2=0x7fb5e01986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:35.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.112+0000 7fb5deffd640 1 --2- 172.21.15.86:0/2449269031 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5e0105390 0x7fb5e01986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:35.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.112+0000 7fb5deffd640 1 -- 172.21.15.86:0/2449269031 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5e019fdc0 con 0x7fb5e0102a60 2024-04-21T08:38:35.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.112+0000 7fb5deffd640 1 --2- 172.21.15.86:0/2449269031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5e0102a60 0x7fb5e01981a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0036040 tx=0x7fb5d00042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:35.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.112+0000 7fb5e4983640 1 -- 172.21.15.86:0/2449269031 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0043020 con 0x7fb5e0102a60 2024-04-21T08:38:35.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.112+0000 7fb5e4983640 1 -- 172.21.15.86:0/2449269031 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5d00054c0 con 0x7fb5e0102a60 2024-04-21T08:38:35.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.112+0000 7fb5e5985640 1 -- 172.21.15.86:0/2449269031 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5e01a0050 con 0x7fb5e0102a60 2024-04-21T08:38:35.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.112+0000 7fb5e4983640 1 -- 172.21.15.86:0/2449269031 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5d0003db0 con 0x7fb5e0102a60 2024-04-21T08:38:35.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.112+0000 7fb5e5985640 1 -- 172.21.15.86:0/2449269031 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5e01a05a0 con 0x7fb5e0102a60 2024-04-21T08:38:35.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.113+0000 7fb5e5985640 1 -- 172.21.15.86:0/2449269031 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb5e00fdf00 con 0x7fb5e0102a60 2024-04-21T08:38:35.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.114+0000 7fb5e4983640 1 -- 172.21.15.86:0/2449269031 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb5d0046020 con 0x7fb5e0102a60 2024-04-21T08:38:35.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.115+0000 7fb5e4983640 1 --2- 172.21.15.86:0/2449269031 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb5cc0a3bb0 0x7fb5cc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:35.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.115+0000 7fb5d7fff640 1 --2- 172.21.15.86:0/2449269031 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb5cc0a3bb0 0x7fb5cc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:35.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.115+0000 7fb5e4983640 1 -- 172.21.15.86:0/2449269031 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fb5d00f30e0 con 0x7fb5e0102a60 2024-04-21T08:38:35.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.115+0000 7fb5d7fff640 1 --2- 172.21.15.86:0/2449269031 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb5cc0a3bb0 0x7fb5cc0a6070 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb5e01996b0 tx=0x7fb5c8005bf0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:35.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.118+0000 7fb5e4983640 1 -- 172.21.15.86:0/2449269031 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb5d004d020 con 0x7fb5e0102a60 2024-04-21T08:38:35.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.334+0000 7fb5e5985640 1 -- 172.21.15.86:0/2449269031 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb5e002d5c0 con 0x7fb5e0102a60 2024-04-21T08:38:35.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.335+0000 7fb5e4983640 1 -- 172.21.15.86:0/2449269031 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7fb5d00bf730 con 0x7fb5e0102a60 2024-04-21T08:38:35.334 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:38:35.334 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:36:47.458637+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24455,24472,24394,24398,24482]},"id":1}]} 2024-04-21T08:38:35.334 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-21T08:38:35.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.336+0000 7fb5e5985640 1 -- 172.21.15.86:0/2449269031 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb5cc0a3bb0 msgr2=0x7fb5cc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:35.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.336+0000 7fb5e5985640 1 --2- 172.21.15.86:0/2449269031 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb5cc0a3bb0 0x7fb5cc0a6070 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7fb5e01996b0 tx=0x7fb5c8005bf0 comp rx=0 tx=0).stop 2024-04-21T08:38:35.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.336+0000 7fb5e5985640 1 -- 172.21.15.86:0/2449269031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5e0102a60 msgr2=0x7fb5e01981a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:35.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.336+0000 7fb5e5985640 1 --2- 172.21.15.86:0/2449269031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5e0102a60 0x7fb5e01981a0 secure :-1 s=READY pgs=437 cs=0 l=1 rev1=1 crypto rx=0x7fb5d0036040 tx=0x7fb5d00042f0 comp rx=0 tx=0).stop 2024-04-21T08:38:35.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.337+0000 7fb5e5985640 1 -- 172.21.15.86:0/2449269031 shutdown_connections 2024-04-21T08:38:35.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.337+0000 7fb5e5985640 1 --2- 172.21.15.86:0/2449269031 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb5cc0a3bb0 0x7fb5cc0a6070 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:35.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.337+0000 7fb5e5985640 1 --2- 172.21.15.86:0/2449269031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5e0107cc0 0x7fb5e019f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:35.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.337+0000 7fb5e5985640 1 --2- 172.21.15.86:0/2449269031 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5e0105390 0x7fb5e01986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:35.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.337+0000 7fb5e5985640 1 --2- 172.21.15.86:0/2449269031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5e0102a60 0x7fb5e01981a0 unknown :-1 s=CLOSED pgs=437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:35.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.337+0000 7fb5e5985640 1 -- 172.21.15.86:0/2449269031 >> 172.21.15.86:0/2449269031 conn(0x7fb5e00fc410 msgr2=0x7fb5e01033c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:35.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.337+0000 7fb5e5985640 1 -- 172.21.15.86:0/2449269031 shutdown_connections 2024-04-21T08:38:35.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.337+0000 7fb5e5985640 1 -- 172.21.15.86:0/2449269031 wait complete. 2024-04-21T08:38:35.348 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:38:35.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.448+0000 7f4edd243640 1 -- 172.21.15.86:0/2217947531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ed8101fa0 msgr2=0x7f4ed81023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:35.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.448+0000 7f4edd243640 1 --2- 172.21.15.86:0/2217947531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ed8101fa0 0x7f4ed81023c0 secure :-1 s=READY pgs=438 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8009900 tx=0x7f4ec802bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:35.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.449+0000 7f4edd243640 1 -- 172.21.15.86:0/2217947531 shutdown_connections 2024-04-21T08:38:35.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.449+0000 7f4edd243640 1 --2- 172.21.15.86:0/2217947531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ed8103b50 0x7f4ed810a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:35.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.449+0000 7f4edd243640 1 --2- 172.21.15.86:0/2217947531 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ed8103190 0x7f4ed8103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:35.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.449+0000 7f4edd243640 1 --2- 172.21.15.86:0/2217947531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ed8101fa0 0x7f4ed81023c0 unknown :-1 s=CLOSED pgs=438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:35.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.449+0000 7f4edd243640 1 -- 172.21.15.86:0/2217947531 >> 172.21.15.86:0/2217947531 conn(0x7f4ed80fd530 msgr2=0x7f4ed80ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:35.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.449+0000 7f4edd243640 1 -- 172.21.15.86:0/2217947531 shutdown_connections 2024-04-21T08:38:35.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.449+0000 7f4edd243640 1 -- 172.21.15.86:0/2217947531 wait complete. 2024-04-21T08:38:35.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.449+0000 7f4edd243640 1 Processor -- start 2024-04-21T08:38:35.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.449+0000 7f4edd243640 1 -- start start 2024-04-21T08:38:35.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.450+0000 7f4edd243640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ed8101fa0 0x7f4ed8198250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:35.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.450+0000 7f4edd243640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ed8103190 0x7f4ed8198790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:35.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.450+0000 7f4edd243640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ed8103b50 0x7f4ed819f800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:35.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.450+0000 7f4edd243640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed810db20 con 0x7f4ed8101fa0 2024-04-21T08:38:35.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.450+0000 7f4edd243640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed810d9a0 con 0x7f4ed8103190 2024-04-21T08:38:35.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.450+0000 7f4edd243640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4ed810dca0 con 0x7f4ed8103b50 2024-04-21T08:38:35.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.450+0000 7f4ed6d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ed8101fa0 0x7f4ed8198250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:35.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.450+0000 7f4ed6d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ed8101fa0 0x7f4ed8198250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60158/0 (socket says 172.21.15.86:60158) 2024-04-21T08:38:35.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.450+0000 7f4ed6d9d640 1 -- 172.21.15.86:0/990580321 learned_addr learned my addr 172.21.15.86:0/990580321 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:35.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.450+0000 7f4ed759e640 1 --2- 172.21.15.86:0/990580321 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ed8103b50 0x7f4ed819f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:35.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.450+0000 7f4ed659c640 1 --2- 172.21.15.86:0/990580321 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ed8103190 0x7f4ed8198790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:35.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.450+0000 7f4ed6d9d640 1 -- 172.21.15.86:0/990580321 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ed8103b50 msgr2=0x7f4ed819f800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:35.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.450+0000 7f4ed6d9d640 1 --2- 172.21.15.86:0/990580321 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ed8103b50 0x7f4ed819f800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:35.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.450+0000 7f4ed6d9d640 1 -- 172.21.15.86:0/990580321 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ed8103190 msgr2=0x7f4ed8198790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:35.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.450+0000 7f4ed6d9d640 1 --2- 172.21.15.86:0/990580321 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ed8103190 0x7f4ed8198790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:35.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.450+0000 7f4ed6d9d640 1 -- 172.21.15.86:0/990580321 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ed819fe70 con 0x7f4ed8101fa0 2024-04-21T08:38:35.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.450+0000 7f4ed6d9d640 1 --2- 172.21.15.86:0/990580321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ed8101fa0 0x7f4ed8198250 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8036040 tx=0x7f4ec80042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:35.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.450+0000 7f4ebffff640 1 -- 172.21.15.86:0/990580321 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec8043020 con 0x7f4ed8101fa0 2024-04-21T08:38:35.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.450+0000 7f4ebffff640 1 -- 172.21.15.86:0/990580321 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ec8004aa0 con 0x7f4ed8101fa0 2024-04-21T08:38:35.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.450+0000 7f4edd243640 1 -- 172.21.15.86:0/990580321 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ed81a0100 con 0x7f4ed8101fa0 2024-04-21T08:38:35.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.450+0000 7f4ebffff640 1 -- 172.21.15.86:0/990580321 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ec8003de0 con 0x7f4ed8101fa0 2024-04-21T08:38:35.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.450+0000 7f4edd243640 1 -- 172.21.15.86:0/990580321 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ed81a06d0 con 0x7f4ed8101fa0 2024-04-21T08:38:35.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.451+0000 7f4edd243640 1 -- 172.21.15.86:0/990580321 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ed81092e0 con 0x7f4ed8101fa0 2024-04-21T08:38:35.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.452+0000 7f4ebffff640 1 -- 172.21.15.86:0/990580321 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4ec8046020 con 0x7f4ed8101fa0 2024-04-21T08:38:35.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.453+0000 7f4ebffff640 1 --2- 172.21.15.86:0/990580321 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f4ec00a3bb0 0x7f4ec00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:35.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.453+0000 7f4ed659c640 1 --2- 172.21.15.86:0/990580321 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f4ec00a3bb0 0x7f4ec00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:35.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.453+0000 7f4ebffff640 1 -- 172.21.15.86:0/990580321 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f4ec80f3040 con 0x7f4ed8101fa0 2024-04-21T08:38:35.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.453+0000 7f4ed659c640 1 --2- 172.21.15.86:0/990580321 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f4ec00a3bb0 0x7f4ec00a6070 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8199760 tx=0x7f4ecc008040 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:35.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.457+0000 7f4ebffff640 1 -- 172.21.15.86:0/990580321 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4ec804d020 con 0x7f4ed8101fa0 2024-04-21T08:38:35.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:35 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3556191574' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:35.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:35 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/137473126' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:35.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:35 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2449269031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:35.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.638+0000 7f4edd243640 1 -- 172.21.15.86:0/990580321 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f4ed8062b50 con 0x7f4ed8101fa0 2024-04-21T08:38:35.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.639+0000 7f4ebffff640 1 -- 172.21.15.86:0/990580321 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24689 (secure 0 0 0) 0x7f4ec80bf690 con 0x7f4ed8101fa0 2024-04-21T08:38:35.639 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:38:35.639 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":93,"fsid":"f8cec7cc-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:22.687329+0000","modified":"2024-04-21T08:38:24.484832+0000","last_up_change":"2024-04-21T08:35:30.048315+0000","last_in_change":"2024-04-21T08:35:10.099496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:45.279109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:18.300782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:19.574636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:22.905632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89b7601f-0620-4db2-90f1-b45cea173738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6803","nonce":2571516897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6805","nonce":2571516897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6809","nonce":2571516897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6807","nonce":2571516897}]},"public_addr":"172.21.15.86:6803/2571516897","cluster_addr":"172.21.15.86:6805/2571516897","heartbeat_back_addr":"172.21.15.86:6809/2571516897","heartbeat_front_addr":"172.21.15.86:6807/2571516897","state":["exists","up"]},{"osd":1,"uuid":"4891031c-f98c-42bc-bb3b-a420855ecc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6801","nonce":74067182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6803","nonce":74067182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6807","nonce":74067182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6805","nonce":74067182}]},"public_addr":"172.21.15.88:6801/74067182","cluster_addr":"172.21.15.88:6803/74067182","heartbeat_back_addr":"172.21.15.88:6807/74067182","heartbeat_front_addr":"172.21.15.88:6805/74067182","state":["exists","up"]},{"osd":2,"uuid":"60d12aa4-9312-4fa2-8d0b-2b3f42caad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6801","nonce":4137659051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6803","nonce":4137659051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6807","nonce":4137659051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6805","nonce":4137659051}]},"public_addr":"172.21.15.179:6801/4137659051","cluster_addr":"172.21.15.179:6803/4137659051","heartbeat_back_addr":"172.21.15.179:6807/4137659051","heartbeat_front_addr":"172.21.15.179:6805/4137659051","state":["exists","up"]},{"osd":3,"uuid":"404c3158-8cb0-4c02-b48a-f9ee5ff7b784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6811","nonce":2309603148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6813","nonce":2309603148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6817","nonce":2309603148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6815","nonce":2309603148}]},"public_addr":"172.21.15.86:6811/2309603148","cluster_addr":"172.21.15.86:6813/2309603148","heartbeat_back_addr":"172.21.15.86:6817/2309603148","heartbeat_front_addr":"172.21.15.86:6815/2309603148","state":["exists","up"]},{"osd":4,"uuid":"458ff202-d781-4ec7-ac93-0b49f835b503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6809","nonce":1818714217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6811","nonce":1818714217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6815","nonce":1818714217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6813","nonce":1818714217}]},"public_addr":"172.21.15.88:6809/1818714217","cluster_addr":"172.21.15.88:6811/1818714217","heartbeat_back_addr":"172.21.15.88:6815/1818714217","heartbeat_front_addr":"172.21.15.88:6813/1818714217","state":["exists","up"]},{"osd":5,"uuid":"8123cc90-bd8b-4204-bf31-46b34e9f589d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6809","nonce":863840990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6811","nonce":863840990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6815","nonce":863840990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6813","nonce":863840990}]},"public_addr":"172.21.15.179:6809/863840990","cluster_addr":"172.21.15.179:6811/863840990","heartbeat_back_addr":"172.21.15.179:6815/863840990","heartbeat_front_addr":"172.21.15.179:6813/863840990","state":["exists","up"]},{"osd":6,"uuid":"aac9d7c7-659a-4110-903a-a871c086066f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6819","nonce":1224231118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6821","nonce":1224231118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6825","nonce":1224231118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6823","nonce":1224231118}]},"public_addr":"172.21.15.86:6819/1224231118","cluster_addr":"172.21.15.86:6821/1224231118","heartbeat_back_addr":"172.21.15.86:6825/1224231118","heartbeat_front_addr":"172.21.15.86:6823/1224231118","state":["exists","up"]},{"osd":7,"uuid":"df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6817","nonce":136629878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6819","nonce":136629878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6823","nonce":136629878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6821","nonce":136629878}]},"public_addr":"172.21.15.88:6817/136629878","cluster_addr":"172.21.15.88:6819/136629878","heartbeat_back_addr":"172.21.15.88:6823/136629878","heartbeat_front_addr":"172.21.15.88:6821/136629878","state":["exists","up"]},{"osd":8,"uuid":"63e59f4b-0426-46db-92f7-c55dae61b5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6817","nonce":1770418957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6819","nonce":1770418957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6823","nonce":1770418957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6821","nonce":1770418957}]},"public_addr":"172.21.15.179:6817/1770418957","cluster_addr":"172.21.15.179:6819/1770418957","heartbeat_back_addr":"172.21.15.179:6823/1770418957","heartbeat_front_addr":"172.21.15.179:6821/1770418957","state":["exists","up"]},{"osd":9,"uuid":"5841bb36-ed6b-45d8-b857-eeab139c44e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6827","nonce":1108500274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6829","nonce":1108500274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6833","nonce":1108500274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6831","nonce":1108500274}]},"public_addr":"172.21.15.86:6827/1108500274","cluster_addr":"172.21.15.86:6829/1108500274","heartbeat_back_addr":"172.21.15.86:6833/1108500274","heartbeat_front_addr":"172.21.15.86:6831/1108500274","state":["exists","up"]},{"osd":10,"uuid":"f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6825","nonce":1294275353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6827","nonce":1294275353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6831","nonce":1294275353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6829","nonce":1294275353}]},"public_addr":"172.21.15.88:6825/1294275353","cluster_addr":"172.21.15.88:6827/1294275353","heartbeat_back_addr":"172.21.15.88:6831/1294275353","heartbeat_front_addr":"172.21.15.88:6829/1294275353","state":["exists","up"]},{"osd":11,"uuid":"e0d4cc76-e7a6-4a6b-909c-95488ccd5a36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6825","nonce":4007070233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6827","nonce":4007070233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6831","nonce":4007070233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6829","nonce":4007070233}]},"public_addr":"172.21.15.179:6825/4007070233","cluster_addr":"172.21.15.179:6827/4007070233","heartbeat_back_addr":"172.21.15.179:6831/4007070233","heartbeat_front_addr":"172.21.15.179:6829/4007070233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:55.210605+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:18.723090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:42.547979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:06.421622+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:30.858315+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:55.455313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.623369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:46.185111+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.103627+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:37.297545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:02.637431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:28.199292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6800/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/1412393225":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6800/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/1687852404":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2020707404":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2568130933":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/3952607648":"2024-04-22T08:28:57.384403+0000","172.21.15.86:6801/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/2756903426":"2024-04-22T08:29:23.190001+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:38:35.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.642+0000 7f4edd243640 1 -- 172.21.15.86:0/990580321 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f4ec00a3bb0 msgr2=0x7f4ec00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:35.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.642+0000 7f4edd243640 1 --2- 172.21.15.86:0/990580321 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f4ec00a3bb0 0x7f4ec00a6070 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7f4ed8199760 tx=0x7f4ecc008040 comp rx=0 tx=0).stop 2024-04-21T08:38:35.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.642+0000 7f4edd243640 1 -- 172.21.15.86:0/990580321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ed8101fa0 msgr2=0x7f4ed8198250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:35.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.642+0000 7f4edd243640 1 --2- 172.21.15.86:0/990580321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ed8101fa0 0x7f4ed8198250 secure :-1 s=READY pgs=439 cs=0 l=1 rev1=1 crypto rx=0x7f4ec8036040 tx=0x7f4ec80042f0 comp rx=0 tx=0).stop 2024-04-21T08:38:35.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.642+0000 7f4edd243640 1 -- 172.21.15.86:0/990580321 shutdown_connections 2024-04-21T08:38:35.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.642+0000 7f4edd243640 1 --2- 172.21.15.86:0/990580321 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f4ec00a3bb0 0x7f4ec00a6070 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:35.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.642+0000 7f4edd243640 1 --2- 172.21.15.86:0/990580321 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ed8103b50 0x7f4ed819f800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:35.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.642+0000 7f4edd243640 1 --2- 172.21.15.86:0/990580321 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ed8103190 0x7f4ed8198790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:35.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.642+0000 7f4edd243640 1 --2- 172.21.15.86:0/990580321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ed8101fa0 0x7f4ed8198250 unknown :-1 s=CLOSED pgs=439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:35.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.642+0000 7f4edd243640 1 -- 172.21.15.86:0/990580321 >> 172.21.15.86:0/990580321 conn(0x7f4ed80fd530 msgr2=0x7f4ed81064b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:35.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.642+0000 7f4edd243640 1 -- 172.21.15.86:0/990580321 shutdown_connections 2024-04-21T08:38:35.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.642+0000 7f4edd243640 1 -- 172.21.15.86:0/990580321 wait complete. 2024-04-21T08:38:35.653 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:31:45.279109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:36:18.300782+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:36:19.574636+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:36:22.905632+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:38:35.653 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:38:35.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:35 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3556191574' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:35.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:35 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/137473126' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:35.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:35 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2449269031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:35.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:35 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3556191574' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:35.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:35 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/137473126' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:35.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:35 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2449269031' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.755+0000 7f4c91bc7640 1 -- 172.21.15.86:0/1308424358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c8c100620 msgr2=0x7f4c8c100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.755+0000 7f4c91bc7640 1 --2- 172.21.15.86:0/1308424358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c8c100620 0x7f4c8c100a40 secure :-1 s=READY pgs=440 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c009900 tx=0x7f4c7c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.755+0000 7f4c91bc7640 1 -- 172.21.15.86:0/1308424358 shutdown_connections 2024-04-21T08:38:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.755+0000 7f4c91bc7640 1 --2- 172.21.15.86:0/1308424358 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4c8c1021d0 0x7f4c8c108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.755+0000 7f4c91bc7640 1 --2- 172.21.15.86:0/1308424358 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c8c101810 0x7f4c8c101c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.755+0000 7f4c91bc7640 1 --2- 172.21.15.86:0/1308424358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c8c100620 0x7f4c8c100a40 unknown :-1 s=CLOSED pgs=440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.755+0000 7f4c91bc7640 1 -- 172.21.15.86:0/1308424358 >> 172.21.15.86:0/1308424358 conn(0x7f4c8c0fbb90 msgr2=0x7f4c8c0fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.755+0000 7f4c91bc7640 1 -- 172.21.15.86:0/1308424358 shutdown_connections 2024-04-21T08:38:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.755+0000 7f4c91bc7640 1 -- 172.21.15.86:0/1308424358 wait complete. 2024-04-21T08:38:35.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.756+0000 7f4c91bc7640 1 Processor -- start 2024-04-21T08:38:35.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.756+0000 7f4c91bc7640 1 -- start start 2024-04-21T08:38:35.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.756+0000 7f4c91bc7640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c8c100620 0x7f4c8c198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:35.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.756+0000 7f4c91bc7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c8c101810 0x7f4c8c198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:35.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.756+0000 7f4c91bc7640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4c8c1021d0 0x7f4c8c19f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:35.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.756+0000 7f4c91bc7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c10c1a0 con 0x7f4c8c101810 2024-04-21T08:38:35.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.756+0000 7f4c91bc7640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c10c020 con 0x7f4c8c1021d0 2024-04-21T08:38:35.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.756+0000 7f4c91bc7640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4c8c10c320 con 0x7f4c8c100620 2024-04-21T08:38:35.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.756+0000 7f4c8affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c8c101810 0x7f4c8c198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:35.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.756+0000 7f4c8affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c8c101810 0x7f4c8c198770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60178/0 (socket says 172.21.15.86:60178) 2024-04-21T08:38:35.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.756+0000 7f4c8affd640 1 -- 172.21.15.86:0/607224544 learned_addr learned my addr 172.21.15.86:0/607224544 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:35.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.756+0000 7f4c8bfff640 1 --2- 172.21.15.86:0/607224544 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4c8c1021d0 0x7f4c8c19f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:35.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.756+0000 7f4c8affd640 1 -- 172.21.15.86:0/607224544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c8c100620 msgr2=0x7f4c8c198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:35.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.756+0000 7f4c8affd640 1 --2- 172.21.15.86:0/607224544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c8c100620 0x7f4c8c198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:35.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.756+0000 7f4c8affd640 1 -- 172.21.15.86:0/607224544 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4c8c1021d0 msgr2=0x7f4c8c19f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:35.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.756+0000 7f4c8affd640 1 --2- 172.21.15.86:0/607224544 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4c8c1021d0 0x7f4c8c19f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:35.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.756+0000 7f4c8affd640 1 -- 172.21.15.86:0/607224544 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4c8c19fe00 con 0x7f4c8c101810 2024-04-21T08:38:35.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.757+0000 7f4c8affd640 1 --2- 172.21.15.86:0/607224544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c8c101810 0x7f4c8c198770 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4c8000d770 tx=0x7f4c8000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:35.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.757+0000 7f4c88ff9640 1 -- 172.21.15.86:0/607224544 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c80017070 con 0x7f4c8c101810 2024-04-21T08:38:35.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.757+0000 7f4c88ff9640 1 -- 172.21.15.86:0/607224544 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4c800054d0 con 0x7f4c8c101810 2024-04-21T08:38:35.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.757+0000 7f4c91bc7640 1 -- 172.21.15.86:0/607224544 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4c8c1a00f0 con 0x7f4c8c101810 2024-04-21T08:38:35.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.757+0000 7f4c88ff9640 1 -- 172.21.15.86:0/607224544 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4c80004c30 con 0x7f4c8c101810 2024-04-21T08:38:35.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.757+0000 7f4c91bc7640 1 -- 172.21.15.86:0/607224544 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4c8c1a05f0 con 0x7f4c8c101810 2024-04-21T08:38:35.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.758+0000 7f4c91bc7640 1 -- 172.21.15.86:0/607224544 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4c8c1a0850 con 0x7f4c8c101810 2024-04-21T08:38:35.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.759+0000 7f4c88ff9640 1 -- 172.21.15.86:0/607224544 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4c80004e10 con 0x7f4c8c101810 2024-04-21T08:38:35.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.759+0000 7f4c88ff9640 1 --2- 172.21.15.86:0/607224544 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f4c740a3bb0 0x7f4c740a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:35.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.759+0000 7f4c8b7fe640 1 --2- 172.21.15.86:0/607224544 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f4c740a3bb0 0x7f4c740a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:35.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.759+0000 7f4c88ff9640 1 -- 172.21.15.86:0/607224544 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f4c800d0250 con 0x7f4c8c101810 2024-04-21T08:38:35.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.760+0000 7f4c8b7fe640 1 --2- 172.21.15.86:0/607224544 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f4c740a3bb0 0x7f4c740a6070 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c005c20 tx=0x7f4c7c000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:35.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.764+0000 7f4c88ff9640 1 -- 172.21.15.86:0/607224544 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4c8009c8a0 con 0x7f4c8c101810 2024-04-21T08:38:35.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.942+0000 7f4c91bc7640 1 -- 172.21.15.86:0/607224544 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f4c8c1a0a20 con 0x7f4c8c101810 2024-04-21T08:38:35.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.943+0000 7f4c88ff9640 1 -- 172.21.15.86:0/607224544 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7f4c8009c1a0 con 0x7f4c8c101810 2024-04-21T08:38:35.942 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-21T08:38:35.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.945+0000 7f4c91bc7640 1 -- 172.21.15.86:0/607224544 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f4c740a3bb0 msgr2=0x7f4c740a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:35.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.945+0000 7f4c91bc7640 1 --2- 172.21.15.86:0/607224544 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f4c740a3bb0 0x7f4c740a6070 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f4c7c005c20 tx=0x7f4c7c000bc0 comp rx=0 tx=0).stop 2024-04-21T08:38:35.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.945+0000 7f4c91bc7640 1 -- 172.21.15.86:0/607224544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c8c101810 msgr2=0x7f4c8c198770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:35.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.945+0000 7f4c91bc7640 1 --2- 172.21.15.86:0/607224544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c8c101810 0x7f4c8c198770 secure :-1 s=READY pgs=441 cs=0 l=1 rev1=1 crypto rx=0x7f4c8000d770 tx=0x7f4c8000dc30 comp rx=0 tx=0).stop 2024-04-21T08:38:35.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.945+0000 7f4c91bc7640 1 -- 172.21.15.86:0/607224544 shutdown_connections 2024-04-21T08:38:35.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.945+0000 7f4c91bc7640 1 --2- 172.21.15.86:0/607224544 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f4c740a3bb0 0x7f4c740a6070 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:35.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.945+0000 7f4c91bc7640 1 --2- 172.21.15.86:0/607224544 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4c8c1021d0 0x7f4c8c19f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:35.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.945+0000 7f4c91bc7640 1 --2- 172.21.15.86:0/607224544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4c8c101810 0x7f4c8c198770 unknown :-1 s=CLOSED pgs=441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:35.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.945+0000 7f4c91bc7640 1 --2- 172.21.15.86:0/607224544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4c8c100620 0x7f4c8c198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:35.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.945+0000 7f4c91bc7640 1 -- 172.21.15.86:0/607224544 >> 172.21.15.86:0/607224544 conn(0x7f4c8c0fbb90 msgr2=0x7f4c8c0fef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:35.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.945+0000 7f4c91bc7640 1 -- 172.21.15.86:0/607224544 shutdown_connections 2024-04-21T08:38:35.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:35.945+0000 7f4c91bc7640 1 -- 172.21.15.86:0/607224544 wait complete. 2024-04-21T08:38:35.956 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:38:36.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.057+0000 7f38eb606640 1 -- 172.21.15.86:0/509494611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e4101810 msgr2=0x7f38e4101c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:36.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.057+0000 7f38eb606640 1 --2- 172.21.15.86:0/509494611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e4101810 0x7f38e4101c90 secure :-1 s=READY pgs=442 cs=0 l=1 rev1=1 crypto rx=0x7f38d400aa10 tx=0x7f38d402cf70 comp rx=0 tx=0).stop 2024-04-21T08:38:36.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.057+0000 7f38eb606640 1 -- 172.21.15.86:0/509494611 shutdown_connections 2024-04-21T08:38:36.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.057+0000 7f38eb606640 1 --2- 172.21.15.86:0/509494611 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38e41021d0 0x7f38e4108a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.057+0000 7f38eb606640 1 --2- 172.21.15.86:0/509494611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e4101810 0x7f38e4101c90 unknown :-1 s=CLOSED pgs=442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.057+0000 7f38eb606640 1 --2- 172.21.15.86:0/509494611 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38e4100620 0x7f38e4100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.057+0000 7f38eb606640 1 -- 172.21.15.86:0/509494611 >> 172.21.15.86:0/509494611 conn(0x7f38e40fbb90 msgr2=0x7f38e40fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:36.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.057+0000 7f38eb606640 1 -- 172.21.15.86:0/509494611 shutdown_connections 2024-04-21T08:38:36.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.057+0000 7f38eb606640 1 -- 172.21.15.86:0/509494611 wait complete. 2024-04-21T08:38:36.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.058+0000 7f38eb606640 1 Processor -- start 2024-04-21T08:38:36.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.058+0000 7f38eb606640 1 -- start start 2024-04-21T08:38:36.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.058+0000 7f38eb606640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38e4100620 0x7f38e4196080 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:36.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.058+0000 7f38eb606640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e4101810 0x7f38e41965c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:36.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.058+0000 7f38eb606640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38e41021d0 0x7f38e419d5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:36.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.058+0000 7f38eb606640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f38e410b920 con 0x7f38e4101810 2024-04-21T08:38:36.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.058+0000 7f38eb606640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f38e410b7a0 con 0x7f38e41021d0 2024-04-21T08:38:36.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.058+0000 7f38eb606640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f38e410baa0 con 0x7f38e4100620 2024-04-21T08:38:36.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.058+0000 7f38e8ba1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e4101810 0x7f38e41965c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:36.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.058+0000 7f38e8ba1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e4101810 0x7f38e41965c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60206/0 (socket says 172.21.15.86:60206) 2024-04-21T08:38:36.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.058+0000 7f38e8ba1640 1 -- 172.21.15.86:0/2257403535 learned_addr learned my addr 172.21.15.86:0/2257403535 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:36.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.058+0000 7f38e93a2640 1 --2- 172.21.15.86:0/2257403535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38e4100620 0x7f38e4196080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:36.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.058+0000 7f38e9ba3640 1 --2- 172.21.15.86:0/2257403535 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38e41021d0 0x7f38e419d5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:36.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.058+0000 7f38e8ba1640 1 -- 172.21.15.86:0/2257403535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38e4100620 msgr2=0x7f38e4196080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:36.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.058+0000 7f38e8ba1640 1 --2- 172.21.15.86:0/2257403535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38e4100620 0x7f38e4196080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.058+0000 7f38e8ba1640 1 -- 172.21.15.86:0/2257403535 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38e41021d0 msgr2=0x7f38e419d5e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:36.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.058+0000 7f38e8ba1640 1 --2- 172.21.15.86:0/2257403535 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38e41021d0 0x7f38e419d5e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.058+0000 7f38e8ba1640 1 -- 172.21.15.86:0/2257403535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38e419dc50 con 0x7f38e4101810 2024-04-21T08:38:36.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.058+0000 7f38e93a2640 1 --2- 172.21.15.86:0/2257403535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38e4100620 0x7f38e4196080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:36.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.058+0000 7f38e9ba3640 1 --2- 172.21.15.86:0/2257403535 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38e41021d0 0x7f38e419d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:36.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.058+0000 7f38e8ba1640 1 --2- 172.21.15.86:0/2257403535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e4101810 0x7f38e41965c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f38d400a340 tx=0x7f38d4004970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:36.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.059+0000 7f38da7fc640 1 -- 172.21.15.86:0/2257403535 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d4043020 con 0x7f38e4101810 2024-04-21T08:38:36.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.059+0000 7f38da7fc640 1 -- 172.21.15.86:0/2257403535 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38d4035c00 con 0x7f38e4101810 2024-04-21T08:38:36.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.059+0000 7f38eb606640 1 -- 172.21.15.86:0/2257403535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38e419dee0 con 0x7f38e4101810 2024-04-21T08:38:36.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.059+0000 7f38da7fc640 1 -- 172.21.15.86:0/2257403535 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38d4003850 con 0x7f38e4101810 2024-04-21T08:38:36.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.059+0000 7f38eb606640 1 -- 172.21.15.86:0/2257403535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38e419e420 con 0x7f38e4101810 2024-04-21T08:38:36.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.060+0000 7f38eb606640 1 -- 172.21.15.86:0/2257403535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f38e4107a60 con 0x7f38e4101810 2024-04-21T08:38:36.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.061+0000 7f38da7fc640 1 -- 172.21.15.86:0/2257403535 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f38d4046020 con 0x7f38e4101810 2024-04-21T08:38:36.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.061+0000 7f38da7fc640 1 --2- 172.21.15.86:0/2257403535 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f38d00a3bb0 0x7f38d00a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:36.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.061+0000 7f38e93a2640 1 --2- 172.21.15.86:0/2257403535 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f38d00a3bb0 0x7f38d00a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:36.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.061+0000 7f38da7fc640 1 -- 172.21.15.86:0/2257403535 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f38d40f4220 con 0x7f38e4101810 2024-04-21T08:38:36.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.062+0000 7f38e93a2640 1 --2- 172.21.15.86:0/2257403535 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f38d00a3bb0 0x7f38d00a6070 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f38e4101670 tx=0x7f38e0005e70 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:36.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.065+0000 7f38da7fc640 1 -- 172.21.15.86:0/2257403535 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f38d404d020 con 0x7f38e4101810 2024-04-21T08:38:36.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.239+0000 7f38eb606640 1 -- 172.21.15.86:0/2257403535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f38e402d750 con 0x7f38e4101810 2024-04-21T08:38:36.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.240+0000 7f38da7fc640 1 -- 172.21.15.86:0/2257403535 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f38d40c0990 con 0x7f38e4101810 2024-04-21T08:38:36.239 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-21T08:38:36.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.241+0000 7f38eb606640 1 -- 172.21.15.86:0/2257403535 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f38d00a3bb0 msgr2=0x7f38d00a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:36.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.241+0000 7f38eb606640 1 --2- 172.21.15.86:0/2257403535 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f38d00a3bb0 0x7f38d00a6070 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7f38e4101670 tx=0x7f38e0005e70 comp rx=0 tx=0).stop 2024-04-21T08:38:36.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.241+0000 7f38eb606640 1 -- 172.21.15.86:0/2257403535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e4101810 msgr2=0x7f38e41965c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:36.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.241+0000 7f38eb606640 1 --2- 172.21.15.86:0/2257403535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e4101810 0x7f38e41965c0 secure :-1 s=READY pgs=443 cs=0 l=1 rev1=1 crypto rx=0x7f38d400a340 tx=0x7f38d4004970 comp rx=0 tx=0).stop 2024-04-21T08:38:36.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.241+0000 7f38eb606640 1 -- 172.21.15.86:0/2257403535 shutdown_connections 2024-04-21T08:38:36.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.242+0000 7f38eb606640 1 --2- 172.21.15.86:0/2257403535 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f38d00a3bb0 0x7f38d00a6070 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.242+0000 7f38eb606640 1 --2- 172.21.15.86:0/2257403535 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38e41021d0 0x7f38e419d5e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.242+0000 7f38eb606640 1 --2- 172.21.15.86:0/2257403535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38e4101810 0x7f38e41965c0 unknown :-1 s=CLOSED pgs=443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.242+0000 7f38eb606640 1 --2- 172.21.15.86:0/2257403535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38e4100620 0x7f38e4196080 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.242+0000 7f38eb606640 1 -- 172.21.15.86:0/2257403535 >> 172.21.15.86:0/2257403535 conn(0x7f38e40fbb90 msgr2=0x7f38e4104c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:36.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.242+0000 7f38eb606640 1 -- 172.21.15.86:0/2257403535 shutdown_connections 2024-04-21T08:38:36.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.242+0000 7f38eb606640 1 -- 172.21.15.86:0/2257403535 wait complete. 2024-04-21T08:38:36.252 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:38:36.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.355+0000 7f39b4bce640 1 -- 172.21.15.86:0/3612821971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39b0102a00 msgr2=0x7f39b0109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:36.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.355+0000 7f39b4bce640 1 --2- 172.21.15.86:0/3612821971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39b0102a00 0x7f39b0109320 secure :-1 s=READY pgs=444 cs=0 l=1 rev1=1 crypto rx=0x7f39a8009ca0 tx=0x7f39a802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.355+0000 7f39b4bce640 1 -- 172.21.15.86:0/3612821971 shutdown_connections 2024-04-21T08:38:36.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.355+0000 7f39b4bce640 1 --2- 172.21.15.86:0/3612821971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39b0102a00 0x7f39b0109320 unknown :-1 s=CLOSED pgs=444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.355+0000 7f39b4bce640 1 --2- 172.21.15.86:0/3612821971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39b0102040 0x7f39b01024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.355+0000 7f39b4bce640 1 --2- 172.21.15.86:0/3612821971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39b0100e60 0x7f39b0101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.355+0000 7f39b4bce640 1 -- 172.21.15.86:0/3612821971 >> 172.21.15.86:0/3612821971 conn(0x7f39b00fc440 msgr2=0x7f39b00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:36.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.355+0000 7f39b4bce640 1 -- 172.21.15.86:0/3612821971 shutdown_connections 2024-04-21T08:38:36.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.355+0000 7f39b4bce640 1 -- 172.21.15.86:0/3612821971 wait complete. 2024-04-21T08:38:36.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.356+0000 7f39b4bce640 1 Processor -- start 2024-04-21T08:38:36.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.356+0000 7f39b4bce640 1 -- start start 2024-04-21T08:38:36.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.356+0000 7f39b4bce640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39b0100e60 0x7f39b01970b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:36.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.356+0000 7f39b4bce640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39b0102040 0x7f39b01975f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:36.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.356+0000 7f39b4bce640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39b0102a00 0x7f39b019e660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:36.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.356+0000 7f39b4bce640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f39b010ca10 con 0x7f39b0102a00 2024-04-21T08:38:36.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.356+0000 7f39b4bce640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f39b010c890 con 0x7f39b0100e60 2024-04-21T08:38:36.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.356+0000 7f39b4bce640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f39b010cb90 con 0x7f39b0102040 2024-04-21T08:38:36.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.356+0000 7f39aed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39b0102a00 0x7f39b019e660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:36.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.356+0000 7f39aed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39b0102a00 0x7f39b019e660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60218/0 (socket says 172.21.15.86:60218) 2024-04-21T08:38:36.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.356+0000 7f39aed9d640 1 -- 172.21.15.86:0/2355081236 learned_addr learned my addr 172.21.15.86:0/2355081236 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:36.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.356+0000 7f39add9b640 1 --2- 172.21.15.86:0/2355081236 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39b0102040 0x7f39b01975f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:36.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.356+0000 7f39ae59c640 1 --2- 172.21.15.86:0/2355081236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39b0100e60 0x7f39b01970b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:36.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.356+0000 7f39aed9d640 1 -- 172.21.15.86:0/2355081236 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39b0102040 msgr2=0x7f39b01975f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:36.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.356+0000 7f39aed9d640 1 --2- 172.21.15.86:0/2355081236 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39b0102040 0x7f39b01975f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.356+0000 7f39aed9d640 1 -- 172.21.15.86:0/2355081236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39b0100e60 msgr2=0x7f39b01970b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:36.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.356+0000 7f39aed9d640 1 --2- 172.21.15.86:0/2355081236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39b0100e60 0x7f39b01970b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.356+0000 7f39aed9d640 1 -- 172.21.15.86:0/2355081236 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39b019ecd0 con 0x7f39b0102a00 2024-04-21T08:38:36.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.356+0000 7f39ae59c640 1 --2- 172.21.15.86:0/2355081236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39b0100e60 0x7f39b01970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:36.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.356+0000 7f39aed9d640 1 --2- 172.21.15.86:0/2355081236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39b0102a00 0x7f39b019e660 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f39a802c700 tx=0x7f39a80346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:36.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.357+0000 7f399f7fe640 1 -- 172.21.15.86:0/2355081236 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a8041020 con 0x7f39b0102a00 2024-04-21T08:38:36.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.357+0000 7f399f7fe640 1 -- 172.21.15.86:0/2355081236 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39a8034ca0 con 0x7f39b0102a00 2024-04-21T08:38:36.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.357+0000 7f39b4bce640 1 -- 172.21.15.86:0/2355081236 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39b019ef60 con 0x7f39b0102a00 2024-04-21T08:38:36.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.357+0000 7f399f7fe640 1 -- 172.21.15.86:0/2355081236 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39a8039040 con 0x7f39b0102a00 2024-04-21T08:38:36.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.357+0000 7f39b4bce640 1 -- 172.21.15.86:0/2355081236 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39b019f3c0 con 0x7f39b0102a00 2024-04-21T08:38:36.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.358+0000 7f39b4bce640 1 -- 172.21.15.86:0/2355081236 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f39b0108200 con 0x7f39b0102a00 2024-04-21T08:38:36.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.359+0000 7f399f7fe640 1 -- 172.21.15.86:0/2355081236 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f39a8044020 con 0x7f39b0102a00 2024-04-21T08:38:36.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.359+0000 7f399f7fe640 1 --2- 172.21.15.86:0/2355081236 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f39980a3bb0 0x7f39980a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:36.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.359+0000 7f39ae59c640 1 --2- 172.21.15.86:0/2355081236 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f39980a3bb0 0x7f39980a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:36.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.359+0000 7f399f7fe640 1 -- 172.21.15.86:0/2355081236 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f39a80f2530 con 0x7f39b0102a00 2024-04-21T08:38:36.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.360+0000 7f39ae59c640 1 --2- 172.21.15.86:0/2355081236 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f39980a3bb0 0x7f39980a6070 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f39b0101bb0 tx=0x7f39a0005f90 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:36.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.364+0000 7f399f7fe640 1 -- 172.21.15.86:0/2355081236 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f39a804b020 con 0x7f39b0102a00 2024-04-21T08:38:36.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.544+0000 7f39b4bce640 1 -- 172.21.15.86:0/2355081236 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f39b0062bf0 con 0x7f39b0102a00 2024-04-21T08:38:36.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.544+0000 7f399f7fe640 1 -- 172.21.15.86:0/2355081236 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7f39a80beb80 con 0x7f39b0102a00 2024-04-21T08:38:36.543 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-21T08:38:36.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.546+0000 7f39b4bce640 1 -- 172.21.15.86:0/2355081236 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f39980a3bb0 msgr2=0x7f39980a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:36.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.546+0000 7f39b4bce640 1 --2- 172.21.15.86:0/2355081236 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f39980a3bb0 0x7f39980a6070 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7f39b0101bb0 tx=0x7f39a0005f90 comp rx=0 tx=0).stop 2024-04-21T08:38:36.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.546+0000 7f39b4bce640 1 -- 172.21.15.86:0/2355081236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39b0102a00 msgr2=0x7f39b019e660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:36.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.546+0000 7f39b4bce640 1 --2- 172.21.15.86:0/2355081236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39b0102a00 0x7f39b019e660 secure :-1 s=READY pgs=445 cs=0 l=1 rev1=1 crypto rx=0x7f39a802c700 tx=0x7f39a80346d0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.546+0000 7f39b4bce640 1 -- 172.21.15.86:0/2355081236 shutdown_connections 2024-04-21T08:38:36.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.546+0000 7f39b4bce640 1 --2- 172.21.15.86:0/2355081236 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f39980a3bb0 0x7f39980a6070 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.546+0000 7f39b4bce640 1 --2- 172.21.15.86:0/2355081236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39b0102a00 0x7f39b019e660 unknown :-1 s=CLOSED pgs=445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.546+0000 7f39b4bce640 1 --2- 172.21.15.86:0/2355081236 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39b0102040 0x7f39b01975f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.546+0000 7f39b4bce640 1 --2- 172.21.15.86:0/2355081236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39b0100e60 0x7f39b01970b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.546+0000 7f39b4bce640 1 -- 172.21.15.86:0/2355081236 >> 172.21.15.86:0/2355081236 conn(0x7f39b00fc440 msgr2=0x7f39b01053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:36.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.546+0000 7f39b4bce640 1 -- 172.21.15.86:0/2355081236 shutdown_connections 2024-04-21T08:38:36.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.546+0000 7f39b4bce640 1 -- 172.21.15.86:0/2355081236 wait complete. 2024-04-21T08:38:36.556 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:36 smithi086 ceph-mon[27939]: pgmap v359: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 12 op/s 2024-04-21T08:38:36.556 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:36 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/990580321' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:36.556 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:36 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/607224544' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:36.556 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:36 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2257403535' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:36.557 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:38:36.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.659+0000 7fb944825640 1 -- 172.21.15.86:0/3524594359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb940100e90 msgr2=0x7fb9401012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:36.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.659+0000 7fb944825640 1 --2- 172.21.15.86:0/3524594359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb940100e90 0x7fb9401012b0 secure :-1 s=READY pgs=446 cs=0 l=1 rev1=1 crypto rx=0x7fb930009900 tx=0x7fb93002bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:36.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.659+0000 7fb944825640 1 -- 172.21.15.86:0/3524594359 shutdown_connections 2024-04-21T08:38:36.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.659+0000 7fb944825640 1 --2- 172.21.15.86:0/3524594359 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9401029a0 0x7fb9401092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.659+0000 7fb944825640 1 --2- 172.21.15.86:0/3524594359 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb940101fe0 0x7fb940102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.659+0000 7fb944825640 1 --2- 172.21.15.86:0/3524594359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb940100e90 0x7fb9401012b0 unknown :-1 s=CLOSED pgs=446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.659+0000 7fb944825640 1 -- 172.21.15.86:0/3524594359 >> 172.21.15.86:0/3524594359 conn(0x7fb9400fc400 msgr2=0x7fb9400fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:36.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.659+0000 7fb944825640 1 -- 172.21.15.86:0/3524594359 shutdown_connections 2024-04-21T08:38:36.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.659+0000 7fb944825640 1 -- 172.21.15.86:0/3524594359 wait complete. 2024-04-21T08:38:36.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.659+0000 7fb944825640 1 Processor -- start 2024-04-21T08:38:36.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.659+0000 7fb944825640 1 -- start start 2024-04-21T08:38:36.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.660+0000 7fb944825640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb940100e90 0x7fb940104e10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:36.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.660+0000 7fb944825640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb940101fe0 0x7fb940103450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:36.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.660+0000 7fb944825640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9401029a0 0x7fb940103990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:36.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.660+0000 7fb944825640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb94010ca60 con 0x7fb940101fe0 2024-04-21T08:38:36.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.660+0000 7fb944825640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb94010c8e0 con 0x7fb940100e90 2024-04-21T08:38:36.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.660+0000 7fb944825640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb94010cbe0 con 0x7fb9401029a0 2024-04-21T08:38:36.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.660+0000 7fb93dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb940101fe0 0x7fb940103450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:36.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.660+0000 7fb93dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb940101fe0 0x7fb940103450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60244/0 (socket says 172.21.15.86:60244) 2024-04-21T08:38:36.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.660+0000 7fb93dd9b640 1 -- 172.21.15.86:0/2468116825 learned_addr learned my addr 172.21.15.86:0/2468116825 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:36.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.660+0000 7fb93e59c640 1 --2- 172.21.15.86:0/2468116825 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb940100e90 0x7fb940104e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:36.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.660+0000 7fb93ed9d640 1 --2- 172.21.15.86:0/2468116825 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9401029a0 0x7fb940103990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:36.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.660+0000 7fb93dd9b640 1 -- 172.21.15.86:0/2468116825 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9401029a0 msgr2=0x7fb940103990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:36.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.660+0000 7fb93dd9b640 1 --2- 172.21.15.86:0/2468116825 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9401029a0 0x7fb940103990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.660+0000 7fb93dd9b640 1 -- 172.21.15.86:0/2468116825 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb940100e90 msgr2=0x7fb940104e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:36.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.660+0000 7fb93dd9b640 1 --2- 172.21.15.86:0/2468116825 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb940100e90 0x7fb940104e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.660+0000 7fb93dd9b640 1 -- 172.21.15.86:0/2468116825 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb940104220 con 0x7fb940101fe0 2024-04-21T08:38:36.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.660+0000 7fb93e59c640 1 --2- 172.21.15.86:0/2468116825 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb940100e90 0x7fb940104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:36.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.660+0000 7fb93ed9d640 1 --2- 172.21.15.86:0/2468116825 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9401029a0 0x7fb940103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:36.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.660+0000 7fb93dd9b640 1 --2- 172.21.15.86:0/2468116825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb940101fe0 0x7fb940103450 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb93400b450 tx=0x7fb93400b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:36.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.660+0000 7fb92f7fe640 1 -- 172.21.15.86:0/2468116825 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb934015020 con 0x7fb940101fe0 2024-04-21T08:38:36.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.660+0000 7fb944825640 1 -- 172.21.15.86:0/2468116825 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9401a2360 con 0x7fb940101fe0 2024-04-21T08:38:36.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.661+0000 7fb92f7fe640 1 -- 172.21.15.86:0/2468116825 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb934011d40 con 0x7fb940101fe0 2024-04-21T08:38:36.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.661+0000 7fb944825640 1 -- 172.21.15.86:0/2468116825 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9401a2980 con 0x7fb940101fe0 2024-04-21T08:38:36.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.661+0000 7fb92f7fe640 1 -- 172.21.15.86:0/2468116825 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb934010e60 con 0x7fb940101fe0 2024-04-21T08:38:36.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.661+0000 7fb944825640 1 -- 172.21.15.86:0/2468116825 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb9401a2ca0 con 0x7fb940101fe0 2024-04-21T08:38:36.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.662+0000 7fb92f7fe640 1 -- 172.21.15.86:0/2468116825 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fb934018020 con 0x7fb940101fe0 2024-04-21T08:38:36.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.663+0000 7fb92f7fe640 1 --2- 172.21.15.86:0/2468116825 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb9280a3bb0 0x7fb9280a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:36.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.663+0000 7fb93e59c640 1 --2- 172.21.15.86:0/2468116825 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb9280a3bb0 0x7fb9280a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:36.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.663+0000 7fb92f7fe640 1 -- 172.21.15.86:0/2468116825 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fb9340d08b0 con 0x7fb940101fe0 2024-04-21T08:38:36.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.664+0000 7fb93e59c640 1 --2- 172.21.15.86:0/2468116825 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb9280a3bb0 0x7fb9280a6070 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb930000c00 tx=0x7fb9300095f0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:36.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.667+0000 7fb92f7fe640 1 -- 172.21.15.86:0/2468116825 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb93401f020 con 0x7fb940101fe0 2024-04-21T08:38:36.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:36 smithi088 ceph-mon[31259]: pgmap v359: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 12 op/s 2024-04-21T08:38:36.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:36 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/990580321' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:36.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:36 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/607224544' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:36.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:36 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2257403535' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:36.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:36 smithi179 ceph-mon[31829]: pgmap v359: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 6.5 KiB/s rd, 12 op/s 2024-04-21T08:38:36.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:36 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/990580321' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:36.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:36 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/607224544' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:36.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:36 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2257403535' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:36.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.842+0000 7fb944825640 1 -- 172.21.15.86:0/2468116825 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fb9401a2e70 con 0x7fb940101fe0 2024-04-21T08:38:36.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.843+0000 7fb92f7fe640 1 -- 172.21.15.86:0/2468116825 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v93) ==== 103+0+11 (secure 0 0 0) 0x7fb93400d040 con 0x7fb940101fe0 2024-04-21T08:38:36.842 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-21T08:38:36.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.845+0000 7fb944825640 1 -- 172.21.15.86:0/2468116825 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb9280a3bb0 msgr2=0x7fb9280a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:36.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.845+0000 7fb944825640 1 --2- 172.21.15.86:0/2468116825 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb9280a3bb0 0x7fb9280a6070 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7fb930000c00 tx=0x7fb9300095f0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.845+0000 7fb944825640 1 -- 172.21.15.86:0/2468116825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb940101fe0 msgr2=0x7fb940103450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:36.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.845+0000 7fb944825640 1 --2- 172.21.15.86:0/2468116825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb940101fe0 0x7fb940103450 secure :-1 s=READY pgs=447 cs=0 l=1 rev1=1 crypto rx=0x7fb93400b450 tx=0x7fb93400b910 comp rx=0 tx=0).stop 2024-04-21T08:38:36.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.845+0000 7fb944825640 1 -- 172.21.15.86:0/2468116825 shutdown_connections 2024-04-21T08:38:36.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.845+0000 7fb944825640 1 --2- 172.21.15.86:0/2468116825 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb9280a3bb0 0x7fb9280a6070 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.845+0000 7fb944825640 1 --2- 172.21.15.86:0/2468116825 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9401029a0 0x7fb940103990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.845+0000 7fb944825640 1 --2- 172.21.15.86:0/2468116825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb940101fe0 0x7fb940103450 unknown :-1 s=CLOSED pgs=447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.845+0000 7fb944825640 1 --2- 172.21.15.86:0/2468116825 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb940100e90 0x7fb940104e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.845+0000 7fb944825640 1 -- 172.21.15.86:0/2468116825 >> 172.21.15.86:0/2468116825 conn(0x7fb9400fc400 msgr2=0x7fb9400fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:36.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.845+0000 7fb944825640 1 -- 172.21.15.86:0/2468116825 shutdown_connections 2024-04-21T08:38:36.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.845+0000 7fb944825640 1 -- 172.21.15.86:0/2468116825 wait complete. 2024-04-21T08:38:36.855 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:38:36.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.957+0000 7f8440288640 1 -- 172.21.15.86:0/2302239635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8438102aa0 msgr2=0x7f8438104e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:36.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.957+0000 7f8440288640 1 --2- 172.21.15.86:0/2302239635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8438102aa0 0x7f8438104e90 secure :-1 s=READY pgs=448 cs=0 l=1 rev1=1 crypto rx=0x7f8434009900 tx=0x7f843402bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:36.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.957+0000 7f8440288640 1 -- 172.21.15.86:0/2302239635 shutdown_connections 2024-04-21T08:38:36.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.957+0000 7f8440288640 1 --2- 172.21.15.86:0/2302239635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8438107d00 0x7f843810a0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.957+0000 7f8440288640 1 --2- 172.21.15.86:0/2302239635 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84381053d0 0x7f84381077c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.957+0000 7f8440288640 1 --2- 172.21.15.86:0/2302239635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8438102aa0 0x7f8438104e90 unknown :-1 s=CLOSED pgs=448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.957+0000 7f8440288640 1 -- 172.21.15.86:0/2302239635 >> 172.21.15.86:0/2302239635 conn(0x7f84380fc470 msgr2=0x7f84380fe890 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:36.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.957+0000 7f8440288640 1 -- 172.21.15.86:0/2302239635 shutdown_connections 2024-04-21T08:38:36.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.957+0000 7f8440288640 1 -- 172.21.15.86:0/2302239635 wait complete. 2024-04-21T08:38:36.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.957+0000 7f8440288640 1 Processor -- start 2024-04-21T08:38:36.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.957+0000 7f8440288640 1 -- start start 2024-04-21T08:38:36.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.958+0000 7f8440288640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8438102aa0 0x7f8438198210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:36.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.958+0000 7f8440288640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84381053d0 0x7f8438198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:36.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.958+0000 7f8440288640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8438107d00 0x7f843819f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:36.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.958+0000 7f8440288640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f843810db00 con 0x7f8438107d00 2024-04-21T08:38:36.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.958+0000 7f8440288640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f843810d980 con 0x7f8438102aa0 2024-04-21T08:38:36.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.958+0000 7f8440288640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f843810dc80 con 0x7f84381053d0 2024-04-21T08:38:36.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.958+0000 7f843e825640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8438107d00 0x7f843819f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:36.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.958+0000 7f843e825640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8438107d00 0x7f843819f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60252/0 (socket says 172.21.15.86:60252) 2024-04-21T08:38:36.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.958+0000 7f843e825640 1 -- 172.21.15.86:0/1852617290 learned_addr learned my addr 172.21.15.86:0/1852617290 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:36.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.958+0000 7f843d823640 1 --2- 172.21.15.86:0/1852617290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84381053d0 0x7f8438198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:36.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.958+0000 7f843e024640 1 --2- 172.21.15.86:0/1852617290 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8438102aa0 0x7f8438198210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:36.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.958+0000 7f843e825640 1 -- 172.21.15.86:0/1852617290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84381053d0 msgr2=0x7f8438198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:36.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.958+0000 7f843e825640 1 --2- 172.21.15.86:0/1852617290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84381053d0 0x7f8438198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.958+0000 7f843e825640 1 -- 172.21.15.86:0/1852617290 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8438102aa0 msgr2=0x7f8438198210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:36.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.958+0000 7f843e825640 1 --2- 172.21.15.86:0/1852617290 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8438102aa0 0x7f8438198210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:36.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.958+0000 7f843e825640 1 -- 172.21.15.86:0/1852617290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f843819fe30 con 0x7f8438107d00 2024-04-21T08:38:36.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.958+0000 7f843e024640 1 --2- 172.21.15.86:0/1852617290 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8438102aa0 0x7f8438198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:36.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.958+0000 7f843e825640 1 --2- 172.21.15.86:0/1852617290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8438107d00 0x7f843819f7c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8430009fd0 tx=0x7f843000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:36.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.958+0000 7f842f7fe640 1 -- 172.21.15.86:0/1852617290 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430017070 con 0x7f8438107d00 2024-04-21T08:38:36.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.958+0000 7f842f7fe640 1 -- 172.21.15.86:0/1852617290 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8430003e20 con 0x7f8438107d00 2024-04-21T08:38:36.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.958+0000 7f8440288640 1 -- 172.21.15.86:0/1852617290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84381a0120 con 0x7f8438107d00 2024-04-21T08:38:36.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.958+0000 7f842f7fe640 1 -- 172.21.15.86:0/1852617290 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8430010460 con 0x7f8438107d00 2024-04-21T08:38:36.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.958+0000 7f8440288640 1 -- 172.21.15.86:0/1852617290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84381a0670 con 0x7f8438107d00 2024-04-21T08:38:36.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.959+0000 7f8440288640 1 -- 172.21.15.86:0/1852617290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84380fdf60 con 0x7f8438107d00 2024-04-21T08:38:36.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.961+0000 7f842f7fe640 1 -- 172.21.15.86:0/1852617290 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f8430003970 con 0x7f8438107d00 2024-04-21T08:38:36.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.961+0000 7f842f7fe640 1 --2- 172.21.15.86:0/1852617290 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f84240a3cb0 0x7f84240a6170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:36.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.961+0000 7f843e024640 1 --2- 172.21.15.86:0/1852617290 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f84240a3cb0 0x7f84240a6170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:36.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.961+0000 7f842f7fe640 1 -- 172.21.15.86:0/1852617290 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f84300d04c0 con 0x7f8438107d00 2024-04-21T08:38:36.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.962+0000 7f843e024640 1 --2- 172.21.15.86:0/1852617290 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f84240a3cb0 0x7f84240a6170 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f84340096c0 tx=0x7f8434000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:36.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:36.965+0000 7f842f7fe640 1 -- 172.21.15.86:0/1852617290 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f843009cb30 con 0x7f8438107d00 2024-04-21T08:38:37.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.182+0000 7f8440288640 1 -- 172.21.15.86:0/1852617290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f843802d5c0 con 0x7f8438107d00 2024-04-21T08:38:37.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.182+0000 7f842f7fe640 1 -- 172.21.15.86:0/1852617290 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 28 v28) ==== 94+0+11935 (secure 0 0 0) 0x7f843009c430 con 0x7f8438107d00 2024-04-21T08:38:37.182 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:38:37.182 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":28,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":28,"flags":50,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":false},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:36:47.458637+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24455,24472,24394,24398,24482]},"id":1}]} 2024-04-21T08:38:37.182 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 28 2024-04-21T08:38:37.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.184+0000 7f8440288640 1 -- 172.21.15.86:0/1852617290 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f84240a3cb0 msgr2=0x7f84240a6170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:37.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.184+0000 7f8440288640 1 --2- 172.21.15.86:0/1852617290 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f84240a3cb0 0x7f84240a6170 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f84340096c0 tx=0x7f8434000bc0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.184+0000 7f8440288640 1 -- 172.21.15.86:0/1852617290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8438107d00 msgr2=0x7f843819f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:37.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.184+0000 7f8440288640 1 --2- 172.21.15.86:0/1852617290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8438107d00 0x7f843819f7c0 secure :-1 s=READY pgs=449 cs=0 l=1 rev1=1 crypto rx=0x7f8430009fd0 tx=0x7f843000b260 comp rx=0 tx=0).stop 2024-04-21T08:38:37.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.184+0000 7f8440288640 1 -- 172.21.15.86:0/1852617290 shutdown_connections 2024-04-21T08:38:37.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.184+0000 7f8440288640 1 --2- 172.21.15.86:0/1852617290 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f84240a3cb0 0x7f84240a6170 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.184+0000 7f8440288640 1 --2- 172.21.15.86:0/1852617290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8438107d00 0x7f843819f7c0 unknown :-1 s=CLOSED pgs=449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.184+0000 7f8440288640 1 --2- 172.21.15.86:0/1852617290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84381053d0 0x7f8438198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.184+0000 7f8440288640 1 --2- 172.21.15.86:0/1852617290 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8438102aa0 0x7f8438198210 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.184+0000 7f8440288640 1 -- 172.21.15.86:0/1852617290 >> 172.21.15.86:0/1852617290 conn(0x7f84380fc470 msgr2=0x7f8438103340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:37.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.184+0000 7f8440288640 1 -- 172.21.15.86:0/1852617290 shutdown_connections 2024-04-21T08:38:37.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.184+0000 7f8440288640 1 -- 172.21.15.86:0/1852617290 wait complete. 2024-04-21T08:38:37.195 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:38:37.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.293+0000 7f9b35967640 1 -- 172.21.15.86:0/1272895357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b30101f90 msgr2=0x7f9b301023b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:37.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.293+0000 7f9b35967640 1 --2- 172.21.15.86:0/1272895357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b30101f90 0x7f9b301023b0 secure :-1 s=READY pgs=450 cs=0 l=1 rev1=1 crypto rx=0x7f9b24009900 tx=0x7f9b2402bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:37.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.294+0000 7f9b35967640 1 -- 172.21.15.86:0/1272895357 shutdown_connections 2024-04-21T08:38:37.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.294+0000 7f9b35967640 1 --2- 172.21.15.86:0/1272895357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b30103b40 0x7f9b3010a3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.294+0000 7f9b35967640 1 --2- 172.21.15.86:0/1272895357 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b30103180 0x7f9b30103600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.294+0000 7f9b35967640 1 --2- 172.21.15.86:0/1272895357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b30101f90 0x7f9b301023b0 unknown :-1 s=CLOSED pgs=450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.294+0000 7f9b35967640 1 -- 172.21.15.86:0/1272895357 >> 172.21.15.86:0/1272895357 conn(0x7f9b300fd500 msgr2=0x7f9b300ff940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:37.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.294+0000 7f9b35967640 1 -- 172.21.15.86:0/1272895357 shutdown_connections 2024-04-21T08:38:37.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.294+0000 7f9b35967640 1 -- 172.21.15.86:0/1272895357 wait complete. 2024-04-21T08:38:37.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.294+0000 7f9b35967640 1 Processor -- start 2024-04-21T08:38:37.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.294+0000 7f9b35967640 1 -- start start 2024-04-21T08:38:37.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.294+0000 7f9b35967640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b30101f90 0x7f9b301981b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.294+0000 7f9b35967640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b30103180 0x7f9b301986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.294+0000 7f9b35967640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b30103b40 0x7f9b3019f760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.294+0000 7f9b35967640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3010db10 con 0x7f9b30103b40 2024-04-21T08:38:37.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.294+0000 7f9b35967640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3010d990 con 0x7f9b30103180 2024-04-21T08:38:37.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.294+0000 7f9b35967640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9b3010dc90 con 0x7f9b30101f90 2024-04-21T08:38:37.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.295+0000 7f9b2f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b30103b40 0x7f9b3019f760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.295+0000 7f9b2f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b30103b40 0x7f9b3019f760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60274/0 (socket says 172.21.15.86:60274) 2024-04-21T08:38:37.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.295+0000 7f9b2f7fe640 1 -- 172.21.15.86:0/171220768 learned_addr learned my addr 172.21.15.86:0/171220768 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:37.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.295+0000 7f9b2e7fc640 1 --2- 172.21.15.86:0/171220768 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b30103180 0x7f9b301986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.295+0000 7f9b2effd640 1 --2- 172.21.15.86:0/171220768 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b30101f90 0x7f9b301981b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.295+0000 7f9b2f7fe640 1 -- 172.21.15.86:0/171220768 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b30101f90 msgr2=0x7f9b301981b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:37.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.295+0000 7f9b2f7fe640 1 --2- 172.21.15.86:0/171220768 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b30101f90 0x7f9b301981b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.295+0000 7f9b2f7fe640 1 -- 172.21.15.86:0/171220768 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b30103180 msgr2=0x7f9b301986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:37.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.295+0000 7f9b2f7fe640 1 --2- 172.21.15.86:0/171220768 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b30103180 0x7f9b301986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.295+0000 7f9b2f7fe640 1 -- 172.21.15.86:0/171220768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b3019fdd0 con 0x7f9b30103b40 2024-04-21T08:38:37.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.295+0000 7f9b2effd640 1 --2- 172.21.15.86:0/171220768 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b30101f90 0x7f9b301981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:37.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.295+0000 7f9b2f7fe640 1 --2- 172.21.15.86:0/171220768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b30103b40 0x7f9b3019f760 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9b2800b970 tx=0x7f9b2800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:37.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.295+0000 7f9b34965640 1 -- 172.21.15.86:0/171220768 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b28019070 con 0x7f9b30103b40 2024-04-21T08:38:37.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.295+0000 7f9b35967640 1 -- 172.21.15.86:0/171220768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b301a00c0 con 0x7f9b30103b40 2024-04-21T08:38:37.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.295+0000 7f9b34965640 1 -- 172.21.15.86:0/171220768 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b28014440 con 0x7f9b30103b40 2024-04-21T08:38:37.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.295+0000 7f9b34965640 1 -- 172.21.15.86:0/171220768 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b280075d0 con 0x7f9b30103b40 2024-04-21T08:38:37.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.295+0000 7f9b35967640 1 -- 172.21.15.86:0/171220768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b301a06e0 con 0x7f9b30103b40 2024-04-21T08:38:37.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.296+0000 7f9b35967640 1 -- 172.21.15.86:0/171220768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9b30109330 con 0x7f9b30103b40 2024-04-21T08:38:37.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.297+0000 7f9b34965640 1 -- 172.21.15.86:0/171220768 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f9b2801c070 con 0x7f9b30103b40 2024-04-21T08:38:37.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.298+0000 7f9b34965640 1 --2- 172.21.15.86:0/171220768 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f9b180a3bb0 0x7f9b180a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.298+0000 7f9b2effd640 1 --2- 172.21.15.86:0/171220768 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f9b180a3bb0 0x7f9b180a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.298+0000 7f9b34965640 1 -- 172.21.15.86:0/171220768 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f9b280cfdd0 con 0x7f9b30103b40 2024-04-21T08:38:37.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.298+0000 7f9b2effd640 1 --2- 172.21.15.86:0/171220768 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f9b180a3bb0 0x7f9b180a6070 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9b240096c0 tx=0x7f9b24000bc0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:37.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.302+0000 7f9b34965640 1 -- 172.21.15.86:0/171220768 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9b2809c420 con 0x7f9b30103b40 2024-04-21T08:38:37.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.486+0000 7f9b35967640 1 -- 172.21.15.86:0/171220768 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f9b30062b50 con 0x7f9b30103b40 2024-04-21T08:38:37.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.487+0000 7f9b34965640 1 -- 172.21.15.86:0/171220768 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24689 (secure 0 0 0) 0x7f9b2809bd20 con 0x7f9b30103b40 2024-04-21T08:38:37.487 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:38:37.487 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":93,"fsid":"f8cec7cc-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:22.687329+0000","modified":"2024-04-21T08:38:24.484832+0000","last_up_change":"2024-04-21T08:35:30.048315+0000","last_in_change":"2024-04-21T08:35:10.099496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:45.279109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:18.300782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:19.574636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:22.905632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89b7601f-0620-4db2-90f1-b45cea173738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6803","nonce":2571516897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6805","nonce":2571516897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6809","nonce":2571516897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6807","nonce":2571516897}]},"public_addr":"172.21.15.86:6803/2571516897","cluster_addr":"172.21.15.86:6805/2571516897","heartbeat_back_addr":"172.21.15.86:6809/2571516897","heartbeat_front_addr":"172.21.15.86:6807/2571516897","state":["exists","up"]},{"osd":1,"uuid":"4891031c-f98c-42bc-bb3b-a420855ecc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6801","nonce":74067182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6803","nonce":74067182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6807","nonce":74067182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6805","nonce":74067182}]},"public_addr":"172.21.15.88:6801/74067182","cluster_addr":"172.21.15.88:6803/74067182","heartbeat_back_addr":"172.21.15.88:6807/74067182","heartbeat_front_addr":"172.21.15.88:6805/74067182","state":["exists","up"]},{"osd":2,"uuid":"60d12aa4-9312-4fa2-8d0b-2b3f42caad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6801","nonce":4137659051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6803","nonce":4137659051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6807","nonce":4137659051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6805","nonce":4137659051}]},"public_addr":"172.21.15.179:6801/4137659051","cluster_addr":"172.21.15.179:6803/4137659051","heartbeat_back_addr":"172.21.15.179:6807/4137659051","heartbeat_front_addr":"172.21.15.179:6805/4137659051","state":["exists","up"]},{"osd":3,"uuid":"404c3158-8cb0-4c02-b48a-f9ee5ff7b784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6811","nonce":2309603148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6813","nonce":2309603148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6817","nonce":2309603148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6815","nonce":2309603148}]},"public_addr":"172.21.15.86:6811/2309603148","cluster_addr":"172.21.15.86:6813/2309603148","heartbeat_back_addr":"172.21.15.86:6817/2309603148","heartbeat_front_addr":"172.21.15.86:6815/2309603148","state":["exists","up"]},{"osd":4,"uuid":"458ff202-d781-4ec7-ac93-0b49f835b503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6809","nonce":1818714217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6811","nonce":1818714217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6815","nonce":1818714217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6813","nonce":1818714217}]},"public_addr":"172.21.15.88:6809/1818714217","cluster_addr":"172.21.15.88:6811/1818714217","heartbeat_back_addr":"172.21.15.88:6815/1818714217","heartbeat_front_addr":"172.21.15.88:6813/1818714217","state":["exists","up"]},{"osd":5,"uuid":"8123cc90-bd8b-4204-bf31-46b34e9f589d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6809","nonce":863840990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6811","nonce":863840990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6815","nonce":863840990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6813","nonce":863840990}]},"public_addr":"172.21.15.179:6809/863840990","cluster_addr":"172.21.15.179:6811/863840990","heartbeat_back_addr":"172.21.15.179:6815/863840990","heartbeat_front_addr":"172.21.15.179:6813/863840990","state":["exists","up"]},{"osd":6,"uuid":"aac9d7c7-659a-4110-903a-a871c086066f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6819","nonce":1224231118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6821","nonce":1224231118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6825","nonce":1224231118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6823","nonce":1224231118}]},"public_addr":"172.21.15.86:6819/1224231118","cluster_addr":"172.21.15.86:6821/1224231118","heartbeat_back_addr":"172.21.15.86:6825/1224231118","heartbeat_front_addr":"172.21.15.86:6823/1224231118","state":["exists","up"]},{"osd":7,"uuid":"df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6817","nonce":136629878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6819","nonce":136629878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6823","nonce":136629878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6821","nonce":136629878}]},"public_addr":"172.21.15.88:6817/136629878","cluster_addr":"172.21.15.88:6819/136629878","heartbeat_back_addr":"172.21.15.88:6823/136629878","heartbeat_front_addr":"172.21.15.88:6821/136629878","state":["exists","up"]},{"osd":8,"uuid":"63e59f4b-0426-46db-92f7-c55dae61b5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6817","nonce":1770418957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6819","nonce":1770418957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6823","nonce":1770418957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6821","nonce":1770418957}]},"public_addr":"172.21.15.179:6817/1770418957","cluster_addr":"172.21.15.179:6819/1770418957","heartbeat_back_addr":"172.21.15.179:6823/1770418957","heartbeat_front_addr":"172.21.15.179:6821/1770418957","state":["exists","up"]},{"osd":9,"uuid":"5841bb36-ed6b-45d8-b857-eeab139c44e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6827","nonce":1108500274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6829","nonce":1108500274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6833","nonce":1108500274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6831","nonce":1108500274}]},"public_addr":"172.21.15.86:6827/1108500274","cluster_addr":"172.21.15.86:6829/1108500274","heartbeat_back_addr":"172.21.15.86:6833/1108500274","heartbeat_front_addr":"172.21.15.86:6831/1108500274","state":["exists","up"]},{"osd":10,"uuid":"f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6825","nonce":1294275353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6827","nonce":1294275353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6831","nonce":1294275353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6829","nonce":1294275353}]},"public_addr":"172.21.15.88:6825/1294275353","cluster_addr":"172.21.15.88:6827/1294275353","heartbeat_back_addr":"172.21.15.88:6831/1294275353","heartbeat_front_addr":"172.21.15.88:6829/1294275353","state":["exists","up"]},{"osd":11,"uuid":"e0d4cc76-e7a6-4a6b-909c-95488ccd5a36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6825","nonce":4007070233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6827","nonce":4007070233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6831","nonce":4007070233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6829","nonce":4007070233}]},"public_addr":"172.21.15.179:6825/4007070233","cluster_addr":"172.21.15.179:6827/4007070233","heartbeat_back_addr":"172.21.15.179:6831/4007070233","heartbeat_front_addr":"172.21.15.179:6829/4007070233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:55.210605+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:18.723090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:42.547979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:06.421622+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:30.858315+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:55.455313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.623369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:46.185111+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.103627+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:37.297545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:02.637431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:28.199292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6800/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/1412393225":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6800/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/1687852404":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2020707404":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2568130933":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/3952607648":"2024-04-22T08:28:57.384403+0000","172.21.15.86:6801/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/2756903426":"2024-04-22T08:29:23.190001+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:38:37.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.489+0000 7f9b35967640 1 -- 172.21.15.86:0/171220768 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f9b180a3bb0 msgr2=0x7f9b180a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:37.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.489+0000 7f9b35967640 1 --2- 172.21.15.86:0/171220768 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f9b180a3bb0 0x7f9b180a6070 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f9b240096c0 tx=0x7f9b24000bc0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.489+0000 7f9b35967640 1 -- 172.21.15.86:0/171220768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b30103b40 msgr2=0x7f9b3019f760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:37.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.489+0000 7f9b35967640 1 --2- 172.21.15.86:0/171220768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b30103b40 0x7f9b3019f760 secure :-1 s=READY pgs=451 cs=0 l=1 rev1=1 crypto rx=0x7f9b2800b970 tx=0x7f9b2800be30 comp rx=0 tx=0).stop 2024-04-21T08:38:37.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.489+0000 7f9b35967640 1 -- 172.21.15.86:0/171220768 shutdown_connections 2024-04-21T08:38:37.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.489+0000 7f9b35967640 1 --2- 172.21.15.86:0/171220768 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f9b180a3bb0 0x7f9b180a6070 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.489+0000 7f9b35967640 1 --2- 172.21.15.86:0/171220768 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b30103b40 0x7f9b3019f760 unknown :-1 s=CLOSED pgs=451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.489+0000 7f9b35967640 1 --2- 172.21.15.86:0/171220768 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b30103180 0x7f9b301986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.489+0000 7f9b35967640 1 --2- 172.21.15.86:0/171220768 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b30101f90 0x7f9b301981b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.489+0000 7f9b35967640 1 -- 172.21.15.86:0/171220768 >> 172.21.15.86:0/171220768 conn(0x7f9b300fd500 msgr2=0x7f9b30106500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:37.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.490+0000 7f9b35967640 1 -- 172.21.15.86:0/171220768 shutdown_connections 2024-04-21T08:38:37.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.490+0000 7f9b35967640 1 -- 172.21.15.86:0/171220768 wait complete. 2024-04-21T08:38:37.499 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:37 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2355081236' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:37.499 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:37 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2468116825' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:37.499 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:37 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1852617290' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:37.501 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 28, 'flags': 50, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': False}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-21T08:36:20.626894+0000', 'modified': '2024-04-21T08:36:47.458637+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24482, 'mds_1': 24472, 'mds_2': 24455, 'mds_3': 24398, 'mds_4': 24394}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14640': {'gid': 14640, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6839/3163123023', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6838', 'nonce': 3163123023}, {'type': 'v1', 'addr': '172.21.15.86:6839', 'nonce': 3163123023}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14652': {'gid': 14652, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6841/3537972023', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6840', 'nonce': 3537972023}, {'type': 'v1', 'addr': '172.21.15.86:6841', 'nonce': 3537972023}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24394': {'gid': 24394, 'name': 'l', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.179:6833/3606421532', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6832', 'nonce': 3606421532}, {'type': 'v1', 'addr': '172.21.15.179:6833', 'nonce': 3606421532}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24398': {'gid': 24398, 'name': 'k', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.88:6833/1872744608', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6832', 'nonce': 1872744608}, {'type': 'v1', 'addr': '172.21.15.88:6833', 'nonce': 1872744608}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.88:6835/454792798', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6834', 'nonce': 454792798}, {'type': 'v1', 'addr': '172.21.15.88:6835', 'nonce': 454792798}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.179:6835/3343050063', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6834', 'nonce': 3343050063}, {'type': 'v1', 'addr': '172.21.15.179:6835', 'nonce': 3343050063}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.179:6837/1280529894', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6836', 'nonce': 1280529894}, {'type': 'v1', 'addr': '172.21.15.179:6837', 'nonce': 1280529894}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.88:6837/1950568471', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6836', 'nonce': 1950568471}, {'type': 'v1', 'addr': '172.21.15.88:6837', 'nonce': 1950568471}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.179:6839/3029310820', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6838', 'nonce': 3029310820}, {'type': 'v1', 'addr': '172.21.15.179:6839', 'nonce': 3029310820}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.88:6839/3969608142', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6838', 'nonce': 3969608142}, {'type': 'v1', 'addr': '172.21.15.88:6839', 'nonce': 3969608142}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24482, 'qdb_cluster': [24455, 24472, 24394, 24398, 24482]} 2024-04-21T08:38:37.501 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-21T08:38:37.501 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-04-21T08:38:37.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.601+0000 7fe52e225640 1 -- 172.21.15.86:0/4108946239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe528103b50 msgr2=0x7fe52810a3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:37.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.601+0000 7fe52e225640 1 --2- 172.21.15.86:0/4108946239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe528103b50 0x7fe52810a3e0 secure :-1 s=READY pgs=452 cs=0 l=1 rev1=1 crypto rx=0x7fe520009ca0 tx=0x7fe52002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.601+0000 7fe52e225640 1 -- 172.21.15.86:0/4108946239 shutdown_connections 2024-04-21T08:38:37.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.601+0000 7fe52e225640 1 --2- 172.21.15.86:0/4108946239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe528103b50 0x7fe52810a3e0 unknown :-1 s=CLOSED pgs=452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.601+0000 7fe52e225640 1 --2- 172.21.15.86:0/4108946239 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe528103190 0x7fe528103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.601+0000 7fe52e225640 1 --2- 172.21.15.86:0/4108946239 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe528101fa0 0x7fe5281023c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.601+0000 7fe52e225640 1 -- 172.21.15.86:0/4108946239 >> 172.21.15.86:0/4108946239 conn(0x7fe5280fd530 msgr2=0x7fe5280ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:37.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.601+0000 7fe52e225640 1 -- 172.21.15.86:0/4108946239 shutdown_connections 2024-04-21T08:38:37.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.601+0000 7fe52e225640 1 -- 172.21.15.86:0/4108946239 wait complete. 2024-04-21T08:38:37.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.602+0000 7fe52e225640 1 Processor -- start 2024-04-21T08:38:37.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.602+0000 7fe52e225640 1 -- start start 2024-04-21T08:38:37.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.602+0000 7fe52e225640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe528101fa0 0x7fe5281981f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.602+0000 7fe52e225640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe528103190 0x7fe528198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.602+0000 7fe52e225640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe528103b50 0x7fe52819f7a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.602+0000 7fe52e225640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe52810db40 con 0x7fe528103b50 2024-04-21T08:38:37.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.602+0000 7fe52e225640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe52810d9c0 con 0x7fe528101fa0 2024-04-21T08:38:37.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.602+0000 7fe52e225640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe52810dcc0 con 0x7fe528103190 2024-04-21T08:38:37.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.602+0000 7fe527fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe528103b50 0x7fe52819f7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.602+0000 7fe527fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe528103b50 0x7fe52819f7a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60306/0 (socket says 172.21.15.86:60306) 2024-04-21T08:38:37.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.602+0000 7fe527fff640 1 -- 172.21.15.86:0/3820896792 learned_addr learned my addr 172.21.15.86:0/3820896792 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:37.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.602+0000 7fe5277fe640 1 --2- 172.21.15.86:0/3820896792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe528101fa0 0x7fe5281981f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.603+0000 7fe527fff640 1 -- 172.21.15.86:0/3820896792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe528103190 msgr2=0x7fe528198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:37.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.603+0000 7fe51effd640 1 --2- 172.21.15.86:0/3820896792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe528103190 0x7fe528198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.603+0000 7fe527fff640 1 --2- 172.21.15.86:0/3820896792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe528103190 0x7fe528198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.603+0000 7fe527fff640 1 -- 172.21.15.86:0/3820896792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe528101fa0 msgr2=0x7fe5281981f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:37.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.603+0000 7fe527fff640 1 --2- 172.21.15.86:0/3820896792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe528101fa0 0x7fe5281981f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.603+0000 7fe527fff640 1 -- 172.21.15.86:0/3820896792 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe52819fe10 con 0x7fe528103b50 2024-04-21T08:38:37.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.603+0000 7fe5277fe640 1 --2- 172.21.15.86:0/3820896792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe528101fa0 0x7fe5281981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:37.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.603+0000 7fe527fff640 1 --2- 172.21.15.86:0/3820896792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe528103b50 0x7fe52819f7a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe52002c700 tx=0x7fe520035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:37.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.603+0000 7fe5257fa640 1 -- 172.21.15.86:0/3820896792 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520039040 con 0x7fe528103b50 2024-04-21T08:38:37.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.603+0000 7fe5257fa640 1 -- 172.21.15.86:0/3820896792 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe520044070 con 0x7fe528103b50 2024-04-21T08:38:37.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.603+0000 7fe52e225640 1 -- 172.21.15.86:0/3820896792 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5281a00a0 con 0x7fe528103b50 2024-04-21T08:38:37.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.603+0000 7fe5257fa640 1 -- 172.21.15.86:0/3820896792 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520004440 con 0x7fe528103b50 2024-04-21T08:38:37.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.603+0000 7fe52e225640 1 -- 172.21.15.86:0/3820896792 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5281a0590 con 0x7fe528103b50 2024-04-21T08:38:37.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.605+0000 7fe5257fa640 1 -- 172.21.15.86:0/3820896792 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fe5200045e0 con 0x7fe528103b50 2024-04-21T08:38:37.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.605+0000 7fe530e14740 1 Processor -- start 2024-04-21T08:38:37.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.606+0000 7fe530e14740 1 -- start start 2024-04-21T08:38:37.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.606+0000 7fe530e14740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557dc6be2e40 0x557dc6be3260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.606+0000 7fe5257fa640 1 --2- 172.21.15.86:0/3820896792 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe5140a3bb0 0x7fe5140a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.606+0000 7fe530e14740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dc6be3830 0x557dc6be3cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.606+0000 7fe530e14740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dc6be42d0 0x557dc6beaeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.606+0000 7fe530e14740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557dc6b47b10 con 0x557dc6be42d0 2024-04-21T08:38:37.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.606+0000 7fe530e14740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x557dc6ae5240 con 0x557dc6be2e40 2024-04-21T08:38:37.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.606+0000 7fe530e14740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x557dc6b43ad0 con 0x557dc6be3830 2024-04-21T08:38:37.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.606+0000 7fe527fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dc6be42d0 0x557dc6beaeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.606+0000 7fe51effd640 1 --2- 172.21.15.86:0/3820896792 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe5140a3bb0 0x7fe5140a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.606+0000 7fe527fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dc6be42d0 0x557dc6beaeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60322/0 (socket says 172.21.15.86:60322) 2024-04-21T08:38:37.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.606+0000 7fe527fff640 1 -- 172.21.15.86:0/2935465433 learned_addr learned my addr 172.21.15.86:0/2935465433 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:37.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.606+0000 7fe5257fa640 1 -- 172.21.15.86:0/3820896792 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fe5200f2ef0 con 0x7fe528103b50 2024-04-21T08:38:37.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.606+0000 7fe51effd640 1 --2- 172.21.15.86:0/2935465433 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dc6be3830 0x557dc6be3cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.606+0000 7fe5277fe640 1 --2- 172.21.15.86:0/2935465433 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557dc6be2e40 0x557dc6be3260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.606+0000 7fe527fff640 1 -- 172.21.15.86:0/2935465433 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dc6be3830 msgr2=0x557dc6be3cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:37.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.606+0000 7fe527fff640 1 --2- 172.21.15.86:0/2935465433 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dc6be3830 0x557dc6be3cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.606+0000 7fe527fff640 1 -- 172.21.15.86:0/2935465433 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557dc6be2e40 msgr2=0x557dc6be3260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:37.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.606+0000 7fe527fff640 1 --2- 172.21.15.86:0/2935465433 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557dc6be2e40 0x557dc6be3260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.606+0000 7fe527fff640 1 -- 172.21.15.86:0/2935465433 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dc685a2d0 con 0x557dc6be42d0 2024-04-21T08:38:37.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.606+0000 7fe51effd640 1 --2- 172.21.15.86:0/3820896792 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe5140a3bb0 0x7fe5140a6070 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe528199700 tx=0x7fe510014840 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:37.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.606+0000 7fe527fff640 1 --2- 172.21.15.86:0/2935465433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dc6be42d0 0x557dc6beaeb0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe520009dc0 tx=0x7fe52002c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad44554d0d69ce2d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:37.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.606+0000 7fe51dffb640 1 -- 172.21.15.86:0/2935465433 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5200c1040 con 0x557dc6be42d0 2024-04-21T08:38:37.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.607+0000 7fe51dffb640 1 -- 172.21.15.86:0/2935465433 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5200098d0 con 0x557dc6be42d0 2024-04-21T08:38:37.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.607+0000 7fe51dffb640 1 -- 172.21.15.86:0/2935465433 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe520090b80 con 0x557dc6be42d0 2024-04-21T08:38:37.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.607+0000 7fe530e14740 1 -- 172.21.15.86:0/2935465433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dc6be42d0 msgr2=0x557dc6beaeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:37.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.607+0000 7fe530e14740 1 --2- 172.21.15.86:0/2935465433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dc6be42d0 0x557dc6beaeb0 secure :-1 s=READY pgs=454 cs=0 l=1 rev1=1 crypto rx=0x7fe520009dc0 tx=0x7fe52002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.607+0000 7fe530e14740 1 -- 172.21.15.86:0/2935465433 shutdown_connections 2024-04-21T08:38:37.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.607+0000 7fe530e14740 1 --2- 172.21.15.86:0/2935465433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dc6be42d0 0x557dc6beaeb0 unknown :-1 s=CLOSED pgs=454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.607+0000 7fe530e14740 1 --2- 172.21.15.86:0/2935465433 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dc6be3830 0x557dc6be3cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.607+0000 7fe530e14740 1 --2- 172.21.15.86:0/2935465433 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557dc6be2e40 0x557dc6be3260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.607+0000 7fe530e14740 1 -- 172.21.15.86:0/2935465433 >> 172.21.15.86:0/2935465433 conn(0x557dc6bdef40 msgr2=0x557dc6bdd490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:37.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.607+0000 7fe530e14740 1 -- 172.21.15.86:0/2935465433 shutdown_connections 2024-04-21T08:38:37.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.607+0000 7fe530e14740 1 -- 172.21.15.86:0/2935465433 wait complete. 2024-04-21T08:38:37.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.607+0000 7fe530e14740 1 -- start start 2024-04-21T08:38:37.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.608+0000 7fe530e14740 1 Processor -- start 2024-04-21T08:38:37.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.608+0000 7fe530e14740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dc6be2e40 0x557dc6becbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.608+0000 7fe530e14740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dc6be3830 0x557dc6bed0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.608+0000 7fe530e14740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557dc6be42d0 0x557dc6bf4130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.608+0000 7fe530e14740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557dc6be4ad0 con 0x557dc6be3830 2024-04-21T08:38:37.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.608+0000 7fe530e14740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x557dc6ad0190 con 0x557dc6be42d0 2024-04-21T08:38:37.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.608+0000 7fe530e14740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x557dc6beb3f0 con 0x557dc6be2e40 2024-04-21T08:38:37.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.608+0000 7fe530e14740 10 client.0 _ll_register_callbacks cb 0x557dc6acb8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:37.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.608+0000 7fe51effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dc6be3830 0x557dc6bed0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.608+0000 7fe51effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dc6be3830 0x557dc6bed0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60334/0 (socket says 172.21.15.86:60334) 2024-04-21T08:38:37.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.608+0000 7fe51effd640 1 -- 172.21.15.86:0/10974659 learned_addr learned my addr 172.21.15.86:0/10974659 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:37.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.608+0000 7fe5277fe640 1 --2- 172.21.15.86:0/10974659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dc6be2e40 0x557dc6becbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.608+0000 7fe51effd640 1 -- 172.21.15.86:0/10974659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dc6be2e40 msgr2=0x557dc6becbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:37.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.608+0000 7fe527fff640 1 --2- 172.21.15.86:0/10974659 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557dc6be42d0 0x557dc6bf4130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.608+0000 7fe51effd640 1 --2- 172.21.15.86:0/10974659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dc6be2e40 0x557dc6becbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.608+0000 7fe51effd640 1 -- 172.21.15.86:0/10974659 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557dc6be42d0 msgr2=0x557dc6bf4130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:37.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.608+0000 7fe51effd640 1 --2- 172.21.15.86:0/10974659 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557dc6be42d0 0x557dc6bf4130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.608+0000 7fe51effd640 1 -- 172.21.15.86:0/10974659 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dc69246b0 con 0x557dc6be3830 2024-04-21T08:38:37.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.608+0000 7fe5277fe640 1 --2- 172.21.15.86:0/10974659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dc6be2e40 0x557dc6becbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:37.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.608+0000 7fe527fff640 1 --2- 172.21.15.86:0/10974659 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557dc6be42d0 0x557dc6bf4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:37.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.608+0000 7fe51effd640 1 --2- 172.21.15.86:0/10974659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dc6be3830 0x557dc6bed0f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe51000bfd0 tx=0x7fe510004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e44bce66488b97cb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:37.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.608+0000 7fe506ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:38:37.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.609+0000 7fe506ffd640 1 -- 172.21.15.86:0/10974659 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510020460 con 0x557dc6be3830 2024-04-21T08:38:37.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.609+0000 7fe506ffd640 1 -- 172.21.15.86:0/10974659 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fe510020600 con 0x557dc6be3830 2024-04-21T08:38:37.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.609+0000 7fe530e14740 1 -- 172.21.15.86:0/10974659 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x557dc6b55530 con 0x557dc6be3830 2024-04-21T08:38:37.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.609+0000 7fe506ffd640 1 -- 172.21.15.86:0/10974659 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe51002e050 con 0x557dc6be3830 2024-04-21T08:38:37.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.609+0000 7fe530e14740 10 client.15456 fetch_fsmap learned FSMap version 28 2024-04-21T08:38:37.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.609+0000 7fe530e14740 1 -- 172.21.15.86:0/10974659 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557dc6bf7510 con 0x557dc6be3830 2024-04-21T08:38:37.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.609+0000 7fe506ffd640 1 -- 172.21.15.86:0/10974659 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x557dc6b55530 con 0x557dc6be3830 2024-04-21T08:38:37.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.609+0000 7fe506ffd640 1 -- 172.21.15.86:0/10974659 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe51001e610 con 0x557dc6be3830 2024-04-21T08:38:37.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.609+0000 7fe530e14740 10 client.15456 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:38:37.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.609+0000 7fe530e14740 10 client.15456 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-21T08:38:37.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.609+0000 7fe530e14740 20 client.15456 populate_metadata read hostname 'smithi086' 2024-04-21T08:38:37.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.609+0000 7fe530e14740 1 --2- 172.21.15.86:0/10974659 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x557dc6bf9c40 0x557dc6bfb0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.609+0000 7fe530e14740 4 client.15456 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:38:37.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.609+0000 7fe530e14740 1 -- 172.21.15.86:0/10974659 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557dc6b4f3a0 con 0x557dc6bf9c40 2024-04-21T08:38:37.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.610+0000 7fe5277fe640 1 --2- 172.21.15.86:0/10974659 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x557dc6bf9c40 0x557dc6bfb0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.610+0000 7fe506ffd640 10 client.15456 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:38:37.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.610+0000 7fe5277fe640 1 --2- 172.21.15.86:0/10974659 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x557dc6bf9c40 0x557dc6bfb0b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=85feb20090b80aa2 server_cookie=8ede224af7c4a8c5 in_seq=0 out_seq=0 2024-04-21T08:38:37.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.611+0000 7fe506ffd640 1 -- 172.21.15.86:0/10974659 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557dc6b4f3a0 con 0x557dc6bf9c40 2024-04-21T08:38:37.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.611+0000 7fe506ffd640 10 client.15456 handle_command_reply: tid=0 2024-04-21T08:38:37.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.611+0000 7fe530e14740 1 client.15456 shutdown 2024-04-21T08:38:37.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.611+0000 7fe530e14740 1 -- 172.21.15.86:0/10974659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dc6be3830 msgr2=0x557dc6bed0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:37.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.611+0000 7fe530e14740 1 --2- 172.21.15.86:0/10974659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dc6be3830 0x557dc6bed0f0 secure :-1 s=READY pgs=455 cs=0 l=1 rev1=1 crypto rx=0x7fe51000bfd0 tx=0x7fe510004730 comp rx=0 tx=0).stop 2024-04-21T08:38:37.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.611+0000 7fe530e14740 1 -- 172.21.15.86:0/10974659 shutdown_connections 2024-04-21T08:38:37.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.611+0000 7fe530e14740 1 --2- 172.21.15.86:0/10974659 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x557dc6bf9c40 0x557dc6bfb0b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.611+0000 7fe530e14740 1 --2- 172.21.15.86:0/10974659 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557dc6be42d0 0x557dc6bf4130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.611+0000 7fe530e14740 1 --2- 172.21.15.86:0/10974659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dc6be3830 0x557dc6bed0f0 unknown :-1 s=CLOSED pgs=455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.611+0000 7fe506ffd640 0 client.15456 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:38:37.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.611+0000 7fe530e14740 1 --2- 172.21.15.86:0/10974659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dc6be2e40 0x557dc6becbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.611+0000 7fe530e14740 1 -- 172.21.15.86:0/10974659 >> 172.21.15.86:0/10974659 conn(0x557dc6bdef40 msgr2=0x557dc6bdca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:37.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.611+0000 7fe530e14740 1 -- 172.21.15.86:0/10974659 shutdown_connections 2024-04-21T08:38:37.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.611+0000 7fe530e14740 1 -- 172.21.15.86:0/10974659 wait complete. 2024-04-21T08:38:37.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.611+0000 7fe530e14740 20 client.15456 trim_cache size 0 max 16384 2024-04-21T08:38:37.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.629+0000 7fe530e14740 1 Processor -- start 2024-04-21T08:38:37.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.629+0000 7fe530e14740 1 -- start start 2024-04-21T08:38:37.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.629+0000 7fe530e14740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557dc6be2e40 0x557dc6be7810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.629+0000 7fe530e14740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dc6be3830 0x557dc6be7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.629+0000 7fe530e14740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dc6be42d0 0x557dc6bf40f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.629+0000 7fe530e14740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557dc6bf93f0 con 0x557dc6be3830 2024-04-21T08:38:37.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.629+0000 7fe530e14740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x557dc6bf9ac0 con 0x557dc6be2e40 2024-04-21T08:38:37.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.629+0000 7fe530e14740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x557dc6bfa190 con 0x557dc6be42d0 2024-04-21T08:38:37.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.630+0000 7fe51effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dc6be3830 0x557dc6be7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.630+0000 7fe51effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dc6be3830 0x557dc6be7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60346/0 (socket says 172.21.15.86:60346) 2024-04-21T08:38:37.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.630+0000 7fe51effd640 1 -- 172.21.15.86:0/640442508 learned_addr learned my addr 172.21.15.86:0/640442508 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:37.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.630+0000 7fe5277fe640 1 --2- 172.21.15.86:0/640442508 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557dc6be2e40 0x557dc6be7810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.630+0000 7fe51effd640 1 -- 172.21.15.86:0/640442508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dc6be42d0 msgr2=0x557dc6bf40f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:37.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.630+0000 7fe51effd640 1 --2- 172.21.15.86:0/640442508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dc6be42d0 0x557dc6bf40f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.630+0000 7fe51effd640 1 -- 172.21.15.86:0/640442508 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557dc6be2e40 msgr2=0x557dc6be7810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:37.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.630+0000 7fe51effd640 1 --2- 172.21.15.86:0/640442508 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557dc6be2e40 0x557dc6be7810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.630+0000 7fe51effd640 1 -- 172.21.15.86:0/640442508 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557dc6be72f0 con 0x557dc6be3830 2024-04-21T08:38:37.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.630+0000 7fe51effd640 1 --2- 172.21.15.86:0/640442508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dc6be3830 0x557dc6be7d50 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe51000b970 tx=0x7fe510011f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efc0b7f5ebdbd72c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:37.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.630+0000 7fe51d7fa640 1 -- 172.21.15.86:0/640442508 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510011c90 con 0x557dc6be3830 2024-04-21T08:38:37.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.630+0000 7fe51d7fa640 1 -- 172.21.15.86:0/640442508 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe510025070 con 0x557dc6be3830 2024-04-21T08:38:37.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.630+0000 7fe51d7fa640 1 -- 172.21.15.86:0/640442508 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe510005540 con 0x557dc6be3830 2024-04-21T08:38:37.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.631+0000 7fe530e14740 1 -- 172.21.15.86:0/640442508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dc6be3830 msgr2=0x557dc6be7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:37.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.631+0000 7fe530e14740 1 --2- 172.21.15.86:0/640442508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dc6be3830 0x557dc6be7d50 secure :-1 s=READY pgs=456 cs=0 l=1 rev1=1 crypto rx=0x7fe51000b970 tx=0x7fe510011f80 comp rx=0 tx=0).stop 2024-04-21T08:38:37.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.631+0000 7fe530e14740 1 -- 172.21.15.86:0/640442508 shutdown_connections 2024-04-21T08:38:37.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.631+0000 7fe530e14740 1 --2- 172.21.15.86:0/640442508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dc6be42d0 0x557dc6bf40f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.631+0000 7fe530e14740 1 --2- 172.21.15.86:0/640442508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dc6be3830 0x557dc6be7d50 unknown :-1 s=CLOSED pgs=456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.631+0000 7fe530e14740 1 --2- 172.21.15.86:0/640442508 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557dc6be2e40 0x557dc6be7810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.631+0000 7fe530e14740 1 -- 172.21.15.86:0/640442508 >> 172.21.15.86:0/640442508 conn(0x557dc6bdef40 msgr2=0x557dc6bd8800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:37.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.631+0000 7fe530e14740 1 -- 172.21.15.86:0/640442508 shutdown_connections 2024-04-21T08:38:37.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.631+0000 7fe530e14740 1 -- 172.21.15.86:0/640442508 wait complete. 2024-04-21T08:38:37.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.631+0000 7fe530e14740 1 -- start start 2024-04-21T08:38:37.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.631+0000 7fe530e14740 1 Processor -- start 2024-04-21T08:38:37.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.632+0000 7fe530e14740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557dc6be2e40 0x557dc6bef050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.632+0000 7fe530e14740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dc6be3830 0x557dc6bef590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.632+0000 7fe530e14740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dc6be42d0 0x557dc6c5dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.632+0000 7fe530e14740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557dc6bfaf30 con 0x557dc6be3830 2024-04-21T08:38:37.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.632+0000 7fe530e14740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x557dc6bfa860 con 0x557dc6be2e40 2024-04-21T08:38:37.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.632+0000 7fe530e14740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x557dc6bfb600 con 0x557dc6be42d0 2024-04-21T08:38:37.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.632+0000 7fe530e14740 10 client.0 _ll_register_callbacks cb 0x7fe510012a40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:37.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.632+0000 7fe51effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dc6be3830 0x557dc6bef590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.632+0000 7fe51effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dc6be3830 0x557dc6bef590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60348/0 (socket says 172.21.15.86:60348) 2024-04-21T08:38:37.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.632+0000 7fe51effd640 1 -- 172.21.15.86:0/2728711516 learned_addr learned my addr 172.21.15.86:0/2728711516 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:37.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.632+0000 7fe5277fe640 1 --2- 172.21.15.86:0/2728711516 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557dc6be2e40 0x557dc6bef050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.632+0000 7fe527fff640 1 --2- 172.21.15.86:0/2728711516 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dc6be42d0 0x557dc6c5dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.632+0000 7fe51effd640 1 -- 172.21.15.86:0/2728711516 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dc6be42d0 msgr2=0x557dc6c5dbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:37.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.632+0000 7fe51effd640 1 --2- 172.21.15.86:0/2728711516 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dc6be42d0 0x557dc6c5dbe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.632+0000 7fe51effd640 1 -- 172.21.15.86:0/2728711516 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557dc6be2e40 msgr2=0x557dc6bef050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:37.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.632+0000 7fe51effd640 1 --2- 172.21.15.86:0/2728711516 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557dc6be2e40 0x557dc6bef050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.632+0000 7fe51effd640 1 -- 172.21.15.86:0/2728711516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557dc6c5e120 con 0x557dc6be3830 2024-04-21T08:38:37.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.632+0000 7fe5277fe640 1 --2- 172.21.15.86:0/2728711516 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557dc6be2e40 0x557dc6bef050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:37.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.632+0000 7fe527fff640 1 --2- 172.21.15.86:0/2728711516 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dc6be42d0 0x557dc6c5dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:37.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.632+0000 7fe51effd640 1 --2- 172.21.15.86:0/2728711516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dc6be3830 0x557dc6bef590 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe510000c00 tx=0x7fe51001eec0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=319fe68dab33c39a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:37.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.632+0000 7fe506ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:38:37.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.632+0000 7fe506ffd640 1 -- 172.21.15.86:0/2728711516 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe51003f020 con 0x557dc6be3830 2024-04-21T08:38:37.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.632+0000 7fe506ffd640 1 -- 172.21.15.86:0/2728711516 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fe51001e410 con 0x557dc6be3830 2024-04-21T08:38:37.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.632+0000 7fe530e14740 1 -- 172.21.15.86:0/2728711516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe510025070 con 0x557dc6be3830 2024-04-21T08:38:37.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.632+0000 7fe506ffd640 1 -- 172.21.15.86:0/2728711516 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fe51004a050 con 0x557dc6be3830 2024-04-21T08:38:37.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.632+0000 7fe530e14740 10 client.15462 fetch_fsmap learned FSMap version 28 2024-04-21T08:38:37.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.632+0000 7fe530e14740 1 -- 172.21.15.86:0/2728711516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x557dc6c5f300 con 0x557dc6be3830 2024-04-21T08:38:37.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.633+0000 7fe506ffd640 1 -- 172.21.15.86:0/2728711516 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe510025070 con 0x557dc6be3830 2024-04-21T08:38:37.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.633+0000 7fe506ffd640 1 -- 172.21.15.86:0/2728711516 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fe51003a480 con 0x557dc6be3830 2024-04-21T08:38:37.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.633+0000 7fe530e14740 10 client.15462 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:38:37.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.633+0000 7fe530e14740 10 client.15462 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-21T08:38:37.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.633+0000 7fe530e14740 20 client.15462 populate_metadata read hostname 'smithi086' 2024-04-21T08:38:37.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.633+0000 7fe530e14740 1 --2- 172.21.15.86:0/2728711516 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x557dc6c63b80 0x557dc6c62890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.633+0000 7fe530e14740 4 client.15462 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:38:37.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.633+0000 7fe530e14740 1 -- 172.21.15.86:0/2728711516 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "status"}) -- 0x557dc6acb960 con 0x557dc6c63b80 2024-04-21T08:38:37.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.633+0000 7fe5277fe640 1 --2- 172.21.15.86:0/2728711516 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x557dc6c63b80 0x557dc6c62890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.633+0000 7fe506ffd640 10 client.15462 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:38:37.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.633+0000 7fe5277fe640 1 --2- 172.21.15.86:0/2728711516 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x557dc6c63b80 0x557dc6c62890 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c6be9c2e7d084d4b server_cookie=8e77896e59e3a31f in_seq=0 out_seq=0 2024-04-21T08:38:37.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.634+0000 7fe506ffd640 1 -- 172.21.15.86:0/2728711516 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x557dc6acb960 con 0x557dc6c63b80 2024-04-21T08:38:37.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.634+0000 7fe506ffd640 10 client.15462 handle_command_reply: tid=0 2024-04-21T08:38:37.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.634+0000 7fe530e14740 1 client.15462 shutdown 2024-04-21T08:38:37.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.634+0000 7fe530e14740 1 -- 172.21.15.86:0/2728711516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dc6be3830 msgr2=0x557dc6bef590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:37.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.634+0000 7fe530e14740 1 --2- 172.21.15.86:0/2728711516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dc6be3830 0x557dc6bef590 secure :-1 s=READY pgs=457 cs=0 l=1 rev1=1 crypto rx=0x7fe510000c00 tx=0x7fe51001eec0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.634+0000 7fe530e14740 1 -- 172.21.15.86:0/2728711516 shutdown_connections 2024-04-21T08:38:37.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.634+0000 7fe530e14740 1 --2- 172.21.15.86:0/2728711516 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557dc6be42d0 0x557dc6c5dbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.634+0000 7fe530e14740 1 --2- 172.21.15.86:0/2728711516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557dc6be3830 0x557dc6bef590 unknown :-1 s=CLOSED pgs=457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.634+0000 7fe530e14740 1 --2- 172.21.15.86:0/2728711516 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x557dc6c63b80 0x557dc6c62890 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.634+0000 7fe530e14740 1 --2- 172.21.15.86:0/2728711516 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557dc6be2e40 0x557dc6bef050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.634+0000 7fe506ffd640 0 client.15462 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:38:37.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.634+0000 7fe530e14740 1 -- 172.21.15.86:0/2728711516 >> 172.21.15.86:0/2728711516 conn(0x557dc6bdef40 msgr2=0x557dc6bdd400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:37.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.634+0000 7fe530e14740 1 -- 172.21.15.86:0/2728711516 shutdown_connections 2024-04-21T08:38:37.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.634+0000 7fe530e14740 1 -- 172.21.15.86:0/2728711516 wait complete. 2024-04-21T08:38:37.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.634+0000 7fe530e14740 20 client.15462 trim_cache size 0 max 16384 2024-04-21T08:38:37.641 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:38:37.641 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "f8cec7cc-ffb8-11ee-bc93-c7b262605968", 2024-04-21T08:38:37.641 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 4, 2024-04-21T08:38:37.641 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24394, 2024-04-21T08:38:37.642 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-21T08:38:37.642 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-21T08:38:37.642 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-21T08:38:37.642 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 111.190549651, 2024-04-21T08:38:37.642 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 28, 2024-04-21T08:38:37.642 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 93, 2024-04-21T08:38:37.642 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 89, 2024-04-21T08:38:37.642 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 174.41893023700001 2024-04-21T08:38:37.642 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:38:37.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.634+0000 7fe52e225640 1 -- 172.21.15.86:0/3820896792 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe5140a3bb0 msgr2=0x7fe5140a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:37.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.634+0000 7fe52e225640 1 --2- 172.21.15.86:0/3820896792 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe5140a3bb0 0x7fe5140a6070 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7fe528199700 tx=0x7fe510014840 comp rx=0 tx=0).stop 2024-04-21T08:38:37.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.634+0000 7fe52e225640 1 -- 172.21.15.86:0/3820896792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe528103b50 msgr2=0x7fe52819f7a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:37.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.634+0000 7fe52e225640 1 --2- 172.21.15.86:0/3820896792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe528103b50 0x7fe52819f7a0 secure :-1 s=READY pgs=453 cs=0 l=1 rev1=1 crypto rx=0x7fe52002c700 tx=0x7fe520035d60 comp rx=0 tx=0).stop 2024-04-21T08:38:37.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.634+0000 7fe52e225640 1 -- 172.21.15.86:0/3820896792 shutdown_connections 2024-04-21T08:38:37.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.634+0000 7fe52e225640 1 --2- 172.21.15.86:0/3820896792 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe5140a3bb0 0x7fe5140a6070 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.634+0000 7fe52e225640 1 --2- 172.21.15.86:0/3820896792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe528103b50 0x7fe52819f7a0 unknown :-1 s=CLOSED pgs=453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.634+0000 7fe52e225640 1 --2- 172.21.15.86:0/3820896792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe528103190 0x7fe528198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.634+0000 7fe52e225640 1 --2- 172.21.15.86:0/3820896792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe528101fa0 0x7fe5281981f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.634+0000 7fe52e225640 1 -- 172.21.15.86:0/3820896792 >> 172.21.15.86:0/3820896792 conn(0x7fe5280fd530 msgr2=0x7fe528106490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:37.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.635+0000 7fe52e225640 1 -- 172.21.15.86:0/3820896792 shutdown_connections 2024-04-21T08:38:37.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.635+0000 7fe52e225640 1 -- 172.21.15.86:0/3820896792 wait complete. 2024-04-21T08:38:37.645 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-04-21T08:38:37.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:37 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2355081236' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:37.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:37 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2468116825' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:37.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:37 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1852617290' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:37.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.745+0000 7f028c225640 1 -- 172.21.15.86:0/2938991070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0284101fa0 msgr2=0x7f02841023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:37.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.745+0000 7f028c225640 1 --2- 172.21.15.86:0/2938991070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0284101fa0 0x7f02841023c0 secure :-1 s=READY pgs=458 cs=0 l=1 rev1=1 crypto rx=0x7f0274009900 tx=0x7f027402bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:37.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.745+0000 7f028c225640 1 -- 172.21.15.86:0/2938991070 shutdown_connections 2024-04-21T08:38:37.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.745+0000 7f028c225640 1 --2- 172.21.15.86:0/2938991070 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0284103b50 0x7f028410a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.745+0000 7f028c225640 1 --2- 172.21.15.86:0/2938991070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0284103190 0x7f0284103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.745+0000 7f028c225640 1 --2- 172.21.15.86:0/2938991070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0284101fa0 0x7f02841023c0 unknown :-1 s=CLOSED pgs=458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.745+0000 7f028c225640 1 -- 172.21.15.86:0/2938991070 >> 172.21.15.86:0/2938991070 conn(0x7f02840fd530 msgr2=0x7f02840ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:37.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.745+0000 7f028c225640 1 -- 172.21.15.86:0/2938991070 shutdown_connections 2024-04-21T08:38:37.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.745+0000 7f028c225640 1 -- 172.21.15.86:0/2938991070 wait complete. 2024-04-21T08:38:37.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.745+0000 7f028c225640 1 Processor -- start 2024-04-21T08:38:37.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.745+0000 7f028c225640 1 -- start start 2024-04-21T08:38:37.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.745+0000 7f028c225640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0284101fa0 0x7f02841983e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.745+0000 7f028c225640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0284103190 0x7f0284198920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.745+0000 7f028c225640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0284103b50 0x7f028419f940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.745+0000 7f028c225640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f028410dba0 con 0x7f0284101fa0 2024-04-21T08:38:37.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.745+0000 7f028c225640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f028410da20 con 0x7f0284103b50 2024-04-21T08:38:37.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.745+0000 7f028c225640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f028410dd20 con 0x7f0284103190 2024-04-21T08:38:37.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.746+0000 7f0289fc1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0284101fa0 0x7f02841983e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.746+0000 7f0289fc1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0284101fa0 0x7f02841983e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60368/0 (socket says 172.21.15.86:60368) 2024-04-21T08:38:37.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.746+0000 7f0289fc1640 1 -- 172.21.15.86:0/233773968 learned_addr learned my addr 172.21.15.86:0/233773968 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:37.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.746+0000 7f0289fc1640 1 -- 172.21.15.86:0/233773968 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0284103190 msgr2=0x7f0284198920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:37.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.746+0000 7f0289fc1640 1 --2- 172.21.15.86:0/233773968 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0284103190 0x7f0284198920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.746+0000 7f0289fc1640 1 -- 172.21.15.86:0/233773968 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0284103b50 msgr2=0x7f028419f940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:37.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.746+0000 7f0289fc1640 1 --2- 172.21.15.86:0/233773968 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0284103b50 0x7f028419f940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.746+0000 7f0289fc1640 1 -- 172.21.15.86:0/233773968 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f028419ffb0 con 0x7f0284101fa0 2024-04-21T08:38:37.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.746+0000 7f0289fc1640 1 --2- 172.21.15.86:0/233773968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0284101fa0 0x7f02841983e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0274036040 tx=0x7f02740042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:37.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.746+0000 7f027affd640 1 -- 172.21.15.86:0/233773968 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0274043020 con 0x7f0284101fa0 2024-04-21T08:38:37.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.746+0000 7f027affd640 1 -- 172.21.15.86:0/233773968 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0274004aa0 con 0x7f0284101fa0 2024-04-21T08:38:37.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.746+0000 7f028c225640 1 -- 172.21.15.86:0/233773968 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02841a0240 con 0x7f0284101fa0 2024-04-21T08:38:37.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.746+0000 7f027affd640 1 -- 172.21.15.86:0/233773968 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0274003de0 con 0x7f0284101fa0 2024-04-21T08:38:37.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.746+0000 7f028c225640 1 -- 172.21.15.86:0/233773968 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02841a0810 con 0x7f0284101fa0 2024-04-21T08:38:37.749 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:37 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2355081236' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:37.749 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:37 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2468116825' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:37.749 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:37 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1852617290' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:37.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.748+0000 7f027affd640 1 -- 172.21.15.86:0/233773968 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f0274046020 con 0x7f0284101fa0 2024-04-21T08:38:37.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.749+0000 7f028ee15740 1 Processor -- start 2024-04-21T08:38:37.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.749+0000 7f028ee15740 1 -- start start 2024-04-21T08:38:37.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.749+0000 7f028ee15740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfedda5e40 0x55bfedda6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.749+0000 7f028ee15740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bfedda6830 0x55bfedda6cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.749+0000 7f028ee15740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfedda72d0 0x55bfeddadeb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.749+0000 7f028ee15740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bfedd0ab10 con 0x55bfedda72d0 2024-04-21T08:38:37.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.749+0000 7f028ee15740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55bfedca8240 con 0x55bfedda6830 2024-04-21T08:38:37.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.749+0000 7f028ee15740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55bfedd06ad0 con 0x55bfedda5e40 2024-04-21T08:38:37.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.749+0000 7f02897c0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfedda72d0 0x55bfeddadeb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.749+0000 7f02897c0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfedda72d0 0x55bfeddadeb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60378/0 (socket says 172.21.15.86:60378) 2024-04-21T08:38:37.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.749+0000 7f02897c0640 1 -- 172.21.15.86:0/1453872234 learned_addr learned my addr 172.21.15.86:0/1453872234 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:37.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.749+0000 7f028a7c2640 1 --2- 172.21.15.86:0/1453872234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfedda5e40 0x55bfedda6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.749+0000 7f0289fc1640 1 --2- 172.21.15.86:0/1453872234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bfedda6830 0x55bfedda6cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.749+0000 7f027affd640 1 --2- 172.21.15.86:0/233773968 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f02700a3bb0 0x7f02700a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.750+0000 7f02897c0640 1 -- 172.21.15.86:0/1453872234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfedda5e40 msgr2=0x55bfedda6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:37.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.750+0000 7f02897c0640 1 --2- 172.21.15.86:0/1453872234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfedda5e40 0x55bfedda6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.750+0000 7f02897c0640 1 -- 172.21.15.86:0/1453872234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bfedda6830 msgr2=0x55bfedda6cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:37.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.750+0000 7f02897c0640 1 --2- 172.21.15.86:0/1453872234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bfedda6830 0x55bfedda6cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.750+0000 7f02897c0640 1 -- 172.21.15.86:0/1453872234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfeda1d2d0 con 0x55bfedda72d0 2024-04-21T08:38:37.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.750+0000 7f028a7c2640 1 --2- 172.21.15.86:0/233773968 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f02700a3bb0 0x7f02700a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.750+0000 7f027affd640 1 -- 172.21.15.86:0/233773968 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f02740f2c80 con 0x7f0284101fa0 2024-04-21T08:38:37.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.750+0000 7f02897c0640 1 --2- 172.21.15.86:0/1453872234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfedda72d0 0x55bfeddadeb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f02800097f0 tx=0x7f0280009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f0fb4e564440d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:37.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.750+0000 7f026ffff640 1 -- 172.21.15.86:0/1453872234 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0280015020 con 0x55bfedda72d0 2024-04-21T08:38:37.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.750+0000 7f028a7c2640 1 --2- 172.21.15.86:0/233773968 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f02700a3bb0 0x7f02700a6070 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x55bfedda1200 tx=0x7f027c004b50 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:37.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.750+0000 7f026ffff640 1 -- 172.21.15.86:0/1453872234 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0280011420 con 0x55bfedda72d0 2024-04-21T08:38:37.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.750+0000 7f026ffff640 1 -- 172.21.15.86:0/1453872234 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02800105c0 con 0x55bfedda72d0 2024-04-21T08:38:37.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.751+0000 7f028ee15740 1 -- 172.21.15.86:0/1453872234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfedda72d0 msgr2=0x55bfeddadeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:37.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.751+0000 7f028ee15740 1 --2- 172.21.15.86:0/1453872234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfedda72d0 0x55bfeddadeb0 secure :-1 s=READY pgs=460 cs=0 l=1 rev1=1 crypto rx=0x7f02800097f0 tx=0x7f0280009cb0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.751+0000 7f028ee15740 1 -- 172.21.15.86:0/1453872234 shutdown_connections 2024-04-21T08:38:37.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.751+0000 7f028ee15740 1 --2- 172.21.15.86:0/1453872234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfedda72d0 0x55bfeddadeb0 unknown :-1 s=CLOSED pgs=460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.751+0000 7f028ee15740 1 --2- 172.21.15.86:0/1453872234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bfedda6830 0x55bfedda6cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.751+0000 7f028ee15740 1 --2- 172.21.15.86:0/1453872234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfedda5e40 0x55bfedda6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.751+0000 7f028ee15740 1 -- 172.21.15.86:0/1453872234 >> 172.21.15.86:0/1453872234 conn(0x55bfedda1f40 msgr2=0x55bfedda0490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:37.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.751+0000 7f028ee15740 1 -- 172.21.15.86:0/1453872234 shutdown_connections 2024-04-21T08:38:37.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.751+0000 7f028ee15740 1 -- 172.21.15.86:0/1453872234 wait complete. 2024-04-21T08:38:37.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.751+0000 7f028ee15740 1 -- start start 2024-04-21T08:38:37.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.751+0000 7f028ee15740 1 Processor -- start 2024-04-21T08:38:37.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.752+0000 7f028ee15740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bfedda5e40 0x55bfeddafe80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.752+0000 7f028ee15740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfedda6830 0x55bfeddb03c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.752+0000 7f028ee15740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfedda72d0 0x55bfeddb7400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.752+0000 7f028ee15740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bfeddae4b0 con 0x55bfedda6830 2024-04-21T08:38:37.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.752+0000 7f028ee15740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55bfedc93190 con 0x55bfedda5e40 2024-04-21T08:38:37.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.752+0000 7f028ee15740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55bfeddae630 con 0x55bfedda72d0 2024-04-21T08:38:37.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.752+0000 7f028ee15740 10 client.0 _ll_register_callbacks cb 0x55bfedc8e8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:37.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.752+0000 7f02897c0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfedda6830 0x55bfeddb03c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.752+0000 7f02897c0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfedda6830 0x55bfeddb03c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60386/0 (socket says 172.21.15.86:60386) 2024-04-21T08:38:37.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.752+0000 7f02897c0640 1 -- 172.21.15.86:0/528129624 learned_addr learned my addr 172.21.15.86:0/528129624 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:37.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.752+0000 7f028a7c2640 1 --2- 172.21.15.86:0/528129624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfedda72d0 0x55bfeddb7400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.752+0000 7f0289fc1640 1 --2- 172.21.15.86:0/528129624 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bfedda5e40 0x55bfeddafe80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.752+0000 7f02897c0640 1 -- 172.21.15.86:0/528129624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfedda72d0 msgr2=0x55bfeddb7400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:37.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.752+0000 7f02897c0640 1 --2- 172.21.15.86:0/528129624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfedda72d0 0x55bfeddb7400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.752+0000 7f02897c0640 1 -- 172.21.15.86:0/528129624 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bfedda5e40 msgr2=0x55bfeddafe80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:37.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.752+0000 7f02897c0640 1 --2- 172.21.15.86:0/528129624 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bfedda5e40 0x55bfeddafe80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.752+0000 7f02897c0640 1 -- 172.21.15.86:0/528129624 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfedae76b0 con 0x55bfedda6830 2024-04-21T08:38:37.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.752+0000 7f028a7c2640 1 --2- 172.21.15.86:0/528129624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfedda72d0 0x55bfeddb7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:37.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.752+0000 7f0289fc1640 1 --2- 172.21.15.86:0/528129624 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bfedda5e40 0x55bfeddafe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:37.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.752+0000 7f02897c0640 1 --2- 172.21.15.86:0/528129624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfedda6830 0x55bfeddb03c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0280009380 tx=0x7f02800093b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=718b21e684927cd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:37.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.752+0000 7f026d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:38:37.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.752+0000 7f026d7fa640 1 -- 172.21.15.86:0/528129624 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0280027060 con 0x55bfedda6830 2024-04-21T08:38:37.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.752+0000 7f026d7fa640 1 -- 172.21.15.86:0/528129624 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f0280011860 con 0x55bfedda6830 2024-04-21T08:38:37.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.752+0000 7f028ee15740 1 -- 172.21.15.86:0/528129624 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0280011420 con 0x55bfedda6830 2024-04-21T08:38:37.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.752+0000 7f026d7fa640 1 -- 172.21.15.86:0/528129624 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f028002e050 con 0x55bfedda6830 2024-04-21T08:38:37.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.752+0000 7f028ee15740 10 client.15474 fetch_fsmap learned FSMap version 28 2024-04-21T08:38:37.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.752+0000 7f028ee15740 1 -- 172.21.15.86:0/528129624 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bfeddba770 con 0x55bfedda6830 2024-04-21T08:38:37.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.753+0000 7f026d7fa640 1 -- 172.21.15.86:0/528129624 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0280011420 con 0x55bfedda6830 2024-04-21T08:38:37.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.753+0000 7f026d7fa640 1 -- 172.21.15.86:0/528129624 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f028001a980 con 0x55bfedda6830 2024-04-21T08:38:37.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.753+0000 7f028ee15740 10 client.15474 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:38:37.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.753+0000 7f028ee15740 10 client.15474 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-21T08:38:37.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.753+0000 7f028ee15740 20 client.15474 populate_metadata read hostname 'smithi086' 2024-04-21T08:38:37.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.753+0000 7f028ee15740 1 --2- 172.21.15.86:0/528129624 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55bfeddbcc40 0x55bfeddbe0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.753+0000 7f028ee15740 4 client.15474 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:38:37.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.753+0000 7f028ee15740 1 -- 172.21.15.86:0/528129624 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bfedd18530 con 0x55bfeddbcc40 2024-04-21T08:38:37.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.753+0000 7f0289fc1640 1 --2- 172.21.15.86:0/528129624 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55bfeddbcc40 0x55bfeddbe0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.754+0000 7f026d7fa640 10 client.15474 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:38:37.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.754+0000 7f0289fc1640 1 --2- 172.21.15.86:0/528129624 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55bfeddbcc40 0x55bfeddbe0b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fd327d3d06b2407c server_cookie=397b60d4c037bad5 in_seq=0 out_seq=0 2024-04-21T08:38:37.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.754+0000 7f026d7fa640 1 -- 172.21.15.86:0/528129624 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bfedd18530 con 0x55bfeddbcc40 2024-04-21T08:38:37.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.754+0000 7f026d7fa640 10 client.15474 handle_command_reply: tid=0 2024-04-21T08:38:37.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.755+0000 7f028ee15740 1 client.15474 shutdown 2024-04-21T08:38:37.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.755+0000 7f028ee15740 1 -- 172.21.15.86:0/528129624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfedda6830 msgr2=0x55bfeddb03c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:37.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.755+0000 7f028ee15740 1 --2- 172.21.15.86:0/528129624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfedda6830 0x55bfeddb03c0 secure :-1 s=READY pgs=461 cs=0 l=1 rev1=1 crypto rx=0x7f0280009380 tx=0x7f02800093b0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.755+0000 7f028ee15740 1 -- 172.21.15.86:0/528129624 shutdown_connections 2024-04-21T08:38:37.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.755+0000 7f028ee15740 1 --2- 172.21.15.86:0/528129624 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55bfeddbcc40 0x55bfeddbe0b0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.755+0000 7f028ee15740 1 --2- 172.21.15.86:0/528129624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfedda72d0 0x55bfeddb7400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.755+0000 7f028ee15740 1 --2- 172.21.15.86:0/528129624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfedda6830 0x55bfeddb03c0 unknown :-1 s=CLOSED pgs=461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.755+0000 7f028ee15740 1 --2- 172.21.15.86:0/528129624 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bfedda5e40 0x55bfeddafe80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.755+0000 7f026d7fa640 0 client.15474 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:38:37.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.755+0000 7f028ee15740 1 -- 172.21.15.86:0/528129624 >> 172.21.15.86:0/528129624 conn(0x55bfedda1f40 msgr2=0x55bfedd9fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:37.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.755+0000 7f028ee15740 1 -- 172.21.15.86:0/528129624 shutdown_connections 2024-04-21T08:38:37.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.755+0000 7f028ee15740 1 -- 172.21.15.86:0/528129624 wait complete. 2024-04-21T08:38:37.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.755+0000 7f028ee15740 20 client.15474 trim_cache size 0 max 16384 2024-04-21T08:38:37.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.773+0000 7f028ee15740 1 Processor -- start 2024-04-21T08:38:37.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.773+0000 7f028ee15740 1 -- start start 2024-04-21T08:38:37.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.773+0000 7f028ee15740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bfedda5e40 0x55bfedda0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.773+0000 7f028ee15740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfedda6830 0x55bfeddb73c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.773+0000 7f028ee15740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfedda72d0 0x55bfedda0940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.773+0000 7f028ee15740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bfeddbc3f0 con 0x55bfedda72d0 2024-04-21T08:38:37.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.773+0000 7f028ee15740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55bfeddbcac0 con 0x55bfedda5e40 2024-04-21T08:38:37.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.773+0000 7f028ee15740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55bfeddbd190 con 0x55bfedda6830 2024-04-21T08:38:37.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.773+0000 7f028a7c2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfedda72d0 0x55bfedda0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.773+0000 7f028a7c2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfedda72d0 0x55bfedda0940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60398/0 (socket says 172.21.15.86:60398) 2024-04-21T08:38:37.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.773+0000 7f028a7c2640 1 -- 172.21.15.86:0/412426624 learned_addr learned my addr 172.21.15.86:0/412426624 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:37.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.773+0000 7f0289fc1640 1 --2- 172.21.15.86:0/412426624 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bfedda5e40 0x55bfedda0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.773+0000 7f02897c0640 1 --2- 172.21.15.86:0/412426624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfedda6830 0x55bfeddb73c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.773+0000 7f028a7c2640 1 -- 172.21.15.86:0/412426624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfedda6830 msgr2=0x55bfeddb73c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:37.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.773+0000 7f028a7c2640 1 --2- 172.21.15.86:0/412426624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfedda6830 0x55bfeddb73c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.773+0000 7f028a7c2640 1 -- 172.21.15.86:0/412426624 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bfedda5e40 msgr2=0x55bfedda0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:37.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.773+0000 7f028a7c2640 1 --2- 172.21.15.86:0/412426624 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bfedda5e40 0x55bfedda0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.773+0000 7f028a7c2640 1 -- 172.21.15.86:0/412426624 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bfedd9be70 con 0x55bfedda72d0 2024-04-21T08:38:37.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.773+0000 7f02897c0640 1 --2- 172.21.15.86:0/412426624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfedda6830 0x55bfeddb73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:37.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.774+0000 7f028a7c2640 1 --2- 172.21.15.86:0/412426624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfedda72d0 0x55bfedda0940 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f027c0166f0 tx=0x7f027c00d030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9733025f4ef7f4be server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:37.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.774+0000 7f026f7fe640 1 -- 172.21.15.86:0/412426624 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c015ac0 con 0x55bfedda72d0 2024-04-21T08:38:37.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.774+0000 7f026f7fe640 1 -- 172.21.15.86:0/412426624 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f027c026070 con 0x55bfedda72d0 2024-04-21T08:38:37.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.774+0000 7f026f7fe640 1 -- 172.21.15.86:0/412426624 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c014570 con 0x55bfedda72d0 2024-04-21T08:38:37.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.774+0000 7f028ee15740 1 -- 172.21.15.86:0/412426624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfedda72d0 msgr2=0x55bfedda0940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:37.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.774+0000 7f028ee15740 1 --2- 172.21.15.86:0/412426624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfedda72d0 0x55bfedda0940 secure :-1 s=READY pgs=462 cs=0 l=1 rev1=1 crypto rx=0x7f027c0166f0 tx=0x7f027c00d030 comp rx=0 tx=0).stop 2024-04-21T08:38:37.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.774+0000 7f028ee15740 1 -- 172.21.15.86:0/412426624 shutdown_connections 2024-04-21T08:38:37.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.774+0000 7f028ee15740 1 --2- 172.21.15.86:0/412426624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfedda72d0 0x55bfedda0940 unknown :-1 s=CLOSED pgs=462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.774+0000 7f028ee15740 1 --2- 172.21.15.86:0/412426624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfedda6830 0x55bfeddb73c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.774+0000 7f028ee15740 1 --2- 172.21.15.86:0/412426624 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bfedda5e40 0x55bfedda0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.774+0000 7f028ee15740 1 -- 172.21.15.86:0/412426624 >> 172.21.15.86:0/412426624 conn(0x55bfedda1f40 msgr2=0x55bfeddaa7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:37.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.775+0000 7f028ee15740 1 -- 172.21.15.86:0/412426624 shutdown_connections 2024-04-21T08:38:37.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.775+0000 7f028ee15740 1 -- 172.21.15.86:0/412426624 wait complete. 2024-04-21T08:38:37.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.775+0000 7f028ee15740 1 -- start start 2024-04-21T08:38:37.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.775+0000 7f028ee15740 1 Processor -- start 2024-04-21T08:38:37.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.775+0000 7f028ee15740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bfedda5e40 0x55bfeddb2bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.775+0000 7f028ee15740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfedda6830 0x55bfeddb3110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.775+0000 7f028ee15740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfedda72d0 0x55bfeddb5f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.775+0000 7f028ee15740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bfeddbdf30 con 0x55bfedda72d0 2024-04-21T08:38:37.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.775+0000 7f028ee15740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55bfeddbd860 con 0x55bfedda5e40 2024-04-21T08:38:37.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.775+0000 7f028ee15740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55bfeddbe600 con 0x55bfedda6830 2024-04-21T08:38:37.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.775+0000 7f028ee15740 10 client.0 _ll_register_callbacks cb 0x7f02740f6700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:37.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.775+0000 7f028a7c2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfedda72d0 0x55bfeddb5f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.775+0000 7f028a7c2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfedda72d0 0x55bfeddb5f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60402/0 (socket says 172.21.15.86:60402) 2024-04-21T08:38:37.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.775+0000 7f028a7c2640 1 -- 172.21.15.86:0/2810631564 learned_addr learned my addr 172.21.15.86:0/2810631564 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:37.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.775+0000 7f0289fc1640 1 --2- 172.21.15.86:0/2810631564 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bfedda5e40 0x55bfeddb2bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.776+0000 7f028a7c2640 1 -- 172.21.15.86:0/2810631564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfedda6830 msgr2=0x55bfeddb3110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:37.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.776+0000 7f02897c0640 1 --2- 172.21.15.86:0/2810631564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfedda6830 0x55bfeddb3110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.776+0000 7f028a7c2640 1 --2- 172.21.15.86:0/2810631564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfedda6830 0x55bfeddb3110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.776+0000 7f028a7c2640 1 -- 172.21.15.86:0/2810631564 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bfedda5e40 msgr2=0x55bfeddb2bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:37.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.776+0000 7f028a7c2640 1 --2- 172.21.15.86:0/2810631564 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bfedda5e40 0x55bfeddb2bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.776+0000 7f028a7c2640 1 -- 172.21.15.86:0/2810631564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bfeddb6590 con 0x55bfedda72d0 2024-04-21T08:38:37.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.776+0000 7f028a7c2640 1 --2- 172.21.15.86:0/2810631564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfedda72d0 0x55bfeddb5f50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f027c007070 tx=0x7f027c021ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=534f6cae399411ff server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:37.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.776+0000 7f026d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:38:37.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.776+0000 7f026d7fa640 1 -- 172.21.15.86:0/2810631564 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f027c015cd0 con 0x55bfedda72d0 2024-04-21T08:38:37.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.776+0000 7f026d7fa640 1 -- 172.21.15.86:0/2810631564 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f027c015e70 con 0x55bfedda72d0 2024-04-21T08:38:37.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.776+0000 7f028ee15740 1 -- 172.21.15.86:0/2810631564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f027c026070 con 0x55bfedda72d0 2024-04-21T08:38:37.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.776+0000 7f026d7fa640 1 -- 172.21.15.86:0/2810631564 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7f027c0323f0 con 0x55bfedda72d0 2024-04-21T08:38:37.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.776+0000 7f028ee15740 10 client.15480 fetch_fsmap learned FSMap version 28 2024-04-21T08:38:37.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.776+0000 7f028ee15740 1 -- 172.21.15.86:0/2810631564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55bfede22bf0 con 0x55bfedda72d0 2024-04-21T08:38:37.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.776+0000 7f026d7fa640 1 -- 172.21.15.86:0/2810631564 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f027c026070 con 0x55bfedda72d0 2024-04-21T08:38:37.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.776+0000 7f026d7fa640 1 -- 172.21.15.86:0/2810631564 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7f027c02a620 con 0x55bfedda72d0 2024-04-21T08:38:37.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.777+0000 7f028ee15740 10 client.15480 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:38:37.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.777+0000 7f028ee15740 10 client.15480 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-21T08:38:37.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.777+0000 7f028ee15740 20 client.15480 populate_metadata read hostname 'smithi086' 2024-04-21T08:38:37.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.777+0000 7f028ee15740 1 --2- 172.21.15.86:0/2810631564 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55bfede271d0 0x55bfede23ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.777+0000 7f028ee15740 4 client.15480 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:38:37.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.777+0000 7f028ee15740 1 -- 172.21.15.86:0/2810631564 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "status"}) -- 0x55bfedd123a0 con 0x55bfede271d0 2024-04-21T08:38:37.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.777+0000 7f0289fc1640 1 --2- 172.21.15.86:0/2810631564 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55bfede271d0 0x55bfede23ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.777+0000 7f0289fc1640 1 --2- 172.21.15.86:0/2810631564 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55bfede271d0 0x55bfede23ce0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=72547564c47e5bb7 server_cookie=c30de203fcaeb9b9 in_seq=0 out_seq=0 2024-04-21T08:38:37.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.777+0000 7f026d7fa640 10 client.15480 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:38:37.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.778+0000 7f026d7fa640 1 -- 172.21.15.86:0/2810631564 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55bfedd123a0 con 0x55bfede271d0 2024-04-21T08:38:37.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.778+0000 7f026d7fa640 10 client.15480 handle_command_reply: tid=0 2024-04-21T08:38:37.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.778+0000 7f028ee15740 1 client.15480 shutdown 2024-04-21T08:38:37.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.778+0000 7f028ee15740 1 -- 172.21.15.86:0/2810631564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfedda72d0 msgr2=0x55bfeddb5f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:37.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.778+0000 7f028ee15740 1 --2- 172.21.15.86:0/2810631564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfedda72d0 0x55bfeddb5f50 secure :-1 s=READY pgs=463 cs=0 l=1 rev1=1 crypto rx=0x7f027c007070 tx=0x7f027c021ab0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.778+0000 7f028ee15740 1 -- 172.21.15.86:0/2810631564 shutdown_connections 2024-04-21T08:38:37.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.778+0000 7f028ee15740 1 --2- 172.21.15.86:0/2810631564 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55bfede271d0 0x55bfede23ce0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.778+0000 7f028ee15740 1 --2- 172.21.15.86:0/2810631564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bfedda72d0 0x55bfeddb5f50 unknown :-1 s=CLOSED pgs=463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.778+0000 7f026d7fa640 0 client.15480 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:38:37.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.778+0000 7f028ee15740 1 --2- 172.21.15.86:0/2810631564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bfedda6830 0x55bfeddb3110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.778+0000 7f028ee15740 1 --2- 172.21.15.86:0/2810631564 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bfedda5e40 0x55bfeddb2bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.778+0000 7f028ee15740 1 -- 172.21.15.86:0/2810631564 >> 172.21.15.86:0/2810631564 conn(0x55bfedda1f40 msgr2=0x55bfeddaa7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:37.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.778+0000 7f028ee15740 1 -- 172.21.15.86:0/2810631564 shutdown_connections 2024-04-21T08:38:37.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.778+0000 7f028ee15740 1 -- 172.21.15.86:0/2810631564 wait complete. 2024-04-21T08:38:37.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.778+0000 7f028ee15740 20 client.15480 trim_cache size 0 max 16384 2024-04-21T08:38:37.785 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:38:37.785 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "f8cec7cc-ffb8-11ee-bc93-c7b262605968", 2024-04-21T08:38:37.785 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 3, 2024-04-21T08:38:37.785 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24398, 2024-04-21T08:38:37.785 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-21T08:38:37.785 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-21T08:38:37.786 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-21T08:38:37.786 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 112.34113922500001, 2024-04-21T08:38:37.786 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 28, 2024-04-21T08:38:37.786 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 93, 2024-04-21T08:38:37.786 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 89, 2024-04-21T08:38:37.786 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 171.73207313899999 2024-04-21T08:38:37.786 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:38:37.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.778+0000 7f028c225640 1 -- 172.21.15.86:0/233773968 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f02700a3bb0 msgr2=0x7f02700a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:37.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.778+0000 7f028c225640 1 --2- 172.21.15.86:0/233773968 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f02700a3bb0 0x7f02700a6070 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x55bfedda1200 tx=0x7f027c004b50 comp rx=0 tx=0).stop 2024-04-21T08:38:37.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.778+0000 7f028c225640 1 -- 172.21.15.86:0/233773968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0284101fa0 msgr2=0x7f02841983e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:37.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.778+0000 7f028c225640 1 --2- 172.21.15.86:0/233773968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0284101fa0 0x7f02841983e0 secure :-1 s=READY pgs=459 cs=0 l=1 rev1=1 crypto rx=0x7f0274036040 tx=0x7f02740042f0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.778+0000 7f028c225640 1 -- 172.21.15.86:0/233773968 shutdown_connections 2024-04-21T08:38:37.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.778+0000 7f028c225640 1 --2- 172.21.15.86:0/233773968 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f02700a3bb0 0x7f02700a6070 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.778+0000 7f028c225640 1 --2- 172.21.15.86:0/233773968 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0284103b50 0x7f028419f940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.778+0000 7f028c225640 1 --2- 172.21.15.86:0/233773968 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0284103190 0x7f0284198920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.778+0000 7f028c225640 1 --2- 172.21.15.86:0/233773968 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0284101fa0 0x7f02841983e0 unknown :-1 s=CLOSED pgs=459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.778+0000 7f028c225640 1 -- 172.21.15.86:0/233773968 >> 172.21.15.86:0/233773968 conn(0x7f02840fd530 msgr2=0x7f0284106650 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:37.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.778+0000 7f028c225640 1 -- 172.21.15.86:0/233773968 shutdown_connections 2024-04-21T08:38:37.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.779+0000 7f028c225640 1 -- 172.21.15.86:0/233773968 wait complete. 2024-04-21T08:38:37.789 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-21T08:38:37.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.888+0000 7fac5324e640 1 -- 172.21.15.86:0/2632161082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac4c102a00 msgr2=0x7fac4c109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:37.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.888+0000 7fac5324e640 1 --2- 172.21.15.86:0/2632161082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac4c102a00 0x7fac4c109320 secure :-1 s=READY pgs=464 cs=0 l=1 rev1=1 crypto rx=0x7fac44009ca0 tx=0x7fac4402c100 comp rx=0 tx=0).stop 2024-04-21T08:38:37.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.888+0000 7fac5324e640 1 -- 172.21.15.86:0/2632161082 shutdown_connections 2024-04-21T08:38:37.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.888+0000 7fac5324e640 1 --2- 172.21.15.86:0/2632161082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac4c102a00 0x7fac4c109320 unknown :-1 s=CLOSED pgs=464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.888+0000 7fac5324e640 1 --2- 172.21.15.86:0/2632161082 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac4c102040 0x7fac4c1024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.888+0000 7fac5324e640 1 --2- 172.21.15.86:0/2632161082 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac4c100e60 0x7fac4c101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.888+0000 7fac5324e640 1 -- 172.21.15.86:0/2632161082 >> 172.21.15.86:0/2632161082 conn(0x7fac4c0fc420 msgr2=0x7fac4c0fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:37.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.888+0000 7fac5324e640 1 -- 172.21.15.86:0/2632161082 shutdown_connections 2024-04-21T08:38:37.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.888+0000 7fac5324e640 1 -- 172.21.15.86:0/2632161082 wait complete. 2024-04-21T08:38:37.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.889+0000 7fac5324e640 1 Processor -- start 2024-04-21T08:38:37.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.889+0000 7fac5324e640 1 -- start start 2024-04-21T08:38:37.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.889+0000 7fac5324e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac4c100e60 0x7fac4c198110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.889+0000 7fac5324e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac4c102040 0x7fac4c198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.889+0000 7fac5324e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac4c102a00 0x7fac4c19f6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.889+0000 7fac5324e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fac4c10ca00 con 0x7fac4c102040 2024-04-21T08:38:37.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.889+0000 7fac5324e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fac4c10c880 con 0x7fac4c100e60 2024-04-21T08:38:37.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.889+0000 7fac5324e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fac4c10cb80 con 0x7fac4c102a00 2024-04-21T08:38:37.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.889+0000 7fac4bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac4c102040 0x7fac4c198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.889+0000 7fac4bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac4c102040 0x7fac4c198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60408/0 (socket says 172.21.15.86:60408) 2024-04-21T08:38:37.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.889+0000 7fac4bfff640 1 -- 172.21.15.86:0/214227480 learned_addr learned my addr 172.21.15.86:0/214227480 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:37.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.890+0000 7fac517eb640 1 --2- 172.21.15.86:0/214227480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac4c102a00 0x7fac4c19f6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.890+0000 7fac50fea640 1 --2- 172.21.15.86:0/214227480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac4c100e60 0x7fac4c198110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.890+0000 7fac4bfff640 1 -- 172.21.15.86:0/214227480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac4c102a00 msgr2=0x7fac4c19f6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:37.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.890+0000 7fac4bfff640 1 --2- 172.21.15.86:0/214227480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac4c102a00 0x7fac4c19f6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.890+0000 7fac4bfff640 1 -- 172.21.15.86:0/214227480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac4c100e60 msgr2=0x7fac4c198110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:37.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.890+0000 7fac4bfff640 1 --2- 172.21.15.86:0/214227480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac4c100e60 0x7fac4c198110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.890+0000 7fac4bfff640 1 -- 172.21.15.86:0/214227480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fac4c19fd30 con 0x7fac4c102040 2024-04-21T08:38:37.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.890+0000 7fac50fea640 1 --2- 172.21.15.86:0/214227480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac4c100e60 0x7fac4c198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:37.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.890+0000 7fac4bfff640 1 --2- 172.21.15.86:0/214227480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac4c102040 0x7fac4c198650 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fac4000d770 tx=0x7fac4000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:37.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.890+0000 7fac49ffb640 1 -- 172.21.15.86:0/214227480 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac40017070 con 0x7fac4c102040 2024-04-21T08:38:37.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.890+0000 7fac49ffb640 1 -- 172.21.15.86:0/214227480 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac400054d0 con 0x7fac4c102040 2024-04-21T08:38:37.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.890+0000 7fac5324e640 1 -- 172.21.15.86:0/214227480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fac4c19ffc0 con 0x7fac4c102040 2024-04-21T08:38:37.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.890+0000 7fac49ffb640 1 -- 172.21.15.86:0/214227480 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac40004c30 con 0x7fac4c102040 2024-04-21T08:38:37.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.890+0000 7fac5324e640 1 -- 172.21.15.86:0/214227480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fac4c1a04e0 con 0x7fac4c102040 2024-04-21T08:38:37.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.892+0000 7fac49ffb640 1 -- 172.21.15.86:0/214227480 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fac40004e10 con 0x7fac4c102040 2024-04-21T08:38:37.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.893+0000 7fac55e43740 1 Processor -- start 2024-04-21T08:38:37.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.893+0000 7fac55e43740 1 -- start start 2024-04-21T08:38:37.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.893+0000 7fac49ffb640 1 --2- 172.21.15.86:0/214227480 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fac340a3c30 0x7fac340a60f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.893+0000 7fac55e43740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b073cade40 0x55b073cae260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.893+0000 7fac55e43740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b073cae830 0x55b073caecb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.893+0000 7fac55e43740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b073caf2d0 0x55b073cb5eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.893+0000 7fac55e43740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b073c12b10 con 0x55b073cae830 2024-04-21T08:38:37.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.893+0000 7fac55e43740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b073bb0240 con 0x55b073caf2d0 2024-04-21T08:38:37.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.893+0000 7fac55e43740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b073c0ead0 con 0x55b073cade40 2024-04-21T08:38:37.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.893+0000 7fac517eb640 1 --2- 172.21.15.86:0/214227480 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fac340a3c30 0x7fac340a60f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.893+0000 7fac4bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b073cae830 0x55b073caecb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.893+0000 7fac4bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b073cae830 0x55b073caecb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60424/0 (socket says 172.21.15.86:60424) 2024-04-21T08:38:37.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.893+0000 7fac4bfff640 1 -- 172.21.15.86:0/3275227692 learned_addr learned my addr 172.21.15.86:0/3275227692 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:37.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.893+0000 7fac49ffb640 1 -- 172.21.15.86:0/214227480 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fac400d02b0 con 0x7fac4c102040 2024-04-21T08:38:37.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.893+0000 7fac50fea640 1 --2- 172.21.15.86:0/3275227692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b073cade40 0x55b073cae260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.893+0000 7fac517eb640 1 --2- 172.21.15.86:0/3275227692 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b073caf2d0 0x55b073cb5eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.893+0000 7fac4bfff640 1 -- 172.21.15.86:0/3275227692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b073cade40 msgr2=0x55b073cae260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:37.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.893+0000 7fac4bfff640 1 --2- 172.21.15.86:0/3275227692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b073cade40 0x55b073cae260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.900 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.893+0000 7fac4bfff640 1 -- 172.21.15.86:0/3275227692 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b073caf2d0 msgr2=0x55b073cb5eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:37.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.893+0000 7fac4bfff640 1 --2- 172.21.15.86:0/3275227692 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b073caf2d0 0x55b073cb5eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.894+0000 7fac4bfff640 1 -- 172.21.15.86:0/3275227692 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b0739252d0 con 0x55b073cae830 2024-04-21T08:38:37.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.894+0000 7fac517eb640 1 --2- 172.21.15.86:0/214227480 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fac340a3c30 0x7fac340a60f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fac44032380 tx=0x7fac44032f30 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:37.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.894+0000 7fac4bfff640 1 --2- 172.21.15.86:0/3275227692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b073cae830 0x55b073caecb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fac4009c130 tx=0x7fac400209b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=294da54a2c07530e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:37.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.894+0000 7fac32ffd640 1 -- 172.21.15.86:0/3275227692 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4009c8a0 con 0x55b073cae830 2024-04-21T08:38:37.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.894+0000 7fac32ffd640 1 -- 172.21.15.86:0/3275227692 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac4009c1a0 con 0x55b073cae830 2024-04-21T08:38:37.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.894+0000 7fac32ffd640 1 -- 172.21.15.86:0/3275227692 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac4009a530 con 0x55b073cae830 2024-04-21T08:38:37.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.894+0000 7fac55e43740 1 -- 172.21.15.86:0/3275227692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b073cae830 msgr2=0x55b073caecb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:37.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.894+0000 7fac55e43740 1 --2- 172.21.15.86:0/3275227692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b073cae830 0x55b073caecb0 secure :-1 s=READY pgs=466 cs=0 l=1 rev1=1 crypto rx=0x7fac4009c130 tx=0x7fac400209b0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.895+0000 7fac55e43740 1 -- 172.21.15.86:0/3275227692 shutdown_connections 2024-04-21T08:38:37.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.895+0000 7fac55e43740 1 --2- 172.21.15.86:0/3275227692 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b073caf2d0 0x55b073cb5eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.895+0000 7fac55e43740 1 --2- 172.21.15.86:0/3275227692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b073cae830 0x55b073caecb0 unknown :-1 s=CLOSED pgs=466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.895+0000 7fac55e43740 1 --2- 172.21.15.86:0/3275227692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b073cade40 0x55b073cae260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.895+0000 7fac55e43740 1 -- 172.21.15.86:0/3275227692 >> 172.21.15.86:0/3275227692 conn(0x55b073ca9f40 msgr2=0x55b073ca8490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:37.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.895+0000 7fac55e43740 1 -- 172.21.15.86:0/3275227692 shutdown_connections 2024-04-21T08:38:37.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.895+0000 7fac55e43740 1 -- 172.21.15.86:0/3275227692 wait complete. 2024-04-21T08:38:37.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.895+0000 7fac55e43740 1 -- start start 2024-04-21T08:38:37.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.895+0000 7fac55e43740 1 Processor -- start 2024-04-21T08:38:37.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.895+0000 7fac55e43740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b073cade40 0x55b073cb7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.895+0000 7fac55e43740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b073cae830 0x55b073cb83f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.895+0000 7fac55e43740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b073caf2d0 0x55b073cbf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.895+0000 7fac55e43740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b073cb64b0 con 0x55b073cade40 2024-04-21T08:38:37.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.895+0000 7fac55e43740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b073b9b190 con 0x55b073caf2d0 2024-04-21T08:38:37.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.895+0000 7fac55e43740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b073cb6630 con 0x55b073cae830 2024-04-21T08:38:37.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.895+0000 7fac55e43740 10 client.0 _ll_register_callbacks cb 0x55b073b968b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:37.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.895+0000 7fac50fea640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b073cade40 0x55b073cb7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.895+0000 7fac50fea640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b073cade40 0x55b073cb7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60436/0 (socket says 172.21.15.86:60436) 2024-04-21T08:38:37.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.895+0000 7fac50fea640 1 -- 172.21.15.86:0/3788962904 learned_addr learned my addr 172.21.15.86:0/3788962904 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:37.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.896+0000 7fac4bfff640 1 --2- 172.21.15.86:0/3788962904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b073cae830 0x55b073cb83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.896+0000 7fac517eb640 1 --2- 172.21.15.86:0/3788962904 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b073caf2d0 0x55b073cbf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.896+0000 7fac50fea640 1 -- 172.21.15.86:0/3788962904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b073cae830 msgr2=0x55b073cb83f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:37.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.896+0000 7fac50fea640 1 --2- 172.21.15.86:0/3788962904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b073cae830 0x55b073cb83f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.896+0000 7fac50fea640 1 -- 172.21.15.86:0/3788962904 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b073caf2d0 msgr2=0x55b073cbf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:37.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.896+0000 7fac50fea640 1 --2- 172.21.15.86:0/3788962904 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b073caf2d0 0x55b073cbf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.896+0000 7fac50fea640 1 -- 172.21.15.86:0/3788962904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b0739ef6b0 con 0x55b073cade40 2024-04-21T08:38:37.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.896+0000 7fac517eb640 1 --2- 172.21.15.86:0/3788962904 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b073caf2d0 0x55b073cbf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:37.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.896+0000 7fac50fea640 1 --2- 172.21.15.86:0/3788962904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b073cade40 0x55b073cb7eb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fac3c010840 tx=0x7fac3c010d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a75e86b3c04f0d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:37.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.896+0000 7fac2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:38:37.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.896+0000 7fac2bfff640 1 -- 172.21.15.86:0/3788962904 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c004440 con 0x55b073cade40 2024-04-21T08:38:37.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.896+0000 7fac55e43740 1 -- 172.21.15.86:0/3788962904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac4009c1a0 con 0x55b073cade40 2024-04-21T08:38:37.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.896+0000 7fac2bfff640 1 -- 172.21.15.86:0/3788962904 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fac3c00e930 con 0x55b073cade40 2024-04-21T08:38:37.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.896+0000 7fac2bfff640 1 -- 172.21.15.86:0/3788962904 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fac3c00eb30 con 0x55b073cade40 2024-04-21T08:38:37.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.896+0000 7fac55e43740 10 client.15492 fetch_fsmap learned FSMap version 28 2024-04-21T08:38:37.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.896+0000 7fac55e43740 1 -- 172.21.15.86:0/3788962904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b073cc28c0 con 0x55b073cade40 2024-04-21T08:38:37.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.896+0000 7fac2bfff640 1 -- 172.21.15.86:0/3788962904 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac4009c1a0 con 0x55b073cade40 2024-04-21T08:38:37.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.896+0000 7fac2bfff640 1 -- 172.21.15.86:0/3788962904 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fac3c017070 con 0x55b073cade40 2024-04-21T08:38:37.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.897+0000 7fac55e43740 10 client.15492 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:38:37.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.897+0000 7fac55e43740 10 client.15492 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-21T08:38:37.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.897+0000 7fac55e43740 20 client.15492 populate_metadata read hostname 'smithi086' 2024-04-21T08:38:37.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.897+0000 7fac55e43740 1 --2- 172.21.15.86:0/3788962904 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55b073cc5030 0x55b073cc64a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.897+0000 7fac55e43740 4 client.15492 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:38:37.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.897+0000 7fac55e43740 1 -- 172.21.15.86:0/3788962904 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b073c20530 con 0x55b073cc5030 2024-04-21T08:38:37.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.897+0000 7fac4bfff640 1 --2- 172.21.15.86:0/3788962904 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55b073cc5030 0x55b073cc64a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.897+0000 7fac2bfff640 10 client.15492 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:38:37.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.897+0000 7fac4bfff640 1 --2- 172.21.15.86:0/3788962904 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55b073cc5030 0x55b073cc64a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=803832f8d2563465 server_cookie=f9aa4b96cfebf0e1 in_seq=0 out_seq=0 2024-04-21T08:38:37.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.898+0000 7fac2bfff640 1 -- 172.21.15.86:0/3788962904 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b073c20530 con 0x55b073cc5030 2024-04-21T08:38:37.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.898+0000 7fac2bfff640 10 client.15492 handle_command_reply: tid=0 2024-04-21T08:38:37.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.898+0000 7fac55e43740 1 client.15492 shutdown 2024-04-21T08:38:37.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.898+0000 7fac55e43740 1 -- 172.21.15.86:0/3788962904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b073cade40 msgr2=0x55b073cb7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:37.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.898+0000 7fac55e43740 1 --2- 172.21.15.86:0/3788962904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b073cade40 0x55b073cb7eb0 secure :-1 s=READY pgs=467 cs=0 l=1 rev1=1 crypto rx=0x7fac3c010840 tx=0x7fac3c010d00 comp rx=0 tx=0).stop 2024-04-21T08:38:37.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.898+0000 7fac55e43740 1 -- 172.21.15.86:0/3788962904 shutdown_connections 2024-04-21T08:38:37.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.898+0000 7fac55e43740 1 --2- 172.21.15.86:0/3788962904 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55b073cc5030 0x55b073cc64a0 crc :-1 s=READY pgs=17 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.898+0000 7fac55e43740 1 --2- 172.21.15.86:0/3788962904 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b073caf2d0 0x55b073cbf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.898+0000 7fac55e43740 1 --2- 172.21.15.86:0/3788962904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b073cae830 0x55b073cb83f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.898+0000 7fac2bfff640 0 client.15492 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:38:37.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.898+0000 7fac55e43740 1 --2- 172.21.15.86:0/3788962904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b073cade40 0x55b073cb7eb0 unknown :-1 s=CLOSED pgs=467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.898+0000 7fac55e43740 1 -- 172.21.15.86:0/3788962904 >> 172.21.15.86:0/3788962904 conn(0x55b073ca9f40 msgr2=0x55b073ca7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:37.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.898+0000 7fac55e43740 1 -- 172.21.15.86:0/3788962904 shutdown_connections 2024-04-21T08:38:37.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.898+0000 7fac55e43740 1 -- 172.21.15.86:0/3788962904 wait complete. 2024-04-21T08:38:37.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.899+0000 7fac55e43740 20 client.15492 trim_cache size 0 max 16384 2024-04-21T08:38:37.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.916+0000 7fac55e43740 1 Processor -- start 2024-04-21T08:38:37.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.916+0000 7fac55e43740 1 -- start start 2024-04-21T08:38:37.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.916+0000 7fac55e43740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b073cade40 0x55b073ca8400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.916+0000 7fac55e43740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b073cae830 0x55b073ca8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.916+0000 7fac55e43740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b073caf2d0 0x55b073cbf350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.916+0000 7fac55e43740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b073cc47e0 con 0x55b073cae830 2024-04-21T08:38:37.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.916+0000 7fac55e43740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b073cc4eb0 con 0x55b073caf2d0 2024-04-21T08:38:37.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.916+0000 7fac55e43740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b073cc5580 con 0x55b073cade40 2024-04-21T08:38:37.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.916+0000 7fac4bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b073cae830 0x55b073ca8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.916+0000 7fac4bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b073cae830 0x55b073ca8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60438/0 (socket says 172.21.15.86:60438) 2024-04-21T08:38:37.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.916+0000 7fac4bfff640 1 -- 172.21.15.86:0/1658423449 learned_addr learned my addr 172.21.15.86:0/1658423449 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:37.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.916+0000 7fac517eb640 1 --2- 172.21.15.86:0/1658423449 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b073caf2d0 0x55b073cbf350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.917+0000 7fac4bfff640 1 -- 172.21.15.86:0/1658423449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b073cade40 msgr2=0x55b073ca8400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:37.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.917+0000 7fac4bfff640 1 --2- 172.21.15.86:0/1658423449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b073cade40 0x55b073ca8400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.917+0000 7fac4bfff640 1 -- 172.21.15.86:0/1658423449 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b073caf2d0 msgr2=0x55b073cbf350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:37.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.917+0000 7fac4bfff640 1 --2- 172.21.15.86:0/1658423449 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b073caf2d0 0x55b073cbf350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.917+0000 7fac4bfff640 1 -- 172.21.15.86:0/1658423449 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b073cc67d0 con 0x55b073cae830 2024-04-21T08:38:37.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.917+0000 7fac4bfff640 1 --2- 172.21.15.86:0/1658423449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b073cae830 0x55b073ca8940 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fac4009bea0 tx=0x7fac40018ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c04a2d6d8ab6521 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:37.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.917+0000 7fac327fc640 1 -- 172.21.15.86:0/1658423449 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac400ba070 con 0x55b073cae830 2024-04-21T08:38:37.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.917+0000 7fac327fc640 1 -- 172.21.15.86:0/1658423449 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac400b4070 con 0x55b073cae830 2024-04-21T08:38:37.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.917+0000 7fac327fc640 1 -- 172.21.15.86:0/1658423449 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac400a4e50 con 0x55b073cae830 2024-04-21T08:38:37.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.917+0000 7fac55e43740 1 -- 172.21.15.86:0/1658423449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b073cae830 msgr2=0x55b073ca8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:37.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.917+0000 7fac55e43740 1 --2- 172.21.15.86:0/1658423449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b073cae830 0x55b073ca8940 secure :-1 s=READY pgs=468 cs=0 l=1 rev1=1 crypto rx=0x7fac4009bea0 tx=0x7fac40018ad0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.917+0000 7fac55e43740 1 -- 172.21.15.86:0/1658423449 shutdown_connections 2024-04-21T08:38:37.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.917+0000 7fac55e43740 1 --2- 172.21.15.86:0/1658423449 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b073caf2d0 0x55b073cbf350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.917+0000 7fac55e43740 1 --2- 172.21.15.86:0/1658423449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b073cae830 0x55b073ca8940 unknown :-1 s=CLOSED pgs=468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.917+0000 7fac55e43740 1 --2- 172.21.15.86:0/1658423449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b073cade40 0x55b073ca8400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.917+0000 7fac55e43740 1 -- 172.21.15.86:0/1658423449 >> 172.21.15.86:0/1658423449 conn(0x55b073ca9f40 msgr2=0x55b073cb27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:37.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.918+0000 7fac55e43740 1 -- 172.21.15.86:0/1658423449 shutdown_connections 2024-04-21T08:38:37.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.918+0000 7fac55e43740 1 -- 172.21.15.86:0/1658423449 wait complete. 2024-04-21T08:38:37.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.918+0000 7fac55e43740 1 -- start start 2024-04-21T08:38:37.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.918+0000 7fac55e43740 1 Processor -- start 2024-04-21T08:38:37.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.918+0000 7fac55e43740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b073cade40 0x55b073cbab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.918+0000 7fac55e43740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b073cae830 0x55b073cbb040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.918+0000 7fac55e43740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b073caf2d0 0x55b073cbe010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.918+0000 7fac55e43740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b073cc6320 con 0x55b073cade40 2024-04-21T08:38:37.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.918+0000 7fac55e43740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b073cc5c50 con 0x55b073cae830 2024-04-21T08:38:37.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.918+0000 7fac55e43740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b073cc69f0 con 0x55b073caf2d0 2024-04-21T08:38:37.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.918+0000 7fac55e43740 10 client.0 _ll_register_callbacks cb 0x7fac3c00b710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:37.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.918+0000 7fac50fea640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b073cade40 0x55b073cbab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.918+0000 7fac50fea640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b073cade40 0x55b073cbab00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60442/0 (socket says 172.21.15.86:60442) 2024-04-21T08:38:37.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.918+0000 7fac50fea640 1 -- 172.21.15.86:0/4260202381 learned_addr learned my addr 172.21.15.86:0/4260202381 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:37.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.918+0000 7fac4bfff640 1 --2- 172.21.15.86:0/4260202381 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b073cae830 0x55b073cbb040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.919+0000 7fac50fea640 1 -- 172.21.15.86:0/4260202381 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b073caf2d0 msgr2=0x55b073cbe010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:37.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.919+0000 7fac50fea640 1 --2- 172.21.15.86:0/4260202381 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b073caf2d0 0x55b073cbe010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.919+0000 7fac50fea640 1 -- 172.21.15.86:0/4260202381 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b073cae830 msgr2=0x55b073cbb040 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:37.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.919+0000 7fac50fea640 1 --2- 172.21.15.86:0/4260202381 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b073cae830 0x55b073cbb040 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.919+0000 7fac50fea640 1 -- 172.21.15.86:0/4260202381 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b073cbe600 con 0x55b073cade40 2024-04-21T08:38:37.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.919+0000 7fac50fea640 1 --2- 172.21.15.86:0/4260202381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b073cade40 0x55b073cbab00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fac3c0196a0 tx=0x7fac3c019b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f161e6075f061e67 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:37.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.919+0000 7fac2bfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:38:37.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.919+0000 7fac2bfff640 1 -- 172.21.15.86:0/4260202381 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fac3c02d060 con 0x55b073cade40 2024-04-21T08:38:37.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.919+0000 7fac2bfff640 1 -- 172.21.15.86:0/4260202381 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fac3c004440 con 0x55b073cade40 2024-04-21T08:38:37.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.919+0000 7fac55e43740 1 -- 172.21.15.86:0/4260202381 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fac400b4070 con 0x55b073cade40 2024-04-21T08:38:37.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.919+0000 7fac2bfff640 1 -- 172.21.15.86:0/4260202381 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fac3c00e990 con 0x55b073cade40 2024-04-21T08:38:37.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.919+0000 7fac55e43740 10 client.15498 fetch_fsmap learned FSMap version 28 2024-04-21T08:38:37.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.919+0000 7fac55e43740 1 -- 172.21.15.86:0/4260202381 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55b073d2afe0 con 0x55b073cade40 2024-04-21T08:38:37.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.919+0000 7fac2bfff640 1 -- 172.21.15.86:0/4260202381 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fac400b4070 con 0x55b073cade40 2024-04-21T08:38:37.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.919+0000 7fac2bfff640 1 -- 172.21.15.86:0/4260202381 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fac3c017070 con 0x55b073cade40 2024-04-21T08:38:37.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.919+0000 7fac55e43740 10 client.15498 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:38:37.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.919+0000 7fac55e43740 10 client.15498 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-21T08:38:37.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.919+0000 7fac55e43740 20 client.15498 populate_metadata read hostname 'smithi086' 2024-04-21T08:38:37.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.919+0000 7fac55e43740 1 --2- 172.21.15.86:0/4260202381 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55b073d2fbf0 0x55b073d2e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:37.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.919+0000 7fac55e43740 4 client.15498 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:38:37.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.919+0000 7fac55e43740 1 -- 172.21.15.86:0/4260202381 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "status"}) -- 0x55b073c1a3a0 con 0x55b073d2fbf0 2024-04-21T08:38:37.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.920+0000 7fac4bfff640 1 --2- 172.21.15.86:0/4260202381 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55b073d2fbf0 0x55b073d2e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:37.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.920+0000 7fac4bfff640 1 --2- 172.21.15.86:0/4260202381 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55b073d2fbf0 0x55b073d2e900 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ee4143ee44a43352 server_cookie=547f3642858b13fe in_seq=0 out_seq=0 2024-04-21T08:38:37.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.920+0000 7fac2bfff640 10 client.15498 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:38:37.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.920+0000 7fac2bfff640 1 -- 172.21.15.86:0/4260202381 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55b073c1a3a0 con 0x55b073d2fbf0 2024-04-21T08:38:37.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.920+0000 7fac2bfff640 10 client.15498 handle_command_reply: tid=0 2024-04-21T08:38:37.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.920+0000 7fac55e43740 1 client.15498 shutdown 2024-04-21T08:38:37.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.920+0000 7fac55e43740 1 -- 172.21.15.86:0/4260202381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b073cade40 msgr2=0x55b073cbab00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:37.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.920+0000 7fac55e43740 1 --2- 172.21.15.86:0/4260202381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b073cade40 0x55b073cbab00 secure :-1 s=READY pgs=469 cs=0 l=1 rev1=1 crypto rx=0x7fac3c0196a0 tx=0x7fac3c019b40 comp rx=0 tx=0).stop 2024-04-21T08:38:37.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.920+0000 7fac55e43740 1 -- 172.21.15.86:0/4260202381 shutdown_connections 2024-04-21T08:38:37.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.920+0000 7fac55e43740 1 --2- 172.21.15.86:0/4260202381 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55b073d2fbf0 0x55b073d2e900 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.920+0000 7fac55e43740 1 --2- 172.21.15.86:0/4260202381 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b073caf2d0 0x55b073cbe010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.920+0000 7fac2bfff640 0 client.15498 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:38:37.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.920+0000 7fac55e43740 1 --2- 172.21.15.86:0/4260202381 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b073cae830 0x55b073cbb040 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.920+0000 7fac55e43740 1 --2- 172.21.15.86:0/4260202381 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b073cade40 0x55b073cbab00 unknown :-1 s=CLOSED pgs=469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.920+0000 7fac55e43740 1 -- 172.21.15.86:0/4260202381 >> 172.21.15.86:0/4260202381 conn(0x55b073ca9f40 msgr2=0x55b073cb27e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:37.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.920+0000 7fac55e43740 1 -- 172.21.15.86:0/4260202381 shutdown_connections 2024-04-21T08:38:37.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.920+0000 7fac55e43740 1 -- 172.21.15.86:0/4260202381 wait complete. 2024-04-21T08:38:37.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.921+0000 7fac55e43740 20 client.15498 trim_cache size 0 max 16384 2024-04-21T08:38:37.927 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:38:37.927 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "f8cec7cc-ffb8-11ee-bc93-c7b262605968", 2024-04-21T08:38:37.927 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 2, 2024-04-21T08:38:37.927 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24455, 2024-04-21T08:38:37.927 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-21T08:38:37.927 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-21T08:38:37.927 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-21T08:38:37.927 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 112.529392676, 2024-04-21T08:38:37.927 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 28, 2024-04-21T08:38:37.928 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 93, 2024-04-21T08:38:37.928 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 89, 2024-04-21T08:38:37.928 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 154.77658488099999 2024-04-21T08:38:37.928 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:38:37.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.921+0000 7fac5324e640 1 -- 172.21.15.86:0/214227480 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fac340a3c30 msgr2=0x7fac340a60f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:37.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.921+0000 7fac5324e640 1 --2- 172.21.15.86:0/214227480 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fac340a3c30 0x7fac340a60f0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7fac44032380 tx=0x7fac44032f30 comp rx=0 tx=0).stop 2024-04-21T08:38:37.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.921+0000 7fac5324e640 1 -- 172.21.15.86:0/214227480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac4c102040 msgr2=0x7fac4c198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:37.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.921+0000 7fac5324e640 1 --2- 172.21.15.86:0/214227480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac4c102040 0x7fac4c198650 secure :-1 s=READY pgs=465 cs=0 l=1 rev1=1 crypto rx=0x7fac4000d770 tx=0x7fac4000dc30 comp rx=0 tx=0).stop 2024-04-21T08:38:37.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.921+0000 7fac5324e640 1 -- 172.21.15.86:0/214227480 shutdown_connections 2024-04-21T08:38:37.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.921+0000 7fac5324e640 1 --2- 172.21.15.86:0/214227480 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fac340a3c30 0x7fac340a60f0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.921+0000 7fac5324e640 1 --2- 172.21.15.86:0/214227480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fac4c102a00 0x7fac4c19f6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.921+0000 7fac5324e640 1 --2- 172.21.15.86:0/214227480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fac4c102040 0x7fac4c198650 unknown :-1 s=CLOSED pgs=465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.921+0000 7fac5324e640 1 --2- 172.21.15.86:0/214227480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fac4c100e60 0x7fac4c198110 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:37.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.921+0000 7fac5324e640 1 -- 172.21.15.86:0/214227480 >> 172.21.15.86:0/214227480 conn(0x7fac4c0fc420 msgr2=0x7fac4c06a550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:37.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.921+0000 7fac5324e640 1 -- 172.21.15.86:0/214227480 shutdown_connections 2024-04-21T08:38:37.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:37.921+0000 7fac5324e640 1 -- 172.21.15.86:0/214227480 wait complete. 2024-04-21T08:38:37.931 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-21T08:38:38.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.032+0000 7fa61a6c1640 1 -- 172.21.15.86:0/364742405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa614102a00 msgr2=0x7fa614109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:38.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.032+0000 7fa61a6c1640 1 --2- 172.21.15.86:0/364742405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa614102a00 0x7fa614109320 secure :-1 s=READY pgs=470 cs=0 l=1 rev1=1 crypto rx=0x7fa60c009ca0 tx=0x7fa60c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.032+0000 7fa61a6c1640 1 -- 172.21.15.86:0/364742405 shutdown_connections 2024-04-21T08:38:38.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.032+0000 7fa61a6c1640 1 --2- 172.21.15.86:0/364742405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa614102a00 0x7fa614109320 unknown :-1 s=CLOSED pgs=470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.032+0000 7fa61a6c1640 1 --2- 172.21.15.86:0/364742405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa614102040 0x7fa6141024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.032+0000 7fa61a6c1640 1 --2- 172.21.15.86:0/364742405 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa614100e60 0x7fa614101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.032+0000 7fa61a6c1640 1 -- 172.21.15.86:0/364742405 >> 172.21.15.86:0/364742405 conn(0x7fa6140fc420 msgr2=0x7fa6140fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:38.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.032+0000 7fa61a6c1640 1 -- 172.21.15.86:0/364742405 shutdown_connections 2024-04-21T08:38:38.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.032+0000 7fa61a6c1640 1 -- 172.21.15.86:0/364742405 wait complete. 2024-04-21T08:38:38.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.032+0000 7fa61a6c1640 1 Processor -- start 2024-04-21T08:38:38.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.032+0000 7fa61a6c1640 1 -- start start 2024-04-21T08:38:38.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.033+0000 7fa61a6c1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa614100e60 0x7fa614196fd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.033+0000 7fa61a6c1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa614102040 0x7fa614197510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.033+0000 7fa61a6c1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa614102a00 0x7fa61419e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.033+0000 7fa61a6c1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa61410c9f0 con 0x7fa614102a00 2024-04-21T08:38:38.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.033+0000 7fa61a6c1640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa61410c870 con 0x7fa614102040 2024-04-21T08:38:38.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.033+0000 7fa61a6c1640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa61410cb70 con 0x7fa614100e60 2024-04-21T08:38:38.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.033+0000 7fa618c5e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa614102a00 0x7fa61419e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:38.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.033+0000 7fa618c5e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa614102a00 0x7fa61419e580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60450/0 (socket says 172.21.15.86:60450) 2024-04-21T08:38:38.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.033+0000 7fa618c5e640 1 -- 172.21.15.86:0/3897864055 learned_addr learned my addr 172.21.15.86:0/3897864055 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:38.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.033+0000 7fa618c5e640 1 -- 172.21.15.86:0/3897864055 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa614100e60 msgr2=0x7fa614196fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:38.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.033+0000 7fa618c5e640 1 --2- 172.21.15.86:0/3897864055 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa614100e60 0x7fa614196fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.033+0000 7fa618c5e640 1 -- 172.21.15.86:0/3897864055 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa614102040 msgr2=0x7fa614197510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:38.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.033+0000 7fa618c5e640 1 --2- 172.21.15.86:0/3897864055 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa614102040 0x7fa614197510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.033+0000 7fa618c5e640 1 -- 172.21.15.86:0/3897864055 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa61419ebf0 con 0x7fa614102a00 2024-04-21T08:38:38.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.033+0000 7fa618c5e640 1 --2- 172.21.15.86:0/3897864055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa614102a00 0x7fa61419e580 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa60c02c700 tx=0x7fa60c035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:38.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.033+0000 7fa6117fa640 1 -- 172.21.15.86:0/3897864055 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60c039040 con 0x7fa614102a00 2024-04-21T08:38:38.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.033+0000 7fa6117fa640 1 -- 172.21.15.86:0/3897864055 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa60c044070 con 0x7fa614102a00 2024-04-21T08:38:38.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.033+0000 7fa61a6c1640 1 -- 172.21.15.86:0/3897864055 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa61419ee80 con 0x7fa614102a00 2024-04-21T08:38:38.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.033+0000 7fa6117fa640 1 -- 172.21.15.86:0/3897864055 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60c004440 con 0x7fa614102a00 2024-04-21T08:38:38.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.033+0000 7fa61a6c1640 1 -- 172.21.15.86:0/3897864055 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa61419f340 con 0x7fa614102a00 2024-04-21T08:38:38.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.035+0000 7fa6117fa640 1 -- 172.21.15.86:0/3897864055 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa60c0045e0 con 0x7fa614102a00 2024-04-21T08:38:38.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.036+0000 7fa61d390740 1 Processor -- start 2024-04-21T08:38:38.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.036+0000 7fa61d390740 1 -- start start 2024-04-21T08:38:38.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.036+0000 7fa61d390740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576aa2efe40 0x5576aa2f0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.036+0000 7fa61d390740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576aa2f0830 0x5576aa2f0cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.036+0000 7fa61d390740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576aa2f12d0 0x5576aa2f7eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.036+0000 7fa61d390740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5576aa254b10 con 0x5576aa2f12d0 2024-04-21T08:38:38.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.036+0000 7fa61d390740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5576aa1f2240 con 0x5576aa2f0830 2024-04-21T08:38:38.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.036+0000 7fa61d390740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5576aa250ad0 con 0x5576aa2efe40 2024-04-21T08:38:38.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.036+0000 7fa6117fa640 1 --2- 172.21.15.86:0/3897864055 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa5fc0a3bb0 0x7fa5fc0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.036+0000 7fa6137fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576aa2f12d0 0x5576aa2f7eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:38.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.036+0000 7fa6137fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576aa2f12d0 0x5576aa2f7eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60454/0 (socket says 172.21.15.86:60454) 2024-04-21T08:38:38.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.036+0000 7fa618c5e640 1 --2- 172.21.15.86:0/3897864055 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa5fc0a3bb0 0x7fa5fc0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:38.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.036+0000 7fa6137fe640 1 -- 172.21.15.86:0/547601658 learned_addr learned my addr 172.21.15.86:0/547601658 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:38.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.036+0000 7fa6137fe640 1 --2- 172.21.15.86:0/547601658 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576aa2efe40 0x5576aa2f0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:38.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.036+0000 7fa613fff640 1 --2- 172.21.15.86:0/547601658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576aa2f0830 0x5576aa2f0cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:38.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.036+0000 7fa6117fa640 1 -- 172.21.15.86:0/3897864055 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fa60c0f2e80 con 0x7fa614102a00 2024-04-21T08:38:38.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.036+0000 7fa6137fe640 1 -- 172.21.15.86:0/547601658 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576aa2efe40 msgr2=0x5576aa2f0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:38.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.036+0000 7fa6137fe640 1 --2- 172.21.15.86:0/547601658 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576aa2efe40 0x5576aa2f0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.036+0000 7fa6137fe640 1 -- 172.21.15.86:0/547601658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576aa2f0830 msgr2=0x5576aa2f0cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:38.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.036+0000 7fa6137fe640 1 --2- 172.21.15.86:0/547601658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576aa2f0830 0x5576aa2f0cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.036+0000 7fa6137fe640 1 -- 172.21.15.86:0/547601658 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576a9f672d0 con 0x5576aa2f12d0 2024-04-21T08:38:38.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.036+0000 7fa613fff640 1 --2- 172.21.15.86:0/547601658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576aa2f0830 0x5576aa2f0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:38.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.036+0000 7fa618c5e640 1 --2- 172.21.15.86:0/3897864055 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa5fc0a3bb0 0x7fa5fc0a6070 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa60c0bed60 tx=0x7fa60c0e9ee0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:38.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.036+0000 7fa6137fe640 1 --2- 172.21.15.86:0/547601658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576aa2f12d0 0x5576aa2f7eb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa604013860 tx=0x7fa60400c270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5e98838fe36a1f41 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:38.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.037+0000 7fa5fa7fc640 1 -- 172.21.15.86:0/547601658 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6040132d0 con 0x5576aa2f12d0 2024-04-21T08:38:38.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.037+0000 7fa5fa7fc640 1 -- 172.21.15.86:0/547601658 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6040035f0 con 0x5576aa2f12d0 2024-04-21T08:38:38.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.037+0000 7fa5fa7fc640 1 -- 172.21.15.86:0/547601658 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa604009840 con 0x5576aa2f12d0 2024-04-21T08:38:38.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.037+0000 7fa61d390740 1 -- 172.21.15.86:0/547601658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576aa2f12d0 msgr2=0x5576aa2f7eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:38.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.037+0000 7fa61d390740 1 --2- 172.21.15.86:0/547601658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576aa2f12d0 0x5576aa2f7eb0 secure :-1 s=READY pgs=472 cs=0 l=1 rev1=1 crypto rx=0x7fa604013860 tx=0x7fa60400c270 comp rx=0 tx=0).stop 2024-04-21T08:38:38.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.037+0000 7fa61d390740 1 -- 172.21.15.86:0/547601658 shutdown_connections 2024-04-21T08:38:38.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.037+0000 7fa61d390740 1 --2- 172.21.15.86:0/547601658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576aa2f12d0 0x5576aa2f7eb0 unknown :-1 s=CLOSED pgs=472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.037+0000 7fa61d390740 1 --2- 172.21.15.86:0/547601658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576aa2f0830 0x5576aa2f0cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.037+0000 7fa61d390740 1 --2- 172.21.15.86:0/547601658 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576aa2efe40 0x5576aa2f0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.037+0000 7fa61d390740 1 -- 172.21.15.86:0/547601658 >> 172.21.15.86:0/547601658 conn(0x5576aa2ebf40 msgr2=0x5576aa2ea490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:38.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.037+0000 7fa61d390740 1 -- 172.21.15.86:0/547601658 shutdown_connections 2024-04-21T08:38:38.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.037+0000 7fa61d390740 1 -- 172.21.15.86:0/547601658 wait complete. 2024-04-21T08:38:38.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.038+0000 7fa61d390740 1 -- start start 2024-04-21T08:38:38.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.038+0000 7fa61d390740 1 Processor -- start 2024-04-21T08:38:38.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.038+0000 7fa61d390740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576aa2efe40 0x5576aa2f9ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.038+0000 7fa61d390740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576aa2f0830 0x5576aa2fa420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.038+0000 7fa61d390740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576aa2f12d0 0x5576aa301460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.038+0000 7fa61d390740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5576aa2f84b0 con 0x5576aa2f12d0 2024-04-21T08:38:38.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.038+0000 7fa61d390740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5576aa1dd190 con 0x5576aa2efe40 2024-04-21T08:38:38.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.038+0000 7fa61d390740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5576aa2f8630 con 0x5576aa2f0830 2024-04-21T08:38:38.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.038+0000 7fa61d390740 10 client.0 _ll_register_callbacks cb 0x5576aa1d88b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:38.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.038+0000 7fa6137fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576aa2f12d0 0x5576aa301460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:38.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.038+0000 7fa6137fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576aa2f12d0 0x5576aa301460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60458/0 (socket says 172.21.15.86:60458) 2024-04-21T08:38:38.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.038+0000 7fa6137fe640 1 -- 172.21.15.86:0/965503895 learned_addr learned my addr 172.21.15.86:0/965503895 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:38.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.038+0000 7fa6137fe640 1 --2- 172.21.15.86:0/965503895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576aa2efe40 0x5576aa2f9ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:38.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.039+0000 7fa6137fe640 1 -- 172.21.15.86:0/965503895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576aa2f0830 msgr2=0x5576aa2fa420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:38.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.039+0000 7fa613fff640 1 --2- 172.21.15.86:0/965503895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576aa2f0830 0x5576aa2fa420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:38.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.039+0000 7fa6137fe640 1 --2- 172.21.15.86:0/965503895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576aa2f0830 0x5576aa2fa420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.039+0000 7fa6137fe640 1 -- 172.21.15.86:0/965503895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576aa2efe40 msgr2=0x5576aa2f9ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:38.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.039+0000 7fa6137fe640 1 --2- 172.21.15.86:0/965503895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576aa2efe40 0x5576aa2f9ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.039+0000 7fa6137fe640 1 -- 172.21.15.86:0/965503895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576aa0316b0 con 0x5576aa2f12d0 2024-04-21T08:38:38.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.039+0000 7fa613fff640 1 --2- 172.21.15.86:0/965503895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576aa2f0830 0x5576aa2fa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:38.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.039+0000 7fa6137fe640 1 --2- 172.21.15.86:0/965503895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576aa2f12d0 0x5576aa301460 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa60400cb60 tx=0x7fa604015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5c469790cc33635 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:38.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.039+0000 7fa5f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:38:38.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.039+0000 7fa5f37fe640 1 -- 172.21.15.86:0/965503895 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6040041a0 con 0x5576aa2f12d0 2024-04-21T08:38:38.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.039+0000 7fa5f37fe640 1 -- 172.21.15.86:0/965503895 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fa604004340 con 0x5576aa2f12d0 2024-04-21T08:38:38.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.039+0000 7fa61d390740 1 -- 172.21.15.86:0/965503895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6040035f0 con 0x5576aa2f12d0 2024-04-21T08:38:38.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.039+0000 7fa5f37fe640 1 -- 172.21.15.86:0/965503895 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa6040130d0 con 0x5576aa2f12d0 2024-04-21T08:38:38.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.039+0000 7fa61d390740 10 client.15510 fetch_fsmap learned FSMap version 28 2024-04-21T08:38:38.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.039+0000 7fa61d390740 1 -- 172.21.15.86:0/965503895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5576aa3047d0 con 0x5576aa2f12d0 2024-04-21T08:38:38.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.039+0000 7fa5f37fe640 1 -- 172.21.15.86:0/965503895 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6040035f0 con 0x5576aa2f12d0 2024-04-21T08:38:38.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.039+0000 7fa5f37fe640 1 -- 172.21.15.86:0/965503895 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa60400f040 con 0x5576aa2f12d0 2024-04-21T08:38:38.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.039+0000 7fa61d390740 10 client.15510 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:38:38.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.039+0000 7fa61d390740 10 client.15510 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-21T08:38:38.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.039+0000 7fa61d390740 20 client.15510 populate_metadata read hostname 'smithi086' 2024-04-21T08:38:38.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.039+0000 7fa61d390740 1 --2- 172.21.15.86:0/965503895 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5576aa306b50 0x5576aa307fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.039+0000 7fa61d390740 4 client.15510 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:38:38.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.039+0000 7fa61d390740 1 -- 172.21.15.86:0/965503895 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576aa262530 con 0x5576aa306b50 2024-04-21T08:38:38.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.040+0000 7fa613fff640 1 --2- 172.21.15.86:0/965503895 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5576aa306b50 0x5576aa307fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:38.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.040+0000 7fa613fff640 1 --2- 172.21.15.86:0/965503895 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5576aa306b50 0x5576aa307fc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=af7ad39793ca717b server_cookie=4438d262cc16848c in_seq=0 out_seq=0 2024-04-21T08:38:38.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.040+0000 7fa5f37fe640 10 client.15510 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:38:38.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.041+0000 7fa5f37fe640 1 -- 172.21.15.86:0/965503895 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5576aa262530 con 0x5576aa306b50 2024-04-21T08:38:38.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.041+0000 7fa5f37fe640 10 client.15510 handle_command_reply: tid=0 2024-04-21T08:38:38.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.041+0000 7fa61d390740 1 client.15510 shutdown 2024-04-21T08:38:38.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.041+0000 7fa61d390740 1 -- 172.21.15.86:0/965503895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576aa2f12d0 msgr2=0x5576aa301460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:38.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.041+0000 7fa61d390740 1 --2- 172.21.15.86:0/965503895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576aa2f12d0 0x5576aa301460 secure :-1 s=READY pgs=473 cs=0 l=1 rev1=1 crypto rx=0x7fa60400cb60 tx=0x7fa604015040 comp rx=0 tx=0).stop 2024-04-21T08:38:38.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.041+0000 7fa61d390740 1 -- 172.21.15.86:0/965503895 shutdown_connections 2024-04-21T08:38:38.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.041+0000 7fa61d390740 1 --2- 172.21.15.86:0/965503895 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5576aa306b50 0x5576aa307fc0 crc :-1 s=READY pgs=18 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.041+0000 7fa61d390740 1 --2- 172.21.15.86:0/965503895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576aa2f12d0 0x5576aa301460 unknown :-1 s=CLOSED pgs=473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.041+0000 7fa61d390740 1 --2- 172.21.15.86:0/965503895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576aa2f0830 0x5576aa2fa420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.041+0000 7fa61d390740 1 --2- 172.21.15.86:0/965503895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576aa2efe40 0x5576aa2f9ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.041+0000 7fa5f37fe640 0 client.15510 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:38:38.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.041+0000 7fa61d390740 1 -- 172.21.15.86:0/965503895 >> 172.21.15.86:0/965503895 conn(0x5576aa2ebf40 msgr2=0x5576aa2e9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:38.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.041+0000 7fa61d390740 1 -- 172.21.15.86:0/965503895 shutdown_connections 2024-04-21T08:38:38.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.041+0000 7fa61d390740 1 -- 172.21.15.86:0/965503895 wait complete. 2024-04-21T08:38:38.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.041+0000 7fa61d390740 20 client.15510 trim_cache size 0 max 16384 2024-04-21T08:38:38.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.059+0000 7fa61d390740 1 Processor -- start 2024-04-21T08:38:38.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.059+0000 7fa61d390740 1 -- start start 2024-04-21T08:38:38.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.059+0000 7fa61d390740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576aa2efe40 0x5576aa305c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.059+0000 7fa61d390740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576aa2f0830 0x5576aa2f9eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.059+0000 7fa61d390740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576aa2f12d0 0x5576aa2fa3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.059+0000 7fa61d390740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5576aa306300 con 0x5576aa2f0830 2024-04-21T08:38:38.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.059+0000 7fa61d390740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5576aa3069d0 con 0x5576aa2efe40 2024-04-21T08:38:38.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.059+0000 7fa61d390740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5576aa3070a0 con 0x5576aa2f12d0 2024-04-21T08:38:38.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.060+0000 7fa613fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576aa2f0830 0x5576aa2f9eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:38.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.060+0000 7fa613fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576aa2f0830 0x5576aa2f9eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60474/0 (socket says 172.21.15.86:60474) 2024-04-21T08:38:38.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.060+0000 7fa613fff640 1 -- 172.21.15.86:0/2406187660 learned_addr learned my addr 172.21.15.86:0/2406187660 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:38.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.060+0000 7fa6137fe640 1 --2- 172.21.15.86:0/2406187660 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576aa2efe40 0x5576aa305c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:38.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.060+0000 7fa613fff640 1 -- 172.21.15.86:0/2406187660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576aa2f12d0 msgr2=0x5576aa2fa3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:38.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.060+0000 7fa613fff640 1 --2- 172.21.15.86:0/2406187660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576aa2f12d0 0x5576aa2fa3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.060+0000 7fa613fff640 1 -- 172.21.15.86:0/2406187660 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576aa2efe40 msgr2=0x5576aa305c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:38.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.060+0000 7fa613fff640 1 --2- 172.21.15.86:0/2406187660 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576aa2efe40 0x5576aa305c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.060+0000 7fa613fff640 1 -- 172.21.15.86:0/2406187660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576aa3082f0 con 0x5576aa2f0830 2024-04-21T08:38:38.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.060+0000 7fa6137fe640 1 --2- 172.21.15.86:0/2406187660 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576aa2efe40 0x5576aa305c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:38.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.060+0000 7fa613fff640 1 --2- 172.21.15.86:0/2406187660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576aa2f0830 0x5576aa2f9eb0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x5576aa2fb3c0 tx=0x7fa608011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f631fc46be97035 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:38.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.060+0000 7fa5f9ffb640 1 -- 172.21.15.86:0/2406187660 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60801c070 con 0x5576aa2f0830 2024-04-21T08:38:38.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.060+0000 7fa5f9ffb640 1 -- 172.21.15.86:0/2406187660 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa608004020 con 0x5576aa2f0830 2024-04-21T08:38:38.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.060+0000 7fa5f9ffb640 1 -- 172.21.15.86:0/2406187660 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa608018850 con 0x5576aa2f0830 2024-04-21T08:38:38.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.061+0000 7fa61d390740 1 -- 172.21.15.86:0/2406187660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576aa2f0830 msgr2=0x5576aa2f9eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:38.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.061+0000 7fa61d390740 1 --2- 172.21.15.86:0/2406187660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576aa2f0830 0x5576aa2f9eb0 secure :-1 s=READY pgs=474 cs=0 l=1 rev1=1 crypto rx=0x5576aa2fb3c0 tx=0x7fa608011810 comp rx=0 tx=0).stop 2024-04-21T08:38:38.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.061+0000 7fa61d390740 1 -- 172.21.15.86:0/2406187660 shutdown_connections 2024-04-21T08:38:38.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.061+0000 7fa61d390740 1 --2- 172.21.15.86:0/2406187660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576aa2f12d0 0x5576aa2fa3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.061+0000 7fa61d390740 1 --2- 172.21.15.86:0/2406187660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576aa2f0830 0x5576aa2f9eb0 unknown :-1 s=CLOSED pgs=474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.061+0000 7fa61d390740 1 --2- 172.21.15.86:0/2406187660 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576aa2efe40 0x5576aa305c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.061+0000 7fa61d390740 1 -- 172.21.15.86:0/2406187660 >> 172.21.15.86:0/2406187660 conn(0x5576aa2ebf40 msgr2=0x5576aa2f4840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:38.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.061+0000 7fa61d390740 1 -- 172.21.15.86:0/2406187660 shutdown_connections 2024-04-21T08:38:38.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.061+0000 7fa61d390740 1 -- 172.21.15.86:0/2406187660 wait complete. 2024-04-21T08:38:38.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.061+0000 7fa61d390740 1 -- start start 2024-04-21T08:38:38.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.061+0000 7fa61d390740 1 Processor -- start 2024-04-21T08:38:38.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.061+0000 7fa61d390740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576aa2efe40 0x5576aa2f57e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.061+0000 7fa61d390740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576aa2f0830 0x5576aa2f5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.061+0000 7fa61d390740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576aa2f12d0 0x5576aa2f6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.061+0000 7fa61d390740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5576aa307e40 con 0x5576aa2f0830 2024-04-21T08:38:38.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.061+0000 7fa61d390740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5576aa307770 con 0x5576aa2efe40 2024-04-21T08:38:38.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.061+0000 7fa61d390740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5576aa308510 con 0x5576aa2f12d0 2024-04-21T08:38:38.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.061+0000 7fa61d390740 10 client.0 _ll_register_callbacks cb 0x7fa60400ccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:38.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.061+0000 7fa613fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576aa2f0830 0x5576aa2f5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:38.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.061+0000 7fa613fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576aa2f0830 0x5576aa2f5d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60480/0 (socket says 172.21.15.86:60480) 2024-04-21T08:38:38.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.062+0000 7fa613fff640 1 -- 172.21.15.86:0/1207704286 learned_addr learned my addr 172.21.15.86:0/1207704286 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:38.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.062+0000 7fa6137fe640 1 --2- 172.21.15.86:0/1207704286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576aa2efe40 0x5576aa2f57e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:38.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.062+0000 7fa6137fe640 1 --2- 172.21.15.86:0/1207704286 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576aa2f12d0 0x5576aa2f6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:38.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.062+0000 7fa613fff640 1 -- 172.21.15.86:0/1207704286 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576aa2f12d0 msgr2=0x5576aa2f6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:38.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.062+0000 7fa613fff640 1 --2- 172.21.15.86:0/1207704286 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576aa2f12d0 0x5576aa2f6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.062+0000 7fa613fff640 1 -- 172.21.15.86:0/1207704286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576aa2efe40 msgr2=0x5576aa2f57e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:38.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.062+0000 7fa613fff640 1 --2- 172.21.15.86:0/1207704286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576aa2efe40 0x5576aa2f57e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.062+0000 7fa613fff640 1 -- 172.21.15.86:0/1207704286 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576aa2f6a90 con 0x5576aa2f0830 2024-04-21T08:38:38.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.062+0000 7fa6137fe640 1 --2- 172.21.15.86:0/1207704286 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576aa2f12d0 0x5576aa2f6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:38.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.062+0000 7fa613fff640 1 --2- 172.21.15.86:0/1207704286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576aa2f0830 0x5576aa2f5d20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa60800a6b0 tx=0x7fa608018390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ea2f14f3c96d727 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:38.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.062+0000 7fa5f37fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:38:38.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.062+0000 7fa5f37fe640 1 -- 172.21.15.86:0/1207704286 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa60801c040 con 0x5576aa2f0830 2024-04-21T08:38:38.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.062+0000 7fa5f37fe640 1 -- 172.21.15.86:0/1207704286 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fa6080037f0 con 0x5576aa2f0830 2024-04-21T08:38:38.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.062+0000 7fa61d390740 1 -- 172.21.15.86:0/1207704286 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa608004020 con 0x5576aa2f0830 2024-04-21T08:38:38.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.062+0000 7fa5f37fe640 1 -- 172.21.15.86:0/1207704286 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa608032050 con 0x5576aa2f0830 2024-04-21T08:38:38.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.062+0000 7fa61d390740 10 client.15516 fetch_fsmap learned FSMap version 28 2024-04-21T08:38:38.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.062+0000 7fa61d390740 1 -- 172.21.15.86:0/1207704286 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x5576aa370fd0 con 0x5576aa2f0830 2024-04-21T08:38:38.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.062+0000 7fa5f37fe640 1 -- 172.21.15.86:0/1207704286 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa608004020 con 0x5576aa2f0830 2024-04-21T08:38:38.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.063+0000 7fa5f37fe640 1 -- 172.21.15.86:0/1207704286 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa608027020 con 0x5576aa2f0830 2024-04-21T08:38:38.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.063+0000 7fa61d390740 10 client.15516 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:38:38.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.063+0000 7fa61d390740 10 client.15516 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-21T08:38:38.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.063+0000 7fa61d390740 20 client.15516 populate_metadata read hostname 'smithi086' 2024-04-21T08:38:38.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.063+0000 7fa61d390740 1 --2- 172.21.15.86:0/1207704286 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5576aa3759b0 0x5576aa3724c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.063+0000 7fa61d390740 4 client.15516 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:38:38.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.063+0000 7fa61d390740 1 -- 172.21.15.86:0/1207704286 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "status"}) -- 0x5576aa25c3a0 con 0x5576aa3759b0 2024-04-21T08:38:38.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.063+0000 7fa6137fe640 1 --2- 172.21.15.86:0/1207704286 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5576aa3759b0 0x5576aa3724c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:38.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.063+0000 7fa6137fe640 1 --2- 172.21.15.86:0/1207704286 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5576aa3759b0 0x5576aa3724c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=80f442f244e0c81 server_cookie=ae9e05855686fee in_seq=0 out_seq=0 2024-04-21T08:38:38.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.063+0000 7fa5f37fe640 10 client.15516 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:38:38.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.064+0000 7fa5f37fe640 1 -- 172.21.15.86:0/1207704286 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x5576aa25c3a0 con 0x5576aa3759b0 2024-04-21T08:38:38.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.064+0000 7fa5f37fe640 10 client.15516 handle_command_reply: tid=0 2024-04-21T08:38:38.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.064+0000 7fa61d390740 1 client.15516 shutdown 2024-04-21T08:38:38.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.064+0000 7fa61d390740 1 -- 172.21.15.86:0/1207704286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576aa2f0830 msgr2=0x5576aa2f5d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:38.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.064+0000 7fa61d390740 1 --2- 172.21.15.86:0/1207704286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576aa2f0830 0x5576aa2f5d20 secure :-1 s=READY pgs=475 cs=0 l=1 rev1=1 crypto rx=0x7fa60800a6b0 tx=0x7fa608018390 comp rx=0 tx=0).stop 2024-04-21T08:38:38.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.064+0000 7fa61d390740 1 -- 172.21.15.86:0/1207704286 shutdown_connections 2024-04-21T08:38:38.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.064+0000 7fa61d390740 1 --2- 172.21.15.86:0/1207704286 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576aa2f12d0 0x5576aa2f6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.064+0000 7fa61d390740 1 --2- 172.21.15.86:0/1207704286 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5576aa3759b0 0x5576aa3724c0 crc :-1 s=READY pgs=19 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.064+0000 7fa61d390740 1 --2- 172.21.15.86:0/1207704286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576aa2f0830 0x5576aa2f5d20 unknown :-1 s=CLOSED pgs=475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.064+0000 7fa61d390740 1 --2- 172.21.15.86:0/1207704286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576aa2efe40 0x5576aa2f57e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.064+0000 7fa5f37fe640 0 client.15516 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:38:38.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.064+0000 7fa61d390740 1 -- 172.21.15.86:0/1207704286 >> 172.21.15.86:0/1207704286 conn(0x5576aa2ebf40 msgr2=0x5576aa2ea420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:38.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.064+0000 7fa61d390740 1 -- 172.21.15.86:0/1207704286 shutdown_connections 2024-04-21T08:38:38.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.064+0000 7fa61d390740 1 -- 172.21.15.86:0/1207704286 wait complete. 2024-04-21T08:38:38.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.064+0000 7fa61d390740 20 client.15516 trim_cache size 0 max 16384 2024-04-21T08:38:38.070 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:38:38.070 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "f8cec7cc-ffb8-11ee-bc93-c7b262605968", 2024-04-21T08:38:38.070 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 1, 2024-04-21T08:38:38.070 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24472, 2024-04-21T08:38:38.070 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-21T08:38:38.071 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-21T08:38:38.071 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-21T08:38:38.071 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 113.682642387, 2024-04-21T08:38:38.071 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 28, 2024-04-21T08:38:38.071 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 93, 2024-04-21T08:38:38.071 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 89, 2024-04-21T08:38:38.071 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 152.13225264299999 2024-04-21T08:38:38.071 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:38:38.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.064+0000 7fa61a6c1640 1 -- 172.21.15.86:0/3897864055 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa5fc0a3bb0 msgr2=0x7fa5fc0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:38.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.064+0000 7fa61a6c1640 1 --2- 172.21.15.86:0/3897864055 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa5fc0a3bb0 0x7fa5fc0a6070 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7fa60c0bed60 tx=0x7fa60c0e9ee0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.064+0000 7fa61a6c1640 1 -- 172.21.15.86:0/3897864055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa614102a00 msgr2=0x7fa61419e580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:38.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.064+0000 7fa61a6c1640 1 --2- 172.21.15.86:0/3897864055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa614102a00 0x7fa61419e580 secure :-1 s=READY pgs=471 cs=0 l=1 rev1=1 crypto rx=0x7fa60c02c700 tx=0x7fa60c035d60 comp rx=0 tx=0).stop 2024-04-21T08:38:38.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.065+0000 7fa61a6c1640 1 -- 172.21.15.86:0/3897864055 shutdown_connections 2024-04-21T08:38:38.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.065+0000 7fa61a6c1640 1 --2- 172.21.15.86:0/3897864055 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa5fc0a3bb0 0x7fa5fc0a6070 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.065+0000 7fa61a6c1640 1 --2- 172.21.15.86:0/3897864055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa614102a00 0x7fa61419e580 unknown :-1 s=CLOSED pgs=471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.065+0000 7fa61a6c1640 1 --2- 172.21.15.86:0/3897864055 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa614102040 0x7fa614197510 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.065+0000 7fa61a6c1640 1 --2- 172.21.15.86:0/3897864055 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa614100e60 0x7fa614196fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.065+0000 7fa61a6c1640 1 -- 172.21.15.86:0/3897864055 >> 172.21.15.86:0/3897864055 conn(0x7fa6140fc420 msgr2=0x7fa614105370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:38.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.065+0000 7fa61a6c1640 1 -- 172.21.15.86:0/3897864055 shutdown_connections 2024-04-21T08:38:38.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.065+0000 7fa61a6c1640 1 -- 172.21.15.86:0/3897864055 wait complete. 2024-04-21T08:38:38.074 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-21T08:38:38.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.174+0000 7fa889693640 1 -- 172.21.15.86:0/1301173098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa884102040 msgr2=0x7fa8841024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:38.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.174+0000 7fa889693640 1 --2- 172.21.15.86:0/1301173098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa884102040 0x7fa8841024c0 secure :-1 s=READY pgs=476 cs=0 l=1 rev1=1 crypto rx=0x7fa878009900 tx=0x7fa87802bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:38.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.174+0000 7fa889693640 1 -- 172.21.15.86:0/1301173098 shutdown_connections 2024-04-21T08:38:38.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.174+0000 7fa889693640 1 --2- 172.21.15.86:0/1301173098 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa884102a00 0x7fa884109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.174+0000 7fa889693640 1 --2- 172.21.15.86:0/1301173098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa884102040 0x7fa8841024c0 unknown :-1 s=CLOSED pgs=476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.174+0000 7fa889693640 1 --2- 172.21.15.86:0/1301173098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa884100e60 0x7fa884101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.174+0000 7fa889693640 1 -- 172.21.15.86:0/1301173098 >> 172.21.15.86:0/1301173098 conn(0x7fa8840fc420 msgr2=0x7fa8840fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:38.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.174+0000 7fa889693640 1 -- 172.21.15.86:0/1301173098 shutdown_connections 2024-04-21T08:38:38.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.174+0000 7fa889693640 1 -- 172.21.15.86:0/1301173098 wait complete. 2024-04-21T08:38:38.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.175+0000 7fa889693640 1 Processor -- start 2024-04-21T08:38:38.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.175+0000 7fa889693640 1 -- start start 2024-04-21T08:38:38.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.175+0000 7fa889693640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa884100e60 0x7fa88406b120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.175+0000 7fa889693640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa884102040 0x7fa88419cef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.175+0000 7fa889693640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa884102a00 0x7fa88419f560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.175+0000 7fa889693640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa88410ca10 con 0x7fa884100e60 2024-04-21T08:38:38.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.175+0000 7fa889693640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa88410c890 con 0x7fa884102040 2024-04-21T08:38:38.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.175+0000 7fa889693640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa88410cb90 con 0x7fa884102a00 2024-04-21T08:38:38.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.175+0000 7fa882ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa884100e60 0x7fa88406b120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:38.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.175+0000 7fa882ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa884100e60 0x7fa88406b120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60490/0 (socket says 172.21.15.86:60490) 2024-04-21T08:38:38.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.175+0000 7fa882ffd640 1 -- 172.21.15.86:0/3919125812 learned_addr learned my addr 172.21.15.86:0/3919125812 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:38.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.176+0000 7fa8827fc640 1 --2- 172.21.15.86:0/3919125812 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa884102040 0x7fa88419cef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:38.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.176+0000 7fa8837fe640 1 --2- 172.21.15.86:0/3919125812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa884102a00 0x7fa88419f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:38.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.176+0000 7fa882ffd640 1 -- 172.21.15.86:0/3919125812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa884102a00 msgr2=0x7fa88419f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:38.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.176+0000 7fa882ffd640 1 --2- 172.21.15.86:0/3919125812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa884102a00 0x7fa88419f560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.176+0000 7fa882ffd640 1 -- 172.21.15.86:0/3919125812 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa884102040 msgr2=0x7fa88419cef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:38.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.176+0000 7fa882ffd640 1 --2- 172.21.15.86:0/3919125812 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa884102040 0x7fa88419cef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.176+0000 7fa882ffd640 1 -- 172.21.15.86:0/3919125812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa88419fe20 con 0x7fa884100e60 2024-04-21T08:38:38.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.176+0000 7fa8827fc640 1 --2- 172.21.15.86:0/3919125812 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa884102040 0x7fa88419cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:38.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.176+0000 7fa882ffd640 1 --2- 172.21.15.86:0/3919125812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa884100e60 0x7fa88406b120 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa87400d950 tx=0x7fa87400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:38.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.176+0000 7fa86bfff640 1 -- 172.21.15.86:0/3919125812 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa874017070 con 0x7fa884100e60 2024-04-21T08:38:38.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.176+0000 7fa889693640 1 -- 172.21.15.86:0/3919125812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa8841a0110 con 0x7fa884100e60 2024-04-21T08:38:38.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.176+0000 7fa86bfff640 1 -- 172.21.15.86:0/3919125812 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa874009c30 con 0x7fa884100e60 2024-04-21T08:38:38.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.176+0000 7fa86bfff640 1 -- 172.21.15.86:0/3919125812 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87400f620 con 0x7fa884100e60 2024-04-21T08:38:38.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.176+0000 7fa889693640 1 -- 172.21.15.86:0/3919125812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8841a0610 con 0x7fa884100e60 2024-04-21T08:38:38.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.178+0000 7fa86bfff640 1 -- 172.21.15.86:0/3919125812 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7fa87401a070 con 0x7fa884100e60 2024-04-21T08:38:38.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.179+0000 7fa88c366740 1 Processor -- start 2024-04-21T08:38:38.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.179+0000 7fa88c366740 1 -- start start 2024-04-21T08:38:38.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.179+0000 7fa88c366740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e526411e40 0x55e526412260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.179+0000 7fa88c366740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e526412830 0x55e526412cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.179+0000 7fa88c366740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5264132d0 0x55e526419eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.179+0000 7fa88c366740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e526376b10 con 0x55e526412830 2024-04-21T08:38:38.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.179+0000 7fa88c366740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55e526314240 con 0x55e526411e40 2024-04-21T08:38:38.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.179+0000 7fa88c366740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e526372ad0 con 0x55e5264132d0 2024-04-21T08:38:38.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.179+0000 7fa86bfff640 1 --2- 172.21.15.86:0/3919125812 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa86c0a3bb0 0x7fa86c0a6070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.179+0000 7fa882ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e526412830 0x55e526412cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:38.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.179+0000 7fa882ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e526412830 0x55e526412cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60494/0 (socket says 172.21.15.86:60494) 2024-04-21T08:38:38.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.179+0000 7fa882ffd640 1 -- 172.21.15.86:0/32049736 learned_addr learned my addr 172.21.15.86:0/32049736 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:38.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.179+0000 7fa8837fe640 1 --2- 172.21.15.86:0/32049736 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e526411e40 0x55e526412260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:38.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.179+0000 7fa8837fe640 1 --2- 172.21.15.86:0/3919125812 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa86c0a3bb0 0x7fa86c0a6070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:38.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.179+0000 7fa8827fc640 1 --2- 172.21.15.86:0/32049736 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5264132d0 0x55e526419eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:38.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.179+0000 7fa86bfff640 1 -- 172.21.15.86:0/3919125812 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fa8740d02e0 con 0x7fa884100e60 2024-04-21T08:38:38.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.179+0000 7fa882ffd640 1 -- 172.21.15.86:0/32049736 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5264132d0 msgr2=0x55e526419eb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:38.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.179+0000 7fa882ffd640 1 --2- 172.21.15.86:0/32049736 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5264132d0 0x55e526419eb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.179+0000 7fa882ffd640 1 -- 172.21.15.86:0/32049736 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e526411e40 msgr2=0x55e526412260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:38.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.179+0000 7fa882ffd640 1 --2- 172.21.15.86:0/32049736 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e526411e40 0x55e526412260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.180+0000 7fa882ffd640 1 -- 172.21.15.86:0/32049736 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5260892d0 con 0x55e526412830 2024-04-21T08:38:38.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.180+0000 7fa8837fe640 1 --2- 172.21.15.86:0/32049736 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e526411e40 0x55e526412260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:38:38.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.180+0000 7fa882ffd640 1 --2- 172.21.15.86:0/32049736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e526412830 0x55e526412cb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa874022000 tx=0x7fa874005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c0aed10b2799eed server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:38.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.180+0000 7fa8837fe640 1 --2- 172.21.15.86:0/3919125812 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa86c0a3bb0 0x7fa86c0a6070 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55e52640d200 tx=0x7fa87c005db0 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:38.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.180+0000 7fa8697fa640 1 -- 172.21.15.86:0/32049736 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa874021070 con 0x55e526412830 2024-04-21T08:38:38.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.180+0000 7fa8697fa640 1 -- 172.21.15.86:0/32049736 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa874009de0 con 0x55e526412830 2024-04-21T08:38:38.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.180+0000 7fa8697fa640 1 -- 172.21.15.86:0/32049736 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8740a0340 con 0x55e526412830 2024-04-21T08:38:38.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.180+0000 7fa88c366740 1 -- 172.21.15.86:0/32049736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e526412830 msgr2=0x55e526412cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:38.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.180+0000 7fa88c366740 1 --2- 172.21.15.86:0/32049736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e526412830 0x55e526412cb0 secure :-1 s=READY pgs=478 cs=0 l=1 rev1=1 crypto rx=0x7fa874022000 tx=0x7fa874005f30 comp rx=0 tx=0).stop 2024-04-21T08:38:38.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.180+0000 7fa88c366740 1 -- 172.21.15.86:0/32049736 shutdown_connections 2024-04-21T08:38:38.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.180+0000 7fa88c366740 1 --2- 172.21.15.86:0/32049736 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e5264132d0 0x55e526419eb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.181+0000 7fa88c366740 1 --2- 172.21.15.86:0/32049736 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e526412830 0x55e526412cb0 unknown :-1 s=CLOSED pgs=478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.181+0000 7fa88c366740 1 --2- 172.21.15.86:0/32049736 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e526411e40 0x55e526412260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.181+0000 7fa88c366740 1 -- 172.21.15.86:0/32049736 >> 172.21.15.86:0/32049736 conn(0x55e52640df40 msgr2=0x55e52640c490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:38.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.181+0000 7fa88c366740 1 -- 172.21.15.86:0/32049736 shutdown_connections 2024-04-21T08:38:38.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.181+0000 7fa88c366740 1 -- 172.21.15.86:0/32049736 wait complete. 2024-04-21T08:38:38.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.181+0000 7fa88c366740 1 -- start start 2024-04-21T08:38:38.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.181+0000 7fa88c366740 1 Processor -- start 2024-04-21T08:38:38.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.181+0000 7fa88c366740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e526411e40 0x55e52641bdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.181+0000 7fa88c366740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e526412830 0x55e52641c330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.181+0000 7fa88c366740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e5264132d0 0x55e526423300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.181+0000 7fa88c366740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e52641a3f0 con 0x55e526412830 2024-04-21T08:38:38.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.181+0000 7fa88c366740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55e5262ff190 con 0x55e5264132d0 2024-04-21T08:38:38.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.181+0000 7fa88c366740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e52641a570 con 0x55e526411e40 2024-04-21T08:38:38.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.181+0000 7fa88c366740 10 client.0 _ll_register_callbacks cb 0x55e5262fa8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:38.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.181+0000 7fa8827fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e526412830 0x55e52641c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:38.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.181+0000 7fa8827fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e526412830 0x55e52641c330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60510/0 (socket says 172.21.15.86:60510) 2024-04-21T08:38:38.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.181+0000 7fa8827fc640 1 -- 172.21.15.86:0/1208848243 learned_addr learned my addr 172.21.15.86:0/1208848243 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:38.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.181+0000 7fa882ffd640 1 --2- 172.21.15.86:0/1208848243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e526411e40 0x55e52641bdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:38.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.181+0000 7fa8837fe640 1 --2- 172.21.15.86:0/1208848243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e5264132d0 0x55e526423300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:38.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.181+0000 7fa8827fc640 1 -- 172.21.15.86:0/1208848243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e526411e40 msgr2=0x55e52641bdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:38.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.181+0000 7fa8827fc640 1 --2- 172.21.15.86:0/1208848243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e526411e40 0x55e52641bdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.181+0000 7fa8827fc640 1 -- 172.21.15.86:0/1208848243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e5264132d0 msgr2=0x55e526423300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:38.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.181+0000 7fa8827fc640 1 --2- 172.21.15.86:0/1208848243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e5264132d0 0x55e526423300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.181+0000 7fa8827fc640 1 -- 172.21.15.86:0/1208848243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5261536b0 con 0x55e526412830 2024-04-21T08:38:38.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.181+0000 7fa8837fe640 1 --2- 172.21.15.86:0/1208848243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e5264132d0 0x55e526423300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:38.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.182+0000 7fa8827fc640 1 --2- 172.21.15.86:0/1208848243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e526412830 0x55e52641c330 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa878000c00 tx=0x7fa8780042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23c378a75d3efd41 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:38.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.182+0000 7fa866ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:38:38.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.182+0000 7fa866ff5640 1 -- 172.21.15.86:0/1208848243 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa878005450 con 0x55e526412830 2024-04-21T08:38:38.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.182+0000 7fa866ff5640 1 -- 172.21.15.86:0/1208848243 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fa8780046c0 con 0x55e526412830 2024-04-21T08:38:38.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.182+0000 7fa88c366740 1 -- 172.21.15.86:0/1208848243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa874009de0 con 0x55e526412830 2024-04-21T08:38:38.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.182+0000 7fa866ff5640 1 -- 172.21.15.86:0/1208848243 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa878052050 con 0x55e526412830 2024-04-21T08:38:38.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.182+0000 7fa88c366740 10 client.15528 fetch_fsmap learned FSMap version 28 2024-04-21T08:38:38.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.182+0000 7fa88c366740 1 -- 172.21.15.86:0/1208848243 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e526426730 con 0x55e526412830 2024-04-21T08:38:38.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.182+0000 7fa866ff5640 1 -- 172.21.15.86:0/1208848243 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa874009de0 con 0x55e526412830 2024-04-21T08:38:38.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.182+0000 7fa866ff5640 1 -- 172.21.15.86:0/1208848243 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa878003970 con 0x55e526412830 2024-04-21T08:38:38.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.182+0000 7fa88c366740 10 client.15528 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:38:38.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.182+0000 7fa88c366740 10 client.15528 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-21T08:38:38.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.182+0000 7fa88c366740 20 client.15528 populate_metadata read hostname 'smithi086' 2024-04-21T08:38:38.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.182+0000 7fa88c366740 1 --2- 172.21.15.86:0/1208848243 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55e526428de0 0x55e52642a250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.182+0000 7fa88c366740 4 client.15528 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:38:38.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.182+0000 7fa88c366740 1 -- 172.21.15.86:0/1208848243 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e526384530 con 0x55e526428de0 2024-04-21T08:38:38.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.183+0000 7fa882ffd640 1 --2- 172.21.15.86:0/1208848243 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55e526428de0 0x55e52642a250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:38.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.183+0000 7fa866ff5640 10 client.15528 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:38:38.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.183+0000 7fa882ffd640 1 --2- 172.21.15.86:0/1208848243 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55e526428de0 0x55e52642a250 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed04b6826cc54e7b server_cookie=a624432594da50f3 in_seq=0 out_seq=0 2024-04-21T08:38:38.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.184+0000 7fa866ff5640 1 -- 172.21.15.86:0/1208848243 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e526384530 con 0x55e526428de0 2024-04-21T08:38:38.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.184+0000 7fa866ff5640 10 client.15528 handle_command_reply: tid=0 2024-04-21T08:38:38.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.184+0000 7fa88c366740 1 client.15528 shutdown 2024-04-21T08:38:38.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.184+0000 7fa88c366740 1 -- 172.21.15.86:0/1208848243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e526412830 msgr2=0x55e52641c330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:38.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.184+0000 7fa88c366740 1 --2- 172.21.15.86:0/1208848243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e526412830 0x55e52641c330 secure :-1 s=READY pgs=479 cs=0 l=1 rev1=1 crypto rx=0x7fa878000c00 tx=0x7fa8780042e0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.184+0000 7fa88c366740 1 -- 172.21.15.86:0/1208848243 shutdown_connections 2024-04-21T08:38:38.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.184+0000 7fa88c366740 1 --2- 172.21.15.86:0/1208848243 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55e526428de0 0x55e52642a250 crc :-1 s=READY pgs=31 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.184+0000 7fa88c366740 1 --2- 172.21.15.86:0/1208848243 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e5264132d0 0x55e526423300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.184+0000 7fa88c366740 1 --2- 172.21.15.86:0/1208848243 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e526412830 0x55e52641c330 unknown :-1 s=CLOSED pgs=479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.184+0000 7fa88c366740 1 --2- 172.21.15.86:0/1208848243 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e526411e40 0x55e52641bdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.184+0000 7fa866ff5640 0 client.15528 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:38:38.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.184+0000 7fa88c366740 1 -- 172.21.15.86:0/1208848243 >> 172.21.15.86:0/1208848243 conn(0x55e52640df40 msgr2=0x55e52640ba00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:38.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.184+0000 7fa88c366740 1 -- 172.21.15.86:0/1208848243 shutdown_connections 2024-04-21T08:38:38.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.184+0000 7fa88c366740 1 -- 172.21.15.86:0/1208848243 wait complete. 2024-04-21T08:38:38.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.184+0000 7fa88c366740 20 client.15528 trim_cache size 0 max 16384 2024-04-21T08:38:38.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.202+0000 7fa88c366740 1 Processor -- start 2024-04-21T08:38:38.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.202+0000 7fa88c366740 1 -- start start 2024-04-21T08:38:38.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.202+0000 7fa88c366740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e526411e40 0x55e526416850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.202+0000 7fa88c366740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e526412830 0x55e5264232c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.202+0000 7fa88c366740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e5264132d0 0x55e526416d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.202+0000 7fa88c366740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e526428590 con 0x55e526411e40 2024-04-21T08:38:38.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.202+0000 7fa88c366740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55e526428c60 con 0x55e5264132d0 2024-04-21T08:38:38.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.202+0000 7fa88c366740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e526429330 con 0x55e526412830 2024-04-21T08:38:38.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.202+0000 7fa882ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e526411e40 0x55e526416850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:38.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.202+0000 7fa882ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e526411e40 0x55e526416850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60522/0 (socket says 172.21.15.86:60522) 2024-04-21T08:38:38.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.202+0000 7fa882ffd640 1 -- 172.21.15.86:0/1724075021 learned_addr learned my addr 172.21.15.86:0/1724075021 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:38.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.202+0000 7fa8827fc640 1 --2- 172.21.15.86:0/1724075021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e526412830 0x55e5264232c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:38.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.202+0000 7fa882ffd640 1 -- 172.21.15.86:0/1724075021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e526412830 msgr2=0x55e5264232c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:38.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.202+0000 7fa8837fe640 1 --2- 172.21.15.86:0/1724075021 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e5264132d0 0x55e526416d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:38.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.202+0000 7fa882ffd640 1 --2- 172.21.15.86:0/1724075021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e526412830 0x55e5264232c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.202+0000 7fa882ffd640 1 -- 172.21.15.86:0/1724075021 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e5264132d0 msgr2=0x55e526416d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:38.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.202+0000 7fa882ffd640 1 --2- 172.21.15.86:0/1724075021 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e5264132d0 0x55e526416d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.202+0000 7fa882ffd640 1 -- 172.21.15.86:0/1724075021 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5264297e0 con 0x55e526411e40 2024-04-21T08:38:38.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.202+0000 7fa8837fe640 1 --2- 172.21.15.86:0/1724075021 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e5264132d0 0x55e526416d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:38.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.202+0000 7fa882ffd640 1 --2- 172.21.15.86:0/1724075021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e526411e40 0x55e526416850 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa87409bed0 tx=0x7fa8740cf040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a342384a13f5deb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:38.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.203+0000 7fa868ff9640 1 -- 172.21.15.86:0/1724075021 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa874021040 con 0x55e526411e40 2024-04-21T08:38:38.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.203+0000 7fa868ff9640 1 -- 172.21.15.86:0/1724075021 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa874005a70 con 0x55e526411e40 2024-04-21T08:38:38.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.203+0000 7fa868ff9640 1 -- 172.21.15.86:0/1724075021 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa87409b620 con 0x55e526411e40 2024-04-21T08:38:38.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.203+0000 7fa88c366740 1 -- 172.21.15.86:0/1724075021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e526411e40 msgr2=0x55e526416850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:38.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.203+0000 7fa88c366740 1 --2- 172.21.15.86:0/1724075021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e526411e40 0x55e526416850 secure :-1 s=READY pgs=480 cs=0 l=1 rev1=1 crypto rx=0x7fa87409bed0 tx=0x7fa8740cf040 comp rx=0 tx=0).stop 2024-04-21T08:38:38.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.203+0000 7fa88c366740 1 -- 172.21.15.86:0/1724075021 shutdown_connections 2024-04-21T08:38:38.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.203+0000 7fa88c366740 1 --2- 172.21.15.86:0/1724075021 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e5264132d0 0x55e526416d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.203+0000 7fa88c366740 1 --2- 172.21.15.86:0/1724075021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e526412830 0x55e5264232c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.203+0000 7fa88c366740 1 --2- 172.21.15.86:0/1724075021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e526411e40 0x55e526416850 unknown :-1 s=CLOSED pgs=480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.203+0000 7fa88c366740 1 -- 172.21.15.86:0/1724075021 >> 172.21.15.86:0/1724075021 conn(0x55e52640df40 msgr2=0x55e52640c400 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:38.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.203+0000 7fa88c366740 1 -- 172.21.15.86:0/1724075021 shutdown_connections 2024-04-21T08:38:38.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.203+0000 7fa88c366740 1 -- 172.21.15.86:0/1724075021 wait complete. 2024-04-21T08:38:38.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.203+0000 7fa88c366740 1 -- start start 2024-04-21T08:38:38.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.203+0000 7fa88c366740 1 Processor -- start 2024-04-21T08:38:38.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.204+0000 7fa88c366740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e526411e40 0x55e52641f2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.204+0000 7fa88c366740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e526412830 0x55e5264217f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.204+0000 7fa88c366740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e5264132d0 0x55e526421d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.204+0000 7fa88c366740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e52642a0d0 con 0x55e526411e40 2024-04-21T08:38:38.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.204+0000 7fa88c366740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55e526429a00 con 0x55e5264132d0 2024-04-21T08:38:38.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.204+0000 7fa88c366740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e52642a7a0 con 0x55e526412830 2024-04-21T08:38:38.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.204+0000 7fa88c366740 10 client.0 _ll_register_callbacks cb 0x7fa878005d50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:38.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.204+0000 7fa882ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e526411e40 0x55e52641f2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:38.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.204+0000 7fa882ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e526411e40 0x55e52641f2a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60538/0 (socket says 172.21.15.86:60538) 2024-04-21T08:38:38.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.204+0000 7fa882ffd640 1 -- 172.21.15.86:0/3000107342 learned_addr learned my addr 172.21.15.86:0/3000107342 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:38.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.204+0000 7fa8827fc640 1 --2- 172.21.15.86:0/3000107342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e526412830 0x55e5264217f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:38.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.204+0000 7fa882ffd640 1 -- 172.21.15.86:0/3000107342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e526412830 msgr2=0x55e5264217f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:38.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.204+0000 7fa882ffd640 1 --2- 172.21.15.86:0/3000107342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e526412830 0x55e5264217f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.204+0000 7fa882ffd640 1 -- 172.21.15.86:0/3000107342 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e5264132d0 msgr2=0x55e526421d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:38.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.204+0000 7fa882ffd640 1 --2- 172.21.15.86:0/3000107342 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e5264132d0 0x55e526421d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.204+0000 7fa882ffd640 1 -- 172.21.15.86:0/3000107342 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5264225a0 con 0x55e526411e40 2024-04-21T08:38:38.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.204+0000 7fa8827fc640 1 --2- 172.21.15.86:0/3000107342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e526412830 0x55e5264217f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:38.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.204+0000 7fa882ffd640 1 --2- 172.21.15.86:0/3000107342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e526411e40 0x55e52641f2a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa87409c160 tx=0x7fa874010c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8254f58b26376c69 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:38.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.204+0000 7fa866ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:38:38.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.204+0000 7fa866ff5640 1 -- 172.21.15.86:0/3000107342 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8740ba060 con 0x55e526411e40 2024-04-21T08:38:38.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.204+0000 7fa866ff5640 1 -- 172.21.15.86:0/3000107342 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fa8740a0170 con 0x55e526411e40 2024-04-21T08:38:38.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.204+0000 7fa88c366740 1 -- 172.21.15.86:0/3000107342 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa874005a70 con 0x55e526411e40 2024-04-21T08:38:38.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.204+0000 7fa866ff5640 1 -- 172.21.15.86:0/3000107342 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=28) ==== 24+0+0 (secure 0 0 0) 0x7fa874021030 con 0x55e526411e40 2024-04-21T08:38:38.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.204+0000 7fa88c366740 10 client.15534 fetch_fsmap learned FSMap version 28 2024-04-21T08:38:38.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.204+0000 7fa88c366740 1 -- 172.21.15.86:0/3000107342 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=28,monmap=4+}) -- 0x55e52648edf0 con 0x55e526411e40 2024-04-21T08:38:38.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.205+0000 7fa866ff5640 1 -- 172.21.15.86:0/3000107342 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa874005a70 con 0x55e526411e40 2024-04-21T08:38:38.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.205+0000 7fa866ff5640 1 -- 172.21.15.86:0/3000107342 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 28) ==== 8362+0+0 (secure 0 0 0) 0x7fa874098110 con 0x55e526411e40 2024-04-21T08:38:38.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.205+0000 7fa88c366740 10 client.15534 fetch_fsmap finished waiting for FSMap version 28 2024-04-21T08:38:38.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.205+0000 7fa88c366740 10 client.15534 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-21T08:38:38.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.205+0000 7fa88c366740 20 client.15534 populate_metadata read hostname 'smithi086' 2024-04-21T08:38:38.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.205+0000 7fa88c366740 1 --2- 172.21.15.86:0/3000107342 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55e526493210 0x55e526491f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.205+0000 7fa88c366740 4 client.15534 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:38:38.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.205+0000 7fa88c366740 1 -- 172.21.15.86:0/3000107342 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "status"}) -- 0x55e52637e3a0 con 0x55e526493210 2024-04-21T08:38:38.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.205+0000 7fa8827fc640 1 --2- 172.21.15.86:0/3000107342 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55e526493210 0x55e526491f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:38.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.205+0000 7fa866ff5640 10 client.15534 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:38:38.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.205+0000 7fa8827fc640 1 --2- 172.21.15.86:0/3000107342 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55e526493210 0x55e526491f20 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40e012a5ea515d7b server_cookie=c210eab9c1041dac in_seq=0 out_seq=0 2024-04-21T08:38:38.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.205+0000 7fa866ff5640 1 -- 172.21.15.86:0/3000107342 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55e52637e3a0 con 0x55e526493210 2024-04-21T08:38:38.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.205+0000 7fa866ff5640 10 client.15534 handle_command_reply: tid=0 2024-04-21T08:38:38.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.206+0000 7fa88c366740 1 client.15534 shutdown 2024-04-21T08:38:38.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.206+0000 7fa88c366740 1 -- 172.21.15.86:0/3000107342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e526411e40 msgr2=0x55e52641f2a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:38.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.206+0000 7fa88c366740 1 --2- 172.21.15.86:0/3000107342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e526411e40 0x55e52641f2a0 secure :-1 s=READY pgs=481 cs=0 l=1 rev1=1 crypto rx=0x7fa87409c160 tx=0x7fa874010c40 comp rx=0 tx=0).stop 2024-04-21T08:38:38.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.206+0000 7fa88c366740 1 -- 172.21.15.86:0/3000107342 shutdown_connections 2024-04-21T08:38:38.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.206+0000 7fa88c366740 1 --2- 172.21.15.86:0/3000107342 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55e526493210 0x55e526491f20 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.206+0000 7fa88c366740 1 --2- 172.21.15.86:0/3000107342 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e5264132d0 0x55e526421d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.206+0000 7fa88c366740 1 --2- 172.21.15.86:0/3000107342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e526412830 0x55e5264217f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.206+0000 7fa866ff5640 0 client.15534 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:38:38.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.206+0000 7fa88c366740 1 --2- 172.21.15.86:0/3000107342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e526411e40 0x55e52641f2a0 unknown :-1 s=CLOSED pgs=481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.206+0000 7fa88c366740 1 -- 172.21.15.86:0/3000107342 >> 172.21.15.86:0/3000107342 conn(0x55e52640df40 msgr2=0x55e52640b170 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:38.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.206+0000 7fa88c366740 1 -- 172.21.15.86:0/3000107342 shutdown_connections 2024-04-21T08:38:38.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.206+0000 7fa88c366740 1 -- 172.21.15.86:0/3000107342 wait complete. 2024-04-21T08:38:38.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.206+0000 7fa88c366740 20 client.15534 trim_cache size 0 max 16384 2024-04-21T08:38:38.212 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:38:38.212 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "f8cec7cc-ffb8-11ee-bc93-c7b262605968", 2024-04-21T08:38:38.212 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-21T08:38:38.212 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24482, 2024-04-21T08:38:38.213 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-21T08:38:38.213 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-21T08:38:38.213 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-21T08:38:38.213 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 136.77872535099999, 2024-04-21T08:38:38.213 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 28, 2024-04-21T08:38:38.213 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 93, 2024-04-21T08:38:38.213 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 89, 2024-04-21T08:38:38.213 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 143.64928575900001 2024-04-21T08:38:38.213 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:38:38.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.206+0000 7fa889693640 1 -- 172.21.15.86:0/3919125812 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa86c0a3bb0 msgr2=0x7fa86c0a6070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:38.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.206+0000 7fa889693640 1 --2- 172.21.15.86:0/3919125812 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa86c0a3bb0 0x7fa86c0a6070 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x55e52640d200 tx=0x7fa87c005db0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.206+0000 7fa889693640 1 -- 172.21.15.86:0/3919125812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa884100e60 msgr2=0x7fa88406b120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:38.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.206+0000 7fa889693640 1 --2- 172.21.15.86:0/3919125812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa884100e60 0x7fa88406b120 secure :-1 s=READY pgs=477 cs=0 l=1 rev1=1 crypto rx=0x7fa87400d950 tx=0x7fa87400de10 comp rx=0 tx=0).stop 2024-04-21T08:38:38.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.206+0000 7fa889693640 1 -- 172.21.15.86:0/3919125812 shutdown_connections 2024-04-21T08:38:38.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.206+0000 7fa889693640 1 --2- 172.21.15.86:0/3919125812 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa86c0a3bb0 0x7fa86c0a6070 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.206+0000 7fa889693640 1 --2- 172.21.15.86:0/3919125812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa884102a00 0x7fa88419f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.206+0000 7fa889693640 1 --2- 172.21.15.86:0/3919125812 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa884102040 0x7fa88419cef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.206+0000 7fa889693640 1 --2- 172.21.15.86:0/3919125812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa884100e60 0x7fa88406b120 unknown :-1 s=CLOSED pgs=477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.206+0000 7fa889693640 1 -- 172.21.15.86:0/3919125812 >> 172.21.15.86:0/3919125812 conn(0x7fa8840fc420 msgr2=0x7fa8841054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:38.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.206+0000 7fa889693640 1 -- 172.21.15.86:0/3919125812 shutdown_connections 2024-04-21T08:38:38.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.206+0000 7fa889693640 1 -- 172.21.15.86:0/3919125812 wait complete. 2024-04-21T08:38:38.216 INFO:teuthology.run_tasks:Running task sequential... 2024-04-21T08:38:38.226 INFO:teuthology.task.sequential:In sequential, running task sequential... 2024-04-21T08:38:38.226 INFO:teuthology.task.sequential:In sequential, running task print... 2024-04-21T08:38:38.227 INFO:teuthology.task.print:Enabling mgr modules 2024-04-21T08:38:38.227 INFO:teuthology.task.sequential:In sequential, running task exec... 2024-04-21T08:38:38.228 INFO:teuthology.task.exec:Executing custom commands... 2024-04-21T08:38:38.228 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi086.front.sepia.ceph.com 2024-04-21T08:38:38.228 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph mgr module enable snap_schedule' 2024-04-21T08:38:38.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.326+0000 7f4038aac640 1 -- 172.21.15.86:0/242926984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4034102040 msgr2=0x7f40341024c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:38.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.326+0000 7f4038aac640 1 --2- 172.21.15.86:0/242926984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4034102040 0x7f40341024c0 secure :-1 s=READY pgs=482 cs=0 l=1 rev1=1 crypto rx=0x7f401c009900 tx=0x7f401c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:38.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.326+0000 7f4038aac640 1 -- 172.21.15.86:0/242926984 shutdown_connections 2024-04-21T08:38:38.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.326+0000 7f4038aac640 1 --2- 172.21.15.86:0/242926984 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4034102a00 0x7f4034109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.326+0000 7f4038aac640 1 --2- 172.21.15.86:0/242926984 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4034102040 0x7f40341024c0 unknown :-1 s=CLOSED pgs=482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.326+0000 7f4038aac640 1 --2- 172.21.15.86:0/242926984 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4034100e60 0x7f4034101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.326+0000 7f4038aac640 1 -- 172.21.15.86:0/242926984 >> 172.21.15.86:0/242926984 conn(0x7f40340fc420 msgr2=0x7f40340fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:38.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.326+0000 7f4038aac640 1 -- 172.21.15.86:0/242926984 shutdown_connections 2024-04-21T08:38:38.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.326+0000 7f4038aac640 1 -- 172.21.15.86:0/242926984 wait complete. 2024-04-21T08:38:38.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.327+0000 7f4038aac640 1 Processor -- start 2024-04-21T08:38:38.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.327+0000 7f4038aac640 1 -- start start 2024-04-21T08:38:38.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.327+0000 7f4038aac640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4034100e60 0x7f40341980d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.327+0000 7f4038aac640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4034102040 0x7f4034198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.327+0000 7f4038aac640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4034102a00 0x7f403419f680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.327+0000 7f4038aac640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f403410daf0 con 0x7f4034102a00 2024-04-21T08:38:38.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.327+0000 7f4038aac640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f403410d970 con 0x7f4034102040 2024-04-21T08:38:38.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.327+0000 7f4038aac640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f403410dc70 con 0x7f4034100e60 2024-04-21T08:38:38.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.327+0000 7f4032d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4034102a00 0x7f403419f680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:38.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.327+0000 7f4032d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4034102a00 0x7f403419f680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60548/0 (socket says 172.21.15.86:60548) 2024-04-21T08:38:38.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.327+0000 7f4032d9d640 1 -- 172.21.15.86:0/3085318051 learned_addr learned my addr 172.21.15.86:0/3085318051 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:38.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.327+0000 7f4031d9b640 1 --2- 172.21.15.86:0/3085318051 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4034102040 0x7f4034198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:38.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.327+0000 7f403259c640 1 --2- 172.21.15.86:0/3085318051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4034100e60 0x7f40341980d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:38.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.327+0000 7f4032d9d640 1 -- 172.21.15.86:0/3085318051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4034100e60 msgr2=0x7f40341980d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:38.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.327+0000 7f4032d9d640 1 --2- 172.21.15.86:0/3085318051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4034100e60 0x7f40341980d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.327+0000 7f4032d9d640 1 -- 172.21.15.86:0/3085318051 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4034102040 msgr2=0x7f4034198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:38.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.327+0000 7f4032d9d640 1 --2- 172.21.15.86:0/3085318051 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4034102040 0x7f4034198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:38.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.327+0000 7f4032d9d640 1 -- 172.21.15.86:0/3085318051 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f403419fcf0 con 0x7f4034102a00 2024-04-21T08:38:38.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.327+0000 7f403259c640 1 --2- 172.21.15.86:0/3085318051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4034100e60 0x7f40341980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:38.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.328+0000 7f4032d9d640 1 --2- 172.21.15.86:0/3085318051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4034102a00 0x7f403419f680 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f402400e970 tx=0x7f402400ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:38.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.328+0000 7f402b7fe640 1 -- 172.21.15.86:0/3085318051 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f402400ccc0 con 0x7f4034102a00 2024-04-21T08:38:38.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.328+0000 7f4038aac640 1 -- 172.21.15.86:0/3085318051 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f403419ffe0 con 0x7f4034102a00 2024-04-21T08:38:38.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.328+0000 7f402b7fe640 1 -- 172.21.15.86:0/3085318051 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f402400ce60 con 0x7f4034102a00 2024-04-21T08:38:38.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.328+0000 7f402b7fe640 1 -- 172.21.15.86:0/3085318051 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4024010430 con 0x7f4034102a00 2024-04-21T08:38:38.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.328+0000 7f4038aac640 1 -- 172.21.15.86:0/3085318051 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40341a0450 con 0x7f4034102a00 2024-04-21T08:38:38.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.329+0000 7f4038aac640 1 -- 172.21.15.86:0/3085318051 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4034108200 con 0x7f4034102a00 2024-04-21T08:38:38.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.330+0000 7f402b7fe640 1 -- 172.21.15.86:0/3085318051 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 18) ==== 138646+0+0 (secure 0 0 0) 0x7f4024004fb0 con 0x7f4034102a00 2024-04-21T08:38:38.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.330+0000 7f402b7fe640 1 --2- 172.21.15.86:0/3085318051 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f40200a3c50 0x7f40200a6110 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:38.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.330+0000 7f403259c640 1 --2- 172.21.15.86:0/3085318051 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f40200a3c50 0x7f40200a6110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:38.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.330+0000 7f402b7fe640 1 -- 172.21.15.86:0/3085318051 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f40240d0de0 con 0x7f4034102a00 2024-04-21T08:38:38.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.331+0000 7f403259c640 1 --2- 172.21.15.86:0/3085318051 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f40200a3c50 0x7f40200a6110 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4034101bb0 tx=0x7f402c008370 comp rx=0 tx=0).ready entity=mgr.14152 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:38.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.334+0000 7f402b7fe640 1 -- 172.21.15.86:0/3085318051 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4024053ff0 con 0x7f4034102a00 2024-04-21T08:38:38.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:38.562+0000 7f4038aac640 1 -- 172.21.15.86:0/3085318051 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr module enable", "module": "snap_schedule"} v 0) -- 0x7f403402d750 con 0x7f4034102a00 2024-04-21T08:38:38.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:38 smithi088 ceph-mon[31259]: pgmap v360: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-21T08:38:38.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:38 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/171220768' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:38.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:38 smithi179 ceph-mon[31829]: pgmap v360: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-21T08:38:38.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:38 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/171220768' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:38.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:38 smithi086 ceph-mon[27939]: pgmap v360: 129 pgs: 129 active+clean; 647 KiB data, 376 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-21T08:38:38.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:38 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/171220768' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:39.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.437+0000 7f402b7fe640 1 -- 172.21.15.86:0/3085318051 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr module enable", "module": "snap_schedule"}]=0 v19) ==== 92+0+0 (secure 0 0 0) 0x7f402409d430 con 0x7f4034102a00 2024-04-21T08:38:39.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.440+0000 7f4038aac640 1 -- 172.21.15.86:0/3085318051 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f40200a3c50 msgr2=0x7f40200a6110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:39.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.440+0000 7f4038aac640 1 --2- 172.21.15.86:0/3085318051 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f40200a3c50 0x7f40200a6110 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f4034101bb0 tx=0x7f402c008370 comp rx=0 tx=0).stop 2024-04-21T08:38:39.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.440+0000 7f4038aac640 1 -- 172.21.15.86:0/3085318051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4034102a00 msgr2=0x7f403419f680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:39.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.440+0000 7f4038aac640 1 --2- 172.21.15.86:0/3085318051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4034102a00 0x7f403419f680 secure :-1 s=READY pgs=483 cs=0 l=1 rev1=1 crypto rx=0x7f402400e970 tx=0x7f402400ed70 comp rx=0 tx=0).stop 2024-04-21T08:38:39.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.441+0000 7f4038aac640 1 -- 172.21.15.86:0/3085318051 shutdown_connections 2024-04-21T08:38:39.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.441+0000 7f4038aac640 1 --2- 172.21.15.86:0/3085318051 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f40200a3c50 0x7f40200a6110 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:39.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.441+0000 7f4038aac640 1 --2- 172.21.15.86:0/3085318051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4034102a00 0x7f403419f680 unknown :-1 s=CLOSED pgs=483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:39.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.441+0000 7f4038aac640 1 --2- 172.21.15.86:0/3085318051 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4034102040 0x7f4034198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:39.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.441+0000 7f4038aac640 1 --2- 172.21.15.86:0/3085318051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4034100e60 0x7f40341980d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:39.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.441+0000 7f4038aac640 1 -- 172.21.15.86:0/3085318051 >> 172.21.15.86:0/3085318051 conn(0x7f40340fc420 msgr2=0x7f40341053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:39.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.442+0000 7f4038aac640 1 -- 172.21.15.86:0/3085318051 shutdown_connections 2024-04-21T08:38:39.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.442+0000 7f4038aac640 1 -- 172.21.15.86:0/3085318051 wait complete. 2024-04-21T08:38:39.458 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/allow_m_granularity true' 2024-04-21T08:38:39.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.578+0000 7fa0c5c11640 1 -- 172.21.15.86:0/3363673904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0c0100e60 msgr2=0x7fa0c0101280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:39.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.578+0000 7fa0c5c11640 1 --2- 172.21.15.86:0/3363673904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0c0100e60 0x7fa0c0101280 secure :-1 s=READY pgs=486 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0009900 tx=0x7fa0b002bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:39.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.578+0000 7fa0c5c11640 1 -- 172.21.15.86:0/3363673904 shutdown_connections 2024-04-21T08:38:39.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.578+0000 7fa0c5c11640 1 --2- 172.21.15.86:0/3363673904 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0c0102a00 0x7fa0c0109320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:39.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.578+0000 7fa0c5c11640 1 --2- 172.21.15.86:0/3363673904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0c0102040 0x7fa0c01024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:39.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.578+0000 7fa0c5c11640 1 --2- 172.21.15.86:0/3363673904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0c0100e60 0x7fa0c0101280 unknown :-1 s=CLOSED pgs=486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:39.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.578+0000 7fa0c5c11640 1 -- 172.21.15.86:0/3363673904 >> 172.21.15.86:0/3363673904 conn(0x7fa0c00fc420 msgr2=0x7fa0c00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:39.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.578+0000 7fa0c5c11640 1 -- 172.21.15.86:0/3363673904 shutdown_connections 2024-04-21T08:38:39.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.578+0000 7fa0c5c11640 1 -- 172.21.15.86:0/3363673904 wait complete. 2024-04-21T08:38:39.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.579+0000 7fa0c5c11640 1 Processor -- start 2024-04-21T08:38:39.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.579+0000 7fa0c5c11640 1 -- start start 2024-04-21T08:38:39.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.579+0000 7fa0c5c11640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0c0100e60 0x7fa0c0197050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:39.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.579+0000 7fa0c5c11640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0c0102040 0x7fa0c0197590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:39.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.579+0000 7fa0c5c11640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0c0102a00 0x7fa0c019e600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:39.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.579+0000 7fa0c5c11640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c010c9d0 con 0x7fa0c0100e60 2024-04-21T08:38:39.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.579+0000 7fa0c5c11640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c010c850 con 0x7fa0c0102040 2024-04-21T08:38:39.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.579+0000 7fa0c5c11640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa0c010cb50 con 0x7fa0c0102a00 2024-04-21T08:38:39.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.579+0000 7fa0bf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0c0100e60 0x7fa0c0197050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:39.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.579+0000 7fa0bf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0c0100e60 0x7fa0c0197050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57234/0 (socket says 172.21.15.86:57234) 2024-04-21T08:38:39.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.579+0000 7fa0bf7fe640 1 -- 172.21.15.86:0/885774209 learned_addr learned my addr 172.21.15.86:0/885774209 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:39.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.579+0000 7fa0beffd640 1 --2- 172.21.15.86:0/885774209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0c0102040 0x7fa0c0197590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:39.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.579+0000 7fa0bffff640 1 --2- 172.21.15.86:0/885774209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0c0102a00 0x7fa0c019e600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:39.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.580+0000 7fa0bf7fe640 1 -- 172.21.15.86:0/885774209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0c0102a00 msgr2=0x7fa0c019e600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:39.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.580+0000 7fa0bf7fe640 1 --2- 172.21.15.86:0/885774209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0c0102a00 0x7fa0c019e600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:39.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.580+0000 7fa0bf7fe640 1 -- 172.21.15.86:0/885774209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0c0102040 msgr2=0x7fa0c0197590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:39.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.580+0000 7fa0bf7fe640 1 --2- 172.21.15.86:0/885774209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0c0102040 0x7fa0c0197590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:39.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.580+0000 7fa0bf7fe640 1 -- 172.21.15.86:0/885774209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0c019ec70 con 0x7fa0c0100e60 2024-04-21T08:38:39.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.580+0000 7fa0beffd640 1 --2- 172.21.15.86:0/885774209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0c0102040 0x7fa0c0197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:38:39.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.580+0000 7fa0bf7fe640 1 --2- 172.21.15.86:0/885774209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0c0100e60 0x7fa0c0197050 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0000c00 tx=0x7fa0b0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:39.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.580+0000 7fa0bcff9640 1 -- 172.21.15.86:0/885774209 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0043020 con 0x7fa0c0100e60 2024-04-21T08:38:39.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.580+0000 7fa0bcff9640 1 -- 172.21.15.86:0/885774209 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0b0004440 con 0x7fa0c0100e60 2024-04-21T08:38:39.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.580+0000 7fa0c5c11640 1 -- 172.21.15.86:0/885774209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0c019ef00 con 0x7fa0c0100e60 2024-04-21T08:38:39.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.580+0000 7fa0bcff9640 1 -- 172.21.15.86:0/885774209 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0003c90 con 0x7fa0c0100e60 2024-04-21T08:38:39.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.580+0000 7fa0c5c11640 1 -- 172.21.15.86:0/885774209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0c019f360 con 0x7fa0c0100e60 2024-04-21T08:38:39.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.581+0000 7fa0c5c11640 1 -- 172.21.15.86:0/885774209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa0c0108200 con 0x7fa0c0100e60 2024-04-21T08:38:39.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.582+0000 7fa0bcff9640 1 -- 172.21.15.86:0/885774209 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fa0b0046020 con 0x7fa0c0100e60 2024-04-21T08:38:39.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.583+0000 7fa0bcff9640 1 --2- 172.21.15.86:0/885774209 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa0a80a3ca0 0x7fa0a80a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:39.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.583+0000 7fa0beffd640 1 -- 172.21.15.86:0/885774209 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa0a80a3ca0 msgr2=0x7fa0a80a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/1364156965 2024-04-21T08:38:39.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.583+0000 7fa0bcff9640 1 -- 172.21.15.86:0/885774209 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fa0b00f2e40 con 0x7fa0c0100e60 2024-04-21T08:38:39.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.583+0000 7fa0beffd640 1 --2- 172.21.15.86:0/885774209 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa0a80a3ca0 0x7fa0a80a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:38:39.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.587+0000 7fa0bcff9640 1 -- 172.21.15.86:0/885774209 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa0b004d020 con 0x7fa0c0100e60 2024-04-21T08:38:39.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:39 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3085318051' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-21T08:38:39.714 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:38:39 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: ignoring --setuser ceph since I am not root 2024-04-21T08:38:39.714 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:38:39 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: ignoring --setgroup ceph since I am not root 2024-04-21T08:38:39.714 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:38:39 smithi088 ceph-mgr[32655]: -- 172.21.15.88:0/2649371755 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b2679fa5a0 con 0x55b2679ce800 2024-04-21T08:38:39.714 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:38:39 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:38:39.619+0000 7fdd1cf2d200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-21T08:38:39.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:39 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3085318051' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-21T08:38:39.746 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:38:39 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: ignoring --setuser ceph since I am not root 2024-04-21T08:38:39.746 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:38:39 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: ignoring --setgroup ceph since I am not root 2024-04-21T08:38:39.746 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:38:39 smithi179 ceph-mgr[33687]: -- 172.21.15.179:0/155184051 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5654451a45a0 con 0x565445178800 2024-04-21T08:38:39.746 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:38:39 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:38:39.616+0000 7f669b429200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-21T08:38:39.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.773+0000 7fa0c5c11640 1 -- 172.21.15.86:0/885774209 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}] v 0) -- 0x7fa0c002d5c0 con 0x7fa0c0100e60 2024-04-21T08:38:39.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.778+0000 7fa0bcff9640 1 -- 172.21.15.86:0/885774209 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/allow_m_granularity}]=0 v29)=0 v29) ==== 138+0+0 (secure 0 0 0) 0x7fa0b00bf430 con 0x7fa0c0100e60 2024-04-21T08:38:39.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.780+0000 7fa0c5c11640 1 -- 172.21.15.86:0/885774209 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa0a80a3ca0 msgr2=0x7fa0a80a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:38:39.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.780+0000 7fa0c5c11640 1 --2- 172.21.15.86:0/885774209 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa0a80a3ca0 0x7fa0a80a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:39.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.780+0000 7fa0c5c11640 1 -- 172.21.15.86:0/885774209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0c0100e60 msgr2=0x7fa0c0197050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:39.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.780+0000 7fa0c5c11640 1 --2- 172.21.15.86:0/885774209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0c0100e60 0x7fa0c0197050 secure :-1 s=READY pgs=487 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0000c00 tx=0x7fa0b0004040 comp rx=0 tx=0).stop 2024-04-21T08:38:39.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.780+0000 7fa0c5c11640 1 -- 172.21.15.86:0/885774209 shutdown_connections 2024-04-21T08:38:39.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.780+0000 7fa0c5c11640 1 --2- 172.21.15.86:0/885774209 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa0a80a3ca0 0x7fa0a80a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:39.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.780+0000 7fa0c5c11640 1 --2- 172.21.15.86:0/885774209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0c0102a00 0x7fa0c019e600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:39.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.780+0000 7fa0c5c11640 1 --2- 172.21.15.86:0/885774209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0c0102040 0x7fa0c0197590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:39.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.780+0000 7fa0c5c11640 1 --2- 172.21.15.86:0/885774209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0c0100e60 0x7fa0c0197050 unknown :-1 s=CLOSED pgs=487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:39.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.780+0000 7fa0c5c11640 1 -- 172.21.15.86:0/885774209 >> 172.21.15.86:0/885774209 conn(0x7fa0c00fc420 msgr2=0x7fa0c01053d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:39.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.780+0000 7fa0c5c11640 1 -- 172.21.15.86:0/885774209 shutdown_connections 2024-04-21T08:38:39.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.780+0000 7fa0c5c11640 1 -- 172.21.15.86:0/885774209 wait complete. 2024-04-21T08:38:39.790 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:39 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3085318051' entity='client.admin' cmd=[{"prefix": "mgr module enable", "module": "snap_schedule"}]: dispatch 2024-04-21T08:38:39.790 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:38:39 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: ignoring --setuser ceph since I am not root 2024-04-21T08:38:39.790 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:38:39 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: ignoring --setgroup ceph since I am not root 2024-04-21T08:38:39.790 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:38:39 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:38:39.639+0000 7faffae22200 -1 mgr[py] Module balancer has missing NOTIFY_TYPES member 2024-04-21T08:38:39.791 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph config set mgr mgr/snap_schedule/dump_on_update true' 2024-04-21T08:38:39.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.893+0000 7fd76fa43640 1 -- 172.21.15.86:0/1043429306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd768107c90 msgr2=0x7fd76810a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:39.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.893+0000 7fd76fa43640 1 --2- 172.21.15.86:0/1043429306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd768107c90 0x7fd76810a0e0 secure :-1 s=READY pgs=488 cs=0 l=1 rev1=1 crypto rx=0x7fd760009ca0 tx=0x7fd76002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:39.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.893+0000 7fd76fa43640 1 -- 172.21.15.86:0/1043429306 shutdown_connections 2024-04-21T08:38:39.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.893+0000 7fd76fa43640 1 --2- 172.21.15.86:0/1043429306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd768107c90 0x7fd76810a0e0 unknown :-1 s=CLOSED pgs=488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:39.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.893+0000 7fd76fa43640 1 --2- 172.21.15.86:0/1043429306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd768105360 0x7fd768107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:39.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.893+0000 7fd76fa43640 1 --2- 172.21.15.86:0/1043429306 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd768102a30 0x7fd768104e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:39.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.893+0000 7fd76fa43640 1 -- 172.21.15.86:0/1043429306 >> 172.21.15.86:0/1043429306 conn(0x7fd7680fc420 msgr2=0x7fd7680fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:39.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.894+0000 7fd76fa43640 1 -- 172.21.15.86:0/1043429306 shutdown_connections 2024-04-21T08:38:39.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.894+0000 7fd76fa43640 1 -- 172.21.15.86:0/1043429306 wait complete. 2024-04-21T08:38:39.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.894+0000 7fd76fa43640 1 Processor -- start 2024-04-21T08:38:39.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.894+0000 7fd76fa43640 1 -- start start 2024-04-21T08:38:39.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.894+0000 7fd76fa43640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd768102a30 0x7fd768102350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:39.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.894+0000 7fd76fa43640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd768105360 0x7fd7681009e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:39.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.894+0000 7fd76fa43640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd768107c90 0x7fd768100f20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:39.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.894+0000 7fd76fa43640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd76810daf0 con 0x7fd768102a30 2024-04-21T08:38:39.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.894+0000 7fd76fa43640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd76810d970 con 0x7fd768105360 2024-04-21T08:38:39.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.894+0000 7fd76fa43640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd76810dc70 con 0x7fd768107c90 2024-04-21T08:38:39.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.894+0000 7fd76d7df640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd768102a30 0x7fd768102350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:39.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.894+0000 7fd76d7df640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd768102a30 0x7fd768102350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57254/0 (socket says 172.21.15.86:57254) 2024-04-21T08:38:39.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.894+0000 7fd76d7df640 1 -- 172.21.15.86:0/841419958 learned_addr learned my addr 172.21.15.86:0/841419958 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:39.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.895+0000 7fd76cfde640 1 --2- 172.21.15.86:0/841419958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd768105360 0x7fd7681009e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:39.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.895+0000 7fd76dfe0640 1 --2- 172.21.15.86:0/841419958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd768107c90 0x7fd768100f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:39.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.895+0000 7fd76d7df640 1 -- 172.21.15.86:0/841419958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd768107c90 msgr2=0x7fd768100f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:39.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.895+0000 7fd76d7df640 1 --2- 172.21.15.86:0/841419958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd768107c90 0x7fd768100f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:39.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.895+0000 7fd76d7df640 1 -- 172.21.15.86:0/841419958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd768105360 msgr2=0x7fd7681009e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:39.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.895+0000 7fd76d7df640 1 --2- 172.21.15.86:0/841419958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd768105360 0x7fd7681009e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:39.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.895+0000 7fd76d7df640 1 -- 172.21.15.86:0/841419958 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd7681017e0 con 0x7fd768102a30 2024-04-21T08:38:39.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.895+0000 7fd76d7df640 1 --2- 172.21.15.86:0/841419958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd768102a30 0x7fd768102350 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd76400e9f0 tx=0x7fd76400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:39.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.895+0000 7fd75e7fc640 1 -- 172.21.15.86:0/841419958 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd76400cd80 con 0x7fd768102a30 2024-04-21T08:38:39.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.895+0000 7fd75e7fc640 1 -- 172.21.15.86:0/841419958 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd7640108e0 con 0x7fd768102a30 2024-04-21T08:38:39.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.895+0000 7fd75e7fc640 1 -- 172.21.15.86:0/841419958 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd764018980 con 0x7fd768102a30 2024-04-21T08:38:39.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.895+0000 7fd76fa43640 1 -- 172.21.15.86:0/841419958 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd7681a2270 con 0x7fd768102a30 2024-04-21T08:38:39.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.895+0000 7fd76fa43640 1 -- 172.21.15.86:0/841419958 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd7681a2790 con 0x7fd768102a30 2024-04-21T08:38:39.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.896+0000 7fd76fa43640 1 -- 172.21.15.86:0/841419958 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd7680fdf10 con 0x7fd768102a30 2024-04-21T08:38:39.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.897+0000 7fd75e7fc640 1 -- 172.21.15.86:0/841419958 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fd764010430 con 0x7fd768102a30 2024-04-21T08:38:39.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.898+0000 7fd75e7fc640 1 --2- 172.21.15.86:0/841419958 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fd7540a3ca0 0x7fd7540a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:39.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.898+0000 7fd76cfde640 1 -- 172.21.15.86:0/841419958 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fd7540a3ca0 msgr2=0x7fd7540a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/1364156965 2024-04-21T08:38:39.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.898+0000 7fd75e7fc640 1 -- 172.21.15.86:0/841419958 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fd7640d0aa0 con 0x7fd768102a30 2024-04-21T08:38:39.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.898+0000 7fd76cfde640 1 --2- 172.21.15.86:0/841419958 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fd7540a3ca0 0x7fd7540a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:38:39.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:39.902+0000 7fd75e7fc640 1 -- 172.21.15.86:0/841419958 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd76409caa0 con 0x7fd768102a30 2024-04-21T08:38:40.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:40.087+0000 7fd76fa43640 1 -- 172.21.15.86:0/841419958 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command([{prefix=config set, name=mgr/snap_schedule/dump_on_update}] v 0) -- 0x7fd76802d5c0 con 0x7fd768102a30 2024-04-21T08:38:40.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:40.091+0000 7fd75e7fc640 1 -- 172.21.15.86:0/841419958 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{prefix=config set, name=mgr/snap_schedule/dump_on_update}]=0 v30)=0 v30) ==== 133+0+0 (secure 0 0 0) 0x7fd7640a0da0 con 0x7fd768102a30 2024-04-21T08:38:40.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:40.093+0000 7fd76fa43640 1 -- 172.21.15.86:0/841419958 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fd7540a3ca0 msgr2=0x7fd7540a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:38:40.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:40.093+0000 7fd76fa43640 1 --2- 172.21.15.86:0/841419958 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fd7540a3ca0 0x7fd7540a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:40.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:40.093+0000 7fd76fa43640 1 -- 172.21.15.86:0/841419958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd768102a30 msgr2=0x7fd768102350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:40.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:40.093+0000 7fd76fa43640 1 --2- 172.21.15.86:0/841419958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd768102a30 0x7fd768102350 secure :-1 s=READY pgs=489 cs=0 l=1 rev1=1 crypto rx=0x7fd76400e9f0 tx=0x7fd76400eeb0 comp rx=0 tx=0).stop 2024-04-21T08:38:40.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:40.093+0000 7fd76fa43640 1 -- 172.21.15.86:0/841419958 shutdown_connections 2024-04-21T08:38:40.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:40.093+0000 7fd76fa43640 1 --2- 172.21.15.86:0/841419958 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fd7540a3ca0 0x7fd7540a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:40.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:40.093+0000 7fd76fa43640 1 --2- 172.21.15.86:0/841419958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd768107c90 0x7fd768100f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:40.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:40.093+0000 7fd76fa43640 1 --2- 172.21.15.86:0/841419958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd768105360 0x7fd7681009e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:40.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:40.093+0000 7fd76fa43640 1 --2- 172.21.15.86:0/841419958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd768102a30 0x7fd768102350 unknown :-1 s=CLOSED pgs=489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:40.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:40.093+0000 7fd76fa43640 1 -- 172.21.15.86:0/841419958 >> 172.21.15.86:0/841419958 conn(0x7fd7680fc420 msgr2=0x7fd768103350 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:40.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:40.093+0000 7fd76fa43640 1 -- 172.21.15.86:0/841419958 shutdown_connections 2024-04-21T08:38:40.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:40.093+0000 7fd76fa43640 1 -- 172.21.15.86:0/841419958 wait complete. 2024-04-21T08:38:40.103 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:38:39 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:38:39.832+0000 7faffae22200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:38:40.103 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:38:39 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:38:39.940+0000 7faffae22200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-21T08:38:40.104 INFO:teuthology.run_tasks:Running task kclient... 2024-04-21T08:38:40.118 INFO:tasks.kclient:Mounting kernel clients... 2024-04-21T08:38:40.118 INFO:tasks.kclient:config is {'client.0': {}} 2024-04-21T08:38:40.119 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:38:40.119 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-21T08:38:40.133 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.admin (id: 0) 2024-04-21T08:38:40.134 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:38:40.134 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.admin 2024-04-21T08:38:40.198 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:38:40.199 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link delete ceph-brx 2024-04-21T08:38:40.212 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:38:39 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:38:39.804+0000 7fdd1cf2d200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:38:40.212 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:38:39 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:38:39.907+0000 7fdd1cf2d200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-21T08:38:40.216 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:38:39 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:38:39.804+0000 7f669b429200 -1 mgr[py] Module orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:38:40.216 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:38:39 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:38:39.907+0000 7f669b429200 -1 mgr[py] Module snap_schedule has missing NOTIFY_TYPES member 2024-04-21T08:38:40.216 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:38:40 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:38:40.217+0000 7f669b429200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-21T08:38:40.310 INFO:tasks.kclient:client.0 config is {'mntopts': ['ms_mode=secure', 'wsync'], 'syntax': 'v1', 'mount_subvol_num': 1} 2024-04-21T08:38:40.311 INFO:tasks.cephfs.mount:cephfs_mntpt = None 2024-04-21T08:38:40.311 INFO:tasks.cephfs.mount:hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-21T08:38:40.311 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph fs ls 2024-04-21T08:38:40.354 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:38:40 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:38:40.264+0000 7faffae22200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-21T08:38:40.463 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:40 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3085318051' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-21T08:38:40.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:40 smithi088 ceph-mon[31259]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-21T08:38:40.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:40 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/885774209' entity='client.admin' 2024-04-21T08:38:40.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:40 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/841419958' entity='client.admin' 2024-04-21T08:38:40.464 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:38:40 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:38:40.212+0000 7fdd1cf2d200 -1 mgr[py] Module progress has missing NOTIFY_TYPES member 2024-04-21T08:38:40.464 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:38:40 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:38:40.298+0000 7fdd1cf2d200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T08:38:40.496 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:38:40 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:38:40.302+0000 7f669b429200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T08:38:40.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:40 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3085318051' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-21T08:38:40.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:40 smithi179 ceph-mon[31829]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-21T08:38:40.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:40 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/885774209' entity='client.admin' 2024-04-21T08:38:40.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:40 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/841419958' entity='client.admin' 2024-04-21T08:38:40.561 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:38:40.616 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:38:40 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:38:40.354+0000 7faffae22200 -1 mgr[py] Module influx has missing NOTIFY_TYPES member 2024-04-21T08:38:40.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:40 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3085318051' entity='client.admin' cmd='[{"prefix": "mgr module enable", "module": "snap_schedule"}]': finished 2024-04-21T08:38:40.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:40 smithi086 ceph-mon[27939]: mgrmap e19: x(active, since 9m), standbys: z, y 2024-04-21T08:38:40.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:40 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/885774209' entity='client.admin' 2024-04-21T08:38:40.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:40 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/841419958' entity='client.admin' 2024-04-21T08:38:41.746 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:38:41 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:38:41.488+0000 7f669b429200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T08:38:41.746 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:38:41 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:38:41.658+0000 7f669b429200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T08:38:41.750 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:38:41 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:38:41.578+0000 7faffae22200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T08:38:41.811 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:38:41 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:38:41.474+0000 7fdd1cf2d200 -1 mgr[py] Module telegraf has missing NOTIFY_TYPES member 2024-04-21T08:38:41.812 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:38:41 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:38:41.643+0000 7fdd1cf2d200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T08:38:41.812 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:38:41 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:38:41.812+0000 7fdd1cf2d200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T08:38:42.064 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:38:41 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:38:41.751+0000 7faffae22200 -1 mgr[py] Module osd_support has missing NOTIFY_TYPES member 2024-04-21T08:38:42.064 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:38:41 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:38:41.928+0000 7faffae22200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T08:38:42.119 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:38:41 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:38:41.831+0000 7f669b429200 -1 mgr[py] Module iostat has missing NOTIFY_TYPES member 2024-04-21T08:38:42.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.314+0000 7fe34033a700 1 -- 172.21.15.86:0/2528962055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe338102a90 msgr2=0x7fe338102f10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:42.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.314+0000 7fe34033a700 1 --2- 172.21.15.86:0/2528962055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe338102a90 0x7fe338102f10 secure :-1 s=READY pgs=490 cs=0 l=1 rev1=1 crypto rx=0x7fe328009990 tx=0x7fe328009ca0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.314+0000 7fe34033a700 1 -- 172.21.15.86:0/2528962055 shutdown_connections 2024-04-21T08:38:42.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.314+0000 7fe34033a700 1 --2- 172.21.15.86:0/2528962055 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe338103450 0x7fe33810f8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.314+0000 7fe34033a700 1 --2- 172.21.15.86:0/2528962055 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe338108a80 0x7fe338108e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.314+0000 7fe34033a700 1 --2- 172.21.15.86:0/2528962055 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe338102a90 0x7fe338102f10 unknown :-1 s=CLOSED pgs=490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.314+0000 7fe34033a700 1 -- 172.21.15.86:0/2528962055 >> 172.21.15.86:0/2528962055 conn(0x7fe338079bf0 msgr2=0x7fe33807a000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:42.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.314+0000 7fe34033a700 1 -- 172.21.15.86:0/2528962055 shutdown_connections 2024-04-21T08:38:42.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.314+0000 7fe34033a700 1 -- 172.21.15.86:0/2528962055 wait complete. 2024-04-21T08:38:42.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.314+0000 7fe34033a700 1 Processor -- start 2024-04-21T08:38:42.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.314+0000 7fe34033a700 1 -- start start 2024-04-21T08:38:42.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.315+0000 7fe34033a700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe338102a90 0x7fe338074d40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:42.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.315+0000 7fe34033a700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe338103450 0x7fe338075280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:42.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.315+0000 7fe34033a700 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe338108a80 0x7fe3380757c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:42.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.315+0000 7fe34033a700 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe338112c90 con 0x7fe338108a80 2024-04-21T08:38:42.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.315+0000 7fe34033a700 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe338112b10 con 0x7fe338103450 2024-04-21T08:38:42.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.315+0000 7fe34033a700 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe338112e10 con 0x7fe338102a90 2024-04-21T08:38:42.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.315+0000 7fe33e0d6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe338102a90 0x7fe338074d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:42.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.315+0000 7fe33d8d5700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe338103450 0x7fe338075280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:42.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.315+0000 7fe33e0d6700 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe338102a90 0x7fe338074d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.86:46924/0 (socket says 172.21.15.86:46924) 2024-04-21T08:38:42.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.315+0000 7fe33d8d5700 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe338103450 0x7fe338075280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.86:36896/0 (socket says 172.21.15.86:36896) 2024-04-21T08:38:42.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.315+0000 7fe33e0d6700 1 -- 172.21.15.86:0/3074087217 learned_addr learned my addr 172.21.15.86:0/3074087217 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:42.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.315+0000 7fe33e8d7700 1 --2- 172.21.15.86:0/3074087217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe338108a80 0x7fe3380757c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:42.317 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:38:42 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:38:42.230+0000 7faffae22200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T08:38:42.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.315+0000 7fe33e0d6700 1 -- 172.21.15.86:0/3074087217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe338103450 msgr2=0x7fe338075280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:42.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.315+0000 7fe33e0d6700 1 --2- 172.21.15.86:0/3074087217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe338103450 0x7fe338075280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.315+0000 7fe33e0d6700 1 -- 172.21.15.86:0/3074087217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe338108a80 msgr2=0x7fe3380757c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:42.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.315+0000 7fe33e0d6700 1 --2- 172.21.15.86:0/3074087217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe338108a80 0x7fe3380757c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.315+0000 7fe33e0d6700 1 -- 172.21.15.86:0/3074087217 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe3381a76b0 con 0x7fe338102a90 2024-04-21T08:38:42.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.316+0000 7fe33e0d6700 1 --2- 172.21.15.86:0/3074087217 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe338102a90 0x7fe338074d40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe33400b530 tx=0x7fe33400b840 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:42.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.316+0000 7fe32f7fe700 1 -- 172.21.15.86:0/3074087217 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe334015020 con 0x7fe338102a90 2024-04-21T08:38:42.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.316+0000 7fe34033a700 1 -- 172.21.15.86:0/3074087217 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe3381a79a0 con 0x7fe338102a90 2024-04-21T08:38:42.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.316+0000 7fe32f7fe700 1 -- 172.21.15.86:0/3074087217 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe334011ce0 con 0x7fe338102a90 2024-04-21T08:38:42.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.316+0000 7fe32f7fe700 1 -- 172.21.15.86:0/3074087217 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe334010dd0 con 0x7fe338102a90 2024-04-21T08:38:42.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.316+0000 7fe34033a700 1 -- 172.21.15.86:0/3074087217 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe3381a7f00 con 0x7fe338102a90 2024-04-21T08:38:42.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.317+0000 7fe34033a700 1 -- 172.21.15.86:0/3074087217 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe338069730 con 0x7fe338102a90 2024-04-21T08:38:42.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.318+0000 7fe32f7fe700 1 -- 172.21.15.86:0/3074087217 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fe334018020 con 0x7fe338102a90 2024-04-21T08:38:42.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.319+0000 7fe32f7fe700 1 --2- 172.21.15.86:0/3074087217 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe3240a3c90 0x7fe3240a6150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:42.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.319+0000 7fe33d8d5700 1 -- 172.21.15.86:0/3074087217 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe3240a3c90 msgr2=0x7fe3240a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/1364156965 2024-04-21T08:38:42.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.319+0000 7fe32f7fe700 1 -- 172.21.15.86:0/3074087217 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fe3340d0460 con 0x7fe338102a90 2024-04-21T08:38:42.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.319+0000 7fe33d8d5700 1 --2- 172.21.15.86:0/3074087217 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe3240a3c90 0x7fe3240a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:38:42.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.325+0000 7fe32f7fe700 1 -- 172.21.15.86:0/3074087217 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe33401f020 con 0x7fe338102a90 2024-04-21T08:38:42.391 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:38:42 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:38:42.098+0000 7fdd1cf2d200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T08:38:42.391 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:38:42 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:38:42.190+0000 7fdd1cf2d200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T08:38:42.391 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:38:42 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:38:42.391+0000 7fdd1cf2d200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T08:38:42.421 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:38:42 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:38:42.120+0000 7f669b429200 -1 mgr[py] Module alerts has missing NOTIFY_TYPES member 2024-04-21T08:38:42.421 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:38:42 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:38:42.213+0000 7f669b429200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T08:38:42.421 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:38:42 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:38:42.421+0000 7f669b429200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T08:38:42.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.520+0000 7fe33d8d5700 1 -- 172.21.15.86:0/3074087217 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe3240a3c90 msgr2=0x7fe3240a6150 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/1364156965 2024-04-21T08:38:42.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.520+0000 7fe33d8d5700 1 --2- 172.21.15.86:0/3074087217 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe3240a3c90 0x7fe3240a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T08:38:42.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.577+0000 7fe34033a700 1 -- 172.21.15.86:0/3074087217 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "fs ls"} v 0) -- 0x7fe33802df80 con 0x7fe338102a90 2024-04-21T08:38:42.577 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:38:42 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:38:42.332+0000 7faffae22200 -1 mgr[py] Module selftest has missing NOTIFY_TYPES member 2024-04-21T08:38:42.577 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:38:42 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:38:42.542+0000 7faffae22200 -1 mgr[py] Module pg_autoscaler has missing NOTIFY_TYPES member 2024-04-21T08:38:42.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.578+0000 7fe32f7fe700 1 -- 172.21.15.86:0/3074087217 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "fs ls"}]=0 v28) ==== 53+0+88 (secure 0 0 0) 0x7fe33409cab0 con 0x7fe338102a90 2024-04-21T08:38:42.578 INFO:teuthology.orchestra.run.smithi086.stdout:name: cephfs, metadata pool: cephfs_metadata, data pools: [cephfs_data cephfs_data_ec ] 2024-04-21T08:38:42.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.580+0000 7fe34033a700 1 -- 172.21.15.86:0/3074087217 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe3240a3c90 msgr2=0x7fe3240a6150 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:38:42.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.580+0000 7fe34033a700 1 --2- 172.21.15.86:0/3074087217 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe3240a3c90 0x7fe3240a6150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.580+0000 7fe34033a700 1 -- 172.21.15.86:0/3074087217 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe338102a90 msgr2=0x7fe338074d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:42.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.580+0000 7fe34033a700 1 --2- 172.21.15.86:0/3074087217 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe338102a90 0x7fe338074d40 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fe33400b530 tx=0x7fe33400b840 comp rx=0 tx=0).stop 2024-04-21T08:38:42.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.580+0000 7fe34033a700 1 -- 172.21.15.86:0/3074087217 shutdown_connections 2024-04-21T08:38:42.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.580+0000 7fe34033a700 1 --2- 172.21.15.86:0/3074087217 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fe3240a3c90 0x7fe3240a6150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.580+0000 7fe34033a700 1 --2- 172.21.15.86:0/3074087217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe338108a80 0x7fe3380757c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.580+0000 7fe34033a700 1 --2- 172.21.15.86:0/3074087217 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe338102a90 0x7fe338074d40 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.580+0000 7fe34033a700 1 --2- 172.21.15.86:0/3074087217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe338103450 0x7fe338075280 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:42.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.580+0000 7fe34033a700 1 -- 172.21.15.86:0/3074087217 >> 172.21.15.86:0/3074087217 conn(0x7fe338079bf0 msgr2=0x7fe338101790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:42.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.580+0000 7fe34033a700 1 -- 172.21.15.86:0/3074087217 shutdown_connections 2024-04-21T08:38:42.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:42.580+0000 7fe34033a700 1 -- 172.21.15.86:0/3074087217 wait complete. 2024-04-21T08:38:42.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:42 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3074087217' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-21T08:38:42.714 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:38:42 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:38:42.583+0000 7fdd1cf2d200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:38:42.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:42 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3074087217' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-21T08:38:42.746 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:38:42 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:38:42.614+0000 7f669b429200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:38:42.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:42 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3074087217' entity='client.admin' cmd=[{"prefix": "fs ls"}]: dispatch 2024-04-21T08:38:42.866 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:38:42 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:38:42.744+0000 7faffae22200 -1 mgr[py] Module test_orchestrator has missing NOTIFY_TYPES member 2024-04-21T08:38:43.400 INFO:tasks.cephfs.mount:Mounting default Ceph FS; just confirmed its presence on cluster 2024-04-21T08:38:43.400 INFO:tasks.cephfs.mount:Mounting Ceph FS. Following are details of mount; remember "None" represents Python type None - 2024-04-21T08:38:43.400 INFO:tasks.cephfs.mount:self.client_remote.hostname = smithi086.front.sepia.ceph.com 2024-04-21T08:38:43.400 INFO:tasks.cephfs.mount:self.client.name = client.0 2024-04-21T08:38:43.400 INFO:tasks.cephfs.mount:self.hostfs_mntpt = /home/ubuntu/cephtest/mnt.0 2024-04-21T08:38:43.400 INFO:tasks.cephfs.mount:self.cephfs_name = None 2024-04-21T08:38:43.400 INFO:tasks.cephfs.mount:self.cephfs_mntpt = None 2024-04-21T08:38:43.400 INFO:tasks.cephfs.mount:self.client_keyring_path = None 2024-04-21T08:38:43.400 INFO:tasks.cephfs.mount:Setting the 'None' netns for '/home/ubuntu/cephtest/mnt.0' 2024-04-21T08:38:43.401 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:38:43.401 DEBUG:teuthology.orchestra.run.smithi086:> ip addr 2024-04-21T08:38:43.455 INFO:teuthology.orchestra.run.smithi086.stdout:1: lo: mtu 65536 qdisc noqueue state UNKNOWN group default qlen 1000 2024-04-21T08:38:43.456 INFO:teuthology.orchestra.run.smithi086.stdout: link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00 2024-04-21T08:38:43.456 INFO:teuthology.orchestra.run.smithi086.stdout: inet 127.0.0.1/8 scope host lo 2024-04-21T08:38:43.456 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-21T08:38:43.456 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 ::1/128 scope host 2024-04-21T08:38:43.456 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-21T08:38:43.456 INFO:teuthology.orchestra.run.smithi086.stdout:2: eno1: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-21T08:38:43.456 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f6 brd ff:ff:ff:ff:ff:ff 2024-04-21T08:38:43.456 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f0 2024-04-21T08:38:43.456 INFO:teuthology.orchestra.run.smithi086.stdout:3: eno2: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-21T08:38:43.456 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:d8:da:f7 brd ff:ff:ff:ff:ff:ff 2024-04-21T08:38:43.456 INFO:teuthology.orchestra.run.smithi086.stdout: altname enp7s0f1 2024-04-21T08:38:43.456 INFO:teuthology.orchestra.run.smithi086.stdout:4: enp3s0f0: mtu 1500 qdisc mq state DOWN group default qlen 1000 2024-04-21T08:38:43.456 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:08 brd ff:ff:ff:ff:ff:ff 2024-04-21T08:38:43.456 INFO:teuthology.orchestra.run.smithi086.stdout:5: enp3s0f1: mtu 1500 qdisc mq state UP group default qlen 1000 2024-04-21T08:38:43.456 INFO:teuthology.orchestra.run.smithi086.stdout: link/ether 0c:c4:7a:8f:d8:09 brd ff:ff:ff:ff:ff:ff 2024-04-21T08:38:43.457 INFO:teuthology.orchestra.run.smithi086.stdout: inet 172.21.15.86/20 brd 172.21.15.255 scope global dynamic noprefixroute enp3s0f1 2024-04-21T08:38:43.457 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft 42078sec preferred_lft 42078sec 2024-04-21T08:38:43.457 INFO:teuthology.orchestra.run.smithi086.stdout: inet6 fe80::ecad:5405:e8c2:c812/64 scope link noprefixroute 2024-04-21T08:38:43.457 INFO:teuthology.orchestra.run.smithi086.stdout: valid_lft forever preferred_lft forever 2024-04-21T08:38:43.457 INFO:tasks.cephfs.mount:Setuping the 'ceph-brx' with 192.168.255.254/16 2024-04-21T08:38:43.457 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T08:38:43.457 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-21T08:38:43.457 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add name ceph-brx type bridge 2024-04-21T08:38:43.457 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr flush dev ceph-brx 2024-04-21T08:38:43.457 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set ceph-brx up 2024-04-21T08:38:43.458 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip addr add 192.168.255.254/16 brd 192.168.255.255 dev ceph-brx 2024-04-21T08:38:43.458 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-21T08:38:43.463 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:38:43 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:38:43.177+0000 7fdd1cf2d200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T08:38:43.496 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:38:43 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:38:43.215+0000 7f669b429200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T08:38:43.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T08:38:43.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:38:43.595 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:38:43.596 DEBUG:teuthology.orchestra.run.smithi086:> echo 1 | sudo tee /proc/sys/net/ipv4/ip_forward 2024-04-21T08:38:43.616 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:38:43 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:38:43.348+0000 7faffae22200 -1 mgr[py] Module rook has missing NOTIFY_TYPES member 2024-04-21T08:38:43.619 INFO:teuthology.orchestra.run.smithi086.stdout:1 2024-04-21T08:38:43.620 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:38:43.621 DEBUG:teuthology.orchestra.run.smithi086:> route 2024-04-21T08:38:43.676 INFO:teuthology.orchestra.run.smithi086.stdout:Kernel IP routing table 2024-04-21T08:38:43.676 INFO:teuthology.orchestra.run.smithi086.stdout:Destination Gateway Genmask Flags Metric Ref Use Iface 2024-04-21T08:38:43.676 INFO:teuthology.orchestra.run.smithi086.stdout:default _gateway 0.0.0.0 UG 100 0 0 enp3s0f1 2024-04-21T08:38:43.676 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.0.0 0.0.0.0 255.255.240.0 U 100 0 0 enp3s0f1 2024-04-21T08:38:43.676 INFO:teuthology.orchestra.run.smithi086.stdout:192.168.0.0 0.0.0.0 255.255.0.0 U 0 0 0 ceph-brx 2024-04-21T08:38:43.676 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T08:38:43.676 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-21T08:38:43.677 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -o enp3s0f1 -i ceph-brx -j ACCEPT 2024-04-21T08:38:43.677 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -A FORWARD -i enp3s0f1 -o ceph-brx -j ACCEPT 2024-04-21T08:38:43.677 DEBUG:teuthology.orchestra.run.smithi086:> sudo iptables -t nat -A POSTROUTING -s 192.168.255.254/16 -o enp3s0f1 -j MASQUERADE 2024-04-21T08:38:43.677 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-21T08:38:43.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T08:38:43.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:38:43.801 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:38:43.801 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-21T08:38:43.816 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:38:43.816 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list-id 2024-04-21T08:38:43.871 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T08:38:43.871 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-21T08:38:43.871 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns add ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-21T08:38:43.871 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns set ceph-ns--home-ubuntu-cephtest-mnt.0 0 2024-04-21T08:38:43.871 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-21T08:38:43.895 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:38:43 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:38:43.596+0000 7fdd1cf2d200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T08:38:43.895 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:38:43 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:38:43.691+0000 7fdd1cf2d200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T08:38:43.895 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:38:43 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:38:43.892+0000 7fdd1cf2d200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T08:38:43.900 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:38:43 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:38:43.800+0000 7faffae22200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T08:38:43.901 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:38:43 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:38:43.901+0000 7faffae22200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T08:38:43.934 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:38:43 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:38:43.637+0000 7f669b429200 -1 mgr[py] Module volumes has missing NOTIFY_TYPES member 2024-04-21T08:38:43.934 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:38:43 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:38:43.734+0000 7f669b429200 -1 mgr[py] Module crash has missing NOTIFY_TYPES member 2024-04-21T08:38:43.934 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:38:43 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:38:43.935+0000 7f669b429200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T08:38:43.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:43 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T08:38:43.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:43 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:38:43.976 INFO:tasks.cephfs.mount:Setuping the netns 'ceph-ns--home-ubuntu-cephtest-mnt.0' with 192.168.0.1/16 2024-04-21T08:38:43.977 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T08:38:43.977 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-21T08:38:43.977 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link add veth0 netns ceph-ns--home-ubuntu-cephtest-mnt.0 type veth peer name brx.0 2024-04-21T08:38:43.977 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip addr add 192.168.0.1/16 brd 192.168.255.255 dev veth0 2024-04-21T08:38:43.977 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set veth0 up 2024-04-21T08:38:43.977 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip link set lo up 2024-04-21T08:38:43.977 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns exec ceph-ns--home-ubuntu-cephtest-mnt.0 ip route add default via 192.168.255.254 2024-04-21T08:38:43.977 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-21T08:38:44.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T08:38:44.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:38:44.089 DEBUG:teuthology.orchestra.run.smithi086:> (cd / && exec stdin-killer --timeout=300 -- bash -c ' 2024-04-21T08:38:44.089 DEBUG:teuthology.orchestra.run.smithi086:> set -e 2024-04-21T08:38:44.089 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set brx.0 up 2024-04-21T08:38:44.089 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link set dev brx.0 master ceph-brx 2024-04-21T08:38:44.089 DEBUG:teuthology.orchestra.run.smithi086:> ') 2024-04-21T08:38:44.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T08:38:44.155 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:38:44 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:38:44.125+0000 7faffae22200 -1 mgr[py] Module rgw has missing NOTIFY_TYPES member 2024-04-21T08:38:44.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:38:44.166 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-21T08:38:44.166 DEBUG:teuthology.orchestra.run.smithi086:> mkdir -p -v /home/ubuntu/cephtest/mnt.0 2024-04-21T08:38:44.180 INFO:teuthology.orchestra.run.smithi086.stdout:mkdir: created directory '/home/ubuntu/cephtest/mnt.0' 2024-04-21T08:38:44.180 INFO:teuthology.orchestra.run:Running command with timeout 60 2024-04-21T08:38:44.181 DEBUG:teuthology.orchestra.run.smithi086:> chmod 0000 /home/ubuntu/cephtest/mnt.0 2024-04-21T08:38:44.235 WARNING:tasks.cephfs.mount:You cannot specify both: cephfs_mntpt and mount_subvol_num 2024-04-21T08:38:44.235 INFO:tasks.cephfs.mount:Mounting subvol 1 for now 2024-04-21T08:38:44.236 INFO:tasks.cephfs.kernel_mount:mounting using device: :/volumes/qa/sv_1/3e019f7c-7e70-4c51-be9e-eaa2c75bd456 2024-04-21T08:38:44.236 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:38:44.236 DEBUG:teuthology.orchestra.run.smithi086:> sudo nsenter --net=/var/run/netns/ceph-ns--home-ubuntu-cephtest-mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage /bin/mount -t ceph :/volumes/qa/sv_1/3e019f7c-7e70-4c51-be9e-eaa2c75bd456 /home/ubuntu/cephtest/mnt.0 -v -o norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-04-21T08:38:44.321 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-04-21T08:38:44.321 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2024-04-21T08:38:44.321 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-04-21T08:38:44.443 INFO:teuthology.orchestra.run.smithi086.stdout:parsing options: rw,norequire_active_mds,conf=/etc/ceph/ceph.conf,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync 2024-04-21T08:38:44.443 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync". 2024-04-21T08:38:44.443 INFO:teuthology.orchestra.run.smithi086.stdout:invalid new device string format 2024-04-21T08:38:44.443 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: resolved to: "172.21.15.86:3300,172.21.15.88:3300,172.21.15.179:3300" 2024-04-21T08:38:44.443 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: trying mount with old device syntax: 172.21.15.86:3300,172.21.15.88:3300,172.21.15.179:3300:/volumes/qa/sv_1/3e019f7c-7e70-4c51-be9e-eaa2c75bd456 2024-04-21T08:38:44.443 INFO:teuthology.orchestra.run.smithi086.stdout:mount.ceph: options "norequire_active_mds,norbytes,name=0,mds_namespace=cephfs,ms_mode=secure,wsync,key=0,fsid=f8cec7cc-ffb8-11ee-bc93-c7b262605968" will pass to kernel 2024-04-21T08:38:44.444 INFO:teuthology.orchestra.run.smithi086.stdout:mount: /home/ubuntu/cephtest/mnt.0 does not contain SELinux labels. 2024-04-21T08:38:44.444 INFO:teuthology.orchestra.run.smithi086.stdout: You just mounted a file system that supports labels which does not 2024-04-21T08:38:44.444 INFO:teuthology.orchestra.run.smithi086.stdout: contain labels, onto an SELinux box. It is likely that confined 2024-04-21T08:38:44.444 INFO:teuthology.orchestra.run.smithi086.stdout: applications will generate AVC messages and not be allowed access to 2024-04-21T08:38:44.444 INFO:teuthology.orchestra.run.smithi086.stdout: this file system. For more details see restorecon(8) and mount(8). 2024-04-21T08:38:44.446 INFO:tasks.cephfs.kernel_mount:mount command passed 2024-04-21T08:38:44.446 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:38:44.446 DEBUG:teuthology.orchestra.run.smithi086:> sudo chmod 1777 /home/ubuntu/cephtest/mnt.0 2024-04-21T08:38:44.473 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-21T08:38:44.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-21T08:38:44.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:38:44.571 INFO:teuthology.orchestra.run.smithi086.stdout:client15579 2024-04-21T08:38:44.571 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-21T08:38:44.571 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-21T08:38:44.588 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-21T08:38:44.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-21T08:38:44.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:38:44.684 INFO:teuthology.orchestra.run.smithi086.stdout:client15579 2024-04-21T08:38:44.684 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=900 -- bash -c 'getfattr --only-values -n ceph.client_id .') 2024-04-21T08:38:44.695 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:38:44 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:38:44.696+0000 7faffae22200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T08:38:44.713 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:38:44 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:38:44.410+0000 7fdd1cf2d200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T08:38:44.713 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:38:44 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:38:44.503+0000 7fdd1cf2d200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T08:38:44.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44 stdin-killer INFO: expiration expected; waiting 900 seconds for command to complete 2024-04-21T08:38:44.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:38:44.740 INFO:teuthology.orchestra.run.smithi086.stdout:client15579 2024-04-21T08:38:44.740 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest/mnt.0 && exec stdin-killer --timeout=300 -- bash -c 'sudo dd if=/sys/kernel/debug/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968.client15579/status') 2024-04-21T08:38:44.746 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:38:44 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:38:44.463+0000 7f669b429200 -1 mgr[py] Module prometheus has missing NOTIFY_TYPES member 2024-04-21T08:38:44.746 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:38:44 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:38:44.558+0000 7f669b429200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T08:38:44.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44 stdin-killer INFO: expiration expected; waiting 300 seconds for command to complete 2024-04-21T08:38:44.839 INFO:teuthology.orchestra.run.smithi086.stdout:instance: client.15579 (3)192.168.0.1:0/2912239477 2024-04-21T08:38:44.839 INFO:teuthology.orchestra.run.smithi086.stdout:blocklisted: false 2024-04-21T08:38:44.840 INFO:teuthology.orchestra.run.smithi086.stderr:0+1 records in 2024-04-21T08:38:44.840 INFO:teuthology.orchestra.run.smithi086.stderr:0+1 records out 2024-04-21T08:38:44.840 INFO:teuthology.orchestra.run.smithi086.stderr:70 bytes copied, 2.4147e-05 s, 2.9 MB/s 2024-04-21T08:38:44.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44 stdin-killer INFO: command exited with status 0: exiting normally with same code! 2024-04-21T08:38:44.847 INFO:teuthology.run_tasks:Running task exec... 2024-04-21T08:38:44.856 INFO:teuthology.task.exec:Executing custom commands... 2024-04-21T08:38:44.857 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi086.front.sepia.ceph.com 2024-04-21T08:38:44.857 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs set cephfs balance_automate true' 2024-04-21T08:38:44.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.964+0000 7fa25ca62640 1 -- 172.21.15.86:0/2221412579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa258105400 msgr2=0x7fa2581077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:44.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.964+0000 7fa25ca62640 1 --2- 172.21.15.86:0/2221412579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa258105400 0x7fa2581077f0 secure :-1 s=READY pgs=492 cs=0 l=1 rev1=1 crypto rx=0x7fa24c009900 tx=0x7fa24c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:44.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.964+0000 7fa25ca62640 1 -- 172.21.15.86:0/2221412579 shutdown_connections 2024-04-21T08:38:44.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.964+0000 7fa25ca62640 1 --2- 172.21.15.86:0/2221412579 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa258107d30 0x7fa25810a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:44.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.964+0000 7fa25ca62640 1 --2- 172.21.15.86:0/2221412579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa258105400 0x7fa2581077f0 unknown :-1 s=CLOSED pgs=492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:44.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.964+0000 7fa25ca62640 1 --2- 172.21.15.86:0/2221412579 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa258102ad0 0x7fa258104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:44.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.964+0000 7fa25ca62640 1 -- 172.21.15.86:0/2221412579 >> 172.21.15.86:0/2221412579 conn(0x7fa2580fc420 msgr2=0x7fa2580fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:44.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.964+0000 7fa25ca62640 1 -- 172.21.15.86:0/2221412579 shutdown_connections 2024-04-21T08:38:44.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.964+0000 7fa25ca62640 1 -- 172.21.15.86:0/2221412579 wait complete. 2024-04-21T08:38:44.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.965+0000 7fa25ca62640 1 Processor -- start 2024-04-21T08:38:44.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.965+0000 7fa25ca62640 1 -- start start 2024-04-21T08:38:44.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.965+0000 7fa25ca62640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa258102ad0 0x7fa258198180 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:44.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.965+0000 7fa25ca62640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa258105400 0x7fa2581986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:44.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.965+0000 7fa25ca62640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa258107d30 0x7fa25819f730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:44.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.965+0000 7fa25ca62640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa25810dad0 con 0x7fa258105400 2024-04-21T08:38:44.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.965+0000 7fa25ca62640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa25810d950 con 0x7fa258102ad0 2024-04-21T08:38:44.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.965+0000 7fa25ca62640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa25810dc50 con 0x7fa258107d30 2024-04-21T08:38:44.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.965+0000 7fa255d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa258105400 0x7fa2581986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:44.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.965+0000 7fa255d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa258105400 0x7fa2581986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57300/0 (socket says 172.21.15.86:57300) 2024-04-21T08:38:44.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.965+0000 7fa255d9b640 1 -- 172.21.15.86:0/465099298 learned_addr learned my addr 172.21.15.86:0/465099298 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:44.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.965+0000 7fa25659c640 1 --2- 172.21.15.86:0/465099298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa258102ad0 0x7fa258198180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:44.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.965+0000 7fa256d9d640 1 --2- 172.21.15.86:0/465099298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa258107d30 0x7fa25819f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:44.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.965+0000 7fa255d9b640 1 -- 172.21.15.86:0/465099298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa258107d30 msgr2=0x7fa25819f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:44.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.965+0000 7fa255d9b640 1 --2- 172.21.15.86:0/465099298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa258107d30 0x7fa25819f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:44.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.965+0000 7fa255d9b640 1 -- 172.21.15.86:0/465099298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa258102ad0 msgr2=0x7fa258198180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:44.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.965+0000 7fa255d9b640 1 --2- 172.21.15.86:0/465099298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa258102ad0 0x7fa258198180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:44.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.965+0000 7fa255d9b640 1 -- 172.21.15.86:0/465099298 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa25819fda0 con 0x7fa258105400 2024-04-21T08:38:44.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.965+0000 7fa25659c640 1 --2- 172.21.15.86:0/465099298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa258102ad0 0x7fa258198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:38:44.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.966+0000 7fa255d9b640 1 --2- 172.21.15.86:0/465099298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa258105400 0x7fa2581986c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa24c000c00 tx=0x7fa24c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:44.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.966+0000 7fa2477fe640 1 -- 172.21.15.86:0/465099298 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24c043020 con 0x7fa258105400 2024-04-21T08:38:44.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.966+0000 7fa25ca62640 1 -- 172.21.15.86:0/465099298 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa2581a0030 con 0x7fa258105400 2024-04-21T08:38:44.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.966+0000 7fa2477fe640 1 -- 172.21.15.86:0/465099298 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa24c004350 con 0x7fa258105400 2024-04-21T08:38:44.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.966+0000 7fa2477fe640 1 -- 172.21.15.86:0/465099298 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa24c003b80 con 0x7fa258105400 2024-04-21T08:38:44.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.966+0000 7fa25ca62640 1 -- 172.21.15.86:0/465099298 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa2581a05d0 con 0x7fa258105400 2024-04-21T08:38:44.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.967+0000 7fa25ca62640 1 -- 172.21.15.86:0/465099298 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa2580fdf10 con 0x7fa258105400 2024-04-21T08:38:44.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.968+0000 7fa2477fe640 1 -- 172.21.15.86:0/465099298 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fa24c046020 con 0x7fa258105400 2024-04-21T08:38:44.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.969+0000 7fa2477fe640 1 --2- 172.21.15.86:0/465099298 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa2400a3ca0 0x7fa2400a6160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:44.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.969+0000 7fa25659c640 1 -- 172.21.15.86:0/465099298 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa2400a3ca0 msgr2=0x7fa2400a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/1364156965 2024-04-21T08:38:44.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.969+0000 7fa2477fe640 1 -- 172.21.15.86:0/465099298 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fa24c0f2d50 con 0x7fa258105400 2024-04-21T08:38:44.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.969+0000 7fa25659c640 1 --2- 172.21.15.86:0/465099298 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa2400a3ca0 0x7fa2400a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:38:44.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:44.974+0000 7fa2477fe640 1 -- 172.21.15.86:0/465099298 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa24c04d020 con 0x7fa258105400 2024-04-21T08:38:45.060 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:38:44 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:38:44.795+0000 7faffae22200 -1 mgr[py] Module devicehealth has missing NOTIFY_TYPES member 2024-04-21T08:38:45.089 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:38:44 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:38:44.746+0000 7fdd1cf2d200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T08:38:45.089 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:38:44 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:38:44.877+0000 7fdd1cf2d200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T08:38:45.089 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:38:44 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:38:44.977+0000 7fdd1cf2d200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T08:38:45.089 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:38:45 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:38:45.090+0000 7fdd1cf2d200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T08:38:45.149 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:38:44 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:38:44.804+0000 7f669b429200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T08:38:45.149 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:38:44 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:38:44.939+0000 7f669b429200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T08:38:45.149 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:38:45 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:38:45.035+0000 7f669b429200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T08:38:45.149 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:38:45 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:38:45.150+0000 7f669b429200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T08:38:45.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.170+0000 7fa25659c640 1 -- 172.21.15.86:0/465099298 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa2400a3ca0 msgr2=0x7fa2400a6160 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/1364156965 2024-04-21T08:38:45.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.170+0000 7fa25659c640 1 --2- 172.21.15.86:0/465099298 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa2400a3ca0 0x7fa2400a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T08:38:45.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.200+0000 7fa25ca62640 1 -- 172.21.15.86:0/465099298 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"} v 0) -- 0x7fa25802d5c0 con 0x7fa258105400 2024-04-21T08:38:45.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.253+0000 7fa2477fe640 1 -- 172.21.15.86:0/465099298 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]=0 v29) ==== 117+0+0 (secure 0 0 0) 0x7fa24c0bf3a0 con 0x7fa258105400 2024-04-21T08:38:45.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.255+0000 7fa25ca62640 1 -- 172.21.15.86:0/465099298 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa2400a3ca0 msgr2=0x7fa2400a6160 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:38:45.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.255+0000 7fa25ca62640 1 --2- 172.21.15.86:0/465099298 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa2400a3ca0 0x7fa2400a6160 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.255+0000 7fa25ca62640 1 -- 172.21.15.86:0/465099298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa258105400 msgr2=0x7fa2581986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.255+0000 7fa25ca62640 1 --2- 172.21.15.86:0/465099298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa258105400 0x7fa2581986c0 secure :-1 s=READY pgs=493 cs=0 l=1 rev1=1 crypto rx=0x7fa24c000c00 tx=0x7fa24c005dc0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.255+0000 7fa25ca62640 1 -- 172.21.15.86:0/465099298 shutdown_connections 2024-04-21T08:38:45.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.255+0000 7fa25ca62640 1 --2- 172.21.15.86:0/465099298 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fa2400a3ca0 0x7fa2400a6160 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.255+0000 7fa25ca62640 1 --2- 172.21.15.86:0/465099298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa258107d30 0x7fa25819f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.255+0000 7fa25ca62640 1 --2- 172.21.15.86:0/465099298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa258105400 0x7fa2581986c0 unknown :-1 s=CLOSED pgs=493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.255+0000 7fa25ca62640 1 --2- 172.21.15.86:0/465099298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa258102ad0 0x7fa258198180 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.255+0000 7fa25ca62640 1 -- 172.21.15.86:0/465099298 >> 172.21.15.86:0/465099298 conn(0x7fa2580fc420 msgr2=0x7fa2581036d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:45.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.255+0000 7fa25ca62640 1 -- 172.21.15.86:0/465099298 shutdown_connections 2024-04-21T08:38:45.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.255+0000 7fa25ca62640 1 -- 172.21.15.86:0/465099298 wait complete. 2024-04-21T08:38:45.268 INFO:teuthology.run_tasks:Running task check-counter... 2024-04-21T08:38:45.279 INFO:teuthology.run_tasks:Running task fwd_scrub... 2024-04-21T08:38:45.292 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:38:45.312 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:45 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/465099298' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-21T08:38:45.313 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:38:45 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:38:45.061+0000 7faffae22200 -1 mgr[py] Module telemetry has missing NOTIFY_TYPES member 2024-04-21T08:38:45.313 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:38:45 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:38:45.200+0000 7faffae22200 -1 mgr[py] Module diskprediction_local has missing NOTIFY_TYPES member 2024-04-21T08:38:45.313 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:38:45 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:38:45.302+0000 7faffae22200 -1 mgr[py] Module osd_perf_query has missing NOTIFY_TYPES member 2024-04-21T08:38:45.387 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:45 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/465099298' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-21T08:38:45.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.393+0000 7f3ff26ba640 1 -- 172.21.15.86:0/3941438738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fec105340 msgr2=0x7f3fec107730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.394+0000 7f3ff26ba640 1 --2- 172.21.15.86:0/3941438738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fec105340 0x7f3fec107730 secure :-1 s=READY pgs=494 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0009900 tx=0x7f3fe002bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:45.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.394+0000 7f3ff26ba640 1 -- 172.21.15.86:0/3941438738 shutdown_connections 2024-04-21T08:38:45.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.394+0000 7f3ff26ba640 1 --2- 172.21.15.86:0/3941438738 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3fec107c70 0x7f3fec10a0c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.394+0000 7f3ff26ba640 1 --2- 172.21.15.86:0/3941438738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fec105340 0x7f3fec107730 unknown :-1 s=CLOSED pgs=494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.394+0000 7f3ff26ba640 1 --2- 172.21.15.86:0/3941438738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3fec102a10 0x7f3fec104e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.394+0000 7f3ff26ba640 1 -- 172.21.15.86:0/3941438738 >> 172.21.15.86:0/3941438738 conn(0x7f3fec0fc400 msgr2=0x7f3fec0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:45.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.394+0000 7f3ff26ba640 1 -- 172.21.15.86:0/3941438738 shutdown_connections 2024-04-21T08:38:45.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.394+0000 7f3ff26ba640 1 -- 172.21.15.86:0/3941438738 wait complete. 2024-04-21T08:38:45.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.394+0000 7f3ff26ba640 1 Processor -- start 2024-04-21T08:38:45.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.394+0000 7f3ff26ba640 1 -- start start 2024-04-21T08:38:45.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.395+0000 7f3ff26ba640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3fec102a10 0x7f3fec1981e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.395+0000 7f3ff26ba640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fec105340 0x7f3fec198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.395+0000 7f3ff26ba640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3fec107c70 0x7f3fec19f790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.395+0000 7f3ff26ba640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3fec10dab0 con 0x7f3fec105340 2024-04-21T08:38:45.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.395+0000 7f3ff26ba640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3fec10d930 con 0x7f3fec102a10 2024-04-21T08:38:45.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.395+0000 7f3ff26ba640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3fec10dc30 con 0x7f3fec107c70 2024-04-21T08:38:45.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.395+0000 7f3feb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fec105340 0x7f3fec198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.395+0000 7f3feb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fec105340 0x7f3fec198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57330/0 (socket says 172.21.15.86:57330) 2024-04-21T08:38:45.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.395+0000 7f3feb7fe640 1 -- 172.21.15.86:0/3159402181 learned_addr learned my addr 172.21.15.86:0/3159402181 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:45.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.395+0000 7f3febfff640 1 --2- 172.21.15.86:0/3159402181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3fec102a10 0x7f3fec1981e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.395+0000 7f3ff0c57640 1 --2- 172.21.15.86:0/3159402181 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3fec107c70 0x7f3fec19f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.395+0000 7f3feb7fe640 1 -- 172.21.15.86:0/3159402181 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3fec107c70 msgr2=0x7f3fec19f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.395+0000 7f3feb7fe640 1 --2- 172.21.15.86:0/3159402181 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3fec107c70 0x7f3fec19f790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.395+0000 7f3feb7fe640 1 -- 172.21.15.86:0/3159402181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3fec102a10 msgr2=0x7f3fec1981e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.395+0000 7f3feb7fe640 1 --2- 172.21.15.86:0/3159402181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3fec102a10 0x7f3fec1981e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.395+0000 7f3feb7fe640 1 -- 172.21.15.86:0/3159402181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3fec19fe00 con 0x7f3fec105340 2024-04-21T08:38:45.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.396+0000 7f3feb7fe640 1 --2- 172.21.15.86:0/3159402181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fec105340 0x7f3fec198720 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0000c00 tx=0x7f3fe0005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:45.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.396+0000 7f3fe97fa640 1 -- 172.21.15.86:0/3159402181 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe0043020 con 0x7f3fec105340 2024-04-21T08:38:45.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.396+0000 7f3fe97fa640 1 -- 172.21.15.86:0/3159402181 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3fe0004350 con 0x7f3fec105340 2024-04-21T08:38:45.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.396+0000 7f3ff26ba640 1 -- 172.21.15.86:0/3159402181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3fec1a0090 con 0x7f3fec105340 2024-04-21T08:38:45.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.396+0000 7f3fe97fa640 1 -- 172.21.15.86:0/3159402181 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3fe0003b80 con 0x7f3fec105340 2024-04-21T08:38:45.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.396+0000 7f3ff26ba640 1 -- 172.21.15.86:0/3159402181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3fec1a05e0 con 0x7f3fec105340 2024-04-21T08:38:45.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.397+0000 7f3ff26ba640 1 -- 172.21.15.86:0/3159402181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3fec0fdef0 con 0x7f3fec105340 2024-04-21T08:38:45.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.398+0000 7f3fe97fa640 1 -- 172.21.15.86:0/3159402181 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f3fe0046020 con 0x7f3fec105340 2024-04-21T08:38:45.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.399+0000 7f3fe97fa640 1 --2- 172.21.15.86:0/3159402181 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f3fd40a3c00 0x7f3fd40a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.399+0000 7f3febfff640 1 -- 172.21.15.86:0/3159402181 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f3fd40a3c00 msgr2=0x7f3fd40a60c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/1364156965 2024-04-21T08:38:45.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.399+0000 7f3fe97fa640 1 -- 172.21.15.86:0/3159402181 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f3fe00f29b0 con 0x7f3fec105340 2024-04-21T08:38:45.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.399+0000 7f3febfff640 1 --2- 172.21.15.86:0/3159402181 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f3fd40a3c00 0x7f3fd40a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:38:45.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.404+0000 7f3fe97fa640 1 -- 172.21.15.86:0/3159402181 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f3fe004d020 con 0x7f3fec105340 2024-04-21T08:38:45.457 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:45 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/465099298' entity='client.admin' cmd=[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]: dispatch 2024-04-21T08:38:45.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.600+0000 7f3febfff640 1 -- 172.21.15.86:0/3159402181 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f3fd40a3c00 msgr2=0x7f3fd40a60c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/1364156965 2024-04-21T08:38:45.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.600+0000 7f3febfff640 1 --2- 172.21.15.86:0/3159402181 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f3fd40a3c00 0x7f3fd40a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.400000 2024-04-21T08:38:45.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.603+0000 7f3ff26ba640 1 -- 172.21.15.86:0/3159402181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f3fec062b50 con 0x7f3fec105340 2024-04-21T08:38:45.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.605+0000 7f3fe97fa640 1 -- 172.21.15.86:0/3159402181 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v93) ==== 74+0+24689 (secure 0 0 0) 0x7f3fe00bf000 con 0x7f3fec105340 2024-04-21T08:38:45.605 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:38:45.605 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":93,"fsid":"f8cec7cc-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:22.687329+0000","modified":"2024-04-21T08:38:24.484832+0000","last_up_change":"2024-04-21T08:35:30.048315+0000","last_in_change":"2024-04-21T08:35:10.099496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:45.279109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:18.300782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:19.574636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:22.905632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89b7601f-0620-4db2-90f1-b45cea173738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6803","nonce":2571516897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6805","nonce":2571516897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6809","nonce":2571516897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6807","nonce":2571516897}]},"public_addr":"172.21.15.86:6803/2571516897","cluster_addr":"172.21.15.86:6805/2571516897","heartbeat_back_addr":"172.21.15.86:6809/2571516897","heartbeat_front_addr":"172.21.15.86:6807/2571516897","state":["exists","up"]},{"osd":1,"uuid":"4891031c-f98c-42bc-bb3b-a420855ecc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6801","nonce":74067182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6803","nonce":74067182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6807","nonce":74067182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6805","nonce":74067182}]},"public_addr":"172.21.15.88:6801/74067182","cluster_addr":"172.21.15.88:6803/74067182","heartbeat_back_addr":"172.21.15.88:6807/74067182","heartbeat_front_addr":"172.21.15.88:6805/74067182","state":["exists","up"]},{"osd":2,"uuid":"60d12aa4-9312-4fa2-8d0b-2b3f42caad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6801","nonce":4137659051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6803","nonce":4137659051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6807","nonce":4137659051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6805","nonce":4137659051}]},"public_addr":"172.21.15.179:6801/4137659051","cluster_addr":"172.21.15.179:6803/4137659051","heartbeat_back_addr":"172.21.15.179:6807/4137659051","heartbeat_front_addr":"172.21.15.179:6805/4137659051","state":["exists","up"]},{"osd":3,"uuid":"404c3158-8cb0-4c02-b48a-f9ee5ff7b784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6811","nonce":2309603148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6813","nonce":2309603148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6817","nonce":2309603148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6815","nonce":2309603148}]},"public_addr":"172.21.15.86:6811/2309603148","cluster_addr":"172.21.15.86:6813/2309603148","heartbeat_back_addr":"172.21.15.86:6817/2309603148","heartbeat_front_addr":"172.21.15.86:6815/2309603148","state":["exists","up"]},{"osd":4,"uuid":"458ff202-d781-4ec7-ac93-0b49f835b503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6809","nonce":1818714217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6811","nonce":1818714217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6815","nonce":1818714217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6813","nonce":1818714217}]},"public_addr":"172.21.15.88:6809/1818714217","cluster_addr":"172.21.15.88:6811/1818714217","heartbeat_back_addr":"172.21.15.88:6815/1818714217","heartbeat_front_addr":"172.21.15.88:6813/1818714217","state":["exists","up"]},{"osd":5,"uuid":"8123cc90-bd8b-4204-bf31-46b34e9f589d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6809","nonce":863840990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6811","nonce":863840990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6815","nonce":863840990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6813","nonce":863840990}]},"public_addr":"172.21.15.179:6809/863840990","cluster_addr":"172.21.15.179:6811/863840990","heartbeat_back_addr":"172.21.15.179:6815/863840990","heartbeat_front_addr":"172.21.15.179:6813/863840990","state":["exists","up"]},{"osd":6,"uuid":"aac9d7c7-659a-4110-903a-a871c086066f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6819","nonce":1224231118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6821","nonce":1224231118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6825","nonce":1224231118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6823","nonce":1224231118}]},"public_addr":"172.21.15.86:6819/1224231118","cluster_addr":"172.21.15.86:6821/1224231118","heartbeat_back_addr":"172.21.15.86:6825/1224231118","heartbeat_front_addr":"172.21.15.86:6823/1224231118","state":["exists","up"]},{"osd":7,"uuid":"df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6817","nonce":136629878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6819","nonce":136629878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6823","nonce":136629878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6821","nonce":136629878}]},"public_addr":"172.21.15.88:6817/136629878","cluster_addr":"172.21.15.88:6819/136629878","heartbeat_back_addr":"172.21.15.88:6823/136629878","heartbeat_front_addr":"172.21.15.88:6821/136629878","state":["exists","up"]},{"osd":8,"uuid":"63e59f4b-0426-46db-92f7-c55dae61b5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6817","nonce":1770418957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6819","nonce":1770418957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6823","nonce":1770418957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6821","nonce":1770418957}]},"public_addr":"172.21.15.179:6817/1770418957","cluster_addr":"172.21.15.179:6819/1770418957","heartbeat_back_addr":"172.21.15.179:6823/1770418957","heartbeat_front_addr":"172.21.15.179:6821/1770418957","state":["exists","up"]},{"osd":9,"uuid":"5841bb36-ed6b-45d8-b857-eeab139c44e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6827","nonce":1108500274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6829","nonce":1108500274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6833","nonce":1108500274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6831","nonce":1108500274}]},"public_addr":"172.21.15.86:6827/1108500274","cluster_addr":"172.21.15.86:6829/1108500274","heartbeat_back_addr":"172.21.15.86:6833/1108500274","heartbeat_front_addr":"172.21.15.86:6831/1108500274","state":["exists","up"]},{"osd":10,"uuid":"f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6825","nonce":1294275353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6827","nonce":1294275353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6831","nonce":1294275353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6829","nonce":1294275353}]},"public_addr":"172.21.15.88:6825/1294275353","cluster_addr":"172.21.15.88:6827/1294275353","heartbeat_back_addr":"172.21.15.88:6831/1294275353","heartbeat_front_addr":"172.21.15.88:6829/1294275353","state":["exists","up"]},{"osd":11,"uuid":"e0d4cc76-e7a6-4a6b-909c-95488ccd5a36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6825","nonce":4007070233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6827","nonce":4007070233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6831","nonce":4007070233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6829","nonce":4007070233}]},"public_addr":"172.21.15.179:6825/4007070233","cluster_addr":"172.21.15.179:6827/4007070233","heartbeat_back_addr":"172.21.15.179:6831/4007070233","heartbeat_front_addr":"172.21.15.179:6829/4007070233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:55.210605+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:18.723090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:42.547979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:06.421622+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:30.858315+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:55.455313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.623369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:46.185111+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.103627+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:37.297545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:02.637431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:28.199292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:6800/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/1412393225":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6800/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/1687852404":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2020707404":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2568130933":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/3952607648":"2024-04-22T08:28:57.384403+0000","172.21.15.86:6801/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/2756903426":"2024-04-22T08:29:23.190001+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:38:45.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.608+0000 7f3ff26ba640 1 -- 172.21.15.86:0/3159402181 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f3fd40a3c00 msgr2=0x7f3fd40a60c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:38:45.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.608+0000 7f3ff26ba640 1 --2- 172.21.15.86:0/3159402181 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f3fd40a3c00 0x7f3fd40a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.608+0000 7f3ff26ba640 1 -- 172.21.15.86:0/3159402181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fec105340 msgr2=0x7f3fec198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.608+0000 7f3ff26ba640 1 --2- 172.21.15.86:0/3159402181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fec105340 0x7f3fec198720 secure :-1 s=READY pgs=495 cs=0 l=1 rev1=1 crypto rx=0x7f3fe0000c00 tx=0x7f3fe0005dc0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.608+0000 7f3ff26ba640 1 -- 172.21.15.86:0/3159402181 shutdown_connections 2024-04-21T08:38:45.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.608+0000 7f3ff26ba640 1 --2- 172.21.15.86:0/3159402181 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f3fd40a3c00 0x7f3fd40a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.608+0000 7f3ff26ba640 1 --2- 172.21.15.86:0/3159402181 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3fec107c70 0x7f3fec19f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.608+0000 7f3ff26ba640 1 --2- 172.21.15.86:0/3159402181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3fec105340 0x7f3fec198720 unknown :-1 s=CLOSED pgs=495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.608+0000 7f3ff26ba640 1 --2- 172.21.15.86:0/3159402181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3fec102a10 0x7f3fec1981e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.608+0000 7f3ff26ba640 1 -- 172.21.15.86:0/3159402181 >> 172.21.15.86:0/3159402181 conn(0x7f3fec0fc400 msgr2=0x7f3fec104860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:45.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.608+0000 7f3ff26ba640 1 -- 172.21.15.86:0/3159402181 shutdown_connections 2024-04-21T08:38:45.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.608+0000 7f3ff26ba640 1 -- 172.21.15.86:0/3159402181 wait complete. 2024-04-21T08:38:45.616 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:38:45 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:38:45.427+0000 7faffae22200 -1 mgr[py] Module rbd_support has missing NOTIFY_TYPES member 2024-04-21T08:38:45.619 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:31:45.279109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:36:18.300782+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:36:19.574636+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:36:22.905632+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:38:45.619 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:38:45.713 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:38:45 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:38:45.388+0000 7fdd1cf2d200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T08:38:45.746 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:38:45 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:38:45.458+0000 7f669b429200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T08:38:45.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.765+0000 7fb3deae0640 1 -- 172.21.15.86:0/3624062049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d8102a30 msgr2=0x7fb3d8104e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.765+0000 7fb3deae0640 1 --2- 172.21.15.86:0/3624062049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d8102a30 0x7fb3d8104e20 secure :-1 s=READY pgs=496 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc009900 tx=0x7fb3cc02bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:45.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.765+0000 7fb3deae0640 1 -- 172.21.15.86:0/3624062049 shutdown_connections 2024-04-21T08:38:45.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.765+0000 7fb3deae0640 1 --2- 172.21.15.86:0/3624062049 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3d8107c90 0x7fb3d810a0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.765+0000 7fb3deae0640 1 --2- 172.21.15.86:0/3624062049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3d8105360 0x7fb3d8107750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.765+0000 7fb3deae0640 1 --2- 172.21.15.86:0/3624062049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d8102a30 0x7fb3d8104e20 unknown :-1 s=CLOSED pgs=496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.765+0000 7fb3deae0640 1 -- 172.21.15.86:0/3624062049 >> 172.21.15.86:0/3624062049 conn(0x7fb3d80fc440 msgr2=0x7fb3d80fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:45.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.765+0000 7fb3deae0640 1 -- 172.21.15.86:0/3624062049 shutdown_connections 2024-04-21T08:38:45.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.765+0000 7fb3deae0640 1 -- 172.21.15.86:0/3624062049 wait complete. 2024-04-21T08:38:45.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.765+0000 7fb3deae0640 1 Processor -- start 2024-04-21T08:38:45.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.766+0000 7fb3deae0640 1 -- start start 2024-04-21T08:38:45.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.766+0000 7fb3deae0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d8102a30 0x7fb3d8198280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.766+0000 7fb3deae0640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3d8105360 0x7fb3d81987c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.766+0000 7fb3deae0640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3d8107c90 0x7fb3d819f830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.766+0000 7fb3deae0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d810daf0 con 0x7fb3d8102a30 2024-04-21T08:38:45.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.766+0000 7fb3deae0640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d810d970 con 0x7fb3d8107c90 2024-04-21T08:38:45.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.766+0000 7fb3deae0640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb3d810dc70 con 0x7fb3d8105360 2024-04-21T08:38:45.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.766+0000 7fb3dc87c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d8102a30 0x7fb3d8198280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.766+0000 7fb3dc87c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d8102a30 0x7fb3d8198280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57334/0 (socket says 172.21.15.86:57334) 2024-04-21T08:38:45.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.766+0000 7fb3dc87c640 1 -- 172.21.15.86:0/4128797878 learned_addr learned my addr 172.21.15.86:0/4128797878 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:45.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.766+0000 7fb3d7fff640 1 --2- 172.21.15.86:0/4128797878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3d8105360 0x7fb3d81987c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.766+0000 7fb3dd07d640 1 --2- 172.21.15.86:0/4128797878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3d8107c90 0x7fb3d819f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:45.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.766+0000 7fb3dc87c640 1 -- 172.21.15.86:0/4128797878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3d8105360 msgr2=0x7fb3d81987c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.766+0000 7fb3dc87c640 1 --2- 172.21.15.86:0/4128797878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3d8105360 0x7fb3d81987c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.766+0000 7fb3dc87c640 1 -- 172.21.15.86:0/4128797878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3d8107c90 msgr2=0x7fb3d819f830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.766+0000 7fb3dc87c640 1 --2- 172.21.15.86:0/4128797878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3d8107c90 0x7fb3d819f830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.766+0000 7fb3dc87c640 1 -- 172.21.15.86:0/4128797878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb3d819fea0 con 0x7fb3d8102a30 2024-04-21T08:38:45.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.766+0000 7fb3dd07d640 1 --2- 172.21.15.86:0/4128797878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3d8107c90 0x7fb3d819f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:45.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.766+0000 7fb3dc87c640 1 --2- 172.21.15.86:0/4128797878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d8102a30 0x7fb3d8198280 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc006010 tx=0x7fb3cc0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:45.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.766+0000 7fb3d5ffb640 1 -- 172.21.15.86:0/4128797878 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc043020 con 0x7fb3d8102a30 2024-04-21T08:38:45.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.766+0000 7fb3deae0640 1 -- 172.21.15.86:0/4128797878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb3d81a0130 con 0x7fb3d8102a30 2024-04-21T08:38:45.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.766+0000 7fb3d5ffb640 1 -- 172.21.15.86:0/4128797878 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3cc0044c0 con 0x7fb3d8102a30 2024-04-21T08:38:45.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.767+0000 7fb3d5ffb640 1 -- 172.21.15.86:0/4128797878 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3cc003c60 con 0x7fb3d8102a30 2024-04-21T08:38:45.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.767+0000 7fb3deae0640 1 -- 172.21.15.86:0/4128797878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb3d81a06d0 con 0x7fb3d8102a30 2024-04-21T08:38:45.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.768+0000 7fb3deae0640 1 -- 172.21.15.86:0/4128797878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb3d80fdf30 con 0x7fb3d8102a30 2024-04-21T08:38:45.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.769+0000 7fb3d5ffb640 1 -- 172.21.15.86:0/4128797878 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7fb3cc046020 con 0x7fb3d8102a30 2024-04-21T08:38:45.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.769+0000 7fb3d5ffb640 1 --2- 172.21.15.86:0/4128797878 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb3c80a3c00 0x7fb3c80a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:45.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.770+0000 7fb3d7fff640 1 -- 172.21.15.86:0/4128797878 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb3c80a3c00 msgr2=0x7fb3c80a60c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/1364156965 2024-04-21T08:38:45.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.770+0000 7fb3d5ffb640 1 -- 172.21.15.86:0/4128797878 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fb3cc0f2be0 con 0x7fb3d8102a30 2024-04-21T08:38:45.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.770+0000 7fb3d7fff640 1 --2- 172.21.15.86:0/4128797878 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb3c80a3c00 0x7fb3c80a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:38:45.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.774+0000 7fb3d5ffb640 1 -- 172.21.15.86:0/4128797878 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb3cc04d020 con 0x7fb3d8102a30 2024-04-21T08:38:45.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.967+0000 7fb3deae0640 1 -- 172.21.15.86:0/4128797878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fb3d8062b50 con 0x7fb3d8102a30 2024-04-21T08:38:45.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.967+0000 7fb3d5ffb640 1 -- 172.21.15.86:0/4128797878 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v93) ==== 93+0+10 (secure 0 0 0) 0x7fb3cc0bf1d0 con 0x7fb3d8102a30 2024-04-21T08:38:45.967 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-21T08:38:45.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.970+0000 7fb3deae0640 1 -- 172.21.15.86:0/4128797878 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb3c80a3c00 msgr2=0x7fb3c80a60c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:38:45.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.970+0000 7fb3deae0640 1 --2- 172.21.15.86:0/4128797878 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb3c80a3c00 0x7fb3c80a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.970+0000 7fb3deae0640 1 -- 172.21.15.86:0/4128797878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d8102a30 msgr2=0x7fb3d8198280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:45.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.970+0000 7fb3deae0640 1 --2- 172.21.15.86:0/4128797878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d8102a30 0x7fb3d8198280 secure :-1 s=READY pgs=497 cs=0 l=1 rev1=1 crypto rx=0x7fb3cc006010 tx=0x7fb3cc0040c0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.970+0000 7fb3deae0640 1 -- 172.21.15.86:0/4128797878 shutdown_connections 2024-04-21T08:38:45.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.970+0000 7fb3deae0640 1 --2- 172.21.15.86:0/4128797878 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb3c80a3c00 0x7fb3c80a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.970+0000 7fb3deae0640 1 --2- 172.21.15.86:0/4128797878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb3d8107c90 0x7fb3d819f830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.970+0000 7fb3deae0640 1 --2- 172.21.15.86:0/4128797878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb3d8105360 0x7fb3d81987c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.970+0000 7fb3deae0640 1 --2- 172.21.15.86:0/4128797878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb3d8102a30 0x7fb3d8198280 unknown :-1 s=CLOSED pgs=497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:45.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.970+0000 7fb3deae0640 1 -- 172.21.15.86:0/4128797878 >> 172.21.15.86:0/4128797878 conn(0x7fb3d80fc440 msgr2=0x7fb3d8103450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:45.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.970+0000 7fb3deae0640 1 -- 172.21.15.86:0/4128797878 shutdown_connections 2024-04-21T08:38:45.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:45.970+0000 7fb3deae0640 1 -- 172.21.15.86:0/4128797878 wait complete. 2024-04-21T08:38:45.979 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:38:45 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:38:45.742+0000 7faffae22200 -1 mgr[py] Module zabbix has missing NOTIFY_TYPES member 2024-04-21T08:38:45.980 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:38:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.083+0000 7f2346d9f640 1 -- 172.21.15.86:0/3423856196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2340103990 msgr2=0x7f234010a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.083+0000 7f2346d9f640 1 --2- 172.21.15.86:0/3423856196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2340103990 0x7f234010a2b0 secure :-1 s=READY pgs=498 cs=0 l=1 rev1=1 crypto rx=0x7f2338009ca0 tx=0x7f233802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.084+0000 7f2346d9f640 1 -- 172.21.15.86:0/3423856196 shutdown_connections 2024-04-21T08:38:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.084+0000 7f2346d9f640 1 --2- 172.21.15.86:0/3423856196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2340103990 0x7f234010a2b0 unknown :-1 s=CLOSED pgs=498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.084+0000 7f2346d9f640 1 --2- 172.21.15.86:0/3423856196 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2340102fd0 0x7f2340103450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.084+0000 7f2346d9f640 1 --2- 172.21.15.86:0/3423856196 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2340101e80 0x7f23401022a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.084+0000 7f2346d9f640 1 -- 172.21.15.86:0/3423856196 >> 172.21.15.86:0/3423856196 conn(0x7f23400fd3f0 msgr2=0x7f23400ff830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.084+0000 7f2346d9f640 1 -- 172.21.15.86:0/3423856196 shutdown_connections 2024-04-21T08:38:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.084+0000 7f2346d9f640 1 -- 172.21.15.86:0/3423856196 wait complete. 2024-04-21T08:38:46.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.084+0000 7f2346d9f640 1 Processor -- start 2024-04-21T08:38:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.084+0000 7f2346d9f640 1 -- start start 2024-04-21T08:38:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.084+0000 7f2346d9f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2340101e80 0x7f2340198050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.084+0000 7f2346d9f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2340102fd0 0x7f2340198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.084+0000 7f2346d9f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2340103990 0x7f234019f600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.084+0000 7f2346d9f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f234010d9e0 con 0x7f2340101e80 2024-04-21T08:38:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.084+0000 7f2346d9f640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f234010d860 con 0x7f2340103990 2024-04-21T08:38:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.084+0000 7f2346d9f640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f234010db60 con 0x7f2340102fd0 2024-04-21T08:38:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.085+0000 7f2345d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2340101e80 0x7f2340198050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.085+0000 7f2345d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2340101e80 0x7f2340198050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57362/0 (socket says 172.21.15.86:57362) 2024-04-21T08:38:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.085+0000 7f2345d9d640 1 -- 172.21.15.86:0/1521255174 learned_addr learned my addr 172.21.15.86:0/1521255174 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.085+0000 7f234559c640 1 --2- 172.21.15.86:0/1521255174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2340102fd0 0x7f2340198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:46.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.085+0000 7f234659e640 1 --2- 172.21.15.86:0/1521255174 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2340103990 0x7f234019f600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.085+0000 7f2345d9d640 1 -- 172.21.15.86:0/1521255174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2340102fd0 msgr2=0x7f2340198590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.085+0000 7f2345d9d640 1 --2- 172.21.15.86:0/1521255174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2340102fd0 0x7f2340198590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.085+0000 7f2345d9d640 1 -- 172.21.15.86:0/1521255174 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2340103990 msgr2=0x7f234019f600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.085+0000 7f2345d9d640 1 --2- 172.21.15.86:0/1521255174 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2340103990 0x7f234019f600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.085+0000 7f2345d9d640 1 -- 172.21.15.86:0/1521255174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f234019fc70 con 0x7f2340101e80 2024-04-21T08:38:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.085+0000 7f2345d9d640 1 --2- 172.21.15.86:0/1521255174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2340101e80 0x7f2340198050 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f233c00e9f0 tx=0x7f233c00edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.085+0000 7f2336ffd640 1 -- 172.21.15.86:0/1521255174 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233c00cd80 con 0x7f2340101e80 2024-04-21T08:38:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.085+0000 7f2346d9f640 1 -- 172.21.15.86:0/1521255174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f234019ff60 con 0x7f2340101e80 2024-04-21T08:38:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.085+0000 7f2336ffd640 1 -- 172.21.15.86:0/1521255174 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f233c0054d0 con 0x7f2340101e80 2024-04-21T08:38:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.085+0000 7f2336ffd640 1 -- 172.21.15.86:0/1521255174 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f233c010690 con 0x7f2340101e80 2024-04-21T08:38:46.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.085+0000 7f2346d9f640 1 -- 172.21.15.86:0/1521255174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23401a03c0 con 0x7f2340101e80 2024-04-21T08:38:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.086+0000 7f2346d9f640 1 -- 172.21.15.86:0/1521255174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2340068400 con 0x7f2340101e80 2024-04-21T08:38:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.087+0000 7f2336ffd640 1 -- 172.21.15.86:0/1521255174 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 19) ==== 138663+0+0 (secure 0 0 0) 0x7f233c004e30 con 0x7f2340101e80 2024-04-21T08:38:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.088+0000 7f2336ffd640 1 --2- 172.21.15.86:0/1521255174 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f232c0a3c00 0x7f232c0a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.088+0000 7f234559c640 1 -- 172.21.15.86:0/1521255174 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f232c0a3c00 msgr2=0x7f232c0a60c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/1364156965 2024-04-21T08:38:46.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.088+0000 7f2336ffd640 1 -- 172.21.15.86:0/1521255174 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7f233c0d0e00 con 0x7f2340101e80 2024-04-21T08:38:46.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.088+0000 7f234559c640 1 --2- 172.21.15.86:0/1521255174 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f232c0a3c00 0x7f232c0a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:38:46.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.096+0000 7f2336ffd640 1 -- 172.21.15.86:0/1521255174 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f233c053ff0 con 0x7f2340101e80 2024-04-21T08:38:46.259 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:38:45 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:38:45.993+0000 7fdd1cf2d200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T08:38:46.259 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:38:46 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y[32646]: 2024-04-21T08:38:46.209+0000 7fdd1cf2d200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T08:38:46.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.274+0000 7f2336ffd640 1 -- 172.21.15.86:0/1521255174 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 20) ==== 138663+0+0 (secure 0 0 0) 0x7f233c0993c0 con 0x7f2340101e80 2024-04-21T08:38:46.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.283+0000 7f2346d9f640 1 -- 172.21.15.86:0/1521255174 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f234002d5c0 con 0x7f2340101e80 2024-04-21T08:38:46.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.284+0000 7f2336ffd640 1 -- 172.21.15.86:0/1521255174 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v93) ==== 104+0+11 (secure 0 0 0) 0x7f233c057130 con 0x7f2340101e80 2024-04-21T08:38:46.283 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-21T08:38:46.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.286+0000 7f2346d9f640 1 -- 172.21.15.86:0/1521255174 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f232c0a3c00 msgr2=0x7f232c0a60c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:38:46.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.286+0000 7f2346d9f640 1 --2- 172.21.15.86:0/1521255174 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f232c0a3c00 0x7f232c0a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.286+0000 7f2346d9f640 1 -- 172.21.15.86:0/1521255174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2340101e80 msgr2=0x7f2340198050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:46.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.286+0000 7f2346d9f640 1 --2- 172.21.15.86:0/1521255174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2340101e80 0x7f2340198050 secure :-1 s=READY pgs=499 cs=0 l=1 rev1=1 crypto rx=0x7f233c00e9f0 tx=0x7f233c00edf0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.286+0000 7f2346d9f640 1 -- 172.21.15.86:0/1521255174 shutdown_connections 2024-04-21T08:38:46.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.286+0000 7f2346d9f640 1 --2- 172.21.15.86:0/1521255174 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7f232c0a3c00 0x7f232c0a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.286+0000 7f2346d9f640 1 --2- 172.21.15.86:0/1521255174 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2340103990 0x7f234019f600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.286+0000 7f2346d9f640 1 --2- 172.21.15.86:0/1521255174 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2340102fd0 0x7f2340198590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.286+0000 7f2346d9f640 1 --2- 172.21.15.86:0/1521255174 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2340101e80 0x7f2340198050 unknown :-1 s=CLOSED pgs=499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.286+0000 7f2346d9f640 1 -- 172.21.15.86:0/1521255174 >> 172.21.15.86:0/1521255174 conn(0x7f23400fd3f0 msgr2=0x7f2340106360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:46.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.286+0000 7f2346d9f640 1 -- 172.21.15.86:0/1521255174 shutdown_connections 2024-04-21T08:38:46.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.286+0000 7f2346d9f640 1 -- 172.21.15.86:0/1521255174 wait complete. 2024-04-21T08:38:46.296 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:38:46.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.400+0000 7fb827650640 1 -- 172.21.15.86:0/4262853286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb820101fe0 msgr2=0x7fb820102460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:46.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.400+0000 7fb827650640 1 --2- 172.21.15.86:0/4262853286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb820101fe0 0x7fb820102460 secure :-1 s=READY pgs=500 cs=0 l=1 rev1=1 crypto rx=0x7fb810009900 tx=0x7fb81002bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:46.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.400+0000 7fb827650640 1 -- 172.21.15.86:0/4262853286 shutdown_connections 2024-04-21T08:38:46.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.400+0000 7fb827650640 1 --2- 172.21.15.86:0/4262853286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8201029a0 0x7fb8201092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.400+0000 7fb827650640 1 --2- 172.21.15.86:0/4262853286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb820101fe0 0x7fb820102460 unknown :-1 s=CLOSED pgs=500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.400+0000 7fb827650640 1 --2- 172.21.15.86:0/4262853286 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb820100e90 0x7fb8201012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.400+0000 7fb827650640 1 -- 172.21.15.86:0/4262853286 >> 172.21.15.86:0/4262853286 conn(0x7fb8200fc400 msgr2=0x7fb8200fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:46.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.400+0000 7fb827650640 1 -- 172.21.15.86:0/4262853286 shutdown_connections 2024-04-21T08:38:46.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.400+0000 7fb827650640 1 -- 172.21.15.86:0/4262853286 wait complete. 2024-04-21T08:38:46.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.400+0000 7fb827650640 1 Processor -- start 2024-04-21T08:38:46.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.400+0000 7fb827650640 1 -- start start 2024-04-21T08:38:46.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.400+0000 7fb827650640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb820100e90 0x7fb820197150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.400+0000 7fb827650640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb820101fe0 0x7fb820197690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.400+0000 7fb827650640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8201029a0 0x7fb82019e700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.400+0000 7fb827650640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb82010c9f0 con 0x7fb820101fe0 2024-04-21T08:38:46.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.401+0000 7fb827650640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb82010c870 con 0x7fb820100e90 2024-04-21T08:38:46.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.401+0000 7fb827650640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb82010cb70 con 0x7fb8201029a0 2024-04-21T08:38:46.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.401+0000 7fb824beb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb820101fe0 0x7fb820197690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:46.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.401+0000 7fb824beb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb820101fe0 0x7fb820197690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57374/0 (socket says 172.21.15.86:57374) 2024-04-21T08:38:46.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.401+0000 7fb824beb640 1 -- 172.21.15.86:0/701212650 learned_addr learned my addr 172.21.15.86:0/701212650 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:46.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.401+0000 7fb8253ec640 1 --2- 172.21.15.86:0/701212650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb820100e90 0x7fb820197150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:46.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.401+0000 7fb825bed640 1 --2- 172.21.15.86:0/701212650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8201029a0 0x7fb82019e700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:46.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.401+0000 7fb824beb640 1 -- 172.21.15.86:0/701212650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8201029a0 msgr2=0x7fb82019e700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:46.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.401+0000 7fb824beb640 1 --2- 172.21.15.86:0/701212650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8201029a0 0x7fb82019e700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.401+0000 7fb824beb640 1 -- 172.21.15.86:0/701212650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb820100e90 msgr2=0x7fb820197150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:46.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.401+0000 7fb824beb640 1 --2- 172.21.15.86:0/701212650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb820100e90 0x7fb820197150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.401+0000 7fb824beb640 1 -- 172.21.15.86:0/701212650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb82019ed70 con 0x7fb820101fe0 2024-04-21T08:38:46.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.401+0000 7fb824beb640 1 --2- 172.21.15.86:0/701212650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb820101fe0 0x7fb820197690 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb810000c00 tx=0x7fb810004270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:46.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.401+0000 7fb8167fc640 1 -- 172.21.15.86:0/701212650 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb810005000 con 0x7fb820101fe0 2024-04-21T08:38:46.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.401+0000 7fb8167fc640 1 -- 172.21.15.86:0/701212650 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb810004aa0 con 0x7fb820101fe0 2024-04-21T08:38:46.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.401+0000 7fb827650640 1 -- 172.21.15.86:0/701212650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb8200690e0 con 0x7fb820101fe0 2024-04-21T08:38:46.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.401+0000 7fb8167fc640 1 -- 172.21.15.86:0/701212650 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb810047480 con 0x7fb820101fe0 2024-04-21T08:38:46.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.401+0000 7fb827650640 1 -- 172.21.15.86:0/701212650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb820069630 con 0x7fb820101fe0 2024-04-21T08:38:46.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.402+0000 7fb827650640 1 -- 172.21.15.86:0/701212650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb820068d70 con 0x7fb820101fe0 2024-04-21T08:38:46.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.403+0000 7fb8167fc640 1 -- 172.21.15.86:0/701212650 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 20) ==== 138663+0+0 (secure 0 0 0) 0x7fb810003780 con 0x7fb820101fe0 2024-04-21T08:38:46.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.404+0000 7fb8167fc640 1 --2- 172.21.15.86:0/701212650 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb80c0a3c00 0x7fb80c0a60c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.404+0000 7fb8253ec640 1 -- 172.21.15.86:0/701212650 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb80c0a3c00 msgr2=0x7fb80c0a60c0 unknown :-1 s=STATE_CONNECTING_RE l=1).process reconnect failed to v2:172.21.15.86:6800/1364156965 2024-04-21T08:38:46.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.404+0000 7fb8167fc640 1 -- 172.21.15.86:0/701212650 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(93..93 src has 1..93) ==== 9150+0+0 (secure 0 0 0) 0x7fb8100f4ce0 con 0x7fb820101fe0 2024-04-21T08:38:46.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.404+0000 7fb8253ec640 1 --2- 172.21.15.86:0/701212650 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb80c0a3c00 0x7fb80c0a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._fault waiting 0.200000 2024-04-21T08:38:46.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.408+0000 7fb8167fc640 1 -- 172.21.15.86:0/701212650 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb8100c1330 con 0x7fb820101fe0 2024-04-21T08:38:46.496 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:38:46 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:38:46.055+0000 7f669b429200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T08:38:46.496 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:38:46 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z[33683]: 2024-04-21T08:38:46.278+0000 7f669b429200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T08:38:46.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:46 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/465099298' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-21T08:38:46.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:46 smithi179 ceph-mon[31829]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:38:46.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:46 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3159402181' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:46.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:46 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/4128797878' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:46.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:46 smithi179 ceph-mon[31829]: Standby manager daemon y restarted 2024-04-21T08:38:46.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:46 smithi179 ceph-mon[31829]: Standby manager daemon y started 2024-04-21T08:38:46.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:46 smithi179 ceph-mon[31829]: from='mgr.? 172.21.15.88:0/1945585371' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-21T08:38:46.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:46 smithi179 ceph-mon[31829]: from='mgr.? 172.21.15.88:0/1945585371' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:38:46.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:46 smithi179 ceph-mon[31829]: from='mgr.? 172.21.15.88:0/1945585371' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-21T08:38:46.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:46 smithi179 ceph-mon[31829]: from='mgr.? 172.21.15.88:0/1945585371' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:38:46.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.591+0000 7fb827650640 1 -- 172.21.15.86:0/701212650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fb82019efa0 con 0x7fb820101fe0 2024-04-21T08:38:46.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.592+0000 7fb8167fc640 1 -- 172.21.15.86:0/701212650 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v93) ==== 100+0+11 (secure 0 0 0) 0x7fb8100c0c30 con 0x7fb820101fe0 2024-04-21T08:38:46.591 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-21T08:38:46.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.594+0000 7fb827650640 1 -- 172.21.15.86:0/701212650 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb80c0a3c00 msgr2=0x7fb80c0a60c0 unknown :-1 s=STATE_CONNECTING l=1).mark_down 2024-04-21T08:38:46.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.594+0000 7fb827650640 1 --2- 172.21.15.86:0/701212650 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb80c0a3c00 0x7fb80c0a60c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.594+0000 7fb827650640 1 -- 172.21.15.86:0/701212650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb820101fe0 msgr2=0x7fb820197690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:46.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.594+0000 7fb827650640 1 --2- 172.21.15.86:0/701212650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb820101fe0 0x7fb820197690 secure :-1 s=READY pgs=501 cs=0 l=1 rev1=1 crypto rx=0x7fb810000c00 tx=0x7fb810004270 comp rx=0 tx=0).stop 2024-04-21T08:38:46.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.594+0000 7fb827650640 1 -- 172.21.15.86:0/701212650 shutdown_connections 2024-04-21T08:38:46.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.594+0000 7fb827650640 1 --2- 172.21.15.86:0/701212650 >> [v2:172.21.15.86:6800/1364156965,v1:172.21.15.86:6801/1364156965] conn(0x7fb80c0a3c00 0x7fb80c0a60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.594+0000 7fb827650640 1 --2- 172.21.15.86:0/701212650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8201029a0 0x7fb82019e700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.594+0000 7fb827650640 1 --2- 172.21.15.86:0/701212650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb820101fe0 0x7fb820197690 unknown :-1 s=CLOSED pgs=501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.594+0000 7fb827650640 1 --2- 172.21.15.86:0/701212650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb820100e90 0x7fb820197150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.594+0000 7fb827650640 1 -- 172.21.15.86:0/701212650 >> 172.21.15.86:0/701212650 conn(0x7fb8200fc400 msgr2=0x7fb8201054e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:46.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.594+0000 7fb827650640 1 -- 172.21.15.86:0/701212650 shutdown_connections 2024-04-21T08:38:46.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.594+0000 7fb827650640 1 -- 172.21.15.86:0/701212650 wait complete. 2024-04-21T08:38:46.603 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:38:46 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:38:46.398+0000 7faffae22200 -1 mgr[py] Module status has missing NOTIFY_TYPES member 2024-04-21T08:38:46.603 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:46 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/465099298' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-21T08:38:46.603 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:46 smithi086 ceph-mon[27939]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:38:46.603 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:46 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3159402181' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:46.603 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:46 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/4128797878' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:46.603 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:46 smithi086 ceph-mon[27939]: Standby manager daemon y restarted 2024-04-21T08:38:46.603 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:46 smithi086 ceph-mon[27939]: Standby manager daemon y started 2024-04-21T08:38:46.603 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:46 smithi086 ceph-mon[27939]: from='mgr.? 172.21.15.88:0/1945585371' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-21T08:38:46.603 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:46 smithi086 ceph-mon[27939]: from='mgr.? 172.21.15.88:0/1945585371' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:38:46.603 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:46 smithi086 ceph-mon[27939]: from='mgr.? 172.21.15.88:0/1945585371' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-21T08:38:46.604 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:46 smithi086 ceph-mon[27939]: from='mgr.? 172.21.15.88:0/1945585371' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:38:46.604 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:38:46.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:46 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/465099298' entity='client.admin' cmd='[{"prefix": "fs set", "fs_name": "cephfs", "var": "balance_automate", "val": "true"}]': finished 2024-04-21T08:38:46.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:46 smithi088 ceph-mon[31259]: fsmap cephfs:5 5 up:standby-replay 2 up:standby 5 up:active 2024-04-21T08:38:46.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:46 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3159402181' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:46.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:46 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/4128797878' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:46.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:46 smithi088 ceph-mon[31259]: Standby manager daemon y restarted 2024-04-21T08:38:46.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:46 smithi088 ceph-mon[31259]: Standby manager daemon y started 2024-04-21T08:38:46.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:46 smithi088 ceph-mon[31259]: from='mgr.? 172.21.15.88:0/1945585371' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/crt"}]: dispatch 2024-04-21T08:38:46.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:46 smithi088 ceph-mon[31259]: from='mgr.? 172.21.15.88:0/1945585371' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:38:46.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:46 smithi088 ceph-mon[31259]: from='mgr.? 172.21.15.88:0/1945585371' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/y/key"}]: dispatch 2024-04-21T08:38:46.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:46 smithi088 ceph-mon[31259]: from='mgr.? 172.21.15.88:0/1945585371' entity='mgr.y' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:38:46.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.716+0000 7f2648b15640 1 -- 172.21.15.86:0/1439205165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2644103230 msgr2=0x7f26441036b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:46.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.716+0000 7f2648b15640 1 --2- 172.21.15.86:0/1439205165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2644103230 0x7f26441036b0 secure :-1 s=READY pgs=505 cs=0 l=1 rev1=1 crypto rx=0x7f2638009930 tx=0x7f263802bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:46.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.716+0000 7f2648b15640 1 -- 172.21.15.86:0/1439205165 shutdown_connections 2024-04-21T08:38:46.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.716+0000 7f2648b15640 1 --2- 172.21.15.86:0/1439205165 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2644103bf0 0x7f264410a480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.716+0000 7f2648b15640 1 --2- 172.21.15.86:0/1439205165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2644103230 0x7f26441036b0 unknown :-1 s=CLOSED pgs=505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.716+0000 7f2648b15640 1 --2- 172.21.15.86:0/1439205165 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2644100260 0x7f2644100680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.716+0000 7f2648b15640 1 -- 172.21.15.86:0/1439205165 >> 172.21.15.86:0/1439205165 conn(0x7f26440fbb90 msgr2=0x7f26440fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:46.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.716+0000 7f2648b15640 1 -- 172.21.15.86:0/1439205165 shutdown_connections 2024-04-21T08:38:46.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.716+0000 7f2648b15640 1 -- 172.21.15.86:0/1439205165 wait complete. 2024-04-21T08:38:46.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.717+0000 7f2648b15640 1 Processor -- start 2024-04-21T08:38:46.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.717+0000 7f2648b15640 1 -- start start 2024-04-21T08:38:46.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.717+0000 7f2648b15640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2644100260 0x7f2644198230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.717+0000 7f2648b15640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2644103230 0x7f2644198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.717+0000 7f2648b15640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2644103bf0 0x7f264419f7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:46.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.717+0000 7f2648b15640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f264410db00 con 0x7f2644103bf0 2024-04-21T08:38:46.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.717+0000 7f2648b15640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f264410d980 con 0x7f2644100260 2024-04-21T08:38:46.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.717+0000 7f2648b15640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f264410dc80 con 0x7f2644103230 2024-04-21T08:38:46.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.717+0000 7f2642d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2644103bf0 0x7f264419f7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:46.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.718+0000 7f2642d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2644103bf0 0x7f264419f7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57410/0 (socket says 172.21.15.86:57410) 2024-04-21T08:38:46.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.718+0000 7f2642d9d640 1 -- 172.21.15.86:0/1774050096 learned_addr learned my addr 172.21.15.86:0/1774050096 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:46.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.718+0000 7f2641d9b640 1 --2- 172.21.15.86:0/1774050096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2644103230 0x7f2644198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:46.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.718+0000 7f264259c640 1 --2- 172.21.15.86:0/1774050096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2644100260 0x7f2644198230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:46.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.718+0000 7f2642d9d640 1 -- 172.21.15.86:0/1774050096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2644103230 msgr2=0x7f2644198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:46.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.718+0000 7f2642d9d640 1 --2- 172.21.15.86:0/1774050096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2644103230 0x7f2644198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.718+0000 7f2642d9d640 1 -- 172.21.15.86:0/1774050096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2644100260 msgr2=0x7f2644198230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:46.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.718+0000 7f2642d9d640 1 --2- 172.21.15.86:0/1774050096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2644100260 0x7f2644198230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.718+0000 7f2642d9d640 1 -- 172.21.15.86:0/1774050096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f264419fe50 con 0x7f2644103bf0 2024-04-21T08:38:46.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.718+0000 7f2642d9d640 1 --2- 172.21.15.86:0/1774050096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2644103bf0 0x7f264419f7e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f263c00eb10 tx=0x7f263c00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:46.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.718+0000 7f26337fe640 1 -- 172.21.15.86:0/1774050096 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c004b20 con 0x7f2644103bf0 2024-04-21T08:38:46.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.718+0000 7f26337fe640 1 -- 172.21.15.86:0/1774050096 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f263c004cc0 con 0x7f2644103bf0 2024-04-21T08:38:46.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.718+0000 7f2648b15640 1 -- 172.21.15.86:0/1774050096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f26441a0140 con 0x7f2644103bf0 2024-04-21T08:38:46.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.718+0000 7f26337fe640 1 -- 172.21.15.86:0/1774050096 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f263c010670 con 0x7f2644103bf0 2024-04-21T08:38:46.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.718+0000 7f2648b15640 1 -- 172.21.15.86:0/1774050096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f26441a0610 con 0x7f2644103bf0 2024-04-21T08:38:46.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.719+0000 7f2648b15640 1 -- 172.21.15.86:0/1774050096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2644109420 con 0x7f2644103bf0 2024-04-21T08:38:46.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.721+0000 7f26337fe640 1 -- 172.21.15.86:0/1774050096 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7f263c005000 con 0x7f2644103bf0 2024-04-21T08:38:46.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.722+0000 7f26337fe640 1 -- 172.21.15.86:0/1774050096 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f263c0d0af0 con 0x7f2644103bf0 2024-04-21T08:38:46.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.725+0000 7f26337fe640 1 -- 172.21.15.86:0/1774050096 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f263c053ff0 con 0x7f2644103bf0 2024-04-21T08:38:46.866 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:38:46 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:38:46.629+0000 7faffae22200 -1 mgr[py] Module nfs has missing NOTIFY_TYPES member 2024-04-21T08:38:46.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.978+0000 7f2648b15640 1 -- 172.21.15.86:0/1774050096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f2644062b80 con 0x7f2644103bf0 2024-04-21T08:38:46.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.979+0000 7f26337fe640 1 -- 172.21.15.86:0/1774050096 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f263c09d400 con 0x7f2644103bf0 2024-04-21T08:38:46.978 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-21T08:38:46.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.981+0000 7f2648b15640 1 -- 172.21.15.86:0/1774050096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2644103bf0 msgr2=0x7f264419f7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:46.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.981+0000 7f2648b15640 1 --2- 172.21.15.86:0/1774050096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2644103bf0 0x7f264419f7e0 secure :-1 s=READY pgs=506 cs=0 l=1 rev1=1 crypto rx=0x7f263c00eb10 tx=0x7f263c00c2d0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.981+0000 7f2648b15640 1 -- 172.21.15.86:0/1774050096 shutdown_connections 2024-04-21T08:38:46.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.981+0000 7f2648b15640 1 --2- 172.21.15.86:0/1774050096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2644103bf0 0x7f264419f7e0 unknown :-1 s=CLOSED pgs=506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.981+0000 7f2648b15640 1 --2- 172.21.15.86:0/1774050096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2644103230 0x7f2644198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.981+0000 7f2648b15640 1 --2- 172.21.15.86:0/1774050096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2644100260 0x7f2644198230 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:46.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.981+0000 7f2648b15640 1 -- 172.21.15.86:0/1774050096 >> 172.21.15.86:0/1774050096 conn(0x7f26440fbb90 msgr2=0x7f26441065f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:46.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.981+0000 7f2648b15640 1 -- 172.21.15.86:0/1774050096 shutdown_connections 2024-04-21T08:38:46.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:46.981+0000 7f2648b15640 1 -- 172.21.15.86:0/1774050096 wait complete. 2024-04-21T08:38:46.992 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:38:47.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.092+0000 7f1e256ba640 1 -- 172.21.15.86:0/2218653714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e20101fa0 msgr2=0x7f1e201023c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:47.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.092+0000 7f1e256ba640 1 --2- 172.21.15.86:0/2218653714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e20101fa0 0x7f1e201023c0 secure :-1 s=READY pgs=513 cs=0 l=1 rev1=1 crypto rx=0x7f1e10009900 tx=0x7f1e1002bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:47.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.092+0000 7f1e256ba640 1 -- 172.21.15.86:0/2218653714 shutdown_connections 2024-04-21T08:38:47.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.092+0000 7f1e256ba640 1 --2- 172.21.15.86:0/2218653714 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e20103b50 0x7f1e2010a3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.092+0000 7f1e256ba640 1 --2- 172.21.15.86:0/2218653714 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e20103190 0x7f1e20103610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.092+0000 7f1e256ba640 1 --2- 172.21.15.86:0/2218653714 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e20101fa0 0x7f1e201023c0 unknown :-1 s=CLOSED pgs=513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.092+0000 7f1e256ba640 1 -- 172.21.15.86:0/2218653714 >> 172.21.15.86:0/2218653714 conn(0x7f1e200fd530 msgr2=0x7f1e200ff950 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:47.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.092+0000 7f1e256ba640 1 -- 172.21.15.86:0/2218653714 shutdown_connections 2024-04-21T08:38:47.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.092+0000 7f1e256ba640 1 -- 172.21.15.86:0/2218653714 wait complete. 2024-04-21T08:38:47.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.092+0000 7f1e256ba640 1 Processor -- start 2024-04-21T08:38:47.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.093+0000 7f1e256ba640 1 -- start start 2024-04-21T08:38:47.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.093+0000 7f1e256ba640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e20101fa0 0x7f1e201981c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:47.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.093+0000 7f1e256ba640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e20103190 0x7f1e20198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:47.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.093+0000 7f1e256ba640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e20103b50 0x7f1e2019f770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:47.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.093+0000 7f1e256ba640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2010dae0 con 0x7f1e20103b50 2024-04-21T08:38:47.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.093+0000 7f1e256ba640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2010d960 con 0x7f1e20103190 2024-04-21T08:38:47.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.093+0000 7f1e256ba640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1e2010dc60 con 0x7f1e20101fa0 2024-04-21T08:38:47.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.093+0000 7f1e1f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e20103b50 0x7f1e2019f770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:47.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.093+0000 7f1e1f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e20103b50 0x7f1e2019f770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57530/0 (socket says 172.21.15.86:57530) 2024-04-21T08:38:47.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.093+0000 7f1e1effd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e20101fa0 0x7f1e201981c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:47.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.093+0000 7f1e1f7fe640 1 -- 172.21.15.86:0/2971531704 learned_addr learned my addr 172.21.15.86:0/2971531704 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:47.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.093+0000 7f1e1e7fc640 1 --2- 172.21.15.86:0/2971531704 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e20103190 0x7f1e20198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:47.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.093+0000 7f1e1effd640 1 -- 172.21.15.86:0/2971531704 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e20103190 msgr2=0x7f1e20198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:47.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.093+0000 7f1e1effd640 1 --2- 172.21.15.86:0/2971531704 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e20103190 0x7f1e20198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.093+0000 7f1e1effd640 1 -- 172.21.15.86:0/2971531704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e20103b50 msgr2=0x7f1e2019f770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:47.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.093+0000 7f1e1effd640 1 --2- 172.21.15.86:0/2971531704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e20103b50 0x7f1e2019f770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.093+0000 7f1e1effd640 1 -- 172.21.15.86:0/2971531704 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e2019fde0 con 0x7f1e20101fa0 2024-04-21T08:38:47.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.093+0000 7f1e1effd640 1 --2- 172.21.15.86:0/2971531704 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e20101fa0 0x7f1e201981c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1e10036040 tx=0x7f1e10004270 comp rx=0 tx=0).ready entity=mon.1 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:47.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.094+0000 7f1e07fff640 1 -- 172.21.15.86:0/2971531704 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e10043020 con 0x7f1e20101fa0 2024-04-21T08:38:47.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.094+0000 7f1e256ba640 1 -- 172.21.15.86:0/2971531704 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e201a0070 con 0x7f1e20101fa0 2024-04-21T08:38:47.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.094+0000 7f1e07fff640 1 -- 172.21.15.86:0/2971531704 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e10004aa0 con 0x7f1e20101fa0 2024-04-21T08:38:47.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.094+0000 7f1e07fff640 1 -- 172.21.15.86:0/2971531704 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e10003c90 con 0x7f1e20101fa0 2024-04-21T08:38:47.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.094+0000 7f1e256ba640 1 -- 172.21.15.86:0/2971531704 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e201a0590 con 0x7f1e20101fa0 2024-04-21T08:38:47.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.095+0000 7f1e256ba640 1 -- 172.21.15.86:0/2971531704 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e20109390 con 0x7f1e20101fa0 2024-04-21T08:38:47.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.096+0000 7f1e07fff640 1 -- 172.21.15.86:0/2971531704 <== mon.1 v2:172.21.15.179:3300/0 4 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7f1e10046020 con 0x7f1e20101fa0 2024-04-21T08:38:47.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.097+0000 7f1e07fff640 1 -- 172.21.15.86:0/2971531704 <== mon.1 v2:172.21.15.179:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f1e100f2a10 con 0x7f1e20101fa0 2024-04-21T08:38:47.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.100+0000 7f1e07fff640 1 -- 172.21.15.86:0/2971531704 <== mon.1 v2:172.21.15.179:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1e1004d020 con 0x7f1e20101fa0 2024-04-21T08:38:47.267 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:38:46 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:38:46.903+0000 7faf5a756700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:38:47.267 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:38:46 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:38:46.903+0000 7faf5a756700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:38:47.267 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:38:46 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:38:46.903+0000 7faf5a756700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:38:47.267 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:38:46 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:38:46.903+0000 7faf5a756700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:38:47.267 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:38:46 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:38:46.903+0000 7faf5a756700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:38:47.267 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:38:46 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:38:46.906+0000 7faf5c79a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:38:47.268 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:38:46 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:38:46.906+0000 7faf5c79a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:38:47.268 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:38:46 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:38:46.906+0000 7faf5c79a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:38:47.268 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:38:46 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:38:46.906+0000 7faf5c79a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:38:47.268 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:38:46 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:38:46.906+0000 7faf5c79a700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:38:47.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.291+0000 7f1e256ba640 1 -- 172.21.15.86:0/2971531704 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f1e2002d790 con 0x7f1e20101fa0 2024-04-21T08:38:47.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.292+0000 7f1e07fff640 1 -- 172.21.15.86:0/2971531704 <== mon.1 v2:172.21.15.179:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+25067 (secure 0 0 0) 0x7f1e100dde20 con 0x7f1e20101fa0 2024-04-21T08:38:47.292 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:38:47.292 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":94,"fsid":"f8cec7cc-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:22.687329+0000","modified":"2024-04-21T08:38:46.631676+0000","last_up_change":"2024-04-21T08:35:30.048315+0000","last_in_change":"2024-04-21T08:35:10.099496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:45.279109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:18.300782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:19.574636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:22.905632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89b7601f-0620-4db2-90f1-b45cea173738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6803","nonce":2571516897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6805","nonce":2571516897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6809","nonce":2571516897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6807","nonce":2571516897}]},"public_addr":"172.21.15.86:6803/2571516897","cluster_addr":"172.21.15.86:6805/2571516897","heartbeat_back_addr":"172.21.15.86:6809/2571516897","heartbeat_front_addr":"172.21.15.86:6807/2571516897","state":["exists","up"]},{"osd":1,"uuid":"4891031c-f98c-42bc-bb3b-a420855ecc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6801","nonce":74067182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6803","nonce":74067182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6807","nonce":74067182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6805","nonce":74067182}]},"public_addr":"172.21.15.88:6801/74067182","cluster_addr":"172.21.15.88:6803/74067182","heartbeat_back_addr":"172.21.15.88:6807/74067182","heartbeat_front_addr":"172.21.15.88:6805/74067182","state":["exists","up"]},{"osd":2,"uuid":"60d12aa4-9312-4fa2-8d0b-2b3f42caad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6801","nonce":4137659051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6803","nonce":4137659051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6807","nonce":4137659051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6805","nonce":4137659051}]},"public_addr":"172.21.15.179:6801/4137659051","cluster_addr":"172.21.15.179:6803/4137659051","heartbeat_back_addr":"172.21.15.179:6807/4137659051","heartbeat_front_addr":"172.21.15.179:6805/4137659051","state":["exists","up"]},{"osd":3,"uuid":"404c3158-8cb0-4c02-b48a-f9ee5ff7b784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6811","nonce":2309603148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6813","nonce":2309603148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6817","nonce":2309603148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6815","nonce":2309603148}]},"public_addr":"172.21.15.86:6811/2309603148","cluster_addr":"172.21.15.86:6813/2309603148","heartbeat_back_addr":"172.21.15.86:6817/2309603148","heartbeat_front_addr":"172.21.15.86:6815/2309603148","state":["exists","up"]},{"osd":4,"uuid":"458ff202-d781-4ec7-ac93-0b49f835b503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6809","nonce":1818714217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6811","nonce":1818714217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6815","nonce":1818714217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6813","nonce":1818714217}]},"public_addr":"172.21.15.88:6809/1818714217","cluster_addr":"172.21.15.88:6811/1818714217","heartbeat_back_addr":"172.21.15.88:6815/1818714217","heartbeat_front_addr":"172.21.15.88:6813/1818714217","state":["exists","up"]},{"osd":5,"uuid":"8123cc90-bd8b-4204-bf31-46b34e9f589d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6809","nonce":863840990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6811","nonce":863840990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6815","nonce":863840990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6813","nonce":863840990}]},"public_addr":"172.21.15.179:6809/863840990","cluster_addr":"172.21.15.179:6811/863840990","heartbeat_back_addr":"172.21.15.179:6815/863840990","heartbeat_front_addr":"172.21.15.179:6813/863840990","state":["exists","up"]},{"osd":6,"uuid":"aac9d7c7-659a-4110-903a-a871c086066f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6819","nonce":1224231118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6821","nonce":1224231118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6825","nonce":1224231118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6823","nonce":1224231118}]},"public_addr":"172.21.15.86:6819/1224231118","cluster_addr":"172.21.15.86:6821/1224231118","heartbeat_back_addr":"172.21.15.86:6825/1224231118","heartbeat_front_addr":"172.21.15.86:6823/1224231118","state":["exists","up"]},{"osd":7,"uuid":"df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6817","nonce":136629878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6819","nonce":136629878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6823","nonce":136629878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6821","nonce":136629878}]},"public_addr":"172.21.15.88:6817/136629878","cluster_addr":"172.21.15.88:6819/136629878","heartbeat_back_addr":"172.21.15.88:6823/136629878","heartbeat_front_addr":"172.21.15.88:6821/136629878","state":["exists","up"]},{"osd":8,"uuid":"63e59f4b-0426-46db-92f7-c55dae61b5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6817","nonce":1770418957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6819","nonce":1770418957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6823","nonce":1770418957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6821","nonce":1770418957}]},"public_addr":"172.21.15.179:6817/1770418957","cluster_addr":"172.21.15.179:6819/1770418957","heartbeat_back_addr":"172.21.15.179:6823/1770418957","heartbeat_front_addr":"172.21.15.179:6821/1770418957","state":["exists","up"]},{"osd":9,"uuid":"5841bb36-ed6b-45d8-b857-eeab139c44e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6827","nonce":1108500274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6829","nonce":1108500274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6833","nonce":1108500274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6831","nonce":1108500274}]},"public_addr":"172.21.15.86:6827/1108500274","cluster_addr":"172.21.15.86:6829/1108500274","heartbeat_back_addr":"172.21.15.86:6833/1108500274","heartbeat_front_addr":"172.21.15.86:6831/1108500274","state":["exists","up"]},{"osd":10,"uuid":"f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6825","nonce":1294275353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6827","nonce":1294275353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6831","nonce":1294275353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6829","nonce":1294275353}]},"public_addr":"172.21.15.88:6825/1294275353","cluster_addr":"172.21.15.88:6827/1294275353","heartbeat_back_addr":"172.21.15.88:6831/1294275353","heartbeat_front_addr":"172.21.15.88:6829/1294275353","state":["exists","up"]},{"osd":11,"uuid":"e0d4cc76-e7a6-4a6b-909c-95488ccd5a36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6825","nonce":4007070233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6827","nonce":4007070233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6831","nonce":4007070233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6829","nonce":4007070233}]},"public_addr":"172.21.15.179:6825/4007070233","cluster_addr":"172.21.15.179:6827/4007070233","heartbeat_back_addr":"172.21.15.179:6831/4007070233","heartbeat_front_addr":"172.21.15.179:6829/4007070233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:55.210605+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:18.723090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:42.547979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:06.421622+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:30.858315+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:55.455313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.623369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:46.185111+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.103627+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:37.297545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:02.637431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:28.199292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1068089552":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/4273877219":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/2707594358":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/4060044103":"2024-04-22T08:38:46.631621+0000","172.21.15.86:6800/1364156965":"2024-04-22T08:38:46.631621+0000","172.21.15.86:6800/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/1364156965":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/1412393225":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6800/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/1687852404":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2020707404":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2568130933":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/3952607648":"2024-04-22T08:28:57.384403+0000","172.21.15.86:6801/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/2756903426":"2024-04-22T08:29:23.190001+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:38:47.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.294+0000 7f1e256ba640 1 -- 172.21.15.86:0/2971531704 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e20101fa0 msgr2=0x7f1e201981c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:47.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.294+0000 7f1e256ba640 1 --2- 172.21.15.86:0/2971531704 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e20101fa0 0x7f1e201981c0 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f1e10036040 tx=0x7f1e10004270 comp rx=0 tx=0).stop 2024-04-21T08:38:47.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.294+0000 7f1e256ba640 1 -- 172.21.15.86:0/2971531704 shutdown_connections 2024-04-21T08:38:47.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.294+0000 7f1e256ba640 1 --2- 172.21.15.86:0/2971531704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e20103b50 0x7f1e2019f770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.294+0000 7f1e256ba640 1 --2- 172.21.15.86:0/2971531704 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e20103190 0x7f1e20198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.294+0000 7f1e256ba640 1 --2- 172.21.15.86:0/2971531704 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e20101fa0 0x7f1e201981c0 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.294+0000 7f1e256ba640 1 -- 172.21.15.86:0/2971531704 >> 172.21.15.86:0/2971531704 conn(0x7f1e200fd530 msgr2=0x7f1e20106560 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:47.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.294+0000 7f1e256ba640 1 -- 172.21.15.86:0/2971531704 shutdown_connections 2024-04-21T08:38:47.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.294+0000 7f1e256ba640 1 -- 172.21.15.86:0/2971531704 wait complete. 2024-04-21T08:38:47.305 INFO:tasks.fwd_scrub.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:31:45.279109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:36:18.300782+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:36:19.574636+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:36:22.905632+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:38:47.305 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:38:47.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.408+0000 7faacdcfd640 1 -- 172.21.15.86:0/1747110863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faac8100e90 msgr2=0x7faac81012b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:47.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.408+0000 7faacdcfd640 1 --2- 172.21.15.86:0/1747110863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faac8100e90 0x7faac81012b0 secure :-1 s=READY pgs=514 cs=0 l=1 rev1=1 crypto rx=0x7faab8009900 tx=0x7faab802bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:47.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.408+0000 7faacdcfd640 1 -- 172.21.15.86:0/1747110863 shutdown_connections 2024-04-21T08:38:47.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.408+0000 7faacdcfd640 1 --2- 172.21.15.86:0/1747110863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faac81029a0 0x7faac81092c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.408+0000 7faacdcfd640 1 --2- 172.21.15.86:0/1747110863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faac8101fe0 0x7faac8102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.408+0000 7faacdcfd640 1 --2- 172.21.15.86:0/1747110863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faac8100e90 0x7faac81012b0 unknown :-1 s=CLOSED pgs=514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.408+0000 7faacdcfd640 1 -- 172.21.15.86:0/1747110863 >> 172.21.15.86:0/1747110863 conn(0x7faac80fc400 msgr2=0x7faac80fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:47.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.408+0000 7faacdcfd640 1 -- 172.21.15.86:0/1747110863 shutdown_connections 2024-04-21T08:38:47.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.408+0000 7faacdcfd640 1 -- 172.21.15.86:0/1747110863 wait complete. 2024-04-21T08:38:47.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.409+0000 7faacdcfd640 1 Processor -- start 2024-04-21T08:38:47.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.409+0000 7faacdcfd640 1 -- start start 2024-04-21T08:38:47.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.409+0000 7faacdcfd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faac8100e90 0x7faac8197140 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:47.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.409+0000 7faacdcfd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faac8101fe0 0x7faac8197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:47.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.409+0000 7faacdcfd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faac81029a0 0x7faac819e6f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:47.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.409+0000 7faacdcfd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faac810c9e0 con 0x7faac8100e90 2024-04-21T08:38:47.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.409+0000 7faacdcfd640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faac810c860 con 0x7faac8101fe0 2024-04-21T08:38:47.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.409+0000 7faacdcfd640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faac810cb60 con 0x7faac81029a0 2024-04-21T08:38:47.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.409+0000 7faac77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faac8100e90 0x7faac8197140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:47.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.409+0000 7faac77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faac8100e90 0x7faac8197140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57542/0 (socket says 172.21.15.86:57542) 2024-04-21T08:38:47.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.409+0000 7faac77fe640 1 -- 172.21.15.86:0/2605923580 learned_addr learned my addr 172.21.15.86:0/2605923580 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:47.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.409+0000 7faac6ffd640 1 --2- 172.21.15.86:0/2605923580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faac8101fe0 0x7faac8197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:47.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.409+0000 7faac7fff640 1 --2- 172.21.15.86:0/2605923580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faac81029a0 0x7faac819e6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:47.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.410+0000 7faac77fe640 1 -- 172.21.15.86:0/2605923580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faac81029a0 msgr2=0x7faac819e6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:47.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.410+0000 7faac77fe640 1 --2- 172.21.15.86:0/2605923580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faac81029a0 0x7faac819e6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.410+0000 7faac77fe640 1 -- 172.21.15.86:0/2605923580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faac8101fe0 msgr2=0x7faac8197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:47.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.410+0000 7faac77fe640 1 --2- 172.21.15.86:0/2605923580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faac8101fe0 0x7faac8197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.410+0000 7faac77fe640 1 -- 172.21.15.86:0/2605923580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faac819ed60 con 0x7faac8100e90 2024-04-21T08:38:47.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.410+0000 7faac77fe640 1 --2- 172.21.15.86:0/2605923580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faac8100e90 0x7faac8197140 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7faab8006010 tx=0x7faab8005e40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:47.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.410+0000 7faac4ff9640 1 -- 172.21.15.86:0/2605923580 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab8043020 con 0x7faac8100e90 2024-04-21T08:38:47.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.410+0000 7faacdcfd640 1 -- 172.21.15.86:0/2605923580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faac819eff0 con 0x7faac8100e90 2024-04-21T08:38:47.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.410+0000 7faac4ff9640 1 -- 172.21.15.86:0/2605923580 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faab80043d0 con 0x7faac8100e90 2024-04-21T08:38:47.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.410+0000 7faac4ff9640 1 -- 172.21.15.86:0/2605923580 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faab8003b80 con 0x7faac8100e90 2024-04-21T08:38:47.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.410+0000 7faacdcfd640 1 -- 172.21.15.86:0/2605923580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faac819f4b0 con 0x7faac8100e90 2024-04-21T08:38:47.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.411+0000 7faacdcfd640 1 -- 172.21.15.86:0/2605923580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7faac81082c0 con 0x7faac8100e90 2024-04-21T08:38:47.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.413+0000 7faac4ff9640 1 -- 172.21.15.86:0/2605923580 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 21) ==== 138331+0+0 (secure 0 0 0) 0x7faab8046020 con 0x7faac8100e90 2024-04-21T08:38:47.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.414+0000 7faac4ff9640 1 -- 172.21.15.86:0/2605923580 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7faab80f2b30 con 0x7faac8100e90 2024-04-21T08:38:47.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.416+0000 7faac4ff9640 1 -- 172.21.15.86:0/2605923580 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7faab804d020 con 0x7faac8100e90 2024-04-21T08:38:47.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.604+0000 7faacdcfd640 1 -- 172.21.15.86:0/2605923580 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7faac802d5c0 con 0x7faac8100e90 2024-04-21T08:38:47.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.604+0000 7faac4ff9640 1 -- 172.21.15.86:0/2605923580 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7faab80bf480 con 0x7faac8100e90 2024-04-21T08:38:47.603 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-21T08:38:47.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.606+0000 7faacdcfd640 1 -- 172.21.15.86:0/2605923580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faac8100e90 msgr2=0x7faac8197140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:47.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.606+0000 7faacdcfd640 1 --2- 172.21.15.86:0/2605923580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faac8100e90 0x7faac8197140 secure :-1 s=READY pgs=515 cs=0 l=1 rev1=1 crypto rx=0x7faab8006010 tx=0x7faab8005e40 comp rx=0 tx=0).stop 2024-04-21T08:38:47.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.606+0000 7faacdcfd640 1 -- 172.21.15.86:0/2605923580 shutdown_connections 2024-04-21T08:38:47.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.606+0000 7faacdcfd640 1 --2- 172.21.15.86:0/2605923580 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faac81029a0 0x7faac819e6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.606+0000 7faacdcfd640 1 --2- 172.21.15.86:0/2605923580 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faac8101fe0 0x7faac8197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.606+0000 7faacdcfd640 1 --2- 172.21.15.86:0/2605923580 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faac8100e90 0x7faac8197140 unknown :-1 s=CLOSED pgs=515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.606+0000 7faacdcfd640 1 -- 172.21.15.86:0/2605923580 >> 172.21.15.86:0/2605923580 conn(0x7faac80fc400 msgr2=0x7faac8105490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:47.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.606+0000 7faacdcfd640 1 -- 172.21.15.86:0/2605923580 shutdown_connections 2024-04-21T08:38:47.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.606+0000 7faacdcfd640 1 -- 172.21.15.86:0/2605923580 wait complete. 2024-04-21T08:38:47.617 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:38:47.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-04-21T08:38:47.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: Standby manager daemon z restarted 2024-04-21T08:38:47.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: Standby manager daemon z started 2024-04-21T08:38:47.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.? 172.21.15.179:0/739109812' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-21T08:38:47.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.? 172.21.15.179:0/739109812' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:38:47.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1521255174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:47.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.? 172.21.15.179:0/739109812' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-21T08:38:47.619 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.? 172.21.15.179:0/739109812' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:38:47.619 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/701212650' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:47.619 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: Active manager daemon x restarted 2024-04-21T08:38:47.619 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: Activating manager daemon x 2024-04-21T08:38:47.619 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: osdmap e94: 12 total, 12 up, 12 in 2024-04-21T08:38:47.619 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: mgrmap e21: x(active, starting, since 0.0153348s), standbys: z, y 2024-04-21T08:38:47.619 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:38:47.619 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:38:47.619 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:38:47.619 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-21T08:38:47.619 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-21T08:38:47.619 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-21T08:38:47.619 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-21T08:38:47.620 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-21T08:38:47.620 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-21T08:38:47.620 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T08:38:47.620 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T08:38:47.620 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T08:38:47.620 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T08:38:47.620 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T08:38:47.620 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T08:38:47.620 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-21T08:38:47.620 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-21T08:38:47.620 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-21T08:38:47.620 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:38:47.620 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:38:47.620 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:38:47.620 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:38:47.621 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:38:47.621 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:38:47.621 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:38:47.621 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:38:47.621 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:38:47.621 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:38:47.621 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:38:47.621 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:38:47.621 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-21T08:38:47.621 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-21T08:38:47.621 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-21T08:38:47.621 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: Manager daemon x is now available 2024-04-21T08:38:47.621 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:38:47.621 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-21T08:38:47.621 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-21T08:38:47.621 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:47 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1774050096' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:47.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-04-21T08:38:47.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: Standby manager daemon z restarted 2024-04-21T08:38:47.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: Standby manager daemon z started 2024-04-21T08:38:47.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.? 172.21.15.179:0/739109812' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-21T08:38:47.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.? 172.21.15.179:0/739109812' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:38:47.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1521255174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:47.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.? 172.21.15.179:0/739109812' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-21T08:38:47.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.? 172.21.15.179:0/739109812' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:38:47.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/701212650' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:47.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: Active manager daemon x restarted 2024-04-21T08:38:47.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: Activating manager daemon x 2024-04-21T08:38:47.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: osdmap e94: 12 total, 12 up, 12 in 2024-04-21T08:38:47.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: mgrmap e21: x(active, starting, since 0.0153348s), standbys: z, y 2024-04-21T08:38:47.716 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:38:47.716 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:38:47.716 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:38:47.716 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-21T08:38:47.716 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-21T08:38:47.716 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-21T08:38:47.716 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-21T08:38:47.716 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-21T08:38:47.716 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-21T08:38:47.716 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T08:38:47.716 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T08:38:47.716 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T08:38:47.716 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T08:38:47.716 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T08:38:47.717 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T08:38:47.717 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-21T08:38:47.717 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-21T08:38:47.717 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-21T08:38:47.717 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:38:47.717 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:38:47.717 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:38:47.717 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:38:47.717 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:38:47.717 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:38:47.717 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:38:47.717 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:38:47.717 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:38:47.717 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:38:47.718 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:38:47.718 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:38:47.718 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-21T08:38:47.718 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-21T08:38:47.718 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-21T08:38:47.718 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: Manager daemon x is now available 2024-04-21T08:38:47.718 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:38:47.718 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-21T08:38:47.718 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-21T08:38:47.718 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:47 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1774050096' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:47.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.742+0000 7fdd1454b640 1 -- 172.21.15.86:0/32823407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd0c1029a0 msgr2=0x7fdd0c1092c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:47.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.742+0000 7fdd1454b640 1 --2- 172.21.15.86:0/32823407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd0c1029a0 0x7fdd0c1092c0 secure :-1 s=READY pgs=516 cs=0 l=1 rev1=1 crypto rx=0x7fdd04009ca0 tx=0x7fdd0402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.743+0000 7fdd1454b640 1 -- 172.21.15.86:0/32823407 shutdown_connections 2024-04-21T08:38:47.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.743+0000 7fdd1454b640 1 --2- 172.21.15.86:0/32823407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd0c1029a0 0x7fdd0c1092c0 unknown :-1 s=CLOSED pgs=516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.743+0000 7fdd1454b640 1 --2- 172.21.15.86:0/32823407 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd0c101fe0 0x7fdd0c102460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.743+0000 7fdd1454b640 1 --2- 172.21.15.86:0/32823407 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd0c100e90 0x7fdd0c1012b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.743+0000 7fdd1454b640 1 -- 172.21.15.86:0/32823407 >> 172.21.15.86:0/32823407 conn(0x7fdd0c0fc400 msgr2=0x7fdd0c0fe840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:47.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.743+0000 7fdd1454b640 1 -- 172.21.15.86:0/32823407 shutdown_connections 2024-04-21T08:38:47.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.743+0000 7fdd1454b640 1 -- 172.21.15.86:0/32823407 wait complete. 2024-04-21T08:38:47.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.743+0000 7fdd1454b640 1 Processor -- start 2024-04-21T08:38:47.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.743+0000 7fdd1454b640 1 -- start start 2024-04-21T08:38:47.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.743+0000 7fdd1454b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd0c100e90 0x7fdd0c069c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:47.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.743+0000 7fdd1454b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd0c101fe0 0x7fdd0c19cf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:47.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.743+0000 7fdd1454b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd0c1029a0 0x7fdd0c19f540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:47.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.743+0000 7fdd1454b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c10c9f0 con 0x7fdd0c100e90 2024-04-21T08:38:47.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.743+0000 7fdd1454b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c10c870 con 0x7fdd0c101fe0 2024-04-21T08:38:47.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.743+0000 7fdd1454b640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdd0c10cb70 con 0x7fdd0c1029a0 2024-04-21T08:38:47.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.743+0000 7fdd122e7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd0c100e90 0x7fdd0c069c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:47.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.743+0000 7fdd122e7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd0c100e90 0x7fdd0c069c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57550/0 (socket says 172.21.15.86:57550) 2024-04-21T08:38:47.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.743+0000 7fdd122e7640 1 -- 172.21.15.86:0/1096425833 learned_addr learned my addr 172.21.15.86:0/1096425833 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:47.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.743+0000 7fdd11ae6640 1 --2- 172.21.15.86:0/1096425833 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd0c101fe0 0x7fdd0c19cf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:47.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.744+0000 7fdd12ae8640 1 --2- 172.21.15.86:0/1096425833 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd0c1029a0 0x7fdd0c19f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:47.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.744+0000 7fdd122e7640 1 -- 172.21.15.86:0/1096425833 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd0c1029a0 msgr2=0x7fdd0c19f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:47.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.744+0000 7fdd122e7640 1 --2- 172.21.15.86:0/1096425833 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd0c1029a0 0x7fdd0c19f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.744+0000 7fdd122e7640 1 -- 172.21.15.86:0/1096425833 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd0c101fe0 msgr2=0x7fdd0c19cf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:47.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.744+0000 7fdd122e7640 1 --2- 172.21.15.86:0/1096425833 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd0c101fe0 0x7fdd0c19cf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.744+0000 7fdd122e7640 1 -- 172.21.15.86:0/1096425833 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdd0c19fe50 con 0x7fdd0c100e90 2024-04-21T08:38:47.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.744+0000 7fdd122e7640 1 --2- 172.21.15.86:0/1096425833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd0c100e90 0x7fdd0c069c50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800d770 tx=0x7fdd0800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:47.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.744+0000 7fdd037fe640 1 -- 172.21.15.86:0/1096425833 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08017070 con 0x7fdd0c100e90 2024-04-21T08:38:47.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.744+0000 7fdd1454b640 1 -- 172.21.15.86:0/1096425833 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdd0c1a00e0 con 0x7fdd0c100e90 2024-04-21T08:38:47.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.744+0000 7fdd037fe640 1 -- 172.21.15.86:0/1096425833 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdd080054d0 con 0x7fdd0c100e90 2024-04-21T08:38:47.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.744+0000 7fdd037fe640 1 -- 172.21.15.86:0/1096425833 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdd08004c30 con 0x7fdd0c100e90 2024-04-21T08:38:47.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.744+0000 7fdd1454b640 1 -- 172.21.15.86:0/1096425833 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdd0c1a05e0 con 0x7fdd0c100e90 2024-04-21T08:38:47.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: mgrmap e20: x(active, since 9m), standbys: z, y 2024-04-21T08:38:47.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: Standby manager daemon z restarted 2024-04-21T08:38:47.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: Standby manager daemon z started 2024-04-21T08:38:47.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.? 172.21.15.179:0/739109812' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/crt"}]: dispatch 2024-04-21T08:38:47.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.? 172.21.15.179:0/739109812' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/crt"}]: dispatch 2024-04-21T08:38:47.748 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1521255174' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:47.748 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.? 172.21.15.179:0/739109812' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/z/key"}]: dispatch 2024-04-21T08:38:47.748 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.? 172.21.15.179:0/739109812' entity='mgr.z' cmd=[{"prefix": "config-key get", "key": "mgr/dashboard/key"}]: dispatch 2024-04-21T08:38:47.748 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/701212650' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:47.748 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: Active manager daemon x restarted 2024-04-21T08:38:47.748 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: Activating manager daemon x 2024-04-21T08:38:47.748 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: osdmap e94: 12 total, 12 up, 12 in 2024-04-21T08:38:47.748 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: mgrmap e21: x(active, starting, since 0.0153348s), standbys: z, y 2024-04-21T08:38:47.748 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "a"}]: dispatch 2024-04-21T08:38:47.748 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "b"}]: dispatch 2024-04-21T08:38:47.749 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mon metadata", "id": "c"}]: dispatch 2024-04-21T08:38:47.749 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "j"}]: dispatch 2024-04-21T08:38:47.749 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "d"}]: dispatch 2024-04-21T08:38:47.749 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-21T08:38:47.749 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-21T08:38:47.749 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-21T08:38:47.749 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-21T08:38:47.749 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T08:38:47.749 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T08:38:47.749 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T08:38:47.749 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T08:38:47.749 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T08:38:47.749 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T08:38:47.749 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "x", "id": "x"}]: dispatch 2024-04-21T08:38:47.750 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "z", "id": "z"}]: dispatch 2024-04-21T08:38:47.750 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mgr metadata", "who": "y", "id": "y"}]: dispatch 2024-04-21T08:38:47.750 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 0}]: dispatch 2024-04-21T08:38:47.750 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 1}]: dispatch 2024-04-21T08:38:47.750 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 2}]: dispatch 2024-04-21T08:38:47.750 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 3}]: dispatch 2024-04-21T08:38:47.750 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 4}]: dispatch 2024-04-21T08:38:47.750 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 5}]: dispatch 2024-04-21T08:38:47.750 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 6}]: dispatch 2024-04-21T08:38:47.750 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 7}]: dispatch 2024-04-21T08:38:47.750 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 8}]: dispatch 2024-04-21T08:38:47.750 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 9}]: dispatch 2024-04-21T08:38:47.750 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 10}]: dispatch 2024-04-21T08:38:47.750 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "osd metadata", "id": 11}]: dispatch 2024-04-21T08:38:47.751 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata"}]: dispatch 2024-04-21T08:38:47.751 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "osd metadata"}]: dispatch 2024-04-21T08:38:47.751 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mon metadata"}]: dispatch 2024-04-21T08:38:47.751 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: Manager daemon x is now available 2024-04-21T08:38:47.751 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:38:47.751 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/mirror_snapshot_schedule"}]: dispatch 2024-04-21T08:38:47.751 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix":"config rm","who":"mgr","name":"mgr/rbd_support/x/trash_purge_schedule"}]: dispatch 2024-04-21T08:38:47.751 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:47 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1774050096' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:47.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.745+0000 7fdd1454b640 1 -- 172.21.15.86:0/1096425833 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdd0c1081c0 con 0x7fdd0c100e90 2024-04-21T08:38:47.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.746+0000 7fdd037fe640 1 -- 172.21.15.86:0/1096425833 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7fdd08004e10 con 0x7fdd0c100e90 2024-04-21T08:38:47.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.747+0000 7fdd037fe640 1 --2- 172.21.15.86:0/1096425833 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdcf80a39b0 0x7fdcf80a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:47.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.747+0000 7fdd11ae6640 1 --2- 172.21.15.86:0/1096425833 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdcf80a39b0 0x7fdcf80a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:47.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.747+0000 7fdd037fe640 1 -- 172.21.15.86:0/1096425833 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fdd080cfea0 con 0x7fdd0c100e90 2024-04-21T08:38:47.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.747+0000 7fdd11ae6640 1 --2- 172.21.15.86:0/1096425833 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdcf80a39b0 0x7fdcf80a5e70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c0699c0 tx=0x7fdcfc00b390 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:47.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.752+0000 7fdd037fe640 1 -- 172.21.15.86:0/1096425833 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdd0809c090 con 0x7fdd0c100e90 2024-04-21T08:38:47.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.931+0000 7fdd1454b640 1 -- 172.21.15.86:0/1096425833 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7fdd0c062b80 con 0x7fdd0c100e90 2024-04-21T08:38:47.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.931+0000 7fdd037fe640 1 -- 172.21.15.86:0/1096425833 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7fdd080a0390 con 0x7fdd0c100e90 2024-04-21T08:38:47.931 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-21T08:38:47.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.933+0000 7fdd1454b640 1 -- 172.21.15.86:0/1096425833 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdcf80a39b0 msgr2=0x7fdcf80a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:47.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.933+0000 7fdd1454b640 1 --2- 172.21.15.86:0/1096425833 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdcf80a39b0 0x7fdcf80a5e70 secure :-1 s=READY pgs=35 cs=0 l=1 rev1=1 crypto rx=0x7fdd0c0699c0 tx=0x7fdcfc00b390 comp rx=0 tx=0).stop 2024-04-21T08:38:47.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.933+0000 7fdd1454b640 1 -- 172.21.15.86:0/1096425833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd0c100e90 msgr2=0x7fdd0c069c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:47.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.933+0000 7fdd1454b640 1 --2- 172.21.15.86:0/1096425833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd0c100e90 0x7fdd0c069c50 secure :-1 s=READY pgs=517 cs=0 l=1 rev1=1 crypto rx=0x7fdd0800d770 tx=0x7fdd0800dc30 comp rx=0 tx=0).stop 2024-04-21T08:38:47.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.933+0000 7fdd1454b640 1 -- 172.21.15.86:0/1096425833 shutdown_connections 2024-04-21T08:38:47.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.933+0000 7fdd1454b640 1 --2- 172.21.15.86:0/1096425833 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdcf80a39b0 0x7fdcf80a5e70 unknown :-1 s=CLOSED pgs=35 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.933+0000 7fdd1454b640 1 --2- 172.21.15.86:0/1096425833 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdd0c1029a0 0x7fdd0c19f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.933+0000 7fdd1454b640 1 --2- 172.21.15.86:0/1096425833 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdd0c101fe0 0x7fdd0c19cf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.933+0000 7fdd1454b640 1 --2- 172.21.15.86:0/1096425833 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdd0c100e90 0x7fdd0c069c50 unknown :-1 s=CLOSED pgs=517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:47.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.933+0000 7fdd1454b640 1 -- 172.21.15.86:0/1096425833 >> 172.21.15.86:0/1096425833 conn(0x7fdd0c0fc400 msgr2=0x7fdd0c105390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:47.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.934+0000 7fdd1454b640 1 -- 172.21.15.86:0/1096425833 shutdown_connections 2024-04-21T08:38:47.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:47.934+0000 7fdd1454b640 1 -- 172.21.15.86:0/1096425833 wait complete. 2024-04-21T08:38:47.944 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:38:48.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.050+0000 7fd571611640 1 -- 172.21.15.86:0/890792688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd56c102470 msgr2=0x7fd56c102890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:48.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.050+0000 7fd571611640 1 --2- 172.21.15.86:0/890792688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd56c102470 0x7fd56c102890 secure :-1 s=READY pgs=518 cs=0 l=1 rev1=1 crypto rx=0x7fd55c009890 tx=0x7fd55c02bb70 comp rx=0 tx=0).stop 2024-04-21T08:38:48.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.050+0000 7fd571611640 1 -- 172.21.15.86:0/890792688 shutdown_connections 2024-04-21T08:38:48.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.050+0000 7fd571611640 1 --2- 172.21.15.86:0/890792688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd56c101ad0 0x7fd56c109370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:48.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.050+0000 7fd571611640 1 --2- 172.21.15.86:0/890792688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd56c101110 0x7fd56c101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:48.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.050+0000 7fd571611640 1 --2- 172.21.15.86:0/890792688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd56c102470 0x7fd56c102890 unknown :-1 s=CLOSED pgs=518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:48.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.050+0000 7fd571611640 1 -- 172.21.15.86:0/890792688 >> 172.21.15.86:0/890792688 conn(0x7fd56c0fbb80 msgr2=0x7fd56c0fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:48.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.050+0000 7fd571611640 1 -- 172.21.15.86:0/890792688 shutdown_connections 2024-04-21T08:38:48.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.050+0000 7fd571611640 1 -- 172.21.15.86:0/890792688 wait complete. 2024-04-21T08:38:48.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.051+0000 7fd571611640 1 Processor -- start 2024-04-21T08:38:48.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.051+0000 7fd571611640 1 -- start start 2024-04-21T08:38:48.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.051+0000 7fd571611640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd56c101110 0x7fd56c105100 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:48.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.051+0000 7fd571611640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd56c101ad0 0x7fd56c103740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:48.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.051+0000 7fd571611640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd56c102470 0x7fd56c103c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:48.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.051+0000 7fd571611640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd56c10c9b0 con 0x7fd56c101110 2024-04-21T08:38:48.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.051+0000 7fd571611640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd56c10c830 con 0x7fd56c101ad0 2024-04-21T08:38:48.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.051+0000 7fd571611640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd56c10cb30 con 0x7fd56c102470 2024-04-21T08:38:48.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.051+0000 7fd56affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd56c101110 0x7fd56c105100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:48.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.051+0000 7fd56affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd56c101110 0x7fd56c105100 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57574/0 (socket says 172.21.15.86:57574) 2024-04-21T08:38:48.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.051+0000 7fd56affd640 1 -- 172.21.15.86:0/3155011016 learned_addr learned my addr 172.21.15.86:0/3155011016 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:48.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.051+0000 7fd56a7fc640 1 --2- 172.21.15.86:0/3155011016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd56c101ad0 0x7fd56c103740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:48.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.051+0000 7fd56b7fe640 1 --2- 172.21.15.86:0/3155011016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd56c102470 0x7fd56c103c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:48.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.051+0000 7fd56affd640 1 -- 172.21.15.86:0/3155011016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd56c102470 msgr2=0x7fd56c103c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:48.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.051+0000 7fd56affd640 1 --2- 172.21.15.86:0/3155011016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd56c102470 0x7fd56c103c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:48.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.051+0000 7fd56affd640 1 -- 172.21.15.86:0/3155011016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd56c101ad0 msgr2=0x7fd56c103740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:48.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.051+0000 7fd56affd640 1 --2- 172.21.15.86:0/3155011016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd56c101ad0 0x7fd56c103740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:48.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.051+0000 7fd56affd640 1 -- 172.21.15.86:0/3155011016 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd56c104510 con 0x7fd56c101110 2024-04-21T08:38:48.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.052+0000 7fd56affd640 1 --2- 172.21.15.86:0/3155011016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd56c101110 0x7fd56c105100 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd55c0040c0 tx=0x7fd55c0040f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:48.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.052+0000 7fd553fff640 1 -- 172.21.15.86:0/3155011016 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55c041020 con 0x7fd56c101110 2024-04-21T08:38:48.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.052+0000 7fd571611640 1 -- 172.21.15.86:0/3155011016 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd56c1a2300 con 0x7fd56c101110 2024-04-21T08:38:48.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.052+0000 7fd553fff640 1 -- 172.21.15.86:0/3155011016 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd55c02c9c0 con 0x7fd56c101110 2024-04-21T08:38:48.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.052+0000 7fd553fff640 1 -- 172.21.15.86:0/3155011016 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd55c039040 con 0x7fd56c101110 2024-04-21T08:38:48.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.052+0000 7fd571611640 1 -- 172.21.15.86:0/3155011016 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd56c1a28a0 con 0x7fd56c101110 2024-04-21T08:38:48.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.053+0000 7fd571611640 1 -- 172.21.15.86:0/3155011016 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd56c1a2b00 con 0x7fd56c101110 2024-04-21T08:38:48.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.054+0000 7fd553fff640 1 -- 172.21.15.86:0/3155011016 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7fd55c044020 con 0x7fd56c101110 2024-04-21T08:38:48.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.055+0000 7fd553fff640 1 --2- 172.21.15.86:0/3155011016 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd5540a39b0 0x7fd5540a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:48.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.055+0000 7fd56a7fc640 1 --2- 172.21.15.86:0/3155011016 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd5540a39b0 0x7fd5540a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:48.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.055+0000 7fd553fff640 1 -- 172.21.15.86:0/3155011016 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fd55c0f21d0 con 0x7fd56c101110 2024-04-21T08:38:48.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.055+0000 7fd56a7fc640 1 --2- 172.21.15.86:0/3155011016 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd5540a39b0 0x7fd5540a5e70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd56c104e70 tx=0x7fd560009260 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:48.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.059+0000 7fd553fff640 1 -- 172.21.15.86:0/3155011016 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd55c04b020 con 0x7fd56c101110 2024-04-21T08:38:48.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.241+0000 7fd571611640 1 -- 172.21.15.86:0/3155011016 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fd56c1a2cd0 con 0x7fd56c101110 2024-04-21T08:38:48.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.241+0000 7fd553fff640 1 -- 172.21.15.86:0/3155011016 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7fd55c0bea10 con 0x7fd56c101110 2024-04-21T08:38:48.240 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-21T08:38:48.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.243+0000 7fd571611640 1 -- 172.21.15.86:0/3155011016 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd5540a39b0 msgr2=0x7fd5540a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.243+0000 7fd571611640 1 --2- 172.21.15.86:0/3155011016 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd5540a39b0 0x7fd5540a5e70 secure :-1 s=READY pgs=36 cs=0 l=1 rev1=1 crypto rx=0x7fd56c104e70 tx=0x7fd560009260 comp rx=0 tx=0).stop 2024-04-21T08:38:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.243+0000 7fd571611640 1 -- 172.21.15.86:0/3155011016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd56c101110 msgr2=0x7fd56c105100 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.243+0000 7fd571611640 1 --2- 172.21.15.86:0/3155011016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd56c101110 0x7fd56c105100 secure :-1 s=READY pgs=519 cs=0 l=1 rev1=1 crypto rx=0x7fd55c0040c0 tx=0x7fd55c0040f0 comp rx=0 tx=0).stop 2024-04-21T08:38:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.243+0000 7fd571611640 1 -- 172.21.15.86:0/3155011016 shutdown_connections 2024-04-21T08:38:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.243+0000 7fd571611640 1 --2- 172.21.15.86:0/3155011016 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd5540a39b0 0x7fd5540a5e70 unknown :-1 s=CLOSED pgs=36 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.243+0000 7fd571611640 1 --2- 172.21.15.86:0/3155011016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd56c102470 0x7fd56c103c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.243+0000 7fd571611640 1 --2- 172.21.15.86:0/3155011016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd56c101ad0 0x7fd56c103740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.243+0000 7fd571611640 1 --2- 172.21.15.86:0/3155011016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd56c101110 0x7fd56c105100 unknown :-1 s=CLOSED pgs=519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.243+0000 7fd571611640 1 -- 172.21.15.86:0/3155011016 >> 172.21.15.86:0/3155011016 conn(0x7fd56c0fbb80 msgr2=0x7fd56c100000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.243+0000 7fd571611640 1 -- 172.21.15.86:0/3155011016 shutdown_connections 2024-04-21T08:38:48.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.243+0000 7fd571611640 1 -- 172.21.15.86:0/3155011016 wait complete. 2024-04-21T08:38:48.253 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:38:48.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.356+0000 7f81b71b2640 1 -- 172.21.15.86:0/1396825553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81b0102a00 msgr2=0x7f81b0109320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:48.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.356+0000 7f81b71b2640 1 --2- 172.21.15.86:0/1396825553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81b0102a00 0x7f81b0109320 secure :-1 s=READY pgs=520 cs=0 l=1 rev1=1 crypto rx=0x7f81a8009ca0 tx=0x7f81a802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:48.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.357+0000 7f81b71b2640 1 -- 172.21.15.86:0/1396825553 shutdown_connections 2024-04-21T08:38:48.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.357+0000 7f81b71b2640 1 --2- 172.21.15.86:0/1396825553 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81b0102a00 0x7f81b0109320 unknown :-1 s=CLOSED pgs=520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:48.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.357+0000 7f81b71b2640 1 --2- 172.21.15.86:0/1396825553 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81b0102040 0x7f81b01024c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:48.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.357+0000 7f81b71b2640 1 --2- 172.21.15.86:0/1396825553 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81b0100e60 0x7f81b0101280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:48.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.357+0000 7f81b71b2640 1 -- 172.21.15.86:0/1396825553 >> 172.21.15.86:0/1396825553 conn(0x7f81b00fc440 msgr2=0x7f81b00fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:48.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.357+0000 7f81b71b2640 1 -- 172.21.15.86:0/1396825553 shutdown_connections 2024-04-21T08:38:48.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.357+0000 7f81b71b2640 1 -- 172.21.15.86:0/1396825553 wait complete. 2024-04-21T08:38:48.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.357+0000 7f81b71b2640 1 Processor -- start 2024-04-21T08:38:48.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.357+0000 7f81b71b2640 1 -- start start 2024-04-21T08:38:48.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.357+0000 7f81b71b2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81b0100e60 0x7f81b01982b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:48.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.357+0000 7f81b71b2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81b0102040 0x7f81b01987f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:48.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.357+0000 7f81b71b2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81b0102a00 0x7f81b019f7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:48.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.357+0000 7f81b71b2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f81b010ca00 con 0x7f81b0102a00 2024-04-21T08:38:48.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.357+0000 7f81b71b2640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f81b010c880 con 0x7f81b0102040 2024-04-21T08:38:48.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.357+0000 7f81b71b2640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f81b010cb80 con 0x7f81b0100e60 2024-04-21T08:38:48.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.357+0000 7f81b574f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81b0102a00 0x7f81b019f7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:48.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.357+0000 7f81b574f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81b0102a00 0x7f81b019f7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57596/0 (socket says 172.21.15.86:57596) 2024-04-21T08:38:48.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.357+0000 7f81b574f640 1 -- 172.21.15.86:0/1054819338 learned_addr learned my addr 172.21.15.86:0/1054819338 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:48.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.357+0000 7f81b4f4e640 1 --2- 172.21.15.86:0/1054819338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81b0100e60 0x7f81b01982b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:48.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.358+0000 7f81affff640 1 --2- 172.21.15.86:0/1054819338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81b0102040 0x7f81b01987f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:48.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.358+0000 7f81b574f640 1 -- 172.21.15.86:0/1054819338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81b0100e60 msgr2=0x7f81b01982b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:48.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.358+0000 7f81b574f640 1 --2- 172.21.15.86:0/1054819338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81b0100e60 0x7f81b01982b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:48.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.358+0000 7f81b574f640 1 -- 172.21.15.86:0/1054819338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81b0102040 msgr2=0x7f81b01987f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:48.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.358+0000 7f81b574f640 1 --2- 172.21.15.86:0/1054819338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81b0102040 0x7f81b01987f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:48.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.358+0000 7f81b574f640 1 -- 172.21.15.86:0/1054819338 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f81b019fe30 con 0x7f81b0102a00 2024-04-21T08:38:48.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.358+0000 7f81b574f640 1 --2- 172.21.15.86:0/1054819338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81b0102a00 0x7f81b019f7c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f81a8005e30 tx=0x7f81a8034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:48.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.358+0000 7f81adffb640 1 -- 172.21.15.86:0/1054819338 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a8041020 con 0x7f81b0102a00 2024-04-21T08:38:48.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.358+0000 7f81adffb640 1 -- 172.21.15.86:0/1054819338 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f81a8034ca0 con 0x7f81b0102a00 2024-04-21T08:38:48.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.358+0000 7f81b71b2640 1 -- 172.21.15.86:0/1054819338 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f81b01a00c0 con 0x7f81b0102a00 2024-04-21T08:38:48.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.358+0000 7f81adffb640 1 -- 172.21.15.86:0/1054819338 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f81a8039040 con 0x7f81b0102a00 2024-04-21T08:38:48.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.358+0000 7f81b71b2640 1 -- 172.21.15.86:0/1054819338 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f81b01a05b0 con 0x7f81b0102a00 2024-04-21T08:38:48.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.359+0000 7f81b71b2640 1 -- 172.21.15.86:0/1054819338 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f81b01a0810 con 0x7f81b0102a00 2024-04-21T08:38:48.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.361+0000 7f81adffb640 1 -- 172.21.15.86:0/1054819338 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f81a8044020 con 0x7f81b0102a00 2024-04-21T08:38:48.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.361+0000 7f81adffb640 1 --2- 172.21.15.86:0/1054819338 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f81a00a39b0 0x7f81a00a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:48.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.361+0000 7f81b4f4e640 1 --2- 172.21.15.86:0/1054819338 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f81a00a39b0 0x7f81a00a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:48.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.362+0000 7f81adffb640 1 -- 172.21.15.86:0/1054819338 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f81a80f2160 con 0x7f81b0102a00 2024-04-21T08:38:48.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.362+0000 7f81b4f4e640 1 --2- 172.21.15.86:0/1054819338 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f81a00a39b0 0x7f81a00a5e70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f81a40095f0 tx=0x7f81a4005f90 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:48.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.366+0000 7f81adffb640 1 -- 172.21.15.86:0/1054819338 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f81a804b020 con 0x7f81b0102a00 2024-04-21T08:38:48.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.562+0000 7f81b71b2640 1 -- 172.21.15.86:0/1054819338 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f81b01a0ba0 con 0x7f81b0102a00 2024-04-21T08:38:48.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.562+0000 7f81adffb640 1 -- 172.21.15.86:0/1054819338 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f81a80be9a0 con 0x7f81b0102a00 2024-04-21T08:38:48.562 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-21T08:38:48.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.565+0000 7f81b71b2640 1 -- 172.21.15.86:0/1054819338 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f81a00a39b0 msgr2=0x7f81a00a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:48.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.565+0000 7f81b71b2640 1 --2- 172.21.15.86:0/1054819338 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f81a00a39b0 0x7f81a00a5e70 secure :-1 s=READY pgs=37 cs=0 l=1 rev1=1 crypto rx=0x7f81a40095f0 tx=0x7f81a4005f90 comp rx=0 tx=0).stop 2024-04-21T08:38:48.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.565+0000 7f81b71b2640 1 -- 172.21.15.86:0/1054819338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81b0102a00 msgr2=0x7f81b019f7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:48.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.565+0000 7f81b71b2640 1 --2- 172.21.15.86:0/1054819338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81b0102a00 0x7f81b019f7c0 secure :-1 s=READY pgs=521 cs=0 l=1 rev1=1 crypto rx=0x7f81a8005e30 tx=0x7f81a8034670 comp rx=0 tx=0).stop 2024-04-21T08:38:48.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.565+0000 7f81b71b2640 1 -- 172.21.15.86:0/1054819338 shutdown_connections 2024-04-21T08:38:48.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.565+0000 7f81b71b2640 1 --2- 172.21.15.86:0/1054819338 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f81a00a39b0 0x7f81a00a5e70 unknown :-1 s=CLOSED pgs=37 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:48.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.565+0000 7f81b71b2640 1 --2- 172.21.15.86:0/1054819338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f81b0102a00 0x7f81b019f7c0 unknown :-1 s=CLOSED pgs=521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:48.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.565+0000 7f81b71b2640 1 --2- 172.21.15.86:0/1054819338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f81b0102040 0x7f81b01987f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:48.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.566+0000 7f81b71b2640 1 --2- 172.21.15.86:0/1054819338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f81b0100e60 0x7f81b01982b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:48.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.566+0000 7f81b71b2640 1 -- 172.21.15.86:0/1054819338 >> 172.21.15.86:0/1054819338 conn(0x7f81b00fc440 msgr2=0x7f81b00fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:48.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.566+0000 7f81b71b2640 1 -- 172.21.15.86:0/1054819338 shutdown_connections 2024-04-21T08:38:48.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.566+0000 7f81b71b2640 1 -- 172.21.15.86:0/1054819338 wait complete. 2024-04-21T08:38:48.577 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:48 smithi086 ceph-mon[27939]: [21/Apr/2024:08:38:47] ENGINE Bus STARTING 2024-04-21T08:38:48.578 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:48 smithi086 ceph-mon[27939]: [21/Apr/2024:08:38:47] ENGINE Serving on http://172.21.15.86:8765 2024-04-21T08:38:48.578 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:48 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2971531704' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:48.578 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:48 smithi086 ceph-mon[27939]: [21/Apr/2024:08:38:47] ENGINE Serving on https://172.21.15.86:7150 2024-04-21T08:38:48.578 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:48 smithi086 ceph-mon[27939]: [21/Apr/2024:08:38:47] ENGINE Bus STARTED 2024-04-21T08:38:48.578 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:48 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2605923580' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:48.578 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:48 smithi086 ceph-mon[27939]: mgrmap e22: x(active, since 1.01804s), standbys: z, y 2024-04-21T08:38:48.578 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:48 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1096425833' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:48.578 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:48 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3155011016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:48.579 INFO:tasks.fwd_scrub:Wait for all MDSs to reach steady state... 2024-04-21T08:38:48.579 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:38:48.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.687+0000 7f79907ce640 1 -- 172.21.15.86:0/4077693124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7988107c50 msgr2=0x7f798810a0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:48.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.687+0000 7f79907ce640 1 --2- 172.21.15.86:0/4077693124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7988107c50 0x7f798810a0a0 secure :-1 s=READY pgs=522 cs=0 l=1 rev1=1 crypto rx=0x7f7980009ca0 tx=0x7f798002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:48.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.687+0000 7f79907ce640 1 -- 172.21.15.86:0/4077693124 shutdown_connections 2024-04-21T08:38:48.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.687+0000 7f79907ce640 1 --2- 172.21.15.86:0/4077693124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7988107c50 0x7f798810a0a0 unknown :-1 s=CLOSED pgs=522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:48.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.687+0000 7f79907ce640 1 --2- 172.21.15.86:0/4077693124 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7988105320 0x7f7988107710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:48.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.687+0000 7f79907ce640 1 --2- 172.21.15.86:0/4077693124 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79881029f0 0x7f7988104de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:48.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.687+0000 7f79907ce640 1 -- 172.21.15.86:0/4077693124 >> 172.21.15.86:0/4077693124 conn(0x7f79880fc3e0 msgr2=0x7f79880fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:48.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.687+0000 7f79907ce640 1 -- 172.21.15.86:0/4077693124 shutdown_connections 2024-04-21T08:38:48.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.688+0000 7f79907ce640 1 -- 172.21.15.86:0/4077693124 wait complete. 2024-04-21T08:38:48.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.688+0000 7f79907ce640 1 Processor -- start 2024-04-21T08:38:48.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.688+0000 7f79907ce640 1 -- start start 2024-04-21T08:38:48.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.688+0000 7f79907ce640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79881029f0 0x7f7988198150 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:48.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.688+0000 7f79907ce640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7988105320 0x7f7988198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:48.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.688+0000 7f79907ce640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7988107c50 0x7f798819f700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:48.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.688+0000 7f79907ce640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f798810dab0 con 0x7f7988105320 2024-04-21T08:38:48.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.688+0000 7f79907ce640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f798810d930 con 0x7f7988107c50 2024-04-21T08:38:48.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.688+0000 7f79907ce640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f798810dc30 con 0x7f79881029f0 2024-04-21T08:38:48.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.688+0000 7f798dd69640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7988105320 0x7f7988198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:48.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.688+0000 7f798dd69640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7988105320 0x7f7988198690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57608/0 (socket says 172.21.15.86:57608) 2024-04-21T08:38:48.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.688+0000 7f798dd69640 1 -- 172.21.15.86:0/3233413766 learned_addr learned my addr 172.21.15.86:0/3233413766 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:48.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.689+0000 7f798e56a640 1 --2- 172.21.15.86:0/3233413766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79881029f0 0x7f7988198150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:48.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.689+0000 7f798ed6b640 1 --2- 172.21.15.86:0/3233413766 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7988107c50 0x7f798819f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:48.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.689+0000 7f798dd69640 1 -- 172.21.15.86:0/3233413766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79881029f0 msgr2=0x7f7988198150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:48.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.689+0000 7f798dd69640 1 --2- 172.21.15.86:0/3233413766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79881029f0 0x7f7988198150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:48.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.689+0000 7f798dd69640 1 -- 172.21.15.86:0/3233413766 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7988107c50 msgr2=0x7f798819f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:48.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.689+0000 7f798dd69640 1 --2- 172.21.15.86:0/3233413766 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7988107c50 0x7f798819f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:48.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.689+0000 7f798dd69640 1 -- 172.21.15.86:0/3233413766 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f798819fd70 con 0x7f7988105320 2024-04-21T08:38:48.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.689+0000 7f798dd69640 1 --2- 172.21.15.86:0/3233413766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7988105320 0x7f7988198690 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f797800e9f0 tx=0x7f797800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:48.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.689+0000 7f797f7fe640 1 -- 172.21.15.86:0/3233413766 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797800ccc0 con 0x7f7988105320 2024-04-21T08:38:48.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.689+0000 7f79907ce640 1 -- 172.21.15.86:0/3233413766 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79881a0060 con 0x7f7988105320 2024-04-21T08:38:48.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.689+0000 7f797f7fe640 1 -- 172.21.15.86:0/3233413766 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79780054d0 con 0x7f7988105320 2024-04-21T08:38:48.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.689+0000 7f797f7fe640 1 -- 172.21.15.86:0/3233413766 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7978010430 con 0x7f7988105320 2024-04-21T08:38:48.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.689+0000 7f79907ce640 1 -- 172.21.15.86:0/3233413766 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79881a05b0 con 0x7f7988105320 2024-04-21T08:38:48.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.690+0000 7f79907ce640 1 -- 172.21.15.86:0/3233413766 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f79880fded0 con 0x7f7988105320 2024-04-21T08:38:48.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.691+0000 7f797f7fe640 1 -- 172.21.15.86:0/3233413766 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f79780046c0 con 0x7f7988105320 2024-04-21T08:38:48.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.692+0000 7f797f7fe640 1 --2- 172.21.15.86:0/3233413766 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f79740a39b0 0x7f79740a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:48.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.692+0000 7f798e56a640 1 --2- 172.21.15.86:0/3233413766 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f79740a39b0 0x7f79740a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:48.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.692+0000 7f797f7fe640 1 -- 172.21.15.86:0/3233413766 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f79780d07d0 con 0x7f7988105320 2024-04-21T08:38:48.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.692+0000 7f798e56a640 1 --2- 172.21.15.86:0/3233413766 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f79740a39b0 0x7f79740a5e70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7984009bd0 tx=0x7f7984005f90 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:48.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.700+0000 7f797f7fe640 1 -- 172.21.15.86:0/3233413766 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f797809d010 con 0x7f7988105320 2024-04-21T08:38:48.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:48 smithi088 ceph-mon[31259]: [21/Apr/2024:08:38:47] ENGINE Bus STARTING 2024-04-21T08:38:48.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:48 smithi088 ceph-mon[31259]: [21/Apr/2024:08:38:47] ENGINE Serving on http://172.21.15.86:8765 2024-04-21T08:38:48.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:48 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2971531704' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:48.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:48 smithi088 ceph-mon[31259]: [21/Apr/2024:08:38:47] ENGINE Serving on https://172.21.15.86:7150 2024-04-21T08:38:48.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:48 smithi088 ceph-mon[31259]: [21/Apr/2024:08:38:47] ENGINE Bus STARTED 2024-04-21T08:38:48.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:48 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2605923580' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:48.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:48 smithi088 ceph-mon[31259]: mgrmap e22: x(active, since 1.01804s), standbys: z, y 2024-04-21T08:38:48.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:48 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1096425833' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:48.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:48 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3155011016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:48.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:48 smithi179 ceph-mon[31829]: [21/Apr/2024:08:38:47] ENGINE Bus STARTING 2024-04-21T08:38:48.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:48 smithi179 ceph-mon[31829]: [21/Apr/2024:08:38:47] ENGINE Serving on http://172.21.15.86:8765 2024-04-21T08:38:48.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:48 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2971531704' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:48.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:48 smithi179 ceph-mon[31829]: [21/Apr/2024:08:38:47] ENGINE Serving on https://172.21.15.86:7150 2024-04-21T08:38:48.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:48 smithi179 ceph-mon[31829]: [21/Apr/2024:08:38:47] ENGINE Bus STARTED 2024-04-21T08:38:48.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:48 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2605923580' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:48.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:48 smithi179 ceph-mon[31829]: mgrmap e22: x(active, since 1.01804s), standbys: z, y 2024-04-21T08:38:48.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:48 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1096425833' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:48.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:48 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3155011016' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:48.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.954+0000 7f79907ce640 1 -- 172.21.15.86:0/3233413766 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7988062b50 con 0x7f7988105320 2024-04-21T08:38:48.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.955+0000 7f797f7fe640 1 -- 172.21.15.86:0/3233413766 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7f797809c910 con 0x7f7988105320 2024-04-21T08:38:48.954 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:38:48.955 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:38:45.202126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24398,24394,24455,24472]},"id":1}]} 2024-04-21T08:38:48.955 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-21T08:38:48.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.957+0000 7f79907ce640 1 -- 172.21.15.86:0/3233413766 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f79740a39b0 msgr2=0x7f79740a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:48.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.957+0000 7f79907ce640 1 --2- 172.21.15.86:0/3233413766 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f79740a39b0 0x7f79740a5e70 secure :-1 s=READY pgs=38 cs=0 l=1 rev1=1 crypto rx=0x7f7984009bd0 tx=0x7f7984005f90 comp rx=0 tx=0).stop 2024-04-21T08:38:48.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.957+0000 7f79907ce640 1 -- 172.21.15.86:0/3233413766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7988105320 msgr2=0x7f7988198690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:48.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.957+0000 7f79907ce640 1 --2- 172.21.15.86:0/3233413766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7988105320 0x7f7988198690 secure :-1 s=READY pgs=523 cs=0 l=1 rev1=1 crypto rx=0x7f797800e9f0 tx=0x7f797800eeb0 comp rx=0 tx=0).stop 2024-04-21T08:38:48.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.957+0000 7f79907ce640 1 -- 172.21.15.86:0/3233413766 shutdown_connections 2024-04-21T08:38:48.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.957+0000 7f79907ce640 1 --2- 172.21.15.86:0/3233413766 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f79740a39b0 0x7f79740a5e70 unknown :-1 s=CLOSED pgs=38 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:48.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.957+0000 7f79907ce640 1 --2- 172.21.15.86:0/3233413766 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7988107c50 0x7f798819f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:48.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.957+0000 7f79907ce640 1 --2- 172.21.15.86:0/3233413766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7988105320 0x7f7988198690 unknown :-1 s=CLOSED pgs=523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:48.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.957+0000 7f79907ce640 1 --2- 172.21.15.86:0/3233413766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79881029f0 0x7f7988198150 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:48.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.957+0000 7f79907ce640 1 -- 172.21.15.86:0/3233413766 >> 172.21.15.86:0/3233413766 conn(0x7f79880fc3e0 msgr2=0x7f79881034d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:48.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.957+0000 7f79907ce640 1 -- 172.21.15.86:0/3233413766 shutdown_connections 2024-04-21T08:38:48.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:48.957+0000 7f79907ce640 1 -- 172.21.15.86:0/3233413766 wait complete. 2024-04-21T08:38:48.968 INFO:tasks.fwd_scrub:Ready to start scrub thrashing 2024-04-21T08:38:48.968 INFO:tasks.fwd_scrub.ceph_manager:waiting for clean 2024-04-21T08:38:48.969 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-21T08:38:49.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.070+0000 7f32d46f2640 1 -- 172.21.15.86:0/439912312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32cc102980 msgr2=0x7f32cc1092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:49.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.070+0000 7f32d46f2640 1 --2- 172.21.15.86:0/439912312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32cc102980 0x7f32cc1092a0 secure :-1 s=READY pgs=525 cs=0 l=1 rev1=1 crypto rx=0x7f32bc009ca0 tx=0x7f32bc02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:49.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.070+0000 7f32d46f2640 1 -- 172.21.15.86:0/439912312 shutdown_connections 2024-04-21T08:38:49.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.070+0000 7f32d46f2640 1 --2- 172.21.15.86:0/439912312 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32cc102980 0x7f32cc1092a0 unknown :-1 s=CLOSED pgs=525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:49.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.070+0000 7f32d46f2640 1 --2- 172.21.15.86:0/439912312 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32cc101fc0 0x7f32cc102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:49.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.070+0000 7f32d46f2640 1 --2- 172.21.15.86:0/439912312 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32cc100e70 0x7f32cc101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:49.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.070+0000 7f32d46f2640 1 -- 172.21.15.86:0/439912312 >> 172.21.15.86:0/439912312 conn(0x7f32cc0fc3e0 msgr2=0x7f32cc0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:49.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.070+0000 7f32d46f2640 1 -- 172.21.15.86:0/439912312 shutdown_connections 2024-04-21T08:38:49.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.070+0000 7f32d46f2640 1 -- 172.21.15.86:0/439912312 wait complete. 2024-04-21T08:38:49.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.070+0000 7f32d46f2640 1 Processor -- start 2024-04-21T08:38:49.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.070+0000 7f32d46f2640 1 -- start start 2024-04-21T08:38:49.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.070+0000 7f32d46f2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32cc100e70 0x7f32cc104df0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:49.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.071+0000 7f32d46f2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32cc101fc0 0x7f32cc103430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:49.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.071+0000 7f32d46f2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32cc102980 0x7f32cc103970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:49.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.071+0000 7f32d46f2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc10c9a0 con 0x7f32cc102980 2024-04-21T08:38:49.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.071+0000 7f32d46f2640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc10c820 con 0x7f32cc101fc0 2024-04-21T08:38:49.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.071+0000 7f32d46f2640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f32cc10cb20 con 0x7f32cc100e70 2024-04-21T08:38:49.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.071+0000 7f32d2c8f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32cc102980 0x7f32cc103970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:49.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.071+0000 7f32d1c8d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32cc101fc0 0x7f32cc103430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:49.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.071+0000 7f32d2c8f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32cc102980 0x7f32cc103970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57632/0 (socket says 172.21.15.86:57632) 2024-04-21T08:38:49.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.071+0000 7f32d2c8f640 1 -- 172.21.15.86:0/3726123234 learned_addr learned my addr 172.21.15.86:0/3726123234 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:49.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.071+0000 7f32d248e640 1 --2- 172.21.15.86:0/3726123234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32cc100e70 0x7f32cc104df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:49.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.071+0000 7f32d2c8f640 1 -- 172.21.15.86:0/3726123234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32cc100e70 msgr2=0x7f32cc104df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:49.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.071+0000 7f32d2c8f640 1 --2- 172.21.15.86:0/3726123234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32cc100e70 0x7f32cc104df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:49.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.071+0000 7f32d2c8f640 1 -- 172.21.15.86:0/3726123234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32cc101fc0 msgr2=0x7f32cc103430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:49.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.071+0000 7f32d2c8f640 1 --2- 172.21.15.86:0/3726123234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32cc101fc0 0x7f32cc103430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:49.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.071+0000 7f32d2c8f640 1 -- 172.21.15.86:0/3726123234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f32cc104200 con 0x7f32cc102980 2024-04-21T08:38:49.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.071+0000 7f32d2c8f640 1 --2- 172.21.15.86:0/3726123234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32cc102980 0x7f32cc103970 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f32bc02c700 tx=0x7f32bc0346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:49.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.071+0000 7f32cb7fe640 1 -- 172.21.15.86:0/3726123234 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc041020 con 0x7f32cc102980 2024-04-21T08:38:49.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.071+0000 7f32d46f2640 1 -- 172.21.15.86:0/3726123234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f32cc1a21d0 con 0x7f32cc102980 2024-04-21T08:38:49.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.071+0000 7f32cb7fe640 1 -- 172.21.15.86:0/3726123234 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f32bc034ca0 con 0x7f32cc102980 2024-04-21T08:38:49.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.071+0000 7f32cb7fe640 1 -- 172.21.15.86:0/3726123234 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f32bc039040 con 0x7f32cc102980 2024-04-21T08:38:49.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.071+0000 7f32d46f2640 1 -- 172.21.15.86:0/3726123234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f32cc1a2720 con 0x7f32cc102980 2024-04-21T08:38:49.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.072+0000 7f32d46f2640 1 -- 172.21.15.86:0/3726123234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f32cc1a2980 con 0x7f32cc102980 2024-04-21T08:38:49.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.074+0000 7f32cb7fe640 1 -- 172.21.15.86:0/3726123234 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 22) ==== 138458+0+0 (secure 0 0 0) 0x7f32bc044020 con 0x7f32cc102980 2024-04-21T08:38:49.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.074+0000 7f32cb7fe640 1 --2- 172.21.15.86:0/3726123234 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f32b40a39b0 0x7f32b40a5e70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:49.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.074+0000 7f32d248e640 1 --2- 172.21.15.86:0/3726123234 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f32b40a39b0 0x7f32b40a5e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:49.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.074+0000 7f32cb7fe640 1 -- 172.21.15.86:0/3726123234 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f32bc0f2610 con 0x7f32cc102980 2024-04-21T08:38:49.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.075+0000 7f32d248e640 1 --2- 172.21.15.86:0/3726123234 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f32b40a39b0 0x7f32b40a5e70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f32c4009bd0 tx=0x7f32c4005f90 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:49.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.082+0000 7f32cb7fe640 1 -- 172.21.15.86:0/3726123234 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f32bc04b020 con 0x7f32cc102980 2024-04-21T08:38:49.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.264+0000 7f32d46f2640 1 -- 172.21.15.86:0/3726123234 --> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f32cc0fe5a0 con 0x7f32b40a39b0 2024-04-21T08:38:49.271 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:49 smithi086 ceph-mon[27939]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:38:49.271 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:49 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1054819338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:49.271 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:49 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:49.271 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:49 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:49.271 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:49 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:49.272 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:49 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:49.272 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:49 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:49.272 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:49 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:49.272 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:49 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3233413766' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:49.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.288+0000 7f32cb7fe640 1 -- 172.21.15.86:0/3726123234 <== mgr.15558 v2:172.21.15.86:6800/2452032403 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339464 (secure 0 0 0) 0x7f32cc0fe5a0 con 0x7f32b40a39b0 2024-04-21T08:38:49.291 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:38:49.293 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-21T08:38:49.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.290+0000 7f32cb7fe640 1 -- 172.21.15.86:0/3726123234 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7f32bc0baee0 con 0x7f32cc102980 2024-04-21T08:38:49.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.293+0000 7f32d46f2640 1 -- 172.21.15.86:0/3726123234 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f32b40a39b0 msgr2=0x7f32b40a5e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:49.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.293+0000 7f32d46f2640 1 --2- 172.21.15.86:0/3726123234 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f32b40a39b0 0x7f32b40a5e70 secure :-1 s=READY pgs=40 cs=0 l=1 rev1=1 crypto rx=0x7f32c4009bd0 tx=0x7f32c4005f90 comp rx=0 tx=0).stop 2024-04-21T08:38:49.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.293+0000 7f32d46f2640 1 -- 172.21.15.86:0/3726123234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32cc102980 msgr2=0x7f32cc103970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:49.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.293+0000 7f32d46f2640 1 --2- 172.21.15.86:0/3726123234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32cc102980 0x7f32cc103970 secure :-1 s=READY pgs=526 cs=0 l=1 rev1=1 crypto rx=0x7f32bc02c700 tx=0x7f32bc0346d0 comp rx=0 tx=0).stop 2024-04-21T08:38:49.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.293+0000 7f32d46f2640 1 -- 172.21.15.86:0/3726123234 shutdown_connections 2024-04-21T08:38:49.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.293+0000 7f32d46f2640 1 --2- 172.21.15.86:0/3726123234 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f32b40a39b0 0x7f32b40a5e70 unknown :-1 s=CLOSED pgs=40 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:49.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.293+0000 7f32d46f2640 1 --2- 172.21.15.86:0/3726123234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f32cc102980 0x7f32cc103970 unknown :-1 s=CLOSED pgs=526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:49.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.293+0000 7f32d46f2640 1 --2- 172.21.15.86:0/3726123234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f32cc101fc0 0x7f32cc103430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:49.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.293+0000 7f32d46f2640 1 --2- 172.21.15.86:0/3726123234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f32cc100e70 0x7f32cc104df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:49.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.293+0000 7f32d46f2640 1 -- 172.21.15.86:0/3726123234 >> 172.21.15.86:0/3726123234 conn(0x7f32cc0fc3e0 msgr2=0x7f32cc0fde90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:49.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.293+0000 7f32d46f2640 1 -- 172.21.15.86:0/3726123234 shutdown_connections 2024-04-21T08:38:49.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.293+0000 7f32d46f2640 1 -- 172.21.15.86:0/3726123234 wait complete. 2024-04-21T08:38:49.303 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-04-21T08:38:48.652229+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1194594,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1486,"num_read_kb":910,"num_write":558,"num_write_kb":4413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":1181659,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":944,"ondisk_log_size":944,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":386032,"kb_used_data":15860,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124555792,"statfs":{"total":1151940427776,"available":1151545131008,"internally_reserved":0,"allocated":16240640,"data_stored":9386888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":378713404},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.992077"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807854+0000","last_change":"2024-04-21T08:36:20.913421+0000","last_active":"2024-04-21T08:38:46.807854+0000","last_peered":"2024-04-21T08:38:46.807854+0000","last_clean":"2024-04-21T08:38:46.807854+0000","last_became_active":"2024-04-21T08:36:20.913017+0000","last_became_peered":"2024-04-21T08:36:20.913017+0000","last_unstale":"2024-04-21T08:38:46.807854+0000","last_undegraded":"2024-04-21T08:38:46.807854+0000","last_fullsized":"2024-04-21T08:38:46.807854+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:16:29.885916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.915359+0000","last_change":"2024-04-21T08:36:20.911675+0000","last_active":"2024-04-21T08:38:46.915359+0000","last_peered":"2024-04-21T08:38:46.915359+0000","last_clean":"2024-04-21T08:38:46.915359+0000","last_became_active":"2024-04-21T08:36:20.910591+0000","last_became_peered":"2024-04-21T08:36:20.910591+0000","last_unstale":"2024-04-21T08:38:46.915359+0000","last_undegraded":"2024-04-21T08:38:46.915359+0000","last_fullsized":"2024-04-21T08:38:46.915359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:01:40.735376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.915070+0000","last_change":"2024-04-21T08:36:20.911978+0000","last_active":"2024-04-21T08:38:46.915070+0000","last_peered":"2024-04-21T08:38:46.915070+0000","last_clean":"2024-04-21T08:38:46.915070+0000","last_became_active":"2024-04-21T08:36:20.911641+0000","last_became_peered":"2024-04-21T08:36:20.911641+0000","last_unstale":"2024-04-21T08:38:46.915070+0000","last_undegraded":"2024-04-21T08:38:46.915070+0000","last_fullsized":"2024-04-21T08:38:46.915070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:14:23.789790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911901+0000","last_change":"2024-04-21T08:36:20.919027+0000","last_active":"2024-04-21T08:38:46.911901+0000","last_peered":"2024-04-21T08:38:46.911901+0000","last_clean":"2024-04-21T08:38:46.911901+0000","last_became_active":"2024-04-21T08:36:20.918740+0000","last_became_peered":"2024-04-21T08:36:20.918740+0000","last_unstale":"2024-04-21T08:38:46.911901+0000","last_undegraded":"2024-04-21T08:38:46.911901+0000","last_fullsized":"2024-04-21T08:38:46.911901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:00:31.066938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655044+0000","last_change":"2024-04-21T08:36:20.907175+0000","last_active":"2024-04-21T08:38:46.655044+0000","last_peered":"2024-04-21T08:38:46.655044+0000","last_clean":"2024-04-21T08:38:46.655044+0000","last_became_active":"2024-04-21T08:36:20.906818+0000","last_became_peered":"2024-04-21T08:36:20.906818+0000","last_unstale":"2024-04-21T08:38:46.655044+0000","last_undegraded":"2024-04-21T08:38:46.655044+0000","last_fullsized":"2024-04-21T08:38:46.655044+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:41:03.332231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638216+0000","last_change":"2024-04-21T08:36:20.905968+0000","last_active":"2024-04-21T08:38:46.638216+0000","last_peered":"2024-04-21T08:38:46.638216+0000","last_clean":"2024-04-21T08:38:46.638216+0000","last_became_active":"2024-04-21T08:36:20.904840+0000","last_became_peered":"2024-04-21T08:36:20.904840+0000","last_unstale":"2024-04-21T08:38:46.638216+0000","last_undegraded":"2024-04-21T08:38:46.638216+0000","last_fullsized":"2024-04-21T08:38:46.638216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:23:46.886241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913568+0000","last_change":"2024-04-21T08:36:20.912045+0000","last_active":"2024-04-21T08:38:46.913568+0000","last_peered":"2024-04-21T08:38:46.913568+0000","last_clean":"2024-04-21T08:38:46.913568+0000","last_became_active":"2024-04-21T08:36:20.911690+0000","last_became_peered":"2024-04-21T08:36:20.911690+0000","last_unstale":"2024-04-21T08:38:46.913568+0000","last_undegraded":"2024-04-21T08:38:46.913568+0000","last_fullsized":"2024-04-21T08:38:46.913568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:06:28.229307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638612+0000","last_change":"2024-04-21T08:36:20.904874+0000","last_active":"2024-04-21T08:38:46.638612+0000","last_peered":"2024-04-21T08:38:46.638612+0000","last_clean":"2024-04-21T08:38:46.638612+0000","last_became_active":"2024-04-21T08:36:20.904217+0000","last_became_peered":"2024-04-21T08:36:20.904217+0000","last_unstale":"2024-04-21T08:38:46.638612+0000","last_undegraded":"2024-04-21T08:38:46.638612+0000","last_fullsized":"2024-04-21T08:38:46.638612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:37:04.423812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.923169+0000","last_change":"2024-04-21T08:36:20.912874+0000","last_active":"2024-04-21T08:38:46.923169+0000","last_peered":"2024-04-21T08:38:46.923169+0000","last_clean":"2024-04-21T08:38:46.923169+0000","last_became_active":"2024-04-21T08:36:20.912569+0000","last_became_peered":"2024-04-21T08:36:20.912569+0000","last_unstale":"2024-04-21T08:38:46.923169+0000","last_undegraded":"2024-04-21T08:38:46.923169+0000","last_fullsized":"2024-04-21T08:38:46.923169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:55:20.763157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":52,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638497+0000","last_change":"2024-04-21T08:37:24.296916+0000","last_active":"2024-04-21T08:38:46.638497+0000","last_peered":"2024-04-21T08:38:46.638497+0000","last_clean":"2024-04-21T08:38:46.638497+0000","last_became_active":"2024-04-21T08:37:24.296547+0000","last_became_peered":"2024-04-21T08:37:24.296547+0000","last_unstale":"2024-04-21T08:38:46.638497+0000","last_undegraded":"2024-04-21T08:38:46.638497+0000","last_fullsized":"2024-04-21T08:38:46.638497+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:47:56.683793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914246+0000","last_change":"2024-04-21T08:36:20.909094+0000","last_active":"2024-04-21T08:38:46.914246+0000","last_peered":"2024-04-21T08:38:46.914246+0000","last_clean":"2024-04-21T08:38:46.914246+0000","last_became_active":"2024-04-21T08:36:20.908634+0000","last_became_peered":"2024-04-21T08:36:20.908634+0000","last_unstale":"2024-04-21T08:38:46.914246+0000","last_undegraded":"2024-04-21T08:38:46.914246+0000","last_fullsized":"2024-04-21T08:38:46.914246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:28:38.450621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911748+0000","last_change":"2024-04-21T08:36:20.909429+0000","last_active":"2024-04-21T08:38:46.911748+0000","last_peered":"2024-04-21T08:38:46.911748+0000","last_clean":"2024-04-21T08:38:46.911748+0000","last_became_active":"2024-04-21T08:36:20.909081+0000","last_became_peered":"2024-04-21T08:36:20.909081+0000","last_unstale":"2024-04-21T08:38:46.911748+0000","last_undegraded":"2024-04-21T08:38:46.911748+0000","last_fullsized":"2024-04-21T08:38:46.911748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:45:46.939125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913919+0000","last_change":"2024-04-21T08:36:20.895894+0000","last_active":"2024-04-21T08:38:46.913919+0000","last_peered":"2024-04-21T08:38:46.913919+0000","last_clean":"2024-04-21T08:38:46.913919+0000","last_became_active":"2024-04-21T08:36:20.895572+0000","last_became_peered":"2024-04-21T08:36:20.895572+0000","last_unstale":"2024-04-21T08:38:46.913919+0000","last_undegraded":"2024-04-21T08:38:46.913919+0000","last_fullsized":"2024-04-21T08:38:46.913919+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:12:48.750555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914638+0000","last_change":"2024-04-21T08:36:20.905356+0000","last_active":"2024-04-21T08:38:46.914638+0000","last_peered":"2024-04-21T08:38:46.914638+0000","last_clean":"2024-04-21T08:38:46.914638+0000","last_became_active":"2024-04-21T08:36:20.904954+0000","last_became_peered":"2024-04-21T08:36:20.904954+0000","last_unstale":"2024-04-21T08:38:46.914638+0000","last_undegraded":"2024-04-21T08:38:46.914638+0000","last_fullsized":"2024-04-21T08:38:46.914638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:45:49.116482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.915244+0000","last_change":"2024-04-21T08:36:20.911550+0000","last_active":"2024-04-21T08:38:46.915244+0000","last_peered":"2024-04-21T08:38:46.915244+0000","last_clean":"2024-04-21T08:38:46.915244+0000","last_became_active":"2024-04-21T08:36:20.910557+0000","last_became_peered":"2024-04-21T08:36:20.910557+0000","last_unstale":"2024-04-21T08:38:46.915244+0000","last_undegraded":"2024-04-21T08:38:46.915244+0000","last_fullsized":"2024-04-21T08:38:46.915244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:09:49.422729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.922413+0000","last_change":"2024-04-21T08:36:20.903954+0000","last_active":"2024-04-21T08:38:46.922413+0000","last_peered":"2024-04-21T08:38:46.922413+0000","last_clean":"2024-04-21T08:38:46.922413+0000","last_became_active":"2024-04-21T08:36:20.903641+0000","last_became_peered":"2024-04-21T08:36:20.903641+0000","last_unstale":"2024-04-21T08:38:46.922413+0000","last_undegraded":"2024-04-21T08:38:46.922413+0000","last_fullsized":"2024-04-21T08:38:46.922413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:08:55.678940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807825+0000","last_change":"2024-04-21T08:36:20.914256+0000","last_active":"2024-04-21T08:38:46.807825+0000","last_peered":"2024-04-21T08:38:46.807825+0000","last_clean":"2024-04-21T08:38:46.807825+0000","last_became_active":"2024-04-21T08:36:20.913858+0000","last_became_peered":"2024-04-21T08:36:20.913858+0000","last_unstale":"2024-04-21T08:38:46.807825+0000","last_undegraded":"2024-04-21T08:38:46.807825+0000","last_fullsized":"2024-04-21T08:38:46.807825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:01:10.276763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.743080+0000","last_change":"2024-04-21T08:36:20.904138+0000","last_active":"2024-04-21T08:38:46.743080+0000","last_peered":"2024-04-21T08:38:46.743080+0000","last_clean":"2024-04-21T08:38:46.743080+0000","last_became_active":"2024-04-21T08:36:20.903670+0000","last_became_peered":"2024-04-21T08:36:20.903670+0000","last_unstale":"2024-04-21T08:38:46.743080+0000","last_undegraded":"2024-04-21T08:38:46.743080+0000","last_fullsized":"2024-04-21T08:38:46.743080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:38:50.160867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639071+0000","last_change":"2024-04-21T08:36:20.910798+0000","last_active":"2024-04-21T08:38:46.639071+0000","last_peered":"2024-04-21T08:38:46.639071+0000","last_clean":"2024-04-21T08:38:46.639071+0000","last_became_active":"2024-04-21T08:36:20.910411+0000","last_became_peered":"2024-04-21T08:36:20.910411+0000","last_unstale":"2024-04-21T08:38:46.639071+0000","last_undegraded":"2024-04-21T08:38:46.639071+0000","last_fullsized":"2024-04-21T08:38:46.639071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:13:12.109818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638623+0000","last_change":"2024-04-21T08:36:24.924831+0000","last_active":"2024-04-21T08:38:46.638623+0000","last_peered":"2024-04-21T08:38:46.638623+0000","last_clean":"2024-04-21T08:38:46.638623+0000","last_became_active":"2024-04-21T08:36:24.924435+0000","last_became_peered":"2024-04-21T08:36:24.924435+0000","last_unstale":"2024-04-21T08:38:46.638623+0000","last_undegraded":"2024-04-21T08:38:46.638623+0000","last_fullsized":"2024-04-21T08:38:46.638623+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:23.447079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":188,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.743359+0000","last_change":"2024-04-21T08:37:24.308024+0000","last_active":"2024-04-21T08:38:46.743359+0000","last_peered":"2024-04-21T08:38:46.743359+0000","last_clean":"2024-04-21T08:38:46.743359+0000","last_became_active":"2024-04-21T08:37:24.297491+0000","last_became_peered":"2024-04-21T08:37:24.297491+0000","last_unstale":"2024-04-21T08:38:46.743359+0000","last_undegraded":"2024-04-21T08:38:46.743359+0000","last_fullsized":"2024-04-21T08:38:46.743359+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:48:10.039545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639158+0000","last_change":"2024-04-21T08:36:20.912851+0000","last_active":"2024-04-21T08:38:46.639158+0000","last_peered":"2024-04-21T08:38:46.639158+0000","last_clean":"2024-04-21T08:38:46.639158+0000","last_became_active":"2024-04-21T08:36:20.912407+0000","last_became_peered":"2024-04-21T08:36:20.912407+0000","last_unstale":"2024-04-21T08:38:46.639158+0000","last_undegraded":"2024-04-21T08:38:46.639158+0000","last_fullsized":"2024-04-21T08:38:46.639158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:45:18.026940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.910950+0000","last_change":"2024-04-21T08:36:24.924052+0000","last_active":"2024-04-21T08:38:46.910950+0000","last_peered":"2024-04-21T08:38:46.910950+0000","last_clean":"2024-04-21T08:38:46.910950+0000","last_became_active":"2024-04-21T08:36:24.923618+0000","last_became_peered":"2024-04-21T08:36:24.923618+0000","last_unstale":"2024-04-21T08:38:46.910950+0000","last_undegraded":"2024-04-21T08:38:46.910950+0000","last_fullsized":"2024-04-21T08:38:46.910950+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:08:18.182386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639180+0000","last_change":"2024-04-21T08:36:20.285291+0000","last_active":"2024-04-21T08:38:46.639180+0000","last_peered":"2024-04-21T08:38:46.639180+0000","last_clean":"2024-04-21T08:38:46.639180+0000","last_became_active":"2024-04-21T08:36:20.284969+0000","last_became_peered":"2024-04-21T08:36:20.284969+0000","last_unstale":"2024-04-21T08:38:46.639180+0000","last_undegraded":"2024-04-21T08:38:46.639180+0000","last_fullsized":"2024-04-21T08:38:46.639180+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:55:29.484439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.915781+0000","last_change":"2024-04-21T08:36:20.902819+0000","last_active":"2024-04-21T08:38:46.915781+0000","last_peered":"2024-04-21T08:38:46.915781+0000","last_clean":"2024-04-21T08:38:46.915781+0000","last_became_active":"2024-04-21T08:36:20.902466+0000","last_became_peered":"2024-04-21T08:36:20.902466+0000","last_unstale":"2024-04-21T08:38:46.915781+0000","last_undegraded":"2024-04-21T08:38:46.915781+0000","last_fullsized":"2024-04-21T08:38:46.915781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:29:40.454026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911156+0000","last_change":"2024-04-21T08:36:24.926480+0000","last_active":"2024-04-21T08:38:46.911156+0000","last_peered":"2024-04-21T08:38:46.911156+0000","last_clean":"2024-04-21T08:38:46.911156+0000","last_became_active":"2024-04-21T08:36:24.926116+0000","last_became_peered":"2024-04-21T08:36:24.926116+0000","last_unstale":"2024-04-21T08:38:46.911156+0000","last_undegraded":"2024-04-21T08:38:46.911156+0000","last_fullsized":"2024-04-21T08:38:46.911156+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:43:18.839396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655562+0000","last_change":"2024-04-21T08:36:20.293086+0000","last_active":"2024-04-21T08:38:46.655562+0000","last_peered":"2024-04-21T08:38:46.655562+0000","last_clean":"2024-04-21T08:38:46.655562+0000","last_became_active":"2024-04-21T08:36:20.291178+0000","last_became_peered":"2024-04-21T08:36:20.291178+0000","last_unstale":"2024-04-21T08:38:46.655562+0000","last_undegraded":"2024-04-21T08:38:46.655562+0000","last_fullsized":"2024-04-21T08:38:46.655562+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:02:32.935930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.922727+0000","last_change":"2024-04-21T08:36:20.909374+0000","last_active":"2024-04-21T08:38:46.922727+0000","last_peered":"2024-04-21T08:38:46.922727+0000","last_clean":"2024-04-21T08:38:46.922727+0000","last_became_active":"2024-04-21T08:36:20.909043+0000","last_became_peered":"2024-04-21T08:36:20.909043+0000","last_unstale":"2024-04-21T08:38:46.922727+0000","last_undegraded":"2024-04-21T08:38:46.922727+0000","last_fullsized":"2024-04-21T08:38:46.922727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:45:45.719942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911098+0000","last_change":"2024-04-21T08:38:24.623194+0000","last_active":"2024-04-21T08:38:46.911098+0000","last_peered":"2024-04-21T08:38:46.911098+0000","last_clean":"2024-04-21T08:38:46.911098+0000","last_became_active":"2024-04-21T08:38:24.622632+0000","last_became_peered":"2024-04-21T08:38:24.622632+0000","last_unstale":"2024-04-21T08:38:46.911098+0000","last_undegraded":"2024-04-21T08:38:46.911098+0000","last_fullsized":"2024-04-21T08:38:46.911098+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:26:39.752161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655409+0000","last_change":"2024-04-21T08:36:20.288644+0000","last_active":"2024-04-21T08:38:46.655409+0000","last_peered":"2024-04-21T08:38:46.655409+0000","last_clean":"2024-04-21T08:38:46.655409+0000","last_became_active":"2024-04-21T08:36:20.288069+0000","last_became_peered":"2024-04-21T08:36:20.288069+0000","last_unstale":"2024-04-21T08:38:46.655409+0000","last_undegraded":"2024-04-21T08:38:46.655409+0000","last_fullsized":"2024-04-21T08:38:46.655409+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:37:36.522713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751376+0000","last_change":"2024-04-21T08:36:20.908505+0000","last_active":"2024-04-21T08:38:46.751376+0000","last_peered":"2024-04-21T08:38:46.751376+0000","last_clean":"2024-04-21T08:38:46.751376+0000","last_became_active":"2024-04-21T08:36:20.908163+0000","last_became_peered":"2024-04-21T08:36:20.908163+0000","last_unstale":"2024-04-21T08:38:46.751376+0000","last_undegraded":"2024-04-21T08:38:46.751376+0000","last_fullsized":"2024-04-21T08:38:46.751376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:54:18.530203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639160+0000","last_change":"2024-04-21T08:36:24.919428+0000","last_active":"2024-04-21T08:38:46.639160+0000","last_peered":"2024-04-21T08:38:46.639160+0000","last_clean":"2024-04-21T08:38:46.639160+0000","last_became_active":"2024-04-21T08:36:24.918722+0000","last_became_peered":"2024-04-21T08:36:24.918722+0000","last_unstale":"2024-04-21T08:38:46.639160+0000","last_undegraded":"2024-04-21T08:38:46.639160+0000","last_fullsized":"2024-04-21T08:38:46.639160+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:30:32.369551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"93'8","reported_seq":168,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751405+0000","last_change":"2024-04-21T08:36:20.287642+0000","last_active":"2024-04-21T08:38:46.751405+0000","last_peered":"2024-04-21T08:38:46.751405+0000","last_clean":"2024-04-21T08:38:46.751405+0000","last_became_active":"2024-04-21T08:36:20.287296+0000","last_became_peered":"2024-04-21T08:36:20.287296+0000","last_unstale":"2024-04-21T08:38:46.751405+0000","last_undegraded":"2024-04-21T08:38:46.751405+0000","last_fullsized":"2024-04-21T08:38:46.751405+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:59:28.579020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1462,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655081+0000","last_change":"2024-04-21T08:36:20.916971+0000","last_active":"2024-04-21T08:38:46.655081+0000","last_peered":"2024-04-21T08:38:46.655081+0000","last_clean":"2024-04-21T08:38:46.655081+0000","last_became_active":"2024-04-21T08:36:20.916622+0000","last_became_peered":"2024-04-21T08:36:20.916622+0000","last_unstale":"2024-04-21T08:38:46.655081+0000","last_undegraded":"2024-04-21T08:38:46.655081+0000","last_fullsized":"2024-04-21T08:38:46.655081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:49:38.858772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638856+0000","last_change":"2024-04-21T08:36:24.924170+0000","last_active":"2024-04-21T08:38:46.638856+0000","last_peered":"2024-04-21T08:38:46.638856+0000","last_clean":"2024-04-21T08:38:46.638856+0000","last_became_active":"2024-04-21T08:36:24.923744+0000","last_became_peered":"2024-04-21T08:36:24.923744+0000","last_unstale":"2024-04-21T08:38:46.638856+0000","last_undegraded":"2024-04-21T08:38:46.638856+0000","last_fullsized":"2024-04-21T08:38:46.638856+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:23:24.605021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638388+0000","last_change":"2024-04-21T08:36:20.286545+0000","last_active":"2024-04-21T08:38:46.638388+0000","last_peered":"2024-04-21T08:38:46.638388+0000","last_clean":"2024-04-21T08:38:46.638388+0000","last_became_active":"2024-04-21T08:36:20.286197+0000","last_became_peered":"2024-04-21T08:36:20.286197+0000","last_unstale":"2024-04-21T08:38:46.638388+0000","last_undegraded":"2024-04-21T08:38:46.638388+0000","last_fullsized":"2024-04-21T08:38:46.638388+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:27:55.430428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751488+0000","last_change":"2024-04-21T08:36:20.907743+0000","last_active":"2024-04-21T08:38:46.751488+0000","last_peered":"2024-04-21T08:38:46.751488+0000","last_clean":"2024-04-21T08:38:46.751488+0000","last_became_active":"2024-04-21T08:36:20.907163+0000","last_became_peered":"2024-04-21T08:36:20.907163+0000","last_unstale":"2024-04-21T08:38:46.751488+0000","last_undegraded":"2024-04-21T08:38:46.751488+0000","last_fullsized":"2024-04-21T08:38:46.751488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:41:00.172353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807766+0000","last_change":"2024-04-21T08:37:24.301529+0000","last_active":"2024-04-21T08:38:46.807766+0000","last_peered":"2024-04-21T08:38:46.807766+0000","last_clean":"2024-04-21T08:38:46.807766+0000","last_became_active":"2024-04-21T08:37:24.301014+0000","last_became_peered":"2024-04-21T08:37:24.301014+0000","last_unstale":"2024-04-21T08:38:46.807766+0000","last_undegraded":"2024-04-21T08:38:46.807766+0000","last_fullsized":"2024-04-21T08:38:46.807766+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:58:20.088501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911966+0000","last_change":"2024-04-21T08:36:20.283143+0000","last_active":"2024-04-21T08:38:46.911966+0000","last_peered":"2024-04-21T08:38:46.911966+0000","last_clean":"2024-04-21T08:38:46.911966+0000","last_became_active":"2024-04-21T08:36:20.282585+0000","last_became_peered":"2024-04-21T08:36:20.282585+0000","last_unstale":"2024-04-21T08:38:46.911966+0000","last_undegraded":"2024-04-21T08:38:46.911966+0000","last_fullsized":"2024-04-21T08:38:46.911966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:12:37.588764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638488+0000","last_change":"2024-04-21T08:36:20.912591+0000","last_active":"2024-04-21T08:38:46.638488+0000","last_peered":"2024-04-21T08:38:46.638488+0000","last_clean":"2024-04-21T08:38:46.638488+0000","last_became_active":"2024-04-21T08:36:20.912174+0000","last_became_peered":"2024-04-21T08:36:20.912174+0000","last_unstale":"2024-04-21T08:38:46.638488+0000","last_undegraded":"2024-04-21T08:38:46.638488+0000","last_fullsized":"2024-04-21T08:38:46.638488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:33:58.949506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.915307+0000","last_change":"2024-04-21T08:36:24.915998+0000","last_active":"2024-04-21T08:38:46.915307+0000","last_peered":"2024-04-21T08:38:46.915307+0000","last_clean":"2024-04-21T08:38:46.915307+0000","last_became_active":"2024-04-21T08:36:24.915621+0000","last_became_peered":"2024-04-21T08:36:24.915621+0000","last_unstale":"2024-04-21T08:38:46.915307+0000","last_undegraded":"2024-04-21T08:38:46.915307+0000","last_fullsized":"2024-04-21T08:38:46.915307+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:43:05.117181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751087+0000","last_change":"2024-04-21T08:36:20.286308+0000","last_active":"2024-04-21T08:38:46.751087+0000","last_peered":"2024-04-21T08:38:46.751087+0000","last_clean":"2024-04-21T08:38:46.751087+0000","last_became_active":"2024-04-21T08:36:20.285975+0000","last_became_peered":"2024-04-21T08:36:20.285975+0000","last_unstale":"2024-04-21T08:38:46.751087+0000","last_undegraded":"2024-04-21T08:38:46.751087+0000","last_fullsized":"2024-04-21T08:38:46.751087+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:55:10.658349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.742770+0000","last_change":"2024-04-21T08:36:20.900502+0000","last_active":"2024-04-21T08:38:46.742770+0000","last_peered":"2024-04-21T08:38:46.742770+0000","last_clean":"2024-04-21T08:38:46.742770+0000","last_became_active":"2024-04-21T08:36:20.900199+0000","last_became_peered":"2024-04-21T08:36:20.900199+0000","last_unstale":"2024-04-21T08:38:46.742770+0000","last_undegraded":"2024-04-21T08:38:46.742770+0000","last_fullsized":"2024-04-21T08:38:46.742770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:17:00.720918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638804+0000","last_change":"2024-04-21T08:36:24.920991+0000","last_active":"2024-04-21T08:38:46.638804+0000","last_peered":"2024-04-21T08:38:46.638804+0000","last_clean":"2024-04-21T08:38:46.638804+0000","last_became_active":"2024-04-21T08:36:24.916611+0000","last_became_peered":"2024-04-21T08:36:24.916611+0000","last_unstale":"2024-04-21T08:38:46.638804+0000","last_undegraded":"2024-04-21T08:38:46.638804+0000","last_fullsized":"2024-04-21T08:38:46.638804+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:01:01.412127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"93'7","reported_seq":283,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:47.633155+0000","last_change":"2024-04-21T08:36:20.284966+0000","last_active":"2024-04-21T08:38:47.633155+0000","last_peered":"2024-04-21T08:38:47.633155+0000","last_clean":"2024-04-21T08:38:47.633155+0000","last_became_active":"2024-04-21T08:36:20.284612+0000","last_became_peered":"2024-04-21T08:36:20.284612+0000","last_unstale":"2024-04-21T08:38:47.633155+0000","last_undegraded":"2024-04-21T08:38:47.633155+0000","last_fullsized":"2024-04-21T08:38:47.633155+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:54:50.095013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":243,"num_read_kb":243,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.923175+0000","last_change":"2024-04-21T08:36:20.903493+0000","last_active":"2024-04-21T08:38:46.923175+0000","last_peered":"2024-04-21T08:38:46.923175+0000","last_clean":"2024-04-21T08:38:46.923175+0000","last_became_active":"2024-04-21T08:36:20.902752+0000","last_became_peered":"2024-04-21T08:36:20.902752+0000","last_unstale":"2024-04-21T08:38:46.923175+0000","last_undegraded":"2024-04-21T08:38:46.923175+0000","last_fullsized":"2024-04-21T08:38:46.923175+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:40.684264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.742602+0000","last_change":"2024-04-21T08:36:24.920505+0000","last_active":"2024-04-21T08:38:46.742602+0000","last_peered":"2024-04-21T08:38:46.742602+0000","last_clean":"2024-04-21T08:38:46.742602+0000","last_became_active":"2024-04-21T08:36:24.919966+0000","last_became_peered":"2024-04-21T08:36:24.919966+0000","last_unstale":"2024-04-21T08:38:46.742602+0000","last_undegraded":"2024-04-21T08:38:46.742602+0000","last_fullsized":"2024-04-21T08:38:46.742602+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:58:05.131815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914242+0000","last_change":"2024-04-21T08:36:20.281720+0000","last_active":"2024-04-21T08:38:46.914242+0000","last_peered":"2024-04-21T08:38:46.914242+0000","last_clean":"2024-04-21T08:38:46.914242+0000","last_became_active":"2024-04-21T08:36:20.279862+0000","last_became_peered":"2024-04-21T08:36:20.279862+0000","last_unstale":"2024-04-21T08:38:46.914242+0000","last_undegraded":"2024-04-21T08:38:46.914242+0000","last_fullsized":"2024-04-21T08:38:46.914242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:18:58.106532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.916030+0000","last_change":"2024-04-21T08:36:20.908520+0000","last_active":"2024-04-21T08:38:46.916030+0000","last_peered":"2024-04-21T08:38:46.916030+0000","last_clean":"2024-04-21T08:38:46.916030+0000","last_became_active":"2024-04-21T08:36:20.908184+0000","last_became_peered":"2024-04-21T08:36:20.908184+0000","last_unstale":"2024-04-21T08:38:46.916030+0000","last_undegraded":"2024-04-21T08:38:46.916030+0000","last_fullsized":"2024-04-21T08:38:46.916030+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:14:33.895864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911416+0000","last_change":"2024-04-21T08:36:24.920697+0000","last_active":"2024-04-21T08:38:46.911416+0000","last_peered":"2024-04-21T08:38:46.911416+0000","last_clean":"2024-04-21T08:38:46.911416+0000","last_became_active":"2024-04-21T08:36:24.919976+0000","last_became_peered":"2024-04-21T08:36:24.919976+0000","last_unstale":"2024-04-21T08:38:46.911416+0000","last_undegraded":"2024-04-21T08:38:46.911416+0000","last_fullsized":"2024-04-21T08:38:46.911416+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:40:09.238887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751422+0000","last_change":"2024-04-21T08:36:20.283531+0000","last_active":"2024-04-21T08:38:46.751422+0000","last_peered":"2024-04-21T08:38:46.751422+0000","last_clean":"2024-04-21T08:38:46.751422+0000","last_became_active":"2024-04-21T08:36:20.283159+0000","last_became_peered":"2024-04-21T08:36:20.283159+0000","last_unstale":"2024-04-21T08:38:46.751422+0000","last_undegraded":"2024-04-21T08:38:46.751422+0000","last_fullsized":"2024-04-21T08:38:46.751422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:57:02.917537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.922897+0000","last_change":"2024-04-21T08:36:20.910113+0000","last_active":"2024-04-21T08:38:46.922897+0000","last_peered":"2024-04-21T08:38:46.922897+0000","last_clean":"2024-04-21T08:38:46.922897+0000","last_became_active":"2024-04-21T08:36:20.909744+0000","last_became_peered":"2024-04-21T08:36:20.909744+0000","last_unstale":"2024-04-21T08:38:46.922897+0000","last_undegraded":"2024-04-21T08:38:46.922897+0000","last_fullsized":"2024-04-21T08:38:46.922897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:16:14.568318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.742678+0000","last_change":"2024-04-21T08:36:24.922232+0000","last_active":"2024-04-21T08:38:46.742678+0000","last_peered":"2024-04-21T08:38:46.742678+0000","last_clean":"2024-04-21T08:38:46.742678+0000","last_became_active":"2024-04-21T08:36:24.921690+0000","last_became_peered":"2024-04-21T08:36:24.921690+0000","last_unstale":"2024-04-21T08:38:46.742678+0000","last_undegraded":"2024-04-21T08:38:46.742678+0000","last_fullsized":"2024-04-21T08:38:46.742678+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:21:42.580203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'42","reported_seq":332,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:47.634217+0000","last_change":"2024-04-21T08:36:20.283574+0000","last_active":"2024-04-21T08:38:47.634217+0000","last_peered":"2024-04-21T08:38:47.634217+0000","last_clean":"2024-04-21T08:38:47.634217+0000","last_became_active":"2024-04-21T08:36:20.283220+0000","last_became_peered":"2024-04-21T08:36:20.283220+0000","last_unstale":"2024-04-21T08:38:47.634217+0000","last_undegraded":"2024-04-21T08:38:47.634217+0000","last_fullsized":"2024-04-21T08:38:47.634217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:55:24.233237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70068,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":257,"num_read_kb":197,"num_write":44,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655458+0000","last_change":"2024-04-21T08:36:20.919264+0000","last_active":"2024-04-21T08:38:46.655458+0000","last_peered":"2024-04-21T08:38:46.655458+0000","last_clean":"2024-04-21T08:38:46.655458+0000","last_became_active":"2024-04-21T08:36:20.918878+0000","last_became_peered":"2024-04-21T08:36:20.918878+0000","last_unstale":"2024-04-21T08:38:46.655458+0000","last_undegraded":"2024-04-21T08:38:46.655458+0000","last_fullsized":"2024-04-21T08:38:46.655458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:24:06.460758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751673+0000","last_change":"2024-04-21T08:36:24.926878+0000","last_active":"2024-04-21T08:38:46.751673+0000","last_peered":"2024-04-21T08:38:46.751673+0000","last_clean":"2024-04-21T08:38:46.751673+0000","last_became_active":"2024-04-21T08:36:24.926431+0000","last_became_peered":"2024-04-21T08:36:24.926431+0000","last_unstale":"2024-04-21T08:38:46.751673+0000","last_undegraded":"2024-04-21T08:38:46.751673+0000","last_fullsized":"2024-04-21T08:38:46.751673+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:51:29.065962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639141+0000","last_change":"2024-04-21T08:36:20.284841+0000","last_active":"2024-04-21T08:38:46.639141+0000","last_peered":"2024-04-21T08:38:46.639141+0000","last_clean":"2024-04-21T08:38:46.639141+0000","last_became_active":"2024-04-21T08:36:20.283888+0000","last_became_peered":"2024-04-21T08:36:20.283888+0000","last_unstale":"2024-04-21T08:38:46.639141+0000","last_undegraded":"2024-04-21T08:38:46.639141+0000","last_fullsized":"2024-04-21T08:38:46.639141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:29:42.659608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.912243+0000","last_change":"2024-04-21T08:36:20.905140+0000","last_active":"2024-04-21T08:38:46.912243+0000","last_peered":"2024-04-21T08:38:46.912243+0000","last_clean":"2024-04-21T08:38:46.912243+0000","last_became_active":"2024-04-21T08:36:20.904821+0000","last_became_peered":"2024-04-21T08:36:20.904821+0000","last_unstale":"2024-04-21T08:38:46.912243+0000","last_undegraded":"2024-04-21T08:38:46.912243+0000","last_fullsized":"2024-04-21T08:38:46.912243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:09:11.962877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807527+0000","last_change":"2024-04-21T08:38:24.834006+0000","last_active":"2024-04-21T08:38:46.807527+0000","last_peered":"2024-04-21T08:38:46.807527+0000","last_clean":"2024-04-21T08:38:46.807527+0000","last_became_active":"2024-04-21T08:38:24.833453+0000","last_became_peered":"2024-04-21T08:38:24.833453+0000","last_unstale":"2024-04-21T08:38:46.807527+0000","last_undegraded":"2024-04-21T08:38:46.807527+0000","last_fullsized":"2024-04-21T08:38:46.807527+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:27:17.094301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751268+0000","last_change":"2024-04-21T08:36:20.290289+0000","last_active":"2024-04-21T08:38:46.751268+0000","last_peered":"2024-04-21T08:38:46.751268+0000","last_clean":"2024-04-21T08:38:46.751268+0000","last_became_active":"2024-04-21T08:36:20.289922+0000","last_became_peered":"2024-04-21T08:36:20.289922+0000","last_unstale":"2024-04-21T08:38:46.751268+0000","last_undegraded":"2024-04-21T08:38:46.751268+0000","last_fullsized":"2024-04-21T08:38:46.751268+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:10:00.926023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914048+0000","last_change":"2024-04-21T08:36:20.906950+0000","last_active":"2024-04-21T08:38:46.914048+0000","last_peered":"2024-04-21T08:38:46.914048+0000","last_clean":"2024-04-21T08:38:46.914048+0000","last_became_active":"2024-04-21T08:36:20.906599+0000","last_became_peered":"2024-04-21T08:36:20.906599+0000","last_unstale":"2024-04-21T08:38:46.914048+0000","last_undegraded":"2024-04-21T08:38:46.914048+0000","last_fullsized":"2024-04-21T08:38:46.914048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:55:27.595997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.743113+0000","last_change":"2024-04-21T08:36:24.924349+0000","last_active":"2024-04-21T08:38:46.743113+0000","last_peered":"2024-04-21T08:38:46.743113+0000","last_clean":"2024-04-21T08:38:46.743113+0000","last_became_active":"2024-04-21T08:36:24.923982+0000","last_became_peered":"2024-04-21T08:36:24.923982+0000","last_unstale":"2024-04-21T08:38:46.743113+0000","last_undegraded":"2024-04-21T08:38:46.743113+0000","last_fullsized":"2024-04-21T08:38:46.743113+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:20:29.600759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"94'9","reported_seq":170,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:47.636667+0000","last_change":"2024-04-21T08:36:20.285629+0000","last_active":"2024-04-21T08:38:47.636667+0000","last_peered":"2024-04-21T08:38:47.636667+0000","last_clean":"2024-04-21T08:38:47.636667+0000","last_became_active":"2024-04-21T08:36:20.285234+0000","last_became_peered":"2024-04-21T08:36:20.285234+0000","last_unstale":"2024-04-21T08:38:47.636667+0000","last_undegraded":"2024-04-21T08:38:47.636667+0000","last_fullsized":"2024-04-21T08:38:47.636667+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:19:32.966146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1902,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":129,"num_read_kb":6,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913981+0000","last_change":"2024-04-21T08:36:20.906819+0000","last_active":"2024-04-21T08:38:46.913981+0000","last_peered":"2024-04-21T08:38:46.913981+0000","last_clean":"2024-04-21T08:38:46.913981+0000","last_became_active":"2024-04-21T08:36:20.906459+0000","last_became_peered":"2024-04-21T08:36:20.906459+0000","last_unstale":"2024-04-21T08:38:46.913981+0000","last_undegraded":"2024-04-21T08:38:46.913981+0000","last_fullsized":"2024-04-21T08:38:46.913981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:12:39.312857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913214+0000","last_change":"2024-04-21T08:36:24.929452+0000","last_active":"2024-04-21T08:38:46.913214+0000","last_peered":"2024-04-21T08:38:46.913214+0000","last_clean":"2024-04-21T08:38:46.913214+0000","last_became_active":"2024-04-21T08:36:24.929039+0000","last_became_peered":"2024-04-21T08:36:24.929039+0000","last_unstale":"2024-04-21T08:38:46.913214+0000","last_undegraded":"2024-04-21T08:38:46.913214+0000","last_fullsized":"2024-04-21T08:38:46.913214+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:16:05.432024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":76,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655055+0000","last_change":"2024-04-21T08:37:24.719766+0000","last_active":"2024-04-21T08:38:46.655055+0000","last_peered":"2024-04-21T08:38:46.655055+0000","last_clean":"2024-04-21T08:38:46.655055+0000","last_became_active":"2024-04-21T08:37:24.708201+0000","last_became_peered":"2024-04-21T08:37:24.708201+0000","last_unstale":"2024-04-21T08:38:46.655055+0000","last_undegraded":"2024-04-21T08:38:46.655055+0000","last_fullsized":"2024-04-21T08:38:46.655055+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:10:51.158061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751195+0000","last_change":"2024-04-21T08:36:20.909076+0000","last_active":"2024-04-21T08:38:46.751195+0000","last_peered":"2024-04-21T08:38:46.751195+0000","last_clean":"2024-04-21T08:38:46.751195+0000","last_became_active":"2024-04-21T08:36:20.908739+0000","last_became_peered":"2024-04-21T08:36:20.908739+0000","last_unstale":"2024-04-21T08:38:46.751195+0000","last_undegraded":"2024-04-21T08:38:46.751195+0000","last_fullsized":"2024-04-21T08:38:46.751195+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:03:56.870477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751077+0000","last_change":"2024-04-21T08:36:24.926871+0000","last_active":"2024-04-21T08:38:46.751077+0000","last_peered":"2024-04-21T08:38:46.751077+0000","last_clean":"2024-04-21T08:38:46.751077+0000","last_became_active":"2024-04-21T08:36:24.926438+0000","last_became_peered":"2024-04-21T08:36:24.926438+0000","last_unstale":"2024-04-21T08:38:46.751077+0000","last_undegraded":"2024-04-21T08:38:46.751077+0000","last_fullsized":"2024-04-21T08:38:46.751077+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:24:48.984906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"94'3","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807724+0000","last_change":"2024-04-21T08:37:24.308038+0000","last_active":"2024-04-21T08:38:46.807724+0000","last_peered":"2024-04-21T08:38:46.807724+0000","last_clean":"2024-04-21T08:38:46.807724+0000","last_became_active":"2024-04-21T08:37:24.301034+0000","last_became_peered":"2024-04-21T08:37:24.301034+0000","last_unstale":"2024-04-21T08:38:46.807724+0000","last_undegraded":"2024-04-21T08:38:46.807724+0000","last_fullsized":"2024-04-21T08:38:46.807724+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:37:06.598400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807693+0000","last_change":"2024-04-21T08:37:24.299260+0000","last_active":"2024-04-21T08:38:46.807693+0000","last_peered":"2024-04-21T08:38:46.807693+0000","last_clean":"2024-04-21T08:38:46.807693+0000","last_became_active":"2024-04-21T08:37:24.298831+0000","last_became_peered":"2024-04-21T08:37:24.298831+0000","last_unstale":"2024-04-21T08:38:46.807693+0000","last_undegraded":"2024-04-21T08:38:46.807693+0000","last_fullsized":"2024-04-21T08:38:46.807693+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:58:29.368027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655140+0000","last_change":"2024-04-21T08:36:24.929420+0000","last_active":"2024-04-21T08:38:46.655140+0000","last_peered":"2024-04-21T08:38:46.655140+0000","last_clean":"2024-04-21T08:38:46.655140+0000","last_became_active":"2024-04-21T08:36:24.928981+0000","last_became_peered":"2024-04-21T08:36:24.928981+0000","last_unstale":"2024-04-21T08:38:46.655140+0000","last_undegraded":"2024-04-21T08:38:46.655140+0000","last_fullsized":"2024-04-21T08:38:46.655140+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:38:00.040456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.742890+0000","last_change":"2024-04-21T08:36:20.279910+0000","last_active":"2024-04-21T08:38:46.742890+0000","last_peered":"2024-04-21T08:38:46.742890+0000","last_clean":"2024-04-21T08:38:46.742890+0000","last_became_active":"2024-04-21T08:36:20.278904+0000","last_became_peered":"2024-04-21T08:36:20.278904+0000","last_unstale":"2024-04-21T08:38:46.742890+0000","last_undegraded":"2024-04-21T08:38:46.742890+0000","last_fullsized":"2024-04-21T08:38:46.742890+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:40:19.209213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638524+0000","last_change":"2024-04-21T08:36:20.895816+0000","last_active":"2024-04-21T08:38:46.638524+0000","last_peered":"2024-04-21T08:38:46.638524+0000","last_clean":"2024-04-21T08:38:46.638524+0000","last_became_active":"2024-04-21T08:36:20.895487+0000","last_became_peered":"2024-04-21T08:36:20.895487+0000","last_unstale":"2024-04-21T08:38:46.638524+0000","last_undegraded":"2024-04-21T08:38:46.638524+0000","last_fullsized":"2024-04-21T08:38:46.638524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:53:52.153683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.912486+0000","last_change":"2024-04-21T08:36:24.917637+0000","last_active":"2024-04-21T08:38:46.912486+0000","last_peered":"2024-04-21T08:38:46.912486+0000","last_clean":"2024-04-21T08:38:46.912486+0000","last_became_active":"2024-04-21T08:36:24.917192+0000","last_became_peered":"2024-04-21T08:36:24.917192+0000","last_unstale":"2024-04-21T08:38:46.912486+0000","last_undegraded":"2024-04-21T08:38:46.912486+0000","last_fullsized":"2024-04-21T08:38:46.912486+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:02:59.843903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":159,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914154+0000","last_change":"2024-04-21T08:36:20.286424+0000","last_active":"2024-04-21T08:38:46.914154+0000","last_peered":"2024-04-21T08:38:46.914154+0000","last_clean":"2024-04-21T08:38:46.914154+0000","last_became_active":"2024-04-21T08:36:20.285767+0000","last_became_peered":"2024-04-21T08:36:20.285767+0000","last_unstale":"2024-04-21T08:38:46.914154+0000","last_undegraded":"2024-04-21T08:38:46.914154+0000","last_fullsized":"2024-04-21T08:38:46.914154+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:53:33.202734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638138+0000","last_change":"2024-04-21T08:37:24.297650+0000","last_active":"2024-04-21T08:38:46.638138+0000","last_peered":"2024-04-21T08:38:46.638138+0000","last_clean":"2024-04-21T08:38:46.638138+0000","last_became_active":"2024-04-21T08:37:24.297250+0000","last_became_peered":"2024-04-21T08:37:24.297250+0000","last_unstale":"2024-04-21T08:38:46.638138+0000","last_undegraded":"2024-04-21T08:38:46.638138+0000","last_fullsized":"2024-04-21T08:38:46.638138+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:21:21.419556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911806+0000","last_change":"2024-04-21T08:36:24.919669+0000","last_active":"2024-04-21T08:38:46.911806+0000","last_peered":"2024-04-21T08:38:46.911806+0000","last_clean":"2024-04-21T08:38:46.911806+0000","last_became_active":"2024-04-21T08:36:24.919281+0000","last_became_peered":"2024-04-21T08:36:24.919281+0000","last_unstale":"2024-04-21T08:38:46.911806+0000","last_undegraded":"2024-04-21T08:38:46.911806+0000","last_fullsized":"2024-04-21T08:38:46.911806+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:44:27.480367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751175+0000","last_change":"2024-04-21T08:37:24.297826+0000","last_active":"2024-04-21T08:38:46.751175+0000","last_peered":"2024-04-21T08:38:46.751175+0000","last_clean":"2024-04-21T08:38:46.751175+0000","last_became_active":"2024-04-21T08:37:24.297397+0000","last_became_peered":"2024-04-21T08:37:24.297397+0000","last_unstale":"2024-04-21T08:38:46.751175+0000","last_undegraded":"2024-04-21T08:38:46.751175+0000","last_fullsized":"2024-04-21T08:38:46.751175+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:52:17.693725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'7","reported_seq":165,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.916204+0000","last_change":"2024-04-21T08:36:20.291033+0000","last_active":"2024-04-21T08:38:46.916204+0000","last_peered":"2024-04-21T08:38:46.916204+0000","last_clean":"2024-04-21T08:38:46.916204+0000","last_became_active":"2024-04-21T08:36:20.290697+0000","last_became_peered":"2024-04-21T08:36:20.290697+0000","last_unstale":"2024-04-21T08:38:46.916204+0000","last_undegraded":"2024-04-21T08:38:46.916204+0000","last_fullsized":"2024-04-21T08:38:46.916204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:22:54.593595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":124,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913716+0000","last_change":"2024-04-21T08:37:24.310936+0000","last_active":"2024-04-21T08:38:46.913716+0000","last_peered":"2024-04-21T08:38:46.913716+0000","last_clean":"2024-04-21T08:38:46.913716+0000","last_became_active":"2024-04-21T08:37:24.297349+0000","last_became_peered":"2024-04-21T08:37:24.297349+0000","last_unstale":"2024-04-21T08:38:46.913716+0000","last_undegraded":"2024-04-21T08:38:46.913716+0000","last_fullsized":"2024-04-21T08:38:46.913716+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:31:39.884609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638671+0000","last_change":"2024-04-21T08:36:20.904197+0000","last_active":"2024-04-21T08:38:46.638671+0000","last_peered":"2024-04-21T08:38:46.638671+0000","last_clean":"2024-04-21T08:38:46.638671+0000","last_became_active":"2024-04-21T08:36:20.903099+0000","last_became_peered":"2024-04-21T08:36:20.903099+0000","last_unstale":"2024-04-21T08:38:46.638671+0000","last_undegraded":"2024-04-21T08:38:46.638671+0000","last_fullsized":"2024-04-21T08:38:46.638671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:16:08.914335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911254+0000","last_change":"2024-04-21T08:36:20.280608+0000","last_active":"2024-04-21T08:38:46.911254+0000","last_peered":"2024-04-21T08:38:46.911254+0000","last_clean":"2024-04-21T08:38:46.911254+0000","last_became_active":"2024-04-21T08:36:20.280172+0000","last_became_peered":"2024-04-21T08:36:20.280172+0000","last_unstale":"2024-04-21T08:38:46.911254+0000","last_undegraded":"2024-04-21T08:38:46.911254+0000","last_fullsized":"2024-04-21T08:38:46.911254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:51:12.284463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":366,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913553+0000","last_change":"2024-04-21T08:35:08.423545+0000","last_active":"2024-04-21T08:38:46.913553+0000","last_peered":"2024-04-21T08:38:46.913553+0000","last_clean":"2024-04-21T08:38:46.913553+0000","last_became_active":"2024-04-21T08:35:08.423027+0000","last_became_peered":"2024-04-21T08:35:08.423027+0000","last_unstale":"2024-04-21T08:38:46.913553+0000","last_undegraded":"2024-04-21T08:38:46.913553+0000","last_fullsized":"2024-04-21T08:38:46.913553+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:31:45.359253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:31:45.359253+0000","last_clean_scrub_stamp":"2024-04-21T08:31:45.359253+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:53:24.310824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751452+0000","last_change":"2024-04-21T08:36:24.922422+0000","last_active":"2024-04-21T08:38:46.751452+0000","last_peered":"2024-04-21T08:38:46.751452+0000","last_clean":"2024-04-21T08:38:46.751452+0000","last_became_active":"2024-04-21T08:36:24.922006+0000","last_became_peered":"2024-04-21T08:36:24.922006+0000","last_unstale":"2024-04-21T08:38:46.751452+0000","last_undegraded":"2024-04-21T08:38:46.751452+0000","last_fullsized":"2024-04-21T08:38:46.751452+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:46:22.840802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911805+0000","last_change":"2024-04-21T08:36:20.911222+0000","last_active":"2024-04-21T08:38:46.911805+0000","last_peered":"2024-04-21T08:38:46.911805+0000","last_clean":"2024-04-21T08:38:46.911805+0000","last_became_active":"2024-04-21T08:36:20.910933+0000","last_became_peered":"2024-04-21T08:36:20.910933+0000","last_unstale":"2024-04-21T08:38:46.911805+0000","last_undegraded":"2024-04-21T08:38:46.911805+0000","last_fullsized":"2024-04-21T08:38:46.911805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:41:25.621266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913633+0000","last_change":"2024-04-21T08:36:20.288409+0000","last_active":"2024-04-21T08:38:46.913633+0000","last_peered":"2024-04-21T08:38:46.913633+0000","last_clean":"2024-04-21T08:38:46.913633+0000","last_became_active":"2024-04-21T08:36:20.287916+0000","last_became_peered":"2024-04-21T08:36:20.287916+0000","last_unstale":"2024-04-21T08:38:46.913633+0000","last_undegraded":"2024-04-21T08:38:46.913633+0000","last_fullsized":"2024-04-21T08:38:46.913633+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:04:31.068352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807586+0000","last_change":"2024-04-21T08:36:24.918166+0000","last_active":"2024-04-21T08:38:46.807586+0000","last_peered":"2024-04-21T08:38:46.807586+0000","last_clean":"2024-04-21T08:38:46.807586+0000","last_became_active":"2024-04-21T08:36:24.917745+0000","last_became_peered":"2024-04-21T08:36:24.917745+0000","last_unstale":"2024-04-21T08:38:46.807586+0000","last_undegraded":"2024-04-21T08:38:46.807586+0000","last_fullsized":"2024-04-21T08:38:46.807586+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:55:41.169681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655417+0000","last_change":"2024-04-21T08:36:20.291079+0000","last_active":"2024-04-21T08:38:46.655417+0000","last_peered":"2024-04-21T08:38:46.655417+0000","last_clean":"2024-04-21T08:38:46.655417+0000","last_became_active":"2024-04-21T08:36:20.290720+0000","last_became_peered":"2024-04-21T08:36:20.290720+0000","last_unstale":"2024-04-21T08:38:46.655417+0000","last_undegraded":"2024-04-21T08:38:46.655417+0000","last_fullsized":"2024-04-21T08:38:46.655417+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:41.269029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.912713+0000","last_change":"2024-04-21T08:36:20.907416+0000","last_active":"2024-04-21T08:38:46.912713+0000","last_peered":"2024-04-21T08:38:46.912713+0000","last_clean":"2024-04-21T08:38:46.912713+0000","last_became_active":"2024-04-21T08:36:20.907005+0000","last_became_peered":"2024-04-21T08:36:20.907005+0000","last_unstale":"2024-04-21T08:38:46.912713+0000","last_undegraded":"2024-04-21T08:38:46.912713+0000","last_fullsized":"2024-04-21T08:38:46.912713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:58:17.979947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638481+0000","last_change":"2024-04-21T08:36:24.921475+0000","last_active":"2024-04-21T08:38:46.638481+0000","last_peered":"2024-04-21T08:38:46.638481+0000","last_clean":"2024-04-21T08:38:46.638481+0000","last_became_active":"2024-04-21T08:36:24.921091+0000","last_became_peered":"2024-04-21T08:36:24.921091+0000","last_unstale":"2024-04-21T08:38:46.638481+0000","last_undegraded":"2024-04-21T08:38:46.638481+0000","last_fullsized":"2024-04-21T08:38:46.638481+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:28:14.224501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'48","reported_seq":105,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.860978+0000","last_change":"2024-04-21T08:36:20.288916+0000","last_active":"2024-04-21T08:38:46.860978+0000","last_peered":"2024-04-21T08:38:46.860978+0000","last_clean":"2024-04-21T08:38:46.860978+0000","last_became_active":"2024-04-21T08:36:20.288575+0000","last_became_peered":"2024-04-21T08:36:20.288575+0000","last_unstale":"2024-04-21T08:38:46.860978+0000","last_undegraded":"2024-04-21T08:38:46.860978+0000","last_fullsized":"2024-04-21T08:38:46.860978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:52:05.795554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639071+0000","last_change":"2024-04-21T08:36:20.912053+0000","last_active":"2024-04-21T08:38:46.639071+0000","last_peered":"2024-04-21T08:38:46.639071+0000","last_clean":"2024-04-21T08:38:46.639071+0000","last_became_active":"2024-04-21T08:36:20.911584+0000","last_became_peered":"2024-04-21T08:36:20.911584+0000","last_unstale":"2024-04-21T08:38:46.639071+0000","last_undegraded":"2024-04-21T08:38:46.639071+0000","last_fullsized":"2024-04-21T08:38:46.639071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:48:36.369954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638589+0000","last_change":"2024-04-21T08:36:24.919212+0000","last_active":"2024-04-21T08:38:46.638589+0000","last_peered":"2024-04-21T08:38:46.638589+0000","last_clean":"2024-04-21T08:38:46.638589+0000","last_became_active":"2024-04-21T08:36:24.918807+0000","last_became_peered":"2024-04-21T08:36:24.918807+0000","last_unstale":"2024-04-21T08:38:46.638589+0000","last_undegraded":"2024-04-21T08:38:46.638589+0000","last_fullsized":"2024-04-21T08:38:46.638589+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:25:50.816775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911766+0000","last_change":"2024-04-21T08:36:20.282536+0000","last_active":"2024-04-21T08:38:46.911766+0000","last_peered":"2024-04-21T08:38:46.911766+0000","last_clean":"2024-04-21T08:38:46.911766+0000","last_became_active":"2024-04-21T08:36:20.282172+0000","last_became_peered":"2024-04-21T08:36:20.282172+0000","last_unstale":"2024-04-21T08:38:46.911766+0000","last_undegraded":"2024-04-21T08:38:46.911766+0000","last_fullsized":"2024-04-21T08:38:46.911766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:15:38.992441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.915413+0000","last_change":"2024-04-21T08:36:20.908416+0000","last_active":"2024-04-21T08:38:46.915413+0000","last_peered":"2024-04-21T08:38:46.915413+0000","last_clean":"2024-04-21T08:38:46.915413+0000","last_became_active":"2024-04-21T08:36:20.908054+0000","last_became_peered":"2024-04-21T08:36:20.908054+0000","last_unstale":"2024-04-21T08:38:46.915413+0000","last_undegraded":"2024-04-21T08:38:46.915413+0000","last_fullsized":"2024-04-21T08:38:46.915413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:23:16.562060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911638+0000","last_change":"2024-04-21T08:36:24.920637+0000","last_active":"2024-04-21T08:38:46.911638+0000","last_peered":"2024-04-21T08:38:46.911638+0000","last_clean":"2024-04-21T08:38:46.911638+0000","last_became_active":"2024-04-21T08:36:24.919921+0000","last_became_peered":"2024-04-21T08:36:24.919921+0000","last_unstale":"2024-04-21T08:38:46.911638+0000","last_undegraded":"2024-04-21T08:38:46.911638+0000","last_fullsized":"2024-04-21T08:38:46.911638+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:20:38.230087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.922412+0000","last_change":"2024-04-21T08:36:20.279332+0000","last_active":"2024-04-21T08:38:46.922412+0000","last_peered":"2024-04-21T08:38:46.922412+0000","last_clean":"2024-04-21T08:38:46.922412+0000","last_became_active":"2024-04-21T08:36:20.278970+0000","last_became_peered":"2024-04-21T08:36:20.278970+0000","last_unstale":"2024-04-21T08:38:46.922412+0000","last_undegraded":"2024-04-21T08:38:46.922412+0000","last_fullsized":"2024-04-21T08:38:46.922412+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:36:16.041610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.654968+0000","last_change":"2024-04-21T08:36:20.919621+0000","last_active":"2024-04-21T08:38:46.654968+0000","last_peered":"2024-04-21T08:38:46.654968+0000","last_clean":"2024-04-21T08:38:46.654968+0000","last_became_active":"2024-04-21T08:36:20.919246+0000","last_became_peered":"2024-04-21T08:36:20.919246+0000","last_unstale":"2024-04-21T08:38:46.654968+0000","last_undegraded":"2024-04-21T08:38:46.654968+0000","last_fullsized":"2024-04-21T08:38:46.654968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:31:59.001501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.743258+0000","last_change":"2024-04-21T08:36:24.920459+0000","last_active":"2024-04-21T08:38:46.743258+0000","last_peered":"2024-04-21T08:38:46.743258+0000","last_clean":"2024-04-21T08:38:46.743258+0000","last_became_active":"2024-04-21T08:36:24.919691+0000","last_became_peered":"2024-04-21T08:36:24.919691+0000","last_unstale":"2024-04-21T08:38:46.743258+0000","last_undegraded":"2024-04-21T08:38:46.743258+0000","last_fullsized":"2024-04-21T08:38:46.743258+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:38:26.655452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638571+0000","last_change":"2024-04-21T08:36:20.284286+0000","last_active":"2024-04-21T08:38:46.638571+0000","last_peered":"2024-04-21T08:38:46.638571+0000","last_clean":"2024-04-21T08:38:46.638571+0000","last_became_active":"2024-04-21T08:36:20.283903+0000","last_became_peered":"2024-04-21T08:36:20.283903+0000","last_unstale":"2024-04-21T08:38:46.638571+0000","last_undegraded":"2024-04-21T08:38:46.638571+0000","last_fullsized":"2024-04-21T08:38:46.638571+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:51:52.409988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807332+0000","last_change":"2024-04-21T08:36:20.911892+0000","last_active":"2024-04-21T08:38:46.807332+0000","last_peered":"2024-04-21T08:38:46.807332+0000","last_clean":"2024-04-21T08:38:46.807332+0000","last_became_active":"2024-04-21T08:36:20.911513+0000","last_became_peered":"2024-04-21T08:36:20.911513+0000","last_unstale":"2024-04-21T08:38:46.807332+0000","last_undegraded":"2024-04-21T08:38:46.807332+0000","last_fullsized":"2024-04-21T08:38:46.807332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:51:13.733534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911539+0000","last_change":"2024-04-21T08:36:24.920347+0000","last_active":"2024-04-21T08:38:46.911539+0000","last_peered":"2024-04-21T08:38:46.911539+0000","last_clean":"2024-04-21T08:38:46.911539+0000","last_became_active":"2024-04-21T08:36:24.919831+0000","last_became_peered":"2024-04-21T08:36:24.919831+0000","last_unstale":"2024-04-21T08:38:46.911539+0000","last_undegraded":"2024-04-21T08:38:46.911539+0000","last_fullsized":"2024-04-21T08:38:46.911539+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:14:27.020476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638672+0000","last_change":"2024-04-21T08:36:20.286786+0000","last_active":"2024-04-21T08:38:46.638672+0000","last_peered":"2024-04-21T08:38:46.638672+0000","last_clean":"2024-04-21T08:38:46.638672+0000","last_became_active":"2024-04-21T08:36:20.286429+0000","last_became_peered":"2024-04-21T08:36:20.286429+0000","last_unstale":"2024-04-21T08:38:46.638672+0000","last_undegraded":"2024-04-21T08:38:46.638672+0000","last_fullsized":"2024-04-21T08:38:46.638672+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:40:57.093370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638983+0000","last_change":"2024-04-21T08:36:20.904767+0000","last_active":"2024-04-21T08:38:46.638983+0000","last_peered":"2024-04-21T08:38:46.638983+0000","last_clean":"2024-04-21T08:38:46.638983+0000","last_became_active":"2024-04-21T08:36:20.904406+0000","last_became_peered":"2024-04-21T08:36:20.904406+0000","last_unstale":"2024-04-21T08:38:46.638983+0000","last_undegraded":"2024-04-21T08:38:46.638983+0000","last_fullsized":"2024-04-21T08:38:46.638983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:11:41.644549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911769+0000","last_change":"2024-04-21T08:36:24.918031+0000","last_active":"2024-04-21T08:38:46.911769+0000","last_peered":"2024-04-21T08:38:46.911769+0000","last_clean":"2024-04-21T08:38:46.911769+0000","last_became_active":"2024-04-21T08:36:24.917618+0000","last_became_peered":"2024-04-21T08:36:24.917618+0000","last_unstale":"2024-04-21T08:38:46.911769+0000","last_undegraded":"2024-04-21T08:38:46.911769+0000","last_fullsized":"2024-04-21T08:38:46.911769+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:50:35.797911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655179+0000","last_change":"2024-04-21T08:36:20.286212+0000","last_active":"2024-04-21T08:38:46.655179+0000","last_peered":"2024-04-21T08:38:46.655179+0000","last_clean":"2024-04-21T08:38:46.655179+0000","last_became_active":"2024-04-21T08:36:20.285878+0000","last_became_peered":"2024-04-21T08:36:20.285878+0000","last_unstale":"2024-04-21T08:38:46.655179+0000","last_undegraded":"2024-04-21T08:38:46.655179+0000","last_fullsized":"2024-04-21T08:38:46.655179+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:19.375455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914242+0000","last_change":"2024-04-21T08:36:20.908335+0000","last_active":"2024-04-21T08:38:46.914242+0000","last_peered":"2024-04-21T08:38:46.914242+0000","last_clean":"2024-04-21T08:38:46.914242+0000","last_became_active":"2024-04-21T08:36:20.907945+0000","last_became_peered":"2024-04-21T08:36:20.907945+0000","last_unstale":"2024-04-21T08:38:46.914242+0000","last_undegraded":"2024-04-21T08:38:46.914242+0000","last_fullsized":"2024-04-21T08:38:46.914242+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:47:17.841511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.922383+0000","last_change":"2024-04-21T08:36:24.915553+0000","last_active":"2024-04-21T08:38:46.922383+0000","last_peered":"2024-04-21T08:38:46.922383+0000","last_clean":"2024-04-21T08:38:46.922383+0000","last_became_active":"2024-04-21T08:36:24.915028+0000","last_became_peered":"2024-04-21T08:36:24.915028+0000","last_unstale":"2024-04-21T08:38:46.922383+0000","last_undegraded":"2024-04-21T08:38:46.922383+0000","last_fullsized":"2024-04-21T08:38:46.922383+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:17:21.192946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'7","reported_seq":164,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639097+0000","last_change":"2024-04-21T08:36:20.279649+0000","last_active":"2024-04-21T08:38:46.639097+0000","last_peered":"2024-04-21T08:38:46.639097+0000","last_clean":"2024-04-21T08:38:46.639097+0000","last_became_active":"2024-04-21T08:36:20.279111+0000","last_became_peered":"2024-04-21T08:36:20.279111+0000","last_unstale":"2024-04-21T08:38:46.639097+0000","last_undegraded":"2024-04-21T08:38:46.639097+0000","last_fullsized":"2024-04-21T08:38:46.639097+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:19:57.380495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914334+0000","last_change":"2024-04-21T08:36:20.911489+0000","last_active":"2024-04-21T08:38:46.914334+0000","last_peered":"2024-04-21T08:38:46.914334+0000","last_clean":"2024-04-21T08:38:46.914334+0000","last_became_active":"2024-04-21T08:36:20.911168+0000","last_became_peered":"2024-04-21T08:36:20.911168+0000","last_unstale":"2024-04-21T08:38:46.914334+0000","last_undegraded":"2024-04-21T08:38:46.914334+0000","last_fullsized":"2024-04-21T08:38:46.914334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:40:41.585841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914668+0000","last_change":"2024-04-21T08:36:24.921175+0000","last_active":"2024-04-21T08:38:46.914668+0000","last_peered":"2024-04-21T08:38:46.914668+0000","last_clean":"2024-04-21T08:38:46.914668+0000","last_became_active":"2024-04-21T08:36:24.920802+0000","last_became_peered":"2024-04-21T08:36:24.920802+0000","last_unstale":"2024-04-21T08:38:46.914668+0000","last_undegraded":"2024-04-21T08:38:46.914668+0000","last_fullsized":"2024-04-21T08:38:46.914668+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:43:12.140016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"94'41","reported_seq":82,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.859069+0000","last_change":"2024-04-21T08:36:20.290202+0000","last_active":"2024-04-21T08:38:46.859069+0000","last_peered":"2024-04-21T08:38:46.859069+0000","last_clean":"2024-04-21T08:38:46.859069+0000","last_became_active":"2024-04-21T08:36:20.289868+0000","last_became_peered":"2024-04-21T08:36:20.289868+0000","last_unstale":"2024-04-21T08:38:46.859069+0000","last_undegraded":"2024-04-21T08:38:46.859069+0000","last_fullsized":"2024-04-21T08:38:46.859069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:07:45.525980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913789+0000","last_change":"2024-04-21T08:36:20.899590+0000","last_active":"2024-04-21T08:38:46.913789+0000","last_peered":"2024-04-21T08:38:46.913789+0000","last_clean":"2024-04-21T08:38:46.913789+0000","last_became_active":"2024-04-21T08:36:20.899254+0000","last_became_peered":"2024-04-21T08:36:20.899254+0000","last_unstale":"2024-04-21T08:38:46.913789+0000","last_undegraded":"2024-04-21T08:38:46.913789+0000","last_fullsized":"2024-04-21T08:38:46.913789+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:34:25.409694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.750892+0000","last_change":"2024-04-21T08:36:24.918310+0000","last_active":"2024-04-21T08:38:46.750892+0000","last_peered":"2024-04-21T08:38:46.750892+0000","last_clean":"2024-04-21T08:38:46.750892+0000","last_became_active":"2024-04-21T08:36:24.917899+0000","last_became_peered":"2024-04-21T08:36:24.917899+0000","last_unstale":"2024-04-21T08:38:46.750892+0000","last_undegraded":"2024-04-21T08:38:46.750892+0000","last_fullsized":"2024-04-21T08:38:46.750892+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:09:13.006884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":51,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913309+0000","last_change":"2024-04-21T08:37:24.304544+0000","last_active":"2024-04-21T08:38:46.913309+0000","last_peered":"2024-04-21T08:38:46.913309+0000","last_clean":"2024-04-21T08:38:46.913309+0000","last_became_active":"2024-04-21T08:37:24.295808+0000","last_became_peered":"2024-04-21T08:37:24.295808+0000","last_unstale":"2024-04-21T08:38:46.913309+0000","last_undegraded":"2024-04-21T08:38:46.913309+0000","last_fullsized":"2024-04-21T08:38:46.913309+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:07:15.797133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914356+0000","last_change":"2024-04-21T08:36:20.912368+0000","last_active":"2024-04-21T08:38:46.914356+0000","last_peered":"2024-04-21T08:38:46.914356+0000","last_clean":"2024-04-21T08:38:46.914356+0000","last_became_active":"2024-04-21T08:36:20.912009+0000","last_became_peered":"2024-04-21T08:36:20.912009+0000","last_unstale":"2024-04-21T08:38:46.914356+0000","last_undegraded":"2024-04-21T08:38:46.914356+0000","last_fullsized":"2024-04-21T08:38:46.914356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:35:40.700314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807399+0000","last_change":"2024-04-21T08:36:20.913472+0000","last_active":"2024-04-21T08:38:46.807399+0000","last_peered":"2024-04-21T08:38:46.807399+0000","last_clean":"2024-04-21T08:38:46.807399+0000","last_became_active":"2024-04-21T08:36:20.913118+0000","last_became_peered":"2024-04-21T08:36:20.913118+0000","last_unstale":"2024-04-21T08:38:46.807399+0000","last_undegraded":"2024-04-21T08:38:46.807399+0000","last_fullsized":"2024-04-21T08:38:46.807399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:31:53.957044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638786+0000","last_change":"2024-04-21T08:36:20.913677+0000","last_active":"2024-04-21T08:38:46.638786+0000","last_peered":"2024-04-21T08:38:46.638786+0000","last_clean":"2024-04-21T08:38:46.638786+0000","last_became_active":"2024-04-21T08:36:20.913097+0000","last_became_peered":"2024-04-21T08:36:20.913097+0000","last_unstale":"2024-04-21T08:38:46.638786+0000","last_undegraded":"2024-04-21T08:38:46.638786+0000","last_fullsized":"2024-04-21T08:38:46.638786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:08:40.388370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638662+0000","last_change":"2024-04-21T08:36:20.906624+0000","last_active":"2024-04-21T08:38:46.638662+0000","last_peered":"2024-04-21T08:38:46.638662+0000","last_clean":"2024-04-21T08:38:46.638662+0000","last_became_active":"2024-04-21T08:36:20.906247+0000","last_became_peered":"2024-04-21T08:36:20.906247+0000","last_unstale":"2024-04-21T08:38:46.638662+0000","last_undegraded":"2024-04-21T08:38:46.638662+0000","last_fullsized":"2024-04-21T08:38:46.638662+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:44:11.855183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.916219+0000","last_change":"2024-04-21T08:36:20.906576+0000","last_active":"2024-04-21T08:38:46.916219+0000","last_peered":"2024-04-21T08:38:46.916219+0000","last_clean":"2024-04-21T08:38:46.916219+0000","last_became_active":"2024-04-21T08:36:20.906231+0000","last_became_peered":"2024-04-21T08:36:20.906231+0000","last_unstale":"2024-04-21T08:38:46.916219+0000","last_undegraded":"2024-04-21T08:38:46.916219+0000","last_fullsized":"2024-04-21T08:38:46.916219+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:42:48.156614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.912593+0000","last_change":"2024-04-21T08:36:20.902551+0000","last_active":"2024-04-21T08:38:46.912593+0000","last_peered":"2024-04-21T08:38:46.912593+0000","last_clean":"2024-04-21T08:38:46.912593+0000","last_became_active":"2024-04-21T08:36:20.902208+0000","last_became_peered":"2024-04-21T08:36:20.902208+0000","last_unstale":"2024-04-21T08:38:46.912593+0000","last_undegraded":"2024-04-21T08:38:46.912593+0000","last_fullsized":"2024-04-21T08:38:46.912593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:36:30.016256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807607+0000","last_change":"2024-04-21T08:36:20.918588+0000","last_active":"2024-04-21T08:38:46.807607+0000","last_peered":"2024-04-21T08:38:46.807607+0000","last_clean":"2024-04-21T08:38:46.807607+0000","last_became_active":"2024-04-21T08:36:20.918263+0000","last_became_peered":"2024-04-21T08:36:20.918263+0000","last_unstale":"2024-04-21T08:38:46.807607+0000","last_undegraded":"2024-04-21T08:38:46.807607+0000","last_fullsized":"2024-04-21T08:38:46.807607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:51:30.340336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639167+0000","last_change":"2024-04-21T08:36:20.911063+0000","last_active":"2024-04-21T08:38:46.639167+0000","last_peered":"2024-04-21T08:38:46.639167+0000","last_clean":"2024-04-21T08:38:46.639167+0000","last_became_active":"2024-04-21T08:36:20.910647+0000","last_became_peered":"2024-04-21T08:36:20.910647+0000","last_unstale":"2024-04-21T08:38:46.639167+0000","last_undegraded":"2024-04-21T08:38:46.639167+0000","last_fullsized":"2024-04-21T08:38:46.639167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:10:31.079782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639213+0000","last_change":"2024-04-21T08:36:20.914079+0000","last_active":"2024-04-21T08:38:46.639213+0000","last_peered":"2024-04-21T08:38:46.639213+0000","last_clean":"2024-04-21T08:38:46.639213+0000","last_became_active":"2024-04-21T08:36:20.912851+0000","last_became_peered":"2024-04-21T08:36:20.912851+0000","last_unstale":"2024-04-21T08:38:46.639213+0000","last_undegraded":"2024-04-21T08:38:46.639213+0000","last_fullsized":"2024-04-21T08:38:46.639213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:53:35.811281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751071+0000","last_change":"2024-04-21T08:36:20.900551+0000","last_active":"2024-04-21T08:38:46.751071+0000","last_peered":"2024-04-21T08:38:46.751071+0000","last_clean":"2024-04-21T08:38:46.751071+0000","last_became_active":"2024-04-21T08:36:20.900173+0000","last_became_peered":"2024-04-21T08:36:20.900173+0000","last_unstale":"2024-04-21T08:38:46.751071+0000","last_undegraded":"2024-04-21T08:38:46.751071+0000","last_fullsized":"2024-04-21T08:38:46.751071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:21.389498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638564+0000","last_change":"2024-04-21T08:36:20.912953+0000","last_active":"2024-04-21T08:38:46.638564+0000","last_peered":"2024-04-21T08:38:46.638564+0000","last_clean":"2024-04-21T08:38:46.638564+0000","last_became_active":"2024-04-21T08:36:20.912223+0000","last_became_peered":"2024-04-21T08:36:20.912223+0000","last_unstale":"2024-04-21T08:38:46.638564+0000","last_undegraded":"2024-04-21T08:38:46.638564+0000","last_fullsized":"2024-04-21T08:38:46.638564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:43:14.277179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.922633+0000","last_change":"2024-04-21T08:36:20.906164+0000","last_active":"2024-04-21T08:38:46.922633+0000","last_peered":"2024-04-21T08:38:46.922633+0000","last_clean":"2024-04-21T08:38:46.922633+0000","last_became_active":"2024-04-21T08:36:20.905749+0000","last_became_peered":"2024-04-21T08:36:20.905749+0000","last_unstale":"2024-04-21T08:38:46.922633+0000","last_undegraded":"2024-04-21T08:38:46.922633+0000","last_fullsized":"2024-04-21T08:38:46.922633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:02:10.569087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913644+0000","last_change":"2024-04-21T08:36:20.900314+0000","last_active":"2024-04-21T08:38:46.913644+0000","last_peered":"2024-04-21T08:38:46.913644+0000","last_clean":"2024-04-21T08:38:46.913644+0000","last_became_active":"2024-04-21T08:36:20.900009+0000","last_became_peered":"2024-04-21T08:36:20.900009+0000","last_unstale":"2024-04-21T08:38:46.913644+0000","last_undegraded":"2024-04-21T08:38:46.913644+0000","last_fullsized":"2024-04-21T08:38:46.913644+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:19:59.997302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807555+0000","last_change":"2024-04-21T08:36:20.920725+0000","last_active":"2024-04-21T08:38:46.807555+0000","last_peered":"2024-04-21T08:38:46.807555+0000","last_clean":"2024-04-21T08:38:46.807555+0000","last_became_active":"2024-04-21T08:36:20.920288+0000","last_became_peered":"2024-04-21T08:36:20.920288+0000","last_unstale":"2024-04-21T08:38:46.807555+0000","last_undegraded":"2024-04-21T08:38:46.807555+0000","last_fullsized":"2024-04-21T08:38:46.807555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:36:18.736388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":603992,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1350,"num_read_kb":791,"num_write":375,"num_write_kb":1473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":806,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1811976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":305,"ondisk_log_size":305,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383722,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1093632,"data_stored":531299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645358,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28764,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716388,"statfs":{"total":95995035648,"available":95965581312,"internally_reserved":0,"allocated":1785856,"data_stored":1190803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808884,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1105920,"data_stored":530003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070521,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28476,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716676,"statfs":{"total":95995035648,"available":95965876224,"internally_reserved":0,"allocated":1490944,"data_stored":923575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332159,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28672,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716480,"statfs":{"total":95995035648,"available":95965675520,"internally_reserved":0,"allocated":1691648,"data_stored":1127969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495684,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1081344,"data_stored":525507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36476,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708676,"statfs":{"total":95995035648,"available":95957684224,"internally_reserved":0,"allocated":1163264,"data_stored":596717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1105920,"data_stored":527495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36480,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708672,"statfs":{"total":95995035648,"available":95957680128,"internally_reserved":0,"allocated":1167360,"data_stored":598085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182482,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1050031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920846,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1089536,"data_stored":530895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757320,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":33156,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93711996,"statfs":{"total":95995035648,"available":95961083904,"internally_reserved":0,"allocated":1843200,"data_stored":1254509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":71932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":73300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":394694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":204718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T08:38:49.309 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-21T08:38:49.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.446+0000 7f317c952640 1 -- 172.21.15.86:0/3993978131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3178102980 msgr2=0x7f31781092a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:49.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.446+0000 7f317c952640 1 --2- 172.21.15.86:0/3993978131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3178102980 0x7f31781092a0 secure :-1 s=READY pgs=527 cs=0 l=1 rev1=1 crypto rx=0x7f3170009ca0 tx=0x7f317002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:49.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.446+0000 7f317c952640 1 -- 172.21.15.86:0/3993978131 shutdown_connections 2024-04-21T08:38:49.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.446+0000 7f317c952640 1 --2- 172.21.15.86:0/3993978131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3178102980 0x7f31781092a0 unknown :-1 s=CLOSED pgs=527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:49.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.446+0000 7f317c952640 1 --2- 172.21.15.86:0/3993978131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3178101fc0 0x7f3178102440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:49.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.446+0000 7f317c952640 1 --2- 172.21.15.86:0/3993978131 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3178100e70 0x7f3178101290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:49.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.446+0000 7f317c952640 1 -- 172.21.15.86:0/3993978131 >> 172.21.15.86:0/3993978131 conn(0x7f31780fc3e0 msgr2=0x7f31780fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:49.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.446+0000 7f317c952640 1 -- 172.21.15.86:0/3993978131 shutdown_connections 2024-04-21T08:38:49.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.446+0000 7f317c952640 1 -- 172.21.15.86:0/3993978131 wait complete. 2024-04-21T08:38:49.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.447+0000 7f317c952640 1 Processor -- start 2024-04-21T08:38:49.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.447+0000 7f317c952640 1 -- start start 2024-04-21T08:38:49.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.447+0000 7f317c952640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3178100e70 0x7f3178198160 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:49.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.447+0000 7f317c952640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3178101fc0 0x7f31781986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:49.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.447+0000 7f317c952640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3178102980 0x7f317819f710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:49.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.447+0000 7f317c952640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f317810c9d0 con 0x7f3178100e70 2024-04-21T08:38:49.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.447+0000 7f317c952640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f317810c850 con 0x7f3178102980 2024-04-21T08:38:49.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.447+0000 7f317c952640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f317810cb50 con 0x7f3178101fc0 2024-04-21T08:38:49.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.447+0000 7f317659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3178100e70 0x7f3178198160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:49.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.448+0000 7f317659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3178100e70 0x7f3178198160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:57658/0 (socket says 172.21.15.86:57658) 2024-04-21T08:38:49.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.448+0000 7f317659c640 1 -- 172.21.15.86:0/2534227838 learned_addr learned my addr 172.21.15.86:0/2534227838 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:49.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.448+0000 7f3176d9d640 1 --2- 172.21.15.86:0/2534227838 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3178102980 0x7f317819f710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:49.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.448+0000 7f3175d9b640 1 --2- 172.21.15.86:0/2534227838 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3178101fc0 0x7f31781986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:49.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.448+0000 7f317659c640 1 -- 172.21.15.86:0/2534227838 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3178101fc0 msgr2=0x7f31781986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:49.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.448+0000 7f317659c640 1 --2- 172.21.15.86:0/2534227838 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3178101fc0 0x7f31781986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:49.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.448+0000 7f317659c640 1 -- 172.21.15.86:0/2534227838 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3178102980 msgr2=0x7f317819f710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:49.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.448+0000 7f317659c640 1 --2- 172.21.15.86:0/2534227838 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3178102980 0x7f317819f710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:49.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.448+0000 7f317659c640 1 -- 172.21.15.86:0/2534227838 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f317819fd80 con 0x7f3178100e70 2024-04-21T08:38:49.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.448+0000 7f317659c640 1 --2- 172.21.15.86:0/2534227838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3178100e70 0x7f3178198160 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f316c00e9f0 tx=0x7f316c00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:49.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.448+0000 7f31677fe640 1 -- 172.21.15.86:0/2534227838 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c00cd80 con 0x7f3178100e70 2024-04-21T08:38:49.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.448+0000 7f31677fe640 1 -- 172.21.15.86:0/2534227838 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f316c0108e0 con 0x7f3178100e70 2024-04-21T08:38:49.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.448+0000 7f317c952640 1 -- 172.21.15.86:0/2534227838 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31781a0070 con 0x7f3178100e70 2024-04-21T08:38:49.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.448+0000 7f31677fe640 1 -- 172.21.15.86:0/2534227838 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f316c018980 con 0x7f3178100e70 2024-04-21T08:38:49.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.448+0000 7f317c952640 1 -- 172.21.15.86:0/2534227838 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31781a0610 con 0x7f3178100e70 2024-04-21T08:38:49.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.449+0000 7f317c952640 1 -- 172.21.15.86:0/2534227838 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f31781a0870 con 0x7f3178100e70 2024-04-21T08:38:49.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.451+0000 7f31677fe640 1 -- 172.21.15.86:0/2534227838 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7f316c010430 con 0x7f3178100e70 2024-04-21T08:38:49.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.451+0000 7f31677fe640 1 --2- 172.21.15.86:0/2534227838 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f31600a3da0 0x7f31600a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:49.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.452+0000 7f3175d9b640 1 --2- 172.21.15.86:0/2534227838 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f31600a3da0 0x7f31600a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:49.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.452+0000 7f31677fe640 1 -- 172.21.15.86:0/2534227838 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f316c0d0e30 con 0x7f3178100e70 2024-04-21T08:38:49.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.452+0000 7f3175d9b640 1 --2- 172.21.15.86:0/2534227838 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f31600a3da0 0x7f31600a6260 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3178199670 tx=0x7f316800b390 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:49.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.456+0000 7f31677fe640 1 -- 172.21.15.86:0/2534227838 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f316c09cb10 con 0x7f3178100e70 2024-04-21T08:38:49.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.638+0000 7f317c952640 1 -- 172.21.15.86:0/2534227838 --> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f31780fe590 con 0x7f31600a3da0 2024-04-21T08:38:49.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.650+0000 7f31677fe640 1 -- 172.21.15.86:0/2534227838 <== mgr.15558 v2:172.21.15.86:6800/2452032403 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339464 (secure 0 0 0) 0x7f31780fe590 con 0x7f31600a3da0 2024-04-21T08:38:49.653 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:38:49.655 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-21T08:38:49.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.653+0000 7f317c952640 1 -- 172.21.15.86:0/2534227838 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f31600a3da0 msgr2=0x7f31600a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:49.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.653+0000 7f317c952640 1 --2- 172.21.15.86:0/2534227838 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f31600a3da0 0x7f31600a6260 secure :-1 s=READY pgs=41 cs=0 l=1 rev1=1 crypto rx=0x7f3178199670 tx=0x7f316800b390 comp rx=0 tx=0).stop 2024-04-21T08:38:49.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.653+0000 7f317c952640 1 -- 172.21.15.86:0/2534227838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3178100e70 msgr2=0x7f3178198160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:49.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.654+0000 7f317c952640 1 --2- 172.21.15.86:0/2534227838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3178100e70 0x7f3178198160 secure :-1 s=READY pgs=528 cs=0 l=1 rev1=1 crypto rx=0x7f316c00e9f0 tx=0x7f316c00eeb0 comp rx=0 tx=0).stop 2024-04-21T08:38:49.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.654+0000 7f317c952640 1 -- 172.21.15.86:0/2534227838 shutdown_connections 2024-04-21T08:38:49.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.654+0000 7f317c952640 1 --2- 172.21.15.86:0/2534227838 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f31600a3da0 0x7f31600a6260 unknown :-1 s=CLOSED pgs=41 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:49.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.654+0000 7f317c952640 1 --2- 172.21.15.86:0/2534227838 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3178102980 0x7f317819f710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:49.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.654+0000 7f317c952640 1 --2- 172.21.15.86:0/2534227838 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3178101fc0 0x7f31781986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:49.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.654+0000 7f317c952640 1 --2- 172.21.15.86:0/2534227838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3178100e70 0x7f3178198160 unknown :-1 s=CLOSED pgs=528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:49.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.654+0000 7f317c952640 1 -- 172.21.15.86:0/2534227838 >> 172.21.15.86:0/2534227838 conn(0x7f31780fc3e0 msgr2=0x7f31780fde80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:49.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.654+0000 7f317c952640 1 -- 172.21.15.86:0/2534227838 shutdown_connections 2024-04-21T08:38:49.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.654+0000 7f317c952640 1 -- 172.21.15.86:0/2534227838 wait complete. 2024-04-21T08:38:49.664 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-04-21T08:38:48.652229+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1194594,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1486,"num_read_kb":910,"num_write":558,"num_write_kb":4413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":1181659,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":944,"ondisk_log_size":944,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":386032,"kb_used_data":15860,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124555792,"statfs":{"total":1151940427776,"available":1151545131008,"internally_reserved":0,"allocated":16240640,"data_stored":9386888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":378713404},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.992077"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807854+0000","last_change":"2024-04-21T08:36:20.913421+0000","last_active":"2024-04-21T08:38:46.807854+0000","last_peered":"2024-04-21T08:38:46.807854+0000","last_clean":"2024-04-21T08:38:46.807854+0000","last_became_active":"2024-04-21T08:36:20.913017+0000","last_became_peered":"2024-04-21T08:36:20.913017+0000","last_unstale":"2024-04-21T08:38:46.807854+0000","last_undegraded":"2024-04-21T08:38:46.807854+0000","last_fullsized":"2024-04-21T08:38:46.807854+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:16:29.885916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.915359+0000","last_change":"2024-04-21T08:36:20.911675+0000","last_active":"2024-04-21T08:38:46.915359+0000","last_peered":"2024-04-21T08:38:46.915359+0000","last_clean":"2024-04-21T08:38:46.915359+0000","last_became_active":"2024-04-21T08:36:20.910591+0000","last_became_peered":"2024-04-21T08:36:20.910591+0000","last_unstale":"2024-04-21T08:38:46.915359+0000","last_undegraded":"2024-04-21T08:38:46.915359+0000","last_fullsized":"2024-04-21T08:38:46.915359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:01:40.735376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.915070+0000","last_change":"2024-04-21T08:36:20.911978+0000","last_active":"2024-04-21T08:38:46.915070+0000","last_peered":"2024-04-21T08:38:46.915070+0000","last_clean":"2024-04-21T08:38:46.915070+0000","last_became_active":"2024-04-21T08:36:20.911641+0000","last_became_peered":"2024-04-21T08:36:20.911641+0000","last_unstale":"2024-04-21T08:38:46.915070+0000","last_undegraded":"2024-04-21T08:38:46.915070+0000","last_fullsized":"2024-04-21T08:38:46.915070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:14:23.789790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911901+0000","last_change":"2024-04-21T08:36:20.919027+0000","last_active":"2024-04-21T08:38:46.911901+0000","last_peered":"2024-04-21T08:38:46.911901+0000","last_clean":"2024-04-21T08:38:46.911901+0000","last_became_active":"2024-04-21T08:36:20.918740+0000","last_became_peered":"2024-04-21T08:36:20.918740+0000","last_unstale":"2024-04-21T08:38:46.911901+0000","last_undegraded":"2024-04-21T08:38:46.911901+0000","last_fullsized":"2024-04-21T08:38:46.911901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:00:31.066938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655044+0000","last_change":"2024-04-21T08:36:20.907175+0000","last_active":"2024-04-21T08:38:46.655044+0000","last_peered":"2024-04-21T08:38:46.655044+0000","last_clean":"2024-04-21T08:38:46.655044+0000","last_became_active":"2024-04-21T08:36:20.906818+0000","last_became_peered":"2024-04-21T08:36:20.906818+0000","last_unstale":"2024-04-21T08:38:46.655044+0000","last_undegraded":"2024-04-21T08:38:46.655044+0000","last_fullsized":"2024-04-21T08:38:46.655044+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:41:03.332231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638216+0000","last_change":"2024-04-21T08:36:20.905968+0000","last_active":"2024-04-21T08:38:46.638216+0000","last_peered":"2024-04-21T08:38:46.638216+0000","last_clean":"2024-04-21T08:38:46.638216+0000","last_became_active":"2024-04-21T08:36:20.904840+0000","last_became_peered":"2024-04-21T08:36:20.904840+0000","last_unstale":"2024-04-21T08:38:46.638216+0000","last_undegraded":"2024-04-21T08:38:46.638216+0000","last_fullsized":"2024-04-21T08:38:46.638216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:23:46.886241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913568+0000","last_change":"2024-04-21T08:36:20.912045+0000","last_active":"2024-04-21T08:38:46.913568+0000","last_peered":"2024-04-21T08:38:46.913568+0000","last_clean":"2024-04-21T08:38:46.913568+0000","last_became_active":"2024-04-21T08:36:20.911690+0000","last_became_peered":"2024-04-21T08:36:20.911690+0000","last_unstale":"2024-04-21T08:38:46.913568+0000","last_undegraded":"2024-04-21T08:38:46.913568+0000","last_fullsized":"2024-04-21T08:38:46.913568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:06:28.229307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638612+0000","last_change":"2024-04-21T08:36:20.904874+0000","last_active":"2024-04-21T08:38:46.638612+0000","last_peered":"2024-04-21T08:38:46.638612+0000","last_clean":"2024-04-21T08:38:46.638612+0000","last_became_active":"2024-04-21T08:36:20.904217+0000","last_became_peered":"2024-04-21T08:36:20.904217+0000","last_unstale":"2024-04-21T08:38:46.638612+0000","last_undegraded":"2024-04-21T08:38:46.638612+0000","last_fullsized":"2024-04-21T08:38:46.638612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:37:04.423812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.923169+0000","last_change":"2024-04-21T08:36:20.912874+0000","last_active":"2024-04-21T08:38:46.923169+0000","last_peered":"2024-04-21T08:38:46.923169+0000","last_clean":"2024-04-21T08:38:46.923169+0000","last_became_active":"2024-04-21T08:36:20.912569+0000","last_became_peered":"2024-04-21T08:36:20.912569+0000","last_unstale":"2024-04-21T08:38:46.923169+0000","last_undegraded":"2024-04-21T08:38:46.923169+0000","last_fullsized":"2024-04-21T08:38:46.923169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:55:20.763157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":52,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638497+0000","last_change":"2024-04-21T08:37:24.296916+0000","last_active":"2024-04-21T08:38:46.638497+0000","last_peered":"2024-04-21T08:38:46.638497+0000","last_clean":"2024-04-21T08:38:46.638497+0000","last_became_active":"2024-04-21T08:37:24.296547+0000","last_became_peered":"2024-04-21T08:37:24.296547+0000","last_unstale":"2024-04-21T08:38:46.638497+0000","last_undegraded":"2024-04-21T08:38:46.638497+0000","last_fullsized":"2024-04-21T08:38:46.638497+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:47:56.683793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914246+0000","last_change":"2024-04-21T08:36:20.909094+0000","last_active":"2024-04-21T08:38:46.914246+0000","last_peered":"2024-04-21T08:38:46.914246+0000","last_clean":"2024-04-21T08:38:46.914246+0000","last_became_active":"2024-04-21T08:36:20.908634+0000","last_became_peered":"2024-04-21T08:36:20.908634+0000","last_unstale":"2024-04-21T08:38:46.914246+0000","last_undegraded":"2024-04-21T08:38:46.914246+0000","last_fullsized":"2024-04-21T08:38:46.914246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:28:38.450621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911748+0000","last_change":"2024-04-21T08:36:20.909429+0000","last_active":"2024-04-21T08:38:46.911748+0000","last_peered":"2024-04-21T08:38:46.911748+0000","last_clean":"2024-04-21T08:38:46.911748+0000","last_became_active":"2024-04-21T08:36:20.909081+0000","last_became_peered":"2024-04-21T08:36:20.909081+0000","last_unstale":"2024-04-21T08:38:46.911748+0000","last_undegraded":"2024-04-21T08:38:46.911748+0000","last_fullsized":"2024-04-21T08:38:46.911748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:45:46.939125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913919+0000","last_change":"2024-04-21T08:36:20.895894+0000","last_active":"2024-04-21T08:38:46.913919+0000","last_peered":"2024-04-21T08:38:46.913919+0000","last_clean":"2024-04-21T08:38:46.913919+0000","last_became_active":"2024-04-21T08:36:20.895572+0000","last_became_peered":"2024-04-21T08:36:20.895572+0000","last_unstale":"2024-04-21T08:38:46.913919+0000","last_undegraded":"2024-04-21T08:38:46.913919+0000","last_fullsized":"2024-04-21T08:38:46.913919+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:12:48.750555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914638+0000","last_change":"2024-04-21T08:36:20.905356+0000","last_active":"2024-04-21T08:38:46.914638+0000","last_peered":"2024-04-21T08:38:46.914638+0000","last_clean":"2024-04-21T08:38:46.914638+0000","last_became_active":"2024-04-21T08:36:20.904954+0000","last_became_peered":"2024-04-21T08:36:20.904954+0000","last_unstale":"2024-04-21T08:38:46.914638+0000","last_undegraded":"2024-04-21T08:38:46.914638+0000","last_fullsized":"2024-04-21T08:38:46.914638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:45:49.116482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.915244+0000","last_change":"2024-04-21T08:36:20.911550+0000","last_active":"2024-04-21T08:38:46.915244+0000","last_peered":"2024-04-21T08:38:46.915244+0000","last_clean":"2024-04-21T08:38:46.915244+0000","last_became_active":"2024-04-21T08:36:20.910557+0000","last_became_peered":"2024-04-21T08:36:20.910557+0000","last_unstale":"2024-04-21T08:38:46.915244+0000","last_undegraded":"2024-04-21T08:38:46.915244+0000","last_fullsized":"2024-04-21T08:38:46.915244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:09:49.422729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.922413+0000","last_change":"2024-04-21T08:36:20.903954+0000","last_active":"2024-04-21T08:38:46.922413+0000","last_peered":"2024-04-21T08:38:46.922413+0000","last_clean":"2024-04-21T08:38:46.922413+0000","last_became_active":"2024-04-21T08:36:20.903641+0000","last_became_peered":"2024-04-21T08:36:20.903641+0000","last_unstale":"2024-04-21T08:38:46.922413+0000","last_undegraded":"2024-04-21T08:38:46.922413+0000","last_fullsized":"2024-04-21T08:38:46.922413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:08:55.678940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807825+0000","last_change":"2024-04-21T08:36:20.914256+0000","last_active":"2024-04-21T08:38:46.807825+0000","last_peered":"2024-04-21T08:38:46.807825+0000","last_clean":"2024-04-21T08:38:46.807825+0000","last_became_active":"2024-04-21T08:36:20.913858+0000","last_became_peered":"2024-04-21T08:36:20.913858+0000","last_unstale":"2024-04-21T08:38:46.807825+0000","last_undegraded":"2024-04-21T08:38:46.807825+0000","last_fullsized":"2024-04-21T08:38:46.807825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:01:10.276763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.743080+0000","last_change":"2024-04-21T08:36:20.904138+0000","last_active":"2024-04-21T08:38:46.743080+0000","last_peered":"2024-04-21T08:38:46.743080+0000","last_clean":"2024-04-21T08:38:46.743080+0000","last_became_active":"2024-04-21T08:36:20.903670+0000","last_became_peered":"2024-04-21T08:36:20.903670+0000","last_unstale":"2024-04-21T08:38:46.743080+0000","last_undegraded":"2024-04-21T08:38:46.743080+0000","last_fullsized":"2024-04-21T08:38:46.743080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:38:50.160867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639071+0000","last_change":"2024-04-21T08:36:20.910798+0000","last_active":"2024-04-21T08:38:46.639071+0000","last_peered":"2024-04-21T08:38:46.639071+0000","last_clean":"2024-04-21T08:38:46.639071+0000","last_became_active":"2024-04-21T08:36:20.910411+0000","last_became_peered":"2024-04-21T08:36:20.910411+0000","last_unstale":"2024-04-21T08:38:46.639071+0000","last_undegraded":"2024-04-21T08:38:46.639071+0000","last_fullsized":"2024-04-21T08:38:46.639071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:13:12.109818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638623+0000","last_change":"2024-04-21T08:36:24.924831+0000","last_active":"2024-04-21T08:38:46.638623+0000","last_peered":"2024-04-21T08:38:46.638623+0000","last_clean":"2024-04-21T08:38:46.638623+0000","last_became_active":"2024-04-21T08:36:24.924435+0000","last_became_peered":"2024-04-21T08:36:24.924435+0000","last_unstale":"2024-04-21T08:38:46.638623+0000","last_undegraded":"2024-04-21T08:38:46.638623+0000","last_fullsized":"2024-04-21T08:38:46.638623+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:23.447079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":188,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.743359+0000","last_change":"2024-04-21T08:37:24.308024+0000","last_active":"2024-04-21T08:38:46.743359+0000","last_peered":"2024-04-21T08:38:46.743359+0000","last_clean":"2024-04-21T08:38:46.743359+0000","last_became_active":"2024-04-21T08:37:24.297491+0000","last_became_peered":"2024-04-21T08:37:24.297491+0000","last_unstale":"2024-04-21T08:38:46.743359+0000","last_undegraded":"2024-04-21T08:38:46.743359+0000","last_fullsized":"2024-04-21T08:38:46.743359+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:48:10.039545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639158+0000","last_change":"2024-04-21T08:36:20.912851+0000","last_active":"2024-04-21T08:38:46.639158+0000","last_peered":"2024-04-21T08:38:46.639158+0000","last_clean":"2024-04-21T08:38:46.639158+0000","last_became_active":"2024-04-21T08:36:20.912407+0000","last_became_peered":"2024-04-21T08:36:20.912407+0000","last_unstale":"2024-04-21T08:38:46.639158+0000","last_undegraded":"2024-04-21T08:38:46.639158+0000","last_fullsized":"2024-04-21T08:38:46.639158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:45:18.026940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.910950+0000","last_change":"2024-04-21T08:36:24.924052+0000","last_active":"2024-04-21T08:38:46.910950+0000","last_peered":"2024-04-21T08:38:46.910950+0000","last_clean":"2024-04-21T08:38:46.910950+0000","last_became_active":"2024-04-21T08:36:24.923618+0000","last_became_peered":"2024-04-21T08:36:24.923618+0000","last_unstale":"2024-04-21T08:38:46.910950+0000","last_undegraded":"2024-04-21T08:38:46.910950+0000","last_fullsized":"2024-04-21T08:38:46.910950+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:08:18.182386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639180+0000","last_change":"2024-04-21T08:36:20.285291+0000","last_active":"2024-04-21T08:38:46.639180+0000","last_peered":"2024-04-21T08:38:46.639180+0000","last_clean":"2024-04-21T08:38:46.639180+0000","last_became_active":"2024-04-21T08:36:20.284969+0000","last_became_peered":"2024-04-21T08:36:20.284969+0000","last_unstale":"2024-04-21T08:38:46.639180+0000","last_undegraded":"2024-04-21T08:38:46.639180+0000","last_fullsized":"2024-04-21T08:38:46.639180+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:55:29.484439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.915781+0000","last_change":"2024-04-21T08:36:20.902819+0000","last_active":"2024-04-21T08:38:46.915781+0000","last_peered":"2024-04-21T08:38:46.915781+0000","last_clean":"2024-04-21T08:38:46.915781+0000","last_became_active":"2024-04-21T08:36:20.902466+0000","last_became_peered":"2024-04-21T08:36:20.902466+0000","last_unstale":"2024-04-21T08:38:46.915781+0000","last_undegraded":"2024-04-21T08:38:46.915781+0000","last_fullsized":"2024-04-21T08:38:46.915781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:29:40.454026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911156+0000","last_change":"2024-04-21T08:36:24.926480+0000","last_active":"2024-04-21T08:38:46.911156+0000","last_peered":"2024-04-21T08:38:46.911156+0000","last_clean":"2024-04-21T08:38:46.911156+0000","last_became_active":"2024-04-21T08:36:24.926116+0000","last_became_peered":"2024-04-21T08:36:24.926116+0000","last_unstale":"2024-04-21T08:38:46.911156+0000","last_undegraded":"2024-04-21T08:38:46.911156+0000","last_fullsized":"2024-04-21T08:38:46.911156+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:43:18.839396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655562+0000","last_change":"2024-04-21T08:36:20.293086+0000","last_active":"2024-04-21T08:38:46.655562+0000","last_peered":"2024-04-21T08:38:46.655562+0000","last_clean":"2024-04-21T08:38:46.655562+0000","last_became_active":"2024-04-21T08:36:20.291178+0000","last_became_peered":"2024-04-21T08:36:20.291178+0000","last_unstale":"2024-04-21T08:38:46.655562+0000","last_undegraded":"2024-04-21T08:38:46.655562+0000","last_fullsized":"2024-04-21T08:38:46.655562+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:02:32.935930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.922727+0000","last_change":"2024-04-21T08:36:20.909374+0000","last_active":"2024-04-21T08:38:46.922727+0000","last_peered":"2024-04-21T08:38:46.922727+0000","last_clean":"2024-04-21T08:38:46.922727+0000","last_became_active":"2024-04-21T08:36:20.909043+0000","last_became_peered":"2024-04-21T08:36:20.909043+0000","last_unstale":"2024-04-21T08:38:46.922727+0000","last_undegraded":"2024-04-21T08:38:46.922727+0000","last_fullsized":"2024-04-21T08:38:46.922727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:45:45.719942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911098+0000","last_change":"2024-04-21T08:38:24.623194+0000","last_active":"2024-04-21T08:38:46.911098+0000","last_peered":"2024-04-21T08:38:46.911098+0000","last_clean":"2024-04-21T08:38:46.911098+0000","last_became_active":"2024-04-21T08:38:24.622632+0000","last_became_peered":"2024-04-21T08:38:24.622632+0000","last_unstale":"2024-04-21T08:38:46.911098+0000","last_undegraded":"2024-04-21T08:38:46.911098+0000","last_fullsized":"2024-04-21T08:38:46.911098+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:26:39.752161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655409+0000","last_change":"2024-04-21T08:36:20.288644+0000","last_active":"2024-04-21T08:38:46.655409+0000","last_peered":"2024-04-21T08:38:46.655409+0000","last_clean":"2024-04-21T08:38:46.655409+0000","last_became_active":"2024-04-21T08:36:20.288069+0000","last_became_peered":"2024-04-21T08:36:20.288069+0000","last_unstale":"2024-04-21T08:38:46.655409+0000","last_undegraded":"2024-04-21T08:38:46.655409+0000","last_fullsized":"2024-04-21T08:38:46.655409+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:37:36.522713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751376+0000","last_change":"2024-04-21T08:36:20.908505+0000","last_active":"2024-04-21T08:38:46.751376+0000","last_peered":"2024-04-21T08:38:46.751376+0000","last_clean":"2024-04-21T08:38:46.751376+0000","last_became_active":"2024-04-21T08:36:20.908163+0000","last_became_peered":"2024-04-21T08:36:20.908163+0000","last_unstale":"2024-04-21T08:38:46.751376+0000","last_undegraded":"2024-04-21T08:38:46.751376+0000","last_fullsized":"2024-04-21T08:38:46.751376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:54:18.530203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639160+0000","last_change":"2024-04-21T08:36:24.919428+0000","last_active":"2024-04-21T08:38:46.639160+0000","last_peered":"2024-04-21T08:38:46.639160+0000","last_clean":"2024-04-21T08:38:46.639160+0000","last_became_active":"2024-04-21T08:36:24.918722+0000","last_became_peered":"2024-04-21T08:36:24.918722+0000","last_unstale":"2024-04-21T08:38:46.639160+0000","last_undegraded":"2024-04-21T08:38:46.639160+0000","last_fullsized":"2024-04-21T08:38:46.639160+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:30:32.369551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"93'8","reported_seq":168,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751405+0000","last_change":"2024-04-21T08:36:20.287642+0000","last_active":"2024-04-21T08:38:46.751405+0000","last_peered":"2024-04-21T08:38:46.751405+0000","last_clean":"2024-04-21T08:38:46.751405+0000","last_became_active":"2024-04-21T08:36:20.287296+0000","last_became_peered":"2024-04-21T08:36:20.287296+0000","last_unstale":"2024-04-21T08:38:46.751405+0000","last_undegraded":"2024-04-21T08:38:46.751405+0000","last_fullsized":"2024-04-21T08:38:46.751405+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:59:28.579020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1462,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655081+0000","last_change":"2024-04-21T08:36:20.916971+0000","last_active":"2024-04-21T08:38:46.655081+0000","last_peered":"2024-04-21T08:38:46.655081+0000","last_clean":"2024-04-21T08:38:46.655081+0000","last_became_active":"2024-04-21T08:36:20.916622+0000","last_became_peered":"2024-04-21T08:36:20.916622+0000","last_unstale":"2024-04-21T08:38:46.655081+0000","last_undegraded":"2024-04-21T08:38:46.655081+0000","last_fullsized":"2024-04-21T08:38:46.655081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:49:38.858772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638856+0000","last_change":"2024-04-21T08:36:24.924170+0000","last_active":"2024-04-21T08:38:46.638856+0000","last_peered":"2024-04-21T08:38:46.638856+0000","last_clean":"2024-04-21T08:38:46.638856+0000","last_became_active":"2024-04-21T08:36:24.923744+0000","last_became_peered":"2024-04-21T08:36:24.923744+0000","last_unstale":"2024-04-21T08:38:46.638856+0000","last_undegraded":"2024-04-21T08:38:46.638856+0000","last_fullsized":"2024-04-21T08:38:46.638856+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:23:24.605021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638388+0000","last_change":"2024-04-21T08:36:20.286545+0000","last_active":"2024-04-21T08:38:46.638388+0000","last_peered":"2024-04-21T08:38:46.638388+0000","last_clean":"2024-04-21T08:38:46.638388+0000","last_became_active":"2024-04-21T08:36:20.286197+0000","last_became_peered":"2024-04-21T08:36:20.286197+0000","last_unstale":"2024-04-21T08:38:46.638388+0000","last_undegraded":"2024-04-21T08:38:46.638388+0000","last_fullsized":"2024-04-21T08:38:46.638388+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:27:55.430428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751488+0000","last_change":"2024-04-21T08:36:20.907743+0000","last_active":"2024-04-21T08:38:46.751488+0000","last_peered":"2024-04-21T08:38:46.751488+0000","last_clean":"2024-04-21T08:38:46.751488+0000","last_became_active":"2024-04-21T08:36:20.907163+0000","last_became_peered":"2024-04-21T08:36:20.907163+0000","last_unstale":"2024-04-21T08:38:46.751488+0000","last_undegraded":"2024-04-21T08:38:46.751488+0000","last_fullsized":"2024-04-21T08:38:46.751488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:41:00.172353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807766+0000","last_change":"2024-04-21T08:37:24.301529+0000","last_active":"2024-04-21T08:38:46.807766+0000","last_peered":"2024-04-21T08:38:46.807766+0000","last_clean":"2024-04-21T08:38:46.807766+0000","last_became_active":"2024-04-21T08:37:24.301014+0000","last_became_peered":"2024-04-21T08:37:24.301014+0000","last_unstale":"2024-04-21T08:38:46.807766+0000","last_undegraded":"2024-04-21T08:38:46.807766+0000","last_fullsized":"2024-04-21T08:38:46.807766+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:58:20.088501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911966+0000","last_change":"2024-04-21T08:36:20.283143+0000","last_active":"2024-04-21T08:38:46.911966+0000","last_peered":"2024-04-21T08:38:46.911966+0000","last_clean":"2024-04-21T08:38:46.911966+0000","last_became_active":"2024-04-21T08:36:20.282585+0000","last_became_peered":"2024-04-21T08:36:20.282585+0000","last_unstale":"2024-04-21T08:38:46.911966+0000","last_undegraded":"2024-04-21T08:38:46.911966+0000","last_fullsized":"2024-04-21T08:38:46.911966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:12:37.588764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638488+0000","last_change":"2024-04-21T08:36:20.912591+0000","last_active":"2024-04-21T08:38:46.638488+0000","last_peered":"2024-04-21T08:38:46.638488+0000","last_clean":"2024-04-21T08:38:46.638488+0000","last_became_active":"2024-04-21T08:36:20.912174+0000","last_became_peered":"2024-04-21T08:36:20.912174+0000","last_unstale":"2024-04-21T08:38:46.638488+0000","last_undegraded":"2024-04-21T08:38:46.638488+0000","last_fullsized":"2024-04-21T08:38:46.638488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:33:58.949506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.915307+0000","last_change":"2024-04-21T08:36:24.915998+0000","last_active":"2024-04-21T08:38:46.915307+0000","last_peered":"2024-04-21T08:38:46.915307+0000","last_clean":"2024-04-21T08:38:46.915307+0000","last_became_active":"2024-04-21T08:36:24.915621+0000","last_became_peered":"2024-04-21T08:36:24.915621+0000","last_unstale":"2024-04-21T08:38:46.915307+0000","last_undegraded":"2024-04-21T08:38:46.915307+0000","last_fullsized":"2024-04-21T08:38:46.915307+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:43:05.117181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751087+0000","last_change":"2024-04-21T08:36:20.286308+0000","last_active":"2024-04-21T08:38:46.751087+0000","last_peered":"2024-04-21T08:38:46.751087+0000","last_clean":"2024-04-21T08:38:46.751087+0000","last_became_active":"2024-04-21T08:36:20.285975+0000","last_became_peered":"2024-04-21T08:36:20.285975+0000","last_unstale":"2024-04-21T08:38:46.751087+0000","last_undegraded":"2024-04-21T08:38:46.751087+0000","last_fullsized":"2024-04-21T08:38:46.751087+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:55:10.658349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.742770+0000","last_change":"2024-04-21T08:36:20.900502+0000","last_active":"2024-04-21T08:38:46.742770+0000","last_peered":"2024-04-21T08:38:46.742770+0000","last_clean":"2024-04-21T08:38:46.742770+0000","last_became_active":"2024-04-21T08:36:20.900199+0000","last_became_peered":"2024-04-21T08:36:20.900199+0000","last_unstale":"2024-04-21T08:38:46.742770+0000","last_undegraded":"2024-04-21T08:38:46.742770+0000","last_fullsized":"2024-04-21T08:38:46.742770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:17:00.720918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638804+0000","last_change":"2024-04-21T08:36:24.920991+0000","last_active":"2024-04-21T08:38:46.638804+0000","last_peered":"2024-04-21T08:38:46.638804+0000","last_clean":"2024-04-21T08:38:46.638804+0000","last_became_active":"2024-04-21T08:36:24.916611+0000","last_became_peered":"2024-04-21T08:36:24.916611+0000","last_unstale":"2024-04-21T08:38:46.638804+0000","last_undegraded":"2024-04-21T08:38:46.638804+0000","last_fullsized":"2024-04-21T08:38:46.638804+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:01:01.412127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"93'7","reported_seq":283,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:47.633155+0000","last_change":"2024-04-21T08:36:20.284966+0000","last_active":"2024-04-21T08:38:47.633155+0000","last_peered":"2024-04-21T08:38:47.633155+0000","last_clean":"2024-04-21T08:38:47.633155+0000","last_became_active":"2024-04-21T08:36:20.284612+0000","last_became_peered":"2024-04-21T08:36:20.284612+0000","last_unstale":"2024-04-21T08:38:47.633155+0000","last_undegraded":"2024-04-21T08:38:47.633155+0000","last_fullsized":"2024-04-21T08:38:47.633155+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:54:50.095013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":243,"num_read_kb":243,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.923175+0000","last_change":"2024-04-21T08:36:20.903493+0000","last_active":"2024-04-21T08:38:46.923175+0000","last_peered":"2024-04-21T08:38:46.923175+0000","last_clean":"2024-04-21T08:38:46.923175+0000","last_became_active":"2024-04-21T08:36:20.902752+0000","last_became_peered":"2024-04-21T08:36:20.902752+0000","last_unstale":"2024-04-21T08:38:46.923175+0000","last_undegraded":"2024-04-21T08:38:46.923175+0000","last_fullsized":"2024-04-21T08:38:46.923175+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:40.684264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.742602+0000","last_change":"2024-04-21T08:36:24.920505+0000","last_active":"2024-04-21T08:38:46.742602+0000","last_peered":"2024-04-21T08:38:46.742602+0000","last_clean":"2024-04-21T08:38:46.742602+0000","last_became_active":"2024-04-21T08:36:24.919966+0000","last_became_peered":"2024-04-21T08:36:24.919966+0000","last_unstale":"2024-04-21T08:38:46.742602+0000","last_undegraded":"2024-04-21T08:38:46.742602+0000","last_fullsized":"2024-04-21T08:38:46.742602+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:58:05.131815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914242+0000","last_change":"2024-04-21T08:36:20.281720+0000","last_active":"2024-04-21T08:38:46.914242+0000","last_peered":"2024-04-21T08:38:46.914242+0000","last_clean":"2024-04-21T08:38:46.914242+0000","last_became_active":"2024-04-21T08:36:20.279862+0000","last_became_peered":"2024-04-21T08:36:20.279862+0000","last_unstale":"2024-04-21T08:38:46.914242+0000","last_undegraded":"2024-04-21T08:38:46.914242+0000","last_fullsized":"2024-04-21T08:38:46.914242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:18:58.106532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.916030+0000","last_change":"2024-04-21T08:36:20.908520+0000","last_active":"2024-04-21T08:38:46.916030+0000","last_peered":"2024-04-21T08:38:46.916030+0000","last_clean":"2024-04-21T08:38:46.916030+0000","last_became_active":"2024-04-21T08:36:20.908184+0000","last_became_peered":"2024-04-21T08:36:20.908184+0000","last_unstale":"2024-04-21T08:38:46.916030+0000","last_undegraded":"2024-04-21T08:38:46.916030+0000","last_fullsized":"2024-04-21T08:38:46.916030+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:14:33.895864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911416+0000","last_change":"2024-04-21T08:36:24.920697+0000","last_active":"2024-04-21T08:38:46.911416+0000","last_peered":"2024-04-21T08:38:46.911416+0000","last_clean":"2024-04-21T08:38:46.911416+0000","last_became_active":"2024-04-21T08:36:24.919976+0000","last_became_peered":"2024-04-21T08:36:24.919976+0000","last_unstale":"2024-04-21T08:38:46.911416+0000","last_undegraded":"2024-04-21T08:38:46.911416+0000","last_fullsized":"2024-04-21T08:38:46.911416+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:40:09.238887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751422+0000","last_change":"2024-04-21T08:36:20.283531+0000","last_active":"2024-04-21T08:38:46.751422+0000","last_peered":"2024-04-21T08:38:46.751422+0000","last_clean":"2024-04-21T08:38:46.751422+0000","last_became_active":"2024-04-21T08:36:20.283159+0000","last_became_peered":"2024-04-21T08:36:20.283159+0000","last_unstale":"2024-04-21T08:38:46.751422+0000","last_undegraded":"2024-04-21T08:38:46.751422+0000","last_fullsized":"2024-04-21T08:38:46.751422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:57:02.917537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.922897+0000","last_change":"2024-04-21T08:36:20.910113+0000","last_active":"2024-04-21T08:38:46.922897+0000","last_peered":"2024-04-21T08:38:46.922897+0000","last_clean":"2024-04-21T08:38:46.922897+0000","last_became_active":"2024-04-21T08:36:20.909744+0000","last_became_peered":"2024-04-21T08:36:20.909744+0000","last_unstale":"2024-04-21T08:38:46.922897+0000","last_undegraded":"2024-04-21T08:38:46.922897+0000","last_fullsized":"2024-04-21T08:38:46.922897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:16:14.568318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.742678+0000","last_change":"2024-04-21T08:36:24.922232+0000","last_active":"2024-04-21T08:38:46.742678+0000","last_peered":"2024-04-21T08:38:46.742678+0000","last_clean":"2024-04-21T08:38:46.742678+0000","last_became_active":"2024-04-21T08:36:24.921690+0000","last_became_peered":"2024-04-21T08:36:24.921690+0000","last_unstale":"2024-04-21T08:38:46.742678+0000","last_undegraded":"2024-04-21T08:38:46.742678+0000","last_fullsized":"2024-04-21T08:38:46.742678+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:21:42.580203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'42","reported_seq":332,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:47.634217+0000","last_change":"2024-04-21T08:36:20.283574+0000","last_active":"2024-04-21T08:38:47.634217+0000","last_peered":"2024-04-21T08:38:47.634217+0000","last_clean":"2024-04-21T08:38:47.634217+0000","last_became_active":"2024-04-21T08:36:20.283220+0000","last_became_peered":"2024-04-21T08:36:20.283220+0000","last_unstale":"2024-04-21T08:38:47.634217+0000","last_undegraded":"2024-04-21T08:38:47.634217+0000","last_fullsized":"2024-04-21T08:38:47.634217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:55:24.233237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70068,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":257,"num_read_kb":197,"num_write":44,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655458+0000","last_change":"2024-04-21T08:36:20.919264+0000","last_active":"2024-04-21T08:38:46.655458+0000","last_peered":"2024-04-21T08:38:46.655458+0000","last_clean":"2024-04-21T08:38:46.655458+0000","last_became_active":"2024-04-21T08:36:20.918878+0000","last_became_peered":"2024-04-21T08:36:20.918878+0000","last_unstale":"2024-04-21T08:38:46.655458+0000","last_undegraded":"2024-04-21T08:38:46.655458+0000","last_fullsized":"2024-04-21T08:38:46.655458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:24:06.460758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751673+0000","last_change":"2024-04-21T08:36:24.926878+0000","last_active":"2024-04-21T08:38:46.751673+0000","last_peered":"2024-04-21T08:38:46.751673+0000","last_clean":"2024-04-21T08:38:46.751673+0000","last_became_active":"2024-04-21T08:36:24.926431+0000","last_became_peered":"2024-04-21T08:36:24.926431+0000","last_unstale":"2024-04-21T08:38:46.751673+0000","last_undegraded":"2024-04-21T08:38:46.751673+0000","last_fullsized":"2024-04-21T08:38:46.751673+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:51:29.065962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639141+0000","last_change":"2024-04-21T08:36:20.284841+0000","last_active":"2024-04-21T08:38:46.639141+0000","last_peered":"2024-04-21T08:38:46.639141+0000","last_clean":"2024-04-21T08:38:46.639141+0000","last_became_active":"2024-04-21T08:36:20.283888+0000","last_became_peered":"2024-04-21T08:36:20.283888+0000","last_unstale":"2024-04-21T08:38:46.639141+0000","last_undegraded":"2024-04-21T08:38:46.639141+0000","last_fullsized":"2024-04-21T08:38:46.639141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:29:42.659608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.912243+0000","last_change":"2024-04-21T08:36:20.905140+0000","last_active":"2024-04-21T08:38:46.912243+0000","last_peered":"2024-04-21T08:38:46.912243+0000","last_clean":"2024-04-21T08:38:46.912243+0000","last_became_active":"2024-04-21T08:36:20.904821+0000","last_became_peered":"2024-04-21T08:36:20.904821+0000","last_unstale":"2024-04-21T08:38:46.912243+0000","last_undegraded":"2024-04-21T08:38:46.912243+0000","last_fullsized":"2024-04-21T08:38:46.912243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:09:11.962877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807527+0000","last_change":"2024-04-21T08:38:24.834006+0000","last_active":"2024-04-21T08:38:46.807527+0000","last_peered":"2024-04-21T08:38:46.807527+0000","last_clean":"2024-04-21T08:38:46.807527+0000","last_became_active":"2024-04-21T08:38:24.833453+0000","last_became_peered":"2024-04-21T08:38:24.833453+0000","last_unstale":"2024-04-21T08:38:46.807527+0000","last_undegraded":"2024-04-21T08:38:46.807527+0000","last_fullsized":"2024-04-21T08:38:46.807527+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:27:17.094301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751268+0000","last_change":"2024-04-21T08:36:20.290289+0000","last_active":"2024-04-21T08:38:46.751268+0000","last_peered":"2024-04-21T08:38:46.751268+0000","last_clean":"2024-04-21T08:38:46.751268+0000","last_became_active":"2024-04-21T08:36:20.289922+0000","last_became_peered":"2024-04-21T08:36:20.289922+0000","last_unstale":"2024-04-21T08:38:46.751268+0000","last_undegraded":"2024-04-21T08:38:46.751268+0000","last_fullsized":"2024-04-21T08:38:46.751268+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:10:00.926023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914048+0000","last_change":"2024-04-21T08:36:20.906950+0000","last_active":"2024-04-21T08:38:46.914048+0000","last_peered":"2024-04-21T08:38:46.914048+0000","last_clean":"2024-04-21T08:38:46.914048+0000","last_became_active":"2024-04-21T08:36:20.906599+0000","last_became_peered":"2024-04-21T08:36:20.906599+0000","last_unstale":"2024-04-21T08:38:46.914048+0000","last_undegraded":"2024-04-21T08:38:46.914048+0000","last_fullsized":"2024-04-21T08:38:46.914048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:55:27.595997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.743113+0000","last_change":"2024-04-21T08:36:24.924349+0000","last_active":"2024-04-21T08:38:46.743113+0000","last_peered":"2024-04-21T08:38:46.743113+0000","last_clean":"2024-04-21T08:38:46.743113+0000","last_became_active":"2024-04-21T08:36:24.923982+0000","last_became_peered":"2024-04-21T08:36:24.923982+0000","last_unstale":"2024-04-21T08:38:46.743113+0000","last_undegraded":"2024-04-21T08:38:46.743113+0000","last_fullsized":"2024-04-21T08:38:46.743113+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:20:29.600759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"94'9","reported_seq":170,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:47.636667+0000","last_change":"2024-04-21T08:36:20.285629+0000","last_active":"2024-04-21T08:38:47.636667+0000","last_peered":"2024-04-21T08:38:47.636667+0000","last_clean":"2024-04-21T08:38:47.636667+0000","last_became_active":"2024-04-21T08:36:20.285234+0000","last_became_peered":"2024-04-21T08:36:20.285234+0000","last_unstale":"2024-04-21T08:38:47.636667+0000","last_undegraded":"2024-04-21T08:38:47.636667+0000","last_fullsized":"2024-04-21T08:38:47.636667+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:19:32.966146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1902,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":129,"num_read_kb":6,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913981+0000","last_change":"2024-04-21T08:36:20.906819+0000","last_active":"2024-04-21T08:38:46.913981+0000","last_peered":"2024-04-21T08:38:46.913981+0000","last_clean":"2024-04-21T08:38:46.913981+0000","last_became_active":"2024-04-21T08:36:20.906459+0000","last_became_peered":"2024-04-21T08:36:20.906459+0000","last_unstale":"2024-04-21T08:38:46.913981+0000","last_undegraded":"2024-04-21T08:38:46.913981+0000","last_fullsized":"2024-04-21T08:38:46.913981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:12:39.312857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913214+0000","last_change":"2024-04-21T08:36:24.929452+0000","last_active":"2024-04-21T08:38:46.913214+0000","last_peered":"2024-04-21T08:38:46.913214+0000","last_clean":"2024-04-21T08:38:46.913214+0000","last_became_active":"2024-04-21T08:36:24.929039+0000","last_became_peered":"2024-04-21T08:36:24.929039+0000","last_unstale":"2024-04-21T08:38:46.913214+0000","last_undegraded":"2024-04-21T08:38:46.913214+0000","last_fullsized":"2024-04-21T08:38:46.913214+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:16:05.432024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":76,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655055+0000","last_change":"2024-04-21T08:37:24.719766+0000","last_active":"2024-04-21T08:38:46.655055+0000","last_peered":"2024-04-21T08:38:46.655055+0000","last_clean":"2024-04-21T08:38:46.655055+0000","last_became_active":"2024-04-21T08:37:24.708201+0000","last_became_peered":"2024-04-21T08:37:24.708201+0000","last_unstale":"2024-04-21T08:38:46.655055+0000","last_undegraded":"2024-04-21T08:38:46.655055+0000","last_fullsized":"2024-04-21T08:38:46.655055+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:10:51.158061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751195+0000","last_change":"2024-04-21T08:36:20.909076+0000","last_active":"2024-04-21T08:38:46.751195+0000","last_peered":"2024-04-21T08:38:46.751195+0000","last_clean":"2024-04-21T08:38:46.751195+0000","last_became_active":"2024-04-21T08:36:20.908739+0000","last_became_peered":"2024-04-21T08:36:20.908739+0000","last_unstale":"2024-04-21T08:38:46.751195+0000","last_undegraded":"2024-04-21T08:38:46.751195+0000","last_fullsized":"2024-04-21T08:38:46.751195+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:03:56.870477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751077+0000","last_change":"2024-04-21T08:36:24.926871+0000","last_active":"2024-04-21T08:38:46.751077+0000","last_peered":"2024-04-21T08:38:46.751077+0000","last_clean":"2024-04-21T08:38:46.751077+0000","last_became_active":"2024-04-21T08:36:24.926438+0000","last_became_peered":"2024-04-21T08:36:24.926438+0000","last_unstale":"2024-04-21T08:38:46.751077+0000","last_undegraded":"2024-04-21T08:38:46.751077+0000","last_fullsized":"2024-04-21T08:38:46.751077+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:24:48.984906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"94'3","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807724+0000","last_change":"2024-04-21T08:37:24.308038+0000","last_active":"2024-04-21T08:38:46.807724+0000","last_peered":"2024-04-21T08:38:46.807724+0000","last_clean":"2024-04-21T08:38:46.807724+0000","last_became_active":"2024-04-21T08:37:24.301034+0000","last_became_peered":"2024-04-21T08:37:24.301034+0000","last_unstale":"2024-04-21T08:38:46.807724+0000","last_undegraded":"2024-04-21T08:38:46.807724+0000","last_fullsized":"2024-04-21T08:38:46.807724+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:37:06.598400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807693+0000","last_change":"2024-04-21T08:37:24.299260+0000","last_active":"2024-04-21T08:38:46.807693+0000","last_peered":"2024-04-21T08:38:46.807693+0000","last_clean":"2024-04-21T08:38:46.807693+0000","last_became_active":"2024-04-21T08:37:24.298831+0000","last_became_peered":"2024-04-21T08:37:24.298831+0000","last_unstale":"2024-04-21T08:38:46.807693+0000","last_undegraded":"2024-04-21T08:38:46.807693+0000","last_fullsized":"2024-04-21T08:38:46.807693+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:58:29.368027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655140+0000","last_change":"2024-04-21T08:36:24.929420+0000","last_active":"2024-04-21T08:38:46.655140+0000","last_peered":"2024-04-21T08:38:46.655140+0000","last_clean":"2024-04-21T08:38:46.655140+0000","last_became_active":"2024-04-21T08:36:24.928981+0000","last_became_peered":"2024-04-21T08:36:24.928981+0000","last_unstale":"2024-04-21T08:38:46.655140+0000","last_undegraded":"2024-04-21T08:38:46.655140+0000","last_fullsized":"2024-04-21T08:38:46.655140+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:38:00.040456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.742890+0000","last_change":"2024-04-21T08:36:20.279910+0000","last_active":"2024-04-21T08:38:46.742890+0000","last_peered":"2024-04-21T08:38:46.742890+0000","last_clean":"2024-04-21T08:38:46.742890+0000","last_became_active":"2024-04-21T08:36:20.278904+0000","last_became_peered":"2024-04-21T08:36:20.278904+0000","last_unstale":"2024-04-21T08:38:46.742890+0000","last_undegraded":"2024-04-21T08:38:46.742890+0000","last_fullsized":"2024-04-21T08:38:46.742890+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:40:19.209213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638524+0000","last_change":"2024-04-21T08:36:20.895816+0000","last_active":"2024-04-21T08:38:46.638524+0000","last_peered":"2024-04-21T08:38:46.638524+0000","last_clean":"2024-04-21T08:38:46.638524+0000","last_became_active":"2024-04-21T08:36:20.895487+0000","last_became_peered":"2024-04-21T08:36:20.895487+0000","last_unstale":"2024-04-21T08:38:46.638524+0000","last_undegraded":"2024-04-21T08:38:46.638524+0000","last_fullsized":"2024-04-21T08:38:46.638524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:53:52.153683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.912486+0000","last_change":"2024-04-21T08:36:24.917637+0000","last_active":"2024-04-21T08:38:46.912486+0000","last_peered":"2024-04-21T08:38:46.912486+0000","last_clean":"2024-04-21T08:38:46.912486+0000","last_became_active":"2024-04-21T08:36:24.917192+0000","last_became_peered":"2024-04-21T08:36:24.917192+0000","last_unstale":"2024-04-21T08:38:46.912486+0000","last_undegraded":"2024-04-21T08:38:46.912486+0000","last_fullsized":"2024-04-21T08:38:46.912486+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:02:59.843903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":159,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914154+0000","last_change":"2024-04-21T08:36:20.286424+0000","last_active":"2024-04-21T08:38:46.914154+0000","last_peered":"2024-04-21T08:38:46.914154+0000","last_clean":"2024-04-21T08:38:46.914154+0000","last_became_active":"2024-04-21T08:36:20.285767+0000","last_became_peered":"2024-04-21T08:36:20.285767+0000","last_unstale":"2024-04-21T08:38:46.914154+0000","last_undegraded":"2024-04-21T08:38:46.914154+0000","last_fullsized":"2024-04-21T08:38:46.914154+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:53:33.202734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638138+0000","last_change":"2024-04-21T08:37:24.297650+0000","last_active":"2024-04-21T08:38:46.638138+0000","last_peered":"2024-04-21T08:38:46.638138+0000","last_clean":"2024-04-21T08:38:46.638138+0000","last_became_active":"2024-04-21T08:37:24.297250+0000","last_became_peered":"2024-04-21T08:37:24.297250+0000","last_unstale":"2024-04-21T08:38:46.638138+0000","last_undegraded":"2024-04-21T08:38:46.638138+0000","last_fullsized":"2024-04-21T08:38:46.638138+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:21:21.419556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911806+0000","last_change":"2024-04-21T08:36:24.919669+0000","last_active":"2024-04-21T08:38:46.911806+0000","last_peered":"2024-04-21T08:38:46.911806+0000","last_clean":"2024-04-21T08:38:46.911806+0000","last_became_active":"2024-04-21T08:36:24.919281+0000","last_became_peered":"2024-04-21T08:36:24.919281+0000","last_unstale":"2024-04-21T08:38:46.911806+0000","last_undegraded":"2024-04-21T08:38:46.911806+0000","last_fullsized":"2024-04-21T08:38:46.911806+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:44:27.480367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751175+0000","last_change":"2024-04-21T08:37:24.297826+0000","last_active":"2024-04-21T08:38:46.751175+0000","last_peered":"2024-04-21T08:38:46.751175+0000","last_clean":"2024-04-21T08:38:46.751175+0000","last_became_active":"2024-04-21T08:37:24.297397+0000","last_became_peered":"2024-04-21T08:37:24.297397+0000","last_unstale":"2024-04-21T08:38:46.751175+0000","last_undegraded":"2024-04-21T08:38:46.751175+0000","last_fullsized":"2024-04-21T08:38:46.751175+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:52:17.693725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'7","reported_seq":165,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.916204+0000","last_change":"2024-04-21T08:36:20.291033+0000","last_active":"2024-04-21T08:38:46.916204+0000","last_peered":"2024-04-21T08:38:46.916204+0000","last_clean":"2024-04-21T08:38:46.916204+0000","last_became_active":"2024-04-21T08:36:20.290697+0000","last_became_peered":"2024-04-21T08:36:20.290697+0000","last_unstale":"2024-04-21T08:38:46.916204+0000","last_undegraded":"2024-04-21T08:38:46.916204+0000","last_fullsized":"2024-04-21T08:38:46.916204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:22:54.593595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":124,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913716+0000","last_change":"2024-04-21T08:37:24.310936+0000","last_active":"2024-04-21T08:38:46.913716+0000","last_peered":"2024-04-21T08:38:46.913716+0000","last_clean":"2024-04-21T08:38:46.913716+0000","last_became_active":"2024-04-21T08:37:24.297349+0000","last_became_peered":"2024-04-21T08:37:24.297349+0000","last_unstale":"2024-04-21T08:38:46.913716+0000","last_undegraded":"2024-04-21T08:38:46.913716+0000","last_fullsized":"2024-04-21T08:38:46.913716+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:31:39.884609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638671+0000","last_change":"2024-04-21T08:36:20.904197+0000","last_active":"2024-04-21T08:38:46.638671+0000","last_peered":"2024-04-21T08:38:46.638671+0000","last_clean":"2024-04-21T08:38:46.638671+0000","last_became_active":"2024-04-21T08:36:20.903099+0000","last_became_peered":"2024-04-21T08:36:20.903099+0000","last_unstale":"2024-04-21T08:38:46.638671+0000","last_undegraded":"2024-04-21T08:38:46.638671+0000","last_fullsized":"2024-04-21T08:38:46.638671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:16:08.914335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911254+0000","last_change":"2024-04-21T08:36:20.280608+0000","last_active":"2024-04-21T08:38:46.911254+0000","last_peered":"2024-04-21T08:38:46.911254+0000","last_clean":"2024-04-21T08:38:46.911254+0000","last_became_active":"2024-04-21T08:36:20.280172+0000","last_became_peered":"2024-04-21T08:36:20.280172+0000","last_unstale":"2024-04-21T08:38:46.911254+0000","last_undegraded":"2024-04-21T08:38:46.911254+0000","last_fullsized":"2024-04-21T08:38:46.911254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:51:12.284463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":366,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913553+0000","last_change":"2024-04-21T08:35:08.423545+0000","last_active":"2024-04-21T08:38:46.913553+0000","last_peered":"2024-04-21T08:38:46.913553+0000","last_clean":"2024-04-21T08:38:46.913553+0000","last_became_active":"2024-04-21T08:35:08.423027+0000","last_became_peered":"2024-04-21T08:35:08.423027+0000","last_unstale":"2024-04-21T08:38:46.913553+0000","last_undegraded":"2024-04-21T08:38:46.913553+0000","last_fullsized":"2024-04-21T08:38:46.913553+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:31:45.359253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:31:45.359253+0000","last_clean_scrub_stamp":"2024-04-21T08:31:45.359253+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:53:24.310824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751452+0000","last_change":"2024-04-21T08:36:24.922422+0000","last_active":"2024-04-21T08:38:46.751452+0000","last_peered":"2024-04-21T08:38:46.751452+0000","last_clean":"2024-04-21T08:38:46.751452+0000","last_became_active":"2024-04-21T08:36:24.922006+0000","last_became_peered":"2024-04-21T08:36:24.922006+0000","last_unstale":"2024-04-21T08:38:46.751452+0000","last_undegraded":"2024-04-21T08:38:46.751452+0000","last_fullsized":"2024-04-21T08:38:46.751452+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:46:22.840802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911805+0000","last_change":"2024-04-21T08:36:20.911222+0000","last_active":"2024-04-21T08:38:46.911805+0000","last_peered":"2024-04-21T08:38:46.911805+0000","last_clean":"2024-04-21T08:38:46.911805+0000","last_became_active":"2024-04-21T08:36:20.910933+0000","last_became_peered":"2024-04-21T08:36:20.910933+0000","last_unstale":"2024-04-21T08:38:46.911805+0000","last_undegraded":"2024-04-21T08:38:46.911805+0000","last_fullsized":"2024-04-21T08:38:46.911805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:41:25.621266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913633+0000","last_change":"2024-04-21T08:36:20.288409+0000","last_active":"2024-04-21T08:38:46.913633+0000","last_peered":"2024-04-21T08:38:46.913633+0000","last_clean":"2024-04-21T08:38:46.913633+0000","last_became_active":"2024-04-21T08:36:20.287916+0000","last_became_peered":"2024-04-21T08:36:20.287916+0000","last_unstale":"2024-04-21T08:38:46.913633+0000","last_undegraded":"2024-04-21T08:38:46.913633+0000","last_fullsized":"2024-04-21T08:38:46.913633+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:04:31.068352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807586+0000","last_change":"2024-04-21T08:36:24.918166+0000","last_active":"2024-04-21T08:38:46.807586+0000","last_peered":"2024-04-21T08:38:46.807586+0000","last_clean":"2024-04-21T08:38:46.807586+0000","last_became_active":"2024-04-21T08:36:24.917745+0000","last_became_peered":"2024-04-21T08:36:24.917745+0000","last_unstale":"2024-04-21T08:38:46.807586+0000","last_undegraded":"2024-04-21T08:38:46.807586+0000","last_fullsized":"2024-04-21T08:38:46.807586+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:55:41.169681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655417+0000","last_change":"2024-04-21T08:36:20.291079+0000","last_active":"2024-04-21T08:38:46.655417+0000","last_peered":"2024-04-21T08:38:46.655417+0000","last_clean":"2024-04-21T08:38:46.655417+0000","last_became_active":"2024-04-21T08:36:20.290720+0000","last_became_peered":"2024-04-21T08:36:20.290720+0000","last_unstale":"2024-04-21T08:38:46.655417+0000","last_undegraded":"2024-04-21T08:38:46.655417+0000","last_fullsized":"2024-04-21T08:38:46.655417+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:41.269029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.912713+0000","last_change":"2024-04-21T08:36:20.907416+0000","last_active":"2024-04-21T08:38:46.912713+0000","last_peered":"2024-04-21T08:38:46.912713+0000","last_clean":"2024-04-21T08:38:46.912713+0000","last_became_active":"2024-04-21T08:36:20.907005+0000","last_became_peered":"2024-04-21T08:36:20.907005+0000","last_unstale":"2024-04-21T08:38:46.912713+0000","last_undegraded":"2024-04-21T08:38:46.912713+0000","last_fullsized":"2024-04-21T08:38:46.912713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:58:17.979947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638481+0000","last_change":"2024-04-21T08:36:24.921475+0000","last_active":"2024-04-21T08:38:46.638481+0000","last_peered":"2024-04-21T08:38:46.638481+0000","last_clean":"2024-04-21T08:38:46.638481+0000","last_became_active":"2024-04-21T08:36:24.921091+0000","last_became_peered":"2024-04-21T08:36:24.921091+0000","last_unstale":"2024-04-21T08:38:46.638481+0000","last_undegraded":"2024-04-21T08:38:46.638481+0000","last_fullsized":"2024-04-21T08:38:46.638481+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:28:14.224501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'48","reported_seq":105,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.860978+0000","last_change":"2024-04-21T08:36:20.288916+0000","last_active":"2024-04-21T08:38:46.860978+0000","last_peered":"2024-04-21T08:38:46.860978+0000","last_clean":"2024-04-21T08:38:46.860978+0000","last_became_active":"2024-04-21T08:36:20.288575+0000","last_became_peered":"2024-04-21T08:36:20.288575+0000","last_unstale":"2024-04-21T08:38:46.860978+0000","last_undegraded":"2024-04-21T08:38:46.860978+0000","last_fullsized":"2024-04-21T08:38:46.860978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:52:05.795554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639071+0000","last_change":"2024-04-21T08:36:20.912053+0000","last_active":"2024-04-21T08:38:46.639071+0000","last_peered":"2024-04-21T08:38:46.639071+0000","last_clean":"2024-04-21T08:38:46.639071+0000","last_became_active":"2024-04-21T08:36:20.911584+0000","last_became_peered":"2024-04-21T08:36:20.911584+0000","last_unstale":"2024-04-21T08:38:46.639071+0000","last_undegraded":"2024-04-21T08:38:46.639071+0000","last_fullsized":"2024-04-21T08:38:46.639071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:48:36.369954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638589+0000","last_change":"2024-04-21T08:36:24.919212+0000","last_active":"2024-04-21T08:38:46.638589+0000","last_peered":"2024-04-21T08:38:46.638589+0000","last_clean":"2024-04-21T08:38:46.638589+0000","last_became_active":"2024-04-21T08:36:24.918807+0000","last_became_peered":"2024-04-21T08:36:24.918807+0000","last_unstale":"2024-04-21T08:38:46.638589+0000","last_undegraded":"2024-04-21T08:38:46.638589+0000","last_fullsized":"2024-04-21T08:38:46.638589+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:25:50.816775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911766+0000","last_change":"2024-04-21T08:36:20.282536+0000","last_active":"2024-04-21T08:38:46.911766+0000","last_peered":"2024-04-21T08:38:46.911766+0000","last_clean":"2024-04-21T08:38:46.911766+0000","last_became_active":"2024-04-21T08:36:20.282172+0000","last_became_peered":"2024-04-21T08:36:20.282172+0000","last_unstale":"2024-04-21T08:38:46.911766+0000","last_undegraded":"2024-04-21T08:38:46.911766+0000","last_fullsized":"2024-04-21T08:38:46.911766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:15:38.992441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.915413+0000","last_change":"2024-04-21T08:36:20.908416+0000","last_active":"2024-04-21T08:38:46.915413+0000","last_peered":"2024-04-21T08:38:46.915413+0000","last_clean":"2024-04-21T08:38:46.915413+0000","last_became_active":"2024-04-21T08:36:20.908054+0000","last_became_peered":"2024-04-21T08:36:20.908054+0000","last_unstale":"2024-04-21T08:38:46.915413+0000","last_undegraded":"2024-04-21T08:38:46.915413+0000","last_fullsized":"2024-04-21T08:38:46.915413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:23:16.562060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911638+0000","last_change":"2024-04-21T08:36:24.920637+0000","last_active":"2024-04-21T08:38:46.911638+0000","last_peered":"2024-04-21T08:38:46.911638+0000","last_clean":"2024-04-21T08:38:46.911638+0000","last_became_active":"2024-04-21T08:36:24.919921+0000","last_became_peered":"2024-04-21T08:36:24.919921+0000","last_unstale":"2024-04-21T08:38:46.911638+0000","last_undegraded":"2024-04-21T08:38:46.911638+0000","last_fullsized":"2024-04-21T08:38:46.911638+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:20:38.230087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.922412+0000","last_change":"2024-04-21T08:36:20.279332+0000","last_active":"2024-04-21T08:38:46.922412+0000","last_peered":"2024-04-21T08:38:46.922412+0000","last_clean":"2024-04-21T08:38:46.922412+0000","last_became_active":"2024-04-21T08:36:20.278970+0000","last_became_peered":"2024-04-21T08:36:20.278970+0000","last_unstale":"2024-04-21T08:38:46.922412+0000","last_undegraded":"2024-04-21T08:38:46.922412+0000","last_fullsized":"2024-04-21T08:38:46.922412+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:36:16.041610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.654968+0000","last_change":"2024-04-21T08:36:20.919621+0000","last_active":"2024-04-21T08:38:46.654968+0000","last_peered":"2024-04-21T08:38:46.654968+0000","last_clean":"2024-04-21T08:38:46.654968+0000","last_became_active":"2024-04-21T08:36:20.919246+0000","last_became_peered":"2024-04-21T08:36:20.919246+0000","last_unstale":"2024-04-21T08:38:46.654968+0000","last_undegraded":"2024-04-21T08:38:46.654968+0000","last_fullsized":"2024-04-21T08:38:46.654968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:31:59.001501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.743258+0000","last_change":"2024-04-21T08:36:24.920459+0000","last_active":"2024-04-21T08:38:46.743258+0000","last_peered":"2024-04-21T08:38:46.743258+0000","last_clean":"2024-04-21T08:38:46.743258+0000","last_became_active":"2024-04-21T08:36:24.919691+0000","last_became_peered":"2024-04-21T08:36:24.919691+0000","last_unstale":"2024-04-21T08:38:46.743258+0000","last_undegraded":"2024-04-21T08:38:46.743258+0000","last_fullsized":"2024-04-21T08:38:46.743258+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:38:26.655452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638571+0000","last_change":"2024-04-21T08:36:20.284286+0000","last_active":"2024-04-21T08:38:46.638571+0000","last_peered":"2024-04-21T08:38:46.638571+0000","last_clean":"2024-04-21T08:38:46.638571+0000","last_became_active":"2024-04-21T08:36:20.283903+0000","last_became_peered":"2024-04-21T08:36:20.283903+0000","last_unstale":"2024-04-21T08:38:46.638571+0000","last_undegraded":"2024-04-21T08:38:46.638571+0000","last_fullsized":"2024-04-21T08:38:46.638571+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:51:52.409988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807332+0000","last_change":"2024-04-21T08:36:20.911892+0000","last_active":"2024-04-21T08:38:46.807332+0000","last_peered":"2024-04-21T08:38:46.807332+0000","last_clean":"2024-04-21T08:38:46.807332+0000","last_became_active":"2024-04-21T08:36:20.911513+0000","last_became_peered":"2024-04-21T08:36:20.911513+0000","last_unstale":"2024-04-21T08:38:46.807332+0000","last_undegraded":"2024-04-21T08:38:46.807332+0000","last_fullsized":"2024-04-21T08:38:46.807332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:51:13.733534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911539+0000","last_change":"2024-04-21T08:36:24.920347+0000","last_active":"2024-04-21T08:38:46.911539+0000","last_peered":"2024-04-21T08:38:46.911539+0000","last_clean":"2024-04-21T08:38:46.911539+0000","last_became_active":"2024-04-21T08:36:24.919831+0000","last_became_peered":"2024-04-21T08:36:24.919831+0000","last_unstale":"2024-04-21T08:38:46.911539+0000","last_undegraded":"2024-04-21T08:38:46.911539+0000","last_fullsized":"2024-04-21T08:38:46.911539+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:14:27.020476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638672+0000","last_change":"2024-04-21T08:36:20.286786+0000","last_active":"2024-04-21T08:38:46.638672+0000","last_peered":"2024-04-21T08:38:46.638672+0000","last_clean":"2024-04-21T08:38:46.638672+0000","last_became_active":"2024-04-21T08:36:20.286429+0000","last_became_peered":"2024-04-21T08:36:20.286429+0000","last_unstale":"2024-04-21T08:38:46.638672+0000","last_undegraded":"2024-04-21T08:38:46.638672+0000","last_fullsized":"2024-04-21T08:38:46.638672+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:40:57.093370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638983+0000","last_change":"2024-04-21T08:36:20.904767+0000","last_active":"2024-04-21T08:38:46.638983+0000","last_peered":"2024-04-21T08:38:46.638983+0000","last_clean":"2024-04-21T08:38:46.638983+0000","last_became_active":"2024-04-21T08:36:20.904406+0000","last_became_peered":"2024-04-21T08:36:20.904406+0000","last_unstale":"2024-04-21T08:38:46.638983+0000","last_undegraded":"2024-04-21T08:38:46.638983+0000","last_fullsized":"2024-04-21T08:38:46.638983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:11:41.644549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911769+0000","last_change":"2024-04-21T08:36:24.918031+0000","last_active":"2024-04-21T08:38:46.911769+0000","last_peered":"2024-04-21T08:38:46.911769+0000","last_clean":"2024-04-21T08:38:46.911769+0000","last_became_active":"2024-04-21T08:36:24.917618+0000","last_became_peered":"2024-04-21T08:36:24.917618+0000","last_unstale":"2024-04-21T08:38:46.911769+0000","last_undegraded":"2024-04-21T08:38:46.911769+0000","last_fullsized":"2024-04-21T08:38:46.911769+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:50:35.797911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655179+0000","last_change":"2024-04-21T08:36:20.286212+0000","last_active":"2024-04-21T08:38:46.655179+0000","last_peered":"2024-04-21T08:38:46.655179+0000","last_clean":"2024-04-21T08:38:46.655179+0000","last_became_active":"2024-04-21T08:36:20.285878+0000","last_became_peered":"2024-04-21T08:36:20.285878+0000","last_unstale":"2024-04-21T08:38:46.655179+0000","last_undegraded":"2024-04-21T08:38:46.655179+0000","last_fullsized":"2024-04-21T08:38:46.655179+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:19.375455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914242+0000","last_change":"2024-04-21T08:36:20.908335+0000","last_active":"2024-04-21T08:38:46.914242+0000","last_peered":"2024-04-21T08:38:46.914242+0000","last_clean":"2024-04-21T08:38:46.914242+0000","last_became_active":"2024-04-21T08:36:20.907945+0000","last_became_peered":"2024-04-21T08:36:20.907945+0000","last_unstale":"2024-04-21T08:38:46.914242+0000","last_undegraded":"2024-04-21T08:38:46.914242+0000","last_fullsized":"2024-04-21T08:38:46.914242+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:47:17.841511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.922383+0000","last_change":"2024-04-21T08:36:24.915553+0000","last_active":"2024-04-21T08:38:46.922383+0000","last_peered":"2024-04-21T08:38:46.922383+0000","last_clean":"2024-04-21T08:38:46.922383+0000","last_became_active":"2024-04-21T08:36:24.915028+0000","last_became_peered":"2024-04-21T08:36:24.915028+0000","last_unstale":"2024-04-21T08:38:46.922383+0000","last_undegraded":"2024-04-21T08:38:46.922383+0000","last_fullsized":"2024-04-21T08:38:46.922383+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:17:21.192946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'7","reported_seq":164,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639097+0000","last_change":"2024-04-21T08:36:20.279649+0000","last_active":"2024-04-21T08:38:46.639097+0000","last_peered":"2024-04-21T08:38:46.639097+0000","last_clean":"2024-04-21T08:38:46.639097+0000","last_became_active":"2024-04-21T08:36:20.279111+0000","last_became_peered":"2024-04-21T08:36:20.279111+0000","last_unstale":"2024-04-21T08:38:46.639097+0000","last_undegraded":"2024-04-21T08:38:46.639097+0000","last_fullsized":"2024-04-21T08:38:46.639097+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:19:57.380495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914334+0000","last_change":"2024-04-21T08:36:20.911489+0000","last_active":"2024-04-21T08:38:46.914334+0000","last_peered":"2024-04-21T08:38:46.914334+0000","last_clean":"2024-04-21T08:38:46.914334+0000","last_became_active":"2024-04-21T08:36:20.911168+0000","last_became_peered":"2024-04-21T08:36:20.911168+0000","last_unstale":"2024-04-21T08:38:46.914334+0000","last_undegraded":"2024-04-21T08:38:46.914334+0000","last_fullsized":"2024-04-21T08:38:46.914334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:40:41.585841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914668+0000","last_change":"2024-04-21T08:36:24.921175+0000","last_active":"2024-04-21T08:38:46.914668+0000","last_peered":"2024-04-21T08:38:46.914668+0000","last_clean":"2024-04-21T08:38:46.914668+0000","last_became_active":"2024-04-21T08:36:24.920802+0000","last_became_peered":"2024-04-21T08:36:24.920802+0000","last_unstale":"2024-04-21T08:38:46.914668+0000","last_undegraded":"2024-04-21T08:38:46.914668+0000","last_fullsized":"2024-04-21T08:38:46.914668+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:43:12.140016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"94'41","reported_seq":82,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.859069+0000","last_change":"2024-04-21T08:36:20.290202+0000","last_active":"2024-04-21T08:38:46.859069+0000","last_peered":"2024-04-21T08:38:46.859069+0000","last_clean":"2024-04-21T08:38:46.859069+0000","last_became_active":"2024-04-21T08:36:20.289868+0000","last_became_peered":"2024-04-21T08:36:20.289868+0000","last_unstale":"2024-04-21T08:38:46.859069+0000","last_undegraded":"2024-04-21T08:38:46.859069+0000","last_fullsized":"2024-04-21T08:38:46.859069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:07:45.525980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913789+0000","last_change":"2024-04-21T08:36:20.899590+0000","last_active":"2024-04-21T08:38:46.913789+0000","last_peered":"2024-04-21T08:38:46.913789+0000","last_clean":"2024-04-21T08:38:46.913789+0000","last_became_active":"2024-04-21T08:36:20.899254+0000","last_became_peered":"2024-04-21T08:36:20.899254+0000","last_unstale":"2024-04-21T08:38:46.913789+0000","last_undegraded":"2024-04-21T08:38:46.913789+0000","last_fullsized":"2024-04-21T08:38:46.913789+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:34:25.409694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.750892+0000","last_change":"2024-04-21T08:36:24.918310+0000","last_active":"2024-04-21T08:38:46.750892+0000","last_peered":"2024-04-21T08:38:46.750892+0000","last_clean":"2024-04-21T08:38:46.750892+0000","last_became_active":"2024-04-21T08:36:24.917899+0000","last_became_peered":"2024-04-21T08:36:24.917899+0000","last_unstale":"2024-04-21T08:38:46.750892+0000","last_undegraded":"2024-04-21T08:38:46.750892+0000","last_fullsized":"2024-04-21T08:38:46.750892+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:09:13.006884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":51,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913309+0000","last_change":"2024-04-21T08:37:24.304544+0000","last_active":"2024-04-21T08:38:46.913309+0000","last_peered":"2024-04-21T08:38:46.913309+0000","last_clean":"2024-04-21T08:38:46.913309+0000","last_became_active":"2024-04-21T08:37:24.295808+0000","last_became_peered":"2024-04-21T08:37:24.295808+0000","last_unstale":"2024-04-21T08:38:46.913309+0000","last_undegraded":"2024-04-21T08:38:46.913309+0000","last_fullsized":"2024-04-21T08:38:46.913309+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:07:15.797133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914356+0000","last_change":"2024-04-21T08:36:20.912368+0000","last_active":"2024-04-21T08:38:46.914356+0000","last_peered":"2024-04-21T08:38:46.914356+0000","last_clean":"2024-04-21T08:38:46.914356+0000","last_became_active":"2024-04-21T08:36:20.912009+0000","last_became_peered":"2024-04-21T08:36:20.912009+0000","last_unstale":"2024-04-21T08:38:46.914356+0000","last_undegraded":"2024-04-21T08:38:46.914356+0000","last_fullsized":"2024-04-21T08:38:46.914356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:35:40.700314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807399+0000","last_change":"2024-04-21T08:36:20.913472+0000","last_active":"2024-04-21T08:38:46.807399+0000","last_peered":"2024-04-21T08:38:46.807399+0000","last_clean":"2024-04-21T08:38:46.807399+0000","last_became_active":"2024-04-21T08:36:20.913118+0000","last_became_peered":"2024-04-21T08:36:20.913118+0000","last_unstale":"2024-04-21T08:38:46.807399+0000","last_undegraded":"2024-04-21T08:38:46.807399+0000","last_fullsized":"2024-04-21T08:38:46.807399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:31:53.957044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638786+0000","last_change":"2024-04-21T08:36:20.913677+0000","last_active":"2024-04-21T08:38:46.638786+0000","last_peered":"2024-04-21T08:38:46.638786+0000","last_clean":"2024-04-21T08:38:46.638786+0000","last_became_active":"2024-04-21T08:36:20.913097+0000","last_became_peered":"2024-04-21T08:36:20.913097+0000","last_unstale":"2024-04-21T08:38:46.638786+0000","last_undegraded":"2024-04-21T08:38:46.638786+0000","last_fullsized":"2024-04-21T08:38:46.638786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:08:40.388370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638662+0000","last_change":"2024-04-21T08:36:20.906624+0000","last_active":"2024-04-21T08:38:46.638662+0000","last_peered":"2024-04-21T08:38:46.638662+0000","last_clean":"2024-04-21T08:38:46.638662+0000","last_became_active":"2024-04-21T08:36:20.906247+0000","last_became_peered":"2024-04-21T08:36:20.906247+0000","last_unstale":"2024-04-21T08:38:46.638662+0000","last_undegraded":"2024-04-21T08:38:46.638662+0000","last_fullsized":"2024-04-21T08:38:46.638662+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:44:11.855183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.916219+0000","last_change":"2024-04-21T08:36:20.906576+0000","last_active":"2024-04-21T08:38:46.916219+0000","last_peered":"2024-04-21T08:38:46.916219+0000","last_clean":"2024-04-21T08:38:46.916219+0000","last_became_active":"2024-04-21T08:36:20.906231+0000","last_became_peered":"2024-04-21T08:36:20.906231+0000","last_unstale":"2024-04-21T08:38:46.916219+0000","last_undegraded":"2024-04-21T08:38:46.916219+0000","last_fullsized":"2024-04-21T08:38:46.916219+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:42:48.156614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.912593+0000","last_change":"2024-04-21T08:36:20.902551+0000","last_active":"2024-04-21T08:38:46.912593+0000","last_peered":"2024-04-21T08:38:46.912593+0000","last_clean":"2024-04-21T08:38:46.912593+0000","last_became_active":"2024-04-21T08:36:20.902208+0000","last_became_peered":"2024-04-21T08:36:20.902208+0000","last_unstale":"2024-04-21T08:38:46.912593+0000","last_undegraded":"2024-04-21T08:38:46.912593+0000","last_fullsized":"2024-04-21T08:38:46.912593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:36:30.016256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807607+0000","last_change":"2024-04-21T08:36:20.918588+0000","last_active":"2024-04-21T08:38:46.807607+0000","last_peered":"2024-04-21T08:38:46.807607+0000","last_clean":"2024-04-21T08:38:46.807607+0000","last_became_active":"2024-04-21T08:36:20.918263+0000","last_became_peered":"2024-04-21T08:36:20.918263+0000","last_unstale":"2024-04-21T08:38:46.807607+0000","last_undegraded":"2024-04-21T08:38:46.807607+0000","last_fullsized":"2024-04-21T08:38:46.807607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:51:30.340336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639167+0000","last_change":"2024-04-21T08:36:20.911063+0000","last_active":"2024-04-21T08:38:46.639167+0000","last_peered":"2024-04-21T08:38:46.639167+0000","last_clean":"2024-04-21T08:38:46.639167+0000","last_became_active":"2024-04-21T08:36:20.910647+0000","last_became_peered":"2024-04-21T08:36:20.910647+0000","last_unstale":"2024-04-21T08:38:46.639167+0000","last_undegraded":"2024-04-21T08:38:46.639167+0000","last_fullsized":"2024-04-21T08:38:46.639167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:10:31.079782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639213+0000","last_change":"2024-04-21T08:36:20.914079+0000","last_active":"2024-04-21T08:38:46.639213+0000","last_peered":"2024-04-21T08:38:46.639213+0000","last_clean":"2024-04-21T08:38:46.639213+0000","last_became_active":"2024-04-21T08:36:20.912851+0000","last_became_peered":"2024-04-21T08:36:20.912851+0000","last_unstale":"2024-04-21T08:38:46.639213+0000","last_undegraded":"2024-04-21T08:38:46.639213+0000","last_fullsized":"2024-04-21T08:38:46.639213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:53:35.811281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751071+0000","last_change":"2024-04-21T08:36:20.900551+0000","last_active":"2024-04-21T08:38:46.751071+0000","last_peered":"2024-04-21T08:38:46.751071+0000","last_clean":"2024-04-21T08:38:46.751071+0000","last_became_active":"2024-04-21T08:36:20.900173+0000","last_became_peered":"2024-04-21T08:36:20.900173+0000","last_unstale":"2024-04-21T08:38:46.751071+0000","last_undegraded":"2024-04-21T08:38:46.751071+0000","last_fullsized":"2024-04-21T08:38:46.751071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:21.389498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638564+0000","last_change":"2024-04-21T08:36:20.912953+0000","last_active":"2024-04-21T08:38:46.638564+0000","last_peered":"2024-04-21T08:38:46.638564+0000","last_clean":"2024-04-21T08:38:46.638564+0000","last_became_active":"2024-04-21T08:36:20.912223+0000","last_became_peered":"2024-04-21T08:36:20.912223+0000","last_unstale":"2024-04-21T08:38:46.638564+0000","last_undegraded":"2024-04-21T08:38:46.638564+0000","last_fullsized":"2024-04-21T08:38:46.638564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:43:14.277179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.922633+0000","last_change":"2024-04-21T08:36:20.906164+0000","last_active":"2024-04-21T08:38:46.922633+0000","last_peered":"2024-04-21T08:38:46.922633+0000","last_clean":"2024-04-21T08:38:46.922633+0000","last_became_active":"2024-04-21T08:36:20.905749+0000","last_became_peered":"2024-04-21T08:36:20.905749+0000","last_unstale":"2024-04-21T08:38:46.922633+0000","last_undegraded":"2024-04-21T08:38:46.922633+0000","last_fullsized":"2024-04-21T08:38:46.922633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:02:10.569087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913644+0000","last_change":"2024-04-21T08:36:20.900314+0000","last_active":"2024-04-21T08:38:46.913644+0000","last_peered":"2024-04-21T08:38:46.913644+0000","last_clean":"2024-04-21T08:38:46.913644+0000","last_became_active":"2024-04-21T08:36:20.900009+0000","last_became_peered":"2024-04-21T08:36:20.900009+0000","last_unstale":"2024-04-21T08:38:46.913644+0000","last_undegraded":"2024-04-21T08:38:46.913644+0000","last_fullsized":"2024-04-21T08:38:46.913644+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:19:59.997302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807555+0000","last_change":"2024-04-21T08:36:20.920725+0000","last_active":"2024-04-21T08:38:46.807555+0000","last_peered":"2024-04-21T08:38:46.807555+0000","last_clean":"2024-04-21T08:38:46.807555+0000","last_became_active":"2024-04-21T08:36:20.920288+0000","last_became_peered":"2024-04-21T08:36:20.920288+0000","last_unstale":"2024-04-21T08:38:46.807555+0000","last_undegraded":"2024-04-21T08:38:46.807555+0000","last_fullsized":"2024-04-21T08:38:46.807555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:36:18.736388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":603992,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1350,"num_read_kb":791,"num_write":375,"num_write_kb":1473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":806,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1811976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":305,"ondisk_log_size":305,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383722,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1093632,"data_stored":531299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645358,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28764,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716388,"statfs":{"total":95995035648,"available":95965581312,"internally_reserved":0,"allocated":1785856,"data_stored":1190803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808884,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1105920,"data_stored":530003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070521,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28476,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716676,"statfs":{"total":95995035648,"available":95965876224,"internally_reserved":0,"allocated":1490944,"data_stored":923575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332159,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28672,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716480,"statfs":{"total":95995035648,"available":95965675520,"internally_reserved":0,"allocated":1691648,"data_stored":1127969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495684,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1081344,"data_stored":525507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36476,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708676,"statfs":{"total":95995035648,"available":95957684224,"internally_reserved":0,"allocated":1163264,"data_stored":596717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1105920,"data_stored":527495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36480,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708672,"statfs":{"total":95995035648,"available":95957680128,"internally_reserved":0,"allocated":1167360,"data_stored":598085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182482,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1050031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920846,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1089536,"data_stored":530895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757320,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":33156,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93711996,"statfs":{"total":95995035648,"available":95961083904,"internally_reserved":0,"allocated":1843200,"data_stored":1254509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":71932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":73300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":394694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":204718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T08:38:49.669 INFO:tasks.fwd_scrub.ceph_manager:clean! 2024-04-21T08:38:49.669 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-21T08:38:49.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:49 smithi088 ceph-mon[31259]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:38:49.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:49 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1054819338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:49.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:49 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:49.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:49 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:49.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:49 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:49.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:49 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:49.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:49 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:49.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:49 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:49.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:49 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3233413766' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:49.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:49 smithi179 ceph-mon[31829]: pgmap v3: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:38:49.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:49 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1054819338' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:49.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:49 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:49.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:49 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:49.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:49 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:49.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:49 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:49.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:49 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:49.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:49 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:49.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:49 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3233413766' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:49.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.807+0000 7f1644550640 1 -- 172.21.15.86:0/3289618213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f163c107cd0 msgr2=0x7f163c10a0c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:49.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.807+0000 7f1644550640 1 --2- 172.21.15.86:0/3289618213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f163c107cd0 0x7f163c10a0c0 secure :-1 s=READY pgs=529 cs=0 l=1 rev1=1 crypto rx=0x7f1634009ca0 tx=0x7f163402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:49.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.807+0000 7f1644550640 1 -- 172.21.15.86:0/3289618213 shutdown_connections 2024-04-21T08:38:49.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.807+0000 7f1644550640 1 --2- 172.21.15.86:0/3289618213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f163c107cd0 0x7f163c10a0c0 unknown :-1 s=CLOSED pgs=529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:49.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.807+0000 7f1644550640 1 --2- 172.21.15.86:0/3289618213 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f163c1053a0 0x7f163c107790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:49.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.807+0000 7f1644550640 1 --2- 172.21.15.86:0/3289618213 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f163c102a70 0x7f163c104e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:49.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.807+0000 7f1644550640 1 -- 172.21.15.86:0/3289618213 >> 172.21.15.86:0/3289618213 conn(0x7f163c0fc3e0 msgr2=0x7f163c0fe820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:49.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.807+0000 7f1644550640 1 -- 172.21.15.86:0/3289618213 shutdown_connections 2024-04-21T08:38:49.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.807+0000 7f1644550640 1 -- 172.21.15.86:0/3289618213 wait complete. 2024-04-21T08:38:49.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.808+0000 7f1644550640 1 Processor -- start 2024-04-21T08:38:49.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.808+0000 7f1644550640 1 -- start start 2024-04-21T08:38:49.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.808+0000 7f1644550640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f163c102a70 0x7f163c198170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:49.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.808+0000 7f1644550640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f163c1053a0 0x7f163c1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:49.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.808+0000 7f1644550640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f163c107cd0 0x7f163c19f720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:49.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.808+0000 7f1644550640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f163c10dac0 con 0x7f163c102a70 2024-04-21T08:38:49.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.808+0000 7f1644550640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f163c10d940 con 0x7f163c1053a0 2024-04-21T08:38:49.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.808+0000 7f1644550640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f163c10dc40 con 0x7f163c107cd0 2024-04-21T08:38:49.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.808+0000 7f16422ec640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f163c102a70 0x7f163c198170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:49.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.808+0000 7f16422ec640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f163c102a70 0x7f163c198170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36018/0 (socket says 172.21.15.86:36018) 2024-04-21T08:38:49.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.808+0000 7f16422ec640 1 -- 172.21.15.86:0/1000079868 learned_addr learned my addr 172.21.15.86:0/1000079868 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:49.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.808+0000 7f1641aeb640 1 --2- 172.21.15.86:0/1000079868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f163c1053a0 0x7f163c1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:49.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.808+0000 7f1642aed640 1 --2- 172.21.15.86:0/1000079868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f163c107cd0 0x7f163c19f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:49.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.808+0000 7f1641aeb640 1 -- 172.21.15.86:0/1000079868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f163c107cd0 msgr2=0x7f163c19f720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:49.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.808+0000 7f1641aeb640 1 --2- 172.21.15.86:0/1000079868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f163c107cd0 0x7f163c19f720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:49.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.808+0000 7f1641aeb640 1 -- 172.21.15.86:0/1000079868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f163c102a70 msgr2=0x7f163c198170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:49.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.808+0000 7f1641aeb640 1 --2- 172.21.15.86:0/1000079868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f163c102a70 0x7f163c198170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:49.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.808+0000 7f1641aeb640 1 -- 172.21.15.86:0/1000079868 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f163c19fd90 con 0x7f163c1053a0 2024-04-21T08:38:49.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.808+0000 7f16422ec640 1 --2- 172.21.15.86:0/1000079868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f163c102a70 0x7f163c198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:38:49.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.808+0000 7f1641aeb640 1 --2- 172.21.15.86:0/1000079868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f163c1053a0 0x7f163c1986b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f163800e9c0 tx=0x7f163800ee80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:49.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.809+0000 7f16337fe640 1 -- 172.21.15.86:0/1000079868 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f163800cd80 con 0x7f163c1053a0 2024-04-21T08:38:49.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.809+0000 7f1644550640 1 -- 172.21.15.86:0/1000079868 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f163c1a0080 con 0x7f163c1053a0 2024-04-21T08:38:49.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.809+0000 7f16337fe640 1 -- 172.21.15.86:0/1000079868 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f16380035f0 con 0x7f163c1053a0 2024-04-21T08:38:49.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.809+0000 7f16337fe640 1 -- 172.21.15.86:0/1000079868 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1638010690 con 0x7f163c1053a0 2024-04-21T08:38:49.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.809+0000 7f1644550640 1 -- 172.21.15.86:0/1000079868 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f163c1a0620 con 0x7f163c1053a0 2024-04-21T08:38:49.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.810+0000 7f1644550640 1 -- 172.21.15.86:0/1000079868 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f163c0fded0 con 0x7f163c1053a0 2024-04-21T08:38:49.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.811+0000 7f16337fe640 1 -- 172.21.15.86:0/1000079868 <== mon.2 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7f16380037a0 con 0x7f163c1053a0 2024-04-21T08:38:49.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.811+0000 7f16337fe640 1 --2- 172.21.15.86:0/1000079868 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f16280a3da0 0x7f16280a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:49.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.811+0000 7f16422ec640 1 --2- 172.21.15.86:0/1000079868 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f16280a3da0 0x7f16280a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:49.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.811+0000 7f16337fe640 1 -- 172.21.15.86:0/1000079868 <== mon.2 v2:172.21.15.88:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f16380d0ed0 con 0x7f163c1053a0 2024-04-21T08:38:49.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.812+0000 7f16422ec640 1 --2- 172.21.15.86:0/1000079868 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f16280a3da0 0x7f16280a6260 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f162c011500 tx=0x7f162c005f90 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:49.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:49.815+0000 7f16337fe640 1 -- 172.21.15.86:0/1000079868 <== mon.2 v2:172.21.15.88:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f163809d200 con 0x7f163c1053a0 2024-04-21T08:38:50.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.004+0000 7f1644550640 1 -- 172.21.15.86:0/1000079868 --> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f163c062390 con 0x7f16280a3da0 2024-04-21T08:38:50.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.013+0000 7f16337fe640 1 -- 172.21.15.86:0/1000079868 <== mgr.15558 v2:172.21.15.86:6800/2452032403 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339464 (secure 0 0 0) 0x7f163c062390 con 0x7f16280a3da0 2024-04-21T08:38:50.019 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:38:50.022 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-21T08:38:50.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.017+0000 7f1644550640 1 -- 172.21.15.86:0/1000079868 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f16280a3da0 msgr2=0x7f16280a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:50.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.017+0000 7f1644550640 1 --2- 172.21.15.86:0/1000079868 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f16280a3da0 0x7f16280a6260 secure :-1 s=READY pgs=42 cs=0 l=1 rev1=1 crypto rx=0x7f162c011500 tx=0x7f162c005f90 comp rx=0 tx=0).stop 2024-04-21T08:38:50.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.017+0000 7f1644550640 1 -- 172.21.15.86:0/1000079868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f163c1053a0 msgr2=0x7f163c1986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:50.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.017+0000 7f1644550640 1 --2- 172.21.15.86:0/1000079868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f163c1053a0 0x7f163c1986b0 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f163800e9c0 tx=0x7f163800ee80 comp rx=0 tx=0).stop 2024-04-21T08:38:50.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.017+0000 7f1644550640 1 -- 172.21.15.86:0/1000079868 shutdown_connections 2024-04-21T08:38:50.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.017+0000 7f1644550640 1 --2- 172.21.15.86:0/1000079868 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f16280a3da0 0x7f16280a6260 unknown :-1 s=CLOSED pgs=42 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.017+0000 7f1644550640 1 --2- 172.21.15.86:0/1000079868 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f163c107cd0 0x7f163c19f720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.017+0000 7f1644550640 1 --2- 172.21.15.86:0/1000079868 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f163c1053a0 0x7f163c1986b0 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.017+0000 7f1644550640 1 --2- 172.21.15.86:0/1000079868 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f163c102a70 0x7f163c198170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.017+0000 7f1644550640 1 -- 172.21.15.86:0/1000079868 >> 172.21.15.86:0/1000079868 conn(0x7f163c0fc3e0 msgr2=0x7f163c103570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:50.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.017+0000 7f1644550640 1 -- 172.21.15.86:0/1000079868 shutdown_connections 2024-04-21T08:38:50.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.017+0000 7f1644550640 1 -- 172.21.15.86:0/1000079868 wait complete. 2024-04-21T08:38:50.029 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":4,"stamp":"2024-04-21T08:38:48.652229+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1194594,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1486,"num_read_kb":910,"num_write":558,"num_write_kb":4413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":1181659,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":944,"ondisk_log_size":944,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":386032,"kb_used_data":15860,"kb_used_omap":18,"kb_used_meta":369837,"kb_avail":1124555792,"statfs":{"total":1151940427776,"available":1151545131008,"internally_reserved":0,"allocated":16240640,"data_stored":9386888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":378713404},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.992077"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807854+0000","last_change":"2024-04-21T08:36:20.913421+0000","last_active":"2024-04-21T08:38:46.807854+0000","last_peered":"2024-04-21T08:38:46.807854+0000","last_clean":"2024-04-21T08:38:46.807854+0000","last_became_active":"2024-04-21T08:36:20.913017+0000","last_became_peered":"2024-04-21T08:36:20.913017+0000","last_unstale":"2024-04-21T08:38:46.807854+0000","last_undegraded":"2024-04-21T08:38:46.807854+0000","last_fullsized":"2024-04-21T08:38:46.807854+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:16:29.885916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.915359+0000","last_change":"2024-04-21T08:36:20.911675+0000","last_active":"2024-04-21T08:38:46.915359+0000","last_peered":"2024-04-21T08:38:46.915359+0000","last_clean":"2024-04-21T08:38:46.915359+0000","last_became_active":"2024-04-21T08:36:20.910591+0000","last_became_peered":"2024-04-21T08:36:20.910591+0000","last_unstale":"2024-04-21T08:38:46.915359+0000","last_undegraded":"2024-04-21T08:38:46.915359+0000","last_fullsized":"2024-04-21T08:38:46.915359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:01:40.735376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.915070+0000","last_change":"2024-04-21T08:36:20.911978+0000","last_active":"2024-04-21T08:38:46.915070+0000","last_peered":"2024-04-21T08:38:46.915070+0000","last_clean":"2024-04-21T08:38:46.915070+0000","last_became_active":"2024-04-21T08:36:20.911641+0000","last_became_peered":"2024-04-21T08:36:20.911641+0000","last_unstale":"2024-04-21T08:38:46.915070+0000","last_undegraded":"2024-04-21T08:38:46.915070+0000","last_fullsized":"2024-04-21T08:38:46.915070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:14:23.789790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911901+0000","last_change":"2024-04-21T08:36:20.919027+0000","last_active":"2024-04-21T08:38:46.911901+0000","last_peered":"2024-04-21T08:38:46.911901+0000","last_clean":"2024-04-21T08:38:46.911901+0000","last_became_active":"2024-04-21T08:36:20.918740+0000","last_became_peered":"2024-04-21T08:36:20.918740+0000","last_unstale":"2024-04-21T08:38:46.911901+0000","last_undegraded":"2024-04-21T08:38:46.911901+0000","last_fullsized":"2024-04-21T08:38:46.911901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:00:31.066938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655044+0000","last_change":"2024-04-21T08:36:20.907175+0000","last_active":"2024-04-21T08:38:46.655044+0000","last_peered":"2024-04-21T08:38:46.655044+0000","last_clean":"2024-04-21T08:38:46.655044+0000","last_became_active":"2024-04-21T08:36:20.906818+0000","last_became_peered":"2024-04-21T08:36:20.906818+0000","last_unstale":"2024-04-21T08:38:46.655044+0000","last_undegraded":"2024-04-21T08:38:46.655044+0000","last_fullsized":"2024-04-21T08:38:46.655044+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:41:03.332231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638216+0000","last_change":"2024-04-21T08:36:20.905968+0000","last_active":"2024-04-21T08:38:46.638216+0000","last_peered":"2024-04-21T08:38:46.638216+0000","last_clean":"2024-04-21T08:38:46.638216+0000","last_became_active":"2024-04-21T08:36:20.904840+0000","last_became_peered":"2024-04-21T08:36:20.904840+0000","last_unstale":"2024-04-21T08:38:46.638216+0000","last_undegraded":"2024-04-21T08:38:46.638216+0000","last_fullsized":"2024-04-21T08:38:46.638216+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:23:46.886241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913568+0000","last_change":"2024-04-21T08:36:20.912045+0000","last_active":"2024-04-21T08:38:46.913568+0000","last_peered":"2024-04-21T08:38:46.913568+0000","last_clean":"2024-04-21T08:38:46.913568+0000","last_became_active":"2024-04-21T08:36:20.911690+0000","last_became_peered":"2024-04-21T08:36:20.911690+0000","last_unstale":"2024-04-21T08:38:46.913568+0000","last_undegraded":"2024-04-21T08:38:46.913568+0000","last_fullsized":"2024-04-21T08:38:46.913568+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:06:28.229307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638612+0000","last_change":"2024-04-21T08:36:20.904874+0000","last_active":"2024-04-21T08:38:46.638612+0000","last_peered":"2024-04-21T08:38:46.638612+0000","last_clean":"2024-04-21T08:38:46.638612+0000","last_became_active":"2024-04-21T08:36:20.904217+0000","last_became_peered":"2024-04-21T08:36:20.904217+0000","last_unstale":"2024-04-21T08:38:46.638612+0000","last_undegraded":"2024-04-21T08:38:46.638612+0000","last_fullsized":"2024-04-21T08:38:46.638612+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:37:04.423812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.923169+0000","last_change":"2024-04-21T08:36:20.912874+0000","last_active":"2024-04-21T08:38:46.923169+0000","last_peered":"2024-04-21T08:38:46.923169+0000","last_clean":"2024-04-21T08:38:46.923169+0000","last_became_active":"2024-04-21T08:36:20.912569+0000","last_became_peered":"2024-04-21T08:36:20.912569+0000","last_unstale":"2024-04-21T08:38:46.923169+0000","last_undegraded":"2024-04-21T08:38:46.923169+0000","last_fullsized":"2024-04-21T08:38:46.923169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:55:20.763157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":52,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638497+0000","last_change":"2024-04-21T08:37:24.296916+0000","last_active":"2024-04-21T08:38:46.638497+0000","last_peered":"2024-04-21T08:38:46.638497+0000","last_clean":"2024-04-21T08:38:46.638497+0000","last_became_active":"2024-04-21T08:37:24.296547+0000","last_became_peered":"2024-04-21T08:37:24.296547+0000","last_unstale":"2024-04-21T08:38:46.638497+0000","last_undegraded":"2024-04-21T08:38:46.638497+0000","last_fullsized":"2024-04-21T08:38:46.638497+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:47:56.683793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914246+0000","last_change":"2024-04-21T08:36:20.909094+0000","last_active":"2024-04-21T08:38:46.914246+0000","last_peered":"2024-04-21T08:38:46.914246+0000","last_clean":"2024-04-21T08:38:46.914246+0000","last_became_active":"2024-04-21T08:36:20.908634+0000","last_became_peered":"2024-04-21T08:36:20.908634+0000","last_unstale":"2024-04-21T08:38:46.914246+0000","last_undegraded":"2024-04-21T08:38:46.914246+0000","last_fullsized":"2024-04-21T08:38:46.914246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:28:38.450621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911748+0000","last_change":"2024-04-21T08:36:20.909429+0000","last_active":"2024-04-21T08:38:46.911748+0000","last_peered":"2024-04-21T08:38:46.911748+0000","last_clean":"2024-04-21T08:38:46.911748+0000","last_became_active":"2024-04-21T08:36:20.909081+0000","last_became_peered":"2024-04-21T08:36:20.909081+0000","last_unstale":"2024-04-21T08:38:46.911748+0000","last_undegraded":"2024-04-21T08:38:46.911748+0000","last_fullsized":"2024-04-21T08:38:46.911748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:45:46.939125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913919+0000","last_change":"2024-04-21T08:36:20.895894+0000","last_active":"2024-04-21T08:38:46.913919+0000","last_peered":"2024-04-21T08:38:46.913919+0000","last_clean":"2024-04-21T08:38:46.913919+0000","last_became_active":"2024-04-21T08:36:20.895572+0000","last_became_peered":"2024-04-21T08:36:20.895572+0000","last_unstale":"2024-04-21T08:38:46.913919+0000","last_undegraded":"2024-04-21T08:38:46.913919+0000","last_fullsized":"2024-04-21T08:38:46.913919+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:12:48.750555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914638+0000","last_change":"2024-04-21T08:36:20.905356+0000","last_active":"2024-04-21T08:38:46.914638+0000","last_peered":"2024-04-21T08:38:46.914638+0000","last_clean":"2024-04-21T08:38:46.914638+0000","last_became_active":"2024-04-21T08:36:20.904954+0000","last_became_peered":"2024-04-21T08:36:20.904954+0000","last_unstale":"2024-04-21T08:38:46.914638+0000","last_undegraded":"2024-04-21T08:38:46.914638+0000","last_fullsized":"2024-04-21T08:38:46.914638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:45:49.116482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.915244+0000","last_change":"2024-04-21T08:36:20.911550+0000","last_active":"2024-04-21T08:38:46.915244+0000","last_peered":"2024-04-21T08:38:46.915244+0000","last_clean":"2024-04-21T08:38:46.915244+0000","last_became_active":"2024-04-21T08:36:20.910557+0000","last_became_peered":"2024-04-21T08:36:20.910557+0000","last_unstale":"2024-04-21T08:38:46.915244+0000","last_undegraded":"2024-04-21T08:38:46.915244+0000","last_fullsized":"2024-04-21T08:38:46.915244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:09:49.422729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.922413+0000","last_change":"2024-04-21T08:36:20.903954+0000","last_active":"2024-04-21T08:38:46.922413+0000","last_peered":"2024-04-21T08:38:46.922413+0000","last_clean":"2024-04-21T08:38:46.922413+0000","last_became_active":"2024-04-21T08:36:20.903641+0000","last_became_peered":"2024-04-21T08:36:20.903641+0000","last_unstale":"2024-04-21T08:38:46.922413+0000","last_undegraded":"2024-04-21T08:38:46.922413+0000","last_fullsized":"2024-04-21T08:38:46.922413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:08:55.678940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807825+0000","last_change":"2024-04-21T08:36:20.914256+0000","last_active":"2024-04-21T08:38:46.807825+0000","last_peered":"2024-04-21T08:38:46.807825+0000","last_clean":"2024-04-21T08:38:46.807825+0000","last_became_active":"2024-04-21T08:36:20.913858+0000","last_became_peered":"2024-04-21T08:36:20.913858+0000","last_unstale":"2024-04-21T08:38:46.807825+0000","last_undegraded":"2024-04-21T08:38:46.807825+0000","last_fullsized":"2024-04-21T08:38:46.807825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:01:10.276763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.743080+0000","last_change":"2024-04-21T08:36:20.904138+0000","last_active":"2024-04-21T08:38:46.743080+0000","last_peered":"2024-04-21T08:38:46.743080+0000","last_clean":"2024-04-21T08:38:46.743080+0000","last_became_active":"2024-04-21T08:36:20.903670+0000","last_became_peered":"2024-04-21T08:36:20.903670+0000","last_unstale":"2024-04-21T08:38:46.743080+0000","last_undegraded":"2024-04-21T08:38:46.743080+0000","last_fullsized":"2024-04-21T08:38:46.743080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:38:50.160867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639071+0000","last_change":"2024-04-21T08:36:20.910798+0000","last_active":"2024-04-21T08:38:46.639071+0000","last_peered":"2024-04-21T08:38:46.639071+0000","last_clean":"2024-04-21T08:38:46.639071+0000","last_became_active":"2024-04-21T08:36:20.910411+0000","last_became_peered":"2024-04-21T08:36:20.910411+0000","last_unstale":"2024-04-21T08:38:46.639071+0000","last_undegraded":"2024-04-21T08:38:46.639071+0000","last_fullsized":"2024-04-21T08:38:46.639071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:13:12.109818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638623+0000","last_change":"2024-04-21T08:36:24.924831+0000","last_active":"2024-04-21T08:38:46.638623+0000","last_peered":"2024-04-21T08:38:46.638623+0000","last_clean":"2024-04-21T08:38:46.638623+0000","last_became_active":"2024-04-21T08:36:24.924435+0000","last_became_peered":"2024-04-21T08:36:24.924435+0000","last_unstale":"2024-04-21T08:38:46.638623+0000","last_undegraded":"2024-04-21T08:38:46.638623+0000","last_fullsized":"2024-04-21T08:38:46.638623+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:23.447079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":188,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.743359+0000","last_change":"2024-04-21T08:37:24.308024+0000","last_active":"2024-04-21T08:38:46.743359+0000","last_peered":"2024-04-21T08:38:46.743359+0000","last_clean":"2024-04-21T08:38:46.743359+0000","last_became_active":"2024-04-21T08:37:24.297491+0000","last_became_peered":"2024-04-21T08:37:24.297491+0000","last_unstale":"2024-04-21T08:38:46.743359+0000","last_undegraded":"2024-04-21T08:38:46.743359+0000","last_fullsized":"2024-04-21T08:38:46.743359+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:48:10.039545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":114,"num_read_kb":114,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639158+0000","last_change":"2024-04-21T08:36:20.912851+0000","last_active":"2024-04-21T08:38:46.639158+0000","last_peered":"2024-04-21T08:38:46.639158+0000","last_clean":"2024-04-21T08:38:46.639158+0000","last_became_active":"2024-04-21T08:36:20.912407+0000","last_became_peered":"2024-04-21T08:36:20.912407+0000","last_unstale":"2024-04-21T08:38:46.639158+0000","last_undegraded":"2024-04-21T08:38:46.639158+0000","last_fullsized":"2024-04-21T08:38:46.639158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:45:18.026940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.910950+0000","last_change":"2024-04-21T08:36:24.924052+0000","last_active":"2024-04-21T08:38:46.910950+0000","last_peered":"2024-04-21T08:38:46.910950+0000","last_clean":"2024-04-21T08:38:46.910950+0000","last_became_active":"2024-04-21T08:36:24.923618+0000","last_became_peered":"2024-04-21T08:36:24.923618+0000","last_unstale":"2024-04-21T08:38:46.910950+0000","last_undegraded":"2024-04-21T08:38:46.910950+0000","last_fullsized":"2024-04-21T08:38:46.910950+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:08:18.182386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639180+0000","last_change":"2024-04-21T08:36:20.285291+0000","last_active":"2024-04-21T08:38:46.639180+0000","last_peered":"2024-04-21T08:38:46.639180+0000","last_clean":"2024-04-21T08:38:46.639180+0000","last_became_active":"2024-04-21T08:36:20.284969+0000","last_became_peered":"2024-04-21T08:36:20.284969+0000","last_unstale":"2024-04-21T08:38:46.639180+0000","last_undegraded":"2024-04-21T08:38:46.639180+0000","last_fullsized":"2024-04-21T08:38:46.639180+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:55:29.484439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.915781+0000","last_change":"2024-04-21T08:36:20.902819+0000","last_active":"2024-04-21T08:38:46.915781+0000","last_peered":"2024-04-21T08:38:46.915781+0000","last_clean":"2024-04-21T08:38:46.915781+0000","last_became_active":"2024-04-21T08:36:20.902466+0000","last_became_peered":"2024-04-21T08:36:20.902466+0000","last_unstale":"2024-04-21T08:38:46.915781+0000","last_undegraded":"2024-04-21T08:38:46.915781+0000","last_fullsized":"2024-04-21T08:38:46.915781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:29:40.454026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911156+0000","last_change":"2024-04-21T08:36:24.926480+0000","last_active":"2024-04-21T08:38:46.911156+0000","last_peered":"2024-04-21T08:38:46.911156+0000","last_clean":"2024-04-21T08:38:46.911156+0000","last_became_active":"2024-04-21T08:36:24.926116+0000","last_became_peered":"2024-04-21T08:36:24.926116+0000","last_unstale":"2024-04-21T08:38:46.911156+0000","last_undegraded":"2024-04-21T08:38:46.911156+0000","last_fullsized":"2024-04-21T08:38:46.911156+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:43:18.839396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655562+0000","last_change":"2024-04-21T08:36:20.293086+0000","last_active":"2024-04-21T08:38:46.655562+0000","last_peered":"2024-04-21T08:38:46.655562+0000","last_clean":"2024-04-21T08:38:46.655562+0000","last_became_active":"2024-04-21T08:36:20.291178+0000","last_became_peered":"2024-04-21T08:36:20.291178+0000","last_unstale":"2024-04-21T08:38:46.655562+0000","last_undegraded":"2024-04-21T08:38:46.655562+0000","last_fullsized":"2024-04-21T08:38:46.655562+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:02:32.935930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.922727+0000","last_change":"2024-04-21T08:36:20.909374+0000","last_active":"2024-04-21T08:38:46.922727+0000","last_peered":"2024-04-21T08:38:46.922727+0000","last_clean":"2024-04-21T08:38:46.922727+0000","last_became_active":"2024-04-21T08:36:20.909043+0000","last_became_peered":"2024-04-21T08:36:20.909043+0000","last_unstale":"2024-04-21T08:38:46.922727+0000","last_undegraded":"2024-04-21T08:38:46.922727+0000","last_fullsized":"2024-04-21T08:38:46.922727+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:45:45.719942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911098+0000","last_change":"2024-04-21T08:38:24.623194+0000","last_active":"2024-04-21T08:38:46.911098+0000","last_peered":"2024-04-21T08:38:46.911098+0000","last_clean":"2024-04-21T08:38:46.911098+0000","last_became_active":"2024-04-21T08:38:24.622632+0000","last_became_peered":"2024-04-21T08:38:24.622632+0000","last_unstale":"2024-04-21T08:38:46.911098+0000","last_undegraded":"2024-04-21T08:38:46.911098+0000","last_fullsized":"2024-04-21T08:38:46.911098+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:26:39.752161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655409+0000","last_change":"2024-04-21T08:36:20.288644+0000","last_active":"2024-04-21T08:38:46.655409+0000","last_peered":"2024-04-21T08:38:46.655409+0000","last_clean":"2024-04-21T08:38:46.655409+0000","last_became_active":"2024-04-21T08:36:20.288069+0000","last_became_peered":"2024-04-21T08:36:20.288069+0000","last_unstale":"2024-04-21T08:38:46.655409+0000","last_undegraded":"2024-04-21T08:38:46.655409+0000","last_fullsized":"2024-04-21T08:38:46.655409+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:37:36.522713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751376+0000","last_change":"2024-04-21T08:36:20.908505+0000","last_active":"2024-04-21T08:38:46.751376+0000","last_peered":"2024-04-21T08:38:46.751376+0000","last_clean":"2024-04-21T08:38:46.751376+0000","last_became_active":"2024-04-21T08:36:20.908163+0000","last_became_peered":"2024-04-21T08:36:20.908163+0000","last_unstale":"2024-04-21T08:38:46.751376+0000","last_undegraded":"2024-04-21T08:38:46.751376+0000","last_fullsized":"2024-04-21T08:38:46.751376+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:54:18.530203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639160+0000","last_change":"2024-04-21T08:36:24.919428+0000","last_active":"2024-04-21T08:38:46.639160+0000","last_peered":"2024-04-21T08:38:46.639160+0000","last_clean":"2024-04-21T08:38:46.639160+0000","last_became_active":"2024-04-21T08:36:24.918722+0000","last_became_peered":"2024-04-21T08:36:24.918722+0000","last_unstale":"2024-04-21T08:38:46.639160+0000","last_undegraded":"2024-04-21T08:38:46.639160+0000","last_fullsized":"2024-04-21T08:38:46.639160+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:30:32.369551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"93'8","reported_seq":168,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751405+0000","last_change":"2024-04-21T08:36:20.287642+0000","last_active":"2024-04-21T08:38:46.751405+0000","last_peered":"2024-04-21T08:38:46.751405+0000","last_clean":"2024-04-21T08:38:46.751405+0000","last_became_active":"2024-04-21T08:36:20.287296+0000","last_became_peered":"2024-04-21T08:36:20.287296+0000","last_unstale":"2024-04-21T08:38:46.751405+0000","last_undegraded":"2024-04-21T08:38:46.751405+0000","last_fullsized":"2024-04-21T08:38:46.751405+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:59:28.579020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1462,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":128,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655081+0000","last_change":"2024-04-21T08:36:20.916971+0000","last_active":"2024-04-21T08:38:46.655081+0000","last_peered":"2024-04-21T08:38:46.655081+0000","last_clean":"2024-04-21T08:38:46.655081+0000","last_became_active":"2024-04-21T08:36:20.916622+0000","last_became_peered":"2024-04-21T08:36:20.916622+0000","last_unstale":"2024-04-21T08:38:46.655081+0000","last_undegraded":"2024-04-21T08:38:46.655081+0000","last_fullsized":"2024-04-21T08:38:46.655081+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:49:38.858772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638856+0000","last_change":"2024-04-21T08:36:24.924170+0000","last_active":"2024-04-21T08:38:46.638856+0000","last_peered":"2024-04-21T08:38:46.638856+0000","last_clean":"2024-04-21T08:38:46.638856+0000","last_became_active":"2024-04-21T08:36:24.923744+0000","last_became_peered":"2024-04-21T08:36:24.923744+0000","last_unstale":"2024-04-21T08:38:46.638856+0000","last_undegraded":"2024-04-21T08:38:46.638856+0000","last_fullsized":"2024-04-21T08:38:46.638856+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:23:24.605021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638388+0000","last_change":"2024-04-21T08:36:20.286545+0000","last_active":"2024-04-21T08:38:46.638388+0000","last_peered":"2024-04-21T08:38:46.638388+0000","last_clean":"2024-04-21T08:38:46.638388+0000","last_became_active":"2024-04-21T08:36:20.286197+0000","last_became_peered":"2024-04-21T08:36:20.286197+0000","last_unstale":"2024-04-21T08:38:46.638388+0000","last_undegraded":"2024-04-21T08:38:46.638388+0000","last_fullsized":"2024-04-21T08:38:46.638388+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:27:55.430428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751488+0000","last_change":"2024-04-21T08:36:20.907743+0000","last_active":"2024-04-21T08:38:46.751488+0000","last_peered":"2024-04-21T08:38:46.751488+0000","last_clean":"2024-04-21T08:38:46.751488+0000","last_became_active":"2024-04-21T08:36:20.907163+0000","last_became_peered":"2024-04-21T08:36:20.907163+0000","last_unstale":"2024-04-21T08:38:46.751488+0000","last_undegraded":"2024-04-21T08:38:46.751488+0000","last_fullsized":"2024-04-21T08:38:46.751488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:41:00.172353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807766+0000","last_change":"2024-04-21T08:37:24.301529+0000","last_active":"2024-04-21T08:38:46.807766+0000","last_peered":"2024-04-21T08:38:46.807766+0000","last_clean":"2024-04-21T08:38:46.807766+0000","last_became_active":"2024-04-21T08:37:24.301014+0000","last_became_peered":"2024-04-21T08:37:24.301014+0000","last_unstale":"2024-04-21T08:38:46.807766+0000","last_undegraded":"2024-04-21T08:38:46.807766+0000","last_fullsized":"2024-04-21T08:38:46.807766+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:58:20.088501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911966+0000","last_change":"2024-04-21T08:36:20.283143+0000","last_active":"2024-04-21T08:38:46.911966+0000","last_peered":"2024-04-21T08:38:46.911966+0000","last_clean":"2024-04-21T08:38:46.911966+0000","last_became_active":"2024-04-21T08:36:20.282585+0000","last_became_peered":"2024-04-21T08:36:20.282585+0000","last_unstale":"2024-04-21T08:38:46.911966+0000","last_undegraded":"2024-04-21T08:38:46.911966+0000","last_fullsized":"2024-04-21T08:38:46.911966+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:12:37.588764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638488+0000","last_change":"2024-04-21T08:36:20.912591+0000","last_active":"2024-04-21T08:38:46.638488+0000","last_peered":"2024-04-21T08:38:46.638488+0000","last_clean":"2024-04-21T08:38:46.638488+0000","last_became_active":"2024-04-21T08:36:20.912174+0000","last_became_peered":"2024-04-21T08:36:20.912174+0000","last_unstale":"2024-04-21T08:38:46.638488+0000","last_undegraded":"2024-04-21T08:38:46.638488+0000","last_fullsized":"2024-04-21T08:38:46.638488+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:33:58.949506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.915307+0000","last_change":"2024-04-21T08:36:24.915998+0000","last_active":"2024-04-21T08:38:46.915307+0000","last_peered":"2024-04-21T08:38:46.915307+0000","last_clean":"2024-04-21T08:38:46.915307+0000","last_became_active":"2024-04-21T08:36:24.915621+0000","last_became_peered":"2024-04-21T08:36:24.915621+0000","last_unstale":"2024-04-21T08:38:46.915307+0000","last_undegraded":"2024-04-21T08:38:46.915307+0000","last_fullsized":"2024-04-21T08:38:46.915307+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:43:05.117181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751087+0000","last_change":"2024-04-21T08:36:20.286308+0000","last_active":"2024-04-21T08:38:46.751087+0000","last_peered":"2024-04-21T08:38:46.751087+0000","last_clean":"2024-04-21T08:38:46.751087+0000","last_became_active":"2024-04-21T08:36:20.285975+0000","last_became_peered":"2024-04-21T08:36:20.285975+0000","last_unstale":"2024-04-21T08:38:46.751087+0000","last_undegraded":"2024-04-21T08:38:46.751087+0000","last_fullsized":"2024-04-21T08:38:46.751087+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:55:10.658349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.742770+0000","last_change":"2024-04-21T08:36:20.900502+0000","last_active":"2024-04-21T08:38:46.742770+0000","last_peered":"2024-04-21T08:38:46.742770+0000","last_clean":"2024-04-21T08:38:46.742770+0000","last_became_active":"2024-04-21T08:36:20.900199+0000","last_became_peered":"2024-04-21T08:36:20.900199+0000","last_unstale":"2024-04-21T08:38:46.742770+0000","last_undegraded":"2024-04-21T08:38:46.742770+0000","last_fullsized":"2024-04-21T08:38:46.742770+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:17:00.720918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638804+0000","last_change":"2024-04-21T08:36:24.920991+0000","last_active":"2024-04-21T08:38:46.638804+0000","last_peered":"2024-04-21T08:38:46.638804+0000","last_clean":"2024-04-21T08:38:46.638804+0000","last_became_active":"2024-04-21T08:36:24.916611+0000","last_became_peered":"2024-04-21T08:36:24.916611+0000","last_unstale":"2024-04-21T08:38:46.638804+0000","last_undegraded":"2024-04-21T08:38:46.638804+0000","last_fullsized":"2024-04-21T08:38:46.638804+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:01:01.412127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"93'7","reported_seq":283,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:47.633155+0000","last_change":"2024-04-21T08:36:20.284966+0000","last_active":"2024-04-21T08:38:47.633155+0000","last_peered":"2024-04-21T08:38:47.633155+0000","last_clean":"2024-04-21T08:38:47.633155+0000","last_became_active":"2024-04-21T08:36:20.284612+0000","last_became_peered":"2024-04-21T08:36:20.284612+0000","last_unstale":"2024-04-21T08:38:47.633155+0000","last_undegraded":"2024-04-21T08:38:47.633155+0000","last_fullsized":"2024-04-21T08:38:47.633155+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:54:50.095013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":243,"num_read_kb":243,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.923175+0000","last_change":"2024-04-21T08:36:20.903493+0000","last_active":"2024-04-21T08:38:46.923175+0000","last_peered":"2024-04-21T08:38:46.923175+0000","last_clean":"2024-04-21T08:38:46.923175+0000","last_became_active":"2024-04-21T08:36:20.902752+0000","last_became_peered":"2024-04-21T08:36:20.902752+0000","last_unstale":"2024-04-21T08:38:46.923175+0000","last_undegraded":"2024-04-21T08:38:46.923175+0000","last_fullsized":"2024-04-21T08:38:46.923175+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:40.684264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.742602+0000","last_change":"2024-04-21T08:36:24.920505+0000","last_active":"2024-04-21T08:38:46.742602+0000","last_peered":"2024-04-21T08:38:46.742602+0000","last_clean":"2024-04-21T08:38:46.742602+0000","last_became_active":"2024-04-21T08:36:24.919966+0000","last_became_peered":"2024-04-21T08:36:24.919966+0000","last_unstale":"2024-04-21T08:38:46.742602+0000","last_undegraded":"2024-04-21T08:38:46.742602+0000","last_fullsized":"2024-04-21T08:38:46.742602+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:58:05.131815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914242+0000","last_change":"2024-04-21T08:36:20.281720+0000","last_active":"2024-04-21T08:38:46.914242+0000","last_peered":"2024-04-21T08:38:46.914242+0000","last_clean":"2024-04-21T08:38:46.914242+0000","last_became_active":"2024-04-21T08:36:20.279862+0000","last_became_peered":"2024-04-21T08:36:20.279862+0000","last_unstale":"2024-04-21T08:38:46.914242+0000","last_undegraded":"2024-04-21T08:38:46.914242+0000","last_fullsized":"2024-04-21T08:38:46.914242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:18:58.106532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.916030+0000","last_change":"2024-04-21T08:36:20.908520+0000","last_active":"2024-04-21T08:38:46.916030+0000","last_peered":"2024-04-21T08:38:46.916030+0000","last_clean":"2024-04-21T08:38:46.916030+0000","last_became_active":"2024-04-21T08:36:20.908184+0000","last_became_peered":"2024-04-21T08:36:20.908184+0000","last_unstale":"2024-04-21T08:38:46.916030+0000","last_undegraded":"2024-04-21T08:38:46.916030+0000","last_fullsized":"2024-04-21T08:38:46.916030+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:14:33.895864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911416+0000","last_change":"2024-04-21T08:36:24.920697+0000","last_active":"2024-04-21T08:38:46.911416+0000","last_peered":"2024-04-21T08:38:46.911416+0000","last_clean":"2024-04-21T08:38:46.911416+0000","last_became_active":"2024-04-21T08:36:24.919976+0000","last_became_peered":"2024-04-21T08:36:24.919976+0000","last_unstale":"2024-04-21T08:38:46.911416+0000","last_undegraded":"2024-04-21T08:38:46.911416+0000","last_fullsized":"2024-04-21T08:38:46.911416+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:40:09.238887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751422+0000","last_change":"2024-04-21T08:36:20.283531+0000","last_active":"2024-04-21T08:38:46.751422+0000","last_peered":"2024-04-21T08:38:46.751422+0000","last_clean":"2024-04-21T08:38:46.751422+0000","last_became_active":"2024-04-21T08:36:20.283159+0000","last_became_peered":"2024-04-21T08:36:20.283159+0000","last_unstale":"2024-04-21T08:38:46.751422+0000","last_undegraded":"2024-04-21T08:38:46.751422+0000","last_fullsized":"2024-04-21T08:38:46.751422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:57:02.917537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.922897+0000","last_change":"2024-04-21T08:36:20.910113+0000","last_active":"2024-04-21T08:38:46.922897+0000","last_peered":"2024-04-21T08:38:46.922897+0000","last_clean":"2024-04-21T08:38:46.922897+0000","last_became_active":"2024-04-21T08:36:20.909744+0000","last_became_peered":"2024-04-21T08:36:20.909744+0000","last_unstale":"2024-04-21T08:38:46.922897+0000","last_undegraded":"2024-04-21T08:38:46.922897+0000","last_fullsized":"2024-04-21T08:38:46.922897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:16:14.568318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.742678+0000","last_change":"2024-04-21T08:36:24.922232+0000","last_active":"2024-04-21T08:38:46.742678+0000","last_peered":"2024-04-21T08:38:46.742678+0000","last_clean":"2024-04-21T08:38:46.742678+0000","last_became_active":"2024-04-21T08:36:24.921690+0000","last_became_peered":"2024-04-21T08:36:24.921690+0000","last_unstale":"2024-04-21T08:38:46.742678+0000","last_undegraded":"2024-04-21T08:38:46.742678+0000","last_fullsized":"2024-04-21T08:38:46.742678+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:21:42.580203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'42","reported_seq":332,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:47.634217+0000","last_change":"2024-04-21T08:36:20.283574+0000","last_active":"2024-04-21T08:38:47.634217+0000","last_peered":"2024-04-21T08:38:47.634217+0000","last_clean":"2024-04-21T08:38:47.634217+0000","last_became_active":"2024-04-21T08:36:20.283220+0000","last_became_peered":"2024-04-21T08:36:20.283220+0000","last_unstale":"2024-04-21T08:38:47.634217+0000","last_undegraded":"2024-04-21T08:38:47.634217+0000","last_fullsized":"2024-04-21T08:38:47.634217+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:55:24.233237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70068,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":257,"num_read_kb":197,"num_write":44,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655458+0000","last_change":"2024-04-21T08:36:20.919264+0000","last_active":"2024-04-21T08:38:46.655458+0000","last_peered":"2024-04-21T08:38:46.655458+0000","last_clean":"2024-04-21T08:38:46.655458+0000","last_became_active":"2024-04-21T08:36:20.918878+0000","last_became_peered":"2024-04-21T08:36:20.918878+0000","last_unstale":"2024-04-21T08:38:46.655458+0000","last_undegraded":"2024-04-21T08:38:46.655458+0000","last_fullsized":"2024-04-21T08:38:46.655458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:24:06.460758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751673+0000","last_change":"2024-04-21T08:36:24.926878+0000","last_active":"2024-04-21T08:38:46.751673+0000","last_peered":"2024-04-21T08:38:46.751673+0000","last_clean":"2024-04-21T08:38:46.751673+0000","last_became_active":"2024-04-21T08:36:24.926431+0000","last_became_peered":"2024-04-21T08:36:24.926431+0000","last_unstale":"2024-04-21T08:38:46.751673+0000","last_undegraded":"2024-04-21T08:38:46.751673+0000","last_fullsized":"2024-04-21T08:38:46.751673+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:51:29.065962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639141+0000","last_change":"2024-04-21T08:36:20.284841+0000","last_active":"2024-04-21T08:38:46.639141+0000","last_peered":"2024-04-21T08:38:46.639141+0000","last_clean":"2024-04-21T08:38:46.639141+0000","last_became_active":"2024-04-21T08:36:20.283888+0000","last_became_peered":"2024-04-21T08:36:20.283888+0000","last_unstale":"2024-04-21T08:38:46.639141+0000","last_undegraded":"2024-04-21T08:38:46.639141+0000","last_fullsized":"2024-04-21T08:38:46.639141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:29:42.659608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.912243+0000","last_change":"2024-04-21T08:36:20.905140+0000","last_active":"2024-04-21T08:38:46.912243+0000","last_peered":"2024-04-21T08:38:46.912243+0000","last_clean":"2024-04-21T08:38:46.912243+0000","last_became_active":"2024-04-21T08:36:20.904821+0000","last_became_peered":"2024-04-21T08:36:20.904821+0000","last_unstale":"2024-04-21T08:38:46.912243+0000","last_undegraded":"2024-04-21T08:38:46.912243+0000","last_fullsized":"2024-04-21T08:38:46.912243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:09:11.962877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807527+0000","last_change":"2024-04-21T08:38:24.834006+0000","last_active":"2024-04-21T08:38:46.807527+0000","last_peered":"2024-04-21T08:38:46.807527+0000","last_clean":"2024-04-21T08:38:46.807527+0000","last_became_active":"2024-04-21T08:38:24.833453+0000","last_became_peered":"2024-04-21T08:38:24.833453+0000","last_unstale":"2024-04-21T08:38:46.807527+0000","last_undegraded":"2024-04-21T08:38:46.807527+0000","last_fullsized":"2024-04-21T08:38:46.807527+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:27:17.094301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751268+0000","last_change":"2024-04-21T08:36:20.290289+0000","last_active":"2024-04-21T08:38:46.751268+0000","last_peered":"2024-04-21T08:38:46.751268+0000","last_clean":"2024-04-21T08:38:46.751268+0000","last_became_active":"2024-04-21T08:36:20.289922+0000","last_became_peered":"2024-04-21T08:36:20.289922+0000","last_unstale":"2024-04-21T08:38:46.751268+0000","last_undegraded":"2024-04-21T08:38:46.751268+0000","last_fullsized":"2024-04-21T08:38:46.751268+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:10:00.926023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914048+0000","last_change":"2024-04-21T08:36:20.906950+0000","last_active":"2024-04-21T08:38:46.914048+0000","last_peered":"2024-04-21T08:38:46.914048+0000","last_clean":"2024-04-21T08:38:46.914048+0000","last_became_active":"2024-04-21T08:36:20.906599+0000","last_became_peered":"2024-04-21T08:36:20.906599+0000","last_unstale":"2024-04-21T08:38:46.914048+0000","last_undegraded":"2024-04-21T08:38:46.914048+0000","last_fullsized":"2024-04-21T08:38:46.914048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:55:27.595997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.743113+0000","last_change":"2024-04-21T08:36:24.924349+0000","last_active":"2024-04-21T08:38:46.743113+0000","last_peered":"2024-04-21T08:38:46.743113+0000","last_clean":"2024-04-21T08:38:46.743113+0000","last_became_active":"2024-04-21T08:36:24.923982+0000","last_became_peered":"2024-04-21T08:36:24.923982+0000","last_unstale":"2024-04-21T08:38:46.743113+0000","last_undegraded":"2024-04-21T08:38:46.743113+0000","last_fullsized":"2024-04-21T08:38:46.743113+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:20:29.600759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"94'9","reported_seq":170,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:47.636667+0000","last_change":"2024-04-21T08:36:20.285629+0000","last_active":"2024-04-21T08:38:47.636667+0000","last_peered":"2024-04-21T08:38:47.636667+0000","last_clean":"2024-04-21T08:38:47.636667+0000","last_became_active":"2024-04-21T08:36:20.285234+0000","last_became_peered":"2024-04-21T08:36:20.285234+0000","last_unstale":"2024-04-21T08:38:47.636667+0000","last_undegraded":"2024-04-21T08:38:47.636667+0000","last_fullsized":"2024-04-21T08:38:47.636667+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:19:32.966146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1902,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":129,"num_read_kb":6,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913981+0000","last_change":"2024-04-21T08:36:20.906819+0000","last_active":"2024-04-21T08:38:46.913981+0000","last_peered":"2024-04-21T08:38:46.913981+0000","last_clean":"2024-04-21T08:38:46.913981+0000","last_became_active":"2024-04-21T08:36:20.906459+0000","last_became_peered":"2024-04-21T08:36:20.906459+0000","last_unstale":"2024-04-21T08:38:46.913981+0000","last_undegraded":"2024-04-21T08:38:46.913981+0000","last_fullsized":"2024-04-21T08:38:46.913981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:12:39.312857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913214+0000","last_change":"2024-04-21T08:36:24.929452+0000","last_active":"2024-04-21T08:38:46.913214+0000","last_peered":"2024-04-21T08:38:46.913214+0000","last_clean":"2024-04-21T08:38:46.913214+0000","last_became_active":"2024-04-21T08:36:24.929039+0000","last_became_peered":"2024-04-21T08:36:24.929039+0000","last_unstale":"2024-04-21T08:38:46.913214+0000","last_undegraded":"2024-04-21T08:38:46.913214+0000","last_fullsized":"2024-04-21T08:38:46.913214+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:16:05.432024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":76,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655055+0000","last_change":"2024-04-21T08:37:24.719766+0000","last_active":"2024-04-21T08:38:46.655055+0000","last_peered":"2024-04-21T08:38:46.655055+0000","last_clean":"2024-04-21T08:38:46.655055+0000","last_became_active":"2024-04-21T08:37:24.708201+0000","last_became_peered":"2024-04-21T08:37:24.708201+0000","last_unstale":"2024-04-21T08:38:46.655055+0000","last_undegraded":"2024-04-21T08:38:46.655055+0000","last_fullsized":"2024-04-21T08:38:46.655055+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:10:51.158061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751195+0000","last_change":"2024-04-21T08:36:20.909076+0000","last_active":"2024-04-21T08:38:46.751195+0000","last_peered":"2024-04-21T08:38:46.751195+0000","last_clean":"2024-04-21T08:38:46.751195+0000","last_became_active":"2024-04-21T08:36:20.908739+0000","last_became_peered":"2024-04-21T08:36:20.908739+0000","last_unstale":"2024-04-21T08:38:46.751195+0000","last_undegraded":"2024-04-21T08:38:46.751195+0000","last_fullsized":"2024-04-21T08:38:46.751195+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:03:56.870477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751077+0000","last_change":"2024-04-21T08:36:24.926871+0000","last_active":"2024-04-21T08:38:46.751077+0000","last_peered":"2024-04-21T08:38:46.751077+0000","last_clean":"2024-04-21T08:38:46.751077+0000","last_became_active":"2024-04-21T08:36:24.926438+0000","last_became_peered":"2024-04-21T08:36:24.926438+0000","last_unstale":"2024-04-21T08:38:46.751077+0000","last_undegraded":"2024-04-21T08:38:46.751077+0000","last_fullsized":"2024-04-21T08:38:46.751077+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:24:48.984906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"94'3","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807724+0000","last_change":"2024-04-21T08:37:24.308038+0000","last_active":"2024-04-21T08:38:46.807724+0000","last_peered":"2024-04-21T08:38:46.807724+0000","last_clean":"2024-04-21T08:38:46.807724+0000","last_became_active":"2024-04-21T08:37:24.301034+0000","last_became_peered":"2024-04-21T08:37:24.301034+0000","last_unstale":"2024-04-21T08:38:46.807724+0000","last_undegraded":"2024-04-21T08:38:46.807724+0000","last_fullsized":"2024-04-21T08:38:46.807724+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:37:06.598400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807693+0000","last_change":"2024-04-21T08:37:24.299260+0000","last_active":"2024-04-21T08:38:46.807693+0000","last_peered":"2024-04-21T08:38:46.807693+0000","last_clean":"2024-04-21T08:38:46.807693+0000","last_became_active":"2024-04-21T08:37:24.298831+0000","last_became_peered":"2024-04-21T08:37:24.298831+0000","last_unstale":"2024-04-21T08:38:46.807693+0000","last_undegraded":"2024-04-21T08:38:46.807693+0000","last_fullsized":"2024-04-21T08:38:46.807693+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:58:29.368027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655140+0000","last_change":"2024-04-21T08:36:24.929420+0000","last_active":"2024-04-21T08:38:46.655140+0000","last_peered":"2024-04-21T08:38:46.655140+0000","last_clean":"2024-04-21T08:38:46.655140+0000","last_became_active":"2024-04-21T08:36:24.928981+0000","last_became_peered":"2024-04-21T08:36:24.928981+0000","last_unstale":"2024-04-21T08:38:46.655140+0000","last_undegraded":"2024-04-21T08:38:46.655140+0000","last_fullsized":"2024-04-21T08:38:46.655140+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:38:00.040456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.742890+0000","last_change":"2024-04-21T08:36:20.279910+0000","last_active":"2024-04-21T08:38:46.742890+0000","last_peered":"2024-04-21T08:38:46.742890+0000","last_clean":"2024-04-21T08:38:46.742890+0000","last_became_active":"2024-04-21T08:36:20.278904+0000","last_became_peered":"2024-04-21T08:36:20.278904+0000","last_unstale":"2024-04-21T08:38:46.742890+0000","last_undegraded":"2024-04-21T08:38:46.742890+0000","last_fullsized":"2024-04-21T08:38:46.742890+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:40:19.209213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638524+0000","last_change":"2024-04-21T08:36:20.895816+0000","last_active":"2024-04-21T08:38:46.638524+0000","last_peered":"2024-04-21T08:38:46.638524+0000","last_clean":"2024-04-21T08:38:46.638524+0000","last_became_active":"2024-04-21T08:36:20.895487+0000","last_became_peered":"2024-04-21T08:36:20.895487+0000","last_unstale":"2024-04-21T08:38:46.638524+0000","last_undegraded":"2024-04-21T08:38:46.638524+0000","last_fullsized":"2024-04-21T08:38:46.638524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:53:52.153683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.912486+0000","last_change":"2024-04-21T08:36:24.917637+0000","last_active":"2024-04-21T08:38:46.912486+0000","last_peered":"2024-04-21T08:38:46.912486+0000","last_clean":"2024-04-21T08:38:46.912486+0000","last_became_active":"2024-04-21T08:36:24.917192+0000","last_became_peered":"2024-04-21T08:36:24.917192+0000","last_unstale":"2024-04-21T08:38:46.912486+0000","last_undegraded":"2024-04-21T08:38:46.912486+0000","last_fullsized":"2024-04-21T08:38:46.912486+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:02:59.843903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":159,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914154+0000","last_change":"2024-04-21T08:36:20.286424+0000","last_active":"2024-04-21T08:38:46.914154+0000","last_peered":"2024-04-21T08:38:46.914154+0000","last_clean":"2024-04-21T08:38:46.914154+0000","last_became_active":"2024-04-21T08:36:20.285767+0000","last_became_peered":"2024-04-21T08:36:20.285767+0000","last_unstale":"2024-04-21T08:38:46.914154+0000","last_undegraded":"2024-04-21T08:38:46.914154+0000","last_fullsized":"2024-04-21T08:38:46.914154+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:53:33.202734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":121,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638138+0000","last_change":"2024-04-21T08:37:24.297650+0000","last_active":"2024-04-21T08:38:46.638138+0000","last_peered":"2024-04-21T08:38:46.638138+0000","last_clean":"2024-04-21T08:38:46.638138+0000","last_became_active":"2024-04-21T08:37:24.297250+0000","last_became_peered":"2024-04-21T08:37:24.297250+0000","last_unstale":"2024-04-21T08:38:46.638138+0000","last_undegraded":"2024-04-21T08:38:46.638138+0000","last_fullsized":"2024-04-21T08:38:46.638138+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:21:21.419556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911806+0000","last_change":"2024-04-21T08:36:24.919669+0000","last_active":"2024-04-21T08:38:46.911806+0000","last_peered":"2024-04-21T08:38:46.911806+0000","last_clean":"2024-04-21T08:38:46.911806+0000","last_became_active":"2024-04-21T08:36:24.919281+0000","last_became_peered":"2024-04-21T08:36:24.919281+0000","last_unstale":"2024-04-21T08:38:46.911806+0000","last_undegraded":"2024-04-21T08:38:46.911806+0000","last_fullsized":"2024-04-21T08:38:46.911806+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:44:27.480367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751175+0000","last_change":"2024-04-21T08:37:24.297826+0000","last_active":"2024-04-21T08:38:46.751175+0000","last_peered":"2024-04-21T08:38:46.751175+0000","last_clean":"2024-04-21T08:38:46.751175+0000","last_became_active":"2024-04-21T08:37:24.297397+0000","last_became_peered":"2024-04-21T08:37:24.297397+0000","last_unstale":"2024-04-21T08:38:46.751175+0000","last_undegraded":"2024-04-21T08:38:46.751175+0000","last_fullsized":"2024-04-21T08:38:46.751175+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:52:17.693725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'7","reported_seq":165,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.916204+0000","last_change":"2024-04-21T08:36:20.291033+0000","last_active":"2024-04-21T08:38:46.916204+0000","last_peered":"2024-04-21T08:38:46.916204+0000","last_clean":"2024-04-21T08:38:46.916204+0000","last_became_active":"2024-04-21T08:36:20.290697+0000","last_became_peered":"2024-04-21T08:36:20.290697+0000","last_unstale":"2024-04-21T08:38:46.916204+0000","last_undegraded":"2024-04-21T08:38:46.916204+0000","last_fullsized":"2024-04-21T08:38:46.916204+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:22:54.593595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":124,"num_read_kb":124,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913716+0000","last_change":"2024-04-21T08:37:24.310936+0000","last_active":"2024-04-21T08:38:46.913716+0000","last_peered":"2024-04-21T08:38:46.913716+0000","last_clean":"2024-04-21T08:38:46.913716+0000","last_became_active":"2024-04-21T08:37:24.297349+0000","last_became_peered":"2024-04-21T08:37:24.297349+0000","last_unstale":"2024-04-21T08:38:46.913716+0000","last_undegraded":"2024-04-21T08:38:46.913716+0000","last_fullsized":"2024-04-21T08:38:46.913716+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:31:39.884609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638671+0000","last_change":"2024-04-21T08:36:20.904197+0000","last_active":"2024-04-21T08:38:46.638671+0000","last_peered":"2024-04-21T08:38:46.638671+0000","last_clean":"2024-04-21T08:38:46.638671+0000","last_became_active":"2024-04-21T08:36:20.903099+0000","last_became_peered":"2024-04-21T08:36:20.903099+0000","last_unstale":"2024-04-21T08:38:46.638671+0000","last_undegraded":"2024-04-21T08:38:46.638671+0000","last_fullsized":"2024-04-21T08:38:46.638671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:16:08.914335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911254+0000","last_change":"2024-04-21T08:36:20.280608+0000","last_active":"2024-04-21T08:38:46.911254+0000","last_peered":"2024-04-21T08:38:46.911254+0000","last_clean":"2024-04-21T08:38:46.911254+0000","last_became_active":"2024-04-21T08:36:20.280172+0000","last_became_peered":"2024-04-21T08:36:20.280172+0000","last_unstale":"2024-04-21T08:38:46.911254+0000","last_undegraded":"2024-04-21T08:38:46.911254+0000","last_fullsized":"2024-04-21T08:38:46.911254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:51:12.284463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"20'137","reported_seq":366,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913553+0000","last_change":"2024-04-21T08:35:08.423545+0000","last_active":"2024-04-21T08:38:46.913553+0000","last_peered":"2024-04-21T08:38:46.913553+0000","last_clean":"2024-04-21T08:38:46.913553+0000","last_became_active":"2024-04-21T08:35:08.423027+0000","last_became_peered":"2024-04-21T08:35:08.423027+0000","last_unstale":"2024-04-21T08:38:46.913553+0000","last_undegraded":"2024-04-21T08:38:46.913553+0000","last_fullsized":"2024-04-21T08:38:46.913553+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:31:45.359253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:31:45.359253+0000","last_clean_scrub_stamp":"2024-04-21T08:31:45.359253+0000","objects_scrubbed":0,"log_size":137,"log_dups_size":0,"ondisk_log_size":137,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:53:24.310824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751452+0000","last_change":"2024-04-21T08:36:24.922422+0000","last_active":"2024-04-21T08:38:46.751452+0000","last_peered":"2024-04-21T08:38:46.751452+0000","last_clean":"2024-04-21T08:38:46.751452+0000","last_became_active":"2024-04-21T08:36:24.922006+0000","last_became_peered":"2024-04-21T08:36:24.922006+0000","last_unstale":"2024-04-21T08:38:46.751452+0000","last_undegraded":"2024-04-21T08:38:46.751452+0000","last_fullsized":"2024-04-21T08:38:46.751452+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:46:22.840802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911805+0000","last_change":"2024-04-21T08:36:20.911222+0000","last_active":"2024-04-21T08:38:46.911805+0000","last_peered":"2024-04-21T08:38:46.911805+0000","last_clean":"2024-04-21T08:38:46.911805+0000","last_became_active":"2024-04-21T08:36:20.910933+0000","last_became_peered":"2024-04-21T08:36:20.910933+0000","last_unstale":"2024-04-21T08:38:46.911805+0000","last_undegraded":"2024-04-21T08:38:46.911805+0000","last_fullsized":"2024-04-21T08:38:46.911805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:41:25.621266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913633+0000","last_change":"2024-04-21T08:36:20.288409+0000","last_active":"2024-04-21T08:38:46.913633+0000","last_peered":"2024-04-21T08:38:46.913633+0000","last_clean":"2024-04-21T08:38:46.913633+0000","last_became_active":"2024-04-21T08:36:20.287916+0000","last_became_peered":"2024-04-21T08:36:20.287916+0000","last_unstale":"2024-04-21T08:38:46.913633+0000","last_undegraded":"2024-04-21T08:38:46.913633+0000","last_fullsized":"2024-04-21T08:38:46.913633+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:04:31.068352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807586+0000","last_change":"2024-04-21T08:36:24.918166+0000","last_active":"2024-04-21T08:38:46.807586+0000","last_peered":"2024-04-21T08:38:46.807586+0000","last_clean":"2024-04-21T08:38:46.807586+0000","last_became_active":"2024-04-21T08:36:24.917745+0000","last_became_peered":"2024-04-21T08:36:24.917745+0000","last_unstale":"2024-04-21T08:38:46.807586+0000","last_undegraded":"2024-04-21T08:38:46.807586+0000","last_fullsized":"2024-04-21T08:38:46.807586+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:55:41.169681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655417+0000","last_change":"2024-04-21T08:36:20.291079+0000","last_active":"2024-04-21T08:38:46.655417+0000","last_peered":"2024-04-21T08:38:46.655417+0000","last_clean":"2024-04-21T08:38:46.655417+0000","last_became_active":"2024-04-21T08:36:20.290720+0000","last_became_peered":"2024-04-21T08:36:20.290720+0000","last_unstale":"2024-04-21T08:38:46.655417+0000","last_undegraded":"2024-04-21T08:38:46.655417+0000","last_fullsized":"2024-04-21T08:38:46.655417+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:41.269029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.912713+0000","last_change":"2024-04-21T08:36:20.907416+0000","last_active":"2024-04-21T08:38:46.912713+0000","last_peered":"2024-04-21T08:38:46.912713+0000","last_clean":"2024-04-21T08:38:46.912713+0000","last_became_active":"2024-04-21T08:36:20.907005+0000","last_became_peered":"2024-04-21T08:36:20.907005+0000","last_unstale":"2024-04-21T08:38:46.912713+0000","last_undegraded":"2024-04-21T08:38:46.912713+0000","last_fullsized":"2024-04-21T08:38:46.912713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:58:17.979947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638481+0000","last_change":"2024-04-21T08:36:24.921475+0000","last_active":"2024-04-21T08:38:46.638481+0000","last_peered":"2024-04-21T08:38:46.638481+0000","last_clean":"2024-04-21T08:38:46.638481+0000","last_became_active":"2024-04-21T08:36:24.921091+0000","last_became_peered":"2024-04-21T08:36:24.921091+0000","last_unstale":"2024-04-21T08:38:46.638481+0000","last_undegraded":"2024-04-21T08:38:46.638481+0000","last_fullsized":"2024-04-21T08:38:46.638481+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:28:14.224501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'48","reported_seq":105,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.860978+0000","last_change":"2024-04-21T08:36:20.288916+0000","last_active":"2024-04-21T08:38:46.860978+0000","last_peered":"2024-04-21T08:38:46.860978+0000","last_clean":"2024-04-21T08:38:46.860978+0000","last_became_active":"2024-04-21T08:36:20.288575+0000","last_became_peered":"2024-04-21T08:36:20.288575+0000","last_unstale":"2024-04-21T08:38:46.860978+0000","last_undegraded":"2024-04-21T08:38:46.860978+0000","last_fullsized":"2024-04-21T08:38:46.860978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:52:05.795554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639071+0000","last_change":"2024-04-21T08:36:20.912053+0000","last_active":"2024-04-21T08:38:46.639071+0000","last_peered":"2024-04-21T08:38:46.639071+0000","last_clean":"2024-04-21T08:38:46.639071+0000","last_became_active":"2024-04-21T08:36:20.911584+0000","last_became_peered":"2024-04-21T08:36:20.911584+0000","last_unstale":"2024-04-21T08:38:46.639071+0000","last_undegraded":"2024-04-21T08:38:46.639071+0000","last_fullsized":"2024-04-21T08:38:46.639071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:48:36.369954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638589+0000","last_change":"2024-04-21T08:36:24.919212+0000","last_active":"2024-04-21T08:38:46.638589+0000","last_peered":"2024-04-21T08:38:46.638589+0000","last_clean":"2024-04-21T08:38:46.638589+0000","last_became_active":"2024-04-21T08:36:24.918807+0000","last_became_peered":"2024-04-21T08:36:24.918807+0000","last_unstale":"2024-04-21T08:38:46.638589+0000","last_undegraded":"2024-04-21T08:38:46.638589+0000","last_fullsized":"2024-04-21T08:38:46.638589+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:25:50.816775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911766+0000","last_change":"2024-04-21T08:36:20.282536+0000","last_active":"2024-04-21T08:38:46.911766+0000","last_peered":"2024-04-21T08:38:46.911766+0000","last_clean":"2024-04-21T08:38:46.911766+0000","last_became_active":"2024-04-21T08:36:20.282172+0000","last_became_peered":"2024-04-21T08:36:20.282172+0000","last_unstale":"2024-04-21T08:38:46.911766+0000","last_undegraded":"2024-04-21T08:38:46.911766+0000","last_fullsized":"2024-04-21T08:38:46.911766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:15:38.992441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.915413+0000","last_change":"2024-04-21T08:36:20.908416+0000","last_active":"2024-04-21T08:38:46.915413+0000","last_peered":"2024-04-21T08:38:46.915413+0000","last_clean":"2024-04-21T08:38:46.915413+0000","last_became_active":"2024-04-21T08:36:20.908054+0000","last_became_peered":"2024-04-21T08:36:20.908054+0000","last_unstale":"2024-04-21T08:38:46.915413+0000","last_undegraded":"2024-04-21T08:38:46.915413+0000","last_fullsized":"2024-04-21T08:38:46.915413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:23:16.562060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911638+0000","last_change":"2024-04-21T08:36:24.920637+0000","last_active":"2024-04-21T08:38:46.911638+0000","last_peered":"2024-04-21T08:38:46.911638+0000","last_clean":"2024-04-21T08:38:46.911638+0000","last_became_active":"2024-04-21T08:36:24.919921+0000","last_became_peered":"2024-04-21T08:36:24.919921+0000","last_unstale":"2024-04-21T08:38:46.911638+0000","last_undegraded":"2024-04-21T08:38:46.911638+0000","last_fullsized":"2024-04-21T08:38:46.911638+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:20:38.230087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.922412+0000","last_change":"2024-04-21T08:36:20.279332+0000","last_active":"2024-04-21T08:38:46.922412+0000","last_peered":"2024-04-21T08:38:46.922412+0000","last_clean":"2024-04-21T08:38:46.922412+0000","last_became_active":"2024-04-21T08:36:20.278970+0000","last_became_peered":"2024-04-21T08:36:20.278970+0000","last_unstale":"2024-04-21T08:38:46.922412+0000","last_undegraded":"2024-04-21T08:38:46.922412+0000","last_fullsized":"2024-04-21T08:38:46.922412+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:36:16.041610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.654968+0000","last_change":"2024-04-21T08:36:20.919621+0000","last_active":"2024-04-21T08:38:46.654968+0000","last_peered":"2024-04-21T08:38:46.654968+0000","last_clean":"2024-04-21T08:38:46.654968+0000","last_became_active":"2024-04-21T08:36:20.919246+0000","last_became_peered":"2024-04-21T08:36:20.919246+0000","last_unstale":"2024-04-21T08:38:46.654968+0000","last_undegraded":"2024-04-21T08:38:46.654968+0000","last_fullsized":"2024-04-21T08:38:46.654968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:31:59.001501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.743258+0000","last_change":"2024-04-21T08:36:24.920459+0000","last_active":"2024-04-21T08:38:46.743258+0000","last_peered":"2024-04-21T08:38:46.743258+0000","last_clean":"2024-04-21T08:38:46.743258+0000","last_became_active":"2024-04-21T08:36:24.919691+0000","last_became_peered":"2024-04-21T08:36:24.919691+0000","last_unstale":"2024-04-21T08:38:46.743258+0000","last_undegraded":"2024-04-21T08:38:46.743258+0000","last_fullsized":"2024-04-21T08:38:46.743258+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:38:26.655452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638571+0000","last_change":"2024-04-21T08:36:20.284286+0000","last_active":"2024-04-21T08:38:46.638571+0000","last_peered":"2024-04-21T08:38:46.638571+0000","last_clean":"2024-04-21T08:38:46.638571+0000","last_became_active":"2024-04-21T08:36:20.283903+0000","last_became_peered":"2024-04-21T08:36:20.283903+0000","last_unstale":"2024-04-21T08:38:46.638571+0000","last_undegraded":"2024-04-21T08:38:46.638571+0000","last_fullsized":"2024-04-21T08:38:46.638571+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:51:52.409988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807332+0000","last_change":"2024-04-21T08:36:20.911892+0000","last_active":"2024-04-21T08:38:46.807332+0000","last_peered":"2024-04-21T08:38:46.807332+0000","last_clean":"2024-04-21T08:38:46.807332+0000","last_became_active":"2024-04-21T08:36:20.911513+0000","last_became_peered":"2024-04-21T08:36:20.911513+0000","last_unstale":"2024-04-21T08:38:46.807332+0000","last_undegraded":"2024-04-21T08:38:46.807332+0000","last_fullsized":"2024-04-21T08:38:46.807332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:51:13.733534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911539+0000","last_change":"2024-04-21T08:36:24.920347+0000","last_active":"2024-04-21T08:38:46.911539+0000","last_peered":"2024-04-21T08:38:46.911539+0000","last_clean":"2024-04-21T08:38:46.911539+0000","last_became_active":"2024-04-21T08:36:24.919831+0000","last_became_peered":"2024-04-21T08:36:24.919831+0000","last_unstale":"2024-04-21T08:38:46.911539+0000","last_undegraded":"2024-04-21T08:38:46.911539+0000","last_fullsized":"2024-04-21T08:38:46.911539+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:14:27.020476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638672+0000","last_change":"2024-04-21T08:36:20.286786+0000","last_active":"2024-04-21T08:38:46.638672+0000","last_peered":"2024-04-21T08:38:46.638672+0000","last_clean":"2024-04-21T08:38:46.638672+0000","last_became_active":"2024-04-21T08:36:20.286429+0000","last_became_peered":"2024-04-21T08:36:20.286429+0000","last_unstale":"2024-04-21T08:38:46.638672+0000","last_undegraded":"2024-04-21T08:38:46.638672+0000","last_fullsized":"2024-04-21T08:38:46.638672+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:40:57.093370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638983+0000","last_change":"2024-04-21T08:36:20.904767+0000","last_active":"2024-04-21T08:38:46.638983+0000","last_peered":"2024-04-21T08:38:46.638983+0000","last_clean":"2024-04-21T08:38:46.638983+0000","last_became_active":"2024-04-21T08:36:20.904406+0000","last_became_peered":"2024-04-21T08:36:20.904406+0000","last_unstale":"2024-04-21T08:38:46.638983+0000","last_undegraded":"2024-04-21T08:38:46.638983+0000","last_fullsized":"2024-04-21T08:38:46.638983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:11:41.644549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911769+0000","last_change":"2024-04-21T08:36:24.918031+0000","last_active":"2024-04-21T08:38:46.911769+0000","last_peered":"2024-04-21T08:38:46.911769+0000","last_clean":"2024-04-21T08:38:46.911769+0000","last_became_active":"2024-04-21T08:36:24.917618+0000","last_became_peered":"2024-04-21T08:36:24.917618+0000","last_unstale":"2024-04-21T08:38:46.911769+0000","last_undegraded":"2024-04-21T08:38:46.911769+0000","last_fullsized":"2024-04-21T08:38:46.911769+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:50:35.797911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655179+0000","last_change":"2024-04-21T08:36:20.286212+0000","last_active":"2024-04-21T08:38:46.655179+0000","last_peered":"2024-04-21T08:38:46.655179+0000","last_clean":"2024-04-21T08:38:46.655179+0000","last_became_active":"2024-04-21T08:36:20.285878+0000","last_became_peered":"2024-04-21T08:36:20.285878+0000","last_unstale":"2024-04-21T08:38:46.655179+0000","last_undegraded":"2024-04-21T08:38:46.655179+0000","last_fullsized":"2024-04-21T08:38:46.655179+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:19.375455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914242+0000","last_change":"2024-04-21T08:36:20.908335+0000","last_active":"2024-04-21T08:38:46.914242+0000","last_peered":"2024-04-21T08:38:46.914242+0000","last_clean":"2024-04-21T08:38:46.914242+0000","last_became_active":"2024-04-21T08:36:20.907945+0000","last_became_peered":"2024-04-21T08:36:20.907945+0000","last_unstale":"2024-04-21T08:38:46.914242+0000","last_undegraded":"2024-04-21T08:38:46.914242+0000","last_fullsized":"2024-04-21T08:38:46.914242+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:47:17.841511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.922383+0000","last_change":"2024-04-21T08:36:24.915553+0000","last_active":"2024-04-21T08:38:46.922383+0000","last_peered":"2024-04-21T08:38:46.922383+0000","last_clean":"2024-04-21T08:38:46.922383+0000","last_became_active":"2024-04-21T08:36:24.915028+0000","last_became_peered":"2024-04-21T08:36:24.915028+0000","last_unstale":"2024-04-21T08:38:46.922383+0000","last_undegraded":"2024-04-21T08:38:46.922383+0000","last_fullsized":"2024-04-21T08:38:46.922383+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:17:21.192946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'7","reported_seq":164,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639097+0000","last_change":"2024-04-21T08:36:20.279649+0000","last_active":"2024-04-21T08:38:46.639097+0000","last_peered":"2024-04-21T08:38:46.639097+0000","last_clean":"2024-04-21T08:38:46.639097+0000","last_became_active":"2024-04-21T08:36:20.279111+0000","last_became_peered":"2024-04-21T08:36:20.279111+0000","last_unstale":"2024-04-21T08:38:46.639097+0000","last_undegraded":"2024-04-21T08:38:46.639097+0000","last_fullsized":"2024-04-21T08:38:46.639097+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:19:57.380495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":123,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914334+0000","last_change":"2024-04-21T08:36:20.911489+0000","last_active":"2024-04-21T08:38:46.914334+0000","last_peered":"2024-04-21T08:38:46.914334+0000","last_clean":"2024-04-21T08:38:46.914334+0000","last_became_active":"2024-04-21T08:36:20.911168+0000","last_became_peered":"2024-04-21T08:36:20.911168+0000","last_unstale":"2024-04-21T08:38:46.914334+0000","last_undegraded":"2024-04-21T08:38:46.914334+0000","last_fullsized":"2024-04-21T08:38:46.914334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:40:41.585841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914668+0000","last_change":"2024-04-21T08:36:24.921175+0000","last_active":"2024-04-21T08:38:46.914668+0000","last_peered":"2024-04-21T08:38:46.914668+0000","last_clean":"2024-04-21T08:38:46.914668+0000","last_became_active":"2024-04-21T08:36:24.920802+0000","last_became_peered":"2024-04-21T08:36:24.920802+0000","last_unstale":"2024-04-21T08:38:46.914668+0000","last_undegraded":"2024-04-21T08:38:46.914668+0000","last_fullsized":"2024-04-21T08:38:46.914668+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:43:12.140016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"94'41","reported_seq":82,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.859069+0000","last_change":"2024-04-21T08:36:20.290202+0000","last_active":"2024-04-21T08:38:46.859069+0000","last_peered":"2024-04-21T08:38:46.859069+0000","last_clean":"2024-04-21T08:38:46.859069+0000","last_became_active":"2024-04-21T08:36:20.289868+0000","last_became_peered":"2024-04-21T08:36:20.289868+0000","last_unstale":"2024-04-21T08:38:46.859069+0000","last_undegraded":"2024-04-21T08:38:46.859069+0000","last_fullsized":"2024-04-21T08:38:46.859069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:07:45.525980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913789+0000","last_change":"2024-04-21T08:36:20.899590+0000","last_active":"2024-04-21T08:38:46.913789+0000","last_peered":"2024-04-21T08:38:46.913789+0000","last_clean":"2024-04-21T08:38:46.913789+0000","last_became_active":"2024-04-21T08:36:20.899254+0000","last_became_peered":"2024-04-21T08:36:20.899254+0000","last_unstale":"2024-04-21T08:38:46.913789+0000","last_undegraded":"2024-04-21T08:38:46.913789+0000","last_fullsized":"2024-04-21T08:38:46.913789+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:34:25.409694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.750892+0000","last_change":"2024-04-21T08:36:24.918310+0000","last_active":"2024-04-21T08:38:46.750892+0000","last_peered":"2024-04-21T08:38:46.750892+0000","last_clean":"2024-04-21T08:38:46.750892+0000","last_became_active":"2024-04-21T08:36:24.917899+0000","last_became_peered":"2024-04-21T08:36:24.917899+0000","last_unstale":"2024-04-21T08:38:46.750892+0000","last_undegraded":"2024-04-21T08:38:46.750892+0000","last_fullsized":"2024-04-21T08:38:46.750892+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:09:13.006884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":51,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913309+0000","last_change":"2024-04-21T08:37:24.304544+0000","last_active":"2024-04-21T08:38:46.913309+0000","last_peered":"2024-04-21T08:38:46.913309+0000","last_clean":"2024-04-21T08:38:46.913309+0000","last_became_active":"2024-04-21T08:37:24.295808+0000","last_became_peered":"2024-04-21T08:37:24.295808+0000","last_unstale":"2024-04-21T08:38:46.913309+0000","last_undegraded":"2024-04-21T08:38:46.913309+0000","last_fullsized":"2024-04-21T08:38:46.913309+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:07:15.797133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914356+0000","last_change":"2024-04-21T08:36:20.912368+0000","last_active":"2024-04-21T08:38:46.914356+0000","last_peered":"2024-04-21T08:38:46.914356+0000","last_clean":"2024-04-21T08:38:46.914356+0000","last_became_active":"2024-04-21T08:36:20.912009+0000","last_became_peered":"2024-04-21T08:36:20.912009+0000","last_unstale":"2024-04-21T08:38:46.914356+0000","last_undegraded":"2024-04-21T08:38:46.914356+0000","last_fullsized":"2024-04-21T08:38:46.914356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:35:40.700314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807399+0000","last_change":"2024-04-21T08:36:20.913472+0000","last_active":"2024-04-21T08:38:46.807399+0000","last_peered":"2024-04-21T08:38:46.807399+0000","last_clean":"2024-04-21T08:38:46.807399+0000","last_became_active":"2024-04-21T08:36:20.913118+0000","last_became_peered":"2024-04-21T08:36:20.913118+0000","last_unstale":"2024-04-21T08:38:46.807399+0000","last_undegraded":"2024-04-21T08:38:46.807399+0000","last_fullsized":"2024-04-21T08:38:46.807399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:31:53.957044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638786+0000","last_change":"2024-04-21T08:36:20.913677+0000","last_active":"2024-04-21T08:38:46.638786+0000","last_peered":"2024-04-21T08:38:46.638786+0000","last_clean":"2024-04-21T08:38:46.638786+0000","last_became_active":"2024-04-21T08:36:20.913097+0000","last_became_peered":"2024-04-21T08:36:20.913097+0000","last_unstale":"2024-04-21T08:38:46.638786+0000","last_undegraded":"2024-04-21T08:38:46.638786+0000","last_fullsized":"2024-04-21T08:38:46.638786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:08:40.388370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638662+0000","last_change":"2024-04-21T08:36:20.906624+0000","last_active":"2024-04-21T08:38:46.638662+0000","last_peered":"2024-04-21T08:38:46.638662+0000","last_clean":"2024-04-21T08:38:46.638662+0000","last_became_active":"2024-04-21T08:36:20.906247+0000","last_became_peered":"2024-04-21T08:36:20.906247+0000","last_unstale":"2024-04-21T08:38:46.638662+0000","last_undegraded":"2024-04-21T08:38:46.638662+0000","last_fullsized":"2024-04-21T08:38:46.638662+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:44:11.855183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.916219+0000","last_change":"2024-04-21T08:36:20.906576+0000","last_active":"2024-04-21T08:38:46.916219+0000","last_peered":"2024-04-21T08:38:46.916219+0000","last_clean":"2024-04-21T08:38:46.916219+0000","last_became_active":"2024-04-21T08:36:20.906231+0000","last_became_peered":"2024-04-21T08:36:20.906231+0000","last_unstale":"2024-04-21T08:38:46.916219+0000","last_undegraded":"2024-04-21T08:38:46.916219+0000","last_fullsized":"2024-04-21T08:38:46.916219+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:42:48.156614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.912593+0000","last_change":"2024-04-21T08:36:20.902551+0000","last_active":"2024-04-21T08:38:46.912593+0000","last_peered":"2024-04-21T08:38:46.912593+0000","last_clean":"2024-04-21T08:38:46.912593+0000","last_became_active":"2024-04-21T08:36:20.902208+0000","last_became_peered":"2024-04-21T08:36:20.902208+0000","last_unstale":"2024-04-21T08:38:46.912593+0000","last_undegraded":"2024-04-21T08:38:46.912593+0000","last_fullsized":"2024-04-21T08:38:46.912593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:36:30.016256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807607+0000","last_change":"2024-04-21T08:36:20.918588+0000","last_active":"2024-04-21T08:38:46.807607+0000","last_peered":"2024-04-21T08:38:46.807607+0000","last_clean":"2024-04-21T08:38:46.807607+0000","last_became_active":"2024-04-21T08:36:20.918263+0000","last_became_peered":"2024-04-21T08:36:20.918263+0000","last_unstale":"2024-04-21T08:38:46.807607+0000","last_undegraded":"2024-04-21T08:38:46.807607+0000","last_fullsized":"2024-04-21T08:38:46.807607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:51:30.340336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639167+0000","last_change":"2024-04-21T08:36:20.911063+0000","last_active":"2024-04-21T08:38:46.639167+0000","last_peered":"2024-04-21T08:38:46.639167+0000","last_clean":"2024-04-21T08:38:46.639167+0000","last_became_active":"2024-04-21T08:36:20.910647+0000","last_became_peered":"2024-04-21T08:36:20.910647+0000","last_unstale":"2024-04-21T08:38:46.639167+0000","last_undegraded":"2024-04-21T08:38:46.639167+0000","last_fullsized":"2024-04-21T08:38:46.639167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:10:31.079782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639213+0000","last_change":"2024-04-21T08:36:20.914079+0000","last_active":"2024-04-21T08:38:46.639213+0000","last_peered":"2024-04-21T08:38:46.639213+0000","last_clean":"2024-04-21T08:38:46.639213+0000","last_became_active":"2024-04-21T08:36:20.912851+0000","last_became_peered":"2024-04-21T08:36:20.912851+0000","last_unstale":"2024-04-21T08:38:46.639213+0000","last_undegraded":"2024-04-21T08:38:46.639213+0000","last_fullsized":"2024-04-21T08:38:46.639213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:53:35.811281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751071+0000","last_change":"2024-04-21T08:36:20.900551+0000","last_active":"2024-04-21T08:38:46.751071+0000","last_peered":"2024-04-21T08:38:46.751071+0000","last_clean":"2024-04-21T08:38:46.751071+0000","last_became_active":"2024-04-21T08:36:20.900173+0000","last_became_peered":"2024-04-21T08:36:20.900173+0000","last_unstale":"2024-04-21T08:38:46.751071+0000","last_undegraded":"2024-04-21T08:38:46.751071+0000","last_fullsized":"2024-04-21T08:38:46.751071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:21.389498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638564+0000","last_change":"2024-04-21T08:36:20.912953+0000","last_active":"2024-04-21T08:38:46.638564+0000","last_peered":"2024-04-21T08:38:46.638564+0000","last_clean":"2024-04-21T08:38:46.638564+0000","last_became_active":"2024-04-21T08:36:20.912223+0000","last_became_peered":"2024-04-21T08:36:20.912223+0000","last_unstale":"2024-04-21T08:38:46.638564+0000","last_undegraded":"2024-04-21T08:38:46.638564+0000","last_fullsized":"2024-04-21T08:38:46.638564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:43:14.277179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.922633+0000","last_change":"2024-04-21T08:36:20.906164+0000","last_active":"2024-04-21T08:38:46.922633+0000","last_peered":"2024-04-21T08:38:46.922633+0000","last_clean":"2024-04-21T08:38:46.922633+0000","last_became_active":"2024-04-21T08:36:20.905749+0000","last_became_peered":"2024-04-21T08:36:20.905749+0000","last_unstale":"2024-04-21T08:38:46.922633+0000","last_undegraded":"2024-04-21T08:38:46.922633+0000","last_fullsized":"2024-04-21T08:38:46.922633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:02:10.569087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913644+0000","last_change":"2024-04-21T08:36:20.900314+0000","last_active":"2024-04-21T08:38:46.913644+0000","last_peered":"2024-04-21T08:38:46.913644+0000","last_clean":"2024-04-21T08:38:46.913644+0000","last_became_active":"2024-04-21T08:36:20.900009+0000","last_became_peered":"2024-04-21T08:36:20.900009+0000","last_unstale":"2024-04-21T08:38:46.913644+0000","last_undegraded":"2024-04-21T08:38:46.913644+0000","last_fullsized":"2024-04-21T08:38:46.913644+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:19:59.997302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807555+0000","last_change":"2024-04-21T08:36:20.920725+0000","last_active":"2024-04-21T08:38:46.807555+0000","last_peered":"2024-04-21T08:38:46.807555+0000","last_clean":"2024-04-21T08:38:46.807555+0000","last_became_active":"2024-04-21T08:36:20.920288+0000","last_became_peered":"2024-04-21T08:36:20.920288+0000","last_unstale":"2024-04-21T08:38:46.807555+0000","last_undegraded":"2024-04-21T08:38:46.807555+0000","last_fullsized":"2024-04-21T08:38:46.807555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:36:18.736388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":136,"num_read_kb":119,"num_write":181,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":137,"ondisk_log_size":137,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":603992,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1350,"num_read_kb":791,"num_write":375,"num_write_kb":1473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":806,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1811976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":305,"ondisk_log_size":305,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383722,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1093632,"data_stored":531299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645358,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28764,"kb_used_data":1744,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716388,"statfs":{"total":95995035648,"available":95965581312,"internally_reserved":0,"allocated":1785856,"data_stored":1190803,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808884,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36420,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708732,"statfs":{"total":95995035648,"available":95957741568,"internally_reserved":0,"allocated":1105920,"data_stored":530003,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070521,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28476,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716676,"statfs":{"total":95995035648,"available":95965876224,"internally_reserved":0,"allocated":1490944,"data_stored":923575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332159,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28672,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716480,"statfs":{"total":95995035648,"available":95965675520,"internally_reserved":0,"allocated":1691648,"data_stored":1127969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495684,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1081344,"data_stored":525507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574940,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36476,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708676,"statfs":{"total":95995035648,"available":95957684224,"internally_reserved":0,"allocated":1163264,"data_stored":596717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738465,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1105920,"data_stored":527495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":1,"apply_latency_ms":1,"commit_latency_ns":1000000,"apply_latency_ns":1000000},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411415,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36480,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708672,"statfs":{"total":95995035648,"available":95957680128,"internally_reserved":0,"allocated":1167360,"data_stored":598085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182482,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1050031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920846,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1089536,"data_stored":530895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757320,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":33156,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":31166,"kb_avail":93711996,"statfs":{"total":95995035648,"available":95961083904,"internally_reserved":0,"allocated":1843200,"data_stored":1254509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":31914437},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":71932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":73300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":394694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":204718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T08:38:50.038 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:38:50.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.171+0000 7f7540700640 1 -- 172.21.15.86:0/794124815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75381020a0 msgr2=0x7f7538102520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:50.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.171+0000 7f7540700640 1 --2- 172.21.15.86:0/794124815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75381020a0 0x7f7538102520 secure :-1 s=READY pgs=530 cs=0 l=1 rev1=1 crypto rx=0x7f752c009900 tx=0x7f752c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:50.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.171+0000 7f7540700640 1 -- 172.21.15.86:0/794124815 shutdown_connections 2024-04-21T08:38:50.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.171+0000 7f7540700640 1 --2- 172.21.15.86:0/794124815 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7538102a60 0x7f75381092f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.171+0000 7f7540700640 1 --2- 172.21.15.86:0/794124815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f75381020a0 0x7f7538102520 unknown :-1 s=CLOSED pgs=530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.171+0000 7f7540700640 1 --2- 172.21.15.86:0/794124815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7538101390 0x7f75381017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.171+0000 7f7540700640 1 -- 172.21.15.86:0/794124815 >> 172.21.15.86:0/794124815 conn(0x7f75380fbb80 msgr2=0x7f75380fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:50.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.171+0000 7f7540700640 1 -- 172.21.15.86:0/794124815 shutdown_connections 2024-04-21T08:38:50.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.171+0000 7f7540700640 1 -- 172.21.15.86:0/794124815 wait complete. 2024-04-21T08:38:50.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.172+0000 7f7540700640 1 Processor -- start 2024-04-21T08:38:50.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.172+0000 7f7540700640 1 -- start start 2024-04-21T08:38:50.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.172+0000 7f7540700640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7538101390 0x7f75381970f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:50.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.172+0000 7f7540700640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75381020a0 0x7f7538197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:50.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.172+0000 7f7540700640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7538102a60 0x7f753819e6a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:50.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.172+0000 7f7540700640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f753810c9c0 con 0x7f7538101390 2024-04-21T08:38:50.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.172+0000 7f7540700640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f753810c840 con 0x7f75381020a0 2024-04-21T08:38:50.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.172+0000 7f7540700640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f753810cb40 con 0x7f7538102a60 2024-04-21T08:38:50.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.173+0000 7f753e49c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7538101390 0x7f75381970f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:50.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.173+0000 7f753e49c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7538101390 0x7f75381970f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36030/0 (socket says 172.21.15.86:36030) 2024-04-21T08:38:50.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.173+0000 7f753e49c640 1 -- 172.21.15.86:0/407883413 learned_addr learned my addr 172.21.15.86:0/407883413 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:50.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.173+0000 7f753dc9b640 1 --2- 172.21.15.86:0/407883413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75381020a0 0x7f7538197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:50.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.173+0000 7f753ec9d640 1 --2- 172.21.15.86:0/407883413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7538102a60 0x7f753819e6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:50.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.173+0000 7f753e49c640 1 -- 172.21.15.86:0/407883413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7538102a60 msgr2=0x7f753819e6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:50.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.173+0000 7f753e49c640 1 --2- 172.21.15.86:0/407883413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7538102a60 0x7f753819e6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.173+0000 7f753e49c640 1 -- 172.21.15.86:0/407883413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75381020a0 msgr2=0x7f7538197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:50.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.173+0000 7f753e49c640 1 --2- 172.21.15.86:0/407883413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75381020a0 0x7f7538197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.173+0000 7f753e49c640 1 -- 172.21.15.86:0/407883413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f753819ed10 con 0x7f7538101390 2024-04-21T08:38:50.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.173+0000 7f753ec9d640 1 --2- 172.21.15.86:0/407883413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7538102a60 0x7f753819e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:50.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.173+0000 7f753e49c640 1 --2- 172.21.15.86:0/407883413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7538101390 0x7f75381970f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f752800eb70 tx=0x7f752800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:50.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.173+0000 7f75377fe640 1 -- 172.21.15.86:0/407883413 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7528003b70 con 0x7f7538101390 2024-04-21T08:38:50.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.173+0000 7f75377fe640 1 -- 172.21.15.86:0/407883413 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f752800ee70 con 0x7f7538101390 2024-04-21T08:38:50.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.173+0000 7f7540700640 1 -- 172.21.15.86:0/407883413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f753819efa0 con 0x7f7538101390 2024-04-21T08:38:50.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.173+0000 7f75377fe640 1 -- 172.21.15.86:0/407883413 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7528010650 con 0x7f7538101390 2024-04-21T08:38:50.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.173+0000 7f7540700640 1 -- 172.21.15.86:0/407883413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f753819f460 con 0x7f7538101390 2024-04-21T08:38:50.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.174+0000 7f7540700640 1 -- 172.21.15.86:0/407883413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f75381082a0 con 0x7f7538101390 2024-04-21T08:38:50.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.176+0000 7f75377fe640 1 -- 172.21.15.86:0/407883413 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7f7528004a40 con 0x7f7538101390 2024-04-21T08:38:50.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.177+0000 7f75377fe640 1 --2- 172.21.15.86:0/407883413 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f75200a3da0 0x7f75200a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:50.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.177+0000 7f753dc9b640 1 --2- 172.21.15.86:0/407883413 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f75200a3da0 0x7f75200a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:50.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.177+0000 7f75377fe640 1 -- 172.21.15.86:0/407883413 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f75280d12c0 con 0x7f7538101390 2024-04-21T08:38:50.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.177+0000 7f753dc9b640 1 --2- 172.21.15.86:0/407883413 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f75200a3da0 0x7f75200a6260 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f752c009230 tx=0x7f752c005fd0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:50.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.181+0000 7f75377fe640 1 -- 172.21.15.86:0/407883413 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f752809d5b0 con 0x7f7538101390 2024-04-21T08:38:50.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.379+0000 7f7540700640 1 -- 172.21.15.86:0/407883413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7538062b80 con 0x7f7538101390 2024-04-21T08:38:50.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.381+0000 7f75377fe640 1 -- 172.21.15.86:0/407883413 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+25067 (secure 0 0 0) 0x7f752809ceb0 con 0x7f7538101390 2024-04-21T08:38:50.381 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:38:50.381 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":94,"fsid":"f8cec7cc-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:22.687329+0000","modified":"2024-04-21T08:38:46.631676+0000","last_up_change":"2024-04-21T08:35:30.048315+0000","last_in_change":"2024-04-21T08:35:10.099496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:45.279109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:18.300782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:19.574636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:22.905632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89b7601f-0620-4db2-90f1-b45cea173738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6803","nonce":2571516897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6805","nonce":2571516897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6809","nonce":2571516897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6807","nonce":2571516897}]},"public_addr":"172.21.15.86:6803/2571516897","cluster_addr":"172.21.15.86:6805/2571516897","heartbeat_back_addr":"172.21.15.86:6809/2571516897","heartbeat_front_addr":"172.21.15.86:6807/2571516897","state":["exists","up"]},{"osd":1,"uuid":"4891031c-f98c-42bc-bb3b-a420855ecc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6801","nonce":74067182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6803","nonce":74067182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6807","nonce":74067182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6805","nonce":74067182}]},"public_addr":"172.21.15.88:6801/74067182","cluster_addr":"172.21.15.88:6803/74067182","heartbeat_back_addr":"172.21.15.88:6807/74067182","heartbeat_front_addr":"172.21.15.88:6805/74067182","state":["exists","up"]},{"osd":2,"uuid":"60d12aa4-9312-4fa2-8d0b-2b3f42caad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6801","nonce":4137659051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6803","nonce":4137659051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6807","nonce":4137659051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6805","nonce":4137659051}]},"public_addr":"172.21.15.179:6801/4137659051","cluster_addr":"172.21.15.179:6803/4137659051","heartbeat_back_addr":"172.21.15.179:6807/4137659051","heartbeat_front_addr":"172.21.15.179:6805/4137659051","state":["exists","up"]},{"osd":3,"uuid":"404c3158-8cb0-4c02-b48a-f9ee5ff7b784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6811","nonce":2309603148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6813","nonce":2309603148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6817","nonce":2309603148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6815","nonce":2309603148}]},"public_addr":"172.21.15.86:6811/2309603148","cluster_addr":"172.21.15.86:6813/2309603148","heartbeat_back_addr":"172.21.15.86:6817/2309603148","heartbeat_front_addr":"172.21.15.86:6815/2309603148","state":["exists","up"]},{"osd":4,"uuid":"458ff202-d781-4ec7-ac93-0b49f835b503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6809","nonce":1818714217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6811","nonce":1818714217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6815","nonce":1818714217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6813","nonce":1818714217}]},"public_addr":"172.21.15.88:6809/1818714217","cluster_addr":"172.21.15.88:6811/1818714217","heartbeat_back_addr":"172.21.15.88:6815/1818714217","heartbeat_front_addr":"172.21.15.88:6813/1818714217","state":["exists","up"]},{"osd":5,"uuid":"8123cc90-bd8b-4204-bf31-46b34e9f589d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6809","nonce":863840990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6811","nonce":863840990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6815","nonce":863840990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6813","nonce":863840990}]},"public_addr":"172.21.15.179:6809/863840990","cluster_addr":"172.21.15.179:6811/863840990","heartbeat_back_addr":"172.21.15.179:6815/863840990","heartbeat_front_addr":"172.21.15.179:6813/863840990","state":["exists","up"]},{"osd":6,"uuid":"aac9d7c7-659a-4110-903a-a871c086066f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6819","nonce":1224231118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6821","nonce":1224231118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6825","nonce":1224231118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6823","nonce":1224231118}]},"public_addr":"172.21.15.86:6819/1224231118","cluster_addr":"172.21.15.86:6821/1224231118","heartbeat_back_addr":"172.21.15.86:6825/1224231118","heartbeat_front_addr":"172.21.15.86:6823/1224231118","state":["exists","up"]},{"osd":7,"uuid":"df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6817","nonce":136629878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6819","nonce":136629878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6823","nonce":136629878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6821","nonce":136629878}]},"public_addr":"172.21.15.88:6817/136629878","cluster_addr":"172.21.15.88:6819/136629878","heartbeat_back_addr":"172.21.15.88:6823/136629878","heartbeat_front_addr":"172.21.15.88:6821/136629878","state":["exists","up"]},{"osd":8,"uuid":"63e59f4b-0426-46db-92f7-c55dae61b5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6817","nonce":1770418957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6819","nonce":1770418957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6823","nonce":1770418957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6821","nonce":1770418957}]},"public_addr":"172.21.15.179:6817/1770418957","cluster_addr":"172.21.15.179:6819/1770418957","heartbeat_back_addr":"172.21.15.179:6823/1770418957","heartbeat_front_addr":"172.21.15.179:6821/1770418957","state":["exists","up"]},{"osd":9,"uuid":"5841bb36-ed6b-45d8-b857-eeab139c44e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6827","nonce":1108500274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6829","nonce":1108500274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6833","nonce":1108500274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6831","nonce":1108500274}]},"public_addr":"172.21.15.86:6827/1108500274","cluster_addr":"172.21.15.86:6829/1108500274","heartbeat_back_addr":"172.21.15.86:6833/1108500274","heartbeat_front_addr":"172.21.15.86:6831/1108500274","state":["exists","up"]},{"osd":10,"uuid":"f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6825","nonce":1294275353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6827","nonce":1294275353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6831","nonce":1294275353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6829","nonce":1294275353}]},"public_addr":"172.21.15.88:6825/1294275353","cluster_addr":"172.21.15.88:6827/1294275353","heartbeat_back_addr":"172.21.15.88:6831/1294275353","heartbeat_front_addr":"172.21.15.88:6829/1294275353","state":["exists","up"]},{"osd":11,"uuid":"e0d4cc76-e7a6-4a6b-909c-95488ccd5a36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6825","nonce":4007070233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6827","nonce":4007070233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6831","nonce":4007070233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6829","nonce":4007070233}]},"public_addr":"172.21.15.179:6825/4007070233","cluster_addr":"172.21.15.179:6827/4007070233","heartbeat_back_addr":"172.21.15.179:6831/4007070233","heartbeat_front_addr":"172.21.15.179:6829/4007070233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:55.210605+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:18.723090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:42.547979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:06.421622+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:30.858315+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:55.455313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.623369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:46.185111+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.103627+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:37.297545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:02.637431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:28.199292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1068089552":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/4273877219":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/2707594358":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/4060044103":"2024-04-22T08:38:46.631621+0000","172.21.15.86:6800/1364156965":"2024-04-22T08:38:46.631621+0000","172.21.15.86:6800/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/1364156965":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/1412393225":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6800/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/1687852404":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2020707404":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2568130933":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/3952607648":"2024-04-22T08:28:57.384403+0000","172.21.15.86:6801/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/2756903426":"2024-04-22T08:29:23.190001+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:38:50.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.385+0000 7f7540700640 1 -- 172.21.15.86:0/407883413 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f75200a3da0 msgr2=0x7f75200a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:50.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.385+0000 7f7540700640 1 --2- 172.21.15.86:0/407883413 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f75200a3da0 0x7f75200a6260 secure :-1 s=READY pgs=43 cs=0 l=1 rev1=1 crypto rx=0x7f752c009230 tx=0x7f752c005fd0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.385+0000 7f7540700640 1 -- 172.21.15.86:0/407883413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7538101390 msgr2=0x7f75381970f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:50.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.385+0000 7f7540700640 1 --2- 172.21.15.86:0/407883413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7538101390 0x7f75381970f0 secure :-1 s=READY pgs=531 cs=0 l=1 rev1=1 crypto rx=0x7f752800eb70 tx=0x7f752800c6a0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.385+0000 7f7540700640 1 -- 172.21.15.86:0/407883413 shutdown_connections 2024-04-21T08:38:50.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.385+0000 7f7540700640 1 --2- 172.21.15.86:0/407883413 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f75200a3da0 0x7f75200a6260 unknown :-1 s=CLOSED pgs=43 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.385+0000 7f7540700640 1 --2- 172.21.15.86:0/407883413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7538102a60 0x7f753819e6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.385+0000 7f7540700640 1 --2- 172.21.15.86:0/407883413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f75381020a0 0x7f7538197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.385+0000 7f7540700640 1 --2- 172.21.15.86:0/407883413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7538101390 0x7f75381970f0 unknown :-1 s=CLOSED pgs=531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.384 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.385+0000 7f7540700640 1 -- 172.21.15.86:0/407883413 >> 172.21.15.86:0/407883413 conn(0x7f75380fbb80 msgr2=0x7f7538105470 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:50.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.385+0000 7f7540700640 1 -- 172.21.15.86:0/407883413 shutdown_connections 2024-04-21T08:38:50.385 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.385+0000 7f7540700640 1 -- 172.21.15.86:0/407883413 wait complete. 2024-04-21T08:38:50.399 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:31:45.279109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:36:18.300782+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:36:19.574636+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:36:22.905632+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:38:50.400 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:38:50.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.507+0000 7f1806832640 1 -- 172.21.15.86:0/1264176139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1800105400 msgr2=0x7f18001077f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:50.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.507+0000 7f1806832640 1 --2- 172.21.15.86:0/1264176139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1800105400 0x7f18001077f0 secure :-1 s=READY pgs=532 cs=0 l=1 rev1=1 crypto rx=0x7f17f0009900 tx=0x7f17f002bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:50.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.507+0000 7f1806832640 1 -- 172.21.15.86:0/1264176139 shutdown_connections 2024-04-21T08:38:50.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.507+0000 7f1806832640 1 --2- 172.21.15.86:0/1264176139 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1800107d30 0x7f180010a120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.507+0000 7f1806832640 1 --2- 172.21.15.86:0/1264176139 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1800105400 0x7f18001077f0 unknown :-1 s=CLOSED pgs=532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.507+0000 7f1806832640 1 --2- 172.21.15.86:0/1264176139 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1800102ad0 0x7f1800104ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.507+0000 7f1806832640 1 -- 172.21.15.86:0/1264176139 >> 172.21.15.86:0/1264176139 conn(0x7f18000fc440 msgr2=0x7f18000fe860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:50.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.507+0000 7f1806832640 1 -- 172.21.15.86:0/1264176139 shutdown_connections 2024-04-21T08:38:50.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.507+0000 7f1806832640 1 -- 172.21.15.86:0/1264176139 wait complete. 2024-04-21T08:38:50.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.507+0000 7f1806832640 1 Processor -- start 2024-04-21T08:38:50.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.507+0000 7f1806832640 1 -- start start 2024-04-21T08:38:50.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.508+0000 7f1806832640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1800102ad0 0x7f18001981d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:50.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.508+0000 7f1806832640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1800105400 0x7f1800198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:50.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.508+0000 7f1806832640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1800107d30 0x7f180019f780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:50.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.508+0000 7f1806832640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f180010db20 con 0x7f1800107d30 2024-04-21T08:38:50.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.508+0000 7f1806832640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f180010d9a0 con 0x7f1800105400 2024-04-21T08:38:50.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.508+0000 7f1806832640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f180010dca0 con 0x7f1800102ad0 2024-04-21T08:38:50.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.508+0000 7f1804dcf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1800107d30 0x7f180019f780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:50.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.508+0000 7f17fffff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1800102ad0 0x7f18001981d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:50.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.508+0000 7f1804dcf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1800107d30 0x7f180019f780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36050/0 (socket says 172.21.15.86:36050) 2024-04-21T08:38:50.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.508+0000 7f1804dcf640 1 -- 172.21.15.86:0/567611901 learned_addr learned my addr 172.21.15.86:0/567611901 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:50.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.508+0000 7f17ff7fe640 1 --2- 172.21.15.86:0/567611901 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1800105400 0x7f1800198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:50.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.508+0000 7f1804dcf640 1 -- 172.21.15.86:0/567611901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1800102ad0 msgr2=0x7f18001981d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:50.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.508+0000 7f1804dcf640 1 --2- 172.21.15.86:0/567611901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1800102ad0 0x7f18001981d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.508+0000 7f1804dcf640 1 -- 172.21.15.86:0/567611901 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1800105400 msgr2=0x7f1800198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:50.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.508+0000 7f1804dcf640 1 --2- 172.21.15.86:0/567611901 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1800105400 0x7f1800198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.508+0000 7f1804dcf640 1 -- 172.21.15.86:0/567611901 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f180019fdf0 con 0x7f1800107d30 2024-04-21T08:38:50.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.508+0000 7f17ff7fe640 1 --2- 172.21.15.86:0/567611901 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1800105400 0x7f1800198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:38:50.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.508+0000 7f1804dcf640 1 --2- 172.21.15.86:0/567611901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1800107d30 0x7f180019f780 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f17f800eb10 tx=0x7f17f800ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:50.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.509+0000 7f17fd7fa640 1 -- 172.21.15.86:0/567611901 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f800ce20 con 0x7f1800107d30 2024-04-21T08:38:50.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.509+0000 7f1806832640 1 -- 172.21.15.86:0/567611901 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f18001a00e0 con 0x7f1800107d30 2024-04-21T08:38:50.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.509+0000 7f17fd7fa640 1 -- 172.21.15.86:0/567611901 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f17f8005020 con 0x7f1800107d30 2024-04-21T08:38:50.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.509+0000 7f17fd7fa640 1 -- 172.21.15.86:0/567611901 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f17f8010690 con 0x7f1800107d30 2024-04-21T08:38:50.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.509+0000 7f1806832640 1 -- 172.21.15.86:0/567611901 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f18001a0600 con 0x7f1800107d30 2024-04-21T08:38:50.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.510+0000 7f1806832640 1 -- 172.21.15.86:0/567611901 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f18000fdf30 con 0x7f1800107d30 2024-04-21T08:38:50.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.511+0000 7f17fd7fa640 1 -- 172.21.15.86:0/567611901 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7f17f8005510 con 0x7f1800107d30 2024-04-21T08:38:50.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.512+0000 7f17fd7fa640 1 --2- 172.21.15.86:0/567611901 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f17e80a3d20 0x7f17e80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:50.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.512+0000 7f17fffff640 1 --2- 172.21.15.86:0/567611901 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f17e80a3d20 0x7f17e80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:50.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.512+0000 7f17fd7fa640 1 -- 172.21.15.86:0/567611901 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f17f80d10f0 con 0x7f1800107d30 2024-04-21T08:38:50.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.512+0000 7f17fffff640 1 --2- 172.21.15.86:0/567611901 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f17e80a3d20 0x7f17e80a61e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f17f400abe0 tx=0x7f17f400a2d0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:50.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.516+0000 7f17fd7fa640 1 -- 172.21.15.86:0/567611901 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f17f809d420 con 0x7f1800107d30 2024-04-21T08:38:50.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:50 smithi086 ceph-mon[27939]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:38:50.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:50 smithi086 ceph-mon[27939]: from='client.15699 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:50.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:50 smithi086 ceph-mon[27939]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-21T08:38:50.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:50 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:50.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:50 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:50.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:50 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:50.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:50 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:50.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:50 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:50.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:50 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:50.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:50 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:50.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:50 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:50.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:50 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:50.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:50 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:50.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:50 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:50.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:50 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:50.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:50 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:50.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:50 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:50.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:50 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:50.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:50 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:50.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:50 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:50.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:50 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:50.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:50 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:50.619 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:50 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:38:50.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:50 smithi088 ceph-mon[31259]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:38:50.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:50 smithi088 ceph-mon[31259]: from='client.15699 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:50.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:50 smithi088 ceph-mon[31259]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-21T08:38:50.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:50 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:50.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:50 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:50.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:50 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:50.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:50 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:50.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:50 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:50.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:50 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:50.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:50 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:50.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:50 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:50.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:50 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:50.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:50 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:50.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:50 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:50.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:50 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:50.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:50 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:50.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:50 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:50.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:50 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:50.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:50 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:50.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:50 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:50.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:50 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:50.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:50 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:50.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:50 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:38:50.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.721+0000 7f1806832640 1 -- 172.21.15.86:0/567611901 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f1800062b50 con 0x7f1800107d30 2024-04-21T08:38:50.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.722+0000 7f17fd7fa640 1 -- 172.21.15.86:0/567611901 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f17f809cd20 con 0x7f1800107d30 2024-04-21T08:38:50.721 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-21T08:38:50.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.724+0000 7f1806832640 1 -- 172.21.15.86:0/567611901 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f17e80a3d20 msgr2=0x7f17e80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:50.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.724+0000 7f1806832640 1 --2- 172.21.15.86:0/567611901 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f17e80a3d20 0x7f17e80a61e0 secure :-1 s=READY pgs=44 cs=0 l=1 rev1=1 crypto rx=0x7f17f400abe0 tx=0x7f17f400a2d0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.724+0000 7f1806832640 1 -- 172.21.15.86:0/567611901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1800107d30 msgr2=0x7f180019f780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:50.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.724+0000 7f1806832640 1 --2- 172.21.15.86:0/567611901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1800107d30 0x7f180019f780 secure :-1 s=READY pgs=533 cs=0 l=1 rev1=1 crypto rx=0x7f17f800eb10 tx=0x7f17f800ef10 comp rx=0 tx=0).stop 2024-04-21T08:38:50.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.724+0000 7f1806832640 1 -- 172.21.15.86:0/567611901 shutdown_connections 2024-04-21T08:38:50.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.724+0000 7f1806832640 1 --2- 172.21.15.86:0/567611901 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f17e80a3d20 0x7f17e80a61e0 unknown :-1 s=CLOSED pgs=44 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.724+0000 7f1806832640 1 --2- 172.21.15.86:0/567611901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1800107d30 0x7f180019f780 unknown :-1 s=CLOSED pgs=533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.724+0000 7f1806832640 1 --2- 172.21.15.86:0/567611901 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1800105400 0x7f1800198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.724+0000 7f1806832640 1 --2- 172.21.15.86:0/567611901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1800102ad0 0x7f18001981d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.724+0000 7f1806832640 1 -- 172.21.15.86:0/567611901 >> 172.21.15.86:0/567611901 conn(0x7f18000fc440 msgr2=0x7f18001034f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:50.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.724+0000 7f1806832640 1 -- 172.21.15.86:0/567611901 shutdown_connections 2024-04-21T08:38:50.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.724+0000 7f1806832640 1 -- 172.21.15.86:0/567611901 wait complete. 2024-04-21T08:38:50.737 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:38:50.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:50 smithi179 ceph-mon[31829]: pgmap v4: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:38:50.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:50 smithi179 ceph-mon[31829]: from='client.15699 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:50.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:50 smithi179 ceph-mon[31829]: mgrmap e23: x(active, since 2s), standbys: z, y 2024-04-21T08:38:50.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:50 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:50.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:50 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:50.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:50 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:50.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:50 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:50.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:50 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:50.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:50 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:50.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:50 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:50.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:50 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:50.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:50 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:50.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:50 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:50.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:50 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:50.748 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:50 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:50.748 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:50 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:50.748 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:50 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:50.748 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:50 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:50.748 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:50 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:50.748 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:50 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:50.748 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:50 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:38:50.748 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:50 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:50.748 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:50 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:38:50.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.841+0000 7f2c0a138640 1 -- 172.21.15.86:0/1749934487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c04105e20 msgr2=0x7f2c04106200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:50.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.841+0000 7f2c0a138640 1 --2- 172.21.15.86:0/1749934487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c04105e20 0x7f2c04106200 secure :-1 s=READY pgs=534 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8009930 tx=0x7f2bf802bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:50.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.841+0000 7f2c0a138640 1 -- 172.21.15.86:0/1749934487 shutdown_connections 2024-04-21T08:38:50.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.841+0000 7f2c0a138640 1 --2- 172.21.15.86:0/1749934487 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c04102350 0x7f2c0410dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.841+0000 7f2c0a138640 1 --2- 172.21.15.86:0/1749934487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c04101990 0x7f2c04101e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.841+0000 7f2c0a138640 1 --2- 172.21.15.86:0/1749934487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c04105e20 0x7f2c04106200 unknown :-1 s=CLOSED pgs=534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.841+0000 7f2c0a138640 1 -- 172.21.15.86:0/1749934487 >> 172.21.15.86:0/1749934487 conn(0x7f2c040760e0 msgr2=0x7f2c040764f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:50.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.841+0000 7f2c0a138640 1 -- 172.21.15.86:0/1749934487 shutdown_connections 2024-04-21T08:38:50.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.842+0000 7f2c0a138640 1 -- 172.21.15.86:0/1749934487 wait complete. 2024-04-21T08:38:50.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.842+0000 7f2c0a138640 1 Processor -- start 2024-04-21T08:38:50.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.842+0000 7f2c0a138640 1 -- start start 2024-04-21T08:38:50.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.842+0000 7f2c0a138640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c04101990 0x7f2c04199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:50.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.842+0000 7f2c0a138640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c04102350 0x7f2c04199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:50.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.842+0000 7f2c0a138640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c04105e20 0x7f2c041a4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:50.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.842+0000 7f2c0a138640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2c041110f0 con 0x7f2c04101990 2024-04-21T08:38:50.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.842+0000 7f2c0a138640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2c04110f70 con 0x7f2c04105e20 2024-04-21T08:38:50.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.842+0000 7f2c0a138640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2c04111270 con 0x7f2c04102350 2024-04-21T08:38:50.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.843+0000 7f2c037fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c04101990 0x7f2c04199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:50.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.843+0000 7f2c037fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c04101990 0x7f2c04199510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36070/0 (socket says 172.21.15.86:36070) 2024-04-21T08:38:50.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.843+0000 7f2c037fe640 1 -- 172.21.15.86:0/1294973919 learned_addr learned my addr 172.21.15.86:0/1294973919 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:50.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.843+0000 7f2c02ffd640 1 --2- 172.21.15.86:0/1294973919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c04102350 0x7f2c04199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:50.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.843+0000 7f2c03fff640 1 --2- 172.21.15.86:0/1294973919 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c04105e20 0x7f2c041a4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:50.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.843+0000 7f2c037fe640 1 -- 172.21.15.86:0/1294973919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c04102350 msgr2=0x7f2c04199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:50.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.843+0000 7f2c037fe640 1 --2- 172.21.15.86:0/1294973919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c04102350 0x7f2c04199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.843+0000 7f2c037fe640 1 -- 172.21.15.86:0/1294973919 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c04105e20 msgr2=0x7f2c041a4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:50.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.843+0000 7f2c037fe640 1 --2- 172.21.15.86:0/1294973919 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c04105e20 0x7f2c041a4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:50.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.843+0000 7f2c037fe640 1 -- 172.21.15.86:0/1294973919 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2c041a4ad0 con 0x7f2c04101990 2024-04-21T08:38:50.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.843+0000 7f2c02ffd640 1 --2- 172.21.15.86:0/1294973919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c04102350 0x7f2c04199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:38:50.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.843+0000 7f2c03fff640 1 --2- 172.21.15.86:0/1294973919 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c04105e20 0x7f2c041a4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:38:50.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.843+0000 7f2c037fe640 1 --2- 172.21.15.86:0/1294973919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c04101990 0x7f2c04199510 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8005ab0 tx=0x7f2bf8005e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:50.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.843+0000 7f2c00ff9640 1 -- 172.21.15.86:0/1294973919 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf8043020 con 0x7f2c04101990 2024-04-21T08:38:50.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.843+0000 7f2c0a138640 1 -- 172.21.15.86:0/1294973919 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2c041a4ca0 con 0x7f2c04101990 2024-04-21T08:38:50.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.843+0000 7f2c00ff9640 1 -- 172.21.15.86:0/1294973919 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bf802ca90 con 0x7f2c04101990 2024-04-21T08:38:50.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.843+0000 7f2c00ff9640 1 -- 172.21.15.86:0/1294973919 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bf80039e0 con 0x7f2c04101990 2024-04-21T08:38:50.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.843+0000 7f2c0a138640 1 -- 172.21.15.86:0/1294973919 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2c041a51c0 con 0x7f2c04101990 2024-04-21T08:38:50.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.844+0000 7f2c0a138640 1 -- 172.21.15.86:0/1294973919 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2c040691c0 con 0x7f2c04101990 2024-04-21T08:38:50.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.846+0000 7f2c00ff9640 1 -- 172.21.15.86:0/1294973919 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7f2bf8046020 con 0x7f2c04101990 2024-04-21T08:38:50.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.846+0000 7f2c00ff9640 1 --2- 172.21.15.86:0/1294973919 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2bec0a3d20 0x7f2bec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:50.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.846+0000 7f2c00ff9640 1 -- 172.21.15.86:0/1294973919 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f2bf80f2d90 con 0x7f2c04101990 2024-04-21T08:38:50.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.846+0000 7f2c02ffd640 1 --2- 172.21.15.86:0/1294973919 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2bec0a3d20 0x7f2bec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:50.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.847+0000 7f2c02ffd640 1 --2- 172.21.15.86:0/1294973919 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2bec0a3d20 0x7f2bec0a61e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2c04101430 tx=0x7f2bf4005f90 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:50.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:50.850+0000 7f2c00ff9640 1 -- 172.21.15.86:0/1294973919 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2bf804d020 con 0x7f2c04101990 2024-04-21T08:38:51.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.048+0000 7f2c0a138640 1 -- 172.21.15.86:0/1294973919 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f2c04108da0 con 0x7f2c04101990 2024-04-21T08:38:51.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.048+0000 7f2c00ff9640 1 -- 172.21.15.86:0/1294973919 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f2bf80bf1d0 con 0x7f2c04101990 2024-04-21T08:38:51.048 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-21T08:38:51.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.050+0000 7f2c0a138640 1 -- 172.21.15.86:0/1294973919 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2bec0a3d20 msgr2=0x7f2bec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:51.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.050+0000 7f2c0a138640 1 --2- 172.21.15.86:0/1294973919 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2bec0a3d20 0x7f2bec0a61e0 secure :-1 s=READY pgs=45 cs=0 l=1 rev1=1 crypto rx=0x7f2c04101430 tx=0x7f2bf4005f90 comp rx=0 tx=0).stop 2024-04-21T08:38:51.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.051+0000 7f2c0a138640 1 -- 172.21.15.86:0/1294973919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c04101990 msgr2=0x7f2c04199510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:51.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.051+0000 7f2c0a138640 1 --2- 172.21.15.86:0/1294973919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c04101990 0x7f2c04199510 secure :-1 s=READY pgs=535 cs=0 l=1 rev1=1 crypto rx=0x7f2bf8005ab0 tx=0x7f2bf8005e00 comp rx=0 tx=0).stop 2024-04-21T08:38:51.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.051+0000 7f2c0a138640 1 -- 172.21.15.86:0/1294973919 shutdown_connections 2024-04-21T08:38:51.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.051+0000 7f2c0a138640 1 --2- 172.21.15.86:0/1294973919 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2bec0a3d20 0x7f2bec0a61e0 unknown :-1 s=CLOSED pgs=45 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:51.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.051+0000 7f2c0a138640 1 --2- 172.21.15.86:0/1294973919 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2c04105e20 0x7f2c041a4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:51.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.051+0000 7f2c0a138640 1 --2- 172.21.15.86:0/1294973919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2c04102350 0x7f2c04199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:51.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.051+0000 7f2c0a138640 1 --2- 172.21.15.86:0/1294973919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2c04101990 0x7f2c04199510 unknown :-1 s=CLOSED pgs=535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:51.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.051+0000 7f2c0a138640 1 -- 172.21.15.86:0/1294973919 >> 172.21.15.86:0/1294973919 conn(0x7f2c040760e0 msgr2=0x7f2c040fe990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:51.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.051+0000 7f2c0a138640 1 -- 172.21.15.86:0/1294973919 shutdown_connections 2024-04-21T08:38:51.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.051+0000 7f2c0a138640 1 -- 172.21.15.86:0/1294973919 wait complete. 2024-04-21T08:38:51.061 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:38:51.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.168+0000 7fa7d2a31640 1 -- 172.21.15.86:0/2977645840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7cc101a40 msgr2=0x7fa7cc105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:51.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.168+0000 7fa7d2a31640 1 --2- 172.21.15.86:0/2977645840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7cc101a40 0x7fa7cc105a80 secure :-1 s=READY pgs=536 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc009900 tx=0x7fa7bc02bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:51.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.168+0000 7fa7d2a31640 1 -- 172.21.15.86:0/2977645840 shutdown_connections 2024-04-21T08:38:51.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.168+0000 7fa7d2a31640 1 --2- 172.21.15.86:0/2977645840 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7cc106160 0x7fa7cc10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:51.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.168+0000 7fa7d2a31640 1 --2- 172.21.15.86:0/2977645840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7cc101a40 0x7fa7cc105a80 unknown :-1 s=CLOSED pgs=536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:51.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.168+0000 7fa7d2a31640 1 --2- 172.21.15.86:0/2977645840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7cc101090 0x7fa7cc101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.168+0000 7fa7d2a31640 1 -- 172.21.15.86:0/2977645840 >> 172.21.15.86:0/2977645840 conn(0x7fa7cc077f50 msgr2=0x7fa7cc078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.168+0000 7fa7d2a31640 1 -- 172.21.15.86:0/2977645840 shutdown_connections 2024-04-21T08:38:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.168+0000 7fa7d2a31640 1 -- 172.21.15.86:0/2977645840 wait complete. 2024-04-21T08:38:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.168+0000 7fa7d2a31640 1 Processor -- start 2024-04-21T08:38:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.169+0000 7fa7d2a31640 1 -- start start 2024-04-21T08:38:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.169+0000 7fa7d2a31640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7cc101090 0x7fa7cc1a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.169+0000 7fa7d2a31640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7cc106160 0x7fa7cc1a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.169+0000 7fa7d2a31640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7cc19a7f0 0x7fa7cc19aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.169+0000 7fa7d2a31640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa7cc112210 con 0x7fa7cc106160 2024-04-21T08:38:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.169+0000 7fa7d2a31640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa7cc112090 con 0x7fa7cc19a7f0 2024-04-21T08:38:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.169+0000 7fa7d2a31640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa7cc112390 con 0x7fa7cc101090 2024-04-21T08:38:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.169+0000 7fa7cb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7cc106160 0x7fa7cc1a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:51.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.169+0000 7fa7cb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7cc106160 0x7fa7cc1a0bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36080/0 (socket says 172.21.15.86:36080) 2024-04-21T08:38:51.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.169+0000 7fa7cb7fe640 1 -- 172.21.15.86:0/2204742226 learned_addr learned my addr 172.21.15.86:0/2204742226 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:51.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.169+0000 7fa7cbfff640 1 --2- 172.21.15.86:0/2204742226 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7cc101090 0x7fa7cc1a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:51.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.169+0000 7fa7d0fce640 1 --2- 172.21.15.86:0/2204742226 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7cc19a7f0 0x7fa7cc19aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:51.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.169+0000 7fa7cb7fe640 1 -- 172.21.15.86:0/2204742226 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7cc101090 msgr2=0x7fa7cc1a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:51.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.169+0000 7fa7cb7fe640 1 --2- 172.21.15.86:0/2204742226 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7cc101090 0x7fa7cc1a0680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:51.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.169+0000 7fa7cb7fe640 1 -- 172.21.15.86:0/2204742226 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7cc19a7f0 msgr2=0x7fa7cc19aca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:51.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.169+0000 7fa7cb7fe640 1 --2- 172.21.15.86:0/2204742226 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7cc19a7f0 0x7fa7cc19aca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:51.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.169+0000 7fa7cb7fe640 1 -- 172.21.15.86:0/2204742226 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa7cc19b380 con 0x7fa7cc106160 2024-04-21T08:38:51.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.169+0000 7fa7d0fce640 1 --2- 172.21.15.86:0/2204742226 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7cc19a7f0 0x7fa7cc19aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:51.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.169+0000 7fa7cbfff640 1 --2- 172.21.15.86:0/2204742226 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7cc101090 0x7fa7cc1a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:38:51.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.170+0000 7fa7cb7fe640 1 --2- 172.21.15.86:0/2204742226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7cc106160 0x7fa7cc1a0bc0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc000c00 tx=0x7fa7bc02c660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:51.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.172+0000 7fa7c97fa640 1 -- 172.21.15.86:0/2204742226 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc043020 con 0x7fa7cc106160 2024-04-21T08:38:51.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.172+0000 7fa7d2a31640 1 -- 172.21.15.86:0/2204742226 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa7cc19b610 con 0x7fa7cc106160 2024-04-21T08:38:51.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.172+0000 7fa7c97fa640 1 -- 172.21.15.86:0/2204742226 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa7bc02caf0 con 0x7fa7cc106160 2024-04-21T08:38:51.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.172+0000 7fa7c97fa640 1 -- 172.21.15.86:0/2204742226 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa7bc003b20 con 0x7fa7cc106160 2024-04-21T08:38:51.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.172+0000 7fa7d2a31640 1 -- 172.21.15.86:0/2204742226 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa7cc1a77f0 con 0x7fa7cc106160 2024-04-21T08:38:51.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.173+0000 7fa7d2a31640 1 -- 172.21.15.86:0/2204742226 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa7cc102560 con 0x7fa7cc106160 2024-04-21T08:38:51.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.174+0000 7fa7c97fa640 1 -- 172.21.15.86:0/2204742226 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 23) ==== 138766+0+0 (secure 0 0 0) 0x7fa7bc046020 con 0x7fa7cc106160 2024-04-21T08:38:51.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.174+0000 7fa7c97fa640 1 --2- 172.21.15.86:0/2204742226 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa7b80a3d20 0x7fa7b80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:51.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.175+0000 7fa7cbfff640 1 --2- 172.21.15.86:0/2204742226 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa7b80a3d20 0x7fa7b80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:51.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.175+0000 7fa7c97fa640 1 -- 172.21.15.86:0/2204742226 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fa7bc0f2a50 con 0x7fa7cc106160 2024-04-21T08:38:51.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.175+0000 7fa7cbfff640 1 --2- 172.21.15.86:0/2204742226 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa7b80a3d20 0x7fa7b80a61e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4009fc0 tx=0x7fa7b4005f90 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:51.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.179+0000 7fa7c97fa640 1 -- 172.21.15.86:0/2204742226 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa7bc04d020 con 0x7fa7cc106160 2024-04-21T08:38:51.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.300+0000 7fa7c97fa640 1 -- 172.21.15.86:0/2204742226 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fa7bc0bae00 con 0x7fa7cc106160 2024-04-21T08:38:51.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:51 smithi086 ceph-mon[27939]: from='client.15705 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:51.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:51 smithi086 ceph-mon[27939]: Adjusting osd_memory_target on smithi179 to 176.2M 2024-04-21T08:38:51.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:51 smithi086 ceph-mon[27939]: Unable to set osd_memory_target on smithi179 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-21T08:38:51.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:51 smithi086 ceph-mon[27939]: Adjusting osd_memory_target on smithi088 to 176.2M 2024-04-21T08:38:51.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:51 smithi086 ceph-mon[27939]: Unable to set osd_memory_target on smithi088 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-21T08:38:51.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:51 smithi086 ceph-mon[27939]: Adjusting osd_memory_target on smithi086 to 176.2M 2024-04-21T08:38:51.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:51 smithi086 ceph-mon[27939]: Unable to set osd_memory_target on smithi086 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-21T08:38:51.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:51 smithi086 ceph-mon[27939]: Updating smithi086:/etc/ceph/ceph.conf 2024-04-21T08:38:51.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:51 smithi086 ceph-mon[27939]: Updating smithi088:/etc/ceph/ceph.conf 2024-04-21T08:38:51.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:51 smithi086 ceph-mon[27939]: Updating smithi179:/etc/ceph/ceph.conf 2024-04-21T08:38:51.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:51 smithi086 ceph-mon[27939]: from='client.25118 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:51.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:51 smithi086 ceph-mon[27939]: Updating smithi086:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:38:51.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:51 smithi086 ceph-mon[27939]: Updating smithi088:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:38:51.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:51 smithi086 ceph-mon[27939]: Updating smithi179:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:38:51.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:51 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/407883413' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:51.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:51 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/567611901' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:51.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:51 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1294973919' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:51.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:51 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:51.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:51 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:51.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:51 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:51.368 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:51 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:51.369 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:51 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:51.369 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:51 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:51.369 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:51 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:51.369 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:51 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:38:51.369 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:51 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:51.369 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:51 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:38:51.369 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:51 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:51.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.372+0000 7fa7d2a31640 1 -- 172.21.15.86:0/2204742226 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7fa7cc101a40 con 0x7fa7cc106160 2024-04-21T08:38:51.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.372+0000 7fa7c97fa640 1 -- 172.21.15.86:0/2204742226 <== mon.0 v2:172.21.15.86:3300/0 8 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7fa7bc0bee90 con 0x7fa7cc106160 2024-04-21T08:38:51.371 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-21T08:38:51.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.375+0000 7fa7d2a31640 1 -- 172.21.15.86:0/2204742226 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa7b80a3d20 msgr2=0x7fa7b80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:51.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.375+0000 7fa7d2a31640 1 --2- 172.21.15.86:0/2204742226 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa7b80a3d20 0x7fa7b80a61e0 secure :-1 s=READY pgs=46 cs=0 l=1 rev1=1 crypto rx=0x7fa7b4009fc0 tx=0x7fa7b4005f90 comp rx=0 tx=0).stop 2024-04-21T08:38:51.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.375+0000 7fa7d2a31640 1 -- 172.21.15.86:0/2204742226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7cc106160 msgr2=0x7fa7cc1a0bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:51.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.375+0000 7fa7d2a31640 1 --2- 172.21.15.86:0/2204742226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7cc106160 0x7fa7cc1a0bc0 secure :-1 s=READY pgs=537 cs=0 l=1 rev1=1 crypto rx=0x7fa7bc000c00 tx=0x7fa7bc02c660 comp rx=0 tx=0).stop 2024-04-21T08:38:51.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.375+0000 7fa7d2a31640 1 -- 172.21.15.86:0/2204742226 shutdown_connections 2024-04-21T08:38:51.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.375+0000 7fa7d2a31640 1 --2- 172.21.15.86:0/2204742226 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa7b80a3d20 0x7fa7b80a61e0 unknown :-1 s=CLOSED pgs=46 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:51.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.375+0000 7fa7d2a31640 1 --2- 172.21.15.86:0/2204742226 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa7cc19a7f0 0x7fa7cc19aca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:51.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.375+0000 7fa7d2a31640 1 --2- 172.21.15.86:0/2204742226 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa7cc106160 0x7fa7cc1a0bc0 unknown :-1 s=CLOSED pgs=537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:51.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.375+0000 7fa7d2a31640 1 --2- 172.21.15.86:0/2204742226 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa7cc101090 0x7fa7cc1a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:51.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.375+0000 7fa7d2a31640 1 -- 172.21.15.86:0/2204742226 >> 172.21.15.86:0/2204742226 conn(0x7fa7cc077f50 msgr2=0x7fa7cc0ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:51.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.375+0000 7fa7d2a31640 1 -- 172.21.15.86:0/2204742226 shutdown_connections 2024-04-21T08:38:51.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.375+0000 7fa7d2a31640 1 -- 172.21.15.86:0/2204742226 wait complete. 2024-04-21T08:38:51.385 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:38:51.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.485+0000 7f33d0a93640 1 -- 172.21.15.86:0/2246273316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33cc1005c0 msgr2=0x7f33cc100a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:51.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.485+0000 7f33d0a93640 1 --2- 172.21.15.86:0/2246273316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33cc1005c0 0x7f33cc100a40 secure :-1 s=READY pgs=538 cs=0 l=1 rev1=1 crypto rx=0x7f33b4009900 tx=0x7f33b402bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:51.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.485+0000 7f33d0a93640 1 -- 172.21.15.86:0/2246273316 shutdown_connections 2024-04-21T08:38:51.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.485+0000 7f33d0a93640 1 --2- 172.21.15.86:0/2246273316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33cc100f80 0x7f33cc10d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:51.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.485+0000 7f33d0a93640 1 --2- 172.21.15.86:0/2246273316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33cc1005c0 0x7f33cc100a40 unknown :-1 s=CLOSED pgs=538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:51.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.485+0000 7f33d0a93640 1 --2- 172.21.15.86:0/2246273316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33cc106680 0x7f33cc106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:51.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.485+0000 7f33d0a93640 1 -- 172.21.15.86:0/2246273316 >> 172.21.15.86:0/2246273316 conn(0x7f33cc0773e0 msgr2=0x7f33cc0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:51.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.485+0000 7f33d0a93640 1 -- 172.21.15.86:0/2246273316 shutdown_connections 2024-04-21T08:38:51.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.485+0000 7f33d0a93640 1 -- 172.21.15.86:0/2246273316 wait complete. 2024-04-21T08:38:51.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.486+0000 7f33d0a93640 1 Processor -- start 2024-04-21T08:38:51.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.486+0000 7f33d0a93640 1 -- start start 2024-04-21T08:38:51.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.486+0000 7f33d0a93640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33cc1005c0 0x7f33cc1994e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:51.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.486+0000 7f33d0a93640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33cc100f80 0x7f33cc199a20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:51.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.486+0000 7f33d0a93640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33cc106680 0x7f33cc1a4560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:51.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.486+0000 7f33d0a93640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc111080 con 0x7f33cc106680 2024-04-21T08:38:51.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.486+0000 7f33d0a93640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc110f00 con 0x7f33cc1005c0 2024-04-21T08:38:51.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.486+0000 7f33d0a93640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f33cc111200 con 0x7f33cc100f80 2024-04-21T08:38:51.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.486+0000 7f33cad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33cc106680 0x7f33cc1a4560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:51.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.486+0000 7f33cad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33cc106680 0x7f33cc1a4560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36092/0 (socket says 172.21.15.86:36092) 2024-04-21T08:38:51.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.486+0000 7f33cad9d640 1 -- 172.21.15.86:0/778735871 learned_addr learned my addr 172.21.15.86:0/778735871 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:51.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.486+0000 7f33c3fff640 1 --2- 172.21.15.86:0/778735871 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33cc100f80 0x7f33cc199a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:51.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.486+0000 7f33ca59c640 1 --2- 172.21.15.86:0/778735871 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33cc1005c0 0x7f33cc1994e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:51.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.487+0000 7f33cad9d640 1 -- 172.21.15.86:0/778735871 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33cc100f80 msgr2=0x7f33cc199a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:51.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.487+0000 7f33cad9d640 1 --2- 172.21.15.86:0/778735871 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33cc100f80 0x7f33cc199a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:51.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.487+0000 7f33cad9d640 1 -- 172.21.15.86:0/778735871 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33cc1005c0 msgr2=0x7f33cc1994e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:51.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.487+0000 7f33cad9d640 1 --2- 172.21.15.86:0/778735871 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33cc1005c0 0x7f33cc1994e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:51.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.487+0000 7f33cad9d640 1 -- 172.21.15.86:0/778735871 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f33cc1a4aa0 con 0x7f33cc106680 2024-04-21T08:38:51.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.487+0000 7f33cad9d640 1 --2- 172.21.15.86:0/778735871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33cc106680 0x7f33cc1a4560 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f33c400eb10 tx=0x7f33c400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:51.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.487+0000 7f33c37fe640 1 -- 172.21.15.86:0/778735871 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c4004b20 con 0x7f33cc106680 2024-04-21T08:38:51.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.487+0000 7f33c37fe640 1 -- 172.21.15.86:0/778735871 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f33c4004cc0 con 0x7f33cc106680 2024-04-21T08:38:51.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.487+0000 7f33c37fe640 1 -- 172.21.15.86:0/778735871 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f33c4010670 con 0x7f33cc106680 2024-04-21T08:38:51.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.487+0000 7f33d0a93640 1 -- 172.21.15.86:0/778735871 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f33cc1a4c70 con 0x7f33cc106680 2024-04-21T08:38:51.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.487+0000 7f33d0a93640 1 -- 172.21.15.86:0/778735871 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33cc1a5140 con 0x7f33cc106680 2024-04-21T08:38:51.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.488+0000 7f33d0a93640 1 -- 172.21.15.86:0/778735871 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f33cc0691c0 con 0x7f33cc106680 2024-04-21T08:38:51.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.489+0000 7f33c37fe640 1 -- 172.21.15.86:0/778735871 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f33c4005020 con 0x7f33cc106680 2024-04-21T08:38:51.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.490+0000 7f33c37fe640 1 --2- 172.21.15.86:0/778735871 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f33b80a3db0 0x7f33b80a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:51.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.490+0000 7f33ca59c640 1 --2- 172.21.15.86:0/778735871 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f33b80a3db0 0x7f33b80a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:51.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.490+0000 7f33c37fe640 1 -- 172.21.15.86:0/778735871 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f33c40d1280 con 0x7f33cc106680 2024-04-21T08:38:51.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.490+0000 7f33ca59c640 1 --2- 172.21.15.86:0/778735871 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f33b80a3db0 0x7f33b80a6270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f33bc005d00 tx=0x7f33bc00b410 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:51.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.494+0000 7f33c37fe640 1 -- 172.21.15.86:0/778735871 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f33c409d440 con 0x7f33cc106680 2024-04-21T08:38:51.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.670+0000 7f33d0a93640 1 -- 172.21.15.86:0/778735871 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f33cc02d5c0 con 0x7f33cc106680 2024-04-21T08:38:51.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.670+0000 7f33c37fe640 1 -- 172.21.15.86:0/778735871 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f33c409cd40 con 0x7f33cc106680 2024-04-21T08:38:51.671 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-21T08:38:51.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.672+0000 7f33d0a93640 1 -- 172.21.15.86:0/778735871 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f33b80a3db0 msgr2=0x7f33b80a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:51.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.672+0000 7f33d0a93640 1 --2- 172.21.15.86:0/778735871 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f33b80a3db0 0x7f33b80a6270 secure :-1 s=READY pgs=47 cs=0 l=1 rev1=1 crypto rx=0x7f33bc005d00 tx=0x7f33bc00b410 comp rx=0 tx=0).stop 2024-04-21T08:38:51.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.672+0000 7f33d0a93640 1 -- 172.21.15.86:0/778735871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33cc106680 msgr2=0x7f33cc1a4560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:51.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.672+0000 7f33d0a93640 1 --2- 172.21.15.86:0/778735871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33cc106680 0x7f33cc1a4560 secure :-1 s=READY pgs=539 cs=0 l=1 rev1=1 crypto rx=0x7f33c400eb10 tx=0x7f33c400c2d0 comp rx=0 tx=0).stop 2024-04-21T08:38:51.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.672+0000 7f33d0a93640 1 -- 172.21.15.86:0/778735871 shutdown_connections 2024-04-21T08:38:51.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.672+0000 7f33d0a93640 1 --2- 172.21.15.86:0/778735871 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f33b80a3db0 0x7f33b80a6270 unknown :-1 s=CLOSED pgs=47 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:51.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.672+0000 7f33d0a93640 1 --2- 172.21.15.86:0/778735871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f33cc106680 0x7f33cc1a4560 unknown :-1 s=CLOSED pgs=539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:51.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.672+0000 7f33d0a93640 1 --2- 172.21.15.86:0/778735871 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33cc100f80 0x7f33cc199a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:51.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.672+0000 7f33d0a93640 1 --2- 172.21.15.86:0/778735871 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33cc1005c0 0x7f33cc1994e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:51.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.672+0000 7f33d0a93640 1 -- 172.21.15.86:0/778735871 >> 172.21.15.86:0/778735871 conn(0x7f33cc0773e0 msgr2=0x7f33cc0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:51.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.672+0000 7f33d0a93640 1 -- 172.21.15.86:0/778735871 shutdown_connections 2024-04-21T08:38:51.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.673+0000 7f33d0a93640 1 -- 172.21.15.86:0/778735871 wait complete. 2024-04-21T08:38:51.684 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:38:51.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:51 smithi088 ceph-mon[31259]: from='client.15705 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:51.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:51 smithi088 ceph-mon[31259]: Adjusting osd_memory_target on smithi179 to 176.2M 2024-04-21T08:38:51.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:51 smithi088 ceph-mon[31259]: Unable to set osd_memory_target on smithi179 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-21T08:38:51.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:51 smithi088 ceph-mon[31259]: Adjusting osd_memory_target on smithi088 to 176.2M 2024-04-21T08:38:51.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:51 smithi088 ceph-mon[31259]: Unable to set osd_memory_target on smithi088 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-21T08:38:51.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:51 smithi088 ceph-mon[31259]: Adjusting osd_memory_target on smithi086 to 176.2M 2024-04-21T08:38:51.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:51 smithi088 ceph-mon[31259]: Unable to set osd_memory_target on smithi086 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-21T08:38:51.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:51 smithi088 ceph-mon[31259]: Updating smithi086:/etc/ceph/ceph.conf 2024-04-21T08:38:51.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:51 smithi088 ceph-mon[31259]: Updating smithi088:/etc/ceph/ceph.conf 2024-04-21T08:38:51.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:51 smithi088 ceph-mon[31259]: Updating smithi179:/etc/ceph/ceph.conf 2024-04-21T08:38:51.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:51 smithi088 ceph-mon[31259]: from='client.25118 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:51.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:51 smithi088 ceph-mon[31259]: Updating smithi086:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:38:51.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:51 smithi088 ceph-mon[31259]: Updating smithi088:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:38:51.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:51 smithi088 ceph-mon[31259]: Updating smithi179:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:38:51.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:51 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/407883413' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:51.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:51 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/567611901' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:51.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:51 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1294973919' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:51.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:51 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:51.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:51 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:51.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:51 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:51.716 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:51 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:51.716 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:51 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:51.716 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:51 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:51.716 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:51 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:51.716 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:51 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:38:51.716 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:51 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:51.716 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:51 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:38:51.716 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:51 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:51.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:51 smithi179 ceph-mon[31829]: from='client.15705 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:51.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:51 smithi179 ceph-mon[31829]: Adjusting osd_memory_target on smithi179 to 176.2M 2024-04-21T08:38:51.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:51 smithi179 ceph-mon[31829]: Unable to set osd_memory_target on smithi179 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-21T08:38:51.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:51 smithi179 ceph-mon[31829]: Adjusting osd_memory_target on smithi088 to 176.2M 2024-04-21T08:38:51.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:51 smithi179 ceph-mon[31829]: Unable to set osd_memory_target on smithi088 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-21T08:38:51.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:51 smithi179 ceph-mon[31829]: Adjusting osd_memory_target on smithi086 to 176.2M 2024-04-21T08:38:51.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:51 smithi179 ceph-mon[31829]: Unable to set osd_memory_target on smithi086 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-21T08:38:51.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:51 smithi179 ceph-mon[31829]: Updating smithi086:/etc/ceph/ceph.conf 2024-04-21T08:38:51.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:51 smithi179 ceph-mon[31829]: Updating smithi088:/etc/ceph/ceph.conf 2024-04-21T08:38:51.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:51 smithi179 ceph-mon[31829]: Updating smithi179:/etc/ceph/ceph.conf 2024-04-21T08:38:51.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:51 smithi179 ceph-mon[31829]: from='client.25118 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:51.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:51 smithi179 ceph-mon[31829]: Updating smithi086:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:38:51.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:51 smithi179 ceph-mon[31829]: Updating smithi088:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:38:51.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:51 smithi179 ceph-mon[31829]: Updating smithi179:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config/ceph.conf 2024-04-21T08:38:51.748 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:51 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/407883413' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:51.748 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:51 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/567611901' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:51.748 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:51 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1294973919' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:51.748 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:51 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:51.748 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:51 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:51.748 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:51 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:51.748 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:51 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:51.748 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:51 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:51.748 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:51 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:51.748 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:51 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:51.748 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:51 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:38:51.748 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:51 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:38:51.749 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:51 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:38:51.749 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:51 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:38:51.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.781+0000 7f5642425640 1 -- 172.21.15.86:0/2566328478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f563c101130 msgr2=0x7f563c101590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:51.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.781+0000 7f5642425640 1 --2- 172.21.15.86:0/2566328478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f563c101130 0x7f563c101590 secure :-1 s=READY pgs=540 cs=0 l=1 rev1=1 crypto rx=0x7f5630009900 tx=0x7f563002bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:51.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.781+0000 7f5642425640 1 -- 172.21.15.86:0/2566328478 shutdown_connections 2024-04-21T08:38:51.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.781+0000 7f5642425640 1 --2- 172.21.15.86:0/2566328478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f563c101ad0 0x7f563c10dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:51.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.781+0000 7f5642425640 1 --2- 172.21.15.86:0/2566328478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f563c101130 0x7f563c101590 unknown :-1 s=CLOSED pgs=540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:51.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.781+0000 7f5642425640 1 --2- 172.21.15.86:0/2566328478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f563c106f10 0x7f563c1072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:51.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.781+0000 7f5642425640 1 -- 172.21.15.86:0/2566328478 >> 172.21.15.86:0/2566328478 conn(0x7f563c077e30 msgr2=0x7f563c078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:51.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.781+0000 7f5642425640 1 -- 172.21.15.86:0/2566328478 shutdown_connections 2024-04-21T08:38:51.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.781+0000 7f5642425640 1 -- 172.21.15.86:0/2566328478 wait complete. 2024-04-21T08:38:51.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.782+0000 7f5642425640 1 Processor -- start 2024-04-21T08:38:51.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.782+0000 7f5642425640 1 -- start start 2024-04-21T08:38:51.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.782+0000 7f5642425640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f563c101130 0x7f563c197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:51.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.782+0000 7f5642425640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f563c101ad0 0x7f563c197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:51.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.782+0000 7f5642425640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f563c106f10 0x7f563c19bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:51.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.782+0000 7f5642425640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f563c1110e0 con 0x7f563c106f10 2024-04-21T08:38:51.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.782+0000 7f5642425640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f563c110f60 con 0x7f563c101130 2024-04-21T08:38:51.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.782+0000 7f5642425640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f563c111260 con 0x7f563c101ad0 2024-04-21T08:38:51.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.782+0000 7f56409c2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f563c106f10 0x7f563c19bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:51.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.782+0000 7f56409c2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f563c106f10 0x7f563c19bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36116/0 (socket says 172.21.15.86:36116) 2024-04-21T08:38:51.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.782+0000 7f56409c2640 1 -- 172.21.15.86:0/4222529660 learned_addr learned my addr 172.21.15.86:0/4222529660 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:51.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.782+0000 7f563bfff640 1 --2- 172.21.15.86:0/4222529660 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f563c101130 0x7f563c197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:51.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.782+0000 7f563b7fe640 1 --2- 172.21.15.86:0/4222529660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f563c101ad0 0x7f563c197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:51.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.783+0000 7f56409c2640 1 -- 172.21.15.86:0/4222529660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f563c101ad0 msgr2=0x7f563c197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:51.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.783+0000 7f56409c2640 1 --2- 172.21.15.86:0/4222529660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f563c101ad0 0x7f563c197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:51.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.783+0000 7f56409c2640 1 -- 172.21.15.86:0/4222529660 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f563c101130 msgr2=0x7f563c197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:51.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.783+0000 7f56409c2640 1 --2- 172.21.15.86:0/4222529660 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f563c101130 0x7f563c197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:51.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.783+0000 7f56409c2640 1 -- 172.21.15.86:0/4222529660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f563c19c670 con 0x7f563c106f10 2024-04-21T08:38:51.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.783+0000 7f56409c2640 1 --2- 172.21.15.86:0/4222529660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f563c106f10 0x7f563c19bf40 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f563400eb10 tx=0x7f563400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:51.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.783+0000 7f56397fa640 1 -- 172.21.15.86:0/4222529660 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5634004b20 con 0x7f563c106f10 2024-04-21T08:38:51.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.783+0000 7f5642425640 1 -- 172.21.15.86:0/4222529660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f563c19c960 con 0x7f563c106f10 2024-04-21T08:38:51.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.783+0000 7f56397fa640 1 -- 172.21.15.86:0/4222529660 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5634004cc0 con 0x7f563c106f10 2024-04-21T08:38:51.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.783+0000 7f56397fa640 1 -- 172.21.15.86:0/4222529660 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5634010670 con 0x7f563c106f10 2024-04-21T08:38:51.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.783+0000 7f5642425640 1 -- 172.21.15.86:0/4222529660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f563c1a41f0 con 0x7f563c106f10 2024-04-21T08:38:51.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.784+0000 7f5642425640 1 -- 172.21.15.86:0/4222529660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f563c069230 con 0x7f563c106f10 2024-04-21T08:38:51.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.785+0000 7f56397fa640 1 -- 172.21.15.86:0/4222529660 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f5634005020 con 0x7f563c106f10 2024-04-21T08:38:51.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.786+0000 7f56397fa640 1 --2- 172.21.15.86:0/4222529660 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f56240a3db0 0x7f56240a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:51.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.786+0000 7f563bfff640 1 --2- 172.21.15.86:0/4222529660 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f56240a3db0 0x7f56240a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:51.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.786+0000 7f56397fa640 1 -- 172.21.15.86:0/4222529660 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f56340d1330 con 0x7f563c106f10 2024-04-21T08:38:51.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.786+0000 7f563bfff640 1 --2- 172.21.15.86:0/4222529660 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f56240a3db0 0x7f56240a6270 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f562c005fd0 tx=0x7f562c00b390 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:51.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:51.790+0000 7f56397fa640 1 -- 172.21.15.86:0/4222529660 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f563409d4f0 con 0x7f563c106f10 2024-04-21T08:38:52.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.010+0000 7f5642425640 1 -- 172.21.15.86:0/4222529660 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f563c1093b0 con 0x7f563c106f10 2024-04-21T08:38:52.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.010+0000 7f56397fa640 1 -- 172.21.15.86:0/4222529660 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7f563409cdf0 con 0x7f563c106f10 2024-04-21T08:38:52.011 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:38:52.011 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:38:45.202126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24398,24394,24455,24472]},"id":1}]} 2024-04-21T08:38:52.011 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-21T08:38:52.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.013+0000 7f5642425640 1 -- 172.21.15.86:0/4222529660 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f56240a3db0 msgr2=0x7f56240a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:52.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.013+0000 7f5642425640 1 --2- 172.21.15.86:0/4222529660 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f56240a3db0 0x7f56240a6270 secure :-1 s=READY pgs=48 cs=0 l=1 rev1=1 crypto rx=0x7f562c005fd0 tx=0x7f562c00b390 comp rx=0 tx=0).stop 2024-04-21T08:38:52.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.013+0000 7f5642425640 1 -- 172.21.15.86:0/4222529660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f563c106f10 msgr2=0x7f563c19bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:52.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.013+0000 7f5642425640 1 --2- 172.21.15.86:0/4222529660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f563c106f10 0x7f563c19bf40 secure :-1 s=READY pgs=541 cs=0 l=1 rev1=1 crypto rx=0x7f563400eb10 tx=0x7f563400c2d0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.013+0000 7f5642425640 1 -- 172.21.15.86:0/4222529660 shutdown_connections 2024-04-21T08:38:52.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.013+0000 7f5642425640 1 --2- 172.21.15.86:0/4222529660 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f56240a3db0 0x7f56240a6270 unknown :-1 s=CLOSED pgs=48 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.013+0000 7f5642425640 1 --2- 172.21.15.86:0/4222529660 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f563c106f10 0x7f563c19bf40 unknown :-1 s=CLOSED pgs=541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.013+0000 7f5642425640 1 --2- 172.21.15.86:0/4222529660 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f563c101ad0 0x7f563c197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.013+0000 7f5642425640 1 --2- 172.21.15.86:0/4222529660 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f563c101130 0x7f563c197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.013+0000 7f5642425640 1 -- 172.21.15.86:0/4222529660 >> 172.21.15.86:0/4222529660 conn(0x7f563c077e30 msgr2=0x7f563c0fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:52.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.013+0000 7f5642425640 1 -- 172.21.15.86:0/4222529660 shutdown_connections 2024-04-21T08:38:52.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.013+0000 7f5642425640 1 -- 172.21.15.86:0/4222529660 wait complete. 2024-04-21T08:38:52.024 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:38:52.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.122+0000 7f7d4acc0640 1 -- 172.21.15.86:0/688922228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d44105e30 msgr2=0x7f7d44106210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:52.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.122+0000 7f7d4acc0640 1 --2- 172.21.15.86:0/688922228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d44105e30 0x7f7d44106210 secure :-1 s=READY pgs=542 cs=0 l=1 rev1=1 crypto rx=0x7f7d34009930 tx=0x7f7d3402bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:52.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.122+0000 7f7d4acc0640 1 -- 172.21.15.86:0/688922228 shutdown_connections 2024-04-21T08:38:52.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.122+0000 7f7d4acc0640 1 --2- 172.21.15.86:0/688922228 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d44101b60 0x7f7d4410dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.122+0000 7f7d4acc0640 1 --2- 172.21.15.86:0/688922228 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d441011a0 0x7f7d44101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.122+0000 7f7d4acc0640 1 --2- 172.21.15.86:0/688922228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d44105e30 0x7f7d44106210 unknown :-1 s=CLOSED pgs=542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.122+0000 7f7d4acc0640 1 -- 172.21.15.86:0/688922228 >> 172.21.15.86:0/688922228 conn(0x7f7d44077e30 msgr2=0x7f7d44078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:52.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.122+0000 7f7d4acc0640 1 -- 172.21.15.86:0/688922228 shutdown_connections 2024-04-21T08:38:52.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.122+0000 7f7d4acc0640 1 -- 172.21.15.86:0/688922228 wait complete. 2024-04-21T08:38:52.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.122+0000 7f7d4acc0640 1 Processor -- start 2024-04-21T08:38:52.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.122+0000 7f7d4acc0640 1 -- start start 2024-04-21T08:38:52.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.122+0000 7f7d4acc0640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d441011a0 0x7f7d441986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.122+0000 7f7d4acc0640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d44101b60 0x7f7d44198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.122+0000 7f7d4acc0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d44105e30 0x7f7d4419cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.122+0000 7f7d4acc0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7d441110e0 con 0x7f7d44105e30 2024-04-21T08:38:52.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.122+0000 7f7d4acc0640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7d44110f60 con 0x7f7d44101b60 2024-04-21T08:38:52.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.122+0000 7f7d4acc0640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7d44111260 con 0x7f7d441011a0 2024-04-21T08:38:52.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.122+0000 7f7d4925d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d44105e30 0x7f7d4419cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.123+0000 7f7d4925d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d44105e30 0x7f7d4419cfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36134/0 (socket says 172.21.15.86:36134) 2024-04-21T08:38:52.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.123+0000 7f7d4925d640 1 -- 172.21.15.86:0/3315821057 learned_addr learned my addr 172.21.15.86:0/3315821057 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:52.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.123+0000 7f7d48a5c640 1 --2- 172.21.15.86:0/3315821057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d441011a0 0x7f7d441986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.123+0000 7f7d4925d640 1 -- 172.21.15.86:0/3315821057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d441011a0 msgr2=0x7f7d441986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:52.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.123+0000 7f7d43fff640 1 --2- 172.21.15.86:0/3315821057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d44101b60 0x7f7d44198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.123+0000 7f7d4925d640 1 --2- 172.21.15.86:0/3315821057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d441011a0 0x7f7d441986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.123+0000 7f7d4925d640 1 -- 172.21.15.86:0/3315821057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d44101b60 msgr2=0x7f7d44198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:52.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.123+0000 7f7d4925d640 1 --2- 172.21.15.86:0/3315821057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d44101b60 0x7f7d44198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.123+0000 7f7d4925d640 1 -- 172.21.15.86:0/3315821057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d4419d6e0 con 0x7f7d44105e30 2024-04-21T08:38:52.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.123+0000 7f7d43fff640 1 --2- 172.21.15.86:0/3315821057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d44101b60 0x7f7d44198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:52.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.123+0000 7f7d4925d640 1 --2- 172.21.15.86:0/3315821057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d44105e30 0x7f7d4419cfb0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c00eaa0 tx=0x7f7d3c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:52.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.123+0000 7f7d41ffb640 1 -- 172.21.15.86:0/3315821057 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c00cdb0 con 0x7f7d44105e30 2024-04-21T08:38:52.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.123+0000 7f7d4acc0640 1 -- 172.21.15.86:0/3315821057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d4419d9d0 con 0x7f7d44105e30 2024-04-21T08:38:52.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.123+0000 7f7d41ffb640 1 -- 172.21.15.86:0/3315821057 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d3c005020 con 0x7f7d44105e30 2024-04-21T08:38:52.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.123+0000 7f7d41ffb640 1 -- 172.21.15.86:0/3315821057 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d3c0116d0 con 0x7f7d44105e30 2024-04-21T08:38:52.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.123+0000 7f7d4acc0640 1 -- 172.21.15.86:0/3315821057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d441a5260 con 0x7f7d44105e30 2024-04-21T08:38:52.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.124+0000 7f7d4acc0640 1 -- 172.21.15.86:0/3315821057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7d44102570 con 0x7f7d44105e30 2024-04-21T08:38:52.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.125+0000 7f7d41ffb640 1 -- 172.21.15.86:0/3315821057 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f7d3c0054a0 con 0x7f7d44105e30 2024-04-21T08:38:52.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.126+0000 7f7d41ffb640 1 --2- 172.21.15.86:0/3315821057 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7d2c0a3db0 0x7f7d2c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.126+0000 7f7d48a5c640 1 --2- 172.21.15.86:0/3315821057 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7d2c0a3db0 0x7f7d2c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.126+0000 7f7d41ffb640 1 -- 172.21.15.86:0/3315821057 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f7d3c0d0f50 con 0x7f7d44105e30 2024-04-21T08:38:52.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.126+0000 7f7d48a5c640 1 --2- 172.21.15.86:0/3315821057 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7d2c0a3db0 0x7f7d2c0a6270 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7d34006010 tx=0x7f7d3402c670 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:52.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.130+0000 7f7d41ffb640 1 -- 172.21.15.86:0/3315821057 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7d3c016030 con 0x7f7d44105e30 2024-04-21T08:38:52.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.312+0000 7f7d4acc0640 1 -- 172.21.15.86:0/3315821057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f7d44068de0 con 0x7f7d44105e30 2024-04-21T08:38:52.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.314+0000 7f7d41ffb640 1 -- 172.21.15.86:0/3315821057 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+25067 (secure 0 0 0) 0x7f7d44068de0 con 0x7f7d44105e30 2024-04-21T08:38:52.314 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:38:52.315 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":94,"fsid":"f8cec7cc-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:22.687329+0000","modified":"2024-04-21T08:38:46.631676+0000","last_up_change":"2024-04-21T08:35:30.048315+0000","last_in_change":"2024-04-21T08:35:10.099496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:45.279109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:18.300782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:19.574636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:22.905632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89b7601f-0620-4db2-90f1-b45cea173738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6803","nonce":2571516897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6805","nonce":2571516897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6809","nonce":2571516897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6807","nonce":2571516897}]},"public_addr":"172.21.15.86:6803/2571516897","cluster_addr":"172.21.15.86:6805/2571516897","heartbeat_back_addr":"172.21.15.86:6809/2571516897","heartbeat_front_addr":"172.21.15.86:6807/2571516897","state":["exists","up"]},{"osd":1,"uuid":"4891031c-f98c-42bc-bb3b-a420855ecc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6801","nonce":74067182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6803","nonce":74067182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6807","nonce":74067182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6805","nonce":74067182}]},"public_addr":"172.21.15.88:6801/74067182","cluster_addr":"172.21.15.88:6803/74067182","heartbeat_back_addr":"172.21.15.88:6807/74067182","heartbeat_front_addr":"172.21.15.88:6805/74067182","state":["exists","up"]},{"osd":2,"uuid":"60d12aa4-9312-4fa2-8d0b-2b3f42caad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6801","nonce":4137659051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6803","nonce":4137659051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6807","nonce":4137659051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6805","nonce":4137659051}]},"public_addr":"172.21.15.179:6801/4137659051","cluster_addr":"172.21.15.179:6803/4137659051","heartbeat_back_addr":"172.21.15.179:6807/4137659051","heartbeat_front_addr":"172.21.15.179:6805/4137659051","state":["exists","up"]},{"osd":3,"uuid":"404c3158-8cb0-4c02-b48a-f9ee5ff7b784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6811","nonce":2309603148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6813","nonce":2309603148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6817","nonce":2309603148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6815","nonce":2309603148}]},"public_addr":"172.21.15.86:6811/2309603148","cluster_addr":"172.21.15.86:6813/2309603148","heartbeat_back_addr":"172.21.15.86:6817/2309603148","heartbeat_front_addr":"172.21.15.86:6815/2309603148","state":["exists","up"]},{"osd":4,"uuid":"458ff202-d781-4ec7-ac93-0b49f835b503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6809","nonce":1818714217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6811","nonce":1818714217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6815","nonce":1818714217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6813","nonce":1818714217}]},"public_addr":"172.21.15.88:6809/1818714217","cluster_addr":"172.21.15.88:6811/1818714217","heartbeat_back_addr":"172.21.15.88:6815/1818714217","heartbeat_front_addr":"172.21.15.88:6813/1818714217","state":["exists","up"]},{"osd":5,"uuid":"8123cc90-bd8b-4204-bf31-46b34e9f589d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6809","nonce":863840990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6811","nonce":863840990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6815","nonce":863840990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6813","nonce":863840990}]},"public_addr":"172.21.15.179:6809/863840990","cluster_addr":"172.21.15.179:6811/863840990","heartbeat_back_addr":"172.21.15.179:6815/863840990","heartbeat_front_addr":"172.21.15.179:6813/863840990","state":["exists","up"]},{"osd":6,"uuid":"aac9d7c7-659a-4110-903a-a871c086066f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6819","nonce":1224231118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6821","nonce":1224231118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6825","nonce":1224231118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6823","nonce":1224231118}]},"public_addr":"172.21.15.86:6819/1224231118","cluster_addr":"172.21.15.86:6821/1224231118","heartbeat_back_addr":"172.21.15.86:6825/1224231118","heartbeat_front_addr":"172.21.15.86:6823/1224231118","state":["exists","up"]},{"osd":7,"uuid":"df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6817","nonce":136629878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6819","nonce":136629878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6823","nonce":136629878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6821","nonce":136629878}]},"public_addr":"172.21.15.88:6817/136629878","cluster_addr":"172.21.15.88:6819/136629878","heartbeat_back_addr":"172.21.15.88:6823/136629878","heartbeat_front_addr":"172.21.15.88:6821/136629878","state":["exists","up"]},{"osd":8,"uuid":"63e59f4b-0426-46db-92f7-c55dae61b5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6817","nonce":1770418957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6819","nonce":1770418957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6823","nonce":1770418957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6821","nonce":1770418957}]},"public_addr":"172.21.15.179:6817/1770418957","cluster_addr":"172.21.15.179:6819/1770418957","heartbeat_back_addr":"172.21.15.179:6823/1770418957","heartbeat_front_addr":"172.21.15.179:6821/1770418957","state":["exists","up"]},{"osd":9,"uuid":"5841bb36-ed6b-45d8-b857-eeab139c44e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6827","nonce":1108500274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6829","nonce":1108500274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6833","nonce":1108500274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6831","nonce":1108500274}]},"public_addr":"172.21.15.86:6827/1108500274","cluster_addr":"172.21.15.86:6829/1108500274","heartbeat_back_addr":"172.21.15.86:6833/1108500274","heartbeat_front_addr":"172.21.15.86:6831/1108500274","state":["exists","up"]},{"osd":10,"uuid":"f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6825","nonce":1294275353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6827","nonce":1294275353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6831","nonce":1294275353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6829","nonce":1294275353}]},"public_addr":"172.21.15.88:6825/1294275353","cluster_addr":"172.21.15.88:6827/1294275353","heartbeat_back_addr":"172.21.15.88:6831/1294275353","heartbeat_front_addr":"172.21.15.88:6829/1294275353","state":["exists","up"]},{"osd":11,"uuid":"e0d4cc76-e7a6-4a6b-909c-95488ccd5a36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6825","nonce":4007070233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6827","nonce":4007070233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6831","nonce":4007070233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6829","nonce":4007070233}]},"public_addr":"172.21.15.179:6825/4007070233","cluster_addr":"172.21.15.179:6827/4007070233","heartbeat_back_addr":"172.21.15.179:6831/4007070233","heartbeat_front_addr":"172.21.15.179:6829/4007070233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:55.210605+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:18.723090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:42.547979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:06.421622+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:30.858315+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:55.455313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.623369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:46.185111+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.103627+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:37.297545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:02.637431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:28.199292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1068089552":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/4273877219":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/2707594358":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/4060044103":"2024-04-22T08:38:46.631621+0000","172.21.15.86:6800/1364156965":"2024-04-22T08:38:46.631621+0000","172.21.15.86:6800/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/1364156965":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/1412393225":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6800/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/1687852404":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2020707404":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2568130933":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/3952607648":"2024-04-22T08:28:57.384403+0000","172.21.15.86:6801/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/2756903426":"2024-04-22T08:29:23.190001+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:38:52.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.316+0000 7f7d4acc0640 1 -- 172.21.15.86:0/3315821057 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7d2c0a3db0 msgr2=0x7f7d2c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:52.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.316+0000 7f7d4acc0640 1 --2- 172.21.15.86:0/3315821057 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7d2c0a3db0 0x7f7d2c0a6270 secure :-1 s=READY pgs=49 cs=0 l=1 rev1=1 crypto rx=0x7f7d34006010 tx=0x7f7d3402c670 comp rx=0 tx=0).stop 2024-04-21T08:38:52.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.316+0000 7f7d4acc0640 1 -- 172.21.15.86:0/3315821057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d44105e30 msgr2=0x7f7d4419cfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:52.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.316+0000 7f7d4acc0640 1 --2- 172.21.15.86:0/3315821057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d44105e30 0x7f7d4419cfb0 secure :-1 s=READY pgs=543 cs=0 l=1 rev1=1 crypto rx=0x7f7d3c00eaa0 tx=0x7f7d3c00ef60 comp rx=0 tx=0).stop 2024-04-21T08:38:52.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.316+0000 7f7d4acc0640 1 -- 172.21.15.86:0/3315821057 shutdown_connections 2024-04-21T08:38:52.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.316+0000 7f7d4acc0640 1 --2- 172.21.15.86:0/3315821057 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7d2c0a3db0 0x7f7d2c0a6270 unknown :-1 s=CLOSED pgs=49 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.316+0000 7f7d4acc0640 1 --2- 172.21.15.86:0/3315821057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d44105e30 0x7f7d4419cfb0 unknown :-1 s=CLOSED pgs=543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.316+0000 7f7d4acc0640 1 --2- 172.21.15.86:0/3315821057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d44101b60 0x7f7d44198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.316+0000 7f7d4acc0640 1 --2- 172.21.15.86:0/3315821057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d441011a0 0x7f7d441986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.316+0000 7f7d4acc0640 1 -- 172.21.15.86:0/3315821057 >> 172.21.15.86:0/3315821057 conn(0x7f7d44077e30 msgr2=0x7f7d440693c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:52.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.316+0000 7f7d4acc0640 1 -- 172.21.15.86:0/3315821057 shutdown_connections 2024-04-21T08:38:52.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.316+0000 7f7d4acc0640 1 -- 172.21.15.86:0/3315821057 wait complete. 2024-04-21T08:38:52.328 DEBUG:tasks.fwd_scrub:Yielding 2024-04-21T08:38:52.328 INFO:teuthology.run_tasks:Running task exec... 2024-04-21T08:38:52.340 INFO:teuthology.task.exec:Executing custom commands... 2024-04-21T08:38:52.340 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi086.front.sepia.ceph.com 2024-04-21T08:38:52.340 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule add --fs=cephfs --path=/ --snap_schedule=1m' 2024-04-21T08:38:52.341 INFO:tasks.fwd_scrub.fs.[cephfs]:start scrubbing fs: cephfs 2024-04-21T08:38:52.341 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:38:52.341 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:38:52.341 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:38:52.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.444+0000 7fdaeb24e640 1 -- 172.21.15.86:0/3813678132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdae4106170 msgr2=0x7fdae410dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:52.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.444+0000 7fdaeb24e640 1 --2- 172.21.15.86:0/3813678132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdae4106170 0x7fdae410dd40 secure :-1 s=READY pgs=544 cs=0 l=1 rev1=1 crypto rx=0x7fdadc00be30 tx=0x7fdadc02dd20 comp rx=0 tx=0).stop 2024-04-21T08:38:52.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.444+0000 7fb9ce28c640 1 -- 172.21.15.86:0/418196933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9c8106170 msgr2=0x7fb9c810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:52.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.444+0000 7fb9ce28c640 1 --2- 172.21.15.86:0/418196933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9c8106170 0x7fb9c810dd40 secure :-1 s=READY pgs=545 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0009ca0 tx=0x7fb9c002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.444+0000 7fb9ce28c640 1 -- 172.21.15.86:0/418196933 shutdown_connections 2024-04-21T08:38:52.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.444+0000 7fb9ce28c640 1 --2- 172.21.15.86:0/418196933 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9c8106170 0x7fb9c810dd40 unknown :-1 s=CLOSED pgs=545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.444+0000 7fb9ce28c640 1 --2- 172.21.15.86:0/418196933 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9c8101aa0 0x7fb9c8105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.444+0000 7fb9ce28c640 1 --2- 172.21.15.86:0/418196933 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9c81010f0 0x7fb9c81014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.444+0000 7fb9ce28c640 1 -- 172.21.15.86:0/418196933 >> 172.21.15.86:0/418196933 conn(0x7fb9c8077fb0 msgr2=0x7fb9c80783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:52.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.444+0000 7fdaeb24e640 1 -- 172.21.15.86:0/3813678132 shutdown_connections 2024-04-21T08:38:52.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.444+0000 7fdaeb24e640 1 --2- 172.21.15.86:0/3813678132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdae4106170 0x7fdae410dd40 unknown :-1 s=CLOSED pgs=544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.444+0000 7fdaeb24e640 1 --2- 172.21.15.86:0/3813678132 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdae4101aa0 0x7fdae4105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.444+0000 7fdaeb24e640 1 --2- 172.21.15.86:0/3813678132 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdae41010f0 0x7fdae41014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.444+0000 7fdaeb24e640 1 -- 172.21.15.86:0/3813678132 >> 172.21.15.86:0/3813678132 conn(0x7fdae4077fd0 msgr2=0x7fdae40783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:52.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.444+0000 7fb9ce28c640 1 -- 172.21.15.86:0/418196933 shutdown_connections 2024-04-21T08:38:52.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.444+0000 7fb9ce28c640 1 -- 172.21.15.86:0/418196933 wait complete. 2024-04-21T08:38:52.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.444+0000 7fdaeb24e640 1 -- 172.21.15.86:0/3813678132 shutdown_connections 2024-04-21T08:38:52.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.444+0000 7fdaeb24e640 1 -- 172.21.15.86:0/3813678132 wait complete. 2024-04-21T08:38:52.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.445+0000 7fdaeb24e640 1 Processor -- start 2024-04-21T08:38:52.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.445+0000 7fb9ce28c640 1 Processor -- start 2024-04-21T08:38:52.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.445+0000 7fdaeb24e640 1 -- start start 2024-04-21T08:38:52.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.445+0000 7fb9ce28c640 1 -- start start 2024-04-21T08:38:52.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.445+0000 7fdaeb24e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdae41010f0 0x7fdae4196470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.445+0000 7fdaeb24e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdae4106170 0x7fdae41969b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.445+0000 7fdaeb24e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdae419ad30 0x7fdae419b1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.445+0000 7fdaeb24e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdae410ffd0 con 0x7fdae419ad30 2024-04-21T08:38:52.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.445+0000 7fdaeb24e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdae410fe50 con 0x7fdae41010f0 2024-04-21T08:38:52.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.445+0000 7fdaeb24e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdae4110150 con 0x7fdae4106170 2024-04-21T08:38:52.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.445+0000 7fdae97eb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdae419ad30 0x7fdae419b1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.445+0000 7fdae97eb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdae419ad30 0x7fdae419b1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36164/0 (socket says 172.21.15.86:36164) 2024-04-21T08:38:52.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.445+0000 7fdae97eb640 1 -- 172.21.15.86:0/834888398 learned_addr learned my addr 172.21.15.86:0/834888398 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:52.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.445+0000 7fb9ce28c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9c81010f0 0x7fb9c8198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.445+0000 7fdae8fea640 1 --2- 172.21.15.86:0/834888398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdae41010f0 0x7fdae4196470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.445+0000 7fb9ce28c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9c8106170 0x7fb9c8198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.445+0000 7fb9ce28c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9c819d010 0x7fb9c819d4c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.445+0000 7fb9ce28c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c81110e0 con 0x7fb9c81010f0 2024-04-21T08:38:52.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.445+0000 7fb9ce28c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c8110f60 con 0x7fb9c819d010 2024-04-21T08:38:52.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.445+0000 7fb9ce28c640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c8111260 con 0x7fb9c8106170 2024-04-21T08:38:52.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.445+0000 7fb9c7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9c81010f0 0x7fb9c8198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.445+0000 7fb9c7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9c81010f0 0x7fb9c8198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36178/0 (socket says 172.21.15.86:36178) 2024-04-21T08:38:52.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.445+0000 7fb9c7fff640 1 -- 172.21.15.86:0/1170022977 learned_addr learned my addr 172.21.15.86:0/1170022977 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:52.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.445+0000 7fdae3fff640 1 --2- 172.21.15.86:0/834888398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdae4106170 0x7fdae41969b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.445+0000 7fb9cc829640 1 --2- 172.21.15.86:0/1170022977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9c819d010 0x7fb9c819d4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.445+0000 7fdae97eb640 1 -- 172.21.15.86:0/834888398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdae4106170 msgr2=0x7fdae41969b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:52.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.445+0000 7fdae97eb640 1 --2- 172.21.15.86:0/834888398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdae4106170 0x7fdae41969b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.445+0000 7fdae97eb640 1 -- 172.21.15.86:0/834888398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdae41010f0 msgr2=0x7fdae4196470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:52.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.445+0000 7fdae97eb640 1 --2- 172.21.15.86:0/834888398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdae41010f0 0x7fdae4196470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.446+0000 7fdae97eb640 1 -- 172.21.15.86:0/834888398 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdae419b820 con 0x7fdae419ad30 2024-04-21T08:38:52.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.446+0000 7fb9c77fe640 1 --2- 172.21.15.86:0/1170022977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9c8106170 0x7fb9c8198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.446+0000 7fb9c7fff640 1 -- 172.21.15.86:0/1170022977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9c8106170 msgr2=0x7fb9c8198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:52.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.446+0000 7fb9c7fff640 1 --2- 172.21.15.86:0/1170022977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9c8106170 0x7fb9c8198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.446+0000 7fb9c7fff640 1 -- 172.21.15.86:0/1170022977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9c819d010 msgr2=0x7fb9c819d4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:52.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.446+0000 7fb9c7fff640 1 --2- 172.21.15.86:0/1170022977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9c819d010 0x7fb9c819d4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.446+0000 7fb9c7fff640 1 -- 172.21.15.86:0/1170022977 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb9c819db00 con 0x7fb9c81010f0 2024-04-21T08:38:52.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.446+0000 7fdae97eb640 1 --2- 172.21.15.86:0/834888398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdae419ad30 0x7fdae419b1e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fdadc02e330 tx=0x7fdadc003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:52.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.446+0000 7fb9c7fff640 1 --2- 172.21.15.86:0/1170022977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9c81010f0 0x7fb9c8198750 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb9b800e9f0 tx=0x7fb9b800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:52.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.446+0000 7fdae1ffb640 1 -- 172.21.15.86:0/834888398 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc03b040 con 0x7fdae419ad30 2024-04-21T08:38:52.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.446+0000 7fdaeb24e640 1 -- 172.21.15.86:0/834888398 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdae419bab0 con 0x7fdae419ad30 2024-04-21T08:38:52.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.446+0000 7fdae1ffb640 1 -- 172.21.15.86:0/834888398 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdadc046070 con 0x7fdae419ad30 2024-04-21T08:38:52.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.446+0000 7fdae1ffb640 1 -- 172.21.15.86:0/834888398 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdadc0369b0 con 0x7fdae419ad30 2024-04-21T08:38:52.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.446+0000 7fb9c57fa640 1 -- 172.21.15.86:0/1170022977 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b800cd80 con 0x7fb9c81010f0 2024-04-21T08:38:52.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.446+0000 7fb9c57fa640 1 -- 172.21.15.86:0/1170022977 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9b80054d0 con 0x7fb9c81010f0 2024-04-21T08:38:52.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.446+0000 7fb9c57fa640 1 -- 172.21.15.86:0/1170022977 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8010690 con 0x7fb9c81010f0 2024-04-21T08:38:52.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.446+0000 7fb9ce28c640 1 -- 172.21.15.86:0/1170022977 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb9c819ddf0 con 0x7fb9c81010f0 2024-04-21T08:38:52.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.446+0000 7fb9ce28c640 1 -- 172.21.15.86:0/1170022977 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9c81a5590 con 0x7fb9c81010f0 2024-04-21T08:38:52.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.446+0000 7fdaeb24e640 1 -- 172.21.15.86:0/834888398 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdae41a3270 con 0x7fdae419ad30 2024-04-21T08:38:52.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.447+0000 7fdaeb24e640 1 -- 172.21.15.86:0/834888398 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdae404fce0 con 0x7fdae419ad30 2024-04-21T08:38:52.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.449+0000 7fb9c57fa640 1 -- 172.21.15.86:0/1170022977 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fb9b8004e90 con 0x7fb9c81010f0 2024-04-21T08:38:52.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.449+0000 7fb9d0b22740 1 Processor -- start 2024-04-21T08:38:52.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.449+0000 7fb9d0b22740 1 -- start start 2024-04-21T08:38:52.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.449+0000 7fb9d0b22740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a8599e8e0 0x556a8599ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.449+0000 7fb9d0b22740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a8599f290 0x556a859a7840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.449+0000 7fb9d0b22740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a859a7e10 0x556a859aa260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.449+0000 7fb9d0b22740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556a85902f00 con 0x556a859a7e10 2024-04-21T08:38:52.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.449+0000 7fb9d0b22740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x556a858a0590 con 0x556a8599f290 2024-04-21T08:38:52.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.449+0000 7fb9d0b22740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556a858feec0 con 0x556a8599e8e0 2024-04-21T08:38:52.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.449+0000 7fb9c77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a859a7e10 0x556a859aa260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.449+0000 7fb9c7fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a8599f290 0x556a859a7840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.449+0000 7fb9c7fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a8599f290 0x556a859a7840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.86:58728/0 (socket says 172.21.15.86:58728) 2024-04-21T08:38:52.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.449+0000 7fb9c7fff640 1 -- 172.21.15.86:0/3995268485 learned_addr learned my addr 172.21.15.86:0/3995268485 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:52.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.449+0000 7fb9cc829640 1 --2- 172.21.15.86:0/3995268485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a8599e8e0 0x556a8599ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.449+0000 7fb9c57fa640 1 --2- 172.21.15.86:0/1170022977 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb9b00a3db0 0x7fb9b00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.450+0000 7fb9c7fff640 1 -- 172.21.15.86:0/3995268485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a8599e8e0 msgr2=0x556a8599ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:52.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.450+0000 7fb9c7fff640 1 --2- 172.21.15.86:0/3995268485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a8599e8e0 0x556a8599ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.450+0000 7fb9c7fff640 1 -- 172.21.15.86:0/3995268485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a859a7e10 msgr2=0x556a859aa260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:52.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.450+0000 7fb9c7fff640 1 --2- 172.21.15.86:0/3995268485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a859a7e10 0x556a859aa260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.450+0000 7fb9c7fff640 1 -- 172.21.15.86:0/3995268485 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a856dd7e0 con 0x556a8599f290 2024-04-21T08:38:52.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.450+0000 7fb9c77fe640 1 --2- 172.21.15.86:0/3995268485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a859a7e10 0x556a859aa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:38:52.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.450+0000 7fb9c57fa640 1 -- 172.21.15.86:0/1170022977 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fb9b80d11c0 con 0x7fb9c81010f0 2024-04-21T08:38:52.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.450+0000 7fb9cc829640 1 --2- 172.21.15.86:0/1170022977 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb9b00a3db0 0x7fb9b00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.450+0000 7fdae1ffb640 1 -- 172.21.15.86:0/834888398 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fdadc009ae0 con 0x7fdae419ad30 2024-04-21T08:38:52.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.450+0000 7fb9c7fff640 1 --2- 172.21.15.86:0/3995268485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a8599f290 0x556a859a7840 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb9b809c980 tx=0x7fb9b809cb80 comp rx=0 tx=0).ready entity=mon.2 client_cookie=2a99e32d0fbc6101 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:52.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.450+0000 7fb9ae7fc640 1 -- 172.21.15.86:0/3995268485 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b809c7c0 con 0x556a8599f290 2024-04-21T08:38:52.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.450+0000 7fb9ae7fc640 1 -- 172.21.15.86:0/3995268485 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9b8005680 con 0x556a8599f290 2024-04-21T08:38:52.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.450+0000 7fb9ae7fc640 1 -- 172.21.15.86:0/3995268485 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9b8005180 con 0x556a8599f290 2024-04-21T08:38:52.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.450+0000 7fdae1ffb640 1 --2- 172.21.15.86:0/834888398 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdad40a3e60 0x7fdad40a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.451+0000 7fdae8fea640 1 --2- 172.21.15.86:0/834888398 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdad40a3e60 0x7fdad40a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.451+0000 7fdae1ffb640 1 -- 172.21.15.86:0/834888398 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fdadc0f5550 con 0x7fdae419ad30 2024-04-21T08:38:52.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.451+0000 7fb9d0b22740 1 -- 172.21.15.86:0/3995268485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a8599f290 msgr2=0x556a859a7840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:52.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.451+0000 7fb9d0b22740 1 --2- 172.21.15.86:0/3995268485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a8599f290 0x556a859a7840 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7fb9b809c980 tx=0x7fb9b809cb80 comp rx=0 tx=0).stop 2024-04-21T08:38:52.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.451+0000 7fb9cc829640 1 --2- 172.21.15.86:0/1170022977 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb9b00a3db0 0x7fb9b00a6270 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0005e30 tx=0x7fb9c00095c0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:52.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.451+0000 7fb9d0b22740 1 -- 172.21.15.86:0/3995268485 shutdown_connections 2024-04-21T08:38:52.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.451+0000 7fb9d0b22740 1 --2- 172.21.15.86:0/3995268485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a859a7e10 0x556a859aa260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.451+0000 7fb9d0b22740 1 --2- 172.21.15.86:0/3995268485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a8599f290 0x556a859a7840 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.451+0000 7fb9d0b22740 1 --2- 172.21.15.86:0/3995268485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a8599e8e0 0x556a8599ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.451+0000 7fb9d0b22740 1 -- 172.21.15.86:0/3995268485 >> 172.21.15.86:0/3995268485 conn(0x556a85998fb0 msgr2=0x556a85999390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:52.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.451+0000 7fb9d0b22740 1 -- 172.21.15.86:0/3995268485 shutdown_connections 2024-04-21T08:38:52.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.451+0000 7fb9d0b22740 1 -- 172.21.15.86:0/3995268485 wait complete. 2024-04-21T08:38:52.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.451+0000 7fdae8fea640 1 --2- 172.21.15.86:0/834888398 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdad40a3e60 0x7fdad40a6320 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdad8009bd0 tx=0x7fdad8005f90 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:52.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.451+0000 7fb9d0b22740 1 -- start start 2024-04-21T08:38:52.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.452+0000 7fb9d0b22740 1 Processor -- start 2024-04-21T08:38:52.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.452+0000 7fb9d0b22740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a8599e8e0 0x556a859a4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.452+0000 7fb9d0b22740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a8599f290 0x556a859a5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.452+0000 7fb9d0b22740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a859a7e10 0x556a859a5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.452+0000 7fb9d0b22740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556a859aad20 con 0x556a859a7e10 2024-04-21T08:38:52.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.452+0000 7fb9d0b22740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x556a858bc580 con 0x556a8599f290 2024-04-21T08:38:52.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.452+0000 7fb9d0b22740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556a859aaea0 con 0x556a8599e8e0 2024-04-21T08:38:52.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.452+0000 7fb9d0b22740 10 client.0 _ll_register_callbacks cb 0x556a858c2650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:52.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.452+0000 7fb9cc829640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a859a7e10 0x556a859a5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.452+0000 7fb9cc829640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a859a7e10 0x556a859a5850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36196/0 (socket says 172.21.15.86:36196) 2024-04-21T08:38:52.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.452+0000 7fb9cc829640 1 -- 172.21.15.86:0/4185610656 learned_addr learned my addr 172.21.15.86:0/4185610656 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:52.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.452+0000 7fb9c7fff640 1 --2- 172.21.15.86:0/4185610656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a8599e8e0 0x556a859a4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.452+0000 7fb9c77fe640 1 --2- 172.21.15.86:0/4185610656 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a8599f290 0x556a859a5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.452+0000 7fb9cc829640 1 -- 172.21.15.86:0/4185610656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a8599e8e0 msgr2=0x556a859a4dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:52.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.452+0000 7fb9cc829640 1 --2- 172.21.15.86:0/4185610656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a8599e8e0 0x556a859a4dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.452+0000 7fb9cc829640 1 -- 172.21.15.86:0/4185610656 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a8599f290 msgr2=0x556a859a5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:52.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.452+0000 7fb9cc829640 1 --2- 172.21.15.86:0/4185610656 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a8599f290 0x556a859a5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.452+0000 7fb9cc829640 1 -- 172.21.15.86:0/4185610656 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a8573df70 con 0x556a859a7e10 2024-04-21T08:38:52.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.452+0000 7fb9cc829640 1 --2- 172.21.15.86:0/4185610656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a859a7e10 0x556a859a5850 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0032fc0 tx=0x7fb9c002c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee68785d5fce6a36 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:52.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.452+0000 7fb9a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:38:52.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.454+0000 7fb9a77fe640 1 -- 172.21.15.86:0/4185610656 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0052060 con 0x556a859a7e10 2024-04-21T08:38:52.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.454+0000 7fb9a77fe640 1 -- 172.21.15.86:0/4185610656 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fb9c0003d80 con 0x556a859a7e10 2024-04-21T08:38:52.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.454+0000 7fb9d0b22740 1 -- 172.21.15.86:0/4185610656 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9b8005680 con 0x556a859a7e10 2024-04-21T08:38:52.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.454+0000 7fdae1ffb640 1 -- 172.21.15.86:0/834888398 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdadc0c1820 con 0x7fdae419ad30 2024-04-21T08:38:52.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.454+0000 7fb9a77fe640 1 -- 172.21.15.86:0/4185610656 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb9c0059050 con 0x556a859a7e10 2024-04-21T08:38:52.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.454+0000 7fb9d0b22740 10 client.15771 fetch_fsmap learned FSMap version 29 2024-04-21T08:38:52.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.454+0000 7fb9d0b22740 1 -- 172.21.15.86:0/4185610656 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556a855d5170 con 0x556a859a7e10 2024-04-21T08:38:52.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.455+0000 7fb9a77fe640 1 -- 172.21.15.86:0/4185610656 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9b8005680 con 0x556a859a7e10 2024-04-21T08:38:52.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.455+0000 7fb9a77fe640 1 -- 172.21.15.86:0/4185610656 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb9c003f040 con 0x556a859a7e10 2024-04-21T08:38:52.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.455+0000 7fb9d0b22740 10 client.15771 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:38:52.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.455+0000 7fb9d0b22740 10 client.15771 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:38:52.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.455+0000 7fb9d0b22740 20 client.15771 populate_metadata read hostname 'smithi086' 2024-04-21T08:38:52.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.455+0000 7fb9d0b22740 1 --2- 172.21.15.86:0/4185610656 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x556a859b82d0 0x556a859ba6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.455+0000 7fb9d0b22740 4 client.15771 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:38:52.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.455+0000 7fb9d0b22740 1 -- 172.21.15.86:0/4185610656 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a85910920 con 0x556a859b82d0 2024-04-21T08:38:52.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.455+0000 7fb9c7fff640 1 --2- 172.21.15.86:0/4185610656 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x556a859b82d0 0x556a859ba6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.455+0000 7fb9a77fe640 10 client.15771 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:38:52.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.456+0000 7fb9c7fff640 1 --2- 172.21.15.86:0/4185610656 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x556a859b82d0 0x556a859ba6f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f81699a1c72a1f71 server_cookie=70c0755013ab9506 in_seq=0 out_seq=0 2024-04-21T08:38:52.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.456+0000 7fb9a77fe640 1 -- 172.21.15.86:0/4185610656 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556a85910920 con 0x556a859b82d0 2024-04-21T08:38:52.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.456+0000 7fb9a77fe640 10 client.15771 handle_command_reply: tid=0 2024-04-21T08:38:52.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.456+0000 7fb9d0b22740 1 client.15771 shutdown 2024-04-21T08:38:52.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.456+0000 7fb9d0b22740 1 -- 172.21.15.86:0/4185610656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a859a7e10 msgr2=0x556a859a5850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:52.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.456+0000 7fb9d0b22740 1 --2- 172.21.15.86:0/4185610656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a859a7e10 0x556a859a5850 secure :-1 s=READY pgs=548 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0032fc0 tx=0x7fb9c002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.456+0000 7fb9d0b22740 1 -- 172.21.15.86:0/4185610656 shutdown_connections 2024-04-21T08:38:52.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.456+0000 7fb9d0b22740 1 --2- 172.21.15.86:0/4185610656 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x556a859b82d0 0x556a859ba6f0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.457+0000 7fb9a77fe640 0 client.15771 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:38:52.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.457+0000 7fb9d0b22740 1 --2- 172.21.15.86:0/4185610656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a859a7e10 0x556a859a5850 unknown :-1 s=CLOSED pgs=548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.457+0000 7fb9d0b22740 1 --2- 172.21.15.86:0/4185610656 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a8599f290 0x556a859a5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.457+0000 7fb9d0b22740 1 --2- 172.21.15.86:0/4185610656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a8599e8e0 0x556a859a4dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.457+0000 7fb9d0b22740 1 -- 172.21.15.86:0/4185610656 >> 172.21.15.86:0/4185610656 conn(0x556a85998fb0 msgr2=0x556a8599ad00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:52.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.457+0000 7fb9d0b22740 1 -- 172.21.15.86:0/4185610656 shutdown_connections 2024-04-21T08:38:52.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.457+0000 7fb9d0b22740 1 -- 172.21.15.86:0/4185610656 wait complete. 2024-04-21T08:38:52.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.457+0000 7fb9d0b22740 20 client.15771 trim_cache size 0 max 16384 2024-04-21T08:38:52.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.474+0000 7fb9d0b22740 1 Processor -- start 2024-04-21T08:38:52.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.474+0000 7fb9d0b22740 1 -- start start 2024-04-21T08:38:52.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.474+0000 7fb9d0b22740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a8599e8e0 0x556a859a7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.474+0000 7fb9d0b22740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a8599f290 0x556a859a3d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.474+0000 7fb9d0b22740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a859a7e10 0x556a859b3be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.474+0000 7fb9d0b22740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c81b1550 con 0x556a859a7e10 2024-04-21T08:38:52.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.474+0000 7fb9d0b22740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c81b3090 con 0x556a8599e8e0 2024-04-21T08:38:52.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.474+0000 7fb9d0b22740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c81b1c20 con 0x556a8599f290 2024-04-21T08:38:52.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.474+0000 7fb9cc829640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a859a7e10 0x556a859b3be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.474+0000 7fb9cc829640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a859a7e10 0x556a859b3be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36212/0 (socket says 172.21.15.86:36212) 2024-04-21T08:38:52.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.474+0000 7fb9cc829640 1 -- 172.21.15.86:0/44253486 learned_addr learned my addr 172.21.15.86:0/44253486 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:52.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.474+0000 7fb9c77fe640 1 --2- 172.21.15.86:0/44253486 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a8599f290 0x556a859a3d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.474+0000 7fb9c7fff640 1 --2- 172.21.15.86:0/44253486 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a8599e8e0 0x556a859a7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.474+0000 7fb9cc829640 1 -- 172.21.15.86:0/44253486 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a8599f290 msgr2=0x556a859a3d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:52.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.474+0000 7fb9cc829640 1 --2- 172.21.15.86:0/44253486 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a8599f290 0x556a859a3d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.474+0000 7fb9cc829640 1 -- 172.21.15.86:0/44253486 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a8599e8e0 msgr2=0x556a859a7500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:52.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.474+0000 7fb9cc829640 1 --2- 172.21.15.86:0/44253486 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a8599e8e0 0x556a859a7500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.474+0000 7fb9cc829640 1 -- 172.21.15.86:0/44253486 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a859aac60 con 0x556a859a7e10 2024-04-21T08:38:52.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.475+0000 7fb9cc829640 1 --2- 172.21.15.86:0/44253486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a859a7e10 0x556a859b3be0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0004af0 tx=0x7fb9c0005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52a7fa0f5985925d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:52.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.475+0000 7fb9adffb640 1 -- 172.21.15.86:0/44253486 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0062070 con 0x556a859a7e10 2024-04-21T08:38:52.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.475+0000 7fb9adffb640 1 -- 172.21.15.86:0/44253486 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9c0032890 con 0x556a859a7e10 2024-04-21T08:38:52.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.475+0000 7fb9adffb640 1 -- 172.21.15.86:0/44253486 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9c0052070 con 0x556a859a7e10 2024-04-21T08:38:52.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.475+0000 7fb9d0b22740 1 -- 172.21.15.86:0/44253486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a859a7e10 msgr2=0x556a859b3be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:52.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.476+0000 7fb9d0b22740 1 --2- 172.21.15.86:0/44253486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a859a7e10 0x556a859b3be0 secure :-1 s=READY pgs=549 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0004af0 tx=0x7fb9c0005dc0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.476+0000 7fb9d0b22740 1 -- 172.21.15.86:0/44253486 shutdown_connections 2024-04-21T08:38:52.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.476+0000 7fb9d0b22740 1 --2- 172.21.15.86:0/44253486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a859a7e10 0x556a859b3be0 unknown :-1 s=CLOSED pgs=549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.476+0000 7fb9d0b22740 1 --2- 172.21.15.86:0/44253486 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a8599f290 0x556a859a3d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.476+0000 7fb9d0b22740 1 --2- 172.21.15.86:0/44253486 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a8599e8e0 0x556a859a7500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.476+0000 7fb9d0b22740 1 -- 172.21.15.86:0/44253486 >> 172.21.15.86:0/44253486 conn(0x556a85998fb0 msgr2=0x556a85998010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:52.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.476+0000 7fb9d0b22740 1 -- 172.21.15.86:0/44253486 shutdown_connections 2024-04-21T08:38:52.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.476+0000 7fb9d0b22740 1 -- 172.21.15.86:0/44253486 wait complete. 2024-04-21T08:38:52.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.476+0000 7fb9d0b22740 1 -- start start 2024-04-21T08:38:52.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.476+0000 7fb9d0b22740 1 Processor -- start 2024-04-21T08:38:52.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.476+0000 7fb9d0b22740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a8599e8e0 0x556a859a1c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.476+0000 7fb9d0b22740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a8599f290 0x556a859a2150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.476+0000 7fb9d0b22740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a859a7e10 0x556a859a2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.476+0000 7fb9d0b22740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c81b29c0 con 0x556a8599f290 2024-04-21T08:38:52.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.476+0000 7fb9d0b22740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c81b22f0 con 0x556a8599e8e0 2024-04-21T08:38:52.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.476+0000 7fb9d0b22740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb9c81af510 con 0x556a859a7e10 2024-04-21T08:38:52.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.476+0000 7fb9d0b22740 10 client.0 _ll_register_callbacks cb 0x7fb9c0037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:52.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.476+0000 7fb9c77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a8599f290 0x556a859a2150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.476+0000 7fb9c77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a8599f290 0x556a859a2150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36228/0 (socket says 172.21.15.86:36228) 2024-04-21T08:38:52.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.476+0000 7fb9c77fe640 1 -- 172.21.15.86:0/3853824709 learned_addr learned my addr 172.21.15.86:0/3853824709 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:52.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.477+0000 7fb9c7fff640 1 --2- 172.21.15.86:0/3853824709 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a8599e8e0 0x556a859a1c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.477+0000 7fb9cc829640 1 --2- 172.21.15.86:0/3853824709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a859a7e10 0x556a859a2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.477+0000 7fb9c77fe640 1 -- 172.21.15.86:0/3853824709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a859a7e10 msgr2=0x556a859a2690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:52.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.477+0000 7fb9c77fe640 1 --2- 172.21.15.86:0/3853824709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a859a7e10 0x556a859a2690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.477+0000 7fb9c77fe640 1 -- 172.21.15.86:0/3853824709 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a8599e8e0 msgr2=0x556a859a1c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:52.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.477+0000 7fb9c77fe640 1 --2- 172.21.15.86:0/3853824709 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a8599e8e0 0x556a859a1c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.477+0000 7fb9c77fe640 1 -- 172.21.15.86:0/3853824709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a85a1d9e0 con 0x556a8599f290 2024-04-21T08:38:52.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.477+0000 7fb9c7fff640 1 --2- 172.21.15.86:0/3853824709 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a8599e8e0 0x556a859a1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:38:52.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.477+0000 7fb9c77fe640 1 --2- 172.21.15.86:0/3853824709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a8599f290 0x556a859a2150 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8068e50 tx=0x7fb9bc009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcfeb137365213b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:52.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.477+0000 7fb9a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:38:52.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.477+0000 7fb9a77fe640 1 -- 172.21.15.86:0/3853824709 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9bc011bc0 con 0x556a8599f290 2024-04-21T08:38:52.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.477+0000 7fb9a77fe640 1 -- 172.21.15.86:0/3853824709 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fb9bc011d60 con 0x556a8599f290 2024-04-21T08:38:52.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.477+0000 7fb9d0b22740 1 -- 172.21.15.86:0/3853824709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9c0032890 con 0x556a8599f290 2024-04-21T08:38:52.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.477+0000 7fb9a77fe640 1 -- 172.21.15.86:0/3853824709 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb9bc02c050 con 0x556a8599f290 2024-04-21T08:38:52.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.477+0000 7fb9d0b22740 10 client.15777 fetch_fsmap learned FSMap version 29 2024-04-21T08:38:52.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.477+0000 7fb9d0b22740 1 -- 172.21.15.86:0/3853824709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556a85a1ed40 con 0x556a8599f290 2024-04-21T08:38:52.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.477+0000 7fb9a77fe640 1 -- 172.21.15.86:0/3853824709 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9c0032890 con 0x556a8599f290 2024-04-21T08:38:52.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.478+0000 7fb9a77fe640 1 -- 172.21.15.86:0/3853824709 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb9bc01d470 con 0x556a8599f290 2024-04-21T08:38:52.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.478+0000 7fb9d0b22740 10 client.15777 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:38:52.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.478+0000 7fb9d0b22740 10 client.15777 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:38:52.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.478+0000 7fb9d0b22740 20 client.15777 populate_metadata read hostname 'smithi086' 2024-04-21T08:38:52.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.478+0000 7fb9d0b22740 1 --2- 172.21.15.86:0/3853824709 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x556a859b82d0 0x556a85a215b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.478+0000 7fb9d0b22740 4 client.15777 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:38:52.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.478+0000 7fb9d0b22740 1 -- 172.21.15.86:0/3853824709 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556a8590a790 con 0x556a859b82d0 2024-04-21T08:38:52.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.478+0000 7fb9c7fff640 1 --2- 172.21.15.86:0/3853824709 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x556a859b82d0 0x556a85a215b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.478+0000 7fb9a77fe640 10 client.15777 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:38:52.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.478+0000 7fb9c7fff640 1 --2- 172.21.15.86:0/3853824709 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x556a859b82d0 0x556a85a215b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5213f2b8eefa2f44 server_cookie=522845625b289c7 in_seq=0 out_seq=0 2024-04-21T08:38:52.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.479+0000 7fb9a77fe640 1 -- 172.21.15.86:0/3853824709 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556a8590a790 con 0x556a859b82d0 2024-04-21T08:38:52.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.479+0000 7fb9a77fe640 10 client.15777 handle_command_reply: tid=0 2024-04-21T08:38:52.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.479+0000 7fb9d0b22740 1 client.15777 shutdown 2024-04-21T08:38:52.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.480+0000 7fb9d0b22740 1 -- 172.21.15.86:0/3853824709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a8599f290 msgr2=0x556a859a2150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:52.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.480+0000 7fb9d0b22740 1 --2- 172.21.15.86:0/3853824709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a8599f290 0x556a859a2150 secure :-1 s=READY pgs=550 cs=0 l=1 rev1=1 crypto rx=0x7fb9c8068e50 tx=0x7fb9bc009f90 comp rx=0 tx=0).stop 2024-04-21T08:38:52.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.480+0000 7fb9d0b22740 1 -- 172.21.15.86:0/3853824709 shutdown_connections 2024-04-21T08:38:52.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.480+0000 7fb9d0b22740 1 --2- 172.21.15.86:0/3853824709 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x556a859b82d0 0x556a85a215b0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.480+0000 7fb9d0b22740 1 --2- 172.21.15.86:0/3853824709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a859a7e10 0x556a859a2690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.480+0000 7fb9d0b22740 1 --2- 172.21.15.86:0/3853824709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a8599f290 0x556a859a2150 unknown :-1 s=CLOSED pgs=550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.480+0000 7fb9d0b22740 1 --2- 172.21.15.86:0/3853824709 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a8599e8e0 0x556a859a1c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.480+0000 7fb9d0b22740 1 -- 172.21.15.86:0/3853824709 >> 172.21.15.86:0/3853824709 conn(0x556a85998fb0 msgr2=0x556a859a4d90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:52.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.480+0000 7fb9a77fe640 0 client.15777 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:38:52.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.480+0000 7fb9d0b22740 1 -- 172.21.15.86:0/3853824709 shutdown_connections 2024-04-21T08:38:52.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.480+0000 7fb9d0b22740 1 -- 172.21.15.86:0/3853824709 wait complete. 2024-04-21T08:38:52.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.480+0000 7fb9d0b22740 20 client.15777 trim_cache size 0 max 16384 2024-04-21T08:38:52.492 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:38:52.492 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-21T08:38:52.492 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "a22b346e-63e5-4ab5-b852-c599e1b8c333", 2024-04-21T08:38:52.492 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-21T08:38:52.492 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:38:52.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.480+0000 7fb9ce28c640 1 -- 172.21.15.86:0/1170022977 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb9b00a3db0 msgr2=0x7fb9b00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:52.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.480+0000 7fb9ce28c640 1 --2- 172.21.15.86:0/1170022977 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb9b00a3db0 0x7fb9b00a6270 secure :-1 s=READY pgs=50 cs=0 l=1 rev1=1 crypto rx=0x7fb9c0005e30 tx=0x7fb9c00095c0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.480+0000 7fb9ce28c640 1 -- 172.21.15.86:0/1170022977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9c81010f0 msgr2=0x7fb9c8198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:52.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.480+0000 7fb9ce28c640 1 --2- 172.21.15.86:0/1170022977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9c81010f0 0x7fb9c8198750 secure :-1 s=READY pgs=547 cs=0 l=1 rev1=1 crypto rx=0x7fb9b800e9f0 tx=0x7fb9b800edf0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.481+0000 7fb9ce28c640 1 -- 172.21.15.86:0/1170022977 shutdown_connections 2024-04-21T08:38:52.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.481+0000 7fb9ce28c640 1 --2- 172.21.15.86:0/1170022977 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb9b00a3db0 0x7fb9b00a6270 unknown :-1 s=CLOSED pgs=50 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.481+0000 7fb9ce28c640 1 --2- 172.21.15.86:0/1170022977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb9c819d010 0x7fb9c819d4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.481+0000 7fb9ce28c640 1 --2- 172.21.15.86:0/1170022977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb9c8106170 0x7fb9c8198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.481+0000 7fb9ce28c640 1 --2- 172.21.15.86:0/1170022977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb9c81010f0 0x7fb9c8198750 unknown :-1 s=CLOSED pgs=547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.481+0000 7fb9ce28c640 1 -- 172.21.15.86:0/1170022977 >> 172.21.15.86:0/1170022977 conn(0x7fb9c8077fb0 msgr2=0x7fb9c806a500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:52.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.481+0000 7fb9ce28c640 1 -- 172.21.15.86:0/1170022977 shutdown_connections 2024-04-21T08:38:52.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.481+0000 7fb9ce28c640 1 -- 172.21.15.86:0/1170022977 wait complete. 2024-04-21T08:38:52.494 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:38:52.495 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:38:52.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.596+0000 7f238ce21640 1 -- 172.21.15.86:0/1783234999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2388101a30 msgr2=0x7f2388105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:52.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.596+0000 7f238ce21640 1 --2- 172.21.15.86:0/1783234999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2388101a30 0x7f2388105a70 secure :-1 s=READY pgs=551 cs=0 l=1 rev1=1 crypto rx=0x7f2378009900 tx=0x7f237802bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:52.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.596+0000 7f238ce21640 1 -- 172.21.15.86:0/1783234999 shutdown_connections 2024-04-21T08:38:52.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.596+0000 7f238ce21640 1 --2- 172.21.15.86:0/1783234999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2388106150 0x7f238810dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.596+0000 7f238ce21640 1 --2- 172.21.15.86:0/1783234999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2388101a30 0x7f2388105a70 unknown :-1 s=CLOSED pgs=551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.596+0000 7f238ce21640 1 --2- 172.21.15.86:0/1783234999 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2388101080 0x7f2388101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.596+0000 7f238ce21640 1 -- 172.21.15.86:0/1783234999 >> 172.21.15.86:0/1783234999 conn(0x7f2388078000 msgr2=0x7f2388078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:52.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.596+0000 7f238ce21640 1 -- 172.21.15.86:0/1783234999 shutdown_connections 2024-04-21T08:38:52.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.596+0000 7f238ce21640 1 -- 172.21.15.86:0/1783234999 wait complete. 2024-04-21T08:38:52.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.597+0000 7f238ce21640 1 Processor -- start 2024-04-21T08:38:52.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.597+0000 7f238ce21640 1 -- start start 2024-04-21T08:38:52.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.597+0000 7f238ce21640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2388101080 0x7f2388198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.597+0000 7f238ce21640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2388106150 0x7f2388198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.597+0000 7f238ce21640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f238819cf30 0x7f238819d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.597+0000 7f238ce21640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f23881121d0 con 0x7f238819cf30 2024-04-21T08:38:52.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.597+0000 7f238ce21640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2388112050 con 0x7f2388101080 2024-04-21T08:38:52.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.597+0000 7f238ce21640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2388112350 con 0x7f2388106150 2024-04-21T08:38:52.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.598+0000 7f2386d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f238819cf30 0x7f238819d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.598+0000 7f2386d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f238819cf30 0x7f238819d3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36252/0 (socket says 172.21.15.86:36252) 2024-04-21T08:38:52.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.598+0000 7f2386d9d640 1 -- 172.21.15.86:0/1459467481 learned_addr learned my addr 172.21.15.86:0/1459467481 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:52.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.598+0000 7f2385d9b640 1 --2- 172.21.15.86:0/1459467481 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2388106150 0x7f2388198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.598+0000 7f238659c640 1 --2- 172.21.15.86:0/1459467481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2388101080 0x7f2388198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.598+0000 7f2386d9d640 1 -- 172.21.15.86:0/1459467481 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2388106150 msgr2=0x7f2388198bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:52.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.598+0000 7f2386d9d640 1 --2- 172.21.15.86:0/1459467481 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2388106150 0x7f2388198bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.598+0000 7f2386d9d640 1 -- 172.21.15.86:0/1459467481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2388101080 msgr2=0x7f2388198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:52.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.598+0000 7f2386d9d640 1 --2- 172.21.15.86:0/1459467481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2388101080 0x7f2388198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.598+0000 7f2386d9d640 1 -- 172.21.15.86:0/1459467481 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f238819da20 con 0x7f238819cf30 2024-04-21T08:38:52.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.598+0000 7f2385d9b640 1 --2- 172.21.15.86:0/1459467481 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2388106150 0x7f2388198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:52.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.598+0000 7f238659c640 1 --2- 172.21.15.86:0/1459467481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2388101080 0x7f2388198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:52.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.598+0000 7f2386d9d640 1 --2- 172.21.15.86:0/1459467481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f238819cf30 0x7f238819d3e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f238000eaa0 tx=0x7f238000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:52.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.598+0000 7f23777fe640 1 -- 172.21.15.86:0/1459467481 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f238000cdb0 con 0x7f238819cf30 2024-04-21T08:38:52.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.598+0000 7f23777fe640 1 -- 172.21.15.86:0/1459467481 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2380005020 con 0x7f238819cf30 2024-04-21T08:38:52.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.598+0000 7f238ce21640 1 -- 172.21.15.86:0/1459467481 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f238819dd10 con 0x7f238819cf30 2024-04-21T08:38:52.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.598+0000 7f23777fe640 1 -- 172.21.15.86:0/1459467481 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23800116d0 con 0x7f238819cf30 2024-04-21T08:38:52.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.598+0000 7f238ce21640 1 -- 172.21.15.86:0/1459467481 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f23881a54d0 con 0x7f238819cf30 2024-04-21T08:38:52.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.600+0000 7f23777fe640 1 -- 172.21.15.86:0/1459467481 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f23800051d0 con 0x7f238819cf30 2024-04-21T08:38:52.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.600+0000 7f238fa0b740 1 Processor -- start 2024-04-21T08:38:52.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.601+0000 7f238fa0b740 1 -- start start 2024-04-21T08:38:52.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.601+0000 7f238fa0b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5652562c02b0 0x5652562c0690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.601+0000 7f238fa0b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652562c0bd0 0x5652562cd950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.601+0000 7f238fa0b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652562cde90 0x5652562d0280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.601+0000 7f238fa0b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5652562251e0 con 0x5652562cde90 2024-04-21T08:38:52.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.601+0000 7f238fa0b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5652561c2390 con 0x5652562c02b0 2024-04-21T08:38:52.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.601+0000 7f238fa0b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5652562211a0 con 0x5652562c0bd0 2024-04-21T08:38:52.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.601+0000 7f2385d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652562cde90 0x5652562d0280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.601+0000 7f2385d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652562cde90 0x5652562d0280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36264/0 (socket says 172.21.15.86:36264) 2024-04-21T08:38:52.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.601+0000 7f2385d9b640 1 -- 172.21.15.86:0/2576165737 learned_addr learned my addr 172.21.15.86:0/2576165737 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:52.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.601+0000 7f2385d9b640 1 --2- 172.21.15.86:0/2576165737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5652562c02b0 0x5652562c0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.601+0000 7f23777fe640 1 --2- 172.21.15.86:0/1459467481 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f23700a3db0 0x7f23700a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.601+0000 7f238659c640 1 --2- 172.21.15.86:0/2576165737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652562c0bd0 0x5652562cd950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.601+0000 7f2385d9b640 1 -- 172.21.15.86:0/2576165737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652562c0bd0 msgr2=0x5652562cd950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:52.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.601+0000 7f2385d9b640 1 --2- 172.21.15.86:0/2576165737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652562c0bd0 0x5652562cd950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.601+0000 7f2386d9d640 1 --2- 172.21.15.86:0/1459467481 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f23700a3db0 0x7f23700a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.601+0000 7f2385d9b640 1 -- 172.21.15.86:0/2576165737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5652562c02b0 msgr2=0x5652562c0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:52.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.601+0000 7f2385d9b640 1 --2- 172.21.15.86:0/2576165737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5652562c02b0 0x5652562c0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.601+0000 7f2385d9b640 1 -- 172.21.15.86:0/2576165737 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652560017d0 con 0x5652562cde90 2024-04-21T08:38:52.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.602+0000 7f23777fe640 1 -- 172.21.15.86:0/1459467481 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f23800d0ec0 con 0x7f238819cf30 2024-04-21T08:38:52.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.602+0000 7f2385d9b640 1 --2- 172.21.15.86:0/2576165737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652562cde90 0x5652562d0280 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f238806b6f0 tx=0x7f2378005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7df04cdb5ecc5360 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:52.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.602+0000 7f2386d9d640 1 --2- 172.21.15.86:0/1459467481 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f23700a3db0 0x7f23700a6270 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f23800ce240 tx=0x7f23800bd040 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:52.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.602+0000 7f2374ff9640 1 -- 172.21.15.86:0/2576165737 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2378049070 con 0x5652562cde90 2024-04-21T08:38:52.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.602+0000 7f2374ff9640 1 -- 172.21.15.86:0/2576165737 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f237802c9e0 con 0x5652562cde90 2024-04-21T08:38:52.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.602+0000 7f2374ff9640 1 -- 172.21.15.86:0/2576165737 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2378043450 con 0x5652562cde90 2024-04-21T08:38:52.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.603+0000 7f238fa0b740 1 -- 172.21.15.86:0/2576165737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652562cde90 msgr2=0x5652562d0280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:52.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.603+0000 7f238fa0b740 1 --2- 172.21.15.86:0/2576165737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652562cde90 0x5652562d0280 secure :-1 s=READY pgs=553 cs=0 l=1 rev1=1 crypto rx=0x7f238806b6f0 tx=0x7f2378005b50 comp rx=0 tx=0).stop 2024-04-21T08:38:52.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.603+0000 7f238fa0b740 1 -- 172.21.15.86:0/2576165737 shutdown_connections 2024-04-21T08:38:52.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.603+0000 7f238fa0b740 1 --2- 172.21.15.86:0/2576165737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652562cde90 0x5652562d0280 unknown :-1 s=CLOSED pgs=553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.603+0000 7f238fa0b740 1 --2- 172.21.15.86:0/2576165737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652562c0bd0 0x5652562cd950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.603+0000 7f238fa0b740 1 --2- 172.21.15.86:0/2576165737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5652562c02b0 0x5652562c0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.603+0000 7f238fa0b740 1 -- 172.21.15.86:0/2576165737 >> 172.21.15.86:0/2576165737 conn(0x5652562bacb0 msgr2=0x5652562bb090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:52.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.603+0000 7f238fa0b740 1 -- 172.21.15.86:0/2576165737 shutdown_connections 2024-04-21T08:38:52.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.603+0000 7f238fa0b740 1 -- 172.21.15.86:0/2576165737 wait complete. 2024-04-21T08:38:52.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.603+0000 7f238fa0b740 1 -- start start 2024-04-21T08:38:52.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.604+0000 7f238fa0b740 1 Processor -- start 2024-04-21T08:38:52.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.604+0000 7f238fa0b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652562c02b0 0x5652562caf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.604+0000 7f238fa0b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652562c0bd0 0x5652562cb4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.604+0000 7f238fa0b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5652562cde90 0x5652562cba00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.604+0000 7f238fa0b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5652562d0bf0 con 0x5652562c0bd0 2024-04-21T08:38:52.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.604+0000 7f238fa0b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5652561de2f0 con 0x5652562cde90 2024-04-21T08:38:52.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.604+0000 7f238fa0b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5652562d0d70 con 0x5652562c02b0 2024-04-21T08:38:52.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.604+0000 7f238659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652562c0bd0 0x5652562cb4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.604+0000 7f238fa0b740 10 client.0 _ll_register_callbacks cb 0x5652561eb580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:52.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.604+0000 7f238659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652562c0bd0 0x5652562cb4c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36274/0 (socket says 172.21.15.86:36274) 2024-04-21T08:38:52.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.604+0000 7f238659c640 1 -- 172.21.15.86:0/429621938 learned_addr learned my addr 172.21.15.86:0/429621938 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:52.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.604+0000 7f2385d9b640 1 --2- 172.21.15.86:0/429621938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652562c02b0 0x5652562caf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.604+0000 7f2385d9b640 1 --2- 172.21.15.86:0/429621938 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5652562cde90 0x5652562cba00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.604+0000 7f238659c640 1 -- 172.21.15.86:0/429621938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652562c02b0 msgr2=0x5652562caf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:52.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.604+0000 7f238659c640 1 --2- 172.21.15.86:0/429621938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652562c02b0 0x5652562caf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.604+0000 7f238659c640 1 -- 172.21.15.86:0/429621938 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5652562cde90 msgr2=0x5652562cba00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:52.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.604+0000 7f238659c640 1 --2- 172.21.15.86:0/429621938 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5652562cde90 0x5652562cba00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.604+0000 7f238659c640 1 -- 172.21.15.86:0/429621938 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565255f373f0 con 0x5652562c0bd0 2024-04-21T08:38:52.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.604+0000 7f2385d9b640 1 --2- 172.21.15.86:0/429621938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652562c02b0 0x5652562caf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:52.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.604+0000 7f238659c640 1 --2- 172.21.15.86:0/429621938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652562c0bd0 0x5652562cb4c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f237c003b70 tx=0x7f237c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1a0f553f6144dcc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:52.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.604+0000 7f236dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:38:52.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.605+0000 7f236dffb640 1 -- 172.21.15.86:0/429621938 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f237c003e60 con 0x5652562c0bd0 2024-04-21T08:38:52.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.605+0000 7f238fa0b740 1 -- 172.21.15.86:0/429621938 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f237802c9e0 con 0x5652562c0bd0 2024-04-21T08:38:52.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.605+0000 7f236dffb640 1 -- 172.21.15.86:0/429621938 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f237c005020 con 0x5652562c0bd0 2024-04-21T08:38:52.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.605+0000 7f236dffb640 1 -- 172.21.15.86:0/429621938 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f237c024050 con 0x5652562c0bd0 2024-04-21T08:38:52.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.605+0000 7f238fa0b740 10 client.15789 fetch_fsmap learned FSMap version 29 2024-04-21T08:38:52.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.605+0000 7f238fa0b740 1 -- 172.21.15.86:0/429621938 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x565255ef70f0 con 0x5652562c0bd0 2024-04-21T08:38:52.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.605+0000 7f236dffb640 1 -- 172.21.15.86:0/429621938 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f237802c9e0 con 0x5652562c0bd0 2024-04-21T08:38:52.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.605+0000 7f236dffb640 1 -- 172.21.15.86:0/429621938 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f237c01b480 con 0x5652562c0bd0 2024-04-21T08:38:52.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.605+0000 7f238fa0b740 10 client.15789 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:38:52.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.605+0000 7f238fa0b740 10 client.15789 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:38:52.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.605+0000 7f238fa0b740 20 client.15789 populate_metadata read hostname 'smithi086' 2024-04-21T08:38:52.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.605+0000 7f238fa0b740 1 --2- 172.21.15.86:0/429621938 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5652562ddf50 0x5652562e0390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.605+0000 7f238fa0b740 4 client.15789 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:38:52.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.605+0000 7f238fa0b740 1 -- 172.21.15.86:0/429621938 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565256232c00 con 0x5652562ddf50 2024-04-21T08:38:52.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.606+0000 7f2385d9b640 1 --2- 172.21.15.86:0/429621938 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5652562ddf50 0x5652562e0390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.606+0000 7f2385d9b640 1 --2- 172.21.15.86:0/429621938 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5652562ddf50 0x5652562e0390 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c631416cfebb0c4b server_cookie=d8e30cd4e1362388 in_seq=0 out_seq=0 2024-04-21T08:38:52.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.606+0000 7f236dffb640 10 client.15789 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:38:52.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.607+0000 7f236dffb640 1 -- 172.21.15.86:0/429621938 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x565256232c00 con 0x5652562ddf50 2024-04-21T08:38:52.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.607+0000 7f236dffb640 10 client.15789 handle_command_reply: tid=0 2024-04-21T08:38:52.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.607+0000 7f238fa0b740 1 client.15789 shutdown 2024-04-21T08:38:52.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.607+0000 7f238fa0b740 1 -- 172.21.15.86:0/429621938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652562c0bd0 msgr2=0x5652562cb4c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:52.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.607+0000 7f238fa0b740 1 --2- 172.21.15.86:0/429621938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652562c0bd0 0x5652562cb4c0 secure :-1 s=READY pgs=554 cs=0 l=1 rev1=1 crypto rx=0x7f237c003b70 tx=0x7f237c0117b0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.607+0000 7f238fa0b740 1 -- 172.21.15.86:0/429621938 shutdown_connections 2024-04-21T08:38:52.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.607+0000 7f238fa0b740 1 --2- 172.21.15.86:0/429621938 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5652562ddf50 0x5652562e0390 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.607+0000 7f238fa0b740 1 --2- 172.21.15.86:0/429621938 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5652562cde90 0x5652562cba00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.607+0000 7f238fa0b740 1 --2- 172.21.15.86:0/429621938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652562c0bd0 0x5652562cb4c0 unknown :-1 s=CLOSED pgs=554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.607+0000 7f238fa0b740 1 --2- 172.21.15.86:0/429621938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652562c02b0 0x5652562caf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.607+0000 7f238fa0b740 1 -- 172.21.15.86:0/429621938 >> 172.21.15.86:0/429621938 conn(0x5652562bacb0 msgr2=0x5652562bca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:52.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.607+0000 7f236dffb640 0 client.15789 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:38:52.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.607+0000 7f238fa0b740 1 -- 172.21.15.86:0/429621938 shutdown_connections 2024-04-21T08:38:52.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.607+0000 7f238fa0b740 1 -- 172.21.15.86:0/429621938 wait complete. 2024-04-21T08:38:52.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.607+0000 7f238fa0b740 20 client.15789 trim_cache size 0 max 16384 2024-04-21T08:38:52.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:52 smithi086 ceph-mon[27939]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:38:52.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:52 smithi086 ceph-mon[27939]: Updating smithi088:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:38:52.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:52 smithi086 ceph-mon[27939]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:38:52.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:52 smithi086 ceph-mon[27939]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:38:52.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:52 smithi086 ceph-mon[27939]: Updating smithi086:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:38:52.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:52 smithi086 ceph-mon[27939]: Updating smithi088:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:38:52.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:52 smithi086 ceph-mon[27939]: Updating smithi179:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:38:52.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:52 smithi086 ceph-mon[27939]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-21T08:38:52.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:52 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2204742226' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:52.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:52 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/778735871' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:52.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:52 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/4222529660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:52.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.645+0000 7f238fa0b740 1 Processor -- start 2024-04-21T08:38:52.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.645+0000 7f238fa0b740 1 -- start start 2024-04-21T08:38:52.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.646+0000 7f238fa0b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652562c02b0 0x5652562d9af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.646+0000 7f238fa0b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652562c0bd0 0x5652562cb820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.646+0000 7f238fa0b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5652562cde90 0x5652562b9bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.646+0000 7f238fa0b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f23881ad0b0 con 0x5652562c02b0 2024-04-21T08:38:52.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.646+0000 7f238fa0b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f23881aebf0 con 0x5652562cde90 2024-04-21T08:38:52.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.646+0000 7f238fa0b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f23881ad780 con 0x5652562c0bd0 2024-04-21T08:38:52.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.646+0000 7f2385d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652562c02b0 0x5652562d9af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.646+0000 7f2385d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652562c02b0 0x5652562d9af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36278/0 (socket says 172.21.15.86:36278) 2024-04-21T08:38:52.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.646+0000 7f2385d9b640 1 -- 172.21.15.86:0/2804764151 learned_addr learned my addr 172.21.15.86:0/2804764151 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:52.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.646+0000 7f2385d9b640 1 -- 172.21.15.86:0/2804764151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652562c0bd0 msgr2=0x5652562cb820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:52.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.646+0000 7f238659c640 1 --2- 172.21.15.86:0/2804764151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652562c0bd0 0x5652562cb820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.646+0000 7f2385d9b640 1 --2- 172.21.15.86:0/2804764151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652562c0bd0 0x5652562cb820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.646+0000 7f2385d9b640 1 -- 172.21.15.86:0/2804764151 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5652562cde90 msgr2=0x5652562b9bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:52.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.646+0000 7f2385d9b640 1 --2- 172.21.15.86:0/2804764151 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5652562cde90 0x5652562b9bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.646+0000 7f2385d9b640 1 -- 172.21.15.86:0/2804764151 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5652562c12d0 con 0x5652562c02b0 2024-04-21T08:38:52.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.646+0000 7f238659c640 1 --2- 172.21.15.86:0/2804764151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652562c0bd0 0x5652562cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:52.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.646+0000 7f2385d9b640 1 --2- 172.21.15.86:0/2804764151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652562c02b0 0x5652562d9af0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2378047040 tx=0x7f2378005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7447986e3fb45af6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:52.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.646+0000 7f236ffff640 1 -- 172.21.15.86:0/2804764151 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2378058030 con 0x5652562c02b0 2024-04-21T08:38:52.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.646+0000 7f236ffff640 1 -- 172.21.15.86:0/2804764151 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23780054d0 con 0x5652562c02b0 2024-04-21T08:38:52.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.646+0000 7f236ffff640 1 -- 172.21.15.86:0/2804764151 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2378049040 con 0x5652562c02b0 2024-04-21T08:38:52.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.647+0000 7f238fa0b740 1 -- 172.21.15.86:0/2804764151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652562c02b0 msgr2=0x5652562d9af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:52.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.647+0000 7f238fa0b740 1 --2- 172.21.15.86:0/2804764151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652562c02b0 0x5652562d9af0 secure :-1 s=READY pgs=555 cs=0 l=1 rev1=1 crypto rx=0x7f2378047040 tx=0x7f2378005b50 comp rx=0 tx=0).stop 2024-04-21T08:38:52.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.647+0000 7f238fa0b740 1 -- 172.21.15.86:0/2804764151 shutdown_connections 2024-04-21T08:38:52.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.647+0000 7f238fa0b740 1 --2- 172.21.15.86:0/2804764151 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5652562cde90 0x5652562b9bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.647+0000 7f238fa0b740 1 --2- 172.21.15.86:0/2804764151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652562c0bd0 0x5652562cb820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.647+0000 7f238fa0b740 1 --2- 172.21.15.86:0/2804764151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652562c02b0 0x5652562d9af0 unknown :-1 s=CLOSED pgs=555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.647+0000 7f238fa0b740 1 -- 172.21.15.86:0/2804764151 >> 172.21.15.86:0/2804764151 conn(0x5652562bacb0 msgr2=0x5652562b94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:52.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.647+0000 7f238fa0b740 1 -- 172.21.15.86:0/2804764151 shutdown_connections 2024-04-21T08:38:52.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.647+0000 7f238fa0b740 1 -- 172.21.15.86:0/2804764151 wait complete. 2024-04-21T08:38:52.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.647+0000 7f238fa0b740 1 -- start start 2024-04-21T08:38:52.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.648+0000 7f238fa0b740 1 Processor -- start 2024-04-21T08:38:52.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.648+0000 7f238fa0b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652562c02b0 0x5652562d6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.648+0000 7f238fa0b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5652562c0bd0 0x5652562c77f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.648+0000 7f238fa0b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652562cde90 0x5652562c7d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.648+0000 7f238fa0b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f23881ae520 con 0x5652562cde90 2024-04-21T08:38:52.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.648+0000 7f238fa0b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f23881ade50 con 0x5652562c0bd0 2024-04-21T08:38:52.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.648+0000 7f238fa0b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f23881ab070 con 0x5652562c02b0 2024-04-21T08:38:52.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.648+0000 7f238fa0b740 10 client.0 _ll_register_callbacks cb 0x7f237c004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:38:52.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.648+0000 7f2385d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652562cde90 0x5652562c7d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.648+0000 7f2385d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652562cde90 0x5652562c7d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36288/0 (socket says 172.21.15.86:36288) 2024-04-21T08:38:52.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.648+0000 7f2385d9b640 1 -- 172.21.15.86:0/1949375432 learned_addr learned my addr 172.21.15.86:0/1949375432 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:52.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.648+0000 7f238659c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5652562c0bd0 0x5652562c77f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.648+0000 7f2385d9b640 1 -- 172.21.15.86:0/1949375432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652562c02b0 msgr2=0x5652562d6260 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:38:52.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.648+0000 7f2385d9b640 1 --2- 172.21.15.86:0/1949375432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652562c02b0 0x5652562d6260 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.648+0000 7f2385d9b640 1 -- 172.21.15.86:0/1949375432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5652562c0bd0 msgr2=0x5652562c77f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:38:52.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.648+0000 7f2385d9b640 1 --2- 172.21.15.86:0/1949375432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5652562c0bd0 0x5652562c77f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.648+0000 7f2385d9b640 1 -- 172.21.15.86:0/1949375432 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5652562c83a0 con 0x5652562cde90 2024-04-21T08:38:52.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.648+0000 7f238659c640 1 --2- 172.21.15.86:0/1949375432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5652562c0bd0 0x5652562c77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:38:52.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.648+0000 7f2385d9b640 1 --2- 172.21.15.86:0/1949375432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652562cde90 0x5652562c7d30 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f237802bc70 tx=0x7f237802c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33fe2d5918ae5f5b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:52.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.648+0000 7f236dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:38:52.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.649+0000 7f236dffb640 1 -- 172.21.15.86:0/1949375432 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2378058070 con 0x5652562cde90 2024-04-21T08:38:52.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.649+0000 7f236dffb640 1 -- 172.21.15.86:0/1949375432 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f237802cbd0 con 0x5652562cde90 2024-04-21T08:38:52.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.649+0000 7f238fa0b740 1 -- 172.21.15.86:0/1949375432 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f23780054d0 con 0x5652562cde90 2024-04-21T08:38:52.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.649+0000 7f236dffb640 1 -- 172.21.15.86:0/1949375432 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2378049070 con 0x5652562cde90 2024-04-21T08:38:52.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.649+0000 7f238fa0b740 10 client.15795 fetch_fsmap learned FSMap version 29 2024-04-21T08:38:52.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.649+0000 7f238fa0b740 1 -- 172.21.15.86:0/1949375432 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5652563442e0 con 0x5652562cde90 2024-04-21T08:38:52.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.649+0000 7f236dffb640 1 -- 172.21.15.86:0/1949375432 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23780054d0 con 0x5652562cde90 2024-04-21T08:38:52.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.649+0000 7f236dffb640 1 -- 172.21.15.86:0/1949375432 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f237804f040 con 0x5652562cde90 2024-04-21T08:38:52.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.649+0000 7f238fa0b740 10 client.15795 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:38:52.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.649+0000 7f238fa0b740 10 client.15795 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:38:52.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.649+0000 7f238fa0b740 20 client.15795 populate_metadata read hostname 'smithi086' 2024-04-21T08:38:52.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.649+0000 7f238fa0b740 1 --2- 172.21.15.86:0/1949375432 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x565256344560 0x565256346950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.649+0000 7f238fa0b740 4 client.15795 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:38:52.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.649+0000 7f238fa0b740 1 -- 172.21.15.86:0/1949375432 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56525622ca70 con 0x565256344560 2024-04-21T08:38:52.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.649+0000 7f238659c640 1 --2- 172.21.15.86:0/1949375432 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x565256344560 0x565256346950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.650+0000 7f236dffb640 10 client.15795 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:38:52.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.650+0000 7f238659c640 1 --2- 172.21.15.86:0/1949375432 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x565256344560 0x565256346950 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=40a9e70aa2587176 server_cookie=78cfdfe84bac76e0 in_seq=0 out_seq=0 2024-04-21T08:38:52.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.650+0000 7f236dffb640 1 -- 172.21.15.86:0/1949375432 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x56525622ca70 con 0x565256344560 2024-04-21T08:38:52.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.650+0000 7f236dffb640 10 client.15795 handle_command_reply: tid=0 2024-04-21T08:38:52.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.650+0000 7f238fa0b740 1 client.15795 shutdown 2024-04-21T08:38:52.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.650+0000 7f238fa0b740 1 -- 172.21.15.86:0/1949375432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652562cde90 msgr2=0x5652562c7d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:52.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.650+0000 7f238fa0b740 1 --2- 172.21.15.86:0/1949375432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652562cde90 0x5652562c7d30 secure :-1 s=READY pgs=556 cs=0 l=1 rev1=1 crypto rx=0x7f237802bc70 tx=0x7f237802c5d0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.650+0000 7f238fa0b740 1 -- 172.21.15.86:0/1949375432 shutdown_connections 2024-04-21T08:38:52.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.650+0000 7f238fa0b740 1 --2- 172.21.15.86:0/1949375432 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x565256344560 0x565256346950 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.650+0000 7f238fa0b740 1 --2- 172.21.15.86:0/1949375432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5652562cde90 0x5652562c7d30 unknown :-1 s=CLOSED pgs=556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.650+0000 7f238fa0b740 1 --2- 172.21.15.86:0/1949375432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5652562c0bd0 0x5652562c77f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.650+0000 7f236dffb640 0 client.15795 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:38:52.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.650+0000 7f238fa0b740 1 --2- 172.21.15.86:0/1949375432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5652562c02b0 0x5652562d6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.650+0000 7f238fa0b740 1 -- 172.21.15.86:0/1949375432 >> 172.21.15.86:0/1949375432 conn(0x5652562bacb0 msgr2=0x5652562b94e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:52.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.650+0000 7f238fa0b740 1 -- 172.21.15.86:0/1949375432 shutdown_connections 2024-04-21T08:38:52.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.650+0000 7f238fa0b740 1 -- 172.21.15.86:0/1949375432 wait complete. 2024-04-21T08:38:52.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.650+0000 7f238fa0b740 20 client.15795 trim_cache size 0 max 16384 2024-04-21T08:38:52.659 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:38:52.659 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:38:52.659 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-21T08:38:52.660 INFO:teuthology.orchestra.run.smithi086.stdout: "a22b346e-63e5-4ab5-b852-c599e1b8c333": { 2024-04-21T08:38:52.660 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-21T08:38:52.660 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "a22b346e-63e5-4ab5-b852-c599e1b8c333", 2024-04-21T08:38:52.660 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-21T08:38:52.660 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:38:52.660 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:38:52.660 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:38:52.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.651+0000 7f238ce21640 1 -- 172.21.15.86:0/1459467481 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f23700a3db0 msgr2=0x7f23700a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:52.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.651+0000 7f238ce21640 1 --2- 172.21.15.86:0/1459467481 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f23700a3db0 0x7f23700a6270 secure :-1 s=READY pgs=52 cs=0 l=1 rev1=1 crypto rx=0x7f23800ce240 tx=0x7f23800bd040 comp rx=0 tx=0).stop 2024-04-21T08:38:52.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.651+0000 7f238ce21640 1 -- 172.21.15.86:0/1459467481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f238819cf30 msgr2=0x7f238819d3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:52.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.651+0000 7f238ce21640 1 --2- 172.21.15.86:0/1459467481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f238819cf30 0x7f238819d3e0 secure :-1 s=READY pgs=552 cs=0 l=1 rev1=1 crypto rx=0x7f238000eaa0 tx=0x7f238000ef60 comp rx=0 tx=0).stop 2024-04-21T08:38:52.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.651+0000 7f238ce21640 1 -- 172.21.15.86:0/1459467481 shutdown_connections 2024-04-21T08:38:52.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.651+0000 7f238ce21640 1 --2- 172.21.15.86:0/1459467481 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f23700a3db0 0x7f23700a6270 unknown :-1 s=CLOSED pgs=52 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.651+0000 7f238ce21640 1 --2- 172.21.15.86:0/1459467481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f238819cf30 0x7f238819d3e0 unknown :-1 s=CLOSED pgs=552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.651+0000 7f238ce21640 1 --2- 172.21.15.86:0/1459467481 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2388106150 0x7f2388198bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.651+0000 7f238ce21640 1 --2- 172.21.15.86:0/1459467481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2388101080 0x7f2388198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.651+0000 7f238ce21640 1 -- 172.21.15.86:0/1459467481 >> 172.21.15.86:0/1459467481 conn(0x7f2388078000 msgr2=0x7f23880fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:52.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.651+0000 7f238ce21640 1 -- 172.21.15.86:0/1459467481 shutdown_connections 2024-04-21T08:38:52.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.651+0000 7f238ce21640 1 -- 172.21.15.86:0/1459467481 wait complete. 2024-04-21T08:38:52.666 INFO:tasks.cephfs.filesystem:scrub status for tag:a22b346e-63e5-4ab5-b852-c599e1b8c333 - {'path': '/', 'tag': 'a22b346e-63e5-4ab5-b852-c599e1b8c333', 'options': 'recursive,force'} 2024-04-21T08:38:52.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.698+0000 7fdaeb24e640 1 -- 172.21.15.86:0/834888398 --> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] -- mgr_command(tid 0: {"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}) -- 0x7fdae4101aa0 con 0x7fdad40a3e60 2024-04-21T08:38:52.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:52 smithi088 ceph-mon[31259]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:38:52.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:52 smithi088 ceph-mon[31259]: Updating smithi088:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:38:52.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:52 smithi088 ceph-mon[31259]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:38:52.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:52 smithi088 ceph-mon[31259]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:38:52.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:52 smithi088 ceph-mon[31259]: Updating smithi086:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:38:52.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:52 smithi088 ceph-mon[31259]: Updating smithi088:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:38:52.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:52 smithi088 ceph-mon[31259]: Updating smithi179:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:38:52.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:52 smithi088 ceph-mon[31259]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-21T08:38:52.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:52 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2204742226' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:52.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:52 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/778735871' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:52.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:52 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/4222529660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:52.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:52 smithi179 ceph-mon[31829]: Updating smithi086:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:38:52.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:52 smithi179 ceph-mon[31829]: Updating smithi088:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:38:52.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:52 smithi179 ceph-mon[31829]: Updating smithi179:/etc/ceph/ceph.client.admin.keyring 2024-04-21T08:38:52.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:52 smithi179 ceph-mon[31829]: pgmap v5: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:38:52.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:52 smithi179 ceph-mon[31829]: Updating smithi086:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:38:52.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:52 smithi179 ceph-mon[31829]: Updating smithi088:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:38:52.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:52 smithi179 ceph-mon[31829]: Updating smithi179:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/config/ceph.client.admin.keyring 2024-04-21T08:38:52.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:52 smithi179 ceph-mon[31829]: mgrmap e24: x(active, since 4s), standbys: z, y 2024-04-21T08:38:52.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:52 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2204742226' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:52.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:52 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/778735871' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:52.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:52 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/4222529660' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:52.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.762+0000 7fdae1ffb640 1 -- 172.21.15.86:0/834888398 <== mgr.15558 v2:172.21.15.86:6800/2452032403 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+24 (secure 0 0 0) 0x7fdae4101aa0 con 0x7fdad40a3e60 2024-04-21T08:38:52.763 INFO:teuthology.orchestra.run.smithi086.stdout:Schedule set for path / 2024-04-21T08:38:52.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.764+0000 7fdaeb24e640 1 -- 172.21.15.86:0/834888398 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdad40a3e60 msgr2=0x7fdad40a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:52.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.764+0000 7fdaeb24e640 1 --2- 172.21.15.86:0/834888398 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdad40a3e60 0x7fdad40a6320 secure :-1 s=READY pgs=51 cs=0 l=1 rev1=1 crypto rx=0x7fdad8009bd0 tx=0x7fdad8005f90 comp rx=0 tx=0).stop 2024-04-21T08:38:52.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.764+0000 7fdaeb24e640 1 -- 172.21.15.86:0/834888398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdae419ad30 msgr2=0x7fdae419b1e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:52.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.764+0000 7fdaeb24e640 1 --2- 172.21.15.86:0/834888398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdae419ad30 0x7fdae419b1e0 secure :-1 s=READY pgs=546 cs=0 l=1 rev1=1 crypto rx=0x7fdadc02e330 tx=0x7fdadc003ee0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.764+0000 7fdaeb24e640 1 -- 172.21.15.86:0/834888398 shutdown_connections 2024-04-21T08:38:52.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.764+0000 7fdaeb24e640 1 --2- 172.21.15.86:0/834888398 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdad40a3e60 0x7fdad40a6320 unknown :-1 s=CLOSED pgs=51 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.764+0000 7fdaeb24e640 1 --2- 172.21.15.86:0/834888398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdae419ad30 0x7fdae419b1e0 unknown :-1 s=CLOSED pgs=546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.764+0000 7fdaeb24e640 1 --2- 172.21.15.86:0/834888398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdae4106170 0x7fdae41969b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.764+0000 7fdaeb24e640 1 --2- 172.21.15.86:0/834888398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdae41010f0 0x7fdae4196470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.764+0000 7fdaeb24e640 1 -- 172.21.15.86:0/834888398 >> 172.21.15.86:0/834888398 conn(0x7fdae4077fd0 msgr2=0x7fdae40feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:52.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.764+0000 7fdaeb24e640 1 -- 172.21.15.86:0/834888398 shutdown_connections 2024-04-21T08:38:52.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.764+0000 7fdaeb24e640 1 -- 172.21.15.86:0/834888398 wait complete. 2024-04-21T08:38:52.775 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule retention add --fs=cephfs --path=/ --retention-spec-or-period=6m3h' 2024-04-21T08:38:52.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.919+0000 7f06b95b2640 1 -- 172.21.15.86:0/3076855665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06b4106170 msgr2=0x7f06b410dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:52.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.919+0000 7f06b95b2640 1 --2- 172.21.15.86:0/3076855665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06b4106170 0x7f06b410dd40 secure :-1 s=READY pgs=557 cs=0 l=1 rev1=1 crypto rx=0x7f06ac009ca0 tx=0x7f06ac02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.919+0000 7f06b95b2640 1 -- 172.21.15.86:0/3076855665 shutdown_connections 2024-04-21T08:38:52.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.919+0000 7f06b95b2640 1 --2- 172.21.15.86:0/3076855665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06b4106170 0x7f06b410dd40 unknown :-1 s=CLOSED pgs=557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.919+0000 7f06b95b2640 1 --2- 172.21.15.86:0/3076855665 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06b4101aa0 0x7f06b4105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.919+0000 7f06b95b2640 1 --2- 172.21.15.86:0/3076855665 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06b41010f0 0x7f06b41014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.919+0000 7f06b95b2640 1 -- 172.21.15.86:0/3076855665 >> 172.21.15.86:0/3076855665 conn(0x7f06b4077fd0 msgr2=0x7f06b40783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:52.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.919+0000 7f06b95b2640 1 -- 172.21.15.86:0/3076855665 shutdown_connections 2024-04-21T08:38:52.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.919+0000 7f06b95b2640 1 -- 172.21.15.86:0/3076855665 wait complete. 2024-04-21T08:38:52.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.920+0000 7f06b95b2640 1 Processor -- start 2024-04-21T08:38:52.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.920+0000 7f06b95b2640 1 -- start start 2024-04-21T08:38:52.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.920+0000 7f06b95b2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06b41010f0 0x7f06b41a0640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.920+0000 7f06b95b2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06b4106170 0x7f06b41a0b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.920+0000 7f06b95b2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06b419a700 0x7f06b419ab80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.920+0000 7f06b95b2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f06b41110e0 con 0x7f06b419a700 2024-04-21T08:38:52.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.920+0000 7f06b95b2640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f06b4110f60 con 0x7f06b4106170 2024-04-21T08:38:52.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.920+0000 7f06b95b2640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f06b4111260 con 0x7f06b41010f0 2024-04-21T08:38:52.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.920+0000 7f06b37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06b419a700 0x7f06b419ab80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.920+0000 7f06b37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06b419a700 0x7f06b419ab80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36304/0 (socket says 172.21.15.86:36304) 2024-04-21T08:38:52.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.920+0000 7f06b37fe640 1 -- 172.21.15.86:0/3888450713 learned_addr learned my addr 172.21.15.86:0/3888450713 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:52.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.920+0000 7f06b2ffd640 1 --2- 172.21.15.86:0/3888450713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06b41010f0 0x7f06b41a0640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.920+0000 7f06abfff640 1 --2- 172.21.15.86:0/3888450713 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06b4106170 0x7f06b41a0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.920+0000 7f06b37fe640 1 -- 172.21.15.86:0/3888450713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06b41010f0 msgr2=0x7f06b41a0640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:52.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.920+0000 7f06b37fe640 1 --2- 172.21.15.86:0/3888450713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06b41010f0 0x7f06b41a0640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.920+0000 7f06b37fe640 1 -- 172.21.15.86:0/3888450713 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06b4106170 msgr2=0x7f06b41a0b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:52.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.920+0000 7f06b37fe640 1 --2- 172.21.15.86:0/3888450713 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06b4106170 0x7f06b41a0b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:52.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.920+0000 7f06b37fe640 1 -- 172.21.15.86:0/3888450713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06b419b300 con 0x7f06b419a700 2024-04-21T08:38:52.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.921+0000 7f06b37fe640 1 --2- 172.21.15.86:0/3888450713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06b419a700 0x7f06b419ab80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f06ac005e30 tx=0x7f06ac034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:52.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.921+0000 7f06b0ff9640 1 -- 172.21.15.86:0/3888450713 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ac039040 con 0x7f06b419a700 2024-04-21T08:38:52.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.921+0000 7f06b0ff9640 1 -- 172.21.15.86:0/3888450713 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06ac044070 con 0x7f06b419a700 2024-04-21T08:38:52.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.921+0000 7f06b95b2640 1 -- 172.21.15.86:0/3888450713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06b419b590 con 0x7f06b419a700 2024-04-21T08:38:52.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.921+0000 7f06b0ff9640 1 -- 172.21.15.86:0/3888450713 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06ac003e60 con 0x7f06b419a700 2024-04-21T08:38:52.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.921+0000 7f06b95b2640 1 -- 172.21.15.86:0/3888450713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06b41a76d0 con 0x7f06b419a700 2024-04-21T08:38:52.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.922+0000 7f06b95b2640 1 -- 172.21.15.86:0/3888450713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f06b4102570 con 0x7f06b419a700 2024-04-21T08:38:52.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.923+0000 7f06b0ff9640 1 -- 172.21.15.86:0/3888450713 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f06ac0324c0 con 0x7f06b419a700 2024-04-21T08:38:52.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.924+0000 7f06b0ff9640 1 --2- 172.21.15.86:0/3888450713 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f06a00a3e60 0x7f06a00a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:52.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.924+0000 7f06b2ffd640 1 --2- 172.21.15.86:0/3888450713 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f06a00a3e60 0x7f06a00a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:52.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.924+0000 7f06b0ff9640 1 -- 172.21.15.86:0/3888450713 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f06ac0f3860 con 0x7f06b419a700 2024-04-21T08:38:52.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.924+0000 7f06b2ffd640 1 --2- 172.21.15.86:0/3888450713 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f06a00a3e60 0x7f06a00a6320 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f06a400bbe0 tx=0x7f06a400b390 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:52.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:52.927+0000 7f06b0ff9640 1 -- 172.21.15.86:0/3888450713 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f06ac0bfb30 con 0x7f06b419a700 2024-04-21T08:38:53.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.127+0000 7f06b95b2640 1 -- 172.21.15.86:0/3888450713 --> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] -- mgr_command(tid 0: {"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}) -- 0x7f06b4101aa0 con 0x7f06a00a3e60 2024-04-21T08:38:53.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.191+0000 7f06b0ff9640 1 -- 172.21.15.86:0/3888450713 <== mgr.15558 v2:172.21.15.86:6800/2452032403 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+26 (secure 0 0 0) 0x7f06b4101aa0 con 0x7f06a00a3e60 2024-04-21T08:38:53.191 INFO:teuthology.orchestra.run.smithi086.stdout:Retention added to path / 2024-04-21T08:38:53.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.193+0000 7f06b95b2640 1 -- 172.21.15.86:0/3888450713 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f06a00a3e60 msgr2=0x7f06a00a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:53.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.193+0000 7f06b95b2640 1 --2- 172.21.15.86:0/3888450713 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f06a00a3e60 0x7f06a00a6320 secure :-1 s=READY pgs=53 cs=0 l=1 rev1=1 crypto rx=0x7f06a400bbe0 tx=0x7f06a400b390 comp rx=0 tx=0).stop 2024-04-21T08:38:53.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.193+0000 7f06b95b2640 1 -- 172.21.15.86:0/3888450713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06b419a700 msgr2=0x7f06b419ab80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:53.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.193+0000 7f06b95b2640 1 --2- 172.21.15.86:0/3888450713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06b419a700 0x7f06b419ab80 secure :-1 s=READY pgs=558 cs=0 l=1 rev1=1 crypto rx=0x7f06ac005e30 tx=0x7f06ac034760 comp rx=0 tx=0).stop 2024-04-21T08:38:53.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.193+0000 7f06b95b2640 1 -- 172.21.15.86:0/3888450713 shutdown_connections 2024-04-21T08:38:53.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.193+0000 7f06b95b2640 1 --2- 172.21.15.86:0/3888450713 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f06a00a3e60 0x7f06a00a6320 unknown :-1 s=CLOSED pgs=53 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.193+0000 7f06b95b2640 1 --2- 172.21.15.86:0/3888450713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06b419a700 0x7f06b419ab80 unknown :-1 s=CLOSED pgs=558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.193+0000 7f06b95b2640 1 --2- 172.21.15.86:0/3888450713 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06b4106170 0x7f06b41a0b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.193+0000 7f06b95b2640 1 --2- 172.21.15.86:0/3888450713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06b41010f0 0x7f06b41a0640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.193+0000 7f06b95b2640 1 -- 172.21.15.86:0/3888450713 >> 172.21.15.86:0/3888450713 conn(0x7f06b4077fd0 msgr2=0x7f06b410cdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:53.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.193+0000 7f06b95b2640 1 -- 172.21.15.86:0/3888450713 shutdown_connections 2024-04-21T08:38:53.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.193+0000 7f06b95b2640 1 -- 172.21.15.86:0/3888450713 wait complete. 2024-04-21T08:38:53.204 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-21T08:38:53.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.302+0000 7ff8f9924640 1 -- 172.21.15.86:0/214210534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8f41021d0 msgr2=0x7ff8f41025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:53.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.302+0000 7ff8f9924640 1 --2- 172.21.15.86:0/214210534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8f41021d0 0x7ff8f41025b0 secure :-1 s=READY pgs=559 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4009900 tx=0x7ff8e402bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:53.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.303+0000 7ff8f9924640 1 -- 172.21.15.86:0/214210534 shutdown_connections 2024-04-21T08:38:53.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.303+0000 7ff8f9924640 1 --2- 172.21.15.86:0/214210534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8f41072a0 0x7ff8f410ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.303+0000 7ff8f9924640 1 --2- 172.21.15.86:0/214210534 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8f4102b80 0x7ff8f4106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.303+0000 7ff8f9924640 1 --2- 172.21.15.86:0/214210534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8f41021d0 0x7ff8f41025b0 unknown :-1 s=CLOSED pgs=559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.303+0000 7ff8f9924640 1 -- 172.21.15.86:0/214210534 >> 172.21.15.86:0/214210534 conn(0x7ff8f40793e0 msgr2=0x7ff8f40797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:53.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.303+0000 7ff8f9924640 1 -- 172.21.15.86:0/214210534 shutdown_connections 2024-04-21T08:38:53.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.303+0000 7ff8f9924640 1 -- 172.21.15.86:0/214210534 wait complete. 2024-04-21T08:38:53.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.303+0000 7ff8f9924640 1 Processor -- start 2024-04-21T08:38:53.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.303+0000 7ff8f9924640 1 -- start start 2024-04-21T08:38:53.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.303+0000 7ff8f9924640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8f41021d0 0x7ff8f4074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:53.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.303+0000 7ff8f9924640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8f41072a0 0x7ff8f4074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:53.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.303+0000 7ff8f9924640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8f4074cd0 0x7ff8f41a6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:53.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.303+0000 7ff8f9924640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f4112260 con 0x7ff8f4074cd0 2024-04-21T08:38:53.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.303+0000 7ff8f9924640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f41120e0 con 0x7ff8f41072a0 2024-04-21T08:38:53.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.303+0000 7ff8f9924640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff8f41123e0 con 0x7ff8f41021d0 2024-04-21T08:38:53.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.303+0000 7ff8f37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8f4074cd0 0x7ff8f41a6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:53.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.304+0000 7ff8f37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8f4074cd0 0x7ff8f41a6850 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36326/0 (socket says 172.21.15.86:36326) 2024-04-21T08:38:53.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.304+0000 7ff8f37fe640 1 -- 172.21.15.86:0/3876282721 learned_addr learned my addr 172.21.15.86:0/3876282721 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:53.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.304+0000 7ff8ebfff640 1 --2- 172.21.15.86:0/3876282721 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8f41072a0 0x7ff8f4074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:53.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.304+0000 7ff8f2ffd640 1 --2- 172.21.15.86:0/3876282721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8f41021d0 0x7ff8f4074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:53.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.304+0000 7ff8f37fe640 1 -- 172.21.15.86:0/3876282721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8f41021d0 msgr2=0x7ff8f4074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:53.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.304+0000 7ff8f37fe640 1 --2- 172.21.15.86:0/3876282721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8f41021d0 0x7ff8f4074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.304+0000 7ff8f37fe640 1 -- 172.21.15.86:0/3876282721 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8f41072a0 msgr2=0x7ff8f4074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:53.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.304+0000 7ff8f37fe640 1 --2- 172.21.15.86:0/3876282721 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8f41072a0 0x7ff8f4074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.304+0000 7ff8f37fe640 1 -- 172.21.15.86:0/3876282721 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff8f41a6f50 con 0x7ff8f4074cd0 2024-04-21T08:38:53.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.304+0000 7ff8ebfff640 1 --2- 172.21.15.86:0/3876282721 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8f41072a0 0x7ff8f4074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:53.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.304+0000 7ff8f2ffd640 1 --2- 172.21.15.86:0/3876282721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8f41021d0 0x7ff8f4074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:53.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.304+0000 7ff8f37fe640 1 --2- 172.21.15.86:0/3876282721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8f4074cd0 0x7ff8f41a6850 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec00b9e0 tx=0x7ff8ec00bea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:53.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.304+0000 7ff8f8922640 1 -- 172.21.15.86:0/3876282721 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec019070 con 0x7ff8f4074cd0 2024-04-21T08:38:53.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.304+0000 7ff8f8922640 1 -- 172.21.15.86:0/3876282721 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff8ec014440 con 0x7ff8f4074cd0 2024-04-21T08:38:53.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.304+0000 7ff8f9924640 1 -- 172.21.15.86:0/3876282721 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff8f41a7240 con 0x7ff8f4074cd0 2024-04-21T08:38:53.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.304+0000 7ff8f8922640 1 -- 172.21.15.86:0/3876282721 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff8ec0075d0 con 0x7ff8f4074cd0 2024-04-21T08:38:53.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.304+0000 7ff8f9924640 1 -- 172.21.15.86:0/3876282721 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff8f41a7740 con 0x7ff8f4074cd0 2024-04-21T08:38:53.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.305+0000 7ff8f9924640 1 -- 172.21.15.86:0/3876282721 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff8f404fce0 con 0x7ff8f4074cd0 2024-04-21T08:38:53.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.306+0000 7ff8f8922640 1 -- 172.21.15.86:0/3876282721 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7ff8ec01c070 con 0x7ff8f4074cd0 2024-04-21T08:38:53.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.307+0000 7ff8f8922640 1 --2- 172.21.15.86:0/3876282721 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff8e00a3e90 0x7ff8e00a6350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:53.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.307+0000 7ff8f2ffd640 1 --2- 172.21.15.86:0/3876282721 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff8e00a3e90 0x7ff8e00a6350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:53.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.307+0000 7ff8f8922640 1 -- 172.21.15.86:0/3876282721 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7ff8ec0d0160 con 0x7ff8f4074cd0 2024-04-21T08:38:53.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.307+0000 7ff8f2ffd640 1 --2- 172.21.15.86:0/3876282721 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff8e00a3e90 0x7ff8e00a6350 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4036040 tx=0x7ff8e4000bc0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:53.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.311+0000 7ff8f8922640 1 -- 172.21.15.86:0/3876282721 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff8ec09c430 con 0x7ff8f4074cd0 2024-04-21T08:38:53.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.511+0000 7ff8f9924640 1 -- 172.21.15.86:0/3876282721 --> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7ff8f410a540 con 0x7ff8e00a3e90 2024-04-21T08:38:53.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.519+0000 7ff8f8922640 1 -- 172.21.15.86:0/3876282721 <== mgr.15558 v2:172.21.15.86:6800/2452032403 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+298 (secure 0 0 0) 0x7ff8e40042f0 con 0x7ff8e00a3e90 2024-04-21T08:38:53.519 INFO:teuthology.orchestra.run.smithi086.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-21T00:00:00", "created": "2024-04-21T08:38:52", "first": null, "last": null, "last_pruned": null, "created_count": 0, "pruned_count": 0, "active": true} 2024-04-21T08:38:53.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.521+0000 7ff8f9924640 1 -- 172.21.15.86:0/3876282721 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff8e00a3e90 msgr2=0x7ff8e00a6350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:53.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.521+0000 7ff8f9924640 1 --2- 172.21.15.86:0/3876282721 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff8e00a3e90 0x7ff8e00a6350 secure :-1 s=READY pgs=54 cs=0 l=1 rev1=1 crypto rx=0x7ff8e4036040 tx=0x7ff8e4000bc0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.521+0000 7ff8f9924640 1 -- 172.21.15.86:0/3876282721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8f4074cd0 msgr2=0x7ff8f41a6850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:53.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.521+0000 7ff8f9924640 1 --2- 172.21.15.86:0/3876282721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8f4074cd0 0x7ff8f41a6850 secure :-1 s=READY pgs=560 cs=0 l=1 rev1=1 crypto rx=0x7ff8ec00b9e0 tx=0x7ff8ec00bea0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.521+0000 7ff8f9924640 1 -- 172.21.15.86:0/3876282721 shutdown_connections 2024-04-21T08:38:53.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.521+0000 7ff8f9924640 1 --2- 172.21.15.86:0/3876282721 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff8e00a3e90 0x7ff8e00a6350 unknown :-1 s=CLOSED pgs=54 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.521+0000 7ff8f9924640 1 --2- 172.21.15.86:0/3876282721 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff8f4074cd0 0x7ff8f41a6850 unknown :-1 s=CLOSED pgs=560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.521+0000 7ff8f9924640 1 --2- 172.21.15.86:0/3876282721 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff8f41072a0 0x7ff8f4074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.521+0000 7ff8f9924640 1 --2- 172.21.15.86:0/3876282721 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff8f41021d0 0x7ff8f4074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.521+0000 7ff8f9924640 1 -- 172.21.15.86:0/3876282721 >> 172.21.15.86:0/3876282721 conn(0x7ff8f40793e0 msgr2=0x7ff8f40ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:53.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.521+0000 7ff8f9924640 1 -- 172.21.15.86:0/3876282721 shutdown_connections 2024-04-21T08:38:53.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.521+0000 7ff8f9924640 1 -- 172.21.15.86:0/3876282721 wait complete. 2024-04-21T08:38:53.531 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:53 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3315821057' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:53.531 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:53 smithi086 ceph-mon[27939]: scrub queued for path: / 2024-04-21T08:38:53.531 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:53 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:38:53.531 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:53 smithi086 ceph-mon[27939]: scrub summary: active paths [/] 2024-04-21T08:38:53.531 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:53 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:38:53.532 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-21T08:38:53.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.632+0000 7f1a0f72c640 1 -- 172.21.15.86:0/1363009008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a08103220 msgr2=0x7f1a08103600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:53.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.632+0000 7f1a0f72c640 1 --2- 172.21.15.86:0/1363009008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a08103220 0x7f1a08103600 secure :-1 s=READY pgs=561 cs=0 l=1 rev1=1 crypto rx=0x7f1a04009ca0 tx=0x7f1a0402c180 comp rx=0 tx=0).stop 2024-04-21T08:38:53.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.633+0000 7f1a0f72c640 1 -- 172.21.15.86:0/1363009008 shutdown_connections 2024-04-21T08:38:53.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.633+0000 7f1a0f72c640 1 --2- 172.21.15.86:0/1363009008 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a0810c560 0x7f1a0810e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.633+0000 7f1a0f72c640 1 --2- 172.21.15.86:0/1363009008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a08103b40 0x7f1a0810c020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.633+0000 7f1a0f72c640 1 --2- 172.21.15.86:0/1363009008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a08103220 0x7f1a08103600 unknown :-1 s=CLOSED pgs=561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.633+0000 7f1a0f72c640 1 -- 172.21.15.86:0/1363009008 >> 172.21.15.86:0/1363009008 conn(0x7f1a08077fd0 msgr2=0x7f1a080783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:53.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.633+0000 7f1a0f72c640 1 -- 172.21.15.86:0/1363009008 shutdown_connections 2024-04-21T08:38:53.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.633+0000 7f1a0f72c640 1 -- 172.21.15.86:0/1363009008 wait complete. 2024-04-21T08:38:53.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.633+0000 7f1a0f72c640 1 Processor -- start 2024-04-21T08:38:53.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.633+0000 7f1a0f72c640 1 -- start start 2024-04-21T08:38:53.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.633+0000 7f1a0f72c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a08103220 0x7f1a08198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:53.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.633+0000 7f1a0f72c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a08103b40 0x7f1a08198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:53.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.633+0000 7f1a0f72c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a0810c560 0x7f1a0819cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:53.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.633+0000 7f1a0f72c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1a081121b0 con 0x7f1a08103220 2024-04-21T08:38:53.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.633+0000 7f1a0f72c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1a08112030 con 0x7f1a0810c560 2024-04-21T08:38:53.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.633+0000 7f1a0f72c640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1a08112330 con 0x7f1a08103b40 2024-04-21T08:38:53.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.633+0000 7f1a0d4c8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a08103220 0x7f1a08198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:53.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.633+0000 7f1a0d4c8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a08103220 0x7f1a08198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36334/0 (socket says 172.21.15.86:36334) 2024-04-21T08:38:53.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.633+0000 7f1a0d4c8640 1 -- 172.21.15.86:0/286392671 learned_addr learned my addr 172.21.15.86:0/286392671 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:53.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.634+0000 7f1a0dcc9640 1 --2- 172.21.15.86:0/286392671 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a0810c560 0x7f1a0819cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:53.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.634+0000 7f1a0ccc7640 1 --2- 172.21.15.86:0/286392671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a08103b40 0x7f1a08198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:53.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.634+0000 7f1a0d4c8640 1 -- 172.21.15.86:0/286392671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a08103b40 msgr2=0x7f1a08198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:53.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.634+0000 7f1a0d4c8640 1 --2- 172.21.15.86:0/286392671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a08103b40 0x7f1a08198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.634+0000 7f1a0d4c8640 1 -- 172.21.15.86:0/286392671 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a0810c560 msgr2=0x7f1a0819cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:53.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.634+0000 7f1a0d4c8640 1 --2- 172.21.15.86:0/286392671 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a0810c560 0x7f1a0819cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.634+0000 7f1a0d4c8640 1 -- 172.21.15.86:0/286392671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a0819d6f0 con 0x7f1a08103220 2024-04-21T08:38:53.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.634+0000 7f1a0ccc7640 1 --2- 172.21.15.86:0/286392671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a08103b40 0x7f1a08198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:53.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.634+0000 7f1a0d4c8640 1 --2- 172.21.15.86:0/286392671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a08103220 0x7f1a08198750 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f1a04005e30 tx=0x7f1a040039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:53.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.634+0000 7f19fe7fc640 1 -- 172.21.15.86:0/286392671 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a04039040 con 0x7f1a08103220 2024-04-21T08:38:53.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.634+0000 7f19fe7fc640 1 -- 172.21.15.86:0/286392671 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a04044070 con 0x7f1a08103220 2024-04-21T08:38:53.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.634+0000 7f1a0f72c640 1 -- 172.21.15.86:0/286392671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a0819d980 con 0x7f1a08103220 2024-04-21T08:38:53.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.634+0000 7f19fe7fc640 1 -- 172.21.15.86:0/286392671 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a04032960 con 0x7f1a08103220 2024-04-21T08:38:53.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.634+0000 7f1a0f72c640 1 -- 172.21.15.86:0/286392671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a081a5270 con 0x7f1a08103220 2024-04-21T08:38:53.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.635+0000 7f1a0f72c640 1 -- 172.21.15.86:0/286392671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1a0804fce0 con 0x7f1a08103220 2024-04-21T08:38:53.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.636+0000 7f19fe7fc640 1 -- 172.21.15.86:0/286392671 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f1a04004580 con 0x7f1a08103220 2024-04-21T08:38:53.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.637+0000 7f19fe7fc640 1 --2- 172.21.15.86:0/286392671 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f19f40a3e60 0x7f19f40a6320 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:53.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.637+0000 7f1a0ccc7640 1 --2- 172.21.15.86:0/286392671 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f19f40a3e60 0x7f19f40a6320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:53.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.637+0000 7f19fe7fc640 1 -- 172.21.15.86:0/286392671 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f1a040f3c70 con 0x7f1a08103220 2024-04-21T08:38:53.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.637+0000 7f1a0ccc7640 1 --2- 172.21.15.86:0/286392671 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f19f40a3e60 0x7f19f40a6320 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f19f80058c0 tx=0x7f19f8005850 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:53.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.641+0000 7f19fe7fc640 1 -- 172.21.15.86:0/286392671 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1a040bff40 con 0x7f1a08103220 2024-04-21T08:38:53.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:53 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3315821057' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:53.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:53 smithi088 ceph-mon[31259]: scrub queued for path: / 2024-04-21T08:38:53.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:53 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:38:53.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:53 smithi088 ceph-mon[31259]: scrub summary: active paths [/] 2024-04-21T08:38:53.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:53 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:38:53.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:53 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3315821057' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:53.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:53 smithi179 ceph-mon[31829]: scrub queued for path: / 2024-04-21T08:38:53.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:53 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:38:53.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:53 smithi179 ceph-mon[31829]: scrub summary: active paths [/] 2024-04-21T08:38:53.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:53 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:38:53.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.840+0000 7f1a0f72c640 1 -- 172.21.15.86:0/286392671 --> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f1a08062390 con 0x7f19f40a3e60 2024-04-21T08:38:53.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.846+0000 7f19fe7fc640 1 -- 172.21.15.86:0/286392671 <== mgr.15558 v2:172.21.15.86:6800/2452032403 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f1a08062390 con 0x7f19f40a3e60 2024-04-21T08:38:53.847 INFO:teuthology.orchestra.run.smithi086.stdout:/ 1m 6m3h 2024-04-21T08:38:53.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.848+0000 7f1a0f72c640 1 -- 172.21.15.86:0/286392671 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f19f40a3e60 msgr2=0x7f19f40a6320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:53.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.848+0000 7f1a0f72c640 1 --2- 172.21.15.86:0/286392671 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f19f40a3e60 0x7f19f40a6320 secure :-1 s=READY pgs=55 cs=0 l=1 rev1=1 crypto rx=0x7f19f80058c0 tx=0x7f19f8005850 comp rx=0 tx=0).stop 2024-04-21T08:38:53.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.848+0000 7f1a0f72c640 1 -- 172.21.15.86:0/286392671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a08103220 msgr2=0x7f1a08198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:53.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.848+0000 7f1a0f72c640 1 --2- 172.21.15.86:0/286392671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a08103220 0x7f1a08198750 secure :-1 s=READY pgs=562 cs=0 l=1 rev1=1 crypto rx=0x7f1a04005e30 tx=0x7f1a040039f0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.848+0000 7f1a0f72c640 1 -- 172.21.15.86:0/286392671 shutdown_connections 2024-04-21T08:38:53.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.848+0000 7f1a0f72c640 1 --2- 172.21.15.86:0/286392671 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f19f40a3e60 0x7f19f40a6320 unknown :-1 s=CLOSED pgs=55 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.848+0000 7f1a0f72c640 1 --2- 172.21.15.86:0/286392671 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a0810c560 0x7f1a0819cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.848+0000 7f1a0f72c640 1 --2- 172.21.15.86:0/286392671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a08103b40 0x7f1a08198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.848+0000 7f1a0f72c640 1 --2- 172.21.15.86:0/286392671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a08103220 0x7f1a08198750 unknown :-1 s=CLOSED pgs=562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:53.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.848+0000 7f1a0f72c640 1 -- 172.21.15.86:0/286392671 >> 172.21.15.86:0/286392671 conn(0x7f1a08077fd0 msgr2=0x7f1a080fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:53.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.849+0000 7f1a0f72c640 1 -- 172.21.15.86:0/286392671 shutdown_connections 2024-04-21T08:38:53.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:53.849+0000 7f1a0f72c640 1 -- 172.21.15.86:0/286392671 wait complete. 2024-04-21T08:38:53.859 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > START_TIME' 2024-04-21T08:38:53.883 INFO:teuthology.run_tasks:Running task full_sequential_finally... 2024-04-21T08:38:53.902 INFO:teuthology.run_tasks:Running task background_exec... 2024-04-21T08:38:53.935 INFO:teuthology.task.background_exec:Running background command on role mon.a host ubuntu@smithi086.front.sepia.ceph.com 2024-04-21T08:38:53.935 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest daemon-helper kill --kill-group bash -c 'while sleep 13; do ceph tell mds.cephfs:0 flush journal; done' 2024-04-21T08:38:53.937 INFO:teuthology.run_tasks:Running task quiescer... 2024-04-21T08:38:53.951 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:38:54.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.048+0000 7f00745b2640 1 -- 172.21.15.86:0/3323248817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f006c102b40 msgr2=0x7f006c106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:54.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.048+0000 7f00745b2640 1 --2- 172.21.15.86:0/3323248817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f006c102b40 0x7f006c106b80 secure :-1 s=READY pgs=563 cs=0 l=1 rev1=1 crypto rx=0x7f005c009900 tx=0x7f005c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:54.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.048+0000 7f00745b2640 1 -- 172.21.15.86:0/3323248817 shutdown_connections 2024-04-21T08:38:54.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.048+0000 7f00745b2640 1 --2- 172.21.15.86:0/3323248817 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f006c107260 0x7f006c10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.048+0000 7f00745b2640 1 --2- 172.21.15.86:0/3323248817 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f006c102b40 0x7f006c106b80 unknown :-1 s=CLOSED pgs=563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.048+0000 7f00745b2640 1 --2- 172.21.15.86:0/3323248817 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f006c102190 0x7f006c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.048+0000 7f00745b2640 1 -- 172.21.15.86:0/3323248817 >> 172.21.15.86:0/3323248817 conn(0x7f006c0793e0 msgr2=0x7f006c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:54.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.048+0000 7f00745b2640 1 -- 172.21.15.86:0/3323248817 shutdown_connections 2024-04-21T08:38:54.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.048+0000 7f00745b2640 1 -- 172.21.15.86:0/3323248817 wait complete. 2024-04-21T08:38:54.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.048+0000 7f00745b2640 1 Processor -- start 2024-04-21T08:38:54.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.049+0000 7f00745b2640 1 -- start start 2024-04-21T08:38:54.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.049+0000 7f00745b2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f006c102190 0x7f006c074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:54.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.049+0000 7f00745b2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f006c107260 0x7f006c074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:54.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.049+0000 7f00745b2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f006c074cd0 0x7f006c1a6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:54.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.049+0000 7f00745b2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f006c1121a0 con 0x7f006c107260 2024-04-21T08:38:54.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.049+0000 7f00745b2640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f006c112020 con 0x7f006c102190 2024-04-21T08:38:54.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.049+0000 7f00745b2640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f006c112320 con 0x7f006c074cd0 2024-04-21T08:38:54.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.049+0000 7f0071b4d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f006c107260 0x7f006c074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:54.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.049+0000 7f0071b4d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f006c107260 0x7f006c074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36340/0 (socket says 172.21.15.86:36340) 2024-04-21T08:38:54.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.049+0000 7f0071b4d640 1 -- 172.21.15.86:0/3848053340 learned_addr learned my addr 172.21.15.86:0/3848053340 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:54.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.049+0000 7f0072b4f640 1 --2- 172.21.15.86:0/3848053340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f006c074cd0 0x7f006c1a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:54.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.049+0000 7f007234e640 1 --2- 172.21.15.86:0/3848053340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f006c102190 0x7f006c074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:54.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.049+0000 7f0071b4d640 1 -- 172.21.15.86:0/3848053340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f006c074cd0 msgr2=0x7f006c1a6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:54.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.049+0000 7f0071b4d640 1 --2- 172.21.15.86:0/3848053340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f006c074cd0 0x7f006c1a6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.049+0000 7f0071b4d640 1 -- 172.21.15.86:0/3848053340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f006c102190 msgr2=0x7f006c074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:54.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.049+0000 7f0071b4d640 1 --2- 172.21.15.86:0/3848053340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f006c102190 0x7f006c074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.049+0000 7f0071b4d640 1 -- 172.21.15.86:0/3848053340 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f006c1a7010 con 0x7f006c107260 2024-04-21T08:38:54.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.049+0000 7f007234e640 1 --2- 172.21.15.86:0/3848053340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f006c102190 0x7f006c074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:54.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.049+0000 7f0071b4d640 1 --2- 172.21.15.86:0/3848053340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f006c107260 0x7f006c074790 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f005c036040 tx=0x7f005c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:54.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.049+0000 7f00637fe640 1 -- 172.21.15.86:0/3848053340 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c043020 con 0x7f006c107260 2024-04-21T08:38:54.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.049+0000 7f00637fe640 1 -- 172.21.15.86:0/3848053340 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f005c02ca70 con 0x7f006c107260 2024-04-21T08:38:54.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.049+0000 7f00745b2640 1 -- 172.21.15.86:0/3848053340 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f006c1a72a0 con 0x7f006c107260 2024-04-21T08:38:54.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.049+0000 7f00637fe640 1 -- 172.21.15.86:0/3848053340 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f005c003ac0 con 0x7f006c107260 2024-04-21T08:38:54.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.049+0000 7f00745b2640 1 -- 172.21.15.86:0/3848053340 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f006c1a77f0 con 0x7f006c107260 2024-04-21T08:38:54.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.051+0000 7f00745b2640 1 -- 172.21.15.86:0/3848053340 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f006c069230 con 0x7f006c107260 2024-04-21T08:38:54.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.052+0000 7f00637fe640 1 -- 172.21.15.86:0/3848053340 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f005c046020 con 0x7f006c107260 2024-04-21T08:38:54.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.052+0000 7f00637fe640 1 --2- 172.21.15.86:0/3848053340 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f00580a3de0 0x7f00580a62a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:54.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.052+0000 7f007234e640 1 --2- 172.21.15.86:0/3848053340 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f00580a3de0 0x7f00580a62a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:54.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.053+0000 7f00637fe640 1 -- 172.21.15.86:0/3848053340 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f005c0f2ce0 con 0x7f006c107260 2024-04-21T08:38:54.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.053+0000 7f007234e640 1 --2- 172.21.15.86:0/3848053340 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f00580a3de0 0x7f00580a62a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0064005fd0 tx=0x7f0064009500 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:54.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.057+0000 7f00637fe640 1 -- 172.21.15.86:0/3848053340 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f005c04d020 con 0x7f006c107260 2024-04-21T08:38:54.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.236+0000 7f00745b2640 1 -- 172.21.15.86:0/3848053340 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f006c10a490 con 0x7f006c107260 2024-04-21T08:38:54.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.237+0000 7f00637fe640 1 -- 172.21.15.86:0/3848053340 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+25067 (secure 0 0 0) 0x7f005c0befb0 con 0x7f006c107260 2024-04-21T08:38:54.237 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:38:54.238 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":94,"fsid":"f8cec7cc-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:22.687329+0000","modified":"2024-04-21T08:38:46.631676+0000","last_up_change":"2024-04-21T08:35:30.048315+0000","last_in_change":"2024-04-21T08:35:10.099496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:45.279109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:18.300782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:19.574636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:22.905632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89b7601f-0620-4db2-90f1-b45cea173738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6803","nonce":2571516897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6805","nonce":2571516897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6809","nonce":2571516897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6807","nonce":2571516897}]},"public_addr":"172.21.15.86:6803/2571516897","cluster_addr":"172.21.15.86:6805/2571516897","heartbeat_back_addr":"172.21.15.86:6809/2571516897","heartbeat_front_addr":"172.21.15.86:6807/2571516897","state":["exists","up"]},{"osd":1,"uuid":"4891031c-f98c-42bc-bb3b-a420855ecc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6801","nonce":74067182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6803","nonce":74067182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6807","nonce":74067182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6805","nonce":74067182}]},"public_addr":"172.21.15.88:6801/74067182","cluster_addr":"172.21.15.88:6803/74067182","heartbeat_back_addr":"172.21.15.88:6807/74067182","heartbeat_front_addr":"172.21.15.88:6805/74067182","state":["exists","up"]},{"osd":2,"uuid":"60d12aa4-9312-4fa2-8d0b-2b3f42caad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6801","nonce":4137659051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6803","nonce":4137659051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6807","nonce":4137659051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6805","nonce":4137659051}]},"public_addr":"172.21.15.179:6801/4137659051","cluster_addr":"172.21.15.179:6803/4137659051","heartbeat_back_addr":"172.21.15.179:6807/4137659051","heartbeat_front_addr":"172.21.15.179:6805/4137659051","state":["exists","up"]},{"osd":3,"uuid":"404c3158-8cb0-4c02-b48a-f9ee5ff7b784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6811","nonce":2309603148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6813","nonce":2309603148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6817","nonce":2309603148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6815","nonce":2309603148}]},"public_addr":"172.21.15.86:6811/2309603148","cluster_addr":"172.21.15.86:6813/2309603148","heartbeat_back_addr":"172.21.15.86:6817/2309603148","heartbeat_front_addr":"172.21.15.86:6815/2309603148","state":["exists","up"]},{"osd":4,"uuid":"458ff202-d781-4ec7-ac93-0b49f835b503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6809","nonce":1818714217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6811","nonce":1818714217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6815","nonce":1818714217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6813","nonce":1818714217}]},"public_addr":"172.21.15.88:6809/1818714217","cluster_addr":"172.21.15.88:6811/1818714217","heartbeat_back_addr":"172.21.15.88:6815/1818714217","heartbeat_front_addr":"172.21.15.88:6813/1818714217","state":["exists","up"]},{"osd":5,"uuid":"8123cc90-bd8b-4204-bf31-46b34e9f589d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6809","nonce":863840990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6811","nonce":863840990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6815","nonce":863840990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6813","nonce":863840990}]},"public_addr":"172.21.15.179:6809/863840990","cluster_addr":"172.21.15.179:6811/863840990","heartbeat_back_addr":"172.21.15.179:6815/863840990","heartbeat_front_addr":"172.21.15.179:6813/863840990","state":["exists","up"]},{"osd":6,"uuid":"aac9d7c7-659a-4110-903a-a871c086066f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6819","nonce":1224231118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6821","nonce":1224231118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6825","nonce":1224231118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6823","nonce":1224231118}]},"public_addr":"172.21.15.86:6819/1224231118","cluster_addr":"172.21.15.86:6821/1224231118","heartbeat_back_addr":"172.21.15.86:6825/1224231118","heartbeat_front_addr":"172.21.15.86:6823/1224231118","state":["exists","up"]},{"osd":7,"uuid":"df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6817","nonce":136629878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6819","nonce":136629878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6823","nonce":136629878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6821","nonce":136629878}]},"public_addr":"172.21.15.88:6817/136629878","cluster_addr":"172.21.15.88:6819/136629878","heartbeat_back_addr":"172.21.15.88:6823/136629878","heartbeat_front_addr":"172.21.15.88:6821/136629878","state":["exists","up"]},{"osd":8,"uuid":"63e59f4b-0426-46db-92f7-c55dae61b5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6817","nonce":1770418957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6819","nonce":1770418957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6823","nonce":1770418957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6821","nonce":1770418957}]},"public_addr":"172.21.15.179:6817/1770418957","cluster_addr":"172.21.15.179:6819/1770418957","heartbeat_back_addr":"172.21.15.179:6823/1770418957","heartbeat_front_addr":"172.21.15.179:6821/1770418957","state":["exists","up"]},{"osd":9,"uuid":"5841bb36-ed6b-45d8-b857-eeab139c44e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6827","nonce":1108500274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6829","nonce":1108500274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6833","nonce":1108500274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6831","nonce":1108500274}]},"public_addr":"172.21.15.86:6827/1108500274","cluster_addr":"172.21.15.86:6829/1108500274","heartbeat_back_addr":"172.21.15.86:6833/1108500274","heartbeat_front_addr":"172.21.15.86:6831/1108500274","state":["exists","up"]},{"osd":10,"uuid":"f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6825","nonce":1294275353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6827","nonce":1294275353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6831","nonce":1294275353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6829","nonce":1294275353}]},"public_addr":"172.21.15.88:6825/1294275353","cluster_addr":"172.21.15.88:6827/1294275353","heartbeat_back_addr":"172.21.15.88:6831/1294275353","heartbeat_front_addr":"172.21.15.88:6829/1294275353","state":["exists","up"]},{"osd":11,"uuid":"e0d4cc76-e7a6-4a6b-909c-95488ccd5a36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6825","nonce":4007070233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6827","nonce":4007070233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6831","nonce":4007070233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6829","nonce":4007070233}]},"public_addr":"172.21.15.179:6825/4007070233","cluster_addr":"172.21.15.179:6827/4007070233","heartbeat_back_addr":"172.21.15.179:6831/4007070233","heartbeat_front_addr":"172.21.15.179:6829/4007070233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:55.210605+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:18.723090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:42.547979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:06.421622+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:30.858315+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:55.455313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.623369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:46.185111+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.103627+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:37.297545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:02.637431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:28.199292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1068089552":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/4273877219":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/2707594358":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/4060044103":"2024-04-22T08:38:46.631621+0000","172.21.15.86:6800/1364156965":"2024-04-22T08:38:46.631621+0000","172.21.15.86:6800/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/1364156965":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/1412393225":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6800/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/1687852404":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2020707404":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2568130933":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/3952607648":"2024-04-22T08:28:57.384403+0000","172.21.15.86:6801/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/2756903426":"2024-04-22T08:29:23.190001+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:38:54.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.239+0000 7f00745b2640 1 -- 172.21.15.86:0/3848053340 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f00580a3de0 msgr2=0x7f00580a62a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:54.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.239+0000 7f00745b2640 1 --2- 172.21.15.86:0/3848053340 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f00580a3de0 0x7f00580a62a0 secure :-1 s=READY pgs=56 cs=0 l=1 rev1=1 crypto rx=0x7f0064005fd0 tx=0x7f0064009500 comp rx=0 tx=0).stop 2024-04-21T08:38:54.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.239+0000 7f00745b2640 1 -- 172.21.15.86:0/3848053340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f006c107260 msgr2=0x7f006c074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:54.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.239+0000 7f00745b2640 1 --2- 172.21.15.86:0/3848053340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f006c107260 0x7f006c074790 secure :-1 s=READY pgs=564 cs=0 l=1 rev1=1 crypto rx=0x7f005c036040 tx=0x7f005c0045e0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.239+0000 7f00745b2640 1 -- 172.21.15.86:0/3848053340 shutdown_connections 2024-04-21T08:38:54.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.239+0000 7f00745b2640 1 --2- 172.21.15.86:0/3848053340 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f00580a3de0 0x7f00580a62a0 unknown :-1 s=CLOSED pgs=56 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.239+0000 7f00745b2640 1 --2- 172.21.15.86:0/3848053340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f006c074cd0 0x7f006c1a6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.239+0000 7f00745b2640 1 --2- 172.21.15.86:0/3848053340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f006c107260 0x7f006c074790 unknown :-1 s=CLOSED pgs=564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.239+0000 7f00745b2640 1 --2- 172.21.15.86:0/3848053340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f006c102190 0x7f006c074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.239+0000 7f00745b2640 1 -- 172.21.15.86:0/3848053340 >> 172.21.15.86:0/3848053340 conn(0x7f006c0793e0 msgr2=0x7f006c0fffd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:54.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.239+0000 7f00745b2640 1 -- 172.21.15.86:0/3848053340 shutdown_connections 2024-04-21T08:38:54.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.239+0000 7f00745b2640 1 -- 172.21.15.86:0/3848053340 wait complete. 2024-04-21T08:38:54.251 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:31:45.279109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:36:18.300782+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:36:19.574636+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:36:22.905632+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:38:54.251 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:38:54.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.354+0000 7efe61315640 1 -- 172.21.15.86:0/3504178877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe5c106160 msgr2=0x7efe5c10dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:54.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.354+0000 7efe61315640 1 --2- 172.21.15.86:0/3504178877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe5c106160 0x7efe5c10dd30 secure :-1 s=READY pgs=565 cs=0 l=1 rev1=1 crypto rx=0x7efe54009ca0 tx=0x7efe5402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.354+0000 7efe61315640 1 -- 172.21.15.86:0/3504178877 shutdown_connections 2024-04-21T08:38:54.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.354+0000 7efe61315640 1 --2- 172.21.15.86:0/3504178877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe5c106160 0x7efe5c10dd30 unknown :-1 s=CLOSED pgs=565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.354+0000 7efe61315640 1 --2- 172.21.15.86:0/3504178877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe5c101a40 0x7efe5c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.354+0000 7efe61315640 1 --2- 172.21.15.86:0/3504178877 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe5c101090 0x7efe5c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.354+0000 7efe61315640 1 -- 172.21.15.86:0/3504178877 >> 172.21.15.86:0/3504178877 conn(0x7efe5c077f50 msgr2=0x7efe5c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:54.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.354+0000 7efe61315640 1 -- 172.21.15.86:0/3504178877 shutdown_connections 2024-04-21T08:38:54.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.354+0000 7efe61315640 1 -- 172.21.15.86:0/3504178877 wait complete. 2024-04-21T08:38:54.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.355+0000 7efe61315640 1 Processor -- start 2024-04-21T08:38:54.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.355+0000 7efe61315640 1 -- start start 2024-04-21T08:38:54.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.355+0000 7efe61315640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe5c101090 0x7efe5c198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:54.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.355+0000 7efe61315640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe5c106160 0x7efe5c198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:54.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.355+0000 7efe61315640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe5c19cf40 0x7efe5c19d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:54.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.355+0000 7efe61315640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c1121e0 con 0x7efe5c101090 2024-04-21T08:38:54.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.355+0000 7efe61315640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c112060 con 0x7efe5c19cf40 2024-04-21T08:38:54.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.355+0000 7efe61315640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c112360 con 0x7efe5c106160 2024-04-21T08:38:54.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.355+0000 7efe5affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe5c101090 0x7efe5c198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:54.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.355+0000 7efe5affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe5c101090 0x7efe5c198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36360/0 (socket says 172.21.15.86:36360) 2024-04-21T08:38:54.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.355+0000 7efe5affd640 1 -- 172.21.15.86:0/2304155821 learned_addr learned my addr 172.21.15.86:0/2304155821 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:54.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.355+0000 7efe53fff640 1 --2- 172.21.15.86:0/2304155821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe5c106160 0x7efe5c198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:54.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.355+0000 7efe5b7fe640 1 --2- 172.21.15.86:0/2304155821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe5c19cf40 0x7efe5c19d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:54.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.355+0000 7efe5affd640 1 -- 172.21.15.86:0/2304155821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe5c106160 msgr2=0x7efe5c198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:54.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.355+0000 7efe5affd640 1 --2- 172.21.15.86:0/2304155821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe5c106160 0x7efe5c198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.355+0000 7efe5affd640 1 -- 172.21.15.86:0/2304155821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe5c19cf40 msgr2=0x7efe5c19d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:54.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.355+0000 7efe5affd640 1 --2- 172.21.15.86:0/2304155821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe5c19cf40 0x7efe5c19d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.355+0000 7efe5affd640 1 -- 172.21.15.86:0/2304155821 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe5c19da30 con 0x7efe5c101090 2024-04-21T08:38:54.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.355+0000 7efe5b7fe640 1 --2- 172.21.15.86:0/2304155821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe5c19cf40 0x7efe5c19d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:54.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.355+0000 7efe5affd640 1 --2- 172.21.15.86:0/2304155821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe5c101090 0x7efe5c198680 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7efe4c00c920 tx=0x7efe4c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:54.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.356+0000 7efe58ff9640 1 -- 172.21.15.86:0/2304155821 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c007c70 con 0x7efe5c101090 2024-04-21T08:38:54.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.356+0000 7efe58ff9640 1 -- 172.21.15.86:0/2304155821 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe4c007e10 con 0x7efe5c101090 2024-04-21T08:38:54.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.356+0000 7efe61315640 1 -- 172.21.15.86:0/2304155821 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe5c19dd20 con 0x7efe5c101090 2024-04-21T08:38:54.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.356+0000 7efe58ff9640 1 -- 172.21.15.86:0/2304155821 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4c018820 con 0x7efe5c101090 2024-04-21T08:38:54.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.356+0000 7efe61315640 1 -- 172.21.15.86:0/2304155821 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe5c1a5560 con 0x7efe5c101090 2024-04-21T08:38:54.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.357+0000 7efe61315640 1 -- 172.21.15.86:0/2304155821 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe5c068d70 con 0x7efe5c101090 2024-04-21T08:38:54.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.358+0000 7efe58ff9640 1 -- 172.21.15.86:0/2304155821 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7efe4c00f450 con 0x7efe5c101090 2024-04-21T08:38:54.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.359+0000 7efe58ff9640 1 --2- 172.21.15.86:0/2304155821 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efe480a3e30 0x7efe480a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:54.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.359+0000 7efe53fff640 1 --2- 172.21.15.86:0/2304155821 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efe480a3e30 0x7efe480a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:54.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.359+0000 7efe58ff9640 1 -- 172.21.15.86:0/2304155821 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7efe4c0d0dd0 con 0x7efe5c101090 2024-04-21T08:38:54.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.359+0000 7efe53fff640 1 --2- 172.21.15.86:0/2304155821 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efe480a3e30 0x7efe480a62f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efe5c100930 tx=0x7efe4400a250 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:54.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.363+0000 7efe58ff9640 1 -- 172.21.15.86:0/2304155821 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efe4c0d4030 con 0x7efe5c101090 2024-04-21T08:38:54.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.543+0000 7efe61315640 1 -- 172.21.15.86:0/2304155821 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7efe5c101a40 con 0x7efe5c101090 2024-04-21T08:38:54.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.543+0000 7efe58ff9640 1 -- 172.21.15.86:0/2304155821 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7efe4c014030 con 0x7efe5c101090 2024-04-21T08:38:54.543 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-21T08:38:54.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.545+0000 7efe61315640 1 -- 172.21.15.86:0/2304155821 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efe480a3e30 msgr2=0x7efe480a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:54.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.545+0000 7efe61315640 1 --2- 172.21.15.86:0/2304155821 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efe480a3e30 0x7efe480a62f0 secure :-1 s=READY pgs=57 cs=0 l=1 rev1=1 crypto rx=0x7efe5c100930 tx=0x7efe4400a250 comp rx=0 tx=0).stop 2024-04-21T08:38:54.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.545+0000 7efe61315640 1 -- 172.21.15.86:0/2304155821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe5c101090 msgr2=0x7efe5c198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:54.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.545+0000 7efe61315640 1 --2- 172.21.15.86:0/2304155821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe5c101090 0x7efe5c198680 secure :-1 s=READY pgs=566 cs=0 l=1 rev1=1 crypto rx=0x7efe4c00c920 tx=0x7efe4c00cde0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.545+0000 7efe61315640 1 -- 172.21.15.86:0/2304155821 shutdown_connections 2024-04-21T08:38:54.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.545+0000 7efe61315640 1 --2- 172.21.15.86:0/2304155821 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efe480a3e30 0x7efe480a62f0 unknown :-1 s=CLOSED pgs=57 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.545+0000 7efe61315640 1 --2- 172.21.15.86:0/2304155821 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe5c19cf40 0x7efe5c19d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.545+0000 7efe61315640 1 --2- 172.21.15.86:0/2304155821 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe5c106160 0x7efe5c198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.545+0000 7efe61315640 1 --2- 172.21.15.86:0/2304155821 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe5c101090 0x7efe5c198680 unknown :-1 s=CLOSED pgs=566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.545+0000 7efe61315640 1 -- 172.21.15.86:0/2304155821 >> 172.21.15.86:0/2304155821 conn(0x7efe5c077f50 msgr2=0x7efe5c0feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:54.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.545+0000 7efe61315640 1 -- 172.21.15.86:0/2304155821 shutdown_connections 2024-04-21T08:38:54.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.545+0000 7efe61315640 1 -- 172.21.15.86:0/2304155821 wait complete. 2024-04-21T08:38:54.555 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:54 smithi086 ceph-mon[27939]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-21T08:38:54.555 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:54 smithi086 ceph-mon[27939]: from='client.15762 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:54.555 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:54 smithi086 ceph-mon[27939]: from='client.15801 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:54.555 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:54 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3848053340' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:54.556 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:38:54.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.658+0000 7f500bb06640 1 -- 172.21.15.86:0/3671356621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5004101a50 msgr2=0x7f5004105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:54.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.658+0000 7f500bb06640 1 --2- 172.21.15.86:0/3671356621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5004101a50 0x7f5004105a90 secure :-1 s=READY pgs=567 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4009900 tx=0x7f4ff402bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:54.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.658+0000 7f500bb06640 1 -- 172.21.15.86:0/3671356621 shutdown_connections 2024-04-21T08:38:54.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.658+0000 7f500bb06640 1 --2- 172.21.15.86:0/3671356621 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5004106170 0x7f500410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.658+0000 7f500bb06640 1 --2- 172.21.15.86:0/3671356621 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5004101a50 0x7f5004105a90 unknown :-1 s=CLOSED pgs=567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.658+0000 7f500bb06640 1 --2- 172.21.15.86:0/3671356621 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f50041010a0 0x7f5004101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.658+0000 7f500bb06640 1 -- 172.21.15.86:0/3671356621 >> 172.21.15.86:0/3671356621 conn(0x7f50040771d0 msgr2=0x7f50040775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:54.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.658+0000 7f500bb06640 1 -- 172.21.15.86:0/3671356621 shutdown_connections 2024-04-21T08:38:54.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.658+0000 7f500bb06640 1 -- 172.21.15.86:0/3671356621 wait complete. 2024-04-21T08:38:54.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.658+0000 7f500bb06640 1 Processor -- start 2024-04-21T08:38:54.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.659+0000 7f500bb06640 1 -- start start 2024-04-21T08:38:54.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.659+0000 7f500bb06640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f50041010a0 0x7f50041975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:54.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.659+0000 7f500bb06640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5004106170 0x7f5004197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:54.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.659+0000 7f500bb06640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f500419be90 0x7f500419c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:54.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.659+0000 7f500bb06640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f50041110b0 con 0x7f5004106170 2024-04-21T08:38:54.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.659+0000 7f500bb06640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5004110f30 con 0x7f500419be90 2024-04-21T08:38:54.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.659+0000 7f500bb06640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5004111230 con 0x7f50041010a0 2024-04-21T08:38:54.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.659+0000 7f50090a1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5004106170 0x7f5004197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:54.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.659+0000 7f50090a1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5004106170 0x7f5004197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36380/0 (socket says 172.21.15.86:36380) 2024-04-21T08:38:54.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.659+0000 7f50090a1640 1 -- 172.21.15.86:0/508432928 learned_addr learned my addr 172.21.15.86:0/508432928 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:54.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.659+0000 7f50098a2640 1 --2- 172.21.15.86:0/508432928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f50041010a0 0x7f50041975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:54.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.659+0000 7f500a0a3640 1 --2- 172.21.15.86:0/508432928 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f500419be90 0x7f500419c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:54.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.659+0000 7f50090a1640 1 -- 172.21.15.86:0/508432928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f50041010a0 msgr2=0x7f50041975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:54.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.659+0000 7f50090a1640 1 --2- 172.21.15.86:0/508432928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f50041010a0 0x7f50041975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.659+0000 7f50090a1640 1 -- 172.21.15.86:0/508432928 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f500419be90 msgr2=0x7f500419c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:54.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.659+0000 7f50090a1640 1 --2- 172.21.15.86:0/508432928 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f500419be90 0x7f500419c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.659+0000 7f50090a1640 1 -- 172.21.15.86:0/508432928 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f500419c980 con 0x7f5004106170 2024-04-21T08:38:54.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.659+0000 7f500a0a3640 1 --2- 172.21.15.86:0/508432928 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f500419be90 0x7f500419c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:54.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.659+0000 7f50090a1640 1 --2- 172.21.15.86:0/508432928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5004106170 0x7f5004197b10 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4005ec0 tx=0x7f4ff402ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:54.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.659+0000 7f4ffaffd640 1 -- 172.21.15.86:0/508432928 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4043020 con 0x7f5004106170 2024-04-21T08:38:54.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.659+0000 7f500bb06640 1 -- 172.21.15.86:0/508432928 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f500419cc10 con 0x7f5004106170 2024-04-21T08:38:54.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.659+0000 7f4ffaffd640 1 -- 172.21.15.86:0/508432928 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ff40052b0 con 0x7f5004106170 2024-04-21T08:38:54.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.659+0000 7f4ffaffd640 1 -- 172.21.15.86:0/508432928 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ff4003d40 con 0x7f5004106170 2024-04-21T08:38:54.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.660+0000 7f500bb06640 1 -- 172.21.15.86:0/508432928 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f50041a4460 con 0x7f5004106170 2024-04-21T08:38:54.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.661+0000 7f500bb06640 1 -- 172.21.15.86:0/508432928 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f50040691c0 con 0x7f5004106170 2024-04-21T08:38:54.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.662+0000 7f4ffaffd640 1 -- 172.21.15.86:0/508432928 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f4ff4046020 con 0x7f5004106170 2024-04-21T08:38:54.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.662+0000 7f4ffaffd640 1 --2- 172.21.15.86:0/508432928 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4ff00a3db0 0x7f4ff00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:54.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.662+0000 7f4ffaffd640 1 -- 172.21.15.86:0/508432928 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f4ff40f3670 con 0x7f5004106170 2024-04-21T08:38:54.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.662+0000 7f50098a2640 1 --2- 172.21.15.86:0/508432928 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4ff00a3db0 0x7f4ff00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:54.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.663+0000 7f50098a2640 1 --2- 172.21.15.86:0/508432928 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4ff00a3db0 0x7f4ff00a6270 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5000005e10 tx=0x7f5000005da0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:54.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.666+0000 7f4ffaffd640 1 -- 172.21.15.86:0/508432928 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4ff404d020 con 0x7f5004106170 2024-04-21T08:38:54.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:54 smithi088 ceph-mon[31259]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-21T08:38:54.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:54 smithi088 ceph-mon[31259]: from='client.15762 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:54.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:54 smithi088 ceph-mon[31259]: from='client.15801 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:54.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:54 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3848053340' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:54.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:54 smithi179 ceph-mon[31829]: pgmap v6: 129 pgs: 129 active+clean; 1.1 MiB data, 377 MiB used, 1.0 TiB / 1.0 TiB avail; 1.0 KiB/s rd, 2 op/s 2024-04-21T08:38:54.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:54 smithi179 ceph-mon[31829]: from='client.15762 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule add", "fs": "cephfs", "path": "/", "snap_schedule": "1m", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:54.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:54 smithi179 ceph-mon[31829]: from='client.15801 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule retention add", "fs": "cephfs", "path": "/", "retention_spec_or_period": "6m3h", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:54.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:54 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3848053340' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:54.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.841+0000 7f500bb06640 1 -- 172.21.15.86:0/508432928 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f5004101a50 con 0x7f5004106170 2024-04-21T08:38:54.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.841+0000 7f4ffaffd640 1 -- 172.21.15.86:0/508432928 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f4ff40bfa60 con 0x7f5004106170 2024-04-21T08:38:54.842 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-21T08:38:54.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.843+0000 7f500bb06640 1 -- 172.21.15.86:0/508432928 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4ff00a3db0 msgr2=0x7f4ff00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:54.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.843+0000 7f500bb06640 1 --2- 172.21.15.86:0/508432928 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4ff00a3db0 0x7f4ff00a6270 secure :-1 s=READY pgs=58 cs=0 l=1 rev1=1 crypto rx=0x7f5000005e10 tx=0x7f5000005da0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.843+0000 7f500bb06640 1 -- 172.21.15.86:0/508432928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5004106170 msgr2=0x7f5004197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:54.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.843+0000 7f500bb06640 1 --2- 172.21.15.86:0/508432928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5004106170 0x7f5004197b10 secure :-1 s=READY pgs=568 cs=0 l=1 rev1=1 crypto rx=0x7f4ff4005ec0 tx=0x7f4ff402ca10 comp rx=0 tx=0).stop 2024-04-21T08:38:54.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.843+0000 7f500bb06640 1 -- 172.21.15.86:0/508432928 shutdown_connections 2024-04-21T08:38:54.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.843+0000 7f500bb06640 1 --2- 172.21.15.86:0/508432928 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4ff00a3db0 0x7f4ff00a6270 unknown :-1 s=CLOSED pgs=58 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.843+0000 7f500bb06640 1 --2- 172.21.15.86:0/508432928 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f500419be90 0x7f500419c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.843+0000 7f500bb06640 1 --2- 172.21.15.86:0/508432928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5004106170 0x7f5004197b10 unknown :-1 s=CLOSED pgs=568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.843+0000 7f500bb06640 1 --2- 172.21.15.86:0/508432928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f50041010a0 0x7f50041975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.843+0000 7f500bb06640 1 -- 172.21.15.86:0/508432928 >> 172.21.15.86:0/508432928 conn(0x7f50040771d0 msgr2=0x7f50040fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:54.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.843+0000 7f500bb06640 1 -- 172.21.15.86:0/508432928 shutdown_connections 2024-04-21T08:38:54.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.843+0000 7f500bb06640 1 -- 172.21.15.86:0/508432928 wait complete. 2024-04-21T08:38:54.855 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:38:54.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.958+0000 7f29ab3f4640 1 -- 172.21.15.86:0/249686265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a406af10 msgr2=0x7f29a410ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:54.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.958+0000 7f29ab3f4640 1 --2- 172.21.15.86:0/249686265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a406af10 0x7f29a410ede0 secure :-1 s=READY pgs=569 cs=0 l=1 rev1=1 crypto rx=0x7f299c009cd0 tx=0x7f299c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.958+0000 7f29ab3f4640 1 -- 172.21.15.86:0/249686265 shutdown_connections 2024-04-21T08:38:54.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.958+0000 7f29ab3f4640 1 --2- 172.21.15.86:0/249686265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a406af10 0x7f29a410ede0 unknown :-1 s=CLOSED pgs=569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.958+0000 7f29ab3f4640 1 --2- 172.21.15.86:0/249686265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a406a550 0x7f29a406a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.958+0000 7f29ab3f4640 1 --2- 172.21.15.86:0/249686265 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29a4108000 0x7f29a41083e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.958+0000 7f29ab3f4640 1 -- 172.21.15.86:0/249686265 >> 172.21.15.86:0/249686265 conn(0x7f29a4076c50 msgr2=0x7f29a4077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:54.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.958+0000 7f29ab3f4640 1 -- 172.21.15.86:0/249686265 shutdown_connections 2024-04-21T08:38:54.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.958+0000 7f29ab3f4640 1 -- 172.21.15.86:0/249686265 wait complete. 2024-04-21T08:38:54.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.958+0000 7f29ab3f4640 1 Processor -- start 2024-04-21T08:38:54.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.958+0000 7f29ab3f4640 1 -- start start 2024-04-21T08:38:54.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.959+0000 7f29ab3f4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a406a550 0x7f29a4198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:54.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.959+0000 7f29ab3f4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a406af10 0x7f29a4198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:54.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.959+0000 7f29ab3f4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29a4108000 0x7f29a419cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:54.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.959+0000 7f29ab3f4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29a4112190 con 0x7f29a406af10 2024-04-21T08:38:54.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.959+0000 7f29ab3f4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f29a4112010 con 0x7f29a4108000 2024-04-21T08:38:54.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.959+0000 7f29ab3f4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f29a4112310 con 0x7f29a406a550 2024-04-21T08:38:54.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.959+0000 7f29a898f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a406af10 0x7f29a4198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:54.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.959+0000 7f29a898f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a406af10 0x7f29a4198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36400/0 (socket says 172.21.15.86:36400) 2024-04-21T08:38:54.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.959+0000 7f29a898f640 1 -- 172.21.15.86:0/433338166 learned_addr learned my addr 172.21.15.86:0/433338166 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:54.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.959+0000 7f29a9190640 1 --2- 172.21.15.86:0/433338166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a406a550 0x7f29a4198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:54.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.959+0000 7f29a9991640 1 --2- 172.21.15.86:0/433338166 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29a4108000 0x7f29a419cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:54.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.959+0000 7f29a898f640 1 -- 172.21.15.86:0/433338166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a406a550 msgr2=0x7f29a4198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:54.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.959+0000 7f29a898f640 1 --2- 172.21.15.86:0/433338166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a406a550 0x7f29a4198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.959+0000 7f29a898f640 1 -- 172.21.15.86:0/433338166 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29a4108000 msgr2=0x7f29a419cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:54.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.959+0000 7f29a898f640 1 --2- 172.21.15.86:0/433338166 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29a4108000 0x7f29a419cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:54.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.959+0000 7f29a898f640 1 -- 172.21.15.86:0/433338166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29a419d700 con 0x7f29a406af10 2024-04-21T08:38:54.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.959+0000 7f29a9991640 1 --2- 172.21.15.86:0/433338166 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29a4108000 0x7f29a419cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:54.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.959+0000 7f29a898f640 1 --2- 172.21.15.86:0/433338166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a406af10 0x7f29a4198c50 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f299400e9f0 tx=0x7f299400edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:54.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.959+0000 7f299a7fc640 1 -- 172.21.15.86:0/433338166 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f299400cd80 con 0x7f29a406af10 2024-04-21T08:38:54.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.959+0000 7f299a7fc640 1 -- 172.21.15.86:0/433338166 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f29940054d0 con 0x7f29a406af10 2024-04-21T08:38:54.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.959+0000 7f29ab3f4640 1 -- 172.21.15.86:0/433338166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29a419d9f0 con 0x7f29a406af10 2024-04-21T08:38:54.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.959+0000 7f299a7fc640 1 -- 172.21.15.86:0/433338166 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2994010690 con 0x7f29a406af10 2024-04-21T08:38:54.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.960+0000 7f29ab3f4640 1 -- 172.21.15.86:0/433338166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29a41a5280 con 0x7f29a406af10 2024-04-21T08:38:54.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.960+0000 7f29ab3f4640 1 -- 172.21.15.86:0/433338166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f29a4103660 con 0x7f29a406af10 2024-04-21T08:38:54.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.962+0000 7f299a7fc640 1 -- 172.21.15.86:0/433338166 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f2994004e50 con 0x7f29a406af10 2024-04-21T08:38:54.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.962+0000 7f299a7fc640 1 --2- 172.21.15.86:0/433338166 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f29900a3db0 0x7f29900a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:54.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.962+0000 7f29a9190640 1 --2- 172.21.15.86:0/433338166 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f29900a3db0 0x7f29900a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:54.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.962+0000 7f299a7fc640 1 -- 172.21.15.86:0/433338166 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f29940d13a0 con 0x7f29a406af10 2024-04-21T08:38:54.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.963+0000 7f29a9190640 1 --2- 172.21.15.86:0/433338166 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f29900a3db0 0x7f29900a6270 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f29a000bb60 tx=0x7f29a000b390 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:54.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:54.966+0000 7f299a7fc640 1 -- 172.21.15.86:0/433338166 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2994053ff0 con 0x7f29a406af10 2024-04-21T08:38:55.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.145+0000 7f29ab3f4640 1 -- 172.21.15.86:0/433338166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f29a402d5c0 con 0x7f29a406af10 2024-04-21T08:38:55.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.145+0000 7f299a7fc640 1 -- 172.21.15.86:0/433338166 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f299409d560 con 0x7f29a406af10 2024-04-21T08:38:55.146 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-21T08:38:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.148+0000 7f29ab3f4640 1 -- 172.21.15.86:0/433338166 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f29900a3db0 msgr2=0x7f29900a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.148+0000 7f29ab3f4640 1 --2- 172.21.15.86:0/433338166 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f29900a3db0 0x7f29900a6270 secure :-1 s=READY pgs=59 cs=0 l=1 rev1=1 crypto rx=0x7f29a000bb60 tx=0x7f29a000b390 comp rx=0 tx=0).stop 2024-04-21T08:38:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.148+0000 7f29ab3f4640 1 -- 172.21.15.86:0/433338166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a406af10 msgr2=0x7f29a4198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.148+0000 7f29ab3f4640 1 --2- 172.21.15.86:0/433338166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a406af10 0x7f29a4198c50 secure :-1 s=READY pgs=570 cs=0 l=1 rev1=1 crypto rx=0x7f299400e9f0 tx=0x7f299400edf0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.148+0000 7f29ab3f4640 1 -- 172.21.15.86:0/433338166 shutdown_connections 2024-04-21T08:38:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.148+0000 7f29ab3f4640 1 --2- 172.21.15.86:0/433338166 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f29900a3db0 0x7f29900a6270 unknown :-1 s=CLOSED pgs=59 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.148+0000 7f29ab3f4640 1 --2- 172.21.15.86:0/433338166 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29a4108000 0x7f29a419cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.148+0000 7f29ab3f4640 1 --2- 172.21.15.86:0/433338166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29a406af10 0x7f29a4198c50 unknown :-1 s=CLOSED pgs=570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.148+0000 7f29ab3f4640 1 --2- 172.21.15.86:0/433338166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f29a406a550 0x7f29a4198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.148+0000 7f29ab3f4640 1 -- 172.21.15.86:0/433338166 >> 172.21.15.86:0/433338166 conn(0x7f29a4076c50 msgr2=0x7f29a4100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:55.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.148+0000 7f29ab3f4640 1 -- 172.21.15.86:0/433338166 shutdown_connections 2024-04-21T08:38:55.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.148+0000 7f29ab3f4640 1 -- 172.21.15.86:0/433338166 wait complete. 2024-04-21T08:38:55.159 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:38:55.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.259+0000 7fc276ec1640 1 -- 172.21.15.86:0/2701982653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc270106160 msgr2=0x7fc27010dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:55.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.259+0000 7fc276ec1640 1 --2- 172.21.15.86:0/2701982653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc270106160 0x7fc27010dd30 secure :-1 s=READY pgs=571 cs=0 l=1 rev1=1 crypto rx=0x7fc268009900 tx=0x7fc26802bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:55.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.259+0000 7fc276ec1640 1 -- 172.21.15.86:0/2701982653 shutdown_connections 2024-04-21T08:38:55.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.259+0000 7fc276ec1640 1 --2- 172.21.15.86:0/2701982653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc270106160 0x7fc27010dd30 unknown :-1 s=CLOSED pgs=571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.259+0000 7fc276ec1640 1 --2- 172.21.15.86:0/2701982653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc270101a40 0x7fc270105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.259+0000 7fc276ec1640 1 --2- 172.21.15.86:0/2701982653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc270101090 0x7fc270101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.259+0000 7fc276ec1640 1 -- 172.21.15.86:0/2701982653 >> 172.21.15.86:0/2701982653 conn(0x7fc270077f50 msgr2=0x7fc270078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:55.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.260+0000 7fc276ec1640 1 -- 172.21.15.86:0/2701982653 shutdown_connections 2024-04-21T08:38:55.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.260+0000 7fc276ec1640 1 -- 172.21.15.86:0/2701982653 wait complete. 2024-04-21T08:38:55.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.260+0000 7fc276ec1640 1 Processor -- start 2024-04-21T08:38:55.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.260+0000 7fc276ec1640 1 -- start start 2024-04-21T08:38:55.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.260+0000 7fc276ec1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc270101090 0x7fc270197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:55.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.260+0000 7fc276ec1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc270106160 0x7fc270197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:55.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.260+0000 7fc276ec1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc27019bef0 0x7fc27019c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:55.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.260+0000 7fc276ec1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc2701110d0 con 0x7fc270106160 2024-04-21T08:38:55.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.260+0000 7fc276ec1640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc270110f50 con 0x7fc27019bef0 2024-04-21T08:38:55.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.260+0000 7fc276ec1640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc270111250 con 0x7fc270101090 2024-04-21T08:38:55.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.260+0000 7fc26ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc270106160 0x7fc270197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:55.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.260+0000 7fc26ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc270106160 0x7fc270197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36424/0 (socket says 172.21.15.86:36424) 2024-04-21T08:38:55.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.260+0000 7fc26ffff640 1 -- 172.21.15.86:0/1709866020 learned_addr learned my addr 172.21.15.86:0/1709866020 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:55.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.261+0000 7fc27545e640 1 --2- 172.21.15.86:0/1709866020 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc27019bef0 0x7fc27019c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:55.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.261+0000 7fc274c5d640 1 --2- 172.21.15.86:0/1709866020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc270101090 0x7fc270197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:55.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.261+0000 7fc26ffff640 1 -- 172.21.15.86:0/1709866020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc270101090 msgr2=0x7fc270197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:55.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.261+0000 7fc26ffff640 1 --2- 172.21.15.86:0/1709866020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc270101090 0x7fc270197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.261+0000 7fc26ffff640 1 -- 172.21.15.86:0/1709866020 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc27019bef0 msgr2=0x7fc27019c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:55.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.261+0000 7fc26ffff640 1 --2- 172.21.15.86:0/1709866020 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc27019bef0 0x7fc27019c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.261+0000 7fc26ffff640 1 -- 172.21.15.86:0/1709866020 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc27019c9e0 con 0x7fc270106160 2024-04-21T08:38:55.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.261+0000 7fc274c5d640 1 --2- 172.21.15.86:0/1709866020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc270101090 0x7fc270197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:55.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.261+0000 7fc27545e640 1 --2- 172.21.15.86:0/1709866020 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc27019bef0 0x7fc27019c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:55.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.261+0000 7fc26ffff640 1 --2- 172.21.15.86:0/1709866020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc270106160 0x7fc270197b70 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc26000cce0 tx=0x7fc26000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:55.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.261+0000 7fc26dffb640 1 -- 172.21.15.86:0/1709866020 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc260004b20 con 0x7fc270106160 2024-04-21T08:38:55.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.261+0000 7fc26dffb640 1 -- 172.21.15.86:0/1709866020 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc260004cc0 con 0x7fc270106160 2024-04-21T08:38:55.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.261+0000 7fc276ec1640 1 -- 172.21.15.86:0/1709866020 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc27019ccd0 con 0x7fc270106160 2024-04-21T08:38:55.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.261+0000 7fc26dffb640 1 -- 172.21.15.86:0/1709866020 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc26000f6b0 con 0x7fc270106160 2024-04-21T08:38:55.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.261+0000 7fc276ec1640 1 -- 172.21.15.86:0/1709866020 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2701a4470 con 0x7fc270106160 2024-04-21T08:38:55.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.262+0000 7fc276ec1640 1 -- 172.21.15.86:0/1709866020 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc270068d70 con 0x7fc270106160 2024-04-21T08:38:55.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.263+0000 7fc26dffb640 1 -- 172.21.15.86:0/1709866020 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fc26000f950 con 0x7fc270106160 2024-04-21T08:38:55.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.263+0000 7fc26dffb640 1 --2- 172.21.15.86:0/1709866020 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc25c0a3db0 0x7fc25c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:55.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.264+0000 7fc274c5d640 1 --2- 172.21.15.86:0/1709866020 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc25c0a3db0 0x7fc25c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:55.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.264+0000 7fc26dffb640 1 -- 172.21.15.86:0/1709866020 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fc2600d0e70 con 0x7fc270106160 2024-04-21T08:38:55.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.264+0000 7fc274c5d640 1 --2- 172.21.15.86:0/1709866020 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc25c0a3db0 0x7fc25c0a6270 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc258005d00 tx=0x7fc258009550 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:55.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.268+0000 7fc26dffb640 1 -- 172.21.15.86:0/1709866020 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc260014030 con 0x7fc270106160 2024-04-21T08:38:55.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.445+0000 7fc276ec1640 1 -- 172.21.15.86:0/1709866020 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc270101a40 con 0x7fc270106160 2024-04-21T08:38:55.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.446+0000 7fc26dffb640 1 -- 172.21.15.86:0/1709866020 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7fc26009d030 con 0x7fc270106160 2024-04-21T08:38:55.446 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-21T08:38:55.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.448+0000 7fc276ec1640 1 -- 172.21.15.86:0/1709866020 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc25c0a3db0 msgr2=0x7fc25c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:55.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.448+0000 7fc276ec1640 1 --2- 172.21.15.86:0/1709866020 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc25c0a3db0 0x7fc25c0a6270 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7fc258005d00 tx=0x7fc258009550 comp rx=0 tx=0).stop 2024-04-21T08:38:55.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.448+0000 7fc276ec1640 1 -- 172.21.15.86:0/1709866020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc270106160 msgr2=0x7fc270197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:55.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.448+0000 7fc276ec1640 1 --2- 172.21.15.86:0/1709866020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc270106160 0x7fc270197b70 secure :-1 s=READY pgs=572 cs=0 l=1 rev1=1 crypto rx=0x7fc26000cce0 tx=0x7fc26000a620 comp rx=0 tx=0).stop 2024-04-21T08:38:55.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.448+0000 7fc276ec1640 1 -- 172.21.15.86:0/1709866020 shutdown_connections 2024-04-21T08:38:55.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.448+0000 7fc276ec1640 1 --2- 172.21.15.86:0/1709866020 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc25c0a3db0 0x7fc25c0a6270 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.448+0000 7fc276ec1640 1 --2- 172.21.15.86:0/1709866020 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc27019bef0 0x7fc27019c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.448+0000 7fc276ec1640 1 --2- 172.21.15.86:0/1709866020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc270106160 0x7fc270197b70 unknown :-1 s=CLOSED pgs=572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.448+0000 7fc276ec1640 1 --2- 172.21.15.86:0/1709866020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc270101090 0x7fc270197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.448+0000 7fc276ec1640 1 -- 172.21.15.86:0/1709866020 >> 172.21.15.86:0/1709866020 conn(0x7fc270077f50 msgr2=0x7fc2700ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:55.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.448+0000 7fc276ec1640 1 -- 172.21.15.86:0/1709866020 shutdown_connections 2024-04-21T08:38:55.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.448+0000 7fc276ec1640 1 -- 172.21.15.86:0/1709866020 wait complete. 2024-04-21T08:38:55.459 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:55 smithi086 ceph-mon[27939]: from='client.15807 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:55.459 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:55 smithi086 ceph-mon[27939]: from='client.15813 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:55.459 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:55 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2304155821' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:55.459 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:55 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/508432928' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:55.459 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:55 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/433338166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:55.460 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:38:55.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.563+0000 7f36c07db640 1 -- 172.21.15.86:0/451564170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36b8106f50 msgr2=0x7f36b8107330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:55.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.563+0000 7f36c07db640 1 --2- 172.21.15.86:0/451564170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36b8106f50 0x7f36b8107330 secure :-1 s=READY pgs=573 cs=0 l=1 rev1=1 crypto rx=0x7f36b4009930 tx=0x7f36b402bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:55.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.563+0000 7f36c07db640 1 -- 172.21.15.86:0/451564170 shutdown_connections 2024-04-21T08:38:55.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.563+0000 7f36c07db640 1 --2- 172.21.15.86:0/451564170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36b8100e80 0x7f36b810ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.563+0000 7f36c07db640 1 --2- 172.21.15.86:0/451564170 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36b81004c0 0x7f36b8100940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.563+0000 7f36c07db640 1 --2- 172.21.15.86:0/451564170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36b8106f50 0x7f36b8107330 unknown :-1 s=CLOSED pgs=573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.563+0000 7f36c07db640 1 -- 172.21.15.86:0/451564170 >> 172.21.15.86:0/451564170 conn(0x7f36b8077450 msgr2=0x7f36b80fe7d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:55.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.563+0000 7f36c07db640 1 -- 172.21.15.86:0/451564170 shutdown_connections 2024-04-21T08:38:55.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.563+0000 7f36c07db640 1 -- 172.21.15.86:0/451564170 wait complete. 2024-04-21T08:38:55.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.563+0000 7f36c07db640 1 Processor -- start 2024-04-21T08:38:55.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.564+0000 7f36c07db640 1 -- start start 2024-04-21T08:38:55.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.564+0000 7f36c07db640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36b81004c0 0x7f36b81016a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:55.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.564+0000 7f36c07db640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36b8100e80 0x7f36b8101be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:55.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.564+0000 7f36c07db640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36b8106f50 0x7f36b8102120 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:55.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.564+0000 7f36c07db640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f36b8112280 con 0x7f36b81004c0 2024-04-21T08:38:55.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.564+0000 7f36c07db640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f36b8112100 con 0x7f36b8100e80 2024-04-21T08:38:55.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.564+0000 7f36c07db640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f36b8112400 con 0x7f36b8106f50 2024-04-21T08:38:55.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.564+0000 7f36be577640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36b81004c0 0x7f36b81016a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:55.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.564+0000 7f36be577640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36b81004c0 0x7f36b81016a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36438/0 (socket says 172.21.15.86:36438) 2024-04-21T08:38:55.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.564+0000 7f36be577640 1 -- 172.21.15.86:0/1258657881 learned_addr learned my addr 172.21.15.86:0/1258657881 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:55.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.564+0000 7f36bed78640 1 --2- 172.21.15.86:0/1258657881 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36b8106f50 0x7f36b8102120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:55.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.564+0000 7f36be577640 1 -- 172.21.15.86:0/1258657881 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36b8106f50 msgr2=0x7f36b8102120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:55.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.564+0000 7f36be577640 1 --2- 172.21.15.86:0/1258657881 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36b8106f50 0x7f36b8102120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.564+0000 7f36be577640 1 -- 172.21.15.86:0/1258657881 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36b8100e80 msgr2=0x7f36b8101be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:55.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.564+0000 7f36be577640 1 --2- 172.21.15.86:0/1258657881 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36b8100e80 0x7f36b8101be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.564+0000 7f36be577640 1 -- 172.21.15.86:0/1258657881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f36b81a6cc0 con 0x7f36b81004c0 2024-04-21T08:38:55.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.564+0000 7f36bed78640 1 --2- 172.21.15.86:0/1258657881 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36b8106f50 0x7f36b8102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:55.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.564+0000 7f36be577640 1 --2- 172.21.15.86:0/1258657881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36b81004c0 0x7f36b81016a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f36b4005940 tx=0x7f36b402cd20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:55.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.564+0000 7f36af7fe640 1 -- 172.21.15.86:0/1258657881 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b4043020 con 0x7f36b81004c0 2024-04-21T08:38:55.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.564+0000 7f36c07db640 1 -- 172.21.15.86:0/1258657881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f36b81a6f50 con 0x7f36b81004c0 2024-04-21T08:38:55.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.564+0000 7f36af7fe640 1 -- 172.21.15.86:0/1258657881 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f36b40043d0 con 0x7f36b81004c0 2024-04-21T08:38:55.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.564+0000 7f36af7fe640 1 -- 172.21.15.86:0/1258657881 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f36b4003af0 con 0x7f36b81004c0 2024-04-21T08:38:55.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.565+0000 7f36c07db640 1 -- 172.21.15.86:0/1258657881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f36b81a7410 con 0x7f36b81004c0 2024-04-21T08:38:55.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.565+0000 7f36c07db640 1 -- 172.21.15.86:0/1258657881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f36b8069230 con 0x7f36b81004c0 2024-04-21T08:38:55.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.567+0000 7f36af7fe640 1 -- 172.21.15.86:0/1258657881 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f36b4046020 con 0x7f36b81004c0 2024-04-21T08:38:55.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.567+0000 7f36af7fe640 1 --2- 172.21.15.86:0/1258657881 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f36a40a3db0 0x7f36a40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:55.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.567+0000 7f36bdd76640 1 --2- 172.21.15.86:0/1258657881 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f36a40a3db0 0x7f36a40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:55.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.567+0000 7f36af7fe640 1 -- 172.21.15.86:0/1258657881 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f36b40f2c90 con 0x7f36b81004c0 2024-04-21T08:38:55.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.567+0000 7f36bdd76640 1 --2- 172.21.15.86:0/1258657881 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f36a40a3db0 0x7f36a40a6270 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f36a80058c0 tx=0x7f36a8005850 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:55.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.571+0000 7f36af7fe640 1 -- 172.21.15.86:0/1258657881 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f36b404d070 con 0x7f36b81004c0 2024-04-21T08:38:55.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:55 smithi088 ceph-mon[31259]: from='client.15807 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:55.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:55 smithi088 ceph-mon[31259]: from='client.15813 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:55.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:55 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2304155821' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:55.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:55 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/508432928' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:55.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:55 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/433338166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:55.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:55 smithi179 ceph-mon[31829]: from='client.15807 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:55.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:55 smithi179 ceph-mon[31829]: from='client.15813 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:38:55.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:55 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2304155821' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:55.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:55 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/508432928' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:55.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:55 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/433338166' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:55.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.749+0000 7f36c07db640 1 -- 172.21.15.86:0/1258657881 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f36b8109ef0 con 0x7f36b81004c0 2024-04-21T08:38:55.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.751+0000 7f36af7fe640 1 -- 172.21.15.86:0/1258657881 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+25067 (secure 0 0 0) 0x7f36b403c350 con 0x7f36b81004c0 2024-04-21T08:38:55.751 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:38:55.751 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":94,"fsid":"f8cec7cc-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:22.687329+0000","modified":"2024-04-21T08:38:46.631676+0000","last_up_change":"2024-04-21T08:35:30.048315+0000","last_in_change":"2024-04-21T08:35:10.099496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:45.279109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:18.300782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:19.574636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:22.905632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89b7601f-0620-4db2-90f1-b45cea173738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6803","nonce":2571516897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6805","nonce":2571516897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6809","nonce":2571516897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6807","nonce":2571516897}]},"public_addr":"172.21.15.86:6803/2571516897","cluster_addr":"172.21.15.86:6805/2571516897","heartbeat_back_addr":"172.21.15.86:6809/2571516897","heartbeat_front_addr":"172.21.15.86:6807/2571516897","state":["exists","up"]},{"osd":1,"uuid":"4891031c-f98c-42bc-bb3b-a420855ecc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6801","nonce":74067182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6803","nonce":74067182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6807","nonce":74067182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6805","nonce":74067182}]},"public_addr":"172.21.15.88:6801/74067182","cluster_addr":"172.21.15.88:6803/74067182","heartbeat_back_addr":"172.21.15.88:6807/74067182","heartbeat_front_addr":"172.21.15.88:6805/74067182","state":["exists","up"]},{"osd":2,"uuid":"60d12aa4-9312-4fa2-8d0b-2b3f42caad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6801","nonce":4137659051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6803","nonce":4137659051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6807","nonce":4137659051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6805","nonce":4137659051}]},"public_addr":"172.21.15.179:6801/4137659051","cluster_addr":"172.21.15.179:6803/4137659051","heartbeat_back_addr":"172.21.15.179:6807/4137659051","heartbeat_front_addr":"172.21.15.179:6805/4137659051","state":["exists","up"]},{"osd":3,"uuid":"404c3158-8cb0-4c02-b48a-f9ee5ff7b784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6811","nonce":2309603148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6813","nonce":2309603148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6817","nonce":2309603148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6815","nonce":2309603148}]},"public_addr":"172.21.15.86:6811/2309603148","cluster_addr":"172.21.15.86:6813/2309603148","heartbeat_back_addr":"172.21.15.86:6817/2309603148","heartbeat_front_addr":"172.21.15.86:6815/2309603148","state":["exists","up"]},{"osd":4,"uuid":"458ff202-d781-4ec7-ac93-0b49f835b503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6809","nonce":1818714217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6811","nonce":1818714217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6815","nonce":1818714217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6813","nonce":1818714217}]},"public_addr":"172.21.15.88:6809/1818714217","cluster_addr":"172.21.15.88:6811/1818714217","heartbeat_back_addr":"172.21.15.88:6815/1818714217","heartbeat_front_addr":"172.21.15.88:6813/1818714217","state":["exists","up"]},{"osd":5,"uuid":"8123cc90-bd8b-4204-bf31-46b34e9f589d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6809","nonce":863840990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6811","nonce":863840990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6815","nonce":863840990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6813","nonce":863840990}]},"public_addr":"172.21.15.179:6809/863840990","cluster_addr":"172.21.15.179:6811/863840990","heartbeat_back_addr":"172.21.15.179:6815/863840990","heartbeat_front_addr":"172.21.15.179:6813/863840990","state":["exists","up"]},{"osd":6,"uuid":"aac9d7c7-659a-4110-903a-a871c086066f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6819","nonce":1224231118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6821","nonce":1224231118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6825","nonce":1224231118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6823","nonce":1224231118}]},"public_addr":"172.21.15.86:6819/1224231118","cluster_addr":"172.21.15.86:6821/1224231118","heartbeat_back_addr":"172.21.15.86:6825/1224231118","heartbeat_front_addr":"172.21.15.86:6823/1224231118","state":["exists","up"]},{"osd":7,"uuid":"df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6817","nonce":136629878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6819","nonce":136629878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6823","nonce":136629878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6821","nonce":136629878}]},"public_addr":"172.21.15.88:6817/136629878","cluster_addr":"172.21.15.88:6819/136629878","heartbeat_back_addr":"172.21.15.88:6823/136629878","heartbeat_front_addr":"172.21.15.88:6821/136629878","state":["exists","up"]},{"osd":8,"uuid":"63e59f4b-0426-46db-92f7-c55dae61b5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6817","nonce":1770418957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6819","nonce":1770418957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6823","nonce":1770418957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6821","nonce":1770418957}]},"public_addr":"172.21.15.179:6817/1770418957","cluster_addr":"172.21.15.179:6819/1770418957","heartbeat_back_addr":"172.21.15.179:6823/1770418957","heartbeat_front_addr":"172.21.15.179:6821/1770418957","state":["exists","up"]},{"osd":9,"uuid":"5841bb36-ed6b-45d8-b857-eeab139c44e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6827","nonce":1108500274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6829","nonce":1108500274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6833","nonce":1108500274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6831","nonce":1108500274}]},"public_addr":"172.21.15.86:6827/1108500274","cluster_addr":"172.21.15.86:6829/1108500274","heartbeat_back_addr":"172.21.15.86:6833/1108500274","heartbeat_front_addr":"172.21.15.86:6831/1108500274","state":["exists","up"]},{"osd":10,"uuid":"f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6825","nonce":1294275353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6827","nonce":1294275353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6831","nonce":1294275353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6829","nonce":1294275353}]},"public_addr":"172.21.15.88:6825/1294275353","cluster_addr":"172.21.15.88:6827/1294275353","heartbeat_back_addr":"172.21.15.88:6831/1294275353","heartbeat_front_addr":"172.21.15.88:6829/1294275353","state":["exists","up"]},{"osd":11,"uuid":"e0d4cc76-e7a6-4a6b-909c-95488ccd5a36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6825","nonce":4007070233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6827","nonce":4007070233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6831","nonce":4007070233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6829","nonce":4007070233}]},"public_addr":"172.21.15.179:6825/4007070233","cluster_addr":"172.21.15.179:6827/4007070233","heartbeat_back_addr":"172.21.15.179:6831/4007070233","heartbeat_front_addr":"172.21.15.179:6829/4007070233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:55.210605+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:18.723090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:42.547979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:06.421622+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:30.858315+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:55.455313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.623369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:46.185111+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.103627+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:37.297545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:02.637431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:28.199292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1068089552":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/4273877219":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/2707594358":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/4060044103":"2024-04-22T08:38:46.631621+0000","172.21.15.86:6800/1364156965":"2024-04-22T08:38:46.631621+0000","172.21.15.86:6800/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/1364156965":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/1412393225":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6800/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/1687852404":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2020707404":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2568130933":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/3952607648":"2024-04-22T08:28:57.384403+0000","172.21.15.86:6801/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/2756903426":"2024-04-22T08:29:23.190001+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:38:55.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.753+0000 7f36c07db640 1 -- 172.21.15.86:0/1258657881 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f36a40a3db0 msgr2=0x7f36a40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:55.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.753+0000 7f36c07db640 1 --2- 172.21.15.86:0/1258657881 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f36a40a3db0 0x7f36a40a6270 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7f36a80058c0 tx=0x7f36a8005850 comp rx=0 tx=0).stop 2024-04-21T08:38:55.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.753+0000 7f36c07db640 1 -- 172.21.15.86:0/1258657881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36b81004c0 msgr2=0x7f36b81016a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:55.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.753+0000 7f36c07db640 1 --2- 172.21.15.86:0/1258657881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36b81004c0 0x7f36b81016a0 secure :-1 s=READY pgs=574 cs=0 l=1 rev1=1 crypto rx=0x7f36b4005940 tx=0x7f36b402cd20 comp rx=0 tx=0).stop 2024-04-21T08:38:55.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.753+0000 7f36c07db640 1 -- 172.21.15.86:0/1258657881 shutdown_connections 2024-04-21T08:38:55.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.753+0000 7f36c07db640 1 --2- 172.21.15.86:0/1258657881 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f36a40a3db0 0x7f36a40a6270 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.753+0000 7f36c07db640 1 --2- 172.21.15.86:0/1258657881 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f36b8106f50 0x7f36b8102120 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.753+0000 7f36c07db640 1 --2- 172.21.15.86:0/1258657881 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f36b8100e80 0x7f36b8101be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.753+0000 7f36c07db640 1 --2- 172.21.15.86:0/1258657881 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f36b81004c0 0x7f36b81016a0 unknown :-1 s=CLOSED pgs=574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.753+0000 7f36c07db640 1 -- 172.21.15.86:0/1258657881 >> 172.21.15.86:0/1258657881 conn(0x7f36b8077450 msgr2=0x7f36b8105020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:55.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.753+0000 7f36c07db640 1 -- 172.21.15.86:0/1258657881 shutdown_connections 2024-04-21T08:38:55.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.753+0000 7f36c07db640 1 -- 172.21.15.86:0/1258657881 wait complete. 2024-04-21T08:38:55.765 INFO:tasks.quiescer.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:31:45.279109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:36:18.300782+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:36:19.574636+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:36:22.905632+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:38:55.765 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:38:55.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.867+0000 7f92487bb640 1 -- 172.21.15.86:0/3926013235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f924006a5d0 msgr2=0x7f924006a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:55.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.867+0000 7f92487bb640 1 --2- 172.21.15.86:0/3926013235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f924006a5d0 0x7f924006a9b0 secure :-1 s=READY pgs=575 cs=0 l=1 rev1=1 crypto rx=0x7f9230009900 tx=0x7f923002bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:55.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.867+0000 7f92487bb640 1 -- 172.21.15.86:0/3926013235 shutdown_connections 2024-04-21T08:38:55.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.867+0000 7f92487bb640 1 --2- 172.21.15.86:0/3926013235 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f924010e960 0x7f9240110d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.867+0000 7f92487bb640 1 --2- 172.21.15.86:0/3926013235 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f924006aef0 0x7f924010e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.867+0000 7f92487bb640 1 --2- 172.21.15.86:0/3926013235 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f924006a5d0 0x7f924006a9b0 unknown :-1 s=CLOSED pgs=575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.867+0000 7f92487bb640 1 -- 172.21.15.86:0/3926013235 >> 172.21.15.86:0/3926013235 conn(0x7f9240077eb0 msgr2=0x7f92400782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:55.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.867+0000 7f92487bb640 1 -- 172.21.15.86:0/3926013235 shutdown_connections 2024-04-21T08:38:55.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.867+0000 7f92487bb640 1 -- 172.21.15.86:0/3926013235 wait complete. 2024-04-21T08:38:55.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.868+0000 7f92487bb640 1 Processor -- start 2024-04-21T08:38:55.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.868+0000 7f92487bb640 1 -- start start 2024-04-21T08:38:55.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.868+0000 7f92487bb640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f924006a5d0 0x7f92401a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:55.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.868+0000 7f92487bb640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f924006aef0 0x7f92401a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:55.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.868+0000 7f92487bb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f924010e960 0x7f924019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:55.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.868+0000 7f92487bb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f92401143e0 con 0x7f924010e960 2024-04-21T08:38:55.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.868+0000 7f92487bb640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9240114260 con 0x7f924006aef0 2024-04-21T08:38:55.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.868+0000 7f92487bb640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9240114560 con 0x7f924006a5d0 2024-04-21T08:38:55.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.868+0000 7f9246d58640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f924010e960 0x7f924019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:55.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.868+0000 7f9246d58640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f924010e960 0x7f924019a780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36452/0 (socket says 172.21.15.86:36452) 2024-04-21T08:38:55.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.868+0000 7f9246d58640 1 -- 172.21.15.86:0/2281660912 learned_addr learned my addr 172.21.15.86:0/2281660912 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:55.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.868+0000 7f9246557640 1 --2- 172.21.15.86:0/2281660912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f924006a5d0 0x7f92401a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:55.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.868+0000 7f9245d56640 1 --2- 172.21.15.86:0/2281660912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f924006aef0 0x7f92401a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:55.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.868+0000 7f9246d58640 1 -- 172.21.15.86:0/2281660912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f924006a5d0 msgr2=0x7f92401a0610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:55.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.868+0000 7f9246d58640 1 --2- 172.21.15.86:0/2281660912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f924006a5d0 0x7f92401a0610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.868+0000 7f9246d58640 1 -- 172.21.15.86:0/2281660912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f924006aef0 msgr2=0x7f92401a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:55.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.868+0000 7f9246d58640 1 --2- 172.21.15.86:0/2281660912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f924006aef0 0x7f92401a0b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:55.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.868+0000 7f9246d58640 1 -- 172.21.15.86:0/2281660912 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f924019af50 con 0x7f924010e960 2024-04-21T08:38:55.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.868+0000 7f9245d56640 1 --2- 172.21.15.86:0/2281660912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f924006aef0 0x7f92401a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:55.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.868+0000 7f9246d58640 1 --2- 172.21.15.86:0/2281660912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f924010e960 0x7f924019a780 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f923800eb40 tx=0x7f923800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:55.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.869+0000 7f92377fe640 1 -- 172.21.15.86:0/2281660912 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9238004b20 con 0x7f924010e960 2024-04-21T08:38:55.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.869+0000 7f92487bb640 1 -- 172.21.15.86:0/2281660912 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f924019b240 con 0x7f924010e960 2024-04-21T08:38:55.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.869+0000 7f92487bb640 1 -- 172.21.15.86:0/2281660912 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f92401a7460 con 0x7f924010e960 2024-04-21T08:38:55.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.869+0000 7f92377fe640 1 -- 172.21.15.86:0/2281660912 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9238004cc0 con 0x7f924010e960 2024-04-21T08:38:55.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.869+0000 7f92377fe640 1 -- 172.21.15.86:0/2281660912 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9238010690 con 0x7f924010e960 2024-04-21T08:38:55.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.870+0000 7f92487bb640 1 -- 172.21.15.86:0/2281660912 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9224005290 con 0x7f924010e960 2024-04-21T08:38:55.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.871+0000 7f92377fe640 1 -- 172.21.15.86:0/2281660912 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f9238010830 con 0x7f924010e960 2024-04-21T08:38:55.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.871+0000 7f92377fe640 1 --2- 172.21.15.86:0/2281660912 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f922c0a3db0 0x7f922c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:55.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.871+0000 7f9246557640 1 --2- 172.21.15.86:0/2281660912 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f922c0a3db0 0x7f922c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:55.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.871+0000 7f92377fe640 1 -- 172.21.15.86:0/2281660912 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f92380d1590 con 0x7f924010e960 2024-04-21T08:38:55.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.872+0000 7f9246557640 1 --2- 172.21.15.86:0/2281660912 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f922c0a3db0 0x7f922c0a6270 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9230006010 tx=0x7f9230005e40 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:55.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:55.875+0000 7f92377fe640 1 -- 172.21.15.86:0/2281660912 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f923809d750 con 0x7f924010e960 2024-04-21T08:38:56.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.050+0000 7f92487bb640 1 -- 172.21.15.86:0/2281660912 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f9224005850 con 0x7f924010e960 2024-04-21T08:38:56.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.051+0000 7f92377fe640 1 -- 172.21.15.86:0/2281660912 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7f923809d050 con 0x7f924010e960 2024-04-21T08:38:56.051 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-21T08:38:56.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.053+0000 7f92487bb640 1 -- 172.21.15.86:0/2281660912 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f922c0a3db0 msgr2=0x7f922c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:56.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.053+0000 7f92487bb640 1 --2- 172.21.15.86:0/2281660912 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f922c0a3db0 0x7f922c0a6270 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7f9230006010 tx=0x7f9230005e40 comp rx=0 tx=0).stop 2024-04-21T08:38:56.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.053+0000 7f92487bb640 1 -- 172.21.15.86:0/2281660912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f924010e960 msgr2=0x7f924019a780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:56.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.053+0000 7f92487bb640 1 --2- 172.21.15.86:0/2281660912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f924010e960 0x7f924019a780 secure :-1 s=READY pgs=576 cs=0 l=1 rev1=1 crypto rx=0x7f923800eb40 tx=0x7f923800c2d0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.053+0000 7f92487bb640 1 -- 172.21.15.86:0/2281660912 shutdown_connections 2024-04-21T08:38:56.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.053+0000 7f92487bb640 1 --2- 172.21.15.86:0/2281660912 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f922c0a3db0 0x7f922c0a6270 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.053+0000 7f92487bb640 1 --2- 172.21.15.86:0/2281660912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f924010e960 0x7f924019a780 unknown :-1 s=CLOSED pgs=576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.053+0000 7f92487bb640 1 --2- 172.21.15.86:0/2281660912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f924006aef0 0x7f92401a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.053+0000 7f92487bb640 1 --2- 172.21.15.86:0/2281660912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f924006a5d0 0x7f92401a0610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.053+0000 7f92487bb640 1 -- 172.21.15.86:0/2281660912 >> 172.21.15.86:0/2281660912 conn(0x7f9240077eb0 msgr2=0x7f924010fed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:56.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.053+0000 7f92487bb640 1 -- 172.21.15.86:0/2281660912 shutdown_connections 2024-04-21T08:38:56.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.053+0000 7f92487bb640 1 -- 172.21.15.86:0/2281660912 wait complete. 2024-04-21T08:38:56.065 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:38:56.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.165+0000 7f476fc48640 1 -- 172.21.15.86:0/4175781269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4768101a50 msgr2=0x7f4768105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:56.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.165+0000 7f476fc48640 1 --2- 172.21.15.86:0/4175781269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4768101a50 0x7f4768105a90 secure :-1 s=READY pgs=577 cs=0 l=1 rev1=1 crypto rx=0x7f4764009900 tx=0x7f476402bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:56.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.165+0000 7f476fc48640 1 -- 172.21.15.86:0/4175781269 shutdown_connections 2024-04-21T08:38:56.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.165+0000 7f476fc48640 1 --2- 172.21.15.86:0/4175781269 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4768106170 0x7f476810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.165+0000 7f476fc48640 1 --2- 172.21.15.86:0/4175781269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4768101a50 0x7f4768105a90 unknown :-1 s=CLOSED pgs=577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.165+0000 7f476fc48640 1 --2- 172.21.15.86:0/4175781269 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47681010a0 0x7f4768101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.165+0000 7f476fc48640 1 -- 172.21.15.86:0/4175781269 >> 172.21.15.86:0/4175781269 conn(0x7f4768077eb0 msgr2=0x7f47680782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.165+0000 7f476fc48640 1 -- 172.21.15.86:0/4175781269 shutdown_connections 2024-04-21T08:38:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.165+0000 7f476fc48640 1 -- 172.21.15.86:0/4175781269 wait complete. 2024-04-21T08:38:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.166+0000 7f476fc48640 1 Processor -- start 2024-04-21T08:38:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.166+0000 7f476fc48640 1 -- start start 2024-04-21T08:38:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.166+0000 7f476fc48640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47681010a0 0x7f47681986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.166+0000 7f476fc48640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4768106170 0x7f4768198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.166+0000 7f476fc48640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f476819cf90 0x7f476819d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.166+0000 7f476fc48640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47681121f0 con 0x7f47681010a0 2024-04-21T08:38:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.166+0000 7f476fc48640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4768112070 con 0x7f4768106170 2024-04-21T08:38:56.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.166+0000 7f476fc48640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4768112370 con 0x7f476819cf90 2024-04-21T08:38:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.166+0000 7f476d9e4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47681010a0 0x7f47681986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.166+0000 7f476d9e4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47681010a0 0x7f47681986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36482/0 (socket says 172.21.15.86:36482) 2024-04-21T08:38:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.166+0000 7f476d9e4640 1 -- 172.21.15.86:0/335669794 learned_addr learned my addr 172.21.15.86:0/335669794 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.166+0000 7f476d1e3640 1 --2- 172.21.15.86:0/335669794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4768106170 0x7f4768198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.166+0000 7f476e1e5640 1 --2- 172.21.15.86:0/335669794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f476819cf90 0x7f476819d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.166+0000 7f476d9e4640 1 -- 172.21.15.86:0/335669794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f476819cf90 msgr2=0x7f476819d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.166+0000 7f476d9e4640 1 --2- 172.21.15.86:0/335669794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f476819cf90 0x7f476819d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.166+0000 7f476d9e4640 1 -- 172.21.15.86:0/335669794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4768106170 msgr2=0x7f4768198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.166+0000 7f476d9e4640 1 --2- 172.21.15.86:0/335669794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4768106170 0x7f4768198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.166+0000 7f476d9e4640 1 -- 172.21.15.86:0/335669794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f476819da80 con 0x7f47681010a0 2024-04-21T08:38:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.166+0000 7f476d1e3640 1 --2- 172.21.15.86:0/335669794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4768106170 0x7f4768198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:56.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.166+0000 7f476e1e5640 1 --2- 172.21.15.86:0/335669794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f476819cf90 0x7f476819d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:56.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.167+0000 7f476d9e4640 1 --2- 172.21.15.86:0/335669794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47681010a0 0x7f47681986d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f475800c920 tx=0x7f475800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:56.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.167+0000 7f475effd640 1 -- 172.21.15.86:0/335669794 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4758007c70 con 0x7f47681010a0 2024-04-21T08:38:56.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.167+0000 7f476fc48640 1 -- 172.21.15.86:0/335669794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f476819dd70 con 0x7f47681010a0 2024-04-21T08:38:56.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.167+0000 7f475effd640 1 -- 172.21.15.86:0/335669794 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4758007e10 con 0x7f47681010a0 2024-04-21T08:38:56.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.167+0000 7f475effd640 1 -- 172.21.15.86:0/335669794 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4758018820 con 0x7f47681010a0 2024-04-21T08:38:56.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.167+0000 7f476fc48640 1 -- 172.21.15.86:0/335669794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47681a55b0 con 0x7f47681010a0 2024-04-21T08:38:56.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.168+0000 7f476fc48640 1 -- 172.21.15.86:0/335669794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47680691c0 con 0x7f47681010a0 2024-04-21T08:38:56.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.169+0000 7f475effd640 1 -- 172.21.15.86:0/335669794 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f475800f450 con 0x7f47681010a0 2024-04-21T08:38:56.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.170+0000 7f475effd640 1 --2- 172.21.15.86:0/335669794 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f47540a3db0 0x7f47540a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:56.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.170+0000 7f476d1e3640 1 --2- 172.21.15.86:0/335669794 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f47540a3db0 0x7f47540a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:56.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.170+0000 7f475effd640 1 -- 172.21.15.86:0/335669794 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f47580d1180 con 0x7f47681010a0 2024-04-21T08:38:56.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.170+0000 7f476d1e3640 1 --2- 172.21.15.86:0/335669794 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f47540a3db0 0x7f47540a6270 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4764036040 tx=0x7f4764000bc0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:56.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.174+0000 7f475effd640 1 -- 172.21.15.86:0/335669794 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4758014030 con 0x7f47681010a0 2024-04-21T08:38:56.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.353+0000 7f476fc48640 1 -- 172.21.15.86:0/335669794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f4768101a50 con 0x7f47681010a0 2024-04-21T08:38:56.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.353+0000 7f475effd640 1 -- 172.21.15.86:0/335669794 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f475809d3d0 con 0x7f47681010a0 2024-04-21T08:38:56.354 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-21T08:38:56.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.355+0000 7f476fc48640 1 -- 172.21.15.86:0/335669794 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f47540a3db0 msgr2=0x7f47540a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:56.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.355+0000 7f476fc48640 1 --2- 172.21.15.86:0/335669794 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f47540a3db0 0x7f47540a6270 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7f4764036040 tx=0x7f4764000bc0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.355+0000 7f476fc48640 1 -- 172.21.15.86:0/335669794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47681010a0 msgr2=0x7f47681986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:56.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.355+0000 7f476fc48640 1 --2- 172.21.15.86:0/335669794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47681010a0 0x7f47681986d0 secure :-1 s=READY pgs=578 cs=0 l=1 rev1=1 crypto rx=0x7f475800c920 tx=0x7f475800cde0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.355+0000 7f476fc48640 1 -- 172.21.15.86:0/335669794 shutdown_connections 2024-04-21T08:38:56.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.355+0000 7f476fc48640 1 --2- 172.21.15.86:0/335669794 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f47540a3db0 0x7f47540a6270 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.355+0000 7f476fc48640 1 --2- 172.21.15.86:0/335669794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f476819cf90 0x7f476819d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.355+0000 7f476fc48640 1 --2- 172.21.15.86:0/335669794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4768106170 0x7f4768198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.355+0000 7f476fc48640 1 --2- 172.21.15.86:0/335669794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47681010a0 0x7f47681986d0 unknown :-1 s=CLOSED pgs=578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.355+0000 7f476fc48640 1 -- 172.21.15.86:0/335669794 >> 172.21.15.86:0/335669794 conn(0x7f4768077eb0 msgr2=0x7f47680fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:56.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.355+0000 7f476fc48640 1 -- 172.21.15.86:0/335669794 shutdown_connections 2024-04-21T08:38:56.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.355+0000 7f476fc48640 1 -- 172.21.15.86:0/335669794 wait complete. 2024-04-21T08:38:56.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:56 smithi086 ceph-mon[27939]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 0 B/s wr, 24 op/s 2024-04-21T08:38:56.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:56 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1709866020' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:56.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:56 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1258657881' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:56.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:56 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2281660912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:56.367 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:38:56.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.469+0000 7f66e877e640 1 -- 172.21.15.86:0/1919202533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66e0101a40 msgr2=0x7f66e0105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:56.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.469+0000 7f66e877e640 1 --2- 172.21.15.86:0/1919202533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66e0101a40 0x7f66e0105a80 secure :-1 s=READY pgs=579 cs=0 l=1 rev1=1 crypto rx=0x7f66d0009900 tx=0x7f66d002bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:56.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.469+0000 7f66e877e640 1 -- 172.21.15.86:0/1919202533 shutdown_connections 2024-04-21T08:38:56.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.469+0000 7f66e877e640 1 --2- 172.21.15.86:0/1919202533 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66e0106160 0x7f66e010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.469+0000 7f66e877e640 1 --2- 172.21.15.86:0/1919202533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66e0101a40 0x7f66e0105a80 unknown :-1 s=CLOSED pgs=579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.469+0000 7f66e877e640 1 --2- 172.21.15.86:0/1919202533 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66e0101090 0x7f66e0101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.469+0000 7f66e877e640 1 -- 172.21.15.86:0/1919202533 >> 172.21.15.86:0/1919202533 conn(0x7f66e0077f50 msgr2=0x7f66e0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:56.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.469+0000 7f66e877e640 1 -- 172.21.15.86:0/1919202533 shutdown_connections 2024-04-21T08:38:56.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.469+0000 7f66e877e640 1 -- 172.21.15.86:0/1919202533 wait complete. 2024-04-21T08:38:56.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.469+0000 7f66e877e640 1 Processor -- start 2024-04-21T08:38:56.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.469+0000 7f66e877e640 1 -- start start 2024-04-21T08:38:56.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.469+0000 7f66e877e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66e0101090 0x7f66e01995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:56.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.469+0000 7f66e877e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66e0106160 0x7f66e0199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:56.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.469+0000 7f66e877e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66e019a090 0x7f66e01a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:56.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.469+0000 7f66e877e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f66e01110a0 con 0x7f66e0101090 2024-04-21T08:38:56.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.469+0000 7f66e877e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f66e0110f20 con 0x7f66e0106160 2024-04-21T08:38:56.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.469+0000 7f66e877e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f66e0111220 con 0x7f66e019a090 2024-04-21T08:38:56.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.470+0000 7f66e651a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66e0101090 0x7f66e01995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:56.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.470+0000 7f66e651a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66e0101090 0x7f66e01995c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36500/0 (socket says 172.21.15.86:36500) 2024-04-21T08:38:56.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.470+0000 7f66e651a640 1 -- 172.21.15.86:0/2229459377 learned_addr learned my addr 172.21.15.86:0/2229459377 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:56.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.470+0000 7f66e5d19640 1 --2- 172.21.15.86:0/2229459377 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66e0106160 0x7f66e0199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:56.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.470+0000 7f66e651a640 1 -- 172.21.15.86:0/2229459377 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66e019a090 msgr2=0x7f66e01a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:56.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.470+0000 7f66e6d1b640 1 --2- 172.21.15.86:0/2229459377 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66e019a090 0x7f66e01a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:56.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.470+0000 7f66e651a640 1 --2- 172.21.15.86:0/2229459377 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66e019a090 0x7f66e01a4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.470+0000 7f66e651a640 1 -- 172.21.15.86:0/2229459377 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66e0106160 msgr2=0x7f66e0199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:56.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.470+0000 7f66e651a640 1 --2- 172.21.15.86:0/2229459377 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66e0106160 0x7f66e0199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.470+0000 7f66e651a640 1 -- 172.21.15.86:0/2229459377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66e01a4e10 con 0x7f66e0101090 2024-04-21T08:38:56.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.470+0000 7f66e5d19640 1 --2- 172.21.15.86:0/2229459377 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66e0106160 0x7f66e0199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:56.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.470+0000 7f66e651a640 1 --2- 172.21.15.86:0/2229459377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66e0101090 0x7f66e01995c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f66dc00c970 tx=0x7f66dc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:56.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.470+0000 7f66d77fe640 1 -- 172.21.15.86:0/2229459377 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66dc007bb0 con 0x7f66e0101090 2024-04-21T08:38:56.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.470+0000 7f66d77fe640 1 -- 172.21.15.86:0/2229459377 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66dc007d50 con 0x7f66e0101090 2024-04-21T08:38:56.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.470+0000 7f66e877e640 1 -- 172.21.15.86:0/2229459377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66e01a5100 con 0x7f66e0101090 2024-04-21T08:38:56.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.470+0000 7f66d77fe640 1 -- 172.21.15.86:0/2229459377 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66dc00f450 con 0x7f66e0101090 2024-04-21T08:38:56.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.470+0000 7f66e877e640 1 -- 172.21.15.86:0/2229459377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66e01a5620 con 0x7f66e0101090 2024-04-21T08:38:56.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.471+0000 7f66e877e640 1 -- 172.21.15.86:0/2229459377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f66e0068d70 con 0x7f66e0101090 2024-04-21T08:38:56.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.472+0000 7f66d77fe640 1 -- 172.21.15.86:0/2229459377 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f66dc004f20 con 0x7f66e0101090 2024-04-21T08:38:56.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.473+0000 7f66d77fe640 1 --2- 172.21.15.86:0/2229459377 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f66cc0a3e30 0x7f66cc0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:56.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.473+0000 7f66e5d19640 1 --2- 172.21.15.86:0/2229459377 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f66cc0a3e30 0x7f66cc0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:56.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.473+0000 7f66d77fe640 1 -- 172.21.15.86:0/2229459377 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f66dc0d0ab0 con 0x7f66e0101090 2024-04-21T08:38:56.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.473+0000 7f66e5d19640 1 --2- 172.21.15.86:0/2229459377 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f66cc0a3e30 0x7f66cc0a62f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f66d0006010 tx=0x7f66d0005a20 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:56.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.477+0000 7f66d77fe640 1 -- 172.21.15.86:0/2229459377 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f66dc014030 con 0x7f66e0101090 2024-04-21T08:38:56.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.655+0000 7f66e877e640 1 -- 172.21.15.86:0/2229459377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f66e0109390 con 0x7f66e0101090 2024-04-21T08:38:56.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.656+0000 7f66d77fe640 1 -- 172.21.15.86:0/2229459377 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f66dc09cc70 con 0x7f66e0101090 2024-04-21T08:38:56.657 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-21T08:38:56.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.658+0000 7f66e877e640 1 -- 172.21.15.86:0/2229459377 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f66cc0a3e30 msgr2=0x7f66cc0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:56.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.658+0000 7f66e877e640 1 --2- 172.21.15.86:0/2229459377 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f66cc0a3e30 0x7f66cc0a62f0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f66d0006010 tx=0x7f66d0005a20 comp rx=0 tx=0).stop 2024-04-21T08:38:56.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.658+0000 7f66e877e640 1 -- 172.21.15.86:0/2229459377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66e0101090 msgr2=0x7f66e01995c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:56.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.658+0000 7f66e877e640 1 --2- 172.21.15.86:0/2229459377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66e0101090 0x7f66e01995c0 secure :-1 s=READY pgs=580 cs=0 l=1 rev1=1 crypto rx=0x7f66dc00c970 tx=0x7f66dc00ce30 comp rx=0 tx=0).stop 2024-04-21T08:38:56.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.658+0000 7f66e877e640 1 -- 172.21.15.86:0/2229459377 shutdown_connections 2024-04-21T08:38:56.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.658+0000 7f66e877e640 1 --2- 172.21.15.86:0/2229459377 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f66cc0a3e30 0x7f66cc0a62f0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.658+0000 7f66e877e640 1 --2- 172.21.15.86:0/2229459377 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66e019a090 0x7f66e01a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.658+0000 7f66e877e640 1 --2- 172.21.15.86:0/2229459377 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66e0106160 0x7f66e0199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.658+0000 7f66e877e640 1 --2- 172.21.15.86:0/2229459377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66e0101090 0x7f66e01995c0 unknown :-1 s=CLOSED pgs=580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.658+0000 7f66e877e640 1 -- 172.21.15.86:0/2229459377 >> 172.21.15.86:0/2229459377 conn(0x7f66e0077f50 msgr2=0x7f66e00ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:56.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.658+0000 7f66e877e640 1 -- 172.21.15.86:0/2229459377 shutdown_connections 2024-04-21T08:38:56.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.658+0000 7f66e877e640 1 -- 172.21.15.86:0/2229459377 wait complete. 2024-04-21T08:38:56.669 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:38:56.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:56 smithi088 ceph-mon[31259]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 0 B/s wr, 24 op/s 2024-04-21T08:38:56.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:56 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1709866020' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:56.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:56 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1258657881' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:56.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:56 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2281660912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:56.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:56 smithi179 ceph-mon[31829]: pgmap v7: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 0 B/s wr, 24 op/s 2024-04-21T08:38:56.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:56 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1709866020' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:56.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:56 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1258657881' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:56.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:56 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2281660912' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:56.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.769+0000 7ffb45ce0640 1 -- 172.21.15.86:0/2580115124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb401010a0 msgr2=0x7ffb40101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.769+0000 7ffb45ce0640 1 --2- 172.21.15.86:0/2580115124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb401010a0 0x7ffb40101480 secure :-1 s=READY pgs=581 cs=0 l=1 rev1=1 crypto rx=0x7ffb30009900 tx=0x7ffb3002bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.769+0000 7ffb45ce0640 1 -- 172.21.15.86:0/2580115124 shutdown_connections 2024-04-21T08:38:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.769+0000 7ffb45ce0640 1 --2- 172.21.15.86:0/2580115124 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb40106170 0x7ffb4010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.769+0000 7ffb45ce0640 1 --2- 172.21.15.86:0/2580115124 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb40101a50 0x7ffb40105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.769+0000 7ffb45ce0640 1 --2- 172.21.15.86:0/2580115124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb401010a0 0x7ffb40101480 unknown :-1 s=CLOSED pgs=581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.769+0000 7ffb45ce0640 1 -- 172.21.15.86:0/2580115124 >> 172.21.15.86:0/2580115124 conn(0x7ffb40077eb0 msgr2=0x7ffb400782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.769+0000 7ffb45ce0640 1 -- 172.21.15.86:0/2580115124 shutdown_connections 2024-04-21T08:38:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.769+0000 7ffb45ce0640 1 -- 172.21.15.86:0/2580115124 wait complete. 2024-04-21T08:38:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.770+0000 7ffb45ce0640 1 Processor -- start 2024-04-21T08:38:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.770+0000 7ffb45ce0640 1 -- start start 2024-04-21T08:38:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.770+0000 7ffb45ce0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb401010a0 0x7ffb401975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.770+0000 7ffb45ce0640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb40106170 0x7ffb40197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:56.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.770+0000 7ffb45ce0640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb4019bea0 0x7ffb4019c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:56.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.770+0000 7ffb45ce0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffb401110e0 con 0x7ffb401010a0 2024-04-21T08:38:56.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.770+0000 7ffb45ce0640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ffb40110f60 con 0x7ffb4019bea0 2024-04-21T08:38:56.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.770+0000 7ffb45ce0640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ffb40111260 con 0x7ffb40106170 2024-04-21T08:38:56.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.770+0000 7ffb3f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb401010a0 0x7ffb401975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:56.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.770+0000 7ffb3f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb401010a0 0x7ffb401975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36512/0 (socket says 172.21.15.86:36512) 2024-04-21T08:38:56.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.770+0000 7ffb3f7fe640 1 -- 172.21.15.86:0/126889366 learned_addr learned my addr 172.21.15.86:0/126889366 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:56.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.770+0000 7ffb3ffff640 1 --2- 172.21.15.86:0/126889366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb4019bea0 0x7ffb4019c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:56.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.770+0000 7ffb3effd640 1 --2- 172.21.15.86:0/126889366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb40106170 0x7ffb40197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:56.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.771+0000 7ffb3f7fe640 1 -- 172.21.15.86:0/126889366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb40106170 msgr2=0x7ffb40197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:56.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.771+0000 7ffb3f7fe640 1 --2- 172.21.15.86:0/126889366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb40106170 0x7ffb40197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.771+0000 7ffb3f7fe640 1 -- 172.21.15.86:0/126889366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb4019bea0 msgr2=0x7ffb4019c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:56.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.771+0000 7ffb3f7fe640 1 --2- 172.21.15.86:0/126889366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb4019bea0 0x7ffb4019c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.771+0000 7ffb3f7fe640 1 -- 172.21.15.86:0/126889366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb4019c990 con 0x7ffb401010a0 2024-04-21T08:38:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.771+0000 7ffb3f7fe640 1 --2- 172.21.15.86:0/126889366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb401010a0 0x7ffb401975e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ffb30005940 tx=0x7ffb3002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.771+0000 7ffb3cff9640 1 -- 172.21.15.86:0/126889366 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb30043020 con 0x7ffb401010a0 2024-04-21T08:38:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.771+0000 7ffb3cff9640 1 -- 172.21.15.86:0/126889366 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb3002cbe0 con 0x7ffb401010a0 2024-04-21T08:38:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.771+0000 7ffb45ce0640 1 -- 172.21.15.86:0/126889366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb4019cc20 con 0x7ffb401010a0 2024-04-21T08:38:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.771+0000 7ffb3cff9640 1 -- 172.21.15.86:0/126889366 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb30003bf0 con 0x7ffb401010a0 2024-04-21T08:38:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.771+0000 7ffb45ce0640 1 -- 172.21.15.86:0/126889366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb401a4440 con 0x7ffb401010a0 2024-04-21T08:38:56.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.772+0000 7ffb45ce0640 1 -- 172.21.15.86:0/126889366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffb400691c0 con 0x7ffb401010a0 2024-04-21T08:38:56.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.773+0000 7ffb3cff9640 1 -- 172.21.15.86:0/126889366 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7ffb30046020 con 0x7ffb401010a0 2024-04-21T08:38:56.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.774+0000 7ffb3cff9640 1 --2- 172.21.15.86:0/126889366 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ffb280a3db0 0x7ffb280a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:56.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.774+0000 7ffb3effd640 1 --2- 172.21.15.86:0/126889366 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ffb280a3db0 0x7ffb280a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:56.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.774+0000 7ffb3cff9640 1 -- 172.21.15.86:0/126889366 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7ffb300f2dd0 con 0x7ffb401010a0 2024-04-21T08:38:56.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.774+0000 7ffb3effd640 1 --2- 172.21.15.86:0/126889366 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ffb280a3db0 0x7ffb280a6270 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffb40100990 tx=0x7ffb3400b390 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:56.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.778+0000 7ffb3cff9640 1 -- 172.21.15.86:0/126889366 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffb300f6030 con 0x7ffb401010a0 2024-04-21T08:38:56.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.956+0000 7ffb45ce0640 1 -- 172.21.15.86:0/126889366 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7ffb40101a50 con 0x7ffb401010a0 2024-04-21T08:38:56.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.956+0000 7ffb3cff9640 1 -- 172.21.15.86:0/126889366 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7ffb3004d020 con 0x7ffb401010a0 2024-04-21T08:38:56.957 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-21T08:38:56.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.958+0000 7ffb45ce0640 1 -- 172.21.15.86:0/126889366 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ffb280a3db0 msgr2=0x7ffb280a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:56.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.958+0000 7ffb45ce0640 1 --2- 172.21.15.86:0/126889366 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ffb280a3db0 0x7ffb280a6270 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7ffb40100990 tx=0x7ffb3400b390 comp rx=0 tx=0).stop 2024-04-21T08:38:56.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.958+0000 7ffb45ce0640 1 -- 172.21.15.86:0/126889366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb401010a0 msgr2=0x7ffb401975e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:56.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.958+0000 7ffb45ce0640 1 --2- 172.21.15.86:0/126889366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb401010a0 0x7ffb401975e0 secure :-1 s=READY pgs=582 cs=0 l=1 rev1=1 crypto rx=0x7ffb30005940 tx=0x7ffb3002c7e0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.958+0000 7ffb45ce0640 1 -- 172.21.15.86:0/126889366 shutdown_connections 2024-04-21T08:38:56.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.958+0000 7ffb45ce0640 1 --2- 172.21.15.86:0/126889366 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ffb280a3db0 0x7ffb280a6270 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.958+0000 7ffb45ce0640 1 --2- 172.21.15.86:0/126889366 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb4019bea0 0x7ffb4019c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.958+0000 7ffb45ce0640 1 --2- 172.21.15.86:0/126889366 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb40106170 0x7ffb40197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.958+0000 7ffb45ce0640 1 --2- 172.21.15.86:0/126889366 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb401010a0 0x7ffb401975e0 unknown :-1 s=CLOSED pgs=582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:56.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.958+0000 7ffb45ce0640 1 -- 172.21.15.86:0/126889366 >> 172.21.15.86:0/126889366 conn(0x7ffb40077eb0 msgr2=0x7ffb400fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:56.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.959+0000 7ffb45ce0640 1 -- 172.21.15.86:0/126889366 shutdown_connections 2024-04-21T08:38:56.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:56.959+0000 7ffb45ce0640 1 -- 172.21.15.86:0/126889366 wait complete. 2024-04-21T08:38:56.970 INFO:tasks.quiescer.ceph_manager:waiting for clean 2024-04-21T08:38:56.970 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-21T08:38:57.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.068+0000 7f4203dd5640 1 -- 172.21.15.86:0/3754052581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41fc102190 msgr2=0x7f41fc102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:57.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.068+0000 7f4203dd5640 1 --2- 172.21.15.86:0/3754052581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41fc102190 0x7f41fc102570 secure :-1 s=READY pgs=583 cs=0 l=1 rev1=1 crypto rx=0x7f41ec009900 tx=0x7f41ec02bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:57.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.068+0000 7f4203dd5640 1 -- 172.21.15.86:0/3754052581 shutdown_connections 2024-04-21T08:38:57.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.068+0000 7f4203dd5640 1 --2- 172.21.15.86:0/3754052581 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41fc107260 0x7f41fc10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.068+0000 7f4203dd5640 1 --2- 172.21.15.86:0/3754052581 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41fc102b40 0x7f41fc106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.068+0000 7f4203dd5640 1 --2- 172.21.15.86:0/3754052581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41fc102190 0x7f41fc102570 unknown :-1 s=CLOSED pgs=583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.068+0000 7f4203dd5640 1 -- 172.21.15.86:0/3754052581 >> 172.21.15.86:0/3754052581 conn(0x7f41fc0793e0 msgr2=0x7f41fc0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:57.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.068+0000 7f4203dd5640 1 -- 172.21.15.86:0/3754052581 shutdown_connections 2024-04-21T08:38:57.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.068+0000 7f4203dd5640 1 -- 172.21.15.86:0/3754052581 wait complete. 2024-04-21T08:38:57.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.068+0000 7f4203dd5640 1 Processor -- start 2024-04-21T08:38:57.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.068+0000 7f4203dd5640 1 -- start start 2024-04-21T08:38:57.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.069+0000 7f4203dd5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41fc102190 0x7f41fc198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:57.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.069+0000 7f4203dd5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41fc107260 0x7f41fc198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:57.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.069+0000 7f4203dd5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41fc19d020 0x7f41fc19d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:57.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.069+0000 7f4203dd5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc1121c0 con 0x7f41fc19d020 2024-04-21T08:38:57.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.069+0000 7f4203dd5640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc112040 con 0x7f41fc102190 2024-04-21T08:38:57.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.069+0000 7f4203dd5640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f41fc112340 con 0x7f41fc107260 2024-04-21T08:38:57.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.069+0000 7f4202372640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41fc19d020 0x7f41fc19d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:57.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.069+0000 7f4202372640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41fc19d020 0x7f41fc19d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36530/0 (socket says 172.21.15.86:36530) 2024-04-21T08:38:57.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.069+0000 7f4202372640 1 -- 172.21.15.86:0/1437785926 learned_addr learned my addr 172.21.15.86:0/1437785926 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:57.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.069+0000 7f4201370640 1 --2- 172.21.15.86:0/1437785926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41fc107260 0x7f41fc198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:57.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.069+0000 7f4201b71640 1 --2- 172.21.15.86:0/1437785926 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41fc102190 0x7f41fc198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:57.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.069+0000 7f4202372640 1 -- 172.21.15.86:0/1437785926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41fc107260 msgr2=0x7f41fc198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:57.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.069+0000 7f4202372640 1 --2- 172.21.15.86:0/1437785926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41fc107260 0x7f41fc198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.069+0000 7f4202372640 1 -- 172.21.15.86:0/1437785926 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41fc102190 msgr2=0x7f41fc198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:57.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.069+0000 7f4202372640 1 --2- 172.21.15.86:0/1437785926 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41fc102190 0x7f41fc198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.069+0000 7f4202372640 1 -- 172.21.15.86:0/1437785926 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f41fc19db10 con 0x7f41fc19d020 2024-04-21T08:38:57.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.069+0000 7f4202372640 1 --2- 172.21.15.86:0/1437785926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41fc19d020 0x7f41fc19d4d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f41f400cad0 tx=0x7f41f400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:57.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.069+0000 7f41f2ffd640 1 -- 172.21.15.86:0/1437785926 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f400ae20 con 0x7f41fc19d020 2024-04-21T08:38:57.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.069+0000 7f4203dd5640 1 -- 172.21.15.86:0/1437785926 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f41fc19de00 con 0x7f41fc19d020 2024-04-21T08:38:57.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.069+0000 7f41f2ffd640 1 -- 172.21.15.86:0/1437785926 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f41f4005020 con 0x7f41fc19d020 2024-04-21T08:38:57.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.069+0000 7f41f2ffd640 1 -- 172.21.15.86:0/1437785926 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f41f400f6b0 con 0x7f41fc19d020 2024-04-21T08:38:57.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.069+0000 7f4203dd5640 1 -- 172.21.15.86:0/1437785926 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f41fc1a55a0 con 0x7f41fc19d020 2024-04-21T08:38:57.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.070+0000 7f4203dd5640 1 -- 172.21.15.86:0/1437785926 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f41fc069230 con 0x7f41fc19d020 2024-04-21T08:38:57.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.072+0000 7f41f2ffd640 1 -- 172.21.15.86:0/1437785926 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f41f40051d0 con 0x7f41fc19d020 2024-04-21T08:38:57.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.072+0000 7f41f2ffd640 1 --2- 172.21.15.86:0/1437785926 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f41e80a3db0 0x7f41e80a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:57.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.072+0000 7f4201b71640 1 --2- 172.21.15.86:0/1437785926 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f41e80a3db0 0x7f41e80a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:57.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.072+0000 7f41f2ffd640 1 -- 172.21.15.86:0/1437785926 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f41f40d10c0 con 0x7f41fc19d020 2024-04-21T08:38:57.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.073+0000 7f4201b71640 1 --2- 172.21.15.86:0/1437785926 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f41e80a3db0 0x7f41e80a6270 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f41ec000c00 tx=0x7f41ec004120 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:57.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.076+0000 7f41f2ffd640 1 -- 172.21.15.86:0/1437785926 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f41f4014030 con 0x7f41fc19d020 2024-04-21T08:38:57.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.248+0000 7f4203dd5640 1 -- 172.21.15.86:0/1437785926 --> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f41fc102b40 con 0x7f41e80a3db0 2024-04-21T08:38:57.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.258+0000 7f41f2ffd640 1 -- 172.21.15.86:0/1437785926 <== mgr.15558 v2:172.21.15.86:6800/2452032403 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339446 (secure 0 0 0) 0x7f41fc102b40 con 0x7f41e80a3db0 2024-04-21T08:38:57.262 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:38:57.263 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-21T08:38:57.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.261+0000 7f4203dd5640 1 -- 172.21.15.86:0/1437785926 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f41e80a3db0 msgr2=0x7f41e80a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:57.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.261+0000 7f4203dd5640 1 --2- 172.21.15.86:0/1437785926 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f41e80a3db0 0x7f41e80a6270 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f41ec000c00 tx=0x7f41ec004120 comp rx=0 tx=0).stop 2024-04-21T08:38:57.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.261+0000 7f4203dd5640 1 -- 172.21.15.86:0/1437785926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41fc19d020 msgr2=0x7f41fc19d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:57.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.261+0000 7f4203dd5640 1 --2- 172.21.15.86:0/1437785926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41fc19d020 0x7f41fc19d4d0 secure :-1 s=READY pgs=584 cs=0 l=1 rev1=1 crypto rx=0x7f41f400cad0 tx=0x7f41f400cf90 comp rx=0 tx=0).stop 2024-04-21T08:38:57.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.261+0000 7f4203dd5640 1 -- 172.21.15.86:0/1437785926 shutdown_connections 2024-04-21T08:38:57.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.261+0000 7f4203dd5640 1 --2- 172.21.15.86:0/1437785926 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f41e80a3db0 0x7f41e80a6270 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.261+0000 7f4203dd5640 1 --2- 172.21.15.86:0/1437785926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f41fc19d020 0x7f41fc19d4d0 unknown :-1 s=CLOSED pgs=584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.261+0000 7f4203dd5640 1 --2- 172.21.15.86:0/1437785926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f41fc107260 0x7f41fc198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.261+0000 7f4203dd5640 1 --2- 172.21.15.86:0/1437785926 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f41fc102190 0x7f41fc198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.261+0000 7f4203dd5640 1 -- 172.21.15.86:0/1437785926 >> 172.21.15.86:0/1437785926 conn(0x7f41fc0793e0 msgr2=0x7f41fc0fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:57.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.261+0000 7f4203dd5640 1 -- 172.21.15.86:0/1437785926 shutdown_connections 2024-04-21T08:38:57.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.261+0000 7f4203dd5640 1 -- 172.21.15.86:0/1437785926 wait complete. 2024-04-21T08:38:57.273 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-04-21T08:38:56.653889+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1194594,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1639,"num_read_kb":1148,"num_write":578,"num_write_kb":4413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":1181659,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":955,"ondisk_log_size":955,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":390584,"kb_used_data":16376,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124551240,"statfs":{"total":1151940427776,"available":1151540469760,"internally_reserved":0,"allocated":16769024,"data_stored":9913268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":382973244},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":153,"num_read_kb":238,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.993736"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807854+0000","last_change":"2024-04-21T08:36:20.913421+0000","last_active":"2024-04-21T08:38:46.807854+0000","last_peered":"2024-04-21T08:38:46.807854+0000","last_clean":"2024-04-21T08:38:46.807854+0000","last_became_active":"2024-04-21T08:36:20.913017+0000","last_became_peered":"2024-04-21T08:36:20.913017+0000","last_unstale":"2024-04-21T08:38:46.807854+0000","last_undegraded":"2024-04-21T08:38:46.807854+0000","last_fullsized":"2024-04-21T08:38:46.807854+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:16:29.885916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.915359+0000","last_change":"2024-04-21T08:36:20.911675+0000","last_active":"2024-04-21T08:38:46.915359+0000","last_peered":"2024-04-21T08:38:46.915359+0000","last_clean":"2024-04-21T08:38:46.915359+0000","last_became_active":"2024-04-21T08:36:20.910591+0000","last_became_peered":"2024-04-21T08:36:20.910591+0000","last_unstale":"2024-04-21T08:38:46.915359+0000","last_undegraded":"2024-04-21T08:38:46.915359+0000","last_fullsized":"2024-04-21T08:38:46.915359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:01:40.735376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.915070+0000","last_change":"2024-04-21T08:36:20.911978+0000","last_active":"2024-04-21T08:38:46.915070+0000","last_peered":"2024-04-21T08:38:46.915070+0000","last_clean":"2024-04-21T08:38:46.915070+0000","last_became_active":"2024-04-21T08:36:20.911641+0000","last_became_peered":"2024-04-21T08:36:20.911641+0000","last_unstale":"2024-04-21T08:38:46.915070+0000","last_undegraded":"2024-04-21T08:38:46.915070+0000","last_fullsized":"2024-04-21T08:38:46.915070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:14:23.789790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911901+0000","last_change":"2024-04-21T08:36:20.919027+0000","last_active":"2024-04-21T08:38:46.911901+0000","last_peered":"2024-04-21T08:38:46.911901+0000","last_clean":"2024-04-21T08:38:46.911901+0000","last_became_active":"2024-04-21T08:36:20.918740+0000","last_became_peered":"2024-04-21T08:36:20.918740+0000","last_unstale":"2024-04-21T08:38:46.911901+0000","last_undegraded":"2024-04-21T08:38:46.911901+0000","last_fullsized":"2024-04-21T08:38:46.911901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:00:31.066938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655099+0000","last_change":"2024-04-21T08:36:20.907175+0000","last_active":"2024-04-21T08:38:46.655099+0000","last_peered":"2024-04-21T08:38:46.655099+0000","last_clean":"2024-04-21T08:38:46.655099+0000","last_became_active":"2024-04-21T08:36:20.906818+0000","last_became_peered":"2024-04-21T08:36:20.906818+0000","last_unstale":"2024-04-21T08:38:46.655099+0000","last_undegraded":"2024-04-21T08:38:46.655099+0000","last_fullsized":"2024-04-21T08:38:46.655099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:41:03.332231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638267+0000","last_change":"2024-04-21T08:36:20.905968+0000","last_active":"2024-04-21T08:38:46.638267+0000","last_peered":"2024-04-21T08:38:46.638267+0000","last_clean":"2024-04-21T08:38:46.638267+0000","last_became_active":"2024-04-21T08:36:20.904840+0000","last_became_peered":"2024-04-21T08:36:20.904840+0000","last_unstale":"2024-04-21T08:38:46.638267+0000","last_undegraded":"2024-04-21T08:38:46.638267+0000","last_fullsized":"2024-04-21T08:38:46.638267+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:23:46.886241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913625+0000","last_change":"2024-04-21T08:36:20.912045+0000","last_active":"2024-04-21T08:38:46.913625+0000","last_peered":"2024-04-21T08:38:46.913625+0000","last_clean":"2024-04-21T08:38:46.913625+0000","last_became_active":"2024-04-21T08:36:20.911690+0000","last_became_peered":"2024-04-21T08:36:20.911690+0000","last_unstale":"2024-04-21T08:38:46.913625+0000","last_undegraded":"2024-04-21T08:38:46.913625+0000","last_fullsized":"2024-04-21T08:38:46.913625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:06:28.229307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638674+0000","last_change":"2024-04-21T08:36:20.904874+0000","last_active":"2024-04-21T08:38:46.638674+0000","last_peered":"2024-04-21T08:38:46.638674+0000","last_clean":"2024-04-21T08:38:46.638674+0000","last_became_active":"2024-04-21T08:36:20.904217+0000","last_became_peered":"2024-04-21T08:36:20.904217+0000","last_unstale":"2024-04-21T08:38:46.638674+0000","last_undegraded":"2024-04-21T08:38:46.638674+0000","last_fullsized":"2024-04-21T08:38:46.638674+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:37:04.423812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.923169+0000","last_change":"2024-04-21T08:36:20.912874+0000","last_active":"2024-04-21T08:38:46.923169+0000","last_peered":"2024-04-21T08:38:46.923169+0000","last_clean":"2024-04-21T08:38:46.923169+0000","last_became_active":"2024-04-21T08:36:20.912569+0000","last_became_peered":"2024-04-21T08:36:20.912569+0000","last_unstale":"2024-04-21T08:38:46.923169+0000","last_undegraded":"2024-04-21T08:38:46.923169+0000","last_fullsized":"2024-04-21T08:38:46.923169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:55:20.763157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":53,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638546+0000","last_change":"2024-04-21T08:37:24.296916+0000","last_active":"2024-04-21T08:38:46.638546+0000","last_peered":"2024-04-21T08:38:46.638546+0000","last_clean":"2024-04-21T08:38:46.638546+0000","last_became_active":"2024-04-21T08:37:24.296547+0000","last_became_peered":"2024-04-21T08:37:24.296547+0000","last_unstale":"2024-04-21T08:38:46.638546+0000","last_undegraded":"2024-04-21T08:38:46.638546+0000","last_fullsized":"2024-04-21T08:38:46.638546+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:47:56.683793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914246+0000","last_change":"2024-04-21T08:36:20.909094+0000","last_active":"2024-04-21T08:38:46.914246+0000","last_peered":"2024-04-21T08:38:46.914246+0000","last_clean":"2024-04-21T08:38:46.914246+0000","last_became_active":"2024-04-21T08:36:20.908634+0000","last_became_peered":"2024-04-21T08:36:20.908634+0000","last_unstale":"2024-04-21T08:38:46.914246+0000","last_undegraded":"2024-04-21T08:38:46.914246+0000","last_fullsized":"2024-04-21T08:38:46.914246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:28:38.450621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911748+0000","last_change":"2024-04-21T08:36:20.909429+0000","last_active":"2024-04-21T08:38:46.911748+0000","last_peered":"2024-04-21T08:38:46.911748+0000","last_clean":"2024-04-21T08:38:46.911748+0000","last_became_active":"2024-04-21T08:36:20.909081+0000","last_became_peered":"2024-04-21T08:36:20.909081+0000","last_unstale":"2024-04-21T08:38:46.911748+0000","last_undegraded":"2024-04-21T08:38:46.911748+0000","last_fullsized":"2024-04-21T08:38:46.911748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:45:46.939125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913919+0000","last_change":"2024-04-21T08:36:20.895894+0000","last_active":"2024-04-21T08:38:46.913919+0000","last_peered":"2024-04-21T08:38:46.913919+0000","last_clean":"2024-04-21T08:38:46.913919+0000","last_became_active":"2024-04-21T08:36:20.895572+0000","last_became_peered":"2024-04-21T08:36:20.895572+0000","last_unstale":"2024-04-21T08:38:46.913919+0000","last_undegraded":"2024-04-21T08:38:46.913919+0000","last_fullsized":"2024-04-21T08:38:46.913919+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:12:48.750555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914638+0000","last_change":"2024-04-21T08:36:20.905356+0000","last_active":"2024-04-21T08:38:46.914638+0000","last_peered":"2024-04-21T08:38:46.914638+0000","last_clean":"2024-04-21T08:38:46.914638+0000","last_became_active":"2024-04-21T08:36:20.904954+0000","last_became_peered":"2024-04-21T08:36:20.904954+0000","last_unstale":"2024-04-21T08:38:46.914638+0000","last_undegraded":"2024-04-21T08:38:46.914638+0000","last_fullsized":"2024-04-21T08:38:46.914638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:45:49.116482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.915244+0000","last_change":"2024-04-21T08:36:20.911550+0000","last_active":"2024-04-21T08:38:46.915244+0000","last_peered":"2024-04-21T08:38:46.915244+0000","last_clean":"2024-04-21T08:38:46.915244+0000","last_became_active":"2024-04-21T08:36:20.910557+0000","last_became_peered":"2024-04-21T08:36:20.910557+0000","last_unstale":"2024-04-21T08:38:46.915244+0000","last_undegraded":"2024-04-21T08:38:46.915244+0000","last_fullsized":"2024-04-21T08:38:46.915244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:09:49.422729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.922413+0000","last_change":"2024-04-21T08:36:20.903954+0000","last_active":"2024-04-21T08:38:46.922413+0000","last_peered":"2024-04-21T08:38:46.922413+0000","last_clean":"2024-04-21T08:38:46.922413+0000","last_became_active":"2024-04-21T08:36:20.903641+0000","last_became_peered":"2024-04-21T08:36:20.903641+0000","last_unstale":"2024-04-21T08:38:46.922413+0000","last_undegraded":"2024-04-21T08:38:46.922413+0000","last_fullsized":"2024-04-21T08:38:46.922413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:08:55.678940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807825+0000","last_change":"2024-04-21T08:36:20.914256+0000","last_active":"2024-04-21T08:38:46.807825+0000","last_peered":"2024-04-21T08:38:46.807825+0000","last_clean":"2024-04-21T08:38:46.807825+0000","last_became_active":"2024-04-21T08:36:20.913858+0000","last_became_peered":"2024-04-21T08:36:20.913858+0000","last_unstale":"2024-04-21T08:38:46.807825+0000","last_undegraded":"2024-04-21T08:38:46.807825+0000","last_fullsized":"2024-04-21T08:38:46.807825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:01:10.276763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.743080+0000","last_change":"2024-04-21T08:36:20.904138+0000","last_active":"2024-04-21T08:38:46.743080+0000","last_peered":"2024-04-21T08:38:46.743080+0000","last_clean":"2024-04-21T08:38:46.743080+0000","last_became_active":"2024-04-21T08:36:20.903670+0000","last_became_peered":"2024-04-21T08:36:20.903670+0000","last_unstale":"2024-04-21T08:38:46.743080+0000","last_undegraded":"2024-04-21T08:38:46.743080+0000","last_fullsized":"2024-04-21T08:38:46.743080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:38:50.160867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639071+0000","last_change":"2024-04-21T08:36:20.910798+0000","last_active":"2024-04-21T08:38:46.639071+0000","last_peered":"2024-04-21T08:38:46.639071+0000","last_clean":"2024-04-21T08:38:46.639071+0000","last_became_active":"2024-04-21T08:36:20.910411+0000","last_became_peered":"2024-04-21T08:36:20.910411+0000","last_unstale":"2024-04-21T08:38:46.639071+0000","last_undegraded":"2024-04-21T08:38:46.639071+0000","last_fullsized":"2024-04-21T08:38:46.639071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:13:12.109818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638676+0000","last_change":"2024-04-21T08:36:24.924831+0000","last_active":"2024-04-21T08:38:46.638676+0000","last_peered":"2024-04-21T08:38:46.638676+0000","last_clean":"2024-04-21T08:38:46.638676+0000","last_became_active":"2024-04-21T08:36:24.924435+0000","last_became_peered":"2024-04-21T08:36:24.924435+0000","last_unstale":"2024-04-21T08:38:46.638676+0000","last_undegraded":"2024-04-21T08:38:46.638676+0000","last_fullsized":"2024-04-21T08:38:46.638676+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:23.447079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":194,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:52.412380+0000","last_change":"2024-04-21T08:37:24.308024+0000","last_active":"2024-04-21T08:38:52.412380+0000","last_peered":"2024-04-21T08:38:52.412380+0000","last_clean":"2024-04-21T08:38:52.412380+0000","last_became_active":"2024-04-21T08:37:24.297491+0000","last_became_peered":"2024-04-21T08:37:24.297491+0000","last_unstale":"2024-04-21T08:38:52.412380+0000","last_undegraded":"2024-04-21T08:38:52.412380+0000","last_fullsized":"2024-04-21T08:38:52.412380+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:48:10.039545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639158+0000","last_change":"2024-04-21T08:36:20.912851+0000","last_active":"2024-04-21T08:38:46.639158+0000","last_peered":"2024-04-21T08:38:46.639158+0000","last_clean":"2024-04-21T08:38:46.639158+0000","last_became_active":"2024-04-21T08:36:20.912407+0000","last_became_peered":"2024-04-21T08:36:20.912407+0000","last_unstale":"2024-04-21T08:38:46.639158+0000","last_undegraded":"2024-04-21T08:38:46.639158+0000","last_fullsized":"2024-04-21T08:38:46.639158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:45:18.026940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.910950+0000","last_change":"2024-04-21T08:36:24.924052+0000","last_active":"2024-04-21T08:38:46.910950+0000","last_peered":"2024-04-21T08:38:46.910950+0000","last_clean":"2024-04-21T08:38:46.910950+0000","last_became_active":"2024-04-21T08:36:24.923618+0000","last_became_peered":"2024-04-21T08:36:24.923618+0000","last_unstale":"2024-04-21T08:38:46.910950+0000","last_undegraded":"2024-04-21T08:38:46.910950+0000","last_fullsized":"2024-04-21T08:38:46.910950+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:08:18.182386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639180+0000","last_change":"2024-04-21T08:36:20.285291+0000","last_active":"2024-04-21T08:38:46.639180+0000","last_peered":"2024-04-21T08:38:46.639180+0000","last_clean":"2024-04-21T08:38:46.639180+0000","last_became_active":"2024-04-21T08:36:20.284969+0000","last_became_peered":"2024-04-21T08:36:20.284969+0000","last_unstale":"2024-04-21T08:38:46.639180+0000","last_undegraded":"2024-04-21T08:38:46.639180+0000","last_fullsized":"2024-04-21T08:38:46.639180+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:55:29.484439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.915781+0000","last_change":"2024-04-21T08:36:20.902819+0000","last_active":"2024-04-21T08:38:46.915781+0000","last_peered":"2024-04-21T08:38:46.915781+0000","last_clean":"2024-04-21T08:38:46.915781+0000","last_became_active":"2024-04-21T08:36:20.902466+0000","last_became_peered":"2024-04-21T08:36:20.902466+0000","last_unstale":"2024-04-21T08:38:46.915781+0000","last_undegraded":"2024-04-21T08:38:46.915781+0000","last_fullsized":"2024-04-21T08:38:46.915781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:29:40.454026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911156+0000","last_change":"2024-04-21T08:36:24.926480+0000","last_active":"2024-04-21T08:38:46.911156+0000","last_peered":"2024-04-21T08:38:46.911156+0000","last_clean":"2024-04-21T08:38:46.911156+0000","last_became_active":"2024-04-21T08:36:24.926116+0000","last_became_peered":"2024-04-21T08:36:24.926116+0000","last_unstale":"2024-04-21T08:38:46.911156+0000","last_undegraded":"2024-04-21T08:38:46.911156+0000","last_fullsized":"2024-04-21T08:38:46.911156+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:43:18.839396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655608+0000","last_change":"2024-04-21T08:36:20.293086+0000","last_active":"2024-04-21T08:38:46.655608+0000","last_peered":"2024-04-21T08:38:46.655608+0000","last_clean":"2024-04-21T08:38:46.655608+0000","last_became_active":"2024-04-21T08:36:20.291178+0000","last_became_peered":"2024-04-21T08:36:20.291178+0000","last_unstale":"2024-04-21T08:38:46.655608+0000","last_undegraded":"2024-04-21T08:38:46.655608+0000","last_fullsized":"2024-04-21T08:38:46.655608+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:02:32.935930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.922777+0000","last_change":"2024-04-21T08:36:20.909374+0000","last_active":"2024-04-21T08:38:46.922777+0000","last_peered":"2024-04-21T08:38:46.922777+0000","last_clean":"2024-04-21T08:38:46.922777+0000","last_became_active":"2024-04-21T08:36:20.909043+0000","last_became_peered":"2024-04-21T08:36:20.909043+0000","last_unstale":"2024-04-21T08:38:46.922777+0000","last_undegraded":"2024-04-21T08:38:46.922777+0000","last_fullsized":"2024-04-21T08:38:46.922777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:45:45.719942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911098+0000","last_change":"2024-04-21T08:38:24.623194+0000","last_active":"2024-04-21T08:38:46.911098+0000","last_peered":"2024-04-21T08:38:46.911098+0000","last_clean":"2024-04-21T08:38:46.911098+0000","last_became_active":"2024-04-21T08:38:24.622632+0000","last_became_peered":"2024-04-21T08:38:24.622632+0000","last_unstale":"2024-04-21T08:38:46.911098+0000","last_undegraded":"2024-04-21T08:38:46.911098+0000","last_fullsized":"2024-04-21T08:38:46.911098+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:26:39.752161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655483+0000","last_change":"2024-04-21T08:36:20.288644+0000","last_active":"2024-04-21T08:38:46.655483+0000","last_peered":"2024-04-21T08:38:46.655483+0000","last_clean":"2024-04-21T08:38:46.655483+0000","last_became_active":"2024-04-21T08:36:20.288069+0000","last_became_peered":"2024-04-21T08:36:20.288069+0000","last_unstale":"2024-04-21T08:38:46.655483+0000","last_undegraded":"2024-04-21T08:38:46.655483+0000","last_fullsized":"2024-04-21T08:38:46.655483+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:37:36.522713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751431+0000","last_change":"2024-04-21T08:36:20.908505+0000","last_active":"2024-04-21T08:38:46.751431+0000","last_peered":"2024-04-21T08:38:46.751431+0000","last_clean":"2024-04-21T08:38:46.751431+0000","last_became_active":"2024-04-21T08:36:20.908163+0000","last_became_peered":"2024-04-21T08:36:20.908163+0000","last_unstale":"2024-04-21T08:38:46.751431+0000","last_undegraded":"2024-04-21T08:38:46.751431+0000","last_fullsized":"2024-04-21T08:38:46.751431+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:54:18.530203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639160+0000","last_change":"2024-04-21T08:36:24.919428+0000","last_active":"2024-04-21T08:38:46.639160+0000","last_peered":"2024-04-21T08:38:46.639160+0000","last_clean":"2024-04-21T08:38:46.639160+0000","last_became_active":"2024-04-21T08:36:24.918722+0000","last_became_peered":"2024-04-21T08:36:24.918722+0000","last_unstale":"2024-04-21T08:38:46.639160+0000","last_undegraded":"2024-04-21T08:38:46.639160+0000","last_fullsized":"2024-04-21T08:38:46.639160+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:30:32.369551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"93'8","reported_seq":174,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:52.641167+0000","last_change":"2024-04-21T08:36:20.287642+0000","last_active":"2024-04-21T08:38:52.641167+0000","last_peered":"2024-04-21T08:38:52.641167+0000","last_clean":"2024-04-21T08:38:52.641167+0000","last_became_active":"2024-04-21T08:36:20.287296+0000","last_became_peered":"2024-04-21T08:36:20.287296+0000","last_unstale":"2024-04-21T08:38:52.641167+0000","last_undegraded":"2024-04-21T08:38:52.641167+0000","last_fullsized":"2024-04-21T08:38:52.641167+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:59:28.579020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1462,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":133,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655128+0000","last_change":"2024-04-21T08:36:20.916971+0000","last_active":"2024-04-21T08:38:46.655128+0000","last_peered":"2024-04-21T08:38:46.655128+0000","last_clean":"2024-04-21T08:38:46.655128+0000","last_became_active":"2024-04-21T08:36:20.916622+0000","last_became_peered":"2024-04-21T08:36:20.916622+0000","last_unstale":"2024-04-21T08:38:46.655128+0000","last_undegraded":"2024-04-21T08:38:46.655128+0000","last_fullsized":"2024-04-21T08:38:46.655128+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:49:38.858772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638856+0000","last_change":"2024-04-21T08:36:24.924170+0000","last_active":"2024-04-21T08:38:46.638856+0000","last_peered":"2024-04-21T08:38:46.638856+0000","last_clean":"2024-04-21T08:38:46.638856+0000","last_became_active":"2024-04-21T08:36:24.923744+0000","last_became_peered":"2024-04-21T08:36:24.923744+0000","last_unstale":"2024-04-21T08:38:46.638856+0000","last_undegraded":"2024-04-21T08:38:46.638856+0000","last_fullsized":"2024-04-21T08:38:46.638856+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:23:24.605021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638388+0000","last_change":"2024-04-21T08:36:20.286545+0000","last_active":"2024-04-21T08:38:46.638388+0000","last_peered":"2024-04-21T08:38:46.638388+0000","last_clean":"2024-04-21T08:38:46.638388+0000","last_became_active":"2024-04-21T08:36:20.286197+0000","last_became_peered":"2024-04-21T08:36:20.286197+0000","last_unstale":"2024-04-21T08:38:46.638388+0000","last_undegraded":"2024-04-21T08:38:46.638388+0000","last_fullsized":"2024-04-21T08:38:46.638388+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:27:55.430428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751535+0000","last_change":"2024-04-21T08:36:20.907743+0000","last_active":"2024-04-21T08:38:46.751535+0000","last_peered":"2024-04-21T08:38:46.751535+0000","last_clean":"2024-04-21T08:38:46.751535+0000","last_became_active":"2024-04-21T08:36:20.907163+0000","last_became_peered":"2024-04-21T08:36:20.907163+0000","last_unstale":"2024-04-21T08:38:46.751535+0000","last_undegraded":"2024-04-21T08:38:46.751535+0000","last_fullsized":"2024-04-21T08:38:46.751535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:41:00.172353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807766+0000","last_change":"2024-04-21T08:37:24.301529+0000","last_active":"2024-04-21T08:38:46.807766+0000","last_peered":"2024-04-21T08:38:46.807766+0000","last_clean":"2024-04-21T08:38:46.807766+0000","last_became_active":"2024-04-21T08:37:24.301014+0000","last_became_peered":"2024-04-21T08:37:24.301014+0000","last_unstale":"2024-04-21T08:38:46.807766+0000","last_undegraded":"2024-04-21T08:38:46.807766+0000","last_fullsized":"2024-04-21T08:38:46.807766+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:58:20.088501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.912093+0000","last_change":"2024-04-21T08:36:20.283143+0000","last_active":"2024-04-21T08:38:46.912093+0000","last_peered":"2024-04-21T08:38:46.912093+0000","last_clean":"2024-04-21T08:38:46.912093+0000","last_became_active":"2024-04-21T08:36:20.282585+0000","last_became_peered":"2024-04-21T08:36:20.282585+0000","last_unstale":"2024-04-21T08:38:46.912093+0000","last_undegraded":"2024-04-21T08:38:46.912093+0000","last_fullsized":"2024-04-21T08:38:46.912093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:12:37.588764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638543+0000","last_change":"2024-04-21T08:36:20.912591+0000","last_active":"2024-04-21T08:38:46.638543+0000","last_peered":"2024-04-21T08:38:46.638543+0000","last_clean":"2024-04-21T08:38:46.638543+0000","last_became_active":"2024-04-21T08:36:20.912174+0000","last_became_peered":"2024-04-21T08:36:20.912174+0000","last_unstale":"2024-04-21T08:38:46.638543+0000","last_undegraded":"2024-04-21T08:38:46.638543+0000","last_fullsized":"2024-04-21T08:38:46.638543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:33:58.949506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.915307+0000","last_change":"2024-04-21T08:36:24.915998+0000","last_active":"2024-04-21T08:38:46.915307+0000","last_peered":"2024-04-21T08:38:46.915307+0000","last_clean":"2024-04-21T08:38:46.915307+0000","last_became_active":"2024-04-21T08:36:24.915621+0000","last_became_peered":"2024-04-21T08:36:24.915621+0000","last_unstale":"2024-04-21T08:38:46.915307+0000","last_undegraded":"2024-04-21T08:38:46.915307+0000","last_fullsized":"2024-04-21T08:38:46.915307+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:43:05.117181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751087+0000","last_change":"2024-04-21T08:36:20.286308+0000","last_active":"2024-04-21T08:38:46.751087+0000","last_peered":"2024-04-21T08:38:46.751087+0000","last_clean":"2024-04-21T08:38:46.751087+0000","last_became_active":"2024-04-21T08:36:20.285975+0000","last_became_peered":"2024-04-21T08:36:20.285975+0000","last_unstale":"2024-04-21T08:38:46.751087+0000","last_undegraded":"2024-04-21T08:38:46.751087+0000","last_fullsized":"2024-04-21T08:38:46.751087+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:55:10.658349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.742821+0000","last_change":"2024-04-21T08:36:20.900502+0000","last_active":"2024-04-21T08:38:46.742821+0000","last_peered":"2024-04-21T08:38:46.742821+0000","last_clean":"2024-04-21T08:38:46.742821+0000","last_became_active":"2024-04-21T08:36:20.900199+0000","last_became_peered":"2024-04-21T08:36:20.900199+0000","last_unstale":"2024-04-21T08:38:46.742821+0000","last_undegraded":"2024-04-21T08:38:46.742821+0000","last_fullsized":"2024-04-21T08:38:46.742821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:17:00.720918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638804+0000","last_change":"2024-04-21T08:36:24.920991+0000","last_active":"2024-04-21T08:38:46.638804+0000","last_peered":"2024-04-21T08:38:46.638804+0000","last_clean":"2024-04-21T08:38:46.638804+0000","last_became_active":"2024-04-21T08:36:24.916611+0000","last_became_peered":"2024-04-21T08:36:24.916611+0000","last_unstale":"2024-04-21T08:38:46.638804+0000","last_undegraded":"2024-04-21T08:38:46.638804+0000","last_fullsized":"2024-04-21T08:38:46.638804+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:01:01.412127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"93'7","reported_seq":293,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:51.661122+0000","last_change":"2024-04-21T08:36:20.284966+0000","last_active":"2024-04-21T08:38:51.661122+0000","last_peered":"2024-04-21T08:38:51.661122+0000","last_clean":"2024-04-21T08:38:51.661122+0000","last_became_active":"2024-04-21T08:36:20.284612+0000","last_became_peered":"2024-04-21T08:36:20.284612+0000","last_unstale":"2024-04-21T08:38:51.661122+0000","last_undegraded":"2024-04-21T08:38:51.661122+0000","last_fullsized":"2024-04-21T08:38:51.661122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:54:50.095013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":253,"num_read_kb":253,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.923175+0000","last_change":"2024-04-21T08:36:20.903493+0000","last_active":"2024-04-21T08:38:46.923175+0000","last_peered":"2024-04-21T08:38:46.923175+0000","last_clean":"2024-04-21T08:38:46.923175+0000","last_became_active":"2024-04-21T08:36:20.902752+0000","last_became_peered":"2024-04-21T08:36:20.902752+0000","last_unstale":"2024-04-21T08:38:46.923175+0000","last_undegraded":"2024-04-21T08:38:46.923175+0000","last_fullsized":"2024-04-21T08:38:46.923175+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:40.684264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.742602+0000","last_change":"2024-04-21T08:36:24.920505+0000","last_active":"2024-04-21T08:38:46.742602+0000","last_peered":"2024-04-21T08:38:46.742602+0000","last_clean":"2024-04-21T08:38:46.742602+0000","last_became_active":"2024-04-21T08:36:24.919966+0000","last_became_peered":"2024-04-21T08:36:24.919966+0000","last_unstale":"2024-04-21T08:38:46.742602+0000","last_undegraded":"2024-04-21T08:38:46.742602+0000","last_fullsized":"2024-04-21T08:38:46.742602+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:58:05.131815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914242+0000","last_change":"2024-04-21T08:36:20.281720+0000","last_active":"2024-04-21T08:38:46.914242+0000","last_peered":"2024-04-21T08:38:46.914242+0000","last_clean":"2024-04-21T08:38:46.914242+0000","last_became_active":"2024-04-21T08:36:20.279862+0000","last_became_peered":"2024-04-21T08:36:20.279862+0000","last_unstale":"2024-04-21T08:38:46.914242+0000","last_undegraded":"2024-04-21T08:38:46.914242+0000","last_fullsized":"2024-04-21T08:38:46.914242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:18:58.106532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.916030+0000","last_change":"2024-04-21T08:36:20.908520+0000","last_active":"2024-04-21T08:38:46.916030+0000","last_peered":"2024-04-21T08:38:46.916030+0000","last_clean":"2024-04-21T08:38:46.916030+0000","last_became_active":"2024-04-21T08:36:20.908184+0000","last_became_peered":"2024-04-21T08:36:20.908184+0000","last_unstale":"2024-04-21T08:38:46.916030+0000","last_undegraded":"2024-04-21T08:38:46.916030+0000","last_fullsized":"2024-04-21T08:38:46.916030+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:14:33.895864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911465+0000","last_change":"2024-04-21T08:36:24.920697+0000","last_active":"2024-04-21T08:38:46.911465+0000","last_peered":"2024-04-21T08:38:46.911465+0000","last_clean":"2024-04-21T08:38:46.911465+0000","last_became_active":"2024-04-21T08:36:24.919976+0000","last_became_peered":"2024-04-21T08:36:24.919976+0000","last_unstale":"2024-04-21T08:38:46.911465+0000","last_undegraded":"2024-04-21T08:38:46.911465+0000","last_fullsized":"2024-04-21T08:38:46.911465+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:40:09.238887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751422+0000","last_change":"2024-04-21T08:36:20.283531+0000","last_active":"2024-04-21T08:38:46.751422+0000","last_peered":"2024-04-21T08:38:46.751422+0000","last_clean":"2024-04-21T08:38:46.751422+0000","last_became_active":"2024-04-21T08:36:20.283159+0000","last_became_peered":"2024-04-21T08:36:20.283159+0000","last_unstale":"2024-04-21T08:38:46.751422+0000","last_undegraded":"2024-04-21T08:38:46.751422+0000","last_fullsized":"2024-04-21T08:38:46.751422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:57:02.917537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.922897+0000","last_change":"2024-04-21T08:36:20.910113+0000","last_active":"2024-04-21T08:38:46.922897+0000","last_peered":"2024-04-21T08:38:46.922897+0000","last_clean":"2024-04-21T08:38:46.922897+0000","last_became_active":"2024-04-21T08:36:20.909744+0000","last_became_peered":"2024-04-21T08:36:20.909744+0000","last_unstale":"2024-04-21T08:38:46.922897+0000","last_undegraded":"2024-04-21T08:38:46.922897+0000","last_fullsized":"2024-04-21T08:38:46.922897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:16:14.568318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.742678+0000","last_change":"2024-04-21T08:36:24.922232+0000","last_active":"2024-04-21T08:38:46.742678+0000","last_peered":"2024-04-21T08:38:46.742678+0000","last_clean":"2024-04-21T08:38:46.742678+0000","last_became_active":"2024-04-21T08:36:24.921690+0000","last_became_peered":"2024-04-21T08:36:24.921690+0000","last_unstale":"2024-04-21T08:38:46.742678+0000","last_undegraded":"2024-04-21T08:38:46.742678+0000","last_fullsized":"2024-04-21T08:38:46.742678+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:21:42.580203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'42","reported_seq":343,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:52.640372+0000","last_change":"2024-04-21T08:36:20.283574+0000","last_active":"2024-04-21T08:38:52.640372+0000","last_peered":"2024-04-21T08:38:52.640372+0000","last_clean":"2024-04-21T08:38:52.640372+0000","last_became_active":"2024-04-21T08:36:20.283220+0000","last_became_peered":"2024-04-21T08:36:20.283220+0000","last_unstale":"2024-04-21T08:38:52.640372+0000","last_undegraded":"2024-04-21T08:38:52.640372+0000","last_fullsized":"2024-04-21T08:38:52.640372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:55:24.233237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70068,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":268,"num_read_kb":203,"num_write":44,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655458+0000","last_change":"2024-04-21T08:36:20.919264+0000","last_active":"2024-04-21T08:38:46.655458+0000","last_peered":"2024-04-21T08:38:46.655458+0000","last_clean":"2024-04-21T08:38:46.655458+0000","last_became_active":"2024-04-21T08:36:20.918878+0000","last_became_peered":"2024-04-21T08:36:20.918878+0000","last_unstale":"2024-04-21T08:38:46.655458+0000","last_undegraded":"2024-04-21T08:38:46.655458+0000","last_fullsized":"2024-04-21T08:38:46.655458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:24:06.460758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751721+0000","last_change":"2024-04-21T08:36:24.926878+0000","last_active":"2024-04-21T08:38:46.751721+0000","last_peered":"2024-04-21T08:38:46.751721+0000","last_clean":"2024-04-21T08:38:46.751721+0000","last_became_active":"2024-04-21T08:36:24.926431+0000","last_became_peered":"2024-04-21T08:36:24.926431+0000","last_unstale":"2024-04-21T08:38:46.751721+0000","last_undegraded":"2024-04-21T08:38:46.751721+0000","last_fullsized":"2024-04-21T08:38:46.751721+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:51:29.065962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639141+0000","last_change":"2024-04-21T08:36:20.284841+0000","last_active":"2024-04-21T08:38:46.639141+0000","last_peered":"2024-04-21T08:38:46.639141+0000","last_clean":"2024-04-21T08:38:46.639141+0000","last_became_active":"2024-04-21T08:36:20.283888+0000","last_became_peered":"2024-04-21T08:36:20.283888+0000","last_unstale":"2024-04-21T08:38:46.639141+0000","last_undegraded":"2024-04-21T08:38:46.639141+0000","last_fullsized":"2024-04-21T08:38:46.639141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:29:42.659608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.912243+0000","last_change":"2024-04-21T08:36:20.905140+0000","last_active":"2024-04-21T08:38:46.912243+0000","last_peered":"2024-04-21T08:38:46.912243+0000","last_clean":"2024-04-21T08:38:46.912243+0000","last_became_active":"2024-04-21T08:36:20.904821+0000","last_became_peered":"2024-04-21T08:36:20.904821+0000","last_unstale":"2024-04-21T08:38:46.912243+0000","last_undegraded":"2024-04-21T08:38:46.912243+0000","last_fullsized":"2024-04-21T08:38:46.912243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:09:11.962877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807527+0000","last_change":"2024-04-21T08:38:24.834006+0000","last_active":"2024-04-21T08:38:46.807527+0000","last_peered":"2024-04-21T08:38:46.807527+0000","last_clean":"2024-04-21T08:38:46.807527+0000","last_became_active":"2024-04-21T08:38:24.833453+0000","last_became_peered":"2024-04-21T08:38:24.833453+0000","last_unstale":"2024-04-21T08:38:46.807527+0000","last_undegraded":"2024-04-21T08:38:46.807527+0000","last_fullsized":"2024-04-21T08:38:46.807527+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:27:17.094301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751268+0000","last_change":"2024-04-21T08:36:20.290289+0000","last_active":"2024-04-21T08:38:46.751268+0000","last_peered":"2024-04-21T08:38:46.751268+0000","last_clean":"2024-04-21T08:38:46.751268+0000","last_became_active":"2024-04-21T08:36:20.289922+0000","last_became_peered":"2024-04-21T08:36:20.289922+0000","last_unstale":"2024-04-21T08:38:46.751268+0000","last_undegraded":"2024-04-21T08:38:46.751268+0000","last_fullsized":"2024-04-21T08:38:46.751268+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:10:00.926023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914048+0000","last_change":"2024-04-21T08:36:20.906950+0000","last_active":"2024-04-21T08:38:46.914048+0000","last_peered":"2024-04-21T08:38:46.914048+0000","last_clean":"2024-04-21T08:38:46.914048+0000","last_became_active":"2024-04-21T08:36:20.906599+0000","last_became_peered":"2024-04-21T08:36:20.906599+0000","last_unstale":"2024-04-21T08:38:46.914048+0000","last_undegraded":"2024-04-21T08:38:46.914048+0000","last_fullsized":"2024-04-21T08:38:46.914048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:55:27.595997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.743162+0000","last_change":"2024-04-21T08:36:24.924349+0000","last_active":"2024-04-21T08:38:46.743162+0000","last_peered":"2024-04-21T08:38:46.743162+0000","last_clean":"2024-04-21T08:38:46.743162+0000","last_became_active":"2024-04-21T08:36:24.923982+0000","last_became_peered":"2024-04-21T08:36:24.923982+0000","last_unstale":"2024-04-21T08:38:46.743162+0000","last_undegraded":"2024-04-21T08:38:46.743162+0000","last_fullsized":"2024-04-21T08:38:46.743162+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:20:29.600759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"94'9","reported_seq":175,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:51.641095+0000","last_change":"2024-04-21T08:36:20.285629+0000","last_active":"2024-04-21T08:38:51.641095+0000","last_peered":"2024-04-21T08:38:51.641095+0000","last_clean":"2024-04-21T08:38:51.641095+0000","last_became_active":"2024-04-21T08:36:20.285234+0000","last_became_peered":"2024-04-21T08:36:20.285234+0000","last_unstale":"2024-04-21T08:38:51.641095+0000","last_undegraded":"2024-04-21T08:38:51.641095+0000","last_fullsized":"2024-04-21T08:38:51.641095+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:19:32.966146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1902,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":134,"num_read_kb":6,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913981+0000","last_change":"2024-04-21T08:36:20.906819+0000","last_active":"2024-04-21T08:38:46.913981+0000","last_peered":"2024-04-21T08:38:46.913981+0000","last_clean":"2024-04-21T08:38:46.913981+0000","last_became_active":"2024-04-21T08:36:20.906459+0000","last_became_peered":"2024-04-21T08:36:20.906459+0000","last_unstale":"2024-04-21T08:38:46.913981+0000","last_undegraded":"2024-04-21T08:38:46.913981+0000","last_fullsized":"2024-04-21T08:38:46.913981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:12:39.312857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913253+0000","last_change":"2024-04-21T08:36:24.929452+0000","last_active":"2024-04-21T08:38:46.913253+0000","last_peered":"2024-04-21T08:38:46.913253+0000","last_clean":"2024-04-21T08:38:46.913253+0000","last_became_active":"2024-04-21T08:36:24.929039+0000","last_became_peered":"2024-04-21T08:36:24.929039+0000","last_unstale":"2024-04-21T08:38:46.913253+0000","last_undegraded":"2024-04-21T08:38:46.913253+0000","last_fullsized":"2024-04-21T08:38:46.913253+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:16:05.432024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":76,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655055+0000","last_change":"2024-04-21T08:37:24.719766+0000","last_active":"2024-04-21T08:38:46.655055+0000","last_peered":"2024-04-21T08:38:46.655055+0000","last_clean":"2024-04-21T08:38:46.655055+0000","last_became_active":"2024-04-21T08:37:24.708201+0000","last_became_peered":"2024-04-21T08:37:24.708201+0000","last_unstale":"2024-04-21T08:38:46.655055+0000","last_undegraded":"2024-04-21T08:38:46.655055+0000","last_fullsized":"2024-04-21T08:38:46.655055+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:10:51.158061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751195+0000","last_change":"2024-04-21T08:36:20.909076+0000","last_active":"2024-04-21T08:38:46.751195+0000","last_peered":"2024-04-21T08:38:46.751195+0000","last_clean":"2024-04-21T08:38:46.751195+0000","last_became_active":"2024-04-21T08:36:20.908739+0000","last_became_peered":"2024-04-21T08:36:20.908739+0000","last_unstale":"2024-04-21T08:38:46.751195+0000","last_undegraded":"2024-04-21T08:38:46.751195+0000","last_fullsized":"2024-04-21T08:38:46.751195+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:03:56.870477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751077+0000","last_change":"2024-04-21T08:36:24.926871+0000","last_active":"2024-04-21T08:38:46.751077+0000","last_peered":"2024-04-21T08:38:46.751077+0000","last_clean":"2024-04-21T08:38:46.751077+0000","last_became_active":"2024-04-21T08:36:24.926438+0000","last_became_peered":"2024-04-21T08:36:24.926438+0000","last_unstale":"2024-04-21T08:38:46.751077+0000","last_undegraded":"2024-04-21T08:38:46.751077+0000","last_fullsized":"2024-04-21T08:38:46.751077+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:24:48.984906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"94'3","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807724+0000","last_change":"2024-04-21T08:37:24.308038+0000","last_active":"2024-04-21T08:38:46.807724+0000","last_peered":"2024-04-21T08:38:46.807724+0000","last_clean":"2024-04-21T08:38:46.807724+0000","last_became_active":"2024-04-21T08:37:24.301034+0000","last_became_peered":"2024-04-21T08:37:24.301034+0000","last_unstale":"2024-04-21T08:38:46.807724+0000","last_undegraded":"2024-04-21T08:38:46.807724+0000","last_fullsized":"2024-04-21T08:38:46.807724+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:37:06.598400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807693+0000","last_change":"2024-04-21T08:37:24.299260+0000","last_active":"2024-04-21T08:38:46.807693+0000","last_peered":"2024-04-21T08:38:46.807693+0000","last_clean":"2024-04-21T08:38:46.807693+0000","last_became_active":"2024-04-21T08:37:24.298831+0000","last_became_peered":"2024-04-21T08:37:24.298831+0000","last_unstale":"2024-04-21T08:38:46.807693+0000","last_undegraded":"2024-04-21T08:38:46.807693+0000","last_fullsized":"2024-04-21T08:38:46.807693+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:58:29.368027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655140+0000","last_change":"2024-04-21T08:36:24.929420+0000","last_active":"2024-04-21T08:38:46.655140+0000","last_peered":"2024-04-21T08:38:46.655140+0000","last_clean":"2024-04-21T08:38:46.655140+0000","last_became_active":"2024-04-21T08:36:24.928981+0000","last_became_peered":"2024-04-21T08:36:24.928981+0000","last_unstale":"2024-04-21T08:38:46.655140+0000","last_undegraded":"2024-04-21T08:38:46.655140+0000","last_fullsized":"2024-04-21T08:38:46.655140+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:38:00.040456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.742890+0000","last_change":"2024-04-21T08:36:20.279910+0000","last_active":"2024-04-21T08:38:46.742890+0000","last_peered":"2024-04-21T08:38:46.742890+0000","last_clean":"2024-04-21T08:38:46.742890+0000","last_became_active":"2024-04-21T08:36:20.278904+0000","last_became_peered":"2024-04-21T08:36:20.278904+0000","last_unstale":"2024-04-21T08:38:46.742890+0000","last_undegraded":"2024-04-21T08:38:46.742890+0000","last_fullsized":"2024-04-21T08:38:46.742890+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:40:19.209213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638524+0000","last_change":"2024-04-21T08:36:20.895816+0000","last_active":"2024-04-21T08:38:46.638524+0000","last_peered":"2024-04-21T08:38:46.638524+0000","last_clean":"2024-04-21T08:38:46.638524+0000","last_became_active":"2024-04-21T08:36:20.895487+0000","last_became_peered":"2024-04-21T08:36:20.895487+0000","last_unstale":"2024-04-21T08:38:46.638524+0000","last_undegraded":"2024-04-21T08:38:46.638524+0000","last_fullsized":"2024-04-21T08:38:46.638524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:53:52.153683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.912486+0000","last_change":"2024-04-21T08:36:24.917637+0000","last_active":"2024-04-21T08:38:46.912486+0000","last_peered":"2024-04-21T08:38:46.912486+0000","last_clean":"2024-04-21T08:38:46.912486+0000","last_became_active":"2024-04-21T08:36:24.917192+0000","last_became_peered":"2024-04-21T08:36:24.917192+0000","last_unstale":"2024-04-21T08:38:46.912486+0000","last_undegraded":"2024-04-21T08:38:46.912486+0000","last_fullsized":"2024-04-21T08:38:46.912486+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:02:59.843903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":164,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:51.411645+0000","last_change":"2024-04-21T08:36:20.286424+0000","last_active":"2024-04-21T08:38:51.411645+0000","last_peered":"2024-04-21T08:38:51.411645+0000","last_clean":"2024-04-21T08:38:51.411645+0000","last_became_active":"2024-04-21T08:36:20.285767+0000","last_became_peered":"2024-04-21T08:36:20.285767+0000","last_unstale":"2024-04-21T08:38:51.411645+0000","last_undegraded":"2024-04-21T08:38:51.411645+0000","last_fullsized":"2024-04-21T08:38:51.411645+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:53:33.202734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638138+0000","last_change":"2024-04-21T08:37:24.297650+0000","last_active":"2024-04-21T08:38:46.638138+0000","last_peered":"2024-04-21T08:38:46.638138+0000","last_clean":"2024-04-21T08:38:46.638138+0000","last_became_active":"2024-04-21T08:37:24.297250+0000","last_became_peered":"2024-04-21T08:37:24.297250+0000","last_unstale":"2024-04-21T08:38:46.638138+0000","last_undegraded":"2024-04-21T08:38:46.638138+0000","last_fullsized":"2024-04-21T08:38:46.638138+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:21:21.419556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911806+0000","last_change":"2024-04-21T08:36:24.919669+0000","last_active":"2024-04-21T08:38:46.911806+0000","last_peered":"2024-04-21T08:38:46.911806+0000","last_clean":"2024-04-21T08:38:46.911806+0000","last_became_active":"2024-04-21T08:36:24.919281+0000","last_became_peered":"2024-04-21T08:36:24.919281+0000","last_unstale":"2024-04-21T08:38:46.911806+0000","last_undegraded":"2024-04-21T08:38:46.911806+0000","last_fullsized":"2024-04-21T08:38:46.911806+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:44:27.480367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751175+0000","last_change":"2024-04-21T08:37:24.297826+0000","last_active":"2024-04-21T08:38:46.751175+0000","last_peered":"2024-04-21T08:38:46.751175+0000","last_clean":"2024-04-21T08:38:46.751175+0000","last_became_active":"2024-04-21T08:37:24.297397+0000","last_became_peered":"2024-04-21T08:37:24.297397+0000","last_unstale":"2024-04-21T08:38:46.751175+0000","last_undegraded":"2024-04-21T08:38:46.751175+0000","last_fullsized":"2024-04-21T08:38:46.751175+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:52:17.693725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'7","reported_seq":170,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:51.640626+0000","last_change":"2024-04-21T08:36:20.291033+0000","last_active":"2024-04-21T08:38:51.640626+0000","last_peered":"2024-04-21T08:38:51.640626+0000","last_clean":"2024-04-21T08:38:51.640626+0000","last_became_active":"2024-04-21T08:36:20.290697+0000","last_became_peered":"2024-04-21T08:36:20.290697+0000","last_unstale":"2024-04-21T08:38:51.640626+0000","last_undegraded":"2024-04-21T08:38:51.640626+0000","last_fullsized":"2024-04-21T08:38:51.640626+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:22:54.593595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":129,"num_read_kb":129,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913716+0000","last_change":"2024-04-21T08:37:24.310936+0000","last_active":"2024-04-21T08:38:46.913716+0000","last_peered":"2024-04-21T08:38:46.913716+0000","last_clean":"2024-04-21T08:38:46.913716+0000","last_became_active":"2024-04-21T08:37:24.297349+0000","last_became_peered":"2024-04-21T08:37:24.297349+0000","last_unstale":"2024-04-21T08:38:46.913716+0000","last_undegraded":"2024-04-21T08:38:46.913716+0000","last_fullsized":"2024-04-21T08:38:46.913716+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:31:39.884609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638671+0000","last_change":"2024-04-21T08:36:20.904197+0000","last_active":"2024-04-21T08:38:46.638671+0000","last_peered":"2024-04-21T08:38:46.638671+0000","last_clean":"2024-04-21T08:38:46.638671+0000","last_became_active":"2024-04-21T08:36:20.903099+0000","last_became_peered":"2024-04-21T08:36:20.903099+0000","last_unstale":"2024-04-21T08:38:46.638671+0000","last_undegraded":"2024-04-21T08:38:46.638671+0000","last_fullsized":"2024-04-21T08:38:46.638671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:16:08.914335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911254+0000","last_change":"2024-04-21T08:36:20.280608+0000","last_active":"2024-04-21T08:38:46.911254+0000","last_peered":"2024-04-21T08:38:46.911254+0000","last_clean":"2024-04-21T08:38:46.911254+0000","last_became_active":"2024-04-21T08:36:20.280172+0000","last_became_peered":"2024-04-21T08:36:20.280172+0000","last_unstale":"2024-04-21T08:38:46.911254+0000","last_undegraded":"2024-04-21T08:38:46.911254+0000","last_fullsized":"2024-04-21T08:38:46.911254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:51:12.284463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"94'148","reported_seq":410,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:48.757849+0000","last_change":"2024-04-21T08:35:08.423545+0000","last_active":"2024-04-21T08:38:48.757849+0000","last_peered":"2024-04-21T08:38:48.757849+0000","last_clean":"2024-04-21T08:38:48.757849+0000","last_became_active":"2024-04-21T08:35:08.423027+0000","last_became_peered":"2024-04-21T08:35:08.423027+0000","last_unstale":"2024-04-21T08:38:48.757849+0000","last_undegraded":"2024-04-21T08:38:48.757849+0000","last_fullsized":"2024-04-21T08:38:48.757849+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:31:45.359253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:31:45.359253+0000","last_clean_scrub_stamp":"2024-04-21T08:31:45.359253+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:53:24.310824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751452+0000","last_change":"2024-04-21T08:36:24.922422+0000","last_active":"2024-04-21T08:38:46.751452+0000","last_peered":"2024-04-21T08:38:46.751452+0000","last_clean":"2024-04-21T08:38:46.751452+0000","last_became_active":"2024-04-21T08:36:24.922006+0000","last_became_peered":"2024-04-21T08:36:24.922006+0000","last_unstale":"2024-04-21T08:38:46.751452+0000","last_undegraded":"2024-04-21T08:38:46.751452+0000","last_fullsized":"2024-04-21T08:38:46.751452+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:46:22.840802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911805+0000","last_change":"2024-04-21T08:36:20.911222+0000","last_active":"2024-04-21T08:38:46.911805+0000","last_peered":"2024-04-21T08:38:46.911805+0000","last_clean":"2024-04-21T08:38:46.911805+0000","last_became_active":"2024-04-21T08:36:20.910933+0000","last_became_peered":"2024-04-21T08:36:20.910933+0000","last_unstale":"2024-04-21T08:38:46.911805+0000","last_undegraded":"2024-04-21T08:38:46.911805+0000","last_fullsized":"2024-04-21T08:38:46.911805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:41:25.621266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913633+0000","last_change":"2024-04-21T08:36:20.288409+0000","last_active":"2024-04-21T08:38:46.913633+0000","last_peered":"2024-04-21T08:38:46.913633+0000","last_clean":"2024-04-21T08:38:46.913633+0000","last_became_active":"2024-04-21T08:36:20.287916+0000","last_became_peered":"2024-04-21T08:36:20.287916+0000","last_unstale":"2024-04-21T08:38:46.913633+0000","last_undegraded":"2024-04-21T08:38:46.913633+0000","last_fullsized":"2024-04-21T08:38:46.913633+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:04:31.068352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807586+0000","last_change":"2024-04-21T08:36:24.918166+0000","last_active":"2024-04-21T08:38:46.807586+0000","last_peered":"2024-04-21T08:38:46.807586+0000","last_clean":"2024-04-21T08:38:46.807586+0000","last_became_active":"2024-04-21T08:36:24.917745+0000","last_became_peered":"2024-04-21T08:36:24.917745+0000","last_unstale":"2024-04-21T08:38:46.807586+0000","last_undegraded":"2024-04-21T08:38:46.807586+0000","last_fullsized":"2024-04-21T08:38:46.807586+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:55:41.169681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655417+0000","last_change":"2024-04-21T08:36:20.291079+0000","last_active":"2024-04-21T08:38:46.655417+0000","last_peered":"2024-04-21T08:38:46.655417+0000","last_clean":"2024-04-21T08:38:46.655417+0000","last_became_active":"2024-04-21T08:36:20.290720+0000","last_became_peered":"2024-04-21T08:36:20.290720+0000","last_unstale":"2024-04-21T08:38:46.655417+0000","last_undegraded":"2024-04-21T08:38:46.655417+0000","last_fullsized":"2024-04-21T08:38:46.655417+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:41.269029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.912713+0000","last_change":"2024-04-21T08:36:20.907416+0000","last_active":"2024-04-21T08:38:46.912713+0000","last_peered":"2024-04-21T08:38:46.912713+0000","last_clean":"2024-04-21T08:38:46.912713+0000","last_became_active":"2024-04-21T08:36:20.907005+0000","last_became_peered":"2024-04-21T08:36:20.907005+0000","last_unstale":"2024-04-21T08:38:46.912713+0000","last_undegraded":"2024-04-21T08:38:46.912713+0000","last_fullsized":"2024-04-21T08:38:46.912713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:58:17.979947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638481+0000","last_change":"2024-04-21T08:36:24.921475+0000","last_active":"2024-04-21T08:38:46.638481+0000","last_peered":"2024-04-21T08:38:46.638481+0000","last_clean":"2024-04-21T08:38:46.638481+0000","last_became_active":"2024-04-21T08:36:24.921091+0000","last_became_peered":"2024-04-21T08:36:24.921091+0000","last_unstale":"2024-04-21T08:38:46.638481+0000","last_undegraded":"2024-04-21T08:38:46.638481+0000","last_fullsized":"2024-04-21T08:38:46.638481+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:28:14.224501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'48","reported_seq":105,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.860978+0000","last_change":"2024-04-21T08:36:20.288916+0000","last_active":"2024-04-21T08:38:46.860978+0000","last_peered":"2024-04-21T08:38:46.860978+0000","last_clean":"2024-04-21T08:38:46.860978+0000","last_became_active":"2024-04-21T08:36:20.288575+0000","last_became_peered":"2024-04-21T08:36:20.288575+0000","last_unstale":"2024-04-21T08:38:46.860978+0000","last_undegraded":"2024-04-21T08:38:46.860978+0000","last_fullsized":"2024-04-21T08:38:46.860978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:52:05.795554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639071+0000","last_change":"2024-04-21T08:36:20.912053+0000","last_active":"2024-04-21T08:38:46.639071+0000","last_peered":"2024-04-21T08:38:46.639071+0000","last_clean":"2024-04-21T08:38:46.639071+0000","last_became_active":"2024-04-21T08:36:20.911584+0000","last_became_peered":"2024-04-21T08:36:20.911584+0000","last_unstale":"2024-04-21T08:38:46.639071+0000","last_undegraded":"2024-04-21T08:38:46.639071+0000","last_fullsized":"2024-04-21T08:38:46.639071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:48:36.369954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638589+0000","last_change":"2024-04-21T08:36:24.919212+0000","last_active":"2024-04-21T08:38:46.638589+0000","last_peered":"2024-04-21T08:38:46.638589+0000","last_clean":"2024-04-21T08:38:46.638589+0000","last_became_active":"2024-04-21T08:36:24.918807+0000","last_became_peered":"2024-04-21T08:36:24.918807+0000","last_unstale":"2024-04-21T08:38:46.638589+0000","last_undegraded":"2024-04-21T08:38:46.638589+0000","last_fullsized":"2024-04-21T08:38:46.638589+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:25:50.816775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911766+0000","last_change":"2024-04-21T08:36:20.282536+0000","last_active":"2024-04-21T08:38:46.911766+0000","last_peered":"2024-04-21T08:38:46.911766+0000","last_clean":"2024-04-21T08:38:46.911766+0000","last_became_active":"2024-04-21T08:36:20.282172+0000","last_became_peered":"2024-04-21T08:36:20.282172+0000","last_unstale":"2024-04-21T08:38:46.911766+0000","last_undegraded":"2024-04-21T08:38:46.911766+0000","last_fullsized":"2024-04-21T08:38:46.911766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:15:38.992441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.915413+0000","last_change":"2024-04-21T08:36:20.908416+0000","last_active":"2024-04-21T08:38:46.915413+0000","last_peered":"2024-04-21T08:38:46.915413+0000","last_clean":"2024-04-21T08:38:46.915413+0000","last_became_active":"2024-04-21T08:36:20.908054+0000","last_became_peered":"2024-04-21T08:36:20.908054+0000","last_unstale":"2024-04-21T08:38:46.915413+0000","last_undegraded":"2024-04-21T08:38:46.915413+0000","last_fullsized":"2024-04-21T08:38:46.915413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:23:16.562060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911638+0000","last_change":"2024-04-21T08:36:24.920637+0000","last_active":"2024-04-21T08:38:46.911638+0000","last_peered":"2024-04-21T08:38:46.911638+0000","last_clean":"2024-04-21T08:38:46.911638+0000","last_became_active":"2024-04-21T08:36:24.919921+0000","last_became_peered":"2024-04-21T08:36:24.919921+0000","last_unstale":"2024-04-21T08:38:46.911638+0000","last_undegraded":"2024-04-21T08:38:46.911638+0000","last_fullsized":"2024-04-21T08:38:46.911638+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:20:38.230087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.922412+0000","last_change":"2024-04-21T08:36:20.279332+0000","last_active":"2024-04-21T08:38:46.922412+0000","last_peered":"2024-04-21T08:38:46.922412+0000","last_clean":"2024-04-21T08:38:46.922412+0000","last_became_active":"2024-04-21T08:36:20.278970+0000","last_became_peered":"2024-04-21T08:36:20.278970+0000","last_unstale":"2024-04-21T08:38:46.922412+0000","last_undegraded":"2024-04-21T08:38:46.922412+0000","last_fullsized":"2024-04-21T08:38:46.922412+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:36:16.041610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.654968+0000","last_change":"2024-04-21T08:36:20.919621+0000","last_active":"2024-04-21T08:38:46.654968+0000","last_peered":"2024-04-21T08:38:46.654968+0000","last_clean":"2024-04-21T08:38:46.654968+0000","last_became_active":"2024-04-21T08:36:20.919246+0000","last_became_peered":"2024-04-21T08:36:20.919246+0000","last_unstale":"2024-04-21T08:38:46.654968+0000","last_undegraded":"2024-04-21T08:38:46.654968+0000","last_fullsized":"2024-04-21T08:38:46.654968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:31:59.001501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.743258+0000","last_change":"2024-04-21T08:36:24.920459+0000","last_active":"2024-04-21T08:38:46.743258+0000","last_peered":"2024-04-21T08:38:46.743258+0000","last_clean":"2024-04-21T08:38:46.743258+0000","last_became_active":"2024-04-21T08:36:24.919691+0000","last_became_peered":"2024-04-21T08:36:24.919691+0000","last_unstale":"2024-04-21T08:38:46.743258+0000","last_undegraded":"2024-04-21T08:38:46.743258+0000","last_fullsized":"2024-04-21T08:38:46.743258+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:38:26.655452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638571+0000","last_change":"2024-04-21T08:36:20.284286+0000","last_active":"2024-04-21T08:38:46.638571+0000","last_peered":"2024-04-21T08:38:46.638571+0000","last_clean":"2024-04-21T08:38:46.638571+0000","last_became_active":"2024-04-21T08:36:20.283903+0000","last_became_peered":"2024-04-21T08:36:20.283903+0000","last_unstale":"2024-04-21T08:38:46.638571+0000","last_undegraded":"2024-04-21T08:38:46.638571+0000","last_fullsized":"2024-04-21T08:38:46.638571+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:51:52.409988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807332+0000","last_change":"2024-04-21T08:36:20.911892+0000","last_active":"2024-04-21T08:38:46.807332+0000","last_peered":"2024-04-21T08:38:46.807332+0000","last_clean":"2024-04-21T08:38:46.807332+0000","last_became_active":"2024-04-21T08:36:20.911513+0000","last_became_peered":"2024-04-21T08:36:20.911513+0000","last_unstale":"2024-04-21T08:38:46.807332+0000","last_undegraded":"2024-04-21T08:38:46.807332+0000","last_fullsized":"2024-04-21T08:38:46.807332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:51:13.733534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911539+0000","last_change":"2024-04-21T08:36:24.920347+0000","last_active":"2024-04-21T08:38:46.911539+0000","last_peered":"2024-04-21T08:38:46.911539+0000","last_clean":"2024-04-21T08:38:46.911539+0000","last_became_active":"2024-04-21T08:36:24.919831+0000","last_became_peered":"2024-04-21T08:36:24.919831+0000","last_unstale":"2024-04-21T08:38:46.911539+0000","last_undegraded":"2024-04-21T08:38:46.911539+0000","last_fullsized":"2024-04-21T08:38:46.911539+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:14:27.020476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638672+0000","last_change":"2024-04-21T08:36:20.286786+0000","last_active":"2024-04-21T08:38:46.638672+0000","last_peered":"2024-04-21T08:38:46.638672+0000","last_clean":"2024-04-21T08:38:46.638672+0000","last_became_active":"2024-04-21T08:36:20.286429+0000","last_became_peered":"2024-04-21T08:36:20.286429+0000","last_unstale":"2024-04-21T08:38:46.638672+0000","last_undegraded":"2024-04-21T08:38:46.638672+0000","last_fullsized":"2024-04-21T08:38:46.638672+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:40:57.093370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638983+0000","last_change":"2024-04-21T08:36:20.904767+0000","last_active":"2024-04-21T08:38:46.638983+0000","last_peered":"2024-04-21T08:38:46.638983+0000","last_clean":"2024-04-21T08:38:46.638983+0000","last_became_active":"2024-04-21T08:36:20.904406+0000","last_became_peered":"2024-04-21T08:36:20.904406+0000","last_unstale":"2024-04-21T08:38:46.638983+0000","last_undegraded":"2024-04-21T08:38:46.638983+0000","last_fullsized":"2024-04-21T08:38:46.638983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:11:41.644549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911769+0000","last_change":"2024-04-21T08:36:24.918031+0000","last_active":"2024-04-21T08:38:46.911769+0000","last_peered":"2024-04-21T08:38:46.911769+0000","last_clean":"2024-04-21T08:38:46.911769+0000","last_became_active":"2024-04-21T08:36:24.917618+0000","last_became_peered":"2024-04-21T08:36:24.917618+0000","last_unstale":"2024-04-21T08:38:46.911769+0000","last_undegraded":"2024-04-21T08:38:46.911769+0000","last_fullsized":"2024-04-21T08:38:46.911769+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:50:35.797911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655179+0000","last_change":"2024-04-21T08:36:20.286212+0000","last_active":"2024-04-21T08:38:46.655179+0000","last_peered":"2024-04-21T08:38:46.655179+0000","last_clean":"2024-04-21T08:38:46.655179+0000","last_became_active":"2024-04-21T08:36:20.285878+0000","last_became_peered":"2024-04-21T08:36:20.285878+0000","last_unstale":"2024-04-21T08:38:46.655179+0000","last_undegraded":"2024-04-21T08:38:46.655179+0000","last_fullsized":"2024-04-21T08:38:46.655179+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:19.375455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914242+0000","last_change":"2024-04-21T08:36:20.908335+0000","last_active":"2024-04-21T08:38:46.914242+0000","last_peered":"2024-04-21T08:38:46.914242+0000","last_clean":"2024-04-21T08:38:46.914242+0000","last_became_active":"2024-04-21T08:36:20.907945+0000","last_became_peered":"2024-04-21T08:36:20.907945+0000","last_unstale":"2024-04-21T08:38:46.914242+0000","last_undegraded":"2024-04-21T08:38:46.914242+0000","last_fullsized":"2024-04-21T08:38:46.914242+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:47:17.841511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.922383+0000","last_change":"2024-04-21T08:36:24.915553+0000","last_active":"2024-04-21T08:38:46.922383+0000","last_peered":"2024-04-21T08:38:46.922383+0000","last_clean":"2024-04-21T08:38:46.922383+0000","last_became_active":"2024-04-21T08:36:24.915028+0000","last_became_peered":"2024-04-21T08:36:24.915028+0000","last_unstale":"2024-04-21T08:38:46.922383+0000","last_undegraded":"2024-04-21T08:38:46.922383+0000","last_fullsized":"2024-04-21T08:38:46.922383+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:17:21.192946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'7","reported_seq":169,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:50.660424+0000","last_change":"2024-04-21T08:36:20.279649+0000","last_active":"2024-04-21T08:38:50.660424+0000","last_peered":"2024-04-21T08:38:50.660424+0000","last_clean":"2024-04-21T08:38:50.660424+0000","last_became_active":"2024-04-21T08:36:20.279111+0000","last_became_peered":"2024-04-21T08:36:20.279111+0000","last_unstale":"2024-04-21T08:38:50.660424+0000","last_undegraded":"2024-04-21T08:38:50.660424+0000","last_fullsized":"2024-04-21T08:38:50.660424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:19:57.380495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":128,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914334+0000","last_change":"2024-04-21T08:36:20.911489+0000","last_active":"2024-04-21T08:38:46.914334+0000","last_peered":"2024-04-21T08:38:46.914334+0000","last_clean":"2024-04-21T08:38:46.914334+0000","last_became_active":"2024-04-21T08:36:20.911168+0000","last_became_peered":"2024-04-21T08:36:20.911168+0000","last_unstale":"2024-04-21T08:38:46.914334+0000","last_undegraded":"2024-04-21T08:38:46.914334+0000","last_fullsized":"2024-04-21T08:38:46.914334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:40:41.585841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914668+0000","last_change":"2024-04-21T08:36:24.921175+0000","last_active":"2024-04-21T08:38:46.914668+0000","last_peered":"2024-04-21T08:38:46.914668+0000","last_clean":"2024-04-21T08:38:46.914668+0000","last_became_active":"2024-04-21T08:36:24.920802+0000","last_became_peered":"2024-04-21T08:36:24.920802+0000","last_unstale":"2024-04-21T08:38:46.914668+0000","last_undegraded":"2024-04-21T08:38:46.914668+0000","last_fullsized":"2024-04-21T08:38:46.914668+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:43:12.140016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"94'41","reported_seq":82,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.859069+0000","last_change":"2024-04-21T08:36:20.290202+0000","last_active":"2024-04-21T08:38:46.859069+0000","last_peered":"2024-04-21T08:38:46.859069+0000","last_clean":"2024-04-21T08:38:46.859069+0000","last_became_active":"2024-04-21T08:36:20.289868+0000","last_became_peered":"2024-04-21T08:36:20.289868+0000","last_unstale":"2024-04-21T08:38:46.859069+0000","last_undegraded":"2024-04-21T08:38:46.859069+0000","last_fullsized":"2024-04-21T08:38:46.859069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:07:45.525980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913789+0000","last_change":"2024-04-21T08:36:20.899590+0000","last_active":"2024-04-21T08:38:46.913789+0000","last_peered":"2024-04-21T08:38:46.913789+0000","last_clean":"2024-04-21T08:38:46.913789+0000","last_became_active":"2024-04-21T08:36:20.899254+0000","last_became_peered":"2024-04-21T08:36:20.899254+0000","last_unstale":"2024-04-21T08:38:46.913789+0000","last_undegraded":"2024-04-21T08:38:46.913789+0000","last_fullsized":"2024-04-21T08:38:46.913789+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:34:25.409694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.750892+0000","last_change":"2024-04-21T08:36:24.918310+0000","last_active":"2024-04-21T08:38:46.750892+0000","last_peered":"2024-04-21T08:38:46.750892+0000","last_clean":"2024-04-21T08:38:46.750892+0000","last_became_active":"2024-04-21T08:36:24.917899+0000","last_became_peered":"2024-04-21T08:36:24.917899+0000","last_unstale":"2024-04-21T08:38:46.750892+0000","last_undegraded":"2024-04-21T08:38:46.750892+0000","last_fullsized":"2024-04-21T08:38:46.750892+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:09:13.006884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":51,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913309+0000","last_change":"2024-04-21T08:37:24.304544+0000","last_active":"2024-04-21T08:38:46.913309+0000","last_peered":"2024-04-21T08:38:46.913309+0000","last_clean":"2024-04-21T08:38:46.913309+0000","last_became_active":"2024-04-21T08:37:24.295808+0000","last_became_peered":"2024-04-21T08:37:24.295808+0000","last_unstale":"2024-04-21T08:38:46.913309+0000","last_undegraded":"2024-04-21T08:38:46.913309+0000","last_fullsized":"2024-04-21T08:38:46.913309+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:07:15.797133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914356+0000","last_change":"2024-04-21T08:36:20.912368+0000","last_active":"2024-04-21T08:38:46.914356+0000","last_peered":"2024-04-21T08:38:46.914356+0000","last_clean":"2024-04-21T08:38:46.914356+0000","last_became_active":"2024-04-21T08:36:20.912009+0000","last_became_peered":"2024-04-21T08:36:20.912009+0000","last_unstale":"2024-04-21T08:38:46.914356+0000","last_undegraded":"2024-04-21T08:38:46.914356+0000","last_fullsized":"2024-04-21T08:38:46.914356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:35:40.700314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807399+0000","last_change":"2024-04-21T08:36:20.913472+0000","last_active":"2024-04-21T08:38:46.807399+0000","last_peered":"2024-04-21T08:38:46.807399+0000","last_clean":"2024-04-21T08:38:46.807399+0000","last_became_active":"2024-04-21T08:36:20.913118+0000","last_became_peered":"2024-04-21T08:36:20.913118+0000","last_unstale":"2024-04-21T08:38:46.807399+0000","last_undegraded":"2024-04-21T08:38:46.807399+0000","last_fullsized":"2024-04-21T08:38:46.807399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:31:53.957044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638786+0000","last_change":"2024-04-21T08:36:20.913677+0000","last_active":"2024-04-21T08:38:46.638786+0000","last_peered":"2024-04-21T08:38:46.638786+0000","last_clean":"2024-04-21T08:38:46.638786+0000","last_became_active":"2024-04-21T08:36:20.913097+0000","last_became_peered":"2024-04-21T08:36:20.913097+0000","last_unstale":"2024-04-21T08:38:46.638786+0000","last_undegraded":"2024-04-21T08:38:46.638786+0000","last_fullsized":"2024-04-21T08:38:46.638786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:08:40.388370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638662+0000","last_change":"2024-04-21T08:36:20.906624+0000","last_active":"2024-04-21T08:38:46.638662+0000","last_peered":"2024-04-21T08:38:46.638662+0000","last_clean":"2024-04-21T08:38:46.638662+0000","last_became_active":"2024-04-21T08:36:20.906247+0000","last_became_peered":"2024-04-21T08:36:20.906247+0000","last_unstale":"2024-04-21T08:38:46.638662+0000","last_undegraded":"2024-04-21T08:38:46.638662+0000","last_fullsized":"2024-04-21T08:38:46.638662+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:44:11.855183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.916219+0000","last_change":"2024-04-21T08:36:20.906576+0000","last_active":"2024-04-21T08:38:46.916219+0000","last_peered":"2024-04-21T08:38:46.916219+0000","last_clean":"2024-04-21T08:38:46.916219+0000","last_became_active":"2024-04-21T08:36:20.906231+0000","last_became_peered":"2024-04-21T08:36:20.906231+0000","last_unstale":"2024-04-21T08:38:46.916219+0000","last_undegraded":"2024-04-21T08:38:46.916219+0000","last_fullsized":"2024-04-21T08:38:46.916219+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:42:48.156614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.912593+0000","last_change":"2024-04-21T08:36:20.902551+0000","last_active":"2024-04-21T08:38:46.912593+0000","last_peered":"2024-04-21T08:38:46.912593+0000","last_clean":"2024-04-21T08:38:46.912593+0000","last_became_active":"2024-04-21T08:36:20.902208+0000","last_became_peered":"2024-04-21T08:36:20.902208+0000","last_unstale":"2024-04-21T08:38:46.912593+0000","last_undegraded":"2024-04-21T08:38:46.912593+0000","last_fullsized":"2024-04-21T08:38:46.912593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:36:30.016256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807607+0000","last_change":"2024-04-21T08:36:20.918588+0000","last_active":"2024-04-21T08:38:46.807607+0000","last_peered":"2024-04-21T08:38:46.807607+0000","last_clean":"2024-04-21T08:38:46.807607+0000","last_became_active":"2024-04-21T08:36:20.918263+0000","last_became_peered":"2024-04-21T08:36:20.918263+0000","last_unstale":"2024-04-21T08:38:46.807607+0000","last_undegraded":"2024-04-21T08:38:46.807607+0000","last_fullsized":"2024-04-21T08:38:46.807607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:51:30.340336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639167+0000","last_change":"2024-04-21T08:36:20.911063+0000","last_active":"2024-04-21T08:38:46.639167+0000","last_peered":"2024-04-21T08:38:46.639167+0000","last_clean":"2024-04-21T08:38:46.639167+0000","last_became_active":"2024-04-21T08:36:20.910647+0000","last_became_peered":"2024-04-21T08:36:20.910647+0000","last_unstale":"2024-04-21T08:38:46.639167+0000","last_undegraded":"2024-04-21T08:38:46.639167+0000","last_fullsized":"2024-04-21T08:38:46.639167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:10:31.079782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639213+0000","last_change":"2024-04-21T08:36:20.914079+0000","last_active":"2024-04-21T08:38:46.639213+0000","last_peered":"2024-04-21T08:38:46.639213+0000","last_clean":"2024-04-21T08:38:46.639213+0000","last_became_active":"2024-04-21T08:36:20.912851+0000","last_became_peered":"2024-04-21T08:36:20.912851+0000","last_unstale":"2024-04-21T08:38:46.639213+0000","last_undegraded":"2024-04-21T08:38:46.639213+0000","last_fullsized":"2024-04-21T08:38:46.639213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:53:35.811281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751071+0000","last_change":"2024-04-21T08:36:20.900551+0000","last_active":"2024-04-21T08:38:46.751071+0000","last_peered":"2024-04-21T08:38:46.751071+0000","last_clean":"2024-04-21T08:38:46.751071+0000","last_became_active":"2024-04-21T08:36:20.900173+0000","last_became_peered":"2024-04-21T08:36:20.900173+0000","last_unstale":"2024-04-21T08:38:46.751071+0000","last_undegraded":"2024-04-21T08:38:46.751071+0000","last_fullsized":"2024-04-21T08:38:46.751071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:21.389498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638564+0000","last_change":"2024-04-21T08:36:20.912953+0000","last_active":"2024-04-21T08:38:46.638564+0000","last_peered":"2024-04-21T08:38:46.638564+0000","last_clean":"2024-04-21T08:38:46.638564+0000","last_became_active":"2024-04-21T08:36:20.912223+0000","last_became_peered":"2024-04-21T08:36:20.912223+0000","last_unstale":"2024-04-21T08:38:46.638564+0000","last_undegraded":"2024-04-21T08:38:46.638564+0000","last_fullsized":"2024-04-21T08:38:46.638564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:43:14.277179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.922633+0000","last_change":"2024-04-21T08:36:20.906164+0000","last_active":"2024-04-21T08:38:46.922633+0000","last_peered":"2024-04-21T08:38:46.922633+0000","last_clean":"2024-04-21T08:38:46.922633+0000","last_became_active":"2024-04-21T08:36:20.905749+0000","last_became_peered":"2024-04-21T08:36:20.905749+0000","last_unstale":"2024-04-21T08:38:46.922633+0000","last_undegraded":"2024-04-21T08:38:46.922633+0000","last_fullsized":"2024-04-21T08:38:46.922633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:02:10.569087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913644+0000","last_change":"2024-04-21T08:36:20.900314+0000","last_active":"2024-04-21T08:38:46.913644+0000","last_peered":"2024-04-21T08:38:46.913644+0000","last_clean":"2024-04-21T08:38:46.913644+0000","last_became_active":"2024-04-21T08:36:20.900009+0000","last_became_peered":"2024-04-21T08:36:20.900009+0000","last_unstale":"2024-04-21T08:38:46.913644+0000","last_undegraded":"2024-04-21T08:38:46.913644+0000","last_fullsized":"2024-04-21T08:38:46.913644+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:19:59.997302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807555+0000","last_change":"2024-04-21T08:36:20.920725+0000","last_active":"2024-04-21T08:38:46.807555+0000","last_peered":"2024-04-21T08:38:46.807555+0000","last_clean":"2024-04-21T08:38:46.807555+0000","last_became_active":"2024-04-21T08:36:20.920288+0000","last_became_peered":"2024-04-21T08:36:20.920288+0000","last_unstale":"2024-04-21T08:38:46.807555+0000","last_undegraded":"2024-04-21T08:38:46.807555+0000","last_fullsized":"2024-04-21T08:38:46.807555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:36:18.736388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":603992,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1402,"num_read_kb":818,"num_write":375,"num_write_kb":1473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":806,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1811976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":305,"ondisk_log_size":305,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383723,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1093632,"data_stored":531299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645359,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28952,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716200,"statfs":{"total":95995035648,"available":95965388800,"internally_reserved":0,"allocated":1921024,"data_stored":1323967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808885,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36804,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708348,"statfs":{"total":95995035648,"available":95957348352,"internally_reserved":0,"allocated":1499136,"data_stored":923219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070522,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28476,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716676,"statfs":{"total":95995035648,"available":95965876224,"internally_reserved":0,"allocated":1490944,"data_stored":923575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332160,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28672,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716480,"statfs":{"total":95995035648,"available":95965675520,"internally_reserved":0,"allocated":1691648,"data_stored":1127969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495685,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1081344,"data_stored":525507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36476,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708676,"statfs":{"total":95995035648,"available":95957684224,"internally_reserved":0,"allocated":1163264,"data_stored":596717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1105920,"data_stored":527495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36480,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708672,"statfs":{"total":95995035648,"available":95957680128,"internally_reserved":0,"allocated":1167360,"data_stored":598085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182483,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1050031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920847,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1089536,"data_stored":530895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757321,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37136,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708016,"statfs":{"total":95995035648,"available":95957008384,"internally_reserved":0,"allocated":1843200,"data_stored":1254509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":71932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":73300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":394694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":204718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T08:38:57.278 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-21T08:38:57.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.411+0000 7fe515ae4640 1 -- 172.21.15.86:0/445428667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe510105e30 msgr2=0x7fe510106210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:57.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.411+0000 7fe515ae4640 1 --2- 172.21.15.86:0/445428667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe510105e30 0x7fe510106210 secure :-1 s=READY pgs=585 cs=0 l=1 rev1=1 crypto rx=0x7fe504009930 tx=0x7fe50402bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:57.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.411+0000 7fe515ae4640 1 -- 172.21.15.86:0/445428667 shutdown_connections 2024-04-21T08:38:57.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.411+0000 7fe515ae4640 1 --2- 172.21.15.86:0/445428667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe510101b60 0x7fe51010dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.411+0000 7fe515ae4640 1 --2- 172.21.15.86:0/445428667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5101011a0 0x7fe510101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.411+0000 7fe515ae4640 1 --2- 172.21.15.86:0/445428667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe510105e30 0x7fe510106210 unknown :-1 s=CLOSED pgs=585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.411+0000 7fe515ae4640 1 -- 172.21.15.86:0/445428667 >> 172.21.15.86:0/445428667 conn(0x7fe510077e30 msgr2=0x7fe510078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:57.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.411+0000 7fe515ae4640 1 -- 172.21.15.86:0/445428667 shutdown_connections 2024-04-21T08:38:57.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.411+0000 7fe515ae4640 1 -- 172.21.15.86:0/445428667 wait complete. 2024-04-21T08:38:57.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.411+0000 7fe515ae4640 1 Processor -- start 2024-04-21T08:38:57.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.412+0000 7fe515ae4640 1 -- start start 2024-04-21T08:38:57.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.412+0000 7fe515ae4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5101011a0 0x7fe51019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:57.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.412+0000 7fe515ae4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe510101b60 0x7fe51019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:57.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.412+0000 7fe515ae4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe510105e30 0x7fe510198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:57.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.412+0000 7fe515ae4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe5101110c0 con 0x7fe510105e30 2024-04-21T08:38:57.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.412+0000 7fe515ae4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe510110f40 con 0x7fe510101b60 2024-04-21T08:38:57.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.412+0000 7fe515ae4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe510111240 con 0x7fe5101011a0 2024-04-21T08:38:57.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.412+0000 7fe50ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe510105e30 0x7fe510198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:57.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.412+0000 7fe50ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe510105e30 0x7fe510198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36550/0 (socket says 172.21.15.86:36550) 2024-04-21T08:38:57.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.412+0000 7fe50ffff640 1 -- 172.21.15.86:0/116184650 learned_addr learned my addr 172.21.15.86:0/116184650 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:57.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.412+0000 7fe50f7fe640 1 --2- 172.21.15.86:0/116184650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5101011a0 0x7fe51019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:57.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.412+0000 7fe50effd640 1 --2- 172.21.15.86:0/116184650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe510101b60 0x7fe51019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:57.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.412+0000 7fe50ffff640 1 -- 172.21.15.86:0/116184650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5101011a0 msgr2=0x7fe51019e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:57.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.412+0000 7fe50ffff640 1 --2- 172.21.15.86:0/116184650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5101011a0 0x7fe51019e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.412+0000 7fe50ffff640 1 -- 172.21.15.86:0/116184650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe510101b60 msgr2=0x7fe51019e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:57.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.412+0000 7fe50ffff640 1 --2- 172.21.15.86:0/116184650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe510101b60 0x7fe51019e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.412+0000 7fe50ffff640 1 -- 172.21.15.86:0/116184650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe510198d10 con 0x7fe510105e30 2024-04-21T08:38:57.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.412+0000 7fe50ffff640 1 --2- 172.21.15.86:0/116184650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe510105e30 0x7fe510198590 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe50800cad0 tx=0x7fe50800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:57.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.412+0000 7fe50cff9640 1 -- 172.21.15.86:0/116184650 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50800ae20 con 0x7fe510105e30 2024-04-21T08:38:57.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.413+0000 7fe50cff9640 1 -- 172.21.15.86:0/116184650 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe508005020 con 0x7fe510105e30 2024-04-21T08:38:57.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.413+0000 7fe515ae4640 1 -- 172.21.15.86:0/116184650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe510199000 con 0x7fe510105e30 2024-04-21T08:38:57.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.413+0000 7fe50cff9640 1 -- 172.21.15.86:0/116184650 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe50800f6b0 con 0x7fe510105e30 2024-04-21T08:38:57.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.413+0000 7fe515ae4640 1 -- 172.21.15.86:0/116184650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5101a52a0 con 0x7fe510105e30 2024-04-21T08:38:57.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.414+0000 7fe515ae4640 1 -- 172.21.15.86:0/116184650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe510102570 con 0x7fe510105e30 2024-04-21T08:38:57.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.415+0000 7fe50cff9640 1 -- 172.21.15.86:0/116184650 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fe5080051d0 con 0x7fe510105e30 2024-04-21T08:38:57.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.415+0000 7fe50cff9640 1 --2- 172.21.15.86:0/116184650 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe4f80a3db0 0x7fe4f80a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:57.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.415+0000 7fe50f7fe640 1 --2- 172.21.15.86:0/116184650 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe4f80a3db0 0x7fe4f80a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:57.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.415+0000 7fe50cff9640 1 -- 172.21.15.86:0/116184650 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fe5080d0dc0 con 0x7fe510105e30 2024-04-21T08:38:57.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.416+0000 7fe50f7fe640 1 --2- 172.21.15.86:0/116184650 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe4f80a3db0 0x7fe4f80a6270 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe504005ab0 tx=0x7fe504005fd0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:57.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.420+0000 7fe50cff9640 1 -- 172.21.15.86:0/116184650 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe508014030 con 0x7fe510105e30 2024-04-21T08:38:57.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.594+0000 7fe515ae4640 1 -- 172.21.15.86:0/116184650 --> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7fe510062390 con 0x7fe4f80a3db0 2024-04-21T08:38:57.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.603+0000 7fe50cff9640 1 -- 172.21.15.86:0/116184650 <== mgr.15558 v2:172.21.15.86:6800/2452032403 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339446 (secure 0 0 0) 0x7fe510062390 con 0x7fe4f80a3db0 2024-04-21T08:38:57.607 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:38:57.608 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-21T08:38:57.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.606+0000 7fe515ae4640 1 -- 172.21.15.86:0/116184650 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe4f80a3db0 msgr2=0x7fe4f80a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:57.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.606+0000 7fe515ae4640 1 --2- 172.21.15.86:0/116184650 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe4f80a3db0 0x7fe4f80a6270 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7fe504005ab0 tx=0x7fe504005fd0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.607+0000 7fe515ae4640 1 -- 172.21.15.86:0/116184650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe510105e30 msgr2=0x7fe510198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:57.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.607+0000 7fe515ae4640 1 --2- 172.21.15.86:0/116184650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe510105e30 0x7fe510198590 secure :-1 s=READY pgs=586 cs=0 l=1 rev1=1 crypto rx=0x7fe50800cad0 tx=0x7fe50800cf90 comp rx=0 tx=0).stop 2024-04-21T08:38:57.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.607+0000 7fe515ae4640 1 -- 172.21.15.86:0/116184650 shutdown_connections 2024-04-21T08:38:57.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.607+0000 7fe515ae4640 1 --2- 172.21.15.86:0/116184650 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe4f80a3db0 0x7fe4f80a6270 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.607+0000 7fe515ae4640 1 --2- 172.21.15.86:0/116184650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe510105e30 0x7fe510198590 unknown :-1 s=CLOSED pgs=586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.607+0000 7fe515ae4640 1 --2- 172.21.15.86:0/116184650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe510101b60 0x7fe51019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.607+0000 7fe515ae4640 1 --2- 172.21.15.86:0/116184650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5101011a0 0x7fe51019e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.607+0000 7fe515ae4640 1 -- 172.21.15.86:0/116184650 >> 172.21.15.86:0/116184650 conn(0x7fe510077e30 msgr2=0x7fe510069480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:57.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.607+0000 7fe515ae4640 1 -- 172.21.15.86:0/116184650 shutdown_connections 2024-04-21T08:38:57.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.607+0000 7fe515ae4640 1 -- 172.21.15.86:0/116184650 wait complete. 2024-04-21T08:38:57.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:57 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/335669794' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:57.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:57 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2229459377' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:57.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:57 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/126889366' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:57.618 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-04-21T08:38:56.653889+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1194594,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1639,"num_read_kb":1148,"num_write":578,"num_write_kb":4413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":1181659,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":955,"ondisk_log_size":955,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":390584,"kb_used_data":16376,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124551240,"statfs":{"total":1151940427776,"available":1151540469760,"internally_reserved":0,"allocated":16769024,"data_stored":9913268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":382973244},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":153,"num_read_kb":238,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.993736"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807854+0000","last_change":"2024-04-21T08:36:20.913421+0000","last_active":"2024-04-21T08:38:46.807854+0000","last_peered":"2024-04-21T08:38:46.807854+0000","last_clean":"2024-04-21T08:38:46.807854+0000","last_became_active":"2024-04-21T08:36:20.913017+0000","last_became_peered":"2024-04-21T08:36:20.913017+0000","last_unstale":"2024-04-21T08:38:46.807854+0000","last_undegraded":"2024-04-21T08:38:46.807854+0000","last_fullsized":"2024-04-21T08:38:46.807854+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:16:29.885916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.915359+0000","last_change":"2024-04-21T08:36:20.911675+0000","last_active":"2024-04-21T08:38:46.915359+0000","last_peered":"2024-04-21T08:38:46.915359+0000","last_clean":"2024-04-21T08:38:46.915359+0000","last_became_active":"2024-04-21T08:36:20.910591+0000","last_became_peered":"2024-04-21T08:36:20.910591+0000","last_unstale":"2024-04-21T08:38:46.915359+0000","last_undegraded":"2024-04-21T08:38:46.915359+0000","last_fullsized":"2024-04-21T08:38:46.915359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:01:40.735376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.915070+0000","last_change":"2024-04-21T08:36:20.911978+0000","last_active":"2024-04-21T08:38:46.915070+0000","last_peered":"2024-04-21T08:38:46.915070+0000","last_clean":"2024-04-21T08:38:46.915070+0000","last_became_active":"2024-04-21T08:36:20.911641+0000","last_became_peered":"2024-04-21T08:36:20.911641+0000","last_unstale":"2024-04-21T08:38:46.915070+0000","last_undegraded":"2024-04-21T08:38:46.915070+0000","last_fullsized":"2024-04-21T08:38:46.915070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:14:23.789790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911901+0000","last_change":"2024-04-21T08:36:20.919027+0000","last_active":"2024-04-21T08:38:46.911901+0000","last_peered":"2024-04-21T08:38:46.911901+0000","last_clean":"2024-04-21T08:38:46.911901+0000","last_became_active":"2024-04-21T08:36:20.918740+0000","last_became_peered":"2024-04-21T08:36:20.918740+0000","last_unstale":"2024-04-21T08:38:46.911901+0000","last_undegraded":"2024-04-21T08:38:46.911901+0000","last_fullsized":"2024-04-21T08:38:46.911901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:00:31.066938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655099+0000","last_change":"2024-04-21T08:36:20.907175+0000","last_active":"2024-04-21T08:38:46.655099+0000","last_peered":"2024-04-21T08:38:46.655099+0000","last_clean":"2024-04-21T08:38:46.655099+0000","last_became_active":"2024-04-21T08:36:20.906818+0000","last_became_peered":"2024-04-21T08:36:20.906818+0000","last_unstale":"2024-04-21T08:38:46.655099+0000","last_undegraded":"2024-04-21T08:38:46.655099+0000","last_fullsized":"2024-04-21T08:38:46.655099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:41:03.332231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638267+0000","last_change":"2024-04-21T08:36:20.905968+0000","last_active":"2024-04-21T08:38:46.638267+0000","last_peered":"2024-04-21T08:38:46.638267+0000","last_clean":"2024-04-21T08:38:46.638267+0000","last_became_active":"2024-04-21T08:36:20.904840+0000","last_became_peered":"2024-04-21T08:36:20.904840+0000","last_unstale":"2024-04-21T08:38:46.638267+0000","last_undegraded":"2024-04-21T08:38:46.638267+0000","last_fullsized":"2024-04-21T08:38:46.638267+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:23:46.886241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913625+0000","last_change":"2024-04-21T08:36:20.912045+0000","last_active":"2024-04-21T08:38:46.913625+0000","last_peered":"2024-04-21T08:38:46.913625+0000","last_clean":"2024-04-21T08:38:46.913625+0000","last_became_active":"2024-04-21T08:36:20.911690+0000","last_became_peered":"2024-04-21T08:36:20.911690+0000","last_unstale":"2024-04-21T08:38:46.913625+0000","last_undegraded":"2024-04-21T08:38:46.913625+0000","last_fullsized":"2024-04-21T08:38:46.913625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:06:28.229307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638674+0000","last_change":"2024-04-21T08:36:20.904874+0000","last_active":"2024-04-21T08:38:46.638674+0000","last_peered":"2024-04-21T08:38:46.638674+0000","last_clean":"2024-04-21T08:38:46.638674+0000","last_became_active":"2024-04-21T08:36:20.904217+0000","last_became_peered":"2024-04-21T08:36:20.904217+0000","last_unstale":"2024-04-21T08:38:46.638674+0000","last_undegraded":"2024-04-21T08:38:46.638674+0000","last_fullsized":"2024-04-21T08:38:46.638674+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:37:04.423812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.923169+0000","last_change":"2024-04-21T08:36:20.912874+0000","last_active":"2024-04-21T08:38:46.923169+0000","last_peered":"2024-04-21T08:38:46.923169+0000","last_clean":"2024-04-21T08:38:46.923169+0000","last_became_active":"2024-04-21T08:36:20.912569+0000","last_became_peered":"2024-04-21T08:36:20.912569+0000","last_unstale":"2024-04-21T08:38:46.923169+0000","last_undegraded":"2024-04-21T08:38:46.923169+0000","last_fullsized":"2024-04-21T08:38:46.923169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:55:20.763157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":53,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638546+0000","last_change":"2024-04-21T08:37:24.296916+0000","last_active":"2024-04-21T08:38:46.638546+0000","last_peered":"2024-04-21T08:38:46.638546+0000","last_clean":"2024-04-21T08:38:46.638546+0000","last_became_active":"2024-04-21T08:37:24.296547+0000","last_became_peered":"2024-04-21T08:37:24.296547+0000","last_unstale":"2024-04-21T08:38:46.638546+0000","last_undegraded":"2024-04-21T08:38:46.638546+0000","last_fullsized":"2024-04-21T08:38:46.638546+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:47:56.683793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914246+0000","last_change":"2024-04-21T08:36:20.909094+0000","last_active":"2024-04-21T08:38:46.914246+0000","last_peered":"2024-04-21T08:38:46.914246+0000","last_clean":"2024-04-21T08:38:46.914246+0000","last_became_active":"2024-04-21T08:36:20.908634+0000","last_became_peered":"2024-04-21T08:36:20.908634+0000","last_unstale":"2024-04-21T08:38:46.914246+0000","last_undegraded":"2024-04-21T08:38:46.914246+0000","last_fullsized":"2024-04-21T08:38:46.914246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:28:38.450621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911748+0000","last_change":"2024-04-21T08:36:20.909429+0000","last_active":"2024-04-21T08:38:46.911748+0000","last_peered":"2024-04-21T08:38:46.911748+0000","last_clean":"2024-04-21T08:38:46.911748+0000","last_became_active":"2024-04-21T08:36:20.909081+0000","last_became_peered":"2024-04-21T08:36:20.909081+0000","last_unstale":"2024-04-21T08:38:46.911748+0000","last_undegraded":"2024-04-21T08:38:46.911748+0000","last_fullsized":"2024-04-21T08:38:46.911748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:45:46.939125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913919+0000","last_change":"2024-04-21T08:36:20.895894+0000","last_active":"2024-04-21T08:38:46.913919+0000","last_peered":"2024-04-21T08:38:46.913919+0000","last_clean":"2024-04-21T08:38:46.913919+0000","last_became_active":"2024-04-21T08:36:20.895572+0000","last_became_peered":"2024-04-21T08:36:20.895572+0000","last_unstale":"2024-04-21T08:38:46.913919+0000","last_undegraded":"2024-04-21T08:38:46.913919+0000","last_fullsized":"2024-04-21T08:38:46.913919+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:12:48.750555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914638+0000","last_change":"2024-04-21T08:36:20.905356+0000","last_active":"2024-04-21T08:38:46.914638+0000","last_peered":"2024-04-21T08:38:46.914638+0000","last_clean":"2024-04-21T08:38:46.914638+0000","last_became_active":"2024-04-21T08:36:20.904954+0000","last_became_peered":"2024-04-21T08:36:20.904954+0000","last_unstale":"2024-04-21T08:38:46.914638+0000","last_undegraded":"2024-04-21T08:38:46.914638+0000","last_fullsized":"2024-04-21T08:38:46.914638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:45:49.116482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.915244+0000","last_change":"2024-04-21T08:36:20.911550+0000","last_active":"2024-04-21T08:38:46.915244+0000","last_peered":"2024-04-21T08:38:46.915244+0000","last_clean":"2024-04-21T08:38:46.915244+0000","last_became_active":"2024-04-21T08:36:20.910557+0000","last_became_peered":"2024-04-21T08:36:20.910557+0000","last_unstale":"2024-04-21T08:38:46.915244+0000","last_undegraded":"2024-04-21T08:38:46.915244+0000","last_fullsized":"2024-04-21T08:38:46.915244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:09:49.422729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.922413+0000","last_change":"2024-04-21T08:36:20.903954+0000","last_active":"2024-04-21T08:38:46.922413+0000","last_peered":"2024-04-21T08:38:46.922413+0000","last_clean":"2024-04-21T08:38:46.922413+0000","last_became_active":"2024-04-21T08:36:20.903641+0000","last_became_peered":"2024-04-21T08:36:20.903641+0000","last_unstale":"2024-04-21T08:38:46.922413+0000","last_undegraded":"2024-04-21T08:38:46.922413+0000","last_fullsized":"2024-04-21T08:38:46.922413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:08:55.678940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807825+0000","last_change":"2024-04-21T08:36:20.914256+0000","last_active":"2024-04-21T08:38:46.807825+0000","last_peered":"2024-04-21T08:38:46.807825+0000","last_clean":"2024-04-21T08:38:46.807825+0000","last_became_active":"2024-04-21T08:36:20.913858+0000","last_became_peered":"2024-04-21T08:36:20.913858+0000","last_unstale":"2024-04-21T08:38:46.807825+0000","last_undegraded":"2024-04-21T08:38:46.807825+0000","last_fullsized":"2024-04-21T08:38:46.807825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:01:10.276763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.743080+0000","last_change":"2024-04-21T08:36:20.904138+0000","last_active":"2024-04-21T08:38:46.743080+0000","last_peered":"2024-04-21T08:38:46.743080+0000","last_clean":"2024-04-21T08:38:46.743080+0000","last_became_active":"2024-04-21T08:36:20.903670+0000","last_became_peered":"2024-04-21T08:36:20.903670+0000","last_unstale":"2024-04-21T08:38:46.743080+0000","last_undegraded":"2024-04-21T08:38:46.743080+0000","last_fullsized":"2024-04-21T08:38:46.743080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:38:50.160867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639071+0000","last_change":"2024-04-21T08:36:20.910798+0000","last_active":"2024-04-21T08:38:46.639071+0000","last_peered":"2024-04-21T08:38:46.639071+0000","last_clean":"2024-04-21T08:38:46.639071+0000","last_became_active":"2024-04-21T08:36:20.910411+0000","last_became_peered":"2024-04-21T08:36:20.910411+0000","last_unstale":"2024-04-21T08:38:46.639071+0000","last_undegraded":"2024-04-21T08:38:46.639071+0000","last_fullsized":"2024-04-21T08:38:46.639071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:13:12.109818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638676+0000","last_change":"2024-04-21T08:36:24.924831+0000","last_active":"2024-04-21T08:38:46.638676+0000","last_peered":"2024-04-21T08:38:46.638676+0000","last_clean":"2024-04-21T08:38:46.638676+0000","last_became_active":"2024-04-21T08:36:24.924435+0000","last_became_peered":"2024-04-21T08:36:24.924435+0000","last_unstale":"2024-04-21T08:38:46.638676+0000","last_undegraded":"2024-04-21T08:38:46.638676+0000","last_fullsized":"2024-04-21T08:38:46.638676+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:23.447079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":194,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:52.412380+0000","last_change":"2024-04-21T08:37:24.308024+0000","last_active":"2024-04-21T08:38:52.412380+0000","last_peered":"2024-04-21T08:38:52.412380+0000","last_clean":"2024-04-21T08:38:52.412380+0000","last_became_active":"2024-04-21T08:37:24.297491+0000","last_became_peered":"2024-04-21T08:37:24.297491+0000","last_unstale":"2024-04-21T08:38:52.412380+0000","last_undegraded":"2024-04-21T08:38:52.412380+0000","last_fullsized":"2024-04-21T08:38:52.412380+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:48:10.039545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639158+0000","last_change":"2024-04-21T08:36:20.912851+0000","last_active":"2024-04-21T08:38:46.639158+0000","last_peered":"2024-04-21T08:38:46.639158+0000","last_clean":"2024-04-21T08:38:46.639158+0000","last_became_active":"2024-04-21T08:36:20.912407+0000","last_became_peered":"2024-04-21T08:36:20.912407+0000","last_unstale":"2024-04-21T08:38:46.639158+0000","last_undegraded":"2024-04-21T08:38:46.639158+0000","last_fullsized":"2024-04-21T08:38:46.639158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:45:18.026940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.910950+0000","last_change":"2024-04-21T08:36:24.924052+0000","last_active":"2024-04-21T08:38:46.910950+0000","last_peered":"2024-04-21T08:38:46.910950+0000","last_clean":"2024-04-21T08:38:46.910950+0000","last_became_active":"2024-04-21T08:36:24.923618+0000","last_became_peered":"2024-04-21T08:36:24.923618+0000","last_unstale":"2024-04-21T08:38:46.910950+0000","last_undegraded":"2024-04-21T08:38:46.910950+0000","last_fullsized":"2024-04-21T08:38:46.910950+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:08:18.182386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639180+0000","last_change":"2024-04-21T08:36:20.285291+0000","last_active":"2024-04-21T08:38:46.639180+0000","last_peered":"2024-04-21T08:38:46.639180+0000","last_clean":"2024-04-21T08:38:46.639180+0000","last_became_active":"2024-04-21T08:36:20.284969+0000","last_became_peered":"2024-04-21T08:36:20.284969+0000","last_unstale":"2024-04-21T08:38:46.639180+0000","last_undegraded":"2024-04-21T08:38:46.639180+0000","last_fullsized":"2024-04-21T08:38:46.639180+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:55:29.484439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.915781+0000","last_change":"2024-04-21T08:36:20.902819+0000","last_active":"2024-04-21T08:38:46.915781+0000","last_peered":"2024-04-21T08:38:46.915781+0000","last_clean":"2024-04-21T08:38:46.915781+0000","last_became_active":"2024-04-21T08:36:20.902466+0000","last_became_peered":"2024-04-21T08:36:20.902466+0000","last_unstale":"2024-04-21T08:38:46.915781+0000","last_undegraded":"2024-04-21T08:38:46.915781+0000","last_fullsized":"2024-04-21T08:38:46.915781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:29:40.454026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911156+0000","last_change":"2024-04-21T08:36:24.926480+0000","last_active":"2024-04-21T08:38:46.911156+0000","last_peered":"2024-04-21T08:38:46.911156+0000","last_clean":"2024-04-21T08:38:46.911156+0000","last_became_active":"2024-04-21T08:36:24.926116+0000","last_became_peered":"2024-04-21T08:36:24.926116+0000","last_unstale":"2024-04-21T08:38:46.911156+0000","last_undegraded":"2024-04-21T08:38:46.911156+0000","last_fullsized":"2024-04-21T08:38:46.911156+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:43:18.839396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655608+0000","last_change":"2024-04-21T08:36:20.293086+0000","last_active":"2024-04-21T08:38:46.655608+0000","last_peered":"2024-04-21T08:38:46.655608+0000","last_clean":"2024-04-21T08:38:46.655608+0000","last_became_active":"2024-04-21T08:36:20.291178+0000","last_became_peered":"2024-04-21T08:36:20.291178+0000","last_unstale":"2024-04-21T08:38:46.655608+0000","last_undegraded":"2024-04-21T08:38:46.655608+0000","last_fullsized":"2024-04-21T08:38:46.655608+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:02:32.935930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.922777+0000","last_change":"2024-04-21T08:36:20.909374+0000","last_active":"2024-04-21T08:38:46.922777+0000","last_peered":"2024-04-21T08:38:46.922777+0000","last_clean":"2024-04-21T08:38:46.922777+0000","last_became_active":"2024-04-21T08:36:20.909043+0000","last_became_peered":"2024-04-21T08:36:20.909043+0000","last_unstale":"2024-04-21T08:38:46.922777+0000","last_undegraded":"2024-04-21T08:38:46.922777+0000","last_fullsized":"2024-04-21T08:38:46.922777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:45:45.719942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911098+0000","last_change":"2024-04-21T08:38:24.623194+0000","last_active":"2024-04-21T08:38:46.911098+0000","last_peered":"2024-04-21T08:38:46.911098+0000","last_clean":"2024-04-21T08:38:46.911098+0000","last_became_active":"2024-04-21T08:38:24.622632+0000","last_became_peered":"2024-04-21T08:38:24.622632+0000","last_unstale":"2024-04-21T08:38:46.911098+0000","last_undegraded":"2024-04-21T08:38:46.911098+0000","last_fullsized":"2024-04-21T08:38:46.911098+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:26:39.752161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655483+0000","last_change":"2024-04-21T08:36:20.288644+0000","last_active":"2024-04-21T08:38:46.655483+0000","last_peered":"2024-04-21T08:38:46.655483+0000","last_clean":"2024-04-21T08:38:46.655483+0000","last_became_active":"2024-04-21T08:36:20.288069+0000","last_became_peered":"2024-04-21T08:36:20.288069+0000","last_unstale":"2024-04-21T08:38:46.655483+0000","last_undegraded":"2024-04-21T08:38:46.655483+0000","last_fullsized":"2024-04-21T08:38:46.655483+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:37:36.522713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751431+0000","last_change":"2024-04-21T08:36:20.908505+0000","last_active":"2024-04-21T08:38:46.751431+0000","last_peered":"2024-04-21T08:38:46.751431+0000","last_clean":"2024-04-21T08:38:46.751431+0000","last_became_active":"2024-04-21T08:36:20.908163+0000","last_became_peered":"2024-04-21T08:36:20.908163+0000","last_unstale":"2024-04-21T08:38:46.751431+0000","last_undegraded":"2024-04-21T08:38:46.751431+0000","last_fullsized":"2024-04-21T08:38:46.751431+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:54:18.530203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639160+0000","last_change":"2024-04-21T08:36:24.919428+0000","last_active":"2024-04-21T08:38:46.639160+0000","last_peered":"2024-04-21T08:38:46.639160+0000","last_clean":"2024-04-21T08:38:46.639160+0000","last_became_active":"2024-04-21T08:36:24.918722+0000","last_became_peered":"2024-04-21T08:36:24.918722+0000","last_unstale":"2024-04-21T08:38:46.639160+0000","last_undegraded":"2024-04-21T08:38:46.639160+0000","last_fullsized":"2024-04-21T08:38:46.639160+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:30:32.369551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"93'8","reported_seq":174,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:52.641167+0000","last_change":"2024-04-21T08:36:20.287642+0000","last_active":"2024-04-21T08:38:52.641167+0000","last_peered":"2024-04-21T08:38:52.641167+0000","last_clean":"2024-04-21T08:38:52.641167+0000","last_became_active":"2024-04-21T08:36:20.287296+0000","last_became_peered":"2024-04-21T08:36:20.287296+0000","last_unstale":"2024-04-21T08:38:52.641167+0000","last_undegraded":"2024-04-21T08:38:52.641167+0000","last_fullsized":"2024-04-21T08:38:52.641167+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:59:28.579020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1462,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":133,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655128+0000","last_change":"2024-04-21T08:36:20.916971+0000","last_active":"2024-04-21T08:38:46.655128+0000","last_peered":"2024-04-21T08:38:46.655128+0000","last_clean":"2024-04-21T08:38:46.655128+0000","last_became_active":"2024-04-21T08:36:20.916622+0000","last_became_peered":"2024-04-21T08:36:20.916622+0000","last_unstale":"2024-04-21T08:38:46.655128+0000","last_undegraded":"2024-04-21T08:38:46.655128+0000","last_fullsized":"2024-04-21T08:38:46.655128+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:49:38.858772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638856+0000","last_change":"2024-04-21T08:36:24.924170+0000","last_active":"2024-04-21T08:38:46.638856+0000","last_peered":"2024-04-21T08:38:46.638856+0000","last_clean":"2024-04-21T08:38:46.638856+0000","last_became_active":"2024-04-21T08:36:24.923744+0000","last_became_peered":"2024-04-21T08:36:24.923744+0000","last_unstale":"2024-04-21T08:38:46.638856+0000","last_undegraded":"2024-04-21T08:38:46.638856+0000","last_fullsized":"2024-04-21T08:38:46.638856+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:23:24.605021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638388+0000","last_change":"2024-04-21T08:36:20.286545+0000","last_active":"2024-04-21T08:38:46.638388+0000","last_peered":"2024-04-21T08:38:46.638388+0000","last_clean":"2024-04-21T08:38:46.638388+0000","last_became_active":"2024-04-21T08:36:20.286197+0000","last_became_peered":"2024-04-21T08:36:20.286197+0000","last_unstale":"2024-04-21T08:38:46.638388+0000","last_undegraded":"2024-04-21T08:38:46.638388+0000","last_fullsized":"2024-04-21T08:38:46.638388+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:27:55.430428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751535+0000","last_change":"2024-04-21T08:36:20.907743+0000","last_active":"2024-04-21T08:38:46.751535+0000","last_peered":"2024-04-21T08:38:46.751535+0000","last_clean":"2024-04-21T08:38:46.751535+0000","last_became_active":"2024-04-21T08:36:20.907163+0000","last_became_peered":"2024-04-21T08:36:20.907163+0000","last_unstale":"2024-04-21T08:38:46.751535+0000","last_undegraded":"2024-04-21T08:38:46.751535+0000","last_fullsized":"2024-04-21T08:38:46.751535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:41:00.172353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807766+0000","last_change":"2024-04-21T08:37:24.301529+0000","last_active":"2024-04-21T08:38:46.807766+0000","last_peered":"2024-04-21T08:38:46.807766+0000","last_clean":"2024-04-21T08:38:46.807766+0000","last_became_active":"2024-04-21T08:37:24.301014+0000","last_became_peered":"2024-04-21T08:37:24.301014+0000","last_unstale":"2024-04-21T08:38:46.807766+0000","last_undegraded":"2024-04-21T08:38:46.807766+0000","last_fullsized":"2024-04-21T08:38:46.807766+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:58:20.088501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.912093+0000","last_change":"2024-04-21T08:36:20.283143+0000","last_active":"2024-04-21T08:38:46.912093+0000","last_peered":"2024-04-21T08:38:46.912093+0000","last_clean":"2024-04-21T08:38:46.912093+0000","last_became_active":"2024-04-21T08:36:20.282585+0000","last_became_peered":"2024-04-21T08:36:20.282585+0000","last_unstale":"2024-04-21T08:38:46.912093+0000","last_undegraded":"2024-04-21T08:38:46.912093+0000","last_fullsized":"2024-04-21T08:38:46.912093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:12:37.588764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638543+0000","last_change":"2024-04-21T08:36:20.912591+0000","last_active":"2024-04-21T08:38:46.638543+0000","last_peered":"2024-04-21T08:38:46.638543+0000","last_clean":"2024-04-21T08:38:46.638543+0000","last_became_active":"2024-04-21T08:36:20.912174+0000","last_became_peered":"2024-04-21T08:36:20.912174+0000","last_unstale":"2024-04-21T08:38:46.638543+0000","last_undegraded":"2024-04-21T08:38:46.638543+0000","last_fullsized":"2024-04-21T08:38:46.638543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:33:58.949506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.915307+0000","last_change":"2024-04-21T08:36:24.915998+0000","last_active":"2024-04-21T08:38:46.915307+0000","last_peered":"2024-04-21T08:38:46.915307+0000","last_clean":"2024-04-21T08:38:46.915307+0000","last_became_active":"2024-04-21T08:36:24.915621+0000","last_became_peered":"2024-04-21T08:36:24.915621+0000","last_unstale":"2024-04-21T08:38:46.915307+0000","last_undegraded":"2024-04-21T08:38:46.915307+0000","last_fullsized":"2024-04-21T08:38:46.915307+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:43:05.117181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751087+0000","last_change":"2024-04-21T08:36:20.286308+0000","last_active":"2024-04-21T08:38:46.751087+0000","last_peered":"2024-04-21T08:38:46.751087+0000","last_clean":"2024-04-21T08:38:46.751087+0000","last_became_active":"2024-04-21T08:36:20.285975+0000","last_became_peered":"2024-04-21T08:36:20.285975+0000","last_unstale":"2024-04-21T08:38:46.751087+0000","last_undegraded":"2024-04-21T08:38:46.751087+0000","last_fullsized":"2024-04-21T08:38:46.751087+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:55:10.658349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.742821+0000","last_change":"2024-04-21T08:36:20.900502+0000","last_active":"2024-04-21T08:38:46.742821+0000","last_peered":"2024-04-21T08:38:46.742821+0000","last_clean":"2024-04-21T08:38:46.742821+0000","last_became_active":"2024-04-21T08:36:20.900199+0000","last_became_peered":"2024-04-21T08:36:20.900199+0000","last_unstale":"2024-04-21T08:38:46.742821+0000","last_undegraded":"2024-04-21T08:38:46.742821+0000","last_fullsized":"2024-04-21T08:38:46.742821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:17:00.720918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638804+0000","last_change":"2024-04-21T08:36:24.920991+0000","last_active":"2024-04-21T08:38:46.638804+0000","last_peered":"2024-04-21T08:38:46.638804+0000","last_clean":"2024-04-21T08:38:46.638804+0000","last_became_active":"2024-04-21T08:36:24.916611+0000","last_became_peered":"2024-04-21T08:36:24.916611+0000","last_unstale":"2024-04-21T08:38:46.638804+0000","last_undegraded":"2024-04-21T08:38:46.638804+0000","last_fullsized":"2024-04-21T08:38:46.638804+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:01:01.412127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"93'7","reported_seq":293,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:51.661122+0000","last_change":"2024-04-21T08:36:20.284966+0000","last_active":"2024-04-21T08:38:51.661122+0000","last_peered":"2024-04-21T08:38:51.661122+0000","last_clean":"2024-04-21T08:38:51.661122+0000","last_became_active":"2024-04-21T08:36:20.284612+0000","last_became_peered":"2024-04-21T08:36:20.284612+0000","last_unstale":"2024-04-21T08:38:51.661122+0000","last_undegraded":"2024-04-21T08:38:51.661122+0000","last_fullsized":"2024-04-21T08:38:51.661122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:54:50.095013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":253,"num_read_kb":253,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.923175+0000","last_change":"2024-04-21T08:36:20.903493+0000","last_active":"2024-04-21T08:38:46.923175+0000","last_peered":"2024-04-21T08:38:46.923175+0000","last_clean":"2024-04-21T08:38:46.923175+0000","last_became_active":"2024-04-21T08:36:20.902752+0000","last_became_peered":"2024-04-21T08:36:20.902752+0000","last_unstale":"2024-04-21T08:38:46.923175+0000","last_undegraded":"2024-04-21T08:38:46.923175+0000","last_fullsized":"2024-04-21T08:38:46.923175+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:40.684264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.742602+0000","last_change":"2024-04-21T08:36:24.920505+0000","last_active":"2024-04-21T08:38:46.742602+0000","last_peered":"2024-04-21T08:38:46.742602+0000","last_clean":"2024-04-21T08:38:46.742602+0000","last_became_active":"2024-04-21T08:36:24.919966+0000","last_became_peered":"2024-04-21T08:36:24.919966+0000","last_unstale":"2024-04-21T08:38:46.742602+0000","last_undegraded":"2024-04-21T08:38:46.742602+0000","last_fullsized":"2024-04-21T08:38:46.742602+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:58:05.131815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914242+0000","last_change":"2024-04-21T08:36:20.281720+0000","last_active":"2024-04-21T08:38:46.914242+0000","last_peered":"2024-04-21T08:38:46.914242+0000","last_clean":"2024-04-21T08:38:46.914242+0000","last_became_active":"2024-04-21T08:36:20.279862+0000","last_became_peered":"2024-04-21T08:36:20.279862+0000","last_unstale":"2024-04-21T08:38:46.914242+0000","last_undegraded":"2024-04-21T08:38:46.914242+0000","last_fullsized":"2024-04-21T08:38:46.914242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:18:58.106532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.916030+0000","last_change":"2024-04-21T08:36:20.908520+0000","last_active":"2024-04-21T08:38:46.916030+0000","last_peered":"2024-04-21T08:38:46.916030+0000","last_clean":"2024-04-21T08:38:46.916030+0000","last_became_active":"2024-04-21T08:36:20.908184+0000","last_became_peered":"2024-04-21T08:36:20.908184+0000","last_unstale":"2024-04-21T08:38:46.916030+0000","last_undegraded":"2024-04-21T08:38:46.916030+0000","last_fullsized":"2024-04-21T08:38:46.916030+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:14:33.895864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911465+0000","last_change":"2024-04-21T08:36:24.920697+0000","last_active":"2024-04-21T08:38:46.911465+0000","last_peered":"2024-04-21T08:38:46.911465+0000","last_clean":"2024-04-21T08:38:46.911465+0000","last_became_active":"2024-04-21T08:36:24.919976+0000","last_became_peered":"2024-04-21T08:36:24.919976+0000","last_unstale":"2024-04-21T08:38:46.911465+0000","last_undegraded":"2024-04-21T08:38:46.911465+0000","last_fullsized":"2024-04-21T08:38:46.911465+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:40:09.238887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751422+0000","last_change":"2024-04-21T08:36:20.283531+0000","last_active":"2024-04-21T08:38:46.751422+0000","last_peered":"2024-04-21T08:38:46.751422+0000","last_clean":"2024-04-21T08:38:46.751422+0000","last_became_active":"2024-04-21T08:36:20.283159+0000","last_became_peered":"2024-04-21T08:36:20.283159+0000","last_unstale":"2024-04-21T08:38:46.751422+0000","last_undegraded":"2024-04-21T08:38:46.751422+0000","last_fullsized":"2024-04-21T08:38:46.751422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:57:02.917537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.922897+0000","last_change":"2024-04-21T08:36:20.910113+0000","last_active":"2024-04-21T08:38:46.922897+0000","last_peered":"2024-04-21T08:38:46.922897+0000","last_clean":"2024-04-21T08:38:46.922897+0000","last_became_active":"2024-04-21T08:36:20.909744+0000","last_became_peered":"2024-04-21T08:36:20.909744+0000","last_unstale":"2024-04-21T08:38:46.922897+0000","last_undegraded":"2024-04-21T08:38:46.922897+0000","last_fullsized":"2024-04-21T08:38:46.922897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:16:14.568318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.742678+0000","last_change":"2024-04-21T08:36:24.922232+0000","last_active":"2024-04-21T08:38:46.742678+0000","last_peered":"2024-04-21T08:38:46.742678+0000","last_clean":"2024-04-21T08:38:46.742678+0000","last_became_active":"2024-04-21T08:36:24.921690+0000","last_became_peered":"2024-04-21T08:36:24.921690+0000","last_unstale":"2024-04-21T08:38:46.742678+0000","last_undegraded":"2024-04-21T08:38:46.742678+0000","last_fullsized":"2024-04-21T08:38:46.742678+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:21:42.580203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'42","reported_seq":343,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:52.640372+0000","last_change":"2024-04-21T08:36:20.283574+0000","last_active":"2024-04-21T08:38:52.640372+0000","last_peered":"2024-04-21T08:38:52.640372+0000","last_clean":"2024-04-21T08:38:52.640372+0000","last_became_active":"2024-04-21T08:36:20.283220+0000","last_became_peered":"2024-04-21T08:36:20.283220+0000","last_unstale":"2024-04-21T08:38:52.640372+0000","last_undegraded":"2024-04-21T08:38:52.640372+0000","last_fullsized":"2024-04-21T08:38:52.640372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:55:24.233237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70068,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":268,"num_read_kb":203,"num_write":44,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655458+0000","last_change":"2024-04-21T08:36:20.919264+0000","last_active":"2024-04-21T08:38:46.655458+0000","last_peered":"2024-04-21T08:38:46.655458+0000","last_clean":"2024-04-21T08:38:46.655458+0000","last_became_active":"2024-04-21T08:36:20.918878+0000","last_became_peered":"2024-04-21T08:36:20.918878+0000","last_unstale":"2024-04-21T08:38:46.655458+0000","last_undegraded":"2024-04-21T08:38:46.655458+0000","last_fullsized":"2024-04-21T08:38:46.655458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:24:06.460758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751721+0000","last_change":"2024-04-21T08:36:24.926878+0000","last_active":"2024-04-21T08:38:46.751721+0000","last_peered":"2024-04-21T08:38:46.751721+0000","last_clean":"2024-04-21T08:38:46.751721+0000","last_became_active":"2024-04-21T08:36:24.926431+0000","last_became_peered":"2024-04-21T08:36:24.926431+0000","last_unstale":"2024-04-21T08:38:46.751721+0000","last_undegraded":"2024-04-21T08:38:46.751721+0000","last_fullsized":"2024-04-21T08:38:46.751721+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:51:29.065962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639141+0000","last_change":"2024-04-21T08:36:20.284841+0000","last_active":"2024-04-21T08:38:46.639141+0000","last_peered":"2024-04-21T08:38:46.639141+0000","last_clean":"2024-04-21T08:38:46.639141+0000","last_became_active":"2024-04-21T08:36:20.283888+0000","last_became_peered":"2024-04-21T08:36:20.283888+0000","last_unstale":"2024-04-21T08:38:46.639141+0000","last_undegraded":"2024-04-21T08:38:46.639141+0000","last_fullsized":"2024-04-21T08:38:46.639141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:29:42.659608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.912243+0000","last_change":"2024-04-21T08:36:20.905140+0000","last_active":"2024-04-21T08:38:46.912243+0000","last_peered":"2024-04-21T08:38:46.912243+0000","last_clean":"2024-04-21T08:38:46.912243+0000","last_became_active":"2024-04-21T08:36:20.904821+0000","last_became_peered":"2024-04-21T08:36:20.904821+0000","last_unstale":"2024-04-21T08:38:46.912243+0000","last_undegraded":"2024-04-21T08:38:46.912243+0000","last_fullsized":"2024-04-21T08:38:46.912243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:09:11.962877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807527+0000","last_change":"2024-04-21T08:38:24.834006+0000","last_active":"2024-04-21T08:38:46.807527+0000","last_peered":"2024-04-21T08:38:46.807527+0000","last_clean":"2024-04-21T08:38:46.807527+0000","last_became_active":"2024-04-21T08:38:24.833453+0000","last_became_peered":"2024-04-21T08:38:24.833453+0000","last_unstale":"2024-04-21T08:38:46.807527+0000","last_undegraded":"2024-04-21T08:38:46.807527+0000","last_fullsized":"2024-04-21T08:38:46.807527+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:27:17.094301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751268+0000","last_change":"2024-04-21T08:36:20.290289+0000","last_active":"2024-04-21T08:38:46.751268+0000","last_peered":"2024-04-21T08:38:46.751268+0000","last_clean":"2024-04-21T08:38:46.751268+0000","last_became_active":"2024-04-21T08:36:20.289922+0000","last_became_peered":"2024-04-21T08:36:20.289922+0000","last_unstale":"2024-04-21T08:38:46.751268+0000","last_undegraded":"2024-04-21T08:38:46.751268+0000","last_fullsized":"2024-04-21T08:38:46.751268+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:10:00.926023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914048+0000","last_change":"2024-04-21T08:36:20.906950+0000","last_active":"2024-04-21T08:38:46.914048+0000","last_peered":"2024-04-21T08:38:46.914048+0000","last_clean":"2024-04-21T08:38:46.914048+0000","last_became_active":"2024-04-21T08:36:20.906599+0000","last_became_peered":"2024-04-21T08:36:20.906599+0000","last_unstale":"2024-04-21T08:38:46.914048+0000","last_undegraded":"2024-04-21T08:38:46.914048+0000","last_fullsized":"2024-04-21T08:38:46.914048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:55:27.595997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.743162+0000","last_change":"2024-04-21T08:36:24.924349+0000","last_active":"2024-04-21T08:38:46.743162+0000","last_peered":"2024-04-21T08:38:46.743162+0000","last_clean":"2024-04-21T08:38:46.743162+0000","last_became_active":"2024-04-21T08:36:24.923982+0000","last_became_peered":"2024-04-21T08:36:24.923982+0000","last_unstale":"2024-04-21T08:38:46.743162+0000","last_undegraded":"2024-04-21T08:38:46.743162+0000","last_fullsized":"2024-04-21T08:38:46.743162+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:20:29.600759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"94'9","reported_seq":175,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:51.641095+0000","last_change":"2024-04-21T08:36:20.285629+0000","last_active":"2024-04-21T08:38:51.641095+0000","last_peered":"2024-04-21T08:38:51.641095+0000","last_clean":"2024-04-21T08:38:51.641095+0000","last_became_active":"2024-04-21T08:36:20.285234+0000","last_became_peered":"2024-04-21T08:36:20.285234+0000","last_unstale":"2024-04-21T08:38:51.641095+0000","last_undegraded":"2024-04-21T08:38:51.641095+0000","last_fullsized":"2024-04-21T08:38:51.641095+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:19:32.966146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1902,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":134,"num_read_kb":6,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913981+0000","last_change":"2024-04-21T08:36:20.906819+0000","last_active":"2024-04-21T08:38:46.913981+0000","last_peered":"2024-04-21T08:38:46.913981+0000","last_clean":"2024-04-21T08:38:46.913981+0000","last_became_active":"2024-04-21T08:36:20.906459+0000","last_became_peered":"2024-04-21T08:36:20.906459+0000","last_unstale":"2024-04-21T08:38:46.913981+0000","last_undegraded":"2024-04-21T08:38:46.913981+0000","last_fullsized":"2024-04-21T08:38:46.913981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:12:39.312857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913253+0000","last_change":"2024-04-21T08:36:24.929452+0000","last_active":"2024-04-21T08:38:46.913253+0000","last_peered":"2024-04-21T08:38:46.913253+0000","last_clean":"2024-04-21T08:38:46.913253+0000","last_became_active":"2024-04-21T08:36:24.929039+0000","last_became_peered":"2024-04-21T08:36:24.929039+0000","last_unstale":"2024-04-21T08:38:46.913253+0000","last_undegraded":"2024-04-21T08:38:46.913253+0000","last_fullsized":"2024-04-21T08:38:46.913253+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:16:05.432024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":76,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655055+0000","last_change":"2024-04-21T08:37:24.719766+0000","last_active":"2024-04-21T08:38:46.655055+0000","last_peered":"2024-04-21T08:38:46.655055+0000","last_clean":"2024-04-21T08:38:46.655055+0000","last_became_active":"2024-04-21T08:37:24.708201+0000","last_became_peered":"2024-04-21T08:37:24.708201+0000","last_unstale":"2024-04-21T08:38:46.655055+0000","last_undegraded":"2024-04-21T08:38:46.655055+0000","last_fullsized":"2024-04-21T08:38:46.655055+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:10:51.158061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751195+0000","last_change":"2024-04-21T08:36:20.909076+0000","last_active":"2024-04-21T08:38:46.751195+0000","last_peered":"2024-04-21T08:38:46.751195+0000","last_clean":"2024-04-21T08:38:46.751195+0000","last_became_active":"2024-04-21T08:36:20.908739+0000","last_became_peered":"2024-04-21T08:36:20.908739+0000","last_unstale":"2024-04-21T08:38:46.751195+0000","last_undegraded":"2024-04-21T08:38:46.751195+0000","last_fullsized":"2024-04-21T08:38:46.751195+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:03:56.870477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751077+0000","last_change":"2024-04-21T08:36:24.926871+0000","last_active":"2024-04-21T08:38:46.751077+0000","last_peered":"2024-04-21T08:38:46.751077+0000","last_clean":"2024-04-21T08:38:46.751077+0000","last_became_active":"2024-04-21T08:36:24.926438+0000","last_became_peered":"2024-04-21T08:36:24.926438+0000","last_unstale":"2024-04-21T08:38:46.751077+0000","last_undegraded":"2024-04-21T08:38:46.751077+0000","last_fullsized":"2024-04-21T08:38:46.751077+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:24:48.984906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"94'3","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807724+0000","last_change":"2024-04-21T08:37:24.308038+0000","last_active":"2024-04-21T08:38:46.807724+0000","last_peered":"2024-04-21T08:38:46.807724+0000","last_clean":"2024-04-21T08:38:46.807724+0000","last_became_active":"2024-04-21T08:37:24.301034+0000","last_became_peered":"2024-04-21T08:37:24.301034+0000","last_unstale":"2024-04-21T08:38:46.807724+0000","last_undegraded":"2024-04-21T08:38:46.807724+0000","last_fullsized":"2024-04-21T08:38:46.807724+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:37:06.598400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807693+0000","last_change":"2024-04-21T08:37:24.299260+0000","last_active":"2024-04-21T08:38:46.807693+0000","last_peered":"2024-04-21T08:38:46.807693+0000","last_clean":"2024-04-21T08:38:46.807693+0000","last_became_active":"2024-04-21T08:37:24.298831+0000","last_became_peered":"2024-04-21T08:37:24.298831+0000","last_unstale":"2024-04-21T08:38:46.807693+0000","last_undegraded":"2024-04-21T08:38:46.807693+0000","last_fullsized":"2024-04-21T08:38:46.807693+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:58:29.368027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655140+0000","last_change":"2024-04-21T08:36:24.929420+0000","last_active":"2024-04-21T08:38:46.655140+0000","last_peered":"2024-04-21T08:38:46.655140+0000","last_clean":"2024-04-21T08:38:46.655140+0000","last_became_active":"2024-04-21T08:36:24.928981+0000","last_became_peered":"2024-04-21T08:36:24.928981+0000","last_unstale":"2024-04-21T08:38:46.655140+0000","last_undegraded":"2024-04-21T08:38:46.655140+0000","last_fullsized":"2024-04-21T08:38:46.655140+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:38:00.040456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.742890+0000","last_change":"2024-04-21T08:36:20.279910+0000","last_active":"2024-04-21T08:38:46.742890+0000","last_peered":"2024-04-21T08:38:46.742890+0000","last_clean":"2024-04-21T08:38:46.742890+0000","last_became_active":"2024-04-21T08:36:20.278904+0000","last_became_peered":"2024-04-21T08:36:20.278904+0000","last_unstale":"2024-04-21T08:38:46.742890+0000","last_undegraded":"2024-04-21T08:38:46.742890+0000","last_fullsized":"2024-04-21T08:38:46.742890+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:40:19.209213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638524+0000","last_change":"2024-04-21T08:36:20.895816+0000","last_active":"2024-04-21T08:38:46.638524+0000","last_peered":"2024-04-21T08:38:46.638524+0000","last_clean":"2024-04-21T08:38:46.638524+0000","last_became_active":"2024-04-21T08:36:20.895487+0000","last_became_peered":"2024-04-21T08:36:20.895487+0000","last_unstale":"2024-04-21T08:38:46.638524+0000","last_undegraded":"2024-04-21T08:38:46.638524+0000","last_fullsized":"2024-04-21T08:38:46.638524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:53:52.153683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.912486+0000","last_change":"2024-04-21T08:36:24.917637+0000","last_active":"2024-04-21T08:38:46.912486+0000","last_peered":"2024-04-21T08:38:46.912486+0000","last_clean":"2024-04-21T08:38:46.912486+0000","last_became_active":"2024-04-21T08:36:24.917192+0000","last_became_peered":"2024-04-21T08:36:24.917192+0000","last_unstale":"2024-04-21T08:38:46.912486+0000","last_undegraded":"2024-04-21T08:38:46.912486+0000","last_fullsized":"2024-04-21T08:38:46.912486+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:02:59.843903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":164,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:51.411645+0000","last_change":"2024-04-21T08:36:20.286424+0000","last_active":"2024-04-21T08:38:51.411645+0000","last_peered":"2024-04-21T08:38:51.411645+0000","last_clean":"2024-04-21T08:38:51.411645+0000","last_became_active":"2024-04-21T08:36:20.285767+0000","last_became_peered":"2024-04-21T08:36:20.285767+0000","last_unstale":"2024-04-21T08:38:51.411645+0000","last_undegraded":"2024-04-21T08:38:51.411645+0000","last_fullsized":"2024-04-21T08:38:51.411645+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:53:33.202734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638138+0000","last_change":"2024-04-21T08:37:24.297650+0000","last_active":"2024-04-21T08:38:46.638138+0000","last_peered":"2024-04-21T08:38:46.638138+0000","last_clean":"2024-04-21T08:38:46.638138+0000","last_became_active":"2024-04-21T08:37:24.297250+0000","last_became_peered":"2024-04-21T08:37:24.297250+0000","last_unstale":"2024-04-21T08:38:46.638138+0000","last_undegraded":"2024-04-21T08:38:46.638138+0000","last_fullsized":"2024-04-21T08:38:46.638138+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:21:21.419556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911806+0000","last_change":"2024-04-21T08:36:24.919669+0000","last_active":"2024-04-21T08:38:46.911806+0000","last_peered":"2024-04-21T08:38:46.911806+0000","last_clean":"2024-04-21T08:38:46.911806+0000","last_became_active":"2024-04-21T08:36:24.919281+0000","last_became_peered":"2024-04-21T08:36:24.919281+0000","last_unstale":"2024-04-21T08:38:46.911806+0000","last_undegraded":"2024-04-21T08:38:46.911806+0000","last_fullsized":"2024-04-21T08:38:46.911806+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:44:27.480367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751175+0000","last_change":"2024-04-21T08:37:24.297826+0000","last_active":"2024-04-21T08:38:46.751175+0000","last_peered":"2024-04-21T08:38:46.751175+0000","last_clean":"2024-04-21T08:38:46.751175+0000","last_became_active":"2024-04-21T08:37:24.297397+0000","last_became_peered":"2024-04-21T08:37:24.297397+0000","last_unstale":"2024-04-21T08:38:46.751175+0000","last_undegraded":"2024-04-21T08:38:46.751175+0000","last_fullsized":"2024-04-21T08:38:46.751175+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:52:17.693725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'7","reported_seq":170,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:51.640626+0000","last_change":"2024-04-21T08:36:20.291033+0000","last_active":"2024-04-21T08:38:51.640626+0000","last_peered":"2024-04-21T08:38:51.640626+0000","last_clean":"2024-04-21T08:38:51.640626+0000","last_became_active":"2024-04-21T08:36:20.290697+0000","last_became_peered":"2024-04-21T08:36:20.290697+0000","last_unstale":"2024-04-21T08:38:51.640626+0000","last_undegraded":"2024-04-21T08:38:51.640626+0000","last_fullsized":"2024-04-21T08:38:51.640626+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:22:54.593595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":129,"num_read_kb":129,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913716+0000","last_change":"2024-04-21T08:37:24.310936+0000","last_active":"2024-04-21T08:38:46.913716+0000","last_peered":"2024-04-21T08:38:46.913716+0000","last_clean":"2024-04-21T08:38:46.913716+0000","last_became_active":"2024-04-21T08:37:24.297349+0000","last_became_peered":"2024-04-21T08:37:24.297349+0000","last_unstale":"2024-04-21T08:38:46.913716+0000","last_undegraded":"2024-04-21T08:38:46.913716+0000","last_fullsized":"2024-04-21T08:38:46.913716+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:31:39.884609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638671+0000","last_change":"2024-04-21T08:36:20.904197+0000","last_active":"2024-04-21T08:38:46.638671+0000","last_peered":"2024-04-21T08:38:46.638671+0000","last_clean":"2024-04-21T08:38:46.638671+0000","last_became_active":"2024-04-21T08:36:20.903099+0000","last_became_peered":"2024-04-21T08:36:20.903099+0000","last_unstale":"2024-04-21T08:38:46.638671+0000","last_undegraded":"2024-04-21T08:38:46.638671+0000","last_fullsized":"2024-04-21T08:38:46.638671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:16:08.914335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911254+0000","last_change":"2024-04-21T08:36:20.280608+0000","last_active":"2024-04-21T08:38:46.911254+0000","last_peered":"2024-04-21T08:38:46.911254+0000","last_clean":"2024-04-21T08:38:46.911254+0000","last_became_active":"2024-04-21T08:36:20.280172+0000","last_became_peered":"2024-04-21T08:36:20.280172+0000","last_unstale":"2024-04-21T08:38:46.911254+0000","last_undegraded":"2024-04-21T08:38:46.911254+0000","last_fullsized":"2024-04-21T08:38:46.911254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:51:12.284463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"94'148","reported_seq":410,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:48.757849+0000","last_change":"2024-04-21T08:35:08.423545+0000","last_active":"2024-04-21T08:38:48.757849+0000","last_peered":"2024-04-21T08:38:48.757849+0000","last_clean":"2024-04-21T08:38:48.757849+0000","last_became_active":"2024-04-21T08:35:08.423027+0000","last_became_peered":"2024-04-21T08:35:08.423027+0000","last_unstale":"2024-04-21T08:38:48.757849+0000","last_undegraded":"2024-04-21T08:38:48.757849+0000","last_fullsized":"2024-04-21T08:38:48.757849+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:31:45.359253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:31:45.359253+0000","last_clean_scrub_stamp":"2024-04-21T08:31:45.359253+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:53:24.310824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751452+0000","last_change":"2024-04-21T08:36:24.922422+0000","last_active":"2024-04-21T08:38:46.751452+0000","last_peered":"2024-04-21T08:38:46.751452+0000","last_clean":"2024-04-21T08:38:46.751452+0000","last_became_active":"2024-04-21T08:36:24.922006+0000","last_became_peered":"2024-04-21T08:36:24.922006+0000","last_unstale":"2024-04-21T08:38:46.751452+0000","last_undegraded":"2024-04-21T08:38:46.751452+0000","last_fullsized":"2024-04-21T08:38:46.751452+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:46:22.840802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911805+0000","last_change":"2024-04-21T08:36:20.911222+0000","last_active":"2024-04-21T08:38:46.911805+0000","last_peered":"2024-04-21T08:38:46.911805+0000","last_clean":"2024-04-21T08:38:46.911805+0000","last_became_active":"2024-04-21T08:36:20.910933+0000","last_became_peered":"2024-04-21T08:36:20.910933+0000","last_unstale":"2024-04-21T08:38:46.911805+0000","last_undegraded":"2024-04-21T08:38:46.911805+0000","last_fullsized":"2024-04-21T08:38:46.911805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:41:25.621266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913633+0000","last_change":"2024-04-21T08:36:20.288409+0000","last_active":"2024-04-21T08:38:46.913633+0000","last_peered":"2024-04-21T08:38:46.913633+0000","last_clean":"2024-04-21T08:38:46.913633+0000","last_became_active":"2024-04-21T08:36:20.287916+0000","last_became_peered":"2024-04-21T08:36:20.287916+0000","last_unstale":"2024-04-21T08:38:46.913633+0000","last_undegraded":"2024-04-21T08:38:46.913633+0000","last_fullsized":"2024-04-21T08:38:46.913633+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:04:31.068352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807586+0000","last_change":"2024-04-21T08:36:24.918166+0000","last_active":"2024-04-21T08:38:46.807586+0000","last_peered":"2024-04-21T08:38:46.807586+0000","last_clean":"2024-04-21T08:38:46.807586+0000","last_became_active":"2024-04-21T08:36:24.917745+0000","last_became_peered":"2024-04-21T08:36:24.917745+0000","last_unstale":"2024-04-21T08:38:46.807586+0000","last_undegraded":"2024-04-21T08:38:46.807586+0000","last_fullsized":"2024-04-21T08:38:46.807586+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:55:41.169681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655417+0000","last_change":"2024-04-21T08:36:20.291079+0000","last_active":"2024-04-21T08:38:46.655417+0000","last_peered":"2024-04-21T08:38:46.655417+0000","last_clean":"2024-04-21T08:38:46.655417+0000","last_became_active":"2024-04-21T08:36:20.290720+0000","last_became_peered":"2024-04-21T08:36:20.290720+0000","last_unstale":"2024-04-21T08:38:46.655417+0000","last_undegraded":"2024-04-21T08:38:46.655417+0000","last_fullsized":"2024-04-21T08:38:46.655417+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:41.269029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.912713+0000","last_change":"2024-04-21T08:36:20.907416+0000","last_active":"2024-04-21T08:38:46.912713+0000","last_peered":"2024-04-21T08:38:46.912713+0000","last_clean":"2024-04-21T08:38:46.912713+0000","last_became_active":"2024-04-21T08:36:20.907005+0000","last_became_peered":"2024-04-21T08:36:20.907005+0000","last_unstale":"2024-04-21T08:38:46.912713+0000","last_undegraded":"2024-04-21T08:38:46.912713+0000","last_fullsized":"2024-04-21T08:38:46.912713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:58:17.979947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638481+0000","last_change":"2024-04-21T08:36:24.921475+0000","last_active":"2024-04-21T08:38:46.638481+0000","last_peered":"2024-04-21T08:38:46.638481+0000","last_clean":"2024-04-21T08:38:46.638481+0000","last_became_active":"2024-04-21T08:36:24.921091+0000","last_became_peered":"2024-04-21T08:36:24.921091+0000","last_unstale":"2024-04-21T08:38:46.638481+0000","last_undegraded":"2024-04-21T08:38:46.638481+0000","last_fullsized":"2024-04-21T08:38:46.638481+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:28:14.224501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'48","reported_seq":105,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.860978+0000","last_change":"2024-04-21T08:36:20.288916+0000","last_active":"2024-04-21T08:38:46.860978+0000","last_peered":"2024-04-21T08:38:46.860978+0000","last_clean":"2024-04-21T08:38:46.860978+0000","last_became_active":"2024-04-21T08:36:20.288575+0000","last_became_peered":"2024-04-21T08:36:20.288575+0000","last_unstale":"2024-04-21T08:38:46.860978+0000","last_undegraded":"2024-04-21T08:38:46.860978+0000","last_fullsized":"2024-04-21T08:38:46.860978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:52:05.795554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639071+0000","last_change":"2024-04-21T08:36:20.912053+0000","last_active":"2024-04-21T08:38:46.639071+0000","last_peered":"2024-04-21T08:38:46.639071+0000","last_clean":"2024-04-21T08:38:46.639071+0000","last_became_active":"2024-04-21T08:36:20.911584+0000","last_became_peered":"2024-04-21T08:36:20.911584+0000","last_unstale":"2024-04-21T08:38:46.639071+0000","last_undegraded":"2024-04-21T08:38:46.639071+0000","last_fullsized":"2024-04-21T08:38:46.639071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:48:36.369954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638589+0000","last_change":"2024-04-21T08:36:24.919212+0000","last_active":"2024-04-21T08:38:46.638589+0000","last_peered":"2024-04-21T08:38:46.638589+0000","last_clean":"2024-04-21T08:38:46.638589+0000","last_became_active":"2024-04-21T08:36:24.918807+0000","last_became_peered":"2024-04-21T08:36:24.918807+0000","last_unstale":"2024-04-21T08:38:46.638589+0000","last_undegraded":"2024-04-21T08:38:46.638589+0000","last_fullsized":"2024-04-21T08:38:46.638589+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:25:50.816775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911766+0000","last_change":"2024-04-21T08:36:20.282536+0000","last_active":"2024-04-21T08:38:46.911766+0000","last_peered":"2024-04-21T08:38:46.911766+0000","last_clean":"2024-04-21T08:38:46.911766+0000","last_became_active":"2024-04-21T08:36:20.282172+0000","last_became_peered":"2024-04-21T08:36:20.282172+0000","last_unstale":"2024-04-21T08:38:46.911766+0000","last_undegraded":"2024-04-21T08:38:46.911766+0000","last_fullsized":"2024-04-21T08:38:46.911766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:15:38.992441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.915413+0000","last_change":"2024-04-21T08:36:20.908416+0000","last_active":"2024-04-21T08:38:46.915413+0000","last_peered":"2024-04-21T08:38:46.915413+0000","last_clean":"2024-04-21T08:38:46.915413+0000","last_became_active":"2024-04-21T08:36:20.908054+0000","last_became_peered":"2024-04-21T08:36:20.908054+0000","last_unstale":"2024-04-21T08:38:46.915413+0000","last_undegraded":"2024-04-21T08:38:46.915413+0000","last_fullsized":"2024-04-21T08:38:46.915413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:23:16.562060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911638+0000","last_change":"2024-04-21T08:36:24.920637+0000","last_active":"2024-04-21T08:38:46.911638+0000","last_peered":"2024-04-21T08:38:46.911638+0000","last_clean":"2024-04-21T08:38:46.911638+0000","last_became_active":"2024-04-21T08:36:24.919921+0000","last_became_peered":"2024-04-21T08:36:24.919921+0000","last_unstale":"2024-04-21T08:38:46.911638+0000","last_undegraded":"2024-04-21T08:38:46.911638+0000","last_fullsized":"2024-04-21T08:38:46.911638+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:20:38.230087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.922412+0000","last_change":"2024-04-21T08:36:20.279332+0000","last_active":"2024-04-21T08:38:46.922412+0000","last_peered":"2024-04-21T08:38:46.922412+0000","last_clean":"2024-04-21T08:38:46.922412+0000","last_became_active":"2024-04-21T08:36:20.278970+0000","last_became_peered":"2024-04-21T08:36:20.278970+0000","last_unstale":"2024-04-21T08:38:46.922412+0000","last_undegraded":"2024-04-21T08:38:46.922412+0000","last_fullsized":"2024-04-21T08:38:46.922412+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:36:16.041610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.654968+0000","last_change":"2024-04-21T08:36:20.919621+0000","last_active":"2024-04-21T08:38:46.654968+0000","last_peered":"2024-04-21T08:38:46.654968+0000","last_clean":"2024-04-21T08:38:46.654968+0000","last_became_active":"2024-04-21T08:36:20.919246+0000","last_became_peered":"2024-04-21T08:36:20.919246+0000","last_unstale":"2024-04-21T08:38:46.654968+0000","last_undegraded":"2024-04-21T08:38:46.654968+0000","last_fullsized":"2024-04-21T08:38:46.654968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:31:59.001501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.743258+0000","last_change":"2024-04-21T08:36:24.920459+0000","last_active":"2024-04-21T08:38:46.743258+0000","last_peered":"2024-04-21T08:38:46.743258+0000","last_clean":"2024-04-21T08:38:46.743258+0000","last_became_active":"2024-04-21T08:36:24.919691+0000","last_became_peered":"2024-04-21T08:36:24.919691+0000","last_unstale":"2024-04-21T08:38:46.743258+0000","last_undegraded":"2024-04-21T08:38:46.743258+0000","last_fullsized":"2024-04-21T08:38:46.743258+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:38:26.655452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638571+0000","last_change":"2024-04-21T08:36:20.284286+0000","last_active":"2024-04-21T08:38:46.638571+0000","last_peered":"2024-04-21T08:38:46.638571+0000","last_clean":"2024-04-21T08:38:46.638571+0000","last_became_active":"2024-04-21T08:36:20.283903+0000","last_became_peered":"2024-04-21T08:36:20.283903+0000","last_unstale":"2024-04-21T08:38:46.638571+0000","last_undegraded":"2024-04-21T08:38:46.638571+0000","last_fullsized":"2024-04-21T08:38:46.638571+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:51:52.409988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807332+0000","last_change":"2024-04-21T08:36:20.911892+0000","last_active":"2024-04-21T08:38:46.807332+0000","last_peered":"2024-04-21T08:38:46.807332+0000","last_clean":"2024-04-21T08:38:46.807332+0000","last_became_active":"2024-04-21T08:36:20.911513+0000","last_became_peered":"2024-04-21T08:36:20.911513+0000","last_unstale":"2024-04-21T08:38:46.807332+0000","last_undegraded":"2024-04-21T08:38:46.807332+0000","last_fullsized":"2024-04-21T08:38:46.807332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:51:13.733534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911539+0000","last_change":"2024-04-21T08:36:24.920347+0000","last_active":"2024-04-21T08:38:46.911539+0000","last_peered":"2024-04-21T08:38:46.911539+0000","last_clean":"2024-04-21T08:38:46.911539+0000","last_became_active":"2024-04-21T08:36:24.919831+0000","last_became_peered":"2024-04-21T08:36:24.919831+0000","last_unstale":"2024-04-21T08:38:46.911539+0000","last_undegraded":"2024-04-21T08:38:46.911539+0000","last_fullsized":"2024-04-21T08:38:46.911539+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:14:27.020476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638672+0000","last_change":"2024-04-21T08:36:20.286786+0000","last_active":"2024-04-21T08:38:46.638672+0000","last_peered":"2024-04-21T08:38:46.638672+0000","last_clean":"2024-04-21T08:38:46.638672+0000","last_became_active":"2024-04-21T08:36:20.286429+0000","last_became_peered":"2024-04-21T08:36:20.286429+0000","last_unstale":"2024-04-21T08:38:46.638672+0000","last_undegraded":"2024-04-21T08:38:46.638672+0000","last_fullsized":"2024-04-21T08:38:46.638672+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:40:57.093370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638983+0000","last_change":"2024-04-21T08:36:20.904767+0000","last_active":"2024-04-21T08:38:46.638983+0000","last_peered":"2024-04-21T08:38:46.638983+0000","last_clean":"2024-04-21T08:38:46.638983+0000","last_became_active":"2024-04-21T08:36:20.904406+0000","last_became_peered":"2024-04-21T08:36:20.904406+0000","last_unstale":"2024-04-21T08:38:46.638983+0000","last_undegraded":"2024-04-21T08:38:46.638983+0000","last_fullsized":"2024-04-21T08:38:46.638983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:11:41.644549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911769+0000","last_change":"2024-04-21T08:36:24.918031+0000","last_active":"2024-04-21T08:38:46.911769+0000","last_peered":"2024-04-21T08:38:46.911769+0000","last_clean":"2024-04-21T08:38:46.911769+0000","last_became_active":"2024-04-21T08:36:24.917618+0000","last_became_peered":"2024-04-21T08:36:24.917618+0000","last_unstale":"2024-04-21T08:38:46.911769+0000","last_undegraded":"2024-04-21T08:38:46.911769+0000","last_fullsized":"2024-04-21T08:38:46.911769+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:50:35.797911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655179+0000","last_change":"2024-04-21T08:36:20.286212+0000","last_active":"2024-04-21T08:38:46.655179+0000","last_peered":"2024-04-21T08:38:46.655179+0000","last_clean":"2024-04-21T08:38:46.655179+0000","last_became_active":"2024-04-21T08:36:20.285878+0000","last_became_peered":"2024-04-21T08:36:20.285878+0000","last_unstale":"2024-04-21T08:38:46.655179+0000","last_undegraded":"2024-04-21T08:38:46.655179+0000","last_fullsized":"2024-04-21T08:38:46.655179+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:19.375455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914242+0000","last_change":"2024-04-21T08:36:20.908335+0000","last_active":"2024-04-21T08:38:46.914242+0000","last_peered":"2024-04-21T08:38:46.914242+0000","last_clean":"2024-04-21T08:38:46.914242+0000","last_became_active":"2024-04-21T08:36:20.907945+0000","last_became_peered":"2024-04-21T08:36:20.907945+0000","last_unstale":"2024-04-21T08:38:46.914242+0000","last_undegraded":"2024-04-21T08:38:46.914242+0000","last_fullsized":"2024-04-21T08:38:46.914242+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:47:17.841511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.922383+0000","last_change":"2024-04-21T08:36:24.915553+0000","last_active":"2024-04-21T08:38:46.922383+0000","last_peered":"2024-04-21T08:38:46.922383+0000","last_clean":"2024-04-21T08:38:46.922383+0000","last_became_active":"2024-04-21T08:36:24.915028+0000","last_became_peered":"2024-04-21T08:36:24.915028+0000","last_unstale":"2024-04-21T08:38:46.922383+0000","last_undegraded":"2024-04-21T08:38:46.922383+0000","last_fullsized":"2024-04-21T08:38:46.922383+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:17:21.192946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'7","reported_seq":169,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:50.660424+0000","last_change":"2024-04-21T08:36:20.279649+0000","last_active":"2024-04-21T08:38:50.660424+0000","last_peered":"2024-04-21T08:38:50.660424+0000","last_clean":"2024-04-21T08:38:50.660424+0000","last_became_active":"2024-04-21T08:36:20.279111+0000","last_became_peered":"2024-04-21T08:36:20.279111+0000","last_unstale":"2024-04-21T08:38:50.660424+0000","last_undegraded":"2024-04-21T08:38:50.660424+0000","last_fullsized":"2024-04-21T08:38:50.660424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:19:57.380495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":128,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914334+0000","last_change":"2024-04-21T08:36:20.911489+0000","last_active":"2024-04-21T08:38:46.914334+0000","last_peered":"2024-04-21T08:38:46.914334+0000","last_clean":"2024-04-21T08:38:46.914334+0000","last_became_active":"2024-04-21T08:36:20.911168+0000","last_became_peered":"2024-04-21T08:36:20.911168+0000","last_unstale":"2024-04-21T08:38:46.914334+0000","last_undegraded":"2024-04-21T08:38:46.914334+0000","last_fullsized":"2024-04-21T08:38:46.914334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:40:41.585841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914668+0000","last_change":"2024-04-21T08:36:24.921175+0000","last_active":"2024-04-21T08:38:46.914668+0000","last_peered":"2024-04-21T08:38:46.914668+0000","last_clean":"2024-04-21T08:38:46.914668+0000","last_became_active":"2024-04-21T08:36:24.920802+0000","last_became_peered":"2024-04-21T08:36:24.920802+0000","last_unstale":"2024-04-21T08:38:46.914668+0000","last_undegraded":"2024-04-21T08:38:46.914668+0000","last_fullsized":"2024-04-21T08:38:46.914668+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:43:12.140016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"94'41","reported_seq":82,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.859069+0000","last_change":"2024-04-21T08:36:20.290202+0000","last_active":"2024-04-21T08:38:46.859069+0000","last_peered":"2024-04-21T08:38:46.859069+0000","last_clean":"2024-04-21T08:38:46.859069+0000","last_became_active":"2024-04-21T08:36:20.289868+0000","last_became_peered":"2024-04-21T08:36:20.289868+0000","last_unstale":"2024-04-21T08:38:46.859069+0000","last_undegraded":"2024-04-21T08:38:46.859069+0000","last_fullsized":"2024-04-21T08:38:46.859069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:07:45.525980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913789+0000","last_change":"2024-04-21T08:36:20.899590+0000","last_active":"2024-04-21T08:38:46.913789+0000","last_peered":"2024-04-21T08:38:46.913789+0000","last_clean":"2024-04-21T08:38:46.913789+0000","last_became_active":"2024-04-21T08:36:20.899254+0000","last_became_peered":"2024-04-21T08:36:20.899254+0000","last_unstale":"2024-04-21T08:38:46.913789+0000","last_undegraded":"2024-04-21T08:38:46.913789+0000","last_fullsized":"2024-04-21T08:38:46.913789+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:34:25.409694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.750892+0000","last_change":"2024-04-21T08:36:24.918310+0000","last_active":"2024-04-21T08:38:46.750892+0000","last_peered":"2024-04-21T08:38:46.750892+0000","last_clean":"2024-04-21T08:38:46.750892+0000","last_became_active":"2024-04-21T08:36:24.917899+0000","last_became_peered":"2024-04-21T08:36:24.917899+0000","last_unstale":"2024-04-21T08:38:46.750892+0000","last_undegraded":"2024-04-21T08:38:46.750892+0000","last_fullsized":"2024-04-21T08:38:46.750892+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:09:13.006884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":51,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913309+0000","last_change":"2024-04-21T08:37:24.304544+0000","last_active":"2024-04-21T08:38:46.913309+0000","last_peered":"2024-04-21T08:38:46.913309+0000","last_clean":"2024-04-21T08:38:46.913309+0000","last_became_active":"2024-04-21T08:37:24.295808+0000","last_became_peered":"2024-04-21T08:37:24.295808+0000","last_unstale":"2024-04-21T08:38:46.913309+0000","last_undegraded":"2024-04-21T08:38:46.913309+0000","last_fullsized":"2024-04-21T08:38:46.913309+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:07:15.797133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914356+0000","last_change":"2024-04-21T08:36:20.912368+0000","last_active":"2024-04-21T08:38:46.914356+0000","last_peered":"2024-04-21T08:38:46.914356+0000","last_clean":"2024-04-21T08:38:46.914356+0000","last_became_active":"2024-04-21T08:36:20.912009+0000","last_became_peered":"2024-04-21T08:36:20.912009+0000","last_unstale":"2024-04-21T08:38:46.914356+0000","last_undegraded":"2024-04-21T08:38:46.914356+0000","last_fullsized":"2024-04-21T08:38:46.914356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:35:40.700314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807399+0000","last_change":"2024-04-21T08:36:20.913472+0000","last_active":"2024-04-21T08:38:46.807399+0000","last_peered":"2024-04-21T08:38:46.807399+0000","last_clean":"2024-04-21T08:38:46.807399+0000","last_became_active":"2024-04-21T08:36:20.913118+0000","last_became_peered":"2024-04-21T08:36:20.913118+0000","last_unstale":"2024-04-21T08:38:46.807399+0000","last_undegraded":"2024-04-21T08:38:46.807399+0000","last_fullsized":"2024-04-21T08:38:46.807399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:31:53.957044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638786+0000","last_change":"2024-04-21T08:36:20.913677+0000","last_active":"2024-04-21T08:38:46.638786+0000","last_peered":"2024-04-21T08:38:46.638786+0000","last_clean":"2024-04-21T08:38:46.638786+0000","last_became_active":"2024-04-21T08:36:20.913097+0000","last_became_peered":"2024-04-21T08:36:20.913097+0000","last_unstale":"2024-04-21T08:38:46.638786+0000","last_undegraded":"2024-04-21T08:38:46.638786+0000","last_fullsized":"2024-04-21T08:38:46.638786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:08:40.388370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638662+0000","last_change":"2024-04-21T08:36:20.906624+0000","last_active":"2024-04-21T08:38:46.638662+0000","last_peered":"2024-04-21T08:38:46.638662+0000","last_clean":"2024-04-21T08:38:46.638662+0000","last_became_active":"2024-04-21T08:36:20.906247+0000","last_became_peered":"2024-04-21T08:36:20.906247+0000","last_unstale":"2024-04-21T08:38:46.638662+0000","last_undegraded":"2024-04-21T08:38:46.638662+0000","last_fullsized":"2024-04-21T08:38:46.638662+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:44:11.855183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.916219+0000","last_change":"2024-04-21T08:36:20.906576+0000","last_active":"2024-04-21T08:38:46.916219+0000","last_peered":"2024-04-21T08:38:46.916219+0000","last_clean":"2024-04-21T08:38:46.916219+0000","last_became_active":"2024-04-21T08:36:20.906231+0000","last_became_peered":"2024-04-21T08:36:20.906231+0000","last_unstale":"2024-04-21T08:38:46.916219+0000","last_undegraded":"2024-04-21T08:38:46.916219+0000","last_fullsized":"2024-04-21T08:38:46.916219+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:42:48.156614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.912593+0000","last_change":"2024-04-21T08:36:20.902551+0000","last_active":"2024-04-21T08:38:46.912593+0000","last_peered":"2024-04-21T08:38:46.912593+0000","last_clean":"2024-04-21T08:38:46.912593+0000","last_became_active":"2024-04-21T08:36:20.902208+0000","last_became_peered":"2024-04-21T08:36:20.902208+0000","last_unstale":"2024-04-21T08:38:46.912593+0000","last_undegraded":"2024-04-21T08:38:46.912593+0000","last_fullsized":"2024-04-21T08:38:46.912593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:36:30.016256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807607+0000","last_change":"2024-04-21T08:36:20.918588+0000","last_active":"2024-04-21T08:38:46.807607+0000","last_peered":"2024-04-21T08:38:46.807607+0000","last_clean":"2024-04-21T08:38:46.807607+0000","last_became_active":"2024-04-21T08:36:20.918263+0000","last_became_peered":"2024-04-21T08:36:20.918263+0000","last_unstale":"2024-04-21T08:38:46.807607+0000","last_undegraded":"2024-04-21T08:38:46.807607+0000","last_fullsized":"2024-04-21T08:38:46.807607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:51:30.340336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639167+0000","last_change":"2024-04-21T08:36:20.911063+0000","last_active":"2024-04-21T08:38:46.639167+0000","last_peered":"2024-04-21T08:38:46.639167+0000","last_clean":"2024-04-21T08:38:46.639167+0000","last_became_active":"2024-04-21T08:36:20.910647+0000","last_became_peered":"2024-04-21T08:36:20.910647+0000","last_unstale":"2024-04-21T08:38:46.639167+0000","last_undegraded":"2024-04-21T08:38:46.639167+0000","last_fullsized":"2024-04-21T08:38:46.639167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:10:31.079782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639213+0000","last_change":"2024-04-21T08:36:20.914079+0000","last_active":"2024-04-21T08:38:46.639213+0000","last_peered":"2024-04-21T08:38:46.639213+0000","last_clean":"2024-04-21T08:38:46.639213+0000","last_became_active":"2024-04-21T08:36:20.912851+0000","last_became_peered":"2024-04-21T08:36:20.912851+0000","last_unstale":"2024-04-21T08:38:46.639213+0000","last_undegraded":"2024-04-21T08:38:46.639213+0000","last_fullsized":"2024-04-21T08:38:46.639213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:53:35.811281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751071+0000","last_change":"2024-04-21T08:36:20.900551+0000","last_active":"2024-04-21T08:38:46.751071+0000","last_peered":"2024-04-21T08:38:46.751071+0000","last_clean":"2024-04-21T08:38:46.751071+0000","last_became_active":"2024-04-21T08:36:20.900173+0000","last_became_peered":"2024-04-21T08:36:20.900173+0000","last_unstale":"2024-04-21T08:38:46.751071+0000","last_undegraded":"2024-04-21T08:38:46.751071+0000","last_fullsized":"2024-04-21T08:38:46.751071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:21.389498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638564+0000","last_change":"2024-04-21T08:36:20.912953+0000","last_active":"2024-04-21T08:38:46.638564+0000","last_peered":"2024-04-21T08:38:46.638564+0000","last_clean":"2024-04-21T08:38:46.638564+0000","last_became_active":"2024-04-21T08:36:20.912223+0000","last_became_peered":"2024-04-21T08:36:20.912223+0000","last_unstale":"2024-04-21T08:38:46.638564+0000","last_undegraded":"2024-04-21T08:38:46.638564+0000","last_fullsized":"2024-04-21T08:38:46.638564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:43:14.277179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.922633+0000","last_change":"2024-04-21T08:36:20.906164+0000","last_active":"2024-04-21T08:38:46.922633+0000","last_peered":"2024-04-21T08:38:46.922633+0000","last_clean":"2024-04-21T08:38:46.922633+0000","last_became_active":"2024-04-21T08:36:20.905749+0000","last_became_peered":"2024-04-21T08:36:20.905749+0000","last_unstale":"2024-04-21T08:38:46.922633+0000","last_undegraded":"2024-04-21T08:38:46.922633+0000","last_fullsized":"2024-04-21T08:38:46.922633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:02:10.569087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913644+0000","last_change":"2024-04-21T08:36:20.900314+0000","last_active":"2024-04-21T08:38:46.913644+0000","last_peered":"2024-04-21T08:38:46.913644+0000","last_clean":"2024-04-21T08:38:46.913644+0000","last_became_active":"2024-04-21T08:36:20.900009+0000","last_became_peered":"2024-04-21T08:36:20.900009+0000","last_unstale":"2024-04-21T08:38:46.913644+0000","last_undegraded":"2024-04-21T08:38:46.913644+0000","last_fullsized":"2024-04-21T08:38:46.913644+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:19:59.997302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807555+0000","last_change":"2024-04-21T08:36:20.920725+0000","last_active":"2024-04-21T08:38:46.807555+0000","last_peered":"2024-04-21T08:38:46.807555+0000","last_clean":"2024-04-21T08:38:46.807555+0000","last_became_active":"2024-04-21T08:36:20.920288+0000","last_became_peered":"2024-04-21T08:36:20.920288+0000","last_unstale":"2024-04-21T08:38:46.807555+0000","last_undegraded":"2024-04-21T08:38:46.807555+0000","last_fullsized":"2024-04-21T08:38:46.807555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:36:18.736388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":603992,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1402,"num_read_kb":818,"num_write":375,"num_write_kb":1473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":806,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1811976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":305,"ondisk_log_size":305,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383723,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1093632,"data_stored":531299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645359,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28952,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716200,"statfs":{"total":95995035648,"available":95965388800,"internally_reserved":0,"allocated":1921024,"data_stored":1323967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808885,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36804,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708348,"statfs":{"total":95995035648,"available":95957348352,"internally_reserved":0,"allocated":1499136,"data_stored":923219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070522,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28476,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716676,"statfs":{"total":95995035648,"available":95965876224,"internally_reserved":0,"allocated":1490944,"data_stored":923575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332160,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28672,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716480,"statfs":{"total":95995035648,"available":95965675520,"internally_reserved":0,"allocated":1691648,"data_stored":1127969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495685,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1081344,"data_stored":525507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36476,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708676,"statfs":{"total":95995035648,"available":95957684224,"internally_reserved":0,"allocated":1163264,"data_stored":596717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1105920,"data_stored":527495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36480,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708672,"statfs":{"total":95995035648,"available":95957680128,"internally_reserved":0,"allocated":1167360,"data_stored":598085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182483,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1050031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920847,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1089536,"data_stored":530895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757321,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37136,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708016,"statfs":{"total":95995035648,"available":95957008384,"internally_reserved":0,"allocated":1843200,"data_stored":1254509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":71932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":73300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":394694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":204718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T08:38:57.623 INFO:tasks.quiescer.ceph_manager:clean! 2024-04-21T08:38:57.623 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2024-04-21T08:38:57.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:57 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/335669794' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:57.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:57 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2229459377' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:57.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:57 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/126889366' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:57.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:57 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/335669794' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:57.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:57 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2229459377' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:38:57.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:57 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/126889366' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:38:57.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.755+0000 7f45d8c5c640 1 -- 172.21.15.86:0/107820934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d41031d0 msgr2=0x7f45d41035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:57.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.755+0000 7f45d8c5c640 1 --2- 172.21.15.86:0/107820934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d41031d0 0x7f45d41035b0 secure :-1 s=READY pgs=587 cs=0 l=1 rev1=1 crypto rx=0x7f45cc009900 tx=0x7f45cc02bcd0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.755+0000 7f45d8c5c640 1 -- 172.21.15.86:0/107820934 shutdown_connections 2024-04-21T08:38:57.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.755+0000 7f45d8c5c640 1 --2- 172.21.15.86:0/107820934 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45d410c5a0 0x7f45d410e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.755+0000 7f45d8c5c640 1 --2- 172.21.15.86:0/107820934 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45d4103af0 0x7f45d410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.755+0000 7f45d8c5c640 1 --2- 172.21.15.86:0/107820934 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d41031d0 0x7f45d41035b0 unknown :-1 s=CLOSED pgs=587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.755+0000 7f45d8c5c640 1 -- 172.21.15.86:0/107820934 >> 172.21.15.86:0/107820934 conn(0x7f45d40771d0 msgr2=0x7f45d40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:57.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.755+0000 7f45d8c5c640 1 -- 172.21.15.86:0/107820934 shutdown_connections 2024-04-21T08:38:57.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.755+0000 7f45d8c5c640 1 -- 172.21.15.86:0/107820934 wait complete. 2024-04-21T08:38:57.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.756+0000 7f45d8c5c640 1 Processor -- start 2024-04-21T08:38:57.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.756+0000 7f45d8c5c640 1 -- start start 2024-04-21T08:38:57.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.756+0000 7f45d8c5c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45d41031d0 0x7f45d419e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:57.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.756+0000 7f45d8c5c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d4103af0 0x7f45d419e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:57.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.756+0000 7f45d8c5c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45d410c5a0 0x7f45d41985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:57.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.756+0000 7f45d8c5c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f45d41121f0 con 0x7f45d4103af0 2024-04-21T08:38:57.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.756+0000 7f45d8c5c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f45d4112070 con 0x7f45d410c5a0 2024-04-21T08:38:57.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.756+0000 7f45d8c5c640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f45d4112370 con 0x7f45d41031d0 2024-04-21T08:38:57.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.756+0000 7f45d1d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d4103af0 0x7f45d419e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:57.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.756+0000 7f45d1d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d4103af0 0x7f45d419e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36584/0 (socket says 172.21.15.86:36584) 2024-04-21T08:38:57.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.756+0000 7f45d1d9b640 1 -- 172.21.15.86:0/746196228 learned_addr learned my addr 172.21.15.86:0/746196228 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:57.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.756+0000 7f45d259c640 1 --2- 172.21.15.86:0/746196228 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45d41031d0 0x7f45d419e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:57.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.756+0000 7f45d2d9d640 1 --2- 172.21.15.86:0/746196228 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45d410c5a0 0x7f45d41985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:57.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.757+0000 7f45d1d9b640 1 -- 172.21.15.86:0/746196228 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45d41031d0 msgr2=0x7f45d419e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:57.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.757+0000 7f45d1d9b640 1 --2- 172.21.15.86:0/746196228 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45d41031d0 0x7f45d419e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.757+0000 7f45d1d9b640 1 -- 172.21.15.86:0/746196228 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45d410c5a0 msgr2=0x7f45d41985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:57.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.757+0000 7f45d1d9b640 1 --2- 172.21.15.86:0/746196228 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45d410c5a0 0x7f45d41985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.757+0000 7f45d1d9b640 1 -- 172.21.15.86:0/746196228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f45d4198d30 con 0x7f45d4103af0 2024-04-21T08:38:57.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.757+0000 7f45d1d9b640 1 --2- 172.21.15.86:0/746196228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d4103af0 0x7f45d419e940 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f45c800e9f0 tx=0x7f45c800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:57.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.757+0000 7f45c37fe640 1 -- 172.21.15.86:0/746196228 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c800cd80 con 0x7f45d4103af0 2024-04-21T08:38:57.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.757+0000 7f45c37fe640 1 -- 172.21.15.86:0/746196228 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f45c80108e0 con 0x7f45d4103af0 2024-04-21T08:38:57.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.757+0000 7f45d8c5c640 1 -- 172.21.15.86:0/746196228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f45d4199020 con 0x7f45d4103af0 2024-04-21T08:38:57.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.757+0000 7f45c37fe640 1 -- 172.21.15.86:0/746196228 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f45c8018980 con 0x7f45d4103af0 2024-04-21T08:38:57.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.757+0000 7f45d8c5c640 1 -- 172.21.15.86:0/746196228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f45d41a52c0 con 0x7f45d4103af0 2024-04-21T08:38:57.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.758+0000 7f45d8c5c640 1 -- 172.21.15.86:0/746196228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f45d4069230 con 0x7f45d4103af0 2024-04-21T08:38:57.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.759+0000 7f45c37fe640 1 -- 172.21.15.86:0/746196228 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f45c8010430 con 0x7f45d4103af0 2024-04-21T08:38:57.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.760+0000 7f45c37fe640 1 --2- 172.21.15.86:0/746196228 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f45bc0a3e30 0x7f45bc0a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:57.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.760+0000 7f45d259c640 1 --2- 172.21.15.86:0/746196228 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f45bc0a3e30 0x7f45bc0a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:57.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.760+0000 7f45c37fe640 1 -- 172.21.15.86:0/746196228 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f45c80d1030 con 0x7f45d4103af0 2024-04-21T08:38:57.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.760+0000 7f45d259c640 1 --2- 172.21.15.86:0/746196228 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f45bc0a3e30 0x7f45bc0a62f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f45cc006010 tx=0x7f45cc005e40 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:57.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.764+0000 7f45c37fe640 1 -- 172.21.15.86:0/746196228 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f45c809d1f0 con 0x7f45d4103af0 2024-04-21T08:38:57.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.938+0000 7f45d8c5c640 1 -- 172.21.15.86:0/746196228 --> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] -- mgr_command(tid 0: {"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}) -- 0x7f45d4062390 con 0x7f45bc0a3e30 2024-04-21T08:38:57.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.947+0000 7f45c37fe640 1 -- 172.21.15.86:0/746196228 <== mgr.15558 v2:172.21.15.86:6800/2452032403 1 ==== mgr_command_reply(tid 0: 0 dumped all) ==== 18+0+339446 (secure 0 0 0) 0x7f45d4062390 con 0x7f45bc0a3e30 2024-04-21T08:38:57.951 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:38:57.952 INFO:teuthology.orchestra.run.smithi086.stderr:dumped all 2024-04-21T08:38:57.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.950+0000 7f45d8c5c640 1 -- 172.21.15.86:0/746196228 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f45bc0a3e30 msgr2=0x7f45bc0a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:57.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.950+0000 7f45d8c5c640 1 --2- 172.21.15.86:0/746196228 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f45bc0a3e30 0x7f45bc0a62f0 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f45cc006010 tx=0x7f45cc005e40 comp rx=0 tx=0).stop 2024-04-21T08:38:57.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.950+0000 7f45d8c5c640 1 -- 172.21.15.86:0/746196228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d4103af0 msgr2=0x7f45d419e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:57.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.950+0000 7f45d8c5c640 1 --2- 172.21.15.86:0/746196228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d4103af0 0x7f45d419e940 secure :-1 s=READY pgs=588 cs=0 l=1 rev1=1 crypto rx=0x7f45c800e9f0 tx=0x7f45c800eeb0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.950+0000 7f45d8c5c640 1 -- 172.21.15.86:0/746196228 shutdown_connections 2024-04-21T08:38:57.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.950+0000 7f45d8c5c640 1 --2- 172.21.15.86:0/746196228 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f45bc0a3e30 0x7f45bc0a62f0 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.950+0000 7f45d8c5c640 1 --2- 172.21.15.86:0/746196228 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f45d410c5a0 0x7f45d41985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.950+0000 7f45d8c5c640 1 --2- 172.21.15.86:0/746196228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f45d4103af0 0x7f45d419e940 unknown :-1 s=CLOSED pgs=588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.950+0000 7f45d8c5c640 1 --2- 172.21.15.86:0/746196228 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f45d41031d0 0x7f45d419e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:57.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.950+0000 7f45d8c5c640 1 -- 172.21.15.86:0/746196228 >> 172.21.15.86:0/746196228 conn(0x7f45d40771d0 msgr2=0x7f45d40ff070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:57.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.950+0000 7f45d8c5c640 1 -- 172.21.15.86:0/746196228 shutdown_connections 2024-04-21T08:38:57.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:57.950+0000 7f45d8c5c640 1 -- 172.21.15.86:0/746196228 wait complete. 2024-04-21T08:38:57.962 INFO:teuthology.orchestra.run.smithi086.stdout:{"pg_ready":true,"pg_map":{"version":8,"stamp":"2024-04-21T08:38:56.653889+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":1194594,"num_objects":104,"num_object_clones":0,"num_object_copies":314,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":104,"num_whiteouts":0,"num_read":1639,"num_read_kb":1148,"num_write":578,"num_write_kb":4413,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":23,"num_bytes_recovered":1181659,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":955,"ondisk_log_size":955,"up":419,"acting":419,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":419,"num_osds":12,"num_per_pool_osds":12,"num_per_pool_omap_osds":12,"kb":1124941824,"kb_used":390584,"kb_used_data":16376,"kb_used_omap":18,"kb_used_meta":373997,"kb_avail":1124551240,"statfs":{"total":1151940427776,"available":1151540469760,"internally_reserved":0,"allocated":16769024,"data_stored":9913268,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":19140,"internal_metadata":382973244},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":153,"num_read_kb":238,"num_write":20,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.993736"},"pg_stats":[{"pgid":"3.3f","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807854+0000","last_change":"2024-04-21T08:36:20.913421+0000","last_active":"2024-04-21T08:38:46.807854+0000","last_peered":"2024-04-21T08:38:46.807854+0000","last_clean":"2024-04-21T08:38:46.807854+0000","last_became_active":"2024-04-21T08:36:20.913017+0000","last_became_peered":"2024-04-21T08:36:20.913017+0000","last_unstale":"2024-04-21T08:38:46.807854+0000","last_undegraded":"2024-04-21T08:38:46.807854+0000","last_fullsized":"2024-04-21T08:38:46.807854+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:16:29.885916+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.3e","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.915359+0000","last_change":"2024-04-21T08:36:20.911675+0000","last_active":"2024-04-21T08:38:46.915359+0000","last_peered":"2024-04-21T08:38:46.915359+0000","last_clean":"2024-04-21T08:38:46.915359+0000","last_became_active":"2024-04-21T08:36:20.910591+0000","last_became_peered":"2024-04-21T08:36:20.910591+0000","last_unstale":"2024-04-21T08:38:46.915359+0000","last_undegraded":"2024-04-21T08:38:46.915359+0000","last_fullsized":"2024-04-21T08:38:46.915359+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:01:40.735376+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,11,10],"acting":[0,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3d","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.915070+0000","last_change":"2024-04-21T08:36:20.911978+0000","last_active":"2024-04-21T08:38:46.915070+0000","last_peered":"2024-04-21T08:38:46.915070+0000","last_clean":"2024-04-21T08:38:46.915070+0000","last_became_active":"2024-04-21T08:36:20.911641+0000","last_became_peered":"2024-04-21T08:36:20.911641+0000","last_unstale":"2024-04-21T08:38:46.915070+0000","last_undegraded":"2024-04-21T08:38:46.915070+0000","last_fullsized":"2024-04-21T08:38:46.915070+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:14:23.789790+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,2,1],"acting":[0,2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.3c","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911901+0000","last_change":"2024-04-21T08:36:20.919027+0000","last_active":"2024-04-21T08:38:46.911901+0000","last_peered":"2024-04-21T08:38:46.911901+0000","last_clean":"2024-04-21T08:38:46.911901+0000","last_became_active":"2024-04-21T08:36:20.918740+0000","last_became_peered":"2024-04-21T08:36:20.918740+0000","last_unstale":"2024-04-21T08:38:46.911901+0000","last_undegraded":"2024-04-21T08:38:46.911901+0000","last_fullsized":"2024-04-21T08:38:46.911901+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:00:31.066938+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,8,2],"acting":[4,8,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.3b","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655099+0000","last_change":"2024-04-21T08:36:20.907175+0000","last_active":"2024-04-21T08:38:46.655099+0000","last_peered":"2024-04-21T08:38:46.655099+0000","last_clean":"2024-04-21T08:38:46.655099+0000","last_became_active":"2024-04-21T08:36:20.906818+0000","last_became_peered":"2024-04-21T08:36:20.906818+0000","last_unstale":"2024-04-21T08:38:46.655099+0000","last_undegraded":"2024-04-21T08:38:46.655099+0000","last_fullsized":"2024-04-21T08:38:46.655099+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:41:03.332231+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,3,6],"acting":[8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.3a","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638267+0000","last_change":"2024-04-21T08:36:20.905968+0000","last_active":"2024-04-21T08:38:46.638267+0000","last_peered":"2024-04-21T08:38:46.638267+0000","last_clean":"2024-04-21T08:38:46.638267+0000","last_became_active":"2024-04-21T08:36:20.904840+0000","last_became_peered":"2024-04-21T08:36:20.904840+0000","last_unstale":"2024-04-21T08:38:46.638267+0000","last_undegraded":"2024-04-21T08:38:46.638267+0000","last_fullsized":"2024-04-21T08:38:46.638267+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:23:46.886241+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,4],"acting":[10,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.39","version":"91'5","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913625+0000","last_change":"2024-04-21T08:36:20.912045+0000","last_active":"2024-04-21T08:38:46.913625+0000","last_peered":"2024-04-21T08:38:46.913625+0000","last_clean":"2024-04-21T08:38:46.913625+0000","last_became_active":"2024-04-21T08:36:20.911690+0000","last_became_peered":"2024-04-21T08:36:20.911690+0000","last_unstale":"2024-04-21T08:38:46.913625+0000","last_undegraded":"2024-04-21T08:38:46.913625+0000","last_fullsized":"2024-04-21T08:38:46.913625+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:06:28.229307+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,9,5],"acting":[7,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.38","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638674+0000","last_change":"2024-04-21T08:36:20.904874+0000","last_active":"2024-04-21T08:38:46.638674+0000","last_peered":"2024-04-21T08:38:46.638674+0000","last_clean":"2024-04-21T08:38:46.638674+0000","last_became_active":"2024-04-21T08:36:20.904217+0000","last_became_peered":"2024-04-21T08:36:20.904217+0000","last_unstale":"2024-04-21T08:38:46.638674+0000","last_undegraded":"2024-04-21T08:38:46.638674+0000","last_fullsized":"2024-04-21T08:38:46.638674+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:37:04.423812+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,10],"acting":[3,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.37","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.923169+0000","last_change":"2024-04-21T08:36:20.912874+0000","last_active":"2024-04-21T08:38:46.923169+0000","last_peered":"2024-04-21T08:38:46.923169+0000","last_clean":"2024-04-21T08:38:46.923169+0000","last_became_active":"2024-04-21T08:36:20.912569+0000","last_became_peered":"2024-04-21T08:36:20.912569+0000","last_unstale":"2024-04-21T08:38:46.923169+0000","last_undegraded":"2024-04-21T08:38:46.923169+0000","last_fullsized":"2024-04-21T08:38:46.923169+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:55:20.763157+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,11],"acting":[1,5,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.36","version":"91'16","reported_seq":53,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638546+0000","last_change":"2024-04-21T08:37:24.296916+0000","last_active":"2024-04-21T08:38:46.638546+0000","last_peered":"2024-04-21T08:38:46.638546+0000","last_clean":"2024-04-21T08:38:46.638546+0000","last_became_active":"2024-04-21T08:37:24.296547+0000","last_became_peered":"2024-04-21T08:37:24.296547+0000","last_unstale":"2024-04-21T08:38:46.638546+0000","last_undegraded":"2024-04-21T08:38:46.638546+0000","last_fullsized":"2024-04-21T08:38:46.638546+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":16,"log_dups_size":0,"ondisk_log_size":16,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:47:56.683793+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,2,9],"acting":[10,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.35","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914246+0000","last_change":"2024-04-21T08:36:20.909094+0000","last_active":"2024-04-21T08:38:46.914246+0000","last_peered":"2024-04-21T08:38:46.914246+0000","last_clean":"2024-04-21T08:38:46.914246+0000","last_became_active":"2024-04-21T08:36:20.908634+0000","last_became_peered":"2024-04-21T08:36:20.908634+0000","last_unstale":"2024-04-21T08:38:46.914246+0000","last_undegraded":"2024-04-21T08:38:46.914246+0000","last_fullsized":"2024-04-21T08:38:46.914246+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:28:38.450621+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,8,1],"acting":[7,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.34","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911748+0000","last_change":"2024-04-21T08:36:20.909429+0000","last_active":"2024-04-21T08:38:46.911748+0000","last_peered":"2024-04-21T08:38:46.911748+0000","last_clean":"2024-04-21T08:38:46.911748+0000","last_became_active":"2024-04-21T08:36:20.909081+0000","last_became_peered":"2024-04-21T08:36:20.909081+0000","last_unstale":"2024-04-21T08:38:46.911748+0000","last_undegraded":"2024-04-21T08:38:46.911748+0000","last_fullsized":"2024-04-21T08:38:46.911748+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:45:46.939125+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,8],"acting":[4,10,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.33","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913919+0000","last_change":"2024-04-21T08:36:20.895894+0000","last_active":"2024-04-21T08:38:46.913919+0000","last_peered":"2024-04-21T08:38:46.913919+0000","last_clean":"2024-04-21T08:38:46.913919+0000","last_became_active":"2024-04-21T08:36:20.895572+0000","last_became_peered":"2024-04-21T08:36:20.895572+0000","last_unstale":"2024-04-21T08:38:46.913919+0000","last_undegraded":"2024-04-21T08:38:46.913919+0000","last_fullsized":"2024-04-21T08:38:46.913919+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:12:48.750555+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,10,5],"acting":[6,10,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.32","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914638+0000","last_change":"2024-04-21T08:36:20.905356+0000","last_active":"2024-04-21T08:38:46.914638+0000","last_peered":"2024-04-21T08:38:46.914638+0000","last_clean":"2024-04-21T08:38:46.914638+0000","last_became_active":"2024-04-21T08:36:20.904954+0000","last_became_peered":"2024-04-21T08:36:20.904954+0000","last_unstale":"2024-04-21T08:38:46.914638+0000","last_undegraded":"2024-04-21T08:38:46.914638+0000","last_fullsized":"2024-04-21T08:38:46.914638+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:45:49.116482+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,8,10],"acting":[0,8,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.31","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.915244+0000","last_change":"2024-04-21T08:36:20.911550+0000","last_active":"2024-04-21T08:38:46.915244+0000","last_peered":"2024-04-21T08:38:46.915244+0000","last_clean":"2024-04-21T08:38:46.915244+0000","last_became_active":"2024-04-21T08:36:20.910557+0000","last_became_peered":"2024-04-21T08:36:20.910557+0000","last_unstale":"2024-04-21T08:38:46.915244+0000","last_undegraded":"2024-04-21T08:38:46.915244+0000","last_fullsized":"2024-04-21T08:38:46.915244+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:09:49.422729+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,2],"acting":[0,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.30","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.922413+0000","last_change":"2024-04-21T08:36:20.903954+0000","last_active":"2024-04-21T08:38:46.922413+0000","last_peered":"2024-04-21T08:38:46.922413+0000","last_clean":"2024-04-21T08:38:46.922413+0000","last_became_active":"2024-04-21T08:36:20.903641+0000","last_became_peered":"2024-04-21T08:36:20.903641+0000","last_unstale":"2024-04-21T08:38:46.922413+0000","last_undegraded":"2024-04-21T08:38:46.922413+0000","last_fullsized":"2024-04-21T08:38:46.922413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:08:55.678940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,0],"acting":[1,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2f","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807825+0000","last_change":"2024-04-21T08:36:20.914256+0000","last_active":"2024-04-21T08:38:46.807825+0000","last_peered":"2024-04-21T08:38:46.807825+0000","last_clean":"2024-04-21T08:38:46.807825+0000","last_became_active":"2024-04-21T08:36:20.913858+0000","last_became_peered":"2024-04-21T08:36:20.913858+0000","last_unstale":"2024-04-21T08:38:46.807825+0000","last_undegraded":"2024-04-21T08:38:46.807825+0000","last_fullsized":"2024-04-21T08:38:46.807825+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:01:10.276763+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,1,0],"acting":[11,1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.2e","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.743080+0000","last_change":"2024-04-21T08:36:20.904138+0000","last_active":"2024-04-21T08:38:46.743080+0000","last_peered":"2024-04-21T08:38:46.743080+0000","last_clean":"2024-04-21T08:38:46.743080+0000","last_became_active":"2024-04-21T08:36:20.903670+0000","last_became_peered":"2024-04-21T08:36:20.903670+0000","last_unstale":"2024-04-21T08:38:46.743080+0000","last_undegraded":"2024-04-21T08:38:46.743080+0000","last_fullsized":"2024-04-21T08:38:46.743080+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:38:50.160867+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,1,10],"acting":[9,1,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.2d","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639071+0000","last_change":"2024-04-21T08:36:20.910798+0000","last_active":"2024-04-21T08:38:46.639071+0000","last_peered":"2024-04-21T08:38:46.639071+0000","last_clean":"2024-04-21T08:38:46.639071+0000","last_became_active":"2024-04-21T08:36:20.910411+0000","last_became_peered":"2024-04-21T08:36:20.910411+0000","last_unstale":"2024-04-21T08:38:46.639071+0000","last_undegraded":"2024-04-21T08:38:46.639071+0000","last_fullsized":"2024-04-21T08:38:46.639071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:13:12.109818+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,6],"acting":[2,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"4.19","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638676+0000","last_change":"2024-04-21T08:36:24.924831+0000","last_active":"2024-04-21T08:38:46.638676+0000","last_peered":"2024-04-21T08:38:46.638676+0000","last_clean":"2024-04-21T08:38:46.638676+0000","last_became_active":"2024-04-21T08:36:24.924435+0000","last_became_peered":"2024-04-21T08:36:24.924435+0000","last_unstale":"2024-04-21T08:38:46.638676+0000","last_undegraded":"2024-04-21T08:38:46.638676+0000","last_fullsized":"2024-04-21T08:38:46.638676+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:29:23.447079+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,9,0],"acting":[2,3,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1f","version":"89'13","reported_seq":194,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:52.412380+0000","last_change":"2024-04-21T08:37:24.308024+0000","last_active":"2024-04-21T08:38:52.412380+0000","last_peered":"2024-04-21T08:38:52.412380+0000","last_clean":"2024-04-21T08:38:52.412380+0000","last_became_active":"2024-04-21T08:37:24.297491+0000","last_became_peered":"2024-04-21T08:37:24.297491+0000","last_unstale":"2024-04-21T08:38:52.412380+0000","last_undegraded":"2024-04-21T08:38:52.412380+0000","last_fullsized":"2024-04-21T08:38:52.412380+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":13,"log_dups_size":0,"ondisk_log_size":13,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:48:10.039545+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":120,"num_read_kb":120,"num_write":14,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":6,"num_bytes_recovered":672,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,11,8],"acting":[9,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.1e","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639158+0000","last_change":"2024-04-21T08:36:20.912851+0000","last_active":"2024-04-21T08:38:46.639158+0000","last_peered":"2024-04-21T08:38:46.639158+0000","last_clean":"2024-04-21T08:38:46.639158+0000","last_became_active":"2024-04-21T08:36:20.912407+0000","last_became_peered":"2024-04-21T08:36:20.912407+0000","last_unstale":"2024-04-21T08:38:46.639158+0000","last_undegraded":"2024-04-21T08:38:46.639158+0000","last_fullsized":"2024-04-21T08:38:46.639158+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:45:18.026940+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,5],"acting":[3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.18","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.910950+0000","last_change":"2024-04-21T08:36:24.924052+0000","last_active":"2024-04-21T08:38:46.910950+0000","last_peered":"2024-04-21T08:38:46.910950+0000","last_clean":"2024-04-21T08:38:46.910950+0000","last_became_active":"2024-04-21T08:36:24.923618+0000","last_became_peered":"2024-04-21T08:36:24.923618+0000","last_unstale":"2024-04-21T08:38:46.910950+0000","last_undegraded":"2024-04-21T08:38:46.910950+0000","last_fullsized":"2024-04-21T08:38:46.910950+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:08:18.182386+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,9,3],"acting":[4,6,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1e","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639180+0000","last_change":"2024-04-21T08:36:20.285291+0000","last_active":"2024-04-21T08:38:46.639180+0000","last_peered":"2024-04-21T08:38:46.639180+0000","last_clean":"2024-04-21T08:38:46.639180+0000","last_became_active":"2024-04-21T08:36:20.284969+0000","last_became_peered":"2024-04-21T08:36:20.284969+0000","last_unstale":"2024-04-21T08:38:46.639180+0000","last_undegraded":"2024-04-21T08:38:46.639180+0000","last_fullsized":"2024-04-21T08:38:46.639180+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:55:29.484439+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,5],"acting":[3,0,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.1f","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.915781+0000","last_change":"2024-04-21T08:36:20.902819+0000","last_active":"2024-04-21T08:38:46.915781+0000","last_peered":"2024-04-21T08:38:46.915781+0000","last_clean":"2024-04-21T08:38:46.915781+0000","last_became_active":"2024-04-21T08:36:20.902466+0000","last_became_peered":"2024-04-21T08:36:20.902466+0000","last_unstale":"2024-04-21T08:38:46.915781+0000","last_undegraded":"2024-04-21T08:38:46.915781+0000","last_fullsized":"2024-04-21T08:38:46.915781+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:29:40.454026+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1,9],"acting":[0,1,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.1b","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911156+0000","last_change":"2024-04-21T08:36:24.926480+0000","last_active":"2024-04-21T08:38:46.911156+0000","last_peered":"2024-04-21T08:38:46.911156+0000","last_clean":"2024-04-21T08:38:46.911156+0000","last_became_active":"2024-04-21T08:36:24.926116+0000","last_became_peered":"2024-04-21T08:36:24.926116+0000","last_unstale":"2024-04-21T08:38:46.911156+0000","last_undegraded":"2024-04-21T08:38:46.911156+0000","last_fullsized":"2024-04-21T08:38:46.911156+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:43:18.839396+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,3,9,5],"acting":[4,3,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1d","version":"89'12","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655608+0000","last_change":"2024-04-21T08:36:20.293086+0000","last_active":"2024-04-21T08:38:46.655608+0000","last_peered":"2024-04-21T08:38:46.655608+0000","last_clean":"2024-04-21T08:38:46.655608+0000","last_became_active":"2024-04-21T08:36:20.291178+0000","last_became_peered":"2024-04-21T08:36:20.291178+0000","last_unstale":"2024-04-21T08:38:46.655608+0000","last_undegraded":"2024-04-21T08:38:46.655608+0000","last_fullsized":"2024-04-21T08:38:46.655608+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:02:32.935930+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":7,"num_object_clones":0,"num_object_copies":21,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":7,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":16,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":7,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1c","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.922777+0000","last_change":"2024-04-21T08:36:20.909374+0000","last_active":"2024-04-21T08:38:46.922777+0000","last_peered":"2024-04-21T08:38:46.922777+0000","last_clean":"2024-04-21T08:38:46.922777+0000","last_became_active":"2024-04-21T08:36:20.909043+0000","last_became_peered":"2024-04-21T08:36:20.909043+0000","last_unstale":"2024-04-21T08:38:46.922777+0000","last_undegraded":"2024-04-21T08:38:46.922777+0000","last_fullsized":"2024-04-21T08:38:46.922777+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:45:45.719942+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,6],"acting":[1,10,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.1a","version":"0'0","reported_seq":31,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911098+0000","last_change":"2024-04-21T08:38:24.623194+0000","last_active":"2024-04-21T08:38:46.911098+0000","last_peered":"2024-04-21T08:38:46.911098+0000","last_clean":"2024-04-21T08:38:46.911098+0000","last_became_active":"2024-04-21T08:38:24.622632+0000","last_became_peered":"2024-04-21T08:38:24.622632+0000","last_unstale":"2024-04-21T08:38:46.911098+0000","last_undegraded":"2024-04-21T08:38:46.911098+0000","last_fullsized":"2024-04-21T08:38:46.911098+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:26:39.752161+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,6,8,3],"acting":[4,6,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.1c","version":"89'5","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655483+0000","last_change":"2024-04-21T08:36:20.288644+0000","last_active":"2024-04-21T08:38:46.655483+0000","last_peered":"2024-04-21T08:38:46.655483+0000","last_clean":"2024-04-21T08:38:46.655483+0000","last_became_active":"2024-04-21T08:36:20.288069+0000","last_became_peered":"2024-04-21T08:36:20.288069+0000","last_unstale":"2024-04-21T08:38:46.655483+0000","last_undegraded":"2024-04-21T08:38:46.655483+0000","last_fullsized":"2024-04-21T08:38:46.655483+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:37:36.522713+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,1,5],"acting":[8,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.1d","version":"91'15","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751431+0000","last_change":"2024-04-21T08:36:20.908505+0000","last_active":"2024-04-21T08:38:46.751431+0000","last_peered":"2024-04-21T08:38:46.751431+0000","last_clean":"2024-04-21T08:38:46.751431+0000","last_became_active":"2024-04-21T08:36:20.908163+0000","last_became_peered":"2024-04-21T08:36:20.908163+0000","last_unstale":"2024-04-21T08:38:46.751431+0000","last_undegraded":"2024-04-21T08:38:46.751431+0000","last_fullsized":"2024-04-21T08:38:46.751431+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:54:18.530203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1d","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639160+0000","last_change":"2024-04-21T08:36:24.919428+0000","last_active":"2024-04-21T08:38:46.639160+0000","last_peered":"2024-04-21T08:38:46.639160+0000","last_clean":"2024-04-21T08:38:46.639160+0000","last_became_active":"2024-04-21T08:36:24.918722+0000","last_became_peered":"2024-04-21T08:36:24.918722+0000","last_unstale":"2024-04-21T08:38:46.639160+0000","last_undegraded":"2024-04-21T08:38:46.639160+0000","last_fullsized":"2024-04-21T08:38:46.639160+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:30:32.369551+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,7,11],"acting":[2,5,7,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1b","version":"93'8","reported_seq":174,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:52.641167+0000","last_change":"2024-04-21T08:36:20.287642+0000","last_active":"2024-04-21T08:38:52.641167+0000","last_peered":"2024-04-21T08:38:52.641167+0000","last_clean":"2024-04-21T08:38:52.641167+0000","last_became_active":"2024-04-21T08:36:20.287296+0000","last_became_peered":"2024-04-21T08:36:20.287296+0000","last_unstale":"2024-04-21T08:38:52.641167+0000","last_undegraded":"2024-04-21T08:38:52.641167+0000","last_fullsized":"2024-04-21T08:38:52.641167+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:59:28.579020+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1462,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":133,"num_read_kb":5,"num_write":5,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7],"acting":[5,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1a","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655128+0000","last_change":"2024-04-21T08:36:20.916971+0000","last_active":"2024-04-21T08:38:46.655128+0000","last_peered":"2024-04-21T08:38:46.655128+0000","last_clean":"2024-04-21T08:38:46.655128+0000","last_became_active":"2024-04-21T08:36:20.916622+0000","last_became_peered":"2024-04-21T08:36:20.916622+0000","last_unstale":"2024-04-21T08:38:46.655128+0000","last_undegraded":"2024-04-21T08:38:46.655128+0000","last_fullsized":"2024-04-21T08:38:46.655128+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:49:38.858772+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,9,5],"acting":[8,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.1c","version":"0'0","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638856+0000","last_change":"2024-04-21T08:36:24.924170+0000","last_active":"2024-04-21T08:38:46.638856+0000","last_peered":"2024-04-21T08:38:46.638856+0000","last_clean":"2024-04-21T08:38:46.638856+0000","last_became_active":"2024-04-21T08:36:24.923744+0000","last_became_peered":"2024-04-21T08:36:24.923744+0000","last_unstale":"2024-04-21T08:38:46.638856+0000","last_undegraded":"2024-04-21T08:38:46.638856+0000","last_fullsized":"2024-04-21T08:38:46.638856+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:23:24.605021+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6,9,5],"acting":[2,6,9,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1a","version":"89'7","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638388+0000","last_change":"2024-04-21T08:36:20.286545+0000","last_active":"2024-04-21T08:38:46.638388+0000","last_peered":"2024-04-21T08:38:46.638388+0000","last_clean":"2024-04-21T08:38:46.638388+0000","last_became_active":"2024-04-21T08:36:20.286197+0000","last_became_peered":"2024-04-21T08:36:20.286197+0000","last_unstale":"2024-04-21T08:38:46.638388+0000","last_undegraded":"2024-04-21T08:38:46.638388+0000","last_fullsized":"2024-04-21T08:38:46.638388+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:27:55.430428+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":9,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,8,6],"acting":[10,8,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.1b","version":"91'8","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751535+0000","last_change":"2024-04-21T08:36:20.907743+0000","last_active":"2024-04-21T08:38:46.751535+0000","last_peered":"2024-04-21T08:38:46.751535+0000","last_clean":"2024-04-21T08:38:46.751535+0000","last_became_active":"2024-04-21T08:36:20.907163+0000","last_became_peered":"2024-04-21T08:36:20.907163+0000","last_unstale":"2024-04-21T08:38:46.751535+0000","last_undegraded":"2024-04-21T08:38:46.751535+0000","last_fullsized":"2024-04-21T08:38:46.751535+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:41:00.172353+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,3],"acting":[5,4,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.1f","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807766+0000","last_change":"2024-04-21T08:37:24.301529+0000","last_active":"2024-04-21T08:38:46.807766+0000","last_peered":"2024-04-21T08:38:46.807766+0000","last_clean":"2024-04-21T08:38:46.807766+0000","last_became_active":"2024-04-21T08:37:24.301014+0000","last_became_peered":"2024-04-21T08:37:24.301014+0000","last_unstale":"2024-04-21T08:38:46.807766+0000","last_undegraded":"2024-04-21T08:38:46.807766+0000","last_fullsized":"2024-04-21T08:38:46.807766+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:58:20.088501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8,0],"acting":[11,4,8,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.19","version":"89'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.912093+0000","last_change":"2024-04-21T08:36:20.283143+0000","last_active":"2024-04-21T08:38:46.912093+0000","last_peered":"2024-04-21T08:38:46.912093+0000","last_clean":"2024-04-21T08:38:46.912093+0000","last_became_active":"2024-04-21T08:36:20.282585+0000","last_became_peered":"2024-04-21T08:36:20.282585+0000","last_unstale":"2024-04-21T08:38:46.912093+0000","last_undegraded":"2024-04-21T08:38:46.912093+0000","last_fullsized":"2024-04-21T08:38:46.912093+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:12:37.588764+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":5,"num_write_kb":6,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,9],"acting":[6,4,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.18","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638543+0000","last_change":"2024-04-21T08:36:20.912591+0000","last_active":"2024-04-21T08:38:46.638543+0000","last_peered":"2024-04-21T08:38:46.638543+0000","last_clean":"2024-04-21T08:38:46.638543+0000","last_became_active":"2024-04-21T08:36:20.912174+0000","last_became_peered":"2024-04-21T08:36:20.912174+0000","last_unstale":"2024-04-21T08:38:46.638543+0000","last_undegraded":"2024-04-21T08:38:46.638543+0000","last_fullsized":"2024-04-21T08:38:46.638543+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:33:58.949506+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,6],"acting":[3,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.915307+0000","last_change":"2024-04-21T08:36:24.915998+0000","last_active":"2024-04-21T08:38:46.915307+0000","last_peered":"2024-04-21T08:38:46.915307+0000","last_clean":"2024-04-21T08:38:46.915307+0000","last_became_active":"2024-04-21T08:36:24.915621+0000","last_became_peered":"2024-04-21T08:36:24.915621+0000","last_unstale":"2024-04-21T08:38:46.915307+0000","last_undegraded":"2024-04-21T08:38:46.915307+0000","last_fullsized":"2024-04-21T08:38:46.915307+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:43:05.117181+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,7,6,1],"acting":[0,7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.18","version":"89'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751087+0000","last_change":"2024-04-21T08:36:20.286308+0000","last_active":"2024-04-21T08:38:46.751087+0000","last_peered":"2024-04-21T08:38:46.751087+0000","last_clean":"2024-04-21T08:38:46.751087+0000","last_became_active":"2024-04-21T08:36:20.285975+0000","last_became_peered":"2024-04-21T08:36:20.285975+0000","last_unstale":"2024-04-21T08:38:46.751087+0000","last_undegraded":"2024-04-21T08:38:46.751087+0000","last_fullsized":"2024-04-21T08:38:46.751087+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:55:10.658349+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":582,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,1,7],"acting":[5,1,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.19","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.742821+0000","last_change":"2024-04-21T08:36:20.900502+0000","last_active":"2024-04-21T08:38:46.742821+0000","last_peered":"2024-04-21T08:38:46.742821+0000","last_clean":"2024-04-21T08:38:46.742821+0000","last_became_active":"2024-04-21T08:36:20.900199+0000","last_became_peered":"2024-04-21T08:36:20.900199+0000","last_unstale":"2024-04-21T08:38:46.742821+0000","last_undegraded":"2024-04-21T08:38:46.742821+0000","last_fullsized":"2024-04-21T08:38:46.742821+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:17:00.720918+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,10],"acting":[9,3,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"4.11","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638804+0000","last_change":"2024-04-21T08:36:24.920991+0000","last_active":"2024-04-21T08:38:46.638804+0000","last_peered":"2024-04-21T08:38:46.638804+0000","last_clean":"2024-04-21T08:38:46.638804+0000","last_became_active":"2024-04-21T08:36:24.916611+0000","last_became_peered":"2024-04-21T08:36:24.916611+0000","last_unstale":"2024-04-21T08:38:46.638804+0000","last_undegraded":"2024-04-21T08:38:46.638804+0000","last_fullsized":"2024-04-21T08:38:46.638804+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:01:01.412127+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,7,1,5],"acting":[3,7,1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.17","version":"93'7","reported_seq":293,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:51.661122+0000","last_change":"2024-04-21T08:36:20.284966+0000","last_active":"2024-04-21T08:38:51.661122+0000","last_peered":"2024-04-21T08:38:51.661122+0000","last_clean":"2024-04-21T08:38:51.661122+0000","last_became_active":"2024-04-21T08:36:20.284612+0000","last_became_peered":"2024-04-21T08:36:20.284612+0000","last_unstale":"2024-04-21T08:38:51.661122+0000","last_undegraded":"2024-04-21T08:38:51.661122+0000","last_fullsized":"2024-04-21T08:38:51.661122+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:54:50.095013+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":180,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":253,"num_read_kb":253,"num_write":8,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,5,9],"acting":[10,5,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.16","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.923175+0000","last_change":"2024-04-21T08:36:20.903493+0000","last_active":"2024-04-21T08:38:46.923175+0000","last_peered":"2024-04-21T08:38:46.923175+0000","last_clean":"2024-04-21T08:38:46.923175+0000","last_became_active":"2024-04-21T08:36:20.902752+0000","last_became_peered":"2024-04-21T08:36:20.902752+0000","last_unstale":"2024-04-21T08:38:46.923175+0000","last_undegraded":"2024-04-21T08:38:46.923175+0000","last_fullsized":"2024-04-21T08:38:46.923175+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:40.684264+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,10,9],"acting":[1,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.10","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.742602+0000","last_change":"2024-04-21T08:36:24.920505+0000","last_active":"2024-04-21T08:38:46.742602+0000","last_peered":"2024-04-21T08:38:46.742602+0000","last_clean":"2024-04-21T08:38:46.742602+0000","last_became_active":"2024-04-21T08:36:24.919966+0000","last_became_peered":"2024-04-21T08:36:24.919966+0000","last_unstale":"2024-04-21T08:38:46.742602+0000","last_undegraded":"2024-04-21T08:38:46.742602+0000","last_fullsized":"2024-04-21T08:38:46.742602+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:58:05.131815+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,0,2,4],"acting":[9,0,2,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.16","version":"89'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914242+0000","last_change":"2024-04-21T08:36:20.281720+0000","last_active":"2024-04-21T08:38:46.914242+0000","last_peered":"2024-04-21T08:38:46.914242+0000","last_clean":"2024-04-21T08:38:46.914242+0000","last_became_active":"2024-04-21T08:36:20.279862+0000","last_became_peered":"2024-04-21T08:36:20.279862+0000","last_unstale":"2024-04-21T08:38:46.914242+0000","last_undegraded":"2024-04-21T08:38:46.914242+0000","last_fullsized":"2024-04-21T08:38:46.914242+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:18:58.106532+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,2],"acting":[7,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.17","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.916030+0000","last_change":"2024-04-21T08:36:20.908520+0000","last_active":"2024-04-21T08:38:46.916030+0000","last_peered":"2024-04-21T08:38:46.916030+0000","last_clean":"2024-04-21T08:38:46.916030+0000","last_became_active":"2024-04-21T08:36:20.908184+0000","last_became_peered":"2024-04-21T08:36:20.908184+0000","last_unstale":"2024-04-21T08:38:46.916030+0000","last_undegraded":"2024-04-21T08:38:46.916030+0000","last_fullsized":"2024-04-21T08:38:46.916030+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:14:33.895864+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4,5],"acting":[0,4,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.13","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911465+0000","last_change":"2024-04-21T08:36:24.920697+0000","last_active":"2024-04-21T08:38:46.911465+0000","last_peered":"2024-04-21T08:38:46.911465+0000","last_clean":"2024-04-21T08:38:46.911465+0000","last_became_active":"2024-04-21T08:36:24.919976+0000","last_became_peered":"2024-04-21T08:36:24.919976+0000","last_unstale":"2024-04-21T08:38:46.911465+0000","last_undegraded":"2024-04-21T08:38:46.911465+0000","last_fullsized":"2024-04-21T08:38:46.911465+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:40:09.238887+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1,2,9],"acting":[4,1,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.15","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751422+0000","last_change":"2024-04-21T08:36:20.283531+0000","last_active":"2024-04-21T08:38:46.751422+0000","last_peered":"2024-04-21T08:38:46.751422+0000","last_clean":"2024-04-21T08:38:46.751422+0000","last_became_active":"2024-04-21T08:36:20.283159+0000","last_became_peered":"2024-04-21T08:36:20.283159+0000","last_unstale":"2024-04-21T08:38:46.751422+0000","last_undegraded":"2024-04-21T08:38:46.751422+0000","last_fullsized":"2024-04-21T08:38:46.751422+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:57:02.917537+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":672,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":7,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,10],"acting":[5,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.14","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.922897+0000","last_change":"2024-04-21T08:36:20.910113+0000","last_active":"2024-04-21T08:38:46.922897+0000","last_peered":"2024-04-21T08:38:46.922897+0000","last_clean":"2024-04-21T08:38:46.922897+0000","last_became_active":"2024-04-21T08:36:20.909744+0000","last_became_peered":"2024-04-21T08:36:20.909744+0000","last_unstale":"2024-04-21T08:38:46.922897+0000","last_undegraded":"2024-04-21T08:38:46.922897+0000","last_fullsized":"2024-04-21T08:38:46.922897+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:16:14.568318+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,8,4],"acting":[1,8,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"4.12","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.742678+0000","last_change":"2024-04-21T08:36:24.922232+0000","last_active":"2024-04-21T08:38:46.742678+0000","last_peered":"2024-04-21T08:38:46.742678+0000","last_clean":"2024-04-21T08:38:46.742678+0000","last_became_active":"2024-04-21T08:36:24.921690+0000","last_became_peered":"2024-04-21T08:36:24.921690+0000","last_unstale":"2024-04-21T08:38:46.742678+0000","last_undegraded":"2024-04-21T08:38:46.742678+0000","last_fullsized":"2024-04-21T08:38:46.742678+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:21:42.580203+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,11,5],"acting":[9,8,11,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.14","version":"94'42","reported_seq":343,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:52.640372+0000","last_change":"2024-04-21T08:36:20.283574+0000","last_active":"2024-04-21T08:38:52.640372+0000","last_peered":"2024-04-21T08:38:52.640372+0000","last_clean":"2024-04-21T08:38:52.640372+0000","last_became_active":"2024-04-21T08:36:20.283220+0000","last_became_peered":"2024-04-21T08:36:20.283220+0000","last_unstale":"2024-04-21T08:38:52.640372+0000","last_undegraded":"2024-04-21T08:38:52.640372+0000","last_fullsized":"2024-04-21T08:38:52.640372+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":42,"log_dups_size":0,"ondisk_log_size":42,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:55:24.233237+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":70068,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":268,"num_read_kb":203,"num_write":44,"num_write_kb":89,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,1,2],"acting":[10,1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"3.15","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655458+0000","last_change":"2024-04-21T08:36:20.919264+0000","last_active":"2024-04-21T08:38:46.655458+0000","last_peered":"2024-04-21T08:38:46.655458+0000","last_clean":"2024-04-21T08:38:46.655458+0000","last_became_active":"2024-04-21T08:36:20.918878+0000","last_became_peered":"2024-04-21T08:36:20.918878+0000","last_unstale":"2024-04-21T08:38:46.655458+0000","last_undegraded":"2024-04-21T08:38:46.655458+0000","last_fullsized":"2024-04-21T08:38:46.655458+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:24:06.460758+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,7],"acting":[8,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.15","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751721+0000","last_change":"2024-04-21T08:36:24.926878+0000","last_active":"2024-04-21T08:38:46.751721+0000","last_peered":"2024-04-21T08:38:46.751721+0000","last_clean":"2024-04-21T08:38:46.751721+0000","last_became_active":"2024-04-21T08:36:24.926431+0000","last_became_peered":"2024-04-21T08:36:24.926431+0000","last_unstale":"2024-04-21T08:38:46.751721+0000","last_undegraded":"2024-04-21T08:38:46.751721+0000","last_fullsized":"2024-04-21T08:38:46.751721+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:51:29.065962+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,8,3,6],"acting":[5,8,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.13","version":"89'9","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639141+0000","last_change":"2024-04-21T08:36:20.284841+0000","last_active":"2024-04-21T08:38:46.639141+0000","last_peered":"2024-04-21T08:38:46.639141+0000","last_clean":"2024-04-21T08:38:46.639141+0000","last_became_active":"2024-04-21T08:36:20.283888+0000","last_became_peered":"2024-04-21T08:36:20.283888+0000","last_unstale":"2024-04-21T08:38:46.639141+0000","last_undegraded":"2024-04-21T08:38:46.639141+0000","last_fullsized":"2024-04-21T08:38:46.639141+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:29:42.659608+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":6,"num_object_clones":0,"num_object_copies":18,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":6,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":13,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,10,3],"acting":[2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.12","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.912243+0000","last_change":"2024-04-21T08:36:20.905140+0000","last_active":"2024-04-21T08:38:46.912243+0000","last_peered":"2024-04-21T08:38:46.912243+0000","last_clean":"2024-04-21T08:38:46.912243+0000","last_became_active":"2024-04-21T08:36:20.904821+0000","last_became_peered":"2024-04-21T08:36:20.904821+0000","last_unstale":"2024-04-21T08:38:46.912243+0000","last_undegraded":"2024-04-21T08:38:46.912243+0000","last_fullsized":"2024-04-21T08:38:46.912243+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:09:11.962877+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,3],"acting":[6,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.14","version":"0'0","reported_seq":32,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807527+0000","last_change":"2024-04-21T08:38:24.834006+0000","last_active":"2024-04-21T08:38:46.807527+0000","last_peered":"2024-04-21T08:38:46.807527+0000","last_clean":"2024-04-21T08:38:46.807527+0000","last_became_active":"2024-04-21T08:38:24.833453+0000","last_became_peered":"2024-04-21T08:38:24.833453+0000","last_unstale":"2024-04-21T08:38:46.807527+0000","last_undegraded":"2024-04-21T08:38:46.807527+0000","last_fullsized":"2024-04-21T08:38:46.807527+0000","mapping_epoch":92,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":93,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:27:17.094301+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,7,6],"acting":[11,8,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.12","version":"89'5","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751268+0000","last_change":"2024-04-21T08:36:20.290289+0000","last_active":"2024-04-21T08:38:46.751268+0000","last_peered":"2024-04-21T08:38:46.751268+0000","last_clean":"2024-04-21T08:38:46.751268+0000","last_became_active":"2024-04-21T08:36:20.289922+0000","last_became_peered":"2024-04-21T08:36:20.289922+0000","last_unstale":"2024-04-21T08:38:46.751268+0000","last_undegraded":"2024-04-21T08:38:46.751268+0000","last_fullsized":"2024-04-21T08:38:46.751268+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:10:00.926023+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":5,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,3,7],"acting":[5,3,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.13","version":"91'15","reported_seq":46,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914048+0000","last_change":"2024-04-21T08:36:20.906950+0000","last_active":"2024-04-21T08:38:46.914048+0000","last_peered":"2024-04-21T08:38:46.914048+0000","last_clean":"2024-04-21T08:38:46.914048+0000","last_became_active":"2024-04-21T08:36:20.906599+0000","last_became_peered":"2024-04-21T08:36:20.906599+0000","last_unstale":"2024-04-21T08:38:46.914048+0000","last_undegraded":"2024-04-21T08:38:46.914048+0000","last_fullsized":"2024-04-21T08:38:46.914048+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:55:27.595997+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,4,6],"acting":[7,4,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.17","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.743162+0000","last_change":"2024-04-21T08:36:24.924349+0000","last_active":"2024-04-21T08:38:46.743162+0000","last_peered":"2024-04-21T08:38:46.743162+0000","last_clean":"2024-04-21T08:38:46.743162+0000","last_became_active":"2024-04-21T08:36:24.923982+0000","last_became_peered":"2024-04-21T08:36:24.923982+0000","last_unstale":"2024-04-21T08:38:46.743162+0000","last_undegraded":"2024-04-21T08:38:46.743162+0000","last_fullsized":"2024-04-21T08:38:46.743162+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:20:29.600759+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,7,0,3],"acting":[9,7,0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.11","version":"94'9","reported_seq":175,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:51.641095+0000","last_change":"2024-04-21T08:36:20.285629+0000","last_active":"2024-04-21T08:38:51.641095+0000","last_peered":"2024-04-21T08:38:51.641095+0000","last_clean":"2024-04-21T08:38:51.641095+0000","last_became_active":"2024-04-21T08:36:20.285234+0000","last_became_peered":"2024-04-21T08:36:20.285234+0000","last_unstale":"2024-04-21T08:38:51.641095+0000","last_undegraded":"2024-04-21T08:38:51.641095+0000","last_fullsized":"2024-04-21T08:38:51.641095+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:19:32.966146+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1902,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":134,"num_read_kb":6,"num_write":9,"num_write_kb":5,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,7,10],"acting":[4,7,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.10","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913981+0000","last_change":"2024-04-21T08:36:20.906819+0000","last_active":"2024-04-21T08:38:46.913981+0000","last_peered":"2024-04-21T08:38:46.913981+0000","last_clean":"2024-04-21T08:38:46.913981+0000","last_became_active":"2024-04-21T08:36:20.906459+0000","last_became_peered":"2024-04-21T08:36:20.906459+0000","last_unstale":"2024-04-21T08:38:46.913981+0000","last_undegraded":"2024-04-21T08:38:46.913981+0000","last_fullsized":"2024-04-21T08:38:46.913981+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:12:39.312857+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.16","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913253+0000","last_change":"2024-04-21T08:36:24.929452+0000","last_active":"2024-04-21T08:38:46.913253+0000","last_peered":"2024-04-21T08:38:46.913253+0000","last_clean":"2024-04-21T08:38:46.913253+0000","last_became_active":"2024-04-21T08:36:24.929039+0000","last_became_peered":"2024-04-21T08:36:24.929039+0000","last_unstale":"2024-04-21T08:38:46.913253+0000","last_undegraded":"2024-04-21T08:38:46.913253+0000","last_fullsized":"2024-04-21T08:38:46.913253+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:16:05.432024+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,9,11],"acting":[6,3,9,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.10","version":"89'17","reported_seq":76,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655055+0000","last_change":"2024-04-21T08:37:24.719766+0000","last_active":"2024-04-21T08:38:46.655055+0000","last_peered":"2024-04-21T08:38:46.655055+0000","last_clean":"2024-04-21T08:38:46.655055+0000","last_became_active":"2024-04-21T08:37:24.708201+0000","last_became_peered":"2024-04-21T08:37:24.708201+0000","last_unstale":"2024-04-21T08:38:46.655055+0000","last_undegraded":"2024-04-21T08:38:46.655055+0000","last_fullsized":"2024-04-21T08:38:46.655055+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":17,"log_dups_size":0,"ondisk_log_size":17,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:10:51.158061+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":8,"num_object_clones":0,"num_object_copies":24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":8,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":20,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":8,"num_bytes_recovered":112,"num_keys_recovered":0,"num_objects_omap":6,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,1],"acting":[8,0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.11","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751195+0000","last_change":"2024-04-21T08:36:20.909076+0000","last_active":"2024-04-21T08:38:46.751195+0000","last_peered":"2024-04-21T08:38:46.751195+0000","last_clean":"2024-04-21T08:38:46.751195+0000","last_became_active":"2024-04-21T08:36:20.908739+0000","last_became_peered":"2024-04-21T08:36:20.908739+0000","last_unstale":"2024-04-21T08:38:46.751195+0000","last_undegraded":"2024-04-21T08:38:46.751195+0000","last_fullsized":"2024-04-21T08:38:46.751195+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:03:56.870477+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,4,1],"acting":[5,4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"4.9","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751077+0000","last_change":"2024-04-21T08:36:24.926871+0000","last_active":"2024-04-21T08:38:46.751077+0000","last_peered":"2024-04-21T08:38:46.751077+0000","last_clean":"2024-04-21T08:38:46.751077+0000","last_became_active":"2024-04-21T08:36:24.926438+0000","last_became_peered":"2024-04-21T08:36:24.926438+0000","last_unstale":"2024-04-21T08:38:46.751077+0000","last_undegraded":"2024-04-21T08:38:46.751077+0000","last_fullsized":"2024-04-21T08:38:46.751077+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:24:48.984906+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,0,3,1],"acting":[5,0,3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.f","version":"94'3","reported_seq":24,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807724+0000","last_change":"2024-04-21T08:37:24.308038+0000","last_active":"2024-04-21T08:38:46.807724+0000","last_peered":"2024-04-21T08:38:46.807724+0000","last_clean":"2024-04-21T08:38:46.807724+0000","last_became_active":"2024-04-21T08:37:24.301034+0000","last_became_peered":"2024-04-21T08:37:24.301034+0000","last_unstale":"2024-04-21T08:38:46.807724+0000","last_undegraded":"2024-04-21T08:38:46.807724+0000","last_fullsized":"2024-04-21T08:38:46.807724+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:37:06.598400+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":4,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":22,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,4],"acting":[11,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.e","version":"91'5","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807693+0000","last_change":"2024-04-21T08:37:24.299260+0000","last_active":"2024-04-21T08:38:46.807693+0000","last_peered":"2024-04-21T08:38:46.807693+0000","last_clean":"2024-04-21T08:38:46.807693+0000","last_became_active":"2024-04-21T08:37:24.298831+0000","last_became_peered":"2024-04-21T08:37:24.298831+0000","last_unstale":"2024-04-21T08:38:46.807693+0000","last_undegraded":"2024-04-21T08:38:46.807693+0000","last_fullsized":"2024-04-21T08:38:46.807693+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:58:29.368027+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,9,4],"acting":[11,9,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.2","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655140+0000","last_change":"2024-04-21T08:36:24.929420+0000","last_active":"2024-04-21T08:38:46.655140+0000","last_peered":"2024-04-21T08:38:46.655140+0000","last_clean":"2024-04-21T08:38:46.655140+0000","last_became_active":"2024-04-21T08:36:24.928981+0000","last_became_peered":"2024-04-21T08:36:24.928981+0000","last_unstale":"2024-04-21T08:38:46.655140+0000","last_undegraded":"2024-04-21T08:38:46.655140+0000","last_fullsized":"2024-04-21T08:38:46.655140+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:38:00.040456+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,10,11,9],"acting":[8,10,11,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"2.4","version":"89'1","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.742890+0000","last_change":"2024-04-21T08:36:20.279910+0000","last_active":"2024-04-21T08:38:46.742890+0000","last_peered":"2024-04-21T08:38:46.742890+0000","last_clean":"2024-04-21T08:38:46.742890+0000","last_became_active":"2024-04-21T08:36:20.278904+0000","last_became_peered":"2024-04-21T08:36:20.278904+0000","last_unstale":"2024-04-21T08:38:46.742890+0000","last_undegraded":"2024-04-21T08:38:46.742890+0000","last_fullsized":"2024-04-21T08:38:46.742890+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:40:19.209213+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,2,0],"acting":[9,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.5","version":"91'4","reported_seq":35,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638524+0000","last_change":"2024-04-21T08:36:20.895816+0000","last_active":"2024-04-21T08:38:46.638524+0000","last_peered":"2024-04-21T08:38:46.638524+0000","last_clean":"2024-04-21T08:38:46.638524+0000","last_became_active":"2024-04-21T08:36:20.895487+0000","last_became_peered":"2024-04-21T08:36:20.895487+0000","last_unstale":"2024-04-21T08:38:46.638524+0000","last_undegraded":"2024-04-21T08:38:46.638524+0000","last_fullsized":"2024-04-21T08:38:46.638524+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:53:52.153683+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,3,6],"acting":[10,3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.4","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.912486+0000","last_change":"2024-04-21T08:36:24.917637+0000","last_active":"2024-04-21T08:38:46.912486+0000","last_peered":"2024-04-21T08:38:46.912486+0000","last_clean":"2024-04-21T08:38:46.912486+0000","last_became_active":"2024-04-21T08:36:24.917192+0000","last_became_peered":"2024-04-21T08:36:24.917192+0000","last_unstale":"2024-04-21T08:38:46.912486+0000","last_undegraded":"2024-04-21T08:38:46.912486+0000","last_fullsized":"2024-04-21T08:38:46.912486+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:02:59.843903+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,0,1,8],"acting":[6,0,1,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.2","version":"89'5","reported_seq":164,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:51.411645+0000","last_change":"2024-04-21T08:36:20.286424+0000","last_active":"2024-04-21T08:38:51.411645+0000","last_peered":"2024-04-21T08:38:51.411645+0000","last_clean":"2024-04-21T08:38:51.411645+0000","last_became_active":"2024-04-21T08:36:20.285767+0000","last_became_peered":"2024-04-21T08:36:20.285767+0000","last_unstale":"2024-04-21T08:38:51.411645+0000","last_undegraded":"2024-04-21T08:38:51.411645+0000","last_fullsized":"2024-04-21T08:38:51.411645+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:53:33.202734+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":588,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":126,"num_read_kb":1,"num_write":4,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,6,1],"acting":[7,6,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.3","version":"91'7","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638138+0000","last_change":"2024-04-21T08:37:24.297650+0000","last_active":"2024-04-21T08:38:46.638138+0000","last_peered":"2024-04-21T08:38:46.638138+0000","last_clean":"2024-04-21T08:38:46.638138+0000","last_became_active":"2024-04-21T08:37:24.297250+0000","last_became_peered":"2024-04-21T08:37:24.297250+0000","last_unstale":"2024-04-21T08:38:46.638138+0000","last_undegraded":"2024-04-21T08:38:46.638138+0000","last_fullsized":"2024-04-21T08:38:46.638138+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:21:21.419556+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[10,9,0],"acting":[10,9,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":10,"acting_primary":10,"purged_snaps":[]},{"pgid":"4.7","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911806+0000","last_change":"2024-04-21T08:36:24.919669+0000","last_active":"2024-04-21T08:38:46.911806+0000","last_peered":"2024-04-21T08:38:46.911806+0000","last_clean":"2024-04-21T08:38:46.911806+0000","last_became_active":"2024-04-21T08:36:24.919281+0000","last_became_peered":"2024-04-21T08:36:24.919281+0000","last_unstale":"2024-04-21T08:38:46.911806+0000","last_undegraded":"2024-04-21T08:38:46.911806+0000","last_fullsized":"2024-04-21T08:38:46.911806+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:44:27.480367+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,5,10,0],"acting":[6,5,10,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.0","version":"91'8","reported_seq":45,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751175+0000","last_change":"2024-04-21T08:37:24.297826+0000","last_active":"2024-04-21T08:38:46.751175+0000","last_peered":"2024-04-21T08:38:46.751175+0000","last_clean":"2024-04-21T08:38:46.751175+0000","last_became_active":"2024-04-21T08:37:24.297397+0000","last_became_peered":"2024-04-21T08:37:24.297397+0000","last_unstale":"2024-04-21T08:38:46.751175+0000","last_undegraded":"2024-04-21T08:38:46.751175+0000","last_fullsized":"2024-04-21T08:38:46.751175+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:52:17.693725+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,8],"acting":[5,2,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.1","version":"94'7","reported_seq":170,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:51.640626+0000","last_change":"2024-04-21T08:36:20.291033+0000","last_active":"2024-04-21T08:38:51.640626+0000","last_peered":"2024-04-21T08:38:51.640626+0000","last_clean":"2024-04-21T08:38:51.640626+0000","last_became_active":"2024-04-21T08:36:20.290697+0000","last_became_peered":"2024-04-21T08:36:20.290697+0000","last_unstale":"2024-04-21T08:38:51.640626+0000","last_undegraded":"2024-04-21T08:38:51.640626+0000","last_fullsized":"2024-04-21T08:38:51.640626+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:22:54.593595+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":129,"num_read_kb":129,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,0],"acting":[6,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.5","version":"89'1","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913716+0000","last_change":"2024-04-21T08:37:24.310936+0000","last_active":"2024-04-21T08:38:46.913716+0000","last_peered":"2024-04-21T08:38:46.913716+0000","last_clean":"2024-04-21T08:38:46.913716+0000","last_became_active":"2024-04-21T08:37:24.297349+0000","last_became_peered":"2024-04-21T08:37:24.297349+0000","last_unstale":"2024-04-21T08:38:46.913716+0000","last_undegraded":"2024-04-21T08:38:46.913716+0000","last_fullsized":"2024-04-21T08:38:46.913716+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:31:39.884609+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,11,10],"acting":[7,5,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.2","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638671+0000","last_change":"2024-04-21T08:36:20.904197+0000","last_active":"2024-04-21T08:38:46.638671+0000","last_peered":"2024-04-21T08:38:46.638671+0000","last_clean":"2024-04-21T08:38:46.638671+0000","last_became_active":"2024-04-21T08:36:20.903099+0000","last_became_peered":"2024-04-21T08:36:20.903099+0000","last_unstale":"2024-04-21T08:38:46.638671+0000","last_undegraded":"2024-04-21T08:38:46.638671+0000","last_fullsized":"2024-04-21T08:38:46.638671+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:16:08.914335+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,5,1],"acting":[3,5,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.3","version":"89'6","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911254+0000","last_change":"2024-04-21T08:36:20.280608+0000","last_active":"2024-04-21T08:38:46.911254+0000","last_peered":"2024-04-21T08:38:46.911254+0000","last_clean":"2024-04-21T08:38:46.911254+0000","last_became_active":"2024-04-21T08:36:20.280172+0000","last_became_peered":"2024-04-21T08:36:20.280172+0000","last_unstale":"2024-04-21T08:38:46.911254+0000","last_undegraded":"2024-04-21T08:38:46.911254+0000","last_fullsized":"2024-04-21T08:38:46.911254+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:51:12.284463+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":68,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":4,"num_read_kb":4,"num_write":8,"num_write_kb":8,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,10,9],"acting":[4,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"1.0","version":"94'148","reported_seq":410,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:48.757849+0000","last_change":"2024-04-21T08:35:08.423545+0000","last_active":"2024-04-21T08:38:48.757849+0000","last_peered":"2024-04-21T08:38:48.757849+0000","last_clean":"2024-04-21T08:38:48.757849+0000","last_became_active":"2024-04-21T08:35:08.423027+0000","last_became_peered":"2024-04-21T08:35:08.423027+0000","last_unstale":"2024-04-21T08:38:48.757849+0000","last_undegraded":"2024-04-21T08:38:48.757849+0000","last_fullsized":"2024-04-21T08:38:48.757849+0000","mapping_epoch":75,"log_start":"0'0","ondisk_log_start":"0'0","created":19,"last_epoch_clean":76,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:31:45.359253+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:31:45.359253+0000","last_clean_scrub_stamp":"2024-04-21T08:31:45.359253+0000","objects_scrubbed":0,"log_size":148,"log_dups_size":0,"ondisk_log_size":148,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:53:24.310824+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,10],"acting":[7,5,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.6","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751452+0000","last_change":"2024-04-21T08:36:24.922422+0000","last_active":"2024-04-21T08:38:46.751452+0000","last_peered":"2024-04-21T08:38:46.751452+0000","last_clean":"2024-04-21T08:38:46.751452+0000","last_became_active":"2024-04-21T08:36:24.922006+0000","last_became_peered":"2024-04-21T08:36:24.922006+0000","last_unstale":"2024-04-21T08:38:46.751452+0000","last_undegraded":"2024-04-21T08:38:46.751452+0000","last_fullsized":"2024-04-21T08:38:46.751452+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:46:22.840802+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,6,10],"acting":[5,9,6,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.1","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911805+0000","last_change":"2024-04-21T08:36:20.911222+0000","last_active":"2024-04-21T08:38:46.911805+0000","last_peered":"2024-04-21T08:38:46.911805+0000","last_clean":"2024-04-21T08:38:46.911805+0000","last_became_active":"2024-04-21T08:36:20.910933+0000","last_became_peered":"2024-04-21T08:36:20.910933+0000","last_unstale":"2024-04-21T08:38:46.911805+0000","last_undegraded":"2024-04-21T08:38:46.911805+0000","last_fullsized":"2024-04-21T08:38:46.911805+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:41:25.621266+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,2],"acting":[6,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"2.0","version":"89'4","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913633+0000","last_change":"2024-04-21T08:36:20.288409+0000","last_active":"2024-04-21T08:38:46.913633+0000","last_peered":"2024-04-21T08:38:46.913633+0000","last_clean":"2024-04-21T08:38:46.913633+0000","last_became_active":"2024-04-21T08:36:20.287916+0000","last_became_peered":"2024-04-21T08:36:20.287916+0000","last_unstale":"2024-04-21T08:38:46.913633+0000","last_undegraded":"2024-04-21T08:38:46.913633+0000","last_fullsized":"2024-04-21T08:38:46.913633+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:04:31.068352+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,5,8],"acting":[7,5,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.3","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807586+0000","last_change":"2024-04-21T08:36:24.918166+0000","last_active":"2024-04-21T08:38:46.807586+0000","last_peered":"2024-04-21T08:38:46.807586+0000","last_clean":"2024-04-21T08:38:46.807586+0000","last_became_active":"2024-04-21T08:36:24.917745+0000","last_became_peered":"2024-04-21T08:36:24.917745+0000","last_unstale":"2024-04-21T08:38:46.807586+0000","last_undegraded":"2024-04-21T08:38:46.807586+0000","last_fullsized":"2024-04-21T08:38:46.807586+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:55:41.169681+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,5,7,6],"acting":[11,5,7,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"2.5","version":"89'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655417+0000","last_change":"2024-04-21T08:36:20.291079+0000","last_active":"2024-04-21T08:38:46.655417+0000","last_peered":"2024-04-21T08:38:46.655417+0000","last_clean":"2024-04-21T08:38:46.655417+0000","last_became_active":"2024-04-21T08:36:20.290720+0000","last_became_peered":"2024-04-21T08:36:20.290720+0000","last_unstale":"2024-04-21T08:38:46.655417+0000","last_undegraded":"2024-04-21T08:38:46.655417+0000","last_fullsized":"2024-04-21T08:38:46.655417+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:16:41.269029+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":112,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":5,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,0,9],"acting":[8,0,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.4","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.912713+0000","last_change":"2024-04-21T08:36:20.907416+0000","last_active":"2024-04-21T08:38:46.912713+0000","last_peered":"2024-04-21T08:38:46.912713+0000","last_clean":"2024-04-21T08:38:46.912713+0000","last_became_active":"2024-04-21T08:36:20.907005+0000","last_became_peered":"2024-04-21T08:36:20.907005+0000","last_unstale":"2024-04-21T08:38:46.912713+0000","last_undegraded":"2024-04-21T08:38:46.912713+0000","last_fullsized":"2024-04-21T08:38:46.912713+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:58:17.979947+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,2,9],"acting":[6,2,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"4.0","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638481+0000","last_change":"2024-04-21T08:36:24.921475+0000","last_active":"2024-04-21T08:38:46.638481+0000","last_peered":"2024-04-21T08:38:46.638481+0000","last_clean":"2024-04-21T08:38:46.638481+0000","last_became_active":"2024-04-21T08:36:24.921091+0000","last_became_peered":"2024-04-21T08:36:24.921091+0000","last_unstale":"2024-04-21T08:38:46.638481+0000","last_undegraded":"2024-04-21T08:38:46.638481+0000","last_fullsized":"2024-04-21T08:38:46.638481+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:28:14.224501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,10,0,7],"acting":[3,10,0,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.6","version":"94'48","reported_seq":105,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.860978+0000","last_change":"2024-04-21T08:36:20.288916+0000","last_active":"2024-04-21T08:38:46.860978+0000","last_peered":"2024-04-21T08:38:46.860978+0000","last_clean":"2024-04-21T08:38:46.860978+0000","last_became_active":"2024-04-21T08:36:20.288575+0000","last_became_peered":"2024-04-21T08:36:20.288575+0000","last_unstale":"2024-04-21T08:38:46.860978+0000","last_undegraded":"2024-04-21T08:38:46.860978+0000","last_fullsized":"2024-04-21T08:38:46.860978+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":48,"log_dups_size":0,"ondisk_log_size":48,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:52:05.795554+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":393216,"num_objects":5,"num_object_clones":0,"num_object_copies":15,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":5,"num_whiteouts":0,"num_read":82,"num_read_kb":70,"num_write":84,"num_write_kb":836,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":4,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,8,3],"acting":[9,8,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"3.7","version":"91'5","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639071+0000","last_change":"2024-04-21T08:36:20.912053+0000","last_active":"2024-04-21T08:38:46.639071+0000","last_peered":"2024-04-21T08:38:46.639071+0000","last_clean":"2024-04-21T08:38:46.639071+0000","last_became_active":"2024-04-21T08:36:20.911584+0000","last_became_peered":"2024-04-21T08:36:20.911584+0000","last_unstale":"2024-04-21T08:38:46.639071+0000","last_undegraded":"2024-04-21T08:38:46.639071+0000","last_fullsized":"2024-04-21T08:38:46.639071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:48:36.369954+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,11,7],"acting":[3,11,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.1","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638589+0000","last_change":"2024-04-21T08:36:24.919212+0000","last_active":"2024-04-21T08:38:46.638589+0000","last_peered":"2024-04-21T08:38:46.638589+0000","last_clean":"2024-04-21T08:38:46.638589+0000","last_became_active":"2024-04-21T08:36:24.918807+0000","last_became_peered":"2024-04-21T08:36:24.918807+0000","last_unstale":"2024-04-21T08:38:46.638589+0000","last_undegraded":"2024-04-21T08:38:46.638589+0000","last_fullsized":"2024-04-21T08:38:46.638589+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:25:50.816775+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1,4,7],"acting":[2,1,4,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.7","version":"89'7","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911766+0000","last_change":"2024-04-21T08:36:20.282536+0000","last_active":"2024-04-21T08:38:46.911766+0000","last_peered":"2024-04-21T08:38:46.911766+0000","last_clean":"2024-04-21T08:38:46.911766+0000","last_became_active":"2024-04-21T08:36:20.282172+0000","last_became_peered":"2024-04-21T08:36:20.282172+0000","last_unstale":"2024-04-21T08:38:46.911766+0000","last_undegraded":"2024-04-21T08:38:46.911766+0000","last_fullsized":"2024-04-21T08:38:46.911766+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:15:38.992441+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":22,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":1,"num_read_kb":1,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,11,6],"acting":[4,11,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"3.6","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.915413+0000","last_change":"2024-04-21T08:36:20.908416+0000","last_active":"2024-04-21T08:38:46.915413+0000","last_peered":"2024-04-21T08:38:46.915413+0000","last_clean":"2024-04-21T08:38:46.915413+0000","last_became_active":"2024-04-21T08:36:20.908054+0000","last_became_peered":"2024-04-21T08:36:20.908054+0000","last_unstale":"2024-04-21T08:38:46.915413+0000","last_undegraded":"2024-04-21T08:38:46.915413+0000","last_fullsized":"2024-04-21T08:38:46.915413+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:23:16.562060+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3,4],"acting":[0,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"4.e","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911638+0000","last_change":"2024-04-21T08:36:24.920637+0000","last_active":"2024-04-21T08:38:46.911638+0000","last_peered":"2024-04-21T08:38:46.911638+0000","last_clean":"2024-04-21T08:38:46.911638+0000","last_became_active":"2024-04-21T08:36:24.919921+0000","last_became_peered":"2024-04-21T08:36:24.919921+0000","last_unstale":"2024-04-21T08:38:46.911638+0000","last_undegraded":"2024-04-21T08:38:46.911638+0000","last_fullsized":"2024-04-21T08:38:46.911638+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T09:20:38.230087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,5,10,9],"acting":[4,5,10,9],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.8","version":"89'2","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.922412+0000","last_change":"2024-04-21T08:36:20.279332+0000","last_active":"2024-04-21T08:38:46.922412+0000","last_peered":"2024-04-21T08:38:46.922412+0000","last_clean":"2024-04-21T08:38:46.922412+0000","last_became_active":"2024-04-21T08:36:20.278970+0000","last_became_peered":"2024-04-21T08:36:20.278970+0000","last_unstale":"2024-04-21T08:38:46.922412+0000","last_undegraded":"2024-04-21T08:38:46.922412+0000","last_fullsized":"2024-04-21T08:38:46.922412+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:36:16.041610+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":3,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":3,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2,0],"acting":[1,2,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.9","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.654968+0000","last_change":"2024-04-21T08:36:20.919621+0000","last_active":"2024-04-21T08:38:46.654968+0000","last_peered":"2024-04-21T08:38:46.654968+0000","last_clean":"2024-04-21T08:38:46.654968+0000","last_became_active":"2024-04-21T08:36:20.919246+0000","last_became_peered":"2024-04-21T08:36:20.919246+0000","last_unstale":"2024-04-21T08:38:46.654968+0000","last_undegraded":"2024-04-21T08:38:46.654968+0000","last_fullsized":"2024-04-21T08:38:46.654968+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:31:59.001501+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,11,10],"acting":[8,11,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"4.f","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.743258+0000","last_change":"2024-04-21T08:36:24.920459+0000","last_active":"2024-04-21T08:38:46.743258+0000","last_peered":"2024-04-21T08:38:46.743258+0000","last_clean":"2024-04-21T08:38:46.743258+0000","last_became_active":"2024-04-21T08:36:24.919691+0000","last_became_peered":"2024-04-21T08:36:24.919691+0000","last_unstale":"2024-04-21T08:38:46.743258+0000","last_undegraded":"2024-04-21T08:38:46.743258+0000","last_fullsized":"2024-04-21T08:38:46.743258+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T16:38:26.655452+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[9,3,2,5],"acting":[9,3,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":9,"acting_primary":9,"purged_snaps":[]},{"pgid":"2.9","version":"89'7","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638571+0000","last_change":"2024-04-21T08:36:20.284286+0000","last_active":"2024-04-21T08:38:46.638571+0000","last_peered":"2024-04-21T08:38:46.638571+0000","last_clean":"2024-04-21T08:38:46.638571+0000","last_became_active":"2024-04-21T08:36:20.283903+0000","last_became_peered":"2024-04-21T08:36:20.283903+0000","last_unstale":"2024-04-21T08:38:46.638571+0000","last_undegraded":"2024-04-21T08:38:46.638571+0000","last_fullsized":"2024-04-21T08:38:46.638571+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:51:52.409988+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":90,"num_objects":4,"num_object_clones":0,"num_object_copies":12,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":4,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":10,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":3,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,9,10],"acting":[3,9,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.8","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807332+0000","last_change":"2024-04-21T08:36:20.911892+0000","last_active":"2024-04-21T08:38:46.807332+0000","last_peered":"2024-04-21T08:38:46.807332+0000","last_clean":"2024-04-21T08:38:46.807332+0000","last_became_active":"2024-04-21T08:36:20.911513+0000","last_became_peered":"2024-04-21T08:36:20.911513+0000","last_unstale":"2024-04-21T08:38:46.807332+0000","last_undegraded":"2024-04-21T08:38:46.807332+0000","last_fullsized":"2024-04-21T08:38:46.807332+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:51:13.733534+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,0,2],"acting":[11,0,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"4.c","version":"89'1","reported_seq":26,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911539+0000","last_change":"2024-04-21T08:36:24.920347+0000","last_active":"2024-04-21T08:38:46.911539+0000","last_peered":"2024-04-21T08:38:46.911539+0000","last_clean":"2024-04-21T08:38:46.911539+0000","last_became_active":"2024-04-21T08:36:24.919831+0000","last_became_peered":"2024-04-21T08:36:24.919831+0000","last_unstale":"2024-04-21T08:38:46.911539+0000","last_undegraded":"2024-04-21T08:38:46.911539+0000","last_fullsized":"2024-04-21T08:38:46.911539+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:14:27.020476+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":117,"num_objects":1,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":1,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":1,"num_write_kb":1,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,9,7,8],"acting":[4,9,7,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.a","version":"89'3","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638672+0000","last_change":"2024-04-21T08:36:20.286786+0000","last_active":"2024-04-21T08:38:46.638672+0000","last_peered":"2024-04-21T08:38:46.638672+0000","last_clean":"2024-04-21T08:38:46.638672+0000","last_became_active":"2024-04-21T08:36:20.286429+0000","last_became_peered":"2024-04-21T08:36:20.286429+0000","last_unstale":"2024-04-21T08:38:46.638672+0000","last_undegraded":"2024-04-21T08:38:46.638672+0000","last_fullsized":"2024-04-21T08:38:46.638672+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":3,"log_dups_size":0,"ondisk_log_size":3,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:40:57.093370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,8],"acting":[2,3,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.b","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638983+0000","last_change":"2024-04-21T08:36:20.904767+0000","last_active":"2024-04-21T08:38:46.638983+0000","last_peered":"2024-04-21T08:38:46.638983+0000","last_clean":"2024-04-21T08:38:46.638983+0000","last_became_active":"2024-04-21T08:36:20.904406+0000","last_became_peered":"2024-04-21T08:36:20.904406+0000","last_unstale":"2024-04-21T08:38:46.638983+0000","last_undegraded":"2024-04-21T08:38:46.638983+0000","last_fullsized":"2024-04-21T08:38:46.638983+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:11:41.644549+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0,10],"acting":[3,0,10],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"4.d","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.911769+0000","last_change":"2024-04-21T08:36:24.918031+0000","last_active":"2024-04-21T08:38:46.911769+0000","last_peered":"2024-04-21T08:38:46.911769+0000","last_clean":"2024-04-21T08:38:46.911769+0000","last_became_active":"2024-04-21T08:36:24.917618+0000","last_became_peered":"2024-04-21T08:36:24.917618+0000","last_unstale":"2024-04-21T08:38:46.911769+0000","last_undegraded":"2024-04-21T08:38:46.911769+0000","last_fullsized":"2024-04-21T08:38:46.911769+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:50:35.797911+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,2,10,3],"acting":[4,2,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]},{"pgid":"2.b","version":"89'4","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.655179+0000","last_change":"2024-04-21T08:36:20.286212+0000","last_active":"2024-04-21T08:38:46.655179+0000","last_peered":"2024-04-21T08:38:46.655179+0000","last_clean":"2024-04-21T08:38:46.655179+0000","last_became_active":"2024-04-21T08:36:20.285878+0000","last_became_peered":"2024-04-21T08:36:20.285878+0000","last_unstale":"2024-04-21T08:38:46.655179+0000","last_undegraded":"2024-04-21T08:38:46.655179+0000","last_fullsized":"2024-04-21T08:38:46.655179+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T10:21:19.375455+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":2,"num_read_kb":2,"num_write":4,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[8,2,7],"acting":[8,2,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":8,"acting_primary":8,"purged_snaps":[]},{"pgid":"3.a","version":"91'5","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914242+0000","last_change":"2024-04-21T08:36:20.908335+0000","last_active":"2024-04-21T08:38:46.914242+0000","last_peered":"2024-04-21T08:38:46.914242+0000","last_clean":"2024-04-21T08:38:46.914242+0000","last_became_active":"2024-04-21T08:36:20.907945+0000","last_became_peered":"2024-04-21T08:36:20.907945+0000","last_unstale":"2024-04-21T08:38:46.914242+0000","last_undegraded":"2024-04-21T08:38:46.914242+0000","last_fullsized":"2024-04-21T08:38:46.914242+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T13:47:17.841511+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,6],"acting":[7,1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.a","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.922383+0000","last_change":"2024-04-21T08:36:24.915553+0000","last_active":"2024-04-21T08:38:46.922383+0000","last_peered":"2024-04-21T08:38:46.922383+0000","last_clean":"2024-04-21T08:38:46.922383+0000","last_became_active":"2024-04-21T08:36:24.915028+0000","last_became_peered":"2024-04-21T08:36:24.915028+0000","last_unstale":"2024-04-21T08:38:46.922383+0000","last_undegraded":"2024-04-21T08:38:46.922383+0000","last_fullsized":"2024-04-21T08:38:46.922383+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:17:21.192946+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5,6,7],"acting":[1,5,6,7],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.c","version":"91'7","reported_seq":169,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:50.660424+0000","last_change":"2024-04-21T08:36:20.279649+0000","last_active":"2024-04-21T08:38:50.660424+0000","last_peered":"2024-04-21T08:38:50.660424+0000","last_clean":"2024-04-21T08:38:50.660424+0000","last_became_active":"2024-04-21T08:36:20.279111+0000","last_became_peered":"2024-04-21T08:36:20.279111+0000","last_unstale":"2024-04-21T08:38:50.660424+0000","last_undegraded":"2024-04-21T08:38:50.660424+0000","last_fullsized":"2024-04-21T08:38:50.660424+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:19:57.380495+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":1702,"num_objects":3,"num_object_clones":0,"num_object_copies":9,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":3,"num_whiteouts":0,"num_read":128,"num_read_kb":3,"num_write":7,"num_write_kb":4,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":1,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,11,0],"acting":[2,11,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.d","version":"91'7","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914334+0000","last_change":"2024-04-21T08:36:20.911489+0000","last_active":"2024-04-21T08:38:46.914334+0000","last_peered":"2024-04-21T08:38:46.914334+0000","last_clean":"2024-04-21T08:38:46.914334+0000","last_became_active":"2024-04-21T08:36:20.911168+0000","last_became_peered":"2024-04-21T08:36:20.911168+0000","last_unstale":"2024-04-21T08:38:46.914334+0000","last_undegraded":"2024-04-21T08:38:46.914334+0000","last_fullsized":"2024-04-21T08:38:46.914334+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:40:41.585841+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,1,11],"acting":[7,1,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.b","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914668+0000","last_change":"2024-04-21T08:36:24.921175+0000","last_active":"2024-04-21T08:38:46.914668+0000","last_peered":"2024-04-21T08:38:46.914668+0000","last_clean":"2024-04-21T08:38:46.914668+0000","last_became_active":"2024-04-21T08:36:24.920802+0000","last_became_peered":"2024-04-21T08:36:24.920802+0000","last_unstale":"2024-04-21T08:38:46.914668+0000","last_undegraded":"2024-04-21T08:38:46.914668+0000","last_fullsized":"2024-04-21T08:38:46.914668+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:43:12.140016+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,9,1,4],"acting":[0,9,1,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.d","version":"94'41","reported_seq":82,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.859069+0000","last_change":"2024-04-21T08:36:20.290202+0000","last_active":"2024-04-21T08:38:46.859069+0000","last_peered":"2024-04-21T08:38:46.859069+0000","last_clean":"2024-04-21T08:38:46.859069+0000","last_became_active":"2024-04-21T08:36:20.289868+0000","last_became_peered":"2024-04-21T08:36:20.289868+0000","last_unstale":"2024-04-21T08:38:46.859069+0000","last_undegraded":"2024-04-21T08:38:46.859069+0000","last_fullsized":"2024-04-21T08:38:46.859069+0000","mapping_epoch":82,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":83,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":41,"log_dups_size":0,"ondisk_log_size":41,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:07:45.525980+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":131670,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":11,"num_read_kb":10,"num_write":49,"num_write_kb":475,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,10,3],"acting":[5,10,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.c","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913789+0000","last_change":"2024-04-21T08:36:20.899590+0000","last_active":"2024-04-21T08:38:46.913789+0000","last_peered":"2024-04-21T08:38:46.913789+0000","last_clean":"2024-04-21T08:38:46.913789+0000","last_became_active":"2024-04-21T08:36:20.899254+0000","last_became_peered":"2024-04-21T08:36:20.899254+0000","last_unstale":"2024-04-21T08:38:46.913789+0000","last_undegraded":"2024-04-21T08:38:46.913789+0000","last_fullsized":"2024-04-21T08:38:46.913789+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:34:25.409694+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,3,5],"acting":[7,3,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"4.8","version":"0'0","reported_seq":25,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.750892+0000","last_change":"2024-04-21T08:36:24.918310+0000","last_active":"2024-04-21T08:38:46.750892+0000","last_peered":"2024-04-21T08:38:46.750892+0000","last_clean":"2024-04-21T08:38:46.750892+0000","last_became_active":"2024-04-21T08:36:24.917899+0000","last_became_peered":"2024-04-21T08:36:24.917899+0000","last_unstale":"2024-04-21T08:38:46.750892+0000","last_undegraded":"2024-04-21T08:38:46.750892+0000","last_fullsized":"2024-04-21T08:38:46.750892+0000","mapping_epoch":87,"log_start":"0'0","ondisk_log_start":"0'0","created":87,"last_epoch_clean":88,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:23.895117+0000","last_clean_scrub_stamp":"2024-04-21T08:36:23.895117+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:09:13.006884+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,2,7,1],"acting":[5,2,7,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"2.e","version":"89'5","reported_seq":51,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913309+0000","last_change":"2024-04-21T08:37:24.304544+0000","last_active":"2024-04-21T08:38:46.913309+0000","last_peered":"2024-04-21T08:38:46.913309+0000","last_clean":"2024-04-21T08:38:46.913309+0000","last_became_active":"2024-04-21T08:37:24.295808+0000","last_became_peered":"2024-04-21T08:37:24.295808+0000","last_unstale":"2024-04-21T08:38:46.913309+0000","last_undegraded":"2024-04-21T08:38:46.913309+0000","last_fullsized":"2024-04-21T08:38:46.913309+0000","mapping_epoch":90,"log_start":"0'0","ondisk_log_start":"0'0","created":82,"last_epoch_clean":91,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:19.258930+0000","last_clean_scrub_stamp":"2024-04-21T08:36:19.258930+0000","objects_scrubbed":0,"log_size":5,"log_dups_size":0,"ondisk_log_size":5,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:07:15.797133+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":6,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":2,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":2,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,3,4],"acting":[6,3,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.f","version":"91'9","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.914356+0000","last_change":"2024-04-21T08:36:20.912368+0000","last_active":"2024-04-21T08:38:46.914356+0000","last_peered":"2024-04-21T08:38:46.914356+0000","last_clean":"2024-04-21T08:38:46.914356+0000","last_became_active":"2024-04-21T08:36:20.912009+0000","last_became_peered":"2024-04-21T08:36:20.912009+0000","last_unstale":"2024-04-21T08:38:46.914356+0000","last_undegraded":"2024-04-21T08:38:46.914356+0000","last_fullsized":"2024-04-21T08:38:46.914356+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:35:40.700314+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,11,8],"acting":[7,11,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"3.20","version":"91'9","reported_seq":41,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807399+0000","last_change":"2024-04-21T08:36:20.913472+0000","last_active":"2024-04-21T08:38:46.807399+0000","last_peered":"2024-04-21T08:38:46.807399+0000","last_clean":"2024-04-21T08:38:46.807399+0000","last_became_active":"2024-04-21T08:36:20.913118+0000","last_became_peered":"2024-04-21T08:36:20.913118+0000","last_unstale":"2024-04-21T08:38:46.807399+0000","last_undegraded":"2024-04-21T08:38:46.807399+0000","last_fullsized":"2024-04-21T08:38:46.807399+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":9,"log_dups_size":0,"ondisk_log_size":9,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:31:53.957044+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,2,5],"acting":[11,2,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.21","version":"91'12","reported_seq":44,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638786+0000","last_change":"2024-04-21T08:36:20.913677+0000","last_active":"2024-04-21T08:38:46.638786+0000","last_peered":"2024-04-21T08:38:46.638786+0000","last_clean":"2024-04-21T08:38:46.638786+0000","last_became_active":"2024-04-21T08:36:20.913097+0000","last_became_peered":"2024-04-21T08:36:20.913097+0000","last_unstale":"2024-04-21T08:38:46.638786+0000","last_undegraded":"2024-04-21T08:38:46.638786+0000","last_fullsized":"2024-04-21T08:38:46.638786+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":12,"log_dups_size":0,"ondisk_log_size":12,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:08:40.388370+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,5,4],"acting":[2,5,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.22","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638662+0000","last_change":"2024-04-21T08:36:20.906624+0000","last_active":"2024-04-21T08:38:46.638662+0000","last_peered":"2024-04-21T08:38:46.638662+0000","last_clean":"2024-04-21T08:38:46.638662+0000","last_became_active":"2024-04-21T08:36:20.906247+0000","last_became_peered":"2024-04-21T08:36:20.906247+0000","last_unstale":"2024-04-21T08:38:46.638662+0000","last_undegraded":"2024-04-21T08:38:46.638662+0000","last_fullsized":"2024-04-21T08:38:46.638662+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:44:11.855183+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,3,0],"acting":[2,3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.23","version":"91'7","reported_seq":39,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.916219+0000","last_change":"2024-04-21T08:36:20.906576+0000","last_active":"2024-04-21T08:38:46.916219+0000","last_peered":"2024-04-21T08:38:46.916219+0000","last_clean":"2024-04-21T08:38:46.916219+0000","last_became_active":"2024-04-21T08:36:20.906231+0000","last_became_peered":"2024-04-21T08:36:20.906231+0000","last_unstale":"2024-04-21T08:38:46.916219+0000","last_undegraded":"2024-04-21T08:38:46.916219+0000","last_fullsized":"2024-04-21T08:38:46.916219+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":7,"log_dups_size":0,"ondisk_log_size":7,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T18:42:48.156614+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,6,4],"acting":[0,6,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"3.24","version":"91'4","reported_seq":36,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.912593+0000","last_change":"2024-04-21T08:36:20.902551+0000","last_active":"2024-04-21T08:38:46.912593+0000","last_peered":"2024-04-21T08:38:46.912593+0000","last_clean":"2024-04-21T08:38:46.912593+0000","last_became_active":"2024-04-21T08:36:20.902208+0000","last_became_peered":"2024-04-21T08:36:20.902208+0000","last_unstale":"2024-04-21T08:38:46.912593+0000","last_undegraded":"2024-04-21T08:38:46.912593+0000","last_fullsized":"2024-04-21T08:38:46.912593+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":4,"log_dups_size":0,"ondisk_log_size":4,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T08:36:30.016256+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,4,0],"acting":[6,4,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.25","version":"91'6","reported_seq":38,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807607+0000","last_change":"2024-04-21T08:36:20.918588+0000","last_active":"2024-04-21T08:38:46.807607+0000","last_peered":"2024-04-21T08:38:46.807607+0000","last_clean":"2024-04-21T08:38:46.807607+0000","last_became_active":"2024-04-21T08:36:20.918263+0000","last_became_peered":"2024-04-21T08:36:20.918263+0000","last_unstale":"2024-04-21T08:38:46.807607+0000","last_undegraded":"2024-04-21T08:38:46.807607+0000","last_fullsized":"2024-04-21T08:38:46.807607+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:51:30.340336+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,8,5],"acting":[11,8,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]},{"pgid":"3.26","version":"91'15","reported_seq":47,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639167+0000","last_change":"2024-04-21T08:36:20.911063+0000","last_active":"2024-04-21T08:38:46.639167+0000","last_peered":"2024-04-21T08:38:46.639167+0000","last_clean":"2024-04-21T08:38:46.639167+0000","last_became_active":"2024-04-21T08:36:20.910647+0000","last_became_peered":"2024-04-21T08:36:20.910647+0000","last_unstale":"2024-04-21T08:38:46.639167+0000","last_undegraded":"2024-04-21T08:38:46.639167+0000","last_fullsized":"2024-04-21T08:38:46.639167+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":15,"log_dups_size":0,"ondisk_log_size":15,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T19:10:31.079782+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,8,1],"acting":[2,8,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"3.27","version":"91'2","reported_seq":34,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.639213+0000","last_change":"2024-04-21T08:36:20.914079+0000","last_active":"2024-04-21T08:38:46.639213+0000","last_peered":"2024-04-21T08:38:46.639213+0000","last_clean":"2024-04-21T08:38:46.639213+0000","last_became_active":"2024-04-21T08:36:20.912851+0000","last_became_peered":"2024-04-21T08:36:20.912851+0000","last_unstale":"2024-04-21T08:38:46.639213+0000","last_undegraded":"2024-04-21T08:38:46.639213+0000","last_fullsized":"2024-04-21T08:38:46.639213+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T11:53:35.811281+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,4,2],"acting":[3,4,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.28","version":"91'6","reported_seq":37,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.751071+0000","last_change":"2024-04-21T08:36:20.900551+0000","last_active":"2024-04-21T08:38:46.751071+0000","last_peered":"2024-04-21T08:38:46.751071+0000","last_clean":"2024-04-21T08:38:46.751071+0000","last_became_active":"2024-04-21T08:36:20.900173+0000","last_became_peered":"2024-04-21T08:36:20.900173+0000","last_unstale":"2024-04-21T08:38:46.751071+0000","last_undegraded":"2024-04-21T08:38:46.751071+0000","last_fullsized":"2024-04-21T08:38:46.751071+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":6,"log_dups_size":0,"ondisk_log_size":6,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T12:31:21.389498+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[5,9,3],"acting":[5,9,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":5,"acting_primary":5,"purged_snaps":[]},{"pgid":"3.29","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.638564+0000","last_change":"2024-04-21T08:36:20.912953+0000","last_active":"2024-04-21T08:38:46.638564+0000","last_peered":"2024-04-21T08:38:46.638564+0000","last_clean":"2024-04-21T08:38:46.638564+0000","last_became_active":"2024-04-21T08:36:20.912223+0000","last_became_peered":"2024-04-21T08:36:20.912223+0000","last_unstale":"2024-04-21T08:38:46.638564+0000","last_undegraded":"2024-04-21T08:38:46.638564+0000","last_fullsized":"2024-04-21T08:38:46.638564+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T14:43:14.277179+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,2,11],"acting":[3,2,11],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"3.2a","version":"91'10","reported_seq":42,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.922633+0000","last_change":"2024-04-21T08:36:20.906164+0000","last_active":"2024-04-21T08:38:46.922633+0000","last_peered":"2024-04-21T08:38:46.922633+0000","last_clean":"2024-04-21T08:38:46.922633+0000","last_became_active":"2024-04-21T08:36:20.905749+0000","last_became_peered":"2024-04-21T08:36:20.905749+0000","last_unstale":"2024-04-21T08:38:46.922633+0000","last_undegraded":"2024-04-21T08:38:46.922633+0000","last_fullsized":"2024-04-21T08:38:46.922633+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":10,"log_dups_size":0,"ondisk_log_size":10,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T20:02:10.569087+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,7,3],"acting":[1,7,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"3.2b","version":"91'11","reported_seq":43,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.913644+0000","last_change":"2024-04-21T08:36:20.900314+0000","last_active":"2024-04-21T08:38:46.913644+0000","last_peered":"2024-04-21T08:38:46.913644+0000","last_clean":"2024-04-21T08:38:46.913644+0000","last_became_active":"2024-04-21T08:36:20.900009+0000","last_became_peered":"2024-04-21T08:36:20.900009+0000","last_unstale":"2024-04-21T08:38:46.913644+0000","last_undegraded":"2024-04-21T08:38:46.913644+0000","last_fullsized":"2024-04-21T08:38:46.913644+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":11,"log_dups_size":0,"ondisk_log_size":11,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T15:19:59.997302+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[6,7,0],"acting":[6,7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":6,"acting_primary":6,"purged_snaps":[]},{"pgid":"3.2c","version":"91'8","reported_seq":40,"reported_epoch":94,"state":"active+clean","last_fresh":"2024-04-21T08:38:46.807555+0000","last_change":"2024-04-21T08:36:20.920725+0000","last_active":"2024-04-21T08:38:46.807555+0000","last_peered":"2024-04-21T08:38:46.807555+0000","last_clean":"2024-04-21T08:38:46.807555+0000","last_became_active":"2024-04-21T08:36:20.920288+0000","last_became_peered":"2024-04-21T08:36:20.920288+0000","last_unstale":"2024-04-21T08:38:46.807555+0000","last_undegraded":"2024-04-21T08:38:46.807555+0000","last_fullsized":"2024-04-21T08:38:46.807555+0000","mapping_epoch":83,"log_start":"0'0","ondisk_log_start":"0'0","created":83,"last_epoch_clean":84,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2024-04-21T08:36:20.261028+0000","last_clean_scrub_stamp":"2024-04-21T08:36:20.261028+0000","objects_scrubbed":0,"log_size":8,"log_dups_size":0,"ondisk_log_size":8,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"periodic scrub scheduled @ 2024-04-22T17:36:18.736388+0000","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[11,4,8],"acting":[11,4,8],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":11,"acting_primary":11,"purged_snaps":[]}],"pool_stats":[{"poolid":4,"num_pg":32,"stat_sum":{"num_bytes":234,"num_objects":2,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":1,"num_bytes_recovered":117,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":32768,"data_stored":32768,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":128,"acting":128,"num_store_stats":12},{"poolid":3,"num_pg":64,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":500,"ondisk_log_size":500,"up":192,"acting":192,"num_store_stats":12},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":6,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":2,"num_whiteouts":0,"num_read":237,"num_read_kb":330,"num_write":201,"num_write_kb":2938,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":4,"num_bytes_recovered":1180736,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":1781760,"data_stored":1771104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":148,"ondisk_log_size":148,"up":3,"acting":3,"num_store_stats":3},{"poolid":2,"num_pg":32,"stat_sum":{"num_bytes":603992,"num_objects":100,"num_object_clones":0,"num_object_copies":300,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":100,"num_whiteouts":0,"num_read":1402,"num_read_kb":818,"num_write":375,"num_write_kb":1473,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":18,"num_bytes_recovered":806,"num_keys_recovered":0,"num_objects_omap":66,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":2174976,"data_stored":1811976,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":305,"ondisk_log_size":305,"up":96,"acting":96,"num_store_stats":12}],"osd_stats":[{"osd":11,"up_from":80,"seq":343597383723,"num_pgs":28,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36408,"kb_used_data":1068,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708744,"statfs":{"total":95995035648,"available":95957753856,"internally_reserved":0,"allocated":1093632,"data_stored":531299,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1596,"internal_metadata":36174276},"hb_peers":[0,1,2,4,5,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":10,"up_from":72,"seq":309237645359,"num_pgs":33,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28952,"kb_used_data":1876,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716200,"statfs":{"total":95995035648,"available":95965388800,"internally_reserved":0,"allocated":1921024,"data_stored":1323967,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":27654592},"hb_peers":[0,1,2,3,4,5,6,8,9,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":9,"up_from":67,"seq":287762808885,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36804,"kb_used_data":1464,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708348,"statfs":{"total":95995035648,"available":95957348352,"internally_reserved":0,"allocated":1499136,"data_stored":923219,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1600,"internal_metadata":36174272},"hb_peers":[0,1,2,3,4,5,7,8,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":8,"up_from":59,"seq":253403070522,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28476,"kb_used_data":1456,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716676,"statfs":{"total":95995035648,"available":95965876224,"internally_reserved":0,"allocated":1490944,"data_stored":923575,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[0,1,2,3,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":7,"up_from":51,"seq":219043332160,"num_pgs":36,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28672,"kb_used_data":1652,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716480,"statfs":{"total":95995035648,"available":95965675520,"internally_reserved":0,"allocated":1691648,"data_stored":1127969,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1590,"internal_metadata":27654602},"hb_peers":[1,2,3,4,5,6,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":6,"up_from":46,"seq":197568495685,"num_pgs":32,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28076,"kb_used_data":1056,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717076,"statfs":{"total":95995035648,"available":95966285824,"internally_reserved":0,"allocated":1081344,"data_stored":525507,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":27654598},"hb_peers":[0,1,2,3,4,5,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":13,"seq":55834574941,"num_pgs":34,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36476,"kb_used_data":1136,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708676,"statfs":{"total":95995035648,"available":95957684224,"internally_reserved":0,"allocated":1163264,"data_stored":596717,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":8,"seq":34359738466,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28100,"kb_used_data":1080,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93717052,"statfs":{"total":95995035648,"available":95966261248,"internally_reserved":0,"allocated":1105920,"data_stored":527495,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":27654597},"hb_peers":[1,2,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":18,"seq":77309411416,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36480,"kb_used_data":1140,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708672,"statfs":{"total":95995035648,"available":95957680128,"internally_reserved":0,"allocated":1167360,"data_stored":598085,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[0,1,3,4,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":3,"up_from":25,"seq":107374182483,"num_pgs":40,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":28600,"kb_used_data":1584,"kb_used_omap":1,"kb_used_meta":27006,"kb_avail":93716552,"statfs":{"total":95995035648,"available":95965749248,"internally_reserved":0,"allocated":1622016,"data_stored":1050031,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1593,"internal_metadata":27654599},"hb_peers":[0,1,2,4,5,6,7,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":4,"up_from":33,"seq":141733920847,"num_pgs":35,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":36404,"kb_used_data":1064,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708748,"statfs":{"total":95995035648,"available":95957757952,"internally_reserved":0,"allocated":1089536,"data_stored":530895,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1594,"internal_metadata":36174278},"hb_peers":[1,2,3,5,6,7,8,9,10,11],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":5,"up_from":38,"seq":163208757321,"num_pgs":44,"num_osds":1,"num_per_pool_osds":1,"num_per_pool_omap_osds":1,"kb":93745152,"kb_used":37136,"kb_used_data":1800,"kb_used_omap":1,"kb_used_meta":35326,"kb_avail":93708016,"statfs":{"total":95995035648,"available":95957008384,"internally_reserved":0,"allocated":1843200,"data_stored":1254509,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":1595,"internal_metadata":36174277},"hb_peers":[0,1,2,3,4,6,7,8,9,10],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":593920,"data_stored":590368,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":40960,"data_stored":2710,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":98304,"data_stored":71932,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":102400,"data_stored":73300,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":557056,"data_stored":525246,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":20480,"data_stored":2014,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":180224,"data_stored":135260,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":16384,"data_stored":722,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":4624,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":421888,"data_stored":394694,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":430080,"data_stored":394338,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":258048,"data_stored":204718,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":24576,"data_stored":2418,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":3,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":0,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":1,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":2,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":3,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":4,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":5,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":6,"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":7,"total":0,"available":0,"internally_reserved":0,"allocated":8192,"data_stored":8192,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":8,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":9,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":10,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":4,"osd":11,"total":0,"available":0,"internally_reserved":0,"allocated":4096,"data_stored":4096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2024-04-21T08:38:57.968 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:38:58.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.100+0000 7fa1bee3f640 1 -- 172.21.15.86:0/2361807279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1b8106f10 msgr2=0x7fa1b81072f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:58.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.100+0000 7fa1bee3f640 1 --2- 172.21.15.86:0/2361807279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1b8106f10 0x7fa1b81072f0 secure :-1 s=READY pgs=589 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac009900 tx=0x7fa1ac02bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:58.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.100+0000 7fa1bee3f640 1 -- 172.21.15.86:0/2361807279 shutdown_connections 2024-04-21T08:38:58.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.100+0000 7fa1bee3f640 1 --2- 172.21.15.86:0/2361807279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1b8101ad0 0x7fa1b810dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.100+0000 7fa1bee3f640 1 --2- 172.21.15.86:0/2361807279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1b8101130 0x7fa1b8101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.100+0000 7fa1bee3f640 1 --2- 172.21.15.86:0/2361807279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1b8106f10 0x7fa1b81072f0 unknown :-1 s=CLOSED pgs=589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.100+0000 7fa1bee3f640 1 -- 172.21.15.86:0/2361807279 >> 172.21.15.86:0/2361807279 conn(0x7fa1b8077e30 msgr2=0x7fa1b8078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:58.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.100+0000 7fa1bee3f640 1 -- 172.21.15.86:0/2361807279 shutdown_connections 2024-04-21T08:38:58.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.100+0000 7fa1bee3f640 1 -- 172.21.15.86:0/2361807279 wait complete. 2024-04-21T08:38:58.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.101+0000 7fa1bee3f640 1 Processor -- start 2024-04-21T08:38:58.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.101+0000 7fa1bee3f640 1 -- start start 2024-04-21T08:38:58.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.101+0000 7fa1bee3f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1b8101130 0x7fa1b819e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:58.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.101+0000 7fa1bee3f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1b8101ad0 0x7fa1b819e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:58.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.101+0000 7fa1bee3f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1b8106f10 0x7fa1b81985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:58.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.101+0000 7fa1bee3f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b81110e0 con 0x7fa1b8101ad0 2024-04-21T08:38:58.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.101+0000 7fa1bee3f640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b8110f60 con 0x7fa1b8106f10 2024-04-21T08:38:58.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.101+0000 7fa1bee3f640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa1b8111260 con 0x7fa1b8101130 2024-04-21T08:38:58.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.101+0000 7fa1b7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1b8101ad0 0x7fa1b819e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:58.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.101+0000 7fa1b7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1b8101ad0 0x7fa1b819e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36600/0 (socket says 172.21.15.86:36600) 2024-04-21T08:38:58.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.101+0000 7fa1b7fff640 1 -- 172.21.15.86:0/977784777 learned_addr learned my addr 172.21.15.86:0/977784777 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:58.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.101+0000 7fa1bd3dc640 1 --2- 172.21.15.86:0/977784777 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1b8106f10 0x7fa1b81985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:58.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.101+0000 7fa1bcbdb640 1 --2- 172.21.15.86:0/977784777 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1b8101130 0x7fa1b819e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:58.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.101+0000 7fa1b7fff640 1 -- 172.21.15.86:0/977784777 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1b8101130 msgr2=0x7fa1b819e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:58.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.102+0000 7fa1b7fff640 1 --2- 172.21.15.86:0/977784777 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1b8101130 0x7fa1b819e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.102+0000 7fa1b7fff640 1 -- 172.21.15.86:0/977784777 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1b8106f10 msgr2=0x7fa1b81985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:58.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.102+0000 7fa1b7fff640 1 --2- 172.21.15.86:0/977784777 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1b8106f10 0x7fa1b81985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.102+0000 7fa1b7fff640 1 -- 172.21.15.86:0/977784777 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1b8198d30 con 0x7fa1b8101ad0 2024-04-21T08:38:58.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.102+0000 7fa1bcbdb640 1 --2- 172.21.15.86:0/977784777 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1b8101130 0x7fa1b819e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:58.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.102+0000 7fa1b7fff640 1 --2- 172.21.15.86:0/977784777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1b8101ad0 0x7fa1b819e940 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa1a400d950 tx=0x7fa1a400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:58.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.102+0000 7fa1b5ffb640 1 -- 172.21.15.86:0/977784777 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a4017070 con 0x7fa1b8101ad0 2024-04-21T08:38:58.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.102+0000 7fa1b5ffb640 1 -- 172.21.15.86:0/977784777 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa1a4009c30 con 0x7fa1b8101ad0 2024-04-21T08:38:58.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.102+0000 7fa1bee3f640 1 -- 172.21.15.86:0/977784777 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1b8199020 con 0x7fa1b8101ad0 2024-04-21T08:38:58.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.102+0000 7fa1b5ffb640 1 -- 172.21.15.86:0/977784777 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1a400f620 con 0x7fa1b8101ad0 2024-04-21T08:38:58.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.102+0000 7fa1bee3f640 1 -- 172.21.15.86:0/977784777 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1b81a52c0 con 0x7fa1b8101ad0 2024-04-21T08:38:58.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.103+0000 7fa1bee3f640 1 -- 172.21.15.86:0/977784777 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa1b8069230 con 0x7fa1b8101ad0 2024-04-21T08:38:58.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.104+0000 7fa1b5ffb640 1 -- 172.21.15.86:0/977784777 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fa1a4010460 con 0x7fa1b8101ad0 2024-04-21T08:38:58.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.105+0000 7fa1b5ffb640 1 --2- 172.21.15.86:0/977784777 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa1a80a3db0 0x7fa1a80a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:58.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.105+0000 7fa1bcbdb640 1 --2- 172.21.15.86:0/977784777 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa1a80a3db0 0x7fa1a80a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:58.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.105+0000 7fa1b5ffb640 1 -- 172.21.15.86:0/977784777 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fa1a40d0970 con 0x7fa1b8101ad0 2024-04-21T08:38:58.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.105+0000 7fa1bcbdb640 1 --2- 172.21.15.86:0/977784777 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa1a80a3db0 0x7fa1a80a6270 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac036040 tx=0x7fa1ac000bc0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:58.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.109+0000 7fa1b5ffb640 1 -- 172.21.15.86:0/977784777 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa1a409cb30 con 0x7fa1b8101ad0 2024-04-21T08:38:58.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.324+0000 7fa1bee3f640 1 -- 172.21.15.86:0/977784777 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa1b81a54f0 con 0x7fa1b8101ad0 2024-04-21T08:38:58.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.325+0000 7fa1b5ffb640 1 -- 172.21.15.86:0/977784777 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7fa1a4009de0 con 0x7fa1b8101ad0 2024-04-21T08:38:58.326 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:38:58.326 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:38:45.202126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24398,24394,24455,24472]},"id":1}]} 2024-04-21T08:38:58.326 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-21T08:38:58.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.327+0000 7fa1bee3f640 1 -- 172.21.15.86:0/977784777 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa1a80a3db0 msgr2=0x7fa1a80a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:58.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.327+0000 7fa1bee3f640 1 --2- 172.21.15.86:0/977784777 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa1a80a3db0 0x7fa1a80a6270 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fa1ac036040 tx=0x7fa1ac000bc0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.327+0000 7fa1bee3f640 1 -- 172.21.15.86:0/977784777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1b8101ad0 msgr2=0x7fa1b819e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:58.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.327+0000 7fa1bee3f640 1 --2- 172.21.15.86:0/977784777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1b8101ad0 0x7fa1b819e940 secure :-1 s=READY pgs=590 cs=0 l=1 rev1=1 crypto rx=0x7fa1a400d950 tx=0x7fa1a400de10 comp rx=0 tx=0).stop 2024-04-21T08:38:58.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.327+0000 7fa1bee3f640 1 -- 172.21.15.86:0/977784777 shutdown_connections 2024-04-21T08:38:58.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.327+0000 7fa1bee3f640 1 --2- 172.21.15.86:0/977784777 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa1a80a3db0 0x7fa1a80a6270 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.327+0000 7fa1bee3f640 1 --2- 172.21.15.86:0/977784777 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1b8106f10 0x7fa1b81985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.327+0000 7fa1bee3f640 1 --2- 172.21.15.86:0/977784777 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1b8101ad0 0x7fa1b819e940 unknown :-1 s=CLOSED pgs=590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.327+0000 7fa1bee3f640 1 --2- 172.21.15.86:0/977784777 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa1b8101130 0x7fa1b819e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.327+0000 7fa1bee3f640 1 -- 172.21.15.86:0/977784777 >> 172.21.15.86:0/977784777 conn(0x7fa1b8077e30 msgr2=0x7fa1b806a860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:58.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.327+0000 7fa1bee3f640 1 -- 172.21.15.86:0/977784777 shutdown_connections 2024-04-21T08:38:58.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.327+0000 7fa1bee3f640 1 -- 172.21.15.86:0/977784777 wait complete. 2024-04-21T08:38:58.338 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:38:58.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.437+0000 7f9010e21640 1 -- 172.21.15.86:0/862193251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f900c069a80 msgr2=0x7f900c10ed80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:58.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.437+0000 7f9010e21640 1 --2- 172.21.15.86:0/862193251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f900c069a80 0x7f900c10ed80 secure :-1 s=READY pgs=591 cs=0 l=1 rev1=1 crypto rx=0x7f9004009cd0 tx=0x7f900402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.438+0000 7f9010e21640 1 -- 172.21.15.86:0/862193251 shutdown_connections 2024-04-21T08:38:58.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.438+0000 7f9010e21640 1 --2- 172.21.15.86:0/862193251 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f900c069a80 0x7f900c10ed80 unknown :-1 s=CLOSED pgs=591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.438+0000 7f9010e21640 1 --2- 172.21.15.86:0/862193251 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f900c0690e0 0x7f900c069540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.438+0000 7f9010e21640 1 --2- 172.21.15.86:0/862193251 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f900c107f70 0x7f900c108350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.438+0000 7f9010e21640 1 -- 172.21.15.86:0/862193251 >> 172.21.15.86:0/862193251 conn(0x7f900c077e30 msgr2=0x7f900c078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:58.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.438+0000 7f9010e21640 1 -- 172.21.15.86:0/862193251 shutdown_connections 2024-04-21T08:38:58.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.438+0000 7f9010e21640 1 -- 172.21.15.86:0/862193251 wait complete. 2024-04-21T08:38:58.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.438+0000 7f9010e21640 1 Processor -- start 2024-04-21T08:38:58.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.438+0000 7f9010e21640 1 -- start start 2024-04-21T08:38:58.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.438+0000 7f9010e21640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f900c0690e0 0x7f900c198600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:58.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.439+0000 7f9010e21640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f900c069a80 0x7f900c198b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:58.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.439+0000 7f9010e21640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f900c107f70 0x7f900c19cec0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:58.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.439+0000 7f9010e21640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f900c112140 con 0x7f900c0690e0 2024-04-21T08:38:58.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.439+0000 7f9010e21640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f900c111fc0 con 0x7f900c107f70 2024-04-21T08:38:58.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.439+0000 7f9010e21640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f900c1122c0 con 0x7f900c069a80 2024-04-21T08:38:58.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.439+0000 7f900a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f900c0690e0 0x7f900c198600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:58.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.439+0000 7f900a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f900c0690e0 0x7f900c198600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36620/0 (socket says 172.21.15.86:36620) 2024-04-21T08:38:58.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.439+0000 7f900a59c640 1 -- 172.21.15.86:0/3661518707 learned_addr learned my addr 172.21.15.86:0/3661518707 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:58.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.439+0000 7f9009d9b640 1 --2- 172.21.15.86:0/3661518707 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f900c069a80 0x7f900c198b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:58.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.439+0000 7f900ad9d640 1 --2- 172.21.15.86:0/3661518707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f900c107f70 0x7f900c19cec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:58.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.439+0000 7f900a59c640 1 -- 172.21.15.86:0/3661518707 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f900c069a80 msgr2=0x7f900c198b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:58.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.439+0000 7f900a59c640 1 --2- 172.21.15.86:0/3661518707 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f900c069a80 0x7f900c198b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.439+0000 7f900a59c640 1 -- 172.21.15.86:0/3661518707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f900c107f70 msgr2=0x7f900c19cec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:58.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.439+0000 7f900a59c640 1 --2- 172.21.15.86:0/3661518707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f900c107f70 0x7f900c19cec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.439+0000 7f900a59c640 1 -- 172.21.15.86:0/3661518707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f900c19d5f0 con 0x7f900c0690e0 2024-04-21T08:38:58.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.439+0000 7f900ad9d640 1 --2- 172.21.15.86:0/3661518707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f900c107f70 0x7f900c19cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:58.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.439+0000 7f9009d9b640 1 --2- 172.21.15.86:0/3661518707 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f900c069a80 0x7f900c198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:58.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.439+0000 7f900a59c640 1 --2- 172.21.15.86:0/3661518707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f900c0690e0 0x7f900c198600 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00d950 tx=0x7f8ffc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:58.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.439+0000 7f8ffb7fe640 1 -- 172.21.15.86:0/3661518707 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc017070 con 0x7f900c0690e0 2024-04-21T08:38:58.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.439+0000 7f8ffb7fe640 1 -- 172.21.15.86:0/3661518707 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ffc009c30 con 0x7f900c0690e0 2024-04-21T08:38:58.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.439+0000 7f9010e21640 1 -- 172.21.15.86:0/3661518707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f900c19d8e0 con 0x7f900c0690e0 2024-04-21T08:38:58.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.439+0000 7f8ffb7fe640 1 -- 172.21.15.86:0/3661518707 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ffc00f620 con 0x7f900c0690e0 2024-04-21T08:38:58.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.439+0000 7f9010e21640 1 -- 172.21.15.86:0/3661518707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f900c1a5170 con 0x7f900c0690e0 2024-04-21T08:38:58.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.440+0000 7f9010e21640 1 -- 172.21.15.86:0/3661518707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f900c068de0 con 0x7f900c0690e0 2024-04-21T08:38:58.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.441+0000 7f8ffb7fe640 1 -- 172.21.15.86:0/3661518707 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f8ffc010460 con 0x7f900c0690e0 2024-04-21T08:38:58.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.442+0000 7f8ffb7fe640 1 --2- 172.21.15.86:0/3661518707 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8ff40a3db0 0x7f8ff40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:58.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.442+0000 7f9009d9b640 1 --2- 172.21.15.86:0/3661518707 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8ff40a3db0 0x7f8ff40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:58.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.442+0000 7f8ffb7fe640 1 -- 172.21.15.86:0/3661518707 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f8ffc0d0a20 con 0x7f900c0690e0 2024-04-21T08:38:58.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.442+0000 7f9009d9b640 1 --2- 172.21.15.86:0/3661518707 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8ff40a3db0 0x7f8ff40a6270 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f90000058c0 tx=0x7f9000005850 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:58.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.447+0000 7f8ffb7fe640 1 -- 172.21.15.86:0/3661518707 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8ffc09cbe0 con 0x7f900c0690e0 2024-04-21T08:38:58.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:58 smithi086 ceph-mon[27939]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 0 B/s wr, 19 op/s 2024-04-21T08:38:58.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:58 smithi086 ceph-mon[27939]: from='client.15879 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:58.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.626+0000 7f9010e21640 1 -- 172.21.15.86:0/3661518707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f900c10a400 con 0x7f900c0690e0 2024-04-21T08:38:58.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.627+0000 7f8ffb7fe640 1 -- 172.21.15.86:0/3661518707 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+25067 (secure 0 0 0) 0x7f8ffc009de0 con 0x7f900c0690e0 2024-04-21T08:38:58.628 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:38:58.628 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":94,"fsid":"f8cec7cc-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:22.687329+0000","modified":"2024-04-21T08:38:46.631676+0000","last_up_change":"2024-04-21T08:35:30.048315+0000","last_in_change":"2024-04-21T08:35:10.099496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:45.279109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:18.300782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:19.574636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:22.905632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89b7601f-0620-4db2-90f1-b45cea173738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6803","nonce":2571516897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6805","nonce":2571516897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6809","nonce":2571516897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6807","nonce":2571516897}]},"public_addr":"172.21.15.86:6803/2571516897","cluster_addr":"172.21.15.86:6805/2571516897","heartbeat_back_addr":"172.21.15.86:6809/2571516897","heartbeat_front_addr":"172.21.15.86:6807/2571516897","state":["exists","up"]},{"osd":1,"uuid":"4891031c-f98c-42bc-bb3b-a420855ecc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6801","nonce":74067182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6803","nonce":74067182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6807","nonce":74067182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6805","nonce":74067182}]},"public_addr":"172.21.15.88:6801/74067182","cluster_addr":"172.21.15.88:6803/74067182","heartbeat_back_addr":"172.21.15.88:6807/74067182","heartbeat_front_addr":"172.21.15.88:6805/74067182","state":["exists","up"]},{"osd":2,"uuid":"60d12aa4-9312-4fa2-8d0b-2b3f42caad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6801","nonce":4137659051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6803","nonce":4137659051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6807","nonce":4137659051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6805","nonce":4137659051}]},"public_addr":"172.21.15.179:6801/4137659051","cluster_addr":"172.21.15.179:6803/4137659051","heartbeat_back_addr":"172.21.15.179:6807/4137659051","heartbeat_front_addr":"172.21.15.179:6805/4137659051","state":["exists","up"]},{"osd":3,"uuid":"404c3158-8cb0-4c02-b48a-f9ee5ff7b784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6811","nonce":2309603148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6813","nonce":2309603148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6817","nonce":2309603148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6815","nonce":2309603148}]},"public_addr":"172.21.15.86:6811/2309603148","cluster_addr":"172.21.15.86:6813/2309603148","heartbeat_back_addr":"172.21.15.86:6817/2309603148","heartbeat_front_addr":"172.21.15.86:6815/2309603148","state":["exists","up"]},{"osd":4,"uuid":"458ff202-d781-4ec7-ac93-0b49f835b503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6809","nonce":1818714217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6811","nonce":1818714217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6815","nonce":1818714217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6813","nonce":1818714217}]},"public_addr":"172.21.15.88:6809/1818714217","cluster_addr":"172.21.15.88:6811/1818714217","heartbeat_back_addr":"172.21.15.88:6815/1818714217","heartbeat_front_addr":"172.21.15.88:6813/1818714217","state":["exists","up"]},{"osd":5,"uuid":"8123cc90-bd8b-4204-bf31-46b34e9f589d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6809","nonce":863840990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6811","nonce":863840990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6815","nonce":863840990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6813","nonce":863840990}]},"public_addr":"172.21.15.179:6809/863840990","cluster_addr":"172.21.15.179:6811/863840990","heartbeat_back_addr":"172.21.15.179:6815/863840990","heartbeat_front_addr":"172.21.15.179:6813/863840990","state":["exists","up"]},{"osd":6,"uuid":"aac9d7c7-659a-4110-903a-a871c086066f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6819","nonce":1224231118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6821","nonce":1224231118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6825","nonce":1224231118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6823","nonce":1224231118}]},"public_addr":"172.21.15.86:6819/1224231118","cluster_addr":"172.21.15.86:6821/1224231118","heartbeat_back_addr":"172.21.15.86:6825/1224231118","heartbeat_front_addr":"172.21.15.86:6823/1224231118","state":["exists","up"]},{"osd":7,"uuid":"df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6817","nonce":136629878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6819","nonce":136629878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6823","nonce":136629878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6821","nonce":136629878}]},"public_addr":"172.21.15.88:6817/136629878","cluster_addr":"172.21.15.88:6819/136629878","heartbeat_back_addr":"172.21.15.88:6823/136629878","heartbeat_front_addr":"172.21.15.88:6821/136629878","state":["exists","up"]},{"osd":8,"uuid":"63e59f4b-0426-46db-92f7-c55dae61b5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6817","nonce":1770418957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6819","nonce":1770418957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6823","nonce":1770418957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6821","nonce":1770418957}]},"public_addr":"172.21.15.179:6817/1770418957","cluster_addr":"172.21.15.179:6819/1770418957","heartbeat_back_addr":"172.21.15.179:6823/1770418957","heartbeat_front_addr":"172.21.15.179:6821/1770418957","state":["exists","up"]},{"osd":9,"uuid":"5841bb36-ed6b-45d8-b857-eeab139c44e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6827","nonce":1108500274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6829","nonce":1108500274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6833","nonce":1108500274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6831","nonce":1108500274}]},"public_addr":"172.21.15.86:6827/1108500274","cluster_addr":"172.21.15.86:6829/1108500274","heartbeat_back_addr":"172.21.15.86:6833/1108500274","heartbeat_front_addr":"172.21.15.86:6831/1108500274","state":["exists","up"]},{"osd":10,"uuid":"f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6825","nonce":1294275353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6827","nonce":1294275353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6831","nonce":1294275353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6829","nonce":1294275353}]},"public_addr":"172.21.15.88:6825/1294275353","cluster_addr":"172.21.15.88:6827/1294275353","heartbeat_back_addr":"172.21.15.88:6831/1294275353","heartbeat_front_addr":"172.21.15.88:6829/1294275353","state":["exists","up"]},{"osd":11,"uuid":"e0d4cc76-e7a6-4a6b-909c-95488ccd5a36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6825","nonce":4007070233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6827","nonce":4007070233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6831","nonce":4007070233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6829","nonce":4007070233}]},"public_addr":"172.21.15.179:6825/4007070233","cluster_addr":"172.21.15.179:6827/4007070233","heartbeat_back_addr":"172.21.15.179:6831/4007070233","heartbeat_front_addr":"172.21.15.179:6829/4007070233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:55.210605+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:18.723090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:42.547979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:06.421622+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:30.858315+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:55.455313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.623369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:46.185111+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.103627+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:37.297545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:02.637431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:28.199292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1068089552":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/4273877219":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/2707594358":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/4060044103":"2024-04-22T08:38:46.631621+0000","172.21.15.86:6800/1364156965":"2024-04-22T08:38:46.631621+0000","172.21.15.86:6800/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/1364156965":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/1412393225":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6800/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/1687852404":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2020707404":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2568130933":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/3952607648":"2024-04-22T08:28:57.384403+0000","172.21.15.86:6801/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/2756903426":"2024-04-22T08:29:23.190001+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:38:58.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.629+0000 7f9010e21640 1 -- 172.21.15.86:0/3661518707 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8ff40a3db0 msgr2=0x7f8ff40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:58.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.629+0000 7f9010e21640 1 --2- 172.21.15.86:0/3661518707 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8ff40a3db0 0x7f8ff40a6270 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f90000058c0 tx=0x7f9000005850 comp rx=0 tx=0).stop 2024-04-21T08:38:58.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.629+0000 7f9010e21640 1 -- 172.21.15.86:0/3661518707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f900c0690e0 msgr2=0x7f900c198600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:58.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.629+0000 7f9010e21640 1 --2- 172.21.15.86:0/3661518707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f900c0690e0 0x7f900c198600 secure :-1 s=READY pgs=592 cs=0 l=1 rev1=1 crypto rx=0x7f8ffc00d950 tx=0x7f8ffc00de10 comp rx=0 tx=0).stop 2024-04-21T08:38:58.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.629+0000 7f9010e21640 1 -- 172.21.15.86:0/3661518707 shutdown_connections 2024-04-21T08:38:58.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.629+0000 7f9010e21640 1 --2- 172.21.15.86:0/3661518707 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8ff40a3db0 0x7f8ff40a6270 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.629+0000 7f9010e21640 1 --2- 172.21.15.86:0/3661518707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f900c107f70 0x7f900c19cec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.629+0000 7f9010e21640 1 --2- 172.21.15.86:0/3661518707 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f900c069a80 0x7f900c198b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.629+0000 7f9010e21640 1 --2- 172.21.15.86:0/3661518707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f900c0690e0 0x7f900c198600 unknown :-1 s=CLOSED pgs=592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.629+0000 7f9010e21640 1 -- 172.21.15.86:0/3661518707 >> 172.21.15.86:0/3661518707 conn(0x7f900c077e30 msgr2=0x7f900c0fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:58.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.629+0000 7f9010e21640 1 -- 172.21.15.86:0/3661518707 shutdown_connections 2024-04-21T08:38:58.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.629+0000 7f9010e21640 1 -- 172.21.15.86:0/3661518707 wait complete. 2024-04-21T08:38:58.641 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:31:45.279109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:36:18.300782+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:36:19.574636+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '85', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:36:22.905632+0000', 'flags': 5, 'flags_names': 'hashpspool,ec_overwrites', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '89', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:38:58.642 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:38:58.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:58 smithi088 ceph-mon[31259]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 0 B/s wr, 19 op/s 2024-04-21T08:38:58.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:58 smithi088 ceph-mon[31259]: from='client.15879 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:58.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.743+0000 7fecbc6d2640 1 -- 172.21.15.86:0/2715227582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecb4103b00 msgr2=0x7fecb410bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:58.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.743+0000 7fecbc6d2640 1 --2- 172.21.15.86:0/2715227582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecb4103b00 0x7fecb410bfe0 secure :-1 s=READY pgs=593 cs=0 l=1 rev1=1 crypto rx=0x7feca4009900 tx=0x7feca402bd60 comp rx=0 tx=0).stop 2024-04-21T08:38:58.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.743+0000 7fecbc6d2640 1 -- 172.21.15.86:0/2715227582 shutdown_connections 2024-04-21T08:38:58.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.743+0000 7fecbc6d2640 1 --2- 172.21.15.86:0/2715227582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecb410c520 0x7fecb410e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.743+0000 7fecbc6d2640 1 --2- 172.21.15.86:0/2715227582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecb4103b00 0x7fecb410bfe0 unknown :-1 s=CLOSED pgs=593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.743+0000 7fecbc6d2640 1 --2- 172.21.15.86:0/2715227582 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fecb41031e0 0x7fecb41035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.743+0000 7fecbc6d2640 1 -- 172.21.15.86:0/2715227582 >> 172.21.15.86:0/2715227582 conn(0x7fecb40771d0 msgr2=0x7fecb40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:58.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.743+0000 7fecbc6d2640 1 -- 172.21.15.86:0/2715227582 shutdown_connections 2024-04-21T08:38:58.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.743+0000 7fecbc6d2640 1 -- 172.21.15.86:0/2715227582 wait complete. 2024-04-21T08:38:58.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.744+0000 7fecbc6d2640 1 Processor -- start 2024-04-21T08:38:58.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.744+0000 7fecbc6d2640 1 -- start start 2024-04-21T08:38:58.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.744+0000 7fecbc6d2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecb41031e0 0x7fecb419e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:58.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.744+0000 7fecbc6d2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fecb4103b00 0x7fecb419e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:58.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.744+0000 7fecbc6d2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecb410c520 0x7fecb4198590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:58.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.744+0000 7fecbc6d2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fecb41121b0 con 0x7fecb410c520 2024-04-21T08:38:58.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.744+0000 7fecbc6d2640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fecb4112030 con 0x7fecb4103b00 2024-04-21T08:38:58.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.744+0000 7fecbc6d2640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fecb4112330 con 0x7fecb41031e0 2024-04-21T08:38:58.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.744+0000 7fecbac6f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecb410c520 0x7fecb4198590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:58.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.744+0000 7fecbac6f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecb410c520 0x7fecb4198590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36650/0 (socket says 172.21.15.86:36650) 2024-04-21T08:38:58.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.744+0000 7fecbac6f640 1 -- 172.21.15.86:0/2902273678 learned_addr learned my addr 172.21.15.86:0/2902273678 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:58.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.744+0000 7fecba46e640 1 --2- 172.21.15.86:0/2902273678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecb41031e0 0x7fecb419e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:58.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.744+0000 7fecb9c6d640 1 --2- 172.21.15.86:0/2902273678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fecb4103b00 0x7fecb419e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:58.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.744+0000 7fecbac6f640 1 -- 172.21.15.86:0/2902273678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecb41031e0 msgr2=0x7fecb419e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:58.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.744+0000 7fecbac6f640 1 --2- 172.21.15.86:0/2902273678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecb41031e0 0x7fecb419e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.744+0000 7fecbac6f640 1 -- 172.21.15.86:0/2902273678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fecb4103b00 msgr2=0x7fecb419e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:58.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.744+0000 7fecbac6f640 1 --2- 172.21.15.86:0/2902273678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fecb4103b00 0x7fecb419e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.744+0000 7fecbac6f640 1 -- 172.21.15.86:0/2902273678 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecb4198d10 con 0x7fecb410c520 2024-04-21T08:38:58.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.744+0000 7fecb9c6d640 1 --2- 172.21.15.86:0/2902273678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fecb4103b00 0x7fecb419e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:58.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.744+0000 7fecbac6f640 1 --2- 172.21.15.86:0/2902273678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecb410c520 0x7fecb4198590 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fecac00eaa0 tx=0x7fecac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:58.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.745+0000 7fecab7fe640 1 -- 172.21.15.86:0/2902273678 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecac00cdb0 con 0x7fecb410c520 2024-04-21T08:38:58.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.745+0000 7fecab7fe640 1 -- 172.21.15.86:0/2902273678 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecac005020 con 0x7fecb410c520 2024-04-21T08:38:58.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.745+0000 7fecbc6d2640 1 -- 172.21.15.86:0/2902273678 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecb4199000 con 0x7fecb410c520 2024-04-21T08:38:58.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.745+0000 7fecab7fe640 1 -- 172.21.15.86:0/2902273678 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecac0116d0 con 0x7fecb410c520 2024-04-21T08:38:58.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.745+0000 7fecbc6d2640 1 -- 172.21.15.86:0/2902273678 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecb41a52a0 con 0x7fecb410c520 2024-04-21T08:38:58.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.746+0000 7fecbc6d2640 1 -- 172.21.15.86:0/2902273678 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fecb40691c0 con 0x7fecb410c520 2024-04-21T08:38:58.749 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:58 smithi179 ceph-mon[31829]: pgmap v8: 129 pgs: 129 active+clean; 1.1 MiB data, 381 MiB used, 1.0 TiB / 1.0 TiB avail; 26 KiB/s rd, 0 B/s wr, 19 op/s 2024-04-21T08:38:58.749 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:58 smithi179 ceph-mon[31829]: from='client.15879 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:58.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.747+0000 7fecab7fe640 1 -- 172.21.15.86:0/2902273678 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fecac0051d0 con 0x7fecb410c520 2024-04-21T08:38:58.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.747+0000 7fecab7fe640 1 --2- 172.21.15.86:0/2902273678 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7feca00a3db0 0x7feca00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:58.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.747+0000 7fecba46e640 1 --2- 172.21.15.86:0/2902273678 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7feca00a3db0 0x7feca00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:58.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.747+0000 7fecab7fe640 1 -- 172.21.15.86:0/2902273678 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fecac0d0f30 con 0x7fecb410c520 2024-04-21T08:38:58.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.748+0000 7fecba46e640 1 --2- 172.21.15.86:0/2902273678 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7feca00a3db0 0x7feca00a6270 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fecb00058c0 tx=0x7fecb000a4c0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:58.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.752+0000 7fecab7fe640 1 -- 172.21.15.86:0/2902273678 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fecac016030 con 0x7fecb410c520 2024-04-21T08:38:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.928+0000 7fecbc6d2640 1 -- 172.21.15.86:0/2902273678 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fecb402d5c0 con 0x7fecb410c520 2024-04-21T08:38:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.928+0000 7fecab7fe640 1 -- 172.21.15.86:0/2902273678 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v94) ==== 93+0+10 (secure 0 0 0) 0x7fecac09d0f0 con 0x7fecb410c520 2024-04-21T08:38:58.929 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-21T08:38:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.930+0000 7fecbc6d2640 1 -- 172.21.15.86:0/2902273678 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7feca00a3db0 msgr2=0x7feca00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.930+0000 7fecbc6d2640 1 --2- 172.21.15.86:0/2902273678 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7feca00a3db0 0x7feca00a6270 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fecb00058c0 tx=0x7fecb000a4c0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.930+0000 7fecbc6d2640 1 -- 172.21.15.86:0/2902273678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecb410c520 msgr2=0x7fecb4198590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.930+0000 7fecbc6d2640 1 --2- 172.21.15.86:0/2902273678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecb410c520 0x7fecb4198590 secure :-1 s=READY pgs=594 cs=0 l=1 rev1=1 crypto rx=0x7fecac00eaa0 tx=0x7fecac00ef60 comp rx=0 tx=0).stop 2024-04-21T08:38:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.930+0000 7fecbc6d2640 1 -- 172.21.15.86:0/2902273678 shutdown_connections 2024-04-21T08:38:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.930+0000 7fecbc6d2640 1 --2- 172.21.15.86:0/2902273678 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7feca00a3db0 0x7feca00a6270 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.930+0000 7fecbc6d2640 1 --2- 172.21.15.86:0/2902273678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecb410c520 0x7fecb4198590 unknown :-1 s=CLOSED pgs=594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.930+0000 7fecbc6d2640 1 --2- 172.21.15.86:0/2902273678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fecb4103b00 0x7fecb419e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.931+0000 7fecbc6d2640 1 --2- 172.21.15.86:0/2902273678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecb41031e0 0x7fecb419e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.931+0000 7fecbc6d2640 1 -- 172.21.15.86:0/2902273678 >> 172.21.15.86:0/2902273678 conn(0x7fecb40771d0 msgr2=0x7fecb40fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.931+0000 7fecbc6d2640 1 -- 172.21.15.86:0/2902273678 shutdown_connections 2024-04-21T08:38:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:58.931+0000 7fecbc6d2640 1 -- 172.21.15.86:0/2902273678 wait complete. 2024-04-21T08:38:58.942 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:38:59.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.042+0000 7f47d0011640 1 -- 172.21.15.86:0/315325290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47c8101090 msgr2=0x7f47c8101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:59.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.042+0000 7f47d0011640 1 --2- 172.21.15.86:0/315325290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47c8101090 0x7f47c8101470 secure :-1 s=READY pgs=595 cs=0 l=1 rev1=1 crypto rx=0x7f47c0009900 tx=0x7f47c002bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:59.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.043+0000 7f47d0011640 1 -- 172.21.15.86:0/315325290 shutdown_connections 2024-04-21T08:38:59.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.043+0000 7f47d0011640 1 --2- 172.21.15.86:0/315325290 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47c8106160 0x7f47c810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.043+0000 7f47d0011640 1 --2- 172.21.15.86:0/315325290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47c8101a40 0x7f47c8105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.043+0000 7f47d0011640 1 --2- 172.21.15.86:0/315325290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47c8101090 0x7f47c8101470 unknown :-1 s=CLOSED pgs=595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.043+0000 7f47d0011640 1 -- 172.21.15.86:0/315325290 >> 172.21.15.86:0/315325290 conn(0x7f47c8077f50 msgr2=0x7f47c8078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:59.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.043+0000 7f47d0011640 1 -- 172.21.15.86:0/315325290 shutdown_connections 2024-04-21T08:38:59.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.043+0000 7f47d0011640 1 -- 172.21.15.86:0/315325290 wait complete. 2024-04-21T08:38:59.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.043+0000 7f47d0011640 1 Processor -- start 2024-04-21T08:38:59.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.043+0000 7f47d0011640 1 -- start start 2024-04-21T08:38:59.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.043+0000 7f47d0011640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47c8101090 0x7f47c81975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:59.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.043+0000 7f47d0011640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47c8106160 0x7f47c8197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:59.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.043+0000 7f47d0011640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47c819bea0 0x7f47c819c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:59.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.043+0000 7f47d0011640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47c81110a0 con 0x7f47c819bea0 2024-04-21T08:38:59.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.043+0000 7f47d0011640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f47c8110f20 con 0x7f47c8106160 2024-04-21T08:38:59.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.043+0000 7f47d0011640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f47c8111220 con 0x7f47c8101090 2024-04-21T08:38:59.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.044+0000 7f47ce5ae640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47c819bea0 0x7f47c819c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:59.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.044+0000 7f47ce5ae640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47c819bea0 0x7f47c819c350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36666/0 (socket says 172.21.15.86:36666) 2024-04-21T08:38:59.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.044+0000 7f47ce5ae640 1 -- 172.21.15.86:0/1269349601 learned_addr learned my addr 172.21.15.86:0/1269349601 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:59.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.044+0000 7f47cddad640 1 --2- 172.21.15.86:0/1269349601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47c8101090 0x7f47c81975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:59.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.044+0000 7f47cd5ac640 1 --2- 172.21.15.86:0/1269349601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47c8106160 0x7f47c8197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:59.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.044+0000 7f47ce5ae640 1 -- 172.21.15.86:0/1269349601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47c8101090 msgr2=0x7f47c81975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:59.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.044+0000 7f47ce5ae640 1 --2- 172.21.15.86:0/1269349601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47c8101090 0x7f47c81975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.044+0000 7f47ce5ae640 1 -- 172.21.15.86:0/1269349601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47c8106160 msgr2=0x7f47c8197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:59.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.044+0000 7f47ce5ae640 1 --2- 172.21.15.86:0/1269349601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47c8106160 0x7f47c8197b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.044+0000 7f47ce5ae640 1 -- 172.21.15.86:0/1269349601 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47c819c990 con 0x7f47c819bea0 2024-04-21T08:38:59.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.044+0000 7f47cd5ac640 1 --2- 172.21.15.86:0/1269349601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47c8106160 0x7f47c8197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:59.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.044+0000 7f47ce5ae640 1 --2- 172.21.15.86:0/1269349601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47c819bea0 0x7f47c819c350 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f47c400cad0 tx=0x7f47c400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:59.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.044+0000 7f47beffd640 1 -- 172.21.15.86:0/1269349601 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c400ae20 con 0x7f47c819bea0 2024-04-21T08:38:59.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.044+0000 7f47beffd640 1 -- 172.21.15.86:0/1269349601 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47c4005020 con 0x7f47c819bea0 2024-04-21T08:38:59.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.044+0000 7f47d0011640 1 -- 172.21.15.86:0/1269349601 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47c819cc80 con 0x7f47c819bea0 2024-04-21T08:38:59.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.044+0000 7f47beffd640 1 -- 172.21.15.86:0/1269349601 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47c400f6b0 con 0x7f47c819bea0 2024-04-21T08:38:59.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.044+0000 7f47d0011640 1 -- 172.21.15.86:0/1269349601 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47c81a4420 con 0x7f47c819bea0 2024-04-21T08:38:59.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.045+0000 7f47d0011640 1 -- 172.21.15.86:0/1269349601 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f47c8068d70 con 0x7f47c819bea0 2024-04-21T08:38:59.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.046+0000 7f47beffd640 1 -- 172.21.15.86:0/1269349601 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f47c40051d0 con 0x7f47c819bea0 2024-04-21T08:38:59.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.047+0000 7f47beffd640 1 --2- 172.21.15.86:0/1269349601 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f47b40a3db0 0x7f47b40a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:59.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.047+0000 7f47cddad640 1 --2- 172.21.15.86:0/1269349601 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f47b40a3db0 0x7f47b40a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:59.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.047+0000 7f47beffd640 1 -- 172.21.15.86:0/1269349601 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f47c40d10c0 con 0x7f47c819bea0 2024-04-21T08:38:59.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.047+0000 7f47cddad640 1 --2- 172.21.15.86:0/1269349601 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f47b40a3db0 0x7f47b40a6270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f47c0009230 tx=0x7f47c0005fd0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:59.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.051+0000 7f47beffd640 1 -- 172.21.15.86:0/1269349601 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f47c4014030 con 0x7f47c819bea0 2024-04-21T08:38:59.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.228+0000 7f47d0011640 1 -- 172.21.15.86:0/1269349601 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f47c8101a40 con 0x7f47c819bea0 2024-04-21T08:38:59.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.229+0000 7f47beffd640 1 -- 172.21.15.86:0/1269349601 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v94) ==== 104+0+11 (secure 0 0 0) 0x7f47c409d280 con 0x7f47c819bea0 2024-04-21T08:38:59.229 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-21T08:38:59.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.230+0000 7f47d0011640 1 -- 172.21.15.86:0/1269349601 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f47b40a3db0 msgr2=0x7f47b40a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:59.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.230+0000 7f47d0011640 1 --2- 172.21.15.86:0/1269349601 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f47b40a3db0 0x7f47b40a6270 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f47c0009230 tx=0x7f47c0005fd0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.230+0000 7f47d0011640 1 -- 172.21.15.86:0/1269349601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47c819bea0 msgr2=0x7f47c819c350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:59.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.230+0000 7f47d0011640 1 --2- 172.21.15.86:0/1269349601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47c819bea0 0x7f47c819c350 secure :-1 s=READY pgs=596 cs=0 l=1 rev1=1 crypto rx=0x7f47c400cad0 tx=0x7f47c400cf90 comp rx=0 tx=0).stop 2024-04-21T08:38:59.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.230+0000 7f47d0011640 1 -- 172.21.15.86:0/1269349601 shutdown_connections 2024-04-21T08:38:59.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.230+0000 7f47d0011640 1 --2- 172.21.15.86:0/1269349601 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f47b40a3db0 0x7f47b40a6270 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.230+0000 7f47d0011640 1 --2- 172.21.15.86:0/1269349601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47c819bea0 0x7f47c819c350 unknown :-1 s=CLOSED pgs=596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.230+0000 7f47d0011640 1 --2- 172.21.15.86:0/1269349601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f47c8106160 0x7f47c8197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.231+0000 7f47d0011640 1 --2- 172.21.15.86:0/1269349601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47c8101090 0x7f47c81975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.231+0000 7f47d0011640 1 -- 172.21.15.86:0/1269349601 >> 172.21.15.86:0/1269349601 conn(0x7f47c8077f50 msgr2=0x7f47c80ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:59.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.231+0000 7f47d0011640 1 -- 172.21.15.86:0/1269349601 shutdown_connections 2024-04-21T08:38:59.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.231+0000 7f47d0011640 1 -- 172.21.15.86:0/1269349601 wait complete. 2024-04-21T08:38:59.242 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:38:59.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.344+0000 7f7045c70640 1 -- 172.21.15.86:0/2581348509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70401021e0 msgr2=0x7f70401025c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:59.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.344+0000 7f7045c70640 1 --2- 172.21.15.86:0/2581348509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70401021e0 0x7f70401025c0 secure :-1 s=READY pgs=597 cs=0 l=1 rev1=1 crypto rx=0x7f7034009900 tx=0x7f703402bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:59.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.344+0000 7f7045c70640 1 -- 172.21.15.86:0/2581348509 shutdown_connections 2024-04-21T08:38:59.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.344+0000 7f7045c70640 1 --2- 172.21.15.86:0/2581348509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70401072b0 0x7f704010ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.344+0000 7f7045c70640 1 --2- 172.21.15.86:0/2581348509 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7040102b90 0x7f7040106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.344+0000 7f7045c70640 1 --2- 172.21.15.86:0/2581348509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70401021e0 0x7f70401025c0 unknown :-1 s=CLOSED pgs=597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.344+0000 7f7045c70640 1 -- 172.21.15.86:0/2581348509 >> 172.21.15.86:0/2581348509 conn(0x7f70400fda60 msgr2=0x7f70400ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:59.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.344+0000 7f7045c70640 1 -- 172.21.15.86:0/2581348509 shutdown_connections 2024-04-21T08:38:59.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.344+0000 7f7045c70640 1 -- 172.21.15.86:0/2581348509 wait complete. 2024-04-21T08:38:59.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.344+0000 7f7045c70640 1 Processor -- start 2024-04-21T08:38:59.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.344+0000 7f7045c70640 1 -- start start 2024-04-21T08:38:59.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.344+0000 7f7045c70640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70401021e0 0x7f704019e640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:59.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.345+0000 7f7045c70640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70401072b0 0x7f704019eb80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:59.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.345+0000 7f7045c70640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7040198750 0x7f7040198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:59.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.345+0000 7f7045c70640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7040112290 con 0x7f7040198750 2024-04-21T08:38:59.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.345+0000 7f7045c70640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7040112110 con 0x7f70401072b0 2024-04-21T08:38:59.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.345+0000 7f7045c70640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7040112410 con 0x7f70401021e0 2024-04-21T08:38:59.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.345+0000 7f703ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7040198750 0x7f7040198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:59.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.345+0000 7f703ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7040198750 0x7f7040198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36680/0 (socket says 172.21.15.86:36680) 2024-04-21T08:38:59.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.345+0000 7f703ffff640 1 -- 172.21.15.86:0/3235256692 learned_addr learned my addr 172.21.15.86:0/3235256692 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:59.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.345+0000 7f703f7fe640 1 --2- 172.21.15.86:0/3235256692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70401021e0 0x7f704019e640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:59.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.345+0000 7f703effd640 1 --2- 172.21.15.86:0/3235256692 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70401072b0 0x7f704019eb80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:59.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.345+0000 7f703ffff640 1 -- 172.21.15.86:0/3235256692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70401021e0 msgr2=0x7f704019e640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:59.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.345+0000 7f703ffff640 1 --2- 172.21.15.86:0/3235256692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70401021e0 0x7f704019e640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.345+0000 7f703ffff640 1 -- 172.21.15.86:0/3235256692 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70401072b0 msgr2=0x7f704019eb80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:59.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.345+0000 7f703ffff640 1 --2- 172.21.15.86:0/3235256692 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70401072b0 0x7f704019eb80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.345+0000 7f703ffff640 1 -- 172.21.15.86:0/3235256692 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7040199350 con 0x7f7040198750 2024-04-21T08:38:59.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.345+0000 7f703ffff640 1 --2- 172.21.15.86:0/3235256692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7040198750 0x7f7040198bd0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f703800b970 tx=0x7f703800be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:59.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.345+0000 7f703cff9640 1 -- 172.21.15.86:0/3235256692 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7038019070 con 0x7f7040198750 2024-04-21T08:38:59.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.345+0000 7f7045c70640 1 -- 172.21.15.86:0/3235256692 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70401a53e0 con 0x7f7040198750 2024-04-21T08:38:59.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.345+0000 7f703cff9640 1 -- 172.21.15.86:0/3235256692 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7038014440 con 0x7f7040198750 2024-04-21T08:38:59.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.345+0000 7f703cff9640 1 -- 172.21.15.86:0/3235256692 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70380075d0 con 0x7f7040198750 2024-04-21T08:38:59.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.345+0000 7f7045c70640 1 -- 172.21.15.86:0/3235256692 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70401a57e0 con 0x7f7040198750 2024-04-21T08:38:59.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.346+0000 7f7045c70640 1 -- 172.21.15.86:0/3235256692 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f70401036b0 con 0x7f7040198750 2024-04-21T08:38:59.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.347+0000 7f703cff9640 1 -- 172.21.15.86:0/3235256692 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f703801c070 con 0x7f7040198750 2024-04-21T08:38:59.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.348+0000 7f703cff9640 1 --2- 172.21.15.86:0/3235256692 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f70280a3db0 0x7f70280a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:59.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.348+0000 7f703f7fe640 1 --2- 172.21.15.86:0/3235256692 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f70280a3db0 0x7f70280a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:59.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.348+0000 7f703cff9640 1 -- 172.21.15.86:0/3235256692 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f70380d06c0 con 0x7f7040198750 2024-04-21T08:38:59.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.348+0000 7f703f7fe640 1 --2- 172.21.15.86:0/3235256692 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f70280a3db0 0x7f70280a6270 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7034006010 tx=0x7f7034005e40 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:59.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.352+0000 7f703cff9640 1 -- 172.21.15.86:0/3235256692 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f703809c340 con 0x7f7040198750 2024-04-21T08:38:59.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.528+0000 7f7045c70640 1 -- 172.21.15.86:0/3235256692 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f704010a4e0 con 0x7f7040198750 2024-04-21T08:38:59.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.529+0000 7f703cff9640 1 -- 172.21.15.86:0/3235256692 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v94) ==== 100+0+11 (secure 0 0 0) 0x7f70380a0640 con 0x7f7040198750 2024-04-21T08:38:59.529 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-21T08:38:59.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.531+0000 7f7045c70640 1 -- 172.21.15.86:0/3235256692 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f70280a3db0 msgr2=0x7f70280a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:59.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.531+0000 7f7045c70640 1 --2- 172.21.15.86:0/3235256692 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f70280a3db0 0x7f70280a6270 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f7034006010 tx=0x7f7034005e40 comp rx=0 tx=0).stop 2024-04-21T08:38:59.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.531+0000 7f7045c70640 1 -- 172.21.15.86:0/3235256692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7040198750 msgr2=0x7f7040198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:59.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.531+0000 7f7045c70640 1 --2- 172.21.15.86:0/3235256692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7040198750 0x7f7040198bd0 secure :-1 s=READY pgs=598 cs=0 l=1 rev1=1 crypto rx=0x7f703800b970 tx=0x7f703800be30 comp rx=0 tx=0).stop 2024-04-21T08:38:59.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.531+0000 7f7045c70640 1 -- 172.21.15.86:0/3235256692 shutdown_connections 2024-04-21T08:38:59.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.531+0000 7f7045c70640 1 --2- 172.21.15.86:0/3235256692 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f70280a3db0 0x7f70280a6270 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.531+0000 7f7045c70640 1 --2- 172.21.15.86:0/3235256692 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7040198750 0x7f7040198bd0 unknown :-1 s=CLOSED pgs=598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.531+0000 7f7045c70640 1 --2- 172.21.15.86:0/3235256692 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f70401072b0 0x7f704019eb80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.531+0000 7f7045c70640 1 --2- 172.21.15.86:0/3235256692 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70401021e0 0x7f704019e640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.531+0000 7f7045c70640 1 -- 172.21.15.86:0/3235256692 >> 172.21.15.86:0/3235256692 conn(0x7f70400fda60 msgr2=0x7f70400fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:59.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.531+0000 7f7045c70640 1 -- 172.21.15.86:0/3235256692 shutdown_connections 2024-04-21T08:38:59.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.531+0000 7f7045c70640 1 -- 172.21.15.86:0/3235256692 wait complete. 2024-04-21T08:38:59.541 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:59 smithi086 ceph-mon[27939]: from='client.15885 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:59.541 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:59 smithi086 ceph-mon[27939]: from='client.15891 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:59.541 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:59 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/977784777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:59.541 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:59 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3661518707' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:59.541 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:59 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2902273678' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:59.541 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:38:59 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1269349601' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:59.542 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:38:59.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.643+0000 7f0493074640 1 -- 172.21.15.86:0/546552945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f048c102b40 msgr2=0x7f048c106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:59.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.643+0000 7f0493074640 1 --2- 172.21.15.86:0/546552945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f048c102b40 0x7f048c106b80 secure :-1 s=READY pgs=599 cs=0 l=1 rev1=1 crypto rx=0x7f0480009900 tx=0x7f048002bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:59.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.643+0000 7f0493074640 1 -- 172.21.15.86:0/546552945 shutdown_connections 2024-04-21T08:38:59.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.643+0000 7f0493074640 1 --2- 172.21.15.86:0/546552945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f048c107260 0x7f048c10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.643+0000 7f0493074640 1 --2- 172.21.15.86:0/546552945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f048c102b40 0x7f048c106b80 unknown :-1 s=CLOSED pgs=599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.643+0000 7f0493074640 1 --2- 172.21.15.86:0/546552945 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f048c102190 0x7f048c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.643+0000 7f0493074640 1 -- 172.21.15.86:0/546552945 >> 172.21.15.86:0/546552945 conn(0x7f048c0793a0 msgr2=0x7f048c0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:59.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.644+0000 7f0493074640 1 -- 172.21.15.86:0/546552945 shutdown_connections 2024-04-21T08:38:59.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.644+0000 7f0493074640 1 -- 172.21.15.86:0/546552945 wait complete. 2024-04-21T08:38:59.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.644+0000 7f0493074640 1 Processor -- start 2024-04-21T08:38:59.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.644+0000 7f0493074640 1 -- start start 2024-04-21T08:38:59.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.644+0000 7f0493074640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f048c102190 0x7f048c198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:59.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.644+0000 7f0493074640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f048c107260 0x7f048c198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:59.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.644+0000 7f0493074640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f048c19cfc0 0x7f048c19d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:59.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.644+0000 7f0493074640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f048c1121a0 con 0x7f048c102190 2024-04-21T08:38:59.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.644+0000 7f0493074640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f048c112020 con 0x7f048c19cfc0 2024-04-21T08:38:59.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.644+0000 7f0493074640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f048c112320 con 0x7f048c107260 2024-04-21T08:38:59.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.644+0000 7f0490e10640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f048c102190 0x7f048c198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:59.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.644+0000 7f0490e10640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f048c102190 0x7f048c198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45758/0 (socket says 172.21.15.86:45758) 2024-04-21T08:38:59.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.644+0000 7f0490e10640 1 -- 172.21.15.86:0/1163078857 learned_addr learned my addr 172.21.15.86:0/1163078857 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:59.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.645+0000 7f0491611640 1 --2- 172.21.15.86:0/1163078857 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f048c19cfc0 0x7f048c19d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:59.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.645+0000 7f048bfff640 1 --2- 172.21.15.86:0/1163078857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f048c107260 0x7f048c198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:59.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.645+0000 7f0490e10640 1 -- 172.21.15.86:0/1163078857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f048c107260 msgr2=0x7f048c198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:59.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.645+0000 7f0490e10640 1 --2- 172.21.15.86:0/1163078857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f048c107260 0x7f048c198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.645+0000 7f0490e10640 1 -- 172.21.15.86:0/1163078857 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f048c19cfc0 msgr2=0x7f048c19d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:59.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.645+0000 7f0490e10640 1 --2- 172.21.15.86:0/1163078857 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f048c19cfc0 0x7f048c19d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.645+0000 7f0490e10640 1 -- 172.21.15.86:0/1163078857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f048c19dab0 con 0x7f048c102190 2024-04-21T08:38:59.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.645+0000 7f048bfff640 1 --2- 172.21.15.86:0/1163078857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f048c107260 0x7f048c198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:38:59.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.645+0000 7f0490e10640 1 --2- 172.21.15.86:0/1163078857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f048c102190 0x7f048c198700 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f047c00c970 tx=0x7f047c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:59.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.645+0000 7f0489ffb640 1 -- 172.21.15.86:0/1163078857 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c007bb0 con 0x7f048c102190 2024-04-21T08:38:59.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.645+0000 7f0489ffb640 1 -- 172.21.15.86:0/1163078857 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f047c007d50 con 0x7f048c102190 2024-04-21T08:38:59.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.645+0000 7f0493074640 1 -- 172.21.15.86:0/1163078857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f048c19dda0 con 0x7f048c102190 2024-04-21T08:38:59.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.645+0000 7f0489ffb640 1 -- 172.21.15.86:0/1163078857 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f047c00f450 con 0x7f048c102190 2024-04-21T08:38:59.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.645+0000 7f0493074640 1 -- 172.21.15.86:0/1163078857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f048c1a5540 con 0x7f048c102190 2024-04-21T08:38:59.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.646+0000 7f0493074640 1 -- 172.21.15.86:0/1163078857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f048c0684e0 con 0x7f048c102190 2024-04-21T08:38:59.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.647+0000 7f0489ffb640 1 -- 172.21.15.86:0/1163078857 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f047c004f20 con 0x7f048c102190 2024-04-21T08:38:59.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.648+0000 7f0489ffb640 1 --2- 172.21.15.86:0/1163078857 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f04740a3db0 0x7f04740a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:59.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.648+0000 7f048bfff640 1 --2- 172.21.15.86:0/1163078857 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f04740a3db0 0x7f04740a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:59.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.648+0000 7f0489ffb640 1 -- 172.21.15.86:0/1163078857 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f047c0d0a90 con 0x7f048c102190 2024-04-21T08:38:59.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.648+0000 7f048bfff640 1 --2- 172.21.15.86:0/1163078857 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f04740a3db0 0x7f04740a6270 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0480036040 tx=0x7f0480000bc0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:59.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.652+0000 7f0489ffb640 1 -- 172.21.15.86:0/1163078857 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f047c014030 con 0x7f048c102190 2024-04-21T08:38:59.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:59 smithi088 ceph-mon[31259]: from='client.15885 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:59.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:59 smithi088 ceph-mon[31259]: from='client.15891 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:59.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:59 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/977784777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:59.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:59 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3661518707' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:59.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:59 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2902273678' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:59.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:38:59 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1269349601' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:59.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:59 smithi179 ceph-mon[31829]: from='client.15885 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:59.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:59 smithi179 ceph-mon[31829]: from='client.15891 -' entity='client.admin' cmd=[{"prefix": "pg dump", "target": ["mon-mgr", ""], "format": "json"}]: dispatch 2024-04-21T08:38:59.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:59 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/977784777' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:38:59.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:59 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3661518707' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:38:59.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:59 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2902273678' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:38:59.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:38:59 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1269349601' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:38:59.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.831+0000 7f0493074640 1 -- 172.21.15.86:0/1163078857 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f048c10a490 con 0x7f048c102190 2024-04-21T08:38:59.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.832+0000 7f0489ffb640 1 -- 172.21.15.86:0/1163078857 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v94) ==== 103+0+11 (secure 0 0 0) 0x7f047c09cc50 con 0x7f048c102190 2024-04-21T08:38:59.832 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-21T08:38:59.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.834+0000 7f0493074640 1 -- 172.21.15.86:0/1163078857 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f04740a3db0 msgr2=0x7f04740a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:59.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.834+0000 7f0493074640 1 --2- 172.21.15.86:0/1163078857 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f04740a3db0 0x7f04740a6270 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f0480036040 tx=0x7f0480000bc0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.834+0000 7f0493074640 1 -- 172.21.15.86:0/1163078857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f048c102190 msgr2=0x7f048c198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:59.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.834+0000 7f0493074640 1 --2- 172.21.15.86:0/1163078857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f048c102190 0x7f048c198700 secure :-1 s=READY pgs=600 cs=0 l=1 rev1=1 crypto rx=0x7f047c00c970 tx=0x7f047c00ce30 comp rx=0 tx=0).stop 2024-04-21T08:38:59.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.834+0000 7f0493074640 1 -- 172.21.15.86:0/1163078857 shutdown_connections 2024-04-21T08:38:59.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.834+0000 7f0493074640 1 --2- 172.21.15.86:0/1163078857 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f04740a3db0 0x7f04740a6270 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.834+0000 7f0493074640 1 --2- 172.21.15.86:0/1163078857 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f048c19cfc0 0x7f048c19d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.834+0000 7f0493074640 1 --2- 172.21.15.86:0/1163078857 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f048c107260 0x7f048c198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.834+0000 7f0493074640 1 --2- 172.21.15.86:0/1163078857 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f048c102190 0x7f048c198700 unknown :-1 s=CLOSED pgs=600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.834+0000 7f0493074640 1 -- 172.21.15.86:0/1163078857 >> 172.21.15.86:0/1163078857 conn(0x7f048c0793a0 msgr2=0x7f048c0fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:59.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.834+0000 7f0493074640 1 -- 172.21.15.86:0/1163078857 shutdown_connections 2024-04-21T08:38:59.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.834+0000 7f0493074640 1 -- 172.21.15.86:0/1163078857 wait complete. 2024-04-21T08:38:59.846 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:38:59.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.944+0000 7f0039e26640 1 -- 172.21.15.86:0/2051962507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0034102200 msgr2=0x7f00341025e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:59.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.944+0000 7f0039e26640 1 --2- 172.21.15.86:0/2051962507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0034102200 0x7f00341025e0 secure :-1 s=READY pgs=601 cs=0 l=1 rev1=1 crypto rx=0x7f0024009900 tx=0x7f002402bc30 comp rx=0 tx=0).stop 2024-04-21T08:38:59.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.944+0000 7f0039e26640 1 -- 172.21.15.86:0/2051962507 shutdown_connections 2024-04-21T08:38:59.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.944+0000 7f0039e26640 1 --2- 172.21.15.86:0/2051962507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00341072d0 0x7f003410eea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.944+0000 7f0039e26640 1 --2- 172.21.15.86:0/2051962507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0034102bb0 0x7f0034106bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.944+0000 7f0039e26640 1 --2- 172.21.15.86:0/2051962507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0034102200 0x7f00341025e0 unknown :-1 s=CLOSED pgs=601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.944+0000 7f0039e26640 1 -- 172.21.15.86:0/2051962507 >> 172.21.15.86:0/2051962507 conn(0x7f00340793e0 msgr2=0x7f00340797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:38:59.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.944+0000 7f0039e26640 1 -- 172.21.15.86:0/2051962507 shutdown_connections 2024-04-21T08:38:59.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.944+0000 7f0039e26640 1 -- 172.21.15.86:0/2051962507 wait complete. 2024-04-21T08:38:59.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.945+0000 7f0039e26640 1 Processor -- start 2024-04-21T08:38:59.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.945+0000 7f0039e26640 1 -- start start 2024-04-21T08:38:59.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.945+0000 7f0039e26640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0034102200 0x7f0034198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:59.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.945+0000 7f0039e26640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00341072d0 0x7f0034198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:59.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.945+0000 7f0039e26640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f003419cfd0 0x7f003419d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:59.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.945+0000 7f0039e26640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0034112210 con 0x7f003419cfd0 2024-04-21T08:38:59.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.945+0000 7f0039e26640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0034112090 con 0x7f00341072d0 2024-04-21T08:38:59.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.945+0000 7f0039e26640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0034112390 con 0x7f0034102200 2024-04-21T08:38:59.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.945+0000 7f0033fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f003419cfd0 0x7f003419d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:59.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.945+0000 7f0033fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f003419cfd0 0x7f003419d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45786/0 (socket says 172.21.15.86:45786) 2024-04-21T08:38:59.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.945+0000 7f0033fff640 1 -- 172.21.15.86:0/1551884384 learned_addr learned my addr 172.21.15.86:0/1551884384 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:38:59.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.945+0000 7f00337fe640 1 --2- 172.21.15.86:0/1551884384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0034102200 0x7f0034198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:59.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.945+0000 7f0032ffd640 1 --2- 172.21.15.86:0/1551884384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00341072d0 0x7f0034198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:59.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.946+0000 7f0033fff640 1 -- 172.21.15.86:0/1551884384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0034102200 msgr2=0x7f0034198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:59.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.946+0000 7f0033fff640 1 --2- 172.21.15.86:0/1551884384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0034102200 0x7f0034198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.946+0000 7f0033fff640 1 -- 172.21.15.86:0/1551884384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00341072d0 msgr2=0x7f0034198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:38:59.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.946+0000 7f0033fff640 1 --2- 172.21.15.86:0/1551884384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00341072d0 0x7f0034198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:38:59.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.946+0000 7f0033fff640 1 -- 172.21.15.86:0/1551884384 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f003419dac0 con 0x7f003419cfd0 2024-04-21T08:38:59.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.946+0000 7f00337fe640 1 --2- 172.21.15.86:0/1551884384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0034102200 0x7f0034198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:38:59.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.946+0000 7f0033fff640 1 --2- 172.21.15.86:0/1551884384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f003419cfd0 0x7f003419d480 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f002c00eaa0 tx=0x7f002c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:59.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.946+0000 7f0030ff9640 1 -- 172.21.15.86:0/1551884384 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c00cdb0 con 0x7f003419cfd0 2024-04-21T08:38:59.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.946+0000 7f0030ff9640 1 -- 172.21.15.86:0/1551884384 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f002c005020 con 0x7f003419cfd0 2024-04-21T08:38:59.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.946+0000 7f0030ff9640 1 -- 172.21.15.86:0/1551884384 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f002c0116d0 con 0x7f003419cfd0 2024-04-21T08:38:59.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.946+0000 7f0039e26640 1 -- 172.21.15.86:0/1551884384 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f003419ddb0 con 0x7f003419cfd0 2024-04-21T08:38:59.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.946+0000 7f0039e26640 1 -- 172.21.15.86:0/1551884384 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f00341a5550 con 0x7f003419cfd0 2024-04-21T08:38:59.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.947+0000 7f0039e26640 1 -- 172.21.15.86:0/1551884384 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f0034068550 con 0x7f003419cfd0 2024-04-21T08:38:59.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.949+0000 7f0030ff9640 1 -- 172.21.15.86:0/1551884384 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f002c0051d0 con 0x7f003419cfd0 2024-04-21T08:38:59.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.949+0000 7f0030ff9640 1 --2- 172.21.15.86:0/1551884384 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f001c0a3db0 0x7f001c0a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:38:59.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.949+0000 7f00337fe640 1 --2- 172.21.15.86:0/1551884384 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f001c0a3db0 0x7f001c0a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:38:59.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.949+0000 7f0030ff9640 1 -- 172.21.15.86:0/1551884384 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f002c0d1350 con 0x7f003419cfd0 2024-04-21T08:38:59.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.950+0000 7f00337fe640 1 --2- 172.21.15.86:0/1551884384 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f001c0a3db0 0x7f001c0a6270 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0024005ab0 tx=0x7f0024005a20 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:38:59.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:38:59.953+0000 7f0030ff9640 1 -- 172.21.15.86:0/1551884384 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f002c016030 con 0x7f003419cfd0 2024-04-21T08:39:00.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.168+0000 7f0039e26640 1 -- 172.21.15.86:0/1551884384 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f0034199a60 con 0x7f003419cfd0 2024-04-21T08:39:00.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.169+0000 7f0030ff9640 1 -- 172.21.15.86:0/1551884384 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7f0034199a60 con 0x7f003419cfd0 2024-04-21T08:39:00.169 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:39:00.169 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:38:45.202126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24398,24394,24455,24472]},"id":1}]} 2024-04-21T08:39:00.169 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-21T08:39:00.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.171+0000 7f0039e26640 1 -- 172.21.15.86:0/1551884384 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f001c0a3db0 msgr2=0x7f001c0a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.171+0000 7f0039e26640 1 --2- 172.21.15.86:0/1551884384 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f001c0a3db0 0x7f001c0a6270 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7f0024005ab0 tx=0x7f0024005a20 comp rx=0 tx=0).stop 2024-04-21T08:39:00.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.171+0000 7f0039e26640 1 -- 172.21.15.86:0/1551884384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f003419cfd0 msgr2=0x7f003419d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.171+0000 7f0039e26640 1 --2- 172.21.15.86:0/1551884384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f003419cfd0 0x7f003419d480 secure :-1 s=READY pgs=602 cs=0 l=1 rev1=1 crypto rx=0x7f002c00eaa0 tx=0x7f002c00ef60 comp rx=0 tx=0).stop 2024-04-21T08:39:00.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.171+0000 7f0039e26640 1 -- 172.21.15.86:0/1551884384 shutdown_connections 2024-04-21T08:39:00.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.171+0000 7f0039e26640 1 --2- 172.21.15.86:0/1551884384 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f001c0a3db0 0x7f001c0a6270 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.171+0000 7f0039e26640 1 --2- 172.21.15.86:0/1551884384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f003419cfd0 0x7f003419d480 unknown :-1 s=CLOSED pgs=602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.171+0000 7f0039e26640 1 --2- 172.21.15.86:0/1551884384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f00341072d0 0x7f0034198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.171+0000 7f0039e26640 1 --2- 172.21.15.86:0/1551884384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0034102200 0x7f0034198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.171+0000 7f0039e26640 1 -- 172.21.15.86:0/1551884384 >> 172.21.15.86:0/1551884384 conn(0x7f00340793e0 msgr2=0x7f00340fff20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:00.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.171+0000 7f0039e26640 1 -- 172.21.15.86:0/1551884384 shutdown_connections 2024-04-21T08:39:00.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.171+0000 7f0039e26640 1 -- 172.21.15.86:0/1551884384 wait complete. 2024-04-21T08:39:00.183 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:39:00.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.293+0000 7efe64375640 1 -- 172.21.15.86:0/928620818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe5c101a00 msgr2=0x7efe5c101e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.293+0000 7efe64375640 1 --2- 172.21.15.86:0/928620818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe5c101a00 0x7efe5c101e80 secure :-1 s=READY pgs=605 cs=0 l=1 rev1=1 crypto rx=0x7efe4c009930 tx=0x7efe4c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:00.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.293+0000 7efe64375640 1 -- 172.21.15.86:0/928620818 shutdown_connections 2024-04-21T08:39:00.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.293+0000 7efe64375640 1 --2- 172.21.15.86:0/928620818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe5c1023c0 0x7efe5c10dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.293+0000 7efe64375640 1 --2- 172.21.15.86:0/928620818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe5c101a00 0x7efe5c101e80 unknown :-1 s=CLOSED pgs=605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.293+0000 7efe64375640 1 --2- 172.21.15.86:0/928620818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe5c105e90 0x7efe5c106270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.293+0000 7efe64375640 1 -- 172.21.15.86:0/928620818 >> 172.21.15.86:0/928620818 conn(0x7efe5c076170 msgr2=0x7efe5c076580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:00.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.293+0000 7efe64375640 1 -- 172.21.15.86:0/928620818 shutdown_connections 2024-04-21T08:39:00.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.293+0000 7efe64375640 1 -- 172.21.15.86:0/928620818 wait complete. 2024-04-21T08:39:00.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.294+0000 7efe64375640 1 Processor -- start 2024-04-21T08:39:00.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.294+0000 7efe64375640 1 -- start start 2024-04-21T08:39:00.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.294+0000 7efe64375640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe5c101a00 0x7efe5c198770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.294+0000 7efe64375640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe5c1023c0 0x7efe5c198cb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.294+0000 7efe64375640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe5c105e90 0x7efe5c19cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.294+0000 7efe64375640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c111160 con 0x7efe5c105e90 2024-04-21T08:39:00.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.294+0000 7efe64375640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c110fe0 con 0x7efe5c1023c0 2024-04-21T08:39:00.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.294+0000 7efe64375640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7efe5c1112e0 con 0x7efe5c101a00 2024-04-21T08:39:00.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.294+0000 7efe62912640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe5c105e90 0x7efe5c19cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.294+0000 7efe62912640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe5c105e90 0x7efe5c19cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45818/0 (socket says 172.21.15.86:45818) 2024-04-21T08:39:00.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.294+0000 7efe62912640 1 -- 172.21.15.86:0/3744683880 learned_addr learned my addr 172.21.15.86:0/3744683880 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:00.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.294+0000 7efe61910640 1 --2- 172.21.15.86:0/3744683880 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe5c1023c0 0x7efe5c198cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.294+0000 7efe62111640 1 --2- 172.21.15.86:0/3744683880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe5c101a00 0x7efe5c198770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.294+0000 7efe62912640 1 -- 172.21.15.86:0/3744683880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe5c101a00 msgr2=0x7efe5c198770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.294+0000 7efe62912640 1 --2- 172.21.15.86:0/3744683880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe5c101a00 0x7efe5c198770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.294+0000 7efe62912640 1 -- 172.21.15.86:0/3744683880 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe5c1023c0 msgr2=0x7efe5c198cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.294+0000 7efe62912640 1 --2- 172.21.15.86:0/3744683880 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe5c1023c0 0x7efe5c198cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.294+0000 7efe62912640 1 -- 172.21.15.86:0/3744683880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe5c19d710 con 0x7efe5c105e90 2024-04-21T08:39:00.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.294+0000 7efe62111640 1 --2- 172.21.15.86:0/3744683880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe5c101a00 0x7efe5c198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:00.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.294+0000 7efe62912640 1 --2- 172.21.15.86:0/3744683880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe5c105e90 0x7efe5c19cfe0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7efe5400eb10 tx=0x7efe5400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:00.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.295+0000 7efe537fe640 1 -- 172.21.15.86:0/3744683880 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe54004b20 con 0x7efe5c105e90 2024-04-21T08:39:00.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.295+0000 7efe537fe640 1 -- 172.21.15.86:0/3744683880 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe54004cc0 con 0x7efe5c105e90 2024-04-21T08:39:00.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.295+0000 7efe537fe640 1 -- 172.21.15.86:0/3744683880 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe54010670 con 0x7efe5c105e90 2024-04-21T08:39:00.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.295+0000 7efe64375640 1 -- 172.21.15.86:0/3744683880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe5c19da00 con 0x7efe5c105e90 2024-04-21T08:39:00.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.295+0000 7efe64375640 1 -- 172.21.15.86:0/3744683880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe5c1a5290 con 0x7efe5c105e90 2024-04-21T08:39:00.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.296+0000 7efe64375640 1 -- 172.21.15.86:0/3744683880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efe5c069230 con 0x7efe5c105e90 2024-04-21T08:39:00.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.297+0000 7efe537fe640 1 -- 172.21.15.86:0/3744683880 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7efe54005000 con 0x7efe5c105e90 2024-04-21T08:39:00.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.298+0000 7efe537fe640 1 --2- 172.21.15.86:0/3744683880 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efe480a3e30 0x7efe480a62f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.298+0000 7efe62111640 1 --2- 172.21.15.86:0/3744683880 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efe480a3e30 0x7efe480a62f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.298+0000 7efe537fe640 1 -- 172.21.15.86:0/3744683880 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7efe540d14b0 con 0x7efe5c105e90 2024-04-21T08:39:00.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.298+0000 7efe62111640 1 --2- 172.21.15.86:0/3744683880 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efe480a3e30 0x7efe480a62f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efe5c077770 tx=0x7efe58005f90 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:00.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.301+0000 7efe537fe640 1 -- 172.21.15.86:0/3744683880 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efe54053ff0 con 0x7efe5c105e90 2024-04-21T08:39:00.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.481+0000 7efe64375640 1 -- 172.21.15.86:0/3744683880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7efe5c10a5c0 con 0x7efe5c105e90 2024-04-21T08:39:00.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.483+0000 7efe537fe640 1 -- 172.21.15.86:0/3744683880 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v94) ==== 74+0+25067 (secure 0 0 0) 0x7efe5409d670 con 0x7efe5c105e90 2024-04-21T08:39:00.483 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:39:00.484 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":94,"fsid":"f8cec7cc-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:22.687329+0000","modified":"2024-04-21T08:38:46.631676+0000","last_up_change":"2024-04-21T08:35:30.048315+0000","last_in_change":"2024-04-21T08:35:10.099496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:45.279109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:18.300782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:19.574636+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"85","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:22.905632+0000","flags":5,"flags_names":"hashpspool,ec_overwrites","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"89","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89b7601f-0620-4db2-90f1-b45cea173738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6803","nonce":2571516897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6805","nonce":2571516897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6809","nonce":2571516897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6807","nonce":2571516897}]},"public_addr":"172.21.15.86:6803/2571516897","cluster_addr":"172.21.15.86:6805/2571516897","heartbeat_back_addr":"172.21.15.86:6809/2571516897","heartbeat_front_addr":"172.21.15.86:6807/2571516897","state":["exists","up"]},{"osd":1,"uuid":"4891031c-f98c-42bc-bb3b-a420855ecc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6801","nonce":74067182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6803","nonce":74067182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6807","nonce":74067182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6805","nonce":74067182}]},"public_addr":"172.21.15.88:6801/74067182","cluster_addr":"172.21.15.88:6803/74067182","heartbeat_back_addr":"172.21.15.88:6807/74067182","heartbeat_front_addr":"172.21.15.88:6805/74067182","state":["exists","up"]},{"osd":2,"uuid":"60d12aa4-9312-4fa2-8d0b-2b3f42caad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6801","nonce":4137659051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6803","nonce":4137659051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6807","nonce":4137659051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6805","nonce":4137659051}]},"public_addr":"172.21.15.179:6801/4137659051","cluster_addr":"172.21.15.179:6803/4137659051","heartbeat_back_addr":"172.21.15.179:6807/4137659051","heartbeat_front_addr":"172.21.15.179:6805/4137659051","state":["exists","up"]},{"osd":3,"uuid":"404c3158-8cb0-4c02-b48a-f9ee5ff7b784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6811","nonce":2309603148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6813","nonce":2309603148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6817","nonce":2309603148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6815","nonce":2309603148}]},"public_addr":"172.21.15.86:6811/2309603148","cluster_addr":"172.21.15.86:6813/2309603148","heartbeat_back_addr":"172.21.15.86:6817/2309603148","heartbeat_front_addr":"172.21.15.86:6815/2309603148","state":["exists","up"]},{"osd":4,"uuid":"458ff202-d781-4ec7-ac93-0b49f835b503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6809","nonce":1818714217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6811","nonce":1818714217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6815","nonce":1818714217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6813","nonce":1818714217}]},"public_addr":"172.21.15.88:6809/1818714217","cluster_addr":"172.21.15.88:6811/1818714217","heartbeat_back_addr":"172.21.15.88:6815/1818714217","heartbeat_front_addr":"172.21.15.88:6813/1818714217","state":["exists","up"]},{"osd":5,"uuid":"8123cc90-bd8b-4204-bf31-46b34e9f589d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6809","nonce":863840990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6811","nonce":863840990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6815","nonce":863840990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6813","nonce":863840990}]},"public_addr":"172.21.15.179:6809/863840990","cluster_addr":"172.21.15.179:6811/863840990","heartbeat_back_addr":"172.21.15.179:6815/863840990","heartbeat_front_addr":"172.21.15.179:6813/863840990","state":["exists","up"]},{"osd":6,"uuid":"aac9d7c7-659a-4110-903a-a871c086066f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6819","nonce":1224231118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6821","nonce":1224231118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6825","nonce":1224231118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6823","nonce":1224231118}]},"public_addr":"172.21.15.86:6819/1224231118","cluster_addr":"172.21.15.86:6821/1224231118","heartbeat_back_addr":"172.21.15.86:6825/1224231118","heartbeat_front_addr":"172.21.15.86:6823/1224231118","state":["exists","up"]},{"osd":7,"uuid":"df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6817","nonce":136629878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6819","nonce":136629878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6823","nonce":136629878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6821","nonce":136629878}]},"public_addr":"172.21.15.88:6817/136629878","cluster_addr":"172.21.15.88:6819/136629878","heartbeat_back_addr":"172.21.15.88:6823/136629878","heartbeat_front_addr":"172.21.15.88:6821/136629878","state":["exists","up"]},{"osd":8,"uuid":"63e59f4b-0426-46db-92f7-c55dae61b5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6817","nonce":1770418957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6819","nonce":1770418957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6823","nonce":1770418957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6821","nonce":1770418957}]},"public_addr":"172.21.15.179:6817/1770418957","cluster_addr":"172.21.15.179:6819/1770418957","heartbeat_back_addr":"172.21.15.179:6823/1770418957","heartbeat_front_addr":"172.21.15.179:6821/1770418957","state":["exists","up"]},{"osd":9,"uuid":"5841bb36-ed6b-45d8-b857-eeab139c44e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6827","nonce":1108500274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6829","nonce":1108500274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6833","nonce":1108500274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6831","nonce":1108500274}]},"public_addr":"172.21.15.86:6827/1108500274","cluster_addr":"172.21.15.86:6829/1108500274","heartbeat_back_addr":"172.21.15.86:6833/1108500274","heartbeat_front_addr":"172.21.15.86:6831/1108500274","state":["exists","up"]},{"osd":10,"uuid":"f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6825","nonce":1294275353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6827","nonce":1294275353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6831","nonce":1294275353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6829","nonce":1294275353}]},"public_addr":"172.21.15.88:6825/1294275353","cluster_addr":"172.21.15.88:6827/1294275353","heartbeat_back_addr":"172.21.15.88:6831/1294275353","heartbeat_front_addr":"172.21.15.88:6829/1294275353","state":["exists","up"]},{"osd":11,"uuid":"e0d4cc76-e7a6-4a6b-909c-95488ccd5a36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6825","nonce":4007070233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6827","nonce":4007070233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6831","nonce":4007070233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6829","nonce":4007070233}]},"public_addr":"172.21.15.179:6825/4007070233","cluster_addr":"172.21.15.179:6827/4007070233","heartbeat_back_addr":"172.21.15.179:6831/4007070233","heartbeat_front_addr":"172.21.15.179:6829/4007070233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:55.210605+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:18.723090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:42.547979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:06.421622+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:30.858315+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:55.455313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.623369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:46.185111+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.103627+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:37.297545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:02.637431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:28.199292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1068089552":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/4273877219":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/2707594358":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/4060044103":"2024-04-22T08:38:46.631621+0000","172.21.15.86:6800/1364156965":"2024-04-22T08:38:46.631621+0000","172.21.15.86:6800/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/1364156965":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/1412393225":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6800/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/1687852404":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2020707404":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2568130933":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/3952607648":"2024-04-22T08:28:57.384403+0000","172.21.15.86:6801/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/2756903426":"2024-04-22T08:29:23.190001+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:39:00.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.485+0000 7efe64375640 1 -- 172.21.15.86:0/3744683880 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efe480a3e30 msgr2=0x7efe480a62f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.485+0000 7efe64375640 1 --2- 172.21.15.86:0/3744683880 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efe480a3e30 0x7efe480a62f0 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7efe5c077770 tx=0x7efe58005f90 comp rx=0 tx=0).stop 2024-04-21T08:39:00.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.485+0000 7efe64375640 1 -- 172.21.15.86:0/3744683880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe5c105e90 msgr2=0x7efe5c19cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.485+0000 7efe64375640 1 --2- 172.21.15.86:0/3744683880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe5c105e90 0x7efe5c19cfe0 secure :-1 s=READY pgs=606 cs=0 l=1 rev1=1 crypto rx=0x7efe5400eb10 tx=0x7efe5400c2d0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.485+0000 7efe64375640 1 -- 172.21.15.86:0/3744683880 shutdown_connections 2024-04-21T08:39:00.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.485+0000 7efe64375640 1 --2- 172.21.15.86:0/3744683880 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efe480a3e30 0x7efe480a62f0 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.485+0000 7efe64375640 1 --2- 172.21.15.86:0/3744683880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe5c105e90 0x7efe5c19cfe0 unknown :-1 s=CLOSED pgs=606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.485+0000 7efe64375640 1 --2- 172.21.15.86:0/3744683880 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe5c1023c0 0x7efe5c198cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.485+0000 7efe64375640 1 --2- 172.21.15.86:0/3744683880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe5c101a00 0x7efe5c198770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.485+0000 7efe64375640 1 -- 172.21.15.86:0/3744683880 >> 172.21.15.86:0/3744683880 conn(0x7efe5c076170 msgr2=0x7efe5c100a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:00.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.485+0000 7efe64375640 1 -- 172.21.15.86:0/3744683880 shutdown_connections 2024-04-21T08:39:00.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.485+0000 7efe64375640 1 -- 172.21.15.86:0/3744683880 wait complete. 2024-04-21T08:39:00.495 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:00 smithi086 ceph-mon[27939]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 54 op/s 2024-04-21T08:39:00.495 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:00 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3235256692' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:39:00.495 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:00 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1163078857' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:39:00.495 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:00 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1551884384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:39:00.496 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:39:00 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:39:00.200+0000 7faf37210700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:39:00.496 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:39:00 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:39:00.200+0000 7faf37210700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:39:00.496 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:39:00 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:39:00.200+0000 7faf37210700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:39:00.496 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:39:00 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:39:00.200+0000 7faf37210700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:39:00.496 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:39:00 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:39:00.200+0000 7faf37210700 -1 client.0 error registering admin socket command: (17) File exists 2024-04-21T08:39:00.497 INFO:tasks.quiescer.fs.[cephfs]:Initializing Quiescer with random seed 684631 2024-04-21T08:39:00.498 DEBUG:tasks.quiescer:Yielding 2024-04-21T08:39:00.498 INFO:teuthology.run_tasks:Running task workunit... 2024-04-21T08:39:00.512 INFO:tasks.workunit:Pulling workunits from ref 4f6bc5273643aaf5660909e85400647a8951c10b 2024-04-21T08:39:00.512 INFO:tasks.workunit:Making a separate scratch dir for every client... 2024-04-21T08:39:00.512 INFO:tasks.workunit:timeout=3h 2024-04-21T08:39:00.512 INFO:tasks.workunit:cleanup=True 2024-04-21T08:39:00.512 DEBUG:teuthology.orchestra.run.smithi086:> stat -- /home/ubuntu/cephtest/mnt.0 2024-04-21T08:39:00.513 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:39:00.528 INFO:teuthology.orchestra.run.smithi086.stdout: File: /home/ubuntu/cephtest/mnt.0 2024-04-21T08:39:00.528 INFO:teuthology.orchestra.run.smithi086.stdout: Size: 0 Blocks: 0 IO Block: 65536 directory 2024-04-21T08:39:00.528 INFO:teuthology.orchestra.run.smithi086.stdout:Device: 73h/115d Inode: 1099511628283 Links: 2 2024-04-21T08:39:00.528 INFO:teuthology.orchestra.run.smithi086.stdout:Access: (1777/drwxrwxrwt) Uid: ( 0/ root) Gid: ( 0/ root) 2024-04-21T08:39:00.528 INFO:teuthology.orchestra.run.smithi086.stdout:Context: system_u:object_r:unlabeled_t:s0 2024-04-21T08:39:00.528 INFO:teuthology.orchestra.run.smithi086.stdout:Access: 2024-04-21 08:36:47.840882086 +0000 2024-04-21T08:39:00.528 INFO:teuthology.orchestra.run.smithi086.stdout:Modify: 2024-04-21 08:36:47.840882086 +0000 2024-04-21T08:39:00.529 INFO:teuthology.orchestra.run.smithi086.stdout:Change: 2024-04-21 08:38:44.471147508 +0000 2024-04-21T08:39:00.529 INFO:teuthology.orchestra.run.smithi086.stdout: Birth: 2024-04-21 08:36:47.840882086 +0000 2024-04-21T08:39:00.529 INFO:tasks.workunit:Did not need to create dir /home/ubuntu/cephtest/mnt.0 2024-04-21T08:39:00.529 DEBUG:teuthology.orchestra.run.smithi086:> cd -- /home/ubuntu/cephtest/mnt.0 && sudo install -d -m 0755 --owner=ubuntu -- client.0 2024-04-21T08:39:00.600 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/leonid-s-usov/ceph-ceph /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 4f6bc5273643aaf5660909e85400647a8951c10b 2024-04-21T08:39:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.614+0000 7f7440fb1640 1 -- 172.21.15.86:0/2015906832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f743c105e30 msgr2=0x7f743c106210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.614+0000 7f7440fb1640 1 --2- 172.21.15.86:0/2015906832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f743c105e30 0x7f743c106210 secure :-1 s=READY pgs=607 cs=0 l=1 rev1=1 crypto rx=0x7f742c009930 tx=0x7f742c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.614+0000 7f7440fb1640 1 -- 172.21.15.86:0/2015906832 shutdown_connections 2024-04-21T08:39:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.614+0000 7f7440fb1640 1 --2- 172.21.15.86:0/2015906832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f743c101b60 0x7f743c10dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.614+0000 7f7440fb1640 1 --2- 172.21.15.86:0/2015906832 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f743c1011a0 0x7f743c101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.614+0000 7f7440fb1640 1 --2- 172.21.15.86:0/2015906832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f743c105e30 0x7f743c106210 unknown :-1 s=CLOSED pgs=607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.614+0000 7f7440fb1640 1 -- 172.21.15.86:0/2015906832 >> 172.21.15.86:0/2015906832 conn(0x7f743c077e30 msgr2=0x7f743c078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.614+0000 7f7440fb1640 1 -- 172.21.15.86:0/2015906832 shutdown_connections 2024-04-21T08:39:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.614+0000 7f7440fb1640 1 -- 172.21.15.86:0/2015906832 wait complete. 2024-04-21T08:39:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.615+0000 7f7440fb1640 1 Processor -- start 2024-04-21T08:39:00.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.615+0000 7f7440fb1640 1 -- start start 2024-04-21T08:39:00.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.615+0000 7f7440fb1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f743c1011a0 0x7f743c197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.615+0000 7f7440fb1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f743c101b60 0x7f743c197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.615+0000 7f7440fb1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f743c105e30 0x7f743c19bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.615+0000 7f7440fb1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f743c1110e0 con 0x7f743c101b60 2024-04-21T08:39:00.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.615+0000 7f7440fb1640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f743c110f60 con 0x7f743c105e30 2024-04-21T08:39:00.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.615+0000 7f7440fb1640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f743c111260 con 0x7f743c1011a0 2024-04-21T08:39:00.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.615+0000 7f7439d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f743c101b60 0x7f743c197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.615+0000 7f7439d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f743c101b60 0x7f743c197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45834/0 (socket says 172.21.15.86:45834) 2024-04-21T08:39:00.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.615+0000 7f7439d9b640 1 -- 172.21.15.86:0/2188911756 learned_addr learned my addr 172.21.15.86:0/2188911756 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:00.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.615+0000 7f743a59c640 1 --2- 172.21.15.86:0/2188911756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f743c1011a0 0x7f743c197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.615+0000 7f743ad9d640 1 --2- 172.21.15.86:0/2188911756 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f743c105e30 0x7f743c19bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.615+0000 7f7439d9b640 1 -- 172.21.15.86:0/2188911756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f743c1011a0 msgr2=0x7f743c197640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.615+0000 7f7439d9b640 1 --2- 172.21.15.86:0/2188911756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f743c1011a0 0x7f743c197640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.616+0000 7f7439d9b640 1 -- 172.21.15.86:0/2188911756 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f743c105e30 msgr2=0x7f743c19bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.616+0000 7f7439d9b640 1 --2- 172.21.15.86:0/2188911756 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f743c105e30 0x7f743c19bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.616+0000 7f7439d9b640 1 -- 172.21.15.86:0/2188911756 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f743c19c630 con 0x7f743c101b60 2024-04-21T08:39:00.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.616+0000 7f7439d9b640 1 --2- 172.21.15.86:0/2188911756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f743c101b60 0x7f743c197b80 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f743000d950 tx=0x7f743000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:00.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.616+0000 7f742b7fe640 1 -- 172.21.15.86:0/2188911756 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7430017070 con 0x7f743c101b60 2024-04-21T08:39:00.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.616+0000 7f742b7fe640 1 -- 172.21.15.86:0/2188911756 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7430009c30 con 0x7f743c101b60 2024-04-21T08:39:00.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.616+0000 7f7440fb1640 1 -- 172.21.15.86:0/2188911756 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f743c19c920 con 0x7f743c101b60 2024-04-21T08:39:00.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.616+0000 7f742b7fe640 1 -- 172.21.15.86:0/2188911756 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f743000f620 con 0x7f743c101b60 2024-04-21T08:39:00.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.616+0000 7f7440fb1640 1 -- 172.21.15.86:0/2188911756 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f743c1a41b0 con 0x7f743c101b60 2024-04-21T08:39:00.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.617+0000 7f7440fb1640 1 -- 172.21.15.86:0/2188911756 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f743c102570 con 0x7f743c101b60 2024-04-21T08:39:00.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.618+0000 7f742b7fe640 1 -- 172.21.15.86:0/2188911756 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f7430010460 con 0x7f743c101b60 2024-04-21T08:39:00.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.619+0000 7f742b7fe640 1 --2- 172.21.15.86:0/2188911756 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f74240a3db0 0x7f74240a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:00.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.619+0000 7f743a59c640 1 --2- 172.21.15.86:0/2188911756 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f74240a3db0 0x7f74240a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:00.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.619+0000 7f742b7fe640 1 -- 172.21.15.86:0/2188911756 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f74300d08c0 con 0x7f743c101b60 2024-04-21T08:39:00.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.619+0000 7f743a59c640 1 --2- 172.21.15.86:0/2188911756 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f74240a3db0 0x7f74240a6270 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f742c005ab0 tx=0x7f742c005fd0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:00.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.624+0000 7f742b7fe640 1 -- 172.21.15.86:0/2188911756 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7430009de0 con 0x7f743c101b60 2024-04-21T08:39:00.629 INFO:tasks.workunit.client.0.smithi086.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2024-04-21T08:39:00.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:00 smithi088 ceph-mon[31259]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 54 op/s 2024-04-21T08:39:00.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:00 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3235256692' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:39:00.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:00 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1163078857' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:39:00.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:00 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1551884384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:39:00.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:00 smithi179 ceph-mon[31829]: pgmap v9: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 83 KiB/s wr, 54 op/s 2024-04-21T08:39:00.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:00 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3235256692' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:39:00.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:00 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1163078857' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:39:00.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:00 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1551884384' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:39:00.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.846+0000 7f7440fb1640 1 -- 172.21.15.86:0/2188911756 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f743c109390 con 0x7f743c101b60 2024-04-21T08:39:00.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.847+0000 7f742b7fe640 1 -- 172.21.15.86:0/2188911756 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7f74300b0e20 con 0x7f743c101b60 2024-04-21T08:39:00.847 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:39:00.847 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:38:45.202126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24398,24394,24455,24472]},"id":1}]} 2024-04-21T08:39:00.848 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-21T08:39:00.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.849+0000 7f7440fb1640 1 -- 172.21.15.86:0/2188911756 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f74240a3db0 msgr2=0x7f74240a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.849+0000 7f7440fb1640 1 --2- 172.21.15.86:0/2188911756 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f74240a3db0 0x7f74240a6270 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f742c005ab0 tx=0x7f742c005fd0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.849+0000 7f7440fb1640 1 -- 172.21.15.86:0/2188911756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f743c101b60 msgr2=0x7f743c197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:00.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.849+0000 7f7440fb1640 1 --2- 172.21.15.86:0/2188911756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f743c101b60 0x7f743c197b80 secure :-1 s=READY pgs=608 cs=0 l=1 rev1=1 crypto rx=0x7f743000d950 tx=0x7f743000de10 comp rx=0 tx=0).stop 2024-04-21T08:39:00.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.849+0000 7f7440fb1640 1 -- 172.21.15.86:0/2188911756 shutdown_connections 2024-04-21T08:39:00.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.849+0000 7f7440fb1640 1 --2- 172.21.15.86:0/2188911756 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f74240a3db0 0x7f74240a6270 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.849+0000 7f7440fb1640 1 --2- 172.21.15.86:0/2188911756 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f743c105e30 0x7f743c19bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.849+0000 7f7440fb1640 1 --2- 172.21.15.86:0/2188911756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f743c101b60 0x7f743c197b80 unknown :-1 s=CLOSED pgs=608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.849+0000 7f7440fb1640 1 --2- 172.21.15.86:0/2188911756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f743c1011a0 0x7f743c197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:00.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.849+0000 7f7440fb1640 1 -- 172.21.15.86:0/2188911756 >> 172.21.15.86:0/2188911756 conn(0x7f743c077e30 msgr2=0x7f743c0fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:00.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.849+0000 7f7440fb1640 1 -- 172.21.15.86:0/2188911756 shutdown_connections 2024-04-21T08:39:00.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:00.849+0000 7f7440fb1640 1 -- 172.21.15.86:0/2188911756 wait complete. 2024-04-21T08:39:00.861 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: mds map: {'epoch': 29, 'flags': 306, 'flags_state': {'joinable': True, 'allow_snaps': True, 'allow_multimds_snaps': True, 'allow_standby_replay': True, 'refuse_client_session': False, 'refuse_standby_for_another_fs': False, 'balance_automate': True}, 'ever_allowed_features': 32, 'explicitly_allowed_features': 32, 'created': '2024-04-21T08:36:20.626894+0000', 'modified': '2024-04-21T08:38:45.202126+0000', 'tableserver': 0, 'root': 0, 'session_timeout': 300, 'session_autoclose': 300, 'required_client_features': {}, 'max_file_size': 1099511627776, 'max_xattr_size': 65536, 'last_failure': 0, 'last_failure_osd_epoch': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}, 'max_mds': 5, 'in': [0, 1, 2, 3, 4], 'up': {'mds_0': 24482, 'mds_1': 24472, 'mds_2': 24455, 'mds_3': 24398, 'mds_4': 24394}, 'failed': [], 'damaged': [], 'stopped': [], 'info': {'gid_14640': {'gid': 14640, 'name': 'g', 'rank': 4, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6839/3163123023', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6838', 'nonce': 3163123023}, {'type': 'v1', 'addr': '172.21.15.86:6839', 'nonce': 3163123023}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_14652': {'gid': 14652, 'name': 'a', 'rank': 3, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.86:6841/3537972023', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.86:6840', 'nonce': 3537972023}, {'type': 'v1', 'addr': '172.21.15.86:6841', 'nonce': 3537972023}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24394': {'gid': 24394, 'name': 'l', 'rank': 4, 'incarnation': 26, 'state': 'up:active', 'state_seq': 17, 'addr': '172.21.15.179:6833/3606421532', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6832', 'nonce': 3606421532}, {'type': 'v1', 'addr': '172.21.15.179:6833', 'nonce': 3606421532}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24398': {'gid': 24398, 'name': 'k', 'rank': 3, 'incarnation': 24, 'state': 'up:active', 'state_seq': 16, 'addr': '172.21.15.88:6833/1872744608', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6832', 'nonce': 1872744608}, {'type': 'v1', 'addr': '172.21.15.88:6833', 'nonce': 1872744608}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24428': {'gid': 24428, 'name': 'e', 'rank': 2, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.88:6835/454792798', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6834', 'nonce': 454792798}, {'type': 'v1', 'addr': '172.21.15.88:6835', 'nonce': 454792798}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24436': {'gid': 24436, 'name': 'f', 'rank': 1, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.179:6835/3343050063', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6834', 'nonce': 3343050063}, {'type': 'v1', 'addr': '172.21.15.179:6835', 'nonce': 3343050063}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24448': {'gid': 24448, 'name': 'c', 'rank': 0, 'incarnation': 0, 'state': 'up:standby-replay', 'state_seq': 1, 'addr': '172.21.15.179:6837/1280529894', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6836', 'nonce': 1280529894}, {'type': 'v1', 'addr': '172.21.15.179:6837', 'nonce': 1280529894}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24455': {'gid': 24455, 'name': 'h', 'rank': 2, 'incarnation': 22, 'state': 'up:active', 'state_seq': 12, 'addr': '172.21.15.88:6837/1950568471', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6836', 'nonce': 1950568471}, {'type': 'v1', 'addr': '172.21.15.88:6837', 'nonce': 1950568471}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24472': {'gid': 24472, 'name': 'i', 'rank': 1, 'incarnation': 20, 'state': 'up:active', 'state_seq': 11, 'addr': '172.21.15.179:6839/3029310820', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.179:6838', 'nonce': 3029310820}, {'type': 'v1', 'addr': '172.21.15.179:6839', 'nonce': 3029310820}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}, 'gid_24482': {'gid': 24482, 'name': 'b', 'rank': 0, 'incarnation': 14, 'state': 'up:active', 'state_seq': 3, 'addr': '172.21.15.88:6839/3969608142', 'addrs': {'addrvec': [{'type': 'v2', 'addr': '172.21.15.88:6838', 'nonce': 3969608142}, {'type': 'v1', 'addr': '172.21.15.88:6839', 'nonce': 3969608142}]}, 'join_fscid': -1, 'export_targets': [], 'features': 4540701547738038271, 'flags': 0, 'compat': {'compat': {}, 'ro_compat': {}, 'incompat': {'feature_1': 'base v0.20', 'feature_2': 'client writeable ranges', 'feature_3': 'default file layouts on dirs', 'feature_4': 'dir inode in separate object', 'feature_5': 'mds uses versioned encoding', 'feature_6': 'dirfrag is stored in omap', 'feature_7': 'mds uses inline data', 'feature_8': 'no anchor table', 'feature_9': 'file layout v2', 'feature_10': 'snaprealm v2', 'feature_11': 'minor log segments', 'feature_12': 'quiesce subvolumes'}}}}, 'data_pools': [3, 4], 'metadata_pool': 2, 'enabled': True, 'fs_name': 'cephfs', 'balancer': '', 'bal_rank_mask': '-1', 'standby_count_wanted': 0, 'qdb_leader': 24482, 'qdb_cluster': [24482, 24398, 24394, 24455, 24472]} 2024-04-21T08:39:00.861 DEBUG:tasks.cephfs.filesystem:are_daemons_healthy: 5/5 2024-04-21T08:39:00.861 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.l status 2024-04-21T08:39:01.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.002+0000 7fd25c5db640 1 -- 172.21.15.86:0/2467256271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2541031d0 msgr2=0x7fd2541035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.002+0000 7fd25c5db640 1 --2- 172.21.15.86:0/2467256271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2541031d0 0x7fd2541035b0 secure :-1 s=READY pgs=609 cs=0 l=1 rev1=1 crypto rx=0x7fd250009900 tx=0x7fd25002bd60 comp rx=0 tx=0).stop 2024-04-21T08:39:01.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.002+0000 7fd25c5db640 1 -- 172.21.15.86:0/2467256271 shutdown_connections 2024-04-21T08:39:01.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.002+0000 7fd25c5db640 1 --2- 172.21.15.86:0/2467256271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd25410c510 0x7fd25410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.002+0000 7fd25c5db640 1 --2- 172.21.15.86:0/2467256271 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd254103af0 0x7fd25410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.002+0000 7fd25c5db640 1 --2- 172.21.15.86:0/2467256271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2541031d0 0x7fd2541035b0 unknown :-1 s=CLOSED pgs=609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.002+0000 7fd25c5db640 1 -- 172.21.15.86:0/2467256271 >> 172.21.15.86:0/2467256271 conn(0x7fd254077f50 msgr2=0x7fd254078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:01.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.002+0000 7fd25c5db640 1 -- 172.21.15.86:0/2467256271 shutdown_connections 2024-04-21T08:39:01.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.002+0000 7fd25c5db640 1 -- 172.21.15.86:0/2467256271 wait complete. 2024-04-21T08:39:01.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.002+0000 7fd25c5db640 1 Processor -- start 2024-04-21T08:39:01.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.002+0000 7fd25c5db640 1 -- start start 2024-04-21T08:39:01.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.003+0000 7fd25c5db640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2541031d0 0x7fd254198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.003+0000 7fd25c5db640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd254103af0 0x7fd254198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.003+0000 7fd25c5db640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd25410c510 0x7fd25419d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.003+0000 7fd25c5db640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd2541121a0 con 0x7fd254103af0 2024-04-21T08:39:01.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.003+0000 7fd25c5db640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd254112020 con 0x7fd2541031d0 2024-04-21T08:39:01.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.003+0000 7fd25c5db640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd254112320 con 0x7fd25410c510 2024-04-21T08:39:01.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.003+0000 7fd259b76640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd254103af0 0x7fd254198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.003+0000 7fd259b76640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd254103af0 0x7fd254198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45848/0 (socket says 172.21.15.86:45848) 2024-04-21T08:39:01.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.003+0000 7fd259b76640 1 -- 172.21.15.86:0/345418349 learned_addr learned my addr 172.21.15.86:0/345418349 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:01.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.003+0000 7fd25a377640 1 --2- 172.21.15.86:0/345418349 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2541031d0 0x7fd254198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.003+0000 7fd25ab78640 1 --2- 172.21.15.86:0/345418349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd25410c510 0x7fd25419d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.003+0000 7fd259b76640 1 -- 172.21.15.86:0/345418349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd25410c510 msgr2=0x7fd25419d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.003+0000 7fd259b76640 1 --2- 172.21.15.86:0/345418349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd25410c510 0x7fd25419d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.003+0000 7fd259b76640 1 -- 172.21.15.86:0/345418349 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2541031d0 msgr2=0x7fd254198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.003+0000 7fd259b76640 1 --2- 172.21.15.86:0/345418349 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2541031d0 0x7fd254198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.003+0000 7fd259b76640 1 -- 172.21.15.86:0/345418349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd25419d730 con 0x7fd254103af0 2024-04-21T08:39:01.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.003+0000 7fd25a377640 1 --2- 172.21.15.86:0/345418349 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2541031d0 0x7fd254198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:01.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.003+0000 7fd25ab78640 1 --2- 172.21.15.86:0/345418349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd25410c510 0x7fd25419d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:01.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.003+0000 7fd259b76640 1 --2- 172.21.15.86:0/345418349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd254103af0 0x7fd254198c80 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fd24400d770 tx=0x7fd24400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:01.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.003+0000 7fd24b7fe640 1 -- 172.21.15.86:0/345418349 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd244017070 con 0x7fd254103af0 2024-04-21T08:39:01.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.003+0000 7fd25c5db640 1 -- 172.21.15.86:0/345418349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd25419da20 con 0x7fd254103af0 2024-04-21T08:39:01.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.003+0000 7fd24b7fe640 1 -- 172.21.15.86:0/345418349 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2440054d0 con 0x7fd254103af0 2024-04-21T08:39:01.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.003+0000 7fd24b7fe640 1 -- 172.21.15.86:0/345418349 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd244004c30 con 0x7fd254103af0 2024-04-21T08:39:01.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.003+0000 7fd25c5db640 1 -- 172.21.15.86:0/345418349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2541a52b0 con 0x7fd254103af0 2024-04-21T08:39:01.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.006+0000 7fd24b7fe640 1 -- 172.21.15.86:0/345418349 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7fd244004e10 con 0x7fd254103af0 2024-04-21T08:39:01.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.006+0000 7fd25f1ce740 1 Processor -- start 2024-04-21T08:39:01.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.006+0000 7fd25f1ce740 1 -- start start 2024-04-21T08:39:01.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.006+0000 7fd25f1ce740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af012347b0 0x55af01234b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.006+0000 7fd25f1ce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af01235160 0x55af0123d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.006+0000 7fd25f1ce740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55af0123dce0 0x55af01240130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.006+0000 7fd25f1ce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55af01198b10 con 0x55af01235160 2024-04-21T08:39:01.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.006+0000 7fd25f1ce740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55af01136240 con 0x55af0123dce0 2024-04-21T08:39:01.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.006+0000 7fd25f1ce740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55af01194ad0 con 0x55af012347b0 2024-04-21T08:39:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.007+0000 7fd25a377640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af01235160 0x55af0123d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.007+0000 7fd25a377640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af01235160 0x55af0123d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45862/0 (socket says 172.21.15.86:45862) 2024-04-21T08:39:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.007+0000 7fd25a377640 1 -- 172.21.15.86:0/619636851 learned_addr learned my addr 172.21.15.86:0/619636851 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.007+0000 7fd24b7fe640 1 --2- 172.21.15.86:0/345418349 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd2400a3db0 0x7fd2400a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.007+0000 7fd259b76640 1 --2- 172.21.15.86:0/619636851 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55af0123dce0 0x55af01240130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.007+0000 7fd25ab78640 1 --2- 172.21.15.86:0/619636851 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af012347b0 0x55af01234b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.007+0000 7fd25ab78640 1 --2- 172.21.15.86:0/345418349 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd2400a3db0 0x7fd2400a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.007+0000 7fd24b7fe640 1 -- 172.21.15.86:0/345418349 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fd2440d0890 con 0x7fd254103af0 2024-04-21T08:39:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.007+0000 7fd25a377640 1 -- 172.21.15.86:0/619636851 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af012347b0 msgr2=0x55af01234b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.007+0000 7fd25a377640 1 --2- 172.21.15.86:0/619636851 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af012347b0 0x55af01234b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.007+0000 7fd25a377640 1 -- 172.21.15.86:0/619636851 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55af0123dce0 msgr2=0x55af01240130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.007+0000 7fd25a377640 1 --2- 172.21.15.86:0/619636851 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55af0123dce0 0x55af01240130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.007+0000 7fd25a377640 1 -- 172.21.15.86:0/619636851 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af00eab2d0 con 0x55af01235160 2024-04-21T08:39:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.007+0000 7fd259b76640 1 --2- 172.21.15.86:0/619636851 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55af0123dce0 0x55af01240130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.007+0000 7fd25a377640 1 --2- 172.21.15.86:0/619636851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af01235160 0x55af0123d710 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd250009900 tx=0x7fd250005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb08efac833e5096 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.007+0000 7fd25ab78640 1 --2- 172.21.15.86:0/345418349 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd2400a3db0 0x7fd2400a6270 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd24c005c40 tx=0x7fd24c009200 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.007+0000 7fd248bfe640 1 -- 172.21.15.86:0/619636851 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2500042c0 con 0x55af01235160 2024-04-21T08:39:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.007+0000 7fd248bfe640 1 -- 172.21.15.86:0/619636851 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd250004460 con 0x55af01235160 2024-04-21T08:39:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.007+0000 7fd248bfe640 1 -- 172.21.15.86:0/619636851 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd250004920 con 0x55af01235160 2024-04-21T08:39:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.008+0000 7fd25f1ce740 1 -- 172.21.15.86:0/619636851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af01235160 msgr2=0x55af0123d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.008+0000 7fd25f1ce740 1 --2- 172.21.15.86:0/619636851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af01235160 0x55af0123d710 secure :-1 s=READY pgs=611 cs=0 l=1 rev1=1 crypto rx=0x7fd250009900 tx=0x7fd250005e80 comp rx=0 tx=0).stop 2024-04-21T08:39:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.008+0000 7fd25f1ce740 1 -- 172.21.15.86:0/619636851 shutdown_connections 2024-04-21T08:39:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.008+0000 7fd25f1ce740 1 --2- 172.21.15.86:0/619636851 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55af0123dce0 0x55af01240130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.008+0000 7fd25f1ce740 1 --2- 172.21.15.86:0/619636851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af01235160 0x55af0123d710 unknown :-1 s=CLOSED pgs=611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.008+0000 7fd25f1ce740 1 --2- 172.21.15.86:0/619636851 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af012347b0 0x55af01234b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.008+0000 7fd25f1ce740 1 -- 172.21.15.86:0/619636851 >> 172.21.15.86:0/619636851 conn(0x55af0122ea60 msgr2=0x55af0122ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.008+0000 7fd25f1ce740 1 -- 172.21.15.86:0/619636851 shutdown_connections 2024-04-21T08:39:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.008+0000 7fd25f1ce740 1 -- 172.21.15.86:0/619636851 wait complete. 2024-04-21T08:39:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.009+0000 7fd25f1ce740 1 -- start start 2024-04-21T08:39:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.009+0000 7fd25f1ce740 1 Processor -- start 2024-04-21T08:39:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.009+0000 7fd25f1ce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af012347b0 0x55af0123ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.009+0000 7fd25f1ce740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55af01235160 0x55af0123b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.009+0000 7fd25f1ce740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af0123dce0 0x55af0123b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.009+0000 7fd25f1ce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55af01240bf0 con 0x55af012347b0 2024-04-21T08:39:01.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.009+0000 7fd25f1ce740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55af01121190 con 0x55af01235160 2024-04-21T08:39:01.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.009+0000 7fd25f1ce740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55af01240d70 con 0x55af0123dce0 2024-04-21T08:39:01.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.009+0000 7fd25f1ce740 10 client.0 _ll_register_callbacks cb 0x55af0111c8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:01.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.009+0000 7fd25a377640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af012347b0 0x55af0123ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.009+0000 7fd25a377640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af012347b0 0x55af0123ac00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45872/0 (socket says 172.21.15.86:45872) 2024-04-21T08:39:01.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.009+0000 7fd25a377640 1 -- 172.21.15.86:0/1954508340 learned_addr learned my addr 172.21.15.86:0/1954508340 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:01.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.009+0000 7fd25ab78640 1 --2- 172.21.15.86:0/1954508340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af0123dce0 0x55af0123b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.009+0000 7fd259b76640 1 --2- 172.21.15.86:0/1954508340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55af01235160 0x55af0123b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.009+0000 7fd25a377640 1 -- 172.21.15.86:0/1954508340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af0123dce0 msgr2=0x55af0123b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.009+0000 7fd25a377640 1 --2- 172.21.15.86:0/1954508340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af0123dce0 0x55af0123b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.009+0000 7fd25a377640 1 -- 172.21.15.86:0/1954508340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55af01235160 msgr2=0x55af0123b140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.009+0000 7fd25a377640 1 --2- 172.21.15.86:0/1954508340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55af01235160 0x55af0123b140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.009+0000 7fd25a377640 1 -- 172.21.15.86:0/1954508340 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af00f756b0 con 0x55af012347b0 2024-04-21T08:39:01.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.009+0000 7fd25a377640 1 --2- 172.21.15.86:0/1954508340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af012347b0 0x55af0123ac00 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd250000c00 tx=0x7fd250004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84bb61a0c121ad96 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:01.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.009+0000 7fd23dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:01.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.010+0000 7fd23dffb640 1 -- 172.21.15.86:0/1954508340 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd250034650 con 0x55af012347b0 2024-04-21T08:39:01.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.010+0000 7fd23dffb640 1 -- 172.21.15.86:0/1954508340 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fd2500347f0 con 0x55af012347b0 2024-04-21T08:39:01.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.010+0000 7fd25f1ce740 1 -- 172.21.15.86:0/1954508340 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd250004460 con 0x55af012347b0 2024-04-21T08:39:01.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.010+0000 7fd23dffb640 1 -- 172.21.15.86:0/1954508340 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd2500349f0 con 0x55af012347b0 2024-04-21T08:39:01.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.010+0000 7fd25f1ce740 10 client.15963 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:01.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.010+0000 7fd25f1ce740 1 -- 172.21.15.86:0/1954508340 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55af0124ce40 con 0x55af012347b0 2024-04-21T08:39:01.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.010+0000 7fd23dffb640 1 -- 172.21.15.86:0/1954508340 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd250004460 con 0x55af012347b0 2024-04-21T08:39:01.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.010+0000 7fd23dffb640 1 -- 172.21.15.86:0/1954508340 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd250050ce0 con 0x55af012347b0 2024-04-21T08:39:01.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.010+0000 7fd25f1ce740 10 client.15963 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:01.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.010+0000 7fd25f1ce740 10 client.15963 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-21T08:39:01.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.010+0000 7fd25f1ce740 20 client.15963 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:01.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.010+0000 7fd25f1ce740 1 --2- 172.21.15.86:0/1954508340 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55af0124ef50 0x55af012503c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.010+0000 7fd25f1ce740 4 client.15963 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:01.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.010+0000 7fd25f1ce740 1 -- 172.21.15.86:0/1954508340 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55af011a6530 con 0x55af0124ef50 2024-04-21T08:39:01.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.010+0000 7fd259b76640 1 --2- 172.21.15.86:0/1954508340 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55af0124ef50 0x55af012503c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.011+0000 7fd259b76640 1 --2- 172.21.15.86:0/1954508340 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55af0124ef50 0x55af012503c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=341f1a6afb831cb1 server_cookie=e84c3c36a8dd0bab in_seq=0 out_seq=0 2024-04-21T08:39:01.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.011+0000 7fd23dffb640 10 client.15963 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:39:01.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.012+0000 7fd23dffb640 1 -- 172.21.15.86:0/1954508340 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55af011a6530 con 0x55af0124ef50 2024-04-21T08:39:01.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.012+0000 7fd23dffb640 10 client.15963 handle_command_reply: tid=0 2024-04-21T08:39:01.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.012+0000 7fd25f1ce740 1 client.15963 shutdown 2024-04-21T08:39:01.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.012+0000 7fd25f1ce740 1 -- 172.21.15.86:0/1954508340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af012347b0 msgr2=0x55af0123ac00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.012+0000 7fd25f1ce740 1 --2- 172.21.15.86:0/1954508340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af012347b0 0x55af0123ac00 secure :-1 s=READY pgs=612 cs=0 l=1 rev1=1 crypto rx=0x7fd250000c00 tx=0x7fd250004610 comp rx=0 tx=0).stop 2024-04-21T08:39:01.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.012+0000 7fd25f1ce740 1 -- 172.21.15.86:0/1954508340 shutdown_connections 2024-04-21T08:39:01.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.012+0000 7fd25f1ce740 1 --2- 172.21.15.86:0/1954508340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af0123dce0 0x55af0123b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.012+0000 7fd25f1ce740 1 --2- 172.21.15.86:0/1954508340 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55af0124ef50 0x55af012503c0 crc :-1 s=READY pgs=32 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.012+0000 7fd25f1ce740 1 --2- 172.21.15.86:0/1954508340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55af01235160 0x55af0123b140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.012+0000 7fd25f1ce740 1 --2- 172.21.15.86:0/1954508340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af012347b0 0x55af0123ac00 unknown :-1 s=CLOSED pgs=612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.012+0000 7fd23dffb640 0 client.15963 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:39:01.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.012+0000 7fd25f1ce740 1 -- 172.21.15.86:0/1954508340 >> 172.21.15.86:0/1954508340 conn(0x55af0122ea60 msgr2=0x55af01230bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:01.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.012+0000 7fd25f1ce740 1 -- 172.21.15.86:0/1954508340 shutdown_connections 2024-04-21T08:39:01.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.012+0000 7fd25f1ce740 1 -- 172.21.15.86:0/1954508340 wait complete. 2024-04-21T08:39:01.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.012+0000 7fd25f1ce740 20 client.15963 trim_cache size 0 max 16384 2024-04-21T08:39:01.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.030+0000 7fd25f1ce740 1 Processor -- start 2024-04-21T08:39:01.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.030+0000 7fd25f1ce740 1 -- start start 2024-04-21T08:39:01.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.030+0000 7fd25f1ce740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55af012347b0 0x55af0123d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.030+0000 7fd25f1ce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af01235160 0x55af0124e880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.030+0000 7fd25f1ce740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af0123dce0 0x55af0123b150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.030+0000 7fd25f1ce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55af0124e700 con 0x55af01235160 2024-04-21T08:39:01.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.030+0000 7fd25f1ce740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55af0124edd0 con 0x55af012347b0 2024-04-21T08:39:01.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.030+0000 7fd25f1ce740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55af0124f4a0 con 0x55af0123dce0 2024-04-21T08:39:01.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.030+0000 7fd259b76640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af01235160 0x55af0124e880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.030+0000 7fd259b76640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af01235160 0x55af0124e880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45876/0 (socket says 172.21.15.86:45876) 2024-04-21T08:39:01.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.030+0000 7fd259b76640 1 -- 172.21.15.86:0/859219036 learned_addr learned my addr 172.21.15.86:0/859219036 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:01.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.030+0000 7fd25a377640 1 --2- 172.21.15.86:0/859219036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55af012347b0 0x55af0123d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.030+0000 7fd25ab78640 1 --2- 172.21.15.86:0/859219036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af0123dce0 0x55af0123b150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.030+0000 7fd259b76640 1 -- 172.21.15.86:0/859219036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af0123dce0 msgr2=0x55af0123b150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.030+0000 7fd259b76640 1 --2- 172.21.15.86:0/859219036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af0123dce0 0x55af0123b150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.030+0000 7fd259b76640 1 -- 172.21.15.86:0/859219036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55af012347b0 msgr2=0x55af0123d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.030+0000 7fd259b76640 1 --2- 172.21.15.86:0/859219036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55af012347b0 0x55af0123d330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.030+0000 7fd259b76640 1 -- 172.21.15.86:0/859219036 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55af012367e0 con 0x55af01235160 2024-04-21T08:39:01.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.030+0000 7fd259b76640 1 --2- 172.21.15.86:0/859219036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af01235160 0x55af0124e880 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd24409c000 tx=0x7fd24406dd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cc82c9ec0b2a223 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:01.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.030+0000 7fd23ffff640 1 -- 172.21.15.86:0/859219036 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd244081ae0 con 0x55af01235160 2024-04-21T08:39:01.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.031+0000 7fd23ffff640 1 -- 172.21.15.86:0/859219036 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2440a0700 con 0x55af01235160 2024-04-21T08:39:01.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.031+0000 7fd23ffff640 1 -- 172.21.15.86:0/859219036 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd244020040 con 0x55af01235160 2024-04-21T08:39:01.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.031+0000 7fd25f1ce740 1 -- 172.21.15.86:0/859219036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af01235160 msgr2=0x55af0124e880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.031+0000 7fd25f1ce740 1 --2- 172.21.15.86:0/859219036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af01235160 0x55af0124e880 secure :-1 s=READY pgs=613 cs=0 l=1 rev1=1 crypto rx=0x7fd24409c000 tx=0x7fd24406dd90 comp rx=0 tx=0).stop 2024-04-21T08:39:01.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.031+0000 7fd25f1ce740 1 -- 172.21.15.86:0/859219036 shutdown_connections 2024-04-21T08:39:01.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.031+0000 7fd25f1ce740 1 --2- 172.21.15.86:0/859219036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af0123dce0 0x55af0123b150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.031+0000 7fd25f1ce740 1 --2- 172.21.15.86:0/859219036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af01235160 0x55af0124e880 unknown :-1 s=CLOSED pgs=613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.031+0000 7fd25f1ce740 1 --2- 172.21.15.86:0/859219036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55af012347b0 0x55af0123d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.031+0000 7fd25f1ce740 1 -- 172.21.15.86:0/859219036 >> 172.21.15.86:0/859219036 conn(0x55af0122ea60 msgr2=0x55af01229800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:01.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.031+0000 7fd25f1ce740 1 -- 172.21.15.86:0/859219036 shutdown_connections 2024-04-21T08:39:01.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.031+0000 7fd25f1ce740 1 -- 172.21.15.86:0/859219036 wait complete. 2024-04-21T08:39:01.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.031+0000 7fd25f1ce740 1 -- start start 2024-04-21T08:39:01.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.032+0000 7fd25f1ce740 1 Processor -- start 2024-04-21T08:39:01.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.032+0000 7fd25f1ce740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55af012347b0 0x55af01258330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.032+0000 7fd25f1ce740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af01235160 0x55af01258870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.032+0000 7fd25f1ce740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af0123dce0 0x55af012532e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.032+0000 7fd25f1ce740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55af01250240 con 0x55af01235160 2024-04-21T08:39:01.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.032+0000 7fd25f1ce740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55af0124fb70 con 0x55af012347b0 2024-04-21T08:39:01.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.032+0000 7fd25f1ce740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55af01250910 con 0x55af0123dce0 2024-04-21T08:39:01.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.032+0000 7fd25f1ce740 10 client.0 _ll_register_callbacks cb 0x7fd24c009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:01.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.032+0000 7fd259b76640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af01235160 0x55af01258870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.032+0000 7fd259b76640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af01235160 0x55af01258870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45886/0 (socket says 172.21.15.86:45886) 2024-04-21T08:39:01.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.032+0000 7fd259b76640 1 -- 172.21.15.86:0/1572898535 learned_addr learned my addr 172.21.15.86:0/1572898535 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:01.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.032+0000 7fd25ab78640 1 --2- 172.21.15.86:0/1572898535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af0123dce0 0x55af012532e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.032+0000 7fd25a377640 1 --2- 172.21.15.86:0/1572898535 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55af012347b0 0x55af01258330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.032+0000 7fd259b76640 1 -- 172.21.15.86:0/1572898535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af0123dce0 msgr2=0x55af012532e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.032+0000 7fd259b76640 1 --2- 172.21.15.86:0/1572898535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af0123dce0 0x55af012532e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.032+0000 7fd259b76640 1 -- 172.21.15.86:0/1572898535 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55af012347b0 msgr2=0x55af01258330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.032+0000 7fd259b76640 1 --2- 172.21.15.86:0/1572898535 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55af012347b0 0x55af01258330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.032+0000 7fd259b76640 1 -- 172.21.15.86:0/1572898535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55af01253820 con 0x55af01235160 2024-04-21T08:39:01.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.032+0000 7fd25ab78640 1 --2- 172.21.15.86:0/1572898535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af0123dce0 0x55af012532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:01.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.032+0000 7fd25a377640 1 --2- 172.21.15.86:0/1572898535 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55af012347b0 0x55af01258330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:01.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.032+0000 7fd259b76640 1 --2- 172.21.15.86:0/1572898535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af01235160 0x55af01258870 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd244021000 tx=0x7fd2440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb656013eb7bc560 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:01.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.032+0000 7fd23dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:01.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.032+0000 7fd23dffb640 1 -- 172.21.15.86:0/1572898535 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd244069070 con 0x55af01235160 2024-04-21T08:39:01.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.032+0000 7fd23dffb640 1 -- 172.21.15.86:0/1572898535 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fd2440989c0 con 0x55af01235160 2024-04-21T08:39:01.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.032+0000 7fd25f1ce740 1 -- 172.21.15.86:0/1572898535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2440a0700 con 0x55af01235160 2024-04-21T08:39:01.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.033+0000 7fd23dffb640 1 -- 172.21.15.86:0/1572898535 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd244020040 con 0x55af01235160 2024-04-21T08:39:01.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.033+0000 7fd25f1ce740 10 client.15969 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:01.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.033+0000 7fd25f1ce740 1 -- 172.21.15.86:0/1572898535 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55af012b55c0 con 0x55af01235160 2024-04-21T08:39:01.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.033+0000 7fd23dffb640 1 -- 172.21.15.86:0/1572898535 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2440a0700 con 0x55af01235160 2024-04-21T08:39:01.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.033+0000 7fd23dffb640 1 -- 172.21.15.86:0/1572898535 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd2440a4bc0 con 0x55af01235160 2024-04-21T08:39:01.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.033+0000 7fd25f1ce740 10 client.15969 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:01.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.033+0000 7fd25f1ce740 10 client.15969 resolve_mds: resolved name 'l' to daemon mds.l 2024-04-21T08:39:01.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.033+0000 7fd25f1ce740 20 client.15969 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:01.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.033+0000 7fd25f1ce740 1 --2- 172.21.15.86:0/1572898535 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55af012b9d70 0x55af012b8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.033+0000 7fd25f1ce740 4 client.15969 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:39:01.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.033+0000 7fd25f1ce740 1 -- 172.21.15.86:0/1572898535 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "status"}) -- 0x55af011a03a0 con 0x55af012b9d70 2024-04-21T08:39:01.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.033+0000 7fd25a377640 1 --2- 172.21.15.86:0/1572898535 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55af012b9d70 0x55af012b8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.034+0000 7fd25a377640 1 --2- 172.21.15.86:0/1572898535 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55af012b9d70 0x55af012b8a80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=58f6f45f9670e3f5 server_cookie=df81e2b192901335 in_seq=0 out_seq=0 2024-04-21T08:39:01.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.034+0000 7fd23dffb640 10 client.15969 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:39:01.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.034+0000 7fd23dffb640 1 -- 172.21.15.86:0/1572898535 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x55af011a03a0 con 0x55af012b9d70 2024-04-21T08:39:01.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.034+0000 7fd23dffb640 10 client.15969 handle_command_reply: tid=0 2024-04-21T08:39:01.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.034+0000 7fd25f1ce740 1 client.15969 shutdown 2024-04-21T08:39:01.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.034+0000 7fd25f1ce740 1 -- 172.21.15.86:0/1572898535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af01235160 msgr2=0x55af01258870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.034+0000 7fd25f1ce740 1 --2- 172.21.15.86:0/1572898535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af01235160 0x55af01258870 secure :-1 s=READY pgs=614 cs=0 l=1 rev1=1 crypto rx=0x7fd244021000 tx=0x7fd2440046c0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.034+0000 7fd25f1ce740 1 -- 172.21.15.86:0/1572898535 shutdown_connections 2024-04-21T08:39:01.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.034+0000 7fd25f1ce740 1 --2- 172.21.15.86:0/1572898535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55af0123dce0 0x55af012532e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.034+0000 7fd25f1ce740 1 --2- 172.21.15.86:0/1572898535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55af01235160 0x55af01258870 unknown :-1 s=CLOSED pgs=614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.034+0000 7fd25f1ce740 1 --2- 172.21.15.86:0/1572898535 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55af012b9d70 0x55af012b8a80 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.034+0000 7fd25f1ce740 1 --2- 172.21.15.86:0/1572898535 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55af012347b0 0x55af01258330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.034+0000 7fd23dffb640 0 client.15969 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:39:01.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.034+0000 7fd25f1ce740 1 -- 172.21.15.86:0/1572898535 >> 172.21.15.86:0/1572898535 conn(0x55af0122ea60 msgr2=0x55af01230120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:01.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.034+0000 7fd25f1ce740 1 -- 172.21.15.86:0/1572898535 shutdown_connections 2024-04-21T08:39:01.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.034+0000 7fd25f1ce740 1 -- 172.21.15.86:0/1572898535 wait complete. 2024-04-21T08:39:01.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.034+0000 7fd25f1ce740 20 client.15969 trim_cache size 0 max 16384 2024-04-21T08:39:01.048 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:39:01.048 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "f8cec7cc-ffb8-11ee-bc93-c7b262605968", 2024-04-21T08:39:01.048 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 4, 2024-04-21T08:39:01.048 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24394, 2024-04-21T08:39:01.048 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-21T08:39:01.049 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-21T08:39:01.049 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-21T08:39:01.049 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 134.59176722699999, 2024-04-21T08:39:01.049 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 29, 2024-04-21T08:39:01.049 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 94, 2024-04-21T08:39:01.049 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 93, 2024-04-21T08:39:01.049 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 197.82014788199999 2024-04-21T08:39:01.049 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:39:01.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.034+0000 7fd25c5db640 1 -- 172.21.15.86:0/345418349 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd2400a3db0 msgr2=0x7fd2400a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.034+0000 7fd25c5db640 1 --2- 172.21.15.86:0/345418349 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd2400a3db0 0x7fd2400a6270 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7fd24c005c40 tx=0x7fd24c009200 comp rx=0 tx=0).stop 2024-04-21T08:39:01.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.034+0000 7fd25c5db640 1 -- 172.21.15.86:0/345418349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd254103af0 msgr2=0x7fd254198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.034+0000 7fd25c5db640 1 --2- 172.21.15.86:0/345418349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd254103af0 0x7fd254198c80 secure :-1 s=READY pgs=610 cs=0 l=1 rev1=1 crypto rx=0x7fd24400d770 tx=0x7fd24400dc30 comp rx=0 tx=0).stop 2024-04-21T08:39:01.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.035+0000 7fd25c5db640 1 -- 172.21.15.86:0/345418349 shutdown_connections 2024-04-21T08:39:01.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.035+0000 7fd25c5db640 1 --2- 172.21.15.86:0/345418349 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd2400a3db0 0x7fd2400a6270 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.035+0000 7fd25c5db640 1 --2- 172.21.15.86:0/345418349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd25410c510 0x7fd25419d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.035+0000 7fd25c5db640 1 --2- 172.21.15.86:0/345418349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd254103af0 0x7fd254198c80 unknown :-1 s=CLOSED pgs=610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.035+0000 7fd25c5db640 1 --2- 172.21.15.86:0/345418349 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2541031d0 0x7fd254198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.035+0000 7fd25c5db640 1 -- 172.21.15.86:0/345418349 >> 172.21.15.86:0/345418349 conn(0x7fd254077f50 msgr2=0x7fd2540fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:01.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.035+0000 7fd25c5db640 1 -- 172.21.15.86:0/345418349 shutdown_connections 2024-04-21T08:39:01.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.035+0000 7fd25c5db640 1 -- 172.21.15.86:0/345418349 wait complete. 2024-04-21T08:39:01.052 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.k status 2024-04-21T08:39:01.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.151+0000 7f6c50ac8640 1 -- 172.21.15.86:0/1628596533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c4c107260 msgr2=0x7f6c4c10ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.151+0000 7f6c50ac8640 1 --2- 172.21.15.86:0/1628596533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c4c107260 0x7f6c4c10ee30 secure :-1 s=READY pgs=615 cs=0 l=1 rev1=1 crypto rx=0x7f6c44009ca0 tx=0x7f6c4402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.151+0000 7f6c50ac8640 1 -- 172.21.15.86:0/1628596533 shutdown_connections 2024-04-21T08:39:01.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.151+0000 7f6c50ac8640 1 --2- 172.21.15.86:0/1628596533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c4c107260 0x7f6c4c10ee30 unknown :-1 s=CLOSED pgs=615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.151+0000 7f6c50ac8640 1 --2- 172.21.15.86:0/1628596533 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c4c102b40 0x7f6c4c106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.151+0000 7f6c50ac8640 1 --2- 172.21.15.86:0/1628596533 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c4c102190 0x7f6c4c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.151+0000 7f6c50ac8640 1 -- 172.21.15.86:0/1628596533 >> 172.21.15.86:0/1628596533 conn(0x7f6c4c0fd990 msgr2=0x7f6c4c0ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:01.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.151+0000 7f6c50ac8640 1 -- 172.21.15.86:0/1628596533 shutdown_connections 2024-04-21T08:39:01.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.151+0000 7f6c50ac8640 1 -- 172.21.15.86:0/1628596533 wait complete. 2024-04-21T08:39:01.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.152+0000 7f6c50ac8640 1 Processor -- start 2024-04-21T08:39:01.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.152+0000 7f6c50ac8640 1 -- start start 2024-04-21T08:39:01.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.152+0000 7f6c50ac8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c4c102190 0x7f6c4c198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.152+0000 7f6c50ac8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c4c107260 0x7f6c4c198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.152+0000 7f6c50ac8640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c4c19cf50 0x7f6c4c19d400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.152+0000 7f6c50ac8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c1121b0 con 0x7f6c4c107260 2024-04-21T08:39:01.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.152+0000 7f6c50ac8640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c112030 con 0x7f6c4c102190 2024-04-21T08:39:01.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.152+0000 7f6c50ac8640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6c4c112330 con 0x7f6c4c19cf50 2024-04-21T08:39:01.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.152+0000 7f6c43fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c4c107260 0x7f6c4c198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.152+0000 7f6c43fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c4c107260 0x7f6c4c198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45908/0 (socket says 172.21.15.86:45908) 2024-04-21T08:39:01.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.152+0000 7f6c43fff640 1 -- 172.21.15.86:0/3241727789 learned_addr learned my addr 172.21.15.86:0/3241727789 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:01.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.152+0000 7f6c4a59c640 1 --2- 172.21.15.86:0/3241727789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c4c102190 0x7f6c4c198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.152+0000 7f6c4ad9d640 1 --2- 172.21.15.86:0/3241727789 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c4c19cf50 0x7f6c4c19d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.153+0000 7f6c43fff640 1 -- 172.21.15.86:0/3241727789 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c4c19cf50 msgr2=0x7f6c4c19d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.153+0000 7f6c43fff640 1 --2- 172.21.15.86:0/3241727789 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c4c19cf50 0x7f6c4c19d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.153+0000 7f6c43fff640 1 -- 172.21.15.86:0/3241727789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c4c102190 msgr2=0x7f6c4c198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.153+0000 7f6c43fff640 1 --2- 172.21.15.86:0/3241727789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c4c102190 0x7f6c4c198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.153+0000 7f6c43fff640 1 -- 172.21.15.86:0/3241727789 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6c4c19da40 con 0x7f6c4c107260 2024-04-21T08:39:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.153+0000 7f6c4a59c640 1 --2- 172.21.15.86:0/3241727789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c4c102190 0x7f6c4c198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.153+0000 7f6c43fff640 1 --2- 172.21.15.86:0/3241727789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c4c107260 0x7f6c4c198bd0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f6c3400d770 tx=0x7f6c3400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.153+0000 7f6c437fe640 1 -- 172.21.15.86:0/3241727789 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c34017070 con 0x7f6c4c107260 2024-04-21T08:39:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.153+0000 7f6c437fe640 1 -- 172.21.15.86:0/3241727789 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c34009c30 con 0x7f6c4c107260 2024-04-21T08:39:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.153+0000 7f6c50ac8640 1 -- 172.21.15.86:0/3241727789 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6c4c19dd30 con 0x7f6c4c107260 2024-04-21T08:39:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.153+0000 7f6c437fe640 1 -- 172.21.15.86:0/3241727789 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3400f620 con 0x7f6c4c107260 2024-04-21T08:39:01.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.153+0000 7f6c50ac8640 1 -- 172.21.15.86:0/3241727789 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6c4c1a5520 con 0x7f6c4c107260 2024-04-21T08:39:01.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.155+0000 7f6c437fe640 1 -- 172.21.15.86:0/3241727789 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f6c3401a070 con 0x7f6c4c107260 2024-04-21T08:39:01.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.155+0000 7f6c437fe640 1 --2- 172.21.15.86:0/3241727789 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f6c380a3db0 0x7f6c380a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.155+0000 7f6c53796740 1 Processor -- start 2024-04-21T08:39:01.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.156+0000 7f6c4a59c640 1 --2- 172.21.15.86:0/3241727789 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f6c380a3db0 0x7f6c380a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.156+0000 7f6c437fe640 1 -- 172.21.15.86:0/3241727789 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f6c340d08a0 con 0x7f6c4c107260 2024-04-21T08:39:01.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.156+0000 7f6c53796740 1 -- start start 2024-04-21T08:39:01.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.156+0000 7f6c53796740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555dacaa7610 0x555dacaa79f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.156+0000 7f6c53796740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555dacaa7fc0 0x555dacab0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.156+0000 7f6c53796740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555dacab1120 0x555dacab15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.156+0000 7f6c53796740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555daca0bb10 con 0x555dacaa7fc0 2024-04-21T08:39:01.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.156+0000 7f6c53796740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x555dac9a9240 con 0x555dacab1120 2024-04-21T08:39:01.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.156+0000 7f6c53796740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555daca07ad0 con 0x555dacaa7610 2024-04-21T08:39:01.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.156+0000 7f6c4a59c640 1 --2- 172.21.15.86:0/3241727789 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f6c380a3db0 0x7f6c380a6270 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c00abe0 tx=0x7f6c3c00a5c0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:01.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.156+0000 7f6c43fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555dacaa7fc0 0x555dacab0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.156+0000 7f6c43fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555dacaa7fc0 0x555dacab0b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45922/0 (socket says 172.21.15.86:45922) 2024-04-21T08:39:01.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.156+0000 7f6c43fff640 1 -- 172.21.15.86:0/265177627 learned_addr learned my addr 172.21.15.86:0/265177627 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:01.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.156+0000 7f6c4ad9d640 1 --2- 172.21.15.86:0/265177627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555dacab1120 0x555dacab15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.156+0000 7f6c4a59c640 1 --2- 172.21.15.86:0/265177627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555dacaa7610 0x555dacaa79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.156+0000 7f6c43fff640 1 -- 172.21.15.86:0/265177627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555dacaa7610 msgr2=0x555dacaa79f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.156+0000 7f6c43fff640 1 --2- 172.21.15.86:0/265177627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555dacaa7610 0x555dacaa79f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.156+0000 7f6c43fff640 1 -- 172.21.15.86:0/265177627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555dacab1120 msgr2=0x555dacab15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.156+0000 7f6c43fff640 1 --2- 172.21.15.86:0/265177627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555dacab1120 0x555dacab15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.156+0000 7f6c43fff640 1 -- 172.21.15.86:0/265177627 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555dac71e2d0 con 0x555dacaa7fc0 2024-04-21T08:39:01.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.157+0000 7f6c43fff640 1 --2- 172.21.15.86:0/265177627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555dacaa7fc0 0x555dacab0b00 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f6c340cdc50 tx=0x7f6c34009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d50b2a42cbddeb63 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:01.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.157+0000 7f6c40ff9640 1 -- 172.21.15.86:0/265177627 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c34021040 con 0x555dacaa7fc0 2024-04-21T08:39:01.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.157+0000 7f6c40ff9640 1 -- 172.21.15.86:0/265177627 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c340a0710 con 0x555dacaa7fc0 2024-04-21T08:39:01.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.157+0000 7f6c40ff9640 1 -- 172.21.15.86:0/265177627 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c340980b0 con 0x555dacaa7fc0 2024-04-21T08:39:01.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.157+0000 7f6c53796740 1 -- 172.21.15.86:0/265177627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555dacaa7fc0 msgr2=0x555dacab0b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.157+0000 7f6c53796740 1 --2- 172.21.15.86:0/265177627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555dacaa7fc0 0x555dacab0b00 secure :-1 s=READY pgs=617 cs=0 l=1 rev1=1 crypto rx=0x7f6c340cdc50 tx=0x7f6c34009de0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.157+0000 7f6c53796740 1 -- 172.21.15.86:0/265177627 shutdown_connections 2024-04-21T08:39:01.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.157+0000 7f6c53796740 1 --2- 172.21.15.86:0/265177627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555dacab1120 0x555dacab15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.157+0000 7f6c53796740 1 --2- 172.21.15.86:0/265177627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555dacaa7fc0 0x555dacab0b00 unknown :-1 s=CLOSED pgs=617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.157+0000 7f6c53796740 1 --2- 172.21.15.86:0/265177627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555dacaa7610 0x555dacaa79f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.157+0000 7f6c53796740 1 -- 172.21.15.86:0/265177627 >> 172.21.15.86:0/265177627 conn(0x555dacaa1a60 msgr2=0x555dacaa1e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:01.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.157+0000 7f6c53796740 1 -- 172.21.15.86:0/265177627 shutdown_connections 2024-04-21T08:39:01.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.158+0000 7f6c53796740 1 -- 172.21.15.86:0/265177627 wait complete. 2024-04-21T08:39:01.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.158+0000 7f6c53796740 1 -- start start 2024-04-21T08:39:01.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.158+0000 7f6c53796740 1 Processor -- start 2024-04-21T08:39:01.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.158+0000 7f6c53796740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555dacaa7610 0x555dacab4b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.158+0000 7f6c53796740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555dacaa7fc0 0x555dacab50d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.158+0000 7f6c53796740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555dacab1120 0x555dacab5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.158+0000 7f6c53796740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555dacab8040 con 0x555dacaa7fc0 2024-04-21T08:39:01.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.158+0000 7f6c53796740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x555dac994190 con 0x555dacab1120 2024-04-21T08:39:01.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.158+0000 7f6c53796740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555dacab81c0 con 0x555dacaa7610 2024-04-21T08:39:01.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.158+0000 7f6c53796740 10 client.0 _ll_register_callbacks cb 0x555dac98f8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:01.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.158+0000 7f6c43fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555dacaa7fc0 0x555dacab50d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.158+0000 7f6c43fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555dacaa7fc0 0x555dacab50d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45928/0 (socket says 172.21.15.86:45928) 2024-04-21T08:39:01.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.158+0000 7f6c4ad9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555dacab1120 0x555dacab5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.158+0000 7f6c43fff640 1 -- 172.21.15.86:0/3243178631 learned_addr learned my addr 172.21.15.86:0/3243178631 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:01.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.158+0000 7f6c4a59c640 1 --2- 172.21.15.86:0/3243178631 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555dacaa7610 0x555dacab4b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.158+0000 7f6c43fff640 1 -- 172.21.15.86:0/3243178631 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555dacaa7610 msgr2=0x555dacab4b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.158+0000 7f6c43fff640 1 --2- 172.21.15.86:0/3243178631 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555dacaa7610 0x555dacab4b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.158+0000 7f6c43fff640 1 -- 172.21.15.86:0/3243178631 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555dacab1120 msgr2=0x555dacab5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.158+0000 7f6c43fff640 1 --2- 172.21.15.86:0/3243178631 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555dacab1120 0x555dacab5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.158+0000 7f6c43fff640 1 -- 172.21.15.86:0/3243178631 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dac7e86b0 con 0x555dacaa7fc0 2024-04-21T08:39:01.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.159+0000 7f6c43fff640 1 --2- 172.21.15.86:0/3243178631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555dacaa7fc0 0x555dacab50d0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f6c34098d30 tx=0x7f6c34005c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1032260459fd36 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:01.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.159+0000 7f6c31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:01.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.159+0000 7f6c31ffb640 1 -- 172.21.15.86:0/3243178631 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c34021030 con 0x555dacaa7fc0 2024-04-21T08:39:01.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.159+0000 7f6c31ffb640 1 -- 172.21.15.86:0/3243178631 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f6c340bb270 con 0x555dacaa7fc0 2024-04-21T08:39:01.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.159+0000 7f6c53796740 1 -- 172.21.15.86:0/3243178631 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c340a0710 con 0x555dacaa7fc0 2024-04-21T08:39:01.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.159+0000 7f6c31ffb640 1 -- 172.21.15.86:0/3243178631 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6c340a4fd0 con 0x555dacaa7fc0 2024-04-21T08:39:01.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.159+0000 7f6c53796740 10 client.15981 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:01.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.159+0000 7f6c53796740 1 -- 172.21.15.86:0/3243178631 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555dacac4330 con 0x555dacaa7fc0 2024-04-21T08:39:01.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.159+0000 7f6c31ffb640 1 -- 172.21.15.86:0/3243178631 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c340a0710 con 0x555dacaa7fc0 2024-04-21T08:39:01.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.159+0000 7f6c31ffb640 1 -- 172.21.15.86:0/3243178631 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6c3407f070 con 0x555dacaa7fc0 2024-04-21T08:39:01.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.159+0000 7f6c53796740 10 client.15981 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:01.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.159+0000 7f6c53796740 10 client.15981 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-21T08:39:01.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.159+0000 7f6c53796740 20 client.15981 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:01.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.159+0000 7f6c53796740 1 --2- 172.21.15.86:0/3243178631 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x555dacac6450 0x555dacac78c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.159+0000 7f6c53796740 4 client.15981 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:01.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.159+0000 7f6c53796740 1 -- 172.21.15.86:0/3243178631 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555daca19530 con 0x555dacac6450 2024-04-21T08:39:01.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.160+0000 7f6c4a59c640 1 --2- 172.21.15.86:0/3243178631 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x555dacac6450 0x555dacac78c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.160+0000 7f6c4a59c640 1 --2- 172.21.15.86:0/3243178631 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x555dacac6450 0x555dacac78c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9e02fd73b4920d28 server_cookie=322cb798e2e113fe in_seq=0 out_seq=0 2024-04-21T08:39:01.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.160+0000 7f6c31ffb640 10 client.15981 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:39:01.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.161+0000 7f6c31ffb640 1 -- 172.21.15.86:0/3243178631 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555daca19530 con 0x555dacac6450 2024-04-21T08:39:01.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.161+0000 7f6c31ffb640 10 client.15981 handle_command_reply: tid=0 2024-04-21T08:39:01.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.161+0000 7f6c53796740 1 client.15981 shutdown 2024-04-21T08:39:01.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.161+0000 7f6c53796740 1 -- 172.21.15.86:0/3243178631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555dacaa7fc0 msgr2=0x555dacab50d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.161+0000 7f6c53796740 1 --2- 172.21.15.86:0/3243178631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555dacaa7fc0 0x555dacab50d0 secure :-1 s=READY pgs=618 cs=0 l=1 rev1=1 crypto rx=0x7f6c34098d30 tx=0x7f6c34005c20 comp rx=0 tx=0).stop 2024-04-21T08:39:01.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.161+0000 7f6c53796740 1 -- 172.21.15.86:0/3243178631 shutdown_connections 2024-04-21T08:39:01.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.161+0000 7f6c53796740 1 --2- 172.21.15.86:0/3243178631 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x555dacac6450 0x555dacac78c0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.161+0000 7f6c53796740 1 --2- 172.21.15.86:0/3243178631 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555dacab1120 0x555dacab5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.161+0000 7f6c31ffb640 0 client.15981 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:39:01.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.161+0000 7f6c53796740 1 --2- 172.21.15.86:0/3243178631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555dacaa7fc0 0x555dacab50d0 unknown :-1 s=CLOSED pgs=618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.161+0000 7f6c53796740 1 --2- 172.21.15.86:0/3243178631 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555dacaa7610 0x555dacab4b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.161+0000 7f6c53796740 1 -- 172.21.15.86:0/3243178631 >> 172.21.15.86:0/3243178631 conn(0x555dacaa1a60 msgr2=0x555dacaa3bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:01.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.161+0000 7f6c53796740 1 -- 172.21.15.86:0/3243178631 shutdown_connections 2024-04-21T08:39:01.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.161+0000 7f6c53796740 1 -- 172.21.15.86:0/3243178631 wait complete. 2024-04-21T08:39:01.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.161+0000 7f6c53796740 20 client.15981 trim_cache size 0 max 16384 2024-04-21T08:39:01.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.178+0000 7f6c53796740 1 Processor -- start 2024-04-21T08:39:01.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.179+0000 7f6c53796740 1 -- start start 2024-04-21T08:39:01.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.179+0000 7f6c53796740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555dacaa7610 0x555dacab72c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.179+0000 7f6c53796740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555dacaa7fc0 0x555dacab08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.179+0000 7f6c53796740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555dacab1120 0x555dacab2510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.179+0000 7f6c53796740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555dacac5c00 con 0x555dacaa7610 2024-04-21T08:39:01.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.179+0000 7f6c53796740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x555dacac62d0 con 0x555dacaa7fc0 2024-04-21T08:39:01.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.179+0000 7f6c53796740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555dacac69a0 con 0x555dacab1120 2024-04-21T08:39:01.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.179+0000 7f6c4a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555dacaa7610 0x555dacab72c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.179+0000 7f6c4a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555dacaa7610 0x555dacab72c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45932/0 (socket says 172.21.15.86:45932) 2024-04-21T08:39:01.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.179+0000 7f6c4a59c640 1 -- 172.21.15.86:0/2889520666 learned_addr learned my addr 172.21.15.86:0/2889520666 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:01.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.179+0000 7f6c43fff640 1 --2- 172.21.15.86:0/2889520666 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555dacaa7fc0 0x555dacab08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.179+0000 7f6c4a59c640 1 -- 172.21.15.86:0/2889520666 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555dacab1120 msgr2=0x555dacab2510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.179+0000 7f6c4a59c640 1 --2- 172.21.15.86:0/2889520666 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555dacab1120 0x555dacab2510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.179+0000 7f6c4a59c640 1 -- 172.21.15.86:0/2889520666 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555dacaa7fc0 msgr2=0x555dacab08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.179+0000 7f6c4a59c640 1 --2- 172.21.15.86:0/2889520666 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555dacaa7fc0 0x555dacab08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.179+0000 7f6c4a59c640 1 -- 172.21.15.86:0/2889520666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555daca9bd80 con 0x555dacaa7610 2024-04-21T08:39:01.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.179+0000 7f6c4a59c640 1 --2- 172.21.15.86:0/2889520666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555dacaa7610 0x555dacab72c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x555dacac8120 tx=0x7f6c3c014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e0af00be80039ff server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:01.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.179+0000 7f6c33fff640 1 -- 172.21.15.86:0/2889520666 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c02b030 con 0x555dacaa7610 2024-04-21T08:39:01.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.179+0000 7f6c33fff640 1 -- 172.21.15.86:0/2889520666 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c3c027bc0 con 0x555dacaa7610 2024-04-21T08:39:01.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.179+0000 7f6c33fff640 1 -- 172.21.15.86:0/2889520666 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c3c0035f0 con 0x555dacaa7610 2024-04-21T08:39:01.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.180+0000 7f6c53796740 1 -- 172.21.15.86:0/2889520666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555dacaa7610 msgr2=0x555dacab72c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.180+0000 7f6c53796740 1 --2- 172.21.15.86:0/2889520666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555dacaa7610 0x555dacab72c0 secure :-1 s=READY pgs=619 cs=0 l=1 rev1=1 crypto rx=0x555dacac8120 tx=0x7f6c3c014000 comp rx=0 tx=0).stop 2024-04-21T08:39:01.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.180+0000 7f6c53796740 1 -- 172.21.15.86:0/2889520666 shutdown_connections 2024-04-21T08:39:01.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.180+0000 7f6c53796740 1 --2- 172.21.15.86:0/2889520666 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555dacab1120 0x555dacab2510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.180+0000 7f6c53796740 1 --2- 172.21.15.86:0/2889520666 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555dacaa7fc0 0x555dacab08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.180+0000 7f6c53796740 1 --2- 172.21.15.86:0/2889520666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555dacaa7610 0x555dacab72c0 unknown :-1 s=CLOSED pgs=619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.180+0000 7f6c53796740 1 -- 172.21.15.86:0/2889520666 >> 172.21.15.86:0/2889520666 conn(0x555dacaa1a60 msgr2=0x555daca9c800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:01.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.180+0000 7f6c53796740 1 -- 172.21.15.86:0/2889520666 shutdown_connections 2024-04-21T08:39:01.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.180+0000 7f6c53796740 1 -- 172.21.15.86:0/2889520666 wait complete. 2024-04-21T08:39:01.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.180+0000 7f6c53796740 1 -- start start 2024-04-21T08:39:01.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.180+0000 7f6c53796740 1 Processor -- start 2024-04-21T08:39:01.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.180+0000 7f6c53796740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555dacaa7610 0x555dacab49c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.180+0000 7f6c53796740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555dacaa7fc0 0x555dacab4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.180+0000 7f6c53796740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555dacab1120 0x555dacab5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.180+0000 7f6c53796740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555dacac7740 con 0x555dacab1120 2024-04-21T08:39:01.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.180+0000 7f6c53796740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x555dacac7070 con 0x555dacaa7610 2024-04-21T08:39:01.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.180+0000 7f6c53796740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555dacac7e10 con 0x555dacaa7fc0 2024-04-21T08:39:01.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.181+0000 7f6c53796740 10 client.0 _ll_register_callbacks cb 0x7f6c44037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:01.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.181+0000 7f6c4ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555dacab1120 0x555dacab5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.181+0000 7f6c4ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555dacab1120 0x555dacab5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45934/0 (socket says 172.21.15.86:45934) 2024-04-21T08:39:01.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.181+0000 7f6c4ad9d640 1 -- 172.21.15.86:0/3291877125 learned_addr learned my addr 172.21.15.86:0/3291877125 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:01.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.181+0000 7f6c4a59c640 1 --2- 172.21.15.86:0/3291877125 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555dacaa7610 0x555dacab49c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.181+0000 7f6c43fff640 1 --2- 172.21.15.86:0/3291877125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555dacaa7fc0 0x555dacab4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.181+0000 7f6c4ad9d640 1 -- 172.21.15.86:0/3291877125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555dacaa7fc0 msgr2=0x555dacab4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.181+0000 7f6c4ad9d640 1 --2- 172.21.15.86:0/3291877125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555dacaa7fc0 0x555dacab4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.181+0000 7f6c4ad9d640 1 -- 172.21.15.86:0/3291877125 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555dacaa7610 msgr2=0x555dacab49c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.181+0000 7f6c4ad9d640 1 --2- 172.21.15.86:0/3291877125 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555dacaa7610 0x555dacab49c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.181+0000 7f6c4ad9d640 1 -- 172.21.15.86:0/3291877125 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555dacacde50 con 0x555dacab1120 2024-04-21T08:39:01.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.181+0000 7f6c4ad9d640 1 --2- 172.21.15.86:0/3291877125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555dacab1120 0x555dacab5440 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f6c44009ca0 tx=0x7f6c44032df0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35cac5194f79c059 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:01.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.181+0000 7f6c31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:01.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.181+0000 7f6c31ffb640 1 -- 172.21.15.86:0/3291877125 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6c44039040 con 0x555dacab1120 2024-04-21T08:39:01.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.181+0000 7f6c31ffb640 1 -- 172.21.15.86:0/3291877125 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f6c440328d0 con 0x555dacab1120 2024-04-21T08:39:01.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.181+0000 7f6c53796740 1 -- 172.21.15.86:0/3291877125 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6c3c027bc0 con 0x555dacab1120 2024-04-21T08:39:01.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.181+0000 7f6c31ffb640 1 -- 172.21.15.86:0/3291877125 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6c44054050 con 0x555dacab1120 2024-04-21T08:39:01.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.181+0000 7f6c53796740 10 client.15987 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:01.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.181+0000 7f6c53796740 1 -- 172.21.15.86:0/3291877125 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555dacb2c750 con 0x555dacab1120 2024-04-21T08:39:01.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.181+0000 7f6c31ffb640 1 -- 172.21.15.86:0/3291877125 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6c3c027bc0 con 0x555dacab1120 2024-04-21T08:39:01.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.182+0000 7f6c31ffb640 1 -- 172.21.15.86:0/3291877125 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6c44004690 con 0x555dacab1120 2024-04-21T08:39:01.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.182+0000 7f6c53796740 10 client.15987 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:01.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.182+0000 7f6c53796740 10 client.15987 resolve_mds: resolved name 'k' to daemon mds.k 2024-04-21T08:39:01.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.182+0000 7f6c53796740 20 client.15987 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:01.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.182+0000 7f6c53796740 1 --2- 172.21.15.86:0/3291877125 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x555dacb31350 0x555dacb30060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.182+0000 7f6c53796740 4 client.15987 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:39:01.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.182+0000 7f6c53796740 1 -- 172.21.15.86:0/3291877125 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "status"}) -- 0x555daca133a0 con 0x555dacb31350 2024-04-21T08:39:01.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.182+0000 7f6c4a59c640 1 --2- 172.21.15.86:0/3291877125 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x555dacb31350 0x555dacb30060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.182+0000 7f6c31ffb640 10 client.15987 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:39:01.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.182+0000 7f6c4a59c640 1 --2- 172.21.15.86:0/3291877125 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x555dacb31350 0x555dacb30060 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=df5977d9d834a89b server_cookie=992990c68b1bd587 in_seq=0 out_seq=0 2024-04-21T08:39:01.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.183+0000 7f6c31ffb640 1 -- 172.21.15.86:0/3291877125 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x555daca133a0 con 0x555dacb31350 2024-04-21T08:39:01.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.183+0000 7f6c31ffb640 10 client.15987 handle_command_reply: tid=0 2024-04-21T08:39:01.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.183+0000 7f6c53796740 1 client.15987 shutdown 2024-04-21T08:39:01.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.183+0000 7f6c53796740 1 -- 172.21.15.86:0/3291877125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555dacab1120 msgr2=0x555dacab5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.183+0000 7f6c53796740 1 --2- 172.21.15.86:0/3291877125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555dacab1120 0x555dacab5440 secure :-1 s=READY pgs=620 cs=0 l=1 rev1=1 crypto rx=0x7f6c44009ca0 tx=0x7f6c44032df0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.183+0000 7f6c53796740 1 -- 172.21.15.86:0/3291877125 shutdown_connections 2024-04-21T08:39:01.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.183+0000 7f6c53796740 1 --2- 172.21.15.86:0/3291877125 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x555dacb31350 0x555dacb30060 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.183+0000 7f6c53796740 1 --2- 172.21.15.86:0/3291877125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555dacab1120 0x555dacab5440 unknown :-1 s=CLOSED pgs=620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.183+0000 7f6c53796740 1 --2- 172.21.15.86:0/3291877125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555dacaa7fc0 0x555dacab4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.183+0000 7f6c31ffb640 0 client.15987 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:39:01.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.183+0000 7f6c53796740 1 --2- 172.21.15.86:0/3291877125 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555dacaa7610 0x555dacab49c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.183+0000 7f6c53796740 1 -- 172.21.15.86:0/3291877125 >> 172.21.15.86:0/3291877125 conn(0x555dacaa1a60 msgr2=0x555dacaaead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:01.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.183+0000 7f6c53796740 1 -- 172.21.15.86:0/3291877125 shutdown_connections 2024-04-21T08:39:01.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.183+0000 7f6c53796740 1 -- 172.21.15.86:0/3291877125 wait complete. 2024-04-21T08:39:01.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.183+0000 7f6c53796740 20 client.15987 trim_cache size 0 max 16384 2024-04-21T08:39:01.191 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:39:01.191 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "f8cec7cc-ffb8-11ee-bc93-c7b262605968", 2024-04-21T08:39:01.192 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 3, 2024-04-21T08:39:01.192 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24398, 2024-04-21T08:39:01.192 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-21T08:39:01.192 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-21T08:39:01.192 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-21T08:39:01.192 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 135.747183495, 2024-04-21T08:39:01.192 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 29, 2024-04-21T08:39:01.192 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 94, 2024-04-21T08:39:01.192 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 93, 2024-04-21T08:39:01.193 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 195.138117002 2024-04-21T08:39:01.193 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:39:01.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.183+0000 7f6c50ac8640 1 -- 172.21.15.86:0/3241727789 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f6c380a3db0 msgr2=0x7f6c380a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.194 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.183+0000 7f6c50ac8640 1 --2- 172.21.15.86:0/3241727789 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f6c380a3db0 0x7f6c380a6270 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7f6c3c00abe0 tx=0x7f6c3c00a5c0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.183+0000 7f6c50ac8640 1 -- 172.21.15.86:0/3241727789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c4c107260 msgr2=0x7f6c4c198bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.183+0000 7f6c50ac8640 1 --2- 172.21.15.86:0/3241727789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c4c107260 0x7f6c4c198bd0 secure :-1 s=READY pgs=616 cs=0 l=1 rev1=1 crypto rx=0x7f6c3400d770 tx=0x7f6c3400dc30 comp rx=0 tx=0).stop 2024-04-21T08:39:01.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.183+0000 7f6c50ac8640 1 -- 172.21.15.86:0/3241727789 shutdown_connections 2024-04-21T08:39:01.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.183+0000 7f6c50ac8640 1 --2- 172.21.15.86:0/3241727789 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f6c380a3db0 0x7f6c380a6270 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.183+0000 7f6c50ac8640 1 --2- 172.21.15.86:0/3241727789 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6c4c19cf50 0x7f6c4c19d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.183+0000 7f6c50ac8640 1 --2- 172.21.15.86:0/3241727789 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6c4c107260 0x7f6c4c198bd0 unknown :-1 s=CLOSED pgs=616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.183+0000 7f6c50ac8640 1 --2- 172.21.15.86:0/3241727789 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6c4c102190 0x7f6c4c198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.183+0000 7f6c50ac8640 1 -- 172.21.15.86:0/3241727789 >> 172.21.15.86:0/3241727789 conn(0x7f6c4c0fd990 msgr2=0x7f6c4c0ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:01.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.183+0000 7f6c50ac8640 1 -- 172.21.15.86:0/3241727789 shutdown_connections 2024-04-21T08:39:01.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.183+0000 7f6c50ac8640 1 -- 172.21.15.86:0/3241727789 wait complete. 2024-04-21T08:39:01.195 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.h status 2024-04-21T08:39:01.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.294+0000 7f0d0e06d640 1 -- 172.21.15.86:0/3381770608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d08107fc0 msgr2=0x7f0d081083a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.294+0000 7f0d0e06d640 1 --2- 172.21.15.86:0/3381770608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d08107fc0 0x7f0d081083a0 secure :-1 s=READY pgs=621 cs=0 l=1 rev1=1 crypto rx=0x7f0cf8009930 tx=0x7f0cf802bc20 comp rx=0 tx=0).stop 2024-04-21T08:39:01.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.294+0000 7f0d0e06d640 1 -- 172.21.15.86:0/3381770608 shutdown_connections 2024-04-21T08:39:01.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.294+0000 7f0d0e06d640 1 --2- 172.21.15.86:0/3381770608 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d0806aed0 0x7f0d0810edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.294+0000 7f0d0e06d640 1 --2- 172.21.15.86:0/3381770608 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d0806a550 0x7f0d0806a990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.294+0000 7f0d0e06d640 1 --2- 172.21.15.86:0/3381770608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d08107fc0 0x7f0d081083a0 unknown :-1 s=CLOSED pgs=621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.294+0000 7f0d0e06d640 1 -- 172.21.15.86:0/3381770608 >> 172.21.15.86:0/3381770608 conn(0x7f0d08076c50 msgr2=0x7f0d08077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.294+0000 7f0d0e06d640 1 -- 172.21.15.86:0/3381770608 shutdown_connections 2024-04-21T08:39:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.294+0000 7f0d0e06d640 1 -- 172.21.15.86:0/3381770608 wait complete. 2024-04-21T08:39:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.295+0000 7f0d0e06d640 1 Processor -- start 2024-04-21T08:39:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.295+0000 7f0d0e06d640 1 -- start start 2024-04-21T08:39:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.295+0000 7f0d0e06d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d0806a550 0x7f0d081a06b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.295+0000 7f0d0e06d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d0806aed0 0x7f0d081a0bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.295+0000 7f0d0e06d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d08107fc0 0x7f0d0819a770 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.295+0000 7f0d0e06d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d08112190 con 0x7f0d0806aed0 2024-04-21T08:39:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.295+0000 7f0d0e06d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0d08112010 con 0x7f0d0806a550 2024-04-21T08:39:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.295+0000 7f0d0e06d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0d08112310 con 0x7f0d08107fc0 2024-04-21T08:39:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.295+0000 7f0d06ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d0806aed0 0x7f0d081a0bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.295+0000 7f0d06ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d0806aed0 0x7f0d081a0bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45948/0 (socket says 172.21.15.86:45948) 2024-04-21T08:39:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.295+0000 7f0d06ffd640 1 -- 172.21.15.86:0/1247081231 learned_addr learned my addr 172.21.15.86:0/1247081231 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.296+0000 7f0d07fff640 1 --2- 172.21.15.86:0/1247081231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d08107fc0 0x7f0d0819a770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.296+0000 7f0d077fe640 1 --2- 172.21.15.86:0/1247081231 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d0806a550 0x7f0d081a06b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.296+0000 7f0d06ffd640 1 -- 172.21.15.86:0/1247081231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d08107fc0 msgr2=0x7f0d0819a770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.296+0000 7f0d06ffd640 1 --2- 172.21.15.86:0/1247081231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d08107fc0 0x7f0d0819a770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.296+0000 7f0d06ffd640 1 -- 172.21.15.86:0/1247081231 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d0806a550 msgr2=0x7f0d081a06b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.296+0000 7f0d06ffd640 1 --2- 172.21.15.86:0/1247081231 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d0806a550 0x7f0d081a06b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.296+0000 7f0d06ffd640 1 -- 172.21.15.86:0/1247081231 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d0819b030 con 0x7f0d0806aed0 2024-04-21T08:39:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.296+0000 7f0d06ffd640 1 --2- 172.21.15.86:0/1247081231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d0806aed0 0x7f0d081a0bf0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc00d770 tx=0x7f0cfc00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.296+0000 7f0d04ff9640 1 -- 172.21.15.86:0/1247081231 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc017070 con 0x7f0d0806aed0 2024-04-21T08:39:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.296+0000 7f0d04ff9640 1 -- 172.21.15.86:0/1247081231 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cfc0054d0 con 0x7f0d0806aed0 2024-04-21T08:39:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.296+0000 7f0d0e06d640 1 -- 172.21.15.86:0/1247081231 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d0819b320 con 0x7f0d0806aed0 2024-04-21T08:39:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.296+0000 7f0d04ff9640 1 -- 172.21.15.86:0/1247081231 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc004c30 con 0x7f0d0806aed0 2024-04-21T08:39:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.296+0000 7f0d0e06d640 1 -- 172.21.15.86:0/1247081231 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d081a7630 con 0x7f0d0806aed0 2024-04-21T08:39:01.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.298+0000 7f0d04ff9640 1 -- 172.21.15.86:0/1247081231 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 24) ==== 138822+0+0 (secure 0 0 0) 0x7f0cfc004e10 con 0x7f0d0806aed0 2024-04-21T08:39:01.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.299+0000 7f0d10c69740 1 Processor -- start 2024-04-21T08:39:01.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.299+0000 7f0d10c69740 1 -- start start 2024-04-21T08:39:01.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.299+0000 7f0d04ff9640 1 --2- 172.21.15.86:0/1247081231 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0cf00a3db0 0x7f0cf00a6270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.299+0000 7f0d10c69740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4a23947b0 0x55c4a2394b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.299+0000 7f0d10c69740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4a2395160 0x55c4a239d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.299+0000 7f0d10c69740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a239dce0 0x55c4a23a0130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.299+0000 7f0d10c69740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c4a22f8b10 con 0x55c4a239dce0 2024-04-21T08:39:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.299+0000 7f0d10c69740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c4a2296240 con 0x55c4a23947b0 2024-04-21T08:39:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.299+0000 7f0d10c69740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c4a22f4ad0 con 0x55c4a2395160 2024-04-21T08:39:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.299+0000 7f0d07fff640 1 --2- 172.21.15.86:0/1247081231 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0cf00a3db0 0x7f0cf00a6270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.299+0000 7f0d04ff9640 1 -- 172.21.15.86:0/1247081231 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f0cfc0d07e0 con 0x7f0d0806aed0 2024-04-21T08:39:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.299+0000 7f0d07fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a239dce0 0x55c4a23a0130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.299+0000 7f0d07fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a239dce0 0x55c4a23a0130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45954/0 (socket says 172.21.15.86:45954) 2024-04-21T08:39:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.299+0000 7f0d07fff640 1 -- 172.21.15.86:0/2736243343 learned_addr learned my addr 172.21.15.86:0/2736243343 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.299+0000 7f0d077fe640 1 --2- 172.21.15.86:0/2736243343 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4a23947b0 0x55c4a2394b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.299+0000 7f0d06ffd640 1 --2- 172.21.15.86:0/2736243343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4a2395160 0x55c4a239d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.299+0000 7f0d07fff640 1 -- 172.21.15.86:0/2736243343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4a2395160 msgr2=0x55c4a239d710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.299+0000 7f0d07fff640 1 --2- 172.21.15.86:0/2736243343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4a2395160 0x55c4a239d710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.299+0000 7f0d07fff640 1 -- 172.21.15.86:0/2736243343 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4a23947b0 msgr2=0x55c4a2394b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.299+0000 7f0d07fff640 1 --2- 172.21.15.86:0/2736243343 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4a23947b0 0x55c4a2394b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.299+0000 7f0d07fff640 1 -- 172.21.15.86:0/2736243343 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4a200b2d0 con 0x55c4a239dce0 2024-04-21T08:39:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.300+0000 7f0d07fff640 1 --2- 172.21.15.86:0/1247081231 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0cf00a3db0 0x7f0cf00a6270 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0d000077f0 tx=0x7f0d00007e40 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.300+0000 7f0d07fff640 1 --2- 172.21.15.86:0/2736243343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a239dce0 0x55c4a23a0130 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f0d00011c40 tx=0x7f0d0001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=139f38a0a600d5ac server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.300+0000 7f0cedffb640 1 -- 172.21.15.86:0/2736243343 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d00022070 con 0x55c4a239dce0 2024-04-21T08:39:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.300+0000 7f0cedffb640 1 -- 172.21.15.86:0/2736243343 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d0000fe70 con 0x55c4a239dce0 2024-04-21T08:39:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.300+0000 7f0cedffb640 1 -- 172.21.15.86:0/2736243343 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d00005550 con 0x55c4a239dce0 2024-04-21T08:39:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.300+0000 7f0d10c69740 1 -- 172.21.15.86:0/2736243343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a239dce0 msgr2=0x55c4a23a0130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.300+0000 7f0d10c69740 1 --2- 172.21.15.86:0/2736243343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a239dce0 0x55c4a23a0130 secure :-1 s=READY pgs=623 cs=0 l=1 rev1=1 crypto rx=0x7f0d00011c40 tx=0x7f0d0001b000 comp rx=0 tx=0).stop 2024-04-21T08:39:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.300+0000 7f0d10c69740 1 -- 172.21.15.86:0/2736243343 shutdown_connections 2024-04-21T08:39:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.300+0000 7f0d10c69740 1 --2- 172.21.15.86:0/2736243343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a239dce0 0x55c4a23a0130 unknown :-1 s=CLOSED pgs=623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.300+0000 7f0d10c69740 1 --2- 172.21.15.86:0/2736243343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4a2395160 0x55c4a239d710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.300+0000 7f0d10c69740 1 --2- 172.21.15.86:0/2736243343 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4a23947b0 0x55c4a2394b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.300+0000 7f0d10c69740 1 -- 172.21.15.86:0/2736243343 >> 172.21.15.86:0/2736243343 conn(0x55c4a238ea60 msgr2=0x55c4a238ee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.300+0000 7f0d10c69740 1 -- 172.21.15.86:0/2736243343 shutdown_connections 2024-04-21T08:39:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.300+0000 7f0d10c69740 1 -- 172.21.15.86:0/2736243343 wait complete. 2024-04-21T08:39:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.301+0000 7f0d10c69740 1 -- start start 2024-04-21T08:39:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.301+0000 7f0d10c69740 1 Processor -- start 2024-04-21T08:39:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.301+0000 7f0d10c69740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4a23947b0 0x55c4a239abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.301+0000 7f0d10c69740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4a2395160 0x55c4a239b130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.301+0000 7f0d10c69740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a239dce0 0x55c4a239b670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.301+0000 7f0d10c69740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c4a23a0bf0 con 0x55c4a239dce0 2024-04-21T08:39:01.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.301+0000 7f0d10c69740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c4a2281190 con 0x55c4a2395160 2024-04-21T08:39:01.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.301+0000 7f0d10c69740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c4a23a0d70 con 0x55c4a23947b0 2024-04-21T08:39:01.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.301+0000 7f0d10c69740 10 client.0 _ll_register_callbacks cb 0x55c4a227c8b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:01.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.301+0000 7f0d07fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a239dce0 0x55c4a239b670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.301+0000 7f0d07fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a239dce0 0x55c4a239b670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45956/0 (socket says 172.21.15.86:45956) 2024-04-21T08:39:01.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.301+0000 7f0d07fff640 1 -- 172.21.15.86:0/3274298064 learned_addr learned my addr 172.21.15.86:0/3274298064 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:01.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.301+0000 7f0d06ffd640 1 --2- 172.21.15.86:0/3274298064 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4a2395160 0x55c4a239b130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.301+0000 7f0d077fe640 1 --2- 172.21.15.86:0/3274298064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4a23947b0 0x55c4a239abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.302+0000 7f0d07fff640 1 -- 172.21.15.86:0/3274298064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4a23947b0 msgr2=0x55c4a239abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.302+0000 7f0d07fff640 1 --2- 172.21.15.86:0/3274298064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4a23947b0 0x55c4a239abf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.302+0000 7f0d07fff640 1 -- 172.21.15.86:0/3274298064 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4a2395160 msgr2=0x55c4a239b130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.302+0000 7f0d07fff640 1 --2- 172.21.15.86:0/3274298064 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4a2395160 0x55c4a239b130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.302+0000 7f0d07fff640 1 -- 172.21.15.86:0/3274298064 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4a20d56b0 con 0x55c4a239dce0 2024-04-21T08:39:01.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.302+0000 7f0d07fff640 1 --2- 172.21.15.86:0/3274298064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a239dce0 0x55c4a239b670 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0d00007580 tx=0x7f0d00004c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1441b7bb7c046521 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:01.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.302+0000 7f0ce6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:01.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.302+0000 7f0ce6ffd640 1 -- 172.21.15.86:0/3274298064 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d00033060 con 0x55c4a239dce0 2024-04-21T08:39:01.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.302+0000 7f0ce6ffd640 1 -- 172.21.15.86:0/3274298064 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f0d00005700 con 0x55c4a239dce0 2024-04-21T08:39:01.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.302+0000 7f0d10c69740 1 -- 172.21.15.86:0/3274298064 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d0000fe70 con 0x55c4a239dce0 2024-04-21T08:39:01.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.302+0000 7f0ce6ffd640 1 -- 172.21.15.86:0/3274298064 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0d0003a050 con 0x55c4a239dce0 2024-04-21T08:39:01.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.302+0000 7f0d10c69740 10 client.15999 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:01.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.302+0000 7f0d10c69740 1 -- 172.21.15.86:0/3274298064 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c4a23ace40 con 0x55c4a239dce0 2024-04-21T08:39:01.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.302+0000 7f0ce6ffd640 1 -- 172.21.15.86:0/3274298064 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d0000fe70 con 0x55c4a239dce0 2024-04-21T08:39:01.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.302+0000 7f0ce6ffd640 1 -- 172.21.15.86:0/3274298064 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0d00012900 con 0x55c4a239dce0 2024-04-21T08:39:01.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.302+0000 7f0d10c69740 10 client.15999 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:01.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.302+0000 7f0d10c69740 10 client.15999 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-21T08:39:01.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.302+0000 7f0d10c69740 20 client.15999 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:01.311 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.303+0000 7f0d10c69740 1 --2- 172.21.15.86:0/3274298064 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55c4a23af0f0 0x55c4a23b0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.303+0000 7f0d10c69740 4 client.15999 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:01.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.303+0000 7f0d10c69740 1 -- 172.21.15.86:0/3274298064 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4a2306530 con 0x55c4a23af0f0 2024-04-21T08:39:01.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.303+0000 7f0d077fe640 1 --2- 172.21.15.86:0/3274298064 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55c4a23af0f0 0x55c4a23b0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.303+0000 7f0ce6ffd640 10 client.15999 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:39:01.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.303+0000 7f0d077fe640 1 --2- 172.21.15.86:0/3274298064 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55c4a23af0f0 0x55c4a23b0560 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7551d0ebc790e8ce server_cookie=77ad4894443cbb6e in_seq=0 out_seq=0 2024-04-21T08:39:01.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.304+0000 7f0ce6ffd640 1 -- 172.21.15.86:0/3274298064 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c4a2306530 con 0x55c4a23af0f0 2024-04-21T08:39:01.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.304+0000 7f0ce6ffd640 10 client.15999 handle_command_reply: tid=0 2024-04-21T08:39:01.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.304+0000 7f0d10c69740 1 client.15999 shutdown 2024-04-21T08:39:01.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.304+0000 7f0d10c69740 1 -- 172.21.15.86:0/3274298064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a239dce0 msgr2=0x55c4a239b670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.304+0000 7f0d10c69740 1 --2- 172.21.15.86:0/3274298064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a239dce0 0x55c4a239b670 secure :-1 s=READY pgs=624 cs=0 l=1 rev1=1 crypto rx=0x7f0d00007580 tx=0x7f0d00004c90 comp rx=0 tx=0).stop 2024-04-21T08:39:01.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.304+0000 7f0d10c69740 1 -- 172.21.15.86:0/3274298064 shutdown_connections 2024-04-21T08:39:01.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.304+0000 7f0d10c69740 1 --2- 172.21.15.86:0/3274298064 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55c4a23af0f0 0x55c4a23b0560 crc :-1 s=READY pgs=33 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.304+0000 7f0d10c69740 1 --2- 172.21.15.86:0/3274298064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a239dce0 0x55c4a239b670 unknown :-1 s=CLOSED pgs=624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.304+0000 7f0d10c69740 1 --2- 172.21.15.86:0/3274298064 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4a2395160 0x55c4a239b130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.304+0000 7f0ce6ffd640 0 client.15999 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:39:01.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.304+0000 7f0d10c69740 1 --2- 172.21.15.86:0/3274298064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4a23947b0 0x55c4a239abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.304+0000 7f0d10c69740 1 -- 172.21.15.86:0/3274298064 >> 172.21.15.86:0/3274298064 conn(0x55c4a238ea60 msgr2=0x55c4a2390bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:01.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.304+0000 7f0d10c69740 1 -- 172.21.15.86:0/3274298064 shutdown_connections 2024-04-21T08:39:01.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.304+0000 7f0d10c69740 1 -- 172.21.15.86:0/3274298064 wait complete. 2024-04-21T08:39:01.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.304+0000 7f0d10c69740 20 client.15999 trim_cache size 0 max 16384 2024-04-21T08:39:01.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.324+0000 7f0d10c69740 1 Processor -- start 2024-04-21T08:39:01.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.324+0000 7f0d10c69740 1 -- start start 2024-04-21T08:39:01.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.325+0000 7f0d10c69740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a23947b0 0x55c4a239d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.325+0000 7f0d10c69740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4a2395160 0x55c4a239b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.325+0000 7f0d10c69740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4a239dce0 0x55c4a239b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.325+0000 7f0d10c69740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c4a23ae8a0 con 0x55c4a23947b0 2024-04-21T08:39:01.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.325+0000 7f0d10c69740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c4a23aef70 con 0x55c4a2395160 2024-04-21T08:39:01.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.325+0000 7f0d10c69740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c4a23af640 con 0x55c4a239dce0 2024-04-21T08:39:01.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.326+0000 7f0d07fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4a239dce0 0x55c4a239b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.326+0000 7f0d07fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4a239dce0 0x55c4a239b680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.86:38032/0 (socket says 172.21.15.86:38032) 2024-04-21T08:39:01.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.326+0000 7f0d07fff640 1 -- 172.21.15.86:0/2839094229 learned_addr learned my addr 172.21.15.86:0/2839094229 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:01.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.327+0000 7f0d06ffd640 1 --2- 172.21.15.86:0/2839094229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4a2395160 0x55c4a239b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.327+0000 7f0d077fe640 1 --2- 172.21.15.86:0/2839094229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a23947b0 0x55c4a239d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.327+0000 7f0d06ffd640 1 -- 172.21.15.86:0/2839094229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4a239dce0 msgr2=0x55c4a239b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.327+0000 7f0d06ffd640 1 --2- 172.21.15.86:0/2839094229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4a239dce0 0x55c4a239b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.327+0000 7f0d06ffd640 1 -- 172.21.15.86:0/2839094229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a23947b0 msgr2=0x55c4a239d320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.327+0000 7f0d06ffd640 1 --2- 172.21.15.86:0/2839094229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a23947b0 0x55c4a239d320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.327+0000 7f0d06ffd640 1 -- 172.21.15.86:0/2839094229 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4a23a00d0 con 0x55c4a2395160 2024-04-21T08:39:01.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.327+0000 7f0d077fe640 1 --2- 172.21.15.86:0/2839094229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a23947b0 0x55c4a239d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:01.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.328+0000 7f0d06ffd640 1 --2- 172.21.15.86:0/2839094229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4a2395160 0x55c4a239b140 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc0cdb00 tx=0x7f0cfc0985c0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=99f6acc3bbb8a621 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:01.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.330+0000 7f0d04ff9640 1 -- 172.21.15.86:0/1247081231 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f0cfc098910 con 0x7f0d0806aed0 2024-04-21T08:39:01.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.332+0000 7f0ced7fa640 1 -- 172.21.15.86:0/2839094229 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc0c1da0 con 0x55c4a2395160 2024-04-21T08:39:01.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.332+0000 7f0ced7fa640 1 -- 172.21.15.86:0/2839094229 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cfc004c30 con 0x55c4a2395160 2024-04-21T08:39:01.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.332+0000 7f0ced7fa640 1 -- 172.21.15.86:0/2839094229 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc0c1930 con 0x55c4a2395160 2024-04-21T08:39:01.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.333+0000 7f0d10c69740 1 -- 172.21.15.86:0/2839094229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4a2395160 msgr2=0x55c4a239b140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.337 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.333+0000 7f0d10c69740 1 --2- 172.21.15.86:0/2839094229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4a2395160 0x55c4a239b140 secure :-1 s=READY pgs=60 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc0cdb00 tx=0x7f0cfc0985c0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.333+0000 7f0d10c69740 1 -- 172.21.15.86:0/2839094229 shutdown_connections 2024-04-21T08:39:01.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.333+0000 7f0d10c69740 1 --2- 172.21.15.86:0/2839094229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4a239dce0 0x55c4a239b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.333+0000 7f0d10c69740 1 --2- 172.21.15.86:0/2839094229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4a2395160 0x55c4a239b140 unknown :-1 s=CLOSED pgs=60 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.333+0000 7f0d10c69740 1 --2- 172.21.15.86:0/2839094229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a23947b0 0x55c4a239d320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.333+0000 7f0d10c69740 1 -- 172.21.15.86:0/2839094229 >> 172.21.15.86:0/2839094229 conn(0x55c4a238ea60 msgr2=0x55c4a2389800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:01.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.333+0000 7f0d10c69740 1 -- 172.21.15.86:0/2839094229 shutdown_connections 2024-04-21T08:39:01.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.333+0000 7f0d10c69740 1 -- 172.21.15.86:0/2839094229 wait complete. 2024-04-21T08:39:01.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.333+0000 7f0d10c69740 1 -- start start 2024-04-21T08:39:01.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.334+0000 7f0d10c69740 1 Processor -- start 2024-04-21T08:39:01.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.334+0000 7f0d10c69740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4a23947b0 0x55c4a23b2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.334+0000 7f0d10c69740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a2395160 0x55c4a23b29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.334+0000 7f0d10c69740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4a239dce0 0x55c4a23b2ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.334+0000 7f0d10c69740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c4a23b03e0 con 0x55c4a2395160 2024-04-21T08:39:01.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.334+0000 7f0d10c69740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c4a23afd10 con 0x55c4a239dce0 2024-04-21T08:39:01.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.334+0000 7f0d10c69740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c4a23b0ab0 con 0x55c4a23947b0 2024-04-21T08:39:01.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.334+0000 7f0d10c69740 10 client.0 _ll_register_callbacks cb 0x7f0cf8009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:01.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.334+0000 7f0d06ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a2395160 0x55c4a23b29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.334+0000 7f0d06ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a2395160 0x55c4a23b29b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45968/0 (socket says 172.21.15.86:45968) 2024-04-21T08:39:01.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.334+0000 7f0d06ffd640 1 -- 172.21.15.86:0/2912776376 learned_addr learned my addr 172.21.15.86:0/2912776376 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:01.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.334+0000 7f0d077fe640 1 --2- 172.21.15.86:0/2912776376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4a23947b0 0x55c4a23b2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.334+0000 7f0d07fff640 1 --2- 172.21.15.86:0/2912776376 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4a239dce0 0x55c4a23b2ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.334+0000 7f0d06ffd640 1 -- 172.21.15.86:0/2912776376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4a23947b0 msgr2=0x55c4a23b2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.334+0000 7f0d06ffd640 1 --2- 172.21.15.86:0/2912776376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4a23947b0 0x55c4a23b2470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.334+0000 7f0d06ffd640 1 -- 172.21.15.86:0/2912776376 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4a239dce0 msgr2=0x55c4a23b2ef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.334+0000 7f0d06ffd640 1 --2- 172.21.15.86:0/2912776376 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4a239dce0 0x55c4a23b2ef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.334+0000 7f0d06ffd640 1 -- 172.21.15.86:0/2912776376 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4a23b6cd0 con 0x55c4a2395160 2024-04-21T08:39:01.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.334+0000 7f0d06ffd640 1 --2- 172.21.15.86:0/2912776376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a2395160 0x55c4a23b29b0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc004de0 tx=0x7f0cfc09c360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d83c1359bf48144f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:01.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.334+0000 7f0ce6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:01.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.334+0000 7f0ce6ffd640 1 -- 172.21.15.86:0/2912776376 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0cfc0c1b10 con 0x55c4a2395160 2024-04-21T08:39:01.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.335+0000 7f0ce6ffd640 1 -- 172.21.15.86:0/2912776376 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f0cfc09a5a0 con 0x55c4a2395160 2024-04-21T08:39:01.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.335+0000 7f0d10c69740 1 -- 172.21.15.86:0/2912776376 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0cfc004c30 con 0x55c4a2395160 2024-04-21T08:39:01.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.335+0000 7f0ce6ffd640 1 -- 172.21.15.86:0/2912776376 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0cfc0c1da0 con 0x55c4a2395160 2024-04-21T08:39:01.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.335+0000 7f0d10c69740 10 client.16005 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:01.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.335+0000 7f0d10c69740 1 -- 172.21.15.86:0/2912776376 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c4a24154b0 con 0x55c4a2395160 2024-04-21T08:39:01.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.335+0000 7f0ce6ffd640 1 -- 172.21.15.86:0/2912776376 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0cfc004c30 con 0x55c4a2395160 2024-04-21T08:39:01.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.335+0000 7f0ce6ffd640 1 -- 172.21.15.86:0/2912776376 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0cfc0bcc70 con 0x55c4a2395160 2024-04-21T08:39:01.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.335+0000 7f0d10c69740 10 client.16005 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:01.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.335+0000 7f0d10c69740 10 client.16005 resolve_mds: resolved name 'h' to daemon mds.h 2024-04-21T08:39:01.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.335+0000 7f0d10c69740 20 client.16005 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:01.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.335+0000 7f0d10c69740 1 --2- 172.21.15.86:0/2912776376 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55c4a2419b50 0x55c4a2418860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.335+0000 7f0d10c69740 4 client.16005 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:39:01.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.335+0000 7f0d10c69740 1 -- 172.21.15.86:0/2912776376 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "status"}) -- 0x55c4a23003a0 con 0x55c4a2419b50 2024-04-21T08:39:01.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.335+0000 7f0d077fe640 1 --2- 172.21.15.86:0/2912776376 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55c4a2419b50 0x55c4a2418860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.336+0000 7f0d077fe640 1 --2- 172.21.15.86:0/2912776376 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55c4a2419b50 0x55c4a2418860 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=badad4eafe3899c9 server_cookie=bd06adc48edcd908 in_seq=0 out_seq=0 2024-04-21T08:39:01.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.336+0000 7f0ce6ffd640 10 client.16005 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:39:01.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.336+0000 7f0ce6ffd640 1 -- 172.21.15.86:0/2912776376 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+327 (crc 0 0 0) 0x55c4a23003a0 con 0x55c4a2419b50 2024-04-21T08:39:01.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.336+0000 7f0ce6ffd640 10 client.16005 handle_command_reply: tid=0 2024-04-21T08:39:01.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.336+0000 7f0d10c69740 1 client.16005 shutdown 2024-04-21T08:39:01.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.336+0000 7f0d10c69740 1 -- 172.21.15.86:0/2912776376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a2395160 msgr2=0x55c4a23b29b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.336+0000 7f0d10c69740 1 --2- 172.21.15.86:0/2912776376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a2395160 0x55c4a23b29b0 secure :-1 s=READY pgs=625 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc004de0 tx=0x7f0cfc09c360 comp rx=0 tx=0).stop 2024-04-21T08:39:01.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.336+0000 7f0d10c69740 1 -- 172.21.15.86:0/2912776376 shutdown_connections 2024-04-21T08:39:01.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.336+0000 7f0d10c69740 1 --2- 172.21.15.86:0/2912776376 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55c4a2419b50 0x55c4a2418860 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.336+0000 7f0d10c69740 1 --2- 172.21.15.86:0/2912776376 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4a239dce0 0x55c4a23b2ef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.336+0000 7f0d10c69740 1 --2- 172.21.15.86:0/2912776376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4a2395160 0x55c4a23b29b0 unknown :-1 s=CLOSED pgs=625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.336+0000 7f0d10c69740 1 --2- 172.21.15.86:0/2912776376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4a23947b0 0x55c4a23b2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.336+0000 7f0d10c69740 1 -- 172.21.15.86:0/2912776376 >> 172.21.15.86:0/2912776376 conn(0x55c4a238ea60 msgr2=0x55c4a23a9010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:01.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.336+0000 7f0ce6ffd640 0 client.16005 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:39:01.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.336+0000 7f0d10c69740 1 -- 172.21.15.86:0/2912776376 shutdown_connections 2024-04-21T08:39:01.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.336+0000 7f0d10c69740 1 -- 172.21.15.86:0/2912776376 wait complete. 2024-04-21T08:39:01.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.336+0000 7f0d10c69740 20 client.16005 trim_cache size 0 max 16384 2024-04-21T08:39:01.342 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:39:01.342 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "f8cec7cc-ffb8-11ee-bc93-c7b262605968", 2024-04-21T08:39:01.342 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 2, 2024-04-21T08:39:01.342 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24455, 2024-04-21T08:39:01.342 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-21T08:39:01.342 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-21T08:39:01.342 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-21T08:39:01.343 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 135.946132799, 2024-04-21T08:39:01.343 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 29, 2024-04-21T08:39:01.343 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 94, 2024-04-21T08:39:01.343 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 93, 2024-04-21T08:39:01.343 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 178.19332903099999 2024-04-21T08:39:01.343 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:39:01.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.337+0000 7f0d0e06d640 1 -- 172.21.15.86:0/1247081231 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0cf00a3db0 msgr2=0x7f0cf00a6270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.337+0000 7f0d0e06d640 1 --2- 172.21.15.86:0/1247081231 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0cf00a3db0 0x7f0cf00a6270 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f0d000077f0 tx=0x7f0d00007e40 comp rx=0 tx=0).stop 2024-04-21T08:39:01.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.337+0000 7f0d0e06d640 1 -- 172.21.15.86:0/1247081231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d0806aed0 msgr2=0x7f0d081a0bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.337+0000 7f0d0e06d640 1 --2- 172.21.15.86:0/1247081231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d0806aed0 0x7f0d081a0bf0 secure :-1 s=READY pgs=622 cs=0 l=1 rev1=1 crypto rx=0x7f0cfc00d770 tx=0x7f0cfc00dc30 comp rx=0 tx=0).stop 2024-04-21T08:39:01.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.337+0000 7f0d0e06d640 1 -- 172.21.15.86:0/1247081231 shutdown_connections 2024-04-21T08:39:01.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.337+0000 7f0d0e06d640 1 --2- 172.21.15.86:0/1247081231 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0cf00a3db0 0x7f0cf00a6270 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.337+0000 7f0d0e06d640 1 --2- 172.21.15.86:0/1247081231 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d08107fc0 0x7f0d0819a770 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.337+0000 7f0d0e06d640 1 --2- 172.21.15.86:0/1247081231 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d0806aed0 0x7f0d081a0bf0 unknown :-1 s=CLOSED pgs=622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.337+0000 7f0d0e06d640 1 --2- 172.21.15.86:0/1247081231 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d0806a550 0x7f0d081a06b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.337+0000 7f0d0e06d640 1 -- 172.21.15.86:0/1247081231 >> 172.21.15.86:0/1247081231 conn(0x7f0d08076c50 msgr2=0x7f0d0810d140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:01.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.337+0000 7f0d0e06d640 1 -- 172.21.15.86:0/1247081231 shutdown_connections 2024-04-21T08:39:01.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.337+0000 7f0d0e06d640 1 -- 172.21.15.86:0/1247081231 wait complete. 2024-04-21T08:39:01.348 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.i status 2024-04-21T08:39:01.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.447+0000 7f5291f85640 1 -- 172.21.15.86:0/1434660858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f528c101a40 msgr2=0x7f528c105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.447+0000 7f5291f85640 1 --2- 172.21.15.86:0/1434660858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f528c101a40 0x7f528c105a80 secure :-1 s=READY pgs=626 cs=0 l=1 rev1=1 crypto rx=0x7f5280009900 tx=0x7f528002bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:01.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.447+0000 7f5291f85640 1 -- 172.21.15.86:0/1434660858 shutdown_connections 2024-04-21T08:39:01.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.447+0000 7f5291f85640 1 --2- 172.21.15.86:0/1434660858 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f528c106160 0x7f528c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.447+0000 7f5291f85640 1 --2- 172.21.15.86:0/1434660858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f528c101a40 0x7f528c105a80 unknown :-1 s=CLOSED pgs=626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.447+0000 7f5291f85640 1 --2- 172.21.15.86:0/1434660858 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f528c101090 0x7f528c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.447+0000 7f5291f85640 1 -- 172.21.15.86:0/1434660858 >> 172.21.15.86:0/1434660858 conn(0x7f528c077f50 msgr2=0x7f528c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:01.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.447+0000 7f5291f85640 1 -- 172.21.15.86:0/1434660858 shutdown_connections 2024-04-21T08:39:01.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.447+0000 7f5291f85640 1 -- 172.21.15.86:0/1434660858 wait complete. 2024-04-21T08:39:01.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.448+0000 7f5291f85640 1 Processor -- start 2024-04-21T08:39:01.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.448+0000 7f5291f85640 1 -- start start 2024-04-21T08:39:01.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.448+0000 7f5291f85640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f528c101090 0x7f528c198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.448+0000 7f5291f85640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f528c106160 0x7f528c198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.448+0000 7f5291f85640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f528c19d000 0x7f528c19d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.448+0000 7f5291f85640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f528c1110a0 con 0x7f528c101090 2024-04-21T08:39:01.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.448+0000 7f5291f85640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f528c110f20 con 0x7f528c106160 2024-04-21T08:39:01.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.448+0000 7f5291f85640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f528c111220 con 0x7f528c19d000 2024-04-21T08:39:01.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.448+0000 7f528b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f528c101090 0x7f528c198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.448+0000 7f528b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f528c101090 0x7f528c198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45984/0 (socket says 172.21.15.86:45984) 2024-04-21T08:39:01.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.448+0000 7f528b7fe640 1 -- 172.21.15.86:0/1203569147 learned_addr learned my addr 172.21.15.86:0/1203569147 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:01.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.449+0000 7f528affd640 1 --2- 172.21.15.86:0/1203569147 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f528c106160 0x7f528c198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.449+0000 7f528bfff640 1 --2- 172.21.15.86:0/1203569147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f528c19d000 0x7f528c19d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.449+0000 7f528b7fe640 1 -- 172.21.15.86:0/1203569147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f528c19d000 msgr2=0x7f528c19d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.449+0000 7f528b7fe640 1 --2- 172.21.15.86:0/1203569147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f528c19d000 0x7f528c19d4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.449+0000 7f528b7fe640 1 -- 172.21.15.86:0/1203569147 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f528c106160 msgr2=0x7f528c198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.449+0000 7f528b7fe640 1 --2- 172.21.15.86:0/1203569147 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f528c106160 0x7f528c198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.449+0000 7f528b7fe640 1 -- 172.21.15.86:0/1203569147 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f528c19daf0 con 0x7f528c101090 2024-04-21T08:39:01.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.449+0000 7f528bfff640 1 --2- 172.21.15.86:0/1203569147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f528c19d000 0x7f528c19d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:01.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.449+0000 7f528affd640 1 --2- 172.21.15.86:0/1203569147 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f528c106160 0x7f528c198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:01.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.449+0000 7f528b7fe640 1 --2- 172.21.15.86:0/1203569147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f528c101090 0x7f528c198740 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f527c00d950 tx=0x7f527c00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:01.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.449+0000 7f5288ff9640 1 -- 172.21.15.86:0/1203569147 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527c017030 con 0x7f528c101090 2024-04-21T08:39:01.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.449+0000 7f5288ff9640 1 -- 172.21.15.86:0/1203569147 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f527c004020 con 0x7f528c101090 2024-04-21T08:39:01.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.449+0000 7f5291f85640 1 -- 172.21.15.86:0/1203569147 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f528c19dde0 con 0x7f528c101090 2024-04-21T08:39:01.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.449+0000 7f5288ff9640 1 -- 172.21.15.86:0/1203569147 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527c0047a0 con 0x7f528c101090 2024-04-21T08:39:01.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.449+0000 7f5291f85640 1 -- 172.21.15.86:0/1203569147 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f528c1a55a0 con 0x7f528c101090 2024-04-21T08:39:01.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.452+0000 7f5288ff9640 1 -- 172.21.15.86:0/1203569147 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f527c009a30 con 0x7f528c101090 2024-04-21T08:39:01.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.452+0000 7f5294b71740 1 Processor -- start 2024-04-21T08:39:01.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.452+0000 7f5294b71740 1 -- start start 2024-04-21T08:39:01.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.452+0000 7f5294b71740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a1d9287b0 0x563a1d928b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.452+0000 7f5294b71740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a1d929160 0x563a1d931710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.452+0000 7f5294b71740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a1d931ce0 0x563a1d934130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.452+0000 7f5294b71740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563a1d88cb10 con 0x563a1d929160 2024-04-21T08:39:01.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.452+0000 7f5294b71740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x563a1d82a240 con 0x563a1d9287b0 2024-04-21T08:39:01.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.452+0000 7f5294b71740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563a1d888ad0 con 0x563a1d931ce0 2024-04-21T08:39:01.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.452+0000 7f528b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a1d929160 0x563a1d931710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.452+0000 7f528b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a1d929160 0x563a1d931710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45988/0 (socket says 172.21.15.86:45988) 2024-04-21T08:39:01.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.452+0000 7f528b7fe640 1 -- 172.21.15.86:0/2344408176 learned_addr learned my addr 172.21.15.86:0/2344408176 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:01.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.452+0000 7f5288ff9640 1 --2- 172.21.15.86:0/1203569147 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f52740a3e40 0x7f52740a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.452+0000 7f528affd640 1 --2- 172.21.15.86:0/2344408176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a1d931ce0 0x563a1d934130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.452+0000 7f528bfff640 1 --2- 172.21.15.86:0/2344408176 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a1d9287b0 0x563a1d928b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.452+0000 7f528bfff640 1 --2- 172.21.15.86:0/1203569147 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f52740a3e40 0x7f52740a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.453+0000 7f5288ff9640 1 -- 172.21.15.86:0/1203569147 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f527c0d0930 con 0x7f528c101090 2024-04-21T08:39:01.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.453+0000 7f528b7fe640 1 -- 172.21.15.86:0/2344408176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a1d931ce0 msgr2=0x563a1d934130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.453+0000 7f528b7fe640 1 --2- 172.21.15.86:0/2344408176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a1d931ce0 0x563a1d934130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.453+0000 7f528b7fe640 1 -- 172.21.15.86:0/2344408176 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a1d9287b0 msgr2=0x563a1d928b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.453+0000 7f528b7fe640 1 --2- 172.21.15.86:0/2344408176 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a1d9287b0 0x563a1d928b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.453+0000 7f528b7fe640 1 -- 172.21.15.86:0/2344408176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a1d59f2d0 con 0x563a1d929160 2024-04-21T08:39:01.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.453+0000 7f528bfff640 1 --2- 172.21.15.86:0/2344408176 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a1d9287b0 0x563a1d928b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:01.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.453+0000 7f528affd640 1 --2- 172.21.15.86:0/2344408176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a1d931ce0 0x563a1d934130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:01.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.453+0000 7f528b7fe640 1 --2- 172.21.15.86:0/2344408176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a1d929160 0x563a1d931710 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f527c0cdc20 tx=0x7f527c004590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec0dd9ed52b3dd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:01.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.453+0000 7f5271ffb640 1 -- 172.21.15.86:0/2344408176 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527c09bc70 con 0x563a1d929160 2024-04-21T08:39:01.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.453+0000 7f528bfff640 1 --2- 172.21.15.86:0/1203569147 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f52740a3e40 0x7f52740a6300 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5284005c40 tx=0x7f5284009200 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:01.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.453+0000 7f5271ffb640 1 -- 172.21.15.86:0/2344408176 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f527c09ca10 con 0x563a1d929160 2024-04-21T08:39:01.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.453+0000 7f5271ffb640 1 -- 172.21.15.86:0/2344408176 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527c009d20 con 0x563a1d929160 2024-04-21T08:39:01.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.454+0000 7f5294b71740 1 -- 172.21.15.86:0/2344408176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a1d929160 msgr2=0x563a1d931710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.454+0000 7f5294b71740 1 --2- 172.21.15.86:0/2344408176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a1d929160 0x563a1d931710 secure :-1 s=READY pgs=628 cs=0 l=1 rev1=1 crypto rx=0x7f527c0cdc20 tx=0x7f527c004590 comp rx=0 tx=0).stop 2024-04-21T08:39:01.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.454+0000 7f5294b71740 1 -- 172.21.15.86:0/2344408176 shutdown_connections 2024-04-21T08:39:01.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.454+0000 7f5294b71740 1 --2- 172.21.15.86:0/2344408176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a1d931ce0 0x563a1d934130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.454+0000 7f5294b71740 1 --2- 172.21.15.86:0/2344408176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a1d929160 0x563a1d931710 unknown :-1 s=CLOSED pgs=628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.454+0000 7f5294b71740 1 --2- 172.21.15.86:0/2344408176 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a1d9287b0 0x563a1d928b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.454+0000 7f5294b71740 1 -- 172.21.15.86:0/2344408176 >> 172.21.15.86:0/2344408176 conn(0x563a1d922a60 msgr2=0x563a1d922e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:01.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.454+0000 7f5294b71740 1 -- 172.21.15.86:0/2344408176 shutdown_connections 2024-04-21T08:39:01.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.454+0000 7f5294b71740 1 -- 172.21.15.86:0/2344408176 wait complete. 2024-04-21T08:39:01.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.454+0000 7f5294b71740 1 -- start start 2024-04-21T08:39:01.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.454+0000 7f5294b71740 1 Processor -- start 2024-04-21T08:39:01.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.455+0000 7f5294b71740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a1d9287b0 0x563a1d92ed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.455+0000 7f5294b71740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a1d929160 0x563a1d92f250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.455+0000 7f5294b71740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a1d931ce0 0x563a1d92f790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.455+0000 7f5294b71740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563a1d934bf0 con 0x563a1d9287b0 2024-04-21T08:39:01.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.455+0000 7f5294b71740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x563a1d815190 con 0x563a1d931ce0 2024-04-21T08:39:01.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.455+0000 7f5294b71740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563a1d934d70 con 0x563a1d929160 2024-04-21T08:39:01.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.455+0000 7f5294b71740 10 client.0 _ll_register_callbacks cb 0x563a1d8108b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:01.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.455+0000 7f528b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a1d9287b0 0x563a1d92ed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.455+0000 7f528b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a1d9287b0 0x563a1d92ed10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46004/0 (socket says 172.21.15.86:46004) 2024-04-21T08:39:01.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.455+0000 7f528b7fe640 1 -- 172.21.15.86:0/1967812720 learned_addr learned my addr 172.21.15.86:0/1967812720 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:01.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.455+0000 7f528affd640 1 --2- 172.21.15.86:0/1967812720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a1d929160 0x563a1d92f250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.455+0000 7f528b7fe640 1 -- 172.21.15.86:0/1967812720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a1d929160 msgr2=0x563a1d92f250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.455+0000 7f528b7fe640 1 --2- 172.21.15.86:0/1967812720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a1d929160 0x563a1d92f250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.455+0000 7f528b7fe640 1 -- 172.21.15.86:0/1967812720 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a1d931ce0 msgr2=0x563a1d92f790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.455+0000 7f528b7fe640 1 --2- 172.21.15.86:0/1967812720 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a1d931ce0 0x563a1d92f790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.455+0000 7f528b7fe640 1 -- 172.21.15.86:0/1967812720 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a1d6696b0 con 0x563a1d9287b0 2024-04-21T08:39:01.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.455+0000 7f528affd640 1 --2- 172.21.15.86:0/1967812720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a1d929160 0x563a1d92f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:01.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.455+0000 7f528b7fe640 1 --2- 172.21.15.86:0/1967812720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a1d9287b0 0x563a1d92ed10 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f527c09c010 tx=0x7f527c009f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3005ce36da60910 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:01.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.455+0000 7f526affd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:01.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.455+0000 7f526affd640 1 -- 172.21.15.86:0/1967812720 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527c09c310 con 0x563a1d9287b0 2024-04-21T08:39:01.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.455+0000 7f526affd640 1 -- 172.21.15.86:0/1967812720 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f527c0ce080 con 0x563a1d9287b0 2024-04-21T08:39:01.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.455+0000 7f5294b71740 1 -- 172.21.15.86:0/1967812720 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f527c09ca10 con 0x563a1d9287b0 2024-04-21T08:39:01.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.455+0000 7f526affd640 1 -- 172.21.15.86:0/1967812720 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f527c0a06c0 con 0x563a1d9287b0 2024-04-21T08:39:01.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.455+0000 7f5294b71740 10 client.16017 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:01.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.455+0000 7f5294b71740 1 -- 172.21.15.86:0/1967812720 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563a1d934bf0 con 0x563a1d9287b0 2024-04-21T08:39:01.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.455+0000 7f526affd640 1 -- 172.21.15.86:0/1967812720 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f527c09ca10 con 0x563a1d9287b0 2024-04-21T08:39:01.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.456+0000 7f526affd640 1 -- 172.21.15.86:0/1967812720 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f527c0a0120 con 0x563a1d9287b0 2024-04-21T08:39:01.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.456+0000 7f5294b71740 10 client.16017 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:01.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.456+0000 7f5294b71740 10 client.16017 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-21T08:39:01.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.456+0000 7f5294b71740 20 client.16017 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:01.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.456+0000 7f5294b71740 1 --2- 172.21.15.86:0/1967812720 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x563a1d9430f0 0x563a1d944560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.456+0000 7f5294b71740 4 client.16017 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:01.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.456+0000 7f5294b71740 1 -- 172.21.15.86:0/1967812720 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a1d89a530 con 0x563a1d9430f0 2024-04-21T08:39:01.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.456+0000 7f528affd640 1 --2- 172.21.15.86:0/1967812720 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x563a1d9430f0 0x563a1d944560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.456+0000 7f526affd640 10 client.16017 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:39:01.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.456+0000 7f528affd640 1 --2- 172.21.15.86:0/1967812720 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x563a1d9430f0 0x563a1d944560 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e22198568eae0698 server_cookie=6878a421eede8b3c in_seq=0 out_seq=0 2024-04-21T08:39:01.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.457+0000 7f526affd640 1 -- 172.21.15.86:0/1967812720 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563a1d89a530 con 0x563a1d9430f0 2024-04-21T08:39:01.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.457+0000 7f526affd640 10 client.16017 handle_command_reply: tid=0 2024-04-21T08:39:01.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.457+0000 7f5294b71740 1 client.16017 shutdown 2024-04-21T08:39:01.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.457+0000 7f5294b71740 1 -- 172.21.15.86:0/1967812720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a1d9287b0 msgr2=0x563a1d92ed10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.457+0000 7f5294b71740 1 --2- 172.21.15.86:0/1967812720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a1d9287b0 0x563a1d92ed10 secure :-1 s=READY pgs=629 cs=0 l=1 rev1=1 crypto rx=0x7f527c09c010 tx=0x7f527c009f50 comp rx=0 tx=0).stop 2024-04-21T08:39:01.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.457+0000 7f5294b71740 1 -- 172.21.15.86:0/1967812720 shutdown_connections 2024-04-21T08:39:01.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.457+0000 7f5294b71740 1 --2- 172.21.15.86:0/1967812720 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a1d931ce0 0x563a1d92f790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.457+0000 7f5294b71740 1 --2- 172.21.15.86:0/1967812720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a1d929160 0x563a1d92f250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.457+0000 7f5294b71740 1 --2- 172.21.15.86:0/1967812720 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x563a1d9430f0 0x563a1d944560 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.457+0000 7f5294b71740 1 --2- 172.21.15.86:0/1967812720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a1d9287b0 0x563a1d92ed10 unknown :-1 s=CLOSED pgs=629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.457+0000 7f526affd640 0 client.16017 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:39:01.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.457+0000 7f5294b71740 1 -- 172.21.15.86:0/1967812720 >> 172.21.15.86:0/1967812720 conn(0x563a1d922a60 msgr2=0x563a1d924bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:01.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.458+0000 7f5294b71740 1 -- 172.21.15.86:0/1967812720 shutdown_connections 2024-04-21T08:39:01.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.458+0000 7f5294b71740 1 -- 172.21.15.86:0/1967812720 wait complete. 2024-04-21T08:39:01.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.458+0000 7f5294b71740 20 client.16017 trim_cache size 0 max 16384 2024-04-21T08:39:01.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.475+0000 7f5294b71740 1 Processor -- start 2024-04-21T08:39:01.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.475+0000 7f5294b71740 1 -- start start 2024-04-21T08:39:01.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.476+0000 7f5294b71740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a1d9287b0 0x563a1d931440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.476+0000 7f5294b71740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a1d929160 0x563a1d91d800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.476+0000 7f5294b71740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a1d931ce0 0x563a1d9420a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.476+0000 7f5294b71740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563a1d9428a0 con 0x563a1d9287b0 2024-04-21T08:39:01.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.476+0000 7f5294b71740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x563a1d942f70 con 0x563a1d929160 2024-04-21T08:39:01.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.476+0000 7f5294b71740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563a1d943640 con 0x563a1d931ce0 2024-04-21T08:39:01.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.476+0000 7f528b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a1d9287b0 0x563a1d931440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.476+0000 7f528b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a1d9287b0 0x563a1d931440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46010/0 (socket says 172.21.15.86:46010) 2024-04-21T08:39:01.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.476+0000 7f528b7fe640 1 -- 172.21.15.86:0/2106691927 learned_addr learned my addr 172.21.15.86:0/2106691927 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:01.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.476+0000 7f528affd640 1 --2- 172.21.15.86:0/2106691927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a1d929160 0x563a1d91d800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.476+0000 7f528b7fe640 1 -- 172.21.15.86:0/2106691927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a1d931ce0 msgr2=0x563a1d9420a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.476+0000 7f528b7fe640 1 --2- 172.21.15.86:0/2106691927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a1d931ce0 0x563a1d9420a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.476+0000 7f528b7fe640 1 -- 172.21.15.86:0/2106691927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a1d929160 msgr2=0x563a1d91d800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.476+0000 7f528b7fe640 1 --2- 172.21.15.86:0/2106691927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a1d929160 0x563a1d91d800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.476+0000 7f528b7fe640 1 -- 172.21.15.86:0/2106691927 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a1d9237f0 con 0x563a1d9287b0 2024-04-21T08:39:01.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.476+0000 7f528b7fe640 1 --2- 172.21.15.86:0/2106691927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a1d9287b0 0x563a1d931440 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f527c0d0d30 tx=0x7f527c09a3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e046224cb7ea452 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:01.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.476+0000 7f52717fa640 1 -- 172.21.15.86:0/2106691927 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527c09bc70 con 0x563a1d9287b0 2024-04-21T08:39:01.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.476+0000 7f52717fa640 1 -- 172.21.15.86:0/2106691927 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f527c0cdad0 con 0x563a1d9287b0 2024-04-21T08:39:01.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.476+0000 7f52717fa640 1 -- 172.21.15.86:0/2106691927 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527c0a4f80 con 0x563a1d9287b0 2024-04-21T08:39:01.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.477+0000 7f5294b71740 1 -- 172.21.15.86:0/2106691927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a1d9287b0 msgr2=0x563a1d931440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.477+0000 7f5294b71740 1 --2- 172.21.15.86:0/2106691927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a1d9287b0 0x563a1d931440 secure :-1 s=READY pgs=630 cs=0 l=1 rev1=1 crypto rx=0x7f527c0d0d30 tx=0x7f527c09a3b0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.477+0000 7f5294b71740 1 -- 172.21.15.86:0/2106691927 shutdown_connections 2024-04-21T08:39:01.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.477+0000 7f5294b71740 1 --2- 172.21.15.86:0/2106691927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a1d931ce0 0x563a1d9420a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.477+0000 7f5294b71740 1 --2- 172.21.15.86:0/2106691927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a1d929160 0x563a1d91d800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.477+0000 7f5294b71740 1 --2- 172.21.15.86:0/2106691927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a1d9287b0 0x563a1d931440 unknown :-1 s=CLOSED pgs=630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.477+0000 7f5294b71740 1 -- 172.21.15.86:0/2106691927 >> 172.21.15.86:0/2106691927 conn(0x563a1d922a60 msgr2=0x563a1d922e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:01.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.477+0000 7f5294b71740 1 -- 172.21.15.86:0/2106691927 shutdown_connections 2024-04-21T08:39:01.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.477+0000 7f5294b71740 1 -- 172.21.15.86:0/2106691927 wait complete. 2024-04-21T08:39:01.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.477+0000 7f5294b71740 1 -- start start 2024-04-21T08:39:01.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.477+0000 7f5294b71740 1 Processor -- start 2024-04-21T08:39:01.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.478+0000 7f5294b71740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a1d9287b0 0x563a1d92b790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.478+0000 7f5294b71740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a1d929160 0x563a1d92bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.478+0000 7f5294b71740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a1d931ce0 0x563a1d94b7b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.478+0000 7f5294b71740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563a1d9443e0 con 0x563a1d9287b0 2024-04-21T08:39:01.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.478+0000 7f5294b71740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x563a1d943d10 con 0x563a1d929160 2024-04-21T08:39:01.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.478+0000 7f5294b71740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563a1d944ab0 con 0x563a1d931ce0 2024-04-21T08:39:01.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.478+0000 7f5294b71740 10 client.0 _ll_register_callbacks cb 0x7f5280009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:01.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.478+0000 7f528b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a1d9287b0 0x563a1d92b790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.478+0000 7f528b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a1d9287b0 0x563a1d92b790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46022/0 (socket says 172.21.15.86:46022) 2024-04-21T08:39:01.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.478+0000 7f528b7fe640 1 -- 172.21.15.86:0/3721328225 learned_addr learned my addr 172.21.15.86:0/3721328225 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:01.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.478+0000 7f528affd640 1 --2- 172.21.15.86:0/3721328225 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a1d929160 0x563a1d92bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.478+0000 7f528bfff640 1 --2- 172.21.15.86:0/3721328225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a1d931ce0 0x563a1d94b7b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.478+0000 7f528b7fe640 1 -- 172.21.15.86:0/3721328225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a1d931ce0 msgr2=0x563a1d94b7b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.478+0000 7f528b7fe640 1 --2- 172.21.15.86:0/3721328225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a1d931ce0 0x563a1d94b7b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.478+0000 7f528b7fe640 1 -- 172.21.15.86:0/3721328225 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a1d929160 msgr2=0x563a1d92bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.478+0000 7f528b7fe640 1 --2- 172.21.15.86:0/3721328225 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a1d929160 0x563a1d92bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.478+0000 7f528b7fe640 1 -- 172.21.15.86:0/3721328225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a1d94bcf0 con 0x563a1d9287b0 2024-04-21T08:39:01.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.478+0000 7f528affd640 1 --2- 172.21.15.86:0/3721328225 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a1d929160 0x563a1d92bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:01.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.478+0000 7f528b7fe640 1 --2- 172.21.15.86:0/3721328225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a1d9287b0 0x563a1d92b790 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f527c09c4d0 tx=0x7f527c004590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aba64d8e37c4d24e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:01.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.478+0000 7f526affd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:01.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.478+0000 7f526affd640 1 -- 172.21.15.86:0/3721328225 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527c09bc70 con 0x563a1d9287b0 2024-04-21T08:39:01.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.478+0000 7f526affd640 1 -- 172.21.15.86:0/3721328225 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f527c0a0120 con 0x563a1d9287b0 2024-04-21T08:39:01.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.478+0000 7f5294b71740 1 -- 172.21.15.86:0/3721328225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f527c0cdad0 con 0x563a1d9287b0 2024-04-21T08:39:01.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.478+0000 7f526affd640 1 -- 172.21.15.86:0/3721328225 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f527c09ca10 con 0x563a1d9287b0 2024-04-21T08:39:01.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.478+0000 7f5294b71740 10 client.16023 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:01.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.478+0000 7f5294b71740 1 -- 172.21.15.86:0/3721328225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563a1d94bcf0 con 0x563a1d9287b0 2024-04-21T08:39:01.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.479+0000 7f526affd640 1 -- 172.21.15.86:0/3721328225 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f527c0cdad0 con 0x563a1d9287b0 2024-04-21T08:39:01.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.479+0000 7f526affd640 1 -- 172.21.15.86:0/3721328225 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f527c098060 con 0x563a1d9287b0 2024-04-21T08:39:01.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.479+0000 7f5294b71740 10 client.16023 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:01.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.479+0000 7f5294b71740 10 client.16023 resolve_mds: resolved name 'i' to daemon mds.i 2024-04-21T08:39:01.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.479+0000 7f5294b71740 20 client.16023 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:01.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.479+0000 7f5294b71740 1 --2- 172.21.15.86:0/3721328225 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x563a1d9ad090 0x563a1d9abda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.479+0000 7f5294b71740 4 client.16023 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:39:01.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.479+0000 7f5294b71740 1 -- 172.21.15.86:0/3721328225 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "status"}) -- 0x563a1d8943a0 con 0x563a1d9ad090 2024-04-21T08:39:01.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.479+0000 7f528affd640 1 --2- 172.21.15.86:0/3721328225 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x563a1d9ad090 0x563a1d9abda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.479+0000 7f526affd640 10 client.16023 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:39:01.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.479+0000 7f528affd640 1 --2- 172.21.15.86:0/3721328225 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x563a1d9ad090 0x563a1d9abda0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ab0ed0b2871667c0 server_cookie=5f3e9d2f128a2687 in_seq=0 out_seq=0 2024-04-21T08:39:01.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.480+0000 7f526affd640 1 -- 172.21.15.86:0/3721328225 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+322 (crc 0 0 0) 0x563a1d8943a0 con 0x563a1d9ad090 2024-04-21T08:39:01.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.480+0000 7f526affd640 10 client.16023 handle_command_reply: tid=0 2024-04-21T08:39:01.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.480+0000 7f5294b71740 1 client.16023 shutdown 2024-04-21T08:39:01.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.480+0000 7f5294b71740 1 -- 172.21.15.86:0/3721328225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a1d9287b0 msgr2=0x563a1d92b790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.480+0000 7f5294b71740 1 --2- 172.21.15.86:0/3721328225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a1d9287b0 0x563a1d92b790 secure :-1 s=READY pgs=631 cs=0 l=1 rev1=1 crypto rx=0x7f527c09c4d0 tx=0x7f527c004590 comp rx=0 tx=0).stop 2024-04-21T08:39:01.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.480+0000 7f5294b71740 1 -- 172.21.15.86:0/3721328225 shutdown_connections 2024-04-21T08:39:01.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.480+0000 7f5294b71740 1 --2- 172.21.15.86:0/3721328225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a1d931ce0 0x563a1d94b7b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.480+0000 7f5294b71740 1 --2- 172.21.15.86:0/3721328225 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a1d929160 0x563a1d92bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.480+0000 7f5294b71740 1 --2- 172.21.15.86:0/3721328225 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x563a1d9ad090 0x563a1d9abda0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.480+0000 7f5294b71740 1 --2- 172.21.15.86:0/3721328225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a1d9287b0 0x563a1d92b790 unknown :-1 s=CLOSED pgs=631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.480+0000 7f526affd640 0 client.16023 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:39:01.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.480+0000 7f5294b71740 1 -- 172.21.15.86:0/3721328225 >> 172.21.15.86:0/3721328225 conn(0x563a1d922a60 msgr2=0x563a1d9420a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:01.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.480+0000 7f5294b71740 1 -- 172.21.15.86:0/3721328225 shutdown_connections 2024-04-21T08:39:01.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.480+0000 7f5294b71740 1 -- 172.21.15.86:0/3721328225 wait complete. 2024-04-21T08:39:01.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.480+0000 7f5294b71740 20 client.16023 trim_cache size 0 max 16384 2024-04-21T08:39:01.488 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:39:01.488 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "f8cec7cc-ffb8-11ee-bc93-c7b262605968", 2024-04-21T08:39:01.488 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 1, 2024-04-21T08:39:01.489 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24472, 2024-04-21T08:39:01.489 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-21T08:39:01.489 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-21T08:39:01.489 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-21T08:39:01.489 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 137.099572395, 2024-04-21T08:39:01.489 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 29, 2024-04-21T08:39:01.489 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 94, 2024-04-21T08:39:01.489 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 93, 2024-04-21T08:39:01.489 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 175.549184258 2024-04-21T08:39:01.489 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:39:01.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.480+0000 7f5291f85640 1 -- 172.21.15.86:0/1203569147 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f52740a3e40 msgr2=0x7f52740a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.480+0000 7f5291f85640 1 --2- 172.21.15.86:0/1203569147 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f52740a3e40 0x7f52740a6300 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f5284005c40 tx=0x7f5284009200 comp rx=0 tx=0).stop 2024-04-21T08:39:01.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.480+0000 7f5291f85640 1 -- 172.21.15.86:0/1203569147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f528c101090 msgr2=0x7f528c198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.480+0000 7f5291f85640 1 --2- 172.21.15.86:0/1203569147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f528c101090 0x7f528c198740 secure :-1 s=READY pgs=627 cs=0 l=1 rev1=1 crypto rx=0x7f527c00d950 tx=0x7f527c00de10 comp rx=0 tx=0).stop 2024-04-21T08:39:01.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.480+0000 7f5291f85640 1 -- 172.21.15.86:0/1203569147 shutdown_connections 2024-04-21T08:39:01.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.480+0000 7f5291f85640 1 --2- 172.21.15.86:0/1203569147 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f52740a3e40 0x7f52740a6300 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.480+0000 7f5291f85640 1 --2- 172.21.15.86:0/1203569147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f528c19d000 0x7f528c19d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.480+0000 7f5291f85640 1 --2- 172.21.15.86:0/1203569147 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f528c106160 0x7f528c198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.480+0000 7f5291f85640 1 --2- 172.21.15.86:0/1203569147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f528c101090 0x7f528c198740 unknown :-1 s=CLOSED pgs=627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.480+0000 7f5291f85640 1 -- 172.21.15.86:0/1203569147 >> 172.21.15.86:0/1203569147 conn(0x7f528c077f50 msgr2=0x7f528c100190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:01.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.480+0000 7f5291f85640 1 -- 172.21.15.86:0/1203569147 shutdown_connections 2024-04-21T08:39:01.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.480+0000 7f5291f85640 1 -- 172.21.15.86:0/1203569147 wait complete. 2024-04-21T08:39:01.492 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:01 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3744683880' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:39:01.492 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:01 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2188911756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:39:01.493 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.b status 2024-04-21T08:39:01.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.592+0000 7f3bff643640 1 -- 172.21.15.86:0/1359115981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bf8102190 msgr2=0x7f3bf8102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.592+0000 7f3bff643640 1 --2- 172.21.15.86:0/1359115981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bf8102190 0x7f3bf8102570 secure :-1 s=READY pgs=632 cs=0 l=1 rev1=1 crypto rx=0x7f3bec009900 tx=0x7f3bec02bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:01.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.592+0000 7f3bff643640 1 -- 172.21.15.86:0/1359115981 shutdown_connections 2024-04-21T08:39:01.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.592+0000 7f3bff643640 1 --2- 172.21.15.86:0/1359115981 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3bf8107260 0x7f3bf810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.592+0000 7f3bff643640 1 --2- 172.21.15.86:0/1359115981 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3bf8102b40 0x7f3bf8106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.592+0000 7f3bff643640 1 --2- 172.21.15.86:0/1359115981 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bf8102190 0x7f3bf8102570 unknown :-1 s=CLOSED pgs=632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.592+0000 7f3bff643640 1 -- 172.21.15.86:0/1359115981 >> 172.21.15.86:0/1359115981 conn(0x7f3bf80fd990 msgr2=0x7f3bf80ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:01.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.592+0000 7f3bff643640 1 -- 172.21.15.86:0/1359115981 shutdown_connections 2024-04-21T08:39:01.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.592+0000 7f3bff643640 1 -- 172.21.15.86:0/1359115981 wait complete. 2024-04-21T08:39:01.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.592+0000 7f3bff643640 1 Processor -- start 2024-04-21T08:39:01.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.592+0000 7f3bff643640 1 -- start start 2024-04-21T08:39:01.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.593+0000 7f3bff643640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3bf8102190 0x7f3bf819e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.593+0000 7f3bff643640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3bf8107260 0x7f3bf819e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.593+0000 7f3bff643640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bf8198600 0x7f3bf8198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.593+0000 7f3bff643640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf81121d0 con 0x7f3bf8198600 2024-04-21T08:39:01.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.593+0000 7f3bff643640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf8112050 con 0x7f3bf8102190 2024-04-21T08:39:01.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.593+0000 7f3bff643640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3bf8112350 con 0x7f3bf8107260 2024-04-21T08:39:01.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.593+0000 7f3bfdbe0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bf8198600 0x7f3bf8198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.606 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.593+0000 7f3bfdbe0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bf8198600 0x7f3bf8198ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46036/0 (socket says 172.21.15.86:46036) 2024-04-21T08:39:01.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.593+0000 7f3bfdbe0640 1 -- 172.21.15.86:0/3123656269 learned_addr learned my addr 172.21.15.86:0/3123656269 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:01.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.593+0000 7f3bfcbde640 1 --2- 172.21.15.86:0/3123656269 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3bf8107260 0x7f3bf819e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.593+0000 7f3bfd3df640 1 --2- 172.21.15.86:0/3123656269 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3bf8102190 0x7f3bf819e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.593+0000 7f3bfdbe0640 1 -- 172.21.15.86:0/3123656269 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3bf8107260 msgr2=0x7f3bf819e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.593+0000 7f3bfdbe0640 1 --2- 172.21.15.86:0/3123656269 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3bf8107260 0x7f3bf819e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.593+0000 7f3bfdbe0640 1 -- 172.21.15.86:0/3123656269 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3bf8102190 msgr2=0x7f3bf819e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.593+0000 7f3bfdbe0640 1 --2- 172.21.15.86:0/3123656269 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3bf8102190 0x7f3bf819e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.593+0000 7f3bfdbe0640 1 -- 172.21.15.86:0/3123656269 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3bf8199120 con 0x7f3bf8198600 2024-04-21T08:39:01.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.593+0000 7f3bfdbe0640 1 --2- 172.21.15.86:0/3123656269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bf8198600 0x7f3bf8198ab0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3bf000eaa0 tx=0x7f3bf000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:01.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.593+0000 7f3bea7fc640 1 -- 172.21.15.86:0/3123656269 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf000cdb0 con 0x7f3bf8198600 2024-04-21T08:39:01.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.593+0000 7f3bea7fc640 1 -- 172.21.15.86:0/3123656269 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bf0005020 con 0x7f3bf8198600 2024-04-21T08:39:01.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.593+0000 7f3bff643640 1 -- 172.21.15.86:0/3123656269 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3bf8199410 con 0x7f3bf8198600 2024-04-21T08:39:01.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.594+0000 7f3bea7fc640 1 -- 172.21.15.86:0/3123656269 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bf00116d0 con 0x7f3bf8198600 2024-04-21T08:39:01.607 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.594+0000 7f3bff643640 1 -- 172.21.15.86:0/3123656269 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3bf81a5610 con 0x7f3bf8198600 2024-04-21T08:39:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.596+0000 7f3bea7fc640 1 -- 172.21.15.86:0/3123656269 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f3bf00051d0 con 0x7f3bf8198600 2024-04-21T08:39:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.596+0000 7f3c02237740 1 Processor -- start 2024-04-21T08:39:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.596+0000 7f3c02237740 1 -- start start 2024-04-21T08:39:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.597+0000 7f3bea7fc640 1 --2- 172.21.15.86:0/3123656269 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f3be40a3e40 0x7f3be40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.597+0000 7f3c02237740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ea6ddb7b0 0x564ea6ddbb90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.597+0000 7f3c02237740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ea6ddc160 0x564ea6de4710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.597+0000 7f3c02237740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ea6de4ce0 0x564ea6de7130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.597+0000 7f3c02237740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564ea6d3fb10 con 0x564ea6ddb7b0 2024-04-21T08:39:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.597+0000 7f3c02237740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564ea6cdd240 con 0x564ea6ddc160 2024-04-21T08:39:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.597+0000 7f3c02237740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564ea6d3bad0 con 0x564ea6de4ce0 2024-04-21T08:39:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.597+0000 7f3bfd3df640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ea6ddb7b0 0x564ea6ddbb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.597+0000 7f3bfcbde640 1 --2- 172.21.15.86:0/3123656269 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f3be40a3e40 0x7f3be40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.597+0000 7f3bea7fc640 1 -- 172.21.15.86:0/3123656269 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f3bf00d0f60 con 0x7f3bf8198600 2024-04-21T08:39:01.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.597+0000 7f3bfd3df640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ea6ddb7b0 0x564ea6ddbb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46042/0 (socket says 172.21.15.86:46042) 2024-04-21T08:39:01.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.597+0000 7f3bfd3df640 1 -- 172.21.15.86:0/1092394527 learned_addr learned my addr 172.21.15.86:0/1092394527 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:01.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.597+0000 7f3bfdbe0640 1 --2- 172.21.15.86:0/1092394527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ea6de4ce0 0x564ea6de7130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.597+0000 7f3bfcbde640 1 --2- 172.21.15.86:0/1092394527 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ea6ddc160 0x564ea6de4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.597+0000 7f3bfd3df640 1 -- 172.21.15.86:0/1092394527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ea6de4ce0 msgr2=0x564ea6de7130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.597+0000 7f3bfd3df640 1 --2- 172.21.15.86:0/1092394527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ea6de4ce0 0x564ea6de7130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.597+0000 7f3bfd3df640 1 -- 172.21.15.86:0/1092394527 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ea6ddc160 msgr2=0x564ea6de4710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.597+0000 7f3bfd3df640 1 --2- 172.21.15.86:0/1092394527 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ea6ddc160 0x564ea6de4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.597+0000 7f3bfd3df640 1 -- 172.21.15.86:0/1092394527 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ea6a522d0 con 0x564ea6ddb7b0 2024-04-21T08:39:01.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.597+0000 7f3bfcbde640 1 --2- 172.21.15.86:0/3123656269 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f3be40a3e40 0x7f3be40a6300 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3bf400b620 tx=0x7f3bf40127c0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:01.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.597+0000 7f3bfd3df640 1 --2- 172.21.15.86:0/1092394527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ea6ddb7b0 0x564ea6ddbb90 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f3bec009900 tx=0x7f3bec005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f0ac44bfb900376 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:01.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.597+0000 7f3be37fe640 1 -- 172.21.15.86:0/1092394527 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bec044020 con 0x564ea6ddb7b0 2024-04-21T08:39:01.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.597+0000 7f3be37fe640 1 -- 172.21.15.86:0/1092394527 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bec02cc90 con 0x564ea6ddb7b0 2024-04-21T08:39:01.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.597+0000 7f3be37fe640 1 -- 172.21.15.86:0/1092394527 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bec0046c0 con 0x564ea6ddb7b0 2024-04-21T08:39:01.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.598+0000 7f3c02237740 1 -- 172.21.15.86:0/1092394527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ea6ddb7b0 msgr2=0x564ea6ddbb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.598+0000 7f3c02237740 1 --2- 172.21.15.86:0/1092394527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ea6ddb7b0 0x564ea6ddbb90 secure :-1 s=READY pgs=634 cs=0 l=1 rev1=1 crypto rx=0x7f3bec009900 tx=0x7f3bec005e70 comp rx=0 tx=0).stop 2024-04-21T08:39:01.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.598+0000 7f3c02237740 1 -- 172.21.15.86:0/1092394527 shutdown_connections 2024-04-21T08:39:01.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.598+0000 7f3c02237740 1 --2- 172.21.15.86:0/1092394527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ea6de4ce0 0x564ea6de7130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.598+0000 7f3c02237740 1 --2- 172.21.15.86:0/1092394527 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ea6ddc160 0x564ea6de4710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.598+0000 7f3c02237740 1 --2- 172.21.15.86:0/1092394527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ea6ddb7b0 0x564ea6ddbb90 unknown :-1 s=CLOSED pgs=634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.598+0000 7f3c02237740 1 -- 172.21.15.86:0/1092394527 >> 172.21.15.86:0/1092394527 conn(0x564ea6dd5a60 msgr2=0x564ea6dd5e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:01.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.598+0000 7f3c02237740 1 -- 172.21.15.86:0/1092394527 shutdown_connections 2024-04-21T08:39:01.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.598+0000 7f3c02237740 1 -- 172.21.15.86:0/1092394527 wait complete. 2024-04-21T08:39:01.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.598+0000 7f3c02237740 1 -- start start 2024-04-21T08:39:01.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.599+0000 7f3c02237740 1 Processor -- start 2024-04-21T08:39:01.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.599+0000 7f3c02237740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ea6ddb7b0 0x564ea6de1cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.599+0000 7f3c02237740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ea6ddc160 0x564ea6de21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.599+0000 7f3c02237740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ea6de4ce0 0x564ea6de2730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.599+0000 7f3c02237740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564ea6de7bf0 con 0x564ea6ddb7b0 2024-04-21T08:39:01.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.599+0000 7f3c02237740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564ea6cc8190 con 0x564ea6ddc160 2024-04-21T08:39:01.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.599+0000 7f3c02237740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564ea6de7d70 con 0x564ea6de4ce0 2024-04-21T08:39:01.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.599+0000 7f3c02237740 10 client.0 _ll_register_callbacks cb 0x564ea6cc38b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:01.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.599+0000 7f3bfd3df640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ea6ddb7b0 0x564ea6de1cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.599+0000 7f3bfd3df640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ea6ddb7b0 0x564ea6de1cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46050/0 (socket says 172.21.15.86:46050) 2024-04-21T08:39:01.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.599+0000 7f3bfd3df640 1 -- 172.21.15.86:0/1114668930 learned_addr learned my addr 172.21.15.86:0/1114668930 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:01.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.599+0000 7f3bfcbde640 1 --2- 172.21.15.86:0/1114668930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ea6ddc160 0x564ea6de21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.599+0000 7f3bfd3df640 1 -- 172.21.15.86:0/1114668930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ea6de4ce0 msgr2=0x564ea6de2730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.599+0000 7f3bfdbe0640 1 --2- 172.21.15.86:0/1114668930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ea6de4ce0 0x564ea6de2730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.599+0000 7f3bfd3df640 1 --2- 172.21.15.86:0/1114668930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ea6de4ce0 0x564ea6de2730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.599+0000 7f3bfd3df640 1 -- 172.21.15.86:0/1114668930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ea6ddc160 msgr2=0x564ea6de21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.599+0000 7f3bfd3df640 1 --2- 172.21.15.86:0/1114668930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ea6ddc160 0x564ea6de21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.599+0000 7f3bfd3df640 1 -- 172.21.15.86:0/1114668930 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ea6b1c6b0 con 0x564ea6ddb7b0 2024-04-21T08:39:01.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.599+0000 7f3bfcbde640 1 --2- 172.21.15.86:0/1114668930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ea6ddc160 0x564ea6de21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:01.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.599+0000 7f3bfd3df640 1 --2- 172.21.15.86:0/1114668930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ea6ddb7b0 0x564ea6de1cb0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f3bec02c4e0 tx=0x7f3bec02ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea29853d6f32be9b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:01.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.599+0000 7f3be0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:01.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.600+0000 7f3be0ff9640 1 -- 172.21.15.86:0/1114668930 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bec04d070 con 0x564ea6ddb7b0 2024-04-21T08:39:01.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.600+0000 7f3be0ff9640 1 -- 172.21.15.86:0/1114668930 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f3bec004460 con 0x564ea6ddb7b0 2024-04-21T08:39:01.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.600+0000 7f3c02237740 1 -- 172.21.15.86:0/1114668930 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bec02cc90 con 0x564ea6ddb7b0 2024-04-21T08:39:01.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.600+0000 7f3be0ff9640 1 -- 172.21.15.86:0/1114668930 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3bec059050 con 0x564ea6ddb7b0 2024-04-21T08:39:01.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.600+0000 7f3c02237740 10 client.16035 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:01.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.600+0000 7f3c02237740 1 -- 172.21.15.86:0/1114668930 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564ea6df3f20 con 0x564ea6ddb7b0 2024-04-21T08:39:01.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.600+0000 7f3be0ff9640 1 -- 172.21.15.86:0/1114668930 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bec02cc90 con 0x564ea6ddb7b0 2024-04-21T08:39:01.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.600+0000 7f3be0ff9640 1 -- 172.21.15.86:0/1114668930 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3bec004a20 con 0x564ea6ddb7b0 2024-04-21T08:39:01.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.600+0000 7f3c02237740 10 client.16035 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:01.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.600+0000 7f3c02237740 10 client.16035 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-21T08:39:01.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.600+0000 7f3c02237740 20 client.16035 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:01.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.600+0000 7f3c02237740 1 --2- 172.21.15.86:0/1114668930 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564ea6df6090 0x564ea6df7500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.600+0000 7f3c02237740 4 client.16035 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:01.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.600+0000 7f3c02237740 1 -- 172.21.15.86:0/1114668930 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ea6d4d530 con 0x564ea6df6090 2024-04-21T08:39:01.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.601+0000 7f3bfcbde640 1 --2- 172.21.15.86:0/1114668930 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564ea6df6090 0x564ea6df7500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.601+0000 7f3be0ff9640 10 client.16035 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:01.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.601+0000 7f3bfcbde640 1 --2- 172.21.15.86:0/1114668930 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564ea6df6090 0x564ea6df7500 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=52d8e187ee1b24e4 server_cookie=92e8ee08a9258a31 in_seq=0 out_seq=0 2024-04-21T08:39:01.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.602+0000 7f3be0ff9640 1 -- 172.21.15.86:0/1114668930 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564ea6d4d530 con 0x564ea6df6090 2024-04-21T08:39:01.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.602+0000 7f3be0ff9640 10 client.16035 handle_command_reply: tid=0 2024-04-21T08:39:01.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.602+0000 7f3c02237740 1 client.16035 shutdown 2024-04-21T08:39:01.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.602+0000 7f3c02237740 1 -- 172.21.15.86:0/1114668930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ea6ddb7b0 msgr2=0x564ea6de1cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.602+0000 7f3c02237740 1 --2- 172.21.15.86:0/1114668930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ea6ddb7b0 0x564ea6de1cb0 secure :-1 s=READY pgs=635 cs=0 l=1 rev1=1 crypto rx=0x7f3bec02c4e0 tx=0x7f3bec02ca10 comp rx=0 tx=0).stop 2024-04-21T08:39:01.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.602+0000 7f3c02237740 1 -- 172.21.15.86:0/1114668930 shutdown_connections 2024-04-21T08:39:01.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.602+0000 7f3c02237740 1 --2- 172.21.15.86:0/1114668930 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564ea6df6090 0x564ea6df7500 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.602+0000 7f3c02237740 1 --2- 172.21.15.86:0/1114668930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ea6de4ce0 0x564ea6de2730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.602+0000 7f3c02237740 1 --2- 172.21.15.86:0/1114668930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ea6ddc160 0x564ea6de21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.602+0000 7f3c02237740 1 --2- 172.21.15.86:0/1114668930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ea6ddb7b0 0x564ea6de1cb0 unknown :-1 s=CLOSED pgs=635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.602+0000 7f3be0ff9640 0 client.16035 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:01.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.602+0000 7f3c02237740 1 -- 172.21.15.86:0/1114668930 >> 172.21.15.86:0/1114668930 conn(0x564ea6dd5a60 msgr2=0x564ea6dd7bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:01.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.602+0000 7f3c02237740 1 -- 172.21.15.86:0/1114668930 shutdown_connections 2024-04-21T08:39:01.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.602+0000 7f3c02237740 1 -- 172.21.15.86:0/1114668930 wait complete. 2024-04-21T08:39:01.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.602+0000 7f3c02237740 20 client.16035 trim_cache size 0 max 16384 2024-04-21T08:39:01.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.620+0000 7f3c02237740 1 Processor -- start 2024-04-21T08:39:01.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.620+0000 7f3c02237740 1 -- start start 2024-04-21T08:39:01.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.620+0000 7f3c02237740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ea6ddb7b0 0x564ea6de43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.620+0000 7f3c02237740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ea6ddc160 0x564ea6de2220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.620+0000 7f3c02237740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ea6de4ce0 0x564ea6de2760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.620+0000 7f3c02237740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564ea6df5840 con 0x564ea6ddb7b0 2024-04-21T08:39:01.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.620+0000 7f3c02237740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564ea6df5f10 con 0x564ea6de4ce0 2024-04-21T08:39:01.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.620+0000 7f3c02237740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564ea6df65e0 con 0x564ea6ddc160 2024-04-21T08:39:01.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.620+0000 7f3bfd3df640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ea6ddb7b0 0x564ea6de43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.620+0000 7f3bfd3df640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ea6ddb7b0 0x564ea6de43e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46054/0 (socket says 172.21.15.86:46054) 2024-04-21T08:39:01.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.620+0000 7f3bfd3df640 1 -- 172.21.15.86:0/3305631359 learned_addr learned my addr 172.21.15.86:0/3305631359 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:01.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.620+0000 7f3bfcbde640 1 --2- 172.21.15.86:0/3305631359 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ea6ddc160 0x564ea6de2220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.620+0000 7f3bfdbe0640 1 --2- 172.21.15.86:0/3305631359 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ea6de4ce0 0x564ea6de2760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.620+0000 7f3bfd3df640 1 -- 172.21.15.86:0/3305631359 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ea6ddc160 msgr2=0x564ea6de2220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.620+0000 7f3bfd3df640 1 --2- 172.21.15.86:0/3305631359 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ea6ddc160 0x564ea6de2220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.620+0000 7f3bfd3df640 1 -- 172.21.15.86:0/3305631359 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ea6de4ce0 msgr2=0x564ea6de2760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.620+0000 7f3bfd3df640 1 --2- 172.21.15.86:0/3305631359 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ea6de4ce0 0x564ea6de2760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.620+0000 7f3bfd3df640 1 -- 172.21.15.86:0/3305631359 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ea6dcfd80 con 0x564ea6ddb7b0 2024-04-21T08:39:01.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.620+0000 7f3bfdbe0640 1 --2- 172.21.15.86:0/3305631359 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ea6de4ce0 0x564ea6de2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:01.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.620+0000 7f3bfd3df640 1 --2- 172.21.15.86:0/3305631359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ea6ddb7b0 0x564ea6de43e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f3bec02c4b0 tx=0x7f3bec000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2c2c9a8598248c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:01.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.621+0000 7f3be2ffd640 1 -- 172.21.15.86:0/3305631359 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bec000e60 con 0x564ea6ddb7b0 2024-04-21T08:39:01.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.621+0000 7f3be2ffd640 1 -- 172.21.15.86:0/3305631359 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bec02cc90 con 0x564ea6ddb7b0 2024-04-21T08:39:01.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.621+0000 7f3be2ffd640 1 -- 172.21.15.86:0/3305631359 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bec044070 con 0x564ea6ddb7b0 2024-04-21T08:39:01.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.621+0000 7f3c02237740 1 -- 172.21.15.86:0/3305631359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ea6ddb7b0 msgr2=0x564ea6de43e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.621+0000 7f3c02237740 1 --2- 172.21.15.86:0/3305631359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ea6ddb7b0 0x564ea6de43e0 secure :-1 s=READY pgs=636 cs=0 l=1 rev1=1 crypto rx=0x7f3bec02c4b0 tx=0x7f3bec000bc0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.621+0000 7f3c02237740 1 -- 172.21.15.86:0/3305631359 shutdown_connections 2024-04-21T08:39:01.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.621+0000 7f3c02237740 1 --2- 172.21.15.86:0/3305631359 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ea6de4ce0 0x564ea6de2760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.621+0000 7f3c02237740 1 --2- 172.21.15.86:0/3305631359 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ea6ddc160 0x564ea6de2220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.621+0000 7f3c02237740 1 --2- 172.21.15.86:0/3305631359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ea6ddb7b0 0x564ea6de43e0 unknown :-1 s=CLOSED pgs=636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.621+0000 7f3c02237740 1 -- 172.21.15.86:0/3305631359 >> 172.21.15.86:0/3305631359 conn(0x564ea6dd5a60 msgr2=0x564ea6dd0800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:01.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.621+0000 7f3c02237740 1 -- 172.21.15.86:0/3305631359 shutdown_connections 2024-04-21T08:39:01.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.621+0000 7f3c02237740 1 -- 172.21.15.86:0/3305631359 wait complete. 2024-04-21T08:39:01.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.621+0000 7f3c02237740 1 -- start start 2024-04-21T08:39:01.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.622+0000 7f3c02237740 1 Processor -- start 2024-04-21T08:39:01.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.622+0000 7f3c02237740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ea6ddb7b0 0x564ea6dff760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.622+0000 7f3c02237740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ea6ddc160 0x564ea6dffca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.622+0000 7f3c02237740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ea6de4ce0 0x564ea6dfa710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.622+0000 7f3c02237740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564ea6df7380 con 0x564ea6ddb7b0 2024-04-21T08:39:01.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.622+0000 7f3c02237740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564ea6df6cb0 con 0x564ea6de4ce0 2024-04-21T08:39:01.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.622+0000 7f3c02237740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564ea6df7a50 con 0x564ea6ddc160 2024-04-21T08:39:01.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.622+0000 7f3c02237740 10 client.0 _ll_register_callbacks cb 0x7f3bf0098be0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:01.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.622+0000 7f3bfd3df640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ea6ddb7b0 0x564ea6dff760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.622+0000 7f3bfd3df640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ea6ddb7b0 0x564ea6dff760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46064/0 (socket says 172.21.15.86:46064) 2024-04-21T08:39:01.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.622+0000 7f3bfd3df640 1 -- 172.21.15.86:0/1591544630 learned_addr learned my addr 172.21.15.86:0/1591544630 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:01.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.622+0000 7f3bfdbe0640 1 --2- 172.21.15.86:0/1591544630 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ea6de4ce0 0x564ea6dfa710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.622+0000 7f3bfcbde640 1 --2- 172.21.15.86:0/1591544630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ea6ddc160 0x564ea6dffca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.622+0000 7f3bfd3df640 1 -- 172.21.15.86:0/1591544630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ea6ddc160 msgr2=0x564ea6dffca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.622+0000 7f3bfd3df640 1 --2- 172.21.15.86:0/1591544630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ea6ddc160 0x564ea6dffca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.622+0000 7f3bfd3df640 1 -- 172.21.15.86:0/1591544630 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ea6de4ce0 msgr2=0x564ea6dfa710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:01.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.622+0000 7f3bfd3df640 1 --2- 172.21.15.86:0/1591544630 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ea6de4ce0 0x564ea6dfa710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.622+0000 7f3bfd3df640 1 -- 172.21.15.86:0/1591544630 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ea6dfae50 con 0x564ea6ddb7b0 2024-04-21T08:39:01.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.622+0000 7f3bfcbde640 1 --2- 172.21.15.86:0/1591544630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ea6ddc160 0x564ea6dffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:01.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.622+0000 7f3bfdbe0640 1 --2- 172.21.15.86:0/1591544630 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ea6de4ce0 0x564ea6dfa710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:01.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.622+0000 7f3bfd3df640 1 --2- 172.21.15.86:0/1591544630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ea6ddb7b0 0x564ea6dff760 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3bec02ca50 tx=0x7f3bec02c640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4719a28c2b97efff server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:01.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.622+0000 7f3be0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:01.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.623+0000 7f3be0ff9640 1 -- 172.21.15.86:0/1591544630 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3bec056070 con 0x564ea6ddb7b0 2024-04-21T08:39:01.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.623+0000 7f3be0ff9640 1 -- 172.21.15.86:0/1591544630 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f3bec0046c0 con 0x564ea6ddb7b0 2024-04-21T08:39:01.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.623+0000 7f3c02237740 1 -- 172.21.15.86:0/1591544630 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3bec02cc90 con 0x564ea6ddb7b0 2024-04-21T08:39:01.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.623+0000 7f3be0ff9640 1 -- 172.21.15.86:0/1591544630 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3bec044040 con 0x564ea6ddb7b0 2024-04-21T08:39:01.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.623+0000 7f3c02237740 10 client.16041 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:01.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.623+0000 7f3c02237740 1 -- 172.21.15.86:0/1591544630 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564ea6e5c9d0 con 0x564ea6ddb7b0 2024-04-21T08:39:01.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.623+0000 7f3be0ff9640 1 -- 172.21.15.86:0/1591544630 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3bec02cc90 con 0x564ea6ddb7b0 2024-04-21T08:39:01.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.623+0000 7f3be0ff9640 1 -- 172.21.15.86:0/1591544630 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3bec040530 con 0x564ea6ddb7b0 2024-04-21T08:39:01.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.623+0000 7f3c02237740 10 client.16041 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:01.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.623+0000 7f3c02237740 10 client.16041 resolve_mds: resolved name 'b' to daemon mds.b 2024-04-21T08:39:01.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.623+0000 7f3c02237740 20 client.16041 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:01.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.623+0000 7f3c02237740 1 --2- 172.21.15.86:0/1591544630 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564ea6e61190 0x564ea6e5fea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:01.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.623+0000 7f3c02237740 4 client.16041 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "status"}] 2024-04-21T08:39:01.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.623+0000 7f3c02237740 1 -- 172.21.15.86:0/1591544630 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "status"}) -- 0x564ea6d473a0 con 0x564ea6e61190 2024-04-21T08:39:01.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.623+0000 7f3bfcbde640 1 --2- 172.21.15.86:0/1591544630 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564ea6e61190 0x564ea6e5fea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:01.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.624+0000 7f3bfcbde640 1 --2- 172.21.15.86:0/1591544630 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564ea6e61190 0x564ea6e5fea0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cdf98e6c3e2d43b server_cookie=a0f75a171a9c81f3 in_seq=0 out_seq=0 2024-04-21T08:39:01.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.624+0000 7f3be0ff9640 10 client.16041 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:01.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.624+0000 7f3be0ff9640 1 -- 172.21.15.86:0/1591544630 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+332 (crc 0 0 0) 0x564ea6d473a0 con 0x564ea6e61190 2024-04-21T08:39:01.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.624+0000 7f3be0ff9640 10 client.16041 handle_command_reply: tid=0 2024-04-21T08:39:01.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.624+0000 7f3c02237740 1 client.16041 shutdown 2024-04-21T08:39:01.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.624+0000 7f3c02237740 1 -- 172.21.15.86:0/1591544630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ea6ddb7b0 msgr2=0x564ea6dff760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.624+0000 7f3c02237740 1 --2- 172.21.15.86:0/1591544630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ea6ddb7b0 0x564ea6dff760 secure :-1 s=READY pgs=637 cs=0 l=1 rev1=1 crypto rx=0x7f3bec02ca50 tx=0x7f3bec02c640 comp rx=0 tx=0).stop 2024-04-21T08:39:01.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.624+0000 7f3c02237740 1 -- 172.21.15.86:0/1591544630 shutdown_connections 2024-04-21T08:39:01.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.624+0000 7f3c02237740 1 --2- 172.21.15.86:0/1591544630 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564ea6e61190 0x564ea6e5fea0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.624+0000 7f3c02237740 1 --2- 172.21.15.86:0/1591544630 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ea6de4ce0 0x564ea6dfa710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.624+0000 7f3be0ff9640 0 client.16041 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:01.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.624+0000 7f3c02237740 1 --2- 172.21.15.86:0/1591544630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ea6ddc160 0x564ea6dffca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.624+0000 7f3c02237740 1 --2- 172.21.15.86:0/1591544630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ea6ddb7b0 0x564ea6dff760 unknown :-1 s=CLOSED pgs=637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.624+0000 7f3c02237740 1 -- 172.21.15.86:0/1591544630 >> 172.21.15.86:0/1591544630 conn(0x564ea6dd5a60 msgr2=0x564ea6df0020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:01.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.624+0000 7f3c02237740 1 -- 172.21.15.86:0/1591544630 shutdown_connections 2024-04-21T08:39:01.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.624+0000 7f3c02237740 1 -- 172.21.15.86:0/1591544630 wait complete. 2024-04-21T08:39:01.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.624+0000 7f3c02237740 20 client.16041 trim_cache size 0 max 16384 2024-04-21T08:39:01.633 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:39:01.633 INFO:teuthology.orchestra.run.smithi086.stdout: "cluster_fsid": "f8cec7cc-ffb8-11ee-bc93-c7b262605968", 2024-04-21T08:39:01.633 INFO:teuthology.orchestra.run.smithi086.stdout: "whoami": 0, 2024-04-21T08:39:01.633 INFO:teuthology.orchestra.run.smithi086.stdout: "id": 24482, 2024-04-21T08:39:01.633 INFO:teuthology.orchestra.run.smithi086.stdout: "want_state": "up:active", 2024-04-21T08:39:01.633 INFO:teuthology.orchestra.run.smithi086.stdout: "state": "up:active", 2024-04-21T08:39:01.633 INFO:teuthology.orchestra.run.smithi086.stdout: "fs_name": "cephfs", 2024-04-21T08:39:01.633 INFO:teuthology.orchestra.run.smithi086.stdout: "rank_uptime": 160.19826070600001, 2024-04-21T08:39:01.633 INFO:teuthology.orchestra.run.smithi086.stdout: "mdsmap_epoch": 29, 2024-04-21T08:39:01.633 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch": 94, 2024-04-21T08:39:01.633 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_epoch_barrier": 93, 2024-04-21T08:39:01.633 INFO:teuthology.orchestra.run.smithi086.stdout: "uptime": 167.06882134400001 2024-04-21T08:39:01.634 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:39:01.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.625+0000 7f3bff643640 1 -- 172.21.15.86:0/3123656269 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f3be40a3e40 msgr2=0x7f3be40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.625+0000 7f3bff643640 1 --2- 172.21.15.86:0/3123656269 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f3be40a3e40 0x7f3be40a6300 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f3bf400b620 tx=0x7f3bf40127c0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.625+0000 7f3bff643640 1 -- 172.21.15.86:0/3123656269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bf8198600 msgr2=0x7f3bf8198ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:01.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.625+0000 7f3bff643640 1 --2- 172.21.15.86:0/3123656269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bf8198600 0x7f3bf8198ab0 secure :-1 s=READY pgs=633 cs=0 l=1 rev1=1 crypto rx=0x7f3bf000eaa0 tx=0x7f3bf000ef60 comp rx=0 tx=0).stop 2024-04-21T08:39:01.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.625+0000 7f3bff643640 1 -- 172.21.15.86:0/3123656269 shutdown_connections 2024-04-21T08:39:01.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.625+0000 7f3bff643640 1 --2- 172.21.15.86:0/3123656269 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f3be40a3e40 0x7f3be40a6300 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.625+0000 7f3bff643640 1 --2- 172.21.15.86:0/3123656269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3bf8198600 0x7f3bf8198ab0 unknown :-1 s=CLOSED pgs=633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.625+0000 7f3bff643640 1 --2- 172.21.15.86:0/3123656269 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3bf8107260 0x7f3bf819e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.625+0000 7f3bff643640 1 --2- 172.21.15.86:0/3123656269 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3bf8102190 0x7f3bf819e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:01.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.625+0000 7f3bff643640 1 -- 172.21.15.86:0/3123656269 >> 172.21.15.86:0/3123656269 conn(0x7f3bf80fd990 msgr2=0x7f3bf80fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:01.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.625+0000 7f3bff643640 1 -- 172.21.15.86:0/3123656269 shutdown_connections 2024-04-21T08:39:01.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:01.625+0000 7f3bff643640 1 -- 172.21.15.86:0/3123656269 wait complete. 2024-04-21T08:39:01.636 INFO:tasks.quiescer:Ready to start quiesce thrashing; initial delay: 120 sec 2024-04-21T08:39:01.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:01 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3744683880' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:39:01.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:01 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2188911756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:39:01.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:01 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3744683880' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:39:01.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:01 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2188911756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:39:02.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:02 smithi086 ceph-mon[27939]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-04-21T08:39:02.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:02 smithi086 ceph-mon[27939]: mgrmap e25: x(active, since 14s), standbys: z, y 2024-04-21T08:39:02.648 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:02 smithi179 ceph-mon[31829]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-04-21T08:39:02.648 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:02 smithi179 ceph-mon[31829]: mgrmap e25: x(active, since 14s), standbys: z, y 2024-04-21T08:39:02.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:02 smithi088 ceph-mon[31259]: pgmap v10: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-04-21T08:39:02.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:02 smithi088 ceph-mon[31259]: mgrmap e25: x(active, since 14s), standbys: z, y 2024-04-21T08:39:04.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:04 smithi086 ceph-mon[27939]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-04-21T08:39:04.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:04 smithi088 ceph-mon[31259]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-04-21T08:39:04.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:04 smithi179 ceph-mon[31829]: pgmap v11: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 76 KiB/s wr, 50 op/s 2024-04-21T08:39:05.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:05 smithi086 ceph-mon[27939]: scrub summary: idle 2024-04-21T08:39:05.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:05 smithi088 ceph-mon[31259]: scrub summary: idle 2024-04-21T08:39:05.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:05 smithi179 ceph-mon[31829]: scrub summary: idle 2024-04-21T08:39:06.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:06 smithi086 ceph-mon[27939]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 99 KiB/s wr, 68 op/s 2024-04-21T08:39:06.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:06 smithi088 ceph-mon[31259]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 99 KiB/s wr, 68 op/s 2024-04-21T08:39:06.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:06 smithi179 ceph-mon[31829]: pgmap v12: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 99 KiB/s wr, 68 op/s 2024-04-21T08:39:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.066+0000 7f8ed51b1640 1 -- 172.21.15.86:0/2293033853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ed0107270 msgr2=0x7f8ed010ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:07.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.066+0000 7f8ed51b1640 1 --2- 172.21.15.86:0/2293033853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ed0107270 0x7f8ed010ee40 secure :-1 s=READY pgs=638 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8009ca0 tx=0x7f8ec802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.066+0000 7f8ed51b1640 1 -- 172.21.15.86:0/2293033853 shutdown_connections 2024-04-21T08:39:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.066+0000 7f8ed51b1640 1 --2- 172.21.15.86:0/2293033853 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ed0107270 0x7f8ed010ee40 unknown :-1 s=CLOSED pgs=638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.066+0000 7f8ed51b1640 1 --2- 172.21.15.86:0/2293033853 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ed0102b50 0x7f8ed0106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.066+0000 7f8ed51b1640 1 --2- 172.21.15.86:0/2293033853 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ed01021a0 0x7f8ed0102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.066+0000 7f8ed51b1640 1 -- 172.21.15.86:0/2293033853 >> 172.21.15.86:0/2293033853 conn(0x7f8ed00fda00 msgr2=0x7f8ed00ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.066+0000 7f8ed51b1640 1 -- 172.21.15.86:0/2293033853 shutdown_connections 2024-04-21T08:39:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.066+0000 7f8ed51b1640 1 -- 172.21.15.86:0/2293033853 wait complete. 2024-04-21T08:39:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.066+0000 7f8ed51b1640 1 Processor -- start 2024-04-21T08:39:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.067+0000 7f8ed51b1640 1 -- start start 2024-04-21T08:39:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.067+0000 7f8ed51b1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ed01021a0 0x7f8ed01985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.067+0000 7f8ed51b1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ed0107270 0x7f8ed0198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.067+0000 7f8ed51b1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ed019cea0 0x7f8ed019d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.067+0000 7f8ed51b1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed01121c0 con 0x7f8ed019cea0 2024-04-21T08:39:07.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.067+0000 7f8ed51b1640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed0112040 con 0x7f8ed01021a0 2024-04-21T08:39:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.067+0000 7f8ed51b1640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed0112340 con 0x7f8ed0107270 2024-04-21T08:39:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.067+0000 7f8ecf59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ed019cea0 0x7f8ed019d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.067+0000 7f8ecf59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ed019cea0 0x7f8ed019d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46078/0 (socket says 172.21.15.86:46078) 2024-04-21T08:39:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.067+0000 7f8ecf59e640 1 -- 172.21.15.86:0/1694306218 learned_addr learned my addr 172.21.15.86:0/1694306218 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.067+0000 7f8ece59c640 1 --2- 172.21.15.86:0/1694306218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ed0107270 0x7f8ed0198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.067+0000 7f8eced9d640 1 --2- 172.21.15.86:0/1694306218 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ed01021a0 0x7f8ed01985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.067+0000 7f8ecf59e640 1 -- 172.21.15.86:0/1694306218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ed0107270 msgr2=0x7f8ed0198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.067+0000 7f8ecf59e640 1 --2- 172.21.15.86:0/1694306218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ed0107270 0x7f8ed0198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.067+0000 7f8ecf59e640 1 -- 172.21.15.86:0/1694306218 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ed01021a0 msgr2=0x7f8ed01985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.067+0000 7f8ecf59e640 1 --2- 172.21.15.86:0/1694306218 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ed01021a0 0x7f8ed01985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.067+0000 7f8ecf59e640 1 -- 172.21.15.86:0/1694306218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed019d990 con 0x7f8ed019cea0 2024-04-21T08:39:07.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.068+0000 7f8ecf59e640 1 --2- 172.21.15.86:0/1694306218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ed019cea0 0x7f8ed019d350 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8005e30 tx=0x7f8ec8034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.068+0000 7f8eb7fff640 1 -- 172.21.15.86:0/1694306218 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8039040 con 0x7f8ed019cea0 2024-04-21T08:39:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.068+0000 7f8ed51b1640 1 -- 172.21.15.86:0/1694306218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ed019dc20 con 0x7f8ed019cea0 2024-04-21T08:39:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.068+0000 7f8eb7fff640 1 -- 172.21.15.86:0/1694306218 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ec8044070 con 0x7f8ed019cea0 2024-04-21T08:39:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.068+0000 7f8eb7fff640 1 -- 172.21.15.86:0/1694306218 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8003e60 con 0x7f8ed019cea0 2024-04-21T08:39:07.070 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.068+0000 7f8ed51b1640 1 -- 172.21.15.86:0/1694306218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ed01a5440 con 0x7f8ed019cea0 2024-04-21T08:39:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.070+0000 7f8eb7fff640 1 -- 172.21.15.86:0/1694306218 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f8ec80324c0 con 0x7f8ed019cea0 2024-04-21T08:39:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.070+0000 7f8ed7d9b740 1 Processor -- start 2024-04-21T08:39:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.070+0000 7f8ed7d9b740 1 -- start start 2024-04-21T08:39:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.071+0000 7f8ed7d9b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d751e6b8d0 0x55d751e6a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.071+0000 7f8ed7d9b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d751e6ab60 0x55d751e6afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.071+0000 7f8ed7d9b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d751e784f0 0x55d751e7a8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:07.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.071+0000 7f8ed7d9b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d751dcf8d0 con 0x55d751e6b8d0 2024-04-21T08:39:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.071+0000 7f8ed7d9b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d751d6ce80 con 0x55d751e784f0 2024-04-21T08:39:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.071+0000 7f8ed7d9b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d751dcb890 con 0x55d751e6ab60 2024-04-21T08:39:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.071+0000 7f8ece59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d751e6b8d0 0x55d751e6a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.071+0000 7f8ece59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d751e6b8d0 0x55d751e6a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46092/0 (socket says 172.21.15.86:46092) 2024-04-21T08:39:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.071+0000 7f8ece59c640 1 -- 172.21.15.86:0/1859685573 learned_addr learned my addr 172.21.15.86:0/1859685573 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.071+0000 7f8eb7fff640 1 --2- 172.21.15.86:0/1694306218 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8eb80a3ee0 0x7f8eb80a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.071+0000 7f8eced9d640 1 --2- 172.21.15.86:0/1859685573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d751e6ab60 0x55d751e6afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.071+0000 7f8ece59c640 1 --2- 172.21.15.86:0/1859685573 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d751e784f0 0x55d751e7a8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.071+0000 7f8ecf59e640 1 --2- 172.21.15.86:0/1694306218 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8eb80a3ee0 0x7f8eb80a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.071+0000 7f8ece59c640 1 -- 172.21.15.86:0/1859685573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d751e6ab60 msgr2=0x55d751e6afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.071+0000 7f8ece59c640 1 --2- 172.21.15.86:0/1859685573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d751e6ab60 0x55d751e6afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.071+0000 7f8ece59c640 1 -- 172.21.15.86:0/1859685573 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d751e784f0 msgr2=0x55d751e7a8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:07.077 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.071+0000 7f8ece59c640 1 --2- 172.21.15.86:0/1859685573 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d751e784f0 0x55d751e7a8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.071+0000 7f8ece59c640 1 -- 172.21.15.86:0/1859685573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d751d8e320 con 0x55d751e6b8d0 2024-04-21T08:39:07.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.071+0000 7f8eb7fff640 1 -- 172.21.15.86:0/1694306218 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f8ec80f2fd0 con 0x7f8ed019cea0 2024-04-21T08:39:07.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.071+0000 7f8ece59c640 1 --2- 172.21.15.86:0/1859685573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d751e6b8d0 0x55d751e6a620 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400bf60 tx=0x7f8ec400b650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cf97e626bcc4587 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:07.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.072+0000 7f8eb57fa640 1 -- 172.21.15.86:0/1859685573 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4004020 con 0x55d751e6b8d0 2024-04-21T08:39:07.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.072+0000 7f8ecf59e640 1 --2- 172.21.15.86:0/1694306218 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8eb80a3ee0 0x7f8eb80a63a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8ec80f03a0 tx=0x7f8ec80df040 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:07.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.072+0000 7f8eb57fa640 1 -- 172.21.15.86:0/1859685573 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ec4004c50 con 0x55d751e6b8d0 2024-04-21T08:39:07.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.072+0000 7f8eb57fa640 1 -- 172.21.15.86:0/1859685573 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4005220 con 0x55d751e6b8d0 2024-04-21T08:39:07.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.072+0000 7f8ed7d9b740 1 -- 172.21.15.86:0/1859685573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d751e6b8d0 msgr2=0x55d751e6a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:07.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.072+0000 7f8ed7d9b740 1 --2- 172.21.15.86:0/1859685573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d751e6b8d0 0x55d751e6a620 secure :-1 s=READY pgs=640 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400bf60 tx=0x7f8ec400b650 comp rx=0 tx=0).stop 2024-04-21T08:39:07.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.072+0000 7f8ed7d9b740 1 -- 172.21.15.86:0/1859685573 shutdown_connections 2024-04-21T08:39:07.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.072+0000 7f8ed7d9b740 1 --2- 172.21.15.86:0/1859685573 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d751e784f0 0x55d751e7a8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.072+0000 7f8ed7d9b740 1 --2- 172.21.15.86:0/1859685573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d751e6ab60 0x55d751e6afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.072+0000 7f8ed7d9b740 1 --2- 172.21.15.86:0/1859685573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d751e6b8d0 0x55d751e6a620 unknown :-1 s=CLOSED pgs=640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.078 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.072+0000 7f8ed7d9b740 1 -- 172.21.15.86:0/1859685573 >> 172.21.15.86:0/1859685573 conn(0x55d751e31c90 msgr2=0x55d751e65850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.072+0000 7f8ed7d9b740 1 -- 172.21.15.86:0/1859685573 shutdown_connections 2024-04-21T08:39:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.072+0000 7f8ed7d9b740 1 -- 172.21.15.86:0/1859685573 wait complete. 2024-04-21T08:39:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.073+0000 7f8ed7d9b740 1 -- start start 2024-04-21T08:39:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.073+0000 7f8ed7d9b740 1 Processor -- start 2024-04-21T08:39:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.073+0000 7f8ed7d9b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d751e6ab60 0x55d751e77de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.073+0000 7f8ed7d9b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d751e6b8d0 0x55d751e751e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.073+0000 7f8ed7d9b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d751e784f0 0x55d751e75720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.073+0000 7f8ed7d9b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d751e7b220 con 0x55d751e6b8d0 2024-04-21T08:39:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.073+0000 7f8ed7d9b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d751d57eb0 con 0x55d751e784f0 2024-04-21T08:39:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.073+0000 7f8ed7d9b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d751e7b3a0 con 0x55d751e6ab60 2024-04-21T08:39:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.073+0000 7f8ed7d9b740 10 client.0 _ll_register_callbacks cb 0x55d751d84b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.073+0000 7f8eced9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d751e6b8d0 0x55d751e751e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.073+0000 7f8eced9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d751e6b8d0 0x55d751e751e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46108/0 (socket says 172.21.15.86:46108) 2024-04-21T08:39:07.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.073+0000 7f8eced9d640 1 -- 172.21.15.86:0/3120522015 learned_addr learned my addr 172.21.15.86:0/3120522015 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.073+0000 7f8ece59c640 1 --2- 172.21.15.86:0/3120522015 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d751e6ab60 0x55d751e77de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.073+0000 7f8ece59c640 1 --2- 172.21.15.86:0/3120522015 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d751e784f0 0x55d751e75720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.073+0000 7f8eced9d640 1 -- 172.21.15.86:0/3120522015 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d751e6ab60 msgr2=0x55d751e77de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.073+0000 7f8eced9d640 1 --2- 172.21.15.86:0/3120522015 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d751e6ab60 0x55d751e77de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.073+0000 7f8eced9d640 1 -- 172.21.15.86:0/3120522015 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d751e784f0 msgr2=0x55d751e75720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.073+0000 7f8eced9d640 1 --2- 172.21.15.86:0/3120522015 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d751e784f0 0x55d751e75720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.073+0000 7f8eced9d640 1 -- 172.21.15.86:0/3120522015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d751ae2210 con 0x55d751e6b8d0 2024-04-21T08:39:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.073+0000 7f8eced9d640 1 --2- 172.21.15.86:0/3120522015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d751e6b8d0 0x55d751e751e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0010810 tx=0x7f8ec0010c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6569a69de0ce724b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.073+0000 7f8eb2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.074+0000 7f8eb2ff5640 1 -- 172.21.15.86:0/3120522015 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec000eb30 con 0x55d751e6b8d0 2024-04-21T08:39:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.074+0000 7f8eb2ff5640 1 -- 172.21.15.86:0/3120522015 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f8ec000ecd0 con 0x55d751e6b8d0 2024-04-21T08:39:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.074+0000 7f8ed7d9b740 1 -- 172.21.15.86:0/3120522015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ec4004c50 con 0x55d751e6b8d0 2024-04-21T08:39:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.074+0000 7f8eb2ff5640 1 -- 172.21.15.86:0/3120522015 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8ec0025050 con 0x55d751e6b8d0 2024-04-21T08:39:07.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.074+0000 7f8ed7d9b740 10 client.16053 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:07.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.074+0000 7f8ed7d9b740 1 -- 172.21.15.86:0/3120522015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d751bae620 con 0x55d751e6b8d0 2024-04-21T08:39:07.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.074+0000 7f8eb2ff5640 1 -- 172.21.15.86:0/3120522015 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ec4004c50 con 0x55d751e6b8d0 2024-04-21T08:39:07.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.074+0000 7f8eb2ff5640 1 -- 172.21.15.86:0/3120522015 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8ec00114b0 con 0x55d751e6b8d0 2024-04-21T08:39:07.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.074+0000 7f8ed7d9b740 10 client.16053 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:07.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.074+0000 7f8ed7d9b740 10 client.16053 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:07.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.074+0000 7f8ed7d9b740 20 client.16053 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:07.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.074+0000 7f8ed7d9b740 1 --2- 172.21.15.86:0/3120522015 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d751e88530 0x55d751e8a920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:07.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.074+0000 7f8ed7d9b740 4 client.16053 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:07.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.074+0000 7f8ed7d9b740 1 -- 172.21.15.86:0/3120522015 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d751de3880 con 0x55d751e88530 2024-04-21T08:39:07.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.074+0000 7f8ece59c640 1 --2- 172.21.15.86:0/3120522015 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d751e88530 0x55d751e8a920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:07.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.075+0000 7f8eb2ff5640 10 client.16053 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:07.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.075+0000 7f8ece59c640 1 --2- 172.21.15.86:0/3120522015 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d751e88530 0x55d751e8a920 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd6d0b36ce47e3c server_cookie=e523037126e58ed6 in_seq=0 out_seq=0 2024-04-21T08:39:07.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.076+0000 7f8eb2ff5640 1 -- 172.21.15.86:0/3120522015 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d751de3880 con 0x55d751e88530 2024-04-21T08:39:07.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.076+0000 7f8eb2ff5640 10 client.16053 handle_command_reply: tid=0 2024-04-21T08:39:07.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.076+0000 7f8ed7d9b740 1 client.16053 shutdown 2024-04-21T08:39:07.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.076+0000 7f8ed7d9b740 1 -- 172.21.15.86:0/3120522015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d751e6b8d0 msgr2=0x55d751e751e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:07.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.076+0000 7f8ed7d9b740 1 --2- 172.21.15.86:0/3120522015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d751e6b8d0 0x55d751e751e0 secure :-1 s=READY pgs=641 cs=0 l=1 rev1=1 crypto rx=0x7f8ec0010810 tx=0x7f8ec0010c10 comp rx=0 tx=0).stop 2024-04-21T08:39:07.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.076+0000 7f8ed7d9b740 1 -- 172.21.15.86:0/3120522015 shutdown_connections 2024-04-21T08:39:07.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.076+0000 7f8ed7d9b740 1 --2- 172.21.15.86:0/3120522015 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d751e88530 0x55d751e8a920 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.076+0000 7f8ed7d9b740 1 --2- 172.21.15.86:0/3120522015 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d751e784f0 0x55d751e75720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.076+0000 7f8ed7d9b740 1 --2- 172.21.15.86:0/3120522015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d751e6b8d0 0x55d751e751e0 unknown :-1 s=CLOSED pgs=641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.076+0000 7f8eb2ff5640 0 client.16053 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:07.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.076+0000 7f8ed7d9b740 1 --2- 172.21.15.86:0/3120522015 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d751e6ab60 0x55d751e77de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.076+0000 7f8ed7d9b740 1 -- 172.21.15.86:0/3120522015 >> 172.21.15.86:0/3120522015 conn(0x55d751e31c90 msgr2=0x55d751e675a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:07.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.076+0000 7f8ed7d9b740 1 -- 172.21.15.86:0/3120522015 shutdown_connections 2024-04-21T08:39:07.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.076+0000 7f8ed7d9b740 1 -- 172.21.15.86:0/3120522015 wait complete. 2024-04-21T08:39:07.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.076+0000 7f8ed7d9b740 20 client.16053 trim_cache size 0 max 16384 2024-04-21T08:39:07.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.099+0000 7f8ed7d9b740 1 Processor -- start 2024-04-21T08:39:07.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.099+0000 7f8ed7d9b740 1 -- start start 2024-04-21T08:39:07.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.100+0000 7f8ed7d9b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d751e6ab60 0x55d751e84150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:07.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.100+0000 7f8ed7d9b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d751e6b8d0 0x55d751e90210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:07.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.100+0000 7f8ed7d9b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d751e784f0 0x55d751e90750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:07.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.100+0000 7f8ed7d9b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed01acfb0 con 0x55d751e6ab60 2024-04-21T08:39:07.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.100+0000 7f8ed7d9b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed01aeaf0 con 0x55d751e784f0 2024-04-21T08:39:07.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.100+0000 7f8ed7d9b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed01ad680 con 0x55d751e6b8d0 2024-04-21T08:39:07.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.100+0000 7f8ece59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d751e6ab60 0x55d751e84150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:07.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.100+0000 7f8ece59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d751e6ab60 0x55d751e84150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46124/0 (socket says 172.21.15.86:46124) 2024-04-21T08:39:07.100 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.100+0000 7f8ece59c640 1 -- 172.21.15.86:0/2634295596 learned_addr learned my addr 172.21.15.86:0/2634295596 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.100+0000 7f8eced9d640 1 --2- 172.21.15.86:0/2634295596 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d751e6b8d0 0x55d751e90210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.100+0000 7f8ece59c640 1 -- 172.21.15.86:0/2634295596 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d751e6b8d0 msgr2=0x55d751e90210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.100+0000 7f8ece59c640 1 --2- 172.21.15.86:0/2634295596 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d751e6b8d0 0x55d751e90210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.100+0000 7f8ece59c640 1 -- 172.21.15.86:0/2634295596 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d751e784f0 msgr2=0x55d751e90750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.100+0000 7f8ece59c640 1 --2- 172.21.15.86:0/2634295596 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d751e784f0 0x55d751e90750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.100+0000 7f8ece59c640 1 -- 172.21.15.86:0/2634295596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d751e69e30 con 0x55d751e6ab60 2024-04-21T08:39:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.100+0000 7f8eced9d640 1 --2- 172.21.15.86:0/2634295596 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d751e6b8d0 0x55d751e90210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.100+0000 7f8ece59c640 1 --2- 172.21.15.86:0/2634295596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d751e6ab60 0x55d751e84150 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4005fd0 tx=0x7f8ec400bd10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94cc0ffbef2cf704 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.100+0000 7f8eb4ff9640 1 -- 172.21.15.86:0/2634295596 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4027040 con 0x55d751e6ab60 2024-04-21T08:39:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.101+0000 7f8eb4ff9640 1 -- 172.21.15.86:0/2634295596 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ec4031160 con 0x55d751e6ab60 2024-04-21T08:39:07.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.101+0000 7f8eb4ff9640 1 -- 172.21.15.86:0/2634295596 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4005220 con 0x55d751e6ab60 2024-04-21T08:39:07.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.101+0000 7f8ed7d9b740 1 -- 172.21.15.86:0/2634295596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d751e6ab60 msgr2=0x55d751e84150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.101+0000 7f8ed7d9b740 1 --2- 172.21.15.86:0/2634295596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d751e6ab60 0x55d751e84150 secure :-1 s=READY pgs=642 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4005fd0 tx=0x7f8ec400bd10 comp rx=0 tx=0).stop 2024-04-21T08:39:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.101+0000 7f8ed7d9b740 1 -- 172.21.15.86:0/2634295596 shutdown_connections 2024-04-21T08:39:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.101+0000 7f8ed7d9b740 1 --2- 172.21.15.86:0/2634295596 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d751e784f0 0x55d751e90750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.101+0000 7f8ed7d9b740 1 --2- 172.21.15.86:0/2634295596 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d751e6b8d0 0x55d751e90210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.101+0000 7f8ed7d9b740 1 --2- 172.21.15.86:0/2634295596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d751e6ab60 0x55d751e84150 unknown :-1 s=CLOSED pgs=642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.101+0000 7f8ed7d9b740 1 -- 172.21.15.86:0/2634295596 >> 172.21.15.86:0/2634295596 conn(0x55d751e31c90 msgr2=0x55d751e70490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.101+0000 7f8ed7d9b740 1 -- 172.21.15.86:0/2634295596 shutdown_connections 2024-04-21T08:39:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.101+0000 7f8ed7d9b740 1 -- 172.21.15.86:0/2634295596 wait complete. 2024-04-21T08:39:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.102+0000 7f8ed7d9b740 1 -- start start 2024-04-21T08:39:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.102+0000 7f8ed7d9b740 1 Processor -- start 2024-04-21T08:39:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.102+0000 7f8ed7d9b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d751e6ab60 0x55d751e80a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:07.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.102+0000 7f8ed7d9b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d751e6b8d0 0x55d751e7bda0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.102+0000 7f8ed7d9b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d751e784f0 0x55d751e7c2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.102+0000 7f8ed7d9b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed01ae420 con 0x55d751e6ab60 2024-04-21T08:39:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.102+0000 7f8ed7d9b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed01add50 con 0x55d751e6b8d0 2024-04-21T08:39:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.102+0000 7f8ed7d9b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed01aaf70 con 0x55d751e784f0 2024-04-21T08:39:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.102+0000 7f8ed7d9b740 10 client.0 _ll_register_callbacks cb 0x7f8ec000c450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.102+0000 7f8ece59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d751e6ab60 0x55d751e80a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.102+0000 7f8ece59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d751e6ab60 0x55d751e80a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46138/0 (socket says 172.21.15.86:46138) 2024-04-21T08:39:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.102+0000 7f8ece59c640 1 -- 172.21.15.86:0/2231031828 learned_addr learned my addr 172.21.15.86:0/2231031828 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.102+0000 7f8eced9d640 1 --2- 172.21.15.86:0/2231031828 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d751e6b8d0 0x55d751e7bda0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.102+0000 7f8ece59c640 1 -- 172.21.15.86:0/2231031828 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d751e784f0 msgr2=0x55d751e7c2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.102+0000 7f8ece59c640 1 --2- 172.21.15.86:0/2231031828 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d751e784f0 0x55d751e7c2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.102+0000 7f8ece59c640 1 -- 172.21.15.86:0/2231031828 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d751e6b8d0 msgr2=0x55d751e7bda0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.102+0000 7f8ece59c640 1 --2- 172.21.15.86:0/2231031828 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d751e6b8d0 0x55d751e7bda0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.102+0000 7f8ece59c640 1 -- 172.21.15.86:0/2231031828 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d751e7cac0 con 0x55d751e6ab60 2024-04-21T08:39:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.103+0000 7f8ece59c640 1 --2- 172.21.15.86:0/2231031828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d751e6ab60 0x55d751e80a30 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400b900 tx=0x7f8ec400b930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5898b670c181f124 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.103+0000 7f8eb2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.103+0000 7f8eb2ff5640 1 -- 172.21.15.86:0/2231031828 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec40314c0 con 0x55d751e6ab60 2024-04-21T08:39:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.103+0000 7f8eb2ff5640 1 -- 172.21.15.86:0/2231031828 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f8ec4031660 con 0x55d751e6ab60 2024-04-21T08:39:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.103+0000 7f8ed7d9b740 1 -- 172.21.15.86:0/2231031828 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ec4031160 con 0x55d751e6ab60 2024-04-21T08:39:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.103+0000 7f8eb2ff5640 1 -- 172.21.15.86:0/2231031828 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8ec40047e0 con 0x55d751e6ab60 2024-04-21T08:39:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.103+0000 7f8ed7d9b740 10 client.16059 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.103+0000 7f8ed7d9b740 1 -- 172.21.15.86:0/2231031828 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d751eeea10 con 0x55d751e6ab60 2024-04-21T08:39:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.103+0000 7f8eb2ff5640 1 -- 172.21.15.86:0/2231031828 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ec4031160 con 0x55d751e6ab60 2024-04-21T08:39:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.103+0000 7f8eb2ff5640 1 -- 172.21.15.86:0/2231031828 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8ec4010460 con 0x55d751e6ab60 2024-04-21T08:39:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.103+0000 7f8ed7d9b740 10 client.16059 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.103+0000 7f8ed7d9b740 10 client.16059 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.103+0000 7f8ed7d9b740 20 client.16059 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:07.107 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.103+0000 7f8ed7d9b740 1 --2- 172.21.15.86:0/2231031828 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d751eeeed0 0x55d751ef12c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:07.108 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.103+0000 7f8ed7d9b740 4 client.16059 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:39:07.108 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.103+0000 7f8ed7d9b740 1 -- 172.21.15.86:0/2231031828 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d751e6a850 con 0x55d751eeeed0 2024-04-21T08:39:07.108 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.104+0000 7f8eced9d640 1 --2- 172.21.15.86:0/2231031828 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d751eeeed0 0x55d751ef12c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:07.108 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.104+0000 7f8eced9d640 1 --2- 172.21.15.86:0/2231031828 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d751eeeed0 0x55d751ef12c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=606a81985f1be914 server_cookie=57da1d848228b683 in_seq=0 out_seq=0 2024-04-21T08:39:07.108 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.104+0000 7f8eb2ff5640 10 client.16059 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:07.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.116+0000 7f8eb2ff5640 1 -- 172.21.15.86:0/2231031828 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d751e6a850 con 0x55d751eeeed0 2024-04-21T08:39:07.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.116+0000 7f8eb2ff5640 10 client.16059 handle_command_reply: tid=0 2024-04-21T08:39:07.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.116+0000 7f8ed7d9b740 1 client.16059 shutdown 2024-04-21T08:39:07.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.117+0000 7f8ed7d9b740 1 -- 172.21.15.86:0/2231031828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d751e6ab60 msgr2=0x55d751e80a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:07.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.117+0000 7f8ed7d9b740 1 --2- 172.21.15.86:0/2231031828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d751e6ab60 0x55d751e80a30 secure :-1 s=READY pgs=643 cs=0 l=1 rev1=1 crypto rx=0x7f8ec400b900 tx=0x7f8ec400b930 comp rx=0 tx=0).stop 2024-04-21T08:39:07.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.117+0000 7f8ed7d9b740 1 -- 172.21.15.86:0/2231031828 shutdown_connections 2024-04-21T08:39:07.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.117+0000 7f8ed7d9b740 1 --2- 172.21.15.86:0/2231031828 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d751eeeed0 0x55d751ef12c0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.117+0000 7f8ed7d9b740 1 --2- 172.21.15.86:0/2231031828 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d751e784f0 0x55d751e7c2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.117+0000 7f8ed7d9b740 1 --2- 172.21.15.86:0/2231031828 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d751e6b8d0 0x55d751e7bda0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.117+0000 7f8ed7d9b740 1 --2- 172.21.15.86:0/2231031828 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d751e6ab60 0x55d751e80a30 unknown :-1 s=CLOSED pgs=643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.117+0000 7f8ed7d9b740 1 -- 172.21.15.86:0/2231031828 >> 172.21.15.86:0/2231031828 conn(0x55d751e31c90 msgr2=0x55d751e66c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:07.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.117+0000 7f8eb2ff5640 0 client.16059 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:07.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.117+0000 7f8ed7d9b740 1 -- 172.21.15.86:0/2231031828 shutdown_connections 2024-04-21T08:39:07.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.117+0000 7f8ed7d9b740 1 -- 172.21.15.86:0/2231031828 wait complete. 2024-04-21T08:39:07.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.117+0000 7f8ed7d9b740 20 client.16059 trim_cache size 0 max 16384 2024-04-21T08:39:07.118 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:39:07.118 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:39:07.118 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:39:07.118 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:39:07.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.117+0000 7f8ed51b1640 1 -- 172.21.15.86:0/1694306218 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8eb80a3ee0 msgr2=0x7f8eb80a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:07.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.117+0000 7f8ed51b1640 1 --2- 172.21.15.86:0/1694306218 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8eb80a3ee0 0x7f8eb80a63a0 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f8ec80f03a0 tx=0x7f8ec80df040 comp rx=0 tx=0).stop 2024-04-21T08:39:07.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.117+0000 7f8ed51b1640 1 -- 172.21.15.86:0/1694306218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ed019cea0 msgr2=0x7f8ed019d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:07.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.117+0000 7f8ed51b1640 1 --2- 172.21.15.86:0/1694306218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ed019cea0 0x7f8ed019d350 secure :-1 s=READY pgs=639 cs=0 l=1 rev1=1 crypto rx=0x7f8ec8005e30 tx=0x7f8ec8034760 comp rx=0 tx=0).stop 2024-04-21T08:39:07.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.117+0000 7f8ed51b1640 1 -- 172.21.15.86:0/1694306218 shutdown_connections 2024-04-21T08:39:07.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.117+0000 7f8ed51b1640 1 --2- 172.21.15.86:0/1694306218 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8eb80a3ee0 0x7f8eb80a63a0 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.117+0000 7f8ed51b1640 1 --2- 172.21.15.86:0/1694306218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ed019cea0 0x7f8ed019d350 unknown :-1 s=CLOSED pgs=639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.118+0000 7f8ed51b1640 1 --2- 172.21.15.86:0/1694306218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ed0107270 0x7f8ed0198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.119 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.118+0000 7f8ed51b1640 1 --2- 172.21.15.86:0/1694306218 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ed01021a0 0x7f8ed01985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:07.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.118+0000 7f8ed51b1640 1 -- 172.21.15.86:0/1694306218 >> 172.21.15.86:0/1694306218 conn(0x7f8ed00fda00 msgr2=0x7f8ed00fe4b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:07.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.118+0000 7f8ed51b1640 1 -- 172.21.15.86:0/1694306218 shutdown_connections 2024-04-21T08:39:07.120 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:07.118+0000 7f8ed51b1640 1 -- 172.21.15.86:0/1694306218 wait complete. 2024-04-21T08:39:08.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:08 smithi086 ceph-mon[27939]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 99 KiB/s wr, 55 op/s 2024-04-21T08:39:08.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:08 smithi088 ceph-mon[31259]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 99 KiB/s wr, 55 op/s 2024-04-21T08:39:08.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:08 smithi179 ceph-mon[31829]: pgmap v13: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 99 KiB/s wr, 55 op/s 2024-04-21T08:39:10.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:10 smithi086 ceph-mon[27939]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 101 KiB/s wr, 64 op/s 2024-04-21T08:39:10.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:10 smithi088 ceph-mon[31259]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 101 KiB/s wr, 64 op/s 2024-04-21T08:39:10.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:10 smithi179 ceph-mon[31829]: pgmap v14: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 101 KiB/s wr, 64 op/s 2024-04-21T08:39:12.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:12 smithi086 ceph-mon[27939]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 28 op/s 2024-04-21T08:39:12.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:12 smithi088 ceph-mon[31259]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 28 op/s 2024-04-21T08:39:12.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:12 smithi179 ceph-mon[31829]: pgmap v15: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 25 KiB/s wr, 28 op/s 2024-04-21T08:39:14.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:14 smithi086 ceph-mon[27939]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 31 op/s 2024-04-21T08:39:14.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:14 smithi088 ceph-mon[31259]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 31 op/s 2024-04-21T08:39:14.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:14 smithi179 ceph-mon[31829]: pgmap v16: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 31 op/s 2024-04-21T08:39:16.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:16 smithi086 ceph-mon[27939]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 31 op/s 2024-04-21T08:39:16.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:16 smithi088 ceph-mon[31259]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 31 op/s 2024-04-21T08:39:16.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:16 smithi179 ceph-mon[31829]: pgmap v17: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 25 KiB/s wr, 31 op/s 2024-04-21T08:39:18.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:18 smithi086 ceph-mon[27939]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-04-21T08:39:18.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:18 smithi088 ceph-mon[31259]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-04-21T08:39:18.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:18 smithi179 ceph-mon[31829]: pgmap v18: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-04-21T08:39:20.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.224+0000 7fee874ba640 1 -- 172.21.15.86:0/1351233115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee80106170 msgr2=0x7fee8010dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:20.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.224+0000 7fee874ba640 1 --2- 172.21.15.86:0/1351233115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee80106170 0x7fee8010dd40 secure :-1 s=READY pgs=644 cs=0 l=1 rev1=1 crypto rx=0x7fee78009ca0 tx=0x7fee7802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:39:20.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.224+0000 7fee874ba640 1 -- 172.21.15.86:0/1351233115 shutdown_connections 2024-04-21T08:39:20.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.224+0000 7fee874ba640 1 --2- 172.21.15.86:0/1351233115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee80106170 0x7fee8010dd40 unknown :-1 s=CLOSED pgs=644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:20.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.224+0000 7fee874ba640 1 --2- 172.21.15.86:0/1351233115 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee80101aa0 0x7fee80105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:20.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.224+0000 7fee874ba640 1 --2- 172.21.15.86:0/1351233115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee801010f0 0x7fee801014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:20.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.224+0000 7fee874ba640 1 -- 172.21.15.86:0/1351233115 >> 172.21.15.86:0/1351233115 conn(0x7fee80077fd0 msgr2=0x7fee800783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:20.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.224+0000 7fee874ba640 1 -- 172.21.15.86:0/1351233115 shutdown_connections 2024-04-21T08:39:20.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.224+0000 7fee874ba640 1 -- 172.21.15.86:0/1351233115 wait complete. 2024-04-21T08:39:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.225+0000 7fee874ba640 1 Processor -- start 2024-04-21T08:39:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.225+0000 7fee874ba640 1 -- start start 2024-04-21T08:39:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.225+0000 7fee874ba640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee801010f0 0x7fee80198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.225+0000 7fee874ba640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee80106170 0x7fee80198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.225+0000 7fee874ba640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee8019cfc0 0x7fee8019d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.225+0000 7fee874ba640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee801110e0 con 0x7fee8019cfc0 2024-04-21T08:39:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.225+0000 7fee874ba640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fee80110f60 con 0x7fee80106170 2024-04-21T08:39:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.225+0000 7fee874ba640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fee80111260 con 0x7fee801010f0 2024-04-21T08:39:20.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.226+0000 7fee85a57640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee8019cfc0 0x7fee8019d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.226+0000 7fee85a57640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee8019cfc0 0x7fee8019d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55352/0 (socket says 172.21.15.86:55352) 2024-04-21T08:39:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.226+0000 7fee85a57640 1 -- 172.21.15.86:0/3669969541 learned_addr learned my addr 172.21.15.86:0/3669969541 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.226+0000 7fee84a55640 1 --2- 172.21.15.86:0/3669969541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee80106170 0x7fee80198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.226+0000 7fee85256640 1 --2- 172.21.15.86:0/3669969541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee801010f0 0x7fee80198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.226+0000 7fee85a57640 1 -- 172.21.15.86:0/3669969541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee801010f0 msgr2=0x7fee80198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.226+0000 7fee85a57640 1 --2- 172.21.15.86:0/3669969541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee801010f0 0x7fee80198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.226+0000 7fee85a57640 1 -- 172.21.15.86:0/3669969541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee80106170 msgr2=0x7fee80198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.226+0000 7fee85a57640 1 --2- 172.21.15.86:0/3669969541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee80106170 0x7fee80198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.226+0000 7fee85a57640 1 -- 172.21.15.86:0/3669969541 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee8019dab0 con 0x7fee8019cfc0 2024-04-21T08:39:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.226+0000 7fee84a55640 1 --2- 172.21.15.86:0/3669969541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee80106170 0x7fee80198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.226+0000 7fee85a57640 1 --2- 172.21.15.86:0/3669969541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee8019cfc0 0x7fee8019d470 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fee78005e30 tx=0x7fee780346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.226+0000 7fee767fc640 1 -- 172.21.15.86:0/3669969541 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee78041020 con 0x7fee8019cfc0 2024-04-21T08:39:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.226+0000 7fee874ba640 1 -- 172.21.15.86:0/3669969541 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee8019dd40 con 0x7fee8019cfc0 2024-04-21T08:39:20.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.226+0000 7fee767fc640 1 -- 172.21.15.86:0/3669969541 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee78032970 con 0x7fee8019cfc0 2024-04-21T08:39:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.226+0000 7fee874ba640 1 -- 172.21.15.86:0/3669969541 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee801a5560 con 0x7fee8019cfc0 2024-04-21T08:39:20.229 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.226+0000 7fee767fc640 1 -- 172.21.15.86:0/3669969541 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee78039040 con 0x7fee8019cfc0 2024-04-21T08:39:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.229+0000 7fee8a185740 1 Processor -- start 2024-04-21T08:39:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.230+0000 7fee767fc640 1 -- 172.21.15.86:0/3669969541 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7fee78044020 con 0x7fee8019cfc0 2024-04-21T08:39:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.230+0000 7fee8a185740 1 -- start start 2024-04-21T08:39:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.230+0000 7fee8a185740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3de31e8d0 0x55f3de31d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.230+0000 7fee8a185740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f3de31db60 0x55f3de31dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:20.231 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.230+0000 7fee8a185740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f3de3270e0 0x55f3de3294d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:20.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.230+0000 7fee8a185740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f3de2828d0 con 0x55f3de31e8d0 2024-04-21T08:39:20.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.230+0000 7fee8a185740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55f3de21fe80 con 0x55f3de31db60 2024-04-21T08:39:20.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.230+0000 7fee8a185740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f3de27e890 con 0x55f3de3270e0 2024-04-21T08:39:20.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.230+0000 7fee84a55640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3de31e8d0 0x55f3de31d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:20.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.230+0000 7fee84a55640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3de31e8d0 0x55f3de31d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55358/0 (socket says 172.21.15.86:55358) 2024-04-21T08:39:20.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.230+0000 7fee84a55640 1 -- 172.21.15.86:0/158772290 learned_addr learned my addr 172.21.15.86:0/158772290 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:20.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.230+0000 7fee85256640 1 --2- 172.21.15.86:0/158772290 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f3de31db60 0x55f3de31dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:20.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.230+0000 7fee767fc640 1 --2- 172.21.15.86:0/3669969541 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fee6c0a3ee0 0x7fee6c0a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:20.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.231+0000 7fee84a55640 1 --2- 172.21.15.86:0/158772290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f3de3270e0 0x55f3de3294d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:20.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.231+0000 7fee84a55640 1 -- 172.21.15.86:0/158772290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f3de3270e0 msgr2=0x55f3de3294d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:20.232 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.231+0000 7fee84a55640 1 --2- 172.21.15.86:0/158772290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f3de3270e0 0x55f3de3294d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.231+0000 7fee84a55640 1 -- 172.21.15.86:0/158772290 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f3de31db60 msgr2=0x55f3de31dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.231+0000 7fee84a55640 1 --2- 172.21.15.86:0/158772290 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f3de31db60 0x55f3de31dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.231+0000 7fee84a55640 1 -- 172.21.15.86:0/158772290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3de241320 con 0x55f3de31e8d0 2024-04-21T08:39:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.231+0000 7fee85256640 1 --2- 172.21.15.86:0/158772290 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f3de31db60 0x55f3de31dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.231+0000 7fee85a57640 1 --2- 172.21.15.86:0/3669969541 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fee6c0a3ee0 0x7fee6c0a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.231+0000 7fee767fc640 1 -- 172.21.15.86:0/3669969541 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fee780f28d0 con 0x7fee8019cfc0 2024-04-21T08:39:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.231+0000 7fee84a55640 1 --2- 172.21.15.86:0/158772290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3de31e8d0 0x55f3de31d620 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fee700058c0 tx=0x7fee70005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdd515f769d8cff4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.231+0000 7fee6b7fe640 1 -- 172.21.15.86:0/158772290 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee70013e60 con 0x55f3de31e8d0 2024-04-21T08:39:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.231+0000 7fee6b7fe640 1 -- 172.21.15.86:0/158772290 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee7000f940 con 0x55f3de31e8d0 2024-04-21T08:39:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.231+0000 7fee6b7fe640 1 -- 172.21.15.86:0/158772290 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee70003910 con 0x55f3de31e8d0 2024-04-21T08:39:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.232+0000 7fee8a185740 1 -- 172.21.15.86:0/158772290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3de31e8d0 msgr2=0x55f3de31d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.232+0000 7fee85a57640 1 --2- 172.21.15.86:0/3669969541 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fee6c0a3ee0 0x7fee6c0a63a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fee780efca0 tx=0x7fee7808fe00 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.232+0000 7fee8a185740 1 --2- 172.21.15.86:0/158772290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3de31e8d0 0x55f3de31d620 secure :-1 s=READY pgs=646 cs=0 l=1 rev1=1 crypto rx=0x7fee700058c0 tx=0x7fee70005850 comp rx=0 tx=0).stop 2024-04-21T08:39:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.232+0000 7fee8a185740 1 -- 172.21.15.86:0/158772290 shutdown_connections 2024-04-21T08:39:20.233 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.232+0000 7fee8a185740 1 --2- 172.21.15.86:0/158772290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f3de3270e0 0x55f3de3294d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.232+0000 7fee8a185740 1 --2- 172.21.15.86:0/158772290 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f3de31db60 0x55f3de31dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.232+0000 7fee8a185740 1 --2- 172.21.15.86:0/158772290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3de31e8d0 0x55f3de31d620 unknown :-1 s=CLOSED pgs=646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.232+0000 7fee8a185740 1 -- 172.21.15.86:0/158772290 >> 172.21.15.86:0/158772290 conn(0x55f3de2e4c90 msgr2=0x55f3de318850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.232+0000 7fee8a185740 1 -- 172.21.15.86:0/158772290 shutdown_connections 2024-04-21T08:39:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.232+0000 7fee8a185740 1 -- 172.21.15.86:0/158772290 wait complete. 2024-04-21T08:39:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.232+0000 7fee8a185740 1 -- start start 2024-04-21T08:39:20.234 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.232+0000 7fee8a185740 1 Processor -- start 2024-04-21T08:39:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.233+0000 7fee8a185740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f3de31db60 0x55f3de326a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:20.236 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.233+0000 7fee8a185740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3de31e8d0 0x55f3de323dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.233+0000 7fee8a185740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f3de3270e0 0x55f3de324320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.233+0000 7fee8a185740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f3de329e10 con 0x55f3de31e8d0 2024-04-21T08:39:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.233+0000 7fee8a185740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55f3de20aeb0 con 0x55f3de3270e0 2024-04-21T08:39:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.233+0000 7fee8a185740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f3de329f90 con 0x55f3de31db60 2024-04-21T08:39:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.233+0000 7fee8a185740 10 client.0 _ll_register_callbacks cb 0x55f3de237b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.233+0000 7fee85256640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3de31e8d0 0x55f3de323dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.233+0000 7fee85256640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3de31e8d0 0x55f3de323dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55362/0 (socket says 172.21.15.86:55362) 2024-04-21T08:39:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.233+0000 7fee84a55640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f3de31db60 0x55f3de326a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.233+0000 7fee85256640 1 -- 172.21.15.86:0/1637711278 learned_addr learned my addr 172.21.15.86:0/1637711278 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.233+0000 7fee84a55640 1 --2- 172.21.15.86:0/1637711278 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f3de3270e0 0x55f3de324320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.233+0000 7fee85256640 1 -- 172.21.15.86:0/1637711278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f3de31db60 msgr2=0x55f3de326a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.233+0000 7fee85256640 1 --2- 172.21.15.86:0/1637711278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f3de31db60 0x55f3de326a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.233+0000 7fee85256640 1 -- 172.21.15.86:0/1637711278 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f3de3270e0 msgr2=0x55f3de324320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.233+0000 7fee85256640 1 --2- 172.21.15.86:0/1637711278 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f3de3270e0 0x55f3de324320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:20.237 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.233+0000 7fee85256640 1 -- 172.21.15.86:0/1637711278 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3ddf95210 con 0x55f3de31e8d0 2024-04-21T08:39:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.233+0000 7fee84a55640 1 --2- 172.21.15.86:0/1637711278 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f3de3270e0 0x55f3de324320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.233+0000 7fee85256640 1 --2- 172.21.15.86:0/1637711278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3de31e8d0 0x55f3de323dc0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fee7c012740 tx=0x7fee7c012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c234d84badf88005 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.233+0000 7fee68ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.234+0000 7fee68ff9640 1 -- 172.21.15.86:0/1637711278 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c004050 con 0x55f3de31e8d0 2024-04-21T08:39:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.234+0000 7fee68ff9640 1 -- 172.21.15.86:0/1637711278 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fee7c0041f0 con 0x55f3de31e8d0 2024-04-21T08:39:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.234+0000 7fee8a185740 1 -- 172.21.15.86:0/1637711278 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee7000f940 con 0x55f3de31e8d0 2024-04-21T08:39:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.234+0000 7fee68ff9640 1 -- 172.21.15.86:0/1637711278 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fee7c0043f0 con 0x55f3de31e8d0 2024-04-21T08:39:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.234+0000 7fee8a185740 10 client.16071 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.234+0000 7fee8a185740 1 -- 172.21.15.86:0/1637711278 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f3de061620 con 0x55f3de31e8d0 2024-04-21T08:39:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.234+0000 7fee68ff9640 1 -- 172.21.15.86:0/1637711278 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee7000f940 con 0x55f3de31e8d0 2024-04-21T08:39:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.234+0000 7fee68ff9640 1 -- 172.21.15.86:0/1637711278 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fee7c003c10 con 0x55f3de31e8d0 2024-04-21T08:39:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.234+0000 7fee8a185740 10 client.16071 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:20.238 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.234+0000 7fee8a185740 10 client.16071 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.234+0000 7fee8a185740 20 client.16071 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.234+0000 7fee8a185740 1 --2- 172.21.15.86:0/1637711278 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f3de3509b0 0x55f3de352da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.234+0000 7fee8a185740 4 client.16071 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.234+0000 7fee8a185740 1 -- 172.21.15.86:0/1637711278 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f3de296880 con 0x55f3de3509b0 2024-04-21T08:39:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.235+0000 7fee84a55640 1 --2- 172.21.15.86:0/1637711278 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f3de3509b0 0x55f3de352da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.235+0000 7fee68ff9640 10 client.16071 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.235+0000 7fee84a55640 1 --2- 172.21.15.86:0/1637711278 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f3de3509b0 0x55f3de352da0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2a71ebaa90b15da server_cookie=11408b9f841b4df4 in_seq=0 out_seq=0 2024-04-21T08:39:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.236+0000 7fee68ff9640 1 -- 172.21.15.86:0/1637711278 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f3de296880 con 0x55f3de3509b0 2024-04-21T08:39:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.236+0000 7fee68ff9640 10 client.16071 handle_command_reply: tid=0 2024-04-21T08:39:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.236+0000 7fee8a185740 1 client.16071 shutdown 2024-04-21T08:39:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.236+0000 7fee8a185740 1 -- 172.21.15.86:0/1637711278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3de31e8d0 msgr2=0x55f3de323dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.236+0000 7fee8a185740 1 --2- 172.21.15.86:0/1637711278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3de31e8d0 0x55f3de323dc0 secure :-1 s=READY pgs=647 cs=0 l=1 rev1=1 crypto rx=0x7fee7c012740 tx=0x7fee7c012c00 comp rx=0 tx=0).stop 2024-04-21T08:39:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.236+0000 7fee8a185740 1 -- 172.21.15.86:0/1637711278 shutdown_connections 2024-04-21T08:39:20.239 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.236+0000 7fee8a185740 1 --2- 172.21.15.86:0/1637711278 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f3de3509b0 0x55f3de352da0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.236+0000 7fee8a185740 1 --2- 172.21.15.86:0/1637711278 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f3de3270e0 0x55f3de324320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.236+0000 7fee8a185740 1 --2- 172.21.15.86:0/1637711278 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3de31e8d0 0x55f3de323dc0 unknown :-1 s=CLOSED pgs=647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.236+0000 7fee8a185740 1 --2- 172.21.15.86:0/1637711278 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f3de31db60 0x55f3de326a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.236+0000 7fee8a185740 1 -- 172.21.15.86:0/1637711278 >> 172.21.15.86:0/1637711278 conn(0x55f3de2e4c90 msgr2=0x55f3de31a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.236+0000 7fee68ff9640 0 client.16071 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.236+0000 7fee8a185740 1 -- 172.21.15.86:0/1637711278 shutdown_connections 2024-04-21T08:39:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.236+0000 7fee8a185740 1 -- 172.21.15.86:0/1637711278 wait complete. 2024-04-21T08:39:20.240 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.236+0000 7fee8a185740 20 client.16071 trim_cache size 0 max 16384 2024-04-21T08:39:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.251+0000 7fee8a185740 1 Processor -- start 2024-04-21T08:39:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.251+0000 7fee8a185740 1 -- start start 2024-04-21T08:39:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.251+0000 7fee8a185740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f3de31db60 0x55f3de3264f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.251+0000 7fee8a185740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3de31e8d0 0x55f3de326a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.251+0000 7fee8a185740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f3de3270e0 0x55f3de358690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.251+0000 7fee8a185740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee80073110 con 0x55f3de31e8d0 2024-04-21T08:39:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.251+0000 7fee8a185740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fee801a6330 con 0x55f3de3270e0 2024-04-21T08:39:20.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.251+0000 7fee8a185740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fee800737e0 con 0x55f3de31db60 2024-04-21T08:39:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.251+0000 7fee85256640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3de31e8d0 0x55f3de326a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.251+0000 7fee85256640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3de31e8d0 0x55f3de326a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55368/0 (socket says 172.21.15.86:55368) 2024-04-21T08:39:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.251+0000 7fee85256640 1 -- 172.21.15.86:0/415062618 learned_addr learned my addr 172.21.15.86:0/415062618 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.251+0000 7fee84a55640 1 --2- 172.21.15.86:0/415062618 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f3de31db60 0x55f3de3264f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.251+0000 7fee84a55640 1 --2- 172.21.15.86:0/415062618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f3de3270e0 0x55f3de358690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.251+0000 7fee85256640 1 -- 172.21.15.86:0/415062618 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f3de31db60 msgr2=0x55f3de3264f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.251+0000 7fee85256640 1 --2- 172.21.15.86:0/415062618 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f3de31db60 0x55f3de3264f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.251+0000 7fee85256640 1 -- 172.21.15.86:0/415062618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f3de3270e0 msgr2=0x55f3de358690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.251+0000 7fee85256640 1 --2- 172.21.15.86:0/415062618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f3de3270e0 0x55f3de358690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.251+0000 7fee85256640 1 -- 172.21.15.86:0/415062618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f3de31ce30 con 0x55f3de31e8d0 2024-04-21T08:39:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.251+0000 7fee84a55640 1 --2- 172.21.15.86:0/415062618 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f3de31db60 0x55f3de3264f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.252+0000 7fee85256640 1 --2- 172.21.15.86:0/415062618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3de31e8d0 0x55f3de326a30 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fee7c012420 tx=0x7fee7c009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43e6819f14f72135 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.252+0000 7fee6affd640 1 -- 172.21.15.86:0/415062618 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c02d020 con 0x55f3de31e8d0 2024-04-21T08:39:20.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.252+0000 7fee6affd640 1 -- 172.21.15.86:0/415062618 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee7c012e50 con 0x55f3de31e8d0 2024-04-21T08:39:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.252+0000 7fee6affd640 1 -- 172.21.15.86:0/415062618 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee7c005040 con 0x55f3de31e8d0 2024-04-21T08:39:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.252+0000 7fee8a185740 1 -- 172.21.15.86:0/415062618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3de31e8d0 msgr2=0x55f3de326a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.253+0000 7fee8a185740 1 --2- 172.21.15.86:0/415062618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3de31e8d0 0x55f3de326a30 secure :-1 s=READY pgs=648 cs=0 l=1 rev1=1 crypto rx=0x7fee7c012420 tx=0x7fee7c009680 comp rx=0 tx=0).stop 2024-04-21T08:39:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.253+0000 7fee8a185740 1 -- 172.21.15.86:0/415062618 shutdown_connections 2024-04-21T08:39:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.253+0000 7fee8a185740 1 --2- 172.21.15.86:0/415062618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f3de3270e0 0x55f3de358690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.253+0000 7fee8a185740 1 --2- 172.21.15.86:0/415062618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3de31e8d0 0x55f3de326a30 unknown :-1 s=CLOSED pgs=648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.253+0000 7fee8a185740 1 --2- 172.21.15.86:0/415062618 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f3de31db60 0x55f3de3264f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.253+0000 7fee8a185740 1 -- 172.21.15.86:0/415062618 >> 172.21.15.86:0/415062618 conn(0x55f3de2e4c90 msgr2=0x55f3de31f080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.253+0000 7fee8a185740 1 -- 172.21.15.86:0/415062618 shutdown_connections 2024-04-21T08:39:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.253+0000 7fee8a185740 1 -- 172.21.15.86:0/415062618 wait complete. 2024-04-21T08:39:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.253+0000 7fee8a185740 1 -- start start 2024-04-21T08:39:20.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.253+0000 7fee8a185740 1 Processor -- start 2024-04-21T08:39:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.253+0000 7fee8a185740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f3de31db60 0x55f3de337ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.253+0000 7fee8a185740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f3de31e8d0 0x55f3de338020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.253+0000 7fee8a185740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3de3270e0 0x55f3de320070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.253+0000 7fee8a185740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee801a5c60 con 0x55f3de3270e0 2024-04-21T08:39:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.253+0000 7fee8a185740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fee80073eb0 con 0x55f3de31e8d0 2024-04-21T08:39:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.253+0000 7fee8a185740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fee800710d0 con 0x55f3de31db60 2024-04-21T08:39:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.254+0000 7fee8a185740 10 client.0 _ll_register_callbacks cb 0x7fee7c009910 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.254+0000 7fee84a55640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3de3270e0 0x55f3de320070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.254+0000 7fee84a55640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3de3270e0 0x55f3de320070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55374/0 (socket says 172.21.15.86:55374) 2024-04-21T08:39:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.254+0000 7fee84a55640 1 -- 172.21.15.86:0/3637393532 learned_addr learned my addr 172.21.15.86:0/3637393532 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:20.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.254+0000 7fee85256640 1 --2- 172.21.15.86:0/3637393532 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f3de31e8d0 0x55f3de338020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.254+0000 7fee84a55640 1 --2- 172.21.15.86:0/3637393532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f3de31db60 0x55f3de337ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.254+0000 7fee84a55640 1 -- 172.21.15.86:0/3637393532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f3de31db60 msgr2=0x55f3de337ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.254+0000 7fee84a55640 1 --2- 172.21.15.86:0/3637393532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f3de31db60 0x55f3de337ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.254+0000 7fee84a55640 1 -- 172.21.15.86:0/3637393532 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f3de31e8d0 msgr2=0x55f3de338020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.254+0000 7fee84a55640 1 --2- 172.21.15.86:0/3637393532 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f3de31e8d0 0x55f3de338020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.254+0000 7fee84a55640 1 -- 172.21.15.86:0/3637393532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3de320630 con 0x55f3de3270e0 2024-04-21T08:39:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.254+0000 7fee84a55640 1 --2- 172.21.15.86:0/3637393532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3de3270e0 0x55f3de320070 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fee70009fd0 tx=0x7fee7c009e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a95a91a44cbce15 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.254+0000 7fee68ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.254+0000 7fee68ff9640 1 -- 172.21.15.86:0/3637393532 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee70025030 con 0x55f3de3270e0 2024-04-21T08:39:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.254+0000 7fee8a185740 1 -- 172.21.15.86:0/3637393532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee7c012e50 con 0x55f3de3270e0 2024-04-21T08:39:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.254+0000 7fee68ff9640 1 -- 172.21.15.86:0/3637393532 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fee7002fae0 con 0x55f3de3270e0 2024-04-21T08:39:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.254+0000 7fee68ff9640 1 -- 172.21.15.86:0/3637393532 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fee700050b0 con 0x55f3de3270e0 2024-04-21T08:39:20.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.255+0000 7fee8a185740 10 client.16077 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.255+0000 7fee8a185740 1 -- 172.21.15.86:0/3637393532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f3de3b66e0 con 0x55f3de3270e0 2024-04-21T08:39:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.255+0000 7fee68ff9640 1 -- 172.21.15.86:0/3637393532 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee7c012e50 con 0x55f3de3270e0 2024-04-21T08:39:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.255+0000 7fee68ff9640 1 -- 172.21.15.86:0/3637393532 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fee7002fce0 con 0x55f3de3270e0 2024-04-21T08:39:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.255+0000 7fee8a185740 10 client.16077 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.255+0000 7fee8a185740 10 client.16077 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.255+0000 7fee8a185740 20 client.16077 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.255+0000 7fee8a185740 1 --2- 172.21.15.86:0/3637393532 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f3de3b6bd0 0x55f3de3b8ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.255+0000 7fee8a185740 4 client.16077 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:39:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.255+0000 7fee8a185740 1 -- 172.21.15.86:0/3637393532 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f3de2223d0 con 0x55f3de3b6bd0 2024-04-21T08:39:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.255+0000 7fee85256640 1 --2- 172.21.15.86:0/3637393532 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f3de3b6bd0 0x55f3de3b8ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.256+0000 7fee68ff9640 10 client.16077 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:20.259 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.256+0000 7fee85256640 1 --2- 172.21.15.86:0/3637393532 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f3de3b6bd0 0x55f3de3b8ff0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7f6cf88598e5562 server_cookie=59ef04e8ce9c0933 in_seq=0 out_seq=0 2024-04-21T08:39:20.260 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.259+0000 7fee68ff9640 1 -- 172.21.15.86:0/3637393532 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f3de2223d0 con 0x55f3de3b6bd0 2024-04-21T08:39:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.259+0000 7fee68ff9640 10 client.16077 handle_command_reply: tid=0 2024-04-21T08:39:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.259+0000 7fee8a185740 1 client.16077 shutdown 2024-04-21T08:39:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.259+0000 7fee8a185740 1 -- 172.21.15.86:0/3637393532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3de3270e0 msgr2=0x55f3de320070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.259+0000 7fee8a185740 1 --2- 172.21.15.86:0/3637393532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3de3270e0 0x55f3de320070 secure :-1 s=READY pgs=649 cs=0 l=1 rev1=1 crypto rx=0x7fee70009fd0 tx=0x7fee7c009e90 comp rx=0 tx=0).stop 2024-04-21T08:39:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.259+0000 7fee8a185740 1 -- 172.21.15.86:0/3637393532 shutdown_connections 2024-04-21T08:39:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.259+0000 7fee8a185740 1 --2- 172.21.15.86:0/3637393532 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f3de3b6bd0 0x55f3de3b8ff0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.260+0000 7fee8a185740 1 --2- 172.21.15.86:0/3637393532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f3de3270e0 0x55f3de320070 unknown :-1 s=CLOSED pgs=649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.260+0000 7fee8a185740 1 --2- 172.21.15.86:0/3637393532 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f3de31e8d0 0x55f3de338020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.260+0000 7fee8a185740 1 --2- 172.21.15.86:0/3637393532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f3de31db60 0x55f3de337ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.260+0000 7fee8a185740 1 -- 172.21.15.86:0/3637393532 >> 172.21.15.86:0/3637393532 conn(0x55f3de2e4c90 msgr2=0x55f3de3235f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.260+0000 7fee68ff9640 0 client.16077 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.260+0000 7fee8a185740 1 -- 172.21.15.86:0/3637393532 shutdown_connections 2024-04-21T08:39:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.260+0000 7fee8a185740 1 -- 172.21.15.86:0/3637393532 wait complete. 2024-04-21T08:39:20.261 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.260+0000 7fee8a185740 20 client.16077 trim_cache size 0 max 16384 2024-04-21T08:39:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:39:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:39:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:39:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:39:20.262 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.260+0000 7fee874ba640 1 -- 172.21.15.86:0/3669969541 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fee6c0a3ee0 msgr2=0x7fee6c0a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.260+0000 7fee874ba640 1 --2- 172.21.15.86:0/3669969541 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fee6c0a3ee0 0x7fee6c0a63a0 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fee780efca0 tx=0x7fee7808fe00 comp rx=0 tx=0).stop 2024-04-21T08:39:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.260+0000 7fee874ba640 1 -- 172.21.15.86:0/3669969541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee8019cfc0 msgr2=0x7fee8019d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.260+0000 7fee874ba640 1 --2- 172.21.15.86:0/3669969541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee8019cfc0 0x7fee8019d470 secure :-1 s=READY pgs=645 cs=0 l=1 rev1=1 crypto rx=0x7fee78005e30 tx=0x7fee780346d0 comp rx=0 tx=0).stop 2024-04-21T08:39:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.260+0000 7fee874ba640 1 -- 172.21.15.86:0/3669969541 shutdown_connections 2024-04-21T08:39:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.260+0000 7fee874ba640 1 --2- 172.21.15.86:0/3669969541 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fee6c0a3ee0 0x7fee6c0a63a0 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.260+0000 7fee874ba640 1 --2- 172.21.15.86:0/3669969541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee8019cfc0 0x7fee8019d470 unknown :-1 s=CLOSED pgs=645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.260+0000 7fee874ba640 1 --2- 172.21.15.86:0/3669969541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee80106170 0x7fee80198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.260+0000 7fee874ba640 1 --2- 172.21.15.86:0/3669969541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee801010f0 0x7fee80198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.260+0000 7fee874ba640 1 -- 172.21.15.86:0/3669969541 >> 172.21.15.86:0/3669969541 conn(0x7fee80077fd0 msgr2=0x7fee80069990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.261+0000 7fee874ba640 1 -- 172.21.15.86:0/3669969541 shutdown_connections 2024-04-21T08:39:20.263 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:20.261+0000 7fee874ba640 1 -- 172.21.15.86:0/3669969541 wait complete. 2024-04-21T08:39:20.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:20 smithi086 ceph-mon[27939]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.8 KiB/s wr, 17 op/s 2024-04-21T08:39:20.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:20 smithi088 ceph-mon[31259]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.8 KiB/s wr, 17 op/s 2024-04-21T08:39:20.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:20 smithi179 ceph-mon[31829]: pgmap v19: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 1.8 KiB/s wr, 17 op/s 2024-04-21T08:39:22.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:22 smithi086 ceph-mon[27939]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-21T08:39:22.667 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:39:22.667 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:39:22.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:22 smithi088 ceph-mon[31259]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-21T08:39:22.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:22 smithi179 ceph-mon[31829]: pgmap v20: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-21T08:39:22.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.827+0000 7fd9a3a81640 1 -- 172.21.15.86:0/546506405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd99c101570 msgr2=0x7fd99c1019f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:22.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.827+0000 7fd9a3a81640 1 --2- 172.21.15.86:0/546506405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd99c101570 0x7fd99c1019f0 secure :-1 s=READY pgs=650 cs=0 l=1 rev1=1 crypto rx=0x7fd98c009900 tx=0x7fd98c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:22.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.827+0000 7fd9a3a81640 1 -- 172.21.15.86:0/546506405 shutdown_connections 2024-04-21T08:39:22.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.827+0000 7fd9a3a81640 1 --2- 172.21.15.86:0/546506405 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd99c101f30 0x7fd99c10dce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:22.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.827+0000 7fd9a3a81640 1 --2- 172.21.15.86:0/546506405 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd99c101570 0x7fd99c1019f0 unknown :-1 s=CLOSED pgs=650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:22.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.827+0000 7fd9a3a81640 1 --2- 172.21.15.86:0/546506405 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd99c105e50 0x7fd99c106230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:22.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.827+0000 7fd9a3a81640 1 -- 172.21.15.86:0/546506405 >> 172.21.15.86:0/546506405 conn(0x7fd99c0773b0 msgr2=0x7fd99c0fe770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:22.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.827+0000 7fd9a3a81640 1 -- 172.21.15.86:0/546506405 shutdown_connections 2024-04-21T08:39:22.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.827+0000 7fd9a3a81640 1 -- 172.21.15.86:0/546506405 wait complete. 2024-04-21T08:39:22.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.828+0000 7fd9a3a81640 1 Processor -- start 2024-04-21T08:39:22.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.828+0000 7fd9a3a81640 1 -- start start 2024-04-21T08:39:22.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.829+0000 7fd9a3a81640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd99c101570 0x7fd99c1975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:22.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.829+0000 7fd9a3a81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd99c101f30 0x7fd99c197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:22.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.829+0000 7fd9a3a81640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd99c105e50 0x7fd99c19be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:22.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.829+0000 7fd9a3a81640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c111080 con 0x7fd99c101f30 2024-04-21T08:39:22.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.829+0000 7fd9a3a81640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c110f00 con 0x7fd99c105e50 2024-04-21T08:39:22.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.829+0000 7fd9a3a81640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd99c111200 con 0x7fd99c101570 2024-04-21T08:39:22.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.829+0000 7fd9a201e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd99c105e50 0x7fd99c19be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:22.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.829+0000 7fd9a101c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd99c101f30 0x7fd99c197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:22.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.829+0000 7fd9a181d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd99c101570 0x7fd99c1975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:22.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.829+0000 7fd9a201e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd99c105e50 0x7fd99c19be80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.86:59378/0 (socket says 172.21.15.86:59378) 2024-04-21T08:39:22.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.829+0000 7fd9a201e640 1 -- 172.21.15.86:0/1450885265 learned_addr learned my addr 172.21.15.86:0/1450885265 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:22.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.830+0000 7fd9a201e640 1 -- 172.21.15.86:0/1450885265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd99c101570 msgr2=0x7fd99c1975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:22.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.830+0000 7fd9a201e640 1 --2- 172.21.15.86:0/1450885265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd99c101570 0x7fd99c1975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:22.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.830+0000 7fd9a201e640 1 -- 172.21.15.86:0/1450885265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd99c101f30 msgr2=0x7fd99c197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:22.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.830+0000 7fd9a201e640 1 --2- 172.21.15.86:0/1450885265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd99c101f30 0x7fd99c197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:22.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.830+0000 7fd9a201e640 1 -- 172.21.15.86:0/1450885265 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd99c19c5b0 con 0x7fd99c105e50 2024-04-21T08:39:22.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.830+0000 7fd9a181d640 1 --2- 172.21.15.86:0/1450885265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd99c101570 0x7fd99c1975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:22.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.830+0000 7fd9a101c640 1 --2- 172.21.15.86:0/1450885265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd99c101f30 0x7fd99c197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:22.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.830+0000 7fd9a201e640 1 --2- 172.21.15.86:0/1450885265 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd99c105e50 0x7fd99c19be80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd99400eb10 tx=0x7fd99400c2d0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:22.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.830+0000 7fd992ffd640 1 -- 172.21.15.86:0/1450885265 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd994004b20 con 0x7fd99c105e50 2024-04-21T08:39:22.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.830+0000 7fd992ffd640 1 -- 172.21.15.86:0/1450885265 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd994004cc0 con 0x7fd99c105e50 2024-04-21T08:39:22.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.830+0000 7fd9a3a81640 1 -- 172.21.15.86:0/1450885265 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd99c19c8a0 con 0x7fd99c105e50 2024-04-21T08:39:22.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.830+0000 7fd9a3a81640 1 -- 172.21.15.86:0/1450885265 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd99c1a4130 con 0x7fd99c105e50 2024-04-21T08:39:22.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.831+0000 7fd992ffd640 1 -- 172.21.15.86:0/1450885265 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd994010670 con 0x7fd99c105e50 2024-04-21T08:39:22.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.834+0000 7fd992ffd640 1 -- 172.21.15.86:0/1450885265 <== mon.2 v2:172.21.15.88:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7fd994005020 con 0x7fd99c105e50 2024-04-21T08:39:22.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.834+0000 7fd9a667a740 1 Processor -- start 2024-04-21T08:39:22.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.834+0000 7fd9a667a740 1 -- start start 2024-04-21T08:39:22.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.834+0000 7fd9a667a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b629241580 0x55b629241960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:22.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.834+0000 7fd9a667a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b629241ea0 0x55b62924a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:22.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.835+0000 7fd9a667a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b62924aa80 0x55b62924ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:22.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.835+0000 7fd9a667a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b6291a61e0 con 0x55b62924aa80 2024-04-21T08:39:22.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.835+0000 7fd9a667a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b629143390 con 0x55b629241ea0 2024-04-21T08:39:22.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.835+0000 7fd9a667a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b6291a21a0 con 0x55b629241580 2024-04-21T08:39:22.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.835+0000 7fd9a101c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b62924aa80 0x55b62924ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:22.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.835+0000 7fd992ffd640 1 --2- 172.21.15.86:0/1450885265 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd9880a3ec0 0x7fd9880a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:22.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.835+0000 7fd9a101c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b62924aa80 0x55b62924ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55402/0 (socket says 172.21.15.86:55402) 2024-04-21T08:39:22.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.835+0000 7fd9a101c640 1 -- 172.21.15.86:0/2340057841 learned_addr learned my addr 172.21.15.86:0/2340057841 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:22.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.835+0000 7fd9a181d640 1 --2- 172.21.15.86:0/2340057841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b629241ea0 0x55b62924a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:22.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.835+0000 7fd9a101c640 1 --2- 172.21.15.86:0/2340057841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b629241580 0x55b629241960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:22.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.835+0000 7fd992ffd640 1 -- 172.21.15.86:0/1450885265 <== mon.2 v2:172.21.15.88:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fd9940d1260 con 0x7fd99c105e50 2024-04-21T08:39:22.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.835+0000 7fd9a201e640 1 --2- 172.21.15.86:0/1450885265 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd9880a3ec0 0x7fd9880a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:22.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.835+0000 7fd9a101c640 1 -- 172.21.15.86:0/2340057841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b629241580 msgr2=0x55b629241960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:22.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.835+0000 7fd9a101c640 1 --2- 172.21.15.86:0/2340057841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b629241580 0x55b629241960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:22.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.835+0000 7fd9a101c640 1 -- 172.21.15.86:0/2340057841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b629241ea0 msgr2=0x55b62924a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:22.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.835+0000 7fd9a101c640 1 --2- 172.21.15.86:0/2340057841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b629241ea0 0x55b62924a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:22.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.835+0000 7fd9a101c640 1 -- 172.21.15.86:0/2340057841 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b628f827d0 con 0x55b62924aa80 2024-04-21T08:39:22.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.835+0000 7fd9a181d640 1 --2- 172.21.15.86:0/2340057841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b629241ea0 0x55b62924a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:22.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.835+0000 7fd9a101c640 1 --2- 172.21.15.86:0/2340057841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b62924aa80 0x55b62924ce70 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd98c02c170 tx=0x7fd98c005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fd56e490b2181aea server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:22.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.836+0000 7fd9a201e640 1 --2- 172.21.15.86:0/1450885265 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd9880a3ec0 0x7fd9880a6380 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd9940ce4e0 tx=0x7fd994009a70 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:22.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.836+0000 7fd987fff640 1 -- 172.21.15.86:0/2340057841 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c004030 con 0x55b62924aa80 2024-04-21T08:39:22.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.836+0000 7fd987fff640 1 -- 172.21.15.86:0/2340057841 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd98c0035f0 con 0x55b62924aa80 2024-04-21T08:39:22.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.836+0000 7fd987fff640 1 -- 172.21.15.86:0/2340057841 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c0426d0 con 0x55b62924aa80 2024-04-21T08:39:22.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.837+0000 7fd9a667a740 1 -- 172.21.15.86:0/2340057841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b62924aa80 msgr2=0x55b62924ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:22.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.837+0000 7fd9a667a740 1 --2- 172.21.15.86:0/2340057841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b62924aa80 0x55b62924ce70 secure :-1 s=READY pgs=651 cs=0 l=1 rev1=1 crypto rx=0x7fd98c02c170 tx=0x7fd98c005b70 comp rx=0 tx=0).stop 2024-04-21T08:39:22.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.837+0000 7fd9a667a740 1 -- 172.21.15.86:0/2340057841 shutdown_connections 2024-04-21T08:39:22.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.837+0000 7fd9a667a740 1 --2- 172.21.15.86:0/2340057841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b62924aa80 0x55b62924ce70 unknown :-1 s=CLOSED pgs=651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:22.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.837+0000 7fd9a667a740 1 --2- 172.21.15.86:0/2340057841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b629241ea0 0x55b62924a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:22.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.837+0000 7fd9a667a740 1 --2- 172.21.15.86:0/2340057841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b629241580 0x55b629241960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:22.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.837+0000 7fd9a667a740 1 -- 172.21.15.86:0/2340057841 >> 172.21.15.86:0/2340057841 conn(0x55b62923bcb0 msgr2=0x55b62923c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:22.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.837+0000 7fd9a667a740 1 -- 172.21.15.86:0/2340057841 shutdown_connections 2024-04-21T08:39:22.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.837+0000 7fd9a667a740 1 -- 172.21.15.86:0/2340057841 wait complete. 2024-04-21T08:39:22.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.838+0000 7fd9a667a740 1 -- start start 2024-04-21T08:39:22.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.838+0000 7fd9a667a740 1 Processor -- start 2024-04-21T08:39:22.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.838+0000 7fd9a667a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b629241580 0x55b629247b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:22.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.839+0000 7fd9a667a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b629241ea0 0x55b629248050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:22.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.839+0000 7fd9a667a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b629248590 0x55b629256d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:22.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.839+0000 7fd9a667a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b62924d810 con 0x55b629241580 2024-04-21T08:39:22.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.839+0000 7fd9a667a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b62915f2f0 con 0x55b629248590 2024-04-21T08:39:22.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.839+0000 7fd9a667a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b62924d990 con 0x55b629241ea0 2024-04-21T08:39:22.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.839+0000 7fd9a101c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b629241580 0x55b629247b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:22.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.839+0000 7fd9a667a740 10 client.0 _ll_register_callbacks cb 0x55b62916c580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:22.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.839+0000 7fd9a101c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b629241580 0x55b629247b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55414/0 (socket says 172.21.15.86:55414) 2024-04-21T08:39:22.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.839+0000 7fd9a101c640 1 -- 172.21.15.86:0/3370988847 learned_addr learned my addr 172.21.15.86:0/3370988847 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:22.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.839+0000 7fd9a181d640 1 --2- 172.21.15.86:0/3370988847 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b629241ea0 0x55b629248050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:22.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.839+0000 7fd9a101c640 1 -- 172.21.15.86:0/3370988847 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b629241ea0 msgr2=0x55b629248050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:22.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.839+0000 7fd9a101c640 1 --2- 172.21.15.86:0/3370988847 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b629241ea0 0x55b629248050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:22.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.839+0000 7fd9a101c640 1 -- 172.21.15.86:0/3370988847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b629248590 msgr2=0x55b629256d10 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-21T08:39:22.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.839+0000 7fd9a101c640 1 --2- 172.21.15.86:0/3370988847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b629248590 0x55b629256d10 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:22.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.839+0000 7fd9a101c640 1 -- 172.21.15.86:0/3370988847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b628eb83f0 con 0x55b629241580 2024-04-21T08:39:22.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.840+0000 7fd9a101c640 1 --2- 172.21.15.86:0/3370988847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b629241580 0x55b629247b10 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fd98c049000 tx=0x7fd98c005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=338a51aaadc692f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:22.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.840+0000 7fd9857fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:22.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.840+0000 7fd9857fa640 1 -- 172.21.15.86:0/3370988847 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c053020 con 0x55b629241580 2024-04-21T08:39:22.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.840+0000 7fd9857fa640 1 -- 172.21.15.86:0/3370988847 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fd98c0039a0 con 0x55b629241580 2024-04-21T08:39:22.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.840+0000 7fd9a667a740 1 -- 172.21.15.86:0/3370988847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd98c0035f0 con 0x55b629241580 2024-04-21T08:39:22.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.840+0000 7fd9857fa640 1 -- 172.21.15.86:0/3370988847 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd98c003ba0 con 0x55b629241580 2024-04-21T08:39:22.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.840+0000 7fd9a667a740 10 client.16089 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:22.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.840+0000 7fd9a667a740 1 -- 172.21.15.86:0/3370988847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b628e780f0 con 0x55b629241580 2024-04-21T08:39:22.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.841+0000 7fd9857fa640 1 -- 172.21.15.86:0/3370988847 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd98c0035f0 con 0x55b629241580 2024-04-21T08:39:22.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.841+0000 7fd9857fa640 1 -- 172.21.15.86:0/3370988847 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd98c04f450 con 0x55b629241580 2024-04-21T08:39:22.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.841+0000 7fd9a667a740 10 client.16089 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:22.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.841+0000 7fd9a667a740 10 client.16089 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:22.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.841+0000 7fd9a667a740 20 client.16089 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:22.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.841+0000 7fd9a667a740 1 --2- 172.21.15.86:0/3370988847 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b62925ae20 0x55b62925d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:22.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.841+0000 7fd9a667a740 4 client.16089 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:22.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.841+0000 7fd9a667a740 1 -- 172.21.15.86:0/3370988847 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6291b3c00 con 0x55b62925ae20 2024-04-21T08:39:22.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.841+0000 7fd9a181d640 1 --2- 172.21.15.86:0/3370988847 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b62925ae20 0x55b62925d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:22.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.842+0000 7fd9a181d640 1 --2- 172.21.15.86:0/3370988847 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b62925ae20 0x55b62925d210 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4700483c2b0ce4c5 server_cookie=fada6006c2472a79 in_seq=0 out_seq=0 2024-04-21T08:39:22.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.842+0000 7fd9857fa640 10 client.16089 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:22.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.842+0000 7fd9857fa640 1 -- 172.21.15.86:0/3370988847 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b6291b3c00 con 0x55b62925ae20 2024-04-21T08:39:22.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.842+0000 7fd9857fa640 10 client.16089 handle_command_reply: tid=0 2024-04-21T08:39:22.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.842+0000 7fd9a667a740 1 client.16089 shutdown 2024-04-21T08:39:22.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.843+0000 7fd9a667a740 1 -- 172.21.15.86:0/3370988847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b629241580 msgr2=0x55b629247b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:22.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.843+0000 7fd9a667a740 1 --2- 172.21.15.86:0/3370988847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b629241580 0x55b629247b10 secure :-1 s=READY pgs=652 cs=0 l=1 rev1=1 crypto rx=0x7fd98c049000 tx=0x7fd98c005fd0 comp rx=0 tx=0).stop 2024-04-21T08:39:22.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.843+0000 7fd9a667a740 1 -- 172.21.15.86:0/3370988847 shutdown_connections 2024-04-21T08:39:22.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.843+0000 7fd9a667a740 1 --2- 172.21.15.86:0/3370988847 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b62925ae20 0x55b62925d210 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:22.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.843+0000 7fd9a667a740 1 --2- 172.21.15.86:0/3370988847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b629248590 0x55b629256d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:22.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.843+0000 7fd9a667a740 1 --2- 172.21.15.86:0/3370988847 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b629241ea0 0x55b629248050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:22.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.843+0000 7fd9857fa640 0 client.16089 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:22.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.843+0000 7fd9a667a740 1 --2- 172.21.15.86:0/3370988847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b629241580 0x55b629247b10 unknown :-1 s=CLOSED pgs=652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:22.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.843+0000 7fd9a667a740 1 -- 172.21.15.86:0/3370988847 >> 172.21.15.86:0/3370988847 conn(0x55b62923bcb0 msgr2=0x55b62923da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:22.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.843+0000 7fd9a667a740 1 -- 172.21.15.86:0/3370988847 shutdown_connections 2024-04-21T08:39:22.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.843+0000 7fd9a667a740 1 -- 172.21.15.86:0/3370988847 wait complete. 2024-04-21T08:39:22.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.843+0000 7fd9a667a740 20 client.16089 trim_cache size 0 max 16384 2024-04-21T08:39:22.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.877+0000 7fd9a667a740 1 Processor -- start 2024-04-21T08:39:22.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.877+0000 7fd9a667a740 1 -- start start 2024-04-21T08:39:22.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.877+0000 7fd9a667a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b62923bcb0 0x55b62923c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:22.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.877+0000 7fd9a667a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b629241580 0x55b62923d810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:22.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.877+0000 7fd9a667a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b629241ea0 0x55b629262840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:22.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.877+0000 7fd9a667a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd98000fe90 con 0x55b629241580 2024-04-21T08:39:22.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.877+0000 7fd9a667a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd9800119d0 con 0x55b62923bcb0 2024-04-21T08:39:22.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.877+0000 7fd9a667a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd980010560 con 0x55b629241ea0 2024-04-21T08:39:22.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.877+0000 7fd9a181d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b629241580 0x55b62923d810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:22.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.878+0000 7fd9a181d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b629241580 0x55b62923d810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55422/0 (socket says 172.21.15.86:55422) 2024-04-21T08:39:22.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.878+0000 7fd9a181d640 1 -- 172.21.15.86:0/1163163100 learned_addr learned my addr 172.21.15.86:0/1163163100 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:22.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.878+0000 7fd9a101c640 1 --2- 172.21.15.86:0/1163163100 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b62923bcb0 0x55b62923c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:22.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.878+0000 7fd9a101c640 1 -- 172.21.15.86:0/1163163100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b629241ea0 msgr2=0x55b629262840 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:39:22.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.878+0000 7fd9a101c640 1 --2- 172.21.15.86:0/1163163100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b629241ea0 0x55b629262840 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:22.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.878+0000 7fd9a101c640 1 -- 172.21.15.86:0/1163163100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b629241580 msgr2=0x55b62923d810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:22.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.878+0000 7fd9a101c640 1 --2- 172.21.15.86:0/1163163100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b629241580 0x55b62923d810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:22.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.878+0000 7fd9a101c640 1 -- 172.21.15.86:0/1163163100 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b62924d1e0 con 0x55b62923bcb0 2024-04-21T08:39:22.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.878+0000 7fd9a181d640 1 --2- 172.21.15.86:0/1163163100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b629241580 0x55b62923d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:22.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.878+0000 7fd9a101c640 1 --2- 172.21.15.86:0/1163163100 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b62923bcb0 0x55b62923c090 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd98c04f8a0 tx=0x7fd98c004240 comp rx=0 tx=0).ready entity=mon.2 client_cookie=44e9dd6b17e90f37 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:22.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.878+0000 7fd9877fe640 1 -- 172.21.15.86:0/1163163100 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c053040 con 0x55b62923bcb0 2024-04-21T08:39:22.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.878+0000 7fd9877fe640 1 -- 172.21.15.86:0/1163163100 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd98c02cc50 con 0x55b62923bcb0 2024-04-21T08:39:22.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.878+0000 7fd9877fe640 1 -- 172.21.15.86:0/1163163100 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd98c042470 con 0x55b62923bcb0 2024-04-21T08:39:22.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.879+0000 7fd9a667a740 1 -- 172.21.15.86:0/1163163100 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b62923bcb0 msgr2=0x55b62923c090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:22.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.879+0000 7fd9a667a740 1 --2- 172.21.15.86:0/1163163100 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b62923bcb0 0x55b62923c090 secure :-1 s=READY pgs=62 cs=0 l=1 rev1=1 crypto rx=0x7fd98c04f8a0 tx=0x7fd98c004240 comp rx=0 tx=0).stop 2024-04-21T08:39:22.883 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.879+0000 7fd9a667a740 1 -- 172.21.15.86:0/1163163100 shutdown_connections 2024-04-21T08:39:22.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.879+0000 7fd9a667a740 1 --2- 172.21.15.86:0/1163163100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b629241ea0 0x55b629262840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:22.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.879+0000 7fd9a667a740 1 --2- 172.21.15.86:0/1163163100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b629241580 0x55b62923d810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:22.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.879+0000 7fd9a667a740 1 --2- 172.21.15.86:0/1163163100 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b62923bcb0 0x55b62923c090 unknown :-1 s=CLOSED pgs=62 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:22.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.879+0000 7fd9a667a740 1 -- 172.21.15.86:0/1163163100 >> 172.21.15.86:0/1163163100 conn(0x55b6292481b0 msgr2=0x55b62923d100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:22.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.879+0000 7fd9a667a740 1 -- 172.21.15.86:0/1163163100 shutdown_connections 2024-04-21T08:39:22.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.879+0000 7fd9a667a740 1 -- 172.21.15.86:0/1163163100 wait complete. 2024-04-21T08:39:22.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.880+0000 7fd9a667a740 1 -- start start 2024-04-21T08:39:22.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.880+0000 7fd9a667a740 1 Processor -- start 2024-04-21T08:39:22.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.880+0000 7fd9a667a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b62923bcb0 0x55b6292ba3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:22.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.880+0000 7fd9a667a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b629241580 0x55b6292ba900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:22.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.880+0000 7fd9a667a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b629241ea0 0x55b62924edf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:22.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.880+0000 7fd9a667a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd980011300 con 0x55b629241580 2024-04-21T08:39:22.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.880+0000 7fd9a667a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd980010c30 con 0x55b629241ea0 2024-04-21T08:39:22.884 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.880+0000 7fd9a667a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd98000de50 con 0x55b62923bcb0 2024-04-21T08:39:22.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.880+0000 7fd9a667a740 10 client.0 _ll_register_callbacks cb 0x7fd98c009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:22.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.881+0000 7fd9a181d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b629241580 0x55b6292ba900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:22.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.881+0000 7fd9a181d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b629241580 0x55b6292ba900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55438/0 (socket says 172.21.15.86:55438) 2024-04-21T08:39:22.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.881+0000 7fd9a181d640 1 -- 172.21.15.86:0/1399378511 learned_addr learned my addr 172.21.15.86:0/1399378511 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:22.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.881+0000 7fd9a101c640 1 --2- 172.21.15.86:0/1399378511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b62923bcb0 0x55b6292ba3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:22.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.881+0000 7fd9a101c640 1 --2- 172.21.15.86:0/1399378511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b629241ea0 0x55b62924edf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:22.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.881+0000 7fd9a181d640 1 -- 172.21.15.86:0/1399378511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b62923bcb0 msgr2=0x55b6292ba3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:22.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.881+0000 7fd9a181d640 1 --2- 172.21.15.86:0/1399378511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b62923bcb0 0x55b6292ba3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:22.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.881+0000 7fd9a181d640 1 -- 172.21.15.86:0/1399378511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b629241ea0 msgr2=0x55b62924edf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:22.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.881+0000 7fd9a181d640 1 --2- 172.21.15.86:0/1399378511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b629241ea0 0x55b62924edf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:22.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.881+0000 7fd9a181d640 1 -- 172.21.15.86:0/1399378511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b62924f530 con 0x55b629241580 2024-04-21T08:39:22.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.881+0000 7fd9a101c640 1 --2- 172.21.15.86:0/1399378511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b62923bcb0 0x55b6292ba3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:22.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.881+0000 7fd9a181d640 1 --2- 172.21.15.86:0/1399378511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b629241580 0x55b6292ba900 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x55b629247370 tx=0x7fd998016d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ec1ba38464b8181 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:22.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.881+0000 7fd9857fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:22.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.881+0000 7fd9857fa640 1 -- 172.21.15.86:0/1399378511 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd998027070 con 0x55b629241580 2024-04-21T08:39:22.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.881+0000 7fd9857fa640 1 -- 172.21.15.86:0/1399378511 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fd9980152e0 con 0x55b629241580 2024-04-21T08:39:22.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.881+0000 7fd9a667a740 1 -- 172.21.15.86:0/1399378511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd98c02cc50 con 0x55b629241580 2024-04-21T08:39:22.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.882+0000 7fd9857fa640 1 -- 172.21.15.86:0/1399378511 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd998032050 con 0x55b629241580 2024-04-21T08:39:22.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.882+0000 7fd9a667a740 10 client.16095 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:22.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.882+0000 7fd9a667a740 1 -- 172.21.15.86:0/1399378511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b6292c1c10 con 0x55b629241580 2024-04-21T08:39:22.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.882+0000 7fd9857fa640 1 -- 172.21.15.86:0/1399378511 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd98c02cc50 con 0x55b629241580 2024-04-21T08:39:22.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.882+0000 7fd9857fa640 1 -- 172.21.15.86:0/1399378511 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd998004930 con 0x55b629241580 2024-04-21T08:39:22.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.882+0000 7fd9a667a740 10 client.16095 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:22.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.882+0000 7fd9a667a740 10 client.16095 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:22.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.882+0000 7fd9a667a740 20 client.16095 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:22.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.882+0000 7fd9a667a740 1 --2- 172.21.15.86:0/1399378511 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b629248590 0x55b6292c43e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:22.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.882+0000 7fd9a667a740 4 client.16095 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:39:22.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.882+0000 7fd9a667a740 1 -- 172.21.15.86:0/1399378511 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b6291ada70 con 0x55b629248590 2024-04-21T08:39:22.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.882+0000 7fd9a101c640 1 --2- 172.21.15.86:0/1399378511 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b629248590 0x55b6292c43e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:22.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.883+0000 7fd9857fa640 10 client.16095 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:22.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.883+0000 7fd9a101c640 1 --2- 172.21.15.86:0/1399378511 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b629248590 0x55b6292c43e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e8e8a850e165db75 server_cookie=721f0f93e1f4fa16 in_seq=0 out_seq=0 2024-04-21T08:39:22.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.883+0000 7fd9857fa640 1 -- 172.21.15.86:0/1399378511 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55b6291ada70 con 0x55b629248590 2024-04-21T08:39:22.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.883+0000 7fd9857fa640 10 client.16095 handle_command_reply: tid=0 2024-04-21T08:39:22.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.883+0000 7fd9a667a740 1 client.16095 shutdown 2024-04-21T08:39:22.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.883+0000 7fd9a667a740 1 -- 172.21.15.86:0/1399378511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b629241580 msgr2=0x55b6292ba900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:22.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.883+0000 7fd9a667a740 1 --2- 172.21.15.86:0/1399378511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b629241580 0x55b6292ba900 secure :-1 s=READY pgs=653 cs=0 l=1 rev1=1 crypto rx=0x55b629247370 tx=0x7fd998016d40 comp rx=0 tx=0).stop 2024-04-21T08:39:22.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.883+0000 7fd9a667a740 1 -- 172.21.15.86:0/1399378511 shutdown_connections 2024-04-21T08:39:22.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.883+0000 7fd9a667a740 1 --2- 172.21.15.86:0/1399378511 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b629248590 0x55b6292c43e0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:22.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.883+0000 7fd9a667a740 1 --2- 172.21.15.86:0/1399378511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b629241ea0 0x55b62924edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:22.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.883+0000 7fd9a667a740 1 --2- 172.21.15.86:0/1399378511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b629241580 0x55b6292ba900 unknown :-1 s=CLOSED pgs=653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:22.887 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.883+0000 7fd9a667a740 1 --2- 172.21.15.86:0/1399378511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b62923bcb0 0x55b6292ba3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:22.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.883+0000 7fd9a667a740 1 -- 172.21.15.86:0/1399378511 >> 172.21.15.86:0/1399378511 conn(0x55b6292481b0 msgr2=0x55b62923d920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:22.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.884+0000 7fd9857fa640 0 client.16095 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:22.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.884+0000 7fd9a667a740 1 -- 172.21.15.86:0/1399378511 shutdown_connections 2024-04-21T08:39:22.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.884+0000 7fd9a667a740 1 -- 172.21.15.86:0/1399378511 wait complete. 2024-04-21T08:39:22.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.884+0000 7fd9a667a740 20 client.16095 trim_cache size 0 max 16384 2024-04-21T08:39:22.888 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:39:22.888 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-21T08:39:22.888 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-21T08:39:22.888 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:39:22.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.884+0000 7fd9a3a81640 1 -- 172.21.15.86:0/1450885265 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd9880a3ec0 msgr2=0x7fd9880a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:22.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.884+0000 7fd9a3a81640 1 --2- 172.21.15.86:0/1450885265 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd9880a3ec0 0x7fd9880a6380 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x7fd9940ce4e0 tx=0x7fd994009a70 comp rx=0 tx=0).stop 2024-04-21T08:39:22.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.884+0000 7fd9a3a81640 1 -- 172.21.15.86:0/1450885265 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd99c105e50 msgr2=0x7fd99c19be80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:22.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.884+0000 7fd9a3a81640 1 --2- 172.21.15.86:0/1450885265 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd99c105e50 0x7fd99c19be80 secure :-1 s=READY pgs=61 cs=0 l=1 rev1=1 crypto rx=0x7fd99400eb10 tx=0x7fd99400c2d0 comp rx=0 tx=0).stop 2024-04-21T08:39:22.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.884+0000 7fd9a3a81640 1 -- 172.21.15.86:0/1450885265 shutdown_connections 2024-04-21T08:39:22.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.885+0000 7fd9a3a81640 1 --2- 172.21.15.86:0/1450885265 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd9880a3ec0 0x7fd9880a6380 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:22.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.885+0000 7fd9a3a81640 1 --2- 172.21.15.86:0/1450885265 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd99c105e50 0x7fd99c19be80 unknown :-1 s=CLOSED pgs=61 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:22.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.885+0000 7fd9a3a81640 1 --2- 172.21.15.86:0/1450885265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd99c101f30 0x7fd99c197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:22.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.885+0000 7fd9a3a81640 1 --2- 172.21.15.86:0/1450885265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd99c101570 0x7fd99c1975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:22.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.885+0000 7fd9a3a81640 1 -- 172.21.15.86:0/1450885265 >> 172.21.15.86:0/1450885265 conn(0x7fd99c0773b0 msgr2=0x7fd99c103f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:22.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.885+0000 7fd9a3a81640 1 -- 172.21.15.86:0/1450885265 shutdown_connections 2024-04-21T08:39:22.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:22.885+0000 7fd9a3a81640 1 -- 172.21.15.86:0/1450885265 wait complete. 2024-04-21T08:39:22.899 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:39:22.899 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:39:23.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.052+0000 7f877baba640 1 -- 172.21.15.86:0/560803396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8774105e30 msgr2=0x7f8774106210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.052+0000 7f877baba640 1 --2- 172.21.15.86:0/560803396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8774105e30 0x7f8774106210 secure :-1 s=READY pgs=654 cs=0 l=1 rev1=1 crypto rx=0x7f8770009930 tx=0x7f877002bc50 comp rx=0 tx=0).stop 2024-04-21T08:39:23.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.052+0000 7f877baba640 1 -- 172.21.15.86:0/560803396 shutdown_connections 2024-04-21T08:39:23.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.052+0000 7f877baba640 1 --2- 172.21.15.86:0/560803396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8774101b60 0x7f877410dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.052+0000 7f877baba640 1 --2- 172.21.15.86:0/560803396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f87741011a0 0x7f8774101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.052+0000 7f877baba640 1 --2- 172.21.15.86:0/560803396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8774105e30 0x7f8774106210 unknown :-1 s=CLOSED pgs=654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.052+0000 7f877baba640 1 -- 172.21.15.86:0/560803396 >> 172.21.15.86:0/560803396 conn(0x7f8774077e30 msgr2=0x7f8774078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:23.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.052+0000 7f877baba640 1 -- 172.21.15.86:0/560803396 shutdown_connections 2024-04-21T08:39:23.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.052+0000 7f877baba640 1 -- 172.21.15.86:0/560803396 wait complete. 2024-04-21T08:39:23.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.053+0000 7f877baba640 1 Processor -- start 2024-04-21T08:39:23.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.053+0000 7f877baba640 1 -- start start 2024-04-21T08:39:23.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.053+0000 7f877baba640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87741011a0 0x7f8774199690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.053+0000 7f877baba640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8774101b60 0x7f8774199bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.053+0000 7f877baba640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8774105e30 0x7f87741a4710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.053+0000 7f877baba640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f87741111b0 con 0x7f8774105e30 2024-04-21T08:39:23.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.053+0000 7f877baba640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8774111030 con 0x7f8774101b60 2024-04-21T08:39:23.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.053+0000 7f877baba640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8774111330 con 0x7f87741011a0 2024-04-21T08:39:23.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.053+0000 7f877a057640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8774105e30 0x7f87741a4710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.053+0000 7f877a057640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8774105e30 0x7f87741a4710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55460/0 (socket says 172.21.15.86:55460) 2024-04-21T08:39:23.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.053+0000 7f877a057640 1 -- 172.21.15.86:0/978893574 learned_addr learned my addr 172.21.15.86:0/978893574 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:23.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.053+0000 7f8779055640 1 --2- 172.21.15.86:0/978893574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8774101b60 0x7f8774199bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.053+0000 7f8779856640 1 --2- 172.21.15.86:0/978893574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87741011a0 0x7f8774199690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.053+0000 7f877a057640 1 -- 172.21.15.86:0/978893574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87741011a0 msgr2=0x7f8774199690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.053+0000 7f877a057640 1 --2- 172.21.15.86:0/978893574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87741011a0 0x7f8774199690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.053+0000 7f877a057640 1 -- 172.21.15.86:0/978893574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8774101b60 msgr2=0x7f8774199bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.053+0000 7f877a057640 1 --2- 172.21.15.86:0/978893574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8774101b60 0x7f8774199bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.053+0000 7f877a057640 1 -- 172.21.15.86:0/978893574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f87741a4c50 con 0x7f8774105e30 2024-04-21T08:39:23.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.053+0000 7f8779856640 1 --2- 172.21.15.86:0/978893574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87741011a0 0x7f8774199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:23.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.053+0000 7f8779055640 1 --2- 172.21.15.86:0/978893574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8774101b60 0x7f8774199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:23.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.054+0000 7f877a057640 1 --2- 172.21.15.86:0/978893574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8774105e30 0x7f87741a4710 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f876c00b990 tx=0x7f876c00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:23.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.054+0000 7f876affd640 1 -- 172.21.15.86:0/978893574 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876c019070 con 0x7f8774105e30 2024-04-21T08:39:23.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.054+0000 7f876affd640 1 -- 172.21.15.86:0/978893574 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f876c00cba0 con 0x7f8774105e30 2024-04-21T08:39:23.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.054+0000 7f877baba640 1 -- 172.21.15.86:0/978893574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f87741a4e20 con 0x7f8774105e30 2024-04-21T08:39:23.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.054+0000 7f876affd640 1 -- 172.21.15.86:0/978893574 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f876c0075d0 con 0x7f8774105e30 2024-04-21T08:39:23.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.054+0000 7f877baba640 1 -- 172.21.15.86:0/978893574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f87741a5230 con 0x7f8774105e30 2024-04-21T08:39:23.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.055+0000 7f877baba640 1 -- 172.21.15.86:0/978893574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8774102570 con 0x7f8774105e30 2024-04-21T08:39:23.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.056+0000 7f876affd640 1 -- 172.21.15.86:0/978893574 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f876c01c070 con 0x7f8774105e30 2024-04-21T08:39:23.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.056+0000 7f876affd640 1 --2- 172.21.15.86:0/978893574 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f87600a3e40 0x7f87600a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.057+0000 7f8779856640 1 --2- 172.21.15.86:0/978893574 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f87600a3e40 0x7f87600a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.057+0000 7f876affd640 1 -- 172.21.15.86:0/978893574 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f876c0d0330 con 0x7f8774105e30 2024-04-21T08:39:23.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.057+0000 7f8779856640 1 --2- 172.21.15.86:0/978893574 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f87600a3e40 0x7f87600a6300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8770005ec0 tx=0x7f8770005fd0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:23.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.061+0000 7f876affd640 1 -- 172.21.15.86:0/978893574 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f876c09c520 con 0x7f8774105e30 2024-04-21T08:39:23.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.286+0000 7f877baba640 1 -- 172.21.15.86:0/978893574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f87740696b0 con 0x7f8774105e30 2024-04-21T08:39:23.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.287+0000 7f876affd640 1 -- 172.21.15.86:0/978893574 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7f87740696b0 con 0x7f8774105e30 2024-04-21T08:39:23.286 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:39:23.286 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:38:45.202126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24398,24394,24455,24472]},"id":1}]} 2024-04-21T08:39:23.286 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-21T08:39:23.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.289+0000 7f877baba640 1 -- 172.21.15.86:0/978893574 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f87600a3e40 msgr2=0x7f87600a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.289+0000 7f877baba640 1 --2- 172.21.15.86:0/978893574 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f87600a3e40 0x7f87600a6300 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f8770005ec0 tx=0x7f8770005fd0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.289+0000 7f877baba640 1 -- 172.21.15.86:0/978893574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8774105e30 msgr2=0x7f87741a4710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.289+0000 7f877baba640 1 --2- 172.21.15.86:0/978893574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8774105e30 0x7f87741a4710 secure :-1 s=READY pgs=655 cs=0 l=1 rev1=1 crypto rx=0x7f876c00b990 tx=0x7f876c00be50 comp rx=0 tx=0).stop 2024-04-21T08:39:23.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.289+0000 7f877baba640 1 -- 172.21.15.86:0/978893574 shutdown_connections 2024-04-21T08:39:23.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.289+0000 7f877baba640 1 --2- 172.21.15.86:0/978893574 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f87600a3e40 0x7f87600a6300 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.289+0000 7f877baba640 1 --2- 172.21.15.86:0/978893574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8774105e30 0x7f87741a4710 unknown :-1 s=CLOSED pgs=655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.289+0000 7f877baba640 1 --2- 172.21.15.86:0/978893574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8774101b60 0x7f8774199bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.289+0000 7f877baba640 1 --2- 172.21.15.86:0/978893574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f87741011a0 0x7f8774199690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.289+0000 7f877baba640 1 -- 172.21.15.86:0/978893574 >> 172.21.15.86:0/978893574 conn(0x7f8774077e30 msgr2=0x7f87740feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:23.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.289+0000 7f877baba640 1 -- 172.21.15.86:0/978893574 shutdown_connections 2024-04-21T08:39:23.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.289+0000 7f877baba640 1 -- 172.21.15.86:0/978893574 wait complete. 2024-04-21T08:39:23.301 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:39:23.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.406+0000 7f78e6a3a640 1 -- 172.21.15.86:0/2801757822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78e01021c0 msgr2=0x7f78e01025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.406+0000 7f78e6a3a640 1 --2- 172.21.15.86:0/2801757822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78e01021c0 0x7f78e01025a0 secure :-1 s=READY pgs=656 cs=0 l=1 rev1=1 crypto rx=0x7f78d0009900 tx=0x7f78d002bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:23.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.406+0000 7f78e6a3a640 1 -- 172.21.15.86:0/2801757822 shutdown_connections 2024-04-21T08:39:23.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.406+0000 7f78e6a3a640 1 --2- 172.21.15.86:0/2801757822 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78e0107290 0x7f78e010ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.406+0000 7f78e6a3a640 1 --2- 172.21.15.86:0/2801757822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78e0102b70 0x7f78e0106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.406+0000 7f78e6a3a640 1 --2- 172.21.15.86:0/2801757822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78e01021c0 0x7f78e01025a0 unknown :-1 s=CLOSED pgs=656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.406+0000 7f78e6a3a640 1 -- 172.21.15.86:0/2801757822 >> 172.21.15.86:0/2801757822 conn(0x7f78e00793e0 msgr2=0x7f78e00797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:23.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.406+0000 7f78e6a3a640 1 -- 172.21.15.86:0/2801757822 shutdown_connections 2024-04-21T08:39:23.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.406+0000 7f78e6a3a640 1 -- 172.21.15.86:0/2801757822 wait complete. 2024-04-21T08:39:23.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.406+0000 7f78e6a3a640 1 Processor -- start 2024-04-21T08:39:23.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.406+0000 7f78e6a3a640 1 -- start start 2024-04-21T08:39:23.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.407+0000 7f78e6a3a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78e0107290 0x7f78e019e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.407+0000 7f78e6a3a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78e019ec00 0x7f78e0198870 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.407+0000 7f78e6a3a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78e0198db0 0x7f78e0199230 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.407+0000 7f78e6a3a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f78e01121f0 con 0x7f78e0198db0 2024-04-21T08:39:23.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.407+0000 7f78e6a3a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f78e0112070 con 0x7f78e019ec00 2024-04-21T08:39:23.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.407+0000 7f78e6a3a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f78e0112370 con 0x7f78e0107290 2024-04-21T08:39:23.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.407+0000 7f78e4fd7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78e0198db0 0x7f78e0199230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.407+0000 7f78e4fd7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78e0198db0 0x7f78e0199230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55490/0 (socket says 172.21.15.86:55490) 2024-04-21T08:39:23.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.407+0000 7f78e4fd7640 1 -- 172.21.15.86:0/3281940705 learned_addr learned my addr 172.21.15.86:0/3281940705 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:23.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.407+0000 7f78dffff640 1 --2- 172.21.15.86:0/3281940705 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78e0107290 0x7f78e019e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.407+0000 7f78df7fe640 1 --2- 172.21.15.86:0/3281940705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78e019ec00 0x7f78e0198870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.407+0000 7f78e4fd7640 1 -- 172.21.15.86:0/3281940705 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78e0107290 msgr2=0x7f78e019e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.407+0000 7f78e4fd7640 1 --2- 172.21.15.86:0/3281940705 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78e0107290 0x7f78e019e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.407+0000 7f78e4fd7640 1 -- 172.21.15.86:0/3281940705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78e019ec00 msgr2=0x7f78e0198870 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.407+0000 7f78e4fd7640 1 --2- 172.21.15.86:0/3281940705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78e019ec00 0x7f78e0198870 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.407+0000 7f78e4fd7640 1 -- 172.21.15.86:0/3281940705 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78e01a5580 con 0x7f78e0198db0 2024-04-21T08:39:23.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.408+0000 7f78e4fd7640 1 --2- 172.21.15.86:0/3281940705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78e0198db0 0x7f78e0199230 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f78d800ec70 tx=0x7f78d800c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:23.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.408+0000 7f78dd7fa640 1 -- 172.21.15.86:0/3281940705 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d8003b70 con 0x7f78e0198db0 2024-04-21T08:39:23.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.408+0000 7f78e6a3a640 1 -- 172.21.15.86:0/3281940705 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78e01a57b0 con 0x7f78e0198db0 2024-04-21T08:39:23.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.408+0000 7f78dd7fa640 1 -- 172.21.15.86:0/3281940705 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78d800ee80 con 0x7f78e0198db0 2024-04-21T08:39:23.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.408+0000 7f78dd7fa640 1 -- 172.21.15.86:0/3281940705 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d8010830 con 0x7f78e0198db0 2024-04-21T08:39:23.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.408+0000 7f78e6a3a640 1 -- 172.21.15.86:0/3281940705 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78e01a5d00 con 0x7f78e0198db0 2024-04-21T08:39:23.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.411+0000 7f78e9622740 1 Processor -- start 2024-04-21T08:39:23.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.411+0000 7f78dd7fa640 1 -- 172.21.15.86:0/3281940705 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f78d8004020 con 0x7f78e0198db0 2024-04-21T08:39:23.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.411+0000 7f78e9622740 1 -- start start 2024-04-21T08:39:23.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.411+0000 7f78e9622740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f601b34d20 0x55f601b35100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.411+0000 7f78e9622740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f601b356d0 0x55f601b21650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.412+0000 7f78e9622740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f601b21c70 0x55f601b220d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.412+0000 7f78e9622740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f601a8c1f0 con 0x55f601b356d0 2024-04-21T08:39:23.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.412+0000 7f78e9622740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55f601a294b0 con 0x55f601b34d20 2024-04-21T08:39:23.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.412+0000 7f78e9622740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f601a881b0 con 0x55f601b21c70 2024-04-21T08:39:23.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.412+0000 7f78dffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f601b356d0 0x55f601b21650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.412+0000 7f78dffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f601b356d0 0x55f601b21650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55500/0 (socket says 172.21.15.86:55500) 2024-04-21T08:39:23.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.412+0000 7f78dffff640 1 -- 172.21.15.86:0/4157294432 learned_addr learned my addr 172.21.15.86:0/4157294432 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:23.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.412+0000 7f78df7fe640 1 --2- 172.21.15.86:0/4157294432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f601b34d20 0x55f601b35100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.412+0000 7f78df7fe640 1 --2- 172.21.15.86:0/4157294432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f601b21c70 0x55f601b220d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.412+0000 7f78dd7fa640 1 --2- 172.21.15.86:0/3281940705 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f78c80a3e40 0x7f78c80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.412+0000 7f78dffff640 1 -- 172.21.15.86:0/4157294432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f601b21c70 msgr2=0x55f601b220d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:23.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.412+0000 7f78dffff640 1 --2- 172.21.15.86:0/4157294432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f601b21c70 0x55f601b220d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.412+0000 7f78dffff640 1 -- 172.21.15.86:0/4157294432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f601b34d20 msgr2=0x55f601b35100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:23.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.412+0000 7f78dffff640 1 --2- 172.21.15.86:0/4157294432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f601b34d20 0x55f601b35100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.412+0000 7f78dffff640 1 -- 172.21.15.86:0/4157294432 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f601a1b1b0 con 0x55f601b356d0 2024-04-21T08:39:23.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.412+0000 7f78dd7fa640 1 -- 172.21.15.86:0/3281940705 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f78d80d11f0 con 0x7f78e0198db0 2024-04-21T08:39:23.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.412+0000 7f78e4fd7640 1 --2- 172.21.15.86:0/3281940705 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f78c80a3e40 0x7f78c80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.412+0000 7f78df7fe640 1 --2- 172.21.15.86:0/4157294432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f601b34d20 0x55f601b35100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:23.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.413+0000 7f78dffff640 1 --2- 172.21.15.86:0/4157294432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f601b356d0 0x55f601b21650 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f78e0101e80 tx=0x7f78d0005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d41246aca77a797 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:23.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.413+0000 7f78e4fd7640 1 --2- 172.21.15.86:0/3281940705 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f78c80a3e40 0x7f78c80a6300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f78d80ce4b0 tx=0x7f78d8004800 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:23.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.413+0000 7f78c67fc640 1 -- 172.21.15.86:0/4157294432 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d0045020 con 0x55f601b356d0 2024-04-21T08:39:23.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.413+0000 7f78c67fc640 1 -- 172.21.15.86:0/4157294432 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78d002c7e0 con 0x55f601b356d0 2024-04-21T08:39:23.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.413+0000 7f78c67fc640 1 -- 172.21.15.86:0/4157294432 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d0005480 con 0x55f601b356d0 2024-04-21T08:39:23.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.414+0000 7f78e9622740 1 -- 172.21.15.86:0/4157294432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f601b356d0 msgr2=0x55f601b21650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.414+0000 7f78e9622740 1 --2- 172.21.15.86:0/4157294432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f601b356d0 0x55f601b21650 secure :-1 s=READY pgs=658 cs=0 l=1 rev1=1 crypto rx=0x7f78e0101e80 tx=0x7f78d0005fd0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.414+0000 7f78e9622740 1 -- 172.21.15.86:0/4157294432 shutdown_connections 2024-04-21T08:39:23.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.414+0000 7f78e9622740 1 --2- 172.21.15.86:0/4157294432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f601b21c70 0x55f601b220d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.414+0000 7f78e9622740 1 --2- 172.21.15.86:0/4157294432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f601b356d0 0x55f601b21650 unknown :-1 s=CLOSED pgs=658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.414+0000 7f78e9622740 1 --2- 172.21.15.86:0/4157294432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f601b34d20 0x55f601b35100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.414+0000 7f78e9622740 1 -- 172.21.15.86:0/4157294432 >> 172.21.15.86:0/4157294432 conn(0x55f601b24110 msgr2=0x55f601b244f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:23.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.416+0000 7f78e9622740 1 -- 172.21.15.86:0/4157294432 shutdown_connections 2024-04-21T08:39:23.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.416+0000 7f78e9622740 1 -- 172.21.15.86:0/4157294432 wait complete. 2024-04-21T08:39:23.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.416+0000 7f78e9622740 1 -- start start 2024-04-21T08:39:23.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.416+0000 7f78e9622740 1 Processor -- start 2024-04-21T08:39:23.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.416+0000 7f78e9622740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f601b21c70 0x55f601b299e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.416+0000 7f78e9622740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f601b34d20 0x55f601b29f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.416+0000 7f78e9622740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f601b356d0 0x55f601b2f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.417+0000 7f78e9622740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f601b25c20 con 0x55f601b356d0 2024-04-21T08:39:23.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.417+0000 7f78e9622740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55f601a45500 con 0x55f601b34d20 2024-04-21T08:39:23.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.417+0000 7f78df7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f601b356d0 0x55f601b2f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.417+0000 7f78df7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f601b21c70 0x55f601b299e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.417+0000 7f78e9622740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f601b25da0 con 0x55f601b21c70 2024-04-21T08:39:23.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.417+0000 7f78df7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f601b21c70 0x55f601b299e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.86:51494/0 (socket says 172.21.15.86:51494) 2024-04-21T08:39:23.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.417+0000 7f78df7fe640 1 -- 172.21.15.86:0/2249127236 learned_addr learned my addr 172.21.15.86:0/2249127236 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:23.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.417+0000 7f78dffff640 1 --2- 172.21.15.86:0/2249127236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f601b34d20 0x55f601b29f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.417+0000 7f78e9622740 10 client.0 _ll_register_callbacks cb 0x55f601a13b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:23.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.417+0000 7f78df7fe640 1 -- 172.21.15.86:0/2249127236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f601b34d20 msgr2=0x55f601b29f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:23.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.417+0000 7f78df7fe640 1 --2- 172.21.15.86:0/2249127236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f601b34d20 0x55f601b29f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.417+0000 7f78df7fe640 1 -- 172.21.15.86:0/2249127236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f601b356d0 msgr2=0x55f601b2f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:23.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.417+0000 7f78df7fe640 1 --2- 172.21.15.86:0/2249127236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f601b356d0 0x55f601b2f700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.417+0000 7f78df7fe640 1 -- 172.21.15.86:0/2249127236 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f60179e3f0 con 0x55f601b21c70 2024-04-21T08:39:23.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.417+0000 7f78df7fe640 1 --2- 172.21.15.86:0/2249127236 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f601b21c70 0x55f601b299e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f78d4011e00 tx=0x7f78d4004730 comp rx=0 tx=0).ready entity=mon.1 client_cookie=752f91f13772920 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:23.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.417+0000 7f78bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-04-21T08:39:23.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.417+0000 7f78bf7fe640 1 -- 172.21.15.86:0/2249127236 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d4011270 con 0x55f601b21c70 2024-04-21T08:39:23.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.418+0000 7f78bf7fe640 1 -- 172.21.15.86:0/2249127236 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f78d4004050 con 0x55f601b21c70 2024-04-21T08:39:23.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.418+0000 7f78e9622740 1 -- 172.21.15.86:0/2249127236 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78d002c7e0 con 0x55f601b21c70 2024-04-21T08:39:23.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.418+0000 7f78bf7fe640 1 -- 172.21.15.86:0/2249127236 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f78d4029050 con 0x55f601b21c70 2024-04-21T08:39:23.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.418+0000 7f78e9622740 10 client.25351 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:23.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.418+0000 7f78e9622740 1 -- 172.21.15.86:0/2249127236 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f6018687d0 con 0x55f601b21c70 2024-04-21T08:39:23.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.418+0000 7f78bf7fe640 1 -- 172.21.15.86:0/2249127236 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78d002c7e0 con 0x55f601b21c70 2024-04-21T08:39:23.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.418+0000 7f78bf7fe640 1 -- 172.21.15.86:0/2249127236 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f78d40047a0 con 0x55f601b21c70 2024-04-21T08:39:23.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.418+0000 7f78e9622740 10 client.25351 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:23.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.418+0000 7f78e9622740 10 client.25351 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:39:23.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.418+0000 7f78e9622740 20 client.25351 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:23.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.418+0000 7f78e9622740 1 --2- 172.21.15.86:0/2249127236 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55f601b51c40 0x55f601b540d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.418+0000 7f78e9622740 4 client.25351 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:23.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.418+0000 7f78e9622740 1 -- 172.21.15.86:0/2249127236 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f601a2b930 con 0x55f601b51c40 2024-04-21T08:39:23.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.419+0000 7f78dffff640 1 --2- 172.21.15.86:0/2249127236 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55f601b51c40 0x55f601b540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.419+0000 7f78dffff640 1 --2- 172.21.15.86:0/2249127236 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55f601b51c40 0x55f601b540d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=55ef8772698e1d58 server_cookie=d31f44a6872e74ad in_seq=0 out_seq=0 2024-04-21T08:39:23.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.419+0000 7f78bf7fe640 10 client.25351 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:39:23.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.420+0000 7f78bf7fe640 1 -- 172.21.15.86:0/2249127236 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f601a2b930 con 0x55f601b51c40 2024-04-21T08:39:23.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.420+0000 7f78bf7fe640 10 client.25351 handle_command_reply: tid=0 2024-04-21T08:39:23.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.420+0000 7f78e9622740 1 client.25351 shutdown 2024-04-21T08:39:23.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.420+0000 7f78e9622740 1 -- 172.21.15.86:0/2249127236 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f601b21c70 msgr2=0x55f601b299e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.420+0000 7f78e9622740 1 --2- 172.21.15.86:0/2249127236 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f601b21c70 0x55f601b299e0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7f78d4011e00 tx=0x7f78d4004730 comp rx=0 tx=0).stop 2024-04-21T08:39:23.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.420+0000 7f78e9622740 1 -- 172.21.15.86:0/2249127236 shutdown_connections 2024-04-21T08:39:23.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.420+0000 7f78e9622740 1 --2- 172.21.15.86:0/2249127236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f601b356d0 0x55f601b2f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.420+0000 7f78e9622740 1 --2- 172.21.15.86:0/2249127236 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55f601b51c40 0x55f601b540d0 crc :-1 s=READY pgs=34 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.420+0000 7f78e9622740 1 --2- 172.21.15.86:0/2249127236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f601b34d20 0x55f601b29f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.420+0000 7f78e9622740 1 --2- 172.21.15.86:0/2249127236 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f601b21c70 0x55f601b299e0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.420+0000 7f78e9622740 1 -- 172.21.15.86:0/2249127236 >> 172.21.15.86:0/2249127236 conn(0x55f601b24110 msgr2=0x55f601b23870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:23.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.420+0000 7f78bf7fe640 0 client.25351 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:39:23.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.420+0000 7f78e9622740 1 -- 172.21.15.86:0/2249127236 shutdown_connections 2024-04-21T08:39:23.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.420+0000 7f78e9622740 1 -- 172.21.15.86:0/2249127236 wait complete. 2024-04-21T08:39:23.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.420+0000 7f78e9622740 20 client.25351 trim_cache size 0 max 16384 2024-04-21T08:39:23.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.433+0000 7f78e9622740 1 Processor -- start 2024-04-21T08:39:23.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.433+0000 7f78e9622740 1 -- start start 2024-04-21T08:39:23.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.434+0000 7f78e9622740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f601b21c70 0x55f601b3a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.434+0000 7f78e9622740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f601b34d20 0x55f601b33760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.434+0000 7f78e9622740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f601b356d0 0x55f601b599a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.434+0000 7f78e9622740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f78e01ad620 con 0x55f601b21c70 2024-04-21T08:39:23.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.434+0000 7f78e9622740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f78e01af160 con 0x55f601b356d0 2024-04-21T08:39:23.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.434+0000 7f78e9622740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f78e01adcf0 con 0x55f601b34d20 2024-04-21T08:39:23.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.434+0000 7f78df7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f601b21c70 0x55f601b3a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.434+0000 7f78df7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f601b21c70 0x55f601b3a2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55514/0 (socket says 172.21.15.86:55514) 2024-04-21T08:39:23.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.434+0000 7f78df7fe640 1 -- 172.21.15.86:0/1033636938 learned_addr learned my addr 172.21.15.86:0/1033636938 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:23.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.434+0000 7f78dffff640 1 --2- 172.21.15.86:0/1033636938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f601b34d20 0x55f601b33760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.434+0000 7f78df7fe640 1 --2- 172.21.15.86:0/1033636938 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f601b356d0 0x55f601b599a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.434+0000 7f78df7fe640 1 -- 172.21.15.86:0/1033636938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f601b34d20 msgr2=0x55f601b33760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:23.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.434+0000 7f78df7fe640 1 --2- 172.21.15.86:0/1033636938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f601b34d20 0x55f601b33760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.434+0000 7f78df7fe640 1 -- 172.21.15.86:0/1033636938 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f601b356d0 msgr2=0x55f601b599a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:23.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.434+0000 7f78df7fe640 1 --2- 172.21.15.86:0/1033636938 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f601b356d0 0x55f601b599a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.434+0000 7f78df7fe640 1 -- 172.21.15.86:0/1033636938 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f601b3b950 con 0x55f601b21c70 2024-04-21T08:39:23.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.434+0000 7f78df7fe640 1 --2- 172.21.15.86:0/1033636938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f601b21c70 0x55f601b3a2c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f78d4001eb0 tx=0x7f78d4009620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b4efe7ff112c751 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:23.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.435+0000 7f78c5ffb640 1 -- 172.21.15.86:0/1033636938 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d4035020 con 0x55f601b21c70 2024-04-21T08:39:23.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.435+0000 7f78c5ffb640 1 -- 172.21.15.86:0/1033636938 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78d4005d30 con 0x55f601b21c70 2024-04-21T08:39:23.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.435+0000 7f78c5ffb640 1 -- 172.21.15.86:0/1033636938 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d402e670 con 0x55f601b21c70 2024-04-21T08:39:23.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.435+0000 7f78e9622740 1 -- 172.21.15.86:0/1033636938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f601b21c70 msgr2=0x55f601b3a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.435+0000 7f78e9622740 1 --2- 172.21.15.86:0/1033636938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f601b21c70 0x55f601b3a2c0 secure :-1 s=READY pgs=659 cs=0 l=1 rev1=1 crypto rx=0x7f78d4001eb0 tx=0x7f78d4009620 comp rx=0 tx=0).stop 2024-04-21T08:39:23.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.435+0000 7f78e9622740 1 -- 172.21.15.86:0/1033636938 shutdown_connections 2024-04-21T08:39:23.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.435+0000 7f78e9622740 1 --2- 172.21.15.86:0/1033636938 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f601b356d0 0x55f601b599a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.435+0000 7f78e9622740 1 --2- 172.21.15.86:0/1033636938 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f601b34d20 0x55f601b33760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.435+0000 7f78e9622740 1 --2- 172.21.15.86:0/1033636938 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f601b21c70 0x55f601b3a2c0 unknown :-1 s=CLOSED pgs=659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.435+0000 7f78e9622740 1 -- 172.21.15.86:0/1033636938 >> 172.21.15.86:0/1033636938 conn(0x55f601b24110 msgr2=0x55f601b23800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:23.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.435+0000 7f78e9622740 1 -- 172.21.15.86:0/1033636938 shutdown_connections 2024-04-21T08:39:23.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.436+0000 7f78e9622740 1 -- 172.21.15.86:0/1033636938 wait complete. 2024-04-21T08:39:23.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.436+0000 7f78e9622740 1 -- start start 2024-04-21T08:39:23.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.436+0000 7f78e9622740 1 Processor -- start 2024-04-21T08:39:23.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.436+0000 7f78e9622740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f601b34d20 0x55f601b65490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.436+0000 7f78e9622740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f601b356d0 0x55f601b60760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.437+0000 7f78e9622740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f601b60e00 0x55f601bb6cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.437+0000 7f78df7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f601b34d20 0x55f601b65490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.437+0000 7f78e9622740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f78e01aea90 con 0x55f601b34d20 2024-04-21T08:39:23.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.437+0000 7f78df7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f601b34d20 0x55f601b65490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55520/0 (socket says 172.21.15.86:55520) 2024-04-21T08:39:23.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.437+0000 7f78df7fe640 1 -- 172.21.15.86:0/555165412 learned_addr learned my addr 172.21.15.86:0/555165412 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:23.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.437+0000 7f78dffff640 1 --2- 172.21.15.86:0/555165412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f601b356d0 0x55f601b60760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.437+0000 7f78e9622740 1 -- 172.21.15.86:0/555165412 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f78e01ae3c0 con 0x55f601b60e00 2024-04-21T08:39:23.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.437+0000 7f78e9622740 1 -- 172.21.15.86:0/555165412 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f78e01ab5e0 con 0x55f601b356d0 2024-04-21T08:39:23.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.437+0000 7f78e9622740 10 client.0 _ll_register_callbacks cb 0x7f78d4011cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:23.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.437+0000 7f78dffff640 1 -- 172.21.15.86:0/555165412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f601b60e00 msgr2=0x55f601bb6cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:23.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.437+0000 7f78dffff640 1 --2- 172.21.15.86:0/555165412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f601b60e00 0x55f601bb6cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.437+0000 7f78dffff640 1 -- 172.21.15.86:0/555165412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f601b34d20 msgr2=0x55f601b65490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:23.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.437+0000 7f78dffff640 1 --2- 172.21.15.86:0/555165412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f601b34d20 0x55f601b65490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.437+0000 7f78dffff640 1 -- 172.21.15.86:0/555165412 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f601b61540 con 0x55f601b356d0 2024-04-21T08:39:23.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.437+0000 7f78e9622740 1 -- 172.21.15.86:0/555165412 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78d4005d30 con 0x55f601b356d0 2024-04-21T08:39:23.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.437+0000 7f78df7fe640 1 --2- 172.21.15.86:0/555165412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f601b34d20 0x55f601b65490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:23.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.437+0000 7f78dffff640 1 --2- 172.21.15.86:0/555165412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f601b356d0 0x55f601b60760 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f78d0004650 tx=0x7f78d00050e0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=501d76a23c3a899a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:23.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.437+0000 7f78bf7fe640 10 client.25357 ms_handle_connect on v2:172.21.15.179:3300/0 2024-04-21T08:39:23.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.437+0000 7f78bf7fe640 1 -- 172.21.15.86:0/555165412 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78d0045040 con 0x55f601b356d0 2024-04-21T08:39:23.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.438+0000 7f78bf7fe640 1 -- 172.21.15.86:0/555165412 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f78d0052050 con 0x55f601b356d0 2024-04-21T08:39:23.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.438+0000 7f78bf7fe640 1 -- 172.21.15.86:0/555165412 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f78d002cc70 con 0x55f601b356d0 2024-04-21T08:39:23.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.438+0000 7f78e9622740 10 client.25357 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:23.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.438+0000 7f78e9622740 1 -- 172.21.15.86:0/555165412 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({fsmap=29}) -- 0x55f601bb8110 con 0x55f601b356d0 2024-04-21T08:39:23.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.438+0000 7f78bf7fe640 1 -- 172.21.15.86:0/555165412 <== mon.1 v2:172.21.15.179:3300/0 4 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f78d0052bc0 con 0x55f601b356d0 2024-04-21T08:39:23.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.438+0000 7f78e9622740 10 client.25357 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:23.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.438+0000 7f78e9622740 10 client.25357 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:39:23.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.438+0000 7f78e9622740 20 client.25357 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:23.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.438+0000 7f78e9622740 1 --2- 172.21.15.86:0/555165412 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55f601bb8340 0x55f601bba730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.438+0000 7f78e9622740 4 client.25357 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:39:23.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.438+0000 7f78e9622740 1 -- 172.21.15.86:0/555165412 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f601a99c10 con 0x55f601bb8340 2024-04-21T08:39:23.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.438+0000 7f78df7fe640 1 --2- 172.21.15.86:0/555165412 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55f601bb8340 0x55f601bba730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.439+0000 7f78bf7fe640 10 client.25357 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:39:23.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.439+0000 7f78df7fe640 1 --2- 172.21.15.86:0/555165412 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55f601bb8340 0x55f601bba730 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6efffdbb94d804cd server_cookie=7dc356a9cfd1d463 in_seq=0 out_seq=0 2024-04-21T08:39:23.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.439+0000 7f78bf7fe640 1 -- 172.21.15.86:0/555165412 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f601a99c10 con 0x55f601bb8340 2024-04-21T08:39:23.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.439+0000 7f78bf7fe640 10 client.25357 handle_command_reply: tid=0 2024-04-21T08:39:23.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.439+0000 7f78e9622740 1 client.25357 shutdown 2024-04-21T08:39:23.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.439+0000 7f78e9622740 1 -- 172.21.15.86:0/555165412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f601b356d0 msgr2=0x55f601b60760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.439+0000 7f78e9622740 1 --2- 172.21.15.86:0/555165412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f601b356d0 0x55f601b60760 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f78d0004650 tx=0x7f78d00050e0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.439+0000 7f78e9622740 1 -- 172.21.15.86:0/555165412 shutdown_connections 2024-04-21T08:39:23.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.439+0000 7f78e9622740 1 --2- 172.21.15.86:0/555165412 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55f601bb8340 0x55f601bba730 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.439+0000 7f78e9622740 1 --2- 172.21.15.86:0/555165412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f601b60e00 0x55f601bb6cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.439+0000 7f78bf7fe640 0 client.25357 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:39:23.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.439+0000 7f78e9622740 1 --2- 172.21.15.86:0/555165412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f601b356d0 0x55f601b60760 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.439+0000 7f78e9622740 1 --2- 172.21.15.86:0/555165412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f601b34d20 0x55f601b65490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.439+0000 7f78e9622740 1 -- 172.21.15.86:0/555165412 >> 172.21.15.86:0/555165412 conn(0x55f601b24110 msgr2=0x55f601b20370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:23.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.439+0000 7f78e9622740 1 -- 172.21.15.86:0/555165412 shutdown_connections 2024-04-21T08:39:23.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.439+0000 7f78e9622740 1 -- 172.21.15.86:0/555165412 wait complete. 2024-04-21T08:39:23.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.440+0000 7f78e9622740 20 client.25357 trim_cache size 0 max 16384 2024-04-21T08:39:23.444 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:39:23.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.440+0000 7f78e6a3a640 1 -- 172.21.15.86:0/3281940705 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f78c80a3e40 msgr2=0x7f78c80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.440+0000 7f78e6a3a640 1 --2- 172.21.15.86:0/3281940705 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f78c80a3e40 0x7f78c80a6300 secure :-1 s=READY pgs=87 cs=0 l=1 rev1=1 crypto rx=0x7f78d80ce4b0 tx=0x7f78d8004800 comp rx=0 tx=0).stop 2024-04-21T08:39:23.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.440+0000 7f78e6a3a640 1 -- 172.21.15.86:0/3281940705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78e0198db0 msgr2=0x7f78e0199230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.440+0000 7f78e6a3a640 1 --2- 172.21.15.86:0/3281940705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78e0198db0 0x7f78e0199230 secure :-1 s=READY pgs=657 cs=0 l=1 rev1=1 crypto rx=0x7f78d800ec70 tx=0x7f78d800c6a0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.440+0000 7f78e6a3a640 1 -- 172.21.15.86:0/3281940705 shutdown_connections 2024-04-21T08:39:23.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.440+0000 7f78e6a3a640 1 --2- 172.21.15.86:0/3281940705 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f78c80a3e40 0x7f78c80a6300 unknown :-1 s=CLOSED pgs=87 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.440+0000 7f78e6a3a640 1 --2- 172.21.15.86:0/3281940705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78e0198db0 0x7f78e0199230 unknown :-1 s=CLOSED pgs=657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.440+0000 7f78e6a3a640 1 --2- 172.21.15.86:0/3281940705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78e019ec00 0x7f78e0198870 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.440+0000 7f78e6a3a640 1 --2- 172.21.15.86:0/3281940705 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78e0107290 0x7f78e019e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.440+0000 7f78e6a3a640 1 -- 172.21.15.86:0/3281940705 >> 172.21.15.86:0/3281940705 conn(0x7f78e00793e0 msgr2=0x7f78e01002b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:23.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.440+0000 7f78e6a3a640 1 -- 172.21.15.86:0/3281940705 shutdown_connections 2024-04-21T08:39:23.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.440+0000 7f78e6a3a640 1 -- 172.21.15.86:0/3281940705 wait complete. 2024-04-21T08:39:23.453 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:39:23.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.556+0000 7fb8675e6640 1 -- 172.21.15.86:0/4042916953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8601031d0 msgr2=0x7fb8601035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.556+0000 7fb8675e6640 1 --2- 172.21.15.86:0/4042916953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8601031d0 0x7fb8601035b0 secure :-1 s=READY pgs=660 cs=0 l=1 rev1=1 crypto rx=0x7fb85c009900 tx=0x7fb85c02bd60 comp rx=0 tx=0).stop 2024-04-21T08:39:23.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.557+0000 7fb8675e6640 1 -- 172.21.15.86:0/4042916953 shutdown_connections 2024-04-21T08:39:23.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.557+0000 7fb8675e6640 1 --2- 172.21.15.86:0/4042916953 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb86010c510 0x7fb86010e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.557+0000 7fb8675e6640 1 --2- 172.21.15.86:0/4042916953 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb860103af0 0x7fb86010bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.557+0000 7fb8675e6640 1 --2- 172.21.15.86:0/4042916953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8601031d0 0x7fb8601035b0 unknown :-1 s=CLOSED pgs=660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.557+0000 7fb8675e6640 1 -- 172.21.15.86:0/4042916953 >> 172.21.15.86:0/4042916953 conn(0x7fb860077f50 msgr2=0x7fb860078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:23.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.557+0000 7fb8675e6640 1 -- 172.21.15.86:0/4042916953 shutdown_connections 2024-04-21T08:39:23.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.557+0000 7fb8675e6640 1 -- 172.21.15.86:0/4042916953 wait complete. 2024-04-21T08:39:23.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.557+0000 7fb8675e6640 1 Processor -- start 2024-04-21T08:39:23.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.557+0000 7fb8675e6640 1 -- start start 2024-04-21T08:39:23.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.558+0000 7fb8675e6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8601031d0 0x7fb86019e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.558+0000 7fb8675e6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb860103af0 0x7fb86019e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.558+0000 7fb8675e6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb86010c510 0x7fb8601985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.558+0000 7fb8675e6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb860112200 con 0x7fb8601031d0 2024-04-21T08:39:23.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.558+0000 7fb8675e6640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb860112080 con 0x7fb860103af0 2024-04-21T08:39:23.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.558+0000 7fb8675e6640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb860112380 con 0x7fb86010c510 2024-04-21T08:39:23.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.558+0000 7fb865382640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8601031d0 0x7fb86019e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.558+0000 7fb865382640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8601031d0 0x7fb86019e430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55532/0 (socket says 172.21.15.86:55532) 2024-04-21T08:39:23.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.558+0000 7fb865382640 1 -- 172.21.15.86:0/3987674671 learned_addr learned my addr 172.21.15.86:0/3987674671 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:23.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.558+0000 7fb864b81640 1 --2- 172.21.15.86:0/3987674671 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb860103af0 0x7fb86019e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.558+0000 7fb865b83640 1 --2- 172.21.15.86:0/3987674671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb86010c510 0x7fb8601985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.558+0000 7fb865382640 1 -- 172.21.15.86:0/3987674671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb86010c510 msgr2=0x7fb8601985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.558+0000 7fb865382640 1 --2- 172.21.15.86:0/3987674671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb86010c510 0x7fb8601985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.558+0000 7fb865382640 1 -- 172.21.15.86:0/3987674671 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb860103af0 msgr2=0x7fb86019e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.558+0000 7fb865382640 1 --2- 172.21.15.86:0/3987674671 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb860103af0 0x7fb86019e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.558+0000 7fb865382640 1 -- 172.21.15.86:0/3987674671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb860198d60 con 0x7fb8601031d0 2024-04-21T08:39:23.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.558+0000 7fb865b83640 1 --2- 172.21.15.86:0/3987674671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb86010c510 0x7fb8601985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:23.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.558+0000 7fb865382640 1 --2- 172.21.15.86:0/3987674671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8601031d0 0x7fb86019e430 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb85c005940 tx=0x7fb85c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:23.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.558+0000 7fb8567fc640 1 -- 172.21.15.86:0/3987674671 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c043020 con 0x7fb8601031d0 2024-04-21T08:39:23.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.558+0000 7fb8675e6640 1 -- 172.21.15.86:0/3987674671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb860198ff0 con 0x7fb8601031d0 2024-04-21T08:39:23.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.559+0000 7fb8567fc640 1 -- 172.21.15.86:0/3987674671 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb85c02dc30 con 0x7fb8601031d0 2024-04-21T08:39:23.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.559+0000 7fb8675e6640 1 -- 172.21.15.86:0/3987674671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8601a52f0 con 0x7fb8601031d0 2024-04-21T08:39:23.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.559+0000 7fb8567fc640 1 -- 172.21.15.86:0/3987674671 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c003a10 con 0x7fb8601031d0 2024-04-21T08:39:23.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.561+0000 7fb86a1d4740 1 Processor -- start 2024-04-21T08:39:23.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.561+0000 7fb86a1d4740 1 -- start start 2024-04-21T08:39:23.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.562+0000 7fb86a1d4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55918b906330 0x55918b906710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.562+0000 7fb86a1d4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55918b906ce0 0x55918b90f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.562+0000 7fb86a1d4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55918b90fe40 0x55918b9102c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.562+0000 7fb86a1d4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55918b86b1f0 con 0x55918b906330 2024-04-21T08:39:23.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.562+0000 7fb86a1d4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55918b8084b0 con 0x55918b906ce0 2024-04-21T08:39:23.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.562+0000 7fb86a1d4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55918b8671b0 con 0x55918b90fe40 2024-04-21T08:39:23.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.562+0000 7fb865b83640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55918b906330 0x55918b906710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.562+0000 7fb865b83640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55918b906330 0x55918b906710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55538/0 (socket says 172.21.15.86:55538) 2024-04-21T08:39:23.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.562+0000 7fb865b83640 1 -- 172.21.15.86:0/3497294036 learned_addr learned my addr 172.21.15.86:0/3497294036 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:23.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.562+0000 7fb864b81640 1 --2- 172.21.15.86:0/3497294036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55918b90fe40 0x55918b9102c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.562+0000 7fb865b83640 1 -- 172.21.15.86:0/3497294036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55918b90fe40 msgr2=0x55918b9102c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:23.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.562+0000 7fb865b83640 1 --2- 172.21.15.86:0/3497294036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55918b90fe40 0x55918b9102c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.562+0000 7fb865b83640 1 -- 172.21.15.86:0/3497294036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55918b906ce0 msgr2=0x55918b90f820 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-21T08:39:23.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.562+0000 7fb865b83640 1 --2- 172.21.15.86:0/3497294036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55918b906ce0 0x55918b90f820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.562+0000 7fb865b83640 1 -- 172.21.15.86:0/3497294036 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55918b7fa1b0 con 0x55918b906330 2024-04-21T08:39:23.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.562+0000 7fb865b83640 1 --2- 172.21.15.86:0/3497294036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55918b906330 0x55918b906710 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fb858007f30 tx=0x7fb85800c750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aefa990fd0c62f4d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:23.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.562+0000 7fb8567fc640 1 -- 172.21.15.86:0/3987674671 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7fb85c046020 con 0x7fb8601031d0 2024-04-21T08:39:23.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.563+0000 7fb84b7fe640 1 -- 172.21.15.86:0/3497294036 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb858014070 con 0x55918b906330 2024-04-21T08:39:23.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.563+0000 7fb84b7fe640 1 -- 172.21.15.86:0/3497294036 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb85800fd00 con 0x55918b906330 2024-04-21T08:39:23.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.563+0000 7fb84b7fe640 1 -- 172.21.15.86:0/3497294036 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85800ecc0 con 0x55918b906330 2024-04-21T08:39:23.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.563+0000 7fb8567fc640 1 --2- 172.21.15.86:0/3987674671 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb84c0a3ec0 0x7fb84c0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.564+0000 7fb86a1d4740 1 -- 172.21.15.86:0/3497294036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55918b906330 msgr2=0x55918b906710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.564+0000 7fb86a1d4740 1 --2- 172.21.15.86:0/3497294036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55918b906330 0x55918b906710 secure :-1 s=READY pgs=662 cs=0 l=1 rev1=1 crypto rx=0x7fb858007f30 tx=0x7fb85800c750 comp rx=0 tx=0).stop 2024-04-21T08:39:23.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.564+0000 7fb8567fc640 1 -- 172.21.15.86:0/3987674671 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fb85c0f2ad0 con 0x7fb8601031d0 2024-04-21T08:39:23.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.564+0000 7fb86a1d4740 1 -- 172.21.15.86:0/3497294036 shutdown_connections 2024-04-21T08:39:23.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.564+0000 7fb86a1d4740 1 --2- 172.21.15.86:0/3497294036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55918b90fe40 0x55918b9102c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.564+0000 7fb86a1d4740 1 --2- 172.21.15.86:0/3497294036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55918b906ce0 0x55918b90f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.564+0000 7fb86a1d4740 1 --2- 172.21.15.86:0/3497294036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55918b906330 0x55918b906710 unknown :-1 s=CLOSED pgs=662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.564+0000 7fb86a1d4740 1 -- 172.21.15.86:0/3497294036 >> 172.21.15.86:0/3497294036 conn(0x55918b900ba0 msgr2=0x55918b900f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:23.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.565+0000 7fb865382640 1 --2- 172.21.15.86:0/3987674671 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb84c0a3ec0 0x7fb84c0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.565+0000 7fb86a1d4740 1 -- 172.21.15.86:0/3497294036 shutdown_connections 2024-04-21T08:39:23.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.565+0000 7fb86a1d4740 1 -- 172.21.15.86:0/3497294036 wait complete. 2024-04-21T08:39:23.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.565+0000 7fb86a1d4740 1 -- start start 2024-04-21T08:39:23.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.565+0000 7fb86a1d4740 1 Processor -- start 2024-04-21T08:39:23.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.566+0000 7fb865382640 1 --2- 172.21.15.86:0/3987674671 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb84c0a3ec0 0x7fb84c0a6380 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb85c0efe60 tx=0x7fb85c02bd60 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:23.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.566+0000 7fb86a1d4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55918b906330 0x55918b916200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.566+0000 7fb86a1d4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55918b906ce0 0x55918b9135c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.566+0000 7fb86a1d4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55918b90fe40 0x55918b913b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.566+0000 7fb86a1d4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55918b916dc0 con 0x55918b906ce0 2024-04-21T08:39:23.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.566+0000 7fb86a1d4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55918b824500 con 0x55918b906330 2024-04-21T08:39:23.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.566+0000 7fb86a1d4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55918b916f40 con 0x55918b90fe40 2024-04-21T08:39:23.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.566+0000 7fb864b81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55918b906ce0 0x55918b9135c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.566+0000 7fb86a1d4740 10 client.0 _ll_register_callbacks cb 0x55918b7f2b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:23.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.566+0000 7fb864b81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55918b906ce0 0x55918b9135c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55554/0 (socket says 172.21.15.86:55554) 2024-04-21T08:39:23.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.566+0000 7fb864b81640 1 -- 172.21.15.86:0/3946442638 learned_addr learned my addr 172.21.15.86:0/3946442638 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:23.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.566+0000 7fb865b83640 1 --2- 172.21.15.86:0/3946442638 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55918b906330 0x55918b916200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.566+0000 7fb865b83640 1 --2- 172.21.15.86:0/3946442638 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55918b90fe40 0x55918b913b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.566+0000 7fb864b81640 1 -- 172.21.15.86:0/3946442638 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55918b90fe40 msgr2=0x55918b913b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:23.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.566+0000 7fb864b81640 1 --2- 172.21.15.86:0/3946442638 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55918b90fe40 0x55918b913b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.566+0000 7fb864b81640 1 -- 172.21.15.86:0/3946442638 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55918b906330 msgr2=0x55918b916200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:23.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.566+0000 7fb864b81640 1 --2- 172.21.15.86:0/3946442638 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55918b906330 0x55918b916200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.566+0000 7fb864b81640 1 -- 172.21.15.86:0/3946442638 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55918b57d3f0 con 0x55918b906ce0 2024-04-21T08:39:23.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.566+0000 7fb864b81640 1 --2- 172.21.15.86:0/3946442638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55918b906ce0 0x55918b9135c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fb850012700 tx=0x7fb850012bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5484ddfd6c024112 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:23.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.566+0000 7fb848ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:23.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.566+0000 7fb848ff9640 1 -- 172.21.15.86:0/3946442638 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb850004050 con 0x55918b906ce0 2024-04-21T08:39:23.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.567+0000 7fb848ff9640 1 -- 172.21.15.86:0/3946442638 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fb8500041f0 con 0x55918b906ce0 2024-04-21T08:39:23.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.567+0000 7fb86a1d4740 1 -- 172.21.15.86:0/3946442638 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb85800fd00 con 0x55918b906ce0 2024-04-21T08:39:23.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.567+0000 7fb848ff9640 1 -- 172.21.15.86:0/3946442638 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb850025050 con 0x55918b906ce0 2024-04-21T08:39:23.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.567+0000 7fb86a1d4740 10 client.16131 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:23.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.567+0000 7fb86a1d4740 1 -- 172.21.15.86:0/3946442638 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55918b6477d0 con 0x55918b906ce0 2024-04-21T08:39:23.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.567+0000 7fb848ff9640 1 -- 172.21.15.86:0/3946442638 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb85000fdd0 con 0x55918b906ce0 2024-04-21T08:39:23.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.567+0000 7fb848ff9640 1 -- 172.21.15.86:0/3946442638 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb850003b60 con 0x55918b906ce0 2024-04-21T08:39:23.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.567+0000 7fb86a1d4740 10 client.16131 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:23.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.567+0000 7fb86a1d4740 10 client.16131 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:39:23.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.567+0000 7fb86a1d4740 20 client.16131 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:23.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.567+0000 7fb86a1d4740 1 --2- 172.21.15.86:0/3946442638 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55918b9240f0 0x55918b926530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.567+0000 7fb86a1d4740 4 client.16131 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:23.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.567+0000 7fb86a1d4740 1 -- 172.21.15.86:0/3946442638 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x7fb85800fd00 con 0x55918b9240f0 2024-04-21T08:39:23.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.568+0000 7fb865b83640 1 --2- 172.21.15.86:0/3946442638 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55918b9240f0 0x55918b926530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.568+0000 7fb848ff9640 10 client.16131 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:39:23.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.568+0000 7fb865b83640 1 --2- 172.21.15.86:0/3946442638 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55918b9240f0 0x55918b926530 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=89b30eac6e46dc27 server_cookie=75a4680fa2c7c3e5 in_seq=0 out_seq=0 2024-04-21T08:39:23.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.569+0000 7fb848ff9640 1 -- 172.21.15.86:0/3946442638 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7fb85800fd00 con 0x55918b9240f0 2024-04-21T08:39:23.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.569+0000 7fb848ff9640 10 client.16131 handle_command_reply: tid=0 2024-04-21T08:39:23.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.569+0000 7fb86a1d4740 1 client.16131 shutdown 2024-04-21T08:39:23.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.569+0000 7fb86a1d4740 1 -- 172.21.15.86:0/3946442638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55918b906ce0 msgr2=0x55918b9135c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.569+0000 7fb86a1d4740 1 --2- 172.21.15.86:0/3946442638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55918b906ce0 0x55918b9135c0 secure :-1 s=READY pgs=663 cs=0 l=1 rev1=1 crypto rx=0x7fb850012700 tx=0x7fb850012bc0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.569+0000 7fb86a1d4740 1 -- 172.21.15.86:0/3946442638 shutdown_connections 2024-04-21T08:39:23.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.569+0000 7fb86a1d4740 1 --2- 172.21.15.86:0/3946442638 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55918b9240f0 0x55918b926530 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.569+0000 7fb86a1d4740 1 --2- 172.21.15.86:0/3946442638 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55918b90fe40 0x55918b913b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.569+0000 7fb86a1d4740 1 --2- 172.21.15.86:0/3946442638 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55918b906ce0 0x55918b9135c0 unknown :-1 s=CLOSED pgs=663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.569+0000 7fb86a1d4740 1 --2- 172.21.15.86:0/3946442638 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55918b906330 0x55918b916200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.569+0000 7fb86a1d4740 1 -- 172.21.15.86:0/3946442638 >> 172.21.15.86:0/3946442638 conn(0x55918b900ba0 msgr2=0x55918b9028f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:23.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.569+0000 7fb848ff9640 0 client.16131 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:39:23.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.569+0000 7fb86a1d4740 1 -- 172.21.15.86:0/3946442638 shutdown_connections 2024-04-21T08:39:23.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.569+0000 7fb86a1d4740 1 -- 172.21.15.86:0/3946442638 wait complete. 2024-04-21T08:39:23.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.569+0000 7fb86a1d4740 20 client.16131 trim_cache size 0 max 16384 2024-04-21T08:39:23.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.582+0000 7fb86a1d4740 1 Processor -- start 2024-04-21T08:39:23.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.582+0000 7fb86a1d4740 1 -- start start 2024-04-21T08:39:23.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.583+0000 7fb86a1d4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55918b906330 0x55918b911230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.583+0000 7fb86a1d4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55918b906ce0 0x55918b91fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.583+0000 7fb86a1d4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55918b90fe40 0x55918b912760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.583+0000 7fb86a1d4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb8601ad130 con 0x55918b906330 2024-04-21T08:39:23.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.583+0000 7fb86a1d4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb8601aec70 con 0x55918b90fe40 2024-04-21T08:39:23.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.583+0000 7fb86a1d4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb8601ad800 con 0x55918b906ce0 2024-04-21T08:39:23.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.583+0000 7fb865b83640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55918b906330 0x55918b911230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.583+0000 7fb865b83640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55918b906330 0x55918b911230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55558/0 (socket says 172.21.15.86:55558) 2024-04-21T08:39:23.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.583+0000 7fb865b83640 1 -- 172.21.15.86:0/1548270488 learned_addr learned my addr 172.21.15.86:0/1548270488 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:23.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.583+0000 7fb864b81640 1 --2- 172.21.15.86:0/1548270488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55918b906ce0 0x55918b91fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.583+0000 7fb865b83640 1 -- 172.21.15.86:0/1548270488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55918b906ce0 msgr2=0x55918b91fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:23.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.583+0000 7fb865b83640 1 --2- 172.21.15.86:0/1548270488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55918b906ce0 0x55918b91fcc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.583+0000 7fb865b83640 1 -- 172.21.15.86:0/1548270488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55918b90fe40 msgr2=0x55918b912760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:23.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.583+0000 7fb865b83640 1 --2- 172.21.15.86:0/1548270488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55918b90fe40 0x55918b912760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.583+0000 7fb865b83640 1 -- 172.21.15.86:0/1548270488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55918b906a80 con 0x55918b906330 2024-04-21T08:39:23.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.583+0000 7fb865b83640 1 --2- 172.21.15.86:0/1548270488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55918b906330 0x55918b911230 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fb85800e7f0 tx=0x7fb858007800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4abe4774f9935d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:23.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.584+0000 7fb84affd640 1 -- 172.21.15.86:0/1548270488 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb858014040 con 0x55918b906330 2024-04-21T08:39:23.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.584+0000 7fb84affd640 1 -- 172.21.15.86:0/1548270488 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb858007a40 con 0x55918b906330 2024-04-21T08:39:23.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.584+0000 7fb84affd640 1 -- 172.21.15.86:0/1548270488 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85800faa0 con 0x55918b906330 2024-04-21T08:39:23.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.584+0000 7fb86a1d4740 1 -- 172.21.15.86:0/1548270488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55918b906330 msgr2=0x55918b911230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.584+0000 7fb86a1d4740 1 --2- 172.21.15.86:0/1548270488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55918b906330 0x55918b911230 secure :-1 s=READY pgs=664 cs=0 l=1 rev1=1 crypto rx=0x7fb85800e7f0 tx=0x7fb858007800 comp rx=0 tx=0).stop 2024-04-21T08:39:23.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.585+0000 7fb86a1d4740 1 -- 172.21.15.86:0/1548270488 shutdown_connections 2024-04-21T08:39:23.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.585+0000 7fb86a1d4740 1 --2- 172.21.15.86:0/1548270488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55918b90fe40 0x55918b912760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.585+0000 7fb86a1d4740 1 --2- 172.21.15.86:0/1548270488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55918b906ce0 0x55918b91fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.585+0000 7fb86a1d4740 1 --2- 172.21.15.86:0/1548270488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55918b906330 0x55918b911230 unknown :-1 s=CLOSED pgs=664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.585+0000 7fb86a1d4740 1 -- 172.21.15.86:0/1548270488 >> 172.21.15.86:0/1548270488 conn(0x55918b900ba0 msgr2=0x55918b8ff370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:23.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.585+0000 7fb86a1d4740 1 -- 172.21.15.86:0/1548270488 shutdown_connections 2024-04-21T08:39:23.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.585+0000 7fb86a1d4740 1 -- 172.21.15.86:0/1548270488 wait complete. 2024-04-21T08:39:23.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.585+0000 7fb86a1d4740 1 -- start start 2024-04-21T08:39:23.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.585+0000 7fb86a1d4740 1 Processor -- start 2024-04-21T08:39:23.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.586+0000 7fb86a1d4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55918b906330 0x55918b91c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.586+0000 7fb86a1d4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55918b906ce0 0x55918b914c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.586+0000 7fb86a1d4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55918b90fe40 0x55918b9151b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.586+0000 7fb86a1d4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb8601ae5a0 con 0x55918b90fe40 2024-04-21T08:39:23.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.586+0000 7fb86a1d4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb8601aded0 con 0x55918b906ce0 2024-04-21T08:39:23.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.586+0000 7fb86a1d4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb8601ab0f0 con 0x55918b906330 2024-04-21T08:39:23.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.586+0000 7fb864b81640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55918b906ce0 0x55918b914c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.586+0000 7fb864b81640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55918b906ce0 0x55918b914c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.86:59512/0 (socket says 172.21.15.86:59512) 2024-04-21T08:39:23.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.586+0000 7fb864b81640 1 -- 172.21.15.86:0/2951175332 learned_addr learned my addr 172.21.15.86:0/2951175332 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:23.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.586+0000 7fb865b83640 1 --2- 172.21.15.86:0/2951175332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55918b90fe40 0x55918b9151b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.586+0000 7fb86a1d4740 10 client.0 _ll_register_callbacks cb 0x7fb850009df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:23.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.587+0000 7fb864b81640 1 -- 172.21.15.86:0/2951175332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55918b906330 msgr2=0x55918b91c3c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:39:23.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.587+0000 7fb864b81640 1 --2- 172.21.15.86:0/2951175332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55918b906330 0x55918b91c3c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.587+0000 7fb864b81640 1 -- 172.21.15.86:0/2951175332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55918b90fe40 msgr2=0x55918b9151b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:23.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.587+0000 7fb864b81640 1 --2- 172.21.15.86:0/2951175332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55918b90fe40 0x55918b9151b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.587+0000 7fb864b81640 1 -- 172.21.15.86:0/2951175332 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55918b915770 con 0x55918b906ce0 2024-04-21T08:39:23.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.587+0000 7fb865b83640 1 --2- 172.21.15.86:0/2951175332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55918b90fe40 0x55918b9151b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:23.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.587+0000 7fb864b81640 1 --2- 172.21.15.86:0/2951175332 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55918b906ce0 0x55918b914c70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb850009f00 tx=0x7fb850010850 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e55799d59f0e7599 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:23.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.587+0000 7fb848ff9640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-04-21T08:39:23.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.587+0000 7fb848ff9640 1 -- 172.21.15.86:0/2951175332 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb850010b80 con 0x55918b906ce0 2024-04-21T08:39:23.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.587+0000 7fb86a1d4740 1 -- 172.21.15.86:0/2951175332 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb858007a40 con 0x55918b906ce0 2024-04-21T08:39:23.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.587+0000 7fb848ff9640 1 -- 172.21.15.86:0/2951175332 <== mon.2 v2:172.21.15.88:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fb850018040 con 0x55918b906ce0 2024-04-21T08:39:23.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.588+0000 7fb848ff9640 1 -- 172.21.15.86:0/2951175332 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb850010d20 con 0x55918b906ce0 2024-04-21T08:39:23.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.588+0000 7fb86a1d4740 10 client.25442 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:23.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.588+0000 7fb86a1d4740 1 -- 172.21.15.86:0/2951175332 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55918b98a440 con 0x55918b906ce0 2024-04-21T08:39:23.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.588+0000 7fb848ff9640 1 -- 172.21.15.86:0/2951175332 <== mon.2 v2:172.21.15.88:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb858007a40 con 0x55918b906ce0 2024-04-21T08:39:23.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.588+0000 7fb848ff9640 1 -- 172.21.15.86:0/2951175332 <== mon.2 v2:172.21.15.88:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb850031030 con 0x55918b906ce0 2024-04-21T08:39:23.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.588+0000 7fb86a1d4740 10 client.25442 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:23.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.588+0000 7fb86a1d4740 10 client.25442 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:39:23.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.588+0000 7fb86a1d4740 20 client.25442 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:23.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.588+0000 7fb86a1d4740 1 --2- 172.21.15.86:0/2951175332 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55918b98abf0 0x55918b98d010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.588+0000 7fb86a1d4740 4 client.25442 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:39:23.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.588+0000 7fb86a1d4740 1 -- 172.21.15.86:0/2951175332 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55918b80a930 con 0x55918b98abf0 2024-04-21T08:39:23.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.588+0000 7fb865b83640 1 --2- 172.21.15.86:0/2951175332 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55918b98abf0 0x55918b98d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.589+0000 7fb848ff9640 10 client.25442 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:39:23.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.589+0000 7fb865b83640 1 --2- 172.21.15.86:0/2951175332 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55918b98abf0 0x55918b98d010 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=74a63b71c87a44da server_cookie=ec0a6d3210e7eab5 in_seq=0 out_seq=0 2024-04-21T08:39:23.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.589+0000 7fb848ff9640 1 -- 172.21.15.86:0/2951175332 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55918b80a930 con 0x55918b98abf0 2024-04-21T08:39:23.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.589+0000 7fb848ff9640 10 client.25442 handle_command_reply: tid=0 2024-04-21T08:39:23.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.589+0000 7fb86a1d4740 1 client.25442 shutdown 2024-04-21T08:39:23.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.589+0000 7fb86a1d4740 1 -- 172.21.15.86:0/2951175332 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55918b906ce0 msgr2=0x55918b914c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.589+0000 7fb86a1d4740 1 --2- 172.21.15.86:0/2951175332 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55918b906ce0 0x55918b914c70 secure :-1 s=READY pgs=63 cs=0 l=1 rev1=1 crypto rx=0x7fb850009f00 tx=0x7fb850010850 comp rx=0 tx=0).stop 2024-04-21T08:39:23.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.589+0000 7fb86a1d4740 1 -- 172.21.15.86:0/2951175332 shutdown_connections 2024-04-21T08:39:23.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.589+0000 7fb86a1d4740 1 --2- 172.21.15.86:0/2951175332 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55918b98abf0 0x55918b98d010 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.589+0000 7fb86a1d4740 1 --2- 172.21.15.86:0/2951175332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55918b90fe40 0x55918b9151b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.589+0000 7fb86a1d4740 1 --2- 172.21.15.86:0/2951175332 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55918b906ce0 0x55918b914c70 unknown :-1 s=CLOSED pgs=63 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.589+0000 7fb86a1d4740 1 --2- 172.21.15.86:0/2951175332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55918b906330 0x55918b91c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.589+0000 7fb86a1d4740 1 -- 172.21.15.86:0/2951175332 >> 172.21.15.86:0/2951175332 conn(0x55918b900ba0 msgr2=0x55918b8ffc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:23.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.589+0000 7fb848ff9640 0 client.25442 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:39:23.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.589+0000 7fb86a1d4740 1 -- 172.21.15.86:0/2951175332 shutdown_connections 2024-04-21T08:39:23.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.589+0000 7fb86a1d4740 1 -- 172.21.15.86:0/2951175332 wait complete. 2024-04-21T08:39:23.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.589+0000 7fb86a1d4740 20 client.25442 trim_cache size 0 max 16384 2024-04-21T08:39:23.593 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:39:23.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.590+0000 7fb8675e6640 1 -- 172.21.15.86:0/3987674671 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb84c0a3ec0 msgr2=0x7fb84c0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.590+0000 7fb8675e6640 1 --2- 172.21.15.86:0/3987674671 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb84c0a3ec0 0x7fb84c0a6380 secure :-1 s=READY pgs=88 cs=0 l=1 rev1=1 crypto rx=0x7fb85c0efe60 tx=0x7fb85c02bd60 comp rx=0 tx=0).stop 2024-04-21T08:39:23.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.590+0000 7fb8675e6640 1 -- 172.21.15.86:0/3987674671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8601031d0 msgr2=0x7fb86019e430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.590+0000 7fb8675e6640 1 --2- 172.21.15.86:0/3987674671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8601031d0 0x7fb86019e430 secure :-1 s=READY pgs=661 cs=0 l=1 rev1=1 crypto rx=0x7fb85c005940 tx=0x7fb85c0045e0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.590+0000 7fb8675e6640 1 -- 172.21.15.86:0/3987674671 shutdown_connections 2024-04-21T08:39:23.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.590+0000 7fb8675e6640 1 --2- 172.21.15.86:0/3987674671 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb84c0a3ec0 0x7fb84c0a6380 unknown :-1 s=CLOSED pgs=88 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.590+0000 7fb8675e6640 1 --2- 172.21.15.86:0/3987674671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb86010c510 0x7fb8601985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.590+0000 7fb8675e6640 1 --2- 172.21.15.86:0/3987674671 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb860103af0 0x7fb86019e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.590+0000 7fb8675e6640 1 --2- 172.21.15.86:0/3987674671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8601031d0 0x7fb86019e430 unknown :-1 s=CLOSED pgs=661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.590+0000 7fb8675e6640 1 -- 172.21.15.86:0/3987674671 >> 172.21.15.86:0/3987674671 conn(0x7fb860077f50 msgr2=0x7fb8600fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:23.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.590+0000 7fb8675e6640 1 -- 172.21.15.86:0/3987674671 shutdown_connections 2024-04-21T08:39:23.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.590+0000 7fb8675e6640 1 -- 172.21.15.86:0/3987674671 wait complete. 2024-04-21T08:39:23.602 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:23 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/978893574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:39:23.603 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:39:23.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.707+0000 7fa13c0f1640 1 -- 172.21.15.86:0/3477660113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa134102b50 msgr2=0x7fa134106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.707+0000 7fa13c0f1640 1 --2- 172.21.15.86:0/3477660113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa134102b50 0x7fa134106b90 secure :-1 s=READY pgs=665 cs=0 l=1 rev1=1 crypto rx=0x7fa124009900 tx=0x7fa12402bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:23.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.707+0000 7fa13c0f1640 1 -- 172.21.15.86:0/3477660113 shutdown_connections 2024-04-21T08:39:23.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.707+0000 7fa13c0f1640 1 --2- 172.21.15.86:0/3477660113 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa134107270 0x7fa13410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.707+0000 7fa13c0f1640 1 --2- 172.21.15.86:0/3477660113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa134102b50 0x7fa134106b90 unknown :-1 s=CLOSED pgs=665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.707+0000 7fa13c0f1640 1 --2- 172.21.15.86:0/3477660113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa1341021a0 0x7fa134102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.707+0000 7fa13c0f1640 1 -- 172.21.15.86:0/3477660113 >> 172.21.15.86:0/3477660113 conn(0x7fa134079350 msgr2=0x7fa134079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:23.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.707+0000 7fa13c0f1640 1 -- 172.21.15.86:0/3477660113 shutdown_connections 2024-04-21T08:39:23.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.707+0000 7fa13c0f1640 1 -- 172.21.15.86:0/3477660113 wait complete. 2024-04-21T08:39:23.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.708+0000 7fa13c0f1640 1 Processor -- start 2024-04-21T08:39:23.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.708+0000 7fa13c0f1640 1 -- start start 2024-04-21T08:39:23.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.708+0000 7fa13c0f1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1341021a0 0x7fa134074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.708+0000 7fa13c0f1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa134107270 0x7fa134074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.708+0000 7fa13c0f1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa134074cd0 0x7fa1341a6b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.708+0000 7fa13c0f1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa134112190 con 0x7fa1341021a0 2024-04-21T08:39:23.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.708+0000 7fa13c0f1640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa134112010 con 0x7fa134074cd0 2024-04-21T08:39:23.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.708+0000 7fa13c0f1640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa134112310 con 0x7fa134107270 2024-04-21T08:39:23.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.708+0000 7fa139e8d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1341021a0 0x7fa134074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.709+0000 7fa139e8d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1341021a0 0x7fa134074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55580/0 (socket says 172.21.15.86:55580) 2024-04-21T08:39:23.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.709+0000 7fa139e8d640 1 -- 172.21.15.86:0/2579262356 learned_addr learned my addr 172.21.15.86:0/2579262356 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:23.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.709+0000 7fa13968c640 1 --2- 172.21.15.86:0/2579262356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa134107270 0x7fa134074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.709+0000 7fa13a68e640 1 --2- 172.21.15.86:0/2579262356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa134074cd0 0x7fa1341a6b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.709+0000 7fa139e8d640 1 -- 172.21.15.86:0/2579262356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa134107270 msgr2=0x7fa134074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.709+0000 7fa139e8d640 1 --2- 172.21.15.86:0/2579262356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa134107270 0x7fa134074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.709+0000 7fa139e8d640 1 -- 172.21.15.86:0/2579262356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa134074cd0 msgr2=0x7fa1341a6b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.709+0000 7fa139e8d640 1 --2- 172.21.15.86:0/2579262356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa134074cd0 0x7fa1341a6b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.709+0000 7fa139e8d640 1 -- 172.21.15.86:0/2579262356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa1341a7240 con 0x7fa1341021a0 2024-04-21T08:39:23.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.709+0000 7fa13a68e640 1 --2- 172.21.15.86:0/2579262356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa134074cd0 0x7fa1341a6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:23.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.709+0000 7fa139e8d640 1 --2- 172.21.15.86:0/2579262356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1341021a0 0x7fa134074250 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fa13000d950 tx=0x7fa13000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:23.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.709+0000 7fa12affd640 1 -- 172.21.15.86:0/2579262356 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa130017030 con 0x7fa1341021a0 2024-04-21T08:39:23.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.709+0000 7fa13c0f1640 1 -- 172.21.15.86:0/2579262356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa1341a7530 con 0x7fa1341021a0 2024-04-21T08:39:23.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.709+0000 7fa12affd640 1 -- 172.21.15.86:0/2579262356 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa130004020 con 0x7fa1341021a0 2024-04-21T08:39:23.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.709+0000 7fa12affd640 1 -- 172.21.15.86:0/2579262356 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1300047a0 con 0x7fa1341021a0 2024-04-21T08:39:23.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.709+0000 7fa13c0f1640 1 -- 172.21.15.86:0/2579262356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa1341a7aa0 con 0x7fa1341021a0 2024-04-21T08:39:23.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.712+0000 7fa13edb6740 1 Processor -- start 2024-04-21T08:39:23.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.712+0000 7fa12affd640 1 -- 172.21.15.86:0/2579262356 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7fa130004940 con 0x7fa1341021a0 2024-04-21T08:39:23.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.712+0000 7fa13edb6740 1 -- start start 2024-04-21T08:39:23.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.712+0000 7fa13edb6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55914f7854d0 0x55914f7858b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.712+0000 7fa13edb6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55914f785e80 0x55914f78e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.712+0000 7fa13edb6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55914f78ea00 0x55914f790e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.712+0000 7fa13edb6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55914f6ea1f0 con 0x55914f78ea00 2024-04-21T08:39:23.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.712+0000 7fa13edb6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55914f6874b0 con 0x55914f785e80 2024-04-21T08:39:23.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.712+0000 7fa13edb6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55914f6e61b0 con 0x55914f7854d0 2024-04-21T08:39:23.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.712+0000 7fa13968c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55914f78ea00 0x55914f790e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.712+0000 7fa13968c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55914f78ea00 0x55914f790e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55594/0 (socket says 172.21.15.86:55594) 2024-04-21T08:39:23.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.712+0000 7fa13968c640 1 -- 172.21.15.86:0/4074264909 learned_addr learned my addr 172.21.15.86:0/4074264909 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:23.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.712+0000 7fa12affd640 1 --2- 172.21.15.86:0/2579262356 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa1200a3e70 0x7fa1200a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.713+0000 7fa13a68e640 1 --2- 172.21.15.86:0/4074264909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55914f7854d0 0x55914f7858b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.713+0000 7fa13a68e640 1 --2- 172.21.15.86:0/2579262356 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa1200a3e70 0x7fa1200a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.713+0000 7fa13968c640 1 -- 172.21.15.86:0/4074264909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55914f7854d0 msgr2=0x55914f7858b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:23.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.713+0000 7fa13968c640 1 --2- 172.21.15.86:0/4074264909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55914f7854d0 0x55914f7858b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.713+0000 7fa13968c640 1 -- 172.21.15.86:0/4074264909 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55914f785e80 msgr2=0x55914f78e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:23.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.713+0000 7fa139e8d640 1 --2- 172.21.15.86:0/4074264909 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55914f785e80 0x55914f78e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.713+0000 7fa12affd640 1 -- 172.21.15.86:0/2579262356 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fa1300d0850 con 0x7fa1341021a0 2024-04-21T08:39:23.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.713+0000 7fa13968c640 1 --2- 172.21.15.86:0/4074264909 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55914f785e80 0x55914f78e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.713+0000 7fa13968c640 1 -- 172.21.15.86:0/4074264909 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55914f6791b0 con 0x55914f78ea00 2024-04-21T08:39:23.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.713+0000 7fa139e8d640 1 --2- 172.21.15.86:0/4074264909 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55914f785e80 0x55914f78e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:23.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.713+0000 7fa13968c640 1 --2- 172.21.15.86:0/4074264909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55914f78ea00 0x55914f790e50 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fa13406b650 tx=0x7fa124005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14415e134e0b8fd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:23.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.713+0000 7fa13a68e640 1 --2- 172.21.15.86:0/2579262356 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa1200a3e70 0x7fa1200a6330 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa12c005fd0 tx=0x7fa12c009900 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:23.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.713+0000 7fa11ffff640 1 -- 172.21.15.86:0/4074264909 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1240040c0 con 0x55914f78ea00 2024-04-21T08:39:23.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.713+0000 7fa11ffff640 1 -- 172.21.15.86:0/4074264909 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa124046070 con 0x55914f78ea00 2024-04-21T08:39:23.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.713+0000 7fa11ffff640 1 -- 172.21.15.86:0/4074264909 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa124005540 con 0x55914f78ea00 2024-04-21T08:39:23.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.714+0000 7fa13edb6740 1 -- 172.21.15.86:0/4074264909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55914f78ea00 msgr2=0x55914f790e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.714+0000 7fa13edb6740 1 --2- 172.21.15.86:0/4074264909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55914f78ea00 0x55914f790e50 secure :-1 s=READY pgs=667 cs=0 l=1 rev1=1 crypto rx=0x7fa13406b650 tx=0x7fa124005c00 comp rx=0 tx=0).stop 2024-04-21T08:39:23.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.714+0000 7fa13edb6740 1 -- 172.21.15.86:0/4074264909 shutdown_connections 2024-04-21T08:39:23.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.714+0000 7fa13edb6740 1 --2- 172.21.15.86:0/4074264909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55914f78ea00 0x55914f790e50 unknown :-1 s=CLOSED pgs=667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.714+0000 7fa13edb6740 1 --2- 172.21.15.86:0/4074264909 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55914f785e80 0x55914f78e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.714+0000 7fa13edb6740 1 --2- 172.21.15.86:0/4074264909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55914f7854d0 0x55914f7858b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.714+0000 7fa13edb6740 1 -- 172.21.15.86:0/4074264909 >> 172.21.15.86:0/4074264909 conn(0x55914f77fba0 msgr2=0x55914f77ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:23.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.714+0000 7fa13edb6740 1 -- 172.21.15.86:0/4074264909 shutdown_connections 2024-04-21T08:39:23.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.714+0000 7fa13edb6740 1 -- 172.21.15.86:0/4074264909 wait complete. 2024-04-21T08:39:23.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.715+0000 7fa13edb6740 1 -- start start 2024-04-21T08:39:23.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.715+0000 7fa13edb6740 1 Processor -- start 2024-04-21T08:39:23.717 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:23 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/978893574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:39:23.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.715+0000 7fa13edb6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55914f7854d0 0x55914f78e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.715+0000 7fa13edb6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55914f785e80 0x55914f78b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.715+0000 7fa13edb6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55914f78ea00 0x55914f78bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.715+0000 7fa13edb6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55914f791910 con 0x55914f7854d0 2024-04-21T08:39:23.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.715+0000 7fa13edb6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55914f6a3500 con 0x55914f785e80 2024-04-21T08:39:23.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.715+0000 7fa13edb6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55914f791a90 con 0x55914f78ea00 2024-04-21T08:39:23.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.715+0000 7fa13edb6740 10 client.0 _ll_register_callbacks cb 0x55914f671b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:23.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.715+0000 7fa139e8d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55914f7854d0 0x55914f78e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.715+0000 7fa139e8d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55914f7854d0 0x55914f78e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55608/0 (socket says 172.21.15.86:55608) 2024-04-21T08:39:23.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.715+0000 7fa139e8d640 1 -- 172.21.15.86:0/4163504574 learned_addr learned my addr 172.21.15.86:0/4163504574 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:23.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.715+0000 7fa13968c640 1 --2- 172.21.15.86:0/4163504574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55914f785e80 0x55914f78b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.716+0000 7fa13a68e640 1 --2- 172.21.15.86:0/4163504574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55914f78ea00 0x55914f78bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.716+0000 7fa139e8d640 1 -- 172.21.15.86:0/4163504574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55914f78ea00 msgr2=0x55914f78bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:23.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.716+0000 7fa139e8d640 1 --2- 172.21.15.86:0/4163504574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55914f78ea00 0x55914f78bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.716+0000 7fa139e8d640 1 -- 172.21.15.86:0/4163504574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55914f785e80 msgr2=0x55914f78b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:23.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.716+0000 7fa139e8d640 1 --2- 172.21.15.86:0/4163504574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55914f785e80 0x55914f78b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.716+0000 7fa139e8d640 1 -- 172.21.15.86:0/4163504574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55914f3fc3f0 con 0x55914f7854d0 2024-04-21T08:39:23.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.716+0000 7fa139e8d640 1 --2- 172.21.15.86:0/4163504574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55914f7854d0 0x55914f78e2f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa13009c1c0 tx=0x7fa13009c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7719ab3d100d790b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:23.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.716+0000 7fa11d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:23.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.716+0000 7fa11d7fa640 1 -- 172.21.15.86:0/4163504574 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa1300b4020 con 0x55914f7854d0 2024-04-21T08:39:23.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.716+0000 7fa11d7fa640 1 -- 172.21.15.86:0/4163504574 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fa1300a0170 con 0x55914f7854d0 2024-04-21T08:39:23.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.716+0000 7fa13edb6740 1 -- 172.21.15.86:0/4163504574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa124046070 con 0x55914f7854d0 2024-04-21T08:39:23.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.716+0000 7fa11d7fa640 1 -- 172.21.15.86:0/4163504574 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa1300a4ea0 con 0x55914f7854d0 2024-04-21T08:39:23.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.716+0000 7fa13edb6740 10 client.16149 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:23.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.716+0000 7fa13edb6740 1 -- 172.21.15.86:0/4163504574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55914f4c67d0 con 0x55914f7854d0 2024-04-21T08:39:23.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.716+0000 7fa11d7fa640 1 -- 172.21.15.86:0/4163504574 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa124046070 con 0x55914f7854d0 2024-04-21T08:39:23.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.717+0000 7fa11d7fa640 1 -- 172.21.15.86:0/4163504574 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa130097f80 con 0x55914f7854d0 2024-04-21T08:39:23.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.717+0000 7fa13edb6740 10 client.16149 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:23.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.717+0000 7fa13edb6740 10 client.16149 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:39:23.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.717+0000 7fa13edb6740 20 client.16149 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:23.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.717+0000 7fa13edb6740 1 --2- 172.21.15.86:0/4163504574 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55914f79f070 0x55914f7a14c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.717+0000 7fa13edb6740 4 client.16149 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:23.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.717+0000 7fa13edb6740 1 -- 172.21.15.86:0/4163504574 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55914f689930 con 0x55914f79f070 2024-04-21T08:39:23.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.717+0000 7fa13968c640 1 --2- 172.21.15.86:0/4163504574 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55914f79f070 0x55914f7a14c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.717+0000 7fa13968c640 1 --2- 172.21.15.86:0/4163504574 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55914f79f070 0x55914f7a14c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e5631549028c3f88 server_cookie=a0ef9c81a25c3434 in_seq=0 out_seq=0 2024-04-21T08:39:23.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.717+0000 7fa11d7fa640 10 client.16149 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:39:23.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.718+0000 7fa11d7fa640 1 -- 172.21.15.86:0/4163504574 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55914f689930 con 0x55914f79f070 2024-04-21T08:39:23.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.718+0000 7fa11d7fa640 10 client.16149 handle_command_reply: tid=0 2024-04-21T08:39:23.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.718+0000 7fa13edb6740 1 client.16149 shutdown 2024-04-21T08:39:23.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.718+0000 7fa13edb6740 1 -- 172.21.15.86:0/4163504574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55914f7854d0 msgr2=0x55914f78e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.718+0000 7fa13edb6740 1 --2- 172.21.15.86:0/4163504574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55914f7854d0 0x55914f78e2f0 secure :-1 s=READY pgs=668 cs=0 l=1 rev1=1 crypto rx=0x7fa13009c1c0 tx=0x7fa13009c9f0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.718+0000 7fa13edb6740 1 -- 172.21.15.86:0/4163504574 shutdown_connections 2024-04-21T08:39:23.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.718+0000 7fa13edb6740 1 --2- 172.21.15.86:0/4163504574 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55914f79f070 0x55914f7a14c0 crc :-1 s=READY pgs=35 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.718+0000 7fa13edb6740 1 --2- 172.21.15.86:0/4163504574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55914f78ea00 0x55914f78bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.718+0000 7fa13edb6740 1 --2- 172.21.15.86:0/4163504574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55914f785e80 0x55914f78b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.718+0000 7fa13edb6740 1 --2- 172.21.15.86:0/4163504574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55914f7854d0 0x55914f78e2f0 unknown :-1 s=CLOSED pgs=668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.718+0000 7fa11d7fa640 0 client.16149 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:39:23.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.719+0000 7fa13edb6740 1 -- 172.21.15.86:0/4163504574 >> 172.21.15.86:0/4163504574 conn(0x55914f77fba0 msgr2=0x55914f7818f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:23.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.719+0000 7fa13edb6740 1 -- 172.21.15.86:0/4163504574 shutdown_connections 2024-04-21T08:39:23.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.719+0000 7fa13edb6740 1 -- 172.21.15.86:0/4163504574 wait complete. 2024-04-21T08:39:23.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.719+0000 7fa13edb6740 20 client.16149 trim_cache size 0 max 16384 2024-04-21T08:39:23.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.732+0000 7fa13edb6740 1 Processor -- start 2024-04-21T08:39:23.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.732+0000 7fa13edb6740 1 -- start start 2024-04-21T08:39:23.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.732+0000 7fa13edb6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55914f7854d0 0x55914f79a7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.732+0000 7fa13edb6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55914f785e80 0x55914f78ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.732+0000 7fa13edb6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55914f78ea00 0x55914f78e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.732+0000 7fa13edb6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa1341b1a20 con 0x55914f78ea00 2024-04-21T08:39:23.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.732+0000 7fa13edb6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa1341b3560 con 0x55914f785e80 2024-04-21T08:39:23.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.732+0000 7fa13edb6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa1341b20f0 con 0x55914f7854d0 2024-04-21T08:39:23.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.732+0000 7fa13a68e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55914f78ea00 0x55914f78e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.732+0000 7fa13968c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55914f785e80 0x55914f78ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.732+0000 7fa13968c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55914f785e80 0x55914f78ddb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.86:59564/0 (socket says 172.21.15.86:59564) 2024-04-21T08:39:23.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.732+0000 7fa13968c640 1 -- 172.21.15.86:0/161798459 learned_addr learned my addr 172.21.15.86:0/161798459 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:23.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.732+0000 7fa13968c640 1 -- 172.21.15.86:0/161798459 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55914f7854d0 msgr2=0x55914f79a7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:23.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.732+0000 7fa13968c640 1 --2- 172.21.15.86:0/161798459 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55914f7854d0 0x55914f79a7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.733+0000 7fa13968c640 1 -- 172.21.15.86:0/161798459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55914f78ea00 msgr2=0x55914f78e2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:23.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.733+0000 7fa13968c640 1 --2- 172.21.15.86:0/161798459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55914f78ea00 0x55914f78e2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.733+0000 7fa13968c640 1 -- 172.21.15.86:0/161798459 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55914f785c20 con 0x55914f785e80 2024-04-21T08:39:23.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.733+0000 7fa13968c640 1 --2- 172.21.15.86:0/161798459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55914f785e80 0x55914f78ddb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa124009850 tx=0x7fa1240050e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e006dde645a04d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:23.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.733+0000 7fa11f7fe640 1 -- 172.21.15.86:0/161798459 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa124058070 con 0x55914f785e80 2024-04-21T08:39:23.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.733+0000 7fa11f7fe640 1 -- 172.21.15.86:0/161798459 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa124046070 con 0x55914f785e80 2024-04-21T08:39:23.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.733+0000 7fa11f7fe640 1 -- 172.21.15.86:0/161798459 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa124052070 con 0x55914f785e80 2024-04-21T08:39:23.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.734+0000 7fa13edb6740 1 -- 172.21.15.86:0/161798459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55914f785e80 msgr2=0x55914f78ddb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.734+0000 7fa13edb6740 1 --2- 172.21.15.86:0/161798459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55914f785e80 0x55914f78ddb0 secure :-1 s=READY pgs=64 cs=0 l=1 rev1=1 crypto rx=0x7fa124009850 tx=0x7fa1240050e0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.734+0000 7fa13edb6740 1 -- 172.21.15.86:0/161798459 shutdown_connections 2024-04-21T08:39:23.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.734+0000 7fa13edb6740 1 --2- 172.21.15.86:0/161798459 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55914f78ea00 0x55914f78e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.734+0000 7fa13edb6740 1 --2- 172.21.15.86:0/161798459 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55914f785e80 0x55914f78ddb0 unknown :-1 s=CLOSED pgs=64 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.734+0000 7fa13edb6740 1 --2- 172.21.15.86:0/161798459 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55914f7854d0 0x55914f79a7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.734+0000 7fa13edb6740 1 -- 172.21.15.86:0/161798459 >> 172.21.15.86:0/161798459 conn(0x55914f77fba0 msgr2=0x55914f78b450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:23.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.734+0000 7fa13edb6740 1 -- 172.21.15.86:0/161798459 shutdown_connections 2024-04-21T08:39:23.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.734+0000 7fa13edb6740 1 -- 172.21.15.86:0/161798459 wait complete. 2024-04-21T08:39:23.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.734+0000 7fa13edb6740 1 -- start start 2024-04-21T08:39:23.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.734+0000 7fa13edb6740 1 Processor -- start 2024-04-21T08:39:23.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.735+0000 7fa13edb6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55914f7854d0 0x55914f796ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.735+0000 7fa13edb6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55914f785e80 0x55914f787e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.735+0000 7fa13edb6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55914f78ea00 0x55914f7883b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.735+0000 7fa13edb6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa1341b2e90 con 0x55914f78ea00 2024-04-21T08:39:23.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.735+0000 7fa13edb6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa1341b27c0 con 0x55914f785e80 2024-04-21T08:39:23.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.735+0000 7fa13edb6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa1341af9e0 con 0x55914f7854d0 2024-04-21T08:39:23.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.735+0000 7fa13edb6740 10 client.0 _ll_register_callbacks cb 0x7fa13009bb90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:23.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.735+0000 7fa13a68e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55914f78ea00 0x55914f7883b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.735+0000 7fa13a68e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55914f78ea00 0x55914f7883b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55624/0 (socket says 172.21.15.86:55624) 2024-04-21T08:39:23.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.735+0000 7fa13a68e640 1 -- 172.21.15.86:0/4124211615 learned_addr learned my addr 172.21.15.86:0/4124211615 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:23.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.735+0000 7fa13968c640 1 --2- 172.21.15.86:0/4124211615 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55914f785e80 0x55914f787e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.735+0000 7fa139e8d640 1 --2- 172.21.15.86:0/4124211615 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55914f7854d0 0x55914f796ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.735+0000 7fa13a68e640 1 -- 172.21.15.86:0/4124211615 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55914f7854d0 msgr2=0x55914f796ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:23.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.735+0000 7fa13a68e640 1 --2- 172.21.15.86:0/4124211615 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55914f7854d0 0x55914f796ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.735+0000 7fa13a68e640 1 -- 172.21.15.86:0/4124211615 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55914f785e80 msgr2=0x55914f787e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:23.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.735+0000 7fa13a68e640 1 --2- 172.21.15.86:0/4124211615 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55914f785e80 0x55914f787e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.735+0000 7fa13a68e640 1 -- 172.21.15.86:0/4124211615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55914f7888f0 con 0x55914f78ea00 2024-04-21T08:39:23.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.735+0000 7fa13968c640 1 --2- 172.21.15.86:0/4124211615 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55914f785e80 0x55914f787e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:23.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.735+0000 7fa13a68e640 1 --2- 172.21.15.86:0/4124211615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55914f78ea00 0x55914f7883b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa12c003a60 tx=0x7fa12c009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f28d793c8206494 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:23.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.735+0000 7fa11d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:23.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.736+0000 7fa11d7fa640 1 -- 172.21.15.86:0/4124211615 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa12c004b20 con 0x55914f78ea00 2024-04-21T08:39:23.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.736+0000 7fa11d7fa640 1 -- 172.21.15.86:0/4124211615 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fa12c004cc0 con 0x55914f78ea00 2024-04-21T08:39:23.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.736+0000 7fa13edb6740 1 -- 172.21.15.86:0/4124211615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa124046070 con 0x55914f78ea00 2024-04-21T08:39:23.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.736+0000 7fa11d7fa640 1 -- 172.21.15.86:0/4124211615 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa12c004040 con 0x55914f78ea00 2024-04-21T08:39:23.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.736+0000 7fa13edb6740 10 client.16155 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:23.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.736+0000 7fa13edb6740 1 -- 172.21.15.86:0/4124211615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55914f805360 con 0x55914f78ea00 2024-04-21T08:39:23.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.736+0000 7fa11d7fa640 1 -- 172.21.15.86:0/4124211615 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa124046070 con 0x55914f78ea00 2024-04-21T08:39:23.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.736+0000 7fa11d7fa640 1 -- 172.21.15.86:0/4124211615 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa12c026070 con 0x55914f78ea00 2024-04-21T08:39:23.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.736+0000 7fa13edb6740 10 client.16155 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:23.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.736+0000 7fa13edb6740 10 client.16155 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:39:23.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.736+0000 7fa13edb6740 20 client.16155 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:23.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.736+0000 7fa13edb6740 1 --2- 172.21.15.86:0/4124211615 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55914f79f070 0x55914f807ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.736+0000 7fa13edb6740 4 client.16155 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:39:23.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.736+0000 7fa13edb6740 1 -- 172.21.15.86:0/4124211615 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55914f6f7c10 con 0x55914f79f070 2024-04-21T08:39:23.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.737+0000 7fa139e8d640 1 --2- 172.21.15.86:0/4124211615 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55914f79f070 0x55914f807ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.737+0000 7fa139e8d640 1 --2- 172.21.15.86:0/4124211615 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55914f79f070 0x55914f807ba0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=682500c58a18fca9 server_cookie=554e84705679381f in_seq=0 out_seq=0 2024-04-21T08:39:23.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.737+0000 7fa11d7fa640 10 client.16155 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:39:23.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.737+0000 7fa11d7fa640 1 -- 172.21.15.86:0/4124211615 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55914f6f7c10 con 0x55914f79f070 2024-04-21T08:39:23.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.737+0000 7fa11d7fa640 10 client.16155 handle_command_reply: tid=0 2024-04-21T08:39:23.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.737+0000 7fa13edb6740 1 client.16155 shutdown 2024-04-21T08:39:23.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.738+0000 7fa13edb6740 1 -- 172.21.15.86:0/4124211615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55914f78ea00 msgr2=0x55914f7883b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.738+0000 7fa13edb6740 1 --2- 172.21.15.86:0/4124211615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55914f78ea00 0x55914f7883b0 secure :-1 s=READY pgs=669 cs=0 l=1 rev1=1 crypto rx=0x7fa12c003a60 tx=0x7fa12c009680 comp rx=0 tx=0).stop 2024-04-21T08:39:23.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.738+0000 7fa13edb6740 1 -- 172.21.15.86:0/4124211615 shutdown_connections 2024-04-21T08:39:23.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.738+0000 7fa13edb6740 1 --2- 172.21.15.86:0/4124211615 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55914f79f070 0x55914f807ba0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.738+0000 7fa13edb6740 1 --2- 172.21.15.86:0/4124211615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55914f78ea00 0x55914f7883b0 unknown :-1 s=CLOSED pgs=669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.738+0000 7fa11d7fa640 0 client.16155 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:39:23.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.738+0000 7fa13edb6740 1 --2- 172.21.15.86:0/4124211615 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55914f785e80 0x55914f787e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.738+0000 7fa13edb6740 1 --2- 172.21.15.86:0/4124211615 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55914f7854d0 0x55914f796ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.738+0000 7fa13edb6740 1 -- 172.21.15.86:0/4124211615 >> 172.21.15.86:0/4124211615 conn(0x55914f77fba0 msgr2=0x55914f77e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:23.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.738+0000 7fa13edb6740 1 -- 172.21.15.86:0/4124211615 shutdown_connections 2024-04-21T08:39:23.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.738+0000 7fa13edb6740 1 -- 172.21.15.86:0/4124211615 wait complete. 2024-04-21T08:39:23.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.738+0000 7fa13edb6740 20 client.16155 trim_cache size 0 max 16384 2024-04-21T08:39:23.742 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:39:23.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.738+0000 7fa13c0f1640 1 -- 172.21.15.86:0/2579262356 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa1200a3e70 msgr2=0x7fa1200a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.738+0000 7fa13c0f1640 1 --2- 172.21.15.86:0/2579262356 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa1200a3e70 0x7fa1200a6330 secure :-1 s=READY pgs=89 cs=0 l=1 rev1=1 crypto rx=0x7fa12c005fd0 tx=0x7fa12c009900 comp rx=0 tx=0).stop 2024-04-21T08:39:23.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.739+0000 7fa13c0f1640 1 -- 172.21.15.86:0/2579262356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1341021a0 msgr2=0x7fa134074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.739+0000 7fa13c0f1640 1 --2- 172.21.15.86:0/2579262356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1341021a0 0x7fa134074250 secure :-1 s=READY pgs=666 cs=0 l=1 rev1=1 crypto rx=0x7fa13000d950 tx=0x7fa13000de10 comp rx=0 tx=0).stop 2024-04-21T08:39:23.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.739+0000 7fa13c0f1640 1 -- 172.21.15.86:0/2579262356 shutdown_connections 2024-04-21T08:39:23.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.739+0000 7fa13c0f1640 1 --2- 172.21.15.86:0/2579262356 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa1200a3e70 0x7fa1200a6330 unknown :-1 s=CLOSED pgs=89 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.739+0000 7fa13c0f1640 1 --2- 172.21.15.86:0/2579262356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa134074cd0 0x7fa1341a6b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.739+0000 7fa13c0f1640 1 --2- 172.21.15.86:0/2579262356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa134107270 0x7fa134074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.739+0000 7fa13c0f1640 1 --2- 172.21.15.86:0/2579262356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa1341021a0 0x7fa134074250 unknown :-1 s=CLOSED pgs=666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.739+0000 7fa13c0f1640 1 -- 172.21.15.86:0/2579262356 >> 172.21.15.86:0/2579262356 conn(0x7fa134079350 msgr2=0x7fa134104d80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:23.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.739+0000 7fa13c0f1640 1 -- 172.21.15.86:0/2579262356 shutdown_connections 2024-04-21T08:39:23.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.739+0000 7fa13c0f1640 1 -- 172.21.15.86:0/2579262356 wait complete. 2024-04-21T08:39:23.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:23 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/978893574' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:39:23.749 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:39:23.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.858+0000 7fb99f515640 1 -- 172.21.15.86:0/2832506880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb998102e90 msgr2=0x7fb998100470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.858+0000 7fb99f515640 1 --2- 172.21.15.86:0/2832506880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb998102e90 0x7fb998100470 secure :-1 s=READY pgs=670 cs=0 l=1 rev1=1 crypto rx=0x7fb994009930 tx=0x7fb99402bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:23.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.858+0000 7fb99f515640 1 -- 172.21.15.86:0/2832506880 shutdown_connections 2024-04-21T08:39:23.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.858+0000 7fb99f515640 1 --2- 172.21.15.86:0/2832506880 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb99810ebf0 0x7fb998110fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.858+0000 7fb99f515640 1 --2- 172.21.15.86:0/2832506880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb998103300 0x7fb9981009b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.858+0000 7fb99f515640 1 --2- 172.21.15.86:0/2832506880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb998102e90 0x7fb998100470 unknown :-1 s=CLOSED pgs=670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.858+0000 7fb99f515640 1 -- 172.21.15.86:0/2832506880 >> 172.21.15.86:0/2832506880 conn(0x7fb998077410 msgr2=0x7fb9980fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:23.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.858+0000 7fb99f515640 1 -- 172.21.15.86:0/2832506880 shutdown_connections 2024-04-21T08:39:23.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.858+0000 7fb99f515640 1 -- 172.21.15.86:0/2832506880 wait complete. 2024-04-21T08:39:23.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.859+0000 7fb99f515640 1 Processor -- start 2024-04-21T08:39:23.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.859+0000 7fb99f515640 1 -- start start 2024-04-21T08:39:23.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.859+0000 7fb99f515640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb998102e90 0x7fb9981a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.859+0000 7fb99f515640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb998103300 0x7fb9981a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.859+0000 7fb99f515640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb99810ebf0 0x7fb99819a7c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.859+0000 7fb99f515640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb9981143e0 con 0x7fb99810ebf0 2024-04-21T08:39:23.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.859+0000 7fb99f515640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb998114260 con 0x7fb998103300 2024-04-21T08:39:23.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.859+0000 7fb99f515640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb998114560 con 0x7fb998102e90 2024-04-21T08:39:23.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.859+0000 7fb99dab2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb99810ebf0 0x7fb99819a7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.859+0000 7fb99dab2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb99810ebf0 0x7fb99819a7c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55642/0 (socket says 172.21.15.86:55642) 2024-04-21T08:39:23.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.859+0000 7fb99dab2640 1 -- 172.21.15.86:0/747256642 learned_addr learned my addr 172.21.15.86:0/747256642 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:23.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.859+0000 7fb99cab0640 1 --2- 172.21.15.86:0/747256642 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb998103300 0x7fb9981a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.860+0000 7fb99dab2640 1 -- 172.21.15.86:0/747256642 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb998102e90 msgr2=0x7fb9981a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.860+0000 7fb99dab2640 1 --2- 172.21.15.86:0/747256642 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb998102e90 0x7fb9981a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.860+0000 7fb99dab2640 1 -- 172.21.15.86:0/747256642 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb998103300 msgr2=0x7fb9981a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.860+0000 7fb99dab2640 1 --2- 172.21.15.86:0/747256642 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb998103300 0x7fb9981a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.860+0000 7fb99dab2640 1 -- 172.21.15.86:0/747256642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb99819af90 con 0x7fb99810ebf0 2024-04-21T08:39:23.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.860+0000 7fb99dab2640 1 --2- 172.21.15.86:0/747256642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb99810ebf0 0x7fb99819a7c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb990009fd0 tx=0x7fb99000b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:23.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.860+0000 7fb98e7fc640 1 -- 172.21.15.86:0/747256642 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb990017070 con 0x7fb99810ebf0 2024-04-21T08:39:23.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.860+0000 7fb98e7fc640 1 -- 172.21.15.86:0/747256642 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb990003e20 con 0x7fb99810ebf0 2024-04-21T08:39:23.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.860+0000 7fb99f515640 1 -- 172.21.15.86:0/747256642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb99819b280 con 0x7fb99810ebf0 2024-04-21T08:39:23.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.860+0000 7fb98e7fc640 1 -- 172.21.15.86:0/747256642 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb990010460 con 0x7fb99810ebf0 2024-04-21T08:39:23.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.860+0000 7fb99f515640 1 -- 172.21.15.86:0/747256642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb9981a74a0 con 0x7fb99810ebf0 2024-04-21T08:39:23.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.863+0000 7fb98e7fc640 1 -- 172.21.15.86:0/747256642 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7fb990003970 con 0x7fb99810ebf0 2024-04-21T08:39:23.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.864+0000 7fb9a21e4740 1 Processor -- start 2024-04-21T08:39:23.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.864+0000 7fb9a21e4740 1 -- start start 2024-04-21T08:39:23.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.864+0000 7fb9a21e4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644d35b44d0 0x5644d35b48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.864+0000 7fb9a21e4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5644d35b4e80 0x5644d35bd430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.864+0000 7fb9a21e4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644d35bda00 0x5644d35bfe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.864+0000 7fb9a21e4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5644d35191f0 con 0x5644d35bda00 2024-04-21T08:39:23.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.864+0000 7fb9a21e4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5644d34b64b0 con 0x5644d35b4e80 2024-04-21T08:39:23.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.864+0000 7fb9a21e4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5644d35151b0 con 0x5644d35b44d0 2024-04-21T08:39:23.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.864+0000 7fb98e7fc640 1 --2- 172.21.15.86:0/747256642 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb9840a3ec0 0x7fb9840a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.864+0000 7fb99cab0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644d35bda00 0x5644d35bfe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.864+0000 7fb99cab0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644d35bda00 0x5644d35bfe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55650/0 (socket says 172.21.15.86:55650) 2024-04-21T08:39:23.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.864+0000 7fb99cab0640 1 -- 172.21.15.86:0/3757644077 learned_addr learned my addr 172.21.15.86:0/3757644077 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:23.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.864+0000 7fb99cab0640 1 --2- 172.21.15.86:0/3757644077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644d35b44d0 0x5644d35b48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.864+0000 7fb99d2b1640 1 --2- 172.21.15.86:0/3757644077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5644d35b4e80 0x5644d35bd430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.864+0000 7fb99dab2640 1 --2- 172.21.15.86:0/747256642 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb9840a3ec0 0x7fb9840a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.865+0000 7fb98e7fc640 1 -- 172.21.15.86:0/747256642 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fb9900d0aa0 con 0x7fb99810ebf0 2024-04-21T08:39:23.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.865+0000 7fb99cab0640 1 -- 172.21.15.86:0/3757644077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644d35b44d0 msgr2=0x5644d35b48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:23.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.865+0000 7fb99cab0640 1 --2- 172.21.15.86:0/3757644077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644d35b44d0 0x5644d35b48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.865+0000 7fb99cab0640 1 -- 172.21.15.86:0/3757644077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5644d35b4e80 msgr2=0x5644d35bd430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:23.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.865+0000 7fb99cab0640 1 --2- 172.21.15.86:0/3757644077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5644d35b4e80 0x5644d35bd430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.865+0000 7fb99cab0640 1 -- 172.21.15.86:0/3757644077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644d34a81b0 con 0x5644d35bda00 2024-04-21T08:39:23.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.865+0000 7fb99d2b1640 1 --2- 172.21.15.86:0/3757644077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5644d35b4e80 0x5644d35bd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:23.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.865+0000 7fb99cab0640 1 --2- 172.21.15.86:0/3757644077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644d35bda00 0x5644d35bfe50 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb98800bf80 tx=0x7fb988005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e526e8df266ffcd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:23.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.865+0000 7fb9837fe640 1 -- 172.21.15.86:0/3757644077 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb988004220 con 0x5644d35bda00 2024-04-21T08:39:23.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.865+0000 7fb99dab2640 1 --2- 172.21.15.86:0/747256642 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb9840a3ec0 0x7fb9840a6380 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb9900cdd60 tx=0x7fb990004020 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:23.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.865+0000 7fb9837fe640 1 -- 172.21.15.86:0/3757644077 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9880043c0 con 0x5644d35bda00 2024-04-21T08:39:23.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.865+0000 7fb9837fe640 1 -- 172.21.15.86:0/3757644077 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb988004de0 con 0x5644d35bda00 2024-04-21T08:39:23.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.866+0000 7fb9a21e4740 1 -- 172.21.15.86:0/3757644077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644d35bda00 msgr2=0x5644d35bfe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.866+0000 7fb9a21e4740 1 --2- 172.21.15.86:0/3757644077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644d35bda00 0x5644d35bfe50 secure :-1 s=READY pgs=672 cs=0 l=1 rev1=1 crypto rx=0x7fb98800bf80 tx=0x7fb988005c90 comp rx=0 tx=0).stop 2024-04-21T08:39:23.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.866+0000 7fb9a21e4740 1 -- 172.21.15.86:0/3757644077 shutdown_connections 2024-04-21T08:39:23.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.866+0000 7fb9a21e4740 1 --2- 172.21.15.86:0/3757644077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644d35bda00 0x5644d35bfe50 unknown :-1 s=CLOSED pgs=672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.866+0000 7fb9a21e4740 1 --2- 172.21.15.86:0/3757644077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5644d35b4e80 0x5644d35bd430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.866+0000 7fb9a21e4740 1 --2- 172.21.15.86:0/3757644077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644d35b44d0 0x5644d35b48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.866+0000 7fb9a21e4740 1 -- 172.21.15.86:0/3757644077 >> 172.21.15.86:0/3757644077 conn(0x5644d35aeba0 msgr2=0x5644d35aef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:23.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.866+0000 7fb9a21e4740 1 -- 172.21.15.86:0/3757644077 shutdown_connections 2024-04-21T08:39:23.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.866+0000 7fb9a21e4740 1 -- 172.21.15.86:0/3757644077 wait complete. 2024-04-21T08:39:23.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.866+0000 7fb9a21e4740 1 -- start start 2024-04-21T08:39:23.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.866+0000 7fb9a21e4740 1 Processor -- start 2024-04-21T08:39:23.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.867+0000 7fb9a21e4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644d35b44d0 0x5644d35bd300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.867+0000 7fb9a21e4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644d35b4e80 0x5644d35ba690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.867+0000 7fb9a21e4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5644d35bda00 0x5644d35bac10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.867+0000 7fb9a21e4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5644d35c0970 con 0x5644d35b4e80 2024-04-21T08:39:23.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.867+0000 7fb9a21e4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5644d34d2500 con 0x5644d35bda00 2024-04-21T08:39:23.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.867+0000 7fb9a21e4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5644d35c0af0 con 0x5644d35b44d0 2024-04-21T08:39:23.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.867+0000 7fb9a21e4740 10 client.0 _ll_register_callbacks cb 0x5644d34a0b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:23.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.867+0000 7fb99d2b1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644d35b4e80 0x5644d35ba690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.867+0000 7fb99d2b1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644d35b4e80 0x5644d35ba690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55658/0 (socket says 172.21.15.86:55658) 2024-04-21T08:39:23.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.867+0000 7fb99d2b1640 1 -- 172.21.15.86:0/3067850866 learned_addr learned my addr 172.21.15.86:0/3067850866 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:23.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.867+0000 7fb99cab0640 1 --2- 172.21.15.86:0/3067850866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5644d35bda00 0x5644d35bac10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.867+0000 7fb99d2b1640 1 -- 172.21.15.86:0/3067850866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644d35b44d0 msgr2=0x5644d35bd300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:23.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.867+0000 7fb99cab0640 1 --2- 172.21.15.86:0/3067850866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644d35b44d0 0x5644d35bd300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.868+0000 7fb99d2b1640 1 --2- 172.21.15.86:0/3067850866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644d35b44d0 0x5644d35bd300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.868+0000 7fb99d2b1640 1 -- 172.21.15.86:0/3067850866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5644d35bda00 msgr2=0x5644d35bac10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:23.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.868+0000 7fb99d2b1640 1 --2- 172.21.15.86:0/3067850866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5644d35bda00 0x5644d35bac10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.868+0000 7fb99d2b1640 1 -- 172.21.15.86:0/3067850866 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644d322b3f0 con 0x5644d35b4e80 2024-04-21T08:39:23.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.868+0000 7fb99cab0640 1 --2- 172.21.15.86:0/3067850866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644d35b44d0 0x5644d35bd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:23.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.868+0000 7fb99d2b1640 1 --2- 172.21.15.86:0/3067850866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644d35b4e80 0x5644d35ba690 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fb994009230 tx=0x7fb994004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91b6987d8ed2adc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:23.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.868+0000 7fb980ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:23.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.868+0000 7fb980ff9640 1 -- 172.21.15.86:0/3067850866 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb994042070 con 0x5644d35b4e80 2024-04-21T08:39:23.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.868+0000 7fb980ff9640 1 -- 172.21.15.86:0/3067850866 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fb994005240 con 0x5644d35b4e80 2024-04-21T08:39:23.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.868+0000 7fb9a21e4740 1 -- 172.21.15.86:0/3067850866 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9880043c0 con 0x5644d35b4e80 2024-04-21T08:39:23.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.868+0000 7fb980ff9640 1 -- 172.21.15.86:0/3067850866 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb99404c050 con 0x5644d35b4e80 2024-04-21T08:39:23.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.868+0000 7fb9a21e4740 10 client.16167 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:23.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.869+0000 7fb9a21e4740 1 -- 172.21.15.86:0/3067850866 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5644d32f57d0 con 0x5644d35b4e80 2024-04-21T08:39:23.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.869+0000 7fb980ff9640 1 -- 172.21.15.86:0/3067850866 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9880043c0 con 0x5644d35b4e80 2024-04-21T08:39:23.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.869+0000 7fb980ff9640 1 -- 172.21.15.86:0/3067850866 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb994004c20 con 0x5644d35b4e80 2024-04-21T08:39:23.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.869+0000 7fb9a21e4740 10 client.16167 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:23.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.869+0000 7fb9a21e4740 10 client.16167 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:39:23.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.869+0000 7fb9a21e4740 20 client.16167 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:23.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.869+0000 7fb9a21e4740 1 --2- 172.21.15.86:0/3067850866 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5644d35d6450 0x5644d35d8870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.869+0000 7fb9a21e4740 4 client.16167 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:23.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.869+0000 7fb9a21e4740 1 -- 172.21.15.86:0/3067850866 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5644d34b8930 con 0x5644d35d6450 2024-04-21T08:39:23.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.869+0000 7fb99cab0640 1 --2- 172.21.15.86:0/3067850866 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5644d35d6450 0x5644d35d8870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.870+0000 7fb980ff9640 10 client.16167 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:39:23.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.870+0000 7fb99cab0640 1 --2- 172.21.15.86:0/3067850866 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5644d35d6450 0x5644d35d8870 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=64ee4a7e9d01f3d7 server_cookie=840aa2f91c7ad9f6 in_seq=0 out_seq=0 2024-04-21T08:39:23.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.871+0000 7fb980ff9640 1 -- 172.21.15.86:0/3067850866 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5644d34b8930 con 0x5644d35d6450 2024-04-21T08:39:23.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.871+0000 7fb980ff9640 10 client.16167 handle_command_reply: tid=0 2024-04-21T08:39:23.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.871+0000 7fb9a21e4740 1 client.16167 shutdown 2024-04-21T08:39:23.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.871+0000 7fb9a21e4740 1 -- 172.21.15.86:0/3067850866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644d35b4e80 msgr2=0x5644d35ba690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.871+0000 7fb9a21e4740 1 --2- 172.21.15.86:0/3067850866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644d35b4e80 0x5644d35ba690 secure :-1 s=READY pgs=673 cs=0 l=1 rev1=1 crypto rx=0x7fb994009230 tx=0x7fb994004610 comp rx=0 tx=0).stop 2024-04-21T08:39:23.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.871+0000 7fb9a21e4740 1 -- 172.21.15.86:0/3067850866 shutdown_connections 2024-04-21T08:39:23.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.871+0000 7fb9a21e4740 1 --2- 172.21.15.86:0/3067850866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5644d35bda00 0x5644d35bac10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.871+0000 7fb9a21e4740 1 --2- 172.21.15.86:0/3067850866 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5644d35d6450 0x5644d35d8870 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.871+0000 7fb980ff9640 0 client.16167 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:39:23.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.871+0000 7fb9a21e4740 1 --2- 172.21.15.86:0/3067850866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644d35b4e80 0x5644d35ba690 unknown :-1 s=CLOSED pgs=673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.871+0000 7fb9a21e4740 1 --2- 172.21.15.86:0/3067850866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644d35b44d0 0x5644d35bd300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.871+0000 7fb9a21e4740 1 -- 172.21.15.86:0/3067850866 >> 172.21.15.86:0/3067850866 conn(0x5644d35aeba0 msgr2=0x5644d35b08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:23.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.871+0000 7fb9a21e4740 1 -- 172.21.15.86:0/3067850866 shutdown_connections 2024-04-21T08:39:23.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.871+0000 7fb9a21e4740 1 -- 172.21.15.86:0/3067850866 wait complete. 2024-04-21T08:39:23.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.871+0000 7fb9a21e4740 20 client.16167 trim_cache size 0 max 16384 2024-04-21T08:39:23.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.884+0000 7fb9a21e4740 1 Processor -- start 2024-04-21T08:39:23.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.885+0000 7fb9a21e4740 1 -- start start 2024-04-21T08:39:23.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.885+0000 7fb9a21e4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644d35b44d0 0x5644d35c9830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.885+0000 7fb9a21e4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5644d35bda00 0x5644d35ada80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.885+0000 7fb9a21e4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644d35ba7b0 0x5644d35bcdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.885+0000 7fb9a21e4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb9981ad0e0 con 0x5644d35b44d0 2024-04-21T08:39:23.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.885+0000 7fb9a21e4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb9981aec20 con 0x5644d35bda00 2024-04-21T08:39:23.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.885+0000 7fb9a21e4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb9981ad7b0 con 0x5644d35ba7b0 2024-04-21T08:39:23.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.885+0000 7fb99cab0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644d35b44d0 0x5644d35c9830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.885 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.885+0000 7fb99cab0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644d35b44d0 0x5644d35c9830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55674/0 (socket says 172.21.15.86:55674) 2024-04-21T08:39:23.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.885+0000 7fb99cab0640 1 -- 172.21.15.86:0/2110928844 learned_addr learned my addr 172.21.15.86:0/2110928844 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:23.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.885+0000 7fb99d2b1640 1 --2- 172.21.15.86:0/2110928844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5644d35bda00 0x5644d35ada80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.885+0000 7fb99cab0640 1 --2- 172.21.15.86:0/2110928844 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644d35ba7b0 0x5644d35bcdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.885+0000 7fb99cab0640 1 -- 172.21.15.86:0/2110928844 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644d35ba7b0 msgr2=0x5644d35bcdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:23.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.885+0000 7fb99cab0640 1 --2- 172.21.15.86:0/2110928844 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644d35ba7b0 0x5644d35bcdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.885+0000 7fb99cab0640 1 -- 172.21.15.86:0/2110928844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5644d35bda00 msgr2=0x5644d35ada80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:23.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.885+0000 7fb99cab0640 1 --2- 172.21.15.86:0/2110928844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5644d35bda00 0x5644d35ada80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.885+0000 7fb99cab0640 1 -- 172.21.15.86:0/2110928844 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5644d35b9640 con 0x5644d35b44d0 2024-04-21T08:39:23.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.886+0000 7fb99cab0640 1 --2- 172.21.15.86:0/2110928844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644d35b44d0 0x5644d35c9830 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb98800b620 tx=0x7fb988004a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76bb96bfd05f9589 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:23.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.886+0000 7fb982ffd640 1 -- 172.21.15.86:0/2110928844 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb98802d240 con 0x5644d35b44d0 2024-04-21T08:39:23.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.886+0000 7fb982ffd640 1 -- 172.21.15.86:0/2110928844 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb98802d890 con 0x5644d35b44d0 2024-04-21T08:39:23.886 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.886+0000 7fb982ffd640 1 -- 172.21.15.86:0/2110928844 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb988010d50 con 0x5644d35b44d0 2024-04-21T08:39:23.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.887+0000 7fb9a21e4740 1 -- 172.21.15.86:0/2110928844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644d35b44d0 msgr2=0x5644d35c9830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.887+0000 7fb9a21e4740 1 --2- 172.21.15.86:0/2110928844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644d35b44d0 0x5644d35c9830 secure :-1 s=READY pgs=674 cs=0 l=1 rev1=1 crypto rx=0x7fb98800b620 tx=0x7fb988004a20 comp rx=0 tx=0).stop 2024-04-21T08:39:23.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.887+0000 7fb9a21e4740 1 -- 172.21.15.86:0/2110928844 shutdown_connections 2024-04-21T08:39:23.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.887+0000 7fb9a21e4740 1 --2- 172.21.15.86:0/2110928844 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644d35ba7b0 0x5644d35bcdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.887+0000 7fb9a21e4740 1 --2- 172.21.15.86:0/2110928844 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5644d35bda00 0x5644d35ada80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.887+0000 7fb9a21e4740 1 --2- 172.21.15.86:0/2110928844 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644d35b44d0 0x5644d35c9830 unknown :-1 s=CLOSED pgs=674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.887+0000 7fb9a21e4740 1 -- 172.21.15.86:0/2110928844 >> 172.21.15.86:0/2110928844 conn(0x5644d35aeba0 msgr2=0x5644d35ad370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:23.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.887+0000 7fb9a21e4740 1 -- 172.21.15.86:0/2110928844 shutdown_connections 2024-04-21T08:39:23.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.887+0000 7fb9a21e4740 1 -- 172.21.15.86:0/2110928844 wait complete. 2024-04-21T08:39:23.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.887+0000 7fb9a21e4740 1 -- start start 2024-04-21T08:39:23.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.887+0000 7fb9a21e4740 1 Processor -- start 2024-04-21T08:39:23.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.888+0000 7fb9a21e4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644d35b44d0 0x5644d35c1a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.888+0000 7fb9a21e4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644d35ba7b0 0x5644d35c1f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.888+0000 7fb9a21e4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5644d35bda00 0x5644d35c24c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.888+0000 7fb9a21e4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb9981ae550 con 0x5644d35ba7b0 2024-04-21T08:39:23.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.888+0000 7fb9a21e4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb9981ade80 con 0x5644d35bda00 2024-04-21T08:39:23.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.888+0000 7fb9a21e4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb9981ab0c0 con 0x5644d35b44d0 2024-04-21T08:39:23.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.888+0000 7fb9a21e4740 10 client.0 _ll_register_callbacks cb 0x7fb9940044d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:23.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.888+0000 7fb99d2b1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644d35ba7b0 0x5644d35c1f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.888+0000 7fb99cab0640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644d35b44d0 0x5644d35c1a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.888+0000 7fb99cab0640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644d35b44d0 0x5644d35c1a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.86:51678/0 (socket says 172.21.15.86:51678) 2024-04-21T08:39:23.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.888+0000 7fb99d2b1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644d35ba7b0 0x5644d35c1f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55682/0 (socket says 172.21.15.86:55682) 2024-04-21T08:39:23.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.888+0000 7fb99cab0640 1 -- 172.21.15.86:0/3796272424 learned_addr learned my addr 172.21.15.86:0/3796272424 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:23.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.888+0000 7fb99cab0640 1 --2- 172.21.15.86:0/3796272424 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5644d35bda00 0x5644d35c24c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.888+0000 7fb99cab0640 1 -- 172.21.15.86:0/3796272424 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5644d35bda00 msgr2=0x5644d35c24c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:23.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.888+0000 7fb99cab0640 1 --2- 172.21.15.86:0/3796272424 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5644d35bda00 0x5644d35c24c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.888+0000 7fb99cab0640 1 -- 172.21.15.86:0/3796272424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644d35ba7b0 msgr2=0x5644d35c1f60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:23.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.888+0000 7fb99cab0640 1 --2- 172.21.15.86:0/3796272424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644d35ba7b0 0x5644d35c1f60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.888+0000 7fb99cab0640 1 -- 172.21.15.86:0/3796272424 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5644d363bc60 con 0x5644d35b44d0 2024-04-21T08:39:23.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.889+0000 7fb99cab0640 1 --2- 172.21.15.86:0/3796272424 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644d35b44d0 0x5644d35c1a20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x5644d35c2a00 tx=0x7fb988031040 comp rx=0 tx=0).ready entity=mon.1 client_cookie=c60d47a38b82bb67 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:23.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.889+0000 7fb980ff9640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-04-21T08:39:23.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.889+0000 7fb980ff9640 1 -- 172.21.15.86:0/3796272424 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb988034040 con 0x5644d35b44d0 2024-04-21T08:39:23.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.889+0000 7fb980ff9640 1 -- 172.21.15.86:0/3796272424 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fb9880106b0 con 0x5644d35b44d0 2024-04-21T08:39:23.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.889+0000 7fb9a21e4740 1 -- 172.21.15.86:0/3796272424 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb98802d890 con 0x5644d35b44d0 2024-04-21T08:39:23.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.889+0000 7fb980ff9640 1 -- 172.21.15.86:0/3796272424 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb988010ad0 con 0x5644d35b44d0 2024-04-21T08:39:23.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.889+0000 7fb9a21e4740 10 client.25390 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:23.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.889+0000 7fb9a21e4740 1 -- 172.21.15.86:0/3796272424 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5644d363cf60 con 0x5644d35b44d0 2024-04-21T08:39:23.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.890+0000 7fb980ff9640 1 -- 172.21.15.86:0/3796272424 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb98802d890 con 0x5644d35b44d0 2024-04-21T08:39:23.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.890+0000 7fb980ff9640 1 -- 172.21.15.86:0/3796272424 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb98803a070 con 0x5644d35b44d0 2024-04-21T08:39:23.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.890+0000 7fb9a21e4740 10 client.25390 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:23.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.890+0000 7fb9a21e4740 10 client.25390 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:39:23.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.890+0000 7fb9a21e4740 20 client.25390 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:23.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.890+0000 7fb9a21e4740 1 --2- 172.21.15.86:0/3796272424 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5644d363d330 0x5644d363f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:23.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.890+0000 7fb9a21e4740 4 client.25390 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:39:23.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.890+0000 7fb9a21e4740 1 -- 172.21.15.86:0/3796272424 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5644d3526c10 con 0x5644d363d330 2024-04-21T08:39:23.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.890+0000 7fb99d2b1640 1 --2- 172.21.15.86:0/3796272424 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5644d363d330 0x5644d363f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:23.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.891+0000 7fb980ff9640 10 client.25390 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:39:23.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.891+0000 7fb99d2b1640 1 --2- 172.21.15.86:0/3796272424 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5644d363d330 0x5644d363f720 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a4381da82f684033 server_cookie=68cac1c68bc64a9d in_seq=0 out_seq=0 2024-04-21T08:39:23.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.891+0000 7fb980ff9640 1 -- 172.21.15.86:0/3796272424 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5644d3526c10 con 0x5644d363d330 2024-04-21T08:39:23.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.891+0000 7fb980ff9640 10 client.25390 handle_command_reply: tid=0 2024-04-21T08:39:23.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.891+0000 7fb9a21e4740 1 client.25390 shutdown 2024-04-21T08:39:23.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.891+0000 7fb9a21e4740 1 -- 172.21.15.86:0/3796272424 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644d35b44d0 msgr2=0x5644d35c1a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.891+0000 7fb9a21e4740 1 --2- 172.21.15.86:0/3796272424 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644d35b44d0 0x5644d35c1a20 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x5644d35c2a00 tx=0x7fb988031040 comp rx=0 tx=0).stop 2024-04-21T08:39:23.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.891+0000 7fb9a21e4740 1 -- 172.21.15.86:0/3796272424 shutdown_connections 2024-04-21T08:39:23.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.891+0000 7fb9a21e4740 1 --2- 172.21.15.86:0/3796272424 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5644d35bda00 0x5644d35c24c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.891+0000 7fb9a21e4740 1 --2- 172.21.15.86:0/3796272424 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5644d363d330 0x5644d363f720 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.891+0000 7fb9a21e4740 1 --2- 172.21.15.86:0/3796272424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5644d35ba7b0 0x5644d35c1f60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.891+0000 7fb980ff9640 0 client.25390 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:39:23.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.891+0000 7fb9a21e4740 1 --2- 172.21.15.86:0/3796272424 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5644d35b44d0 0x5644d35c1a20 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.891+0000 7fb9a21e4740 1 -- 172.21.15.86:0/3796272424 >> 172.21.15.86:0/3796272424 conn(0x5644d35aeba0 msgr2=0x5644d35adc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:23.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.891+0000 7fb9a21e4740 1 -- 172.21.15.86:0/3796272424 shutdown_connections 2024-04-21T08:39:23.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.891+0000 7fb9a21e4740 1 -- 172.21.15.86:0/3796272424 wait complete. 2024-04-21T08:39:23.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.891+0000 7fb9a21e4740 20 client.25390 trim_cache size 0 max 16384 2024-04-21T08:39:23.896 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:39:23.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.892+0000 7fb99f515640 1 -- 172.21.15.86:0/747256642 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb9840a3ec0 msgr2=0x7fb9840a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.892+0000 7fb99f515640 1 --2- 172.21.15.86:0/747256642 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb9840a3ec0 0x7fb9840a6380 secure :-1 s=READY pgs=90 cs=0 l=1 rev1=1 crypto rx=0x7fb9900cdd60 tx=0x7fb990004020 comp rx=0 tx=0).stop 2024-04-21T08:39:23.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.892+0000 7fb99f515640 1 -- 172.21.15.86:0/747256642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb99810ebf0 msgr2=0x7fb99819a7c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:23.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.892+0000 7fb99f515640 1 --2- 172.21.15.86:0/747256642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb99810ebf0 0x7fb99819a7c0 secure :-1 s=READY pgs=671 cs=0 l=1 rev1=1 crypto rx=0x7fb990009fd0 tx=0x7fb99000b260 comp rx=0 tx=0).stop 2024-04-21T08:39:23.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.892+0000 7fb99f515640 1 -- 172.21.15.86:0/747256642 shutdown_connections 2024-04-21T08:39:23.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.892+0000 7fb99f515640 1 --2- 172.21.15.86:0/747256642 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb9840a3ec0 0x7fb9840a6380 unknown :-1 s=CLOSED pgs=90 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.892+0000 7fb99f515640 1 --2- 172.21.15.86:0/747256642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb99810ebf0 0x7fb99819a7c0 unknown :-1 s=CLOSED pgs=671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.892+0000 7fb99f515640 1 --2- 172.21.15.86:0/747256642 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb998103300 0x7fb9981a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.892+0000 7fb99f515640 1 --2- 172.21.15.86:0/747256642 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb998102e90 0x7fb9981a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:23.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.892+0000 7fb99f515640 1 -- 172.21.15.86:0/747256642 >> 172.21.15.86:0/747256642 conn(0x7fb998077410 msgr2=0x7fb9981101e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:23.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.892+0000 7fb99f515640 1 -- 172.21.15.86:0/747256642 shutdown_connections 2024-04-21T08:39:23.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:23.892+0000 7fb99f515640 1 -- 172.21.15.86:0/747256642 wait complete. 2024-04-21T08:39:23.902 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:39:24.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.006+0000 7f1dfe3b2640 1 -- 172.21.15.86:0/646023264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1df8106160 msgr2=0x7f1df810dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:24.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.006+0000 7f1dfe3b2640 1 --2- 172.21.15.86:0/646023264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1df8106160 0x7f1df810dd30 secure :-1 s=READY pgs=675 cs=0 l=1 rev1=1 crypto rx=0x7f1df0009ca0 tx=0x7f1df002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:39:24.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.006+0000 7f1dfe3b2640 1 -- 172.21.15.86:0/646023264 shutdown_connections 2024-04-21T08:39:24.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.006+0000 7f1dfe3b2640 1 --2- 172.21.15.86:0/646023264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1df8106160 0x7f1df810dd30 unknown :-1 s=CLOSED pgs=675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:24.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.006+0000 7f1dfe3b2640 1 --2- 172.21.15.86:0/646023264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1df8101a40 0x7f1df8105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:24.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.006+0000 7f1dfe3b2640 1 --2- 172.21.15.86:0/646023264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1df8101090 0x7f1df8101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:24.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.006+0000 7f1dfe3b2640 1 -- 172.21.15.86:0/646023264 >> 172.21.15.86:0/646023264 conn(0x7f1df8077f50 msgr2=0x7f1df8078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:24.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.006+0000 7f1dfe3b2640 1 -- 172.21.15.86:0/646023264 shutdown_connections 2024-04-21T08:39:24.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.006+0000 7f1dfe3b2640 1 -- 172.21.15.86:0/646023264 wait complete. 2024-04-21T08:39:24.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.007+0000 7f1dfe3b2640 1 Processor -- start 2024-04-21T08:39:24.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.007+0000 7f1dfe3b2640 1 -- start start 2024-04-21T08:39:24.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.007+0000 7f1dfe3b2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1df8101090 0x7f1df81987d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:24.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.007+0000 7f1dfe3b2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1df8106160 0x7f1df8198d10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:24.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.007+0000 7f1dfe3b2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1df819bfe0 0x7f1df819c490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:24.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.007+0000 7f1dfe3b2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1df81110d0 con 0x7f1df8106160 2024-04-21T08:39:24.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.007+0000 7f1dfe3b2640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1df8110f50 con 0x7f1df819bfe0 2024-04-21T08:39:24.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.007+0000 7f1dfe3b2640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1df8111250 con 0x7f1df8101090 2024-04-21T08:39:24.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.008+0000 7f1df77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1df8106160 0x7f1df8198d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:24.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.008+0000 7f1df77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1df8106160 0x7f1df8198d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55692/0 (socket says 172.21.15.86:55692) 2024-04-21T08:39:24.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.008+0000 7f1df77fe640 1 -- 172.21.15.86:0/62269222 learned_addr learned my addr 172.21.15.86:0/62269222 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:24.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.008+0000 7f1dfc94f640 1 --2- 172.21.15.86:0/62269222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1df819bfe0 0x7f1df819c490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:24.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.008+0000 7f1df77fe640 1 -- 172.21.15.86:0/62269222 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1df8101090 msgr2=0x7f1df81987d0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T08:39:24.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.008+0000 7f1df7fff640 1 --2- 172.21.15.86:0/62269222 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1df8101090 0x7f1df81987d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:24.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.008+0000 7f1df77fe640 1 --2- 172.21.15.86:0/62269222 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1df8101090 0x7f1df81987d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:24.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.008+0000 7f1df77fe640 1 -- 172.21.15.86:0/62269222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1df819bfe0 msgr2=0x7f1df819c490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:24.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.008+0000 7f1df77fe640 1 --2- 172.21.15.86:0/62269222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1df819bfe0 0x7f1df819c490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:24.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.008+0000 7f1df77fe640 1 -- 172.21.15.86:0/62269222 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1df819cad0 con 0x7f1df8106160 2024-04-21T08:39:24.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.008+0000 7f1dfc94f640 1 --2- 172.21.15.86:0/62269222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1df819bfe0 0x7f1df819c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:24.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.008+0000 7f1df77fe640 1 --2- 172.21.15.86:0/62269222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1df8106160 0x7f1df8198d10 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f1dec00e9f0 tx=0x7f1dec00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:24.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.008+0000 7f1df57fa640 1 -- 172.21.15.86:0/62269222 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec00ccc0 con 0x7f1df8106160 2024-04-21T08:39:24.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.009+0000 7f1df57fa640 1 -- 172.21.15.86:0/62269222 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dec0054d0 con 0x7f1df8106160 2024-04-21T08:39:24.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.009+0000 7f1dfe3b2640 1 -- 172.21.15.86:0/62269222 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1df819cdc0 con 0x7f1df8106160 2024-04-21T08:39:24.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.009+0000 7f1df57fa640 1 -- 172.21.15.86:0/62269222 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec010430 con 0x7f1df8106160 2024-04-21T08:39:24.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.009+0000 7f1dfe3b2640 1 -- 172.21.15.86:0/62269222 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1df81a5660 con 0x7f1df8106160 2024-04-21T08:39:24.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.011+0000 7f1df57fa640 1 -- 172.21.15.86:0/62269222 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f1dec0046c0 con 0x7f1df8106160 2024-04-21T08:39:24.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.012+0000 7f1df57fa640 1 --2- 172.21.15.86:0/62269222 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1de00a3f40 0x7f1de00a6400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:24.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.012+0000 7f1df7fff640 1 --2- 172.21.15.86:0/62269222 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1de00a3f40 0x7f1de00a6400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:24.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.012+0000 7f1df57fa640 1 -- 172.21.15.86:0/62269222 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f1dec0d1220 con 0x7f1df8106160 2024-04-21T08:39:24.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.012+0000 7f1e00fa6740 1 Processor -- start 2024-04-21T08:39:24.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.012+0000 7f1e00fa6740 1 -- start start 2024-04-21T08:39:24.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.012+0000 7f1df7fff640 1 --2- 172.21.15.86:0/62269222 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1de00a3f40 0x7f1de00a6400 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1de8009aa0 tx=0x7f1de8005f90 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:24.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.012+0000 7f1e00fa6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c86a566d20 0x55c86a567100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:24.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.012+0000 7f1e00fa6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86a5676d0 0x55c86a553650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:24.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.012+0000 7f1e00fa6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c86a553c70 0x55c86a5540d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:24.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.012+0000 7f1e00fa6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c86a4be1f0 con 0x55c86a5676d0 2024-04-21T08:39:24.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.013+0000 7f1e00fa6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c86a45b4b0 con 0x55c86a566d20 2024-04-21T08:39:24.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.013+0000 7f1e00fa6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c86a4ba1b0 con 0x55c86a553c70 2024-04-21T08:39:24.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.013+0000 7f1df77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86a5676d0 0x55c86a553650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:24.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.013+0000 7f1df77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86a5676d0 0x55c86a553650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55704/0 (socket says 172.21.15.86:55704) 2024-04-21T08:39:24.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.013+0000 7f1df77fe640 1 -- 172.21.15.86:0/2778239061 learned_addr learned my addr 172.21.15.86:0/2778239061 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:24.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.013+0000 7f1df7fff640 1 --2- 172.21.15.86:0/2778239061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c86a566d20 0x55c86a567100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:24.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.013+0000 7f1df77fe640 1 -- 172.21.15.86:0/2778239061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c86a553c70 msgr2=0x55c86a5540d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:24.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.013+0000 7f1dfc94f640 1 --2- 172.21.15.86:0/2778239061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c86a553c70 0x55c86a5540d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:24.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.013+0000 7f1df77fe640 1 --2- 172.21.15.86:0/2778239061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c86a553c70 0x55c86a5540d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:24.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.013+0000 7f1df77fe640 1 -- 172.21.15.86:0/2778239061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c86a566d20 msgr2=0x55c86a567100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:24.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.013+0000 7f1df77fe640 1 --2- 172.21.15.86:0/2778239061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c86a566d20 0x55c86a567100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:24.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.013+0000 7f1df77fe640 1 -- 172.21.15.86:0/2778239061 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c86a44d1b0 con 0x55c86a5676d0 2024-04-21T08:39:24.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.013+0000 7f1df7fff640 1 --2- 172.21.15.86:0/2778239061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c86a566d20 0x55c86a567100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:24.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.013+0000 7f1dfc94f640 1 --2- 172.21.15.86:0/2778239061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c86a553c70 0x55c86a5540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:24.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.013+0000 7f1df77fe640 1 --2- 172.21.15.86:0/2778239061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86a5676d0 0x55c86a553650 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f1dec09c8c0 tx=0x7f1dec005680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8fb28e89688fef30 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:24.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.013+0000 7f1dde7fc640 1 -- 172.21.15.86:0/2778239061 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec0b6070 con 0x55c86a5676d0 2024-04-21T08:39:24.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.013+0000 7f1dde7fc640 1 -- 172.21.15.86:0/2778239061 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dec0a0f70 con 0x55c86a5676d0 2024-04-21T08:39:24.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.013+0000 7f1dde7fc640 1 -- 172.21.15.86:0/2778239061 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec0a53a0 con 0x55c86a5676d0 2024-04-21T08:39:24.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.014+0000 7f1e00fa6740 1 -- 172.21.15.86:0/2778239061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86a5676d0 msgr2=0x55c86a553650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:24.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.014+0000 7f1e00fa6740 1 --2- 172.21.15.86:0/2778239061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86a5676d0 0x55c86a553650 secure :-1 s=READY pgs=677 cs=0 l=1 rev1=1 crypto rx=0x7f1dec09c8c0 tx=0x7f1dec005680 comp rx=0 tx=0).stop 2024-04-21T08:39:24.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.014+0000 7f1e00fa6740 1 -- 172.21.15.86:0/2778239061 shutdown_connections 2024-04-21T08:39:24.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.014+0000 7f1e00fa6740 1 --2- 172.21.15.86:0/2778239061 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c86a553c70 0x55c86a5540d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:24.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.014+0000 7f1e00fa6740 1 --2- 172.21.15.86:0/2778239061 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86a5676d0 0x55c86a553650 unknown :-1 s=CLOSED pgs=677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:24.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.014+0000 7f1e00fa6740 1 --2- 172.21.15.86:0/2778239061 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c86a566d20 0x55c86a567100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:24.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.014+0000 7f1e00fa6740 1 -- 172.21.15.86:0/2778239061 >> 172.21.15.86:0/2778239061 conn(0x55c86a556110 msgr2=0x55c86a5564f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:24.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.015+0000 7f1e00fa6740 1 -- 172.21.15.86:0/2778239061 shutdown_connections 2024-04-21T08:39:24.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.015+0000 7f1e00fa6740 1 -- 172.21.15.86:0/2778239061 wait complete. 2024-04-21T08:39:24.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.015+0000 7f1e00fa6740 1 -- start start 2024-04-21T08:39:24.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.015+0000 7f1e00fa6740 1 Processor -- start 2024-04-21T08:39:24.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.016+0000 7f1e00fa6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c86a553c70 0x55c86a56d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:24.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.016+0000 7f1e00fa6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c86a566d20 0x55c86a56a8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:24.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.016+0000 7f1e00fa6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86a5676d0 0x55c86a56ae10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:24.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.016+0000 7f1e00fa6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c86a56e160 con 0x55c86a5676d0 2024-04-21T08:39:24.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.016+0000 7f1e00fa6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c86a477500 con 0x55c86a566d20 2024-04-21T08:39:24.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.016+0000 7f1e00fa6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c86a56e2e0 con 0x55c86a553c70 2024-04-21T08:39:24.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.016+0000 7f1e00fa6740 10 client.0 _ll_register_callbacks cb 0x55c86a445b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:24.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.016+0000 7f1df77fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c86a566d20 0x55c86a56a8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:24.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.016+0000 7f1df77fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c86a566d20 0x55c86a56a8d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.86:59646/0 (socket says 172.21.15.86:59646) 2024-04-21T08:39:24.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.016+0000 7f1df77fe640 1 -- 172.21.15.86:0/3291006499 learned_addr learned my addr 172.21.15.86:0/3291006499 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:24.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.016+0000 7f1df7fff640 1 --2- 172.21.15.86:0/3291006499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c86a553c70 0x55c86a56d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:24.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.016+0000 7f1dfc94f640 1 --2- 172.21.15.86:0/3291006499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86a5676d0 0x55c86a56ae10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:24.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.016+0000 7f1df77fe640 1 -- 172.21.15.86:0/3291006499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c86a553c70 msgr2=0x55c86a56d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:24.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.016+0000 7f1df77fe640 1 --2- 172.21.15.86:0/3291006499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c86a553c70 0x55c86a56d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:24.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.016+0000 7f1df77fe640 1 -- 172.21.15.86:0/3291006499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86a5676d0 msgr2=0x55c86a56ae10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:24.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.016+0000 7f1df77fe640 1 --2- 172.21.15.86:0/3291006499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86a5676d0 0x55c86a56ae10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:24.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.016+0000 7f1df77fe640 1 -- 172.21.15.86:0/3291006499 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c86a1d03f0 con 0x55c86a566d20 2024-04-21T08:39:24.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.016+0000 7f1dfc94f640 1 --2- 172.21.15.86:0/3291006499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86a5676d0 0x55c86a56ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:24.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.017+0000 7f1df77fe640 1 --2- 172.21.15.86:0/3291006499 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c86a566d20 0x55c86a56a8d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1dec09cb50 tx=0x7f1dec017000 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b3a8314cffdb6cbb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:24.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.017+0000 7f1dd77fe640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-04-21T08:39:24.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.017+0000 7f1dd77fe640 1 -- 172.21.15.86:0/3291006499 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1dec0b6040 con 0x55c86a566d20 2024-04-21T08:39:24.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.017+0000 7f1dd77fe640 1 -- 172.21.15.86:0/3291006499 <== mon.2 v2:172.21.15.88:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f1dec0ce410 con 0x55c86a566d20 2024-04-21T08:39:24.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.017+0000 7f1e00fa6740 1 -- 172.21.15.86:0/3291006499 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1dec0a0f70 con 0x55c86a566d20 2024-04-21T08:39:24.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.017+0000 7f1dd77fe640 1 -- 172.21.15.86:0/3291006499 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1dec0a5830 con 0x55c86a566d20 2024-04-21T08:39:24.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.017+0000 7f1e00fa6740 10 client.25478 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:24.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.017+0000 7f1e00fa6740 1 -- 172.21.15.86:0/3291006499 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c86a29a7d0 con 0x55c86a566d20 2024-04-21T08:39:24.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.017+0000 7f1dd77fe640 1 -- 172.21.15.86:0/3291006499 <== mon.2 v2:172.21.15.88:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1dec0a0f70 con 0x55c86a566d20 2024-04-21T08:39:24.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.017+0000 7f1dd77fe640 1 -- 172.21.15.86:0/3291006499 <== mon.2 v2:172.21.15.88:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1dec005070 con 0x55c86a566d20 2024-04-21T08:39:24.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.017+0000 7f1e00fa6740 10 client.25478 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:24.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.018+0000 7f1e00fa6740 10 client.25478 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:24.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.018+0000 7f1e00fa6740 20 client.25478 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:24.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.018+0000 7f1e00fa6740 1 --2- 172.21.15.86:0/3291006499 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c86a57b180 0x55c86a57d570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:24.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.018+0000 7f1e00fa6740 4 client.25478 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:24.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.018+0000 7f1e00fa6740 1 -- 172.21.15.86:0/3291006499 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c86a45d930 con 0x55c86a57b180 2024-04-21T08:39:24.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.018+0000 7f1df7fff640 1 --2- 172.21.15.86:0/3291006499 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c86a57b180 0x55c86a57d570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:24.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.018+0000 7f1df7fff640 1 --2- 172.21.15.86:0/3291006499 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c86a57b180 0x55c86a57d570 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7cfc452bb8534b2b server_cookie=1a85d3b4a6052e2e in_seq=0 out_seq=0 2024-04-21T08:39:24.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.018+0000 7f1dd77fe640 10 client.25478 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:24.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.019+0000 7f1dd77fe640 1 -- 172.21.15.86:0/3291006499 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c86a45d930 con 0x55c86a57b180 2024-04-21T08:39:24.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.019+0000 7f1dd77fe640 10 client.25478 handle_command_reply: tid=0 2024-04-21T08:39:24.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.019+0000 7f1e00fa6740 1 client.25478 shutdown 2024-04-21T08:39:24.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.019+0000 7f1e00fa6740 1 -- 172.21.15.86:0/3291006499 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c86a566d20 msgr2=0x55c86a56a8d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:24.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.019+0000 7f1e00fa6740 1 --2- 172.21.15.86:0/3291006499 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c86a566d20 0x55c86a56a8d0 secure :-1 s=READY pgs=65 cs=0 l=1 rev1=1 crypto rx=0x7f1dec09cb50 tx=0x7f1dec017000 comp rx=0 tx=0).stop 2024-04-21T08:39:24.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.019+0000 7f1e00fa6740 1 -- 172.21.15.86:0/3291006499 shutdown_connections 2024-04-21T08:39:24.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.019+0000 7f1e00fa6740 1 --2- 172.21.15.86:0/3291006499 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c86a57b180 0x55c86a57d570 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:24.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.019+0000 7f1dd77fe640 0 client.25478 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:24.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.019+0000 7f1e00fa6740 1 --2- 172.21.15.86:0/3291006499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86a5676d0 0x55c86a56ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:24.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.019+0000 7f1e00fa6740 1 --2- 172.21.15.86:0/3291006499 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c86a566d20 0x55c86a56a8d0 unknown :-1 s=CLOSED pgs=65 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:24.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.019+0000 7f1e00fa6740 1 --2- 172.21.15.86:0/3291006499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c86a553c70 0x55c86a56d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:24.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.019+0000 7f1e00fa6740 1 -- 172.21.15.86:0/3291006499 >> 172.21.15.86:0/3291006499 conn(0x55c86a556110 msgr2=0x55c86a555870 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:24.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.019+0000 7f1e00fa6740 1 -- 172.21.15.86:0/3291006499 shutdown_connections 2024-04-21T08:39:24.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.019+0000 7f1e00fa6740 1 -- 172.21.15.86:0/3291006499 wait complete. 2024-04-21T08:39:24.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.020+0000 7f1e00fa6740 20 client.25478 trim_cache size 0 max 16384 2024-04-21T08:39:24.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.045+0000 7f1e00fa6740 1 Processor -- start 2024-04-21T08:39:24.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.045+0000 7f1e00fa6740 1 -- start start 2024-04-21T08:39:24.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.045+0000 7f1e00fa6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c86a556110 0x55c86a564e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:24.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.045+0000 7f1e00fa6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c86a553c70 0x55c86a56d000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:24.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.045+0000 7f1e00fa6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86a566d20 0x55c86a580590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:24.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.045+0000 7f1e00fa6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1df8073160 con 0x55c86a566d20 2024-04-21T08:39:24.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.046+0000 7f1e00fa6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1df81a64a0 con 0x55c86a553c70 2024-04-21T08:39:24.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.046+0000 7f1e00fa6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1df8073830 con 0x55c86a556110 2024-04-21T08:39:24.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.046+0000 7f1dfc94f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86a566d20 0x55c86a580590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:24.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.046+0000 7f1dfc94f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86a566d20 0x55c86a580590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55722/0 (socket says 172.21.15.86:55722) 2024-04-21T08:39:24.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.046+0000 7f1dfc94f640 1 -- 172.21.15.86:0/3626591923 learned_addr learned my addr 172.21.15.86:0/3626591923 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:24.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.046+0000 7f1df77fe640 1 --2- 172.21.15.86:0/3626591923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c86a553c70 0x55c86a56d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:24.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.046+0000 7f1df7fff640 1 --2- 172.21.15.86:0/3626591923 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c86a556110 0x55c86a564e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:24.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.046+0000 7f1dfc94f640 1 -- 172.21.15.86:0/3626591923 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c86a556110 msgr2=0x55c86a564e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:24.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.046+0000 7f1dfc94f640 1 --2- 172.21.15.86:0/3626591923 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c86a556110 0x55c86a564e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:24.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.046+0000 7f1dfc94f640 1 -- 172.21.15.86:0/3626591923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c86a553c70 msgr2=0x55c86a56d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:24.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.046+0000 7f1dfc94f640 1 --2- 172.21.15.86:0/3626591923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c86a553c70 0x55c86a56d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:24.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.046+0000 7f1dfc94f640 1 -- 172.21.15.86:0/3626591923 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c86a567470 con 0x55c86a566d20 2024-04-21T08:39:24.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.046+0000 7f1dfc94f640 1 --2- 172.21.15.86:0/3626591923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86a566d20 0x55c86a580590 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f1df0036000 tx=0x7f1df0005ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=679f62ee83b456fa server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:24.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.046+0000 7f1dddffb640 1 -- 172.21.15.86:0/3626591923 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df0044030 con 0x55c86a566d20 2024-04-21T08:39:24.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.047+0000 7f1dddffb640 1 -- 172.21.15.86:0/3626591923 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1df0032970 con 0x55c86a566d20 2024-04-21T08:39:24.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.047+0000 7f1dddffb640 1 -- 172.21.15.86:0/3626591923 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1df0039040 con 0x55c86a566d20 2024-04-21T08:39:24.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.047+0000 7f1e00fa6740 1 -- 172.21.15.86:0/3626591923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86a566d20 msgr2=0x55c86a580590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:24.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.047+0000 7f1e00fa6740 1 --2- 172.21.15.86:0/3626591923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86a566d20 0x55c86a580590 secure :-1 s=READY pgs=678 cs=0 l=1 rev1=1 crypto rx=0x7f1df0036000 tx=0x7f1df0005ef0 comp rx=0 tx=0).stop 2024-04-21T08:39:24.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.047+0000 7f1e00fa6740 1 -- 172.21.15.86:0/3626591923 shutdown_connections 2024-04-21T08:39:24.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.047+0000 7f1e00fa6740 1 --2- 172.21.15.86:0/3626591923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86a566d20 0x55c86a580590 unknown :-1 s=CLOSED pgs=678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:24.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.047+0000 7f1e00fa6740 1 --2- 172.21.15.86:0/3626591923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c86a553c70 0x55c86a56d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:24.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.047+0000 7f1e00fa6740 1 --2- 172.21.15.86:0/3626591923 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c86a556110 0x55c86a564e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:24.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.047+0000 7f1e00fa6740 1 -- 172.21.15.86:0/3626591923 >> 172.21.15.86:0/3626591923 conn(0x55c86a56af70 msgr2=0x55c86a564750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:24.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.047+0000 7f1e00fa6740 1 -- 172.21.15.86:0/3626591923 shutdown_connections 2024-04-21T08:39:24.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.047+0000 7f1e00fa6740 1 -- 172.21.15.86:0/3626591923 wait complete. 2024-04-21T08:39:24.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.048+0000 7f1e00fa6740 1 -- start start 2024-04-21T08:39:24.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.048+0000 7f1e00fa6740 1 Processor -- start 2024-04-21T08:39:24.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.048+0000 7f1e00fa6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86a553c70 0x55c86a580780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:24.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.048+0000 7f1e00fa6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c86a556110 0x55c86a57db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:24.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.048+0000 7f1e00fa6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c86a566d20 0x55c86a57e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:24.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.048+0000 7f1e00fa6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1df81a5dd0 con 0x55c86a553c70 2024-04-21T08:39:24.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.048+0000 7f1e00fa6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1df8073f00 con 0x55c86a556110 2024-04-21T08:39:24.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.048+0000 7f1e00fa6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1df8071120 con 0x55c86a566d20 2024-04-21T08:39:24.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.048+0000 7f1e00fa6740 10 client.0 _ll_register_callbacks cb 0x7f1dec0b61e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:24.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.048+0000 7f1df7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86a553c70 0x55c86a580780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:24.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.048+0000 7f1df7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86a553c70 0x55c86a580780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55732/0 (socket says 172.21.15.86:55732) 2024-04-21T08:39:24.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.048+0000 7f1df7fff640 1 -- 172.21.15.86:0/2012427315 learned_addr learned my addr 172.21.15.86:0/2012427315 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:24.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.049+0000 7f1df77fe640 1 --2- 172.21.15.86:0/2012427315 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c86a556110 0x55c86a57db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:24.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.049+0000 7f1dfc94f640 1 --2- 172.21.15.86:0/2012427315 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c86a566d20 0x55c86a57e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:24.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.049+0000 7f1df7fff640 1 -- 172.21.15.86:0/2012427315 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c86a566d20 msgr2=0x55c86a57e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:24.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.049+0000 7f1df7fff640 1 --2- 172.21.15.86:0/2012427315 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c86a566d20 0x55c86a57e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:24.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.049+0000 7f1df7fff640 1 -- 172.21.15.86:0/2012427315 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c86a556110 msgr2=0x55c86a57db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:24.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.049+0000 7f1df7fff640 1 --2- 172.21.15.86:0/2012427315 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c86a556110 0x55c86a57db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:24.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.049+0000 7f1df7fff640 1 -- 172.21.15.86:0/2012427315 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c86a57e690 con 0x55c86a553c70 2024-04-21T08:39:24.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.049+0000 7f1df7fff640 1 --2- 172.21.15.86:0/2012427315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86a553c70 0x55c86a580780 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x55c86a5622f0 tx=0x7f1de8004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c74a526da3ad3910 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:24.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.049+0000 7f1dd77fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:24.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.049+0000 7f1dd77fe640 1 -- 172.21.15.86:0/2012427315 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1de8005020 con 0x55c86a553c70 2024-04-21T08:39:24.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.049+0000 7f1dd77fe640 1 -- 172.21.15.86:0/2012427315 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f1de80051c0 con 0x55c86a553c70 2024-04-21T08:39:24.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.049+0000 7f1e00fa6740 1 -- 172.21.15.86:0/2012427315 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1df0032970 con 0x55c86a553c70 2024-04-21T08:39:24.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.049+0000 7f1dd77fe640 1 -- 172.21.15.86:0/2012427315 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1de804a050 con 0x55c86a553c70 2024-04-21T08:39:24.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.049+0000 7f1e00fa6740 10 client.16191 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:24.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.049+0000 7f1e00fa6740 1 -- 172.21.15.86:0/2012427315 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c86a5e11d0 con 0x55c86a553c70 2024-04-21T08:39:24.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.050+0000 7f1dd77fe640 1 -- 172.21.15.86:0/2012427315 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1df0032970 con 0x55c86a553c70 2024-04-21T08:39:24.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.050+0000 7f1dd77fe640 1 -- 172.21.15.86:0/2012427315 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1de800f040 con 0x55c86a553c70 2024-04-21T08:39:24.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.050+0000 7f1e00fa6740 10 client.16191 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:24.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.050+0000 7f1e00fa6740 10 client.16191 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:24.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.050+0000 7f1e00fa6740 20 client.16191 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:24.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.050+0000 7f1e00fa6740 1 --2- 172.21.15.86:0/2012427315 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c86a5676d0 0x55c86a5e3a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:24.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.050+0000 7f1e00fa6740 4 client.16191 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:39:24.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.050+0000 7f1e00fa6740 1 -- 172.21.15.86:0/2012427315 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c86a4cbc10 con 0x55c86a5676d0 2024-04-21T08:39:24.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.050+0000 7f1df77fe640 1 --2- 172.21.15.86:0/2012427315 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c86a5676d0 0x55c86a5e3a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:24.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.051+0000 7f1dd77fe640 10 client.16191 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:24.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.051+0000 7f1df77fe640 1 --2- 172.21.15.86:0/2012427315 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c86a5676d0 0x55c86a5e3a10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4c017a0cb9e301ac server_cookie=a07754c272e077dd in_seq=0 out_seq=0 2024-04-21T08:39:24.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.051+0000 7f1dd77fe640 1 -- 172.21.15.86:0/2012427315 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c86a4cbc10 con 0x55c86a5676d0 2024-04-21T08:39:24.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.051+0000 7f1dd77fe640 10 client.16191 handle_command_reply: tid=0 2024-04-21T08:39:24.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.051+0000 7f1e00fa6740 1 client.16191 shutdown 2024-04-21T08:39:24.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.051+0000 7f1e00fa6740 1 -- 172.21.15.86:0/2012427315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86a553c70 msgr2=0x55c86a580780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:24.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.051+0000 7f1e00fa6740 1 --2- 172.21.15.86:0/2012427315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86a553c70 0x55c86a580780 secure :-1 s=READY pgs=679 cs=0 l=1 rev1=1 crypto rx=0x55c86a5622f0 tx=0x7f1de8004730 comp rx=0 tx=0).stop 2024-04-21T08:39:24.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.051+0000 7f1e00fa6740 1 -- 172.21.15.86:0/2012427315 shutdown_connections 2024-04-21T08:39:24.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.051+0000 7f1e00fa6740 1 --2- 172.21.15.86:0/2012427315 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c86a5676d0 0x55c86a5e3a10 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:24.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.051+0000 7f1e00fa6740 1 --2- 172.21.15.86:0/2012427315 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c86a566d20 0x55c86a57e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:24.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.051+0000 7f1e00fa6740 1 --2- 172.21.15.86:0/2012427315 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c86a556110 0x55c86a57db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:24.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.051+0000 7f1dd77fe640 0 client.16191 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:24.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.051+0000 7f1e00fa6740 1 --2- 172.21.15.86:0/2012427315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c86a553c70 0x55c86a580780 unknown :-1 s=CLOSED pgs=679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:24.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.051+0000 7f1e00fa6740 1 -- 172.21.15.86:0/2012427315 >> 172.21.15.86:0/2012427315 conn(0x55c86a56af70 msgr2=0x55c86a576e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:24.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.051+0000 7f1e00fa6740 1 -- 172.21.15.86:0/2012427315 shutdown_connections 2024-04-21T08:39:24.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.051+0000 7f1e00fa6740 1 -- 172.21.15.86:0/2012427315 wait complete. 2024-04-21T08:39:24.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.051+0000 7f1e00fa6740 20 client.16191 trim_cache size 0 max 16384 2024-04-21T08:39:24.055 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:39:24.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.052+0000 7f1dfe3b2640 1 -- 172.21.15.86:0/62269222 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1de00a3f40 msgr2=0x7f1de00a6400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:24.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.052+0000 7f1dfe3b2640 1 --2- 172.21.15.86:0/62269222 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1de00a3f40 0x7f1de00a6400 secure :-1 s=READY pgs=91 cs=0 l=1 rev1=1 crypto rx=0x7f1de8009aa0 tx=0x7f1de8005f90 comp rx=0 tx=0).stop 2024-04-21T08:39:24.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.052+0000 7f1dfe3b2640 1 -- 172.21.15.86:0/62269222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1df8106160 msgr2=0x7f1df8198d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:24.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.052+0000 7f1dfe3b2640 1 --2- 172.21.15.86:0/62269222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1df8106160 0x7f1df8198d10 secure :-1 s=READY pgs=676 cs=0 l=1 rev1=1 crypto rx=0x7f1dec00e9f0 tx=0x7f1dec00eeb0 comp rx=0 tx=0).stop 2024-04-21T08:39:24.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.052+0000 7f1dfe3b2640 1 -- 172.21.15.86:0/62269222 shutdown_connections 2024-04-21T08:39:24.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.052+0000 7f1dfe3b2640 1 --2- 172.21.15.86:0/62269222 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1de00a3f40 0x7f1de00a6400 unknown :-1 s=CLOSED pgs=91 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:24.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.052+0000 7f1dfe3b2640 1 --2- 172.21.15.86:0/62269222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1df819bfe0 0x7f1df819c490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:24.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.052+0000 7f1dfe3b2640 1 --2- 172.21.15.86:0/62269222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1df8106160 0x7f1df8198d10 unknown :-1 s=CLOSED pgs=676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:24.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.052+0000 7f1dfe3b2640 1 --2- 172.21.15.86:0/62269222 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1df8101090 0x7f1df81987d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:24.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.052+0000 7f1dfe3b2640 1 -- 172.21.15.86:0/62269222 >> 172.21.15.86:0/62269222 conn(0x7f1df8077f50 msgr2=0x7f1df8069420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:24.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.052+0000 7f1dfe3b2640 1 -- 172.21.15.86:0/62269222 shutdown_connections 2024-04-21T08:39:24.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:24.052+0000 7f1dfe3b2640 1 -- 172.21.15.86:0/62269222 wait complete. 2024-04-21T08:39:24.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:24 smithi086 ceph-mon[27939]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-21T08:39:24.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:24 smithi088 ceph-mon[31259]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-21T08:39:24.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:24 smithi179 ceph-mon[31829]: pgmap v21: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.4 KiB/s rd, 255 B/s wr, 12 op/s 2024-04-21T08:39:25.063 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:39:25.064 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:39:25.064 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:39:25.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.188+0000 7f603c243640 1 -- 172.21.15.86:0/3346046255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6034107260 msgr2=0x7f603410ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:25.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.188+0000 7f603c243640 1 --2- 172.21.15.86:0/3346046255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6034107260 0x7f603410ee30 secure :-1 s=READY pgs=680 cs=0 l=1 rev1=1 crypto rx=0x7f602c009ca0 tx=0x7f602c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.188+0000 7f603c243640 1 -- 172.21.15.86:0/3346046255 shutdown_connections 2024-04-21T08:39:25.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.188+0000 7f603c243640 1 --2- 172.21.15.86:0/3346046255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6034107260 0x7f603410ee30 unknown :-1 s=CLOSED pgs=680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.188+0000 7f603c243640 1 --2- 172.21.15.86:0/3346046255 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f603406b1c0 0x7f6034106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.188+0000 7f603c243640 1 --2- 172.21.15.86:0/3346046255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f603406a810 0x7f603406abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.188+0000 7f603c243640 1 -- 172.21.15.86:0/3346046255 >> 172.21.15.86:0/3346046255 conn(0x7f6034077fb0 msgr2=0x7f60340783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:25.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.188+0000 7f603c243640 1 -- 172.21.15.86:0/3346046255 shutdown_connections 2024-04-21T08:39:25.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.188+0000 7f603c243640 1 -- 172.21.15.86:0/3346046255 wait complete. 2024-04-21T08:39:25.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.189+0000 7f603c243640 1 Processor -- start 2024-04-21T08:39:25.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.189+0000 7f603c243640 1 -- start start 2024-04-21T08:39:25.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.189+0000 7f603c243640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f603406a810 0x7f6034198530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:25.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.189+0000 7f603c243640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6034107260 0x7f6034198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:25.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.189+0000 7f603c243640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f603419cdf0 0x7f603419d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:25.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.189+0000 7f603c243640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60341121d0 con 0x7f603406a810 2024-04-21T08:39:25.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.189+0000 7f6039fdf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f603406a810 0x7f6034198530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:25.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.189+0000 7f6039fdf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f603406a810 0x7f6034198530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55748/0 (socket says 172.21.15.86:55748) 2024-04-21T08:39:25.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.189+0000 7f603c243640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6034112050 con 0x7f6034107260 2024-04-21T08:39:25.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.189+0000 7f603c243640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6034112350 con 0x7f603419cdf0 2024-04-21T08:39:25.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.189+0000 7f6039fdf640 1 -- 172.21.15.86:0/2790702371 learned_addr learned my addr 172.21.15.86:0/2790702371 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:25.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.190+0000 7f60397de640 1 --2- 172.21.15.86:0/2790702371 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6034107260 0x7f6034198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:25.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.190+0000 7f603a7e0640 1 --2- 172.21.15.86:0/2790702371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f603419cdf0 0x7f603419d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:25.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.190+0000 7f6039fdf640 1 -- 172.21.15.86:0/2790702371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f603419cdf0 msgr2=0x7f603419d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:25.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.190+0000 7f6039fdf640 1 --2- 172.21.15.86:0/2790702371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f603419cdf0 0x7f603419d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.190+0000 7f6039fdf640 1 -- 172.21.15.86:0/2790702371 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6034107260 msgr2=0x7f6034198a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:25.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.190+0000 7f6039fdf640 1 --2- 172.21.15.86:0/2790702371 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6034107260 0x7f6034198a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.190+0000 7f6039fdf640 1 -- 172.21.15.86:0/2790702371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f603419d8e0 con 0x7f603406a810 2024-04-21T08:39:25.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.190+0000 7f603a7e0640 1 --2- 172.21.15.86:0/2790702371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f603419cdf0 0x7f603419d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:25.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.190+0000 7f6039fdf640 1 --2- 172.21.15.86:0/2790702371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f603406a810 0x7f6034198530 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f603000d770 tx=0x7f603000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:25.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.190+0000 7f602affd640 1 -- 172.21.15.86:0/2790702371 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030017070 con 0x7f603406a810 2024-04-21T08:39:25.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.190+0000 7f602affd640 1 -- 172.21.15.86:0/2790702371 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60300054d0 con 0x7f603406a810 2024-04-21T08:39:25.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.190+0000 7f603c243640 1 -- 172.21.15.86:0/2790702371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f603419dbd0 con 0x7f603406a810 2024-04-21T08:39:25.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.190+0000 7f602affd640 1 -- 172.21.15.86:0/2790702371 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030004c30 con 0x7f603406a810 2024-04-21T08:39:25.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.190+0000 7f603c243640 1 -- 172.21.15.86:0/2790702371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60341a53c0 con 0x7f603406a810 2024-04-21T08:39:25.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.193+0000 7f602affd640 1 -- 172.21.15.86:0/2790702371 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f6030004e10 con 0x7f603406a810 2024-04-21T08:39:25.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.193+0000 7f602affd640 1 --2- 172.21.15.86:0/2790702371 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f60200a3ec0 0x7f60200a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:25.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.194+0000 7f602affd640 1 -- 172.21.15.86:0/2790702371 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f60300d0870 con 0x7f603406a810 2024-04-21T08:39:25.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.194+0000 7f60397de640 1 --2- 172.21.15.86:0/2790702371 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f60200a3ec0 0x7f60200a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:25.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.194+0000 7f603ee35740 1 Processor -- start 2024-04-21T08:39:25.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.194+0000 7f60397de640 1 --2- 172.21.15.86:0/2790702371 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f60200a3ec0 0x7f60200a6380 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f60340787a0 tx=0x7f602400b390 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:25.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.194+0000 7f603ee35740 1 -- start start 2024-04-21T08:39:25.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.194+0000 7f603ee35740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d0c85688e0 0x55d0c8568cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:25.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.194+0000 7f603ee35740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0c8569290 0x55d0c8571840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:25.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.194+0000 7f603ee35740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0c8571e10 0x55d0c8574260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:25.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.194+0000 7f603ee35740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d0c84ccf00 con 0x55d0c8569290 2024-04-21T08:39:25.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.194+0000 7f603ee35740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d0c846a590 con 0x55d0c8571e10 2024-04-21T08:39:25.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.194+0000 7f603ee35740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d0c84c8ec0 con 0x55d0c85688e0 2024-04-21T08:39:25.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.195+0000 7f60397de640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0c8569290 0x55d0c8571840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:25.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.195+0000 7f60397de640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0c8569290 0x55d0c8571840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55754/0 (socket says 172.21.15.86:55754) 2024-04-21T08:39:25.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.195+0000 7f60397de640 1 -- 172.21.15.86:0/120224057 learned_addr learned my addr 172.21.15.86:0/120224057 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:25.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.195+0000 7f603a7e0640 1 --2- 172.21.15.86:0/120224057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0c8571e10 0x55d0c8574260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:25.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.195+0000 7f6039fdf640 1 --2- 172.21.15.86:0/120224057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d0c85688e0 0x55d0c8568cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:25.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.195+0000 7f603a7e0640 1 -- 172.21.15.86:0/120224057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d0c85688e0 msgr2=0x55d0c8568cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:25.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.195+0000 7f603a7e0640 1 --2- 172.21.15.86:0/120224057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d0c85688e0 0x55d0c8568cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.195+0000 7f603a7e0640 1 -- 172.21.15.86:0/120224057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0c8569290 msgr2=0x55d0c8571840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:25.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.195+0000 7f603a7e0640 1 --2- 172.21.15.86:0/120224057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0c8569290 0x55d0c8571840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.195+0000 7f603a7e0640 1 -- 172.21.15.86:0/120224057 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0c82a77e0 con 0x55d0c8571e10 2024-04-21T08:39:25.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.195+0000 7f60397de640 1 --2- 172.21.15.86:0/120224057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0c8569290 0x55d0c8571840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:25.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.195+0000 7f603a7e0640 1 --2- 172.21.15.86:0/120224057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0c8571e10 0x55d0c8574260 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f602c037040 tx=0x7f602c009800 comp rx=0 tx=0).ready entity=mon.2 client_cookie=631aac02cbbcae1f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:25.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.196+0000 7f601ffff640 1 -- 172.21.15.86:0/120224057 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c039050 con 0x55d0c8571e10 2024-04-21T08:39:25.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.196+0000 7f601ffff640 1 -- 172.21.15.86:0/120224057 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f602c045070 con 0x55d0c8571e10 2024-04-21T08:39:25.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.196+0000 7f601ffff640 1 -- 172.21.15.86:0/120224057 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c0042a0 con 0x55d0c8571e10 2024-04-21T08:39:25.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.197+0000 7f603ee35740 1 -- 172.21.15.86:0/120224057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0c8571e10 msgr2=0x55d0c8574260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:25.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.197+0000 7f603ee35740 1 --2- 172.21.15.86:0/120224057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0c8571e10 0x55d0c8574260 secure :-1 s=READY pgs=66 cs=0 l=1 rev1=1 crypto rx=0x7f602c037040 tx=0x7f602c009800 comp rx=0 tx=0).stop 2024-04-21T08:39:25.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.197+0000 7f603ee35740 1 -- 172.21.15.86:0/120224057 shutdown_connections 2024-04-21T08:39:25.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.197+0000 7f603ee35740 1 --2- 172.21.15.86:0/120224057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0c8571e10 0x55d0c8574260 unknown :-1 s=CLOSED pgs=66 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.197+0000 7f603ee35740 1 --2- 172.21.15.86:0/120224057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0c8569290 0x55d0c8571840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.197+0000 7f603ee35740 1 --2- 172.21.15.86:0/120224057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d0c85688e0 0x55d0c8568cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.197+0000 7f603ee35740 1 -- 172.21.15.86:0/120224057 >> 172.21.15.86:0/120224057 conn(0x55d0c8562fb0 msgr2=0x55d0c8563390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:25.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.197+0000 7f603ee35740 1 -- 172.21.15.86:0/120224057 shutdown_connections 2024-04-21T08:39:25.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.197+0000 7f603ee35740 1 -- 172.21.15.86:0/120224057 wait complete. 2024-04-21T08:39:25.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.197+0000 7f603ee35740 1 -- start start 2024-04-21T08:39:25.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.197+0000 7f603ee35740 1 Processor -- start 2024-04-21T08:39:25.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.198+0000 7f603ee35740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d0c85688e0 0x55d0c856edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:25.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.198+0000 7f603ee35740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0c8569290 0x55d0c856f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:25.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.198+0000 7f603ee35740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0c8571e10 0x55d0c856f830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:25.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.198+0000 7f603ee35740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d0c8574d20 con 0x55d0c8571e10 2024-04-21T08:39:25.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.198+0000 7f603ee35740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d0c8486580 con 0x55d0c8569290 2024-04-21T08:39:25.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.198+0000 7f603ee35740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d0c8574ea0 con 0x55d0c85688e0 2024-04-21T08:39:25.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.198+0000 7f603ee35740 10 client.0 _ll_register_callbacks cb 0x55d0c848c650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:25.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.198+0000 7f603a7e0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0c8571e10 0x55d0c856f830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:25.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.198+0000 7f603a7e0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0c8571e10 0x55d0c856f830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55768/0 (socket says 172.21.15.86:55768) 2024-04-21T08:39:25.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.198+0000 7f603a7e0640 1 -- 172.21.15.86:0/3380716709 learned_addr learned my addr 172.21.15.86:0/3380716709 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:25.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.198+0000 7f6039fdf640 1 --2- 172.21.15.86:0/3380716709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d0c85688e0 0x55d0c856edb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:25.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.198+0000 7f603a7e0640 1 -- 172.21.15.86:0/3380716709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d0c85688e0 msgr2=0x55d0c856edb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:25.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.198+0000 7f60397de640 1 --2- 172.21.15.86:0/3380716709 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0c8569290 0x55d0c856f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:25.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.198+0000 7f603a7e0640 1 --2- 172.21.15.86:0/3380716709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d0c85688e0 0x55d0c856edb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.198+0000 7f603a7e0640 1 -- 172.21.15.86:0/3380716709 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0c8569290 msgr2=0x55d0c856f2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:25.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.198+0000 7f603a7e0640 1 --2- 172.21.15.86:0/3380716709 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0c8569290 0x55d0c856f2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.198+0000 7f603a7e0640 1 -- 172.21.15.86:0/3380716709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0c8307f70 con 0x55d0c8571e10 2024-04-21T08:39:25.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.198+0000 7f6039fdf640 1 --2- 172.21.15.86:0/3380716709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d0c85688e0 0x55d0c856edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:25.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.198+0000 7f60397de640 1 --2- 172.21.15.86:0/3380716709 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0c8569290 0x55d0c856f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:25.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.199+0000 7f603a7e0640 1 --2- 172.21.15.86:0/3380716709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0c8571e10 0x55d0c856f830 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f602c042040 tx=0x7f602c02c890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d23ca291ccdc72b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:25.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.199+0000 7f601d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:25.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.199+0000 7f601d7fa640 1 -- 172.21.15.86:0/3380716709 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f602c04f070 con 0x55d0c8571e10 2024-04-21T08:39:25.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.199+0000 7f601d7fa640 1 -- 172.21.15.86:0/3380716709 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f602c003f20 con 0x55d0c8571e10 2024-04-21T08:39:25.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.199+0000 7f603ee35740 1 -- 172.21.15.86:0/3380716709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f602c045070 con 0x55d0c8571e10 2024-04-21T08:39:25.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.199+0000 7f601d7fa640 1 -- 172.21.15.86:0/3380716709 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f602c032950 con 0x55d0c8571e10 2024-04-21T08:39:25.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.199+0000 7f603ee35740 10 client.16203 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:25.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.199+0000 7f603ee35740 1 -- 172.21.15.86:0/3380716709 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d0c819f170 con 0x55d0c8571e10 2024-04-21T08:39:25.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.199+0000 7f601d7fa640 1 -- 172.21.15.86:0/3380716709 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f602c045070 con 0x55d0c8571e10 2024-04-21T08:39:25.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.199+0000 7f601d7fa640 1 -- 172.21.15.86:0/3380716709 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f602c040930 con 0x55d0c8571e10 2024-04-21T08:39:25.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.200+0000 7f603ee35740 10 client.16203 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:25.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.200+0000 7f603ee35740 10 client.16203 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:25.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.200+0000 7f603ee35740 20 client.16203 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:25.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.200+0000 7f603ee35740 1 --2- 172.21.15.86:0/3380716709 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d0c8582140 0x55d0c8584530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:25.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.200+0000 7f603ee35740 4 client.16203 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:25.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.200+0000 7f603ee35740 1 -- 172.21.15.86:0/3380716709 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d0c84da920 con 0x55d0c8582140 2024-04-21T08:39:25.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.200+0000 7f6039fdf640 1 --2- 172.21.15.86:0/3380716709 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d0c8582140 0x55d0c8584530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:25.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.200+0000 7f6039fdf640 1 --2- 172.21.15.86:0/3380716709 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d0c8582140 0x55d0c8584530 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=525f5ab09af9e39c server_cookie=c4e5a6b06eb1dafa in_seq=0 out_seq=0 2024-04-21T08:39:25.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.200+0000 7f601d7fa640 10 client.16203 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:25.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.201+0000 7f601d7fa640 1 -- 172.21.15.86:0/3380716709 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d0c84da920 con 0x55d0c8582140 2024-04-21T08:39:25.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.201+0000 7f601d7fa640 10 client.16203 handle_command_reply: tid=0 2024-04-21T08:39:25.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.201+0000 7f603ee35740 1 client.16203 shutdown 2024-04-21T08:39:25.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.201+0000 7f603ee35740 1 -- 172.21.15.86:0/3380716709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0c8571e10 msgr2=0x55d0c856f830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:25.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.201+0000 7f603ee35740 1 --2- 172.21.15.86:0/3380716709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0c8571e10 0x55d0c856f830 secure :-1 s=READY pgs=682 cs=0 l=1 rev1=1 crypto rx=0x7f602c042040 tx=0x7f602c02c890 comp rx=0 tx=0).stop 2024-04-21T08:39:25.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.201+0000 7f603ee35740 1 -- 172.21.15.86:0/3380716709 shutdown_connections 2024-04-21T08:39:25.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.201+0000 7f603ee35740 1 --2- 172.21.15.86:0/3380716709 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d0c8582140 0x55d0c8584530 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.201+0000 7f603ee35740 1 --2- 172.21.15.86:0/3380716709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0c8571e10 0x55d0c856f830 unknown :-1 s=CLOSED pgs=682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.201+0000 7f601d7fa640 0 client.16203 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:25.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.201+0000 7f603ee35740 1 --2- 172.21.15.86:0/3380716709 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0c8569290 0x55d0c856f2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.201+0000 7f603ee35740 1 --2- 172.21.15.86:0/3380716709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d0c85688e0 0x55d0c856edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.201+0000 7f603ee35740 1 -- 172.21.15.86:0/3380716709 >> 172.21.15.86:0/3380716709 conn(0x55d0c8562fb0 msgr2=0x55d0c8564d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:25.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.201+0000 7f603ee35740 1 -- 172.21.15.86:0/3380716709 shutdown_connections 2024-04-21T08:39:25.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.201+0000 7f603ee35740 1 -- 172.21.15.86:0/3380716709 wait complete. 2024-04-21T08:39:25.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.202+0000 7f603ee35740 20 client.16203 trim_cache size 0 max 16384 2024-04-21T08:39:25.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.235+0000 7f603ee35740 1 Processor -- start 2024-04-21T08:39:25.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.235+0000 7f603ee35740 1 -- start start 2024-04-21T08:39:25.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.235+0000 7f603ee35740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0c85688e0 0x55d0c857dbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:25.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.235+0000 7f603ee35740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0c8569290 0x55d0c856f650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:25.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.235+0000 7f603ee35740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d0c8571e10 0x55d0c8589ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:25.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.236+0000 7f603ee35740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60341aefc0 con 0x55d0c85688e0 2024-04-21T08:39:25.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.236+0000 7f603ee35740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f60341b0b00 con 0x55d0c8569290 2024-04-21T08:39:25.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.236+0000 7f603ee35740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f60341af690 con 0x55d0c8571e10 2024-04-21T08:39:25.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.236+0000 7f6039fdf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0c85688e0 0x55d0c857dbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:25.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.236+0000 7f6039fdf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0c85688e0 0x55d0c857dbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55772/0 (socket says 172.21.15.86:55772) 2024-04-21T08:39:25.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.236+0000 7f6039fdf640 1 -- 172.21.15.86:0/138281281 learned_addr learned my addr 172.21.15.86:0/138281281 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:25.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.236+0000 7f603a7e0640 1 --2- 172.21.15.86:0/138281281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d0c8571e10 0x55d0c8589ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:25.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.236+0000 7f60397de640 1 --2- 172.21.15.86:0/138281281 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0c8569290 0x55d0c856f650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:25.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.236+0000 7f6039fdf640 1 -- 172.21.15.86:0/138281281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d0c8571e10 msgr2=0x55d0c8589ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:25.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.236+0000 7f6039fdf640 1 --2- 172.21.15.86:0/138281281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d0c8571e10 0x55d0c8589ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.236+0000 7f6039fdf640 1 -- 172.21.15.86:0/138281281 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0c8569290 msgr2=0x55d0c856f650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:25.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.236+0000 7f6039fdf640 1 --2- 172.21.15.86:0/138281281 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0c8569290 0x55d0c856f650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.236+0000 7f6039fdf640 1 -- 172.21.15.86:0/138281281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d0c856d300 con 0x55d0c85688e0 2024-04-21T08:39:25.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.236+0000 7f603a7e0640 1 --2- 172.21.15.86:0/138281281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d0c8571e10 0x55d0c8589ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:25.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.236+0000 7f60397de640 1 --2- 172.21.15.86:0/138281281 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0c8569290 0x55d0c856f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:25.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.236+0000 7f6039fdf640 1 --2- 172.21.15.86:0/138281281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0c85688e0 0x55d0c857dbe0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f6030009610 tx=0x7f603001d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5f321882273a4d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:25.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.236+0000 7f601f7fe640 1 -- 172.21.15.86:0/138281281 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60300b4020 con 0x55d0c85688e0 2024-04-21T08:39:25.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.236+0000 7f601f7fe640 1 -- 172.21.15.86:0/138281281 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f603009a550 con 0x55d0c85688e0 2024-04-21T08:39:25.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.236+0000 7f601f7fe640 1 -- 172.21.15.86:0/138281281 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6030020040 con 0x55d0c85688e0 2024-04-21T08:39:25.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.237+0000 7f603ee35740 1 -- 172.21.15.86:0/138281281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0c85688e0 msgr2=0x55d0c857dbe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:25.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.237+0000 7f603ee35740 1 --2- 172.21.15.86:0/138281281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0c85688e0 0x55d0c857dbe0 secure :-1 s=READY pgs=683 cs=0 l=1 rev1=1 crypto rx=0x7f6030009610 tx=0x7f603001d040 comp rx=0 tx=0).stop 2024-04-21T08:39:25.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.237+0000 7f603ee35740 1 -- 172.21.15.86:0/138281281 shutdown_connections 2024-04-21T08:39:25.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.237+0000 7f603ee35740 1 --2- 172.21.15.86:0/138281281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d0c8571e10 0x55d0c8589ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.237+0000 7f603ee35740 1 --2- 172.21.15.86:0/138281281 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0c8569290 0x55d0c856f650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.237+0000 7f603ee35740 1 --2- 172.21.15.86:0/138281281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0c85688e0 0x55d0c857dbe0 unknown :-1 s=CLOSED pgs=683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.237+0000 7f603ee35740 1 -- 172.21.15.86:0/138281281 >> 172.21.15.86:0/138281281 conn(0x55d0c8562fb0 msgr2=0x55d0c8562010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:25.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.237+0000 7f603ee35740 1 -- 172.21.15.86:0/138281281 shutdown_connections 2024-04-21T08:39:25.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.237+0000 7f603ee35740 1 -- 172.21.15.86:0/138281281 wait complete. 2024-04-21T08:39:25.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.238+0000 7f603ee35740 1 -- start start 2024-04-21T08:39:25.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.238+0000 7f603ee35740 1 Processor -- start 2024-04-21T08:39:25.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.238+0000 7f603ee35740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d0c85688e0 0x55d0c8575ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:25.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.238+0000 7f603ee35740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0c8569290 0x55d0c8576400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:25.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.239+0000 7f60397de640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0c8569290 0x55d0c8576400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:25.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.239+0000 7f60397de640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0c8569290 0x55d0c8576400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55786/0 (socket says 172.21.15.86:55786) 2024-04-21T08:39:25.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.239+0000 7f603ee35740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0c8571e10 0x55d0c85e7340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:25.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.239+0000 7f60397de640 1 -- 172.21.15.86:0/2802621678 learned_addr learned my addr 172.21.15.86:0/2802621678 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:25.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.239+0000 7f603ee35740 1 -- 172.21.15.86:0/2802621678 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60341b0430 con 0x55d0c8569290 2024-04-21T08:39:25.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.239+0000 7f603ee35740 1 -- 172.21.15.86:0/2802621678 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f60341afd60 con 0x55d0c8571e10 2024-04-21T08:39:25.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.239+0000 7f603ee35740 1 -- 172.21.15.86:0/2802621678 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f60341acf80 con 0x55d0c85688e0 2024-04-21T08:39:25.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.239+0000 7f603ee35740 10 client.0 _ll_register_callbacks cb 0x7f602400b830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:25.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.239+0000 7f6039fdf640 1 --2- 172.21.15.86:0/2802621678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d0c85688e0 0x55d0c8575ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:25.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.239+0000 7f603a7e0640 1 --2- 172.21.15.86:0/2802621678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0c8571e10 0x55d0c85e7340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:25.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.239+0000 7f60397de640 1 -- 172.21.15.86:0/2802621678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d0c85688e0 msgr2=0x55d0c8575ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:25.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.239+0000 7f60397de640 1 --2- 172.21.15.86:0/2802621678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d0c85688e0 0x55d0c8575ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.239+0000 7f60397de640 1 -- 172.21.15.86:0/2802621678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0c8571e10 msgr2=0x55d0c85e7340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:25.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.239+0000 7f60397de640 1 --2- 172.21.15.86:0/2802621678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0c8571e10 0x55d0c85e7340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.239+0000 7f60397de640 1 -- 172.21.15.86:0/2802621678 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d0c85e7880 con 0x55d0c8569290 2024-04-21T08:39:25.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.239+0000 7f603a7e0640 1 --2- 172.21.15.86:0/2802621678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0c8571e10 0x55d0c85e7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:25.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.239+0000 7f6039fdf640 1 --2- 172.21.15.86:0/2802621678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d0c85688e0 0x55d0c8575ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:25.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.239+0000 7f60397de640 1 --2- 172.21.15.86:0/2802621678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0c8569290 0x55d0c8576400 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f602400b940 tx=0x7f6024004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=354351fdd86b675b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:25.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.239+0000 7f601d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:25.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.240+0000 7f601d7fa640 1 -- 172.21.15.86:0/2802621678 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6024005020 con 0x55d0c8569290 2024-04-21T08:39:25.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.240+0000 7f601d7fa640 1 -- 172.21.15.86:0/2802621678 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f60240051c0 con 0x55d0c8569290 2024-04-21T08:39:25.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.240+0000 7f603ee35740 1 -- 172.21.15.86:0/2802621678 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f603009a550 con 0x55d0c8569290 2024-04-21T08:39:25.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.240+0000 7f601d7fa640 1 -- 172.21.15.86:0/2802621678 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f60240053c0 con 0x55d0c8569290 2024-04-21T08:39:25.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.240+0000 7f603ee35740 10 client.16209 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:25.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.240+0000 7f603ee35740 1 -- 172.21.15.86:0/2802621678 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d0c85e8a30 con 0x55d0c8569290 2024-04-21T08:39:25.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.240+0000 7f601d7fa640 1 -- 172.21.15.86:0/2802621678 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f603009a550 con 0x55d0c8569290 2024-04-21T08:39:25.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.240+0000 7f601d7fa640 1 -- 172.21.15.86:0/2802621678 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6024004930 con 0x55d0c8569290 2024-04-21T08:39:25.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.240+0000 7f603ee35740 10 client.16209 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:25.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.240+0000 7f603ee35740 10 client.16209 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:25.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.240+0000 7f603ee35740 20 client.16209 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:25.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.240+0000 7f603ee35740 1 --2- 172.21.15.86:0/2802621678 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d0c85e9260 0x55d0c85eb680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:25.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.240+0000 7f603ee35740 4 client.16209 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:39:25.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.240+0000 7f603ee35740 1 -- 172.21.15.86:0/2802621678 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d0c84d4790 con 0x55d0c85e9260 2024-04-21T08:39:25.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.241+0000 7f6039fdf640 1 --2- 172.21.15.86:0/2802621678 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d0c85e9260 0x55d0c85eb680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:25.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.241+0000 7f601d7fa640 10 client.16209 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:25.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.241+0000 7f6039fdf640 1 --2- 172.21.15.86:0/2802621678 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d0c85e9260 0x55d0c85eb680 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96ff92936de43f91 server_cookie=c33189134579b133 in_seq=0 out_seq=0 2024-04-21T08:39:25.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.242+0000 7f601d7fa640 1 -- 172.21.15.86:0/2802621678 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d0c84d4790 con 0x55d0c85e9260 2024-04-21T08:39:25.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.242+0000 7f601d7fa640 10 client.16209 handle_command_reply: tid=0 2024-04-21T08:39:25.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.242+0000 7f603ee35740 1 client.16209 shutdown 2024-04-21T08:39:25.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.242+0000 7f603ee35740 1 -- 172.21.15.86:0/2802621678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0c8569290 msgr2=0x55d0c8576400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:25.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.242+0000 7f603ee35740 1 --2- 172.21.15.86:0/2802621678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0c8569290 0x55d0c8576400 secure :-1 s=READY pgs=684 cs=0 l=1 rev1=1 crypto rx=0x7f602400b940 tx=0x7f6024004730 comp rx=0 tx=0).stop 2024-04-21T08:39:25.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.242+0000 7f603ee35740 1 -- 172.21.15.86:0/2802621678 shutdown_connections 2024-04-21T08:39:25.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.242+0000 7f603ee35740 1 --2- 172.21.15.86:0/2802621678 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d0c85e9260 0x55d0c85eb680 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.243+0000 7f601d7fa640 0 client.16209 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:25.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.242+0000 7f603ee35740 1 --2- 172.21.15.86:0/2802621678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d0c8571e10 0x55d0c85e7340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.243+0000 7f603ee35740 1 --2- 172.21.15.86:0/2802621678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d0c8569290 0x55d0c8576400 unknown :-1 s=CLOSED pgs=684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.243+0000 7f603ee35740 1 --2- 172.21.15.86:0/2802621678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d0c85688e0 0x55d0c8575ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.243+0000 7f603ee35740 1 -- 172.21.15.86:0/2802621678 >> 172.21.15.86:0/2802621678 conn(0x55d0c8562fb0 msgr2=0x55d0c8564050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:25.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.243+0000 7f603ee35740 1 -- 172.21.15.86:0/2802621678 shutdown_connections 2024-04-21T08:39:25.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.243+0000 7f603ee35740 1 -- 172.21.15.86:0/2802621678 wait complete. 2024-04-21T08:39:25.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.243+0000 7f603ee35740 20 client.16209 trim_cache size 0 max 16384 2024-04-21T08:39:25.246 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:39:25.246 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-21T08:39:25.247 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "93733a9d-4340-43f9-b190-52578cdd3b3f", 2024-04-21T08:39:25.247 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-21T08:39:25.247 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:39:25.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.243+0000 7f603c243640 1 -- 172.21.15.86:0/2790702371 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f60200a3ec0 msgr2=0x7f60200a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:25.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.243+0000 7f603c243640 1 --2- 172.21.15.86:0/2790702371 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f60200a3ec0 0x7f60200a6380 secure :-1 s=READY pgs=92 cs=0 l=1 rev1=1 crypto rx=0x7f60340787a0 tx=0x7f602400b390 comp rx=0 tx=0).stop 2024-04-21T08:39:25.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.243+0000 7f603c243640 1 -- 172.21.15.86:0/2790702371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f603406a810 msgr2=0x7f6034198530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:25.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.243+0000 7f603c243640 1 --2- 172.21.15.86:0/2790702371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f603406a810 0x7f6034198530 secure :-1 s=READY pgs=681 cs=0 l=1 rev1=1 crypto rx=0x7f603000d770 tx=0x7f603000dc30 comp rx=0 tx=0).stop 2024-04-21T08:39:25.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.243+0000 7f603c243640 1 -- 172.21.15.86:0/2790702371 shutdown_connections 2024-04-21T08:39:25.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.243+0000 7f603c243640 1 --2- 172.21.15.86:0/2790702371 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f60200a3ec0 0x7f60200a6380 unknown :-1 s=CLOSED pgs=92 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.243+0000 7f603c243640 1 --2- 172.21.15.86:0/2790702371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f603419cdf0 0x7f603419d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.243+0000 7f603c243640 1 --2- 172.21.15.86:0/2790702371 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6034107260 0x7f6034198a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.243+0000 7f603c243640 1 --2- 172.21.15.86:0/2790702371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f603406a810 0x7f6034198530 unknown :-1 s=CLOSED pgs=681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.243+0000 7f603c243640 1 -- 172.21.15.86:0/2790702371 >> 172.21.15.86:0/2790702371 conn(0x7f6034077fb0 msgr2=0x7f60340febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:25.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.244+0000 7f603c243640 1 -- 172.21.15.86:0/2790702371 shutdown_connections 2024-04-21T08:39:25.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.244+0000 7f603c243640 1 -- 172.21.15.86:0/2790702371 wait complete. 2024-04-21T08:39:25.257 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:39:25.257 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:39:25.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.364+0000 7ff9bab6d640 1 -- 172.21.15.86:0/575683076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9b4106150 msgr2=0x7ff9b410dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:25.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.364+0000 7ff9bab6d640 1 --2- 172.21.15.86:0/575683076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9b4106150 0x7ff9b410dd20 secure :-1 s=READY pgs=685 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac009ca0 tx=0x7ff9ac02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.364+0000 7ff9bab6d640 1 -- 172.21.15.86:0/575683076 shutdown_connections 2024-04-21T08:39:25.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.364+0000 7ff9bab6d640 1 --2- 172.21.15.86:0/575683076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9b4106150 0x7ff9b410dd20 unknown :-1 s=CLOSED pgs=685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.364+0000 7ff9bab6d640 1 --2- 172.21.15.86:0/575683076 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9b4101a30 0x7ff9b4105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.364+0000 7ff9bab6d640 1 --2- 172.21.15.86:0/575683076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9b4101080 0x7ff9b4101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.364+0000 7ff9bab6d640 1 -- 172.21.15.86:0/575683076 >> 172.21.15.86:0/575683076 conn(0x7ff9b4078000 msgr2=0x7ff9b4078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:25.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.364+0000 7ff9bab6d640 1 -- 172.21.15.86:0/575683076 shutdown_connections 2024-04-21T08:39:25.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.364+0000 7ff9bab6d640 1 -- 172.21.15.86:0/575683076 wait complete. 2024-04-21T08:39:25.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.365+0000 7ff9bab6d640 1 Processor -- start 2024-04-21T08:39:25.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.365+0000 7ff9bab6d640 1 -- start start 2024-04-21T08:39:25.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.365+0000 7ff9bab6d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9b4101080 0x7ff9b410e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:25.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.365+0000 7ff9bab6d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9b4106150 0x7ff9b410ebf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:25.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.365+0000 7ff9bab6d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9b410f130 0x7ff9b41a6860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:25.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.365+0000 7ff9bab6d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b41121d0 con 0x7ff9b410f130 2024-04-21T08:39:25.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.365+0000 7ff9bab6d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b4112050 con 0x7ff9b4101080 2024-04-21T08:39:25.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.365+0000 7ff9bab6d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b4112350 con 0x7ff9b4106150 2024-04-21T08:39:25.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.365+0000 7ff9b910a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9b410f130 0x7ff9b41a6860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:25.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.365+0000 7ff9b3fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9b4106150 0x7ff9b410ebf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:25.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.365+0000 7ff9b910a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9b410f130 0x7ff9b41a6860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55812/0 (socket says 172.21.15.86:55812) 2024-04-21T08:39:25.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.365+0000 7ff9b910a640 1 -- 172.21.15.86:0/3317289482 learned_addr learned my addr 172.21.15.86:0/3317289482 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:25.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.365+0000 7ff9b8909640 1 --2- 172.21.15.86:0/3317289482 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9b4101080 0x7ff9b410e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:25.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.365+0000 7ff9b910a640 1 -- 172.21.15.86:0/3317289482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9b4106150 msgr2=0x7ff9b410ebf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:25.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.365+0000 7ff9b910a640 1 --2- 172.21.15.86:0/3317289482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9b4106150 0x7ff9b410ebf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.365+0000 7ff9b910a640 1 -- 172.21.15.86:0/3317289482 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9b4101080 msgr2=0x7ff9b410e6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:25.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.365+0000 7ff9b910a640 1 --2- 172.21.15.86:0/3317289482 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9b4101080 0x7ff9b410e6b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.365+0000 7ff9b910a640 1 -- 172.21.15.86:0/3317289482 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff9b41a6f00 con 0x7ff9b410f130 2024-04-21T08:39:25.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.365+0000 7ff9b3fff640 1 --2- 172.21.15.86:0/3317289482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9b4106150 0x7ff9b410ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:25.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.365+0000 7ff9b8909640 1 --2- 172.21.15.86:0/3317289482 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9b4101080 0x7ff9b410e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:25.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.366+0000 7ff9b910a640 1 --2- 172.21.15.86:0/3317289482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9b410f130 0x7ff9b41a6860 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac005e30 tx=0x7ff9ac034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:25.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.366+0000 7ff9b1ffb640 1 -- 172.21.15.86:0/3317289482 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac041020 con 0x7ff9b410f130 2024-04-21T08:39:25.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.366+0000 7ff9bab6d640 1 -- 172.21.15.86:0/3317289482 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff9b41a7190 con 0x7ff9b410f130 2024-04-21T08:39:25.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.366+0000 7ff9b1ffb640 1 -- 172.21.15.86:0/3317289482 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9ac032970 con 0x7ff9b410f130 2024-04-21T08:39:25.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.366+0000 7ff9b1ffb640 1 -- 172.21.15.86:0/3317289482 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9ac039040 con 0x7ff9b410f130 2024-04-21T08:39:25.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.366+0000 7ff9bab6d640 1 -- 172.21.15.86:0/3317289482 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9b41a76d0 con 0x7ff9b410f130 2024-04-21T08:39:25.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.369+0000 7ff9bd835740 1 Processor -- start 2024-04-21T08:39:25.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.369+0000 7ff9b1ffb640 1 -- 172.21.15.86:0/3317289482 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7ff9ac044020 con 0x7ff9b410f130 2024-04-21T08:39:25.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.370+0000 7ff9bd835740 1 -- start start 2024-04-21T08:39:25.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.370+0000 7ff9bd835740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a6755390 0x5579a6755770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:25.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.370+0000 7ff9bd835740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579a6755d40 0x5579a6766d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:25.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.370+0000 7ff9bd835740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579a6767290 0x5579a6769680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:25.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.370+0000 7ff9bd835740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5579a66ba1e0 con 0x5579a6755390 2024-04-21T08:39:25.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.370+0000 7ff9bd835740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5579a6657390 con 0x5579a6755d40 2024-04-21T08:39:25.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.370+0000 7ff9bd835740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5579a66b61a0 con 0x5579a6767290 2024-04-21T08:39:25.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.370+0000 7ff9b1ffb640 1 --2- 172.21.15.86:0/3317289482 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff9a40a3e40 0x7ff9a40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:25.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.370+0000 7ff9b8909640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579a6755d40 0x5579a6766d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:25.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.370+0000 7ff9b8909640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579a6755d40 0x5579a6766d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.86:59756/0 (socket says 172.21.15.86:59756) 2024-04-21T08:39:25.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.370+0000 7ff9b8909640 1 -- 172.21.15.86:0/4039184051 learned_addr learned my addr 172.21.15.86:0/4039184051 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:25.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.370+0000 7ff9b1ffb640 1 -- 172.21.15.86:0/3317289482 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7ff9ac0f2a70 con 0x7ff9b410f130 2024-04-21T08:39:25.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.371+0000 7ff9b910a640 1 --2- 172.21.15.86:0/3317289482 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff9a40a3e40 0x7ff9a40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:25.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.371+0000 7ff9b3fff640 1 --2- 172.21.15.86:0/4039184051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a6755390 0x5579a6755770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:25.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.371+0000 7ff9b8909640 1 -- 172.21.15.86:0/4039184051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579a6767290 msgr2=0x5579a6769680 unknown :-1 s=STATE_CONNECTING l=0).mark_down 2024-04-21T08:39:25.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.371+0000 7ff9b8909640 1 --2- 172.21.15.86:0/4039184051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579a6767290 0x5579a6769680 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.371+0000 7ff9b8909640 1 -- 172.21.15.86:0/4039184051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a6755390 msgr2=0x5579a6755770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:25.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.371+0000 7ff9b8909640 1 --2- 172.21.15.86:0/4039184051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a6755390 0x5579a6755770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.371+0000 7ff9b8909640 1 -- 172.21.15.86:0/4039184051 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579a64967d0 con 0x5579a6755d40 2024-04-21T08:39:25.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.371+0000 7ff9b3fff640 1 --2- 172.21.15.86:0/4039184051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a6755390 0x5579a6755770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:25.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.371+0000 7ff9b910a640 1 --2- 172.21.15.86:0/3317289482 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff9a40a3e40 0x7ff9a40a6300 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac0efe40 tx=0x7ff9ac0095c0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:25.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.371+0000 7ff9b8909640 1 --2- 172.21.15.86:0/4039184051 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579a6755d40 0x5579a6766d50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff9a800f4f0 tx=0x7ff9a800f9b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=77b7878d056c18cd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:25.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.372+0000 7ff99effd640 1 -- 172.21.15.86:0/4039184051 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a8005220 con 0x5579a6755d40 2024-04-21T08:39:25.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.372+0000 7ff99effd640 1 -- 172.21.15.86:0/4039184051 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9a8014070 con 0x5579a6755d40 2024-04-21T08:39:25.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.372+0000 7ff99effd640 1 -- 172.21.15.86:0/4039184051 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a8010a50 con 0x5579a6755d40 2024-04-21T08:39:25.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.373+0000 7ff9bd835740 1 -- 172.21.15.86:0/4039184051 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579a6755d40 msgr2=0x5579a6766d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:25.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.373+0000 7ff9bd835740 1 --2- 172.21.15.86:0/4039184051 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579a6755d40 0x5579a6766d50 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7ff9a800f4f0 tx=0x7ff9a800f9b0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.373+0000 7ff9bd835740 1 -- 172.21.15.86:0/4039184051 shutdown_connections 2024-04-21T08:39:25.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.373+0000 7ff9bd835740 1 --2- 172.21.15.86:0/4039184051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579a6767290 0x5579a6769680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.373+0000 7ff9bd835740 1 --2- 172.21.15.86:0/4039184051 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579a6755d40 0x5579a6766d50 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.373+0000 7ff9bd835740 1 --2- 172.21.15.86:0/4039184051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a6755390 0x5579a6755770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.373+0000 7ff9bd835740 1 -- 172.21.15.86:0/4039184051 >> 172.21.15.86:0/4039184051 conn(0x5579a674fcb0 msgr2=0x5579a6750090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:25.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.373+0000 7ff9bd835740 1 -- 172.21.15.86:0/4039184051 shutdown_connections 2024-04-21T08:39:25.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.373+0000 7ff9bd835740 1 -- 172.21.15.86:0/4039184051 wait complete. 2024-04-21T08:39:25.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.373+0000 7ff9bd835740 1 -- start start 2024-04-21T08:39:25.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.373+0000 7ff9bd835740 1 Processor -- start 2024-04-21T08:39:25.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.374+0000 7ff9bd835740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579a6755390 0x5579a67642b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:25.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.374+0000 7ff9bd835740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a6755d40 0x5579a67647f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:25.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.374+0000 7ff9bd835740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579a6767290 0x5579a6764d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:25.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.374+0000 7ff9bd835740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5579a676a050 con 0x5579a6755d40 2024-04-21T08:39:25.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.374+0000 7ff9bd835740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5579a66732f0 con 0x5579a6767290 2024-04-21T08:39:25.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.374+0000 7ff9bd835740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5579a676a1d0 con 0x5579a6755390 2024-04-21T08:39:25.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.374+0000 7ff9bd835740 10 client.0 _ll_register_callbacks cb 0x5579a6680580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:25.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.374+0000 7ff9b8909640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a6755d40 0x5579a67647f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:25.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.374+0000 7ff9b8909640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a6755d40 0x5579a67647f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55826/0 (socket says 172.21.15.86:55826) 2024-04-21T08:39:25.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.374+0000 7ff9b8909640 1 -- 172.21.15.86:0/1400723533 learned_addr learned my addr 172.21.15.86:0/1400723533 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:25.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.374+0000 7ff9b3fff640 1 --2- 172.21.15.86:0/1400723533 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579a6755390 0x5579a67642b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:25.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.374+0000 7ff9b3fff640 1 --2- 172.21.15.86:0/1400723533 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579a6767290 0x5579a6764d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:25.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.374+0000 7ff9b8909640 1 -- 172.21.15.86:0/1400723533 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579a6755390 msgr2=0x5579a67642b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:25.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.374+0000 7ff9b8909640 1 --2- 172.21.15.86:0/1400723533 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579a6755390 0x5579a67642b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.374+0000 7ff9b8909640 1 -- 172.21.15.86:0/1400723533 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579a6767290 msgr2=0x5579a6764d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:25.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.374+0000 7ff9b8909640 1 --2- 172.21.15.86:0/1400723533 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579a6767290 0x5579a6764d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.374+0000 7ff9b8909640 1 -- 172.21.15.86:0/1400723533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579a63cc3f0 con 0x5579a6755d40 2024-04-21T08:39:25.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.374+0000 7ff9b3fff640 1 --2- 172.21.15.86:0/1400723533 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579a6767290 0x5579a6764d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:25.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.374+0000 7ff9b8909640 1 --2- 172.21.15.86:0/1400723533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a6755d40 0x5579a67647f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ff9a800ac10 tx=0x7ff9a800ac40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a65a9fb075fe677 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:25.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.374+0000 7ff993fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:25.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.374+0000 7ff993fff640 1 -- 172.21.15.86:0/1400723533 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a8025070 con 0x5579a6755d40 2024-04-21T08:39:25.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.375+0000 7ff9bd835740 1 -- 172.21.15.86:0/1400723533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9a8014070 con 0x5579a6755d40 2024-04-21T08:39:25.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.375+0000 7ff993fff640 1 -- 172.21.15.86:0/1400723533 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7ff9a80046c0 con 0x5579a6755d40 2024-04-21T08:39:25.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.375+0000 7ff993fff640 1 -- 172.21.15.86:0/1400723533 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff9a8010aa0 con 0x5579a6755d40 2024-04-21T08:39:25.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.375+0000 7ff9bd835740 10 client.16218 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:25.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.375+0000 7ff9bd835740 1 -- 172.21.15.86:0/1400723533 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5579a638c0f0 con 0x5579a6755d40 2024-04-21T08:39:25.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.375+0000 7ff993fff640 1 -- 172.21.15.86:0/1400723533 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9a8014070 con 0x5579a6755d40 2024-04-21T08:39:25.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.375+0000 7ff993fff640 1 -- 172.21.15.86:0/1400723533 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff9a80186f0 con 0x5579a6755d40 2024-04-21T08:39:25.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.375+0000 7ff9bd835740 10 client.16218 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:25.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.375+0000 7ff9bd835740 10 client.16218 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:25.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.375+0000 7ff9bd835740 20 client.16218 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:25.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.375+0000 7ff9bd835740 1 --2- 172.21.15.86:0/1400723533 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5579a67770d0 0x5579a67794c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:25.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.375+0000 7ff9bd835740 4 client.16218 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:25.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.375+0000 7ff9bd835740 1 -- 172.21.15.86:0/1400723533 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579a66c7c00 con 0x5579a67770d0 2024-04-21T08:39:25.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.376+0000 7ff9b3fff640 1 --2- 172.21.15.86:0/1400723533 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5579a67770d0 0x5579a67794c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:25.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.376+0000 7ff9b3fff640 1 --2- 172.21.15.86:0/1400723533 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5579a67770d0 0x5579a67794c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=100c20e4d857034f server_cookie=d0be948bdd1dfb14 in_seq=0 out_seq=0 2024-04-21T08:39:25.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.376+0000 7ff993fff640 10 client.16218 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:25.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.377+0000 7ff993fff640 1 -- 172.21.15.86:0/1400723533 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5579a66c7c00 con 0x5579a67770d0 2024-04-21T08:39:25.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.377+0000 7ff993fff640 10 client.16218 handle_command_reply: tid=0 2024-04-21T08:39:25.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.377+0000 7ff9bd835740 1 client.16218 shutdown 2024-04-21T08:39:25.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.377+0000 7ff9bd835740 1 -- 172.21.15.86:0/1400723533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a6755d40 msgr2=0x5579a67647f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:25.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.377+0000 7ff9bd835740 1 --2- 172.21.15.86:0/1400723533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a6755d40 0x5579a67647f0 secure :-1 s=READY pgs=687 cs=0 l=1 rev1=1 crypto rx=0x7ff9a800ac10 tx=0x7ff9a800ac40 comp rx=0 tx=0).stop 2024-04-21T08:39:25.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.377+0000 7ff9bd835740 1 -- 172.21.15.86:0/1400723533 shutdown_connections 2024-04-21T08:39:25.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.377+0000 7ff9bd835740 1 --2- 172.21.15.86:0/1400723533 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5579a67770d0 0x5579a67794c0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.377+0000 7ff993fff640 0 client.16218 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:25.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.377+0000 7ff9bd835740 1 --2- 172.21.15.86:0/1400723533 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579a6767290 0x5579a6764d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.377+0000 7ff9bd835740 1 --2- 172.21.15.86:0/1400723533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a6755d40 0x5579a67647f0 unknown :-1 s=CLOSED pgs=687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.377+0000 7ff9bd835740 1 --2- 172.21.15.86:0/1400723533 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579a6755390 0x5579a67642b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.377+0000 7ff9bd835740 1 -- 172.21.15.86:0/1400723533 >> 172.21.15.86:0/1400723533 conn(0x5579a674fcb0 msgr2=0x5579a6751a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:25.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.377+0000 7ff9bd835740 1 -- 172.21.15.86:0/1400723533 shutdown_connections 2024-04-21T08:39:25.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.377+0000 7ff9bd835740 1 -- 172.21.15.86:0/1400723533 wait complete. 2024-04-21T08:39:25.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.377+0000 7ff9bd835740 20 client.16218 trim_cache size 0 max 16384 2024-04-21T08:39:25.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.394+0000 7ff9bd835740 1 Processor -- start 2024-04-21T08:39:25.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.394+0000 7ff9bd835740 1 -- start start 2024-04-21T08:39:25.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.394+0000 7ff9bd835740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579a674fcb0 0x5579a67669e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:25.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.394+0000 7ff9bd835740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a6755390 0x5579a6772f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:25.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.395+0000 7ff9bd835740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579a6755d40 0x5579a677eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:25.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.395+0000 7ff9bd835740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b41ad0c0 con 0x5579a6755390 2024-04-21T08:39:25.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.395+0000 7ff9bd835740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b41aec00 con 0x5579a6755d40 2024-04-21T08:39:25.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.395+0000 7ff9bd835740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b41ad790 con 0x5579a674fcb0 2024-04-21T08:39:25.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.395+0000 7ff9b8909640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a6755390 0x5579a6772f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:25.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.395+0000 7ff9b8909640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a6755390 0x5579a6772f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55842/0 (socket says 172.21.15.86:55842) 2024-04-21T08:39:25.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.395+0000 7ff9b8909640 1 -- 172.21.15.86:0/2414953656 learned_addr learned my addr 172.21.15.86:0/2414953656 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:25.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.395+0000 7ff9b3fff640 1 --2- 172.21.15.86:0/2414953656 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579a6755d40 0x5579a677eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:25.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.395+0000 7ff9b8909640 1 -- 172.21.15.86:0/2414953656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579a674fcb0 msgr2=0x5579a67669e0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:39:25.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.395+0000 7ff9b8909640 1 --2- 172.21.15.86:0/2414953656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579a674fcb0 0x5579a67669e0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.395+0000 7ff9b8909640 1 -- 172.21.15.86:0/2414953656 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579a6755d40 msgr2=0x5579a677eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:25.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.395+0000 7ff9b8909640 1 --2- 172.21.15.86:0/2414953656 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579a6755d40 0x5579a677eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.395+0000 7ff9b8909640 1 -- 172.21.15.86:0/2414953656 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579a67699b0 con 0x5579a6755390 2024-04-21T08:39:25.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.395+0000 7ff9b8909640 1 --2- 172.21.15.86:0/2414953656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a6755390 0x5579a6772f50 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff9a800f3e0 tx=0x7ff9a8005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a6678c1b16ba1d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:25.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.395+0000 7ff99e7fc640 1 -- 172.21.15.86:0/2414953656 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a8025040 con 0x5579a6755390 2024-04-21T08:39:25.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.395+0000 7ff99e7fc640 1 -- 172.21.15.86:0/2414953656 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9a8014030 con 0x5579a6755390 2024-04-21T08:39:25.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.395+0000 7ff99e7fc640 1 -- 172.21.15.86:0/2414953656 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a80048c0 con 0x5579a6755390 2024-04-21T08:39:25.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.396+0000 7ff9bd835740 1 -- 172.21.15.86:0/2414953656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a6755390 msgr2=0x5579a6772f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:25.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.396+0000 7ff9bd835740 1 --2- 172.21.15.86:0/2414953656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a6755390 0x5579a6772f50 secure :-1 s=READY pgs=688 cs=0 l=1 rev1=1 crypto rx=0x7ff9a800f3e0 tx=0x7ff9a8005850 comp rx=0 tx=0).stop 2024-04-21T08:39:25.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.396+0000 7ff9bd835740 1 -- 172.21.15.86:0/2414953656 shutdown_connections 2024-04-21T08:39:25.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.396+0000 7ff9bd835740 1 --2- 172.21.15.86:0/2414953656 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579a6755d40 0x5579a677eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.396+0000 7ff9bd835740 1 --2- 172.21.15.86:0/2414953656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a6755390 0x5579a6772f50 unknown :-1 s=CLOSED pgs=688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.396+0000 7ff9bd835740 1 --2- 172.21.15.86:0/2414953656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579a674fcb0 0x5579a67669e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.396+0000 7ff9bd835740 1 -- 172.21.15.86:0/2414953656 >> 172.21.15.86:0/2414953656 conn(0x5579a6762ef0 msgr2=0x5579a6751990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:25.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.396+0000 7ff9bd835740 1 -- 172.21.15.86:0/2414953656 shutdown_connections 2024-04-21T08:39:25.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.396+0000 7ff9bd835740 1 -- 172.21.15.86:0/2414953656 wait complete. 2024-04-21T08:39:25.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.397+0000 7ff9bd835740 1 -- start start 2024-04-21T08:39:25.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.397+0000 7ff9bd835740 1 Processor -- start 2024-04-21T08:39:25.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.397+0000 7ff9bd835740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579a674fcb0 0x5579a676b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:25.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.397+0000 7ff9bd835740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a6755390 0x5579a676ba50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:25.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.397+0000 7ff9bd835740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579a6755d40 0x5579a67dc3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:25.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.397+0000 7ff9bd835740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b41ae530 con 0x5579a6755390 2024-04-21T08:39:25.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.397+0000 7ff9bd835740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b41ade60 con 0x5579a6755d40 2024-04-21T08:39:25.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.397+0000 7ff9bd835740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff9b41ab080 con 0x5579a674fcb0 2024-04-21T08:39:25.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.397+0000 7ff9b8909640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a6755390 0x5579a676ba50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:25.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.397+0000 7ff9b8909640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a6755390 0x5579a676ba50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55854/0 (socket says 172.21.15.86:55854) 2024-04-21T08:39:25.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.397+0000 7ff9b8909640 1 -- 172.21.15.86:0/1657643096 learned_addr learned my addr 172.21.15.86:0/1657643096 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:25.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.397+0000 7ff9bd835740 10 client.0 _ll_register_callbacks cb 0x7ff9a800af10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:25.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.398+0000 7ff9b3fff640 1 --2- 172.21.15.86:0/1657643096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579a6755d40 0x5579a67dc3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:25.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.398+0000 7ff9b3fff640 1 --2- 172.21.15.86:0/1657643096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579a674fcb0 0x5579a676b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:25.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.398+0000 7ff9b8909640 1 -- 172.21.15.86:0/1657643096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579a674fcb0 msgr2=0x5579a676b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:25.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.398+0000 7ff9b8909640 1 --2- 172.21.15.86:0/1657643096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579a674fcb0 0x5579a676b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.398+0000 7ff9b8909640 1 -- 172.21.15.86:0/1657643096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579a6755d40 msgr2=0x5579a67dc3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:25.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.398+0000 7ff9b8909640 1 --2- 172.21.15.86:0/1657643096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579a6755d40 0x5579a67dc3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.398+0000 7ff9b8909640 1 -- 172.21.15.86:0/1657643096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579a67dc900 con 0x5579a6755390 2024-04-21T08:39:25.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.398+0000 7ff9b8909640 1 --2- 172.21.15.86:0/1657643096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a6755390 0x5579a676ba50 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff9a800feb0 tx=0x7ff9a800f9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec95e2db16a9ac8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:25.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.398+0000 7ff993fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:25.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.398+0000 7ff993fff640 1 -- 172.21.15.86:0/1657643096 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9a8042060 con 0x5579a6755390 2024-04-21T08:39:25.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.398+0000 7ff993fff640 1 -- 172.21.15.86:0/1657643096 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7ff9a8004a70 con 0x5579a6755390 2024-04-21T08:39:25.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.398+0000 7ff9bd835740 1 -- 172.21.15.86:0/1657643096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9a8014030 con 0x5579a6755390 2024-04-21T08:39:25.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.398+0000 7ff993fff640 1 -- 172.21.15.86:0/1657643096 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff9a8025070 con 0x5579a6755390 2024-04-21T08:39:25.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.398+0000 7ff9bd835740 10 client.16224 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:25.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.398+0000 7ff9bd835740 1 -- 172.21.15.86:0/1657643096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5579a67ddb80 con 0x5579a6755390 2024-04-21T08:39:25.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.399+0000 7ff993fff640 1 -- 172.21.15.86:0/1657643096 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9a80187d0 con 0x5579a6755390 2024-04-21T08:39:25.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.399+0000 7ff993fff640 1 -- 172.21.15.86:0/1657643096 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff9a8021c30 con 0x5579a6755390 2024-04-21T08:39:25.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.399+0000 7ff9bd835740 10 client.16224 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:25.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.399+0000 7ff9bd835740 10 client.16224 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:25.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.399+0000 7ff9bd835740 20 client.16224 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:25.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.399+0000 7ff9bd835740 1 --2- 172.21.15.86:0/1657643096 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5579a6767290 0x5579a67e0060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:25.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.399+0000 7ff9bd835740 4 client.16224 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:39:25.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.399+0000 7ff9bd835740 1 -- 172.21.15.86:0/1657643096 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x7ff9a8014030 con 0x5579a6767290 2024-04-21T08:39:25.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.399+0000 7ff9b3fff640 1 --2- 172.21.15.86:0/1657643096 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5579a6767290 0x5579a67e0060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:25.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.400+0000 7ff9b3fff640 1 --2- 172.21.15.86:0/1657643096 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5579a6767290 0x5579a67e0060 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=68a756dfe1d57058 server_cookie=6ebed4ae2a003e23 in_seq=0 out_seq=0 2024-04-21T08:39:25.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.400+0000 7ff993fff640 10 client.16224 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:25.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.400+0000 7ff993fff640 1 -- 172.21.15.86:0/1657643096 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x7ff9a8014030 con 0x5579a6767290 2024-04-21T08:39:25.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.400+0000 7ff993fff640 10 client.16224 handle_command_reply: tid=0 2024-04-21T08:39:25.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.400+0000 7ff9bd835740 1 client.16224 shutdown 2024-04-21T08:39:25.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.400+0000 7ff9bd835740 1 -- 172.21.15.86:0/1657643096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a6755390 msgr2=0x5579a676ba50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:25.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.400+0000 7ff9bd835740 1 --2- 172.21.15.86:0/1657643096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a6755390 0x5579a676ba50 secure :-1 s=READY pgs=689 cs=0 l=1 rev1=1 crypto rx=0x7ff9a800feb0 tx=0x7ff9a800f9b0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.400+0000 7ff9bd835740 1 -- 172.21.15.86:0/1657643096 shutdown_connections 2024-04-21T08:39:25.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.400+0000 7ff9bd835740 1 --2- 172.21.15.86:0/1657643096 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5579a6767290 0x5579a67e0060 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.400+0000 7ff993fff640 0 client.16224 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:25.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.400+0000 7ff9bd835740 1 --2- 172.21.15.86:0/1657643096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579a6755d40 0x5579a67dc3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.400+0000 7ff9bd835740 1 --2- 172.21.15.86:0/1657643096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579a6755390 0x5579a676ba50 unknown :-1 s=CLOSED pgs=689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.400+0000 7ff9bd835740 1 --2- 172.21.15.86:0/1657643096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579a674fcb0 0x5579a676b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.400+0000 7ff9bd835740 1 -- 172.21.15.86:0/1657643096 >> 172.21.15.86:0/1657643096 conn(0x5579a6762ef0 msgr2=0x5579a674e4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:25.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.400+0000 7ff9bd835740 1 -- 172.21.15.86:0/1657643096 shutdown_connections 2024-04-21T08:39:25.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.400+0000 7ff9bd835740 1 -- 172.21.15.86:0/1657643096 wait complete. 2024-04-21T08:39:25.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.400+0000 7ff9bd835740 20 client.16224 trim_cache size 0 max 16384 2024-04-21T08:39:25.406 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:39:25.406 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:39:25.406 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-21T08:39:25.406 INFO:teuthology.orchestra.run.smithi086.stdout: "93733a9d-4340-43f9-b190-52578cdd3b3f": { 2024-04-21T08:39:25.406 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-21T08:39:25.406 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "93733a9d-4340-43f9-b190-52578cdd3b3f", 2024-04-21T08:39:25.406 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-21T08:39:25.407 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:39:25.407 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:39:25.407 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:39:25.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.401+0000 7ff9bab6d640 1 -- 172.21.15.86:0/3317289482 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff9a40a3e40 msgr2=0x7ff9a40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:25.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.401+0000 7ff9bab6d640 1 --2- 172.21.15.86:0/3317289482 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff9a40a3e40 0x7ff9a40a6300 secure :-1 s=READY pgs=93 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac0efe40 tx=0x7ff9ac0095c0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.401+0000 7ff9bab6d640 1 -- 172.21.15.86:0/3317289482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9b410f130 msgr2=0x7ff9b41a6860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:25.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.401+0000 7ff9bab6d640 1 --2- 172.21.15.86:0/3317289482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9b410f130 0x7ff9b41a6860 secure :-1 s=READY pgs=686 cs=0 l=1 rev1=1 crypto rx=0x7ff9ac005e30 tx=0x7ff9ac034670 comp rx=0 tx=0).stop 2024-04-21T08:39:25.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.401+0000 7ff9bab6d640 1 -- 172.21.15.86:0/3317289482 shutdown_connections 2024-04-21T08:39:25.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.401+0000 7ff9bab6d640 1 --2- 172.21.15.86:0/3317289482 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff9a40a3e40 0x7ff9a40a6300 unknown :-1 s=CLOSED pgs=93 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.401+0000 7ff9bab6d640 1 --2- 172.21.15.86:0/3317289482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9b410f130 0x7ff9b41a6860 unknown :-1 s=CLOSED pgs=686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.401+0000 7ff9bab6d640 1 --2- 172.21.15.86:0/3317289482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9b4106150 0x7ff9b410ebf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.401+0000 7ff9bab6d640 1 --2- 172.21.15.86:0/3317289482 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff9b4101080 0x7ff9b410e6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:25.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.401+0000 7ff9bab6d640 1 -- 172.21.15.86:0/3317289482 >> 172.21.15.86:0/3317289482 conn(0x7ff9b4078000 msgr2=0x7ff9b40feed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:25.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.401+0000 7ff9bab6d640 1 -- 172.21.15.86:0/3317289482 shutdown_connections 2024-04-21T08:39:25.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:25.401+0000 7ff9bab6d640 1 -- 172.21.15.86:0/3317289482 wait complete. 2024-04-21T08:39:25.411 INFO:tasks.cephfs.filesystem:scrub status for tag:93733a9d-4340-43f9-b190-52578cdd3b3f - {'path': '/', 'tag': '93733a9d-4340-43f9-b190-52578cdd3b3f', 'options': 'recursive,force'} 2024-04-21T08:39:26.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:26 smithi086 ceph-mon[27939]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-21T08:39:26.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:26 smithi086 ceph-mon[27939]: scrub queued for path: / 2024-04-21T08:39:26.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:26 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:39:26.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:26 smithi086 ceph-mon[27939]: scrub summary: active paths [/] 2024-04-21T08:39:26.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:26 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:39:26.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:26 smithi088 ceph-mon[31259]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-21T08:39:26.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:26 smithi088 ceph-mon[31259]: scrub queued for path: / 2024-04-21T08:39:26.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:26 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:39:26.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:26 smithi088 ceph-mon[31259]: scrub summary: active paths [/] 2024-04-21T08:39:26.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:26 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:39:26.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:26 smithi179 ceph-mon[31829]: pgmap v22: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-21T08:39:26.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:26 smithi179 ceph-mon[31829]: scrub queued for path: / 2024-04-21T08:39:26.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:26 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:39:26.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:26 smithi179 ceph-mon[31829]: scrub summary: active paths [/] 2024-04-21T08:39:26.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:26 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:39:28.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:28 smithi086 ceph-mon[27939]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-21T08:39:28.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:28 smithi088 ceph-mon[31259]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-21T08:39:28.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:28 smithi179 ceph-mon[31829]: pgmap v23: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 255 B/s wr, 8 op/s 2024-04-21T08:39:30.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:30 smithi086 ceph-mon[27939]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-21T08:39:30.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:30 smithi086 ceph-mon[27939]: scrub summary: idle 2024-04-21T08:39:30.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:30 smithi088 ceph-mon[31259]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-21T08:39:30.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:30 smithi088 ceph-mon[31259]: scrub summary: idle 2024-04-21T08:39:30.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:30 smithi179 ceph-mon[31829]: pgmap v24: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-21T08:39:30.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:30 smithi179 ceph-mon[31829]: scrub summary: idle 2024-04-21T08:39:32.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:32 smithi088 ceph-mon[31259]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-21T08:39:32.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:32 smithi179 ceph-mon[31829]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-21T08:39:32.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:32 smithi086 ceph-mon[27939]: pgmap v25: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 255 B/s wr, 9 op/s 2024-04-21T08:39:33.349 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.348+0000 7f3ce7d96640 1 -- 172.21.15.86:0/958234359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ce01031e0 msgr2=0x7f3ce01035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:33.349 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.348+0000 7f3ce7d96640 1 --2- 172.21.15.86:0/958234359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ce01031e0 0x7f3ce01035c0 secure :-1 s=READY pgs=690 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc009900 tx=0x7f3cdc02bd60 comp rx=0 tx=0).stop 2024-04-21T08:39:33.349 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.348+0000 7f3ce7d96640 1 -- 172.21.15.86:0/958234359 shutdown_connections 2024-04-21T08:39:33.349 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.348+0000 7f3ce7d96640 1 --2- 172.21.15.86:0/958234359 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ce010c520 0x7f3ce010e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.349 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.348+0000 7f3ce7d96640 1 --2- 172.21.15.86:0/958234359 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ce0103b00 0x7f3ce010bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.349 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.348+0000 7f3ce7d96640 1 --2- 172.21.15.86:0/958234359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ce01031e0 0x7f3ce01035c0 unknown :-1 s=CLOSED pgs=690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.349 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.348+0000 7f3ce7d96640 1 -- 172.21.15.86:0/958234359 >> 172.21.15.86:0/958234359 conn(0x7f3ce0077fd0 msgr2=0x7f3ce00783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:33.350 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.348+0000 7f3ce7d96640 1 -- 172.21.15.86:0/958234359 shutdown_connections 2024-04-21T08:39:33.350 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.348+0000 7f3ce7d96640 1 -- 172.21.15.86:0/958234359 wait complete. 2024-04-21T08:39:33.350 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.348+0000 7f3ce7d96640 1 Processor -- start 2024-04-21T08:39:33.350 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.349+0000 7f3ce7d96640 1 -- start start 2024-04-21T08:39:33.350 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.349+0000 7f3ce7d96640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ce01031e0 0x7f3ce01986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.350 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.349+0000 7f3ce7d96640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ce0103b00 0x7f3ce0198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.350 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.349+0000 7f3ce7d96640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ce010c520 0x7f3ce019cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.350 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.349+0000 7f3ce7d96640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce01121b0 con 0x7f3ce0103b00 2024-04-21T08:39:33.350 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.349+0000 7f3ce7d96640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce0112030 con 0x7f3ce01031e0 2024-04-21T08:39:33.350 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.349+0000 7f3ce7d96640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce0112330 con 0x7f3ce010c520 2024-04-21T08:39:33.350 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.349+0000 7f3ce5331640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ce0103b00 0x7f3ce0198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.350 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.349+0000 7f3ce5331640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ce0103b00 0x7f3ce0198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59634/0 (socket says 172.21.15.86:59634) 2024-04-21T08:39:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.349+0000 7f3ce5331640 1 -- 172.21.15.86:0/1091952773 learned_addr learned my addr 172.21.15.86:0/1091952773 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.349+0000 7f3ce6333640 1 --2- 172.21.15.86:0/1091952773 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ce010c520 0x7f3ce019cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.349+0000 7f3ce5b32640 1 --2- 172.21.15.86:0/1091952773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ce01031e0 0x7f3ce01986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.349+0000 7f3ce5331640 1 -- 172.21.15.86:0/1091952773 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ce010c520 msgr2=0x7f3ce019cf70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.349+0000 7f3ce5331640 1 --2- 172.21.15.86:0/1091952773 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ce010c520 0x7f3ce019cf70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.349+0000 7f3ce5331640 1 -- 172.21.15.86:0/1091952773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ce01031e0 msgr2=0x7f3ce01986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.349+0000 7f3ce5331640 1 --2- 172.21.15.86:0/1091952773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ce01031e0 0x7f3ce01986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.349+0000 7f3ce5331640 1 -- 172.21.15.86:0/1091952773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3ce019d6a0 con 0x7f3ce0103b00 2024-04-21T08:39:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.349+0000 7f3ce6333640 1 --2- 172.21.15.86:0/1091952773 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ce010c520 0x7f3ce019cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.349+0000 7f3ce5331640 1 --2- 172.21.15.86:0/1091952773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ce0103b00 0x7f3ce0198bf0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f3cd000e9f0 tx=0x7f3cd000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.350+0000 7f3cd6ffd640 1 -- 172.21.15.86:0/1091952773 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd000cd80 con 0x7f3ce0103b00 2024-04-21T08:39:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.350+0000 7f3cd6ffd640 1 -- 172.21.15.86:0/1091952773 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cd00108e0 con 0x7f3ce0103b00 2024-04-21T08:39:33.351 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.350+0000 7f3ce7d96640 1 -- 172.21.15.86:0/1091952773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3ce019d990 con 0x7f3ce0103b00 2024-04-21T08:39:33.352 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.350+0000 7f3cd6ffd640 1 -- 172.21.15.86:0/1091952773 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd0018980 con 0x7f3ce0103b00 2024-04-21T08:39:33.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.350+0000 7f3ce7d96640 1 -- 172.21.15.86:0/1091952773 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3ce0100d60 con 0x7f3ce0103b00 2024-04-21T08:39:33.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.353+0000 7f3cea989740 1 Processor -- start 2024-04-21T08:39:33.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.353+0000 7f3cea989740 1 -- start start 2024-04-21T08:39:33.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.353+0000 7f3cea989740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629465ea8d0 0x5629465e9620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.353+0000 7f3cea989740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629465e9b60 0x5629465e9fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.353+0000 7f3cea989740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629465f30e0 0x5629465f54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.353+0000 7f3cea989740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56294654e8d0 con 0x5629465ea8d0 2024-04-21T08:39:33.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.353+0000 7f3cea989740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5629464ebe80 con 0x5629465e9b60 2024-04-21T08:39:33.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.353+0000 7f3cea989740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56294654a890 con 0x5629465f30e0 2024-04-21T08:39:33.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.353+0000 7f3cd6ffd640 1 -- 172.21.15.86:0/1091952773 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f3cd0018b20 con 0x7f3ce0103b00 2024-04-21T08:39:33.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.353+0000 7f3ce6333640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629465ea8d0 0x5629465e9620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.353+0000 7f3ce6333640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629465ea8d0 0x5629465e9620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59650/0 (socket says 172.21.15.86:59650) 2024-04-21T08:39:33.355 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.353+0000 7f3ce6333640 1 -- 172.21.15.86:0/2934218594 learned_addr learned my addr 172.21.15.86:0/2934218594 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:33.355 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.354+0000 7f3ce5b32640 1 --2- 172.21.15.86:0/2934218594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629465e9b60 0x5629465e9fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.355 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.354+0000 7f3ce6333640 1 -- 172.21.15.86:0/2934218594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629465f30e0 msgr2=0x5629465f54d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:33.355 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.354+0000 7f3ce6333640 1 --2- 172.21.15.86:0/2934218594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629465f30e0 0x5629465f54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.355 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.354+0000 7f3ce6333640 1 -- 172.21.15.86:0/2934218594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629465e9b60 msgr2=0x5629465e9fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:33.355 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.354+0000 7f3ce6333640 1 --2- 172.21.15.86:0/2934218594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629465e9b60 0x5629465e9fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.355 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.354+0000 7f3ce6333640 1 -- 172.21.15.86:0/2934218594 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56294650d320 con 0x5629465ea8d0 2024-04-21T08:39:33.355 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.354+0000 7f3cd6ffd640 1 --2- 172.21.15.86:0/1091952773 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f3ccc0a3f60 0x7f3ccc0a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.355 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.354+0000 7f3ce6333640 1 --2- 172.21.15.86:0/2934218594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629465ea8d0 0x5629465e9620 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8011790 tx=0x7f3cd8011c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3067d15d9e26127a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:33.355 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.354+0000 7f3ccbfff640 1 -- 172.21.15.86:0/2934218594 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd8003c30 con 0x5629465ea8d0 2024-04-21T08:39:33.355 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.354+0000 7f3cd6ffd640 1 -- 172.21.15.86:0/1091952773 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f3cd00d10e0 con 0x7f3ce0103b00 2024-04-21T08:39:33.355 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.354+0000 7f3ccbfff640 1 -- 172.21.15.86:0/2934218594 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cd800fe50 con 0x5629465ea8d0 2024-04-21T08:39:33.355 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.354+0000 7f3ce5b32640 1 --2- 172.21.15.86:0/1091952773 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f3ccc0a3f60 0x7f3ccc0a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.354+0000 7f3ccbfff640 1 -- 172.21.15.86:0/2934218594 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd80193e0 con 0x5629465ea8d0 2024-04-21T08:39:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.355+0000 7f3cea989740 1 -- 172.21.15.86:0/2934218594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629465ea8d0 msgr2=0x5629465e9620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.355+0000 7f3cea989740 1 --2- 172.21.15.86:0/2934218594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629465ea8d0 0x5629465e9620 secure :-1 s=READY pgs=692 cs=0 l=1 rev1=1 crypto rx=0x7f3cd8011790 tx=0x7f3cd8011c50 comp rx=0 tx=0).stop 2024-04-21T08:39:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.355+0000 7f3ce5b32640 1 --2- 172.21.15.86:0/1091952773 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f3ccc0a3f60 0x7f3ccc0a6420 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc006010 tx=0x7f3cdc005e40 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.355+0000 7f3cea989740 1 -- 172.21.15.86:0/2934218594 shutdown_connections 2024-04-21T08:39:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.355+0000 7f3cea989740 1 --2- 172.21.15.86:0/2934218594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629465f30e0 0x5629465f54d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.355+0000 7f3cea989740 1 --2- 172.21.15.86:0/2934218594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629465e9b60 0x5629465e9fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.355+0000 7f3cea989740 1 --2- 172.21.15.86:0/2934218594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629465ea8d0 0x5629465e9620 unknown :-1 s=CLOSED pgs=692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.355+0000 7f3cea989740 1 -- 172.21.15.86:0/2934218594 >> 172.21.15.86:0/2934218594 conn(0x5629465b0c90 msgr2=0x5629465e4850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.355+0000 7f3cea989740 1 -- 172.21.15.86:0/2934218594 shutdown_connections 2024-04-21T08:39:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.355+0000 7f3cea989740 1 -- 172.21.15.86:0/2934218594 wait complete. 2024-04-21T08:39:33.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.356+0000 7f3cea989740 1 -- start start 2024-04-21T08:39:33.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.356+0000 7f3cea989740 1 Processor -- start 2024-04-21T08:39:33.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.356+0000 7f3cea989740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629465e9b60 0x5629465f2ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.356+0000 7f3cea989740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629465ea8d0 0x5629465efe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.356+0000 7f3cea989740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629465f30e0 0x5629465f0460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.357+0000 7f3cea989740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5629465f5e10 con 0x5629465e9b60 2024-04-21T08:39:33.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.357+0000 7f3ce5b32640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629465e9b60 0x5629465f2ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.357+0000 7f3cea989740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5629464d6eb0 con 0x5629465ea8d0 2024-04-21T08:39:33.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.357+0000 7f3ce5b32640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629465e9b60 0x5629465f2ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59662/0 (socket says 172.21.15.86:59662) 2024-04-21T08:39:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.357+0000 7f3ce5b32640 1 -- 172.21.15.86:0/43684626 learned_addr learned my addr 172.21.15.86:0/43684626 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.357+0000 7f3cea989740 1 -- 172.21.15.86:0/43684626 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5629465f5f90 con 0x5629465f30e0 2024-04-21T08:39:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.357+0000 7f3ce5331640 1 --2- 172.21.15.86:0/43684626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629465ea8d0 0x5629465efe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.357+0000 7f3ce6333640 1 --2- 172.21.15.86:0/43684626 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629465f30e0 0x5629465f0460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.357+0000 7f3cea989740 10 client.0 _ll_register_callbacks cb 0x562946503b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.357+0000 7f3ce5331640 1 -- 172.21.15.86:0/43684626 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629465f30e0 msgr2=0x5629465f0460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.357+0000 7f3ce5331640 1 --2- 172.21.15.86:0/43684626 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629465f30e0 0x5629465f0460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.357+0000 7f3ce5331640 1 -- 172.21.15.86:0/43684626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629465e9b60 msgr2=0x5629465f2ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.357+0000 7f3ce5331640 1 --2- 172.21.15.86:0/43684626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629465e9b60 0x5629465f2ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.357+0000 7f3ce5331640 1 -- 172.21.15.86:0/43684626 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562946261210 con 0x5629465ea8d0 2024-04-21T08:39:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.357+0000 7f3ce5b32640 1 --2- 172.21.15.86:0/43684626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629465e9b60 0x5629465f2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.357+0000 7f3ce6333640 1 --2- 172.21.15.86:0/43684626 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629465f30e0 0x5629465f0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.357+0000 7f3ce5331640 1 --2- 172.21.15.86:0/43684626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629465ea8d0 0x5629465efe50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3cd00ce340 tx=0x7f3cd009ccb0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9fc3e0ac6a99a6c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.357+0000 7f3cc97fa640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-04-21T08:39:33.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.358+0000 7f3cc97fa640 1 -- 172.21.15.86:0/43684626 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd00d32d0 con 0x5629465ea8d0 2024-04-21T08:39:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.358+0000 7f3cea989740 1 -- 172.21.15.86:0/43684626 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cd800fe50 con 0x5629465ea8d0 2024-04-21T08:39:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.358+0000 7f3cc97fa640 1 -- 172.21.15.86:0/43684626 <== mon.2 v2:172.21.15.88:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f3cd0010430 con 0x5629465ea8d0 2024-04-21T08:39:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.358+0000 7f3cc97fa640 1 -- 172.21.15.86:0/43684626 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3cd0010630 con 0x5629465ea8d0 2024-04-21T08:39:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.358+0000 7f3cea989740 10 client.25520 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.358+0000 7f3cea989740 1 -- 172.21.15.86:0/43684626 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56294632d620 con 0x5629465ea8d0 2024-04-21T08:39:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.358+0000 7f3cc97fa640 1 -- 172.21.15.86:0/43684626 <== mon.2 v2:172.21.15.88:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cd800fe50 con 0x5629465ea8d0 2024-04-21T08:39:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.358+0000 7f3cc97fa640 1 -- 172.21.15.86:0/43684626 <== mon.2 v2:172.21.15.88:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3cd00988c0 con 0x5629465ea8d0 2024-04-21T08:39:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.358+0000 7f3cea989740 10 client.25520 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.358+0000 7f3cea989740 10 client.25520 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.358+0000 7f3cea989740 20 client.25520 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.358+0000 7f3cea989740 1 --2- 172.21.15.86:0/43684626 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x562946603150 0x562946605540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.358+0000 7f3cea989740 4 client.25520 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:33.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.358+0000 7f3cea989740 1 -- 172.21.15.86:0/43684626 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562946562880 con 0x562946603150 2024-04-21T08:39:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.359+0000 7f3ce5b32640 1 --2- 172.21.15.86:0/43684626 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x562946603150 0x562946605540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.359+0000 7f3ce5b32640 1 --2- 172.21.15.86:0/43684626 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x562946603150 0x562946605540 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=27ccb0fd3fbd4c41 server_cookie=a482697cc9afe96c in_seq=0 out_seq=0 2024-04-21T08:39:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.359+0000 7f3cc97fa640 10 client.25520 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.360+0000 7f3cc97fa640 1 -- 172.21.15.86:0/43684626 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562946562880 con 0x562946603150 2024-04-21T08:39:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.360+0000 7f3cc97fa640 10 client.25520 handle_command_reply: tid=0 2024-04-21T08:39:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.360+0000 7f3cea989740 1 client.25520 shutdown 2024-04-21T08:39:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.360+0000 7f3cea989740 1 -- 172.21.15.86:0/43684626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629465ea8d0 msgr2=0x5629465efe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.360+0000 7f3cea989740 1 --2- 172.21.15.86:0/43684626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629465ea8d0 0x5629465efe50 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7f3cd00ce340 tx=0x7f3cd009ccb0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.360+0000 7f3cea989740 1 -- 172.21.15.86:0/43684626 shutdown_connections 2024-04-21T08:39:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.360+0000 7f3cea989740 1 --2- 172.21.15.86:0/43684626 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x562946603150 0x562946605540 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.360+0000 7f3cea989740 1 --2- 172.21.15.86:0/43684626 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629465f30e0 0x5629465f0460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.360+0000 7f3cea989740 1 --2- 172.21.15.86:0/43684626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629465ea8d0 0x5629465efe50 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.360+0000 7f3cea989740 1 --2- 172.21.15.86:0/43684626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629465e9b60 0x5629465f2ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.365 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.360+0000 7f3cea989740 1 -- 172.21.15.86:0/43684626 >> 172.21.15.86:0/43684626 conn(0x5629465b0c90 msgr2=0x5629465e65a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:33.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.360+0000 7f3cc97fa640 0 client.25520 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:33.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.360+0000 7f3cea989740 1 -- 172.21.15.86:0/43684626 shutdown_connections 2024-04-21T08:39:33.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.360+0000 7f3cea989740 1 -- 172.21.15.86:0/43684626 wait complete. 2024-04-21T08:39:33.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.360+0000 7f3cea989740 20 client.25520 trim_cache size 0 max 16384 2024-04-21T08:39:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.377+0000 7f3cea989740 1 Processor -- start 2024-04-21T08:39:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.377+0000 7f3cea989740 1 -- start start 2024-04-21T08:39:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.377+0000 7f3cea989740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629465e9b60 0x5629465efdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.377+0000 7f3cea989740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629465ea8d0 0x5629465fed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.377+0000 7f3cea989740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629465f30e0 0x5629465f2580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.377+0000 7f3cea989740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce01b14d0 con 0x5629465e9b60 2024-04-21T08:39:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.377+0000 7f3cea989740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce01b3010 con 0x5629465f30e0 2024-04-21T08:39:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.377+0000 7f3cea989740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce01b1ba0 con 0x5629465ea8d0 2024-04-21T08:39:33.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.377+0000 7f3ce5b32640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629465e9b60 0x5629465efdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.377+0000 7f3ce5b32640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629465e9b60 0x5629465efdf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59670/0 (socket says 172.21.15.86:59670) 2024-04-21T08:39:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.377+0000 7f3ce5b32640 1 -- 172.21.15.86:0/746564762 learned_addr learned my addr 172.21.15.86:0/746564762 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.377+0000 7f3ce6333640 1 --2- 172.21.15.86:0/746564762 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629465f30e0 0x5629465f2580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.377+0000 7f3ce5331640 1 --2- 172.21.15.86:0/746564762 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629465ea8d0 0x5629465fed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.377+0000 7f3ce5b32640 1 -- 172.21.15.86:0/746564762 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629465ea8d0 msgr2=0x5629465fed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.377+0000 7f3ce5b32640 1 --2- 172.21.15.86:0/746564762 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629465ea8d0 0x5629465fed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.377+0000 7f3ce5b32640 1 -- 172.21.15.86:0/746564762 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629465f30e0 msgr2=0x5629465f2580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.377+0000 7f3ce5b32640 1 --2- 172.21.15.86:0/746564762 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629465f30e0 0x5629465f2580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.377+0000 7f3ce5b32640 1 -- 172.21.15.86:0/746564762 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629465e8e30 con 0x5629465e9b60 2024-04-21T08:39:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.377+0000 7f3ce5331640 1 --2- 172.21.15.86:0/746564762 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629465ea8d0 0x5629465fed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.378+0000 7f3ce5b32640 1 --2- 172.21.15.86:0/746564762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629465e9b60 0x5629465efdf0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc036040 tx=0x7f3cdc040000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=318b483d983b1065 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:33.378 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.378+0000 7f3ccb7fe640 1 -- 172.21.15.86:0/746564762 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc005040 con 0x5629465e9b60 2024-04-21T08:39:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.378+0000 7f3ccb7fe640 1 -- 172.21.15.86:0/746564762 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cdc050250 con 0x5629465e9b60 2024-04-21T08:39:33.379 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.378+0000 7f3ccb7fe640 1 -- 172.21.15.86:0/746564762 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cdc0048e0 con 0x5629465e9b60 2024-04-21T08:39:33.381 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.379+0000 7f3cea989740 1 -- 172.21.15.86:0/746564762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629465e9b60 msgr2=0x5629465efdf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:33.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.379+0000 7f3cea989740 1 --2- 172.21.15.86:0/746564762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629465e9b60 0x5629465efdf0 secure :-1 s=READY pgs=693 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc036040 tx=0x7f3cdc040000 comp rx=0 tx=0).stop 2024-04-21T08:39:33.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.379+0000 7f3cea989740 1 -- 172.21.15.86:0/746564762 shutdown_connections 2024-04-21T08:39:33.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.379+0000 7f3cea989740 1 --2- 172.21.15.86:0/746564762 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629465f30e0 0x5629465f2580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.379+0000 7f3cea989740 1 --2- 172.21.15.86:0/746564762 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629465ea8d0 0x5629465fed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.379+0000 7f3cea989740 1 --2- 172.21.15.86:0/746564762 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629465e9b60 0x5629465efdf0 unknown :-1 s=CLOSED pgs=693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.379+0000 7f3cea989740 1 -- 172.21.15.86:0/746564762 >> 172.21.15.86:0/746564762 conn(0x5629465b0c90 msgr2=0x5629465ef6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:33.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.379+0000 7f3cea989740 1 -- 172.21.15.86:0/746564762 shutdown_connections 2024-04-21T08:39:33.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.379+0000 7f3cea989740 1 -- 172.21.15.86:0/746564762 wait complete. 2024-04-21T08:39:33.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.379+0000 7f3cea989740 1 -- start start 2024-04-21T08:39:33.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.379+0000 7f3cea989740 1 Processor -- start 2024-04-21T08:39:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.379+0000 7f3cea989740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629465e9b60 0x5629465fb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.379+0000 7f3cea989740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629465ea8d0 0x5629465fb760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.379+0000 7f3cea989740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629465f30e0 0x5629465ec0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.379+0000 7f3cea989740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce01b2940 con 0x5629465ea8d0 2024-04-21T08:39:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.379+0000 7f3cea989740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce01b2270 con 0x5629465e9b60 2024-04-21T08:39:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.379+0000 7f3cea989740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3ce01af490 con 0x5629465f30e0 2024-04-21T08:39:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.380+0000 7f3cea989740 10 client.0 _ll_register_callbacks cb 0x7f3cd0009700 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.380+0000 7f3ce5331640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629465ea8d0 0x5629465fb760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.380+0000 7f3ce5331640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629465ea8d0 0x5629465fb760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:59672/0 (socket says 172.21.15.86:59672) 2024-04-21T08:39:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.380+0000 7f3ce5331640 1 -- 172.21.15.86:0/4240024094 learned_addr learned my addr 172.21.15.86:0/4240024094 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:33.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.380+0000 7f3ce5b32640 1 --2- 172.21.15.86:0/4240024094 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629465e9b60 0x5629465fb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.380+0000 7f3ce6333640 1 --2- 172.21.15.86:0/4240024094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629465f30e0 0x5629465ec0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.380+0000 7f3ce5331640 1 -- 172.21.15.86:0/4240024094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629465f30e0 msgr2=0x5629465ec0e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.380+0000 7f3ce5331640 1 --2- 172.21.15.86:0/4240024094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629465f30e0 0x5629465ec0e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.380+0000 7f3ce5331640 1 -- 172.21.15.86:0/4240024094 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629465e9b60 msgr2=0x5629465fb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.380+0000 7f3ce5331640 1 --2- 172.21.15.86:0/4240024094 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629465e9b60 0x5629465fb220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.380+0000 7f3ce5331640 1 -- 172.21.15.86:0/4240024094 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629465ec620 con 0x5629465ea8d0 2024-04-21T08:39:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.380+0000 7f3ce6333640 1 --2- 172.21.15.86:0/4240024094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629465f30e0 0x5629465ec0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.380+0000 7f3ce5331640 1 --2- 172.21.15.86:0/4240024094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629465ea8d0 0x5629465fb760 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f3cd000c6a0 tx=0x7f3cd000cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b38228257e2d90a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.380+0000 7f3cc97fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.380+0000 7f3cc97fa640 1 -- 172.21.15.86:0/4240024094 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3cd009c4d0 con 0x5629465ea8d0 2024-04-21T08:39:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.380+0000 7f3cc97fa640 1 -- 172.21.15.86:0/4240024094 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f3cd009bad0 con 0x5629465ea8d0 2024-04-21T08:39:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.380+0000 7f3cea989740 1 -- 172.21.15.86:0/4240024094 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3cdc050250 con 0x5629465ea8d0 2024-04-21T08:39:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.381+0000 7f3cc97fa640 1 -- 172.21.15.86:0/4240024094 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3cd0098e10 con 0x5629465ea8d0 2024-04-21T08:39:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.381+0000 7f3cea989740 10 client.16242 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:33.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.381+0000 7f3cea989740 1 -- 172.21.15.86:0/4240024094 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562946668ce0 con 0x5629465ea8d0 2024-04-21T08:39:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.381+0000 7f3cc97fa640 1 -- 172.21.15.86:0/4240024094 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3cdc050250 con 0x5629465ea8d0 2024-04-21T08:39:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.381+0000 7f3cc97fa640 1 -- 172.21.15.86:0/4240024094 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3cd00a2200 con 0x5629465ea8d0 2024-04-21T08:39:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.381+0000 7f3cea989740 10 client.16242 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.381+0000 7f3cea989740 10 client.16242 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.381+0000 7f3cea989740 20 client.16242 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.381+0000 7f3cea989740 1 --2- 172.21.15.86:0/4240024094 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x562946669460 0x56294666b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.381+0000 7f3cea989740 4 client.16242 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:39:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.381+0000 7f3cea989740 1 -- 172.21.15.86:0/4240024094 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5629464ee3d0 con 0x562946669460 2024-04-21T08:39:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.381+0000 7f3ce5b32640 1 --2- 172.21.15.86:0/4240024094 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x562946669460 0x56294666b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.382+0000 7f3cc97fa640 10 client.16242 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:33.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.382+0000 7f3ce5b32640 1 --2- 172.21.15.86:0/4240024094 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x562946669460 0x56294666b880 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22a5c8d6d9b27857 server_cookie=aee21c8ec1fbed44 in_seq=0 out_seq=0 2024-04-21T08:39:33.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.386+0000 7f3cc97fa640 1 -- 172.21.15.86:0/4240024094 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5629464ee3d0 con 0x562946669460 2024-04-21T08:39:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.386+0000 7f3cc97fa640 10 client.16242 handle_command_reply: tid=0 2024-04-21T08:39:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.386+0000 7f3cea989740 1 client.16242 shutdown 2024-04-21T08:39:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.386+0000 7f3cea989740 1 -- 172.21.15.86:0/4240024094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629465ea8d0 msgr2=0x5629465fb760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.386+0000 7f3cea989740 1 --2- 172.21.15.86:0/4240024094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629465ea8d0 0x5629465fb760 secure :-1 s=READY pgs=694 cs=0 l=1 rev1=1 crypto rx=0x7f3cd000c6a0 tx=0x7f3cd000cf20 comp rx=0 tx=0).stop 2024-04-21T08:39:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.386+0000 7f3cea989740 1 -- 172.21.15.86:0/4240024094 shutdown_connections 2024-04-21T08:39:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.386+0000 7f3cea989740 1 --2- 172.21.15.86:0/4240024094 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x562946669460 0x56294666b880 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.386+0000 7f3cea989740 1 --2- 172.21.15.86:0/4240024094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629465f30e0 0x5629465ec0e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.386+0000 7f3cea989740 1 --2- 172.21.15.86:0/4240024094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629465ea8d0 0x5629465fb760 unknown :-1 s=CLOSED pgs=694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.386+0000 7f3cea989740 1 --2- 172.21.15.86:0/4240024094 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629465e9b60 0x5629465fb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.386+0000 7f3cea989740 1 -- 172.21.15.86:0/4240024094 >> 172.21.15.86:0/4240024094 conn(0x5629465b0c90 msgr2=0x5629465ee740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.386+0000 7f3cc97fa640 0 client.16242 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.386+0000 7f3cea989740 1 -- 172.21.15.86:0/4240024094 shutdown_connections 2024-04-21T08:39:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.386+0000 7f3cea989740 1 -- 172.21.15.86:0/4240024094 wait complete. 2024-04-21T08:39:33.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.386+0000 7f3cea989740 20 client.16242 trim_cache size 0 max 16384 2024-04-21T08:39:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:39:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:39:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:39:33.388 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:39:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.387+0000 7f3ce7d96640 1 -- 172.21.15.86:0/1091952773 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f3ccc0a3f60 msgr2=0x7f3ccc0a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.387+0000 7f3ce7d96640 1 --2- 172.21.15.86:0/1091952773 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f3ccc0a3f60 0x7f3ccc0a6420 secure :-1 s=READY pgs=94 cs=0 l=1 rev1=1 crypto rx=0x7f3cdc006010 tx=0x7f3cdc005e40 comp rx=0 tx=0).stop 2024-04-21T08:39:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.387+0000 7f3ce7d96640 1 -- 172.21.15.86:0/1091952773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ce0103b00 msgr2=0x7f3ce0198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.387+0000 7f3ce7d96640 1 --2- 172.21.15.86:0/1091952773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ce0103b00 0x7f3ce0198bf0 secure :-1 s=READY pgs=691 cs=0 l=1 rev1=1 crypto rx=0x7f3cd000e9f0 tx=0x7f3cd000eeb0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.387+0000 7f3ce7d96640 1 -- 172.21.15.86:0/1091952773 shutdown_connections 2024-04-21T08:39:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.387+0000 7f3ce7d96640 1 --2- 172.21.15.86:0/1091952773 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f3ccc0a3f60 0x7f3ccc0a6420 unknown :-1 s=CLOSED pgs=94 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.387+0000 7f3ce7d96640 1 --2- 172.21.15.86:0/1091952773 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3ce010c520 0x7f3ce019cf70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.387+0000 7f3ce7d96640 1 --2- 172.21.15.86:0/1091952773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3ce0103b00 0x7f3ce0198bf0 unknown :-1 s=CLOSED pgs=691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.387+0000 7f3ce7d96640 1 --2- 172.21.15.86:0/1091952773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3ce01031e0 0x7f3ce01986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:33.389 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.387+0000 7f3ce7d96640 1 -- 172.21.15.86:0/1091952773 >> 172.21.15.86:0/1091952773 conn(0x7f3ce0077fd0 msgr2=0x7f3ce00fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.387+0000 7f3ce7d96640 1 -- 172.21.15.86:0/1091952773 shutdown_connections 2024-04-21T08:39:33.390 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:33.387+0000 7f3ce7d96640 1 -- 172.21.15.86:0/1091952773 wait complete. 2024-04-21T08:39:34.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:34 smithi088 ceph-mon[31259]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-21T08:39:34.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:34 smithi179 ceph-mon[31829]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-21T08:39:34.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:34 smithi086 ceph-mon[27939]: pgmap v26: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 255 B/s wr, 13 op/s 2024-04-21T08:39:36.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:36 smithi088 ceph-mon[31259]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-04-21T08:39:36.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:36 smithi179 ceph-mon[31829]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-04-21T08:39:36.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:36 smithi086 ceph-mon[27939]: pgmap v27: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-04-21T08:39:38.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:38 smithi088 ceph-mon[31259]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-04-21T08:39:38.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:38 smithi179 ceph-mon[31829]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-04-21T08:39:38.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:38 smithi086 ceph-mon[27939]: pgmap v28: 129 pgs: 129 active+clean; 1.3 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 9 op/s 2024-04-21T08:39:40.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:40 smithi088 ceph-mon[31259]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-04-21T08:39:40.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:40 smithi179 ceph-mon[31829]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-04-21T08:39:40.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:40 smithi086 ceph-mon[27939]: pgmap v29: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 8.7 KiB/s rd, 1.5 KiB/s wr, 14 op/s 2024-04-21T08:39:42.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:42 smithi088 ceph-mon[31259]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-21T08:39:42.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:42 smithi179 ceph-mon[31829]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-21T08:39:42.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:42 smithi086 ceph-mon[27939]: pgmap v30: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-21T08:39:44.617 INFO:tasks.workunit.client.0.smithi086.stderr:Note: switching to '4f6bc5273643aaf5660909e85400647a8951c10b'. 2024-04-21T08:39:44.617 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-04-21T08:39:44.618 INFO:tasks.workunit.client.0.smithi086.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2024-04-21T08:39:44.618 INFO:tasks.workunit.client.0.smithi086.stderr:changes and commit them, and you can discard any commits you make in this 2024-04-21T08:39:44.618 INFO:tasks.workunit.client.0.smithi086.stderr:state without impacting any branches by switching back to a branch. 2024-04-21T08:39:44.618 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-04-21T08:39:44.618 INFO:tasks.workunit.client.0.smithi086.stderr:If you want to create a new branch to retain commits you create, you may 2024-04-21T08:39:44.618 INFO:tasks.workunit.client.0.smithi086.stderr:do so (now or later) by using -c with the switch command. Example: 2024-04-21T08:39:44.618 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-04-21T08:39:44.618 INFO:tasks.workunit.client.0.smithi086.stderr: git switch -c 2024-04-21T08:39:44.618 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-04-21T08:39:44.618 INFO:tasks.workunit.client.0.smithi086.stderr:Or undo this operation with: 2024-04-21T08:39:44.618 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-04-21T08:39:44.618 INFO:tasks.workunit.client.0.smithi086.stderr: git switch - 2024-04-21T08:39:44.618 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-04-21T08:39:44.619 INFO:tasks.workunit.client.0.smithi086.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2024-04-21T08:39:44.619 INFO:tasks.workunit.client.0.smithi086.stderr: 2024-04-21T08:39:44.619 INFO:tasks.workunit.client.0.smithi086.stderr:HEAD is now at 4f6bc527364 use ubuntu for fs/workload tests 2024-04-21T08:39:44.621 DEBUG:teuthology.orchestra.run.smithi086:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2024-04-21T08:39:44.661 INFO:tasks.workunit.client.0.smithi086.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2024-04-21T08:39:44.663 INFO:tasks.workunit.client.0.smithi086.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-21T08:39:44.663 INFO:tasks.workunit.client.0.smithi086.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2024-04-21T08:39:44.668 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:44 smithi086 ceph-mon[27939]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-04-21T08:39:44.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:44 smithi088 ceph-mon[31259]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-04-21T08:39:44.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:44 smithi179 ceph-mon[31829]: pgmap v31: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 7.5 KiB/s rd, 1.5 KiB/s wr, 13 op/s 2024-04-21T08:39:45.215 INFO:tasks.workunit.client.0.smithi086.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2024-04-21T08:39:45.268 INFO:tasks.workunit.client.0.smithi086.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2024-04-21T08:39:45.308 INFO:tasks.workunit.client.0.smithi086.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2024-04-21T08:39:45.309 INFO:tasks.workunit.client.0.smithi086.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-21T08:39:45.309 INFO:tasks.workunit.client.0.smithi086.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2024-04-21T08:39:45.347 INFO:tasks.workunit.client.0.smithi086.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2024-04-21T08:39:45.351 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-21T08:39:45.351 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2024-04-21T08:39:45.411 INFO:tasks.workunit:Running workunits matching fs/test_o_trunc.sh on client.0... 2024-04-21T08:39:45.412 INFO:tasks.workunit:Running workunit fs/test_o_trunc.sh... 2024-04-21T08:39:45.413 DEBUG:teuthology.orchestra.run.smithi086:workunit test fs/test_o_trunc.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=4f6bc5273643aaf5660909e85400647a8951c10b TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-04-21T08:39:45.472 INFO:tasks.workunit.client.0.smithi086.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc.sh 2024-04-21T08:39:45.472 INFO:tasks.workunit.client.0.smithi086.stderr:+ mydir=/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs 2024-04-21T08:39:45.472 INFO:tasks.workunit.client.0.smithi086.stderr:+ /home/ubuntu/cephtest/clone.client.0/qa/workunits/fs/test_o_trunc trunc.foo 600 2024-04-21T08:39:46.480 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.478+0000 7f21debc7640 1 -- 172.21.15.86:0/4003739168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21d8101070 msgr2=0x7f21d8106c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:46.480 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.478+0000 7f21debc7640 1 --2- 172.21.15.86:0/4003739168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21d8101070 0x7f21d8106c00 secure :-1 s=READY pgs=695 cs=0 l=1 rev1=1 crypto rx=0x7f21c4009900 tx=0x7f21c402bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:46.480 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.478+0000 7f21debc7640 1 -- 172.21.15.86:0/4003739168 shutdown_connections 2024-04-21T08:39:46.480 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.478+0000 7f21debc7640 1 --2- 172.21.15.86:0/4003739168 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21d8107330 0x7f21d810eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.478+0000 7f21debc7640 1 --2- 172.21.15.86:0/4003739168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21d8101070 0x7f21d8106c00 unknown :-1 s=CLOSED pgs=695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.478+0000 7f21debc7640 1 --2- 172.21.15.86:0/4003739168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21d81006c0 0x7f21d8100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.478+0000 7f21debc7640 1 -- 172.21.15.86:0/4003739168 >> 172.21.15.86:0/4003739168 conn(0x7f21d8077390 msgr2=0x7f21d80fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.478+0000 7f21debc7640 1 -- 172.21.15.86:0/4003739168 shutdown_connections 2024-04-21T08:39:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.478+0000 7f21debc7640 1 -- 172.21.15.86:0/4003739168 wait complete. 2024-04-21T08:39:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.479+0000 7f21debc7640 1 Processor -- start 2024-04-21T08:39:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.479+0000 7f21debc7640 1 -- start start 2024-04-21T08:39:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.479+0000 7f21debc7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21d81006c0 0x7f21d8198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.479+0000 7f21debc7640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21d8107330 0x7f21d8198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.479+0000 7f21debc7640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21d819cfe0 0x7f21d819d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:46.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.479+0000 7f21debc7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f21d8112200 con 0x7f21d81006c0 2024-04-21T08:39:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.479+0000 7f21debc7640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f21d8112080 con 0x7f21d8107330 2024-04-21T08:39:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.479+0000 7f21debc7640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f21d8112380 con 0x7f21d819cfe0 2024-04-21T08:39:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.479+0000 7f21dc963640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21d81006c0 0x7f21d8198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.479+0000 7f21dc963640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21d81006c0 0x7f21d8198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36366/0 (socket says 172.21.15.86:36366) 2024-04-21T08:39:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.479+0000 7f21dc963640 1 -- 172.21.15.86:0/2001087781 learned_addr learned my addr 172.21.15.86:0/2001087781 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.479+0000 7f21d7fff640 1 --2- 172.21.15.86:0/2001087781 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21d8107330 0x7f21d8198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.479+0000 7f21dd164640 1 --2- 172.21.15.86:0/2001087781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21d819cfe0 0x7f21d819d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.479+0000 7f21dc963640 1 -- 172.21.15.86:0/2001087781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21d819cfe0 msgr2=0x7f21d819d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.479+0000 7f21dc963640 1 --2- 172.21.15.86:0/2001087781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21d819cfe0 0x7f21d819d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.479+0000 7f21dc963640 1 -- 172.21.15.86:0/2001087781 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21d8107330 msgr2=0x7f21d8198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.479+0000 7f21dc963640 1 --2- 172.21.15.86:0/2001087781 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21d8107330 0x7f21d8198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:46.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.479+0000 7f21dc963640 1 -- 172.21.15.86:0/2001087781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f21d819dad0 con 0x7f21d81006c0 2024-04-21T08:39:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.479+0000 7f21dc963640 1 --2- 172.21.15.86:0/2001087781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21d81006c0 0x7f21d8198720 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00c970 tx=0x7f21cc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.479+0000 7f21d5ffb640 1 -- 172.21.15.86:0/2001087781 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc007bb0 con 0x7f21d81006c0 2024-04-21T08:39:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.480+0000 7f21debc7640 1 -- 172.21.15.86:0/2001087781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f21d819ddc0 con 0x7f21d81006c0 2024-04-21T08:39:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.480+0000 7f21d5ffb640 1 -- 172.21.15.86:0/2001087781 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21cc007d50 con 0x7f21d81006c0 2024-04-21T08:39:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.480+0000 7f21d5ffb640 1 -- 172.21.15.86:0/2001087781 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc00f450 con 0x7f21d81006c0 2024-04-21T08:39:46.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.480+0000 7f21debc7640 1 -- 172.21.15.86:0/2001087781 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f21d81a5580 con 0x7f21d81006c0 2024-04-21T08:39:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.482+0000 7f21d5ffb640 1 -- 172.21.15.86:0/2001087781 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f21cc004f20 con 0x7f21d81006c0 2024-04-21T08:39:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.483+0000 7f21e17bb740 1 Processor -- start 2024-04-21T08:39:46.489 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.483+0000 7f21e17bb740 1 -- start start 2024-04-21T08:39:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.483+0000 7f21d5ffb640 1 --2- 172.21.15.86:0/2001087781 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f21c80a3ee0 0x7f21c80a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.483+0000 7f21e17bb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1cc0488d0 0x55c1cc047620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.483+0000 7f21e17bb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c1cc047b60 0x55c1cc047fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.483+0000 7f21e17bb740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c1cc0510e0 0x55c1cc0534d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.483+0000 7f21e17bb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c1cbfac8d0 con 0x55c1cc0488d0 2024-04-21T08:39:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.483+0000 7f21e17bb740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c1cbf49e80 con 0x55c1cc0510e0 2024-04-21T08:39:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.483+0000 7f21e17bb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c1cbfa8890 con 0x55c1cc047b60 2024-04-21T08:39:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.483+0000 7f21dd164640 1 --2- 172.21.15.86:0/2001087781 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f21c80a3ee0 0x7f21c80a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.483+0000 7f21dc963640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1cc0488d0 0x55c1cc047620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.483+0000 7f21d5ffb640 1 -- 172.21.15.86:0/2001087781 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f21cc0d0ba0 con 0x7f21d81006c0 2024-04-21T08:39:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.483+0000 7f21dc963640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1cc0488d0 0x55c1cc047620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36378/0 (socket says 172.21.15.86:36378) 2024-04-21T08:39:46.490 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.483+0000 7f21dc963640 1 -- 172.21.15.86:0/3096871640 learned_addr learned my addr 172.21.15.86:0/3096871640 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.483+0000 7f21dd164640 1 --2- 172.21.15.86:0/3096871640 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c1cc0510e0 0x55c1cc0534d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.483+0000 7f21d7fff640 1 --2- 172.21.15.86:0/3096871640 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c1cc047b60 0x55c1cc047fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.484+0000 7f21dc963640 1 -- 172.21.15.86:0/3096871640 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c1cc047b60 msgr2=0x55c1cc047fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.484+0000 7f21dc963640 1 --2- 172.21.15.86:0/3096871640 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c1cc047b60 0x55c1cc047fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.484+0000 7f21dc963640 1 -- 172.21.15.86:0/3096871640 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c1cc0510e0 msgr2=0x55c1cc0534d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.484+0000 7f21dc963640 1 --2- 172.21.15.86:0/3096871640 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c1cc0510e0 0x55c1cc0534d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.484+0000 7f21dc963640 1 -- 172.21.15.86:0/3096871640 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1cbf6b320 con 0x55c1cc0488d0 2024-04-21T08:39:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.484+0000 7f21dd164640 1 --2- 172.21.15.86:0/2001087781 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f21c80a3ee0 0x7f21c80a63a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f21d0005fd0 tx=0x7f21d0009d50 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.484+0000 7f21dc963640 1 --2- 172.21.15.86:0/3096871640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1cc0488d0 0x55c1cc047620 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f21cc098f50 tx=0x7f21cc09bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acf4a283ade3031c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.484+0000 7f21c2ffd640 1 -- 172.21.15.86:0/3096871640 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc01d070 con 0x55c1cc0488d0 2024-04-21T08:39:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.484+0000 7f21c2ffd640 1 -- 172.21.15.86:0/3096871640 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21cc09c580 con 0x55c1cc0488d0 2024-04-21T08:39:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.484+0000 7f21c2ffd640 1 -- 172.21.15.86:0/3096871640 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc098bf0 con 0x55c1cc0488d0 2024-04-21T08:39:46.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.485+0000 7f21e17bb740 1 -- 172.21.15.86:0/3096871640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1cc0488d0 msgr2=0x55c1cc047620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.485+0000 7f21e17bb740 1 --2- 172.21.15.86:0/3096871640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1cc0488d0 0x55c1cc047620 secure :-1 s=READY pgs=697 cs=0 l=1 rev1=1 crypto rx=0x7f21cc098f50 tx=0x7f21cc09bee0 comp rx=0 tx=0).stop 2024-04-21T08:39:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.485+0000 7f21e17bb740 1 -- 172.21.15.86:0/3096871640 shutdown_connections 2024-04-21T08:39:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.485+0000 7f21e17bb740 1 --2- 172.21.15.86:0/3096871640 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c1cc0510e0 0x55c1cc0534d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.485+0000 7f21e17bb740 1 --2- 172.21.15.86:0/3096871640 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c1cc047b60 0x55c1cc047fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.485+0000 7f21e17bb740 1 --2- 172.21.15.86:0/3096871640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1cc0488d0 0x55c1cc047620 unknown :-1 s=CLOSED pgs=697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.485+0000 7f21e17bb740 1 -- 172.21.15.86:0/3096871640 >> 172.21.15.86:0/3096871640 conn(0x55c1cc00ec90 msgr2=0x55c1cc042850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.485+0000 7f21e17bb740 1 -- 172.21.15.86:0/3096871640 shutdown_connections 2024-04-21T08:39:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.485+0000 7f21e17bb740 1 -- 172.21.15.86:0/3096871640 wait complete. 2024-04-21T08:39:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.485+0000 7f21e17bb740 1 -- start start 2024-04-21T08:39:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.485+0000 7f21e17bb740 1 Processor -- start 2024-04-21T08:39:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.486+0000 7f21e17bb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c1cc047b60 0x55c1cc050b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.486+0000 7f21e17bb740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c1cc0488d0 0x55c1cc04dea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:46.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.486+0000 7f21e17bb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1cc0510e0 0x55c1cc04e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.486+0000 7f21e17bb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c1cc053db0 con 0x55c1cc0510e0 2024-04-21T08:39:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.486+0000 7f21e17bb740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c1cbf34eb0 con 0x55c1cc0488d0 2024-04-21T08:39:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.486+0000 7f21e17bb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c1cc053f30 con 0x55c1cc047b60 2024-04-21T08:39:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.486+0000 7f21e17bb740 10 client.0 _ll_register_callbacks cb 0x55c1cbf61b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.486+0000 7f21dd164640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1cc0510e0 0x55c1cc04e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.486+0000 7f21dd164640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1cc0510e0 0x55c1cc04e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36386/0 (socket says 172.21.15.86:36386) 2024-04-21T08:39:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.486+0000 7f21d7fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c1cc0488d0 0x55c1cc04dea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.486+0000 7f21dd164640 1 -- 172.21.15.86:0/154715430 learned_addr learned my addr 172.21.15.86:0/154715430 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.486+0000 7f21dc963640 1 --2- 172.21.15.86:0/154715430 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c1cc047b60 0x55c1cc050b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.486+0000 7f21dd164640 1 -- 172.21.15.86:0/154715430 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c1cc047b60 msgr2=0x55c1cc050b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.486+0000 7f21dd164640 1 --2- 172.21.15.86:0/154715430 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c1cc047b60 0x55c1cc050b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.486+0000 7f21dd164640 1 -- 172.21.15.86:0/154715430 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c1cc0488d0 msgr2=0x55c1cc04dea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.486+0000 7f21dd164640 1 --2- 172.21.15.86:0/154715430 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c1cc0488d0 0x55c1cc04dea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:46.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.486+0000 7f21dd164640 1 -- 172.21.15.86:0/154715430 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1cbcbf210 con 0x55c1cc0510e0 2024-04-21T08:39:46.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.486+0000 7f21dd164640 1 --2- 172.21.15.86:0/154715430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1cc0510e0 0x55c1cc04e5e0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f21d0005c00 tx=0x7f21d0009240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e112087149a352b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:46.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.486+0000 7f21b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:46.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.487+0000 7f21b7fff640 1 -- 172.21.15.86:0/154715430 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21d0025030 con 0x55c1cc0510e0 2024-04-21T08:39:46.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.487+0000 7f21b7fff640 1 -- 172.21.15.86:0/154715430 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f21d0004250 con 0x55c1cc0510e0 2024-04-21T08:39:46.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.487+0000 7f21e17bb740 1 -- 172.21.15.86:0/154715430 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21cc09c580 con 0x55c1cc0510e0 2024-04-21T08:39:46.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.487+0000 7f21b7fff640 1 -- 172.21.15.86:0/154715430 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f21d0032050 con 0x55c1cc0510e0 2024-04-21T08:39:46.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.487+0000 7f21e17bb740 10 client.16254 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:46.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.487+0000 7f21e17bb740 1 -- 172.21.15.86:0/154715430 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c1cbd8b620 con 0x55c1cc0510e0 2024-04-21T08:39:46.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.487+0000 7f21b7fff640 1 -- 172.21.15.86:0/154715430 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21cc09c580 con 0x55c1cc0510e0 2024-04-21T08:39:46.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.487+0000 7f21b7fff640 1 -- 172.21.15.86:0/154715430 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f21d00219e0 con 0x55c1cc0510e0 2024-04-21T08:39:46.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.487+0000 7f21e17bb740 10 client.16254 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:46.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.487+0000 7f21e17bb740 10 client.16254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:46.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.487+0000 7f21e17bb740 20 client.16254 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.487+0000 7f21e17bb740 1 --2- 172.21.15.86:0/154715430 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c1cc061100 0x55c1cc0634f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.487+0000 7f21e17bb740 4 client.16254 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.487+0000 7f21e17bb740 1 -- 172.21.15.86:0/154715430 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1cbfc0880 con 0x55c1cc061100 2024-04-21T08:39:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.487+0000 7f21dc963640 1 --2- 172.21.15.86:0/154715430 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c1cc061100 0x55c1cc0634f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.488+0000 7f21b7fff640 10 client.16254 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.488+0000 7f21dc963640 1 --2- 172.21.15.86:0/154715430 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c1cc061100 0x55c1cc0634f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f8922378a268fd16 server_cookie=b80ff1e55d72b2 in_seq=0 out_seq=0 2024-04-21T08:39:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.489+0000 7f21b7fff640 1 -- 172.21.15.86:0/154715430 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c1cbfc0880 con 0x55c1cc061100 2024-04-21T08:39:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.489+0000 7f21b7fff640 10 client.16254 handle_command_reply: tid=0 2024-04-21T08:39:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.489+0000 7f21e17bb740 1 client.16254 shutdown 2024-04-21T08:39:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.489+0000 7f21e17bb740 1 -- 172.21.15.86:0/154715430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1cc0510e0 msgr2=0x55c1cc04e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.489+0000 7f21e17bb740 1 --2- 172.21.15.86:0/154715430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1cc0510e0 0x55c1cc04e5e0 secure :-1 s=READY pgs=698 cs=0 l=1 rev1=1 crypto rx=0x7f21d0005c00 tx=0x7f21d0009240 comp rx=0 tx=0).stop 2024-04-21T08:39:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.489+0000 7f21e17bb740 1 -- 172.21.15.86:0/154715430 shutdown_connections 2024-04-21T08:39:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.489+0000 7f21e17bb740 1 --2- 172.21.15.86:0/154715430 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c1cc061100 0x55c1cc0634f0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:46.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.489+0000 7f21e17bb740 1 --2- 172.21.15.86:0/154715430 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1cc0510e0 0x55c1cc04e5e0 unknown :-1 s=CLOSED pgs=698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:46.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.489+0000 7f21e17bb740 1 --2- 172.21.15.86:0/154715430 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c1cc0488d0 0x55c1cc04dea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:46.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.489+0000 7f21b7fff640 0 client.16254 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:46.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.489+0000 7f21e17bb740 1 --2- 172.21.15.86:0/154715430 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c1cc047b60 0x55c1cc050b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:46.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.489+0000 7f21e17bb740 1 -- 172.21.15.86:0/154715430 >> 172.21.15.86:0/154715430 conn(0x55c1cc00ec90 msgr2=0x55c1cc0445a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:46.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.489+0000 7f21e17bb740 1 -- 172.21.15.86:0/154715430 shutdown_connections 2024-04-21T08:39:46.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.489+0000 7f21e17bb740 1 -- 172.21.15.86:0/154715430 wait complete. 2024-04-21T08:39:46.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.489+0000 7f21e17bb740 20 client.16254 trim_cache size 0 max 16384 2024-04-21T08:39:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.504+0000 7f21e17bb740 1 Processor -- start 2024-04-21T08:39:46.509 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.504+0000 7f21e17bb740 1 -- start start 2024-04-21T08:39:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.504+0000 7f21e17bb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1cc047b60 0x55c1cc05ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.504+0000 7f21e17bb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c1cc0488d0 0x55c1cc068dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.504+0000 7f21e17bb740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c1cc0510e0 0x55c1cc069300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.504+0000 7f21e17bb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f21d81af1b0 con 0x55c1cc047b60 2024-04-21T08:39:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.504+0000 7f21e17bb740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f21d81b0cf0 con 0x55c1cc0510e0 2024-04-21T08:39:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.504+0000 7f21e17bb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f21d81af880 con 0x55c1cc0488d0 2024-04-21T08:39:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.504+0000 7f21dc963640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1cc047b60 0x55c1cc05ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.504+0000 7f21dc963640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1cc047b60 0x55c1cc05ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36402/0 (socket says 172.21.15.86:36402) 2024-04-21T08:39:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.504+0000 7f21dc963640 1 -- 172.21.15.86:0/3865601954 learned_addr learned my addr 172.21.15.86:0/3865601954 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.504+0000 7f21d7fff640 1 --2- 172.21.15.86:0/3865601954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c1cc0488d0 0x55c1cc068dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:46.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.504+0000 7f21dc963640 1 -- 172.21.15.86:0/3865601954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c1cc0488d0 msgr2=0x55c1cc068dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.504+0000 7f21dc963640 1 --2- 172.21.15.86:0/3865601954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c1cc0488d0 0x55c1cc068dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.504+0000 7f21dd164640 1 --2- 172.21.15.86:0/3865601954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c1cc0510e0 0x55c1cc069300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.504+0000 7f21dc963640 1 -- 172.21.15.86:0/3865601954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c1cc0510e0 msgr2=0x55c1cc069300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.505+0000 7f21dc963640 1 --2- 172.21.15.86:0/3865601954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c1cc0510e0 0x55c1cc069300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.505+0000 7f21dc963640 1 -- 172.21.15.86:0/3865601954 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c1cc046e30 con 0x55c1cc047b60 2024-04-21T08:39:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.505+0000 7f21dd164640 1 --2- 172.21.15.86:0/3865601954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c1cc0510e0 0x55c1cc069300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.505+0000 7f21dc963640 1 --2- 172.21.15.86:0/3865601954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1cc047b60 0x55c1cc05ccb0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0a4cc0 tx=0x7f21cc087040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85c9bb4ae72997cf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.505+0000 7f21c27fc640 1 -- 172.21.15.86:0/3865601954 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc01d040 con 0x55c1cc047b60 2024-04-21T08:39:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.505+0000 7f21c27fc640 1 -- 172.21.15.86:0/3865601954 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21cc0cdd40 con 0x55c1cc047b60 2024-04-21T08:39:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.505+0000 7f21c27fc640 1 -- 172.21.15.86:0/3865601954 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc0047a0 con 0x55c1cc047b60 2024-04-21T08:39:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.505+0000 7f21e17bb740 1 -- 172.21.15.86:0/3865601954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1cc047b60 msgr2=0x55c1cc05ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:46.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.505+0000 7f21e17bb740 1 --2- 172.21.15.86:0/3865601954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1cc047b60 0x55c1cc05ccb0 secure :-1 s=READY pgs=699 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0a4cc0 tx=0x7f21cc087040 comp rx=0 tx=0).stop 2024-04-21T08:39:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.505+0000 7f21e17bb740 1 -- 172.21.15.86:0/3865601954 shutdown_connections 2024-04-21T08:39:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.505+0000 7f21e17bb740 1 --2- 172.21.15.86:0/3865601954 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c1cc0510e0 0x55c1cc069300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.505+0000 7f21e17bb740 1 --2- 172.21.15.86:0/3865601954 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c1cc0488d0 0x55c1cc068dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.505+0000 7f21e17bb740 1 --2- 172.21.15.86:0/3865601954 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1cc047b60 0x55c1cc05ccb0 unknown :-1 s=CLOSED pgs=699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.505+0000 7f21e17bb740 1 -- 172.21.15.86:0/3865601954 >> 172.21.15.86:0/3865601954 conn(0x55c1cc00ec90 msgr2=0x55c1cc049080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.505+0000 7f21e17bb740 1 -- 172.21.15.86:0/3865601954 shutdown_connections 2024-04-21T08:39:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.506+0000 7f21e17bb740 1 -- 172.21.15.86:0/3865601954 wait complete. 2024-04-21T08:39:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.506+0000 7f21e17bb740 1 -- start start 2024-04-21T08:39:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.506+0000 7f21e17bb740 1 Processor -- start 2024-04-21T08:39:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.506+0000 7f21e17bb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1cc047b60 0x55c1cc04acf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:46.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.506+0000 7f21e17bb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c1cc0488d0 0x55c1cc04b230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.506+0000 7f21e17bb740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c1cc0510e0 0x55c1cc04b770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.506+0000 7f21e17bb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f21d81b0620 con 0x55c1cc047b60 2024-04-21T08:39:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.506+0000 7f21e17bb740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f21d81aff50 con 0x55c1cc0510e0 2024-04-21T08:39:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.506+0000 7f21e17bb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f21d81ad170 con 0x55c1cc0488d0 2024-04-21T08:39:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.506+0000 7f21e17bb740 10 client.0 _ll_register_callbacks cb 0x7f21d00097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.506+0000 7f21dc963640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1cc047b60 0x55c1cc04acf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.506+0000 7f21dc963640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1cc047b60 0x55c1cc04acf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36406/0 (socket says 172.21.15.86:36406) 2024-04-21T08:39:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.506+0000 7f21dc963640 1 -- 172.21.15.86:0/2902207124 learned_addr learned my addr 172.21.15.86:0/2902207124 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.506+0000 7f21d7fff640 1 --2- 172.21.15.86:0/2902207124 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c1cc0488d0 0x55c1cc04b230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.507+0000 7f21dd164640 1 --2- 172.21.15.86:0/2902207124 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c1cc0510e0 0x55c1cc04b770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.507+0000 7f21dc963640 1 -- 172.21.15.86:0/2902207124 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c1cc0488d0 msgr2=0x55c1cc04b230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.507+0000 7f21dc963640 1 --2- 172.21.15.86:0/2902207124 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c1cc0488d0 0x55c1cc04b230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.507+0000 7f21dc963640 1 -- 172.21.15.86:0/2902207124 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c1cc0510e0 msgr2=0x55c1cc04b770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:46.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.507+0000 7f21dc963640 1 --2- 172.21.15.86:0/2902207124 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c1cc0510e0 0x55c1cc04b770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.507+0000 7f21dc963640 1 -- 172.21.15.86:0/2902207124 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c1cc0c6790 con 0x55c1cc047b60 2024-04-21T08:39:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.507+0000 7f21dd164640 1 --2- 172.21.15.86:0/2902207124 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c1cc0510e0 0x55c1cc04b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.507+0000 7f21dc963640 1 --2- 172.21.15.86:0/2902207124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1cc047b60 0x55c1cc04acf0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0989e0 tx=0x7f21cc09c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3156849688dc0ce9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.507+0000 7f21b7fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.507+0000 7f21b7fff640 1 -- 172.21.15.86:0/2902207124 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f21cc01d070 con 0x55c1cc047b60 2024-04-21T08:39:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.507+0000 7f21b7fff640 1 -- 172.21.15.86:0/2902207124 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f21cc098bf0 con 0x55c1cc047b60 2024-04-21T08:39:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.507+0000 7f21e17bb740 1 -- 172.21.15.86:0/2902207124 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f21cc0cdd40 con 0x55c1cc047b60 2024-04-21T08:39:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.507+0000 7f21b7fff640 1 -- 172.21.15.86:0/2902207124 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f21cc09cc80 con 0x55c1cc047b60 2024-04-21T08:39:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.507+0000 7f21e17bb740 10 client.16260 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.507+0000 7f21e17bb740 1 -- 172.21.15.86:0/2902207124 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c1cc0c79f0 con 0x55c1cc047b60 2024-04-21T08:39:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.507+0000 7f21b7fff640 1 -- 172.21.15.86:0/2902207124 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f21cc0cdd40 con 0x55c1cc047b60 2024-04-21T08:39:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.507+0000 7f21b7fff640 1 -- 172.21.15.86:0/2902207124 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f21cc09b9d0 con 0x55c1cc047b60 2024-04-21T08:39:46.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.507+0000 7f21e17bb740 10 client.16260 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.507+0000 7f21e17bb740 10 client.16260 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.507+0000 7f21e17bb740 20 client.16260 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.507+0000 7f21e17bb740 1 --2- 172.21.15.86:0/2902207124 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c1cc0c7c20 0x55c1cc0ca010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.507+0000 7f21e17bb740 4 client.16260 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:39:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.507+0000 7f21e17bb740 1 -- 172.21.15.86:0/2902207124 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c1cbf4c3d0 con 0x55c1cc0c7c20 2024-04-21T08:39:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.508+0000 7f21d7fff640 1 --2- 172.21.15.86:0/2902207124 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c1cc0c7c20 0x55c1cc0ca010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.508+0000 7f21b7fff640 10 client.16260 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:46.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.508+0000 7f21d7fff640 1 --2- 172.21.15.86:0/2902207124 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c1cc0c7c20 0x55c1cc0ca010 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=30c3fde963969751 server_cookie=6d4530337dac01b7 in_seq=0 out_seq=0 2024-04-21T08:39:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.517+0000 7f21b7fff640 1 -- 172.21.15.86:0/2902207124 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c1cbf4c3d0 con 0x55c1cc0c7c20 2024-04-21T08:39:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.517+0000 7f21b7fff640 10 client.16260 handle_command_reply: tid=0 2024-04-21T08:39:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.517+0000 7f21e17bb740 1 client.16260 shutdown 2024-04-21T08:39:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.517+0000 7f21e17bb740 1 -- 172.21.15.86:0/2902207124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1cc047b60 msgr2=0x55c1cc04acf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.517+0000 7f21e17bb740 1 --2- 172.21.15.86:0/2902207124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1cc047b60 0x55c1cc04acf0 secure :-1 s=READY pgs=700 cs=0 l=1 rev1=1 crypto rx=0x7f21cc0989e0 tx=0x7f21cc09c6c0 comp rx=0 tx=0).stop 2024-04-21T08:39:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.517+0000 7f21e17bb740 1 -- 172.21.15.86:0/2902207124 shutdown_connections 2024-04-21T08:39:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.517+0000 7f21e17bb740 1 --2- 172.21.15.86:0/2902207124 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c1cc0c7c20 0x55c1cc0ca010 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.517+0000 7f21e17bb740 1 --2- 172.21.15.86:0/2902207124 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c1cc0510e0 0x55c1cc04b770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:46.518 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.517+0000 7f21e17bb740 1 --2- 172.21.15.86:0/2902207124 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c1cc0488d0 0x55c1cc04b230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.517+0000 7f21e17bb740 1 --2- 172.21.15.86:0/2902207124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1cc047b60 0x55c1cc04acf0 unknown :-1 s=CLOSED pgs=700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.517+0000 7f21b7fff640 0 client.16260 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.517+0000 7f21e17bb740 1 -- 172.21.15.86:0/2902207124 >> 172.21.15.86:0/2902207124 conn(0x55c1cc00ec90 msgr2=0x55c1cc043b90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.517+0000 7f21e17bb740 1 -- 172.21.15.86:0/2902207124 shutdown_connections 2024-04-21T08:39:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.517+0000 7f21e17bb740 1 -- 172.21.15.86:0/2902207124 wait complete. 2024-04-21T08:39:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.517+0000 7f21e17bb740 20 client.16260 trim_cache size 0 max 16384 2024-04-21T08:39:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:39:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:39:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:39:46.519 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:39:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.518+0000 7f21debc7640 1 -- 172.21.15.86:0/2001087781 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f21c80a3ee0 msgr2=0x7f21c80a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:46.520 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.518+0000 7f21debc7640 1 --2- 172.21.15.86:0/2001087781 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f21c80a3ee0 0x7f21c80a63a0 secure :-1 s=READY pgs=95 cs=0 l=1 rev1=1 crypto rx=0x7f21d0005fd0 tx=0x7f21d0009d50 comp rx=0 tx=0).stop 2024-04-21T08:39:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.518+0000 7f21debc7640 1 -- 172.21.15.86:0/2001087781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21d81006c0 msgr2=0x7f21d8198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.518+0000 7f21debc7640 1 --2- 172.21.15.86:0/2001087781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21d81006c0 0x7f21d8198720 secure :-1 s=READY pgs=696 cs=0 l=1 rev1=1 crypto rx=0x7f21cc00c970 tx=0x7f21cc00ce30 comp rx=0 tx=0).stop 2024-04-21T08:39:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.518+0000 7f21debc7640 1 -- 172.21.15.86:0/2001087781 shutdown_connections 2024-04-21T08:39:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.518+0000 7f21debc7640 1 --2- 172.21.15.86:0/2001087781 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f21c80a3ee0 0x7f21c80a63a0 unknown :-1 s=CLOSED pgs=95 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.518+0000 7f21debc7640 1 --2- 172.21.15.86:0/2001087781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f21d819cfe0 0x7f21d819d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.518+0000 7f21debc7640 1 --2- 172.21.15.86:0/2001087781 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f21d8107330 0x7f21d8198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.518+0000 7f21debc7640 1 --2- 172.21.15.86:0/2001087781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f21d81006c0 0x7f21d8198720 unknown :-1 s=CLOSED pgs=696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.518+0000 7f21debc7640 1 -- 172.21.15.86:0/2001087781 >> 172.21.15.86:0/2001087781 conn(0x7f21d8077390 msgr2=0x7f21d81054f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.518+0000 7f21debc7640 1 -- 172.21.15.86:0/2001087781 shutdown_connections 2024-04-21T08:39:46.521 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:46.518+0000 7f21debc7640 1 -- 172.21.15.86:0/2001087781 wait complete. 2024-04-21T08:39:46.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:46 smithi088 ceph-mon[31259]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-21T08:39:46.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:46 smithi179 ceph-mon[31829]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-21T08:39:46.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:46 smithi086 ceph-mon[27939]: pgmap v32: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-21T08:39:48.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:48 smithi088 ceph-mon[31259]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-21T08:39:48.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:48 smithi179 ceph-mon[31829]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-21T08:39:48.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:48 smithi086 ceph-mon[27939]: pgmap v33: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 5.4 KiB/s rd, 1.5 KiB/s wr, 9 op/s 2024-04-21T08:39:50.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:50 smithi088 ceph-mon[31259]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.5 KiB/s wr, 16 op/s 2024-04-21T08:39:50.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:50 smithi179 ceph-mon[31829]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.5 KiB/s wr, 16 op/s 2024-04-21T08:39:50.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:50 smithi086 ceph-mon[27939]: pgmap v34: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.5 KiB/s wr, 16 op/s 2024-04-21T08:39:52.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:52 smithi088 ceph-mon[31259]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.0 KiB/s wr, 12 op/s 2024-04-21T08:39:52.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:52 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:39:52.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:52 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:39:52.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:52 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:39:52.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:52 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:39:52.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:52 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:39:52.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:52 smithi179 ceph-mon[31829]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.0 KiB/s wr, 12 op/s 2024-04-21T08:39:52.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:52 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:39:52.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:52 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:39:52.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:52 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:39:52.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:52 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:39:52.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:52 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:39:52.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:52 smithi086 ceph-mon[27939]: pgmap v35: 129 pgs: 129 active+clean; 1.4 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 5.0 KiB/s wr, 12 op/s 2024-04-21T08:39:52.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:52 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:39:52.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:52 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:39:52.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:52 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:39:52.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:52 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:39:52.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:52 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:39:53.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:53 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:39:53.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:53 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:39:53.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:53 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:39:53.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:53 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:39:53.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:53 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:39:53.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:53 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:39:53.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:53 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:39:53.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:53 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:39:53.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:53 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:39:54.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:54 smithi088 ceph-mon[31259]: pgmap v36: 129 pgs: 129 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-04-21T08:39:54.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:54 smithi179 ceph-mon[31829]: pgmap v36: 129 pgs: 129 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-04-21T08:39:54.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:54 smithi086 ceph-mon[27939]: pgmap v36: 129 pgs: 129 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 12 KiB/s wr, 18 op/s 2024-04-21T08:39:55.412 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:39:55.412 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:39:55.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.525+0000 7f3899c81640 1 -- 172.21.15.86:0/2235774240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38941031f0 msgr2=0x7f38941035d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:55.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.525+0000 7f3899c81640 1 --2- 172.21.15.86:0/2235774240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38941031f0 0x7f38941035d0 secure :-1 s=READY pgs=701 cs=0 l=1 rev1=1 crypto rx=0x7f3884009900 tx=0x7f388402bd60 comp rx=0 tx=0).stop 2024-04-21T08:39:55.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.525+0000 7f3899c81640 1 -- 172.21.15.86:0/2235774240 shutdown_connections 2024-04-21T08:39:55.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.525+0000 7f3899c81640 1 --2- 172.21.15.86:0/2235774240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f389410c530 0x7f389410e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.525+0000 7f3899c81640 1 --2- 172.21.15.86:0/2235774240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3894103b10 0x7f389410bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.525+0000 7f3899c81640 1 --2- 172.21.15.86:0/2235774240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38941031f0 0x7f38941035d0 unknown :-1 s=CLOSED pgs=701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.525+0000 7f3899c81640 1 -- 172.21.15.86:0/2235774240 >> 172.21.15.86:0/2235774240 conn(0x7f38940771d0 msgr2=0x7f38940775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:55.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.526+0000 7f3899c81640 1 -- 172.21.15.86:0/2235774240 shutdown_connections 2024-04-21T08:39:55.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.526+0000 7f3899c81640 1 -- 172.21.15.86:0/2235774240 wait complete. 2024-04-21T08:39:55.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.526+0000 7f3899c81640 1 Processor -- start 2024-04-21T08:39:55.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.526+0000 7f3899c81640 1 -- start start 2024-04-21T08:39:55.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.526+0000 7f3899c81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38941031f0 0x7f3894100e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:55.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.526+0000 7f3899c81640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3894103b10 0x7f3894101380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:55.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.527+0000 7f3899c81640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f389410c530 0x7f38941018c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:55.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.527+0000 7f3899c81640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3894112210 con 0x7f38941031f0 2024-04-21T08:39:55.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.527+0000 7f3899c81640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3894112090 con 0x7f3894103b10 2024-04-21T08:39:55.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.527+0000 7f3899c81640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3894112390 con 0x7f389410c530 2024-04-21T08:39:55.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.527+0000 7f38937fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38941031f0 0x7f3894100e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:55.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.527+0000 7f38937fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38941031f0 0x7f3894100e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45364/0 (socket says 172.21.15.86:45364) 2024-04-21T08:39:55.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.527+0000 7f38937fe640 1 -- 172.21.15.86:0/991391411 learned_addr learned my addr 172.21.15.86:0/991391411 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:55.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.527+0000 7f3892ffd640 1 --2- 172.21.15.86:0/991391411 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3894103b10 0x7f3894101380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:55.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.527+0000 7f38937fe640 1 -- 172.21.15.86:0/991391411 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f389410c530 msgr2=0x7f38941018c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:55.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.527+0000 7f38937fe640 1 --2- 172.21.15.86:0/991391411 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f389410c530 0x7f38941018c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.527+0000 7f38937fe640 1 -- 172.21.15.86:0/991391411 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3894103b10 msgr2=0x7f3894101380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:55.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.527+0000 7f38937fe640 1 --2- 172.21.15.86:0/991391411 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3894103b10 0x7f3894101380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.527+0000 7f38937fe640 1 -- 172.21.15.86:0/991391411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38941a6bd0 con 0x7f38941031f0 2024-04-21T08:39:55.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.527+0000 7f3892ffd640 1 --2- 172.21.15.86:0/991391411 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3894103b10 0x7f3894101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:55.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.527+0000 7f38937fe640 1 --2- 172.21.15.86:0/991391411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38941031f0 0x7f3894100e40 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f3884000c00 tx=0x7f388402d920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:55.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.527+0000 7f3890ff9640 1 -- 172.21.15.86:0/991391411 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884043020 con 0x7f38941031f0 2024-04-21T08:39:55.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.527+0000 7f3890ff9640 1 -- 172.21.15.86:0/991391411 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f388402dde0 con 0x7f38941031f0 2024-04-21T08:39:55.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.527+0000 7f3899c81640 1 -- 172.21.15.86:0/991391411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38941a6e60 con 0x7f38941031f0 2024-04-21T08:39:55.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.527+0000 7f3890ff9640 1 -- 172.21.15.86:0/991391411 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3884003b80 con 0x7f38941031f0 2024-04-21T08:39:55.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.527+0000 7f3899c81640 1 -- 172.21.15.86:0/991391411 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38941a7320 con 0x7f38941031f0 2024-04-21T08:39:55.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.530+0000 7f3890ff9640 1 -- 172.21.15.86:0/991391411 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f3884046020 con 0x7f38941031f0 2024-04-21T08:39:55.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.530+0000 7f389c875740 1 Processor -- start 2024-04-21T08:39:55.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.530+0000 7f389c875740 1 -- start start 2024-04-21T08:39:55.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.530+0000 7f389c875740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc6cf4580 0x555fc6cf4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:55.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.530+0000 7f3890ff9640 1 --2- 172.21.15.86:0/991391411 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f387c0a3e70 0x7f387c0a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:55.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.530+0000 7f389c875740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555fc6cf4ea0 0x555fc6cfd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:55.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.530+0000 7f389c875740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fc6cfda80 0x555fc6cffe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:55.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.530+0000 7f389c875740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555fc6c591e0 con 0x555fc6cf4580 2024-04-21T08:39:55.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.530+0000 7f389c875740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x555fc6bf6390 con 0x555fc6cf4ea0 2024-04-21T08:39:55.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.530+0000 7f389c875740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555fc6c551a0 con 0x555fc6cfda80 2024-04-21T08:39:55.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.530+0000 7f38937fe640 1 --2- 172.21.15.86:0/991391411 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f387c0a3e70 0x7f387c0a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:55.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.530+0000 7f3893fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc6cf4580 0x555fc6cf4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:55.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.530+0000 7f3893fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc6cf4580 0x555fc6cf4960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45380/0 (socket says 172.21.15.86:45380) 2024-04-21T08:39:55.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.530+0000 7f3893fff640 1 -- 172.21.15.86:0/548015361 learned_addr learned my addr 172.21.15.86:0/548015361 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:55.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.530+0000 7f3890ff9640 1 -- 172.21.15.86:0/991391411 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f38840f2ef0 con 0x7f38941031f0 2024-04-21T08:39:55.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.530+0000 7f3892ffd640 1 --2- 172.21.15.86:0/548015361 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555fc6cf4ea0 0x555fc6cfd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:55.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.531+0000 7f3893fff640 1 --2- 172.21.15.86:0/548015361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fc6cfda80 0x555fc6cffe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:55.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.531+0000 7f3893fff640 1 -- 172.21.15.86:0/548015361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fc6cfda80 msgr2=0x555fc6cffe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:55.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.531+0000 7f3893fff640 1 --2- 172.21.15.86:0/548015361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fc6cfda80 0x555fc6cffe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.531+0000 7f3893fff640 1 -- 172.21.15.86:0/548015361 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555fc6cf4ea0 msgr2=0x555fc6cfd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:55.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.531+0000 7f3893fff640 1 --2- 172.21.15.86:0/548015361 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555fc6cf4ea0 0x555fc6cfd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.531+0000 7f3893fff640 1 -- 172.21.15.86:0/548015361 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fc6a357d0 con 0x555fc6cf4580 2024-04-21T08:39:55.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.531+0000 7f38937fe640 1 --2- 172.21.15.86:0/991391411 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f387c0a3e70 0x7f387c0a6330 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f38840f02c0 tx=0x7f3884089480 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:55.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.531+0000 7f3893fff640 1 --2- 172.21.15.86:0/548015361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc6cf4580 0x555fc6cf4960 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f388c009fd0 tx=0x7f388c009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40fb4acb5fec52e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:55.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.531+0000 7f3879ffb640 1 -- 172.21.15.86:0/548015361 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c0150f0 con 0x555fc6cf4580 2024-04-21T08:39:55.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.531+0000 7f3879ffb640 1 -- 172.21.15.86:0/548015361 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f388c010550 con 0x555fc6cf4580 2024-04-21T08:39:55.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.531+0000 7f3879ffb640 1 -- 172.21.15.86:0/548015361 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c005280 con 0x555fc6cf4580 2024-04-21T08:39:55.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.532+0000 7f389c875740 1 -- 172.21.15.86:0/548015361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc6cf4580 msgr2=0x555fc6cf4960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:55.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.532+0000 7f389c875740 1 --2- 172.21.15.86:0/548015361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc6cf4580 0x555fc6cf4960 secure :-1 s=READY pgs=703 cs=0 l=1 rev1=1 crypto rx=0x7f388c009fd0 tx=0x7f388c009710 comp rx=0 tx=0).stop 2024-04-21T08:39:55.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.532+0000 7f389c875740 1 -- 172.21.15.86:0/548015361 shutdown_connections 2024-04-21T08:39:55.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.532+0000 7f389c875740 1 --2- 172.21.15.86:0/548015361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fc6cfda80 0x555fc6cffe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.532+0000 7f389c875740 1 --2- 172.21.15.86:0/548015361 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555fc6cf4ea0 0x555fc6cfd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.532+0000 7f389c875740 1 --2- 172.21.15.86:0/548015361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc6cf4580 0x555fc6cf4960 unknown :-1 s=CLOSED pgs=703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.532+0000 7f389c875740 1 -- 172.21.15.86:0/548015361 >> 172.21.15.86:0/548015361 conn(0x555fc6ceecb0 msgr2=0x555fc6cef090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:55.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.532+0000 7f389c875740 1 -- 172.21.15.86:0/548015361 shutdown_connections 2024-04-21T08:39:55.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.532+0000 7f389c875740 1 -- 172.21.15.86:0/548015361 wait complete. 2024-04-21T08:39:55.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.532+0000 7f389c875740 1 -- start start 2024-04-21T08:39:55.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.532+0000 7f389c875740 1 Processor -- start 2024-04-21T08:39:55.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.533+0000 7f389c875740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fc6cf4580 0x555fc6cfaab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:55.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.533+0000 7f389c875740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc6cf4ea0 0x555fc6cfaff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:55.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.533+0000 7f389c875740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555fc6cfda80 0x555fc6cfb530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:55.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.533+0000 7f389c875740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555fc6d00870 con 0x555fc6cf4ea0 2024-04-21T08:39:55.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.533+0000 7f389c875740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x555fc6c122f0 con 0x555fc6cfda80 2024-04-21T08:39:55.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.533+0000 7f389c875740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555fc6d009f0 con 0x555fc6cf4580 2024-04-21T08:39:55.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.533+0000 7f389c875740 10 client.0 _ll_register_callbacks cb 0x555fc6c1f580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:55.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.533+0000 7f3892ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc6cf4ea0 0x555fc6cfaff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:55.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.533+0000 7f3892ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc6cf4ea0 0x555fc6cfaff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45396/0 (socket says 172.21.15.86:45396) 2024-04-21T08:39:55.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.533+0000 7f3892ffd640 1 -- 172.21.15.86:0/3994704473 learned_addr learned my addr 172.21.15.86:0/3994704473 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:55.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.533+0000 7f3893fff640 1 --2- 172.21.15.86:0/3994704473 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fc6cf4580 0x555fc6cfaab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:55.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.533+0000 7f3893fff640 1 --2- 172.21.15.86:0/3994704473 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555fc6cfda80 0x555fc6cfb530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:55.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.533+0000 7f3892ffd640 1 -- 172.21.15.86:0/3994704473 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fc6cf4580 msgr2=0x555fc6cfaab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:55.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.533+0000 7f3892ffd640 1 --2- 172.21.15.86:0/3994704473 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fc6cf4580 0x555fc6cfaab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.533+0000 7f3892ffd640 1 -- 172.21.15.86:0/3994704473 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555fc6cfda80 msgr2=0x555fc6cfb530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:55.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.533+0000 7f3892ffd640 1 --2- 172.21.15.86:0/3994704473 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555fc6cfda80 0x555fc6cfb530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.533+0000 7f3892ffd640 1 -- 172.21.15.86:0/3994704473 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fc696b3f0 con 0x555fc6cf4ea0 2024-04-21T08:39:55.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.533+0000 7f3892ffd640 1 --2- 172.21.15.86:0/3994704473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc6cf4ea0 0x555fc6cfaff0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f3888009f30 tx=0x7f3888011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e3f8672a02acb75 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:55.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.533+0000 7f3872ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:55.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.533+0000 7f3872ffd640 1 -- 172.21.15.86:0/3994704473 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3888012860 con 0x555fc6cf4ea0 2024-04-21T08:39:55.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.533+0000 7f3872ffd640 1 -- 172.21.15.86:0/3994704473 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f3888012a00 con 0x555fc6cf4ea0 2024-04-21T08:39:55.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.533+0000 7f389c875740 1 -- 172.21.15.86:0/3994704473 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f388c010550 con 0x555fc6cf4ea0 2024-04-21T08:39:55.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.534+0000 7f3872ffd640 1 -- 172.21.15.86:0/3994704473 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3888025050 con 0x555fc6cf4ea0 2024-04-21T08:39:55.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.534+0000 7f389c875740 10 client.16272 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:55.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.534+0000 7f389c875740 1 -- 172.21.15.86:0/3994704473 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555fc692b0f0 con 0x555fc6cf4ea0 2024-04-21T08:39:55.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.534+0000 7f3872ffd640 1 -- 172.21.15.86:0/3994704473 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f388c010550 con 0x555fc6cf4ea0 2024-04-21T08:39:55.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.534+0000 7f3872ffd640 1 -- 172.21.15.86:0/3994704473 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f388800f040 con 0x555fc6cf4ea0 2024-04-21T08:39:55.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.534+0000 7f389c875740 10 client.16272 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:55.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.534+0000 7f389c875740 10 client.16272 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:55.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.534+0000 7f389c875740 20 client.16272 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:55.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.534+0000 7f389c875740 1 --2- 172.21.15.86:0/3994704473 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x555fc6d0dc40 0x555fc6d10080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:55.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.534+0000 7f389c875740 4 client.16272 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:55.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.534+0000 7f389c875740 1 -- 172.21.15.86:0/3994704473 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555fc6c66c00 con 0x555fc6d0dc40 2024-04-21T08:39:55.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.534+0000 7f3893fff640 1 --2- 172.21.15.86:0/3994704473 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x555fc6d0dc40 0x555fc6d10080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:55.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.535+0000 7f3893fff640 1 --2- 172.21.15.86:0/3994704473 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x555fc6d0dc40 0x555fc6d10080 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db83ee34770b2a55 server_cookie=2534d6674afd583c in_seq=0 out_seq=0 2024-04-21T08:39:55.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.535+0000 7f3872ffd640 10 client.16272 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:55.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.535+0000 7f3872ffd640 1 -- 172.21.15.86:0/3994704473 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555fc6c66c00 con 0x555fc6d0dc40 2024-04-21T08:39:55.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.535+0000 7f3872ffd640 10 client.16272 handle_command_reply: tid=0 2024-04-21T08:39:55.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.535+0000 7f389c875740 1 client.16272 shutdown 2024-04-21T08:39:55.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.535+0000 7f389c875740 1 -- 172.21.15.86:0/3994704473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc6cf4ea0 msgr2=0x555fc6cfaff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:55.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.535+0000 7f389c875740 1 --2- 172.21.15.86:0/3994704473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc6cf4ea0 0x555fc6cfaff0 secure :-1 s=READY pgs=704 cs=0 l=1 rev1=1 crypto rx=0x7f3888009f30 tx=0x7f3888011870 comp rx=0 tx=0).stop 2024-04-21T08:39:55.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.535+0000 7f389c875740 1 -- 172.21.15.86:0/3994704473 shutdown_connections 2024-04-21T08:39:55.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.535+0000 7f389c875740 1 --2- 172.21.15.86:0/3994704473 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x555fc6d0dc40 0x555fc6d10080 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.535+0000 7f3872ffd640 0 client.16272 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:55.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.535+0000 7f389c875740 1 --2- 172.21.15.86:0/3994704473 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555fc6cfda80 0x555fc6cfb530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.535+0000 7f389c875740 1 --2- 172.21.15.86:0/3994704473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc6cf4ea0 0x555fc6cfaff0 unknown :-1 s=CLOSED pgs=704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.535+0000 7f389c875740 1 --2- 172.21.15.86:0/3994704473 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fc6cf4580 0x555fc6cfaab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.536+0000 7f389c875740 1 -- 172.21.15.86:0/3994704473 >> 172.21.15.86:0/3994704473 conn(0x555fc6ceecb0 msgr2=0x555fc6cf0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:55.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.536+0000 7f389c875740 1 -- 172.21.15.86:0/3994704473 shutdown_connections 2024-04-21T08:39:55.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.536+0000 7f389c875740 1 -- 172.21.15.86:0/3994704473 wait complete. 2024-04-21T08:39:55.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.536+0000 7f389c875740 20 client.16272 trim_cache size 0 max 16384 2024-04-21T08:39:55.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.553+0000 7f389c875740 1 Processor -- start 2024-04-21T08:39:55.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.553+0000 7f389c875740 1 -- start start 2024-04-21T08:39:55.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.553+0000 7f389c875740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc6cf4580 0x555fc6d09790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:55.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.553+0000 7f389c875740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555fc6cf4ea0 0x555fc6cfb350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:55.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.553+0000 7f389c875740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fc6cfda80 0x555fc6cedbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:55.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.553+0000 7f389c875740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f38941b14f0 con 0x555fc6cf4580 2024-04-21T08:39:55.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.553+0000 7f389c875740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f38941b3030 con 0x555fc6cf4ea0 2024-04-21T08:39:55.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.553+0000 7f389c875740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f38941b1bc0 con 0x555fc6cfda80 2024-04-21T08:39:55.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.553+0000 7f3893fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc6cf4580 0x555fc6d09790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:55.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.553+0000 7f3893fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc6cf4580 0x555fc6d09790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45400/0 (socket says 172.21.15.86:45400) 2024-04-21T08:39:55.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.553+0000 7f3893fff640 1 -- 172.21.15.86:0/2899667368 learned_addr learned my addr 172.21.15.86:0/2899667368 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:55.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.553+0000 7f3892ffd640 1 --2- 172.21.15.86:0/2899667368 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555fc6cf4ea0 0x555fc6cfb350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:55.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.553+0000 7f3893fff640 1 --2- 172.21.15.86:0/2899667368 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fc6cfda80 0x555fc6cedbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:55.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.553+0000 7f3893fff640 1 -- 172.21.15.86:0/2899667368 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fc6cfda80 msgr2=0x555fc6cedbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:55.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.553+0000 7f3893fff640 1 --2- 172.21.15.86:0/2899667368 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fc6cfda80 0x555fc6cedbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.553+0000 7f3893fff640 1 -- 172.21.15.86:0/2899667368 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555fc6cf4ea0 msgr2=0x555fc6cfb350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:55.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.553+0000 7f3893fff640 1 --2- 172.21.15.86:0/2899667368 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555fc6cf4ea0 0x555fc6cfb350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.553+0000 7f3893fff640 1 -- 172.21.15.86:0/2899667368 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555fc6cef630 con 0x555fc6cf4580 2024-04-21T08:39:55.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.553+0000 7f3893fff640 1 --2- 172.21.15.86:0/2899667368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc6cf4580 0x555fc6d09790 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f388c015fd0 tx=0x7f388c009710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a758b166c26d73f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:55.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.553+0000 7f38797fa640 1 -- 172.21.15.86:0/2899667368 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c02e570 con 0x555fc6cf4580 2024-04-21T08:39:55.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.553+0000 7f38797fa640 1 -- 172.21.15.86:0/2899667368 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f388c02e710 con 0x555fc6cf4580 2024-04-21T08:39:55.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.553+0000 7f38797fa640 1 -- 172.21.15.86:0/2899667368 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f388c022440 con 0x555fc6cf4580 2024-04-21T08:39:55.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.554+0000 7f389c875740 1 -- 172.21.15.86:0/2899667368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc6cf4580 msgr2=0x555fc6d09790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:55.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.554+0000 7f389c875740 1 --2- 172.21.15.86:0/2899667368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc6cf4580 0x555fc6d09790 secure :-1 s=READY pgs=705 cs=0 l=1 rev1=1 crypto rx=0x7f388c015fd0 tx=0x7f388c009710 comp rx=0 tx=0).stop 2024-04-21T08:39:55.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.554+0000 7f389c875740 1 -- 172.21.15.86:0/2899667368 shutdown_connections 2024-04-21T08:39:55.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.554+0000 7f389c875740 1 --2- 172.21.15.86:0/2899667368 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fc6cfda80 0x555fc6cedbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.554+0000 7f389c875740 1 --2- 172.21.15.86:0/2899667368 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555fc6cf4ea0 0x555fc6cfb350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.554+0000 7f389c875740 1 --2- 172.21.15.86:0/2899667368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc6cf4580 0x555fc6d09790 unknown :-1 s=CLOSED pgs=705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.554+0000 7f389c875740 1 -- 172.21.15.86:0/2899667368 >> 172.21.15.86:0/2899667368 conn(0x555fc6ceecb0 msgr2=0x555fc6ced4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:55.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.554+0000 7f389c875740 1 -- 172.21.15.86:0/2899667368 shutdown_connections 2024-04-21T08:39:55.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.554+0000 7f389c875740 1 -- 172.21.15.86:0/2899667368 wait complete. 2024-04-21T08:39:55.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.554+0000 7f389c875740 1 -- start start 2024-04-21T08:39:55.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.555+0000 7f389c875740 1 Processor -- start 2024-04-21T08:39:55.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.555+0000 7f389c875740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fc6cf4580 0x555fc6cf7700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:55.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.555+0000 7f389c875740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc6cf4ea0 0x555fc6cf7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:55.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.555+0000 7f389c875740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555fc6cfda80 0x555fc6cf8180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:55.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.555+0000 7f389c875740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f38941b2960 con 0x555fc6cf4ea0 2024-04-21T08:39:55.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.555+0000 7f389c875740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f38941b2290 con 0x555fc6cfda80 2024-04-21T08:39:55.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.555+0000 7f389c875740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f38941af4b0 con 0x555fc6cf4580 2024-04-21T08:39:55.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.555+0000 7f389c875740 10 client.0 _ll_register_callbacks cb 0x7f38880097c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:55.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.555+0000 7f3892ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc6cf4ea0 0x555fc6cf7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:55.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.555+0000 7f3892ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc6cf4ea0 0x555fc6cf7c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45410/0 (socket says 172.21.15.86:45410) 2024-04-21T08:39:55.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.555+0000 7f3892ffd640 1 -- 172.21.15.86:0/2743602496 learned_addr learned my addr 172.21.15.86:0/2743602496 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:55.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.555+0000 7f3893fff640 1 --2- 172.21.15.86:0/2743602496 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fc6cf4580 0x555fc6cf7700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:55.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.555+0000 7f3893fff640 1 --2- 172.21.15.86:0/2743602496 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555fc6cfda80 0x555fc6cf8180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:55.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.555+0000 7f3892ffd640 1 -- 172.21.15.86:0/2743602496 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fc6cf4580 msgr2=0x555fc6cf7700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:55.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.555+0000 7f3892ffd640 1 --2- 172.21.15.86:0/2743602496 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fc6cf4580 0x555fc6cf7700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.555+0000 7f3892ffd640 1 -- 172.21.15.86:0/2743602496 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555fc6cfda80 msgr2=0x555fc6cf8180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:55.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.555+0000 7f3892ffd640 1 --2- 172.21.15.86:0/2743602496 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555fc6cfda80 0x555fc6cf8180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.555+0000 7f3892ffd640 1 -- 172.21.15.86:0/2743602496 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555fc6d73200 con 0x555fc6cf4ea0 2024-04-21T08:39:55.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.555+0000 7f3893fff640 1 --2- 172.21.15.86:0/2743602496 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555fc6cfda80 0x555fc6cf8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:55.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.556+0000 7f3892ffd640 1 --2- 172.21.15.86:0/2743602496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc6cf4ea0 0x555fc6cf7c40 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x555fc6cfc080 tx=0x7f3888005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a61222d34912c187 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:55.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.556+0000 7f3872ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:55.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.556+0000 7f3872ffd640 1 -- 172.21.15.86:0/2743602496 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38880055f0 con 0x555fc6cf4ea0 2024-04-21T08:39:55.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.556+0000 7f3872ffd640 1 -- 172.21.15.86:0/2743602496 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f3888009a80 con 0x555fc6cf4ea0 2024-04-21T08:39:55.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.556+0000 7f389c875740 1 -- 172.21.15.86:0/2743602496 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f388c02e710 con 0x555fc6cf4ea0 2024-04-21T08:39:55.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.556+0000 7f3872ffd640 1 -- 172.21.15.86:0/2743602496 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f388801a3f0 con 0x555fc6cf4ea0 2024-04-21T08:39:55.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.556+0000 7f389c875740 10 client.16278 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:55.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.556+0000 7f389c875740 1 -- 172.21.15.86:0/2743602496 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555fc6d74450 con 0x555fc6cf4ea0 2024-04-21T08:39:55.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.556+0000 7f3872ffd640 1 -- 172.21.15.86:0/2743602496 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f388c02e710 con 0x555fc6cf4ea0 2024-04-21T08:39:55.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.556+0000 7f3872ffd640 1 -- 172.21.15.86:0/2743602496 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f38880128e0 con 0x555fc6cf4ea0 2024-04-21T08:39:55.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.556+0000 7f389c875740 10 client.16278 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:55.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.556+0000 7f389c875740 10 client.16278 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:55.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.556+0000 7f389c875740 20 client.16278 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:55.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.556+0000 7f389c875740 1 --2- 172.21.15.86:0/2743602496 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x555fc6d0dc40 0x555fc6d76ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:55.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.556+0000 7f389c875740 4 client.16278 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:39:55.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.556+0000 7f389c875740 1 -- 172.21.15.86:0/2743602496 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x555fc6c60a70 con 0x555fc6d0dc40 2024-04-21T08:39:55.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.556+0000 7f3893fff640 1 --2- 172.21.15.86:0/2743602496 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x555fc6d0dc40 0x555fc6d76ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:55.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.557+0000 7f3872ffd640 10 client.16278 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:55.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.557+0000 7f3893fff640 1 --2- 172.21.15.86:0/2743602496 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x555fc6d0dc40 0x555fc6d76ba0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=951c4f1896aacb13 server_cookie=67d3be45dd944355 in_seq=0 out_seq=0 2024-04-21T08:39:55.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.557+0000 7f3872ffd640 1 -- 172.21.15.86:0/2743602496 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x555fc6c60a70 con 0x555fc6d0dc40 2024-04-21T08:39:55.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.557+0000 7f3872ffd640 10 client.16278 handle_command_reply: tid=0 2024-04-21T08:39:55.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.557+0000 7f389c875740 1 client.16278 shutdown 2024-04-21T08:39:55.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.557+0000 7f389c875740 1 -- 172.21.15.86:0/2743602496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc6cf4ea0 msgr2=0x555fc6cf7c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:55.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.557+0000 7f389c875740 1 --2- 172.21.15.86:0/2743602496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc6cf4ea0 0x555fc6cf7c40 secure :-1 s=READY pgs=706 cs=0 l=1 rev1=1 crypto rx=0x555fc6cfc080 tx=0x7f3888005270 comp rx=0 tx=0).stop 2024-04-21T08:39:55.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.557+0000 7f389c875740 1 -- 172.21.15.86:0/2743602496 shutdown_connections 2024-04-21T08:39:55.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.557+0000 7f389c875740 1 --2- 172.21.15.86:0/2743602496 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x555fc6d0dc40 0x555fc6d76ba0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.557+0000 7f389c875740 1 --2- 172.21.15.86:0/2743602496 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555fc6cfda80 0x555fc6cf8180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.557+0000 7f389c875740 1 --2- 172.21.15.86:0/2743602496 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555fc6cf4ea0 0x555fc6cf7c40 unknown :-1 s=CLOSED pgs=706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.557+0000 7f3872ffd640 0 client.16278 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:55.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.557+0000 7f389c875740 1 --2- 172.21.15.86:0/2743602496 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555fc6cf4580 0x555fc6cf7700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.557+0000 7f389c875740 1 -- 172.21.15.86:0/2743602496 >> 172.21.15.86:0/2743602496 conn(0x555fc6ceecb0 msgr2=0x555fc6cf0970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:55.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.557+0000 7f389c875740 1 -- 172.21.15.86:0/2743602496 shutdown_connections 2024-04-21T08:39:55.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.557+0000 7f389c875740 1 -- 172.21.15.86:0/2743602496 wait complete. 2024-04-21T08:39:55.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.557+0000 7f389c875740 20 client.16278 trim_cache size 0 max 16384 2024-04-21T08:39:55.565 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:39:55.566 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-21T08:39:55.566 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-21T08:39:55.566 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:39:55.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.558+0000 7f3899c81640 1 -- 172.21.15.86:0/991391411 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f387c0a3e70 msgr2=0x7f387c0a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:55.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.558+0000 7f3899c81640 1 --2- 172.21.15.86:0/991391411 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f387c0a3e70 0x7f387c0a6330 secure :-1 s=READY pgs=96 cs=0 l=1 rev1=1 crypto rx=0x7f38840f02c0 tx=0x7f3884089480 comp rx=0 tx=0).stop 2024-04-21T08:39:55.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.558+0000 7f3899c81640 1 -- 172.21.15.86:0/991391411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38941031f0 msgr2=0x7f3894100e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:55.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.558+0000 7f3899c81640 1 --2- 172.21.15.86:0/991391411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38941031f0 0x7f3894100e40 secure :-1 s=READY pgs=702 cs=0 l=1 rev1=1 crypto rx=0x7f3884000c00 tx=0x7f388402d920 comp rx=0 tx=0).stop 2024-04-21T08:39:55.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.558+0000 7f3899c81640 1 -- 172.21.15.86:0/991391411 shutdown_connections 2024-04-21T08:39:55.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.558+0000 7f3899c81640 1 --2- 172.21.15.86:0/991391411 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f387c0a3e70 0x7f387c0a6330 unknown :-1 s=CLOSED pgs=96 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.558+0000 7f3899c81640 1 --2- 172.21.15.86:0/991391411 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f389410c530 0x7f38941018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.558+0000 7f3899c81640 1 --2- 172.21.15.86:0/991391411 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3894103b10 0x7f3894101380 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.558+0000 7f3899c81640 1 --2- 172.21.15.86:0/991391411 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38941031f0 0x7f3894100e40 unknown :-1 s=CLOSED pgs=702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.558+0000 7f3899c81640 1 -- 172.21.15.86:0/991391411 >> 172.21.15.86:0/991391411 conn(0x7f38940771d0 msgr2=0x7f38940fdd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:55.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.558+0000 7f3899c81640 1 -- 172.21.15.86:0/991391411 shutdown_connections 2024-04-21T08:39:55.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.558+0000 7f3899c81640 1 -- 172.21.15.86:0/991391411 wait complete. 2024-04-21T08:39:55.568 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:39:55.569 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:39:55.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.667+0000 7f8ea365c640 1 -- 172.21.15.86:0/3364772209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e9c107260 msgr2=0x7f8e9c10ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:55.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.667+0000 7f8ea365c640 1 --2- 172.21.15.86:0/3364772209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e9c107260 0x7f8e9c10ee30 secure :-1 s=READY pgs=707 cs=0 l=1 rev1=1 crypto rx=0x7f8e94009900 tx=0x7f8e9402bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:55.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.667+0000 7f8ea365c640 1 -- 172.21.15.86:0/3364772209 shutdown_connections 2024-04-21T08:39:55.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.667+0000 7f8ea365c640 1 --2- 172.21.15.86:0/3364772209 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e9c107260 0x7f8e9c10ee30 unknown :-1 s=CLOSED pgs=707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.667+0000 7f8ea365c640 1 --2- 172.21.15.86:0/3364772209 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e9c102b40 0x7f8e9c106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.667+0000 7f8ea365c640 1 --2- 172.21.15.86:0/3364772209 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8e9c102190 0x7f8e9c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.667+0000 7f8ea365c640 1 -- 172.21.15.86:0/3364772209 >> 172.21.15.86:0/3364772209 conn(0x7f8e9c0793e0 msgr2=0x7f8e9c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:55.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.668+0000 7f8ea365c640 1 -- 172.21.15.86:0/3364772209 shutdown_connections 2024-04-21T08:39:55.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.668+0000 7f8ea365c640 1 -- 172.21.15.86:0/3364772209 wait complete. 2024-04-21T08:39:55.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.668+0000 7f8ea365c640 1 Processor -- start 2024-04-21T08:39:55.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.668+0000 7f8ea365c640 1 -- start start 2024-04-21T08:39:55.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.668+0000 7f8ea365c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8e9c102190 0x7f8e9c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:55.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.668+0000 7f8ea365c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e9c107260 0x7f8e9c198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:55.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.668+0000 7f8ea365c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e9c19cfb0 0x7f8e9c19d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:55.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.668+0000 7f8ea365c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9c1121b0 con 0x7f8e9c19cfb0 2024-04-21T08:39:55.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.668+0000 7f8ea365c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9c112030 con 0x7f8e9c102190 2024-04-21T08:39:55.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.668+0000 7f8ea365c640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8e9c112330 con 0x7f8e9c107260 2024-04-21T08:39:55.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.668+0000 7f8ea1bf9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e9c19cfb0 0x7f8e9c19d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:55.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.668+0000 7f8ea1bf9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e9c19cfb0 0x7f8e9c19d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45436/0 (socket says 172.21.15.86:45436) 2024-04-21T08:39:55.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.668+0000 7f8ea1bf9640 1 -- 172.21.15.86:0/3828276445 learned_addr learned my addr 172.21.15.86:0/3828276445 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:55.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.669+0000 7f8ea13f8640 1 --2- 172.21.15.86:0/3828276445 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8e9c102190 0x7f8e9c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:55.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.669+0000 7f8ea0bf7640 1 --2- 172.21.15.86:0/3828276445 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e9c107260 0x7f8e9c198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:55.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.669+0000 7f8ea1bf9640 1 -- 172.21.15.86:0/3828276445 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e9c107260 msgr2=0x7f8e9c198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:55.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.669+0000 7f8ea1bf9640 1 --2- 172.21.15.86:0/3828276445 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e9c107260 0x7f8e9c198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.669+0000 7f8ea1bf9640 1 -- 172.21.15.86:0/3828276445 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8e9c102190 msgr2=0x7f8e9c1986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:55.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.669+0000 7f8ea1bf9640 1 --2- 172.21.15.86:0/3828276445 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8e9c102190 0x7f8e9c1986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.669+0000 7f8ea1bf9640 1 -- 172.21.15.86:0/3828276445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e9c19daa0 con 0x7f8e9c19cfb0 2024-04-21T08:39:55.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.669+0000 7f8ea13f8640 1 --2- 172.21.15.86:0/3828276445 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8e9c102190 0x7f8e9c1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:55.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.669+0000 7f8ea0bf7640 1 --2- 172.21.15.86:0/3828276445 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e9c107260 0x7f8e9c198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:55.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.669+0000 7f8ea1bf9640 1 --2- 172.21.15.86:0/3828276445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e9c19cfb0 0x7f8e9c19d460 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f8e94009230 tx=0x7f8e9402c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:55.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.669+0000 7f8e927fc640 1 -- 172.21.15.86:0/3828276445 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e94043020 con 0x7f8e9c19cfb0 2024-04-21T08:39:55.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.669+0000 7f8e927fc640 1 -- 172.21.15.86:0/3828276445 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e9402caf0 con 0x7f8e9c19cfb0 2024-04-21T08:39:55.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.669+0000 7f8ea365c640 1 -- 172.21.15.86:0/3828276445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e9c19dd30 con 0x7f8e9c19cfb0 2024-04-21T08:39:55.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.669+0000 7f8e927fc640 1 -- 172.21.15.86:0/3828276445 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e94003990 con 0x7f8e9c19cfb0 2024-04-21T08:39:55.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.669+0000 7f8ea365c640 1 -- 172.21.15.86:0/3828276445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e9c1a5580 con 0x7f8e9c19cfb0 2024-04-21T08:39:55.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.670+0000 7f8ea365c640 1 -- 172.21.15.86:0/3828276445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8e9c069230 con 0x7f8e9c19cfb0 2024-04-21T08:39:55.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.671+0000 7f8e927fc640 1 -- 172.21.15.86:0/3828276445 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f8e94046020 con 0x7f8e9c19cfb0 2024-04-21T08:39:55.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.671+0000 7f8e927fc640 1 --2- 172.21.15.86:0/3828276445 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8e880a3ec0 0x7f8e880a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:55.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.672+0000 7f8ea13f8640 1 --2- 172.21.15.86:0/3828276445 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8e880a3ec0 0x7f8e880a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:55.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.672+0000 7f8e927fc640 1 -- 172.21.15.86:0/3828276445 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f8e940f2f70 con 0x7f8e9c19cfb0 2024-04-21T08:39:55.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.672+0000 7f8ea13f8640 1 --2- 172.21.15.86:0/3828276445 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8e880a3ec0 0x7f8e880a6380 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8e98005fd0 tx=0x7f8e98009500 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:55.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.676+0000 7f8e927fc640 1 -- 172.21.15.86:0/3828276445 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8e9404d020 con 0x7f8e9c19cfb0 2024-04-21T08:39:55.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.893+0000 7f8ea365c640 1 -- 172.21.15.86:0/3828276445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8e9c102b40 con 0x7f8e9c19cfb0 2024-04-21T08:39:55.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.894+0000 7f8e927fc640 1 -- 172.21.15.86:0/3828276445 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7f8e9403c350 con 0x7f8e9c19cfb0 2024-04-21T08:39:55.893 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:39:55.893 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:38:45.202126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24398,24394,24455,24472]},"id":1}]} 2024-04-21T08:39:55.893 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-21T08:39:55.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.896+0000 7f8ea365c640 1 -- 172.21.15.86:0/3828276445 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8e880a3ec0 msgr2=0x7f8e880a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:55.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.896+0000 7f8ea365c640 1 --2- 172.21.15.86:0/3828276445 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8e880a3ec0 0x7f8e880a6380 secure :-1 s=READY pgs=97 cs=0 l=1 rev1=1 crypto rx=0x7f8e98005fd0 tx=0x7f8e98009500 comp rx=0 tx=0).stop 2024-04-21T08:39:55.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.896+0000 7f8ea365c640 1 -- 172.21.15.86:0/3828276445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e9c19cfb0 msgr2=0x7f8e9c19d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:55.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.896+0000 7f8ea365c640 1 --2- 172.21.15.86:0/3828276445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e9c19cfb0 0x7f8e9c19d460 secure :-1 s=READY pgs=708 cs=0 l=1 rev1=1 crypto rx=0x7f8e94009230 tx=0x7f8e9402c710 comp rx=0 tx=0).stop 2024-04-21T08:39:55.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.896+0000 7f8ea365c640 1 -- 172.21.15.86:0/3828276445 shutdown_connections 2024-04-21T08:39:55.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.896+0000 7f8ea365c640 1 --2- 172.21.15.86:0/3828276445 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8e880a3ec0 0x7f8e880a6380 unknown :-1 s=CLOSED pgs=97 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.896+0000 7f8ea365c640 1 --2- 172.21.15.86:0/3828276445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e9c19cfb0 0x7f8e9c19d460 unknown :-1 s=CLOSED pgs=708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.896+0000 7f8ea365c640 1 --2- 172.21.15.86:0/3828276445 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e9c107260 0x7f8e9c198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.896+0000 7f8ea365c640 1 --2- 172.21.15.86:0/3828276445 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8e9c102190 0x7f8e9c1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:55.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.896+0000 7f8ea365c640 1 -- 172.21.15.86:0/3828276445 >> 172.21.15.86:0/3828276445 conn(0x7f8e9c0793e0 msgr2=0x7f8e9c0ffe50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:55.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.896+0000 7f8ea365c640 1 -- 172.21.15.86:0/3828276445 shutdown_connections 2024-04-21T08:39:55.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:55.896+0000 7f8ea365c640 1 -- 172.21.15.86:0/3828276445 wait complete. 2024-04-21T08:39:55.906 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:39:56.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.008+0000 7f85e17e6640 1 -- 172.21.15.86:0/3212868832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85dc06a840 msgr2=0x7f85dc06ac20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.008+0000 7f85e17e6640 1 --2- 172.21.15.86:0/3212868832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85dc06a840 0x7f85dc06ac20 secure :-1 s=READY pgs=709 cs=0 l=1 rev1=1 crypto rx=0x7f85cc009900 tx=0x7f85cc02bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:56.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.008+0000 7f85e17e6640 1 -- 172.21.15.86:0/3212868832 shutdown_connections 2024-04-21T08:39:56.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.008+0000 7f85e17e6640 1 --2- 172.21.15.86:0/3212868832 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85dc107270 0x7f85dc10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.008+0000 7f85e17e6640 1 --2- 172.21.15.86:0/3212868832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85dc06b1f0 0x7f85dc106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.008+0000 7f85e17e6640 1 --2- 172.21.15.86:0/3212868832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85dc06a840 0x7f85dc06ac20 unknown :-1 s=CLOSED pgs=709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.008+0000 7f85e17e6640 1 -- 172.21.15.86:0/3212868832 >> 172.21.15.86:0/3212868832 conn(0x7f85dc077eb0 msgr2=0x7f85dc0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:56.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.008+0000 7f85e17e6640 1 -- 172.21.15.86:0/3212868832 shutdown_connections 2024-04-21T08:39:56.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.008+0000 7f85e17e6640 1 -- 172.21.15.86:0/3212868832 wait complete. 2024-04-21T08:39:56.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.009+0000 7f85e17e6640 1 Processor -- start 2024-04-21T08:39:56.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.009+0000 7f85e17e6640 1 -- start start 2024-04-21T08:39:56.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.009+0000 7f85e17e6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85dc06a840 0x7f85dc19e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.009+0000 7f85e17e6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85dc107270 0x7f85dc19e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.009+0000 7f85e17e6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85dc198590 0x7f85dc198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.009+0000 7f85e17e6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc1121d0 con 0x7f85dc06a840 2024-04-21T08:39:56.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.009+0000 7f85e17e6640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc112050 con 0x7f85dc107270 2024-04-21T08:39:56.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.009+0000 7f85e17e6640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc112350 con 0x7f85dc198590 2024-04-21T08:39:56.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.009+0000 7f85daffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85dc06a840 0x7f85dc19e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.009+0000 7f85daffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85dc06a840 0x7f85dc19e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45456/0 (socket says 172.21.15.86:45456) 2024-04-21T08:39:56.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.009+0000 7f85daffd640 1 -- 172.21.15.86:0/3044459354 learned_addr learned my addr 172.21.15.86:0/3044459354 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:56.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.009+0000 7f85da7fc640 1 --2- 172.21.15.86:0/3044459354 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85dc107270 0x7f85dc19e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.010+0000 7f85daffd640 1 -- 172.21.15.86:0/3044459354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85dc198590 msgr2=0x7f85dc198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.010+0000 7f85db7fe640 1 --2- 172.21.15.86:0/3044459354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85dc198590 0x7f85dc198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.010+0000 7f85daffd640 1 --2- 172.21.15.86:0/3044459354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85dc198590 0x7f85dc198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.010+0000 7f85daffd640 1 -- 172.21.15.86:0/3044459354 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85dc107270 msgr2=0x7f85dc19e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.010+0000 7f85daffd640 1 --2- 172.21.15.86:0/3044459354 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85dc107270 0x7f85dc19e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.010+0000 7f85daffd640 1 -- 172.21.15.86:0/3044459354 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85dc1990b0 con 0x7f85dc06a840 2024-04-21T08:39:56.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.010+0000 7f85da7fc640 1 --2- 172.21.15.86:0/3044459354 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85dc107270 0x7f85dc19e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:56.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.010+0000 7f85db7fe640 1 --2- 172.21.15.86:0/3044459354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85dc198590 0x7f85dc198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:56.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.010+0000 7f85daffd640 1 --2- 172.21.15.86:0/3044459354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85dc06a840 0x7f85dc19e3e0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f85cc005940 tx=0x7f85cc02c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:56.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.010+0000 7f85c3fff640 1 -- 172.21.15.86:0/3044459354 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc043020 con 0x7f85dc06a840 2024-04-21T08:39:56.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.010+0000 7f85e17e6640 1 -- 172.21.15.86:0/3044459354 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85dc199340 con 0x7f85dc06a840 2024-04-21T08:39:56.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.010+0000 7f85c3fff640 1 -- 172.21.15.86:0/3044459354 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85cc02cbe0 con 0x7f85dc06a840 2024-04-21T08:39:56.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.010+0000 7f85c3fff640 1 -- 172.21.15.86:0/3044459354 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85cc003bf0 con 0x7f85dc06a840 2024-04-21T08:39:56.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.010+0000 7f85e17e6640 1 -- 172.21.15.86:0/3044459354 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85dc1a5590 con 0x7f85dc06a840 2024-04-21T08:39:56.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.012+0000 7f85c3fff640 1 -- 172.21.15.86:0/3044459354 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f85cc046020 con 0x7f85dc06a840 2024-04-21T08:39:56.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.012+0000 7f85e43d1740 1 Processor -- start 2024-04-21T08:39:56.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.013+0000 7f85e43d1740 1 -- start start 2024-04-21T08:39:56.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.013+0000 7f85e43d1740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558fc614b4d0 0x558fc614b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.013+0000 7f85e43d1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558fc614be80 0x558fc6154430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.013+0000 7f85e43d1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc6154a00 0x558fc6156e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.013+0000 7f85e43d1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558fc60b01f0 con 0x558fc6154a00 2024-04-21T08:39:56.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.013+0000 7f85e43d1740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x558fc604d4b0 con 0x558fc614b4d0 2024-04-21T08:39:56.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.013+0000 7f85e43d1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558fc60ac1b0 con 0x558fc614be80 2024-04-21T08:39:56.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.013+0000 7f85da7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc6154a00 0x558fc6156e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.013+0000 7f85da7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc6154a00 0x558fc6156e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45460/0 (socket says 172.21.15.86:45460) 2024-04-21T08:39:56.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.013+0000 7f85da7fc640 1 -- 172.21.15.86:0/731261497 learned_addr learned my addr 172.21.15.86:0/731261497 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:56.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.013+0000 7f85daffd640 1 --2- 172.21.15.86:0/731261497 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558fc614be80 0x558fc6154430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.013+0000 7f85db7fe640 1 --2- 172.21.15.86:0/731261497 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558fc614b4d0 0x558fc614b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.013+0000 7f85c3fff640 1 --2- 172.21.15.86:0/3044459354 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f85c40a3e40 0x7f85c40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.013+0000 7f85da7fc640 1 -- 172.21.15.86:0/731261497 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558fc614be80 msgr2=0x558fc6154430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.013+0000 7f85da7fc640 1 --2- 172.21.15.86:0/731261497 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558fc614be80 0x558fc6154430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.013+0000 7f85da7fc640 1 -- 172.21.15.86:0/731261497 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558fc614b4d0 msgr2=0x558fc614b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.013+0000 7f85da7fc640 1 --2- 172.21.15.86:0/731261497 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558fc614b4d0 0x558fc614b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.013+0000 7f85da7fc640 1 -- 172.21.15.86:0/731261497 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fc603f1b0 con 0x558fc6154a00 2024-04-21T08:39:56.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.013+0000 7f85db7fe640 1 --2- 172.21.15.86:0/3044459354 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f85c40a3e40 0x7f85c40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.013+0000 7f85c3fff640 1 -- 172.21.15.86:0/3044459354 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f85cc0f3020 con 0x7f85dc06a840 2024-04-21T08:39:56.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.013+0000 7f85da7fc640 1 --2- 172.21.15.86:0/731261497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc6154a00 0x558fc6156e50 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f85d0009bc0 tx=0x7f85d000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6011e71b6e714997 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:56.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.013+0000 7f85c17fa640 1 -- 172.21.15.86:0/731261497 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d0016070 con 0x558fc6154a00 2024-04-21T08:39:56.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.013+0000 7f85c17fa640 1 -- 172.21.15.86:0/731261497 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85d0011920 con 0x558fc6154a00 2024-04-21T08:39:56.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.014+0000 7f85c17fa640 1 -- 172.21.15.86:0/731261497 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d0010870 con 0x558fc6154a00 2024-04-21T08:39:56.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.014+0000 7f85e43d1740 1 -- 172.21.15.86:0/731261497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc6154a00 msgr2=0x558fc6156e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.014+0000 7f85e43d1740 1 --2- 172.21.15.86:0/731261497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc6154a00 0x558fc6156e50 secure :-1 s=READY pgs=711 cs=0 l=1 rev1=1 crypto rx=0x7f85d0009bc0 tx=0x7f85d000e6c0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.014+0000 7f85db7fe640 1 --2- 172.21.15.86:0/3044459354 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f85c40a3e40 0x7f85c40a6300 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f85d4005fd0 tx=0x7f85d4004a40 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:56.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.014+0000 7f85e43d1740 1 -- 172.21.15.86:0/731261497 shutdown_connections 2024-04-21T08:39:56.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.014+0000 7f85e43d1740 1 --2- 172.21.15.86:0/731261497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc6154a00 0x558fc6156e50 unknown :-1 s=CLOSED pgs=711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.014+0000 7f85e43d1740 1 --2- 172.21.15.86:0/731261497 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558fc614be80 0x558fc6154430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.014+0000 7f85e43d1740 1 --2- 172.21.15.86:0/731261497 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558fc614b4d0 0x558fc614b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.014+0000 7f85e43d1740 1 -- 172.21.15.86:0/731261497 >> 172.21.15.86:0/731261497 conn(0x558fc6145ba0 msgr2=0x558fc6145f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:56.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.014+0000 7f85e43d1740 1 -- 172.21.15.86:0/731261497 shutdown_connections 2024-04-21T08:39:56.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.014+0000 7f85e43d1740 1 -- 172.21.15.86:0/731261497 wait complete. 2024-04-21T08:39:56.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.014+0000 7f85e43d1740 1 -- start start 2024-04-21T08:39:56.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.015+0000 7f85e43d1740 1 Processor -- start 2024-04-21T08:39:56.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.015+0000 7f85e43d1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558fc614b4d0 0x558fc61542d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.015+0000 7f85e43d1740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558fc614be80 0x558fc6151660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.015+0000 7f85e43d1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc6154a00 0x558fc6151bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.015+0000 7f85e43d1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558fc6157970 con 0x558fc6154a00 2024-04-21T08:39:56.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.015+0000 7f85e43d1740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x558fc6069500 con 0x558fc614be80 2024-04-21T08:39:56.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.015+0000 7f85e43d1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558fc6157af0 con 0x558fc614b4d0 2024-04-21T08:39:56.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.015+0000 7f85e43d1740 10 client.0 _ll_register_callbacks cb 0x558fc6037b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:56.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.015+0000 7f85db7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc6154a00 0x558fc6151bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.015+0000 7f85db7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc6154a00 0x558fc6151bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45464/0 (socket says 172.21.15.86:45464) 2024-04-21T08:39:56.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.015+0000 7f85db7fe640 1 -- 172.21.15.86:0/1680816897 learned_addr learned my addr 172.21.15.86:0/1680816897 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:56.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.015+0000 7f85daffd640 1 --2- 172.21.15.86:0/1680816897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558fc614b4d0 0x558fc61542d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.015+0000 7f85da7fc640 1 --2- 172.21.15.86:0/1680816897 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558fc614be80 0x558fc6151660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.015+0000 7f85db7fe640 1 -- 172.21.15.86:0/1680816897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558fc614b4d0 msgr2=0x558fc61542d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.015+0000 7f85db7fe640 1 --2- 172.21.15.86:0/1680816897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558fc614b4d0 0x558fc61542d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.015+0000 7f85db7fe640 1 -- 172.21.15.86:0/1680816897 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558fc614be80 msgr2=0x558fc6151660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.015+0000 7f85db7fe640 1 --2- 172.21.15.86:0/1680816897 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558fc614be80 0x558fc6151660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.015+0000 7f85db7fe640 1 -- 172.21.15.86:0/1680816897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fc5dc23f0 con 0x558fc6154a00 2024-04-21T08:39:56.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.015+0000 7f85daffd640 1 --2- 172.21.15.86:0/1680816897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558fc614b4d0 0x558fc61542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:56.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.015+0000 7f85db7fe640 1 --2- 172.21.15.86:0/1680816897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc6154a00 0x558fc6151bc0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f85d40119f0 tx=0x7f85d4008420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=622338a4e42c39f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:56.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.015+0000 7f85beff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:56.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.015+0000 7f85beff5640 1 -- 172.21.15.86:0/1680816897 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d401cd80 con 0x558fc6154a00 2024-04-21T08:39:56.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.016+0000 7f85beff5640 1 -- 172.21.15.86:0/1680816897 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f85d401a450 con 0x558fc6154a00 2024-04-21T08:39:56.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.016+0000 7f85e43d1740 1 -- 172.21.15.86:0/1680816897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85d0011920 con 0x558fc6154a00 2024-04-21T08:39:56.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.016+0000 7f85beff5640 1 -- 172.21.15.86:0/1680816897 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f85d401a650 con 0x558fc6154a00 2024-04-21T08:39:56.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.016+0000 7f85e43d1740 10 client.16296 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:56.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.016+0000 7f85e43d1740 1 -- 172.21.15.86:0/1680816897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558fc5e8c7d0 con 0x558fc6154a00 2024-04-21T08:39:56.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.016+0000 7f85beff5640 1 -- 172.21.15.86:0/1680816897 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85d0011920 con 0x558fc6154a00 2024-04-21T08:39:56.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.016+0000 7f85beff5640 1 -- 172.21.15.86:0/1680816897 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f85d4023070 con 0x558fc6154a00 2024-04-21T08:39:56.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.016+0000 7f85e43d1740 10 client.16296 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:56.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.016+0000 7f85e43d1740 10 client.16296 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:39:56.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.016+0000 7f85e43d1740 20 client.16296 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:56.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.016+0000 7f85e43d1740 1 --2- 172.21.15.86:0/1680816897 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x558fc6164c80 0x558fc61670d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.016+0000 7f85e43d1740 4 client.16296 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:56.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.016+0000 7f85e43d1740 1 -- 172.21.15.86:0/1680816897 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558fc604f930 con 0x558fc6164c80 2024-04-21T08:39:56.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.016+0000 7f85daffd640 1 --2- 172.21.15.86:0/1680816897 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x558fc6164c80 0x558fc61670d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.017+0000 7f85daffd640 1 --2- 172.21.15.86:0/1680816897 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x558fc6164c80 0x558fc61670d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=898a113968bbbfbf server_cookie=ac2c7234034c5cd4 in_seq=0 out_seq=0 2024-04-21T08:39:56.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.017+0000 7f85beff5640 10 client.16296 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:39:56.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.018+0000 7f85beff5640 1 -- 172.21.15.86:0/1680816897 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558fc604f930 con 0x558fc6164c80 2024-04-21T08:39:56.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.018+0000 7f85beff5640 10 client.16296 handle_command_reply: tid=0 2024-04-21T08:39:56.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.018+0000 7f85e43d1740 1 client.16296 shutdown 2024-04-21T08:39:56.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.018+0000 7f85e43d1740 1 -- 172.21.15.86:0/1680816897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc6154a00 msgr2=0x558fc6151bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.018+0000 7f85e43d1740 1 --2- 172.21.15.86:0/1680816897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc6154a00 0x558fc6151bc0 secure :-1 s=READY pgs=712 cs=0 l=1 rev1=1 crypto rx=0x7f85d40119f0 tx=0x7f85d4008420 comp rx=0 tx=0).stop 2024-04-21T08:39:56.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.018+0000 7f85e43d1740 1 -- 172.21.15.86:0/1680816897 shutdown_connections 2024-04-21T08:39:56.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.018+0000 7f85e43d1740 1 --2- 172.21.15.86:0/1680816897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc6154a00 0x558fc6151bc0 unknown :-1 s=CLOSED pgs=712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.018+0000 7f85e43d1740 1 --2- 172.21.15.86:0/1680816897 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x558fc6164c80 0x558fc61670d0 crc :-1 s=READY pgs=36 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.018+0000 7f85e43d1740 1 --2- 172.21.15.86:0/1680816897 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558fc614be80 0x558fc6151660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.018+0000 7f85beff5640 0 client.16296 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:39:56.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.018+0000 7f85e43d1740 1 --2- 172.21.15.86:0/1680816897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558fc614b4d0 0x558fc61542d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.018+0000 7f85e43d1740 1 -- 172.21.15.86:0/1680816897 >> 172.21.15.86:0/1680816897 conn(0x558fc6145ba0 msgr2=0x558fc61478f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:56.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.018+0000 7f85e43d1740 1 -- 172.21.15.86:0/1680816897 shutdown_connections 2024-04-21T08:39:56.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.018+0000 7f85e43d1740 1 -- 172.21.15.86:0/1680816897 wait complete. 2024-04-21T08:39:56.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.018+0000 7f85e43d1740 20 client.16296 trim_cache size 0 max 16384 2024-04-21T08:39:56.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.031+0000 7f85e43d1740 1 Processor -- start 2024-04-21T08:39:56.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.031+0000 7f85e43d1740 1 -- start start 2024-04-21T08:39:56.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.031+0000 7f85e43d1740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558fc614b4d0 0x558fc6160830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.031+0000 7f85e43d1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558fc614be80 0x558fc6145f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.031+0000 7f85e43d1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc6154a00 0x558fc6153d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.031+0000 7f85e43d1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc1af2c0 con 0x558fc6154a00 2024-04-21T08:39:56.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.031+0000 7f85e43d1740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc1b0e00 con 0x558fc614b4d0 2024-04-21T08:39:56.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.031+0000 7f85e43d1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc1af990 con 0x558fc614be80 2024-04-21T08:39:56.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.031+0000 7f85db7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc6154a00 0x558fc6153d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.031+0000 7f85db7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc6154a00 0x558fc6153d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45474/0 (socket says 172.21.15.86:45474) 2024-04-21T08:39:56.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.031+0000 7f85db7fe640 1 -- 172.21.15.86:0/3692176643 learned_addr learned my addr 172.21.15.86:0/3692176643 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:56.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.031+0000 7f85daffd640 1 --2- 172.21.15.86:0/3692176643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558fc614b4d0 0x558fc6160830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.031+0000 7f85db7fe640 1 -- 172.21.15.86:0/3692176643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558fc614be80 msgr2=0x558fc6145f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.031+0000 7f85db7fe640 1 --2- 172.21.15.86:0/3692176643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558fc614be80 0x558fc6145f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.031+0000 7f85db7fe640 1 -- 172.21.15.86:0/3692176643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558fc614b4d0 msgr2=0x558fc6160830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.031+0000 7f85db7fe640 1 --2- 172.21.15.86:0/3692176643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558fc614b4d0 0x558fc6160830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.031+0000 7f85db7fe640 1 -- 172.21.15.86:0/3692176643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558fc6150610 con 0x558fc6154a00 2024-04-21T08:39:56.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.032+0000 7f85db7fe640 1 --2- 172.21.15.86:0/3692176643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc6154a00 0x558fc6153d90 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f85d4025b00 tx=0x7f85d4008c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5f25969fbd1ce0d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:56.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.032+0000 7f85c0ff9640 1 -- 172.21.15.86:0/3692176643 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4038070 con 0x558fc6154a00 2024-04-21T08:39:56.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.032+0000 7f85c0ff9640 1 -- 172.21.15.86:0/3692176643 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85d401c0b0 con 0x558fc6154a00 2024-04-21T08:39:56.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.032+0000 7f85c0ff9640 1 -- 172.21.15.86:0/3692176643 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4025de0 con 0x558fc6154a00 2024-04-21T08:39:56.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.032+0000 7f85e43d1740 1 -- 172.21.15.86:0/3692176643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc6154a00 msgr2=0x558fc6153d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.032+0000 7f85e43d1740 1 --2- 172.21.15.86:0/3692176643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc6154a00 0x558fc6153d90 secure :-1 s=READY pgs=713 cs=0 l=1 rev1=1 crypto rx=0x7f85d4025b00 tx=0x7f85d4008c00 comp rx=0 tx=0).stop 2024-04-21T08:39:56.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.032+0000 7f85e43d1740 1 -- 172.21.15.86:0/3692176643 shutdown_connections 2024-04-21T08:39:56.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.032+0000 7f85e43d1740 1 --2- 172.21.15.86:0/3692176643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc6154a00 0x558fc6153d90 unknown :-1 s=CLOSED pgs=713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.032+0000 7f85e43d1740 1 --2- 172.21.15.86:0/3692176643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558fc614be80 0x558fc6145f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.032+0000 7f85e43d1740 1 --2- 172.21.15.86:0/3692176643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558fc614b4d0 0x558fc6160830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.032+0000 7f85e43d1740 1 -- 172.21.15.86:0/3692176643 >> 172.21.15.86:0/3692176643 conn(0x558fc6145ba0 msgr2=0x558fc6144c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:56.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.032+0000 7f85e43d1740 1 -- 172.21.15.86:0/3692176643 shutdown_connections 2024-04-21T08:39:56.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.032+0000 7f85e43d1740 1 -- 172.21.15.86:0/3692176643 wait complete. 2024-04-21T08:39:56.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.033+0000 7f85e43d1740 1 -- start start 2024-04-21T08:39:56.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.033+0000 7f85e43d1740 1 Processor -- start 2024-04-21T08:39:56.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.033+0000 7f85e43d1740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558fc614b4d0 0x558fc6158990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.033+0000 7f85e43d1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558fc614be80 0x558fc6158ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.033+0000 7f85e43d1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc6154a00 0x558fc6159410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.033+0000 7f85e43d1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc1b0730 con 0x558fc6154a00 2024-04-21T08:39:56.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.033+0000 7f85e43d1740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc1b0060 con 0x558fc614b4d0 2024-04-21T08:39:56.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.033+0000 7f85e43d1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f85dc1ad280 con 0x558fc614be80 2024-04-21T08:39:56.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.033+0000 7f85e43d1740 10 client.0 _ll_register_callbacks cb 0x7f85d4011670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:56.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.033+0000 7f85db7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc6154a00 0x558fc6159410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.033+0000 7f85db7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc6154a00 0x558fc6159410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45482/0 (socket says 172.21.15.86:45482) 2024-04-21T08:39:56.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.033+0000 7f85db7fe640 1 -- 172.21.15.86:0/294931979 learned_addr learned my addr 172.21.15.86:0/294931979 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:56.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.033+0000 7f85da7fc640 1 --2- 172.21.15.86:0/294931979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558fc614be80 0x558fc6158ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.033+0000 7f85daffd640 1 --2- 172.21.15.86:0/294931979 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558fc614b4d0 0x558fc6158990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.033+0000 7f85db7fe640 1 -- 172.21.15.86:0/294931979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558fc614be80 msgr2=0x558fc6158ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.033+0000 7f85db7fe640 1 --2- 172.21.15.86:0/294931979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558fc614be80 0x558fc6158ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.033+0000 7f85db7fe640 1 -- 172.21.15.86:0/294931979 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558fc614b4d0 msgr2=0x558fc6158990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.033+0000 7f85db7fe640 1 --2- 172.21.15.86:0/294931979 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558fc614b4d0 0x558fc6158990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.033+0000 7f85db7fe640 1 -- 172.21.15.86:0/294931979 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558fc61ca450 con 0x558fc6154a00 2024-04-21T08:39:56.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.033+0000 7f85daffd640 1 --2- 172.21.15.86:0/294931979 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558fc614b4d0 0x558fc6158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:56.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.033+0000 7f85db7fe640 1 --2- 172.21.15.86:0/294931979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc6154a00 0x558fc6159410 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f85d400c010 tx=0x7f85d4008ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca677079f078515f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:56.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.033+0000 7f85beff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:56.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.034+0000 7f85beff5640 1 -- 172.21.15.86:0/294931979 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85d4042060 con 0x558fc6154a00 2024-04-21T08:39:56.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.034+0000 7f85beff5640 1 -- 172.21.15.86:0/294931979 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f85d401aa00 con 0x558fc6154a00 2024-04-21T08:39:56.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.034+0000 7f85e43d1740 1 -- 172.21.15.86:0/294931979 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85d401c0b0 con 0x558fc6154a00 2024-04-21T08:39:56.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.034+0000 7f85beff5640 1 -- 172.21.15.86:0/294931979 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f85d4049050 con 0x558fc6154a00 2024-04-21T08:39:56.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.034+0000 7f85e43d1740 10 client.16302 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:56.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.034+0000 7f85e43d1740 1 -- 172.21.15.86:0/294931979 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558fc61cb730 con 0x558fc6154a00 2024-04-21T08:39:56.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.034+0000 7f85beff5640 1 -- 172.21.15.86:0/294931979 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85d401c0b0 con 0x558fc6154a00 2024-04-21T08:39:56.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.034+0000 7f85beff5640 1 -- 172.21.15.86:0/294931979 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f85d4023070 con 0x558fc6154a00 2024-04-21T08:39:56.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.034+0000 7f85e43d1740 10 client.16302 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:56.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.034+0000 7f85e43d1740 10 client.16302 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:39:56.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.034+0000 7f85e43d1740 20 client.16302 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:56.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.034+0000 7f85e43d1740 1 --2- 172.21.15.86:0/294931979 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x558fc61cbcc0 0x558fc61ce0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.034+0000 7f85e43d1740 4 client.16302 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:39:56.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.034+0000 7f85e43d1740 1 -- 172.21.15.86:0/294931979 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558fc60bdc10 con 0x558fc61cbcc0 2024-04-21T08:39:56.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.034+0000 7f85daffd640 1 --2- 172.21.15.86:0/294931979 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x558fc61cbcc0 0x558fc61ce0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.035+0000 7f85beff5640 10 client.16302 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:39:56.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.035+0000 7f85daffd640 1 --2- 172.21.15.86:0/294931979 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x558fc61cbcc0 0x558fc61ce0e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=742582bf44c1893d server_cookie=4caa84f5eb55a659 in_seq=0 out_seq=0 2024-04-21T08:39:56.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.035+0000 7f85beff5640 1 -- 172.21.15.86:0/294931979 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558fc60bdc10 con 0x558fc61cbcc0 2024-04-21T08:39:56.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.035+0000 7f85beff5640 10 client.16302 handle_command_reply: tid=0 2024-04-21T08:39:56.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.035+0000 7f85e43d1740 1 client.16302 shutdown 2024-04-21T08:39:56.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.035+0000 7f85e43d1740 1 -- 172.21.15.86:0/294931979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc6154a00 msgr2=0x558fc6159410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.035+0000 7f85e43d1740 1 --2- 172.21.15.86:0/294931979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc6154a00 0x558fc6159410 secure :-1 s=READY pgs=714 cs=0 l=1 rev1=1 crypto rx=0x7f85d400c010 tx=0x7f85d4008ad0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.035+0000 7f85e43d1740 1 -- 172.21.15.86:0/294931979 shutdown_connections 2024-04-21T08:39:56.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.035+0000 7f85e43d1740 1 --2- 172.21.15.86:0/294931979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558fc6154a00 0x558fc6159410 unknown :-1 s=CLOSED pgs=714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.035+0000 7f85e43d1740 1 --2- 172.21.15.86:0/294931979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558fc614be80 0x558fc6158ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.035+0000 7f85e43d1740 1 --2- 172.21.15.86:0/294931979 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x558fc61cbcc0 0x558fc61ce0e0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.035+0000 7f85e43d1740 1 --2- 172.21.15.86:0/294931979 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558fc614b4d0 0x558fc6158990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.035+0000 7f85beff5640 0 client.16302 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:39:56.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.035+0000 7f85e43d1740 1 -- 172.21.15.86:0/294931979 >> 172.21.15.86:0/294931979 conn(0x558fc6145ba0 msgr2=0x558fc6144c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:56.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.035+0000 7f85e43d1740 1 -- 172.21.15.86:0/294931979 shutdown_connections 2024-04-21T08:39:56.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.035+0000 7f85e43d1740 1 -- 172.21.15.86:0/294931979 wait complete. 2024-04-21T08:39:56.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.035+0000 7f85e43d1740 20 client.16302 trim_cache size 0 max 16384 2024-04-21T08:39:56.041 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:39:56.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.036+0000 7f85e17e6640 1 -- 172.21.15.86:0/3044459354 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f85c40a3e40 msgr2=0x7f85c40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.036+0000 7f85e17e6640 1 --2- 172.21.15.86:0/3044459354 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f85c40a3e40 0x7f85c40a6300 secure :-1 s=READY pgs=98 cs=0 l=1 rev1=1 crypto rx=0x7f85d4005fd0 tx=0x7f85d4004a40 comp rx=0 tx=0).stop 2024-04-21T08:39:56.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.036+0000 7f85e17e6640 1 -- 172.21.15.86:0/3044459354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85dc06a840 msgr2=0x7f85dc19e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.036+0000 7f85e17e6640 1 --2- 172.21.15.86:0/3044459354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85dc06a840 0x7f85dc19e3e0 secure :-1 s=READY pgs=710 cs=0 l=1 rev1=1 crypto rx=0x7f85cc005940 tx=0x7f85cc02c7e0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.036+0000 7f85e17e6640 1 -- 172.21.15.86:0/3044459354 shutdown_connections 2024-04-21T08:39:56.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.036+0000 7f85e17e6640 1 --2- 172.21.15.86:0/3044459354 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f85c40a3e40 0x7f85c40a6300 unknown :-1 s=CLOSED pgs=98 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.036+0000 7f85e17e6640 1 --2- 172.21.15.86:0/3044459354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85dc198590 0x7f85dc198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.036+0000 7f85e17e6640 1 --2- 172.21.15.86:0/3044459354 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85dc107270 0x7f85dc19e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.036+0000 7f85e17e6640 1 --2- 172.21.15.86:0/3044459354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85dc06a840 0x7f85dc19e3e0 unknown :-1 s=CLOSED pgs=710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.036+0000 7f85e17e6640 1 -- 172.21.15.86:0/3044459354 >> 172.21.15.86:0/3044459354 conn(0x7f85dc077eb0 msgr2=0x7f85dc108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:56.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.036+0000 7f85e17e6640 1 -- 172.21.15.86:0/3044459354 shutdown_connections 2024-04-21T08:39:56.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.036+0000 7f85e17e6640 1 -- 172.21.15.86:0/3044459354 wait complete. 2024-04-21T08:39:56.045 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:39:56.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.146+0000 7f74bb152640 1 -- 172.21.15.86:0/2986399945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74b4101a50 msgr2=0x7f74b4105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.146+0000 7f74bb152640 1 --2- 172.21.15.86:0/2986399945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74b4101a50 0x7f74b4105a90 secure :-1 s=READY pgs=715 cs=0 l=1 rev1=1 crypto rx=0x7f74a0009900 tx=0x7f74a002bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:56.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.146+0000 7f74bb152640 1 -- 172.21.15.86:0/2986399945 shutdown_connections 2024-04-21T08:39:56.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.146+0000 7f74bb152640 1 --2- 172.21.15.86:0/2986399945 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74b4106170 0x7f74b410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.146+0000 7f74bb152640 1 --2- 172.21.15.86:0/2986399945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74b4101a50 0x7f74b4105a90 unknown :-1 s=CLOSED pgs=715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.146+0000 7f74bb152640 1 --2- 172.21.15.86:0/2986399945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74b41010a0 0x7f74b4101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.146+0000 7f74bb152640 1 -- 172.21.15.86:0/2986399945 >> 172.21.15.86:0/2986399945 conn(0x7f74b4077eb0 msgr2=0x7f74b40782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:56.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.146+0000 7f74bb152640 1 -- 172.21.15.86:0/2986399945 shutdown_connections 2024-04-21T08:39:56.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.146+0000 7f74bb152640 1 -- 172.21.15.86:0/2986399945 wait complete. 2024-04-21T08:39:56.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.147+0000 7f74bb152640 1 Processor -- start 2024-04-21T08:39:56.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.147+0000 7f74bb152640 1 -- start start 2024-04-21T08:39:56.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.147+0000 7f74bb152640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74b41010a0 0x7f74b4197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.147+0000 7f74bb152640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74b4106170 0x7f74b4197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.147+0000 7f74bb152640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74b419bef0 0x7f74b419c3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.147+0000 7f74bb152640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f74b41110b0 con 0x7f74b41010a0 2024-04-21T08:39:56.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.147+0000 7f74bb152640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f74b4110f30 con 0x7f74b4106170 2024-04-21T08:39:56.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.147+0000 7f74bb152640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f74b4111230 con 0x7f74b419bef0 2024-04-21T08:39:56.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.147+0000 7f74b8eee640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74b41010a0 0x7f74b4197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.147+0000 7f74b8eee640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74b41010a0 0x7f74b4197630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45490/0 (socket says 172.21.15.86:45490) 2024-04-21T08:39:56.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.147+0000 7f74b8eee640 1 -- 172.21.15.86:0/2434696424 learned_addr learned my addr 172.21.15.86:0/2434696424 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:56.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.147+0000 7f74b3fff640 1 --2- 172.21.15.86:0/2434696424 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74b4106170 0x7f74b4197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.147+0000 7f74b96ef640 1 --2- 172.21.15.86:0/2434696424 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74b419bef0 0x7f74b419c3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.147+0000 7f74b8eee640 1 -- 172.21.15.86:0/2434696424 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74b419bef0 msgr2=0x7f74b419c3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.147+0000 7f74b8eee640 1 --2- 172.21.15.86:0/2434696424 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74b419bef0 0x7f74b419c3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.147+0000 7f74b8eee640 1 -- 172.21.15.86:0/2434696424 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74b4106170 msgr2=0x7f74b4197b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.147+0000 7f74b8eee640 1 --2- 172.21.15.86:0/2434696424 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74b4106170 0x7f74b4197b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.147+0000 7f74b8eee640 1 -- 172.21.15.86:0/2434696424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f74b419c9e0 con 0x7f74b41010a0 2024-04-21T08:39:56.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.147+0000 7f74b3fff640 1 --2- 172.21.15.86:0/2434696424 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74b4106170 0x7f74b4197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:56.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.147+0000 7f74b96ef640 1 --2- 172.21.15.86:0/2434696424 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74b419bef0 0x7f74b419c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:56.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.148+0000 7f74b8eee640 1 --2- 172.21.15.86:0/2434696424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74b41010a0 0x7f74b4197630 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f74a800d950 tx=0x7f74a800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:56.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.148+0000 7f74b1ffb640 1 -- 172.21.15.86:0/2434696424 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a8017030 con 0x7f74b41010a0 2024-04-21T08:39:56.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.148+0000 7f74b1ffb640 1 -- 172.21.15.86:0/2434696424 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74a8004020 con 0x7f74b41010a0 2024-04-21T08:39:56.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.148+0000 7f74bb152640 1 -- 172.21.15.86:0/2434696424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f74b419ccd0 con 0x7f74b41010a0 2024-04-21T08:39:56.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.148+0000 7f74b1ffb640 1 -- 172.21.15.86:0/2434696424 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a80047a0 con 0x7f74b41010a0 2024-04-21T08:39:56.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.148+0000 7f74bb152640 1 -- 172.21.15.86:0/2434696424 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74b41a4490 con 0x7f74b41010a0 2024-04-21T08:39:56.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.150+0000 7f74b1ffb640 1 -- 172.21.15.86:0/2434696424 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f74a8009a30 con 0x7f74b41010a0 2024-04-21T08:39:56.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.151+0000 7f74bde27740 1 Processor -- start 2024-04-21T08:39:56.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.151+0000 7f74bde27740 1 -- start start 2024-04-21T08:39:56.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.151+0000 7f74b1ffb640 1 --2- 172.21.15.86:0/2434696424 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f74a40a3e40 0x7f74a40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.151+0000 7f74bde27740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1e508ccc0 0x55a1e508d0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.151+0000 7f74bde27740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a1e508d670 0x55a1e5091580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.151+0000 7f74bde27740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1e5091ba0 0x55a1e5092020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.151+0000 7f74bde27740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a1e4fee1f0 con 0x55a1e508ccc0 2024-04-21T08:39:56.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.151+0000 7f74bde27740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55a1e4f8b4b0 con 0x55a1e508d670 2024-04-21T08:39:56.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.151+0000 7f74bde27740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a1e4fea1b0 con 0x55a1e5091ba0 2024-04-21T08:39:56.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.151+0000 7f74b8eee640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1e508ccc0 0x55a1e508d0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.151+0000 7f74b96ef640 1 --2- 172.21.15.86:0/2434696424 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f74a40a3e40 0x7f74a40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.151+0000 7f74b8eee640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1e508ccc0 0x55a1e508d0a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45496/0 (socket says 172.21.15.86:45496) 2024-04-21T08:39:56.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.151+0000 7f74b8eee640 1 -- 172.21.15.86:0/901937482 learned_addr learned my addr 172.21.15.86:0/901937482 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:56.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.151+0000 7f74b1ffb640 1 -- 172.21.15.86:0/2434696424 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f74a80d0910 con 0x7f74b41010a0 2024-04-21T08:39:56.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.151+0000 7f74b96ef640 1 --2- 172.21.15.86:0/901937482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1e5091ba0 0x55a1e5092020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.151+0000 7f74b3fff640 1 --2- 172.21.15.86:0/901937482 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a1e508d670 0x55a1e5091580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.151+0000 7f74b8eee640 1 -- 172.21.15.86:0/901937482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1e5091ba0 msgr2=0x55a1e5092020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.151+0000 7f74b8eee640 1 --2- 172.21.15.86:0/901937482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1e5091ba0 0x55a1e5092020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.151+0000 7f74b8eee640 1 -- 172.21.15.86:0/901937482 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a1e508d670 msgr2=0x55a1e5091580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.151+0000 7f74b8eee640 1 --2- 172.21.15.86:0/901937482 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a1e508d670 0x55a1e5091580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.151+0000 7f74b8eee640 1 -- 172.21.15.86:0/901937482 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1e4f7d1b0 con 0x55a1e508ccc0 2024-04-21T08:39:56.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.151+0000 7f74b8eee640 1 --2- 172.21.15.86:0/901937482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1e508ccc0 0x55a1e508d0a0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f74a8098750 tx=0x7f74a8004550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d31979f78eca5111 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:56.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.151+0000 7f74b96ef640 1 --2- 172.21.15.86:0/2434696424 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f74a40a3e40 0x7f74a40a6300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f74ac005fd0 tx=0x7f74ac008860 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:56.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.152+0000 7f749effd640 1 -- 172.21.15.86:0/901937482 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a8096e60 con 0x55a1e508ccc0 2024-04-21T08:39:56.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.152+0000 7f749effd640 1 -- 172.21.15.86:0/901937482 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74a809c2f0 con 0x55a1e508ccc0 2024-04-21T08:39:56.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.152+0000 7f749effd640 1 -- 172.21.15.86:0/901937482 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a80a4f60 con 0x55a1e508ccc0 2024-04-21T08:39:56.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.152+0000 7f74bde27740 1 -- 172.21.15.86:0/901937482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1e508ccc0 msgr2=0x55a1e508d0a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.152+0000 7f74bde27740 1 --2- 172.21.15.86:0/901937482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1e508ccc0 0x55a1e508d0a0 secure :-1 s=READY pgs=717 cs=0 l=1 rev1=1 crypto rx=0x7f74a8098750 tx=0x7f74a8004550 comp rx=0 tx=0).stop 2024-04-21T08:39:56.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.152+0000 7f74bde27740 1 -- 172.21.15.86:0/901937482 shutdown_connections 2024-04-21T08:39:56.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.152+0000 7f74bde27740 1 --2- 172.21.15.86:0/901937482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1e5091ba0 0x55a1e5092020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.152+0000 7f74bde27740 1 --2- 172.21.15.86:0/901937482 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a1e508d670 0x55a1e5091580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.152+0000 7f74bde27740 1 --2- 172.21.15.86:0/901937482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1e508ccc0 0x55a1e508d0a0 unknown :-1 s=CLOSED pgs=717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.152+0000 7f74bde27740 1 -- 172.21.15.86:0/901937482 >> 172.21.15.86:0/901937482 conn(0x55a1e5083690 msgr2=0x55a1e5083a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:56.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.152+0000 7f74bde27740 1 -- 172.21.15.86:0/901937482 shutdown_connections 2024-04-21T08:39:56.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.152+0000 7f74bde27740 1 -- 172.21.15.86:0/901937482 wait complete. 2024-04-21T08:39:56.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.153+0000 7f74bde27740 1 -- start start 2024-04-21T08:39:56.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.153+0000 7f74bde27740 1 Processor -- start 2024-04-21T08:39:56.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.153+0000 7f74bde27740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a1e508ccc0 0x55a1e5099270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.153+0000 7f74bde27740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1e508d670 0x55a1e5096670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.153+0000 7f74bde27740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1e5091ba0 0x55a1e5096ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.153+0000 7f74bde27740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a1e5099d90 con 0x55a1e508d670 2024-04-21T08:39:56.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.153+0000 7f74bde27740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55a1e4fa7500 con 0x55a1e508ccc0 2024-04-21T08:39:56.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.153+0000 7f74bde27740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a1e5099f10 con 0x55a1e5091ba0 2024-04-21T08:39:56.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.153+0000 7f74bde27740 10 client.0 _ll_register_callbacks cb 0x55a1e4f75b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:56.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.153+0000 7f74b3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1e508d670 0x55a1e5096670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.153+0000 7f74b3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1e508d670 0x55a1e5096670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45504/0 (socket says 172.21.15.86:45504) 2024-04-21T08:39:56.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.153+0000 7f74b3fff640 1 -- 172.21.15.86:0/231740281 learned_addr learned my addr 172.21.15.86:0/231740281 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:56.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.153+0000 7f74b8eee640 1 --2- 172.21.15.86:0/231740281 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a1e508ccc0 0x55a1e5099270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.153+0000 7f74b96ef640 1 --2- 172.21.15.86:0/231740281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1e5091ba0 0x55a1e5096ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.153+0000 7f74b3fff640 1 -- 172.21.15.86:0/231740281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1e5091ba0 msgr2=0x55a1e5096ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.153+0000 7f74b3fff640 1 --2- 172.21.15.86:0/231740281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1e5091ba0 0x55a1e5096ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.153+0000 7f74b3fff640 1 -- 172.21.15.86:0/231740281 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a1e508ccc0 msgr2=0x55a1e5099270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.153+0000 7f74b3fff640 1 --2- 172.21.15.86:0/231740281 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a1e508ccc0 0x55a1e5099270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.153+0000 7f74b3fff640 1 -- 172.21.15.86:0/231740281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1e4d003f0 con 0x55a1e508d670 2024-04-21T08:39:56.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.154+0000 7f74b3fff640 1 --2- 172.21.15.86:0/231740281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1e508d670 0x55a1e5096670 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f74a0000c00 tx=0x7f74a002ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f4f654b6dc9ae90 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:56.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.154+0000 7f7493fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:56.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.154+0000 7f7493fff640 1 -- 172.21.15.86:0/231740281 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a0005450 con 0x55a1e508d670 2024-04-21T08:39:56.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.154+0000 7f7493fff640 1 -- 172.21.15.86:0/231740281 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f74a00046c0 con 0x55a1e508d670 2024-04-21T08:39:56.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.154+0000 7f74bde27740 1 -- 172.21.15.86:0/231740281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74a809c2f0 con 0x55a1e508d670 2024-04-21T08:39:56.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.154+0000 7f7493fff640 1 -- 172.21.15.86:0/231740281 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f74a00055f0 con 0x55a1e508d670 2024-04-21T08:39:56.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.154+0000 7f74bde27740 10 client.16314 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:56.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.154+0000 7f74bde27740 1 -- 172.21.15.86:0/231740281 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a1e4dca7d0 con 0x55a1e508d670 2024-04-21T08:39:56.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.154+0000 7f7493fff640 1 -- 172.21.15.86:0/231740281 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74a809c2f0 con 0x55a1e508d670 2024-04-21T08:39:56.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.154+0000 7f7493fff640 1 -- 172.21.15.86:0/231740281 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f74a00037b0 con 0x55a1e508d670 2024-04-21T08:39:56.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.154+0000 7f74bde27740 10 client.16314 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:56.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.154+0000 7f74bde27740 10 client.16314 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:39:56.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.154+0000 7f74bde27740 20 client.16314 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:56.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.154+0000 7f74bde27740 1 --2- 172.21.15.86:0/231740281 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55a1e50a70d0 0x55a1e50a9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.154+0000 7f74bde27740 4 client.16314 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:56.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.154+0000 7f74bde27740 1 -- 172.21.15.86:0/231740281 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a1e4f8d930 con 0x55a1e50a70d0 2024-04-21T08:39:56.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.155+0000 7f74b8eee640 1 --2- 172.21.15.86:0/231740281 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55a1e50a70d0 0x55a1e50a9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.155+0000 7f74b8eee640 1 --2- 172.21.15.86:0/231740281 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55a1e50a70d0 0x55a1e50a9510 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5ec32f6145ae4d6 server_cookie=f4bedb1269499808 in_seq=0 out_seq=0 2024-04-21T08:39:56.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.155+0000 7f7493fff640 10 client.16314 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:39:56.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.156+0000 7f7493fff640 1 -- 172.21.15.86:0/231740281 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a1e4f8d930 con 0x55a1e50a70d0 2024-04-21T08:39:56.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.156+0000 7f7493fff640 10 client.16314 handle_command_reply: tid=0 2024-04-21T08:39:56.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.156+0000 7f74bde27740 1 client.16314 shutdown 2024-04-21T08:39:56.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.156+0000 7f74bde27740 1 -- 172.21.15.86:0/231740281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1e508d670 msgr2=0x55a1e5096670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.156+0000 7f74bde27740 1 --2- 172.21.15.86:0/231740281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1e508d670 0x55a1e5096670 secure :-1 s=READY pgs=718 cs=0 l=1 rev1=1 crypto rx=0x7f74a0000c00 tx=0x7f74a002ca90 comp rx=0 tx=0).stop 2024-04-21T08:39:56.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.156+0000 7f74bde27740 1 -- 172.21.15.86:0/231740281 shutdown_connections 2024-04-21T08:39:56.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.156+0000 7f74bde27740 1 --2- 172.21.15.86:0/231740281 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55a1e50a70d0 0x55a1e50a9510 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.156+0000 7f74bde27740 1 --2- 172.21.15.86:0/231740281 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1e5091ba0 0x55a1e5096ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.156+0000 7f74bde27740 1 --2- 172.21.15.86:0/231740281 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1e508d670 0x55a1e5096670 unknown :-1 s=CLOSED pgs=718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.156+0000 7f7493fff640 0 client.16314 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:39:56.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.156+0000 7f74bde27740 1 --2- 172.21.15.86:0/231740281 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a1e508ccc0 0x55a1e5099270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.156+0000 7f74bde27740 1 -- 172.21.15.86:0/231740281 >> 172.21.15.86:0/231740281 conn(0x55a1e5083690 msgr2=0x55a1e5092f20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:56.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.156+0000 7f74bde27740 1 -- 172.21.15.86:0/231740281 shutdown_connections 2024-04-21T08:39:56.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.156+0000 7f74bde27740 1 -- 172.21.15.86:0/231740281 wait complete. 2024-04-21T08:39:56.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.156+0000 7f74bde27740 20 client.16314 trim_cache size 0 max 16384 2024-04-21T08:39:56.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.169+0000 7f74bde27740 1 Processor -- start 2024-04-21T08:39:56.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.169+0000 7f74bde27740 1 -- start start 2024-04-21T08:39:56.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.170+0000 7f74bde27740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1e508ccc0 0x55a1e50a2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.170+0000 7f74bde27740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a1e508d670 0x55a1e5091340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.170+0000 7f74bde27740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1e5091ba0 0x55a1e50aede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.170+0000 7f74bde27740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f74b41ad0f0 con 0x55a1e5091ba0 2024-04-21T08:39:56.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.170+0000 7f74bde27740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f74b41aec30 con 0x55a1e508d670 2024-04-21T08:39:56.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.170+0000 7f74bde27740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f74b41ad7c0 con 0x55a1e508ccc0 2024-04-21T08:39:56.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.170+0000 7f74b96ef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1e5091ba0 0x55a1e50aede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.170+0000 7f74b96ef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1e5091ba0 0x55a1e50aede0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45516/0 (socket says 172.21.15.86:45516) 2024-04-21T08:39:56.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.170+0000 7f74b96ef640 1 -- 172.21.15.86:0/2388212332 learned_addr learned my addr 172.21.15.86:0/2388212332 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:56.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.170+0000 7f74b3fff640 1 --2- 172.21.15.86:0/2388212332 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a1e508d670 0x55a1e5091340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.170+0000 7f74b8eee640 1 --2- 172.21.15.86:0/2388212332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1e508ccc0 0x55a1e50a2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.170+0000 7f74b96ef640 1 -- 172.21.15.86:0/2388212332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1e508ccc0 msgr2=0x55a1e50a2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.170+0000 7f74b96ef640 1 --2- 172.21.15.86:0/2388212332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1e508ccc0 0x55a1e50a2c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.170+0000 7f74b96ef640 1 -- 172.21.15.86:0/2388212332 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a1e508d670 msgr2=0x55a1e5091340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.170+0000 7f74b96ef640 1 --2- 172.21.15.86:0/2388212332 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a1e508d670 0x55a1e5091340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.170+0000 7f74b96ef640 1 -- 172.21.15.86:0/2388212332 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a1e5094bc0 con 0x55a1e5091ba0 2024-04-21T08:39:56.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.170+0000 7f74b3fff640 1 --2- 172.21.15.86:0/2388212332 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a1e508d670 0x55a1e5091340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:56.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.170+0000 7f74b8eee640 1 --2- 172.21.15.86:0/2388212332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1e508ccc0 0x55a1e50a2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:56.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.170+0000 7f74b96ef640 1 --2- 172.21.15.86:0/2388212332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1e5091ba0 0x55a1e50aede0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f74ac008680 tx=0x7f74ac017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46123b8915bd9f13 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:56.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.170+0000 7f749e7fc640 1 -- 172.21.15.86:0/2388212332 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac0142d0 con 0x55a1e5091ba0 2024-04-21T08:39:56.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.170+0000 7f749e7fc640 1 -- 172.21.15.86:0/2388212332 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74ac012d10 con 0x55a1e5091ba0 2024-04-21T08:39:56.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.170+0000 7f749e7fc640 1 -- 172.21.15.86:0/2388212332 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74ac00ecb0 con 0x55a1e5091ba0 2024-04-21T08:39:56.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.171+0000 7f74bde27740 1 -- 172.21.15.86:0/2388212332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1e5091ba0 msgr2=0x55a1e50aede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.171+0000 7f74bde27740 1 --2- 172.21.15.86:0/2388212332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1e5091ba0 0x55a1e50aede0 secure :-1 s=READY pgs=719 cs=0 l=1 rev1=1 crypto rx=0x7f74ac008680 tx=0x7f74ac017000 comp rx=0 tx=0).stop 2024-04-21T08:39:56.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.171+0000 7f74bde27740 1 -- 172.21.15.86:0/2388212332 shutdown_connections 2024-04-21T08:39:56.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.171+0000 7f74bde27740 1 --2- 172.21.15.86:0/2388212332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1e5091ba0 0x55a1e50aede0 unknown :-1 s=CLOSED pgs=719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.171+0000 7f74bde27740 1 --2- 172.21.15.86:0/2388212332 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a1e508d670 0x55a1e5091340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.171+0000 7f74bde27740 1 --2- 172.21.15.86:0/2388212332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1e508ccc0 0x55a1e50a2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.171+0000 7f74bde27740 1 -- 172.21.15.86:0/2388212332 >> 172.21.15.86:0/2388212332 conn(0x55a1e5083690 msgr2=0x55a1e5090c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:56.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.171+0000 7f74bde27740 1 -- 172.21.15.86:0/2388212332 shutdown_connections 2024-04-21T08:39:56.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.171+0000 7f74bde27740 1 -- 172.21.15.86:0/2388212332 wait complete. 2024-04-21T08:39:56.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.171+0000 7f74bde27740 1 -- start start 2024-04-21T08:39:56.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.171+0000 7f74bde27740 1 Processor -- start 2024-04-21T08:39:56.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.172+0000 7f74bde27740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1e508ccc0 0x55a1e509f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.172+0000 7f74bde27740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1e508d670 0x55a1e50881e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.172+0000 7f74bde27740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a1e5091ba0 0x55a1e5088720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.172+0000 7f74bde27740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f74b41ae560 con 0x55a1e508d670 2024-04-21T08:39:56.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.172+0000 7f74bde27740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f74b41ade90 con 0x55a1e5091ba0 2024-04-21T08:39:56.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.172+0000 7f74bde27740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f74b41ab0b0 con 0x55a1e508ccc0 2024-04-21T08:39:56.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.172+0000 7f74bde27740 10 client.0 _ll_register_callbacks cb 0x7f74a0004540 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:56.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.172+0000 7f74b3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1e508d670 0x55a1e50881e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.172+0000 7f74b3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1e508d670 0x55a1e50881e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45528/0 (socket says 172.21.15.86:45528) 2024-04-21T08:39:56.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.172+0000 7f74b3fff640 1 -- 172.21.15.86:0/648055617 learned_addr learned my addr 172.21.15.86:0/648055617 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:56.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.172+0000 7f74b3fff640 1 -- 172.21.15.86:0/648055617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1e508ccc0 msgr2=0x55a1e509f560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.172+0000 7f74b3fff640 1 --2- 172.21.15.86:0/648055617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1e508ccc0 0x55a1e509f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.172+0000 7f74b3fff640 1 -- 172.21.15.86:0/648055617 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a1e5091ba0 msgr2=0x55a1e5088720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.172+0000 7f74b3fff640 1 --2- 172.21.15.86:0/648055617 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a1e5091ba0 0x55a1e5088720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.172+0000 7f74b3fff640 1 -- 172.21.15.86:0/648055617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1e5088f30 con 0x55a1e508d670 2024-04-21T08:39:56.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.172+0000 7f7493fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:56.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.172+0000 7f74b3fff640 1 --2- 172.21.15.86:0/648055617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1e508d670 0x55a1e50881e0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f74a0006010 tx=0x7f74a0005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f8259a02f8b1de8a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:56.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.172+0000 7f7493fff640 1 -- 172.21.15.86:0/648055617 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74a002c5d0 con 0x55a1e508d670 2024-04-21T08:39:56.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.172+0000 7f7493fff640 1 -- 172.21.15.86:0/648055617 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f74a00055f0 con 0x55a1e508d670 2024-04-21T08:39:56.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.172+0000 7f74bde27740 1 -- 172.21.15.86:0/648055617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74ac012d10 con 0x55a1e508d670 2024-04-21T08:39:56.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.172+0000 7f7493fff640 1 -- 172.21.15.86:0/648055617 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f74a0062050 con 0x55a1e508d670 2024-04-21T08:39:56.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.172+0000 7f74bde27740 10 client.16320 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:56.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.172+0000 7f74bde27740 1 -- 172.21.15.86:0/648055617 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a1e5111e40 con 0x55a1e508d670 2024-04-21T08:39:56.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.173+0000 7f7493fff640 1 -- 172.21.15.86:0/648055617 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74ac012d10 con 0x55a1e508d670 2024-04-21T08:39:56.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.173+0000 7f7493fff640 1 -- 172.21.15.86:0/648055617 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f74a005d030 con 0x55a1e508d670 2024-04-21T08:39:56.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.173+0000 7f74bde27740 10 client.16320 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:56.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.173+0000 7f74bde27740 10 client.16320 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:39:56.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.173+0000 7f74bde27740 20 client.16320 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:56.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.173+0000 7f74bde27740 1 --2- 172.21.15.86:0/648055617 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55a1e5112510 0x55a1e5114930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.173+0000 7f74bde27740 4 client.16320 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:39:56.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.173+0000 7f74bde27740 1 -- 172.21.15.86:0/648055617 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a1e4ffbc10 con 0x55a1e5112510 2024-04-21T08:39:56.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.173+0000 7f74b8eee640 1 --2- 172.21.15.86:0/648055617 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55a1e5112510 0x55a1e5114930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.173+0000 7f7493fff640 10 client.16320 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:39:56.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.173+0000 7f74b8eee640 1 --2- 172.21.15.86:0/648055617 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55a1e5112510 0x55a1e5114930 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9a23f6d6f5b7c8cd server_cookie=c8122f01a352b8a5 in_seq=0 out_seq=0 2024-04-21T08:39:56.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.174+0000 7f7493fff640 1 -- 172.21.15.86:0/648055617 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a1e4ffbc10 con 0x55a1e5112510 2024-04-21T08:39:56.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.174+0000 7f7493fff640 10 client.16320 handle_command_reply: tid=0 2024-04-21T08:39:56.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.174+0000 7f74bde27740 1 client.16320 shutdown 2024-04-21T08:39:56.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.174+0000 7f74bde27740 1 -- 172.21.15.86:0/648055617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1e508d670 msgr2=0x55a1e50881e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.174+0000 7f74bde27740 1 --2- 172.21.15.86:0/648055617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1e508d670 0x55a1e50881e0 secure :-1 s=READY pgs=720 cs=0 l=1 rev1=1 crypto rx=0x7f74a0006010 tx=0x7f74a0005e80 comp rx=0 tx=0).stop 2024-04-21T08:39:56.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.174+0000 7f74bde27740 1 -- 172.21.15.86:0/648055617 shutdown_connections 2024-04-21T08:39:56.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.174+0000 7f74bde27740 1 --2- 172.21.15.86:0/648055617 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55a1e5112510 0x55a1e5114930 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.174+0000 7f74bde27740 1 --2- 172.21.15.86:0/648055617 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a1e5091ba0 0x55a1e5088720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.174+0000 7f74bde27740 1 --2- 172.21.15.86:0/648055617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a1e508d670 0x55a1e50881e0 unknown :-1 s=CLOSED pgs=720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.174+0000 7f7493fff640 0 client.16320 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:39:56.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.174+0000 7f74bde27740 1 --2- 172.21.15.86:0/648055617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a1e508ccc0 0x55a1e509f560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.174+0000 7f74bde27740 1 -- 172.21.15.86:0/648055617 >> 172.21.15.86:0/648055617 conn(0x55a1e5083690 msgr2=0x55a1e508bac0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:56.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.174+0000 7f74bde27740 1 -- 172.21.15.86:0/648055617 shutdown_connections 2024-04-21T08:39:56.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.174+0000 7f74bde27740 1 -- 172.21.15.86:0/648055617 wait complete. 2024-04-21T08:39:56.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.174+0000 7f74bde27740 20 client.16320 trim_cache size 0 max 16384 2024-04-21T08:39:56.180 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:39:56.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.174+0000 7f74bb152640 1 -- 172.21.15.86:0/2434696424 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f74a40a3e40 msgr2=0x7f74a40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.174+0000 7f74bb152640 1 --2- 172.21.15.86:0/2434696424 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f74a40a3e40 0x7f74a40a6300 secure :-1 s=READY pgs=99 cs=0 l=1 rev1=1 crypto rx=0x7f74ac005fd0 tx=0x7f74ac008860 comp rx=0 tx=0).stop 2024-04-21T08:39:56.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.174+0000 7f74bb152640 1 -- 172.21.15.86:0/2434696424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74b41010a0 msgr2=0x7f74b4197630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.174+0000 7f74bb152640 1 --2- 172.21.15.86:0/2434696424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74b41010a0 0x7f74b4197630 secure :-1 s=READY pgs=716 cs=0 l=1 rev1=1 crypto rx=0x7f74a800d950 tx=0x7f74a800de10 comp rx=0 tx=0).stop 2024-04-21T08:39:56.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.175+0000 7f74bb152640 1 -- 172.21.15.86:0/2434696424 shutdown_connections 2024-04-21T08:39:56.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.175+0000 7f74bb152640 1 --2- 172.21.15.86:0/2434696424 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f74a40a3e40 0x7f74a40a6300 unknown :-1 s=CLOSED pgs=99 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.175+0000 7f74bb152640 1 --2- 172.21.15.86:0/2434696424 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74b419bef0 0x7f74b419c3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.175+0000 7f74bb152640 1 --2- 172.21.15.86:0/2434696424 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74b4106170 0x7f74b4197b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.175+0000 7f74bb152640 1 --2- 172.21.15.86:0/2434696424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74b41010a0 0x7f74b4197630 unknown :-1 s=CLOSED pgs=716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.175+0000 7f74bb152640 1 -- 172.21.15.86:0/2434696424 >> 172.21.15.86:0/2434696424 conn(0x7f74b4077eb0 msgr2=0x7f74b40ff050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:56.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.175+0000 7f74bb152640 1 -- 172.21.15.86:0/2434696424 shutdown_connections 2024-04-21T08:39:56.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.175+0000 7f74bb152640 1 -- 172.21.15.86:0/2434696424 wait complete. 2024-04-21T08:39:56.184 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:39:56.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.285+0000 7f6a8973b640 1 -- 172.21.15.86:0/3372161258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a84101a40 msgr2=0x7f6a84105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.285+0000 7f6a8973b640 1 --2- 172.21.15.86:0/3372161258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a84101a40 0x7f6a84105a80 secure :-1 s=READY pgs=721 cs=0 l=1 rev1=1 crypto rx=0x7f6a78009900 tx=0x7f6a7802bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:56.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.286+0000 7f6a8973b640 1 -- 172.21.15.86:0/3372161258 shutdown_connections 2024-04-21T08:39:56.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.286+0000 7f6a8973b640 1 --2- 172.21.15.86:0/3372161258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a84106160 0x7f6a8410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.286+0000 7f6a8973b640 1 --2- 172.21.15.86:0/3372161258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a84101a40 0x7f6a84105a80 unknown :-1 s=CLOSED pgs=721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.286+0000 7f6a8973b640 1 --2- 172.21.15.86:0/3372161258 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a84101090 0x7f6a84101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.286+0000 7f6a8973b640 1 -- 172.21.15.86:0/3372161258 >> 172.21.15.86:0/3372161258 conn(0x7f6a84077f50 msgr2=0x7f6a84078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:56.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.286+0000 7f6a8973b640 1 -- 172.21.15.86:0/3372161258 shutdown_connections 2024-04-21T08:39:56.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.286+0000 7f6a8973b640 1 -- 172.21.15.86:0/3372161258 wait complete. 2024-04-21T08:39:56.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.286+0000 7f6a8973b640 1 Processor -- start 2024-04-21T08:39:56.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.286+0000 7f6a8973b640 1 -- start start 2024-04-21T08:39:56.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.287+0000 7f6a8973b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a84101090 0x7f6a84102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.287+0000 7f6a8973b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a84106160 0x7f6a84103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.287+0000 7f6a8973b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a841035a0 0x7f6a841a6940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.287+0000 7f6a8973b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6a84111080 con 0x7f6a841035a0 2024-04-21T08:39:56.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.287+0000 7f6a8973b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6a84110f00 con 0x7f6a84106160 2024-04-21T08:39:56.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.287+0000 7f6a8973b640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6a84111200 con 0x7f6a84101090 2024-04-21T08:39:56.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.287+0000 7f6a837fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a841035a0 0x7f6a841a6940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.287+0000 7f6a827fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a84106160 0x7f6a84103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.287+0000 7f6a82ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a84101090 0x7f6a84102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.287+0000 7f6a827fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a84106160 0x7f6a84103060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.86:46694/0 (socket says 172.21.15.86:46694) 2024-04-21T08:39:56.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.287+0000 7f6a827fc640 1 -- 172.21.15.86:0/3310338888 learned_addr learned my addr 172.21.15.86:0/3310338888 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:56.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.287+0000 7f6a837fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a841035a0 0x7f6a841a6940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45550/0 (socket says 172.21.15.86:45550) 2024-04-21T08:39:56.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.287+0000 7f6a837fe640 1 -- 172.21.15.86:0/3310338888 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a84101090 msgr2=0x7f6a84102b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.287+0000 7f6a837fe640 1 --2- 172.21.15.86:0/3310338888 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a84101090 0x7f6a84102b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.287+0000 7f6a837fe640 1 -- 172.21.15.86:0/3310338888 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a84106160 msgr2=0x7f6a84103060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.287+0000 7f6a837fe640 1 --2- 172.21.15.86:0/3310338888 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a84106160 0x7f6a84103060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.287+0000 7f6a837fe640 1 -- 172.21.15.86:0/3310338888 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a841a7040 con 0x7f6a841035a0 2024-04-21T08:39:56.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.287+0000 7f6a837fe640 1 --2- 172.21.15.86:0/3310338888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a841035a0 0x7f6a841a6940 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c00c710 tx=0x7f6a7c00cbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:56.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.287+0000 7f6a6bfff640 1 -- 172.21.15.86:0/3310338888 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c004220 con 0x7f6a841035a0 2024-04-21T08:39:56.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.288+0000 7f6a6bfff640 1 -- 172.21.15.86:0/3310338888 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a7c00ce20 con 0x7f6a841035a0 2024-04-21T08:39:56.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.288+0000 7f6a8973b640 1 -- 172.21.15.86:0/3310338888 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a841a7330 con 0x7f6a841035a0 2024-04-21T08:39:56.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.288+0000 7f6a6bfff640 1 -- 172.21.15.86:0/3310338888 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7c003d50 con 0x7f6a841035a0 2024-04-21T08:39:56.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.288+0000 7f6a8973b640 1 -- 172.21.15.86:0/3310338888 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a841a7830 con 0x7f6a841035a0 2024-04-21T08:39:56.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.290+0000 7f6a6bfff640 1 -- 172.21.15.86:0/3310338888 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f6a7c0079b0 con 0x7f6a841035a0 2024-04-21T08:39:56.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.290+0000 7f6a8c405740 1 Processor -- start 2024-04-21T08:39:56.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.290+0000 7f6a8c405740 1 -- start start 2024-04-21T08:39:56.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.291+0000 7f6a8c405740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e665cf4d0 0x564e665cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.291+0000 7f6a8c405740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564e665cfe80 0x564e665d8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.291+0000 7f6a8c405740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e665d8a00 0x564e665dae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.291+0000 7f6a8c405740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564e665341f0 con 0x564e665cf4d0 2024-04-21T08:39:56.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.291+0000 7f6a8c405740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564e664d14b0 con 0x564e665cfe80 2024-04-21T08:39:56.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.291+0000 7f6a8c405740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564e665301b0 con 0x564e665d8a00 2024-04-21T08:39:56.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.291+0000 7f6a827fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e665cf4d0 0x564e665cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.291+0000 7f6a827fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e665cf4d0 0x564e665cf8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45564/0 (socket says 172.21.15.86:45564) 2024-04-21T08:39:56.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.291+0000 7f6a827fc640 1 -- 172.21.15.86:0/2058718806 learned_addr learned my addr 172.21.15.86:0/2058718806 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:56.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.291+0000 7f6a82ffd640 1 --2- 172.21.15.86:0/2058718806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564e665cfe80 0x564e665d8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.291+0000 7f6a6bfff640 1 --2- 172.21.15.86:0/3310338888 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f6a6c0a3e70 0x7f6a6c0a6330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.291+0000 7f6a827fc640 1 --2- 172.21.15.86:0/2058718806 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e665d8a00 0x564e665dae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.291+0000 7f6a837fe640 1 --2- 172.21.15.86:0/3310338888 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f6a6c0a3e70 0x7f6a6c0a6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.291+0000 7f6a827fc640 1 -- 172.21.15.86:0/2058718806 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e665d8a00 msgr2=0x564e665dae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.291+0000 7f6a827fc640 1 --2- 172.21.15.86:0/2058718806 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e665d8a00 0x564e665dae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.291+0000 7f6a827fc640 1 -- 172.21.15.86:0/2058718806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564e665cfe80 msgr2=0x564e665d8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.291+0000 7f6a827fc640 1 --2- 172.21.15.86:0/2058718806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564e665cfe80 0x564e665d8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.291+0000 7f6a827fc640 1 -- 172.21.15.86:0/2058718806 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e664c31b0 con 0x564e665cf4d0 2024-04-21T08:39:56.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.291+0000 7f6a82ffd640 1 --2- 172.21.15.86:0/2058718806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564e665cfe80 0x564e665d8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:56.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.291+0000 7f6a6bfff640 1 -- 172.21.15.86:0/3310338888 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f6a7c0d0980 con 0x7f6a841035a0 2024-04-21T08:39:56.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.291+0000 7f6a827fc640 1 --2- 172.21.15.86:0/2058718806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e665cf4d0 0x564e665cf8b0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f6a78005ec0 tx=0x7f6a78005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7faabf374771f63 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:56.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.291+0000 7f6a697fa640 1 -- 172.21.15.86:0/2058718806 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a7802c9e0 con 0x564e665cf4d0 2024-04-21T08:39:56.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.291+0000 7f6a837fe640 1 --2- 172.21.15.86:0/3310338888 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f6a6c0a3e70 0x7f6a6c0a6330 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c0cdc40 tx=0x7f6a7c0c7470 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:56.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.292+0000 7f6a697fa640 1 -- 172.21.15.86:0/2058718806 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a78048070 con 0x564e665cf4d0 2024-04-21T08:39:56.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.292+0000 7f6a697fa640 1 -- 172.21.15.86:0/2058718806 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a78004b80 con 0x564e665cf4d0 2024-04-21T08:39:56.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.292+0000 7f6a8c405740 1 -- 172.21.15.86:0/2058718806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e665cf4d0 msgr2=0x564e665cf8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.292+0000 7f6a8c405740 1 --2- 172.21.15.86:0/2058718806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e665cf4d0 0x564e665cf8b0 secure :-1 s=READY pgs=723 cs=0 l=1 rev1=1 crypto rx=0x7f6a78005ec0 tx=0x7f6a78005e30 comp rx=0 tx=0).stop 2024-04-21T08:39:56.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.292+0000 7f6a8c405740 1 -- 172.21.15.86:0/2058718806 shutdown_connections 2024-04-21T08:39:56.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.292+0000 7f6a8c405740 1 --2- 172.21.15.86:0/2058718806 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e665d8a00 0x564e665dae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.292+0000 7f6a8c405740 1 --2- 172.21.15.86:0/2058718806 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564e665cfe80 0x564e665d8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.292+0000 7f6a8c405740 1 --2- 172.21.15.86:0/2058718806 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e665cf4d0 0x564e665cf8b0 unknown :-1 s=CLOSED pgs=723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.292+0000 7f6a8c405740 1 -- 172.21.15.86:0/2058718806 >> 172.21.15.86:0/2058718806 conn(0x564e665c9ba0 msgr2=0x564e665c9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:56.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.292+0000 7f6a8c405740 1 -- 172.21.15.86:0/2058718806 shutdown_connections 2024-04-21T08:39:56.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.292+0000 7f6a8c405740 1 -- 172.21.15.86:0/2058718806 wait complete. 2024-04-21T08:39:56.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.293+0000 7f6a8c405740 1 -- start start 2024-04-21T08:39:56.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.293+0000 7f6a8c405740 1 Processor -- start 2024-04-21T08:39:56.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.293+0000 7f6a8c405740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e665cf4d0 0x564e665d82d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.293+0000 7f6a8c405740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e665cfe80 0x564e665d5660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.293+0000 7f6a8c405740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564e665d8a00 0x564e665d5bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.293+0000 7f6a8c405740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564e665db910 con 0x564e665cf4d0 2024-04-21T08:39:56.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.293+0000 7f6a8c405740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564e664ed500 con 0x564e665d8a00 2024-04-21T08:39:56.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.293+0000 7f6a8c405740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564e665dba90 con 0x564e665cfe80 2024-04-21T08:39:56.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.293+0000 7f6a8c405740 10 client.0 _ll_register_callbacks cb 0x564e664bbb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:56.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.293+0000 7f6a827fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e665cf4d0 0x564e665d82d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.293+0000 7f6a827fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e665cf4d0 0x564e665d82d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45570/0 (socket says 172.21.15.86:45570) 2024-04-21T08:39:56.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.293+0000 7f6a827fc640 1 -- 172.21.15.86:0/2540413396 learned_addr learned my addr 172.21.15.86:0/2540413396 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:56.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.293+0000 7f6a827fc640 1 --2- 172.21.15.86:0/2540413396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564e665d8a00 0x564e665d5bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.293+0000 7f6a82ffd640 1 --2- 172.21.15.86:0/2540413396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e665cfe80 0x564e665d5660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.293+0000 7f6a827fc640 1 -- 172.21.15.86:0/2540413396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e665cfe80 msgr2=0x564e665d5660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.293+0000 7f6a827fc640 1 --2- 172.21.15.86:0/2540413396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e665cfe80 0x564e665d5660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.293+0000 7f6a827fc640 1 -- 172.21.15.86:0/2540413396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564e665d8a00 msgr2=0x564e665d5bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.293+0000 7f6a827fc640 1 --2- 172.21.15.86:0/2540413396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564e665d8a00 0x564e665d5bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.293+0000 7f6a827fc640 1 -- 172.21.15.86:0/2540413396 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e662463f0 con 0x564e665cf4d0 2024-04-21T08:39:56.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.293+0000 7f6a82ffd640 1 --2- 172.21.15.86:0/2540413396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e665cfe80 0x564e665d5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:56.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.293+0000 7f6a827fc640 1 --2- 172.21.15.86:0/2540413396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e665cf4d0 0x564e665d82d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f6a78036040 tx=0x7f6a78004760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec6a81b9d72f7268 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:56.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.293+0000 7f6a66ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:56.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.294+0000 7f6a66ff5640 1 -- 172.21.15.86:0/2540413396 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a78054070 con 0x564e665cf4d0 2024-04-21T08:39:56.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.294+0000 7f6a66ff5640 1 -- 172.21.15.86:0/2540413396 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f6a78003b00 con 0x564e665cf4d0 2024-04-21T08:39:56.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.294+0000 7f6a8c405740 1 -- 172.21.15.86:0/2540413396 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a78048070 con 0x564e665cf4d0 2024-04-21T08:39:56.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.294+0000 7f6a66ff5640 1 -- 172.21.15.86:0/2540413396 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6a78003d00 con 0x564e665cf4d0 2024-04-21T08:39:56.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.294+0000 7f6a8c405740 10 client.16332 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:56.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.294+0000 7f6a8c405740 1 -- 172.21.15.86:0/2540413396 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564e663107d0 con 0x564e665cf4d0 2024-04-21T08:39:56.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.294+0000 7f6a66ff5640 1 -- 172.21.15.86:0/2540413396 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a78048070 con 0x564e665cf4d0 2024-04-21T08:39:56.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.294+0000 7f6a66ff5640 1 -- 172.21.15.86:0/2540413396 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6a78034440 con 0x564e665cf4d0 2024-04-21T08:39:56.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.294+0000 7f6a8c405740 10 client.16332 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:56.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.294+0000 7f6a8c405740 10 client.16332 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:39:56.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.294+0000 7f6a8c405740 20 client.16332 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:56.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.294+0000 7f6a8c405740 1 --2- 172.21.15.86:0/2540413396 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x564e665e8a00 0x564e665eadf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.294+0000 7f6a8c405740 4 client.16332 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:56.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.294+0000 7f6a8c405740 1 -- 172.21.15.86:0/2540413396 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564e664d3930 con 0x564e665e8a00 2024-04-21T08:39:56.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.295+0000 7f6a82ffd640 1 --2- 172.21.15.86:0/2540413396 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x564e665e8a00 0x564e665eadf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.295+0000 7f6a66ff5640 10 client.16332 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:39:56.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.295+0000 7f6a82ffd640 1 --2- 172.21.15.86:0/2540413396 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x564e665e8a00 0x564e665eadf0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=61554852c6ac556 server_cookie=2ed9ffd718efac81 in_seq=0 out_seq=0 2024-04-21T08:39:56.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.296+0000 7f6a66ff5640 1 -- 172.21.15.86:0/2540413396 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564e664d3930 con 0x564e665e8a00 2024-04-21T08:39:56.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.296+0000 7f6a66ff5640 10 client.16332 handle_command_reply: tid=0 2024-04-21T08:39:56.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.296+0000 7f6a8c405740 1 client.16332 shutdown 2024-04-21T08:39:56.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.296+0000 7f6a8c405740 1 -- 172.21.15.86:0/2540413396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e665cf4d0 msgr2=0x564e665d82d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.296+0000 7f6a8c405740 1 --2- 172.21.15.86:0/2540413396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e665cf4d0 0x564e665d82d0 secure :-1 s=READY pgs=724 cs=0 l=1 rev1=1 crypto rx=0x7f6a78036040 tx=0x7f6a78004760 comp rx=0 tx=0).stop 2024-04-21T08:39:56.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.296+0000 7f6a8c405740 1 -- 172.21.15.86:0/2540413396 shutdown_connections 2024-04-21T08:39:56.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.296+0000 7f6a8c405740 1 --2- 172.21.15.86:0/2540413396 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x564e665e8a00 0x564e665eadf0 crc :-1 s=READY pgs=37 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.296+0000 7f6a8c405740 1 --2- 172.21.15.86:0/2540413396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564e665d8a00 0x564e665d5bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.296+0000 7f6a66ff5640 0 client.16332 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:39:56.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.296+0000 7f6a8c405740 1 --2- 172.21.15.86:0/2540413396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e665cfe80 0x564e665d5660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.296+0000 7f6a8c405740 1 --2- 172.21.15.86:0/2540413396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e665cf4d0 0x564e665d82d0 unknown :-1 s=CLOSED pgs=724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.296+0000 7f6a8c405740 1 -- 172.21.15.86:0/2540413396 >> 172.21.15.86:0/2540413396 conn(0x564e665c9ba0 msgr2=0x564e665cb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:56.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.296+0000 7f6a8c405740 1 -- 172.21.15.86:0/2540413396 shutdown_connections 2024-04-21T08:39:56.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.296+0000 7f6a8c405740 1 -- 172.21.15.86:0/2540413396 wait complete. 2024-04-21T08:39:56.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.296+0000 7f6a8c405740 20 client.16332 trim_cache size 0 max 16384 2024-04-21T08:39:56.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.309+0000 7f6a8c405740 1 Processor -- start 2024-04-21T08:39:56.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.309+0000 7f6a8c405740 1 -- start start 2024-04-21T08:39:56.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.310+0000 7f6a8c405740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e665cf4d0 0x564e665e47d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.310+0000 7f6a8c405740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564e665cfe80 0x564e665c8a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.310+0000 7f6a8c405740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e665d8a00 0x564e665d7d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.310+0000 7f6a8c405740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6a841b1450 con 0x564e665d8a00 2024-04-21T08:39:56.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.310+0000 7f6a8c405740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6a841b2f90 con 0x564e665cfe80 2024-04-21T08:39:56.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.310+0000 7f6a8c405740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6a841b1b20 con 0x564e665cf4d0 2024-04-21T08:39:56.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.310+0000 7f6a827fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e665d8a00 0x564e665d7d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.310+0000 7f6a827fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e665d8a00 0x564e665d7d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45582/0 (socket says 172.21.15.86:45582) 2024-04-21T08:39:56.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.310+0000 7f6a827fc640 1 -- 172.21.15.86:0/2734788964 learned_addr learned my addr 172.21.15.86:0/2734788964 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:56.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.310+0000 7f6a827fc640 1 --2- 172.21.15.86:0/2734788964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e665cf4d0 0x564e665e47d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.310+0000 7f6a82ffd640 1 --2- 172.21.15.86:0/2734788964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564e665cfe80 0x564e665c8a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.310+0000 7f6a827fc640 1 -- 172.21.15.86:0/2734788964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e665cf4d0 msgr2=0x564e665e47d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.310+0000 7f6a827fc640 1 --2- 172.21.15.86:0/2734788964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e665cf4d0 0x564e665e47d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.310+0000 7f6a827fc640 1 -- 172.21.15.86:0/2734788964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564e665cfe80 msgr2=0x564e665c8a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.310+0000 7f6a827fc640 1 --2- 172.21.15.86:0/2734788964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564e665cfe80 0x564e665c8a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.310+0000 7f6a827fc640 1 -- 172.21.15.86:0/2734788964 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564e665c93d0 con 0x564e665d8a00 2024-04-21T08:39:56.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.310+0000 7f6a82ffd640 1 --2- 172.21.15.86:0/2734788964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564e665cfe80 0x564e665c8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:56.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.310+0000 7f6a827fc640 1 --2- 172.21.15.86:0/2734788964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e665d8a00 0x564e665d7d90 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6a780047f0 tx=0x7f6a78005e30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96088faac0006ddf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:56.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.310+0000 7f6a68ff9640 1 -- 172.21.15.86:0/2734788964 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a78054040 con 0x564e665d8a00 2024-04-21T08:39:56.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.310+0000 7f6a68ff9640 1 -- 172.21.15.86:0/2734788964 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a78048040 con 0x564e665d8a00 2024-04-21T08:39:56.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.310+0000 7f6a68ff9640 1 -- 172.21.15.86:0/2734788964 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a78003b60 con 0x564e665d8a00 2024-04-21T08:39:56.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.311+0000 7f6a8c405740 1 -- 172.21.15.86:0/2734788964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e665d8a00 msgr2=0x564e665d7d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.311+0000 7f6a8c405740 1 --2- 172.21.15.86:0/2734788964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e665d8a00 0x564e665d7d90 secure :-1 s=READY pgs=725 cs=0 l=1 rev1=1 crypto rx=0x7f6a780047f0 tx=0x7f6a78005e30 comp rx=0 tx=0).stop 2024-04-21T08:39:56.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.311+0000 7f6a8c405740 1 -- 172.21.15.86:0/2734788964 shutdown_connections 2024-04-21T08:39:56.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.311+0000 7f6a8c405740 1 --2- 172.21.15.86:0/2734788964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e665d8a00 0x564e665d7d90 unknown :-1 s=CLOSED pgs=725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.311+0000 7f6a8c405740 1 --2- 172.21.15.86:0/2734788964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564e665cfe80 0x564e665c8a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.311+0000 7f6a8c405740 1 --2- 172.21.15.86:0/2734788964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e665cf4d0 0x564e665e47d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.311+0000 7f6a8c405740 1 -- 172.21.15.86:0/2734788964 >> 172.21.15.86:0/2734788964 conn(0x564e665c9ba0 msgr2=0x564e665c8370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:56.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.311+0000 7f6a8c405740 1 -- 172.21.15.86:0/2734788964 shutdown_connections 2024-04-21T08:39:56.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.311+0000 7f6a8c405740 1 -- 172.21.15.86:0/2734788964 wait complete. 2024-04-21T08:39:56.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.311+0000 7f6a8c405740 1 -- start start 2024-04-21T08:39:56.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.311+0000 7f6a8c405740 1 Processor -- start 2024-04-21T08:39:56.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.312+0000 7f6a8c405740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e665cf4d0 0x564e665dbe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.312+0000 7f6a8c405740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564e665cfe80 0x564e665dc3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.312+0000 7f6a8c405740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e665d8a00 0x564e6664cbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.312+0000 7f6a8c405740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6a841b28c0 con 0x564e665cf4d0 2024-04-21T08:39:56.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.312+0000 7f6a8c405740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6a841b21f0 con 0x564e665cfe80 2024-04-21T08:39:56.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.312+0000 7f6a8c405740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6a841af410 con 0x564e665d8a00 2024-04-21T08:39:56.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.312+0000 7f6a8c405740 10 client.0 _ll_register_callbacks cb 0x7f6a7802ca70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:56.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.312+0000 7f6a827fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e665cf4d0 0x564e665dbe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.312+0000 7f6a827fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e665cf4d0 0x564e665dbe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45594/0 (socket says 172.21.15.86:45594) 2024-04-21T08:39:56.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.312+0000 7f6a827fc640 1 -- 172.21.15.86:0/3795528058 learned_addr learned my addr 172.21.15.86:0/3795528058 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:56.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.312+0000 7f6a827fc640 1 --2- 172.21.15.86:0/3795528058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e665d8a00 0x564e6664cbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.312+0000 7f6a82ffd640 1 --2- 172.21.15.86:0/3795528058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564e665cfe80 0x564e665dc3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.312+0000 7f6a827fc640 1 -- 172.21.15.86:0/3795528058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e665d8a00 msgr2=0x564e6664cbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.312+0000 7f6a827fc640 1 --2- 172.21.15.86:0/3795528058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e665d8a00 0x564e6664cbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.312+0000 7f6a827fc640 1 -- 172.21.15.86:0/3795528058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564e665cfe80 msgr2=0x564e665dc3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.312+0000 7f6a827fc640 1 --2- 172.21.15.86:0/3795528058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564e665cfe80 0x564e665dc3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.312+0000 7f6a827fc640 1 -- 172.21.15.86:0/3795528058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564e6664d180 con 0x564e665cf4d0 2024-04-21T08:39:56.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.312+0000 7f6a82ffd640 1 --2- 172.21.15.86:0/3795528058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564e665cfe80 0x564e665dc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:56.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.312+0000 7f6a827fc640 1 --2- 172.21.15.86:0/3795528058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e665cf4d0 0x564e665dbe90 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f6a780345b0 tx=0x7f6a78005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=357e693326603be9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:56.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.312+0000 7f6a66ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:56.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.312+0000 7f6a66ff5640 1 -- 172.21.15.86:0/3795528058 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a78054040 con 0x564e665cf4d0 2024-04-21T08:39:56.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.312+0000 7f6a66ff5640 1 -- 172.21.15.86:0/3795528058 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f6a78004450 con 0x564e665cf4d0 2024-04-21T08:39:56.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.312+0000 7f6a8c405740 1 -- 172.21.15.86:0/3795528058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6a78048040 con 0x564e665cf4d0 2024-04-21T08:39:56.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.313+0000 7f6a66ff5640 1 -- 172.21.15.86:0/3795528058 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6a78003b10 con 0x564e665cf4d0 2024-04-21T08:39:56.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.313+0000 7f6a8c405740 10 client.16338 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:56.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.313+0000 7f6a8c405740 1 -- 172.21.15.86:0/3795528058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564e6664e480 con 0x564e665cf4d0 2024-04-21T08:39:56.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.313+0000 7f6a66ff5640 1 -- 172.21.15.86:0/3795528058 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a78048040 con 0x564e665cf4d0 2024-04-21T08:39:56.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.313+0000 7f6a66ff5640 1 -- 172.21.15.86:0/3795528058 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6a780594d0 con 0x564e665cf4d0 2024-04-21T08:39:56.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.313+0000 7f6a8c405740 10 client.16338 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:56.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.313+0000 7f6a8c405740 10 client.16338 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:39:56.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.313+0000 7f6a8c405740 20 client.16338 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:56.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.313+0000 7f6a8c405740 1 --2- 172.21.15.86:0/3795528058 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x564e6664ea30 0x564e66650e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.313+0000 7f6a8c405740 4 client.16338 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:39:56.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.313+0000 7f6a8c405740 1 -- 172.21.15.86:0/3795528058 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564e66541c10 con 0x564e6664ea30 2024-04-21T08:39:56.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.313+0000 7f6a82ffd640 1 --2- 172.21.15.86:0/3795528058 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x564e6664ea30 0x564e66650e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.314+0000 7f6a66ff5640 10 client.16338 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:39:56.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.314+0000 7f6a82ffd640 1 --2- 172.21.15.86:0/3795528058 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x564e6664ea30 0x564e66650e20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7ce5913f93b184dc server_cookie=b57db8e4cbf7688d in_seq=0 out_seq=0 2024-04-21T08:39:56.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.314+0000 7f6a66ff5640 1 -- 172.21.15.86:0/3795528058 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564e66541c10 con 0x564e6664ea30 2024-04-21T08:39:56.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.314+0000 7f6a66ff5640 10 client.16338 handle_command_reply: tid=0 2024-04-21T08:39:56.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.314+0000 7f6a8c405740 1 client.16338 shutdown 2024-04-21T08:39:56.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.314+0000 7f6a8c405740 1 -- 172.21.15.86:0/3795528058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e665cf4d0 msgr2=0x564e665dbe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.314+0000 7f6a8c405740 1 --2- 172.21.15.86:0/3795528058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e665cf4d0 0x564e665dbe90 secure :-1 s=READY pgs=726 cs=0 l=1 rev1=1 crypto rx=0x7f6a780345b0 tx=0x7f6a78005c90 comp rx=0 tx=0).stop 2024-04-21T08:39:56.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.314+0000 7f6a8c405740 1 -- 172.21.15.86:0/3795528058 shutdown_connections 2024-04-21T08:39:56.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.314+0000 7f6a8c405740 1 --2- 172.21.15.86:0/3795528058 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x564e6664ea30 0x564e66650e20 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.314+0000 7f6a8c405740 1 --2- 172.21.15.86:0/3795528058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564e665d8a00 0x564e6664cbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.314+0000 7f6a66ff5640 0 client.16338 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:39:56.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.314+0000 7f6a8c405740 1 --2- 172.21.15.86:0/3795528058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564e665cfe80 0x564e665dc3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.314+0000 7f6a8c405740 1 --2- 172.21.15.86:0/3795528058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564e665cf4d0 0x564e665dbe90 unknown :-1 s=CLOSED pgs=726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.314+0000 7f6a8c405740 1 -- 172.21.15.86:0/3795528058 >> 172.21.15.86:0/3795528058 conn(0x564e665c9ba0 msgr2=0x564e665c8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:56.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.314+0000 7f6a8c405740 1 -- 172.21.15.86:0/3795528058 shutdown_connections 2024-04-21T08:39:56.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.314+0000 7f6a8c405740 1 -- 172.21.15.86:0/3795528058 wait complete. 2024-04-21T08:39:56.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.314+0000 7f6a8c405740 20 client.16338 trim_cache size 0 max 16384 2024-04-21T08:39:56.320 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:39:56.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.314+0000 7f6a8973b640 1 -- 172.21.15.86:0/3310338888 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f6a6c0a3e70 msgr2=0x7f6a6c0a6330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.314+0000 7f6a8973b640 1 --2- 172.21.15.86:0/3310338888 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f6a6c0a3e70 0x7f6a6c0a6330 secure :-1 s=READY pgs=100 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c0cdc40 tx=0x7f6a7c0c7470 comp rx=0 tx=0).stop 2024-04-21T08:39:56.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.314+0000 7f6a8973b640 1 -- 172.21.15.86:0/3310338888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a841035a0 msgr2=0x7f6a841a6940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.314+0000 7f6a8973b640 1 --2- 172.21.15.86:0/3310338888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a841035a0 0x7f6a841a6940 secure :-1 s=READY pgs=722 cs=0 l=1 rev1=1 crypto rx=0x7f6a7c00c710 tx=0x7f6a7c00cbd0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.315+0000 7f6a8973b640 1 -- 172.21.15.86:0/3310338888 shutdown_connections 2024-04-21T08:39:56.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.315+0000 7f6a8973b640 1 --2- 172.21.15.86:0/3310338888 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f6a6c0a3e70 0x7f6a6c0a6330 unknown :-1 s=CLOSED pgs=100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.315+0000 7f6a8973b640 1 --2- 172.21.15.86:0/3310338888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a841035a0 0x7f6a841a6940 unknown :-1 s=CLOSED pgs=722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.315+0000 7f6a8973b640 1 --2- 172.21.15.86:0/3310338888 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a84106160 0x7f6a84103060 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.315+0000 7f6a8973b640 1 --2- 172.21.15.86:0/3310338888 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a84101090 0x7f6a84102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.315+0000 7f6a8973b640 1 -- 172.21.15.86:0/3310338888 >> 172.21.15.86:0/3310338888 conn(0x7f6a84077f50 msgr2=0x7f6a84069500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:56.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.315+0000 7f6a8973b640 1 -- 172.21.15.86:0/3310338888 shutdown_connections 2024-04-21T08:39:56.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.315+0000 7f6a8973b640 1 -- 172.21.15.86:0/3310338888 wait complete. 2024-04-21T08:39:56.324 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:39:56.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.425+0000 7efe4fda4640 1 -- 172.21.15.86:0/1445189021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe481014a0 msgr2=0x7efe48106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.425+0000 7efe4fda4640 1 --2- 172.21.15.86:0/1445189021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe481014a0 0x7efe48106bd0 secure :-1 s=READY pgs=727 cs=0 l=1 rev1=1 crypto rx=0x7efe38009900 tx=0x7efe3802bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:56.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.425+0000 7efe4fda4640 1 -- 172.21.15.86:0/1445189021 shutdown_connections 2024-04-21T08:39:56.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.425+0000 7efe4fda4640 1 --2- 172.21.15.86:0/1445189021 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe481072b0 0x7efe4810eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.425+0000 7efe4fda4640 1 --2- 172.21.15.86:0/1445189021 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe481014a0 0x7efe48106bd0 unknown :-1 s=CLOSED pgs=727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.425+0000 7efe4fda4640 1 --2- 172.21.15.86:0/1445189021 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe48100af0 0x7efe48100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.425+0000 7efe4fda4640 1 -- 172.21.15.86:0/1445189021 >> 172.21.15.86:0/1445189021 conn(0x7efe480760e0 msgr2=0x7efe480764f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:56.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.425+0000 7efe4fda4640 1 -- 172.21.15.86:0/1445189021 shutdown_connections 2024-04-21T08:39:56.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.425+0000 7efe4fda4640 1 -- 172.21.15.86:0/1445189021 wait complete. 2024-04-21T08:39:56.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.426+0000 7efe4fda4640 1 Processor -- start 2024-04-21T08:39:56.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.426+0000 7efe4fda4640 1 -- start start 2024-04-21T08:39:56.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.426+0000 7efe4fda4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe48100af0 0x7efe4819e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.426+0000 7efe4fda4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe481072b0 0x7efe4819e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.426+0000 7efe4fda4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe48198610 0x7efe48198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.426+0000 7efe4fda4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe48112230 con 0x7efe48100af0 2024-04-21T08:39:56.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.426+0000 7efe4fda4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efe481120b0 con 0x7efe481072b0 2024-04-21T08:39:56.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.426+0000 7efe4fda4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7efe481123b0 con 0x7efe48198610 2024-04-21T08:39:56.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.426+0000 7efe4db40640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe48100af0 0x7efe4819e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.426+0000 7efe4db40640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe48100af0 0x7efe4819e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45616/0 (socket says 172.21.15.86:45616) 2024-04-21T08:39:56.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.426+0000 7efe4db40640 1 -- 172.21.15.86:0/1098115357 learned_addr learned my addr 172.21.15.86:0/1098115357 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:56.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.426+0000 7efe4d33f640 1 --2- 172.21.15.86:0/1098115357 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe481072b0 0x7efe4819e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.426+0000 7efe4e341640 1 --2- 172.21.15.86:0/1098115357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe48198610 0x7efe48198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.426+0000 7efe4db40640 1 -- 172.21.15.86:0/1098115357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe48198610 msgr2=0x7efe48198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.426+0000 7efe4db40640 1 --2- 172.21.15.86:0/1098115357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe48198610 0x7efe48198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.426+0000 7efe4db40640 1 -- 172.21.15.86:0/1098115357 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe481072b0 msgr2=0x7efe4819e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.426+0000 7efe4db40640 1 --2- 172.21.15.86:0/1098115357 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe481072b0 0x7efe4819e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.426+0000 7efe4db40640 1 -- 172.21.15.86:0/1098115357 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efe48199130 con 0x7efe48100af0 2024-04-21T08:39:56.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.426+0000 7efe4d33f640 1 --2- 172.21.15.86:0/1098115357 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe481072b0 0x7efe4819e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:56.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.426+0000 7efe4e341640 1 --2- 172.21.15.86:0/1098115357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe48198610 0x7efe48198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:56.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.426+0000 7efe4db40640 1 --2- 172.21.15.86:0/1098115357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe48100af0 0x7efe4819e460 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7efe4400c970 tx=0x7efe4400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:56.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.427+0000 7efe3effd640 1 -- 172.21.15.86:0/1098115357 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44007bb0 con 0x7efe48100af0 2024-04-21T08:39:56.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.427+0000 7efe3effd640 1 -- 172.21.15.86:0/1098115357 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe44007d50 con 0x7efe48100af0 2024-04-21T08:39:56.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.427+0000 7efe4fda4640 1 -- 172.21.15.86:0/1098115357 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efe48199420 con 0x7efe48100af0 2024-04-21T08:39:56.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.427+0000 7efe3effd640 1 -- 172.21.15.86:0/1098115357 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4400f450 con 0x7efe48100af0 2024-04-21T08:39:56.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.427+0000 7efe4fda4640 1 -- 172.21.15.86:0/1098115357 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efe481a5620 con 0x7efe48100af0 2024-04-21T08:39:56.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.429+0000 7efe3effd640 1 -- 172.21.15.86:0/1098115357 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7efe44004f20 con 0x7efe48100af0 2024-04-21T08:39:56.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.429+0000 7efe52999740 1 Processor -- start 2024-04-21T08:39:56.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.429+0000 7efe52999740 1 -- start start 2024-04-21T08:39:56.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.430+0000 7efe52999740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b0c7bb4d0 0x558b0c7bb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.430+0000 7efe52999740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b0c7bbe80 0x558b0c7c4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.430+0000 7efe52999740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b0c7c4a00 0x558b0c7c6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.430+0000 7efe52999740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558b0c7201f0 con 0x558b0c7bb4d0 2024-04-21T08:39:56.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.430+0000 7efe52999740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x558b0c6bd4b0 con 0x558b0c7c4a00 2024-04-21T08:39:56.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.430+0000 7efe52999740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558b0c71c1b0 con 0x558b0c7bbe80 2024-04-21T08:39:56.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.430+0000 7efe3effd640 1 --2- 172.21.15.86:0/1098115357 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efe340a3e40 0x7efe340a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.430+0000 7efe4db40640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b0c7bbe80 0x558b0c7c4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.430+0000 7efe4e341640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b0c7bb4d0 0x558b0c7bb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.430+0000 7efe4e341640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b0c7bb4d0 0x558b0c7bb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45628/0 (socket says 172.21.15.86:45628) 2024-04-21T08:39:56.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.430+0000 7efe4d33f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b0c7c4a00 0x558b0c7c6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.430+0000 7efe4e341640 1 -- 172.21.15.86:0/2422171952 learned_addr learned my addr 172.21.15.86:0/2422171952 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:56.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.430+0000 7efe4e341640 1 --2- 172.21.15.86:0/1098115357 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efe340a3e40 0x7efe340a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.430+0000 7efe3effd640 1 -- 172.21.15.86:0/1098115357 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7efe440d0c50 con 0x7efe48100af0 2024-04-21T08:39:56.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.430+0000 7efe4e341640 1 -- 172.21.15.86:0/2422171952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b0c7bbe80 msgr2=0x558b0c7c4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.430+0000 7efe4e341640 1 --2- 172.21.15.86:0/2422171952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b0c7bbe80 0x558b0c7c4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.430+0000 7efe4e341640 1 -- 172.21.15.86:0/2422171952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b0c7c4a00 msgr2=0x558b0c7c6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.430+0000 7efe4e341640 1 --2- 172.21.15.86:0/2422171952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b0c7c4a00 0x558b0c7c6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.430+0000 7efe4e341640 1 -- 172.21.15.86:0/2422171952 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b0c6af1b0 con 0x558b0c7bb4d0 2024-04-21T08:39:56.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.431+0000 7efe4e341640 1 --2- 172.21.15.86:0/2422171952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b0c7bb4d0 0x558b0c7bb8b0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7efe4000aee0 tx=0x7efe4000d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=516b228fdba907bd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:56.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.431+0000 7efe4e341640 1 --2- 172.21.15.86:0/1098115357 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efe340a3e40 0x7efe340a6300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efe4000afd0 tx=0x7efe4001e000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:56.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.431+0000 7efe33fff640 1 -- 172.21.15.86:0/2422171952 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4002c070 con 0x558b0c7bb4d0 2024-04-21T08:39:56.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.431+0000 7efe33fff640 1 -- 172.21.15.86:0/2422171952 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe40013bf0 con 0x558b0c7bb4d0 2024-04-21T08:39:56.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.431+0000 7efe33fff640 1 -- 172.21.15.86:0/2422171952 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe40027e00 con 0x558b0c7bb4d0 2024-04-21T08:39:56.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.431+0000 7efe52999740 1 -- 172.21.15.86:0/2422171952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b0c7bb4d0 msgr2=0x558b0c7bb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.431+0000 7efe52999740 1 --2- 172.21.15.86:0/2422171952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b0c7bb4d0 0x558b0c7bb8b0 secure :-1 s=READY pgs=729 cs=0 l=1 rev1=1 crypto rx=0x7efe4000aee0 tx=0x7efe4000d6b0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.431+0000 7efe52999740 1 -- 172.21.15.86:0/2422171952 shutdown_connections 2024-04-21T08:39:56.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.431+0000 7efe52999740 1 --2- 172.21.15.86:0/2422171952 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b0c7c4a00 0x558b0c7c6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.431+0000 7efe52999740 1 --2- 172.21.15.86:0/2422171952 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b0c7bbe80 0x558b0c7c4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.431+0000 7efe52999740 1 --2- 172.21.15.86:0/2422171952 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b0c7bb4d0 0x558b0c7bb8b0 unknown :-1 s=CLOSED pgs=729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.432+0000 7efe52999740 1 -- 172.21.15.86:0/2422171952 >> 172.21.15.86:0/2422171952 conn(0x558b0c7b5ba0 msgr2=0x558b0c7b5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:56.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.432+0000 7efe52999740 1 -- 172.21.15.86:0/2422171952 shutdown_connections 2024-04-21T08:39:56.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.432+0000 7efe52999740 1 -- 172.21.15.86:0/2422171952 wait complete. 2024-04-21T08:39:56.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.432+0000 7efe52999740 1 -- start start 2024-04-21T08:39:56.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.432+0000 7efe52999740 1 Processor -- start 2024-04-21T08:39:56.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.432+0000 7efe52999740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b0c7bb4d0 0x558b0c7c42e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.432+0000 7efe52999740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b0c7bbe80 0x558b0c7c1670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.432+0000 7efe52999740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b0c7c4a00 0x558b0c7c1bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.432+0000 7efe52999740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558b0c7c7910 con 0x558b0c7c4a00 2024-04-21T08:39:56.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.432+0000 7efe52999740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x558b0c6d9500 con 0x558b0c7bb4d0 2024-04-21T08:39:56.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.432+0000 7efe52999740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558b0c7c7a90 con 0x558b0c7bbe80 2024-04-21T08:39:56.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.432+0000 7efe52999740 10 client.0 _ll_register_callbacks cb 0x558b0c6a7b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.432+0000 7efe4e341640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b0c7c4a00 0x558b0c7c1bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.432+0000 7efe4e341640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b0c7c4a00 0x558b0c7c1bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45642/0 (socket says 172.21.15.86:45642) 2024-04-21T08:39:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.432+0000 7efe4e341640 1 -- 172.21.15.86:0/1388733309 learned_addr learned my addr 172.21.15.86:0/1388733309 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.432+0000 7efe4db40640 1 --2- 172.21.15.86:0/1388733309 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b0c7bb4d0 0x558b0c7c42e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.433+0000 7efe4d33f640 1 --2- 172.21.15.86:0/1388733309 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b0c7bbe80 0x558b0c7c1670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.433+0000 7efe4e341640 1 -- 172.21.15.86:0/1388733309 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b0c7bbe80 msgr2=0x558b0c7c1670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.433+0000 7efe4e341640 1 --2- 172.21.15.86:0/1388733309 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b0c7bbe80 0x558b0c7c1670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.433+0000 7efe4e341640 1 -- 172.21.15.86:0/1388733309 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b0c7bb4d0 msgr2=0x558b0c7c42e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.433+0000 7efe4e341640 1 --2- 172.21.15.86:0/1388733309 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b0c7bb4d0 0x558b0c7c42e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.433+0000 7efe4e341640 1 -- 172.21.15.86:0/1388733309 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b0c4323f0 con 0x558b0c7c4a00 2024-04-21T08:39:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.433+0000 7efe4d33f640 1 --2- 172.21.15.86:0/1388733309 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b0c7bbe80 0x558b0c7c1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.433+0000 7efe4e341640 1 --2- 172.21.15.86:0/1388733309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b0c7c4a00 0x558b0c7c1bd0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7efe400279e0 tx=0x7efe4000a820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79659c0b963ac297 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.433+0000 7efe317fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.433+0000 7efe317fa640 1 -- 172.21.15.86:0/1388733309 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4002c040 con 0x558b0c7c4a00 2024-04-21T08:39:56.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.433+0000 7efe317fa640 1 -- 172.21.15.86:0/1388733309 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7efe40027b40 con 0x558b0c7c4a00 2024-04-21T08:39:56.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.433+0000 7efe52999740 1 -- 172.21.15.86:0/1388733309 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe40013bf0 con 0x558b0c7c4a00 2024-04-21T08:39:56.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.433+0000 7efe317fa640 1 -- 172.21.15.86:0/1388733309 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efe40027d40 con 0x558b0c7c4a00 2024-04-21T08:39:56.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.433+0000 7efe52999740 10 client.16350 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:56.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.433+0000 7efe52999740 1 -- 172.21.15.86:0/1388733309 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558b0c4fc7d0 con 0x558b0c7c4a00 2024-04-21T08:39:56.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.433+0000 7efe317fa640 1 -- 172.21.15.86:0/1388733309 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe40013bf0 con 0x558b0c7c4a00 2024-04-21T08:39:56.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.433+0000 7efe317fa640 1 -- 172.21.15.86:0/1388733309 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7efe40038030 con 0x558b0c7c4a00 2024-04-21T08:39:56.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.434+0000 7efe52999740 10 client.16350 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:56.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.434+0000 7efe52999740 10 client.16350 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:39:56.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.434+0000 7efe52999740 20 client.16350 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:56.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.434+0000 7efe52999740 1 --2- 172.21.15.86:0/1388733309 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x558b0c7d4b70 0x558b0c7d6f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.434+0000 7efe52999740 4 client.16350 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:56.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.434+0000 7efe52999740 1 -- 172.21.15.86:0/1388733309 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b0c6bf930 con 0x558b0c7d4b70 2024-04-21T08:39:56.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.434+0000 7efe4db40640 1 --2- 172.21.15.86:0/1388733309 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x558b0c7d4b70 0x558b0c7d6f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.434+0000 7efe4db40640 1 --2- 172.21.15.86:0/1388733309 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x558b0c7d4b70 0x558b0c7d6f60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4424089222d089f server_cookie=c3eec0154f519e89 in_seq=0 out_seq=0 2024-04-21T08:39:56.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.434+0000 7efe317fa640 10 client.16350 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:39:56.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.435+0000 7efe317fa640 1 -- 172.21.15.86:0/1388733309 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558b0c6bf930 con 0x558b0c7d4b70 2024-04-21T08:39:56.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.435+0000 7efe317fa640 10 client.16350 handle_command_reply: tid=0 2024-04-21T08:39:56.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.435+0000 7efe52999740 1 client.16350 shutdown 2024-04-21T08:39:56.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.435+0000 7efe52999740 1 -- 172.21.15.86:0/1388733309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b0c7c4a00 msgr2=0x558b0c7c1bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.435+0000 7efe52999740 1 --2- 172.21.15.86:0/1388733309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b0c7c4a00 0x558b0c7c1bd0 secure :-1 s=READY pgs=730 cs=0 l=1 rev1=1 crypto rx=0x7efe400279e0 tx=0x7efe4000a820 comp rx=0 tx=0).stop 2024-04-21T08:39:56.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.435+0000 7efe52999740 1 -- 172.21.15.86:0/1388733309 shutdown_connections 2024-04-21T08:39:56.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.435+0000 7efe52999740 1 --2- 172.21.15.86:0/1388733309 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x558b0c7d4b70 0x558b0c7d6f60 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.435+0000 7efe52999740 1 --2- 172.21.15.86:0/1388733309 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b0c7c4a00 0x558b0c7c1bd0 unknown :-1 s=CLOSED pgs=730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.435+0000 7efe52999740 1 --2- 172.21.15.86:0/1388733309 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b0c7bbe80 0x558b0c7c1670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.435+0000 7efe52999740 1 --2- 172.21.15.86:0/1388733309 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b0c7bb4d0 0x558b0c7c42e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.435+0000 7efe317fa640 0 client.16350 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:39:56.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.435+0000 7efe52999740 1 -- 172.21.15.86:0/1388733309 >> 172.21.15.86:0/1388733309 conn(0x558b0c7b5ba0 msgr2=0x558b0c7b78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:56.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.435+0000 7efe52999740 1 -- 172.21.15.86:0/1388733309 shutdown_connections 2024-04-21T08:39:56.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.435+0000 7efe52999740 1 -- 172.21.15.86:0/1388733309 wait complete. 2024-04-21T08:39:56.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.435+0000 7efe52999740 20 client.16350 trim_cache size 0 max 16384 2024-04-21T08:39:56.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.448+0000 7efe52999740 1 Processor -- start 2024-04-21T08:39:56.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.448+0000 7efe52999740 1 -- start start 2024-04-21T08:39:56.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.449+0000 7efe52999740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b0c7bb4d0 0x558b0c7d07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.449+0000 7efe52999740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b0c7bbe80 0x558b0c7bcea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.449+0000 7efe52999740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b0c7c4a00 0x558b0c7c3da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.449+0000 7efe52999740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe481ad120 con 0x558b0c7bb4d0 2024-04-21T08:39:56.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.449+0000 7efe52999740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efe481aec60 con 0x558b0c7bbe80 2024-04-21T08:39:56.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.449+0000 7efe52999740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7efe481ad7f0 con 0x558b0c7c4a00 2024-04-21T08:39:56.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.449+0000 7efe4db40640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b0c7bb4d0 0x558b0c7d07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.449+0000 7efe4db40640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b0c7bb4d0 0x558b0c7d07d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45656/0 (socket says 172.21.15.86:45656) 2024-04-21T08:39:56.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.449+0000 7efe4db40640 1 -- 172.21.15.86:0/3365215214 learned_addr learned my addr 172.21.15.86:0/3365215214 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:56.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.449+0000 7efe4db40640 1 -- 172.21.15.86:0/3365215214 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b0c7c4a00 msgr2=0x558b0c7c3da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.449+0000 7efe4db40640 1 --2- 172.21.15.86:0/3365215214 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b0c7c4a00 0x558b0c7c3da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.449+0000 7efe4db40640 1 -- 172.21.15.86:0/3365215214 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b0c7bbe80 msgr2=0x558b0c7bcea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.449+0000 7efe4db40640 1 --2- 172.21.15.86:0/3365215214 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b0c7bbe80 0x558b0c7bcea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.449+0000 7efe4db40640 1 -- 172.21.15.86:0/3365215214 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b0c7b0f70 con 0x558b0c7bb4d0 2024-04-21T08:39:56.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.449+0000 7efe4db40640 1 --2- 172.21.15.86:0/3365215214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b0c7bb4d0 0x558b0c7d07d0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7efe4409c330 tx=0x7efe4406f120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccdf3ae5f3ae1b1f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:56.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.449+0000 7efe337fe640 1 -- 172.21.15.86:0/3365215214 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4401d040 con 0x558b0c7bb4d0 2024-04-21T08:39:56.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.449+0000 7efe337fe640 1 -- 172.21.15.86:0/3365215214 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe4409c630 con 0x558b0c7bb4d0 2024-04-21T08:39:56.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.449+0000 7efe337fe640 1 -- 172.21.15.86:0/3365215214 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe44098ca0 con 0x558b0c7bb4d0 2024-04-21T08:39:56.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.450+0000 7efe52999740 1 -- 172.21.15.86:0/3365215214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b0c7bb4d0 msgr2=0x558b0c7d07d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.450+0000 7efe52999740 1 --2- 172.21.15.86:0/3365215214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b0c7bb4d0 0x558b0c7d07d0 secure :-1 s=READY pgs=731 cs=0 l=1 rev1=1 crypto rx=0x7efe4409c330 tx=0x7efe4406f120 comp rx=0 tx=0).stop 2024-04-21T08:39:56.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.450+0000 7efe52999740 1 -- 172.21.15.86:0/3365215214 shutdown_connections 2024-04-21T08:39:56.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.450+0000 7efe52999740 1 --2- 172.21.15.86:0/3365215214 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b0c7c4a00 0x558b0c7c3da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.450+0000 7efe52999740 1 --2- 172.21.15.86:0/3365215214 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b0c7bbe80 0x558b0c7bcea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.450+0000 7efe52999740 1 --2- 172.21.15.86:0/3365215214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b0c7bb4d0 0x558b0c7d07d0 unknown :-1 s=CLOSED pgs=731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.450+0000 7efe52999740 1 -- 172.21.15.86:0/3365215214 >> 172.21.15.86:0/3365215214 conn(0x558b0c7b5ba0 msgr2=0x558b0c7b4c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:56.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.450+0000 7efe52999740 1 -- 172.21.15.86:0/3365215214 shutdown_connections 2024-04-21T08:39:56.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.450+0000 7efe52999740 1 -- 172.21.15.86:0/3365215214 wait complete. 2024-04-21T08:39:56.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.450+0000 7efe52999740 1 -- start start 2024-04-21T08:39:56.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.450+0000 7efe52999740 1 Processor -- start 2024-04-21T08:39:56.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.451+0000 7efe52999740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b0c7bb4d0 0x558b0c7be7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.451+0000 7efe52999740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b0c7bbe80 0x558b0c7bed10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.451+0000 7efe52999740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b0c7c4a00 0x558b0c839e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.451+0000 7efe52999740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efe481ae590 con 0x558b0c7c4a00 2024-04-21T08:39:56.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.451+0000 7efe52999740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efe481adec0 con 0x558b0c7bbe80 2024-04-21T08:39:56.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.451+0000 7efe52999740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7efe481ab0e0 con 0x558b0c7bb4d0 2024-04-21T08:39:56.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.451+0000 7efe52999740 10 client.0 _ll_register_callbacks cb 0x7efe44098290 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:56.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.451+0000 7efe4e341640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b0c7c4a00 0x558b0c839e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.451+0000 7efe4e341640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b0c7c4a00 0x558b0c839e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45672/0 (socket says 172.21.15.86:45672) 2024-04-21T08:39:56.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.451+0000 7efe4e341640 1 -- 172.21.15.86:0/3261785101 learned_addr learned my addr 172.21.15.86:0/3261785101 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:56.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.451+0000 7efe4db40640 1 --2- 172.21.15.86:0/3261785101 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b0c7bb4d0 0x558b0c7be7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.451+0000 7efe4d33f640 1 --2- 172.21.15.86:0/3261785101 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b0c7bbe80 0x558b0c7bed10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.451+0000 7efe4e341640 1 -- 172.21.15.86:0/3261785101 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b0c7bb4d0 msgr2=0x558b0c7be7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.451+0000 7efe4e341640 1 --2- 172.21.15.86:0/3261785101 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b0c7bb4d0 0x558b0c7be7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.451+0000 7efe4e341640 1 -- 172.21.15.86:0/3261785101 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b0c7bbe80 msgr2=0x558b0c7bed10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.451+0000 7efe4e341640 1 --2- 172.21.15.86:0/3261785101 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b0c7bbe80 0x558b0c7bed10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.451+0000 7efe4e341640 1 -- 172.21.15.86:0/3261785101 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b0c83a370 con 0x558b0c7c4a00 2024-04-21T08:39:56.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.451+0000 7efe4d33f640 1 --2- 172.21.15.86:0/3261785101 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b0c7bbe80 0x558b0c7bed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:56.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.451+0000 7efe4e341640 1 --2- 172.21.15.86:0/3261785101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b0c7c4a00 0x558b0c839e30 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7efe4000aee0 tx=0x7efe40028ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a92e91503bd4abda server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:56.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.451+0000 7efe317fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:56.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.451+0000 7efe317fa640 1 -- 172.21.15.86:0/3261785101 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efe4002c040 con 0x558b0c7c4a00 2024-04-21T08:39:56.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.451+0000 7efe52999740 1 -- 172.21.15.86:0/3261785101 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efe4409c630 con 0x558b0c7c4a00 2024-04-21T08:39:56.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.451+0000 7efe317fa640 1 -- 172.21.15.86:0/3261785101 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7efe4001c050 con 0x558b0c7c4a00 2024-04-21T08:39:56.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.451+0000 7efe317fa640 1 -- 172.21.15.86:0/3261785101 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efe40027a10 con 0x558b0c7c4a00 2024-04-21T08:39:56.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.451+0000 7efe52999740 10 client.16356 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:56.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.451+0000 7efe52999740 1 -- 172.21.15.86:0/3261785101 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558b0c83b530 con 0x558b0c7c4a00 2024-04-21T08:39:56.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.452+0000 7efe317fa640 1 -- 172.21.15.86:0/3261785101 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efe4409c630 con 0x558b0c7c4a00 2024-04-21T08:39:56.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.452+0000 7efe317fa640 1 -- 172.21.15.86:0/3261785101 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7efe40038040 con 0x558b0c7c4a00 2024-04-21T08:39:56.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.452+0000 7efe52999740 10 client.16356 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:56.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.452+0000 7efe52999740 10 client.16356 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:39:56.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.452+0000 7efe52999740 20 client.16356 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:56.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.452+0000 7efe52999740 1 --2- 172.21.15.86:0/3261785101 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x558b0c83bc10 0x558b0c83e030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.452+0000 7efe52999740 4 client.16356 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:39:56.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.452+0000 7efe52999740 1 -- 172.21.15.86:0/3261785101 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558b0c72dc10 con 0x558b0c83bc10 2024-04-21T08:39:56.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.452+0000 7efe4db40640 1 --2- 172.21.15.86:0/3261785101 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x558b0c83bc10 0x558b0c83e030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.452+0000 7efe317fa640 10 client.16356 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:39:56.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.452+0000 7efe4db40640 1 --2- 172.21.15.86:0/3261785101 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x558b0c83bc10 0x558b0c83e030 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8fda4b19c98a629e server_cookie=a5148605388602f7 in_seq=0 out_seq=0 2024-04-21T08:39:56.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.453+0000 7efe317fa640 1 -- 172.21.15.86:0/3261785101 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558b0c72dc10 con 0x558b0c83bc10 2024-04-21T08:39:56.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.453+0000 7efe317fa640 10 client.16356 handle_command_reply: tid=0 2024-04-21T08:39:56.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.453+0000 7efe52999740 1 client.16356 shutdown 2024-04-21T08:39:56.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.453+0000 7efe52999740 1 -- 172.21.15.86:0/3261785101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b0c7c4a00 msgr2=0x558b0c839e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.453+0000 7efe52999740 1 --2- 172.21.15.86:0/3261785101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b0c7c4a00 0x558b0c839e30 secure :-1 s=READY pgs=732 cs=0 l=1 rev1=1 crypto rx=0x7efe4000aee0 tx=0x7efe40028ef0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.453+0000 7efe52999740 1 -- 172.21.15.86:0/3261785101 shutdown_connections 2024-04-21T08:39:56.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.453+0000 7efe52999740 1 --2- 172.21.15.86:0/3261785101 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x558b0c83bc10 0x558b0c83e030 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.453+0000 7efe52999740 1 --2- 172.21.15.86:0/3261785101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b0c7c4a00 0x558b0c839e30 unknown :-1 s=CLOSED pgs=732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.453+0000 7efe52999740 1 --2- 172.21.15.86:0/3261785101 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b0c7bbe80 0x558b0c7bed10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.453+0000 7efe52999740 1 --2- 172.21.15.86:0/3261785101 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b0c7bb4d0 0x558b0c7be7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.453+0000 7efe317fa640 0 client.16356 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:39:56.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.453+0000 7efe52999740 1 -- 172.21.15.86:0/3261785101 >> 172.21.15.86:0/3261785101 conn(0x558b0c7b5ba0 msgr2=0x558b0c7b6ea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:56.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.453+0000 7efe52999740 1 -- 172.21.15.86:0/3261785101 shutdown_connections 2024-04-21T08:39:56.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.453+0000 7efe52999740 1 -- 172.21.15.86:0/3261785101 wait complete. 2024-04-21T08:39:56.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.453+0000 7efe52999740 20 client.16356 trim_cache size 0 max 16384 2024-04-21T08:39:56.459 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:39:56.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.453+0000 7efe4fda4640 1 -- 172.21.15.86:0/1098115357 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efe340a3e40 msgr2=0x7efe340a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.453+0000 7efe4fda4640 1 --2- 172.21.15.86:0/1098115357 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efe340a3e40 0x7efe340a6300 secure :-1 s=READY pgs=101 cs=0 l=1 rev1=1 crypto rx=0x7efe4000afd0 tx=0x7efe4001e000 comp rx=0 tx=0).stop 2024-04-21T08:39:56.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.453+0000 7efe4fda4640 1 -- 172.21.15.86:0/1098115357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe48100af0 msgr2=0x7efe4819e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.453+0000 7efe4fda4640 1 --2- 172.21.15.86:0/1098115357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe48100af0 0x7efe4819e460 secure :-1 s=READY pgs=728 cs=0 l=1 rev1=1 crypto rx=0x7efe4400c970 tx=0x7efe4400ce30 comp rx=0 tx=0).stop 2024-04-21T08:39:56.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.453+0000 7efe4fda4640 1 -- 172.21.15.86:0/1098115357 shutdown_connections 2024-04-21T08:39:56.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.453+0000 7efe4fda4640 1 --2- 172.21.15.86:0/1098115357 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efe340a3e40 0x7efe340a6300 unknown :-1 s=CLOSED pgs=101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.453+0000 7efe4fda4640 1 --2- 172.21.15.86:0/1098115357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efe48198610 0x7efe48198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.453+0000 7efe4fda4640 1 --2- 172.21.15.86:0/1098115357 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efe481072b0 0x7efe4819e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.453+0000 7efe4fda4640 1 --2- 172.21.15.86:0/1098115357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efe48100af0 0x7efe4819e460 unknown :-1 s=CLOSED pgs=728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.453+0000 7efe4fda4640 1 -- 172.21.15.86:0/1098115357 >> 172.21.15.86:0/1098115357 conn(0x7efe480760e0 msgr2=0x7efe480fecf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:56.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.454+0000 7efe4fda4640 1 -- 172.21.15.86:0/1098115357 shutdown_connections 2024-04-21T08:39:56.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.454+0000 7efe4fda4640 1 -- 172.21.15.86:0/1098115357 wait complete. 2024-04-21T08:39:56.462 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:56 smithi086 ceph-mon[27939]: pgmap v37: 129 pgs: 129 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-04-21T08:39:56.462 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:56 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3828276445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:39:56.463 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:39:56.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.564+0000 7fe61024e640 1 -- 172.21.15.86:0/3300160872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe60810e830 msgr2=0x7fe608110c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.564+0000 7fe61024e640 1 --2- 172.21.15.86:0/3300160872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe60810e830 0x7fe608110c20 secure :-1 s=READY pgs=733 cs=0 l=1 rev1=1 crypto rx=0x7fe600009ca0 tx=0x7fe60002c180 comp rx=0 tx=0).stop 2024-04-21T08:39:56.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.564+0000 7fe61024e640 1 -- 172.21.15.86:0/3300160872 shutdown_connections 2024-04-21T08:39:56.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.564+0000 7fe61024e640 1 --2- 172.21.15.86:0/3300160872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe60810e830 0x7fe608110c20 unknown :-1 s=CLOSED pgs=733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.564+0000 7fe61024e640 1 --2- 172.21.15.86:0/3300160872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe608074b40 0x7fe60810e2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.564+0000 7fe61024e640 1 --2- 172.21.15.86:0/3300160872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe608074220 0x7fe608074600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.564+0000 7fe61024e640 1 -- 172.21.15.86:0/3300160872 >> 172.21.15.86:0/3300160872 conn(0x7fe6080793a0 msgr2=0x7fe6080797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:56.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.565+0000 7fe61024e640 1 -- 172.21.15.86:0/3300160872 shutdown_connections 2024-04-21T08:39:56.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.565+0000 7fe61024e640 1 -- 172.21.15.86:0/3300160872 wait complete. 2024-04-21T08:39:56.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.565+0000 7fe61024e640 1 Processor -- start 2024-04-21T08:39:56.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.565+0000 7fe61024e640 1 -- start start 2024-04-21T08:39:56.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.565+0000 7fe61024e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe608074220 0x7fe6081a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.565+0000 7fe61024e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe608074b40 0x7fe6081a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.565+0000 7fe61024e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe60810e830 0x7fe60819a800 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.565+0000 7fe61024e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe6081143c0 con 0x7fe608074220 2024-04-21T08:39:56.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.565+0000 7fe61024e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe608114240 con 0x7fe608074b40 2024-04-21T08:39:56.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.565+0000 7fe61024e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe608114540 con 0x7fe60810e830 2024-04-21T08:39:56.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.565+0000 7fe60dfea640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe608074220 0x7fe6081a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.565+0000 7fe60dfea640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe608074220 0x7fe6081a0690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45698/0 (socket says 172.21.15.86:45698) 2024-04-21T08:39:56.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.565+0000 7fe60dfea640 1 -- 172.21.15.86:0/4162165262 learned_addr learned my addr 172.21.15.86:0/4162165262 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:56.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.566+0000 7fe60d7e9640 1 --2- 172.21.15.86:0/4162165262 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe608074b40 0x7fe6081a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.566+0000 7fe60e7eb640 1 --2- 172.21.15.86:0/4162165262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe60810e830 0x7fe60819a800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.566+0000 7fe60dfea640 1 -- 172.21.15.86:0/4162165262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe60810e830 msgr2=0x7fe60819a800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.566+0000 7fe60dfea640 1 --2- 172.21.15.86:0/4162165262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe60810e830 0x7fe60819a800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.566+0000 7fe60dfea640 1 -- 172.21.15.86:0/4162165262 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe608074b40 msgr2=0x7fe6081a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.566+0000 7fe60dfea640 1 --2- 172.21.15.86:0/4162165262 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe608074b40 0x7fe6081a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.566+0000 7fe60dfea640 1 -- 172.21.15.86:0/4162165262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe60819afd0 con 0x7fe608074220 2024-04-21T08:39:56.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.566+0000 7fe60d7e9640 1 --2- 172.21.15.86:0/4162165262 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe608074b40 0x7fe6081a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:56.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.566+0000 7fe60dfea640 1 --2- 172.21.15.86:0/4162165262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe608074220 0x7fe6081a0690 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fe60400e9f0 tx=0x7fe60400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:56.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.566+0000 7fe5feffd640 1 -- 172.21.15.86:0/4162165262 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60400cd80 con 0x7fe608074220 2024-04-21T08:39:56.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.566+0000 7fe61024e640 1 -- 172.21.15.86:0/4162165262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe60819b2c0 con 0x7fe608074220 2024-04-21T08:39:56.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.566+0000 7fe5feffd640 1 -- 172.21.15.86:0/4162165262 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe6040108e0 con 0x7fe608074220 2024-04-21T08:39:56.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.566+0000 7fe5feffd640 1 -- 172.21.15.86:0/4162165262 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe604018980 con 0x7fe608074220 2024-04-21T08:39:56.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.566+0000 7fe61024e640 1 -- 172.21.15.86:0/4162165262 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe6081a74e0 con 0x7fe608074220 2024-04-21T08:39:56.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.569+0000 7fe5feffd640 1 -- 172.21.15.86:0/4162165262 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7fe604010430 con 0x7fe608074220 2024-04-21T08:39:56.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.569+0000 7fe612e43740 1 Processor -- start 2024-04-21T08:39:56.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.569+0000 7fe612e43740 1 -- start start 2024-04-21T08:39:56.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.569+0000 7fe612e43740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7592eb4d0 0x55d7592eb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.569+0000 7fe612e43740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d7592ebe80 0x55d7592f4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.569+0000 7fe612e43740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7592f4a00 0x55d7592f6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.569+0000 7fe612e43740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d7592501f0 con 0x55d7592f4a00 2024-04-21T08:39:56.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.569+0000 7fe612e43740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d7591ed4b0 con 0x55d7592ebe80 2024-04-21T08:39:56.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.569+0000 7fe612e43740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d75924c1b0 con 0x55d7592eb4d0 2024-04-21T08:39:56.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.569+0000 7fe60d7e9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7592f4a00 0x55d7592f6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.569+0000 7fe60d7e9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7592f4a00 0x55d7592f6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45704/0 (socket says 172.21.15.86:45704) 2024-04-21T08:39:56.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.569+0000 7fe60d7e9640 1 -- 172.21.15.86:0/2122555904 learned_addr learned my addr 172.21.15.86:0/2122555904 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:56.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.569+0000 7fe60dfea640 1 --2- 172.21.15.86:0/2122555904 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d7592ebe80 0x55d7592f4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.570+0000 7fe60e7eb640 1 --2- 172.21.15.86:0/2122555904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7592eb4d0 0x55d7592eb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.570+0000 7fe5feffd640 1 --2- 172.21.15.86:0/4162165262 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe5f40a3e40 0x7fe5f40a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.570+0000 7fe60d7e9640 1 -- 172.21.15.86:0/2122555904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7592eb4d0 msgr2=0x55d7592eb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.570+0000 7fe60d7e9640 1 --2- 172.21.15.86:0/2122555904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7592eb4d0 0x55d7592eb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.570+0000 7fe60d7e9640 1 -- 172.21.15.86:0/2122555904 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d7592ebe80 msgr2=0x55d7592f4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.570+0000 7fe60d7e9640 1 --2- 172.21.15.86:0/2122555904 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d7592ebe80 0x55d7592f4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.570+0000 7fe60d7e9640 1 -- 172.21.15.86:0/2122555904 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7591df1b0 con 0x55d7592f4a00 2024-04-21T08:39:56.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.570+0000 7fe60dfea640 1 --2- 172.21.15.86:0/2122555904 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d7592ebe80 0x55d7592f4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:56.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.570+0000 7fe60e7eb640 1 --2- 172.21.15.86:0/4162165262 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe5f40a3e40 0x7fe5f40a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.570+0000 7fe5feffd640 1 -- 172.21.15.86:0/4162165262 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fe6040d1260 con 0x7fe608074220 2024-04-21T08:39:56.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.570+0000 7fe60d7e9640 1 --2- 172.21.15.86:0/2122555904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7592f4a00 0x55d7592f6e50 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8009c30 tx=0x7fe5f80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=439a8dfd2156ce13 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:56.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.570+0000 7fe5f3fff640 1 -- 172.21.15.86:0/2122555904 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8012450 con 0x55d7592f4a00 2024-04-21T08:39:56.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.570+0000 7fe5f3fff640 1 -- 172.21.15.86:0/2122555904 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5f8012aa0 con 0x55d7592f4a00 2024-04-21T08:39:56.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.570+0000 7fe5f3fff640 1 -- 172.21.15.86:0/2122555904 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f800fa90 con 0x55d7592f4a00 2024-04-21T08:39:56.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.571+0000 7fe612e43740 1 -- 172.21.15.86:0/2122555904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7592f4a00 msgr2=0x55d7592f6e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.571+0000 7fe612e43740 1 --2- 172.21.15.86:0/2122555904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7592f4a00 0x55d7592f6e50 secure :-1 s=READY pgs=735 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8009c30 tx=0x7fe5f80117b0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.571+0000 7fe60e7eb640 1 --2- 172.21.15.86:0/4162165262 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe5f40a3e40 0x7fe5f40a6300 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe600006010 tx=0x7fe60002e040 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:56.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.571+0000 7fe612e43740 1 -- 172.21.15.86:0/2122555904 shutdown_connections 2024-04-21T08:39:56.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.571+0000 7fe612e43740 1 --2- 172.21.15.86:0/2122555904 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7592f4a00 0x55d7592f6e50 unknown :-1 s=CLOSED pgs=735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.571+0000 7fe612e43740 1 --2- 172.21.15.86:0/2122555904 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d7592ebe80 0x55d7592f4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.571+0000 7fe612e43740 1 --2- 172.21.15.86:0/2122555904 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7592eb4d0 0x55d7592eb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.571+0000 7fe612e43740 1 -- 172.21.15.86:0/2122555904 >> 172.21.15.86:0/2122555904 conn(0x55d7592e5ba0 msgr2=0x55d7592e5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:56.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.571+0000 7fe612e43740 1 -- 172.21.15.86:0/2122555904 shutdown_connections 2024-04-21T08:39:56.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.571+0000 7fe612e43740 1 -- 172.21.15.86:0/2122555904 wait complete. 2024-04-21T08:39:56.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.571+0000 7fe612e43740 1 -- start start 2024-04-21T08:39:56.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.571+0000 7fe612e43740 1 Processor -- start 2024-04-21T08:39:56.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.572+0000 7fe612e43740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7592eb4d0 0x55d7592f42f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.572+0000 7fe612e43740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7592ebe80 0x55d7592f1680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.572+0000 7fe612e43740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d7592f4a00 0x55d7592f1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.572+0000 7fe612e43740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d7592f7970 con 0x55d7592eb4d0 2024-04-21T08:39:56.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.572+0000 7fe612e43740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d759209500 con 0x55d7592f4a00 2024-04-21T08:39:56.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.572+0000 7fe612e43740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d7592f7af0 con 0x55d7592ebe80 2024-04-21T08:39:56.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.572+0000 7fe612e43740 10 client.0 _ll_register_callbacks cb 0x55d7591d7b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:56.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.572+0000 7fe60dfea640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7592eb4d0 0x55d7592f42f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.572+0000 7fe60dfea640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7592eb4d0 0x55d7592f42f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45720/0 (socket says 172.21.15.86:45720) 2024-04-21T08:39:56.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.572+0000 7fe60dfea640 1 -- 172.21.15.86:0/2212694228 learned_addr learned my addr 172.21.15.86:0/2212694228 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:56.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.572+0000 7fe60d7e9640 1 --2- 172.21.15.86:0/2212694228 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7592ebe80 0x55d7592f1680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.572+0000 7fe60e7eb640 1 --2- 172.21.15.86:0/2212694228 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d7592f4a00 0x55d7592f1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.572+0000 7fe60dfea640 1 -- 172.21.15.86:0/2212694228 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7592ebe80 msgr2=0x55d7592f1680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.572+0000 7fe60dfea640 1 --2- 172.21.15.86:0/2212694228 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7592ebe80 0x55d7592f1680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.572+0000 7fe60dfea640 1 -- 172.21.15.86:0/2212694228 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d7592f4a00 msgr2=0x55d7592f1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.572+0000 7fe60dfea640 1 --2- 172.21.15.86:0/2212694228 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d7592f4a00 0x55d7592f1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.572+0000 7fe60dfea640 1 -- 172.21.15.86:0/2212694228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d758f623f0 con 0x55d7592eb4d0 2024-04-21T08:39:56.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.572+0000 7fe60d7e9640 1 --2- 172.21.15.86:0/2212694228 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7592ebe80 0x55d7592f1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:39:56.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.572+0000 7fe60e7eb640 1 --2- 172.21.15.86:0/2212694228 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d7592f4a00 0x55d7592f1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:56.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.572+0000 7fe60dfea640 1 --2- 172.21.15.86:0/2212694228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7592eb4d0 0x55d7592f42f0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe60409c940 tx=0x7fe60400cf20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4cc49aec4c1ce18f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:56.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.572+0000 7fe5f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:56.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.572+0000 7fe5f17fa640 1 -- 172.21.15.86:0/2212694228 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe60409d340 con 0x55d7592eb4d0 2024-04-21T08:39:56.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.572+0000 7fe5f17fa640 1 -- 172.21.15.86:0/2212694228 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fe6040bd0b0 con 0x55d7592eb4d0 2024-04-21T08:39:56.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.572+0000 7fe612e43740 1 -- 172.21.15.86:0/2212694228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5f8012aa0 con 0x55d7592eb4d0 2024-04-21T08:39:56.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.573+0000 7fe5f17fa640 1 -- 172.21.15.86:0/2212694228 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe6040bd2b0 con 0x55d7592eb4d0 2024-04-21T08:39:56.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.573+0000 7fe612e43740 10 client.16368 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:56.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.573+0000 7fe612e43740 1 -- 172.21.15.86:0/2212694228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d75902c7d0 con 0x55d7592eb4d0 2024-04-21T08:39:56.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.573+0000 7fe5f17fa640 1 -- 172.21.15.86:0/2212694228 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5f8012aa0 con 0x55d7592eb4d0 2024-04-21T08:39:56.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.573+0000 7fe5f17fa640 1 -- 172.21.15.86:0/2212694228 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe60409c090 con 0x55d7592eb4d0 2024-04-21T08:39:56.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.573+0000 7fe612e43740 10 client.16368 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:56.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.573+0000 7fe612e43740 10 client.16368 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.573+0000 7fe612e43740 20 client.16368 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.573+0000 7fe612e43740 1 --2- 172.21.15.86:0/2212694228 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d759304ba0 0x55d759306ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.573+0000 7fe612e43740 4 client.16368 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.573+0000 7fe612e43740 1 -- 172.21.15.86:0/2212694228 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d7591ef930 con 0x55d759304ba0 2024-04-21T08:39:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.573+0000 7fe60d7e9640 1 --2- 172.21.15.86:0/2212694228 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d759304ba0 0x55d759306ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.574+0000 7fe5f17fa640 10 client.16368 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.574+0000 7fe60d7e9640 1 --2- 172.21.15.86:0/2212694228 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d759304ba0 0x55d759306ff0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79c79de4598156b6 server_cookie=5a4076aa935ceff5 in_seq=0 out_seq=0 2024-04-21T08:39:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.574+0000 7fe5f17fa640 1 -- 172.21.15.86:0/2212694228 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d7591ef930 con 0x55d759304ba0 2024-04-21T08:39:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.574+0000 7fe5f17fa640 10 client.16368 handle_command_reply: tid=0 2024-04-21T08:39:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.574+0000 7fe612e43740 1 client.16368 shutdown 2024-04-21T08:39:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.574+0000 7fe612e43740 1 -- 172.21.15.86:0/2212694228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7592eb4d0 msgr2=0x55d7592f42f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.574+0000 7fe612e43740 1 --2- 172.21.15.86:0/2212694228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7592eb4d0 0x55d7592f42f0 secure :-1 s=READY pgs=736 cs=0 l=1 rev1=1 crypto rx=0x7fe60409c940 tx=0x7fe60400cf20 comp rx=0 tx=0).stop 2024-04-21T08:39:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.574+0000 7fe612e43740 1 -- 172.21.15.86:0/2212694228 shutdown_connections 2024-04-21T08:39:56.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.574+0000 7fe612e43740 1 --2- 172.21.15.86:0/2212694228 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d759304ba0 0x55d759306ff0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.574+0000 7fe612e43740 1 --2- 172.21.15.86:0/2212694228 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d7592f4a00 0x55d7592f1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.574+0000 7fe612e43740 1 --2- 172.21.15.86:0/2212694228 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7592ebe80 0x55d7592f1680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.574+0000 7fe612e43740 1 --2- 172.21.15.86:0/2212694228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7592eb4d0 0x55d7592f42f0 unknown :-1 s=CLOSED pgs=736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.574+0000 7fe5f17fa640 0 client.16368 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:56.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.574+0000 7fe612e43740 1 -- 172.21.15.86:0/2212694228 >> 172.21.15.86:0/2212694228 conn(0x55d7592e5ba0 msgr2=0x55d7592e78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:56.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.575+0000 7fe612e43740 1 -- 172.21.15.86:0/2212694228 shutdown_connections 2024-04-21T08:39:56.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.575+0000 7fe612e43740 1 -- 172.21.15.86:0/2212694228 wait complete. 2024-04-21T08:39:56.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.575+0000 7fe612e43740 20 client.16368 trim_cache size 0 max 16384 2024-04-21T08:39:56.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.588+0000 7fe612e43740 1 Processor -- start 2024-04-21T08:39:56.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.588+0000 7fe612e43740 1 -- start start 2024-04-21T08:39:56.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.588+0000 7fe612e43740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d7592eb4d0 0x55d759300830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.588+0000 7fe612e43740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7592ebe80 0x55d7592e76e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.588+0000 7fe612e43740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7592f4a00 0x55d7592f3db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.588+0000 7fe612e43740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe6081b15b0 con 0x55d7592ebe80 2024-04-21T08:39:56.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.588+0000 7fe612e43740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe6081b30f0 con 0x55d7592eb4d0 2024-04-21T08:39:56.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.588+0000 7fe612e43740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe6081b1c80 con 0x55d7592f4a00 2024-04-21T08:39:56.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.588+0000 7fe60d7e9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7592ebe80 0x55d7592e76e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.588+0000 7fe60d7e9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7592ebe80 0x55d7592e76e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45724/0 (socket says 172.21.15.86:45724) 2024-04-21T08:39:56.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.588+0000 7fe60d7e9640 1 -- 172.21.15.86:0/2357730747 learned_addr learned my addr 172.21.15.86:0/2357730747 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:56.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.588+0000 7fe60dfea640 1 --2- 172.21.15.86:0/2357730747 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d7592eb4d0 0x55d759300830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.588+0000 7fe60e7eb640 1 --2- 172.21.15.86:0/2357730747 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7592f4a00 0x55d7592f3db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.588+0000 7fe60d7e9640 1 -- 172.21.15.86:0/2357730747 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7592f4a00 msgr2=0x55d7592f3db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.588+0000 7fe60d7e9640 1 --2- 172.21.15.86:0/2357730747 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7592f4a00 0x55d7592f3db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.588+0000 7fe60d7e9640 1 -- 172.21.15.86:0/2357730747 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d7592eb4d0 msgr2=0x55d759300830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.588+0000 7fe60d7e9640 1 --2- 172.21.15.86:0/2357730747 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d7592eb4d0 0x55d759300830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.588+0000 7fe60d7e9640 1 -- 172.21.15.86:0/2357730747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d7592eb090 con 0x55d7592ebe80 2024-04-21T08:39:56.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.588+0000 7fe60d7e9640 1 --2- 172.21.15.86:0/2357730747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7592ebe80 0x55d7592e76e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8009c00 tx=0x7fe5f8009b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b27e427f3488f9fd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:56.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.589+0000 7fe5f37fe640 1 -- 172.21.15.86:0/2357730747 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f8034070 con 0x55d7592ebe80 2024-04-21T08:39:56.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.589+0000 7fe5f37fe640 1 -- 172.21.15.86:0/2357730747 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5f8012900 con 0x55d7592ebe80 2024-04-21T08:39:56.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.589+0000 7fe5f37fe640 1 -- 172.21.15.86:0/2357730747 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe5f80055e0 con 0x55d7592ebe80 2024-04-21T08:39:56.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.589+0000 7fe612e43740 1 -- 172.21.15.86:0/2357730747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7592ebe80 msgr2=0x55d7592e76e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.589+0000 7fe612e43740 1 --2- 172.21.15.86:0/2357730747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7592ebe80 0x55d7592e76e0 secure :-1 s=READY pgs=737 cs=0 l=1 rev1=1 crypto rx=0x7fe5f8009c00 tx=0x7fe5f8009b30 comp rx=0 tx=0).stop 2024-04-21T08:39:56.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.589+0000 7fe612e43740 1 -- 172.21.15.86:0/2357730747 shutdown_connections 2024-04-21T08:39:56.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.589+0000 7fe612e43740 1 --2- 172.21.15.86:0/2357730747 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7592f4a00 0x55d7592f3db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.589+0000 7fe612e43740 1 --2- 172.21.15.86:0/2357730747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7592ebe80 0x55d7592e76e0 unknown :-1 s=CLOSED pgs=737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.589+0000 7fe612e43740 1 --2- 172.21.15.86:0/2357730747 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d7592eb4d0 0x55d759300830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.589+0000 7fe612e43740 1 -- 172.21.15.86:0/2357730747 >> 172.21.15.86:0/2357730747 conn(0x55d7592e5ba0 msgr2=0x55d7592e6fd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:56.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.589+0000 7fe612e43740 1 -- 172.21.15.86:0/2357730747 shutdown_connections 2024-04-21T08:39:56.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.589+0000 7fe612e43740 1 -- 172.21.15.86:0/2357730747 wait complete. 2024-04-21T08:39:56.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.589+0000 7fe612e43740 1 -- start start 2024-04-21T08:39:56.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.589+0000 7fe612e43740 1 Processor -- start 2024-04-21T08:39:56.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.590+0000 7fe612e43740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d7592eb4d0 0x55d7592fcf30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.590+0000 7fe612e43740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7592ebe80 0x55d7592eded0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.590+0000 7fe612e43740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7592f4a00 0x55d7592ee410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.590+0000 7fe612e43740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe6081b2a20 con 0x55d7592f4a00 2024-04-21T08:39:56.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.590+0000 7fe612e43740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe6081b2350 con 0x55d7592eb4d0 2024-04-21T08:39:56.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.590+0000 7fe612e43740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe6081af570 con 0x55d7592ebe80 2024-04-21T08:39:56.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.590+0000 7fe612e43740 10 client.0 _ll_register_callbacks cb 0x7fe60409ca10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:56.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.590+0000 7fe60e7eb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7592f4a00 0x55d7592ee410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.590+0000 7fe60e7eb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7592f4a00 0x55d7592ee410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45734/0 (socket says 172.21.15.86:45734) 2024-04-21T08:39:56.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.590+0000 7fe60e7eb640 1 -- 172.21.15.86:0/1676069064 learned_addr learned my addr 172.21.15.86:0/1676069064 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:56.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.590+0000 7fe60dfea640 1 --2- 172.21.15.86:0/1676069064 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d7592eb4d0 0x55d7592fcf30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.590+0000 7fe60d7e9640 1 --2- 172.21.15.86:0/1676069064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7592ebe80 0x55d7592eded0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.590+0000 7fe60e7eb640 1 -- 172.21.15.86:0/1676069064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7592ebe80 msgr2=0x55d7592eded0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.590+0000 7fe60e7eb640 1 --2- 172.21.15.86:0/1676069064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7592ebe80 0x55d7592eded0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.590+0000 7fe60e7eb640 1 -- 172.21.15.86:0/1676069064 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d7592eb4d0 msgr2=0x55d7592fcf30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:56.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.590+0000 7fe60e7eb640 1 --2- 172.21.15.86:0/1676069064 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d7592eb4d0 0x55d7592fcf30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.590+0000 7fe60e7eb640 1 -- 172.21.15.86:0/1676069064 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d7592ee950 con 0x55d7592f4a00 2024-04-21T08:39:56.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.590+0000 7fe60e7eb640 1 --2- 172.21.15.86:0/1676069064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7592f4a00 0x55d7592ee410 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fe600009ca0 tx=0x7fe600005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31555cf9f141842b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:56.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.590+0000 7fe5f17fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:56.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.591+0000 7fe5f17fa640 1 -- 172.21.15.86:0/1676069064 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe600039040 con 0x55d7592f4a00 2024-04-21T08:39:56.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.591+0000 7fe5f17fa640 1 -- 172.21.15.86:0/1676069064 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fe600034850 con 0x55d7592f4a00 2024-04-21T08:39:56.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.591+0000 7fe612e43740 1 -- 172.21.15.86:0/1676069064 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe5f8012900 con 0x55d7592f4a00 2024-04-21T08:39:56.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.591+0000 7fe5f17fa640 1 -- 172.21.15.86:0/1676069064 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe600034a50 con 0x55d7592f4a00 2024-04-21T08:39:56.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.591+0000 7fe612e43740 10 client.16374 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:56.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.591+0000 7fe612e43740 1 -- 172.21.15.86:0/1676069064 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d75936ae00 con 0x55d7592f4a00 2024-04-21T08:39:56.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.591+0000 7fe5f17fa640 1 -- 172.21.15.86:0/1676069064 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe5f8012900 con 0x55d7592f4a00 2024-04-21T08:39:56.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.591+0000 7fe5f17fa640 1 -- 172.21.15.86:0/1676069064 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe60004c3f0 con 0x55d7592f4a00 2024-04-21T08:39:56.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.591+0000 7fe612e43740 10 client.16374 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:56.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.591+0000 7fe612e43740 10 client.16374 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:56.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.591+0000 7fe612e43740 20 client.16374 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:56.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.591+0000 7fe612e43740 1 --2- 172.21.15.86:0/1676069064 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d75936b580 0x55d75936d9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:56.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.591+0000 7fe612e43740 4 client.16374 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:39:56.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.591+0000 7fe612e43740 1 -- 172.21.15.86:0/1676069064 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d75925dc10 con 0x55d75936b580 2024-04-21T08:39:56.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.591+0000 7fe60dfea640 1 --2- 172.21.15.86:0/1676069064 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d75936b580 0x55d75936d9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:56.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.592+0000 7fe5f17fa640 10 client.16374 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:56.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.592+0000 7fe60dfea640 1 --2- 172.21.15.86:0/1676069064 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d75936b580 0x55d75936d9a0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3878bf101ce3021 server_cookie=99e1d93f0d767299 in_seq=0 out_seq=0 2024-04-21T08:39:56.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.592+0000 7fe5f17fa640 1 -- 172.21.15.86:0/1676069064 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d75925dc10 con 0x55d75936b580 2024-04-21T08:39:56.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.592+0000 7fe5f17fa640 10 client.16374 handle_command_reply: tid=0 2024-04-21T08:39:56.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.592+0000 7fe612e43740 1 client.16374 shutdown 2024-04-21T08:39:56.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.592+0000 7fe612e43740 1 -- 172.21.15.86:0/1676069064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7592f4a00 msgr2=0x55d7592ee410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.592+0000 7fe612e43740 1 --2- 172.21.15.86:0/1676069064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7592f4a00 0x55d7592ee410 secure :-1 s=READY pgs=738 cs=0 l=1 rev1=1 crypto rx=0x7fe600009ca0 tx=0x7fe600005d80 comp rx=0 tx=0).stop 2024-04-21T08:39:56.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.592+0000 7fe612e43740 1 -- 172.21.15.86:0/1676069064 shutdown_connections 2024-04-21T08:39:56.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.592+0000 7fe612e43740 1 --2- 172.21.15.86:0/1676069064 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d75936b580 0x55d75936d9a0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.592+0000 7fe612e43740 1 --2- 172.21.15.86:0/1676069064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d7592f4a00 0x55d7592ee410 unknown :-1 s=CLOSED pgs=738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.592+0000 7fe612e43740 1 --2- 172.21.15.86:0/1676069064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d7592ebe80 0x55d7592eded0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.592+0000 7fe612e43740 1 --2- 172.21.15.86:0/1676069064 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d7592eb4d0 0x55d7592fcf30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.592+0000 7fe5f17fa640 0 client.16374 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:56.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.592+0000 7fe612e43740 1 -- 172.21.15.86:0/1676069064 >> 172.21.15.86:0/1676069064 conn(0x55d7592e5ba0 msgr2=0x55d7592e4c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:56.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.592+0000 7fe612e43740 1 -- 172.21.15.86:0/1676069064 shutdown_connections 2024-04-21T08:39:56.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.592+0000 7fe612e43740 1 -- 172.21.15.86:0/1676069064 wait complete. 2024-04-21T08:39:56.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.592+0000 7fe612e43740 20 client.16374 trim_cache size 0 max 16384 2024-04-21T08:39:56.598 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:39:56.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.593+0000 7fe61024e640 1 -- 172.21.15.86:0/4162165262 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe5f40a3e40 msgr2=0x7fe5f40a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.593+0000 7fe61024e640 1 --2- 172.21.15.86:0/4162165262 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe5f40a3e40 0x7fe5f40a6300 secure :-1 s=READY pgs=102 cs=0 l=1 rev1=1 crypto rx=0x7fe600006010 tx=0x7fe60002e040 comp rx=0 tx=0).stop 2024-04-21T08:39:56.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.593+0000 7fe61024e640 1 -- 172.21.15.86:0/4162165262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe608074220 msgr2=0x7fe6081a0690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:56.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.593+0000 7fe61024e640 1 --2- 172.21.15.86:0/4162165262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe608074220 0x7fe6081a0690 secure :-1 s=READY pgs=734 cs=0 l=1 rev1=1 crypto rx=0x7fe60400e9f0 tx=0x7fe60400eeb0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.593+0000 7fe61024e640 1 -- 172.21.15.86:0/4162165262 shutdown_connections 2024-04-21T08:39:56.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.593+0000 7fe61024e640 1 --2- 172.21.15.86:0/4162165262 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe5f40a3e40 0x7fe5f40a6300 unknown :-1 s=CLOSED pgs=102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.593+0000 7fe61024e640 1 --2- 172.21.15.86:0/4162165262 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe60810e830 0x7fe60819a800 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.593+0000 7fe61024e640 1 --2- 172.21.15.86:0/4162165262 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe608074b40 0x7fe6081a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.593+0000 7fe61024e640 1 --2- 172.21.15.86:0/4162165262 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe608074220 0x7fe6081a0690 unknown :-1 s=CLOSED pgs=734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:56.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.593+0000 7fe61024e640 1 -- 172.21.15.86:0/4162165262 >> 172.21.15.86:0/4162165262 conn(0x7fe6080793a0 msgr2=0x7fe6081002e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:56.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.593+0000 7fe61024e640 1 -- 172.21.15.86:0/4162165262 shutdown_connections 2024-04-21T08:39:56.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:56.593+0000 7fe61024e640 1 -- 172.21.15.86:0/4162165262 wait complete. 2024-04-21T08:39:56.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:56 smithi088 ceph-mon[31259]: pgmap v37: 129 pgs: 129 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-04-21T08:39:56.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:56 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3828276445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:39:56.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:56 smithi179 ceph-mon[31829]: pgmap v37: 129 pgs: 129 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-04-21T08:39:56.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:56 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3828276445' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:39:57.603 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:39:57.603 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:39:57.603 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:39:57.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.714+0000 7f97f2d3d640 1 -- 172.21.15.86:0/517664647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97ec06a550 msgr2=0x7f97ec06a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:57.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.714+0000 7f97f2d3d640 1 --2- 172.21.15.86:0/517664647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97ec06a550 0x7f97ec06a990 secure :-1 s=READY pgs=739 cs=0 l=1 rev1=1 crypto rx=0x7f97d8009930 tx=0x7f97d802bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:57.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.714+0000 7f97f2d3d640 1 -- 172.21.15.86:0/517664647 shutdown_connections 2024-04-21T08:39:57.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.714+0000 7f97f2d3d640 1 --2- 172.21.15.86:0/517664647 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97ec06aed0 0x7f97ec10edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.714+0000 7f97f2d3d640 1 --2- 172.21.15.86:0/517664647 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97ec06a550 0x7f97ec06a990 unknown :-1 s=CLOSED pgs=739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.714+0000 7f97f2d3d640 1 --2- 172.21.15.86:0/517664647 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97ec107fc0 0x7f97ec1083a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.714+0000 7f97f2d3d640 1 -- 172.21.15.86:0/517664647 >> 172.21.15.86:0/517664647 conn(0x7f97ec076c50 msgr2=0x7f97ec077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:57.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.714+0000 7f97f2d3d640 1 -- 172.21.15.86:0/517664647 shutdown_connections 2024-04-21T08:39:57.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.714+0000 7f97f2d3d640 1 -- 172.21.15.86:0/517664647 wait complete. 2024-04-21T08:39:57.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.715+0000 7f97f2d3d640 1 Processor -- start 2024-04-21T08:39:57.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.715+0000 7f97f2d3d640 1 -- start start 2024-04-21T08:39:57.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.715+0000 7f97f2d3d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97ec06a550 0x7f97ec198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:57.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.715+0000 7f97f2d3d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97ec06aed0 0x7f97ec198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:57.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.715+0000 7f97f2d3d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97ec107fc0 0x7f97ec19d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:57.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.715+0000 7f97f2d3d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec112190 con 0x7f97ec06a550 2024-04-21T08:39:57.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.715+0000 7f97f2d3d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec112010 con 0x7f97ec06aed0 2024-04-21T08:39:57.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.715+0000 7f97f2d3d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec112310 con 0x7f97ec107fc0 2024-04-21T08:39:57.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.715+0000 7f97f0ad9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97ec06a550 0x7f97ec198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:57.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.715+0000 7f97f0ad9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97ec06a550 0x7f97ec198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45760/0 (socket says 172.21.15.86:45760) 2024-04-21T08:39:57.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.715+0000 7f97f0ad9640 1 -- 172.21.15.86:0/1732887514 learned_addr learned my addr 172.21.15.86:0/1732887514 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:57.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.715+0000 7f97f12da640 1 --2- 172.21.15.86:0/1732887514 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97ec107fc0 0x7f97ec19d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:57.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.715+0000 7f97f0ad9640 1 -- 172.21.15.86:0/1732887514 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97ec107fc0 msgr2=0x7f97ec19d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:57.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.715+0000 7f97f0ad9640 1 --2- 172.21.15.86:0/1732887514 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97ec107fc0 0x7f97ec19d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.715+0000 7f97ebfff640 1 --2- 172.21.15.86:0/1732887514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97ec06aed0 0x7f97ec198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:57.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.715+0000 7f97f0ad9640 1 -- 172.21.15.86:0/1732887514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97ec06aed0 msgr2=0x7f97ec198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:57.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.715+0000 7f97f0ad9640 1 --2- 172.21.15.86:0/1732887514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97ec06aed0 0x7f97ec198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.715+0000 7f97f0ad9640 1 -- 172.21.15.86:0/1732887514 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97ec19d740 con 0x7f97ec06a550 2024-04-21T08:39:57.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.715+0000 7f97ebfff640 1 --2- 172.21.15.86:0/1732887514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97ec06aed0 0x7f97ec198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:57.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.715+0000 7f97f0ad9640 1 --2- 172.21.15.86:0/1732887514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97ec06a550 0x7f97ec198750 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f97e000d950 tx=0x7f97e000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:57.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.716+0000 7f97e9ffb640 1 -- 172.21.15.86:0/1732887514 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e0017070 con 0x7f97ec06a550 2024-04-21T08:39:57.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.716+0000 7f97e9ffb640 1 -- 172.21.15.86:0/1732887514 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97e0009c30 con 0x7f97ec06a550 2024-04-21T08:39:57.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.716+0000 7f97f2d3d640 1 -- 172.21.15.86:0/1732887514 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97ec19da30 con 0x7f97ec06a550 2024-04-21T08:39:57.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.716+0000 7f97e9ffb640 1 -- 172.21.15.86:0/1732887514 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e000f620 con 0x7f97ec06a550 2024-04-21T08:39:57.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.716+0000 7f97f2d3d640 1 -- 172.21.15.86:0/1732887514 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97ec1a52c0 con 0x7f97ec06a550 2024-04-21T08:39:57.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.718+0000 7f97e9ffb640 1 -- 172.21.15.86:0/1732887514 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f97e0010460 con 0x7f97ec06a550 2024-04-21T08:39:57.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.718+0000 7f97f5a06740 1 Processor -- start 2024-04-21T08:39:57.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.718+0000 7f97f5a06740 1 -- start start 2024-04-21T08:39:57.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.718+0000 7f97e9ffb640 1 --2- 172.21.15.86:0/1732887514 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f97dc0a3ec0 0x7f97dc0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:57.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.718+0000 7f97f5a06740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6b66d58e0 0x55a6b66d5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:57.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.718+0000 7f97f5a06740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6b66d6290 0x55a6b66de840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:57.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.719+0000 7f97f5a06740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6b66dee10 0x55a6b66e1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:57.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.719+0000 7f97f5a06740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a6b6639f00 con 0x55a6b66d6290 2024-04-21T08:39:57.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.719+0000 7f97f12da640 1 --2- 172.21.15.86:0/1732887514 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f97dc0a3ec0 0x7f97dc0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:57.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.719+0000 7f97f5a06740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55a6b65d7590 con 0x55a6b66d58e0 2024-04-21T08:39:57.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.719+0000 7f97f5a06740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a6b6635ec0 con 0x55a6b66dee10 2024-04-21T08:39:57.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.719+0000 7f97e9ffb640 1 -- 172.21.15.86:0/1732887514 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f97e00d0930 con 0x7f97ec06a550 2024-04-21T08:39:57.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.719+0000 7f97ebfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6b66d6290 0x55a6b66de840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:57.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.719+0000 7f97ebfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6b66d6290 0x55a6b66de840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45772/0 (socket says 172.21.15.86:45772) 2024-04-21T08:39:57.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.719+0000 7f97ebfff640 1 -- 172.21.15.86:0/945705655 learned_addr learned my addr 172.21.15.86:0/945705655 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:57.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.719+0000 7f97f0ad9640 1 --2- 172.21.15.86:0/945705655 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6b66d58e0 0x55a6b66d5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:57.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.719+0000 7f97f12da640 1 --2- 172.21.15.86:0/945705655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6b66dee10 0x55a6b66e1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:57.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.719+0000 7f97ebfff640 1 -- 172.21.15.86:0/945705655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6b66dee10 msgr2=0x55a6b66e1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:57.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.719+0000 7f97ebfff640 1 --2- 172.21.15.86:0/945705655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6b66dee10 0x55a6b66e1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.719+0000 7f97ebfff640 1 -- 172.21.15.86:0/945705655 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6b66d58e0 msgr2=0x55a6b66d5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:57.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.719+0000 7f97ebfff640 1 --2- 172.21.15.86:0/945705655 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6b66d58e0 0x55a6b66d5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.719+0000 7f97ebfff640 1 -- 172.21.15.86:0/945705655 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6b64147e0 con 0x55a6b66d6290 2024-04-21T08:39:57.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.719+0000 7f97f12da640 1 --2- 172.21.15.86:0/945705655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6b66dee10 0x55a6b66e1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:57.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.719+0000 7f97f12da640 1 --2- 172.21.15.86:0/1732887514 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f97dc0a3ec0 0x7f97dc0a6380 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f97e4005fd0 tx=0x7f97e4010740 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:57.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.719+0000 7f97ebfff640 1 --2- 172.21.15.86:0/945705655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6b66d6290 0x55a6b66de840 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f97d8006010 tx=0x7f97d802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f80caa662ee192f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:57.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.719+0000 7f97d6ffd640 1 -- 172.21.15.86:0/945705655 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d8043020 con 0x55a6b66d6290 2024-04-21T08:39:57.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.719+0000 7f97d6ffd640 1 -- 172.21.15.86:0/945705655 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97d802c7e0 con 0x55a6b66d6290 2024-04-21T08:39:57.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.719+0000 7f97d6ffd640 1 -- 172.21.15.86:0/945705655 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97d80055c0 con 0x55a6b66d6290 2024-04-21T08:39:57.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.720+0000 7f97f5a06740 1 -- 172.21.15.86:0/945705655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6b66d6290 msgr2=0x55a6b66de840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:57.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.720+0000 7f97f5a06740 1 --2- 172.21.15.86:0/945705655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6b66d6290 0x55a6b66de840 secure :-1 s=READY pgs=741 cs=0 l=1 rev1=1 crypto rx=0x7f97d8006010 tx=0x7f97d802c650 comp rx=0 tx=0).stop 2024-04-21T08:39:57.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.720+0000 7f97f5a06740 1 -- 172.21.15.86:0/945705655 shutdown_connections 2024-04-21T08:39:57.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.720+0000 7f97f5a06740 1 --2- 172.21.15.86:0/945705655 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6b66dee10 0x55a6b66e1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.720+0000 7f97f5a06740 1 --2- 172.21.15.86:0/945705655 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6b66d6290 0x55a6b66de840 unknown :-1 s=CLOSED pgs=741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.720+0000 7f97f5a06740 1 --2- 172.21.15.86:0/945705655 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6b66d58e0 0x55a6b66d5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.720+0000 7f97f5a06740 1 -- 172.21.15.86:0/945705655 >> 172.21.15.86:0/945705655 conn(0x55a6b66cffb0 msgr2=0x55a6b66d0390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:57.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.720+0000 7f97f5a06740 1 -- 172.21.15.86:0/945705655 shutdown_connections 2024-04-21T08:39:57.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.720+0000 7f97f5a06740 1 -- 172.21.15.86:0/945705655 wait complete. 2024-04-21T08:39:57.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.720+0000 7f97f5a06740 1 -- start start 2024-04-21T08:39:57.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.720+0000 7f97f5a06740 1 Processor -- start 2024-04-21T08:39:57.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.721+0000 7f97f5a06740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6b66d58e0 0x55a6b66dbc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:57.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.721+0000 7f97f5a06740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6b66d6290 0x55a6b66dc1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:57.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.721+0000 7f97f5a06740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6b66dee10 0x55a6b66dc6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:57.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.721+0000 7f97f5a06740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a6b66e1d20 con 0x55a6b66d58e0 2024-04-21T08:39:57.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.721+0000 7f97f5a06740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55a6b65f3580 con 0x55a6b66dee10 2024-04-21T08:39:57.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.721+0000 7f97f5a06740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a6b66e1ea0 con 0x55a6b66d6290 2024-04-21T08:39:57.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.721+0000 7f97f5a06740 10 client.0 _ll_register_callbacks cb 0x55a6b65f9650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:57.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.721+0000 7f97f0ad9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6b66d58e0 0x55a6b66dbc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:57.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.721+0000 7f97f0ad9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6b66d58e0 0x55a6b66dbc70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45778/0 (socket says 172.21.15.86:45778) 2024-04-21T08:39:57.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.721+0000 7f97f0ad9640 1 -- 172.21.15.86:0/3466700048 learned_addr learned my addr 172.21.15.86:0/3466700048 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:57.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.721+0000 7f97ebfff640 1 --2- 172.21.15.86:0/3466700048 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6b66d6290 0x55a6b66dc1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:57.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.721+0000 7f97f12da640 1 --2- 172.21.15.86:0/3466700048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6b66dee10 0x55a6b66dc6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:57.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.721+0000 7f97f0ad9640 1 -- 172.21.15.86:0/3466700048 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6b66d6290 msgr2=0x55a6b66dc1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:57.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.721+0000 7f97f0ad9640 1 --2- 172.21.15.86:0/3466700048 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6b66d6290 0x55a6b66dc1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.721+0000 7f97f0ad9640 1 -- 172.21.15.86:0/3466700048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6b66dee10 msgr2=0x55a6b66dc6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:57.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.721+0000 7f97f0ad9640 1 --2- 172.21.15.86:0/3466700048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6b66dee10 0x55a6b66dc6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.721+0000 7f97f0ad9640 1 -- 172.21.15.86:0/3466700048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6b6474f70 con 0x55a6b66d58e0 2024-04-21T08:39:57.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.721+0000 7f97f0ad9640 1 --2- 172.21.15.86:0/3466700048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6b66d58e0 0x55a6b66dbc70 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f97e009c2a0 tx=0x7f97e004a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=834c0c38de3ec40d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:57.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.721+0000 7f97cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:57.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.721+0000 7f97cbfff640 1 -- 172.21.15.86:0/3466700048 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e0020030 con 0x55a6b66d58e0 2024-04-21T08:39:57.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.722+0000 7f97cbfff640 1 -- 172.21.15.86:0/3466700048 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f97e004c630 con 0x55a6b66d58e0 2024-04-21T08:39:57.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.722+0000 7f97f5a06740 1 -- 172.21.15.86:0/3466700048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97d802c7e0 con 0x55a6b66d58e0 2024-04-21T08:39:57.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.722+0000 7f97cbfff640 1 -- 172.21.15.86:0/3466700048 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f97e00a06c0 con 0x55a6b66d58e0 2024-04-21T08:39:57.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.722+0000 7f97f5a06740 10 client.16386 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:57.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.722+0000 7f97f5a06740 1 -- 172.21.15.86:0/3466700048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a6b630c170 con 0x55a6b66d58e0 2024-04-21T08:39:57.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.722+0000 7f97cbfff640 1 -- 172.21.15.86:0/3466700048 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97d802c7e0 con 0x55a6b66d58e0 2024-04-21T08:39:57.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.722+0000 7f97cbfff640 1 -- 172.21.15.86:0/3466700048 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f97e004a110 con 0x55a6b66d58e0 2024-04-21T08:39:57.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.722+0000 7f97f5a06740 10 client.16386 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:57.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.722+0000 7f97f5a06740 10 client.16386 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:57.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.722+0000 7f97f5a06740 20 client.16386 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:57.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.722+0000 7f97f5a06740 1 --2- 172.21.15.86:0/3466700048 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a6b66ef120 0x55a6b66f1530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:57.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.722+0000 7f97f5a06740 4 client.16386 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:57.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.722+0000 7f97f5a06740 1 -- 172.21.15.86:0/3466700048 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a6b6647920 con 0x55a6b66ef120 2024-04-21T08:39:57.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.722+0000 7f97ebfff640 1 --2- 172.21.15.86:0/3466700048 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a6b66ef120 0x55a6b66f1530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:57.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.723+0000 7f97ebfff640 1 --2- 172.21.15.86:0/3466700048 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a6b66ef120 0x55a6b66f1530 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=907f89510824842d server_cookie=a1398afd0cc456b3 in_seq=0 out_seq=0 2024-04-21T08:39:57.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.723+0000 7f97cbfff640 10 client.16386 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:57.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.723+0000 7f97cbfff640 1 -- 172.21.15.86:0/3466700048 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a6b6647920 con 0x55a6b66ef120 2024-04-21T08:39:57.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.723+0000 7f97cbfff640 10 client.16386 handle_command_reply: tid=0 2024-04-21T08:39:57.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.723+0000 7f97f5a06740 1 client.16386 shutdown 2024-04-21T08:39:57.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.724+0000 7f97f5a06740 1 -- 172.21.15.86:0/3466700048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6b66d58e0 msgr2=0x55a6b66dbc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:57.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.724+0000 7f97f5a06740 1 --2- 172.21.15.86:0/3466700048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6b66d58e0 0x55a6b66dbc70 secure :-1 s=READY pgs=742 cs=0 l=1 rev1=1 crypto rx=0x7f97e009c2a0 tx=0x7f97e004a690 comp rx=0 tx=0).stop 2024-04-21T08:39:57.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.724+0000 7f97f5a06740 1 -- 172.21.15.86:0/3466700048 shutdown_connections 2024-04-21T08:39:57.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.724+0000 7f97f5a06740 1 --2- 172.21.15.86:0/3466700048 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a6b66ef120 0x55a6b66f1530 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.724+0000 7f97f5a06740 1 --2- 172.21.15.86:0/3466700048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6b66dee10 0x55a6b66dc6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.724+0000 7f97cbfff640 0 client.16386 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:57.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.724+0000 7f97f5a06740 1 --2- 172.21.15.86:0/3466700048 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6b66d6290 0x55a6b66dc1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.724+0000 7f97f5a06740 1 --2- 172.21.15.86:0/3466700048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6b66d58e0 0x55a6b66dbc70 unknown :-1 s=CLOSED pgs=742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.724+0000 7f97f5a06740 1 -- 172.21.15.86:0/3466700048 >> 172.21.15.86:0/3466700048 conn(0x55a6b66cffb0 msgr2=0x55a6b66d1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:57.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.724+0000 7f97f5a06740 1 -- 172.21.15.86:0/3466700048 shutdown_connections 2024-04-21T08:39:57.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.724+0000 7f97f5a06740 1 -- 172.21.15.86:0/3466700048 wait complete. 2024-04-21T08:39:57.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.724+0000 7f97f5a06740 20 client.16386 trim_cache size 0 max 16384 2024-04-21T08:39:57.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.741+0000 7f97f5a06740 1 Processor -- start 2024-04-21T08:39:57.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.741+0000 7f97f5a06740 1 -- start start 2024-04-21T08:39:57.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.741+0000 7f97f5a06740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6b66d58e0 0x55a6b66de3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:57.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.741+0000 7f97f5a06740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6b66d6290 0x55a6b66eac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:57.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.741+0000 7f97f5a06740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6b66dee10 0x55a6b66d1aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:57.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.741+0000 7f97f5a06740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec1af2c0 con 0x55a6b66dee10 2024-04-21T08:39:57.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.741+0000 7f97f5a06740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec1b0e00 con 0x55a6b66d58e0 2024-04-21T08:39:57.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.741+0000 7f97f5a06740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec1af990 con 0x55a6b66d6290 2024-04-21T08:39:57.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.741+0000 7f97f12da640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6b66dee10 0x55a6b66d1aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:57.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.742+0000 7f97f12da640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6b66dee10 0x55a6b66d1aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45792/0 (socket says 172.21.15.86:45792) 2024-04-21T08:39:57.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.742+0000 7f97f12da640 1 -- 172.21.15.86:0/882441008 learned_addr learned my addr 172.21.15.86:0/882441008 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:57.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.742+0000 7f97ebfff640 1 --2- 172.21.15.86:0/882441008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6b66d6290 0x55a6b66eac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:57.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.742+0000 7f97f0ad9640 1 --2- 172.21.15.86:0/882441008 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6b66d58e0 0x55a6b66de3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:57.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.742+0000 7f97f12da640 1 -- 172.21.15.86:0/882441008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6b66d6290 msgr2=0x55a6b66eac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:57.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.742+0000 7f97f12da640 1 --2- 172.21.15.86:0/882441008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6b66d6290 0x55a6b66eac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.742+0000 7f97f12da640 1 -- 172.21.15.86:0/882441008 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6b66d58e0 msgr2=0x55a6b66de3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:57.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.742+0000 7f97f12da640 1 --2- 172.21.15.86:0/882441008 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6b66d58e0 0x55a6b66de3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.742+0000 7f97f12da640 1 -- 172.21.15.86:0/882441008 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a6b66cfde0 con 0x55a6b66dee10 2024-04-21T08:39:57.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.742+0000 7f97f12da640 1 --2- 172.21.15.86:0/882441008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6b66dee10 0x55a6b66d1aa0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f97e4012980 tx=0x7f97e4015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebd0f70898d485fb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:57.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.742+0000 7f97d67fc640 1 -- 172.21.15.86:0/882441008 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e4011b60 con 0x55a6b66dee10 2024-04-21T08:39:57.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.742+0000 7f97d67fc640 1 -- 172.21.15.86:0/882441008 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97e4011d00 con 0x55a6b66dee10 2024-04-21T08:39:57.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.742+0000 7f97d67fc640 1 -- 172.21.15.86:0/882441008 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e40205f0 con 0x55a6b66dee10 2024-04-21T08:39:57.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.743+0000 7f97f5a06740 1 -- 172.21.15.86:0/882441008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6b66dee10 msgr2=0x55a6b66d1aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:57.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.743+0000 7f97f5a06740 1 --2- 172.21.15.86:0/882441008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6b66dee10 0x55a6b66d1aa0 secure :-1 s=READY pgs=743 cs=0 l=1 rev1=1 crypto rx=0x7f97e4012980 tx=0x7f97e4015000 comp rx=0 tx=0).stop 2024-04-21T08:39:57.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.743+0000 7f97f5a06740 1 -- 172.21.15.86:0/882441008 shutdown_connections 2024-04-21T08:39:57.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.743+0000 7f97f5a06740 1 --2- 172.21.15.86:0/882441008 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6b66dee10 0x55a6b66d1aa0 unknown :-1 s=CLOSED pgs=743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.743+0000 7f97f5a06740 1 --2- 172.21.15.86:0/882441008 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6b66d6290 0x55a6b66eac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.743+0000 7f97f5a06740 1 --2- 172.21.15.86:0/882441008 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6b66d58e0 0x55a6b66de3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.743+0000 7f97f5a06740 1 -- 172.21.15.86:0/882441008 >> 172.21.15.86:0/882441008 conn(0x55a6b66cffb0 msgr2=0x55a6b66d1390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:57.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.743+0000 7f97f5a06740 1 -- 172.21.15.86:0/882441008 shutdown_connections 2024-04-21T08:39:57.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.743+0000 7f97f5a06740 1 -- 172.21.15.86:0/882441008 wait complete. 2024-04-21T08:39:57.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.743+0000 7f97f5a06740 1 -- start start 2024-04-21T08:39:57.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.743+0000 7f97f5a06740 1 Processor -- start 2024-04-21T08:39:57.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.744+0000 7f97f5a06740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6b66d58e0 0x55a6b66e2c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:57.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.744+0000 7f97f5a06740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6b66d6290 0x55a6b66e3160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:57.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.744+0000 7f97f5a06740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6b66dee10 0x55a6b66e36a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:57.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.744+0000 7f97f5a06740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec1b0730 con 0x55a6b66d58e0 2024-04-21T08:39:57.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.744+0000 7f97f5a06740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec1b0060 con 0x55a6b66dee10 2024-04-21T08:39:57.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.744+0000 7f97f5a06740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f97ec1ad280 con 0x55a6b66d6290 2024-04-21T08:39:57.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.744+0000 7f97f5a06740 10 client.0 _ll_register_callbacks cb 0x7f97e004a7b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:57.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.744+0000 7f97f0ad9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6b66d58e0 0x55a6b66e2c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:57.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.744+0000 7f97f0ad9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6b66d58e0 0x55a6b66e2c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45802/0 (socket says 172.21.15.86:45802) 2024-04-21T08:39:57.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.744+0000 7f97f0ad9640 1 -- 172.21.15.86:0/2888538824 learned_addr learned my addr 172.21.15.86:0/2888538824 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:57.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.744+0000 7f97ebfff640 1 --2- 172.21.15.86:0/2888538824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6b66d6290 0x55a6b66e3160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:57.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.744+0000 7f97f12da640 1 --2- 172.21.15.86:0/2888538824 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6b66dee10 0x55a6b66e36a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:57.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.744+0000 7f97f0ad9640 1 -- 172.21.15.86:0/2888538824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6b66d6290 msgr2=0x55a6b66e3160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:57.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.744+0000 7f97f0ad9640 1 --2- 172.21.15.86:0/2888538824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6b66d6290 0x55a6b66e3160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.744+0000 7f97f0ad9640 1 -- 172.21.15.86:0/2888538824 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6b66dee10 msgr2=0x55a6b66e36a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:57.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.744+0000 7f97f0ad9640 1 --2- 172.21.15.86:0/2888538824 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6b66dee10 0x55a6b66e36a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.744+0000 7f97f0ad9640 1 -- 172.21.15.86:0/2888538824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a6b67546f0 con 0x55a6b66d58e0 2024-04-21T08:39:57.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.744+0000 7f97f12da640 1 --2- 172.21.15.86:0/2888538824 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6b66dee10 0x55a6b66e36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:57.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.744+0000 7f97f0ad9640 1 --2- 172.21.15.86:0/2888538824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6b66d58e0 0x55a6b66e2c20 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f97e009c010 tx=0x7f97e004c430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bfc518614a87746 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:57.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.744+0000 7f97cbfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:57.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.744+0000 7f97cbfff640 1 -- 172.21.15.86:0/2888538824 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97e0014030 con 0x55a6b66d58e0 2024-04-21T08:39:57.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.744+0000 7f97cbfff640 1 -- 172.21.15.86:0/2888538824 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f97e009b760 con 0x55a6b66d58e0 2024-04-21T08:39:57.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.744+0000 7f97f5a06740 1 -- 172.21.15.86:0/2888538824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f97e4011d00 con 0x55a6b66d58e0 2024-04-21T08:39:57.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.744+0000 7f97cbfff640 1 -- 172.21.15.86:0/2888538824 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f97e0020040 con 0x55a6b66d58e0 2024-04-21T08:39:57.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.744+0000 7f97f5a06740 10 client.16392 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:57.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.744+0000 7f97f5a06740 1 -- 172.21.15.86:0/2888538824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a6b6755a10 con 0x55a6b66d58e0 2024-04-21T08:39:57.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.745+0000 7f97cbfff640 1 -- 172.21.15.86:0/2888538824 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f97e4011d00 con 0x55a6b66d58e0 2024-04-21T08:39:57.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.745+0000 7f97cbfff640 1 -- 172.21.15.86:0/2888538824 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f97e00510f0 con 0x55a6b66d58e0 2024-04-21T08:39:57.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.745+0000 7f97f5a06740 10 client.16392 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:57.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.745+0000 7f97f5a06740 10 client.16392 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:57.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.745+0000 7f97f5a06740 20 client.16392 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:57.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.745+0000 7f97f5a06740 1 --2- 172.21.15.86:0/2888538824 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a6b6756190 0x55a6b67585b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:57.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.745+0000 7f97f5a06740 4 client.16392 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:39:57.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.745+0000 7f97f5a06740 1 -- 172.21.15.86:0/2888538824 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a6b6641790 con 0x55a6b6756190 2024-04-21T08:39:57.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.745+0000 7f97ebfff640 1 --2- 172.21.15.86:0/2888538824 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a6b6756190 0x55a6b67585b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:57.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.745+0000 7f97cbfff640 10 client.16392 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:57.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.745+0000 7f97ebfff640 1 --2- 172.21.15.86:0/2888538824 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a6b6756190 0x55a6b67585b0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5433e20163a775e7 server_cookie=cab5020fb863e765 in_seq=0 out_seq=0 2024-04-21T08:39:57.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.746+0000 7f97cbfff640 1 -- 172.21.15.86:0/2888538824 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a6b6641790 con 0x55a6b6756190 2024-04-21T08:39:57.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.746+0000 7f97cbfff640 10 client.16392 handle_command_reply: tid=0 2024-04-21T08:39:57.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.746+0000 7f97f5a06740 1 client.16392 shutdown 2024-04-21T08:39:57.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.746+0000 7f97f5a06740 1 -- 172.21.15.86:0/2888538824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6b66d58e0 msgr2=0x55a6b66e2c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:57.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.746+0000 7f97f5a06740 1 --2- 172.21.15.86:0/2888538824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6b66d58e0 0x55a6b66e2c20 secure :-1 s=READY pgs=744 cs=0 l=1 rev1=1 crypto rx=0x7f97e009c010 tx=0x7f97e004c430 comp rx=0 tx=0).stop 2024-04-21T08:39:57.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.747+0000 7f97f5a06740 1 -- 172.21.15.86:0/2888538824 shutdown_connections 2024-04-21T08:39:57.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.747+0000 7f97f5a06740 1 --2- 172.21.15.86:0/2888538824 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a6b6756190 0x55a6b67585b0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.747+0000 7f97f5a06740 1 --2- 172.21.15.86:0/2888538824 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a6b66dee10 0x55a6b66e36a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.747+0000 7f97cbfff640 0 client.16392 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:57.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.747+0000 7f97f5a06740 1 --2- 172.21.15.86:0/2888538824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a6b66d6290 0x55a6b66e3160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.747+0000 7f97f5a06740 1 --2- 172.21.15.86:0/2888538824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a6b66d58e0 0x55a6b66e2c20 unknown :-1 s=CLOSED pgs=744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.747+0000 7f97f5a06740 1 -- 172.21.15.86:0/2888538824 >> 172.21.15.86:0/2888538824 conn(0x55a6b66cffb0 msgr2=0x55a6b66d1c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:57.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.747+0000 7f97f5a06740 1 -- 172.21.15.86:0/2888538824 shutdown_connections 2024-04-21T08:39:57.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.747+0000 7f97f5a06740 1 -- 172.21.15.86:0/2888538824 wait complete. 2024-04-21T08:39:57.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.747+0000 7f97f5a06740 20 client.16392 trim_cache size 0 max 16384 2024-04-21T08:39:57.752 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:39:57.752 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-21T08:39:57.752 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "be9262df-7152-4ace-bfdc-404253b8f9eb", 2024-04-21T08:39:57.752 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-21T08:39:57.752 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:39:57.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.747+0000 7f97f2d3d640 1 -- 172.21.15.86:0/1732887514 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f97dc0a3ec0 msgr2=0x7f97dc0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:57.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.747+0000 7f97f2d3d640 1 --2- 172.21.15.86:0/1732887514 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f97dc0a3ec0 0x7f97dc0a6380 secure :-1 s=READY pgs=103 cs=0 l=1 rev1=1 crypto rx=0x7f97e4005fd0 tx=0x7f97e4010740 comp rx=0 tx=0).stop 2024-04-21T08:39:57.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.747+0000 7f97f2d3d640 1 -- 172.21.15.86:0/1732887514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97ec06a550 msgr2=0x7f97ec198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:57.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.747+0000 7f97f2d3d640 1 --2- 172.21.15.86:0/1732887514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97ec06a550 0x7f97ec198750 secure :-1 s=READY pgs=740 cs=0 l=1 rev1=1 crypto rx=0x7f97e000d950 tx=0x7f97e000de10 comp rx=0 tx=0).stop 2024-04-21T08:39:57.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.747+0000 7f97f2d3d640 1 -- 172.21.15.86:0/1732887514 shutdown_connections 2024-04-21T08:39:57.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.747+0000 7f97f2d3d640 1 --2- 172.21.15.86:0/1732887514 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f97dc0a3ec0 0x7f97dc0a6380 unknown :-1 s=CLOSED pgs=103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.747+0000 7f97f2d3d640 1 --2- 172.21.15.86:0/1732887514 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f97ec107fc0 0x7f97ec19d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.747+0000 7f97f2d3d640 1 --2- 172.21.15.86:0/1732887514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f97ec06aed0 0x7f97ec198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.747+0000 7f97f2d3d640 1 --2- 172.21.15.86:0/1732887514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f97ec06a550 0x7f97ec198750 unknown :-1 s=CLOSED pgs=740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.747+0000 7f97f2d3d640 1 -- 172.21.15.86:0/1732887514 >> 172.21.15.86:0/1732887514 conn(0x7f97ec076c50 msgr2=0x7f97ec100020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:57.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.748+0000 7f97f2d3d640 1 -- 172.21.15.86:0/1732887514 shutdown_connections 2024-04-21T08:39:57.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.748+0000 7f97f2d3d640 1 -- 172.21.15.86:0/1732887514 wait complete. 2024-04-21T08:39:57.757 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:39:57.757 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:39:57.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.858+0000 7f66d26d2640 1 -- 172.21.15.86:0/3165320751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66cc101090 msgr2=0x7f66cc101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:57.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.858+0000 7f66d26d2640 1 --2- 172.21.15.86:0/3165320751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66cc101090 0x7f66cc101470 secure :-1 s=READY pgs=745 cs=0 l=1 rev1=1 crypto rx=0x7f66b4009900 tx=0x7f66b402bc30 comp rx=0 tx=0).stop 2024-04-21T08:39:57.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.858+0000 7f66d26d2640 1 -- 172.21.15.86:0/3165320751 shutdown_connections 2024-04-21T08:39:57.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.858+0000 7f66d26d2640 1 --2- 172.21.15.86:0/3165320751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66cc106160 0x7f66cc10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.858+0000 7f66d26d2640 1 --2- 172.21.15.86:0/3165320751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66cc101a40 0x7f66cc105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.858+0000 7f66d26d2640 1 --2- 172.21.15.86:0/3165320751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66cc101090 0x7f66cc101470 unknown :-1 s=CLOSED pgs=745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.858+0000 7f66d26d2640 1 -- 172.21.15.86:0/3165320751 >> 172.21.15.86:0/3165320751 conn(0x7f66cc0771d0 msgr2=0x7f66cc0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:57.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.858+0000 7f66d26d2640 1 -- 172.21.15.86:0/3165320751 shutdown_connections 2024-04-21T08:39:57.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.858+0000 7f66d26d2640 1 -- 172.21.15.86:0/3165320751 wait complete. 2024-04-21T08:39:57.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.858+0000 7f66d26d2640 1 Processor -- start 2024-04-21T08:39:57.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.858+0000 7f66d26d2640 1 -- start start 2024-04-21T08:39:57.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.859+0000 7f66d26d2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66cc101090 0x7f66cc1975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:57.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.859+0000 7f66d26d2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66cc106160 0x7f66cc197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:57.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.859+0000 7f66d26d2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66cc19bea0 0x7f66cc19c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:57.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.859+0000 7f66d26d2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f66cc1110c0 con 0x7f66cc101090 2024-04-21T08:39:57.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.859+0000 7f66d26d2640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f66cc110f40 con 0x7f66cc106160 2024-04-21T08:39:57.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.859+0000 7f66d26d2640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f66cc111240 con 0x7f66cc19bea0 2024-04-21T08:39:57.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.859+0000 7f66cbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66cc101090 0x7f66cc1975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:57.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.859+0000 7f66cbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66cc101090 0x7f66cc1975e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45810/0 (socket says 172.21.15.86:45810) 2024-04-21T08:39:57.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.859+0000 7f66cbfff640 1 -- 172.21.15.86:0/3384218865 learned_addr learned my addr 172.21.15.86:0/3384218865 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:57.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.859+0000 7f66d0c6f640 1 --2- 172.21.15.86:0/3384218865 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66cc19bea0 0x7f66cc19c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:57.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.859+0000 7f66cbfff640 1 -- 172.21.15.86:0/3384218865 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66cc19bea0 msgr2=0x7f66cc19c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:57.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.859+0000 7f66cbfff640 1 --2- 172.21.15.86:0/3384218865 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66cc19bea0 0x7f66cc19c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.859+0000 7f66cbfff640 1 -- 172.21.15.86:0/3384218865 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66cc106160 msgr2=0x7f66cc197b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:57.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.859+0000 7f66cbfff640 1 --2- 172.21.15.86:0/3384218865 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66cc106160 0x7f66cc197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.859+0000 7f66cbfff640 1 -- 172.21.15.86:0/3384218865 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66cc19c990 con 0x7f66cc101090 2024-04-21T08:39:57.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.859+0000 7f66d0c6f640 1 --2- 172.21.15.86:0/3384218865 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66cc19bea0 0x7f66cc19c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:57.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.859+0000 7f66cbfff640 1 --2- 172.21.15.86:0/3384218865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66cc101090 0x7f66cc1975e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f66b4005940 tx=0x7f66b402c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:57.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.859+0000 7f66c97fa640 1 -- 172.21.15.86:0/3384218865 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b4043020 con 0x7f66cc101090 2024-04-21T08:39:57.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.859+0000 7f66c97fa640 1 -- 172.21.15.86:0/3384218865 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66b402ca70 con 0x7f66cc101090 2024-04-21T08:39:57.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.859+0000 7f66d26d2640 1 -- 172.21.15.86:0/3384218865 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66cc19cc20 con 0x7f66cc101090 2024-04-21T08:39:57.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.860+0000 7f66c97fa640 1 -- 172.21.15.86:0/3384218865 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b40039f0 con 0x7f66cc101090 2024-04-21T08:39:57.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.860+0000 7f66d26d2640 1 -- 172.21.15.86:0/3384218865 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66cc1a4440 con 0x7f66cc101090 2024-04-21T08:39:57.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.862+0000 7f66c97fa640 1 -- 172.21.15.86:0/3384218865 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f66b4046020 con 0x7f66cc101090 2024-04-21T08:39:57.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.862+0000 7f66d53a3740 1 Processor -- start 2024-04-21T08:39:57.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.862+0000 7f66d53a3740 1 -- start start 2024-04-21T08:39:57.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.862+0000 7f66c97fa640 1 --2- 172.21.15.86:0/3384218865 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f66b80a3e40 0x7f66b80a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:57.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.862+0000 7f66d53a3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ee3da270 0x5610ee3da650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:57.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.862+0000 7f66d53a3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610ee3dab90 0x5610ee3e5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:57.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.862+0000 7f66d53a3740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5610ee3e5c90 0x5610ee3e8080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:57.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.862+0000 7f66d53a3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5610ee33f1e0 con 0x5610ee3da270 2024-04-21T08:39:57.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.862+0000 7f66d53a3740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5610ee2dc390 con 0x5610ee3e5c90 2024-04-21T08:39:57.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.862+0000 7f66d53a3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5610ee33b1a0 con 0x5610ee3dab90 2024-04-21T08:39:57.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.862+0000 7f66cbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ee3da270 0x5610ee3da650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:57.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.862+0000 7f66d0c6f640 1 --2- 172.21.15.86:0/3384218865 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f66b80a3e40 0x7f66b80a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:57.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.862+0000 7f66cbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ee3da270 0x5610ee3da650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45822/0 (socket says 172.21.15.86:45822) 2024-04-21T08:39:57.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.862+0000 7f66cbfff640 1 -- 172.21.15.86:0/3771491906 learned_addr learned my addr 172.21.15.86:0/3771491906 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:57.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.862+0000 7f66c97fa640 1 -- 172.21.15.86:0/3384218865 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f66b40f2ed0 con 0x7f66cc101090 2024-04-21T08:39:57.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.862+0000 7f66d0c6f640 1 --2- 172.21.15.86:0/3771491906 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5610ee3e5c90 0x5610ee3e8080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:57.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.863+0000 7f66cbfff640 1 -- 172.21.15.86:0/3771491906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610ee3dab90 msgr2=0x5610ee3e5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:57.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.863+0000 7f66cbfff640 1 --2- 172.21.15.86:0/3771491906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610ee3dab90 0x5610ee3e5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.863+0000 7f66cbfff640 1 -- 172.21.15.86:0/3771491906 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5610ee3e5c90 msgr2=0x5610ee3e8080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:57.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.863+0000 7f66cbfff640 1 --2- 172.21.15.86:0/3771491906 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5610ee3e5c90 0x5610ee3e8080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.863+0000 7f66cbfff640 1 -- 172.21.15.86:0/3771491906 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610ee11b7d0 con 0x5610ee3da270 2024-04-21T08:39:57.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.863+0000 7f66d0c6f640 1 --2- 172.21.15.86:0/3384218865 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f66b80a3e40 0x7f66b80a6300 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f66c40094d0 tx=0x7f66c4013b20 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:57.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.863+0000 7f66cbfff640 1 --2- 172.21.15.86:0/3771491906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ee3da270 0x5610ee3da650 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f66b40be950 tx=0x7f66b40ba660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79dce0d840e98d54 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:57.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.863+0000 7f66c27fc640 1 -- 172.21.15.86:0/3771491906 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b40d1e40 con 0x5610ee3da270 2024-04-21T08:39:57.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.863+0000 7f66c27fc640 1 -- 172.21.15.86:0/3771491906 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66b40bf0c0 con 0x5610ee3da270 2024-04-21T08:39:57.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.863+0000 7f66c27fc640 1 -- 172.21.15.86:0/3771491906 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b40d2020 con 0x5610ee3da270 2024-04-21T08:39:57.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.864+0000 7f66d53a3740 1 -- 172.21.15.86:0/3771491906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ee3da270 msgr2=0x5610ee3da650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:57.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.864+0000 7f66d53a3740 1 --2- 172.21.15.86:0/3771491906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ee3da270 0x5610ee3da650 secure :-1 s=READY pgs=747 cs=0 l=1 rev1=1 crypto rx=0x7f66b40be950 tx=0x7f66b40ba660 comp rx=0 tx=0).stop 2024-04-21T08:39:57.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.864+0000 7f66d53a3740 1 -- 172.21.15.86:0/3771491906 shutdown_connections 2024-04-21T08:39:57.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.864+0000 7f66d53a3740 1 --2- 172.21.15.86:0/3771491906 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5610ee3e5c90 0x5610ee3e8080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.864+0000 7f66d53a3740 1 --2- 172.21.15.86:0/3771491906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610ee3dab90 0x5610ee3e5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.864+0000 7f66d53a3740 1 --2- 172.21.15.86:0/3771491906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ee3da270 0x5610ee3da650 unknown :-1 s=CLOSED pgs=747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.864+0000 7f66d53a3740 1 -- 172.21.15.86:0/3771491906 >> 172.21.15.86:0/3771491906 conn(0x5610ee3d4cb0 msgr2=0x5610ee3d5090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:57.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.864+0000 7f66d53a3740 1 -- 172.21.15.86:0/3771491906 shutdown_connections 2024-04-21T08:39:57.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.864+0000 7f66d53a3740 1 -- 172.21.15.86:0/3771491906 wait complete. 2024-04-21T08:39:57.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.864+0000 7f66d53a3740 1 -- start start 2024-04-21T08:39:57.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.864+0000 7f66d53a3740 1 Processor -- start 2024-04-21T08:39:57.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.864+0000 7f66d53a3740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5610ee3da270 0x5610ee3e2cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:57.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.864+0000 7f66d53a3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ee3dab90 0x5610ee3e3200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:57.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.864+0000 7f66d53a3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610ee3e5c90 0x5610ee3e3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:57.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.864+0000 7f66d53a3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5610ee3e8840 con 0x5610ee3dab90 2024-04-21T08:39:57.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.864+0000 7f66d53a3740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5610ee2f82f0 con 0x5610ee3da270 2024-04-21T08:39:57.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.864+0000 7f66d53a3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5610ee3e89c0 con 0x5610ee3e5c90 2024-04-21T08:39:57.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.865+0000 7f66d53a3740 10 client.0 _ll_register_callbacks cb 0x5610ee305580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:57.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.865+0000 7f66cb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ee3dab90 0x5610ee3e3200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:57.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.865+0000 7f66cb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ee3dab90 0x5610ee3e3200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45840/0 (socket says 172.21.15.86:45840) 2024-04-21T08:39:57.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.865+0000 7f66cb7fe640 1 -- 172.21.15.86:0/3220909180 learned_addr learned my addr 172.21.15.86:0/3220909180 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:57.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.865+0000 7f66cbfff640 1 --2- 172.21.15.86:0/3220909180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5610ee3da270 0x5610ee3e2cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:57.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.865+0000 7f66cb7fe640 1 -- 172.21.15.86:0/3220909180 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610ee3e5c90 msgr2=0x5610ee3e3740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:57.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.865+0000 7f66d0c6f640 1 --2- 172.21.15.86:0/3220909180 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610ee3e5c90 0x5610ee3e3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:57.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.865+0000 7f66cb7fe640 1 --2- 172.21.15.86:0/3220909180 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610ee3e5c90 0x5610ee3e3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.865+0000 7f66cb7fe640 1 -- 172.21.15.86:0/3220909180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5610ee3da270 msgr2=0x5610ee3e2cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:57.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.865+0000 7f66cb7fe640 1 --2- 172.21.15.86:0/3220909180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5610ee3da270 0x5610ee3e2cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.865+0000 7f66cb7fe640 1 -- 172.21.15.86:0/3220909180 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610ee0513f0 con 0x5610ee3dab90 2024-04-21T08:39:57.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.865+0000 7f66cb7fe640 1 --2- 172.21.15.86:0/3220909180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ee3dab90 0x5610ee3e3200 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00e600 tx=0x7f66bc00eac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=403b04db448cc0c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:57.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.865+0000 7f66ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:57.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.865+0000 7f66ab7fe640 1 -- 172.21.15.86:0/3220909180 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66bc005220 con 0x5610ee3dab90 2024-04-21T08:39:57.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.865+0000 7f66ab7fe640 1 -- 172.21.15.86:0/3220909180 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f66bc0053c0 con 0x5610ee3dab90 2024-04-21T08:39:57.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.865+0000 7f66d53a3740 1 -- 172.21.15.86:0/3220909180 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66b40bf0c0 con 0x5610ee3dab90 2024-04-21T08:39:57.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.865+0000 7f66ab7fe640 1 -- 172.21.15.86:0/3220909180 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f66bc0055c0 con 0x5610ee3dab90 2024-04-21T08:39:57.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.865+0000 7f66d53a3740 10 client.16404 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:57.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.865+0000 7f66d53a3740 1 -- 172.21.15.86:0/3220909180 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5610ee0110f0 con 0x5610ee3dab90 2024-04-21T08:39:57.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.866+0000 7f66ab7fe640 1 -- 172.21.15.86:0/3220909180 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66b40bf0c0 con 0x5610ee3dab90 2024-04-21T08:39:57.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.866+0000 7f66ab7fe640 1 -- 172.21.15.86:0/3220909180 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f66bc010bf0 con 0x5610ee3dab90 2024-04-21T08:39:57.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.866+0000 7f66d53a3740 10 client.16404 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:57.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.866+0000 7f66d53a3740 10 client.16404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:57.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.866+0000 7f66d53a3740 20 client.16404 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:57.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.866+0000 7f66d53a3740 1 --2- 172.21.15.86:0/3220909180 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5610ee3f5a70 0x5610ee3f7ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:57.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.866+0000 7f66d53a3740 4 client.16404 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:57.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.866+0000 7f66d53a3740 1 -- 172.21.15.86:0/3220909180 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5610ee34cc00 con 0x5610ee3f5a70 2024-04-21T08:39:57.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.866+0000 7f66cbfff640 1 --2- 172.21.15.86:0/3220909180 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5610ee3f5a70 0x5610ee3f7ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:57.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.866+0000 7f66ab7fe640 10 client.16404 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:57.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.866+0000 7f66cbfff640 1 --2- 172.21.15.86:0/3220909180 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5610ee3f5a70 0x5610ee3f7ee0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fdbe68e6c8bc5e30 server_cookie=2ee98abd9181c6b2 in_seq=0 out_seq=0 2024-04-21T08:39:57.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.867+0000 7f66ab7fe640 1 -- 172.21.15.86:0/3220909180 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5610ee34cc00 con 0x5610ee3f5a70 2024-04-21T08:39:57.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.867+0000 7f66ab7fe640 10 client.16404 handle_command_reply: tid=0 2024-04-21T08:39:57.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.867+0000 7f66d53a3740 1 client.16404 shutdown 2024-04-21T08:39:57.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.867+0000 7f66d53a3740 1 -- 172.21.15.86:0/3220909180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ee3dab90 msgr2=0x5610ee3e3200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:57.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.867+0000 7f66d53a3740 1 --2- 172.21.15.86:0/3220909180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ee3dab90 0x5610ee3e3200 secure :-1 s=READY pgs=748 cs=0 l=1 rev1=1 crypto rx=0x7f66bc00e600 tx=0x7f66bc00eac0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.867+0000 7f66d53a3740 1 -- 172.21.15.86:0/3220909180 shutdown_connections 2024-04-21T08:39:57.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.867+0000 7f66d53a3740 1 --2- 172.21.15.86:0/3220909180 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5610ee3f5a70 0x5610ee3f7ee0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.867+0000 7f66d53a3740 1 --2- 172.21.15.86:0/3220909180 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610ee3e5c90 0x5610ee3e3740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.867+0000 7f66ab7fe640 0 client.16404 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:57.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.867+0000 7f66d53a3740 1 --2- 172.21.15.86:0/3220909180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ee3dab90 0x5610ee3e3200 unknown :-1 s=CLOSED pgs=748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.867+0000 7f66d53a3740 1 --2- 172.21.15.86:0/3220909180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5610ee3da270 0x5610ee3e2cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.867+0000 7f66d53a3740 1 -- 172.21.15.86:0/3220909180 >> 172.21.15.86:0/3220909180 conn(0x5610ee3d4cb0 msgr2=0x5610ee3d6a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:57.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.867+0000 7f66d53a3740 1 -- 172.21.15.86:0/3220909180 shutdown_connections 2024-04-21T08:39:57.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.867+0000 7f66d53a3740 1 -- 172.21.15.86:0/3220909180 wait complete. 2024-04-21T08:39:57.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.867+0000 7f66d53a3740 20 client.16404 trim_cache size 0 max 16384 2024-04-21T08:39:57.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.884+0000 7f66d53a3740 1 Processor -- start 2024-04-21T08:39:57.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.884+0000 7f66d53a3740 1 -- start start 2024-04-21T08:39:57.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.885+0000 7f66d53a3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ee3da270 0x5610ee3e53f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:57.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.885+0000 7f66d53a3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610ee3dab90 0x5610ee3e1c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:57.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.885+0000 7f66d53a3740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5610ee3e5c90 0x5610ee3f1700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:57.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.885+0000 7f66d53a3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f66cc1ad020 con 0x5610ee3da270 2024-04-21T08:39:57.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.885+0000 7f66d53a3740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f66cc1aeb60 con 0x5610ee3e5c90 2024-04-21T08:39:57.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.885+0000 7f66d53a3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f66cc1ad6f0 con 0x5610ee3dab90 2024-04-21T08:39:57.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.885+0000 7f66cbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ee3da270 0x5610ee3e53f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:57.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.885+0000 7f66cbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ee3da270 0x5610ee3e53f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45850/0 (socket says 172.21.15.86:45850) 2024-04-21T08:39:57.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.885+0000 7f66cbfff640 1 -- 172.21.15.86:0/398988663 learned_addr learned my addr 172.21.15.86:0/398988663 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:57.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.885+0000 7f66cb7fe640 1 --2- 172.21.15.86:0/398988663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610ee3dab90 0x5610ee3e1c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:57.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.885+0000 7f66d0c6f640 1 --2- 172.21.15.86:0/398988663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5610ee3e5c90 0x5610ee3f1700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:57.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.885+0000 7f66cbfff640 1 -- 172.21.15.86:0/398988663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610ee3dab90 msgr2=0x5610ee3e1c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:57.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.885+0000 7f66cbfff640 1 --2- 172.21.15.86:0/398988663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610ee3dab90 0x5610ee3e1c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.885+0000 7f66cbfff640 1 -- 172.21.15.86:0/398988663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5610ee3e5c90 msgr2=0x5610ee3f1700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:57.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.885+0000 7f66cbfff640 1 --2- 172.21.15.86:0/398988663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5610ee3e5c90 0x5610ee3f1700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.885+0000 7f66cbfff640 1 -- 172.21.15.86:0/398988663 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5610ee3e83b0 con 0x5610ee3da270 2024-04-21T08:39:57.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.885+0000 7f66d0c6f640 1 --2- 172.21.15.86:0/398988663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5610ee3e5c90 0x5610ee3f1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:57.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.885+0000 7f66cbfff640 1 --2- 172.21.15.86:0/398988663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ee3da270 0x5610ee3e53f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f66b4005ec0 tx=0x7f66b4005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad359b547ffde2c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:57.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.885+0000 7f66c1ffb640 1 -- 172.21.15.86:0/398988663 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b40d1a80 con 0x5610ee3da270 2024-04-21T08:39:57.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.885+0000 7f66c1ffb640 1 -- 172.21.15.86:0/398988663 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66b402cc20 con 0x5610ee3da270 2024-04-21T08:39:57.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.885+0000 7f66c1ffb640 1 -- 172.21.15.86:0/398988663 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66b40d1c60 con 0x5610ee3da270 2024-04-21T08:39:57.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.886+0000 7f66d53a3740 1 -- 172.21.15.86:0/398988663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ee3da270 msgr2=0x5610ee3e53f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:57.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.886+0000 7f66d53a3740 1 --2- 172.21.15.86:0/398988663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ee3da270 0x5610ee3e53f0 secure :-1 s=READY pgs=749 cs=0 l=1 rev1=1 crypto rx=0x7f66b4005ec0 tx=0x7f66b4005b50 comp rx=0 tx=0).stop 2024-04-21T08:39:57.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.886+0000 7f66d53a3740 1 -- 172.21.15.86:0/398988663 shutdown_connections 2024-04-21T08:39:57.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.886+0000 7f66d53a3740 1 --2- 172.21.15.86:0/398988663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5610ee3e5c90 0x5610ee3f1700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.886+0000 7f66d53a3740 1 --2- 172.21.15.86:0/398988663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610ee3dab90 0x5610ee3e1c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.886+0000 7f66d53a3740 1 --2- 172.21.15.86:0/398988663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ee3da270 0x5610ee3e53f0 unknown :-1 s=CLOSED pgs=749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.886+0000 7f66d53a3740 1 -- 172.21.15.86:0/398988663 >> 172.21.15.86:0/398988663 conn(0x5610ee3d4cb0 msgr2=0x5610ee3d6920 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:57.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.886+0000 7f66d53a3740 1 -- 172.21.15.86:0/398988663 shutdown_connections 2024-04-21T08:39:57.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.886+0000 7f66d53a3740 1 -- 172.21.15.86:0/398988663 wait complete. 2024-04-21T08:39:57.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.886+0000 7f66d53a3740 1 -- start start 2024-04-21T08:39:57.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.886+0000 7f66d53a3740 1 Processor -- start 2024-04-21T08:39:57.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.887+0000 7f66d53a3740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5610ee3da270 0x5610ee3e98a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:57.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.887+0000 7f66d53a3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ee3dab90 0x5610ee3e9de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:57.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.887+0000 7f66d53a3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610ee3e5c90 0x5610ee3ea320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:57.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.887+0000 7f66d53a3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f66cc1ae490 con 0x5610ee3dab90 2024-04-21T08:39:57.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.887+0000 7f66d53a3740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f66cc1addc0 con 0x5610ee3da270 2024-04-21T08:39:57.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.887+0000 7f66d53a3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f66cc1aafe0 con 0x5610ee3e5c90 2024-04-21T08:39:57.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.887+0000 7f66d53a3740 10 client.0 _ll_register_callbacks cb 0x7f66bc0095b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:57.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.887+0000 7f66cb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ee3dab90 0x5610ee3e9de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:57.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.887+0000 7f66cb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ee3dab90 0x5610ee3e9de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45856/0 (socket says 172.21.15.86:45856) 2024-04-21T08:39:57.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.887+0000 7f66cbfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5610ee3da270 0x5610ee3e98a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:57.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.887+0000 7f66cb7fe640 1 -- 172.21.15.86:0/1079486800 learned_addr learned my addr 172.21.15.86:0/1079486800 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:57.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.887+0000 7f66d0c6f640 1 --2- 172.21.15.86:0/1079486800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610ee3e5c90 0x5610ee3ea320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:57.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.887+0000 7f66cb7fe640 1 -- 172.21.15.86:0/1079486800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610ee3e5c90 msgr2=0x5610ee3ea320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:57.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.887+0000 7f66cb7fe640 1 --2- 172.21.15.86:0/1079486800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610ee3e5c90 0x5610ee3ea320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.887+0000 7f66cb7fe640 1 -- 172.21.15.86:0/1079486800 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5610ee3da270 msgr2=0x5610ee3e98a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:57.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.887+0000 7f66cb7fe640 1 --2- 172.21.15.86:0/1079486800 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5610ee3da270 0x5610ee3e98a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.887+0000 7f66cb7fe640 1 -- 172.21.15.86:0/1079486800 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5610ee45b300 con 0x5610ee3dab90 2024-04-21T08:39:57.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.887+0000 7f66cb7fe640 1 --2- 172.21.15.86:0/1079486800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ee3dab90 0x5610ee3e9de0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f66bc000c00 tx=0x7f66bc0109e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d27116bf3357b88f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:57.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.887+0000 7f66ab7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:57.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.887+0000 7f66ab7fe640 1 -- 172.21.15.86:0/1079486800 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f66bc004020 con 0x5610ee3dab90 2024-04-21T08:39:57.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.887+0000 7f66ab7fe640 1 -- 172.21.15.86:0/1079486800 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f66bc0041c0 con 0x5610ee3dab90 2024-04-21T08:39:57.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.887+0000 7f66d53a3740 1 -- 172.21.15.86:0/1079486800 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f66b402cc20 con 0x5610ee3dab90 2024-04-21T08:39:57.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.888+0000 7f66ab7fe640 1 -- 172.21.15.86:0/1079486800 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f66bc038050 con 0x5610ee3dab90 2024-04-21T08:39:57.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.888+0000 7f66d53a3740 10 client.16410 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:57.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.888+0000 7f66d53a3740 1 -- 172.21.15.86:0/1079486800 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5610ee45c560 con 0x5610ee3dab90 2024-04-21T08:39:57.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.888+0000 7f66ab7fe640 1 -- 172.21.15.86:0/1079486800 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f66b402cc20 con 0x5610ee3dab90 2024-04-21T08:39:57.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.888+0000 7f66ab7fe640 1 -- 172.21.15.86:0/1079486800 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f66bc003680 con 0x5610ee3dab90 2024-04-21T08:39:57.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.888+0000 7f66d53a3740 10 client.16410 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:57.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.888+0000 7f66d53a3740 10 client.16410 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:57.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.888+0000 7f66d53a3740 20 client.16410 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:57.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.888+0000 7f66d53a3740 1 --2- 172.21.15.86:0/1079486800 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5610ee45cd10 0x5610ee45f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:57.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.888+0000 7f66d53a3740 4 client.16410 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:39:57.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.888+0000 7f66d53a3740 1 -- 172.21.15.86:0/1079486800 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5610ee346a70 con 0x5610ee45cd10 2024-04-21T08:39:57.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.888+0000 7f66cbfff640 1 --2- 172.21.15.86:0/1079486800 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5610ee45cd10 0x5610ee45f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:57.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.889+0000 7f66cbfff640 1 --2- 172.21.15.86:0/1079486800 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5610ee45cd10 0x5610ee45f100 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b02abaab49e279f8 server_cookie=cef037b03df28da1 in_seq=0 out_seq=0 2024-04-21T08:39:57.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.889+0000 7f66ab7fe640 10 client.16410 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:57.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.889+0000 7f66ab7fe640 1 -- 172.21.15.86:0/1079486800 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5610ee346a70 con 0x5610ee45cd10 2024-04-21T08:39:57.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.889+0000 7f66ab7fe640 10 client.16410 handle_command_reply: tid=0 2024-04-21T08:39:57.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.889+0000 7f66d53a3740 1 client.16410 shutdown 2024-04-21T08:39:57.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.889+0000 7f66d53a3740 1 -- 172.21.15.86:0/1079486800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ee3dab90 msgr2=0x5610ee3e9de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:57.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.889+0000 7f66d53a3740 1 --2- 172.21.15.86:0/1079486800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ee3dab90 0x5610ee3e9de0 secure :-1 s=READY pgs=750 cs=0 l=1 rev1=1 crypto rx=0x7f66bc000c00 tx=0x7f66bc0109e0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.889+0000 7f66d53a3740 1 -- 172.21.15.86:0/1079486800 shutdown_connections 2024-04-21T08:39:57.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.889+0000 7f66d53a3740 1 --2- 172.21.15.86:0/1079486800 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5610ee45cd10 0x5610ee45f100 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.889+0000 7f66d53a3740 1 --2- 172.21.15.86:0/1079486800 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5610ee3e5c90 0x5610ee3ea320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.889+0000 7f66d53a3740 1 --2- 172.21.15.86:0/1079486800 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5610ee3dab90 0x5610ee3e9de0 unknown :-1 s=CLOSED pgs=750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.889+0000 7f66d53a3740 1 --2- 172.21.15.86:0/1079486800 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5610ee3da270 0x5610ee3e98a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.889+0000 7f66ab7fe640 0 client.16410 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:57.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.889+0000 7f66d53a3740 1 -- 172.21.15.86:0/1079486800 >> 172.21.15.86:0/1079486800 conn(0x5610ee3d4cb0 msgr2=0x5610ee3e2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:57.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.889+0000 7f66d53a3740 1 -- 172.21.15.86:0/1079486800 shutdown_connections 2024-04-21T08:39:57.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.889+0000 7f66d53a3740 1 -- 172.21.15.86:0/1079486800 wait complete. 2024-04-21T08:39:57.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.889+0000 7f66d53a3740 20 client.16410 trim_cache size 0 max 16384 2024-04-21T08:39:57.895 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:39:57.895 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:39:57.895 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-21T08:39:57.895 INFO:teuthology.orchestra.run.smithi086.stdout: "be9262df-7152-4ace-bfdc-404253b8f9eb": { 2024-04-21T08:39:57.895 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-21T08:39:57.895 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "be9262df-7152-4ace-bfdc-404253b8f9eb", 2024-04-21T08:39:57.895 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-21T08:39:57.895 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:39:57.896 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:39:57.896 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:39:57.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.890+0000 7f66d26d2640 1 -- 172.21.15.86:0/3384218865 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f66b80a3e40 msgr2=0x7f66b80a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:57.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.890+0000 7f66d26d2640 1 --2- 172.21.15.86:0/3384218865 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f66b80a3e40 0x7f66b80a6300 secure :-1 s=READY pgs=104 cs=0 l=1 rev1=1 crypto rx=0x7f66c40094d0 tx=0x7f66c4013b20 comp rx=0 tx=0).stop 2024-04-21T08:39:57.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.890+0000 7f66d26d2640 1 -- 172.21.15.86:0/3384218865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66cc101090 msgr2=0x7f66cc1975e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:57.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.890+0000 7f66d26d2640 1 --2- 172.21.15.86:0/3384218865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66cc101090 0x7f66cc1975e0 secure :-1 s=READY pgs=746 cs=0 l=1 rev1=1 crypto rx=0x7f66b4005940 tx=0x7f66b402c5f0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.890+0000 7f66d26d2640 1 -- 172.21.15.86:0/3384218865 shutdown_connections 2024-04-21T08:39:57.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.890+0000 7f66d26d2640 1 --2- 172.21.15.86:0/3384218865 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f66b80a3e40 0x7f66b80a6300 unknown :-1 s=CLOSED pgs=104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.890+0000 7f66d26d2640 1 --2- 172.21.15.86:0/3384218865 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f66cc19bea0 0x7f66cc19c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.890+0000 7f66d26d2640 1 --2- 172.21.15.86:0/3384218865 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f66cc106160 0x7f66cc197b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.890+0000 7f66d26d2640 1 --2- 172.21.15.86:0/3384218865 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f66cc101090 0x7f66cc1975e0 unknown :-1 s=CLOSED pgs=746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:57.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.890+0000 7f66d26d2640 1 -- 172.21.15.86:0/3384218865 >> 172.21.15.86:0/3384218865 conn(0x7f66cc0771d0 msgr2=0x7f66cc0fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:57.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.890+0000 7f66d26d2640 1 -- 172.21.15.86:0/3384218865 shutdown_connections 2024-04-21T08:39:57.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:39:57.890+0000 7f66d26d2640 1 -- 172.21.15.86:0/3384218865 wait complete. 2024-04-21T08:39:57.899 INFO:tasks.cephfs.filesystem:scrub status for tag:be9262df-7152-4ace-bfdc-404253b8f9eb - {'path': '/', 'tag': 'be9262df-7152-4ace-bfdc-404253b8f9eb', 'options': 'recursive,force'} 2024-04-21T08:39:58.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:58 smithi088 ceph-mon[31259]: pgmap v38: 129 pgs: 129 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-04-21T08:39:58.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:58 smithi179 ceph-mon[31829]: pgmap v38: 129 pgs: 129 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-04-21T08:39:58.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:58 smithi086 ceph-mon[27939]: pgmap v38: 129 pgs: 129 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 12 KiB/s wr, 14 op/s 2024-04-21T08:39:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.614+0000 7f53b9081640 1 -- 172.21.15.86:0/2927622679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53b4103200 msgr2=0x7f53b41035e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:59.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.614+0000 7f53b9081640 1 --2- 172.21.15.86:0/2927622679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53b4103200 0x7f53b41035e0 secure :-1 s=READY pgs=751 cs=0 l=1 rev1=1 crypto rx=0x7f53a4009ca0 tx=0x7f53a402c180 comp rx=0 tx=0).stop 2024-04-21T08:39:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.615+0000 7f53b9081640 1 -- 172.21.15.86:0/2927622679 shutdown_connections 2024-04-21T08:39:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.615+0000 7f53b9081640 1 --2- 172.21.15.86:0/2927622679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53b410c540 0x7f53b410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.615+0000 7f53b9081640 1 --2- 172.21.15.86:0/2927622679 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53b4103b20 0x7f53b410c000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.615+0000 7f53b9081640 1 --2- 172.21.15.86:0/2927622679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53b4103200 0x7f53b41035e0 unknown :-1 s=CLOSED pgs=751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.615+0000 7f53b9081640 1 -- 172.21.15.86:0/2927622679 >> 172.21.15.86:0/2927622679 conn(0x7f53b4077fd0 msgr2=0x7f53b40783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.615+0000 7f53b9081640 1 -- 172.21.15.86:0/2927622679 shutdown_connections 2024-04-21T08:39:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.615+0000 7f53b9081640 1 -- 172.21.15.86:0/2927622679 wait complete. 2024-04-21T08:39:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.615+0000 7f53b9081640 1 Processor -- start 2024-04-21T08:39:59.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.616+0000 7f53b9081640 1 -- start start 2024-04-21T08:39:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.616+0000 7f53b9081640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53b4103200 0x7f53b4198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.616+0000 7f53b9081640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53b4103b20 0x7f53b4198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:59.617 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.616+0000 7f53b9081640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53b410c540 0x7f53b419cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.616+0000 7f53b9081640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f53b41121a0 con 0x7f53b4103200 2024-04-21T08:39:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.616+0000 7f53b9081640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f53b4112020 con 0x7f53b410c540 2024-04-21T08:39:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.616+0000 7f53b9081640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f53b4112320 con 0x7f53b4103b20 2024-04-21T08:39:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.616+0000 7f53b2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53b4103200 0x7f53b4198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.616+0000 7f53b2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53b4103200 0x7f53b4198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50798/0 (socket says 172.21.15.86:50798) 2024-04-21T08:39:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.616+0000 7f53b2d9d640 1 -- 172.21.15.86:0/782683437 learned_addr learned my addr 172.21.15.86:0/782683437 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.616+0000 7f53b259c640 1 --2- 172.21.15.86:0/782683437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53b4103b20 0x7f53b4198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.616+0000 7f53b359e640 1 --2- 172.21.15.86:0/782683437 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53b410c540 0x7f53b419cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.616+0000 7f53b2d9d640 1 -- 172.21.15.86:0/782683437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53b4103b20 msgr2=0x7f53b4198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.616+0000 7f53b2d9d640 1 --2- 172.21.15.86:0/782683437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53b4103b20 0x7f53b4198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.616+0000 7f53b2d9d640 1 -- 172.21.15.86:0/782683437 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53b410c540 msgr2=0x7f53b419cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.616+0000 7f53b2d9d640 1 --2- 172.21.15.86:0/782683437 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53b410c540 0x7f53b419cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:59.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.616+0000 7f53b2d9d640 1 -- 172.21.15.86:0/782683437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f53b419d6e0 con 0x7f53b4103200 2024-04-21T08:39:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.617+0000 7f53b2d9d640 1 --2- 172.21.15.86:0/782683437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53b4103200 0x7f53b4198740 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f53a4005e30 tx=0x7f53a40039f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.617+0000 7f539bfff640 1 -- 172.21.15.86:0/782683437 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a4039040 con 0x7f53b4103200 2024-04-21T08:39:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.617+0000 7f539bfff640 1 -- 172.21.15.86:0/782683437 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53a4044070 con 0x7f53b4103200 2024-04-21T08:39:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.617+0000 7f539bfff640 1 -- 172.21.15.86:0/782683437 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a4032960 con 0x7f53b4103200 2024-04-21T08:39:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.617+0000 7f53b9081640 1 -- 172.21.15.86:0/782683437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f53b419d970 con 0x7f53b4103200 2024-04-21T08:39:59.619 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.617+0000 7f53b9081640 1 -- 172.21.15.86:0/782683437 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f53b419dde0 con 0x7f53b4103200 2024-04-21T08:39:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.619+0000 7f539bfff640 1 -- 172.21.15.86:0/782683437 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f53a4004580 con 0x7f53b4103200 2024-04-21T08:39:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.620+0000 7f53bbc7b740 1 Processor -- start 2024-04-21T08:39:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.620+0000 7f539bfff640 1 --2- 172.21.15.86:0/782683437 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f539c0a3ee0 0x7f539c0a63a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.620+0000 7f53bbc7b740 1 -- start start 2024-04-21T08:39:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.620+0000 7f53b359e640 1 --2- 172.21.15.86:0/782683437 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f539c0a3ee0 0x7f539c0a63a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.620+0000 7f539bfff640 1 -- 172.21.15.86:0/782683437 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f53a40f3270 con 0x7f53b4103200 2024-04-21T08:39:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.620+0000 7f53bbc7b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56176a4658d0 0x56176a464620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.620+0000 7f53bbc7b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56176a464b60 0x56176a464fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.620+0000 7f53bbc7b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176a4724f0 0x56176a4748e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.620+0000 7f53bbc7b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56176a3c98d0 con 0x56176a4724f0 2024-04-21T08:39:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.620+0000 7f53bbc7b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x56176a366e80 con 0x56176a4658d0 2024-04-21T08:39:59.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.620+0000 7f53bbc7b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56176a3c5890 con 0x56176a464b60 2024-04-21T08:39:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.620+0000 7f53b359e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176a4724f0 0x56176a4748e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.620+0000 7f53b259c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56176a464b60 0x56176a464fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.620+0000 7f53b259c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56176a464b60 0x56176a464fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.86:43034/0 (socket says 172.21.15.86:43034) 2024-04-21T08:39:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.620+0000 7f53b359e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176a4724f0 0x56176a4748e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50804/0 (socket says 172.21.15.86:50804) 2024-04-21T08:39:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.620+0000 7f53b259c640 1 -- 172.21.15.86:0/863033705 learned_addr learned my addr 172.21.15.86:0/863033705 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.620+0000 7f53b2d9d640 1 --2- 172.21.15.86:0/863033705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56176a4658d0 0x56176a464620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.620+0000 7f53b359e640 1 --2- 172.21.15.86:0/782683437 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f539c0a3ee0 0x7f539c0a63a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f53ac004600 tx=0x7f53ac00b9d0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.620+0000 7f53b359e640 1 -- 172.21.15.86:0/863033705 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56176a464b60 msgr2=0x56176a464fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.620+0000 7f53b359e640 1 --2- 172.21.15.86:0/863033705 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56176a464b60 0x56176a464fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.620+0000 7f53b359e640 1 -- 172.21.15.86:0/863033705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56176a4658d0 msgr2=0x56176a464620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.620+0000 7f53b359e640 1 --2- 172.21.15.86:0/863033705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56176a4658d0 0x56176a464620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.620+0000 7f53b359e640 1 -- 172.21.15.86:0/863033705 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56176a388320 con 0x56176a4724f0 2024-04-21T08:39:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.620+0000 7f53b259c640 1 --2- 172.21.15.86:0/863033705 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56176a464b60 0x56176a464fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:59.627 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.621+0000 7f53b359e640 1 --2- 172.21.15.86:0/863033705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176a4724f0 0x56176a4748e0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f53ac00bfd0 tx=0x7f53ac01b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eff94f79f1d0dac server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:59.628 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.621+0000 7f53997fa640 1 -- 172.21.15.86:0/863033705 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ac013a70 con 0x56176a4724f0 2024-04-21T08:39:59.628 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.621+0000 7f53997fa640 1 -- 172.21.15.86:0/863033705 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53ac013c10 con 0x56176a4724f0 2024-04-21T08:39:59.628 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.621+0000 7f53997fa640 1 -- 172.21.15.86:0/863033705 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53ac01e420 con 0x56176a4724f0 2024-04-21T08:39:59.628 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.621+0000 7f53bbc7b740 1 -- 172.21.15.86:0/863033705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176a4724f0 msgr2=0x56176a4748e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:59.628 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.621+0000 7f53bbc7b740 1 --2- 172.21.15.86:0/863033705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176a4724f0 0x56176a4748e0 secure :-1 s=READY pgs=753 cs=0 l=1 rev1=1 crypto rx=0x7f53ac00bfd0 tx=0x7f53ac01b000 comp rx=0 tx=0).stop 2024-04-21T08:39:59.628 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.622+0000 7f53bbc7b740 1 -- 172.21.15.86:0/863033705 shutdown_connections 2024-04-21T08:39:59.628 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.622+0000 7f53bbc7b740 1 --2- 172.21.15.86:0/863033705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176a4724f0 0x56176a4748e0 unknown :-1 s=CLOSED pgs=753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:59.628 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.622+0000 7f53bbc7b740 1 --2- 172.21.15.86:0/863033705 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56176a464b60 0x56176a464fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:59.628 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.622+0000 7f53bbc7b740 1 --2- 172.21.15.86:0/863033705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56176a4658d0 0x56176a464620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:59.628 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.622+0000 7f53bbc7b740 1 -- 172.21.15.86:0/863033705 >> 172.21.15.86:0/863033705 conn(0x56176a42bc90 msgr2=0x56176a45f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:59.628 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.622+0000 7f53bbc7b740 1 -- 172.21.15.86:0/863033705 shutdown_connections 2024-04-21T08:39:59.628 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.622+0000 7f53bbc7b740 1 -- 172.21.15.86:0/863033705 wait complete. 2024-04-21T08:39:59.628 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.622+0000 7f53bbc7b740 1 -- start start 2024-04-21T08:39:59.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.622+0000 7f53bbc7b740 1 Processor -- start 2024-04-21T08:39:59.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.622+0000 7f53bbc7b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56176a464b60 0x56176a471ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:59.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.622+0000 7f53bbc7b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176a4658d0 0x56176a46f270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:59.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.622+0000 7f53bbc7b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56176a4724f0 0x56176a46f880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:59.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.622+0000 7f53bbc7b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56176a475220 con 0x56176a4658d0 2024-04-21T08:39:59.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.622+0000 7f53bbc7b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x56176a351eb0 con 0x56176a464b60 2024-04-21T08:39:59.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.622+0000 7f53bbc7b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56176a4753a0 con 0x56176a4724f0 2024-04-21T08:39:59.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.622+0000 7f53bbc7b740 10 client.0 _ll_register_callbacks cb 0x56176a37eb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:59.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.622+0000 7f53b259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176a4658d0 0x56176a46f270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:59.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.622+0000 7f53b259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176a4658d0 0x56176a46f270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50818/0 (socket says 172.21.15.86:50818) 2024-04-21T08:39:59.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.622+0000 7f53b2d9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56176a464b60 0x56176a471ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:59.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.622+0000 7f53b259c640 1 -- 172.21.15.86:0/746675111 learned_addr learned my addr 172.21.15.86:0/746675111 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:59.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.623+0000 7f53b359e640 1 --2- 172.21.15.86:0/746675111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56176a4724f0 0x56176a46f880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:59.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.623+0000 7f53b259c640 1 -- 172.21.15.86:0/746675111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56176a4724f0 msgr2=0x56176a46f880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:59.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.623+0000 7f53b259c640 1 --2- 172.21.15.86:0/746675111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56176a4724f0 0x56176a46f880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.623+0000 7f53b259c640 1 -- 172.21.15.86:0/746675111 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56176a464b60 msgr2=0x56176a471ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.623+0000 7f53b259c640 1 --2- 172.21.15.86:0/746675111 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56176a464b60 0x56176a471ee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.623+0000 7f53b259c640 1 -- 172.21.15.86:0/746675111 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56176a0dc210 con 0x56176a4658d0 2024-04-21T08:39:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.623+0000 7f53b2d9d640 1 --2- 172.21.15.86:0/746675111 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56176a464b60 0x56176a471ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:39:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.623+0000 7f53b259c640 1 --2- 172.21.15.86:0/746675111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176a4658d0 0x56176a46f270 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f53a800baa0 tx=0x7f53a800bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d517ad10366ae357 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.623+0000 7f5396ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.623+0000 7f5396ff5640 1 -- 172.21.15.86:0/746675111 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a8011440 con 0x56176a4658d0 2024-04-21T08:39:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.623+0000 7f5396ff5640 1 -- 172.21.15.86:0/746675111 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f53a80115e0 con 0x56176a4658d0 2024-04-21T08:39:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.623+0000 7f53bbc7b740 1 -- 172.21.15.86:0/746675111 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53ac013c10 con 0x56176a4658d0 2024-04-21T08:39:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.623+0000 7f5396ff5640 1 -- 172.21.15.86:0/746675111 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f53a80117e0 con 0x56176a4658d0 2024-04-21T08:39:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.623+0000 7f53bbc7b740 10 client.16422 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.623+0000 7f53bbc7b740 1 -- 172.21.15.86:0/746675111 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56176a1a8620 con 0x56176a4658d0 2024-04-21T08:39:59.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.623+0000 7f5396ff5640 1 -- 172.21.15.86:0/746675111 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53ac013c10 con 0x56176a4658d0 2024-04-21T08:39:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.624+0000 7f5396ff5640 1 -- 172.21.15.86:0/746675111 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f53a801b9f0 con 0x56176a4658d0 2024-04-21T08:39:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.624+0000 7f53bbc7b740 10 client.16422 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.624+0000 7f53bbc7b740 10 client.16422 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.624+0000 7f53bbc7b740 20 client.16422 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.624+0000 7f53bbc7b740 1 --2- 172.21.15.86:0/746675111 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56176a4825a0 0x56176a4849f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.624+0000 7f53bbc7b740 4 client.16422 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:39:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.624+0000 7f53bbc7b740 1 -- 172.21.15.86:0/746675111 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56176a3dd880 con 0x56176a4825a0 2024-04-21T08:39:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.624+0000 7f53b2d9d640 1 --2- 172.21.15.86:0/746675111 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56176a4825a0 0x56176a4849f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.624+0000 7f53b2d9d640 1 --2- 172.21.15.86:0/746675111 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56176a4825a0 0x56176a4849f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1531a9748a4fff8 server_cookie=37daa5a507f80d49 in_seq=0 out_seq=0 2024-04-21T08:39:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.624+0000 7f5396ff5640 10 client.16422 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.625+0000 7f5396ff5640 1 -- 172.21.15.86:0/746675111 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56176a3dd880 con 0x56176a4825a0 2024-04-21T08:39:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.625+0000 7f5396ff5640 10 client.16422 handle_command_reply: tid=0 2024-04-21T08:39:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.625+0000 7f53bbc7b740 1 client.16422 shutdown 2024-04-21T08:39:59.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.625+0000 7f53bbc7b740 1 -- 172.21.15.86:0/746675111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176a4658d0 msgr2=0x56176a46f270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.625+0000 7f53bbc7b740 1 --2- 172.21.15.86:0/746675111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176a4658d0 0x56176a46f270 secure :-1 s=READY pgs=754 cs=0 l=1 rev1=1 crypto rx=0x7f53a800baa0 tx=0x7f53a800bf60 comp rx=0 tx=0).stop 2024-04-21T08:39:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.625+0000 7f53bbc7b740 1 -- 172.21.15.86:0/746675111 shutdown_connections 2024-04-21T08:39:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.625+0000 7f53bbc7b740 1 --2- 172.21.15.86:0/746675111 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56176a4825a0 0x56176a4849f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.625+0000 7f53bbc7b740 1 --2- 172.21.15.86:0/746675111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56176a4724f0 0x56176a46f880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.625+0000 7f53bbc7b740 1 --2- 172.21.15.86:0/746675111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176a4658d0 0x56176a46f270 unknown :-1 s=CLOSED pgs=754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.625+0000 7f53bbc7b740 1 --2- 172.21.15.86:0/746675111 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56176a464b60 0x56176a471ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.625+0000 7f5396ff5640 0 client.16422 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.625+0000 7f53bbc7b740 1 -- 172.21.15.86:0/746675111 >> 172.21.15.86:0/746675111 conn(0x56176a42bc90 msgr2=0x56176a4615a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.625+0000 7f53bbc7b740 1 -- 172.21.15.86:0/746675111 shutdown_connections 2024-04-21T08:39:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.625+0000 7f53bbc7b740 1 -- 172.21.15.86:0/746675111 wait complete. 2024-04-21T08:39:59.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.625+0000 7f53bbc7b740 20 client.16422 trim_cache size 0 max 16384 2024-04-21T08:39:59.645 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.641+0000 7f53bbc7b740 1 Processor -- start 2024-04-21T08:39:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.641+0000 7f53bbc7b740 1 -- start start 2024-04-21T08:39:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.641+0000 7f53bbc7b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56176a464b60 0x56176a47e120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.641+0000 7f53bbc7b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176a4658d0 0x56176a4719a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.641+0000 7f53bbc7b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56176a4724f0 0x56176a471ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.641+0000 7f53bbc7b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f53b41acff0 con 0x56176a4658d0 2024-04-21T08:39:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.641+0000 7f53bbc7b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f53b41aeb30 con 0x56176a4724f0 2024-04-21T08:39:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.641+0000 7f53bbc7b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f53b41ad6c0 con 0x56176a464b60 2024-04-21T08:39:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.641+0000 7f53b259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176a4658d0 0x56176a4719a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.641+0000 7f53b259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176a4658d0 0x56176a4719a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50820/0 (socket says 172.21.15.86:50820) 2024-04-21T08:39:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.641+0000 7f53b259c640 1 -- 172.21.15.86:0/2767108418 learned_addr learned my addr 172.21.15.86:0/2767108418 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.642+0000 7f53b2d9d640 1 --2- 172.21.15.86:0/2767108418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56176a464b60 0x56176a47e120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.642+0000 7f53b259c640 1 -- 172.21.15.86:0/2767108418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56176a464b60 msgr2=0x56176a47e120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:59.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.642+0000 7f53b359e640 1 --2- 172.21.15.86:0/2767108418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56176a4724f0 0x56176a471ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.642+0000 7f53b259c640 1 --2- 172.21.15.86:0/2767108418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56176a464b60 0x56176a47e120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.642+0000 7f53b259c640 1 -- 172.21.15.86:0/2767108418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56176a4724f0 msgr2=0x56176a471ee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.642+0000 7f53b259c640 1 --2- 172.21.15.86:0/2767108418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56176a4724f0 0x56176a471ee0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.642+0000 7f53b259c640 1 -- 172.21.15.86:0/2767108418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56176a45af30 con 0x56176a4658d0 2024-04-21T08:39:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.642+0000 7f53b2d9d640 1 --2- 172.21.15.86:0/2767108418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56176a464b60 0x56176a47e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:39:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.642+0000 7f53b259c640 1 --2- 172.21.15.86:0/2767108418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176a4658d0 0x56176a4719a0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f53a8003b00 tx=0x7f53a800b400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb0d1ef6c8a72c0d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.642+0000 7f5398ff9640 1 -- 172.21.15.86:0/2767108418 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a8011440 con 0x56176a4658d0 2024-04-21T08:39:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.642+0000 7f5398ff9640 1 -- 172.21.15.86:0/2767108418 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53a802e070 con 0x56176a4658d0 2024-04-21T08:39:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.642+0000 7f5398ff9640 1 -- 172.21.15.86:0/2767108418 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a801b420 con 0x56176a4658d0 2024-04-21T08:39:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.643+0000 7f53bbc7b740 1 -- 172.21.15.86:0/2767108418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176a4658d0 msgr2=0x56176a4719a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.643+0000 7f53bbc7b740 1 --2- 172.21.15.86:0/2767108418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176a4658d0 0x56176a4719a0 secure :-1 s=READY pgs=755 cs=0 l=1 rev1=1 crypto rx=0x7f53a8003b00 tx=0x7f53a800b400 comp rx=0 tx=0).stop 2024-04-21T08:39:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.643+0000 7f53bbc7b740 1 -- 172.21.15.86:0/2767108418 shutdown_connections 2024-04-21T08:39:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.643+0000 7f53bbc7b740 1 --2- 172.21.15.86:0/2767108418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56176a4724f0 0x56176a471ee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.643+0000 7f53bbc7b740 1 --2- 172.21.15.86:0/2767108418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176a4658d0 0x56176a4719a0 unknown :-1 s=CLOSED pgs=755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.643+0000 7f53bbc7b740 1 --2- 172.21.15.86:0/2767108418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56176a464b60 0x56176a47e120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:59.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.643+0000 7f53bbc7b740 1 -- 172.21.15.86:0/2767108418 >> 172.21.15.86:0/2767108418 conn(0x56176a42bc90 msgr2=0x56176a46db40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.643+0000 7f53bbc7b740 1 -- 172.21.15.86:0/2767108418 shutdown_connections 2024-04-21T08:39:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.643+0000 7f53bbc7b740 1 -- 172.21.15.86:0/2767108418 wait complete. 2024-04-21T08:39:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.643+0000 7f53bbc7b740 1 -- start start 2024-04-21T08:39:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.643+0000 7f53bbc7b740 1 Processor -- start 2024-04-21T08:39:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.643+0000 7f53bbc7b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56176a464b60 0x56176a4758c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.643+0000 7f53bbc7b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176a4658d0 0x56176a475e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.643+0000 7f53bbc7b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56176a4724f0 0x56176a4e0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.643+0000 7f53bbc7b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f53b41ae460 con 0x56176a4658d0 2024-04-21T08:39:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.643+0000 7f53bbc7b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f53b41add90 con 0x56176a464b60 2024-04-21T08:39:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.643+0000 7f53bbc7b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f53b41aafb0 con 0x56176a4724f0 2024-04-21T08:39:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.643+0000 7f53bbc7b740 10 client.0 _ll_register_callbacks cb 0x7f53a8004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:39:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.643+0000 7f53b259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176a4658d0 0x56176a475e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.643+0000 7f53b259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176a4658d0 0x56176a475e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50830/0 (socket says 172.21.15.86:50830) 2024-04-21T08:39:59.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.643+0000 7f53b259c640 1 -- 172.21.15.86:0/949706240 learned_addr learned my addr 172.21.15.86:0/949706240 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:39:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.644+0000 7f53b2d9d640 1 --2- 172.21.15.86:0/949706240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56176a464b60 0x56176a4758c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.644+0000 7f53b359e640 1 --2- 172.21.15.86:0/949706240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56176a4724f0 0x56176a4e0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.644+0000 7f53b259c640 1 -- 172.21.15.86:0/949706240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56176a4724f0 msgr2=0x56176a4e0ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.644+0000 7f53b259c640 1 --2- 172.21.15.86:0/949706240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56176a4724f0 0x56176a4e0ba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.644+0000 7f53b259c640 1 -- 172.21.15.86:0/949706240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56176a464b60 msgr2=0x56176a4758c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:39:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.644+0000 7f53b259c640 1 --2- 172.21.15.86:0/949706240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56176a464b60 0x56176a4758c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.644+0000 7f53b259c640 1 -- 172.21.15.86:0/949706240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56176a4e1160 con 0x56176a4658d0 2024-04-21T08:39:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.644+0000 7f53b259c640 1 --2- 172.21.15.86:0/949706240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176a4658d0 0x56176a475e00 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f53a802b040 tx=0x7f53a800b590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d868abee1e64556 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:39:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.644+0000 7f5396ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:39:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.644+0000 7f5396ff5640 1 -- 172.21.15.86:0/949706240 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53a80057b0 con 0x56176a4658d0 2024-04-21T08:39:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.644+0000 7f5396ff5640 1 -- 172.21.15.86:0/949706240 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f53a8011980 con 0x56176a4658d0 2024-04-21T08:39:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.644+0000 7f53bbc7b740 1 -- 172.21.15.86:0/949706240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f53a802e070 con 0x56176a4658d0 2024-04-21T08:39:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.644+0000 7f5396ff5640 1 -- 172.21.15.86:0/949706240 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f53a8011b80 con 0x56176a4658d0 2024-04-21T08:39:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.644+0000 7f53bbc7b740 10 client.16428 fetch_fsmap learned FSMap version 29 2024-04-21T08:39:59.649 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.644+0000 7f53bbc7b740 1 -- 172.21.15.86:0/949706240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56176a4e8450 con 0x56176a4658d0 2024-04-21T08:39:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.644+0000 7f5396ff5640 1 -- 172.21.15.86:0/949706240 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f53a802e070 con 0x56176a4658d0 2024-04-21T08:39:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.645+0000 7f5396ff5640 1 -- 172.21.15.86:0/949706240 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f53a80107d0 con 0x56176a4658d0 2024-04-21T08:39:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.645+0000 7f53bbc7b740 10 client.16428 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:39:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.645+0000 7f53bbc7b740 10 client.16428 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:39:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.645+0000 7f53bbc7b740 20 client.16428 populate_metadata read hostname 'smithi086' 2024-04-21T08:39:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.645+0000 7f53bbc7b740 1 --2- 172.21.15.86:0/949706240 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56176a4e8980 0x56176a4eada0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:39:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.645+0000 7f53bbc7b740 4 client.16428 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:39:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.645+0000 7f53bbc7b740 1 -- 172.21.15.86:0/949706240 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56176a3693d0 con 0x56176a4e8980 2024-04-21T08:39:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.645+0000 7f53b2d9d640 1 --2- 172.21.15.86:0/949706240 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56176a4e8980 0x56176a4eada0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:39:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.645+0000 7f5396ff5640 10 client.16428 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:59.650 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.645+0000 7f53b2d9d640 1 --2- 172.21.15.86:0/949706240 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56176a4e8980 0x56176a4eada0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e1b96da322b7923 server_cookie=cb3c84acb7660052 in_seq=0 out_seq=0 2024-04-21T08:39:59.657 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.657+0000 7f5396ff5640 1 -- 172.21.15.86:0/949706240 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56176a3693d0 con 0x56176a4e8980 2024-04-21T08:39:59.657 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.657+0000 7f5396ff5640 10 client.16428 handle_command_reply: tid=0 2024-04-21T08:39:59.657 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.658+0000 7f53bbc7b740 1 client.16428 shutdown 2024-04-21T08:39:59.657 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.658+0000 7f53bbc7b740 1 -- 172.21.15.86:0/949706240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176a4658d0 msgr2=0x56176a475e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.658+0000 7f53bbc7b740 1 --2- 172.21.15.86:0/949706240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176a4658d0 0x56176a475e00 secure :-1 s=READY pgs=756 cs=0 l=1 rev1=1 crypto rx=0x7f53a802b040 tx=0x7f53a800b590 comp rx=0 tx=0).stop 2024-04-21T08:39:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.658+0000 7f53bbc7b740 1 -- 172.21.15.86:0/949706240 shutdown_connections 2024-04-21T08:39:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.658+0000 7f53bbc7b740 1 --2- 172.21.15.86:0/949706240 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56176a4e8980 0x56176a4eada0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.658+0000 7f53bbc7b740 1 --2- 172.21.15.86:0/949706240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56176a4724f0 0x56176a4e0ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.658+0000 7f5396ff5640 0 client.16428 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:39:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.658+0000 7f53bbc7b740 1 --2- 172.21.15.86:0/949706240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56176a4658d0 0x56176a475e00 unknown :-1 s=CLOSED pgs=756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.658+0000 7f53bbc7b740 1 --2- 172.21.15.86:0/949706240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56176a464b60 0x56176a4758c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.658+0000 7f53bbc7b740 1 -- 172.21.15.86:0/949706240 >> 172.21.15.86:0/949706240 conn(0x56176a42bc90 msgr2=0x56176a45e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.658+0000 7f53bbc7b740 1 -- 172.21.15.86:0/949706240 shutdown_connections 2024-04-21T08:39:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.658+0000 7f53bbc7b740 1 -- 172.21.15.86:0/949706240 wait complete. 2024-04-21T08:39:59.658 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.658+0000 7f53bbc7b740 20 client.16428 trim_cache size 0 max 16384 2024-04-21T08:39:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:39:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:39:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:39:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:39:59.659 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.658+0000 7f53b9081640 1 -- 172.21.15.86:0/782683437 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f539c0a3ee0 msgr2=0x7f539c0a63a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.658+0000 7f53b9081640 1 --2- 172.21.15.86:0/782683437 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f539c0a3ee0 0x7f539c0a63a0 secure :-1 s=READY pgs=105 cs=0 l=1 rev1=1 crypto rx=0x7f53ac004600 tx=0x7f53ac00b9d0 comp rx=0 tx=0).stop 2024-04-21T08:39:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.658+0000 7f53b9081640 1 -- 172.21.15.86:0/782683437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53b4103200 msgr2=0x7f53b4198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:39:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.658+0000 7f53b9081640 1 --2- 172.21.15.86:0/782683437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53b4103200 0x7f53b4198740 secure :-1 s=READY pgs=752 cs=0 l=1 rev1=1 crypto rx=0x7f53a4005e30 tx=0x7f53a40039f0 comp rx=0 tx=0).stop 2024-04-21T08:39:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.658+0000 7f53b9081640 1 -- 172.21.15.86:0/782683437 shutdown_connections 2024-04-21T08:39:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.658+0000 7f53b9081640 1 --2- 172.21.15.86:0/782683437 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f539c0a3ee0 0x7f539c0a63a0 unknown :-1 s=CLOSED pgs=105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.658+0000 7f53b9081640 1 --2- 172.21.15.86:0/782683437 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f53b410c540 0x7f53b419cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.658+0000 7f53b9081640 1 --2- 172.21.15.86:0/782683437 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f53b4103b20 0x7f53b4198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.658+0000 7f53b9081640 1 --2- 172.21.15.86:0/782683437 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f53b4103200 0x7f53b4198740 unknown :-1 s=CLOSED pgs=752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:39:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.658+0000 7f53b9081640 1 -- 172.21.15.86:0/782683437 >> 172.21.15.86:0/782683437 conn(0x7f53b4077fd0 msgr2=0x7f53b40fdfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:39:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.659+0000 7f53b9081640 1 -- 172.21.15.86:0/782683437 shutdown_connections 2024-04-21T08:39:59.660 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:39:59.659+0000 7f53b9081640 1 -- 172.21.15.86:0/782683437 wait complete. 2024-04-21T08:39:59.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:59 smithi088 ceph-mon[31259]: scrub queued for path: / 2024-04-21T08:39:59.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:59 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:39:59.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:59 smithi088 ceph-mon[31259]: scrub summary: active paths [/] 2024-04-21T08:39:59.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:39:59 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:39:59.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:59 smithi179 ceph-mon[31829]: scrub queued for path: / 2024-04-21T08:39:59.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:59 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:39:59.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:59 smithi179 ceph-mon[31829]: scrub summary: active paths [/] 2024-04-21T08:39:59.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:39:59 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:39:59.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:59 smithi086 ceph-mon[27939]: scrub queued for path: / 2024-04-21T08:39:59.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:59 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:39:59.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:59 smithi086 ceph-mon[27939]: scrub summary: active paths [/] 2024-04-21T08:39:59.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:39:59 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:40:00.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:00 smithi088 ceph-mon[31259]: pgmap v39: 129 pgs: 129 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-04-21T08:40:00.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:00 smithi088 ceph-mon[31259]: scrub summary: idle 2024-04-21T08:40:00.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:00 smithi088 ceph-mon[31259]: overall HEALTH_OK 2024-04-21T08:40:00.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:00 smithi179 ceph-mon[31829]: pgmap v39: 129 pgs: 129 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-04-21T08:40:00.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:00 smithi179 ceph-mon[31829]: scrub summary: idle 2024-04-21T08:40:00.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:00 smithi179 ceph-mon[31829]: overall HEALTH_OK 2024-04-21T08:40:00.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:00 smithi086 ceph-mon[27939]: pgmap v39: 129 pgs: 129 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 20 KiB/s wr, 21 op/s 2024-04-21T08:40:00.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:00 smithi086 ceph-mon[27939]: scrub summary: idle 2024-04-21T08:40:00.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:00 smithi086 ceph-mon[27939]: overall HEALTH_OK 2024-04-21T08:40:02.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:02 smithi088 ceph-mon[31259]: pgmap v40: 129 pgs: 129 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-04-21T08:40:02.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:02 smithi179 ceph-mon[31829]: pgmap v40: 129 pgs: 129 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-04-21T08:40:02.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:02 smithi086 ceph-mon[27939]: pgmap v40: 129 pgs: 129 active+clean; 1.5 MiB data, 382 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 15 KiB/s wr, 13 op/s 2024-04-21T08:40:04.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:04 smithi088 ceph-mon[31259]: pgmap v41: 129 pgs: 129 active+clean; 1.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 47 KiB/s wr, 38 op/s 2024-04-21T08:40:04.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:04 smithi179 ceph-mon[31829]: pgmap v41: 129 pgs: 129 active+clean; 1.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 47 KiB/s wr, 38 op/s 2024-04-21T08:40:04.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:04 smithi086 ceph-mon[27939]: pgmap v41: 129 pgs: 129 active+clean; 1.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 47 KiB/s wr, 38 op/s 2024-04-21T08:40:06.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:06 smithi088 ceph-mon[31259]: pgmap v42: 129 pgs: 129 active+clean; 1.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 40 KiB/s wr, 31 op/s 2024-04-21T08:40:06.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:06 smithi179 ceph-mon[31829]: pgmap v42: 129 pgs: 129 active+clean; 1.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 40 KiB/s wr, 31 op/s 2024-04-21T08:40:06.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:06 smithi086 ceph-mon[27939]: pgmap v42: 129 pgs: 129 active+clean; 1.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 40 KiB/s wr, 31 op/s 2024-04-21T08:40:08.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:08 smithi088 ceph-mon[31259]: pgmap v43: 129 pgs: 129 active+clean; 1.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 40 KiB/s wr, 31 op/s 2024-04-21T08:40:08.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:08 smithi179 ceph-mon[31829]: pgmap v43: 129 pgs: 129 active+clean; 1.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 40 KiB/s wr, 31 op/s 2024-04-21T08:40:08.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:08 smithi086 ceph-mon[27939]: pgmap v43: 129 pgs: 129 active+clean; 1.6 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 40 KiB/s wr, 31 op/s 2024-04-21T08:40:10.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:10 smithi088 ceph-mon[31259]: pgmap v44: 129 pgs: 129 active+clean; 1.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 48 KiB/s wr, 38 op/s 2024-04-21T08:40:10.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:10 smithi179 ceph-mon[31829]: pgmap v44: 129 pgs: 129 active+clean; 1.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 48 KiB/s wr, 38 op/s 2024-04-21T08:40:10.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:10 smithi086 ceph-mon[27939]: pgmap v44: 129 pgs: 129 active+clean; 1.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 48 KiB/s wr, 38 op/s 2024-04-21T08:40:12.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:12 smithi088 ceph-mon[31259]: pgmap v45: 129 pgs: 129 active+clean; 1.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 40 KiB/s wr, 31 op/s 2024-04-21T08:40:12.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:12 smithi179 ceph-mon[31829]: pgmap v45: 129 pgs: 129 active+clean; 1.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 40 KiB/s wr, 31 op/s 2024-04-21T08:40:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.755+0000 7f3815474640 1 -- 172.21.15.86:0/509240869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38101072a0 msgr2=0x7f381010ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:12.756 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.755+0000 7f3815474640 1 --2- 172.21.15.86:0/509240869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38101072a0 0x7f381010ee70 secure :-1 s=READY pgs=757 cs=0 l=1 rev1=1 crypto rx=0x7f3808009ca0 tx=0x7f380802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.755+0000 7f3815474640 1 -- 172.21.15.86:0/509240869 shutdown_connections 2024-04-21T08:40:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.755+0000 7f3815474640 1 --2- 172.21.15.86:0/509240869 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f38101072a0 0x7f381010ee70 unknown :-1 s=CLOSED pgs=757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.755+0000 7f3815474640 1 --2- 172.21.15.86:0/509240869 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3810102b80 0x7f3810106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.755+0000 7f3815474640 1 --2- 172.21.15.86:0/509240869 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38101021d0 0x7f38101025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.755+0000 7f3815474640 1 -- 172.21.15.86:0/509240869 >> 172.21.15.86:0/509240869 conn(0x7f38100793e0 msgr2=0x7f38100797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.755+0000 7f3815474640 1 -- 172.21.15.86:0/509240869 shutdown_connections 2024-04-21T08:40:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.755+0000 7f3815474640 1 -- 172.21.15.86:0/509240869 wait complete. 2024-04-21T08:40:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.756+0000 7f3815474640 1 Processor -- start 2024-04-21T08:40:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.756+0000 7f3815474640 1 -- start start 2024-04-21T08:40:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.756+0000 7f3815474640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38101021d0 0x7f3810074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.756+0000 7f3815474640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38101072a0 0x7f3810074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.756+0000 7f3815474640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3810074cd0 0x7f38101a6780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:12.757 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.756+0000 7f3815474640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f38101121f0 con 0x7f3810074cd0 2024-04-21T08:40:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.756+0000 7f3815474640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3810112070 con 0x7f38101021d0 2024-04-21T08:40:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.756+0000 7f3815474640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3810112370 con 0x7f38101072a0 2024-04-21T08:40:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.756+0000 7f380f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3810074cd0 0x7f38101a6780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.756+0000 7f380f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3810074cd0 0x7f38101a6780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44260/0 (socket says 172.21.15.86:44260) 2024-04-21T08:40:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.756+0000 7f380f7fe640 1 -- 172.21.15.86:0/2876705723 learned_addr learned my addr 172.21.15.86:0/2876705723 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.756+0000 7f3807fff640 1 --2- 172.21.15.86:0/2876705723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38101072a0 0x7f3810074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.756+0000 7f380effd640 1 --2- 172.21.15.86:0/2876705723 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38101021d0 0x7f3810074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.756+0000 7f380f7fe640 1 -- 172.21.15.86:0/2876705723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38101072a0 msgr2=0x7f3810074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.756+0000 7f380f7fe640 1 --2- 172.21.15.86:0/2876705723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38101072a0 0x7f3810074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.756+0000 7f380f7fe640 1 -- 172.21.15.86:0/2876705723 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38101021d0 msgr2=0x7f3810074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.756+0000 7f380f7fe640 1 --2- 172.21.15.86:0/2876705723 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38101021d0 0x7f3810074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.758 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.756+0000 7f380f7fe640 1 -- 172.21.15.86:0/2876705723 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38101a6e80 con 0x7f3810074cd0 2024-04-21T08:40:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.756+0000 7f380effd640 1 --2- 172.21.15.86:0/2876705723 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38101021d0 0x7f3810074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.756+0000 7f380f7fe640 1 --2- 172.21.15.86:0/2876705723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3810074cd0 0x7f38101a6780 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f3808005e30 tx=0x7f38080346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.757+0000 7f380cff9640 1 -- 172.21.15.86:0/2876705723 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808041020 con 0x7f3810074cd0 2024-04-21T08:40:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.757+0000 7f380cff9640 1 -- 172.21.15.86:0/2876705723 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3808032970 con 0x7f3810074cd0 2024-04-21T08:40:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.757+0000 7f3815474640 1 -- 172.21.15.86:0/2876705723 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f38101a7110 con 0x7f3810074cd0 2024-04-21T08:40:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.757+0000 7f380cff9640 1 -- 172.21.15.86:0/2876705723 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808039040 con 0x7f3810074cd0 2024-04-21T08:40:12.759 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.757+0000 7f3815474640 1 -- 172.21.15.86:0/2876705723 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38101a7650 con 0x7f3810074cd0 2024-04-21T08:40:12.766 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.759+0000 7f380cff9640 1 -- 172.21.15.86:0/2876705723 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 25) ==== 138879+0+0 (secure 0 0 0) 0x7f3808044020 con 0x7f3810074cd0 2024-04-21T08:40:12.766 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.759+0000 7f381806e740 1 Processor -- start 2024-04-21T08:40:12.766 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.759+0000 7f381806e740 1 -- start start 2024-04-21T08:40:12.766 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.759+0000 7f380cff9640 1 --2- 172.21.15.86:0/2876705723 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f37fc0a3f90 0x7f37fc0a6450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:12.766 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.760+0000 7f3807fff640 1 --2- 172.21.15.86:0/2876705723 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f37fc0a3f90 0x7f37fc0a6450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:12.766 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.760+0000 7f380cff9640 1 -- 172.21.15.86:0/2876705723 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f38080f28f0 con 0x7f3810074cd0 2024-04-21T08:40:12.766 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.760+0000 7f381806e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608eb4e38d0 0x5608eb4e2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:12.766 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.760+0000 7f381806e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608eb4e2b60 0x5608eb4e2fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:12.766 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.760+0000 7f381806e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608eb4ec0e0 0x5608eb4ee4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:12.766 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.760+0000 7f381806e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5608eb4478d0 con 0x5608eb4e38d0 2024-04-21T08:40:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.760+0000 7f381806e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5608eb3e4e80 con 0x5608eb4ec0e0 2024-04-21T08:40:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.760+0000 7f381806e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5608eb443890 con 0x5608eb4e2b60 2024-04-21T08:40:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.760+0000 7f380effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608eb4e38d0 0x5608eb4e2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.760+0000 7f380effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608eb4e38d0 0x5608eb4e2620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44276/0 (socket says 172.21.15.86:44276) 2024-04-21T08:40:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.760+0000 7f380effd640 1 -- 172.21.15.86:0/3833876006 learned_addr learned my addr 172.21.15.86:0/3833876006 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.760+0000 7f3807fff640 1 --2- 172.21.15.86:0/2876705723 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f37fc0a3f90 0x7f37fc0a6450 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3810075ce0 tx=0x7f37f8005f90 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.760+0000 7f380f7fe640 1 --2- 172.21.15.86:0/3833876006 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608eb4ec0e0 0x5608eb4ee4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.760+0000 7f380effd640 1 -- 172.21.15.86:0/3833876006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608eb4e2b60 msgr2=0x5608eb4e2fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.760+0000 7f380effd640 1 --2- 172.21.15.86:0/3833876006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608eb4e2b60 0x5608eb4e2fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.760+0000 7f380effd640 1 -- 172.21.15.86:0/3833876006 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608eb4ec0e0 msgr2=0x5608eb4ee4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.760+0000 7f380effd640 1 --2- 172.21.15.86:0/3833876006 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608eb4ec0e0 0x5608eb4ee4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.760+0000 7f380effd640 1 -- 172.21.15.86:0/3833876006 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608eb406320 con 0x5608eb4e38d0 2024-04-21T08:40:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.761+0000 7f380effd640 1 --2- 172.21.15.86:0/3833876006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608eb4e38d0 0x5608eb4e2620 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f3800008c60 tx=0x7f380000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32bc79bb9cc6da0a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:12.767 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.761+0000 7f38057fa640 1 -- 172.21.15.86:0/3833876006 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3800014070 con 0x5608eb4e38d0 2024-04-21T08:40:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.761+0000 7f38057fa640 1 -- 172.21.15.86:0/3833876006 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38000047a0 con 0x5608eb4e38d0 2024-04-21T08:40:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.761+0000 7f38057fa640 1 -- 172.21.15.86:0/3833876006 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3800010690 con 0x5608eb4e38d0 2024-04-21T08:40:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.761+0000 7f381806e740 1 -- 172.21.15.86:0/3833876006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608eb4e38d0 msgr2=0x5608eb4e2620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.761+0000 7f381806e740 1 --2- 172.21.15.86:0/3833876006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608eb4e38d0 0x5608eb4e2620 secure :-1 s=READY pgs=759 cs=0 l=1 rev1=1 crypto rx=0x7f3800008c60 tx=0x7f380000e720 comp rx=0 tx=0).stop 2024-04-21T08:40:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.762+0000 7f381806e740 1 -- 172.21.15.86:0/3833876006 shutdown_connections 2024-04-21T08:40:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.762+0000 7f381806e740 1 --2- 172.21.15.86:0/3833876006 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608eb4ec0e0 0x5608eb4ee4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.762+0000 7f381806e740 1 --2- 172.21.15.86:0/3833876006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608eb4e2b60 0x5608eb4e2fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.762+0000 7f381806e740 1 --2- 172.21.15.86:0/3833876006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608eb4e38d0 0x5608eb4e2620 unknown :-1 s=CLOSED pgs=759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.762+0000 7f381806e740 1 -- 172.21.15.86:0/3833876006 >> 172.21.15.86:0/3833876006 conn(0x5608eb4a9c90 msgr2=0x5608eb4dd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.762+0000 7f381806e740 1 -- 172.21.15.86:0/3833876006 shutdown_connections 2024-04-21T08:40:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.762+0000 7f381806e740 1 -- 172.21.15.86:0/3833876006 wait complete. 2024-04-21T08:40:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.762+0000 7f381806e740 1 -- start start 2024-04-21T08:40:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.762+0000 7f381806e740 1 Processor -- start 2024-04-21T08:40:12.768 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.762+0000 7f381806e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608eb4e2b60 0x5608eb4eba20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.762+0000 7f381806e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608eb4e38d0 0x5608eb4e8e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.762+0000 7f381806e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608eb4ec0e0 0x5608eb4e93c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.762+0000 7f381806e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5608eb4eee10 con 0x5608eb4ec0e0 2024-04-21T08:40:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.762+0000 7f381806e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5608eb3cfeb0 con 0x5608eb4e38d0 2024-04-21T08:40:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.762+0000 7f381806e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5608eb4eef90 con 0x5608eb4e2b60 2024-04-21T08:40:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.762+0000 7f381806e740 10 client.0 _ll_register_callbacks cb 0x5608eb3fcb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.762+0000 7f380f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608eb4ec0e0 0x5608eb4e93c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.762+0000 7f380f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608eb4ec0e0 0x5608eb4e93c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44290/0 (socket says 172.21.15.86:44290) 2024-04-21T08:40:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.762+0000 7f380f7fe640 1 -- 172.21.15.86:0/3226854410 learned_addr learned my addr 172.21.15.86:0/3226854410 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.762+0000 7f3807fff640 1 --2- 172.21.15.86:0/3226854410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608eb4e38d0 0x5608eb4e8e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.762+0000 7f380effd640 1 --2- 172.21.15.86:0/3226854410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608eb4e2b60 0x5608eb4eba20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:12.769 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.763+0000 7f380f7fe640 1 -- 172.21.15.86:0/3226854410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608eb4e2b60 msgr2=0x5608eb4eba20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.763+0000 7f380f7fe640 1 --2- 172.21.15.86:0/3226854410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608eb4e2b60 0x5608eb4eba20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.763+0000 7f380f7fe640 1 -- 172.21.15.86:0/3226854410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608eb4e38d0 msgr2=0x5608eb4e8e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.763+0000 7f380f7fe640 1 --2- 172.21.15.86:0/3226854410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608eb4e38d0 0x5608eb4e8e20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.763+0000 7f380f7fe640 1 -- 172.21.15.86:0/3226854410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608eb15a210 con 0x5608eb4ec0e0 2024-04-21T08:40:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.763+0000 7f380effd640 1 --2- 172.21.15.86:0/3226854410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608eb4e2b60 0x5608eb4eba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.763+0000 7f380f7fe640 1 --2- 172.21.15.86:0/3226854410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608eb4ec0e0 0x5608eb4e93c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f3808009dc0 tx=0x7f38080be1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ffa4a02523b1bdd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.763+0000 7f37f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:40:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.763+0000 7f37f67fc640 1 -- 172.21.15.86:0/3226854410 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38080d7060 con 0x5608eb4ec0e0 2024-04-21T08:40:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.763+0000 7f37f67fc640 1 -- 172.21.15.86:0/3226854410 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f38080c6dc0 con 0x5608eb4ec0e0 2024-04-21T08:40:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.763+0000 7f381806e740 1 -- 172.21.15.86:0/3226854410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38000047a0 con 0x5608eb4ec0e0 2024-04-21T08:40:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.763+0000 7f37f67fc640 1 -- 172.21.15.86:0/3226854410 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f38080348f0 con 0x5608eb4ec0e0 2024-04-21T08:40:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.763+0000 7f381806e740 10 client.16440 fetch_fsmap learned FSMap version 29 2024-04-21T08:40:12.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.763+0000 7f381806e740 1 -- 172.21.15.86:0/3226854410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5608eb226620 con 0x5608eb4ec0e0 2024-04-21T08:40:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.763+0000 7f37f67fc640 1 -- 172.21.15.86:0/3226854410 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38000047a0 con 0x5608eb4ec0e0 2024-04-21T08:40:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.763+0000 7f37f67fc640 1 -- 172.21.15.86:0/3226854410 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f380803c030 con 0x5608eb4ec0e0 2024-04-21T08:40:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.763+0000 7f381806e740 10 client.16440 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:40:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.763+0000 7f381806e740 10 client.16440 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.763+0000 7f381806e740 20 client.16440 populate_metadata read hostname 'smithi086' 2024-04-21T08:40:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.763+0000 7f381806e740 1 --2- 172.21.15.86:0/3226854410 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5608eb4fc090 0x5608eb4fe4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.764+0000 7f381806e740 4 client.16440 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:40:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.764+0000 7f381806e740 1 -- 172.21.15.86:0/3226854410 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608eb45b880 con 0x5608eb4fc090 2024-04-21T08:40:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.764+0000 7f380effd640 1 --2- 172.21.15.86:0/3226854410 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5608eb4fc090 0x5608eb4fe4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.764+0000 7f380effd640 1 --2- 172.21.15.86:0/3226854410 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5608eb4fc090 0x5608eb4fe4e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3fa3800d8fcaf2a0 server_cookie=f3e742ebd81be88a in_seq=0 out_seq=0 2024-04-21T08:40:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.764+0000 7f37f67fc640 10 client.16440 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:40:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.765+0000 7f37f67fc640 1 -- 172.21.15.86:0/3226854410 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5608eb45b880 con 0x5608eb4fc090 2024-04-21T08:40:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.765+0000 7f37f67fc640 10 client.16440 handle_command_reply: tid=0 2024-04-21T08:40:12.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.765+0000 7f381806e740 1 client.16440 shutdown 2024-04-21T08:40:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.765+0000 7f381806e740 1 -- 172.21.15.86:0/3226854410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608eb4ec0e0 msgr2=0x5608eb4e93c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.765+0000 7f381806e740 1 --2- 172.21.15.86:0/3226854410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608eb4ec0e0 0x5608eb4e93c0 secure :-1 s=READY pgs=760 cs=0 l=1 rev1=1 crypto rx=0x7f3808009dc0 tx=0x7f38080be1b0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.765+0000 7f381806e740 1 -- 172.21.15.86:0/3226854410 shutdown_connections 2024-04-21T08:40:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.765+0000 7f381806e740 1 --2- 172.21.15.86:0/3226854410 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5608eb4fc090 0x5608eb4fe4e0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.765+0000 7f381806e740 1 --2- 172.21.15.86:0/3226854410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608eb4ec0e0 0x5608eb4e93c0 unknown :-1 s=CLOSED pgs=760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.765+0000 7f381806e740 1 --2- 172.21.15.86:0/3226854410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608eb4e38d0 0x5608eb4e8e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.765+0000 7f37f67fc640 0 client.16440 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:40:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.765+0000 7f381806e740 1 --2- 172.21.15.86:0/3226854410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608eb4e2b60 0x5608eb4eba20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.765+0000 7f381806e740 1 -- 172.21.15.86:0/3226854410 >> 172.21.15.86:0/3226854410 conn(0x5608eb4a9c90 msgr2=0x5608eb4df5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.765+0000 7f381806e740 1 -- 172.21.15.86:0/3226854410 shutdown_connections 2024-04-21T08:40:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.765+0000 7f381806e740 1 -- 172.21.15.86:0/3226854410 wait complete. 2024-04-21T08:40:12.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.765+0000 7f381806e740 20 client.16440 trim_cache size 0 max 16384 2024-04-21T08:40:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.780+0000 7f381806e740 1 Processor -- start 2024-04-21T08:40:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.780+0000 7f381806e740 1 -- start start 2024-04-21T08:40:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.780+0000 7f381806e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608eb4e2b60 0x5608eb4f7d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.780+0000 7f381806e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608eb4e38d0 0x5608eb503df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.780+0000 7f381806e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608eb4ec0e0 0x5608eb504330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.780+0000 7f381806e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f38101b13c0 con 0x5608eb4ec0e0 2024-04-21T08:40:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.780+0000 7f381806e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f38101b2f00 con 0x5608eb4e38d0 2024-04-21T08:40:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.780+0000 7f381806e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f38101b1a90 con 0x5608eb4e2b60 2024-04-21T08:40:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.780+0000 7f380f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608eb4ec0e0 0x5608eb504330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:12.784 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.780+0000 7f380f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608eb4ec0e0 0x5608eb504330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44298/0 (socket says 172.21.15.86:44298) 2024-04-21T08:40:12.785 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.780+0000 7f380f7fe640 1 -- 172.21.15.86:0/214042431 learned_addr learned my addr 172.21.15.86:0/214042431 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:12.785 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.780+0000 7f3807fff640 1 --2- 172.21.15.86:0/214042431 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608eb4e38d0 0x5608eb503df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:12.785 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.781+0000 7f380effd640 1 --2- 172.21.15.86:0/214042431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608eb4e2b60 0x5608eb4f7d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:12.785 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.781+0000 7f380f7fe640 1 -- 172.21.15.86:0/214042431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608eb4e2b60 msgr2=0x5608eb4f7d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:12.785 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.781+0000 7f380f7fe640 1 --2- 172.21.15.86:0/214042431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608eb4e2b60 0x5608eb4f7d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.785 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.781+0000 7f380f7fe640 1 -- 172.21.15.86:0/214042431 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608eb4e38d0 msgr2=0x5608eb503df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:12.785 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.781+0000 7f380f7fe640 1 --2- 172.21.15.86:0/214042431 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608eb4e38d0 0x5608eb503df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.785 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.781+0000 7f380f7fe640 1 -- 172.21.15.86:0/214042431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5608eb4d8f30 con 0x5608eb4ec0e0 2024-04-21T08:40:12.785 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.781+0000 7f380effd640 1 --2- 172.21.15.86:0/214042431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608eb4e2b60 0x5608eb4f7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:40:12.785 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.781+0000 7f3807fff640 1 --2- 172.21.15.86:0/214042431 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608eb4e38d0 0x5608eb503df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:12.785 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.781+0000 7f380f7fe640 1 --2- 172.21.15.86:0/214042431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608eb4ec0e0 0x5608eb504330 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f3808009430 tx=0x7f38080dd3b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25be420c44cbe57b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:12.785 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.781+0000 7f3804ff9640 1 -- 172.21.15.86:0/214042431 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38080bdf20 con 0x5608eb4ec0e0 2024-04-21T08:40:12.785 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.781+0000 7f3804ff9640 1 -- 172.21.15.86:0/214042431 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3808034e50 con 0x5608eb4ec0e0 2024-04-21T08:40:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.781+0000 7f3804ff9640 1 -- 172.21.15.86:0/214042431 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3808003e30 con 0x5608eb4ec0e0 2024-04-21T08:40:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.782+0000 7f381806e740 1 -- 172.21.15.86:0/214042431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608eb4ec0e0 msgr2=0x5608eb504330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.782+0000 7f381806e740 1 --2- 172.21.15.86:0/214042431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608eb4ec0e0 0x5608eb504330 secure :-1 s=READY pgs=761 cs=0 l=1 rev1=1 crypto rx=0x7f3808009430 tx=0x7f38080dd3b0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.782+0000 7f381806e740 1 -- 172.21.15.86:0/214042431 shutdown_connections 2024-04-21T08:40:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.782+0000 7f381806e740 1 --2- 172.21.15.86:0/214042431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608eb4ec0e0 0x5608eb504330 unknown :-1 s=CLOSED pgs=761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.782+0000 7f381806e740 1 --2- 172.21.15.86:0/214042431 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608eb4e38d0 0x5608eb503df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.782+0000 7f381806e740 1 --2- 172.21.15.86:0/214042431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608eb4e2b60 0x5608eb4f7d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.782+0000 7f381806e740 1 -- 172.21.15.86:0/214042431 >> 172.21.15.86:0/214042431 conn(0x5608eb4a9c90 msgr2=0x5608eb4e76f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.782+0000 7f381806e740 1 -- 172.21.15.86:0/214042431 shutdown_connections 2024-04-21T08:40:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.782+0000 7f381806e740 1 -- 172.21.15.86:0/214042431 wait complete. 2024-04-21T08:40:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.782+0000 7f381806e740 1 -- start start 2024-04-21T08:40:12.786 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.782+0000 7f381806e740 1 Processor -- start 2024-04-21T08:40:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.782+0000 7f381806e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608eb4e2b60 0x5608eb4e5b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.782+0000 7f381806e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608eb4e38d0 0x5608eb4e60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.782+0000 7f381806e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608eb4ec0e0 0x5608eb4e6600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.782+0000 7f381806e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f38101b2830 con 0x5608eb4ec0e0 2024-04-21T08:40:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.782+0000 7f381806e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f38101b2160 con 0x5608eb4e2b60 2024-04-21T08:40:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.782+0000 7f381806e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f38101af380 con 0x5608eb4e38d0 2024-04-21T08:40:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.782+0000 7f381806e740 10 client.0 _ll_register_callbacks cb 0x7f38080b8f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.782+0000 7f380f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608eb4ec0e0 0x5608eb4e6600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.782+0000 7f380f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608eb4ec0e0 0x5608eb4e6600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44304/0 (socket says 172.21.15.86:44304) 2024-04-21T08:40:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.782+0000 7f380f7fe640 1 -- 172.21.15.86:0/2708270136 learned_addr learned my addr 172.21.15.86:0/2708270136 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.783+0000 7f3807fff640 1 --2- 172.21.15.86:0/2708270136 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608eb4e38d0 0x5608eb4e60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.783+0000 7f380effd640 1 --2- 172.21.15.86:0/2708270136 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608eb4e2b60 0x5608eb4e5b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.783+0000 7f380f7fe640 1 -- 172.21.15.86:0/2708270136 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608eb4e38d0 msgr2=0x5608eb4e60c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:12.787 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.783+0000 7f380f7fe640 1 --2- 172.21.15.86:0/2708270136 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608eb4e38d0 0x5608eb4e60c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.783+0000 7f380f7fe640 1 -- 172.21.15.86:0/2708270136 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608eb4e2b60 msgr2=0x5608eb4e5b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:12.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.783+0000 7f380f7fe640 1 --2- 172.21.15.86:0/2708270136 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608eb4e2b60 0x5608eb4e5b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.783+0000 7f380f7fe640 1 -- 172.21.15.86:0/2708270136 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5608eb561710 con 0x5608eb4ec0e0 2024-04-21T08:40:12.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.783+0000 7f3807fff640 1 --2- 172.21.15.86:0/2708270136 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608eb4e38d0 0x5608eb4e60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:40:12.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.783+0000 7f380effd640 1 --2- 172.21.15.86:0/2708270136 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608eb4e2b60 0x5608eb4e5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:40:12.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.783+0000 7f380f7fe640 1 --2- 172.21.15.86:0/2708270136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608eb4ec0e0 0x5608eb4e6600 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f38080c6b20 tx=0x7f38080d7d90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41745006f093bf1d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:12.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.783+0000 7f37f67fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:40:12.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.783+0000 7f37f67fc640 1 -- 172.21.15.86:0/2708270136 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38080bdf20 con 0x5608eb4ec0e0 2024-04-21T08:40:12.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.783+0000 7f37f67fc640 1 -- 172.21.15.86:0/2708270136 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f3808004b40 con 0x5608eb4ec0e0 2024-04-21T08:40:12.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.783+0000 7f381806e740 1 -- 172.21.15.86:0/2708270136 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3808034e50 con 0x5608eb4ec0e0 2024-04-21T08:40:12.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.783+0000 7f37f67fc640 1 -- 172.21.15.86:0/2708270136 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3808003e30 con 0x5608eb4ec0e0 2024-04-21T08:40:12.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.783+0000 7f381806e740 10 client.16446 fetch_fsmap learned FSMap version 29 2024-04-21T08:40:12.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.783+0000 7f381806e740 1 -- 172.21.15.86:0/2708270136 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5608eb561710 con 0x5608eb4ec0e0 2024-04-21T08:40:12.788 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.783+0000 7f37f67fc640 1 -- 172.21.15.86:0/2708270136 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3808034e50 con 0x5608eb4ec0e0 2024-04-21T08:40:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.783+0000 7f37f67fc640 1 -- 172.21.15.86:0/2708270136 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f38080bd830 con 0x5608eb4ec0e0 2024-04-21T08:40:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.784+0000 7f381806e740 10 client.16446 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:40:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.784+0000 7f381806e740 10 client.16446 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.784+0000 7f381806e740 20 client.16446 populate_metadata read hostname 'smithi086' 2024-04-21T08:40:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.784+0000 7f381806e740 1 --2- 172.21.15.86:0/2708270136 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5608eb562c50 0x5608eb565040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.784+0000 7f381806e740 4 client.16446 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:40:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.784+0000 7f381806e740 1 -- 172.21.15.86:0/2708270136 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5608eb4e2850 con 0x5608eb562c50 2024-04-21T08:40:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.784+0000 7f380effd640 1 --2- 172.21.15.86:0/2708270136 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5608eb562c50 0x5608eb565040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.784+0000 7f37f67fc640 10 client.16446 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:40:12.789 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.784+0000 7f380effd640 1 --2- 172.21.15.86:0/2708270136 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5608eb562c50 0x5608eb565040 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d258547300c2aa7e server_cookie=c2eff0e3a1a364b5 in_seq=0 out_seq=0 2024-04-21T08:40:12.797 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.798+0000 7f37f67fc640 1 -- 172.21.15.86:0/2708270136 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5608eb4e2850 con 0x5608eb562c50 2024-04-21T08:40:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.798+0000 7f37f67fc640 10 client.16446 handle_command_reply: tid=0 2024-04-21T08:40:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.798+0000 7f381806e740 1 client.16446 shutdown 2024-04-21T08:40:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.798+0000 7f381806e740 1 -- 172.21.15.86:0/2708270136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608eb4ec0e0 msgr2=0x5608eb4e6600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.798+0000 7f381806e740 1 --2- 172.21.15.86:0/2708270136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608eb4ec0e0 0x5608eb4e6600 secure :-1 s=READY pgs=762 cs=0 l=1 rev1=1 crypto rx=0x7f38080c6b20 tx=0x7f38080d7d90 comp rx=0 tx=0).stop 2024-04-21T08:40:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.798+0000 7f381806e740 1 -- 172.21.15.86:0/2708270136 shutdown_connections 2024-04-21T08:40:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.798+0000 7f381806e740 1 --2- 172.21.15.86:0/2708270136 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5608eb562c50 0x5608eb565040 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.798+0000 7f381806e740 1 --2- 172.21.15.86:0/2708270136 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608eb4ec0e0 0x5608eb4e6600 unknown :-1 s=CLOSED pgs=762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.798+0000 7f381806e740 1 --2- 172.21.15.86:0/2708270136 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5608eb4e38d0 0x5608eb4e60c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.798+0000 7f381806e740 1 --2- 172.21.15.86:0/2708270136 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608eb4e2b60 0x5608eb4e5b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.798+0000 7f37f67fc640 0 client.16446 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:40:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.798+0000 7f381806e740 1 -- 172.21.15.86:0/2708270136 >> 172.21.15.86:0/2708270136 conn(0x5608eb4a9c90 msgr2=0x5608eb4dc040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:12.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.798+0000 7f381806e740 1 -- 172.21.15.86:0/2708270136 shutdown_connections 2024-04-21T08:40:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.798+0000 7f381806e740 1 -- 172.21.15.86:0/2708270136 wait complete. 2024-04-21T08:40:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.798+0000 7f381806e740 20 client.16446 trim_cache size 0 max 16384 2024-04-21T08:40:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:40:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:40:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:40:12.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:40:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.799+0000 7f3815474640 1 -- 172.21.15.86:0/2876705723 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f37fc0a3f90 msgr2=0x7f37fc0a6450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.799+0000 7f3815474640 1 --2- 172.21.15.86:0/2876705723 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f37fc0a3f90 0x7f37fc0a6450 secure :-1 s=READY pgs=106 cs=0 l=1 rev1=1 crypto rx=0x7f3810075ce0 tx=0x7f37f8005f90 comp rx=0 tx=0).stop 2024-04-21T08:40:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.799+0000 7f3815474640 1 -- 172.21.15.86:0/2876705723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3810074cd0 msgr2=0x7f38101a6780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.799+0000 7f3815474640 1 --2- 172.21.15.86:0/2876705723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3810074cd0 0x7f38101a6780 secure :-1 s=READY pgs=758 cs=0 l=1 rev1=1 crypto rx=0x7f3808005e30 tx=0x7f38080346d0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.799+0000 7f3815474640 1 -- 172.21.15.86:0/2876705723 shutdown_connections 2024-04-21T08:40:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.799+0000 7f3815474640 1 --2- 172.21.15.86:0/2876705723 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f37fc0a3f90 0x7f37fc0a6450 unknown :-1 s=CLOSED pgs=106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.799+0000 7f3815474640 1 --2- 172.21.15.86:0/2876705723 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3810074cd0 0x7f38101a6780 unknown :-1 s=CLOSED pgs=758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.799+0000 7f3815474640 1 --2- 172.21.15.86:0/2876705723 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38101072a0 0x7f3810074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.799+0000 7f3815474640 1 --2- 172.21.15.86:0/2876705723 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38101021d0 0x7f3810074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.799+0000 7f3815474640 1 -- 172.21.15.86:0/2876705723 >> 172.21.15.86:0/2876705723 conn(0x7f38100793e0 msgr2=0x7f38100ffbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.799+0000 7f3815474640 1 -- 172.21.15.86:0/2876705723 shutdown_connections 2024-04-21T08:40:12.801 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:12.799+0000 7f3815474640 1 -- 172.21.15.86:0/2876705723 wait complete. 2024-04-21T08:40:12.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:12 smithi086 ceph-mon[27939]: pgmap v45: 129 pgs: 129 active+clean; 1.7 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 40 KiB/s wr, 31 op/s 2024-04-21T08:40:14.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:14 smithi088 ceph-mon[31259]: pgmap v46: 129 pgs: 129 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 47 KiB/s wr, 37 op/s 2024-04-21T08:40:14.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:14 smithi179 ceph-mon[31829]: pgmap v46: 129 pgs: 129 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 47 KiB/s wr, 37 op/s 2024-04-21T08:40:14.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:14 smithi086 ceph-mon[27939]: pgmap v46: 129 pgs: 129 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 47 KiB/s wr, 37 op/s 2024-04-21T08:40:16.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:16 smithi088 ceph-mon[31259]: pgmap v47: 129 pgs: 129 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-04-21T08:40:16.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:16 smithi179 ceph-mon[31829]: pgmap v47: 129 pgs: 129 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-04-21T08:40:16.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:16 smithi086 ceph-mon[27939]: pgmap v47: 129 pgs: 129 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-04-21T08:40:18.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:18 smithi088 ceph-mon[31259]: pgmap v48: 129 pgs: 129 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-04-21T08:40:18.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:18 smithi179 ceph-mon[31829]: pgmap v48: 129 pgs: 129 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-04-21T08:40:18.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:18 smithi086 ceph-mon[27939]: pgmap v48: 129 pgs: 129 active+clean; 1.8 MiB data, 383 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 15 KiB/s wr, 12 op/s 2024-04-21T08:40:20.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:20 smithi088 ceph-mon[31259]: pgmap v49: 129 pgs: 129 active+clean; 1.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 22 op/s 2024-04-21T08:40:20.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:20 smithi088 ceph-mon[31259]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-21T08:40:20.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:20 smithi179 ceph-mon[31829]: pgmap v49: 129 pgs: 129 active+clean; 1.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 22 op/s 2024-04-21T08:40:20.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:20 smithi179 ceph-mon[31829]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-21T08:40:20.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:20 smithi086 ceph-mon[27939]: pgmap v49: 129 pgs: 129 active+clean; 1.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 22 op/s 2024-04-21T08:40:20.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:20 smithi086 ceph-mon[27939]: mgrmap e26: x(active, since 92s), standbys: z, y 2024-04-21T08:40:22.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:22 smithi179 ceph-mon[31829]: pgmap v50: 129 pgs: 129 active+clean; 1.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-21T08:40:22.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:22 smithi086 ceph-mon[27939]: pgmap v50: 129 pgs: 129 active+clean; 1.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-21T08:40:22.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:22 smithi088 ceph-mon[31259]: pgmap v50: 129 pgs: 129 active+clean; 1.9 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 19 KiB/s wr, 16 op/s 2024-04-21T08:40:24.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:24 smithi179 ceph-mon[31829]: pgmap v51: 129 pgs: 129 active+clean; 2.0 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 22 op/s 2024-04-21T08:40:24.754 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:24 smithi088 ceph-mon[31259]: pgmap v51: 129 pgs: 129 active+clean; 2.0 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 22 op/s 2024-04-21T08:40:24.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:24 smithi086 ceph-mon[27939]: pgmap v51: 129 pgs: 129 active+clean; 2.0 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 28 KiB/s wr, 22 op/s 2024-04-21T08:40:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.896+0000 7f6dd62c0640 1 -- 172.21.15.86:0/4213470975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6dd01010f0 msgr2=0x7f6dd01014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.896+0000 7f6dd62c0640 1 --2- 172.21.15.86:0/4213470975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6dd01010f0 0x7f6dd01014d0 secure :-1 s=READY pgs=763 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4009900 tx=0x7f6dc402bc30 comp rx=0 tx=0).stop 2024-04-21T08:40:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.896+0000 7f6dd62c0640 1 -- 172.21.15.86:0/4213470975 shutdown_connections 2024-04-21T08:40:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.896+0000 7f6dd62c0640 1 --2- 172.21.15.86:0/4213470975 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6dd0106170 0x7f6dd010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.896+0000 7f6dd62c0640 1 --2- 172.21.15.86:0/4213470975 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6dd0101aa0 0x7f6dd0105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.896+0000 7f6dd62c0640 1 --2- 172.21.15.86:0/4213470975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6dd01010f0 0x7f6dd01014d0 unknown :-1 s=CLOSED pgs=763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.896+0000 7f6dd62c0640 1 -- 172.21.15.86:0/4213470975 >> 172.21.15.86:0/4213470975 conn(0x7f6dd0077fd0 msgr2=0x7f6dd00783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.896+0000 7f6dd62c0640 1 -- 172.21.15.86:0/4213470975 shutdown_connections 2024-04-21T08:40:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.896+0000 7f6dd62c0640 1 -- 172.21.15.86:0/4213470975 wait complete. 2024-04-21T08:40:25.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.897+0000 7f6dd62c0640 1 Processor -- start 2024-04-21T08:40:25.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.897+0000 7f6dd62c0640 1 -- start start 2024-04-21T08:40:25.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.897+0000 7f6dd62c0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6dd01010f0 0x7f6dd01986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.897+0000 7f6dd62c0640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6dd0106170 0x7f6dd0198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.897+0000 7f6dd62c0640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6dd019cf80 0x7f6dd019d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.897+0000 7f6dd62c0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd01110d0 con 0x7f6dd01010f0 2024-04-21T08:40:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.897+0000 7f6dd62c0640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd0110f50 con 0x7f6dd0106170 2024-04-21T08:40:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.897+0000 7f6dd62c0640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd0111250 con 0x7f6dd019cf80 2024-04-21T08:40:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.897+0000 7f6dcffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6dd01010f0 0x7f6dd01986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.897+0000 7f6dcffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6dd01010f0 0x7f6dd01986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35908/0 (socket says 172.21.15.86:35908) 2024-04-21T08:40:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.897+0000 7f6dcffff640 1 -- 172.21.15.86:0/719532738 learned_addr learned my addr 172.21.15.86:0/719532738 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.897+0000 7f6dcf7fe640 1 --2- 172.21.15.86:0/719532738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6dd0106170 0x7f6dd0198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.898+0000 7f6dd485d640 1 --2- 172.21.15.86:0/719532738 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6dd019cf80 0x7f6dd019d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.898+0000 7f6dcffff640 1 -- 172.21.15.86:0/719532738 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6dd019cf80 msgr2=0x7f6dd019d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.898+0000 7f6dcffff640 1 --2- 172.21.15.86:0/719532738 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6dd019cf80 0x7f6dd019d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.898+0000 7f6dcffff640 1 -- 172.21.15.86:0/719532738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6dd0106170 msgr2=0x7f6dd0198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:25.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.898+0000 7f6dcffff640 1 --2- 172.21.15.86:0/719532738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6dd0106170 0x7f6dd0198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.898+0000 7f6dcffff640 1 -- 172.21.15.86:0/719532738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6dd019da70 con 0x7f6dd01010f0 2024-04-21T08:40:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.898+0000 7f6dcffff640 1 --2- 172.21.15.86:0/719532738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6dd01010f0 0x7f6dd01986c0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4005940 tx=0x7f6dc40045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.898+0000 7f6dcd7fa640 1 -- 172.21.15.86:0/719532738 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4043020 con 0x7f6dd01010f0 2024-04-21T08:40:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.898+0000 7f6dd62c0640 1 -- 172.21.15.86:0/719532738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6dd019dd00 con 0x7f6dd01010f0 2024-04-21T08:40:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.898+0000 7f6dcd7fa640 1 -- 172.21.15.86:0/719532738 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dc402caf0 con 0x7f6dd01010f0 2024-04-21T08:40:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.898+0000 7f6dcd7fa640 1 -- 172.21.15.86:0/719532738 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc4003af0 con 0x7f6dd01010f0 2024-04-21T08:40:25.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.898+0000 7f6dd62c0640 1 -- 172.21.15.86:0/719532738 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6dd01a5520 con 0x7f6dd01010f0 2024-04-21T08:40:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.901+0000 7f6dcd7fa640 1 -- 172.21.15.86:0/719532738 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6dc4046020 con 0x7f6dd01010f0 2024-04-21T08:40:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.901+0000 7f6dd8f81740 1 Processor -- start 2024-04-21T08:40:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.901+0000 7f6dd8f81740 1 -- start start 2024-04-21T08:40:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.901+0000 7f6dcd7fa640 1 --2- 172.21.15.86:0/719532738 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f6db80a3dc0 0x7f6db80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.902+0000 7f6dd8f81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608573698d0 0x560857368620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.902+0000 7f6dd8f81740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560857368b60 0x560857368fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.902+0000 7f6dd8f81740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608573720e0 0x5608573744d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.902+0000 7f6dd8f81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5608572cd8d0 con 0x5608573698d0 2024-04-21T08:40:25.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.902+0000 7f6dd8f81740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x56085726ae80 con 0x5608573720e0 2024-04-21T08:40:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.902+0000 7f6dd8f81740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5608572c9890 con 0x560857368b60 2024-04-21T08:40:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.902+0000 7f6dd485d640 1 --2- 172.21.15.86:0/719532738 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f6db80a3dc0 0x7f6db80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.902+0000 7f6dcffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608573698d0 0x560857368620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.902+0000 7f6dcd7fa640 1 -- 172.21.15.86:0/719532738 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f6dc40f2ba0 con 0x7f6dd01010f0 2024-04-21T08:40:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.902+0000 7f6dcffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608573698d0 0x560857368620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35914/0 (socket says 172.21.15.86:35914) 2024-04-21T08:40:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.902+0000 7f6dcffff640 1 -- 172.21.15.86:0/384460569 learned_addr learned my addr 172.21.15.86:0/384460569 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.902+0000 7f6dd485d640 1 --2- 172.21.15.86:0/384460569 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608573720e0 0x5608573744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.902+0000 7f6dcf7fe640 1 --2- 172.21.15.86:0/384460569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560857368b60 0x560857368fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.902+0000 7f6dcffff640 1 -- 172.21.15.86:0/384460569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560857368b60 msgr2=0x560857368fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.902+0000 7f6dcffff640 1 --2- 172.21.15.86:0/384460569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560857368b60 0x560857368fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.902+0000 7f6dcffff640 1 -- 172.21.15.86:0/384460569 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608573720e0 msgr2=0x5608573744d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.902+0000 7f6dcffff640 1 --2- 172.21.15.86:0/384460569 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608573720e0 0x5608573744d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:25.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.902+0000 7f6dcffff640 1 -- 172.21.15.86:0/384460569 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56085728c320 con 0x5608573698d0 2024-04-21T08:40:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.902+0000 7f6dcf7fe640 1 --2- 172.21.15.86:0/384460569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560857368b60 0x560857368fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.902+0000 7f6dd485d640 1 --2- 172.21.15.86:0/384460569 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608573720e0 0x5608573744d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.902+0000 7f6dd485d640 1 --2- 172.21.15.86:0/719532738 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f6db80a3dc0 0x7f6db80a6280 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8009fd0 tx=0x7f6dc8013b00 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.902+0000 7f6dcffff640 1 --2- 172.21.15.86:0/384460569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608573698d0 0x560857368620 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f6dc40c7020 tx=0x7f6dc40be770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3df1e89ab1f550a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.903+0000 7f6db67fc640 1 -- 172.21.15.86:0/384460569 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc40befe0 con 0x5608573698d0 2024-04-21T08:40:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.903+0000 7f6db67fc640 1 -- 172.21.15.86:0/384460569 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dc408e980 con 0x5608573698d0 2024-04-21T08:40:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.903+0000 7f6db67fc640 1 -- 172.21.15.86:0/384460569 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc40a9110 con 0x5608573698d0 2024-04-21T08:40:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.903+0000 7f6dd8f81740 1 -- 172.21.15.86:0/384460569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608573698d0 msgr2=0x560857368620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.903+0000 7f6dd8f81740 1 --2- 172.21.15.86:0/384460569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608573698d0 0x560857368620 secure :-1 s=READY pgs=765 cs=0 l=1 rev1=1 crypto rx=0x7f6dc40c7020 tx=0x7f6dc40be770 comp rx=0 tx=0).stop 2024-04-21T08:40:25.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.903+0000 7f6dd8f81740 1 -- 172.21.15.86:0/384460569 shutdown_connections 2024-04-21T08:40:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.903+0000 7f6dd8f81740 1 --2- 172.21.15.86:0/384460569 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608573720e0 0x5608573744d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.903+0000 7f6dd8f81740 1 --2- 172.21.15.86:0/384460569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560857368b60 0x560857368fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.903+0000 7f6dd8f81740 1 --2- 172.21.15.86:0/384460569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608573698d0 0x560857368620 unknown :-1 s=CLOSED pgs=765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.903+0000 7f6dd8f81740 1 -- 172.21.15.86:0/384460569 >> 172.21.15.86:0/384460569 conn(0x56085732fc90 msgr2=0x560857363850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.904+0000 7f6dd8f81740 1 -- 172.21.15.86:0/384460569 shutdown_connections 2024-04-21T08:40:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.904+0000 7f6dd8f81740 1 -- 172.21.15.86:0/384460569 wait complete. 2024-04-21T08:40:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.904+0000 7f6dd8f81740 1 -- start start 2024-04-21T08:40:25.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.904+0000 7f6dd8f81740 1 Processor -- start 2024-04-21T08:40:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.904+0000 7f6dd8f81740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560857368b60 0x560857371af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.904+0000 7f6dd8f81740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608573698d0 0x56085736ee80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.904+0000 7f6dd8f81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608573720e0 0x56085736f490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.904+0000 7f6dd8f81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560857378fe0 con 0x5608573720e0 2024-04-21T08:40:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.904+0000 7f6dd8f81740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x560857255eb0 con 0x5608573698d0 2024-04-21T08:40:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.904+0000 7f6dd8f81740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x560857379160 con 0x560857368b60 2024-04-21T08:40:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.904+0000 7f6dd8f81740 10 client.0 _ll_register_callbacks cb 0x560857282b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.905+0000 7f6dd485d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608573720e0 0x56085736f490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:25.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.905+0000 7f6dd485d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608573720e0 0x56085736f490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35918/0 (socket says 172.21.15.86:35918) 2024-04-21T08:40:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.905+0000 7f6dd485d640 1 -- 172.21.15.86:0/3386364547 learned_addr learned my addr 172.21.15.86:0/3386364547 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.905+0000 7f6dcf7fe640 1 --2- 172.21.15.86:0/3386364547 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608573698d0 0x56085736ee80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.905+0000 7f6dcffff640 1 --2- 172.21.15.86:0/3386364547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560857368b60 0x560857371af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.905+0000 7f6dd485d640 1 -- 172.21.15.86:0/3386364547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560857368b60 msgr2=0x560857371af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.905+0000 7f6dd485d640 1 --2- 172.21.15.86:0/3386364547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560857368b60 0x560857371af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.905+0000 7f6dd485d640 1 -- 172.21.15.86:0/3386364547 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608573698d0 msgr2=0x56085736ee80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.905+0000 7f6dd485d640 1 --2- 172.21.15.86:0/3386364547 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608573698d0 0x56085736ee80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.905+0000 7f6dd485d640 1 -- 172.21.15.86:0/3386364547 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560856fe0210 con 0x5608573720e0 2024-04-21T08:40:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.905+0000 7f6dd485d640 1 --2- 172.21.15.86:0/3386364547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608573720e0 0x56085736f490 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800fa10 tx=0x7f6dc8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7cf9147a957ca4fe server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.905+0000 7f6daf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:40:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.905+0000 7f6daf7fe640 1 -- 172.21.15.86:0/3386364547 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8024020 con 0x5608573720e0 2024-04-21T08:40:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.905+0000 7f6daf7fe640 1 -- 172.21.15.86:0/3386364547 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f6dc8004530 con 0x5608573720e0 2024-04-21T08:40:25.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.905+0000 7f6dd8f81740 1 -- 172.21.15.86:0/3386364547 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6dc408e980 con 0x5608573720e0 2024-04-21T08:40:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.906+0000 7f6daf7fe640 1 -- 172.21.15.86:0/3386364547 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6dc802f050 con 0x5608573720e0 2024-04-21T08:40:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.906+0000 7f6dd8f81740 10 client.16458 fetch_fsmap learned FSMap version 29 2024-04-21T08:40:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.906+0000 7f6dd8f81740 1 -- 172.21.15.86:0/3386364547 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5608570ac620 con 0x5608573720e0 2024-04-21T08:40:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.906+0000 7f6daf7fe640 1 -- 172.21.15.86:0/3386364547 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dc408e980 con 0x5608573720e0 2024-04-21T08:40:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.906+0000 7f6daf7fe640 1 -- 172.21.15.86:0/3386364547 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6dc801f070 con 0x5608573720e0 2024-04-21T08:40:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.906+0000 7f6dd8f81740 10 client.16458 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:40:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.906+0000 7f6dd8f81740 10 client.16458 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.906+0000 7f6dd8f81740 20 client.16458 populate_metadata read hostname 'smithi086' 2024-04-21T08:40:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.906+0000 7f6dd8f81740 1 --2- 172.21.15.86:0/3386364547 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5608573863d0 0x5608573887c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.906+0000 7f6dd8f81740 4 client.16458 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:40:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.906+0000 7f6dd8f81740 1 -- 172.21.15.86:0/3386364547 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5608572e1880 con 0x5608573863d0 2024-04-21T08:40:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.906+0000 7f6dcffff640 1 --2- 172.21.15.86:0/3386364547 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5608573863d0 0x5608573887c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.907+0000 7f6dcffff640 1 --2- 172.21.15.86:0/3386364547 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5608573863d0 0x5608573887c0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80a94e2dd3f4312b server_cookie=acbe9a05f7ec561e in_seq=0 out_seq=0 2024-04-21T08:40:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.907+0000 7f6daf7fe640 10 client.16458 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:40:25.912 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.907+0000 7f6daf7fe640 1 -- 172.21.15.86:0/3386364547 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5608572e1880 con 0x5608573863d0 2024-04-21T08:40:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.907+0000 7f6daf7fe640 10 client.16458 handle_command_reply: tid=0 2024-04-21T08:40:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.908+0000 7f6dd8f81740 1 client.16458 shutdown 2024-04-21T08:40:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.908+0000 7f6dd8f81740 1 -- 172.21.15.86:0/3386364547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608573720e0 msgr2=0x56085736f490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.908+0000 7f6dd8f81740 1 --2- 172.21.15.86:0/3386364547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608573720e0 0x56085736f490 secure :-1 s=READY pgs=766 cs=0 l=1 rev1=1 crypto rx=0x7f6dc800fa10 tx=0x7f6dc8004ab0 comp rx=0 tx=0).stop 2024-04-21T08:40:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.908+0000 7f6dd8f81740 1 -- 172.21.15.86:0/3386364547 shutdown_connections 2024-04-21T08:40:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.908+0000 7f6dd8f81740 1 --2- 172.21.15.86:0/3386364547 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5608573863d0 0x5608573887c0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.908+0000 7f6dd8f81740 1 --2- 172.21.15.86:0/3386364547 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608573720e0 0x56085736f490 unknown :-1 s=CLOSED pgs=766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.908+0000 7f6dd8f81740 1 --2- 172.21.15.86:0/3386364547 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608573698d0 0x56085736ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.908+0000 7f6daf7fe640 0 client.16458 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:40:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.908+0000 7f6dd8f81740 1 --2- 172.21.15.86:0/3386364547 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560857368b60 0x560857371af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.908+0000 7f6dd8f81740 1 -- 172.21.15.86:0/3386364547 >> 172.21.15.86:0/3386364547 conn(0x56085732fc90 msgr2=0x5608573655a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.908+0000 7f6dd8f81740 1 -- 172.21.15.86:0/3386364547 shutdown_connections 2024-04-21T08:40:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.908+0000 7f6dd8f81740 1 -- 172.21.15.86:0/3386364547 wait complete. 2024-04-21T08:40:25.913 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.908+0000 7f6dd8f81740 20 client.16458 trim_cache size 0 max 16384 2024-04-21T08:40:25.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.923+0000 7f6dd8f81740 1 Processor -- start 2024-04-21T08:40:25.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.923+0000 7f6dd8f81740 1 -- start start 2024-04-21T08:40:25.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.923+0000 7f6dd8f81740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560857368b60 0x560857381ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:25.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.923+0000 7f6dd8f81740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608573698d0 0x560857365390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:25.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.923+0000 7f6dd8f81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608573720e0 0x5608573715b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:25.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.923+0000 7f6dd8f81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd01acfd0 con 0x5608573720e0 2024-04-21T08:40:25.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.923+0000 7f6dd8f81740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd01aeb10 con 0x5608573698d0 2024-04-21T08:40:25.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.923+0000 7f6dd8f81740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd01ad6a0 con 0x560857368b60 2024-04-21T08:40:25.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.923+0000 7f6dd485d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608573720e0 0x5608573715b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:25.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.923+0000 7f6dd485d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608573720e0 0x5608573715b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35920/0 (socket says 172.21.15.86:35920) 2024-04-21T08:40:25.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.923+0000 7f6dd485d640 1 -- 172.21.15.86:0/106771861 learned_addr learned my addr 172.21.15.86:0/106771861 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:25.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.923+0000 7f6dcf7fe640 1 --2- 172.21.15.86:0/106771861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608573698d0 0x560857365390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:25.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.924+0000 7f6dcffff640 1 --2- 172.21.15.86:0/106771861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560857368b60 0x560857381ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:25.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.924+0000 7f6dd485d640 1 -- 172.21.15.86:0/106771861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560857368b60 msgr2=0x560857381ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.924+0000 7f6dd485d640 1 --2- 172.21.15.86:0/106771861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560857368b60 0x560857381ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.924+0000 7f6dd485d640 1 -- 172.21.15.86:0/106771861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608573698d0 msgr2=0x560857365390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.924+0000 7f6dd485d640 1 --2- 172.21.15.86:0/106771861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608573698d0 0x560857365390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.924+0000 7f6dd485d640 1 -- 172.21.15.86:0/106771861 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560857368990 con 0x5608573720e0 2024-04-21T08:40:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.924+0000 7f6dd485d640 1 --2- 172.21.15.86:0/106771861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608573720e0 0x5608573715b0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8003fd0 tx=0x7f6dc800f4e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e590ecfb852c6d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.924+0000 7f6db5ffb640 1 -- 172.21.15.86:0/106771861 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8024030 con 0x5608573720e0 2024-04-21T08:40:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.924+0000 7f6db5ffb640 1 -- 172.21.15.86:0/106771861 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6dc8013460 con 0x5608573720e0 2024-04-21T08:40:25.925 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.924+0000 7f6db5ffb640 1 -- 172.21.15.86:0/106771861 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8010690 con 0x5608573720e0 2024-04-21T08:40:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.925+0000 7f6dd8f81740 1 -- 172.21.15.86:0/106771861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608573720e0 msgr2=0x5608573715b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.925+0000 7f6dd8f81740 1 --2- 172.21.15.86:0/106771861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608573720e0 0x5608573715b0 secure :-1 s=READY pgs=767 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8003fd0 tx=0x7f6dc800f4e0 comp rx=0 tx=0).stop 2024-04-21T08:40:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.925+0000 7f6dd8f81740 1 -- 172.21.15.86:0/106771861 shutdown_connections 2024-04-21T08:40:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.925+0000 7f6dd8f81740 1 --2- 172.21.15.86:0/106771861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608573720e0 0x5608573715b0 unknown :-1 s=CLOSED pgs=767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.925+0000 7f6dd8f81740 1 --2- 172.21.15.86:0/106771861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608573698d0 0x560857365390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.925+0000 7f6dd8f81740 1 --2- 172.21.15.86:0/106771861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560857368b60 0x560857381ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.925+0000 7f6dd8f81740 1 -- 172.21.15.86:0/106771861 >> 172.21.15.86:0/106771861 conn(0x56085732fc90 msgr2=0x560857364c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.925+0000 7f6dd8f81740 1 -- 172.21.15.86:0/106771861 shutdown_connections 2024-04-21T08:40:25.929 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.925+0000 7f6dd8f81740 1 -- 172.21.15.86:0/106771861 wait complete. 2024-04-21T08:40:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.925+0000 7f6dd8f81740 1 -- start start 2024-04-21T08:40:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.926+0000 7f6dd8f81740 1 Processor -- start 2024-04-21T08:40:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.926+0000 7f6dd8f81740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560857368b60 0x56085737e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.926+0000 7f6dd8f81740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608573698d0 0x56085737e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.926+0000 7f6dd8f81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608573720e0 0x56085736b6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.926+0000 7f6dd8f81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd01ae440 con 0x5608573720e0 2024-04-21T08:40:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.926+0000 7f6dd8f81740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd01add70 con 0x5608573698d0 2024-04-21T08:40:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.926+0000 7f6dd8f81740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6dd01aaf90 con 0x560857368b60 2024-04-21T08:40:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.926+0000 7f6dd8f81740 10 client.0 _ll_register_callbacks cb 0x7f6dc8003970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.926+0000 7f6dd485d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608573720e0 0x56085736b6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.926+0000 7f6dd485d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608573720e0 0x56085736b6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35930/0 (socket says 172.21.15.86:35930) 2024-04-21T08:40:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.926+0000 7f6dd485d640 1 -- 172.21.15.86:0/3486270041 learned_addr learned my addr 172.21.15.86:0/3486270041 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:25.930 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.926+0000 7f6dcffff640 1 --2- 172.21.15.86:0/3486270041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560857368b60 0x56085737e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.926+0000 7f6dcf7fe640 1 --2- 172.21.15.86:0/3486270041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608573698d0 0x56085737e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.926+0000 7f6dd485d640 1 -- 172.21.15.86:0/3486270041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560857368b60 msgr2=0x56085737e3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.926+0000 7f6dd485d640 1 --2- 172.21.15.86:0/3486270041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560857368b60 0x56085737e3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.926+0000 7f6dd485d640 1 -- 172.21.15.86:0/3486270041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608573698d0 msgr2=0x56085737e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.926+0000 7f6dd485d640 1 --2- 172.21.15.86:0/3486270041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608573698d0 0x56085737e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.926+0000 7f6dd485d640 1 -- 172.21.15.86:0/3486270041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56085736bf00 con 0x5608573720e0 2024-04-21T08:40:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.926+0000 7f6dcf7fe640 1 --2- 172.21.15.86:0/3486270041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608573698d0 0x56085737e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.927+0000 7f6dd485d640 1 --2- 172.21.15.86:0/3486270041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608573720e0 0x56085736b6d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8009a90 tx=0x7f6dc8004d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6834f26d375d8da4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.927+0000 7f6daf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:40:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.927+0000 7f6daf7fe640 1 -- 172.21.15.86:0/3486270041 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6dc8024040 con 0x5608573720e0 2024-04-21T08:40:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.927+0000 7f6daf7fe640 1 -- 172.21.15.86:0/3486270041 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f6dc8010430 con 0x5608573720e0 2024-04-21T08:40:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.927+0000 7f6dd8f81740 1 -- 172.21.15.86:0/3486270041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56085726d3d0 con 0x5608573720e0 2024-04-21T08:40:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.927+0000 7f6daf7fe640 1 -- 172.21.15.86:0/3486270041 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6dc8049050 con 0x5608573720e0 2024-04-21T08:40:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.927+0000 7f6dd8f81740 10 client.16464 fetch_fsmap learned FSMap version 29 2024-04-21T08:40:25.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.927+0000 7f6dd8f81740 1 -- 172.21.15.86:0/3486270041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5608573ec510 con 0x5608573720e0 2024-04-21T08:40:25.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.927+0000 7f6daf7fe640 1 -- 172.21.15.86:0/3486270041 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56085726d3d0 con 0x5608573720e0 2024-04-21T08:40:25.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.927+0000 7f6daf7fe640 1 -- 172.21.15.86:0/3486270041 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6dc8044030 con 0x5608573720e0 2024-04-21T08:40:25.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.927+0000 7f6dd8f81740 10 client.16464 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:40:25.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.927+0000 7f6dd8f81740 10 client.16464 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:25.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.927+0000 7f6dd8f81740 20 client.16464 populate_metadata read hostname 'smithi086' 2024-04-21T08:40:25.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.927+0000 7f6dd8f81740 1 --2- 172.21.15.86:0/3486270041 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5608573ecb70 0x5608573eef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:25.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.927+0000 7f6dd8f81740 4 client.16464 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:40:25.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.927+0000 7f6dd8f81740 1 -- 172.21.15.86:0/3486270041 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5608572db2f0 con 0x5608573ecb70 2024-04-21T08:40:25.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.928+0000 7f6dcffff640 1 --2- 172.21.15.86:0/3486270041 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5608573ecb70 0x5608573eef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:25.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.928+0000 7f6dcffff640 1 --2- 172.21.15.86:0/3486270041 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5608573ecb70 0x5608573eef60 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5810b7b00311084 server_cookie=1cbc1fc692a463a7 in_seq=0 out_seq=0 2024-04-21T08:40:25.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.928+0000 7f6daf7fe640 10 client.16464 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:40:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.942+0000 7f6daf7fe640 1 -- 172.21.15.86:0/3486270041 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5608572db2f0 con 0x5608573ecb70 2024-04-21T08:40:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.942+0000 7f6daf7fe640 10 client.16464 handle_command_reply: tid=0 2024-04-21T08:40:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.942+0000 7f6dd8f81740 1 client.16464 shutdown 2024-04-21T08:40:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.942+0000 7f6dd8f81740 1 -- 172.21.15.86:0/3486270041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608573720e0 msgr2=0x56085736b6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.942+0000 7f6dd8f81740 1 --2- 172.21.15.86:0/3486270041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608573720e0 0x56085736b6d0 secure :-1 s=READY pgs=768 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8009a90 tx=0x7f6dc8004d10 comp rx=0 tx=0).stop 2024-04-21T08:40:25.942 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.943+0000 7f6dd8f81740 1 -- 172.21.15.86:0/3486270041 shutdown_connections 2024-04-21T08:40:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.943+0000 7f6dd8f81740 1 --2- 172.21.15.86:0/3486270041 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5608573ecb70 0x5608573eef60 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.943+0000 7f6dd8f81740 1 --2- 172.21.15.86:0/3486270041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5608573720e0 0x56085736b6d0 unknown :-1 s=CLOSED pgs=768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.943+0000 7f6dd8f81740 1 --2- 172.21.15.86:0/3486270041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5608573698d0 0x56085737e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.943+0000 7f6daf7fe640 0 client.16464 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:40:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.943+0000 7f6dd8f81740 1 --2- 172.21.15.86:0/3486270041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560857368b60 0x56085737e3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.943+0000 7f6dd8f81740 1 -- 172.21.15.86:0/3486270041 >> 172.21.15.86:0/3486270041 conn(0x56085732fc90 msgr2=0x5608573628d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.943+0000 7f6dd8f81740 1 -- 172.21.15.86:0/3486270041 shutdown_connections 2024-04-21T08:40:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.943+0000 7f6dd8f81740 1 -- 172.21.15.86:0/3486270041 wait complete. 2024-04-21T08:40:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.943+0000 7f6dd8f81740 20 client.16464 trim_cache size 0 max 16384 2024-04-21T08:40:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:40:25.943 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:40:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:40:25.944 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:40:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.943+0000 7f6dd62c0640 1 -- 172.21.15.86:0/719532738 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f6db80a3dc0 msgr2=0x7f6db80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.943+0000 7f6dd62c0640 1 --2- 172.21.15.86:0/719532738 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f6db80a3dc0 0x7f6db80a6280 secure :-1 s=READY pgs=107 cs=0 l=1 rev1=1 crypto rx=0x7f6dc8009fd0 tx=0x7f6dc8013b00 comp rx=0 tx=0).stop 2024-04-21T08:40:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.943+0000 7f6dd62c0640 1 -- 172.21.15.86:0/719532738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6dd01010f0 msgr2=0x7f6dd01986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.943+0000 7f6dd62c0640 1 --2- 172.21.15.86:0/719532738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6dd01010f0 0x7f6dd01986c0 secure :-1 s=READY pgs=764 cs=0 l=1 rev1=1 crypto rx=0x7f6dc4005940 tx=0x7f6dc40045e0 comp rx=0 tx=0).stop 2024-04-21T08:40:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.943+0000 7f6dd62c0640 1 -- 172.21.15.86:0/719532738 shutdown_connections 2024-04-21T08:40:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.943+0000 7f6dd62c0640 1 --2- 172.21.15.86:0/719532738 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f6db80a3dc0 0x7f6db80a6280 unknown :-1 s=CLOSED pgs=107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.943+0000 7f6dd62c0640 1 --2- 172.21.15.86:0/719532738 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6dd019cf80 0x7f6dd019d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.943+0000 7f6dd62c0640 1 --2- 172.21.15.86:0/719532738 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6dd0106170 0x7f6dd0198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.943+0000 7f6dd62c0640 1 --2- 172.21.15.86:0/719532738 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6dd01010f0 0x7f6dd01986c0 unknown :-1 s=CLOSED pgs=764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.943+0000 7f6dd62c0640 1 -- 172.21.15.86:0/719532738 >> 172.21.15.86:0/719532738 conn(0x7f6dd0077fd0 msgr2=0x7f6dd0069960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.943+0000 7f6dd62c0640 1 -- 172.21.15.86:0/719532738 shutdown_connections 2024-04-21T08:40:25.945 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:25.943+0000 7f6dd62c0640 1 -- 172.21.15.86:0/719532738 wait complete. 2024-04-21T08:40:26.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:26 smithi179 ceph-mon[31829]: pgmap v52: 129 pgs: 129 active+clean; 2.0 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-04-21T08:40:26.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:26 smithi086 ceph-mon[27939]: pgmap v52: 129 pgs: 129 active+clean; 2.0 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-04-21T08:40:26.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:26 smithi088 ceph-mon[31259]: pgmap v52: 129 pgs: 129 active+clean; 2.0 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-04-21T08:40:27.900 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:40:27.900 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:40:28.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.015+0000 7fe891869640 1 -- 172.21.15.86:0/3061888986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe88c101a40 msgr2=0x7fe88c105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.015+0000 7fe891869640 1 --2- 172.21.15.86:0/3061888986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe88c101a40 0x7fe88c105a80 secure :-1 s=READY pgs=769 cs=0 l=1 rev1=1 crypto rx=0x7fe880009900 tx=0x7fe88002bc30 comp rx=0 tx=0).stop 2024-04-21T08:40:28.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.015+0000 7fe891869640 1 -- 172.21.15.86:0/3061888986 shutdown_connections 2024-04-21T08:40:28.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.015+0000 7fe891869640 1 --2- 172.21.15.86:0/3061888986 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe88c106160 0x7fe88c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.015+0000 7fe891869640 1 --2- 172.21.15.86:0/3061888986 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe88c101a40 0x7fe88c105a80 unknown :-1 s=CLOSED pgs=769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.015+0000 7fe891869640 1 --2- 172.21.15.86:0/3061888986 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe88c101090 0x7fe88c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.015+0000 7fe891869640 1 -- 172.21.15.86:0/3061888986 >> 172.21.15.86:0/3061888986 conn(0x7fe88c0771d0 msgr2=0x7fe88c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:28.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.015+0000 7fe891869640 1 -- 172.21.15.86:0/3061888986 shutdown_connections 2024-04-21T08:40:28.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.015+0000 7fe891869640 1 -- 172.21.15.86:0/3061888986 wait complete. 2024-04-21T08:40:28.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.016+0000 7fe891869640 1 Processor -- start 2024-04-21T08:40:28.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.016+0000 7fe891869640 1 -- start start 2024-04-21T08:40:28.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.016+0000 7fe891869640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe88c101090 0x7fe88c10e6c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.016+0000 7fe891869640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe88c106160 0x7fe88c10ec00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.016+0000 7fe891869640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe88c10f140 0x7fe88c1a68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.016+0000 7fe891869640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c1121e0 con 0x7fe88c10f140 2024-04-21T08:40:28.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.016+0000 7fe891869640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c112060 con 0x7fe88c106160 2024-04-21T08:40:28.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.016+0000 7fe891869640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c112360 con 0x7fe88c101090 2024-04-21T08:40:28.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.016+0000 7fe88b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe88c10f140 0x7fe88c1a68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.016+0000 7fe88b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe88c10f140 0x7fe88c1a68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35940/0 (socket says 172.21.15.86:35940) 2024-04-21T08:40:28.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.016+0000 7fe88b7fe640 1 -- 172.21.15.86:0/3178909402 learned_addr learned my addr 172.21.15.86:0/3178909402 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:28.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.016+0000 7fe88a7fc640 1 --2- 172.21.15.86:0/3178909402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe88c106160 0x7fe88c10ec00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.016+0000 7fe88affd640 1 --2- 172.21.15.86:0/3178909402 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe88c101090 0x7fe88c10e6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.016+0000 7fe88b7fe640 1 -- 172.21.15.86:0/3178909402 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe88c101090 msgr2=0x7fe88c10e6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.016+0000 7fe88b7fe640 1 --2- 172.21.15.86:0/3178909402 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe88c101090 0x7fe88c10e6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.016+0000 7fe88b7fe640 1 -- 172.21.15.86:0/3178909402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe88c106160 msgr2=0x7fe88c10ec00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.016+0000 7fe88b7fe640 1 --2- 172.21.15.86:0/3178909402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe88c106160 0x7fe88c10ec00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.016+0000 7fe88b7fe640 1 -- 172.21.15.86:0/3178909402 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe88c1a6ff0 con 0x7fe88c10f140 2024-04-21T08:40:28.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.017+0000 7fe88b7fe640 1 --2- 172.21.15.86:0/3178909402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe88c10f140 0x7fe88c1a68f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fe88400eaa0 tx=0x7fe88400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:28.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.017+0000 7fe890867640 1 -- 172.21.15.86:0/3178909402 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe88400cdb0 con 0x7fe88c10f140 2024-04-21T08:40:28.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.017+0000 7fe890867640 1 -- 172.21.15.86:0/3178909402 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe884005020 con 0x7fe88c10f140 2024-04-21T08:40:28.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.017+0000 7fe891869640 1 -- 172.21.15.86:0/3178909402 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe88c1a72e0 con 0x7fe88c10f140 2024-04-21T08:40:28.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.017+0000 7fe890867640 1 -- 172.21.15.86:0/3178909402 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe8840116d0 con 0x7fe88c10f140 2024-04-21T08:40:28.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.017+0000 7fe891869640 1 -- 172.21.15.86:0/3178909402 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe88c1a7800 con 0x7fe88c10f140 2024-04-21T08:40:28.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.019+0000 7fe890867640 1 -- 172.21.15.86:0/3178909402 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe8840051d0 con 0x7fe88c10f140 2024-04-21T08:40:28.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.019+0000 7fe894457740 1 Processor -- start 2024-04-21T08:40:28.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.019+0000 7fe894457740 1 -- start start 2024-04-21T08:40:28.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.020+0000 7fe894457740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558242141580 0x558242141960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.020+0000 7fe894457740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558242141ea0 0x55824214a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.020+0000 7fe894457740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55824214aa80 0x55824214ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.020+0000 7fe894457740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5582420a61e0 con 0x558242141580 2024-04-21T08:40:28.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.020+0000 7fe894457740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x558242043390 con 0x558242141ea0 2024-04-21T08:40:28.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.020+0000 7fe894457740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5582420a21a0 con 0x55824214aa80 2024-04-21T08:40:28.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.020+0000 7fe890867640 1 --2- 172.21.15.86:0/3178909402 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe8740a3d50 0x7fe8740a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.020+0000 7fe88a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558242141580 0x558242141960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.020+0000 7fe88a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558242141580 0x558242141960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35944/0 (socket says 172.21.15.86:35944) 2024-04-21T08:40:28.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.020+0000 7fe88a7fc640 1 -- 172.21.15.86:0/4067400888 learned_addr learned my addr 172.21.15.86:0/4067400888 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:28.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.020+0000 7fe88b7fe640 1 --2- 172.21.15.86:0/3178909402 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe8740a3d50 0x7fe8740a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.020+0000 7fe88affd640 1 --2- 172.21.15.86:0/4067400888 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558242141ea0 0x55824214a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.020+0000 7fe88a7fc640 1 --2- 172.21.15.86:0/4067400888 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55824214aa80 0x55824214ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.020+0000 7fe890867640 1 -- 172.21.15.86:0/3178909402 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fe8840d0c00 con 0x7fe88c10f140 2024-04-21T08:40:28.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.020+0000 7fe88a7fc640 1 -- 172.21.15.86:0/4067400888 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55824214aa80 msgr2=0x55824214ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.020+0000 7fe88a7fc640 1 --2- 172.21.15.86:0/4067400888 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55824214aa80 0x55824214ce70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.020+0000 7fe88a7fc640 1 -- 172.21.15.86:0/4067400888 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558242141ea0 msgr2=0x55824214a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.020+0000 7fe88a7fc640 1 --2- 172.21.15.86:0/4067400888 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558242141ea0 0x55824214a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.020+0000 7fe88a7fc640 1 -- 172.21.15.86:0/4067400888 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558241e827d0 con 0x558242141580 2024-04-21T08:40:28.027 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.020+0000 7fe88affd640 1 --2- 172.21.15.86:0/4067400888 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558242141ea0 0x55824214a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:28.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.020+0000 7fe88a7fc640 1 --2- 172.21.15.86:0/4067400888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558242141580 0x558242141960 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fe88c06a3c0 tx=0x7fe880005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d46bcf8c9267b12 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:28.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.020+0000 7fe88b7fe640 1 --2- 172.21.15.86:0/3178909402 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe8740a3d50 0x7fe8740a6210 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe88409c5e0 tx=0x7fe884087040 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:28.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.021+0000 7fe871ffb640 1 -- 172.21.15.86:0/4067400888 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe880047020 con 0x558242141580 2024-04-21T08:40:28.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.021+0000 7fe871ffb640 1 -- 172.21.15.86:0/4067400888 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe88002cc90 con 0x558242141580 2024-04-21T08:40:28.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.021+0000 7fe871ffb640 1 -- 172.21.15.86:0/4067400888 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe880004920 con 0x558242141580 2024-04-21T08:40:28.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.021+0000 7fe894457740 1 -- 172.21.15.86:0/4067400888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558242141580 msgr2=0x558242141960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.021+0000 7fe894457740 1 --2- 172.21.15.86:0/4067400888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558242141580 0x558242141960 secure :-1 s=READY pgs=771 cs=0 l=1 rev1=1 crypto rx=0x7fe88c06a3c0 tx=0x7fe880005b70 comp rx=0 tx=0).stop 2024-04-21T08:40:28.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.021+0000 7fe894457740 1 -- 172.21.15.86:0/4067400888 shutdown_connections 2024-04-21T08:40:28.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.021+0000 7fe894457740 1 --2- 172.21.15.86:0/4067400888 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55824214aa80 0x55824214ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.021+0000 7fe894457740 1 --2- 172.21.15.86:0/4067400888 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558242141ea0 0x55824214a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.021+0000 7fe894457740 1 --2- 172.21.15.86:0/4067400888 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558242141580 0x558242141960 unknown :-1 s=CLOSED pgs=771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.021+0000 7fe894457740 1 -- 172.21.15.86:0/4067400888 >> 172.21.15.86:0/4067400888 conn(0x55824213bcb0 msgr2=0x55824213c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:28.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.021+0000 7fe894457740 1 -- 172.21.15.86:0/4067400888 shutdown_connections 2024-04-21T08:40:28.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.021+0000 7fe894457740 1 -- 172.21.15.86:0/4067400888 wait complete. 2024-04-21T08:40:28.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.022+0000 7fe894457740 1 -- start start 2024-04-21T08:40:28.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.022+0000 7fe894457740 1 Processor -- start 2024-04-21T08:40:28.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.022+0000 7fe894457740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558242141580 0x558242147b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.022+0000 7fe894457740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558242141ea0 0x558242148050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.022+0000 7fe894457740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55824214aa80 0x558242148590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.022+0000 7fe894457740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55824214d810 con 0x558242141580 2024-04-21T08:40:28.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.022+0000 7fe894457740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55824205f2f0 con 0x55824214aa80 2024-04-21T08:40:28.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.022+0000 7fe894457740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55824214d990 con 0x558242141ea0 2024-04-21T08:40:28.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.022+0000 7fe894457740 10 client.0 _ll_register_callbacks cb 0x55824206c580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:28.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.022+0000 7fe88a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558242141580 0x558242147b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.022+0000 7fe88a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558242141580 0x558242147b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35954/0 (socket says 172.21.15.86:35954) 2024-04-21T08:40:28.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.022+0000 7fe88a7fc640 1 -- 172.21.15.86:0/2591558134 learned_addr learned my addr 172.21.15.86:0/2591558134 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:28.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.022+0000 7fe88a7fc640 1 --2- 172.21.15.86:0/2591558134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55824214aa80 0x558242148590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.023+0000 7fe88affd640 1 --2- 172.21.15.86:0/2591558134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558242141ea0 0x558242148050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.023+0000 7fe88a7fc640 1 -- 172.21.15.86:0/2591558134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558242141ea0 msgr2=0x558242148050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.023+0000 7fe88a7fc640 1 --2- 172.21.15.86:0/2591558134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558242141ea0 0x558242148050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.023+0000 7fe88a7fc640 1 -- 172.21.15.86:0/2591558134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55824214aa80 msgr2=0x558242148590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.023+0000 7fe88a7fc640 1 --2- 172.21.15.86:0/2591558134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55824214aa80 0x558242148590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.023+0000 7fe88a7fc640 1 -- 172.21.15.86:0/2591558134 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558241db83f0 con 0x558242141580 2024-04-21T08:40:28.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.023+0000 7fe88affd640 1 --2- 172.21.15.86:0/2591558134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558242141ea0 0x558242148050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:40:28.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.023+0000 7fe88a7fc640 1 --2- 172.21.15.86:0/2591558134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558242141580 0x558242147b10 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fe880005970 tx=0x7fe8800055e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ede284dc2d7ac1bd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:28.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.023+0000 7fe86affd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:40:28.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.023+0000 7fe86affd640 1 -- 172.21.15.86:0/2591558134 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe880047070 con 0x558242141580 2024-04-21T08:40:28.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.023+0000 7fe86affd640 1 -- 172.21.15.86:0/2591558134 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fe880003a00 con 0x558242141580 2024-04-21T08:40:28.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.023+0000 7fe894457740 1 -- 172.21.15.86:0/2591558134 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe88002cc90 con 0x558242141580 2024-04-21T08:40:28.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.023+0000 7fe86affd640 1 -- 172.21.15.86:0/2591558134 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe88005f050 con 0x558242141580 2024-04-21T08:40:28.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.023+0000 7fe894457740 10 client.16476 fetch_fsmap learned FSMap version 29 2024-04-21T08:40:28.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.023+0000 7fe894457740 1 -- 172.21.15.86:0/2591558134 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558241d780f0 con 0x558242141580 2024-04-21T08:40:28.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.023+0000 7fe86affd640 1 -- 172.21.15.86:0/2591558134 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe88002cc90 con 0x558242141580 2024-04-21T08:40:28.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.023+0000 7fe86affd640 1 -- 172.21.15.86:0/2591558134 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe880040040 con 0x558242141580 2024-04-21T08:40:28.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.023+0000 7fe894457740 10 client.16476 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:40:28.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.023+0000 7fe894457740 10 client.16476 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:28.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.023+0000 7fe894457740 20 client.16476 populate_metadata read hostname 'smithi086' 2024-04-21T08:40:28.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.023+0000 7fe894457740 1 --2- 172.21.15.86:0/2591558134 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55824215a8f0 0x55824215cce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.023+0000 7fe894457740 4 client.16476 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:40:28.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.023+0000 7fe894457740 1 -- 172.21.15.86:0/2591558134 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582420b3c00 con 0x55824215a8f0 2024-04-21T08:40:28.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.024+0000 7fe88affd640 1 --2- 172.21.15.86:0/2591558134 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55824215a8f0 0x55824215cce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.024+0000 7fe86affd640 10 client.16476 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:40:28.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.024+0000 7fe88affd640 1 --2- 172.21.15.86:0/2591558134 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55824215a8f0 0x55824215cce0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4b89ec8369e6fc98 server_cookie=34be59e6270767e2 in_seq=0 out_seq=0 2024-04-21T08:40:28.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.025+0000 7fe86affd640 1 -- 172.21.15.86:0/2591558134 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5582420b3c00 con 0x55824215a8f0 2024-04-21T08:40:28.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.025+0000 7fe86affd640 10 client.16476 handle_command_reply: tid=0 2024-04-21T08:40:28.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.025+0000 7fe894457740 1 client.16476 shutdown 2024-04-21T08:40:28.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.025+0000 7fe894457740 1 -- 172.21.15.86:0/2591558134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558242141580 msgr2=0x558242147b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.025+0000 7fe894457740 1 --2- 172.21.15.86:0/2591558134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558242141580 0x558242147b10 secure :-1 s=READY pgs=772 cs=0 l=1 rev1=1 crypto rx=0x7fe880005970 tx=0x7fe8800055e0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.025+0000 7fe894457740 1 -- 172.21.15.86:0/2591558134 shutdown_connections 2024-04-21T08:40:28.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.025+0000 7fe894457740 1 --2- 172.21.15.86:0/2591558134 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55824215a8f0 0x55824215cce0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.025+0000 7fe894457740 1 --2- 172.21.15.86:0/2591558134 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55824214aa80 0x558242148590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.025+0000 7fe86affd640 0 client.16476 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:40:28.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.025+0000 7fe894457740 1 --2- 172.21.15.86:0/2591558134 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558242141ea0 0x558242148050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.025+0000 7fe894457740 1 --2- 172.21.15.86:0/2591558134 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558242141580 0x558242147b10 unknown :-1 s=CLOSED pgs=772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.025+0000 7fe894457740 1 -- 172.21.15.86:0/2591558134 >> 172.21.15.86:0/2591558134 conn(0x55824213bcb0 msgr2=0x55824213da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:28.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.025+0000 7fe894457740 1 -- 172.21.15.86:0/2591558134 shutdown_connections 2024-04-21T08:40:28.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.025+0000 7fe894457740 1 -- 172.21.15.86:0/2591558134 wait complete. 2024-04-21T08:40:28.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.025+0000 7fe894457740 20 client.16476 trim_cache size 0 max 16384 2024-04-21T08:40:28.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.042+0000 7fe894457740 1 Processor -- start 2024-04-21T08:40:28.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.042+0000 7fe894457740 1 -- start start 2024-04-21T08:40:28.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.042+0000 7fe894457740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558242141580 0x55824214a240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.042+0000 7fe894457740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558242141ea0 0x558242146a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.042+0000 7fe894457740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55824214aa80 0x5582421566d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.042+0000 7fe894457740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c1b13d0 con 0x55824214aa80 2024-04-21T08:40:28.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.042+0000 7fe894457740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c1b2f10 con 0x558242141ea0 2024-04-21T08:40:28.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.042+0000 7fe894457740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c1b1aa0 con 0x558242141580 2024-04-21T08:40:28.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.043+0000 7fe88a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55824214aa80 0x5582421566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.043+0000 7fe88a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55824214aa80 0x5582421566d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35968/0 (socket says 172.21.15.86:35968) 2024-04-21T08:40:28.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.043+0000 7fe88a7fc640 1 -- 172.21.15.86:0/536343895 learned_addr learned my addr 172.21.15.86:0/536343895 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:28.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.043+0000 7fe88affd640 1 --2- 172.21.15.86:0/536343895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558242141ea0 0x558242146a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.043+0000 7fe88a7fc640 1 -- 172.21.15.86:0/536343895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558242141580 msgr2=0x55824214a240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.043+0000 7fe88a7fc640 1 --2- 172.21.15.86:0/536343895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558242141580 0x55824214a240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.043+0000 7fe88a7fc640 1 -- 172.21.15.86:0/536343895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558242141ea0 msgr2=0x558242146a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.043+0000 7fe88a7fc640 1 --2- 172.21.15.86:0/536343895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558242141ea0 0x558242146a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.043+0000 7fe88a7fc640 1 -- 172.21.15.86:0/536343895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55824214d930 con 0x55824214aa80 2024-04-21T08:40:28.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.043+0000 7fe88a7fc640 1 --2- 172.21.15.86:0/536343895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55824214aa80 0x5582421566d0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fe880009850 tx=0x7fe88004cea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=797f2171acdec5d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:28.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.043+0000 7fe8717fa640 1 -- 172.21.15.86:0/536343895 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe880068070 con 0x55824214aa80 2024-04-21T08:40:28.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.043+0000 7fe8717fa640 1 -- 172.21.15.86:0/536343895 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe880056070 con 0x55824214aa80 2024-04-21T08:40:28.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.043+0000 7fe8717fa640 1 -- 172.21.15.86:0/536343895 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe880047070 con 0x55824214aa80 2024-04-21T08:40:28.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.044+0000 7fe894457740 1 -- 172.21.15.86:0/536343895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55824214aa80 msgr2=0x5582421566d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.044+0000 7fe894457740 1 --2- 172.21.15.86:0/536343895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55824214aa80 0x5582421566d0 secure :-1 s=READY pgs=773 cs=0 l=1 rev1=1 crypto rx=0x7fe880009850 tx=0x7fe88004cea0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.044+0000 7fe894457740 1 -- 172.21.15.86:0/536343895 shutdown_connections 2024-04-21T08:40:28.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.044+0000 7fe894457740 1 --2- 172.21.15.86:0/536343895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55824214aa80 0x5582421566d0 unknown :-1 s=CLOSED pgs=773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.044+0000 7fe894457740 1 --2- 172.21.15.86:0/536343895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558242141ea0 0x558242146a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.044+0000 7fe894457740 1 --2- 172.21.15.86:0/536343895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558242141580 0x55824214a240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.044+0000 7fe894457740 1 -- 172.21.15.86:0/536343895 >> 172.21.15.86:0/536343895 conn(0x55824213bcb0 msgr2=0x55824213ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:28.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.044+0000 7fe894457740 1 -- 172.21.15.86:0/536343895 shutdown_connections 2024-04-21T08:40:28.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.044+0000 7fe894457740 1 -- 172.21.15.86:0/536343895 wait complete. 2024-04-21T08:40:28.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.044+0000 7fe894457740 1 -- start start 2024-04-21T08:40:28.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.044+0000 7fe894457740 1 Processor -- start 2024-04-21T08:40:28.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.044+0000 7fe894457740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558242141580 0x558242152e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.044+0000 7fe894457740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558242141ea0 0x55824214e1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.045+0000 7fe894457740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55824214aa80 0x55824214e730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.045+0000 7fe894457740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c1b2840 con 0x55824214aa80 2024-04-21T08:40:28.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.045+0000 7fe894457740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c1b2170 con 0x558242141580 2024-04-21T08:40:28.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.045+0000 7fe894457740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe88c1af390 con 0x558242141ea0 2024-04-21T08:40:28.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.045+0000 7fe894457740 10 client.0 _ll_register_callbacks cb 0x7fe8800041e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:28.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.045+0000 7fe88a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55824214aa80 0x55824214e730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.045+0000 7fe88a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55824214aa80 0x55824214e730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35984/0 (socket says 172.21.15.86:35984) 2024-04-21T08:40:28.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.045+0000 7fe88a7fc640 1 -- 172.21.15.86:0/2742670623 learned_addr learned my addr 172.21.15.86:0/2742670623 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:28.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.045+0000 7fe88a7fc640 1 --2- 172.21.15.86:0/2742670623 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558242141580 0x558242152e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.045+0000 7fe88affd640 1 --2- 172.21.15.86:0/2742670623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558242141ea0 0x55824214e1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.045+0000 7fe88a7fc640 1 -- 172.21.15.86:0/2742670623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558242141ea0 msgr2=0x55824214e1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.045+0000 7fe88a7fc640 1 --2- 172.21.15.86:0/2742670623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558242141ea0 0x55824214e1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.045+0000 7fe88a7fc640 1 -- 172.21.15.86:0/2742670623 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558242141580 msgr2=0x558242152e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.045+0000 7fe88a7fc640 1 --2- 172.21.15.86:0/2742670623 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558242141580 0x558242152e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.045+0000 7fe88a7fc640 1 -- 172.21.15.86:0/2742670623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55824214edf0 con 0x55824214aa80 2024-04-21T08:40:28.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.045+0000 7fe88affd640 1 --2- 172.21.15.86:0/2742670623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558242141ea0 0x55824214e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:40:28.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.045+0000 7fe88a7fc640 1 --2- 172.21.15.86:0/2742670623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55824214aa80 0x55824214e730 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fe88004cdf0 tx=0x7fe8800044d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e7019b789909596c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:28.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.045+0000 7fe86affd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:40:28.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.045+0000 7fe86affd640 1 -- 172.21.15.86:0/2742670623 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe880068070 con 0x55824214aa80 2024-04-21T08:40:28.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.045+0000 7fe86affd640 1 -- 172.21.15.86:0/2742670623 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fe880003800 con 0x55824214aa80 2024-04-21T08:40:28.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.045+0000 7fe894457740 1 -- 172.21.15.86:0/2742670623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe880056070 con 0x55824214aa80 2024-04-21T08:40:28.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.045+0000 7fe86affd640 1 -- 172.21.15.86:0/2742670623 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe880075050 con 0x55824214aa80 2024-04-21T08:40:28.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.045+0000 7fe894457740 10 client.16482 fetch_fsmap learned FSMap version 29 2024-04-21T08:40:28.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.045+0000 7fe894457740 1 -- 172.21.15.86:0/2742670623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5582421c09d0 con 0x55824214aa80 2024-04-21T08:40:28.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.046+0000 7fe86affd640 1 -- 172.21.15.86:0/2742670623 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe880056070 con 0x55824214aa80 2024-04-21T08:40:28.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.046+0000 7fe86affd640 1 -- 172.21.15.86:0/2742670623 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe88004e600 con 0x55824214aa80 2024-04-21T08:40:28.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.046+0000 7fe894457740 10 client.16482 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:40:28.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.046+0000 7fe894457740 10 client.16482 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:28.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.046+0000 7fe894457740 20 client.16482 populate_metadata read hostname 'smithi086' 2024-04-21T08:40:28.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.046+0000 7fe894457740 1 --2- 172.21.15.86:0/2742670623 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5582421c1070 0x5582421c3490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.046+0000 7fe894457740 4 client.16482 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:40:28.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.046+0000 7fe894457740 1 -- 172.21.15.86:0/2742670623 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5582420ada70 con 0x5582421c1070 2024-04-21T08:40:28.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.046+0000 7fe88affd640 1 --2- 172.21.15.86:0/2742670623 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5582421c1070 0x5582421c3490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.046+0000 7fe86affd640 10 client.16482 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:40:28.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.046+0000 7fe88affd640 1 --2- 172.21.15.86:0/2742670623 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5582421c1070 0x5582421c3490 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c0badc63d4cdf571 server_cookie=a5bcdb950cf76f44 in_seq=0 out_seq=0 2024-04-21T08:40:28.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.047+0000 7fe86affd640 1 -- 172.21.15.86:0/2742670623 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5582420ada70 con 0x5582421c1070 2024-04-21T08:40:28.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.047+0000 7fe86affd640 10 client.16482 handle_command_reply: tid=0 2024-04-21T08:40:28.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.047+0000 7fe894457740 1 client.16482 shutdown 2024-04-21T08:40:28.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.047+0000 7fe894457740 1 -- 172.21.15.86:0/2742670623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55824214aa80 msgr2=0x55824214e730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.047+0000 7fe894457740 1 --2- 172.21.15.86:0/2742670623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55824214aa80 0x55824214e730 secure :-1 s=READY pgs=774 cs=0 l=1 rev1=1 crypto rx=0x7fe88004cdf0 tx=0x7fe8800044d0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.047+0000 7fe894457740 1 -- 172.21.15.86:0/2742670623 shutdown_connections 2024-04-21T08:40:28.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.047+0000 7fe894457740 1 --2- 172.21.15.86:0/2742670623 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5582421c1070 0x5582421c3490 crc :-1 s=READY pgs=82 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.047+0000 7fe894457740 1 --2- 172.21.15.86:0/2742670623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55824214aa80 0x55824214e730 unknown :-1 s=CLOSED pgs=774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.047+0000 7fe894457740 1 --2- 172.21.15.86:0/2742670623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558242141ea0 0x55824214e1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.047+0000 7fe86affd640 0 client.16482 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:40:28.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.047+0000 7fe894457740 1 --2- 172.21.15.86:0/2742670623 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558242141580 0x558242152e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.047+0000 7fe894457740 1 -- 172.21.15.86:0/2742670623 >> 172.21.15.86:0/2742670623 conn(0x55824213bcb0 msgr2=0x558242147ad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:28.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.047+0000 7fe894457740 1 -- 172.21.15.86:0/2742670623 shutdown_connections 2024-04-21T08:40:28.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.047+0000 7fe894457740 1 -- 172.21.15.86:0/2742670623 wait complete. 2024-04-21T08:40:28.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.047+0000 7fe894457740 20 client.16482 trim_cache size 0 max 16384 2024-04-21T08:40:28.054 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:40:28.054 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-21T08:40:28.054 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-21T08:40:28.054 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:40:28.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.047+0000 7fe891869640 1 -- 172.21.15.86:0/3178909402 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe8740a3d50 msgr2=0x7fe8740a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.047+0000 7fe891869640 1 --2- 172.21.15.86:0/3178909402 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe8740a3d50 0x7fe8740a6210 secure :-1 s=READY pgs=108 cs=0 l=1 rev1=1 crypto rx=0x7fe88409c5e0 tx=0x7fe884087040 comp rx=0 tx=0).stop 2024-04-21T08:40:28.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.047+0000 7fe891869640 1 -- 172.21.15.86:0/3178909402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe88c10f140 msgr2=0x7fe88c1a68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.047+0000 7fe891869640 1 --2- 172.21.15.86:0/3178909402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe88c10f140 0x7fe88c1a68f0 secure :-1 s=READY pgs=770 cs=0 l=1 rev1=1 crypto rx=0x7fe88400eaa0 tx=0x7fe88400ef60 comp rx=0 tx=0).stop 2024-04-21T08:40:28.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.048+0000 7fe891869640 1 -- 172.21.15.86:0/3178909402 shutdown_connections 2024-04-21T08:40:28.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.048+0000 7fe891869640 1 --2- 172.21.15.86:0/3178909402 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe8740a3d50 0x7fe8740a6210 unknown :-1 s=CLOSED pgs=108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.048+0000 7fe891869640 1 --2- 172.21.15.86:0/3178909402 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe88c10f140 0x7fe88c1a68f0 unknown :-1 s=CLOSED pgs=770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.048+0000 7fe891869640 1 --2- 172.21.15.86:0/3178909402 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe88c106160 0x7fe88c10ec00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.048+0000 7fe891869640 1 --2- 172.21.15.86:0/3178909402 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe88c101090 0x7fe88c10e6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.048+0000 7fe891869640 1 -- 172.21.15.86:0/3178909402 >> 172.21.15.86:0/3178909402 conn(0x7fe88c0771d0 msgr2=0x7fe88c0fee30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:28.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.048+0000 7fe891869640 1 -- 172.21.15.86:0/3178909402 shutdown_connections 2024-04-21T08:40:28.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.048+0000 7fe891869640 1 -- 172.21.15.86:0/3178909402 wait complete. 2024-04-21T08:40:28.058 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:40:28.059 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:40:28.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.157+0000 7ff919787640 1 -- 172.21.15.86:0/2043876038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff914069a80 msgr2=0x7ff91410eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.157+0000 7ff919787640 1 --2- 172.21.15.86:0/2043876038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff914069a80 0x7ff91410eda0 secure :-1 s=READY pgs=775 cs=0 l=1 rev1=1 crypto rx=0x7ff90c009ca0 tx=0x7ff90c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.157+0000 7ff919787640 1 -- 172.21.15.86:0/2043876038 shutdown_connections 2024-04-21T08:40:28.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.157+0000 7ff919787640 1 --2- 172.21.15.86:0/2043876038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff914069a80 0x7ff91410eda0 unknown :-1 s=CLOSED pgs=775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.157+0000 7ff919787640 1 --2- 172.21.15.86:0/2043876038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff9140690e0 0x7ff914069540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.157+0000 7ff919787640 1 --2- 172.21.15.86:0/2043876038 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff914107fe0 0x7ff9141083c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.157+0000 7ff919787640 1 -- 172.21.15.86:0/2043876038 >> 172.21.15.86:0/2043876038 conn(0x7ff914077e30 msgr2=0x7ff914078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:28.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.157+0000 7ff919787640 1 -- 172.21.15.86:0/2043876038 shutdown_connections 2024-04-21T08:40:28.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.157+0000 7ff919787640 1 -- 172.21.15.86:0/2043876038 wait complete. 2024-04-21T08:40:28.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.157+0000 7ff919787640 1 Processor -- start 2024-04-21T08:40:28.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.157+0000 7ff919787640 1 -- start start 2024-04-21T08:40:28.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.158+0000 7ff919787640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9140690e0 0x7ff914198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.158+0000 7ff919787640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff914069a80 0x7ff914198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.158+0000 7ff919787640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff914107fe0 0x7ff91419cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.158+0000 7ff919787640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff914112160 con 0x7ff9140690e0 2024-04-21T08:40:28.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.158+0000 7ff919787640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff914111fe0 con 0x7ff914107fe0 2024-04-21T08:40:28.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.158+0000 7ff919787640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff9141122e0 con 0x7ff914069a80 2024-04-21T08:40:28.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.158+0000 7ff912ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9140690e0 0x7ff914198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.158+0000 7ff912ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9140690e0 0x7ff914198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35998/0 (socket says 172.21.15.86:35998) 2024-04-21T08:40:28.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.158+0000 7ff912ffd640 1 -- 172.21.15.86:0/4193782194 learned_addr learned my addr 172.21.15.86:0/4193782194 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:28.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.158+0000 7ff9137fe640 1 --2- 172.21.15.86:0/4193782194 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff914107fe0 0x7ff91419cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.158+0000 7ff9127fc640 1 --2- 172.21.15.86:0/4193782194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff914069a80 0x7ff914198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.158+0000 7ff912ffd640 1 -- 172.21.15.86:0/4193782194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff914069a80 msgr2=0x7ff914198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.158+0000 7ff912ffd640 1 --2- 172.21.15.86:0/4193782194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff914069a80 0x7ff914198bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.158+0000 7ff912ffd640 1 -- 172.21.15.86:0/4193782194 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff914107fe0 msgr2=0x7ff91419cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.158+0000 7ff912ffd640 1 --2- 172.21.15.86:0/4193782194 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff914107fe0 0x7ff91419cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.158+0000 7ff912ffd640 1 -- 172.21.15.86:0/4193782194 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff91419d670 con 0x7ff9140690e0 2024-04-21T08:40:28.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.158+0000 7ff912ffd640 1 --2- 172.21.15.86:0/4193782194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9140690e0 0x7ff914198680 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff90400d950 tx=0x7ff90400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:28.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.158+0000 7ff8fbfff640 1 -- 172.21.15.86:0/4193782194 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff904017070 con 0x7ff9140690e0 2024-04-21T08:40:28.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.158+0000 7ff919787640 1 -- 172.21.15.86:0/4193782194 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff91419d960 con 0x7ff9140690e0 2024-04-21T08:40:28.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.158+0000 7ff8fbfff640 1 -- 172.21.15.86:0/4193782194 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff904009c30 con 0x7ff9140690e0 2024-04-21T08:40:28.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.158+0000 7ff8fbfff640 1 -- 172.21.15.86:0/4193782194 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff90400f620 con 0x7ff9140690e0 2024-04-21T08:40:28.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.158+0000 7ff919787640 1 -- 172.21.15.86:0/4193782194 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff9141a51f0 con 0x7ff9140690e0 2024-04-21T08:40:28.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.159+0000 7ff919787640 1 -- 172.21.15.86:0/4193782194 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ff914068de0 con 0x7ff9140690e0 2024-04-21T08:40:28.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.160+0000 7ff8fbfff640 1 -- 172.21.15.86:0/4193782194 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff904010460 con 0x7ff9140690e0 2024-04-21T08:40:28.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.161+0000 7ff8fbfff640 1 --2- 172.21.15.86:0/4193782194 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff8fc0a3d20 0x7ff8fc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.161+0000 7ff9127fc640 1 --2- 172.21.15.86:0/4193782194 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff8fc0a3d20 0x7ff8fc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.161+0000 7ff8fbfff640 1 -- 172.21.15.86:0/4193782194 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7ff9040d07d0 con 0x7ff9140690e0 2024-04-21T08:40:28.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.162+0000 7ff9127fc640 1 --2- 172.21.15.86:0/4193782194 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff8fc0a3d20 0x7ff8fc0a61e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff9140ff6a0 tx=0x7ff908005f90 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:28.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.166+0000 7ff8fbfff640 1 -- 172.21.15.86:0/4193782194 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ff90409cb00 con 0x7ff9140690e0 2024-04-21T08:40:28.386 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.386+0000 7ff919787640 1 -- 172.21.15.86:0/4193782194 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ff9141999d0 con 0x7ff9140690e0 2024-04-21T08:40:28.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.386+0000 7ff8fbfff640 1 -- 172.21.15.86:0/4193782194 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7ff904009de0 con 0x7ff9140690e0 2024-04-21T08:40:28.387 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:40:28.387 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:38:45.202126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24398,24394,24455,24472]},"id":1}]} 2024-04-21T08:40:28.387 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-21T08:40:28.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.389+0000 7ff919787640 1 -- 172.21.15.86:0/4193782194 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff8fc0a3d20 msgr2=0x7ff8fc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.389+0000 7ff919787640 1 --2- 172.21.15.86:0/4193782194 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff8fc0a3d20 0x7ff8fc0a61e0 secure :-1 s=READY pgs=109 cs=0 l=1 rev1=1 crypto rx=0x7ff9140ff6a0 tx=0x7ff908005f90 comp rx=0 tx=0).stop 2024-04-21T08:40:28.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.389+0000 7ff919787640 1 -- 172.21.15.86:0/4193782194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9140690e0 msgr2=0x7ff914198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.389+0000 7ff919787640 1 --2- 172.21.15.86:0/4193782194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9140690e0 0x7ff914198680 secure :-1 s=READY pgs=776 cs=0 l=1 rev1=1 crypto rx=0x7ff90400d950 tx=0x7ff90400de10 comp rx=0 tx=0).stop 2024-04-21T08:40:28.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.389+0000 7ff919787640 1 -- 172.21.15.86:0/4193782194 shutdown_connections 2024-04-21T08:40:28.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.389+0000 7ff919787640 1 --2- 172.21.15.86:0/4193782194 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff8fc0a3d20 0x7ff8fc0a61e0 unknown :-1 s=CLOSED pgs=109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.389+0000 7ff919787640 1 --2- 172.21.15.86:0/4193782194 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff914107fe0 0x7ff91419cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.389+0000 7ff919787640 1 --2- 172.21.15.86:0/4193782194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff914069a80 0x7ff914198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.389+0000 7ff919787640 1 --2- 172.21.15.86:0/4193782194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff9140690e0 0x7ff914198680 unknown :-1 s=CLOSED pgs=776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.389+0000 7ff919787640 1 -- 172.21.15.86:0/4193782194 >> 172.21.15.86:0/4193782194 conn(0x7ff914077e30 msgr2=0x7ff9140ffe70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:28.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.389+0000 7ff919787640 1 -- 172.21.15.86:0/4193782194 shutdown_connections 2024-04-21T08:40:28.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.389+0000 7ff919787640 1 -- 172.21.15.86:0/4193782194 wait complete. 2024-04-21T08:40:28.400 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:40:28.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.500+0000 7ffb943ab640 1 -- 172.21.15.86:0/3385580832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb8c1031d0 msgr2=0x7ffb8c1035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.500+0000 7ffb943ab640 1 --2- 172.21.15.86:0/3385580832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb8c1031d0 0x7ffb8c1035b0 secure :-1 s=READY pgs=777 cs=0 l=1 rev1=1 crypto rx=0x7ffb88009900 tx=0x7ffb8802bd60 comp rx=0 tx=0).stop 2024-04-21T08:40:28.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.501+0000 7ffb943ab640 1 -- 172.21.15.86:0/3385580832 shutdown_connections 2024-04-21T08:40:28.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.501+0000 7ffb943ab640 1 --2- 172.21.15.86:0/3385580832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb8c10c510 0x7ffb8c10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.501+0000 7ffb943ab640 1 --2- 172.21.15.86:0/3385580832 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb8c103af0 0x7ffb8c10bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.501+0000 7ffb943ab640 1 --2- 172.21.15.86:0/3385580832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb8c1031d0 0x7ffb8c1035b0 unknown :-1 s=CLOSED pgs=777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.501+0000 7ffb943ab640 1 -- 172.21.15.86:0/3385580832 >> 172.21.15.86:0/3385580832 conn(0x7ffb8c077f50 msgr2=0x7ffb8c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:28.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.501+0000 7ffb943ab640 1 -- 172.21.15.86:0/3385580832 shutdown_connections 2024-04-21T08:40:28.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.501+0000 7ffb943ab640 1 -- 172.21.15.86:0/3385580832 wait complete. 2024-04-21T08:40:28.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.501+0000 7ffb943ab640 1 Processor -- start 2024-04-21T08:40:28.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.501+0000 7ffb943ab640 1 -- start start 2024-04-21T08:40:28.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.501+0000 7ffb943ab640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb8c1031d0 0x7ffb8c198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.501+0000 7ffb943ab640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb8c103af0 0x7ffb8c198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.501+0000 7ffb943ab640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb8c10c510 0x7ffb8c19d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.501+0000 7ffb943ab640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c1121a0 con 0x7ffb8c1031d0 2024-04-21T08:40:28.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.501+0000 7ffb943ab640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c112020 con 0x7ffb8c103af0 2024-04-21T08:40:28.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.501+0000 7ffb943ab640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c112320 con 0x7ffb8c10c510 2024-04-21T08:40:28.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.501+0000 7ffb92147640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb8c1031d0 0x7ffb8c198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.501+0000 7ffb92147640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb8c1031d0 0x7ffb8c198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36008/0 (socket says 172.21.15.86:36008) 2024-04-21T08:40:28.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.501+0000 7ffb92147640 1 -- 172.21.15.86:0/1770891131 learned_addr learned my addr 172.21.15.86:0/1770891131 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:28.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.502+0000 7ffb92948640 1 --2- 172.21.15.86:0/1770891131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb8c10c510 0x7ffb8c19d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.502+0000 7ffb91946640 1 --2- 172.21.15.86:0/1770891131 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb8c103af0 0x7ffb8c198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.502+0000 7ffb92147640 1 -- 172.21.15.86:0/1770891131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb8c10c510 msgr2=0x7ffb8c19d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.502+0000 7ffb92147640 1 --2- 172.21.15.86:0/1770891131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb8c10c510 0x7ffb8c19d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.502+0000 7ffb92147640 1 -- 172.21.15.86:0/1770891131 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb8c103af0 msgr2=0x7ffb8c198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.502+0000 7ffb92147640 1 --2- 172.21.15.86:0/1770891131 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb8c103af0 0x7ffb8c198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.502+0000 7ffb92147640 1 -- 172.21.15.86:0/1770891131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffb8c19d730 con 0x7ffb8c1031d0 2024-04-21T08:40:28.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.502+0000 7ffb92147640 1 --2- 172.21.15.86:0/1770891131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb8c1031d0 0x7ffb8c198740 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ffb88005940 tx=0x7ffb8802d920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:28.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.502+0000 7ffb837fe640 1 -- 172.21.15.86:0/1770891131 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb88043020 con 0x7ffb8c1031d0 2024-04-21T08:40:28.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.502+0000 7ffb837fe640 1 -- 172.21.15.86:0/1770891131 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb8802dd20 con 0x7ffb8c1031d0 2024-04-21T08:40:28.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.502+0000 7ffb943ab640 1 -- 172.21.15.86:0/1770891131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffb8c19d9c0 con 0x7ffb8c1031d0 2024-04-21T08:40:28.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.502+0000 7ffb837fe640 1 -- 172.21.15.86:0/1770891131 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb88003b80 con 0x7ffb8c1031d0 2024-04-21T08:40:28.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.502+0000 7ffb943ab640 1 -- 172.21.15.86:0/1770891131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffb8c1a52b0 con 0x7ffb8c1031d0 2024-04-21T08:40:28.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.504+0000 7ffb837fe640 1 -- 172.21.15.86:0/1770891131 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ffb88046020 con 0x7ffb8c1031d0 2024-04-21T08:40:28.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.505+0000 7ffb96f9d740 1 Processor -- start 2024-04-21T08:40:28.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.505+0000 7ffb96f9d740 1 -- start start 2024-04-21T08:40:28.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.505+0000 7ffb96f9d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591afd2a3b0 0x5591afd2a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.505+0000 7ffb96f9d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591afd2ad60 0x5591afd33890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.505+0000 7ffb96f9d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5591afd33e60 0x5591afd342c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.505+0000 7ffb96f9d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5591afc8f1f0 con 0x5591afd2ad60 2024-04-21T08:40:28.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.505+0000 7ffb96f9d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5591afc2c4b0 con 0x5591afd33e60 2024-04-21T08:40:28.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.505+0000 7ffb96f9d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5591afc8b1b0 con 0x5591afd2a3b0 2024-04-21T08:40:28.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.505+0000 7ffb92147640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591afd2ad60 0x5591afd33890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.505+0000 7ffb92147640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591afd2ad60 0x5591afd33890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36022/0 (socket says 172.21.15.86:36022) 2024-04-21T08:40:28.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.505+0000 7ffb92147640 1 -- 172.21.15.86:0/724686927 learned_addr learned my addr 172.21.15.86:0/724686927 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:28.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.505+0000 7ffb837fe640 1 --2- 172.21.15.86:0/1770891131 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ffb780a3d20 0x7ffb780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.505+0000 7ffb91946640 1 --2- 172.21.15.86:0/724686927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5591afd33e60 0x5591afd342c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.505+0000 7ffb92948640 1 --2- 172.21.15.86:0/724686927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591afd2a3b0 0x5591afd2a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.505+0000 7ffb92948640 1 --2- 172.21.15.86:0/1770891131 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ffb780a3d20 0x7ffb780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.506+0000 7ffb837fe640 1 -- 172.21.15.86:0/1770891131 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7ffb880f2c80 con 0x7ffb8c1031d0 2024-04-21T08:40:28.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.506+0000 7ffb92147640 1 -- 172.21.15.86:0/724686927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591afd2a3b0 msgr2=0x5591afd2a790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.506+0000 7ffb92147640 1 --2- 172.21.15.86:0/724686927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591afd2a3b0 0x5591afd2a790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.506+0000 7ffb92147640 1 -- 172.21.15.86:0/724686927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5591afd33e60 msgr2=0x5591afd342c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.506+0000 7ffb92147640 1 --2- 172.21.15.86:0/724686927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5591afd33e60 0x5591afd342c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.506+0000 7ffb92147640 1 -- 172.21.15.86:0/724686927 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591afc1e1b0 con 0x5591afd2ad60 2024-04-21T08:40:28.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.506+0000 7ffb91946640 1 --2- 172.21.15.86:0/724686927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5591afd33e60 0x5591afd342c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:40:28.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.506+0000 7ffb92147640 1 --2- 172.21.15.86:0/724686927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591afd2ad60 0x5591afd33890 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ffb880be6c0 tx=0x7ffb880ba660 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9d714d487d832dd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:28.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.506+0000 7ffb92948640 1 --2- 172.21.15.86:0/1770891131 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ffb780a3d20 0x7ffb780a61e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ffb84009fd0 tx=0x7ffb84009600 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:28.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.506+0000 7ffb80bfe640 1 -- 172.21.15.86:0/724686927 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb880be320 con 0x5591afd2ad60 2024-04-21T08:40:28.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.506+0000 7ffb80bfe640 1 -- 172.21.15.86:0/724686927 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb880bf0c0 con 0x5591afd2ad60 2024-04-21T08:40:28.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.506+0000 7ffb80bfe640 1 -- 172.21.15.86:0/724686927 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb88005460 con 0x5591afd2ad60 2024-04-21T08:40:28.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.507+0000 7ffb96f9d740 1 -- 172.21.15.86:0/724686927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591afd2ad60 msgr2=0x5591afd33890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.507+0000 7ffb96f9d740 1 --2- 172.21.15.86:0/724686927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591afd2ad60 0x5591afd33890 secure :-1 s=READY pgs=779 cs=0 l=1 rev1=1 crypto rx=0x7ffb880be6c0 tx=0x7ffb880ba660 comp rx=0 tx=0).stop 2024-04-21T08:40:28.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.507+0000 7ffb96f9d740 1 -- 172.21.15.86:0/724686927 shutdown_connections 2024-04-21T08:40:28.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.507+0000 7ffb96f9d740 1 --2- 172.21.15.86:0/724686927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5591afd33e60 0x5591afd342c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.507+0000 7ffb96f9d740 1 --2- 172.21.15.86:0/724686927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591afd2ad60 0x5591afd33890 unknown :-1 s=CLOSED pgs=779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.507+0000 7ffb96f9d740 1 --2- 172.21.15.86:0/724686927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591afd2a3b0 0x5591afd2a790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.507+0000 7ffb96f9d740 1 -- 172.21.15.86:0/724686927 >> 172.21.15.86:0/724686927 conn(0x5591afd24ba0 msgr2=0x5591afd24f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:28.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.507+0000 7ffb96f9d740 1 -- 172.21.15.86:0/724686927 shutdown_connections 2024-04-21T08:40:28.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.507+0000 7ffb96f9d740 1 -- 172.21.15.86:0/724686927 wait complete. 2024-04-21T08:40:28.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.507+0000 7ffb96f9d740 1 -- start start 2024-04-21T08:40:28.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.507+0000 7ffb96f9d740 1 Processor -- start 2024-04-21T08:40:28.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.507+0000 7ffb96f9d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5591afd2a3b0 0x5591afd3a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.507+0000 7ffb96f9d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591afd2ad60 0x5591afd37640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.507+0000 7ffb96f9d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591afd33e60 0x5591afd37ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.507+0000 7ffb96f9d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5591afd3ab60 con 0x5591afd2ad60 2024-04-21T08:40:28.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.507+0000 7ffb96f9d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5591afc48500 con 0x5591afd2a3b0 2024-04-21T08:40:28.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.507+0000 7ffb96f9d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5591afd3ace0 con 0x5591afd33e60 2024-04-21T08:40:28.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.507+0000 7ffb96f9d740 10 client.0 _ll_register_callbacks cb 0x5591afc16b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:28.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.507+0000 7ffb91946640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591afd2ad60 0x5591afd37640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.508+0000 7ffb91946640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591afd2ad60 0x5591afd37640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36036/0 (socket says 172.21.15.86:36036) 2024-04-21T08:40:28.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.508+0000 7ffb91946640 1 -- 172.21.15.86:0/2980468974 learned_addr learned my addr 172.21.15.86:0/2980468974 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:28.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.508+0000 7ffb92147640 1 --2- 172.21.15.86:0/2980468974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5591afd2a3b0 0x5591afd3a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.508+0000 7ffb92948640 1 --2- 172.21.15.86:0/2980468974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591afd33e60 0x5591afd37ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.508+0000 7ffb91946640 1 -- 172.21.15.86:0/2980468974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591afd33e60 msgr2=0x5591afd37ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.508+0000 7ffb91946640 1 --2- 172.21.15.86:0/2980468974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591afd33e60 0x5591afd37ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.508+0000 7ffb91946640 1 -- 172.21.15.86:0/2980468974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5591afd2a3b0 msgr2=0x5591afd3a2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.508+0000 7ffb91946640 1 --2- 172.21.15.86:0/2980468974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5591afd2a3b0 0x5591afd3a2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.508+0000 7ffb91946640 1 -- 172.21.15.86:0/2980468974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591af9a13f0 con 0x5591afd2ad60 2024-04-21T08:40:28.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.508+0000 7ffb92147640 1 --2- 172.21.15.86:0/2980468974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5591afd2a3b0 0x5591afd3a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:28.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.508+0000 7ffb91946640 1 --2- 172.21.15.86:0/2980468974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591afd2ad60 0x5591afd37640 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c011780 tx=0x7ffb7c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27d95e8740351432 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:28.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.508+0000 7ffb75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:40:28.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.508+0000 7ffb75ffb640 1 -- 172.21.15.86:0/2980468974 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb7c00fe70 con 0x5591afd2ad60 2024-04-21T08:40:28.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.508+0000 7ffb75ffb640 1 -- 172.21.15.86:0/2980468974 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7ffb7c004020 con 0x5591afd2ad60 2024-04-21T08:40:28.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.508+0000 7ffb96f9d740 1 -- 172.21.15.86:0/2980468974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb880bf0c0 con 0x5591afd2ad60 2024-04-21T08:40:28.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.508+0000 7ffb75ffb640 1 -- 172.21.15.86:0/2980468974 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffb7c024050 con 0x5591afd2ad60 2024-04-21T08:40:28.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.508+0000 7ffb96f9d740 10 client.16500 fetch_fsmap learned FSMap version 29 2024-04-21T08:40:28.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.508+0000 7ffb96f9d740 1 -- 172.21.15.86:0/2980468974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5591afa6b7d0 con 0x5591afd2ad60 2024-04-21T08:40:28.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.508+0000 7ffb75ffb640 1 -- 172.21.15.86:0/2980468974 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb880bf0c0 con 0x5591afd2ad60 2024-04-21T08:40:28.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.508+0000 7ffb75ffb640 1 -- 172.21.15.86:0/2980468974 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ffb7c018030 con 0x5591afd2ad60 2024-04-21T08:40:28.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.509+0000 7ffb96f9d740 10 client.16500 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:40:28.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.509+0000 7ffb96f9d740 10 client.16500 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:40:28.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.509+0000 7ffb96f9d740 20 client.16500 populate_metadata read hostname 'smithi086' 2024-04-21T08:40:28.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.509+0000 7ffb96f9d740 1 --2- 172.21.15.86:0/2980468974 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x5591afd47e40 0x5591afd4a260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.509+0000 7ffb96f9d740 4 client.16500 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:40:28.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.509+0000 7ffb96f9d740 1 -- 172.21.15.86:0/2980468974 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5591afc2e930 con 0x5591afd47e40 2024-04-21T08:40:28.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.509+0000 7ffb92147640 1 --2- 172.21.15.86:0/2980468974 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x5591afd47e40 0x5591afd4a260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.509+0000 7ffb75ffb640 10 client.16500 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:40:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.509+0000 7ffb92147640 1 --2- 172.21.15.86:0/2980468974 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x5591afd47e40 0x5591afd4a260 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c56735a5e04c8d50 server_cookie=15173d42fd1155a7 in_seq=0 out_seq=0 2024-04-21T08:40:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.510+0000 7ffb75ffb640 1 -- 172.21.15.86:0/2980468974 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5591afc2e930 con 0x5591afd47e40 2024-04-21T08:40:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.510+0000 7ffb75ffb640 10 client.16500 handle_command_reply: tid=0 2024-04-21T08:40:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.510+0000 7ffb96f9d740 1 client.16500 shutdown 2024-04-21T08:40:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.510+0000 7ffb96f9d740 1 -- 172.21.15.86:0/2980468974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591afd2ad60 msgr2=0x5591afd37640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.510+0000 7ffb96f9d740 1 --2- 172.21.15.86:0/2980468974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591afd2ad60 0x5591afd37640 secure :-1 s=READY pgs=780 cs=0 l=1 rev1=1 crypto rx=0x7ffb7c011780 tx=0x7ffb7c011c40 comp rx=0 tx=0).stop 2024-04-21T08:40:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.510+0000 7ffb96f9d740 1 -- 172.21.15.86:0/2980468974 shutdown_connections 2024-04-21T08:40:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.510+0000 7ffb96f9d740 1 --2- 172.21.15.86:0/2980468974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591afd33e60 0x5591afd37ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.510+0000 7ffb96f9d740 1 --2- 172.21.15.86:0/2980468974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591afd2ad60 0x5591afd37640 unknown :-1 s=CLOSED pgs=780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.510+0000 7ffb96f9d740 1 --2- 172.21.15.86:0/2980468974 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x5591afd47e40 0x5591afd4a260 crc :-1 s=READY pgs=38 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.510+0000 7ffb96f9d740 1 --2- 172.21.15.86:0/2980468974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5591afd2a3b0 0x5591afd3a2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.510+0000 7ffb75ffb640 0 client.16500 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:40:28.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.510+0000 7ffb96f9d740 1 -- 172.21.15.86:0/2980468974 >> 172.21.15.86:0/2980468974 conn(0x5591afd24ba0 msgr2=0x5591afd268f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:28.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.510+0000 7ffb96f9d740 1 -- 172.21.15.86:0/2980468974 shutdown_connections 2024-04-21T08:40:28.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.510+0000 7ffb96f9d740 1 -- 172.21.15.86:0/2980468974 wait complete. 2024-04-21T08:40:28.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.510+0000 7ffb96f9d740 20 client.16500 trim_cache size 0 max 16384 2024-04-21T08:40:28.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.523+0000 7ffb96f9d740 1 Processor -- start 2024-04-21T08:40:28.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.523+0000 7ffb96f9d740 1 -- start start 2024-04-21T08:40:28.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.524+0000 7ffb96f9d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5591afd2a3b0 0x5591afd35250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.524+0000 7ffb96f9d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591afd2ad60 0x5591afd39d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.524+0000 7ffb96f9d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591afd33e60 0x5591afd3a2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.524+0000 7ffb96f9d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c1ad0a0 con 0x5591afd33e60 2024-04-21T08:40:28.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.524+0000 7ffb96f9d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c1aebe0 con 0x5591afd2a3b0 2024-04-21T08:40:28.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.524+0000 7ffb96f9d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c1ad770 con 0x5591afd2ad60 2024-04-21T08:40:28.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.524+0000 7ffb92948640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591afd33e60 0x5591afd3a2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.524+0000 7ffb92948640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591afd33e60 0x5591afd3a2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36052/0 (socket says 172.21.15.86:36052) 2024-04-21T08:40:28.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.524+0000 7ffb92948640 1 -- 172.21.15.86:0/1746109515 learned_addr learned my addr 172.21.15.86:0/1746109515 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:28.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.524+0000 7ffb92147640 1 --2- 172.21.15.86:0/1746109515 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5591afd2a3b0 0x5591afd35250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.524+0000 7ffb91946640 1 --2- 172.21.15.86:0/1746109515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591afd2ad60 0x5591afd39d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.524+0000 7ffb92948640 1 -- 172.21.15.86:0/1746109515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591afd2ad60 msgr2=0x5591afd39d70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.524+0000 7ffb92948640 1 --2- 172.21.15.86:0/1746109515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591afd2ad60 0x5591afd39d70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.524+0000 7ffb92948640 1 -- 172.21.15.86:0/1746109515 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5591afd2a3b0 msgr2=0x5591afd35250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.524+0000 7ffb92948640 1 --2- 172.21.15.86:0/1746109515 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5591afd2a3b0 0x5591afd35250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.524+0000 7ffb92948640 1 -- 172.21.15.86:0/1746109515 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5591afd20470 con 0x5591afd33e60 2024-04-21T08:40:28.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.524+0000 7ffb91946640 1 --2- 172.21.15.86:0/1746109515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591afd2ad60 0x5591afd39d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:40:28.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.524+0000 7ffb92948640 1 --2- 172.21.15.86:0/1746109515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591afd33e60 0x5591afd3a2b0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ffb84005c40 tx=0x7ffb8400c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc5f95a7fd3b26a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:28.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.524+0000 7ffb77fff640 1 -- 172.21.15.86:0/1746109515 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb8402d070 con 0x5591afd33e60 2024-04-21T08:40:28.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.524+0000 7ffb77fff640 1 -- 172.21.15.86:0/1746109515 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb8401ea20 con 0x5591afd33e60 2024-04-21T08:40:28.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.524+0000 7ffb77fff640 1 -- 172.21.15.86:0/1746109515 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb84028850 con 0x5591afd33e60 2024-04-21T08:40:28.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.525+0000 7ffb96f9d740 1 -- 172.21.15.86:0/1746109515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591afd33e60 msgr2=0x5591afd3a2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.525+0000 7ffb96f9d740 1 --2- 172.21.15.86:0/1746109515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591afd33e60 0x5591afd3a2b0 secure :-1 s=READY pgs=781 cs=0 l=1 rev1=1 crypto rx=0x7ffb84005c40 tx=0x7ffb8400c000 comp rx=0 tx=0).stop 2024-04-21T08:40:28.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.525+0000 7ffb96f9d740 1 -- 172.21.15.86:0/1746109515 shutdown_connections 2024-04-21T08:40:28.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.525+0000 7ffb96f9d740 1 --2- 172.21.15.86:0/1746109515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591afd33e60 0x5591afd3a2b0 unknown :-1 s=CLOSED pgs=781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.525+0000 7ffb96f9d740 1 --2- 172.21.15.86:0/1746109515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591afd2ad60 0x5591afd39d70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.525+0000 7ffb96f9d740 1 --2- 172.21.15.86:0/1746109515 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5591afd2a3b0 0x5591afd35250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.525+0000 7ffb96f9d740 1 -- 172.21.15.86:0/1746109515 >> 172.21.15.86:0/1746109515 conn(0x5591afd24ba0 msgr2=0x5591afd26830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:28.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.525+0000 7ffb96f9d740 1 -- 172.21.15.86:0/1746109515 shutdown_connections 2024-04-21T08:40:28.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.525+0000 7ffb96f9d740 1 -- 172.21.15.86:0/1746109515 wait complete. 2024-04-21T08:40:28.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.525+0000 7ffb96f9d740 1 -- start start 2024-04-21T08:40:28.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.525+0000 7ffb96f9d740 1 Processor -- start 2024-04-21T08:40:28.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.525+0000 7ffb96f9d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591afd2a3b0 0x5591afd40280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.526+0000 7ffb96f9d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5591afd2ad60 0x5591afd34820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.526+0000 7ffb96f9d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591afd33e60 0x5591afd34d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.526+0000 7ffb96f9d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c1ae510 con 0x5591afd2a3b0 2024-04-21T08:40:28.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.526+0000 7ffb96f9d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c1ade40 con 0x5591afd2ad60 2024-04-21T08:40:28.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.526+0000 7ffb96f9d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ffb8c1ab060 con 0x5591afd33e60 2024-04-21T08:40:28.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.526+0000 7ffb96f9d740 10 client.0 _ll_register_callbacks cb 0x7ffb7c00cf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:28.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.526+0000 7ffb92147640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591afd2a3b0 0x5591afd40280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.526+0000 7ffb92147640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591afd2a3b0 0x5591afd40280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36064/0 (socket says 172.21.15.86:36064) 2024-04-21T08:40:28.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.526+0000 7ffb92147640 1 -- 172.21.15.86:0/768218600 learned_addr learned my addr 172.21.15.86:0/768218600 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:28.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.526+0000 7ffb91946640 1 --2- 172.21.15.86:0/768218600 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5591afd2ad60 0x5591afd34820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.526+0000 7ffb92147640 1 -- 172.21.15.86:0/768218600 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591afd33e60 msgr2=0x5591afd34d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.526+0000 7ffb92147640 1 --2- 172.21.15.86:0/768218600 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591afd33e60 0x5591afd34d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.526+0000 7ffb92147640 1 -- 172.21.15.86:0/768218600 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5591afd2ad60 msgr2=0x5591afd34820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.526+0000 7ffb92147640 1 --2- 172.21.15.86:0/768218600 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5591afd2ad60 0x5591afd34820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.526+0000 7ffb92147640 1 -- 172.21.15.86:0/768218600 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5591afd35560 con 0x5591afd2a3b0 2024-04-21T08:40:28.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.526+0000 7ffb92147640 1 --2- 172.21.15.86:0/768218600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591afd2a3b0 0x5591afd40280 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ffb880bb390 tx=0x7ffb880be440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8bd14fc5065538e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:28.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.526+0000 7ffb75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:40:28.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.526+0000 7ffb75ffb640 1 -- 172.21.15.86:0/768218600 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffb880d7070 con 0x5591afd2a3b0 2024-04-21T08:40:28.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.526+0000 7ffb75ffb640 1 -- 172.21.15.86:0/768218600 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7ffb88003d20 con 0x5591afd2a3b0 2024-04-21T08:40:28.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.526+0000 7ffb96f9d740 1 -- 172.21.15.86:0/768218600 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffb8401ea20 con 0x5591afd2a3b0 2024-04-21T08:40:28.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.526+0000 7ffb75ffb640 1 -- 172.21.15.86:0/768218600 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffb880bf0c0 con 0x5591afd2a3b0 2024-04-21T08:40:28.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.526+0000 7ffb96f9d740 10 client.16506 fetch_fsmap learned FSMap version 29 2024-04-21T08:40:28.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.526+0000 7ffb96f9d740 1 -- 172.21.15.86:0/768218600 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5591afdae300 con 0x5591afd2a3b0 2024-04-21T08:40:28.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.526+0000 7ffb75ffb640 1 -- 172.21.15.86:0/768218600 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffb8401ea20 con 0x5591afd2a3b0 2024-04-21T08:40:28.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.527+0000 7ffb75ffb640 1 -- 172.21.15.86:0/768218600 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ffb880bec20 con 0x5591afd2a3b0 2024-04-21T08:40:28.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.527+0000 7ffb96f9d740 10 client.16506 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:40:28.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.527+0000 7ffb96f9d740 10 client.16506 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:40:28.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.527+0000 7ffb96f9d740 20 client.16506 populate_metadata read hostname 'smithi086' 2024-04-21T08:40:28.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.527+0000 7ffb96f9d740 1 --2- 172.21.15.86:0/768218600 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x5591afdae910 0x5591afdb0d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.527+0000 7ffb96f9d740 4 client.16506 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:40:28.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.527+0000 7ffb96f9d740 1 -- 172.21.15.86:0/768218600 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5591afc9cc10 con 0x5591afdae910 2024-04-21T08:40:28.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.527+0000 7ffb91946640 1 --2- 172.21.15.86:0/768218600 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x5591afdae910 0x5591afdb0d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.527+0000 7ffb91946640 1 --2- 172.21.15.86:0/768218600 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x5591afdae910 0x5591afdb0d30 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=4418b6c0d4aaa197 server_cookie=690c79dbbff83ff2 in_seq=0 out_seq=0 2024-04-21T08:40:28.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.527+0000 7ffb75ffb640 10 client.16506 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:40:28.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.527+0000 7ffb75ffb640 1 -- 172.21.15.86:0/768218600 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5591afc9cc10 con 0x5591afdae910 2024-04-21T08:40:28.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.527+0000 7ffb75ffb640 10 client.16506 handle_command_reply: tid=0 2024-04-21T08:40:28.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.527+0000 7ffb96f9d740 1 client.16506 shutdown 2024-04-21T08:40:28.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.528+0000 7ffb96f9d740 1 -- 172.21.15.86:0/768218600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591afd2a3b0 msgr2=0x5591afd40280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.528+0000 7ffb96f9d740 1 --2- 172.21.15.86:0/768218600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591afd2a3b0 0x5591afd40280 secure :-1 s=READY pgs=782 cs=0 l=1 rev1=1 crypto rx=0x7ffb880bb390 tx=0x7ffb880be440 comp rx=0 tx=0).stop 2024-04-21T08:40:28.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.528+0000 7ffb96f9d740 1 -- 172.21.15.86:0/768218600 shutdown_connections 2024-04-21T08:40:28.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.528+0000 7ffb96f9d740 1 --2- 172.21.15.86:0/768218600 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5591afd33e60 0x5591afd34d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.528+0000 7ffb96f9d740 1 --2- 172.21.15.86:0/768218600 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x5591afdae910 0x5591afdb0d30 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.528+0000 7ffb96f9d740 1 --2- 172.21.15.86:0/768218600 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5591afd2ad60 0x5591afd34820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.528+0000 7ffb75ffb640 0 client.16506 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:40:28.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.528+0000 7ffb96f9d740 1 --2- 172.21.15.86:0/768218600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5591afd2a3b0 0x5591afd40280 unknown :-1 s=CLOSED pgs=782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.528+0000 7ffb96f9d740 1 -- 172.21.15.86:0/768218600 >> 172.21.15.86:0/768218600 conn(0x5591afd24ba0 msgr2=0x5591afd36ed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:28.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.528+0000 7ffb96f9d740 1 -- 172.21.15.86:0/768218600 shutdown_connections 2024-04-21T08:40:28.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.528+0000 7ffb96f9d740 1 -- 172.21.15.86:0/768218600 wait complete. 2024-04-21T08:40:28.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.528+0000 7ffb96f9d740 20 client.16506 trim_cache size 0 max 16384 2024-04-21T08:40:28.536 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:40:28.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.528+0000 7ffb943ab640 1 -- 172.21.15.86:0/1770891131 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ffb780a3d20 msgr2=0x7ffb780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.528+0000 7ffb943ab640 1 --2- 172.21.15.86:0/1770891131 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ffb780a3d20 0x7ffb780a61e0 secure :-1 s=READY pgs=110 cs=0 l=1 rev1=1 crypto rx=0x7ffb84009fd0 tx=0x7ffb84009600 comp rx=0 tx=0).stop 2024-04-21T08:40:28.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.528+0000 7ffb943ab640 1 -- 172.21.15.86:0/1770891131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb8c1031d0 msgr2=0x7ffb8c198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.528+0000 7ffb943ab640 1 --2- 172.21.15.86:0/1770891131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb8c1031d0 0x7ffb8c198740 secure :-1 s=READY pgs=778 cs=0 l=1 rev1=1 crypto rx=0x7ffb88005940 tx=0x7ffb8802d920 comp rx=0 tx=0).stop 2024-04-21T08:40:28.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.528+0000 7ffb943ab640 1 -- 172.21.15.86:0/1770891131 shutdown_connections 2024-04-21T08:40:28.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.528+0000 7ffb943ab640 1 --2- 172.21.15.86:0/1770891131 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ffb780a3d20 0x7ffb780a61e0 unknown :-1 s=CLOSED pgs=110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.528+0000 7ffb943ab640 1 --2- 172.21.15.86:0/1770891131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffb8c10c510 0x7ffb8c19d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.528+0000 7ffb943ab640 1 --2- 172.21.15.86:0/1770891131 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffb8c103af0 0x7ffb8c198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.528+0000 7ffb943ab640 1 --2- 172.21.15.86:0/1770891131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffb8c1031d0 0x7ffb8c198740 unknown :-1 s=CLOSED pgs=778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.528+0000 7ffb943ab640 1 -- 172.21.15.86:0/1770891131 >> 172.21.15.86:0/1770891131 conn(0x7ffb8c077f50 msgr2=0x7ffb8c0fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:28.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.528+0000 7ffb943ab640 1 -- 172.21.15.86:0/1770891131 shutdown_connections 2024-04-21T08:40:28.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.528+0000 7ffb943ab640 1 -- 172.21.15.86:0/1770891131 wait complete. 2024-04-21T08:40:28.539 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:40:28.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.639+0000 7ffa0d6a5640 1 -- 172.21.15.86:0/285029351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa08101b40 msgr2=0x7ffa0810dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.639+0000 7ffa0d6a5640 1 --2- 172.21.15.86:0/285029351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa08101b40 0x7ffa0810dc80 secure :-1 s=READY pgs=783 cs=0 l=1 rev1=1 crypto rx=0x7ffa00009cd0 tx=0x7ffa0002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.639+0000 7ffa0d6a5640 1 -- 172.21.15.86:0/285029351 shutdown_connections 2024-04-21T08:40:28.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.639+0000 7ffa0d6a5640 1 --2- 172.21.15.86:0/285029351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa08101b40 0x7ffa0810dc80 unknown :-1 s=CLOSED pgs=783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.639+0000 7ffa0d6a5640 1 --2- 172.21.15.86:0/285029351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa081011a0 0x7ffa08101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.639+0000 7ffa0d6a5640 1 --2- 172.21.15.86:0/285029351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa08107030 0x7ffa08107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.639+0000 7ffa0d6a5640 1 -- 172.21.15.86:0/285029351 >> 172.21.15.86:0/285029351 conn(0x7ffa08076c50 msgr2=0x7ffa08077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:28.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.639+0000 7ffa0d6a5640 1 -- 172.21.15.86:0/285029351 shutdown_connections 2024-04-21T08:40:28.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.639+0000 7ffa0d6a5640 1 -- 172.21.15.86:0/285029351 wait complete. 2024-04-21T08:40:28.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.640+0000 7ffa0d6a5640 1 Processor -- start 2024-04-21T08:40:28.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.640+0000 7ffa0d6a5640 1 -- start start 2024-04-21T08:40:28.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.640+0000 7ffa0d6a5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa081011a0 0x7ffa08199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.640+0000 7ffa0d6a5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa08101b40 0x7ffa08199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.640+0000 7ffa0d6a5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa08107030 0x7ffa081a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.640+0000 7ffa0d6a5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffa081110c0 con 0x7ffa08101b40 2024-04-21T08:40:28.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.640+0000 7ffa0d6a5640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ffa08110f40 con 0x7ffa081011a0 2024-04-21T08:40:28.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.640+0000 7ffa0d6a5640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ffa08111240 con 0x7ffa08107030 2024-04-21T08:40:28.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.640+0000 7ffa067fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa08101b40 0x7ffa08199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.640+0000 7ffa067fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa08101b40 0x7ffa08199ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36080/0 (socket says 172.21.15.86:36080) 2024-04-21T08:40:28.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.640+0000 7ffa067fc640 1 -- 172.21.15.86:0/40634067 learned_addr learned my addr 172.21.15.86:0/40634067 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:28.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.640+0000 7ffa077fe640 1 --2- 172.21.15.86:0/40634067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa08107030 0x7ffa081a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.640+0000 7ffa06ffd640 1 --2- 172.21.15.86:0/40634067 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa081011a0 0x7ffa08199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.641+0000 7ffa067fc640 1 -- 172.21.15.86:0/40634067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa08107030 msgr2=0x7ffa081a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.641+0000 7ffa067fc640 1 --2- 172.21.15.86:0/40634067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa08107030 0x7ffa081a4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.641+0000 7ffa067fc640 1 -- 172.21.15.86:0/40634067 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa081011a0 msgr2=0x7ffa08199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.641+0000 7ffa067fc640 1 --2- 172.21.15.86:0/40634067 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa081011a0 0x7ffa08199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.641+0000 7ffa067fc640 1 -- 172.21.15.86:0/40634067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa081a4b40 con 0x7ffa08101b40 2024-04-21T08:40:28.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.641+0000 7ffa077fe640 1 --2- 172.21.15.86:0/40634067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa08107030 0x7ffa081a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:28.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.641+0000 7ffa067fc640 1 --2- 172.21.15.86:0/40634067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa08101b40 0x7ffa08199ac0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00d950 tx=0x7ff9fc00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:28.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.641+0000 7ff9effff640 1 -- 172.21.15.86:0/40634067 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc017070 con 0x7ffa08101b40 2024-04-21T08:40:28.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.641+0000 7ff9effff640 1 -- 172.21.15.86:0/40634067 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9fc009c30 con 0x7ffa08101b40 2024-04-21T08:40:28.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.641+0000 7ffa0d6a5640 1 -- 172.21.15.86:0/40634067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa081a4d10 con 0x7ffa08101b40 2024-04-21T08:40:28.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.641+0000 7ff9effff640 1 -- 172.21.15.86:0/40634067 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc00f620 con 0x7ffa08101b40 2024-04-21T08:40:28.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.641+0000 7ffa0d6a5640 1 -- 172.21.15.86:0/40634067 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa081a51d0 con 0x7ffa08101b40 2024-04-21T08:40:28.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.643+0000 7ff9effff640 1 -- 172.21.15.86:0/40634067 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff9fc010460 con 0x7ffa08101b40 2024-04-21T08:40:28.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.643+0000 7ffa10373740 1 Processor -- start 2024-04-21T08:40:28.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.643+0000 7ffa10373740 1 -- start start 2024-04-21T08:40:28.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.643+0000 7ffa10373740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae8d3d04d0 0x55ae8d3d08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.643+0000 7ffa10373740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae8d3d0e80 0x55ae8d3d9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.643+0000 7ffa10373740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d3d9a00 0x55ae8d3dbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.643+0000 7ffa10373740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ae8d3351f0 con 0x55ae8d3d9a00 2024-04-21T08:40:28.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.643+0000 7ffa10373740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55ae8d2d24b0 con 0x55ae8d3d04d0 2024-04-21T08:40:28.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.643+0000 7ffa10373740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ae8d3311b0 con 0x55ae8d3d0e80 2024-04-21T08:40:28.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.644+0000 7ff9effff640 1 --2- 172.21.15.86:0/40634067 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff9f00a3da0 0x7ff9f00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.644+0000 7ffa067fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d3d9a00 0x55ae8d3dbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.644+0000 7ffa077fe640 1 --2- 172.21.15.86:0/40634067 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff9f00a3da0 0x7ff9f00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.644+0000 7ffa067fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d3d9a00 0x55ae8d3dbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36092/0 (socket says 172.21.15.86:36092) 2024-04-21T08:40:28.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.644+0000 7ffa067fc640 1 -- 172.21.15.86:0/2736523635 learned_addr learned my addr 172.21.15.86:0/2736523635 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:28.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.644+0000 7ff9effff640 1 -- 172.21.15.86:0/40634067 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7ff9fc0d08c0 con 0x7ffa08101b40 2024-04-21T08:40:28.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.644+0000 7ffa077fe640 1 --2- 172.21.15.86:0/2736523635 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae8d3d04d0 0x55ae8d3d08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.644+0000 7ffa06ffd640 1 --2- 172.21.15.86:0/2736523635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae8d3d0e80 0x55ae8d3d9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.644+0000 7ffa067fc640 1 -- 172.21.15.86:0/2736523635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae8d3d0e80 msgr2=0x55ae8d3d9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.644+0000 7ffa067fc640 1 --2- 172.21.15.86:0/2736523635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae8d3d0e80 0x55ae8d3d9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.644+0000 7ffa067fc640 1 -- 172.21.15.86:0/2736523635 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae8d3d04d0 msgr2=0x55ae8d3d08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.644+0000 7ffa067fc640 1 --2- 172.21.15.86:0/2736523635 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae8d3d04d0 0x55ae8d3d08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.644+0000 7ffa067fc640 1 -- 172.21.15.86:0/2736523635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae8d2c41b0 con 0x55ae8d3d9a00 2024-04-21T08:40:28.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.644+0000 7ffa077fe640 1 --2- 172.21.15.86:0/40634067 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff9f00a3da0 0x7ff9f00a6260 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ffa00006010 tx=0x7ffa0002cef0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:28.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.644+0000 7ffa067fc640 1 --2- 172.21.15.86:0/2736523635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d3d9a00 0x55ae8d3dbe50 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc0a4bf0 tx=0x7ff9fc06efa0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=515fdd8e941750c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:28.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.644+0000 7ff9ed7fa640 1 -- 172.21.15.86:0/2736523635 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc020070 con 0x55ae8d3d9a00 2024-04-21T08:40:28.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.644+0000 7ff9ed7fa640 1 -- 172.21.15.86:0/2736523635 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9fc09cbb0 con 0x55ae8d3d9a00 2024-04-21T08:40:28.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.644+0000 7ff9ed7fa640 1 -- 172.21.15.86:0/2736523635 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc0a0860 con 0x55ae8d3d9a00 2024-04-21T08:40:28.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.645+0000 7ffa10373740 1 -- 172.21.15.86:0/2736523635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d3d9a00 msgr2=0x55ae8d3dbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.645+0000 7ffa10373740 1 --2- 172.21.15.86:0/2736523635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d3d9a00 0x55ae8d3dbe50 secure :-1 s=READY pgs=785 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc0a4bf0 tx=0x7ff9fc06efa0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.645+0000 7ffa10373740 1 -- 172.21.15.86:0/2736523635 shutdown_connections 2024-04-21T08:40:28.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.645+0000 7ffa10373740 1 --2- 172.21.15.86:0/2736523635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d3d9a00 0x55ae8d3dbe50 unknown :-1 s=CLOSED pgs=785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.645+0000 7ffa10373740 1 --2- 172.21.15.86:0/2736523635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae8d3d0e80 0x55ae8d3d9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.645+0000 7ffa10373740 1 --2- 172.21.15.86:0/2736523635 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae8d3d04d0 0x55ae8d3d08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.645+0000 7ffa10373740 1 -- 172.21.15.86:0/2736523635 >> 172.21.15.86:0/2736523635 conn(0x55ae8d3caba0 msgr2=0x55ae8d3caf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:28.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.645+0000 7ffa10373740 1 -- 172.21.15.86:0/2736523635 shutdown_connections 2024-04-21T08:40:28.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.645+0000 7ffa10373740 1 -- 172.21.15.86:0/2736523635 wait complete. 2024-04-21T08:40:28.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.645+0000 7ffa10373740 1 -- start start 2024-04-21T08:40:28.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.645+0000 7ffa10373740 1 Processor -- start 2024-04-21T08:40:28.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.646+0000 7ffa10373740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae8d3d04d0 0x55ae8d3d92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.646+0000 7ffa10373740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d3d0e80 0x55ae8d3d6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.646+0000 7ffa10373740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae8d3d9a00 0x55ae8d3d6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.646+0000 7ffa10373740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ae8d3dc910 con 0x55ae8d3d0e80 2024-04-21T08:40:28.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.646+0000 7ffa10373740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55ae8d2ee500 con 0x55ae8d3d9a00 2024-04-21T08:40:28.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.646+0000 7ffa10373740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ae8d3dca90 con 0x55ae8d3d04d0 2024-04-21T08:40:28.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.646+0000 7ffa10373740 10 client.0 _ll_register_callbacks cb 0x55ae8d2bcb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:28.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.646+0000 7ffa067fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d3d0e80 0x55ae8d3d6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.646+0000 7ffa067fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d3d0e80 0x55ae8d3d6670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36102/0 (socket says 172.21.15.86:36102) 2024-04-21T08:40:28.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.646+0000 7ffa067fc640 1 -- 172.21.15.86:0/1399477602 learned_addr learned my addr 172.21.15.86:0/1399477602 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:28.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.646+0000 7ffa077fe640 1 --2- 172.21.15.86:0/1399477602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae8d3d9a00 0x55ae8d3d6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.646+0000 7ffa06ffd640 1 --2- 172.21.15.86:0/1399477602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae8d3d04d0 0x55ae8d3d92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.646+0000 7ffa067fc640 1 -- 172.21.15.86:0/1399477602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae8d3d04d0 msgr2=0x55ae8d3d92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.646+0000 7ffa067fc640 1 --2- 172.21.15.86:0/1399477602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae8d3d04d0 0x55ae8d3d92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.646+0000 7ffa067fc640 1 -- 172.21.15.86:0/1399477602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae8d3d9a00 msgr2=0x55ae8d3d6bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.646+0000 7ffa067fc640 1 --2- 172.21.15.86:0/1399477602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae8d3d9a00 0x55ae8d3d6bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.646+0000 7ffa067fc640 1 -- 172.21.15.86:0/1399477602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae8d0473f0 con 0x55ae8d3d0e80 2024-04-21T08:40:28.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.646+0000 7ffa077fe640 1 --2- 172.21.15.86:0/1399477602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae8d3d9a00 0x55ae8d3d6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:28.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.646+0000 7ffa067fc640 1 --2- 172.21.15.86:0/1399477602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d3d0e80 0x55ae8d3d6670 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc09c560 tx=0x7ff9fc009de0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46c679ee05c557fe server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:28.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.646+0000 7ff9eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:40:28.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.646+0000 7ff9eaff5640 1 -- 172.21.15.86:0/1399477602 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc020040 con 0x55ae8d3d0e80 2024-04-21T08:40:28.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.646+0000 7ff9eaff5640 1 -- 172.21.15.86:0/1399477602 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7ff9fc0b6ba0 con 0x55ae8d3d0e80 2024-04-21T08:40:28.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.646+0000 7ffa10373740 1 -- 172.21.15.86:0/1399477602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9fc09cbb0 con 0x55ae8d3d0e80 2024-04-21T08:40:28.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.646+0000 7ff9eaff5640 1 -- 172.21.15.86:0/1399477602 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff9fc0a0860 con 0x55ae8d3d0e80 2024-04-21T08:40:28.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.646+0000 7ffa10373740 10 client.16518 fetch_fsmap learned FSMap version 29 2024-04-21T08:40:28.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.646+0000 7ffa10373740 1 -- 172.21.15.86:0/1399477602 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ae8d1117d0 con 0x55ae8d3d0e80 2024-04-21T08:40:28.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.647+0000 7ff9eaff5640 1 -- 172.21.15.86:0/1399477602 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9fc09cbb0 con 0x55ae8d3d0e80 2024-04-21T08:40:28.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.647+0000 7ff9eaff5640 1 -- 172.21.15.86:0/1399477602 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff9fc098200 con 0x55ae8d3d0e80 2024-04-21T08:40:28.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.647+0000 7ffa10373740 10 client.16518 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:40:28.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.647+0000 7ffa10373740 10 client.16518 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:40:28.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.647+0000 7ffa10373740 20 client.16518 populate_metadata read hostname 'smithi086' 2024-04-21T08:40:28.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.647+0000 7ffa10373740 1 --2- 172.21.15.86:0/1399477602 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55ae8d3e9a30 0x55ae8d3ebe20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.647+0000 7ffa10373740 4 client.16518 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:40:28.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.647+0000 7ffa10373740 1 -- 172.21.15.86:0/1399477602 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae8d2d4930 con 0x55ae8d3e9a30 2024-04-21T08:40:28.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.647+0000 7ffa06ffd640 1 --2- 172.21.15.86:0/1399477602 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55ae8d3e9a30 0x55ae8d3ebe20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.647+0000 7ff9eaff5640 10 client.16518 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:40:28.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.647+0000 7ffa06ffd640 1 --2- 172.21.15.86:0/1399477602 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55ae8d3e9a30 0x55ae8d3ebe20 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2c4f4e4763fbe7eb server_cookie=33d475f1b3c5a66b in_seq=0 out_seq=0 2024-04-21T08:40:28.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.648+0000 7ff9eaff5640 1 -- 172.21.15.86:0/1399477602 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ae8d2d4930 con 0x55ae8d3e9a30 2024-04-21T08:40:28.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.648+0000 7ff9eaff5640 10 client.16518 handle_command_reply: tid=0 2024-04-21T08:40:28.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.648+0000 7ffa10373740 1 client.16518 shutdown 2024-04-21T08:40:28.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.648+0000 7ffa10373740 1 -- 172.21.15.86:0/1399477602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d3d0e80 msgr2=0x55ae8d3d6670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.648+0000 7ffa10373740 1 --2- 172.21.15.86:0/1399477602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d3d0e80 0x55ae8d3d6670 secure :-1 s=READY pgs=786 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc09c560 tx=0x7ff9fc009de0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.648+0000 7ffa10373740 1 -- 172.21.15.86:0/1399477602 shutdown_connections 2024-04-21T08:40:28.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.648+0000 7ffa10373740 1 --2- 172.21.15.86:0/1399477602 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55ae8d3e9a30 0x55ae8d3ebe20 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.648+0000 7ffa10373740 1 --2- 172.21.15.86:0/1399477602 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae8d3d9a00 0x55ae8d3d6bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.648+0000 7ffa10373740 1 --2- 172.21.15.86:0/1399477602 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d3d0e80 0x55ae8d3d6670 unknown :-1 s=CLOSED pgs=786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.648+0000 7ff9eaff5640 0 client.16518 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:40:28.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.648+0000 7ffa10373740 1 --2- 172.21.15.86:0/1399477602 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae8d3d04d0 0x55ae8d3d92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.648+0000 7ffa10373740 1 -- 172.21.15.86:0/1399477602 >> 172.21.15.86:0/1399477602 conn(0x55ae8d3caba0 msgr2=0x55ae8d3cc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:28.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.648+0000 7ffa10373740 1 -- 172.21.15.86:0/1399477602 shutdown_connections 2024-04-21T08:40:28.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.648+0000 7ffa10373740 1 -- 172.21.15.86:0/1399477602 wait complete. 2024-04-21T08:40:28.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.649+0000 7ffa10373740 20 client.16518 trim_cache size 0 max 16384 2024-04-21T08:40:28.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.661+0000 7ffa10373740 1 Processor -- start 2024-04-21T08:40:28.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.661+0000 7ffa10373740 1 -- start start 2024-04-21T08:40:28.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.662+0000 7ffa10373740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae8d3d04d0 0x55ae8d3e5800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.662+0000 7ffa10373740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d3d0e80 0x55ae8d3cc880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.662+0000 7ffa10373740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae8d3d9a00 0x55ae8d3d8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.662+0000 7ffa10373740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffa081af310 con 0x55ae8d3d0e80 2024-04-21T08:40:28.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.662+0000 7ffa10373740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ffa081b0e50 con 0x55ae8d3d04d0 2024-04-21T08:40:28.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.662+0000 7ffa10373740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ffa081af9e0 con 0x55ae8d3d9a00 2024-04-21T08:40:28.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.662+0000 7ffa067fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d3d0e80 0x55ae8d3cc880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.662+0000 7ffa067fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d3d0e80 0x55ae8d3cc880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36106/0 (socket says 172.21.15.86:36106) 2024-04-21T08:40:28.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.662+0000 7ffa067fc640 1 -- 172.21.15.86:0/1783078318 learned_addr learned my addr 172.21.15.86:0/1783078318 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:28.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.662+0000 7ffa077fe640 1 --2- 172.21.15.86:0/1783078318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae8d3d9a00 0x55ae8d3d8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.662+0000 7ffa06ffd640 1 --2- 172.21.15.86:0/1783078318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae8d3d04d0 0x55ae8d3e5800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.662+0000 7ffa067fc640 1 -- 172.21.15.86:0/1783078318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae8d3d9a00 msgr2=0x55ae8d3d8da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.662+0000 7ffa067fc640 1 --2- 172.21.15.86:0/1783078318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae8d3d9a00 0x55ae8d3d8da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.662+0000 7ffa067fc640 1 -- 172.21.15.86:0/1783078318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae8d3d04d0 msgr2=0x55ae8d3e5800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.662+0000 7ffa067fc640 1 --2- 172.21.15.86:0/1783078318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae8d3d04d0 0x55ae8d3e5800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.662+0000 7ffa067fc640 1 -- 172.21.15.86:0/1783078318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae8d3cf790 con 0x55ae8d3d0e80 2024-04-21T08:40:28.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.662+0000 7ffa067fc640 1 --2- 172.21.15.86:0/1783078318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d3d0e80 0x55ae8d3cc880 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc000c00 tx=0x7ff9fc005880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a04d86f40296423 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:28.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.662+0000 7ff9ecff9640 1 -- 172.21.15.86:0/1783078318 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc020070 con 0x55ae8d3d0e80 2024-04-21T08:40:28.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.662+0000 7ff9ecff9640 1 -- 172.21.15.86:0/1783078318 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9fc0a5120 con 0x55ae8d3d0e80 2024-04-21T08:40:28.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.662+0000 7ff9ecff9640 1 -- 172.21.15.86:0/1783078318 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff9fc0b6e00 con 0x55ae8d3d0e80 2024-04-21T08:40:28.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.663+0000 7ffa10373740 1 -- 172.21.15.86:0/1783078318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d3d0e80 msgr2=0x55ae8d3cc880 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.663+0000 7ffa10373740 1 --2- 172.21.15.86:0/1783078318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d3d0e80 0x55ae8d3cc880 secure :-1 s=READY pgs=787 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc000c00 tx=0x7ff9fc005880 comp rx=0 tx=0).stop 2024-04-21T08:40:28.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.663+0000 7ffa10373740 1 -- 172.21.15.86:0/1783078318 shutdown_connections 2024-04-21T08:40:28.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.663+0000 7ffa10373740 1 --2- 172.21.15.86:0/1783078318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae8d3d9a00 0x55ae8d3d8da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.663+0000 7ffa10373740 1 --2- 172.21.15.86:0/1783078318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d3d0e80 0x55ae8d3cc880 unknown :-1 s=CLOSED pgs=787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.663+0000 7ffa10373740 1 --2- 172.21.15.86:0/1783078318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae8d3d04d0 0x55ae8d3e5800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.663+0000 7ffa10373740 1 -- 172.21.15.86:0/1783078318 >> 172.21.15.86:0/1783078318 conn(0x55ae8d3caba0 msgr2=0x55ae8d3caf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:28.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.663+0000 7ffa10373740 1 -- 172.21.15.86:0/1783078318 shutdown_connections 2024-04-21T08:40:28.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.663+0000 7ffa10373740 1 -- 172.21.15.86:0/1783078318 wait complete. 2024-04-21T08:40:28.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.663+0000 7ffa10373740 1 -- start start 2024-04-21T08:40:28.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.663+0000 7ffa10373740 1 Processor -- start 2024-04-21T08:40:28.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.663+0000 7ffa10373740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae8d3d04d0 0x55ae8d3dd730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.663+0000 7ffa10373740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae8d3d0e80 0x55ae8d3ddc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.663+0000 7ffa10373740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d3d9a00 0x55ae8d3de1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.664+0000 7ffa10373740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffa081b0780 con 0x55ae8d3d9a00 2024-04-21T08:40:28.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.664+0000 7ffa10373740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ffa081b00b0 con 0x55ae8d3d04d0 2024-04-21T08:40:28.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.664+0000 7ffa10373740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ffa081ad2d0 con 0x55ae8d3d0e80 2024-04-21T08:40:28.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.664+0000 7ffa10373740 10 client.0 _ll_register_callbacks cb 0x7ff9fc09be10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:28.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.664+0000 7ffa077fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d3d9a00 0x55ae8d3de1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.664+0000 7ffa077fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d3d9a00 0x55ae8d3de1b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36114/0 (socket says 172.21.15.86:36114) 2024-04-21T08:40:28.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.664+0000 7ffa077fe640 1 -- 172.21.15.86:0/1513013801 learned_addr learned my addr 172.21.15.86:0/1513013801 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:28.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.664+0000 7ffa067fc640 1 --2- 172.21.15.86:0/1513013801 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae8d3d0e80 0x55ae8d3ddc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.664+0000 7ffa06ffd640 1 --2- 172.21.15.86:0/1513013801 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae8d3d04d0 0x55ae8d3dd730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.664+0000 7ffa077fe640 1 -- 172.21.15.86:0/1513013801 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae8d3d0e80 msgr2=0x55ae8d3ddc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.664+0000 7ffa077fe640 1 --2- 172.21.15.86:0/1513013801 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae8d3d0e80 0x55ae8d3ddc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.664+0000 7ffa077fe640 1 -- 172.21.15.86:0/1513013801 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae8d3d04d0 msgr2=0x55ae8d3dd730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.664+0000 7ffa077fe640 1 --2- 172.21.15.86:0/1513013801 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae8d3d04d0 0x55ae8d3dd730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.664+0000 7ffa077fe640 1 -- 172.21.15.86:0/1513013801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae8d44ec30 con 0x55ae8d3d9a00 2024-04-21T08:40:28.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.664+0000 7ffa077fe640 1 --2- 172.21.15.86:0/1513013801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d3d9a00 0x55ae8d3de1b0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ffa00009cd0 tx=0x7ffa00044000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d24a7e39e82392f6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:28.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.664+0000 7ff9eaff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:40:28.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.664+0000 7ff9eaff5640 1 -- 172.21.15.86:0/1513013801 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa00054020 con 0x55ae8d3d9a00 2024-04-21T08:40:28.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.664+0000 7ff9eaff5640 1 -- 172.21.15.86:0/1513013801 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7ffa00003d80 con 0x55ae8d3d9a00 2024-04-21T08:40:28.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.664+0000 7ffa10373740 1 -- 172.21.15.86:0/1513013801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff9fc0a5120 con 0x55ae8d3d9a00 2024-04-21T08:40:28.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.664+0000 7ff9eaff5640 1 -- 172.21.15.86:0/1513013801 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffa0005f050 con 0x55ae8d3d9a00 2024-04-21T08:40:28.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.664+0000 7ffa10373740 10 client.16524 fetch_fsmap learned FSMap version 29 2024-04-21T08:40:28.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.664+0000 7ffa10373740 1 -- 172.21.15.86:0/1513013801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ae8d44fed0 con 0x55ae8d3d9a00 2024-04-21T08:40:28.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.665+0000 7ff9eaff5640 1 -- 172.21.15.86:0/1513013801 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff9fc0a5120 con 0x55ae8d3d9a00 2024-04-21T08:40:28.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.665+0000 7ff9eaff5640 1 -- 172.21.15.86:0/1513013801 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ffa00050750 con 0x55ae8d3d9a00 2024-04-21T08:40:28.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.665+0000 7ffa10373740 10 client.16524 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:40:28.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.665+0000 7ffa10373740 10 client.16524 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:40:28.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.665+0000 7ffa10373740 20 client.16524 populate_metadata read hostname 'smithi086' 2024-04-21T08:40:28.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.665+0000 7ffa10373740 1 --2- 172.21.15.86:0/1513013801 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55ae8d4506d0 0x55ae8d452af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.665+0000 7ffa10373740 4 client.16524 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:40:28.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.665+0000 7ffa10373740 1 -- 172.21.15.86:0/1513013801 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ae8d342c10 con 0x55ae8d4506d0 2024-04-21T08:40:28.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.665+0000 7ffa06ffd640 1 --2- 172.21.15.86:0/1513013801 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55ae8d4506d0 0x55ae8d452af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.665+0000 7ff9eaff5640 10 client.16524 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:40:28.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.665+0000 7ffa06ffd640 1 --2- 172.21.15.86:0/1513013801 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55ae8d4506d0 0x55ae8d452af0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=df20a15bb2042df5 server_cookie=2d549ab1659e3d9b in_seq=0 out_seq=0 2024-04-21T08:40:28.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.666+0000 7ff9eaff5640 1 -- 172.21.15.86:0/1513013801 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ae8d342c10 con 0x55ae8d4506d0 2024-04-21T08:40:28.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.666+0000 7ff9eaff5640 10 client.16524 handle_command_reply: tid=0 2024-04-21T08:40:28.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.666+0000 7ffa10373740 1 client.16524 shutdown 2024-04-21T08:40:28.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.666+0000 7ffa10373740 1 -- 172.21.15.86:0/1513013801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d3d9a00 msgr2=0x55ae8d3de1b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.666+0000 7ffa10373740 1 --2- 172.21.15.86:0/1513013801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d3d9a00 0x55ae8d3de1b0 secure :-1 s=READY pgs=788 cs=0 l=1 rev1=1 crypto rx=0x7ffa00009cd0 tx=0x7ffa00044000 comp rx=0 tx=0).stop 2024-04-21T08:40:28.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.666+0000 7ffa10373740 1 -- 172.21.15.86:0/1513013801 shutdown_connections 2024-04-21T08:40:28.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.666+0000 7ffa10373740 1 --2- 172.21.15.86:0/1513013801 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55ae8d4506d0 0x55ae8d452af0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.666+0000 7ffa10373740 1 --2- 172.21.15.86:0/1513013801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae8d3d9a00 0x55ae8d3de1b0 unknown :-1 s=CLOSED pgs=788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.666+0000 7ff9eaff5640 0 client.16524 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:40:28.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.666+0000 7ffa10373740 1 --2- 172.21.15.86:0/1513013801 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae8d3d0e80 0x55ae8d3ddc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.666+0000 7ffa10373740 1 --2- 172.21.15.86:0/1513013801 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae8d3d04d0 0x55ae8d3dd730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.666+0000 7ffa10373740 1 -- 172.21.15.86:0/1513013801 >> 172.21.15.86:0/1513013801 conn(0x55ae8d3caba0 msgr2=0x55ae8d3d1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:28.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.666+0000 7ffa10373740 1 -- 172.21.15.86:0/1513013801 shutdown_connections 2024-04-21T08:40:28.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.666+0000 7ffa10373740 1 -- 172.21.15.86:0/1513013801 wait complete. 2024-04-21T08:40:28.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.666+0000 7ffa10373740 20 client.16524 trim_cache size 0 max 16384 2024-04-21T08:40:28.673 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:40:28.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.666+0000 7ffa0d6a5640 1 -- 172.21.15.86:0/40634067 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff9f00a3da0 msgr2=0x7ff9f00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.666+0000 7ffa0d6a5640 1 --2- 172.21.15.86:0/40634067 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff9f00a3da0 0x7ff9f00a6260 secure :-1 s=READY pgs=111 cs=0 l=1 rev1=1 crypto rx=0x7ffa00006010 tx=0x7ffa0002cef0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.666+0000 7ffa0d6a5640 1 -- 172.21.15.86:0/40634067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa08101b40 msgr2=0x7ffa08199ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.666+0000 7ffa0d6a5640 1 --2- 172.21.15.86:0/40634067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa08101b40 0x7ffa08199ac0 secure :-1 s=READY pgs=784 cs=0 l=1 rev1=1 crypto rx=0x7ff9fc00d950 tx=0x7ff9fc00de10 comp rx=0 tx=0).stop 2024-04-21T08:40:28.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.666+0000 7ffa0d6a5640 1 -- 172.21.15.86:0/40634067 shutdown_connections 2024-04-21T08:40:28.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.666+0000 7ffa0d6a5640 1 --2- 172.21.15.86:0/40634067 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff9f00a3da0 0x7ff9f00a6260 unknown :-1 s=CLOSED pgs=111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.666+0000 7ffa0d6a5640 1 --2- 172.21.15.86:0/40634067 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa08107030 0x7ffa081a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.666+0000 7ffa0d6a5640 1 --2- 172.21.15.86:0/40634067 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa08101b40 0x7ffa08199ac0 unknown :-1 s=CLOSED pgs=784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.666+0000 7ffa0d6a5640 1 --2- 172.21.15.86:0/40634067 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa081011a0 0x7ffa08199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.666+0000 7ffa0d6a5640 1 -- 172.21.15.86:0/40634067 >> 172.21.15.86:0/40634067 conn(0x7ffa08076c50 msgr2=0x7ffa080fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:28.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.666+0000 7ffa0d6a5640 1 -- 172.21.15.86:0/40634067 shutdown_connections 2024-04-21T08:40:28.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.667+0000 7ffa0d6a5640 1 -- 172.21.15.86:0/40634067 wait complete. 2024-04-21T08:40:28.676 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:28 smithi086 ceph-mon[27939]: pgmap v53: 129 pgs: 129 active+clean; 2.0 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-04-21T08:40:28.676 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:28 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/4193782194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:40:28.677 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:40:28.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:28 smithi179 ceph-mon[31829]: pgmap v53: 129 pgs: 129 active+clean; 2.0 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-04-21T08:40:28.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:28 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/4193782194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:40:28.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.776+0000 7f0a0806d640 1 -- 172.21.15.86:0/3863439005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a0006b230 msgr2=0x7f0a00106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.776+0000 7f0a0806d640 1 --2- 172.21.15.86:0/3863439005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a0006b230 0x7f0a00106b90 secure :-1 s=READY pgs=789 cs=0 l=1 rev1=1 crypto rx=0x7f09f0009900 tx=0x7f09f002bc30 comp rx=0 tx=0).stop 2024-04-21T08:40:28.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.776+0000 7f0a0806d640 1 -- 172.21.15.86:0/3863439005 shutdown_connections 2024-04-21T08:40:28.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.776+0000 7f0a0806d640 1 --2- 172.21.15.86:0/3863439005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a00107270 0x7f0a0010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.776+0000 7f0a0806d640 1 --2- 172.21.15.86:0/3863439005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a0006b230 0x7f0a00106b90 unknown :-1 s=CLOSED pgs=789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.776+0000 7f0a0806d640 1 --2- 172.21.15.86:0/3863439005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a0006a880 0x7f0a0006ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.776+0000 7f0a0806d640 1 -- 172.21.15.86:0/3863439005 >> 172.21.15.86:0/3863439005 conn(0x7f0a00077eb0 msgr2=0x7f0a000782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:28.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.776+0000 7f0a0806d640 1 -- 172.21.15.86:0/3863439005 shutdown_connections 2024-04-21T08:40:28.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.776+0000 7f0a0806d640 1 -- 172.21.15.86:0/3863439005 wait complete. 2024-04-21T08:40:28.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.777+0000 7f0a0806d640 1 Processor -- start 2024-04-21T08:40:28.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.777+0000 7f0a0806d640 1 -- start start 2024-04-21T08:40:28.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.777+0000 7f0a0806d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a0006a880 0x7f0a0019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.777+0000 7f0a0806d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a00107270 0x7f0a0019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.777+0000 7f0a0806d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a00198590 0x7f0a00198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.777+0000 7f0a0806d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0a001121b0 con 0x7f0a00198590 2024-04-21T08:40:28.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.777+0000 7f0a0806d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0a00112030 con 0x7f0a0006a880 2024-04-21T08:40:28.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.777+0000 7f0a0806d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0a00112330 con 0x7f0a00107270 2024-04-21T08:40:28.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.777+0000 7f0a0660a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a00198590 0x7f0a00198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.777+0000 7f0a0660a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a00198590 0x7f0a00198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36128/0 (socket says 172.21.15.86:36128) 2024-04-21T08:40:28.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.777+0000 7f0a0660a640 1 -- 172.21.15.86:0/1331980294 learned_addr learned my addr 172.21.15.86:0/1331980294 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:28.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.777+0000 7f0a05e09640 1 --2- 172.21.15.86:0/1331980294 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a0006a880 0x7f0a0019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.777+0000 7f0a05608640 1 --2- 172.21.15.86:0/1331980294 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a00107270 0x7f0a0019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.778+0000 7f0a0660a640 1 -- 172.21.15.86:0/1331980294 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a00107270 msgr2=0x7f0a0019e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.778+0000 7f0a0660a640 1 --2- 172.21.15.86:0/1331980294 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a00107270 0x7f0a0019e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.778+0000 7f0a0660a640 1 -- 172.21.15.86:0/1331980294 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a0006a880 msgr2=0x7f0a0019e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.778+0000 7f0a0660a640 1 --2- 172.21.15.86:0/1331980294 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a0006a880 0x7f0a0019e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.778+0000 7f0a0660a640 1 -- 172.21.15.86:0/1331980294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0a001990b0 con 0x7f0a00198590 2024-04-21T08:40:28.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.778+0000 7f0a05608640 1 --2- 172.21.15.86:0/1331980294 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a00107270 0x7f0a0019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:28.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.778+0000 7f0a0660a640 1 --2- 172.21.15.86:0/1331980294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a00198590 0x7f0a00198a40 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f09f800eaa0 tx=0x7f09f800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:28.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.778+0000 7f09f6ffd640 1 -- 172.21.15.86:0/1331980294 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f800cdb0 con 0x7f0a00198590 2024-04-21T08:40:28.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.778+0000 7f09f6ffd640 1 -- 172.21.15.86:0/1331980294 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09f8005020 con 0x7f0a00198590 2024-04-21T08:40:28.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.778+0000 7f0a0806d640 1 -- 172.21.15.86:0/1331980294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0a001993a0 con 0x7f0a00198590 2024-04-21T08:40:28.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.778+0000 7f09f6ffd640 1 -- 172.21.15.86:0/1331980294 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f80116d0 con 0x7f0a00198590 2024-04-21T08:40:28.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.778+0000 7f0a0806d640 1 -- 172.21.15.86:0/1331980294 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0a001a55f0 con 0x7f0a00198590 2024-04-21T08:40:28.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.780+0000 7f09f6ffd640 1 -- 172.21.15.86:0/1331980294 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f09f80051d0 con 0x7f0a00198590 2024-04-21T08:40:28.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.780+0000 7f0a0ac66740 1 Processor -- start 2024-04-21T08:40:28.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.781+0000 7f0a0ac66740 1 -- start start 2024-04-21T08:40:28.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.781+0000 7f0a0ac66740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ded3e17330 0x55ded3e17710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.781+0000 7f0a0ac66740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded3e17ce0 0x55ded3e20820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.781+0000 7f0a0ac66740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ded3e20e40 0x55ded3e212c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.781+0000 7f0a0ac66740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ded3d7c1f0 con 0x55ded3e17330 2024-04-21T08:40:28.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.781+0000 7f0a0ac66740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55ded3d194b0 con 0x55ded3e20e40 2024-04-21T08:40:28.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.781+0000 7f0a0ac66740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ded3d781b0 con 0x55ded3e17ce0 2024-04-21T08:40:28.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.781+0000 7f0a05608640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ded3e17330 0x55ded3e17710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.781+0000 7f0a05608640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ded3e17330 0x55ded3e17710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36142/0 (socket says 172.21.15.86:36142) 2024-04-21T08:40:28.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.781+0000 7f0a05608640 1 -- 172.21.15.86:0/3461859070 learned_addr learned my addr 172.21.15.86:0/3461859070 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:28.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.781+0000 7f0a05e09640 1 --2- 172.21.15.86:0/3461859070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded3e17ce0 0x55ded3e20820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.781+0000 7f09f6ffd640 1 --2- 172.21.15.86:0/1331980294 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f09ec0a3d20 0x7f09ec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.781+0000 7f0a05608640 1 --2- 172.21.15.86:0/3461859070 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ded3e20e40 0x55ded3e212c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.781+0000 7f0a0660a640 1 --2- 172.21.15.86:0/1331980294 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f09ec0a3d20 0x7f09ec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.781+0000 7f0a05608640 1 -- 172.21.15.86:0/3461859070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded3e17ce0 msgr2=0x55ded3e20820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.781+0000 7f0a05608640 1 --2- 172.21.15.86:0/3461859070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded3e17ce0 0x55ded3e20820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.781+0000 7f0a05608640 1 -- 172.21.15.86:0/3461859070 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ded3e20e40 msgr2=0x55ded3e212c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.781+0000 7f0a05608640 1 --2- 172.21.15.86:0/3461859070 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ded3e20e40 0x55ded3e212c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.781+0000 7f0a05608640 1 -- 172.21.15.86:0/3461859070 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ded3d0b1b0 con 0x55ded3e17330 2024-04-21T08:40:28.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.781+0000 7f0a05e09640 1 --2- 172.21.15.86:0/3461859070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded3e17ce0 0x55ded3e20820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:40:28.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.781+0000 7f09f6ffd640 1 -- 172.21.15.86:0/1331980294 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f09f80d0cb0 con 0x7f0a00198590 2024-04-21T08:40:28.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.782+0000 7f0a05608640 1 --2- 172.21.15.86:0/3461859070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ded3e17330 0x55ded3e17710 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f09f0005c40 tx=0x7f09f0005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea218688558c3d9b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:28.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.782+0000 7f0a0660a640 1 --2- 172.21.15.86:0/1331980294 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f09ec0a3d20 0x7f09ec0a61e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f09f809c870 tx=0x7f09f801c040 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:28.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.782+0000 7f09ebfff640 1 -- 172.21.15.86:0/3461859070 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f002c7e0 con 0x55ded3e17330 2024-04-21T08:40:28.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.782+0000 7f09ebfff640 1 -- 172.21.15.86:0/3461859070 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09f0048070 con 0x55ded3e17330 2024-04-21T08:40:28.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.782+0000 7f09ebfff640 1 -- 172.21.15.86:0/3461859070 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f0004900 con 0x55ded3e17330 2024-04-21T08:40:28.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.783+0000 7f0a0ac66740 1 -- 172.21.15.86:0/3461859070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ded3e17330 msgr2=0x55ded3e17710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.783+0000 7f0a0ac66740 1 --2- 172.21.15.86:0/3461859070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ded3e17330 0x55ded3e17710 secure :-1 s=READY pgs=791 cs=0 l=1 rev1=1 crypto rx=0x7f09f0005c40 tx=0x7f09f0005bd0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.783+0000 7f0a0ac66740 1 -- 172.21.15.86:0/3461859070 shutdown_connections 2024-04-21T08:40:28.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.783+0000 7f0a0ac66740 1 --2- 172.21.15.86:0/3461859070 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ded3e20e40 0x55ded3e212c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.783+0000 7f0a0ac66740 1 --2- 172.21.15.86:0/3461859070 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded3e17ce0 0x55ded3e20820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.783+0000 7f0a0ac66740 1 --2- 172.21.15.86:0/3461859070 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ded3e17330 0x55ded3e17710 unknown :-1 s=CLOSED pgs=791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.783+0000 7f0a0ac66740 1 -- 172.21.15.86:0/3461859070 >> 172.21.15.86:0/3461859070 conn(0x55ded3e11ba0 msgr2=0x55ded3e11f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:28.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.783+0000 7f0a0ac66740 1 -- 172.21.15.86:0/3461859070 shutdown_connections 2024-04-21T08:40:28.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.783+0000 7f0a0ac66740 1 -- 172.21.15.86:0/3461859070 wait complete. 2024-04-21T08:40:28.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.783+0000 7f0a0ac66740 1 -- start start 2024-04-21T08:40:28.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.783+0000 7f0a0ac66740 1 Processor -- start 2024-04-21T08:40:28.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.783+0000 7f0a0ac66740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ded3e17330 0x55ded3e27270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.783+0000 7f0a0ac66740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded3e17ce0 0x55ded3e24600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.783+0000 7f0a0ac66740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ded3e20e40 0x55ded3e24ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.784+0000 7f0a0ac66740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ded3e27d60 con 0x55ded3e20e40 2024-04-21T08:40:28.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.784+0000 7f0a0ac66740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55ded3d35500 con 0x55ded3e17330 2024-04-21T08:40:28.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.784+0000 7f0a0ac66740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ded3e27ee0 con 0x55ded3e17ce0 2024-04-21T08:40:28.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.784+0000 7f0a0ac66740 10 client.0 _ll_register_callbacks cb 0x55ded3d03b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:28.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.784+0000 7f0a05608640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ded3e20e40 0x55ded3e24ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.784+0000 7f0a05608640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ded3e20e40 0x55ded3e24ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36152/0 (socket says 172.21.15.86:36152) 2024-04-21T08:40:28.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.784+0000 7f0a05608640 1 -- 172.21.15.86:0/3633350845 learned_addr learned my addr 172.21.15.86:0/3633350845 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:28.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.784+0000 7f0a05608640 1 --2- 172.21.15.86:0/3633350845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ded3e17330 0x55ded3e27270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.784+0000 7f0a05e09640 1 --2- 172.21.15.86:0/3633350845 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded3e17ce0 0x55ded3e24600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.784+0000 7f0a05608640 1 -- 172.21.15.86:0/3633350845 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded3e17ce0 msgr2=0x55ded3e24600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.784+0000 7f0a05608640 1 --2- 172.21.15.86:0/3633350845 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded3e17ce0 0x55ded3e24600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.784+0000 7f0a05608640 1 -- 172.21.15.86:0/3633350845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ded3e17330 msgr2=0x55ded3e27270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.784+0000 7f0a05608640 1 --2- 172.21.15.86:0/3633350845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ded3e17330 0x55ded3e27270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.784+0000 7f0a05608640 1 -- 172.21.15.86:0/3633350845 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ded3a8e3f0 con 0x55ded3e20e40 2024-04-21T08:40:28.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.784+0000 7f0a05e09640 1 --2- 172.21.15.86:0/3633350845 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded3e17ce0 0x55ded3e24600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:28.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.784+0000 7f0a05608640 1 --2- 172.21.15.86:0/3633350845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ded3e20e40 0x55ded3e24ba0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f09f0005910 tx=0x7f09f0005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2b3f1266c6a674c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:28.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.784+0000 7f09e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:40:28.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.784+0000 7f09e97fa640 1 -- 172.21.15.86:0/3633350845 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f0051030 con 0x55ded3e20e40 2024-04-21T08:40:28.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.784+0000 7f09e97fa640 1 -- 172.21.15.86:0/3633350845 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f09f0005240 con 0x55ded3e20e40 2024-04-21T08:40:28.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.784+0000 7f0a0ac66740 1 -- 172.21.15.86:0/3633350845 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09f0048070 con 0x55ded3e20e40 2024-04-21T08:40:28.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.784+0000 7f09e97fa640 1 -- 172.21.15.86:0/3633350845 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f09f0061050 con 0x55ded3e20e40 2024-04-21T08:40:28.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.784+0000 7f0a0ac66740 10 client.16536 fetch_fsmap learned FSMap version 29 2024-04-21T08:40:28.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.784+0000 7f0a0ac66740 1 -- 172.21.15.86:0/3633350845 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ded3b587d0 con 0x55ded3e20e40 2024-04-21T08:40:28.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.785+0000 7f09e97fa640 1 -- 172.21.15.86:0/3633350845 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09f0048070 con 0x55ded3e20e40 2024-04-21T08:40:28.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.785+0000 7f09e97fa640 1 -- 172.21.15.86:0/3633350845 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f09f0003820 con 0x55ded3e20e40 2024-04-21T08:40:28.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.785+0000 7f0a0ac66740 10 client.16536 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:40:28.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.785+0000 7f0a0ac66740 10 client.16536 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:40:28.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.785+0000 7f0a0ac66740 20 client.16536 populate_metadata read hostname 'smithi086' 2024-04-21T08:40:28.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.785+0000 7f0a0ac66740 1 --2- 172.21.15.86:0/3633350845 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55ded3e34ed0 0x55ded3e372c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.785+0000 7f0a0ac66740 4 client.16536 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:40:28.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.785+0000 7f0a0ac66740 1 -- 172.21.15.86:0/3633350845 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ded3d1b930 con 0x55ded3e34ed0 2024-04-21T08:40:28.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.785+0000 7f0a05e09640 1 --2- 172.21.15.86:0/3633350845 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55ded3e34ed0 0x55ded3e372c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.785+0000 7f09e97fa640 10 client.16536 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:40:28.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.785+0000 7f0a05e09640 1 --2- 172.21.15.86:0/3633350845 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55ded3e34ed0 0x55ded3e372c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=73dfb3b877a31bef server_cookie=678003006c8bd20e in_seq=0 out_seq=0 2024-04-21T08:40:28.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.786+0000 7f09e97fa640 1 -- 172.21.15.86:0/3633350845 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ded3d1b930 con 0x55ded3e34ed0 2024-04-21T08:40:28.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.786+0000 7f09e97fa640 10 client.16536 handle_command_reply: tid=0 2024-04-21T08:40:28.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.786+0000 7f0a0ac66740 1 client.16536 shutdown 2024-04-21T08:40:28.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.786+0000 7f0a0ac66740 1 -- 172.21.15.86:0/3633350845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ded3e20e40 msgr2=0x55ded3e24ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.786+0000 7f0a0ac66740 1 --2- 172.21.15.86:0/3633350845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ded3e20e40 0x55ded3e24ba0 secure :-1 s=READY pgs=792 cs=0 l=1 rev1=1 crypto rx=0x7f09f0005910 tx=0x7f09f0005620 comp rx=0 tx=0).stop 2024-04-21T08:40:28.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.786+0000 7f0a0ac66740 1 -- 172.21.15.86:0/3633350845 shutdown_connections 2024-04-21T08:40:28.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.786+0000 7f0a0ac66740 1 --2- 172.21.15.86:0/3633350845 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55ded3e34ed0 0x55ded3e372c0 crc :-1 s=READY pgs=39 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.786+0000 7f0a0ac66740 1 --2- 172.21.15.86:0/3633350845 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ded3e20e40 0x55ded3e24ba0 unknown :-1 s=CLOSED pgs=792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.786+0000 7f0a0ac66740 1 --2- 172.21.15.86:0/3633350845 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded3e17ce0 0x55ded3e24600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.786+0000 7f0a0ac66740 1 --2- 172.21.15.86:0/3633350845 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ded3e17330 0x55ded3e27270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.786+0000 7f09e97fa640 0 client.16536 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:40:28.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.786+0000 7f0a0ac66740 1 -- 172.21.15.86:0/3633350845 >> 172.21.15.86:0/3633350845 conn(0x55ded3e11ba0 msgr2=0x55ded3e138f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:28.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.786+0000 7f0a0ac66740 1 -- 172.21.15.86:0/3633350845 shutdown_connections 2024-04-21T08:40:28.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.786+0000 7f0a0ac66740 1 -- 172.21.15.86:0/3633350845 wait complete. 2024-04-21T08:40:28.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.786+0000 7f0a0ac66740 20 client.16536 trim_cache size 0 max 16384 2024-04-21T08:40:28.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.799+0000 7f0a0ac66740 1 Processor -- start 2024-04-21T08:40:28.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.800+0000 7f0a0ac66740 1 -- start start 2024-04-21T08:40:28.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.800+0000 7f0a0ac66740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ded3e17330 0x55ded3e22230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.800+0000 7f0a0ac66740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ded3e17ce0 0x55ded3e26d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.800+0000 7f0a0ac66740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded3e20e40 0x55ded3e27270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.800+0000 7f0a0ac66740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0a001acff0 con 0x55ded3e17ce0 2024-04-21T08:40:28.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.800+0000 7f0a0ac66740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0a001aeb30 con 0x55ded3e17330 2024-04-21T08:40:28.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.800+0000 7f0a0ac66740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0a001ad6c0 con 0x55ded3e20e40 2024-04-21T08:40:28.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.800+0000 7f0a05e09640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ded3e17ce0 0x55ded3e26d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.800+0000 7f0a05e09640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ded3e17ce0 0x55ded3e26d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36160/0 (socket says 172.21.15.86:36160) 2024-04-21T08:40:28.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.800+0000 7f0a05e09640 1 -- 172.21.15.86:0/1377651431 learned_addr learned my addr 172.21.15.86:0/1377651431 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:28.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.800+0000 7f0a05e09640 1 -- 172.21.15.86:0/1377651431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded3e20e40 msgr2=0x55ded3e27270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.800+0000 7f0a05e09640 1 --2- 172.21.15.86:0/1377651431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded3e20e40 0x55ded3e27270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.800+0000 7f0a05e09640 1 -- 172.21.15.86:0/1377651431 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ded3e17330 msgr2=0x55ded3e22230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.800+0000 7f0a05e09640 1 --2- 172.21.15.86:0/1377651431 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ded3e17330 0x55ded3e22230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.800+0000 7f0a05e09640 1 -- 172.21.15.86:0/1377651431 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ded3e0c070 con 0x55ded3e17ce0 2024-04-21T08:40:28.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.800+0000 7f0a05e09640 1 --2- 172.21.15.86:0/1377651431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ded3e17ce0 0x55ded3e26d30 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f09fc005d00 tx=0x7f09fc00b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4992cbf0e2cc98bb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:28.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.800+0000 7f09eb7fe640 1 -- 172.21.15.86:0/1377651431 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09fc004220 con 0x55ded3e17ce0 2024-04-21T08:40:28.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.800+0000 7f09eb7fe640 1 -- 172.21.15.86:0/1377651431 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09fc023070 con 0x55ded3e17ce0 2024-04-21T08:40:28.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.800+0000 7f09eb7fe640 1 -- 172.21.15.86:0/1377651431 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09fc0055e0 con 0x55ded3e17ce0 2024-04-21T08:40:28.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.801+0000 7f0a0ac66740 1 -- 172.21.15.86:0/1377651431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ded3e17ce0 msgr2=0x55ded3e26d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.801+0000 7f0a0ac66740 1 --2- 172.21.15.86:0/1377651431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ded3e17ce0 0x55ded3e26d30 secure :-1 s=READY pgs=793 cs=0 l=1 rev1=1 crypto rx=0x7f09fc005d00 tx=0x7f09fc00b410 comp rx=0 tx=0).stop 2024-04-21T08:40:28.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.801+0000 7f0a0ac66740 1 -- 172.21.15.86:0/1377651431 shutdown_connections 2024-04-21T08:40:28.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.801+0000 7f0a0ac66740 1 --2- 172.21.15.86:0/1377651431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded3e20e40 0x55ded3e27270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.801+0000 7f0a0ac66740 1 --2- 172.21.15.86:0/1377651431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ded3e17ce0 0x55ded3e26d30 unknown :-1 s=CLOSED pgs=793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.801+0000 7f0a0ac66740 1 --2- 172.21.15.86:0/1377651431 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ded3e17330 0x55ded3e22230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.801+0000 7f0a0ac66740 1 -- 172.21.15.86:0/1377651431 >> 172.21.15.86:0/1377651431 conn(0x55ded3e11ba0 msgr2=0x55ded3e30a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:28.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.801+0000 7f0a0ac66740 1 -- 172.21.15.86:0/1377651431 shutdown_connections 2024-04-21T08:40:28.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.801+0000 7f0a0ac66740 1 -- 172.21.15.86:0/1377651431 wait complete. 2024-04-21T08:40:28.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.801+0000 7f0a0ac66740 1 -- start start 2024-04-21T08:40:28.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.801+0000 7f0a0ac66740 1 Processor -- start 2024-04-21T08:40:28.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.802+0000 7f0a0ac66740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded3e17330 0x55ded3e38550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.802+0000 7f0a0ac66740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ded3e17ce0 0x55ded3e38a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.802+0000 7f0a0ac66740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ded3e20e40 0x55ded3e94080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.802+0000 7f0a0ac66740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0a001ae460 con 0x55ded3e20e40 2024-04-21T08:40:28.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.802+0000 7f0a0ac66740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0a001add90 con 0x55ded3e17ce0 2024-04-21T08:40:28.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.802+0000 7f0a0ac66740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0a001aafb0 con 0x55ded3e17330 2024-04-21T08:40:28.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.802+0000 7f0a0ac66740 10 client.0 _ll_register_callbacks cb 0x7f09f00047f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:28.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.802+0000 7f0a05608640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ded3e20e40 0x55ded3e94080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.802+0000 7f0a05608640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ded3e20e40 0x55ded3e94080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36162/0 (socket says 172.21.15.86:36162) 2024-04-21T08:40:28.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.802+0000 7f0a05608640 1 -- 172.21.15.86:0/3869095521 learned_addr learned my addr 172.21.15.86:0/3869095521 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:28.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.802+0000 7f0a05e09640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ded3e17ce0 0x55ded3e38a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.802+0000 7f0a05608640 1 --2- 172.21.15.86:0/3869095521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded3e17330 0x55ded3e38550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.802+0000 7f0a05608640 1 -- 172.21.15.86:0/3869095521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded3e17330 msgr2=0x55ded3e38550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.802+0000 7f0a05608640 1 --2- 172.21.15.86:0/3869095521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded3e17330 0x55ded3e38550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.802+0000 7f0a05608640 1 -- 172.21.15.86:0/3869095521 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ded3e17ce0 msgr2=0x55ded3e38a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.802+0000 7f0a05608640 1 --2- 172.21.15.86:0/3869095521 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ded3e17ce0 0x55ded3e38a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.802+0000 7f0a05608640 1 -- 172.21.15.86:0/3869095521 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ded3e94670 con 0x55ded3e20e40 2024-04-21T08:40:28.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.802+0000 7f0a05608640 1 --2- 172.21.15.86:0/3869095521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ded3e20e40 0x55ded3e94080 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f09f00041f0 tx=0x7f09f0005a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d7a0bbe59808d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:28.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.802+0000 7f09e97fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:40:28.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.802+0000 7f09e97fa640 1 -- 172.21.15.86:0/3869095521 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f09f0051040 con 0x55ded3e20e40 2024-04-21T08:40:28.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.802+0000 7f09e97fa640 1 -- 172.21.15.86:0/3869095521 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f09f0003e10 con 0x55ded3e20e40 2024-04-21T08:40:28.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.802+0000 7f0a0ac66740 1 -- 172.21.15.86:0/3869095521 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f09fc023070 con 0x55ded3e20e40 2024-04-21T08:40:28.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.802+0000 7f09e97fa640 1 -- 172.21.15.86:0/3869095521 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f09f002c5d0 con 0x55ded3e20e40 2024-04-21T08:40:28.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.802+0000 7f0a0ac66740 10 client.16542 fetch_fsmap learned FSMap version 29 2024-04-21T08:40:28.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.802+0000 7f0a0ac66740 1 -- 172.21.15.86:0/3869095521 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ded3e9b9f0 con 0x55ded3e20e40 2024-04-21T08:40:28.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.803+0000 7f09e97fa640 1 -- 172.21.15.86:0/3869095521 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f09fc023070 con 0x55ded3e20e40 2024-04-21T08:40:28.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.803+0000 7f09e97fa640 1 -- 172.21.15.86:0/3869095521 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f09f0004f20 con 0x55ded3e20e40 2024-04-21T08:40:28.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.803+0000 7f0a0ac66740 10 client.16542 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:40:28.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.803+0000 7f0a0ac66740 10 client.16542 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:40:28.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.803+0000 7f0a0ac66740 20 client.16542 populate_metadata read hostname 'smithi086' 2024-04-21T08:40:28.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.803+0000 7f0a0ac66740 1 --2- 172.21.15.86:0/3869095521 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55ded3e9c110 0x55ded3e9e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.803+0000 7f0a0ac66740 4 client.16542 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:40:28.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.803+0000 7f0a0ac66740 1 -- 172.21.15.86:0/3869095521 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ded3d89c10 con 0x55ded3e9c110 2024-04-21T08:40:28.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.803+0000 7f0a05e09640 1 --2- 172.21.15.86:0/3869095521 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55ded3e9c110 0x55ded3e9e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.803+0000 7f09e97fa640 10 client.16542 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:40:28.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.803+0000 7f0a05e09640 1 --2- 172.21.15.86:0/3869095521 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55ded3e9c110 0x55ded3e9e530 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=ed232d798c741700 server_cookie=d2e2188f27efaf23 in_seq=0 out_seq=0 2024-04-21T08:40:28.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.804+0000 7f09e97fa640 1 -- 172.21.15.86:0/3869095521 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ded3d89c10 con 0x55ded3e9c110 2024-04-21T08:40:28.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.804+0000 7f09e97fa640 10 client.16542 handle_command_reply: tid=0 2024-04-21T08:40:28.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.804+0000 7f0a0ac66740 1 client.16542 shutdown 2024-04-21T08:40:28.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.804+0000 7f0a0ac66740 1 -- 172.21.15.86:0/3869095521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ded3e20e40 msgr2=0x55ded3e94080 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.804+0000 7f0a0ac66740 1 --2- 172.21.15.86:0/3869095521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ded3e20e40 0x55ded3e94080 secure :-1 s=READY pgs=794 cs=0 l=1 rev1=1 crypto rx=0x7f09f00041f0 tx=0x7f09f0005a40 comp rx=0 tx=0).stop 2024-04-21T08:40:28.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.804+0000 7f0a0ac66740 1 -- 172.21.15.86:0/3869095521 shutdown_connections 2024-04-21T08:40:28.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.804+0000 7f0a0ac66740 1 --2- 172.21.15.86:0/3869095521 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55ded3e9c110 0x55ded3e9e530 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.804+0000 7f0a0ac66740 1 --2- 172.21.15.86:0/3869095521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ded3e20e40 0x55ded3e94080 unknown :-1 s=CLOSED pgs=794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.804+0000 7f0a0ac66740 1 --2- 172.21.15.86:0/3869095521 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ded3e17ce0 0x55ded3e38a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.804+0000 7f0a0ac66740 1 --2- 172.21.15.86:0/3869095521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ded3e17330 0x55ded3e38550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.804+0000 7f09e97fa640 0 client.16542 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:40:28.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.804+0000 7f0a0ac66740 1 -- 172.21.15.86:0/3869095521 >> 172.21.15.86:0/3869095521 conn(0x55ded3e11ba0 msgr2=0x55ded3e10370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:28.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.804+0000 7f0a0ac66740 1 -- 172.21.15.86:0/3869095521 shutdown_connections 2024-04-21T08:40:28.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.804+0000 7f0a0ac66740 1 -- 172.21.15.86:0/3869095521 wait complete. 2024-04-21T08:40:28.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.804+0000 7f0a0ac66740 20 client.16542 trim_cache size 0 max 16384 2024-04-21T08:40:28.811 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:40:28.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.804+0000 7f0a0806d640 1 -- 172.21.15.86:0/1331980294 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f09ec0a3d20 msgr2=0x7f09ec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.804+0000 7f0a0806d640 1 --2- 172.21.15.86:0/1331980294 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f09ec0a3d20 0x7f09ec0a61e0 secure :-1 s=READY pgs=112 cs=0 l=1 rev1=1 crypto rx=0x7f09f809c870 tx=0x7f09f801c040 comp rx=0 tx=0).stop 2024-04-21T08:40:28.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.804+0000 7f0a0806d640 1 -- 172.21.15.86:0/1331980294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a00198590 msgr2=0x7f0a00198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.804+0000 7f0a0806d640 1 --2- 172.21.15.86:0/1331980294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a00198590 0x7f0a00198a40 secure :-1 s=READY pgs=790 cs=0 l=1 rev1=1 crypto rx=0x7f09f800eaa0 tx=0x7f09f800ef60 comp rx=0 tx=0).stop 2024-04-21T08:40:28.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.804+0000 7f0a0806d640 1 -- 172.21.15.86:0/1331980294 shutdown_connections 2024-04-21T08:40:28.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.804+0000 7f0a0806d640 1 --2- 172.21.15.86:0/1331980294 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f09ec0a3d20 0x7f09ec0a61e0 unknown :-1 s=CLOSED pgs=112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.804+0000 7f0a0806d640 1 --2- 172.21.15.86:0/1331980294 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0a00198590 0x7f0a00198a40 unknown :-1 s=CLOSED pgs=790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.804+0000 7f0a0806d640 1 --2- 172.21.15.86:0/1331980294 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0a00107270 0x7f0a0019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.804+0000 7f0a0806d640 1 --2- 172.21.15.86:0/1331980294 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0a0006a880 0x7f0a0019e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.804+0000 7f0a0806d640 1 -- 172.21.15.86:0/1331980294 >> 172.21.15.86:0/1331980294 conn(0x7f0a00077eb0 msgr2=0x7f0a00108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:28.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.805+0000 7f0a0806d640 1 -- 172.21.15.86:0/1331980294 shutdown_connections 2024-04-21T08:40:28.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.805+0000 7f0a0806d640 1 -- 172.21.15.86:0/1331980294 wait complete. 2024-04-21T08:40:28.815 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:40:28.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.916+0000 7f12bfaba640 1 -- 172.21.15.86:0/1912926854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b8107250 msgr2=0x7f12b810ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.916+0000 7f12bfaba640 1 --2- 172.21.15.86:0/1912926854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b8107250 0x7f12b810ee20 secure :-1 s=READY pgs=795 cs=0 l=1 rev1=1 crypto rx=0x7f12b0009ca0 tx=0x7f12b002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.916+0000 7f12bfaba640 1 -- 172.21.15.86:0/1912926854 shutdown_connections 2024-04-21T08:40:28.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.916+0000 7f12bfaba640 1 --2- 172.21.15.86:0/1912926854 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b8107250 0x7f12b810ee20 unknown :-1 s=CLOSED pgs=795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.916+0000 7f12bfaba640 1 --2- 172.21.15.86:0/1912926854 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12b8102b30 0x7f12b8106b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.916+0000 7f12bfaba640 1 --2- 172.21.15.86:0/1912926854 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12b8102180 0x7f12b8102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.917+0000 7f12bfaba640 1 -- 172.21.15.86:0/1912926854 >> 172.21.15.86:0/1912926854 conn(0x7f12b80fd9c0 msgr2=0x7f12b80ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:28.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.917+0000 7f12bfaba640 1 -- 172.21.15.86:0/1912926854 shutdown_connections 2024-04-21T08:40:28.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.917+0000 7f12bfaba640 1 -- 172.21.15.86:0/1912926854 wait complete. 2024-04-21T08:40:28.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.917+0000 7f12bfaba640 1 Processor -- start 2024-04-21T08:40:28.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.917+0000 7f12bfaba640 1 -- start start 2024-04-21T08:40:28.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.917+0000 7f12bfaba640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12b8102180 0x7f12b8198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.917+0000 7f12bfaba640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b8107250 0x7f12b8198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.917+0000 7f12bfaba640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12b819cfd0 0x7f12b819d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.917+0000 7f12bfaba640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f12b81121a0 con 0x7f12b8107250 2024-04-21T08:40:28.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.917+0000 7f12bfaba640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f12b8112020 con 0x7f12b8102180 2024-04-21T08:40:28.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.917+0000 7f12bfaba640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f12b8112320 con 0x7f12b819cfd0 2024-04-21T08:40:28.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.917+0000 7f12bd055640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b8107250 0x7f12b8198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.918+0000 7f12bd856640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12b8102180 0x7f12b8198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.918+0000 7f12bd055640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b8107250 0x7f12b8198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36184/0 (socket says 172.21.15.86:36184) 2024-04-21T08:40:28.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.918+0000 7f12bd055640 1 -- 172.21.15.86:0/3404589049 learned_addr learned my addr 172.21.15.86:0/3404589049 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:28.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.918+0000 7f12be057640 1 --2- 172.21.15.86:0/3404589049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12b819cfd0 0x7f12b819d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.918+0000 7f12bd055640 1 -- 172.21.15.86:0/3404589049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12b819cfd0 msgr2=0x7f12b819d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.918+0000 7f12bd055640 1 --2- 172.21.15.86:0/3404589049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12b819cfd0 0x7f12b819d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.918+0000 7f12bd055640 1 -- 172.21.15.86:0/3404589049 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12b8102180 msgr2=0x7f12b8198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.918+0000 7f12bd055640 1 --2- 172.21.15.86:0/3404589049 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12b8102180 0x7f12b8198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.918+0000 7f12bd055640 1 -- 172.21.15.86:0/3404589049 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12b819dac0 con 0x7f12b8107250 2024-04-21T08:40:28.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.918+0000 7f12bd055640 1 --2- 172.21.15.86:0/3404589049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b8107250 0x7f12b8198ca0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f12a800d770 tx=0x7f12a800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:28.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.918+0000 7f12aeffd640 1 -- 172.21.15.86:0/3404589049 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a8017070 con 0x7f12b8107250 2024-04-21T08:40:28.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.918+0000 7f12bfaba640 1 -- 172.21.15.86:0/3404589049 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12b819ddb0 con 0x7f12b8107250 2024-04-21T08:40:28.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.918+0000 7f12aeffd640 1 -- 172.21.15.86:0/3404589049 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12a8009c30 con 0x7f12b8107250 2024-04-21T08:40:28.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.918+0000 7f12aeffd640 1 -- 172.21.15.86:0/3404589049 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a800f620 con 0x7f12b8107250 2024-04-21T08:40:28.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.918+0000 7f12bfaba640 1 -- 172.21.15.86:0/3404589049 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12b81a55a0 con 0x7f12b8107250 2024-04-21T08:40:28.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.920+0000 7f12aeffd640 1 -- 172.21.15.86:0/3404589049 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f12a801a070 con 0x7f12b8107250 2024-04-21T08:40:28.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.921+0000 7f12c2783740 1 Processor -- start 2024-04-21T08:40:28.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.921+0000 7f12c2783740 1 -- start start 2024-04-21T08:40:28.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.921+0000 7f12c2783740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55837a5193b0 0x55837a519790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.921+0000 7f12c2783740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55837a519d60 0x55837a522890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.921+0000 7f12c2783740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55837a522e60 0x55837a5232c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.921+0000 7f12c2783740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55837a47e1f0 con 0x55837a519d60 2024-04-21T08:40:28.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.921+0000 7f12c2783740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55837a41b4b0 con 0x55837a5193b0 2024-04-21T08:40:28.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.921+0000 7f12c2783740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55837a47a1b0 con 0x55837a522e60 2024-04-21T08:40:28.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.921+0000 7f12aeffd640 1 --2- 172.21.15.86:0/3404589049 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f12a40a3d20 0x7f12a40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.921+0000 7f12bd856640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55837a519d60 0x55837a522890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.921+0000 7f12be057640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55837a5193b0 0x55837a519790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.921+0000 7f12be057640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55837a5193b0 0x55837a519790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.86:57048/0 (socket says 172.21.15.86:57048) 2024-04-21T08:40:28.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.921+0000 7f12bd856640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55837a519d60 0x55837a522890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36186/0 (socket says 172.21.15.86:36186) 2024-04-21T08:40:28.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.921+0000 7f12be057640 1 -- 172.21.15.86:0/3767495810 learned_addr learned my addr 172.21.15.86:0/3767495810 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:28.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.921+0000 7f12bd055640 1 --2- 172.21.15.86:0/3767495810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55837a522e60 0x55837a5232c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.921+0000 7f12be057640 1 --2- 172.21.15.86:0/3404589049 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f12a40a3d20 0x7f12a40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.921+0000 7f12aeffd640 1 -- 172.21.15.86:0/3404589049 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f12a80d05a0 con 0x7f12b8107250 2024-04-21T08:40:28.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.922+0000 7f12bd856640 1 -- 172.21.15.86:0/3767495810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55837a522e60 msgr2=0x55837a5232c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.922+0000 7f12bd856640 1 --2- 172.21.15.86:0/3767495810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55837a522e60 0x55837a5232c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.922+0000 7f12bd856640 1 -- 172.21.15.86:0/3767495810 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55837a5193b0 msgr2=0x55837a519790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.922+0000 7f12bd856640 1 --2- 172.21.15.86:0/3767495810 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55837a5193b0 0x55837a519790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.922+0000 7f12bd856640 1 -- 172.21.15.86:0/3767495810 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55837a40d1b0 con 0x55837a519d60 2024-04-21T08:40:28.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.922+0000 7f12be057640 1 --2- 172.21.15.86:0/3767495810 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55837a5193b0 0x55837a519790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:40:28.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.922+0000 7f12bd856640 1 --2- 172.21.15.86:0/3767495810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55837a519d60 0x55837a522890 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f12b4009c20 tx=0x7f12b4011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=da6ac9ba481c2ff1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:28.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.922+0000 7f12be057640 1 --2- 172.21.15.86:0/3404589049 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f12a40a3d20 0x7f12a40a61e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f12b002cfd0 tx=0x7f12b0005d80 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:28.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.922+0000 7f12a3fff640 1 -- 172.21.15.86:0/3767495810 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b4004050 con 0x55837a519d60 2024-04-21T08:40:28.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.922+0000 7f12a3fff640 1 -- 172.21.15.86:0/3767495810 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12b40054d0 con 0x55837a519d60 2024-04-21T08:40:28.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.922+0000 7f12a3fff640 1 -- 172.21.15.86:0/3767495810 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b4004c20 con 0x55837a519d60 2024-04-21T08:40:28.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.923+0000 7f12c2783740 1 -- 172.21.15.86:0/3767495810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55837a519d60 msgr2=0x55837a522890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.923+0000 7f12c2783740 1 --2- 172.21.15.86:0/3767495810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55837a519d60 0x55837a522890 secure :-1 s=READY pgs=797 cs=0 l=1 rev1=1 crypto rx=0x7f12b4009c20 tx=0x7f12b4011810 comp rx=0 tx=0).stop 2024-04-21T08:40:28.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.923+0000 7f12c2783740 1 -- 172.21.15.86:0/3767495810 shutdown_connections 2024-04-21T08:40:28.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.923+0000 7f12c2783740 1 --2- 172.21.15.86:0/3767495810 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55837a522e60 0x55837a5232c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.923+0000 7f12c2783740 1 --2- 172.21.15.86:0/3767495810 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55837a519d60 0x55837a522890 unknown :-1 s=CLOSED pgs=797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.923+0000 7f12c2783740 1 --2- 172.21.15.86:0/3767495810 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55837a5193b0 0x55837a519790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.923+0000 7f12c2783740 1 -- 172.21.15.86:0/3767495810 >> 172.21.15.86:0/3767495810 conn(0x55837a513ba0 msgr2=0x55837a513f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:28.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.923+0000 7f12c2783740 1 -- 172.21.15.86:0/3767495810 shutdown_connections 2024-04-21T08:40:28.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.923+0000 7f12c2783740 1 -- 172.21.15.86:0/3767495810 wait complete. 2024-04-21T08:40:28.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.923+0000 7f12c2783740 1 -- start start 2024-04-21T08:40:28.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.923+0000 7f12c2783740 1 Processor -- start 2024-04-21T08:40:28.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.924+0000 7f12c2783740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55837a5193b0 0x55837a529210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.924+0000 7f12c2783740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55837a519d60 0x55837a5265a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.924+0000 7f12c2783740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55837a522e60 0x55837a526ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.924+0000 7f12c2783740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55837a529da0 con 0x55837a5193b0 2024-04-21T08:40:28.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.924+0000 7f12c2783740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55837a437500 con 0x55837a522e60 2024-04-21T08:40:28.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.924+0000 7f12c2783740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55837a529f20 con 0x55837a519d60 2024-04-21T08:40:28.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.924+0000 7f12c2783740 10 client.0 _ll_register_callbacks cb 0x55837a405b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:28.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.924+0000 7f12bd856640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55837a5193b0 0x55837a529210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.924+0000 7f12bd856640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55837a5193b0 0x55837a529210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36196/0 (socket says 172.21.15.86:36196) 2024-04-21T08:40:28.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.924+0000 7f12bd856640 1 -- 172.21.15.86:0/1989111287 learned_addr learned my addr 172.21.15.86:0/1989111287 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:28.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.924+0000 7f12bd055640 1 --2- 172.21.15.86:0/1989111287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55837a519d60 0x55837a5265a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.924+0000 7f12be057640 1 --2- 172.21.15.86:0/1989111287 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55837a522e60 0x55837a526ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.924+0000 7f12bd856640 1 -- 172.21.15.86:0/1989111287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55837a519d60 msgr2=0x55837a5265a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.924+0000 7f12bd856640 1 --2- 172.21.15.86:0/1989111287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55837a519d60 0x55837a5265a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.924+0000 7f12bd856640 1 -- 172.21.15.86:0/1989111287 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55837a522e60 msgr2=0x55837a526ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.924+0000 7f12bd856640 1 --2- 172.21.15.86:0/1989111287 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55837a522e60 0x55837a526ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.924+0000 7f12bd856640 1 -- 172.21.15.86:0/1989111287 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55837a1903f0 con 0x55837a5193b0 2024-04-21T08:40:28.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.924+0000 7f12bd856640 1 --2- 172.21.15.86:0/1989111287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55837a5193b0 0x55837a529210 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f12b400a000 tx=0x7f12b4005680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef8abf0bbde1e3ab server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:28.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.924+0000 7f12a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:40:28.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.924+0000 7f12a17fa640 1 -- 172.21.15.86:0/1989111287 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b4029070 con 0x55837a5193b0 2024-04-21T08:40:28.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.924+0000 7f12a17fa640 1 -- 172.21.15.86:0/1989111287 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f12b4003a00 con 0x55837a5193b0 2024-04-21T08:40:28.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.924+0000 7f12c2783740 1 -- 172.21.15.86:0/1989111287 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12b40054d0 con 0x55837a5193b0 2024-04-21T08:40:28.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.925+0000 7f12a17fa640 1 -- 172.21.15.86:0/1989111287 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f12b4035050 con 0x55837a5193b0 2024-04-21T08:40:28.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.925+0000 7f12c2783740 10 client.16554 fetch_fsmap learned FSMap version 29 2024-04-21T08:40:28.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.925+0000 7f12c2783740 1 -- 172.21.15.86:0/1989111287 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55837a25a7d0 con 0x55837a5193b0 2024-04-21T08:40:28.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.925+0000 7f12a17fa640 1 -- 172.21.15.86:0/1989111287 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12b40054d0 con 0x55837a5193b0 2024-04-21T08:40:28.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.925+0000 7f12a17fa640 1 -- 172.21.15.86:0/1989111287 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f12b40051a0 con 0x55837a5193b0 2024-04-21T08:40:28.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.925+0000 7f12c2783740 10 client.16554 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:40:28.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.925+0000 7f12c2783740 10 client.16554 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:40:28.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.925+0000 7f12c2783740 20 client.16554 populate_metadata read hostname 'smithi086' 2024-04-21T08:40:28.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.925+0000 7f12c2783740 1 --2- 172.21.15.86:0/1989111287 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55837a536ea0 0x55837a539290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.925+0000 7f12c2783740 4 client.16554 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:40:28.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.925+0000 7f12c2783740 1 -- 172.21.15.86:0/1989111287 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55837a41d930 con 0x55837a536ea0 2024-04-21T08:40:28.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.925+0000 7f12bd055640 1 --2- 172.21.15.86:0/1989111287 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55837a536ea0 0x55837a539290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.926+0000 7f12bd055640 1 --2- 172.21.15.86:0/1989111287 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55837a536ea0 0x55837a539290 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ef73236fa26e147b server_cookie=a3e88548dd6d55c6 in_seq=0 out_seq=0 2024-04-21T08:40:28.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.926+0000 7f12a17fa640 10 client.16554 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:40:28.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.926+0000 7f12a17fa640 1 -- 172.21.15.86:0/1989111287 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55837a41d930 con 0x55837a536ea0 2024-04-21T08:40:28.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.926+0000 7f12a17fa640 10 client.16554 handle_command_reply: tid=0 2024-04-21T08:40:28.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.926+0000 7f12c2783740 1 client.16554 shutdown 2024-04-21T08:40:28.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.926+0000 7f12c2783740 1 -- 172.21.15.86:0/1989111287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55837a5193b0 msgr2=0x55837a529210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.926+0000 7f12c2783740 1 --2- 172.21.15.86:0/1989111287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55837a5193b0 0x55837a529210 secure :-1 s=READY pgs=798 cs=0 l=1 rev1=1 crypto rx=0x7f12b400a000 tx=0x7f12b4005680 comp rx=0 tx=0).stop 2024-04-21T08:40:28.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.926+0000 7f12c2783740 1 -- 172.21.15.86:0/1989111287 shutdown_connections 2024-04-21T08:40:28.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.926+0000 7f12c2783740 1 --2- 172.21.15.86:0/1989111287 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55837a522e60 0x55837a526ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.926+0000 7f12c2783740 1 --2- 172.21.15.86:0/1989111287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55837a519d60 0x55837a5265a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.926+0000 7f12c2783740 1 --2- 172.21.15.86:0/1989111287 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55837a536ea0 0x55837a539290 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.926+0000 7f12c2783740 1 --2- 172.21.15.86:0/1989111287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55837a5193b0 0x55837a529210 unknown :-1 s=CLOSED pgs=798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.926+0000 7f12a17fa640 0 client.16554 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:40:28.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.926+0000 7f12c2783740 1 -- 172.21.15.86:0/1989111287 >> 172.21.15.86:0/1989111287 conn(0x55837a513ba0 msgr2=0x55837a5158f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:28.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.927+0000 7f12c2783740 1 -- 172.21.15.86:0/1989111287 shutdown_connections 2024-04-21T08:40:28.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.927+0000 7f12c2783740 1 -- 172.21.15.86:0/1989111287 wait complete. 2024-04-21T08:40:28.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.927+0000 7f12c2783740 20 client.16554 trim_cache size 0 max 16384 2024-04-21T08:40:28.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.940+0000 7f12c2783740 1 Processor -- start 2024-04-21T08:40:28.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.940+0000 7f12c2783740 1 -- start start 2024-04-21T08:40:28.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.940+0000 7f12c2783740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55837a5193b0 0x55837a524250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.940+0000 7f12c2783740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55837a519d60 0x55837a532ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.940+0000 7f12c2783740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55837a522e60 0x55837a525450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.940+0000 7f12c2783740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f12b81ad050 con 0x55837a519d60 2024-04-21T08:40:28.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.940+0000 7f12c2783740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f12b81aeb90 con 0x55837a522e60 2024-04-21T08:40:28.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.940+0000 7f12c2783740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f12b81ad720 con 0x55837a5193b0 2024-04-21T08:40:28.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.940+0000 7f12bd055640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55837a519d60 0x55837a532ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.940+0000 7f12bd055640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55837a519d60 0x55837a532ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36206/0 (socket says 172.21.15.86:36206) 2024-04-21T08:40:28.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.940+0000 7f12bd055640 1 -- 172.21.15.86:0/1301750799 learned_addr learned my addr 172.21.15.86:0/1301750799 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:28.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.940+0000 7f12bd856640 1 --2- 172.21.15.86:0/1301750799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55837a5193b0 0x55837a524250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.940+0000 7f12bd055640 1 -- 172.21.15.86:0/1301750799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55837a5193b0 msgr2=0x55837a524250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.940+0000 7f12bd055640 1 --2- 172.21.15.86:0/1301750799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55837a5193b0 0x55837a524250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.940+0000 7f12bd055640 1 -- 172.21.15.86:0/1301750799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55837a522e60 msgr2=0x55837a525450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.940+0000 7f12be057640 1 --2- 172.21.15.86:0/1301750799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55837a522e60 0x55837a525450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.940+0000 7f12bd055640 1 --2- 172.21.15.86:0/1301750799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55837a522e60 0x55837a525450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.940+0000 7f12bd055640 1 -- 172.21.15.86:0/1301750799 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55837a518560 con 0x55837a519d60 2024-04-21T08:40:28.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.940+0000 7f12bd856640 1 --2- 172.21.15.86:0/1301750799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55837a5193b0 0x55837a524250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:28.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.940+0000 7f12be057640 1 --2- 172.21.15.86:0/1301750799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55837a522e60 0x55837a525450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:40:28.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.940+0000 7f12bd055640 1 --2- 172.21.15.86:0/1301750799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55837a519d60 0x55837a532ca0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009610 tx=0x7f12a8016b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=335157d47575d8dd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:28.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.940+0000 7f12a37fe640 1 -- 172.21.15.86:0/1301750799 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a8085070 con 0x55837a519d60 2024-04-21T08:40:28.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.940+0000 7f12a37fe640 1 -- 172.21.15.86:0/1301750799 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12a80a0580 con 0x55837a519d60 2024-04-21T08:40:28.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.940+0000 7f12a37fe640 1 -- 172.21.15.86:0/1301750799 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12a8021040 con 0x55837a519d60 2024-04-21T08:40:28.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.941+0000 7f12c2783740 1 -- 172.21.15.86:0/1301750799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55837a519d60 msgr2=0x55837a532ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.941+0000 7f12c2783740 1 --2- 172.21.15.86:0/1301750799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55837a519d60 0x55837a532ca0 secure :-1 s=READY pgs=799 cs=0 l=1 rev1=1 crypto rx=0x7f12a8009610 tx=0x7f12a8016b40 comp rx=0 tx=0).stop 2024-04-21T08:40:28.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.941+0000 7f12c2783740 1 -- 172.21.15.86:0/1301750799 shutdown_connections 2024-04-21T08:40:28.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.941+0000 7f12c2783740 1 --2- 172.21.15.86:0/1301750799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55837a522e60 0x55837a525450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.941+0000 7f12c2783740 1 --2- 172.21.15.86:0/1301750799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55837a519d60 0x55837a532ca0 unknown :-1 s=CLOSED pgs=799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.941+0000 7f12c2783740 1 --2- 172.21.15.86:0/1301750799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55837a5193b0 0x55837a524250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.941+0000 7f12c2783740 1 -- 172.21.15.86:0/1301750799 >> 172.21.15.86:0/1301750799 conn(0x55837a513ba0 msgr2=0x55837a524d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:28.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.941+0000 7f12c2783740 1 -- 172.21.15.86:0/1301750799 shutdown_connections 2024-04-21T08:40:28.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.941+0000 7f12c2783740 1 -- 172.21.15.86:0/1301750799 wait complete. 2024-04-21T08:40:28.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.941+0000 7f12c2783740 1 -- start start 2024-04-21T08:40:28.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.941+0000 7f12c2783740 1 Processor -- start 2024-04-21T08:40:28.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.942+0000 7f12c2783740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55837a5193b0 0x55837a52f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.942+0000 7f12c2783740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55837a519d60 0x55837a52f6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.942+0000 7f12c2783740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55837a522e60 0x55837a527af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.942+0000 7f12c2783740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f12b81ae4c0 con 0x55837a522e60 2024-04-21T08:40:28.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.942+0000 7f12c2783740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f12b81addf0 con 0x55837a5193b0 2024-04-21T08:40:28.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.942+0000 7f12c2783740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f12b81ab010 con 0x55837a519d60 2024-04-21T08:40:28.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.942+0000 7f12c2783740 10 client.0 _ll_register_callbacks cb 0x7f12b4009a70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:28.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.942+0000 7f12be057640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55837a522e60 0x55837a527af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.942+0000 7f12be057640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55837a522e60 0x55837a527af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36214/0 (socket says 172.21.15.86:36214) 2024-04-21T08:40:28.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.942+0000 7f12be057640 1 -- 172.21.15.86:0/2973891824 learned_addr learned my addr 172.21.15.86:0/2973891824 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:28.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.942+0000 7f12bd856640 1 --2- 172.21.15.86:0/2973891824 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55837a5193b0 0x55837a52f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.942+0000 7f12bd055640 1 --2- 172.21.15.86:0/2973891824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55837a519d60 0x55837a52f6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.942+0000 7f12be057640 1 -- 172.21.15.86:0/2973891824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55837a519d60 msgr2=0x55837a52f6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.942+0000 7f12be057640 1 --2- 172.21.15.86:0/2973891824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55837a519d60 0x55837a52f6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.942+0000 7f12be057640 1 -- 172.21.15.86:0/2973891824 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55837a5193b0 msgr2=0x55837a52f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:28.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.942+0000 7f12be057640 1 --2- 172.21.15.86:0/2973891824 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55837a5193b0 0x55837a52f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.942+0000 7f12be057640 1 -- 172.21.15.86:0/2973891824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55837a5282f0 con 0x55837a522e60 2024-04-21T08:40:28.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.942+0000 7f12bd055640 1 --2- 172.21.15.86:0/2973891824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55837a519d60 0x55837a52f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:40:28.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.942+0000 7f12be057640 1 --2- 172.21.15.86:0/2973891824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55837a522e60 0x55837a527af0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f12b0033000 tx=0x7f12b002c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4664f10e8089687f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:28.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.942+0000 7f12a17fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:40:28.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.942+0000 7f12a17fa640 1 -- 172.21.15.86:0/2973891824 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12b0039040 con 0x55837a522e60 2024-04-21T08:40:28.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.943+0000 7f12a17fa640 1 -- 172.21.15.86:0/2973891824 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f12b0004240 con 0x55837a522e60 2024-04-21T08:40:28.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.943+0000 7f12c2783740 1 -- 172.21.15.86:0/2973891824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12a80a0580 con 0x55837a522e60 2024-04-21T08:40:28.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.943+0000 7f12a17fa640 1 -- 172.21.15.86:0/2973891824 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f12b0004440 con 0x55837a522e60 2024-04-21T08:40:28.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.943+0000 7f12c2783740 10 client.16560 fetch_fsmap learned FSMap version 29 2024-04-21T08:40:28.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.943+0000 7f12c2783740 1 -- 172.21.15.86:0/2973891824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55837a59cdf0 con 0x55837a522e60 2024-04-21T08:40:28.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.943+0000 7f12a17fa640 1 -- 172.21.15.86:0/2973891824 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12a80a0580 con 0x55837a522e60 2024-04-21T08:40:28.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.943+0000 7f12a17fa640 1 -- 172.21.15.86:0/2973891824 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f12b0034440 con 0x55837a522e60 2024-04-21T08:40:28.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.943+0000 7f12c2783740 10 client.16560 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:40:28.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.943+0000 7f12c2783740 10 client.16560 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:40:28.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.943+0000 7f12c2783740 20 client.16560 populate_metadata read hostname 'smithi086' 2024-04-21T08:40:28.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.943+0000 7f12c2783740 1 --2- 172.21.15.86:0/2973891824 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55837a59d5c0 0x55837a59f9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:28.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.943+0000 7f12c2783740 4 client.16560 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:40:28.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.943+0000 7f12c2783740 1 -- 172.21.15.86:0/2973891824 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55837a48bc10 con 0x55837a59d5c0 2024-04-21T08:40:28.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.943+0000 7f12bd856640 1 --2- 172.21.15.86:0/2973891824 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55837a59d5c0 0x55837a59f9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:28.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.944+0000 7f12bd856640 1 --2- 172.21.15.86:0/2973891824 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55837a59d5c0 0x55837a59f9e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6f222c899cacf67c server_cookie=1f317b96d68c604c in_seq=0 out_seq=0 2024-04-21T08:40:28.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.944+0000 7f12a17fa640 10 client.16560 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:40:28.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.944+0000 7f12a17fa640 1 -- 172.21.15.86:0/2973891824 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55837a48bc10 con 0x55837a59d5c0 2024-04-21T08:40:28.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.944+0000 7f12a17fa640 10 client.16560 handle_command_reply: tid=0 2024-04-21T08:40:28.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.944+0000 7f12c2783740 1 client.16560 shutdown 2024-04-21T08:40:28.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.944+0000 7f12c2783740 1 -- 172.21.15.86:0/2973891824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55837a522e60 msgr2=0x55837a527af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.944+0000 7f12c2783740 1 --2- 172.21.15.86:0/2973891824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55837a522e60 0x55837a527af0 secure :-1 s=READY pgs=800 cs=0 l=1 rev1=1 crypto rx=0x7f12b0033000 tx=0x7f12b002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.944+0000 7f12c2783740 1 -- 172.21.15.86:0/2973891824 shutdown_connections 2024-04-21T08:40:28.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.944+0000 7f12c2783740 1 --2- 172.21.15.86:0/2973891824 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55837a59d5c0 0x55837a59f9e0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.944+0000 7f12c2783740 1 --2- 172.21.15.86:0/2973891824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55837a522e60 0x55837a527af0 unknown :-1 s=CLOSED pgs=800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.944+0000 7f12a17fa640 0 client.16560 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:40:28.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.944+0000 7f12c2783740 1 --2- 172.21.15.86:0/2973891824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55837a519d60 0x55837a52f6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.944+0000 7f12c2783740 1 --2- 172.21.15.86:0/2973891824 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55837a5193b0 0x55837a52f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.944+0000 7f12c2783740 1 -- 172.21.15.86:0/2973891824 >> 172.21.15.86:0/2973891824 conn(0x55837a513ba0 msgr2=0x55837a5255d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:28.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.944+0000 7f12c2783740 1 -- 172.21.15.86:0/2973891824 shutdown_connections 2024-04-21T08:40:28.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.944+0000 7f12c2783740 1 -- 172.21.15.86:0/2973891824 wait complete. 2024-04-21T08:40:28.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.944+0000 7f12c2783740 20 client.16560 trim_cache size 0 max 16384 2024-04-21T08:40:28.951 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:40:28.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.945+0000 7f12bfaba640 1 -- 172.21.15.86:0/3404589049 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f12a40a3d20 msgr2=0x7f12a40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.945+0000 7f12bfaba640 1 --2- 172.21.15.86:0/3404589049 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f12a40a3d20 0x7f12a40a61e0 secure :-1 s=READY pgs=113 cs=0 l=1 rev1=1 crypto rx=0x7f12b002cfd0 tx=0x7f12b0005d80 comp rx=0 tx=0).stop 2024-04-21T08:40:28.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.945+0000 7f12bfaba640 1 -- 172.21.15.86:0/3404589049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b8107250 msgr2=0x7f12b8198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:28.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.945+0000 7f12bfaba640 1 --2- 172.21.15.86:0/3404589049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b8107250 0x7f12b8198ca0 secure :-1 s=READY pgs=796 cs=0 l=1 rev1=1 crypto rx=0x7f12a800d770 tx=0x7f12a800dc30 comp rx=0 tx=0).stop 2024-04-21T08:40:28.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.945+0000 7f12bfaba640 1 -- 172.21.15.86:0/3404589049 shutdown_connections 2024-04-21T08:40:28.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.945+0000 7f12bfaba640 1 --2- 172.21.15.86:0/3404589049 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f12a40a3d20 0x7f12a40a61e0 unknown :-1 s=CLOSED pgs=113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.945+0000 7f12bfaba640 1 --2- 172.21.15.86:0/3404589049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12b819cfd0 0x7f12b819d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.945+0000 7f12bfaba640 1 --2- 172.21.15.86:0/3404589049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12b8107250 0x7f12b8198ca0 unknown :-1 s=CLOSED pgs=796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.945+0000 7f12bfaba640 1 --2- 172.21.15.86:0/3404589049 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12b8102180 0x7f12b8198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:28.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.945+0000 7f12bfaba640 1 -- 172.21.15.86:0/3404589049 >> 172.21.15.86:0/3404589049 conn(0x7f12b80fd9c0 msgr2=0x7f12b80ffd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:28.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.945+0000 7f12bfaba640 1 -- 172.21.15.86:0/3404589049 shutdown_connections 2024-04-21T08:40:28.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:28.945+0000 7f12bfaba640 1 -- 172.21.15.86:0/3404589049 wait complete. 2024-04-21T08:40:28.955 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:40:28.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:28 smithi088 ceph-mon[31259]: pgmap v53: 129 pgs: 129 active+clean; 2.0 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-04-21T08:40:28.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:28 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/4193782194' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:40:29.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.055+0000 7fd9bc5d5640 1 -- 172.21.15.86:0/4200135624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9b41031d0 msgr2=0x7fd9b41035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:29.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.055+0000 7fd9bc5d5640 1 --2- 172.21.15.86:0/4200135624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9b41031d0 0x7fd9b41035b0 secure :-1 s=READY pgs=801 cs=0 l=1 rev1=1 crypto rx=0x7fd9b0009900 tx=0x7fd9b002bd60 comp rx=0 tx=0).stop 2024-04-21T08:40:29.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.056+0000 7fd9bc5d5640 1 -- 172.21.15.86:0/4200135624 shutdown_connections 2024-04-21T08:40:29.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.056+0000 7fd9bc5d5640 1 --2- 172.21.15.86:0/4200135624 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9b410c510 0x7fd9b410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:29.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.056+0000 7fd9bc5d5640 1 --2- 172.21.15.86:0/4200135624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9b4103af0 0x7fd9b410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:29.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.056+0000 7fd9bc5d5640 1 --2- 172.21.15.86:0/4200135624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9b41031d0 0x7fd9b41035b0 unknown :-1 s=CLOSED pgs=801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:29.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.056+0000 7fd9bc5d5640 1 -- 172.21.15.86:0/4200135624 >> 172.21.15.86:0/4200135624 conn(0x7fd9b4077f50 msgr2=0x7fd9b4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:29.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.056+0000 7fd9bc5d5640 1 -- 172.21.15.86:0/4200135624 shutdown_connections 2024-04-21T08:40:29.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.056+0000 7fd9bc5d5640 1 -- 172.21.15.86:0/4200135624 wait complete. 2024-04-21T08:40:29.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.056+0000 7fd9bc5d5640 1 Processor -- start 2024-04-21T08:40:29.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.056+0000 7fd9bc5d5640 1 -- start start 2024-04-21T08:40:29.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.056+0000 7fd9bc5d5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9b41031d0 0x7fd9b419e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:29.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.056+0000 7fd9bc5d5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9b4103af0 0x7fd9b419e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:29.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.056+0000 7fd9bc5d5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9b410c510 0x7fd9b41985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:29.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.056+0000 7fd9bc5d5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b41121a0 con 0x7fd9b410c510 2024-04-21T08:40:29.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.056+0000 7fd9bc5d5640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b4112020 con 0x7fd9b41031d0 2024-04-21T08:40:29.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.056+0000 7fd9bc5d5640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b4112320 con 0x7fd9b4103af0 2024-04-21T08:40:29.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.056+0000 7fd9bab72640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9b410c510 0x7fd9b41985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:29.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.056+0000 7fd9bab72640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9b410c510 0x7fd9b41985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36224/0 (socket says 172.21.15.86:36224) 2024-04-21T08:40:29.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.056+0000 7fd9bab72640 1 -- 172.21.15.86:0/361938985 learned_addr learned my addr 172.21.15.86:0/361938985 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:29.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.057+0000 7fd9ba371640 1 --2- 172.21.15.86:0/361938985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9b41031d0 0x7fd9b419e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:29.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.057+0000 7fd9bab72640 1 -- 172.21.15.86:0/361938985 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9b4103af0 msgr2=0x7fd9b419e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:29.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.057+0000 7fd9bab72640 1 --2- 172.21.15.86:0/361938985 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9b4103af0 0x7fd9b419e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:29.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.057+0000 7fd9bab72640 1 -- 172.21.15.86:0/361938985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9b41031d0 msgr2=0x7fd9b419e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:29.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.057+0000 7fd9bab72640 1 --2- 172.21.15.86:0/361938985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9b41031d0 0x7fd9b419e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:29.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.057+0000 7fd9bab72640 1 -- 172.21.15.86:0/361938985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9b4198d30 con 0x7fd9b410c510 2024-04-21T08:40:29.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.057+0000 7fd9bab72640 1 --2- 172.21.15.86:0/361938985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9b410c510 0x7fd9b41985b0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac00eb10 tx=0x7fd9ac00c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:29.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.057+0000 7fd9ab7fe640 1 -- 172.21.15.86:0/361938985 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac004b20 con 0x7fd9b410c510 2024-04-21T08:40:29.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.057+0000 7fd9ab7fe640 1 -- 172.21.15.86:0/361938985 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9ac004cc0 con 0x7fd9b410c510 2024-04-21T08:40:29.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.057+0000 7fd9bc5d5640 1 -- 172.21.15.86:0/361938985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9b4199020 con 0x7fd9b410c510 2024-04-21T08:40:29.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.057+0000 7fd9ab7fe640 1 -- 172.21.15.86:0/361938985 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac010670 con 0x7fd9b410c510 2024-04-21T08:40:29.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.057+0000 7fd9bc5d5640 1 -- 172.21.15.86:0/361938985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9b41a52c0 con 0x7fd9b410c510 2024-04-21T08:40:29.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.059+0000 7fd9ab7fe640 1 -- 172.21.15.86:0/361938985 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd9ac005020 con 0x7fd9b410c510 2024-04-21T08:40:29.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.060+0000 7fd9bf1ca740 1 Processor -- start 2024-04-21T08:40:29.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.060+0000 7fd9bf1ca740 1 -- start start 2024-04-21T08:40:29.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.060+0000 7fd9ab7fe640 1 --2- 172.21.15.86:0/361938985 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd9a00a3d20 0x7fd9a00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:29.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.060+0000 7fd9bf1ca740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e2ba930330 0x55e2ba930710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:29.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.060+0000 7fd9bf1ca740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ba930ce0 0x55e2ba939820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:29.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.060+0000 7fd9bf1ca740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e2ba939e40 0x55e2ba93a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:29.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.060+0000 7fd9bf1ca740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e2ba8951f0 con 0x55e2ba930ce0 2024-04-21T08:40:29.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.060+0000 7fd9bf1ca740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55e2ba8324b0 con 0x55e2ba939e40 2024-04-21T08:40:29.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.060+0000 7fd9bf1ca740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e2ba8911b0 con 0x55e2ba930330 2024-04-21T08:40:29.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.060+0000 7fd9b9b70640 1 --2- 172.21.15.86:0/361938985 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd9a00a3d20 0x7fd9a00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:29.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.060+0000 7fd9ab7fe640 1 -- 172.21.15.86:0/361938985 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fd9ac0d1100 con 0x7fd9b410c510 2024-04-21T08:40:29.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.060+0000 7fd9b9b70640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ba930ce0 0x55e2ba939820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:29.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.060+0000 7fd9ba371640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e2ba930330 0x55e2ba930710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:29.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.060+0000 7fd9b9b70640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ba930ce0 0x55e2ba939820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36226/0 (socket says 172.21.15.86:36226) 2024-04-21T08:40:29.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.060+0000 7fd9b9b70640 1 -- 172.21.15.86:0/395535358 learned_addr learned my addr 172.21.15.86:0/395535358 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:29.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.060+0000 7fd9bab72640 1 --2- 172.21.15.86:0/395535358 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e2ba939e40 0x55e2ba93a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:29.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.061+0000 7fd9bab72640 1 -- 172.21.15.86:0/395535358 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e2ba930330 msgr2=0x55e2ba930710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:29.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.061+0000 7fd9bab72640 1 --2- 172.21.15.86:0/395535358 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e2ba930330 0x55e2ba930710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:29.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.061+0000 7fd9bab72640 1 -- 172.21.15.86:0/395535358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ba930ce0 msgr2=0x55e2ba939820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:29.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.061+0000 7fd9bab72640 1 --2- 172.21.15.86:0/395535358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ba930ce0 0x55e2ba939820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:29.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.061+0000 7fd9bab72640 1 -- 172.21.15.86:0/395535358 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2ba8241b0 con 0x55e2ba939e40 2024-04-21T08:40:29.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.061+0000 7fd9b9b70640 1 --2- 172.21.15.86:0/395535358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ba930ce0 0x55e2ba939820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:40:29.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.061+0000 7fd9ba371640 1 --2- 172.21.15.86:0/395535358 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e2ba930330 0x55e2ba930710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:40:29.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.061+0000 7fd9b9b70640 1 --2- 172.21.15.86:0/361938985 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd9a00a3d20 0x7fd9a00a61e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4013720 tx=0x7fd9a400c8c0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:29.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.061+0000 7fd9bab72640 1 --2- 172.21.15.86:0/395535358 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e2ba939e40 0x55e2ba93a2c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac0ce4e0 tx=0x7fd9ac09cbc0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5cec5fb6514f742b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:29.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.061+0000 7fd9a8bfe640 1 -- 172.21.15.86:0/395535358 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac09d430 con 0x55e2ba939e40 2024-04-21T08:40:29.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.061+0000 7fd9a8bfe640 1 -- 172.21.15.86:0/395535358 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9ac09cd30 con 0x55e2ba939e40 2024-04-21T08:40:29.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.061+0000 7fd9a8bfe640 1 -- 172.21.15.86:0/395535358 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac013190 con 0x55e2ba939e40 2024-04-21T08:40:29.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.062+0000 7fd9bf1ca740 1 -- 172.21.15.86:0/395535358 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e2ba939e40 msgr2=0x55e2ba93a2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:29.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.062+0000 7fd9bf1ca740 1 --2- 172.21.15.86:0/395535358 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e2ba939e40 0x55e2ba93a2c0 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac0ce4e0 tx=0x7fd9ac09cbc0 comp rx=0 tx=0).stop 2024-04-21T08:40:29.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.062+0000 7fd9bf1ca740 1 -- 172.21.15.86:0/395535358 shutdown_connections 2024-04-21T08:40:29.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.062+0000 7fd9bf1ca740 1 --2- 172.21.15.86:0/395535358 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e2ba939e40 0x55e2ba93a2c0 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:29.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.062+0000 7fd9bf1ca740 1 --2- 172.21.15.86:0/395535358 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ba930ce0 0x55e2ba939820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:29.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.062+0000 7fd9bf1ca740 1 --2- 172.21.15.86:0/395535358 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e2ba930330 0x55e2ba930710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:29.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.062+0000 7fd9bf1ca740 1 -- 172.21.15.86:0/395535358 >> 172.21.15.86:0/395535358 conn(0x55e2ba92aba0 msgr2=0x55e2ba92af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:29.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.062+0000 7fd9bf1ca740 1 -- 172.21.15.86:0/395535358 shutdown_connections 2024-04-21T08:40:29.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.062+0000 7fd9bf1ca740 1 -- 172.21.15.86:0/395535358 wait complete. 2024-04-21T08:40:29.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.062+0000 7fd9bf1ca740 1 -- start start 2024-04-21T08:40:29.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.062+0000 7fd9bf1ca740 1 Processor -- start 2024-04-21T08:40:29.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.062+0000 7fd9bf1ca740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e2ba930330 0x55e2ba9401a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:29.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.062+0000 7fd9bf1ca740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e2ba930ce0 0x55e2ba93d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:29.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.062+0000 7fd9bf1ca740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ba939e40 0x55e2ba93da70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:29.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.062+0000 7fd9bf1ca740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e2ba940dc0 con 0x55e2ba939e40 2024-04-21T08:40:29.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.062+0000 7fd9bf1ca740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55e2ba84e500 con 0x55e2ba930ce0 2024-04-21T08:40:29.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.062+0000 7fd9bf1ca740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e2ba940f40 con 0x55e2ba930330 2024-04-21T08:40:29.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.062+0000 7fd9bf1ca740 10 client.0 _ll_register_callbacks cb 0x55e2ba81cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:29.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.063+0000 7fd9bab72640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ba939e40 0x55e2ba93da70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:29.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.063+0000 7fd9bab72640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ba939e40 0x55e2ba93da70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36238/0 (socket says 172.21.15.86:36238) 2024-04-21T08:40:29.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.063+0000 7fd9bab72640 1 -- 172.21.15.86:0/3958008221 learned_addr learned my addr 172.21.15.86:0/3958008221 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:29.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.063+0000 7fd9b9b70640 1 --2- 172.21.15.86:0/3958008221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e2ba930ce0 0x55e2ba93d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:29.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.063+0000 7fd9ba371640 1 --2- 172.21.15.86:0/3958008221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e2ba930330 0x55e2ba9401a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:29.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.063+0000 7fd9bab72640 1 -- 172.21.15.86:0/3958008221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e2ba930330 msgr2=0x55e2ba9401a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:29.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.063+0000 7fd9bab72640 1 --2- 172.21.15.86:0/3958008221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e2ba930330 0x55e2ba9401a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:29.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.063+0000 7fd9bab72640 1 -- 172.21.15.86:0/3958008221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e2ba930ce0 msgr2=0x55e2ba93d530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:29.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.063+0000 7fd9bab72640 1 --2- 172.21.15.86:0/3958008221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e2ba930ce0 0x55e2ba93d530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:29.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.063+0000 7fd9bab72640 1 -- 172.21.15.86:0/3958008221 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2ba5a73f0 con 0x55e2ba939e40 2024-04-21T08:40:29.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.063+0000 7fd9bab72640 1 --2- 172.21.15.86:0/3958008221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ba939e40 0x55e2ba93da70 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac09ca30 tx=0x7fd9ac00cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2d993f5e99f204a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:29.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.063+0000 7fd99dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:40:29.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.063+0000 7fd99dffb640 1 -- 172.21.15.86:0/3958008221 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac0a10e0 con 0x55e2ba939e40 2024-04-21T08:40:29.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.063+0000 7fd99dffb640 1 -- 172.21.15.86:0/3958008221 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fd9ac0a5510 con 0x55e2ba939e40 2024-04-21T08:40:29.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.063+0000 7fd9bf1ca740 1 -- 172.21.15.86:0/3958008221 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9ac09cd30 con 0x55e2ba939e40 2024-04-21T08:40:29.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.063+0000 7fd99dffb640 1 -- 172.21.15.86:0/3958008221 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd9ac0a59a0 con 0x55e2ba939e40 2024-04-21T08:40:29.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.063+0000 7fd9bf1ca740 10 client.16572 fetch_fsmap learned FSMap version 29 2024-04-21T08:40:29.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.063+0000 7fd9bf1ca740 1 -- 172.21.15.86:0/3958008221 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e2ba6717d0 con 0x55e2ba939e40 2024-04-21T08:40:29.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.063+0000 7fd99dffb640 1 -- 172.21.15.86:0/3958008221 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9ac09cd30 con 0x55e2ba939e40 2024-04-21T08:40:29.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.064+0000 7fd99dffb640 1 -- 172.21.15.86:0/3958008221 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd9ac0bd110 con 0x55e2ba939e40 2024-04-21T08:40:29.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.064+0000 7fd9bf1ca740 10 client.16572 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:40:29.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.064+0000 7fd9bf1ca740 10 client.16572 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:29.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.064+0000 7fd9bf1ca740 20 client.16572 populate_metadata read hostname 'smithi086' 2024-04-21T08:40:29.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.064+0000 7fd9bf1ca740 1 --2- 172.21.15.86:0/3958008221 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55e2ba94de90 0x55e2ba950280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:29.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.064+0000 7fd9bf1ca740 4 client.16572 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:40:29.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.064+0000 7fd9bf1ca740 1 -- 172.21.15.86:0/3958008221 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e2ba834930 con 0x55e2ba94de90 2024-04-21T08:40:29.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.064+0000 7fd9ba371640 1 --2- 172.21.15.86:0/3958008221 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55e2ba94de90 0x55e2ba950280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:29.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.064+0000 7fd9ba371640 1 --2- 172.21.15.86:0/3958008221 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55e2ba94de90 0x55e2ba950280 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=18f63138b25ac0bc server_cookie=383b6933a802c203 in_seq=0 out_seq=0 2024-04-21T08:40:29.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.064+0000 7fd99dffb640 10 client.16572 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:40:29.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.065+0000 7fd99dffb640 1 -- 172.21.15.86:0/3958008221 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e2ba834930 con 0x55e2ba94de90 2024-04-21T08:40:29.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.065+0000 7fd99dffb640 10 client.16572 handle_command_reply: tid=0 2024-04-21T08:40:29.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.065+0000 7fd9bf1ca740 1 client.16572 shutdown 2024-04-21T08:40:29.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.065+0000 7fd9bf1ca740 1 -- 172.21.15.86:0/3958008221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ba939e40 msgr2=0x55e2ba93da70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:29.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.065+0000 7fd9bf1ca740 1 --2- 172.21.15.86:0/3958008221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ba939e40 0x55e2ba93da70 secure :-1 s=READY pgs=803 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac09ca30 tx=0x7fd9ac00cf70 comp rx=0 tx=0).stop 2024-04-21T08:40:29.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.065+0000 7fd9bf1ca740 1 -- 172.21.15.86:0/3958008221 shutdown_connections 2024-04-21T08:40:29.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.065+0000 7fd9bf1ca740 1 --2- 172.21.15.86:0/3958008221 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55e2ba94de90 0x55e2ba950280 crc :-1 s=READY pgs=83 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:29.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.065+0000 7fd9bf1ca740 1 --2- 172.21.15.86:0/3958008221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ba939e40 0x55e2ba93da70 unknown :-1 s=CLOSED pgs=803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:29.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.065+0000 7fd99dffb640 0 client.16572 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:40:29.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.065+0000 7fd9bf1ca740 1 --2- 172.21.15.86:0/3958008221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e2ba930ce0 0x55e2ba93d530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:29.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.065+0000 7fd9bf1ca740 1 --2- 172.21.15.86:0/3958008221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e2ba930330 0x55e2ba9401a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:29.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.065+0000 7fd9bf1ca740 1 -- 172.21.15.86:0/3958008221 >> 172.21.15.86:0/3958008221 conn(0x55e2ba92aba0 msgr2=0x55e2ba92c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:29.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.065+0000 7fd9bf1ca740 1 -- 172.21.15.86:0/3958008221 shutdown_connections 2024-04-21T08:40:29.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.065+0000 7fd9bf1ca740 1 -- 172.21.15.86:0/3958008221 wait complete. 2024-04-21T08:40:29.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.065+0000 7fd9bf1ca740 20 client.16572 trim_cache size 0 max 16384 2024-04-21T08:40:29.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.078+0000 7fd9bf1ca740 1 Processor -- start 2024-04-21T08:40:29.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.078+0000 7fd9bf1ca740 1 -- start start 2024-04-21T08:40:29.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.079+0000 7fd9bf1ca740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e2ba930330 0x55e2ba949910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:29.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.079+0000 7fd9bf1ca740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e2ba930ce0 0x55e2ba93fc60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:29.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.079+0000 7fd9bf1ca740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ba939e40 0x55e2ba9401a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:29.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.079+0000 7fd9bf1ca740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b41ad100 con 0x55e2ba939e40 2024-04-21T08:40:29.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.079+0000 7fd9bf1ca740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b41aec40 con 0x55e2ba930ce0 2024-04-21T08:40:29.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.079+0000 7fd9bf1ca740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b41ad7d0 con 0x55e2ba930330 2024-04-21T08:40:29.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.079+0000 7fd9bab72640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ba939e40 0x55e2ba9401a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:29.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.079+0000 7fd9bab72640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ba939e40 0x55e2ba9401a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36240/0 (socket says 172.21.15.86:36240) 2024-04-21T08:40:29.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.079+0000 7fd9bab72640 1 -- 172.21.15.86:0/2269619620 learned_addr learned my addr 172.21.15.86:0/2269619620 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:29.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.079+0000 7fd9b9b70640 1 --2- 172.21.15.86:0/2269619620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e2ba930ce0 0x55e2ba93fc60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:29.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.079+0000 7fd9bab72640 1 -- 172.21.15.86:0/2269619620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e2ba930330 msgr2=0x55e2ba949910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:29.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.079+0000 7fd9bab72640 1 --2- 172.21.15.86:0/2269619620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e2ba930330 0x55e2ba949910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:29.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.079+0000 7fd9bab72640 1 -- 172.21.15.86:0/2269619620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e2ba930ce0 msgr2=0x55e2ba93fc60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:29.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.079+0000 7fd9bab72640 1 --2- 172.21.15.86:0/2269619620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e2ba930ce0 0x55e2ba93fc60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:29.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.079+0000 7fd9bab72640 1 -- 172.21.15.86:0/2269619620 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e2ba924fe0 con 0x55e2ba939e40 2024-04-21T08:40:29.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.079+0000 7fd9b9b70640 1 --2- 172.21.15.86:0/2269619620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e2ba930ce0 0x55e2ba93fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:40:29.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.079+0000 7fd9bab72640 1 --2- 172.21.15.86:0/2269619620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ba939e40 0x55e2ba9401a0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac09c800 tx=0x7fd9ac09cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=df8d18605245e331 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:29.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.079+0000 7fd99ffff640 1 -- 172.21.15.86:0/2269619620 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac0ce970 con 0x55e2ba939e40 2024-04-21T08:40:29.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.079+0000 7fd99ffff640 1 -- 172.21.15.86:0/2269619620 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9ac07f070 con 0x55e2ba939e40 2024-04-21T08:40:29.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.079+0000 7fd99ffff640 1 -- 172.21.15.86:0/2269619620 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9ac09b030 con 0x55e2ba939e40 2024-04-21T08:40:29.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.080+0000 7fd9bf1ca740 1 -- 172.21.15.86:0/2269619620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ba939e40 msgr2=0x55e2ba9401a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:29.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.080+0000 7fd9bf1ca740 1 --2- 172.21.15.86:0/2269619620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ba939e40 0x55e2ba9401a0 secure :-1 s=READY pgs=804 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac09c800 tx=0x7fd9ac09cbc0 comp rx=0 tx=0).stop 2024-04-21T08:40:29.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.080+0000 7fd9bf1ca740 1 -- 172.21.15.86:0/2269619620 shutdown_connections 2024-04-21T08:40:29.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.080+0000 7fd9bf1ca740 1 --2- 172.21.15.86:0/2269619620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ba939e40 0x55e2ba9401a0 unknown :-1 s=CLOSED pgs=804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:29.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.080+0000 7fd9bf1ca740 1 --2- 172.21.15.86:0/2269619620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e2ba930ce0 0x55e2ba93fc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:29.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.080+0000 7fd9bf1ca740 1 --2- 172.21.15.86:0/2269619620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e2ba930330 0x55e2ba949910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:29.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.080+0000 7fd9bf1ca740 1 -- 172.21.15.86:0/2269619620 >> 172.21.15.86:0/2269619620 conn(0x55e2ba92aba0 msgr2=0x55e2ba949200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:29.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.080+0000 7fd9bf1ca740 1 -- 172.21.15.86:0/2269619620 shutdown_connections 2024-04-21T08:40:29.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.080+0000 7fd9bf1ca740 1 -- 172.21.15.86:0/2269619620 wait complete. 2024-04-21T08:40:29.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.080+0000 7fd9bf1ca740 1 -- start start 2024-04-21T08:40:29.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.080+0000 7fd9bf1ca740 1 Processor -- start 2024-04-21T08:40:29.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.080+0000 7fd9bf1ca740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ba930330 0x55e2ba9ab430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:29.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.080+0000 7fd9bf1ca740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e2ba930ce0 0x55e2ba9ab970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:29.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.080+0000 7fd9bf1ca740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e2ba939e40 0x55e2ba950670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:29.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.080+0000 7fd9bf1ca740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b41ae570 con 0x55e2ba930330 2024-04-21T08:40:29.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.080+0000 7fd9bf1ca740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b41adea0 con 0x55e2ba930ce0 2024-04-21T08:40:29.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.080+0000 7fd9bf1ca740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd9b41ab0c0 con 0x55e2ba939e40 2024-04-21T08:40:29.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.080+0000 7fd9bf1ca740 10 client.0 _ll_register_callbacks cb 0x7fd9b0009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:29.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.080+0000 7fd9ba371640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ba930330 0x55e2ba9ab430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:29.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.080+0000 7fd9ba371640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ba930330 0x55e2ba9ab430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36250/0 (socket says 172.21.15.86:36250) 2024-04-21T08:40:29.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.080+0000 7fd9ba371640 1 -- 172.21.15.86:0/1280297370 learned_addr learned my addr 172.21.15.86:0/1280297370 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:29.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.081+0000 7fd9b9b70640 1 --2- 172.21.15.86:0/1280297370 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e2ba930ce0 0x55e2ba9ab970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:29.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.081+0000 7fd9ba371640 1 -- 172.21.15.86:0/1280297370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e2ba939e40 msgr2=0x55e2ba950670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:29.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.081+0000 7fd9ba371640 1 --2- 172.21.15.86:0/1280297370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e2ba939e40 0x55e2ba950670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:29.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.081+0000 7fd9ba371640 1 -- 172.21.15.86:0/1280297370 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e2ba930ce0 msgr2=0x55e2ba9ab970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:29.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.081+0000 7fd9ba371640 1 --2- 172.21.15.86:0/1280297370 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e2ba930ce0 0x55e2ba9ab970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:29.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.081+0000 7fd9ba371640 1 -- 172.21.15.86:0/1280297370 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e2ba950e50 con 0x55e2ba930330 2024-04-21T08:40:29.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.081+0000 7fd9b9b70640 1 --2- 172.21.15.86:0/1280297370 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e2ba930ce0 0x55e2ba9ab970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:29.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.081+0000 7fd9ba371640 1 --2- 172.21.15.86:0/1280297370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ba930330 0x55e2ba9ab430 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fd9b002c2a0 tx=0x7fd9b00044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd119cde6a60a711 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:29.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.081+0000 7fd99dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:40:29.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.081+0000 7fd99dffb640 1 -- 172.21.15.86:0/1280297370 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9b002cb20 con 0x55e2ba930330 2024-04-21T08:40:29.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.081+0000 7fd99dffb640 1 -- 172.21.15.86:0/1280297370 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fd9b003a040 con 0x55e2ba930330 2024-04-21T08:40:29.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.081+0000 7fd9bf1ca740 1 -- 172.21.15.86:0/1280297370 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9ac07f070 con 0x55e2ba930330 2024-04-21T08:40:29.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.081+0000 7fd99dffb640 1 -- 172.21.15.86:0/1280297370 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd9b0056050 con 0x55e2ba930330 2024-04-21T08:40:29.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.081+0000 7fd9bf1ca740 10 client.16578 fetch_fsmap learned FSMap version 29 2024-04-21T08:40:29.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.081+0000 7fd9bf1ca740 1 -- 172.21.15.86:0/1280297370 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e2ba9b3b50 con 0x55e2ba930330 2024-04-21T08:40:29.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.081+0000 7fd99dffb640 1 -- 172.21.15.86:0/1280297370 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9ac07f070 con 0x55e2ba930330 2024-04-21T08:40:29.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.081+0000 7fd99dffb640 1 -- 172.21.15.86:0/1280297370 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd9b0004920 con 0x55e2ba930330 2024-04-21T08:40:29.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.081+0000 7fd9bf1ca740 10 client.16578 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:40:29.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.081+0000 7fd9bf1ca740 10 client.16578 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:29.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.081+0000 7fd9bf1ca740 20 client.16578 populate_metadata read hostname 'smithi086' 2024-04-21T08:40:29.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.081+0000 7fd9bf1ca740 1 --2- 172.21.15.86:0/1280297370 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55e2ba9b4280 0x55e2ba9b66c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:29.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.082+0000 7fd9bf1ca740 4 client.16578 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:40:29.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.082+0000 7fd9bf1ca740 1 -- 172.21.15.86:0/1280297370 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e2ba8a2c10 con 0x55e2ba9b4280 2024-04-21T08:40:29.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.082+0000 7fd9b9b70640 1 --2- 172.21.15.86:0/1280297370 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55e2ba9b4280 0x55e2ba9b66c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:29.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.082+0000 7fd9b9b70640 1 --2- 172.21.15.86:0/1280297370 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55e2ba9b4280 0x55e2ba9b66c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec67710d44c41962 server_cookie=d46c09adb03240ae in_seq=0 out_seq=0 2024-04-21T08:40:29.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.082+0000 7fd99dffb640 10 client.16578 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:40:29.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.082+0000 7fd99dffb640 1 -- 172.21.15.86:0/1280297370 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e2ba8a2c10 con 0x55e2ba9b4280 2024-04-21T08:40:29.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.082+0000 7fd99dffb640 10 client.16578 handle_command_reply: tid=0 2024-04-21T08:40:29.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.082+0000 7fd9bf1ca740 1 client.16578 shutdown 2024-04-21T08:40:29.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.082+0000 7fd9bf1ca740 1 -- 172.21.15.86:0/1280297370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ba930330 msgr2=0x55e2ba9ab430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:29.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.082+0000 7fd9bf1ca740 1 --2- 172.21.15.86:0/1280297370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ba930330 0x55e2ba9ab430 secure :-1 s=READY pgs=805 cs=0 l=1 rev1=1 crypto rx=0x7fd9b002c2a0 tx=0x7fd9b00044c0 comp rx=0 tx=0).stop 2024-04-21T08:40:29.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.083+0000 7fd9bf1ca740 1 -- 172.21.15.86:0/1280297370 shutdown_connections 2024-04-21T08:40:29.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.083+0000 7fd9bf1ca740 1 --2- 172.21.15.86:0/1280297370 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55e2ba9b4280 0x55e2ba9b66c0 crc :-1 s=READY pgs=84 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:29.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.083+0000 7fd9bf1ca740 1 --2- 172.21.15.86:0/1280297370 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e2ba939e40 0x55e2ba950670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:29.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.083+0000 7fd99dffb640 0 client.16578 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:40:29.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.083+0000 7fd9bf1ca740 1 --2- 172.21.15.86:0/1280297370 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e2ba930ce0 0x55e2ba9ab970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:29.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.083+0000 7fd9bf1ca740 1 --2- 172.21.15.86:0/1280297370 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e2ba930330 0x55e2ba9ab430 unknown :-1 s=CLOSED pgs=805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:29.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.083+0000 7fd9bf1ca740 1 -- 172.21.15.86:0/1280297370 >> 172.21.15.86:0/1280297370 conn(0x55e2ba92aba0 msgr2=0x55e2ba93b9c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:29.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.083+0000 7fd9bf1ca740 1 -- 172.21.15.86:0/1280297370 shutdown_connections 2024-04-21T08:40:29.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.083+0000 7fd9bf1ca740 1 -- 172.21.15.86:0/1280297370 wait complete. 2024-04-21T08:40:29.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.083+0000 7fd9bf1ca740 20 client.16578 trim_cache size 0 max 16384 2024-04-21T08:40:29.090 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:40:29.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.083+0000 7fd9bc5d5640 1 -- 172.21.15.86:0/361938985 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd9a00a3d20 msgr2=0x7fd9a00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:29.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.083+0000 7fd9bc5d5640 1 --2- 172.21.15.86:0/361938985 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd9a00a3d20 0x7fd9a00a61e0 secure :-1 s=READY pgs=114 cs=0 l=1 rev1=1 crypto rx=0x7fd9a4013720 tx=0x7fd9a400c8c0 comp rx=0 tx=0).stop 2024-04-21T08:40:29.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.083+0000 7fd9bc5d5640 1 -- 172.21.15.86:0/361938985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9b410c510 msgr2=0x7fd9b41985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:29.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.083+0000 7fd9bc5d5640 1 --2- 172.21.15.86:0/361938985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9b410c510 0x7fd9b41985b0 secure :-1 s=READY pgs=802 cs=0 l=1 rev1=1 crypto rx=0x7fd9ac00eb10 tx=0x7fd9ac00c2d0 comp rx=0 tx=0).stop 2024-04-21T08:40:29.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.083+0000 7fd9bc5d5640 1 -- 172.21.15.86:0/361938985 shutdown_connections 2024-04-21T08:40:29.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.083+0000 7fd9bc5d5640 1 --2- 172.21.15.86:0/361938985 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd9a00a3d20 0x7fd9a00a61e0 unknown :-1 s=CLOSED pgs=114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:29.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.083+0000 7fd9bc5d5640 1 --2- 172.21.15.86:0/361938985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9b410c510 0x7fd9b41985b0 unknown :-1 s=CLOSED pgs=802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:29.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.083+0000 7fd9bc5d5640 1 --2- 172.21.15.86:0/361938985 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9b4103af0 0x7fd9b419e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:29.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.083+0000 7fd9bc5d5640 1 --2- 172.21.15.86:0/361938985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9b41031d0 0x7fd9b419e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:29.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.083+0000 7fd9bc5d5640 1 -- 172.21.15.86:0/361938985 >> 172.21.15.86:0/361938985 conn(0x7fd9b4077f50 msgr2=0x7fd9b40fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:29.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.083+0000 7fd9bc5d5640 1 -- 172.21.15.86:0/361938985 shutdown_connections 2024-04-21T08:40:29.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:29.083+0000 7fd9bc5d5640 1 -- 172.21.15.86:0/361938985 wait complete. 2024-04-21T08:40:30.095 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:40:30.095 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:40:30.095 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:40:30.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.210+0000 7f9b362c1640 1 -- 172.21.15.86:0/2774954259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b30102210 msgr2=0x7f9b301025f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:30.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.210+0000 7f9b362c1640 1 --2- 172.21.15.86:0/2774954259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b30102210 0x7f9b301025f0 secure :-1 s=READY pgs=806 cs=0 l=1 rev1=1 crypto rx=0x7f9b20009900 tx=0x7f9b2002bc30 comp rx=0 tx=0).stop 2024-04-21T08:40:30.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.210+0000 7f9b362c1640 1 -- 172.21.15.86:0/2774954259 shutdown_connections 2024-04-21T08:40:30.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.210+0000 7f9b362c1640 1 --2- 172.21.15.86:0/2774954259 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b301072e0 0x7f9b3010eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.210+0000 7f9b362c1640 1 --2- 172.21.15.86:0/2774954259 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b30102bc0 0x7f9b30106c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.210+0000 7f9b362c1640 1 --2- 172.21.15.86:0/2774954259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b30102210 0x7f9b301025f0 unknown :-1 s=CLOSED pgs=806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.210+0000 7f9b362c1640 1 -- 172.21.15.86:0/2774954259 >> 172.21.15.86:0/2774954259 conn(0x7f9b300fdab0 msgr2=0x7f9b300ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:30.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.210+0000 7f9b362c1640 1 -- 172.21.15.86:0/2774954259 shutdown_connections 2024-04-21T08:40:30.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.210+0000 7f9b362c1640 1 -- 172.21.15.86:0/2774954259 wait complete. 2024-04-21T08:40:30.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.210+0000 7f9b362c1640 1 Processor -- start 2024-04-21T08:40:30.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.210+0000 7f9b362c1640 1 -- start start 2024-04-21T08:40:30.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.211+0000 7f9b362c1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b30102210 0x7f9b3019e650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:30.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.211+0000 7f9b362c1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b301072e0 0x7f9b3019eb90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:30.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.211+0000 7f9b362c1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b30198710 0x7f9b30198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:30.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.211+0000 7f9b362c1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9b30112240 con 0x7f9b301072e0 2024-04-21T08:40:30.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.211+0000 7f9b362c1640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9b301120c0 con 0x7f9b30198710 2024-04-21T08:40:30.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.211+0000 7f9b362c1640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9b301123c0 con 0x7f9b30102210 2024-04-21T08:40:30.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.211+0000 7f9b2f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b301072e0 0x7f9b3019eb90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:30.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.211+0000 7f9b2f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b301072e0 0x7f9b3019eb90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54876/0 (socket says 172.21.15.86:54876) 2024-04-21T08:40:30.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.211+0000 7f9b2f7fe640 1 -- 172.21.15.86:0/2989558081 learned_addr learned my addr 172.21.15.86:0/2989558081 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:30.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.211+0000 7f9b2ffff640 1 --2- 172.21.15.86:0/2989558081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b30102210 0x7f9b3019e650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:30.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.211+0000 7f9b3485e640 1 --2- 172.21.15.86:0/2989558081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b30198710 0x7f9b30198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:30.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.211+0000 7f9b2f7fe640 1 -- 172.21.15.86:0/2989558081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b30102210 msgr2=0x7f9b3019e650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:30.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.211+0000 7f9b2f7fe640 1 --2- 172.21.15.86:0/2989558081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b30102210 0x7f9b3019e650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.211+0000 7f9b2f7fe640 1 -- 172.21.15.86:0/2989558081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b30198710 msgr2=0x7f9b30198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:30.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.211+0000 7f9b2f7fe640 1 --2- 172.21.15.86:0/2989558081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b30198710 0x7f9b30198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.211+0000 7f9b2f7fe640 1 -- 172.21.15.86:0/2989558081 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9b30199310 con 0x7f9b301072e0 2024-04-21T08:40:30.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.211+0000 7f9b3485e640 1 --2- 172.21.15.86:0/2989558081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b30198710 0x7f9b30198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:30.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.211+0000 7f9b2f7fe640 1 --2- 172.21.15.86:0/2989558081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b301072e0 0x7f9b3019eb90 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f9b1800d740 tx=0x7f9b1800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:30.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.212+0000 7f9b2d7fa640 1 -- 172.21.15.86:0/2989558081 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b18017070 con 0x7f9b301072e0 2024-04-21T08:40:30.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.212+0000 7f9b2d7fa640 1 -- 172.21.15.86:0/2989558081 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b1800de50 con 0x7f9b301072e0 2024-04-21T08:40:30.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.212+0000 7f9b362c1640 1 -- 172.21.15.86:0/2989558081 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9b301a53f0 con 0x7f9b301072e0 2024-04-21T08:40:30.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.212+0000 7f9b2d7fa640 1 -- 172.21.15.86:0/2989558081 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b18004ce0 con 0x7f9b301072e0 2024-04-21T08:40:30.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.212+0000 7f9b362c1640 1 -- 172.21.15.86:0/2989558081 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9b301a57f0 con 0x7f9b301072e0 2024-04-21T08:40:30.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.214+0000 7f9b2d7fa640 1 -- 172.21.15.86:0/2989558081 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9b18004e80 con 0x7f9b301072e0 2024-04-21T08:40:30.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.214+0000 7f9b38f93740 1 Processor -- start 2024-04-21T08:40:30.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.214+0000 7f9b38f93740 1 -- start start 2024-04-21T08:40:30.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.214+0000 7f9b2d7fa640 1 --2- 172.21.15.86:0/2989558081 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f9b1c0a3da0 0x7f9b1c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:30.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.214+0000 7f9b38f93740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa05b868e0 0x55fa05b86cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:30.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.214+0000 7f9b38f93740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fa05b87290 0x55fa05b8f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:30.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.214+0000 7f9b38f93740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa05b8fe10 0x55fa05b92260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:30.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.214+0000 7f9b38f93740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fa05aeaf00 con 0x55fa05b8fe10 2024-04-21T08:40:30.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.214+0000 7f9b38f93740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55fa05a88590 con 0x55fa05b87290 2024-04-21T08:40:30.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.214+0000 7f9b38f93740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55fa05ae6ec0 con 0x55fa05b868e0 2024-04-21T08:40:30.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.214+0000 7f9b3485e640 1 --2- 172.21.15.86:0/2989558081 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f9b1c0a3da0 0x7f9b1c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:30.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.214+0000 7f9b3485e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa05b8fe10 0x55fa05b92260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:30.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.214+0000 7f9b2d7fa640 1 -- 172.21.15.86:0/2989558081 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f9b180d0490 con 0x7f9b301072e0 2024-04-21T08:40:30.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.215+0000 7f9b3485e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa05b8fe10 0x55fa05b92260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54888/0 (socket says 172.21.15.86:54888) 2024-04-21T08:40:30.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.215+0000 7f9b3485e640 1 -- 172.21.15.86:0/3261781483 learned_addr learned my addr 172.21.15.86:0/3261781483 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:30.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.215+0000 7f9b2f7fe640 1 --2- 172.21.15.86:0/3261781483 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fa05b87290 0x55fa05b8f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:30.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.215+0000 7f9b2ffff640 1 --2- 172.21.15.86:0/3261781483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa05b868e0 0x55fa05b86cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:30.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.215+0000 7f9b3485e640 1 -- 172.21.15.86:0/3261781483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa05b868e0 msgr2=0x55fa05b86cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:30.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.215+0000 7f9b3485e640 1 --2- 172.21.15.86:0/3261781483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa05b868e0 0x55fa05b86cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.215+0000 7f9b3485e640 1 -- 172.21.15.86:0/3261781483 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fa05b87290 msgr2=0x55fa05b8f840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:30.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.215+0000 7f9b3485e640 1 --2- 172.21.15.86:0/3261781483 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fa05b87290 0x55fa05b8f840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.215+0000 7f9b3485e640 1 -- 172.21.15.86:0/3261781483 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa058c57e0 con 0x55fa05b8fe10 2024-04-21T08:40:30.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.215+0000 7f9b3485e640 1 --2- 172.21.15.86:0/2989558081 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f9b1c0a3da0 0x7f9b1c0a6260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9b28012be0 tx=0x7f9b28009db0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:30.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.215+0000 7f9b3485e640 1 --2- 172.21.15.86:0/3261781483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa05b8fe10 0x55fa05b92260 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f9b28005b30 tx=0x7f9b2801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0419532e251b043 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:30.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.215+0000 7f9b267fc640 1 -- 172.21.15.86:0/3261781483 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2800c7a0 con 0x55fa05b8fe10 2024-04-21T08:40:30.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.215+0000 7f9b267fc640 1 -- 172.21.15.86:0/3261781483 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b28025070 con 0x55fa05b8fe10 2024-04-21T08:40:30.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.215+0000 7f9b267fc640 1 -- 172.21.15.86:0/3261781483 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b28020c50 con 0x55fa05b8fe10 2024-04-21T08:40:30.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.216+0000 7f9b38f93740 1 -- 172.21.15.86:0/3261781483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa05b8fe10 msgr2=0x55fa05b92260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:30.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.216+0000 7f9b38f93740 1 --2- 172.21.15.86:0/3261781483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa05b8fe10 0x55fa05b92260 secure :-1 s=READY pgs=808 cs=0 l=1 rev1=1 crypto rx=0x7f9b28005b30 tx=0x7f9b2801c000 comp rx=0 tx=0).stop 2024-04-21T08:40:30.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.216+0000 7f9b38f93740 1 -- 172.21.15.86:0/3261781483 shutdown_connections 2024-04-21T08:40:30.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.216+0000 7f9b38f93740 1 --2- 172.21.15.86:0/3261781483 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa05b8fe10 0x55fa05b92260 unknown :-1 s=CLOSED pgs=808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.216+0000 7f9b38f93740 1 --2- 172.21.15.86:0/3261781483 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fa05b87290 0x55fa05b8f840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.216+0000 7f9b38f93740 1 --2- 172.21.15.86:0/3261781483 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa05b868e0 0x55fa05b86cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.216+0000 7f9b38f93740 1 -- 172.21.15.86:0/3261781483 >> 172.21.15.86:0/3261781483 conn(0x55fa05b80fb0 msgr2=0x55fa05b81390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:30.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.216+0000 7f9b38f93740 1 -- 172.21.15.86:0/3261781483 shutdown_connections 2024-04-21T08:40:30.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.216+0000 7f9b38f93740 1 -- 172.21.15.86:0/3261781483 wait complete. 2024-04-21T08:40:30.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.216+0000 7f9b38f93740 1 -- start start 2024-04-21T08:40:30.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.216+0000 7f9b38f93740 1 Processor -- start 2024-04-21T08:40:30.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.217+0000 7f9b38f93740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa05b868e0 0x55fa05b8cd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:30.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.217+0000 7f9b38f93740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fa05b87290 0x55fa05b8d2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:30.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.217+0000 7f9b38f93740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa05b8fe10 0x55fa05b8d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:30.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.217+0000 7f9b38f93740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fa05b92d20 con 0x55fa05b8fe10 2024-04-21T08:40:30.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.217+0000 7f9b38f93740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55fa05aa4580 con 0x55fa05b87290 2024-04-21T08:40:30.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.217+0000 7f9b38f93740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55fa05b92ea0 con 0x55fa05b868e0 2024-04-21T08:40:30.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.217+0000 7f9b38f93740 10 client.0 _ll_register_callbacks cb 0x55fa05aaa650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:30.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.217+0000 7f9b3485e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa05b8fe10 0x55fa05b8d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:30.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.217+0000 7f9b3485e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa05b8fe10 0x55fa05b8d7e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54894/0 (socket says 172.21.15.86:54894) 2024-04-21T08:40:30.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.217+0000 7f9b3485e640 1 -- 172.21.15.86:0/3894534152 learned_addr learned my addr 172.21.15.86:0/3894534152 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:30.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.217+0000 7f9b2f7fe640 1 --2- 172.21.15.86:0/3894534152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fa05b87290 0x55fa05b8d2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:30.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.217+0000 7f9b2ffff640 1 --2- 172.21.15.86:0/3894534152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa05b868e0 0x55fa05b8cd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:30.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.217+0000 7f9b3485e640 1 -- 172.21.15.86:0/3894534152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa05b868e0 msgr2=0x55fa05b8cd60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:30.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.217+0000 7f9b3485e640 1 --2- 172.21.15.86:0/3894534152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa05b868e0 0x55fa05b8cd60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.217+0000 7f9b3485e640 1 -- 172.21.15.86:0/3894534152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fa05b87290 msgr2=0x55fa05b8d2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:30.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.217+0000 7f9b3485e640 1 --2- 172.21.15.86:0/3894534152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fa05b87290 0x55fa05b8d2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.217+0000 7f9b3485e640 1 -- 172.21.15.86:0/3894534152 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa05925f70 con 0x55fa05b8fe10 2024-04-21T08:40:30.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.217+0000 7f9b3485e640 1 --2- 172.21.15.86:0/3894534152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa05b8fe10 0x55fa05b8d7e0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f9b280109a0 tx=0x7f9b280206e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcb89ebec69d131b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:30.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.217+0000 7f9b0f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:40:30.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.217+0000 7f9b0f7fe640 1 -- 172.21.15.86:0/3894534152 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2802f070 con 0x55fa05b8fe10 2024-04-21T08:40:30.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.217+0000 7f9b0f7fe640 1 -- 172.21.15.86:0/3894534152 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f9b28020de0 con 0x55fa05b8fe10 2024-04-21T08:40:30.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.218+0000 7f9b38f93740 1 -- 172.21.15.86:0/3894534152 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b28025070 con 0x55fa05b8fe10 2024-04-21T08:40:30.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.218+0000 7f9b0f7fe640 1 -- 172.21.15.86:0/3894534152 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9b2803b050 con 0x55fa05b8fe10 2024-04-21T08:40:30.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.218+0000 7f9b38f93740 10 client.16590 fetch_fsmap learned FSMap version 29 2024-04-21T08:40:30.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.218+0000 7f9b38f93740 1 -- 172.21.15.86:0/3894534152 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fa057bd170 con 0x55fa05b8fe10 2024-04-21T08:40:30.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.218+0000 7f9b0f7fe640 1 -- 172.21.15.86:0/3894534152 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b28025070 con 0x55fa05b8fe10 2024-04-21T08:40:30.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.218+0000 7f9b0f7fe640 1 -- 172.21.15.86:0/3894534152 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9b2801f070 con 0x55fa05b8fe10 2024-04-21T08:40:30.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.218+0000 7f9b38f93740 10 client.16590 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:40:30.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.218+0000 7f9b38f93740 10 client.16590 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:30.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.218+0000 7f9b38f93740 20 client.16590 populate_metadata read hostname 'smithi086' 2024-04-21T08:40:30.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.218+0000 7f9b38f93740 1 --2- 172.21.15.86:0/3894534152 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55fa05ba0200 0x55fa05ba25f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:30.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.218+0000 7f9b38f93740 4 client.16590 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:40:30.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.218+0000 7f9b38f93740 1 -- 172.21.15.86:0/3894534152 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fa05af8920 con 0x55fa05ba0200 2024-04-21T08:40:30.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.218+0000 7f9b2ffff640 1 --2- 172.21.15.86:0/3894534152 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55fa05ba0200 0x55fa05ba25f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:30.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.219+0000 7f9b0f7fe640 10 client.16590 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:40:30.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.219+0000 7f9b2ffff640 1 --2- 172.21.15.86:0/3894534152 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55fa05ba0200 0x55fa05ba25f0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=90aeed5a56084b9d server_cookie=958f554d14177436 in_seq=0 out_seq=0 2024-04-21T08:40:30.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.219+0000 7f9b0f7fe640 1 -- 172.21.15.86:0/3894534152 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fa05af8920 con 0x55fa05ba0200 2024-04-21T08:40:30.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.220+0000 7f9b0f7fe640 10 client.16590 handle_command_reply: tid=0 2024-04-21T08:40:30.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.220+0000 7f9b38f93740 1 client.16590 shutdown 2024-04-21T08:40:30.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.220+0000 7f9b38f93740 1 -- 172.21.15.86:0/3894534152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa05b8fe10 msgr2=0x55fa05b8d7e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:30.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.220+0000 7f9b38f93740 1 --2- 172.21.15.86:0/3894534152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa05b8fe10 0x55fa05b8d7e0 secure :-1 s=READY pgs=809 cs=0 l=1 rev1=1 crypto rx=0x7f9b280109a0 tx=0x7f9b280206e0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.220+0000 7f9b38f93740 1 -- 172.21.15.86:0/3894534152 shutdown_connections 2024-04-21T08:40:30.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.220+0000 7f9b38f93740 1 --2- 172.21.15.86:0/3894534152 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55fa05ba0200 0x55fa05ba25f0 crc :-1 s=READY pgs=85 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.220+0000 7f9b38f93740 1 --2- 172.21.15.86:0/3894534152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa05b8fe10 0x55fa05b8d7e0 unknown :-1 s=CLOSED pgs=809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.220+0000 7f9b38f93740 1 --2- 172.21.15.86:0/3894534152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fa05b87290 0x55fa05b8d2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.220+0000 7f9b0f7fe640 0 client.16590 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:40:30.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.220+0000 7f9b38f93740 1 --2- 172.21.15.86:0/3894534152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa05b868e0 0x55fa05b8cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.220+0000 7f9b38f93740 1 -- 172.21.15.86:0/3894534152 >> 172.21.15.86:0/3894534152 conn(0x55fa05b80fb0 msgr2=0x55fa05b82d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:30.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.220+0000 7f9b38f93740 1 -- 172.21.15.86:0/3894534152 shutdown_connections 2024-04-21T08:40:30.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.220+0000 7f9b38f93740 1 -- 172.21.15.86:0/3894534152 wait complete. 2024-04-21T08:40:30.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.220+0000 7f9b38f93740 20 client.16590 trim_cache size 0 max 16384 2024-04-21T08:40:30.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.237+0000 7f9b38f93740 1 Processor -- start 2024-04-21T08:40:30.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.237+0000 7f9b38f93740 1 -- start start 2024-04-21T08:40:30.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.237+0000 7f9b38f93740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa05b868e0 0x55fa05b9bc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:30.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.237+0000 7f9b38f93740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fa05b87290 0x55fa05b8d600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:30.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.237+0000 7f9b38f93740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa05b8fe10 0x55fa05ba7e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:30.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.237+0000 7f9b38f93740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9b301af2c0 con 0x55fa05b8fe10 2024-04-21T08:40:30.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.237+0000 7f9b38f93740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9b301b0e00 con 0x55fa05b87290 2024-04-21T08:40:30.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.237+0000 7f9b38f93740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9b301af990 con 0x55fa05b868e0 2024-04-21T08:40:30.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.238+0000 7f9b3485e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa05b8fe10 0x55fa05ba7e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:30.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.238+0000 7f9b3485e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa05b8fe10 0x55fa05ba7e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54908/0 (socket says 172.21.15.86:54908) 2024-04-21T08:40:30.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.238+0000 7f9b3485e640 1 -- 172.21.15.86:0/3075804878 learned_addr learned my addr 172.21.15.86:0/3075804878 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:30.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.238+0000 7f9b2f7fe640 1 --2- 172.21.15.86:0/3075804878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fa05b87290 0x55fa05b8d600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:30.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.238+0000 7f9b2ffff640 1 --2- 172.21.15.86:0/3075804878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa05b868e0 0x55fa05b9bc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:30.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.238+0000 7f9b3485e640 1 -- 172.21.15.86:0/3075804878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa05b868e0 msgr2=0x55fa05b9bc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:30.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.238+0000 7f9b3485e640 1 --2- 172.21.15.86:0/3075804878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa05b868e0 0x55fa05b9bc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.238+0000 7f9b3485e640 1 -- 172.21.15.86:0/3075804878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fa05b87290 msgr2=0x55fa05b8d600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:30.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.238+0000 7f9b3485e640 1 --2- 172.21.15.86:0/3075804878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fa05b87290 0x55fa05b8d600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.238+0000 7f9b3485e640 1 -- 172.21.15.86:0/3075804878 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fa05b87030 con 0x55fa05b8fe10 2024-04-21T08:40:30.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.238+0000 7f9b2f7fe640 1 --2- 172.21.15.86:0/3075804878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fa05b87290 0x55fa05b8d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:30.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.238+0000 7f9b3485e640 1 --2- 172.21.15.86:0/3075804878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa05b8fe10 0x55fa05ba7e00 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f9b2800c6b0 tx=0x7f9b28009970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=afed68a657303547 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:30.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.238+0000 7f9b25ffb640 1 -- 172.21.15.86:0/3075804878 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b2802f070 con 0x55fa05b8fe10 2024-04-21T08:40:30.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.238+0000 7f9b25ffb640 1 -- 172.21.15.86:0/3075804878 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b28025040 con 0x55fa05b8fe10 2024-04-21T08:40:30.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.238+0000 7f9b25ffb640 1 -- 172.21.15.86:0/3075804878 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b28004a00 con 0x55fa05b8fe10 2024-04-21T08:40:30.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.239+0000 7f9b38f93740 1 -- 172.21.15.86:0/3075804878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa05b8fe10 msgr2=0x55fa05ba7e00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:30.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.239+0000 7f9b38f93740 1 --2- 172.21.15.86:0/3075804878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa05b8fe10 0x55fa05ba7e00 secure :-1 s=READY pgs=810 cs=0 l=1 rev1=1 crypto rx=0x7f9b2800c6b0 tx=0x7f9b28009970 comp rx=0 tx=0).stop 2024-04-21T08:40:30.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.239+0000 7f9b38f93740 1 -- 172.21.15.86:0/3075804878 shutdown_connections 2024-04-21T08:40:30.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.239+0000 7f9b38f93740 1 --2- 172.21.15.86:0/3075804878 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa05b8fe10 0x55fa05ba7e00 unknown :-1 s=CLOSED pgs=810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.239+0000 7f9b38f93740 1 --2- 172.21.15.86:0/3075804878 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fa05b87290 0x55fa05b8d600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.239+0000 7f9b38f93740 1 --2- 172.21.15.86:0/3075804878 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa05b868e0 0x55fa05b9bc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.239+0000 7f9b38f93740 1 -- 172.21.15.86:0/3075804878 >> 172.21.15.86:0/3075804878 conn(0x55fa05b80fb0 msgr2=0x55fa05b8bd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:30.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.239+0000 7f9b38f93740 1 -- 172.21.15.86:0/3075804878 shutdown_connections 2024-04-21T08:40:30.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.239+0000 7f9b38f93740 1 -- 172.21.15.86:0/3075804878 wait complete. 2024-04-21T08:40:30.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.239+0000 7f9b38f93740 1 -- start start 2024-04-21T08:40:30.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.239+0000 7f9b38f93740 1 Processor -- start 2024-04-21T08:40:30.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.239+0000 7f9b38f93740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fa05b868e0 0x55fa05b93db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:30.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.239+0000 7f9b38f93740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa05b87290 0x55fa05b942f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:30.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.239+0000 7f9b38f93740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa05b8fe10 0x55fa05b94830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:30.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.239+0000 7f9b38f93740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9b301b0730 con 0x55fa05b87290 2024-04-21T08:40:30.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.239+0000 7f9b38f93740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9b301b0060 con 0x55fa05b868e0 2024-04-21T08:40:30.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.239+0000 7f9b38f93740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9b301ad280 con 0x55fa05b8fe10 2024-04-21T08:40:30.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.239+0000 7f9b38f93740 10 client.0 _ll_register_callbacks cb 0x7f9b18098400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:30.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.240+0000 7f9b2f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa05b87290 0x55fa05b942f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:30.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.240+0000 7f9b2f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa05b87290 0x55fa05b942f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54914/0 (socket says 172.21.15.86:54914) 2024-04-21T08:40:30.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.240+0000 7f9b2f7fe640 1 -- 172.21.15.86:0/1108455978 learned_addr learned my addr 172.21.15.86:0/1108455978 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:30.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.240+0000 7f9b3485e640 1 --2- 172.21.15.86:0/1108455978 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa05b8fe10 0x55fa05b94830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:30.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.240+0000 7f9b2ffff640 1 --2- 172.21.15.86:0/1108455978 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fa05b868e0 0x55fa05b93db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:30.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.240+0000 7f9b2f7fe640 1 -- 172.21.15.86:0/1108455978 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa05b8fe10 msgr2=0x55fa05b94830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:30.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.240+0000 7f9b2f7fe640 1 --2- 172.21.15.86:0/1108455978 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa05b8fe10 0x55fa05b94830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.240+0000 7f9b2f7fe640 1 -- 172.21.15.86:0/1108455978 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fa05b868e0 msgr2=0x55fa05b93db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:30.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.240+0000 7f9b2f7fe640 1 --2- 172.21.15.86:0/1108455978 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fa05b868e0 0x55fa05b93db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.240+0000 7f9b2f7fe640 1 -- 172.21.15.86:0/1108455978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fa05c058c0 con 0x55fa05b87290 2024-04-21T08:40:30.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.240+0000 7f9b2ffff640 1 --2- 172.21.15.86:0/1108455978 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fa05b868e0 0x55fa05b93db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:40:30.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.240+0000 7f9b2f7fe640 1 --2- 172.21.15.86:0/1108455978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa05b87290 0x55fa05b942f0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f9b18098a90 tx=0x7f9b180a0530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1f7c442130c93822 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:30.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.240+0000 7f9b0f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:40:30.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.240+0000 7f9b0f7fe640 1 -- 172.21.15.86:0/1108455978 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9b18020040 con 0x55fa05b87290 2024-04-21T08:40:30.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.240+0000 7f9b0f7fe640 1 -- 172.21.15.86:0/1108455978 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f9b180d0150 con 0x55fa05b87290 2024-04-21T08:40:30.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.240+0000 7f9b38f93740 1 -- 172.21.15.86:0/1108455978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9b28025040 con 0x55fa05b87290 2024-04-21T08:40:30.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.240+0000 7f9b0f7fe640 1 -- 172.21.15.86:0/1108455978 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9b1809a480 con 0x55fa05b87290 2024-04-21T08:40:30.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.240+0000 7f9b38f93740 10 client.16596 fetch_fsmap learned FSMap version 29 2024-04-21T08:40:30.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.240+0000 7f9b38f93740 1 -- 172.21.15.86:0/1108455978 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fa05c06c00 con 0x55fa05b87290 2024-04-21T08:40:30.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.240+0000 7f9b0f7fe640 1 -- 172.21.15.86:0/1108455978 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9b28025040 con 0x55fa05b87290 2024-04-21T08:40:30.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.241+0000 7f9b0f7fe640 1 -- 172.21.15.86:0/1108455978 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9b180a48a0 con 0x55fa05b87290 2024-04-21T08:40:30.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.241+0000 7f9b38f93740 10 client.16596 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:40:30.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.241+0000 7f9b38f93740 10 client.16596 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:30.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.241+0000 7f9b38f93740 20 client.16596 populate_metadata read hostname 'smithi086' 2024-04-21T08:40:30.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.241+0000 7f9b38f93740 1 --2- 172.21.15.86:0/1108455978 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55fa05c07320 0x55fa05c09740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:30.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.241+0000 7f9b38f93740 4 client.16596 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:40:30.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.241+0000 7f9b38f93740 1 -- 172.21.15.86:0/1108455978 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55fa05af2790 con 0x55fa05c07320 2024-04-21T08:40:30.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.241+0000 7f9b2ffff640 1 --2- 172.21.15.86:0/1108455978 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55fa05c07320 0x55fa05c09740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:30.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.241+0000 7f9b0f7fe640 10 client.16596 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:40:30.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.241+0000 7f9b2ffff640 1 --2- 172.21.15.86:0/1108455978 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55fa05c07320 0x55fa05c09740 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7895dc370b0390d5 server_cookie=f15f041464917526 in_seq=0 out_seq=0 2024-04-21T08:40:30.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.243+0000 7f9b0f7fe640 1 -- 172.21.15.86:0/1108455978 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55fa05af2790 con 0x55fa05c07320 2024-04-21T08:40:30.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.243+0000 7f9b0f7fe640 10 client.16596 handle_command_reply: tid=0 2024-04-21T08:40:30.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.243+0000 7f9b38f93740 1 client.16596 shutdown 2024-04-21T08:40:30.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.243+0000 7f9b38f93740 1 -- 172.21.15.86:0/1108455978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa05b87290 msgr2=0x55fa05b942f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:30.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.243+0000 7f9b38f93740 1 --2- 172.21.15.86:0/1108455978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa05b87290 0x55fa05b942f0 secure :-1 s=READY pgs=811 cs=0 l=1 rev1=1 crypto rx=0x7f9b18098a90 tx=0x7f9b180a0530 comp rx=0 tx=0).stop 2024-04-21T08:40:30.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.243+0000 7f9b38f93740 1 -- 172.21.15.86:0/1108455978 shutdown_connections 2024-04-21T08:40:30.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.243+0000 7f9b38f93740 1 --2- 172.21.15.86:0/1108455978 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55fa05c07320 0x55fa05c09740 crc :-1 s=READY pgs=86 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.243+0000 7f9b38f93740 1 --2- 172.21.15.86:0/1108455978 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fa05b8fe10 0x55fa05b94830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.243+0000 7f9b38f93740 1 --2- 172.21.15.86:0/1108455978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fa05b87290 0x55fa05b942f0 unknown :-1 s=CLOSED pgs=811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.243+0000 7f9b38f93740 1 --2- 172.21.15.86:0/1108455978 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fa05b868e0 0x55fa05b93db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.243+0000 7f9b38f93740 1 -- 172.21.15.86:0/1108455978 >> 172.21.15.86:0/1108455978 conn(0x55fa05b80fb0 msgr2=0x55fa05b7f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:30.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.243+0000 7f9b0f7fe640 0 client.16596 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:40:30.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.243+0000 7f9b38f93740 1 -- 172.21.15.86:0/1108455978 shutdown_connections 2024-04-21T08:40:30.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.243+0000 7f9b38f93740 1 -- 172.21.15.86:0/1108455978 wait complete. 2024-04-21T08:40:30.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.243+0000 7f9b38f93740 20 client.16596 trim_cache size 0 max 16384 2024-04-21T08:40:30.250 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:40:30.250 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-21T08:40:30.250 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "f6024a57-724d-4ca4-8145-a026dcc1395e", 2024-04-21T08:40:30.251 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-21T08:40:30.251 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:40:30.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.244+0000 7f9b362c1640 1 -- 172.21.15.86:0/2989558081 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f9b1c0a3da0 msgr2=0x7f9b1c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:30.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.244+0000 7f9b362c1640 1 --2- 172.21.15.86:0/2989558081 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f9b1c0a3da0 0x7f9b1c0a6260 secure :-1 s=READY pgs=115 cs=0 l=1 rev1=1 crypto rx=0x7f9b28012be0 tx=0x7f9b28009db0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.244+0000 7f9b362c1640 1 -- 172.21.15.86:0/2989558081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b301072e0 msgr2=0x7f9b3019eb90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:30.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.244+0000 7f9b362c1640 1 --2- 172.21.15.86:0/2989558081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b301072e0 0x7f9b3019eb90 secure :-1 s=READY pgs=807 cs=0 l=1 rev1=1 crypto rx=0x7f9b1800d740 tx=0x7f9b1800dc00 comp rx=0 tx=0).stop 2024-04-21T08:40:30.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.244+0000 7f9b362c1640 1 -- 172.21.15.86:0/2989558081 shutdown_connections 2024-04-21T08:40:30.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.244+0000 7f9b362c1640 1 --2- 172.21.15.86:0/2989558081 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f9b1c0a3da0 0x7f9b1c0a6260 unknown :-1 s=CLOSED pgs=115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.244+0000 7f9b362c1640 1 --2- 172.21.15.86:0/2989558081 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9b30198710 0x7f9b30198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.244+0000 7f9b362c1640 1 --2- 172.21.15.86:0/2989558081 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9b301072e0 0x7f9b3019eb90 unknown :-1 s=CLOSED pgs=807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.244+0000 7f9b362c1640 1 --2- 172.21.15.86:0/2989558081 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9b30102210 0x7f9b3019e650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.244+0000 7f9b362c1640 1 -- 172.21.15.86:0/2989558081 >> 172.21.15.86:0/2989558081 conn(0x7f9b300fdab0 msgr2=0x7f9b300fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:30.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.244+0000 7f9b362c1640 1 -- 172.21.15.86:0/2989558081 shutdown_connections 2024-04-21T08:40:30.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.244+0000 7f9b362c1640 1 -- 172.21.15.86:0/2989558081 wait complete. 2024-04-21T08:40:30.255 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:40:30.255 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:40:30.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.355+0000 7f79882c0640 1 -- 172.21.15.86:0/1814751396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7980106150 msgr2=0x7f798010dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:30.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.355+0000 7f79882c0640 1 --2- 172.21.15.86:0/1814751396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7980106150 0x7f798010dd20 secure :-1 s=READY pgs=812 cs=0 l=1 rev1=1 crypto rx=0x7f797c009900 tx=0x7f797c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:40:30.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.355+0000 7f79882c0640 1 -- 172.21.15.86:0/1814751396 shutdown_connections 2024-04-21T08:40:30.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.355+0000 7f79882c0640 1 --2- 172.21.15.86:0/1814751396 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7980106150 0x7f798010dd20 unknown :-1 s=CLOSED pgs=812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.355+0000 7f79882c0640 1 --2- 172.21.15.86:0/1814751396 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7980101a30 0x7f7980105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.355+0000 7f79882c0640 1 --2- 172.21.15.86:0/1814751396 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7980101080 0x7f7980101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.355+0000 7f79882c0640 1 -- 172.21.15.86:0/1814751396 >> 172.21.15.86:0/1814751396 conn(0x7f7980078000 msgr2=0x7f7980078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:30.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.355+0000 7f79882c0640 1 -- 172.21.15.86:0/1814751396 shutdown_connections 2024-04-21T08:40:30.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.355+0000 7f79882c0640 1 -- 172.21.15.86:0/1814751396 wait complete. 2024-04-21T08:40:30.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.356+0000 7f79882c0640 1 Processor -- start 2024-04-21T08:40:30.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.356+0000 7f79882c0640 1 -- start start 2024-04-21T08:40:30.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.356+0000 7f79882c0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7980101080 0x7f79801986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:30.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.356+0000 7f79882c0640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7980106150 0x7f7980198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:30.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.356+0000 7f79882c0640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f798019cf90 0x7f798019d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:30.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.356+0000 7f79882c0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f79801121d0 con 0x7f7980101080 2024-04-21T08:40:30.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.356+0000 7f79882c0640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7980112050 con 0x7f798019cf90 2024-04-21T08:40:30.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.356+0000 7f79882c0640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7980112350 con 0x7f7980106150 2024-04-21T08:40:30.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.356+0000 7f798605c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7980101080 0x7f79801986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:30.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.356+0000 7f798605c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7980101080 0x7f79801986d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54938/0 (socket says 172.21.15.86:54938) 2024-04-21T08:40:30.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.356+0000 7f798605c640 1 -- 172.21.15.86:0/811869505 learned_addr learned my addr 172.21.15.86:0/811869505 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:30.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.357+0000 7f798605c640 1 -- 172.21.15.86:0/811869505 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7980106150 msgr2=0x7f7980198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:30.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.357+0000 7f798605c640 1 --2- 172.21.15.86:0/811869505 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7980106150 0x7f7980198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.357+0000 7f798605c640 1 -- 172.21.15.86:0/811869505 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f798019cf90 msgr2=0x7f798019d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:30.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.357+0000 7f798605c640 1 --2- 172.21.15.86:0/811869505 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f798019cf90 0x7f798019d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.357+0000 7f798605c640 1 -- 172.21.15.86:0/811869505 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f798019da80 con 0x7f7980101080 2024-04-21T08:40:30.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.357+0000 7f798605c640 1 --2- 172.21.15.86:0/811869505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7980101080 0x7f79801986d0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f797800cce0 tx=0x7f797800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:30.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.357+0000 7f79777fe640 1 -- 172.21.15.86:0/811869505 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7978004b20 con 0x7f7980101080 2024-04-21T08:40:30.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.357+0000 7f79777fe640 1 -- 172.21.15.86:0/811869505 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7978004cc0 con 0x7f7980101080 2024-04-21T08:40:30.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.357+0000 7f79882c0640 1 -- 172.21.15.86:0/811869505 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f798019dd70 con 0x7f7980101080 2024-04-21T08:40:30.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.357+0000 7f79777fe640 1 -- 172.21.15.86:0/811869505 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797800f7c0 con 0x7f7980101080 2024-04-21T08:40:30.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.357+0000 7f79882c0640 1 -- 172.21.15.86:0/811869505 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79801a5510 con 0x7f7980101080 2024-04-21T08:40:30.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.359+0000 7f79777fe640 1 -- 172.21.15.86:0/811869505 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7978005560 con 0x7f7980101080 2024-04-21T08:40:30.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.359+0000 7f798af81740 1 Processor -- start 2024-04-21T08:40:30.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.360+0000 7f798af81740 1 -- start start 2024-04-21T08:40:30.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.360+0000 7f798af81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560bf83f0580 0x560bf83f0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:30.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.360+0000 7f798af81740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560bf83f0ea0 0x560bf83f9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:30.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.360+0000 7f798af81740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560bf83f9a80 0x560bf83fbe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:30.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.360+0000 7f798af81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560bf83551e0 con 0x560bf83f0580 2024-04-21T08:40:30.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.360+0000 7f798af81740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x560bf82f2390 con 0x560bf83f9a80 2024-04-21T08:40:30.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.360+0000 7f798af81740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x560bf83511a0 con 0x560bf83f0ea0 2024-04-21T08:40:30.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.360+0000 7f798685d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560bf83f0580 0x560bf83f0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:30.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.360+0000 7f798685d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560bf83f0580 0x560bf83f0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54954/0 (socket says 172.21.15.86:54954) 2024-04-21T08:40:30.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.360+0000 7f798685d640 1 -- 172.21.15.86:0/657097313 learned_addr learned my addr 172.21.15.86:0/657097313 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:30.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.360+0000 7f79777fe640 1 --2- 172.21.15.86:0/811869505 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f796c0a3d20 0x7f796c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:30.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.360+0000 7f798585b640 1 --2- 172.21.15.86:0/657097313 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560bf83f9a80 0x560bf83fbe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:30.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.360+0000 7f798605c640 1 --2- 172.21.15.86:0/657097313 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560bf83f0ea0 0x560bf83f9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:30.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.360+0000 7f798685d640 1 --2- 172.21.15.86:0/811869505 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f796c0a3d20 0x7f796c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:30.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.360+0000 7f79777fe640 1 -- 172.21.15.86:0/811869505 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f79780d0ce0 con 0x7f7980101080 2024-04-21T08:40:30.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.360+0000 7f798685d640 1 -- 172.21.15.86:0/657097313 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560bf83f0ea0 msgr2=0x560bf83f9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:30.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.360+0000 7f798685d640 1 --2- 172.21.15.86:0/657097313 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560bf83f0ea0 0x560bf83f9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.360+0000 7f798685d640 1 -- 172.21.15.86:0/657097313 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560bf83f9a80 msgr2=0x560bf83fbe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:30.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.360+0000 7f798685d640 1 --2- 172.21.15.86:0/657097313 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560bf83f9a80 0x560bf83fbe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.360+0000 7f798685d640 1 -- 172.21.15.86:0/657097313 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bf81317d0 con 0x560bf83f0580 2024-04-21T08:40:30.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.361+0000 7f798685d640 1 --2- 172.21.15.86:0/657097313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560bf83f0580 0x560bf83f0960 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f797c036040 tx=0x7f797c0043a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90a92819e06d4572 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:30.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.361+0000 7f7974bfe640 1 -- 172.21.15.86:0/657097313 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797c0035f0 con 0x560bf83f0580 2024-04-21T08:40:30.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.361+0000 7f798685d640 1 --2- 172.21.15.86:0/811869505 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f796c0a3d20 0x7f796c0a61e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f797c04b000 tx=0x7f797c047000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:30.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.361+0000 7f7974bfe640 1 -- 172.21.15.86:0/657097313 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f797c003c40 con 0x560bf83f0580 2024-04-21T08:40:30.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.361+0000 7f7974bfe640 1 -- 172.21.15.86:0/657097313 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797c034950 con 0x560bf83f0580 2024-04-21T08:40:30.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.361+0000 7f798af81740 1 -- 172.21.15.86:0/657097313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560bf83f0580 msgr2=0x560bf83f0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:30.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.362+0000 7f798af81740 1 --2- 172.21.15.86:0/657097313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560bf83f0580 0x560bf83f0960 secure :-1 s=READY pgs=814 cs=0 l=1 rev1=1 crypto rx=0x7f797c036040 tx=0x7f797c0043a0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.362+0000 7f798af81740 1 -- 172.21.15.86:0/657097313 shutdown_connections 2024-04-21T08:40:30.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.362+0000 7f798af81740 1 --2- 172.21.15.86:0/657097313 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560bf83f9a80 0x560bf83fbe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.362+0000 7f798af81740 1 --2- 172.21.15.86:0/657097313 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560bf83f0ea0 0x560bf83f9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.362+0000 7f798af81740 1 --2- 172.21.15.86:0/657097313 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560bf83f0580 0x560bf83f0960 unknown :-1 s=CLOSED pgs=814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.362+0000 7f798af81740 1 -- 172.21.15.86:0/657097313 >> 172.21.15.86:0/657097313 conn(0x560bf83eacb0 msgr2=0x560bf83eb090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:30.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.362+0000 7f798af81740 1 -- 172.21.15.86:0/657097313 shutdown_connections 2024-04-21T08:40:30.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.362+0000 7f798af81740 1 -- 172.21.15.86:0/657097313 wait complete. 2024-04-21T08:40:30.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.362+0000 7f798af81740 1 -- start start 2024-04-21T08:40:30.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.362+0000 7f798af81740 1 Processor -- start 2024-04-21T08:40:30.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.362+0000 7f798af81740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560bf83f0580 0x560bf83f6b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:30.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.362+0000 7f798af81740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560bf83f0ea0 0x560bf83f70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:30.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.362+0000 7f798af81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560bf83f9a80 0x560bf83f7600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:30.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.362+0000 7f798af81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560bf83fc870 con 0x560bf83f9a80 2024-04-21T08:40:30.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.362+0000 7f798af81740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x560bf830e2f0 con 0x560bf83f0580 2024-04-21T08:40:30.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.362+0000 7f798af81740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x560bf83fc9f0 con 0x560bf83f0ea0 2024-04-21T08:40:30.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.362+0000 7f798af81740 10 client.0 _ll_register_callbacks cb 0x560bf831b580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:30.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.362+0000 7f798685d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560bf83f9a80 0x560bf83f7600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:30.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.362+0000 7f798685d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560bf83f9a80 0x560bf83f7600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54970/0 (socket says 172.21.15.86:54970) 2024-04-21T08:40:30.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.362+0000 7f798685d640 1 -- 172.21.15.86:0/2319253154 learned_addr learned my addr 172.21.15.86:0/2319253154 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:30.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.362+0000 7f798605c640 1 --2- 172.21.15.86:0/2319253154 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560bf83f0580 0x560bf83f6b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:30.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.363+0000 7f798585b640 1 --2- 172.21.15.86:0/2319253154 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560bf83f0ea0 0x560bf83f70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:30.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.363+0000 7f798685d640 1 -- 172.21.15.86:0/2319253154 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560bf83f0ea0 msgr2=0x560bf83f70c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:30.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.363+0000 7f798685d640 1 --2- 172.21.15.86:0/2319253154 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560bf83f0ea0 0x560bf83f70c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.363+0000 7f798685d640 1 -- 172.21.15.86:0/2319253154 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560bf83f0580 msgr2=0x560bf83f6b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:30.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.363+0000 7f798685d640 1 --2- 172.21.15.86:0/2319253154 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560bf83f0580 0x560bf83f6b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.363+0000 7f798685d640 1 -- 172.21.15.86:0/2319253154 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bf80673f0 con 0x560bf83f9a80 2024-04-21T08:40:30.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.363+0000 7f798585b640 1 --2- 172.21.15.86:0/2319253154 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560bf83f0ea0 0x560bf83f70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:30.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.363+0000 7f798685d640 1 --2- 172.21.15.86:0/2319253154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560bf83f9a80 0x560bf83f7600 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f797c0345b0 tx=0x7f797c034440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=181e27c2e710a1c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:30.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.363+0000 7f7969ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:40:30.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.363+0000 7f7969ffb640 1 -- 172.21.15.86:0/2319253154 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797c0035f0 con 0x560bf83f9a80 2024-04-21T08:40:30.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.363+0000 7f7969ffb640 1 -- 172.21.15.86:0/2319253154 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f797c003790 con 0x560bf83f9a80 2024-04-21T08:40:30.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.363+0000 7f798af81740 1 -- 172.21.15.86:0/2319253154 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f797c003c40 con 0x560bf83f9a80 2024-04-21T08:40:30.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.363+0000 7f7969ffb640 1 -- 172.21.15.86:0/2319253154 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f797c003990 con 0x560bf83f9a80 2024-04-21T08:40:30.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.363+0000 7f798af81740 10 client.16608 fetch_fsmap learned FSMap version 29 2024-04-21T08:40:30.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.363+0000 7f798af81740 1 -- 172.21.15.86:0/2319253154 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560bf80270f0 con 0x560bf83f9a80 2024-04-21T08:40:30.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.363+0000 7f7969ffb640 1 -- 172.21.15.86:0/2319253154 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f797c003c40 con 0x560bf83f9a80 2024-04-21T08:40:30.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.363+0000 7f7969ffb640 1 -- 172.21.15.86:0/2319253154 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f797c040040 con 0x560bf83f9a80 2024-04-21T08:40:30.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.363+0000 7f798af81740 10 client.16608 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:40:30.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.363+0000 7f798af81740 10 client.16608 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:30.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.364+0000 7f798af81740 20 client.16608 populate_metadata read hostname 'smithi086' 2024-04-21T08:40:30.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.364+0000 7f798af81740 1 --2- 172.21.15.86:0/2319253154 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x560bf8409950 0x560bf840bd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:30.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.364+0000 7f798af81740 4 client.16608 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:40:30.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.364+0000 7f798af81740 1 -- 172.21.15.86:0/2319253154 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560bf8362c00 con 0x560bf8409950 2024-04-21T08:40:30.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.364+0000 7f798605c640 1 --2- 172.21.15.86:0/2319253154 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x560bf8409950 0x560bf840bd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:30.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.364+0000 7f7969ffb640 10 client.16608 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:40:30.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.364+0000 7f798605c640 1 --2- 172.21.15.86:0/2319253154 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x560bf8409950 0x560bf840bd40 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ac2acad1aa3723a server_cookie=d54390e0b8ae6f49 in_seq=0 out_seq=0 2024-04-21T08:40:30.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.365+0000 7f7969ffb640 1 -- 172.21.15.86:0/2319253154 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560bf8362c00 con 0x560bf8409950 2024-04-21T08:40:30.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.365+0000 7f7969ffb640 10 client.16608 handle_command_reply: tid=0 2024-04-21T08:40:30.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.365+0000 7f798af81740 1 client.16608 shutdown 2024-04-21T08:40:30.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.365+0000 7f798af81740 1 -- 172.21.15.86:0/2319253154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560bf83f9a80 msgr2=0x560bf83f7600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:30.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.365+0000 7f798af81740 1 --2- 172.21.15.86:0/2319253154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560bf83f9a80 0x560bf83f7600 secure :-1 s=READY pgs=815 cs=0 l=1 rev1=1 crypto rx=0x7f797c0345b0 tx=0x7f797c034440 comp rx=0 tx=0).stop 2024-04-21T08:40:30.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.365+0000 7f798af81740 1 -- 172.21.15.86:0/2319253154 shutdown_connections 2024-04-21T08:40:30.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.365+0000 7f798af81740 1 --2- 172.21.15.86:0/2319253154 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x560bf8409950 0x560bf840bd40 crc :-1 s=READY pgs=87 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.365+0000 7f798af81740 1 --2- 172.21.15.86:0/2319253154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560bf83f9a80 0x560bf83f7600 unknown :-1 s=CLOSED pgs=815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.365+0000 7f798af81740 1 --2- 172.21.15.86:0/2319253154 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560bf83f0ea0 0x560bf83f70c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.365+0000 7f7969ffb640 0 client.16608 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:40:30.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.365+0000 7f798af81740 1 --2- 172.21.15.86:0/2319253154 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560bf83f0580 0x560bf83f6b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.365+0000 7f798af81740 1 -- 172.21.15.86:0/2319253154 >> 172.21.15.86:0/2319253154 conn(0x560bf83eacb0 msgr2=0x560bf83eca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:30.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.365+0000 7f798af81740 1 -- 172.21.15.86:0/2319253154 shutdown_connections 2024-04-21T08:40:30.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.365+0000 7f798af81740 1 -- 172.21.15.86:0/2319253154 wait complete. 2024-04-21T08:40:30.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.365+0000 7f798af81740 20 client.16608 trim_cache size 0 max 16384 2024-04-21T08:40:30.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.382+0000 7f798af81740 1 Processor -- start 2024-04-21T08:40:30.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.382+0000 7f798af81740 1 -- start start 2024-04-21T08:40:30.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.382+0000 7f798af81740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560bf83f0580 0x560bf83f92b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:30.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.382+0000 7f798af81740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560bf83f0ea0 0x560bf8405730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:30.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.382+0000 7f798af81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560bf83f9a80 0x560bf84112e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:30.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.382+0000 7f798af81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f79801af2a0 con 0x560bf83f9a80 2024-04-21T08:40:30.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.382+0000 7f798af81740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f79801b0de0 con 0x560bf83f0ea0 2024-04-21T08:40:30.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.382+0000 7f798af81740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f79801af970 con 0x560bf83f0580 2024-04-21T08:40:30.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.382+0000 7f798685d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560bf83f9a80 0x560bf84112e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:30.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.382+0000 7f798685d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560bf83f9a80 0x560bf84112e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54976/0 (socket says 172.21.15.86:54976) 2024-04-21T08:40:30.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.382+0000 7f798685d640 1 -- 172.21.15.86:0/2461843220 learned_addr learned my addr 172.21.15.86:0/2461843220 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:30.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.383+0000 7f798605c640 1 --2- 172.21.15.86:0/2461843220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560bf83f0580 0x560bf83f92b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:30.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.383+0000 7f798585b640 1 --2- 172.21.15.86:0/2461843220 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560bf83f0ea0 0x560bf8405730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:30.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.383+0000 7f798685d640 1 -- 172.21.15.86:0/2461843220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560bf83f0580 msgr2=0x560bf83f92b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:30.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.383+0000 7f798685d640 1 --2- 172.21.15.86:0/2461843220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560bf83f0580 0x560bf83f92b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.383+0000 7f798685d640 1 -- 172.21.15.86:0/2461843220 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560bf83f0ea0 msgr2=0x560bf8405730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:30.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.383+0000 7f798685d640 1 --2- 172.21.15.86:0/2461843220 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560bf83f0ea0 0x560bf8405730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.383+0000 7f798685d640 1 -- 172.21.15.86:0/2461843220 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560bf83ea7e0 con 0x560bf83f9a80 2024-04-21T08:40:30.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.383+0000 7f798585b640 1 --2- 172.21.15.86:0/2461843220 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560bf83f0ea0 0x560bf8405730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:40:30.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.383+0000 7f798685d640 1 --2- 172.21.15.86:0/2461843220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560bf83f9a80 0x560bf84112e0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f797c006010 tx=0x7f797c004510 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b486f0cf3c62b95 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:30.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.383+0000 7f796bfff640 1 -- 172.21.15.86:0/2461843220 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797c042470 con 0x560bf83f9a80 2024-04-21T08:40:30.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.383+0000 7f796bfff640 1 -- 172.21.15.86:0/2461843220 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f797c003b30 con 0x560bf83f9a80 2024-04-21T08:40:30.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.383+0000 7f796bfff640 1 -- 172.21.15.86:0/2461843220 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797c0035f0 con 0x560bf83f9a80 2024-04-21T08:40:30.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.383+0000 7f798af81740 1 -- 172.21.15.86:0/2461843220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560bf83f9a80 msgr2=0x560bf84112e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:30.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.383+0000 7f798af81740 1 --2- 172.21.15.86:0/2461843220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560bf83f9a80 0x560bf84112e0 secure :-1 s=READY pgs=816 cs=0 l=1 rev1=1 crypto rx=0x7f797c006010 tx=0x7f797c004510 comp rx=0 tx=0).stop 2024-04-21T08:40:30.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.383+0000 7f798af81740 1 -- 172.21.15.86:0/2461843220 shutdown_connections 2024-04-21T08:40:30.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.383+0000 7f798af81740 1 --2- 172.21.15.86:0/2461843220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560bf83f9a80 0x560bf84112e0 unknown :-1 s=CLOSED pgs=816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.383+0000 7f798af81740 1 --2- 172.21.15.86:0/2461843220 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560bf83f0ea0 0x560bf8405730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.383+0000 7f798af81740 1 --2- 172.21.15.86:0/2461843220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560bf83f0580 0x560bf83f92b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.383+0000 7f798af81740 1 -- 172.21.15.86:0/2461843220 >> 172.21.15.86:0/2461843220 conn(0x560bf83eacb0 msgr2=0x560bf83f5450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:30.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.384+0000 7f798af81740 1 -- 172.21.15.86:0/2461843220 shutdown_connections 2024-04-21T08:40:30.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.384+0000 7f798af81740 1 -- 172.21.15.86:0/2461843220 wait complete. 2024-04-21T08:40:30.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.384+0000 7f798af81740 1 -- start start 2024-04-21T08:40:30.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.384+0000 7f798af81740 1 Processor -- start 2024-04-21T08:40:30.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.384+0000 7f798af81740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560bf83f0580 0x560bf8467e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:30.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.384+0000 7f798af81740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560bf83f0ea0 0x560bf8468340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:30.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.384+0000 7f798af81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560bf83f9a80 0x560bf83fdfb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:30.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.384+0000 7f798af81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f79801b0710 con 0x560bf83f9a80 2024-04-21T08:40:30.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.384+0000 7f798af81740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f79801b0040 con 0x560bf83f0ea0 2024-04-21T08:40:30.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.384+0000 7f798af81740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f79801ad260 con 0x560bf83f0580 2024-04-21T08:40:30.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.384+0000 7f798af81740 10 client.0 _ll_register_callbacks cb 0x7f7970005e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:30.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.384+0000 7f798685d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560bf83f9a80 0x560bf83fdfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:30.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.384+0000 7f798685d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560bf83f9a80 0x560bf83fdfb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54978/0 (socket says 172.21.15.86:54978) 2024-04-21T08:40:30.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.384+0000 7f798685d640 1 -- 172.21.15.86:0/375685307 learned_addr learned my addr 172.21.15.86:0/375685307 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:30.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.384+0000 7f798605c640 1 --2- 172.21.15.86:0/375685307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560bf83f0580 0x560bf8467e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:30.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.384+0000 7f798585b640 1 --2- 172.21.15.86:0/375685307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560bf83f0ea0 0x560bf8468340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:30.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.384+0000 7f798685d640 1 -- 172.21.15.86:0/375685307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560bf83f0580 msgr2=0x560bf8467e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:30.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.384+0000 7f798685d640 1 --2- 172.21.15.86:0/375685307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560bf83f0580 0x560bf8467e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.384+0000 7f798685d640 1 -- 172.21.15.86:0/375685307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560bf83f0ea0 msgr2=0x560bf8468340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:30.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.384+0000 7f798685d640 1 --2- 172.21.15.86:0/375685307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560bf83f0ea0 0x560bf8468340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.384+0000 7f798685d640 1 -- 172.21.15.86:0/375685307 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560bf83fe6a0 con 0x560bf83f9a80 2024-04-21T08:40:30.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.384+0000 7f798585b640 1 --2- 172.21.15.86:0/375685307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560bf83f0ea0 0x560bf8468340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:30.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.385+0000 7f798685d640 1 --2- 172.21.15.86:0/375685307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560bf83f9a80 0x560bf83fdfb0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f797c036040 tx=0x7f797c009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=498dfafc6a7b1489 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:30.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.385+0000 7f7969ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:40:30.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.385+0000 7f7969ffb640 1 -- 172.21.15.86:0/375685307 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f797c034570 con 0x560bf83f9a80 2024-04-21T08:40:30.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.385+0000 7f7969ffb640 1 -- 172.21.15.86:0/375685307 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f797c034710 con 0x560bf83f9a80 2024-04-21T08:40:30.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.385+0000 7f798af81740 1 -- 172.21.15.86:0/375685307 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f797c003b30 con 0x560bf83f9a80 2024-04-21T08:40:30.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.385+0000 7f7969ffb640 1 -- 172.21.15.86:0/375685307 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f797c034950 con 0x560bf83f9a80 2024-04-21T08:40:30.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.385+0000 7f798af81740 10 client.16614 fetch_fsmap learned FSMap version 29 2024-04-21T08:40:30.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.385+0000 7f798af81740 1 -- 172.21.15.86:0/375685307 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560bf84705f0 con 0x560bf83f9a80 2024-04-21T08:40:30.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.385+0000 7f7969ffb640 1 -- 172.21.15.86:0/375685307 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f797c003b30 con 0x560bf83f9a80 2024-04-21T08:40:30.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.385+0000 7f7969ffb640 1 -- 172.21.15.86:0/375685307 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f797c055a20 con 0x560bf83f9a80 2024-04-21T08:40:30.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.385+0000 7f798af81740 10 client.16614 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:40:30.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.385+0000 7f798af81740 10 client.16614 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:30.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.385+0000 7f798af81740 20 client.16614 populate_metadata read hostname 'smithi086' 2024-04-21T08:40:30.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.385+0000 7f798af81740 1 --2- 172.21.15.86:0/375685307 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x560bf8470a20 0x560bf8472e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:30.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.385+0000 7f798af81740 4 client.16614 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:40:30.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.385+0000 7f798af81740 1 -- 172.21.15.86:0/375685307 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560bf835ca70 con 0x560bf8470a20 2024-04-21T08:40:30.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.386+0000 7f798605c640 1 --2- 172.21.15.86:0/375685307 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x560bf8470a20 0x560bf8472e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:30.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.386+0000 7f798605c640 1 --2- 172.21.15.86:0/375685307 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x560bf8470a20 0x560bf8472e10 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b76f26a6e7048c3 server_cookie=e12f0e322be16992 in_seq=0 out_seq=0 2024-04-21T08:40:30.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.386+0000 7f7969ffb640 10 client.16614 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:40:30.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.386+0000 7f7969ffb640 1 -- 172.21.15.86:0/375685307 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x560bf835ca70 con 0x560bf8470a20 2024-04-21T08:40:30.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.386+0000 7f7969ffb640 10 client.16614 handle_command_reply: tid=0 2024-04-21T08:40:30.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.386+0000 7f798af81740 1 client.16614 shutdown 2024-04-21T08:40:30.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.386+0000 7f798af81740 1 -- 172.21.15.86:0/375685307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560bf83f9a80 msgr2=0x560bf83fdfb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:30.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.386+0000 7f798af81740 1 --2- 172.21.15.86:0/375685307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560bf83f9a80 0x560bf83fdfb0 secure :-1 s=READY pgs=817 cs=0 l=1 rev1=1 crypto rx=0x7f797c036040 tx=0x7f797c009800 comp rx=0 tx=0).stop 2024-04-21T08:40:30.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.386+0000 7f798af81740 1 -- 172.21.15.86:0/375685307 shutdown_connections 2024-04-21T08:40:30.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.386+0000 7f798af81740 1 --2- 172.21.15.86:0/375685307 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x560bf8470a20 0x560bf8472e10 crc :-1 s=READY pgs=88 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.386+0000 7f798af81740 1 --2- 172.21.15.86:0/375685307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560bf83f9a80 0x560bf83fdfb0 unknown :-1 s=CLOSED pgs=817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.386+0000 7f798af81740 1 --2- 172.21.15.86:0/375685307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560bf83f0ea0 0x560bf8468340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.386+0000 7f7969ffb640 0 client.16614 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:40:30.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.386+0000 7f798af81740 1 --2- 172.21.15.86:0/375685307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560bf83f0580 0x560bf8467e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.386+0000 7f798af81740 1 -- 172.21.15.86:0/375685307 >> 172.21.15.86:0/375685307 conn(0x560bf83eacb0 msgr2=0x560bf83ec000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:30.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.386+0000 7f798af81740 1 -- 172.21.15.86:0/375685307 shutdown_connections 2024-04-21T08:40:30.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.386+0000 7f798af81740 1 -- 172.21.15.86:0/375685307 wait complete. 2024-04-21T08:40:30.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.386+0000 7f798af81740 20 client.16614 trim_cache size 0 max 16384 2024-04-21T08:40:30.397 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:40:30.397 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:40:30.397 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-21T08:40:30.397 INFO:teuthology.orchestra.run.smithi086.stdout: "f6024a57-724d-4ca4-8145-a026dcc1395e": { 2024-04-21T08:40:30.397 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-21T08:40:30.397 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "f6024a57-724d-4ca4-8145-a026dcc1395e", 2024-04-21T08:40:30.397 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-21T08:40:30.397 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:40:30.398 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:40:30.398 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:40:30.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.387+0000 7f79882c0640 1 -- 172.21.15.86:0/811869505 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f796c0a3d20 msgr2=0x7f796c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:30.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.387+0000 7f79882c0640 1 --2- 172.21.15.86:0/811869505 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f796c0a3d20 0x7f796c0a61e0 secure :-1 s=READY pgs=116 cs=0 l=1 rev1=1 crypto rx=0x7f797c04b000 tx=0x7f797c047000 comp rx=0 tx=0).stop 2024-04-21T08:40:30.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.387+0000 7f79882c0640 1 -- 172.21.15.86:0/811869505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7980101080 msgr2=0x7f79801986d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:30.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.387+0000 7f79882c0640 1 --2- 172.21.15.86:0/811869505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7980101080 0x7f79801986d0 secure :-1 s=READY pgs=813 cs=0 l=1 rev1=1 crypto rx=0x7f797800cce0 tx=0x7f797800a620 comp rx=0 tx=0).stop 2024-04-21T08:40:30.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.387+0000 7f79882c0640 1 -- 172.21.15.86:0/811869505 shutdown_connections 2024-04-21T08:40:30.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.387+0000 7f79882c0640 1 --2- 172.21.15.86:0/811869505 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f796c0a3d20 0x7f796c0a61e0 unknown :-1 s=CLOSED pgs=116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.387+0000 7f79882c0640 1 --2- 172.21.15.86:0/811869505 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f798019cf90 0x7f798019d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.387+0000 7f79882c0640 1 --2- 172.21.15.86:0/811869505 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7980106150 0x7f7980198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.387+0000 7f79882c0640 1 --2- 172.21.15.86:0/811869505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7980101080 0x7f79801986d0 unknown :-1 s=CLOSED pgs=813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:30.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.387+0000 7f79882c0640 1 -- 172.21.15.86:0/811869505 >> 172.21.15.86:0/811869505 conn(0x7f7980078000 msgr2=0x7f79800fef90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:30.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.387+0000 7f79882c0640 1 -- 172.21.15.86:0/811869505 shutdown_connections 2024-04-21T08:40:30.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:40:30.387+0000 7f79882c0640 1 -- 172.21.15.86:0/811869505 wait complete. 2024-04-21T08:40:30.400 INFO:tasks.cephfs.filesystem:scrub status for tag:f6024a57-724d-4ca4-8145-a026dcc1395e - {'path': '/', 'tag': 'f6024a57-724d-4ca4-8145-a026dcc1395e', 'options': 'recursive,force'} 2024-04-21T08:40:30.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:30 smithi179 ceph-mon[31829]: pgmap v54: 129 pgs: 129 active+clean; 2.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 26 op/s 2024-04-21T08:40:30.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:30 smithi086 ceph-mon[27939]: pgmap v54: 129 pgs: 129 active+clean; 2.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 26 op/s 2024-04-21T08:40:30.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:30 smithi088 ceph-mon[31259]: pgmap v54: 129 pgs: 129 active+clean; 2.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 33 KiB/s wr, 26 op/s 2024-04-21T08:40:31.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:31 smithi179 ceph-mon[31829]: scrub queued for path: / 2024-04-21T08:40:31.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:31 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:40:31.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:31 smithi179 ceph-mon[31829]: scrub summary: active paths [/] 2024-04-21T08:40:31.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:31 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:40:31.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:31 smithi086 ceph-mon[27939]: scrub queued for path: / 2024-04-21T08:40:31.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:31 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:40:31.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:31 smithi086 ceph-mon[27939]: scrub summary: active paths [/] 2024-04-21T08:40:31.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:31 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:40:31.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:31 smithi088 ceph-mon[31259]: scrub queued for path: / 2024-04-21T08:40:31.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:31 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:40:31.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:31 smithi088 ceph-mon[31259]: scrub summary: active paths [/] 2024-04-21T08:40:31.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:31 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:40:32.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:32 smithi086 ceph-mon[27939]: pgmap v55: 129 pgs: 129 active+clean; 2.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-04-21T08:40:32.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:32 smithi088 ceph-mon[31259]: pgmap v55: 129 pgs: 129 active+clean; 2.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-04-21T08:40:32.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:32 smithi179 ceph-mon[31829]: pgmap v55: 129 pgs: 129 active+clean; 2.1 MiB data, 388 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 16 op/s 2024-04-21T08:40:34.765 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:34 smithi088 ceph-mon[31259]: pgmap v56: 129 pgs: 129 active+clean; 2.2 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 23 op/s 2024-04-21T08:40:34.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:34 smithi086 ceph-mon[27939]: pgmap v56: 129 pgs: 129 active+clean; 2.2 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 23 op/s 2024-04-21T08:40:34.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:34 smithi179 ceph-mon[31829]: pgmap v56: 129 pgs: 129 active+clean; 2.2 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 29 KiB/s wr, 23 op/s 2024-04-21T08:40:35.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:35 smithi086 ceph-mon[27939]: scrub summary: idle 2024-04-21T08:40:35.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:35 smithi088 ceph-mon[31259]: scrub summary: idle 2024-04-21T08:40:35.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:35 smithi179 ceph-mon[31829]: scrub summary: idle 2024-04-21T08:40:36.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:36 smithi086 ceph-mon[27939]: pgmap v57: 129 pgs: 129 active+clean; 2.2 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-21T08:40:36.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:36 smithi088 ceph-mon[31259]: pgmap v57: 129 pgs: 129 active+clean; 2.2 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-21T08:40:36.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:36 smithi179 ceph-mon[31829]: pgmap v57: 129 pgs: 129 active+clean; 2.2 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-21T08:40:38.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:38 smithi086 ceph-mon[27939]: pgmap v58: 129 pgs: 129 active+clean; 2.2 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-21T08:40:38.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:38 smithi088 ceph-mon[31259]: pgmap v58: 129 pgs: 129 active+clean; 2.2 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-21T08:40:38.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:38 smithi179 ceph-mon[31829]: pgmap v58: 129 pgs: 129 active+clean; 2.2 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 20 KiB/s wr, 16 op/s 2024-04-21T08:40:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.038+0000 7f919f9a4640 1 -- 172.21.15.86:0/3902619248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9198106170 msgr2=0x7f919810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.038+0000 7f919f9a4640 1 --2- 172.21.15.86:0/3902619248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9198106170 0x7f919810dd40 secure :-1 s=READY pgs=818 cs=0 l=1 rev1=1 crypto rx=0x7f9190009ca0 tx=0x7f919002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:40:39.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.038+0000 7f919f9a4640 1 -- 172.21.15.86:0/3902619248 shutdown_connections 2024-04-21T08:40:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.038+0000 7f919f9a4640 1 --2- 172.21.15.86:0/3902619248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9198106170 0x7f919810dd40 unknown :-1 s=CLOSED pgs=818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.038+0000 7f919f9a4640 1 --2- 172.21.15.86:0/3902619248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9198101aa0 0x7f9198105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.038+0000 7f919f9a4640 1 --2- 172.21.15.86:0/3902619248 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91981010f0 0x7f91981014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.038+0000 7f919f9a4640 1 -- 172.21.15.86:0/3902619248 >> 172.21.15.86:0/3902619248 conn(0x7f9198077fd0 msgr2=0x7f91980783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.038+0000 7f919f9a4640 1 -- 172.21.15.86:0/3902619248 shutdown_connections 2024-04-21T08:40:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.038+0000 7f919f9a4640 1 -- 172.21.15.86:0/3902619248 wait complete. 2024-04-21T08:40:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.039+0000 7f919f9a4640 1 Processor -- start 2024-04-21T08:40:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.039+0000 7f919f9a4640 1 -- start start 2024-04-21T08:40:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.039+0000 7f919f9a4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91981010f0 0x7f91981986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.039+0000 7f919f9a4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9198106170 0x7f9198198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.039+0000 7f919f9a4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f919819cf90 0x7f919819d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.039+0000 7f919f9a4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f91981121f0 con 0x7f9198106170 2024-04-21T08:40:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.039+0000 7f919f9a4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9198112070 con 0x7f91981010f0 2024-04-21T08:40:39.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.039+0000 7f919f9a4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9198112370 con 0x7f919819cf90 2024-04-21T08:40:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.039+0000 7f919cf3f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9198106170 0x7f9198198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.039+0000 7f919cf3f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9198106170 0x7f9198198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54988/0 (socket says 172.21.15.86:54988) 2024-04-21T08:40:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.039+0000 7f919cf3f640 1 -- 172.21.15.86:0/2148216000 learned_addr learned my addr 172.21.15.86:0/2148216000 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.039+0000 7f919d740640 1 --2- 172.21.15.86:0/2148216000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91981010f0 0x7f91981986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.039+0000 7f919df41640 1 --2- 172.21.15.86:0/2148216000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f919819cf90 0x7f919819d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.039+0000 7f919cf3f640 1 -- 172.21.15.86:0/2148216000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f919819cf90 msgr2=0x7f919819d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.039+0000 7f919cf3f640 1 --2- 172.21.15.86:0/2148216000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f919819cf90 0x7f919819d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.039+0000 7f919cf3f640 1 -- 172.21.15.86:0/2148216000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91981010f0 msgr2=0x7f91981986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.039+0000 7f919cf3f640 1 --2- 172.21.15.86:0/2148216000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91981010f0 0x7f91981986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.039+0000 7f919cf3f640 1 -- 172.21.15.86:0/2148216000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f919819da80 con 0x7f9198106170 2024-04-21T08:40:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.039+0000 7f919d740640 1 --2- 172.21.15.86:0/2148216000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91981010f0 0x7f91981986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.040+0000 7f919cf3f640 1 --2- 172.21.15.86:0/2148216000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9198106170 0x7f9198198c10 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f919400d770 tx=0x7f919400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.040+0000 7f918a7fc640 1 -- 172.21.15.86:0/2148216000 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9194017070 con 0x7f9198106170 2024-04-21T08:40:39.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.040+0000 7f918a7fc640 1 -- 172.21.15.86:0/2148216000 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9194009c30 con 0x7f9198106170 2024-04-21T08:40:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.040+0000 7f919f9a4640 1 -- 172.21.15.86:0/2148216000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f919819dd70 con 0x7f9198106170 2024-04-21T08:40:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.040+0000 7f918a7fc640 1 -- 172.21.15.86:0/2148216000 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f919400f620 con 0x7f9198106170 2024-04-21T08:40:39.043 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.040+0000 7f919f9a4640 1 -- 172.21.15.86:0/2148216000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f91981a55b0 con 0x7f9198106170 2024-04-21T08:40:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.042+0000 7f918a7fc640 1 -- 172.21.15.86:0/2148216000 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f919401a070 con 0x7f9198106170 2024-04-21T08:40:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.043+0000 7f91a259a740 1 Processor -- start 2024-04-21T08:40:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.043+0000 7f91a259a740 1 -- start start 2024-04-21T08:40:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.043+0000 7f91a259a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5021d58d0 0x55f5021d4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.043+0000 7f91a259a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f5021d4b60 0x55f5021d4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.043+0000 7f918a7fc640 1 --2- 172.21.15.86:0/2148216000 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f91840a3e40 0x7f91840a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.043+0000 7f91a259a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5021de0e0 0x55f5021e04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.043+0000 7f91a259a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f5021398d0 con 0x55f5021d58d0 2024-04-21T08:40:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.043+0000 7f91a259a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55f5020d6e80 con 0x55f5021d4b60 2024-04-21T08:40:39.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.043+0000 7f91a259a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f502135890 con 0x55f5021de0e0 2024-04-21T08:40:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.043+0000 7f919df41640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5021d58d0 0x55f5021d4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.043+0000 7f919df41640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5021d58d0 0x55f5021d4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54990/0 (socket says 172.21.15.86:54990) 2024-04-21T08:40:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.043+0000 7f919d740640 1 --2- 172.21.15.86:0/2148216000 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f91840a3e40 0x7f91840a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.043+0000 7f919df41640 1 -- 172.21.15.86:0/1862816843 learned_addr learned my addr 172.21.15.86:0/1862816843 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.043+0000 7f918a7fc640 1 -- 172.21.15.86:0/2148216000 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f91940d0620 con 0x7f9198106170 2024-04-21T08:40:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.043+0000 7f919cf3f640 1 --2- 172.21.15.86:0/1862816843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f5021d4b60 0x55f5021d4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.043+0000 7f919df41640 1 --2- 172.21.15.86:0/1862816843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5021de0e0 0x55f5021e04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.043+0000 7f919df41640 1 -- 172.21.15.86:0/1862816843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5021de0e0 msgr2=0x55f5021e04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.043+0000 7f919df41640 1 --2- 172.21.15.86:0/1862816843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5021de0e0 0x55f5021e04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.043+0000 7f919df41640 1 -- 172.21.15.86:0/1862816843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f5021d4b60 msgr2=0x55f5021d4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.043+0000 7f919df41640 1 --2- 172.21.15.86:0/1862816843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f5021d4b60 0x55f5021d4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.044+0000 7f919df41640 1 -- 172.21.15.86:0/1862816843 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5020f8320 con 0x55f5021d58d0 2024-04-21T08:40:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.044+0000 7f919d740640 1 --2- 172.21.15.86:0/2148216000 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f91840a3e40 0x7f91840a6300 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f918c0058c0 tx=0x7f918c005850 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:39.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.044+0000 7f919df41640 1 --2- 172.21.15.86:0/1862816843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5021d58d0 0x55f5021d4620 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f9190004af0 tx=0x7f9190004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b41909b4df36b00 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.044+0000 7f91837fe640 1 -- 172.21.15.86:0/1862816843 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9190039040 con 0x55f5021d58d0 2024-04-21T08:40:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.044+0000 7f91837fe640 1 -- 172.21.15.86:0/1862816843 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9190032970 con 0x55f5021d58d0 2024-04-21T08:40:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.044+0000 7f91837fe640 1 -- 172.21.15.86:0/1862816843 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9190004230 con 0x55f5021d58d0 2024-04-21T08:40:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.045+0000 7f91a259a740 1 -- 172.21.15.86:0/1862816843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5021d58d0 msgr2=0x55f5021d4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.045+0000 7f91a259a740 1 --2- 172.21.15.86:0/1862816843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5021d58d0 0x55f5021d4620 secure :-1 s=READY pgs=820 cs=0 l=1 rev1=1 crypto rx=0x7f9190004af0 tx=0x7f9190004a40 comp rx=0 tx=0).stop 2024-04-21T08:40:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.045+0000 7f91a259a740 1 -- 172.21.15.86:0/1862816843 shutdown_connections 2024-04-21T08:40:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.045+0000 7f91a259a740 1 --2- 172.21.15.86:0/1862816843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5021de0e0 0x55f5021e04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.045+0000 7f91a259a740 1 --2- 172.21.15.86:0/1862816843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f5021d4b60 0x55f5021d4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.045+0000 7f91a259a740 1 --2- 172.21.15.86:0/1862816843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5021d58d0 0x55f5021d4620 unknown :-1 s=CLOSED pgs=820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.045+0000 7f91a259a740 1 -- 172.21.15.86:0/1862816843 >> 172.21.15.86:0/1862816843 conn(0x55f50219bc90 msgr2=0x55f5021cf850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.045+0000 7f91a259a740 1 -- 172.21.15.86:0/1862816843 shutdown_connections 2024-04-21T08:40:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.045+0000 7f91a259a740 1 -- 172.21.15.86:0/1862816843 wait complete. 2024-04-21T08:40:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.045+0000 7f91a259a740 1 -- start start 2024-04-21T08:40:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.045+0000 7f91a259a740 1 Processor -- start 2024-04-21T08:40:39.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.046+0000 7f91a259a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5021d4b60 0x55f5021dd9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.046+0000 7f91a259a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5021d58d0 0x55f5021dadc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.046+0000 7f91a259a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f5021de0e0 0x55f5021db300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.046+0000 7f91a259a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f5021e0db0 con 0x55f5021d58d0 2024-04-21T08:40:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.046+0000 7f91a259a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55f5020c1eb0 con 0x55f5021de0e0 2024-04-21T08:40:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.046+0000 7f91a259a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f5021e0f30 con 0x55f5021d4b60 2024-04-21T08:40:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.046+0000 7f91a259a740 10 client.0 _ll_register_callbacks cb 0x55f5020eeb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.046+0000 7f919cf3f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5021d58d0 0x55f5021dadc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.046+0000 7f919cf3f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5021d58d0 0x55f5021dadc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54994/0 (socket says 172.21.15.86:54994) 2024-04-21T08:40:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.046+0000 7f919cf3f640 1 -- 172.21.15.86:0/717558009 learned_addr learned my addr 172.21.15.86:0/717558009 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.046+0000 7f919df41640 1 --2- 172.21.15.86:0/717558009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f5021de0e0 0x55f5021db300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.046+0000 7f919d740640 1 --2- 172.21.15.86:0/717558009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5021d4b60 0x55f5021dd9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.046+0000 7f919cf3f640 1 -- 172.21.15.86:0/717558009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5021d4b60 msgr2=0x55f5021dd9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:39.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.046+0000 7f919cf3f640 1 --2- 172.21.15.86:0/717558009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5021d4b60 0x55f5021dd9c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.046+0000 7f919cf3f640 1 -- 172.21.15.86:0/717558009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f5021de0e0 msgr2=0x55f5021db300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.046+0000 7f919cf3f640 1 --2- 172.21.15.86:0/717558009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f5021de0e0 0x55f5021db300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.046+0000 7f919cf3f640 1 -- 172.21.15.86:0/717558009 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f501e4c210 con 0x55f5021d58d0 2024-04-21T08:40:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.046+0000 7f919df41640 1 --2- 172.21.15.86:0/717558009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f5021de0e0 0x55f5021db300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.046+0000 7f919cf3f640 1 --2- 172.21.15.86:0/717558009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5021d58d0 0x55f5021dadc0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f91940cda60 tx=0x7f9194009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17363c58768f148b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.046+0000 7f9180ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:40:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.046+0000 7f9180ff9640 1 -- 172.21.15.86:0/717558009 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9194021040 con 0x55f5021d58d0 2024-04-21T08:40:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.046+0000 7f9180ff9640 1 -- 172.21.15.86:0/717558009 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f91940cdc80 con 0x55f5021d58d0 2024-04-21T08:40:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.046+0000 7f91a259a740 1 -- 172.21.15.86:0/717558009 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9190032970 con 0x55f5021d58d0 2024-04-21T08:40:39.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.046+0000 7f9180ff9640 1 -- 172.21.15.86:0/717558009 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f91940cde80 con 0x55f5021d58d0 2024-04-21T08:40:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.046+0000 7f91a259a740 10 client.16626 fetch_fsmap learned FSMap version 29 2024-04-21T08:40:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.046+0000 7f91a259a740 1 -- 172.21.15.86:0/717558009 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f501f18620 con 0x55f5021d58d0 2024-04-21T08:40:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.047+0000 7f9180ff9640 1 -- 172.21.15.86:0/717558009 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9190032970 con 0x55f5021d58d0 2024-04-21T08:40:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.047+0000 7f9180ff9640 1 -- 172.21.15.86:0/717558009 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f91940d0270 con 0x55f5021d58d0 2024-04-21T08:40:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.047+0000 7f91a259a740 10 client.16626 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:40:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.047+0000 7f91a259a740 10 client.16626 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.047+0000 7f91a259a740 20 client.16626 populate_metadata read hostname 'smithi086' 2024-04-21T08:40:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.047+0000 7f91a259a740 1 --2- 172.21.15.86:0/717558009 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f5021edfe0 0x55f5021f03f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.047+0000 7f91a259a740 4 client.16626 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:40:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.047+0000 7f91a259a740 1 -- 172.21.15.86:0/717558009 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f50214d880 con 0x55f5021edfe0 2024-04-21T08:40:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.047+0000 7f919d740640 1 --2- 172.21.15.86:0/717558009 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f5021edfe0 0x55f5021f03f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.048+0000 7f919d740640 1 --2- 172.21.15.86:0/717558009 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f5021edfe0 0x55f5021f03f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bf29418ade79e474 server_cookie=a243430cd2c37abe in_seq=0 out_seq=0 2024-04-21T08:40:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.048+0000 7f9180ff9640 10 client.16626 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:40:39.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.048+0000 7f9180ff9640 1 -- 172.21.15.86:0/717558009 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f50214d880 con 0x55f5021edfe0 2024-04-21T08:40:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.048+0000 7f9180ff9640 10 client.16626 handle_command_reply: tid=0 2024-04-21T08:40:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.048+0000 7f91a259a740 1 client.16626 shutdown 2024-04-21T08:40:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.048+0000 7f91a259a740 1 -- 172.21.15.86:0/717558009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5021d58d0 msgr2=0x55f5021dadc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.048+0000 7f91a259a740 1 --2- 172.21.15.86:0/717558009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5021d58d0 0x55f5021dadc0 secure :-1 s=READY pgs=821 cs=0 l=1 rev1=1 crypto rx=0x7f91940cda60 tx=0x7f9194009ea0 comp rx=0 tx=0).stop 2024-04-21T08:40:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.048+0000 7f91a259a740 1 -- 172.21.15.86:0/717558009 shutdown_connections 2024-04-21T08:40:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.048+0000 7f91a259a740 1 --2- 172.21.15.86:0/717558009 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f5021edfe0 0x55f5021f03f0 crc :-1 s=READY pgs=89 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.048+0000 7f91a259a740 1 --2- 172.21.15.86:0/717558009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f5021de0e0 0x55f5021db300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.048+0000 7f91a259a740 1 --2- 172.21.15.86:0/717558009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5021d58d0 0x55f5021dadc0 unknown :-1 s=CLOSED pgs=821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.048+0000 7f9180ff9640 0 client.16626 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:40:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.048+0000 7f91a259a740 1 --2- 172.21.15.86:0/717558009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5021d4b60 0x55f5021dd9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.048+0000 7f91a259a740 1 -- 172.21.15.86:0/717558009 >> 172.21.15.86:0/717558009 conn(0x55f50219bc90 msgr2=0x55f5021d15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.049+0000 7f91a259a740 1 -- 172.21.15.86:0/717558009 shutdown_connections 2024-04-21T08:40:39.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.049+0000 7f91a259a740 1 -- 172.21.15.86:0/717558009 wait complete. 2024-04-21T08:40:39.057 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.049+0000 7f91a259a740 20 client.16626 trim_cache size 0 max 16384 2024-04-21T08:40:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.063+0000 7f91a259a740 1 Processor -- start 2024-04-21T08:40:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.063+0000 7f91a259a740 1 -- start start 2024-04-21T08:40:39.065 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.064+0000 7f91a259a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5021d4b60 0x55f5021e9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.064+0000 7f91a259a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f5021d58d0 0x55f5021d4080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.064+0000 7f91a259a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5021de0e0 0x55f5021f5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.064+0000 7f91a259a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f91981af140 con 0x55f5021d4b60 2024-04-21T08:40:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.064+0000 7f91a259a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f91981b0c80 con 0x55f5021d58d0 2024-04-21T08:40:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.064+0000 7f91a259a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f91981af810 con 0x55f5021de0e0 2024-04-21T08:40:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.064+0000 7f919d740640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5021d4b60 0x55f5021e9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.064+0000 7f919d740640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5021d4b60 0x55f5021e9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54996/0 (socket says 172.21.15.86:54996) 2024-04-21T08:40:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.064+0000 7f919d740640 1 -- 172.21.15.86:0/1682818994 learned_addr learned my addr 172.21.15.86:0/1682818994 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.064+0000 7f919df41640 1 --2- 172.21.15.86:0/1682818994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5021de0e0 0x55f5021f5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:39.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.064+0000 7f919d740640 1 -- 172.21.15.86:0/1682818994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5021de0e0 msgr2=0x55f5021f5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.064+0000 7f919cf3f640 1 --2- 172.21.15.86:0/1682818994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f5021d58d0 0x55f5021d4080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.064+0000 7f919d740640 1 --2- 172.21.15.86:0/1682818994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5021de0e0 0x55f5021f5d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.064+0000 7f919d740640 1 -- 172.21.15.86:0/1682818994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f5021d58d0 msgr2=0x55f5021d4080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.064+0000 7f919d740640 1 --2- 172.21.15.86:0/1682818994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f5021d58d0 0x55f5021d4080 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.064+0000 7f919d740640 1 -- 172.21.15.86:0/1682818994 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f5021f0a20 con 0x55f5021d4b60 2024-04-21T08:40:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.064+0000 7f919d740640 1 --2- 172.21.15.86:0/1682818994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5021d4b60 0x55f5021e9cb0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x55f5021f0c20 tx=0x7f918c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4eedd9a807233a1f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.064+0000 7f9182ffd640 1 -- 172.21.15.86:0/1682818994 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c004020 con 0x55f5021d4b60 2024-04-21T08:40:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.064+0000 7f9182ffd640 1 -- 172.21.15.86:0/1682818994 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f918c011d40 con 0x55f5021d4b60 2024-04-21T08:40:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.064+0000 7f9182ffd640 1 -- 172.21.15.86:0/1682818994 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c0257a0 con 0x55f5021d4b60 2024-04-21T08:40:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.065+0000 7f91a259a740 1 -- 172.21.15.86:0/1682818994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5021d4b60 msgr2=0x55f5021e9cb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.065+0000 7f91a259a740 1 --2- 172.21.15.86:0/1682818994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5021d4b60 0x55f5021e9cb0 secure :-1 s=READY pgs=822 cs=0 l=1 rev1=1 crypto rx=0x55f5021f0c20 tx=0x7f918c015000 comp rx=0 tx=0).stop 2024-04-21T08:40:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.065+0000 7f91a259a740 1 -- 172.21.15.86:0/1682818994 shutdown_connections 2024-04-21T08:40:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.065+0000 7f91a259a740 1 --2- 172.21.15.86:0/1682818994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5021de0e0 0x55f5021f5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:39.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.065+0000 7f91a259a740 1 --2- 172.21.15.86:0/1682818994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f5021d58d0 0x55f5021d4080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.065+0000 7f91a259a740 1 --2- 172.21.15.86:0/1682818994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5021d4b60 0x55f5021e9cb0 unknown :-1 s=CLOSED pgs=822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.065+0000 7f91a259a740 1 -- 172.21.15.86:0/1682818994 >> 172.21.15.86:0/1682818994 conn(0x55f50219bc90 msgr2=0x55f5021d6030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.065+0000 7f91a259a740 1 -- 172.21.15.86:0/1682818994 shutdown_connections 2024-04-21T08:40:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.065+0000 7f91a259a740 1 -- 172.21.15.86:0/1682818994 wait complete. 2024-04-21T08:40:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.065+0000 7f91a259a740 1 -- start start 2024-04-21T08:40:39.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.065+0000 7f91a259a740 1 Processor -- start 2024-04-21T08:40:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.066+0000 7f91a259a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5021d4b60 0x55f5021d7b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.066+0000 7f91a259a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5021d58d0 0x55f5021d80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.066+0000 7f91a259a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f5021de0e0 0x55f5021d8610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.066+0000 7f91a259a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f91981b05b0 con 0x55f5021d4b60 2024-04-21T08:40:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.066+0000 7f91a259a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f91981afee0 con 0x55f5021de0e0 2024-04-21T08:40:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.066+0000 7f91a259a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f91981ad100 con 0x55f5021d58d0 2024-04-21T08:40:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.066+0000 7f91a259a740 10 client.0 _ll_register_callbacks cb 0x7f91940985e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.066+0000 7f919d740640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5021d4b60 0x55f5021d7b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.066+0000 7f919d740640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5021d4b60 0x55f5021d7b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54998/0 (socket says 172.21.15.86:54998) 2024-04-21T08:40:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.066+0000 7f919d740640 1 -- 172.21.15.86:0/2202262507 learned_addr learned my addr 172.21.15.86:0/2202262507 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:39.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.066+0000 7f919cf3f640 1 --2- 172.21.15.86:0/2202262507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5021d58d0 0x55f5021d80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.066+0000 7f919df41640 1 --2- 172.21.15.86:0/2202262507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f5021de0e0 0x55f5021d8610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.066+0000 7f919d740640 1 -- 172.21.15.86:0/2202262507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5021d58d0 msgr2=0x55f5021d80d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.066+0000 7f919d740640 1 --2- 172.21.15.86:0/2202262507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5021d58d0 0x55f5021d80d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.066+0000 7f919d740640 1 -- 172.21.15.86:0/2202262507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f5021de0e0 msgr2=0x55f5021d8610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.066+0000 7f919d740640 1 --2- 172.21.15.86:0/2202262507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f5021de0e0 0x55f5021d8610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.066+0000 7f919d740640 1 -- 172.21.15.86:0/2202262507 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f5021d3e30 con 0x55f5021d4b60 2024-04-21T08:40:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.066+0000 7f919d740640 1 --2- 172.21.15.86:0/2202262507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5021d4b60 0x55f5021d7b90 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f918c00b540 tx=0x7f918c00b570 comp rx=0 tx=0).ready entity=mon.0 client_cookie=787112782d24c41f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.066+0000 7f9180ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:40:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.067+0000 7f9180ff9640 1 -- 172.21.15.86:0/2202262507 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f918c034020 con 0x55f5021d4b60 2024-04-21T08:40:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.067+0000 7f9180ff9640 1 -- 172.21.15.86:0/2202262507 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f918c005240 con 0x55f5021d4b60 2024-04-21T08:40:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.067+0000 7f91a259a740 1 -- 172.21.15.86:0/2202262507 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f918c011d40 con 0x55f5021d4b60 2024-04-21T08:40:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.067+0000 7f9180ff9640 1 -- 172.21.15.86:0/2202262507 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f918c005440 con 0x55f5021d4b60 2024-04-21T08:40:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.067+0000 7f91a259a740 10 client.16632 fetch_fsmap learned FSMap version 29 2024-04-21T08:40:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.067+0000 7f91a259a740 1 -- 172.21.15.86:0/2202262507 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f502254790 con 0x55f5021d4b60 2024-04-21T08:40:39.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.067+0000 7f9180ff9640 1 -- 172.21.15.86:0/2202262507 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f918c011d40 con 0x55f5021d4b60 2024-04-21T08:40:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.067+0000 7f9180ff9640 1 -- 172.21.15.86:0/2202262507 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f918c004830 con 0x55f5021d4b60 2024-04-21T08:40:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.067+0000 7f91a259a740 10 client.16632 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:40:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.067+0000 7f91a259a740 10 client.16632 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.067+0000 7f91a259a740 20 client.16632 populate_metadata read hostname 'smithi086' 2024-04-21T08:40:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.067+0000 7f91a259a740 1 --2- 172.21.15.86:0/2202262507 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f502254ea0 0x55f5022572c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.067+0000 7f91a259a740 4 client.16632 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:40:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.067+0000 7f91a259a740 1 -- 172.21.15.86:0/2202262507 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f5021d4850 con 0x55f502254ea0 2024-04-21T08:40:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.067+0000 7f919cf3f640 1 --2- 172.21.15.86:0/2202262507 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f502254ea0 0x55f5022572c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.068+0000 7f9180ff9640 10 client.16632 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:40:39.073 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.068+0000 7f919cf3f640 1 --2- 172.21.15.86:0/2202262507 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f502254ea0 0x55f5022572c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f2adef8f27e13766 server_cookie=d779b88a92a1a50c in_seq=0 out_seq=0 2024-04-21T08:40:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.082+0000 7f9180ff9640 1 -- 172.21.15.86:0/2202262507 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f5021d4850 con 0x55f502254ea0 2024-04-21T08:40:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.082+0000 7f9180ff9640 10 client.16632 handle_command_reply: tid=0 2024-04-21T08:40:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.082+0000 7f91a259a740 1 client.16632 shutdown 2024-04-21T08:40:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.082+0000 7f91a259a740 1 -- 172.21.15.86:0/2202262507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5021d4b60 msgr2=0x55f5021d7b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.082+0000 7f91a259a740 1 --2- 172.21.15.86:0/2202262507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5021d4b60 0x55f5021d7b90 secure :-1 s=READY pgs=823 cs=0 l=1 rev1=1 crypto rx=0x7f918c00b540 tx=0x7f918c00b570 comp rx=0 tx=0).stop 2024-04-21T08:40:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.082+0000 7f91a259a740 1 -- 172.21.15.86:0/2202262507 shutdown_connections 2024-04-21T08:40:39.083 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.082+0000 7f91a259a740 1 --2- 172.21.15.86:0/2202262507 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f502254ea0 0x55f5022572c0 crc :-1 s=READY pgs=90 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.082+0000 7f91a259a740 1 --2- 172.21.15.86:0/2202262507 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f5021de0e0 0x55f5021d8610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.082+0000 7f9180ff9640 0 client.16632 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:40:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.082+0000 7f91a259a740 1 --2- 172.21.15.86:0/2202262507 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f5021d58d0 0x55f5021d80d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.082+0000 7f91a259a740 1 --2- 172.21.15.86:0/2202262507 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f5021d4b60 0x55f5021d7b90 unknown :-1 s=CLOSED pgs=823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.082+0000 7f91a259a740 1 -- 172.21.15.86:0/2202262507 >> 172.21.15.86:0/2202262507 conn(0x55f50219bc90 msgr2=0x55f5021d95b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.082+0000 7f91a259a740 1 -- 172.21.15.86:0/2202262507 shutdown_connections 2024-04-21T08:40:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.082+0000 7f91a259a740 1 -- 172.21.15.86:0/2202262507 wait complete. 2024-04-21T08:40:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.082+0000 7f91a259a740 20 client.16632 trim_cache size 0 max 16384 2024-04-21T08:40:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:40:39.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:40:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:40:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:40:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.083+0000 7f919f9a4640 1 -- 172.21.15.86:0/2148216000 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f91840a3e40 msgr2=0x7f91840a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.083+0000 7f919f9a4640 1 --2- 172.21.15.86:0/2148216000 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f91840a3e40 0x7f91840a6300 secure :-1 s=READY pgs=117 cs=0 l=1 rev1=1 crypto rx=0x7f918c0058c0 tx=0x7f918c005850 comp rx=0 tx=0).stop 2024-04-21T08:40:39.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.083+0000 7f919f9a4640 1 -- 172.21.15.86:0/2148216000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9198106170 msgr2=0x7f9198198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.083+0000 7f919f9a4640 1 --2- 172.21.15.86:0/2148216000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9198106170 0x7f9198198c10 secure :-1 s=READY pgs=819 cs=0 l=1 rev1=1 crypto rx=0x7f919400d770 tx=0x7f919400dc30 comp rx=0 tx=0).stop 2024-04-21T08:40:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.083+0000 7f919f9a4640 1 -- 172.21.15.86:0/2148216000 shutdown_connections 2024-04-21T08:40:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.083+0000 7f919f9a4640 1 --2- 172.21.15.86:0/2148216000 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f91840a3e40 0x7f91840a6300 unknown :-1 s=CLOSED pgs=117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.083+0000 7f919f9a4640 1 --2- 172.21.15.86:0/2148216000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f919819cf90 0x7f919819d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.083+0000 7f919f9a4640 1 --2- 172.21.15.86:0/2148216000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9198106170 0x7f9198198c10 unknown :-1 s=CLOSED pgs=819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.083+0000 7f919f9a4640 1 --2- 172.21.15.86:0/2148216000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f91981010f0 0x7f91981986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.083+0000 7f919f9a4640 1 -- 172.21.15.86:0/2148216000 >> 172.21.15.86:0/2148216000 conn(0x7f9198077fd0 msgr2=0x7f91980ff040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.083+0000 7f919f9a4640 1 -- 172.21.15.86:0/2148216000 shutdown_connections 2024-04-21T08:40:39.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:39.083+0000 7f919f9a4640 1 -- 172.21.15.86:0/2148216000 wait complete. 2024-04-21T08:40:40.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:40 smithi086 ceph-mon[27939]: pgmap v59: 129 pgs: 129 active+clean; 2.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 23 op/s 2024-04-21T08:40:40.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:40 smithi088 ceph-mon[31259]: pgmap v59: 129 pgs: 129 active+clean; 2.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 23 op/s 2024-04-21T08:40:40.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:40 smithi179 ceph-mon[31829]: pgmap v59: 129 pgs: 129 active+clean; 2.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 23 op/s 2024-04-21T08:40:42.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:42 smithi086 ceph-mon[27939]: pgmap v60: 129 pgs: 129 active+clean; 2.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-04-21T08:40:42.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:42 smithi088 ceph-mon[31259]: pgmap v60: 129 pgs: 129 active+clean; 2.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-04-21T08:40:42.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:42 smithi179 ceph-mon[31829]: pgmap v60: 129 pgs: 129 active+clean; 2.3 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 17 KiB/s wr, 14 op/s 2024-04-21T08:40:43.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:43 smithi086 ceph-mon[27939]: pgmap v61: 129 pgs: 129 active+clean; 2.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 23 op/s 2024-04-21T08:40:43.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:43 smithi088 ceph-mon[31259]: pgmap v61: 129 pgs: 129 active+clean; 2.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 23 op/s 2024-04-21T08:40:43.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:43 smithi179 ceph-mon[31829]: pgmap v61: 129 pgs: 129 active+clean; 2.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 23 op/s 2024-04-21T08:40:45.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:45 smithi179 ceph-mon[31829]: pgmap v62: 129 pgs: 129 active+clean; 2.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-21T08:40:46.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:45 smithi086 ceph-mon[27939]: pgmap v62: 129 pgs: 129 active+clean; 2.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-21T08:40:46.213 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:45 smithi088 ceph-mon[31259]: pgmap v62: 129 pgs: 129 active+clean; 2.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-21T08:40:47.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:47 smithi179 ceph-mon[31829]: pgmap v63: 129 pgs: 129 active+clean; 2.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-21T08:40:48.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:47 smithi086 ceph-mon[27939]: pgmap v63: 129 pgs: 129 active+clean; 2.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-21T08:40:48.213 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:47 smithi088 ceph-mon[31259]: pgmap v63: 129 pgs: 129 active+clean; 2.4 MiB data, 389 MiB used, 1.0 TiB / 1.0 TiB avail; 21 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-21T08:40:48.966 INFO:tasks.workunit.client.0.smithi086.stdout:1/600: open fd = 3 2024-04-21T08:40:48.966 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.966 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.966 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.966 INFO:tasks.workunit.client.0.smithi086.stdout:2/600: open fd = 3 2024-04-21T08:40:48.966 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.966 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.966 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.966 INFO:tasks.workunit.client.0.smithi086.stdout:3/600: open fd = 3 2024-04-21T08:40:48.966 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.967 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.967 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.967 INFO:tasks.workunit.client.0.smithi086.stdout:4/600: open fd = 3 2024-04-21T08:40:48.967 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.967 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.967 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.967 INFO:tasks.workunit.client.0.smithi086.stdout:5/600: open fd = 3 2024-04-21T08:40:48.967 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.967 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.967 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.967 INFO:tasks.workunit.client.0.smithi086.stdout:6/600: open fd = 3 2024-04-21T08:40:48.967 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.967 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.967 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.968 INFO:tasks.workunit.client.0.smithi086.stdout:7/600: open fd = 3 2024-04-21T08:40:48.968 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.968 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.968 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.968 INFO:tasks.workunit.client.0.smithi086.stdout:8/600: open fd = 3 2024-04-21T08:40:48.968 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.968 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.968 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.968 INFO:tasks.workunit.client.0.smithi086.stdout:9/600: open fd = 3 2024-04-21T08:40:48.968 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.968 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.968 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.968 INFO:tasks.workunit.client.0.smithi086.stdout:10/600: open fd = 3 2024-04-21T08:40:48.968 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.968 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.969 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.969 INFO:tasks.workunit.client.0.smithi086.stdout:11/600: open fd = 3 2024-04-21T08:40:48.969 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.969 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.969 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.969 INFO:tasks.workunit.client.0.smithi086.stdout:12/600: open fd = 3 2024-04-21T08:40:48.969 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.969 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.969 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.969 INFO:tasks.workunit.client.0.smithi086.stdout:13/600: open fd = 3 2024-04-21T08:40:48.969 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.969 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.969 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.969 INFO:tasks.workunit.client.0.smithi086.stdout:14/600: open fd = 3 2024-04-21T08:40:48.969 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.970 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.970 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.970 INFO:tasks.workunit.client.0.smithi086.stdout:15/600: open fd = 3 2024-04-21T08:40:48.970 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.970 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.970 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.970 INFO:tasks.workunit.client.0.smithi086.stdout:16/600: open fd = 3 2024-04-21T08:40:48.970 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.970 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.970 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.970 INFO:tasks.workunit.client.0.smithi086.stdout:17/600: open fd = 3 2024-04-21T08:40:48.970 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.970 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.970 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.970 INFO:tasks.workunit.client.0.smithi086.stdout:18/600: open fd = 3 2024-04-21T08:40:48.971 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.971 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.971 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.971 INFO:tasks.workunit.client.0.smithi086.stdout:19/600: open fd = 3 2024-04-21T08:40:48.971 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.971 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.971 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.971 INFO:tasks.workunit.client.0.smithi086.stdout:20/600: open fd = 3 2024-04-21T08:40:48.971 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.971 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.971 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.971 INFO:tasks.workunit.client.0.smithi086.stdout:21/600: open fd = 3 2024-04-21T08:40:48.971 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.971 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.971 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.972 INFO:tasks.workunit.client.0.smithi086.stdout:22/600: open fd = 3 2024-04-21T08:40:48.972 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.972 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.972 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.972 INFO:tasks.workunit.client.0.smithi086.stdout:23/600: open fd = 3 2024-04-21T08:40:48.972 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.972 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.972 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.972 INFO:tasks.workunit.client.0.smithi086.stdout:24/600: open fd = 3 2024-04-21T08:40:48.972 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.972 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.972 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.972 INFO:tasks.workunit.client.0.smithi086.stdout:25/600: open fd = 3 2024-04-21T08:40:48.972 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.972 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.973 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.973 INFO:tasks.workunit.client.0.smithi086.stdout:26/600: open fd = 3 2024-04-21T08:40:48.973 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.973 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.973 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.973 INFO:tasks.workunit.client.0.smithi086.stdout:27/600: open fd = 3 2024-04-21T08:40:48.973 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.973 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.973 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.973 INFO:tasks.workunit.client.0.smithi086.stdout:28/600: open fd = 3 2024-04-21T08:40:48.973 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.973 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.973 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.973 INFO:tasks.workunit.client.0.smithi086.stdout:29/600: open fd = 3 2024-04-21T08:40:48.973 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.974 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.974 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.974 INFO:tasks.workunit.client.0.smithi086.stdout:30/600: open fd = 3 2024-04-21T08:40:48.974 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.974 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.974 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.974 INFO:tasks.workunit.client.0.smithi086.stdout:31/600: open fd = 3 2024-04-21T08:40:48.974 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.974 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.974 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.974 INFO:tasks.workunit.client.0.smithi086.stdout:32/600: open fd = 3 2024-04-21T08:40:48.974 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.974 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.974 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.975 INFO:tasks.workunit.client.0.smithi086.stdout:33/600: open fd = 3 2024-04-21T08:40:48.975 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.975 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.975 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.975 INFO:tasks.workunit.client.0.smithi086.stdout:34/600: open fd = 3 2024-04-21T08:40:48.975 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.975 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.975 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.975 INFO:tasks.workunit.client.0.smithi086.stdout:35/600: open fd = 3 2024-04-21T08:40:48.975 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.975 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.975 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.975 INFO:tasks.workunit.client.0.smithi086.stdout:36/600: open fd = 3 2024-04-21T08:40:48.975 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.975 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.976 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.976 INFO:tasks.workunit.client.0.smithi086.stdout:37/600: open fd = 3 2024-04-21T08:40:48.976 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.976 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.976 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.976 INFO:tasks.workunit.client.0.smithi086.stdout:38/600: open fd = 3 2024-04-21T08:40:48.976 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.976 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.976 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.976 INFO:tasks.workunit.client.0.smithi086.stdout:39/600: open fd = 3 2024-04-21T08:40:48.976 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.976 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.976 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.976 INFO:tasks.workunit.client.0.smithi086.stdout:40/600: open fd = 3 2024-04-21T08:40:48.977 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.977 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.977 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.977 INFO:tasks.workunit.client.0.smithi086.stdout:41/600: open fd = 3 2024-04-21T08:40:48.977 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.977 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.977 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.977 INFO:tasks.workunit.client.0.smithi086.stdout:42/600: open fd = 3 2024-04-21T08:40:48.977 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.977 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.977 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.977 INFO:tasks.workunit.client.0.smithi086.stdout:43/600: open fd = 3 2024-04-21T08:40:48.977 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.977 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.978 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.978 INFO:tasks.workunit.client.0.smithi086.stdout:44/600: open fd = 3 2024-04-21T08:40:48.978 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.978 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.978 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.978 INFO:tasks.workunit.client.0.smithi086.stdout:45/600: open fd = 3 2024-04-21T08:40:48.978 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.978 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.978 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.978 INFO:tasks.workunit.client.0.smithi086.stdout:46/600: open fd = 3 2024-04-21T08:40:48.978 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.978 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.978 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.978 INFO:tasks.workunit.client.0.smithi086.stdout:47/600: open fd = 3 2024-04-21T08:40:48.979 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.979 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.979 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.979 INFO:tasks.workunit.client.0.smithi086.stdout:48/600: open fd = 3 2024-04-21T08:40:48.979 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.979 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.979 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.979 INFO:tasks.workunit.client.0.smithi086.stdout:49/600: open fd = 3 2024-04-21T08:40:48.979 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.979 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.979 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.979 INFO:tasks.workunit.client.0.smithi086.stdout:50/600: open fd = 3 2024-04-21T08:40:48.979 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.979 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.979 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.980 INFO:tasks.workunit.client.0.smithi086.stdout:51/600: open fd = 3 2024-04-21T08:40:48.980 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.980 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.980 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.980 INFO:tasks.workunit.client.0.smithi086.stdout:52/600: open fd = 3 2024-04-21T08:40:48.980 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.980 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.980 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.980 INFO:tasks.workunit.client.0.smithi086.stdout:53/600: open fd = 3 2024-04-21T08:40:48.980 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.980 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.980 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.980 INFO:tasks.workunit.client.0.smithi086.stdout:54/600: open fd = 3 2024-04-21T08:40:48.980 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.980 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.981 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.981 INFO:tasks.workunit.client.0.smithi086.stdout:55/600: open fd = 3 2024-04-21T08:40:48.981 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.981 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.981 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.981 INFO:tasks.workunit.client.0.smithi086.stdout:56/600: open fd = 3 2024-04-21T08:40:48.981 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.981 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.981 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.981 INFO:tasks.workunit.client.0.smithi086.stdout:57/600: open fd = 3 2024-04-21T08:40:48.981 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.981 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.981 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.981 INFO:tasks.workunit.client.0.smithi086.stdout:58/600: open fd = 3 2024-04-21T08:40:48.981 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.982 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.982 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.982 INFO:tasks.workunit.client.0.smithi086.stdout:59/600: open fd = 3 2024-04-21T08:40:48.982 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.982 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.982 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.982 INFO:tasks.workunit.client.0.smithi086.stdout:60/600: open fd = 3 2024-04-21T08:40:48.982 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.982 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.982 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.982 INFO:tasks.workunit.client.0.smithi086.stdout:61/600: open fd = 3 2024-04-21T08:40:48.982 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.982 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.982 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.983 INFO:tasks.workunit.client.0.smithi086.stdout:62/600: open fd = 3 2024-04-21T08:40:48.983 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.983 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.983 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:48.983 INFO:tasks.workunit.client.0.smithi086.stdout:63/600: open fd = 3 2024-04-21T08:40:48.983 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.983 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:40:48.983 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:40:49.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:49 smithi179 ceph-mon[31829]: pgmap v64: 129 pgs: 129 active+clean; 2.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 22 op/s 2024-04-21T08:40:50.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:49 smithi086 ceph-mon[27939]: pgmap v64: 129 pgs: 129 active+clean; 2.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 22 op/s 2024-04-21T08:40:50.213 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:49 smithi088 ceph-mon[31259]: pgmap v64: 129 pgs: 129 active+clean; 2.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 22 op/s 2024-04-21T08:40:51.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:51 smithi179 ceph-mon[31829]: pgmap v65: 129 pgs: 129 active+clean; 2.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-21T08:40:52.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:51 smithi086 ceph-mon[27939]: pgmap v65: 129 pgs: 129 active+clean; 2.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-21T08:40:52.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.180+0000 7fcc97c7a640 1 -- 172.21.15.86:0/307049629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc90102e50 msgr2=0x7fcc9006a730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:52.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.180+0000 7fcc97c7a640 1 --2- 172.21.15.86:0/307049629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc90102e50 0x7fcc9006a730 secure :-1 s=READY pgs=824 cs=0 l=1 rev1=1 crypto rx=0x7fcc8c009900 tx=0x7fcc8c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:40:52.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.180+0000 7fcc97c7a640 1 -- 172.21.15.86:0/307049629 shutdown_connections 2024-04-21T08:40:52.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.180+0000 7fcc97c7a640 1 --2- 172.21.15.86:0/307049629 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc9010e830 0x7fcc90110c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:52.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.180+0000 7fcc97c7a640 1 --2- 172.21.15.86:0/307049629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc9006ac70 0x7fcc9006b0d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:52.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.180+0000 7fcc97c7a640 1 --2- 172.21.15.86:0/307049629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc90102e50 0x7fcc9006a730 unknown :-1 s=CLOSED pgs=824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:52.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.180+0000 7fcc97c7a640 1 -- 172.21.15.86:0/307049629 >> 172.21.15.86:0/307049629 conn(0x7fcc90077c30 msgr2=0x7fcc90078040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:52.182 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.180+0000 7fcc97c7a640 1 -- 172.21.15.86:0/307049629 shutdown_connections 2024-04-21T08:40:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.180+0000 7fcc97c7a640 1 -- 172.21.15.86:0/307049629 wait complete. 2024-04-21T08:40:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.181+0000 7fcc97c7a640 1 Processor -- start 2024-04-21T08:40:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.181+0000 7fcc97c7a640 1 -- start start 2024-04-21T08:40:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.181+0000 7fcc97c7a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc9006ac70 0x7fcc901a05d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.181+0000 7fcc97c7a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc90102e50 0x7fcc901a0b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.181+0000 7fcc97c7a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc9010e830 0x7fcc9019a690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.181+0000 7fcc97c7a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcc901143b0 con 0x7fcc90102e50 2024-04-21T08:40:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.181+0000 7fcc97c7a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcc90114230 con 0x7fcc9006ac70 2024-04-21T08:40:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.181+0000 7fcc97c7a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fcc90114530 con 0x7fcc9010e830 2024-04-21T08:40:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.181+0000 7fcc95215640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc90102e50 0x7fcc901a0b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.181+0000 7fcc95215640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc90102e50 0x7fcc901a0b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54680/0 (socket says 172.21.15.86:54680) 2024-04-21T08:40:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.181+0000 7fcc95215640 1 -- 172.21.15.86:0/728425980 learned_addr learned my addr 172.21.15.86:0/728425980 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.182+0000 7fcc96217640 1 --2- 172.21.15.86:0/728425980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc9010e830 0x7fcc9019a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.182+0000 7fcc95a16640 1 --2- 172.21.15.86:0/728425980 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc9006ac70 0x7fcc901a05d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:52.183 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.182+0000 7fcc95215640 1 -- 172.21.15.86:0/728425980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc9010e830 msgr2=0x7fcc9019a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:52.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.182+0000 7fcc95215640 1 --2- 172.21.15.86:0/728425980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc9010e830 0x7fcc9019a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:52.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.182+0000 7fcc95215640 1 -- 172.21.15.86:0/728425980 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc9006ac70 msgr2=0x7fcc901a05d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:52.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.182+0000 7fcc95215640 1 --2- 172.21.15.86:0/728425980 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc9006ac70 0x7fcc901a05d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:52.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.182+0000 7fcc95215640 1 -- 172.21.15.86:0/728425980 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc9019af50 con 0x7fcc90102e50 2024-04-21T08:40:52.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.182+0000 7fcc95215640 1 --2- 172.21.15.86:0/728425980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc90102e50 0x7fcc901a0b10 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fcc8000c890 tx=0x7fcc8000cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:52.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.182+0000 7fcc86ffd640 1 -- 172.21.15.86:0/728425980 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc80004220 con 0x7fcc90102e50 2024-04-21T08:40:52.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.182+0000 7fcc86ffd640 1 -- 172.21.15.86:0/728425980 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc800043c0 con 0x7fcc90102e50 2024-04-21T08:40:52.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.182+0000 7fcc97c7a640 1 -- 172.21.15.86:0/728425980 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc9019b240 con 0x7fcc90102e50 2024-04-21T08:40:52.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.182+0000 7fcc86ffd640 1 -- 172.21.15.86:0/728425980 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc80004a00 con 0x7fcc90102e50 2024-04-21T08:40:52.184 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.182+0000 7fcc97c7a640 1 -- 172.21.15.86:0/728425980 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc901a7550 con 0x7fcc90102e50 2024-04-21T08:40:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.184+0000 7fcc86ffd640 1 -- 172.21.15.86:0/728425980 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcc800079b0 con 0x7fcc90102e50 2024-04-21T08:40:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.185+0000 7fcc9a86a740 1 Processor -- start 2024-04-21T08:40:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.185+0000 7fcc9a86a740 1 -- start start 2024-04-21T08:40:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.185+0000 7fcc9a86a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2c2b698d0 0x55b2c2b68620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.185+0000 7fcc9a86a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c2b68b60 0x55b2c2b68fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.185+0000 7fcc9a86a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2c2b720e0 0x55b2c2b744d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.185+0000 7fcc9a86a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b2c2acd8d0 con 0x55b2c2b720e0 2024-04-21T08:40:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.185+0000 7fcc9a86a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b2c2a6ae80 con 0x55b2c2b698d0 2024-04-21T08:40:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.185+0000 7fcc9a86a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b2c2ac9890 con 0x55b2c2b68b60 2024-04-21T08:40:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.186+0000 7fcc86ffd640 1 --2- 172.21.15.86:0/728425980 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fcc7c0a3e40 0x7fcc7c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:52.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.186+0000 7fcc95215640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2c2b720e0 0x55b2c2b744d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.186+0000 7fcc95215640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2c2b720e0 0x55b2c2b744d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54696/0 (socket says 172.21.15.86:54696) 2024-04-21T08:40:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.186+0000 7fcc95215640 1 -- 172.21.15.86:0/1751827084 learned_addr learned my addr 172.21.15.86:0/1751827084 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.186+0000 7fcc96217640 1 --2- 172.21.15.86:0/728425980 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fcc7c0a3e40 0x7fcc7c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.186+0000 7fcc95a16640 1 --2- 172.21.15.86:0/1751827084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c2b68b60 0x55b2c2b68fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.186+0000 7fcc96217640 1 --2- 172.21.15.86:0/1751827084 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2c2b698d0 0x55b2c2b68620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.186+0000 7fcc86ffd640 1 -- 172.21.15.86:0/728425980 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fcc800d07d0 con 0x7fcc90102e50 2024-04-21T08:40:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.186+0000 7fcc95215640 1 -- 172.21.15.86:0/1751827084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c2b68b60 msgr2=0x55b2c2b68fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.186+0000 7fcc95215640 1 --2- 172.21.15.86:0/1751827084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c2b68b60 0x55b2c2b68fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.186+0000 7fcc95215640 1 -- 172.21.15.86:0/1751827084 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2c2b698d0 msgr2=0x55b2c2b68620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.186+0000 7fcc95215640 1 --2- 172.21.15.86:0/1751827084 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2c2b698d0 0x55b2c2b68620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.186+0000 7fcc95215640 1 -- 172.21.15.86:0/1751827084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2c2a8c320 con 0x55b2c2b720e0 2024-04-21T08:40:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.186+0000 7fcc96217640 1 --2- 172.21.15.86:0/728425980 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fcc7c0a3e40 0x7fcc7c0a6300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcc88005fd0 tx=0x7fcc88009e70 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.186+0000 7fcc95215640 1 --2- 172.21.15.86:0/1751827084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2c2b720e0 0x55b2c2b744d0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fcc80005fd0 tx=0x7fcc8009bfb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2902d7ce4f4178e9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:52.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.186+0000 7fcc7bfff640 1 -- 172.21.15.86:0/1751827084 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8001d050 con 0x55b2c2b720e0 2024-04-21T08:40:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.186+0000 7fcc7bfff640 1 -- 172.21.15.86:0/1751827084 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc8009c470 con 0x55b2c2b720e0 2024-04-21T08:40:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.187+0000 7fcc7bfff640 1 -- 172.21.15.86:0/1751827084 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc80007dc0 con 0x55b2c2b720e0 2024-04-21T08:40:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.187+0000 7fcc9a86a740 1 -- 172.21.15.86:0/1751827084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2c2b720e0 msgr2=0x55b2c2b744d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.187+0000 7fcc9a86a740 1 --2- 172.21.15.86:0/1751827084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2c2b720e0 0x55b2c2b744d0 secure :-1 s=READY pgs=826 cs=0 l=1 rev1=1 crypto rx=0x7fcc80005fd0 tx=0x7fcc8009bfb0 comp rx=0 tx=0).stop 2024-04-21T08:40:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.187+0000 7fcc9a86a740 1 -- 172.21.15.86:0/1751827084 shutdown_connections 2024-04-21T08:40:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.187+0000 7fcc9a86a740 1 --2- 172.21.15.86:0/1751827084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2c2b720e0 0x55b2c2b744d0 unknown :-1 s=CLOSED pgs=826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.187+0000 7fcc9a86a740 1 --2- 172.21.15.86:0/1751827084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c2b68b60 0x55b2c2b68fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.187+0000 7fcc9a86a740 1 --2- 172.21.15.86:0/1751827084 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2c2b698d0 0x55b2c2b68620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.187+0000 7fcc9a86a740 1 -- 172.21.15.86:0/1751827084 >> 172.21.15.86:0/1751827084 conn(0x55b2c2b2fc90 msgr2=0x55b2c2b63850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.187+0000 7fcc9a86a740 1 -- 172.21.15.86:0/1751827084 shutdown_connections 2024-04-21T08:40:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.187+0000 7fcc9a86a740 1 -- 172.21.15.86:0/1751827084 wait complete. 2024-04-21T08:40:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.188+0000 7fcc9a86a740 1 -- start start 2024-04-21T08:40:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.188+0000 7fcc9a86a740 1 Processor -- start 2024-04-21T08:40:52.193 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.188+0000 7fcc9a86a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2c2b68b60 0x55b2c2b71ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.188+0000 7fcc9a86a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2c2b698d0 0x55b2c2b6ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.188+0000 7fcc9a86a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c2b720e0 0x55b2c2b6f460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.188+0000 7fcc9a86a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b2c2b74e10 con 0x55b2c2b698d0 2024-04-21T08:40:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.188+0000 7fcc9a86a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b2c2a55eb0 con 0x55b2c2b68b60 2024-04-21T08:40:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.188+0000 7fcc9a86a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b2c2b74f90 con 0x55b2c2b720e0 2024-04-21T08:40:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.188+0000 7fcc9a86a740 10 client.0 _ll_register_callbacks cb 0x55b2c2a82b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.188+0000 7fcc95215640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2c2b698d0 0x55b2c2b6ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.188+0000 7fcc95215640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2c2b698d0 0x55b2c2b6ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54698/0 (socket says 172.21.15.86:54698) 2024-04-21T08:40:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.188+0000 7fcc95215640 1 -- 172.21.15.86:0/3559091293 learned_addr learned my addr 172.21.15.86:0/3559091293 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.189+0000 7fcc95a16640 1 --2- 172.21.15.86:0/3559091293 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2c2b68b60 0x55b2c2b71ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.189+0000 7fcc96217640 1 --2- 172.21.15.86:0/3559091293 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c2b720e0 0x55b2c2b6f460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.189+0000 7fcc95215640 1 -- 172.21.15.86:0/3559091293 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c2b720e0 msgr2=0x55b2c2b6f460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.189+0000 7fcc95215640 1 --2- 172.21.15.86:0/3559091293 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c2b720e0 0x55b2c2b6f460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:52.194 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.189+0000 7fcc95215640 1 -- 172.21.15.86:0/3559091293 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2c2b68b60 msgr2=0x55b2c2b71ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.189+0000 7fcc95215640 1 --2- 172.21.15.86:0/3559091293 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2c2b68b60 0x55b2c2b71ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.189+0000 7fcc95215640 1 -- 172.21.15.86:0/3559091293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2c27e0210 con 0x55b2c2b698d0 2024-04-21T08:40:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.189+0000 7fcc96217640 1 --2- 172.21.15.86:0/3559091293 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c2b720e0 0x55b2c2b6f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:40:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.189+0000 7fcc95215640 1 --2- 172.21.15.86:0/3559091293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2c2b698d0 0x55b2c2b6ee50 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fcc800a4b00 tx=0x7fcc800a0770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c6ab1a90fc1d8c5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.189+0000 7fcc797fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:40:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.189+0000 7fcc797fa640 1 -- 172.21.15.86:0/3559091293 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc800b4070 con 0x55b2c2b698d0 2024-04-21T08:40:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.189+0000 7fcc797fa640 1 -- 172.21.15.86:0/3559091293 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fcc800d03c0 con 0x55b2c2b698d0 2024-04-21T08:40:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.189+0000 7fcc9a86a740 1 -- 172.21.15.86:0/3559091293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc8009c470 con 0x55b2c2b698d0 2024-04-21T08:40:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.189+0000 7fcc797fa640 1 -- 172.21.15.86:0/3559091293 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcc8001d070 con 0x55b2c2b698d0 2024-04-21T08:40:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.189+0000 7fcc9a86a740 10 client.16644 fetch_fsmap learned FSMap version 29 2024-04-21T08:40:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.189+0000 7fcc9a86a740 1 -- 172.21.15.86:0/3559091293 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b2c28ac620 con 0x55b2c2b698d0 2024-04-21T08:40:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.189+0000 7fcc797fa640 1 -- 172.21.15.86:0/3559091293 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc8009c470 con 0x55b2c2b698d0 2024-04-21T08:40:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.189+0000 7fcc797fa640 1 -- 172.21.15.86:0/3559091293 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcc800d0bc0 con 0x55b2c2b698d0 2024-04-21T08:40:52.195 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.190+0000 7fcc9a86a740 10 client.16644 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:40:52.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.190+0000 7fcc9a86a740 10 client.16644 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:52.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.190+0000 7fcc9a86a740 20 client.16644 populate_metadata read hostname 'smithi086' 2024-04-21T08:40:52.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.190+0000 7fcc9a86a740 1 --2- 172.21.15.86:0/3559091293 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b2c2b81ff0 0x55b2c2b843e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:52.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.190+0000 7fcc9a86a740 4 client.16644 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:40:52.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.190+0000 7fcc9a86a740 1 -- 172.21.15.86:0/3559091293 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b2c2ae1880 con 0x55b2c2b81ff0 2024-04-21T08:40:52.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.190+0000 7fcc95a16640 1 --2- 172.21.15.86:0/3559091293 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b2c2b81ff0 0x55b2c2b843e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:52.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.191+0000 7fcc95a16640 1 --2- 172.21.15.86:0/3559091293 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b2c2b81ff0 0x55b2c2b843e0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=eb451bcb5f36eeca server_cookie=f92d90ea5857b51 in_seq=0 out_seq=0 2024-04-21T08:40:52.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.191+0000 7fcc797fa640 10 client.16644 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:40:52.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.191+0000 7fcc797fa640 1 -- 172.21.15.86:0/3559091293 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b2c2ae1880 con 0x55b2c2b81ff0 2024-04-21T08:40:52.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.191+0000 7fcc797fa640 10 client.16644 handle_command_reply: tid=0 2024-04-21T08:40:52.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.191+0000 7fcc9a86a740 1 client.16644 shutdown 2024-04-21T08:40:52.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.191+0000 7fcc9a86a740 1 -- 172.21.15.86:0/3559091293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2c2b698d0 msgr2=0x55b2c2b6ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:52.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.191+0000 7fcc9a86a740 1 --2- 172.21.15.86:0/3559091293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2c2b698d0 0x55b2c2b6ee50 secure :-1 s=READY pgs=827 cs=0 l=1 rev1=1 crypto rx=0x7fcc800a4b00 tx=0x7fcc800a0770 comp rx=0 tx=0).stop 2024-04-21T08:40:52.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.191+0000 7fcc9a86a740 1 -- 172.21.15.86:0/3559091293 shutdown_connections 2024-04-21T08:40:52.196 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.191+0000 7fcc9a86a740 1 --2- 172.21.15.86:0/3559091293 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b2c2b81ff0 0x55b2c2b843e0 crc :-1 s=READY pgs=91 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.192+0000 7fcc9a86a740 1 --2- 172.21.15.86:0/3559091293 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c2b720e0 0x55b2c2b6f460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.192+0000 7fcc9a86a740 1 --2- 172.21.15.86:0/3559091293 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2c2b698d0 0x55b2c2b6ee50 unknown :-1 s=CLOSED pgs=827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.192+0000 7fcc797fa640 0 client.16644 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:40:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.192+0000 7fcc9a86a740 1 --2- 172.21.15.86:0/3559091293 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2c2b68b60 0x55b2c2b71ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.192+0000 7fcc9a86a740 1 -- 172.21.15.86:0/3559091293 >> 172.21.15.86:0/3559091293 conn(0x55b2c2b2fc90 msgr2=0x55b2c2b655a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.192+0000 7fcc9a86a740 1 -- 172.21.15.86:0/3559091293 shutdown_connections 2024-04-21T08:40:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.192+0000 7fcc9a86a740 1 -- 172.21.15.86:0/3559091293 wait complete. 2024-04-21T08:40:52.197 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.192+0000 7fcc9a86a740 20 client.16644 trim_cache size 0 max 16384 2024-04-21T08:40:52.210 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.206+0000 7fcc9a86a740 1 Processor -- start 2024-04-21T08:40:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.206+0000 7fcc9a86a740 1 -- start start 2024-04-21T08:40:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.207+0000 7fcc9a86a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c2b68b60 0x55b2c2b6de30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.207+0000 7fcc9a86a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2c2b698d0 0x55b2c2b71580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.207+0000 7fcc9a86a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2c2b720e0 0x55b2c2b71ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.207+0000 7fcc9a86a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcc901b13f0 con 0x55b2c2b698d0 2024-04-21T08:40:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.207+0000 7fcc9a86a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcc901b2f30 con 0x55b2c2b720e0 2024-04-21T08:40:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.207+0000 7fcc9a86a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fcc901b1ac0 con 0x55b2c2b68b60 2024-04-21T08:40:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.207+0000 7fcc95215640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2c2b698d0 0x55b2c2b71580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.207+0000 7fcc95215640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2c2b698d0 0x55b2c2b71580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54704/0 (socket says 172.21.15.86:54704) 2024-04-21T08:40:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.207+0000 7fcc95215640 1 -- 172.21.15.86:0/1885075673 learned_addr learned my addr 172.21.15.86:0/1885075673 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.207+0000 7fcc96217640 1 --2- 172.21.15.86:0/1885075673 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2c2b720e0 0x55b2c2b71ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:52.211 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.207+0000 7fcc95a16640 1 --2- 172.21.15.86:0/1885075673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c2b68b60 0x55b2c2b6de30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.207+0000 7fcc95215640 1 -- 172.21.15.86:0/1885075673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c2b68b60 msgr2=0x55b2c2b6de30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.207+0000 7fcc95215640 1 --2- 172.21.15.86:0/1885075673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c2b68b60 0x55b2c2b6de30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.207+0000 7fcc95215640 1 -- 172.21.15.86:0/1885075673 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2c2b720e0 msgr2=0x55b2c2b71ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.207+0000 7fcc95215640 1 --2- 172.21.15.86:0/1885075673 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2c2b720e0 0x55b2c2b71ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.207+0000 7fcc95215640 1 -- 172.21.15.86:0/1885075673 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2c2b67e30 con 0x55b2c2b698d0 2024-04-21T08:40:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.207+0000 7fcc96217640 1 --2- 172.21.15.86:0/1885075673 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2c2b720e0 0x55b2c2b71ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.207+0000 7fcc95215640 1 --2- 172.21.15.86:0/1885075673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2c2b698d0 0x55b2c2b71580 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fcc80098e40 tx=0x7fcc8009a980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=abdf915095fef042 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.207+0000 7fcc7b7fe640 1 -- 172.21.15.86:0/1885075673 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc800b4040 con 0x55b2c2b698d0 2024-04-21T08:40:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.207+0000 7fcc7b7fe640 1 -- 172.21.15.86:0/1885075673 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc8009bbc0 con 0x55b2c2b698d0 2024-04-21T08:40:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.207+0000 7fcc7b7fe640 1 -- 172.21.15.86:0/1885075673 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc800b0e60 con 0x55b2c2b698d0 2024-04-21T08:40:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.208+0000 7fcc9a86a740 1 -- 172.21.15.86:0/1885075673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2c2b698d0 msgr2=0x55b2c2b71580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.208+0000 7fcc9a86a740 1 --2- 172.21.15.86:0/1885075673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2c2b698d0 0x55b2c2b71580 secure :-1 s=READY pgs=828 cs=0 l=1 rev1=1 crypto rx=0x7fcc80098e40 tx=0x7fcc8009a980 comp rx=0 tx=0).stop 2024-04-21T08:40:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.208+0000 7fcc9a86a740 1 -- 172.21.15.86:0/1885075673 shutdown_connections 2024-04-21T08:40:52.212 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.208+0000 7fcc9a86a740 1 --2- 172.21.15.86:0/1885075673 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2c2b720e0 0x55b2c2b71ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.208+0000 7fcc9a86a740 1 --2- 172.21.15.86:0/1885075673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2c2b698d0 0x55b2c2b71580 unknown :-1 s=CLOSED pgs=828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.208+0000 7fcc9a86a740 1 --2- 172.21.15.86:0/1885075673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c2b68b60 0x55b2c2b6de30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.208+0000 7fcc9a86a740 1 -- 172.21.15.86:0/1885075673 >> 172.21.15.86:0/1885075673 conn(0x55b2c2b2fc90 msgr2=0x55b2c2b6d720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.208+0000 7fcc9a86a740 1 -- 172.21.15.86:0/1885075673 shutdown_connections 2024-04-21T08:40:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.208+0000 7fcc9a86a740 1 -- 172.21.15.86:0/1885075673 wait complete. 2024-04-21T08:40:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.208+0000 7fcc9a86a740 1 -- start start 2024-04-21T08:40:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.208+0000 7fcc9a86a740 1 Processor -- start 2024-04-21T08:40:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.209+0000 7fcc9a86a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c2b68b60 0x55b2c2b873a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.209+0000 7fcc9a86a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2c2b698d0 0x55b2c2b878e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.209+0000 7fcc9a86a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2c2b720e0 0x55b2c2b828d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.209+0000 7fcc9a86a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcc901b2860 con 0x55b2c2b720e0 2024-04-21T08:40:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.209+0000 7fcc9a86a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcc901b2190 con 0x55b2c2b698d0 2024-04-21T08:40:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.209+0000 7fcc9a86a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fcc901af3b0 con 0x55b2c2b68b60 2024-04-21T08:40:52.213 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.209+0000 7fcc9a86a740 10 client.0 _ll_register_callbacks cb 0x7fcc88011720 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:40:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.209+0000 7fcc96217640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2c2b720e0 0x55b2c2b828d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.209+0000 7fcc96217640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2c2b720e0 0x55b2c2b828d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54710/0 (socket says 172.21.15.86:54710) 2024-04-21T08:40:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.209+0000 7fcc96217640 1 -- 172.21.15.86:0/1025337368 learned_addr learned my addr 172.21.15.86:0/1025337368 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:40:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.209+0000 7fcc95a16640 1 --2- 172.21.15.86:0/1025337368 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c2b68b60 0x55b2c2b873a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.209+0000 7fcc95215640 1 --2- 172.21.15.86:0/1025337368 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2c2b698d0 0x55b2c2b878e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.209+0000 7fcc96217640 1 -- 172.21.15.86:0/1025337368 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c2b68b60 msgr2=0x55b2c2b873a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.209+0000 7fcc96217640 1 --2- 172.21.15.86:0/1025337368 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c2b68b60 0x55b2c2b873a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.209+0000 7fcc96217640 1 -- 172.21.15.86:0/1025337368 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2c2b698d0 msgr2=0x55b2c2b878e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:40:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.209+0000 7fcc96217640 1 --2- 172.21.15.86:0/1025337368 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2c2b698d0 0x55b2c2b878e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.209+0000 7fcc96217640 1 -- 172.21.15.86:0/1025337368 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2c2b83100 con 0x55b2c2b720e0 2024-04-21T08:40:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.209+0000 7fcc95a16640 1 --2- 172.21.15.86:0/1025337368 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c2b68b60 0x55b2c2b873a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:40:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.209+0000 7fcc95215640 1 --2- 172.21.15.86:0/1025337368 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2c2b698d0 0x55b2c2b878e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:40:52.214 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.209+0000 7fcc96217640 1 --2- 172.21.15.86:0/1025337368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2c2b720e0 0x55b2c2b828d0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fcc88009590 tx=0x7fcc88009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=362e70eeed37acf6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:40:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.209+0000 7fcc797fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:40:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.210+0000 7fcc797fa640 1 -- 172.21.15.86:0/1025337368 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcc8802e060 con 0x55b2c2b720e0 2024-04-21T08:40:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.210+0000 7fcc797fa640 1 -- 172.21.15.86:0/1025337368 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fcc8801a070 con 0x55b2c2b720e0 2024-04-21T08:40:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.210+0000 7fcc9a86a740 1 -- 172.21.15.86:0/1025337368 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcc8009bbc0 con 0x55b2c2b720e0 2024-04-21T08:40:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.210+0000 7fcc797fa640 1 -- 172.21.15.86:0/1025337368 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcc88035050 con 0x55b2c2b720e0 2024-04-21T08:40:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.210+0000 7fcc9a86a740 10 client.16650 fetch_fsmap learned FSMap version 29 2024-04-21T08:40:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.210+0000 7fcc9a86a740 1 -- 172.21.15.86:0/1025337368 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b2c2be7dd0 con 0x55b2c2b720e0 2024-04-21T08:40:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.210+0000 7fcc797fa640 1 -- 172.21.15.86:0/1025337368 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcc8009bbc0 con 0x55b2c2b720e0 2024-04-21T08:40:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.210+0000 7fcc797fa640 1 -- 172.21.15.86:0/1025337368 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcc88020bd0 con 0x55b2c2b720e0 2024-04-21T08:40:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.210+0000 7fcc9a86a740 10 client.16650 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:40:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.210+0000 7fcc9a86a740 10 client.16650 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:40:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.210+0000 7fcc9a86a740 20 client.16650 populate_metadata read hostname 'smithi086' 2024-04-21T08:40:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.210+0000 7fcc9a86a740 1 --2- 172.21.15.86:0/1025337368 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b2c2be8580 0x55b2c2bea9a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:40:52.215 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.210+0000 7fcc9a86a740 4 client.16650 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:40:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.210+0000 7fcc9a86a740 1 -- 172.21.15.86:0/1025337368 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b2c2a6d3d0 con 0x55b2c2be8580 2024-04-21T08:40:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.210+0000 7fcc95a16640 1 --2- 172.21.15.86:0/1025337368 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b2c2be8580 0x55b2c2bea9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:40:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.211+0000 7fcc95a16640 1 --2- 172.21.15.86:0/1025337368 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b2c2be8580 0x55b2c2bea9a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50716f85ccdd27d9 server_cookie=72cd8d78ded18dfb in_seq=0 out_seq=0 2024-04-21T08:40:52.216 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.211+0000 7fcc797fa640 10 client.16650 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:40:52.216 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:51 smithi088 ceph-mon[31259]: pgmap v65: 129 pgs: 129 active+clean; 2.5 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-21T08:40:52.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.225+0000 7fcc797fa640 1 -- 172.21.15.86:0/1025337368 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b2c2a6d3d0 con 0x55b2c2be8580 2024-04-21T08:40:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.225+0000 7fcc797fa640 10 client.16650 handle_command_reply: tid=0 2024-04-21T08:40:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.225+0000 7fcc9a86a740 1 client.16650 shutdown 2024-04-21T08:40:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.225+0000 7fcc9a86a740 1 -- 172.21.15.86:0/1025337368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2c2b720e0 msgr2=0x55b2c2b828d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.225+0000 7fcc9a86a740 1 --2- 172.21.15.86:0/1025337368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2c2b720e0 0x55b2c2b828d0 secure :-1 s=READY pgs=829 cs=0 l=1 rev1=1 crypto rx=0x7fcc88009590 tx=0x7fcc88009500 comp rx=0 tx=0).stop 2024-04-21T08:40:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.225+0000 7fcc9a86a740 1 -- 172.21.15.86:0/1025337368 shutdown_connections 2024-04-21T08:40:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.225+0000 7fcc9a86a740 1 --2- 172.21.15.86:0/1025337368 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b2c2be8580 0x55b2c2bea9a0 crc :-1 s=READY pgs=92 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.225+0000 7fcc9a86a740 1 --2- 172.21.15.86:0/1025337368 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2c2b720e0 0x55b2c2b828d0 unknown :-1 s=CLOSED pgs=829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.225+0000 7fcc9a86a740 1 --2- 172.21.15.86:0/1025337368 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2c2b698d0 0x55b2c2b878e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.225+0000 7fcc9a86a740 1 --2- 172.21.15.86:0/1025337368 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2c2b68b60 0x55b2c2b873a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.225+0000 7fcc797fa640 0 client.16650 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:40:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.225+0000 7fcc9a86a740 1 -- 172.21.15.86:0/1025337368 >> 172.21.15.86:0/1025337368 conn(0x55b2c2b2fc90 msgr2=0x55b2c2b628d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.225+0000 7fcc9a86a740 1 -- 172.21.15.86:0/1025337368 shutdown_connections 2024-04-21T08:40:52.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.225+0000 7fcc9a86a740 1 -- 172.21.15.86:0/1025337368 wait complete. 2024-04-21T08:40:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.225+0000 7fcc9a86a740 20 client.16650 trim_cache size 0 max 16384 2024-04-21T08:40:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:40:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:40:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:40:52.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:40:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.226+0000 7fcc97c7a640 1 -- 172.21.15.86:0/728425980 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fcc7c0a3e40 msgr2=0x7fcc7c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.226+0000 7fcc97c7a640 1 --2- 172.21.15.86:0/728425980 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fcc7c0a3e40 0x7fcc7c0a6300 secure :-1 s=READY pgs=118 cs=0 l=1 rev1=1 crypto rx=0x7fcc88005fd0 tx=0x7fcc88009e70 comp rx=0 tx=0).stop 2024-04-21T08:40:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.226+0000 7fcc97c7a640 1 -- 172.21.15.86:0/728425980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc90102e50 msgr2=0x7fcc901a0b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:40:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.226+0000 7fcc97c7a640 1 --2- 172.21.15.86:0/728425980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc90102e50 0x7fcc901a0b10 secure :-1 s=READY pgs=825 cs=0 l=1 rev1=1 crypto rx=0x7fcc8000c890 tx=0x7fcc8000cd50 comp rx=0 tx=0).stop 2024-04-21T08:40:52.227 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.226+0000 7fcc97c7a640 1 -- 172.21.15.86:0/728425980 shutdown_connections 2024-04-21T08:40:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.226+0000 7fcc97c7a640 1 --2- 172.21.15.86:0/728425980 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fcc7c0a3e40 0x7fcc7c0a6300 unknown :-1 s=CLOSED pgs=118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.226+0000 7fcc97c7a640 1 --2- 172.21.15.86:0/728425980 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc9010e830 0x7fcc9019a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.226+0000 7fcc97c7a640 1 --2- 172.21.15.86:0/728425980 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc90102e50 0x7fcc901a0b10 unknown :-1 s=CLOSED pgs=825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.226+0000 7fcc97c7a640 1 --2- 172.21.15.86:0/728425980 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc9006ac70 0x7fcc901a05d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:40:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.226+0000 7fcc97c7a640 1 -- 172.21.15.86:0/728425980 >> 172.21.15.86:0/728425980 conn(0x7fcc90077c30 msgr2=0x7fcc90100120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:40:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.226+0000 7fcc97c7a640 1 -- 172.21.15.86:0/728425980 shutdown_connections 2024-04-21T08:40:52.228 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:40:52.226+0000 7fcc97c7a640 1 -- 172.21.15.86:0/728425980 wait complete. 2024-04-21T08:40:53.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:52 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:40:53.213 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:52 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:40:53.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:52 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:40:54.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:54 smithi179 ceph-mon[31829]: pgmap v66: 129 pgs: 129 active+clean; 2.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 25 op/s 2024-04-21T08:40:54.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:54 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:40:54.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:54 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:40:54.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:54 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:40:54.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:54 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:40:54.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:54 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:40:54.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:54 smithi086 ceph-mon[27939]: pgmap v66: 129 pgs: 129 active+clean; 2.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 25 op/s 2024-04-21T08:40:54.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:54 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:40:54.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:54 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:40:54.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:54 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:40:54.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:54 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:40:54.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:54 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:40:54.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:54 smithi088 ceph-mon[31259]: pgmap v66: 129 pgs: 129 active+clean; 2.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 33 KiB/s wr, 25 op/s 2024-04-21T08:40:54.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:54 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:40:54.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:54 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:40:54.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:54 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:40:54.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:54 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:40:54.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:54 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:40:56.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:56 smithi179 ceph-mon[31829]: pgmap v67: 129 pgs: 129 active+clean; 2.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-21T08:40:56.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:56 smithi086 ceph-mon[27939]: pgmap v67: 129 pgs: 129 active+clean; 2.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-21T08:40:56.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:56 smithi088 ceph-mon[31259]: pgmap v67: 129 pgs: 129 active+clean; 2.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-21T08:40:58.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:40:58 smithi179 ceph-mon[31829]: pgmap v68: 129 pgs: 129 active+clean; 2.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-21T08:40:58.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:40:58 smithi086 ceph-mon[27939]: pgmap v68: 129 pgs: 129 active+clean; 2.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-21T08:40:58.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:40:58 smithi088 ceph-mon[31259]: pgmap v68: 129 pgs: 129 active+clean; 2.6 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 20 KiB/s rd, 21 KiB/s wr, 15 op/s 2024-04-21T08:41:00.401 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:41:00.401 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:41:00.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:00 smithi179 ceph-mon[31829]: pgmap v69: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 22 op/s 2024-04-21T08:41:00.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.510+0000 7f43bf8f2640 1 -- 172.21.15.86:0/3172191288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43b8106660 msgr2=0x7f43b8106a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:00.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.510+0000 7f43bf8f2640 1 --2- 172.21.15.86:0/3172191288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43b8106660 0x7f43b8106a40 secure :-1 s=READY pgs=830 cs=0 l=1 rev1=1 crypto rx=0x7f43b4009930 tx=0x7f43b402bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:00.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.510+0000 7f43bf8f2640 1 -- 172.21.15.86:0/3172191288 shutdown_connections 2024-04-21T08:41:00.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.510+0000 7f43bf8f2640 1 --2- 172.21.15.86:0/3172191288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43b8100f80 0x7f43b810d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.510+0000 7f43bf8f2640 1 --2- 172.21.15.86:0/3172191288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43b81005c0 0x7f43b8100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.510+0000 7f43bf8f2640 1 --2- 172.21.15.86:0/3172191288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43b8106660 0x7f43b8106a40 unknown :-1 s=CLOSED pgs=830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.510+0000 7f43bf8f2640 1 -- 172.21.15.86:0/3172191288 >> 172.21.15.86:0/3172191288 conn(0x7f43b80773a0 msgr2=0x7f43b80fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:00.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.510+0000 7f43bf8f2640 1 -- 172.21.15.86:0/3172191288 shutdown_connections 2024-04-21T08:41:00.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.510+0000 7f43bf8f2640 1 -- 172.21.15.86:0/3172191288 wait complete. 2024-04-21T08:41:00.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.510+0000 7f43bf8f2640 1 Processor -- start 2024-04-21T08:41:00.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.510+0000 7f43bf8f2640 1 -- start start 2024-04-21T08:41:00.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.511+0000 7f43bf8f2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43b81005c0 0x7f43b8199530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.511+0000 7f43bf8f2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43b8100f80 0x7f43b8199a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.511+0000 7f43bf8f2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43b8106660 0x7f43b81a45b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.511+0000 7f43bf8f2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f43b8111090 con 0x7f43b8106660 2024-04-21T08:41:00.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.511+0000 7f43bf8f2640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f43b8110f10 con 0x7f43b81005c0 2024-04-21T08:41:00.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.511+0000 7f43bf8f2640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f43b8111210 con 0x7f43b8100f80 2024-04-21T08:41:00.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.511+0000 7f43bde8f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43b8106660 0x7f43b81a45b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:00.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.511+0000 7f43bde8f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43b8106660 0x7f43b81a45b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60254/0 (socket says 172.21.15.86:60254) 2024-04-21T08:41:00.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.511+0000 7f43bde8f640 1 -- 172.21.15.86:0/1931527568 learned_addr learned my addr 172.21.15.86:0/1931527568 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:00.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.511+0000 7f43bd68e640 1 --2- 172.21.15.86:0/1931527568 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43b81005c0 0x7f43b8199530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:00.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.511+0000 7f43bce8d640 1 --2- 172.21.15.86:0/1931527568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43b8100f80 0x7f43b8199a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:00.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.511+0000 7f43bde8f640 1 -- 172.21.15.86:0/1931527568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43b8100f80 msgr2=0x7f43b8199a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:00.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.511+0000 7f43bde8f640 1 --2- 172.21.15.86:0/1931527568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43b8100f80 0x7f43b8199a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.511+0000 7f43bde8f640 1 -- 172.21.15.86:0/1931527568 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43b81005c0 msgr2=0x7f43b8199530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:00.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.511+0000 7f43bde8f640 1 --2- 172.21.15.86:0/1931527568 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43b81005c0 0x7f43b8199530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.511+0000 7f43bde8f640 1 -- 172.21.15.86:0/1931527568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f43b81a4af0 con 0x7f43b8106660 2024-04-21T08:41:00.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.511+0000 7f43bce8d640 1 --2- 172.21.15.86:0/1931527568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43b8100f80 0x7f43b8199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:00.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.511+0000 7f43bde8f640 1 --2- 172.21.15.86:0/1931527568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43b8106660 0x7f43b81a45b0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f43b000cad0 tx=0x7f43b000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:00.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.511+0000 7f43ae7fc640 1 -- 172.21.15.86:0/1931527568 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b000ae20 con 0x7f43b8106660 2024-04-21T08:41:00.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.511+0000 7f43ae7fc640 1 -- 172.21.15.86:0/1931527568 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43b0005020 con 0x7f43b8106660 2024-04-21T08:41:00.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.511+0000 7f43ae7fc640 1 -- 172.21.15.86:0/1931527568 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b000f6b0 con 0x7f43b8106660 2024-04-21T08:41:00.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.511+0000 7f43bf8f2640 1 -- 172.21.15.86:0/1931527568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f43b81a4d20 con 0x7f43b8106660 2024-04-21T08:41:00.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.511+0000 7f43bf8f2640 1 -- 172.21.15.86:0/1931527568 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f43b81a51f0 con 0x7f43b8106660 2024-04-21T08:41:00.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.513+0000 7f43ae7fc640 1 -- 172.21.15.86:0/1931527568 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f43b00051d0 con 0x7f43b8106660 2024-04-21T08:41:00.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.514+0000 7f43c25c4740 1 Processor -- start 2024-04-21T08:41:00.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.514+0000 7f43c25c4740 1 -- start start 2024-04-21T08:41:00.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.514+0000 7f43c25c4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6375f8580 0x55f6375f8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.514+0000 7f43c25c4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f6375f8ea0 0x55f637601540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.514+0000 7f43c25c4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f637601a80 0x55f637603e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.514+0000 7f43c25c4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f63755d1e0 con 0x55f6375f8580 2024-04-21T08:41:00.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.514+0000 7f43ae7fc640 1 --2- 172.21.15.86:0/1931527568 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f43a40a3d20 0x7f43a40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.514+0000 7f43c25c4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55f6374fa390 con 0x55f6375f8ea0 2024-04-21T08:41:00.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.514+0000 7f43c25c4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f6375591a0 con 0x55f637601a80 2024-04-21T08:41:00.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.514+0000 7f43bce8d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6375f8580 0x55f6375f8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:00.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.514+0000 7f43bce8d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6375f8580 0x55f6375f8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60266/0 (socket says 172.21.15.86:60266) 2024-04-21T08:41:00.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.514+0000 7f43bde8f640 1 --2- 172.21.15.86:0/1931527568 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f43a40a3d20 0x7f43a40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:00.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.514+0000 7f43bce8d640 1 -- 172.21.15.86:0/2678027276 learned_addr learned my addr 172.21.15.86:0/2678027276 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:00.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.514+0000 7f43bd68e640 1 --2- 172.21.15.86:0/2678027276 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f6375f8ea0 0x55f637601540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:00.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.514+0000 7f43bce8d640 1 --2- 172.21.15.86:0/2678027276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f637601a80 0x55f637603e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:00.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.514+0000 7f43ae7fc640 1 -- 172.21.15.86:0/1931527568 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f43b00d0cb0 con 0x7f43b8106660 2024-04-21T08:41:00.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.515+0000 7f43bce8d640 1 -- 172.21.15.86:0/2678027276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f637601a80 msgr2=0x55f637603e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:00.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.515+0000 7f43bce8d640 1 --2- 172.21.15.86:0/2678027276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f637601a80 0x55f637603e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.515+0000 7f43bce8d640 1 -- 172.21.15.86:0/2678027276 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f6375f8ea0 msgr2=0x55f637601540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:00.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.515+0000 7f43bce8d640 1 --2- 172.21.15.86:0/2678027276 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f6375f8ea0 0x55f637601540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.515+0000 7f43bce8d640 1 -- 172.21.15.86:0/2678027276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f6373397d0 con 0x55f6375f8580 2024-04-21T08:41:00.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.515+0000 7f43bde8f640 1 --2- 172.21.15.86:0/1931527568 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f43a40a3d20 0x7f43a40a61e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f43b009c5e0 tx=0x7f43b006e320 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:00.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.515+0000 7f43bce8d640 1 --2- 172.21.15.86:0/2678027276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6375f8580 0x55f6375f8960 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f43a8012900 tx=0x7f43a8008f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b6bb512eacc6474 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:00.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.515+0000 7f43a37fe640 1 -- 172.21.15.86:0/2678027276 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a8010b80 con 0x55f6375f8580 2024-04-21T08:41:00.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.515+0000 7f43a37fe640 1 -- 172.21.15.86:0/2678027276 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43a8010d20 con 0x55f6375f8580 2024-04-21T08:41:00.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.515+0000 7f43a37fe640 1 -- 172.21.15.86:0/2678027276 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a8005350 con 0x55f6375f8580 2024-04-21T08:41:00.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.516+0000 7f43c25c4740 1 -- 172.21.15.86:0/2678027276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6375f8580 msgr2=0x55f6375f8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:00.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.516+0000 7f43c25c4740 1 --2- 172.21.15.86:0/2678027276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6375f8580 0x55f6375f8960 secure :-1 s=READY pgs=832 cs=0 l=1 rev1=1 crypto rx=0x7f43a8012900 tx=0x7f43a8008f90 comp rx=0 tx=0).stop 2024-04-21T08:41:00.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.516+0000 7f43c25c4740 1 -- 172.21.15.86:0/2678027276 shutdown_connections 2024-04-21T08:41:00.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.516+0000 7f43c25c4740 1 --2- 172.21.15.86:0/2678027276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f637601a80 0x55f637603e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.516+0000 7f43c25c4740 1 --2- 172.21.15.86:0/2678027276 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f6375f8ea0 0x55f637601540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.516+0000 7f43c25c4740 1 --2- 172.21.15.86:0/2678027276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6375f8580 0x55f6375f8960 unknown :-1 s=CLOSED pgs=832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.516+0000 7f43c25c4740 1 -- 172.21.15.86:0/2678027276 >> 172.21.15.86:0/2678027276 conn(0x55f6375f2cb0 msgr2=0x55f6375f3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:00.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.516+0000 7f43c25c4740 1 -- 172.21.15.86:0/2678027276 shutdown_connections 2024-04-21T08:41:00.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.516+0000 7f43c25c4740 1 -- 172.21.15.86:0/2678027276 wait complete. 2024-04-21T08:41:00.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.516+0000 7f43c25c4740 1 -- start start 2024-04-21T08:41:00.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.516+0000 7f43c25c4740 1 Processor -- start 2024-04-21T08:41:00.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.516+0000 7f43c25c4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6375f8580 0x55f6375feae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.516+0000 7f43c25c4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6375f8ea0 0x55f6375ff020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.516+0000 7f43c25c4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f637601a80 0x55f6375ff560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.516+0000 7f43c25c4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f637604870 con 0x55f6375f8580 2024-04-21T08:41:00.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.516+0000 7f43c25c4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55f6375162f0 con 0x55f637601a80 2024-04-21T08:41:00.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.516+0000 7f43c25c4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f6376049f0 con 0x55f6375f8ea0 2024-04-21T08:41:00.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.516+0000 7f43c25c4740 10 client.0 _ll_register_callbacks cb 0x55f637523580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:00.523 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.517+0000 7f43bce8d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6375f8580 0x55f6375feae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:00.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.517+0000 7f43bce8d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6375f8580 0x55f6375feae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60278/0 (socket says 172.21.15.86:60278) 2024-04-21T08:41:00.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.517+0000 7f43bce8d640 1 -- 172.21.15.86:0/756673175 learned_addr learned my addr 172.21.15.86:0/756673175 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:00.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.517+0000 7f43bd68e640 1 --2- 172.21.15.86:0/756673175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6375f8ea0 0x55f6375ff020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:00.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.517+0000 7f43bce8d640 1 --2- 172.21.15.86:0/756673175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f637601a80 0x55f6375ff560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:00.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.517+0000 7f43bce8d640 1 -- 172.21.15.86:0/756673175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6375f8ea0 msgr2=0x55f6375ff020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:00.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.517+0000 7f43bce8d640 1 --2- 172.21.15.86:0/756673175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6375f8ea0 0x55f6375ff020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.517+0000 7f43bce8d640 1 -- 172.21.15.86:0/756673175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f637601a80 msgr2=0x55f6375ff560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:00.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.517+0000 7f43bce8d640 1 --2- 172.21.15.86:0/756673175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f637601a80 0x55f6375ff560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.517+0000 7f43bce8d640 1 -- 172.21.15.86:0/756673175 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f63726f3f0 con 0x55f6375f8580 2024-04-21T08:41:00.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.517+0000 7f43bce8d640 1 --2- 172.21.15.86:0/756673175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6375f8580 0x55f6375feae0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f43a8010ff0 tx=0x7f43a80083f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff5af3f2db23b54f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:00.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.517+0000 7f43a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:00.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.517+0000 7f43a0ff9640 1 -- 172.21.15.86:0/756673175 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a8005540 con 0x55f6375f8580 2024-04-21T08:41:00.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.517+0000 7f43a0ff9640 1 -- 172.21.15.86:0/756673175 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f43a80046c0 con 0x55f6375f8580 2024-04-21T08:41:00.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.517+0000 7f43c25c4740 1 -- 172.21.15.86:0/756673175 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43a8010d20 con 0x55f6375f8580 2024-04-21T08:41:00.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.517+0000 7f43a0ff9640 1 -- 172.21.15.86:0/756673175 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f43a80056e0 con 0x55f6375f8580 2024-04-21T08:41:00.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.517+0000 7f43c25c4740 10 client.16662 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:00.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.517+0000 7f43c25c4740 1 -- 172.21.15.86:0/756673175 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f63722f0f0 con 0x55f6375f8580 2024-04-21T08:41:00.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.517+0000 7f43a0ff9640 1 -- 172.21.15.86:0/756673175 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43a8010d20 con 0x55f6375f8580 2024-04-21T08:41:00.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.518+0000 7f43a0ff9640 1 -- 172.21.15.86:0/756673175 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f43a802b020 con 0x55f6375f8580 2024-04-21T08:41:00.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.518+0000 7f43c25c4740 10 client.16662 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:00.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.518+0000 7f43c25c4740 10 client.16662 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:00.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.518+0000 7f43c25c4740 20 client.16662 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:00.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.518+0000 7f43c25c4740 1 --2- 172.21.15.86:0/756673175 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f637611a40 0x55f637613e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.518+0000 7f43c25c4740 4 client.16662 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:00.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.518+0000 7f43c25c4740 1 -- 172.21.15.86:0/756673175 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f63756ac00 con 0x55f637611a40 2024-04-21T08:41:00.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.518+0000 7f43bd68e640 1 --2- 172.21.15.86:0/756673175 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f637611a40 0x55f637613e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:00.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.518+0000 7f43a0ff9640 10 client.16662 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:00.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.518+0000 7f43bd68e640 1 --2- 172.21.15.86:0/756673175 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f637611a40 0x55f637613e30 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bdc9172ec324fa9 server_cookie=f5534eac8b5ad124 in_seq=0 out_seq=0 2024-04-21T08:41:00.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.519+0000 7f43a0ff9640 1 -- 172.21.15.86:0/756673175 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f63756ac00 con 0x55f637611a40 2024-04-21T08:41:00.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.519+0000 7f43a0ff9640 10 client.16662 handle_command_reply: tid=0 2024-04-21T08:41:00.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.519+0000 7f43c25c4740 1 client.16662 shutdown 2024-04-21T08:41:00.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.519+0000 7f43c25c4740 1 -- 172.21.15.86:0/756673175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6375f8580 msgr2=0x55f6375feae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:00.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.519+0000 7f43c25c4740 1 --2- 172.21.15.86:0/756673175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6375f8580 0x55f6375feae0 secure :-1 s=READY pgs=833 cs=0 l=1 rev1=1 crypto rx=0x7f43a8010ff0 tx=0x7f43a80083f0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.519+0000 7f43c25c4740 1 -- 172.21.15.86:0/756673175 shutdown_connections 2024-04-21T08:41:00.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.519+0000 7f43c25c4740 1 --2- 172.21.15.86:0/756673175 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f637611a40 0x55f637613e30 crc :-1 s=READY pgs=93 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.519+0000 7f43c25c4740 1 --2- 172.21.15.86:0/756673175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f637601a80 0x55f6375ff560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.519+0000 7f43c25c4740 1 --2- 172.21.15.86:0/756673175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6375f8ea0 0x55f6375ff020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.519+0000 7f43c25c4740 1 --2- 172.21.15.86:0/756673175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6375f8580 0x55f6375feae0 unknown :-1 s=CLOSED pgs=833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.519+0000 7f43a0ff9640 0 client.16662 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:00.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.519+0000 7f43c25c4740 1 -- 172.21.15.86:0/756673175 >> 172.21.15.86:0/756673175 conn(0x55f6375f2cb0 msgr2=0x55f6375f4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:00.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.519+0000 7f43c25c4740 1 -- 172.21.15.86:0/756673175 shutdown_connections 2024-04-21T08:41:00.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.519+0000 7f43c25c4740 1 -- 172.21.15.86:0/756673175 wait complete. 2024-04-21T08:41:00.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.519+0000 7f43c25c4740 20 client.16662 trim_cache size 0 max 16384 2024-04-21T08:41:00.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.536+0000 7f43c25c4740 1 Processor -- start 2024-04-21T08:41:00.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.536+0000 7f43c25c4740 1 -- start start 2024-04-21T08:41:00.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.537+0000 7f43c25c4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6375f8580 0x55f637601210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.537+0000 7f43c25c4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f6375f8ea0 0x55f63760d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.537+0000 7f43c25c4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f637601a80 0x55f6375f1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.537+0000 7f43c25c4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f43b81af3b0 con 0x55f6375f8580 2024-04-21T08:41:00.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.537+0000 7f43c25c4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f43b81b0ef0 con 0x55f6375f8ea0 2024-04-21T08:41:00.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.537+0000 7f43c25c4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f43b81afa80 con 0x55f637601a80 2024-04-21T08:41:00.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.537+0000 7f43bce8d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6375f8580 0x55f637601210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:00.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.537+0000 7f43bce8d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6375f8580 0x55f637601210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60290/0 (socket says 172.21.15.86:60290) 2024-04-21T08:41:00.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.537+0000 7f43bce8d640 1 -- 172.21.15.86:0/889460146 learned_addr learned my addr 172.21.15.86:0/889460146 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:00.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.537+0000 7f43bd68e640 1 --2- 172.21.15.86:0/889460146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f6375f8ea0 0x55f63760d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:00.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.537+0000 7f43bce8d640 1 -- 172.21.15.86:0/889460146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f637601a80 msgr2=0x55f6375f1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:00.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.537+0000 7f43bce8d640 1 --2- 172.21.15.86:0/889460146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f637601a80 0x55f6375f1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.537+0000 7f43bce8d640 1 -- 172.21.15.86:0/889460146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f6375f8ea0 msgr2=0x55f63760d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:00.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.537+0000 7f43bce8d640 1 --2- 172.21.15.86:0/889460146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f6375f8ea0 0x55f63760d730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.537+0000 7f43bce8d640 1 -- 172.21.15.86:0/889460146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f637604750 con 0x55f6375f8580 2024-04-21T08:41:00.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.537+0000 7f43bce8d640 1 --2- 172.21.15.86:0/889460146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6375f8580 0x55f637601210 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f43a8005b90 tx=0x7f43a802da20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d08cc93553abf060 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:00.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.537+0000 7f43a2ffd640 1 -- 172.21.15.86:0/889460146 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a803f070 con 0x55f6375f8580 2024-04-21T08:41:00.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.537+0000 7f43a2ffd640 1 -- 172.21.15.86:0/889460146 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43a8022070 con 0x55f6375f8580 2024-04-21T08:41:00.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.537+0000 7f43a2ffd640 1 -- 172.21.15.86:0/889460146 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43a80052a0 con 0x55f6375f8580 2024-04-21T08:41:00.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.538+0000 7f43c25c4740 1 -- 172.21.15.86:0/889460146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6375f8580 msgr2=0x55f637601210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:00.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.538+0000 7f43c25c4740 1 --2- 172.21.15.86:0/889460146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6375f8580 0x55f637601210 secure :-1 s=READY pgs=834 cs=0 l=1 rev1=1 crypto rx=0x7f43a8005b90 tx=0x7f43a802da20 comp rx=0 tx=0).stop 2024-04-21T08:41:00.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.538+0000 7f43c25c4740 1 -- 172.21.15.86:0/889460146 shutdown_connections 2024-04-21T08:41:00.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.538+0000 7f43c25c4740 1 --2- 172.21.15.86:0/889460146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f637601a80 0x55f6375f1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.538+0000 7f43c25c4740 1 --2- 172.21.15.86:0/889460146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f6375f8ea0 0x55f63760d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.538+0000 7f43c25c4740 1 --2- 172.21.15.86:0/889460146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6375f8580 0x55f637601210 unknown :-1 s=CLOSED pgs=834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.538+0000 7f43c25c4740 1 -- 172.21.15.86:0/889460146 >> 172.21.15.86:0/889460146 conn(0x55f6375f2cb0 msgr2=0x55f6375f14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:00.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.538+0000 7f43c25c4740 1 -- 172.21.15.86:0/889460146 shutdown_connections 2024-04-21T08:41:00.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.538+0000 7f43c25c4740 1 -- 172.21.15.86:0/889460146 wait complete. 2024-04-21T08:41:00.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.538+0000 7f43c25c4740 1 -- start start 2024-04-21T08:41:00.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.538+0000 7f43c25c4740 1 Processor -- start 2024-04-21T08:41:00.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.539+0000 7f43c25c4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6375f8580 0x55f637609d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.539+0000 7f43c25c4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6375f8ea0 0x55f637605200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.539+0000 7f43c25c4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f637601a80 0x55f637605740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.539+0000 7f43c25c4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f43b81b0820 con 0x55f6375f8ea0 2024-04-21T08:41:00.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.539+0000 7f43c25c4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f43b81b0150 con 0x55f637601a80 2024-04-21T08:41:00.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.539+0000 7f43c25c4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f43b81ad370 con 0x55f6375f8580 2024-04-21T08:41:00.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.539+0000 7f43c25c4740 10 client.0 _ll_register_callbacks cb 0x7f43b4009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:00.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.539+0000 7f43bd68e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6375f8ea0 0x55f637605200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:00.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.539+0000 7f43bd68e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6375f8ea0 0x55f637605200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60294/0 (socket says 172.21.15.86:60294) 2024-04-21T08:41:00.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.539+0000 7f43bd68e640 1 -- 172.21.15.86:0/1330359228 learned_addr learned my addr 172.21.15.86:0/1330359228 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:00.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.539+0000 7f43bce8d640 1 --2- 172.21.15.86:0/1330359228 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6375f8580 0x55f637609d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:00.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.539+0000 7f43bd68e640 1 -- 172.21.15.86:0/1330359228 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6375f8580 msgr2=0x55f637609d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:00.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.539+0000 7f43bd68e640 1 --2- 172.21.15.86:0/1330359228 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6375f8580 0x55f637609d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.539+0000 7f43bd68e640 1 -- 172.21.15.86:0/1330359228 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f637601a80 msgr2=0x55f637605740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:00.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.539+0000 7f43bd68e640 1 --2- 172.21.15.86:0/1330359228 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f637601a80 0x55f637605740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.539+0000 7f43bd68e640 1 -- 172.21.15.86:0/1330359228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f637605c80 con 0x55f6375f8ea0 2024-04-21T08:41:00.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.539+0000 7f43bd68e640 1 --2- 172.21.15.86:0/1330359228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6375f8ea0 0x55f637605200 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f43b4009340 tx=0x7f43b4004520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3dc51494b480f4bb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:00.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.539+0000 7f43a0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:00.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.539+0000 7f43a0ff9640 1 -- 172.21.15.86:0/1330359228 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f43b402c9e0 con 0x55f6375f8ea0 2024-04-21T08:41:00.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.539+0000 7f43c25c4740 1 -- 172.21.15.86:0/1330359228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f43a8022070 con 0x55f6375f8ea0 2024-04-21T08:41:00.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.539+0000 7f43a0ff9640 1 -- 172.21.15.86:0/1330359228 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f43b402cb80 con 0x55f6375f8ea0 2024-04-21T08:41:00.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.540+0000 7f43a0ff9640 1 -- 172.21.15.86:0/1330359228 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f43b4058050 con 0x55f6375f8ea0 2024-04-21T08:41:00.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.540+0000 7f43c25c4740 10 client.16668 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:00.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.540+0000 7f43c25c4740 1 -- 172.21.15.86:0/1330359228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f637677b50 con 0x55f6375f8ea0 2024-04-21T08:41:00.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.540+0000 7f43a0ff9640 1 -- 172.21.15.86:0/1330359228 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f43a8022070 con 0x55f6375f8ea0 2024-04-21T08:41:00.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.540+0000 7f43a0ff9640 1 -- 172.21.15.86:0/1330359228 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f43b404a6b0 con 0x55f6375f8ea0 2024-04-21T08:41:00.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.540+0000 7f43c25c4740 10 client.16668 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:00.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.540+0000 7f43c25c4740 10 client.16668 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:00.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.540+0000 7f43c25c4740 20 client.16668 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:00.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.540+0000 7f43c25c4740 1 --2- 172.21.15.86:0/1330359228 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f637678320 0x55f63767a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.540+0000 7f43c25c4740 4 client.16668 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:41:00.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.540+0000 7f43c25c4740 1 -- 172.21.15.86:0/1330359228 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f637564a70 con 0x55f637678320 2024-04-21T08:41:00.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.540+0000 7f43bce8d640 1 --2- 172.21.15.86:0/1330359228 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f637678320 0x55f63767a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:00.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.541+0000 7f43bce8d640 1 --2- 172.21.15.86:0/1330359228 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f637678320 0x55f63767a740 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a55c640193e463be server_cookie=a719d046030cbed4 in_seq=0 out_seq=0 2024-04-21T08:41:00.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.541+0000 7f43a0ff9640 10 client.16668 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.541+0000 7f43a0ff9640 1 -- 172.21.15.86:0/1330359228 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55f637564a70 con 0x55f637678320 2024-04-21T08:41:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.541+0000 7f43a0ff9640 10 client.16668 handle_command_reply: tid=0 2024-04-21T08:41:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.541+0000 7f43c25c4740 1 client.16668 shutdown 2024-04-21T08:41:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.541+0000 7f43c25c4740 1 -- 172.21.15.86:0/1330359228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6375f8ea0 msgr2=0x55f637605200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.541+0000 7f43c25c4740 1 --2- 172.21.15.86:0/1330359228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6375f8ea0 0x55f637605200 secure :-1 s=READY pgs=835 cs=0 l=1 rev1=1 crypto rx=0x7f43b4009340 tx=0x7f43b4004520 comp rx=0 tx=0).stop 2024-04-21T08:41:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.541+0000 7f43c25c4740 1 -- 172.21.15.86:0/1330359228 shutdown_connections 2024-04-21T08:41:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.541+0000 7f43c25c4740 1 --2- 172.21.15.86:0/1330359228 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f637678320 0x55f63767a740 crc :-1 s=READY pgs=94 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.541+0000 7f43c25c4740 1 --2- 172.21.15.86:0/1330359228 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f637601a80 0x55f637605740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.541+0000 7f43c25c4740 1 --2- 172.21.15.86:0/1330359228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f6375f8ea0 0x55f637605200 unknown :-1 s=CLOSED pgs=835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.541+0000 7f43a0ff9640 0 client.16668 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.541+0000 7f43c25c4740 1 --2- 172.21.15.86:0/1330359228 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f6375f8580 0x55f637609d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.541+0000 7f43c25c4740 1 -- 172.21.15.86:0/1330359228 >> 172.21.15.86:0/1330359228 conn(0x55f6375f2cb0 msgr2=0x55f6375ff330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.541+0000 7f43c25c4740 1 -- 172.21.15.86:0/1330359228 shutdown_connections 2024-04-21T08:41:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.541+0000 7f43c25c4740 1 -- 172.21.15.86:0/1330359228 wait complete. 2024-04-21T08:41:00.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.541+0000 7f43c25c4740 20 client.16668 trim_cache size 0 max 16384 2024-04-21T08:41:00.548 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:41:00.548 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-21T08:41:00.548 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-21T08:41:00.548 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:41:00.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.541+0000 7f43bf8f2640 1 -- 172.21.15.86:0/1931527568 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f43a40a3d20 msgr2=0x7f43a40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:00.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.541+0000 7f43bf8f2640 1 --2- 172.21.15.86:0/1931527568 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f43a40a3d20 0x7f43a40a61e0 secure :-1 s=READY pgs=119 cs=0 l=1 rev1=1 crypto rx=0x7f43b009c5e0 tx=0x7f43b006e320 comp rx=0 tx=0).stop 2024-04-21T08:41:00.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.541+0000 7f43bf8f2640 1 -- 172.21.15.86:0/1931527568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43b8106660 msgr2=0x7f43b81a45b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:00.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.541+0000 7f43bf8f2640 1 --2- 172.21.15.86:0/1931527568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43b8106660 0x7f43b81a45b0 secure :-1 s=READY pgs=831 cs=0 l=1 rev1=1 crypto rx=0x7f43b000cad0 tx=0x7f43b000cf90 comp rx=0 tx=0).stop 2024-04-21T08:41:00.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.542+0000 7f43bf8f2640 1 -- 172.21.15.86:0/1931527568 shutdown_connections 2024-04-21T08:41:00.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.542+0000 7f43bf8f2640 1 --2- 172.21.15.86:0/1931527568 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f43a40a3d20 0x7f43a40a61e0 unknown :-1 s=CLOSED pgs=119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.542+0000 7f43bf8f2640 1 --2- 172.21.15.86:0/1931527568 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f43b8106660 0x7f43b81a45b0 unknown :-1 s=CLOSED pgs=831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.542+0000 7f43bf8f2640 1 --2- 172.21.15.86:0/1931527568 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f43b8100f80 0x7f43b8199a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.542+0000 7f43bf8f2640 1 --2- 172.21.15.86:0/1931527568 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f43b81005c0 0x7f43b8199530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.542+0000 7f43bf8f2640 1 -- 172.21.15.86:0/1931527568 >> 172.21.15.86:0/1931527568 conn(0x7f43b80773a0 msgr2=0x7f43b80feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:00.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.542+0000 7f43bf8f2640 1 -- 172.21.15.86:0/1931527568 shutdown_connections 2024-04-21T08:41:00.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.542+0000 7f43bf8f2640 1 -- 172.21.15.86:0/1931527568 wait complete. 2024-04-21T08:41:00.550 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:00 smithi086 ceph-mon[27939]: pgmap v69: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 22 op/s 2024-04-21T08:41:00.551 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:41:00.552 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:41:00.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.651+0000 7efd36a4a640 1 -- 172.21.15.86:0/1757901363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd30106f50 msgr2=0x7efd30107330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:00.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.651+0000 7efd36a4a640 1 --2- 172.21.15.86:0/1757901363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd30106f50 0x7efd30107330 secure :-1 s=READY pgs=836 cs=0 l=1 rev1=1 crypto rx=0x7efd24009900 tx=0x7efd2402bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:00.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.651+0000 7efd36a4a640 1 -- 172.21.15.86:0/1757901363 shutdown_connections 2024-04-21T08:41:00.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.651+0000 7efd36a4a640 1 --2- 172.21.15.86:0/1757901363 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd30069aa0 0x7efd3010ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.651+0000 7efd36a4a640 1 --2- 172.21.15.86:0/1757901363 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd300690e0 0x7efd30069560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.651+0000 7efd36a4a640 1 --2- 172.21.15.86:0/1757901363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd30106f50 0x7efd30107330 unknown :-1 s=CLOSED pgs=836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.651+0000 7efd36a4a640 1 -- 172.21.15.86:0/1757901363 >> 172.21.15.86:0/1757901363 conn(0x7efd30077e30 msgr2=0x7efd30078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:00.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.651+0000 7efd36a4a640 1 -- 172.21.15.86:0/1757901363 shutdown_connections 2024-04-21T08:41:00.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.651+0000 7efd36a4a640 1 -- 172.21.15.86:0/1757901363 wait complete. 2024-04-21T08:41:00.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.651+0000 7efd36a4a640 1 Processor -- start 2024-04-21T08:41:00.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.651+0000 7efd36a4a640 1 -- start start 2024-04-21T08:41:00.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.652+0000 7efd36a4a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd300690e0 0x7efd30198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.652+0000 7efd36a4a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd30069aa0 0x7efd30198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.652+0000 7efd36a4a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd30106f50 0x7efd3019cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.652+0000 7efd36a4a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efd30112170 con 0x7efd300690e0 2024-04-21T08:41:00.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.652+0000 7efd36a4a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efd30111ff0 con 0x7efd30106f50 2024-04-21T08:41:00.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.652+0000 7efd36a4a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7efd301122f0 con 0x7efd30069aa0 2024-04-21T08:41:00.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.652+0000 7efd2ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd300690e0 0x7efd30198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:00.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.652+0000 7efd2ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd300690e0 0x7efd30198650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60322/0 (socket says 172.21.15.86:60322) 2024-04-21T08:41:00.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.652+0000 7efd2ffff640 1 -- 172.21.15.86:0/950526153 learned_addr learned my addr 172.21.15.86:0/950526153 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:00.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.652+0000 7efd2f7fe640 1 --2- 172.21.15.86:0/950526153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd30069aa0 0x7efd30198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:00.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.652+0000 7efd34fe7640 1 --2- 172.21.15.86:0/950526153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd30106f50 0x7efd3019cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:00.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.652+0000 7efd2ffff640 1 -- 172.21.15.86:0/950526153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd30069aa0 msgr2=0x7efd30198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:00.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.652+0000 7efd2ffff640 1 --2- 172.21.15.86:0/950526153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd30069aa0 0x7efd30198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.652+0000 7efd2ffff640 1 -- 172.21.15.86:0/950526153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd30106f50 msgr2=0x7efd3019cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:00.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.652+0000 7efd2ffff640 1 --2- 172.21.15.86:0/950526153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd30106f50 0x7efd3019cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.652+0000 7efd2ffff640 1 -- 172.21.15.86:0/950526153 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd3019d640 con 0x7efd300690e0 2024-04-21T08:41:00.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.652+0000 7efd34fe7640 1 --2- 172.21.15.86:0/950526153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd30106f50 0x7efd3019cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:00.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.652+0000 7efd2ffff640 1 --2- 172.21.15.86:0/950526153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd300690e0 0x7efd30198650 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7efd24005940 tx=0x7efd240042f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:00.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.652+0000 7efd2d7fa640 1 -- 172.21.15.86:0/950526153 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd24043020 con 0x7efd300690e0 2024-04-21T08:41:00.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.652+0000 7efd2d7fa640 1 -- 172.21.15.86:0/950526153 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd240054c0 con 0x7efd300690e0 2024-04-21T08:41:00.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.652+0000 7efd36a4a640 1 -- 172.21.15.86:0/950526153 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd3019d8d0 con 0x7efd300690e0 2024-04-21T08:41:00.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.652+0000 7efd2d7fa640 1 -- 172.21.15.86:0/950526153 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd24003d40 con 0x7efd300690e0 2024-04-21T08:41:00.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.652+0000 7efd36a4a640 1 -- 172.21.15.86:0/950526153 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd301a51c0 con 0x7efd300690e0 2024-04-21T08:41:00.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.653+0000 7efd36a4a640 1 -- 172.21.15.86:0/950526153 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd30068de0 con 0x7efd300690e0 2024-04-21T08:41:00.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.654+0000 7efd2d7fa640 1 -- 172.21.15.86:0/950526153 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efd24046020 con 0x7efd300690e0 2024-04-21T08:41:00.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.655+0000 7efd2d7fa640 1 --2- 172.21.15.86:0/950526153 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efd180a3d20 0x7efd180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:00.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.655+0000 7efd2d7fa640 1 -- 172.21.15.86:0/950526153 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7efd240f2a80 con 0x7efd300690e0 2024-04-21T08:41:00.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.655+0000 7efd2f7fe640 1 --2- 172.21.15.86:0/950526153 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efd180a3d20 0x7efd180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:00.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.655+0000 7efd2f7fe640 1 --2- 172.21.15.86:0/950526153 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efd180a3d20 0x7efd180a61e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efd20005fd0 tx=0x7efd20005e20 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:00.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.660+0000 7efd2d7fa640 1 -- 172.21.15.86:0/950526153 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efd240beec0 con 0x7efd300690e0 2024-04-21T08:41:00.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:00 smithi088 ceph-mon[31259]: pgmap v69: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 22 op/s 2024-04-21T08:41:00.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.879+0000 7efd36a4a640 1 -- 172.21.15.86:0/950526153 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd301999a0 con 0x7efd300690e0 2024-04-21T08:41:00.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.880+0000 7efd2d7fa640 1 -- 172.21.15.86:0/950526153 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7efd240be7c0 con 0x7efd300690e0 2024-04-21T08:41:00.880 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:41:00.880 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:38:45.202126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24398,24394,24455,24472]},"id":1}]} 2024-04-21T08:41:00.880 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-21T08:41:00.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.882+0000 7efd36a4a640 1 -- 172.21.15.86:0/950526153 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efd180a3d20 msgr2=0x7efd180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:00.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.882+0000 7efd36a4a640 1 --2- 172.21.15.86:0/950526153 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efd180a3d20 0x7efd180a61e0 secure :-1 s=READY pgs=120 cs=0 l=1 rev1=1 crypto rx=0x7efd20005fd0 tx=0x7efd20005e20 comp rx=0 tx=0).stop 2024-04-21T08:41:00.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.882+0000 7efd36a4a640 1 -- 172.21.15.86:0/950526153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd300690e0 msgr2=0x7efd30198650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:00.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.882+0000 7efd36a4a640 1 --2- 172.21.15.86:0/950526153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd300690e0 0x7efd30198650 secure :-1 s=READY pgs=837 cs=0 l=1 rev1=1 crypto rx=0x7efd24005940 tx=0x7efd240042f0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.882+0000 7efd36a4a640 1 -- 172.21.15.86:0/950526153 shutdown_connections 2024-04-21T08:41:00.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.882+0000 7efd36a4a640 1 --2- 172.21.15.86:0/950526153 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efd180a3d20 0x7efd180a61e0 unknown :-1 s=CLOSED pgs=120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.882+0000 7efd36a4a640 1 --2- 172.21.15.86:0/950526153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd30106f50 0x7efd3019cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.882+0000 7efd36a4a640 1 --2- 172.21.15.86:0/950526153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd30069aa0 0x7efd30198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.882+0000 7efd36a4a640 1 --2- 172.21.15.86:0/950526153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd300690e0 0x7efd30198650 unknown :-1 s=CLOSED pgs=837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:00.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.882+0000 7efd36a4a640 1 -- 172.21.15.86:0/950526153 >> 172.21.15.86:0/950526153 conn(0x7efd30077e30 msgr2=0x7efd300fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:00.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.882+0000 7efd36a4a640 1 -- 172.21.15.86:0/950526153 shutdown_connections 2024-04-21T08:41:00.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.882+0000 7efd36a4a640 1 -- 172.21.15.86:0/950526153 wait complete. 2024-04-21T08:41:00.893 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:41:01.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.996+0000 7ff0d9493640 1 -- 172.21.15.86:0/1773512318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0d4103af0 msgr2=0x7ff0d410bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.996+0000 7ff0d9493640 1 --2- 172.21.15.86:0/1773512318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0d4103af0 0x7ff0d410bfd0 secure :-1 s=READY pgs=838 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8009900 tx=0x7ff0c802bd60 comp rx=0 tx=0).stop 2024-04-21T08:41:01.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.996+0000 7ff0d9493640 1 -- 172.21.15.86:0/1773512318 shutdown_connections 2024-04-21T08:41:01.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.996+0000 7ff0d9493640 1 --2- 172.21.15.86:0/1773512318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0d410c510 0x7ff0d410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.996+0000 7ff0d9493640 1 --2- 172.21.15.86:0/1773512318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0d4103af0 0x7ff0d410bfd0 unknown :-1 s=CLOSED pgs=838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.996+0000 7ff0d9493640 1 --2- 172.21.15.86:0/1773512318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0d41031d0 0x7ff0d41035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.996+0000 7ff0d9493640 1 -- 172.21.15.86:0/1773512318 >> 172.21.15.86:0/1773512318 conn(0x7ff0d4077f50 msgr2=0x7ff0d4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:01.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.996+0000 7ff0d9493640 1 -- 172.21.15.86:0/1773512318 shutdown_connections 2024-04-21T08:41:01.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.996+0000 7ff0d9493640 1 -- 172.21.15.86:0/1773512318 wait complete. 2024-04-21T08:41:01.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.996+0000 7ff0d9493640 1 Processor -- start 2024-04-21T08:41:01.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.996+0000 7ff0d9493640 1 -- start start 2024-04-21T08:41:01.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.997+0000 7ff0d9493640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0d41031d0 0x7ff0d419e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.997+0000 7ff0d9493640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0d4103af0 0x7ff0d419ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.997+0000 7ff0d9493640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0d410c510 0x7ff0d4198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.997+0000 7ff0d9493640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d4112200 con 0x7ff0d4103af0 2024-04-21T08:41:01.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.997+0000 7ff0d9493640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d4112080 con 0x7ff0d41031d0 2024-04-21T08:41:01.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.997+0000 7ff0d9493640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d4112380 con 0x7ff0d410c510 2024-04-21T08:41:01.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.997+0000 7ff0d27fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0d4103af0 0x7ff0d419ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.997+0000 7ff0d27fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0d4103af0 0x7ff0d419ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60342/0 (socket says 172.21.15.86:60342) 2024-04-21T08:41:01.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.997+0000 7ff0d27fc640 1 -- 172.21.15.86:0/2464434382 learned_addr learned my addr 172.21.15.86:0/2464434382 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:01.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.997+0000 7ff0d2ffd640 1 --2- 172.21.15.86:0/2464434382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0d41031d0 0x7ff0d419e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.997+0000 7ff0d27fc640 1 -- 172.21.15.86:0/2464434382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0d410c510 msgr2=0x7ff0d4198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.997+0000 7ff0d27fc640 1 --2- 172.21.15.86:0/2464434382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0d410c510 0x7ff0d4198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.997+0000 7ff0d27fc640 1 -- 172.21.15.86:0/2464434382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0d41031d0 msgr2=0x7ff0d419e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.997+0000 7ff0d27fc640 1 --2- 172.21.15.86:0/2464434382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0d41031d0 0x7ff0d419e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.997+0000 7ff0d27fc640 1 -- 172.21.15.86:0/2464434382 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff0d4198f10 con 0x7ff0d4103af0 2024-04-21T08:41:01.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.997+0000 7ff0d2ffd640 1 --2- 172.21.15.86:0/2464434382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0d41031d0 0x7ff0d419e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:01.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.997+0000 7ff0d27fc640 1 --2- 172.21.15.86:0/2464434382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0d4103af0 0x7ff0d419ead0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8005940 tx=0x7ff0c802d920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:01.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.997+0000 7ff0bbfff640 1 -- 172.21.15.86:0/2464434382 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8043020 con 0x7ff0d4103af0 2024-04-21T08:41:01.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.997+0000 7ff0bbfff640 1 -- 172.21.15.86:0/2464434382 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0c802dd20 con 0x7ff0d4103af0 2024-04-21T08:41:01.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.997+0000 7ff0d9493640 1 -- 172.21.15.86:0/2464434382 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff0d41991a0 con 0x7ff0d4103af0 2024-04-21T08:41:01.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.997+0000 7ff0bbfff640 1 -- 172.21.15.86:0/2464434382 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c8003b80 con 0x7ff0d4103af0 2024-04-21T08:41:01.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.997+0000 7ff0d9493640 1 -- 172.21.15.86:0/2464434382 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff0d41a5510 con 0x7ff0d4103af0 2024-04-21T08:41:01.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:00.999+0000 7ff0bbfff640 1 -- 172.21.15.86:0/2464434382 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff0c8046020 con 0x7ff0d4103af0 2024-04-21T08:41:01.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.000+0000 7ff0dc167740 1 Processor -- start 2024-04-21T08:41:01.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.000+0000 7ff0dc167740 1 -- start start 2024-04-21T08:41:01.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.000+0000 7ff0dc167740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55993c9f14d0 0x55993c9f18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.000+0000 7ff0dc167740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55993c9f1e80 0x55993c9fa430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.000+0000 7ff0dc167740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55993c9faa00 0x55993c9fce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.000+0000 7ff0dc167740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55993c9561f0 con 0x55993c9f14d0 2024-04-21T08:41:01.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.000+0000 7ff0dc167740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55993c8f34b0 con 0x55993c9faa00 2024-04-21T08:41:01.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.000+0000 7ff0dc167740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55993c9521b0 con 0x55993c9f1e80 2024-04-21T08:41:01.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.000+0000 7ff0bbfff640 1 --2- 172.21.15.86:0/2464434382 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff0bc0a3d20 0x7ff0bc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.000+0000 7ff0d37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55993c9f14d0 0x55993c9f18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.000+0000 7ff0d37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55993c9f14d0 0x55993c9f18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60356/0 (socket says 172.21.15.86:60356) 2024-04-21T08:41:01.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.000+0000 7ff0d37fe640 1 -- 172.21.15.86:0/2940489681 learned_addr learned my addr 172.21.15.86:0/2940489681 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:01.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.000+0000 7ff0d37fe640 1 --2- 172.21.15.86:0/2464434382 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff0bc0a3d20 0x7ff0bc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.000+0000 7ff0d2ffd640 1 --2- 172.21.15.86:0/2940489681 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55993c9f1e80 0x55993c9fa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.000+0000 7ff0bbfff640 1 -- 172.21.15.86:0/2464434382 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7ff0c80f2d20 con 0x7ff0d4103af0 2024-04-21T08:41:01.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.000+0000 7ff0d27fc640 1 --2- 172.21.15.86:0/2940489681 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55993c9faa00 0x55993c9fce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.000+0000 7ff0d37fe640 1 -- 172.21.15.86:0/2940489681 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55993c9f1e80 msgr2=0x55993c9fa430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.000+0000 7ff0d37fe640 1 --2- 172.21.15.86:0/2940489681 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55993c9f1e80 0x55993c9fa430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.000+0000 7ff0d37fe640 1 -- 172.21.15.86:0/2940489681 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55993c9faa00 msgr2=0x55993c9fce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.000+0000 7ff0d37fe640 1 --2- 172.21.15.86:0/2940489681 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55993c9faa00 0x55993c9fce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.000+0000 7ff0d37fe640 1 -- 172.21.15.86:0/2940489681 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55993c8e51b0 con 0x55993c9f14d0 2024-04-21T08:41:01.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.000+0000 7ff0d2ffd640 1 --2- 172.21.15.86:0/2940489681 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55993c9f1e80 0x55993c9fa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:01.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.000+0000 7ff0d27fc640 1 --2- 172.21.15.86:0/2940489681 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55993c9faa00 0x55993c9fce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:01.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.001+0000 7ff0d37fe640 1 --2- 172.21.15.86:0/2940489681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55993c9f14d0 0x55993c9f18b0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc012a20 tx=0x7ff0cc012ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef9ef1351963a03f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:01.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.001+0000 7ff0d37fe640 1 --2- 172.21.15.86:0/2464434382 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff0bc0a3d20 0x7ff0bc0a61e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00f7f0 tx=0x7ff0cc014000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:01.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.001+0000 7ff0b97fa640 1 -- 172.21.15.86:0/2940489681 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc020070 con 0x55993c9f14d0 2024-04-21T08:41:01.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.001+0000 7ff0b97fa640 1 -- 172.21.15.86:0/2940489681 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0cc00d830 con 0x55993c9f14d0 2024-04-21T08:41:01.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.001+0000 7ff0b97fa640 1 -- 172.21.15.86:0/2940489681 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0cc0047a0 con 0x55993c9f14d0 2024-04-21T08:41:01.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.001+0000 7ff0dc167740 1 -- 172.21.15.86:0/2940489681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55993c9f14d0 msgr2=0x55993c9f18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.001+0000 7ff0dc167740 1 --2- 172.21.15.86:0/2940489681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55993c9f14d0 0x55993c9f18b0 secure :-1 s=READY pgs=840 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc012a20 tx=0x7ff0cc012ee0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.001+0000 7ff0dc167740 1 -- 172.21.15.86:0/2940489681 shutdown_connections 2024-04-21T08:41:01.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.001+0000 7ff0dc167740 1 --2- 172.21.15.86:0/2940489681 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55993c9faa00 0x55993c9fce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.001+0000 7ff0dc167740 1 --2- 172.21.15.86:0/2940489681 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55993c9f1e80 0x55993c9fa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.001+0000 7ff0dc167740 1 --2- 172.21.15.86:0/2940489681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55993c9f14d0 0x55993c9f18b0 unknown :-1 s=CLOSED pgs=840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.001+0000 7ff0dc167740 1 -- 172.21.15.86:0/2940489681 >> 172.21.15.86:0/2940489681 conn(0x55993c9ebba0 msgr2=0x55993c9ebf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:01.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.001+0000 7ff0dc167740 1 -- 172.21.15.86:0/2940489681 shutdown_connections 2024-04-21T08:41:01.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.001+0000 7ff0dc167740 1 -- 172.21.15.86:0/2940489681 wait complete. 2024-04-21T08:41:01.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.002+0000 7ff0dc167740 1 -- start start 2024-04-21T08:41:01.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.002+0000 7ff0dc167740 1 Processor -- start 2024-04-21T08:41:01.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.002+0000 7ff0dc167740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55993c9f14d0 0x55993c9fa310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.002+0000 7ff0dc167740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55993c9f1e80 0x55993c9f76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.002+0000 7ff0dc167740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55993c9faa00 0x55993c9f7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.002+0000 7ff0dc167740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55993c9fd910 con 0x55993c9f1e80 2024-04-21T08:41:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.002+0000 7ff0dc167740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55993c90f500 con 0x55993c9faa00 2024-04-21T08:41:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.002+0000 7ff0dc167740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55993c9fda90 con 0x55993c9f14d0 2024-04-21T08:41:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.002+0000 7ff0dc167740 10 client.0 _ll_register_callbacks cb 0x55993c8ddb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.002+0000 7ff0d27fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55993c9f1e80 0x55993c9f76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.002+0000 7ff0d27fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55993c9f1e80 0x55993c9f76a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60370/0 (socket says 172.21.15.86:60370) 2024-04-21T08:41:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.002+0000 7ff0d27fc640 1 -- 172.21.15.86:0/573640673 learned_addr learned my addr 172.21.15.86:0/573640673 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.002+0000 7ff0d37fe640 1 --2- 172.21.15.86:0/573640673 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55993c9faa00 0x55993c9f7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.002+0000 7ff0d2ffd640 1 --2- 172.21.15.86:0/573640673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55993c9f14d0 0x55993c9fa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.002+0000 7ff0d27fc640 1 -- 172.21.15.86:0/573640673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55993c9f14d0 msgr2=0x55993c9fa310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.002+0000 7ff0d27fc640 1 --2- 172.21.15.86:0/573640673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55993c9f14d0 0x55993c9fa310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.002+0000 7ff0d27fc640 1 -- 172.21.15.86:0/573640673 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55993c9faa00 msgr2=0x55993c9f7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.002+0000 7ff0d27fc640 1 --2- 172.21.15.86:0/573640673 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55993c9faa00 0x55993c9f7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.002+0000 7ff0d27fc640 1 -- 172.21.15.86:0/573640673 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55993c6683f0 con 0x55993c9f1e80 2024-04-21T08:41:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.002+0000 7ff0d2ffd640 1 --2- 172.21.15.86:0/573640673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55993c9f14d0 0x55993c9fa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.003+0000 7ff0d27fc640 1 --2- 172.21.15.86:0/573640673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55993c9f1e80 0x55993c9f76a0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8005ec0 tx=0x7ff0c802bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2055be00b903d21c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.003+0000 7ff0b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.003+0000 7ff0b6ff5640 1 -- 172.21.15.86:0/573640673 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c80bf160 con 0x55993c9f1e80 2024-04-21T08:41:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.003+0000 7ff0b6ff5640 1 -- 172.21.15.86:0/573640673 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7ff0c80f0380 con 0x55993c9f1e80 2024-04-21T08:41:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.003+0000 7ff0dc167740 1 -- 172.21.15.86:0/573640673 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55993c8f5930 con 0x55993c9f1e80 2024-04-21T08:41:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.003+0000 7ff0b6ff5640 1 -- 172.21.15.86:0/573640673 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff0c80f0580 con 0x55993c9f1e80 2024-04-21T08:41:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.003+0000 7ff0dc167740 10 client.16686 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.003+0000 7ff0dc167740 1 -- 172.21.15.86:0/573640673 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55993c7327d0 con 0x55993c9f1e80 2024-04-21T08:41:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.003+0000 7ff0b6ff5640 1 -- 172.21.15.86:0/573640673 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55993c8f5930 con 0x55993c9f1e80 2024-04-21T08:41:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.003+0000 7ff0b6ff5640 1 -- 172.21.15.86:0/573640673 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff0c80c7240 con 0x55993c9f1e80 2024-04-21T08:41:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.003+0000 7ff0dc167740 10 client.16686 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.003+0000 7ff0dc167740 10 client.16686 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:41:01.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.003+0000 7ff0dc167740 20 client.16686 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:01.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.003+0000 7ff0dc167740 1 --2- 172.21.15.86:0/573640673 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55993ca0ac20 0x55993ca0d050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.003+0000 7ff0dc167740 4 client.16686 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:01.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.003+0000 7ff0dc167740 1 -- 172.21.15.86:0/573640673 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55993c963c10 con 0x55993ca0ac20 2024-04-21T08:41:01.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.004+0000 7ff0d2ffd640 1 --2- 172.21.15.86:0/573640673 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55993ca0ac20 0x55993ca0d050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.004+0000 7ff0d2ffd640 1 --2- 172.21.15.86:0/573640673 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55993ca0ac20 0x55993ca0d050 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1bdb81c960e112d server_cookie=1a868e4585b30d73 in_seq=0 out_seq=0 2024-04-21T08:41:01.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.004+0000 7ff0b6ff5640 10 client.16686 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:41:01.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.005+0000 7ff0b6ff5640 1 -- 172.21.15.86:0/573640673 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55993c963c10 con 0x55993ca0ac20 2024-04-21T08:41:01.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.005+0000 7ff0b6ff5640 10 client.16686 handle_command_reply: tid=0 2024-04-21T08:41:01.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.005+0000 7ff0dc167740 1 client.16686 shutdown 2024-04-21T08:41:01.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.005+0000 7ff0dc167740 1 -- 172.21.15.86:0/573640673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55993c9f1e80 msgr2=0x55993c9f76a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.005+0000 7ff0dc167740 1 --2- 172.21.15.86:0/573640673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55993c9f1e80 0x55993c9f76a0 secure :-1 s=READY pgs=841 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8005ec0 tx=0x7ff0c802bd60 comp rx=0 tx=0).stop 2024-04-21T08:41:01.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.005+0000 7ff0dc167740 1 -- 172.21.15.86:0/573640673 shutdown_connections 2024-04-21T08:41:01.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.005+0000 7ff0dc167740 1 --2- 172.21.15.86:0/573640673 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55993ca0ac20 0x55993ca0d050 crc :-1 s=READY pgs=40 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.005+0000 7ff0dc167740 1 --2- 172.21.15.86:0/573640673 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55993c9faa00 0x55993c9f7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.005+0000 7ff0b6ff5640 0 client.16686 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:41:01.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.005+0000 7ff0dc167740 1 --2- 172.21.15.86:0/573640673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55993c9f1e80 0x55993c9f76a0 unknown :-1 s=CLOSED pgs=841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.005+0000 7ff0dc167740 1 --2- 172.21.15.86:0/573640673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55993c9f14d0 0x55993c9fa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.005+0000 7ff0dc167740 1 -- 172.21.15.86:0/573640673 >> 172.21.15.86:0/573640673 conn(0x55993c9ebba0 msgr2=0x55993c9ed8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:01.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.005+0000 7ff0dc167740 1 -- 172.21.15.86:0/573640673 shutdown_connections 2024-04-21T08:41:01.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.005+0000 7ff0dc167740 1 -- 172.21.15.86:0/573640673 wait complete. 2024-04-21T08:41:01.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.005+0000 7ff0dc167740 20 client.16686 trim_cache size 0 max 16384 2024-04-21T08:41:01.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.018+0000 7ff0dc167740 1 Processor -- start 2024-04-21T08:41:01.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.018+0000 7ff0dc167740 1 -- start start 2024-04-21T08:41:01.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.018+0000 7ff0dc167740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55993c9f14d0 0x55993ca06800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.018+0000 7ff0dc167740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55993c9f1e80 0x55993c9f9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.018+0000 7ff0dc167740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55993c9faa00 0x55993c9fa310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.018+0000 7ff0dc167740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d41ad170 con 0x55993c9f14d0 2024-04-21T08:41:01.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.018+0000 7ff0dc167740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d41aecb0 con 0x55993c9f1e80 2024-04-21T08:41:01.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.018+0000 7ff0dc167740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d41ad840 con 0x55993c9faa00 2024-04-21T08:41:01.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.018+0000 7ff0d2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55993c9f14d0 0x55993ca06800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.018+0000 7ff0d2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55993c9f14d0 0x55993ca06800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60372/0 (socket says 172.21.15.86:60372) 2024-04-21T08:41:01.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.018+0000 7ff0d2ffd640 1 -- 172.21.15.86:0/3797425588 learned_addr learned my addr 172.21.15.86:0/3797425588 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:01.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.019+0000 7ff0d2ffd640 1 -- 172.21.15.86:0/3797425588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55993c9faa00 msgr2=0x55993c9fa310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.019+0000 7ff0d2ffd640 1 --2- 172.21.15.86:0/3797425588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55993c9faa00 0x55993c9fa310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.019+0000 7ff0d2ffd640 1 -- 172.21.15.86:0/3797425588 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55993c9f1e80 msgr2=0x55993c9f9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.019+0000 7ff0d2ffd640 1 --2- 172.21.15.86:0/3797425588 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55993c9f1e80 0x55993c9f9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.019+0000 7ff0d2ffd640 1 -- 172.21.15.86:0/3797425588 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55993c9fd180 con 0x55993c9f14d0 2024-04-21T08:41:01.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.019+0000 7ff0d2ffd640 1 --2- 172.21.15.86:0/3797425588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55993c9f14d0 0x55993ca06800 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x55993c9f2280 tx=0x7ff0c4007600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2571056ba92cac server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:01.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.019+0000 7ff0b8ff9640 1 -- 172.21.15.86:0/3797425588 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c401a5b0 con 0x55993c9f14d0 2024-04-21T08:41:01.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.019+0000 7ff0b8ff9640 1 -- 172.21.15.86:0/3797425588 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0c401a750 con 0x55993c9f14d0 2024-04-21T08:41:01.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.019+0000 7ff0b8ff9640 1 -- 172.21.15.86:0/3797425588 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c400e600 con 0x55993c9f14d0 2024-04-21T08:41:01.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.019+0000 7ff0dc167740 1 -- 172.21.15.86:0/3797425588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55993c9f14d0 msgr2=0x55993ca06800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.019+0000 7ff0dc167740 1 --2- 172.21.15.86:0/3797425588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55993c9f14d0 0x55993ca06800 secure :-1 s=READY pgs=842 cs=0 l=1 rev1=1 crypto rx=0x55993c9f2280 tx=0x7ff0c4007600 comp rx=0 tx=0).stop 2024-04-21T08:41:01.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.020+0000 7ff0dc167740 1 -- 172.21.15.86:0/3797425588 shutdown_connections 2024-04-21T08:41:01.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.020+0000 7ff0dc167740 1 --2- 172.21.15.86:0/3797425588 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55993c9faa00 0x55993c9fa310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.020+0000 7ff0dc167740 1 --2- 172.21.15.86:0/3797425588 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55993c9f1e80 0x55993c9f9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.020+0000 7ff0dc167740 1 --2- 172.21.15.86:0/3797425588 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55993c9f14d0 0x55993ca06800 unknown :-1 s=CLOSED pgs=842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.020+0000 7ff0dc167740 1 -- 172.21.15.86:0/3797425588 >> 172.21.15.86:0/3797425588 conn(0x55993c9ebba0 msgr2=0x55993c9eac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:01.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.020+0000 7ff0dc167740 1 -- 172.21.15.86:0/3797425588 shutdown_connections 2024-04-21T08:41:01.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.020+0000 7ff0dc167740 1 -- 172.21.15.86:0/3797425588 wait complete. 2024-04-21T08:41:01.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.020+0000 7ff0dc167740 1 -- start start 2024-04-21T08:41:01.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.020+0000 7ff0dc167740 1 Processor -- start 2024-04-21T08:41:01.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.020+0000 7ff0dc167740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55993c9f14d0 0x55993ca02a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.020+0000 7ff0dc167740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55993c9f1e80 0x55993ca02f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.020+0000 7ff0dc167740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55993c9faa00 0x55993c9feef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.020+0000 7ff0dc167740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d41ae5e0 con 0x55993c9f1e80 2024-04-21T08:41:01.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.020+0000 7ff0dc167740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d41adf10 con 0x55993c9faa00 2024-04-21T08:41:01.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.020+0000 7ff0dc167740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff0d41ab130 con 0x55993c9f14d0 2024-04-21T08:41:01.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.020+0000 7ff0dc167740 10 client.0 _ll_register_callbacks cb 0x7ff0c80be830 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:01.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.020+0000 7ff0d27fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55993c9f1e80 0x55993ca02f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.020+0000 7ff0d27fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55993c9f1e80 0x55993ca02f60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60380/0 (socket says 172.21.15.86:60380) 2024-04-21T08:41:01.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.020+0000 7ff0d27fc640 1 -- 172.21.15.86:0/4101655587 learned_addr learned my addr 172.21.15.86:0/4101655587 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:01.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.020+0000 7ff0d37fe640 1 --2- 172.21.15.86:0/4101655587 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55993c9faa00 0x55993c9feef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.020+0000 7ff0d27fc640 1 -- 172.21.15.86:0/4101655587 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55993c9f14d0 msgr2=0x55993ca02a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.039 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.020+0000 7ff0d2ffd640 1 --2- 172.21.15.86:0/4101655587 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55993c9f14d0 0x55993ca02a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.020+0000 7ff0d27fc640 1 --2- 172.21.15.86:0/4101655587 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55993c9f14d0 0x55993ca02a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.020+0000 7ff0d27fc640 1 -- 172.21.15.86:0/4101655587 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55993c9faa00 msgr2=0x55993c9feef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.021+0000 7ff0d27fc640 1 --2- 172.21.15.86:0/4101655587 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55993c9faa00 0x55993c9feef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.021+0000 7ff0d27fc640 1 -- 172.21.15.86:0/4101655587 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55993c9ff680 con 0x55993c9f1e80 2024-04-21T08:41:01.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.021+0000 7ff0d37fe640 1 --2- 172.21.15.86:0/4101655587 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55993c9faa00 0x55993c9feef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:01.040 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.021+0000 7ff0d27fc640 1 --2- 172.21.15.86:0/4101655587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55993c9f1e80 0x55993ca02f60 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ff0c80be9f0 tx=0x7ff0c8005be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be7350480f75122 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:01.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.021+0000 7ff0b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:01.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.021+0000 7ff0b6ff5640 1 -- 172.21.15.86:0/4101655587 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0c803c350 con 0x55993c9f1e80 2024-04-21T08:41:01.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.021+0000 7ff0b6ff5640 1 -- 172.21.15.86:0/4101655587 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7ff0c80f02f0 con 0x55993c9f1e80 2024-04-21T08:41:01.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.021+0000 7ff0dc167740 1 -- 172.21.15.86:0/4101655587 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff0c401a750 con 0x55993c9f1e80 2024-04-21T08:41:01.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.021+0000 7ff0b6ff5640 1 -- 172.21.15.86:0/4101655587 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff0c80cc050 con 0x55993c9f1e80 2024-04-21T08:41:01.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.021+0000 7ff0dc167740 10 client.16692 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:01.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.021+0000 7ff0dc167740 1 -- 172.21.15.86:0/4101655587 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55993ca71c20 con 0x55993c9f1e80 2024-04-21T08:41:01.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.021+0000 7ff0b6ff5640 1 -- 172.21.15.86:0/4101655587 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff0c401a750 con 0x55993c9f1e80 2024-04-21T08:41:01.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.021+0000 7ff0b6ff5640 1 -- 172.21.15.86:0/4101655587 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff0c80a90b0 con 0x55993c9f1e80 2024-04-21T08:41:01.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.021+0000 7ff0dc167740 10 client.16692 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:01.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.021+0000 7ff0dc167740 10 client.16692 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:41:01.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.021+0000 7ff0dc167740 20 client.16692 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:01.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.021+0000 7ff0dc167740 1 --2- 172.21.15.86:0/4101655587 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55993ca0ac20 0x55993ca741f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.021+0000 7ff0dc167740 4 client.16692 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:41:01.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.021+0000 7ff0dc167740 1 -- 172.21.15.86:0/4101655587 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55993c95da80 con 0x55993ca0ac20 2024-04-21T08:41:01.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.022+0000 7ff0d2ffd640 1 --2- 172.21.15.86:0/4101655587 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55993ca0ac20 0x55993ca741f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.022+0000 7ff0b6ff5640 10 client.16692 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:41:01.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.022+0000 7ff0d2ffd640 1 --2- 172.21.15.86:0/4101655587 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55993ca0ac20 0x55993ca741f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=227d248181db789c server_cookie=2bba9c3aa1e3c890 in_seq=0 out_seq=0 2024-04-21T08:41:01.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.022+0000 7ff0b6ff5640 1 -- 172.21.15.86:0/4101655587 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55993c95da80 con 0x55993ca0ac20 2024-04-21T08:41:01.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.022+0000 7ff0b6ff5640 10 client.16692 handle_command_reply: tid=0 2024-04-21T08:41:01.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.022+0000 7ff0dc167740 1 client.16692 shutdown 2024-04-21T08:41:01.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.022+0000 7ff0dc167740 1 -- 172.21.15.86:0/4101655587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55993c9f1e80 msgr2=0x55993ca02f60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.022+0000 7ff0dc167740 1 --2- 172.21.15.86:0/4101655587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55993c9f1e80 0x55993ca02f60 secure :-1 s=READY pgs=843 cs=0 l=1 rev1=1 crypto rx=0x7ff0c80be9f0 tx=0x7ff0c8005be0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.022+0000 7ff0dc167740 1 -- 172.21.15.86:0/4101655587 shutdown_connections 2024-04-21T08:41:01.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.022+0000 7ff0dc167740 1 --2- 172.21.15.86:0/4101655587 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55993ca0ac20 0x55993ca741f0 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.022+0000 7ff0dc167740 1 --2- 172.21.15.86:0/4101655587 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55993c9faa00 0x55993c9feef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.022+0000 7ff0b6ff5640 0 client.16692 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:41:01.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.022+0000 7ff0dc167740 1 --2- 172.21.15.86:0/4101655587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55993c9f1e80 0x55993ca02f60 unknown :-1 s=CLOSED pgs=843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.022+0000 7ff0dc167740 1 --2- 172.21.15.86:0/4101655587 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55993c9f14d0 0x55993ca02a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.022+0000 7ff0dc167740 1 -- 172.21.15.86:0/4101655587 >> 172.21.15.86:0/4101655587 conn(0x55993c9ebba0 msgr2=0x55993c9ea370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:01.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.022+0000 7ff0dc167740 1 -- 172.21.15.86:0/4101655587 shutdown_connections 2024-04-21T08:41:01.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.022+0000 7ff0dc167740 1 -- 172.21.15.86:0/4101655587 wait complete. 2024-04-21T08:41:01.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.023+0000 7ff0dc167740 20 client.16692 trim_cache size 0 max 16384 2024-04-21T08:41:01.044 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:41:01.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.023+0000 7ff0d9493640 1 -- 172.21.15.86:0/2464434382 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff0bc0a3d20 msgr2=0x7ff0bc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.023+0000 7ff0d9493640 1 --2- 172.21.15.86:0/2464434382 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff0bc0a3d20 0x7ff0bc0a61e0 secure :-1 s=READY pgs=121 cs=0 l=1 rev1=1 crypto rx=0x7ff0cc00f7f0 tx=0x7ff0cc014000 comp rx=0 tx=0).stop 2024-04-21T08:41:01.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.023+0000 7ff0d9493640 1 -- 172.21.15.86:0/2464434382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0d4103af0 msgr2=0x7ff0d419ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.023+0000 7ff0d9493640 1 --2- 172.21.15.86:0/2464434382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0d4103af0 0x7ff0d419ead0 secure :-1 s=READY pgs=839 cs=0 l=1 rev1=1 crypto rx=0x7ff0c8005940 tx=0x7ff0c802d920 comp rx=0 tx=0).stop 2024-04-21T08:41:01.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.023+0000 7ff0d9493640 1 -- 172.21.15.86:0/2464434382 shutdown_connections 2024-04-21T08:41:01.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.023+0000 7ff0d9493640 1 --2- 172.21.15.86:0/2464434382 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff0bc0a3d20 0x7ff0bc0a61e0 unknown :-1 s=CLOSED pgs=121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.023+0000 7ff0d9493640 1 --2- 172.21.15.86:0/2464434382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff0d410c510 0x7ff0d4198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.023+0000 7ff0d9493640 1 --2- 172.21.15.86:0/2464434382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff0d4103af0 0x7ff0d419ead0 unknown :-1 s=CLOSED pgs=839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.023+0000 7ff0d9493640 1 --2- 172.21.15.86:0/2464434382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff0d41031d0 0x7ff0d419e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.023+0000 7ff0d9493640 1 -- 172.21.15.86:0/2464434382 >> 172.21.15.86:0/2464434382 conn(0x7ff0d4077f50 msgr2=0x7ff0d40fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:01.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.023+0000 7ff0d9493640 1 -- 172.21.15.86:0/2464434382 shutdown_connections 2024-04-21T08:41:01.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.023+0000 7ff0d9493640 1 -- 172.21.15.86:0/2464434382 wait complete. 2024-04-21T08:41:01.047 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:41:01.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.147+0000 7ff400f15640 1 -- 172.21.15.86:0/883723663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3fc10c580 msgr2=0x7ff3fc10e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.147+0000 7ff400f15640 1 --2- 172.21.15.86:0/883723663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3fc10c580 0x7ff3fc10e9b0 secure :-1 s=READY pgs=844 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4009ca0 tx=0x7ff3f402c180 comp rx=0 tx=0).stop 2024-04-21T08:41:01.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.147+0000 7ff400f15640 1 -- 172.21.15.86:0/883723663 shutdown_connections 2024-04-21T08:41:01.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.147+0000 7ff400f15640 1 --2- 172.21.15.86:0/883723663 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3fc10c580 0x7ff3fc10e9b0 unknown :-1 s=CLOSED pgs=844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.147+0000 7ff400f15640 1 --2- 172.21.15.86:0/883723663 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3fc103ad0 0x7ff3fc10bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.147+0000 7ff400f15640 1 --2- 172.21.15.86:0/883723663 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3fc1031b0 0x7ff3fc103590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.147+0000 7ff400f15640 1 -- 172.21.15.86:0/883723663 >> 172.21.15.86:0/883723663 conn(0x7ff3fc0771d0 msgr2=0x7ff3fc0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:01.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.147+0000 7ff400f15640 1 -- 172.21.15.86:0/883723663 shutdown_connections 2024-04-21T08:41:01.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.147+0000 7ff400f15640 1 -- 172.21.15.86:0/883723663 wait complete. 2024-04-21T08:41:01.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.148+0000 7ff400f15640 1 Processor -- start 2024-04-21T08:41:01.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.148+0000 7ff400f15640 1 -- start start 2024-04-21T08:41:01.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.148+0000 7ff400f15640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3fc1031b0 0x7ff3fc198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.148+0000 7ff400f15640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3fc103ad0 0x7ff3fc198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.148+0000 7ff400f15640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3fc10c580 0x7ff3fc19cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.148+0000 7ff400f15640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff3fc1121a0 con 0x7ff3fc10c580 2024-04-21T08:41:01.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.148+0000 7ff400f15640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff3fc112020 con 0x7ff3fc103ad0 2024-04-21T08:41:01.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.148+0000 7ff400f15640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff3fc112320 con 0x7ff3fc1031b0 2024-04-21T08:41:01.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.148+0000 7ff3fad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3fc10c580 0x7ff3fc19cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.148+0000 7ff3fad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3fc10c580 0x7ff3fc19cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60384/0 (socket says 172.21.15.86:60384) 2024-04-21T08:41:01.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.148+0000 7ff3fad9d640 1 -- 172.21.15.86:0/3438721157 learned_addr learned my addr 172.21.15.86:0/3438721157 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:01.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.148+0000 7ff3f3fff640 1 --2- 172.21.15.86:0/3438721157 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3fc103ad0 0x7ff3fc198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.148+0000 7ff3fa59c640 1 --2- 172.21.15.86:0/3438721157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3fc1031b0 0x7ff3fc198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.148+0000 7ff3fad9d640 1 -- 172.21.15.86:0/3438721157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3fc1031b0 msgr2=0x7ff3fc198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.148+0000 7ff3fad9d640 1 --2- 172.21.15.86:0/3438721157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3fc1031b0 0x7ff3fc198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.148+0000 7ff3fad9d640 1 -- 172.21.15.86:0/3438721157 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3fc103ad0 msgr2=0x7ff3fc198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.148+0000 7ff3fad9d640 1 --2- 172.21.15.86:0/3438721157 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3fc103ad0 0x7ff3fc198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.148+0000 7ff3fad9d640 1 -- 172.21.15.86:0/3438721157 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3fc19d6f0 con 0x7ff3fc10c580 2024-04-21T08:41:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.148+0000 7ff3fa59c640 1 --2- 172.21.15.86:0/3438721157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3fc1031b0 0x7ff3fc198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.149+0000 7ff3fad9d640 1 --2- 172.21.15.86:0/3438721157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3fc10c580 0x7ff3fc19cfc0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4004af0 tx=0x7ff3f4003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.149+0000 7ff3f37fe640 1 -- 172.21.15.86:0/3438721157 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4039040 con 0x7ff3fc10c580 2024-04-21T08:41:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.149+0000 7ff3f37fe640 1 -- 172.21.15.86:0/3438721157 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3f4044070 con 0x7ff3fc10c580 2024-04-21T08:41:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.149+0000 7ff400f15640 1 -- 172.21.15.86:0/3438721157 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3fc19d980 con 0x7ff3fc10c580 2024-04-21T08:41:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.149+0000 7ff3f37fe640 1 -- 172.21.15.86:0/3438721157 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3f4032850 con 0x7ff3fc10c580 2024-04-21T08:41:01.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.149+0000 7ff400f15640 1 -- 172.21.15.86:0/3438721157 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3fc1a5270 con 0x7ff3fc10c580 2024-04-21T08:41:01.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.151+0000 7ff3f37fe640 1 -- 172.21.15.86:0/3438721157 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff3f40329f0 con 0x7ff3fc10c580 2024-04-21T08:41:01.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.151+0000 7ff403be9740 1 Processor -- start 2024-04-21T08:41:01.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.151+0000 7ff403be9740 1 -- start start 2024-04-21T08:41:01.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.151+0000 7ff403be9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef7f8ed4d0 0x55ef7f8ed8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.151+0000 7ff403be9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ef7f8ede80 0x55ef7f8f6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.151+0000 7ff403be9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ef7f8f6a00 0x55ef7f8f8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.151+0000 7ff403be9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ef7f8521f0 con 0x55ef7f8ed4d0 2024-04-21T08:41:01.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.151+0000 7ff403be9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55ef7f7ef4b0 con 0x55ef7f8ede80 2024-04-21T08:41:01.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.151+0000 7ff403be9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ef7f84e1b0 con 0x55ef7f8f6a00 2024-04-21T08:41:01.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.152+0000 7ff3f37fe640 1 --2- 172.21.15.86:0/3438721157 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff3e80a3d20 0x7ff3e80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.152+0000 7ff3f3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef7f8ed4d0 0x55ef7f8ed8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.152+0000 7ff3f3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef7f8ed4d0 0x55ef7f8ed8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60386/0 (socket says 172.21.15.86:60386) 2024-04-21T08:41:01.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.152+0000 7ff3f3fff640 1 -- 172.21.15.86:0/2133500357 learned_addr learned my addr 172.21.15.86:0/2133500357 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:01.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.152+0000 7ff3fad9d640 1 --2- 172.21.15.86:0/3438721157 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff3e80a3d20 0x7ff3e80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.152+0000 7ff3f37fe640 1 -- 172.21.15.86:0/3438721157 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7ff3f40f2fc0 con 0x7ff3fc10c580 2024-04-21T08:41:01.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.152+0000 7ff3fa59c640 1 --2- 172.21.15.86:0/2133500357 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ef7f8ede80 0x55ef7f8f6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.152+0000 7ff3f3fff640 1 -- 172.21.15.86:0/2133500357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ef7f8f6a00 msgr2=0x55ef7f8f8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.152+0000 7ff3f3fff640 1 --2- 172.21.15.86:0/2133500357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ef7f8f6a00 0x55ef7f8f8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.152+0000 7ff3f3fff640 1 -- 172.21.15.86:0/2133500357 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ef7f8ede80 msgr2=0x55ef7f8f6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.152+0000 7ff3f3fff640 1 --2- 172.21.15.86:0/2133500357 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ef7f8ede80 0x55ef7f8f6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.152+0000 7ff3f3fff640 1 -- 172.21.15.86:0/2133500357 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef7f7e11b0 con 0x55ef7f8ed4d0 2024-04-21T08:41:01.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.152+0000 7ff3fad9d640 1 --2- 172.21.15.86:0/3438721157 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff3e80a3d20 0x7ff3e80a61e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff3f402c760 tx=0x7ff3f402c180 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:01.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.152+0000 7ff3f3fff640 1 --2- 172.21.15.86:0/2133500357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef7f8ed4d0 0x55ef7f8ed8b0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7ff3e400bcd0 tx=0x7ff3e40127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26304593bd8f419b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:01.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.152+0000 7ff3f0ff9640 1 -- 172.21.15.86:0/2133500357 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e40120d0 con 0x55ef7f8ed4d0 2024-04-21T08:41:01.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.152+0000 7ff3f0ff9640 1 -- 172.21.15.86:0/2133500357 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3e4016070 con 0x55ef7f8ed4d0 2024-04-21T08:41:01.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.152+0000 7ff3f0ff9640 1 -- 172.21.15.86:0/2133500357 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e4010430 con 0x55ef7f8ed4d0 2024-04-21T08:41:01.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.153+0000 7ff403be9740 1 -- 172.21.15.86:0/2133500357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef7f8ed4d0 msgr2=0x55ef7f8ed8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.153+0000 7ff403be9740 1 --2- 172.21.15.86:0/2133500357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef7f8ed4d0 0x55ef7f8ed8b0 secure :-1 s=READY pgs=846 cs=0 l=1 rev1=1 crypto rx=0x7ff3e400bcd0 tx=0x7ff3e40127c0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.153+0000 7ff403be9740 1 -- 172.21.15.86:0/2133500357 shutdown_connections 2024-04-21T08:41:01.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.153+0000 7ff403be9740 1 --2- 172.21.15.86:0/2133500357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ef7f8f6a00 0x55ef7f8f8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.153+0000 7ff403be9740 1 --2- 172.21.15.86:0/2133500357 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ef7f8ede80 0x55ef7f8f6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.153+0000 7ff403be9740 1 --2- 172.21.15.86:0/2133500357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef7f8ed4d0 0x55ef7f8ed8b0 unknown :-1 s=CLOSED pgs=846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.153+0000 7ff403be9740 1 -- 172.21.15.86:0/2133500357 >> 172.21.15.86:0/2133500357 conn(0x55ef7f8e7ba0 msgr2=0x55ef7f8e7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:01.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.153+0000 7ff403be9740 1 -- 172.21.15.86:0/2133500357 shutdown_connections 2024-04-21T08:41:01.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.153+0000 7ff403be9740 1 -- 172.21.15.86:0/2133500357 wait complete. 2024-04-21T08:41:01.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.153+0000 7ff403be9740 1 -- start start 2024-04-21T08:41:01.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.154+0000 7ff403be9740 1 Processor -- start 2024-04-21T08:41:01.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.154+0000 7ff403be9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ef7f8ed4d0 0x55ef7f8f6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.154+0000 7ff403be9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ef7f8ede80 0x55ef7f8f3690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.154+0000 7ff403be9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef7f8f6a00 0x55ef7f8f3c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.154+0000 7ff403be9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ef7f8f9970 con 0x55ef7f8f6a00 2024-04-21T08:41:01.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.154+0000 7ff403be9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55ef7f80b500 con 0x55ef7f8ed4d0 2024-04-21T08:41:01.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.154+0000 7ff403be9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ef7f8f9af0 con 0x55ef7f8ede80 2024-04-21T08:41:01.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.154+0000 7ff403be9740 10 client.0 _ll_register_callbacks cb 0x55ef7f7d9b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:01.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.154+0000 7ff3f3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef7f8f6a00 0x55ef7f8f3c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.154+0000 7ff3f3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef7f8f6a00 0x55ef7f8f3c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60400/0 (socket says 172.21.15.86:60400) 2024-04-21T08:41:01.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.154+0000 7ff3f3fff640 1 -- 172.21.15.86:0/2358259582 learned_addr learned my addr 172.21.15.86:0/2358259582 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:01.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.154+0000 7ff3fa59c640 1 --2- 172.21.15.86:0/2358259582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ef7f8ede80 0x55ef7f8f3690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.154+0000 7ff3f3fff640 1 -- 172.21.15.86:0/2358259582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ef7f8ede80 msgr2=0x55ef7f8f3690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.154+0000 7ff3f3fff640 1 --2- 172.21.15.86:0/2358259582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ef7f8ede80 0x55ef7f8f3690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.154+0000 7ff3f3fff640 1 -- 172.21.15.86:0/2358259582 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ef7f8ed4d0 msgr2=0x55ef7f8f6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.154+0000 7ff3f3fff640 1 --2- 172.21.15.86:0/2358259582 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ef7f8ed4d0 0x55ef7f8f6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.154+0000 7ff3f3fff640 1 -- 172.21.15.86:0/2358259582 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef7f5643f0 con 0x55ef7f8f6a00 2024-04-21T08:41:01.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.154+0000 7ff3f3fff640 1 --2- 172.21.15.86:0/2358259582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef7f8f6a00 0x55ef7f8f3c10 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4005940 tx=0x7ff3e400bf00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=511a85ec0433d402 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:01.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.154+0000 7ff3e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:01.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.154+0000 7ff3e1ffb640 1 -- 172.21.15.86:0/2358259582 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e4003e70 con 0x55ef7f8f6a00 2024-04-21T08:41:01.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.154+0000 7ff403be9740 1 -- 172.21.15.86:0/2358259582 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3e4016070 con 0x55ef7f8f6a00 2024-04-21T08:41:01.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.154+0000 7ff3e1ffb640 1 -- 172.21.15.86:0/2358259582 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7ff3e4004010 con 0x55ef7f8f6a00 2024-04-21T08:41:01.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.155+0000 7ff3e1ffb640 1 -- 172.21.15.86:0/2358259582 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff3e402e050 con 0x55ef7f8f6a00 2024-04-21T08:41:01.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.155+0000 7ff403be9740 10 client.16704 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:01.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.155+0000 7ff403be9740 1 -- 172.21.15.86:0/2358259582 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ef7f62e7d0 con 0x55ef7f8f6a00 2024-04-21T08:41:01.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.155+0000 7ff3e1ffb640 1 -- 172.21.15.86:0/2358259582 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3e4016070 con 0x55ef7f8f6a00 2024-04-21T08:41:01.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.155+0000 7ff3e1ffb640 1 -- 172.21.15.86:0/2358259582 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff3e4010c60 con 0x55ef7f8f6a00 2024-04-21T08:41:01.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.155+0000 7ff403be9740 10 client.16704 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:01.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.155+0000 7ff403be9740 10 client.16704 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:41:01.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.155+0000 7ff403be9740 20 client.16704 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:01.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.155+0000 7ff403be9740 1 --2- 172.21.15.86:0/2358259582 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55ef7f906ae0 0x55ef7f908ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.155+0000 7ff403be9740 4 client.16704 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:01.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.155+0000 7ff403be9740 1 -- 172.21.15.86:0/2358259582 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ef7f7f1930 con 0x55ef7f906ae0 2024-04-21T08:41:01.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.155+0000 7ff3fa59c640 1 --2- 172.21.15.86:0/2358259582 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55ef7f906ae0 0x55ef7f908ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.156+0000 7ff3fa59c640 1 --2- 172.21.15.86:0/2358259582 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55ef7f906ae0 0x55ef7f908ed0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=50a49f130cfe0dcf server_cookie=6c810f7ce83d28ad in_seq=0 out_seq=0 2024-04-21T08:41:01.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.156+0000 7ff3e1ffb640 10 client.16704 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:41:01.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.156+0000 7ff3e1ffb640 1 -- 172.21.15.86:0/2358259582 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ef7f7f1930 con 0x55ef7f906ae0 2024-04-21T08:41:01.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.156+0000 7ff3e1ffb640 10 client.16704 handle_command_reply: tid=0 2024-04-21T08:41:01.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.156+0000 7ff403be9740 1 client.16704 shutdown 2024-04-21T08:41:01.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.156+0000 7ff403be9740 1 -- 172.21.15.86:0/2358259582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef7f8f6a00 msgr2=0x55ef7f8f3c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.156+0000 7ff403be9740 1 --2- 172.21.15.86:0/2358259582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef7f8f6a00 0x55ef7f8f3c10 secure :-1 s=READY pgs=847 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4005940 tx=0x7ff3e400bf00 comp rx=0 tx=0).stop 2024-04-21T08:41:01.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.156+0000 7ff403be9740 1 -- 172.21.15.86:0/2358259582 shutdown_connections 2024-04-21T08:41:01.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.156+0000 7ff403be9740 1 --2- 172.21.15.86:0/2358259582 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55ef7f906ae0 0x55ef7f908ed0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.156+0000 7ff403be9740 1 --2- 172.21.15.86:0/2358259582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef7f8f6a00 0x55ef7f8f3c10 unknown :-1 s=CLOSED pgs=847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.156+0000 7ff403be9740 1 --2- 172.21.15.86:0/2358259582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ef7f8ede80 0x55ef7f8f3690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.156+0000 7ff403be9740 1 --2- 172.21.15.86:0/2358259582 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ef7f8ed4d0 0x55ef7f8f6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.156+0000 7ff403be9740 1 -- 172.21.15.86:0/2358259582 >> 172.21.15.86:0/2358259582 conn(0x55ef7f8e7ba0 msgr2=0x55ef7f8e98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:01.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.157+0000 7ff3e1ffb640 0 client.16704 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:41:01.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.157+0000 7ff403be9740 1 -- 172.21.15.86:0/2358259582 shutdown_connections 2024-04-21T08:41:01.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.157+0000 7ff403be9740 1 -- 172.21.15.86:0/2358259582 wait complete. 2024-04-21T08:41:01.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.157+0000 7ff403be9740 20 client.16704 trim_cache size 0 max 16384 2024-04-21T08:41:01.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.170+0000 7ff403be9740 1 Processor -- start 2024-04-21T08:41:01.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.170+0000 7ff403be9740 1 -- start start 2024-04-21T08:41:01.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.170+0000 7ff403be9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ef7f8ed4d0 0x55ef7f902890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.170+0000 7ff403be9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ef7f8ede80 0x55ef7f8f5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.170+0000 7ff403be9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef7f8f6a00 0x55ef7f8f6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.170+0000 7ff403be9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff3fc1af260 con 0x55ef7f8f6a00 2024-04-21T08:41:01.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.170+0000 7ff403be9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff3fc1b0da0 con 0x55ef7f8ed4d0 2024-04-21T08:41:01.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.170+0000 7ff403be9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff3fc1af930 con 0x55ef7f8ede80 2024-04-21T08:41:01.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.170+0000 7ff3f3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef7f8f6a00 0x55ef7f8f6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.170+0000 7ff3f3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef7f8f6a00 0x55ef7f8f6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60402/0 (socket says 172.21.15.86:60402) 2024-04-21T08:41:01.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.170+0000 7ff3f3fff640 1 -- 172.21.15.86:0/21084894 learned_addr learned my addr 172.21.15.86:0/21084894 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:01.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.170+0000 7ff3f3fff640 1 --2- 172.21.15.86:0/21084894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ef7f8ed4d0 0x55ef7f902890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.170+0000 7ff3fa59c640 1 --2- 172.21.15.86:0/21084894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ef7f8ede80 0x55ef7f8f5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.170+0000 7ff3f3fff640 1 -- 172.21.15.86:0/21084894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ef7f8ede80 msgr2=0x55ef7f8f5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.170+0000 7ff3f3fff640 1 --2- 172.21.15.86:0/21084894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ef7f8ede80 0x55ef7f8f5dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.170+0000 7ff3f3fff640 1 -- 172.21.15.86:0/21084894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ef7f8ed4d0 msgr2=0x55ef7f902890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.170+0000 7ff3f3fff640 1 --2- 172.21.15.86:0/21084894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ef7f8ed4d0 0x55ef7f902890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.170+0000 7ff3f3fff640 1 -- 172.21.15.86:0/21084894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ef7f8f9020 con 0x55ef7f8f6a00 2024-04-21T08:41:01.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.170+0000 7ff3f3fff640 1 --2- 172.21.15.86:0/21084894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef7f8f6a00 0x55ef7f8f6300 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4001eb0 tx=0x7ff3e40127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16d34dd339e812b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:01.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.171+0000 7ff3e3fff640 1 -- 172.21.15.86:0/21084894 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e40280b0 con 0x55ef7f8f6a00 2024-04-21T08:41:01.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.171+0000 7ff3e3fff640 1 -- 172.21.15.86:0/21084894 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3e4016040 con 0x55ef7f8f6a00 2024-04-21T08:41:01.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.171+0000 7ff3e3fff640 1 -- 172.21.15.86:0/21084894 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e40107b0 con 0x55ef7f8f6a00 2024-04-21T08:41:01.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.171+0000 7ff403be9740 1 -- 172.21.15.86:0/21084894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef7f8f6a00 msgr2=0x55ef7f8f6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.171+0000 7ff403be9740 1 --2- 172.21.15.86:0/21084894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef7f8f6a00 0x55ef7f8f6300 secure :-1 s=READY pgs=848 cs=0 l=1 rev1=1 crypto rx=0x7ff3e4001eb0 tx=0x7ff3e40127c0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.171+0000 7ff403be9740 1 -- 172.21.15.86:0/21084894 shutdown_connections 2024-04-21T08:41:01.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.171+0000 7ff403be9740 1 --2- 172.21.15.86:0/21084894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef7f8f6a00 0x55ef7f8f6300 unknown :-1 s=CLOSED pgs=848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.171+0000 7ff403be9740 1 --2- 172.21.15.86:0/21084894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ef7f8ede80 0x55ef7f8f5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.171+0000 7ff403be9740 1 --2- 172.21.15.86:0/21084894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ef7f8ed4d0 0x55ef7f902890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.171+0000 7ff403be9740 1 -- 172.21.15.86:0/21084894 >> 172.21.15.86:0/21084894 conn(0x55ef7f8e7ba0 msgr2=0x55ef7f8f26c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:01.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.171+0000 7ff403be9740 1 -- 172.21.15.86:0/21084894 shutdown_connections 2024-04-21T08:41:01.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.171+0000 7ff403be9740 1 -- 172.21.15.86:0/21084894 wait complete. 2024-04-21T08:41:01.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.171+0000 7ff403be9740 1 -- start start 2024-04-21T08:41:01.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.171+0000 7ff403be9740 1 Processor -- start 2024-04-21T08:41:01.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.172+0000 7ff403be9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ef7f8ed4d0 0x55ef7f8ff2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.172+0000 7ff403be9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ef7f8ede80 0x55ef7f8f0dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.172+0000 7ff403be9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef7f8f6a00 0x55ef7f8ef900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.172+0000 7ff403be9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff3fc1b06d0 con 0x55ef7f8f6a00 2024-04-21T08:41:01.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.172+0000 7ff403be9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff3fc1b0000 con 0x55ef7f8ed4d0 2024-04-21T08:41:01.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.172+0000 7ff403be9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff3fc1ad220 con 0x55ef7f8ede80 2024-04-21T08:41:01.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.172+0000 7ff403be9740 10 client.0 _ll_register_callbacks cb 0x7ff3e400bdc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:01.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.172+0000 7ff3f3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef7f8f6a00 0x55ef7f8ef900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.172+0000 7ff3f3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef7f8f6a00 0x55ef7f8ef900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60410/0 (socket says 172.21.15.86:60410) 2024-04-21T08:41:01.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.172+0000 7ff3f3fff640 1 -- 172.21.15.86:0/1220339593 learned_addr learned my addr 172.21.15.86:0/1220339593 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:01.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.172+0000 7ff3f3fff640 1 --2- 172.21.15.86:0/1220339593 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ef7f8ed4d0 0x55ef7f8ff2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.172+0000 7ff3fa59c640 1 --2- 172.21.15.86:0/1220339593 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ef7f8ede80 0x55ef7f8f0dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.172+0000 7ff3f3fff640 1 -- 172.21.15.86:0/1220339593 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ef7f8ede80 msgr2=0x55ef7f8f0dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.172+0000 7ff3f3fff640 1 --2- 172.21.15.86:0/1220339593 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ef7f8ede80 0x55ef7f8f0dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.172+0000 7ff3f3fff640 1 -- 172.21.15.86:0/1220339593 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ef7f8ed4d0 msgr2=0x55ef7f8ff2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.172+0000 7ff3f3fff640 1 --2- 172.21.15.86:0/1220339593 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ef7f8ed4d0 0x55ef7f8ff2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.172+0000 7ff3f3fff640 1 -- 172.21.15.86:0/1220339593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ef7f8efe40 con 0x55ef7f8f6a00 2024-04-21T08:41:01.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.172+0000 7ff3f3fff640 1 --2- 172.21.15.86:0/1220339593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef7f8f6a00 0x55ef7f8ef900 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ff3e40058d0 tx=0x7ff3e400b800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d52c78d360d3243 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:01.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.172+0000 7ff3e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:01.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.172+0000 7ff3e1ffb640 1 -- 172.21.15.86:0/1220339593 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff3e40282b0 con 0x55ef7f8f6a00 2024-04-21T08:41:01.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.172+0000 7ff3e1ffb640 1 -- 172.21.15.86:0/1220339593 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7ff3e4028b30 con 0x55ef7f8f6a00 2024-04-21T08:41:01.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.172+0000 7ff403be9740 1 -- 172.21.15.86:0/1220339593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3e4016040 con 0x55ef7f8f6a00 2024-04-21T08:41:01.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.173+0000 7ff3e1ffb640 1 -- 172.21.15.86:0/1220339593 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff3e4028d30 con 0x55ef7f8f6a00 2024-04-21T08:41:01.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.173+0000 7ff403be9740 10 client.16710 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:01.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.173+0000 7ff403be9740 1 -- 172.21.15.86:0/1220339593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ef7f96c4b0 con 0x55ef7f8f6a00 2024-04-21T08:41:01.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.173+0000 7ff3e1ffb640 1 -- 172.21.15.86:0/1220339593 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3e4016040 con 0x55ef7f8f6a00 2024-04-21T08:41:01.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.173+0000 7ff3e1ffb640 1 -- 172.21.15.86:0/1220339593 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff3e40106a0 con 0x55ef7f8f6a00 2024-04-21T08:41:01.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.173+0000 7ff403be9740 10 client.16710 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:01.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.173+0000 7ff403be9740 10 client.16710 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:41:01.179 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.173+0000 7ff403be9740 20 client.16710 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:01.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.173+0000 7ff403be9740 1 --2- 172.21.15.86:0/1220339593 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55ef7f96cad0 0x55ef7f96eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.173+0000 7ff403be9740 4 client.16710 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:41:01.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.173+0000 7ff403be9740 1 -- 172.21.15.86:0/1220339593 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ef7f85fc10 con 0x55ef7f96cad0 2024-04-21T08:41:01.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.173+0000 7ff3fa59c640 1 --2- 172.21.15.86:0/1220339593 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55ef7f96cad0 0x55ef7f96eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.174+0000 7ff3e1ffb640 10 client.16710 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:41:01.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.174+0000 7ff3fa59c640 1 --2- 172.21.15.86:0/1220339593 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55ef7f96cad0 0x55ef7f96eec0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=483a7ec04fc6342f server_cookie=a5f0cbb82fdcc297 in_seq=0 out_seq=0 2024-04-21T08:41:01.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.174+0000 7ff3e1ffb640 1 -- 172.21.15.86:0/1220339593 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ef7f85fc10 con 0x55ef7f96cad0 2024-04-21T08:41:01.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.174+0000 7ff3e1ffb640 10 client.16710 handle_command_reply: tid=0 2024-04-21T08:41:01.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.174+0000 7ff403be9740 1 client.16710 shutdown 2024-04-21T08:41:01.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.174+0000 7ff403be9740 1 -- 172.21.15.86:0/1220339593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef7f8f6a00 msgr2=0x55ef7f8ef900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.174+0000 7ff403be9740 1 --2- 172.21.15.86:0/1220339593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef7f8f6a00 0x55ef7f8ef900 secure :-1 s=READY pgs=849 cs=0 l=1 rev1=1 crypto rx=0x7ff3e40058d0 tx=0x7ff3e400b800 comp rx=0 tx=0).stop 2024-04-21T08:41:01.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.174+0000 7ff403be9740 1 -- 172.21.15.86:0/1220339593 shutdown_connections 2024-04-21T08:41:01.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.174+0000 7ff403be9740 1 --2- 172.21.15.86:0/1220339593 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55ef7f96cad0 0x55ef7f96eec0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.174+0000 7ff403be9740 1 --2- 172.21.15.86:0/1220339593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ef7f8f6a00 0x55ef7f8ef900 unknown :-1 s=CLOSED pgs=849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.174+0000 7ff3e1ffb640 0 client.16710 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:41:01.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.174+0000 7ff403be9740 1 --2- 172.21.15.86:0/1220339593 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ef7f8ede80 0x55ef7f8f0dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.174+0000 7ff403be9740 1 --2- 172.21.15.86:0/1220339593 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ef7f8ed4d0 0x55ef7f8ff2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.174+0000 7ff403be9740 1 -- 172.21.15.86:0/1220339593 >> 172.21.15.86:0/1220339593 conn(0x55ef7f8e7ba0 msgr2=0x55ef7f8e6370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:01.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.174+0000 7ff403be9740 1 -- 172.21.15.86:0/1220339593 shutdown_connections 2024-04-21T08:41:01.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.174+0000 7ff403be9740 1 -- 172.21.15.86:0/1220339593 wait complete. 2024-04-21T08:41:01.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.174+0000 7ff403be9740 20 client.16710 trim_cache size 0 max 16384 2024-04-21T08:41:01.181 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:41:01.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.175+0000 7ff400f15640 1 -- 172.21.15.86:0/3438721157 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff3e80a3d20 msgr2=0x7ff3e80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.175+0000 7ff400f15640 1 --2- 172.21.15.86:0/3438721157 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff3e80a3d20 0x7ff3e80a61e0 secure :-1 s=READY pgs=122 cs=0 l=1 rev1=1 crypto rx=0x7ff3f402c760 tx=0x7ff3f402c180 comp rx=0 tx=0).stop 2024-04-21T08:41:01.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.175+0000 7ff400f15640 1 -- 172.21.15.86:0/3438721157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3fc10c580 msgr2=0x7ff3fc19cfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.175+0000 7ff400f15640 1 --2- 172.21.15.86:0/3438721157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3fc10c580 0x7ff3fc19cfc0 secure :-1 s=READY pgs=845 cs=0 l=1 rev1=1 crypto rx=0x7ff3f4004af0 tx=0x7ff3f4003ba0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.175+0000 7ff400f15640 1 -- 172.21.15.86:0/3438721157 shutdown_connections 2024-04-21T08:41:01.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.175+0000 7ff400f15640 1 --2- 172.21.15.86:0/3438721157 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff3e80a3d20 0x7ff3e80a61e0 unknown :-1 s=CLOSED pgs=122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.175+0000 7ff400f15640 1 --2- 172.21.15.86:0/3438721157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff3fc10c580 0x7ff3fc19cfc0 unknown :-1 s=CLOSED pgs=845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.175+0000 7ff400f15640 1 --2- 172.21.15.86:0/3438721157 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff3fc103ad0 0x7ff3fc198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.175+0000 7ff400f15640 1 --2- 172.21.15.86:0/3438721157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff3fc1031b0 0x7ff3fc198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.175+0000 7ff400f15640 1 -- 172.21.15.86:0/3438721157 >> 172.21.15.86:0/3438721157 conn(0x7ff3fc0771d0 msgr2=0x7ff3fc0fdd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:01.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.175+0000 7ff400f15640 1 -- 172.21.15.86:0/3438721157 shutdown_connections 2024-04-21T08:41:01.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.175+0000 7ff400f15640 1 -- 172.21.15.86:0/3438721157 wait complete. 2024-04-21T08:41:01.184 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:41:01.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.286+0000 7f7c26643640 1 -- 172.21.15.86:0/542133119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c20101090 msgr2=0x7f7c20101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.286+0000 7f7c26643640 1 --2- 172.21.15.86:0/542133119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c20101090 0x7f7c20101470 secure :-1 s=READY pgs=850 cs=0 l=1 rev1=1 crypto rx=0x7f7c10009900 tx=0x7f7c1002bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:01.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.286+0000 7f7c26643640 1 -- 172.21.15.86:0/542133119 shutdown_connections 2024-04-21T08:41:01.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.286+0000 7f7c26643640 1 --2- 172.21.15.86:0/542133119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c20106160 0x7f7c2010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.286+0000 7f7c26643640 1 --2- 172.21.15.86:0/542133119 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c20101a40 0x7f7c20105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.286+0000 7f7c26643640 1 --2- 172.21.15.86:0/542133119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c20101090 0x7f7c20101470 unknown :-1 s=CLOSED pgs=850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.286+0000 7f7c26643640 1 -- 172.21.15.86:0/542133119 >> 172.21.15.86:0/542133119 conn(0x7f7c20077f50 msgr2=0x7f7c20078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:01.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.286+0000 7f7c26643640 1 -- 172.21.15.86:0/542133119 shutdown_connections 2024-04-21T08:41:01.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.286+0000 7f7c26643640 1 -- 172.21.15.86:0/542133119 wait complete. 2024-04-21T08:41:01.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.287+0000 7f7c26643640 1 Processor -- start 2024-04-21T08:41:01.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.287+0000 7f7c26643640 1 -- start start 2024-04-21T08:41:01.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.287+0000 7f7c26643640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c20101090 0x7f7c201995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.287+0000 7f7c26643640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c20106160 0x7f7c20199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.287+0000 7f7c26643640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c2019a090 0x7f7c201a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.287+0000 7f7c26643640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7c201110a0 con 0x7f7c2019a090 2024-04-21T08:41:01.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.287+0000 7f7c26643640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7c20110f20 con 0x7f7c20106160 2024-04-21T08:41:01.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.287+0000 7f7c26643640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7c20111220 con 0x7f7c20101090 2024-04-21T08:41:01.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.287+0000 7f7c24be0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c2019a090 0x7f7c201a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.287+0000 7f7c24be0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c2019a090 0x7f7c201a4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60434/0 (socket says 172.21.15.86:60434) 2024-04-21T08:41:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.287+0000 7f7c24be0640 1 -- 172.21.15.86:0/3218090297 learned_addr learned my addr 172.21.15.86:0/3218090297 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.288+0000 7f7c1f7fe640 1 --2- 172.21.15.86:0/3218090297 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c20106160 0x7f7c20199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.288+0000 7f7c1ffff640 1 --2- 172.21.15.86:0/3218090297 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c20101090 0x7f7c201995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.288+0000 7f7c24be0640 1 -- 172.21.15.86:0/3218090297 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c20101090 msgr2=0x7f7c201995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.288+0000 7f7c24be0640 1 --2- 172.21.15.86:0/3218090297 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c20101090 0x7f7c201995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.288+0000 7f7c24be0640 1 -- 172.21.15.86:0/3218090297 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c20106160 msgr2=0x7f7c20199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.288+0000 7f7c24be0640 1 --2- 172.21.15.86:0/3218090297 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c20106160 0x7f7c20199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.288+0000 7f7c24be0640 1 -- 172.21.15.86:0/3218090297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7c201a4e10 con 0x7f7c2019a090 2024-04-21T08:41:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.288+0000 7f7c1f7fe640 1 --2- 172.21.15.86:0/3218090297 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c20106160 0x7f7c20199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.288+0000 7f7c24be0640 1 --2- 172.21.15.86:0/3218090297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c2019a090 0x7f7c201a4640 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f7c1800eaa0 tx=0x7f7c1800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.288+0000 7f7c1d7fa640 1 -- 172.21.15.86:0/3218090297 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1800cdb0 con 0x7f7c2019a090 2024-04-21T08:41:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.288+0000 7f7c1d7fa640 1 -- 172.21.15.86:0/3218090297 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c18005020 con 0x7f7c2019a090 2024-04-21T08:41:01.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.288+0000 7f7c26643640 1 -- 172.21.15.86:0/3218090297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7c201a5100 con 0x7f7c2019a090 2024-04-21T08:41:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.288+0000 7f7c1d7fa640 1 -- 172.21.15.86:0/3218090297 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c180116d0 con 0x7f7c2019a090 2024-04-21T08:41:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.288+0000 7f7c26643640 1 -- 172.21.15.86:0/3218090297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7c201a5650 con 0x7f7c2019a090 2024-04-21T08:41:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.290+0000 7f7c1d7fa640 1 -- 172.21.15.86:0/3218090297 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7c180051d0 con 0x7f7c2019a090 2024-04-21T08:41:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.291+0000 7f7c29238740 1 Processor -- start 2024-04-21T08:41:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.291+0000 7f7c29238740 1 -- start start 2024-04-21T08:41:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.291+0000 7f7c29238740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5602b84d0 0x55d5602b88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.291+0000 7f7c29238740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5602b8e80 0x55d5602c1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.291+0000 7f7c29238740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5602c1a00 0x55d5602c3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.291+0000 7f7c29238740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d56021d1f0 con 0x55d5602b8e80 2024-04-21T08:41:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.291+0000 7f7c29238740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d5601ba4b0 con 0x55d5602b84d0 2024-04-21T08:41:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.291+0000 7f7c29238740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d5602191b0 con 0x55d5602c1a00 2024-04-21T08:41:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.291+0000 7f7c1d7fa640 1 --2- 172.21.15.86:0/3218090297 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7c080a3da0 0x7f7c080a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.291+0000 7f7c1ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5602b8e80 0x55d5602c1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.291+0000 7f7c1ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5602b8e80 0x55d5602c1430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60440/0 (socket says 172.21.15.86:60440) 2024-04-21T08:41:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.291+0000 7f7c1ffff640 1 -- 172.21.15.86:0/172158862 learned_addr learned my addr 172.21.15.86:0/172158862 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:01.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.291+0000 7f7c1f7fe640 1 --2- 172.21.15.86:0/172158862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5602c1a00 0x55d5602c3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.291+0000 7f7c24be0640 1 --2- 172.21.15.86:0/3218090297 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7c080a3da0 0x7f7c080a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.291+0000 7f7c1f7fe640 1 --2- 172.21.15.86:0/172158862 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5602b84d0 0x55d5602b88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.291+0000 7f7c1d7fa640 1 -- 172.21.15.86:0/3218090297 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f7c180d0d20 con 0x7f7c2019a090 2024-04-21T08:41:01.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.291+0000 7f7c1ffff640 1 -- 172.21.15.86:0/172158862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5602c1a00 msgr2=0x55d5602c3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.291+0000 7f7c1ffff640 1 --2- 172.21.15.86:0/172158862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5602c1a00 0x55d5602c3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.291+0000 7f7c1ffff640 1 -- 172.21.15.86:0/172158862 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5602b84d0 msgr2=0x55d5602b88b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.291+0000 7f7c1ffff640 1 --2- 172.21.15.86:0/172158862 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5602b84d0 0x55d5602b88b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.291+0000 7f7c1ffff640 1 -- 172.21.15.86:0/172158862 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5601ac1b0 con 0x55d5602b8e80 2024-04-21T08:41:01.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.292+0000 7f7c24be0640 1 --2- 172.21.15.86:0/3218090297 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7c080a3da0 0x7f7c080a6260 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7c1800cfd0 tx=0x7f7c18087040 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:01.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.292+0000 7f7c1ffff640 1 --2- 172.21.15.86:0/172158862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5602b8e80 0x55d5602c1430 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f7c10009900 tx=0x7f7c10005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0bd2f41e5e23827 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:01.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.292+0000 7f7c067fc640 1 -- 172.21.15.86:0/172158862 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c10044020 con 0x55d5602b8e80 2024-04-21T08:41:01.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.292+0000 7f7c067fc640 1 -- 172.21.15.86:0/172158862 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c1002cc90 con 0x55d5602b8e80 2024-04-21T08:41:01.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.292+0000 7f7c067fc640 1 -- 172.21.15.86:0/172158862 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c100046c0 con 0x55d5602b8e80 2024-04-21T08:41:01.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.292+0000 7f7c29238740 1 -- 172.21.15.86:0/172158862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5602b8e80 msgr2=0x55d5602c1430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.292+0000 7f7c29238740 1 --2- 172.21.15.86:0/172158862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5602b8e80 0x55d5602c1430 secure :-1 s=READY pgs=852 cs=0 l=1 rev1=1 crypto rx=0x7f7c10009900 tx=0x7f7c10005e70 comp rx=0 tx=0).stop 2024-04-21T08:41:01.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.292+0000 7f7c29238740 1 -- 172.21.15.86:0/172158862 shutdown_connections 2024-04-21T08:41:01.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.292+0000 7f7c29238740 1 --2- 172.21.15.86:0/172158862 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5602c1a00 0x55d5602c3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.292+0000 7f7c29238740 1 --2- 172.21.15.86:0/172158862 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5602b8e80 0x55d5602c1430 unknown :-1 s=CLOSED pgs=852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.292+0000 7f7c29238740 1 --2- 172.21.15.86:0/172158862 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5602b84d0 0x55d5602b88b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.292+0000 7f7c29238740 1 -- 172.21.15.86:0/172158862 >> 172.21.15.86:0/172158862 conn(0x55d5602b2ba0 msgr2=0x55d5602b2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:01.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.292+0000 7f7c29238740 1 -- 172.21.15.86:0/172158862 shutdown_connections 2024-04-21T08:41:01.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.292+0000 7f7c29238740 1 -- 172.21.15.86:0/172158862 wait complete. 2024-04-21T08:41:01.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.293+0000 7f7c29238740 1 -- start start 2024-04-21T08:41:01.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.293+0000 7f7c29238740 1 Processor -- start 2024-04-21T08:41:01.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.293+0000 7f7c29238740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5602b84d0 0x55d5602c1320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.293+0000 7f7c29238740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5602b8e80 0x55d5602be6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.293+0000 7f7c29238740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5602c1a00 0x55d5602bec50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.293+0000 7f7c29238740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d5602c4910 con 0x55d5602b8e80 2024-04-21T08:41:01.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.293+0000 7f7c29238740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d5601d6500 con 0x55d5602c1a00 2024-04-21T08:41:01.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.293+0000 7f7c29238740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d5602c4a90 con 0x55d5602b84d0 2024-04-21T08:41:01.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.293+0000 7f7c29238740 10 client.0 _ll_register_callbacks cb 0x55d5601a4b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:01.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.293+0000 7f7c1ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5602b8e80 0x55d5602be6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.293+0000 7f7c1ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5602b8e80 0x55d5602be6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60446/0 (socket says 172.21.15.86:60446) 2024-04-21T08:41:01.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.293+0000 7f7c1ffff640 1 -- 172.21.15.86:0/265051666 learned_addr learned my addr 172.21.15.86:0/265051666 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:01.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.293+0000 7f7c1f7fe640 1 --2- 172.21.15.86:0/265051666 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5602b84d0 0x55d5602c1320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.293+0000 7f7c1f7fe640 1 --2- 172.21.15.86:0/265051666 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5602c1a00 0x55d5602bec50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.294+0000 7f7c1ffff640 1 -- 172.21.15.86:0/265051666 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5602b84d0 msgr2=0x55d5602c1320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.294+0000 7f7c1ffff640 1 --2- 172.21.15.86:0/265051666 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5602b84d0 0x55d5602c1320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.294+0000 7f7c1ffff640 1 -- 172.21.15.86:0/265051666 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5602c1a00 msgr2=0x55d5602bec50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.294+0000 7f7c1ffff640 1 --2- 172.21.15.86:0/265051666 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5602c1a00 0x55d5602bec50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.294+0000 7f7c1ffff640 1 -- 172.21.15.86:0/265051666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d55ff2f3f0 con 0x55d5602b8e80 2024-04-21T08:41:01.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.294+0000 7f7c1f7fe640 1 --2- 172.21.15.86:0/265051666 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5602c1a00 0x55d5602bec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:01.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.294+0000 7f7c1ffff640 1 --2- 172.21.15.86:0/265051666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5602b8e80 0x55d5602be6b0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f7c1002c170 tx=0x7f7c1002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1073198c275858fe server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:01.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.294+0000 7f7bff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.294+0000 7f7bff7fe640 1 -- 172.21.15.86:0/265051666 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c1004d070 con 0x55d5602b8e80 2024-04-21T08:41:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.294+0000 7f7bff7fe640 1 -- 172.21.15.86:0/265051666 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f7c100037f0 con 0x55d5602b8e80 2024-04-21T08:41:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.294+0000 7f7c29238740 1 -- 172.21.15.86:0/265051666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c1002cc90 con 0x55d5602b8e80 2024-04-21T08:41:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.294+0000 7f7bff7fe640 1 -- 172.21.15.86:0/265051666 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7c10059050 con 0x55d5602b8e80 2024-04-21T08:41:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.294+0000 7f7c29238740 10 client.16722 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.294+0000 7f7c29238740 1 -- 172.21.15.86:0/265051666 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d55fff97d0 con 0x55d5602b8e80 2024-04-21T08:41:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.294+0000 7f7bff7fe640 1 -- 172.21.15.86:0/265051666 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c1002cc90 con 0x55d5602b8e80 2024-04-21T08:41:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.294+0000 7f7bff7fe640 1 -- 172.21.15.86:0/265051666 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7c100408b0 con 0x55d5602b8e80 2024-04-21T08:41:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.294+0000 7f7c29238740 10 client.16722 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.294+0000 7f7c29238740 10 client.16722 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:41:01.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.294+0000 7f7c29238740 20 client.16722 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.294+0000 7f7c29238740 1 --2- 172.21.15.86:0/265051666 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55d5602d1a40 0x55d5602d3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.294+0000 7f7c29238740 4 client.16722 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.294+0000 7f7c29238740 1 -- 172.21.15.86:0/265051666 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5601bc930 con 0x55d5602d1a40 2024-04-21T08:41:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.295+0000 7f7c1f7fe640 1 --2- 172.21.15.86:0/265051666 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55d5602d1a40 0x55d5602d3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.295+0000 7f7c1f7fe640 1 --2- 172.21.15.86:0/265051666 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55d5602d1a40 0x55d5602d3e30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a3b921da8fd94dca server_cookie=db4e73e209fd96c5 in_seq=0 out_seq=0 2024-04-21T08:41:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.295+0000 7f7bff7fe640 10 client.16722 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:41:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.296+0000 7f7bff7fe640 1 -- 172.21.15.86:0/265051666 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d5601bc930 con 0x55d5602d1a40 2024-04-21T08:41:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.296+0000 7f7bff7fe640 10 client.16722 handle_command_reply: tid=0 2024-04-21T08:41:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.296+0000 7f7c29238740 1 client.16722 shutdown 2024-04-21T08:41:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.296+0000 7f7c29238740 1 -- 172.21.15.86:0/265051666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5602b8e80 msgr2=0x55d5602be6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.296+0000 7f7c29238740 1 --2- 172.21.15.86:0/265051666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5602b8e80 0x55d5602be6b0 secure :-1 s=READY pgs=853 cs=0 l=1 rev1=1 crypto rx=0x7f7c1002c170 tx=0x7f7c1002bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.296+0000 7f7c29238740 1 -- 172.21.15.86:0/265051666 shutdown_connections 2024-04-21T08:41:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.296+0000 7f7c29238740 1 --2- 172.21.15.86:0/265051666 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55d5602d1a40 0x55d5602d3e30 crc :-1 s=READY pgs=41 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.296+0000 7f7c29238740 1 --2- 172.21.15.86:0/265051666 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5602c1a00 0x55d5602bec50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.296+0000 7f7c29238740 1 --2- 172.21.15.86:0/265051666 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5602b8e80 0x55d5602be6b0 unknown :-1 s=CLOSED pgs=853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.296+0000 7f7bff7fe640 0 client.16722 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:41:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.296+0000 7f7c29238740 1 --2- 172.21.15.86:0/265051666 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5602b84d0 0x55d5602c1320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.296+0000 7f7c29238740 1 -- 172.21.15.86:0/265051666 >> 172.21.15.86:0/265051666 conn(0x55d5602b2ba0 msgr2=0x55d5602b48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.296+0000 7f7c29238740 1 -- 172.21.15.86:0/265051666 shutdown_connections 2024-04-21T08:41:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.296+0000 7f7c29238740 1 -- 172.21.15.86:0/265051666 wait complete. 2024-04-21T08:41:01.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.296+0000 7f7c29238740 20 client.16722 trim_cache size 0 max 16384 2024-04-21T08:41:01.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.309+0000 7f7c29238740 1 Processor -- start 2024-04-21T08:41:01.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.309+0000 7f7c29238740 1 -- start start 2024-04-21T08:41:01.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.309+0000 7f7c29238740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5602b84d0 0x55d5602bd810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.309+0000 7f7c29238740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5602b8e80 0x55d5602b4700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.309+0000 7f7c29238740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5602c1a00 0x55d5602c0de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.309+0000 7f7c29238740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7c201af340 con 0x55d5602c1a00 2024-04-21T08:41:01.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.309+0000 7f7c29238740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7c201b0e80 con 0x55d5602b84d0 2024-04-21T08:41:01.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.309+0000 7f7c29238740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7c201afa10 con 0x55d5602b8e80 2024-04-21T08:41:01.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.309+0000 7f7c1f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5602c1a00 0x55d5602c0de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.310+0000 7f7c1f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5602c1a00 0x55d5602c0de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60452/0 (socket says 172.21.15.86:60452) 2024-04-21T08:41:01.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.310+0000 7f7c1f7fe640 1 -- 172.21.15.86:0/1472770993 learned_addr learned my addr 172.21.15.86:0/1472770993 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:01.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.310+0000 7f7c1f7fe640 1 --2- 172.21.15.86:0/1472770993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5602b84d0 0x55d5602bd810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.310+0000 7f7c1ffff640 1 --2- 172.21.15.86:0/1472770993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5602b8e80 0x55d5602b4700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.310+0000 7f7c1f7fe640 1 -- 172.21.15.86:0/1472770993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5602b8e80 msgr2=0x55d5602b4700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.310+0000 7f7c1f7fe640 1 --2- 172.21.15.86:0/1472770993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5602b8e80 0x55d5602b4700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.310+0000 7f7c1f7fe640 1 -- 172.21.15.86:0/1472770993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5602b84d0 msgr2=0x55d5602bd810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.310+0000 7f7c1f7fe640 1 --2- 172.21.15.86:0/1472770993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5602b84d0 0x55d5602bd810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.310+0000 7f7c1f7fe640 1 -- 172.21.15.86:0/1472770993 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5602bcc00 con 0x55d5602c1a00 2024-04-21T08:41:01.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.310+0000 7f7c1f7fe640 1 --2- 172.21.15.86:0/1472770993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5602c1a00 0x55d5602c0de0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f7c140058c0 tx=0x7f7c14005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d48a32bb27ebc58e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:01.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.310+0000 7f7c05ffb640 1 -- 172.21.15.86:0/1472770993 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c14012c10 con 0x55d5602c1a00 2024-04-21T08:41:01.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.310+0000 7f7c05ffb640 1 -- 172.21.15.86:0/1472770993 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c14012db0 con 0x55d5602c1a00 2024-04-21T08:41:01.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.310+0000 7f7c05ffb640 1 -- 172.21.15.86:0/1472770993 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c140117c0 con 0x55d5602c1a00 2024-04-21T08:41:01.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.311+0000 7f7c29238740 1 -- 172.21.15.86:0/1472770993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5602c1a00 msgr2=0x55d5602c0de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.311+0000 7f7c29238740 1 --2- 172.21.15.86:0/1472770993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5602c1a00 0x55d5602c0de0 secure :-1 s=READY pgs=854 cs=0 l=1 rev1=1 crypto rx=0x7f7c140058c0 tx=0x7f7c14005850 comp rx=0 tx=0).stop 2024-04-21T08:41:01.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.311+0000 7f7c29238740 1 -- 172.21.15.86:0/1472770993 shutdown_connections 2024-04-21T08:41:01.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.311+0000 7f7c29238740 1 --2- 172.21.15.86:0/1472770993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5602c1a00 0x55d5602c0de0 unknown :-1 s=CLOSED pgs=854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.311+0000 7f7c29238740 1 --2- 172.21.15.86:0/1472770993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5602b8e80 0x55d5602b4700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.311+0000 7f7c29238740 1 --2- 172.21.15.86:0/1472770993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5602b84d0 0x55d5602bd810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.311+0000 7f7c29238740 1 -- 172.21.15.86:0/1472770993 >> 172.21.15.86:0/1472770993 conn(0x55d5602b2ba0 msgr2=0x55d5602b3ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:01.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.311+0000 7f7c29238740 1 -- 172.21.15.86:0/1472770993 shutdown_connections 2024-04-21T08:41:01.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.311+0000 7f7c29238740 1 -- 172.21.15.86:0/1472770993 wait complete. 2024-04-21T08:41:01.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.311+0000 7f7c29238740 1 -- start start 2024-04-21T08:41:01.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.311+0000 7f7c29238740 1 Processor -- start 2024-04-21T08:41:01.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.311+0000 7f7c29238740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5602b84d0 0x55d5602d6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.311+0000 7f7c29238740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5602b8e80 0x55d5602d73b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.311+0000 7f7c29238740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5602c1a00 0x55d5602d4630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.311+0000 7f7c29238740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7c201b07b0 con 0x55d5602b84d0 2024-04-21T08:41:01.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.311+0000 7f7c29238740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7c201b00e0 con 0x55d5602b8e80 2024-04-21T08:41:01.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.311+0000 7f7c29238740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7c201ad300 con 0x55d5602c1a00 2024-04-21T08:41:01.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.311+0000 7f7c29238740 10 client.0 _ll_register_callbacks cb 0x7f7c1002cb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:01.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.312+0000 7f7c1f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5602b84d0 0x55d5602d6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.312+0000 7f7c1f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5602b84d0 0x55d5602d6e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60460/0 (socket says 172.21.15.86:60460) 2024-04-21T08:41:01.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.312+0000 7f7c1f7fe640 1 -- 172.21.15.86:0/2473111100 learned_addr learned my addr 172.21.15.86:0/2473111100 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:01.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.312+0000 7f7c1ffff640 1 --2- 172.21.15.86:0/2473111100 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5602b8e80 0x55d5602d73b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.312+0000 7f7c1f7fe640 1 -- 172.21.15.86:0/2473111100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5602c1a00 msgr2=0x55d5602d4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.312+0000 7f7c1f7fe640 1 --2- 172.21.15.86:0/2473111100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5602c1a00 0x55d5602d4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.312+0000 7f7c1f7fe640 1 -- 172.21.15.86:0/2473111100 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5602b8e80 msgr2=0x55d5602d73b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.312+0000 7f7c1f7fe640 1 --2- 172.21.15.86:0/2473111100 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5602b8e80 0x55d5602d73b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.312+0000 7f7c1f7fe640 1 -- 172.21.15.86:0/2473111100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5602d4e30 con 0x55d5602b84d0 2024-04-21T08:41:01.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.312+0000 7f7c1f7fe640 1 --2- 172.21.15.86:0/2473111100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5602b84d0 0x55d5602d6e70 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f7c1400c560 tx=0x7f7c14012c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a38c2d7201f0288 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:01.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.312+0000 7f7bff7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:01.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.312+0000 7f7bff7fe640 1 -- 172.21.15.86:0/2473111100 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7c14005520 con 0x55d5602b84d0 2024-04-21T08:41:01.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.312+0000 7f7bff7fe640 1 -- 172.21.15.86:0/2473111100 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f7c140046c0 con 0x55d5602b84d0 2024-04-21T08:41:01.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.312+0000 7f7c29238740 1 -- 172.21.15.86:0/2473111100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7c14012db0 con 0x55d5602b84d0 2024-04-21T08:41:01.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.312+0000 7f7bff7fe640 1 -- 172.21.15.86:0/2473111100 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7c1403a050 con 0x55d5602b84d0 2024-04-21T08:41:01.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.312+0000 7f7c29238740 10 client.16728 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:01.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.312+0000 7f7c29238740 1 -- 172.21.15.86:0/2473111100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d5603378d0 con 0x55d5602b84d0 2024-04-21T08:41:01.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.312+0000 7f7bff7fe640 1 -- 172.21.15.86:0/2473111100 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7c14012db0 con 0x55d5602b84d0 2024-04-21T08:41:01.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.313+0000 7f7bff7fe640 1 -- 172.21.15.86:0/2473111100 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7c14033060 con 0x55d5602b84d0 2024-04-21T08:41:01.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.313+0000 7f7c29238740 10 client.16728 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:01.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.313+0000 7f7c29238740 10 client.16728 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:41:01.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.313+0000 7f7c29238740 20 client.16728 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:01.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.313+0000 7f7c29238740 1 --2- 172.21.15.86:0/2473111100 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55d560337f90 0x55d56033a3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.313+0000 7f7c29238740 4 client.16728 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:41:01.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.313+0000 7f7c29238740 1 -- 172.21.15.86:0/2473111100 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d56022ac10 con 0x55d560337f90 2024-04-21T08:41:01.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.313+0000 7f7c1ffff640 1 --2- 172.21.15.86:0/2473111100 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55d560337f90 0x55d56033a3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.313+0000 7f7bff7fe640 10 client.16728 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:41:01.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.313+0000 7f7c1ffff640 1 --2- 172.21.15.86:0/2473111100 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55d560337f90 0x55d56033a3b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=205e720fafab8b7b server_cookie=fb35fc04bd953c43 in_seq=0 out_seq=0 2024-04-21T08:41:01.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.313+0000 7f7bff7fe640 1 -- 172.21.15.86:0/2473111100 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d56022ac10 con 0x55d560337f90 2024-04-21T08:41:01.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.313+0000 7f7bff7fe640 10 client.16728 handle_command_reply: tid=0 2024-04-21T08:41:01.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.313+0000 7f7c29238740 1 client.16728 shutdown 2024-04-21T08:41:01.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.314+0000 7f7c29238740 1 -- 172.21.15.86:0/2473111100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5602b84d0 msgr2=0x55d5602d6e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.314+0000 7f7c29238740 1 --2- 172.21.15.86:0/2473111100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5602b84d0 0x55d5602d6e70 secure :-1 s=READY pgs=855 cs=0 l=1 rev1=1 crypto rx=0x7f7c1400c560 tx=0x7f7c14012c00 comp rx=0 tx=0).stop 2024-04-21T08:41:01.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.314+0000 7f7c29238740 1 -- 172.21.15.86:0/2473111100 shutdown_connections 2024-04-21T08:41:01.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.314+0000 7f7c29238740 1 --2- 172.21.15.86:0/2473111100 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55d560337f90 0x55d56033a3b0 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.314+0000 7f7c29238740 1 --2- 172.21.15.86:0/2473111100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5602c1a00 0x55d5602d4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.314+0000 7f7c29238740 1 --2- 172.21.15.86:0/2473111100 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5602b8e80 0x55d5602d73b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.314+0000 7f7c29238740 1 --2- 172.21.15.86:0/2473111100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5602b84d0 0x55d5602d6e70 unknown :-1 s=CLOSED pgs=855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.314+0000 7f7bff7fe640 0 client.16728 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:41:01.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.314+0000 7f7c29238740 1 -- 172.21.15.86:0/2473111100 >> 172.21.15.86:0/2473111100 conn(0x55d5602b2ba0 msgr2=0x55d5602cd570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:01.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.314+0000 7f7c29238740 1 -- 172.21.15.86:0/2473111100 shutdown_connections 2024-04-21T08:41:01.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.314+0000 7f7c29238740 1 -- 172.21.15.86:0/2473111100 wait complete. 2024-04-21T08:41:01.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.314+0000 7f7c29238740 20 client.16728 trim_cache size 0 max 16384 2024-04-21T08:41:01.323 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:41:01.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.314+0000 7f7c26643640 1 -- 172.21.15.86:0/3218090297 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7c080a3da0 msgr2=0x7f7c080a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.314+0000 7f7c26643640 1 --2- 172.21.15.86:0/3218090297 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7c080a3da0 0x7f7c080a6260 secure :-1 s=READY pgs=123 cs=0 l=1 rev1=1 crypto rx=0x7f7c1800cfd0 tx=0x7f7c18087040 comp rx=0 tx=0).stop 2024-04-21T08:41:01.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.314+0000 7f7c26643640 1 -- 172.21.15.86:0/3218090297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c2019a090 msgr2=0x7f7c201a4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.314+0000 7f7c26643640 1 --2- 172.21.15.86:0/3218090297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c2019a090 0x7f7c201a4640 secure :-1 s=READY pgs=851 cs=0 l=1 rev1=1 crypto rx=0x7f7c1800eaa0 tx=0x7f7c1800ef60 comp rx=0 tx=0).stop 2024-04-21T08:41:01.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.314+0000 7f7c26643640 1 -- 172.21.15.86:0/3218090297 shutdown_connections 2024-04-21T08:41:01.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.314+0000 7f7c26643640 1 --2- 172.21.15.86:0/3218090297 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7c080a3da0 0x7f7c080a6260 unknown :-1 s=CLOSED pgs=123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.314+0000 7f7c26643640 1 --2- 172.21.15.86:0/3218090297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7c2019a090 0x7f7c201a4640 unknown :-1 s=CLOSED pgs=851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.314+0000 7f7c26643640 1 --2- 172.21.15.86:0/3218090297 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7c20106160 0x7f7c20199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.314+0000 7f7c26643640 1 --2- 172.21.15.86:0/3218090297 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7c20101090 0x7f7c201995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.314+0000 7f7c26643640 1 -- 172.21.15.86:0/3218090297 >> 172.21.15.86:0/3218090297 conn(0x7f7c20077f50 msgr2=0x7f7c200ff200 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:01.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.314+0000 7f7c26643640 1 -- 172.21.15.86:0/3218090297 shutdown_connections 2024-04-21T08:41:01.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.314+0000 7f7c26643640 1 -- 172.21.15.86:0/3218090297 wait complete. 2024-04-21T08:41:01.326 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:41:01.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.427+0000 7f7085e4e640 1 -- 172.21.15.86:0/2170993016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70801018e0 msgr2=0x7f7080105da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.427+0000 7f7085e4e640 1 --2- 172.21.15.86:0/2170993016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70801018e0 0x7f7080105da0 secure :-1 s=READY pgs=856 cs=0 l=1 rev1=1 crypto rx=0x7f7068009900 tx=0x7f706802bd60 comp rx=0 tx=0).stop 2024-04-21T08:41:01.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.427+0000 7f7085e4e640 1 -- 172.21.15.86:0/2170993016 shutdown_connections 2024-04-21T08:41:01.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.427+0000 7f7085e4e640 1 --2- 172.21.15.86:0/2170993016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70801062e0 0x7f7080106770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.427+0000 7f7085e4e640 1 --2- 172.21.15.86:0/2170993016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70801018e0 0x7f7080105da0 unknown :-1 s=CLOSED pgs=856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.427+0000 7f7085e4e640 1 --2- 172.21.15.86:0/2170993016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7080100fc0 0x7f70801013a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.427+0000 7f7085e4e640 1 -- 172.21.15.86:0/2170993016 >> 172.21.15.86:0/2170993016 conn(0x7f7080077f50 msgr2=0x7f7080078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:01.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.427+0000 7f7085e4e640 1 -- 172.21.15.86:0/2170993016 shutdown_connections 2024-04-21T08:41:01.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.427+0000 7f7085e4e640 1 -- 172.21.15.86:0/2170993016 wait complete. 2024-04-21T08:41:01.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.427+0000 7f7085e4e640 1 Processor -- start 2024-04-21T08:41:01.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.427+0000 7f7085e4e640 1 -- start start 2024-04-21T08:41:01.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.427+0000 7f7085e4e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7080100fc0 0x7f708019ee60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.427+0000 7f7085e4e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70801018e0 0x7f708019f3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.427+0000 7f7085e4e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70801062e0 0x7f70801a3720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.427+0000 7f7085e4e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7080118820 con 0x7f70801018e0 2024-04-21T08:41:01.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.427+0000 7f7085e4e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f70801186a0 con 0x7f7080100fc0 2024-04-21T08:41:01.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.427+0000 7f7085e4e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f70801189a0 con 0x7f70801062e0 2024-04-21T08:41:01.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.428+0000 7f7077fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70801018e0 0x7f708019f3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.428+0000 7f7077fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70801018e0 0x7f708019f3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60480/0 (socket says 172.21.15.86:60480) 2024-04-21T08:41:01.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.428+0000 7f7077fff640 1 -- 172.21.15.86:0/3874522197 learned_addr learned my addr 172.21.15.86:0/3874522197 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:01.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.428+0000 7f7077fff640 1 -- 172.21.15.86:0/3874522197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70801062e0 msgr2=0x7f70801a3720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.428+0000 7f7077fff640 1 --2- 172.21.15.86:0/3874522197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70801062e0 0x7f70801a3720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.428+0000 7f707f7fe640 1 --2- 172.21.15.86:0/3874522197 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7080100fc0 0x7f708019ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.428+0000 7f7077fff640 1 -- 172.21.15.86:0/3874522197 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7080100fc0 msgr2=0x7f708019ee60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.428+0000 7f7077fff640 1 --2- 172.21.15.86:0/3874522197 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7080100fc0 0x7f708019ee60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.428+0000 7f7077fff640 1 -- 172.21.15.86:0/3874522197 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f70801a3e50 con 0x7f70801018e0 2024-04-21T08:41:01.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.428+0000 7f707f7fe640 1 --2- 172.21.15.86:0/3874522197 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7080100fc0 0x7f708019ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:01.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.428+0000 7f7077fff640 1 --2- 172.21.15.86:0/3874522197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70801018e0 0x7f708019f3a0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f7068005940 tx=0x7f706802d920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:01.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.428+0000 7f707d7fa640 1 -- 172.21.15.86:0/3874522197 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7068043020 con 0x7f70801018e0 2024-04-21T08:41:01.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.428+0000 7f707d7fa640 1 -- 172.21.15.86:0/3874522197 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f706802dd20 con 0x7f70801018e0 2024-04-21T08:41:01.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.428+0000 7f7085e4e640 1 -- 172.21.15.86:0/3874522197 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f70801a40e0 con 0x7f70801018e0 2024-04-21T08:41:01.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.428+0000 7f707d7fa640 1 -- 172.21.15.86:0/3874522197 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7068003b80 con 0x7f70801018e0 2024-04-21T08:41:01.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.428+0000 7f7085e4e640 1 -- 172.21.15.86:0/3874522197 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f70801ab9d0 con 0x7f70801018e0 2024-04-21T08:41:01.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.431+0000 7f707d7fa640 1 -- 172.21.15.86:0/3874522197 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7068046020 con 0x7f70801018e0 2024-04-21T08:41:01.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.431+0000 7f7088a42740 1 Processor -- start 2024-04-21T08:41:01.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.431+0000 7f7088a42740 1 -- start start 2024-04-21T08:41:01.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.431+0000 7f7088a42740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f954a4c4d0 0x55f954a4c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.431+0000 7f7088a42740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f954a4ce80 0x55f954a55430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.431+0000 7f7088a42740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f954a55a00 0x55f954a57e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.431+0000 7f7088a42740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f9549b11f0 con 0x55f954a4ce80 2024-04-21T08:41:01.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.431+0000 7f7088a42740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55f95494e4b0 con 0x55f954a55a00 2024-04-21T08:41:01.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.431+0000 7f7088a42740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f9549ad1b0 con 0x55f954a4c4d0 2024-04-21T08:41:01.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.431+0000 7f707f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f954a4ce80 0x55f954a55430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.431+0000 7f707f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f954a4ce80 0x55f954a55430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60486/0 (socket says 172.21.15.86:60486) 2024-04-21T08:41:01.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.431+0000 7f707f7fe640 1 -- 172.21.15.86:0/2089713248 learned_addr learned my addr 172.21.15.86:0/2089713248 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:01.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.431+0000 7f707ffff640 1 --2- 172.21.15.86:0/2089713248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f954a4c4d0 0x55f954a4c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.431+0000 7f7077fff640 1 --2- 172.21.15.86:0/2089713248 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f954a55a00 0x55f954a57e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.431+0000 7f707d7fa640 1 --2- 172.21.15.86:0/3874522197 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f706c0a3d20 0x7f706c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.432+0000 7f707ffff640 1 --2- 172.21.15.86:0/3874522197 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f706c0a3d20 0x7f706c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.432+0000 7f707f7fe640 1 -- 172.21.15.86:0/2089713248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f954a4c4d0 msgr2=0x55f954a4c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.432+0000 7f707f7fe640 1 --2- 172.21.15.86:0/2089713248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f954a4c4d0 0x55f954a4c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.432+0000 7f707f7fe640 1 -- 172.21.15.86:0/2089713248 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f954a55a00 msgr2=0x55f954a57e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.432+0000 7f707f7fe640 1 --2- 172.21.15.86:0/2089713248 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f954a55a00 0x55f954a57e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.432+0000 7f707f7fe640 1 -- 172.21.15.86:0/2089713248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f9549401b0 con 0x55f954a4ce80 2024-04-21T08:41:01.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.432+0000 7f707d7fa640 1 -- 172.21.15.86:0/3874522197 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f70680f2cc0 con 0x7f70801018e0 2024-04-21T08:41:01.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.432+0000 7f707f7fe640 1 --2- 172.21.15.86:0/2089713248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f954a4ce80 0x55f954a55430 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f707000aa10 tx=0x7f707000aed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f0371987a21aa1c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:01.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.432+0000 7f707ffff640 1 --2- 172.21.15.86:0/3874522197 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f706c0a3d20 0x7f706c0a61e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7078009270 tx=0x7f7078009200 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:01.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.432+0000 7f7075ffb640 1 -- 172.21.15.86:0/2089713248 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7070015070 con 0x55f954a4ce80 2024-04-21T08:41:01.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.432+0000 7f7075ffb640 1 -- 172.21.15.86:0/2089713248 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f707000cc60 con 0x55f954a4ce80 2024-04-21T08:41:01.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.432+0000 7f7075ffb640 1 -- 172.21.15.86:0/2089713248 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70700185c0 con 0x55f954a4ce80 2024-04-21T08:41:01.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.433+0000 7f7088a42740 1 -- 172.21.15.86:0/2089713248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f954a4ce80 msgr2=0x55f954a55430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.433+0000 7f7088a42740 1 --2- 172.21.15.86:0/2089713248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f954a4ce80 0x55f954a55430 secure :-1 s=READY pgs=858 cs=0 l=1 rev1=1 crypto rx=0x7f707000aa10 tx=0x7f707000aed0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.433+0000 7f7088a42740 1 -- 172.21.15.86:0/2089713248 shutdown_connections 2024-04-21T08:41:01.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.433+0000 7f7088a42740 1 --2- 172.21.15.86:0/2089713248 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f954a55a00 0x55f954a57e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.433+0000 7f7088a42740 1 --2- 172.21.15.86:0/2089713248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f954a4ce80 0x55f954a55430 unknown :-1 s=CLOSED pgs=858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.433+0000 7f7088a42740 1 --2- 172.21.15.86:0/2089713248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f954a4c4d0 0x55f954a4c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.433+0000 7f7088a42740 1 -- 172.21.15.86:0/2089713248 >> 172.21.15.86:0/2089713248 conn(0x55f954a46ba0 msgr2=0x55f954a46f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:01.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.433+0000 7f7088a42740 1 -- 172.21.15.86:0/2089713248 shutdown_connections 2024-04-21T08:41:01.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.433+0000 7f7088a42740 1 -- 172.21.15.86:0/2089713248 wait complete. 2024-04-21T08:41:01.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.433+0000 7f7088a42740 1 -- start start 2024-04-21T08:41:01.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.433+0000 7f7088a42740 1 Processor -- start 2024-04-21T08:41:01.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.434+0000 7f7088a42740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f954a4c4d0 0x55f954a55250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.434+0000 7f7088a42740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f954a4ce80 0x55f954a525e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.434+0000 7f7088a42740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f954a55a00 0x55f954a52b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.434+0000 7f7088a42740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f954a58970 con 0x55f954a55a00 2024-04-21T08:41:01.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.434+0000 7f7088a42740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55f95496a500 con 0x55f954a4c4d0 2024-04-21T08:41:01.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.434+0000 7f7088a42740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f954a58af0 con 0x55f954a4ce80 2024-04-21T08:41:01.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.434+0000 7f7088a42740 10 client.0 _ll_register_callbacks cb 0x55f954938b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:01.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.434+0000 7f707ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f954a55a00 0x55f954a52b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.434+0000 7f707ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f954a55a00 0x55f954a52b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60492/0 (socket says 172.21.15.86:60492) 2024-04-21T08:41:01.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.434+0000 7f707ffff640 1 -- 172.21.15.86:0/2706112116 learned_addr learned my addr 172.21.15.86:0/2706112116 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:01.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.434+0000 7f7077fff640 1 --2- 172.21.15.86:0/2706112116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f954a4ce80 0x55f954a525e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.434+0000 7f707f7fe640 1 --2- 172.21.15.86:0/2706112116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f954a4c4d0 0x55f954a55250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.434+0000 7f707ffff640 1 -- 172.21.15.86:0/2706112116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f954a4ce80 msgr2=0x55f954a525e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.434+0000 7f707ffff640 1 --2- 172.21.15.86:0/2706112116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f954a4ce80 0x55f954a525e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.434+0000 7f707ffff640 1 -- 172.21.15.86:0/2706112116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f954a4c4d0 msgr2=0x55f954a55250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.434+0000 7f707ffff640 1 --2- 172.21.15.86:0/2706112116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f954a4c4d0 0x55f954a55250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.434+0000 7f707ffff640 1 -- 172.21.15.86:0/2706112116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f9546c33f0 con 0x55f954a55a00 2024-04-21T08:41:01.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.434+0000 7f707f7fe640 1 --2- 172.21.15.86:0/2706112116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f954a4c4d0 0x55f954a55250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:01.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.434+0000 7f707ffff640 1 --2- 172.21.15.86:0/2706112116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f954a55a00 0x55f954a52b20 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f7078003db0 tx=0x7f7078004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea3ba0303839e89 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:01.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.434+0000 7f705effd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:01.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.434+0000 7f705effd640 1 -- 172.21.15.86:0/2706112116 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f707801c050 con 0x55f954a55a00 2024-04-21T08:41:01.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.434+0000 7f705effd640 1 -- 172.21.15.86:0/2706112116 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f70780047b0 con 0x55f954a55a00 2024-04-21T08:41:01.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.434+0000 7f7088a42740 1 -- 172.21.15.86:0/2706112116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f707000cc60 con 0x55f954a55a00 2024-04-21T08:41:01.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.434+0000 7f705effd640 1 -- 172.21.15.86:0/2706112116 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f707802f050 con 0x55f954a55a00 2024-04-21T08:41:01.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.435+0000 7f7088a42740 10 client.16740 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:01.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.435+0000 7f7088a42740 1 -- 172.21.15.86:0/2706112116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f95478d7d0 con 0x55f954a55a00 2024-04-21T08:41:01.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.435+0000 7f705effd640 1 -- 172.21.15.86:0/2706112116 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f707000cc60 con 0x55f954a55a00 2024-04-21T08:41:01.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.435+0000 7f705effd640 1 -- 172.21.15.86:0/2706112116 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f707800f420 con 0x55f954a55a00 2024-04-21T08:41:01.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.435+0000 7f7088a42740 10 client.16740 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:01.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.435+0000 7f7088a42740 10 client.16740 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:41:01.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.435+0000 7f7088a42740 20 client.16740 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:01.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.435+0000 7f7088a42740 1 --2- 172.21.15.86:0/2706112116 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55f954a65c20 0x55f954a68090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.435+0000 7f7088a42740 4 client.16740 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:01.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.435+0000 7f7088a42740 1 -- 172.21.15.86:0/2706112116 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f954950930 con 0x55f954a65c20 2024-04-21T08:41:01.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.435+0000 7f707f7fe640 1 --2- 172.21.15.86:0/2706112116 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55f954a65c20 0x55f954a68090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.436+0000 7f707f7fe640 1 --2- 172.21.15.86:0/2706112116 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55f954a65c20 0x55f954a68090 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7005480515631b98 server_cookie=b0d9ecf6234323b4 in_seq=0 out_seq=0 2024-04-21T08:41:01.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.436+0000 7f705effd640 10 client.16740 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:41:01.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.436+0000 7f705effd640 1 -- 172.21.15.86:0/2706112116 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f954950930 con 0x55f954a65c20 2024-04-21T08:41:01.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.436+0000 7f705effd640 10 client.16740 handle_command_reply: tid=0 2024-04-21T08:41:01.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.436+0000 7f7088a42740 1 client.16740 shutdown 2024-04-21T08:41:01.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.437+0000 7f7088a42740 1 -- 172.21.15.86:0/2706112116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f954a55a00 msgr2=0x55f954a52b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.437+0000 7f7088a42740 1 --2- 172.21.15.86:0/2706112116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f954a55a00 0x55f954a52b20 secure :-1 s=READY pgs=859 cs=0 l=1 rev1=1 crypto rx=0x7f7078003db0 tx=0x7f7078004a40 comp rx=0 tx=0).stop 2024-04-21T08:41:01.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.437+0000 7f7088a42740 1 -- 172.21.15.86:0/2706112116 shutdown_connections 2024-04-21T08:41:01.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.437+0000 7f7088a42740 1 --2- 172.21.15.86:0/2706112116 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55f954a65c20 0x55f954a68090 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.437+0000 7f7088a42740 1 --2- 172.21.15.86:0/2706112116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f954a55a00 0x55f954a52b20 unknown :-1 s=CLOSED pgs=859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.437+0000 7f7088a42740 1 --2- 172.21.15.86:0/2706112116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f954a4ce80 0x55f954a525e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.437+0000 7f705effd640 0 client.16740 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:41:01.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.437+0000 7f7088a42740 1 --2- 172.21.15.86:0/2706112116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f954a4c4d0 0x55f954a55250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.437+0000 7f7088a42740 1 -- 172.21.15.86:0/2706112116 >> 172.21.15.86:0/2706112116 conn(0x55f954a46ba0 msgr2=0x55f954a488f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:01.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.437+0000 7f7088a42740 1 -- 172.21.15.86:0/2706112116 shutdown_connections 2024-04-21T08:41:01.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.437+0000 7f7088a42740 1 -- 172.21.15.86:0/2706112116 wait complete. 2024-04-21T08:41:01.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.437+0000 7f7088a42740 20 client.16740 trim_cache size 0 max 16384 2024-04-21T08:41:01.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.450+0000 7f7088a42740 1 Processor -- start 2024-04-21T08:41:01.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.450+0000 7f7088a42740 1 -- start start 2024-04-21T08:41:01.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.450+0000 7f7088a42740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f954a4c4d0 0x55f954a61860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.450+0000 7f7088a42740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f954a4ce80 0x55f954a4d9c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.450+0000 7f7088a42740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f954a55a00 0x55f954a54d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.450+0000 7f7088a42740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f70801b58f0 con 0x55f954a4c4d0 2024-04-21T08:41:01.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.450+0000 7f7088a42740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f70801b7430 con 0x55f954a4ce80 2024-04-21T08:41:01.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.450+0000 7f7088a42740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f70801b5fc0 con 0x55f954a55a00 2024-04-21T08:41:01.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.450+0000 7f707f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f954a4c4d0 0x55f954a61860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.450+0000 7f707f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f954a4c4d0 0x55f954a61860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60506/0 (socket says 172.21.15.86:60506) 2024-04-21T08:41:01.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.450+0000 7f707f7fe640 1 -- 172.21.15.86:0/2087318241 learned_addr learned my addr 172.21.15.86:0/2087318241 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:01.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.450+0000 7f7077fff640 1 --2- 172.21.15.86:0/2087318241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f954a4ce80 0x55f954a4d9c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.450+0000 7f707f7fe640 1 -- 172.21.15.86:0/2087318241 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f954a55a00 msgr2=0x55f954a54d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.450+0000 7f707ffff640 1 --2- 172.21.15.86:0/2087318241 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f954a55a00 0x55f954a54d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.450+0000 7f707f7fe640 1 --2- 172.21.15.86:0/2087318241 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f954a55a00 0x55f954a54d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.450+0000 7f707f7fe640 1 -- 172.21.15.86:0/2087318241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f954a4ce80 msgr2=0x55f954a4d9c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.450+0000 7f707f7fe640 1 --2- 172.21.15.86:0/2087318241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f954a4ce80 0x55f954a4d9c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.450+0000 7f707f7fe640 1 -- 172.21.15.86:0/2087318241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f954a582b0 con 0x55f954a4c4d0 2024-04-21T08:41:01.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.450+0000 7f7077fff640 1 --2- 172.21.15.86:0/2087318241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f954a4ce80 0x55f954a4d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:01.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.450+0000 7f707ffff640 1 --2- 172.21.15.86:0/2087318241 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f954a55a00 0x55f954a54d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:01.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.451+0000 7f707f7fe640 1 --2- 172.21.15.86:0/2087318241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f954a4c4d0 0x55f954a61860 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f707000cb00 tx=0x7f7070011e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de2ad04e20fbd692 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:01.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.451+0000 7f70757fa640 1 -- 172.21.15.86:0/2087318241 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7070015040 con 0x55f954a4c4d0 2024-04-21T08:41:01.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.451+0000 7f70757fa640 1 -- 172.21.15.86:0/2087318241 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7070006bc0 con 0x55f954a4c4d0 2024-04-21T08:41:01.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.451+0000 7f70757fa640 1 -- 172.21.15.86:0/2087318241 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7070026460 con 0x55f954a4c4d0 2024-04-21T08:41:01.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.451+0000 7f7088a42740 1 -- 172.21.15.86:0/2087318241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f954a4c4d0 msgr2=0x55f954a61860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.451+0000 7f7088a42740 1 --2- 172.21.15.86:0/2087318241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f954a4c4d0 0x55f954a61860 secure :-1 s=READY pgs=860 cs=0 l=1 rev1=1 crypto rx=0x7f707000cb00 tx=0x7f7070011e60 comp rx=0 tx=0).stop 2024-04-21T08:41:01.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.451+0000 7f7088a42740 1 -- 172.21.15.86:0/2087318241 shutdown_connections 2024-04-21T08:41:01.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.451+0000 7f7088a42740 1 --2- 172.21.15.86:0/2087318241 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f954a55a00 0x55f954a54d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.451+0000 7f7088a42740 1 --2- 172.21.15.86:0/2087318241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f954a4ce80 0x55f954a4d9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.451+0000 7f7088a42740 1 --2- 172.21.15.86:0/2087318241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f954a4c4d0 0x55f954a61860 unknown :-1 s=CLOSED pgs=860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.451+0000 7f7088a42740 1 -- 172.21.15.86:0/2087318241 >> 172.21.15.86:0/2087318241 conn(0x55f954a46ba0 msgr2=0x55f954a4d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:01.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.451+0000 7f7088a42740 1 -- 172.21.15.86:0/2087318241 shutdown_connections 2024-04-21T08:41:01.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.451+0000 7f7088a42740 1 -- 172.21.15.86:0/2087318241 wait complete. 2024-04-21T08:41:01.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.452+0000 7f7088a42740 1 -- start start 2024-04-21T08:41:01.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.452+0000 7f7088a42740 1 Processor -- start 2024-04-21T08:41:01.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.452+0000 7f7088a42740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f954a4c4d0 0x55f954a5df60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.452+0000 7f7088a42740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f954a4ce80 0x55f954a4ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.452+0000 7f7088a42740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f954a55a00 0x55f954a4f390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.452+0000 7f7088a42740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f70801b6d60 con 0x55f954a4c4d0 2024-04-21T08:41:01.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.452+0000 7f7088a42740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f70801b6690 con 0x55f954a55a00 2024-04-21T08:41:01.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.452+0000 7f7088a42740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f70801b38b0 con 0x55f954a4ce80 2024-04-21T08:41:01.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.452+0000 7f7088a42740 10 client.0 _ll_register_callbacks cb 0x7f7078003a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:01.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.452+0000 7f707f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f954a4c4d0 0x55f954a5df60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.452+0000 7f707f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f954a4c4d0 0x55f954a5df60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60518/0 (socket says 172.21.15.86:60518) 2024-04-21T08:41:01.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.452+0000 7f707f7fe640 1 -- 172.21.15.86:0/3752854486 learned_addr learned my addr 172.21.15.86:0/3752854486 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:01.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.452+0000 7f707ffff640 1 --2- 172.21.15.86:0/3752854486 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f954a55a00 0x55f954a4f390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.452+0000 7f7077fff640 1 --2- 172.21.15.86:0/3752854486 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f954a4ce80 0x55f954a4ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.452+0000 7f707f7fe640 1 -- 172.21.15.86:0/3752854486 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f954a4ce80 msgr2=0x55f954a4ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.452+0000 7f707f7fe640 1 --2- 172.21.15.86:0/3752854486 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f954a4ce80 0x55f954a4ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.452+0000 7f707f7fe640 1 -- 172.21.15.86:0/3752854486 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f954a55a00 msgr2=0x55f954a4f390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.452+0000 7f707f7fe640 1 --2- 172.21.15.86:0/3752854486 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f954a55a00 0x55f954a4f390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.452+0000 7f707f7fe640 1 -- 172.21.15.86:0/3752854486 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f954a4f8d0 con 0x55f954a4c4d0 2024-04-21T08:41:01.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.453+0000 7f707f7fe640 1 --2- 172.21.15.86:0/3752854486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f954a4c4d0 0x55f954a5df60 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f7070000c00 tx=0x7f707000c520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb741f849bb5ff83 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:01.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.453+0000 7f705effd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:01.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.453+0000 7f705effd640 1 -- 172.21.15.86:0/3752854486 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7070015070 con 0x55f954a4c4d0 2024-04-21T08:41:01.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.453+0000 7f705effd640 1 -- 172.21.15.86:0/3752854486 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f70700116c0 con 0x55f954a4c4d0 2024-04-21T08:41:01.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.453+0000 7f7088a42740 1 -- 172.21.15.86:0/3752854486 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7070006bc0 con 0x55f954a4c4d0 2024-04-21T08:41:01.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.453+0000 7f705effd640 1 -- 172.21.15.86:0/3752854486 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f70700118c0 con 0x55f954a4c4d0 2024-04-21T08:41:01.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.453+0000 7f7088a42740 10 client.16746 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:01.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.453+0000 7f7088a42740 1 -- 172.21.15.86:0/3752854486 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f954acbe00 con 0x55f954a4c4d0 2024-04-21T08:41:01.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.453+0000 7f705effd640 1 -- 172.21.15.86:0/3752854486 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7070006bc0 con 0x55f954a4c4d0 2024-04-21T08:41:01.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.453+0000 7f705effd640 1 -- 172.21.15.86:0/3752854486 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f707002f070 con 0x55f954a4c4d0 2024-04-21T08:41:01.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.453+0000 7f7088a42740 10 client.16746 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:01.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.453+0000 7f7088a42740 10 client.16746 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:41:01.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.453+0000 7f7088a42740 20 client.16746 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:01.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.453+0000 7f7088a42740 1 --2- 172.21.15.86:0/3752854486 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55f954acc1e0 0x55f954ace5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.453+0000 7f7088a42740 4 client.16746 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:41:01.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.453+0000 7f7088a42740 1 -- 172.21.15.86:0/3752854486 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f954a52eb0 con 0x55f954acc1e0 2024-04-21T08:41:01.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.454+0000 7f7077fff640 1 --2- 172.21.15.86:0/3752854486 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55f954acc1e0 0x55f954ace5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.454+0000 7f705effd640 10 client.16746 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:41:01.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.454+0000 7f7077fff640 1 --2- 172.21.15.86:0/3752854486 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55f954acc1e0 0x55f954ace5d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=43a96c314157a52a server_cookie=c0491001f852baa8 in_seq=0 out_seq=0 2024-04-21T08:41:01.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.454+0000 7f705effd640 1 -- 172.21.15.86:0/3752854486 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f954a52eb0 con 0x55f954acc1e0 2024-04-21T08:41:01.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.454+0000 7f705effd640 10 client.16746 handle_command_reply: tid=0 2024-04-21T08:41:01.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.454+0000 7f7088a42740 1 client.16746 shutdown 2024-04-21T08:41:01.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.454+0000 7f7088a42740 1 -- 172.21.15.86:0/3752854486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f954a4c4d0 msgr2=0x55f954a5df60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.454+0000 7f7088a42740 1 --2- 172.21.15.86:0/3752854486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f954a4c4d0 0x55f954a5df60 secure :-1 s=READY pgs=861 cs=0 l=1 rev1=1 crypto rx=0x7f7070000c00 tx=0x7f707000c520 comp rx=0 tx=0).stop 2024-04-21T08:41:01.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.454+0000 7f7088a42740 1 -- 172.21.15.86:0/3752854486 shutdown_connections 2024-04-21T08:41:01.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.454+0000 7f7088a42740 1 --2- 172.21.15.86:0/3752854486 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f954a55a00 0x55f954a4f390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.454+0000 7f7088a42740 1 --2- 172.21.15.86:0/3752854486 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f954a4ce80 0x55f954a4ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.454+0000 7f7088a42740 1 --2- 172.21.15.86:0/3752854486 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55f954acc1e0 0x55f954ace5d0 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.454+0000 7f7088a42740 1 --2- 172.21.15.86:0/3752854486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f954a4c4d0 0x55f954a5df60 unknown :-1 s=CLOSED pgs=861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.454+0000 7f705effd640 0 client.16746 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:41:01.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.454+0000 7f7088a42740 1 -- 172.21.15.86:0/3752854486 >> 172.21.15.86:0/3752854486 conn(0x55f954a46ba0 msgr2=0x55f954a4d260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:01.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.454+0000 7f7088a42740 1 -- 172.21.15.86:0/3752854486 shutdown_connections 2024-04-21T08:41:01.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.454+0000 7f7088a42740 1 -- 172.21.15.86:0/3752854486 wait complete. 2024-04-21T08:41:01.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.455+0000 7f7088a42740 20 client.16746 trim_cache size 0 max 16384 2024-04-21T08:41:01.463 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:41:01.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.455+0000 7f7085e4e640 1 -- 172.21.15.86:0/3874522197 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f706c0a3d20 msgr2=0x7f706c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.455+0000 7f7085e4e640 1 --2- 172.21.15.86:0/3874522197 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f706c0a3d20 0x7f706c0a61e0 secure :-1 s=READY pgs=124 cs=0 l=1 rev1=1 crypto rx=0x7f7078009270 tx=0x7f7078009200 comp rx=0 tx=0).stop 2024-04-21T08:41:01.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.455+0000 7f7085e4e640 1 -- 172.21.15.86:0/3874522197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70801018e0 msgr2=0x7f708019f3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.455+0000 7f7085e4e640 1 --2- 172.21.15.86:0/3874522197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70801018e0 0x7f708019f3a0 secure :-1 s=READY pgs=857 cs=0 l=1 rev1=1 crypto rx=0x7f7068005940 tx=0x7f706802d920 comp rx=0 tx=0).stop 2024-04-21T08:41:01.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.455+0000 7f7085e4e640 1 -- 172.21.15.86:0/3874522197 shutdown_connections 2024-04-21T08:41:01.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.455+0000 7f7085e4e640 1 --2- 172.21.15.86:0/3874522197 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f706c0a3d20 0x7f706c0a61e0 unknown :-1 s=CLOSED pgs=124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.455+0000 7f7085e4e640 1 --2- 172.21.15.86:0/3874522197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f70801062e0 0x7f70801a3720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.455+0000 7f7085e4e640 1 --2- 172.21.15.86:0/3874522197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f70801018e0 0x7f708019f3a0 unknown :-1 s=CLOSED pgs=857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.455+0000 7f7085e4e640 1 --2- 172.21.15.86:0/3874522197 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7080100fc0 0x7f708019ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.455+0000 7f7085e4e640 1 -- 172.21.15.86:0/3874522197 >> 172.21.15.86:0/3874522197 conn(0x7f7080077f50 msgr2=0x7f7080103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:01.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.455+0000 7f7085e4e640 1 -- 172.21.15.86:0/3874522197 shutdown_connections 2024-04-21T08:41:01.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.455+0000 7f7085e4e640 1 -- 172.21.15.86:0/3874522197 wait complete. 2024-04-21T08:41:01.466 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:01 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/950526153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:41:01.467 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:41:01.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.567+0000 7fe12d985640 1 -- 172.21.15.86:0/2972934842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe128103af0 msgr2=0x7fe12810bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.567+0000 7fe12d985640 1 --2- 172.21.15.86:0/2972934842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe128103af0 0x7fe12810bfd0 secure :-1 s=READY pgs=862 cs=0 l=1 rev1=1 crypto rx=0x7fe110009900 tx=0x7fe11002bd60 comp rx=0 tx=0).stop 2024-04-21T08:41:01.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.567+0000 7fe12d985640 1 -- 172.21.15.86:0/2972934842 shutdown_connections 2024-04-21T08:41:01.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.567+0000 7fe12d985640 1 --2- 172.21.15.86:0/2972934842 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe12810c510 0x7fe12810e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.567+0000 7fe12d985640 1 --2- 172.21.15.86:0/2972934842 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe128103af0 0x7fe12810bfd0 unknown :-1 s=CLOSED pgs=862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.567+0000 7fe12d985640 1 --2- 172.21.15.86:0/2972934842 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe1281031d0 0x7fe1281035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.567+0000 7fe12d985640 1 -- 172.21.15.86:0/2972934842 >> 172.21.15.86:0/2972934842 conn(0x7fe128077f50 msgr2=0x7fe128078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:01.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.567+0000 7fe12d985640 1 -- 172.21.15.86:0/2972934842 shutdown_connections 2024-04-21T08:41:01.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.567+0000 7fe12d985640 1 -- 172.21.15.86:0/2972934842 wait complete. 2024-04-21T08:41:01.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.568+0000 7fe12d985640 1 Processor -- start 2024-04-21T08:41:01.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.568+0000 7fe12d985640 1 -- start start 2024-04-21T08:41:01.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.568+0000 7fe12d985640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe1281031d0 0x7fe12819e590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.568+0000 7fe12d985640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe128103af0 0x7fe12819ead0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.568+0000 7fe12d985640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe12810c510 0x7fe128198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.568+0000 7fe12d985640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe128112200 con 0x7fe128103af0 2024-04-21T08:41:01.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.568+0000 7fe12d985640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe128112080 con 0x7fe1281031d0 2024-04-21T08:41:01.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.568+0000 7fe12d985640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe128112380 con 0x7fe12810c510 2024-04-21T08:41:01.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.568+0000 7fe11e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe128103af0 0x7fe12819ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.568+0000 7fe11e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe128103af0 0x7fe12819ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60540/0 (socket says 172.21.15.86:60540) 2024-04-21T08:41:01.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.568+0000 7fe11e7fc640 1 -- 172.21.15.86:0/4053234154 learned_addr learned my addr 172.21.15.86:0/4053234154 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:01.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.569+0000 7fe1277fe640 1 --2- 172.21.15.86:0/4053234154 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe12810c510 0x7fe128198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.569+0000 7fe126ffd640 1 --2- 172.21.15.86:0/4053234154 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe1281031d0 0x7fe12819e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.569+0000 7fe11e7fc640 1 -- 172.21.15.86:0/4053234154 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe12810c510 msgr2=0x7fe128198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.569+0000 7fe11e7fc640 1 --2- 172.21.15.86:0/4053234154 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe12810c510 0x7fe128198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.569+0000 7fe11e7fc640 1 -- 172.21.15.86:0/4053234154 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe1281031d0 msgr2=0x7fe12819e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.569+0000 7fe11e7fc640 1 --2- 172.21.15.86:0/4053234154 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe1281031d0 0x7fe12819e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.569+0000 7fe11e7fc640 1 -- 172.21.15.86:0/4053234154 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe128198f10 con 0x7fe128103af0 2024-04-21T08:41:01.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.569+0000 7fe126ffd640 1 --2- 172.21.15.86:0/4053234154 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe1281031d0 0x7fe12819e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:01.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.569+0000 7fe11e7fc640 1 --2- 172.21.15.86:0/4053234154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe128103af0 0x7fe12819ead0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fe110005940 tx=0x7fe11002d9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:01.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.569+0000 7fe12c983640 1 -- 172.21.15.86:0/4053234154 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe110043020 con 0x7fe128103af0 2024-04-21T08:41:01.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.569+0000 7fe12d985640 1 -- 172.21.15.86:0/4053234154 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe1281991a0 con 0x7fe128103af0 2024-04-21T08:41:01.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.569+0000 7fe12c983640 1 -- 172.21.15.86:0/4053234154 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe11002dda0 con 0x7fe128103af0 2024-04-21T08:41:01.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.569+0000 7fe12c983640 1 -- 172.21.15.86:0/4053234154 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe110003bf0 con 0x7fe128103af0 2024-04-21T08:41:01.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.569+0000 7fe12d985640 1 -- 172.21.15.86:0/4053234154 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe1281a5510 con 0x7fe128103af0 2024-04-21T08:41:01.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.571+0000 7fe12c983640 1 -- 172.21.15.86:0/4053234154 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe110046020 con 0x7fe128103af0 2024-04-21T08:41:01.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.571+0000 7fe130575740 1 Processor -- start 2024-04-21T08:41:01.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.571+0000 7fe130575740 1 -- start start 2024-04-21T08:41:01.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.572+0000 7fe130575740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f39a19f4d0 0x55f39a19f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.572+0000 7fe130575740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f39a19fe80 0x55f39a1a8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.572+0000 7fe130575740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39a1a8a00 0x55f39a1aae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.572+0000 7fe130575740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f39a1041f0 con 0x55f39a1a8a00 2024-04-21T08:41:01.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.572+0000 7fe130575740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55f39a0a14b0 con 0x55f39a19fe80 2024-04-21T08:41:01.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.572+0000 7fe130575740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f39a1001b0 con 0x55f39a19f4d0 2024-04-21T08:41:01.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.572+0000 7fe12c983640 1 --2- 172.21.15.86:0/4053234154 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe1140a3d20 0x7fe1140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.572+0000 7fe11e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39a1a8a00 0x55f39a1aae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.572+0000 7fe1277fe640 1 --2- 172.21.15.86:0/4053234154 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe1140a3d20 0x7fe1140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.572+0000 7fe11e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39a1a8a00 0x55f39a1aae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60556/0 (socket says 172.21.15.86:60556) 2024-04-21T08:41:01.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.572+0000 7fe11e7fc640 1 -- 172.21.15.86:0/1737444920 learned_addr learned my addr 172.21.15.86:0/1737444920 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:01.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.572+0000 7fe1277fe640 1 --2- 172.21.15.86:0/1737444920 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f39a19f4d0 0x55f39a19f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.572+0000 7fe126ffd640 1 --2- 172.21.15.86:0/1737444920 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f39a19fe80 0x55f39a1a8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.572+0000 7fe12c983640 1 -- 172.21.15.86:0/4053234154 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fe1100f2ee0 con 0x7fe128103af0 2024-04-21T08:41:01.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.572+0000 7fe11e7fc640 1 -- 172.21.15.86:0/1737444920 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f39a19f4d0 msgr2=0x55f39a19f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.572+0000 7fe11e7fc640 1 --2- 172.21.15.86:0/1737444920 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f39a19f4d0 0x55f39a19f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.572+0000 7fe11e7fc640 1 -- 172.21.15.86:0/1737444920 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f39a19fe80 msgr2=0x55f39a1a8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.572+0000 7fe11e7fc640 1 --2- 172.21.15.86:0/1737444920 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f39a19fe80 0x55f39a1a8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.572+0000 7fe11e7fc640 1 -- 172.21.15.86:0/1737444920 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f39a0931b0 con 0x55f39a1a8a00 2024-04-21T08:41:01.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.572+0000 7fe1277fe640 1 --2- 172.21.15.86:0/4053234154 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe1140a3d20 0x7fe1140a61e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe120005c40 tx=0x7fe12000a920 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:01.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.572+0000 7fe11e7fc640 1 --2- 172.21.15.86:0/1737444920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39a1a8a00 0x55f39a1aae50 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe110000c00 tx=0x7fe11002df50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=47164c0cf55df246 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:01.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.573+0000 7fe11dffb640 1 -- 172.21.15.86:0/1737444920 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1100bf320 con 0x55f39a1a8a00 2024-04-21T08:41:01.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.573+0000 7fe11dffb640 1 -- 172.21.15.86:0/1737444920 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1100bec20 con 0x55f39a1a8a00 2024-04-21T08:41:01.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.573+0000 7fe11dffb640 1 -- 172.21.15.86:0/1737444920 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe1100df0c0 con 0x55f39a1a8a00 2024-04-21T08:41:01.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.573+0000 7fe130575740 1 -- 172.21.15.86:0/1737444920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39a1a8a00 msgr2=0x55f39a1aae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.573+0000 7fe130575740 1 --2- 172.21.15.86:0/1737444920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39a1a8a00 0x55f39a1aae50 secure :-1 s=READY pgs=864 cs=0 l=1 rev1=1 crypto rx=0x7fe110000c00 tx=0x7fe11002df50 comp rx=0 tx=0).stop 2024-04-21T08:41:01.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.573+0000 7fe130575740 1 -- 172.21.15.86:0/1737444920 shutdown_connections 2024-04-21T08:41:01.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.573+0000 7fe130575740 1 --2- 172.21.15.86:0/1737444920 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39a1a8a00 0x55f39a1aae50 unknown :-1 s=CLOSED pgs=864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.573+0000 7fe130575740 1 --2- 172.21.15.86:0/1737444920 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f39a19fe80 0x55f39a1a8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.573+0000 7fe130575740 1 --2- 172.21.15.86:0/1737444920 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f39a19f4d0 0x55f39a19f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.573+0000 7fe130575740 1 -- 172.21.15.86:0/1737444920 >> 172.21.15.86:0/1737444920 conn(0x55f39a199ba0 msgr2=0x55f39a199f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:01.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.573+0000 7fe130575740 1 -- 172.21.15.86:0/1737444920 shutdown_connections 2024-04-21T08:41:01.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.573+0000 7fe130575740 1 -- 172.21.15.86:0/1737444920 wait complete. 2024-04-21T08:41:01.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.574+0000 7fe130575740 1 -- start start 2024-04-21T08:41:01.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.574+0000 7fe130575740 1 Processor -- start 2024-04-21T08:41:01.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.574+0000 7fe130575740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f39a19f4d0 0x55f39a1a8390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.574+0000 7fe130575740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f39a19fe80 0x55f39a1a5720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.574+0000 7fe130575740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39a1a8a00 0x55f39a1a5d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.574+0000 7fe130575740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f39a0a14b0 con 0x55f39a1a8a00 2024-04-21T08:41:01.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.574+0000 7fe130575740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55f39a0bd500 con 0x55f39a19f4d0 2024-04-21T08:41:01.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.574+0000 7fe130575740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f39a1041f0 con 0x55f39a19fe80 2024-04-21T08:41:01.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.574+0000 7fe130575740 10 client.0 _ll_register_callbacks cb 0x55f39a08bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:01.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.574+0000 7fe1277fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39a1a8a00 0x55f39a1a5d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.574+0000 7fe1277fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39a1a8a00 0x55f39a1a5d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60564/0 (socket says 172.21.15.86:60564) 2024-04-21T08:41:01.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.574+0000 7fe1277fe640 1 -- 172.21.15.86:0/2135133945 learned_addr learned my addr 172.21.15.86:0/2135133945 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:01.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.574+0000 7fe126ffd640 1 --2- 172.21.15.86:0/2135133945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f39a19f4d0 0x55f39a1a8390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.574+0000 7fe11e7fc640 1 --2- 172.21.15.86:0/2135133945 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f39a19fe80 0x55f39a1a5720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.574+0000 7fe1277fe640 1 -- 172.21.15.86:0/2135133945 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f39a19fe80 msgr2=0x55f39a1a5720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.574+0000 7fe1277fe640 1 --2- 172.21.15.86:0/2135133945 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f39a19fe80 0x55f39a1a5720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.574+0000 7fe1277fe640 1 -- 172.21.15.86:0/2135133945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f39a19f4d0 msgr2=0x55f39a1a8390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.574+0000 7fe1277fe640 1 --2- 172.21.15.86:0/2135133945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f39a19f4d0 0x55f39a1a8390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.574+0000 7fe1277fe640 1 -- 172.21.15.86:0/2135133945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f399e163f0 con 0x55f39a1a8a00 2024-04-21T08:41:01.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.575+0000 7fe1277fe640 1 --2- 172.21.15.86:0/2135133945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39a1a8a00 0x55f39a1a5d30 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fe12000a640 tx=0x7fe120018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f47baf9ff6abe79 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:01.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.575+0000 7fe106ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:01.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.575+0000 7fe106ffd640 1 -- 172.21.15.86:0/2135133945 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe120025070 con 0x55f39a1a8a00 2024-04-21T08:41:01.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.575+0000 7fe106ffd640 1 -- 172.21.15.86:0/2135133945 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fe120003b70 con 0x55f39a1a8a00 2024-04-21T08:41:01.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.575+0000 7fe130575740 1 -- 172.21.15.86:0/2135133945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe1100bec20 con 0x55f39a1a8a00 2024-04-21T08:41:01.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.575+0000 7fe106ffd640 1 -- 172.21.15.86:0/2135133945 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe120003d70 con 0x55f39a1a8a00 2024-04-21T08:41:01.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.575+0000 7fe130575740 10 client.16758 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:01.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.575+0000 7fe130575740 1 -- 172.21.15.86:0/2135133945 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f399ee07d0 con 0x55f39a1a8a00 2024-04-21T08:41:01.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.575+0000 7fe106ffd640 1 -- 172.21.15.86:0/2135133945 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe1100bec20 con 0x55f39a1a8a00 2024-04-21T08:41:01.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.575+0000 7fe106ffd640 1 -- 172.21.15.86:0/2135133945 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe120005020 con 0x55f39a1a8a00 2024-04-21T08:41:01.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.575+0000 7fe130575740 10 client.16758 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:01.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.575+0000 7fe130575740 10 client.16758 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:01.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.575+0000 7fe130575740 20 client.16758 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:01.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.575+0000 7fe130575740 1 --2- 172.21.15.86:0/2135133945 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f39a1b86d0 0x55f39a1bab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.575+0000 7fe130575740 4 client.16758 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:01.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.575+0000 7fe130575740 1 -- 172.21.15.86:0/2135133945 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f39a0a3930 con 0x55f39a1b86d0 2024-04-21T08:41:01.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.576+0000 7fe126ffd640 1 --2- 172.21.15.86:0/2135133945 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f39a1b86d0 0x55f39a1bab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.576+0000 7fe126ffd640 1 --2- 172.21.15.86:0/2135133945 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f39a1b86d0 0x55f39a1bab30 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b3e785417a3cc97 server_cookie=a49f41ab14a6327b in_seq=0 out_seq=0 2024-04-21T08:41:01.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.576+0000 7fe106ffd640 10 client.16758 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:01.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.577+0000 7fe106ffd640 1 -- 172.21.15.86:0/2135133945 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f39a0a3930 con 0x55f39a1b86d0 2024-04-21T08:41:01.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.577+0000 7fe106ffd640 10 client.16758 handle_command_reply: tid=0 2024-04-21T08:41:01.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.577+0000 7fe130575740 1 client.16758 shutdown 2024-04-21T08:41:01.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.577+0000 7fe130575740 1 -- 172.21.15.86:0/2135133945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39a1a8a00 msgr2=0x55f39a1a5d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.577+0000 7fe130575740 1 --2- 172.21.15.86:0/2135133945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39a1a8a00 0x55f39a1a5d30 secure :-1 s=READY pgs=865 cs=0 l=1 rev1=1 crypto rx=0x7fe12000a640 tx=0x7fe120018000 comp rx=0 tx=0).stop 2024-04-21T08:41:01.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.577+0000 7fe130575740 1 -- 172.21.15.86:0/2135133945 shutdown_connections 2024-04-21T08:41:01.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.577+0000 7fe130575740 1 --2- 172.21.15.86:0/2135133945 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f39a1b86d0 0x55f39a1bab30 crc :-1 s=READY pgs=95 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.577+0000 7fe130575740 1 --2- 172.21.15.86:0/2135133945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39a1a8a00 0x55f39a1a5d30 unknown :-1 s=CLOSED pgs=865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.577+0000 7fe130575740 1 --2- 172.21.15.86:0/2135133945 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f39a19fe80 0x55f39a1a5720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.577+0000 7fe130575740 1 --2- 172.21.15.86:0/2135133945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f39a19f4d0 0x55f39a1a8390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.577+0000 7fe106ffd640 0 client.16758 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:01.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.577+0000 7fe130575740 1 -- 172.21.15.86:0/2135133945 >> 172.21.15.86:0/2135133945 conn(0x55f39a199ba0 msgr2=0x55f39a19b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:01.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.577+0000 7fe130575740 1 -- 172.21.15.86:0/2135133945 shutdown_connections 2024-04-21T08:41:01.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.577+0000 7fe130575740 1 -- 172.21.15.86:0/2135133945 wait complete. 2024-04-21T08:41:01.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.577+0000 7fe130575740 20 client.16758 trim_cache size 0 max 16384 2024-04-21T08:41:01.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.590+0000 7fe130575740 1 Processor -- start 2024-04-21T08:41:01.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.590+0000 7fe130575740 1 -- start start 2024-04-21T08:41:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.590+0000 7fe130575740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39a19f4d0 0x55f39a1b4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.590+0000 7fe130575740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f39a19fe80 0x55f39a19b700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.590+0000 7fe130575740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f39a1a8a00 0x55f39a1a7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.590+0000 7fe130575740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe1281af370 con 0x55f39a19f4d0 2024-04-21T08:41:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.590+0000 7fe130575740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe1281b0eb0 con 0x55f39a19fe80 2024-04-21T08:41:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.590+0000 7fe130575740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe1281afa40 con 0x55f39a1a8a00 2024-04-21T08:41:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.590+0000 7fe126ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39a19f4d0 0x55f39a1b4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.590+0000 7fe126ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39a19f4d0 0x55f39a1b4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60574/0 (socket says 172.21.15.86:60574) 2024-04-21T08:41:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.590+0000 7fe126ffd640 1 -- 172.21.15.86:0/2546900654 learned_addr learned my addr 172.21.15.86:0/2546900654 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.591+0000 7fe11e7fc640 1 --2- 172.21.15.86:0/2546900654 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f39a19fe80 0x55f39a19b700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.591+0000 7fe126ffd640 1 -- 172.21.15.86:0/2546900654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f39a1a8a00 msgr2=0x55f39a1a7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.596 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.591+0000 7fe1277fe640 1 --2- 172.21.15.86:0/2546900654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f39a1a8a00 0x55f39a1a7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.591+0000 7fe126ffd640 1 --2- 172.21.15.86:0/2546900654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f39a1a8a00 0x55f39a1a7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.591+0000 7fe126ffd640 1 -- 172.21.15.86:0/2546900654 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f39a19fe80 msgr2=0x55f39a19b700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.591+0000 7fe126ffd640 1 --2- 172.21.15.86:0/2546900654 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f39a19fe80 0x55f39a19b700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.591+0000 7fe126ffd640 1 -- 172.21.15.86:0/2546900654 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f39a19fc10 con 0x55f39a19f4d0 2024-04-21T08:41:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.591+0000 7fe1277fe640 1 --2- 172.21.15.86:0/2546900654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f39a1a8a00 0x55f39a1a7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.591+0000 7fe11e7fc640 1 --2- 172.21.15.86:0/2546900654 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f39a19fe80 0x55f39a19b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.591+0000 7fe126ffd640 1 --2- 172.21.15.86:0/2546900654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39a19f4d0 0x55f39a1b4320 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x55f39a1bb3a0 tx=0x7fe1180046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac4e5dda68b0ae98 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.591+0000 7fe11d7fa640 1 -- 172.21.15.86:0/2546900654 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe11801f070 con 0x55f39a19f4d0 2024-04-21T08:41:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.591+0000 7fe11d7fa640 1 -- 172.21.15.86:0/2546900654 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe11801be10 con 0x55f39a19f4d0 2024-04-21T08:41:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.591+0000 7fe11d7fa640 1 -- 172.21.15.86:0/2546900654 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe118013600 con 0x55f39a19f4d0 2024-04-21T08:41:01.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.591+0000 7fe130575740 1 -- 172.21.15.86:0/2546900654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39a19f4d0 msgr2=0x55f39a1b4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.591+0000 7fe130575740 1 --2- 172.21.15.86:0/2546900654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39a19f4d0 0x55f39a1b4320 secure :-1 s=READY pgs=866 cs=0 l=1 rev1=1 crypto rx=0x55f39a1bb3a0 tx=0x7fe1180046c0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.592+0000 7fe130575740 1 -- 172.21.15.86:0/2546900654 shutdown_connections 2024-04-21T08:41:01.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.592+0000 7fe130575740 1 --2- 172.21.15.86:0/2546900654 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f39a1a8a00 0x55f39a1a7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.592+0000 7fe130575740 1 --2- 172.21.15.86:0/2546900654 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f39a19fe80 0x55f39a19b700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.592+0000 7fe130575740 1 --2- 172.21.15.86:0/2546900654 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39a19f4d0 0x55f39a1b4320 unknown :-1 s=CLOSED pgs=866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.592+0000 7fe130575740 1 -- 172.21.15.86:0/2546900654 >> 172.21.15.86:0/2546900654 conn(0x55f39a199ba0 msgr2=0x55f39a19aff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:01.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.592+0000 7fe130575740 1 -- 172.21.15.86:0/2546900654 shutdown_connections 2024-04-21T08:41:01.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.592+0000 7fe130575740 1 -- 172.21.15.86:0/2546900654 wait complete. 2024-04-21T08:41:01.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.592+0000 7fe130575740 1 -- start start 2024-04-21T08:41:01.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.592+0000 7fe130575740 1 Processor -- start 2024-04-21T08:41:01.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.592+0000 7fe130575740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f39a19f4d0 0x55f39a1b0be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.592+0000 7fe130575740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f39a19fe80 0x55f39a1a2130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.592+0000 7fe130575740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39a1a8a00 0x55f39a1a2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.592+0000 7fe130575740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe1281b07e0 con 0x55f39a1a8a00 2024-04-21T08:41:01.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.592+0000 7fe130575740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe1281b0110 con 0x55f39a19f4d0 2024-04-21T08:41:01.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.592+0000 7fe130575740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe1281ad330 con 0x55f39a19fe80 2024-04-21T08:41:01.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.592+0000 7fe130575740 10 client.0 _ll_register_callbacks cb 0x7fe120010b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:01.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.592+0000 7fe1277fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39a1a8a00 0x55f39a1a2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.592+0000 7fe1277fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39a1a8a00 0x55f39a1a2670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60578/0 (socket says 172.21.15.86:60578) 2024-04-21T08:41:01.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.592+0000 7fe1277fe640 1 -- 172.21.15.86:0/2094169341 learned_addr learned my addr 172.21.15.86:0/2094169341 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:01.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.593+0000 7fe11e7fc640 1 --2- 172.21.15.86:0/2094169341 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f39a19fe80 0x55f39a1a2130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.593+0000 7fe126ffd640 1 --2- 172.21.15.86:0/2094169341 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f39a19f4d0 0x55f39a1b0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.593+0000 7fe1277fe640 1 -- 172.21.15.86:0/2094169341 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f39a19fe80 msgr2=0x55f39a1a2130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.593+0000 7fe1277fe640 1 --2- 172.21.15.86:0/2094169341 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f39a19fe80 0x55f39a1a2130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.593+0000 7fe1277fe640 1 -- 172.21.15.86:0/2094169341 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f39a19f4d0 msgr2=0x55f39a1b0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.593+0000 7fe1277fe640 1 --2- 172.21.15.86:0/2094169341 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f39a19f4d0 0x55f39a1b0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.593+0000 7fe1277fe640 1 -- 172.21.15.86:0/2094169341 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f39a1a2ea0 con 0x55f39a1a8a00 2024-04-21T08:41:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.593+0000 7fe1277fe640 1 --2- 172.21.15.86:0/2094169341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39a1a8a00 0x55f39a1a2670 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fe12000ac80 tx=0x7fe120003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2900e683d03899de server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.593+0000 7fe106ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.593+0000 7fe106ffd640 1 -- 172.21.15.86:0/2094169341 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe120025040 con 0x55f39a1a8a00 2024-04-21T08:41:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.593+0000 7fe106ffd640 1 -- 172.21.15.86:0/2094169341 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fe120004c10 con 0x55f39a1a8a00 2024-04-21T08:41:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.593+0000 7fe130575740 1 -- 172.21.15.86:0/2094169341 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe11801be10 con 0x55f39a1a8a00 2024-04-21T08:41:01.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.593+0000 7fe106ffd640 1 -- 172.21.15.86:0/2094169341 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe120040050 con 0x55f39a1a8a00 2024-04-21T08:41:01.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.593+0000 7fe130575740 10 client.16764 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:01.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.593+0000 7fe130575740 1 -- 172.21.15.86:0/2094169341 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f39a21ec40 con 0x55f39a1a8a00 2024-04-21T08:41:01.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.593+0000 7fe106ffd640 1 -- 172.21.15.86:0/2094169341 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe11801be10 con 0x55f39a1a8a00 2024-04-21T08:41:01.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.593+0000 7fe106ffd640 1 -- 172.21.15.86:0/2094169341 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe120004140 con 0x55f39a1a8a00 2024-04-21T08:41:01.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.593+0000 7fe130575740 10 client.16764 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:01.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.593+0000 7fe130575740 10 client.16764 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:01.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.593+0000 7fe130575740 20 client.16764 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:01.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.593+0000 7fe130575740 1 --2- 172.21.15.86:0/2094169341 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f39a21f390 0x55f39a2217b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.593+0000 7fe130575740 4 client.16764 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:41:01.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.593+0000 7fe130575740 1 -- 172.21.15.86:0/2094169341 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f39a111c10 con 0x55f39a21f390 2024-04-21T08:41:01.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.594+0000 7fe126ffd640 1 --2- 172.21.15.86:0/2094169341 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f39a21f390 0x55f39a2217b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.594+0000 7fe106ffd640 10 client.16764 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:01.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.594+0000 7fe126ffd640 1 --2- 172.21.15.86:0/2094169341 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f39a21f390 0x55f39a2217b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c81a2d85bcdfd6d5 server_cookie=9706ffb8bce3fcf0 in_seq=0 out_seq=0 2024-04-21T08:41:01.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.594+0000 7fe106ffd640 1 -- 172.21.15.86:0/2094169341 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f39a111c10 con 0x55f39a21f390 2024-04-21T08:41:01.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.594+0000 7fe106ffd640 10 client.16764 handle_command_reply: tid=0 2024-04-21T08:41:01.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.594+0000 7fe130575740 1 client.16764 shutdown 2024-04-21T08:41:01.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.594+0000 7fe130575740 1 -- 172.21.15.86:0/2094169341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39a1a8a00 msgr2=0x55f39a1a2670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.594+0000 7fe130575740 1 --2- 172.21.15.86:0/2094169341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39a1a8a00 0x55f39a1a2670 secure :-1 s=READY pgs=867 cs=0 l=1 rev1=1 crypto rx=0x7fe12000ac80 tx=0x7fe120003ba0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.595+0000 7fe130575740 1 -- 172.21.15.86:0/2094169341 shutdown_connections 2024-04-21T08:41:01.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.595+0000 7fe130575740 1 --2- 172.21.15.86:0/2094169341 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f39a21f390 0x55f39a2217b0 crc :-1 s=READY pgs=96 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.595+0000 7fe130575740 1 --2- 172.21.15.86:0/2094169341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f39a1a8a00 0x55f39a1a2670 unknown :-1 s=CLOSED pgs=867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.595+0000 7fe106ffd640 0 client.16764 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:01.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.595+0000 7fe130575740 1 --2- 172.21.15.86:0/2094169341 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f39a19fe80 0x55f39a1a2130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.595+0000 7fe130575740 1 --2- 172.21.15.86:0/2094169341 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f39a19f4d0 0x55f39a1b0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.595+0000 7fe130575740 1 -- 172.21.15.86:0/2094169341 >> 172.21.15.86:0/2094169341 conn(0x55f39a199ba0 msgr2=0x55f39a19b880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:01.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.595+0000 7fe130575740 1 -- 172.21.15.86:0/2094169341 shutdown_connections 2024-04-21T08:41:01.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.595+0000 7fe130575740 1 -- 172.21.15.86:0/2094169341 wait complete. 2024-04-21T08:41:01.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.595+0000 7fe130575740 20 client.16764 trim_cache size 0 max 16384 2024-04-21T08:41:01.603 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:41:01.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.595+0000 7fe12d985640 1 -- 172.21.15.86:0/4053234154 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe1140a3d20 msgr2=0x7fe1140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.595+0000 7fe12d985640 1 --2- 172.21.15.86:0/4053234154 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe1140a3d20 0x7fe1140a61e0 secure :-1 s=READY pgs=125 cs=0 l=1 rev1=1 crypto rx=0x7fe120005c40 tx=0x7fe12000a920 comp rx=0 tx=0).stop 2024-04-21T08:41:01.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.595+0000 7fe12d985640 1 -- 172.21.15.86:0/4053234154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe128103af0 msgr2=0x7fe12819ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.595+0000 7fe12d985640 1 --2- 172.21.15.86:0/4053234154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe128103af0 0x7fe12819ead0 secure :-1 s=READY pgs=863 cs=0 l=1 rev1=1 crypto rx=0x7fe110005940 tx=0x7fe11002d9a0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.595+0000 7fe12d985640 1 -- 172.21.15.86:0/4053234154 shutdown_connections 2024-04-21T08:41:01.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.595+0000 7fe12d985640 1 --2- 172.21.15.86:0/4053234154 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe1140a3d20 0x7fe1140a61e0 unknown :-1 s=CLOSED pgs=125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.595+0000 7fe12d985640 1 --2- 172.21.15.86:0/4053234154 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe12810c510 0x7fe128198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.595+0000 7fe12d985640 1 --2- 172.21.15.86:0/4053234154 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe128103af0 0x7fe12819ead0 unknown :-1 s=CLOSED pgs=863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.595+0000 7fe12d985640 1 --2- 172.21.15.86:0/4053234154 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe1281031d0 0x7fe12819e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.595+0000 7fe12d985640 1 -- 172.21.15.86:0/4053234154 >> 172.21.15.86:0/4053234154 conn(0x7fe128077f50 msgr2=0x7fe1280fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:01.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.595+0000 7fe12d985640 1 -- 172.21.15.86:0/4053234154 shutdown_connections 2024-04-21T08:41:01.605 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.595+0000 7fe12d985640 1 -- 172.21.15.86:0/4053234154 wait complete. 2024-04-21T08:41:01.637 INFO:tasks.quiescer.fs.[cephfs]:Total duration (43.8) is longer than `split_if_longer` (35.0), will split into two consecutive quiesces 2024-04-21T08:41:01.637 INFO:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.9 2024-04-21T08:41:01.637 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:41:01.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:01 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/950526153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:41:01.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.736+0000 7f7998596640 1 -- 172.21.15.86:0/2741898205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7990069620 msgr2=0x7f7990069aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.736+0000 7f7998596640 1 --2- 172.21.15.86:0/2741898205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7990069620 0x7f7990069aa0 secure :-1 s=READY pgs=868 cs=0 l=1 rev1=1 crypto rx=0x7f7980009900 tx=0x7f798002bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:01.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.736+0000 7f7998596640 1 -- 172.21.15.86:0/2741898205 shutdown_connections 2024-04-21T08:41:01.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.737+0000 7f7998596640 1 --2- 172.21.15.86:0/2741898205 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f799010e810 0x7f7990110c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.737+0000 7f7998596640 1 --2- 172.21.15.86:0/2741898205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7990069620 0x7f7990069aa0 unknown :-1 s=CLOSED pgs=868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.737+0000 7f7998596640 1 --2- 172.21.15.86:0/2741898205 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7990102e30 0x7f79900690e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.737+0000 7f7998596640 1 -- 172.21.15.86:0/2741898205 >> 172.21.15.86:0/2741898205 conn(0x7f7990077e30 msgr2=0x7f7990078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:01.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.737+0000 7f7998596640 1 -- 172.21.15.86:0/2741898205 shutdown_connections 2024-04-21T08:41:01.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.737+0000 7f7998596640 1 -- 172.21.15.86:0/2741898205 wait complete. 2024-04-21T08:41:01.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.737+0000 7f7998596640 1 Processor -- start 2024-04-21T08:41:01.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.737+0000 7f7998596640 1 -- start start 2024-04-21T08:41:01.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.737+0000 7f7998596640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7990069620 0x7f79901a0660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.737+0000 7f7998596640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7990102e30 0x7f79901a0ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.737+0000 7f7998596640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f799010e810 0x7f799019a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.737+0000 7f7998596640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f79901143a0 con 0x7f7990102e30 2024-04-21T08:41:01.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.737+0000 7f7998596640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7990114220 con 0x7f799010e810 2024-04-21T08:41:01.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.737+0000 7f7998596640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7990114520 con 0x7f7990069620 2024-04-21T08:41:01.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.737+0000 7f7995b31640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7990102e30 0x7f79901a0ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.737+0000 7f7995b31640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7990102e30 0x7f79901a0ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60598/0 (socket says 172.21.15.86:60598) 2024-04-21T08:41:01.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.737+0000 7f7995b31640 1 -- 172.21.15.86:0/2370387356 learned_addr learned my addr 172.21.15.86:0/2370387356 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:01.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.737+0000 7f7996332640 1 --2- 172.21.15.86:0/2370387356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7990069620 0x7f79901a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.738+0000 7f7996b33640 1 --2- 172.21.15.86:0/2370387356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f799010e810 0x7f799019a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.738+0000 7f7995b31640 1 -- 172.21.15.86:0/2370387356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7990069620 msgr2=0x7f79901a0660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.738+0000 7f7995b31640 1 --2- 172.21.15.86:0/2370387356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7990069620 0x7f79901a0660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.738+0000 7f7995b31640 1 -- 172.21.15.86:0/2370387356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f799010e810 msgr2=0x7f799019a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.738+0000 7f7995b31640 1 --2- 172.21.15.86:0/2370387356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f799010e810 0x7f799019a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.738+0000 7f7995b31640 1 -- 172.21.15.86:0/2370387356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f799019afe0 con 0x7f7990102e30 2024-04-21T08:41:01.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.738+0000 7f7995b31640 1 --2- 172.21.15.86:0/2370387356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7990102e30 0x7f79901a0ba0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f7980005940 tx=0x7f798002c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:01.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.738+0000 7f79877fe640 1 -- 172.21.15.86:0/2370387356 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7980043020 con 0x7f7990102e30 2024-04-21T08:41:01.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.738+0000 7f79877fe640 1 -- 172.21.15.86:0/2370387356 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f798002cc60 con 0x7f7990102e30 2024-04-21T08:41:01.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.738+0000 7f7998596640 1 -- 172.21.15.86:0/2370387356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f799019b270 con 0x7f7990102e30 2024-04-21T08:41:01.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.738+0000 7f79877fe640 1 -- 172.21.15.86:0/2370387356 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7980003c30 con 0x7f7990102e30 2024-04-21T08:41:01.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.738+0000 7f7998596640 1 -- 172.21.15.86:0/2370387356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79901a75e0 con 0x7f7990102e30 2024-04-21T08:41:01.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.739+0000 7f7998596640 1 -- 172.21.15.86:0/2370387356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f799019b5e0 con 0x7f7990102e30 2024-04-21T08:41:01.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.740+0000 7f79877fe640 1 -- 172.21.15.86:0/2370387356 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7980046020 con 0x7f7990102e30 2024-04-21T08:41:01.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.741+0000 7f79877fe640 1 --2- 172.21.15.86:0/2370387356 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f797c0a3d20 0x7f797c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:01.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.741+0000 7f7996332640 1 --2- 172.21.15.86:0/2370387356 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f797c0a3d20 0x7f797c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:01.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.741+0000 7f79877fe640 1 -- 172.21.15.86:0/2370387356 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f79800f3390 con 0x7f7990102e30 2024-04-21T08:41:01.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.741+0000 7f7996332640 1 --2- 172.21.15.86:0/2370387356 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f797c0a3d20 0x7f797c0a61e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f798c0058c0 tx=0x7f798c00a250 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:01.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.744+0000 7f79877fe640 1 -- 172.21.15.86:0/2370387356 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f798004d020 con 0x7f7990102e30 2024-04-21T08:41:01.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:01 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/950526153' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:41:01.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.961+0000 7f7998596640 1 -- 172.21.15.86:0/2370387356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f799019bc50 con 0x7f7990102e30 2024-04-21T08:41:01.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.961+0000 7f79877fe640 1 -- 172.21.15.86:0/2370387356 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7f79800bf7d0 con 0x7f7990102e30 2024-04-21T08:41:01.961 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:41:01.961 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:38:45.202126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24398,24394,24455,24472]},"id":1}]} 2024-04-21T08:41:01.961 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-21T08:41:01.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.963+0000 7f7998596640 1 -- 172.21.15.86:0/2370387356 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f797c0a3d20 msgr2=0x7f797c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.963+0000 7f7998596640 1 --2- 172.21.15.86:0/2370387356 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f797c0a3d20 0x7f797c0a61e0 secure :-1 s=READY pgs=126 cs=0 l=1 rev1=1 crypto rx=0x7f798c0058c0 tx=0x7f798c00a250 comp rx=0 tx=0).stop 2024-04-21T08:41:01.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.963+0000 7f7998596640 1 -- 172.21.15.86:0/2370387356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7990102e30 msgr2=0x7f79901a0ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:01.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.963+0000 7f7998596640 1 --2- 172.21.15.86:0/2370387356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7990102e30 0x7f79901a0ba0 secure :-1 s=READY pgs=869 cs=0 l=1 rev1=1 crypto rx=0x7f7980005940 tx=0x7f798002c860 comp rx=0 tx=0).stop 2024-04-21T08:41:01.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.963+0000 7f7998596640 1 -- 172.21.15.86:0/2370387356 shutdown_connections 2024-04-21T08:41:01.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.963+0000 7f7998596640 1 --2- 172.21.15.86:0/2370387356 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f797c0a3d20 0x7f797c0a61e0 unknown :-1 s=CLOSED pgs=126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.963+0000 7f7998596640 1 --2- 172.21.15.86:0/2370387356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f799010e810 0x7f799019a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.963+0000 7f7998596640 1 --2- 172.21.15.86:0/2370387356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7990102e30 0x7f79901a0ba0 unknown :-1 s=CLOSED pgs=869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.963+0000 7f7998596640 1 --2- 172.21.15.86:0/2370387356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7990069620 0x7f79901a0660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:01.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.963+0000 7f7998596640 1 -- 172.21.15.86:0/2370387356 >> 172.21.15.86:0/2370387356 conn(0x7f7990077e30 msgr2=0x7f799010f6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:01.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.963+0000 7f7998596640 1 -- 172.21.15.86:0/2370387356 shutdown_connections 2024-04-21T08:41:01.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:01.963+0000 7f7998596640 1 -- 172.21.15.86:0/2370387356 wait complete. 2024-04-21T08:41:01.974 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 30 --expiration 81.9 --await' 2024-04-21T08:41:01.974 DEBUG:teuthology.orchestra.run.smithi086:> timeout 45 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24482 quiesce db / --timeout 30 --expiration 81.9 --await 2024-04-21T08:41:02.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.081+0000 7efc9e870640 1 -- 172.21.15.86:0/4063410183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc98101a40 msgr2=0x7efc98105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:02.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.081+0000 7efc9e870640 1 --2- 172.21.15.86:0/4063410183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc98101a40 0x7efc98105a80 secure :-1 s=READY pgs=870 cs=0 l=1 rev1=1 crypto rx=0x7efc8c009900 tx=0x7efc8c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:02.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.081+0000 7efc9e870640 1 -- 172.21.15.86:0/4063410183 shutdown_connections 2024-04-21T08:41:02.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.081+0000 7efc9e870640 1 --2- 172.21.15.86:0/4063410183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc98106160 0x7efc9810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.081+0000 7efc9e870640 1 --2- 172.21.15.86:0/4063410183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc98101a40 0x7efc98105a80 unknown :-1 s=CLOSED pgs=870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.081+0000 7efc9e870640 1 --2- 172.21.15.86:0/4063410183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc98101090 0x7efc98101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.081+0000 7efc9e870640 1 -- 172.21.15.86:0/4063410183 >> 172.21.15.86:0/4063410183 conn(0x7efc980771d0 msgr2=0x7efc980775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:02.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.081+0000 7efc9e870640 1 -- 172.21.15.86:0/4063410183 shutdown_connections 2024-04-21T08:41:02.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.081+0000 7efc9e870640 1 -- 172.21.15.86:0/4063410183 wait complete. 2024-04-21T08:41:02.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.082+0000 7efc9e870640 1 Processor -- start 2024-04-21T08:41:02.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.082+0000 7efc9e870640 1 -- start start 2024-04-21T08:41:02.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.082+0000 7efc9e870640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc98101090 0x7efc98197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.082+0000 7efc9e870640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc98106160 0x7efc98197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.082+0000 7efc9e870640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc9819bf20 0x7efc9819c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.082+0000 7efc9e870640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efc981110a0 con 0x7efc98106160 2024-04-21T08:41:02.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.082+0000 7efc9e870640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efc98110f20 con 0x7efc9819bf20 2024-04-21T08:41:02.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.082+0000 7efc9e870640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7efc98111220 con 0x7efc98101090 2024-04-21T08:41:02.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.082+0000 7efc977fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc98106160 0x7efc98197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.082+0000 7efc977fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc98106160 0x7efc98197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60626/0 (socket says 172.21.15.86:60626) 2024-04-21T08:41:02.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.082+0000 7efc977fe640 1 -- 172.21.15.86:0/1103533103 learned_addr learned my addr 172.21.15.86:0/1103533103 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:02.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.082+0000 7efc9ce0d640 1 --2- 172.21.15.86:0/1103533103 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc9819bf20 0x7efc9819c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.082+0000 7efc97fff640 1 --2- 172.21.15.86:0/1103533103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc98101090 0x7efc98197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.083+0000 7efc977fe640 1 -- 172.21.15.86:0/1103533103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc98101090 msgr2=0x7efc98197660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:02.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.083+0000 7efc977fe640 1 --2- 172.21.15.86:0/1103533103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc98101090 0x7efc98197660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.083+0000 7efc977fe640 1 -- 172.21.15.86:0/1103533103 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc9819bf20 msgr2=0x7efc9819c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:02.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.083+0000 7efc977fe640 1 --2- 172.21.15.86:0/1103533103 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc9819bf20 0x7efc9819c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.083+0000 7efc977fe640 1 -- 172.21.15.86:0/1103533103 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc9819ca10 con 0x7efc98106160 2024-04-21T08:41:02.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.083+0000 7efc977fe640 1 --2- 172.21.15.86:0/1103533103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc98106160 0x7efc98197ba0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7efc8c036040 tx=0x7efc8c02c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:02.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.083+0000 7efc957fa640 1 -- 172.21.15.86:0/1103533103 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8c043020 con 0x7efc98106160 2024-04-21T08:41:02.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.083+0000 7efc9e870640 1 -- 172.21.15.86:0/1103533103 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc9819cca0 con 0x7efc98106160 2024-04-21T08:41:02.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.083+0000 7efc957fa640 1 -- 172.21.15.86:0/1103533103 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc8c02cbe0 con 0x7efc98106160 2024-04-21T08:41:02.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.083+0000 7efc957fa640 1 -- 172.21.15.86:0/1103533103 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8c003bf0 con 0x7efc98106160 2024-04-21T08:41:02.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.083+0000 7efc9e870640 1 -- 172.21.15.86:0/1103533103 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc981a44f0 con 0x7efc98106160 2024-04-21T08:41:02.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.085+0000 7efc957fa640 1 -- 172.21.15.86:0/1103533103 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efc8c046020 con 0x7efc98106160 2024-04-21T08:41:02.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.085+0000 7efca145c740 1 Processor -- start 2024-04-21T08:41:02.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.086+0000 7efca145c740 1 -- start start 2024-04-21T08:41:02.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.086+0000 7efc957fa640 1 --2- 172.21.15.86:0/1103533103 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efc800a3d20 0x7efc800a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.086+0000 7efca145c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1e4fca7d0 0x55d1e4fcabb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.086+0000 7efca145c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1e4fcb180 0x55d1e4fd5b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.086+0000 7efca145c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d1e4fd60d0 0x55d1e4fd84c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.086+0000 7efc9ce0d640 1 --2- 172.21.15.86:0/1103533103 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efc800a3d20 0x7efc800a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.086+0000 7efca145c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d1e4f2b0b0 con 0x55d1e4fca7d0 2024-04-21T08:41:02.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.086+0000 7efca145c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d1e4edc680 con 0x55d1e4fd60d0 2024-04-21T08:41:02.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.086+0000 7efca145c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d1e4ee8c60 con 0x55d1e4fcb180 2024-04-21T08:41:02.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.086+0000 7efc957fa640 1 -- 172.21.15.86:0/1103533103 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7efc8c0f2c80 con 0x7efc98106160 2024-04-21T08:41:02.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.086+0000 7efc97fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1e4fca7d0 0x55d1e4fcabb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.086+0000 7efc97fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1e4fca7d0 0x55d1e4fcabb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60634/0 (socket says 172.21.15.86:60634) 2024-04-21T08:41:02.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.086+0000 7efc97fff640 1 -- 172.21.15.86:0/1971959872 learned_addr learned my addr 172.21.15.86:0/1971959872 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:02.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.086+0000 7efc977fe640 1 --2- 172.21.15.86:0/1971959872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1e4fcb180 0x55d1e4fd5b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.086+0000 7efc97fff640 1 -- 172.21.15.86:0/1971959872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1e4fcb180 msgr2=0x55d1e4fd5b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:02.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.086+0000 7efc97fff640 1 --2- 172.21.15.86:0/1971959872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1e4fcb180 0x55d1e4fd5b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.086+0000 7efc97fff640 1 -- 172.21.15.86:0/1971959872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d1e4fd60d0 msgr2=0x55d1e4fd84c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:02.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.086+0000 7efc97fff640 1 --2- 172.21.15.86:0/1971959872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d1e4fd60d0 0x55d1e4fd84c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.086+0000 7efc97fff640 1 -- 172.21.15.86:0/1971959872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1e4d0dc90 con 0x55d1e4fca7d0 2024-04-21T08:41:02.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.086+0000 7efc9ce0d640 1 --2- 172.21.15.86:0/1103533103 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efc800a3d20 0x7efc800a61e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efc90005fd0 tx=0x7efc90009600 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:02.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.086+0000 7efc97fff640 1 --2- 172.21.15.86:0/1971959872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1e4fca7d0 0x55d1e4fcabb0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7efc8800bcb0 tx=0x7efc8800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7fccb198b038bc9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:02.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.086+0000 7efc7e7fc640 1 -- 172.21.15.86:0/1971959872 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc88015070 con 0x55d1e4fca7d0 2024-04-21T08:41:02.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.086+0000 7efc7e7fc640 1 -- 172.21.15.86:0/1971959872 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc8801a910 con 0x55d1e4fca7d0 2024-04-21T08:41:02.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.086+0000 7efc7e7fc640 1 -- 172.21.15.86:0/1971959872 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc880118c0 con 0x55d1e4fca7d0 2024-04-21T08:41:02.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.087+0000 7efca145c740 1 -- 172.21.15.86:0/1971959872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1e4fca7d0 msgr2=0x55d1e4fcabb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:02.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.087+0000 7efca145c740 1 --2- 172.21.15.86:0/1971959872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1e4fca7d0 0x55d1e4fcabb0 secure :-1 s=READY pgs=872 cs=0 l=1 rev1=1 crypto rx=0x7efc8800bcb0 tx=0x7efc8800f730 comp rx=0 tx=0).stop 2024-04-21T08:41:02.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.087+0000 7efca145c740 1 -- 172.21.15.86:0/1971959872 shutdown_connections 2024-04-21T08:41:02.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.087+0000 7efca145c740 1 --2- 172.21.15.86:0/1971959872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d1e4fd60d0 0x55d1e4fd84c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.087+0000 7efca145c740 1 --2- 172.21.15.86:0/1971959872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1e4fcb180 0x55d1e4fd5b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.087+0000 7efca145c740 1 --2- 172.21.15.86:0/1971959872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1e4fca7d0 0x55d1e4fcabb0 unknown :-1 s=CLOSED pgs=872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.087+0000 7efca145c740 1 -- 172.21.15.86:0/1971959872 >> 172.21.15.86:0/1971959872 conn(0x55d1e4fc50f0 msgr2=0x55d1e4fc54d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:02.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.087+0000 7efca145c740 1 -- 172.21.15.86:0/1971959872 shutdown_connections 2024-04-21T08:41:02.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.087+0000 7efca145c740 1 -- 172.21.15.86:0/1971959872 wait complete. 2024-04-21T08:41:02.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.087+0000 7efca145c740 1 -- start start 2024-04-21T08:41:02.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.088+0000 7efca145c740 1 Processor -- start 2024-04-21T08:41:02.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.088+0000 7efca145c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1e4fca7d0 0x55d1e4fd31d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.088+0000 7efca145c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d1e4fcb180 0x55d1e4fd3710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.088+0000 7efca145c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1e4fd60d0 0x55d1e4fd3c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.088+0000 7efca145c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d1e4f2e400 con 0x55d1e4fd60d0 2024-04-21T08:41:02.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.088+0000 7efca145c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d1e4eea840 con 0x55d1e4fcb180 2024-04-21T08:41:02.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.088+0000 7efca145c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d1e4f33720 con 0x55d1e4fca7d0 2024-04-21T08:41:02.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.088+0000 7efca145c740 10 client.0 _ll_register_callbacks cb 0x55d1e4f3bea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:02.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.088+0000 7efc9ce0d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1e4fd60d0 0x55d1e4fd3c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.088+0000 7efc9ce0d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1e4fd60d0 0x55d1e4fd3c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60636/0 (socket says 172.21.15.86:60636) 2024-04-21T08:41:02.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.088+0000 7efc9ce0d640 1 -- 172.21.15.86:0/4138256488 learned_addr learned my addr 172.21.15.86:0/4138256488 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:02.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.088+0000 7efc977fe640 1 --2- 172.21.15.86:0/4138256488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d1e4fcb180 0x55d1e4fd3710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.088+0000 7efc97fff640 1 --2- 172.21.15.86:0/4138256488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1e4fca7d0 0x55d1e4fd31d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.088+0000 7efc9ce0d640 1 -- 172.21.15.86:0/4138256488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1e4fca7d0 msgr2=0x55d1e4fd31d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:02.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.088+0000 7efc9ce0d640 1 --2- 172.21.15.86:0/4138256488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1e4fca7d0 0x55d1e4fd31d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.088+0000 7efc9ce0d640 1 -- 172.21.15.86:0/4138256488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d1e4fcb180 msgr2=0x55d1e4fd3710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:02.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.088+0000 7efc9ce0d640 1 --2- 172.21.15.86:0/4138256488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d1e4fcb180 0x55d1e4fd3710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.088+0000 7efc9ce0d640 1 -- 172.21.15.86:0/4138256488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1e4d6e420 con 0x55d1e4fd60d0 2024-04-21T08:41:02.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.088+0000 7efc977fe640 1 --2- 172.21.15.86:0/4138256488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d1e4fcb180 0x55d1e4fd3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:02.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.088+0000 7efc9ce0d640 1 --2- 172.21.15.86:0/4138256488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1e4fd60d0 0x55d1e4fd3c50 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7efc90012bd0 tx=0x7efc90009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2788efa0803511bf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:02.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.088+0000 7efc777fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:02.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.089+0000 7efc777fe640 1 -- 172.21.15.86:0/4138256488 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc90023070 con 0x55d1e4fd60d0 2024-04-21T08:41:02.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.089+0000 7efca145c740 1 -- 172.21.15.86:0/4138256488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc8801a910 con 0x55d1e4fd60d0 2024-04-21T08:41:02.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.089+0000 7efc777fe640 1 -- 172.21.15.86:0/4138256488 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7efc90011e00 con 0x55d1e4fd60d0 2024-04-21T08:41:02.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.089+0000 7efc777fe640 1 -- 172.21.15.86:0/4138256488 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efc9002e050 con 0x55d1e4fd60d0 2024-04-21T08:41:02.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.089+0000 7efca145c740 10 client.16782 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:02.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.089+0000 7efca145c740 1 -- 172.21.15.86:0/4138256488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d1e4c01580 con 0x55d1e4fd60d0 2024-04-21T08:41:02.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.089+0000 7efc777fe640 1 -- 172.21.15.86:0/4138256488 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc8801a910 con 0x55d1e4fd60d0 2024-04-21T08:41:02.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.089+0000 7efc777fe640 1 -- 172.21.15.86:0/4138256488 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7efc9000f040 con 0x55d1e4fd60d0 2024-04-21T08:41:02.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.089+0000 7efca145c740 10 client.16782 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:02.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.089+0000 7efca145c740 10 client.16782 resolve_mds: validated gid 24482 aka daemon mds.b 2024-04-21T08:41:02.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.089+0000 7efca145c740 20 client.16782 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:02.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.089+0000 7efca145c740 1 --2- 172.21.15.86:0/4138256488 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d1e4fe62f0 0x55d1e4fe8770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.089+0000 7efca145c740 4 client.16782 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:02.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.089+0000 7efca145c740 1 -- 172.21.15.86:0/4138256488 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d1e4f3cb10 con 0x55d1e4fe62f0 2024-04-21T08:41:02.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.089+0000 7efc97fff640 1 --2- 172.21.15.86:0/4138256488 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d1e4fe62f0 0x55d1e4fe8770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.090+0000 7efc777fe640 10 client.16782 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:02.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.090+0000 7efc97fff640 1 --2- 172.21.15.86:0/4138256488 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d1e4fe62f0 0x55d1e4fe8770 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fc3c7de7ddedf1ee server_cookie=ea9ee482a0e09805 in_seq=0 out_seq=0 2024-04-21T08:41:02.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.090+0000 7efc777fe640 1 -- 172.21.15.86:0/4138256488 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d1e4f3cb10 con 0x55d1e4fe62f0 2024-04-21T08:41:02.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.090+0000 7efc777fe640 10 client.16782 handle_command_reply: tid=0 2024-04-21T08:41:02.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.090+0000 7efca145c740 1 client.16782 shutdown 2024-04-21T08:41:02.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.091+0000 7efca145c740 1 -- 172.21.15.86:0/4138256488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1e4fd60d0 msgr2=0x55d1e4fd3c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:02.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.091+0000 7efca145c740 1 --2- 172.21.15.86:0/4138256488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1e4fd60d0 0x55d1e4fd3c50 secure :-1 s=READY pgs=873 cs=0 l=1 rev1=1 crypto rx=0x7efc90012bd0 tx=0x7efc90009b20 comp rx=0 tx=0).stop 2024-04-21T08:41:02.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.091+0000 7efca145c740 1 -- 172.21.15.86:0/4138256488 shutdown_connections 2024-04-21T08:41:02.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.091+0000 7efca145c740 1 --2- 172.21.15.86:0/4138256488 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d1e4fe62f0 0x55d1e4fe8770 crc :-1 s=READY pgs=97 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.091+0000 7efca145c740 1 --2- 172.21.15.86:0/4138256488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1e4fd60d0 0x55d1e4fd3c50 unknown :-1 s=CLOSED pgs=873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.091+0000 7efca145c740 1 --2- 172.21.15.86:0/4138256488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d1e4fcb180 0x55d1e4fd3710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.091+0000 7efc777fe640 0 client.16782 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:02.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.091+0000 7efca145c740 1 --2- 172.21.15.86:0/4138256488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1e4fca7d0 0x55d1e4fd31d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.091+0000 7efca145c740 1 -- 172.21.15.86:0/4138256488 >> 172.21.15.86:0/4138256488 conn(0x55d1e4fc50f0 msgr2=0x55d1e4fc6e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:02.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.091+0000 7efca145c740 1 -- 172.21.15.86:0/4138256488 shutdown_connections 2024-04-21T08:41:02.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.091+0000 7efca145c740 1 -- 172.21.15.86:0/4138256488 wait complete. 2024-04-21T08:41:02.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.091+0000 7efca145c740 20 client.16782 trim_cache size 0 max 16384 2024-04-21T08:41:02.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.107+0000 7efca145c740 1 Processor -- start 2024-04-21T08:41:02.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.107+0000 7efca145c740 1 -- start start 2024-04-21T08:41:02.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.108+0000 7efca145c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1e4fca7d0 0x55d1e4fe1a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.108+0000 7efca145c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1e4fcb180 0x55d1e4fd38a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.108+0000 7efca145c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d1e4fd60d0 0x55d1e4fee080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.108+0000 7efca145c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efc981aec70 con 0x55d1e4fca7d0 2024-04-21T08:41:02.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.108+0000 7efca145c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efc981ad800 con 0x55d1e4fd60d0 2024-04-21T08:41:02.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.108+0000 7efca145c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7efc981aded0 con 0x55d1e4fcb180 2024-04-21T08:41:02.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.108+0000 7efc97fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1e4fca7d0 0x55d1e4fe1a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.108+0000 7efc97fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1e4fca7d0 0x55d1e4fe1a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60650/0 (socket says 172.21.15.86:60650) 2024-04-21T08:41:02.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.108+0000 7efc97fff640 1 -- 172.21.15.86:0/1303193626 learned_addr learned my addr 172.21.15.86:0/1303193626 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:02.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.108+0000 7efc977fe640 1 --2- 172.21.15.86:0/1303193626 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1e4fcb180 0x55d1e4fd38a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.108+0000 7efc97fff640 1 -- 172.21.15.86:0/1303193626 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1e4fcb180 msgr2=0x55d1e4fd38a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:02.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.108+0000 7efc97fff640 1 --2- 172.21.15.86:0/1303193626 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1e4fcb180 0x55d1e4fd38a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.108+0000 7efc97fff640 1 -- 172.21.15.86:0/1303193626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d1e4fd60d0 msgr2=0x55d1e4fee080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:02.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.108+0000 7efc97fff640 1 --2- 172.21.15.86:0/1303193626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d1e4fd60d0 0x55d1e4fee080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.108+0000 7efc97fff640 1 -- 172.21.15.86:0/1303193626 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d1e4fd21a0 con 0x55d1e4fca7d0 2024-04-21T08:41:02.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.108+0000 7efc97fff640 1 --2- 172.21.15.86:0/1303193626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1e4fca7d0 0x55d1e4fe1a90 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7efc880114b0 tx=0x7efc8800ff80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=374a5e0e09a1160f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:02.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.108+0000 7efc7dffb640 1 -- 172.21.15.86:0/1303193626 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc88015040 con 0x55d1e4fca7d0 2024-04-21T08:41:02.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.108+0000 7efc7dffb640 1 -- 172.21.15.86:0/1303193626 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc88005580 con 0x55d1e4fca7d0 2024-04-21T08:41:02.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.108+0000 7efc7dffb640 1 -- 172.21.15.86:0/1303193626 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc88004bb0 con 0x55d1e4fca7d0 2024-04-21T08:41:02.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.109+0000 7efca145c740 1 -- 172.21.15.86:0/1303193626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1e4fca7d0 msgr2=0x55d1e4fe1a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:02.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.109+0000 7efca145c740 1 --2- 172.21.15.86:0/1303193626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1e4fca7d0 0x55d1e4fe1a90 secure :-1 s=READY pgs=874 cs=0 l=1 rev1=1 crypto rx=0x7efc880114b0 tx=0x7efc8800ff80 comp rx=0 tx=0).stop 2024-04-21T08:41:02.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.109+0000 7efca145c740 1 -- 172.21.15.86:0/1303193626 shutdown_connections 2024-04-21T08:41:02.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.109+0000 7efca145c740 1 --2- 172.21.15.86:0/1303193626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d1e4fd60d0 0x55d1e4fee080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.109+0000 7efca145c740 1 --2- 172.21.15.86:0/1303193626 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1e4fcb180 0x55d1e4fd38a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.109+0000 7efca145c740 1 --2- 172.21.15.86:0/1303193626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1e4fca7d0 0x55d1e4fe1a90 unknown :-1 s=CLOSED pgs=874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.109+0000 7efca145c740 1 -- 172.21.15.86:0/1303193626 >> 172.21.15.86:0/1303193626 conn(0x55d1e4fc50f0 msgr2=0x55d1e4fd3190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:02.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.109+0000 7efca145c740 1 -- 172.21.15.86:0/1303193626 shutdown_connections 2024-04-21T08:41:02.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.109+0000 7efca145c740 1 -- 172.21.15.86:0/1303193626 wait complete. 2024-04-21T08:41:02.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.109+0000 7efca145c740 1 -- start start 2024-04-21T08:41:02.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.109+0000 7efca145c740 1 Processor -- start 2024-04-21T08:41:02.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.110+0000 7efca145c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1e4fca7d0 0x55d1e4fde190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.110+0000 7efca145c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1e4fcb180 0x55d1e4fdaf70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.110+0000 7efca145c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d1e4fd60d0 0x55d1e4fdb4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.110+0000 7efca145c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efc981ab0f0 con 0x55d1e4fcb180 2024-04-21T08:41:02.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.110+0000 7efca145c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efc981ae5a0 con 0x55d1e4fd60d0 2024-04-21T08:41:02.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.110+0000 7efca145c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7efc981ab780 con 0x55d1e4fca7d0 2024-04-21T08:41:02.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.110+0000 7efca145c740 10 client.0 _ll_register_callbacks cb 0x7efc90016070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:02.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.110+0000 7efc977fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1e4fcb180 0x55d1e4fdaf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.110+0000 7efc977fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1e4fcb180 0x55d1e4fdaf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60654/0 (socket says 172.21.15.86:60654) 2024-04-21T08:41:02.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.110+0000 7efc977fe640 1 -- 172.21.15.86:0/2458939627 learned_addr learned my addr 172.21.15.86:0/2458939627 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.110+0000 7efc9ce0d640 1 --2- 172.21.15.86:0/2458939627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d1e4fd60d0 0x55d1e4fdb4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.110+0000 7efc97fff640 1 --2- 172.21.15.86:0/2458939627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1e4fca7d0 0x55d1e4fde190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.110+0000 7efc977fe640 1 -- 172.21.15.86:0/2458939627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1e4fca7d0 msgr2=0x55d1e4fde190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.110+0000 7efc977fe640 1 --2- 172.21.15.86:0/2458939627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1e4fca7d0 0x55d1e4fde190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.110+0000 7efc977fe640 1 -- 172.21.15.86:0/2458939627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d1e4fd60d0 msgr2=0x55d1e4fdb4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.110+0000 7efc977fe640 1 --2- 172.21.15.86:0/2458939627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d1e4fd60d0 0x55d1e4fdb4b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.110+0000 7efc977fe640 1 -- 172.21.15.86:0/2458939627 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d1e4fdb9f0 con 0x55d1e4fcb180 2024-04-21T08:41:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.110+0000 7efc977fe640 1 --2- 172.21.15.86:0/2458939627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1e4fcb180 0x55d1e4fdaf70 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7efc8c0f01d0 tx=0x7efc8c02cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3b8ab4273074e43 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.110+0000 7efc777fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.110+0000 7efc777fe640 1 -- 172.21.15.86:0/2458939627 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc8c0be9c0 con 0x55d1e4fcb180 2024-04-21T08:41:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.110+0000 7efc777fe640 1 -- 172.21.15.86:0/2458939627 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7efc8c0bb030 con 0x55d1e4fcb180 2024-04-21T08:41:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.110+0000 7efca145c740 1 -- 172.21.15.86:0/2458939627 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc88005580 con 0x55d1e4fcb180 2024-04-21T08:41:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.110+0000 7efc777fe640 1 -- 172.21.15.86:0/2458939627 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efc8c088050 con 0x55d1e4fcb180 2024-04-21T08:41:02.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.110+0000 7efca145c740 10 client.16788 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:02.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.110+0000 7efca145c740 1 -- 172.21.15.86:0/2458939627 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d1e504d590 con 0x55d1e4fcb180 2024-04-21T08:41:02.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.110+0000 7efc777fe640 1 -- 172.21.15.86:0/2458939627 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc88005580 con 0x55d1e4fcb180 2024-04-21T08:41:02.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.111+0000 7efc777fe640 1 -- 172.21.15.86:0/2458939627 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7efc8c089240 con 0x55d1e4fcb180 2024-04-21T08:41:02.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.111+0000 7efca145c740 10 client.16788 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:02.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.111+0000 7efca145c740 10 client.16788 resolve_mds: validated gid 24482 aka daemon mds.b 2024-04-21T08:41:02.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.111+0000 7efca145c740 20 client.16788 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:02.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.111+0000 7efca145c740 1 --2- 172.21.15.86:0/2458939627 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d1e504dd80 0x55d1e50501a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.111+0000 7efca145c740 4 client.16788 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 30.0, "expiration": 81.9, "await": true}] 2024-04-21T08:41:02.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.111+0000 7efca145c740 1 -- 172.21.15.86:0/2458939627 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 30.0, "expiration": 81.9, "await": true}) -- 0x55d1e4f36980 con 0x55d1e504dd80 2024-04-21T08:41:02.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.111+0000 7efc97fff640 1 --2- 172.21.15.86:0/2458939627 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d1e504dd80 0x55d1e50501a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.111+0000 7efc777fe640 10 client.16788 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:02.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.111+0000 7efc97fff640 1 --2- 172.21.15.86:0/2458939627 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d1e504dd80 0x55d1e50501a0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5754784ff7fd174d server_cookie=30baeeabbe2bb701 in_seq=0 out_seq=0 2024-04-21T08:41:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.125+0000 7efc777fe640 1 -- 172.21.15.86:0/2458939627 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55d1e4f36980 con 0x55d1e504dd80 2024-04-21T08:41:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.125+0000 7efc777fe640 10 client.16788 handle_command_reply: tid=0 2024-04-21T08:41:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.125+0000 7efca145c740 1 client.16788 shutdown 2024-04-21T08:41:02.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.125+0000 7efca145c740 1 -- 172.21.15.86:0/2458939627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1e4fcb180 msgr2=0x55d1e4fdaf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:02.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.125+0000 7efca145c740 1 --2- 172.21.15.86:0/2458939627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1e4fcb180 0x55d1e4fdaf70 secure :-1 s=READY pgs=875 cs=0 l=1 rev1=1 crypto rx=0x7efc8c0f01d0 tx=0x7efc8c02cd90 comp rx=0 tx=0).stop 2024-04-21T08:41:02.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.125+0000 7efca145c740 1 -- 172.21.15.86:0/2458939627 shutdown_connections 2024-04-21T08:41:02.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.125+0000 7efca145c740 1 --2- 172.21.15.86:0/2458939627 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d1e504dd80 0x55d1e50501a0 crc :-1 s=READY pgs=98 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.125+0000 7efca145c740 1 --2- 172.21.15.86:0/2458939627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d1e4fd60d0 0x55d1e4fdb4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.125+0000 7efc777fe640 0 client.16788 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:02.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.125+0000 7efca145c740 1 --2- 172.21.15.86:0/2458939627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d1e4fcb180 0x55d1e4fdaf70 unknown :-1 s=CLOSED pgs=875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.125+0000 7efca145c740 1 --2- 172.21.15.86:0/2458939627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d1e4fca7d0 0x55d1e4fde190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.125+0000 7efca145c740 1 -- 172.21.15.86:0/2458939627 >> 172.21.15.86:0/2458939627 conn(0x55d1e4fc50f0 msgr2=0x55d1e4fc41d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:02.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.125+0000 7efca145c740 1 -- 172.21.15.86:0/2458939627 shutdown_connections 2024-04-21T08:41:02.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.125+0000 7efca145c740 1 -- 172.21.15.86:0/2458939627 wait complete. 2024-04-21T08:41:02.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.126+0000 7efca145c740 20 client.16788 trim_cache size 0 max 16384 2024-04-21T08:41:02.127 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:41:02.127 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 29, 2024-04-21T08:41:02.127 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24482, 2024-04-21T08:41:02.127 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 2, 2024-04-21T08:41:02.127 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-04-21T08:41:02.127 INFO:teuthology.orchestra.run.smithi086.stdout: "c2a8e01a": { 2024-04-21T08:41:02.127 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 2, 2024-04-21T08:41:02.127 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-04-21T08:41:02.127 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-21T08:41:02.127 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-04-21T08:41:02.127 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-21T08:41:02.127 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:41:02.128 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 30.0, 2024-04-21T08:41:02.128 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 81.9, 2024-04-21T08:41:02.128 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-04-21T08:41:02.128 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-04-21T08:41:02.128 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-04-21T08:41:02.128 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-21T08:41:02.128 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-04-21T08:41:02.128 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-21T08:41:02.128 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:41:02.128 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:41:02.128 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:41:02.128 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:41:02.128 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:41:02.128 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:41:02.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.126+0000 7efc9e870640 1 -- 172.21.15.86:0/1103533103 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efc800a3d20 msgr2=0x7efc800a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:02.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.126+0000 7efc9e870640 1 --2- 172.21.15.86:0/1103533103 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efc800a3d20 0x7efc800a61e0 secure :-1 s=READY pgs=127 cs=0 l=1 rev1=1 crypto rx=0x7efc90005fd0 tx=0x7efc90009600 comp rx=0 tx=0).stop 2024-04-21T08:41:02.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.126+0000 7efc9e870640 1 -- 172.21.15.86:0/1103533103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc98106160 msgr2=0x7efc98197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:02.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.126+0000 7efc9e870640 1 --2- 172.21.15.86:0/1103533103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc98106160 0x7efc98197ba0 secure :-1 s=READY pgs=871 cs=0 l=1 rev1=1 crypto rx=0x7efc8c036040 tx=0x7efc8c02c7e0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.126+0000 7efc9e870640 1 -- 172.21.15.86:0/1103533103 shutdown_connections 2024-04-21T08:41:02.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.126+0000 7efc9e870640 1 --2- 172.21.15.86:0/1103533103 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efc800a3d20 0x7efc800a61e0 unknown :-1 s=CLOSED pgs=127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.126+0000 7efc9e870640 1 --2- 172.21.15.86:0/1103533103 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc9819bf20 0x7efc9819c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.126+0000 7efc9e870640 1 --2- 172.21.15.86:0/1103533103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc98106160 0x7efc98197ba0 unknown :-1 s=CLOSED pgs=871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.126+0000 7efc9e870640 1 --2- 172.21.15.86:0/1103533103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc98101090 0x7efc98197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.126+0000 7efc9e870640 1 -- 172.21.15.86:0/1103533103 >> 172.21.15.86:0/1103533103 conn(0x7efc980771d0 msgr2=0x7efc980fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:02.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.126+0000 7efc9e870640 1 -- 172.21.15.86:0/1103533103 shutdown_connections 2024-04-21T08:41:02.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.126+0000 7efc9e870640 1 -- 172.21.15.86:0/1103533103 wait complete. 2024-04-21T08:41:02.136 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced, set_id: c2a8e01a, quiesce duration: 21.9 2024-04-21T08:41:02.608 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:41:02.608 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:41:02.608 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:41:02.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:02 smithi086 ceph-mon[27939]: pgmap v70: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-21T08:41:02.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:02 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2370387356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:41:02.648 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:02 smithi179 ceph-mon[31829]: pgmap v70: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-21T08:41:02.648 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:02 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2370387356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:41:02.718 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:02 smithi088 ceph-mon[31259]: pgmap v70: 129 pgs: 129 active+clean; 2.7 MiB data, 390 MiB used, 1.0 TiB / 1.0 TiB avail; 22 KiB/s rd, 20 KiB/s wr, 15 op/s 2024-04-21T08:41:02.718 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:02 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2370387356' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:41:02.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.714+0000 7f05aa546640 1 -- 172.21.15.86:0/939981633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05a4101090 msgr2=0x7f05a4101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:02.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.714+0000 7f05aa546640 1 --2- 172.21.15.86:0/939981633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05a4101090 0x7f05a4101470 secure :-1 s=READY pgs=876 cs=0 l=1 rev1=1 crypto rx=0x7f0594009900 tx=0x7f059402bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:02.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.714+0000 7f05aa546640 1 -- 172.21.15.86:0/939981633 shutdown_connections 2024-04-21T08:41:02.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.714+0000 7f05aa546640 1 --2- 172.21.15.86:0/939981633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05a4106160 0x7f05a410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.714+0000 7f05aa546640 1 --2- 172.21.15.86:0/939981633 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05a4101a40 0x7f05a4105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.714+0000 7f05aa546640 1 --2- 172.21.15.86:0/939981633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05a4101090 0x7f05a4101470 unknown :-1 s=CLOSED pgs=876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.714+0000 7f05aa546640 1 -- 172.21.15.86:0/939981633 >> 172.21.15.86:0/939981633 conn(0x7f05a4077f50 msgr2=0x7f05a4078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:02.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.714+0000 7f05aa546640 1 -- 172.21.15.86:0/939981633 shutdown_connections 2024-04-21T08:41:02.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.714+0000 7f05aa546640 1 -- 172.21.15.86:0/939981633 wait complete. 2024-04-21T08:41:02.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.715+0000 7f05aa546640 1 Processor -- start 2024-04-21T08:41:02.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.715+0000 7f05aa546640 1 -- start start 2024-04-21T08:41:02.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.715+0000 7f05aa546640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05a4101090 0x7f05a419e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.715+0000 7f05aa546640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05a4106160 0x7f05a419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.715+0000 7f05aa546640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05a41985f0 0x7f05a4198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.715+0000 7f05aa546640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f05a41110d0 con 0x7f05a4106160 2024-04-21T08:41:02.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.715+0000 7f05aa546640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f05a4110f50 con 0x7f05a4101090 2024-04-21T08:41:02.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.715+0000 7f05aa546640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f05a4111250 con 0x7f05a41985f0 2024-04-21T08:41:02.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.716+0000 7f05a37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05a4106160 0x7f05a419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.716+0000 7f05a37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05a4106160 0x7f05a419e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60662/0 (socket says 172.21.15.86:60662) 2024-04-21T08:41:02.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.716+0000 7f05a37fe640 1 -- 172.21.15.86:0/272186145 learned_addr learned my addr 172.21.15.86:0/272186145 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:02.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.716+0000 7f05a8ae3640 1 --2- 172.21.15.86:0/272186145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05a41985f0 0x7f05a4198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.716+0000 7f05a3fff640 1 --2- 172.21.15.86:0/272186145 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05a4101090 0x7f05a419e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.716+0000 7f05a37fe640 1 -- 172.21.15.86:0/272186145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05a41985f0 msgr2=0x7f05a4198aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:02.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.716+0000 7f05a37fe640 1 --2- 172.21.15.86:0/272186145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05a41985f0 0x7f05a4198aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.716+0000 7f05a37fe640 1 -- 172.21.15.86:0/272186145 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05a4101090 msgr2=0x7f05a419e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:02.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.716+0000 7f05a37fe640 1 --2- 172.21.15.86:0/272186145 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05a4101090 0x7f05a419e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.716+0000 7f05a37fe640 1 -- 172.21.15.86:0/272186145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f05a4199110 con 0x7f05a4106160 2024-04-21T08:41:02.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.716+0000 7f05a3fff640 1 --2- 172.21.15.86:0/272186145 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05a4101090 0x7f05a419e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:02.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.716+0000 7f05a37fe640 1 --2- 172.21.15.86:0/272186145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05a4106160 0x7f05a419e930 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f059800d770 tx=0x7f059800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:02.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.716+0000 7f05a17fa640 1 -- 172.21.15.86:0/272186145 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0598017070 con 0x7f05a4106160 2024-04-21T08:41:02.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.716+0000 7f05a17fa640 1 -- 172.21.15.86:0/272186145 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0598009c30 con 0x7f05a4106160 2024-04-21T08:41:02.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.716+0000 7f05aa546640 1 -- 172.21.15.86:0/272186145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f05a4199400 con 0x7f05a4106160 2024-04-21T08:41:02.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.716+0000 7f05a17fa640 1 -- 172.21.15.86:0/272186145 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059800f620 con 0x7f05a4106160 2024-04-21T08:41:02.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.716+0000 7f05aa546640 1 -- 172.21.15.86:0/272186145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f05a41a5600 con 0x7f05a4106160 2024-04-21T08:41:02.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.718+0000 7f05a17fa640 1 -- 172.21.15.86:0/272186145 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f059801a070 con 0x7f05a4106160 2024-04-21T08:41:02.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.719+0000 7f05ad20b740 1 Processor -- start 2024-04-21T08:41:02.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.719+0000 7f05ad20b740 1 -- start start 2024-04-21T08:41:02.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.719+0000 7f05a17fa640 1 --2- 172.21.15.86:0/272186145 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f058c0a3d20 0x7f058c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.719+0000 7f05ad20b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564cf5d848e0 0x564cf5d84cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.719+0000 7f05ad20b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cf5d85290 0x564cf5d8d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.719+0000 7f05ad20b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cf5d8de10 0x564cf5d90260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.719+0000 7f05ad20b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564cf5ce8f00 con 0x564cf5d85290 2024-04-21T08:41:02.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.719+0000 7f05ad20b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564cf5c86590 con 0x564cf5d848e0 2024-04-21T08:41:02.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.719+0000 7f05ad20b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564cf5ce4ec0 con 0x564cf5d8de10 2024-04-21T08:41:02.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.719+0000 7f05a8ae3640 1 --2- 172.21.15.86:0/272186145 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f058c0a3d20 0x7f058c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.719+0000 7f05a17fa640 1 -- 172.21.15.86:0/272186145 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f05980d06e0 con 0x7f05a4106160 2024-04-21T08:41:02.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.719+0000 7f05a37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cf5d85290 0x564cf5d8d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.719+0000 7f05a37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cf5d85290 0x564cf5d8d840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60678/0 (socket says 172.21.15.86:60678) 2024-04-21T08:41:02.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.719+0000 7f05a37fe640 1 -- 172.21.15.86:0/3752306847 learned_addr learned my addr 172.21.15.86:0/3752306847 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:02.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.719+0000 7f05a8ae3640 1 --2- 172.21.15.86:0/3752306847 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cf5d8de10 0x564cf5d90260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.719+0000 7f05a3fff640 1 --2- 172.21.15.86:0/3752306847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564cf5d848e0 0x564cf5d84cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.719+0000 7f05a37fe640 1 -- 172.21.15.86:0/3752306847 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cf5d8de10 msgr2=0x564cf5d90260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:02.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.719+0000 7f05a37fe640 1 --2- 172.21.15.86:0/3752306847 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cf5d8de10 0x564cf5d90260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.719+0000 7f05a37fe640 1 -- 172.21.15.86:0/3752306847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564cf5d848e0 msgr2=0x564cf5d84cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:02.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.719+0000 7f05a37fe640 1 --2- 172.21.15.86:0/3752306847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564cf5d848e0 0x564cf5d84cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.719+0000 7f05a37fe640 1 -- 172.21.15.86:0/3752306847 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cf5ac37e0 con 0x564cf5d85290 2024-04-21T08:41:02.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.719+0000 7f05a8ae3640 1 --2- 172.21.15.86:0/272186145 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f058c0a3d20 0x7f058c0a61e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f059c005fd0 tx=0x7f059c009e60 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:02.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.719+0000 7f05a37fe640 1 --2- 172.21.15.86:0/3752306847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cf5d85290 0x564cf5d8d840 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f0598000c00 tx=0x7f0598024cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=99650e8b0da9be11 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:02.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.719+0000 7f058a7fc640 1 -- 172.21.15.86:0/3752306847 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0598021040 con 0x564cf5d85290 2024-04-21T08:41:02.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.720+0000 7f058a7fc640 1 -- 172.21.15.86:0/3752306847 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0598009de0 con 0x564cf5d85290 2024-04-21T08:41:02.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.720+0000 7f058a7fc640 1 -- 172.21.15.86:0/3752306847 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0598098980 con 0x564cf5d85290 2024-04-21T08:41:02.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.720+0000 7f05ad20b740 1 -- 172.21.15.86:0/3752306847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cf5d85290 msgr2=0x564cf5d8d840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:02.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.720+0000 7f05ad20b740 1 --2- 172.21.15.86:0/3752306847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cf5d85290 0x564cf5d8d840 secure :-1 s=READY pgs=878 cs=0 l=1 rev1=1 crypto rx=0x7f0598000c00 tx=0x7f0598024cf0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.720+0000 7f05ad20b740 1 -- 172.21.15.86:0/3752306847 shutdown_connections 2024-04-21T08:41:02.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.720+0000 7f05ad20b740 1 --2- 172.21.15.86:0/3752306847 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cf5d8de10 0x564cf5d90260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.720+0000 7f05ad20b740 1 --2- 172.21.15.86:0/3752306847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cf5d85290 0x564cf5d8d840 unknown :-1 s=CLOSED pgs=878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.720+0000 7f05ad20b740 1 --2- 172.21.15.86:0/3752306847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564cf5d848e0 0x564cf5d84cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.720+0000 7f05ad20b740 1 -- 172.21.15.86:0/3752306847 >> 172.21.15.86:0/3752306847 conn(0x564cf5d7efb0 msgr2=0x564cf5d7f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:02.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.720+0000 7f05ad20b740 1 -- 172.21.15.86:0/3752306847 shutdown_connections 2024-04-21T08:41:02.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.720+0000 7f05ad20b740 1 -- 172.21.15.86:0/3752306847 wait complete. 2024-04-21T08:41:02.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.721+0000 7f05ad20b740 1 -- start start 2024-04-21T08:41:02.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.721+0000 7f05ad20b740 1 Processor -- start 2024-04-21T08:41:02.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.721+0000 7f05ad20b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cf5d848e0 0x564cf5d8ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.721+0000 7f05ad20b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cf5d85290 0x564cf5d8b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.721+0000 7f05ad20b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564cf5d8de10 0x564cf5d8b880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.721+0000 7f05ad20b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564cf5d90cc0 con 0x564cf5d85290 2024-04-21T08:41:02.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.721+0000 7f05ad20b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564cf5ca2580 con 0x564cf5d8de10 2024-04-21T08:41:02.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.721+0000 7f05ad20b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564cf5d90e40 con 0x564cf5d848e0 2024-04-21T08:41:02.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.721+0000 7f05ad20b740 10 client.0 _ll_register_callbacks cb 0x564cf5ca8650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:02.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.721+0000 7f05a37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cf5d85290 0x564cf5d8b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.721+0000 7f05a37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cf5d85290 0x564cf5d8b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60692/0 (socket says 172.21.15.86:60692) 2024-04-21T08:41:02.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.721+0000 7f05a37fe640 1 -- 172.21.15.86:0/2174067548 learned_addr learned my addr 172.21.15.86:0/2174067548 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:02.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.721+0000 7f05a8ae3640 1 --2- 172.21.15.86:0/2174067548 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564cf5d8de10 0x564cf5d8b880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.721+0000 7f05a3fff640 1 --2- 172.21.15.86:0/2174067548 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cf5d848e0 0x564cf5d8ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.721+0000 7f05a37fe640 1 -- 172.21.15.86:0/2174067548 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cf5d848e0 msgr2=0x564cf5d8ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:02.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.721+0000 7f05a37fe640 1 --2- 172.21.15.86:0/2174067548 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cf5d848e0 0x564cf5d8ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.721+0000 7f05a37fe640 1 -- 172.21.15.86:0/2174067548 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564cf5d8de10 msgr2=0x564cf5d8b880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:02.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.721+0000 7f05a37fe640 1 --2- 172.21.15.86:0/2174067548 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564cf5d8de10 0x564cf5d8b880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.721+0000 7f05a37fe640 1 -- 172.21.15.86:0/2174067548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cf5b23f70 con 0x564cf5d85290 2024-04-21T08:41:02.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.722+0000 7f05a37fe640 1 --2- 172.21.15.86:0/2174067548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cf5d85290 0x564cf5d8b340 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f059800d740 tx=0x7f05980cdae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c62546c65a24868 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:02.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.722+0000 7f05837fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:02.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.722+0000 7f05837fe640 1 -- 172.21.15.86:0/2174067548 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05980b4070 con 0x564cf5d85290 2024-04-21T08:41:02.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.722+0000 7f05837fe640 1 -- 172.21.15.86:0/2174067548 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f059800e350 con 0x564cf5d85290 2024-04-21T08:41:02.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.722+0000 7f05ad20b740 1 -- 172.21.15.86:0/2174067548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0598009de0 con 0x564cf5d85290 2024-04-21T08:41:02.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.722+0000 7f05837fe640 1 -- 172.21.15.86:0/2174067548 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0598021070 con 0x564cf5d85290 2024-04-21T08:41:02.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.722+0000 7f05ad20b740 10 client.16800 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:02.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.722+0000 7f05ad20b740 1 -- 172.21.15.86:0/2174067548 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564cf59bb170 con 0x564cf5d85290 2024-04-21T08:41:02.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.722+0000 7f05837fe640 1 -- 172.21.15.86:0/2174067548 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f059807d030 con 0x564cf5d85290 2024-04-21T08:41:02.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.722+0000 7f05837fe640 1 -- 172.21.15.86:0/2174067548 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f05980cdd40 con 0x564cf5d85290 2024-04-21T08:41:02.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.722+0000 7f05ad20b740 10 client.16800 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:02.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.722+0000 7f05ad20b740 10 client.16800 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:02.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.722+0000 7f05ad20b740 20 client.16800 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:02.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.722+0000 7f05ad20b740 1 --2- 172.21.15.86:0/2174067548 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564cf5d9e130 0x564cf5da0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.722+0000 7f05ad20b740 4 client.16800 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:02.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.722+0000 7f05ad20b740 1 -- 172.21.15.86:0/2174067548 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564cf5cf6920 con 0x564cf5d9e130 2024-04-21T08:41:02.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.722+0000 7f05a3fff640 1 --2- 172.21.15.86:0/2174067548 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564cf5d9e130 0x564cf5da0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.723+0000 7f05837fe640 10 client.16800 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:02.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.723+0000 7f05a3fff640 1 --2- 172.21.15.86:0/2174067548 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564cf5d9e130 0x564cf5da0520 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=44d639dd858aa17 server_cookie=494dacb36fd5182d in_seq=0 out_seq=0 2024-04-21T08:41:02.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.724+0000 7f05837fe640 1 -- 172.21.15.86:0/2174067548 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564cf5cf6920 con 0x564cf5d9e130 2024-04-21T08:41:02.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.724+0000 7f05837fe640 10 client.16800 handle_command_reply: tid=0 2024-04-21T08:41:02.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.724+0000 7f05ad20b740 1 client.16800 shutdown 2024-04-21T08:41:02.734 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.724+0000 7f05ad20b740 1 -- 172.21.15.86:0/2174067548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cf5d85290 msgr2=0x564cf5d8b340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:02.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.724+0000 7f05ad20b740 1 --2- 172.21.15.86:0/2174067548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cf5d85290 0x564cf5d8b340 secure :-1 s=READY pgs=879 cs=0 l=1 rev1=1 crypto rx=0x7f059800d740 tx=0x7f05980cdae0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.724+0000 7f05ad20b740 1 -- 172.21.15.86:0/2174067548 shutdown_connections 2024-04-21T08:41:02.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.724+0000 7f05ad20b740 1 --2- 172.21.15.86:0/2174067548 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564cf5d9e130 0x564cf5da0520 crc :-1 s=READY pgs=99 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.724+0000 7f05ad20b740 1 --2- 172.21.15.86:0/2174067548 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564cf5d8de10 0x564cf5d8b880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.724+0000 7f05ad20b740 1 --2- 172.21.15.86:0/2174067548 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cf5d85290 0x564cf5d8b340 unknown :-1 s=CLOSED pgs=879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.724+0000 7f05837fe640 0 client.16800 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:02.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.724+0000 7f05ad20b740 1 --2- 172.21.15.86:0/2174067548 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cf5d848e0 0x564cf5d8ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.724+0000 7f05ad20b740 1 -- 172.21.15.86:0/2174067548 >> 172.21.15.86:0/2174067548 conn(0x564cf5d7efb0 msgr2=0x564cf5d80d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:02.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.724+0000 7f05ad20b740 1 -- 172.21.15.86:0/2174067548 shutdown_connections 2024-04-21T08:41:02.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.724+0000 7f05ad20b740 1 -- 172.21.15.86:0/2174067548 wait complete. 2024-04-21T08:41:02.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.724+0000 7f05ad20b740 20 client.16800 trim_cache size 0 max 16384 2024-04-21T08:41:02.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.741+0000 7f05ad20b740 1 Processor -- start 2024-04-21T08:41:02.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.741+0000 7f05ad20b740 1 -- start start 2024-04-21T08:41:02.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.741+0000 7f05ad20b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cf5d848e0 0x564cf5d99b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.741+0000 7f05ad20b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cf5d85290 0x564cf5d8b6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.741+0000 7f05ad20b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564cf5d8de10 0x564cf5d80a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.741+0000 7f05ad20b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f05a41af250 con 0x564cf5d848e0 2024-04-21T08:41:02.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.741+0000 7f05ad20b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f05a41b0d90 con 0x564cf5d8de10 2024-04-21T08:41:02.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.741+0000 7f05ad20b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f05a41af920 con 0x564cf5d85290 2024-04-21T08:41:02.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.741+0000 7f05a3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cf5d848e0 0x564cf5d99b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.741+0000 7f05a3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cf5d848e0 0x564cf5d99b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60706/0 (socket says 172.21.15.86:60706) 2024-04-21T08:41:02.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.741+0000 7f05a3fff640 1 -- 172.21.15.86:0/3369606337 learned_addr learned my addr 172.21.15.86:0/3369606337 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:02.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.742+0000 7f05a37fe640 1 --2- 172.21.15.86:0/3369606337 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cf5d85290 0x564cf5d8b6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.742+0000 7f05a3fff640 1 -- 172.21.15.86:0/3369606337 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cf5d85290 msgr2=0x564cf5d8b6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:02.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.742+0000 7f05a3fff640 1 --2- 172.21.15.86:0/3369606337 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cf5d85290 0x564cf5d8b6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.742+0000 7f05a3fff640 1 -- 172.21.15.86:0/3369606337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564cf5d8de10 msgr2=0x564cf5d80a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:02.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.742+0000 7f05a8ae3640 1 --2- 172.21.15.86:0/3369606337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564cf5d8de10 0x564cf5d80a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.742+0000 7f05a3fff640 1 --2- 172.21.15.86:0/3369606337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564cf5d8de10 0x564cf5d80a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.742+0000 7f05a3fff640 1 -- 172.21.15.86:0/3369606337 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564cf5d83ba0 con 0x564cf5d848e0 2024-04-21T08:41:02.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.742+0000 7f05a3fff640 1 --2- 172.21.15.86:0/3369606337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cf5d848e0 0x564cf5d99b80 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f0594009900 tx=0x7f0594000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fcf2fd0975cdc3b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:02.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.742+0000 7f0589ffb640 1 -- 172.21.15.86:0/3369606337 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f059404e070 con 0x564cf5d848e0 2024-04-21T08:41:02.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.742+0000 7f0589ffb640 1 -- 172.21.15.86:0/3369606337 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f059402cc90 con 0x564cf5d848e0 2024-04-21T08:41:02.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.742+0000 7f0589ffb640 1 -- 172.21.15.86:0/3369606337 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0594049770 con 0x564cf5d848e0 2024-04-21T08:41:02.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.743+0000 7f05ad20b740 1 -- 172.21.15.86:0/3369606337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cf5d848e0 msgr2=0x564cf5d99b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:02.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.743+0000 7f05ad20b740 1 --2- 172.21.15.86:0/3369606337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cf5d848e0 0x564cf5d99b80 secure :-1 s=READY pgs=880 cs=0 l=1 rev1=1 crypto rx=0x7f0594009900 tx=0x7f0594000bc0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.743+0000 7f05ad20b740 1 -- 172.21.15.86:0/3369606337 shutdown_connections 2024-04-21T08:41:02.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.743+0000 7f05ad20b740 1 --2- 172.21.15.86:0/3369606337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564cf5d8de10 0x564cf5d80a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.743+0000 7f05ad20b740 1 --2- 172.21.15.86:0/3369606337 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cf5d85290 0x564cf5d8b6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.743+0000 7f05ad20b740 1 --2- 172.21.15.86:0/3369606337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cf5d848e0 0x564cf5d99b80 unknown :-1 s=CLOSED pgs=880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.743+0000 7f05ad20b740 1 -- 172.21.15.86:0/3369606337 >> 172.21.15.86:0/3369606337 conn(0x564cf5d7efb0 msgr2=0x564cf5d80330 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:02.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.743+0000 7f05ad20b740 1 -- 172.21.15.86:0/3369606337 shutdown_connections 2024-04-21T08:41:02.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.743+0000 7f05ad20b740 1 -- 172.21.15.86:0/3369606337 wait complete. 2024-04-21T08:41:02.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.743+0000 7f05ad20b740 1 -- start start 2024-04-21T08:41:02.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.743+0000 7f05ad20b740 1 Processor -- start 2024-04-21T08:41:02.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.743+0000 7f05ad20b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cf5d848e0 0x564cf5d87cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.743+0000 7f05ad20b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cf5d85290 0x564cf5d88230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.743+0000 7f05ad20b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564cf5d8de10 0x564cf5d88770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.743+0000 7f05ad20b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f05a41b06c0 con 0x564cf5d85290 2024-04-21T08:41:02.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.743+0000 7f05ad20b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f05a41afff0 con 0x564cf5d8de10 2024-04-21T08:41:02.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.743+0000 7f05ad20b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f05a41ad210 con 0x564cf5d848e0 2024-04-21T08:41:02.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.743+0000 7f05ad20b740 10 client.0 _ll_register_callbacks cb 0x7f059c015760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:02.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.743+0000 7f05a37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cf5d85290 0x564cf5d88230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.744+0000 7f05a37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cf5d85290 0x564cf5d88230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60718/0 (socket says 172.21.15.86:60718) 2024-04-21T08:41:02.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.744+0000 7f05a37fe640 1 -- 172.21.15.86:0/2293278435 learned_addr learned my addr 172.21.15.86:0/2293278435 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:02.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.744+0000 7f05a3fff640 1 --2- 172.21.15.86:0/2293278435 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cf5d848e0 0x564cf5d87cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.744+0000 7f05a37fe640 1 -- 172.21.15.86:0/2293278435 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cf5d848e0 msgr2=0x564cf5d87cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:02.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.744+0000 7f05a37fe640 1 --2- 172.21.15.86:0/2293278435 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cf5d848e0 0x564cf5d87cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.744+0000 7f05a37fe640 1 -- 172.21.15.86:0/2293278435 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564cf5d8de10 msgr2=0x564cf5d88770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:02.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.744+0000 7f05a37fe640 1 --2- 172.21.15.86:0/2293278435 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564cf5d8de10 0x564cf5d88770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.744+0000 7f05a37fe640 1 -- 172.21.15.86:0/2293278435 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564cf5e035e0 con 0x564cf5d85290 2024-04-21T08:41:02.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.744+0000 7f05a37fe640 1 --2- 172.21.15.86:0/2293278435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cf5d85290 0x564cf5d88230 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f0598000c00 tx=0x7f0598098bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2165e870cd6abd1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:02.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.744+0000 7f05837fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:02.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.744+0000 7f05837fe640 1 -- 172.21.15.86:0/2293278435 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f05980b4070 con 0x564cf5d85290 2024-04-21T08:41:02.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.744+0000 7f05837fe640 1 -- 172.21.15.86:0/2293278435 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f05980980b0 con 0x564cf5d85290 2024-04-21T08:41:02.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.744+0000 7f05ad20b740 1 -- 172.21.15.86:0/2293278435 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f059402cc90 con 0x564cf5d85290 2024-04-21T08:41:02.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.744+0000 7f05837fe640 1 -- 172.21.15.86:0/2293278435 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f05980985b0 con 0x564cf5d85290 2024-04-21T08:41:02.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.744+0000 7f05ad20b740 10 client.16806 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:02.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.744+0000 7f05ad20b740 1 -- 172.21.15.86:0/2293278435 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564cf5e04900 con 0x564cf5d85290 2024-04-21T08:41:02.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.744+0000 7f05837fe640 1 -- 172.21.15.86:0/2293278435 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f059402cc90 con 0x564cf5d85290 2024-04-21T08:41:02.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.744+0000 7f05837fe640 1 -- 172.21.15.86:0/2293278435 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f05980a9080 con 0x564cf5d85290 2024-04-21T08:41:02.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.744+0000 7f05ad20b740 10 client.16806 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:02.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.745+0000 7f05ad20b740 10 client.16806 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:02.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.745+0000 7f05ad20b740 20 client.16806 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:02.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.745+0000 7f05ad20b740 1 --2- 172.21.15.86:0/2293278435 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564cf5e04d90 0x564cf5e071f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.745+0000 7f05ad20b740 4 client.16806 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:41:02.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.745+0000 7f05ad20b740 1 -- 172.21.15.86:0/2293278435 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x564cf5cf0790 con 0x564cf5e04d90 2024-04-21T08:41:02.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.745+0000 7f05a3fff640 1 --2- 172.21.15.86:0/2293278435 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564cf5e04d90 0x564cf5e071f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.745+0000 7f05837fe640 10 client.16806 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:02.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.745+0000 7f05a3fff640 1 --2- 172.21.15.86:0/2293278435 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564cf5e04d90 0x564cf5e071f0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e76a81da6540182c server_cookie=90034966cf854c0a in_seq=0 out_seq=0 2024-04-21T08:41:02.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.746+0000 7f05837fe640 1 -- 172.21.15.86:0/2293278435 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x564cf5cf0790 con 0x564cf5e04d90 2024-04-21T08:41:02.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.746+0000 7f05837fe640 10 client.16806 handle_command_reply: tid=0 2024-04-21T08:41:02.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.746+0000 7f05ad20b740 1 client.16806 shutdown 2024-04-21T08:41:02.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.747+0000 7f05ad20b740 1 -- 172.21.15.86:0/2293278435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cf5d85290 msgr2=0x564cf5d88230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:02.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.747+0000 7f05ad20b740 1 --2- 172.21.15.86:0/2293278435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cf5d85290 0x564cf5d88230 secure :-1 s=READY pgs=881 cs=0 l=1 rev1=1 crypto rx=0x7f0598000c00 tx=0x7f0598098bb0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.747+0000 7f05ad20b740 1 -- 172.21.15.86:0/2293278435 shutdown_connections 2024-04-21T08:41:02.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.747+0000 7f05ad20b740 1 --2- 172.21.15.86:0/2293278435 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564cf5e04d90 0x564cf5e071f0 crc :-1 s=READY pgs=100 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.747+0000 7f05ad20b740 1 --2- 172.21.15.86:0/2293278435 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564cf5d8de10 0x564cf5d88770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.747+0000 7f05ad20b740 1 --2- 172.21.15.86:0/2293278435 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564cf5d85290 0x564cf5d88230 unknown :-1 s=CLOSED pgs=881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.747+0000 7f05ad20b740 1 --2- 172.21.15.86:0/2293278435 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564cf5d848e0 0x564cf5d87cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.747+0000 7f05837fe640 0 client.16806 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:02.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.747+0000 7f05ad20b740 1 -- 172.21.15.86:0/2293278435 >> 172.21.15.86:0/2293278435 conn(0x564cf5d7efb0 msgr2=0x564cf5d80bc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:02.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.747+0000 7f05ad20b740 1 -- 172.21.15.86:0/2293278435 shutdown_connections 2024-04-21T08:41:02.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.747+0000 7f05ad20b740 1 -- 172.21.15.86:0/2293278435 wait complete. 2024-04-21T08:41:02.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.747+0000 7f05ad20b740 20 client.16806 trim_cache size 0 max 16384 2024-04-21T08:41:02.752 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:41:02.752 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-21T08:41:02.752 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "e0bda25e-ed53-4040-ab17-597d95a75b62", 2024-04-21T08:41:02.752 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-21T08:41:02.753 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:41:02.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.747+0000 7f05aa546640 1 -- 172.21.15.86:0/272186145 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f058c0a3d20 msgr2=0x7f058c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:02.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.747+0000 7f05aa546640 1 --2- 172.21.15.86:0/272186145 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f058c0a3d20 0x7f058c0a61e0 secure :-1 s=READY pgs=128 cs=0 l=1 rev1=1 crypto rx=0x7f059c005fd0 tx=0x7f059c009e60 comp rx=0 tx=0).stop 2024-04-21T08:41:02.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.747+0000 7f05aa546640 1 -- 172.21.15.86:0/272186145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05a4106160 msgr2=0x7f05a419e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:02.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.747+0000 7f05aa546640 1 --2- 172.21.15.86:0/272186145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05a4106160 0x7f05a419e930 secure :-1 s=READY pgs=877 cs=0 l=1 rev1=1 crypto rx=0x7f059800d770 tx=0x7f059800dc30 comp rx=0 tx=0).stop 2024-04-21T08:41:02.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.747+0000 7f05aa546640 1 -- 172.21.15.86:0/272186145 shutdown_connections 2024-04-21T08:41:02.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.747+0000 7f05aa546640 1 --2- 172.21.15.86:0/272186145 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f058c0a3d20 0x7f058c0a61e0 unknown :-1 s=CLOSED pgs=128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.747+0000 7f05aa546640 1 --2- 172.21.15.86:0/272186145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f05a41985f0 0x7f05a4198aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.747+0000 7f05aa546640 1 --2- 172.21.15.86:0/272186145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f05a4106160 0x7f05a419e930 unknown :-1 s=CLOSED pgs=877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.747+0000 7f05aa546640 1 --2- 172.21.15.86:0/272186145 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f05a4101090 0x7f05a419e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.747+0000 7f05aa546640 1 -- 172.21.15.86:0/272186145 >> 172.21.15.86:0/272186145 conn(0x7f05a4077f50 msgr2=0x7f05a4104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:02.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.747+0000 7f05aa546640 1 -- 172.21.15.86:0/272186145 shutdown_connections 2024-04-21T08:41:02.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.748+0000 7f05aa546640 1 -- 172.21.15.86:0/272186145 wait complete. 2024-04-21T08:41:02.757 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:41:02.757 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:41:02.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.900+0000 7f7fb534a640 1 -- 172.21.15.86:0/446395489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fb0100d60 msgr2=0x7f7fb010ee00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:02.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.900+0000 7f7fb534a640 1 --2- 172.21.15.86:0/446395489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fb0100d60 0x7f7fb010ee00 secure :-1 s=READY pgs=882 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8009cd0 tx=0x7f7fa802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.900+0000 7f7fb534a640 1 -- 172.21.15.86:0/446395489 shutdown_connections 2024-04-21T08:41:02.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.900+0000 7f7fb534a640 1 --2- 172.21.15.86:0/446395489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fb0100d60 0x7f7fb010ee00 unknown :-1 s=CLOSED pgs=882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.900+0000 7f7fb534a640 1 --2- 172.21.15.86:0/446395489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7fb01003e0 0x7f7fb0100820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.900+0000 7f7fb534a640 1 --2- 172.21.15.86:0/446395489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fb0107fd0 0x7f7fb01083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.900+0000 7f7fb534a640 1 -- 172.21.15.86:0/446395489 >> 172.21.15.86:0/446395489 conn(0x7f7fb0077360 msgr2=0x7f7fb00fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:02.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.900+0000 7f7fb534a640 1 -- 172.21.15.86:0/446395489 shutdown_connections 2024-04-21T08:41:02.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.900+0000 7f7fb534a640 1 -- 172.21.15.86:0/446395489 wait complete. 2024-04-21T08:41:02.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.900+0000 7f7fb534a640 1 Processor -- start 2024-04-21T08:41:02.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.901+0000 7f7fb534a640 1 -- start start 2024-04-21T08:41:02.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.901+0000 7f7fb534a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7fb01003e0 0x7f7fb0198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.901+0000 7f7fb534a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fb0100d60 0x7f7fb0198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.901+0000 7f7fb534a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fb0107fd0 0x7f7fb019cff0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.901+0000 7f7fb534a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb01121b0 con 0x7f7fb0100d60 2024-04-21T08:41:02.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.901+0000 7f7fb534a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb0112030 con 0x7f7fb0107fd0 2024-04-21T08:41:02.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.901+0000 7f7fb534a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb0112330 con 0x7f7fb01003e0 2024-04-21T08:41:02.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.901+0000 7f7fae7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fb0100d60 0x7f7fb0198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.901+0000 7f7fae7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fb0100d60 0x7f7fb0198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60740/0 (socket says 172.21.15.86:60740) 2024-04-21T08:41:02.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.901+0000 7f7fae7fc640 1 -- 172.21.15.86:0/2166903452 learned_addr learned my addr 172.21.15.86:0/2166903452 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:02.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.901+0000 7f7faeffd640 1 --2- 172.21.15.86:0/2166903452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7fb01003e0 0x7f7fb0198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.901+0000 7f7fae7fc640 1 -- 172.21.15.86:0/2166903452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7fb01003e0 msgr2=0x7f7fb0198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:02.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.901+0000 7f7fae7fc640 1 --2- 172.21.15.86:0/2166903452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7fb01003e0 0x7f7fb0198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.901+0000 7f7fae7fc640 1 -- 172.21.15.86:0/2166903452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fb0107fd0 msgr2=0x7f7fb019cff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:02.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.901+0000 7f7fae7fc640 1 --2- 172.21.15.86:0/2166903452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fb0107fd0 0x7f7fb019cff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.901+0000 7f7fae7fc640 1 -- 172.21.15.86:0/2166903452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7fb019d720 con 0x7f7fb0100d60 2024-04-21T08:41:02.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.901+0000 7f7fae7fc640 1 --2- 172.21.15.86:0/2166903452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fb0100d60 0x7f7fb0198c70 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f7fa400d770 tx=0x7f7fa400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:02.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.901+0000 7f7f97fff640 1 -- 172.21.15.86:0/2166903452 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4017070 con 0x7f7fb0100d60 2024-04-21T08:41:02.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.901+0000 7f7fb534a640 1 -- 172.21.15.86:0/2166903452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7fb019da10 con 0x7f7fb0100d60 2024-04-21T08:41:02.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.901+0000 7f7f97fff640 1 -- 172.21.15.86:0/2166903452 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fa40054d0 con 0x7f7fb0100d60 2024-04-21T08:41:02.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.902+0000 7f7f97fff640 1 -- 172.21.15.86:0/2166903452 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4004c30 con 0x7f7fb0100d60 2024-04-21T08:41:02.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.902+0000 7f7fb534a640 1 -- 172.21.15.86:0/2166903452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7fb01a52a0 con 0x7f7fb0100d60 2024-04-21T08:41:02.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.904+0000 7f7f97fff640 1 -- 172.21.15.86:0/2166903452 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7fa4004e10 con 0x7f7fb0100d60 2024-04-21T08:41:02.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.904+0000 7f7fb800f740 1 Processor -- start 2024-04-21T08:41:02.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.904+0000 7f7fb800f740 1 -- start start 2024-04-21T08:41:02.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.904+0000 7f7fb800f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562143672230 0x562143672610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.904+0000 7f7fb800f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562143672b50 0x56214367f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.904+0000 7f7f97fff640 1 --2- 172.21.15.86:0/2166903452 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7f980a3d20 0x7f7f980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.904+0000 7f7fb800f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56214367fe90 0x562143682280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.904+0000 7f7fb800f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5621435d71e0 con 0x562143672b50 2024-04-21T08:41:02.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.904+0000 7f7fb800f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x562143574390 con 0x56214367fe90 2024-04-21T08:41:02.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.904+0000 7f7fb800f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5621435d31a0 con 0x562143672230 2024-04-21T08:41:02.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.904+0000 7f7fae7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562143672b50 0x56214367f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.904+0000 7f7faf7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562143672230 0x562143672610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.904+0000 7f7fae7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562143672b50 0x56214367f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60748/0 (socket says 172.21.15.86:60748) 2024-04-21T08:41:02.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.904+0000 7f7fae7fc640 1 -- 172.21.15.86:0/1322958673 learned_addr learned my addr 172.21.15.86:0/1322958673 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:02.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.904+0000 7f7faeffd640 1 --2- 172.21.15.86:0/2166903452 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7f980a3d20 0x7f7f980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.905+0000 7f7faf7fe640 1 --2- 172.21.15.86:0/1322958673 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56214367fe90 0x562143682280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.905+0000 7f7f97fff640 1 -- 172.21.15.86:0/2166903452 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f7fa40d0640 con 0x7f7fb0100d60 2024-04-21T08:41:02.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.905+0000 7f7fae7fc640 1 -- 172.21.15.86:0/1322958673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562143672230 msgr2=0x562143672610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:02.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.905+0000 7f7fae7fc640 1 --2- 172.21.15.86:0/1322958673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562143672230 0x562143672610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.905+0000 7f7fae7fc640 1 -- 172.21.15.86:0/1322958673 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56214367fe90 msgr2=0x562143682280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:02.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.905+0000 7f7fae7fc640 1 --2- 172.21.15.86:0/1322958673 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56214367fe90 0x562143682280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.905+0000 7f7fae7fc640 1 -- 172.21.15.86:0/1322958673 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621433b37d0 con 0x562143672b50 2024-04-21T08:41:02.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.905+0000 7f7faeffd640 1 --2- 172.21.15.86:0/2166903452 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7f980a3d20 0x7f7f980a61e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0009b50 tx=0x7f7fa0005f90 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:02.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.905+0000 7f7fae7fc640 1 --2- 172.21.15.86:0/1322958673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562143672b50 0x56214367f950 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4009610 tx=0x7f7fa4098640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=687f208f426a49c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:02.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.905+0000 7f7f957fa640 1 -- 172.21.15.86:0/1322958673 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa40cf070 con 0x562143672b50 2024-04-21T08:41:02.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.905+0000 7f7f957fa640 1 -- 172.21.15.86:0/1322958673 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fa409c970 con 0x562143672b50 2024-04-21T08:41:02.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.905+0000 7f7f957fa640 1 -- 172.21.15.86:0/1322958673 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4020070 con 0x562143672b50 2024-04-21T08:41:02.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.906+0000 7f7fb800f740 1 -- 172.21.15.86:0/1322958673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562143672b50 msgr2=0x56214367f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:02.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.906+0000 7f7fb800f740 1 --2- 172.21.15.86:0/1322958673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562143672b50 0x56214367f950 secure :-1 s=READY pgs=884 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4009610 tx=0x7f7fa4098640 comp rx=0 tx=0).stop 2024-04-21T08:41:02.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.906+0000 7f7fb800f740 1 -- 172.21.15.86:0/1322958673 shutdown_connections 2024-04-21T08:41:02.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.906+0000 7f7fb800f740 1 --2- 172.21.15.86:0/1322958673 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56214367fe90 0x562143682280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.906+0000 7f7fb800f740 1 --2- 172.21.15.86:0/1322958673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562143672b50 0x56214367f950 unknown :-1 s=CLOSED pgs=884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.906+0000 7f7fb800f740 1 --2- 172.21.15.86:0/1322958673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562143672230 0x562143672610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.906+0000 7f7fb800f740 1 -- 172.21.15.86:0/1322958673 >> 172.21.15.86:0/1322958673 conn(0x56214366ccb0 msgr2=0x56214366d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:02.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.906+0000 7f7fb800f740 1 -- 172.21.15.86:0/1322958673 shutdown_connections 2024-04-21T08:41:02.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.906+0000 7f7fb800f740 1 -- 172.21.15.86:0/1322958673 wait complete. 2024-04-21T08:41:02.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.906+0000 7f7fb800f740 1 -- start start 2024-04-21T08:41:02.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.906+0000 7f7fb800f740 1 Processor -- start 2024-04-21T08:41:02.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.906+0000 7f7fb800f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562143672230 0x56214367ce60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.906+0000 7f7fb800f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562143672b50 0x56214367d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.906+0000 7f7fb800f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56214367fe90 0x56214367d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.906+0000 7f7fb800f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562143682c10 con 0x562143672b50 2024-04-21T08:41:02.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.906+0000 7f7fb800f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5621435902f0 con 0x56214367fe90 2024-04-21T08:41:02.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.906+0000 7f7fb800f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562143682d90 con 0x562143672230 2024-04-21T08:41:02.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.906+0000 7f7fb800f740 10 client.0 _ll_register_callbacks cb 0x56214359d580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:02.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.907+0000 7f7fae7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562143672b50 0x56214367d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.907+0000 7f7fae7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562143672b50 0x56214367d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60754/0 (socket says 172.21.15.86:60754) 2024-04-21T08:41:02.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.907+0000 7f7fae7fc640 1 -- 172.21.15.86:0/3976426677 learned_addr learned my addr 172.21.15.86:0/3976426677 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:02.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.907+0000 7f7faeffd640 1 --2- 172.21.15.86:0/3976426677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562143672230 0x56214367ce60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.907+0000 7f7faf7fe640 1 --2- 172.21.15.86:0/3976426677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56214367fe90 0x56214367d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.907+0000 7f7fae7fc640 1 -- 172.21.15.86:0/3976426677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562143672230 msgr2=0x56214367ce60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:02.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.907+0000 7f7fae7fc640 1 --2- 172.21.15.86:0/3976426677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562143672230 0x56214367ce60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.907+0000 7f7fae7fc640 1 -- 172.21.15.86:0/3976426677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56214367fe90 msgr2=0x56214367d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:02.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.907+0000 7f7fae7fc640 1 --2- 172.21.15.86:0/3976426677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56214367fe90 0x56214367d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.907+0000 7f7fae7fc640 1 -- 172.21.15.86:0/3976426677 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621432e93f0 con 0x562143672b50 2024-04-21T08:41:02.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.907+0000 7f7fae7fc640 1 --2- 172.21.15.86:0/3976426677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562143672b50 0x56214367d3a0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4098c40 tx=0x7f7fa4098b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2957c05bdacd5c4d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:02.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.907+0000 7f7f92ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:02.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.907+0000 7f7f92ff5640 1 -- 172.21.15.86:0/3976426677 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa40b4020 con 0x562143672b50 2024-04-21T08:41:02.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.907+0000 7f7f92ff5640 1 -- 172.21.15.86:0/3976426677 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f7fa4008230 con 0x562143672b50 2024-04-21T08:41:02.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.907+0000 7f7fb800f740 1 -- 172.21.15.86:0/3976426677 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fa409c970 con 0x562143672b50 2024-04-21T08:41:02.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.907+0000 7f7f92ff5640 1 -- 172.21.15.86:0/3976426677 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7fa40cf070 con 0x562143672b50 2024-04-21T08:41:02.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.907+0000 7f7fb800f740 10 client.16818 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:02.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.907+0000 7f7fb800f740 1 -- 172.21.15.86:0/3976426677 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5621432a90f0 con 0x562143672b50 2024-04-21T08:41:02.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.907+0000 7f7f92ff5640 1 -- 172.21.15.86:0/3976426677 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fa409c970 con 0x562143672b50 2024-04-21T08:41:02.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.908+0000 7f7f92ff5640 1 -- 172.21.15.86:0/3976426677 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7fa40d0270 con 0x562143672b50 2024-04-21T08:41:02.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.908+0000 7f7fb800f740 10 client.16818 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:02.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.908+0000 7f7fb800f740 10 client.16818 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:02.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.908+0000 7f7fb800f740 20 client.16818 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:02.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.908+0000 7f7fb800f740 1 --2- 172.21.15.86:0/3976426677 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56214368fc80 0x562143692070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.908+0000 7f7fb800f740 4 client.16818 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:02.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.908+0000 7f7fb800f740 1 -- 172.21.15.86:0/3976426677 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5621435e4c00 con 0x56214368fc80 2024-04-21T08:41:02.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.908+0000 7f7faeffd640 1 --2- 172.21.15.86:0/3976426677 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56214368fc80 0x562143692070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.908+0000 7f7f92ff5640 10 client.16818 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:02.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.908+0000 7f7faeffd640 1 --2- 172.21.15.86:0/3976426677 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56214368fc80 0x562143692070 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b86cb364ef90fc server_cookie=e00f677ba66bd42a in_seq=0 out_seq=0 2024-04-21T08:41:02.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.909+0000 7f7f92ff5640 1 -- 172.21.15.86:0/3976426677 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5621435e4c00 con 0x56214368fc80 2024-04-21T08:41:02.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.909+0000 7f7f92ff5640 10 client.16818 handle_command_reply: tid=0 2024-04-21T08:41:02.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.909+0000 7f7fb800f740 1 client.16818 shutdown 2024-04-21T08:41:02.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.909+0000 7f7fb800f740 1 -- 172.21.15.86:0/3976426677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562143672b50 msgr2=0x56214367d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:02.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.909+0000 7f7fb800f740 1 --2- 172.21.15.86:0/3976426677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562143672b50 0x56214367d3a0 secure :-1 s=READY pgs=885 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4098c40 tx=0x7f7fa4098b10 comp rx=0 tx=0).stop 2024-04-21T08:41:02.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.909+0000 7f7fb800f740 1 -- 172.21.15.86:0/3976426677 shutdown_connections 2024-04-21T08:41:02.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.909+0000 7f7fb800f740 1 --2- 172.21.15.86:0/3976426677 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56214368fc80 0x562143692070 crc :-1 s=READY pgs=101 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.909+0000 7f7fb800f740 1 --2- 172.21.15.86:0/3976426677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56214367fe90 0x56214367d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.909+0000 7f7fb800f740 1 --2- 172.21.15.86:0/3976426677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562143672b50 0x56214367d3a0 unknown :-1 s=CLOSED pgs=885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.909+0000 7f7f92ff5640 0 client.16818 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:02.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.909+0000 7f7fb800f740 1 --2- 172.21.15.86:0/3976426677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562143672230 0x56214367ce60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.909+0000 7f7fb800f740 1 -- 172.21.15.86:0/3976426677 >> 172.21.15.86:0/3976426677 conn(0x56214366ccb0 msgr2=0x56214366ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:02.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.909+0000 7f7fb800f740 1 -- 172.21.15.86:0/3976426677 shutdown_connections 2024-04-21T08:41:02.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.909+0000 7f7fb800f740 1 -- 172.21.15.86:0/3976426677 wait complete. 2024-04-21T08:41:02.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.909+0000 7f7fb800f740 20 client.16818 trim_cache size 0 max 16384 2024-04-21T08:41:02.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.927+0000 7f7fb800f740 1 Processor -- start 2024-04-21T08:41:02.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.927+0000 7f7fb800f740 1 -- start start 2024-04-21T08:41:02.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.927+0000 7f7fb800f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562143672230 0x56214367f590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.927+0000 7f7fb800f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562143672b50 0x56214366bbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.927+0000 7f7fb800f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56214367fe90 0x562143695090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.927+0000 7f7fb800f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb01ad0e0 con 0x562143672b50 2024-04-21T08:41:02.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.927+0000 7f7fb800f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb01aec20 con 0x562143672230 2024-04-21T08:41:02.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.927+0000 7f7fb800f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb01ad7b0 con 0x56214367fe90 2024-04-21T08:41:02.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.927+0000 7f7fae7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562143672b50 0x56214366bbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.927+0000 7f7fae7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562143672b50 0x56214366bbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60762/0 (socket says 172.21.15.86:60762) 2024-04-21T08:41:02.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.927+0000 7f7fae7fc640 1 -- 172.21.15.86:0/3472271727 learned_addr learned my addr 172.21.15.86:0/3472271727 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:02.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.927+0000 7f7faf7fe640 1 --2- 172.21.15.86:0/3472271727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56214367fe90 0x562143695090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.927+0000 7f7fae7fc640 1 -- 172.21.15.86:0/3472271727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56214367fe90 msgr2=0x562143695090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:02.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.927+0000 7f7fae7fc640 1 --2- 172.21.15.86:0/3472271727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56214367fe90 0x562143695090 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.927+0000 7f7faeffd640 1 --2- 172.21.15.86:0/3472271727 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562143672230 0x56214367f590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.927+0000 7f7fae7fc640 1 -- 172.21.15.86:0/3472271727 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562143672230 msgr2=0x56214367f590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:02.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.927+0000 7f7fae7fc640 1 --2- 172.21.15.86:0/3472271727 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562143672230 0x56214367f590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.927+0000 7f7fae7fc640 1 -- 172.21.15.86:0/3472271727 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5621436718a0 con 0x562143672b50 2024-04-21T08:41:02.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.927+0000 7f7fae7fc640 1 --2- 172.21.15.86:0/3472271727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562143672b50 0x56214366bbf0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4009610 tx=0x7f7fa4098640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d98e37f8a9e0c14 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:02.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.927+0000 7f7f94ff9640 1 -- 172.21.15.86:0/3472271727 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa40b4040 con 0x562143672b50 2024-04-21T08:41:02.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.927+0000 7f7f94ff9640 1 -- 172.21.15.86:0/3472271727 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fa40086b0 con 0x562143672b50 2024-04-21T08:41:02.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.927+0000 7f7f94ff9640 1 -- 172.21.15.86:0/3472271727 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa4020040 con 0x562143672b50 2024-04-21T08:41:02.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.928+0000 7f7fb800f740 1 -- 172.21.15.86:0/3472271727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562143672b50 msgr2=0x56214366bbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:02.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.928+0000 7f7fb800f740 1 --2- 172.21.15.86:0/3472271727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562143672b50 0x56214366bbf0 secure :-1 s=READY pgs=886 cs=0 l=1 rev1=1 crypto rx=0x7f7fa4009610 tx=0x7f7fa4098640 comp rx=0 tx=0).stop 2024-04-21T08:41:02.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.928+0000 7f7fb800f740 1 -- 172.21.15.86:0/3472271727 shutdown_connections 2024-04-21T08:41:02.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.928+0000 7f7fb800f740 1 --2- 172.21.15.86:0/3472271727 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56214367fe90 0x562143695090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.928+0000 7f7fb800f740 1 --2- 172.21.15.86:0/3472271727 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562143672b50 0x56214366bbf0 unknown :-1 s=CLOSED pgs=886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.928+0000 7f7fb800f740 1 --2- 172.21.15.86:0/3472271727 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562143672230 0x56214367f590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.928+0000 7f7fb800f740 1 -- 172.21.15.86:0/3472271727 >> 172.21.15.86:0/3472271727 conn(0x56214366ccb0 msgr2=0x56214366b4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:02.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.928+0000 7f7fb800f740 1 -- 172.21.15.86:0/3472271727 shutdown_connections 2024-04-21T08:41:02.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.928+0000 7f7fb800f740 1 -- 172.21.15.86:0/3472271727 wait complete. 2024-04-21T08:41:02.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.928+0000 7f7fb800f740 1 -- start start 2024-04-21T08:41:02.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.928+0000 7f7fb800f740 1 Processor -- start 2024-04-21T08:41:02.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.929+0000 7f7fb800f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562143672230 0x562143692b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.929+0000 7f7fb800f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562143672b50 0x562143693080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.929+0000 7f7fb800f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56214367fe90 0x5621436935c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.929+0000 7f7fb800f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb01ae550 con 0x56214367fe90 2024-04-21T08:41:02.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.929+0000 7f7fb800f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb01ade80 con 0x562143672230 2024-04-21T08:41:02.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.929+0000 7f7fb800f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7fb01ab0a0 con 0x562143672b50 2024-04-21T08:41:02.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.929+0000 7f7fb800f740 10 client.0 _ll_register_callbacks cb 0x7f7fa0005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:02.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.929+0000 7f7faf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56214367fe90 0x5621436935c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.929+0000 7f7faf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56214367fe90 0x5621436935c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60768/0 (socket says 172.21.15.86:60768) 2024-04-21T08:41:02.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.929+0000 7f7faf7fe640 1 -- 172.21.15.86:0/3783272392 learned_addr learned my addr 172.21.15.86:0/3783272392 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:02.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.929+0000 7f7faeffd640 1 --2- 172.21.15.86:0/3783272392 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562143672230 0x562143692b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.929+0000 7f7fae7fc640 1 --2- 172.21.15.86:0/3783272392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562143672b50 0x562143693080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.929+0000 7f7faf7fe640 1 -- 172.21.15.86:0/3783272392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562143672b50 msgr2=0x562143693080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:02.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.929+0000 7f7faf7fe640 1 --2- 172.21.15.86:0/3783272392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562143672b50 0x562143693080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.929+0000 7f7faf7fe640 1 -- 172.21.15.86:0/3783272392 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562143672230 msgr2=0x562143692b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:02.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.929+0000 7f7faf7fe640 1 --2- 172.21.15.86:0/3783272392 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562143672230 0x562143692b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.929+0000 7f7faf7fe640 1 -- 172.21.15.86:0/3783272392 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5621436f4eb0 con 0x56214367fe90 2024-04-21T08:41:02.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.929+0000 7f7faf7fe640 1 --2- 172.21.15.86:0/3783272392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56214367fe90 0x5621436935c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8036000 tx=0x7f7fa8032fb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=34ba66632046fb00 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:02.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.929+0000 7f7f92ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:02.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.929+0000 7f7f92ff5640 1 -- 172.21.15.86:0/3783272392 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7fa8039040 con 0x56214367fe90 2024-04-21T08:41:02.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.929+0000 7f7f92ff5640 1 -- 172.21.15.86:0/3783272392 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f7fa8003b70 con 0x56214367fe90 2024-04-21T08:41:02.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.929+0000 7f7fb800f740 1 -- 172.21.15.86:0/3783272392 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7fa40086b0 con 0x56214367fe90 2024-04-21T08:41:02.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.930+0000 7f7f92ff5640 1 -- 172.21.15.86:0/3783272392 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7fa8054050 con 0x56214367fe90 2024-04-21T08:41:02.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.930+0000 7f7fb800f740 10 client.16824 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:02.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.930+0000 7f7fb800f740 1 -- 172.21.15.86:0/3783272392 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5621436f61c0 con 0x56214367fe90 2024-04-21T08:41:02.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.930+0000 7f7f92ff5640 1 -- 172.21.15.86:0/3783272392 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7fa40086b0 con 0x56214367fe90 2024-04-21T08:41:02.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.930+0000 7f7f92ff5640 1 -- 172.21.15.86:0/3783272392 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7fa803f040 con 0x56214367fe90 2024-04-21T08:41:02.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.930+0000 7f7fb800f740 10 client.16824 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:02.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.930+0000 7f7fb800f740 10 client.16824 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:02.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.930+0000 7f7fb800f740 20 client.16824 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:02.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.930+0000 7f7fb800f740 1 --2- 172.21.15.86:0/3783272392 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5621436f6970 0x5621436f8d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:02.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.930+0000 7f7fb800f740 4 client.16824 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:41:02.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.930+0000 7f7fb800f740 1 -- 172.21.15.86:0/3783272392 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5621435dea70 con 0x5621436f6970 2024-04-21T08:41:02.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.930+0000 7f7faeffd640 1 --2- 172.21.15.86:0/3783272392 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5621436f6970 0x5621436f8d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:02.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.931+0000 7f7f92ff5640 10 client.16824 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:02.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.931+0000 7f7faeffd640 1 --2- 172.21.15.86:0/3783272392 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5621436f6970 0x5621436f8d90 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dac703ed948d1c0d server_cookie=8d1d7a8ce03df00d in_seq=0 out_seq=0 2024-04-21T08:41:02.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.931+0000 7f7f92ff5640 1 -- 172.21.15.86:0/3783272392 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5621435dea70 con 0x5621436f6970 2024-04-21T08:41:02.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.931+0000 7f7f92ff5640 10 client.16824 handle_command_reply: tid=0 2024-04-21T08:41:02.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.931+0000 7f7fb800f740 1 client.16824 shutdown 2024-04-21T08:41:02.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.931+0000 7f7fb800f740 1 -- 172.21.15.86:0/3783272392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56214367fe90 msgr2=0x5621436935c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:02.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.931+0000 7f7fb800f740 1 --2- 172.21.15.86:0/3783272392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56214367fe90 0x5621436935c0 secure :-1 s=READY pgs=887 cs=0 l=1 rev1=1 crypto rx=0x7f7fa8036000 tx=0x7f7fa8032fb0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.931+0000 7f7fb800f740 1 -- 172.21.15.86:0/3783272392 shutdown_connections 2024-04-21T08:41:02.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.931+0000 7f7fb800f740 1 --2- 172.21.15.86:0/3783272392 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5621436f6970 0x5621436f8d90 crc :-1 s=READY pgs=102 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.931+0000 7f7fb800f740 1 --2- 172.21.15.86:0/3783272392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56214367fe90 0x5621436935c0 unknown :-1 s=CLOSED pgs=887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.931+0000 7f7f92ff5640 0 client.16824 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:02.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.931+0000 7f7fb800f740 1 --2- 172.21.15.86:0/3783272392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562143672b50 0x562143693080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.931+0000 7f7fb800f740 1 --2- 172.21.15.86:0/3783272392 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562143672230 0x562143692b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.931+0000 7f7fb800f740 1 -- 172.21.15.86:0/3783272392 >> 172.21.15.86:0/3783272392 conn(0x56214366ccb0 msgr2=0x56214366bd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:02.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.931+0000 7f7fb800f740 1 -- 172.21.15.86:0/3783272392 shutdown_connections 2024-04-21T08:41:02.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.931+0000 7f7fb800f740 1 -- 172.21.15.86:0/3783272392 wait complete. 2024-04-21T08:41:02.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.931+0000 7f7fb800f740 20 client.16824 trim_cache size 0 max 16384 2024-04-21T08:41:02.941 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:41:02.941 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:41:02.942 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-21T08:41:02.942 INFO:teuthology.orchestra.run.smithi086.stdout: "e0bda25e-ed53-4040-ab17-597d95a75b62": { 2024-04-21T08:41:02.942 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-21T08:41:02.942 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "e0bda25e-ed53-4040-ab17-597d95a75b62", 2024-04-21T08:41:02.942 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-21T08:41:02.942 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:41:02.942 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:41:02.942 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:41:02.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.931+0000 7f7fb534a640 1 -- 172.21.15.86:0/2166903452 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7f980a3d20 msgr2=0x7f7f980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:02.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.931+0000 7f7fb534a640 1 --2- 172.21.15.86:0/2166903452 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7f980a3d20 0x7f7f980a61e0 secure :-1 s=READY pgs=129 cs=0 l=1 rev1=1 crypto rx=0x7f7fa0009b50 tx=0x7f7fa0005f90 comp rx=0 tx=0).stop 2024-04-21T08:41:02.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.931+0000 7f7fb534a640 1 -- 172.21.15.86:0/2166903452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fb0100d60 msgr2=0x7f7fb0198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:02.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.931+0000 7f7fb534a640 1 --2- 172.21.15.86:0/2166903452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fb0100d60 0x7f7fb0198c70 secure :-1 s=READY pgs=883 cs=0 l=1 rev1=1 crypto rx=0x7f7fa400d770 tx=0x7f7fa400dc30 comp rx=0 tx=0).stop 2024-04-21T08:41:02.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.931+0000 7f7fb534a640 1 -- 172.21.15.86:0/2166903452 shutdown_connections 2024-04-21T08:41:02.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.931+0000 7f7fb534a640 1 --2- 172.21.15.86:0/2166903452 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7f980a3d20 0x7f7f980a61e0 unknown :-1 s=CLOSED pgs=129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.931+0000 7f7fb534a640 1 --2- 172.21.15.86:0/2166903452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7fb0107fd0 0x7f7fb019cff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.931+0000 7f7fb534a640 1 --2- 172.21.15.86:0/2166903452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7fb0100d60 0x7f7fb0198c70 unknown :-1 s=CLOSED pgs=883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.931+0000 7f7fb534a640 1 --2- 172.21.15.86:0/2166903452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7fb01003e0 0x7f7fb0198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:02.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.931+0000 7f7fb534a640 1 -- 172.21.15.86:0/2166903452 >> 172.21.15.86:0/2166903452 conn(0x7f7fb0077360 msgr2=0x7f7fb00feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:02.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.932+0000 7f7fb534a640 1 -- 172.21.15.86:0/2166903452 shutdown_connections 2024-04-21T08:41:02.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:02.932+0000 7f7fb534a640 1 -- 172.21.15.86:0/2166903452 wait complete. 2024-04-21T08:41:02.944 INFO:tasks.cephfs.filesystem:scrub status for tag:e0bda25e-ed53-4040-ab17-597d95a75b62 - {'path': '/', 'tag': 'e0bda25e-ed53-4040-ab17-597d95a75b62', 'options': 'recursive,force'} 2024-04-21T08:41:04.516 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:04 smithi088 ceph-mon[31259]: pgmap v71: 129 pgs: 129 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 53 KiB/s wr, 37 op/s 2024-04-21T08:41:04.516 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:04 smithi088 ceph-mon[31259]: scrub queued for path: / 2024-04-21T08:41:04.516 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:04 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:41:04.516 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:04 smithi088 ceph-mon[31259]: scrub summary: active paths [/] 2024-04-21T08:41:04.517 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:04 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:41:04.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:04 smithi086 ceph-mon[27939]: pgmap v71: 129 pgs: 129 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 53 KiB/s wr, 37 op/s 2024-04-21T08:41:04.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:04 smithi086 ceph-mon[27939]: scrub queued for path: / 2024-04-21T08:41:04.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:04 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:41:04.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:04 smithi086 ceph-mon[27939]: scrub summary: active paths [/] 2024-04-21T08:41:04.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:04 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:41:04.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:04 smithi179 ceph-mon[31829]: pgmap v71: 129 pgs: 129 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 53 KiB/s wr, 37 op/s 2024-04-21T08:41:04.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:04 smithi179 ceph-mon[31829]: scrub queued for path: / 2024-04-21T08:41:04.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:04 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:41:04.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:04 smithi179 ceph-mon[31829]: scrub summary: active paths [/] 2024-04-21T08:41:04.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:04 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:41:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.322+0000 7f429a58f640 1 -- 172.21.15.86:0/1667951433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4294106170 msgr2=0x7f429410dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.322+0000 7f429a58f640 1 --2- 172.21.15.86:0/1667951433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4294106170 0x7f429410dd40 secure :-1 s=READY pgs=888 cs=0 l=1 rev1=1 crypto rx=0x7f428c009ca0 tx=0x7f428c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:41:05.323 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.322+0000 7f429a58f640 1 -- 172.21.15.86:0/1667951433 shutdown_connections 2024-04-21T08:41:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.322+0000 7f429a58f640 1 --2- 172.21.15.86:0/1667951433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4294106170 0x7f429410dd40 unknown :-1 s=CLOSED pgs=888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.322+0000 7f429a58f640 1 --2- 172.21.15.86:0/1667951433 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4294101aa0 0x7f4294105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.322+0000 7f429a58f640 1 --2- 172.21.15.86:0/1667951433 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42941010f0 0x7f42941014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.322+0000 7f429a58f640 1 -- 172.21.15.86:0/1667951433 >> 172.21.15.86:0/1667951433 conn(0x7f4294077fd0 msgr2=0x7f42940783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.322+0000 7f429a58f640 1 -- 172.21.15.86:0/1667951433 shutdown_connections 2024-04-21T08:41:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.322+0000 7f429a58f640 1 -- 172.21.15.86:0/1667951433 wait complete. 2024-04-21T08:41:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.322+0000 7f429a58f640 1 Processor -- start 2024-04-21T08:41:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.322+0000 7f429a58f640 1 -- start start 2024-04-21T08:41:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.323+0000 7f429a58f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42941010f0 0x7f42941975a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.323+0000 7f429a58f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4294106170 0x7f4294197ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.323+0000 7f429a58f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f429419be10 0x7f429419c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.323+0000 7f429a58f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f42941110e0 con 0x7f4294106170 2024-04-21T08:41:05.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.323+0000 7f429a58f640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4294110f60 con 0x7f42941010f0 2024-04-21T08:41:05.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.323+0000 7f429a58f640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4294111260 con 0x7f429419be10 2024-04-21T08:41:05.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.323+0000 7f42937fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4294106170 0x7f4294197ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:05.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.323+0000 7f42937fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4294106170 0x7f4294197ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60796/0 (socket says 172.21.15.86:60796) 2024-04-21T08:41:05.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.323+0000 7f42937fe640 1 -- 172.21.15.86:0/1392417930 learned_addr learned my addr 172.21.15.86:0/1392417930 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:05.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.323+0000 7f4293fff640 1 --2- 172.21.15.86:0/1392417930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42941010f0 0x7f42941975a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:05.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.323+0000 7f4298b2c640 1 --2- 172.21.15.86:0/1392417930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f429419be10 0x7f429419c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:05.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.323+0000 7f42937fe640 1 -- 172.21.15.86:0/1392417930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f429419be10 msgr2=0x7f429419c2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:05.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.323+0000 7f42937fe640 1 --2- 172.21.15.86:0/1392417930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f429419be10 0x7f429419c2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:05.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.323+0000 7f42937fe640 1 -- 172.21.15.86:0/1392417930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42941010f0 msgr2=0x7f42941975a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:05.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.323+0000 7f42937fe640 1 --2- 172.21.15.86:0/1392417930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42941010f0 0x7f42941975a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:05.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.323+0000 7f42937fe640 1 -- 172.21.15.86:0/1392417930 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f429419c900 con 0x7f4294106170 2024-04-21T08:41:05.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.323+0000 7f4298b2c640 1 --2- 172.21.15.86:0/1392417930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f429419be10 0x7f429419c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:05.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.323+0000 7f4293fff640 1 --2- 172.21.15.86:0/1392417930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42941010f0 0x7f42941975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:41:05.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.323+0000 7f42937fe640 1 --2- 172.21.15.86:0/1392417930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4294106170 0x7f4294197ae0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f428800c970 tx=0x7f428800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:05.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.324+0000 7f42917fa640 1 -- 172.21.15.86:0/1392417930 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4288007bb0 con 0x7f4294106170 2024-04-21T08:41:05.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.324+0000 7f42917fa640 1 -- 172.21.15.86:0/1392417930 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4288007d50 con 0x7f4294106170 2024-04-21T08:41:05.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.324+0000 7f429a58f640 1 -- 172.21.15.86:0/1392417930 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f429419cbf0 con 0x7f4294106170 2024-04-21T08:41:05.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.324+0000 7f42917fa640 1 -- 172.21.15.86:0/1392417930 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4288018730 con 0x7f4294106170 2024-04-21T08:41:05.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.324+0000 7f429a58f640 1 -- 172.21.15.86:0/1392417930 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42941a4390 con 0x7f4294106170 2024-04-21T08:41:05.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.326+0000 7f42917fa640 1 -- 172.21.15.86:0/1392417930 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f428800f990 con 0x7f4294106170 2024-04-21T08:41:05.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.326+0000 7f429d17c740 1 Processor -- start 2024-04-21T08:41:05.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.327+0000 7f429d17c740 1 -- start start 2024-04-21T08:41:05.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.327+0000 7f42917fa640 1 --2- 172.21.15.86:0/1392417930 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f427c0a3e40 0x7f427c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:05.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.327+0000 7f429d17c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639c0592ae0 0x5639c0591830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:05.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.327+0000 7f4298b2c640 1 --2- 172.21.15.86:0/1392417930 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f427c0a3e40 0x7f427c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:05.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.327+0000 7f429d17c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639c0591d70 0x5639c05921f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:05.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.327+0000 7f429d17c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639c059b2f0 0x5639c059d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:05.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.327+0000 7f429d17c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5639c04f48d0 con 0x5639c059b2f0 2024-04-21T08:41:05.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.327+0000 7f429d17c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5639c0491e80 con 0x5639c0591d70 2024-04-21T08:41:05.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.327+0000 7f429d17c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5639c04f0890 con 0x5639c0592ae0 2024-04-21T08:41:05.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.327+0000 7f42917fa640 1 -- 172.21.15.86:0/1392417930 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f42880d0ba0 con 0x7f4294106170 2024-04-21T08:41:05.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.327+0000 7f4298b2c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639c059b2f0 0x5639c059d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:05.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.327+0000 7f4298b2c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639c059b2f0 0x5639c059d6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60804/0 (socket says 172.21.15.86:60804) 2024-04-21T08:41:05.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.327+0000 7f4298b2c640 1 -- 172.21.15.86:0/1034590005 learned_addr learned my addr 172.21.15.86:0/1034590005 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:05.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.327+0000 7f4293fff640 1 --2- 172.21.15.86:0/1034590005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639c0592ae0 0x5639c0591830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:05.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.327+0000 7f42937fe640 1 --2- 172.21.15.86:0/1034590005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639c0591d70 0x5639c05921f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:05.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.327+0000 7f4298b2c640 1 -- 172.21.15.86:0/1034590005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639c0592ae0 msgr2=0x5639c0591830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:05.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.327+0000 7f4298b2c640 1 --2- 172.21.15.86:0/1034590005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639c0592ae0 0x5639c0591830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:05.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.327+0000 7f4298b2c640 1 -- 172.21.15.86:0/1034590005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639c0591d70 msgr2=0x5639c05921f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:05.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.327+0000 7f4298b2c640 1 --2- 172.21.15.86:0/1034590005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639c0591d70 0x5639c05921f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:05.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.327+0000 7f4298b2c640 1 -- 172.21.15.86:0/1034590005 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639c04b3320 con 0x5639c059b2f0 2024-04-21T08:41:05.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.327+0000 7f4298b2c640 1 --2- 172.21.15.86:0/1392417930 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f427c0a3e40 0x7f427c0a6300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f428c037040 tx=0x7f428c02cca0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:05.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.327+0000 7f4298b2c640 1 --2- 172.21.15.86:0/1034590005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639c059b2f0 0x5639c059d6e0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f428c005cc0 tx=0x7f428c046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11a402270682e805 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:05.334 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.328+0000 7f427a7fc640 1 -- 172.21.15.86:0/1034590005 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428c053070 con 0x5639c059b2f0 2024-04-21T08:41:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.328+0000 7f427a7fc640 1 -- 172.21.15.86:0/1034590005 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f428c03a050 con 0x5639c059b2f0 2024-04-21T08:41:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.328+0000 7f427a7fc640 1 -- 172.21.15.86:0/1034590005 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428c004610 con 0x5639c059b2f0 2024-04-21T08:41:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.328+0000 7f429d17c740 1 -- 172.21.15.86:0/1034590005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639c059b2f0 msgr2=0x5639c059d6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.328+0000 7f429d17c740 1 --2- 172.21.15.86:0/1034590005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639c059b2f0 0x5639c059d6e0 secure :-1 s=READY pgs=890 cs=0 l=1 rev1=1 crypto rx=0x7f428c005cc0 tx=0x7f428c046000 comp rx=0 tx=0).stop 2024-04-21T08:41:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.328+0000 7f429d17c740 1 -- 172.21.15.86:0/1034590005 shutdown_connections 2024-04-21T08:41:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.328+0000 7f429d17c740 1 --2- 172.21.15.86:0/1034590005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639c059b2f0 0x5639c059d6e0 unknown :-1 s=CLOSED pgs=890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.328+0000 7f429d17c740 1 --2- 172.21.15.86:0/1034590005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639c0591d70 0x5639c05921f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.328+0000 7f429d17c740 1 --2- 172.21.15.86:0/1034590005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639c0592ae0 0x5639c0591830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.328+0000 7f429d17c740 1 -- 172.21.15.86:0/1034590005 >> 172.21.15.86:0/1034590005 conn(0x5639c0556c90 msgr2=0x5639c058ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.328+0000 7f429d17c740 1 -- 172.21.15.86:0/1034590005 shutdown_connections 2024-04-21T08:41:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.328+0000 7f429d17c740 1 -- 172.21.15.86:0/1034590005 wait complete. 2024-04-21T08:41:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.329+0000 7f429d17c740 1 -- start start 2024-04-21T08:41:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.329+0000 7f429d17c740 1 Processor -- start 2024-04-21T08:41:05.335 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.329+0000 7f429d17c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639c0591d70 0x5639c059ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.329+0000 7f429d17c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639c0592ae0 0x5639c0598010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.329+0000 7f429d17c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639c059b2f0 0x5639c0598590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.329+0000 7f429d17c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5639c059dfc0 con 0x5639c059b2f0 2024-04-21T08:41:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.329+0000 7f429d17c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5639c047ceb0 con 0x5639c0591d70 2024-04-21T08:41:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.329+0000 7f429d17c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5639c059e140 con 0x5639c0592ae0 2024-04-21T08:41:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.329+0000 7f429d17c740 10 client.0 _ll_register_callbacks cb 0x5639c04a9b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.329+0000 7f4298b2c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639c059b2f0 0x5639c0598590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.329+0000 7f4293fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639c0591d70 0x5639c059ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.329+0000 7f42937fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639c0592ae0 0x5639c0598010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:05.336 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.329+0000 7f4298b2c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639c059b2f0 0x5639c0598590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60820/0 (socket says 172.21.15.86:60820) 2024-04-21T08:41:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.329+0000 7f4298b2c640 1 -- 172.21.15.86:0/2673438582 learned_addr learned my addr 172.21.15.86:0/2673438582 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.330+0000 7f4298b2c640 1 -- 172.21.15.86:0/2673438582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639c0592ae0 msgr2=0x5639c0598010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.330+0000 7f4298b2c640 1 --2- 172.21.15.86:0/2673438582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639c0592ae0 0x5639c0598010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.330+0000 7f4298b2c640 1 -- 172.21.15.86:0/2673438582 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639c0591d70 msgr2=0x5639c059ac80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.330+0000 7f4298b2c640 1 --2- 172.21.15.86:0/2673438582 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639c0591d70 0x5639c059ac80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.330+0000 7f4298b2c640 1 -- 172.21.15.86:0/2673438582 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639c0207210 con 0x5639c059b2f0 2024-04-21T08:41:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.330+0000 7f4298b2c640 1 --2- 172.21.15.86:0/2673438582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639c059b2f0 0x5639c0598590 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f428c004af0 tx=0x7f428c02c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94c12426d556cc67 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.330+0000 7f42737fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.330+0000 7f42737fe640 1 -- 172.21.15.86:0/2673438582 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428c05c060 con 0x5639c059b2f0 2024-04-21T08:41:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.330+0000 7f42737fe640 1 -- 172.21.15.86:0/2673438582 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f428c004d40 con 0x5639c059b2f0 2024-04-21T08:41:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.330+0000 7f429d17c740 1 -- 172.21.15.86:0/2673438582 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f428c03a050 con 0x5639c059b2f0 2024-04-21T08:41:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.330+0000 7f42737fe640 1 -- 172.21.15.86:0/2673438582 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f428c063050 con 0x5639c059b2f0 2024-04-21T08:41:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.330+0000 7f429d17c740 10 client.16836 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:05.337 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.330+0000 7f429d17c740 1 -- 172.21.15.86:0/2673438582 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5639c02d3620 con 0x5639c059b2f0 2024-04-21T08:41:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.330+0000 7f42737fe640 1 -- 172.21.15.86:0/2673438582 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f428c03a050 con 0x5639c059b2f0 2024-04-21T08:41:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.331+0000 7f42737fe640 1 -- 172.21.15.86:0/2673438582 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f428c034560 con 0x5639c059b2f0 2024-04-21T08:41:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.331+0000 7f429d17c740 10 client.16836 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.331+0000 7f429d17c740 10 client.16836 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.331+0000 7f429d17c740 20 client.16836 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.331+0000 7f429d17c740 1 --2- 172.21.15.86:0/2673438582 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5639c05ab110 0x5639c05ad500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.331+0000 7f429d17c740 4 client.16836 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.331+0000 7f429d17c740 1 -- 172.21.15.86:0/2673438582 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639c0508880 con 0x5639c05ab110 2024-04-21T08:41:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.331+0000 7f4293fff640 1 --2- 172.21.15.86:0/2673438582 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5639c05ab110 0x5639c05ad500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.332+0000 7f4293fff640 1 --2- 172.21.15.86:0/2673438582 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5639c05ab110 0x5639c05ad500 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=98dff0154898cf12 server_cookie=6442151acbe8583e in_seq=0 out_seq=0 2024-04-21T08:41:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.332+0000 7f42737fe640 10 client.16836 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.332+0000 7f42737fe640 1 -- 172.21.15.86:0/2673438582 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5639c0508880 con 0x5639c05ab110 2024-04-21T08:41:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.332+0000 7f42737fe640 10 client.16836 handle_command_reply: tid=0 2024-04-21T08:41:05.338 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.332+0000 7f429d17c740 1 client.16836 shutdown 2024-04-21T08:41:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.332+0000 7f429d17c740 1 -- 172.21.15.86:0/2673438582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639c059b2f0 msgr2=0x5639c0598590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.332+0000 7f429d17c740 1 --2- 172.21.15.86:0/2673438582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639c059b2f0 0x5639c0598590 secure :-1 s=READY pgs=891 cs=0 l=1 rev1=1 crypto rx=0x7f428c004af0 tx=0x7f428c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:41:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.332+0000 7f429d17c740 1 -- 172.21.15.86:0/2673438582 shutdown_connections 2024-04-21T08:41:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.332+0000 7f429d17c740 1 --2- 172.21.15.86:0/2673438582 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5639c05ab110 0x5639c05ad500 crc :-1 s=READY pgs=103 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.332+0000 7f429d17c740 1 --2- 172.21.15.86:0/2673438582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639c059b2f0 0x5639c0598590 unknown :-1 s=CLOSED pgs=891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.332+0000 7f429d17c740 1 --2- 172.21.15.86:0/2673438582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639c0592ae0 0x5639c0598010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.332+0000 7f42737fe640 0 client.16836 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.332+0000 7f429d17c740 1 --2- 172.21.15.86:0/2673438582 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639c0591d70 0x5639c059ac80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.332+0000 7f429d17c740 1 -- 172.21.15.86:0/2673438582 >> 172.21.15.86:0/2673438582 conn(0x5639c0556c90 msgr2=0x5639c058e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.333+0000 7f429d17c740 1 -- 172.21.15.86:0/2673438582 shutdown_connections 2024-04-21T08:41:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.333+0000 7f429d17c740 1 -- 172.21.15.86:0/2673438582 wait complete. 2024-04-21T08:41:05.339 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.333+0000 7f429d17c740 20 client.16836 trim_cache size 0 max 16384 2024-04-21T08:41:05.351 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.347+0000 7f429d17c740 1 Processor -- start 2024-04-21T08:41:05.351 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.347+0000 7f429d17c740 1 -- start start 2024-04-21T08:41:05.351 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.348+0000 7f429d17c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639c0591d70 0x5639c05937e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:05.351 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.348+0000 7f429d17c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639c0592ae0 0x5639c059a740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:05.351 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.348+0000 7f429d17c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639c059b2f0 0x5639c059ac80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:05.351 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.348+0000 7f429d17c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f42941acff0 con 0x5639c059b2f0 2024-04-21T08:41:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.348+0000 7f429d17c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f42941aeb30 con 0x5639c0591d70 2024-04-21T08:41:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.348+0000 7f429d17c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f42941ad6c0 con 0x5639c0592ae0 2024-04-21T08:41:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.348+0000 7f4298b2c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639c059b2f0 0x5639c059ac80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.348+0000 7f4298b2c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639c059b2f0 0x5639c059ac80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60824/0 (socket says 172.21.15.86:60824) 2024-04-21T08:41:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.348+0000 7f4298b2c640 1 -- 172.21.15.86:0/1785644476 learned_addr learned my addr 172.21.15.86:0/1785644476 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.348+0000 7f4293fff640 1 --2- 172.21.15.86:0/1785644476 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639c0591d70 0x5639c05937e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.348+0000 7f4298b2c640 1 -- 172.21.15.86:0/1785644476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639c0592ae0 msgr2=0x5639c059a740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.348+0000 7f4298b2c640 1 --2- 172.21.15.86:0/1785644476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639c0592ae0 0x5639c059a740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.348+0000 7f4298b2c640 1 -- 172.21.15.86:0/1785644476 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639c0591d70 msgr2=0x5639c05937e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.348+0000 7f4298b2c640 1 --2- 172.21.15.86:0/1785644476 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639c0591d70 0x5639c05937e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.348+0000 7f4298b2c640 1 -- 172.21.15.86:0/1785644476 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639c05965a0 con 0x5639c059b2f0 2024-04-21T08:41:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.348+0000 7f4298b2c640 1 --2- 172.21.15.86:0/1785644476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639c059b2f0 0x5639c059ac80 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f428c009ca0 tx=0x7f428c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8f41a96802ad8e8d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.348+0000 7f4279ffb640 1 -- 172.21.15.86:0/1785644476 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428c05c070 con 0x5639c059b2f0 2024-04-21T08:41:05.352 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.348+0000 7f4279ffb640 1 -- 172.21.15.86:0/1785644476 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f428c004b40 con 0x5639c059b2f0 2024-04-21T08:41:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.348+0000 7f4279ffb640 1 -- 172.21.15.86:0/1785644476 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f428c053040 con 0x5639c059b2f0 2024-04-21T08:41:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.349+0000 7f429d17c740 1 -- 172.21.15.86:0/1785644476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639c059b2f0 msgr2=0x5639c059ac80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.349+0000 7f429d17c740 1 --2- 172.21.15.86:0/1785644476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639c059b2f0 0x5639c059ac80 secure :-1 s=READY pgs=892 cs=0 l=1 rev1=1 crypto rx=0x7f428c009ca0 tx=0x7f428c004a40 comp rx=0 tx=0).stop 2024-04-21T08:41:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.349+0000 7f429d17c740 1 -- 172.21.15.86:0/1785644476 shutdown_connections 2024-04-21T08:41:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.349+0000 7f429d17c740 1 --2- 172.21.15.86:0/1785644476 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639c059b2f0 0x5639c059ac80 unknown :-1 s=CLOSED pgs=892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.349+0000 7f429d17c740 1 --2- 172.21.15.86:0/1785644476 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639c0592ae0 0x5639c059a740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.349+0000 7f429d17c740 1 --2- 172.21.15.86:0/1785644476 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639c0591d70 0x5639c05937e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.349+0000 7f429d17c740 1 -- 172.21.15.86:0/1785644476 >> 172.21.15.86:0/1785644476 conn(0x5639c0556c90 msgr2=0x5639c05930d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.349+0000 7f429d17c740 1 -- 172.21.15.86:0/1785644476 shutdown_connections 2024-04-21T08:41:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.349+0000 7f429d17c740 1 -- 172.21.15.86:0/1785644476 wait complete. 2024-04-21T08:41:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.349+0000 7f429d17c740 1 -- start start 2024-04-21T08:41:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.349+0000 7f429d17c740 1 Processor -- start 2024-04-21T08:41:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.350+0000 7f429d17c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639c0591d70 0x5639c05adfc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.350+0000 7f429d17c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639c0592ae0 0x5639c05ae500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:05.353 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.350+0000 7f429d17c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639c059b2f0 0x5639c05aea40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.350+0000 7f429d17c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f42941ae460 con 0x5639c0592ae0 2024-04-21T08:41:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.350+0000 7f429d17c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f42941add90 con 0x5639c0591d70 2024-04-21T08:41:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.350+0000 7f429d17c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f42941aafb0 con 0x5639c059b2f0 2024-04-21T08:41:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.350+0000 7f429d17c740 10 client.0 _ll_register_callbacks cb 0x7f428c051070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.350+0000 7f42937fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639c0592ae0 0x5639c05ae500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.350+0000 7f42937fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639c0592ae0 0x5639c05ae500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60836/0 (socket says 172.21.15.86:60836) 2024-04-21T08:41:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.350+0000 7f42937fe640 1 -- 172.21.15.86:0/2319560104 learned_addr learned my addr 172.21.15.86:0/2319560104 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.350+0000 7f4293fff640 1 --2- 172.21.15.86:0/2319560104 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639c0591d70 0x5639c05adfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.350+0000 7f4298b2c640 1 --2- 172.21.15.86:0/2319560104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639c059b2f0 0x5639c05aea40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.350+0000 7f42937fe640 1 -- 172.21.15.86:0/2319560104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639c059b2f0 msgr2=0x5639c05aea40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.350+0000 7f42937fe640 1 --2- 172.21.15.86:0/2319560104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639c059b2f0 0x5639c05aea40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.350+0000 7f42937fe640 1 -- 172.21.15.86:0/2319560104 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639c0591d70 msgr2=0x5639c05adfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:05.354 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.350+0000 7f42937fe640 1 --2- 172.21.15.86:0/2319560104 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639c0591d70 0x5639c05adfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.350+0000 7f42937fe640 1 -- 172.21.15.86:0/2319560104 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639c0610340 con 0x5639c0592ae0 2024-04-21T08:41:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.350+0000 7f4293fff640 1 --2- 172.21.15.86:0/2319560104 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639c0591d70 0x5639c05adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.350+0000 7f4298b2c640 1 --2- 172.21.15.86:0/2319560104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639c059b2f0 0x5639c05aea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.350+0000 7f42937fe640 1 --2- 172.21.15.86:0/2319560104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639c0592ae0 0x5639c05ae500 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f428800c940 tx=0x7f4288098b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d142bc62a3b4431c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.350+0000 7f42737fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.350+0000 7f42737fe640 1 -- 172.21.15.86:0/2319560104 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42880ce3c0 con 0x5639c0592ae0 2024-04-21T08:41:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.350+0000 7f42737fe640 1 -- 172.21.15.86:0/2319560104 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f42880bb680 con 0x5639c0592ae0 2024-04-21T08:41:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.350+0000 7f429d17c740 1 -- 172.21.15.86:0/2319560104 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f428c004b40 con 0x5639c0592ae0 2024-04-21T08:41:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.351+0000 7f42737fe640 1 -- 172.21.15.86:0/2319560104 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f42880a0b40 con 0x5639c0592ae0 2024-04-21T08:41:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.351+0000 7f429d17c740 10 client.16842 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.351+0000 7f429d17c740 1 -- 172.21.15.86:0/2319560104 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5639c06115f0 con 0x5639c0592ae0 2024-04-21T08:41:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.351+0000 7f42737fe640 1 -- 172.21.15.86:0/2319560104 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f428c004b40 con 0x5639c0592ae0 2024-04-21T08:41:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.351+0000 7f42737fe640 1 -- 172.21.15.86:0/2319560104 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4288020030 con 0x5639c0592ae0 2024-04-21T08:41:05.355 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.351+0000 7f429d17c740 10 client.16842 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:05.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.351+0000 7f429d17c740 10 client.16842 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:05.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.351+0000 7f429d17c740 20 client.16842 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:05.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.351+0000 7f429d17c740 1 --2- 172.21.15.86:0/2319560104 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5639c0611da0 0x5639c06141c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:05.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.351+0000 7f429d17c740 4 client.16842 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:41:05.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.351+0000 7f429d17c740 1 -- 172.21.15.86:0/2319560104 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5639c04943d0 con 0x5639c0611da0 2024-04-21T08:41:05.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.351+0000 7f4293fff640 1 --2- 172.21.15.86:0/2319560104 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5639c0611da0 0x5639c06141c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:05.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.351+0000 7f4293fff640 1 --2- 172.21.15.86:0/2319560104 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5639c0611da0 0x5639c06141c0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=85a3dd897bf8b7 server_cookie=a5afcf435be3e982 in_seq=0 out_seq=0 2024-04-21T08:41:05.356 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.351+0000 7f42737fe640 10 client.16842 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:05.366 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.366+0000 7f42737fe640 1 -- 172.21.15.86:0/2319560104 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5639c04943d0 con 0x5639c0611da0 2024-04-21T08:41:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.366+0000 7f42737fe640 10 client.16842 handle_command_reply: tid=0 2024-04-21T08:41:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.366+0000 7f429d17c740 1 client.16842 shutdown 2024-04-21T08:41:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.367+0000 7f429d17c740 1 -- 172.21.15.86:0/2319560104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639c0592ae0 msgr2=0x5639c05ae500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.367+0000 7f429d17c740 1 --2- 172.21.15.86:0/2319560104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639c0592ae0 0x5639c05ae500 secure :-1 s=READY pgs=893 cs=0 l=1 rev1=1 crypto rx=0x7f428800c940 tx=0x7f4288098b30 comp rx=0 tx=0).stop 2024-04-21T08:41:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.367+0000 7f429d17c740 1 -- 172.21.15.86:0/2319560104 shutdown_connections 2024-04-21T08:41:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.367+0000 7f429d17c740 1 --2- 172.21.15.86:0/2319560104 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5639c0611da0 0x5639c06141c0 crc :-1 s=READY pgs=104 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.367+0000 7f429d17c740 1 --2- 172.21.15.86:0/2319560104 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639c059b2f0 0x5639c05aea40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.367+0000 7f429d17c740 1 --2- 172.21.15.86:0/2319560104 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639c0592ae0 0x5639c05ae500 unknown :-1 s=CLOSED pgs=893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.367+0000 7f42737fe640 0 client.16842 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.367+0000 7f429d17c740 1 --2- 172.21.15.86:0/2319560104 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639c0591d70 0x5639c05adfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.367+0000 7f429d17c740 1 -- 172.21.15.86:0/2319560104 >> 172.21.15.86:0/2319560104 conn(0x5639c0556c90 msgr2=0x5639c05a63d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.367+0000 7f429d17c740 1 -- 172.21.15.86:0/2319560104 shutdown_connections 2024-04-21T08:41:05.367 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.367+0000 7f429d17c740 1 -- 172.21.15.86:0/2319560104 wait complete. 2024-04-21T08:41:05.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.367+0000 7f429d17c740 20 client.16842 trim_cache size 0 max 16384 2024-04-21T08:41:05.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:41:05.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:41:05.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:41:05.368 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:41:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.367+0000 7f429a58f640 1 -- 172.21.15.86:0/1392417930 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f427c0a3e40 msgr2=0x7f427c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.367+0000 7f429a58f640 1 --2- 172.21.15.86:0/1392417930 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f427c0a3e40 0x7f427c0a6300 secure :-1 s=READY pgs=130 cs=0 l=1 rev1=1 crypto rx=0x7f428c037040 tx=0x7f428c02cca0 comp rx=0 tx=0).stop 2024-04-21T08:41:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.367+0000 7f429a58f640 1 -- 172.21.15.86:0/1392417930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4294106170 msgr2=0x7f4294197ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.367+0000 7f429a58f640 1 --2- 172.21.15.86:0/1392417930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4294106170 0x7f4294197ae0 secure :-1 s=READY pgs=889 cs=0 l=1 rev1=1 crypto rx=0x7f428800c970 tx=0x7f428800ce30 comp rx=0 tx=0).stop 2024-04-21T08:41:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.367+0000 7f429a58f640 1 -- 172.21.15.86:0/1392417930 shutdown_connections 2024-04-21T08:41:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.367+0000 7f429a58f640 1 --2- 172.21.15.86:0/1392417930 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f427c0a3e40 0x7f427c0a6300 unknown :-1 s=CLOSED pgs=130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.368+0000 7f429a58f640 1 --2- 172.21.15.86:0/1392417930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f429419be10 0x7f429419c2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.368+0000 7f429a58f640 1 --2- 172.21.15.86:0/1392417930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4294106170 0x7f4294197ae0 unknown :-1 s=CLOSED pgs=889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.368+0000 7f429a58f640 1 --2- 172.21.15.86:0/1392417930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42941010f0 0x7f42941975a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:05.369 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.368+0000 7f429a58f640 1 -- 172.21.15.86:0/1392417930 >> 172.21.15.86:0/1392417930 conn(0x7f4294077fd0 msgr2=0x7f42940fef50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:05.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.368+0000 7f429a58f640 1 -- 172.21.15.86:0/1392417930 shutdown_connections 2024-04-21T08:41:05.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:05.368+0000 7f429a58f640 1 -- 172.21.15.86:0/1392417930 wait complete. 2024-04-21T08:41:05.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:05 smithi086 ceph-mon[27939]: scrub summary: idle 2024-04-21T08:41:05.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:05 smithi088 ceph-mon[31259]: scrub summary: idle 2024-04-21T08:41:05.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:05 smithi179 ceph-mon[31829]: scrub summary: idle 2024-04-21T08:41:06.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:06 smithi086 ceph-mon[27939]: pgmap v72: 129 pgs: 129 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 40 KiB/s wr, 27 op/s 2024-04-21T08:41:06.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:06 smithi088 ceph-mon[31259]: pgmap v72: 129 pgs: 129 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 40 KiB/s wr, 27 op/s 2024-04-21T08:41:06.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:06 smithi179 ceph-mon[31829]: pgmap v72: 129 pgs: 129 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 40 KiB/s wr, 27 op/s 2024-04-21T08:41:08.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:08 smithi086 ceph-mon[27939]: pgmap v73: 129 pgs: 129 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 40 KiB/s wr, 27 op/s 2024-04-21T08:41:08.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:08 smithi088 ceph-mon[31259]: pgmap v73: 129 pgs: 129 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 40 KiB/s wr, 27 op/s 2024-04-21T08:41:08.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:08 smithi179 ceph-mon[31829]: pgmap v73: 129 pgs: 129 active+clean; 2.8 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 40 KiB/s wr, 27 op/s 2024-04-21T08:41:10.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:10 smithi086 ceph-mon[27939]: pgmap v74: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 45 KiB/s wr, 36 op/s 2024-04-21T08:41:10.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:10 smithi088 ceph-mon[31259]: pgmap v74: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 45 KiB/s wr, 36 op/s 2024-04-21T08:41:10.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:10 smithi179 ceph-mon[31829]: pgmap v74: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 45 KiB/s wr, 36 op/s 2024-04-21T08:41:12.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:12 smithi086 ceph-mon[27939]: pgmap v75: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 37 KiB/s wr, 30 op/s 2024-04-21T08:41:12.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:12 smithi088 ceph-mon[31259]: pgmap v75: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 37 KiB/s wr, 30 op/s 2024-04-21T08:41:12.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:12 smithi179 ceph-mon[31829]: pgmap v75: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 37 KiB/s wr, 30 op/s 2024-04-21T08:41:14.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:14 smithi086 ceph-mon[27939]: pgmap v76: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 37 KiB/s wr, 34 op/s 2024-04-21T08:41:14.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:14 smithi088 ceph-mon[31259]: pgmap v76: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 37 KiB/s wr, 34 op/s 2024-04-21T08:41:14.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:14 smithi179 ceph-mon[31829]: pgmap v76: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 37 KiB/s wr, 34 op/s 2024-04-21T08:41:16.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:16 smithi086 ceph-mon[27939]: pgmap v77: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.9 KiB/s wr, 12 op/s 2024-04-21T08:41:16.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:16 smithi088 ceph-mon[31259]: pgmap v77: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.9 KiB/s wr, 12 op/s 2024-04-21T08:41:16.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:16 smithi179 ceph-mon[31829]: pgmap v77: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.9 KiB/s wr, 12 op/s 2024-04-21T08:41:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.460+0000 7fe6713ef640 1 -- 172.21.15.86:0/4110536670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe66c108050 msgr2=0x7fe66c108430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.460+0000 7fe6713ef640 1 --2- 172.21.15.86:0/4110536670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe66c108050 0x7fe66c108430 secure :-1 s=READY pgs=894 cs=0 l=1 rev1=1 crypto rx=0x7fe65c009930 tx=0x7fe65c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.461+0000 7fe6713ef640 1 -- 172.21.15.86:0/4110536670 shutdown_connections 2024-04-21T08:41:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.461+0000 7fe6713ef640 1 --2- 172.21.15.86:0/4110536670 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe66c06b140 0x7fe66c10edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:18.461 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.461+0000 7fe6713ef640 1 --2- 172.21.15.86:0/4110536670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe66c06a780 0x7fe66c06ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.461+0000 7fe6713ef640 1 --2- 172.21.15.86:0/4110536670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe66c108050 0x7fe66c108430 unknown :-1 s=CLOSED pgs=894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.461+0000 7fe6713ef640 1 -- 172.21.15.86:0/4110536670 >> 172.21.15.86:0/4110536670 conn(0x7fe66c076c50 msgr2=0x7fe66c077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.461+0000 7fe6713ef640 1 -- 172.21.15.86:0/4110536670 shutdown_connections 2024-04-21T08:41:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.461+0000 7fe6713ef640 1 -- 172.21.15.86:0/4110536670 wait complete. 2024-04-21T08:41:18.462 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.461+0000 7fe6713ef640 1 Processor -- start 2024-04-21T08:41:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.461+0000 7fe6713ef640 1 -- start start 2024-04-21T08:41:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.462+0000 7fe6713ef640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe66c06a780 0x7fe66c101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.462+0000 7fe6713ef640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe66c06b140 0x7fe66c1017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.462+0000 7fe6713ef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe66c108050 0x7fe66c101cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:18.464 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.462+0000 7fe6713ef640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c112180 con 0x7fe66c108050 2024-04-21T08:41:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.462+0000 7fe6713ef640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c112000 con 0x7fe66c06b140 2024-04-21T08:41:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.462+0000 7fe6713ef640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c112300 con 0x7fe66c06a780 2024-04-21T08:41:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.462+0000 7fe66b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe66c108050 0x7fe66c101cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.462+0000 7fe66b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe66c108050 0x7fe66c101cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50034/0 (socket says 172.21.15.86:50034) 2024-04-21T08:41:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.462+0000 7fe66b7fe640 1 -- 172.21.15.86:0/3426888138 learned_addr learned my addr 172.21.15.86:0/3426888138 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.462+0000 7fe66a7fc640 1 --2- 172.21.15.86:0/3426888138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe66c06b140 0x7fe66c1017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.462+0000 7fe66affd640 1 --2- 172.21.15.86:0/3426888138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe66c06a780 0x7fe66c101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.462+0000 7fe66b7fe640 1 -- 172.21.15.86:0/3426888138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe66c06a780 msgr2=0x7fe66c101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.462+0000 7fe66b7fe640 1 --2- 172.21.15.86:0/3426888138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe66c06a780 0x7fe66c101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.462+0000 7fe66b7fe640 1 -- 172.21.15.86:0/3426888138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe66c06b140 msgr2=0x7fe66c1017b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.462+0000 7fe66b7fe640 1 --2- 172.21.15.86:0/3426888138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe66c06b140 0x7fe66c1017b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.462+0000 7fe66b7fe640 1 -- 172.21.15.86:0/3426888138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe66c1a6b40 con 0x7fe66c108050 2024-04-21T08:41:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.462+0000 7fe66a7fc640 1 --2- 172.21.15.86:0/3426888138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe66c06b140 0x7fe66c1017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:18.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.462+0000 7fe66b7fe640 1 --2- 172.21.15.86:0/3426888138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe66c108050 0x7fe66c101cf0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fe66400eaa0 tx=0x7fe66400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:18.466 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.463+0000 7fe653fff640 1 -- 172.21.15.86:0/3426888138 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66400cdb0 con 0x7fe66c108050 2024-04-21T08:41:18.466 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.463+0000 7fe653fff640 1 -- 172.21.15.86:0/3426888138 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe664005020 con 0x7fe66c108050 2024-04-21T08:41:18.466 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.463+0000 7fe6713ef640 1 -- 172.21.15.86:0/3426888138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe66c1a6e30 con 0x7fe66c108050 2024-04-21T08:41:18.466 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.463+0000 7fe653fff640 1 -- 172.21.15.86:0/3426888138 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe6640116d0 con 0x7fe66c108050 2024-04-21T08:41:18.466 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.463+0000 7fe6713ef640 1 -- 172.21.15.86:0/3426888138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe66c1a7300 con 0x7fe66c108050 2024-04-21T08:41:18.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.465+0000 7fe653fff640 1 -- 172.21.15.86:0/3426888138 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe6640051d0 con 0x7fe66c108050 2024-04-21T08:41:18.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.465+0000 7fe673fd9740 1 Processor -- start 2024-04-21T08:41:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.465+0000 7fe673fd9740 1 -- start start 2024-04-21T08:41:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.465+0000 7fe653fff640 1 --2- 172.21.15.86:0/3426888138 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe6540a3dc0 0x7fe6540a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.465+0000 7fe673fd9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3a7848d0 0x559c3a783620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.465+0000 7fe673fd9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c3a783b60 0x559c3a783fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.466+0000 7fe673fd9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c3a78d0e0 0x559c3a78f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.466+0000 7fe673fd9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559c3a6e88d0 con 0x559c3a78d0e0 2024-04-21T08:41:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.466+0000 7fe673fd9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559c3a685e80 con 0x559c3a783b60 2024-04-21T08:41:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.466+0000 7fe673fd9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559c3a6e4890 con 0x559c3a7848d0 2024-04-21T08:41:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.466+0000 7fe66a7fc640 1 --2- 172.21.15.86:0/3426888138 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe6540a3dc0 0x7fe6540a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.466+0000 7fe653fff640 1 -- 172.21.15.86:0/3426888138 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fe6640d0dc0 con 0x7fe66c108050 2024-04-21T08:41:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.466+0000 7fe66b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c3a78d0e0 0x559c3a78f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.466+0000 7fe66b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c3a78d0e0 0x559c3a78f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50050/0 (socket says 172.21.15.86:50050) 2024-04-21T08:41:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.466+0000 7fe66b7fe640 1 -- 172.21.15.86:0/3295898924 learned_addr learned my addr 172.21.15.86:0/3295898924 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:18.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.466+0000 7fe66affd640 1 --2- 172.21.15.86:0/3295898924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3a7848d0 0x559c3a783620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:18.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.466+0000 7fe66a7fc640 1 --2- 172.21.15.86:0/3295898924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c3a783b60 0x559c3a783fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:18.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.466+0000 7fe66b7fe640 1 -- 172.21.15.86:0/3295898924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3a7848d0 msgr2=0x559c3a783620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:18.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.466+0000 7fe66b7fe640 1 --2- 172.21.15.86:0/3295898924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3a7848d0 0x559c3a783620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:18.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.466+0000 7fe66b7fe640 1 -- 172.21.15.86:0/3295898924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c3a783b60 msgr2=0x559c3a783fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:18.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.466+0000 7fe66b7fe640 1 --2- 172.21.15.86:0/3295898924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c3a783b60 0x559c3a783fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:18.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.466+0000 7fe66b7fe640 1 -- 172.21.15.86:0/3295898924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c3a6a7320 con 0x559c3a78d0e0 2024-04-21T08:41:18.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.466+0000 7fe66a7fc640 1 --2- 172.21.15.86:0/3426888138 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe6540a3dc0 0x7fe6540a6280 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe66c102d00 tx=0x7fe6600137d0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:18.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.466+0000 7fe66b7fe640 1 --2- 172.21.15.86:0/3295898924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c3a78d0e0 0x559c3a78f4d0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fe66409c6f0 tx=0x7fe66409d1b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f10e9d9c2c910d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:18.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.466+0000 7fe6517fa640 1 -- 172.21.15.86:0/3295898924 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66401f040 con 0x559c3a78d0e0 2024-04-21T08:41:18.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.466+0000 7fe6517fa640 1 -- 172.21.15.86:0/3295898924 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe66409c9f0 con 0x559c3a78d0e0 2024-04-21T08:41:18.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.466+0000 7fe6517fa640 1 -- 172.21.15.86:0/3295898924 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe664059f80 con 0x559c3a78d0e0 2024-04-21T08:41:18.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.467+0000 7fe673fd9740 1 -- 172.21.15.86:0/3295898924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c3a78d0e0 msgr2=0x559c3a78f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.467+0000 7fe673fd9740 1 --2- 172.21.15.86:0/3295898924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c3a78d0e0 0x559c3a78f4d0 secure :-1 s=READY pgs=896 cs=0 l=1 rev1=1 crypto rx=0x7fe66409c6f0 tx=0x7fe66409d1b0 comp rx=0 tx=0).stop 2024-04-21T08:41:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.467+0000 7fe673fd9740 1 -- 172.21.15.86:0/3295898924 shutdown_connections 2024-04-21T08:41:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.467+0000 7fe673fd9740 1 --2- 172.21.15.86:0/3295898924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c3a78d0e0 0x559c3a78f4d0 unknown :-1 s=CLOSED pgs=896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.467+0000 7fe673fd9740 1 --2- 172.21.15.86:0/3295898924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c3a783b60 0x559c3a783fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.467+0000 7fe673fd9740 1 --2- 172.21.15.86:0/3295898924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3a7848d0 0x559c3a783620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.467+0000 7fe673fd9740 1 -- 172.21.15.86:0/3295898924 >> 172.21.15.86:0/3295898924 conn(0x559c3a74ac90 msgr2=0x559c3a77e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.467+0000 7fe673fd9740 1 -- 172.21.15.86:0/3295898924 shutdown_connections 2024-04-21T08:41:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.467+0000 7fe673fd9740 1 -- 172.21.15.86:0/3295898924 wait complete. 2024-04-21T08:41:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.467+0000 7fe673fd9740 1 -- start start 2024-04-21T08:41:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.468+0000 7fe673fd9740 1 Processor -- start 2024-04-21T08:41:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.468+0000 7fe673fd9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c3a783b60 0x559c3a78ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.468+0000 7fe673fd9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3a7848d0 0x559c3a789de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.468+0000 7fe673fd9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c3a78d0e0 0x559c3a78a340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:18.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.468+0000 7fe673fd9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559c3a78fe10 con 0x559c3a78d0e0 2024-04-21T08:41:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.468+0000 7fe673fd9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559c3a670eb0 con 0x559c3a783b60 2024-04-21T08:41:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.468+0000 7fe673fd9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559c3a78ff90 con 0x559c3a7848d0 2024-04-21T08:41:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.468+0000 7fe673fd9740 10 client.0 _ll_register_callbacks cb 0x559c3a69db40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.468+0000 7fe66b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c3a78d0e0 0x559c3a78a340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.468+0000 7fe66b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c3a78d0e0 0x559c3a78a340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50054/0 (socket says 172.21.15.86:50054) 2024-04-21T08:41:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.468+0000 7fe66b7fe640 1 -- 172.21.15.86:0/151354131 learned_addr learned my addr 172.21.15.86:0/151354131 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.468+0000 7fe66affd640 1 --2- 172.21.15.86:0/151354131 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c3a783b60 0x559c3a78ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.468+0000 7fe66a7fc640 1 --2- 172.21.15.86:0/151354131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3a7848d0 0x559c3a789de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.468+0000 7fe66b7fe640 1 -- 172.21.15.86:0/151354131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3a7848d0 msgr2=0x559c3a789de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.468+0000 7fe66b7fe640 1 --2- 172.21.15.86:0/151354131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3a7848d0 0x559c3a789de0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.468+0000 7fe66b7fe640 1 -- 172.21.15.86:0/151354131 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c3a783b60 msgr2=0x559c3a78ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.468+0000 7fe66b7fe640 1 --2- 172.21.15.86:0/151354131 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c3a783b60 0x559c3a78ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.468+0000 7fe66b7fe640 1 -- 172.21.15.86:0/151354131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c3a3fb210 con 0x559c3a78d0e0 2024-04-21T08:41:18.476 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.468+0000 7fe66b7fe640 1 --2- 172.21.15.86:0/151354131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c3a78d0e0 0x559c3a78a340 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fe66400efd0 tx=0x7fe6640a0ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d12d568daa68d474 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.468+0000 7fe64eff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.468+0000 7fe64eff5640 1 -- 172.21.15.86:0/151354131 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66401f070 con 0x559c3a78d0e0 2024-04-21T08:41:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.469+0000 7fe64eff5640 1 -- 172.21.15.86:0/151354131 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fe6640bbbb0 con 0x559c3a78d0e0 2024-04-21T08:41:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.469+0000 7fe673fd9740 1 -- 172.21.15.86:0/151354131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe66409c9f0 con 0x559c3a78d0e0 2024-04-21T08:41:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.469+0000 7fe64eff5640 1 -- 172.21.15.86:0/151354131 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe6640b7020 con 0x559c3a78d0e0 2024-04-21T08:41:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.469+0000 7fe673fd9740 10 client.16854 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.469+0000 7fe673fd9740 1 -- 172.21.15.86:0/151354131 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559c3a4c7620 con 0x559c3a78d0e0 2024-04-21T08:41:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.469+0000 7fe64eff5640 1 -- 172.21.15.86:0/151354131 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe66409c9f0 con 0x559c3a78d0e0 2024-04-21T08:41:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.469+0000 7fe64eff5640 1 -- 172.21.15.86:0/151354131 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe66400f040 con 0x559c3a78d0e0 2024-04-21T08:41:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.469+0000 7fe673fd9740 10 client.16854 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.469+0000 7fe673fd9740 10 client.16854 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.469+0000 7fe673fd9740 20 client.16854 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.469+0000 7fe673fd9740 1 --2- 172.21.15.86:0/151354131 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559c3a79cf40 0x559c3a79f330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:18.477 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.469+0000 7fe673fd9740 4 client.16854 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.469+0000 7fe673fd9740 1 -- 172.21.15.86:0/151354131 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c3a6fc880 con 0x559c3a79cf40 2024-04-21T08:41:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.469+0000 7fe66affd640 1 --2- 172.21.15.86:0/151354131 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559c3a79cf40 0x559c3a79f330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.470+0000 7fe64eff5640 10 client.16854 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.470+0000 7fe66affd640 1 --2- 172.21.15.86:0/151354131 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559c3a79cf40 0x559c3a79f330 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5d28ecf35f167f0b server_cookie=9a3d03b7ce8ca4f1 in_seq=0 out_seq=0 2024-04-21T08:41:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.470+0000 7fe64eff5640 1 -- 172.21.15.86:0/151354131 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559c3a6fc880 con 0x559c3a79cf40 2024-04-21T08:41:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.470+0000 7fe64eff5640 10 client.16854 handle_command_reply: tid=0 2024-04-21T08:41:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.470+0000 7fe673fd9740 1 client.16854 shutdown 2024-04-21T08:41:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.471+0000 7fe673fd9740 1 -- 172.21.15.86:0/151354131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c3a78d0e0 msgr2=0x559c3a78a340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.471+0000 7fe673fd9740 1 --2- 172.21.15.86:0/151354131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c3a78d0e0 0x559c3a78a340 secure :-1 s=READY pgs=897 cs=0 l=1 rev1=1 crypto rx=0x7fe66400efd0 tx=0x7fe6640a0ee0 comp rx=0 tx=0).stop 2024-04-21T08:41:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.471+0000 7fe673fd9740 1 -- 172.21.15.86:0/151354131 shutdown_connections 2024-04-21T08:41:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.471+0000 7fe673fd9740 1 --2- 172.21.15.86:0/151354131 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559c3a79cf40 0x559c3a79f330 crc :-1 s=READY pgs=105 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:18.478 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.471+0000 7fe673fd9740 1 --2- 172.21.15.86:0/151354131 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c3a78d0e0 0x559c3a78a340 unknown :-1 s=CLOSED pgs=897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.471+0000 7fe673fd9740 1 --2- 172.21.15.86:0/151354131 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3a7848d0 0x559c3a789de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.471+0000 7fe673fd9740 1 --2- 172.21.15.86:0/151354131 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c3a783b60 0x559c3a78ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.471+0000 7fe64eff5640 0 client.16854 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.471+0000 7fe673fd9740 1 -- 172.21.15.86:0/151354131 >> 172.21.15.86:0/151354131 conn(0x559c3a74ac90 msgr2=0x559c3a7805a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.471+0000 7fe673fd9740 1 -- 172.21.15.86:0/151354131 shutdown_connections 2024-04-21T08:41:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.471+0000 7fe673fd9740 1 -- 172.21.15.86:0/151354131 wait complete. 2024-04-21T08:41:18.479 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.471+0000 7fe673fd9740 20 client.16854 trim_cache size 0 max 16384 2024-04-21T08:41:18.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.485+0000 7fe673fd9740 1 Processor -- start 2024-04-21T08:41:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.485+0000 7fe673fd9740 1 -- start start 2024-04-21T08:41:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.485+0000 7fe673fd9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3a783b60 0x559c3a798d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.486+0000 7fe673fd9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c3a7848d0 0x559c3a78c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.486+0000 7fe673fd9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c3a78d0e0 0x559c3a78ca50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.486+0000 7fe673fd9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c1b1400 con 0x559c3a7848d0 2024-04-21T08:41:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.486+0000 7fe673fd9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c1b2f40 con 0x559c3a78d0e0 2024-04-21T08:41:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.486+0000 7fe673fd9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c1b1ad0 con 0x559c3a783b60 2024-04-21T08:41:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.486+0000 7fe66a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c3a7848d0 0x559c3a78c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.486+0000 7fe66a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c3a7848d0 0x559c3a78c510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50056/0 (socket says 172.21.15.86:50056) 2024-04-21T08:41:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.486+0000 7fe66a7fc640 1 -- 172.21.15.86:0/2928633077 learned_addr learned my addr 172.21.15.86:0/2928633077 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:18.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.486+0000 7fe66affd640 1 --2- 172.21.15.86:0/2928633077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3a783b60 0x559c3a798d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:18.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.486+0000 7fe66a7fc640 1 -- 172.21.15.86:0/2928633077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3a783b60 msgr2=0x559c3a798d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:18.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.486+0000 7fe66b7fe640 1 --2- 172.21.15.86:0/2928633077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c3a78d0e0 0x559c3a78ca50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:18.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.486+0000 7fe66a7fc640 1 --2- 172.21.15.86:0/2928633077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3a783b60 0x559c3a798d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:18.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.486+0000 7fe66a7fc640 1 -- 172.21.15.86:0/2928633077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c3a78d0e0 msgr2=0x559c3a78ca50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:18.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.486+0000 7fe66a7fc640 1 --2- 172.21.15.86:0/2928633077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c3a78d0e0 0x559c3a78ca50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:18.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.486+0000 7fe66a7fc640 1 -- 172.21.15.86:0/2928633077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c3a782e30 con 0x559c3a7848d0 2024-04-21T08:41:18.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.486+0000 7fe66affd640 1 --2- 172.21.15.86:0/2928633077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3a783b60 0x559c3a798d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:18.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.486+0000 7fe66b7fe640 1 --2- 172.21.15.86:0/2928633077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c3a78d0e0 0x559c3a78ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:18.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.486+0000 7fe66a7fc640 1 --2- 172.21.15.86:0/2928633077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c3a7848d0 0x559c3a78c510 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fe660015010 tx=0x7fe660016000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c07deffd16392c12 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:18.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.486+0000 7fe650ff9640 1 -- 172.21.15.86:0/2928633077 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe660026070 con 0x559c3a7848d0 2024-04-21T08:41:18.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.486+0000 7fe650ff9640 1 -- 172.21.15.86:0/2928633077 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe660020900 con 0x559c3a7848d0 2024-04-21T08:41:18.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.486+0000 7fe650ff9640 1 -- 172.21.15.86:0/2928633077 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66002a9a0 con 0x559c3a7848d0 2024-04-21T08:41:18.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.487+0000 7fe673fd9740 1 -- 172.21.15.86:0/2928633077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c3a7848d0 msgr2=0x559c3a78c510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.487+0000 7fe673fd9740 1 --2- 172.21.15.86:0/2928633077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c3a7848d0 0x559c3a78c510 secure :-1 s=READY pgs=898 cs=0 l=1 rev1=1 crypto rx=0x7fe660015010 tx=0x7fe660016000 comp rx=0 tx=0).stop 2024-04-21T08:41:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.487+0000 7fe673fd9740 1 -- 172.21.15.86:0/2928633077 shutdown_connections 2024-04-21T08:41:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.487+0000 7fe673fd9740 1 --2- 172.21.15.86:0/2928633077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c3a78d0e0 0x559c3a78ca50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.487+0000 7fe673fd9740 1 --2- 172.21.15.86:0/2928633077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c3a7848d0 0x559c3a78c510 unknown :-1 s=CLOSED pgs=898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.487+0000 7fe673fd9740 1 --2- 172.21.15.86:0/2928633077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3a783b60 0x559c3a798d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.487+0000 7fe673fd9740 1 -- 172.21.15.86:0/2928633077 >> 172.21.15.86:0/2928633077 conn(0x559c3a74ac90 msgr2=0x559c3a785060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.487+0000 7fe673fd9740 1 -- 172.21.15.86:0/2928633077 shutdown_connections 2024-04-21T08:41:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.487+0000 7fe673fd9740 1 -- 172.21.15.86:0/2928633077 wait complete. 2024-04-21T08:41:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.487+0000 7fe673fd9740 1 -- start start 2024-04-21T08:41:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.487+0000 7fe673fd9740 1 Processor -- start 2024-04-21T08:41:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.487+0000 7fe673fd9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3a783b60 0x559c3a786e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.487+0000 7fe673fd9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c3a7848d0 0x559c3a787390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.487+0000 7fe673fd9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c3a78d0e0 0x559c3a802170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:18.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.487+0000 7fe673fd9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c1b2870 con 0x559c3a78d0e0 2024-04-21T08:41:18.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.487+0000 7fe673fd9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c1b21a0 con 0x559c3a7848d0 2024-04-21T08:41:18.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.487+0000 7fe673fd9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe66c1af3c0 con 0x559c3a783b60 2024-04-21T08:41:18.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.488+0000 7fe673fd9740 10 client.0 _ll_register_callbacks cb 0x7fe664074810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:18.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.488+0000 7fe66b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c3a78d0e0 0x559c3a802170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:18.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.488+0000 7fe66b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c3a78d0e0 0x559c3a802170 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50068/0 (socket says 172.21.15.86:50068) 2024-04-21T08:41:18.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.488+0000 7fe66b7fe640 1 -- 172.21.15.86:0/389904298 learned_addr learned my addr 172.21.15.86:0/389904298 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:18.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.488+0000 7fe66a7fc640 1 --2- 172.21.15.86:0/389904298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c3a7848d0 0x559c3a787390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:18.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.488+0000 7fe66affd640 1 --2- 172.21.15.86:0/389904298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3a783b60 0x559c3a786e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:18.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.488+0000 7fe66b7fe640 1 -- 172.21.15.86:0/389904298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3a783b60 msgr2=0x559c3a786e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:18.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.488+0000 7fe66b7fe640 1 --2- 172.21.15.86:0/389904298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3a783b60 0x559c3a786e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:18.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.488+0000 7fe66b7fe640 1 -- 172.21.15.86:0/389904298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c3a7848d0 msgr2=0x559c3a787390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:18.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.488+0000 7fe66b7fe640 1 --2- 172.21.15.86:0/389904298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c3a7848d0 0x559c3a787390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:18.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.488+0000 7fe66b7fe640 1 -- 172.21.15.86:0/389904298 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c3a8026b0 con 0x559c3a78d0e0 2024-04-21T08:41:18.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.488+0000 7fe66b7fe640 1 --2- 172.21.15.86:0/389904298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c3a78d0e0 0x559c3a802170 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fe6640993c0 tx=0x7fe66409c5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4260820a1271ff93 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:18.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.488+0000 7fe64eff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:18.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.488+0000 7fe64eff5640 1 -- 172.21.15.86:0/389904298 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe66401f040 con 0x559c3a78d0e0 2024-04-21T08:41:18.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.488+0000 7fe64eff5640 1 -- 172.21.15.86:0/389904298 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fe6640bbdc0 con 0x559c3a78d0e0 2024-04-21T08:41:18.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.488+0000 7fe673fd9740 1 -- 172.21.15.86:0/389904298 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe660020900 con 0x559c3a78d0e0 2024-04-21T08:41:18.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.488+0000 7fe64eff5640 1 -- 172.21.15.86:0/389904298 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe6640ce630 con 0x559c3a78d0e0 2024-04-21T08:41:18.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.488+0000 7fe673fd9740 10 client.16860 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:18.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.488+0000 7fe673fd9740 1 -- 172.21.15.86:0/389904298 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559c3a8037d0 con 0x559c3a78d0e0 2024-04-21T08:41:18.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.488+0000 7fe64eff5640 1 -- 172.21.15.86:0/389904298 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe660020900 con 0x559c3a78d0e0 2024-04-21T08:41:18.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.489+0000 7fe64eff5640 1 -- 172.21.15.86:0/389904298 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe66409b920 con 0x559c3a78d0e0 2024-04-21T08:41:18.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.489+0000 7fe673fd9740 10 client.16860 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:18.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.489+0000 7fe673fd9740 10 client.16860 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:18.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.489+0000 7fe673fd9740 20 client.16860 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:18.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.489+0000 7fe673fd9740 1 --2- 172.21.15.86:0/389904298 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559c3a803d90 0x559c3a8061b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:18.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.489+0000 7fe673fd9740 4 client.16860 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:41:18.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.489+0000 7fe673fd9740 1 -- 172.21.15.86:0/389904298 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x559c3a6883d0 con 0x559c3a803d90 2024-04-21T08:41:18.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.489+0000 7fe66affd640 1 --2- 172.21.15.86:0/389904298 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559c3a803d90 0x559c3a8061b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:18.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.489+0000 7fe64eff5640 10 client.16860 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:18.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.489+0000 7fe66affd640 1 --2- 172.21.15.86:0/389904298 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559c3a803d90 0x559c3a8061b0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0c8ae1a4b1dac48 server_cookie=ebcdba386fc681d8 in_seq=0 out_seq=0 2024-04-21T08:41:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.500+0000 7fe64eff5640 1 -- 172.21.15.86:0/389904298 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x559c3a6883d0 con 0x559c3a803d90 2024-04-21T08:41:18.501 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.500+0000 7fe64eff5640 10 client.16860 handle_command_reply: tid=0 2024-04-21T08:41:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.500+0000 7fe673fd9740 1 client.16860 shutdown 2024-04-21T08:41:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.500+0000 7fe673fd9740 1 -- 172.21.15.86:0/389904298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c3a78d0e0 msgr2=0x559c3a802170 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.500+0000 7fe673fd9740 1 --2- 172.21.15.86:0/389904298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c3a78d0e0 0x559c3a802170 secure :-1 s=READY pgs=899 cs=0 l=1 rev1=1 crypto rx=0x7fe6640993c0 tx=0x7fe66409c5a0 comp rx=0 tx=0).stop 2024-04-21T08:41:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.501+0000 7fe673fd9740 1 -- 172.21.15.86:0/389904298 shutdown_connections 2024-04-21T08:41:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.501+0000 7fe673fd9740 1 --2- 172.21.15.86:0/389904298 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559c3a803d90 0x559c3a8061b0 crc :-1 s=READY pgs=106 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.501+0000 7fe673fd9740 1 --2- 172.21.15.86:0/389904298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c3a78d0e0 0x559c3a802170 unknown :-1 s=CLOSED pgs=899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.501+0000 7fe673fd9740 1 --2- 172.21.15.86:0/389904298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c3a7848d0 0x559c3a787390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.501+0000 7fe64eff5640 0 client.16860 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.501+0000 7fe673fd9740 1 --2- 172.21.15.86:0/389904298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3a783b60 0x559c3a786e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.501+0000 7fe673fd9740 1 -- 172.21.15.86:0/389904298 >> 172.21.15.86:0/389904298 conn(0x559c3a74ac90 msgr2=0x559c3a77fc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.501+0000 7fe673fd9740 1 -- 172.21.15.86:0/389904298 shutdown_connections 2024-04-21T08:41:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.501+0000 7fe673fd9740 1 -- 172.21.15.86:0/389904298 wait complete. 2024-04-21T08:41:18.502 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.501+0000 7fe673fd9740 20 client.16860 trim_cache size 0 max 16384 2024-04-21T08:41:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:41:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:41:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:41:18.503 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:41:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.501+0000 7fe6713ef640 1 -- 172.21.15.86:0/3426888138 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe6540a3dc0 msgr2=0x7fe6540a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.501+0000 7fe6713ef640 1 --2- 172.21.15.86:0/3426888138 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe6540a3dc0 0x7fe6540a6280 secure :-1 s=READY pgs=131 cs=0 l=1 rev1=1 crypto rx=0x7fe66c102d00 tx=0x7fe6600137d0 comp rx=0 tx=0).stop 2024-04-21T08:41:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.501+0000 7fe6713ef640 1 -- 172.21.15.86:0/3426888138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe66c108050 msgr2=0x7fe66c101cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.501+0000 7fe6713ef640 1 --2- 172.21.15.86:0/3426888138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe66c108050 0x7fe66c101cf0 secure :-1 s=READY pgs=895 cs=0 l=1 rev1=1 crypto rx=0x7fe66400eaa0 tx=0x7fe66400ef60 comp rx=0 tx=0).stop 2024-04-21T08:41:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.501+0000 7fe6713ef640 1 -- 172.21.15.86:0/3426888138 shutdown_connections 2024-04-21T08:41:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.501+0000 7fe6713ef640 1 --2- 172.21.15.86:0/3426888138 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe6540a3dc0 0x7fe6540a6280 unknown :-1 s=CLOSED pgs=131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.501+0000 7fe6713ef640 1 --2- 172.21.15.86:0/3426888138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe66c108050 0x7fe66c101cf0 unknown :-1 s=CLOSED pgs=895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:18.504 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.501+0000 7fe6713ef640 1 --2- 172.21.15.86:0/3426888138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe66c06b140 0x7fe66c1017b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.501+0000 7fe6713ef640 1 --2- 172.21.15.86:0/3426888138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe66c06a780 0x7fe66c101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.501+0000 7fe6713ef640 1 -- 172.21.15.86:0/3426888138 >> 172.21.15.86:0/3426888138 conn(0x7fe66c076c50 msgr2=0x7fe66c0ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.501+0000 7fe6713ef640 1 -- 172.21.15.86:0/3426888138 shutdown_connections 2024-04-21T08:41:18.505 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:18.501+0000 7fe6713ef640 1 -- 172.21.15.86:0/3426888138 wait complete. 2024-04-21T08:41:18.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:18 smithi086 ceph-mon[27939]: pgmap v78: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.9 KiB/s wr, 12 op/s 2024-04-21T08:41:18.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:18 smithi088 ceph-mon[31259]: pgmap v78: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.9 KiB/s wr, 12 op/s 2024-04-21T08:41:18.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:18 smithi179 ceph-mon[31829]: pgmap v78: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 4.9 KiB/s wr, 12 op/s 2024-04-21T08:41:20.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:20 smithi086 ceph-mon[27939]: pgmap v79: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.2 KiB/s wr, 17 op/s 2024-04-21T08:41:20.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:20 smithi088 ceph-mon[31259]: pgmap v79: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.2 KiB/s wr, 17 op/s 2024-04-21T08:41:20.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:20 smithi179 ceph-mon[31829]: pgmap v79: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 6.2 KiB/s wr, 17 op/s 2024-04-21T08:41:22.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:22 smithi086 ceph-mon[27939]: pgmap v80: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-21T08:41:22.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:22 smithi088 ceph-mon[31259]: pgmap v80: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-21T08:41:22.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:22 smithi179 ceph-mon[31829]: pgmap v80: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.3 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-21T08:41:24.038 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:41:24.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.147+0000 7f575916d640 1 -- 172.21.15.86:0/424872353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5754101110 msgr2=0x7f5754101590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:24.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.147+0000 7f575916d640 1 --2- 172.21.15.86:0/424872353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5754101110 0x7f5754101590 secure :-1 s=READY pgs=900 cs=0 l=1 rev1=1 crypto rx=0x7f573c009900 tx=0x7f573c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:24.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.147+0000 7f575916d640 1 -- 172.21.15.86:0/424872353 shutdown_connections 2024-04-21T08:41:24.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.147+0000 7f575916d640 1 --2- 172.21.15.86:0/424872353 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5754101ad0 0x7f575410dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.147+0000 7f575916d640 1 --2- 172.21.15.86:0/424872353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5754101110 0x7f5754101590 unknown :-1 s=CLOSED pgs=900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.147+0000 7f575916d640 1 --2- 172.21.15.86:0/424872353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5754106f30 0x7f5754107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.147+0000 7f575916d640 1 -- 172.21.15.86:0/424872353 >> 172.21.15.86:0/424872353 conn(0x7f5754077e30 msgr2=0x7f5754078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:24.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.147+0000 7f575916d640 1 -- 172.21.15.86:0/424872353 shutdown_connections 2024-04-21T08:41:24.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.147+0000 7f575916d640 1 -- 172.21.15.86:0/424872353 wait complete. 2024-04-21T08:41:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.148+0000 7f575916d640 1 Processor -- start 2024-04-21T08:41:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.148+0000 7f575916d640 1 -- start start 2024-04-21T08:41:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.148+0000 7f575916d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5754101110 0x7f57541986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.148+0000 7f575916d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5754101ad0 0x7f5754198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.148+0000 7f575916d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5754106f30 0x7f575419cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.148+0000 7f575916d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5754112200 con 0x7f5754101ad0 2024-04-21T08:41:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.148+0000 7f575916d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5754112080 con 0x7f5754106f30 2024-04-21T08:41:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.148+0000 7f575916d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5754112380 con 0x7f5754101110 2024-04-21T08:41:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.148+0000 7f574bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5754101ad0 0x7f5754198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.148+0000 7f574bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5754101ad0 0x7f5754198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42232/0 (socket says 172.21.15.86:42232) 2024-04-21T08:41:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.148+0000 7f574bfff640 1 -- 172.21.15.86:0/1340617512 learned_addr learned my addr 172.21.15.86:0/1340617512 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.149+0000 7f575359e640 1 --2- 172.21.15.86:0/1340617512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5754106f30 0x7f575419cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:24.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.149+0000 7f5752d9d640 1 --2- 172.21.15.86:0/1340617512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5754101110 0x7f57541986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:24.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.149+0000 7f574bfff640 1 -- 172.21.15.86:0/1340617512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5754101110 msgr2=0x7f57541986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:24.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.149+0000 7f574bfff640 1 --2- 172.21.15.86:0/1340617512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5754101110 0x7f57541986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.149+0000 7f574bfff640 1 -- 172.21.15.86:0/1340617512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5754106f30 msgr2=0x7f575419cf10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:24.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.149+0000 7f574bfff640 1 --2- 172.21.15.86:0/1340617512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5754106f30 0x7f575419cf10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.149+0000 7f574bfff640 1 -- 172.21.15.86:0/1340617512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f575419d640 con 0x7f5754101ad0 2024-04-21T08:41:24.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.149+0000 7f574bfff640 1 --2- 172.21.15.86:0/1340617512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5754101ad0 0x7f5754198be0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f573c005940 tx=0x7f573c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:24.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.149+0000 7f5750d99640 1 -- 172.21.15.86:0/1340617512 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c043020 con 0x7f5754101ad0 2024-04-21T08:41:24.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.149+0000 7f5750d99640 1 -- 172.21.15.86:0/1340617512 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f573c02caf0 con 0x7f5754101ad0 2024-04-21T08:41:24.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.149+0000 7f575916d640 1 -- 172.21.15.86:0/1340617512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f575419d8d0 con 0x7f5754101ad0 2024-04-21T08:41:24.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.149+0000 7f5750d99640 1 -- 172.21.15.86:0/1340617512 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f573c003af0 con 0x7f5754101ad0 2024-04-21T08:41:24.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.149+0000 7f575916d640 1 -- 172.21.15.86:0/1340617512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57541a51c0 con 0x7f5754101ad0 2024-04-21T08:41:24.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.150+0000 7f575916d640 1 -- 172.21.15.86:0/1340617512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5754069230 con 0x7f5754101ad0 2024-04-21T08:41:24.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.151+0000 7f5750d99640 1 -- 172.21.15.86:0/1340617512 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f573c046020 con 0x7f5754101ad0 2024-04-21T08:41:24.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.152+0000 7f5750d99640 1 --2- 172.21.15.86:0/1340617512 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f57400a3da0 0x7f57400a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:24.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.152+0000 7f5752d9d640 1 --2- 172.21.15.86:0/1340617512 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f57400a3da0 0x7f57400a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:24.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.152+0000 7f5750d99640 1 -- 172.21.15.86:0/1340617512 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f573c0f3200 con 0x7f5754101ad0 2024-04-21T08:41:24.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.153+0000 7f5752d9d640 1 --2- 172.21.15.86:0/1340617512 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f57400a3da0 0x7f57400a6260 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5744005fd0 tx=0x7f5744005850 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:24.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.156+0000 7f5750d99640 1 -- 172.21.15.86:0/1340617512 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f573c04d020 con 0x7f5754101ad0 2024-04-21T08:41:24.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:24 smithi086 ceph-mon[27939]: pgmap v81: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-04-21T08:41:24.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.376+0000 7f575916d640 1 -- 172.21.15.86:0/1340617512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f57541999a0 con 0x7f5754101ad0 2024-04-21T08:41:24.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.376+0000 7f5750d99640 1 -- 172.21.15.86:0/1340617512 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7f57541999a0 con 0x7f5754101ad0 2024-04-21T08:41:24.377 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:41:24.377 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:38:45.202126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24398,24394,24455,24472]},"id":1}]} 2024-04-21T08:41:24.377 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-21T08:41:24.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.378+0000 7f575916d640 1 -- 172.21.15.86:0/1340617512 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f57400a3da0 msgr2=0x7f57400a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:24.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.378+0000 7f575916d640 1 --2- 172.21.15.86:0/1340617512 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f57400a3da0 0x7f57400a6260 secure :-1 s=READY pgs=132 cs=0 l=1 rev1=1 crypto rx=0x7f5744005fd0 tx=0x7f5744005850 comp rx=0 tx=0).stop 2024-04-21T08:41:24.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.378+0000 7f575916d640 1 -- 172.21.15.86:0/1340617512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5754101ad0 msgr2=0x7f5754198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:24.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.378+0000 7f575916d640 1 --2- 172.21.15.86:0/1340617512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5754101ad0 0x7f5754198be0 secure :-1 s=READY pgs=901 cs=0 l=1 rev1=1 crypto rx=0x7f573c005940 tx=0x7f573c0045e0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.378+0000 7f575916d640 1 -- 172.21.15.86:0/1340617512 shutdown_connections 2024-04-21T08:41:24.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.378+0000 7f575916d640 1 --2- 172.21.15.86:0/1340617512 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f57400a3da0 0x7f57400a6260 unknown :-1 s=CLOSED pgs=132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.378+0000 7f575916d640 1 --2- 172.21.15.86:0/1340617512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5754106f30 0x7f575419cf10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.378+0000 7f575916d640 1 --2- 172.21.15.86:0/1340617512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5754101ad0 0x7f5754198be0 unknown :-1 s=CLOSED pgs=901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.378+0000 7f575916d640 1 --2- 172.21.15.86:0/1340617512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5754101110 0x7f57541986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.378+0000 7f575916d640 1 -- 172.21.15.86:0/1340617512 >> 172.21.15.86:0/1340617512 conn(0x7f5754077e30 msgr2=0x7f57540ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:24.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.378+0000 7f575916d640 1 -- 172.21.15.86:0/1340617512 shutdown_connections 2024-04-21T08:41:24.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.379+0000 7f575916d640 1 -- 172.21.15.86:0/1340617512 wait complete. 2024-04-21T08:41:24.389 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id c2a8e01a --release --await' 2024-04-21T08:41:24.390 DEBUG:teuthology.orchestra.run.smithi086:> timeout 45 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24482 quiesce db --set-id c2a8e01a --release --await 2024-04-21T08:41:24.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.494+0000 7f1b84011640 1 -- 172.21.15.86:0/3132376019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b7c107320 msgr2=0x7f1b7c06aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:24.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.494+0000 7f1b84011640 1 --2- 172.21.15.86:0/3132376019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b7c107320 0x7f1b7c06aa90 secure :-1 s=READY pgs=902 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c009900 tx=0x7f1b6c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:24.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.494+0000 7f1b84011640 1 -- 172.21.15.86:0/3132376019 shutdown_connections 2024-04-21T08:41:24.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.494+0000 7f1b84011640 1 --2- 172.21.15.86:0/3132376019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b7c06b170 0x7f1b7c104990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.494+0000 7f1b84011640 1 --2- 172.21.15.86:0/3132376019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b7c107320 0x7f1b7c06aa90 unknown :-1 s=CLOSED pgs=902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.494+0000 7f1b84011640 1 --2- 172.21.15.86:0/3132376019 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b7c106f40 0x7f1b7c06a550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.494+0000 7f1b84011640 1 -- 172.21.15.86:0/3132376019 >> 172.21.15.86:0/3132376019 conn(0x7f1b7c076c50 msgr2=0x7f1b7c077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:24.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.494+0000 7f1b84011640 1 -- 172.21.15.86:0/3132376019 shutdown_connections 2024-04-21T08:41:24.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.494+0000 7f1b84011640 1 -- 172.21.15.86:0/3132376019 wait complete. 2024-04-21T08:41:24.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.495+0000 7f1b84011640 1 Processor -- start 2024-04-21T08:41:24.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.495+0000 7f1b84011640 1 -- start start 2024-04-21T08:41:24.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.495+0000 7f1b84011640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b7c106f40 0x7f1b7c19ed80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:24.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.495+0000 7f1b84011640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b7c107320 0x7f1b7c19f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:24.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.495+0000 7f1b84011640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b7c1a3640 0x7f1b7c1a3af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:24.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.495+0000 7f1b84011640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c1187e0 con 0x7f1b7c1a3640 2024-04-21T08:41:24.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.495+0000 7f1b84011640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c118660 con 0x7f1b7c107320 2024-04-21T08:41:24.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.495+0000 7f1b84011640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c118960 con 0x7f1b7c106f40 2024-04-21T08:41:24.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.495+0000 7f1b825ae640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b7c1a3640 0x7f1b7c1a3af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:24.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.495+0000 7f1b825ae640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b7c1a3640 0x7f1b7c1a3af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42250/0 (socket says 172.21.15.86:42250) 2024-04-21T08:41:24.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.495+0000 7f1b825ae640 1 -- 172.21.15.86:0/2206857169 learned_addr learned my addr 172.21.15.86:0/2206857169 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:24.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.495+0000 7f1b815ac640 1 --2- 172.21.15.86:0/2206857169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b7c107320 0x7f1b7c19f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:24.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.495+0000 7f1b81dad640 1 --2- 172.21.15.86:0/2206857169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b7c106f40 0x7f1b7c19ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:24.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.495+0000 7f1b825ae640 1 -- 172.21.15.86:0/2206857169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b7c106f40 msgr2=0x7f1b7c19ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:24.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.495+0000 7f1b825ae640 1 --2- 172.21.15.86:0/2206857169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b7c106f40 0x7f1b7c19ed80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.495+0000 7f1b825ae640 1 -- 172.21.15.86:0/2206857169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b7c107320 msgr2=0x7f1b7c19f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:24.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.495+0000 7f1b825ae640 1 --2- 172.21.15.86:0/2206857169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b7c107320 0x7f1b7c19f2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.495+0000 7f1b825ae640 1 -- 172.21.15.86:0/2206857169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1b7c1a4130 con 0x7f1b7c1a3640 2024-04-21T08:41:24.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.495+0000 7f1b81dad640 1 --2- 172.21.15.86:0/2206857169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b7c106f40 0x7f1b7c19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:24.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.496+0000 7f1b825ae640 1 --2- 172.21.15.86:0/2206857169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b7c1a3640 0x7f1b7c1a3af0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f1b7400d720 tx=0x7f1b7400dbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:24.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.496+0000 7f1b72ffd640 1 -- 172.21.15.86:0/2206857169 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b74017070 con 0x7f1b7c1a3640 2024-04-21T08:41:24.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.496+0000 7f1b72ffd640 1 -- 172.21.15.86:0/2206857169 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b7400de30 con 0x7f1b7c1a3640 2024-04-21T08:41:24.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.496+0000 7f1b84011640 1 -- 172.21.15.86:0/2206857169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1b7c1a4420 con 0x7f1b7c1a3640 2024-04-21T08:41:24.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.496+0000 7f1b72ffd640 1 -- 172.21.15.86:0/2206857169 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b74004d00 con 0x7f1b7c1a3640 2024-04-21T08:41:24.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.496+0000 7f1b84011640 1 -- 172.21.15.86:0/2206857169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1b7c1abc10 con 0x7f1b7c1a3640 2024-04-21T08:41:24.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.498+0000 7f1b72ffd640 1 -- 172.21.15.86:0/2206857169 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1b74004ea0 con 0x7f1b7c1a3640 2024-04-21T08:41:24.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.498+0000 7f1b86c00740 1 Processor -- start 2024-04-21T08:41:24.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.498+0000 7f1b86c00740 1 -- start start 2024-04-21T08:41:24.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.498+0000 7f1b86c00740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b565af1310 0x55b565af16f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:24.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.498+0000 7f1b86c00740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b565af1c30 0x55b565afa2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:24.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.498+0000 7f1b86c00740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b565afa810 0x55b565afcc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:24.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.498+0000 7f1b86c00740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b565a56500 con 0x55b565afa810 2024-04-21T08:41:24.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.498+0000 7f1b86c00740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b5659fc6a0 con 0x55b565af1c30 2024-04-21T08:41:24.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.498+0000 7f1b86c00740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b565a524c0 con 0x55b565af1310 2024-04-21T08:41:24.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.499+0000 7f1b815ac640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b565afa810 0x55b565afcc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:24.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.499+0000 7f1b72ffd640 1 --2- 172.21.15.86:0/2206857169 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1b680a3d20 0x7f1b680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:24.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.499+0000 7f1b815ac640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b565afa810 0x55b565afcc00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42254/0 (socket says 172.21.15.86:42254) 2024-04-21T08:41:24.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.499+0000 7f1b815ac640 1 -- 172.21.15.86:0/1327731447 learned_addr learned my addr 172.21.15.86:0/1327731447 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:24.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.499+0000 7f1b81dad640 1 --2- 172.21.15.86:0/1327731447 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b565af1c30 0x55b565afa2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:24.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.499+0000 7f1b815ac640 1 --2- 172.21.15.86:0/1327731447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b565af1310 0x55b565af16f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:24.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.499+0000 7f1b825ae640 1 --2- 172.21.15.86:0/2206857169 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1b680a3d20 0x7f1b680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:24.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.499+0000 7f1b72ffd640 1 -- 172.21.15.86:0/2206857169 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f1b740d0530 con 0x7f1b7c1a3640 2024-04-21T08:41:24.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.499+0000 7f1b81dad640 1 -- 172.21.15.86:0/1327731447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b565af1310 msgr2=0x55b565af16f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:24.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.499+0000 7f1b81dad640 1 --2- 172.21.15.86:0/1327731447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b565af1310 0x55b565af16f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.499+0000 7f1b81dad640 1 -- 172.21.15.86:0/1327731447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b565afa810 msgr2=0x55b565afcc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:24.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.499+0000 7f1b81dad640 1 --2- 172.21.15.86:0/1327731447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b565afa810 0x55b565afcc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.499+0000 7f1b81dad640 1 -- 172.21.15.86:0/1327731447 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b565a1b5e0 con 0x55b565af1c30 2024-04-21T08:41:24.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.499+0000 7f1b815ac640 1 --2- 172.21.15.86:0/1327731447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b565afa810 0x55b565afcc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:41:24.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.499+0000 7f1b825ae640 1 --2- 172.21.15.86:0/2206857169 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1b680a3d20 0x7f1b680a61e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1b7409be60 tx=0x7f1b74086c90 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:24.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.499+0000 7f1b81dad640 1 --2- 172.21.15.86:0/1327731447 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b565af1c30 0x55b565afa2d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1b78005c40 tx=0x7f1b7800d710 comp rx=0 tx=0).ready entity=mon.2 client_cookie=42b5a3ae3b44a56 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:24.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.499+0000 7f1b67fff640 1 -- 172.21.15.86:0/1327731447 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b7800f830 con 0x55b565af1c30 2024-04-21T08:41:24.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.500+0000 7f1b67fff640 1 -- 172.21.15.86:0/1327731447 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b78008e10 con 0x55b565af1c30 2024-04-21T08:41:24.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.500+0000 7f1b67fff640 1 -- 172.21.15.86:0/1327731447 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b78018d80 con 0x55b565af1c30 2024-04-21T08:41:24.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.500+0000 7f1b86c00740 1 -- 172.21.15.86:0/1327731447 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b565af1c30 msgr2=0x55b565afa2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:24.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.500+0000 7f1b86c00740 1 --2- 172.21.15.86:0/1327731447 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b565af1c30 0x55b565afa2d0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x7f1b78005c40 tx=0x7f1b7800d710 comp rx=0 tx=0).stop 2024-04-21T08:41:24.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.500+0000 7f1b86c00740 1 -- 172.21.15.86:0/1327731447 shutdown_connections 2024-04-21T08:41:24.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.500+0000 7f1b86c00740 1 --2- 172.21.15.86:0/1327731447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b565afa810 0x55b565afcc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.500+0000 7f1b86c00740 1 --2- 172.21.15.86:0/1327731447 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b565af1c30 0x55b565afa2d0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.500+0000 7f1b86c00740 1 --2- 172.21.15.86:0/1327731447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b565af1310 0x55b565af16f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.500+0000 7f1b86c00740 1 -- 172.21.15.86:0/1327731447 >> 172.21.15.86:0/1327731447 conn(0x55b565aeba40 msgr2=0x55b565aebe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:24.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.500+0000 7f1b86c00740 1 -- 172.21.15.86:0/1327731447 shutdown_connections 2024-04-21T08:41:24.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.500+0000 7f1b86c00740 1 -- 172.21.15.86:0/1327731447 wait complete. 2024-04-21T08:41:24.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.501+0000 7f1b86c00740 1 -- start start 2024-04-21T08:41:24.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.501+0000 7f1b86c00740 1 Processor -- start 2024-04-21T08:41:24.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.501+0000 7f1b86c00740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b565af1310 0x55b565af7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:24.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.501+0000 7f1b86c00740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b565af1c30 0x55b565af7bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:24.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.501+0000 7f1b86c00740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b565afa810 0x55b565af80f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:24.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.501+0000 7f1b86c00740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b565afd600 con 0x55b565af1310 2024-04-21T08:41:24.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.501+0000 7f1b86c00740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b565a0fa60 con 0x55b565afa810 2024-04-21T08:41:24.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.501+0000 7f1b86c00740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b565afd780 con 0x55b565af1c30 2024-04-21T08:41:24.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.501+0000 7f1b86c00740 10 client.0 _ll_register_callbacks cb 0x55b565768110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:24.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.501+0000 7f1b815ac640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b565af1310 0x55b565af7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:24.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.501+0000 7f1b815ac640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b565af1310 0x55b565af7670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42262/0 (socket says 172.21.15.86:42262) 2024-04-21T08:41:24.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.501+0000 7f1b815ac640 1 -- 172.21.15.86:0/3242003300 learned_addr learned my addr 172.21.15.86:0/3242003300 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:24.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.501+0000 7f1b815ac640 1 --2- 172.21.15.86:0/3242003300 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b565afa810 0x55b565af80f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:24.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.501+0000 7f1b81dad640 1 --2- 172.21.15.86:0/3242003300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b565af1c30 0x55b565af7bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:24.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.501+0000 7f1b815ac640 1 -- 172.21.15.86:0/3242003300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b565af1c30 msgr2=0x55b565af7bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:24.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.501+0000 7f1b815ac640 1 --2- 172.21.15.86:0/3242003300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b565af1c30 0x55b565af7bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.501+0000 7f1b815ac640 1 -- 172.21.15.86:0/3242003300 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b565afa810 msgr2=0x55b565af80f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:24.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.501+0000 7f1b815ac640 1 --2- 172.21.15.86:0/3242003300 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b565afa810 0x55b565af80f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.501+0000 7f1b815ac640 1 -- 172.21.15.86:0/3242003300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b565768920 con 0x55b565af1310 2024-04-21T08:41:24.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.501+0000 7f1b81dad640 1 --2- 172.21.15.86:0/3242003300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b565af1c30 0x55b565af7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:24.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.502+0000 7f1b815ac640 1 --2- 172.21.15.86:0/3242003300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b565af1310 0x55b565af7670 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c009230 tx=0x7f1b6c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8038484fdd47e48b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:24.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.502+0000 7f1b657fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:24.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.502+0000 7f1b657fa640 1 -- 172.21.15.86:0/3242003300 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c005450 con 0x55b565af1310 2024-04-21T08:41:24.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.502+0000 7f1b657fa640 1 -- 172.21.15.86:0/3242003300 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f1b6c0046c0 con 0x55b565af1310 2024-04-21T08:41:24.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.502+0000 7f1b86c00740 1 -- 172.21.15.86:0/3242003300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b78008e10 con 0x55b565af1310 2024-04-21T08:41:24.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.502+0000 7f1b657fa640 1 -- 172.21.15.86:0/3242003300 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1b6c054050 con 0x55b565af1310 2024-04-21T08:41:24.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.502+0000 7f1b86c00740 10 client.16878 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:24.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.502+0000 7f1b86c00740 1 -- 172.21.15.86:0/3242003300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b565834d30 con 0x55b565af1310 2024-04-21T08:41:24.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.502+0000 7f1b657fa640 1 -- 172.21.15.86:0/3242003300 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b78008e10 con 0x55b565af1310 2024-04-21T08:41:24.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.502+0000 7f1b657fa640 1 -- 172.21.15.86:0/3242003300 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1b6c040040 con 0x55b565af1310 2024-04-21T08:41:24.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.502+0000 7f1b86c00740 10 client.16878 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:24.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.502+0000 7f1b86c00740 10 client.16878 resolve_mds: validated gid 24482 aka daemon mds.b 2024-04-21T08:41:24.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.502+0000 7f1b86c00740 20 client.16878 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:24.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.502+0000 7f1b86c00740 1 --2- 172.21.15.86:0/3242003300 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b565b0a910 0x55b565b0cd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:24.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.502+0000 7f1b86c00740 4 client.16878 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:24.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.502+0000 7f1b86c00740 1 -- 172.21.15.86:0/3242003300 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b565a29a10 con 0x55b565b0a910 2024-04-21T08:41:24.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.503+0000 7f1b81dad640 1 --2- 172.21.15.86:0/3242003300 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b565b0a910 0x55b565b0cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:24.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.503+0000 7f1b81dad640 1 --2- 172.21.15.86:0/3242003300 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b565b0a910 0x55b565b0cd90 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8cb29f471325bb63 server_cookie=2f0330c938e3a89f in_seq=0 out_seq=0 2024-04-21T08:41:24.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.503+0000 7f1b657fa640 10 client.16878 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:24.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.504+0000 7f1b657fa640 1 -- 172.21.15.86:0/3242003300 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b565a29a10 con 0x55b565b0a910 2024-04-21T08:41:24.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.504+0000 7f1b657fa640 10 client.16878 handle_command_reply: tid=0 2024-04-21T08:41:24.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.504+0000 7f1b86c00740 1 client.16878 shutdown 2024-04-21T08:41:24.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.504+0000 7f1b86c00740 1 -- 172.21.15.86:0/3242003300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b565af1310 msgr2=0x55b565af7670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:24.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.504+0000 7f1b86c00740 1 --2- 172.21.15.86:0/3242003300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b565af1310 0x55b565af7670 secure :-1 s=READY pgs=904 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c009230 tx=0x7f1b6c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:24.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.504+0000 7f1b86c00740 1 -- 172.21.15.86:0/3242003300 shutdown_connections 2024-04-21T08:41:24.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.504+0000 7f1b86c00740 1 --2- 172.21.15.86:0/3242003300 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b565b0a910 0x55b565b0cd90 crc :-1 s=READY pgs=107 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.504+0000 7f1b86c00740 1 --2- 172.21.15.86:0/3242003300 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b565afa810 0x55b565af80f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.504+0000 7f1b86c00740 1 --2- 172.21.15.86:0/3242003300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b565af1c30 0x55b565af7bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.504+0000 7f1b86c00740 1 --2- 172.21.15.86:0/3242003300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b565af1310 0x55b565af7670 unknown :-1 s=CLOSED pgs=904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.504+0000 7f1b657fa640 0 client.16878 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:24.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.504+0000 7f1b86c00740 1 -- 172.21.15.86:0/3242003300 >> 172.21.15.86:0/3242003300 conn(0x55b565aeba40 msgr2=0x55b565aed790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:24.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.504+0000 7f1b86c00740 1 -- 172.21.15.86:0/3242003300 shutdown_connections 2024-04-21T08:41:24.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.504+0000 7f1b86c00740 1 -- 172.21.15.86:0/3242003300 wait complete. 2024-04-21T08:41:24.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.504+0000 7f1b86c00740 20 client.16878 trim_cache size 0 max 16384 2024-04-21T08:41:24.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.521+0000 7f1b86c00740 1 Processor -- start 2024-04-21T08:41:24.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.521+0000 7f1b86c00740 1 -- start start 2024-04-21T08:41:24.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.521+0000 7f1b86c00740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b565af1310 0x55b565af9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:24.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.521+0000 7f1b86c00740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b565af1c30 0x55b565b064c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:24.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.521+0000 7f1b86c00740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b565afa810 0x55b565aeab00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:24.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.521+0000 7f1b86c00740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c1b59a0 con 0x55b565af1310 2024-04-21T08:41:24.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.521+0000 7f1b86c00740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c1b74e0 con 0x55b565af1c30 2024-04-21T08:41:24.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.521+0000 7f1b86c00740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c1b6070 con 0x55b565afa810 2024-04-21T08:41:24.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.521+0000 7f1b815ac640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b565af1310 0x55b565af9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:24.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.521+0000 7f1b815ac640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b565af1310 0x55b565af9da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42274/0 (socket says 172.21.15.86:42274) 2024-04-21T08:41:24.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.521+0000 7f1b815ac640 1 -- 172.21.15.86:0/3789500578 learned_addr learned my addr 172.21.15.86:0/3789500578 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:24.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.522+0000 7f1b81dad640 1 --2- 172.21.15.86:0/3789500578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b565af1c30 0x55b565b064c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:24.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.522+0000 7f1b815ac640 1 -- 172.21.15.86:0/3789500578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b565afa810 msgr2=0x55b565aeab00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:24.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.522+0000 7f1b815ac640 1 --2- 172.21.15.86:0/3789500578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b565afa810 0x55b565aeab00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.522+0000 7f1b815ac640 1 -- 172.21.15.86:0/3789500578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b565af1c30 msgr2=0x55b565b064c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:24.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.522+0000 7f1b815ac640 1 --2- 172.21.15.86:0/3789500578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b565af1c30 0x55b565b064c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.522+0000 7f1b815ac640 1 -- 172.21.15.86:0/3789500578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b565ae9fb0 con 0x55b565af1310 2024-04-21T08:41:24.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.522+0000 7f1b81dad640 1 --2- 172.21.15.86:0/3789500578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b565af1c30 0x55b565b064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:24.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.522+0000 7f1b815ac640 1 --2- 172.21.15.86:0/3789500578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b565af1310 0x55b565af9da0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c004520 tx=0x7f1b6c02cb10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b945c7424331797 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:24.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.522+0000 7f1b677fe640 1 -- 172.21.15.86:0/3789500578 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c051020 con 0x55b565af1310 2024-04-21T08:41:24.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.522+0000 7f1b677fe640 1 -- 172.21.15.86:0/3789500578 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b6c003f90 con 0x55b565af1310 2024-04-21T08:41:24.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.522+0000 7f1b677fe640 1 -- 172.21.15.86:0/3789500578 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c004b20 con 0x55b565af1310 2024-04-21T08:41:24.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.523+0000 7f1b86c00740 1 -- 172.21.15.86:0/3789500578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b565af1310 msgr2=0x55b565af9da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:24.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.523+0000 7f1b86c00740 1 --2- 172.21.15.86:0/3789500578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b565af1310 0x55b565af9da0 secure :-1 s=READY pgs=905 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c004520 tx=0x7f1b6c02cb10 comp rx=0 tx=0).stop 2024-04-21T08:41:24.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.523+0000 7f1b86c00740 1 -- 172.21.15.86:0/3789500578 shutdown_connections 2024-04-21T08:41:24.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.523+0000 7f1b86c00740 1 --2- 172.21.15.86:0/3789500578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b565afa810 0x55b565aeab00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.523+0000 7f1b86c00740 1 --2- 172.21.15.86:0/3789500578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b565af1c30 0x55b565b064c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.523+0000 7f1b86c00740 1 --2- 172.21.15.86:0/3789500578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b565af1310 0x55b565af9da0 unknown :-1 s=CLOSED pgs=905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.523+0000 7f1b86c00740 1 -- 172.21.15.86:0/3789500578 >> 172.21.15.86:0/3789500578 conn(0x55b565aeba40 msgr2=0x55b565af66b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:24.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.523+0000 7f1b86c00740 1 -- 172.21.15.86:0/3789500578 shutdown_connections 2024-04-21T08:41:24.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.523+0000 7f1b86c00740 1 -- 172.21.15.86:0/3789500578 wait complete. 2024-04-21T08:41:24.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.523+0000 7f1b86c00740 1 -- start start 2024-04-21T08:41:24.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.523+0000 7f1b86c00740 1 Processor -- start 2024-04-21T08:41:24.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.523+0000 7f1b86c00740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b565af1310 0x55b565afb630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:24.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.523+0000 7f1b86c00740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b565af1c30 0x55b565afbb70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:24.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.523+0000 7f1b86c00740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b565afa810 0x55b565afc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:24.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.523+0000 7f1b86c00740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c1b6e10 con 0x55b565af1310 2024-04-21T08:41:24.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.523+0000 7f1b86c00740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c1b6740 con 0x55b565af1c30 2024-04-21T08:41:24.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.523+0000 7f1b86c00740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1b7c1b3960 con 0x55b565afa810 2024-04-21T08:41:24.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.523+0000 7f1b86c00740 10 client.0 _ll_register_callbacks cb 0x7f1b6c02ccb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:24.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.523+0000 7f1b815ac640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b565af1310 0x55b565afb630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:24.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.524+0000 7f1b815ac640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b565af1310 0x55b565afb630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42280/0 (socket says 172.21.15.86:42280) 2024-04-21T08:41:24.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.524+0000 7f1b815ac640 1 -- 172.21.15.86:0/232506964 learned_addr learned my addr 172.21.15.86:0/232506964 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:24.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.524+0000 7f1b81dad640 1 --2- 172.21.15.86:0/232506964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b565af1c30 0x55b565afbb70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:24.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.524+0000 7f1b815ac640 1 -- 172.21.15.86:0/232506964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b565afa810 msgr2=0x55b565afc0b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:24.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.524+0000 7f1b815ac640 1 --2- 172.21.15.86:0/232506964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b565afa810 0x55b565afc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.524+0000 7f1b815ac640 1 -- 172.21.15.86:0/232506964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b565af1c30 msgr2=0x55b565afbb70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:24.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.524+0000 7f1b815ac640 1 --2- 172.21.15.86:0/232506964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b565af1c30 0x55b565afbb70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.524+0000 7f1b815ac640 1 -- 172.21.15.86:0/232506964 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b565b00bb0 con 0x55b565af1310 2024-04-21T08:41:24.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.524+0000 7f1b815ac640 1 --2- 172.21.15.86:0/232506964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b565af1310 0x55b565afb630 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c003b10 tx=0x7f1b6c0035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7470efcc735cb64c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:24.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.524+0000 7f1b657fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:24.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.524+0000 7f1b657fa640 1 -- 172.21.15.86:0/232506964 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1b6c051030 con 0x55b565af1310 2024-04-21T08:41:24.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.524+0000 7f1b657fa640 1 -- 172.21.15.86:0/232506964 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f1b6c005450 con 0x55b565af1310 2024-04-21T08:41:24.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.524+0000 7f1b86c00740 1 -- 172.21.15.86:0/232506964 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1b6c003f90 con 0x55b565af1310 2024-04-21T08:41:24.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.524+0000 7f1b657fa640 1 -- 172.21.15.86:0/232506964 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1b6c0046c0 con 0x55b565af1310 2024-04-21T08:41:24.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.524+0000 7f1b86c00740 10 client.16884 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:24.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.524+0000 7f1b86c00740 1 -- 172.21.15.86:0/232506964 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b565b711c0 con 0x55b565af1310 2024-04-21T08:41:24.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.524+0000 7f1b657fa640 1 -- 172.21.15.86:0/232506964 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1b6c003f90 con 0x55b565af1310 2024-04-21T08:41:24.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.525+0000 7f1b657fa640 1 -- 172.21.15.86:0/232506964 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1b6c043840 con 0x55b565af1310 2024-04-21T08:41:24.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.525+0000 7f1b86c00740 10 client.16884 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:24.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.525+0000 7f1b86c00740 10 client.16884 resolve_mds: validated gid 24482 aka daemon mds.b 2024-04-21T08:41:24.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.525+0000 7f1b86c00740 20 client.16884 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:24.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.525+0000 7f1b86c00740 1 --2- 172.21.15.86:0/232506964 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b565b717d0 0x55b565b73bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:24.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.525+0000 7f1b86c00740 4 client.16884 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "c2a8e01a", "release": true, "await": true}] 2024-04-21T08:41:24.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.525+0000 7f1b86c00740 1 -- 172.21.15.86:0/232506964 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "quiesce db", "set_id": "c2a8e01a", "release": true, "await": true}) -- 0x55b565a63f20 con 0x55b565b717d0 2024-04-21T08:41:24.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.525+0000 7f1b81dad640 1 --2- 172.21.15.86:0/232506964 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b565b717d0 0x55b565b73bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:24.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.525+0000 7f1b657fa640 10 client.16884 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:24.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.525+0000 7f1b81dad640 1 --2- 172.21.15.86:0/232506964 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b565b717d0 0x55b565b73bc0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c4fbeff85ca2a03b server_cookie=c267edd760c31d0c in_seq=0 out_seq=0 2024-04-21T08:41:24.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.529+0000 7f1b657fa640 1 -- 172.21.15.86:0/232506964 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55b565a63f20 con 0x55b565b717d0 2024-04-21T08:41:24.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.529+0000 7f1b657fa640 10 client.16884 handle_command_reply: tid=0 2024-04-21T08:41:24.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.529+0000 7f1b86c00740 1 client.16884 shutdown 2024-04-21T08:41:24.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.529+0000 7f1b86c00740 1 -- 172.21.15.86:0/232506964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b565af1310 msgr2=0x55b565afb630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:24.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.529+0000 7f1b86c00740 1 --2- 172.21.15.86:0/232506964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b565af1310 0x55b565afb630 secure :-1 s=READY pgs=906 cs=0 l=1 rev1=1 crypto rx=0x7f1b6c003b10 tx=0x7f1b6c0035f0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.529+0000 7f1b86c00740 1 -- 172.21.15.86:0/232506964 shutdown_connections 2024-04-21T08:41:24.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.529+0000 7f1b86c00740 1 --2- 172.21.15.86:0/232506964 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b565b717d0 0x55b565b73bc0 crc :-1 s=READY pgs=108 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.529+0000 7f1b86c00740 1 --2- 172.21.15.86:0/232506964 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b565afa810 0x55b565afc0b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.529+0000 7f1b86c00740 1 --2- 172.21.15.86:0/232506964 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b565af1c30 0x55b565afbb70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.529+0000 7f1b657fa640 0 client.16884 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:24.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.529+0000 7f1b86c00740 1 --2- 172.21.15.86:0/232506964 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b565af1310 0x55b565afb630 unknown :-1 s=CLOSED pgs=906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.529+0000 7f1b86c00740 1 -- 172.21.15.86:0/232506964 >> 172.21.15.86:0/232506964 conn(0x55b565aeba40 msgr2=0x55b565af7ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:24.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.529+0000 7f1b86c00740 1 -- 172.21.15.86:0/232506964 shutdown_connections 2024-04-21T08:41:24.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.529+0000 7f1b86c00740 1 -- 172.21.15.86:0/232506964 wait complete. 2024-04-21T08:41:24.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.529+0000 7f1b86c00740 20 client.16884 trim_cache size 0 max 16384 2024-04-21T08:41:24.533 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:41:24.533 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 29, 2024-04-21T08:41:24.533 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24482, 2024-04-21T08:41:24.533 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 4, 2024-04-21T08:41:24.533 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-04-21T08:41:24.533 INFO:teuthology.orchestra.run.smithi086.stdout: "c2a8e01a": { 2024-04-21T08:41:24.533 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 4, 2024-04-21T08:41:24.533 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-04-21T08:41:24.533 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-21T08:41:24.533 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-04-21T08:41:24.533 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-21T08:41:24.533 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:41:24.533 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 30.0, 2024-04-21T08:41:24.533 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 81.9, 2024-04-21T08:41:24.534 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-04-21T08:41:24.534 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-04-21T08:41:24.534 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-04-21T08:41:24.534 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-21T08:41:24.534 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-04-21T08:41:24.534 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-21T08:41:24.534 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:41:24.534 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:41:24.534 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:41:24.534 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:41:24.534 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:41:24.534 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:41:24.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.530+0000 7f1b84011640 1 -- 172.21.15.86:0/2206857169 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1b680a3d20 msgr2=0x7f1b680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:24.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.530+0000 7f1b84011640 1 --2- 172.21.15.86:0/2206857169 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1b680a3d20 0x7f1b680a61e0 secure :-1 s=READY pgs=133 cs=0 l=1 rev1=1 crypto rx=0x7f1b7409be60 tx=0x7f1b74086c90 comp rx=0 tx=0).stop 2024-04-21T08:41:24.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.530+0000 7f1b84011640 1 -- 172.21.15.86:0/2206857169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b7c1a3640 msgr2=0x7f1b7c1a3af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:24.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.530+0000 7f1b84011640 1 --2- 172.21.15.86:0/2206857169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b7c1a3640 0x7f1b7c1a3af0 secure :-1 s=READY pgs=903 cs=0 l=1 rev1=1 crypto rx=0x7f1b7400d720 tx=0x7f1b7400dbe0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.530+0000 7f1b84011640 1 -- 172.21.15.86:0/2206857169 shutdown_connections 2024-04-21T08:41:24.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.530+0000 7f1b84011640 1 --2- 172.21.15.86:0/2206857169 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1b680a3d20 0x7f1b680a61e0 unknown :-1 s=CLOSED pgs=133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.530+0000 7f1b84011640 1 --2- 172.21.15.86:0/2206857169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1b7c1a3640 0x7f1b7c1a3af0 unknown :-1 s=CLOSED pgs=903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.530+0000 7f1b84011640 1 --2- 172.21.15.86:0/2206857169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1b7c107320 0x7f1b7c19f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.530+0000 7f1b84011640 1 --2- 172.21.15.86:0/2206857169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1b7c106f40 0x7f1b7c19ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.530+0000 7f1b84011640 1 -- 172.21.15.86:0/2206857169 >> 172.21.15.86:0/2206857169 conn(0x7f1b7c076c50 msgr2=0x7f1b7c0fe720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:24.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.530+0000 7f1b84011640 1 -- 172.21.15.86:0/2206857169 shutdown_connections 2024-04-21T08:41:24.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.530+0000 7f1b84011640 1 -- 172.21.15.86:0/2206857169 wait complete. 2024-04-21T08:41:24.540 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set_id: c2a8e01a, seconds elapsed: 22.9 2024-04-21T08:41:24.540 INFO:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 21.9 2024-04-21T08:41:24.540 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:41:24.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.639+0000 7f98326c1640 1 -- 172.21.15.86:0/1855548915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f982c105ee0 msgr2=0x7f982c1062c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:24.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.639+0000 7f98326c1640 1 --2- 172.21.15.86:0/1855548915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f982c105ee0 0x7f982c1062c0 secure :-1 s=READY pgs=907 cs=0 l=1 rev1=1 crypto rx=0x7f981c009900 tx=0x7f981c02bc80 comp rx=0 tx=0).stop 2024-04-21T08:41:24.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.639+0000 7f98326c1640 1 -- 172.21.15.86:0/1855548915 shutdown_connections 2024-04-21T08:41:24.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.639+0000 7f98326c1640 1 --2- 172.21.15.86:0/1855548915 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f982c101b60 0x7f982c10dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.639+0000 7f98326c1640 1 --2- 172.21.15.86:0/1855548915 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f982c1011a0 0x7f982c101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.639+0000 7f98326c1640 1 --2- 172.21.15.86:0/1855548915 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f982c105ee0 0x7f982c1062c0 unknown :-1 s=CLOSED pgs=907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.639+0000 7f98326c1640 1 -- 172.21.15.86:0/1855548915 >> 172.21.15.86:0/1855548915 conn(0x7f982c077c20 msgr2=0x7f982c078030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:24.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.639+0000 7f98326c1640 1 -- 172.21.15.86:0/1855548915 shutdown_connections 2024-04-21T08:41:24.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.639+0000 7f98326c1640 1 -- 172.21.15.86:0/1855548915 wait complete. 2024-04-21T08:41:24.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.640+0000 7f98326c1640 1 Processor -- start 2024-04-21T08:41:24.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.640+0000 7f98326c1640 1 -- start start 2024-04-21T08:41:24.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.640+0000 7f98326c1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f982c1011a0 0x7f982c19e580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:24.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.640+0000 7f98326c1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f982c101b60 0x7f982c19eac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:24.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.640+0000 7f98326c1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f982c105ee0 0x7f982c198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:24.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.640+0000 7f98326c1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f982c1111d0 con 0x7f982c101b60 2024-04-21T08:41:24.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.640+0000 7f98326c1640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f982c111050 con 0x7f982c1011a0 2024-04-21T08:41:24.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.640+0000 7f98326c1640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f982c111350 con 0x7f982c105ee0 2024-04-21T08:41:24.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.640+0000 7f982b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f982c101b60 0x7f982c19eac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:24.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.640+0000 7f982b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f982c101b60 0x7f982c19eac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42300/0 (socket says 172.21.15.86:42300) 2024-04-21T08:41:24.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.640+0000 7f982b7fe640 1 -- 172.21.15.86:0/3222091684 learned_addr learned my addr 172.21.15.86:0/3222091684 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:24.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.640+0000 7f982bfff640 1 --2- 172.21.15.86:0/3222091684 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f982c1011a0 0x7f982c19e580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:24.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.640+0000 7f9830c5e640 1 --2- 172.21.15.86:0/3222091684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f982c105ee0 0x7f982c198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:24.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.640+0000 7f982b7fe640 1 -- 172.21.15.86:0/3222091684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f982c105ee0 msgr2=0x7f982c198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:24.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.640+0000 7f982b7fe640 1 --2- 172.21.15.86:0/3222091684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f982c105ee0 0x7f982c198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.640+0000 7f982b7fe640 1 -- 172.21.15.86:0/3222091684 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f982c1011a0 msgr2=0x7f982c19e580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:24.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.640+0000 7f982b7fe640 1 --2- 172.21.15.86:0/3222091684 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f982c1011a0 0x7f982c19e580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.640+0000 7f982b7fe640 1 -- 172.21.15.86:0/3222091684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f982c198f00 con 0x7f982c101b60 2024-04-21T08:41:24.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.640+0000 7f9830c5e640 1 --2- 172.21.15.86:0/3222091684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f982c105ee0 0x7f982c198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:24.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.640+0000 7f982bfff640 1 --2- 172.21.15.86:0/3222091684 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f982c1011a0 0x7f982c19e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:24.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.640+0000 7f982b7fe640 1 --2- 172.21.15.86:0/3222091684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f982c101b60 0x7f982c19eac0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f982000b4a0 tx=0x7f982000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:24.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.641+0000 7f98297fa640 1 -- 172.21.15.86:0/3222091684 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9820014070 con 0x7f982c101b60 2024-04-21T08:41:24.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.641+0000 7f98297fa640 1 -- 172.21.15.86:0/3222091684 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9820010ac0 con 0x7f982c101b60 2024-04-21T08:41:24.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.641+0000 7f98326c1640 1 -- 172.21.15.86:0/3222091684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f982c1991f0 con 0x7f982c101b60 2024-04-21T08:41:24.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.641+0000 7f98297fa640 1 -- 172.21.15.86:0/3222091684 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9820017c30 con 0x7f982c101b60 2024-04-21T08:41:24.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.641+0000 7f98326c1640 1 -- 172.21.15.86:0/3222091684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f982c1a5500 con 0x7f982c101b60 2024-04-21T08:41:24.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.642+0000 7f98326c1640 1 -- 172.21.15.86:0/3222091684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f982c102570 con 0x7f982c101b60 2024-04-21T08:41:24.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.643+0000 7f98297fa640 1 -- 172.21.15.86:0/3222091684 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9820017070 con 0x7f982c101b60 2024-04-21T08:41:24.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.643+0000 7f98297fa640 1 --2- 172.21.15.86:0/3222091684 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f98140a3d20 0x7f98140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:24.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.644+0000 7f982bfff640 1 --2- 172.21.15.86:0/3222091684 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f98140a3d20 0x7f98140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:24.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.644+0000 7f98297fa640 1 -- 172.21.15.86:0/3222091684 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f98200d0280 con 0x7f982c101b60 2024-04-21T08:41:24.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.644+0000 7f982bfff640 1 --2- 172.21.15.86:0/3222091684 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f98140a3d20 0x7f98140a61e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f981c005ab0 tx=0x7f981c005a20 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:24.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.647+0000 7f98297fa640 1 -- 172.21.15.86:0/3222091684 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f982009c6c0 con 0x7f982c101b60 2024-04-21T08:41:24.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:24 smithi088 ceph-mon[31259]: pgmap v81: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-04-21T08:41:24.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:24 smithi179 ceph-mon[31829]: pgmap v81: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 4.2 KiB/s wr, 14 op/s 2024-04-21T08:41:24.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.864+0000 7f98326c1640 1 -- 172.21.15.86:0/3222091684 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f982c10a360 con 0x7f982c101b60 2024-04-21T08:41:24.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.864+0000 7f98297fa640 1 -- 172.21.15.86:0/3222091684 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7f982009bfc0 con 0x7f982c101b60 2024-04-21T08:41:24.864 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:41:24.865 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:38:45.202126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24398,24394,24455,24472]},"id":1}]} 2024-04-21T08:41:24.865 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-21T08:41:24.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.866+0000 7f98326c1640 1 -- 172.21.15.86:0/3222091684 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f98140a3d20 msgr2=0x7f98140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:24.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.866+0000 7f98326c1640 1 --2- 172.21.15.86:0/3222091684 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f98140a3d20 0x7f98140a61e0 secure :-1 s=READY pgs=134 cs=0 l=1 rev1=1 crypto rx=0x7f981c005ab0 tx=0x7f981c005a20 comp rx=0 tx=0).stop 2024-04-21T08:41:24.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.866+0000 7f98326c1640 1 -- 172.21.15.86:0/3222091684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f982c101b60 msgr2=0x7f982c19eac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:24.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.866+0000 7f98326c1640 1 --2- 172.21.15.86:0/3222091684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f982c101b60 0x7f982c19eac0 secure :-1 s=READY pgs=908 cs=0 l=1 rev1=1 crypto rx=0x7f982000b4a0 tx=0x7f982000b960 comp rx=0 tx=0).stop 2024-04-21T08:41:24.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.866+0000 7f98326c1640 1 -- 172.21.15.86:0/3222091684 shutdown_connections 2024-04-21T08:41:24.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.866+0000 7f98326c1640 1 --2- 172.21.15.86:0/3222091684 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f98140a3d20 0x7f98140a61e0 unknown :-1 s=CLOSED pgs=134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.866+0000 7f98326c1640 1 --2- 172.21.15.86:0/3222091684 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f982c105ee0 0x7f982c198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.866+0000 7f98326c1640 1 --2- 172.21.15.86:0/3222091684 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f982c101b60 0x7f982c19eac0 unknown :-1 s=CLOSED pgs=908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.866+0000 7f98326c1640 1 --2- 172.21.15.86:0/3222091684 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f982c1011a0 0x7f982c19e580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.866+0000 7f98326c1640 1 -- 172.21.15.86:0/3222091684 >> 172.21.15.86:0/3222091684 conn(0x7f982c077c20 msgr2=0x7f982c06a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:24.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.866+0000 7f98326c1640 1 -- 172.21.15.86:0/3222091684 shutdown_connections 2024-04-21T08:41:24.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.867+0000 7f98326c1640 1 -- 172.21.15.86:0/3222091684 wait complete. 2024-04-21T08:41:24.877 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 30 --expiration 81.9 --await' 2024-04-21T08:41:24.878 DEBUG:teuthology.orchestra.run.smithi086:> timeout 45 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24482 quiesce db / --timeout 30 --expiration 81.9 --await 2024-04-21T08:41:24.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.986+0000 7f67bcc77640 1 -- 172.21.15.86:0/2421288484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67b8102220 msgr2=0x7f67b8102600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:24.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.986+0000 7f67bcc77640 1 --2- 172.21.15.86:0/2421288484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67b8102220 0x7f67b8102600 secure :-1 s=READY pgs=909 cs=0 l=1 rev1=1 crypto rx=0x7f67a8009900 tx=0x7f67a802bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:24.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.986+0000 7f67bcc77640 1 -- 172.21.15.86:0/2421288484 shutdown_connections 2024-04-21T08:41:24.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.986+0000 7f67bcc77640 1 --2- 172.21.15.86:0/2421288484 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67b81072f0 0x7f67b810eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.986+0000 7f67bcc77640 1 --2- 172.21.15.86:0/2421288484 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67b8102bd0 0x7f67b8106c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.986+0000 7f67bcc77640 1 --2- 172.21.15.86:0/2421288484 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67b8102220 0x7f67b8102600 unknown :-1 s=CLOSED pgs=909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.986+0000 7f67bcc77640 1 -- 172.21.15.86:0/2421288484 >> 172.21.15.86:0/2421288484 conn(0x7f67b80fda80 msgr2=0x7f67b80ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:24.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.986+0000 7f67bcc77640 1 -- 172.21.15.86:0/2421288484 shutdown_connections 2024-04-21T08:41:24.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.986+0000 7f67bcc77640 1 -- 172.21.15.86:0/2421288484 wait complete. 2024-04-21T08:41:24.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.987+0000 7f67bcc77640 1 Processor -- start 2024-04-21T08:41:24.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.987+0000 7f67bcc77640 1 -- start start 2024-04-21T08:41:24.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.987+0000 7f67bcc77640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67b8102220 0x7f67b819e5e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:24.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.987+0000 7f67bcc77640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67b81072f0 0x7f67b819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.987+0000 7f67bcc77640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67b81986a0 0x7f67b8198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.987+0000 7f67bcc77640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f67b8112300 con 0x7f67b8102220 2024-04-21T08:41:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.987+0000 7f67bcc77640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f67b8112180 con 0x7f67b81072f0 2024-04-21T08:41:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.987+0000 7f67bcc77640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f67b8112480 con 0x7f67b81986a0 2024-04-21T08:41:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.987+0000 7f67b659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67b8102220 0x7f67b819e5e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.987+0000 7f67b659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67b8102220 0x7f67b819e5e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42326/0 (socket says 172.21.15.86:42326) 2024-04-21T08:41:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.987+0000 7f67b659c640 1 -- 172.21.15.86:0/3224380234 learned_addr learned my addr 172.21.15.86:0/3224380234 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.987+0000 7f67b5d9b640 1 --2- 172.21.15.86:0/3224380234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67b81072f0 0x7f67b819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.987+0000 7f67b6d9d640 1 --2- 172.21.15.86:0/3224380234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67b81986a0 0x7f67b8198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.987+0000 7f67b659c640 1 -- 172.21.15.86:0/3224380234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67b81986a0 msgr2=0x7f67b8198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.987+0000 7f67b659c640 1 --2- 172.21.15.86:0/3224380234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67b81986a0 0x7f67b8198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.987+0000 7f67b659c640 1 -- 172.21.15.86:0/3224380234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67b81072f0 msgr2=0x7f67b819eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.987+0000 7f67b659c640 1 --2- 172.21.15.86:0/3224380234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67b81072f0 0x7f67b819eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.987+0000 7f67b659c640 1 -- 172.21.15.86:0/3224380234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f67b81992a0 con 0x7f67b8102220 2024-04-21T08:41:24.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.988+0000 7f67b659c640 1 --2- 172.21.15.86:0/3224380234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67b8102220 0x7f67b819e5e0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f67a8005ab0 tx=0x7f67a802ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:24.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.988+0000 7f67a77fe640 1 -- 172.21.15.86:0/3224380234 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a8043020 con 0x7f67b8102220 2024-04-21T08:41:24.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.988+0000 7f67a77fe640 1 -- 172.21.15.86:0/3224380234 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67a802cc80 con 0x7f67b8102220 2024-04-21T08:41:24.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.988+0000 7f67bcc77640 1 -- 172.21.15.86:0/3224380234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f67b8199530 con 0x7f67b8102220 2024-04-21T08:41:24.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.988+0000 7f67a77fe640 1 -- 172.21.15.86:0/3224380234 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a8003d40 con 0x7f67b8102220 2024-04-21T08:41:24.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.988+0000 7f67bcc77640 1 -- 172.21.15.86:0/3224380234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f67b81a56d0 con 0x7f67b8102220 2024-04-21T08:41:24.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.990+0000 7f67a77fe640 1 -- 172.21.15.86:0/3224380234 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f67a8046020 con 0x7f67b8102220 2024-04-21T08:41:24.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.990+0000 7f67bf861740 1 Processor -- start 2024-04-21T08:41:24.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.990+0000 7f67bf861740 1 -- start start 2024-04-21T08:41:24.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.990+0000 7f67bf861740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5641233899c0 0x564123389da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:24.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.990+0000 7f67bf861740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56412338a2e0 0x564123392980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:24.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.990+0000 7f67a77fe640 1 --2- 172.21.15.86:0/3224380234 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f67a00a3d20 0x7f67a00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:24.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.990+0000 7f67bf861740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564123392ec0 0x5641233952b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:24.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.991+0000 7f67bf861740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5641232ea0b0 con 0x564123392ec0 2024-04-21T08:41:24.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.991+0000 7f67bf861740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x56412329b680 con 0x5641233899c0 2024-04-21T08:41:24.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.991+0000 7f67bf861740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5641232a7c60 con 0x56412338a2e0 2024-04-21T08:41:24.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.991+0000 7f67b5d9b640 1 --2- 172.21.15.86:0/3224380234 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f67a00a3d20 0x7f67a00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:24.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.991+0000 7f67b6d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564123392ec0 0x5641233952b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:24.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.991+0000 7f67b6d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564123392ec0 0x5641233952b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42340/0 (socket says 172.21.15.86:42340) 2024-04-21T08:41:24.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.991+0000 7f67b6d9d640 1 -- 172.21.15.86:0/1686613080 learned_addr learned my addr 172.21.15.86:0/1686613080 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:24.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.991+0000 7f67b659c640 1 --2- 172.21.15.86:0/1686613080 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56412338a2e0 0x564123392980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:24.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.991+0000 7f67a77fe640 1 -- 172.21.15.86:0/3224380234 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f67a80f2ea0 con 0x7f67b8102220 2024-04-21T08:41:24.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.991+0000 7f67b6d9d640 1 --2- 172.21.15.86:0/1686613080 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5641233899c0 0x564123389da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:24.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.991+0000 7f67b6d9d640 1 -- 172.21.15.86:0/1686613080 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56412338a2e0 msgr2=0x564123392980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:24.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.991+0000 7f67b6d9d640 1 --2- 172.21.15.86:0/1686613080 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56412338a2e0 0x564123392980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.991+0000 7f67b6d9d640 1 -- 172.21.15.86:0/1686613080 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5641233899c0 msgr2=0x564123389da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:24.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.991+0000 7f67b6d9d640 1 --2- 172.21.15.86:0/1686613080 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5641233899c0 0x564123389da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.991+0000 7f67b6d9d640 1 -- 172.21.15.86:0/1686613080 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5641230ccc90 con 0x564123392ec0 2024-04-21T08:41:24.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.991+0000 7f67b5d9b640 1 --2- 172.21.15.86:0/3224380234 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f67a00a3d20 0x7f67a00a61e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f67b8199ef0 tx=0x7f67ac00a4c0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:24.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.991+0000 7f67b6d9d640 1 --2- 172.21.15.86:0/1686613080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564123392ec0 0x5641233952b0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f67b0009fd0 tx=0x7f67b00099a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9471ab5a938d3b78 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:24.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.991+0000 7f67a4ff9640 1 -- 172.21.15.86:0/1686613080 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b001d070 con 0x564123392ec0 2024-04-21T08:41:24.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.991+0000 7f67a4ff9640 1 -- 172.21.15.86:0/1686613080 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67b00149b0 con 0x564123392ec0 2024-04-21T08:41:24.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.991+0000 7f67a4ff9640 1 -- 172.21.15.86:0/1686613080 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67b0018930 con 0x564123392ec0 2024-04-21T08:41:24.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.992+0000 7f67bf861740 1 -- 172.21.15.86:0/1686613080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564123392ec0 msgr2=0x5641233952b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:24.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.992+0000 7f67bf861740 1 --2- 172.21.15.86:0/1686613080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564123392ec0 0x5641233952b0 secure :-1 s=READY pgs=911 cs=0 l=1 rev1=1 crypto rx=0x7f67b0009fd0 tx=0x7f67b00099a0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.992+0000 7f67bf861740 1 -- 172.21.15.86:0/1686613080 shutdown_connections 2024-04-21T08:41:24.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.992+0000 7f67bf861740 1 --2- 172.21.15.86:0/1686613080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564123392ec0 0x5641233952b0 unknown :-1 s=CLOSED pgs=911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.992+0000 7f67bf861740 1 --2- 172.21.15.86:0/1686613080 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56412338a2e0 0x564123392980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.992+0000 7f67bf861740 1 --2- 172.21.15.86:0/1686613080 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5641233899c0 0x564123389da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:24.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.992+0000 7f67bf861740 1 -- 172.21.15.86:0/1686613080 >> 172.21.15.86:0/1686613080 conn(0x5641233840f0 msgr2=0x5641233844d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:24.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.992+0000 7f67bf861740 1 -- 172.21.15.86:0/1686613080 shutdown_connections 2024-04-21T08:41:24.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.992+0000 7f67bf861740 1 -- 172.21.15.86:0/1686613080 wait complete. 2024-04-21T08:41:25.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.992+0000 7f67bf861740 1 -- start start 2024-04-21T08:41:25.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.993+0000 7f67bf861740 1 Processor -- start 2024-04-21T08:41:25.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.993+0000 7f67bf861740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641233899c0 0x56412338ff90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:25.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.993+0000 7f67bf861740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56412338a2e0 0x5641233904d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:25.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.993+0000 7f67bf861740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564123392ec0 0x564123390a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:25.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.993+0000 7f67bf861740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5641232ed400 con 0x5641233899c0 2024-04-21T08:41:25.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.993+0000 7f67bf861740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5641232a9840 con 0x56412338a2e0 2024-04-21T08:41:25.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.993+0000 7f67bf861740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5641232f2720 con 0x564123392ec0 2024-04-21T08:41:25.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.993+0000 7f67bf861740 10 client.0 _ll_register_callbacks cb 0x5641232faea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:25.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.993+0000 7f67b659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641233899c0 0x56412338ff90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:25.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.993+0000 7f67b659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641233899c0 0x56412338ff90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42356/0 (socket says 172.21.15.86:42356) 2024-04-21T08:41:25.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.993+0000 7f67b659c640 1 -- 172.21.15.86:0/902155323 learned_addr learned my addr 172.21.15.86:0/902155323 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:25.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.993+0000 7f67b6d9d640 1 --2- 172.21.15.86:0/902155323 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564123392ec0 0x564123390a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:25.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.993+0000 7f67b5d9b640 1 --2- 172.21.15.86:0/902155323 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56412338a2e0 0x5641233904d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:25.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.993+0000 7f67b659c640 1 -- 172.21.15.86:0/902155323 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564123392ec0 msgr2=0x564123390a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:25.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.993+0000 7f67b659c640 1 --2- 172.21.15.86:0/902155323 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564123392ec0 0x564123390a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:25.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.993+0000 7f67b659c640 1 -- 172.21.15.86:0/902155323 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56412338a2e0 msgr2=0x5641233904d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:25.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.993+0000 7f67b659c640 1 --2- 172.21.15.86:0/902155323 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56412338a2e0 0x5641233904d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:25.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.993+0000 7f67b659c640 1 -- 172.21.15.86:0/902155323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56412312d420 con 0x5641233899c0 2024-04-21T08:41:25.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.993+0000 7f67b659c640 1 --2- 172.21.15.86:0/902155323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641233899c0 0x56412338ff90 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f67a80be8e0 tx=0x7f67a802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd324ad60cd886d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:25.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.993+0000 7f679dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:25.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.993+0000 7f679dffb640 1 -- 172.21.15.86:0/902155323 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a80be540 con 0x5641233899c0 2024-04-21T08:41:25.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.993+0000 7f679dffb640 1 -- 172.21.15.86:0/902155323 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f67a80bcee0 con 0x5641233899c0 2024-04-21T08:41:25.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.993+0000 7f67bf861740 1 -- 172.21.15.86:0/902155323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67b00149b0 con 0x5641233899c0 2024-04-21T08:41:25.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.994+0000 7f679dffb640 1 -- 172.21.15.86:0/902155323 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f67a8004c50 con 0x5641233899c0 2024-04-21T08:41:25.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.994+0000 7f67bf861740 10 client.16902 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:25.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.994+0000 7f67bf861740 1 -- 172.21.15.86:0/902155323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564122fc0580 con 0x5641233899c0 2024-04-21T08:41:25.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.994+0000 7f679dffb640 1 -- 172.21.15.86:0/902155323 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67a80b6020 con 0x5641233899c0 2024-04-21T08:41:25.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.994+0000 7f679dffb640 1 -- 172.21.15.86:0/902155323 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f67a80df0b0 con 0x5641233899c0 2024-04-21T08:41:25.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.994+0000 7f67bf861740 10 client.16902 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:25.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.994+0000 7f67bf861740 10 client.16902 resolve_mds: validated gid 24482 aka daemon mds.b 2024-04-21T08:41:25.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.994+0000 7f67bf861740 20 client.16902 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:25.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.994+0000 7f67bf861740 1 --2- 172.21.15.86:0/902155323 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5641233a2cd0 0x5641233a5140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:25.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.994+0000 7f67bf861740 4 client.16902 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:25.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.994+0000 7f67bf861740 1 -- 172.21.15.86:0/902155323 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5641232fbb10 con 0x5641233a2cd0 2024-04-21T08:41:25.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.994+0000 7f67b5d9b640 1 --2- 172.21.15.86:0/902155323 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5641233a2cd0 0x5641233a5140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:25.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.995+0000 7f679dffb640 10 client.16902 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:25.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.995+0000 7f67b5d9b640 1 --2- 172.21.15.86:0/902155323 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5641233a2cd0 0x5641233a5140 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a192cb57d27f40b7 server_cookie=a27dc68fd4f987d8 in_seq=0 out_seq=0 2024-04-21T08:41:25.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.995+0000 7f679dffb640 1 -- 172.21.15.86:0/902155323 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5641232fbb10 con 0x5641233a2cd0 2024-04-21T08:41:25.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.995+0000 7f679dffb640 10 client.16902 handle_command_reply: tid=0 2024-04-21T08:41:25.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.995+0000 7f67bf861740 1 client.16902 shutdown 2024-04-21T08:41:25.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.995+0000 7f67bf861740 1 -- 172.21.15.86:0/902155323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641233899c0 msgr2=0x56412338ff90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:25.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.995+0000 7f67bf861740 1 --2- 172.21.15.86:0/902155323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641233899c0 0x56412338ff90 secure :-1 s=READY pgs=912 cs=0 l=1 rev1=1 crypto rx=0x7f67a80be8e0 tx=0x7f67a802bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:25.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.995+0000 7f67bf861740 1 -- 172.21.15.86:0/902155323 shutdown_connections 2024-04-21T08:41:25.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.995+0000 7f67bf861740 1 --2- 172.21.15.86:0/902155323 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5641233a2cd0 0x5641233a5140 crc :-1 s=READY pgs=109 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:25.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.995+0000 7f67bf861740 1 --2- 172.21.15.86:0/902155323 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564123392ec0 0x564123390a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:25.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.995+0000 7f67bf861740 1 --2- 172.21.15.86:0/902155323 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56412338a2e0 0x5641233904d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:25.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.995+0000 7f67bf861740 1 --2- 172.21.15.86:0/902155323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641233899c0 0x56412338ff90 unknown :-1 s=CLOSED pgs=912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:25.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.995+0000 7f67bf861740 1 -- 172.21.15.86:0/902155323 >> 172.21.15.86:0/902155323 conn(0x5641233840f0 msgr2=0x564123385e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:25.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.995+0000 7f679dffb640 0 client.16902 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:25.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.996+0000 7f67bf861740 1 -- 172.21.15.86:0/902155323 shutdown_connections 2024-04-21T08:41:25.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.996+0000 7f67bf861740 1 -- 172.21.15.86:0/902155323 wait complete. 2024-04-21T08:41:25.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:24.996+0000 7f67bf861740 20 client.16902 trim_cache size 0 max 16384 2024-04-21T08:41:25.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.012+0000 7f67bf861740 1 Processor -- start 2024-04-21T08:41:25.017 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.012+0000 7f67bf861740 1 -- start start 2024-04-21T08:41:25.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.012+0000 7f67bf861740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641233899c0 0x5641233926c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:25.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.012+0000 7f67bf861740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56412338a2e0 0x56412339b160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:25.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.012+0000 7f67bf861740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564123392ec0 0x564123385db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:25.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.012+0000 7f67bf861740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f67b81b0e20 con 0x5641233899c0 2024-04-21T08:41:25.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.012+0000 7f67bf861740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f67b81af9b0 con 0x564123392ec0 2024-04-21T08:41:25.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.012+0000 7f67bf861740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f67b81b0080 con 0x56412338a2e0 2024-04-21T08:41:25.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.013+0000 7f67b659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641233899c0 0x5641233926c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:25.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.013+0000 7f67b659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641233899c0 0x5641233926c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42370/0 (socket says 172.21.15.86:42370) 2024-04-21T08:41:25.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.013+0000 7f67b659c640 1 -- 172.21.15.86:0/3335238595 learned_addr learned my addr 172.21.15.86:0/3335238595 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:25.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.013+0000 7f67b6d9d640 1 --2- 172.21.15.86:0/3335238595 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564123392ec0 0x564123385db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:25.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.013+0000 7f67b659c640 1 -- 172.21.15.86:0/3335238595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56412338a2e0 msgr2=0x56412339b160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:25.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.013+0000 7f67b659c640 1 --2- 172.21.15.86:0/3335238595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56412338a2e0 0x56412339b160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:25.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.013+0000 7f67b659c640 1 -- 172.21.15.86:0/3335238595 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564123392ec0 msgr2=0x564123385db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:25.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.013+0000 7f67b659c640 1 --2- 172.21.15.86:0/3335238595 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564123392ec0 0x564123385db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:25.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.013+0000 7f67b659c640 1 -- 172.21.15.86:0/3335238595 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56412338a100 con 0x5641233899c0 2024-04-21T08:41:25.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.013+0000 7f67b6d9d640 1 --2- 172.21.15.86:0/3335238595 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564123392ec0 0x564123385db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:25.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.013+0000 7f67b659c640 1 --2- 172.21.15.86:0/3335238595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641233899c0 0x5641233926c0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f67a80092a0 tx=0x7f67a80f31d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4518f68bbf61f44c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:25.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.013+0000 7f679ffff640 1 -- 172.21.15.86:0/3335238595 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a80be9b0 con 0x5641233899c0 2024-04-21T08:41:25.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.013+0000 7f679ffff640 1 -- 172.21.15.86:0/3335238595 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67a80b6020 con 0x5641233899c0 2024-04-21T08:41:25.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.013+0000 7f679ffff640 1 -- 172.21.15.86:0/3335238595 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67a80c7850 con 0x5641233899c0 2024-04-21T08:41:25.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.014+0000 7f67bf861740 1 -- 172.21.15.86:0/3335238595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641233899c0 msgr2=0x5641233926c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:25.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.014+0000 7f67bf861740 1 --2- 172.21.15.86:0/3335238595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641233899c0 0x5641233926c0 secure :-1 s=READY pgs=913 cs=0 l=1 rev1=1 crypto rx=0x7f67a80092a0 tx=0x7f67a80f31d0 comp rx=0 tx=0).stop 2024-04-21T08:41:25.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.014+0000 7f67bf861740 1 -- 172.21.15.86:0/3335238595 shutdown_connections 2024-04-21T08:41:25.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.014+0000 7f67bf861740 1 --2- 172.21.15.86:0/3335238595 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564123392ec0 0x564123385db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:25.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.014+0000 7f67bf861740 1 --2- 172.21.15.86:0/3335238595 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56412338a2e0 0x56412339b160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:25.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.014+0000 7f67bf861740 1 --2- 172.21.15.86:0/3335238595 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5641233899c0 0x5641233926c0 unknown :-1 s=CLOSED pgs=913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:25.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.014+0000 7f67bf861740 1 -- 172.21.15.86:0/3335238595 >> 172.21.15.86:0/3335238595 conn(0x5641233840f0 msgr2=0x564123382940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:25.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.014+0000 7f67bf861740 1 -- 172.21.15.86:0/3335238595 shutdown_connections 2024-04-21T08:41:25.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.014+0000 7f67bf861740 1 -- 172.21.15.86:0/3335238595 wait complete. 2024-04-21T08:41:25.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.014+0000 7f67bf861740 1 -- start start 2024-04-21T08:41:25.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.014+0000 7f67bf861740 1 Processor -- start 2024-04-21T08:41:25.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.014+0000 7f67bf861740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5641233899c0 0x56412338dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:25.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.014+0000 7f67bf861740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56412338a2e0 0x56412338c220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:25.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.014+0000 7f67bf861740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564123392ec0 0x56412338c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:25.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.014+0000 7f67bf861740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f67b81ad2a0 con 0x56412338a2e0 2024-04-21T08:41:25.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.014+0000 7f67bf861740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f67b81b0750 con 0x564123392ec0 2024-04-21T08:41:25.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.014+0000 7f67bf861740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f67b81ad930 con 0x5641233899c0 2024-04-21T08:41:25.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.014+0000 7f67bf861740 10 client.0 _ll_register_callbacks cb 0x7f67a8005670 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:25.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.014+0000 7f67b5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56412338a2e0 0x56412338c220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:25.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.014+0000 7f67b5d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56412338a2e0 0x56412338c220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42386/0 (socket says 172.21.15.86:42386) 2024-04-21T08:41:25.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.014+0000 7f67b5d9b640 1 -- 172.21.15.86:0/4043814105 learned_addr learned my addr 172.21.15.86:0/4043814105 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:25.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.014+0000 7f67b6d9d640 1 --2- 172.21.15.86:0/4043814105 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564123392ec0 0x56412338c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:25.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.014+0000 7f67b659c640 1 --2- 172.21.15.86:0/4043814105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5641233899c0 0x56412338dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:25.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.014+0000 7f67b5d9b640 1 -- 172.21.15.86:0/4043814105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5641233899c0 msgr2=0x56412338dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:25.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.015+0000 7f67b5d9b640 1 --2- 172.21.15.86:0/4043814105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5641233899c0 0x56412338dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:25.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.015+0000 7f67b5d9b640 1 -- 172.21.15.86:0/4043814105 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564123392ec0 msgr2=0x56412338c7a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:25.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.015+0000 7f67b5d9b640 1 --2- 172.21.15.86:0/4043814105 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564123392ec0 0x56412338c7a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:25.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.015+0000 7f67b5d9b640 1 -- 172.21.15.86:0/4043814105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56412338cfd0 con 0x56412338a2e0 2024-04-21T08:41:25.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.015+0000 7f67b6d9d640 1 --2- 172.21.15.86:0/4043814105 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564123392ec0 0x56412338c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:25.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.015+0000 7f67b659c640 1 --2- 172.21.15.86:0/4043814105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5641233899c0 0x56412338dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:25.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.015+0000 7f67b5d9b640 1 --2- 172.21.15.86:0/4043814105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56412338a2e0 0x56412338c220 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f67ac0058c0 tx=0x7f67ac0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=25608e0336953f81 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:25.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.015+0000 7f679dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:25.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.015+0000 7f679dffb640 1 -- 172.21.15.86:0/4043814105 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f67ac029070 con 0x56412338a2e0 2024-04-21T08:41:25.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.015+0000 7f679dffb640 1 -- 172.21.15.86:0/4043814105 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f67ac025250 con 0x56412338a2e0 2024-04-21T08:41:25.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.015+0000 7f67bf861740 1 -- 172.21.15.86:0/4043814105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f67a80b6020 con 0x56412338a2e0 2024-04-21T08:41:25.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.015+0000 7f679dffb640 1 -- 172.21.15.86:0/4043814105 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f67ac035050 con 0x56412338a2e0 2024-04-21T08:41:25.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.015+0000 7f67bf861740 10 client.16908 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:25.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.015+0000 7f67bf861740 1 -- 172.21.15.86:0/4043814105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56412340e310 con 0x56412338a2e0 2024-04-21T08:41:25.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.015+0000 7f679dffb640 1 -- 172.21.15.86:0/4043814105 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f67a80b6020 con 0x56412338a2e0 2024-04-21T08:41:25.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.015+0000 7f679dffb640 1 -- 172.21.15.86:0/4043814105 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f67ac010960 con 0x56412338a2e0 2024-04-21T08:41:25.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.015+0000 7f67bf861740 10 client.16908 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:25.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.015+0000 7f67bf861740 10 client.16908 resolve_mds: validated gid 24482 aka daemon mds.b 2024-04-21T08:41:25.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.015+0000 7f67bf861740 20 client.16908 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:25.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.015+0000 7f67bf861740 1 --2- 172.21.15.86:0/4043814105 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56412340eb10 0x564123410f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:25.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.015+0000 7f67bf861740 4 client.16908 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 30.0, "expiration": 81.9, "await": true}] 2024-04-21T08:41:25.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.015+0000 7f67bf861740 1 -- 172.21.15.86:0/4043814105 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 30.0, "expiration": 81.9, "await": true}) -- 0x5641232f5980 con 0x56412340eb10 2024-04-21T08:41:25.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.016+0000 7f67b659c640 1 --2- 172.21.15.86:0/4043814105 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56412340eb10 0x564123410f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:25.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.016+0000 7f67b659c640 1 --2- 172.21.15.86:0/4043814105 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56412340eb10 0x564123410f30 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db2c6fd41815823c server_cookie=3c406d33e281ee22 in_seq=0 out_seq=0 2024-04-21T08:41:25.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.016+0000 7f679dffb640 10 client.16908 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:25.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.029+0000 7f679dffb640 1 -- 172.21.15.86:0/4043814105 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x5641232f5980 con 0x56412340eb10 2024-04-21T08:41:25.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.029+0000 7f679dffb640 10 client.16908 handle_command_reply: tid=0 2024-04-21T08:41:25.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.029+0000 7f67bf861740 1 client.16908 shutdown 2024-04-21T08:41:25.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.029+0000 7f67bf861740 1 -- 172.21.15.86:0/4043814105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56412338a2e0 msgr2=0x56412338c220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:25.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.029+0000 7f67bf861740 1 --2- 172.21.15.86:0/4043814105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56412338a2e0 0x56412338c220 secure :-1 s=READY pgs=914 cs=0 l=1 rev1=1 crypto rx=0x7f67ac0058c0 tx=0x7f67ac0046c0 comp rx=0 tx=0).stop 2024-04-21T08:41:25.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.029+0000 7f67bf861740 1 -- 172.21.15.86:0/4043814105 shutdown_connections 2024-04-21T08:41:25.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.029+0000 7f67bf861740 1 --2- 172.21.15.86:0/4043814105 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56412340eb10 0x564123410f30 crc :-1 s=READY pgs=110 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:25.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.029+0000 7f67bf861740 1 --2- 172.21.15.86:0/4043814105 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564123392ec0 0x56412338c7a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:25.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.029+0000 7f679dffb640 0 client.16908 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:25.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.029+0000 7f67bf861740 1 --2- 172.21.15.86:0/4043814105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56412338a2e0 0x56412338c220 unknown :-1 s=CLOSED pgs=914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:25.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.029+0000 7f67bf861740 1 --2- 172.21.15.86:0/4043814105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5641233899c0 0x56412338dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:25.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.029+0000 7f67bf861740 1 -- 172.21.15.86:0/4043814105 >> 172.21.15.86:0/4043814105 conn(0x5641233840f0 msgr2=0x5641233831d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:25.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.029+0000 7f67bf861740 1 -- 172.21.15.86:0/4043814105 shutdown_connections 2024-04-21T08:41:25.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.029+0000 7f67bf861740 1 -- 172.21.15.86:0/4043814105 wait complete. 2024-04-21T08:41:25.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.029+0000 7f67bf861740 20 client.16908 trim_cache size 0 max 16384 2024-04-21T08:41:25.031 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:41:25.031 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 29, 2024-04-21T08:41:25.031 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24482, 2024-04-21T08:41:25.031 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 6, 2024-04-21T08:41:25.031 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-04-21T08:41:25.031 INFO:teuthology.orchestra.run.smithi086.stdout: "ea3fd78e": { 2024-04-21T08:41:25.032 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 6, 2024-04-21T08:41:25.032 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-04-21T08:41:25.032 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-21T08:41:25.032 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-04-21T08:41:25.032 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-21T08:41:25.032 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:41:25.032 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 30.0, 2024-04-21T08:41:25.032 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 81.9, 2024-04-21T08:41:25.032 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-04-21T08:41:25.032 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-04-21T08:41:25.032 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-04-21T08:41:25.032 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-21T08:41:25.032 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-04-21T08:41:25.032 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-21T08:41:25.033 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:41:25.033 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:41:25.033 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:41:25.033 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:41:25.033 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:41:25.033 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:41:25.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.030+0000 7f67bcc77640 1 -- 172.21.15.86:0/3224380234 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f67a00a3d20 msgr2=0x7f67a00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:25.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.030+0000 7f67bcc77640 1 --2- 172.21.15.86:0/3224380234 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f67a00a3d20 0x7f67a00a61e0 secure :-1 s=READY pgs=135 cs=0 l=1 rev1=1 crypto rx=0x7f67b8199ef0 tx=0x7f67ac00a4c0 comp rx=0 tx=0).stop 2024-04-21T08:41:25.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.030+0000 7f67bcc77640 1 -- 172.21.15.86:0/3224380234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67b8102220 msgr2=0x7f67b819e5e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:25.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.030+0000 7f67bcc77640 1 --2- 172.21.15.86:0/3224380234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67b8102220 0x7f67b819e5e0 secure :-1 s=READY pgs=910 cs=0 l=1 rev1=1 crypto rx=0x7f67a8005ab0 tx=0x7f67a802ca90 comp rx=0 tx=0).stop 2024-04-21T08:41:25.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.030+0000 7f67bcc77640 1 -- 172.21.15.86:0/3224380234 shutdown_connections 2024-04-21T08:41:25.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.030+0000 7f67bcc77640 1 --2- 172.21.15.86:0/3224380234 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f67a00a3d20 0x7f67a00a61e0 unknown :-1 s=CLOSED pgs=135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:25.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.030+0000 7f67bcc77640 1 --2- 172.21.15.86:0/3224380234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f67b81986a0 0x7f67b8198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:25.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.030+0000 7f67bcc77640 1 --2- 172.21.15.86:0/3224380234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f67b81072f0 0x7f67b819eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:25.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.030+0000 7f67bcc77640 1 --2- 172.21.15.86:0/3224380234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f67b8102220 0x7f67b819e5e0 unknown :-1 s=CLOSED pgs=910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:25.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.030+0000 7f67bcc77640 1 -- 172.21.15.86:0/3224380234 >> 172.21.15.86:0/3224380234 conn(0x7f67b80fda80 msgr2=0x7f67b80fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:25.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.030+0000 7f67bcc77640 1 -- 172.21.15.86:0/3224380234 shutdown_connections 2024-04-21T08:41:25.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:25.030+0000 7f67bcc77640 1 -- 172.21.15.86:0/3224380234 wait complete. 2024-04-21T08:41:25.041 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced, set_id: ea3fd78e, quiesce duration: 21.9 2024-04-21T08:41:25.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:25 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1340617512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:41:25.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:25 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3222091684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:41:25.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:25 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1340617512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:41:25.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:25 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3222091684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:41:25.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:25 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1340617512' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:41:25.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:25 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3222091684' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:41:26.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:26 smithi086 ceph-mon[27939]: pgmap v82: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-21T08:41:26.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:26 smithi088 ceph-mon[31259]: pgmap v82: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-21T08:41:26.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:26 smithi179 ceph-mon[31829]: pgmap v82: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-21T08:41:28.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:28 smithi086 ceph-mon[27939]: pgmap v83: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-21T08:41:28.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:28 smithi088 ceph-mon[31259]: pgmap v83: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-21T08:41:28.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:28 smithi179 ceph-mon[31829]: pgmap v83: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 4.2 KiB/s wr, 10 op/s 2024-04-21T08:41:30.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:30 smithi086 ceph-mon[27939]: pgmap v84: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.1 KiB/s wr, 15 op/s 2024-04-21T08:41:30.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:30 smithi088 ceph-mon[31259]: pgmap v84: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.1 KiB/s wr, 15 op/s 2024-04-21T08:41:30.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:30 smithi179 ceph-mon[31829]: pgmap v84: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 7.1 KiB/s wr, 15 op/s 2024-04-21T08:41:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.598+0000 7fc2f7c62640 1 -- 172.21.15.86:0/163765822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f006e470 msgr2=0x7fc2f011b0b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.598+0000 7fc2f7c62640 1 --2- 172.21.15.86:0/163765822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f006e470 0x7fc2f011b0b0 secure :-1 s=READY pgs=915 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0009900 tx=0x7fc2e002bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.598+0000 7fc2f7c62640 1 -- 172.21.15.86:0/163765822 shutdown_connections 2024-04-21T08:41:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.598+0000 7fc2f7c62640 1 --2- 172.21.15.86:0/163765822 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2f011b5f0 0x7fc2f011d9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.599+0000 7fc2f7c62640 1 --2- 172.21.15.86:0/163765822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f006e470 0x7fc2f011b0b0 unknown :-1 s=CLOSED pgs=915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.599+0000 7fc2f7c62640 1 --2- 172.21.15.86:0/163765822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2f006db50 0x7fc2f006df30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.599+0000 7fc2f7c62640 1 -- 172.21.15.86:0/163765822 >> 172.21.15.86:0/163765822 conn(0x7fc2f006d1b0 msgr2=0x7fc2f010af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.599+0000 7fc2f7c62640 1 -- 172.21.15.86:0/163765822 shutdown_connections 2024-04-21T08:41:31.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.599+0000 7fc2f7c62640 1 -- 172.21.15.86:0/163765822 wait complete. 2024-04-21T08:41:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.599+0000 7fc2f7c62640 1 Processor -- start 2024-04-21T08:41:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.599+0000 7fc2f7c62640 1 -- start start 2024-04-21T08:41:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.599+0000 7fc2f7c62640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2f006db50 0x7fc2f01a7490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.599+0000 7fc2f7c62640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f006e470 0x7fc2f01a79d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.599+0000 7fc2f7c62640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2f011b5f0 0x7fc2f01abd50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.599+0000 7fc2f7c62640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f0120f70 con 0x7fc2f006e470 2024-04-21T08:41:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.599+0000 7fc2f7c62640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f0120df0 con 0x7fc2f006db50 2024-04-21T08:41:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.599+0000 7fc2f7c62640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f01210f0 con 0x7fc2f011b5f0 2024-04-21T08:41:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.600+0000 7fc2f51fd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f006e470 0x7fc2f01a79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.600+0000 7fc2f51fd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f006e470 0x7fc2f01a79d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48576/0 (socket says 172.21.15.86:48576) 2024-04-21T08:41:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.600+0000 7fc2f51fd640 1 -- 172.21.15.86:0/779031120 learned_addr learned my addr 172.21.15.86:0/779031120 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.600+0000 7fc2f59fe640 1 --2- 172.21.15.86:0/779031120 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2f006db50 0x7fc2f01a7490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.600+0000 7fc2f61ff640 1 --2- 172.21.15.86:0/779031120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2f011b5f0 0x7fc2f01abd50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:31.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.600+0000 7fc2f51fd640 1 -- 172.21.15.86:0/779031120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2f011b5f0 msgr2=0x7fc2f01abd50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.600+0000 7fc2f51fd640 1 --2- 172.21.15.86:0/779031120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2f011b5f0 0x7fc2f01abd50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.600+0000 7fc2f51fd640 1 -- 172.21.15.86:0/779031120 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2f006db50 msgr2=0x7fc2f01a7490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.600+0000 7fc2f51fd640 1 --2- 172.21.15.86:0/779031120 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2f006db50 0x7fc2f01a7490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.600+0000 7fc2f51fd640 1 -- 172.21.15.86:0/779031120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc2f01ac480 con 0x7fc2f006e470 2024-04-21T08:41:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.600+0000 7fc2f51fd640 1 --2- 172.21.15.86:0/779031120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f006e470 0x7fc2f01a79d0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0005940 tx=0x7fc2e00045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.600+0000 7fc2e6ffd640 1 -- 172.21.15.86:0/779031120 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e0043020 con 0x7fc2f006e470 2024-04-21T08:41:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.600+0000 7fc2f7c62640 1 -- 172.21.15.86:0/779031120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc2f01ac710 con 0x7fc2f006e470 2024-04-21T08:41:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.600+0000 7fc2e6ffd640 1 -- 172.21.15.86:0/779031120 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2e002caf0 con 0x7fc2f006e470 2024-04-21T08:41:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.600+0000 7fc2e6ffd640 1 -- 172.21.15.86:0/779031120 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e0003af0 con 0x7fc2f006e470 2024-04-21T08:41:31.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.600+0000 7fc2f7c62640 1 -- 172.21.15.86:0/779031120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc2f01b4000 con 0x7fc2f006e470 2024-04-21T08:41:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.603+0000 7fc2e6ffd640 1 -- 172.21.15.86:0/779031120 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc2e0046020 con 0x7fc2f006e470 2024-04-21T08:41:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.603+0000 7fc2fa84b740 1 Processor -- start 2024-04-21T08:41:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.603+0000 7fc2fa84b740 1 -- start start 2024-04-21T08:41:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.603+0000 7fc2fa84b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56268ac508d0 0x56268ac4f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.603+0000 7fc2fa84b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56268ac4fb60 0x56268ac4ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.603+0000 7fc2fa84b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56268ac590e0 0x56268ac5b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.603+0000 7fc2fa84b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56268abb48d0 con 0x56268ac508d0 2024-04-21T08:41:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.604+0000 7fc2fa84b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x56268ab51e80 con 0x56268ac590e0 2024-04-21T08:41:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.604+0000 7fc2fa84b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56268abb0890 con 0x56268ac4fb60 2024-04-21T08:41:31.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.604+0000 7fc2e6ffd640 1 --2- 172.21.15.86:0/779031120 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc2dc0a3dc0 0x7fc2dc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.604+0000 7fc2f61ff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56268ac508d0 0x56268ac4f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.604+0000 7fc2f61ff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56268ac508d0 0x56268ac4f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48578/0 (socket says 172.21.15.86:48578) 2024-04-21T08:41:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.604+0000 7fc2f61ff640 1 -- 172.21.15.86:0/2261113059 learned_addr learned my addr 172.21.15.86:0/2261113059 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.604+0000 7fc2f61ff640 1 --2- 172.21.15.86:0/779031120 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc2dc0a3dc0 0x7fc2dc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.604+0000 7fc2e6ffd640 1 -- 172.21.15.86:0/779031120 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fc2e00f2ba0 con 0x7fc2f006e470 2024-04-21T08:41:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.604+0000 7fc2f51fd640 1 --2- 172.21.15.86:0/2261113059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56268ac590e0 0x56268ac5b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.604+0000 7fc2f59fe640 1 --2- 172.21.15.86:0/2261113059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56268ac4fb60 0x56268ac4ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.604+0000 7fc2f61ff640 1 -- 172.21.15.86:0/2261113059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56268ac4fb60 msgr2=0x56268ac4ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.604+0000 7fc2f61ff640 1 --2- 172.21.15.86:0/2261113059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56268ac4fb60 0x56268ac4ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.604+0000 7fc2f61ff640 1 -- 172.21.15.86:0/2261113059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56268ac590e0 msgr2=0x56268ac5b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.604+0000 7fc2f61ff640 1 --2- 172.21.15.86:0/2261113059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56268ac590e0 0x56268ac5b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.604+0000 7fc2f61ff640 1 -- 172.21.15.86:0/2261113059 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56268ab73320 con 0x56268ac508d0 2024-04-21T08:41:31.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.604+0000 7fc2f59fe640 1 --2- 172.21.15.86:0/2261113059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56268ac4fb60 0x56268ac4ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.604+0000 7fc2f61ff640 1 --2- 172.21.15.86:0/2261113059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56268ac508d0 0x56268ac4f620 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8011da0 tx=0x7fc2e800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d2c79f43145caf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.604+0000 7fc2f61ff640 1 --2- 172.21.15.86:0/779031120 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc2dc0a3dc0 0x7fc2dc0a6280 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc2e800fc90 tx=0x7fc2e801e000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.604+0000 7fc2dbfff640 1 -- 172.21.15.86:0/2261113059 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e8022070 con 0x56268ac508d0 2024-04-21T08:41:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.604+0000 7fc2dbfff640 1 -- 172.21.15.86:0/2261113059 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2e8009dc0 con 0x56268ac508d0 2024-04-21T08:41:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.604+0000 7fc2dbfff640 1 -- 172.21.15.86:0/2261113059 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e8004050 con 0x56268ac508d0 2024-04-21T08:41:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.605+0000 7fc2fa84b740 1 -- 172.21.15.86:0/2261113059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56268ac508d0 msgr2=0x56268ac4f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.605+0000 7fc2fa84b740 1 --2- 172.21.15.86:0/2261113059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56268ac508d0 0x56268ac4f620 secure :-1 s=READY pgs=917 cs=0 l=1 rev1=1 crypto rx=0x7fc2e8011da0 tx=0x7fc2e800f790 comp rx=0 tx=0).stop 2024-04-21T08:41:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.605+0000 7fc2fa84b740 1 -- 172.21.15.86:0/2261113059 shutdown_connections 2024-04-21T08:41:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.605+0000 7fc2fa84b740 1 --2- 172.21.15.86:0/2261113059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56268ac590e0 0x56268ac5b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.605+0000 7fc2fa84b740 1 --2- 172.21.15.86:0/2261113059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56268ac4fb60 0x56268ac4ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.605+0000 7fc2fa84b740 1 --2- 172.21.15.86:0/2261113059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56268ac508d0 0x56268ac4f620 unknown :-1 s=CLOSED pgs=917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.605+0000 7fc2fa84b740 1 -- 172.21.15.86:0/2261113059 >> 172.21.15.86:0/2261113059 conn(0x56268ac16c90 msgr2=0x56268ac4a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.605+0000 7fc2fa84b740 1 -- 172.21.15.86:0/2261113059 shutdown_connections 2024-04-21T08:41:31.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.605+0000 7fc2fa84b740 1 -- 172.21.15.86:0/2261113059 wait complete. 2024-04-21T08:41:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.606+0000 7fc2fa84b740 1 -- start start 2024-04-21T08:41:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.606+0000 7fc2fa84b740 1 Processor -- start 2024-04-21T08:41:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.606+0000 7fc2fa84b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56268ac4fb60 0x56268ac58b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.606+0000 7fc2fa84b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56268ac508d0 0x56268ac55e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.606+0000 7fc2fa84b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56268ac590e0 0x56268ac564a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.606+0000 7fc2fa84b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56268ac5bdb0 con 0x56268ac4fb60 2024-04-21T08:41:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.606+0000 7fc2fa84b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x56268ab3ceb0 con 0x56268ac590e0 2024-04-21T08:41:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.606+0000 7fc2fa84b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56268ac5bf30 con 0x56268ac508d0 2024-04-21T08:41:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.606+0000 7fc2fa84b740 10 client.0 _ll_register_callbacks cb 0x56268ab69b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.606+0000 7fc2f59fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56268ac4fb60 0x56268ac58b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.606+0000 7fc2f59fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56268ac4fb60 0x56268ac58b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48582/0 (socket says 172.21.15.86:48582) 2024-04-21T08:41:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.606+0000 7fc2f59fe640 1 -- 172.21.15.86:0/2573278564 learned_addr learned my addr 172.21.15.86:0/2573278564 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.607+0000 7fc2f61ff640 1 --2- 172.21.15.86:0/2573278564 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56268ac590e0 0x56268ac564a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:31.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.607+0000 7fc2f59fe640 1 -- 172.21.15.86:0/2573278564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56268ac508d0 msgr2=0x56268ac55e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.607+0000 7fc2f59fe640 1 --2- 172.21.15.86:0/2573278564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56268ac508d0 0x56268ac55e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.607+0000 7fc2f59fe640 1 -- 172.21.15.86:0/2573278564 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56268ac590e0 msgr2=0x56268ac564a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.607+0000 7fc2f59fe640 1 --2- 172.21.15.86:0/2573278564 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56268ac590e0 0x56268ac564a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.607+0000 7fc2f59fe640 1 -- 172.21.15.86:0/2573278564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56268a8c7210 con 0x56268ac4fb60 2024-04-21T08:41:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.607+0000 7fc2f59fe640 1 --2- 172.21.15.86:0/2573278564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56268ac4fb60 0x56268ac58b00 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec00b910 tx=0x7fc2ec004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f18e6221b3074 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.607+0000 7fc2d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.607+0000 7fc2d97fa640 1 -- 172.21.15.86:0/2573278564 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2ec005020 con 0x56268ac4fb60 2024-04-21T08:41:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.607+0000 7fc2d97fa640 1 -- 172.21.15.86:0/2573278564 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fc2ec0051c0 con 0x56268ac4fb60 2024-04-21T08:41:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.607+0000 7fc2fa84b740 1 -- 172.21.15.86:0/2573278564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2e8009dc0 con 0x56268ac4fb60 2024-04-21T08:41:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.607+0000 7fc2d97fa640 1 -- 172.21.15.86:0/2573278564 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc2ec023050 con 0x56268ac4fb60 2024-04-21T08:41:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.607+0000 7fc2fa84b740 10 client.16920 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.607+0000 7fc2fa84b740 1 -- 172.21.15.86:0/2573278564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56268a993620 con 0x56268ac4fb60 2024-04-21T08:41:31.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.607+0000 7fc2d97fa640 1 -- 172.21.15.86:0/2573278564 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2e8009dc0 con 0x56268ac4fb60 2024-04-21T08:41:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.608+0000 7fc2d97fa640 1 -- 172.21.15.86:0/2573278564 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc2ec012570 con 0x56268ac4fb60 2024-04-21T08:41:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.608+0000 7fc2fa84b740 10 client.16920 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.608+0000 7fc2fa84b740 10 client.16920 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.608+0000 7fc2fa84b740 20 client.16920 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.608+0000 7fc2fa84b740 1 --2- 172.21.15.86:0/2573278564 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56268ac690e0 0x56268ac6b4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.608+0000 7fc2fa84b740 4 client.16920 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.608+0000 7fc2fa84b740 1 -- 172.21.15.86:0/2573278564 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56268abc8880 con 0x56268ac690e0 2024-04-21T08:41:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.608+0000 7fc2f51fd640 1 --2- 172.21.15.86:0/2573278564 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56268ac690e0 0x56268ac6b4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.608+0000 7fc2d97fa640 10 client.16920 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.608+0000 7fc2f51fd640 1 --2- 172.21.15.86:0/2573278564 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56268ac690e0 0x56268ac6b4f0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34ae1f2cd449c358 server_cookie=75ce2d948eedeb82 in_seq=0 out_seq=0 2024-04-21T08:41:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.609+0000 7fc2d97fa640 1 -- 172.21.15.86:0/2573278564 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56268abc8880 con 0x56268ac690e0 2024-04-21T08:41:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.609+0000 7fc2d97fa640 10 client.16920 handle_command_reply: tid=0 2024-04-21T08:41:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.609+0000 7fc2fa84b740 1 client.16920 shutdown 2024-04-21T08:41:31.614 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.609+0000 7fc2fa84b740 1 -- 172.21.15.86:0/2573278564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56268ac4fb60 msgr2=0x56268ac58b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.609+0000 7fc2fa84b740 1 --2- 172.21.15.86:0/2573278564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56268ac4fb60 0x56268ac58b00 secure :-1 s=READY pgs=918 cs=0 l=1 rev1=1 crypto rx=0x7fc2ec00b910 tx=0x7fc2ec004610 comp rx=0 tx=0).stop 2024-04-21T08:41:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.609+0000 7fc2fa84b740 1 -- 172.21.15.86:0/2573278564 shutdown_connections 2024-04-21T08:41:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.609+0000 7fc2fa84b740 1 --2- 172.21.15.86:0/2573278564 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56268ac690e0 0x56268ac6b4f0 crc :-1 s=READY pgs=111 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.609+0000 7fc2fa84b740 1 --2- 172.21.15.86:0/2573278564 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56268ac590e0 0x56268ac564a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.609+0000 7fc2fa84b740 1 --2- 172.21.15.86:0/2573278564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56268ac508d0 0x56268ac55e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.609+0000 7fc2fa84b740 1 --2- 172.21.15.86:0/2573278564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56268ac4fb60 0x56268ac58b00 unknown :-1 s=CLOSED pgs=918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.609+0000 7fc2d97fa640 0 client.16920 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.609+0000 7fc2fa84b740 1 -- 172.21.15.86:0/2573278564 >> 172.21.15.86:0/2573278564 conn(0x56268ac16c90 msgr2=0x56268ac4c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.609+0000 7fc2fa84b740 1 -- 172.21.15.86:0/2573278564 shutdown_connections 2024-04-21T08:41:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.609+0000 7fc2fa84b740 1 -- 172.21.15.86:0/2573278564 wait complete. 2024-04-21T08:41:31.615 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.609+0000 7fc2fa84b740 20 client.16920 trim_cache size 0 max 16384 2024-04-21T08:41:31.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.624+0000 7fc2fa84b740 1 Processor -- start 2024-04-21T08:41:31.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.624+0000 7fc2fa84b740 1 -- start start 2024-04-21T08:41:31.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.624+0000 7fc2fa84b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56268ac4fb60 0x56268ac64cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:31.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.624+0000 7fc2fa84b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56268ac508d0 0x56268ac585c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:31.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.624+0000 7fc2fa84b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56268ac590e0 0x56268ac58b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:31.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.624+0000 7fc2fa84b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f01bdf90 con 0x56268ac508d0 2024-04-21T08:41:31.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.624+0000 7fc2fa84b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f01bfad0 con 0x56268ac4fb60 2024-04-21T08:41:31.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.624+0000 7fc2fa84b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f01be660 con 0x56268ac590e0 2024-04-21T08:41:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.624+0000 7fc2f51fd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56268ac508d0 0x56268ac585c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.624+0000 7fc2f51fd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56268ac508d0 0x56268ac585c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48590/0 (socket says 172.21.15.86:48590) 2024-04-21T08:41:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.624+0000 7fc2f51fd640 1 -- 172.21.15.86:0/4160925212 learned_addr learned my addr 172.21.15.86:0/4160925212 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.624+0000 7fc2f51fd640 1 -- 172.21.15.86:0/4160925212 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56268ac590e0 msgr2=0x56268ac58b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.624+0000 7fc2f59fe640 1 --2- 172.21.15.86:0/4160925212 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56268ac4fb60 0x56268ac64cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.624+0000 7fc2f51fd640 1 --2- 172.21.15.86:0/4160925212 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56268ac590e0 0x56268ac58b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.624+0000 7fc2f51fd640 1 -- 172.21.15.86:0/4160925212 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56268ac4fb60 msgr2=0x56268ac64cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.624+0000 7fc2f51fd640 1 --2- 172.21.15.86:0/4160925212 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56268ac4fb60 0x56268ac64cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.624+0000 7fc2f51fd640 1 -- 172.21.15.86:0/4160925212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56268ac4ee30 con 0x56268ac508d0 2024-04-21T08:41:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.624+0000 7fc2f59fe640 1 --2- 172.21.15.86:0/4160925212 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56268ac4fb60 0x56268ac64cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.625+0000 7fc2f51fd640 1 --2- 172.21.15.86:0/4160925212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56268ac508d0 0x56268ac585c0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0006010 tx=0x7fc2e004a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d3270f50062c7e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.625+0000 7fc2db7fe640 1 -- 172.21.15.86:0/4160925212 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e002cca0 con 0x56268ac508d0 2024-04-21T08:41:31.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.625+0000 7fc2db7fe640 1 -- 172.21.15.86:0/4160925212 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2e00befe0 con 0x56268ac508d0 2024-04-21T08:41:31.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.625+0000 7fc2db7fe640 1 -- 172.21.15.86:0/4160925212 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e0003c90 con 0x56268ac508d0 2024-04-21T08:41:31.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.625+0000 7fc2fa84b740 1 -- 172.21.15.86:0/4160925212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56268ac508d0 msgr2=0x56268ac585c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:31.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.625+0000 7fc2fa84b740 1 --2- 172.21.15.86:0/4160925212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56268ac508d0 0x56268ac585c0 secure :-1 s=READY pgs=919 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0006010 tx=0x7fc2e004a040 comp rx=0 tx=0).stop 2024-04-21T08:41:31.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.625+0000 7fc2fa84b740 1 -- 172.21.15.86:0/4160925212 shutdown_connections 2024-04-21T08:41:31.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.625+0000 7fc2fa84b740 1 --2- 172.21.15.86:0/4160925212 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56268ac590e0 0x56268ac58b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:31.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.625+0000 7fc2fa84b740 1 --2- 172.21.15.86:0/4160925212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56268ac508d0 0x56268ac585c0 unknown :-1 s=CLOSED pgs=919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:31.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.625+0000 7fc2fa84b740 1 --2- 172.21.15.86:0/4160925212 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56268ac4fb60 0x56268ac64cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:31.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.625+0000 7fc2fa84b740 1 -- 172.21.15.86:0/4160925212 >> 172.21.15.86:0/4160925212 conn(0x56268ac16c90 msgr2=0x56268ac51030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:31.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.625+0000 7fc2fa84b740 1 -- 172.21.15.86:0/4160925212 shutdown_connections 2024-04-21T08:41:31.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.625+0000 7fc2fa84b740 1 -- 172.21.15.86:0/4160925212 wait complete. 2024-04-21T08:41:31.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.626+0000 7fc2fa84b740 1 -- start start 2024-04-21T08:41:31.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.626+0000 7fc2fa84b740 1 Processor -- start 2024-04-21T08:41:31.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.626+0000 7fc2fa84b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56268ac4fb60 0x56268ac59ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:31.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.626+0000 7fc2fa84b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56268ac508d0 0x56268ac5a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:31.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.626+0000 7fc2fa84b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56268ac590e0 0x56268ac601c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:31.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.626+0000 7fc2fa84b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f01bf400 con 0x56268ac508d0 2024-04-21T08:41:31.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.626+0000 7fc2fa84b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f01bed30 con 0x56268ac590e0 2024-04-21T08:41:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.626+0000 7fc2fa84b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc2f01bbf50 con 0x56268ac4fb60 2024-04-21T08:41:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.626+0000 7fc2fa84b740 10 client.0 _ll_register_callbacks cb 0x7fc2ec004040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.626+0000 7fc2f51fd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56268ac508d0 0x56268ac5a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.626+0000 7fc2f51fd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56268ac508d0 0x56268ac5a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48592/0 (socket says 172.21.15.86:48592) 2024-04-21T08:41:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.626+0000 7fc2f51fd640 1 -- 172.21.15.86:0/3039644860 learned_addr learned my addr 172.21.15.86:0/3039644860 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.626+0000 7fc2f59fe640 1 --2- 172.21.15.86:0/3039644860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56268ac4fb60 0x56268ac59ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.626+0000 7fc2f61ff640 1 --2- 172.21.15.86:0/3039644860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56268ac590e0 0x56268ac601c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.627+0000 7fc2f51fd640 1 -- 172.21.15.86:0/3039644860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56268ac4fb60 msgr2=0x56268ac59ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.627+0000 7fc2f51fd640 1 --2- 172.21.15.86:0/3039644860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56268ac4fb60 0x56268ac59ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.627+0000 7fc2f51fd640 1 -- 172.21.15.86:0/3039644860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56268ac590e0 msgr2=0x56268ac601c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.627+0000 7fc2f51fd640 1 --2- 172.21.15.86:0/3039644860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56268ac590e0 0x56268ac601c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.627+0000 7fc2f51fd640 1 -- 172.21.15.86:0/3039644860 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56268ac60700 con 0x56268ac508d0 2024-04-21T08:41:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.627+0000 7fc2f51fd640 1 --2- 172.21.15.86:0/3039644860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56268ac508d0 0x56268ac5a000 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00be6d0 tx=0x7fc2e002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=579e1efbbd914951 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.627+0000 7fc2d97fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:31.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.627+0000 7fc2d97fa640 1 -- 172.21.15.86:0/3039644860 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc2e00dd060 con 0x56268ac508d0 2024-04-21T08:41:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.627+0000 7fc2d97fa640 1 -- 172.21.15.86:0/3039644860 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fc2e0003c90 con 0x56268ac508d0 2024-04-21T08:41:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.627+0000 7fc2fa84b740 1 -- 172.21.15.86:0/3039644860 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc2e00befe0 con 0x56268ac508d0 2024-04-21T08:41:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.627+0000 7fc2d97fa640 1 -- 172.21.15.86:0/3039644860 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc2e00c7550 con 0x56268ac508d0 2024-04-21T08:41:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.627+0000 7fc2fa84b740 10 client.16926 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.627+0000 7fc2fa84b740 1 -- 172.21.15.86:0/3039644860 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56268ac61950 con 0x56268ac508d0 2024-04-21T08:41:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.627+0000 7fc2d97fa640 1 -- 172.21.15.86:0/3039644860 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc2e00befe0 con 0x56268ac508d0 2024-04-21T08:41:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.627+0000 7fc2d97fa640 1 -- 172.21.15.86:0/3039644860 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc2e00beb40 con 0x56268ac508d0 2024-04-21T08:41:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.627+0000 7fc2fa84b740 10 client.16926 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.627+0000 7fc2fa84b740 10 client.16926 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.627+0000 7fc2fa84b740 20 client.16926 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.628+0000 7fc2fa84b740 1 --2- 172.21.15.86:0/3039644860 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56268accf920 0x56268acd1d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.628+0000 7fc2fa84b740 4 client.16926 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:41:31.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.628+0000 7fc2fa84b740 1 -- 172.21.15.86:0/3039644860 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x56268ab543d0 con 0x56268accf920 2024-04-21T08:41:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.628+0000 7fc2f59fe640 1 --2- 172.21.15.86:0/3039644860 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56268accf920 0x56268acd1d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.628+0000 7fc2f59fe640 1 --2- 172.21.15.86:0/3039644860 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56268accf920 0x56268acd1d40 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c49a94666ea2b349 server_cookie=89f461b3eea5b315 in_seq=0 out_seq=0 2024-04-21T08:41:31.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.628+0000 7fc2d97fa640 10 client.16926 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:31.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.637+0000 7fc2d97fa640 1 -- 172.21.15.86:0/3039644860 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x56268ab543d0 con 0x56268accf920 2024-04-21T08:41:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.637+0000 7fc2d97fa640 10 client.16926 handle_command_reply: tid=0 2024-04-21T08:41:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.637+0000 7fc2fa84b740 1 client.16926 shutdown 2024-04-21T08:41:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.637+0000 7fc2fa84b740 1 -- 172.21.15.86:0/3039644860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56268ac508d0 msgr2=0x56268ac5a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.637+0000 7fc2fa84b740 1 --2- 172.21.15.86:0/3039644860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56268ac508d0 0x56268ac5a000 secure :-1 s=READY pgs=920 cs=0 l=1 rev1=1 crypto rx=0x7fc2e00be6d0 tx=0x7fc2e002bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.637+0000 7fc2fa84b740 1 -- 172.21.15.86:0/3039644860 shutdown_connections 2024-04-21T08:41:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.637+0000 7fc2fa84b740 1 --2- 172.21.15.86:0/3039644860 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56268accf920 0x56268acd1d40 crc :-1 s=READY pgs=112 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.637+0000 7fc2fa84b740 1 --2- 172.21.15.86:0/3039644860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56268ac590e0 0x56268ac601c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.637+0000 7fc2fa84b740 1 --2- 172.21.15.86:0/3039644860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56268ac508d0 0x56268ac5a000 unknown :-1 s=CLOSED pgs=920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.637+0000 7fc2d97fa640 0 client.16926 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.637+0000 7fc2fa84b740 1 --2- 172.21.15.86:0/3039644860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56268ac4fb60 0x56268ac59ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:31.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.637+0000 7fc2fa84b740 1 -- 172.21.15.86:0/3039644860 >> 172.21.15.86:0/3039644860 conn(0x56268ac16c90 msgr2=0x56268ac546a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.637+0000 7fc2fa84b740 1 -- 172.21.15.86:0/3039644860 shutdown_connections 2024-04-21T08:41:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.637+0000 7fc2fa84b740 1 -- 172.21.15.86:0/3039644860 wait complete. 2024-04-21T08:41:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.637+0000 7fc2fa84b740 20 client.16926 trim_cache size 0 max 16384 2024-04-21T08:41:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:41:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:41:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:41:31.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:41:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.638+0000 7fc2f7c62640 1 -- 172.21.15.86:0/779031120 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc2dc0a3dc0 msgr2=0x7fc2dc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.638+0000 7fc2f7c62640 1 --2- 172.21.15.86:0/779031120 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc2dc0a3dc0 0x7fc2dc0a6280 secure :-1 s=READY pgs=136 cs=0 l=1 rev1=1 crypto rx=0x7fc2e800fc90 tx=0x7fc2e801e000 comp rx=0 tx=0).stop 2024-04-21T08:41:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.638+0000 7fc2f7c62640 1 -- 172.21.15.86:0/779031120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f006e470 msgr2=0x7fc2f01a79d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.638+0000 7fc2f7c62640 1 --2- 172.21.15.86:0/779031120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f006e470 0x7fc2f01a79d0 secure :-1 s=READY pgs=916 cs=0 l=1 rev1=1 crypto rx=0x7fc2e0005940 tx=0x7fc2e00045e0 comp rx=0 tx=0).stop 2024-04-21T08:41:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.638+0000 7fc2f7c62640 1 -- 172.21.15.86:0/779031120 shutdown_connections 2024-04-21T08:41:31.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.638+0000 7fc2f7c62640 1 --2- 172.21.15.86:0/779031120 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc2dc0a3dc0 0x7fc2dc0a6280 unknown :-1 s=CLOSED pgs=136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.638+0000 7fc2f7c62640 1 --2- 172.21.15.86:0/779031120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc2f011b5f0 0x7fc2f01abd50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.638+0000 7fc2f7c62640 1 --2- 172.21.15.86:0/779031120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc2f006e470 0x7fc2f01a79d0 unknown :-1 s=CLOSED pgs=916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.638+0000 7fc2f7c62640 1 --2- 172.21.15.86:0/779031120 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc2f006db50 0x7fc2f01a7490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.638+0000 7fc2f7c62640 1 -- 172.21.15.86:0/779031120 >> 172.21.15.86:0/779031120 conn(0x7fc2f006d1b0 msgr2=0x7fc2f011c970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.638+0000 7fc2f7c62640 1 -- 172.21.15.86:0/779031120 shutdown_connections 2024-04-21T08:41:31.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:31.638+0000 7fc2f7c62640 1 -- 172.21.15.86:0/779031120 wait complete. 2024-04-21T08:41:32.548 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:32 smithi179 ceph-mon[31829]: pgmap v85: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.8 KiB/s wr, 11 op/s 2024-04-21T08:41:32.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:32 smithi086 ceph-mon[27939]: pgmap v85: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.8 KiB/s wr, 11 op/s 2024-04-21T08:41:32.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:32 smithi088 ceph-mon[31259]: pgmap v85: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.6 KiB/s rd, 5.8 KiB/s wr, 11 op/s 2024-04-21T08:41:32.945 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:41:32.945 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:41:33.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.056+0000 7f6f25759640 1 -- 172.21.15.86:0/269666720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f20101a30 msgr2=0x7f6f20105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.056+0000 7f6f25759640 1 --2- 172.21.15.86:0/269666720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f20101a30 0x7f6f20105a70 secure :-1 s=READY pgs=921 cs=0 l=1 rev1=1 crypto rx=0x7f6f08009900 tx=0x7f6f0802bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:33.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.056+0000 7f6f25759640 1 -- 172.21.15.86:0/269666720 shutdown_connections 2024-04-21T08:41:33.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.056+0000 7f6f25759640 1 --2- 172.21.15.86:0/269666720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f20106150 0x7f6f2010dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.056+0000 7f6f25759640 1 --2- 172.21.15.86:0/269666720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f20101a30 0x7f6f20105a70 unknown :-1 s=CLOSED pgs=921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.056+0000 7f6f25759640 1 --2- 172.21.15.86:0/269666720 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f20101080 0x7f6f20101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.056+0000 7f6f25759640 1 -- 172.21.15.86:0/269666720 >> 172.21.15.86:0/269666720 conn(0x7f6f20078000 msgr2=0x7f6f20078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:33.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.056+0000 7f6f25759640 1 -- 172.21.15.86:0/269666720 shutdown_connections 2024-04-21T08:41:33.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.056+0000 7f6f25759640 1 -- 172.21.15.86:0/269666720 wait complete. 2024-04-21T08:41:33.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.057+0000 7f6f25759640 1 Processor -- start 2024-04-21T08:41:33.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.057+0000 7f6f25759640 1 -- start start 2024-04-21T08:41:33.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.057+0000 7f6f25759640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f20101080 0x7f6f2019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.057+0000 7f6f25759640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f20106150 0x7f6f2019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.057+0000 7f6f25759640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f201985e0 0x7f6f20198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.057+0000 7f6f25759640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6f201110f0 con 0x7f6f20106150 2024-04-21T08:41:33.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.057+0000 7f6f25759640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6f20110f70 con 0x7f6f201985e0 2024-04-21T08:41:33.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.057+0000 7f6f25759640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6f20111270 con 0x7f6f20101080 2024-04-21T08:41:33.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.057+0000 7f6f167fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f20106150 0x7f6f2019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.057+0000 7f6f167fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f20106150 0x7f6f2019e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48602/0 (socket says 172.21.15.86:48602) 2024-04-21T08:41:33.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.057+0000 7f6f167fc640 1 -- 172.21.15.86:0/4088022301 learned_addr learned my addr 172.21.15.86:0/4088022301 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:33.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.057+0000 7f6f1f7fe640 1 --2- 172.21.15.86:0/4088022301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f201985e0 0x7f6f20198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.057+0000 7f6f167fc640 1 -- 172.21.15.86:0/4088022301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f20101080 msgr2=0x7f6f2019e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.057+0000 7f6f167fc640 1 --2- 172.21.15.86:0/4088022301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f20101080 0x7f6f2019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.058+0000 7f6f167fc640 1 -- 172.21.15.86:0/4088022301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f201985e0 msgr2=0x7f6f20198a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.058+0000 7f6f167fc640 1 --2- 172.21.15.86:0/4088022301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f201985e0 0x7f6f20198a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.058+0000 7f6f167fc640 1 -- 172.21.15.86:0/4088022301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6f20199100 con 0x7f6f20106150 2024-04-21T08:41:33.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.058+0000 7f6f1f7fe640 1 --2- 172.21.15.86:0/4088022301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f201985e0 0x7f6f20198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:33.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.058+0000 7f6f167fc640 1 --2- 172.21.15.86:0/4088022301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f20106150 0x7f6f2019e920 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f6f08036040 tx=0x7f6f0802c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:33.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.058+0000 7f6f1cff9640 1 -- 172.21.15.86:0/4088022301 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f08043020 con 0x7f6f20106150 2024-04-21T08:41:33.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.058+0000 7f6f25759640 1 -- 172.21.15.86:0/4088022301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6f20199390 con 0x7f6f20106150 2024-04-21T08:41:33.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.058+0000 7f6f1cff9640 1 -- 172.21.15.86:0/4088022301 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f0802cbe0 con 0x7f6f20106150 2024-04-21T08:41:33.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.058+0000 7f6f1cff9640 1 -- 172.21.15.86:0/4088022301 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f08003bf0 con 0x7f6f20106150 2024-04-21T08:41:33.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.058+0000 7f6f25759640 1 -- 172.21.15.86:0/4088022301 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6f201a55e0 con 0x7f6f20106150 2024-04-21T08:41:33.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.061+0000 7f6f1cff9640 1 -- 172.21.15.86:0/4088022301 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6f08046020 con 0x7f6f20106150 2024-04-21T08:41:33.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.061+0000 7f6f2842c740 1 Processor -- start 2024-04-21T08:41:33.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.061+0000 7f6f2842c740 1 -- start start 2024-04-21T08:41:33.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.061+0000 7f6f2842c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba4d264580 0x55ba4d264960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.061+0000 7f6f2842c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba4d264ea0 0x55ba4d26d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.061+0000 7f6f2842c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba4d26da80 0x55ba4d26fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.061+0000 7f6f2842c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ba4d1c91e0 con 0x55ba4d264ea0 2024-04-21T08:41:33.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.061+0000 7f6f2842c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55ba4d166390 con 0x55ba4d264580 2024-04-21T08:41:33.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.061+0000 7f6f2842c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ba4d1c51a0 con 0x55ba4d26da80 2024-04-21T08:41:33.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.061+0000 7f6f1cff9640 1 --2- 172.21.15.86:0/4088022301 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f6f0c0a3d20 0x7f6f0c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.061+0000 7f6f1effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba4d264ea0 0x55ba4d26d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.061+0000 7f6f1effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba4d264ea0 0x55ba4d26d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48612/0 (socket says 172.21.15.86:48612) 2024-04-21T08:41:33.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.061+0000 7f6f1effd640 1 -- 172.21.15.86:0/1246402342 learned_addr learned my addr 172.21.15.86:0/1246402342 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:33.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.062+0000 7f6f1f7fe640 1 --2- 172.21.15.86:0/1246402342 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba4d264580 0x55ba4d264960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.062+0000 7f6f1f7fe640 1 --2- 172.21.15.86:0/4088022301 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f6f0c0a3d20 0x7f6f0c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.062+0000 7f6f1cff9640 1 -- 172.21.15.86:0/4088022301 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f6f080f2d60 con 0x7f6f20106150 2024-04-21T08:41:33.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.062+0000 7f6f167fc640 1 --2- 172.21.15.86:0/1246402342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba4d26da80 0x55ba4d26fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.062+0000 7f6f1effd640 1 -- 172.21.15.86:0/1246402342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba4d26da80 msgr2=0x55ba4d26fe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:33.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.062+0000 7f6f1effd640 1 --2- 172.21.15.86:0/1246402342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba4d26da80 0x55ba4d26fe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.062+0000 7f6f1effd640 1 -- 172.21.15.86:0/1246402342 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba4d264580 msgr2=0x55ba4d264960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:33.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.062+0000 7f6f1effd640 1 --2- 172.21.15.86:0/1246402342 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba4d264580 0x55ba4d264960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.062+0000 7f6f1effd640 1 -- 172.21.15.86:0/1246402342 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba4cfa57d0 con 0x55ba4d264ea0 2024-04-21T08:41:33.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.062+0000 7f6f167fc640 1 --2- 172.21.15.86:0/1246402342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba4d26da80 0x55ba4d26fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:33.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.062+0000 7f6f1f7fe640 1 --2- 172.21.15.86:0/4088022301 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f6f0c0a3d20 0x7f6f0c0a61e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6f18009440 tx=0x7f6f180093d0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:33.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.062+0000 7f6f1effd640 1 --2- 172.21.15.86:0/1246402342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba4d264ea0 0x55ba4d26d540 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f6f1000bb10 tx=0x7f6f1000d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a7c911846439288 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:33.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.062+0000 7f6f157fa640 1 -- 172.21.15.86:0/1246402342 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f10016070 con 0x55ba4d264ea0 2024-04-21T08:41:33.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.062+0000 7f6f157fa640 1 -- 172.21.15.86:0/1246402342 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f1000dce0 con 0x55ba4d264ea0 2024-04-21T08:41:33.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.062+0000 7f6f157fa640 1 -- 172.21.15.86:0/1246402342 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f10010460 con 0x55ba4d264ea0 2024-04-21T08:41:33.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.063+0000 7f6f2842c740 1 -- 172.21.15.86:0/1246402342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba4d264ea0 msgr2=0x55ba4d26d540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.063+0000 7f6f2842c740 1 --2- 172.21.15.86:0/1246402342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba4d264ea0 0x55ba4d26d540 secure :-1 s=READY pgs=923 cs=0 l=1 rev1=1 crypto rx=0x7f6f1000bb10 tx=0x7f6f1000d2e0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.063+0000 7f6f2842c740 1 -- 172.21.15.86:0/1246402342 shutdown_connections 2024-04-21T08:41:33.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.063+0000 7f6f2842c740 1 --2- 172.21.15.86:0/1246402342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba4d26da80 0x55ba4d26fe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.063+0000 7f6f2842c740 1 --2- 172.21.15.86:0/1246402342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba4d264ea0 0x55ba4d26d540 unknown :-1 s=CLOSED pgs=923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.063+0000 7f6f2842c740 1 --2- 172.21.15.86:0/1246402342 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba4d264580 0x55ba4d264960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.063+0000 7f6f2842c740 1 -- 172.21.15.86:0/1246402342 >> 172.21.15.86:0/1246402342 conn(0x55ba4d25ecb0 msgr2=0x55ba4d25f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:33.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.063+0000 7f6f2842c740 1 -- 172.21.15.86:0/1246402342 shutdown_connections 2024-04-21T08:41:33.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.063+0000 7f6f2842c740 1 -- 172.21.15.86:0/1246402342 wait complete. 2024-04-21T08:41:33.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.063+0000 7f6f2842c740 1 -- start start 2024-04-21T08:41:33.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.063+0000 7f6f2842c740 1 Processor -- start 2024-04-21T08:41:33.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.064+0000 7f6f2842c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba4d264580 0x55ba4d26aa60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.064+0000 7f6f2842c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba4d264ea0 0x55ba4d26afa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.064+0000 7f6f2842c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba4d26da80 0x55ba4d26b4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.064+0000 7f6f2842c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ba4d270870 con 0x55ba4d26da80 2024-04-21T08:41:33.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.064+0000 7f6f2842c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55ba4d1822f0 con 0x55ba4d264ea0 2024-04-21T08:41:33.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.064+0000 7f6f2842c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ba4d2709f0 con 0x55ba4d264580 2024-04-21T08:41:33.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.064+0000 7f6f2842c740 10 client.0 _ll_register_callbacks cb 0x55ba4d18f580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:33.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.064+0000 7f6f1f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba4d26da80 0x55ba4d26b4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.064+0000 7f6f1f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba4d26da80 0x55ba4d26b4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48618/0 (socket says 172.21.15.86:48618) 2024-04-21T08:41:33.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.064+0000 7f6f1f7fe640 1 -- 172.21.15.86:0/3555160840 learned_addr learned my addr 172.21.15.86:0/3555160840 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:33.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.064+0000 7f6f167fc640 1 --2- 172.21.15.86:0/3555160840 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba4d264ea0 0x55ba4d26afa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.064+0000 7f6f1effd640 1 --2- 172.21.15.86:0/3555160840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba4d264580 0x55ba4d26aa60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.064+0000 7f6f1f7fe640 1 -- 172.21.15.86:0/3555160840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba4d264580 msgr2=0x55ba4d26aa60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:33.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.064+0000 7f6f1f7fe640 1 --2- 172.21.15.86:0/3555160840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba4d264580 0x55ba4d26aa60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.064+0000 7f6f1f7fe640 1 -- 172.21.15.86:0/3555160840 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba4d264ea0 msgr2=0x55ba4d26afa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:33.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.064+0000 7f6f1f7fe640 1 --2- 172.21.15.86:0/3555160840 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba4d264ea0 0x55ba4d26afa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.064+0000 7f6f1f7fe640 1 -- 172.21.15.86:0/3555160840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba4cedb3f0 con 0x55ba4d26da80 2024-04-21T08:41:33.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.064+0000 7f6f1effd640 1 --2- 172.21.15.86:0/3555160840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba4d264580 0x55ba4d26aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:33.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.064+0000 7f6f1f7fe640 1 --2- 172.21.15.86:0/3555160840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba4d26da80 0x55ba4d26b4e0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f6f18016e20 tx=0x7f6f18004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1c069a617cb053 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:33.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.064+0000 7f6f067fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:33.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.064+0000 7f6f067fc640 1 -- 172.21.15.86:0/3555160840 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f18029060 con 0x55ba4d26da80 2024-04-21T08:41:33.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.064+0000 7f6f067fc640 1 -- 172.21.15.86:0/3555160840 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f6f1801b0d0 con 0x55ba4d26da80 2024-04-21T08:41:33.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.064+0000 7f6f2842c740 1 -- 172.21.15.86:0/3555160840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f1000dce0 con 0x55ba4d26da80 2024-04-21T08:41:33.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.065+0000 7f6f067fc640 1 -- 172.21.15.86:0/3555160840 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6f18030050 con 0x55ba4d26da80 2024-04-21T08:41:33.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.065+0000 7f6f2842c740 10 client.16938 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:33.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.065+0000 7f6f2842c740 1 -- 172.21.15.86:0/3555160840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ba4ce9b0f0 con 0x55ba4d26da80 2024-04-21T08:41:33.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.065+0000 7f6f067fc640 1 -- 172.21.15.86:0/3555160840 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f1000dce0 con 0x55ba4d26da80 2024-04-21T08:41:33.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.065+0000 7f6f067fc640 1 -- 172.21.15.86:0/3555160840 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6f18024b50 con 0x55ba4d26da80 2024-04-21T08:41:33.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.065+0000 7f6f2842c740 10 client.16938 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:33.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.065+0000 7f6f2842c740 10 client.16938 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:33.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.065+0000 7f6f2842c740 20 client.16938 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:33.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.065+0000 7f6f2842c740 1 --2- 172.21.15.86:0/3555160840 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ba4d27da00 0x55ba4d27fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.065+0000 7f6f2842c740 4 client.16938 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:33.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.065+0000 7f6f2842c740 1 -- 172.21.15.86:0/3555160840 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba4d1d6c00 con 0x55ba4d27da00 2024-04-21T08:41:33.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.065+0000 7f6f1effd640 1 --2- 172.21.15.86:0/3555160840 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ba4d27da00 0x55ba4d27fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.066+0000 7f6f067fc640 10 client.16938 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:33.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.066+0000 7f6f1effd640 1 --2- 172.21.15.86:0/3555160840 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ba4d27da00 0x55ba4d27fe40 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c862203822dd5638 server_cookie=b0792de305852ed3 in_seq=0 out_seq=0 2024-04-21T08:41:33.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.066+0000 7f6f067fc640 1 -- 172.21.15.86:0/3555160840 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ba4d1d6c00 con 0x55ba4d27da00 2024-04-21T08:41:33.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.066+0000 7f6f067fc640 10 client.16938 handle_command_reply: tid=0 2024-04-21T08:41:33.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.066+0000 7f6f2842c740 1 client.16938 shutdown 2024-04-21T08:41:33.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.067+0000 7f6f2842c740 1 -- 172.21.15.86:0/3555160840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba4d26da80 msgr2=0x55ba4d26b4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.067+0000 7f6f2842c740 1 --2- 172.21.15.86:0/3555160840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba4d26da80 0x55ba4d26b4e0 secure :-1 s=READY pgs=924 cs=0 l=1 rev1=1 crypto rx=0x7f6f18016e20 tx=0x7f6f18004b50 comp rx=0 tx=0).stop 2024-04-21T08:41:33.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.067+0000 7f6f2842c740 1 -- 172.21.15.86:0/3555160840 shutdown_connections 2024-04-21T08:41:33.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.067+0000 7f6f2842c740 1 --2- 172.21.15.86:0/3555160840 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ba4d27da00 0x55ba4d27fe40 crc :-1 s=READY pgs=113 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.067+0000 7f6f2842c740 1 --2- 172.21.15.86:0/3555160840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba4d26da80 0x55ba4d26b4e0 unknown :-1 s=CLOSED pgs=924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.067+0000 7f6f2842c740 1 --2- 172.21.15.86:0/3555160840 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba4d264ea0 0x55ba4d26afa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.067+0000 7f6f2842c740 1 --2- 172.21.15.86:0/3555160840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba4d264580 0x55ba4d26aa60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.067+0000 7f6f067fc640 0 client.16938 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:33.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.067+0000 7f6f2842c740 1 -- 172.21.15.86:0/3555160840 >> 172.21.15.86:0/3555160840 conn(0x55ba4d25ecb0 msgr2=0x55ba4d260a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:33.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.067+0000 7f6f2842c740 1 -- 172.21.15.86:0/3555160840 shutdown_connections 2024-04-21T08:41:33.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.067+0000 7f6f2842c740 1 -- 172.21.15.86:0/3555160840 wait complete. 2024-04-21T08:41:33.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.067+0000 7f6f2842c740 20 client.16938 trim_cache size 0 max 16384 2024-04-21T08:41:33.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.084+0000 7f6f2842c740 1 Processor -- start 2024-04-21T08:41:33.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.084+0000 7f6f2842c740 1 -- start start 2024-04-21T08:41:33.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.084+0000 7f6f2842c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba4d264580 0x55ba4d279760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.084+0000 7f6f2842c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba4d264ea0 0x55ba4d2699a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.084+0000 7f6f2842c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba4d26da80 0x55ba4d285710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.084+0000 7f6f2842c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6f201af290 con 0x55ba4d264580 2024-04-21T08:41:33.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.084+0000 7f6f2842c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6f201b0dd0 con 0x55ba4d26da80 2024-04-21T08:41:33.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.084+0000 7f6f2842c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6f201af960 con 0x55ba4d264ea0 2024-04-21T08:41:33.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.085+0000 7f6f1effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba4d264580 0x55ba4d279760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.085+0000 7f6f1effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba4d264580 0x55ba4d279760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48634/0 (socket says 172.21.15.86:48634) 2024-04-21T08:41:33.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.085+0000 7f6f1effd640 1 -- 172.21.15.86:0/970994038 learned_addr learned my addr 172.21.15.86:0/970994038 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:33.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.085+0000 7f6f1f7fe640 1 --2- 172.21.15.86:0/970994038 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba4d26da80 0x55ba4d285710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.085+0000 7f6f1effd640 1 -- 172.21.15.86:0/970994038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba4d264ea0 msgr2=0x55ba4d2699a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:33.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.085+0000 7f6f1effd640 1 --2- 172.21.15.86:0/970994038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba4d264ea0 0x55ba4d2699a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.085+0000 7f6f1effd640 1 -- 172.21.15.86:0/970994038 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba4d26da80 msgr2=0x55ba4d285710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:33.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.085+0000 7f6f1effd640 1 --2- 172.21.15.86:0/970994038 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba4d26da80 0x55ba4d285710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.085+0000 7f6f1effd640 1 -- 172.21.15.86:0/970994038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba4d263900 con 0x55ba4d264580 2024-04-21T08:41:33.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.085+0000 7f6f1effd640 1 --2- 172.21.15.86:0/970994038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba4d264580 0x55ba4d279760 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f6f1000ba30 tx=0x7f6f10004a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61179e5f26e0104c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:33.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.085+0000 7f6f14ff9640 1 -- 172.21.15.86:0/970994038 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f10016030 con 0x55ba4d264580 2024-04-21T08:41:33.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.085+0000 7f6f14ff9640 1 -- 172.21.15.86:0/970994038 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f100044f0 con 0x55ba4d264580 2024-04-21T08:41:33.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.085+0000 7f6f14ff9640 1 -- 172.21.15.86:0/970994038 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f100035f0 con 0x55ba4d264580 2024-04-21T08:41:33.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.086+0000 7f6f2842c740 1 -- 172.21.15.86:0/970994038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba4d264580 msgr2=0x55ba4d279760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.086+0000 7f6f2842c740 1 --2- 172.21.15.86:0/970994038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba4d264580 0x55ba4d279760 secure :-1 s=READY pgs=925 cs=0 l=1 rev1=1 crypto rx=0x7f6f1000ba30 tx=0x7f6f10004a70 comp rx=0 tx=0).stop 2024-04-21T08:41:33.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.086+0000 7f6f2842c740 1 -- 172.21.15.86:0/970994038 shutdown_connections 2024-04-21T08:41:33.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.086+0000 7f6f2842c740 1 --2- 172.21.15.86:0/970994038 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba4d26da80 0x55ba4d285710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.086+0000 7f6f2842c740 1 --2- 172.21.15.86:0/970994038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba4d264ea0 0x55ba4d2699a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.086+0000 7f6f2842c740 1 --2- 172.21.15.86:0/970994038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba4d264580 0x55ba4d279760 unknown :-1 s=CLOSED pgs=925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.086+0000 7f6f2842c740 1 -- 172.21.15.86:0/970994038 >> 172.21.15.86:0/970994038 conn(0x55ba4d25ecb0 msgr2=0x55ba4d26ca10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:33.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.086+0000 7f6f2842c740 1 -- 172.21.15.86:0/970994038 shutdown_connections 2024-04-21T08:41:33.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.086+0000 7f6f2842c740 1 -- 172.21.15.86:0/970994038 wait complete. 2024-04-21T08:41:33.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.086+0000 7f6f2842c740 1 -- start start 2024-04-21T08:41:33.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.086+0000 7f6f2842c740 1 Processor -- start 2024-04-21T08:41:33.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.086+0000 7f6f2842c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba4d264580 0x55ba4d275c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.086+0000 7f6f2842c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba4d264ea0 0x55ba4d2761b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.086+0000 7f6f2842c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba4d26da80 0x55ba4d272c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.086+0000 7f6f2842c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6f201b0700 con 0x55ba4d264580 2024-04-21T08:41:33.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.086+0000 7f6f2842c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6f201b0030 con 0x55ba4d26da80 2024-04-21T08:41:33.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.086+0000 7f6f2842c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6f201ad250 con 0x55ba4d264ea0 2024-04-21T08:41:33.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.087+0000 7f6f2842c740 10 client.0 _ll_register_callbacks cb 0x7f6f1800adc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:33.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.087+0000 7f6f1effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba4d264580 0x55ba4d275c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.087+0000 7f6f1effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba4d264580 0x55ba4d275c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48640/0 (socket says 172.21.15.86:48640) 2024-04-21T08:41:33.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.087+0000 7f6f1effd640 1 -- 172.21.15.86:0/2907632363 learned_addr learned my addr 172.21.15.86:0/2907632363 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:33.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.087+0000 7f6f1f7fe640 1 --2- 172.21.15.86:0/2907632363 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba4d26da80 0x55ba4d272c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.087+0000 7f6f167fc640 1 --2- 172.21.15.86:0/2907632363 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba4d264ea0 0x55ba4d2761b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.087+0000 7f6f1effd640 1 -- 172.21.15.86:0/2907632363 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba4d264ea0 msgr2=0x55ba4d2761b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:33.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.087+0000 7f6f1effd640 1 --2- 172.21.15.86:0/2907632363 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba4d264ea0 0x55ba4d2761b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.087+0000 7f6f1effd640 1 -- 172.21.15.86:0/2907632363 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba4d26da80 msgr2=0x55ba4d272c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:33.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.087+0000 7f6f1effd640 1 --2- 172.21.15.86:0/2907632363 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba4d26da80 0x55ba4d272c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.087+0000 7f6f1effd640 1 -- 172.21.15.86:0/2907632363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba4d273280 con 0x55ba4d264580 2024-04-21T08:41:33.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.087+0000 7f6f167fc640 1 --2- 172.21.15.86:0/2907632363 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba4d264ea0 0x55ba4d2761b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:33.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.087+0000 7f6f1effd640 1 --2- 172.21.15.86:0/2907632363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba4d264580 0x55ba4d275c70 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f6f1000e010 tx=0x7f6f10004a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=acde20dba3089075 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:33.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.087+0000 7f6f067fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:33.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.087+0000 7f6f067fc640 1 -- 172.21.15.86:0/2907632363 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6f10033060 con 0x55ba4d264580 2024-04-21T08:41:33.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.087+0000 7f6f2842c740 1 -- 172.21.15.86:0/2907632363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6f100044f0 con 0x55ba4d264580 2024-04-21T08:41:33.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.087+0000 7f6f067fc640 1 -- 172.21.15.86:0/2907632363 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f6f100244d0 con 0x55ba4d264580 2024-04-21T08:41:33.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.087+0000 7f6f067fc640 1 -- 172.21.15.86:0/2907632363 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6f1003a050 con 0x55ba4d264580 2024-04-21T08:41:33.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.087+0000 7f6f2842c740 10 client.16944 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:33.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.087+0000 7f6f2842c740 1 -- 172.21.15.86:0/2907632363 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ba4d2e48a0 con 0x55ba4d264580 2024-04-21T08:41:33.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.088+0000 7f6f067fc640 1 -- 172.21.15.86:0/2907632363 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6f100044f0 con 0x55ba4d264580 2024-04-21T08:41:33.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.088+0000 7f6f067fc640 1 -- 172.21.15.86:0/2907632363 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6f10010620 con 0x55ba4d264580 2024-04-21T08:41:33.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.088+0000 7f6f2842c740 10 client.16944 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:33.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.088+0000 7f6f2842c740 10 client.16944 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:33.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.088+0000 7f6f2842c740 20 client.16944 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:33.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.088+0000 7f6f2842c740 1 --2- 172.21.15.86:0/2907632363 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ba4d2e4d90 0x55ba4d2e71b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.088+0000 7f6f2842c740 4 client.16944 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:41:33.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.088+0000 7f6f2842c740 1 -- 172.21.15.86:0/2907632363 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ba4d1d0a70 con 0x55ba4d2e4d90 2024-04-21T08:41:33.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.088+0000 7f6f167fc640 1 --2- 172.21.15.86:0/2907632363 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ba4d2e4d90 0x55ba4d2e71b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.088+0000 7f6f067fc640 10 client.16944 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:33.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.088+0000 7f6f167fc640 1 --2- 172.21.15.86:0/2907632363 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ba4d2e4d90 0x55ba4d2e71b0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=228600a40d10c21e server_cookie=758534b7a27ce455 in_seq=0 out_seq=0 2024-04-21T08:41:33.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.089+0000 7f6f067fc640 1 -- 172.21.15.86:0/2907632363 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55ba4d1d0a70 con 0x55ba4d2e4d90 2024-04-21T08:41:33.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.089+0000 7f6f067fc640 10 client.16944 handle_command_reply: tid=0 2024-04-21T08:41:33.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.089+0000 7f6f2842c740 1 client.16944 shutdown 2024-04-21T08:41:33.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.089+0000 7f6f2842c740 1 -- 172.21.15.86:0/2907632363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba4d264580 msgr2=0x55ba4d275c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.089+0000 7f6f2842c740 1 --2- 172.21.15.86:0/2907632363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba4d264580 0x55ba4d275c70 secure :-1 s=READY pgs=926 cs=0 l=1 rev1=1 crypto rx=0x7f6f1000e010 tx=0x7f6f10004a00 comp rx=0 tx=0).stop 2024-04-21T08:41:33.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.089+0000 7f6f2842c740 1 -- 172.21.15.86:0/2907632363 shutdown_connections 2024-04-21T08:41:33.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.089+0000 7f6f2842c740 1 --2- 172.21.15.86:0/2907632363 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ba4d2e4d90 0x55ba4d2e71b0 crc :-1 s=READY pgs=114 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.089+0000 7f6f2842c740 1 --2- 172.21.15.86:0/2907632363 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba4d26da80 0x55ba4d272c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.089+0000 7f6f2842c740 1 --2- 172.21.15.86:0/2907632363 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba4d264ea0 0x55ba4d2761b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.089+0000 7f6f067fc640 0 client.16944 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:33.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.089+0000 7f6f2842c740 1 --2- 172.21.15.86:0/2907632363 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba4d264580 0x55ba4d275c70 unknown :-1 s=CLOSED pgs=926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.089+0000 7f6f2842c740 1 -- 172.21.15.86:0/2907632363 >> 172.21.15.86:0/2907632363 conn(0x55ba4d25ecb0 msgr2=0x55ba4d26c0a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:33.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.089+0000 7f6f2842c740 1 -- 172.21.15.86:0/2907632363 shutdown_connections 2024-04-21T08:41:33.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.089+0000 7f6f2842c740 1 -- 172.21.15.86:0/2907632363 wait complete. 2024-04-21T08:41:33.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.089+0000 7f6f2842c740 20 client.16944 trim_cache size 0 max 16384 2024-04-21T08:41:33.095 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:41:33.095 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-21T08:41:33.095 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-21T08:41:33.095 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:41:33.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.089+0000 7f6f25759640 1 -- 172.21.15.86:0/4088022301 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f6f0c0a3d20 msgr2=0x7f6f0c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.089+0000 7f6f25759640 1 --2- 172.21.15.86:0/4088022301 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f6f0c0a3d20 0x7f6f0c0a61e0 secure :-1 s=READY pgs=137 cs=0 l=1 rev1=1 crypto rx=0x7f6f18009440 tx=0x7f6f180093d0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.089+0000 7f6f25759640 1 -- 172.21.15.86:0/4088022301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f20106150 msgr2=0x7f6f2019e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.089+0000 7f6f25759640 1 --2- 172.21.15.86:0/4088022301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f20106150 0x7f6f2019e920 secure :-1 s=READY pgs=922 cs=0 l=1 rev1=1 crypto rx=0x7f6f08036040 tx=0x7f6f0802c7e0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.089+0000 7f6f25759640 1 -- 172.21.15.86:0/4088022301 shutdown_connections 2024-04-21T08:41:33.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.089+0000 7f6f25759640 1 --2- 172.21.15.86:0/4088022301 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f6f0c0a3d20 0x7f6f0c0a61e0 unknown :-1 s=CLOSED pgs=137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.089+0000 7f6f25759640 1 --2- 172.21.15.86:0/4088022301 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6f201985e0 0x7f6f20198a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.089+0000 7f6f25759640 1 --2- 172.21.15.86:0/4088022301 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6f20106150 0x7f6f2019e920 unknown :-1 s=CLOSED pgs=922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.089+0000 7f6f25759640 1 --2- 172.21.15.86:0/4088022301 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6f20101080 0x7f6f2019e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.089+0000 7f6f25759640 1 -- 172.21.15.86:0/4088022301 >> 172.21.15.86:0/4088022301 conn(0x7f6f20078000 msgr2=0x7f6f200694a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:33.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.090+0000 7f6f25759640 1 -- 172.21.15.86:0/4088022301 shutdown_connections 2024-04-21T08:41:33.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.090+0000 7f6f25759640 1 -- 172.21.15.86:0/4088022301 wait complete. 2024-04-21T08:41:33.099 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:41:33.099 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:41:33.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.199+0000 7f7edea48640 1 -- 172.21.15.86:0/1117379623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed8105e30 msgr2=0x7f7ed8106210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.199+0000 7f7edea48640 1 --2- 172.21.15.86:0/1117379623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed8105e30 0x7f7ed8106210 secure :-1 s=READY pgs=927 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8009930 tx=0x7f7ec802bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:33.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.199+0000 7f7edea48640 1 -- 172.21.15.86:0/1117379623 shutdown_connections 2024-04-21T08:41:33.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.199+0000 7f7edea48640 1 --2- 172.21.15.86:0/1117379623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ed8101b60 0x7f7ed810dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.199+0000 7f7edea48640 1 --2- 172.21.15.86:0/1117379623 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ed81011a0 0x7f7ed8101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.199+0000 7f7edea48640 1 --2- 172.21.15.86:0/1117379623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed8105e30 0x7f7ed8106210 unknown :-1 s=CLOSED pgs=927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.199+0000 7f7edea48640 1 -- 172.21.15.86:0/1117379623 >> 172.21.15.86:0/1117379623 conn(0x7f7ed8077e30 msgr2=0x7f7ed8078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:33.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.199+0000 7f7edea48640 1 -- 172.21.15.86:0/1117379623 shutdown_connections 2024-04-21T08:41:33.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.199+0000 7f7edea48640 1 -- 172.21.15.86:0/1117379623 wait complete. 2024-04-21T08:41:33.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.199+0000 7f7edea48640 1 Processor -- start 2024-04-21T08:41:33.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.199+0000 7f7edea48640 1 -- start start 2024-04-21T08:41:33.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.199+0000 7f7edea48640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ed81011a0 0x7f7ed8198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.199+0000 7f7edea48640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed8101b60 0x7f7ed8198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.199+0000 7f7edea48640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ed8105e30 0x7f7ed819d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.200+0000 7f7edea48640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed8112240 con 0x7f7ed8101b60 2024-04-21T08:41:33.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.200+0000 7f7edea48640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed81120c0 con 0x7f7ed81011a0 2024-04-21T08:41:33.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.200+0000 7f7edea48640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7ed81123c0 con 0x7f7ed8105e30 2024-04-21T08:41:33.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.200+0000 7f7ed77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed8101b60 0x7f7ed8198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.200+0000 7f7ed77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed8101b60 0x7f7ed8198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48650/0 (socket says 172.21.15.86:48650) 2024-04-21T08:41:33.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.200+0000 7f7ed77fe640 1 -- 172.21.15.86:0/3306843758 learned_addr learned my addr 172.21.15.86:0/3306843758 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:33.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.200+0000 7f7ed7fff640 1 --2- 172.21.15.86:0/3306843758 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ed81011a0 0x7f7ed8198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.200+0000 7f7ed77fe640 1 -- 172.21.15.86:0/3306843758 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ed8105e30 msgr2=0x7f7ed819d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.200+0000 7f7ed77fe640 1 --2- 172.21.15.86:0/3306843758 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ed8105e30 0x7f7ed819d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.200+0000 7f7ed77fe640 1 -- 172.21.15.86:0/3306843758 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ed81011a0 msgr2=0x7f7ed8198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.200+0000 7f7ed77fe640 1 --2- 172.21.15.86:0/3306843758 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ed81011a0 0x7f7ed8198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.200+0000 7f7ed77fe640 1 -- 172.21.15.86:0/3306843758 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ed819d750 con 0x7f7ed8101b60 2024-04-21T08:41:33.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.200+0000 7f7ed77fe640 1 --2- 172.21.15.86:0/3306843758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed8101b60 0x7f7ed8198ca0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc00e9f0 tx=0x7f7ecc00eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:33.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.200+0000 7f7ed57fa640 1 -- 172.21.15.86:0/3306843758 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc00cd80 con 0x7f7ed8101b60 2024-04-21T08:41:33.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.200+0000 7f7edea48640 1 -- 172.21.15.86:0/3306843758 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ed819da40 con 0x7f7ed8101b60 2024-04-21T08:41:33.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.200+0000 7f7ed57fa640 1 -- 172.21.15.86:0/3306843758 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ecc0108e0 con 0x7f7ed8101b60 2024-04-21T08:41:33.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.200+0000 7f7ed57fa640 1 -- 172.21.15.86:0/3306843758 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ecc018980 con 0x7f7ed8101b60 2024-04-21T08:41:33.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.200+0000 7f7edea48640 1 -- 172.21.15.86:0/3306843758 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ed81a52d0 con 0x7f7ed8101b60 2024-04-21T08:41:33.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.201+0000 7f7edea48640 1 -- 172.21.15.86:0/3306843758 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f7ed8102570 con 0x7f7ed8101b60 2024-04-21T08:41:33.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.202+0000 7f7ed57fa640 1 -- 172.21.15.86:0/3306843758 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7ecc010430 con 0x7f7ed8101b60 2024-04-21T08:41:33.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.203+0000 7f7ed57fa640 1 --2- 172.21.15.86:0/3306843758 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7ec00a3d20 0x7f7ec00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.203+0000 7f7ed7fff640 1 --2- 172.21.15.86:0/3306843758 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7ec00a3d20 0x7f7ec00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.203+0000 7f7ed57fa640 1 -- 172.21.15.86:0/3306843758 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f7ecc0d0ee0 con 0x7f7ed8101b60 2024-04-21T08:41:33.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.203+0000 7f7ed7fff640 1 --2- 172.21.15.86:0/3306843758 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7ec00a3d20 0x7f7ec00a61e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8005ab0 tx=0x7f7ec8005fd0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:33.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.208+0000 7f7ed57fa640 1 -- 172.21.15.86:0/3306843758 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f7ecc09d210 con 0x7f7ed8101b60 2024-04-21T08:41:33.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.423+0000 7f7edea48640 1 -- 172.21.15.86:0/3306843758 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f7ed802d5c0 con 0x7f7ed8101b60 2024-04-21T08:41:33.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.424+0000 7f7ed57fa640 1 -- 172.21.15.86:0/3306843758 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7f7ecc09cb10 con 0x7f7ed8101b60 2024-04-21T08:41:33.423 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:41:33.423 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:38:45.202126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24398,24394,24455,24472]},"id":1}]} 2024-04-21T08:41:33.423 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-21T08:41:33.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.426+0000 7f7edea48640 1 -- 172.21.15.86:0/3306843758 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7ec00a3d20 msgr2=0x7f7ec00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.426+0000 7f7edea48640 1 --2- 172.21.15.86:0/3306843758 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7ec00a3d20 0x7f7ec00a61e0 secure :-1 s=READY pgs=138 cs=0 l=1 rev1=1 crypto rx=0x7f7ec8005ab0 tx=0x7f7ec8005fd0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.426+0000 7f7edea48640 1 -- 172.21.15.86:0/3306843758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed8101b60 msgr2=0x7f7ed8198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.426+0000 7f7edea48640 1 --2- 172.21.15.86:0/3306843758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed8101b60 0x7f7ed8198ca0 secure :-1 s=READY pgs=928 cs=0 l=1 rev1=1 crypto rx=0x7f7ecc00e9f0 tx=0x7f7ecc00eeb0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.426+0000 7f7edea48640 1 -- 172.21.15.86:0/3306843758 shutdown_connections 2024-04-21T08:41:33.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.426+0000 7f7edea48640 1 --2- 172.21.15.86:0/3306843758 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7ec00a3d20 0x7f7ec00a61e0 unknown :-1 s=CLOSED pgs=138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.426+0000 7f7edea48640 1 --2- 172.21.15.86:0/3306843758 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ed8105e30 0x7f7ed819d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.426+0000 7f7edea48640 1 --2- 172.21.15.86:0/3306843758 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ed8101b60 0x7f7ed8198ca0 unknown :-1 s=CLOSED pgs=928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.426+0000 7f7edea48640 1 --2- 172.21.15.86:0/3306843758 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ed81011a0 0x7f7ed8198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.426+0000 7f7edea48640 1 -- 172.21.15.86:0/3306843758 >> 172.21.15.86:0/3306843758 conn(0x7f7ed8077e30 msgr2=0x7f7ed80feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:33.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.426+0000 7f7edea48640 1 -- 172.21.15.86:0/3306843758 shutdown_connections 2024-04-21T08:41:33.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.426+0000 7f7edea48640 1 -- 172.21.15.86:0/3306843758 wait complete. 2024-04-21T08:41:33.436 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:41:33.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.536+0000 7f668b3f4640 1 -- 172.21.15.86:0/1175570535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6684101130 msgr2=0x7f6684101590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.536+0000 7f668b3f4640 1 --2- 172.21.15.86:0/1175570535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6684101130 0x7f6684101590 secure :-1 s=READY pgs=929 cs=0 l=1 rev1=1 crypto rx=0x7f6674009900 tx=0x7f667402bc80 comp rx=0 tx=0).stop 2024-04-21T08:41:33.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.536+0000 7f668b3f4640 1 -- 172.21.15.86:0/1175570535 shutdown_connections 2024-04-21T08:41:33.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.536+0000 7f668b3f4640 1 --2- 172.21.15.86:0/1175570535 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6684101ad0 0x7f668410dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.536+0000 7f668b3f4640 1 --2- 172.21.15.86:0/1175570535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6684101130 0x7f6684101590 unknown :-1 s=CLOSED pgs=929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.536+0000 7f668b3f4640 1 --2- 172.21.15.86:0/1175570535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6684105ea0 0x7f6684106280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.536+0000 7f668b3f4640 1 -- 172.21.15.86:0/1175570535 >> 172.21.15.86:0/1175570535 conn(0x7f6684077e30 msgr2=0x7f6684078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:33.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.536+0000 7f668b3f4640 1 -- 172.21.15.86:0/1175570535 shutdown_connections 2024-04-21T08:41:33.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.536+0000 7f668b3f4640 1 -- 172.21.15.86:0/1175570535 wait complete. 2024-04-21T08:41:33.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.536+0000 7f668b3f4640 1 Processor -- start 2024-04-21T08:41:33.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.537+0000 7f668b3f4640 1 -- start start 2024-04-21T08:41:33.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.537+0000 7f668b3f4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6684101130 0x7f6684199730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.537+0000 7f668b3f4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6684101ad0 0x7f6684199c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.537+0000 7f668b3f4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6684105ea0 0x7f66841a47b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.537+0000 7f668b3f4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6684111250 con 0x7f6684105ea0 2024-04-21T08:41:33.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.537+0000 7f668b3f4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f66841110d0 con 0x7f6684101130 2024-04-21T08:41:33.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.537+0000 7f668b3f4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f66841113d0 con 0x7f6684101ad0 2024-04-21T08:41:33.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.537+0000 7f6689991640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6684105ea0 0x7f66841a47b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.537+0000 7f6689991640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6684105ea0 0x7f66841a47b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48662/0 (socket says 172.21.15.86:48662) 2024-04-21T08:41:33.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.537+0000 7f6689991640 1 -- 172.21.15.86:0/3986490859 learned_addr learned my addr 172.21.15.86:0/3986490859 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:33.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.537+0000 7f6689190640 1 --2- 172.21.15.86:0/3986490859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6684101130 0x7f6684199730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.537+0000 7f6689991640 1 -- 172.21.15.86:0/3986490859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6684101ad0 msgr2=0x7f6684199c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.537+0000 7f6689991640 1 --2- 172.21.15.86:0/3986490859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6684101ad0 0x7f6684199c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.537+0000 7f6689991640 1 -- 172.21.15.86:0/3986490859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6684101130 msgr2=0x7f6684199730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.537+0000 7f6689991640 1 --2- 172.21.15.86:0/3986490859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6684101130 0x7f6684199730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.537+0000 7f6689991640 1 -- 172.21.15.86:0/3986490859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f66841a4cf0 con 0x7f6684105ea0 2024-04-21T08:41:33.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.537+0000 7f6689190640 1 --2- 172.21.15.86:0/3986490859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6684101130 0x7f6684199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:33.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.537+0000 7f6689991640 1 --2- 172.21.15.86:0/3986490859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6684105ea0 0x7f66841a47b0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f667c00eaa0 tx=0x7f667c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:33.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.537+0000 7f667a7fc640 1 -- 172.21.15.86:0/3986490859 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c00cdb0 con 0x7f6684105ea0 2024-04-21T08:41:33.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.537+0000 7f667a7fc640 1 -- 172.21.15.86:0/3986490859 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f667c005020 con 0x7f6684105ea0 2024-04-21T08:41:33.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.537+0000 7f668b3f4640 1 -- 172.21.15.86:0/3986490859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f66841a4ec0 con 0x7f6684105ea0 2024-04-21T08:41:33.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.537+0000 7f667a7fc640 1 -- 172.21.15.86:0/3986490859 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f667c0116d0 con 0x7f6684105ea0 2024-04-21T08:41:33.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.537+0000 7f668b3f4640 1 -- 172.21.15.86:0/3986490859 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f66841a5390 con 0x7f6684105ea0 2024-04-21T08:41:33.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.539+0000 7f667a7fc640 1 -- 172.21.15.86:0/3986490859 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f667c0051d0 con 0x7f6684105ea0 2024-04-21T08:41:33.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.540+0000 7f668dfe7740 1 Processor -- start 2024-04-21T08:41:33.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.540+0000 7f668dfe7740 1 -- start start 2024-04-21T08:41:33.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.540+0000 7f668dfe7740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a44179330 0x556a44179710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.540+0000 7f668dfe7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a44179ce0 0x556a44182820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.540+0000 7f668dfe7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a44182e40 0x556a441832c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.540+0000 7f668dfe7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556a440de1f0 con 0x556a44179ce0 2024-04-21T08:41:33.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.540+0000 7f668dfe7740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x556a4407b4b0 con 0x556a44179330 2024-04-21T08:41:33.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.540+0000 7f668dfe7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556a440da1b0 con 0x556a44182e40 2024-04-21T08:41:33.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.540+0000 7f667a7fc640 1 --2- 172.21.15.86:0/3986490859 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f66700a3d20 0x7f66700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.540+0000 7f6689190640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a44179ce0 0x556a44182820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.540+0000 7f6689190640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a44179ce0 0x556a44182820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48678/0 (socket says 172.21.15.86:48678) 2024-04-21T08:41:33.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.540+0000 7f6689190640 1 -- 172.21.15.86:0/831587099 learned_addr learned my addr 172.21.15.86:0/831587099 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:33.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.540+0000 7f668898f640 1 --2- 172.21.15.86:0/831587099 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a44179330 0x556a44179710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.540+0000 7f6689991640 1 --2- 172.21.15.86:0/3986490859 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f66700a3d20 0x7f66700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.541+0000 7f667a7fc640 1 -- 172.21.15.86:0/3986490859 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f667c0d0e40 con 0x7f6684105ea0 2024-04-21T08:41:33.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.541+0000 7f6689190640 1 -- 172.21.15.86:0/831587099 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a44182e40 msgr2=0x556a441832c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:33.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.541+0000 7f668898f640 1 --2- 172.21.15.86:0/831587099 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a44182e40 0x556a441832c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.541+0000 7f6689190640 1 --2- 172.21.15.86:0/831587099 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a44182e40 0x556a441832c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.541+0000 7f6689190640 1 -- 172.21.15.86:0/831587099 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a44179330 msgr2=0x556a44179710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:33.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.541+0000 7f6689190640 1 --2- 172.21.15.86:0/831587099 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a44179330 0x556a44179710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.541+0000 7f6689190640 1 -- 172.21.15.86:0/831587099 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a4406d1b0 con 0x556a44179ce0 2024-04-21T08:41:33.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.541+0000 7f668898f640 1 --2- 172.21.15.86:0/831587099 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a44182e40 0x556a441832c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:33.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.541+0000 7f6689190640 1 --2- 172.21.15.86:0/831587099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a44179ce0 0x556a44182820 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f668000aa10 tx=0x7f668000aed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f38fc3487880586d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:33.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.541+0000 7f6689991640 1 --2- 172.21.15.86:0/3986490859 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f66700a3d20 0x7f66700a61e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f667c09c770 tx=0x7f667c087040 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:33.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.541+0000 7f666f7fe640 1 -- 172.21.15.86:0/831587099 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6680015070 con 0x556a44179ce0 2024-04-21T08:41:33.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.541+0000 7f666f7fe640 1 -- 172.21.15.86:0/831587099 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f668000cc60 con 0x556a44179ce0 2024-04-21T08:41:33.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.541+0000 7f666f7fe640 1 -- 172.21.15.86:0/831587099 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6680018560 con 0x556a44179ce0 2024-04-21T08:41:33.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.542+0000 7f668dfe7740 1 -- 172.21.15.86:0/831587099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a44179ce0 msgr2=0x556a44182820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.542+0000 7f668dfe7740 1 --2- 172.21.15.86:0/831587099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a44179ce0 0x556a44182820 secure :-1 s=READY pgs=931 cs=0 l=1 rev1=1 crypto rx=0x7f668000aa10 tx=0x7f668000aed0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.542+0000 7f668dfe7740 1 -- 172.21.15.86:0/831587099 shutdown_connections 2024-04-21T08:41:33.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.542+0000 7f668dfe7740 1 --2- 172.21.15.86:0/831587099 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a44182e40 0x556a441832c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.542+0000 7f668dfe7740 1 --2- 172.21.15.86:0/831587099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a44179ce0 0x556a44182820 unknown :-1 s=CLOSED pgs=931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.542+0000 7f668dfe7740 1 --2- 172.21.15.86:0/831587099 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a44179330 0x556a44179710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.542+0000 7f668dfe7740 1 -- 172.21.15.86:0/831587099 >> 172.21.15.86:0/831587099 conn(0x556a44173ba0 msgr2=0x556a44173f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:33.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.542+0000 7f668dfe7740 1 -- 172.21.15.86:0/831587099 shutdown_connections 2024-04-21T08:41:33.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.542+0000 7f668dfe7740 1 -- 172.21.15.86:0/831587099 wait complete. 2024-04-21T08:41:33.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.542+0000 7f668dfe7740 1 -- start start 2024-04-21T08:41:33.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.542+0000 7f668dfe7740 1 Processor -- start 2024-04-21T08:41:33.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.542+0000 7f668dfe7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a44179330 0x556a441891b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.542+0000 7f668dfe7740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a44179ce0 0x556a44186540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.542+0000 7f668dfe7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a44182e40 0x556a44186a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.542+0000 7f668dfe7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556a44189dc0 con 0x556a44179330 2024-04-21T08:41:33.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.542+0000 7f668dfe7740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x556a44097500 con 0x556a44179ce0 2024-04-21T08:41:33.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.542+0000 7f668dfe7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556a44189f40 con 0x556a44182e40 2024-04-21T08:41:33.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.543+0000 7f668dfe7740 10 client.0 _ll_register_callbacks cb 0x556a44065b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:33.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.543+0000 7f668898f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a44179330 0x556a441891b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.543+0000 7f668898f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a44179330 0x556a441891b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48682/0 (socket says 172.21.15.86:48682) 2024-04-21T08:41:33.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.543+0000 7f668898f640 1 -- 172.21.15.86:0/107929971 learned_addr learned my addr 172.21.15.86:0/107929971 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:33.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.543+0000 7f6689190640 1 --2- 172.21.15.86:0/107929971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a44179ce0 0x556a44186540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.543+0000 7f668898f640 1 --2- 172.21.15.86:0/107929971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a44182e40 0x556a44186a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.543+0000 7f668898f640 1 -- 172.21.15.86:0/107929971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a44182e40 msgr2=0x556a44186a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:33.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.543+0000 7f668898f640 1 --2- 172.21.15.86:0/107929971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a44182e40 0x556a44186a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.543+0000 7f668898f640 1 -- 172.21.15.86:0/107929971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a44179ce0 msgr2=0x556a44186540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:33.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.543+0000 7f668898f640 1 --2- 172.21.15.86:0/107929971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a44179ce0 0x556a44186540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.543+0000 7f668898f640 1 -- 172.21.15.86:0/107929971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a43df03f0 con 0x556a44179330 2024-04-21T08:41:33.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.543+0000 7f668898f640 1 --2- 172.21.15.86:0/107929971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a44179330 0x556a441891b0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f668406b0c0 tx=0x7f6674005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6fe80d4497df9e55 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:33.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.543+0000 7f666cff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:33.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.543+0000 7f666cff9640 1 -- 172.21.15.86:0/107929971 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6674045070 con 0x556a44179330 2024-04-21T08:41:33.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.543+0000 7f666cff9640 1 -- 172.21.15.86:0/107929971 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f6674005240 con 0x556a44179330 2024-04-21T08:41:33.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.543+0000 7f668dfe7740 1 -- 172.21.15.86:0/107929971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f668000cc60 con 0x556a44179330 2024-04-21T08:41:33.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.543+0000 7f666cff9640 1 -- 172.21.15.86:0/107929971 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6674051050 con 0x556a44179330 2024-04-21T08:41:33.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.543+0000 7f668dfe7740 10 client.16962 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:33.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.543+0000 7f668dfe7740 1 -- 172.21.15.86:0/107929971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556a43eba7d0 con 0x556a44179330 2024-04-21T08:41:33.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.543+0000 7f666cff9640 1 -- 172.21.15.86:0/107929971 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f668000cc60 con 0x556a44179330 2024-04-21T08:41:33.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.544+0000 7f666cff9640 1 -- 172.21.15.86:0/107929971 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f66740035f0 con 0x556a44179330 2024-04-21T08:41:33.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.544+0000 7f668dfe7740 10 client.16962 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:33.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.544+0000 7f668dfe7740 10 client.16962 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:41:33.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.544+0000 7f668dfe7740 20 client.16962 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:33.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.544+0000 7f668dfe7740 1 --2- 172.21.15.86:0/107929971 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x556a44196ec0 0x556a44199310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.544+0000 7f668dfe7740 4 client.16962 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:33.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.544+0000 7f668dfe7740 1 -- 172.21.15.86:0/107929971 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556a4407d930 con 0x556a44196ec0 2024-04-21T08:41:33.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.544+0000 7f6689190640 1 --2- 172.21.15.86:0/107929971 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x556a44196ec0 0x556a44199310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.544+0000 7f6689190640 1 --2- 172.21.15.86:0/107929971 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x556a44196ec0 0x556a44199310 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a0e0480445b18f5d server_cookie=11b99d741b3a335d in_seq=0 out_seq=0 2024-04-21T08:41:33.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.544+0000 7f666cff9640 10 client.16962 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:41:33.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.545+0000 7f666cff9640 1 -- 172.21.15.86:0/107929971 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556a4407d930 con 0x556a44196ec0 2024-04-21T08:41:33.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.545+0000 7f666cff9640 10 client.16962 handle_command_reply: tid=0 2024-04-21T08:41:33.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.545+0000 7f668dfe7740 1 client.16962 shutdown 2024-04-21T08:41:33.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.545+0000 7f668dfe7740 1 -- 172.21.15.86:0/107929971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a44179330 msgr2=0x556a441891b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.545+0000 7f668dfe7740 1 --2- 172.21.15.86:0/107929971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a44179330 0x556a441891b0 secure :-1 s=READY pgs=932 cs=0 l=1 rev1=1 crypto rx=0x7f668406b0c0 tx=0x7f6674005fd0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.545+0000 7f668dfe7740 1 -- 172.21.15.86:0/107929971 shutdown_connections 2024-04-21T08:41:33.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.545+0000 7f668dfe7740 1 --2- 172.21.15.86:0/107929971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a44182e40 0x556a44186a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.545+0000 7f668dfe7740 1 --2- 172.21.15.86:0/107929971 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x556a44196ec0 0x556a44199310 crc :-1 s=READY pgs=42 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.545+0000 7f668dfe7740 1 --2- 172.21.15.86:0/107929971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a44179ce0 0x556a44186540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.545+0000 7f666cff9640 0 client.16962 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:41:33.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.545+0000 7f668dfe7740 1 --2- 172.21.15.86:0/107929971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a44179330 0x556a441891b0 unknown :-1 s=CLOSED pgs=932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.545+0000 7f668dfe7740 1 -- 172.21.15.86:0/107929971 >> 172.21.15.86:0/107929971 conn(0x556a44173ba0 msgr2=0x556a441758f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:33.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.546+0000 7f668dfe7740 1 -- 172.21.15.86:0/107929971 shutdown_connections 2024-04-21T08:41:33.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.546+0000 7f668dfe7740 1 -- 172.21.15.86:0/107929971 wait complete. 2024-04-21T08:41:33.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.546+0000 7f668dfe7740 20 client.16962 trim_cache size 0 max 16384 2024-04-21T08:41:33.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.559+0000 7f668dfe7740 1 Processor -- start 2024-04-21T08:41:33.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.559+0000 7f668dfe7740 1 -- start start 2024-04-21T08:41:33.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.559+0000 7f668dfe7740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a44179330 0x556a44192cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.559+0000 7f668dfe7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a44179ce0 0x556a441856c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.559+0000 7f668dfe7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a44182e40 0x556a44188c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.559+0000 7f668dfe7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f66841ad150 con 0x556a44179ce0 2024-04-21T08:41:33.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.559+0000 7f668dfe7740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f66841aec90 con 0x556a44179330 2024-04-21T08:41:33.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.559+0000 7f668dfe7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f66841ad820 con 0x556a44182e40 2024-04-21T08:41:33.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.559+0000 7f6689190640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a44179ce0 0x556a441856c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.559+0000 7f6689190640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a44179ce0 0x556a441856c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48692/0 (socket says 172.21.15.86:48692) 2024-04-21T08:41:33.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.559+0000 7f6689190640 1 -- 172.21.15.86:0/3675434786 learned_addr learned my addr 172.21.15.86:0/3675434786 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:33.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.559+0000 7f668898f640 1 --2- 172.21.15.86:0/3675434786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a44179330 0x556a44192cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.559+0000 7f6689190640 1 -- 172.21.15.86:0/3675434786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a44182e40 msgr2=0x556a44188c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:33.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.559+0000 7f6689190640 1 --2- 172.21.15.86:0/3675434786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a44182e40 0x556a44188c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.559+0000 7f6689190640 1 -- 172.21.15.86:0/3675434786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a44179330 msgr2=0x556a44192cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:33.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.559+0000 7f6689190640 1 --2- 172.21.15.86:0/3675434786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a44179330 0x556a44192cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.559+0000 7f6689190640 1 -- 172.21.15.86:0/3675434786 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556a44184ab0 con 0x556a44179ce0 2024-04-21T08:41:33.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.559+0000 7f6689190640 1 --2- 172.21.15.86:0/3675434786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a44179ce0 0x556a441856c0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x556a441786a0 tx=0x7f668000a180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ff8c37d89f25edf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:33.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.559+0000 7f666effd640 1 -- 172.21.15.86:0/3675434786 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6680028070 con 0x556a44179ce0 2024-04-21T08:41:33.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.560+0000 7f666effd640 1 -- 172.21.15.86:0/3675434786 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6680006be0 con 0x556a44179ce0 2024-04-21T08:41:33.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.560+0000 7f666effd640 1 -- 172.21.15.86:0/3675434786 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6680015040 con 0x556a44179ce0 2024-04-21T08:41:33.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.560+0000 7f668dfe7740 1 -- 172.21.15.86:0/3675434786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a44179ce0 msgr2=0x556a441856c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.560+0000 7f668dfe7740 1 --2- 172.21.15.86:0/3675434786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a44179ce0 0x556a441856c0 secure :-1 s=READY pgs=933 cs=0 l=1 rev1=1 crypto rx=0x556a441786a0 tx=0x7f668000a180 comp rx=0 tx=0).stop 2024-04-21T08:41:33.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.560+0000 7f668dfe7740 1 -- 172.21.15.86:0/3675434786 shutdown_connections 2024-04-21T08:41:33.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.560+0000 7f668dfe7740 1 --2- 172.21.15.86:0/3675434786 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a44182e40 0x556a44188c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.560+0000 7f668dfe7740 1 --2- 172.21.15.86:0/3675434786 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a44179ce0 0x556a441856c0 unknown :-1 s=CLOSED pgs=933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.560+0000 7f668dfe7740 1 --2- 172.21.15.86:0/3675434786 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a44179330 0x556a44192cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.560+0000 7f668dfe7740 1 -- 172.21.15.86:0/3675434786 >> 172.21.15.86:0/3675434786 conn(0x556a44173ba0 msgr2=0x556a44174ee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:33.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.560+0000 7f668dfe7740 1 -- 172.21.15.86:0/3675434786 shutdown_connections 2024-04-21T08:41:33.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.560+0000 7f668dfe7740 1 -- 172.21.15.86:0/3675434786 wait complete. 2024-04-21T08:41:33.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.560+0000 7f668dfe7740 1 -- start start 2024-04-21T08:41:33.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.560+0000 7f668dfe7740 1 Processor -- start 2024-04-21T08:41:33.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.561+0000 7f668dfe7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a44179330 0x556a4418f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.561+0000 7f668dfe7740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a44179ce0 0x556a44187d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.561+0000 7f668dfe7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a44182e40 0x556a44188290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.561+0000 7f668dfe7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f66841ae5c0 con 0x556a44182e40 2024-04-21T08:41:33.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.561+0000 7f668dfe7740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f66841adef0 con 0x556a44179ce0 2024-04-21T08:41:33.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.561+0000 7f668dfe7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f66841ab110 con 0x556a44179330 2024-04-21T08:41:33.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.561+0000 7f668dfe7740 10 client.0 _ll_register_callbacks cb 0x7f6674009610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:33.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.561+0000 7f668898f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a44182e40 0x556a44188290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.561+0000 7f668898f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a44182e40 0x556a44188290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48706/0 (socket says 172.21.15.86:48706) 2024-04-21T08:41:33.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.561+0000 7f668898f640 1 -- 172.21.15.86:0/1417214042 learned_addr learned my addr 172.21.15.86:0/1417214042 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:33.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.561+0000 7f668898f640 1 --2- 172.21.15.86:0/1417214042 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a44179330 0x556a4418f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.561+0000 7f6689190640 1 --2- 172.21.15.86:0/1417214042 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a44179ce0 0x556a44187d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.561+0000 7f668898f640 1 -- 172.21.15.86:0/1417214042 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a44179330 msgr2=0x556a4418f4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:33.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.561+0000 7f668898f640 1 --2- 172.21.15.86:0/1417214042 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a44179330 0x556a4418f4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.561+0000 7f668898f640 1 -- 172.21.15.86:0/1417214042 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a44179ce0 msgr2=0x556a44187d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:33.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.561+0000 7f668898f640 1 --2- 172.21.15.86:0/1417214042 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a44179ce0 0x556a44187d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.561+0000 7f668898f640 1 -- 172.21.15.86:0/1417214042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556a44188950 con 0x556a44182e40 2024-04-21T08:41:33.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.561+0000 7f668898f640 1 --2- 172.21.15.86:0/1417214042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a44182e40 0x556a44188290 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f6674009850 tx=0x7f6674005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=199ecaa3299899b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:33.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.561+0000 7f666cff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:33.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.561+0000 7f666cff9640 1 -- 172.21.15.86:0/1417214042 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6674045040 con 0x556a44182e40 2024-04-21T08:41:33.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.561+0000 7f666cff9640 1 -- 172.21.15.86:0/1417214042 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f6674034440 con 0x556a44182e40 2024-04-21T08:41:33.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.561+0000 7f668dfe7740 1 -- 172.21.15.86:0/1417214042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6680006be0 con 0x556a44182e40 2024-04-21T08:41:33.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.562+0000 7f666cff9640 1 -- 172.21.15.86:0/1417214042 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6674063050 con 0x556a44182e40 2024-04-21T08:41:33.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.562+0000 7f668dfe7740 10 client.16968 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:33.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.562+0000 7f668dfe7740 1 -- 172.21.15.86:0/1417214042 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556a441fd3a0 con 0x556a44182e40 2024-04-21T08:41:33.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.562+0000 7f666cff9640 1 -- 172.21.15.86:0/1417214042 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6680006be0 con 0x556a44182e40 2024-04-21T08:41:33.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.562+0000 7f666cff9640 1 -- 172.21.15.86:0/1417214042 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f667405e030 con 0x556a44182e40 2024-04-21T08:41:33.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.562+0000 7f668dfe7740 10 client.16968 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:33.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.562+0000 7f668dfe7740 10 client.16968 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:41:33.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.562+0000 7f668dfe7740 20 client.16968 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:33.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.562+0000 7f668dfe7740 1 --2- 172.21.15.86:0/1417214042 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x556a441fdaf0 0x556a441fff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.562+0000 7f668dfe7740 4 client.16968 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:41:33.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.562+0000 7f668dfe7740 1 -- 172.21.15.86:0/1417214042 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556a440ebc10 con 0x556a441fdaf0 2024-04-21T08:41:33.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.562+0000 7f6689190640 1 --2- 172.21.15.86:0/1417214042 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x556a441fdaf0 0x556a441fff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.563+0000 7f666cff9640 10 client.16968 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:41:33.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.563+0000 7f6689190640 1 --2- 172.21.15.86:0/1417214042 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x556a441fdaf0 0x556a441fff10 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=44cca7ab1cf64599 server_cookie=c7cce320197664af in_seq=0 out_seq=0 2024-04-21T08:41:33.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.563+0000 7f666cff9640 1 -- 172.21.15.86:0/1417214042 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556a440ebc10 con 0x556a441fdaf0 2024-04-21T08:41:33.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.563+0000 7f666cff9640 10 client.16968 handle_command_reply: tid=0 2024-04-21T08:41:33.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.563+0000 7f668dfe7740 1 client.16968 shutdown 2024-04-21T08:41:33.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.563+0000 7f668dfe7740 1 -- 172.21.15.86:0/1417214042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a44182e40 msgr2=0x556a44188290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.563+0000 7f668dfe7740 1 --2- 172.21.15.86:0/1417214042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a44182e40 0x556a44188290 secure :-1 s=READY pgs=934 cs=0 l=1 rev1=1 crypto rx=0x7f6674009850 tx=0x7f6674005c90 comp rx=0 tx=0).stop 2024-04-21T08:41:33.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.563+0000 7f668dfe7740 1 -- 172.21.15.86:0/1417214042 shutdown_connections 2024-04-21T08:41:33.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.563+0000 7f668dfe7740 1 --2- 172.21.15.86:0/1417214042 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556a44182e40 0x556a44188290 unknown :-1 s=CLOSED pgs=934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.563+0000 7f668dfe7740 1 --2- 172.21.15.86:0/1417214042 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x556a441fdaf0 0x556a441fff10 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.563+0000 7f668dfe7740 1 --2- 172.21.15.86:0/1417214042 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556a44179ce0 0x556a44187d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.563+0000 7f668dfe7740 1 --2- 172.21.15.86:0/1417214042 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556a44179330 0x556a4418f4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.563+0000 7f666cff9640 0 client.16968 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:41:33.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.563+0000 7f668dfe7740 1 -- 172.21.15.86:0/1417214042 >> 172.21.15.86:0/1417214042 conn(0x556a44173ba0 msgr2=0x556a44172c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:33.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.563+0000 7f668dfe7740 1 -- 172.21.15.86:0/1417214042 shutdown_connections 2024-04-21T08:41:33.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.563+0000 7f668dfe7740 1 -- 172.21.15.86:0/1417214042 wait complete. 2024-04-21T08:41:33.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.563+0000 7f668dfe7740 20 client.16968 trim_cache size 0 max 16384 2024-04-21T08:41:33.569 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:41:33.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.564+0000 7f668b3f4640 1 -- 172.21.15.86:0/3986490859 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f66700a3d20 msgr2=0x7f66700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.564+0000 7f668b3f4640 1 --2- 172.21.15.86:0/3986490859 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f66700a3d20 0x7f66700a61e0 secure :-1 s=READY pgs=139 cs=0 l=1 rev1=1 crypto rx=0x7f667c09c770 tx=0x7f667c087040 comp rx=0 tx=0).stop 2024-04-21T08:41:33.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.564+0000 7f668b3f4640 1 -- 172.21.15.86:0/3986490859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6684105ea0 msgr2=0x7f66841a47b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.564+0000 7f668b3f4640 1 --2- 172.21.15.86:0/3986490859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6684105ea0 0x7f66841a47b0 secure :-1 s=READY pgs=930 cs=0 l=1 rev1=1 crypto rx=0x7f667c00eaa0 tx=0x7f667c00ef60 comp rx=0 tx=0).stop 2024-04-21T08:41:33.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.564+0000 7f668b3f4640 1 -- 172.21.15.86:0/3986490859 shutdown_connections 2024-04-21T08:41:33.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.564+0000 7f668b3f4640 1 --2- 172.21.15.86:0/3986490859 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f66700a3d20 0x7f66700a61e0 unknown :-1 s=CLOSED pgs=139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.564+0000 7f668b3f4640 1 --2- 172.21.15.86:0/3986490859 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6684105ea0 0x7f66841a47b0 unknown :-1 s=CLOSED pgs=930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.564+0000 7f668b3f4640 1 --2- 172.21.15.86:0/3986490859 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6684101ad0 0x7f6684199c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.564+0000 7f668b3f4640 1 --2- 172.21.15.86:0/3986490859 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6684101130 0x7f6684199730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.564+0000 7f668b3f4640 1 -- 172.21.15.86:0/3986490859 >> 172.21.15.86:0/3986490859 conn(0x7f6684077e30 msgr2=0x7f66840fefc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:33.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.564+0000 7f668b3f4640 1 -- 172.21.15.86:0/3986490859 shutdown_connections 2024-04-21T08:41:33.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.564+0000 7f668b3f4640 1 -- 172.21.15.86:0/3986490859 wait complete. 2024-04-21T08:41:33.573 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:41:33.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.674+0000 7f2fc6332640 1 -- 172.21.15.86:0/1063401701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fc0105ea0 msgr2=0x7f2fc0106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.674+0000 7f2fc6332640 1 --2- 172.21.15.86:0/1063401701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fc0105ea0 0x7f2fc0106280 secure :-1 s=READY pgs=935 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0009900 tx=0x7f2fb002bcb0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.674+0000 7f2fc6332640 1 -- 172.21.15.86:0/1063401701 shutdown_connections 2024-04-21T08:41:33.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.674+0000 7f2fc6332640 1 --2- 172.21.15.86:0/1063401701 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2fc0101ad0 0x7f2fc010dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.674+0000 7f2fc6332640 1 --2- 172.21.15.86:0/1063401701 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fc0101130 0x7f2fc0101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.674+0000 7f2fc6332640 1 --2- 172.21.15.86:0/1063401701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fc0105ea0 0x7f2fc0106280 unknown :-1 s=CLOSED pgs=935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.674+0000 7f2fc6332640 1 -- 172.21.15.86:0/1063401701 >> 172.21.15.86:0/1063401701 conn(0x7f2fc0077e30 msgr2=0x7f2fc0078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:33.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.674+0000 7f2fc6332640 1 -- 172.21.15.86:0/1063401701 shutdown_connections 2024-04-21T08:41:33.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.674+0000 7f2fc6332640 1 -- 172.21.15.86:0/1063401701 wait complete. 2024-04-21T08:41:33.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.674+0000 7f2fc6332640 1 Processor -- start 2024-04-21T08:41:33.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.674+0000 7f2fc6332640 1 -- start start 2024-04-21T08:41:33.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.675+0000 7f2fc6332640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2fc0101130 0x7f2fc019e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.675+0000 7f2fc6332640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fc0101ad0 0x7f2fc019eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.675+0000 7f2fc6332640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fc0105ea0 0x7f2fc0198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.675+0000 7f2fc6332640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc0112370 con 0x7f2fc0105ea0 2024-04-21T08:41:33.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.675+0000 7f2fc6332640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc01121f0 con 0x7f2fc0101ad0 2024-04-21T08:41:33.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.675+0000 7f2fc6332640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc01124f0 con 0x7f2fc0101130 2024-04-21T08:41:33.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.675+0000 7f2fc48cf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fc0105ea0 0x7f2fc0198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.675+0000 7f2fc48cf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fc0105ea0 0x7f2fc0198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48728/0 (socket says 172.21.15.86:48728) 2024-04-21T08:41:33.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.675+0000 7f2fc48cf640 1 -- 172.21.15.86:0/2852667627 learned_addr learned my addr 172.21.15.86:0/2852667627 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:33.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.675+0000 7f2fbf7fe640 1 --2- 172.21.15.86:0/2852667627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fc0101ad0 0x7f2fc019eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.675+0000 7f2fbffff640 1 --2- 172.21.15.86:0/2852667627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2fc0101130 0x7f2fc019e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.675+0000 7f2fc48cf640 1 -- 172.21.15.86:0/2852667627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2fc0101130 msgr2=0x7f2fc019e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.675+0000 7f2fc48cf640 1 --2- 172.21.15.86:0/2852667627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2fc0101130 0x7f2fc019e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.675+0000 7f2fc48cf640 1 -- 172.21.15.86:0/2852667627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fc0101ad0 msgr2=0x7f2fc019eb60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.675+0000 7f2fc48cf640 1 --2- 172.21.15.86:0/2852667627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fc0101ad0 0x7f2fc019eb60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.675+0000 7f2fc48cf640 1 -- 172.21.15.86:0/2852667627 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fc0198ff0 con 0x7f2fc0105ea0 2024-04-21T08:41:33.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.675+0000 7f2fbf7fe640 1 --2- 172.21.15.86:0/2852667627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fc0101ad0 0x7f2fc019eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:33.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.675+0000 7f2fc48cf640 1 --2- 172.21.15.86:0/2852667627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fc0105ea0 0x7f2fc0198730 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f2fb800b990 tx=0x7f2fb800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:33.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.676+0000 7f2fbd7fa640 1 -- 172.21.15.86:0/2852667627 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb8019070 con 0x7f2fc0105ea0 2024-04-21T08:41:33.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.676+0000 7f2fc6332640 1 -- 172.21.15.86:0/2852667627 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fc01992e0 con 0x7f2fc0105ea0 2024-04-21T08:41:33.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.676+0000 7f2fbd7fa640 1 -- 172.21.15.86:0/2852667627 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fb800cba0 con 0x7f2fc0105ea0 2024-04-21T08:41:33.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.676+0000 7f2fbd7fa640 1 -- 172.21.15.86:0/2852667627 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb80075d0 con 0x7f2fc0105ea0 2024-04-21T08:41:33.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.676+0000 7f2fc6332640 1 -- 172.21.15.86:0/2852667627 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fc01a55a0 con 0x7f2fc0105ea0 2024-04-21T08:41:33.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.678+0000 7f2fbd7fa640 1 -- 172.21.15.86:0/2852667627 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2fb801c070 con 0x7f2fc0105ea0 2024-04-21T08:41:33.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.678+0000 7f2fc9003740 1 Processor -- start 2024-04-21T08:41:33.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.678+0000 7f2fc9003740 1 -- start start 2024-04-21T08:41:33.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.678+0000 7f2fc9003740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5648826bc330 0x5648826bc710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.678+0000 7f2fc9003740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648826bcce0 0x5648826c5820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.678+0000 7f2fc9003740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648826c5e40 0x5648826c62c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.678+0000 7f2fc9003740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5648826211f0 con 0x5648826bcce0 2024-04-21T08:41:33.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.678+0000 7f2fc9003740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5648825be4b0 con 0x5648826bc330 2024-04-21T08:41:33.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.678+0000 7f2fc9003740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56488261d1b0 con 0x5648826c5e40 2024-04-21T08:41:33.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.678+0000 7f2fbd7fa640 1 --2- 172.21.15.86:0/2852667627 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2fa80a3d20 0x7f2fa80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.679+0000 7f2fbffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648826bcce0 0x5648826c5820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.679+0000 7f2fbffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648826bcce0 0x5648826c5820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48744/0 (socket says 172.21.15.86:48744) 2024-04-21T08:41:33.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.679+0000 7f2fbffff640 1 -- 172.21.15.86:0/1641545749 learned_addr learned my addr 172.21.15.86:0/1641545749 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:33.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.679+0000 7f2fc48cf640 1 --2- 172.21.15.86:0/2852667627 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2fa80a3d20 0x7f2fa80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.679+0000 7f2fbf7fe640 1 --2- 172.21.15.86:0/1641545749 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5648826bc330 0x5648826bc710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.679+0000 7f2fbd7fa640 1 -- 172.21.15.86:0/2852667627 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f2fb80d0080 con 0x7f2fc0105ea0 2024-04-21T08:41:33.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.679+0000 7f2fbffff640 1 -- 172.21.15.86:0/1641545749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648826c5e40 msgr2=0x5648826c62c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:33.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.679+0000 7f2fbf7fe640 1 --2- 172.21.15.86:0/1641545749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648826c5e40 0x5648826c62c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.679+0000 7f2fbffff640 1 --2- 172.21.15.86:0/1641545749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648826c5e40 0x5648826c62c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.679+0000 7f2fbffff640 1 -- 172.21.15.86:0/1641545749 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5648826bc330 msgr2=0x5648826bc710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:33.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.679+0000 7f2fbffff640 1 --2- 172.21.15.86:0/1641545749 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5648826bc330 0x5648826bc710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.679+0000 7f2fbffff640 1 -- 172.21.15.86:0/1641545749 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648825b01b0 con 0x5648826bcce0 2024-04-21T08:41:33.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.679+0000 7f2fc48cf640 1 --2- 172.21.15.86:0/2852667627 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2fa80a3d20 0x7f2fa80a61e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2fb809bd50 tx=0x7f2fb8014c50 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:33.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.679+0000 7f2fbffff640 1 --2- 172.21.15.86:0/1641545749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648826bcce0 0x5648826c5820 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0006010 tx=0x7f2fb0005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8fb84766d805cd0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:33.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.679+0000 7f2fa67fc640 1 -- 172.21.15.86:0/1641545749 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb002c8a0 con 0x5648826bcce0 2024-04-21T08:41:33.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.679+0000 7f2fa67fc640 1 -- 172.21.15.86:0/1641545749 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fb002ca40 con 0x5648826bcce0 2024-04-21T08:41:33.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.679+0000 7f2fa67fc640 1 -- 172.21.15.86:0/1641545749 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb00046c0 con 0x5648826bcce0 2024-04-21T08:41:33.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.680+0000 7f2fc9003740 1 -- 172.21.15.86:0/1641545749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648826bcce0 msgr2=0x5648826c5820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.680+0000 7f2fc9003740 1 --2- 172.21.15.86:0/1641545749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648826bcce0 0x5648826c5820 secure :-1 s=READY pgs=937 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0006010 tx=0x7f2fb0005dd0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.680+0000 7f2fc9003740 1 -- 172.21.15.86:0/1641545749 shutdown_connections 2024-04-21T08:41:33.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.680+0000 7f2fc9003740 1 --2- 172.21.15.86:0/1641545749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648826c5e40 0x5648826c62c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.680+0000 7f2fc9003740 1 --2- 172.21.15.86:0/1641545749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648826bcce0 0x5648826c5820 unknown :-1 s=CLOSED pgs=937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.680+0000 7f2fc9003740 1 --2- 172.21.15.86:0/1641545749 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5648826bc330 0x5648826bc710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.680+0000 7f2fc9003740 1 -- 172.21.15.86:0/1641545749 >> 172.21.15.86:0/1641545749 conn(0x5648826b6ba0 msgr2=0x5648826b6f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:33.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.680+0000 7f2fc9003740 1 -- 172.21.15.86:0/1641545749 shutdown_connections 2024-04-21T08:41:33.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.680+0000 7f2fc9003740 1 -- 172.21.15.86:0/1641545749 wait complete. 2024-04-21T08:41:33.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.680+0000 7f2fc9003740 1 -- start start 2024-04-21T08:41:33.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.681+0000 7f2fc9003740 1 Processor -- start 2024-04-21T08:41:33.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.681+0000 7f2fc9003740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648826bc330 0x5648826cc1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.681+0000 7f2fc9003740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5648826bcce0 0x5648826c9580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.681+0000 7f2fc9003740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648826c5e40 0x5648826c9ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.681+0000 7f2fc9003740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5648826ccd60 con 0x5648826bc330 2024-04-21T08:41:33.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.681+0000 7f2fc9003740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5648825da500 con 0x5648826bcce0 2024-04-21T08:41:33.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.681+0000 7f2fc9003740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5648826ccee0 con 0x5648826c5e40 2024-04-21T08:41:33.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.681+0000 7f2fc9003740 10 client.0 _ll_register_callbacks cb 0x5648825a8b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:33.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.681+0000 7f2fbf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648826bc330 0x5648826cc1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.681+0000 7f2fbf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648826bc330 0x5648826cc1f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48746/0 (socket says 172.21.15.86:48746) 2024-04-21T08:41:33.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.681+0000 7f2fbf7fe640 1 -- 172.21.15.86:0/80316353 learned_addr learned my addr 172.21.15.86:0/80316353 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:33.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.681+0000 7f2fbf7fe640 1 -- 172.21.15.86:0/80316353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648826c5e40 msgr2=0x5648826c9ac0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:41:33.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.681+0000 7f2fbf7fe640 1 --2- 172.21.15.86:0/80316353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648826c5e40 0x5648826c9ac0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.681+0000 7f2fbffff640 1 --2- 172.21.15.86:0/80316353 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5648826bcce0 0x5648826c9580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.681+0000 7f2fbf7fe640 1 -- 172.21.15.86:0/80316353 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5648826bcce0 msgr2=0x5648826c9580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:33.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.681+0000 7f2fbf7fe640 1 --2- 172.21.15.86:0/80316353 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5648826bcce0 0x5648826c9580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.681+0000 7f2fbf7fe640 1 -- 172.21.15.86:0/80316353 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648823333f0 con 0x5648826bc330 2024-04-21T08:41:33.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.681+0000 7f2fbffff640 1 --2- 172.21.15.86:0/80316353 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5648826bcce0 0x5648826c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:33.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.681+0000 7f2fbf7fe640 1 --2- 172.21.15.86:0/80316353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648826bc330 0x5648826cc1f0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4011470 tx=0x7f2fb4011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=196e98ec7239959a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:33.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.681+0000 7f2f9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:33.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.681+0000 7f2f9f7fe640 1 -- 172.21.15.86:0/80316353 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb4012a40 con 0x5648826bc330 2024-04-21T08:41:33.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.681+0000 7f2f9f7fe640 1 -- 172.21.15.86:0/80316353 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f2fb4012be0 con 0x5648826bc330 2024-04-21T08:41:33.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.682+0000 7f2fc9003740 1 -- 172.21.15.86:0/80316353 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fb002ca40 con 0x5648826bc330 2024-04-21T08:41:33.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.682+0000 7f2f9f7fe640 1 -- 172.21.15.86:0/80316353 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2fb4012de0 con 0x5648826bc330 2024-04-21T08:41:33.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.682+0000 7f2fc9003740 10 client.16980 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:33.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.682+0000 7f2fc9003740 1 -- 172.21.15.86:0/80316353 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5648823fd7d0 con 0x5648826bc330 2024-04-21T08:41:33.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.682+0000 7f2f9f7fe640 1 -- 172.21.15.86:0/80316353 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fb002ca40 con 0x5648826bc330 2024-04-21T08:41:33.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.682+0000 7f2f9f7fe640 1 -- 172.21.15.86:0/80316353 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2fb4005430 con 0x5648826bc330 2024-04-21T08:41:33.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.682+0000 7f2fc9003740 10 client.16980 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:33.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.682+0000 7f2fc9003740 10 client.16980 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:41:33.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.682+0000 7f2fc9003740 20 client.16980 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:33.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.682+0000 7f2fc9003740 1 --2- 172.21.15.86:0/80316353 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x5648826da040 0x5648826dc4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.682+0000 7f2fc9003740 4 client.16980 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:33.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.682+0000 7f2fc9003740 1 -- 172.21.15.86:0/80316353 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648825c0930 con 0x5648826da040 2024-04-21T08:41:33.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.682+0000 7f2fbffff640 1 --2- 172.21.15.86:0/80316353 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x5648826da040 0x5648826dc4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.683+0000 7f2f9f7fe640 10 client.16980 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:41:33.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.683+0000 7f2fbffff640 1 --2- 172.21.15.86:0/80316353 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x5648826da040 0x5648826dc4c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5272c6a410d4b293 server_cookie=d2c5b97b9b1538e3 in_seq=0 out_seq=0 2024-04-21T08:41:33.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.683+0000 7f2f9f7fe640 1 -- 172.21.15.86:0/80316353 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5648825c0930 con 0x5648826da040 2024-04-21T08:41:33.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.683+0000 7f2f9f7fe640 10 client.16980 handle_command_reply: tid=0 2024-04-21T08:41:33.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.683+0000 7f2fc9003740 1 client.16980 shutdown 2024-04-21T08:41:33.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.683+0000 7f2fc9003740 1 -- 172.21.15.86:0/80316353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648826bc330 msgr2=0x5648826cc1f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.683+0000 7f2fc9003740 1 --2- 172.21.15.86:0/80316353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648826bc330 0x5648826cc1f0 secure :-1 s=READY pgs=938 cs=0 l=1 rev1=1 crypto rx=0x7f2fb4011470 tx=0x7f2fb4011930 comp rx=0 tx=0).stop 2024-04-21T08:41:33.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.684+0000 7f2fc9003740 1 -- 172.21.15.86:0/80316353 shutdown_connections 2024-04-21T08:41:33.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.684+0000 7f2fc9003740 1 --2- 172.21.15.86:0/80316353 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x5648826da040 0x5648826dc4c0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.684+0000 7f2fc9003740 1 --2- 172.21.15.86:0/80316353 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648826c5e40 0x5648826c9ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.684+0000 7f2fc9003740 1 --2- 172.21.15.86:0/80316353 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5648826bcce0 0x5648826c9580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.684+0000 7f2f9f7fe640 0 client.16980 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:41:33.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.684+0000 7f2fc9003740 1 --2- 172.21.15.86:0/80316353 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648826bc330 0x5648826cc1f0 unknown :-1 s=CLOSED pgs=938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.684+0000 7f2fc9003740 1 -- 172.21.15.86:0/80316353 >> 172.21.15.86:0/80316353 conn(0x5648826b6ba0 msgr2=0x5648826b88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:33.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.684+0000 7f2fc9003740 1 -- 172.21.15.86:0/80316353 shutdown_connections 2024-04-21T08:41:33.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.684+0000 7f2fc9003740 1 -- 172.21.15.86:0/80316353 wait complete. 2024-04-21T08:41:33.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.684+0000 7f2fc9003740 20 client.16980 trim_cache size 0 max 16384 2024-04-21T08:41:33.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.697+0000 7f2fc9003740 1 Processor -- start 2024-04-21T08:41:33.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.697+0000 7f2fc9003740 1 -- start start 2024-04-21T08:41:33.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.697+0000 7f2fc9003740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648826bc330 0x5648826c7230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.697+0000 7f2fc9003740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648826bcce0 0x5648826d5c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.697+0000 7f2fc9003740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5648826c5e40 0x5648826b5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.697+0000 7f2fc9003740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc01ad1c0 con 0x5648826bcce0 2024-04-21T08:41:33.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.697+0000 7f2fc9003740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc01aed00 con 0x5648826c5e40 2024-04-21T08:41:33.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.697+0000 7f2fc9003740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc01ad890 con 0x5648826bc330 2024-04-21T08:41:33.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.697+0000 7f2fbffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648826bcce0 0x5648826d5c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.697+0000 7f2fbffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648826bcce0 0x5648826d5c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48762/0 (socket says 172.21.15.86:48762) 2024-04-21T08:41:33.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.697+0000 7f2fbffff640 1 -- 172.21.15.86:0/2593345314 learned_addr learned my addr 172.21.15.86:0/2593345314 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:33.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.697+0000 7f2fbf7fe640 1 --2- 172.21.15.86:0/2593345314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648826bc330 0x5648826c7230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.697+0000 7f2fbffff640 1 -- 172.21.15.86:0/2593345314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648826bc330 msgr2=0x5648826c7230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:33.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.697+0000 7f2fbffff640 1 --2- 172.21.15.86:0/2593345314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648826bc330 0x5648826c7230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.697+0000 7f2fbffff640 1 -- 172.21.15.86:0/2593345314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5648826c5e40 msgr2=0x5648826b5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:33.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.697+0000 7f2fbffff640 1 --2- 172.21.15.86:0/2593345314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5648826c5e40 0x5648826b5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.697+0000 7f2fbffff640 1 -- 172.21.15.86:0/2593345314 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648826bca80 con 0x5648826bcce0 2024-04-21T08:41:33.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.697+0000 7f2fbf7fe640 1 --2- 172.21.15.86:0/2593345314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648826bc330 0x5648826c7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:33.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.697+0000 7f2fbffff640 1 --2- 172.21.15.86:0/2593345314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648826bcce0 0x5648826d5c90 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0005ec0 tx=0x7f2fb00053a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55393f2cede22b6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:33.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.698+0000 7f2fa5ffb640 1 -- 172.21.15.86:0/2593345314 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb00055c0 con 0x5648826bcce0 2024-04-21T08:41:33.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.698+0000 7f2fa5ffb640 1 -- 172.21.15.86:0/2593345314 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fb002c9c0 con 0x5648826bcce0 2024-04-21T08:41:33.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.698+0000 7f2fa5ffb640 1 -- 172.21.15.86:0/2593345314 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb004d6a0 con 0x5648826bcce0 2024-04-21T08:41:33.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.698+0000 7f2fc9003740 1 -- 172.21.15.86:0/2593345314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648826bcce0 msgr2=0x5648826d5c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.698+0000 7f2fc9003740 1 --2- 172.21.15.86:0/2593345314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648826bcce0 0x5648826d5c90 secure :-1 s=READY pgs=939 cs=0 l=1 rev1=1 crypto rx=0x7f2fb0005ec0 tx=0x7f2fb00053a0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.698+0000 7f2fc9003740 1 -- 172.21.15.86:0/2593345314 shutdown_connections 2024-04-21T08:41:33.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.698+0000 7f2fc9003740 1 --2- 172.21.15.86:0/2593345314 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5648826c5e40 0x5648826b5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.698+0000 7f2fc9003740 1 --2- 172.21.15.86:0/2593345314 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648826bcce0 0x5648826d5c90 unknown :-1 s=CLOSED pgs=939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.698+0000 7f2fc9003740 1 --2- 172.21.15.86:0/2593345314 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648826bc330 0x5648826c7230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.698+0000 7f2fc9003740 1 -- 172.21.15.86:0/2593345314 >> 172.21.15.86:0/2593345314 conn(0x5648826b6ba0 msgr2=0x5648826b5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:33.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.698+0000 7f2fc9003740 1 -- 172.21.15.86:0/2593345314 shutdown_connections 2024-04-21T08:41:33.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.698+0000 7f2fc9003740 1 -- 172.21.15.86:0/2593345314 wait complete. 2024-04-21T08:41:33.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.698+0000 7f2fc9003740 1 -- start start 2024-04-21T08:41:33.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.699+0000 7f2fc9003740 1 Processor -- start 2024-04-21T08:41:33.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.699+0000 7f2fc9003740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5648826bc330 0x5648826d2390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.699+0000 7f2fc9003740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648826bcce0 0x5648826cacb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.699+0000 7f2fc9003740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648826c5e40 0x5648826cb1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.699+0000 7f2fc9003740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc01ae630 con 0x5648826bcce0 2024-04-21T08:41:33.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.699+0000 7f2fc9003740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc01adf60 con 0x5648826bc330 2024-04-21T08:41:33.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.699+0000 7f2fc9003740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2fc01ab180 con 0x5648826c5e40 2024-04-21T08:41:33.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.699+0000 7f2fc9003740 10 client.0 _ll_register_callbacks cb 0x7f2fb400d950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:33.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.699+0000 7f2fbffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648826bcce0 0x5648826cacb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.699+0000 7f2fbffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648826bcce0 0x5648826cacb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48776/0 (socket says 172.21.15.86:48776) 2024-04-21T08:41:33.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.699+0000 7f2fbffff640 1 -- 172.21.15.86:0/2654914870 learned_addr learned my addr 172.21.15.86:0/2654914870 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:33.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.699+0000 7f2fbf7fe640 1 --2- 172.21.15.86:0/2654914870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648826c5e40 0x5648826cb1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.699+0000 7f2fbf7fe640 1 --2- 172.21.15.86:0/2654914870 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5648826bc330 0x5648826d2390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.699+0000 7f2fbffff640 1 -- 172.21.15.86:0/2654914870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648826c5e40 msgr2=0x5648826cb1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:33.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.699+0000 7f2fbffff640 1 --2- 172.21.15.86:0/2654914870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648826c5e40 0x5648826cb1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.699+0000 7f2fbffff640 1 -- 172.21.15.86:0/2654914870 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5648826bc330 msgr2=0x5648826d2390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:33.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.699+0000 7f2fbffff640 1 --2- 172.21.15.86:0/2654914870 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5648826bc330 0x5648826d2390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.699+0000 7f2fbffff640 1 -- 172.21.15.86:0/2654914870 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648826cb7e0 con 0x5648826bcce0 2024-04-21T08:41:33.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.699+0000 7f2fbf7fe640 1 --2- 172.21.15.86:0/2654914870 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5648826bc330 0x5648826d2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:33.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.699+0000 7f2fbffff640 1 --2- 172.21.15.86:0/2654914870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648826bcce0 0x5648826cacb0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f2fb004f040 tx=0x7f2fb0005410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62251537d8953575 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:33.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.699+0000 7f2f9f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:33.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.700+0000 7f2f9f7fe640 1 -- 172.21.15.86:0/2654914870 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fb0004040 con 0x5648826bcce0 2024-04-21T08:41:33.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.700+0000 7f2f9f7fe640 1 -- 172.21.15.86:0/2654914870 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f2fb0052040 con 0x5648826bcce0 2024-04-21T08:41:33.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.700+0000 7f2fc9003740 1 -- 172.21.15.86:0/2654914870 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fb002c9c0 con 0x5648826bcce0 2024-04-21T08:41:33.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.700+0000 7f2f9f7fe640 1 -- 172.21.15.86:0/2654914870 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2fb0063050 con 0x5648826bcce0 2024-04-21T08:41:33.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.700+0000 7f2fc9003740 10 client.16986 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:33.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.700+0000 7f2fc9003740 1 -- 172.21.15.86:0/2654914870 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564882740390 con 0x5648826bcce0 2024-04-21T08:41:33.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.700+0000 7f2f9f7fe640 1 -- 172.21.15.86:0/2654914870 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fb002c9c0 con 0x5648826bcce0 2024-04-21T08:41:33.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.700+0000 7f2f9f7fe640 1 -- 172.21.15.86:0/2654914870 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2fb0003800 con 0x5648826bcce0 2024-04-21T08:41:33.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.700+0000 7f2fc9003740 10 client.16986 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:33.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.700+0000 7f2fc9003740 10 client.16986 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:41:33.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.700+0000 7f2fc9003740 20 client.16986 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:33.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.700+0000 7f2fc9003740 1 --2- 172.21.15.86:0/2654914870 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x564882740680 0x564882742aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.700+0000 7f2fc9003740 4 client.16986 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:41:33.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.700+0000 7f2fc9003740 1 -- 172.21.15.86:0/2654914870 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56488262ec10 con 0x564882740680 2024-04-21T08:41:33.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.700+0000 7f2fbf7fe640 1 --2- 172.21.15.86:0/2654914870 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x564882740680 0x564882742aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.701+0000 7f2fbf7fe640 1 --2- 172.21.15.86:0/2654914870 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x564882740680 0x564882742aa0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=13c0d94264ec2aa3 server_cookie=ec5ad2554be61025 in_seq=0 out_seq=0 2024-04-21T08:41:33.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.701+0000 7f2f9f7fe640 10 client.16986 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:41:33.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.701+0000 7f2f9f7fe640 1 -- 172.21.15.86:0/2654914870 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56488262ec10 con 0x564882740680 2024-04-21T08:41:33.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.701+0000 7f2f9f7fe640 10 client.16986 handle_command_reply: tid=0 2024-04-21T08:41:33.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.701+0000 7f2fc9003740 1 client.16986 shutdown 2024-04-21T08:41:33.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.701+0000 7f2fc9003740 1 -- 172.21.15.86:0/2654914870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648826bcce0 msgr2=0x5648826cacb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.701+0000 7f2fc9003740 1 --2- 172.21.15.86:0/2654914870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648826bcce0 0x5648826cacb0 secure :-1 s=READY pgs=940 cs=0 l=1 rev1=1 crypto rx=0x7f2fb004f040 tx=0x7f2fb0005410 comp rx=0 tx=0).stop 2024-04-21T08:41:33.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.701+0000 7f2fc9003740 1 -- 172.21.15.86:0/2654914870 shutdown_connections 2024-04-21T08:41:33.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.701+0000 7f2fc9003740 1 --2- 172.21.15.86:0/2654914870 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x564882740680 0x564882742aa0 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.701+0000 7f2fc9003740 1 --2- 172.21.15.86:0/2654914870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5648826c5e40 0x5648826cb1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.701+0000 7f2f9f7fe640 0 client.16986 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:41:33.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.701+0000 7f2fc9003740 1 --2- 172.21.15.86:0/2654914870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5648826bcce0 0x5648826cacb0 unknown :-1 s=CLOSED pgs=940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.701+0000 7f2fc9003740 1 --2- 172.21.15.86:0/2654914870 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5648826bc330 0x5648826d2390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.701+0000 7f2fc9003740 1 -- 172.21.15.86:0/2654914870 >> 172.21.15.86:0/2654914870 conn(0x5648826b6ba0 msgr2=0x5648826b5c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:33.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.701+0000 7f2fc9003740 1 -- 172.21.15.86:0/2654914870 shutdown_connections 2024-04-21T08:41:33.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.701+0000 7f2fc9003740 1 -- 172.21.15.86:0/2654914870 wait complete. 2024-04-21T08:41:33.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.701+0000 7f2fc9003740 20 client.16986 trim_cache size 0 max 16384 2024-04-21T08:41:33.708 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:41:33.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.702+0000 7f2fc6332640 1 -- 172.21.15.86:0/2852667627 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2fa80a3d20 msgr2=0x7f2fa80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.702+0000 7f2fc6332640 1 --2- 172.21.15.86:0/2852667627 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2fa80a3d20 0x7f2fa80a61e0 secure :-1 s=READY pgs=140 cs=0 l=1 rev1=1 crypto rx=0x7f2fb809bd50 tx=0x7f2fb8014c50 comp rx=0 tx=0).stop 2024-04-21T08:41:33.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.702+0000 7f2fc6332640 1 -- 172.21.15.86:0/2852667627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fc0105ea0 msgr2=0x7f2fc0198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.702+0000 7f2fc6332640 1 --2- 172.21.15.86:0/2852667627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fc0105ea0 0x7f2fc0198730 secure :-1 s=READY pgs=936 cs=0 l=1 rev1=1 crypto rx=0x7f2fb800b990 tx=0x7f2fb800be50 comp rx=0 tx=0).stop 2024-04-21T08:41:33.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.702+0000 7f2fc6332640 1 -- 172.21.15.86:0/2852667627 shutdown_connections 2024-04-21T08:41:33.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.702+0000 7f2fc6332640 1 --2- 172.21.15.86:0/2852667627 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2fa80a3d20 0x7f2fa80a61e0 unknown :-1 s=CLOSED pgs=140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.702+0000 7f2fc6332640 1 --2- 172.21.15.86:0/2852667627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fc0105ea0 0x7f2fc0198730 unknown :-1 s=CLOSED pgs=936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.702+0000 7f2fc6332640 1 --2- 172.21.15.86:0/2852667627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fc0101ad0 0x7f2fc019eb60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.702+0000 7f2fc6332640 1 --2- 172.21.15.86:0/2852667627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2fc0101130 0x7f2fc019e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.702+0000 7f2fc6332640 1 -- 172.21.15.86:0/2852667627 >> 172.21.15.86:0/2852667627 conn(0x7f2fc0077e30 msgr2=0x7f2fc00fef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:33.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.702+0000 7f2fc6332640 1 -- 172.21.15.86:0/2852667627 shutdown_connections 2024-04-21T08:41:33.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.702+0000 7f2fc6332640 1 -- 172.21.15.86:0/2852667627 wait complete. 2024-04-21T08:41:33.712 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:41:33.813 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.812+0000 7f5287f75640 1 -- 172.21.15.86:0/3388919813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5280101090 msgr2=0x7f5280106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.812+0000 7f5287f75640 1 --2- 172.21.15.86:0/3388919813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5280101090 0x7f5280106bc0 secure :-1 s=READY pgs=941 cs=0 l=1 rev1=1 crypto rx=0x7f5270009900 tx=0x7f527002bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:33.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.812+0000 7f5287f75640 1 -- 172.21.15.86:0/3388919813 shutdown_connections 2024-04-21T08:41:33.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.812+0000 7f5287f75640 1 --2- 172.21.15.86:0/3388919813 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f52801072f0 0x7f528010ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.812+0000 7f5287f75640 1 --2- 172.21.15.86:0/3388919813 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5280101090 0x7f5280106bc0 unknown :-1 s=CLOSED pgs=941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.812+0000 7f5287f75640 1 --2- 172.21.15.86:0/3388919813 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52801006e0 0x7f5280100ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.812+0000 7f5287f75640 1 -- 172.21.15.86:0/3388919813 >> 172.21.15.86:0/3388919813 conn(0x7f52800773e0 msgr2=0x7f52800fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:33.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.812+0000 7f5287f75640 1 -- 172.21.15.86:0/3388919813 shutdown_connections 2024-04-21T08:41:33.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.812+0000 7f5287f75640 1 -- 172.21.15.86:0/3388919813 wait complete. 2024-04-21T08:41:33.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.812+0000 7f5287f75640 1 Processor -- start 2024-04-21T08:41:33.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.812+0000 7f5287f75640 1 -- start start 2024-04-21T08:41:33.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.813+0000 7f5287f75640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52801006e0 0x7f5280198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.814 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.813+0000 7f5287f75640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52801072f0 0x7f5280198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.813+0000 7f5287f75640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f528019cfe0 0x7f528019d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.813+0000 7f5287f75640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f52801121c0 con 0x7f52801006e0 2024-04-21T08:41:33.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.813+0000 7f5287f75640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5280112040 con 0x7f52801072f0 2024-04-21T08:41:33.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.813+0000 7f5287f75640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5280112340 con 0x7f528019cfe0 2024-04-21T08:41:33.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.813+0000 7f5285d11640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52801006e0 0x7f5280198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.813+0000 7f5285d11640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52801006e0 0x7f5280198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48790/0 (socket says 172.21.15.86:48790) 2024-04-21T08:41:33.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.813+0000 7f5285d11640 1 -- 172.21.15.86:0/126914837 learned_addr learned my addr 172.21.15.86:0/126914837 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:33.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.813+0000 7f5285510640 1 --2- 172.21.15.86:0/126914837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52801072f0 0x7f5280198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.813+0000 7f5286512640 1 --2- 172.21.15.86:0/126914837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f528019cfe0 0x7f528019d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.813+0000 7f5285d11640 1 -- 172.21.15.86:0/126914837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f528019cfe0 msgr2=0x7f528019d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.813+0000 7f5285d11640 1 --2- 172.21.15.86:0/126914837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f528019cfe0 0x7f528019d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.813+0000 7f5285d11640 1 -- 172.21.15.86:0/126914837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52801072f0 msgr2=0x7f5280198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.815 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.813+0000 7f5285d11640 1 --2- 172.21.15.86:0/126914837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52801072f0 0x7f5280198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.813+0000 7f5285d11640 1 -- 172.21.15.86:0/126914837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f528019dad0 con 0x7f52801006e0 2024-04-21T08:41:33.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.813+0000 7f5286512640 1 --2- 172.21.15.86:0/126914837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f528019cfe0 0x7f528019d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:33.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.813+0000 7f5285d11640 1 --2- 172.21.15.86:0/126914837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52801006e0 0x7f5280198720 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f527800cce0 tx=0x7f527800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:33.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.813+0000 7f5276ffd640 1 -- 172.21.15.86:0/126914837 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527800f650 con 0x7f52801006e0 2024-04-21T08:41:33.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.813+0000 7f5287f75640 1 -- 172.21.15.86:0/126914837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f528019ddc0 con 0x7f52801006e0 2024-04-21T08:41:33.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.813+0000 7f5276ffd640 1 -- 172.21.15.86:0/126914837 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f527800fca0 con 0x7f52801006e0 2024-04-21T08:41:33.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.813+0000 7f5276ffd640 1 -- 172.21.15.86:0/126914837 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5278018b40 con 0x7f52801006e0 2024-04-21T08:41:33.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.813+0000 7f5287f75640 1 -- 172.21.15.86:0/126914837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f52801a5580 con 0x7f52801006e0 2024-04-21T08:41:33.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.816+0000 7f5276ffd640 1 -- 172.21.15.86:0/126914837 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f527800f7f0 con 0x7f52801006e0 2024-04-21T08:41:33.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.816+0000 7f528ab65740 1 Processor -- start 2024-04-21T08:41:33.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.816+0000 7f528ab65740 1 -- start start 2024-04-21T08:41:33.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.816+0000 7f5276ffd640 1 --2- 172.21.15.86:0/126914837 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f526c0a3d20 0x7f526c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.816+0000 7f528ab65740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640a6e67330 0x5640a6e67710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.817+0000 7f528ab65740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a6e67ce0 0x5640a6e70820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.817+0000 7f528ab65740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640a6e70e40 0x5640a6e712c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.817+0000 7f528ab65740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5640a6dcc1f0 con 0x5640a6e67ce0 2024-04-21T08:41:33.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.817+0000 7f528ab65740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5640a6d694b0 con 0x5640a6e70e40 2024-04-21T08:41:33.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.817+0000 7f528ab65740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5640a6dc81b0 con 0x5640a6e67330 2024-04-21T08:41:33.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.817+0000 7f5276ffd640 1 -- 172.21.15.86:0/126914837 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f52780d1000 con 0x7f52801006e0 2024-04-21T08:41:33.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.817+0000 7f5286512640 1 --2- 172.21.15.86:0/126914837 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f526c0a3d20 0x7f526c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.817+0000 7f5285510640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a6e67ce0 0x5640a6e70820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.817+0000 7f5285510640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a6e67ce0 0x5640a6e70820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48806/0 (socket says 172.21.15.86:48806) 2024-04-21T08:41:33.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.817+0000 7f5285510640 1 -- 172.21.15.86:0/3641730450 learned_addr learned my addr 172.21.15.86:0/3641730450 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:33.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.817+0000 7f5286512640 1 --2- 172.21.15.86:0/3641730450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640a6e70e40 0x5640a6e712c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.817+0000 7f5285510640 1 -- 172.21.15.86:0/3641730450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640a6e67330 msgr2=0x5640a6e67710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:33.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.817+0000 7f5285510640 1 --2- 172.21.15.86:0/3641730450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640a6e67330 0x5640a6e67710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.817+0000 7f5285510640 1 -- 172.21.15.86:0/3641730450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640a6e70e40 msgr2=0x5640a6e712c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:33.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.817+0000 7f5285510640 1 --2- 172.21.15.86:0/3641730450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640a6e70e40 0x5640a6e712c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.817+0000 7f5285510640 1 -- 172.21.15.86:0/3641730450 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640a6d5b1b0 con 0x5640a6e67ce0 2024-04-21T08:41:33.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.817+0000 7f5286512640 1 --2- 172.21.15.86:0/126914837 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f526c0a3d20 0x7f526c0a61e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f527c009f10 tx=0x7f527c0147e0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:33.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.817+0000 7f5285510640 1 --2- 172.21.15.86:0/3641730450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a6e67ce0 0x5640a6e70820 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f5280069ee0 tx=0x7f527002c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d797864fdb8b30c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:33.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.817+0000 7f526bfff640 1 -- 172.21.15.86:0/3641730450 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5270045020 con 0x5640a6e67ce0 2024-04-21T08:41:33.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.817+0000 7f526bfff640 1 -- 172.21.15.86:0/3641730450 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f527002c7e0 con 0x5640a6e67ce0 2024-04-21T08:41:33.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.817+0000 7f526bfff640 1 -- 172.21.15.86:0/3641730450 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f52700046c0 con 0x5640a6e67ce0 2024-04-21T08:41:33.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.818+0000 7f528ab65740 1 -- 172.21.15.86:0/3641730450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a6e67ce0 msgr2=0x5640a6e70820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.818+0000 7f528ab65740 1 --2- 172.21.15.86:0/3641730450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a6e67ce0 0x5640a6e70820 secure :-1 s=READY pgs=943 cs=0 l=1 rev1=1 crypto rx=0x7f5280069ee0 tx=0x7f527002c6c0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.818+0000 7f528ab65740 1 -- 172.21.15.86:0/3641730450 shutdown_connections 2024-04-21T08:41:33.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.818+0000 7f528ab65740 1 --2- 172.21.15.86:0/3641730450 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640a6e70e40 0x5640a6e712c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.818+0000 7f528ab65740 1 --2- 172.21.15.86:0/3641730450 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a6e67ce0 0x5640a6e70820 unknown :-1 s=CLOSED pgs=943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.818+0000 7f528ab65740 1 --2- 172.21.15.86:0/3641730450 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640a6e67330 0x5640a6e67710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.818+0000 7f528ab65740 1 -- 172.21.15.86:0/3641730450 >> 172.21.15.86:0/3641730450 conn(0x5640a6e61ba0 msgr2=0x5640a6e61f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:33.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.818+0000 7f528ab65740 1 -- 172.21.15.86:0/3641730450 shutdown_connections 2024-04-21T08:41:33.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.818+0000 7f528ab65740 1 -- 172.21.15.86:0/3641730450 wait complete. 2024-04-21T08:41:33.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.818+0000 7f528ab65740 1 -- start start 2024-04-21T08:41:33.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.818+0000 7f528ab65740 1 Processor -- start 2024-04-21T08:41:33.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.819+0000 7f528ab65740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640a6e67330 0x5640a6e77240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.819+0000 7f528ab65740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640a6e67ce0 0x5640a6e745d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.819+0000 7f528ab65740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a6e70e40 0x5640a6e74b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.819+0000 7f528ab65740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5640a6e77d60 con 0x5640a6e70e40 2024-04-21T08:41:33.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.819+0000 7f528ab65740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5640a6d85500 con 0x5640a6e67330 2024-04-21T08:41:33.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.819+0000 7f528ab65740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5640a6e77ee0 con 0x5640a6e67ce0 2024-04-21T08:41:33.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.819+0000 7f528ab65740 10 client.0 _ll_register_callbacks cb 0x5640a6d53b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:33.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.819+0000 7f5286512640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a6e70e40 0x5640a6e74b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.819+0000 7f5286512640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a6e70e40 0x5640a6e74b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48808/0 (socket says 172.21.15.86:48808) 2024-04-21T08:41:33.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.819+0000 7f5286512640 1 -- 172.21.15.86:0/3668019759 learned_addr learned my addr 172.21.15.86:0/3668019759 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:33.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.819+0000 7f5285510640 1 --2- 172.21.15.86:0/3668019759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640a6e67ce0 0x5640a6e745d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.819+0000 7f5285d11640 1 --2- 172.21.15.86:0/3668019759 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640a6e67330 0x5640a6e77240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.819+0000 7f5286512640 1 -- 172.21.15.86:0/3668019759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640a6e67ce0 msgr2=0x5640a6e745d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:33.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.819+0000 7f5286512640 1 --2- 172.21.15.86:0/3668019759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640a6e67ce0 0x5640a6e745d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.819+0000 7f5286512640 1 -- 172.21.15.86:0/3668019759 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640a6e67330 msgr2=0x5640a6e77240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:33.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.819+0000 7f5286512640 1 --2- 172.21.15.86:0/3668019759 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640a6e67330 0x5640a6e77240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.819+0000 7f5286512640 1 -- 172.21.15.86:0/3668019759 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640a6ade3f0 con 0x5640a6e70e40 2024-04-21T08:41:33.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.819+0000 7f5285d11640 1 --2- 172.21.15.86:0/3668019759 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640a6e67330 0x5640a6e77240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:33.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.819+0000 7f5286512640 1 --2- 172.21.15.86:0/3668019759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a6e70e40 0x5640a6e74b30 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f527c00f5b0 tx=0x7f527c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf4005730b6a72f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:33.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.819+0000 7f52697fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:33.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.820+0000 7f52697fa640 1 -- 172.21.15.86:0/3668019759 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527c026070 con 0x5640a6e70e40 2024-04-21T08:41:33.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.820+0000 7f52697fa640 1 -- 172.21.15.86:0/3668019759 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f527c005220 con 0x5640a6e70e40 2024-04-21T08:41:33.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.820+0000 7f528ab65740 1 -- 172.21.15.86:0/3668019759 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f527002c7e0 con 0x5640a6e70e40 2024-04-21T08:41:33.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.820+0000 7f52697fa640 1 -- 172.21.15.86:0/3668019759 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f527c031050 con 0x5640a6e70e40 2024-04-21T08:41:33.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.820+0000 7f528ab65740 10 client.16998 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:33.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.820+0000 7f528ab65740 1 -- 172.21.15.86:0/3668019759 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5640a6ba87d0 con 0x5640a6e70e40 2024-04-21T08:41:33.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.820+0000 7f52697fa640 1 -- 172.21.15.86:0/3668019759 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f527002c7e0 con 0x5640a6e70e40 2024-04-21T08:41:33.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.820+0000 7f52697fa640 1 -- 172.21.15.86:0/3668019759 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f527c00f040 con 0x5640a6e70e40 2024-04-21T08:41:33.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.820+0000 7f528ab65740 10 client.16998 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:33.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.820+0000 7f528ab65740 10 client.16998 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:41:33.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.820+0000 7f528ab65740 20 client.16998 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:33.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.820+0000 7f528ab65740 1 --2- 172.21.15.86:0/3668019759 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5640a6e84fd0 0x5640a6e87470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.820+0000 7f528ab65740 4 client.16998 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:33.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.820+0000 7f528ab65740 1 -- 172.21.15.86:0/3668019759 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640a6d6b930 con 0x5640a6e84fd0 2024-04-21T08:41:33.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.820+0000 7f5285d11640 1 --2- 172.21.15.86:0/3668019759 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5640a6e84fd0 0x5640a6e87470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.821+0000 7f52697fa640 10 client.16998 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:41:33.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.821+0000 7f5285d11640 1 --2- 172.21.15.86:0/3668019759 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5640a6e84fd0 0x5640a6e87470 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b65f775c11af5cda server_cookie=cf6c2a51ceef96f5 in_seq=0 out_seq=0 2024-04-21T08:41:33.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.821+0000 7f52697fa640 1 -- 172.21.15.86:0/3668019759 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5640a6d6b930 con 0x5640a6e84fd0 2024-04-21T08:41:33.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.821+0000 7f52697fa640 10 client.16998 handle_command_reply: tid=0 2024-04-21T08:41:33.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.821+0000 7f528ab65740 1 client.16998 shutdown 2024-04-21T08:41:33.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.822+0000 7f528ab65740 1 -- 172.21.15.86:0/3668019759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a6e70e40 msgr2=0x5640a6e74b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.822+0000 7f528ab65740 1 --2- 172.21.15.86:0/3668019759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a6e70e40 0x5640a6e74b30 secure :-1 s=READY pgs=944 cs=0 l=1 rev1=1 crypto rx=0x7f527c00f5b0 tx=0x7f527c01a000 comp rx=0 tx=0).stop 2024-04-21T08:41:33.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.822+0000 7f528ab65740 1 -- 172.21.15.86:0/3668019759 shutdown_connections 2024-04-21T08:41:33.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.822+0000 7f528ab65740 1 --2- 172.21.15.86:0/3668019759 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5640a6e84fd0 0x5640a6e87470 crc :-1 s=READY pgs=43 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.822+0000 7f528ab65740 1 --2- 172.21.15.86:0/3668019759 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a6e70e40 0x5640a6e74b30 unknown :-1 s=CLOSED pgs=944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.822+0000 7f52697fa640 0 client.16998 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:41:33.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.822+0000 7f528ab65740 1 --2- 172.21.15.86:0/3668019759 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640a6e67ce0 0x5640a6e745d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.822+0000 7f528ab65740 1 --2- 172.21.15.86:0/3668019759 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640a6e67330 0x5640a6e77240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.822+0000 7f528ab65740 1 -- 172.21.15.86:0/3668019759 >> 172.21.15.86:0/3668019759 conn(0x5640a6e61ba0 msgr2=0x5640a6e638f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:33.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.822+0000 7f528ab65740 1 -- 172.21.15.86:0/3668019759 shutdown_connections 2024-04-21T08:41:33.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.822+0000 7f528ab65740 1 -- 172.21.15.86:0/3668019759 wait complete. 2024-04-21T08:41:33.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.822+0000 7f528ab65740 20 client.16998 trim_cache size 0 max 16384 2024-04-21T08:41:33.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.835+0000 7f528ab65740 1 Processor -- start 2024-04-21T08:41:33.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.835+0000 7f528ab65740 1 -- start start 2024-04-21T08:41:33.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.835+0000 7f528ab65740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640a6e67330 0x5640a6e80c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.835+0000 7f528ab65740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a6e67ce0 0x5640a6e6f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.835+0000 7f528ab65740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640a6e70e40 0x5640a6e73730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.835+0000 7f528ab65740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f52801acfd0 con 0x5640a6e67ce0 2024-04-21T08:41:33.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.835+0000 7f528ab65740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f52801aeb10 con 0x5640a6e67330 2024-04-21T08:41:33.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.835+0000 7f528ab65740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f52801ad6a0 con 0x5640a6e70e40 2024-04-21T08:41:33.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.835+0000 7f5285510640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a6e67ce0 0x5640a6e6f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.835+0000 7f5285510640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a6e67ce0 0x5640a6e6f800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48816/0 (socket says 172.21.15.86:48816) 2024-04-21T08:41:33.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.835+0000 7f5285510640 1 -- 172.21.15.86:0/3757407620 learned_addr learned my addr 172.21.15.86:0/3757407620 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:33.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.835+0000 7f5285d11640 1 --2- 172.21.15.86:0/3757407620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640a6e67330 0x5640a6e80c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.835+0000 7f5285510640 1 -- 172.21.15.86:0/3757407620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640a6e70e40 msgr2=0x5640a6e73730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:33.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.835+0000 7f5285510640 1 --2- 172.21.15.86:0/3757407620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640a6e70e40 0x5640a6e73730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.835+0000 7f5285510640 1 -- 172.21.15.86:0/3757407620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640a6e67330 msgr2=0x5640a6e80c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:33.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.835+0000 7f5285510640 1 --2- 172.21.15.86:0/3757407620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640a6e67330 0x5640a6e80c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.835+0000 7f5285510640 1 -- 172.21.15.86:0/3757407620 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640a6e62ba0 con 0x5640a6e67ce0 2024-04-21T08:41:33.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.836+0000 7f5285510640 1 --2- 172.21.15.86:0/3757407620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a6e67ce0 0x5640a6e6f800 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f5270009900 tx=0x7f5270000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21742ac39d770ca0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:33.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.836+0000 7f526b7fe640 1 -- 172.21.15.86:0/3757407620 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527004e070 con 0x5640a6e67ce0 2024-04-21T08:41:33.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.836+0000 7f526b7fe640 1 -- 172.21.15.86:0/3757407620 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f527002cc90 con 0x5640a6e67ce0 2024-04-21T08:41:33.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.836+0000 7f526b7fe640 1 -- 172.21.15.86:0/3757407620 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5270045070 con 0x5640a6e67ce0 2024-04-21T08:41:33.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.836+0000 7f528ab65740 1 -- 172.21.15.86:0/3757407620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a6e67ce0 msgr2=0x5640a6e6f800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.836+0000 7f528ab65740 1 --2- 172.21.15.86:0/3757407620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a6e67ce0 0x5640a6e6f800 secure :-1 s=READY pgs=945 cs=0 l=1 rev1=1 crypto rx=0x7f5270009900 tx=0x7f5270000bc0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.836+0000 7f528ab65740 1 -- 172.21.15.86:0/3757407620 shutdown_connections 2024-04-21T08:41:33.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.836+0000 7f528ab65740 1 --2- 172.21.15.86:0/3757407620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640a6e70e40 0x5640a6e73730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.836+0000 7f528ab65740 1 --2- 172.21.15.86:0/3757407620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a6e67ce0 0x5640a6e6f800 unknown :-1 s=CLOSED pgs=945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.836+0000 7f528ab65740 1 --2- 172.21.15.86:0/3757407620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640a6e67330 0x5640a6e80c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.836+0000 7f528ab65740 1 -- 172.21.15.86:0/3757407620 >> 172.21.15.86:0/3757407620 conn(0x5640a6e61ba0 msgr2=0x5640a6e60370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:33.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.836+0000 7f528ab65740 1 -- 172.21.15.86:0/3757407620 shutdown_connections 2024-04-21T08:41:33.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.836+0000 7f528ab65740 1 -- 172.21.15.86:0/3757407620 wait complete. 2024-04-21T08:41:33.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.837+0000 7f528ab65740 1 -- start start 2024-04-21T08:41:33.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.837+0000 7f528ab65740 1 Processor -- start 2024-04-21T08:41:33.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.837+0000 7f528ab65740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640a6e67330 0x5640a6e7d400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.837+0000 7f528ab65740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640a6e67ce0 0x5640a6e75da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.837+0000 7f528ab65740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a6e70e40 0x5640a6e762e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.837+0000 7f528ab65740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f52801ae440 con 0x5640a6e70e40 2024-04-21T08:41:33.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.837+0000 7f528ab65740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f52801add70 con 0x5640a6e67ce0 2024-04-21T08:41:33.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.837+0000 7f528ab65740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f52801aaf90 con 0x5640a6e67330 2024-04-21T08:41:33.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.837+0000 7f528ab65740 10 client.0 _ll_register_callbacks cb 0x7f527c00f490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:33.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.837+0000 7f5286512640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a6e70e40 0x5640a6e762e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.837+0000 7f5286512640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a6e70e40 0x5640a6e762e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48820/0 (socket says 172.21.15.86:48820) 2024-04-21T08:41:33.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.837+0000 7f5286512640 1 -- 172.21.15.86:0/2659632341 learned_addr learned my addr 172.21.15.86:0/2659632341 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:33.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.837+0000 7f5285510640 1 --2- 172.21.15.86:0/2659632341 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640a6e67ce0 0x5640a6e75da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.837+0000 7f5285d11640 1 --2- 172.21.15.86:0/2659632341 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640a6e67330 0x5640a6e7d400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.837+0000 7f5286512640 1 -- 172.21.15.86:0/2659632341 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640a6e67330 msgr2=0x5640a6e7d400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:33.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.837+0000 7f5286512640 1 --2- 172.21.15.86:0/2659632341 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640a6e67330 0x5640a6e7d400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.837+0000 7f5286512640 1 -- 172.21.15.86:0/2659632341 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640a6e67ce0 msgr2=0x5640a6e75da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:33.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.837+0000 7f5286512640 1 --2- 172.21.15.86:0/2659632341 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640a6e67ce0 0x5640a6e75da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.838+0000 7f5286512640 1 -- 172.21.15.86:0/2659632341 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640a6e76950 con 0x5640a6e70e40 2024-04-21T08:41:33.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.838+0000 7f5285510640 1 --2- 172.21.15.86:0/2659632341 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640a6e67ce0 0x5640a6e75da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:33.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.838+0000 7f5286512640 1 --2- 172.21.15.86:0/2659632341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a6e70e40 0x5640a6e762e0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f527c0096c0 tx=0x7f527c009a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c33fbb820f1a9c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:33.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.838+0000 7f52697fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:33.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.838+0000 7f52697fa640 1 -- 172.21.15.86:0/2659632341 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f527c03a060 con 0x5640a6e70e40 2024-04-21T08:41:33.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.838+0000 7f52697fa640 1 -- 172.21.15.86:0/2659632341 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f527c005040 con 0x5640a6e70e40 2024-04-21T08:41:33.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.838+0000 7f528ab65740 1 -- 172.21.15.86:0/2659632341 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f527002cc90 con 0x5640a6e70e40 2024-04-21T08:41:33.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.838+0000 7f52697fa640 1 -- 172.21.15.86:0/2659632341 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f527c041050 con 0x5640a6e70e40 2024-04-21T08:41:33.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.838+0000 7f528ab65740 10 client.17004 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:33.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.838+0000 7f528ab65740 1 -- 172.21.15.86:0/2659632341 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5640a6eeb470 con 0x5640a6e70e40 2024-04-21T08:41:33.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.838+0000 7f52697fa640 1 -- 172.21.15.86:0/2659632341 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f527002cc90 con 0x5640a6e70e40 2024-04-21T08:41:33.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.838+0000 7f52697fa640 1 -- 172.21.15.86:0/2659632341 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f527c004860 con 0x5640a6e70e40 2024-04-21T08:41:33.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.838+0000 7f528ab65740 10 client.17004 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:33.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.838+0000 7f528ab65740 10 client.17004 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:41:33.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.838+0000 7f528ab65740 20 client.17004 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:33.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.838+0000 7f528ab65740 1 --2- 172.21.15.86:0/2659632341 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5640a6eebbc0 0x5640a6eedfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.838+0000 7f528ab65740 4 client.17004 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:41:33.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.838+0000 7f528ab65740 1 -- 172.21.15.86:0/2659632341 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5640a6dd9c10 con 0x5640a6eebbc0 2024-04-21T08:41:33.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.838+0000 7f5285d11640 1 --2- 172.21.15.86:0/2659632341 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5640a6eebbc0 0x5640a6eedfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.839+0000 7f52697fa640 10 client.17004 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:41:33.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.839+0000 7f5285d11640 1 --2- 172.21.15.86:0/2659632341 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5640a6eebbc0 0x5640a6eedfe0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=da8dfca45556a5b9 server_cookie=8c5ab86ffe9cfbe3 in_seq=0 out_seq=0 2024-04-21T08:41:33.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.839+0000 7f52697fa640 1 -- 172.21.15.86:0/2659632341 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5640a6dd9c10 con 0x5640a6eebbc0 2024-04-21T08:41:33.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.839+0000 7f52697fa640 10 client.17004 handle_command_reply: tid=0 2024-04-21T08:41:33.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.839+0000 7f528ab65740 1 client.17004 shutdown 2024-04-21T08:41:33.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.839+0000 7f528ab65740 1 -- 172.21.15.86:0/2659632341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a6e70e40 msgr2=0x5640a6e762e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.839+0000 7f528ab65740 1 --2- 172.21.15.86:0/2659632341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a6e70e40 0x5640a6e762e0 secure :-1 s=READY pgs=946 cs=0 l=1 rev1=1 crypto rx=0x7f527c0096c0 tx=0x7f527c009a40 comp rx=0 tx=0).stop 2024-04-21T08:41:33.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.839+0000 7f528ab65740 1 -- 172.21.15.86:0/2659632341 shutdown_connections 2024-04-21T08:41:33.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.839+0000 7f528ab65740 1 --2- 172.21.15.86:0/2659632341 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5640a6eebbc0 0x5640a6eedfe0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.839+0000 7f528ab65740 1 --2- 172.21.15.86:0/2659632341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640a6e70e40 0x5640a6e762e0 unknown :-1 s=CLOSED pgs=946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.839+0000 7f528ab65740 1 --2- 172.21.15.86:0/2659632341 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640a6e67ce0 0x5640a6e75da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.839+0000 7f528ab65740 1 --2- 172.21.15.86:0/2659632341 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640a6e67330 0x5640a6e7d400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.839+0000 7f52697fa640 0 client.17004 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:41:33.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.839+0000 7f528ab65740 1 -- 172.21.15.86:0/2659632341 >> 172.21.15.86:0/2659632341 conn(0x5640a6e61ba0 msgr2=0x5640a6e60370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:33.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.839+0000 7f528ab65740 1 -- 172.21.15.86:0/2659632341 shutdown_connections 2024-04-21T08:41:33.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.839+0000 7f528ab65740 1 -- 172.21.15.86:0/2659632341 wait complete. 2024-04-21T08:41:33.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.839+0000 7f528ab65740 20 client.17004 trim_cache size 0 max 16384 2024-04-21T08:41:33.846 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:41:33.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.840+0000 7f5287f75640 1 -- 172.21.15.86:0/126914837 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f526c0a3d20 msgr2=0x7f526c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.840+0000 7f5287f75640 1 --2- 172.21.15.86:0/126914837 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f526c0a3d20 0x7f526c0a61e0 secure :-1 s=READY pgs=141 cs=0 l=1 rev1=1 crypto rx=0x7f527c009f10 tx=0x7f527c0147e0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.840+0000 7f5287f75640 1 -- 172.21.15.86:0/126914837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52801006e0 msgr2=0x7f5280198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.840+0000 7f5287f75640 1 --2- 172.21.15.86:0/126914837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52801006e0 0x7f5280198720 secure :-1 s=READY pgs=942 cs=0 l=1 rev1=1 crypto rx=0x7f527800cce0 tx=0x7f527800a620 comp rx=0 tx=0).stop 2024-04-21T08:41:33.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.840+0000 7f5287f75640 1 -- 172.21.15.86:0/126914837 shutdown_connections 2024-04-21T08:41:33.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.840+0000 7f5287f75640 1 --2- 172.21.15.86:0/126914837 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f526c0a3d20 0x7f526c0a61e0 unknown :-1 s=CLOSED pgs=141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.840+0000 7f5287f75640 1 --2- 172.21.15.86:0/126914837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f528019cfe0 0x7f528019d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.840+0000 7f5287f75640 1 --2- 172.21.15.86:0/126914837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f52801072f0 0x7f5280198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.840+0000 7f5287f75640 1 --2- 172.21.15.86:0/126914837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f52801006e0 0x7f5280198720 unknown :-1 s=CLOSED pgs=942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.840+0000 7f5287f75640 1 -- 172.21.15.86:0/126914837 >> 172.21.15.86:0/126914837 conn(0x7f52800773e0 msgr2=0x7f5280105500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:33.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.840+0000 7f5287f75640 1 -- 172.21.15.86:0/126914837 shutdown_connections 2024-04-21T08:41:33.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.840+0000 7f5287f75640 1 -- 172.21.15.86:0/126914837 wait complete. 2024-04-21T08:41:33.849 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:41:33.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.951+0000 7f44f785c640 1 -- 172.21.15.86:0/181075439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44f0102bc0 msgr2=0x7f44f0106c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.951+0000 7f44f785c640 1 --2- 172.21.15.86:0/181075439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44f0102bc0 0x7f44f0106c00 secure :-1 s=READY pgs=947 cs=0 l=1 rev1=1 crypto rx=0x7f44e0009900 tx=0x7f44e002bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:33.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.951+0000 7f44f785c640 1 -- 172.21.15.86:0/181075439 shutdown_connections 2024-04-21T08:41:33.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.951+0000 7f44f785c640 1 --2- 172.21.15.86:0/181075439 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44f01072e0 0x7f44f010eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.951+0000 7f44f785c640 1 --2- 172.21.15.86:0/181075439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44f0102bc0 0x7f44f0106c00 unknown :-1 s=CLOSED pgs=947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.951+0000 7f44f785c640 1 --2- 172.21.15.86:0/181075439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44f0102210 0x7f44f01025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.951+0000 7f44f785c640 1 -- 172.21.15.86:0/181075439 >> 172.21.15.86:0/181075439 conn(0x7f44f00fdab0 msgr2=0x7f44f00ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:33.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.951+0000 7f44f785c640 1 -- 172.21.15.86:0/181075439 shutdown_connections 2024-04-21T08:41:33.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.951+0000 7f44f785c640 1 -- 172.21.15.86:0/181075439 wait complete. 2024-04-21T08:41:33.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.951+0000 7f44f785c640 1 Processor -- start 2024-04-21T08:41:33.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.951+0000 7f44f785c640 1 -- start start 2024-04-21T08:41:33.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.952+0000 7f44f785c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44f0102210 0x7f44f019e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.952+0000 7f44f785c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44f01072e0 0x7f44f019e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.952+0000 7f44f785c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44f0198670 0x7f44f0198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.952+0000 7f44f785c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44f0112220 con 0x7f44f0102210 2024-04-21T08:41:33.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.952+0000 7f44f785c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f44f01120a0 con 0x7f44f01072e0 2024-04-21T08:41:33.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.952+0000 7f44f785c640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f44f01123a0 con 0x7f44f0198670 2024-04-21T08:41:33.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.952+0000 7f44f55f8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44f0102210 0x7f44f019e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.952+0000 7f44f55f8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44f0102210 0x7f44f019e470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48844/0 (socket says 172.21.15.86:48844) 2024-04-21T08:41:33.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.952+0000 7f44f55f8640 1 -- 172.21.15.86:0/3445278670 learned_addr learned my addr 172.21.15.86:0/3445278670 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:33.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.952+0000 7f44f4df7640 1 --2- 172.21.15.86:0/3445278670 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44f01072e0 0x7f44f019e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.952+0000 7f44f5df9640 1 --2- 172.21.15.86:0/3445278670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44f0198670 0x7f44f0198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.952+0000 7f44f55f8640 1 -- 172.21.15.86:0/3445278670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44f0198670 msgr2=0x7f44f0198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.952+0000 7f44f55f8640 1 --2- 172.21.15.86:0/3445278670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44f0198670 0x7f44f0198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.952+0000 7f44f55f8640 1 -- 172.21.15.86:0/3445278670 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44f01072e0 msgr2=0x7f44f019e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.952+0000 7f44f55f8640 1 --2- 172.21.15.86:0/3445278670 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44f01072e0 0x7f44f019e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.952+0000 7f44f55f8640 1 -- 172.21.15.86:0/3445278670 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44f0199190 con 0x7f44f0102210 2024-04-21T08:41:33.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.952+0000 7f44f5df9640 1 --2- 172.21.15.86:0/3445278670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44f0198670 0x7f44f0198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:33.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.952+0000 7f44f55f8640 1 --2- 172.21.15.86:0/3445278670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44f0102210 0x7f44f019e470 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f44ec00d950 tx=0x7f44ec00de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:33.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.952+0000 7f44e67fc640 1 -- 172.21.15.86:0/3445278670 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ec017070 con 0x7f44f0102210 2024-04-21T08:41:33.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.952+0000 7f44e67fc640 1 -- 172.21.15.86:0/3445278670 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44ec009c30 con 0x7f44f0102210 2024-04-21T08:41:33.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.952+0000 7f44f785c640 1 -- 172.21.15.86:0/3445278670 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44f0199480 con 0x7f44f0102210 2024-04-21T08:41:33.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.952+0000 7f44e67fc640 1 -- 172.21.15.86:0/3445278670 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ec00f620 con 0x7f44f0102210 2024-04-21T08:41:33.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.952+0000 7f44f785c640 1 -- 172.21.15.86:0/3445278670 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44f01a56d0 con 0x7f44f0102210 2024-04-21T08:41:33.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.954+0000 7f44e67fc640 1 -- 172.21.15.86:0/3445278670 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f44ec010460 con 0x7f44f0102210 2024-04-21T08:41:33.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.955+0000 7f44fa454740 1 Processor -- start 2024-04-21T08:41:33.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.955+0000 7f44e67fc640 1 --2- 172.21.15.86:0/3445278670 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f44dc0a3da0 0x7f44dc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.955+0000 7f44fa454740 1 -- start start 2024-04-21T08:41:33.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.955+0000 7f44f5df9640 1 --2- 172.21.15.86:0/3445278670 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f44dc0a3da0 0x7f44dc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.955+0000 7f44e67fc640 1 -- 172.21.15.86:0/3445278670 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f44ec0d0730 con 0x7f44f0102210 2024-04-21T08:41:33.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.955+0000 7f44fa454740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588d649a4d0 0x5588d649a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.955+0000 7f44fa454740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d649ae80 0x5588d64a3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.955+0000 7f44fa454740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588d64a3a00 0x5588d64a5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.955+0000 7f44fa454740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5588d63ff1f0 con 0x5588d649ae80 2024-04-21T08:41:33.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.955+0000 7f44fa454740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5588d639c4b0 con 0x5588d64a3a00 2024-04-21T08:41:33.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.955+0000 7f44fa454740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5588d63fb1b0 con 0x5588d649a4d0 2024-04-21T08:41:33.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.955+0000 7f44f4df7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d649ae80 0x5588d64a3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.955+0000 7f44f4df7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d649ae80 0x5588d64a3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48860/0 (socket says 172.21.15.86:48860) 2024-04-21T08:41:33.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.955+0000 7f44f4df7640 1 -- 172.21.15.86:0/3445357499 learned_addr learned my addr 172.21.15.86:0/3445357499 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:33.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.955+0000 7f44f55f8640 1 --2- 172.21.15.86:0/3445357499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588d649a4d0 0x5588d649a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.955+0000 7f44f5df9640 1 --2- 172.21.15.86:0/3445357499 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588d64a3a00 0x5588d64a5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.956+0000 7f44f4df7640 1 -- 172.21.15.86:0/3445357499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588d649a4d0 msgr2=0x5588d649a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:33.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.956+0000 7f44f4df7640 1 --2- 172.21.15.86:0/3445357499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588d649a4d0 0x5588d649a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.956+0000 7f44f5df9640 1 --2- 172.21.15.86:0/3445278670 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f44dc0a3da0 0x7f44dc0a6260 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f44e8008c60 tx=0x7f44e800d710 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:33.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.956+0000 7f44f4df7640 1 -- 172.21.15.86:0/3445357499 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588d64a3a00 msgr2=0x5588d64a5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:33.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.956+0000 7f44f4df7640 1 --2- 172.21.15.86:0/3445357499 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588d64a3a00 0x5588d64a5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.956+0000 7f44f4df7640 1 -- 172.21.15.86:0/3445357499 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588d638e1b0 con 0x5588d649ae80 2024-04-21T08:41:33.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.956+0000 7f44f55f8640 1 --2- 172.21.15.86:0/3445357499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588d649a4d0 0x5588d649a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:33.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.956+0000 7f44f5df9640 1 --2- 172.21.15.86:0/3445357499 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588d64a3a00 0x5588d64a5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:33.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.956+0000 7f44f4df7640 1 --2- 172.21.15.86:0/3445357499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d649ae80 0x5588d64a3430 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f44e0009900 tx=0x7f44e002c6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=462d858d14811057 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:33.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.956+0000 7f44db7fe640 1 -- 172.21.15.86:0/3445357499 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e0045020 con 0x5588d649ae80 2024-04-21T08:41:33.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.956+0000 7f44db7fe640 1 -- 172.21.15.86:0/3445357499 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44e002c7e0 con 0x5588d649ae80 2024-04-21T08:41:33.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.956+0000 7f44db7fe640 1 -- 172.21.15.86:0/3445357499 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e00046c0 con 0x5588d649ae80 2024-04-21T08:41:33.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.957+0000 7f44fa454740 1 -- 172.21.15.86:0/3445357499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d649ae80 msgr2=0x5588d64a3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.957+0000 7f44fa454740 1 --2- 172.21.15.86:0/3445357499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d649ae80 0x5588d64a3430 secure :-1 s=READY pgs=949 cs=0 l=1 rev1=1 crypto rx=0x7f44e0009900 tx=0x7f44e002c6c0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.957+0000 7f44fa454740 1 -- 172.21.15.86:0/3445357499 shutdown_connections 2024-04-21T08:41:33.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.957+0000 7f44fa454740 1 --2- 172.21.15.86:0/3445357499 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588d64a3a00 0x5588d64a5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.957+0000 7f44fa454740 1 --2- 172.21.15.86:0/3445357499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d649ae80 0x5588d64a3430 unknown :-1 s=CLOSED pgs=949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.957+0000 7f44fa454740 1 --2- 172.21.15.86:0/3445357499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588d649a4d0 0x5588d649a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.957+0000 7f44fa454740 1 -- 172.21.15.86:0/3445357499 >> 172.21.15.86:0/3445357499 conn(0x5588d6494ba0 msgr2=0x5588d6494f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:33.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.957+0000 7f44fa454740 1 -- 172.21.15.86:0/3445357499 shutdown_connections 2024-04-21T08:41:33.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.957+0000 7f44fa454740 1 -- 172.21.15.86:0/3445357499 wait complete. 2024-04-21T08:41:33.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.957+0000 7f44fa454740 1 -- start start 2024-04-21T08:41:33.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.957+0000 7f44fa454740 1 Processor -- start 2024-04-21T08:41:33.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.957+0000 7f44fa454740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588d649a4d0 0x5588d64a32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.957+0000 7f44fa454740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d649ae80 0x5588d64a0660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.957+0000 7f44fa454740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588d64a3a00 0x5588d64a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.957+0000 7f44fa454740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5588d64a6910 con 0x5588d649ae80 2024-04-21T08:41:33.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.957+0000 7f44fa454740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5588d63b8500 con 0x5588d649a4d0 2024-04-21T08:41:33.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.957+0000 7f44fa454740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5588d64a6a90 con 0x5588d64a3a00 2024-04-21T08:41:33.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.958+0000 7f44fa454740 10 client.0 _ll_register_callbacks cb 0x5588d6386b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:33.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.958+0000 7f44f4df7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d649ae80 0x5588d64a0660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.958+0000 7f44f4df7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d649ae80 0x5588d64a0660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48872/0 (socket says 172.21.15.86:48872) 2024-04-21T08:41:33.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.958+0000 7f44f4df7640 1 -- 172.21.15.86:0/3811545186 learned_addr learned my addr 172.21.15.86:0/3811545186 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:33.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.958+0000 7f44f5df9640 1 --2- 172.21.15.86:0/3811545186 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588d64a3a00 0x5588d64a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.958+0000 7f44f55f8640 1 --2- 172.21.15.86:0/3811545186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588d649a4d0 0x5588d64a32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.958+0000 7f44f4df7640 1 -- 172.21.15.86:0/3811545186 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588d64a3a00 msgr2=0x5588d64a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:33.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.958+0000 7f44f4df7640 1 --2- 172.21.15.86:0/3811545186 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588d64a3a00 0x5588d64a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.958+0000 7f44f4df7640 1 -- 172.21.15.86:0/3811545186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588d649a4d0 msgr2=0x5588d64a32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:33.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.958+0000 7f44f4df7640 1 --2- 172.21.15.86:0/3811545186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588d649a4d0 0x5588d64a32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.958+0000 7f44f4df7640 1 -- 172.21.15.86:0/3811545186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588d61113f0 con 0x5588d649ae80 2024-04-21T08:41:33.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.958+0000 7f44f55f8640 1 --2- 172.21.15.86:0/3811545186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588d649a4d0 0x5588d64a32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:33.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.958+0000 7f44f4df7640 1 --2- 172.21.15.86:0/3811545186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d649ae80 0x5588d64a0660 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f44e0036040 tx=0x7f44e002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed6bb382be0170e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:33.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.958+0000 7f44d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:33.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.958+0000 7f44d8ff9640 1 -- 172.21.15.86:0/3811545186 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e00035f0 con 0x5588d649ae80 2024-04-21T08:41:33.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.958+0000 7f44d8ff9640 1 -- 172.21.15.86:0/3811545186 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f44e0003790 con 0x5588d649ae80 2024-04-21T08:41:33.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.958+0000 7f44fa454740 1 -- 172.21.15.86:0/3811545186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44e002c7e0 con 0x5588d649ae80 2024-04-21T08:41:33.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.958+0000 7f44d8ff9640 1 -- 172.21.15.86:0/3811545186 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f44e0045020 con 0x5588d649ae80 2024-04-21T08:41:33.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.958+0000 7f44fa454740 10 client.17016 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:33.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.958+0000 7f44fa454740 1 -- 172.21.15.86:0/3811545186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5588d61db7d0 con 0x5588d649ae80 2024-04-21T08:41:33.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.959+0000 7f44d8ff9640 1 -- 172.21.15.86:0/3811545186 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44e002c7e0 con 0x5588d649ae80 2024-04-21T08:41:33.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.959+0000 7f44d8ff9640 1 -- 172.21.15.86:0/3811545186 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f44e0004db0 con 0x5588d649ae80 2024-04-21T08:41:33.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.959+0000 7f44fa454740 10 client.17016 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:33.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.959+0000 7f44fa454740 10 client.17016 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:41:33.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.959+0000 7f44fa454740 20 client.17016 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:33.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.959+0000 7f44fa454740 1 --2- 172.21.15.86:0/3811545186 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5588d64b3a00 0x5588d64b5df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.959+0000 7f44fa454740 4 client.17016 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:33.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.959+0000 7f44fa454740 1 -- 172.21.15.86:0/3811545186 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5588d639e930 con 0x5588d64b3a00 2024-04-21T08:41:33.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.959+0000 7f44f55f8640 1 --2- 172.21.15.86:0/3811545186 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5588d64b3a00 0x5588d64b5df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.959+0000 7f44d8ff9640 10 client.17016 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:41:33.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.959+0000 7f44f55f8640 1 --2- 172.21.15.86:0/3811545186 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5588d64b3a00 0x5588d64b5df0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c76390a495cf5a10 server_cookie=913ec8f09cfb878c in_seq=0 out_seq=0 2024-04-21T08:41:33.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.960+0000 7f44d8ff9640 1 -- 172.21.15.86:0/3811545186 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x7f44ec081ae0 con 0x5588d64b3a00 2024-04-21T08:41:33.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.960+0000 7f44d8ff9640 10 client.17016 handle_command_reply: tid=0 2024-04-21T08:41:33.967 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.960+0000 7f44fa454740 1 client.17016 shutdown 2024-04-21T08:41:33.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.960+0000 7f44fa454740 1 -- 172.21.15.86:0/3811545186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d649ae80 msgr2=0x5588d64a0660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.960+0000 7f44fa454740 1 --2- 172.21.15.86:0/3811545186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d649ae80 0x5588d64a0660 secure :-1 s=READY pgs=950 cs=0 l=1 rev1=1 crypto rx=0x7f44e0036040 tx=0x7f44e002bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:33.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.960+0000 7f44fa454740 1 -- 172.21.15.86:0/3811545186 shutdown_connections 2024-04-21T08:41:33.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.960+0000 7f44fa454740 1 --2- 172.21.15.86:0/3811545186 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588d64a3a00 0x5588d64a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.960+0000 7f44fa454740 1 --2- 172.21.15.86:0/3811545186 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5588d64b3a00 0x5588d64b5df0 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.960+0000 7f44fa454740 1 --2- 172.21.15.86:0/3811545186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d649ae80 0x5588d64a0660 unknown :-1 s=CLOSED pgs=950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.960+0000 7f44fa454740 1 --2- 172.21.15.86:0/3811545186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588d649a4d0 0x5588d64a32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.960+0000 7f44fa454740 1 -- 172.21.15.86:0/3811545186 >> 172.21.15.86:0/3811545186 conn(0x5588d6494ba0 msgr2=0x5588d64968f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:33.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.960+0000 7f44d8ff9640 0 client.17016 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:41:33.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.961+0000 7f44fa454740 1 -- 172.21.15.86:0/3811545186 shutdown_connections 2024-04-21T08:41:33.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.961+0000 7f44fa454740 1 -- 172.21.15.86:0/3811545186 wait complete. 2024-04-21T08:41:33.968 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.961+0000 7f44fa454740 20 client.17016 trim_cache size 0 max 16384 2024-04-21T08:41:33.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.974+0000 7f44fa454740 1 Processor -- start 2024-04-21T08:41:33.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.974+0000 7f44fa454740 1 -- start start 2024-04-21T08:41:33.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.974+0000 7f44fa454740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588d649a4d0 0x5588d649f510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.974+0000 7f44fa454740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588d649ae80 0x5588d64a2d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.974+0000 7f44fa454740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d64a3a00 0x5588d64a32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.974+0000 7f44fa454740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44f01af250 con 0x5588d64a3a00 2024-04-21T08:41:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.974+0000 7f44fa454740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f44f01b0d90 con 0x5588d649ae80 2024-04-21T08:41:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.974+0000 7f44fa454740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f44f01af920 con 0x5588d649a4d0 2024-04-21T08:41:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.974+0000 7f44f5df9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d64a3a00 0x5588d64a32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.974+0000 7f44f5df9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d64a3a00 0x5588d64a32d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48888/0 (socket says 172.21.15.86:48888) 2024-04-21T08:41:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.974+0000 7f44f5df9640 1 -- 172.21.15.86:0/4049732699 learned_addr learned my addr 172.21.15.86:0/4049732699 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.974+0000 7f44f4df7640 1 --2- 172.21.15.86:0/4049732699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588d649ae80 0x5588d64a2d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.974+0000 7f44f5df9640 1 -- 172.21.15.86:0/4049732699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588d649a4d0 msgr2=0x5588d649f510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.974+0000 7f44f55f8640 1 --2- 172.21.15.86:0/4049732699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588d649a4d0 0x5588d649f510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.974+0000 7f44f5df9640 1 --2- 172.21.15.86:0/4049732699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588d649a4d0 0x5588d649f510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.974+0000 7f44f5df9640 1 -- 172.21.15.86:0/4049732699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588d649ae80 msgr2=0x5588d64a2d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.974+0000 7f44f5df9640 1 --2- 172.21.15.86:0/4049732699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588d649ae80 0x5588d64a2d90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.974+0000 7f44f5df9640 1 -- 172.21.15.86:0/4049732699 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588d649ac20 con 0x5588d64a3a00 2024-04-21T08:41:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.974+0000 7f44f4df7640 1 --2- 172.21.15.86:0/4049732699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588d649ae80 0x5588d64a2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:33.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.974+0000 7f44f55f8640 1 --2- 172.21.15.86:0/4049732699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588d649a4d0 0x5588d649f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:33.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.974+0000 7f44f5df9640 1 --2- 172.21.15.86:0/4049732699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d64a3a00 0x5588d64a32d0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f44e8011440 tx=0x7f44e8018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af775048c6955147 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:33.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.975+0000 7f44daffd640 1 -- 172.21.15.86:0/4049732699 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e801e850 con 0x5588d64a3a00 2024-04-21T08:41:33.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.975+0000 7f44daffd640 1 -- 172.21.15.86:0/4049732699 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44e8022070 con 0x5588d64a3a00 2024-04-21T08:41:33.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.975+0000 7f44daffd640 1 -- 172.21.15.86:0/4049732699 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e8027e40 con 0x5588d64a3a00 2024-04-21T08:41:33.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.975+0000 7f44fa454740 1 -- 172.21.15.86:0/4049732699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d64a3a00 msgr2=0x5588d64a32d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.975+0000 7f44fa454740 1 --2- 172.21.15.86:0/4049732699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d64a3a00 0x5588d64a32d0 secure :-1 s=READY pgs=951 cs=0 l=1 rev1=1 crypto rx=0x7f44e8011440 tx=0x7f44e8018000 comp rx=0 tx=0).stop 2024-04-21T08:41:33.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.975+0000 7f44fa454740 1 -- 172.21.15.86:0/4049732699 shutdown_connections 2024-04-21T08:41:33.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.975+0000 7f44fa454740 1 --2- 172.21.15.86:0/4049732699 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d64a3a00 0x5588d64a32d0 unknown :-1 s=CLOSED pgs=951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.975+0000 7f44fa454740 1 --2- 172.21.15.86:0/4049732699 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588d649ae80 0x5588d64a2d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.975+0000 7f44fa454740 1 --2- 172.21.15.86:0/4049732699 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588d649a4d0 0x5588d649f510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.975+0000 7f44fa454740 1 -- 172.21.15.86:0/4049732699 >> 172.21.15.86:0/4049732699 conn(0x5588d6494ba0 msgr2=0x5588d649ee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:33.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.975+0000 7f44fa454740 1 -- 172.21.15.86:0/4049732699 shutdown_connections 2024-04-21T08:41:33.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.975+0000 7f44fa454740 1 -- 172.21.15.86:0/4049732699 wait complete. 2024-04-21T08:41:33.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.975+0000 7f44fa454740 1 -- start start 2024-04-21T08:41:33.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.976+0000 7f44fa454740 1 Processor -- start 2024-04-21T08:41:33.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.976+0000 7f44fa454740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588d649a4d0 0x5588d64b6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.976+0000 7f44fa454740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d649ae80 0x5588d64b6d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.976+0000 7f44fa454740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588d64a3a00 0x5588d64b7280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.976+0000 7f44fa454740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44f01b06c0 con 0x5588d649ae80 2024-04-21T08:41:33.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.976+0000 7f44fa454740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f44f01afff0 con 0x5588d64a3a00 2024-04-21T08:41:33.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.976+0000 7f44fa454740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f44f01ad210 con 0x5588d649a4d0 2024-04-21T08:41:33.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.976+0000 7f44fa454740 10 client.0 _ll_register_callbacks cb 0x7f44e002cd70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:33.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.976+0000 7f44f4df7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d649ae80 0x5588d64b6d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.976+0000 7f44f4df7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d649ae80 0x5588d64b6d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48898/0 (socket says 172.21.15.86:48898) 2024-04-21T08:41:33.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.976+0000 7f44f4df7640 1 -- 172.21.15.86:0/3885303071 learned_addr learned my addr 172.21.15.86:0/3885303071 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:33.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.976+0000 7f44f5df9640 1 --2- 172.21.15.86:0/3885303071 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588d64a3a00 0x5588d64b7280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.976+0000 7f44f55f8640 1 --2- 172.21.15.86:0/3885303071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588d649a4d0 0x5588d64b6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.976+0000 7f44f4df7640 1 -- 172.21.15.86:0/3885303071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588d649a4d0 msgr2=0x5588d64b6800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:33.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.976+0000 7f44f4df7640 1 --2- 172.21.15.86:0/3885303071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588d649a4d0 0x5588d64b6800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.976+0000 7f44f4df7640 1 -- 172.21.15.86:0/3885303071 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588d64a3a00 msgr2=0x5588d64b7280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:33.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.976+0000 7f44f4df7640 1 --2- 172.21.15.86:0/3885303071 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588d64a3a00 0x5588d64b7280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.976+0000 7f44f4df7640 1 -- 172.21.15.86:0/3885303071 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588d651cff0 con 0x5588d649ae80 2024-04-21T08:41:33.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.976+0000 7f44f4df7640 1 --2- 172.21.15.86:0/3885303071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d649ae80 0x5588d64b6d40 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f44e0006010 tx=0x7f44e0005470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3c32585f29c72b27 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:33.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.976+0000 7f44d8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:33.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.977+0000 7f44d8ff9640 1 -- 172.21.15.86:0/3885303071 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44e0000e60 con 0x5588d649ae80 2024-04-21T08:41:33.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.977+0000 7f44d8ff9640 1 -- 172.21.15.86:0/3885303071 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f44e0056b00 con 0x5588d649ae80 2024-04-21T08:41:33.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.977+0000 7f44fa454740 1 -- 172.21.15.86:0/3885303071 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44e8022070 con 0x5588d649ae80 2024-04-21T08:41:33.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.977+0000 7f44d8ff9640 1 -- 172.21.15.86:0/3885303071 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f44e0056d00 con 0x5588d649ae80 2024-04-21T08:41:33.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.977+0000 7f44fa454740 10 client.17022 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:33.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.977+0000 7f44fa454740 1 -- 172.21.15.86:0/3885303071 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5588d651e200 con 0x5588d649ae80 2024-04-21T08:41:33.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.977+0000 7f44d8ff9640 1 -- 172.21.15.86:0/3885303071 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44e8022070 con 0x5588d649ae80 2024-04-21T08:41:33.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.977+0000 7f44d8ff9640 1 -- 172.21.15.86:0/3885303071 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f44e0041480 con 0x5588d649ae80 2024-04-21T08:41:33.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.977+0000 7f44fa454740 10 client.17022 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:33.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.977+0000 7f44fa454740 10 client.17022 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:41:33.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.977+0000 7f44fa454740 20 client.17022 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:33.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.977+0000 7f44fa454740 1 --2- 172.21.15.86:0/3885303071 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5588d651e760 0x5588d6520b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:33.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.977+0000 7f44fa454740 4 client.17022 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:41:33.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.977+0000 7f44fa454740 1 -- 172.21.15.86:0/3885303071 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5588d640cc10 con 0x5588d651e760 2024-04-21T08:41:33.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.977+0000 7f44f55f8640 1 --2- 172.21.15.86:0/3885303071 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5588d651e760 0x5588d6520b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:33.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.978+0000 7f44f55f8640 1 --2- 172.21.15.86:0/3885303071 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5588d651e760 0x5588d6520b80 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b86acbc0f5257325 server_cookie=2167c970c4157195 in_seq=0 out_seq=0 2024-04-21T08:41:33.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.978+0000 7f44d8ff9640 10 client.17022 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:41:33.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.978+0000 7f44d8ff9640 1 -- 172.21.15.86:0/3885303071 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5588d640cc10 con 0x5588d651e760 2024-04-21T08:41:33.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.978+0000 7f44d8ff9640 10 client.17022 handle_command_reply: tid=0 2024-04-21T08:41:33.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.978+0000 7f44fa454740 1 client.17022 shutdown 2024-04-21T08:41:33.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.978+0000 7f44fa454740 1 -- 172.21.15.86:0/3885303071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d649ae80 msgr2=0x5588d64b6d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.978+0000 7f44fa454740 1 --2- 172.21.15.86:0/3885303071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d649ae80 0x5588d64b6d40 secure :-1 s=READY pgs=952 cs=0 l=1 rev1=1 crypto rx=0x7f44e0006010 tx=0x7f44e0005470 comp rx=0 tx=0).stop 2024-04-21T08:41:33.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.978+0000 7f44fa454740 1 -- 172.21.15.86:0/3885303071 shutdown_connections 2024-04-21T08:41:33.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.978+0000 7f44fa454740 1 --2- 172.21.15.86:0/3885303071 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588d64a3a00 0x5588d64b7280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.978+0000 7f44fa454740 1 --2- 172.21.15.86:0/3885303071 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5588d651e760 0x5588d6520b80 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.978+0000 7f44fa454740 1 --2- 172.21.15.86:0/3885303071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588d649ae80 0x5588d64b6d40 unknown :-1 s=CLOSED pgs=952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.978+0000 7f44fa454740 1 --2- 172.21.15.86:0/3885303071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588d649a4d0 0x5588d64b6800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.978+0000 7f44d8ff9640 0 client.17022 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:41:33.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.978+0000 7f44fa454740 1 -- 172.21.15.86:0/3885303071 >> 172.21.15.86:0/3885303071 conn(0x5588d6494ba0 msgr2=0x5588d6493c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:33.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.978+0000 7f44fa454740 1 -- 172.21.15.86:0/3885303071 shutdown_connections 2024-04-21T08:41:33.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.978+0000 7f44fa454740 1 -- 172.21.15.86:0/3885303071 wait complete. 2024-04-21T08:41:33.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.978+0000 7f44fa454740 20 client.17022 trim_cache size 0 max 16384 2024-04-21T08:41:33.984 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:41:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.979+0000 7f44f785c640 1 -- 172.21.15.86:0/3445278670 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f44dc0a3da0 msgr2=0x7f44dc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.979+0000 7f44f785c640 1 --2- 172.21.15.86:0/3445278670 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f44dc0a3da0 0x7f44dc0a6260 secure :-1 s=READY pgs=142 cs=0 l=1 rev1=1 crypto rx=0x7f44e8008c60 tx=0x7f44e800d710 comp rx=0 tx=0).stop 2024-04-21T08:41:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.979+0000 7f44f785c640 1 -- 172.21.15.86:0/3445278670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44f0102210 msgr2=0x7f44f019e470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.979+0000 7f44f785c640 1 --2- 172.21.15.86:0/3445278670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44f0102210 0x7f44f019e470 secure :-1 s=READY pgs=948 cs=0 l=1 rev1=1 crypto rx=0x7f44ec00d950 tx=0x7f44ec00de10 comp rx=0 tx=0).stop 2024-04-21T08:41:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.979+0000 7f44f785c640 1 -- 172.21.15.86:0/3445278670 shutdown_connections 2024-04-21T08:41:33.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.979+0000 7f44f785c640 1 --2- 172.21.15.86:0/3445278670 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f44dc0a3da0 0x7f44dc0a6260 unknown :-1 s=CLOSED pgs=142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.979+0000 7f44f785c640 1 --2- 172.21.15.86:0/3445278670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44f0198670 0x7f44f0198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.979+0000 7f44f785c640 1 --2- 172.21.15.86:0/3445278670 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44f01072e0 0x7f44f019e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.979+0000 7f44f785c640 1 --2- 172.21.15.86:0/3445278670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44f0102210 0x7f44f019e470 unknown :-1 s=CLOSED pgs=948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:33.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.979+0000 7f44f785c640 1 -- 172.21.15.86:0/3445278670 >> 172.21.15.86:0/3445278670 conn(0x7f44f00fdab0 msgr2=0x7f44f00fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:33.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.979+0000 7f44f785c640 1 -- 172.21.15.86:0/3445278670 shutdown_connections 2024-04-21T08:41:33.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:33.979+0000 7f44f785c640 1 -- 172.21.15.86:0/3445278670 wait complete. 2024-04-21T08:41:33.988 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:41:34.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.089+0000 7f397b1fd640 1 -- 172.21.15.86:0/827278379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39741021a0 msgr2=0x7f3974102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:34.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.089+0000 7f397b1fd640 1 --2- 172.21.15.86:0/827278379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39741021a0 0x7f3974102580 secure :-1 s=READY pgs=953 cs=0 l=1 rev1=1 crypto rx=0x7f3968009900 tx=0x7f396802bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:34.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.089+0000 7f397b1fd640 1 -- 172.21.15.86:0/827278379 shutdown_connections 2024-04-21T08:41:34.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.089+0000 7f397b1fd640 1 --2- 172.21.15.86:0/827278379 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3974107270 0x7f397410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:34.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.089+0000 7f397b1fd640 1 --2- 172.21.15.86:0/827278379 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3974102b50 0x7f3974106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:34.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.089+0000 7f397b1fd640 1 --2- 172.21.15.86:0/827278379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39741021a0 0x7f3974102580 unknown :-1 s=CLOSED pgs=953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:34.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.089+0000 7f397b1fd640 1 -- 172.21.15.86:0/827278379 >> 172.21.15.86:0/827278379 conn(0x7f39740fd9c0 msgr2=0x7f39740ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:34.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.089+0000 7f397b1fd640 1 -- 172.21.15.86:0/827278379 shutdown_connections 2024-04-21T08:41:34.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.089+0000 7f397b1fd640 1 -- 172.21.15.86:0/827278379 wait complete. 2024-04-21T08:41:34.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.090+0000 7f397b1fd640 1 Processor -- start 2024-04-21T08:41:34.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.090+0000 7f397b1fd640 1 -- start start 2024-04-21T08:41:34.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.090+0000 7f397b1fd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39741021a0 0x7f397419e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:34.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.090+0000 7f397b1fd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3974107270 0x7f397419e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:34.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.090+0000 7f397b1fd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39741985d0 0x7f3974198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:34.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.090+0000 7f397b1fd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f39741121d0 con 0x7f39741021a0 2024-04-21T08:41:34.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.090+0000 7f397b1fd640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3974112050 con 0x7f39741985d0 2024-04-21T08:41:34.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.090+0000 7f397b1fd640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3974112350 con 0x7f3974107270 2024-04-21T08:41:34.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.090+0000 7f3978f99640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39741021a0 0x7f397419e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:34.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.090+0000 7f3978f99640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39741021a0 0x7f397419e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48930/0 (socket says 172.21.15.86:48930) 2024-04-21T08:41:34.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.090+0000 7f3978f99640 1 -- 172.21.15.86:0/759510188 learned_addr learned my addr 172.21.15.86:0/759510188 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:34.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.090+0000 7f397979a640 1 --2- 172.21.15.86:0/759510188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39741985d0 0x7f3974198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:34.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.090+0000 7f3978f99640 1 -- 172.21.15.86:0/759510188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3974107270 msgr2=0x7f397419e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:34.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.090+0000 7f3978f99640 1 --2- 172.21.15.86:0/759510188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3974107270 0x7f397419e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:34.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.090+0000 7f3978f99640 1 -- 172.21.15.86:0/759510188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39741985d0 msgr2=0x7f3974198a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:34.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.090+0000 7f3978f99640 1 --2- 172.21.15.86:0/759510188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39741985d0 0x7f3974198a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:34.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.090+0000 7f3978f99640 1 -- 172.21.15.86:0/759510188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39741990f0 con 0x7f39741021a0 2024-04-21T08:41:34.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.090+0000 7f397979a640 1 --2- 172.21.15.86:0/759510188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39741985d0 0x7f3974198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:34.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.091+0000 7f3978f99640 1 --2- 172.21.15.86:0/759510188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39741021a0 0x7f397419e420 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f3968005940 tx=0x7f396802c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:34.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.091+0000 7f3971ffb640 1 -- 172.21.15.86:0/759510188 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3968043020 con 0x7f39741021a0 2024-04-21T08:41:34.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.091+0000 7f3971ffb640 1 -- 172.21.15.86:0/759510188 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f396802cbe0 con 0x7f39741021a0 2024-04-21T08:41:34.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.091+0000 7f397b1fd640 1 -- 172.21.15.86:0/759510188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3974199380 con 0x7f39741021a0 2024-04-21T08:41:34.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.091+0000 7f3971ffb640 1 -- 172.21.15.86:0/759510188 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3968003bf0 con 0x7f39741021a0 2024-04-21T08:41:34.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.091+0000 7f397b1fd640 1 -- 172.21.15.86:0/759510188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39741a55d0 con 0x7f39741021a0 2024-04-21T08:41:34.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.093+0000 7f3971ffb640 1 -- 172.21.15.86:0/759510188 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3968046020 con 0x7f39741021a0 2024-04-21T08:41:34.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.093+0000 7f397dddf740 1 Processor -- start 2024-04-21T08:41:34.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.094+0000 7f397dddf740 1 -- start start 2024-04-21T08:41:34.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.094+0000 7f3971ffb640 1 --2- 172.21.15.86:0/759510188 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f39640a3d20 0x7f39640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:34.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.094+0000 7f397dddf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d5e043b0 0x5576d5e04790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:34.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.094+0000 7f397dddf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d5e04d60 0x5576d5e0d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:34.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.094+0000 7f397dddf740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d5e0de60 0x5576d5e0e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:34.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.094+0000 7f397dddf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5576d5d691f0 con 0x5576d5e043b0 2024-04-21T08:41:34.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.094+0000 7f397dddf740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5576d5d064b0 con 0x5576d5e0de60 2024-04-21T08:41:34.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.094+0000 7f397dddf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5576d5d651b0 con 0x5576d5e04d60 2024-04-21T08:41:34.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.094+0000 7f397979a640 1 --2- 172.21.15.86:0/759510188 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f39640a3d20 0x7f39640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:34.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.094+0000 7f3971ffb640 1 -- 172.21.15.86:0/759510188 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f39680f2d90 con 0x7f39741021a0 2024-04-21T08:41:34.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.094+0000 7f3978f99640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d5e043b0 0x5576d5e04790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:34.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.094+0000 7f3978f99640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d5e043b0 0x5576d5e04790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48936/0 (socket says 172.21.15.86:48936) 2024-04-21T08:41:34.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.094+0000 7f3978f99640 1 -- 172.21.15.86:0/2297284225 learned_addr learned my addr 172.21.15.86:0/2297284225 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:34.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.094+0000 7f3973fff640 1 --2- 172.21.15.86:0/2297284225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d5e04d60 0x5576d5e0d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:34.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.094+0000 7f3978f99640 1 -- 172.21.15.86:0/2297284225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d5e04d60 msgr2=0x5576d5e0d890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:34.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.094+0000 7f3978f99640 1 --2- 172.21.15.86:0/2297284225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d5e04d60 0x5576d5e0d890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:34.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.094+0000 7f3978f99640 1 -- 172.21.15.86:0/2297284225 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d5e0de60 msgr2=0x5576d5e0e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:34.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.094+0000 7f397979a640 1 --2- 172.21.15.86:0/2297284225 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d5e0de60 0x5576d5e0e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:34.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.094+0000 7f3978f99640 1 --2- 172.21.15.86:0/2297284225 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d5e0de60 0x5576d5e0e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:34.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.094+0000 7f3978f99640 1 -- 172.21.15.86:0/2297284225 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576d5cf81b0 con 0x5576d5e043b0 2024-04-21T08:41:34.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.094+0000 7f397979a640 1 --2- 172.21.15.86:0/2297284225 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d5e0de60 0x5576d5e0e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:34.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.094+0000 7f397979a640 1 --2- 172.21.15.86:0/759510188 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f39640a3d20 0x7f39640a61e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f396c0092f0 tx=0x7f396c010740 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:34.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.094+0000 7f3978f99640 1 --2- 172.21.15.86:0/2297284225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d5e043b0 0x5576d5e04790 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f3968005ec0 tx=0x7f39680ba770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=340181704dacb4aa server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:34.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.094+0000 7f395effd640 1 -- 172.21.15.86:0/2297284225 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3968035ad0 con 0x5576d5e043b0 2024-04-21T08:41:34.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.095+0000 7f395effd640 1 -- 172.21.15.86:0/2297284225 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f396800b0a0 con 0x5576d5e043b0 2024-04-21T08:41:34.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.095+0000 7f395effd640 1 -- 172.21.15.86:0/2297284225 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396800aeb0 con 0x5576d5e043b0 2024-04-21T08:41:34.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.095+0000 7f397dddf740 1 -- 172.21.15.86:0/2297284225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d5e043b0 msgr2=0x5576d5e04790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:34.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.095+0000 7f397dddf740 1 --2- 172.21.15.86:0/2297284225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d5e043b0 0x5576d5e04790 secure :-1 s=READY pgs=955 cs=0 l=1 rev1=1 crypto rx=0x7f3968005ec0 tx=0x7f39680ba770 comp rx=0 tx=0).stop 2024-04-21T08:41:34.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.095+0000 7f397dddf740 1 -- 172.21.15.86:0/2297284225 shutdown_connections 2024-04-21T08:41:34.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.095+0000 7f397dddf740 1 --2- 172.21.15.86:0/2297284225 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d5e0de60 0x5576d5e0e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:34.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.095+0000 7f397dddf740 1 --2- 172.21.15.86:0/2297284225 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d5e04d60 0x5576d5e0d890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:34.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.095+0000 7f397dddf740 1 --2- 172.21.15.86:0/2297284225 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d5e043b0 0x5576d5e04790 unknown :-1 s=CLOSED pgs=955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:34.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.095+0000 7f397dddf740 1 -- 172.21.15.86:0/2297284225 >> 172.21.15.86:0/2297284225 conn(0x5576d5dfeba0 msgr2=0x5576d5dfef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:34.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.095+0000 7f397dddf740 1 -- 172.21.15.86:0/2297284225 shutdown_connections 2024-04-21T08:41:34.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.095+0000 7f397dddf740 1 -- 172.21.15.86:0/2297284225 wait complete. 2024-04-21T08:41:34.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.095+0000 7f397dddf740 1 -- start start 2024-04-21T08:41:34.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.096+0000 7f397dddf740 1 Processor -- start 2024-04-21T08:41:34.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.096+0000 7f397dddf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d5e043b0 0x5576d5e142f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:34.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.096+0000 7f397dddf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d5e04d60 0x5576d5e115e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:34.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.096+0000 7f397dddf740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d5e0de60 0x5576d5e11b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:34.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.096+0000 7f397dddf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5576d5e14b60 con 0x5576d5e04d60 2024-04-21T08:41:34.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.096+0000 7f397dddf740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5576d5d22500 con 0x5576d5e0de60 2024-04-21T08:41:34.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.096+0000 7f397dddf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5576d5e14ce0 con 0x5576d5e043b0 2024-04-21T08:41:34.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.096+0000 7f397dddf740 10 client.0 _ll_register_callbacks cb 0x5576d5cf0b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:34.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.096+0000 7f3973fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d5e04d60 0x5576d5e115e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:34.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.096+0000 7f3973fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d5e04d60 0x5576d5e115e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48952/0 (socket says 172.21.15.86:48952) 2024-04-21T08:41:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.096+0000 7f3973fff640 1 -- 172.21.15.86:0/1059743586 learned_addr learned my addr 172.21.15.86:0/1059743586 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.096+0000 7f397979a640 1 --2- 172.21.15.86:0/1059743586 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d5e0de60 0x5576d5e11b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.096+0000 7f3978f99640 1 --2- 172.21.15.86:0/1059743586 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d5e043b0 0x5576d5e142f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.096+0000 7f3973fff640 1 -- 172.21.15.86:0/1059743586 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d5e043b0 msgr2=0x5576d5e142f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.096+0000 7f3973fff640 1 --2- 172.21.15.86:0/1059743586 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d5e043b0 0x5576d5e142f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.096+0000 7f3973fff640 1 -- 172.21.15.86:0/1059743586 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d5e0de60 msgr2=0x5576d5e11b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.096+0000 7f3973fff640 1 --2- 172.21.15.86:0/1059743586 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d5e0de60 0x5576d5e11b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.096+0000 7f3973fff640 1 -- 172.21.15.86:0/1059743586 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576d5a7b3f0 con 0x5576d5e04d60 2024-04-21T08:41:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.097+0000 7f3973fff640 1 --2- 172.21.15.86:0/1059743586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d5e04d60 0x5576d5e115e0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f396000bd00 tx=0x7f396000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=212113b600a679e4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.097+0000 7f3953fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.097+0000 7f3953fff640 1 -- 172.21.15.86:0/1059743586 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3960015070 con 0x5576d5e04d60 2024-04-21T08:41:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.097+0000 7f3953fff640 1 -- 172.21.15.86:0/1059743586 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f3960004230 con 0x5576d5e04d60 2024-04-21T08:41:34.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.097+0000 7f397dddf740 1 -- 172.21.15.86:0/1059743586 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5576d5d08930 con 0x5576d5e04d60 2024-04-21T08:41:34.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.097+0000 7f3953fff640 1 -- 172.21.15.86:0/1059743586 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3960022050 con 0x5576d5e04d60 2024-04-21T08:41:34.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.097+0000 7f397dddf740 10 client.17034 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:34.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.097+0000 7f397dddf740 1 -- 172.21.15.86:0/1059743586 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5576d5b457d0 con 0x5576d5e04d60 2024-04-21T08:41:34.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.097+0000 7f3953fff640 1 -- 172.21.15.86:0/1059743586 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5576d5d08930 con 0x5576d5e04d60 2024-04-21T08:41:34.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.097+0000 7f3953fff640 1 -- 172.21.15.86:0/1059743586 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3960011510 con 0x5576d5e04d60 2024-04-21T08:41:34.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.097+0000 7f397dddf740 10 client.17034 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:34.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.097+0000 7f397dddf740 10 client.17034 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:34.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.097+0000 7f397dddf740 20 client.17034 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:34.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.097+0000 7f397dddf740 1 --2- 172.21.15.86:0/1059743586 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5576d5e21df0 0x5576d5e24210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:34.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.097+0000 7f397dddf740 4 client.17034 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:34.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.097+0000 7f397dddf740 1 -- 172.21.15.86:0/1059743586 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576d5d76c10 con 0x5576d5e21df0 2024-04-21T08:41:34.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.097+0000 7f3978f99640 1 --2- 172.21.15.86:0/1059743586 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5576d5e21df0 0x5576d5e24210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:34.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.098+0000 7f3978f99640 1 --2- 172.21.15.86:0/1059743586 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5576d5e21df0 0x5576d5e24210 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2773e4f1b5a86e13 server_cookie=67193456b3131680 in_seq=0 out_seq=0 2024-04-21T08:41:34.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.098+0000 7f3953fff640 10 client.17034 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:34.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.099+0000 7f3953fff640 1 -- 172.21.15.86:0/1059743586 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5576d5d76c10 con 0x5576d5e21df0 2024-04-21T08:41:34.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.099+0000 7f3953fff640 10 client.17034 handle_command_reply: tid=0 2024-04-21T08:41:34.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.099+0000 7f397dddf740 1 client.17034 shutdown 2024-04-21T08:41:34.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.099+0000 7f397dddf740 1 -- 172.21.15.86:0/1059743586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d5e04d60 msgr2=0x5576d5e115e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:34.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.099+0000 7f397dddf740 1 --2- 172.21.15.86:0/1059743586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d5e04d60 0x5576d5e115e0 secure :-1 s=READY pgs=956 cs=0 l=1 rev1=1 crypto rx=0x7f396000bd00 tx=0x7f396000f730 comp rx=0 tx=0).stop 2024-04-21T08:41:34.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.099+0000 7f397dddf740 1 -- 172.21.15.86:0/1059743586 shutdown_connections 2024-04-21T08:41:34.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.099+0000 7f397dddf740 1 --2- 172.21.15.86:0/1059743586 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5576d5e21df0 0x5576d5e24210 crc :-1 s=READY pgs=115 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:34.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.099+0000 7f397dddf740 1 --2- 172.21.15.86:0/1059743586 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d5e0de60 0x5576d5e11b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:34.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.099+0000 7f397dddf740 1 --2- 172.21.15.86:0/1059743586 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d5e04d60 0x5576d5e115e0 unknown :-1 s=CLOSED pgs=956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:34.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.099+0000 7f3953fff640 0 client.17034 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:34.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.099+0000 7f397dddf740 1 --2- 172.21.15.86:0/1059743586 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d5e043b0 0x5576d5e142f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:34.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.099+0000 7f397dddf740 1 -- 172.21.15.86:0/1059743586 >> 172.21.15.86:0/1059743586 conn(0x5576d5dfeba0 msgr2=0x5576d5e008f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:34.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.099+0000 7f397dddf740 1 -- 172.21.15.86:0/1059743586 shutdown_connections 2024-04-21T08:41:34.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.099+0000 7f397dddf740 1 -- 172.21.15.86:0/1059743586 wait complete. 2024-04-21T08:41:34.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.099+0000 7f397dddf740 20 client.17034 trim_cache size 0 max 16384 2024-04-21T08:41:34.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.112+0000 7f397dddf740 1 Processor -- start 2024-04-21T08:41:34.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.112+0000 7f397dddf740 1 -- start start 2024-04-21T08:41:34.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.112+0000 7f397dddf740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d5e043b0 0x5576d5e11580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:34.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.112+0000 7f397dddf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d5e04d60 0x5576d5e29ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:34.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.112+0000 7f397dddf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d5e0de60 0x5576d5e2a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:34.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.112+0000 7f397dddf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f39741ad0e0 con 0x5576d5e0de60 2024-04-21T08:41:34.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.112+0000 7f397dddf740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f39741aec20 con 0x5576d5e043b0 2024-04-21T08:41:34.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.112+0000 7f397dddf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f39741ad7b0 con 0x5576d5e04d60 2024-04-21T08:41:34.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.112+0000 7f397979a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d5e0de60 0x5576d5e2a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:34.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.112+0000 7f397979a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d5e0de60 0x5576d5e2a000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48960/0 (socket says 172.21.15.86:48960) 2024-04-21T08:41:34.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.112+0000 7f397979a640 1 -- 172.21.15.86:0/2646643162 learned_addr learned my addr 172.21.15.86:0/2646643162 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:34.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.113+0000 7f3978f99640 1 --2- 172.21.15.86:0/2646643162 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d5e043b0 0x5576d5e11580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:34.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.113+0000 7f3973fff640 1 --2- 172.21.15.86:0/2646643162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d5e04d60 0x5576d5e29ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:34.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.113+0000 7f397979a640 1 -- 172.21.15.86:0/2646643162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d5e04d60 msgr2=0x5576d5e29ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:34.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.113+0000 7f397979a640 1 --2- 172.21.15.86:0/2646643162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d5e04d60 0x5576d5e29ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:34.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.113+0000 7f397979a640 1 -- 172.21.15.86:0/2646643162 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d5e043b0 msgr2=0x5576d5e11580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:34.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.113+0000 7f397979a640 1 --2- 172.21.15.86:0/2646643162 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d5e043b0 0x5576d5e11580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:34.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.113+0000 7f397979a640 1 -- 172.21.15.86:0/2646643162 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576d5e20940 con 0x5576d5e0de60 2024-04-21T08:41:34.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.113+0000 7f3978f99640 1 --2- 172.21.15.86:0/2646643162 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d5e043b0 0x5576d5e11580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:34.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.113+0000 7f397979a640 1 --2- 172.21.15.86:0/2646643162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d5e0de60 0x5576d5e2a000 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f396c0128a0 tx=0x7f396c015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=89753c1839a5ff7a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:34.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.113+0000 7f395e7fc640 1 -- 172.21.15.86:0/2646643162 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c011b60 con 0x5576d5e0de60 2024-04-21T08:41:34.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.113+0000 7f395e7fc640 1 -- 172.21.15.86:0/2646643162 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f396c012e20 con 0x5576d5e0de60 2024-04-21T08:41:34.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.113+0000 7f395e7fc640 1 -- 172.21.15.86:0/2646643162 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c0205f0 con 0x5576d5e0de60 2024-04-21T08:41:34.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.114+0000 7f397dddf740 1 -- 172.21.15.86:0/2646643162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d5e0de60 msgr2=0x5576d5e2a000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:34.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.114+0000 7f397dddf740 1 --2- 172.21.15.86:0/2646643162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d5e0de60 0x5576d5e2a000 secure :-1 s=READY pgs=957 cs=0 l=1 rev1=1 crypto rx=0x7f396c0128a0 tx=0x7f396c015000 comp rx=0 tx=0).stop 2024-04-21T08:41:34.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.114+0000 7f397dddf740 1 -- 172.21.15.86:0/2646643162 shutdown_connections 2024-04-21T08:41:34.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.114+0000 7f397dddf740 1 --2- 172.21.15.86:0/2646643162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d5e0de60 0x5576d5e2a000 unknown :-1 s=CLOSED pgs=957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:34.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.114+0000 7f397dddf740 1 --2- 172.21.15.86:0/2646643162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d5e04d60 0x5576d5e29ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:34.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.114+0000 7f397dddf740 1 --2- 172.21.15.86:0/2646643162 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d5e043b0 0x5576d5e11580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:34.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.114+0000 7f397dddf740 1 -- 172.21.15.86:0/2646643162 >> 172.21.15.86:0/2646643162 conn(0x5576d5dfeba0 msgr2=0x5576d5e10e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:34.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.114+0000 7f397dddf740 1 -- 172.21.15.86:0/2646643162 shutdown_connections 2024-04-21T08:41:34.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.114+0000 7f397dddf740 1 -- 172.21.15.86:0/2646643162 wait complete. 2024-04-21T08:41:34.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.114+0000 7f397dddf740 1 -- start start 2024-04-21T08:41:34.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.114+0000 7f397dddf740 1 Processor -- start 2024-04-21T08:41:34.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.114+0000 7f397dddf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d5e043b0 0x5576d5e19f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:34.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.114+0000 7f397dddf740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d5e04d60 0x5576d5e1a470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:34.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.114+0000 7f397dddf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d5e0de60 0x5576d5e0e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:34.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.114+0000 7f397dddf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f39741ae550 con 0x5576d5e0de60 2024-04-21T08:41:34.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.114+0000 7f397dddf740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f39741ade80 con 0x5576d5e04d60 2024-04-21T08:41:34.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.114+0000 7f397dddf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f39741ab0a0 con 0x5576d5e043b0 2024-04-21T08:41:34.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.114+0000 7f397dddf740 10 client.0 _ll_register_callbacks cb 0x7f396000b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:34.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.114+0000 7f397979a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d5e0de60 0x5576d5e0e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:34.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.115+0000 7f397979a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d5e0de60 0x5576d5e0e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48972/0 (socket says 172.21.15.86:48972) 2024-04-21T08:41:34.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.115+0000 7f397979a640 1 -- 172.21.15.86:0/4013763487 learned_addr learned my addr 172.21.15.86:0/4013763487 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:34.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.115+0000 7f3973fff640 1 --2- 172.21.15.86:0/4013763487 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d5e04d60 0x5576d5e1a470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:34.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.115+0000 7f3978f99640 1 --2- 172.21.15.86:0/4013763487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d5e043b0 0x5576d5e19f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:34.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.115+0000 7f397979a640 1 -- 172.21.15.86:0/4013763487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d5e043b0 msgr2=0x5576d5e19f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:34.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.115+0000 7f397979a640 1 --2- 172.21.15.86:0/4013763487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d5e043b0 0x5576d5e19f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:34.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.115+0000 7f397979a640 1 -- 172.21.15.86:0/4013763487 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d5e04d60 msgr2=0x5576d5e1a470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:34.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.115+0000 7f397979a640 1 --2- 172.21.15.86:0/4013763487 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d5e04d60 0x5576d5e1a470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:34.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.115+0000 7f397979a640 1 -- 172.21.15.86:0/4013763487 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576d5e0ec20 con 0x5576d5e0de60 2024-04-21T08:41:34.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.115+0000 7f3973fff640 1 --2- 172.21.15.86:0/4013763487 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d5e04d60 0x5576d5e1a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:34.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.115+0000 7f3978f99640 1 --2- 172.21.15.86:0/4013763487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d5e043b0 0x5576d5e19f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:34.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.115+0000 7f397979a640 1 --2- 172.21.15.86:0/4013763487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d5e0de60 0x5576d5e0e3f0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f396c021d60 tx=0x7f396c011be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a058d59fcf011045 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:34.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.115+0000 7f3953fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:34.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.115+0000 7f3953fff640 1 -- 172.21.15.86:0/4013763487 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f396c037060 con 0x5576d5e0de60 2024-04-21T08:41:34.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.115+0000 7f3953fff640 1 -- 172.21.15.86:0/4013763487 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f396c011e60 con 0x5576d5e0de60 2024-04-21T08:41:34.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.115+0000 7f397dddf740 1 -- 172.21.15.86:0/4013763487 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f396c012e20 con 0x5576d5e0de60 2024-04-21T08:41:34.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.115+0000 7f3953fff640 1 -- 172.21.15.86:0/4013763487 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f396c03e050 con 0x5576d5e0de60 2024-04-21T08:41:34.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.115+0000 7f397dddf740 10 client.17040 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:34.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.115+0000 7f397dddf740 1 -- 172.21.15.86:0/4013763487 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5576d5e87ea0 con 0x5576d5e0de60 2024-04-21T08:41:34.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.115+0000 7f3953fff640 1 -- 172.21.15.86:0/4013763487 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f396c012e20 con 0x5576d5e0de60 2024-04-21T08:41:34.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.116+0000 7f3953fff640 1 -- 172.21.15.86:0/4013763487 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f396c00f040 con 0x5576d5e0de60 2024-04-21T08:41:34.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.116+0000 7f397dddf740 10 client.17040 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:34.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.116+0000 7f397dddf740 10 client.17040 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:34.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.116+0000 7f397dddf740 20 client.17040 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:34.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.116+0000 7f397dddf740 1 --2- 172.21.15.86:0/4013763487 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5576d5e88530 0x5576d5e8a950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:34.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.116+0000 7f397dddf740 4 client.17040 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:41:34.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.116+0000 7f397dddf740 1 -- 172.21.15.86:0/4013763487 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5576d5d70a80 con 0x5576d5e88530 2024-04-21T08:41:34.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.116+0000 7f3978f99640 1 --2- 172.21.15.86:0/4013763487 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5576d5e88530 0x5576d5e8a950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:34.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.116+0000 7f3953fff640 10 client.17040 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:34.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.116+0000 7f3978f99640 1 --2- 172.21.15.86:0/4013763487 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5576d5e88530 0x5576d5e8a950 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32f3013213dba603 server_cookie=ad0f03f0b55df3b9 in_seq=0 out_seq=0 2024-04-21T08:41:34.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.116+0000 7f3953fff640 1 -- 172.21.15.86:0/4013763487 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5576d5d70a80 con 0x5576d5e88530 2024-04-21T08:41:34.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.116+0000 7f3953fff640 10 client.17040 handle_command_reply: tid=0 2024-04-21T08:41:34.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.116+0000 7f397dddf740 1 client.17040 shutdown 2024-04-21T08:41:34.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.116+0000 7f397dddf740 1 -- 172.21.15.86:0/4013763487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d5e0de60 msgr2=0x5576d5e0e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:34.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.116+0000 7f397dddf740 1 --2- 172.21.15.86:0/4013763487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d5e0de60 0x5576d5e0e3f0 secure :-1 s=READY pgs=958 cs=0 l=1 rev1=1 crypto rx=0x7f396c021d60 tx=0x7f396c011be0 comp rx=0 tx=0).stop 2024-04-21T08:41:34.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.117+0000 7f397dddf740 1 -- 172.21.15.86:0/4013763487 shutdown_connections 2024-04-21T08:41:34.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.117+0000 7f397dddf740 1 --2- 172.21.15.86:0/4013763487 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5576d5e88530 0x5576d5e8a950 crc :-1 s=READY pgs=116 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:34.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.117+0000 7f397dddf740 1 --2- 172.21.15.86:0/4013763487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d5e0de60 0x5576d5e0e3f0 unknown :-1 s=CLOSED pgs=958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:34.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.117+0000 7f3953fff640 0 client.17040 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:34.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.117+0000 7f397dddf740 1 --2- 172.21.15.86:0/4013763487 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d5e04d60 0x5576d5e1a470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:34.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.117+0000 7f397dddf740 1 --2- 172.21.15.86:0/4013763487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d5e043b0 0x5576d5e19f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:34.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.117+0000 7f397dddf740 1 -- 172.21.15.86:0/4013763487 >> 172.21.15.86:0/4013763487 conn(0x5576d5dfeba0 msgr2=0x5576d5e0cde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:34.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.117+0000 7f397dddf740 1 -- 172.21.15.86:0/4013763487 shutdown_connections 2024-04-21T08:41:34.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.117+0000 7f397dddf740 1 -- 172.21.15.86:0/4013763487 wait complete. 2024-04-21T08:41:34.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.117+0000 7f397dddf740 20 client.17040 trim_cache size 0 max 16384 2024-04-21T08:41:34.123 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:41:34.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.117+0000 7f397b1fd640 1 -- 172.21.15.86:0/759510188 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f39640a3d20 msgr2=0x7f39640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:34.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.117+0000 7f397b1fd640 1 --2- 172.21.15.86:0/759510188 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f39640a3d20 0x7f39640a61e0 secure :-1 s=READY pgs=143 cs=0 l=1 rev1=1 crypto rx=0x7f396c0092f0 tx=0x7f396c010740 comp rx=0 tx=0).stop 2024-04-21T08:41:34.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.117+0000 7f397b1fd640 1 -- 172.21.15.86:0/759510188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39741021a0 msgr2=0x7f397419e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:34.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.117+0000 7f397b1fd640 1 --2- 172.21.15.86:0/759510188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39741021a0 0x7f397419e420 secure :-1 s=READY pgs=954 cs=0 l=1 rev1=1 crypto rx=0x7f3968005940 tx=0x7f396802c7e0 comp rx=0 tx=0).stop 2024-04-21T08:41:34.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.117+0000 7f397b1fd640 1 -- 172.21.15.86:0/759510188 shutdown_connections 2024-04-21T08:41:34.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.117+0000 7f397b1fd640 1 --2- 172.21.15.86:0/759510188 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f39640a3d20 0x7f39640a61e0 unknown :-1 s=CLOSED pgs=143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:34.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.117+0000 7f397b1fd640 1 --2- 172.21.15.86:0/759510188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39741985d0 0x7f3974198a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:34.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.117+0000 7f397b1fd640 1 --2- 172.21.15.86:0/759510188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3974107270 0x7f397419e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:34.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.117+0000 7f397b1fd640 1 --2- 172.21.15.86:0/759510188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39741021a0 0x7f397419e420 unknown :-1 s=CLOSED pgs=954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:34.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.117+0000 7f397b1fd640 1 -- 172.21.15.86:0/759510188 >> 172.21.15.86:0/759510188 conn(0x7f39740fd9c0 msgr2=0x7f39740fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:34.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.117+0000 7f397b1fd640 1 -- 172.21.15.86:0/759510188 shutdown_connections 2024-04-21T08:41:34.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:34.117+0000 7f397b1fd640 1 -- 172.21.15.86:0/759510188 wait complete. 2024-04-21T08:41:34.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:34 smithi086 ceph-mon[27939]: pgmap v86: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 17 op/s 2024-04-21T08:41:34.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:34 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3306843758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:41:34.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:34 smithi088 ceph-mon[31259]: pgmap v86: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 17 op/s 2024-04-21T08:41:34.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:34 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3306843758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:41:34.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:34 smithi179 ceph-mon[31829]: pgmap v86: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 8.7 KiB/s wr, 17 op/s 2024-04-21T08:41:34.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:34 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3306843758' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:41:35.127 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:41:35.128 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:41:35.128 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:41:35.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.243+0000 7f2fb2798640 1 -- 172.21.15.86:0/1512834256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fac106380 msgr2=0x7f2fac10df50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:35.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.243+0000 7f2fb2798640 1 --2- 172.21.15.86:0/1512834256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fac106380 0x7f2fac10df50 secure :-1 s=READY pgs=959 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4009ca0 tx=0x7f2fa402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.243+0000 7f2fb2798640 1 -- 172.21.15.86:0/1512834256 shutdown_connections 2024-04-21T08:41:35.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.243+0000 7f2fb2798640 1 --2- 172.21.15.86:0/1512834256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fac106380 0x7f2fac10df50 unknown :-1 s=CLOSED pgs=959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.243+0000 7f2fb2798640 1 --2- 172.21.15.86:0/1512834256 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2fac102180 0x7f2fac105ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.243+0000 7f2fb2798640 1 --2- 172.21.15.86:0/1512834256 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fac1017d0 0x7f2fac101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.243+0000 7f2fb2798640 1 -- 172.21.15.86:0/1512834256 >> 172.21.15.86:0/1512834256 conn(0x7f2fac077410 msgr2=0x7f2fac0fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:35.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.243+0000 7f2fb2798640 1 -- 172.21.15.86:0/1512834256 shutdown_connections 2024-04-21T08:41:35.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.243+0000 7f2fb2798640 1 -- 172.21.15.86:0/1512834256 wait complete. 2024-04-21T08:41:35.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.244+0000 7f2fb2798640 1 Processor -- start 2024-04-21T08:41:35.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.244+0000 7f2fb2798640 1 -- start start 2024-04-21T08:41:35.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.244+0000 7f2fb2798640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2fac1017d0 0x7f2fac19e610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.244+0000 7f2fb2798640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fac106380 0x7f2fac19eb50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.244+0000 7f2fb2798640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fac198780 0x7f2fac198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.244+0000 7f2fb2798640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2fac1112f0 con 0x7f2fac198780 2024-04-21T08:41:35.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.244+0000 7f2fb2798640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2fac111170 con 0x7f2fac106380 2024-04-21T08:41:35.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.244+0000 7f2fb2798640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2fac111470 con 0x7f2fac1017d0 2024-04-21T08:41:35.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.244+0000 7f2fb0d35640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fac198780 0x7f2fac198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.244+0000 7f2fb0d35640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fac198780 0x7f2fac198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48986/0 (socket says 172.21.15.86:48986) 2024-04-21T08:41:35.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.244+0000 7f2fb0d35640 1 -- 172.21.15.86:0/4077924230 learned_addr learned my addr 172.21.15.86:0/4077924230 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:35.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.245+0000 7f2fab7fe640 1 --2- 172.21.15.86:0/4077924230 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fac106380 0x7f2fac19eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.245+0000 7f2fb0d35640 1 -- 172.21.15.86:0/4077924230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2fac1017d0 msgr2=0x7f2fac19e610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:35.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.245+0000 7f2fb0d35640 1 --2- 172.21.15.86:0/4077924230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2fac1017d0 0x7f2fac19e610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.245+0000 7f2fb0d35640 1 -- 172.21.15.86:0/4077924230 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fac106380 msgr2=0x7f2fac19eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:35.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.245+0000 7f2fb0d35640 1 --2- 172.21.15.86:0/4077924230 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fac106380 0x7f2fac19eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.245+0000 7f2fb0d35640 1 -- 172.21.15.86:0/4077924230 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2fac199310 con 0x7f2fac198780 2024-04-21T08:41:35.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.245+0000 7f2fab7fe640 1 --2- 172.21.15.86:0/4077924230 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fac106380 0x7f2fac19eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:35.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.245+0000 7f2fb0d35640 1 --2- 172.21.15.86:0/4077924230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fac198780 0x7f2fac198c30 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4005e30 tx=0x7f2fa4032420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:35.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.245+0000 7f2fa97fa640 1 -- 172.21.15.86:0/4077924230 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4041020 con 0x7f2fac198780 2024-04-21T08:41:35.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.245+0000 7f2fa97fa640 1 -- 172.21.15.86:0/4077924230 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fa4032970 con 0x7f2fac198780 2024-04-21T08:41:35.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.245+0000 7f2fb2798640 1 -- 172.21.15.86:0/4077924230 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2fac1995a0 con 0x7f2fac198780 2024-04-21T08:41:35.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.245+0000 7f2fa97fa640 1 -- 172.21.15.86:0/4077924230 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa4039040 con 0x7f2fac198780 2024-04-21T08:41:35.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.245+0000 7f2fb2798640 1 -- 172.21.15.86:0/4077924230 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2fac1a5780 con 0x7f2fac198780 2024-04-21T08:41:35.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.247+0000 7f2fa97fa640 1 -- 172.21.15.86:0/4077924230 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2fa4044020 con 0x7f2fac198780 2024-04-21T08:41:35.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.248+0000 7f2fb5381740 1 Processor -- start 2024-04-21T08:41:35.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.248+0000 7f2fb5381740 1 -- start start 2024-04-21T08:41:35.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.248+0000 7f2fb5381740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653358208e0 0x565335820cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.248+0000 7f2fb5381740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x565335821290 0x565335829840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.248+0000 7f2fb5381740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565335829e10 0x56533582c260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.248+0000 7f2fb5381740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x565335784f00 con 0x565335829e10 2024-04-21T08:41:35.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.248+0000 7f2fb5381740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x565335722590 con 0x565335821290 2024-04-21T08:41:35.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.248+0000 7f2fb5381740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x565335780ec0 con 0x5653358208e0 2024-04-21T08:41:35.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.248+0000 7f2fa97fa640 1 --2- 172.21.15.86:0/4077924230 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2f940a3da0 0x7f2f940a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.248+0000 7f2fab7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565335829e10 0x56533582c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.248+0000 7f2fb0d35640 1 --2- 172.21.15.86:0/4077924230 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2f940a3da0 0x7f2f940a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.248+0000 7f2fab7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565335829e10 0x56533582c260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48988/0 (socket says 172.21.15.86:48988) 2024-04-21T08:41:35.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.248+0000 7f2fab7fe640 1 -- 172.21.15.86:0/3715174703 learned_addr learned my addr 172.21.15.86:0/3715174703 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:35.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.248+0000 7f2fabfff640 1 --2- 172.21.15.86:0/3715174703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x565335821290 0x565335829840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.248+0000 7f2fab7fe640 1 --2- 172.21.15.86:0/3715174703 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653358208e0 0x565335820cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.248+0000 7f2fa97fa640 1 -- 172.21.15.86:0/4077924230 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f2fa40f2820 con 0x7f2fac198780 2024-04-21T08:41:35.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.248+0000 7f2fab7fe640 1 -- 172.21.15.86:0/3715174703 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653358208e0 msgr2=0x565335820cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:35.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.248+0000 7f2fab7fe640 1 --2- 172.21.15.86:0/3715174703 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653358208e0 0x565335820cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.248+0000 7f2fab7fe640 1 -- 172.21.15.86:0/3715174703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x565335821290 msgr2=0x565335829840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:35.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.248+0000 7f2fab7fe640 1 --2- 172.21.15.86:0/3715174703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x565335821290 0x565335829840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.248+0000 7f2fab7fe640 1 -- 172.21.15.86:0/3715174703 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56533555f7e0 con 0x565335829e10 2024-04-21T08:41:35.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.249+0000 7f2fb0d35640 1 --2- 172.21.15.86:0/4077924230 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2f940a3da0 0x7f2f940a6260 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2fa40c6ca0 tx=0x7f2fa40ba0e0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:35.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.249+0000 7f2fab7fe640 1 --2- 172.21.15.86:0/3715174703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565335829e10 0x56533582c260 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0005fd0 tx=0x7f2fa000e3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59f6922aae13a44f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:35.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.249+0000 7f2f927fc640 1 -- 172.21.15.86:0/3715174703 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa000b680 con 0x565335829e10 2024-04-21T08:41:35.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.249+0000 7f2f927fc640 1 -- 172.21.15.86:0/3715174703 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fa000bcd0 con 0x565335829e10 2024-04-21T08:41:35.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.249+0000 7f2f927fc640 1 -- 172.21.15.86:0/3715174703 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa0011cc0 con 0x565335829e10 2024-04-21T08:41:35.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.249+0000 7f2fb5381740 1 -- 172.21.15.86:0/3715174703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565335829e10 msgr2=0x56533582c260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:35.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.249+0000 7f2fb5381740 1 --2- 172.21.15.86:0/3715174703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565335829e10 0x56533582c260 secure :-1 s=READY pgs=961 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0005fd0 tx=0x7f2fa000e3d0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.249+0000 7f2fb5381740 1 -- 172.21.15.86:0/3715174703 shutdown_connections 2024-04-21T08:41:35.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.249+0000 7f2fb5381740 1 --2- 172.21.15.86:0/3715174703 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565335829e10 0x56533582c260 unknown :-1 s=CLOSED pgs=961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.249+0000 7f2fb5381740 1 --2- 172.21.15.86:0/3715174703 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x565335821290 0x565335829840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.249+0000 7f2fb5381740 1 --2- 172.21.15.86:0/3715174703 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653358208e0 0x565335820cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.249+0000 7f2fb5381740 1 -- 172.21.15.86:0/3715174703 >> 172.21.15.86:0/3715174703 conn(0x56533581afb0 msgr2=0x56533581b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:35.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.249+0000 7f2fb5381740 1 -- 172.21.15.86:0/3715174703 shutdown_connections 2024-04-21T08:41:35.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.249+0000 7f2fb5381740 1 -- 172.21.15.86:0/3715174703 wait complete. 2024-04-21T08:41:35.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.250+0000 7f2fb5381740 1 -- start start 2024-04-21T08:41:35.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.250+0000 7f2fb5381740 1 Processor -- start 2024-04-21T08:41:35.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.250+0000 7f2fb5381740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653358208e0 0x565335826e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.250+0000 7f2fb5381740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565335821290 0x565335827360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.250+0000 7f2fb5381740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x565335829e10 0x5653358278a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.250+0000 7f2fb5381740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56533582cd80 con 0x5653358208e0 2024-04-21T08:41:35.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.250+0000 7f2fb5381740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x56533573e580 con 0x565335829e10 2024-04-21T08:41:35.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.250+0000 7f2fb5381740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56533582cf00 con 0x565335821290 2024-04-21T08:41:35.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.250+0000 7f2fb5381740 10 client.0 _ll_register_callbacks cb 0x565335744650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:35.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.250+0000 7f2fab7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653358208e0 0x565335826e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.250+0000 7f2fab7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653358208e0 0x565335826e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48992/0 (socket says 172.21.15.86:48992) 2024-04-21T08:41:35.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.250+0000 7f2fab7fe640 1 -- 172.21.15.86:0/342509339 learned_addr learned my addr 172.21.15.86:0/342509339 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:35.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.251+0000 7f2fabfff640 1 --2- 172.21.15.86:0/342509339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565335821290 0x565335827360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.251+0000 7f2fab7fe640 1 -- 172.21.15.86:0/342509339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565335821290 msgr2=0x565335827360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:35.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.251+0000 7f2fab7fe640 1 --2- 172.21.15.86:0/342509339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565335821290 0x565335827360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.251+0000 7f2fab7fe640 1 -- 172.21.15.86:0/342509339 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x565335829e10 msgr2=0x5653358278a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:35.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.251+0000 7f2fab7fe640 1 --2- 172.21.15.86:0/342509339 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x565335829e10 0x5653358278a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.251+0000 7f2fab7fe640 1 -- 172.21.15.86:0/342509339 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653355bff70 con 0x5653358208e0 2024-04-21T08:41:35.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.251+0000 7f2fab7fe640 1 --2- 172.21.15.86:0/342509339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653358208e0 0x565335826e20 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f2fa000bfa0 tx=0x7f2fa0013520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ddd4671fd046be4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:35.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.251+0000 7f2f8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:35.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.251+0000 7f2f8b7fe640 1 -- 172.21.15.86:0/342509339 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa0011e50 con 0x5653358208e0 2024-04-21T08:41:35.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.251+0000 7f2f8b7fe640 1 -- 172.21.15.86:0/342509339 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f2fa000b680 con 0x5653358208e0 2024-04-21T08:41:35.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.251+0000 7f2fb5381740 1 -- 172.21.15.86:0/342509339 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2fa000bcd0 con 0x5653358208e0 2024-04-21T08:41:35.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.251+0000 7f2f8b7fe640 1 -- 172.21.15.86:0/342509339 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2fa002f050 con 0x5653358208e0 2024-04-21T08:41:35.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.251+0000 7f2fb5381740 10 client.17052 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:35.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.251+0000 7f2fb5381740 1 -- 172.21.15.86:0/342509339 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x565335457170 con 0x5653358208e0 2024-04-21T08:41:35.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.251+0000 7f2f8b7fe640 1 -- 172.21.15.86:0/342509339 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2fa000bcd0 con 0x5653358208e0 2024-04-21T08:41:35.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.251+0000 7f2f8b7fe640 1 -- 172.21.15.86:0/342509339 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2fa0025030 con 0x5653358208e0 2024-04-21T08:41:35.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.252+0000 7f2fb5381740 10 client.17052 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:35.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.252+0000 7f2fb5381740 10 client.17052 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:35.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.252+0000 7f2fb5381740 20 client.17052 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:35.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.252+0000 7f2fb5381740 1 --2- 172.21.15.86:0/342509339 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56533583a2b0 0x56533583c6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.252+0000 7f2fb5381740 4 client.17052 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:35.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.252+0000 7f2fb5381740 1 -- 172.21.15.86:0/342509339 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x565335792920 con 0x56533583a2b0 2024-04-21T08:41:35.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.252+0000 7f2fabfff640 1 --2- 172.21.15.86:0/342509339 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56533583a2b0 0x56533583c6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.252+0000 7f2f8b7fe640 10 client.17052 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:35.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.252+0000 7f2fabfff640 1 --2- 172.21.15.86:0/342509339 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56533583a2b0 0x56533583c6a0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f1d894ef3cd565f1 server_cookie=6d737b9b3ba3c451 in_seq=0 out_seq=0 2024-04-21T08:41:35.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.253+0000 7f2f8b7fe640 1 -- 172.21.15.86:0/342509339 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x565335792920 con 0x56533583a2b0 2024-04-21T08:41:35.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.253+0000 7f2f8b7fe640 10 client.17052 handle_command_reply: tid=0 2024-04-21T08:41:35.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.253+0000 7f2fb5381740 1 client.17052 shutdown 2024-04-21T08:41:35.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.253+0000 7f2fb5381740 1 -- 172.21.15.86:0/342509339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653358208e0 msgr2=0x565335826e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:35.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.253+0000 7f2fb5381740 1 --2- 172.21.15.86:0/342509339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653358208e0 0x565335826e20 secure :-1 s=READY pgs=962 cs=0 l=1 rev1=1 crypto rx=0x7f2fa000bfa0 tx=0x7f2fa0013520 comp rx=0 tx=0).stop 2024-04-21T08:41:35.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.253+0000 7f2fb5381740 1 -- 172.21.15.86:0/342509339 shutdown_connections 2024-04-21T08:41:35.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.253+0000 7f2fb5381740 1 --2- 172.21.15.86:0/342509339 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56533583a2b0 0x56533583c6a0 crc :-1 s=READY pgs=117 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.253+0000 7f2fb5381740 1 --2- 172.21.15.86:0/342509339 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x565335829e10 0x5653358278a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.253+0000 7f2fb5381740 1 --2- 172.21.15.86:0/342509339 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565335821290 0x565335827360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.253+0000 7f2f8b7fe640 0 client.17052 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:35.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.253+0000 7f2fb5381740 1 --2- 172.21.15.86:0/342509339 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653358208e0 0x565335826e20 unknown :-1 s=CLOSED pgs=962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.253+0000 7f2fb5381740 1 -- 172.21.15.86:0/342509339 >> 172.21.15.86:0/342509339 conn(0x56533581afb0 msgr2=0x56533581cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:35.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.253+0000 7f2fb5381740 1 -- 172.21.15.86:0/342509339 shutdown_connections 2024-04-21T08:41:35.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.253+0000 7f2fb5381740 1 -- 172.21.15.86:0/342509339 wait complete. 2024-04-21T08:41:35.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.253+0000 7f2fb5381740 20 client.17052 trim_cache size 0 max 16384 2024-04-21T08:41:35.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.271+0000 7f2fb5381740 1 Processor -- start 2024-04-21T08:41:35.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.271+0000 7f2fb5381740 1 -- start start 2024-04-21T08:41:35.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.271+0000 7f2fb5381740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5653358208e0 0x565335829550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.271+0000 7f2fb5381740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565335821290 0x565335835c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.271+0000 7f2fb5381740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565335829e10 0x565335819e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.271+0000 7f2fb5381740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2fac1af2f0 con 0x565335821290 2024-04-21T08:41:35.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.271+0000 7f2fb5381740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2fac1b0e30 con 0x5653358208e0 2024-04-21T08:41:35.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.271+0000 7f2fb5381740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2fac1af9c0 con 0x565335829e10 2024-04-21T08:41:35.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.271+0000 7f2fabfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565335821290 0x565335835c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.271+0000 7f2fabfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565335821290 0x565335835c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48994/0 (socket says 172.21.15.86:48994) 2024-04-21T08:41:35.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.271+0000 7f2fabfff640 1 -- 172.21.15.86:0/3494028212 learned_addr learned my addr 172.21.15.86:0/3494028212 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:35.317 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.271+0000 7f2fabfff640 1 -- 172.21.15.86:0/3494028212 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565335829e10 msgr2=0x565335819e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:35.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.271+0000 7f2fabfff640 1 --2- 172.21.15.86:0/3494028212 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565335829e10 0x565335819e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.271+0000 7f2fabfff640 1 -- 172.21.15.86:0/3494028212 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5653358208e0 msgr2=0x565335829550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:35.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.271+0000 7f2fab7fe640 1 --2- 172.21.15.86:0/3494028212 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5653358208e0 0x565335829550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.271+0000 7f2fabfff640 1 --2- 172.21.15.86:0/3494028212 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5653358208e0 0x565335829550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.271+0000 7f2fabfff640 1 -- 172.21.15.86:0/3494028212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56533582c5d0 con 0x565335821290 2024-04-21T08:41:35.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.271+0000 7f2fabfff640 1 --2- 172.21.15.86:0/3494028212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565335821290 0x565335835c40 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c009c60 tx=0x7f2f9c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8377ac833de13943 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:35.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.272+0000 7f2f91ffb640 1 -- 172.21.15.86:0/3494028212 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c022070 con 0x565335821290 2024-04-21T08:41:35.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.272+0000 7f2f91ffb640 1 -- 172.21.15.86:0/3494028212 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f9c01d8e0 con 0x565335821290 2024-04-21T08:41:35.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.272+0000 7f2f91ffb640 1 -- 172.21.15.86:0/3494028212 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2f9c003b90 con 0x565335821290 2024-04-21T08:41:35.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.272+0000 7f2fb5381740 1 -- 172.21.15.86:0/3494028212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565335821290 msgr2=0x565335835c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:35.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.272+0000 7f2fb5381740 1 --2- 172.21.15.86:0/3494028212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565335821290 0x565335835c40 secure :-1 s=READY pgs=963 cs=0 l=1 rev1=1 crypto rx=0x7f2f9c009c60 tx=0x7f2f9c0046c0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.272+0000 7f2fb5381740 1 -- 172.21.15.86:0/3494028212 shutdown_connections 2024-04-21T08:41:35.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.272+0000 7f2fb5381740 1 --2- 172.21.15.86:0/3494028212 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565335829e10 0x565335819e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.318 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.272+0000 7f2fb5381740 1 --2- 172.21.15.86:0/3494028212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565335821290 0x565335835c40 unknown :-1 s=CLOSED pgs=963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.272+0000 7f2fb5381740 1 --2- 172.21.15.86:0/3494028212 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5653358208e0 0x565335829550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.272+0000 7f2fb5381740 1 -- 172.21.15.86:0/3494028212 >> 172.21.15.86:0/3494028212 conn(0x56533581afb0 msgr2=0x565335819780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:35.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.272+0000 7f2fb5381740 1 -- 172.21.15.86:0/3494028212 shutdown_connections 2024-04-21T08:41:35.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.272+0000 7f2fb5381740 1 -- 172.21.15.86:0/3494028212 wait complete. 2024-04-21T08:41:35.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.272+0000 7f2fb5381740 1 -- start start 2024-04-21T08:41:35.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.273+0000 7f2fb5381740 1 Processor -- start 2024-04-21T08:41:35.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.273+0000 7f2fb5381740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653358208e0 0x56533582de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.273+0000 7f2fb5381740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x565335821290 0x56533582e390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.273+0000 7f2fb5381740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565335829e10 0x56533582e910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.273+0000 7f2fb5381740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2fac1b0760 con 0x5653358208e0 2024-04-21T08:41:35.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.273+0000 7f2fb5381740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2fac1b0090 con 0x565335821290 2024-04-21T08:41:35.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.273+0000 7f2fb5381740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2fac1ad2b0 con 0x565335829e10 2024-04-21T08:41:35.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.273+0000 7f2fb5381740 10 client.0 _ll_register_callbacks cb 0x7f2f9c009570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:35.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.273+0000 7f2fab7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653358208e0 0x56533582de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.273+0000 7f2fab7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653358208e0 0x56533582de50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49000/0 (socket says 172.21.15.86:49000) 2024-04-21T08:41:35.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.273+0000 7f2fab7fe640 1 -- 172.21.15.86:0/2857748068 learned_addr learned my addr 172.21.15.86:0/2857748068 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:35.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.273+0000 7f2fab7fe640 1 --2- 172.21.15.86:0/2857748068 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565335829e10 0x56533582e910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.273+0000 7f2fabfff640 1 --2- 172.21.15.86:0/2857748068 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x565335821290 0x56533582e390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.273+0000 7f2fab7fe640 1 -- 172.21.15.86:0/2857748068 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565335829e10 msgr2=0x56533582e910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:35.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.273+0000 7f2fab7fe640 1 --2- 172.21.15.86:0/2857748068 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565335829e10 0x56533582e910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.273+0000 7f2fab7fe640 1 -- 172.21.15.86:0/2857748068 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x565335821290 msgr2=0x56533582e390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:35.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.273+0000 7f2fab7fe640 1 --2- 172.21.15.86:0/2857748068 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x565335821290 0x56533582e390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.273+0000 7f2fab7fe640 1 -- 172.21.15.86:0/2857748068 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56533589f940 con 0x5653358208e0 2024-04-21T08:41:35.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.273+0000 7f2fabfff640 1 --2- 172.21.15.86:0/2857748068 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x565335821290 0x56533582e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:35.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.273+0000 7f2fab7fe640 1 --2- 172.21.15.86:0/2857748068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653358208e0 0x56533582de50 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0005c80 tx=0x7f2fa00118c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bfa7898fa1d54568 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:35.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.273+0000 7f2f8b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:35.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.273+0000 7f2f8b7fe640 1 -- 172.21.15.86:0/2857748068 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2fa000e870 con 0x5653358208e0 2024-04-21T08:41:35.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.274+0000 7f2f8b7fe640 1 -- 172.21.15.86:0/2857748068 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f2fa0004960 con 0x5653358208e0 2024-04-21T08:41:35.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.274+0000 7f2fb5381740 1 -- 172.21.15.86:0/2857748068 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2f9c01d8e0 con 0x5653358208e0 2024-04-21T08:41:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.274+0000 7f2f8b7fe640 1 -- 172.21.15.86:0/2857748068 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2fa0004b60 con 0x5653358208e0 2024-04-21T08:41:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.274+0000 7f2fb5381740 10 client.17058 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.274+0000 7f2fb5381740 1 -- 172.21.15.86:0/2857748068 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5653358a0c80 con 0x5653358208e0 2024-04-21T08:41:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.274+0000 7f2f8b7fe640 1 -- 172.21.15.86:0/2857748068 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2f9c01d8e0 con 0x5653358208e0 2024-04-21T08:41:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.274+0000 7f2f8b7fe640 1 -- 172.21.15.86:0/2857748068 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2fa0025050 con 0x5653358208e0 2024-04-21T08:41:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.274+0000 7f2fb5381740 10 client.17058 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.274+0000 7f2fb5381740 10 client.17058 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.274+0000 7f2fb5381740 20 client.17058 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.274+0000 7f2fb5381740 1 --2- 172.21.15.86:0/2857748068 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5653358a1310 0x5653358a3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.274+0000 7f2fb5381740 4 client.17058 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:41:35.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.274+0000 7f2fb5381740 1 -- 172.21.15.86:0/2857748068 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x56533578c790 con 0x5653358a1310 2024-04-21T08:41:35.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.274+0000 7f2fabfff640 1 --2- 172.21.15.86:0/2857748068 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5653358a1310 0x5653358a3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.275+0000 7f2f8b7fe640 10 client.17058 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:35.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.275+0000 7f2fabfff640 1 --2- 172.21.15.86:0/2857748068 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5653358a1310 0x5653358a3730 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7fd78404bcdc83d3 server_cookie=de3bc618d3a982ac in_seq=0 out_seq=0 2024-04-21T08:41:35.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.276+0000 7f2f8b7fe640 1 -- 172.21.15.86:0/2857748068 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x56533578c790 con 0x5653358a1310 2024-04-21T08:41:35.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.276+0000 7f2f8b7fe640 10 client.17058 handle_command_reply: tid=0 2024-04-21T08:41:35.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.276+0000 7f2fb5381740 1 client.17058 shutdown 2024-04-21T08:41:35.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.276+0000 7f2fb5381740 1 -- 172.21.15.86:0/2857748068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653358208e0 msgr2=0x56533582de50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:35.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.276+0000 7f2fb5381740 1 --2- 172.21.15.86:0/2857748068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653358208e0 0x56533582de50 secure :-1 s=READY pgs=964 cs=0 l=1 rev1=1 crypto rx=0x7f2fa0005c80 tx=0x7f2fa00118c0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.276+0000 7f2fb5381740 1 -- 172.21.15.86:0/2857748068 shutdown_connections 2024-04-21T08:41:35.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.276+0000 7f2fb5381740 1 --2- 172.21.15.86:0/2857748068 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5653358a1310 0x5653358a3730 crc :-1 s=READY pgs=118 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.276+0000 7f2fb5381740 1 --2- 172.21.15.86:0/2857748068 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565335829e10 0x56533582e910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.276+0000 7f2fb5381740 1 --2- 172.21.15.86:0/2857748068 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x565335821290 0x56533582e390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.276+0000 7f2fb5381740 1 --2- 172.21.15.86:0/2857748068 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653358208e0 0x56533582de50 unknown :-1 s=CLOSED pgs=964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.276+0000 7f2f8b7fe640 0 client.17058 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:35.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.276+0000 7f2fb5381740 1 -- 172.21.15.86:0/2857748068 >> 172.21.15.86:0/2857748068 conn(0x56533581afb0 msgr2=0x565335827670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:35.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.276+0000 7f2fb5381740 1 -- 172.21.15.86:0/2857748068 shutdown_connections 2024-04-21T08:41:35.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.276+0000 7f2fb5381740 1 -- 172.21.15.86:0/2857748068 wait complete. 2024-04-21T08:41:35.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.276+0000 7f2fb5381740 20 client.17058 trim_cache size 0 max 16384 2024-04-21T08:41:35.323 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:41:35.323 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-21T08:41:35.323 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "1a0ecd81-d904-4483-bd50-fbdd506135c4", 2024-04-21T08:41:35.323 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-21T08:41:35.324 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:41:35.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.277+0000 7f2fb2798640 1 -- 172.21.15.86:0/4077924230 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2f940a3da0 msgr2=0x7f2f940a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:35.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.277+0000 7f2fb2798640 1 --2- 172.21.15.86:0/4077924230 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2f940a3da0 0x7f2f940a6260 secure :-1 s=READY pgs=144 cs=0 l=1 rev1=1 crypto rx=0x7f2fa40c6ca0 tx=0x7f2fa40ba0e0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.277+0000 7f2fb2798640 1 -- 172.21.15.86:0/4077924230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fac198780 msgr2=0x7f2fac198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:35.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.277+0000 7f2fb2798640 1 --2- 172.21.15.86:0/4077924230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fac198780 0x7f2fac198c30 secure :-1 s=READY pgs=960 cs=0 l=1 rev1=1 crypto rx=0x7f2fa4005e30 tx=0x7f2fa4032420 comp rx=0 tx=0).stop 2024-04-21T08:41:35.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.277+0000 7f2fb2798640 1 -- 172.21.15.86:0/4077924230 shutdown_connections 2024-04-21T08:41:35.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.277+0000 7f2fb2798640 1 --2- 172.21.15.86:0/4077924230 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2f940a3da0 0x7f2f940a6260 unknown :-1 s=CLOSED pgs=144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.277+0000 7f2fb2798640 1 --2- 172.21.15.86:0/4077924230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2fac198780 0x7f2fac198c30 unknown :-1 s=CLOSED pgs=960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.277+0000 7f2fb2798640 1 --2- 172.21.15.86:0/4077924230 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2fac106380 0x7f2fac19eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.277+0000 7f2fb2798640 1 --2- 172.21.15.86:0/4077924230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2fac1017d0 0x7f2fac19e610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.277+0000 7f2fb2798640 1 -- 172.21.15.86:0/4077924230 >> 172.21.15.86:0/4077924230 conn(0x7f2fac077410 msgr2=0x7f2fac1006f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:35.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.277+0000 7f2fb2798640 1 -- 172.21.15.86:0/4077924230 shutdown_connections 2024-04-21T08:41:35.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.277+0000 7f2fb2798640 1 -- 172.21.15.86:0/4077924230 wait complete. 2024-04-21T08:41:35.326 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:41:35.326 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:41:35.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.426+0000 7fc7694c1640 1 -- 172.21.15.86:0/154770563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc764103ae0 msgr2=0x7fc76410bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:35.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.426+0000 7fc7694c1640 1 --2- 172.21.15.86:0/154770563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc764103ae0 0x7fc76410bfc0 secure :-1 s=READY pgs=965 cs=0 l=1 rev1=1 crypto rx=0x7fc758009900 tx=0x7fc75802bd60 comp rx=0 tx=0).stop 2024-04-21T08:41:35.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.426+0000 7fc7694c1640 1 -- 172.21.15.86:0/154770563 shutdown_connections 2024-04-21T08:41:35.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.426+0000 7fc7694c1640 1 --2- 172.21.15.86:0/154770563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc76410c500 0x7fc76410e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.426+0000 7fc7694c1640 1 --2- 172.21.15.86:0/154770563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc764103ae0 0x7fc76410bfc0 unknown :-1 s=CLOSED pgs=965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.426+0000 7fc7694c1640 1 --2- 172.21.15.86:0/154770563 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7641031c0 0x7fc7641035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.426+0000 7fc7694c1640 1 -- 172.21.15.86:0/154770563 >> 172.21.15.86:0/154770563 conn(0x7fc764078000 msgr2=0x7fc764078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:35.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.427+0000 7fc7694c1640 1 -- 172.21.15.86:0/154770563 shutdown_connections 2024-04-21T08:41:35.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.427+0000 7fc7694c1640 1 -- 172.21.15.86:0/154770563 wait complete. 2024-04-21T08:41:35.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.427+0000 7fc7694c1640 1 Processor -- start 2024-04-21T08:41:35.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.427+0000 7fc7694c1640 1 -- start start 2024-04-21T08:41:35.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.427+0000 7fc7694c1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7641031c0 0x7fc7641986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.427+0000 7fc7694c1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc764103ae0 0x7fc764198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.427+0000 7fc7694c1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc76410c500 0x7fc76419cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.427+0000 7fc7694c1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc764112190 con 0x7fc76410c500 2024-04-21T08:41:35.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.427+0000 7fc7694c1640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc764112010 con 0x7fc764103ae0 2024-04-21T08:41:35.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.427+0000 7fc7694c1640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc764112310 con 0x7fc7641031c0 2024-04-21T08:41:35.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.428+0000 7fc7637fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc76410c500 0x7fc76419cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.428+0000 7fc7637fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc76410c500 0x7fc76419cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49018/0 (socket says 172.21.15.86:49018) 2024-04-21T08:41:35.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.428+0000 7fc7637fe640 1 -- 172.21.15.86:0/1263342642 learned_addr learned my addr 172.21.15.86:0/1263342642 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:35.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.428+0000 7fc762ffd640 1 --2- 172.21.15.86:0/1263342642 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7641031c0 0x7fc7641986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.428+0000 7fc7627fc640 1 --2- 172.21.15.86:0/1263342642 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc764103ae0 0x7fc764198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.428+0000 7fc7637fe640 1 -- 172.21.15.86:0/1263342642 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7641031c0 msgr2=0x7fc7641986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:35.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.428+0000 7fc7637fe640 1 --2- 172.21.15.86:0/1263342642 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7641031c0 0x7fc7641986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.428+0000 7fc7637fe640 1 -- 172.21.15.86:0/1263342642 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc764103ae0 msgr2=0x7fc764198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:35.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.428+0000 7fc7637fe640 1 --2- 172.21.15.86:0/1263342642 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc764103ae0 0x7fc764198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.428+0000 7fc7637fe640 1 -- 172.21.15.86:0/1263342642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc76419d6a0 con 0x7fc76410c500 2024-04-21T08:41:35.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.428+0000 7fc7627fc640 1 --2- 172.21.15.86:0/1263342642 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc764103ae0 0x7fc764198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:35.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.428+0000 7fc7637fe640 1 --2- 172.21.15.86:0/1263342642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc76410c500 0x7fc76419cf70 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fc75400cce0 tx=0x7fc75400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:35.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.428+0000 7fc74bfff640 1 -- 172.21.15.86:0/1263342642 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75400f450 con 0x7fc76410c500 2024-04-21T08:41:35.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.428+0000 7fc7694c1640 1 -- 172.21.15.86:0/1263342642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc76419d990 con 0x7fc76410c500 2024-04-21T08:41:35.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.428+0000 7fc74bfff640 1 -- 172.21.15.86:0/1263342642 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc75400faa0 con 0x7fc76410c500 2024-04-21T08:41:35.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.428+0000 7fc74bfff640 1 -- 172.21.15.86:0/1263342642 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc754018b70 con 0x7fc76410c500 2024-04-21T08:41:35.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.428+0000 7fc7694c1640 1 -- 172.21.15.86:0/1263342642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc7641a5220 con 0x7fc76410c500 2024-04-21T08:41:35.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.430+0000 7fc74bfff640 1 -- 172.21.15.86:0/1263342642 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc754018d10 con 0x7fc76410c500 2024-04-21T08:41:35.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.431+0000 7fc76c192740 1 Processor -- start 2024-04-21T08:41:35.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.431+0000 7fc76c192740 1 -- start start 2024-04-21T08:41:35.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.431+0000 7fc74bfff640 1 --2- 172.21.15.86:0/1263342642 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc74c0a3d20 0x7fc74c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.431+0000 7fc76c192740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561cf0ee2b0 0x5561cf0ee690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.431+0000 7fc76c192740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5561cf0eebd0 0x5561cf0fb950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.431+0000 7fc76c192740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5561cf0fbe90 0x5561cf0fe280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.431+0000 7fc76c192740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5561cf0531e0 con 0x5561cf0ee2b0 2024-04-21T08:41:35.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.431+0000 7fc76c192740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5561ceff0390 con 0x5561cf0fbe90 2024-04-21T08:41:35.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.431+0000 7fc76c192740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5561cf04f1a0 con 0x5561cf0eebd0 2024-04-21T08:41:35.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.431+0000 7fc74bfff640 1 -- 172.21.15.86:0/1263342642 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fc7540d0f80 con 0x7fc76410c500 2024-04-21T08:41:35.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.431+0000 7fc7627fc640 1 --2- 172.21.15.86:0/1263342642 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc74c0a3d20 0x7fc74c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.431+0000 7fc762ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561cf0ee2b0 0x5561cf0ee690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.431+0000 7fc762ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561cf0ee2b0 0x5561cf0ee690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49022/0 (socket says 172.21.15.86:49022) 2024-04-21T08:41:35.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.431+0000 7fc762ffd640 1 -- 172.21.15.86:0/1948748290 learned_addr learned my addr 172.21.15.86:0/1948748290 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:35.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.431+0000 7fc7627fc640 1 --2- 172.21.15.86:0/1948748290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5561cf0eebd0 0x5561cf0fb950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.431+0000 7fc7637fe640 1 --2- 172.21.15.86:0/1948748290 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5561cf0fbe90 0x5561cf0fe280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.431+0000 7fc762ffd640 1 -- 172.21.15.86:0/1948748290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5561cf0eebd0 msgr2=0x5561cf0fb950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:35.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.431+0000 7fc762ffd640 1 --2- 172.21.15.86:0/1948748290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5561cf0eebd0 0x5561cf0fb950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.431+0000 7fc762ffd640 1 -- 172.21.15.86:0/1948748290 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5561cf0fbe90 msgr2=0x5561cf0fe280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:35.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.431+0000 7fc762ffd640 1 --2- 172.21.15.86:0/1948748290 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5561cf0fbe90 0x5561cf0fe280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.431+0000 7fc762ffd640 1 -- 172.21.15.86:0/1948748290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561cee2f7d0 con 0x5561cf0ee2b0 2024-04-21T08:41:35.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.431+0000 7fc7627fc640 1 --2- 172.21.15.86:0/1263342642 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc74c0a3d20 0x7fc74c0a61e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc758036040 tx=0x7fc758004390 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:35.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.432+0000 7fc762ffd640 1 --2- 172.21.15.86:0/1948748290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561cf0ee2b0 0x5561cf0ee690 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fc75c00b760 tx=0x7fc75c00bc20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=320fbac4851f40f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:35.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.432+0000 7fc7497fa640 1 -- 172.21.15.86:0/1948748290 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c003d50 con 0x5561cf0ee2b0 2024-04-21T08:41:35.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.432+0000 7fc7497fa640 1 -- 172.21.15.86:0/1948748290 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc75c015070 con 0x5561cf0ee2b0 2024-04-21T08:41:35.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.432+0000 7fc7497fa640 1 -- 172.21.15.86:0/1948748290 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc75c004c90 con 0x5561cf0ee2b0 2024-04-21T08:41:35.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.432+0000 7fc76c192740 1 -- 172.21.15.86:0/1948748290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561cf0ee2b0 msgr2=0x5561cf0ee690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:35.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.432+0000 7fc76c192740 1 --2- 172.21.15.86:0/1948748290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561cf0ee2b0 0x5561cf0ee690 secure :-1 s=READY pgs=967 cs=0 l=1 rev1=1 crypto rx=0x7fc75c00b760 tx=0x7fc75c00bc20 comp rx=0 tx=0).stop 2024-04-21T08:41:35.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.432+0000 7fc76c192740 1 -- 172.21.15.86:0/1948748290 shutdown_connections 2024-04-21T08:41:35.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.432+0000 7fc76c192740 1 --2- 172.21.15.86:0/1948748290 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5561cf0fbe90 0x5561cf0fe280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.432+0000 7fc76c192740 1 --2- 172.21.15.86:0/1948748290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5561cf0eebd0 0x5561cf0fb950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.432+0000 7fc76c192740 1 --2- 172.21.15.86:0/1948748290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561cf0ee2b0 0x5561cf0ee690 unknown :-1 s=CLOSED pgs=967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.432+0000 7fc76c192740 1 -- 172.21.15.86:0/1948748290 >> 172.21.15.86:0/1948748290 conn(0x5561cf0e8cb0 msgr2=0x5561cf0e9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:35.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.432+0000 7fc76c192740 1 -- 172.21.15.86:0/1948748290 shutdown_connections 2024-04-21T08:41:35.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.432+0000 7fc76c192740 1 -- 172.21.15.86:0/1948748290 wait complete. 2024-04-21T08:41:35.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.433+0000 7fc76c192740 1 -- start start 2024-04-21T08:41:35.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.433+0000 7fc76c192740 1 Processor -- start 2024-04-21T08:41:35.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.433+0000 7fc76c192740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5561cf0ee2b0 0x5561cf0f8f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.433+0000 7fc76c192740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561cf0eebd0 0x5561cf0f9450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.433+0000 7fc76c192740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5561cf0fbe90 0x5561cf0f9990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.433+0000 7fc76c192740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5561cf0fec50 con 0x5561cf0eebd0 2024-04-21T08:41:35.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.433+0000 7fc76c192740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5561cf00c2f0 con 0x5561cf0ee2b0 2024-04-21T08:41:35.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.433+0000 7fc76c192740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5561cf0fedd0 con 0x5561cf0fbe90 2024-04-21T08:41:35.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.433+0000 7fc76c192740 10 client.0 _ll_register_callbacks cb 0x5561cf019580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:35.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.433+0000 7fc7627fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561cf0eebd0 0x5561cf0f9450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.433+0000 7fc7627fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561cf0eebd0 0x5561cf0f9450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49024/0 (socket says 172.21.15.86:49024) 2024-04-21T08:41:35.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.433+0000 7fc7627fc640 1 -- 172.21.15.86:0/591962641 learned_addr learned my addr 172.21.15.86:0/591962641 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:35.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.433+0000 7fc762ffd640 1 --2- 172.21.15.86:0/591962641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5561cf0ee2b0 0x5561cf0f8f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.433+0000 7fc7637fe640 1 --2- 172.21.15.86:0/591962641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5561cf0fbe90 0x5561cf0f9990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.433+0000 7fc7627fc640 1 -- 172.21.15.86:0/591962641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5561cf0fbe90 msgr2=0x5561cf0f9990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:35.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.433+0000 7fc7627fc640 1 --2- 172.21.15.86:0/591962641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5561cf0fbe90 0x5561cf0f9990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.433+0000 7fc7627fc640 1 -- 172.21.15.86:0/591962641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5561cf0ee2b0 msgr2=0x5561cf0f8f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:35.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.433+0000 7fc7627fc640 1 --2- 172.21.15.86:0/591962641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5561cf0ee2b0 0x5561cf0f8f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.433+0000 7fc7627fc640 1 -- 172.21.15.86:0/591962641 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561ced653f0 con 0x5561cf0eebd0 2024-04-21T08:41:35.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.434+0000 7fc7627fc640 1 --2- 172.21.15.86:0/591962641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561cf0eebd0 0x5561cf0f9450 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fc758035000 tx=0x7fc75802bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21f04bc4c63c1a69 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:35.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.434+0000 7fc746ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:35.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.434+0000 7fc746ff5640 1 -- 172.21.15.86:0/591962641 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc758053060 con 0x5561cf0eebd0 2024-04-21T08:41:35.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.434+0000 7fc746ff5640 1 -- 172.21.15.86:0/591962641 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fc758004ad0 con 0x5561cf0eebd0 2024-04-21T08:41:35.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.434+0000 7fc76c192740 1 -- 172.21.15.86:0/591962641 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc75c015070 con 0x5561cf0eebd0 2024-04-21T08:41:35.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.434+0000 7fc746ff5640 1 -- 172.21.15.86:0/591962641 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc75805a050 con 0x5561cf0eebd0 2024-04-21T08:41:35.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.434+0000 7fc76c192740 10 client.17070 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:35.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.434+0000 7fc76c192740 1 -- 172.21.15.86:0/591962641 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5561ced250f0 con 0x5561cf0eebd0 2024-04-21T08:41:35.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.434+0000 7fc746ff5640 1 -- 172.21.15.86:0/591962641 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc75c015070 con 0x5561cf0eebd0 2024-04-21T08:41:35.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.434+0000 7fc746ff5640 1 -- 172.21.15.86:0/591962641 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc758040040 con 0x5561cf0eebd0 2024-04-21T08:41:35.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.434+0000 7fc76c192740 10 client.17070 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:35.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.434+0000 7fc76c192740 10 client.17070 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:35.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.434+0000 7fc76c192740 20 client.17070 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:35.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.434+0000 7fc76c192740 1 --2- 172.21.15.86:0/591962641 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5561cf10bf30 0x5561cf10e350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.434+0000 7fc76c192740 4 client.17070 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:35.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.434+0000 7fc76c192740 1 -- 172.21.15.86:0/591962641 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5561cf060c00 con 0x5561cf10bf30 2024-04-21T08:41:35.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.435+0000 7fc762ffd640 1 --2- 172.21.15.86:0/591962641 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5561cf10bf30 0x5561cf10e350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.435+0000 7fc762ffd640 1 --2- 172.21.15.86:0/591962641 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5561cf10bf30 0x5561cf10e350 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d3e6fde8750814 server_cookie=9124d18f4be36268 in_seq=0 out_seq=0 2024-04-21T08:41:35.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.435+0000 7fc746ff5640 10 client.17070 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:35.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.436+0000 7fc746ff5640 1 -- 172.21.15.86:0/591962641 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5561cf060c00 con 0x5561cf10bf30 2024-04-21T08:41:35.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.436+0000 7fc746ff5640 10 client.17070 handle_command_reply: tid=0 2024-04-21T08:41:35.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.436+0000 7fc76c192740 1 client.17070 shutdown 2024-04-21T08:41:35.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.436+0000 7fc76c192740 1 -- 172.21.15.86:0/591962641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561cf0eebd0 msgr2=0x5561cf0f9450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:35.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.436+0000 7fc76c192740 1 --2- 172.21.15.86:0/591962641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561cf0eebd0 0x5561cf0f9450 secure :-1 s=READY pgs=968 cs=0 l=1 rev1=1 crypto rx=0x7fc758035000 tx=0x7fc75802bd60 comp rx=0 tx=0).stop 2024-04-21T08:41:35.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.436+0000 7fc76c192740 1 -- 172.21.15.86:0/591962641 shutdown_connections 2024-04-21T08:41:35.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.436+0000 7fc76c192740 1 --2- 172.21.15.86:0/591962641 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5561cf10bf30 0x5561cf10e350 crc :-1 s=READY pgs=119 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.436+0000 7fc76c192740 1 --2- 172.21.15.86:0/591962641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5561cf0fbe90 0x5561cf0f9990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.436+0000 7fc76c192740 1 --2- 172.21.15.86:0/591962641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561cf0eebd0 0x5561cf0f9450 unknown :-1 s=CLOSED pgs=968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.436+0000 7fc76c192740 1 --2- 172.21.15.86:0/591962641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5561cf0ee2b0 0x5561cf0f8f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.436+0000 7fc746ff5640 0 client.17070 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:35.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.436+0000 7fc76c192740 1 -- 172.21.15.86:0/591962641 >> 172.21.15.86:0/591962641 conn(0x5561cf0e8cb0 msgr2=0x5561cf0eaa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:35.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.436+0000 7fc76c192740 1 -- 172.21.15.86:0/591962641 shutdown_connections 2024-04-21T08:41:35.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.436+0000 7fc76c192740 1 -- 172.21.15.86:0/591962641 wait complete. 2024-04-21T08:41:35.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.436+0000 7fc76c192740 20 client.17070 trim_cache size 0 max 16384 2024-04-21T08:41:35.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.453+0000 7fc76c192740 1 Processor -- start 2024-04-21T08:41:35.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.453+0000 7fc76c192740 1 -- start start 2024-04-21T08:41:35.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.453+0000 7fc76c192740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5561cf0ee2b0 0x5561cf0fb640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.453+0000 7fc76c192740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5561cf0eebd0 0x5561cf107b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.453+0000 7fc76c192740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561cf0fbe90 0x5561cf0ea780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.453+0000 7fc76c192740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc7641ad060 con 0x5561cf0fbe90 2024-04-21T08:41:35.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.453+0000 7fc76c192740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc7641aeba0 con 0x5561cf0ee2b0 2024-04-21T08:41:35.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.453+0000 7fc76c192740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc7641ad730 con 0x5561cf0eebd0 2024-04-21T08:41:35.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.453+0000 7fc7637fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561cf0fbe90 0x5561cf0ea780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.453+0000 7fc7637fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561cf0fbe90 0x5561cf0ea780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49028/0 (socket says 172.21.15.86:49028) 2024-04-21T08:41:35.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.453+0000 7fc7637fe640 1 -- 172.21.15.86:0/2748306385 learned_addr learned my addr 172.21.15.86:0/2748306385 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:35.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.453+0000 7fc762ffd640 1 --2- 172.21.15.86:0/2748306385 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5561cf0ee2b0 0x5561cf0fb640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.453+0000 7fc7637fe640 1 -- 172.21.15.86:0/2748306385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5561cf0eebd0 msgr2=0x5561cf107b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:35.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.453+0000 7fc7637fe640 1 --2- 172.21.15.86:0/2748306385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5561cf0eebd0 0x5561cf107b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.453+0000 7fc7637fe640 1 -- 172.21.15.86:0/2748306385 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5561cf0ee2b0 msgr2=0x5561cf0fb640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:35.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.453+0000 7fc7637fe640 1 --2- 172.21.15.86:0/2748306385 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5561cf0ee2b0 0x5561cf0fb640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.453+0000 7fc7637fe640 1 -- 172.21.15.86:0/2748306385 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5561cf0ed670 con 0x5561cf0fbe90 2024-04-21T08:41:35.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.453+0000 7fc7637fe640 1 --2- 172.21.15.86:0/2748306385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561cf0fbe90 0x5561cf0ea780 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fc754099580 tx=0x7fc75400af30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=33a0111a5506396a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:35.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.454+0000 7fc748ff9640 1 -- 172.21.15.86:0/2748306385 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7540b6070 con 0x5561cf0fbe90 2024-04-21T08:41:35.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.454+0000 7fc748ff9640 1 -- 172.21.15.86:0/2748306385 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7540a0f60 con 0x5561cf0fbe90 2024-04-21T08:41:35.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.454+0000 7fc748ff9640 1 -- 172.21.15.86:0/2748306385 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc754099220 con 0x5561cf0fbe90 2024-04-21T08:41:35.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.454+0000 7fc76c192740 1 -- 172.21.15.86:0/2748306385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561cf0fbe90 msgr2=0x5561cf0ea780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:35.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.454+0000 7fc76c192740 1 --2- 172.21.15.86:0/2748306385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561cf0fbe90 0x5561cf0ea780 secure :-1 s=READY pgs=969 cs=0 l=1 rev1=1 crypto rx=0x7fc754099580 tx=0x7fc75400af30 comp rx=0 tx=0).stop 2024-04-21T08:41:35.463 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.454+0000 7fc76c192740 1 -- 172.21.15.86:0/2748306385 shutdown_connections 2024-04-21T08:41:35.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.454+0000 7fc76c192740 1 --2- 172.21.15.86:0/2748306385 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561cf0fbe90 0x5561cf0ea780 unknown :-1 s=CLOSED pgs=969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.454+0000 7fc76c192740 1 --2- 172.21.15.86:0/2748306385 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5561cf0eebd0 0x5561cf107b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.454+0000 7fc76c192740 1 --2- 172.21.15.86:0/2748306385 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5561cf0ee2b0 0x5561cf0fb640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.454+0000 7fc76c192740 1 -- 172.21.15.86:0/2748306385 >> 172.21.15.86:0/2748306385 conn(0x5561cf0e8cb0 msgr2=0x5561cf0ea070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:35.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.454+0000 7fc76c192740 1 -- 172.21.15.86:0/2748306385 shutdown_connections 2024-04-21T08:41:35.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.454+0000 7fc76c192740 1 -- 172.21.15.86:0/2748306385 wait complete. 2024-04-21T08:41:35.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.454+0000 7fc76c192740 1 -- start start 2024-04-21T08:41:35.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.454+0000 7fc76c192740 1 Processor -- start 2024-04-21T08:41:35.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.455+0000 7fc76c192740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5561cf0ee2b0 0x5561cf0f6300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.455+0000 7fc76c192740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5561cf0eebd0 0x5561cf0f6840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.455+0000 7fc76c192740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561cf0fbe90 0x5561cf171490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.455+0000 7fc76c192740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc7641ae4d0 con 0x5561cf0fbe90 2024-04-21T08:41:35.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.455+0000 7fc76c192740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc7641ade00 con 0x5561cf0eebd0 2024-04-21T08:41:35.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.455+0000 7fc76c192740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc7641ab020 con 0x5561cf0ee2b0 2024-04-21T08:41:35.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.455+0000 7fc76c192740 10 client.0 _ll_register_callbacks cb 0x7fc758009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:35.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.455+0000 7fc7637fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561cf0fbe90 0x5561cf171490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.455+0000 7fc7637fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561cf0fbe90 0x5561cf171490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49036/0 (socket says 172.21.15.86:49036) 2024-04-21T08:41:35.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.455+0000 7fc7637fe640 1 -- 172.21.15.86:0/3850082648 learned_addr learned my addr 172.21.15.86:0/3850082648 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:35.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.455+0000 7fc7627fc640 1 --2- 172.21.15.86:0/3850082648 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5561cf0eebd0 0x5561cf0f6840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.455+0000 7fc762ffd640 1 --2- 172.21.15.86:0/3850082648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5561cf0ee2b0 0x5561cf0f6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.455+0000 7fc7637fe640 1 -- 172.21.15.86:0/3850082648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5561cf0ee2b0 msgr2=0x5561cf0f6300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:35.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.455+0000 7fc7637fe640 1 --2- 172.21.15.86:0/3850082648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5561cf0ee2b0 0x5561cf0f6300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.455+0000 7fc7637fe640 1 -- 172.21.15.86:0/3850082648 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5561cf0eebd0 msgr2=0x5561cf0f6840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:35.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.455+0000 7fc7637fe640 1 --2- 172.21.15.86:0/3850082648 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5561cf0eebd0 0x5561cf0f6840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.455+0000 7fc7637fe640 1 -- 172.21.15.86:0/3850082648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5561cf1719d0 con 0x5561cf0fbe90 2024-04-21T08:41:35.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.455+0000 7fc7627fc640 1 --2- 172.21.15.86:0/3850082648 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5561cf0eebd0 0x5561cf0f6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:41:35.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.455+0000 7fc7637fe640 1 --2- 172.21.15.86:0/3850082648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561cf0fbe90 0x5561cf171490 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fc75409c8b0 tx=0x7fc7540a5890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57918ff54cbf9de3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:35.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.455+0000 7fc746ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:35.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.455+0000 7fc746ff5640 1 -- 172.21.15.86:0/3850082648 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc7540b6040 con 0x5561cf0fbe90 2024-04-21T08:41:35.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.455+0000 7fc746ff5640 1 -- 172.21.15.86:0/3850082648 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fc7540a5390 con 0x5561cf0fbe90 2024-04-21T08:41:35.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.455+0000 7fc76c192740 1 -- 172.21.15.86:0/3850082648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc7540a0f60 con 0x5561cf0fbe90 2024-04-21T08:41:35.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.455+0000 7fc746ff5640 1 -- 172.21.15.86:0/3850082648 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc75400fd90 con 0x5561cf0fbe90 2024-04-21T08:41:35.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.455+0000 7fc76c192740 10 client.17076 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:35.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.455+0000 7fc76c192740 1 -- 172.21.15.86:0/3850082648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5561cf172ce0 con 0x5561cf0fbe90 2024-04-21T08:41:35.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.456+0000 7fc746ff5640 1 -- 172.21.15.86:0/3850082648 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc7540a0f60 con 0x5561cf0fbe90 2024-04-21T08:41:35.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.456+0000 7fc746ff5640 1 -- 172.21.15.86:0/3850082648 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc754098900 con 0x5561cf0fbe90 2024-04-21T08:41:35.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.456+0000 7fc76c192740 10 client.17076 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:35.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.456+0000 7fc76c192740 10 client.17076 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:35.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.456+0000 7fc76c192740 20 client.17076 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:35.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.456+0000 7fc76c192740 1 --2- 172.21.15.86:0/3850082648 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5561cf1732a0 0x5561cf175690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:35.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.456+0000 7fc76c192740 4 client.17076 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:41:35.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.456+0000 7fc76c192740 1 -- 172.21.15.86:0/3850082648 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5561cf05aa70 con 0x5561cf1732a0 2024-04-21T08:41:35.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.456+0000 7fc762ffd640 1 --2- 172.21.15.86:0/3850082648 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5561cf1732a0 0x5561cf175690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:35.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.456+0000 7fc746ff5640 10 client.17076 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:35.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.456+0000 7fc762ffd640 1 --2- 172.21.15.86:0/3850082648 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5561cf1732a0 0x5561cf175690 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fde8051e1fb3ece6 server_cookie=e2c54d6603a74c3b in_seq=0 out_seq=0 2024-04-21T08:41:35.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.456+0000 7fc746ff5640 1 -- 172.21.15.86:0/3850082648 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5561cf05aa70 con 0x5561cf1732a0 2024-04-21T08:41:35.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.456+0000 7fc746ff5640 10 client.17076 handle_command_reply: tid=0 2024-04-21T08:41:35.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.457+0000 7fc76c192740 1 client.17076 shutdown 2024-04-21T08:41:35.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.457+0000 7fc76c192740 1 -- 172.21.15.86:0/3850082648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561cf0fbe90 msgr2=0x5561cf171490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:35.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.457+0000 7fc76c192740 1 --2- 172.21.15.86:0/3850082648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561cf0fbe90 0x5561cf171490 secure :-1 s=READY pgs=970 cs=0 l=1 rev1=1 crypto rx=0x7fc75409c8b0 tx=0x7fc7540a5890 comp rx=0 tx=0).stop 2024-04-21T08:41:35.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.457+0000 7fc76c192740 1 -- 172.21.15.86:0/3850082648 shutdown_connections 2024-04-21T08:41:35.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.457+0000 7fc76c192740 1 --2- 172.21.15.86:0/3850082648 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5561cf1732a0 0x5561cf175690 crc :-1 s=READY pgs=120 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.457+0000 7fc746ff5640 0 client.17076 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:35.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.457+0000 7fc76c192740 1 --2- 172.21.15.86:0/3850082648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5561cf0fbe90 0x5561cf171490 unknown :-1 s=CLOSED pgs=970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.457+0000 7fc76c192740 1 --2- 172.21.15.86:0/3850082648 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5561cf0eebd0 0x5561cf0f6840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.457+0000 7fc76c192740 1 --2- 172.21.15.86:0/3850082648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5561cf0ee2b0 0x5561cf0f6300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.457+0000 7fc76c192740 1 -- 172.21.15.86:0/3850082648 >> 172.21.15.86:0/3850082648 conn(0x5561cf0e8cb0 msgr2=0x5561cf0f73b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:35.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.457+0000 7fc76c192740 1 -- 172.21.15.86:0/3850082648 shutdown_connections 2024-04-21T08:41:35.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.457+0000 7fc76c192740 1 -- 172.21.15.86:0/3850082648 wait complete. 2024-04-21T08:41:35.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.457+0000 7fc76c192740 20 client.17076 trim_cache size 0 max 16384 2024-04-21T08:41:35.468 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:41:35.468 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:41:35.468 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-21T08:41:35.468 INFO:teuthology.orchestra.run.smithi086.stdout: "1a0ecd81-d904-4483-bd50-fbdd506135c4": { 2024-04-21T08:41:35.469 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-21T08:41:35.469 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "1a0ecd81-d904-4483-bd50-fbdd506135c4", 2024-04-21T08:41:35.469 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-21T08:41:35.469 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:41:35.469 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:41:35.469 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:41:35.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.457+0000 7fc7694c1640 1 -- 172.21.15.86:0/1263342642 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc74c0a3d20 msgr2=0x7fc74c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:35.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.457+0000 7fc7694c1640 1 --2- 172.21.15.86:0/1263342642 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc74c0a3d20 0x7fc74c0a61e0 secure :-1 s=READY pgs=145 cs=0 l=1 rev1=1 crypto rx=0x7fc758036040 tx=0x7fc758004390 comp rx=0 tx=0).stop 2024-04-21T08:41:35.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.457+0000 7fc7694c1640 1 -- 172.21.15.86:0/1263342642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc76410c500 msgr2=0x7fc76419cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:35.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.457+0000 7fc7694c1640 1 --2- 172.21.15.86:0/1263342642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc76410c500 0x7fc76419cf70 secure :-1 s=READY pgs=966 cs=0 l=1 rev1=1 crypto rx=0x7fc75400cce0 tx=0x7fc75400a620 comp rx=0 tx=0).stop 2024-04-21T08:41:35.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.457+0000 7fc7694c1640 1 -- 172.21.15.86:0/1263342642 shutdown_connections 2024-04-21T08:41:35.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.457+0000 7fc7694c1640 1 --2- 172.21.15.86:0/1263342642 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc74c0a3d20 0x7fc74c0a61e0 unknown :-1 s=CLOSED pgs=145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.457+0000 7fc7694c1640 1 --2- 172.21.15.86:0/1263342642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc76410c500 0x7fc76419cf70 unknown :-1 s=CLOSED pgs=966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.457+0000 7fc7694c1640 1 --2- 172.21.15.86:0/1263342642 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc764103ae0 0x7fc764198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.457+0000 7fc7694c1640 1 --2- 172.21.15.86:0/1263342642 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc7641031c0 0x7fc7641986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:35.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.457+0000 7fc7694c1640 1 -- 172.21.15.86:0/1263342642 >> 172.21.15.86:0/1263342642 conn(0x7fc764078000 msgr2=0x7fc7640fdf00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:35.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.457+0000 7fc7694c1640 1 -- 172.21.15.86:0/1263342642 shutdown_connections 2024-04-21T08:41:35.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:35.457+0000 7fc7694c1640 1 -- 172.21.15.86:0/1263342642 wait complete. 2024-04-21T08:41:35.471 INFO:tasks.cephfs.filesystem:scrub status for tag:1a0ecd81-d904-4483-bd50-fbdd506135c4 - {'path': '/', 'tag': '1a0ecd81-d904-4483-bd50-fbdd506135c4', 'options': 'recursive,force'} 2024-04-21T08:41:36.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:36 smithi086 ceph-mon[27939]: pgmap v87: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-04-21T08:41:36.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:36 smithi086 ceph-mon[27939]: scrub queued for path: / 2024-04-21T08:41:36.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:36 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:41:36.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:36 smithi086 ceph-mon[27939]: scrub summary: active paths [/] 2024-04-21T08:41:36.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:36 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:41:36.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:36 smithi088 ceph-mon[31259]: pgmap v87: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-04-21T08:41:36.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:36 smithi088 ceph-mon[31259]: scrub queued for path: / 2024-04-21T08:41:36.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:36 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:41:36.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:36 smithi088 ceph-mon[31259]: scrub summary: active paths [/] 2024-04-21T08:41:36.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:36 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:41:36.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:36 smithi179 ceph-mon[31829]: pgmap v87: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-04-21T08:41:36.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:36 smithi179 ceph-mon[31829]: scrub queued for path: / 2024-04-21T08:41:36.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:36 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:41:36.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:36 smithi179 ceph-mon[31829]: scrub summary: active paths [/] 2024-04-21T08:41:36.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:36 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:41:38.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:38 smithi086 ceph-mon[27939]: pgmap v88: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-04-21T08:41:38.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:38 smithi088 ceph-mon[31259]: pgmap v88: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-04-21T08:41:38.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:38 smithi179 ceph-mon[31829]: pgmap v88: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.1 KiB/s rd, 5.7 KiB/s wr, 11 op/s 2024-04-21T08:41:40.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:40 smithi086 ceph-mon[27939]: pgmap v89: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.7 KiB/s wr, 16 op/s 2024-04-21T08:41:40.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:40 smithi086 ceph-mon[27939]: scrub summary: idle 2024-04-21T08:41:40.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:40 smithi088 ceph-mon[31259]: pgmap v89: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.7 KiB/s wr, 16 op/s 2024-04-21T08:41:40.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:40 smithi088 ceph-mon[31259]: scrub summary: idle 2024-04-21T08:41:40.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:40 smithi179 ceph-mon[31829]: pgmap v89: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 12 KiB/s rd, 5.7 KiB/s wr, 16 op/s 2024-04-21T08:41:40.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:40 smithi179 ceph-mon[31829]: scrub summary: idle 2024-04-21T08:41:42.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:42 smithi086 ceph-mon[27939]: pgmap v90: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-04-21T08:41:42.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:42 smithi088 ceph-mon[31259]: pgmap v90: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-04-21T08:41:42.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:42 smithi179 ceph-mon[31829]: pgmap v90: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 6.8 KiB/s rd, 2.9 KiB/s wr, 12 op/s 2024-04-21T08:41:44.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:44 smithi086 ceph-mon[27939]: pgmap v91: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 2.9 KiB/s wr, 16 op/s 2024-04-21T08:41:44.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:44 smithi088 ceph-mon[31259]: pgmap v91: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 2.9 KiB/s wr, 16 op/s 2024-04-21T08:41:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.734+0000 7faa07025640 1 -- 172.21.15.86:0/121798006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa001010f0 msgr2=0x7faa001014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.734+0000 7faa07025640 1 --2- 172.21.15.86:0/121798006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa001010f0 0x7faa001014d0 secure :-1 s=READY pgs=971 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4009900 tx=0x7fa9f402bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.734+0000 7faa07025640 1 -- 172.21.15.86:0/121798006 shutdown_connections 2024-04-21T08:41:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.734+0000 7faa07025640 1 --2- 172.21.15.86:0/121798006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa00106170 0x7faa0010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:44.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.734+0000 7faa07025640 1 --2- 172.21.15.86:0/121798006 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa00101aa0 0x7faa00105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.734+0000 7faa07025640 1 --2- 172.21.15.86:0/121798006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa001010f0 0x7faa001014d0 unknown :-1 s=CLOSED pgs=971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.734+0000 7faa07025640 1 -- 172.21.15.86:0/121798006 >> 172.21.15.86:0/121798006 conn(0x7faa00077fd0 msgr2=0x7faa000783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.734+0000 7faa07025640 1 -- 172.21.15.86:0/121798006 shutdown_connections 2024-04-21T08:41:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.734+0000 7faa07025640 1 -- 172.21.15.86:0/121798006 wait complete. 2024-04-21T08:41:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.735+0000 7faa07025640 1 Processor -- start 2024-04-21T08:41:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.735+0000 7faa07025640 1 -- start start 2024-04-21T08:41:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.735+0000 7faa07025640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa001010f0 0x7faa0019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.735+0000 7faa07025640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa00106170 0x7faa0019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.735+0000 7faa07025640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa00198590 0x7faa00198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.735+0000 7faa07025640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa00112230 con 0x7faa00106170 2024-04-21T08:41:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.735+0000 7faa07025640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faa001120b0 con 0x7faa00198590 2024-04-21T08:41:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.735+0000 7faa07025640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faa001123b0 con 0x7faa001010f0 2024-04-21T08:41:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.735+0000 7fa9fffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa00106170 0x7faa0019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:44.737 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.735+0000 7fa9fffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa00106170 0x7faa0019e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39248/0 (socket says 172.21.15.86:39248) 2024-04-21T08:41:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.735+0000 7fa9fffff640 1 -- 172.21.15.86:0/1546595157 learned_addr learned my addr 172.21.15.86:0/1546595157 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.736+0000 7faa055c2640 1 --2- 172.21.15.86:0/1546595157 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa00198590 0x7faa00198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.736+0000 7fa9fffff640 1 -- 172.21.15.86:0/1546595157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa001010f0 msgr2=0x7faa0019e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.736+0000 7fa9fffff640 1 --2- 172.21.15.86:0/1546595157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa001010f0 0x7faa0019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.736+0000 7fa9fffff640 1 -- 172.21.15.86:0/1546595157 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa00198590 msgr2=0x7faa00198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.736+0000 7fa9fffff640 1 --2- 172.21.15.86:0/1546595157 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa00198590 0x7faa00198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.736+0000 7fa9fffff640 1 -- 172.21.15.86:0/1546595157 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa001990b0 con 0x7faa00106170 2024-04-21T08:41:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.736+0000 7fa9fffff640 1 --2- 172.21.15.86:0/1546595157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa00106170 0x7faa0019e920 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec00e9c0 tx=0x7fa9ec00ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.736+0000 7fa9fdffb640 1 -- 172.21.15.86:0/1546595157 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec00cd80 con 0x7faa00106170 2024-04-21T08:41:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.736+0000 7fa9fdffb640 1 -- 172.21.15.86:0/1546595157 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9ec0054d0 con 0x7faa00106170 2024-04-21T08:41:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.736+0000 7faa07025640 1 -- 172.21.15.86:0/1546595157 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa001993a0 con 0x7faa00106170 2024-04-21T08:41:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.736+0000 7fa9fdffb640 1 -- 172.21.15.86:0/1546595157 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9ec010430 con 0x7faa00106170 2024-04-21T08:41:44.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.736+0000 7faa07025640 1 -- 172.21.15.86:0/1546595157 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa001a55f0 con 0x7faa00106170 2024-04-21T08:41:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.739+0000 7fa9fdffb640 1 -- 172.21.15.86:0/1546595157 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa9ec004fd0 con 0x7faa00106170 2024-04-21T08:41:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.739+0000 7faa09c12740 1 Processor -- start 2024-04-21T08:41:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.739+0000 7faa09c12740 1 -- start start 2024-04-21T08:41:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.739+0000 7fa9fdffb640 1 --2- 172.21.15.86:0/1546595157 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa9f00a3dc0 0x7fa9f00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.740+0000 7faa09c12740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed8f6738d0 0x55ed8f672620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.740+0000 7faa09c12740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ed8f672b60 0x55ed8f672fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:44.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.740+0000 7faa09c12740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8f67c0e0 0x55ed8f67e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.740+0000 7faa09c12740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ed8f5d78d0 con 0x55ed8f67c0e0 2024-04-21T08:41:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.740+0000 7faa09c12740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55ed8f574e80 con 0x55ed8f672b60 2024-04-21T08:41:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.740+0000 7faa09c12740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ed8f5d3890 con 0x55ed8f6738d0 2024-04-21T08:41:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.740+0000 7faa055c2640 1 --2- 172.21.15.86:0/1546595157 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa9f00a3dc0 0x7fa9f00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.740+0000 7fa9fdffb640 1 -- 172.21.15.86:0/1546595157 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fa9ec0d1080 con 0x7faa00106170 2024-04-21T08:41:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.740+0000 7faa055c2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8f67c0e0 0x55ed8f67e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.740+0000 7faa055c2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8f67c0e0 0x55ed8f67e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39264/0 (socket says 172.21.15.86:39264) 2024-04-21T08:41:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.740+0000 7faa055c2640 1 -- 172.21.15.86:0/1067836722 learned_addr learned my addr 172.21.15.86:0/1067836722 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.740+0000 7fa9fffff640 1 --2- 172.21.15.86:0/1067836722 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ed8f672b60 0x55ed8f672fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:44.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.740+0000 7faa055c2640 1 --2- 172.21.15.86:0/1546595157 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa9f00a3dc0 0x7fa9f00a6280 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800c830 tx=0x7fa9f800cc30 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.740+0000 7faa055c2640 1 -- 172.21.15.86:0/1067836722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed8f6738d0 msgr2=0x55ed8f672620 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:41:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.740+0000 7faa055c2640 1 --2- 172.21.15.86:0/1067836722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed8f6738d0 0x55ed8f672620 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.740+0000 7faa055c2640 1 -- 172.21.15.86:0/1067836722 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ed8f672b60 msgr2=0x55ed8f672fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.740+0000 7faa055c2640 1 --2- 172.21.15.86:0/1067836722 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ed8f672b60 0x55ed8f672fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.740+0000 7faa055c2640 1 -- 172.21.15.86:0/1067836722 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed8f596320 con 0x55ed8f67c0e0 2024-04-21T08:41:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.740+0000 7faa055c2640 1 --2- 172.21.15.86:0/1067836722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8f67c0e0 0x55ed8f67e4d0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009d60 tx=0x7fa9f801d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=28a8756f74caadbc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.741+0000 7fa9eaffd640 1 -- 172.21.15.86:0/1067836722 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f801cd80 con 0x55ed8f67c0e0 2024-04-21T08:41:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.741+0000 7fa9eaffd640 1 -- 172.21.15.86:0/1067836722 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9f8020070 con 0x55ed8f67c0e0 2024-04-21T08:41:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.741+0000 7fa9eaffd640 1 -- 172.21.15.86:0/1067836722 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8023610 con 0x55ed8f67c0e0 2024-04-21T08:41:44.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.741+0000 7faa09c12740 1 -- 172.21.15.86:0/1067836722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8f67c0e0 msgr2=0x55ed8f67e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.741+0000 7faa09c12740 1 --2- 172.21.15.86:0/1067836722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8f67c0e0 0x55ed8f67e4d0 secure :-1 s=READY pgs=973 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8009d60 tx=0x7fa9f801d000 comp rx=0 tx=0).stop 2024-04-21T08:41:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.741+0000 7faa09c12740 1 -- 172.21.15.86:0/1067836722 shutdown_connections 2024-04-21T08:41:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.741+0000 7faa09c12740 1 --2- 172.21.15.86:0/1067836722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8f67c0e0 0x55ed8f67e4d0 unknown :-1 s=CLOSED pgs=973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.741+0000 7faa09c12740 1 --2- 172.21.15.86:0/1067836722 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ed8f672b60 0x55ed8f672fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.741+0000 7faa09c12740 1 --2- 172.21.15.86:0/1067836722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed8f6738d0 0x55ed8f672620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.741+0000 7faa09c12740 1 -- 172.21.15.86:0/1067836722 >> 172.21.15.86:0/1067836722 conn(0x55ed8f639c90 msgr2=0x55ed8f66d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.741+0000 7faa09c12740 1 -- 172.21.15.86:0/1067836722 shutdown_connections 2024-04-21T08:41:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.741+0000 7faa09c12740 1 -- 172.21.15.86:0/1067836722 wait complete. 2024-04-21T08:41:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.742+0000 7faa09c12740 1 -- start start 2024-04-21T08:41:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.742+0000 7faa09c12740 1 Processor -- start 2024-04-21T08:41:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.742+0000 7faa09c12740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed8f672b60 0x55ed8f67bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.742+0000 7faa09c12740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ed8f6738d0 0x55ed8f678e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:44.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.742+0000 7faa09c12740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8f67c0e0 0x55ed8f679450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.742+0000 7faa09c12740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ed8f67edb0 con 0x55ed8f67c0e0 2024-04-21T08:41:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.742+0000 7faa09c12740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55ed8f55feb0 con 0x55ed8f6738d0 2024-04-21T08:41:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.742+0000 7faa09c12740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ed8f67ef30 con 0x55ed8f672b60 2024-04-21T08:41:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.742+0000 7faa09c12740 10 client.0 _ll_register_callbacks cb 0x55ed8f58cb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.742+0000 7faa055c2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8f67c0e0 0x55ed8f679450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.742+0000 7faa055c2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8f67c0e0 0x55ed8f679450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39270/0 (socket says 172.21.15.86:39270) 2024-04-21T08:41:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.742+0000 7faa055c2640 1 -- 172.21.15.86:0/239835612 learned_addr learned my addr 172.21.15.86:0/239835612 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.742+0000 7fa9fffff640 1 --2- 172.21.15.86:0/239835612 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ed8f6738d0 0x55ed8f678e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.743+0000 7faa04dc1640 1 --2- 172.21.15.86:0/239835612 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed8f672b60 0x55ed8f67bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.743+0000 7faa055c2640 1 -- 172.21.15.86:0/239835612 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed8f672b60 msgr2=0x55ed8f67bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.743+0000 7faa055c2640 1 --2- 172.21.15.86:0/239835612 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed8f672b60 0x55ed8f67bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.743+0000 7faa055c2640 1 -- 172.21.15.86:0/239835612 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ed8f6738d0 msgr2=0x55ed8f678e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:44.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.743+0000 7faa055c2640 1 --2- 172.21.15.86:0/239835612 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ed8f6738d0 0x55ed8f678e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.743+0000 7faa055c2640 1 -- 172.21.15.86:0/239835612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed8f2ea210 con 0x55ed8f67c0e0 2024-04-21T08:41:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.743+0000 7faa055c2640 1 --2- 172.21.15.86:0/239835612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8f67c0e0 0x55ed8f679450 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8007680 tx=0x7fa9f8004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2793f82228fee2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.743+0000 7fa9dffff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.743+0000 7fa9dffff640 1 -- 172.21.15.86:0/239835612 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8030020 con 0x55ed8f67c0e0 2024-04-21T08:41:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.743+0000 7fa9dffff640 1 -- 172.21.15.86:0/239835612 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fa9f801a990 con 0x55ed8f67c0e0 2024-04-21T08:41:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.743+0000 7faa09c12740 1 -- 172.21.15.86:0/239835612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9f8020070 con 0x55ed8f67c0e0 2024-04-21T08:41:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.743+0000 7fa9dffff640 1 -- 172.21.15.86:0/239835612 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa9f803b050 con 0x55ed8f67c0e0 2024-04-21T08:41:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.743+0000 7faa09c12740 10 client.17088 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.743+0000 7faa09c12740 1 -- 172.21.15.86:0/239835612 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ed8f3b6620 con 0x55ed8f67c0e0 2024-04-21T08:41:44.751 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.743+0000 7fa9dffff640 1 -- 172.21.15.86:0/239835612 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9f8020070 con 0x55ed8f67c0e0 2024-04-21T08:41:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.744+0000 7fa9dffff640 1 -- 172.21.15.86:0/239835612 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa9f802cba0 con 0x55ed8f67c0e0 2024-04-21T08:41:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.744+0000 7faa09c12740 10 client.17088 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.744+0000 7faa09c12740 10 client.17088 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.744+0000 7faa09c12740 20 client.17088 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.744+0000 7faa09c12740 1 --2- 172.21.15.86:0/239835612 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ed8f68c050 0x55ed8f68e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.744+0000 7faa09c12740 4 client.17088 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.744+0000 7faa09c12740 1 -- 172.21.15.86:0/239835612 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ed8f5eb880 con 0x55ed8f68c050 2024-04-21T08:41:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.744+0000 7faa04dc1640 1 --2- 172.21.15.86:0/239835612 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ed8f68c050 0x55ed8f68e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.744+0000 7faa04dc1640 1 --2- 172.21.15.86:0/239835612 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ed8f68c050 0x55ed8f68e440 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a1b587bca2fc6bf server_cookie=da2f8c14e670ad5d in_seq=0 out_seq=0 2024-04-21T08:41:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.744+0000 7fa9dffff640 10 client.17088 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.745+0000 7fa9dffff640 1 -- 172.21.15.86:0/239835612 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ed8f5eb880 con 0x55ed8f68c050 2024-04-21T08:41:44.752 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.745+0000 7fa9dffff640 10 client.17088 handle_command_reply: tid=0 2024-04-21T08:41:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.745+0000 7faa09c12740 1 client.17088 shutdown 2024-04-21T08:41:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.745+0000 7faa09c12740 1 -- 172.21.15.86:0/239835612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8f67c0e0 msgr2=0x55ed8f679450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.745+0000 7faa09c12740 1 --2- 172.21.15.86:0/239835612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8f67c0e0 0x55ed8f679450 secure :-1 s=READY pgs=974 cs=0 l=1 rev1=1 crypto rx=0x7fa9f8007680 tx=0x7fa9f8004b50 comp rx=0 tx=0).stop 2024-04-21T08:41:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.745+0000 7faa09c12740 1 -- 172.21.15.86:0/239835612 shutdown_connections 2024-04-21T08:41:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.745+0000 7faa09c12740 1 --2- 172.21.15.86:0/239835612 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ed8f68c050 0x55ed8f68e440 crc :-1 s=READY pgs=121 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.745+0000 7faa09c12740 1 --2- 172.21.15.86:0/239835612 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8f67c0e0 0x55ed8f679450 unknown :-1 s=CLOSED pgs=974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.745+0000 7faa09c12740 1 --2- 172.21.15.86:0/239835612 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ed8f6738d0 0x55ed8f678e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.745+0000 7faa09c12740 1 --2- 172.21.15.86:0/239835612 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed8f672b60 0x55ed8f67bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.745+0000 7fa9dffff640 0 client.17088 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.745+0000 7faa09c12740 1 -- 172.21.15.86:0/239835612 >> 172.21.15.86:0/239835612 conn(0x55ed8f639c90 msgr2=0x55ed8f66f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.745+0000 7faa09c12740 1 -- 172.21.15.86:0/239835612 shutdown_connections 2024-04-21T08:41:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.745+0000 7faa09c12740 1 -- 172.21.15.86:0/239835612 wait complete. 2024-04-21T08:41:44.753 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.745+0000 7faa09c12740 20 client.17088 trim_cache size 0 max 16384 2024-04-21T08:41:44.754 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:44 smithi179 ceph-mon[31829]: pgmap v91: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 8.9 KiB/s rd, 2.9 KiB/s wr, 16 op/s 2024-04-21T08:41:44.761 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.760+0000 7faa09c12740 1 Processor -- start 2024-04-21T08:41:44.761 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.760+0000 7faa09c12740 1 -- start start 2024-04-21T08:41:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.760+0000 7faa09c12740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8f672b60 0x55ed8f67b570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.760+0000 7faa09c12740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ed8f6738d0 0x55ed8f67bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.760+0000 7faa09c12740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed8f67c0e0 0x55ed8f693a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.760+0000 7faa09c12740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa001af3a0 con 0x55ed8f672b60 2024-04-21T08:41:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.760+0000 7faa09c12740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faa001b0ee0 con 0x55ed8f6738d0 2024-04-21T08:41:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.760+0000 7faa09c12740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faa001afa70 con 0x55ed8f67c0e0 2024-04-21T08:41:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.761+0000 7faa04dc1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8f672b60 0x55ed8f67b570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.761+0000 7faa04dc1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8f672b60 0x55ed8f67b570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39282/0 (socket says 172.21.15.86:39282) 2024-04-21T08:41:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.761+0000 7faa04dc1640 1 -- 172.21.15.86:0/4092093799 learned_addr learned my addr 172.21.15.86:0/4092093799 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.761+0000 7fa9fffff640 1 --2- 172.21.15.86:0/4092093799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ed8f6738d0 0x55ed8f67bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.761+0000 7faa04dc1640 1 -- 172.21.15.86:0/4092093799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed8f67c0e0 msgr2=0x55ed8f693a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.761+0000 7faa04dc1640 1 --2- 172.21.15.86:0/4092093799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed8f67c0e0 0x55ed8f693a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:44.762 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.761+0000 7faa04dc1640 1 -- 172.21.15.86:0/4092093799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ed8f6738d0 msgr2=0x55ed8f67bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.761+0000 7faa04dc1640 1 --2- 172.21.15.86:0/4092093799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ed8f6738d0 0x55ed8f67bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.761+0000 7faa04dc1640 1 -- 172.21.15.86:0/4092093799 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ed8f679a50 con 0x55ed8f672b60 2024-04-21T08:41:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.761+0000 7faa04dc1640 1 --2- 172.21.15.86:0/4092093799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8f672b60 0x55ed8f67b570 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4036040 tx=0x7fa9f4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa8189ea9307995f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.761+0000 7fa9ea7fc640 1 -- 172.21.15.86:0/4092093799 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4004240 con 0x55ed8f672b60 2024-04-21T08:41:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.761+0000 7fa9ea7fc640 1 -- 172.21.15.86:0/4092093799 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9f4048070 con 0x55ed8f672b60 2024-04-21T08:41:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.761+0000 7fa9ea7fc640 1 -- 172.21.15.86:0/4092093799 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f4005630 con 0x55ed8f672b60 2024-04-21T08:41:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.762+0000 7faa09c12740 1 -- 172.21.15.86:0/4092093799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8f672b60 msgr2=0x55ed8f67b570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.762+0000 7faa09c12740 1 --2- 172.21.15.86:0/4092093799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8f672b60 0x55ed8f67b570 secure :-1 s=READY pgs=975 cs=0 l=1 rev1=1 crypto rx=0x7fa9f4036040 tx=0x7fa9f4000bc0 comp rx=0 tx=0).stop 2024-04-21T08:41:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.762+0000 7faa09c12740 1 -- 172.21.15.86:0/4092093799 shutdown_connections 2024-04-21T08:41:44.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.762+0000 7faa09c12740 1 --2- 172.21.15.86:0/4092093799 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed8f67c0e0 0x55ed8f693a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.762+0000 7faa09c12740 1 --2- 172.21.15.86:0/4092093799 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ed8f6738d0 0x55ed8f67bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.762+0000 7faa09c12740 1 --2- 172.21.15.86:0/4092093799 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8f672b60 0x55ed8f67b570 unknown :-1 s=CLOSED pgs=975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.762+0000 7faa09c12740 1 -- 172.21.15.86:0/4092093799 >> 172.21.15.86:0/4092093799 conn(0x55ed8f639c90 msgr2=0x55ed8f66f530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.762+0000 7faa09c12740 1 -- 172.21.15.86:0/4092093799 shutdown_connections 2024-04-21T08:41:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.762+0000 7faa09c12740 1 -- 172.21.15.86:0/4092093799 wait complete. 2024-04-21T08:41:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.762+0000 7faa09c12740 1 -- start start 2024-04-21T08:41:44.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.762+0000 7faa09c12740 1 Processor -- start 2024-04-21T08:41:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.763+0000 7faa09c12740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed8f672b60 0x55ed8f6841c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.763+0000 7faa09c12740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ed8f6738d0 0x55ed8f684700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.763+0000 7faa09c12740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8f67c0e0 0x55ed8f6755b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.763+0000 7faa09c12740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa001b0810 con 0x55ed8f67c0e0 2024-04-21T08:41:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.763+0000 7faa09c12740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faa001b0140 con 0x55ed8f6738d0 2024-04-21T08:41:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.763+0000 7faa09c12740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faa001ad360 con 0x55ed8f672b60 2024-04-21T08:41:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.763+0000 7faa09c12740 10 client.0 _ll_register_callbacks cb 0x7fa9ec0048b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.763+0000 7faa055c2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8f67c0e0 0x55ed8f6755b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.763+0000 7faa055c2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8f67c0e0 0x55ed8f6755b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39290/0 (socket says 172.21.15.86:39290) 2024-04-21T08:41:44.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.763+0000 7faa055c2640 1 -- 172.21.15.86:0/335985166 learned_addr learned my addr 172.21.15.86:0/335985166 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:44.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.763+0000 7fa9fffff640 1 --2- 172.21.15.86:0/335985166 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ed8f6738d0 0x55ed8f684700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:44.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.763+0000 7faa04dc1640 1 --2- 172.21.15.86:0/335985166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed8f672b60 0x55ed8f6841c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:44.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.763+0000 7faa055c2640 1 -- 172.21.15.86:0/335985166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed8f672b60 msgr2=0x55ed8f6841c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:44.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.763+0000 7faa055c2640 1 --2- 172.21.15.86:0/335985166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed8f672b60 0x55ed8f6841c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:44.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.763+0000 7faa055c2640 1 -- 172.21.15.86:0/335985166 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ed8f6738d0 msgr2=0x55ed8f684700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:44.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.763+0000 7faa055c2640 1 --2- 172.21.15.86:0/335985166 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ed8f6738d0 0x55ed8f684700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:44.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.763+0000 7faa055c2640 1 -- 172.21.15.86:0/335985166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ed8f671e30 con 0x55ed8f67c0e0 2024-04-21T08:41:44.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.763+0000 7faa055c2640 1 --2- 172.21.15.86:0/335985166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8f67c0e0 0x55ed8f6755b0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fa9f802cf70 tx=0x7fa9f8024600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=540f7d47b9976ba3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:44.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.763+0000 7fa9dffff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:44.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.763+0000 7fa9dffff640 1 -- 172.21.15.86:0/335985166 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa9f8044060 con 0x55ed8f67c0e0 2024-04-21T08:41:44.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.763+0000 7fa9dffff640 1 -- 172.21.15.86:0/335985166 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fa9f801a030 con 0x55ed8f67c0e0 2024-04-21T08:41:44.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.763+0000 7faa09c12740 1 -- 172.21.15.86:0/335985166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa9f4048070 con 0x55ed8f67c0e0 2024-04-21T08:41:44.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.763+0000 7fa9dffff640 1 -- 172.21.15.86:0/335985166 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa9f804b050 con 0x55ed8f67c0e0 2024-04-21T08:41:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.763+0000 7faa09c12740 10 client.17094 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.763+0000 7faa09c12740 1 -- 172.21.15.86:0/335985166 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ed8f6f1ca0 con 0x55ed8f67c0e0 2024-04-21T08:41:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.764+0000 7fa9dffff640 1 -- 172.21.15.86:0/335985166 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa9f4048070 con 0x55ed8f67c0e0 2024-04-21T08:41:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.764+0000 7fa9dffff640 1 -- 172.21.15.86:0/335985166 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa9f801a990 con 0x55ed8f67c0e0 2024-04-21T08:41:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.764+0000 7faa09c12740 10 client.17094 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.764+0000 7faa09c12740 10 client.17094 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.764+0000 7faa09c12740 20 client.17094 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.764+0000 7faa09c12740 1 --2- 172.21.15.86:0/335985166 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ed8f6f2300 0x55ed8f6f4720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.764+0000 7faa09c12740 4 client.17094 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:41:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.764+0000 7faa09c12740 1 -- 172.21.15.86:0/335985166 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ed8f5773d0 con 0x55ed8f6f2300 2024-04-21T08:41:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.764+0000 7faa04dc1640 1 --2- 172.21.15.86:0/335985166 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ed8f6f2300 0x55ed8f6f4720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.764+0000 7fa9dffff640 10 client.17094 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.764+0000 7faa04dc1640 1 --2- 172.21.15.86:0/335985166 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ed8f6f2300 0x55ed8f6f4720 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dacd7c09e5e269d1 server_cookie=2fe1f9b97a1cd089 in_seq=0 out_seq=0 2024-04-21T08:41:44.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.770+0000 7fa9dffff640 1 -- 172.21.15.86:0/335985166 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ed8f5773d0 con 0x55ed8f6f2300 2024-04-21T08:41:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.770+0000 7fa9dffff640 10 client.17094 handle_command_reply: tid=0 2024-04-21T08:41:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.770+0000 7faa09c12740 1 client.17094 shutdown 2024-04-21T08:41:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.770+0000 7faa09c12740 1 -- 172.21.15.86:0/335985166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8f67c0e0 msgr2=0x55ed8f6755b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.770+0000 7faa09c12740 1 --2- 172.21.15.86:0/335985166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8f67c0e0 0x55ed8f6755b0 secure :-1 s=READY pgs=976 cs=0 l=1 rev1=1 crypto rx=0x7fa9f802cf70 tx=0x7fa9f8024600 comp rx=0 tx=0).stop 2024-04-21T08:41:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.770+0000 7faa09c12740 1 -- 172.21.15.86:0/335985166 shutdown_connections 2024-04-21T08:41:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.770+0000 7faa09c12740 1 --2- 172.21.15.86:0/335985166 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ed8f6f2300 0x55ed8f6f4720 crc :-1 s=READY pgs=122 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.770+0000 7faa09c12740 1 --2- 172.21.15.86:0/335985166 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ed8f67c0e0 0x55ed8f6755b0 unknown :-1 s=CLOSED pgs=976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.770+0000 7faa09c12740 1 --2- 172.21.15.86:0/335985166 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ed8f6738d0 0x55ed8f684700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.770+0000 7fa9dffff640 0 client.17094 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.770+0000 7faa09c12740 1 --2- 172.21.15.86:0/335985166 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ed8f672b60 0x55ed8f6841c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.770+0000 7faa09c12740 1 -- 172.21.15.86:0/335985166 >> 172.21.15.86:0/335985166 conn(0x55ed8f639c90 msgr2=0x55ed8f66ea10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.770+0000 7faa09c12740 1 -- 172.21.15.86:0/335985166 shutdown_connections 2024-04-21T08:41:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.770+0000 7faa09c12740 1 -- 172.21.15.86:0/335985166 wait complete. 2024-04-21T08:41:44.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.770+0000 7faa09c12740 20 client.17094 trim_cache size 0 max 16384 2024-04-21T08:41:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:41:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:41:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:41:44.774 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:41:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.771+0000 7faa07025640 1 -- 172.21.15.86:0/1546595157 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa9f00a3dc0 msgr2=0x7fa9f00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.771+0000 7faa07025640 1 --2- 172.21.15.86:0/1546595157 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa9f00a3dc0 0x7fa9f00a6280 secure :-1 s=READY pgs=146 cs=0 l=1 rev1=1 crypto rx=0x7fa9f800c830 tx=0x7fa9f800cc30 comp rx=0 tx=0).stop 2024-04-21T08:41:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.771+0000 7faa07025640 1 -- 172.21.15.86:0/1546595157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa00106170 msgr2=0x7faa0019e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.771+0000 7faa07025640 1 --2- 172.21.15.86:0/1546595157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa00106170 0x7faa0019e920 secure :-1 s=READY pgs=972 cs=0 l=1 rev1=1 crypto rx=0x7fa9ec00e9c0 tx=0x7fa9ec00ee80 comp rx=0 tx=0).stop 2024-04-21T08:41:44.775 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.771+0000 7faa07025640 1 -- 172.21.15.86:0/1546595157 shutdown_connections 2024-04-21T08:41:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.771+0000 7faa07025640 1 --2- 172.21.15.86:0/1546595157 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa9f00a3dc0 0x7fa9f00a6280 unknown :-1 s=CLOSED pgs=146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.771+0000 7faa07025640 1 --2- 172.21.15.86:0/1546595157 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa00198590 0x7faa00198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.771+0000 7faa07025640 1 --2- 172.21.15.86:0/1546595157 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa00106170 0x7faa0019e920 unknown :-1 s=CLOSED pgs=972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.771+0000 7faa07025640 1 --2- 172.21.15.86:0/1546595157 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa001010f0 0x7faa0019e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.771+0000 7faa07025640 1 -- 172.21.15.86:0/1546595157 >> 172.21.15.86:0/1546595157 conn(0x7faa00077fd0 msgr2=0x7faa000ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.771+0000 7faa07025640 1 -- 172.21.15.86:0/1546595157 shutdown_connections 2024-04-21T08:41:44.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:44.771+0000 7faa07025640 1 -- 172.21.15.86:0/1546595157 wait complete. 2024-04-21T08:41:46.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:46 smithi086 ceph-mon[27939]: pgmap v92: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-04-21T08:41:46.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:46 smithi088 ceph-mon[31259]: pgmap v92: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-04-21T08:41:46.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:46 smithi179 ceph-mon[31829]: pgmap v92: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-04-21T08:41:46.941 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:41:47.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.053+0000 7fb6bb862640 1 -- 172.21.15.86:0/4038204790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6b41128f0 msgr2=0x7fb6b4112cd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:47.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.053+0000 7fb6bb862640 1 --2- 172.21.15.86:0/4038204790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6b41128f0 0x7fb6b4112cd0 secure :-1 s=READY pgs=977 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0009930 tx=0x7fb6b002bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:47.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.053+0000 7fb6bb862640 1 -- 172.21.15.86:0/4038204790 shutdown_connections 2024-04-21T08:41:47.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.053+0000 7fb6bb862640 1 --2- 172.21.15.86:0/4038204790 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6b410d2a0 0x7fb6b41197b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.053+0000 7fb6bb862640 1 --2- 172.21.15.86:0/4038204790 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6b410c8e0 0x7fb6b410cd60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.053+0000 7fb6bb862640 1 --2- 172.21.15.86:0/4038204790 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6b41128f0 0x7fb6b4112cd0 unknown :-1 s=CLOSED pgs=977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.053+0000 7fb6bb862640 1 -- 172.21.15.86:0/4038204790 >> 172.21.15.86:0/4038204790 conn(0x7fb6b406d0d0 msgr2=0x7fb6b410af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:47.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.053+0000 7fb6bb862640 1 -- 172.21.15.86:0/4038204790 shutdown_connections 2024-04-21T08:41:47.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.053+0000 7fb6bb862640 1 -- 172.21.15.86:0/4038204790 wait complete. 2024-04-21T08:41:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.053+0000 7fb6bb862640 1 Processor -- start 2024-04-21T08:41:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.053+0000 7fb6bb862640 1 -- start start 2024-04-21T08:41:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.053+0000 7fb6bb862640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6b410c8e0 0x7fb6b41a3130 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.054+0000 7fb6bb862640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6b410d2a0 0x7fb6b41a3670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.054+0000 7fb6bb862640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6b41128f0 0x7fb6b41a79f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.054+0000 7fb6bb862640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b411cbb0 con 0x7fb6b41128f0 2024-04-21T08:41:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.054+0000 7fb6bb862640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b411ca30 con 0x7fb6b410c8e0 2024-04-21T08:41:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.054+0000 7fb6bb862640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb6b411cd30 con 0x7fb6b410d2a0 2024-04-21T08:41:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.054+0000 7fb6b9dff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6b41128f0 0x7fb6b41a79f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.054+0000 7fb6b9dff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6b41128f0 0x7fb6b41a79f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39306/0 (socket says 172.21.15.86:39306) 2024-04-21T08:41:47.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.054+0000 7fb6b9dff640 1 -- 172.21.15.86:0/318341346 learned_addr learned my addr 172.21.15.86:0/318341346 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:47.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.054+0000 7fb6b95fe640 1 --2- 172.21.15.86:0/318341346 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6b410c8e0 0x7fb6b41a3130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:47.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.054+0000 7fb6b8dfd640 1 --2- 172.21.15.86:0/318341346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6b410d2a0 0x7fb6b41a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:47.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.054+0000 7fb6b9dff640 1 -- 172.21.15.86:0/318341346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6b410d2a0 msgr2=0x7fb6b41a3670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:47.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.054+0000 7fb6b9dff640 1 --2- 172.21.15.86:0/318341346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6b410d2a0 0x7fb6b41a3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.054+0000 7fb6b9dff640 1 -- 172.21.15.86:0/318341346 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6b410c8e0 msgr2=0x7fb6b41a3130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:47.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.054+0000 7fb6b9dff640 1 --2- 172.21.15.86:0/318341346 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6b410c8e0 0x7fb6b41a3130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.054+0000 7fb6b9dff640 1 -- 172.21.15.86:0/318341346 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb6b41a8120 con 0x7fb6b41128f0 2024-04-21T08:41:47.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.054+0000 7fb6b9dff640 1 --2- 172.21.15.86:0/318341346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6b41128f0 0x7fb6b41a79f0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac00eaa0 tx=0x7fb6ac00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:47.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.054+0000 7fb6aa7fc640 1 -- 172.21.15.86:0/318341346 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac00cdb0 con 0x7fb6b41128f0 2024-04-21T08:41:47.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.054+0000 7fb6bb862640 1 -- 172.21.15.86:0/318341346 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb6b41a8410 con 0x7fb6b41128f0 2024-04-21T08:41:47.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.054+0000 7fb6aa7fc640 1 -- 172.21.15.86:0/318341346 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb6ac005020 con 0x7fb6b41128f0 2024-04-21T08:41:47.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.054+0000 7fb6aa7fc640 1 -- 172.21.15.86:0/318341346 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb6ac0116d0 con 0x7fb6b41128f0 2024-04-21T08:41:47.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.054+0000 7fb6bb862640 1 -- 172.21.15.86:0/318341346 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb6b41afca0 con 0x7fb6b41128f0 2024-04-21T08:41:47.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.055+0000 7fb6bb862640 1 -- 172.21.15.86:0/318341346 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb6b4068de0 con 0x7fb6b41128f0 2024-04-21T08:41:47.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.056+0000 7fb6aa7fc640 1 -- 172.21.15.86:0/318341346 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb6ac0054a0 con 0x7fb6b41128f0 2024-04-21T08:41:47.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.057+0000 7fb6aa7fc640 1 --2- 172.21.15.86:0/318341346 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb6a00a3d20 0x7fb6a00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:47.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.057+0000 7fb6b95fe640 1 --2- 172.21.15.86:0/318341346 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb6a00a3d20 0x7fb6a00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:47.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.057+0000 7fb6aa7fc640 1 -- 172.21.15.86:0/318341346 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fb6ac0d0d50 con 0x7fb6b41128f0 2024-04-21T08:41:47.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.057+0000 7fb6b95fe640 1 --2- 172.21.15.86:0/318341346 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb6a00a3d20 0x7fb6a00a61e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0036040 tx=0x7fb6b0000bc0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:47.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.062+0000 7fb6aa7fc640 1 -- 172.21.15.86:0/318341346 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb6ac016030 con 0x7fb6b41128f0 2024-04-21T08:41:47.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.279+0000 7fb6bb862640 1 -- 172.21.15.86:0/318341346 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb6b4114eb0 con 0x7fb6b41128f0 2024-04-21T08:41:47.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.279+0000 7fb6aa7fc640 1 -- 172.21.15.86:0/318341346 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7fb6ac09d080 con 0x7fb6b41128f0 2024-04-21T08:41:47.279 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:41:47.279 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:38:45.202126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24398,24394,24455,24472]},"id":1}]} 2024-04-21T08:41:47.279 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-21T08:41:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.282+0000 7fb6bb862640 1 -- 172.21.15.86:0/318341346 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb6a00a3d20 msgr2=0x7fb6a00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.282+0000 7fb6bb862640 1 --2- 172.21.15.86:0/318341346 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb6a00a3d20 0x7fb6a00a61e0 secure :-1 s=READY pgs=147 cs=0 l=1 rev1=1 crypto rx=0x7fb6b0036040 tx=0x7fb6b0000bc0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.282+0000 7fb6bb862640 1 -- 172.21.15.86:0/318341346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6b41128f0 msgr2=0x7fb6b41a79f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.282+0000 7fb6bb862640 1 --2- 172.21.15.86:0/318341346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6b41128f0 0x7fb6b41a79f0 secure :-1 s=READY pgs=978 cs=0 l=1 rev1=1 crypto rx=0x7fb6ac00eaa0 tx=0x7fb6ac00ef60 comp rx=0 tx=0).stop 2024-04-21T08:41:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.282+0000 7fb6bb862640 1 -- 172.21.15.86:0/318341346 shutdown_connections 2024-04-21T08:41:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.282+0000 7fb6bb862640 1 --2- 172.21.15.86:0/318341346 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb6a00a3d20 0x7fb6a00a61e0 unknown :-1 s=CLOSED pgs=147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.282+0000 7fb6bb862640 1 --2- 172.21.15.86:0/318341346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb6b41128f0 0x7fb6b41a79f0 unknown :-1 s=CLOSED pgs=978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.282+0000 7fb6bb862640 1 --2- 172.21.15.86:0/318341346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb6b410d2a0 0x7fb6b41a3670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.282+0000 7fb6bb862640 1 --2- 172.21.15.86:0/318341346 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb6b410c8e0 0x7fb6b41a3130 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.282+0000 7fb6bb862640 1 -- 172.21.15.86:0/318341346 >> 172.21.15.86:0/318341346 conn(0x7fb6b406d0d0 msgr2=0x7fb6b410f9b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:47.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.282+0000 7fb6bb862640 1 -- 172.21.15.86:0/318341346 shutdown_connections 2024-04-21T08:41:47.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.282+0000 7fb6bb862640 1 -- 172.21.15.86:0/318341346 wait complete. 2024-04-21T08:41:47.292 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id ea3fd78e --release --await' 2024-04-21T08:41:47.292 DEBUG:teuthology.orchestra.run.smithi086:> timeout 45 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24482 quiesce db --set-id ea3fd78e --release --await 2024-04-21T08:41:47.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.397+0000 7fb189598640 1 -- 172.21.15.86:0/308624636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1841031d0 msgr2=0x7fb1841035b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:47.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.397+0000 7fb189598640 1 --2- 172.21.15.86:0/308624636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1841031d0 0x7fb1841035b0 secure :-1 s=READY pgs=979 cs=0 l=1 rev1=1 crypto rx=0x7fb174009ca0 tx=0x7fb17402c230 comp rx=0 tx=0).stop 2024-04-21T08:41:47.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.397+0000 7fb189598640 1 -- 172.21.15.86:0/308624636 shutdown_connections 2024-04-21T08:41:47.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.397+0000 7fb189598640 1 --2- 172.21.15.86:0/308624636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb18410c510 0x7fb18410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.397+0000 7fb189598640 1 --2- 172.21.15.86:0/308624636 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb184103af0 0x7fb18410bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.397+0000 7fb189598640 1 --2- 172.21.15.86:0/308624636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb1841031d0 0x7fb1841035b0 unknown :-1 s=CLOSED pgs=979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.397+0000 7fb189598640 1 -- 172.21.15.86:0/308624636 >> 172.21.15.86:0/308624636 conn(0x7fb184077f50 msgr2=0x7fb184078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:47.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.397+0000 7fb189598640 1 -- 172.21.15.86:0/308624636 shutdown_connections 2024-04-21T08:41:47.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.397+0000 7fb189598640 1 -- 172.21.15.86:0/308624636 wait complete. 2024-04-21T08:41:47.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.398+0000 7fb189598640 1 Processor -- start 2024-04-21T08:41:47.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.398+0000 7fb189598640 1 -- start start 2024-04-21T08:41:47.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.398+0000 7fb189598640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1841031d0 0x7fb18419e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:47.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.398+0000 7fb189598640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb184103af0 0x7fb18419e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:47.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.398+0000 7fb189598640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb18410c510 0x7fb1841985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:47.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.398+0000 7fb189598640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb184112200 con 0x7fb18410c510 2024-04-21T08:41:47.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.398+0000 7fb189598640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb184112080 con 0x7fb184103af0 2024-04-21T08:41:47.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.398+0000 7fb189598640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb184112380 con 0x7fb1841031d0 2024-04-21T08:41:47.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.398+0000 7fb1837fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb18410c510 0x7fb1841985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:47.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.399+0000 7fb1837fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb18410c510 0x7fb1841985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39318/0 (socket says 172.21.15.86:39318) 2024-04-21T08:41:47.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.399+0000 7fb1837fe640 1 -- 172.21.15.86:0/4270213076 learned_addr learned my addr 172.21.15.86:0/4270213076 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:47.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.399+0000 7fb1827fc640 1 --2- 172.21.15.86:0/4270213076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb184103af0 0x7fb18419e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:47.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.399+0000 7fb1837fe640 1 -- 172.21.15.86:0/4270213076 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1841031d0 msgr2=0x7fb18419e430 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T08:41:47.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.399+0000 7fb1837fe640 1 --2- 172.21.15.86:0/4270213076 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1841031d0 0x7fb18419e430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.399+0000 7fb1837fe640 1 -- 172.21.15.86:0/4270213076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb184103af0 msgr2=0x7fb18419e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:47.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.399+0000 7fb1837fe640 1 --2- 172.21.15.86:0/4270213076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb184103af0 0x7fb18419e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.399+0000 7fb1837fe640 1 -- 172.21.15.86:0/4270213076 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb184198d60 con 0x7fb18410c510 2024-04-21T08:41:47.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.399+0000 7fb1837fe640 1 --2- 172.21.15.86:0/4270213076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb18410c510 0x7fb1841985e0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fb17c009fd0 tx=0x7fb17c00b8e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:47.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.399+0000 7fb16bfff640 1 -- 172.21.15.86:0/4270213076 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17c011830 con 0x7fb18410c510 2024-04-21T08:41:47.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.399+0000 7fb16bfff640 1 -- 172.21.15.86:0/4270213076 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb17c016070 con 0x7fb18410c510 2024-04-21T08:41:47.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.399+0000 7fb189598640 1 -- 172.21.15.86:0/4270213076 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb184199050 con 0x7fb18410c510 2024-04-21T08:41:47.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.399+0000 7fb16bfff640 1 -- 172.21.15.86:0/4270213076 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17c010de0 con 0x7fb18410c510 2024-04-21T08:41:47.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.399+0000 7fb189598640 1 -- 172.21.15.86:0/4270213076 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb1841a52f0 con 0x7fb18410c510 2024-04-21T08:41:47.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.401+0000 7fb16bfff640 1 -- 172.21.15.86:0/4270213076 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb17c0119d0 con 0x7fb18410c510 2024-04-21T08:41:47.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.401+0000 7fb18c182740 1 Processor -- start 2024-04-21T08:41:47.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.401+0000 7fb18c182740 1 -- start start 2024-04-21T08:41:47.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.402+0000 7fb18c182740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff2d126310 0x55ff2d1266f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:47.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.402+0000 7fb18c182740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff2d126c30 0x55ff2d12f2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:47.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.402+0000 7fb18c182740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ff2d12f810 0x55ff2d131c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:47.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.402+0000 7fb18c182740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ff2d08b500 con 0x55ff2d126c30 2024-04-21T08:41:47.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.402+0000 7fb18c182740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55ff2d0316a0 con 0x55ff2d12f810 2024-04-21T08:41:47.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.402+0000 7fb18c182740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ff2d0874c0 con 0x55ff2d126310 2024-04-21T08:41:47.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.402+0000 7fb16bfff640 1 --2- 172.21.15.86:0/4270213076 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb16c0a3d20 0x7fb16c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:47.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.402+0000 7fb182ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff2d126c30 0x55ff2d12f2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:47.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.402+0000 7fb182ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff2d126c30 0x55ff2d12f2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39328/0 (socket says 172.21.15.86:39328) 2024-04-21T08:41:47.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.402+0000 7fb182ffd640 1 -- 172.21.15.86:0/2242489961 learned_addr learned my addr 172.21.15.86:0/2242489961 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:47.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.402+0000 7fb1837fe640 1 --2- 172.21.15.86:0/4270213076 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb16c0a3d20 0x7fb16c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:47.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.402+0000 7fb1827fc640 1 --2- 172.21.15.86:0/2242489961 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff2d126310 0x55ff2d1266f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:47.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.402+0000 7fb182ffd640 1 -- 172.21.15.86:0/2242489961 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff2d126310 msgr2=0x55ff2d1266f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:47.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.402+0000 7fb1827fc640 1 --2- 172.21.15.86:0/2242489961 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ff2d12f810 0x55ff2d131c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:47.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.402+0000 7fb182ffd640 1 --2- 172.21.15.86:0/2242489961 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff2d126310 0x55ff2d1266f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.402+0000 7fb182ffd640 1 -- 172.21.15.86:0/2242489961 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ff2d12f810 msgr2=0x55ff2d131c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:47.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.402+0000 7fb182ffd640 1 --2- 172.21.15.86:0/2242489961 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ff2d12f810 0x55ff2d131c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.402+0000 7fb182ffd640 1 -- 172.21.15.86:0/2242489961 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff2d0505e0 con 0x55ff2d126c30 2024-04-21T08:41:47.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.402+0000 7fb1827fc640 1 --2- 172.21.15.86:0/2242489961 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ff2d12f810 0x55ff2d131c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:47.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.402+0000 7fb16bfff640 1 -- 172.21.15.86:0/4270213076 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fb17c0d0190 con 0x7fb18410c510 2024-04-21T08:41:47.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.402+0000 7fb182ffd640 1 --2- 172.21.15.86:0/2242489961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff2d126c30 0x55ff2d12f2d0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fb174005e30 tx=0x7fb174009800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=57004adecc67214d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:47.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.402+0000 7fb1837fe640 1 --2- 172.21.15.86:0/4270213076 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb16c0a3d20 0x7fb16c0a61e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb17c098330 tx=0x7fb17c06e9c0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:47.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.402+0000 7fb1697fa640 1 -- 172.21.15.86:0/2242489961 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb174041020 con 0x55ff2d126c30 2024-04-21T08:41:47.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.402+0000 7fb1697fa640 1 -- 172.21.15.86:0/2242489961 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb174003e20 con 0x55ff2d126c30 2024-04-21T08:41:47.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.402+0000 7fb1697fa640 1 -- 172.21.15.86:0/2242489961 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb174039040 con 0x55ff2d126c30 2024-04-21T08:41:47.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.403+0000 7fb18c182740 1 -- 172.21.15.86:0/2242489961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff2d126c30 msgr2=0x55ff2d12f2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:47.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.403+0000 7fb18c182740 1 --2- 172.21.15.86:0/2242489961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff2d126c30 0x55ff2d12f2d0 secure :-1 s=READY pgs=981 cs=0 l=1 rev1=1 crypto rx=0x7fb174005e30 tx=0x7fb174009800 comp rx=0 tx=0).stop 2024-04-21T08:41:47.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.403+0000 7fb18c182740 1 -- 172.21.15.86:0/2242489961 shutdown_connections 2024-04-21T08:41:47.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.403+0000 7fb18c182740 1 --2- 172.21.15.86:0/2242489961 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ff2d12f810 0x55ff2d131c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.403+0000 7fb18c182740 1 --2- 172.21.15.86:0/2242489961 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff2d126c30 0x55ff2d12f2d0 unknown :-1 s=CLOSED pgs=981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.403+0000 7fb18c182740 1 --2- 172.21.15.86:0/2242489961 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff2d126310 0x55ff2d1266f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.403+0000 7fb18c182740 1 -- 172.21.15.86:0/2242489961 >> 172.21.15.86:0/2242489961 conn(0x55ff2d120a40 msgr2=0x55ff2d120e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:47.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.403+0000 7fb18c182740 1 -- 172.21.15.86:0/2242489961 shutdown_connections 2024-04-21T08:41:47.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.403+0000 7fb18c182740 1 -- 172.21.15.86:0/2242489961 wait complete. 2024-04-21T08:41:47.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.403+0000 7fb18c182740 1 -- start start 2024-04-21T08:41:47.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.403+0000 7fb18c182740 1 Processor -- start 2024-04-21T08:41:47.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.404+0000 7fb18c182740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff2d126310 0x55ff2d12c680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:47.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.404+0000 7fb18c182740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ff2d126c30 0x55ff2d12cbc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:47.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.404+0000 7fb18c182740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff2d12f810 0x55ff2d12d100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:47.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.404+0000 7fb18c182740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ff2d1325a0 con 0x55ff2d126310 2024-04-21T08:41:47.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.404+0000 7fb18c182740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55ff2d044a60 con 0x55ff2d126c30 2024-04-21T08:41:47.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.404+0000 7fb18c182740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ff2d132720 con 0x55ff2d12f810 2024-04-21T08:41:47.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.404+0000 7fb18c182740 10 client.0 _ll_register_callbacks cb 0x55ff2cd9d110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:47.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.404+0000 7fb1827fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff2d126310 0x55ff2d12c680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:47.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.404+0000 7fb1827fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff2d126310 0x55ff2d12c680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39330/0 (socket says 172.21.15.86:39330) 2024-04-21T08:41:47.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.404+0000 7fb1827fc640 1 -- 172.21.15.86:0/4175737453 learned_addr learned my addr 172.21.15.86:0/4175737453 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:47.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.404+0000 7fb1827fc640 1 --2- 172.21.15.86:0/4175737453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff2d12f810 0x55ff2d12d100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:47.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.404+0000 7fb182ffd640 1 --2- 172.21.15.86:0/4175737453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ff2d126c30 0x55ff2d12cbc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:47.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.404+0000 7fb1827fc640 1 -- 172.21.15.86:0/4175737453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff2d12f810 msgr2=0x55ff2d12d100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:47.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.404+0000 7fb1827fc640 1 --2- 172.21.15.86:0/4175737453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff2d12f810 0x55ff2d12d100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.404+0000 7fb1827fc640 1 -- 172.21.15.86:0/4175737453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ff2d126c30 msgr2=0x55ff2d12cbc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:47.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.404+0000 7fb1827fc640 1 --2- 172.21.15.86:0/4175737453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ff2d126c30 0x55ff2d12cbc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.404+0000 7fb1827fc640 1 -- 172.21.15.86:0/4175737453 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff2cd9d920 con 0x55ff2d126310 2024-04-21T08:41:47.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.404+0000 7fb1827fc640 1 --2- 172.21.15.86:0/4175737453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff2d126310 0x55ff2d12c680 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fb17800df60 tx=0x7fb1780046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1c85c43ab21548f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:47.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.404+0000 7fb166ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:47.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.404+0000 7fb166ff5640 1 -- 172.21.15.86:0/4175737453 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb17800eae0 con 0x55ff2d126310 2024-04-21T08:41:47.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.404+0000 7fb166ff5640 1 -- 172.21.15.86:0/4175737453 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fb17800ec80 con 0x55ff2d126310 2024-04-21T08:41:47.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.404+0000 7fb18c182740 1 -- 172.21.15.86:0/4175737453 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb174003e20 con 0x55ff2d126310 2024-04-21T08:41:47.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.405+0000 7fb166ff5640 1 -- 172.21.15.86:0/4175737453 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb178023050 con 0x55ff2d126310 2024-04-21T08:41:47.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.405+0000 7fb18c182740 10 client.17112 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:47.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.405+0000 7fb18c182740 1 -- 172.21.15.86:0/4175737453 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ff2ce69d30 con 0x55ff2d126310 2024-04-21T08:41:47.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.405+0000 7fb166ff5640 1 -- 172.21.15.86:0/4175737453 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb174003e20 con 0x55ff2d126310 2024-04-21T08:41:47.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.405+0000 7fb166ff5640 1 -- 172.21.15.86:0/4175737453 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb178004a70 con 0x55ff2d126310 2024-04-21T08:41:47.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.405+0000 7fb18c182740 10 client.17112 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:47.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.405+0000 7fb18c182740 10 client.17112 resolve_mds: validated gid 24482 aka daemon mds.b 2024-04-21T08:41:47.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.405+0000 7fb18c182740 20 client.17112 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:47.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.405+0000 7fb18c182740 1 --2- 172.21.15.86:0/4175737453 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ff2d13fa50 0x55ff2d141eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:47.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.405+0000 7fb18c182740 4 client.17112 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:47.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.405+0000 7fb18c182740 1 -- 172.21.15.86:0/4175737453 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ff2d05ea10 con 0x55ff2d13fa50 2024-04-21T08:41:47.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.405+0000 7fb182ffd640 1 --2- 172.21.15.86:0/4175737453 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ff2d13fa50 0x55ff2d141eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:47.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.406+0000 7fb166ff5640 10 client.17112 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:47.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.406+0000 7fb182ffd640 1 --2- 172.21.15.86:0/4175737453 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ff2d13fa50 0x55ff2d141eb0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da22b4c2508b37f7 server_cookie=8a7a48031a34b374 in_seq=0 out_seq=0 2024-04-21T08:41:47.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.406+0000 7fb166ff5640 1 -- 172.21.15.86:0/4175737453 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ff2d05ea10 con 0x55ff2d13fa50 2024-04-21T08:41:47.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.406+0000 7fb166ff5640 10 client.17112 handle_command_reply: tid=0 2024-04-21T08:41:47.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.406+0000 7fb18c182740 1 client.17112 shutdown 2024-04-21T08:41:47.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.406+0000 7fb18c182740 1 -- 172.21.15.86:0/4175737453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff2d126310 msgr2=0x55ff2d12c680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:47.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.406+0000 7fb18c182740 1 --2- 172.21.15.86:0/4175737453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff2d126310 0x55ff2d12c680 secure :-1 s=READY pgs=982 cs=0 l=1 rev1=1 crypto rx=0x7fb17800df60 tx=0x7fb1780046c0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.406+0000 7fb18c182740 1 -- 172.21.15.86:0/4175737453 shutdown_connections 2024-04-21T08:41:47.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.406+0000 7fb18c182740 1 --2- 172.21.15.86:0/4175737453 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ff2d13fa50 0x55ff2d141eb0 crc :-1 s=READY pgs=123 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.406+0000 7fb18c182740 1 --2- 172.21.15.86:0/4175737453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff2d12f810 0x55ff2d12d100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.406+0000 7fb166ff5640 0 client.17112 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:47.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.406+0000 7fb18c182740 1 --2- 172.21.15.86:0/4175737453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ff2d126c30 0x55ff2d12cbc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.406+0000 7fb18c182740 1 --2- 172.21.15.86:0/4175737453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff2d126310 0x55ff2d12c680 unknown :-1 s=CLOSED pgs=982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.406+0000 7fb18c182740 1 -- 172.21.15.86:0/4175737453 >> 172.21.15.86:0/4175737453 conn(0x55ff2d120a40 msgr2=0x55ff2d122790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:47.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.407+0000 7fb18c182740 1 -- 172.21.15.86:0/4175737453 shutdown_connections 2024-04-21T08:41:47.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.407+0000 7fb18c182740 1 -- 172.21.15.86:0/4175737453 wait complete. 2024-04-21T08:41:47.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.407+0000 7fb18c182740 20 client.17112 trim_cache size 0 max 16384 2024-04-21T08:41:47.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.423+0000 7fb18c182740 1 Processor -- start 2024-04-21T08:41:47.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.423+0000 7fb18c182740 1 -- start start 2024-04-21T08:41:47.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.423+0000 7fb18c182740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ff2d120a40 0x55ff2d13b4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:47.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.423+0000 7fb18c182740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff2d126310 0x55ff2d147780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:47.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.423+0000 7fb18c182740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff2d126c30 0x55ff2d147cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:47.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.423+0000 7fb18c182740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb1841af2c0 con 0x55ff2d126c30 2024-04-21T08:41:47.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.423+0000 7fb18c182740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb1841b0e00 con 0x55ff2d120a40 2024-04-21T08:41:47.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.423+0000 7fb18c182740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb1841af990 con 0x55ff2d126310 2024-04-21T08:41:47.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.424+0000 7fb1827fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff2d126c30 0x55ff2d147cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:47.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.424+0000 7fb1827fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff2d126c30 0x55ff2d147cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39342/0 (socket says 172.21.15.86:39342) 2024-04-21T08:41:47.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.424+0000 7fb1827fc640 1 -- 172.21.15.86:0/2190173909 learned_addr learned my addr 172.21.15.86:0/2190173909 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:47.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.424+0000 7fb182ffd640 1 --2- 172.21.15.86:0/2190173909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff2d126310 0x55ff2d147780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:47.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.424+0000 7fb1827fc640 1 --2- 172.21.15.86:0/2190173909 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ff2d120a40 0x55ff2d13b4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:47.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.424+0000 7fb1827fc640 1 -- 172.21.15.86:0/2190173909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff2d126310 msgr2=0x55ff2d147780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:47.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.424+0000 7fb1827fc640 1 --2- 172.21.15.86:0/2190173909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff2d126310 0x55ff2d147780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.424+0000 7fb1827fc640 1 -- 172.21.15.86:0/2190173909 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ff2d120a40 msgr2=0x55ff2d13b4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:47.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.424+0000 7fb1827fc640 1 --2- 172.21.15.86:0/2190173909 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ff2d120a40 0x55ff2d13b4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.424+0000 7fb1827fc640 1 -- 172.21.15.86:0/2190173909 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ff2d13e5a0 con 0x55ff2d126c30 2024-04-21T08:41:47.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.424+0000 7fb1827fc640 1 --2- 172.21.15.86:0/2190173909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff2d126c30 0x55ff2d147cc0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fb178004e70 tx=0x7fb178005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=133fc36e31701d31 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:47.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.424+0000 7fb168ff9640 1 -- 172.21.15.86:0/2190173909 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb178012110 con 0x55ff2d126c30 2024-04-21T08:41:47.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.424+0000 7fb168ff9640 1 -- 172.21.15.86:0/2190173909 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1780122b0 con 0x55ff2d126c30 2024-04-21T08:41:47.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.424+0000 7fb168ff9640 1 -- 172.21.15.86:0/2190173909 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb178010a70 con 0x55ff2d126c30 2024-04-21T08:41:47.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.425+0000 7fb18c182740 1 -- 172.21.15.86:0/2190173909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff2d126c30 msgr2=0x55ff2d147cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:47.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.425+0000 7fb18c182740 1 --2- 172.21.15.86:0/2190173909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff2d126c30 0x55ff2d147cc0 secure :-1 s=READY pgs=983 cs=0 l=1 rev1=1 crypto rx=0x7fb178004e70 tx=0x7fb178005e50 comp rx=0 tx=0).stop 2024-04-21T08:41:47.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.425+0000 7fb18c182740 1 -- 172.21.15.86:0/2190173909 shutdown_connections 2024-04-21T08:41:47.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.425+0000 7fb18c182740 1 --2- 172.21.15.86:0/2190173909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff2d126c30 0x55ff2d147cc0 unknown :-1 s=CLOSED pgs=983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.425+0000 7fb18c182740 1 --2- 172.21.15.86:0/2190173909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff2d126310 0x55ff2d147780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.425+0000 7fb18c182740 1 --2- 172.21.15.86:0/2190173909 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ff2d120a40 0x55ff2d13b4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.425+0000 7fb18c182740 1 -- 172.21.15.86:0/2190173909 >> 172.21.15.86:0/2190173909 conn(0x55ff2d11fb80 msgr2=0x55ff2d121e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:47.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.425+0000 7fb18c182740 1 -- 172.21.15.86:0/2190173909 shutdown_connections 2024-04-21T08:41:47.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.425+0000 7fb18c182740 1 -- 172.21.15.86:0/2190173909 wait complete. 2024-04-21T08:41:47.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.425+0000 7fb18c182740 1 -- start start 2024-04-21T08:41:47.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.425+0000 7fb18c182740 1 Processor -- start 2024-04-21T08:41:47.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.425+0000 7fb18c182740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff2d120a40 0x55ff2d134b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:47.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.425+0000 7fb18c182740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ff2d126310 0x55ff2d1350d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:47.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.425+0000 7fb18c182740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff2d126c30 0x55ff2d1a6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:47.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.425+0000 7fb18c182740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb1841b0730 con 0x55ff2d120a40 2024-04-21T08:41:47.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.425+0000 7fb18c182740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb1841b0060 con 0x55ff2d126310 2024-04-21T08:41:47.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.425+0000 7fb18c182740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb1841ad280 con 0x55ff2d126c30 2024-04-21T08:41:47.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.425+0000 7fb18c182740 10 client.0 _ll_register_callbacks cb 0x55ff2d12f760 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:47.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.425+0000 7fb1827fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff2d120a40 0x55ff2d134b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:47.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.426+0000 7fb1827fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff2d120a40 0x55ff2d134b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39354/0 (socket says 172.21.15.86:39354) 2024-04-21T08:41:47.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.426+0000 7fb1827fc640 1 -- 172.21.15.86:0/2058649012 learned_addr learned my addr 172.21.15.86:0/2058649012 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:47.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.426+0000 7fb182ffd640 1 --2- 172.21.15.86:0/2058649012 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ff2d126310 0x55ff2d1350d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:47.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.426+0000 7fb1827fc640 1 -- 172.21.15.86:0/2058649012 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff2d126c30 msgr2=0x55ff2d1a6000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:47.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.426+0000 7fb1827fc640 1 --2- 172.21.15.86:0/2058649012 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff2d126c30 0x55ff2d1a6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.426+0000 7fb1827fc640 1 -- 172.21.15.86:0/2058649012 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ff2d126310 msgr2=0x55ff2d1350d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:47.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.426+0000 7fb1827fc640 1 --2- 172.21.15.86:0/2058649012 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ff2d126310 0x55ff2d1350d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.426+0000 7fb1827fc640 1 -- 172.21.15.86:0/2058649012 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ff2d1a6540 con 0x55ff2d120a40 2024-04-21T08:41:47.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.426+0000 7fb1827fc640 1 --2- 172.21.15.86:0/2058649012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff2d120a40 0x55ff2d134b90 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fb178012500 tx=0x7fb178005c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c50ac1f1765dfeea server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:47.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.426+0000 7fb166ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:47.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.426+0000 7fb166ff5640 1 -- 172.21.15.86:0/2058649012 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb178012110 con 0x55ff2d120a40 2024-04-21T08:41:47.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.426+0000 7fb166ff5640 1 -- 172.21.15.86:0/2058649012 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fb17800ecf0 con 0x55ff2d120a40 2024-04-21T08:41:47.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.426+0000 7fb18c182740 1 -- 172.21.15.86:0/2058649012 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb1780122b0 con 0x55ff2d120a40 2024-04-21T08:41:47.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.426+0000 7fb166ff5640 1 -- 172.21.15.86:0/2058649012 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb17803d050 con 0x55ff2d120a40 2024-04-21T08:41:47.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.426+0000 7fb18c182740 10 client.17118 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:47.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.426+0000 7fb18c182740 1 -- 172.21.15.86:0/2058649012 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ff2d1a77f0 con 0x55ff2d120a40 2024-04-21T08:41:47.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.426+0000 7fb166ff5640 1 -- 172.21.15.86:0/2058649012 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb1780122b0 con 0x55ff2d120a40 2024-04-21T08:41:47.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.426+0000 7fb166ff5640 1 -- 172.21.15.86:0/2058649012 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb178004f90 con 0x55ff2d120a40 2024-04-21T08:41:47.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.427+0000 7fb18c182740 10 client.17118 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:47.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.427+0000 7fb18c182740 10 client.17118 resolve_mds: validated gid 24482 aka daemon mds.b 2024-04-21T08:41:47.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.427+0000 7fb18c182740 20 client.17118 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:47.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.427+0000 7fb18c182740 1 --2- 172.21.15.86:0/2058649012 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ff2d12f810 0x55ff2d1a9dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:47.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.427+0000 7fb18c182740 4 client.17118 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "ea3fd78e", "release": true, "await": true}] 2024-04-21T08:41:47.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.427+0000 7fb18c182740 1 -- 172.21.15.86:0/2058649012 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "quiesce db", "set_id": "ea3fd78e", "release": true, "await": true}) -- 0x55ff2d098f20 con 0x55ff2d12f810 2024-04-21T08:41:47.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.427+0000 7fb182ffd640 1 --2- 172.21.15.86:0/2058649012 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ff2d12f810 0x55ff2d1a9dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:47.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.427+0000 7fb166ff5640 10 client.17118 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:47.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.427+0000 7fb182ffd640 1 --2- 172.21.15.86:0/2058649012 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ff2d12f810 0x55ff2d1a9dc0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b77f64dabb7201b8 server_cookie=1f53575933ca6e28 in_seq=0 out_seq=0 2024-04-21T08:41:47.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.430+0000 7fb166ff5640 1 -- 172.21.15.86:0/2058649012 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+588 (crc 0 0 0) 0x55ff2d098f20 con 0x55ff2d12f810 2024-04-21T08:41:47.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.430+0000 7fb166ff5640 10 client.17118 handle_command_reply: tid=0 2024-04-21T08:41:47.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.431+0000 7fb18c182740 1 client.17118 shutdown 2024-04-21T08:41:47.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.431+0000 7fb18c182740 1 -- 172.21.15.86:0/2058649012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff2d120a40 msgr2=0x55ff2d134b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:47.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.431+0000 7fb18c182740 1 --2- 172.21.15.86:0/2058649012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff2d120a40 0x55ff2d134b90 secure :-1 s=READY pgs=984 cs=0 l=1 rev1=1 crypto rx=0x7fb178012500 tx=0x7fb178005c60 comp rx=0 tx=0).stop 2024-04-21T08:41:47.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.431+0000 7fb18c182740 1 -- 172.21.15.86:0/2058649012 shutdown_connections 2024-04-21T08:41:47.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.431+0000 7fb18c182740 1 --2- 172.21.15.86:0/2058649012 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ff2d12f810 0x55ff2d1a9dc0 crc :-1 s=READY pgs=124 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.431+0000 7fb18c182740 1 --2- 172.21.15.86:0/2058649012 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ff2d126c30 0x55ff2d1a6000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.431+0000 7fb166ff5640 0 client.17118 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:47.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.431+0000 7fb18c182740 1 --2- 172.21.15.86:0/2058649012 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ff2d126310 0x55ff2d1350d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.431+0000 7fb18c182740 1 --2- 172.21.15.86:0/2058649012 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ff2d120a40 0x55ff2d134b90 unknown :-1 s=CLOSED pgs=984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.431+0000 7fb18c182740 1 -- 172.21.15.86:0/2058649012 >> 172.21.15.86:0/2058649012 conn(0x55ff2d11fb80 msgr2=0x55ff2d12c640 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:47.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.431+0000 7fb18c182740 1 -- 172.21.15.86:0/2058649012 shutdown_connections 2024-04-21T08:41:47.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.431+0000 7fb18c182740 1 -- 172.21.15.86:0/2058649012 wait complete. 2024-04-21T08:41:47.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.431+0000 7fb18c182740 20 client.17118 trim_cache size 0 max 16384 2024-04-21T08:41:47.434 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:41:47.434 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 29, 2024-04-21T08:41:47.435 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24482, 2024-04-21T08:41:47.435 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 8, 2024-04-21T08:41:47.435 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-04-21T08:41:47.435 INFO:teuthology.orchestra.run.smithi086.stdout: "ea3fd78e": { 2024-04-21T08:41:47.435 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 8, 2024-04-21T08:41:47.435 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-04-21T08:41:47.435 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-21T08:41:47.435 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-04-21T08:41:47.435 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-21T08:41:47.435 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:41:47.435 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 30.0, 2024-04-21T08:41:47.435 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 81.9, 2024-04-21T08:41:47.435 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-04-21T08:41:47.435 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-04-21T08:41:47.436 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-04-21T08:41:47.436 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-21T08:41:47.436 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-04-21T08:41:47.436 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-21T08:41:47.436 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:41:47.436 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:41:47.436 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:41:47.436 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:41:47.436 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:41:47.436 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:41:47.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.431+0000 7fb189598640 1 -- 172.21.15.86:0/4270213076 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb16c0a3d20 msgr2=0x7fb16c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:47.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.431+0000 7fb189598640 1 --2- 172.21.15.86:0/4270213076 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb16c0a3d20 0x7fb16c0a61e0 secure :-1 s=READY pgs=148 cs=0 l=1 rev1=1 crypto rx=0x7fb17c098330 tx=0x7fb17c06e9c0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.431+0000 7fb189598640 1 -- 172.21.15.86:0/4270213076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb18410c510 msgr2=0x7fb1841985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:47.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.431+0000 7fb189598640 1 --2- 172.21.15.86:0/4270213076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb18410c510 0x7fb1841985e0 secure :-1 s=READY pgs=980 cs=0 l=1 rev1=1 crypto rx=0x7fb17c009fd0 tx=0x7fb17c00b8e0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.431+0000 7fb189598640 1 -- 172.21.15.86:0/4270213076 shutdown_connections 2024-04-21T08:41:47.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.431+0000 7fb189598640 1 --2- 172.21.15.86:0/4270213076 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb16c0a3d20 0x7fb16c0a61e0 unknown :-1 s=CLOSED pgs=148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.431+0000 7fb189598640 1 --2- 172.21.15.86:0/4270213076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb18410c510 0x7fb1841985e0 unknown :-1 s=CLOSED pgs=980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.431+0000 7fb189598640 1 --2- 172.21.15.86:0/4270213076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb184103af0 0x7fb18419e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.431+0000 7fb189598640 1 --2- 172.21.15.86:0/4270213076 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb1841031d0 0x7fb18419e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:47.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.431+0000 7fb189598640 1 -- 172.21.15.86:0/4270213076 >> 172.21.15.86:0/4270213076 conn(0x7fb184077f50 msgr2=0x7fb1840fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:47.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.431+0000 7fb189598640 1 -- 172.21.15.86:0/4270213076 shutdown_connections 2024-04-21T08:41:47.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:41:47.431+0000 7fb189598640 1 -- 172.21.15.86:0/4270213076 wait complete. 2024-04-21T08:41:47.441 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set_id: ea3fd78e, seconds elapsed: 22.9 2024-04-21T08:41:47.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:47 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/318341346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:41:47.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:47 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/318341346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:41:47.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:47 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/318341346' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:41:48.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:48 smithi086 ceph-mon[27939]: pgmap v93: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-04-21T08:41:48.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:48 smithi088 ceph-mon[31259]: pgmap v93: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-04-21T08:41:48.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:48 smithi179 ceph-mon[31829]: pgmap v93: 129 pgs: 129 active+clean; 2.9 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 5.6 KiB/s rd, 9 op/s 2024-04-21T08:41:50.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:50 smithi086 ceph-mon[27939]: pgmap v94: 129 pgs: 129 active+clean; 3.0 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.3 KiB/s wr, 14 op/s 2024-04-21T08:41:50.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:50 smithi088 ceph-mon[31259]: pgmap v94: 129 pgs: 129 active+clean; 3.0 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.3 KiB/s wr, 14 op/s 2024-04-21T08:41:50.750 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:50 smithi179 ceph-mon[31829]: pgmap v94: 129 pgs: 129 active+clean; 3.0 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 2.3 KiB/s wr, 14 op/s 2024-04-21T08:41:52.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:52 smithi086 ceph-mon[27939]: pgmap v95: 129 pgs: 129 active+clean; 3.0 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-04-21T08:41:52.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:52 smithi088 ceph-mon[31259]: pgmap v95: 129 pgs: 129 active+clean; 3.0 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-04-21T08:41:52.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:52 smithi179 ceph-mon[31829]: pgmap v95: 129 pgs: 129 active+clean; 3.0 MiB data, 391 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.3 KiB/s wr, 9 op/s 2024-04-21T08:41:54.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:54 smithi086 ceph-mon[27939]: pgmap v96: 129 pgs: 129 active+clean; 3.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-04-21T08:41:54.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:54 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:41:54.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:54 smithi088 ceph-mon[31259]: pgmap v96: 129 pgs: 129 active+clean; 3.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-04-21T08:41:54.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:54 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:41:54.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:54 smithi179 ceph-mon[31829]: pgmap v96: 129 pgs: 129 active+clean; 3.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 13 KiB/s wr, 15 op/s 2024-04-21T08:41:54.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:54 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:41:55.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:55 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:41:55.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:55 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:41:55.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:55 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:41:55.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:55 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:41:55.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:55 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:41:55.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:55 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:41:55.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:55 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:41:55.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:55 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:41:55.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:55 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:41:56.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:56 smithi086 ceph-mon[27939]: pgmap v97: 129 pgs: 129 active+clean; 3.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-04-21T08:41:56.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:56 smithi088 ceph-mon[31259]: pgmap v97: 129 pgs: 129 active+clean; 3.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-04-21T08:41:56.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:56 smithi179 ceph-mon[31829]: pgmap v97: 129 pgs: 129 active+clean; 3.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-04-21T08:41:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.866+0000 7f0d4ae2e640 1 -- 172.21.15.86:0/2564030260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d44101a90 msgr2=0x7f0d44105ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.866+0000 7f0d4ae2e640 1 --2- 172.21.15.86:0/2564030260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d44101a90 0x7f0d44105ad0 secure :-1 s=READY pgs=985 cs=0 l=1 rev1=1 crypto rx=0x7f0d30009900 tx=0x7f0d3002bc30 comp rx=0 tx=0).stop 2024-04-21T08:41:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.866+0000 7f0d4ae2e640 1 -- 172.21.15.86:0/2564030260 shutdown_connections 2024-04-21T08:41:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.866+0000 7f0d4ae2e640 1 --2- 172.21.15.86:0/2564030260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d441061b0 0x7f0d4410dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.866+0000 7f0d4ae2e640 1 --2- 172.21.15.86:0/2564030260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d44101a90 0x7f0d44105ad0 unknown :-1 s=CLOSED pgs=985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.866+0000 7f0d4ae2e640 1 --2- 172.21.15.86:0/2564030260 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d441010e0 0x7f0d441014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:57.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.866+0000 7f0d4ae2e640 1 -- 172.21.15.86:0/2564030260 >> 172.21.15.86:0/2564030260 conn(0x7f0d440771d0 msgr2=0x7f0d440775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.866+0000 7f0d4ae2e640 1 -- 172.21.15.86:0/2564030260 shutdown_connections 2024-04-21T08:41:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.866+0000 7f0d4ae2e640 1 -- 172.21.15.86:0/2564030260 wait complete. 2024-04-21T08:41:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.867+0000 7f0d4ae2e640 1 Processor -- start 2024-04-21T08:41:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.867+0000 7f0d4ae2e640 1 -- start start 2024-04-21T08:41:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.867+0000 7f0d4ae2e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d441010e0 0x7f0d44197550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.867+0000 7f0d4ae2e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d441061b0 0x7f0d44197a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.867+0000 7f0d4ae2e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d4419be10 0x7f0d4419c2c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.867+0000 7f0d4ae2e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d441110f0 con 0x7f0d4419be10 2024-04-21T08:41:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.867+0000 7f0d4ae2e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0d44110f70 con 0x7f0d441061b0 2024-04-21T08:41:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.867+0000 7f0d4ae2e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0d44111270 con 0x7f0d441010e0 2024-04-21T08:41:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.867+0000 7f0d493cb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d4419be10 0x7f0d4419c2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.867+0000 7f0d493cb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d4419be10 0x7f0d4419c2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44102/0 (socket says 172.21.15.86:44102) 2024-04-21T08:41:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.867+0000 7f0d493cb640 1 -- 172.21.15.86:0/2005177641 learned_addr learned my addr 172.21.15.86:0/2005177641 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:57.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.867+0000 7f0d43fff640 1 --2- 172.21.15.86:0/2005177641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d441061b0 0x7f0d44197a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.868+0000 7f0d48bca640 1 --2- 172.21.15.86:0/2005177641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d441010e0 0x7f0d44197550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.868+0000 7f0d493cb640 1 -- 172.21.15.86:0/2005177641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d441010e0 msgr2=0x7f0d44197550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.868+0000 7f0d493cb640 1 --2- 172.21.15.86:0/2005177641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d441010e0 0x7f0d44197550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.868+0000 7f0d493cb640 1 -- 172.21.15.86:0/2005177641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d441061b0 msgr2=0x7f0d44197a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.868+0000 7f0d493cb640 1 --2- 172.21.15.86:0/2005177641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d441061b0 0x7f0d44197a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.868+0000 7f0d493cb640 1 -- 172.21.15.86:0/2005177641 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d4419c900 con 0x7f0d4419be10 2024-04-21T08:41:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.868+0000 7f0d493cb640 1 --2- 172.21.15.86:0/2005177641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d4419be10 0x7f0d4419c2c0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00eaa0 tx=0x7f0d3c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.868+0000 7f0d41ffb640 1 -- 172.21.15.86:0/2005177641 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c00cdb0 con 0x7f0d4419be10 2024-04-21T08:41:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.868+0000 7f0d41ffb640 1 -- 172.21.15.86:0/2005177641 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d3c005020 con 0x7f0d4419be10 2024-04-21T08:41:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.868+0000 7f0d4ae2e640 1 -- 172.21.15.86:0/2005177641 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d4419cbf0 con 0x7f0d4419be10 2024-04-21T08:41:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.868+0000 7f0d41ffb640 1 -- 172.21.15.86:0/2005177641 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c0116d0 con 0x7f0d4419be10 2024-04-21T08:41:57.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.868+0000 7f0d4ae2e640 1 -- 172.21.15.86:0/2005177641 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d441a43b0 con 0x7f0d4419be10 2024-04-21T08:41:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.870+0000 7f0d41ffb640 1 -- 172.21.15.86:0/2005177641 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0d3c0051d0 con 0x7f0d4419be10 2024-04-21T08:41:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.871+0000 7f0d4da17740 1 Processor -- start 2024-04-21T08:41:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.871+0000 7f0d4da17740 1 -- start start 2024-04-21T08:41:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.871+0000 7f0d41ffb640 1 --2- 172.21.15.86:0/2005177641 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0d340a3dc0 0x7f0d340a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.871+0000 7f0d4da17740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c550768d0 0x560c55075620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.871+0000 7f0d4da17740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c55075b60 0x560c55075fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.871+0000 7f0d4da17740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560c5507f0e0 0x560c550814d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:57.877 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.871+0000 7f0d4da17740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560c54fda8d0 con 0x560c55075b60 2024-04-21T08:41:57.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.871+0000 7f0d4da17740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x560c54f77e80 con 0x560c550768d0 2024-04-21T08:41:57.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.871+0000 7f0d4da17740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x560c54fd6890 con 0x560c5507f0e0 2024-04-21T08:41:57.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.871+0000 7f0d43fff640 1 --2- 172.21.15.86:0/2005177641 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0d340a3dc0 0x7f0d340a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.871+0000 7f0d41ffb640 1 -- 172.21.15.86:0/2005177641 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f0d3c0d0cb0 con 0x7f0d4419be10 2024-04-21T08:41:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.871+0000 7f0d43fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c55075b60 0x560c55075fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.871+0000 7f0d43fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c55075b60 0x560c55075fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44104/0 (socket says 172.21.15.86:44104) 2024-04-21T08:41:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.871+0000 7f0d43fff640 1 -- 172.21.15.86:0/3923622670 learned_addr learned my addr 172.21.15.86:0/3923622670 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.871+0000 7f0d493cb640 1 --2- 172.21.15.86:0/3923622670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560c5507f0e0 0x560c550814d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.871+0000 7f0d48bca640 1 --2- 172.21.15.86:0/3923622670 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c550768d0 0x560c55075620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.872+0000 7f0d43fff640 1 -- 172.21.15.86:0/3923622670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560c5507f0e0 msgr2=0x560c550814d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.872+0000 7f0d43fff640 1 --2- 172.21.15.86:0/3923622670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560c5507f0e0 0x560c550814d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.872+0000 7f0d43fff640 1 -- 172.21.15.86:0/3923622670 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c550768d0 msgr2=0x560c55075620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.872+0000 7f0d43fff640 1 --2- 172.21.15.86:0/3923622670 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c550768d0 0x560c55075620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:57.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.872+0000 7f0d43fff640 1 -- 172.21.15.86:0/3923622670 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c54f99320 con 0x560c55075b60 2024-04-21T08:41:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.872+0000 7f0d43fff640 1 --2- 172.21.15.86:0/2005177641 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0d340a3dc0 0x7f0d340a6280 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0d4406a570 tx=0x7f0d3002c6c0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.872+0000 7f0d43fff640 1 --2- 172.21.15.86:0/3923622670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c55075b60 0x560c55075fe0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f0d30035000 tx=0x7f0d30047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e13f3d7a78c0ec72 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.872+0000 7f0d2effd640 1 -- 172.21.15.86:0/3923622670 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d30053070 con 0x560c55075b60 2024-04-21T08:41:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.872+0000 7f0d2effd640 1 -- 172.21.15.86:0/3923622670 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d3002cba0 con 0x560c55075b60 2024-04-21T08:41:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.872+0000 7f0d2effd640 1 -- 172.21.15.86:0/3923622670 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d30003850 con 0x560c55075b60 2024-04-21T08:41:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.873+0000 7f0d4da17740 1 -- 172.21.15.86:0/3923622670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c55075b60 msgr2=0x560c55075fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.873+0000 7f0d4da17740 1 --2- 172.21.15.86:0/3923622670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c55075b60 0x560c55075fe0 secure :-1 s=READY pgs=987 cs=0 l=1 rev1=1 crypto rx=0x7f0d30035000 tx=0x7f0d30047000 comp rx=0 tx=0).stop 2024-04-21T08:41:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.873+0000 7f0d4da17740 1 -- 172.21.15.86:0/3923622670 shutdown_connections 2024-04-21T08:41:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.873+0000 7f0d4da17740 1 --2- 172.21.15.86:0/3923622670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560c5507f0e0 0x560c550814d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.873+0000 7f0d4da17740 1 --2- 172.21.15.86:0/3923622670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c55075b60 0x560c55075fe0 unknown :-1 s=CLOSED pgs=987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.873+0000 7f0d4da17740 1 --2- 172.21.15.86:0/3923622670 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c550768d0 0x560c55075620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:57.880 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.873+0000 7f0d4da17740 1 -- 172.21.15.86:0/3923622670 >> 172.21.15.86:0/3923622670 conn(0x560c5503cc90 msgr2=0x560c55070850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.873+0000 7f0d4da17740 1 -- 172.21.15.86:0/3923622670 shutdown_connections 2024-04-21T08:41:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.873+0000 7f0d4da17740 1 -- 172.21.15.86:0/3923622670 wait complete. 2024-04-21T08:41:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.873+0000 7f0d4da17740 1 -- start start 2024-04-21T08:41:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.873+0000 7f0d4da17740 1 Processor -- start 2024-04-21T08:41:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.874+0000 7f0d4da17740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c55075b60 0x560c5507eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.874+0000 7f0d4da17740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560c550768d0 0x560c5507be70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.874+0000 7f0d4da17740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c5507f0e0 0x560c5507c480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.874+0000 7f0d4da17740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560c55081db0 con 0x560c5507f0e0 2024-04-21T08:41:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.874+0000 7f0d4da17740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x560c54f62eb0 con 0x560c55075b60 2024-04-21T08:41:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.874+0000 7f0d4da17740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x560c55081f30 con 0x560c550768d0 2024-04-21T08:41:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.874+0000 7f0d4da17740 10 client.0 _ll_register_callbacks cb 0x560c54f8fb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.874+0000 7f0d493cb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c5507f0e0 0x560c5507c480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.874+0000 7f0d493cb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c5507f0e0 0x560c5507c480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44118/0 (socket says 172.21.15.86:44118) 2024-04-21T08:41:57.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.874+0000 7f0d493cb640 1 -- 172.21.15.86:0/3157382010 learned_addr learned my addr 172.21.15.86:0/3157382010 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.874+0000 7f0d48bca640 1 --2- 172.21.15.86:0/3157382010 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c55075b60 0x560c5507eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.874+0000 7f0d43fff640 1 --2- 172.21.15.86:0/3157382010 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560c550768d0 0x560c5507be70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.874+0000 7f0d493cb640 1 -- 172.21.15.86:0/3157382010 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560c550768d0 msgr2=0x560c5507be70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.874+0000 7f0d493cb640 1 --2- 172.21.15.86:0/3157382010 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560c550768d0 0x560c5507be70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.874+0000 7f0d493cb640 1 -- 172.21.15.86:0/3157382010 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c55075b60 msgr2=0x560c5507eae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.874+0000 7f0d493cb640 1 --2- 172.21.15.86:0/3157382010 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c55075b60 0x560c5507eae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.874+0000 7f0d493cb640 1 -- 172.21.15.86:0/3157382010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c54ced210 con 0x560c5507f0e0 2024-04-21T08:41:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.874+0000 7f0d43fff640 1 --2- 172.21.15.86:0/3157382010 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560c550768d0 0x560c5507be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.874+0000 7f0d48bca640 1 --2- 172.21.15.86:0/3157382010 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c55075b60 0x560c5507eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:41:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.874+0000 7f0d493cb640 1 --2- 172.21.15.86:0/3157382010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c5507f0e0 0x560c5507c480 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c098cf0 tx=0x7f0d3c09c330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c3b39b097a4af30 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.874+0000 7f0d23fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:57.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.874+0000 7f0d23fff640 1 -- 172.21.15.86:0/3157382010 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c01f040 con 0x560c5507f0e0 2024-04-21T08:41:57.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.874+0000 7f0d23fff640 1 -- 172.21.15.86:0/3157382010 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f0d3c0d09f0 con 0x560c5507f0e0 2024-04-21T08:41:57.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.874+0000 7f0d4da17740 1 -- 172.21.15.86:0/3157382010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d3002cba0 con 0x560c5507f0e0 2024-04-21T08:41:57.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.875+0000 7f0d23fff640 1 -- 172.21.15.86:0/3157382010 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0d3c09c8e0 con 0x560c5507f0e0 2024-04-21T08:41:57.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.875+0000 7f0d4da17740 10 client.17130 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:57.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.875+0000 7f0d4da17740 1 -- 172.21.15.86:0/3157382010 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560c54db9620 con 0x560c5507f0e0 2024-04-21T08:41:57.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.875+0000 7f0d23fff640 1 -- 172.21.15.86:0/3157382010 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d3002cba0 con 0x560c5507f0e0 2024-04-21T08:41:57.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.875+0000 7f0d23fff640 1 -- 172.21.15.86:0/3157382010 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0d3c00f040 con 0x560c5507f0e0 2024-04-21T08:41:57.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.875+0000 7f0d4da17740 10 client.17130 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:57.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.875+0000 7f0d4da17740 10 client.17130 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:57.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.875+0000 7f0d4da17740 20 client.17130 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:57.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.875+0000 7f0d4da17740 1 --2- 172.21.15.86:0/3157382010 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x560c5508f0d0 0x560c55091500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:57.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.875+0000 7f0d4da17740 4 client.17130 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:41:57.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.875+0000 7f0d4da17740 1 -- 172.21.15.86:0/3157382010 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560c54fee880 con 0x560c5508f0d0 2024-04-21T08:41:57.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.875+0000 7f0d48bca640 1 --2- 172.21.15.86:0/3157382010 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x560c5508f0d0 0x560c55091500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:57.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.876+0000 7f0d23fff640 10 client.17130 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:57.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.876+0000 7f0d48bca640 1 --2- 172.21.15.86:0/3157382010 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x560c5508f0d0 0x560c55091500 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e3b330b483bc2d60 server_cookie=1556f840ac2d5d55 in_seq=0 out_seq=0 2024-04-21T08:41:57.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.876+0000 7f0d23fff640 1 -- 172.21.15.86:0/3157382010 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560c54fee880 con 0x560c5508f0d0 2024-04-21T08:41:57.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.876+0000 7f0d23fff640 10 client.17130 handle_command_reply: tid=0 2024-04-21T08:41:57.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.877+0000 7f0d4da17740 1 client.17130 shutdown 2024-04-21T08:41:57.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.877+0000 7f0d4da17740 1 -- 172.21.15.86:0/3157382010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c5507f0e0 msgr2=0x560c5507c480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:57.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.877+0000 7f0d4da17740 1 --2- 172.21.15.86:0/3157382010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c5507f0e0 0x560c5507c480 secure :-1 s=READY pgs=988 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c098cf0 tx=0x7f0d3c09c330 comp rx=0 tx=0).stop 2024-04-21T08:41:57.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.877+0000 7f0d4da17740 1 -- 172.21.15.86:0/3157382010 shutdown_connections 2024-04-21T08:41:57.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.877+0000 7f0d4da17740 1 --2- 172.21.15.86:0/3157382010 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x560c5508f0d0 0x560c55091500 crc :-1 s=READY pgs=125 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:57.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.877+0000 7f0d4da17740 1 --2- 172.21.15.86:0/3157382010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c5507f0e0 0x560c5507c480 unknown :-1 s=CLOSED pgs=988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:57.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.877+0000 7f0d23fff640 0 client.17130 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:57.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.877+0000 7f0d4da17740 1 --2- 172.21.15.86:0/3157382010 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560c550768d0 0x560c5507be70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:57.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.877+0000 7f0d4da17740 1 --2- 172.21.15.86:0/3157382010 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c55075b60 0x560c5507eae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:57.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.877+0000 7f0d4da17740 1 -- 172.21.15.86:0/3157382010 >> 172.21.15.86:0/3157382010 conn(0x560c5503cc90 msgr2=0x560c550725a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:57.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.877+0000 7f0d4da17740 1 -- 172.21.15.86:0/3157382010 shutdown_connections 2024-04-21T08:41:57.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.877+0000 7f0d4da17740 1 -- 172.21.15.86:0/3157382010 wait complete. 2024-04-21T08:41:57.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.877+0000 7f0d4da17740 20 client.17130 trim_cache size 0 max 16384 2024-04-21T08:41:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.892+0000 7f0d4da17740 1 Processor -- start 2024-04-21T08:41:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.892+0000 7f0d4da17740 1 -- start start 2024-04-21T08:41:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.892+0000 7f0d4da17740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560c55075b60 0x560c5508acb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.892+0000 7f0d4da17740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c550768d0 0x560c5507e5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.892+0000 7f0d4da17740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c5507f0e0 0x560c5507eae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.892+0000 7f0d4da17740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d441ad040 con 0x560c5507f0e0 2024-04-21T08:41:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.892+0000 7f0d4da17740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0d441aeb80 con 0x560c550768d0 2024-04-21T08:41:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.892+0000 7f0d4da17740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0d441ad710 con 0x560c55075b60 2024-04-21T08:41:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.892+0000 7f0d493cb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c5507f0e0 0x560c5507eae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:57.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.892+0000 7f0d493cb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c5507f0e0 0x560c5507eae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44122/0 (socket says 172.21.15.86:44122) 2024-04-21T08:41:57.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.892+0000 7f0d493cb640 1 -- 172.21.15.86:0/1341687213 learned_addr learned my addr 172.21.15.86:0/1341687213 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:57.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.892+0000 7f0d493cb640 1 -- 172.21.15.86:0/1341687213 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560c55075b60 msgr2=0x560c5508acb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:41:57.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.892+0000 7f0d493cb640 1 --2- 172.21.15.86:0/1341687213 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560c55075b60 0x560c5508acb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:57.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.892+0000 7f0d493cb640 1 -- 172.21.15.86:0/1341687213 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c550768d0 msgr2=0x560c5507e5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:57.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.892+0000 7f0d493cb640 1 --2- 172.21.15.86:0/1341687213 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c550768d0 0x560c5507e5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:57.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.892+0000 7f0d493cb640 1 -- 172.21.15.86:0/1341687213 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560c5506bf30 con 0x560c5507f0e0 2024-04-21T08:41:57.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.892+0000 7f0d493cb640 1 --2- 172.21.15.86:0/1341687213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c5507f0e0 0x560c5507eae0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c0a5020 tx=0x7f0d3c09adf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b3e8ca9421077a1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:57.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.893+0000 7f0d2e7fc640 1 -- 172.21.15.86:0/1341687213 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c0bb070 con 0x560c5507f0e0 2024-04-21T08:41:57.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.893+0000 7f0d2e7fc640 1 -- 172.21.15.86:0/1341687213 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d3c0a5550 con 0x560c5507f0e0 2024-04-21T08:41:57.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.893+0000 7f0d2e7fc640 1 -- 172.21.15.86:0/1341687213 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3c01f040 con 0x560c5507f0e0 2024-04-21T08:41:57.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.893+0000 7f0d4da17740 1 -- 172.21.15.86:0/1341687213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c5507f0e0 msgr2=0x560c5507eae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:57.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.893+0000 7f0d4da17740 1 --2- 172.21.15.86:0/1341687213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c5507f0e0 0x560c5507eae0 secure :-1 s=READY pgs=989 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c0a5020 tx=0x7f0d3c09adf0 comp rx=0 tx=0).stop 2024-04-21T08:41:57.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.893+0000 7f0d4da17740 1 -- 172.21.15.86:0/1341687213 shutdown_connections 2024-04-21T08:41:57.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.893+0000 7f0d4da17740 1 --2- 172.21.15.86:0/1341687213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c5507f0e0 0x560c5507eae0 unknown :-1 s=CLOSED pgs=989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:57.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.893+0000 7f0d4da17740 1 --2- 172.21.15.86:0/1341687213 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c550768d0 0x560c5507e5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:57.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.893+0000 7f0d4da17740 1 --2- 172.21.15.86:0/1341687213 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560c55075b60 0x560c5508acb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:57.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.893+0000 7f0d4da17740 1 -- 172.21.15.86:0/1341687213 >> 172.21.15.86:0/1341687213 conn(0x560c5503cc90 msgr2=0x560c55077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:57.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.893+0000 7f0d4da17740 1 -- 172.21.15.86:0/1341687213 shutdown_connections 2024-04-21T08:41:57.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.893+0000 7f0d4da17740 1 -- 172.21.15.86:0/1341687213 wait complete. 2024-04-21T08:41:57.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.894+0000 7f0d4da17740 1 -- start start 2024-04-21T08:41:57.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.894+0000 7f0d4da17740 1 Processor -- start 2024-04-21T08:41:57.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.894+0000 7f0d4da17740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c55075b60 0x560c550873b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:57.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.894+0000 7f0d4da17740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c550768d0 0x560c55078700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:57.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.894+0000 7f0d4da17740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560c5507f0e0 0x560c55078c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:57.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.894+0000 7f0d4da17740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d441ae4b0 con 0x560c55075b60 2024-04-21T08:41:57.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.894+0000 7f0d4da17740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0d441adde0 con 0x560c550768d0 2024-04-21T08:41:57.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.894+0000 7f0d4da17740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0d441ab000 con 0x560c5507f0e0 2024-04-21T08:41:57.901 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.894+0000 7f0d4da17740 10 client.0 _ll_register_callbacks cb 0x7f0d3c09c6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:41:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.894+0000 7f0d48bca640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c55075b60 0x560c550873b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.894+0000 7f0d48bca640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c55075b60 0x560c550873b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44124/0 (socket says 172.21.15.86:44124) 2024-04-21T08:41:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.894+0000 7f0d48bca640 1 -- 172.21.15.86:0/1185760230 learned_addr learned my addr 172.21.15.86:0/1185760230 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:41:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.894+0000 7f0d43fff640 1 --2- 172.21.15.86:0/1185760230 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c550768d0 0x560c55078700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.894+0000 7f0d493cb640 1 --2- 172.21.15.86:0/1185760230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560c5507f0e0 0x560c55078c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.895+0000 7f0d48bca640 1 -- 172.21.15.86:0/1185760230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560c5507f0e0 msgr2=0x560c55078c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.895+0000 7f0d48bca640 1 --2- 172.21.15.86:0/1185760230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560c5507f0e0 0x560c55078c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.895+0000 7f0d48bca640 1 -- 172.21.15.86:0/1185760230 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c550768d0 msgr2=0x560c55078700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:41:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.895+0000 7f0d48bca640 1 --2- 172.21.15.86:0/1185760230 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c550768d0 0x560c55078700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.895+0000 7f0d48bca640 1 -- 172.21.15.86:0/1185760230 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560c55079180 con 0x560c55075b60 2024-04-21T08:41:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.895+0000 7f0d48bca640 1 --2- 172.21.15.86:0/1185760230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c55075b60 0x560c550873b0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f0d38000c00 tx=0x7f0d38012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b3d9891651be75 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:41:57.902 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.895+0000 7f0d23fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:41:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.895+0000 7f0d23fff640 1 -- 172.21.15.86:0/1185760230 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d3801f070 con 0x560c55075b60 2024-04-21T08:41:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.895+0000 7f0d23fff640 1 -- 172.21.15.86:0/1185760230 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f0d38005020 con 0x560c55075b60 2024-04-21T08:41:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.895+0000 7f0d4da17740 1 -- 172.21.15.86:0/1185760230 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d3c0a5550 con 0x560c55075b60 2024-04-21T08:41:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.895+0000 7f0d23fff640 1 -- 172.21.15.86:0/1185760230 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0d3802a050 con 0x560c55075b60 2024-04-21T08:41:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.895+0000 7f0d4da17740 10 client.17136 fetch_fsmap learned FSMap version 29 2024-04-21T08:41:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.895+0000 7f0d4da17740 1 -- 172.21.15.86:0/1185760230 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560c550f5460 con 0x560c55075b60 2024-04-21T08:41:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.895+0000 7f0d23fff640 1 -- 172.21.15.86:0/1185760230 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d3c0a5550 con 0x560c55075b60 2024-04-21T08:41:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.895+0000 7f0d23fff640 1 -- 172.21.15.86:0/1185760230 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0d38004730 con 0x560c55075b60 2024-04-21T08:41:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.895+0000 7f0d4da17740 10 client.17136 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:41:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.895+0000 7f0d4da17740 10 client.17136 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:41:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.895+0000 7f0d4da17740 20 client.17136 populate_metadata read hostname 'smithi086' 2024-04-21T08:41:57.903 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.895+0000 7f0d4da17740 1 --2- 172.21.15.86:0/1185760230 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x560c550f5bc0 0x560c550f7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:41:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.895+0000 7f0d4da17740 4 client.17136 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:41:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.895+0000 7f0d4da17740 1 -- 172.21.15.86:0/1185760230 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x560c54f7a3d0 con 0x560c550f5bc0 2024-04-21T08:41:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.896+0000 7f0d43fff640 1 --2- 172.21.15.86:0/1185760230 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x560c550f5bc0 0x560c550f7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:41:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.896+0000 7f0d23fff640 10 client.17136 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:57.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.896+0000 7f0d43fff640 1 --2- 172.21.15.86:0/1185760230 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x560c550f5bc0 0x560c550f7fe0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e81583ff98c7a68d server_cookie=dc76cad804269d11 in_seq=0 out_seq=0 2024-04-21T08:41:57.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.908+0000 7f0d23fff640 1 -- 172.21.15.86:0/1185760230 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x560c54f7a3d0 con 0x560c550f5bc0 2024-04-21T08:41:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.908+0000 7f0d23fff640 10 client.17136 handle_command_reply: tid=0 2024-04-21T08:41:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.908+0000 7f0d4da17740 1 client.17136 shutdown 2024-04-21T08:41:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.908+0000 7f0d4da17740 1 -- 172.21.15.86:0/1185760230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c55075b60 msgr2=0x560c550873b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.908+0000 7f0d4da17740 1 --2- 172.21.15.86:0/1185760230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c55075b60 0x560c550873b0 secure :-1 s=READY pgs=990 cs=0 l=1 rev1=1 crypto rx=0x7f0d38000c00 tx=0x7f0d38012820 comp rx=0 tx=0).stop 2024-04-21T08:41:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.908+0000 7f0d4da17740 1 -- 172.21.15.86:0/1185760230 shutdown_connections 2024-04-21T08:41:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.908+0000 7f0d4da17740 1 --2- 172.21.15.86:0/1185760230 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x560c550f5bc0 0x560c550f7fe0 crc :-1 s=READY pgs=126 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.908+0000 7f0d4da17740 1 --2- 172.21.15.86:0/1185760230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560c5507f0e0 0x560c55078c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.908+0000 7f0d4da17740 1 --2- 172.21.15.86:0/1185760230 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560c550768d0 0x560c55078700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.908+0000 7f0d4da17740 1 --2- 172.21.15.86:0/1185760230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560c55075b60 0x560c550873b0 unknown :-1 s=CLOSED pgs=990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.908+0000 7f0d23fff640 0 client.17136 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:41:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.908+0000 7f0d4da17740 1 -- 172.21.15.86:0/1185760230 >> 172.21.15.86:0/1185760230 conn(0x560c5503cc90 msgr2=0x560c5507a6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.908+0000 7f0d4da17740 1 -- 172.21.15.86:0/1185760230 shutdown_connections 2024-04-21T08:41:57.908 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.908+0000 7f0d4da17740 1 -- 172.21.15.86:0/1185760230 wait complete. 2024-04-21T08:41:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.908+0000 7f0d4da17740 20 client.17136 trim_cache size 0 max 16384 2024-04-21T08:41:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:41:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:41:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:41:57.909 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:41:57.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.909+0000 7f0d4ae2e640 1 -- 172.21.15.86:0/2005177641 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0d340a3dc0 msgr2=0x7f0d340a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:57.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.909+0000 7f0d4ae2e640 1 --2- 172.21.15.86:0/2005177641 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0d340a3dc0 0x7f0d340a6280 secure :-1 s=READY pgs=149 cs=0 l=1 rev1=1 crypto rx=0x7f0d4406a570 tx=0x7f0d3002c6c0 comp rx=0 tx=0).stop 2024-04-21T08:41:57.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.909+0000 7f0d4ae2e640 1 -- 172.21.15.86:0/2005177641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d4419be10 msgr2=0x7f0d4419c2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:41:57.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.909+0000 7f0d4ae2e640 1 --2- 172.21.15.86:0/2005177641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d4419be10 0x7f0d4419c2c0 secure :-1 s=READY pgs=986 cs=0 l=1 rev1=1 crypto rx=0x7f0d3c00eaa0 tx=0x7f0d3c00ef60 comp rx=0 tx=0).stop 2024-04-21T08:41:57.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.909+0000 7f0d4ae2e640 1 -- 172.21.15.86:0/2005177641 shutdown_connections 2024-04-21T08:41:57.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.909+0000 7f0d4ae2e640 1 --2- 172.21.15.86:0/2005177641 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0d340a3dc0 0x7f0d340a6280 unknown :-1 s=CLOSED pgs=149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:57.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.909+0000 7f0d4ae2e640 1 --2- 172.21.15.86:0/2005177641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d4419be10 0x7f0d4419c2c0 unknown :-1 s=CLOSED pgs=986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:57.910 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.909+0000 7f0d4ae2e640 1 --2- 172.21.15.86:0/2005177641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d441061b0 0x7f0d44197a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:57.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.909+0000 7f0d4ae2e640 1 --2- 172.21.15.86:0/2005177641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d441010e0 0x7f0d44197550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:41:57.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.909+0000 7f0d4ae2e640 1 -- 172.21.15.86:0/2005177641 >> 172.21.15.86:0/2005177641 conn(0x7f0d440771d0 msgr2=0x7f0d440fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:41:57.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.909+0000 7f0d4ae2e640 1 -- 172.21.15.86:0/2005177641 shutdown_connections 2024-04-21T08:41:57.911 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:41:57.909+0000 7f0d4ae2e640 1 -- 172.21.15.86:0/2005177641 wait complete. 2024-04-21T08:41:58.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:41:58 smithi086 ceph-mon[27939]: pgmap v98: 129 pgs: 129 active+clean; 3.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-04-21T08:41:58.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:41:58 smithi088 ceph-mon[31259]: pgmap v98: 129 pgs: 129 active+clean; 3.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-04-21T08:41:58.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:41:58 smithi179 ceph-mon[31829]: pgmap v98: 129 pgs: 129 active+clean; 3.1 MiB data, 395 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 13 KiB/s wr, 11 op/s 2024-04-21T08:42:00.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:00 smithi086 ceph-mon[27939]: pgmap v99: 129 pgs: 129 active+clean; 3.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-04-21T08:42:00.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:00 smithi088 ceph-mon[31259]: pgmap v99: 129 pgs: 129 active+clean; 3.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-04-21T08:42:00.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:00 smithi179 ceph-mon[31829]: pgmap v99: 129 pgs: 129 active+clean; 3.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 25 KiB/s rd, 23 KiB/s wr, 17 op/s 2024-04-21T08:42:02.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:02 smithi086 ceph-mon[27939]: pgmap v100: 129 pgs: 129 active+clean; 3.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-04-21T08:42:02.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:02 smithi088 ceph-mon[31259]: pgmap v100: 129 pgs: 129 active+clean; 3.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-04-21T08:42:02.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:02 smithi179 ceph-mon[31829]: pgmap v100: 129 pgs: 129 active+clean; 3.2 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 23 KiB/s rd, 21 KiB/s wr, 12 op/s 2024-04-21T08:42:04.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:04 smithi086 ceph-mon[27939]: pgmap v101: 129 pgs: 129 active+clean; 3.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 58 KiB/s wr, 36 op/s 2024-04-21T08:42:04.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:04 smithi088 ceph-mon[31259]: pgmap v101: 129 pgs: 129 active+clean; 3.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 58 KiB/s wr, 36 op/s 2024-04-21T08:42:04.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:04 smithi179 ceph-mon[31829]: pgmap v101: 129 pgs: 129 active+clean; 3.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 58 KiB/s wr, 36 op/s 2024-04-21T08:42:05.472 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:42:05.473 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:42:05.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.585+0000 7f1eca69a640 1 -- 172.21.15.86:0/385478057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ec4106150 msgr2=0x7f1ec410dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:05.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.585+0000 7f1eca69a640 1 --2- 172.21.15.86:0/385478057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ec4106150 0x7f1ec410dd20 secure :-1 s=READY pgs=991 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc009ca0 tx=0x7f1ebc02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.585+0000 7f1eca69a640 1 -- 172.21.15.86:0/385478057 shutdown_connections 2024-04-21T08:42:05.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.585+0000 7f1eca69a640 1 --2- 172.21.15.86:0/385478057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ec4106150 0x7f1ec410dd20 unknown :-1 s=CLOSED pgs=991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.585+0000 7f1eca69a640 1 --2- 172.21.15.86:0/385478057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ec4101a30 0x7f1ec4105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.585+0000 7f1eca69a640 1 --2- 172.21.15.86:0/385478057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ec4101080 0x7f1ec4101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.585+0000 7f1eca69a640 1 -- 172.21.15.86:0/385478057 >> 172.21.15.86:0/385478057 conn(0x7f1ec4078000 msgr2=0x7f1ec4078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:05.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.585+0000 7f1eca69a640 1 -- 172.21.15.86:0/385478057 shutdown_connections 2024-04-21T08:42:05.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.585+0000 7f1eca69a640 1 -- 172.21.15.86:0/385478057 wait complete. 2024-04-21T08:42:05.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.585+0000 7f1eca69a640 1 Processor -- start 2024-04-21T08:42:05.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.585+0000 7f1eca69a640 1 -- start start 2024-04-21T08:42:05.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.586+0000 7f1eca69a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ec4101080 0x7f1ec4199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:05.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.586+0000 7f1eca69a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ec4106150 0x7f1ec4199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:05.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.586+0000 7f1eca69a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ec4199fa0 0x7f1ec41a45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:05.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.586+0000 7f1eca69a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec41110a0 con 0x7f1ec4101080 2024-04-21T08:42:05.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.586+0000 7f1eca69a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec4110f20 con 0x7f1ec4199fa0 2024-04-21T08:42:05.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.586+0000 7f1eca69a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec4111220 con 0x7f1ec4106150 2024-04-21T08:42:05.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.586+0000 7f1ec3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ec4101080 0x7f1ec4199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:05.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.586+0000 7f1ec3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ec4101080 0x7f1ec4199520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40498/0 (socket says 172.21.15.86:40498) 2024-04-21T08:42:05.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.586+0000 7f1ec3fff640 1 -- 172.21.15.86:0/4085688921 learned_addr learned my addr 172.21.15.86:0/4085688921 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:05.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.586+0000 7f1ec8c37640 1 --2- 172.21.15.86:0/4085688921 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ec4199fa0 0x7f1ec41a45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:05.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.586+0000 7f1ec37fe640 1 --2- 172.21.15.86:0/4085688921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ec4106150 0x7f1ec4199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:05.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.586+0000 7f1ec3fff640 1 -- 172.21.15.86:0/4085688921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ec4106150 msgr2=0x7f1ec4199a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:05.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.586+0000 7f1ec3fff640 1 --2- 172.21.15.86:0/4085688921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ec4106150 0x7f1ec4199a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.586+0000 7f1ec3fff640 1 -- 172.21.15.86:0/4085688921 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ec4199fa0 msgr2=0x7f1ec41a45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:05.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.586+0000 7f1ec3fff640 1 --2- 172.21.15.86:0/4085688921 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ec4199fa0 0x7f1ec41a45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.586+0000 7f1ec3fff640 1 -- 172.21.15.86:0/4085688921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1ec41a4d40 con 0x7f1ec4101080 2024-04-21T08:42:05.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.586+0000 7f1ec3fff640 1 --2- 172.21.15.86:0/4085688921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ec4101080 0x7f1ec4199520 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f1eb400d950 tx=0x7f1eb400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:05.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.586+0000 7f1ec17fa640 1 -- 172.21.15.86:0/4085688921 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4017030 con 0x7f1ec4101080 2024-04-21T08:42:05.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.586+0000 7f1ec17fa640 1 -- 172.21.15.86:0/4085688921 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1eb4004020 con 0x7f1ec4101080 2024-04-21T08:42:05.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.586+0000 7f1eca69a640 1 -- 172.21.15.86:0/4085688921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1ec41a4fd0 con 0x7f1ec4101080 2024-04-21T08:42:05.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.586+0000 7f1ec17fa640 1 -- 172.21.15.86:0/4085688921 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb40047a0 con 0x7f1ec4101080 2024-04-21T08:42:05.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.586+0000 7f1eca69a640 1 -- 172.21.15.86:0/4085688921 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1ec41a54f0 con 0x7f1ec4101080 2024-04-21T08:42:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.589+0000 7f1ec17fa640 1 -- 172.21.15.86:0/4085688921 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1eb4009a30 con 0x7f1ec4101080 2024-04-21T08:42:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.589+0000 7f1ecd36b740 1 Processor -- start 2024-04-21T08:42:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.589+0000 7f1ecd36b740 1 -- start start 2024-04-21T08:42:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.589+0000 7f1ec17fa640 1 --2- 172.21.15.86:0/4085688921 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1eac0a3d50 0x7f1eac0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.589+0000 7f1ecd36b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55956bed8580 0x55956bed8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.589+0000 7f1ecd36b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55956bed8ea0 0x55956bee1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.589+0000 7f1ecd36b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956bee1a80 0x55956bee3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.589+0000 7f1ecd36b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55956be3d1e0 con 0x55956bee1a80 2024-04-21T08:42:05.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.589+0000 7f1ecd36b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55956bdda390 con 0x55956bed8580 2024-04-21T08:42:05.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.589+0000 7f1ecd36b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55956be391a0 con 0x55956bed8ea0 2024-04-21T08:42:05.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.589+0000 7f1ec8c37640 1 --2- 172.21.15.86:0/4085688921 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1eac0a3d50 0x7f1eac0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:05.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.590+0000 7f1ec17fa640 1 -- 172.21.15.86:0/4085688921 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f1eb40d0450 con 0x7f1ec4101080 2024-04-21T08:42:05.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.590+0000 7f1ec37fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55956bed8ea0 0x55956bee1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:05.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.590+0000 7f1ec8c37640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956bee1a80 0x55956bee3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:05.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.590+0000 7f1ec37fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55956bed8ea0 0x55956bee1540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.86:33030/0 (socket says 172.21.15.86:33030) 2024-04-21T08:42:05.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.590+0000 7f1ec37fe640 1 -- 172.21.15.86:0/3176859277 learned_addr learned my addr 172.21.15.86:0/3176859277 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:05.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.590+0000 7f1ec3fff640 1 --2- 172.21.15.86:0/3176859277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55956bed8580 0x55956bed8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:05.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.590+0000 7f1ec37fe640 1 -- 172.21.15.86:0/3176859277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55956bed8580 msgr2=0x55956bed8960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:05.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.590+0000 7f1ec37fe640 1 --2- 172.21.15.86:0/3176859277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55956bed8580 0x55956bed8960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.590+0000 7f1ec37fe640 1 -- 172.21.15.86:0/3176859277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956bee1a80 msgr2=0x55956bee3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:05.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.590+0000 7f1ec37fe640 1 --2- 172.21.15.86:0/3176859277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956bee1a80 0x55956bee3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.590+0000 7f1ec37fe640 1 -- 172.21.15.86:0/3176859277 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55956bc197d0 con 0x55956bed8ea0 2024-04-21T08:42:05.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.590+0000 7f1ec8c37640 1 --2- 172.21.15.86:0/4085688921 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1eac0a3d50 0x7f1eac0a6210 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc004af0 tx=0x7f1ebc009960 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:05.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.590+0000 7f1ec37fe640 1 --2- 172.21.15.86:0/3176859277 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55956bed8ea0 0x55956bee1540 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8009c20 tx=0x7f1eb8011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=bd12eb1055c43367 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:05.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.590+0000 7f1eaa7fc640 1 -- 172.21.15.86:0/3176859277 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8004050 con 0x55956bed8ea0 2024-04-21T08:42:05.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.590+0000 7f1eaa7fc640 1 -- 172.21.15.86:0/3176859277 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1eb80054d0 con 0x55956bed8ea0 2024-04-21T08:42:05.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.590+0000 7f1eaa7fc640 1 -- 172.21.15.86:0/3176859277 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb8004bf0 con 0x55956bed8ea0 2024-04-21T08:42:05.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.591+0000 7f1ecd36b740 1 -- 172.21.15.86:0/3176859277 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55956bed8ea0 msgr2=0x55956bee1540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:05.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.591+0000 7f1ecd36b740 1 --2- 172.21.15.86:0/3176859277 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55956bed8ea0 0x55956bee1540 secure :-1 s=READY pgs=67 cs=0 l=1 rev1=1 crypto rx=0x7f1eb8009c20 tx=0x7f1eb8011810 comp rx=0 tx=0).stop 2024-04-21T08:42:05.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.591+0000 7f1ecd36b740 1 -- 172.21.15.86:0/3176859277 shutdown_connections 2024-04-21T08:42:05.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.591+0000 7f1ecd36b740 1 --2- 172.21.15.86:0/3176859277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956bee1a80 0x55956bee3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.591+0000 7f1ecd36b740 1 --2- 172.21.15.86:0/3176859277 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55956bed8ea0 0x55956bee1540 unknown :-1 s=CLOSED pgs=67 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.591+0000 7f1ecd36b740 1 --2- 172.21.15.86:0/3176859277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55956bed8580 0x55956bed8960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.591+0000 7f1ecd36b740 1 -- 172.21.15.86:0/3176859277 >> 172.21.15.86:0/3176859277 conn(0x55956bed2cb0 msgr2=0x55956bed3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:05.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.591+0000 7f1ecd36b740 1 -- 172.21.15.86:0/3176859277 shutdown_connections 2024-04-21T08:42:05.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.591+0000 7f1ecd36b740 1 -- 172.21.15.86:0/3176859277 wait complete. 2024-04-21T08:42:05.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.591+0000 7f1ecd36b740 1 -- start start 2024-04-21T08:42:05.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.592+0000 7f1ecd36b740 1 Processor -- start 2024-04-21T08:42:05.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.592+0000 7f1ecd36b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55956bed8580 0x55956bedea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:05.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.592+0000 7f1ecd36b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55956bed8ea0 0x55956bedefc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:05.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.592+0000 7f1ecd36b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956bee1a80 0x55956bedf500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:05.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.592+0000 7f1ecd36b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55956bee4870 con 0x55956bee1a80 2024-04-21T08:42:05.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.592+0000 7f1ecd36b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55956bdf62f0 con 0x55956bed8580 2024-04-21T08:42:05.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.592+0000 7f1ecd36b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55956bee49f0 con 0x55956bed8ea0 2024-04-21T08:42:05.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.592+0000 7f1ecd36b740 10 client.0 _ll_register_callbacks cb 0x55956be03580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:05.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.592+0000 7f1ec8c37640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956bee1a80 0x55956bedf500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:05.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.592+0000 7f1ec8c37640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956bee1a80 0x55956bedf500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40526/0 (socket says 172.21.15.86:40526) 2024-04-21T08:42:05.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.592+0000 7f1ec8c37640 1 -- 172.21.15.86:0/3218222098 learned_addr learned my addr 172.21.15.86:0/3218222098 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:05.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.592+0000 7f1ec3fff640 1 --2- 172.21.15.86:0/3218222098 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55956bed8580 0x55956bedea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:05.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.592+0000 7f1ec37fe640 1 --2- 172.21.15.86:0/3218222098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55956bed8ea0 0x55956bedefc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:05.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.592+0000 7f1ec8c37640 1 -- 172.21.15.86:0/3218222098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55956bed8ea0 msgr2=0x55956bedefc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:05.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.592+0000 7f1ec8c37640 1 --2- 172.21.15.86:0/3218222098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55956bed8ea0 0x55956bedefc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.592+0000 7f1ec8c37640 1 -- 172.21.15.86:0/3218222098 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55956bed8580 msgr2=0x55956bedea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:05.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.592+0000 7f1ec8c37640 1 --2- 172.21.15.86:0/3218222098 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55956bed8580 0x55956bedea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.592+0000 7f1ec8c37640 1 -- 172.21.15.86:0/3218222098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55956bb4f3f0 con 0x55956bee1a80 2024-04-21T08:42:05.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.592+0000 7f1ec37fe640 1 --2- 172.21.15.86:0/3218222098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55956bed8ea0 0x55956bedefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:05.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.593+0000 7f1ec8c37640 1 --2- 172.21.15.86:0/3218222098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956bee1a80 0x55956bedf500 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc009c60 tx=0x7f1ebc005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6de6ed18f03f56a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:05.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.593+0000 7f1ea37fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:05.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.593+0000 7f1ea37fe640 1 -- 172.21.15.86:0/3218222098 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc051060 con 0x55956bee1a80 2024-04-21T08:42:05.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.593+0000 7f1ea37fe640 1 -- 172.21.15.86:0/3218222098 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f1ebc004040 con 0x55956bee1a80 2024-04-21T08:42:05.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.593+0000 7f1ecd36b740 1 -- 172.21.15.86:0/3218222098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1eb80054d0 con 0x55956bee1a80 2024-04-21T08:42:05.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.593+0000 7f1ea37fe640 1 -- 172.21.15.86:0/3218222098 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1ebc058050 con 0x55956bee1a80 2024-04-21T08:42:05.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.593+0000 7f1ecd36b740 10 client.17148 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:05.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.593+0000 7f1ecd36b740 1 -- 172.21.15.86:0/3218222098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55956bb0f0f0 con 0x55956bee1a80 2024-04-21T08:42:05.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.593+0000 7f1ea37fe640 1 -- 172.21.15.86:0/3218222098 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1eb80054d0 con 0x55956bee1a80 2024-04-21T08:42:05.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.593+0000 7f1ea37fe640 1 -- 172.21.15.86:0/3218222098 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1ebc04b070 con 0x55956bee1a80 2024-04-21T08:42:05.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.593+0000 7f1ecd36b740 10 client.17148 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:05.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.593+0000 7f1ecd36b740 10 client.17148 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:05.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.593+0000 7f1ecd36b740 20 client.17148 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:05.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.593+0000 7f1ecd36b740 1 --2- 172.21.15.86:0/3218222098 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55956bef1a40 0x55956bef3ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:05.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.593+0000 7f1ecd36b740 4 client.17148 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:05.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.593+0000 7f1ecd36b740 1 -- 172.21.15.86:0/3218222098 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55956be4ac00 con 0x55956bef1a40 2024-04-21T08:42:05.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.594+0000 7f1ec3fff640 1 --2- 172.21.15.86:0/3218222098 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55956bef1a40 0x55956bef3ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:05.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.594+0000 7f1ea37fe640 10 client.17148 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:05.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.594+0000 7f1ec3fff640 1 --2- 172.21.15.86:0/3218222098 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55956bef1a40 0x55956bef3ea0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f4aeb8ae23098ee server_cookie=9d18bc8691a200b6 in_seq=0 out_seq=0 2024-04-21T08:42:05.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.595+0000 7f1ea37fe640 1 -- 172.21.15.86:0/3218222098 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55956be4ac00 con 0x55956bef1a40 2024-04-21T08:42:05.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.595+0000 7f1ea37fe640 10 client.17148 handle_command_reply: tid=0 2024-04-21T08:42:05.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.595+0000 7f1ecd36b740 1 client.17148 shutdown 2024-04-21T08:42:05.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.595+0000 7f1ecd36b740 1 -- 172.21.15.86:0/3218222098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956bee1a80 msgr2=0x55956bedf500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:05.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.595+0000 7f1ecd36b740 1 --2- 172.21.15.86:0/3218222098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956bee1a80 0x55956bedf500 secure :-1 s=READY pgs=993 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc009c60 tx=0x7f1ebc005c50 comp rx=0 tx=0).stop 2024-04-21T08:42:05.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.595+0000 7f1ecd36b740 1 -- 172.21.15.86:0/3218222098 shutdown_connections 2024-04-21T08:42:05.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.595+0000 7f1ecd36b740 1 --2- 172.21.15.86:0/3218222098 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55956bef1a40 0x55956bef3ea0 crc :-1 s=READY pgs=127 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.595+0000 7f1ecd36b740 1 --2- 172.21.15.86:0/3218222098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956bee1a80 0x55956bedf500 unknown :-1 s=CLOSED pgs=993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.595+0000 7f1ea37fe640 0 client.17148 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:05.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.595+0000 7f1ecd36b740 1 --2- 172.21.15.86:0/3218222098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55956bed8ea0 0x55956bedefc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.595+0000 7f1ecd36b740 1 --2- 172.21.15.86:0/3218222098 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55956bed8580 0x55956bedea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.595+0000 7f1ecd36b740 1 -- 172.21.15.86:0/3218222098 >> 172.21.15.86:0/3218222098 conn(0x55956bed2cb0 msgr2=0x55956bed4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:05.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.595+0000 7f1ecd36b740 1 -- 172.21.15.86:0/3218222098 shutdown_connections 2024-04-21T08:42:05.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.595+0000 7f1ecd36b740 1 -- 172.21.15.86:0/3218222098 wait complete. 2024-04-21T08:42:05.603 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.595+0000 7f1ecd36b740 20 client.17148 trim_cache size 0 max 16384 2024-04-21T08:42:05.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.612+0000 7f1ecd36b740 1 Processor -- start 2024-04-21T08:42:05.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.612+0000 7f1ecd36b740 1 -- start start 2024-04-21T08:42:05.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.612+0000 7f1ecd36b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956bed8580 0x55956beed730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:05.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.612+0000 7f1ecd36b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55956bed8ea0 0x55956bed1bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:05.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.612+0000 7f1ecd36b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55956bee1a80 0x55956bef9770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:05.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.612+0000 7f1ecd36b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec41af310 con 0x55956bed8580 2024-04-21T08:42:05.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.612+0000 7f1ecd36b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec41b0e50 con 0x55956bee1a80 2024-04-21T08:42:05.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.612+0000 7f1ecd36b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec41af9e0 con 0x55956bed8ea0 2024-04-21T08:42:05.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.613+0000 7f1ec3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956bed8580 0x55956beed730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:05.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.613+0000 7f1ec3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956bed8580 0x55956beed730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40534/0 (socket says 172.21.15.86:40534) 2024-04-21T08:42:05.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.613+0000 7f1ec3fff640 1 -- 172.21.15.86:0/1596128289 learned_addr learned my addr 172.21.15.86:0/1596128289 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:05.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.613+0000 7f1ec37fe640 1 --2- 172.21.15.86:0/1596128289 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55956bed8ea0 0x55956bed1bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:05.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.613+0000 7f1ec3fff640 1 -- 172.21.15.86:0/1596128289 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55956bed8ea0 msgr2=0x55956bed1bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:05.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.613+0000 7f1ec3fff640 1 --2- 172.21.15.86:0/1596128289 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55956bed8ea0 0x55956bed1bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.613+0000 7f1ec8c37640 1 --2- 172.21.15.86:0/1596128289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55956bee1a80 0x55956bef9770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:05.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.613+0000 7f1ec3fff640 1 -- 172.21.15.86:0/1596128289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55956bee1a80 msgr2=0x55956bef9770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:05.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.613+0000 7f1ec3fff640 1 --2- 172.21.15.86:0/1596128289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55956bee1a80 0x55956bef9770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.613+0000 7f1ec3fff640 1 -- 172.21.15.86:0/1596128289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55956bee3e70 con 0x55956bed8580 2024-04-21T08:42:05.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.613+0000 7f1ec8c37640 1 --2- 172.21.15.86:0/1596128289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55956bee1a80 0x55956bef9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:05.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.613+0000 7f1ec3fff640 1 --2- 172.21.15.86:0/1596128289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956bed8580 0x55956beed730 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f1eb409be20 tx=0x7f1eb40cf040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd3be07b77317935 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:05.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.613+0000 7f1ea9ffb640 1 -- 172.21.15.86:0/1596128289 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb4065f40 con 0x55956bed8580 2024-04-21T08:42:05.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.613+0000 7f1ea9ffb640 1 -- 172.21.15.86:0/1596128289 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1eb40056a0 con 0x55956bed8580 2024-04-21T08:42:05.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.613+0000 7f1ea9ffb640 1 -- 172.21.15.86:0/1596128289 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1eb40a4d90 con 0x55956bed8580 2024-04-21T08:42:05.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.614+0000 7f1ecd36b740 1 -- 172.21.15.86:0/1596128289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956bed8580 msgr2=0x55956beed730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:05.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.614+0000 7f1ecd36b740 1 --2- 172.21.15.86:0/1596128289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956bed8580 0x55956beed730 secure :-1 s=READY pgs=994 cs=0 l=1 rev1=1 crypto rx=0x7f1eb409be20 tx=0x7f1eb40cf040 comp rx=0 tx=0).stop 2024-04-21T08:42:05.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.614+0000 7f1ecd36b740 1 -- 172.21.15.86:0/1596128289 shutdown_connections 2024-04-21T08:42:05.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.614+0000 7f1ecd36b740 1 --2- 172.21.15.86:0/1596128289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55956bee1a80 0x55956bef9770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.614+0000 7f1ecd36b740 1 --2- 172.21.15.86:0/1596128289 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55956bed8ea0 0x55956bed1bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.614+0000 7f1ecd36b740 1 --2- 172.21.15.86:0/1596128289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956bed8580 0x55956beed730 unknown :-1 s=CLOSED pgs=994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.614+0000 7f1ecd36b740 1 -- 172.21.15.86:0/1596128289 >> 172.21.15.86:0/1596128289 conn(0x55956bed2cb0 msgr2=0x55956bed14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:05.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.614+0000 7f1ecd36b740 1 -- 172.21.15.86:0/1596128289 shutdown_connections 2024-04-21T08:42:05.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.614+0000 7f1ecd36b740 1 -- 172.21.15.86:0/1596128289 wait complete. 2024-04-21T08:42:05.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.614+0000 7f1ecd36b740 1 -- start start 2024-04-21T08:42:05.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.614+0000 7f1ecd36b740 1 Processor -- start 2024-04-21T08:42:05.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.615+0000 7f1ecd36b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55956bed8580 0x55956bedbaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:05.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.615+0000 7f1ecd36b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55956bed8ea0 0x55956bedbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:05.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.615+0000 7f1ecd36b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956bee1a80 0x55956bf56fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:05.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.615+0000 7f1ecd36b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec41b0780 con 0x55956bee1a80 2024-04-21T08:42:05.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.615+0000 7f1ecd36b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec41b00b0 con 0x55956bed8580 2024-04-21T08:42:05.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.615+0000 7f1ecd36b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1ec41ad2d0 con 0x55956bed8ea0 2024-04-21T08:42:05.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.615+0000 7f1ecd36b740 10 client.0 _ll_register_callbacks cb 0x7f1ebc009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:05.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.615+0000 7f1ec8c37640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956bee1a80 0x55956bf56fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:05.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.615+0000 7f1ec8c37640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956bee1a80 0x55956bf56fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40538/0 (socket says 172.21.15.86:40538) 2024-04-21T08:42:05.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.615+0000 7f1ec8c37640 1 -- 172.21.15.86:0/3357035488 learned_addr learned my addr 172.21.15.86:0/3357035488 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:05.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.615+0000 7f1ec37fe640 1 --2- 172.21.15.86:0/3357035488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55956bed8ea0 0x55956bedbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:05.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.615+0000 7f1ec3fff640 1 --2- 172.21.15.86:0/3357035488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55956bed8580 0x55956bedbaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:05.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.615+0000 7f1ec8c37640 1 -- 172.21.15.86:0/3357035488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55956bed8ea0 msgr2=0x55956bedbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:05.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.615+0000 7f1ec8c37640 1 --2- 172.21.15.86:0/3357035488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55956bed8ea0 0x55956bedbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.615+0000 7f1ec8c37640 1 -- 172.21.15.86:0/3357035488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55956bed8580 msgr2=0x55956bedbaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:05.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.615+0000 7f1ec8c37640 1 --2- 172.21.15.86:0/3357035488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55956bed8580 0x55956bedbaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.615+0000 7f1ec8c37640 1 -- 172.21.15.86:0/3357035488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55956bf57520 con 0x55956bee1a80 2024-04-21T08:42:05.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.615+0000 7f1ec8c37640 1 --2- 172.21.15.86:0/3357035488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956bee1a80 0x55956bf56fe0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc005d30 tx=0x7f1ebc009360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fb2fdaec18e43c1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:05.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.615+0000 7f1ea37fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:05.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.615+0000 7f1ea37fe640 1 -- 172.21.15.86:0/3357035488 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ebc051040 con 0x55956bee1a80 2024-04-21T08:42:05.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.615+0000 7f1ea37fe640 1 -- 172.21.15.86:0/3357035488 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f1ebc004f50 con 0x55956bee1a80 2024-04-21T08:42:05.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.615+0000 7f1ecd36b740 1 -- 172.21.15.86:0/3357035488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1eb40056a0 con 0x55956bee1a80 2024-04-21T08:42:05.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.615+0000 7f1ea37fe640 1 -- 172.21.15.86:0/3357035488 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1ebc039040 con 0x55956bee1a80 2024-04-21T08:42:05.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.615+0000 7f1ecd36b740 10 client.17154 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:05.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.615+0000 7f1ecd36b740 1 -- 172.21.15.86:0/3357035488 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55956bf585a0 con 0x55956bee1a80 2024-04-21T08:42:05.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.616+0000 7f1ea37fe640 1 -- 172.21.15.86:0/3357035488 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1eb40056a0 con 0x55956bee1a80 2024-04-21T08:42:05.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.616+0000 7f1ea37fe640 1 -- 172.21.15.86:0/3357035488 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1ebc04b040 con 0x55956bee1a80 2024-04-21T08:42:05.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.616+0000 7f1ecd36b740 10 client.17154 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:05.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.616+0000 7f1ecd36b740 10 client.17154 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:05.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.616+0000 7f1ecd36b740 20 client.17154 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:05.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.616+0000 7f1ecd36b740 1 --2- 172.21.15.86:0/3357035488 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55956bf58d20 0x55956bf5b140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:05.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.616+0000 7f1ecd36b740 4 client.17154 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:42:05.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.616+0000 7f1ecd36b740 1 -- 172.21.15.86:0/3357035488 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55956be44a70 con 0x55956bf58d20 2024-04-21T08:42:05.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.616+0000 7f1ec3fff640 1 --2- 172.21.15.86:0/3357035488 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55956bf58d20 0x55956bf5b140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:05.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.616+0000 7f1ea37fe640 10 client.17154 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:05.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.616+0000 7f1ec3fff640 1 --2- 172.21.15.86:0/3357035488 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55956bf58d20 0x55956bf5b140 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=79868c09fa8f5dca server_cookie=765a5a4cf25542e7 in_seq=0 out_seq=0 2024-04-21T08:42:05.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.617+0000 7f1ea37fe640 1 -- 172.21.15.86:0/3357035488 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55956be44a70 con 0x55956bf58d20 2024-04-21T08:42:05.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.617+0000 7f1ea37fe640 10 client.17154 handle_command_reply: tid=0 2024-04-21T08:42:05.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.617+0000 7f1ecd36b740 1 client.17154 shutdown 2024-04-21T08:42:05.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.617+0000 7f1ecd36b740 1 -- 172.21.15.86:0/3357035488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956bee1a80 msgr2=0x55956bf56fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:05.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.617+0000 7f1ecd36b740 1 --2- 172.21.15.86:0/3357035488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956bee1a80 0x55956bf56fe0 secure :-1 s=READY pgs=995 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc005d30 tx=0x7f1ebc009360 comp rx=0 tx=0).stop 2024-04-21T08:42:05.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.617+0000 7f1ecd36b740 1 -- 172.21.15.86:0/3357035488 shutdown_connections 2024-04-21T08:42:05.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.617+0000 7f1ecd36b740 1 --2- 172.21.15.86:0/3357035488 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55956bf58d20 0x55956bf5b140 crc :-1 s=READY pgs=128 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.617+0000 7f1ecd36b740 1 --2- 172.21.15.86:0/3357035488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55956bee1a80 0x55956bf56fe0 unknown :-1 s=CLOSED pgs=995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.617+0000 7f1ea37fe640 0 client.17154 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:05.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.617+0000 7f1ecd36b740 1 --2- 172.21.15.86:0/3357035488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55956bed8ea0 0x55956bedbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.617+0000 7f1ecd36b740 1 --2- 172.21.15.86:0/3357035488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55956bed8580 0x55956bedbaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.617+0000 7f1ecd36b740 1 -- 172.21.15.86:0/3357035488 >> 172.21.15.86:0/3357035488 conn(0x55956bed2cb0 msgr2=0x55956bed14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:05.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.617+0000 7f1ecd36b740 1 -- 172.21.15.86:0/3357035488 shutdown_connections 2024-04-21T08:42:05.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.617+0000 7f1ecd36b740 1 -- 172.21.15.86:0/3357035488 wait complete. 2024-04-21T08:42:05.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.617+0000 7f1ecd36b740 20 client.17154 trim_cache size 0 max 16384 2024-04-21T08:42:05.624 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:42:05.624 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-21T08:42:05.624 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-21T08:42:05.624 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:42:05.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.617+0000 7f1eca69a640 1 -- 172.21.15.86:0/4085688921 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1eac0a3d50 msgr2=0x7f1eac0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:05.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.617+0000 7f1eca69a640 1 --2- 172.21.15.86:0/4085688921 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1eac0a3d50 0x7f1eac0a6210 secure :-1 s=READY pgs=150 cs=0 l=1 rev1=1 crypto rx=0x7f1ebc004af0 tx=0x7f1ebc009960 comp rx=0 tx=0).stop 2024-04-21T08:42:05.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.617+0000 7f1eca69a640 1 -- 172.21.15.86:0/4085688921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ec4101080 msgr2=0x7f1ec4199520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:05.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.617+0000 7f1eca69a640 1 --2- 172.21.15.86:0/4085688921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ec4101080 0x7f1ec4199520 secure :-1 s=READY pgs=992 cs=0 l=1 rev1=1 crypto rx=0x7f1eb400d950 tx=0x7f1eb400de10 comp rx=0 tx=0).stop 2024-04-21T08:42:05.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.618+0000 7f1eca69a640 1 -- 172.21.15.86:0/4085688921 shutdown_connections 2024-04-21T08:42:05.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.618+0000 7f1eca69a640 1 --2- 172.21.15.86:0/4085688921 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1eac0a3d50 0x7f1eac0a6210 unknown :-1 s=CLOSED pgs=150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.618+0000 7f1eca69a640 1 --2- 172.21.15.86:0/4085688921 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1ec4199fa0 0x7f1ec41a45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.618+0000 7f1eca69a640 1 --2- 172.21.15.86:0/4085688921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1ec4106150 0x7f1ec4199a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.618+0000 7f1eca69a640 1 --2- 172.21.15.86:0/4085688921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1ec4101080 0x7f1ec4199520 unknown :-1 s=CLOSED pgs=992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.618+0000 7f1eca69a640 1 -- 172.21.15.86:0/4085688921 >> 172.21.15.86:0/4085688921 conn(0x7f1ec4078000 msgr2=0x7f1ec40fef20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:05.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.618+0000 7f1eca69a640 1 -- 172.21.15.86:0/4085688921 shutdown_connections 2024-04-21T08:42:05.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.618+0000 7f1eca69a640 1 -- 172.21.15.86:0/4085688921 wait complete. 2024-04-21T08:42:05.628 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:42:05.629 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:42:05.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.727+0000 7fc5c3e25640 1 -- 172.21.15.86:0/3959198744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5bc102b50 msgr2=0x7fc5bc106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:05.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.727+0000 7fc5c3e25640 1 --2- 172.21.15.86:0/3959198744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5bc102b50 0x7fc5bc106b90 secure :-1 s=READY pgs=996 cs=0 l=1 rev1=1 crypto rx=0x7fc5ac009900 tx=0x7fc5ac02bc30 comp rx=0 tx=0).stop 2024-04-21T08:42:05.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.727+0000 7fc5c3e25640 1 -- 172.21.15.86:0/3959198744 shutdown_connections 2024-04-21T08:42:05.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.727+0000 7fc5c3e25640 1 --2- 172.21.15.86:0/3959198744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc5bc107270 0x7fc5bc10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.727+0000 7fc5c3e25640 1 --2- 172.21.15.86:0/3959198744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5bc102b50 0x7fc5bc106b90 unknown :-1 s=CLOSED pgs=996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.727+0000 7fc5c3e25640 1 --2- 172.21.15.86:0/3959198744 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5bc1021a0 0x7fc5bc102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.727+0000 7fc5c3e25640 1 -- 172.21.15.86:0/3959198744 >> 172.21.15.86:0/3959198744 conn(0x7fc5bc0793a0 msgr2=0x7fc5bc0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:05.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.727+0000 7fc5c3e25640 1 -- 172.21.15.86:0/3959198744 shutdown_connections 2024-04-21T08:42:05.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.727+0000 7fc5c3e25640 1 -- 172.21.15.86:0/3959198744 wait complete. 2024-04-21T08:42:05.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.727+0000 7fc5c3e25640 1 Processor -- start 2024-04-21T08:42:05.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.727+0000 7fc5c3e25640 1 -- start start 2024-04-21T08:42:05.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.727+0000 7fc5c3e25640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc5bc1021a0 0x7fc5bc198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:05.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.727+0000 7fc5c3e25640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5bc107270 0x7fc5bc198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:05.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.727+0000 7fc5c3e25640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5bc19cfd0 0x7fc5bc19d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:05.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.728+0000 7fc5c3e25640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc5bc112190 con 0x7fc5bc19cfd0 2024-04-21T08:42:05.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.728+0000 7fc5c3e25640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc5bc112010 con 0x7fc5bc107270 2024-04-21T08:42:05.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.728+0000 7fc5c3e25640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc5bc112310 con 0x7fc5bc1021a0 2024-04-21T08:42:05.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.728+0000 7fc5c23c2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5bc19cfd0 0x7fc5bc19d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:05.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.728+0000 7fc5c23c2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5bc19cfd0 0x7fc5bc19d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40566/0 (socket says 172.21.15.86:40566) 2024-04-21T08:42:05.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.728+0000 7fc5c23c2640 1 -- 172.21.15.86:0/2754431965 learned_addr learned my addr 172.21.15.86:0/2754431965 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:05.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.728+0000 7fc5c13c0640 1 --2- 172.21.15.86:0/2754431965 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5bc107270 0x7fc5bc198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:05.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.728+0000 7fc5c1bc1640 1 --2- 172.21.15.86:0/2754431965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc5bc1021a0 0x7fc5bc198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:05.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.728+0000 7fc5c23c2640 1 -- 172.21.15.86:0/2754431965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc5bc1021a0 msgr2=0x7fc5bc198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:05.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.728+0000 7fc5c23c2640 1 --2- 172.21.15.86:0/2754431965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc5bc1021a0 0x7fc5bc198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.728+0000 7fc5c23c2640 1 -- 172.21.15.86:0/2754431965 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5bc107270 msgr2=0x7fc5bc198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:05.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.728+0000 7fc5c23c2640 1 --2- 172.21.15.86:0/2754431965 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5bc107270 0x7fc5bc198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.728+0000 7fc5c23c2640 1 -- 172.21.15.86:0/2754431965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc5bc19dac0 con 0x7fc5bc19cfd0 2024-04-21T08:42:05.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.728+0000 7fc5c23c2640 1 --2- 172.21.15.86:0/2754431965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5bc19cfd0 0x7fc5bc19d480 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fc5b800d950 tx=0x7fc5b800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:05.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.728+0000 7fc5b2ffd640 1 -- 172.21.15.86:0/2754431965 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b8017030 con 0x7fc5bc19cfd0 2024-04-21T08:42:05.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.728+0000 7fc5c3e25640 1 -- 172.21.15.86:0/2754431965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc5bc19ddb0 con 0x7fc5bc19cfd0 2024-04-21T08:42:05.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.728+0000 7fc5b2ffd640 1 -- 172.21.15.86:0/2754431965 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc5b8004020 con 0x7fc5bc19cfd0 2024-04-21T08:42:05.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.728+0000 7fc5b2ffd640 1 -- 172.21.15.86:0/2754431965 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc5b8003b50 con 0x7fc5bc19cfd0 2024-04-21T08:42:05.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.728+0000 7fc5c3e25640 1 -- 172.21.15.86:0/2754431965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc5bc1a5520 con 0x7fc5bc19cfd0 2024-04-21T08:42:05.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.729+0000 7fc5c3e25640 1 -- 172.21.15.86:0/2754431965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc5bc069230 con 0x7fc5bc19cfd0 2024-04-21T08:42:05.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.730+0000 7fc5b2ffd640 1 -- 172.21.15.86:0/2754431965 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc5b801a070 con 0x7fc5bc19cfd0 2024-04-21T08:42:05.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.731+0000 7fc5b2ffd640 1 --2- 172.21.15.86:0/2754431965 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc5a80a3d20 0x7fc5a80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:05.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.731+0000 7fc5c1bc1640 1 --2- 172.21.15.86:0/2754431965 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc5a80a3d20 0x7fc5a80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:05.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.731+0000 7fc5b2ffd640 1 -- 172.21.15.86:0/2754431965 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fc5b80d0380 con 0x7fc5bc19cfd0 2024-04-21T08:42:05.733 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.731+0000 7fc5c1bc1640 1 --2- 172.21.15.86:0/2754431965 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc5a80a3d20 0x7fc5a80a61e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4005fd0 tx=0x7fc5b4009500 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:05.735 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.736+0000 7fc5b2ffd640 1 -- 172.21.15.86:0/2754431965 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc5b8021070 con 0x7fc5bc19cfd0 2024-04-21T08:42:05.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.949+0000 7fc5c3e25640 1 -- 172.21.15.86:0/2754431965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fc5bc102b50 con 0x7fc5bc19cfd0 2024-04-21T08:42:05.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.950+0000 7fc5b2ffd640 1 -- 172.21.15.86:0/2754431965 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7fc5b809c6b0 con 0x7fc5bc19cfd0 2024-04-21T08:42:05.950 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:42:05.951 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:38:45.202126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24398,24394,24455,24472]},"id":1}]} 2024-04-21T08:42:05.951 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-21T08:42:05.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.952+0000 7fc5c3e25640 1 -- 172.21.15.86:0/2754431965 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc5a80a3d20 msgr2=0x7fc5a80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:05.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.952+0000 7fc5c3e25640 1 --2- 172.21.15.86:0/2754431965 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc5a80a3d20 0x7fc5a80a61e0 secure :-1 s=READY pgs=151 cs=0 l=1 rev1=1 crypto rx=0x7fc5b4005fd0 tx=0x7fc5b4009500 comp rx=0 tx=0).stop 2024-04-21T08:42:05.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.952+0000 7fc5c3e25640 1 -- 172.21.15.86:0/2754431965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5bc19cfd0 msgr2=0x7fc5bc19d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:05.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.952+0000 7fc5c3e25640 1 --2- 172.21.15.86:0/2754431965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5bc19cfd0 0x7fc5bc19d480 secure :-1 s=READY pgs=997 cs=0 l=1 rev1=1 crypto rx=0x7fc5b800d950 tx=0x7fc5b800de10 comp rx=0 tx=0).stop 2024-04-21T08:42:05.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.952+0000 7fc5c3e25640 1 -- 172.21.15.86:0/2754431965 shutdown_connections 2024-04-21T08:42:05.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.952+0000 7fc5c3e25640 1 --2- 172.21.15.86:0/2754431965 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc5a80a3d20 0x7fc5a80a61e0 unknown :-1 s=CLOSED pgs=151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.952+0000 7fc5c3e25640 1 --2- 172.21.15.86:0/2754431965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc5bc19cfd0 0x7fc5bc19d480 unknown :-1 s=CLOSED pgs=997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.952+0000 7fc5c3e25640 1 --2- 172.21.15.86:0/2754431965 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc5bc107270 0x7fc5bc198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.952+0000 7fc5c3e25640 1 --2- 172.21.15.86:0/2754431965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc5bc1021a0 0x7fc5bc198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:05.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.952+0000 7fc5c3e25640 1 -- 172.21.15.86:0/2754431965 >> 172.21.15.86:0/2754431965 conn(0x7fc5bc0793a0 msgr2=0x7fc5bc0fffb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:05.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.952+0000 7fc5c3e25640 1 -- 172.21.15.86:0/2754431965 shutdown_connections 2024-04-21T08:42:05.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:05.952+0000 7fc5c3e25640 1 -- 172.21.15.86:0/2754431965 wait complete. 2024-04-21T08:42:05.963 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:42:06.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.063+0000 7fdb17966640 1 -- 172.21.15.86:0/719784045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb1010c510 msgr2=0x7fdb1010e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.063+0000 7fdb17966640 1 --2- 172.21.15.86:0/719784045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb1010c510 0x7fdb1010e9b0 secure :-1 s=READY pgs=998 cs=0 l=1 rev1=1 crypto rx=0x7fdb08009ca0 tx=0x7fdb0802c230 comp rx=0 tx=0).stop 2024-04-21T08:42:06.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.063+0000 7fdb17966640 1 -- 172.21.15.86:0/719784045 shutdown_connections 2024-04-21T08:42:06.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.063+0000 7fdb17966640 1 --2- 172.21.15.86:0/719784045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb1010c510 0x7fdb1010e9b0 unknown :-1 s=CLOSED pgs=998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.063+0000 7fdb17966640 1 --2- 172.21.15.86:0/719784045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb10103af0 0x7fdb1010bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.063+0000 7fdb17966640 1 --2- 172.21.15.86:0/719784045 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb101031d0 0x7fdb101035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.063+0000 7fdb17966640 1 -- 172.21.15.86:0/719784045 >> 172.21.15.86:0/719784045 conn(0x7fdb10077f50 msgr2=0x7fdb10078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:06.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.063+0000 7fdb17966640 1 -- 172.21.15.86:0/719784045 shutdown_connections 2024-04-21T08:42:06.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.063+0000 7fdb17966640 1 -- 172.21.15.86:0/719784045 wait complete. 2024-04-21T08:42:06.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.063+0000 7fdb17966640 1 Processor -- start 2024-04-21T08:42:06.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.063+0000 7fdb17966640 1 -- start start 2024-04-21T08:42:06.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.063+0000 7fdb17966640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb101031d0 0x7fdb10198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.063+0000 7fdb17966640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb10103af0 0x7fdb10198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.063+0000 7fdb17966640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb1010c510 0x7fdb1019d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.063+0000 7fdb17966640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdb10112200 con 0x7fdb10103af0 2024-04-21T08:42:06.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.063+0000 7fdb17966640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdb10112080 con 0x7fdb101031d0 2024-04-21T08:42:06.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.063+0000 7fdb17966640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdb10112380 con 0x7fdb1010c510 2024-04-21T08:42:06.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.064+0000 7fdb14f01640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb10103af0 0x7fdb10198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.064+0000 7fdb14f01640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb10103af0 0x7fdb10198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40590/0 (socket says 172.21.15.86:40590) 2024-04-21T08:42:06.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.064+0000 7fdb14f01640 1 -- 172.21.15.86:0/1502030300 learned_addr learned my addr 172.21.15.86:0/1502030300 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:06.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.064+0000 7fdb15702640 1 --2- 172.21.15.86:0/1502030300 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb101031d0 0x7fdb10198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.064+0000 7fdb15f03640 1 --2- 172.21.15.86:0/1502030300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb1010c510 0x7fdb1019d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.064+0000 7fdb14f01640 1 -- 172.21.15.86:0/1502030300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb1010c510 msgr2=0x7fdb1019d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.064+0000 7fdb14f01640 1 --2- 172.21.15.86:0/1502030300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb1010c510 0x7fdb1019d020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.064+0000 7fdb14f01640 1 -- 172.21.15.86:0/1502030300 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb101031d0 msgr2=0x7fdb10198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.064+0000 7fdb14f01640 1 --2- 172.21.15.86:0/1502030300 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb101031d0 0x7fdb10198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.064+0000 7fdb14f01640 1 -- 172.21.15.86:0/1502030300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb1019d750 con 0x7fdb10103af0 2024-04-21T08:42:06.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.064+0000 7fdb14f01640 1 --2- 172.21.15.86:0/1502030300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb10103af0 0x7fdb10198ca0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fdb0000b4a0 tx=0x7fdb0000b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:06.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.064+0000 7fdb067fc640 1 -- 172.21.15.86:0/1502030300 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb00014070 con 0x7fdb10103af0 2024-04-21T08:42:06.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.064+0000 7fdb067fc640 1 -- 172.21.15.86:0/1502030300 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb00010ac0 con 0x7fdb10103af0 2024-04-21T08:42:06.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.064+0000 7fdb067fc640 1 -- 172.21.15.86:0/1502030300 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb00017c30 con 0x7fdb10103af0 2024-04-21T08:42:06.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.064+0000 7fdb17966640 1 -- 172.21.15.86:0/1502030300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb1019da40 con 0x7fdb10103af0 2024-04-21T08:42:06.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.064+0000 7fdb17966640 1 -- 172.21.15.86:0/1502030300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb101a52d0 con 0x7fdb10103af0 2024-04-21T08:42:06.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.066+0000 7fdb067fc640 1 -- 172.21.15.86:0/1502030300 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdb00017070 con 0x7fdb10103af0 2024-04-21T08:42:06.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.067+0000 7fdb1a630740 1 Processor -- start 2024-04-21T08:42:06.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.067+0000 7fdb1a630740 1 -- start start 2024-04-21T08:42:06.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.067+0000 7fdb1a630740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55950a6f34d0 0x55950a6f38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.067+0000 7fdb1a630740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55950a6f3e80 0x55950a6fc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.067+0000 7fdb1a630740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55950a6fca00 0x55950a6fee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.067+0000 7fdb1a630740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55950a6581f0 con 0x55950a6fca00 2024-04-21T08:42:06.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.067+0000 7fdb1a630740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55950a5f54b0 con 0x55950a6f3e80 2024-04-21T08:42:06.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.067+0000 7fdb1a630740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55950a6541b0 con 0x55950a6f34d0 2024-04-21T08:42:06.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.067+0000 7fdb067fc640 1 --2- 172.21.15.86:0/1502030300 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdafc0a3d20 0x7fdafc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.067+0000 7fdb14f01640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55950a6fca00 0x55950a6fee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.067+0000 7fdb14f01640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55950a6fca00 0x55950a6fee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40594/0 (socket says 172.21.15.86:40594) 2024-04-21T08:42:06.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.067+0000 7fdb14f01640 1 -- 172.21.15.86:0/4032375098 learned_addr learned my addr 172.21.15.86:0/4032375098 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:06.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.067+0000 7fdb15f03640 1 --2- 172.21.15.86:0/4032375098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55950a6f34d0 0x55950a6f38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.067+0000 7fdb15702640 1 --2- 172.21.15.86:0/4032375098 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55950a6f3e80 0x55950a6fc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.067+0000 7fdb15f03640 1 --2- 172.21.15.86:0/1502030300 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdafc0a3d20 0x7fdafc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.067+0000 7fdb067fc640 1 -- 172.21.15.86:0/1502030300 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fdb000cfdf0 con 0x7fdb10103af0 2024-04-21T08:42:06.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.067+0000 7fdb14f01640 1 -- 172.21.15.86:0/4032375098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55950a6f34d0 msgr2=0x55950a6f38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.067+0000 7fdb14f01640 1 --2- 172.21.15.86:0/4032375098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55950a6f34d0 0x55950a6f38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.067+0000 7fdb14f01640 1 -- 172.21.15.86:0/4032375098 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55950a6f3e80 msgr2=0x55950a6fc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.067+0000 7fdb14f01640 1 --2- 172.21.15.86:0/4032375098 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55950a6f3e80 0x55950a6fc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.067+0000 7fdb14f01640 1 -- 172.21.15.86:0/4032375098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55950a5e71b0 con 0x55950a6fca00 2024-04-21T08:42:06.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.067+0000 7fdb15702640 1 --2- 172.21.15.86:0/4032375098 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55950a6f3e80 0x55950a6fc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:42:06.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.068+0000 7fdb14f01640 1 --2- 172.21.15.86:0/4032375098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55950a6fca00 0x55950a6fee50 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fdb00098500 tx=0x7fdb00097f00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f347386644768ccf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:06.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.068+0000 7fdb15f03640 1 --2- 172.21.15.86:0/1502030300 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdafc0a3d20 0x7fdafc0a61e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdb08009a60 tx=0x7fdb0802c230 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:06.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.068+0000 7fdafb7fe640 1 -- 172.21.15.86:0/4032375098 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0001e070 con 0x55950a6fca00 2024-04-21T08:42:06.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.068+0000 7fdafb7fe640 1 -- 172.21.15.86:0/4032375098 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb0009c230 con 0x55950a6fca00 2024-04-21T08:42:06.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.068+0000 7fdafb7fe640 1 -- 172.21.15.86:0/4032375098 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb00010c70 con 0x55950a6fca00 2024-04-21T08:42:06.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.068+0000 7fdb1a630740 1 -- 172.21.15.86:0/4032375098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55950a6fca00 msgr2=0x55950a6fee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.068+0000 7fdb1a630740 1 --2- 172.21.15.86:0/4032375098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55950a6fca00 0x55950a6fee50 secure :-1 s=READY pgs=1000 cs=0 l=1 rev1=1 crypto rx=0x7fdb00098500 tx=0x7fdb00097f00 comp rx=0 tx=0).stop 2024-04-21T08:42:06.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.068+0000 7fdb1a630740 1 -- 172.21.15.86:0/4032375098 shutdown_connections 2024-04-21T08:42:06.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.068+0000 7fdb1a630740 1 --2- 172.21.15.86:0/4032375098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55950a6fca00 0x55950a6fee50 unknown :-1 s=CLOSED pgs=1000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.068+0000 7fdb1a630740 1 --2- 172.21.15.86:0/4032375098 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55950a6f3e80 0x55950a6fc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.068+0000 7fdb1a630740 1 --2- 172.21.15.86:0/4032375098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55950a6f34d0 0x55950a6f38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.068+0000 7fdb1a630740 1 -- 172.21.15.86:0/4032375098 >> 172.21.15.86:0/4032375098 conn(0x55950a6edba0 msgr2=0x55950a6edf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:06.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.069+0000 7fdb1a630740 1 -- 172.21.15.86:0/4032375098 shutdown_connections 2024-04-21T08:42:06.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.069+0000 7fdb1a630740 1 -- 172.21.15.86:0/4032375098 wait complete. 2024-04-21T08:42:06.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.069+0000 7fdb1a630740 1 -- start start 2024-04-21T08:42:06.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.069+0000 7fdb1a630740 1 Processor -- start 2024-04-21T08:42:06.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.069+0000 7fdb1a630740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55950a6f34d0 0x55950a6fc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.069+0000 7fdb1a630740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55950a6f3e80 0x55950a6f9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.069+0000 7fdb1a630740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55950a6fca00 0x55950a6f9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.069+0000 7fdb1a630740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55950a6ff8b0 con 0x55950a6fca00 2024-04-21T08:42:06.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.069+0000 7fdb1a630740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55950a611500 con 0x55950a6f3e80 2024-04-21T08:42:06.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.069+0000 7fdb1a630740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55950a6ffa30 con 0x55950a6f34d0 2024-04-21T08:42:06.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.069+0000 7fdb1a630740 10 client.0 _ll_register_callbacks cb 0x55950a5dfb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:06.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.069+0000 7fdb15f03640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55950a6fca00 0x55950a6f9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.069+0000 7fdb15f03640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55950a6fca00 0x55950a6f9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40604/0 (socket says 172.21.15.86:40604) 2024-04-21T08:42:06.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.069+0000 7fdb15f03640 1 -- 172.21.15.86:0/2245910412 learned_addr learned my addr 172.21.15.86:0/2245910412 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:06.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.070+0000 7fdb14f01640 1 --2- 172.21.15.86:0/2245910412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55950a6f3e80 0x55950a6f9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.070+0000 7fdb15702640 1 --2- 172.21.15.86:0/2245910412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55950a6f34d0 0x55950a6fc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.070+0000 7fdb15f03640 1 -- 172.21.15.86:0/2245910412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55950a6f34d0 msgr2=0x55950a6fc3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.070+0000 7fdb15f03640 1 --2- 172.21.15.86:0/2245910412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55950a6f34d0 0x55950a6fc3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.070+0000 7fdb15f03640 1 -- 172.21.15.86:0/2245910412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55950a6f3e80 msgr2=0x55950a6f9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.070+0000 7fdb15f03640 1 --2- 172.21.15.86:0/2245910412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55950a6f3e80 0x55950a6f9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.070+0000 7fdb15f03640 1 -- 172.21.15.86:0/2245910412 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55950a36a3f0 con 0x55950a6fca00 2024-04-21T08:42:06.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.070+0000 7fdb14f01640 1 --2- 172.21.15.86:0/2245910412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55950a6f3e80 0x55950a6f9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:06.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.070+0000 7fdb15f03640 1 --2- 172.21.15.86:0/2245910412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55950a6fca00 0x55950a6f9e70 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fdb08036000 tx=0x7fdb080095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b560673ec33071 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:06.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.070+0000 7fdaf8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:06.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.070+0000 7fdaf8ff9640 1 -- 172.21.15.86:0/2245910412 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb08039070 con 0x55950a6fca00 2024-04-21T08:42:06.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.070+0000 7fdaf8ff9640 1 -- 172.21.15.86:0/2245910412 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fdb080041c0 con 0x55950a6fca00 2024-04-21T08:42:06.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.070+0000 7fdb1a630740 1 -- 172.21.15.86:0/2245910412 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb0009c230 con 0x55950a6fca00 2024-04-21T08:42:06.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.070+0000 7fdaf8ff9640 1 -- 172.21.15.86:0/2245910412 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdb080043c0 con 0x55950a6fca00 2024-04-21T08:42:06.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.070+0000 7fdb1a630740 10 client.17172 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:06.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.070+0000 7fdb1a630740 1 -- 172.21.15.86:0/2245910412 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55950a4347d0 con 0x55950a6fca00 2024-04-21T08:42:06.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.070+0000 7fdaf8ff9640 1 -- 172.21.15.86:0/2245910412 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb0009c230 con 0x55950a6fca00 2024-04-21T08:42:06.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.070+0000 7fdaf8ff9640 1 -- 172.21.15.86:0/2245910412 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdb08034ae0 con 0x55950a6fca00 2024-04-21T08:42:06.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.071+0000 7fdb1a630740 10 client.17172 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:06.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.071+0000 7fdb1a630740 10 client.17172 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:42:06.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.071+0000 7fdb1a630740 20 client.17172 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:06.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.071+0000 7fdb1a630740 1 --2- 172.21.15.86:0/2245910412 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55950a70cc10 0x55950a70f040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.071+0000 7fdb1a630740 4 client.17172 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:06.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.071+0000 7fdb1a630740 1 -- 172.21.15.86:0/2245910412 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55950a5f7930 con 0x55950a70cc10 2024-04-21T08:42:06.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.071+0000 7fdb15702640 1 --2- 172.21.15.86:0/2245910412 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55950a70cc10 0x55950a70f040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.071+0000 7fdb15702640 1 --2- 172.21.15.86:0/2245910412 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55950a70cc10 0x55950a70f040 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5cc0015f9035eb7d server_cookie=5bb4c572034bcd9e in_seq=0 out_seq=0 2024-04-21T08:42:06.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.071+0000 7fdaf8ff9640 10 client.17172 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:42:06.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.072+0000 7fdaf8ff9640 1 -- 172.21.15.86:0/2245910412 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55950a5f7930 con 0x55950a70cc10 2024-04-21T08:42:06.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.072+0000 7fdaf8ff9640 10 client.17172 handle_command_reply: tid=0 2024-04-21T08:42:06.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.072+0000 7fdb1a630740 1 client.17172 shutdown 2024-04-21T08:42:06.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.072+0000 7fdb1a630740 1 -- 172.21.15.86:0/2245910412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55950a6fca00 msgr2=0x55950a6f9e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.072+0000 7fdb1a630740 1 --2- 172.21.15.86:0/2245910412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55950a6fca00 0x55950a6f9e70 secure :-1 s=READY pgs=1001 cs=0 l=1 rev1=1 crypto rx=0x7fdb08036000 tx=0x7fdb080095c0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.072+0000 7fdb1a630740 1 -- 172.21.15.86:0/2245910412 shutdown_connections 2024-04-21T08:42:06.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.072+0000 7fdb1a630740 1 --2- 172.21.15.86:0/2245910412 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55950a6fca00 0x55950a6f9e70 unknown :-1 s=CLOSED pgs=1001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.072+0000 7fdb1a630740 1 --2- 172.21.15.86:0/2245910412 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55950a70cc10 0x55950a70f040 crc :-1 s=READY pgs=44 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.072+0000 7fdb1a630740 1 --2- 172.21.15.86:0/2245910412 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55950a6f3e80 0x55950a6f9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.072+0000 7fdb1a630740 1 --2- 172.21.15.86:0/2245910412 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55950a6f34d0 0x55950a6fc3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.072+0000 7fdaf8ff9640 0 client.17172 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:42:06.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.072+0000 7fdb1a630740 1 -- 172.21.15.86:0/2245910412 >> 172.21.15.86:0/2245910412 conn(0x55950a6edba0 msgr2=0x55950a6ef8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:06.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.072+0000 7fdb1a630740 1 -- 172.21.15.86:0/2245910412 shutdown_connections 2024-04-21T08:42:06.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.072+0000 7fdb1a630740 1 -- 172.21.15.86:0/2245910412 wait complete. 2024-04-21T08:42:06.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.072+0000 7fdb1a630740 20 client.17172 trim_cache size 0 max 16384 2024-04-21T08:42:06.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.085+0000 7fdb1a630740 1 Processor -- start 2024-04-21T08:42:06.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.086+0000 7fdb1a630740 1 -- start start 2024-04-21T08:42:06.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.086+0000 7fdb1a630740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55950a6f34d0 0x55950a708770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.086+0000 7fdb1a630740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55950a6f3e80 0x55950a714910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.086+0000 7fdb1a630740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55950a6fca00 0x55950a714e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.086+0000 7fdb1a630740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdb101af360 con 0x55950a6f34d0 2024-04-21T08:42:06.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.086+0000 7fdb1a630740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdb101b0ea0 con 0x55950a6fca00 2024-04-21T08:42:06.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.086+0000 7fdb1a630740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdb101afa30 con 0x55950a6f3e80 2024-04-21T08:42:06.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.086+0000 7fdb15702640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55950a6f34d0 0x55950a708770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.086+0000 7fdb15702640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55950a6f34d0 0x55950a708770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40612/0 (socket says 172.21.15.86:40612) 2024-04-21T08:42:06.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.086+0000 7fdb15702640 1 -- 172.21.15.86:0/1985105116 learned_addr learned my addr 172.21.15.86:0/1985105116 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:06.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.086+0000 7fdb15f03640 1 --2- 172.21.15.86:0/1985105116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55950a6fca00 0x55950a714e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.086+0000 7fdb14f01640 1 --2- 172.21.15.86:0/1985105116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55950a6f3e80 0x55950a714910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.086+0000 7fdb15702640 1 -- 172.21.15.86:0/1985105116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55950a6f3e80 msgr2=0x55950a714910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.086+0000 7fdb15702640 1 --2- 172.21.15.86:0/1985105116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55950a6f3e80 0x55950a714910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.086+0000 7fdb15702640 1 -- 172.21.15.86:0/1985105116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55950a6fca00 msgr2=0x55950a714e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.086+0000 7fdb15702640 1 --2- 172.21.15.86:0/1985105116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55950a6fca00 0x55950a714e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.086+0000 7fdb15702640 1 -- 172.21.15.86:0/1985105116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55950a6fa470 con 0x55950a6f34d0 2024-04-21T08:42:06.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.086+0000 7fdb15702640 1 --2- 172.21.15.86:0/1985105116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55950a6f34d0 0x55950a708770 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x55950a70f8b0 tx=0x7fdb0c00b390 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bc9806304f319f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:06.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.086+0000 7fdafaffd640 1 -- 172.21.15.86:0/1985105116 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c003ca0 con 0x55950a6f34d0 2024-04-21T08:42:06.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.086+0000 7fdafaffd640 1 -- 172.21.15.86:0/1985105116 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb0c023020 con 0x55950a6f34d0 2024-04-21T08:42:06.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.086+0000 7fdafaffd640 1 -- 172.21.15.86:0/1985105116 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb0c0117a0 con 0x55950a6f34d0 2024-04-21T08:42:06.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.087+0000 7fdb1a630740 1 -- 172.21.15.86:0/1985105116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55950a6f34d0 msgr2=0x55950a708770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.087+0000 7fdb1a630740 1 --2- 172.21.15.86:0/1985105116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55950a6f34d0 0x55950a708770 secure :-1 s=READY pgs=1002 cs=0 l=1 rev1=1 crypto rx=0x55950a70f8b0 tx=0x7fdb0c00b390 comp rx=0 tx=0).stop 2024-04-21T08:42:06.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.087+0000 7fdb1a630740 1 -- 172.21.15.86:0/1985105116 shutdown_connections 2024-04-21T08:42:06.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.087+0000 7fdb1a630740 1 --2- 172.21.15.86:0/1985105116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55950a6fca00 0x55950a714e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.087+0000 7fdb1a630740 1 --2- 172.21.15.86:0/1985105116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55950a6f3e80 0x55950a714910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.087+0000 7fdb1a630740 1 --2- 172.21.15.86:0/1985105116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55950a6f34d0 0x55950a708770 unknown :-1 s=CLOSED pgs=1002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.087+0000 7fdb1a630740 1 -- 172.21.15.86:0/1985105116 >> 172.21.15.86:0/1985105116 conn(0x55950a6edba0 msgr2=0x55950a6ecc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:06.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.087+0000 7fdb1a630740 1 -- 172.21.15.86:0/1985105116 shutdown_connections 2024-04-21T08:42:06.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.087+0000 7fdb1a630740 1 -- 172.21.15.86:0/1985105116 wait complete. 2024-04-21T08:42:06.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.087+0000 7fdb1a630740 1 -- start start 2024-04-21T08:42:06.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.087+0000 7fdb1a630740 1 Processor -- start 2024-04-21T08:42:06.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.088+0000 7fdb1a630740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55950a6f34d0 0x55950a704d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.088+0000 7fdb1a630740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55950a6f3e80 0x55950a701080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.088+0000 7fdb1a630740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55950a6fca00 0x55950a7015c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.088+0000 7fdb1a630740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdb101b07d0 con 0x55950a6fca00 2024-04-21T08:42:06.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.088+0000 7fdb1a630740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdb101b0100 con 0x55950a6f3e80 2024-04-21T08:42:06.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.088+0000 7fdb1a630740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdb101ad320 con 0x55950a6f34d0 2024-04-21T08:42:06.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.088+0000 7fdb1a630740 10 client.0 _ll_register_callbacks cb 0x7fdb08032a20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:06.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.088+0000 7fdb15f03640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55950a6fca00 0x55950a7015c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.088+0000 7fdb15f03640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55950a6fca00 0x55950a7015c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40624/0 (socket says 172.21.15.86:40624) 2024-04-21T08:42:06.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.088+0000 7fdb15f03640 1 -- 172.21.15.86:0/2550597041 learned_addr learned my addr 172.21.15.86:0/2550597041 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:06.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.088+0000 7fdb14f01640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55950a6f3e80 0x55950a701080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.088+0000 7fdb15702640 1 --2- 172.21.15.86:0/2550597041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55950a6f34d0 0x55950a704d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.088+0000 7fdb15f03640 1 -- 172.21.15.86:0/2550597041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55950a6f34d0 msgr2=0x55950a704d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.088+0000 7fdb15f03640 1 --2- 172.21.15.86:0/2550597041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55950a6f34d0 0x55950a704d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.088+0000 7fdb15f03640 1 -- 172.21.15.86:0/2550597041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55950a6f3e80 msgr2=0x55950a701080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.088+0000 7fdb15f03640 1 --2- 172.21.15.86:0/2550597041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55950a6f3e80 0x55950a701080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.088+0000 7fdb15f03640 1 -- 172.21.15.86:0/2550597041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55950a701b00 con 0x55950a6fca00 2024-04-21T08:42:06.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.088+0000 7fdb14f01640 1 --2- 172.21.15.86:0/2550597041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55950a6f3e80 0x55950a701080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:42:06.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.088+0000 7fdb15f03640 1 --2- 172.21.15.86:0/2550597041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55950a6fca00 0x55950a7015c0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fdb0802c840 tx=0x7fdb0802c8b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71dceec4f80d90fc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:06.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.088+0000 7fdaf8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:06.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.088+0000 7fdaf8ff9640 1 -- 172.21.15.86:0/2550597041 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb08065020 con 0x55950a6fca00 2024-04-21T08:42:06.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.089+0000 7fdaf8ff9640 1 -- 172.21.15.86:0/2550597041 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fdb08004770 con 0x55950a6fca00 2024-04-21T08:42:06.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.089+0000 7fdb1a630740 1 -- 172.21.15.86:0/2550597041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb0c023020 con 0x55950a6fca00 2024-04-21T08:42:06.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.089+0000 7fdaf8ff9640 1 -- 172.21.15.86:0/2550597041 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdb08070050 con 0x55950a6fca00 2024-04-21T08:42:06.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.089+0000 7fdb1a630740 10 client.17178 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:06.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.089+0000 7fdb1a630740 1 -- 172.21.15.86:0/2550597041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55950a773e50 con 0x55950a6fca00 2024-04-21T08:42:06.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.089+0000 7fdaf8ff9640 1 -- 172.21.15.86:0/2550597041 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb0c023020 con 0x55950a6fca00 2024-04-21T08:42:06.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.089+0000 7fdaf8ff9640 1 -- 172.21.15.86:0/2550597041 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdb08034970 con 0x55950a6fca00 2024-04-21T08:42:06.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.089+0000 7fdb1a630740 10 client.17178 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:06.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.089+0000 7fdb1a630740 10 client.17178 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:42:06.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.089+0000 7fdb1a630740 20 client.17178 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:06.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.089+0000 7fdb1a630740 1 --2- 172.21.15.86:0/2550597041 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55950a774620 0x55950a776a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.089+0000 7fdb1a630740 4 client.17178 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:42:06.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.089+0000 7fdb1a630740 1 -- 172.21.15.86:0/2550597041 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55950a665c10 con 0x55950a774620 2024-04-21T08:42:06.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.089+0000 7fdb15702640 1 --2- 172.21.15.86:0/2550597041 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55950a774620 0x55950a776a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.090+0000 7fdaf8ff9640 10 client.17178 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:42:06.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.090+0000 7fdb15702640 1 --2- 172.21.15.86:0/2550597041 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55950a774620 0x55950a776a40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=6bb873f686aa5f7e server_cookie=9c1c7baf3388fae8 in_seq=0 out_seq=0 2024-04-21T08:42:06.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.090+0000 7fdaf8ff9640 1 -- 172.21.15.86:0/2550597041 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55950a665c10 con 0x55950a774620 2024-04-21T08:42:06.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.090+0000 7fdaf8ff9640 10 client.17178 handle_command_reply: tid=0 2024-04-21T08:42:06.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.090+0000 7fdb1a630740 1 client.17178 shutdown 2024-04-21T08:42:06.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.090+0000 7fdb1a630740 1 -- 172.21.15.86:0/2550597041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55950a6fca00 msgr2=0x55950a7015c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.090+0000 7fdb1a630740 1 --2- 172.21.15.86:0/2550597041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55950a6fca00 0x55950a7015c0 secure :-1 s=READY pgs=1003 cs=0 l=1 rev1=1 crypto rx=0x7fdb0802c840 tx=0x7fdb0802c8b0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.090+0000 7fdb1a630740 1 -- 172.21.15.86:0/2550597041 shutdown_connections 2024-04-21T08:42:06.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.090+0000 7fdb1a630740 1 --2- 172.21.15.86:0/2550597041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55950a6fca00 0x55950a7015c0 unknown :-1 s=CLOSED pgs=1003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.090+0000 7fdb1a630740 1 --2- 172.21.15.86:0/2550597041 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55950a774620 0x55950a776a40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.090+0000 7fdb1a630740 1 --2- 172.21.15.86:0/2550597041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55950a6f3e80 0x55950a701080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.090+0000 7fdaf8ff9640 0 client.17178 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:42:06.097 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.090+0000 7fdb1a630740 1 --2- 172.21.15.86:0/2550597041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55950a6f34d0 0x55950a704d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.090+0000 7fdb1a630740 1 -- 172.21.15.86:0/2550597041 >> 172.21.15.86:0/2550597041 conn(0x55950a6edba0 msgr2=0x55950a6ec370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:06.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.090+0000 7fdb1a630740 1 -- 172.21.15.86:0/2550597041 shutdown_connections 2024-04-21T08:42:06.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.090+0000 7fdb1a630740 1 -- 172.21.15.86:0/2550597041 wait complete. 2024-04-21T08:42:06.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.090+0000 7fdb1a630740 20 client.17178 trim_cache size 0 max 16384 2024-04-21T08:42:06.098 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:42:06.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.091+0000 7fdb17966640 1 -- 172.21.15.86:0/1502030300 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdafc0a3d20 msgr2=0x7fdafc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.091+0000 7fdb17966640 1 --2- 172.21.15.86:0/1502030300 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdafc0a3d20 0x7fdafc0a61e0 secure :-1 s=READY pgs=152 cs=0 l=1 rev1=1 crypto rx=0x7fdb08009a60 tx=0x7fdb0802c230 comp rx=0 tx=0).stop 2024-04-21T08:42:06.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.091+0000 7fdb17966640 1 -- 172.21.15.86:0/1502030300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb10103af0 msgr2=0x7fdb10198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.091+0000 7fdb17966640 1 --2- 172.21.15.86:0/1502030300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb10103af0 0x7fdb10198ca0 secure :-1 s=READY pgs=999 cs=0 l=1 rev1=1 crypto rx=0x7fdb0000b4a0 tx=0x7fdb0000b960 comp rx=0 tx=0).stop 2024-04-21T08:42:06.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.091+0000 7fdb17966640 1 -- 172.21.15.86:0/1502030300 shutdown_connections 2024-04-21T08:42:06.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.091+0000 7fdb17966640 1 --2- 172.21.15.86:0/1502030300 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdafc0a3d20 0x7fdafc0a61e0 unknown :-1 s=CLOSED pgs=152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.091+0000 7fdb17966640 1 --2- 172.21.15.86:0/1502030300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb1010c510 0x7fdb1019d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.091+0000 7fdb17966640 1 --2- 172.21.15.86:0/1502030300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb10103af0 0x7fdb10198ca0 unknown :-1 s=CLOSED pgs=999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.091+0000 7fdb17966640 1 --2- 172.21.15.86:0/1502030300 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb101031d0 0x7fdb10198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.091+0000 7fdb17966640 1 -- 172.21.15.86:0/1502030300 >> 172.21.15.86:0/1502030300 conn(0x7fdb10077f50 msgr2=0x7fdb100fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:06.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.091+0000 7fdb17966640 1 -- 172.21.15.86:0/1502030300 shutdown_connections 2024-04-21T08:42:06.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.091+0000 7fdb17966640 1 -- 172.21.15.86:0/1502030300 wait complete. 2024-04-21T08:42:06.101 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:42:06.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.201+0000 7f7ed1ac0640 1 -- 172.21.15.86:0/1220343776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ecc106170 msgr2=0x7f7ecc10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.201+0000 7f7ed1ac0640 1 --2- 172.21.15.86:0/1220343776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ecc106170 0x7f7ecc10dd40 secure :-1 s=READY pgs=1004 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4009900 tx=0x7f7ec402bc30 comp rx=0 tx=0).stop 2024-04-21T08:42:06.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.201+0000 7f7ed1ac0640 1 -- 172.21.15.86:0/1220343776 shutdown_connections 2024-04-21T08:42:06.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.201+0000 7f7ed1ac0640 1 --2- 172.21.15.86:0/1220343776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ecc106170 0x7f7ecc10dd40 unknown :-1 s=CLOSED pgs=1004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.201+0000 7f7ed1ac0640 1 --2- 172.21.15.86:0/1220343776 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ecc101a50 0x7f7ecc105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.201+0000 7f7ed1ac0640 1 --2- 172.21.15.86:0/1220343776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ecc1010a0 0x7f7ecc101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.201+0000 7f7ed1ac0640 1 -- 172.21.15.86:0/1220343776 >> 172.21.15.86:0/1220343776 conn(0x7f7ecc077eb0 msgr2=0x7f7ecc0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:06.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.201+0000 7f7ed1ac0640 1 -- 172.21.15.86:0/1220343776 shutdown_connections 2024-04-21T08:42:06.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.201+0000 7f7ed1ac0640 1 -- 172.21.15.86:0/1220343776 wait complete. 2024-04-21T08:42:06.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.201+0000 7f7ed1ac0640 1 Processor -- start 2024-04-21T08:42:06.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.201+0000 7f7ed1ac0640 1 -- start start 2024-04-21T08:42:06.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.202+0000 7f7ed1ac0640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ecc1010a0 0x7f7ecc1975e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.202+0000 7f7ed1ac0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ecc106170 0x7f7ecc197b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.202+0000 7f7ed1ac0640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ecc19bea0 0x7f7ecc19c350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.202+0000 7f7ed1ac0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc1110e0 con 0x7f7ecc106170 2024-04-21T08:42:06.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.202+0000 7f7ed1ac0640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc110f60 con 0x7f7ecc1010a0 2024-04-21T08:42:06.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.202+0000 7f7ed1ac0640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc111260 con 0x7f7ecc19bea0 2024-04-21T08:42:06.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.202+0000 7f7ecaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ecc106170 0x7f7ecc197b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.202+0000 7f7ecaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ecc106170 0x7f7ecc197b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40630/0 (socket says 172.21.15.86:40630) 2024-04-21T08:42:06.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.202+0000 7f7ecaffd640 1 -- 172.21.15.86:0/658592371 learned_addr learned my addr 172.21.15.86:0/658592371 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:06.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.202+0000 7f7ecb7fe640 1 --2- 172.21.15.86:0/658592371 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ecc1010a0 0x7f7ecc1975e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.202+0000 7f7ecbfff640 1 --2- 172.21.15.86:0/658592371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ecc19bea0 0x7f7ecc19c350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.202+0000 7f7ecaffd640 1 -- 172.21.15.86:0/658592371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ecc19bea0 msgr2=0x7f7ecc19c350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.202+0000 7f7ecaffd640 1 --2- 172.21.15.86:0/658592371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ecc19bea0 0x7f7ecc19c350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.202+0000 7f7ecaffd640 1 -- 172.21.15.86:0/658592371 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ecc1010a0 msgr2=0x7f7ecc1975e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.202+0000 7f7ecaffd640 1 --2- 172.21.15.86:0/658592371 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ecc1010a0 0x7f7ecc1975e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.202+0000 7f7ecaffd640 1 -- 172.21.15.86:0/658592371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7ecc19c990 con 0x7f7ecc106170 2024-04-21T08:42:06.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.202+0000 7f7ecbfff640 1 --2- 172.21.15.86:0/658592371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ecc19bea0 0x7f7ecc19c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:06.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.202+0000 7f7ecaffd640 1 --2- 172.21.15.86:0/658592371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ecc106170 0x7f7ecc197b20 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400e9c0 tx=0x7f7eb400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:06.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.202+0000 7f7ec8ff9640 1 -- 172.21.15.86:0/658592371 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb400cd80 con 0x7f7ecc106170 2024-04-21T08:42:06.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.202+0000 7f7ec8ff9640 1 -- 172.21.15.86:0/658592371 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7eb40054d0 con 0x7f7ecc106170 2024-04-21T08:42:06.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.202+0000 7f7ed1ac0640 1 -- 172.21.15.86:0/658592371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7ecc19cc80 con 0x7f7ecc106170 2024-04-21T08:42:06.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.202+0000 7f7ec8ff9640 1 -- 172.21.15.86:0/658592371 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb4010430 con 0x7f7ecc106170 2024-04-21T08:42:06.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.202+0000 7f7ed1ac0640 1 -- 172.21.15.86:0/658592371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7ecc1a4470 con 0x7f7ecc106170 2024-04-21T08:42:06.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.205+0000 7f7ec8ff9640 1 -- 172.21.15.86:0/658592371 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7eb4004fb0 con 0x7f7ecc106170 2024-04-21T08:42:06.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.205+0000 7f7ed4781740 1 Processor -- start 2024-04-21T08:42:06.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.205+0000 7f7ed4781740 1 -- start start 2024-04-21T08:42:06.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.205+0000 7f7ed4781740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557539e9a4d0 0x557539e9a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.205+0000 7f7ed4781740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557539e9ae80 0x557539ea3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.205+0000 7f7ed4781740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557539ea3a00 0x557539ea5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.205+0000 7f7ed4781740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557539dff1f0 con 0x557539e9ae80 2024-04-21T08:42:06.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.205+0000 7f7ed4781740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x557539d9c4b0 con 0x557539ea3a00 2024-04-21T08:42:06.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.205+0000 7f7ed4781740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x557539dfb1b0 con 0x557539e9a4d0 2024-04-21T08:42:06.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.205+0000 7f7ecb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557539e9ae80 0x557539ea3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.205+0000 7f7ecb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557539e9ae80 0x557539ea3430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40640/0 (socket says 172.21.15.86:40640) 2024-04-21T08:42:06.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.205+0000 7f7ecb7fe640 1 -- 172.21.15.86:0/163489886 learned_addr learned my addr 172.21.15.86:0/163489886 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:06.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.205+0000 7f7ec8ff9640 1 --2- 172.21.15.86:0/658592371 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7eb80a3d20 0x7f7eb80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.205+0000 7f7ecbfff640 1 --2- 172.21.15.86:0/163489886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557539e9a4d0 0x557539e9a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.205+0000 7f7ecaffd640 1 --2- 172.21.15.86:0/163489886 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557539ea3a00 0x557539ea5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.205+0000 7f7ecb7fe640 1 -- 172.21.15.86:0/163489886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557539e9a4d0 msgr2=0x557539e9a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.205+0000 7f7ecb7fe640 1 --2- 172.21.15.86:0/163489886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557539e9a4d0 0x557539e9a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.205+0000 7f7ecb7fe640 1 -- 172.21.15.86:0/163489886 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557539ea3a00 msgr2=0x557539ea5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.205+0000 7f7ecb7fe640 1 --2- 172.21.15.86:0/163489886 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557539ea3a00 0x557539ea5e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.205+0000 7f7ecb7fe640 1 -- 172.21.15.86:0/163489886 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557539d8e1b0 con 0x557539e9ae80 2024-04-21T08:42:06.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.205+0000 7f7ecbfff640 1 --2- 172.21.15.86:0/163489886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557539e9a4d0 0x557539e9a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:06.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.205+0000 7f7ecaffd640 1 --2- 172.21.15.86:0/163489886 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557539ea3a00 0x557539ea5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:06.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.205+0000 7f7ecbfff640 1 --2- 172.21.15.86:0/658592371 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7eb80a3d20 0x7f7eb80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.205+0000 7f7ec8ff9640 1 -- 172.21.15.86:0/658592371 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f7eb40d0f90 con 0x7f7ecc106170 2024-04-21T08:42:06.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.206+0000 7f7ecb7fe640 1 --2- 172.21.15.86:0/163489886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557539e9ae80 0x557539ea3430 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc0103f0 tx=0x7f7ebc0108b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=235399e47deb1116 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:06.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.206+0000 7f7ec1ffb640 1 -- 172.21.15.86:0/163489886 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc016070 con 0x557539e9ae80 2024-04-21T08:42:06.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.206+0000 7f7ec1ffb640 1 -- 172.21.15.86:0/163489886 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ebc004d20 con 0x557539e9ae80 2024-04-21T08:42:06.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.206+0000 7f7ec1ffb640 1 -- 172.21.15.86:0/163489886 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ebc0053e0 con 0x557539e9ae80 2024-04-21T08:42:06.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.206+0000 7f7ed4781740 1 -- 172.21.15.86:0/163489886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557539e9ae80 msgr2=0x557539ea3430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.206+0000 7f7ecbfff640 1 --2- 172.21.15.86:0/658592371 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7eb80a3d20 0x7f7eb80a61e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4005ab0 tx=0x7f7ec4009260 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:06.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.206+0000 7f7ed4781740 1 --2- 172.21.15.86:0/163489886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557539e9ae80 0x557539ea3430 secure :-1 s=READY pgs=1006 cs=0 l=1 rev1=1 crypto rx=0x7f7ebc0103f0 tx=0x7f7ebc0108b0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.206+0000 7f7ed4781740 1 -- 172.21.15.86:0/163489886 shutdown_connections 2024-04-21T08:42:06.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.206+0000 7f7ed4781740 1 --2- 172.21.15.86:0/163489886 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557539ea3a00 0x557539ea5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.206+0000 7f7ed4781740 1 --2- 172.21.15.86:0/163489886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557539e9ae80 0x557539ea3430 unknown :-1 s=CLOSED pgs=1006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.206+0000 7f7ed4781740 1 --2- 172.21.15.86:0/163489886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557539e9a4d0 0x557539e9a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.206+0000 7f7ed4781740 1 -- 172.21.15.86:0/163489886 >> 172.21.15.86:0/163489886 conn(0x557539e94ba0 msgr2=0x557539e94f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:06.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.206+0000 7f7ed4781740 1 -- 172.21.15.86:0/163489886 shutdown_connections 2024-04-21T08:42:06.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.206+0000 7f7ed4781740 1 -- 172.21.15.86:0/163489886 wait complete. 2024-04-21T08:42:06.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.207+0000 7f7ed4781740 1 -- start start 2024-04-21T08:42:06.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.207+0000 7f7ed4781740 1 Processor -- start 2024-04-21T08:42:06.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.207+0000 7f7ed4781740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557539e9a4d0 0x557539eab740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.207+0000 7f7ed4781740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557539e9ae80 0x557539eabc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.207+0000 7f7ed4781740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557539ea3a00 0x557539ea2770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.207+0000 7f7ed4781740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557539ea6970 con 0x557539e9ae80 2024-04-21T08:42:06.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.207+0000 7f7ed4781740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x557539db8500 con 0x557539e9a4d0 2024-04-21T08:42:06.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.207+0000 7f7ed4781740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x557539ea6af0 con 0x557539ea3a00 2024-04-21T08:42:06.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.207+0000 7f7ed4781740 10 client.0 _ll_register_callbacks cb 0x557539d86b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:06.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.207+0000 7f7ecaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557539e9ae80 0x557539eabc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.207+0000 7f7ecaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557539e9ae80 0x557539eabc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40646/0 (socket says 172.21.15.86:40646) 2024-04-21T08:42:06.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.207+0000 7f7ecaffd640 1 -- 172.21.15.86:0/2595518840 learned_addr learned my addr 172.21.15.86:0/2595518840 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:06.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.207+0000 7f7ecb7fe640 1 --2- 172.21.15.86:0/2595518840 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557539e9a4d0 0x557539eab740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.208+0000 7f7ecbfff640 1 --2- 172.21.15.86:0/2595518840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557539ea3a00 0x557539ea2770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.208+0000 7f7ecaffd640 1 -- 172.21.15.86:0/2595518840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557539ea3a00 msgr2=0x557539ea2770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.208+0000 7f7ecaffd640 1 --2- 172.21.15.86:0/2595518840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557539ea3a00 0x557539ea2770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.208+0000 7f7ecaffd640 1 -- 172.21.15.86:0/2595518840 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557539e9a4d0 msgr2=0x557539eab740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.208+0000 7f7ecaffd640 1 --2- 172.21.15.86:0/2595518840 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557539e9a4d0 0x557539eab740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.208+0000 7f7ecaffd640 1 -- 172.21.15.86:0/2595518840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557539b113f0 con 0x557539e9ae80 2024-04-21T08:42:06.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.208+0000 7f7ecb7fe640 1 --2- 172.21.15.86:0/2595518840 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557539e9a4d0 0x557539eab740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:06.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.208+0000 7f7ecaffd640 1 --2- 172.21.15.86:0/2595518840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557539e9ae80 0x557539eabc80 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f7eb406f680 tx=0x7f7eb406c4a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c578bd17ddad0a56 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:06.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.208+0000 7f7eaaffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:06.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.208+0000 7f7eaaffd640 1 -- 172.21.15.86:0/2595518840 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7eb40676d0 con 0x557539e9ae80 2024-04-21T08:42:06.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.208+0000 7f7eaaffd640 1 -- 172.21.15.86:0/2595518840 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f7eb4099230 con 0x557539e9ae80 2024-04-21T08:42:06.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.208+0000 7f7ed4781740 1 -- 172.21.15.86:0/2595518840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ebc004d20 con 0x557539e9ae80 2024-04-21T08:42:06.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.208+0000 7f7eaaffd640 1 -- 172.21.15.86:0/2595518840 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7eb409d2c0 con 0x557539e9ae80 2024-04-21T08:42:06.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.208+0000 7f7ed4781740 10 client.17190 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:06.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.208+0000 7f7ed4781740 1 -- 172.21.15.86:0/2595518840 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557539bdb7d0 con 0x557539e9ae80 2024-04-21T08:42:06.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.208+0000 7f7eaaffd640 1 -- 172.21.15.86:0/2595518840 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ebc004d20 con 0x557539e9ae80 2024-04-21T08:42:06.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.208+0000 7f7eaaffd640 1 -- 172.21.15.86:0/2595518840 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7eb40ce5f0 con 0x557539e9ae80 2024-04-21T08:42:06.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.208+0000 7f7ed4781740 10 client.17190 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:06.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.208+0000 7f7ed4781740 10 client.17190 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:42:06.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.208+0000 7f7ed4781740 20 client.17190 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:06.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.208+0000 7f7ed4781740 1 --2- 172.21.15.86:0/2595518840 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x557539eb5ca0 0x557539eb8090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.208+0000 7f7ed4781740 4 client.17190 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:06.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.208+0000 7f7ed4781740 1 -- 172.21.15.86:0/2595518840 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557539d9e930 con 0x557539eb5ca0 2024-04-21T08:42:06.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.209+0000 7f7ecb7fe640 1 --2- 172.21.15.86:0/2595518840 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x557539eb5ca0 0x557539eb8090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.209+0000 7f7ecb7fe640 1 --2- 172.21.15.86:0/2595518840 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x557539eb5ca0 0x557539eb8090 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7136d76fca1a52c0 server_cookie=fdbc10cc94f998aa in_seq=0 out_seq=0 2024-04-21T08:42:06.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.209+0000 7f7eaaffd640 10 client.17190 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:42:06.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.210+0000 7f7eaaffd640 1 -- 172.21.15.86:0/2595518840 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557539d9e930 con 0x557539eb5ca0 2024-04-21T08:42:06.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.210+0000 7f7eaaffd640 10 client.17190 handle_command_reply: tid=0 2024-04-21T08:42:06.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.210+0000 7f7ed4781740 1 client.17190 shutdown 2024-04-21T08:42:06.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.210+0000 7f7ed4781740 1 -- 172.21.15.86:0/2595518840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557539e9ae80 msgr2=0x557539eabc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.210+0000 7f7ed4781740 1 --2- 172.21.15.86:0/2595518840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557539e9ae80 0x557539eabc80 secure :-1 s=READY pgs=1007 cs=0 l=1 rev1=1 crypto rx=0x7f7eb406f680 tx=0x7f7eb406c4a0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.210+0000 7f7ed4781740 1 -- 172.21.15.86:0/2595518840 shutdown_connections 2024-04-21T08:42:06.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.210+0000 7f7ed4781740 1 --2- 172.21.15.86:0/2595518840 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x557539eb5ca0 0x557539eb8090 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.210+0000 7f7ed4781740 1 --2- 172.21.15.86:0/2595518840 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557539ea3a00 0x557539ea2770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.210+0000 7f7ed4781740 1 --2- 172.21.15.86:0/2595518840 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557539e9ae80 0x557539eabc80 unknown :-1 s=CLOSED pgs=1007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.210+0000 7f7eaaffd640 0 client.17190 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:42:06.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.210+0000 7f7ed4781740 1 --2- 172.21.15.86:0/2595518840 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557539e9a4d0 0x557539eab740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.210+0000 7f7ed4781740 1 -- 172.21.15.86:0/2595518840 >> 172.21.15.86:0/2595518840 conn(0x557539e94ba0 msgr2=0x557539e968f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:06.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.210+0000 7f7ed4781740 1 -- 172.21.15.86:0/2595518840 shutdown_connections 2024-04-21T08:42:06.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.210+0000 7f7ed4781740 1 -- 172.21.15.86:0/2595518840 wait complete. 2024-04-21T08:42:06.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.210+0000 7f7ed4781740 20 client.17190 trim_cache size 0 max 16384 2024-04-21T08:42:06.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.223+0000 7f7ed4781740 1 Processor -- start 2024-04-21T08:42:06.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.223+0000 7f7ed4781740 1 -- start start 2024-04-21T08:42:06.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.223+0000 7f7ed4781740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557539e9a4d0 0x557539ebd910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.223+0000 7f7ed4781740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557539e9ae80 0x557539ebde50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.223+0000 7f7ed4781740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557539ea3a00 0x557539e9c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.223+0000 7f7ed4781740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc1ad0d0 con 0x557539ea3a00 2024-04-21T08:42:06.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.223+0000 7f7ed4781740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc1aec10 con 0x557539e9a4d0 2024-04-21T08:42:06.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.223+0000 7f7ed4781740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc1ad7a0 con 0x557539e9ae80 2024-04-21T08:42:06.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.223+0000 7f7ecbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557539ea3a00 0x557539e9c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.223+0000 7f7ecbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557539ea3a00 0x557539e9c3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40656/0 (socket says 172.21.15.86:40656) 2024-04-21T08:42:06.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.223+0000 7f7ecbfff640 1 -- 172.21.15.86:0/1666232902 learned_addr learned my addr 172.21.15.86:0/1666232902 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:06.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.224+0000 7f7ecb7fe640 1 --2- 172.21.15.86:0/1666232902 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557539e9a4d0 0x557539ebd910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.224+0000 7f7ecbfff640 1 -- 172.21.15.86:0/1666232902 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557539e9ae80 msgr2=0x557539ebde50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.224+0000 7f7ecbfff640 1 --2- 172.21.15.86:0/1666232902 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557539e9ae80 0x557539ebde50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.224+0000 7f7ecbfff640 1 -- 172.21.15.86:0/1666232902 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557539e9a4d0 msgr2=0x557539ebd910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.224+0000 7f7ecbfff640 1 --2- 172.21.15.86:0/1666232902 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557539e9a4d0 0x557539ebd910 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.224+0000 7f7ecbfff640 1 -- 172.21.15.86:0/1666232902 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557539e94310 con 0x557539ea3a00 2024-04-21T08:42:06.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.224+0000 7f7ecbfff640 1 --2- 172.21.15.86:0/1666232902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557539ea3a00 0x557539e9c3e0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4009900 tx=0x7f7ec403d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd47bb880bb6305e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:06.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.224+0000 7f7ec17fa640 1 -- 172.21.15.86:0/1666232902 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec402cc60 con 0x557539ea3a00 2024-04-21T08:42:06.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.224+0000 7f7ec17fa640 1 -- 172.21.15.86:0/1666232902 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ec4042040 con 0x557539ea3a00 2024-04-21T08:42:06.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.224+0000 7f7ec17fa640 1 -- 172.21.15.86:0/1666232902 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec40035f0 con 0x557539ea3a00 2024-04-21T08:42:06.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.224+0000 7f7ed4781740 1 -- 172.21.15.86:0/1666232902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557539ea3a00 msgr2=0x557539e9c3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.224+0000 7f7ed4781740 1 --2- 172.21.15.86:0/1666232902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557539ea3a00 0x557539e9c3e0 secure :-1 s=READY pgs=1008 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4009900 tx=0x7f7ec403d000 comp rx=0 tx=0).stop 2024-04-21T08:42:06.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.225+0000 7f7ed4781740 1 -- 172.21.15.86:0/1666232902 shutdown_connections 2024-04-21T08:42:06.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.225+0000 7f7ed4781740 1 --2- 172.21.15.86:0/1666232902 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557539ea3a00 0x557539e9c3e0 unknown :-1 s=CLOSED pgs=1008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.225+0000 7f7ed4781740 1 --2- 172.21.15.86:0/1666232902 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557539e9ae80 0x557539ebde50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.225+0000 7f7ed4781740 1 --2- 172.21.15.86:0/1666232902 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557539e9a4d0 0x557539ebd910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.225+0000 7f7ed4781740 1 -- 172.21.15.86:0/1666232902 >> 172.21.15.86:0/1666232902 conn(0x557539e94ba0 msgr2=0x557539e93c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:06.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.225+0000 7f7ed4781740 1 -- 172.21.15.86:0/1666232902 shutdown_connections 2024-04-21T08:42:06.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.225+0000 7f7ed4781740 1 -- 172.21.15.86:0/1666232902 wait complete. 2024-04-21T08:42:06.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.225+0000 7f7ed4781740 1 -- start start 2024-04-21T08:42:06.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.225+0000 7f7ed4781740 1 Processor -- start 2024-04-21T08:42:06.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.225+0000 7f7ed4781740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557539e9a4d0 0x557539eadec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.225+0000 7f7ed4781740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557539e9ae80 0x557539eae400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.225+0000 7f7ed4781740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557539ea3a00 0x557539f1b180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.225+0000 7f7ed4781740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc1ae540 con 0x557539ea3a00 2024-04-21T08:42:06.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.225+0000 7f7ed4781740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc1ade70 con 0x557539e9a4d0 2024-04-21T08:42:06.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.225+0000 7f7ed4781740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7ecc1ab090 con 0x557539e9ae80 2024-04-21T08:42:06.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.225+0000 7f7ed4781740 10 client.0 _ll_register_callbacks cb 0x7f7eb409c990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:06.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.225+0000 7f7ecbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557539ea3a00 0x557539f1b180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.225+0000 7f7ecbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557539ea3a00 0x557539f1b180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40662/0 (socket says 172.21.15.86:40662) 2024-04-21T08:42:06.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.225+0000 7f7ecbfff640 1 -- 172.21.15.86:0/2173403574 learned_addr learned my addr 172.21.15.86:0/2173403574 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:06.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.226+0000 7f7ecaffd640 1 --2- 172.21.15.86:0/2173403574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557539e9ae80 0x557539eae400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.226+0000 7f7ecb7fe640 1 --2- 172.21.15.86:0/2173403574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557539e9a4d0 0x557539eadec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.226+0000 7f7ecbfff640 1 -- 172.21.15.86:0/2173403574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557539e9ae80 msgr2=0x557539eae400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.226+0000 7f7ecbfff640 1 --2- 172.21.15.86:0/2173403574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557539e9ae80 0x557539eae400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.226+0000 7f7ecbfff640 1 -- 172.21.15.86:0/2173403574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557539e9a4d0 msgr2=0x557539eadec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.226+0000 7f7ecbfff640 1 --2- 172.21.15.86:0/2173403574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557539e9a4d0 0x557539eadec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.226+0000 7f7ecbfff640 1 -- 172.21.15.86:0/2173403574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557539f1b770 con 0x557539ea3a00 2024-04-21T08:42:06.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.226+0000 7f7ecaffd640 1 --2- 172.21.15.86:0/2173403574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557539e9ae80 0x557539eae400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:06.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.226+0000 7f7ecb7fe640 1 --2- 172.21.15.86:0/2173403574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557539e9a4d0 0x557539eadec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:06.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.226+0000 7f7ecbfff640 1 --2- 172.21.15.86:0/2173403574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557539ea3a00 0x557539f1b180 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f7ec402c780 tx=0x7f7ec4004610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95b28cc1a3a61987 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:06.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.226+0000 7f7eaaffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:06.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.226+0000 7f7eaaffd640 1 -- 172.21.15.86:0/2173403574 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7ec4003e30 con 0x557539ea3a00 2024-04-21T08:42:06.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.226+0000 7f7eaaffd640 1 -- 172.21.15.86:0/2173403574 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f7ec4004c90 con 0x557539ea3a00 2024-04-21T08:42:06.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.226+0000 7f7ed4781740 1 -- 172.21.15.86:0/2173403574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7ec4042040 con 0x557539ea3a00 2024-04-21T08:42:06.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.226+0000 7f7eaaffd640 1 -- 172.21.15.86:0/2173403574 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7ec4064050 con 0x557539ea3a00 2024-04-21T08:42:06.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.226+0000 7f7ed4781740 10 client.17196 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:06.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.226+0000 7f7ed4781740 1 -- 172.21.15.86:0/2173403574 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557539f1ca50 con 0x557539ea3a00 2024-04-21T08:42:06.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.226+0000 7f7eaaffd640 1 -- 172.21.15.86:0/2173403574 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7ec4042040 con 0x557539ea3a00 2024-04-21T08:42:06.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.226+0000 7f7eaaffd640 1 -- 172.21.15.86:0/2173403574 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7ec4034710 con 0x557539ea3a00 2024-04-21T08:42:06.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.226+0000 7f7ed4781740 10 client.17196 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:06.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.226+0000 7f7ed4781740 10 client.17196 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:42:06.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.226+0000 7f7ed4781740 20 client.17196 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:06.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.226+0000 7f7ed4781740 1 --2- 172.21.15.86:0/2173403574 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x557539f1d170 0x557539f1f560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.226+0000 7f7ed4781740 4 client.17196 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:42:06.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.226+0000 7f7ed4781740 1 -- 172.21.15.86:0/2173403574 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557539e0cc10 con 0x557539f1d170 2024-04-21T08:42:06.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.227+0000 7f7ecb7fe640 1 --2- 172.21.15.86:0/2173403574 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x557539f1d170 0x557539f1f560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.227+0000 7f7eaaffd640 10 client.17196 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:42:06.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.227+0000 7f7ecb7fe640 1 --2- 172.21.15.86:0/2173403574 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x557539f1d170 0x557539f1f560 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=58a2a4ebd969d1bc server_cookie=65ea568de0091e2f in_seq=0 out_seq=0 2024-04-21T08:42:06.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.227+0000 7f7eaaffd640 1 -- 172.21.15.86:0/2173403574 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557539e0cc10 con 0x557539f1d170 2024-04-21T08:42:06.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.227+0000 7f7eaaffd640 10 client.17196 handle_command_reply: tid=0 2024-04-21T08:42:06.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.227+0000 7f7ed4781740 1 client.17196 shutdown 2024-04-21T08:42:06.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.228+0000 7f7ed4781740 1 -- 172.21.15.86:0/2173403574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557539ea3a00 msgr2=0x557539f1b180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.228+0000 7f7ed4781740 1 --2- 172.21.15.86:0/2173403574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557539ea3a00 0x557539f1b180 secure :-1 s=READY pgs=1009 cs=0 l=1 rev1=1 crypto rx=0x7f7ec402c780 tx=0x7f7ec4004610 comp rx=0 tx=0).stop 2024-04-21T08:42:06.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.228+0000 7f7ed4781740 1 -- 172.21.15.86:0/2173403574 shutdown_connections 2024-04-21T08:42:06.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.228+0000 7f7ed4781740 1 --2- 172.21.15.86:0/2173403574 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x557539f1d170 0x557539f1f560 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.228+0000 7f7ed4781740 1 --2- 172.21.15.86:0/2173403574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557539ea3a00 0x557539f1b180 unknown :-1 s=CLOSED pgs=1009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.228+0000 7f7eaaffd640 0 client.17196 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:42:06.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.228+0000 7f7ed4781740 1 --2- 172.21.15.86:0/2173403574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557539e9ae80 0x557539eae400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.228+0000 7f7ed4781740 1 --2- 172.21.15.86:0/2173403574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557539e9a4d0 0x557539eadec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.228+0000 7f7ed4781740 1 -- 172.21.15.86:0/2173403574 >> 172.21.15.86:0/2173403574 conn(0x557539e94ba0 msgr2=0x557539e9d2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:06.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.228+0000 7f7ed4781740 1 -- 172.21.15.86:0/2173403574 shutdown_connections 2024-04-21T08:42:06.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.228+0000 7f7ed4781740 1 -- 172.21.15.86:0/2173403574 wait complete. 2024-04-21T08:42:06.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.228+0000 7f7ed4781740 20 client.17196 trim_cache size 0 max 16384 2024-04-21T08:42:06.236 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:42:06.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.228+0000 7f7ed1ac0640 1 -- 172.21.15.86:0/658592371 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7eb80a3d20 msgr2=0x7f7eb80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.228+0000 7f7ed1ac0640 1 --2- 172.21.15.86:0/658592371 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7eb80a3d20 0x7f7eb80a61e0 secure :-1 s=READY pgs=153 cs=0 l=1 rev1=1 crypto rx=0x7f7ec4005ab0 tx=0x7f7ec4009260 comp rx=0 tx=0).stop 2024-04-21T08:42:06.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.228+0000 7f7ed1ac0640 1 -- 172.21.15.86:0/658592371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ecc106170 msgr2=0x7f7ecc197b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.228+0000 7f7ed1ac0640 1 --2- 172.21.15.86:0/658592371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ecc106170 0x7f7ecc197b20 secure :-1 s=READY pgs=1005 cs=0 l=1 rev1=1 crypto rx=0x7f7eb400e9c0 tx=0x7f7eb400ee80 comp rx=0 tx=0).stop 2024-04-21T08:42:06.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.228+0000 7f7ed1ac0640 1 -- 172.21.15.86:0/658592371 shutdown_connections 2024-04-21T08:42:06.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.228+0000 7f7ed1ac0640 1 --2- 172.21.15.86:0/658592371 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7eb80a3d20 0x7f7eb80a61e0 unknown :-1 s=CLOSED pgs=153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.228+0000 7f7ed1ac0640 1 --2- 172.21.15.86:0/658592371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7ecc19bea0 0x7f7ecc19c350 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.228+0000 7f7ed1ac0640 1 --2- 172.21.15.86:0/658592371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7ecc106170 0x7f7ecc197b20 unknown :-1 s=CLOSED pgs=1005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.228+0000 7f7ed1ac0640 1 --2- 172.21.15.86:0/658592371 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7ecc1010a0 0x7f7ecc1975e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.228+0000 7f7ed1ac0640 1 -- 172.21.15.86:0/658592371 >> 172.21.15.86:0/658592371 conn(0x7f7ecc077eb0 msgr2=0x7f7ecc100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:06.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.228+0000 7f7ed1ac0640 1 -- 172.21.15.86:0/658592371 shutdown_connections 2024-04-21T08:42:06.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.229+0000 7f7ed1ac0640 1 -- 172.21.15.86:0/658592371 wait complete. 2024-04-21T08:42:06.239 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:42:06.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.339+0000 7f298be8c640 1 -- 172.21.15.86:0/4098447819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2984106170 msgr2=0x7f298410dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.339+0000 7f298be8c640 1 --2- 172.21.15.86:0/4098447819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2984106170 0x7f298410dd40 secure :-1 s=READY pgs=1010 cs=0 l=1 rev1=1 crypto rx=0x7f2980009900 tx=0x7f298002bc30 comp rx=0 tx=0).stop 2024-04-21T08:42:06.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.340+0000 7f298be8c640 1 -- 172.21.15.86:0/4098447819 shutdown_connections 2024-04-21T08:42:06.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.340+0000 7f298be8c640 1 --2- 172.21.15.86:0/4098447819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2984106170 0x7f298410dd40 unknown :-1 s=CLOSED pgs=1010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.340+0000 7f298be8c640 1 --2- 172.21.15.86:0/4098447819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2984101a50 0x7f2984105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.340+0000 7f298be8c640 1 --2- 172.21.15.86:0/4098447819 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f29841010a0 0x7f2984101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.340+0000 7f298be8c640 1 -- 172.21.15.86:0/4098447819 >> 172.21.15.86:0/4098447819 conn(0x7f2984077eb0 msgr2=0x7f29840782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:06.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.340+0000 7f298be8c640 1 -- 172.21.15.86:0/4098447819 shutdown_connections 2024-04-21T08:42:06.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.340+0000 7f298be8c640 1 -- 172.21.15.86:0/4098447819 wait complete. 2024-04-21T08:42:06.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.340+0000 7f298be8c640 1 Processor -- start 2024-04-21T08:42:06.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.340+0000 7f298be8c640 1 -- start start 2024-04-21T08:42:06.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.340+0000 7f298be8c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29841010a0 0x7f298410e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.340+0000 7f298be8c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2984106170 0x7f298410ec10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.340+0000 7f298be8c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f298410f150 0x7f29841a6880 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.340+0000 7f298be8c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29841121f0 con 0x7f29841010a0 2024-04-21T08:42:06.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.340+0000 7f298be8c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2984112070 con 0x7f298410f150 2024-04-21T08:42:06.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.340+0000 7f298be8c640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2984112370 con 0x7f2984106170 2024-04-21T08:42:06.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.340+0000 7f2989c28640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29841010a0 0x7f298410e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.341+0000 7f2989c28640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29841010a0 0x7f298410e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40678/0 (socket says 172.21.15.86:40678) 2024-04-21T08:42:06.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.341+0000 7f2989c28640 1 -- 172.21.15.86:0/1568057451 learned_addr learned my addr 172.21.15.86:0/1568057451 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:06.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.341+0000 7f2989427640 1 --2- 172.21.15.86:0/1568057451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2984106170 0x7f298410ec10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.341+0000 7f298a429640 1 --2- 172.21.15.86:0/1568057451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f298410f150 0x7f29841a6880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.341+0000 7f2989c28640 1 -- 172.21.15.86:0/1568057451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2984106170 msgr2=0x7f298410ec10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.341+0000 7f2989c28640 1 --2- 172.21.15.86:0/1568057451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2984106170 0x7f298410ec10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.341+0000 7f2989c28640 1 -- 172.21.15.86:0/1568057451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f298410f150 msgr2=0x7f29841a6880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.341+0000 7f2989c28640 1 --2- 172.21.15.86:0/1568057451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f298410f150 0x7f29841a6880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.341+0000 7f2989c28640 1 -- 172.21.15.86:0/1568057451 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f29841a6f80 con 0x7f29841010a0 2024-04-21T08:42:06.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.341+0000 7f298a429640 1 --2- 172.21.15.86:0/1568057451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f298410f150 0x7f29841a6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:06.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.341+0000 7f2989c28640 1 --2- 172.21.15.86:0/1568057451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29841010a0 0x7f298410e6d0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f297c00cce0 tx=0x7f297c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:06.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.341+0000 7f297affd640 1 -- 172.21.15.86:0/1568057451 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c004b20 con 0x7f29841010a0 2024-04-21T08:42:06.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.341+0000 7f297affd640 1 -- 172.21.15.86:0/1568057451 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f297c004cc0 con 0x7f29841010a0 2024-04-21T08:42:06.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.341+0000 7f298be8c640 1 -- 172.21.15.86:0/1568057451 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f29841a7270 con 0x7f29841010a0 2024-04-21T08:42:06.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.341+0000 7f297affd640 1 -- 172.21.15.86:0/1568057451 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c00f6b0 con 0x7f29841010a0 2024-04-21T08:42:06.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.341+0000 7f298be8c640 1 -- 172.21.15.86:0/1568057451 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f29841a77c0 con 0x7f29841010a0 2024-04-21T08:42:06.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.343+0000 7f297affd640 1 -- 172.21.15.86:0/1568057451 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f297c00f850 con 0x7f29841010a0 2024-04-21T08:42:06.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.343+0000 7f298e722740 1 Processor -- start 2024-04-21T08:42:06.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.344+0000 7f298e722740 1 -- start start 2024-04-21T08:42:06.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.344+0000 7f298e722740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f8c224d0 0x5614f8c228b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.344+0000 7f298e722740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5614f8c22e80 0x5614f8c2b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.344+0000 7f298e722740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614f8c2ba00 0x5614f8c2de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.344+0000 7f298e722740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5614f8b871f0 con 0x5614f8c224d0 2024-04-21T08:42:06.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.344+0000 7f298e722740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5614f8b244b0 con 0x5614f8c22e80 2024-04-21T08:42:06.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.344+0000 7f298e722740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5614f8b831b0 con 0x5614f8c2ba00 2024-04-21T08:42:06.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.344+0000 7f297affd640 1 --2- 172.21.15.86:0/1568057451 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f29700a3d50 0x7f29700a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.344+0000 7f298a429640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f8c224d0 0x5614f8c228b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.344+0000 7f298a429640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f8c224d0 0x5614f8c228b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40694/0 (socket says 172.21.15.86:40694) 2024-04-21T08:42:06.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.344+0000 7f298a429640 1 -- 172.21.15.86:0/288132102 learned_addr learned my addr 172.21.15.86:0/288132102 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:06.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.344+0000 7f2989427640 1 --2- 172.21.15.86:0/288132102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614f8c2ba00 0x5614f8c2de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.344+0000 7f2989c28640 1 --2- 172.21.15.86:0/288132102 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5614f8c22e80 0x5614f8c2b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.344+0000 7f298a429640 1 --2- 172.21.15.86:0/1568057451 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f29700a3d50 0x7f29700a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.344+0000 7f297affd640 1 -- 172.21.15.86:0/1568057451 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f297c0d0cb0 con 0x7f29841010a0 2024-04-21T08:42:06.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.344+0000 7f298a429640 1 -- 172.21.15.86:0/288132102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614f8c2ba00 msgr2=0x5614f8c2de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.344+0000 7f298a429640 1 --2- 172.21.15.86:0/288132102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614f8c2ba00 0x5614f8c2de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.344+0000 7f298a429640 1 -- 172.21.15.86:0/288132102 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5614f8c22e80 msgr2=0x5614f8c2b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.344+0000 7f298a429640 1 --2- 172.21.15.86:0/288132102 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5614f8c22e80 0x5614f8c2b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.344+0000 7f298a429640 1 -- 172.21.15.86:0/288132102 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614f8b161b0 con 0x5614f8c224d0 2024-04-21T08:42:06.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.344+0000 7f2989c28640 1 --2- 172.21.15.86:0/288132102 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5614f8c22e80 0x5614f8c2b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:06.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.344+0000 7f2989427640 1 --2- 172.21.15.86:0/288132102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614f8c2ba00 0x5614f8c2de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:42:06.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.344+0000 7f298a429640 1 --2- 172.21.15.86:0/288132102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f8c224d0 0x5614f8c228b0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f298002c780 tx=0x7f29800044c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f89f11bb4426eb1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:06.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.345+0000 7f296ffff640 1 -- 172.21.15.86:0/288132102 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298002ca80 con 0x5614f8c224d0 2024-04-21T08:42:06.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.345+0000 7f296ffff640 1 -- 172.21.15.86:0/288132102 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f298002cc20 con 0x5614f8c224d0 2024-04-21T08:42:06.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.345+0000 7f296ffff640 1 -- 172.21.15.86:0/288132102 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f29800038a0 con 0x5614f8c224d0 2024-04-21T08:42:06.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.345+0000 7f298e722740 1 -- 172.21.15.86:0/288132102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f8c224d0 msgr2=0x5614f8c228b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.345+0000 7f298a429640 1 --2- 172.21.15.86:0/1568057451 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f29700a3d50 0x7f29700a6210 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2980005ec0 tx=0x7f2980048000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:06.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.345+0000 7f298e722740 1 --2- 172.21.15.86:0/288132102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f8c224d0 0x5614f8c228b0 secure :-1 s=READY pgs=1012 cs=0 l=1 rev1=1 crypto rx=0x7f298002c780 tx=0x7f29800044c0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.345+0000 7f298e722740 1 -- 172.21.15.86:0/288132102 shutdown_connections 2024-04-21T08:42:06.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.345+0000 7f298e722740 1 --2- 172.21.15.86:0/288132102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614f8c2ba00 0x5614f8c2de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.345+0000 7f298e722740 1 --2- 172.21.15.86:0/288132102 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5614f8c22e80 0x5614f8c2b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.345+0000 7f298e722740 1 --2- 172.21.15.86:0/288132102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f8c224d0 0x5614f8c228b0 unknown :-1 s=CLOSED pgs=1012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.345+0000 7f298e722740 1 -- 172.21.15.86:0/288132102 >> 172.21.15.86:0/288132102 conn(0x5614f8c1cba0 msgr2=0x5614f8c1cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:06.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.345+0000 7f298e722740 1 -- 172.21.15.86:0/288132102 shutdown_connections 2024-04-21T08:42:06.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.346+0000 7f298e722740 1 -- 172.21.15.86:0/288132102 wait complete. 2024-04-21T08:42:06.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.346+0000 7f298e722740 1 -- start start 2024-04-21T08:42:06.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.346+0000 7f298e722740 1 Processor -- start 2024-04-21T08:42:06.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.346+0000 7f298e722740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5614f8c224d0 0x5614f8c2b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.346+0000 7f298e722740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614f8c22e80 0x5614f8c28700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.346+0000 7f298e722740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f8c2ba00 0x5614f8c28d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.346+0000 7f298e722740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5614f8c2e970 con 0x5614f8c2ba00 2024-04-21T08:42:06.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.346+0000 7f298e722740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5614f8b40500 con 0x5614f8c224d0 2024-04-21T08:42:06.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.346+0000 7f298e722740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5614f8c2eaf0 con 0x5614f8c22e80 2024-04-21T08:42:06.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.346+0000 7f298e722740 10 client.0 _ll_register_callbacks cb 0x5614f8b0eb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:06.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.346+0000 7f298a429640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f8c2ba00 0x5614f8c28d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.346+0000 7f298a429640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f8c2ba00 0x5614f8c28d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40704/0 (socket says 172.21.15.86:40704) 2024-04-21T08:42:06.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.346+0000 7f298a429640 1 -- 172.21.15.86:0/2074456925 learned_addr learned my addr 172.21.15.86:0/2074456925 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:06.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.346+0000 7f2989c28640 1 --2- 172.21.15.86:0/2074456925 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5614f8c224d0 0x5614f8c2b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.347+0000 7f2989427640 1 --2- 172.21.15.86:0/2074456925 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614f8c22e80 0x5614f8c28700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.347+0000 7f298a429640 1 -- 172.21.15.86:0/2074456925 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614f8c22e80 msgr2=0x5614f8c28700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.347+0000 7f298a429640 1 --2- 172.21.15.86:0/2074456925 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614f8c22e80 0x5614f8c28700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.347+0000 7f298a429640 1 -- 172.21.15.86:0/2074456925 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5614f8c224d0 msgr2=0x5614f8c2b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.347+0000 7f298a429640 1 --2- 172.21.15.86:0/2074456925 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5614f8c224d0 0x5614f8c2b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.347+0000 7f298a429640 1 -- 172.21.15.86:0/2074456925 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614f88993f0 con 0x5614f8c2ba00 2024-04-21T08:42:06.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.347+0000 7f298a429640 1 --2- 172.21.15.86:0/2074456925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f8c2ba00 0x5614f8c28d10 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f298002c170 tx=0x7f298002cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=deb735eb21908d84 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:06.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.347+0000 7f296d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:06.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.347+0000 7f296d7fa640 1 -- 172.21.15.86:0/2074456925 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f298005c060 con 0x5614f8c2ba00 2024-04-21T08:42:06.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.347+0000 7f296d7fa640 1 -- 172.21.15.86:0/2074456925 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f29800054d0 con 0x5614f8c2ba00 2024-04-21T08:42:06.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.347+0000 7f298e722740 1 -- 172.21.15.86:0/2074456925 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f298002cc20 con 0x5614f8c2ba00 2024-04-21T08:42:06.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.347+0000 7f296d7fa640 1 -- 172.21.15.86:0/2074456925 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2980063050 con 0x5614f8c2ba00 2024-04-21T08:42:06.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.347+0000 7f298e722740 10 client.17208 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:06.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.347+0000 7f298e722740 1 -- 172.21.15.86:0/2074456925 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5614f89637d0 con 0x5614f8c2ba00 2024-04-21T08:42:06.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.347+0000 7f296d7fa640 1 -- 172.21.15.86:0/2074456925 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f298002cc20 con 0x5614f8c2ba00 2024-04-21T08:42:06.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.347+0000 7f296d7fa640 1 -- 172.21.15.86:0/2074456925 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2980040070 con 0x5614f8c2ba00 2024-04-21T08:42:06.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.348+0000 7f298e722740 10 client.17208 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:06.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.348+0000 7f298e722740 10 client.17208 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:42:06.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.348+0000 7f298e722740 20 client.17208 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:06.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.348+0000 7f298e722740 1 --2- 172.21.15.86:0/2074456925 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5614f8c3ba50 0x5614f8c3de40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.348+0000 7f298e722740 4 client.17208 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:06.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.348+0000 7f298e722740 1 -- 172.21.15.86:0/2074456925 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5614f8b26930 con 0x5614f8c3ba50 2024-04-21T08:42:06.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.348+0000 7f2989c28640 1 --2- 172.21.15.86:0/2074456925 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5614f8c3ba50 0x5614f8c3de40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.348+0000 7f296d7fa640 10 client.17208 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:42:06.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.348+0000 7f2989c28640 1 --2- 172.21.15.86:0/2074456925 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5614f8c3ba50 0x5614f8c3de40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=399f0703088f4e45 server_cookie=c171e574335a0672 in_seq=0 out_seq=0 2024-04-21T08:42:06.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.349+0000 7f296d7fa640 1 -- 172.21.15.86:0/2074456925 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5614f8b26930 con 0x5614f8c3ba50 2024-04-21T08:42:06.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.349+0000 7f296d7fa640 10 client.17208 handle_command_reply: tid=0 2024-04-21T08:42:06.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.349+0000 7f298e722740 1 client.17208 shutdown 2024-04-21T08:42:06.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.349+0000 7f298e722740 1 -- 172.21.15.86:0/2074456925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f8c2ba00 msgr2=0x5614f8c28d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.349+0000 7f298e722740 1 --2- 172.21.15.86:0/2074456925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f8c2ba00 0x5614f8c28d10 secure :-1 s=READY pgs=1013 cs=0 l=1 rev1=1 crypto rx=0x7f298002c170 tx=0x7f298002cdd0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.349+0000 7f298e722740 1 -- 172.21.15.86:0/2074456925 shutdown_connections 2024-04-21T08:42:06.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.349+0000 7f298e722740 1 --2- 172.21.15.86:0/2074456925 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5614f8c3ba50 0x5614f8c3de40 crc :-1 s=READY pgs=45 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.349+0000 7f298e722740 1 --2- 172.21.15.86:0/2074456925 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f8c2ba00 0x5614f8c28d10 unknown :-1 s=CLOSED pgs=1013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.349+0000 7f298e722740 1 --2- 172.21.15.86:0/2074456925 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614f8c22e80 0x5614f8c28700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.349+0000 7f298e722740 1 --2- 172.21.15.86:0/2074456925 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5614f8c224d0 0x5614f8c2b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.349+0000 7f296d7fa640 0 client.17208 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:42:06.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.349+0000 7f298e722740 1 -- 172.21.15.86:0/2074456925 >> 172.21.15.86:0/2074456925 conn(0x5614f8c1cba0 msgr2=0x5614f8c1e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:06.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.349+0000 7f298e722740 1 -- 172.21.15.86:0/2074456925 shutdown_connections 2024-04-21T08:42:06.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.349+0000 7f298e722740 1 -- 172.21.15.86:0/2074456925 wait complete. 2024-04-21T08:42:06.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.349+0000 7f298e722740 20 client.17208 trim_cache size 0 max 16384 2024-04-21T08:42:06.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.362+0000 7f298e722740 1 Processor -- start 2024-04-21T08:42:06.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.363+0000 7f298e722740 1 -- start start 2024-04-21T08:42:06.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.363+0000 7f298e722740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f8c224d0 0x5614f8c37450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.363+0000 7f298e722740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5614f8c22e80 0x5614f8c2ae30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.363+0000 7f298e722740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614f8c2ba00 0x5614f8c2b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.363+0000 7f298e722740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29841b1470 con 0x5614f8c224d0 2024-04-21T08:42:06.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.363+0000 7f298e722740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f29841b2fb0 con 0x5614f8c22e80 2024-04-21T08:42:06.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.363+0000 7f298e722740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f29841b1b40 con 0x5614f8c2ba00 2024-04-21T08:42:06.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.363+0000 7f2989c28640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f8c224d0 0x5614f8c37450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.363+0000 7f2989c28640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f8c224d0 0x5614f8c37450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40714/0 (socket says 172.21.15.86:40714) 2024-04-21T08:42:06.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.363+0000 7f2989c28640 1 -- 172.21.15.86:0/901216999 learned_addr learned my addr 172.21.15.86:0/901216999 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:06.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.363+0000 7f2989427640 1 --2- 172.21.15.86:0/901216999 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5614f8c22e80 0x5614f8c2ae30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.363+0000 7f298a429640 1 --2- 172.21.15.86:0/901216999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614f8c2ba00 0x5614f8c2b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.363+0000 7f2989c28640 1 -- 172.21.15.86:0/901216999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614f8c2ba00 msgr2=0x5614f8c2b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.363+0000 7f2989c28640 1 --2- 172.21.15.86:0/901216999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614f8c2ba00 0x5614f8c2b370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.363+0000 7f2989c28640 1 -- 172.21.15.86:0/901216999 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5614f8c22e80 msgr2=0x5614f8c2ae30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.363+0000 7f2989c28640 1 --2- 172.21.15.86:0/901216999 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5614f8c22e80 0x5614f8c2ae30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.363+0000 7f2989c28640 1 -- 172.21.15.86:0/901216999 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5614f8c1d4e0 con 0x5614f8c224d0 2024-04-21T08:42:06.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.363+0000 7f2989c28640 1 --2- 172.21.15.86:0/901216999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f8c224d0 0x5614f8c37450 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f297c09c870 tx=0x7f297c01c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48ba0daebac99123 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:06.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.363+0000 7f296f7fe640 1 -- 172.21.15.86:0/901216999 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c0a5550 con 0x5614f8c224d0 2024-04-21T08:42:06.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.363+0000 7f296f7fe640 1 -- 172.21.15.86:0/901216999 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f297c09abe0 con 0x5614f8c224d0 2024-04-21T08:42:06.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.363+0000 7f296f7fe640 1 -- 172.21.15.86:0/901216999 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c0a50c0 con 0x5614f8c224d0 2024-04-21T08:42:06.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.364+0000 7f298e722740 1 -- 172.21.15.86:0/901216999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f8c224d0 msgr2=0x5614f8c37450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.364+0000 7f298e722740 1 --2- 172.21.15.86:0/901216999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f8c224d0 0x5614f8c37450 secure :-1 s=READY pgs=1014 cs=0 l=1 rev1=1 crypto rx=0x7f297c09c870 tx=0x7f297c01c7e0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.364+0000 7f298e722740 1 -- 172.21.15.86:0/901216999 shutdown_connections 2024-04-21T08:42:06.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.364+0000 7f298e722740 1 --2- 172.21.15.86:0/901216999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614f8c2ba00 0x5614f8c2b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.364+0000 7f298e722740 1 --2- 172.21.15.86:0/901216999 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5614f8c22e80 0x5614f8c2ae30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.364+0000 7f298e722740 1 --2- 172.21.15.86:0/901216999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f8c224d0 0x5614f8c37450 unknown :-1 s=CLOSED pgs=1014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.364+0000 7f298e722740 1 -- 172.21.15.86:0/901216999 >> 172.21.15.86:0/901216999 conn(0x5614f8c1cba0 msgr2=0x5614f8c36d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:06.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.364+0000 7f298e722740 1 -- 172.21.15.86:0/901216999 shutdown_connections 2024-04-21T08:42:06.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.364+0000 7f298e722740 1 -- 172.21.15.86:0/901216999 wait complete. 2024-04-21T08:42:06.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.364+0000 7f298e722740 1 -- start start 2024-04-21T08:42:06.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.364+0000 7f298e722740 1 Processor -- start 2024-04-21T08:42:06.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.365+0000 7f298e722740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f8c224d0 0x5614f8c41240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.365+0000 7f298e722740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614f8c22e80 0x5614f8c3e5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.365+0000 7f298e722740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5614f8c2ba00 0x5614f8c3eb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.365+0000 7f298e722740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f29841b28e0 con 0x5614f8c224d0 2024-04-21T08:42:06.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.365+0000 7f298e722740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f29841b2210 con 0x5614f8c2ba00 2024-04-21T08:42:06.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.365+0000 7f298e722740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f29841af430 con 0x5614f8c22e80 2024-04-21T08:42:06.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.365+0000 7f298e722740 10 client.0 _ll_register_callbacks cb 0x7f2980003990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:06.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.365+0000 7f2989c28640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f8c224d0 0x5614f8c41240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.365+0000 7f2989c28640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f8c224d0 0x5614f8c41240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40726/0 (socket says 172.21.15.86:40726) 2024-04-21T08:42:06.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.365+0000 7f2989c28640 1 -- 172.21.15.86:0/1719235038 learned_addr learned my addr 172.21.15.86:0/1719235038 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:06.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.365+0000 7f298a429640 1 --2- 172.21.15.86:0/1719235038 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5614f8c2ba00 0x5614f8c3eb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.365+0000 7f2989427640 1 --2- 172.21.15.86:0/1719235038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614f8c22e80 0x5614f8c3e5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.365+0000 7f2989c28640 1 -- 172.21.15.86:0/1719235038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614f8c22e80 msgr2=0x5614f8c3e5d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.365+0000 7f2989c28640 1 --2- 172.21.15.86:0/1719235038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614f8c22e80 0x5614f8c3e5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.365+0000 7f2989c28640 1 -- 172.21.15.86:0/1719235038 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5614f8c2ba00 msgr2=0x5614f8c3eb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.365+0000 7f2989c28640 1 --2- 172.21.15.86:0/1719235038 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5614f8c2ba00 0x5614f8c3eb10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.365+0000 7f2989c28640 1 -- 172.21.15.86:0/1719235038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5614f8c3f2f0 con 0x5614f8c224d0 2024-04-21T08:42:06.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.365+0000 7f2989c28640 1 --2- 172.21.15.86:0/1719235038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f8c224d0 0x5614f8c41240 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f297c0a5020 tx=0x7f297c09adf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=711296e6b2994e5b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:06.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.365+0000 7f296d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:06.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.365+0000 7f296d7fa640 1 -- 172.21.15.86:0/1719235038 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f297c0bb060 con 0x5614f8c224d0 2024-04-21T08:42:06.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.366+0000 7f296d7fa640 1 -- 172.21.15.86:0/1719235038 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f297c087050 con 0x5614f8c224d0 2024-04-21T08:42:06.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.366+0000 7f298e722740 1 -- 172.21.15.86:0/1719235038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f297c09abe0 con 0x5614f8c224d0 2024-04-21T08:42:06.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.366+0000 7f296d7fa640 1 -- 172.21.15.86:0/1719235038 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f297c0ce510 con 0x5614f8c224d0 2024-04-21T08:42:06.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.366+0000 7f298e722740 10 client.17214 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:06.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.366+0000 7f298e722740 1 -- 172.21.15.86:0/1719235038 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5614f8ca1c40 con 0x5614f8c224d0 2024-04-21T08:42:06.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.366+0000 7f296d7fa640 1 -- 172.21.15.86:0/1719235038 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f297c09abe0 con 0x5614f8c224d0 2024-04-21T08:42:06.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.366+0000 7f296d7fa640 1 -- 172.21.15.86:0/1719235038 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f297c067170 con 0x5614f8c224d0 2024-04-21T08:42:06.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.366+0000 7f298e722740 10 client.17214 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:06.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.366+0000 7f298e722740 10 client.17214 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:42:06.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.366+0000 7f298e722740 20 client.17214 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:06.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.366+0000 7f298e722740 1 --2- 172.21.15.86:0/1719235038 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5614f8c3ba50 0x5614f8ca4270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.366+0000 7f298e722740 4 client.17214 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:42:06.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.366+0000 7f298e722740 1 -- 172.21.15.86:0/1719235038 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5614f8b94c10 con 0x5614f8c3ba50 2024-04-21T08:42:06.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.366+0000 7f2989427640 1 --2- 172.21.15.86:0/1719235038 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5614f8c3ba50 0x5614f8ca4270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.367+0000 7f296d7fa640 10 client.17214 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:42:06.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.367+0000 7f2989427640 1 --2- 172.21.15.86:0/1719235038 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5614f8c3ba50 0x5614f8ca4270 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=de29bdb014259ae5 server_cookie=fb62d1b43be3cfac in_seq=0 out_seq=0 2024-04-21T08:42:06.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.367+0000 7f296d7fa640 1 -- 172.21.15.86:0/1719235038 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5614f8b94c10 con 0x5614f8c3ba50 2024-04-21T08:42:06.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.367+0000 7f296d7fa640 10 client.17214 handle_command_reply: tid=0 2024-04-21T08:42:06.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.367+0000 7f298e722740 1 client.17214 shutdown 2024-04-21T08:42:06.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.367+0000 7f298e722740 1 -- 172.21.15.86:0/1719235038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f8c224d0 msgr2=0x5614f8c41240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.367+0000 7f298e722740 1 --2- 172.21.15.86:0/1719235038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f8c224d0 0x5614f8c41240 secure :-1 s=READY pgs=1015 cs=0 l=1 rev1=1 crypto rx=0x7f297c0a5020 tx=0x7f297c09adf0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.367+0000 7f298e722740 1 -- 172.21.15.86:0/1719235038 shutdown_connections 2024-04-21T08:42:06.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.367+0000 7f298e722740 1 --2- 172.21.15.86:0/1719235038 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5614f8c3ba50 0x5614f8ca4270 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.367+0000 7f298e722740 1 --2- 172.21.15.86:0/1719235038 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5614f8c2ba00 0x5614f8c3eb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.367+0000 7f298e722740 1 --2- 172.21.15.86:0/1719235038 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5614f8c22e80 0x5614f8c3e5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.367+0000 7f296d7fa640 0 client.17214 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:42:06.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.367+0000 7f298e722740 1 --2- 172.21.15.86:0/1719235038 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5614f8c224d0 0x5614f8c41240 unknown :-1 s=CLOSED pgs=1015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.367+0000 7f298e722740 1 -- 172.21.15.86:0/1719235038 >> 172.21.15.86:0/1719235038 conn(0x5614f8c1cba0 msgr2=0x5614f8c375d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:06.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.367+0000 7f298e722740 1 -- 172.21.15.86:0/1719235038 shutdown_connections 2024-04-21T08:42:06.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.367+0000 7f298e722740 1 -- 172.21.15.86:0/1719235038 wait complete. 2024-04-21T08:42:06.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.367+0000 7f298e722740 20 client.17214 trim_cache size 0 max 16384 2024-04-21T08:42:06.374 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:42:06.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.368+0000 7f298be8c640 1 -- 172.21.15.86:0/1568057451 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f29700a3d50 msgr2=0x7f29700a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.368+0000 7f298be8c640 1 --2- 172.21.15.86:0/1568057451 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f29700a3d50 0x7f29700a6210 secure :-1 s=READY pgs=154 cs=0 l=1 rev1=1 crypto rx=0x7f2980005ec0 tx=0x7f2980048000 comp rx=0 tx=0).stop 2024-04-21T08:42:06.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.368+0000 7f298be8c640 1 -- 172.21.15.86:0/1568057451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29841010a0 msgr2=0x7f298410e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.368+0000 7f298be8c640 1 --2- 172.21.15.86:0/1568057451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29841010a0 0x7f298410e6d0 secure :-1 s=READY pgs=1011 cs=0 l=1 rev1=1 crypto rx=0x7f297c00cce0 tx=0x7f297c00a620 comp rx=0 tx=0).stop 2024-04-21T08:42:06.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.368+0000 7f298be8c640 1 -- 172.21.15.86:0/1568057451 shutdown_connections 2024-04-21T08:42:06.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.368+0000 7f298be8c640 1 --2- 172.21.15.86:0/1568057451 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f29700a3d50 0x7f29700a6210 unknown :-1 s=CLOSED pgs=154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.368+0000 7f298be8c640 1 --2- 172.21.15.86:0/1568057451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f298410f150 0x7f29841a6880 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.368+0000 7f298be8c640 1 --2- 172.21.15.86:0/1568057451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2984106170 0x7f298410ec10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.368+0000 7f298be8c640 1 --2- 172.21.15.86:0/1568057451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f29841010a0 0x7f298410e6d0 unknown :-1 s=CLOSED pgs=1011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.368+0000 7f298be8c640 1 -- 172.21.15.86:0/1568057451 >> 172.21.15.86:0/1568057451 conn(0x7f2984077eb0 msgr2=0x7f2984100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:06.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.368+0000 7f298be8c640 1 -- 172.21.15.86:0/1568057451 shutdown_connections 2024-04-21T08:42:06.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.368+0000 7f298be8c640 1 -- 172.21.15.86:0/1568057451 wait complete. 2024-04-21T08:42:06.378 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:42:06.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.477+0000 7f4729532640 1 -- 172.21.15.86:0/3332288018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4724107030 msgr2=0x7f4724107410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.477+0000 7f4729532640 1 --2- 172.21.15.86:0/3332288018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4724107030 0x7f4724107410 secure :-1 s=READY pgs=1016 cs=0 l=1 rev1=1 crypto rx=0x7f4714009930 tx=0x7f471402bc30 comp rx=0 tx=0).stop 2024-04-21T08:42:06.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.477+0000 7f4729532640 1 -- 172.21.15.86:0/3332288018 shutdown_connections 2024-04-21T08:42:06.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.477+0000 7f4729532640 1 --2- 172.21.15.86:0/3332288018 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4724101b40 0x7f472410dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.477+0000 7f4729532640 1 --2- 172.21.15.86:0/3332288018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f47241011a0 0x7f4724101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.477+0000 7f4729532640 1 --2- 172.21.15.86:0/3332288018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4724107030 0x7f4724107410 unknown :-1 s=CLOSED pgs=1016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.477+0000 7f4729532640 1 -- 172.21.15.86:0/3332288018 >> 172.21.15.86:0/3332288018 conn(0x7f4724076c50 msgr2=0x7f4724077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:06.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.477+0000 7f4729532640 1 -- 172.21.15.86:0/3332288018 shutdown_connections 2024-04-21T08:42:06.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.477+0000 7f4729532640 1 -- 172.21.15.86:0/3332288018 wait complete. 2024-04-21T08:42:06.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.478+0000 7f4729532640 1 Processor -- start 2024-04-21T08:42:06.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.478+0000 7f4729532640 1 -- start start 2024-04-21T08:42:06.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.478+0000 7f4729532640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47241011a0 0x7f47241995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.478+0000 7f4729532640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4724101b40 0x7f4724199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.478+0000 7f4729532640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4724107030 0x7f47241a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.478+0000 7f4729532640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47241110c0 con 0x7f47241011a0 2024-04-21T08:42:06.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.478+0000 7f4729532640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4724110f40 con 0x7f4724107030 2024-04-21T08:42:06.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.478+0000 7f4729532640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4724111240 con 0x7f4724101b40 2024-04-21T08:42:06.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.478+0000 7f4722ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47241011a0 0x7f47241995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.478+0000 7f4722ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47241011a0 0x7f47241995e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40744/0 (socket says 172.21.15.86:40744) 2024-04-21T08:42:06.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.478+0000 7f4722ffd640 1 -- 172.21.15.86:0/329550375 learned_addr learned my addr 172.21.15.86:0/329550375 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:06.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.478+0000 7f47237fe640 1 --2- 172.21.15.86:0/329550375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4724107030 0x7f47241a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.478+0000 7f4722ffd640 1 -- 172.21.15.86:0/329550375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4724101b40 msgr2=0x7f4724199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.478+0000 7f471bfff640 1 --2- 172.21.15.86:0/329550375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4724101b40 0x7f4724199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.478+0000 7f4722ffd640 1 --2- 172.21.15.86:0/329550375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4724101b40 0x7f4724199b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.478+0000 7f4722ffd640 1 -- 172.21.15.86:0/329550375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4724107030 msgr2=0x7f47241a4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.478+0000 7f4722ffd640 1 --2- 172.21.15.86:0/329550375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4724107030 0x7f47241a4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.478+0000 7f4722ffd640 1 -- 172.21.15.86:0/329550375 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f47241a4ba0 con 0x7f47241011a0 2024-04-21T08:42:06.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.478+0000 7f47237fe640 1 --2- 172.21.15.86:0/329550375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4724107030 0x7f47241a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:06.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.479+0000 7f4722ffd640 1 --2- 172.21.15.86:0/329550375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47241011a0 0x7f47241995e0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f4714005940 tx=0x7f471402ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:06.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.479+0000 7f4720ff9640 1 -- 172.21.15.86:0/329550375 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4714043020 con 0x7f47241011a0 2024-04-21T08:42:06.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.479+0000 7f4720ff9640 1 -- 172.21.15.86:0/329550375 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f471402cc80 con 0x7f47241011a0 2024-04-21T08:42:06.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.479+0000 7f4729532640 1 -- 172.21.15.86:0/329550375 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f47241a4d70 con 0x7f47241011a0 2024-04-21T08:42:06.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.479+0000 7f4720ff9640 1 -- 172.21.15.86:0/329550375 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4714003c70 con 0x7f47241011a0 2024-04-21T08:42:06.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.479+0000 7f4729532640 1 -- 172.21.15.86:0/329550375 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f47241a5200 con 0x7f47241011a0 2024-04-21T08:42:06.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.481+0000 7f4720ff9640 1 -- 172.21.15.86:0/329550375 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4714046020 con 0x7f47241011a0 2024-04-21T08:42:06.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.481+0000 7f472c1ff740 1 Processor -- start 2024-04-21T08:42:06.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.481+0000 7f472c1ff740 1 -- start start 2024-04-21T08:42:06.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.482+0000 7f472c1ff740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b7676c4d0 0x558b7676c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.482+0000 7f472c1ff740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b7676ce80 0x558b76775430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.482+0000 7f472c1ff740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b76775a00 0x558b76777e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.482+0000 7f472c1ff740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558b766d11f0 con 0x558b7676ce80 2024-04-21T08:42:06.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.482+0000 7f472c1ff740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x558b7666e4b0 con 0x558b76775a00 2024-04-21T08:42:06.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.482+0000 7f472c1ff740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558b766cd1b0 con 0x558b7676c4d0 2024-04-21T08:42:06.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.482+0000 7f4722ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b7676ce80 0x558b76775430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.482+0000 7f4722ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b7676ce80 0x558b76775430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40754/0 (socket says 172.21.15.86:40754) 2024-04-21T08:42:06.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.482+0000 7f4722ffd640 1 -- 172.21.15.86:0/608090264 learned_addr learned my addr 172.21.15.86:0/608090264 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:06.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.482+0000 7f471bfff640 1 --2- 172.21.15.86:0/608090264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b76775a00 0x558b76777e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.482+0000 7f47237fe640 1 --2- 172.21.15.86:0/608090264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b7676c4d0 0x558b7676c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.482+0000 7f4720ff9640 1 --2- 172.21.15.86:0/329550375 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f47100a3d20 0x7f47100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.482+0000 7f4722ffd640 1 -- 172.21.15.86:0/608090264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b7676c4d0 msgr2=0x558b7676c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.482+0000 7f4722ffd640 1 --2- 172.21.15.86:0/608090264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b7676c4d0 0x558b7676c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.482+0000 7f4722ffd640 1 -- 172.21.15.86:0/608090264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b76775a00 msgr2=0x558b76777e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.482+0000 7f4722ffd640 1 --2- 172.21.15.86:0/608090264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b76775a00 0x558b76777e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.482+0000 7f4722ffd640 1 -- 172.21.15.86:0/608090264 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b766601b0 con 0x558b7676ce80 2024-04-21T08:42:06.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.482+0000 7f471bfff640 1 --2- 172.21.15.86:0/608090264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b76775a00 0x558b76777e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:06.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.482+0000 7f47237fe640 1 --2- 172.21.15.86:0/608090264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b7676c4d0 0x558b7676c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:06.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.482+0000 7f47237fe640 1 --2- 172.21.15.86:0/329550375 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f47100a3d20 0x7f47100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.482+0000 7f4720ff9640 1 -- 172.21.15.86:0/329550375 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f47140f2e50 con 0x7f47241011a0 2024-04-21T08:42:06.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.482+0000 7f4722ffd640 1 --2- 172.21.15.86:0/608090264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b7676ce80 0x558b76775430 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f47140f0280 tx=0x7f4714089190 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a77744def0babe6e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:06.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.483+0000 7f47237fe640 1 --2- 172.21.15.86:0/329550375 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f47100a3d20 0x7f47100a61e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f471c00af40 tx=0x7f471c004a40 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:06.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.483+0000 7f47197fa640 1 -- 172.21.15.86:0/608090264 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47140be6d0 con 0x558b7676ce80 2024-04-21T08:42:06.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.483+0000 7f47197fa640 1 -- 172.21.15.86:0/608090264 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f47140035f0 con 0x558b7676ce80 2024-04-21T08:42:06.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.483+0000 7f47197fa640 1 -- 172.21.15.86:0/608090264 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f47140f6a30 con 0x558b7676ce80 2024-04-21T08:42:06.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.483+0000 7f472c1ff740 1 -- 172.21.15.86:0/608090264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b7676ce80 msgr2=0x558b76775430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.483+0000 7f472c1ff740 1 --2- 172.21.15.86:0/608090264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b7676ce80 0x558b76775430 secure :-1 s=READY pgs=1018 cs=0 l=1 rev1=1 crypto rx=0x7f47140f0280 tx=0x7f4714089190 comp rx=0 tx=0).stop 2024-04-21T08:42:06.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.483+0000 7f472c1ff740 1 -- 172.21.15.86:0/608090264 shutdown_connections 2024-04-21T08:42:06.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.483+0000 7f472c1ff740 1 --2- 172.21.15.86:0/608090264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b76775a00 0x558b76777e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.483+0000 7f472c1ff740 1 --2- 172.21.15.86:0/608090264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b7676ce80 0x558b76775430 unknown :-1 s=CLOSED pgs=1018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.483+0000 7f472c1ff740 1 --2- 172.21.15.86:0/608090264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b7676c4d0 0x558b7676c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.483+0000 7f472c1ff740 1 -- 172.21.15.86:0/608090264 >> 172.21.15.86:0/608090264 conn(0x558b76766ba0 msgr2=0x558b76766f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:06.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.483+0000 7f472c1ff740 1 -- 172.21.15.86:0/608090264 shutdown_connections 2024-04-21T08:42:06.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.484+0000 7f472c1ff740 1 -- 172.21.15.86:0/608090264 wait complete. 2024-04-21T08:42:06.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.484+0000 7f472c1ff740 1 -- start start 2024-04-21T08:42:06.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.484+0000 7f472c1ff740 1 Processor -- start 2024-04-21T08:42:06.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.484+0000 7f472c1ff740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b7676c4d0 0x558b767752c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.484+0000 7f472c1ff740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b7676ce80 0x558b76772650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.484+0000 7f472c1ff740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b76775a00 0x558b76772bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.484+0000 7f472c1ff740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558b76778730 con 0x558b7676ce80 2024-04-21T08:42:06.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.484+0000 7f472c1ff740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x558b7668a500 con 0x558b76775a00 2024-04-21T08:42:06.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.484+0000 7f472c1ff740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558b767788b0 con 0x558b7676c4d0 2024-04-21T08:42:06.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.484+0000 7f472c1ff740 10 client.0 _ll_register_callbacks cb 0x558b76658b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:06.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.484+0000 7f471bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b7676ce80 0x558b76772650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.484+0000 7f471bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b7676ce80 0x558b76772650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40758/0 (socket says 172.21.15.86:40758) 2024-04-21T08:42:06.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.484+0000 7f471bfff640 1 -- 172.21.15.86:0/2261160513 learned_addr learned my addr 172.21.15.86:0/2261160513 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:06.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.484+0000 7f47237fe640 1 --2- 172.21.15.86:0/2261160513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b76775a00 0x558b76772bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.484+0000 7f4722ffd640 1 --2- 172.21.15.86:0/2261160513 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b7676c4d0 0x558b767752c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.485+0000 7f471bfff640 1 -- 172.21.15.86:0/2261160513 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b7676c4d0 msgr2=0x558b767752c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.485+0000 7f471bfff640 1 --2- 172.21.15.86:0/2261160513 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b7676c4d0 0x558b767752c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.485+0000 7f471bfff640 1 -- 172.21.15.86:0/2261160513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b76775a00 msgr2=0x558b76772bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.485+0000 7f471bfff640 1 --2- 172.21.15.86:0/2261160513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b76775a00 0x558b76772bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.485+0000 7f471bfff640 1 -- 172.21.15.86:0/2261160513 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b763e33f0 con 0x558b7676ce80 2024-04-21T08:42:06.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.485+0000 7f4722ffd640 1 --2- 172.21.15.86:0/2261160513 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b7676c4d0 0x558b767752c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:06.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.485+0000 7f471bfff640 1 --2- 172.21.15.86:0/2261160513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b7676ce80 0x558b76772650 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f470c005fd0 tx=0x7f470c00fb70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36d8b4813f032d7e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:06.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.485+0000 7f470a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:06.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.485+0000 7f470a7fc640 1 -- 172.21.15.86:0/2261160513 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470c016070 con 0x558b7676ce80 2024-04-21T08:42:06.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.485+0000 7f470a7fc640 1 -- 172.21.15.86:0/2261160513 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f470c00bb00 con 0x558b7676ce80 2024-04-21T08:42:06.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.485+0000 7f472c1ff740 1 -- 172.21.15.86:0/2261160513 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x558b76670930 con 0x558b7676ce80 2024-04-21T08:42:06.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.485+0000 7f470a7fc640 1 -- 172.21.15.86:0/2261160513 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f470c00bd00 con 0x558b7676ce80 2024-04-21T08:42:06.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.485+0000 7f472c1ff740 10 client.17226 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:06.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.485+0000 7f472c1ff740 1 -- 172.21.15.86:0/2261160513 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558b764ad7d0 con 0x558b7676ce80 2024-04-21T08:42:06.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.485+0000 7f470a7fc640 1 -- 172.21.15.86:0/2261160513 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x558b76670930 con 0x558b7676ce80 2024-04-21T08:42:06.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.485+0000 7f470a7fc640 1 -- 172.21.15.86:0/2261160513 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f470c004bb0 con 0x558b7676ce80 2024-04-21T08:42:06.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.485+0000 7f472c1ff740 10 client.17226 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:06.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.485+0000 7f472c1ff740 10 client.17226 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:42:06.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.485+0000 7f472c1ff740 20 client.17226 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:06.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.485+0000 7f472c1ff740 1 --2- 172.21.15.86:0/2261160513 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x558b76785980 0x558b76787da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.485+0000 7f472c1ff740 4 client.17226 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:06.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.485+0000 7f472c1ff740 1 -- 172.21.15.86:0/2261160513 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558b766dec10 con 0x558b76785980 2024-04-21T08:42:06.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.486+0000 7f4722ffd640 1 --2- 172.21.15.86:0/2261160513 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x558b76785980 0x558b76787da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.486+0000 7f4722ffd640 1 --2- 172.21.15.86:0/2261160513 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x558b76785980 0x558b76787da0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3330aabd33710b16 server_cookie=2e3a0ac5904ae04e in_seq=0 out_seq=0 2024-04-21T08:42:06.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.486+0000 7f470a7fc640 10 client.17226 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:42:06.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.487+0000 7f470a7fc640 1 -- 172.21.15.86:0/2261160513 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558b766dec10 con 0x558b76785980 2024-04-21T08:42:06.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.487+0000 7f470a7fc640 10 client.17226 handle_command_reply: tid=0 2024-04-21T08:42:06.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.487+0000 7f472c1ff740 1 client.17226 shutdown 2024-04-21T08:42:06.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.487+0000 7f472c1ff740 1 -- 172.21.15.86:0/2261160513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b7676ce80 msgr2=0x558b76772650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.487+0000 7f472c1ff740 1 --2- 172.21.15.86:0/2261160513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b7676ce80 0x558b76772650 secure :-1 s=READY pgs=1019 cs=0 l=1 rev1=1 crypto rx=0x7f470c005fd0 tx=0x7f470c00fb70 comp rx=0 tx=0).stop 2024-04-21T08:42:06.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.487+0000 7f472c1ff740 1 -- 172.21.15.86:0/2261160513 shutdown_connections 2024-04-21T08:42:06.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.487+0000 7f472c1ff740 1 --2- 172.21.15.86:0/2261160513 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b76775a00 0x558b76772bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.487+0000 7f472c1ff740 1 --2- 172.21.15.86:0/2261160513 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x558b76785980 0x558b76787da0 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.487+0000 7f472c1ff740 1 --2- 172.21.15.86:0/2261160513 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b7676ce80 0x558b76772650 unknown :-1 s=CLOSED pgs=1019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.487+0000 7f472c1ff740 1 --2- 172.21.15.86:0/2261160513 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b7676c4d0 0x558b767752c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.487+0000 7f470a7fc640 0 client.17226 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:42:06.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.487+0000 7f472c1ff740 1 -- 172.21.15.86:0/2261160513 >> 172.21.15.86:0/2261160513 conn(0x558b76766ba0 msgr2=0x558b767688f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:06.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.487+0000 7f472c1ff740 1 -- 172.21.15.86:0/2261160513 shutdown_connections 2024-04-21T08:42:06.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.487+0000 7f472c1ff740 1 -- 172.21.15.86:0/2261160513 wait complete. 2024-04-21T08:42:06.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.487+0000 7f472c1ff740 20 client.17226 trim_cache size 0 max 16384 2024-04-21T08:42:06.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.500+0000 7f472c1ff740 1 Processor -- start 2024-04-21T08:42:06.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.500+0000 7f472c1ff740 1 -- start start 2024-04-21T08:42:06.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.500+0000 7f472c1ff740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b7676c4d0 0x558b76781430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.500+0000 7f472c1ff740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b7676ce80 0x558b76766f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.500+0000 7f472c1ff740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b76775a00 0x558b76774d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.500+0000 7f472c1ff740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47241af370 con 0x558b7676ce80 2024-04-21T08:42:06.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.500+0000 7f472c1ff740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f47241b0eb0 con 0x558b7676c4d0 2024-04-21T08:42:06.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.500+0000 7f472c1ff740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f47241afa40 con 0x558b76775a00 2024-04-21T08:42:06.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.500+0000 7f471bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b7676ce80 0x558b76766f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.500+0000 7f471bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b7676ce80 0x558b76766f80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40774/0 (socket says 172.21.15.86:40774) 2024-04-21T08:42:06.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.500+0000 7f471bfff640 1 -- 172.21.15.86:0/963878661 learned_addr learned my addr 172.21.15.86:0/963878661 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:06.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.501+0000 7f47237fe640 1 --2- 172.21.15.86:0/963878661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b76775a00 0x558b76774d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.501+0000 7f471bfff640 1 -- 172.21.15.86:0/963878661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b76775a00 msgr2=0x558b76774d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.501+0000 7f471bfff640 1 --2- 172.21.15.86:0/963878661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b76775a00 0x558b76774d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.501+0000 7f471bfff640 1 -- 172.21.15.86:0/963878661 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b7676c4d0 msgr2=0x558b76781430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.501+0000 7f471bfff640 1 --2- 172.21.15.86:0/963878661 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b7676c4d0 0x558b76781430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.501+0000 7f471bfff640 1 -- 172.21.15.86:0/963878661 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558b76762470 con 0x558b7676ce80 2024-04-21T08:42:06.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.501+0000 7f47237fe640 1 --2- 172.21.15.86:0/963878661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b76775a00 0x558b76774d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:06.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.501+0000 7f471bfff640 1 --2- 172.21.15.86:0/963878661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b7676ce80 0x558b76766f80 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f470c0105f0 tx=0x7f470c00b820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70157e5465dd7d24 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:06.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.501+0000 7f4718ff9640 1 -- 172.21.15.86:0/963878661 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470c016040 con 0x558b7676ce80 2024-04-21T08:42:06.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.501+0000 7f4718ff9640 1 -- 172.21.15.86:0/963878661 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f470c00bb00 con 0x558b7676ce80 2024-04-21T08:42:06.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.501+0000 7f4718ff9640 1 -- 172.21.15.86:0/963878661 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f470c004d50 con 0x558b7676ce80 2024-04-21T08:42:06.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.502+0000 7f472c1ff740 1 -- 172.21.15.86:0/963878661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b7676ce80 msgr2=0x558b76766f80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.502+0000 7f472c1ff740 1 --2- 172.21.15.86:0/963878661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b7676ce80 0x558b76766f80 secure :-1 s=READY pgs=1020 cs=0 l=1 rev1=1 crypto rx=0x7f470c0105f0 tx=0x7f470c00b820 comp rx=0 tx=0).stop 2024-04-21T08:42:06.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.502+0000 7f472c1ff740 1 -- 172.21.15.86:0/963878661 shutdown_connections 2024-04-21T08:42:06.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.502+0000 7f472c1ff740 1 --2- 172.21.15.86:0/963878661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b76775a00 0x558b76774d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.502+0000 7f472c1ff740 1 --2- 172.21.15.86:0/963878661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b7676ce80 0x558b76766f80 unknown :-1 s=CLOSED pgs=1020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.502+0000 7f472c1ff740 1 --2- 172.21.15.86:0/963878661 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b7676c4d0 0x558b76781430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.502+0000 7f472c1ff740 1 -- 172.21.15.86:0/963878661 >> 172.21.15.86:0/963878661 conn(0x558b76766ba0 msgr2=0x558b7676dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:06.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.502+0000 7f472c1ff740 1 -- 172.21.15.86:0/963878661 shutdown_connections 2024-04-21T08:42:06.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.502+0000 7f472c1ff740 1 -- 172.21.15.86:0/963878661 wait complete. 2024-04-21T08:42:06.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.502+0000 7f472c1ff740 1 -- start start 2024-04-21T08:42:06.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.502+0000 7f472c1ff740 1 Processor -- start 2024-04-21T08:42:06.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.502+0000 7f472c1ff740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b7676c4d0 0x558b7677d940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.502+0000 7f472c1ff740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b7676ce80 0x558b7677de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.502+0000 7f472c1ff740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b76775a00 0x558b7676ee60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.502+0000 7f472c1ff740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f47241b07e0 con 0x558b76775a00 2024-04-21T08:42:06.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.502+0000 7f472c1ff740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f47241b0110 con 0x558b7676ce80 2024-04-21T08:42:06.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.502+0000 7f472c1ff740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f47241ad330 con 0x558b7676c4d0 2024-04-21T08:42:06.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.502+0000 7f472c1ff740 10 client.0 _ll_register_callbacks cb 0x7f470c00dee0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:06.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.502+0000 7f47237fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b76775a00 0x558b7676ee60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.502+0000 7f47237fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b76775a00 0x558b7676ee60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40790/0 (socket says 172.21.15.86:40790) 2024-04-21T08:42:06.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.502+0000 7f47237fe640 1 -- 172.21.15.86:0/3534261991 learned_addr learned my addr 172.21.15.86:0/3534261991 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:06.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.503+0000 7f471bfff640 1 --2- 172.21.15.86:0/3534261991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b7676ce80 0x558b7677de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.503+0000 7f4722ffd640 1 --2- 172.21.15.86:0/3534261991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b7676c4d0 0x558b7677d940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.503+0000 7f47237fe640 1 -- 172.21.15.86:0/3534261991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b7676c4d0 msgr2=0x558b7677d940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.503+0000 7f47237fe640 1 --2- 172.21.15.86:0/3534261991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b7676c4d0 0x558b7677d940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.503+0000 7f47237fe640 1 -- 172.21.15.86:0/3534261991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b7676ce80 msgr2=0x558b7677de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.503+0000 7f47237fe640 1 --2- 172.21.15.86:0/3534261991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b7676ce80 0x558b7677de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.503+0000 7f47237fe640 1 -- 172.21.15.86:0/3534261991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558b7676f660 con 0x558b76775a00 2024-04-21T08:42:06.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.503+0000 7f47237fe640 1 --2- 172.21.15.86:0/3534261991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b76775a00 0x558b7676ee60 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f471c010c30 tx=0x7f471c004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f1fdd3c956c31f1b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:06.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.503+0000 7f470a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:06.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.503+0000 7f470a7fc640 1 -- 172.21.15.86:0/3534261991 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f471c02b060 con 0x558b76775a00 2024-04-21T08:42:06.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.503+0000 7f470a7fc640 1 -- 172.21.15.86:0/3534261991 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f471c01d8c0 con 0x558b76775a00 2024-04-21T08:42:06.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.503+0000 7f472c1ff740 1 -- 172.21.15.86:0/3534261991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f470c00bb00 con 0x558b76775a00 2024-04-21T08:42:06.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.503+0000 7f470a7fc640 1 -- 172.21.15.86:0/3534261991 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f471c032050 con 0x558b76775a00 2024-04-21T08:42:06.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.503+0000 7f472c1ff740 10 client.17232 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:06.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.503+0000 7f472c1ff740 1 -- 172.21.15.86:0/3534261991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558b767ebb20 con 0x558b76775a00 2024-04-21T08:42:06.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.503+0000 7f470a7fc640 1 -- 172.21.15.86:0/3534261991 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f470c00bb00 con 0x558b76775a00 2024-04-21T08:42:06.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.503+0000 7f470a7fc640 1 -- 172.21.15.86:0/3534261991 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f471c019b70 con 0x558b76775a00 2024-04-21T08:42:06.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.503+0000 7f472c1ff740 10 client.17232 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:06.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.503+0000 7f472c1ff740 10 client.17232 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:42:06.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.503+0000 7f472c1ff740 20 client.17232 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:06.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.503+0000 7f472c1ff740 1 --2- 172.21.15.86:0/3534261991 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x558b767ec2d0 0x558b767ee6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.503+0000 7f472c1ff740 4 client.17232 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:42:06.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.503+0000 7f472c1ff740 1 -- 172.21.15.86:0/3534261991 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558b766d8a80 con 0x558b767ec2d0 2024-04-21T08:42:06.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.504+0000 7f4722ffd640 1 --2- 172.21.15.86:0/3534261991 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x558b767ec2d0 0x558b767ee6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.504+0000 7f470a7fc640 10 client.17232 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:42:06.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.504+0000 7f4722ffd640 1 --2- 172.21.15.86:0/3534261991 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x558b767ec2d0 0x558b767ee6f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bbd42e385eff13b0 server_cookie=b543ddba20170ce3 in_seq=0 out_seq=0 2024-04-21T08:42:06.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.504+0000 7f470a7fc640 1 -- 172.21.15.86:0/3534261991 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558b766d8a80 con 0x558b767ec2d0 2024-04-21T08:42:06.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.504+0000 7f470a7fc640 10 client.17232 handle_command_reply: tid=0 2024-04-21T08:42:06.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.504+0000 7f472c1ff740 1 client.17232 shutdown 2024-04-21T08:42:06.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.504+0000 7f472c1ff740 1 -- 172.21.15.86:0/3534261991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b76775a00 msgr2=0x558b7676ee60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.504+0000 7f472c1ff740 1 --2- 172.21.15.86:0/3534261991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b76775a00 0x558b7676ee60 secure :-1 s=READY pgs=1021 cs=0 l=1 rev1=1 crypto rx=0x7f471c010c30 tx=0x7f471c004ab0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.505+0000 7f472c1ff740 1 -- 172.21.15.86:0/3534261991 shutdown_connections 2024-04-21T08:42:06.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.505+0000 7f472c1ff740 1 --2- 172.21.15.86:0/3534261991 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x558b767ec2d0 0x558b767ee6f0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.505+0000 7f472c1ff740 1 --2- 172.21.15.86:0/3534261991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558b76775a00 0x558b7676ee60 unknown :-1 s=CLOSED pgs=1021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.505+0000 7f472c1ff740 1 --2- 172.21.15.86:0/3534261991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558b7676ce80 0x558b7677de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.505+0000 7f470a7fc640 0 client.17232 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:42:06.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.505+0000 7f472c1ff740 1 --2- 172.21.15.86:0/3534261991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558b7676c4d0 0x558b7677d940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.505+0000 7f472c1ff740 1 -- 172.21.15.86:0/3534261991 >> 172.21.15.86:0/3534261991 conn(0x558b76766ba0 msgr2=0x558b76765c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:06.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.505+0000 7f472c1ff740 1 -- 172.21.15.86:0/3534261991 shutdown_connections 2024-04-21T08:42:06.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.505+0000 7f472c1ff740 1 -- 172.21.15.86:0/3534261991 wait complete. 2024-04-21T08:42:06.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.505+0000 7f472c1ff740 20 client.17232 trim_cache size 0 max 16384 2024-04-21T08:42:06.512 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:42:06.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.505+0000 7f4729532640 1 -- 172.21.15.86:0/329550375 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f47100a3d20 msgr2=0x7f47100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.505+0000 7f4729532640 1 --2- 172.21.15.86:0/329550375 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f47100a3d20 0x7f47100a61e0 secure :-1 s=READY pgs=155 cs=0 l=1 rev1=1 crypto rx=0x7f471c00af40 tx=0x7f471c004a40 comp rx=0 tx=0).stop 2024-04-21T08:42:06.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.505+0000 7f4729532640 1 -- 172.21.15.86:0/329550375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47241011a0 msgr2=0x7f47241995e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.505+0000 7f4729532640 1 --2- 172.21.15.86:0/329550375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47241011a0 0x7f47241995e0 secure :-1 s=READY pgs=1017 cs=0 l=1 rev1=1 crypto rx=0x7f4714005940 tx=0x7f471402ca90 comp rx=0 tx=0).stop 2024-04-21T08:42:06.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.505+0000 7f4729532640 1 -- 172.21.15.86:0/329550375 shutdown_connections 2024-04-21T08:42:06.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.505+0000 7f4729532640 1 --2- 172.21.15.86:0/329550375 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f47100a3d20 0x7f47100a61e0 unknown :-1 s=CLOSED pgs=155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.505+0000 7f4729532640 1 --2- 172.21.15.86:0/329550375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4724107030 0x7f47241a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.505+0000 7f4729532640 1 --2- 172.21.15.86:0/329550375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4724101b40 0x7f4724199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.505+0000 7f4729532640 1 --2- 172.21.15.86:0/329550375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f47241011a0 0x7f47241995e0 unknown :-1 s=CLOSED pgs=1017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.505+0000 7f4729532640 1 -- 172.21.15.86:0/329550375 >> 172.21.15.86:0/329550375 conn(0x7f4724076c50 msgr2=0x7f47240ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:06.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.505+0000 7f4729532640 1 -- 172.21.15.86:0/329550375 shutdown_connections 2024-04-21T08:42:06.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.505+0000 7f4729532640 1 -- 172.21.15.86:0/329550375 wait complete. 2024-04-21T08:42:06.515 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:06 smithi086 ceph-mon[27939]: pgmap v102: 129 pgs: 129 active+clean; 3.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 47 KiB/s wr, 30 op/s 2024-04-21T08:42:06.515 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:06 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2754431965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:42:06.516 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:42:06.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.616+0000 7f0740b2e640 1 -- 172.21.15.86:0/564319256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f073c103b00 msgr2=0x7f073c10bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.616+0000 7f0740b2e640 1 --2- 172.21.15.86:0/564319256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f073c103b00 0x7f073c10bfe0 secure :-1 s=READY pgs=1022 cs=0 l=1 rev1=1 crypto rx=0x7f0730009900 tx=0x7f073002bd60 comp rx=0 tx=0).stop 2024-04-21T08:42:06.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.617+0000 7f0740b2e640 1 -- 172.21.15.86:0/564319256 shutdown_connections 2024-04-21T08:42:06.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.617+0000 7f0740b2e640 1 --2- 172.21.15.86:0/564319256 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f073c10c520 0x7f073c10e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.617+0000 7f0740b2e640 1 --2- 172.21.15.86:0/564319256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f073c103b00 0x7f073c10bfe0 unknown :-1 s=CLOSED pgs=1022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.617+0000 7f0740b2e640 1 --2- 172.21.15.86:0/564319256 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f073c1031e0 0x7f073c1035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.617+0000 7f0740b2e640 1 -- 172.21.15.86:0/564319256 >> 172.21.15.86:0/564319256 conn(0x7f073c0771d0 msgr2=0x7f073c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:06.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.617+0000 7f0740b2e640 1 -- 172.21.15.86:0/564319256 shutdown_connections 2024-04-21T08:42:06.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.617+0000 7f0740b2e640 1 -- 172.21.15.86:0/564319256 wait complete. 2024-04-21T08:42:06.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.617+0000 7f0740b2e640 1 Processor -- start 2024-04-21T08:42:06.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.617+0000 7f0740b2e640 1 -- start start 2024-04-21T08:42:06.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.618+0000 7f0740b2e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f073c1031e0 0x7f073c1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.618+0000 7f0740b2e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f073c103b00 0x7f073c198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.618+0000 7f0740b2e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f073c10c520 0x7f073c19cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.618+0000 7f0740b2e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f073c1121e0 con 0x7f073c103b00 2024-04-21T08:42:06.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.618+0000 7f0740b2e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f073c112060 con 0x7f073c1031e0 2024-04-21T08:42:06.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.618+0000 7f0740b2e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f073c112360 con 0x7f073c10c520 2024-04-21T08:42:06.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.618+0000 7f0739d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f073c103b00 0x7f073c198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.618+0000 7f0739d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f073c103b00 0x7f073c198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40814/0 (socket says 172.21.15.86:40814) 2024-04-21T08:42:06.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.618+0000 7f0739d9b640 1 -- 172.21.15.86:0/1491831593 learned_addr learned my addr 172.21.15.86:0/1491831593 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:06.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.618+0000 7f073a59c640 1 --2- 172.21.15.86:0/1491831593 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f073c1031e0 0x7f073c1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.618+0000 7f073ad9d640 1 --2- 172.21.15.86:0/1491831593 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f073c10c520 0x7f073c19cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.618+0000 7f0739d9b640 1 -- 172.21.15.86:0/1491831593 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f073c10c520 msgr2=0x7f073c19cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.618+0000 7f0739d9b640 1 --2- 172.21.15.86:0/1491831593 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f073c10c520 0x7f073c19cf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.618+0000 7f0739d9b640 1 -- 172.21.15.86:0/1491831593 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f073c1031e0 msgr2=0x7f073c1986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.618+0000 7f0739d9b640 1 --2- 172.21.15.86:0/1491831593 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f073c1031e0 0x7f073c1986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.618+0000 7f0739d9b640 1 -- 172.21.15.86:0/1491831593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f073c19d6b0 con 0x7f073c103b00 2024-04-21T08:42:06.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.618+0000 7f0739d9b640 1 --2- 172.21.15.86:0/1491831593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f073c103b00 0x7f073c198c00 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f0730005ab0 tx=0x7f073002dbd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:06.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.618+0000 7f072b7fe640 1 -- 172.21.15.86:0/1491831593 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0730043020 con 0x7f073c103b00 2024-04-21T08:42:06.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.618+0000 7f072b7fe640 1 -- 172.21.15.86:0/1491831593 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f07300052b0 con 0x7f073c103b00 2024-04-21T08:42:06.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.618+0000 7f0740b2e640 1 -- 172.21.15.86:0/1491831593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f073c19d940 con 0x7f073c103b00 2024-04-21T08:42:06.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.618+0000 7f072b7fe640 1 -- 172.21.15.86:0/1491831593 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0730003cb0 con 0x7f073c103b00 2024-04-21T08:42:06.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.618+0000 7f0740b2e640 1 -- 172.21.15.86:0/1491831593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f073c1a5230 con 0x7f073c103b00 2024-04-21T08:42:06.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.621+0000 7f072b7fe640 1 -- 172.21.15.86:0/1491831593 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0730046020 con 0x7f073c103b00 2024-04-21T08:42:06.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.621+0000 7f07437f3740 1 Processor -- start 2024-04-21T08:42:06.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.621+0000 7f07437f3740 1 -- start start 2024-04-21T08:42:06.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.621+0000 7f07437f3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c368254d0 0x559c368258b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.621+0000 7f07437f3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c36825e80 0x559c3682e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.622+0000 7f07437f3740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c3682ea00 0x559c36830e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.622+0000 7f07437f3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559c3678a1f0 con 0x559c368254d0 2024-04-21T08:42:06.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.622+0000 7f07437f3740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559c367274b0 con 0x559c3682ea00 2024-04-21T08:42:06.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.622+0000 7f07437f3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559c367861b0 con 0x559c36825e80 2024-04-21T08:42:06.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.622+0000 7f073ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c368254d0 0x559c368258b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.622+0000 7f073ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c368254d0 0x559c368258b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40818/0 (socket says 172.21.15.86:40818) 2024-04-21T08:42:06.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.622+0000 7f073ad9d640 1 -- 172.21.15.86:0/2021626137 learned_addr learned my addr 172.21.15.86:0/2021626137 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:06.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.622+0000 7f072b7fe640 1 --2- 172.21.15.86:0/1491831593 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f07240a3d20 0x7f07240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.622+0000 7f073a59c640 1 --2- 172.21.15.86:0/2021626137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c36825e80 0x559c3682e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.622+0000 7f0739d9b640 1 --2- 172.21.15.86:0/2021626137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c3682ea00 0x559c36830e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.622+0000 7f073ad9d640 1 -- 172.21.15.86:0/2021626137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c36825e80 msgr2=0x559c3682e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.622+0000 7f073ad9d640 1 --2- 172.21.15.86:0/2021626137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c36825e80 0x559c3682e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.622+0000 7f073ad9d640 1 -- 172.21.15.86:0/2021626137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c3682ea00 msgr2=0x559c36830e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.622+0000 7f073ad9d640 1 --2- 172.21.15.86:0/2021626137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c3682ea00 0x559c36830e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.622+0000 7f073ad9d640 1 -- 172.21.15.86:0/2021626137 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c367191b0 con 0x559c368254d0 2024-04-21T08:42:06.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.622+0000 7f0739d9b640 1 --2- 172.21.15.86:0/2021626137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c3682ea00 0x559c36830e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:06.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.622+0000 7f072b7fe640 1 -- 172.21.15.86:0/1491831593 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f07300f2e10 con 0x7f073c103b00 2024-04-21T08:42:06.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.622+0000 7f073ad9d640 1 --2- 172.21.15.86:0/1491831593 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f07240a3d20 0x7f07240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.622+0000 7f073ad9d640 1 --2- 172.21.15.86:0/2021626137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c368254d0 0x559c368258b0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f0734007f50 tx=0x7f073400e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=88b77bab6954d9db server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:06.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.622+0000 7f0728ff9640 1 -- 172.21.15.86:0/2021626137 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0734022070 con 0x559c368254d0 2024-04-21T08:42:06.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.622+0000 7f073ad9d640 1 --2- 172.21.15.86:0/1491831593 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f07240a3d20 0x7f07240a61e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f073401f000 tx=0x7f0734020000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:06.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.622+0000 7f0728ff9640 1 -- 172.21.15.86:0/2021626137 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0734011b70 con 0x559c368254d0 2024-04-21T08:42:06.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.622+0000 7f0728ff9640 1 -- 172.21.15.86:0/2021626137 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f073401a400 con 0x559c368254d0 2024-04-21T08:42:06.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.623+0000 7f07437f3740 1 -- 172.21.15.86:0/2021626137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c368254d0 msgr2=0x559c368258b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.623+0000 7f07437f3740 1 --2- 172.21.15.86:0/2021626137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c368254d0 0x559c368258b0 secure :-1 s=READY pgs=1024 cs=0 l=1 rev1=1 crypto rx=0x7f0734007f50 tx=0x7f073400e7e0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.623+0000 7f07437f3740 1 -- 172.21.15.86:0/2021626137 shutdown_connections 2024-04-21T08:42:06.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.623+0000 7f07437f3740 1 --2- 172.21.15.86:0/2021626137 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c3682ea00 0x559c36830e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.623+0000 7f07437f3740 1 --2- 172.21.15.86:0/2021626137 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c36825e80 0x559c3682e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.623+0000 7f07437f3740 1 --2- 172.21.15.86:0/2021626137 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c368254d0 0x559c368258b0 unknown :-1 s=CLOSED pgs=1024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.623+0000 7f07437f3740 1 -- 172.21.15.86:0/2021626137 >> 172.21.15.86:0/2021626137 conn(0x559c3681fba0 msgr2=0x559c3681ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:06.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.623+0000 7f07437f3740 1 -- 172.21.15.86:0/2021626137 shutdown_connections 2024-04-21T08:42:06.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.623+0000 7f07437f3740 1 -- 172.21.15.86:0/2021626137 wait complete. 2024-04-21T08:42:06.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.624+0000 7f07437f3740 1 -- start start 2024-04-21T08:42:06.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.624+0000 7f07437f3740 1 Processor -- start 2024-04-21T08:42:06.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.624+0000 7f07437f3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c368254d0 0x559c3682e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.624+0000 7f07437f3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c36825e80 0x559c3682b680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.624+0000 7f07437f3740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c3682ea00 0x559c3682bc00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.624+0000 7f07437f3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559c36831970 con 0x559c368254d0 2024-04-21T08:42:06.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.624+0000 7f07437f3740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559c36743500 con 0x559c3682ea00 2024-04-21T08:42:06.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.624+0000 7f07437f3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559c36831af0 con 0x559c36825e80 2024-04-21T08:42:06.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.624+0000 7f07437f3740 10 client.0 _ll_register_callbacks cb 0x559c36711b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:06.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.624+0000 7f073a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c368254d0 0x559c3682e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.624+0000 7f073a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c368254d0 0x559c3682e2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40820/0 (socket says 172.21.15.86:40820) 2024-04-21T08:42:06.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.624+0000 7f073a59c640 1 -- 172.21.15.86:0/773856218 learned_addr learned my addr 172.21.15.86:0/773856218 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:06.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.624+0000 7f0739d9b640 1 --2- 172.21.15.86:0/773856218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c36825e80 0x559c3682b680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.624+0000 7f073ad9d640 1 --2- 172.21.15.86:0/773856218 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c3682ea00 0x559c3682bc00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.624+0000 7f073a59c640 1 -- 172.21.15.86:0/773856218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c36825e80 msgr2=0x559c3682b680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.624+0000 7f073a59c640 1 --2- 172.21.15.86:0/773856218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c36825e80 0x559c3682b680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.624+0000 7f073a59c640 1 -- 172.21.15.86:0/773856218 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c3682ea00 msgr2=0x559c3682bc00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.624+0000 7f073a59c640 1 --2- 172.21.15.86:0/773856218 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c3682ea00 0x559c3682bc00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.624+0000 7f073a59c640 1 -- 172.21.15.86:0/773856218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c3649c3f0 con 0x559c368254d0 2024-04-21T08:42:06.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.624+0000 7f073ad9d640 1 --2- 172.21.15.86:0/773856218 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c3682ea00 0x559c3682bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:06.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.624+0000 7f073a59c640 1 --2- 172.21.15.86:0/773856218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c368254d0 0x559c3682e2f0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f072c0107c0 tx=0x7f072c010c80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e091948969580e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:06.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.624+0000 7f0721ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:06.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.625+0000 7f0721ffb640 1 -- 172.21.15.86:0/773856218 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072c00cb10 con 0x559c368254d0 2024-04-21T08:42:06.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.625+0000 7f0721ffb640 1 -- 172.21.15.86:0/773856218 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f072c00ccb0 con 0x559c368254d0 2024-04-21T08:42:06.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.625+0000 7f07437f3740 1 -- 172.21.15.86:0/773856218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0734011b70 con 0x559c368254d0 2024-04-21T08:42:06.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.625+0000 7f0721ffb640 1 -- 172.21.15.86:0/773856218 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f072c011440 con 0x559c368254d0 2024-04-21T08:42:06.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.625+0000 7f07437f3740 10 client.17244 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:06.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.625+0000 7f07437f3740 1 -- 172.21.15.86:0/773856218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559c365667d0 con 0x559c368254d0 2024-04-21T08:42:06.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.625+0000 7f0721ffb640 1 -- 172.21.15.86:0/773856218 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0734011b70 con 0x559c368254d0 2024-04-21T08:42:06.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.625+0000 7f0721ffb640 1 -- 172.21.15.86:0/773856218 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f072c01a550 con 0x559c368254d0 2024-04-21T08:42:06.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.625+0000 7f07437f3740 10 client.17244 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:06.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.625+0000 7f07437f3740 10 client.17244 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:06.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.625+0000 7f07437f3740 20 client.17244 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:06.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.625+0000 7f07437f3740 1 --2- 172.21.15.86:0/773856218 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559c3683ecc0 0x559c36841130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.625+0000 7f07437f3740 4 client.17244 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:06.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.625+0000 7f07437f3740 1 -- 172.21.15.86:0/773856218 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559c36729930 con 0x559c3683ecc0 2024-04-21T08:42:06.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.625+0000 7f0739d9b640 1 --2- 172.21.15.86:0/773856218 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559c3683ecc0 0x559c36841130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.626+0000 7f0721ffb640 10 client.17244 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:06.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.626+0000 7f0739d9b640 1 --2- 172.21.15.86:0/773856218 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559c3683ecc0 0x559c36841130 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6a8c6536a461daca server_cookie=ff86101777008588 in_seq=0 out_seq=0 2024-04-21T08:42:06.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.626+0000 7f0721ffb640 1 -- 172.21.15.86:0/773856218 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559c36729930 con 0x559c3683ecc0 2024-04-21T08:42:06.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.626+0000 7f0721ffb640 10 client.17244 handle_command_reply: tid=0 2024-04-21T08:42:06.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.626+0000 7f07437f3740 1 client.17244 shutdown 2024-04-21T08:42:06.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.627+0000 7f07437f3740 1 -- 172.21.15.86:0/773856218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c368254d0 msgr2=0x559c3682e2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.627+0000 7f07437f3740 1 --2- 172.21.15.86:0/773856218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c368254d0 0x559c3682e2f0 secure :-1 s=READY pgs=1025 cs=0 l=1 rev1=1 crypto rx=0x7f072c0107c0 tx=0x7f072c010c80 comp rx=0 tx=0).stop 2024-04-21T08:42:06.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.627+0000 7f07437f3740 1 -- 172.21.15.86:0/773856218 shutdown_connections 2024-04-21T08:42:06.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.627+0000 7f07437f3740 1 --2- 172.21.15.86:0/773856218 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559c3683ecc0 0x559c36841130 crc :-1 s=READY pgs=129 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.627+0000 7f07437f3740 1 --2- 172.21.15.86:0/773856218 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c3682ea00 0x559c3682bc00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.627+0000 7f07437f3740 1 --2- 172.21.15.86:0/773856218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c36825e80 0x559c3682b680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.627+0000 7f07437f3740 1 --2- 172.21.15.86:0/773856218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c368254d0 0x559c3682e2f0 unknown :-1 s=CLOSED pgs=1025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.627+0000 7f0721ffb640 0 client.17244 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:06.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.627+0000 7f07437f3740 1 -- 172.21.15.86:0/773856218 >> 172.21.15.86:0/773856218 conn(0x559c3681fba0 msgr2=0x559c368218f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:06.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.627+0000 7f07437f3740 1 -- 172.21.15.86:0/773856218 shutdown_connections 2024-04-21T08:42:06.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.627+0000 7f07437f3740 1 -- 172.21.15.86:0/773856218 wait complete. 2024-04-21T08:42:06.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.627+0000 7f07437f3740 20 client.17244 trim_cache size 0 max 16384 2024-04-21T08:42:06.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.640+0000 7f07437f3740 1 Processor -- start 2024-04-21T08:42:06.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.640+0000 7f07437f3740 1 -- start start 2024-04-21T08:42:06.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.640+0000 7f07437f3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c368254d0 0x559c3683a890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.640+0000 7f07437f3740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c36825e80 0x559c3681ea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.640+0000 7f07437f3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3682ea00 0x559c3682ddb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.640+0000 7f07437f3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f073c1af130 con 0x559c368254d0 2024-04-21T08:42:06.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.640+0000 7f07437f3740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f073c1b0c70 con 0x559c36825e80 2024-04-21T08:42:06.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.640+0000 7f07437f3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f073c1af800 con 0x559c3682ea00 2024-04-21T08:42:06.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.640+0000 7f073a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c368254d0 0x559c3683a890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.640+0000 7f073a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c368254d0 0x559c3683a890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40828/0 (socket says 172.21.15.86:40828) 2024-04-21T08:42:06.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.640+0000 7f073a59c640 1 -- 172.21.15.86:0/2912085240 learned_addr learned my addr 172.21.15.86:0/2912085240 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:06.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.640+0000 7f0739d9b640 1 --2- 172.21.15.86:0/2912085240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c36825e80 0x559c3681ea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.640+0000 7f073ad9d640 1 --2- 172.21.15.86:0/2912085240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3682ea00 0x559c3682ddb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.640+0000 7f073a59c640 1 -- 172.21.15.86:0/2912085240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3682ea00 msgr2=0x559c3682ddb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.640+0000 7f073a59c640 1 --2- 172.21.15.86:0/2912085240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3682ea00 0x559c3682ddb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.640+0000 7f073a59c640 1 -- 172.21.15.86:0/2912085240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c36825e80 msgr2=0x559c3681ea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.640+0000 7f073a59c640 1 --2- 172.21.15.86:0/2912085240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c36825e80 0x559c3681ea80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.640+0000 7f073a59c640 1 -- 172.21.15.86:0/2912085240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559c36831ae0 con 0x559c368254d0 2024-04-21T08:42:06.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.640+0000 7f0739d9b640 1 --2- 172.21.15.86:0/2912085240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c36825e80 0x559c3681ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:06.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.641+0000 7f073a59c640 1 --2- 172.21.15.86:0/2912085240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c368254d0 0x559c3683a890 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f072c01a9c0 tx=0x7f072c00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8d980b1dbbd6651 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:06.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.641+0000 7f0723fff640 1 -- 172.21.15.86:0/2912085240 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072c02d020 con 0x559c368254d0 2024-04-21T08:42:06.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.641+0000 7f0723fff640 1 -- 172.21.15.86:0/2912085240 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f072c01a6f0 con 0x559c368254d0 2024-04-21T08:42:06.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.641+0000 7f0723fff640 1 -- 172.21.15.86:0/2912085240 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f072c0035f0 con 0x559c368254d0 2024-04-21T08:42:06.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.641+0000 7f07437f3740 1 -- 172.21.15.86:0/2912085240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c368254d0 msgr2=0x559c3683a890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.641+0000 7f07437f3740 1 --2- 172.21.15.86:0/2912085240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c368254d0 0x559c3683a890 secure :-1 s=READY pgs=1026 cs=0 l=1 rev1=1 crypto rx=0x7f072c01a9c0 tx=0x7f072c00e6c0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.641+0000 7f07437f3740 1 -- 172.21.15.86:0/2912085240 shutdown_connections 2024-04-21T08:42:06.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.641+0000 7f07437f3740 1 --2- 172.21.15.86:0/2912085240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c3682ea00 0x559c3682ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.641+0000 7f07437f3740 1 --2- 172.21.15.86:0/2912085240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c36825e80 0x559c3681ea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.641+0000 7f07437f3740 1 --2- 172.21.15.86:0/2912085240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c368254d0 0x559c3683a890 unknown :-1 s=CLOSED pgs=1026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.641+0000 7f07437f3740 1 -- 172.21.15.86:0/2912085240 >> 172.21.15.86:0/2912085240 conn(0x559c3681fba0 msgr2=0x559c3681e370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:06.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.641+0000 7f07437f3740 1 -- 172.21.15.86:0/2912085240 shutdown_connections 2024-04-21T08:42:06.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.641+0000 7f07437f3740 1 -- 172.21.15.86:0/2912085240 wait complete. 2024-04-21T08:42:06.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.642+0000 7f07437f3740 1 -- start start 2024-04-21T08:42:06.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.642+0000 7f07437f3740 1 Processor -- start 2024-04-21T08:42:06.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.642+0000 7f07437f3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c368254d0 0x559c368284b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.642+0000 7f07437f3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c36825e80 0x559c368289f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.642+0000 7f07437f3740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c3682ea00 0x559c36828f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.642+0000 7f07437f3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f073c1b05a0 con 0x559c36825e80 2024-04-21T08:42:06.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.642+0000 7f07437f3740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f073c1afed0 con 0x559c3682ea00 2024-04-21T08:42:06.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.642+0000 7f07437f3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f073c1ad0f0 con 0x559c368254d0 2024-04-21T08:42:06.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.642+0000 7f07437f3740 10 client.0 _ll_register_callbacks cb 0x7f072c0103d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:06.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.642+0000 7f0739d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c36825e80 0x559c368289f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.642+0000 7f0739d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c36825e80 0x559c368289f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40832/0 (socket says 172.21.15.86:40832) 2024-04-21T08:42:06.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.642+0000 7f0739d9b640 1 -- 172.21.15.86:0/3785551319 learned_addr learned my addr 172.21.15.86:0/3785551319 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:06.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.642+0000 7f073ad9d640 1 --2- 172.21.15.86:0/3785551319 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c3682ea00 0x559c36828f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.642+0000 7f0739d9b640 1 -- 172.21.15.86:0/3785551319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c368254d0 msgr2=0x559c368284b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:42:06.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.642+0000 7f0739d9b640 1 --2- 172.21.15.86:0/3785551319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c368254d0 0x559c368284b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.642+0000 7f0739d9b640 1 -- 172.21.15.86:0/3785551319 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c3682ea00 msgr2=0x559c36828f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:06.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.642+0000 7f0739d9b640 1 --2- 172.21.15.86:0/3785551319 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c3682ea00 0x559c36828f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.642+0000 7f0739d9b640 1 -- 172.21.15.86:0/3785551319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559c368a43d0 con 0x559c36825e80 2024-04-21T08:42:06.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.642+0000 7f073ad9d640 1 --2- 172.21.15.86:0/3785551319 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c3682ea00 0x559c36828f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:06.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.642+0000 7f0739d9b640 1 --2- 172.21.15.86:0/3785551319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c36825e80 0x559c368289f0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f07300bafb0 tx=0x7f0730090590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a12d6b0a54e9c168 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:06.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.642+0000 7f0721ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:06.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.642+0000 7f0721ffb640 1 -- 172.21.15.86:0/3785551319 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07300bf250 con 0x559c36825e80 2024-04-21T08:42:06.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.642+0000 7f0721ffb640 1 -- 172.21.15.86:0/3785551319 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f0730005460 con 0x559c36825e80 2024-04-21T08:42:06.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.642+0000 7f07437f3740 1 -- 172.21.15.86:0/3785551319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f072c01a6f0 con 0x559c36825e80 2024-04-21T08:42:06.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.643+0000 7f0721ffb640 1 -- 172.21.15.86:0/3785551319 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f073003c350 con 0x559c36825e80 2024-04-21T08:42:06.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.643+0000 7f07437f3740 10 client.17250 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:06.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.643+0000 7f07437f3740 1 -- 172.21.15.86:0/3785551319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559c368a56b0 con 0x559c36825e80 2024-04-21T08:42:06.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.643+0000 7f0721ffb640 1 -- 172.21.15.86:0/3785551319 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f072c01a6f0 con 0x559c36825e80 2024-04-21T08:42:06.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.643+0000 7f0721ffb640 1 -- 172.21.15.86:0/3785551319 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f07300df040 con 0x559c36825e80 2024-04-21T08:42:06.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.643+0000 7f07437f3740 10 client.17250 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:06.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.643+0000 7f07437f3740 10 client.17250 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:06.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.643+0000 7f07437f3740 20 client.17250 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:06.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.643+0000 7f07437f3740 1 --2- 172.21.15.86:0/3785551319 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559c368a5d90 0x559c368a81b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:06.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.643+0000 7f07437f3740 4 client.17250 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:42:06.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.643+0000 7f07437f3740 1 -- 172.21.15.86:0/3785551319 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559c36797c10 con 0x559c368a5d90 2024-04-21T08:42:06.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.643+0000 7f073a59c640 1 --2- 172.21.15.86:0/3785551319 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559c368a5d90 0x559c368a81b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:06.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.643+0000 7f0721ffb640 10 client.17250 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:06.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.643+0000 7f073a59c640 1 --2- 172.21.15.86:0/3785551319 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559c368a5d90 0x559c368a81b0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8165e1c089fb136b server_cookie=51455b61fbd0ea4a in_seq=0 out_seq=0 2024-04-21T08:42:06.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.644+0000 7f0721ffb640 1 -- 172.21.15.86:0/3785551319 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559c36797c10 con 0x559c368a5d90 2024-04-21T08:42:06.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.644+0000 7f0721ffb640 10 client.17250 handle_command_reply: tid=0 2024-04-21T08:42:06.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.644+0000 7f07437f3740 1 client.17250 shutdown 2024-04-21T08:42:06.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.644+0000 7f07437f3740 1 -- 172.21.15.86:0/3785551319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c36825e80 msgr2=0x559c368289f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.644+0000 7f07437f3740 1 --2- 172.21.15.86:0/3785551319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c36825e80 0x559c368289f0 secure :-1 s=READY pgs=1027 cs=0 l=1 rev1=1 crypto rx=0x7f07300bafb0 tx=0x7f0730090590 comp rx=0 tx=0).stop 2024-04-21T08:42:06.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.644+0000 7f07437f3740 1 -- 172.21.15.86:0/3785551319 shutdown_connections 2024-04-21T08:42:06.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.644+0000 7f07437f3740 1 --2- 172.21.15.86:0/3785551319 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559c368a5d90 0x559c368a81b0 crc :-1 s=READY pgs=130 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.644+0000 7f07437f3740 1 --2- 172.21.15.86:0/3785551319 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559c3682ea00 0x559c36828f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.644+0000 7f07437f3740 1 --2- 172.21.15.86:0/3785551319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559c36825e80 0x559c368289f0 unknown :-1 s=CLOSED pgs=1027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.644+0000 7f07437f3740 1 --2- 172.21.15.86:0/3785551319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559c368254d0 0x559c368284b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.644+0000 7f0721ffb640 0 client.17250 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:06.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.644+0000 7f07437f3740 1 -- 172.21.15.86:0/3785551319 >> 172.21.15.86:0/3785551319 conn(0x559c3681fba0 msgr2=0x559c3681ec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:06.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.644+0000 7f07437f3740 1 -- 172.21.15.86:0/3785551319 shutdown_connections 2024-04-21T08:42:06.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.644+0000 7f07437f3740 1 -- 172.21.15.86:0/3785551319 wait complete. 2024-04-21T08:42:06.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.644+0000 7f07437f3740 20 client.17250 trim_cache size 0 max 16384 2024-04-21T08:42:06.652 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:42:06.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.644+0000 7f0740b2e640 1 -- 172.21.15.86:0/1491831593 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f07240a3d20 msgr2=0x7f07240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.644+0000 7f0740b2e640 1 --2- 172.21.15.86:0/1491831593 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f07240a3d20 0x7f07240a61e0 secure :-1 s=READY pgs=156 cs=0 l=1 rev1=1 crypto rx=0x7f073401f000 tx=0x7f0734020000 comp rx=0 tx=0).stop 2024-04-21T08:42:06.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.644+0000 7f0740b2e640 1 -- 172.21.15.86:0/1491831593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f073c103b00 msgr2=0x7f073c198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:06.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.644+0000 7f0740b2e640 1 --2- 172.21.15.86:0/1491831593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f073c103b00 0x7f073c198c00 secure :-1 s=READY pgs=1023 cs=0 l=1 rev1=1 crypto rx=0x7f0730005ab0 tx=0x7f073002dbd0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.644+0000 7f0740b2e640 1 -- 172.21.15.86:0/1491831593 shutdown_connections 2024-04-21T08:42:06.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.644+0000 7f0740b2e640 1 --2- 172.21.15.86:0/1491831593 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f07240a3d20 0x7f07240a61e0 unknown :-1 s=CLOSED pgs=156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.644+0000 7f0740b2e640 1 --2- 172.21.15.86:0/1491831593 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f073c10c520 0x7f073c19cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.644+0000 7f0740b2e640 1 --2- 172.21.15.86:0/1491831593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f073c103b00 0x7f073c198c00 unknown :-1 s=CLOSED pgs=1023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.644+0000 7f0740b2e640 1 --2- 172.21.15.86:0/1491831593 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f073c1031e0 0x7f073c1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:06.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.644+0000 7f0740b2e640 1 -- 172.21.15.86:0/1491831593 >> 172.21.15.86:0/1491831593 conn(0x7f073c0771d0 msgr2=0x7f073c0fef10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:06.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.645+0000 7f0740b2e640 1 -- 172.21.15.86:0/1491831593 shutdown_connections 2024-04-21T08:42:06.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:06.645+0000 7f0740b2e640 1 -- 172.21.15.86:0/1491831593 wait complete. 2024-04-21T08:42:06.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:06 smithi088 ceph-mon[31259]: pgmap v102: 129 pgs: 129 active+clean; 3.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 47 KiB/s wr, 30 op/s 2024-04-21T08:42:06.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:06 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2754431965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:42:06.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:06 smithi179 ceph-mon[31829]: pgmap v102: 129 pgs: 129 active+clean; 3.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 47 KiB/s wr, 30 op/s 2024-04-21T08:42:06.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:06 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2754431965' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:42:07.656 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:42:07.656 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:42:07.656 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:42:07.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.770+0000 7fa0ed359640 1 -- 172.21.15.86:0/2150578761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e8101130 msgr2=0x7fa0e8101590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:07.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.770+0000 7fa0ed359640 1 --2- 172.21.15.86:0/2150578761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e8101130 0x7fa0e8101590 secure :-1 s=READY pgs=1028 cs=0 l=1 rev1=1 crypto rx=0x7fa0dc009900 tx=0x7fa0dc02bc80 comp rx=0 tx=0).stop 2024-04-21T08:42:07.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.770+0000 7fa0ed359640 1 -- 172.21.15.86:0/2150578761 shutdown_connections 2024-04-21T08:42:07.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.770+0000 7fa0ed359640 1 --2- 172.21.15.86:0/2150578761 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0e8101ad0 0x7fa0e810dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.770+0000 7fa0ed359640 1 --2- 172.21.15.86:0/2150578761 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e8101130 0x7fa0e8101590 unknown :-1 s=CLOSED pgs=1028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.770+0000 7fa0ed359640 1 --2- 172.21.15.86:0/2150578761 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0e8105ea0 0x7fa0e8106280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.770+0000 7fa0ed359640 1 -- 172.21.15.86:0/2150578761 >> 172.21.15.86:0/2150578761 conn(0x7fa0e8077e30 msgr2=0x7fa0e8078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:07.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.770+0000 7fa0ed359640 1 -- 172.21.15.86:0/2150578761 shutdown_connections 2024-04-21T08:42:07.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.770+0000 7fa0ed359640 1 -- 172.21.15.86:0/2150578761 wait complete. 2024-04-21T08:42:07.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.770+0000 7fa0ed359640 1 Processor -- start 2024-04-21T08:42:07.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.771+0000 7fa0ed359640 1 -- start start 2024-04-21T08:42:07.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.771+0000 7fa0ed359640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0e8101130 0x7fa0e8197750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:07.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.771+0000 7fa0ed359640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0e8101ad0 0x7fa0e8197c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:07.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.771+0000 7fa0ed359640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e8105ea0 0x7fa0e819c010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:07.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.771+0000 7fa0ed359640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e81111f0 con 0x7fa0e8105ea0 2024-04-21T08:42:07.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.771+0000 7fa0ed359640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e8111070 con 0x7fa0e8101130 2024-04-21T08:42:07.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.771+0000 7fa0ed359640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e8111370 con 0x7fa0e8101ad0 2024-04-21T08:42:07.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.771+0000 7fa0e77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e8105ea0 0x7fa0e819c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:07.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.771+0000 7fa0e77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e8105ea0 0x7fa0e819c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40856/0 (socket says 172.21.15.86:40856) 2024-04-21T08:42:07.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.771+0000 7fa0e77fe640 1 -- 172.21.15.86:0/764892566 learned_addr learned my addr 172.21.15.86:0/764892566 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:07.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.771+0000 7fa0e6ffd640 1 --2- 172.21.15.86:0/764892566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0e8101130 0x7fa0e8197750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:07.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.771+0000 7fa0e67fc640 1 --2- 172.21.15.86:0/764892566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0e8101ad0 0x7fa0e8197c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:07.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.771+0000 7fa0e77fe640 1 -- 172.21.15.86:0/764892566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0e8101ad0 msgr2=0x7fa0e8197c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:07.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.771+0000 7fa0e77fe640 1 --2- 172.21.15.86:0/764892566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0e8101ad0 0x7fa0e8197c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.771+0000 7fa0e77fe640 1 -- 172.21.15.86:0/764892566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0e8101130 msgr2=0x7fa0e8197750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:07.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.771+0000 7fa0e77fe640 1 --2- 172.21.15.86:0/764892566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0e8101130 0x7fa0e8197750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.771+0000 7fa0e77fe640 1 -- 172.21.15.86:0/764892566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0e819c740 con 0x7fa0e8105ea0 2024-04-21T08:42:07.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.771+0000 7fa0e67fc640 1 --2- 172.21.15.86:0/764892566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0e8101ad0 0x7fa0e8197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:07.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.772+0000 7fa0e77fe640 1 --2- 172.21.15.86:0/764892566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e8105ea0 0x7fa0e819c010 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fa0e000eaa0 tx=0x7fa0e000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:07.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.772+0000 7fa0cffff640 1 -- 172.21.15.86:0/764892566 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e000cdb0 con 0x7fa0e8105ea0 2024-04-21T08:42:07.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.772+0000 7fa0ed359640 1 -- 172.21.15.86:0/764892566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0e819ca30 con 0x7fa0e8105ea0 2024-04-21T08:42:07.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.772+0000 7fa0cffff640 1 -- 172.21.15.86:0/764892566 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0e0005020 con 0x7fa0e8105ea0 2024-04-21T08:42:07.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.772+0000 7fa0cffff640 1 -- 172.21.15.86:0/764892566 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0e00116d0 con 0x7fa0e8105ea0 2024-04-21T08:42:07.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.772+0000 7fa0ed359640 1 -- 172.21.15.86:0/764892566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0e81a42c0 con 0x7fa0e8105ea0 2024-04-21T08:42:07.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.774+0000 7fa0cffff640 1 -- 172.21.15.86:0/764892566 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa0e00051d0 con 0x7fa0e8105ea0 2024-04-21T08:42:07.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.774+0000 7fa0f002c740 1 Processor -- start 2024-04-21T08:42:07.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.774+0000 7fa0f002c740 1 -- start start 2024-04-21T08:42:07.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.775+0000 7fa0cffff640 1 --2- 172.21.15.86:0/764892566 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa0d00a3d20 0x7fa0d00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:07.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.775+0000 7fa0f002c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564812663780 0x564812663b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:07.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.775+0000 7fa0f002c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564812664130 0x56481266ea50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:07.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.775+0000 7fa0f002c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56481266f020 0x564812671430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:07.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.775+0000 7fa0f002c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5648125c7f00 con 0x564812663780 2024-04-21T08:42:07.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.775+0000 7fa0f002c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564812565590 con 0x56481266f020 2024-04-21T08:42:07.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.775+0000 7fa0f002c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5648125c3ec0 con 0x564812664130 2024-04-21T08:42:07.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.775+0000 7fa0e67fc640 1 --2- 172.21.15.86:0/764892566 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa0d00a3d20 0x7fa0d00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:07.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.775+0000 7fa0cffff640 1 -- 172.21.15.86:0/764892566 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fa0e00d0d80 con 0x7fa0e8105ea0 2024-04-21T08:42:07.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.775+0000 7fa0e6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564812663780 0x564812663b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:07.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.775+0000 7fa0e6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564812663780 0x564812663b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40872/0 (socket says 172.21.15.86:40872) 2024-04-21T08:42:07.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.775+0000 7fa0e6ffd640 1 -- 172.21.15.86:0/947925745 learned_addr learned my addr 172.21.15.86:0/947925745 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:07.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.775+0000 7fa0e67fc640 1 --2- 172.21.15.86:0/947925745 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564812664130 0x56481266ea50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:07.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.775+0000 7fa0e77fe640 1 --2- 172.21.15.86:0/947925745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56481266f020 0x564812671430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:07.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.775+0000 7fa0e6ffd640 1 -- 172.21.15.86:0/947925745 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564812664130 msgr2=0x56481266ea50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:07.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.775+0000 7fa0e6ffd640 1 --2- 172.21.15.86:0/947925745 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564812664130 0x56481266ea50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.775+0000 7fa0e6ffd640 1 -- 172.21.15.86:0/947925745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56481266f020 msgr2=0x564812671430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:07.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.775+0000 7fa0e6ffd640 1 --2- 172.21.15.86:0/947925745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56481266f020 0x564812671430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.775+0000 7fa0e6ffd640 1 -- 172.21.15.86:0/947925745 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648123a27e0 con 0x564812663780 2024-04-21T08:42:07.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.775+0000 7fa0e67fc640 1 --2- 172.21.15.86:0/947925745 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564812664130 0x56481266ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:07.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.775+0000 7fa0e77fe640 1 --2- 172.21.15.86:0/947925745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56481266f020 0x564812671430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:07.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.775+0000 7fa0e67fc640 1 --2- 172.21.15.86:0/764892566 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa0d00a3d20 0x7fa0d00a61e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8100d20 tx=0x7fa0dc02cbe0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:07.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.775+0000 7fa0e6ffd640 1 --2- 172.21.15.86:0/947925745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564812663780 0x564812663b60 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fa0d800ac10 tx=0x7fa0d800f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d148cf1bc10a9942 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:07.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.776+0000 7fa0cd7fa640 1 -- 172.21.15.86:0/947925745 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d801d070 con 0x564812663780 2024-04-21T08:42:07.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.776+0000 7fa0cd7fa640 1 -- 172.21.15.86:0/947925745 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0d8018af0 con 0x564812663780 2024-04-21T08:42:07.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.776+0000 7fa0cd7fa640 1 -- 172.21.15.86:0/947925745 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8010980 con 0x564812663780 2024-04-21T08:42:07.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.776+0000 7fa0f002c740 1 -- 172.21.15.86:0/947925745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564812663780 msgr2=0x564812663b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:07.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.776+0000 7fa0f002c740 1 --2- 172.21.15.86:0/947925745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564812663780 0x564812663b60 secure :-1 s=READY pgs=1030 cs=0 l=1 rev1=1 crypto rx=0x7fa0d800ac10 tx=0x7fa0d800f790 comp rx=0 tx=0).stop 2024-04-21T08:42:07.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.776+0000 7fa0f002c740 1 -- 172.21.15.86:0/947925745 shutdown_connections 2024-04-21T08:42:07.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.776+0000 7fa0f002c740 1 --2- 172.21.15.86:0/947925745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56481266f020 0x564812671430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.776+0000 7fa0f002c740 1 --2- 172.21.15.86:0/947925745 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564812664130 0x56481266ea50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.776+0000 7fa0f002c740 1 --2- 172.21.15.86:0/947925745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564812663780 0x564812663b60 unknown :-1 s=CLOSED pgs=1030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.776+0000 7fa0f002c740 1 -- 172.21.15.86:0/947925745 >> 172.21.15.86:0/947925745 conn(0x56481265dfb0 msgr2=0x56481265e390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:07.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.776+0000 7fa0f002c740 1 -- 172.21.15.86:0/947925745 shutdown_connections 2024-04-21T08:42:07.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.776+0000 7fa0f002c740 1 -- 172.21.15.86:0/947925745 wait complete. 2024-04-21T08:42:07.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.777+0000 7fa0f002c740 1 -- start start 2024-04-21T08:42:07.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.777+0000 7fa0f002c740 1 Processor -- start 2024-04-21T08:42:07.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.777+0000 7fa0f002c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564812663780 0x56481266bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:07.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.777+0000 7fa0f002c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564812664130 0x56481266c4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:07.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.777+0000 7fa0f002c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56481266f020 0x56481266ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:07.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.777+0000 7fa0f002c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564812671f50 con 0x564812663780 2024-04-21T08:42:07.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.777+0000 7fa0f002c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564812581580 con 0x56481266f020 2024-04-21T08:42:07.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.777+0000 7fa0f002c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5648126720d0 con 0x564812664130 2024-04-21T08:42:07.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.777+0000 7fa0f002c740 10 client.0 _ll_register_callbacks cb 0x564812587650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:07.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.777+0000 7fa0e6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564812663780 0x56481266bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:07.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.777+0000 7fa0e6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564812663780 0x56481266bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40882/0 (socket says 172.21.15.86:40882) 2024-04-21T08:42:07.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.777+0000 7fa0e6ffd640 1 -- 172.21.15.86:0/311004429 learned_addr learned my addr 172.21.15.86:0/311004429 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:07.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.777+0000 7fa0e77fe640 1 --2- 172.21.15.86:0/311004429 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56481266f020 0x56481266ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:07.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.777+0000 7fa0e67fc640 1 --2- 172.21.15.86:0/311004429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564812664130 0x56481266c4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:07.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.777+0000 7fa0e6ffd640 1 -- 172.21.15.86:0/311004429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564812664130 msgr2=0x56481266c4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:07.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.777+0000 7fa0e6ffd640 1 --2- 172.21.15.86:0/311004429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564812664130 0x56481266c4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.777+0000 7fa0e6ffd640 1 -- 172.21.15.86:0/311004429 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56481266f020 msgr2=0x56481266ca00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:07.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.777+0000 7fa0e6ffd640 1 --2- 172.21.15.86:0/311004429 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56481266f020 0x56481266ca00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.777+0000 7fa0e6ffd640 1 -- 172.21.15.86:0/311004429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564812402f70 con 0x564812663780 2024-04-21T08:42:07.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.778+0000 7fa0e6ffd640 1 --2- 172.21.15.86:0/311004429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564812663780 0x56481266bf80 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fa0d801b040 tx=0x7fa0d80186f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5dd0e31102ceb15 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:07.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.778+0000 7fa0caff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:07.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.778+0000 7fa0caff5640 1 -- 172.21.15.86:0/311004429 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8027020 con 0x564812663780 2024-04-21T08:42:07.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.778+0000 7fa0caff5640 1 -- 172.21.15.86:0/311004429 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fa0d8005040 con 0x564812663780 2024-04-21T08:42:07.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.778+0000 7fa0f002c740 1 -- 172.21.15.86:0/311004429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0d8018af0 con 0x564812663780 2024-04-21T08:42:07.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.778+0000 7fa0caff5640 1 -- 172.21.15.86:0/311004429 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa0d801d040 con 0x564812663780 2024-04-21T08:42:07.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.778+0000 7fa0f002c740 10 client.17262 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:07.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.778+0000 7fa0f002c740 1 -- 172.21.15.86:0/311004429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56481229a170 con 0x564812663780 2024-04-21T08:42:07.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.778+0000 7fa0caff5640 1 -- 172.21.15.86:0/311004429 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0d8018af0 con 0x564812663780 2024-04-21T08:42:07.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.778+0000 7fa0caff5640 1 -- 172.21.15.86:0/311004429 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa0d8010790 con 0x564812663780 2024-04-21T08:42:07.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.778+0000 7fa0f002c740 10 client.17262 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:07.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.778+0000 7fa0f002c740 10 client.17262 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:07.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.778+0000 7fa0f002c740 20 client.17262 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:07.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.778+0000 7fa0f002c740 1 --2- 172.21.15.86:0/311004429 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56481267f410 0x564812681800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:07.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.778+0000 7fa0f002c740 4 client.17262 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:07.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.778+0000 7fa0f002c740 1 -- 172.21.15.86:0/311004429 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5648125d5920 con 0x56481267f410 2024-04-21T08:42:07.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.779+0000 7fa0e67fc640 1 --2- 172.21.15.86:0/311004429 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56481267f410 0x564812681800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:07.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.779+0000 7fa0caff5640 10 client.17262 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:07.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.779+0000 7fa0e67fc640 1 --2- 172.21.15.86:0/311004429 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56481267f410 0x564812681800 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=71e78cb3e9f0851d server_cookie=cf164b3044eef26e in_seq=0 out_seq=0 2024-04-21T08:42:07.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.780+0000 7fa0caff5640 1 -- 172.21.15.86:0/311004429 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5648125d5920 con 0x56481267f410 2024-04-21T08:42:07.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.780+0000 7fa0caff5640 10 client.17262 handle_command_reply: tid=0 2024-04-21T08:42:07.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.780+0000 7fa0f002c740 1 client.17262 shutdown 2024-04-21T08:42:07.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.780+0000 7fa0f002c740 1 -- 172.21.15.86:0/311004429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564812663780 msgr2=0x56481266bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:07.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.780+0000 7fa0f002c740 1 --2- 172.21.15.86:0/311004429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564812663780 0x56481266bf80 secure :-1 s=READY pgs=1031 cs=0 l=1 rev1=1 crypto rx=0x7fa0d801b040 tx=0x7fa0d80186f0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.780+0000 7fa0f002c740 1 -- 172.21.15.86:0/311004429 shutdown_connections 2024-04-21T08:42:07.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.780+0000 7fa0f002c740 1 --2- 172.21.15.86:0/311004429 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56481267f410 0x564812681800 crc :-1 s=READY pgs=131 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.780+0000 7fa0f002c740 1 --2- 172.21.15.86:0/311004429 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56481266f020 0x56481266ca00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.780+0000 7fa0f002c740 1 --2- 172.21.15.86:0/311004429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564812664130 0x56481266c4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.780+0000 7fa0f002c740 1 --2- 172.21.15.86:0/311004429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564812663780 0x56481266bf80 unknown :-1 s=CLOSED pgs=1031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.780+0000 7fa0caff5640 0 client.17262 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:07.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.780+0000 7fa0f002c740 1 -- 172.21.15.86:0/311004429 >> 172.21.15.86:0/311004429 conn(0x56481265dfb0 msgr2=0x56481265fd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:07.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.780+0000 7fa0f002c740 1 -- 172.21.15.86:0/311004429 shutdown_connections 2024-04-21T08:42:07.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.780+0000 7fa0f002c740 1 -- 172.21.15.86:0/311004429 wait complete. 2024-04-21T08:42:07.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.780+0000 7fa0f002c740 20 client.17262 trim_cache size 0 max 16384 2024-04-21T08:42:07.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.797+0000 7fa0f002c740 1 Processor -- start 2024-04-21T08:42:07.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.797+0000 7fa0f002c740 1 -- start start 2024-04-21T08:42:07.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.798+0000 7fa0f002c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564812663780 0x56481266e6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:07.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.798+0000 7fa0f002c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564812664130 0x56481267ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:07.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.798+0000 7fa0f002c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56481266f020 0x564812686e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:07.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.798+0000 7fa0f002c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e81ad040 con 0x564812663780 2024-04-21T08:42:07.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.798+0000 7fa0f002c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e81aeb80 con 0x56481266f020 2024-04-21T08:42:07.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.798+0000 7fa0f002c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e81ad710 con 0x564812664130 2024-04-21T08:42:07.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.798+0000 7fa0e6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564812663780 0x56481266e6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:07.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.798+0000 7fa0e6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564812663780 0x56481266e6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40898/0 (socket says 172.21.15.86:40898) 2024-04-21T08:42:07.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.798+0000 7fa0e6ffd640 1 -- 172.21.15.86:0/4248108005 learned_addr learned my addr 172.21.15.86:0/4248108005 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:07.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.798+0000 7fa0e67fc640 1 --2- 172.21.15.86:0/4248108005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564812664130 0x56481267ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:07.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.798+0000 7fa0e6ffd640 1 -- 172.21.15.86:0/4248108005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564812664130 msgr2=0x56481267ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:07.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.798+0000 7fa0e6ffd640 1 --2- 172.21.15.86:0/4248108005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564812664130 0x56481267ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.798+0000 7fa0e6ffd640 1 -- 172.21.15.86:0/4248108005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56481266f020 msgr2=0x564812686e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:07.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.798+0000 7fa0e77fe640 1 --2- 172.21.15.86:0/4248108005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56481266f020 0x564812686e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:07.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.798+0000 7fa0e6ffd640 1 --2- 172.21.15.86:0/4248108005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56481266f020 0x564812686e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.798+0000 7fa0e6ffd640 1 -- 172.21.15.86:0/4248108005 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5648126645c0 con 0x564812663780 2024-04-21T08:42:07.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.798+0000 7fa0e6ffd640 1 --2- 172.21.15.86:0/4248108005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564812663780 0x56481266e6b0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fa0d800a9a0 tx=0x7fa0d8001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0692dea10e969e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:07.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.798+0000 7fa0ccff9640 1 -- 172.21.15.86:0/4248108005 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8027070 con 0x564812663780 2024-04-21T08:42:07.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.798+0000 7fa0ccff9640 1 -- 172.21.15.86:0/4248108005 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0d800a570 con 0x564812663780 2024-04-21T08:42:07.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.798+0000 7fa0ccff9640 1 -- 172.21.15.86:0/4248108005 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8005290 con 0x564812663780 2024-04-21T08:42:07.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.799+0000 7fa0f002c740 1 -- 172.21.15.86:0/4248108005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564812663780 msgr2=0x56481266e6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:07.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.799+0000 7fa0f002c740 1 --2- 172.21.15.86:0/4248108005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564812663780 0x56481266e6b0 secure :-1 s=READY pgs=1032 cs=0 l=1 rev1=1 crypto rx=0x7fa0d800a9a0 tx=0x7fa0d8001e50 comp rx=0 tx=0).stop 2024-04-21T08:42:07.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.799+0000 7fa0f002c740 1 -- 172.21.15.86:0/4248108005 shutdown_connections 2024-04-21T08:42:07.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.799+0000 7fa0f002c740 1 --2- 172.21.15.86:0/4248108005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56481266f020 0x564812686e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.799+0000 7fa0f002c740 1 --2- 172.21.15.86:0/4248108005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564812664130 0x56481267ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.799+0000 7fa0f002c740 1 --2- 172.21.15.86:0/4248108005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564812663780 0x56481266e6b0 unknown :-1 s=CLOSED pgs=1032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.799+0000 7fa0f002c740 1 -- 172.21.15.86:0/4248108005 >> 172.21.15.86:0/4248108005 conn(0x56481265dfb0 msgr2=0x56481266b010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:07.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.799+0000 7fa0f002c740 1 -- 172.21.15.86:0/4248108005 shutdown_connections 2024-04-21T08:42:07.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.799+0000 7fa0f002c740 1 -- 172.21.15.86:0/4248108005 wait complete. 2024-04-21T08:42:07.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.799+0000 7fa0f002c740 1 -- start start 2024-04-21T08:42:07.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.799+0000 7fa0f002c740 1 Processor -- start 2024-04-21T08:42:07.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.799+0000 7fa0f002c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564812663780 0x564812668d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:07.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.799+0000 7fa0f002c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564812664130 0x5648126692d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:07.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.799+0000 7fa0f002c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56481266f020 0x564812669810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:07.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.799+0000 7fa0f002c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e81ae4b0 con 0x564812663780 2024-04-21T08:42:07.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.799+0000 7fa0f002c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e81adde0 con 0x56481266f020 2024-04-21T08:42:07.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.799+0000 7fa0f002c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa0e81ab000 con 0x564812664130 2024-04-21T08:42:07.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.799+0000 7fa0f002c740 10 client.0 _ll_register_callbacks cb 0x7fa0d801b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:07.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.800+0000 7fa0e6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564812663780 0x564812668d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:07.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.800+0000 7fa0e6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564812663780 0x564812668d90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40900/0 (socket says 172.21.15.86:40900) 2024-04-21T08:42:07.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.800+0000 7fa0e6ffd640 1 -- 172.21.15.86:0/3753027624 learned_addr learned my addr 172.21.15.86:0/3753027624 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:07.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.800+0000 7fa0e77fe640 1 --2- 172.21.15.86:0/3753027624 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56481266f020 0x564812669810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:07.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.800+0000 7fa0e67fc640 1 --2- 172.21.15.86:0/3753027624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564812664130 0x5648126692d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:07.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.800+0000 7fa0e6ffd640 1 -- 172.21.15.86:0/3753027624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564812664130 msgr2=0x5648126692d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:07.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.800+0000 7fa0e6ffd640 1 --2- 172.21.15.86:0/3753027624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564812664130 0x5648126692d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.800+0000 7fa0e6ffd640 1 -- 172.21.15.86:0/3753027624 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56481266f020 msgr2=0x564812669810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:07.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.800+0000 7fa0e6ffd640 1 --2- 172.21.15.86:0/3753027624 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56481266f020 0x564812669810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.800+0000 7fa0e6ffd640 1 -- 172.21.15.86:0/3753027624 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5648126e4760 con 0x564812663780 2024-04-21T08:42:07.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.800+0000 7fa0e6ffd640 1 --2- 172.21.15.86:0/3753027624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564812663780 0x564812668d90 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8018c40 tx=0x7fa0d8010da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d43635b7152ea9e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:07.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.800+0000 7fa0caff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:07.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.800+0000 7fa0caff5640 1 -- 172.21.15.86:0/3753027624 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0d8027040 con 0x564812663780 2024-04-21T08:42:07.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.800+0000 7fa0f002c740 1 -- 172.21.15.86:0/3753027624 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0d800a570 con 0x564812663780 2024-04-21T08:42:07.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.800+0000 7fa0caff5640 1 -- 172.21.15.86:0/3753027624 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fa0d8005040 con 0x564812663780 2024-04-21T08:42:07.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.800+0000 7fa0caff5640 1 -- 172.21.15.86:0/3753027624 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa0d8010970 con 0x564812663780 2024-04-21T08:42:07.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.800+0000 7fa0f002c740 10 client.17268 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:07.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.800+0000 7fa0f002c740 1 -- 172.21.15.86:0/3753027624 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5648126e5aa0 con 0x564812663780 2024-04-21T08:42:07.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.800+0000 7fa0caff5640 1 -- 172.21.15.86:0/3753027624 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0d800a570 con 0x564812663780 2024-04-21T08:42:07.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.801+0000 7fa0caff5640 1 -- 172.21.15.86:0/3753027624 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa0d8004c00 con 0x564812663780 2024-04-21T08:42:07.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.801+0000 7fa0f002c740 10 client.17268 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:07.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.801+0000 7fa0f002c740 10 client.17268 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:07.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.801+0000 7fa0f002c740 20 client.17268 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:07.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.801+0000 7fa0f002c740 1 --2- 172.21.15.86:0/3753027624 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5648126e5f50 0x5648126e8370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:07.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.801+0000 7fa0f002c740 4 client.17268 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:42:07.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.801+0000 7fa0f002c740 1 -- 172.21.15.86:0/3753027624 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5648125cf790 con 0x5648126e5f50 2024-04-21T08:42:07.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.801+0000 7fa0e67fc640 1 --2- 172.21.15.86:0/3753027624 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5648126e5f50 0x5648126e8370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:07.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.801+0000 7fa0caff5640 10 client.17268 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:07.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.801+0000 7fa0e67fc640 1 --2- 172.21.15.86:0/3753027624 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5648126e5f50 0x5648126e8370 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3c578115b85a538e server_cookie=742614440f62670a in_seq=0 out_seq=0 2024-04-21T08:42:07.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.802+0000 7fa0caff5640 1 -- 172.21.15.86:0/3753027624 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5648125cf790 con 0x5648126e5f50 2024-04-21T08:42:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.802+0000 7fa0caff5640 10 client.17268 handle_command_reply: tid=0 2024-04-21T08:42:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.802+0000 7fa0f002c740 1 client.17268 shutdown 2024-04-21T08:42:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.802+0000 7fa0f002c740 1 -- 172.21.15.86:0/3753027624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564812663780 msgr2=0x564812668d90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.802+0000 7fa0f002c740 1 --2- 172.21.15.86:0/3753027624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564812663780 0x564812668d90 secure :-1 s=READY pgs=1033 cs=0 l=1 rev1=1 crypto rx=0x7fa0d8018c40 tx=0x7fa0d8010da0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.803+0000 7fa0f002c740 1 -- 172.21.15.86:0/3753027624 shutdown_connections 2024-04-21T08:42:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.803+0000 7fa0f002c740 1 --2- 172.21.15.86:0/3753027624 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5648126e5f50 0x5648126e8370 crc :-1 s=READY pgs=132 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.803+0000 7fa0f002c740 1 --2- 172.21.15.86:0/3753027624 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56481266f020 0x564812669810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.803+0000 7fa0f002c740 1 --2- 172.21.15.86:0/3753027624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564812664130 0x5648126692d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.803+0000 7fa0f002c740 1 --2- 172.21.15.86:0/3753027624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564812663780 0x564812668d90 unknown :-1 s=CLOSED pgs=1033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.803+0000 7fa0f002c740 1 -- 172.21.15.86:0/3753027624 >> 172.21.15.86:0/3753027624 conn(0x56481265dfb0 msgr2=0x56481266a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.803+0000 7fa0caff5640 0 client.17268 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.803+0000 7fa0f002c740 1 -- 172.21.15.86:0/3753027624 shutdown_connections 2024-04-21T08:42:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.803+0000 7fa0f002c740 1 -- 172.21.15.86:0/3753027624 wait complete. 2024-04-21T08:42:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.803+0000 7fa0f002c740 20 client.17268 trim_cache size 0 max 16384 2024-04-21T08:42:07.810 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:42:07.810 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-21T08:42:07.810 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "c444f00b-2808-409a-adbc-a43363093035", 2024-04-21T08:42:07.810 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-21T08:42:07.810 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:42:07.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.803+0000 7fa0ed359640 1 -- 172.21.15.86:0/764892566 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa0d00a3d20 msgr2=0x7fa0d00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:07.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.803+0000 7fa0ed359640 1 --2- 172.21.15.86:0/764892566 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa0d00a3d20 0x7fa0d00a61e0 secure :-1 s=READY pgs=157 cs=0 l=1 rev1=1 crypto rx=0x7fa0e8100d20 tx=0x7fa0dc02cbe0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.803+0000 7fa0ed359640 1 -- 172.21.15.86:0/764892566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e8105ea0 msgr2=0x7fa0e819c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:07.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.803+0000 7fa0ed359640 1 --2- 172.21.15.86:0/764892566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e8105ea0 0x7fa0e819c010 secure :-1 s=READY pgs=1029 cs=0 l=1 rev1=1 crypto rx=0x7fa0e000eaa0 tx=0x7fa0e000ef60 comp rx=0 tx=0).stop 2024-04-21T08:42:07.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.803+0000 7fa0ed359640 1 -- 172.21.15.86:0/764892566 shutdown_connections 2024-04-21T08:42:07.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.803+0000 7fa0ed359640 1 --2- 172.21.15.86:0/764892566 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa0d00a3d20 0x7fa0d00a61e0 unknown :-1 s=CLOSED pgs=157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.803+0000 7fa0ed359640 1 --2- 172.21.15.86:0/764892566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0e8105ea0 0x7fa0e819c010 unknown :-1 s=CLOSED pgs=1029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.803+0000 7fa0ed359640 1 --2- 172.21.15.86:0/764892566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0e8101ad0 0x7fa0e8197c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.803+0000 7fa0ed359640 1 --2- 172.21.15.86:0/764892566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0e8101130 0x7fa0e8197750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.803+0000 7fa0ed359640 1 -- 172.21.15.86:0/764892566 >> 172.21.15.86:0/764892566 conn(0x7fa0e8077e30 msgr2=0x7fa0e80fefa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:07.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.803+0000 7fa0ed359640 1 -- 172.21.15.86:0/764892566 shutdown_connections 2024-04-21T08:42:07.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.803+0000 7fa0ed359640 1 -- 172.21.15.86:0/764892566 wait complete. 2024-04-21T08:42:07.814 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:42:07.815 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:42:07.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.915+0000 7f7ccaec0640 1 -- 172.21.15.86:0/1484456295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cc4103b10 msgr2=0x7f7cc410bff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:07.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.915+0000 7f7ccaec0640 1 --2- 172.21.15.86:0/1484456295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cc4103b10 0x7f7cc410bff0 secure :-1 s=READY pgs=1034 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0009900 tx=0x7f7cb002bd90 comp rx=0 tx=0).stop 2024-04-21T08:42:07.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.915+0000 7f7ccaec0640 1 -- 172.21.15.86:0/1484456295 shutdown_connections 2024-04-21T08:42:07.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.915+0000 7f7ccaec0640 1 --2- 172.21.15.86:0/1484456295 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7cc410c530 0x7f7cc410e9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.915+0000 7f7ccaec0640 1 --2- 172.21.15.86:0/1484456295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cc4103b10 0x7f7cc410bff0 unknown :-1 s=CLOSED pgs=1034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.915+0000 7f7ccaec0640 1 --2- 172.21.15.86:0/1484456295 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7cc41031f0 0x7f7cc41035d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.915+0000 7f7ccaec0640 1 -- 172.21.15.86:0/1484456295 >> 172.21.15.86:0/1484456295 conn(0x7f7cc40771d0 msgr2=0x7f7cc40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:07.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.915+0000 7f7ccaec0640 1 -- 172.21.15.86:0/1484456295 shutdown_connections 2024-04-21T08:42:07.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.915+0000 7f7ccaec0640 1 -- 172.21.15.86:0/1484456295 wait complete. 2024-04-21T08:42:07.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.915+0000 7f7ccaec0640 1 Processor -- start 2024-04-21T08:42:07.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.915+0000 7f7ccaec0640 1 -- start start 2024-04-21T08:42:07.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.916+0000 7f7ccaec0640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7cc41031f0 0x7f7cc4100e40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:07.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.916+0000 7f7ccaec0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cc4103b10 0x7f7cc4101380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:07.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.916+0000 7f7ccaec0640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7cc410c530 0x7f7cc41018c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:07.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.916+0000 7f7ccaec0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc4112290 con 0x7f7cc4103b10 2024-04-21T08:42:07.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.916+0000 7f7ccaec0640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc4112110 con 0x7f7cc410c530 2024-04-21T08:42:07.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.916+0000 7f7ccaec0640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc4112410 con 0x7f7cc41031f0 2024-04-21T08:42:07.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.916+0000 7f7cc3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cc4103b10 0x7f7cc4101380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:07.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.916+0000 7f7cc3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cc4103b10 0x7f7cc4101380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40920/0 (socket says 172.21.15.86:40920) 2024-04-21T08:42:07.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.916+0000 7f7cc3fff640 1 -- 172.21.15.86:0/1195524866 learned_addr learned my addr 172.21.15.86:0/1195524866 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:07.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.916+0000 7f7cc8c5c640 1 --2- 172.21.15.86:0/1195524866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7cc41031f0 0x7f7cc4100e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:07.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.916+0000 7f7cc945d640 1 --2- 172.21.15.86:0/1195524866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7cc410c530 0x7f7cc41018c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:07.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.916+0000 7f7cc3fff640 1 -- 172.21.15.86:0/1195524866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7cc41031f0 msgr2=0x7f7cc4100e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:07.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.916+0000 7f7cc3fff640 1 --2- 172.21.15.86:0/1195524866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7cc41031f0 0x7f7cc4100e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.916+0000 7f7cc3fff640 1 -- 172.21.15.86:0/1195524866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7cc410c530 msgr2=0x7f7cc41018c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:07.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.916+0000 7f7cc3fff640 1 --2- 172.21.15.86:0/1195524866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7cc410c530 0x7f7cc41018c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.916+0000 7f7cc3fff640 1 -- 172.21.15.86:0/1195524866 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7cc41a6d30 con 0x7f7cc4103b10 2024-04-21T08:42:07.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.916+0000 7f7cc945d640 1 --2- 172.21.15.86:0/1195524866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7cc410c530 0x7f7cc41018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:07.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.916+0000 7f7cc3fff640 1 --2- 172.21.15.86:0/1195524866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cc4103b10 0x7f7cc4101380 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0005ab0 tx=0x7f7cb002d9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:07.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.916+0000 7f7cc1ffb640 1 -- 172.21.15.86:0/1195524866 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0043020 con 0x7f7cc4103b10 2024-04-21T08:42:07.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.916+0000 7f7cc1ffb640 1 -- 172.21.15.86:0/1195524866 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cb002ddd0 con 0x7f7cc4103b10 2024-04-21T08:42:07.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.916+0000 7f7ccaec0640 1 -- 172.21.15.86:0/1195524866 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7cc41a6fc0 con 0x7f7cc4103b10 2024-04-21T08:42:07.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.916+0000 7f7cc1ffb640 1 -- 172.21.15.86:0/1195524866 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb0003b80 con 0x7f7cc4103b10 2024-04-21T08:42:07.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.917+0000 7f7ccaec0640 1 -- 172.21.15.86:0/1195524866 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7cc41a7480 con 0x7f7cc4103b10 2024-04-21T08:42:07.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.919+0000 7f7cc1ffb640 1 -- 172.21.15.86:0/1195524866 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7cb0046020 con 0x7f7cc4103b10 2024-04-21T08:42:07.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.919+0000 7f7ccdb81740 1 Processor -- start 2024-04-21T08:42:07.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.919+0000 7f7ccdb81740 1 -- start start 2024-04-21T08:42:07.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.919+0000 7f7cc1ffb640 1 --2- 172.21.15.86:0/1195524866 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7cb40a3d20 0x7f7cb40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:07.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.919+0000 7f7ccdb81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55798bb99580 0x55798bb99960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:07.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.919+0000 7f7ccdb81740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55798bb99ea0 0x55798bba2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:07.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.919+0000 7f7cc945d640 1 --2- 172.21.15.86:0/1195524866 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7cb40a3d20 0x7f7cb40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:07.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.919+0000 7f7ccdb81740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55798bba2a80 0x55798bba4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:07.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.919+0000 7f7cc1ffb640 1 -- 172.21.15.86:0/1195524866 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f7cb00f2dc0 con 0x7f7cc4103b10 2024-04-21T08:42:07.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.919+0000 7f7ccdb81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55798bafe1e0 con 0x55798bb99580 2024-04-21T08:42:07.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.919+0000 7f7ccdb81740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55798ba9b390 con 0x55798bba2a80 2024-04-21T08:42:07.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.919+0000 7f7ccdb81740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55798bafa1a0 con 0x55798bb99ea0 2024-04-21T08:42:07.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.919+0000 7f7cc8c5c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55798bb99580 0x55798bb99960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:07.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.920+0000 7f7cc8c5c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55798bb99580 0x55798bb99960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40926/0 (socket says 172.21.15.86:40926) 2024-04-21T08:42:07.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.920+0000 7f7cc8c5c640 1 -- 172.21.15.86:0/3213441860 learned_addr learned my addr 172.21.15.86:0/3213441860 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:07.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.920+0000 7f7cc945d640 1 --2- 172.21.15.86:0/3213441860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55798bba2a80 0x55798bba4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:07.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.920+0000 7f7cc8c5c640 1 -- 172.21.15.86:0/3213441860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55798bb99ea0 msgr2=0x55798bba2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:07.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.920+0000 7f7cc3fff640 1 --2- 172.21.15.86:0/3213441860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55798bb99ea0 0x55798bba2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:07.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.920+0000 7f7cc8c5c640 1 --2- 172.21.15.86:0/3213441860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55798bb99ea0 0x55798bba2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.920+0000 7f7cc8c5c640 1 -- 172.21.15.86:0/3213441860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55798bba2a80 msgr2=0x55798bba4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:07.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.920+0000 7f7cc8c5c640 1 --2- 172.21.15.86:0/3213441860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55798bba2a80 0x55798bba4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.920+0000 7f7cc8c5c640 1 -- 172.21.15.86:0/3213441860 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55798b8da7d0 con 0x55798bb99580 2024-04-21T08:42:07.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.920+0000 7f7cc3fff640 1 --2- 172.21.15.86:0/3213441860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55798bb99ea0 0x55798bba2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:07.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.920+0000 7f7cc945d640 1 --2- 172.21.15.86:0/3213441860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55798bba2a80 0x55798bba4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:07.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.920+0000 7f7cc945d640 1 --2- 172.21.15.86:0/1195524866 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7cb40a3d20 0x7f7cb40a61e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc007ea0 tx=0x7f7cbc00c690 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:07.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.920+0000 7f7cc8c5c640 1 --2- 172.21.15.86:0/3213441860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55798bb99580 0x55798bb99960 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800bcb0 tx=0x7f7cb800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b06e968d9729dea server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:07.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.920+0000 7f7caeffd640 1 -- 172.21.15.86:0/3213441860 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8015070 con 0x55798bb99580 2024-04-21T08:42:07.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.920+0000 7f7caeffd640 1 -- 172.21.15.86:0/3213441860 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cb801a910 con 0x55798bb99580 2024-04-21T08:42:07.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.920+0000 7f7caeffd640 1 -- 172.21.15.86:0/3213441860 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb80118b0 con 0x55798bb99580 2024-04-21T08:42:07.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.921+0000 7f7ccdb81740 1 -- 172.21.15.86:0/3213441860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55798bb99580 msgr2=0x55798bb99960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:07.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.921+0000 7f7ccdb81740 1 --2- 172.21.15.86:0/3213441860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55798bb99580 0x55798bb99960 secure :-1 s=READY pgs=1036 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800bcb0 tx=0x7f7cb800f730 comp rx=0 tx=0).stop 2024-04-21T08:42:07.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.921+0000 7f7ccdb81740 1 -- 172.21.15.86:0/3213441860 shutdown_connections 2024-04-21T08:42:07.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.921+0000 7f7ccdb81740 1 --2- 172.21.15.86:0/3213441860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55798bba2a80 0x55798bba4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.921+0000 7f7ccdb81740 1 --2- 172.21.15.86:0/3213441860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55798bb99ea0 0x55798bba2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.921+0000 7f7ccdb81740 1 --2- 172.21.15.86:0/3213441860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55798bb99580 0x55798bb99960 unknown :-1 s=CLOSED pgs=1036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.921+0000 7f7ccdb81740 1 -- 172.21.15.86:0/3213441860 >> 172.21.15.86:0/3213441860 conn(0x55798bb93cb0 msgr2=0x55798bb94090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:07.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.921+0000 7f7ccdb81740 1 -- 172.21.15.86:0/3213441860 shutdown_connections 2024-04-21T08:42:07.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.921+0000 7f7ccdb81740 1 -- 172.21.15.86:0/3213441860 wait complete. 2024-04-21T08:42:07.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.921+0000 7f7ccdb81740 1 -- start start 2024-04-21T08:42:07.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.921+0000 7f7ccdb81740 1 Processor -- start 2024-04-21T08:42:07.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.922+0000 7f7ccdb81740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55798bb99580 0x55798bb9faa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:07.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.922+0000 7f7ccdb81740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55798bb99ea0 0x55798bb9ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:07.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.922+0000 7f7ccdb81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55798bba2a80 0x55798bba0520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:07.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.922+0000 7f7ccdb81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55798bba5870 con 0x55798bba2a80 2024-04-21T08:42:07.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.922+0000 7f7ccdb81740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55798bab72f0 con 0x55798bb99ea0 2024-04-21T08:42:07.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.922+0000 7f7ccdb81740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55798bba59f0 con 0x55798bb99580 2024-04-21T08:42:07.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.922+0000 7f7ccdb81740 10 client.0 _ll_register_callbacks cb 0x55798bac4580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:07.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.922+0000 7f7cc945d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55798bba2a80 0x55798bba0520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:07.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.922+0000 7f7cc945d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55798bba2a80 0x55798bba0520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40932/0 (socket says 172.21.15.86:40932) 2024-04-21T08:42:07.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.922+0000 7f7cc945d640 1 -- 172.21.15.86:0/1960744900 learned_addr learned my addr 172.21.15.86:0/1960744900 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:07.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.922+0000 7f7cc3fff640 1 --2- 172.21.15.86:0/1960744900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55798bb99ea0 0x55798bb9ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:07.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.922+0000 7f7cc945d640 1 -- 172.21.15.86:0/1960744900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55798bb99580 msgr2=0x55798bb9faa0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:42:07.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.922+0000 7f7cc945d640 1 --2- 172.21.15.86:0/1960744900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55798bb99580 0x55798bb9faa0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.922+0000 7f7cc945d640 1 -- 172.21.15.86:0/1960744900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55798bb99ea0 msgr2=0x55798bb9ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:07.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.922+0000 7f7cc945d640 1 --2- 172.21.15.86:0/1960744900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55798bb99ea0 0x55798bb9ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.922+0000 7f7cc945d640 1 -- 172.21.15.86:0/1960744900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55798b8103f0 con 0x55798bba2a80 2024-04-21T08:42:07.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.922+0000 7f7cc945d640 1 --2- 172.21.15.86:0/1960744900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55798bba2a80 0x55798bba0520 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc011e90 tx=0x7f7cbc004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9cd1e6243d73a074 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:07.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.922+0000 7f7ca3fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:07.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.922+0000 7f7ca3fff640 1 -- 172.21.15.86:0/1960744900 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cbc021030 con 0x55798bba2a80 2024-04-21T08:42:07.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.922+0000 7f7ca3fff640 1 -- 172.21.15.86:0/1960744900 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f7cbc003a10 con 0x55798bba2a80 2024-04-21T08:42:07.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.922+0000 7f7ccdb81740 1 -- 172.21.15.86:0/1960744900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cb801a910 con 0x55798bba2a80 2024-04-21T08:42:07.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.922+0000 7f7ca3fff640 1 -- 172.21.15.86:0/1960744900 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7cbc02b050 con 0x55798bba2a80 2024-04-21T08:42:07.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.922+0000 7f7ccdb81740 10 client.17280 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:07.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.922+0000 7f7ccdb81740 1 -- 172.21.15.86:0/1960744900 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55798b7d00f0 con 0x55798bba2a80 2024-04-21T08:42:07.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.923+0000 7f7ca3fff640 1 -- 172.21.15.86:0/1960744900 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cb801a910 con 0x55798bba2a80 2024-04-21T08:42:07.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.923+0000 7f7ca3fff640 1 -- 172.21.15.86:0/1960744900 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7cbc00f5e0 con 0x55798bba2a80 2024-04-21T08:42:07.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.923+0000 7f7ccdb81740 10 client.17280 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:07.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.923+0000 7f7ccdb81740 10 client.17280 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:07.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.923+0000 7f7ccdb81740 20 client.17280 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:07.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.923+0000 7f7ccdb81740 1 --2- 172.21.15.86:0/1960744900 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55798bbb2b70 0x55798bbb4ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:07.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.923+0000 7f7ccdb81740 4 client.17280 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:07.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.923+0000 7f7ccdb81740 1 -- 172.21.15.86:0/1960744900 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55798bb0bc00 con 0x55798bbb2b70 2024-04-21T08:42:07.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.923+0000 7f7cc8c5c640 1 --2- 172.21.15.86:0/1960744900 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55798bbb2b70 0x55798bbb4ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:07.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.924+0000 7f7ca3fff640 10 client.17280 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:07.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.924+0000 7f7cc8c5c640 1 --2- 172.21.15.86:0/1960744900 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55798bbb2b70 0x55798bbb4ff0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcbb7b7a7e653649 server_cookie=900f483c7664364d in_seq=0 out_seq=0 2024-04-21T08:42:07.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.924+0000 7f7ca3fff640 1 -- 172.21.15.86:0/1960744900 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55798bb0bc00 con 0x55798bbb2b70 2024-04-21T08:42:07.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.924+0000 7f7ca3fff640 10 client.17280 handle_command_reply: tid=0 2024-04-21T08:42:07.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.924+0000 7f7ccdb81740 1 client.17280 shutdown 2024-04-21T08:42:07.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.924+0000 7f7ccdb81740 1 -- 172.21.15.86:0/1960744900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55798bba2a80 msgr2=0x55798bba0520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:07.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.924+0000 7f7ccdb81740 1 --2- 172.21.15.86:0/1960744900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55798bba2a80 0x55798bba0520 secure :-1 s=READY pgs=1037 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc011e90 tx=0x7f7cbc004a60 comp rx=0 tx=0).stop 2024-04-21T08:42:07.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.924+0000 7f7ccdb81740 1 -- 172.21.15.86:0/1960744900 shutdown_connections 2024-04-21T08:42:07.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.924+0000 7f7ccdb81740 1 --2- 172.21.15.86:0/1960744900 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55798bbb2b70 0x55798bbb4ff0 crc :-1 s=READY pgs=133 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.924+0000 7f7ccdb81740 1 --2- 172.21.15.86:0/1960744900 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55798bba2a80 0x55798bba0520 unknown :-1 s=CLOSED pgs=1037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.924+0000 7f7ca3fff640 0 client.17280 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:07.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.924+0000 7f7ccdb81740 1 --2- 172.21.15.86:0/1960744900 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55798bb99ea0 0x55798bb9ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.924+0000 7f7ccdb81740 1 --2- 172.21.15.86:0/1960744900 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55798bb99580 0x55798bb9faa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.924+0000 7f7ccdb81740 1 -- 172.21.15.86:0/1960744900 >> 172.21.15.86:0/1960744900 conn(0x55798bb93cb0 msgr2=0x55798bb95a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:07.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.925+0000 7f7ccdb81740 1 -- 172.21.15.86:0/1960744900 shutdown_connections 2024-04-21T08:42:07.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.925+0000 7f7ccdb81740 1 -- 172.21.15.86:0/1960744900 wait complete. 2024-04-21T08:42:07.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.925+0000 7f7ccdb81740 20 client.17280 trim_cache size 0 max 16384 2024-04-21T08:42:07.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.942+0000 7f7ccdb81740 1 Processor -- start 2024-04-21T08:42:07.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.942+0000 7f7ccdb81740 1 -- start start 2024-04-21T08:42:07.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.942+0000 7f7ccdb81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55798bb99580 0x55798bba21d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:07.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.942+0000 7f7ccdb81740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55798bb99ea0 0x55798bbae790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:07.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.942+0000 7f7ccdb81740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55798bba2a80 0x55798bbba8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:07.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.942+0000 7f7ccdb81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc41b1500 con 0x55798bb99580 2024-04-21T08:42:07.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.942+0000 7f7ccdb81740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc41b3040 con 0x55798bba2a80 2024-04-21T08:42:07.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.942+0000 7f7ccdb81740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc41b1bd0 con 0x55798bb99ea0 2024-04-21T08:42:07.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.942+0000 7f7cc8c5c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55798bb99580 0x55798bba21d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:07.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.942+0000 7f7cc8c5c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55798bb99580 0x55798bba21d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40948/0 (socket says 172.21.15.86:40948) 2024-04-21T08:42:07.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.942+0000 7f7cc8c5c640 1 -- 172.21.15.86:0/3961172793 learned_addr learned my addr 172.21.15.86:0/3961172793 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:07.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.942+0000 7f7cc8c5c640 1 -- 172.21.15.86:0/3961172793 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55798bb99ea0 msgr2=0x55798bbae790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:07.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.942+0000 7f7cc8c5c640 1 --2- 172.21.15.86:0/3961172793 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55798bb99ea0 0x55798bbae790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.942+0000 7f7cc8c5c640 1 -- 172.21.15.86:0/3961172793 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55798bba2a80 msgr2=0x55798bbba8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:07.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.942+0000 7f7cc945d640 1 --2- 172.21.15.86:0/3961172793 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55798bba2a80 0x55798bbba8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:07.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.942+0000 7f7cc8c5c640 1 --2- 172.21.15.86:0/3961172793 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55798bba2a80 0x55798bbba8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.942+0000 7f7cc8c5c640 1 -- 172.21.15.86:0/3961172793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55798bba4e70 con 0x55798bb99580 2024-04-21T08:42:07.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.942+0000 7f7cc945d640 1 --2- 172.21.15.86:0/3961172793 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55798bba2a80 0x55798bbba8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:07.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.943+0000 7f7cc8c5c640 1 --2- 172.21.15.86:0/3961172793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55798bb99580 0x55798bba21d0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800fc30 tx=0x7f7cb8011550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=35fa07c70c41cf30 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:07.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.943+0000 7f7cae7fc640 1 -- 172.21.15.86:0/3961172793 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb802a030 con 0x55798bb99580 2024-04-21T08:42:07.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.943+0000 7f7cae7fc640 1 -- 172.21.15.86:0/3961172793 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cb8005040 con 0x55798bb99580 2024-04-21T08:42:07.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.943+0000 7f7cae7fc640 1 -- 172.21.15.86:0/3961172793 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8015040 con 0x55798bb99580 2024-04-21T08:42:07.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.943+0000 7f7ccdb81740 1 -- 172.21.15.86:0/3961172793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55798bb99580 msgr2=0x55798bba21d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:07.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.943+0000 7f7ccdb81740 1 --2- 172.21.15.86:0/3961172793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55798bb99580 0x55798bba21d0 secure :-1 s=READY pgs=1038 cs=0 l=1 rev1=1 crypto rx=0x7f7cb800fc30 tx=0x7f7cb8011550 comp rx=0 tx=0).stop 2024-04-21T08:42:07.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.943+0000 7f7ccdb81740 1 -- 172.21.15.86:0/3961172793 shutdown_connections 2024-04-21T08:42:07.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.943+0000 7f7ccdb81740 1 --2- 172.21.15.86:0/3961172793 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55798bba2a80 0x55798bbba8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.943+0000 7f7ccdb81740 1 --2- 172.21.15.86:0/3961172793 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55798bb99ea0 0x55798bbae790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.943+0000 7f7ccdb81740 1 --2- 172.21.15.86:0/3961172793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55798bb99580 0x55798bba21d0 unknown :-1 s=CLOSED pgs=1038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.943+0000 7f7ccdb81740 1 -- 172.21.15.86:0/3961172793 >> 172.21.15.86:0/3961172793 conn(0x55798bb93cb0 msgr2=0x55798bb9ad60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:07.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.943+0000 7f7ccdb81740 1 -- 172.21.15.86:0/3961172793 shutdown_connections 2024-04-21T08:42:07.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.943+0000 7f7ccdb81740 1 -- 172.21.15.86:0/3961172793 wait complete. 2024-04-21T08:42:07.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.944+0000 7f7ccdb81740 1 -- start start 2024-04-21T08:42:07.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.944+0000 7f7ccdb81740 1 Processor -- start 2024-04-21T08:42:07.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.944+0000 7f7ccdb81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55798bb99580 0x55798bbaad70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:07.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.944+0000 7f7ccdb81740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55798bb99ea0 0x55798bba6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:07.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.944+0000 7f7ccdb81740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55798bba2a80 0x55798bba67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:07.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.944+0000 7f7ccdb81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc41b2970 con 0x55798bb99580 2024-04-21T08:42:07.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.944+0000 7f7ccdb81740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc41b22a0 con 0x55798bba2a80 2024-04-21T08:42:07.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.944+0000 7f7ccdb81740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7cc41af4c0 con 0x55798bb99ea0 2024-04-21T08:42:07.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.944+0000 7f7ccdb81740 10 client.0 _ll_register_callbacks cb 0x7f7cbc016030 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:07.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.944+0000 7f7cc8c5c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55798bb99580 0x55798bbaad70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:07.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.944+0000 7f7cc8c5c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55798bb99580 0x55798bbaad70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:40964/0 (socket says 172.21.15.86:40964) 2024-04-21T08:42:07.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.944+0000 7f7cc8c5c640 1 -- 172.21.15.86:0/3082239268 learned_addr learned my addr 172.21.15.86:0/3082239268 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:07.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.944+0000 7f7cc945d640 1 --2- 172.21.15.86:0/3082239268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55798bba2a80 0x55798bba67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:07.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.945+0000 7f7cc8c5c640 1 -- 172.21.15.86:0/3082239268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55798bb99ea0 msgr2=0x55798bba6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:07.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.945+0000 7f7cc8c5c640 1 --2- 172.21.15.86:0/3082239268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55798bb99ea0 0x55798bba6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.945+0000 7f7cc8c5c640 1 -- 172.21.15.86:0/3082239268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55798bba2a80 msgr2=0x55798bba67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:07.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.945+0000 7f7cc8c5c640 1 --2- 172.21.15.86:0/3082239268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55798bba2a80 0x55798bba67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.945+0000 7f7cc8c5c640 1 -- 172.21.15.86:0/3082239268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55798bba6ce0 con 0x55798bb99580 2024-04-21T08:42:07.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.945+0000 7f7cc945d640 1 --2- 172.21.15.86:0/3082239268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55798bba2a80 0x55798bba67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:07.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.945+0000 7f7cc8c5c640 1 --2- 172.21.15.86:0/3082239268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55798bb99580 0x55798bbaad70 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f7cb80114a0 tx=0x7f7cb800b950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36c0c51278922a4f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:07.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.945+0000 7f7ca3fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:07.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.945+0000 7f7ca3fff640 1 -- 172.21.15.86:0/3082239268 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7cb8034070 con 0x55798bb99580 2024-04-21T08:42:07.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.945+0000 7f7ca3fff640 1 -- 172.21.15.86:0/3082239268 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f7cb8004020 con 0x55798bb99580 2024-04-21T08:42:07.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.945+0000 7f7ccdb81740 1 -- 172.21.15.86:0/3082239268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7cb8005040 con 0x55798bb99580 2024-04-21T08:42:07.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.945+0000 7f7ca3fff640 1 -- 172.21.15.86:0/3082239268 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7cb8015070 con 0x55798bb99580 2024-04-21T08:42:07.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.945+0000 7f7ccdb81740 10 client.17286 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:07.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.945+0000 7f7ccdb81740 1 -- 172.21.15.86:0/3082239268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55798bc18dc0 con 0x55798bb99580 2024-04-21T08:42:07.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.945+0000 7f7ca3fff640 1 -- 172.21.15.86:0/3082239268 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7cb8005040 con 0x55798bb99580 2024-04-21T08:42:07.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.945+0000 7f7ca3fff640 1 -- 172.21.15.86:0/3082239268 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7cb8003680 con 0x55798bb99580 2024-04-21T08:42:07.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.945+0000 7f7ccdb81740 10 client.17286 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:07.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.945+0000 7f7ccdb81740 10 client.17286 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:07.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.945+0000 7f7ccdb81740 20 client.17286 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:07.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.945+0000 7f7ccdb81740 1 --2- 172.21.15.86:0/3082239268 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55798bbb2b70 0x55798bc1b2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:07.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.945+0000 7f7ccdb81740 4 client.17286 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:42:07.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.945+0000 7f7ccdb81740 1 -- 172.21.15.86:0/3082239268 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55798bb05a70 con 0x55798bbb2b70 2024-04-21T08:42:07.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.946+0000 7f7cc3fff640 1 --2- 172.21.15.86:0/3082239268 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55798bbb2b70 0x55798bc1b2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:07.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.946+0000 7f7cc3fff640 1 --2- 172.21.15.86:0/3082239268 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55798bbb2b70 0x55798bc1b2f0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9e6b3b3eaa86d9f server_cookie=74196ccf920961d0 in_seq=0 out_seq=0 2024-04-21T08:42:07.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.946+0000 7f7ca3fff640 10 client.17286 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:07.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.946+0000 7f7ca3fff640 1 -- 172.21.15.86:0/3082239268 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55798bb05a70 con 0x55798bbb2b70 2024-04-21T08:42:07.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.946+0000 7f7ca3fff640 10 client.17286 handle_command_reply: tid=0 2024-04-21T08:42:07.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.946+0000 7f7ccdb81740 1 client.17286 shutdown 2024-04-21T08:42:07.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.946+0000 7f7ccdb81740 1 -- 172.21.15.86:0/3082239268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55798bb99580 msgr2=0x55798bbaad70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:07.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.946+0000 7f7ccdb81740 1 --2- 172.21.15.86:0/3082239268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55798bb99580 0x55798bbaad70 secure :-1 s=READY pgs=1039 cs=0 l=1 rev1=1 crypto rx=0x7f7cb80114a0 tx=0x7f7cb800b950 comp rx=0 tx=0).stop 2024-04-21T08:42:07.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.946+0000 7f7ccdb81740 1 -- 172.21.15.86:0/3082239268 shutdown_connections 2024-04-21T08:42:07.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.946+0000 7f7ccdb81740 1 --2- 172.21.15.86:0/3082239268 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55798bbb2b70 0x55798bc1b2f0 crc :-1 s=READY pgs=134 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.946+0000 7f7ccdb81740 1 --2- 172.21.15.86:0/3082239268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55798bba2a80 0x55798bba67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.946+0000 7f7ca3fff640 0 client.17286 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:07.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.946+0000 7f7ccdb81740 1 --2- 172.21.15.86:0/3082239268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55798bb99ea0 0x55798bba6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.946+0000 7f7ccdb81740 1 --2- 172.21.15.86:0/3082239268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55798bb99580 0x55798bbaad70 unknown :-1 s=CLOSED pgs=1039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.946+0000 7f7ccdb81740 1 -- 172.21.15.86:0/3082239268 >> 172.21.15.86:0/3082239268 conn(0x55798bb93cb0 msgr2=0x55798bb924e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:07.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.947+0000 7f7ccdb81740 1 -- 172.21.15.86:0/3082239268 shutdown_connections 2024-04-21T08:42:07.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.947+0000 7f7ccdb81740 1 -- 172.21.15.86:0/3082239268 wait complete. 2024-04-21T08:42:07.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.947+0000 7f7ccdb81740 20 client.17286 trim_cache size 0 max 16384 2024-04-21T08:42:07.954 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:42:07.954 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:42:07.954 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-21T08:42:07.954 INFO:teuthology.orchestra.run.smithi086.stdout: "c444f00b-2808-409a-adbc-a43363093035": { 2024-04-21T08:42:07.954 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-21T08:42:07.954 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "c444f00b-2808-409a-adbc-a43363093035", 2024-04-21T08:42:07.954 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-21T08:42:07.955 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:42:07.955 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:42:07.955 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:42:07.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.947+0000 7f7ccaec0640 1 -- 172.21.15.86:0/1195524866 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7cb40a3d20 msgr2=0x7f7cb40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:07.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.947+0000 7f7ccaec0640 1 --2- 172.21.15.86:0/1195524866 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7cb40a3d20 0x7f7cb40a61e0 secure :-1 s=READY pgs=158 cs=0 l=1 rev1=1 crypto rx=0x7f7cbc007ea0 tx=0x7f7cbc00c690 comp rx=0 tx=0).stop 2024-04-21T08:42:07.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.947+0000 7f7ccaec0640 1 -- 172.21.15.86:0/1195524866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cc4103b10 msgr2=0x7f7cc4101380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:07.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.947+0000 7f7ccaec0640 1 --2- 172.21.15.86:0/1195524866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cc4103b10 0x7f7cc4101380 secure :-1 s=READY pgs=1035 cs=0 l=1 rev1=1 crypto rx=0x7f7cb0005ab0 tx=0x7f7cb002d9d0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.947+0000 7f7ccaec0640 1 -- 172.21.15.86:0/1195524866 shutdown_connections 2024-04-21T08:42:07.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.947+0000 7f7ccaec0640 1 --2- 172.21.15.86:0/1195524866 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7cb40a3d20 0x7f7cb40a61e0 unknown :-1 s=CLOSED pgs=158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.947+0000 7f7ccaec0640 1 --2- 172.21.15.86:0/1195524866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7cc410c530 0x7f7cc41018c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.947+0000 7f7ccaec0640 1 --2- 172.21.15.86:0/1195524866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7cc4103b10 0x7f7cc4101380 unknown :-1 s=CLOSED pgs=1035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.947+0000 7f7ccaec0640 1 --2- 172.21.15.86:0/1195524866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7cc41031f0 0x7f7cc4100e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:07.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.947+0000 7f7ccaec0640 1 -- 172.21.15.86:0/1195524866 >> 172.21.15.86:0/1195524866 conn(0x7f7cc40771d0 msgr2=0x7f7cc40ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:07.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.947+0000 7f7ccaec0640 1 -- 172.21.15.86:0/1195524866 shutdown_connections 2024-04-21T08:42:07.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:07.947+0000 7f7ccaec0640 1 -- 172.21.15.86:0/1195524866 wait complete. 2024-04-21T08:42:07.958 INFO:tasks.cephfs.filesystem:scrub status for tag:c444f00b-2808-409a-adbc-a43363093035 - {'path': '/', 'tag': 'c444f00b-2808-409a-adbc-a43363093035', 'options': 'recursive,force'} 2024-04-21T08:42:08.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:08 smithi086 ceph-mon[27939]: pgmap v103: 129 pgs: 129 active+clean; 3.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 47 KiB/s wr, 30 op/s 2024-04-21T08:42:08.715 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:08 smithi088 ceph-mon[31259]: pgmap v103: 129 pgs: 129 active+clean; 3.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 47 KiB/s wr, 30 op/s 2024-04-21T08:42:08.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:08 smithi179 ceph-mon[31829]: pgmap v103: 129 pgs: 129 active+clean; 3.3 MiB data, 396 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 47 KiB/s wr, 30 op/s 2024-04-21T08:42:09.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:09 smithi086 ceph-mon[27939]: scrub queued for path: / 2024-04-21T08:42:09.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:09 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:42:09.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:09 smithi086 ceph-mon[27939]: scrub summary: active paths [/] 2024-04-21T08:42:09.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:09 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:42:09.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:09 smithi088 ceph-mon[31259]: scrub queued for path: / 2024-04-21T08:42:09.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:09 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:42:09.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:09 smithi088 ceph-mon[31259]: scrub summary: active paths [/] 2024-04-21T08:42:09.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:09 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:42:09.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:09 smithi179 ceph-mon[31829]: scrub queued for path: / 2024-04-21T08:42:09.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:09 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:42:09.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:09 smithi179 ceph-mon[31829]: scrub summary: active paths [/] 2024-04-21T08:42:09.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:09 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:42:10.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:10 smithi086 ceph-mon[27939]: pgmap v104: 129 pgs: 129 active+clean; 3.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 61 KiB/s wr, 36 op/s 2024-04-21T08:42:10.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:10 smithi086 ceph-mon[27939]: scrub summary: idle 2024-04-21T08:42:10.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:10 smithi088 ceph-mon[31259]: pgmap v104: 129 pgs: 129 active+clean; 3.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 61 KiB/s wr, 36 op/s 2024-04-21T08:42:10.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:10 smithi088 ceph-mon[31259]: scrub summary: idle 2024-04-21T08:42:10.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:10 smithi179 ceph-mon[31829]: pgmap v104: 129 pgs: 129 active+clean; 3.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 61 KiB/s wr, 36 op/s 2024-04-21T08:42:10.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:10 smithi179 ceph-mon[31829]: scrub summary: idle 2024-04-21T08:42:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.003+0000 7fea39e81640 1 -- 172.21.15.86:0/843284080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea3406b120 msgr2=0x7fea3410edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.003+0000 7fea39e81640 1 --2- 172.21.15.86:0/843284080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea3406b120 0x7fea3410edf0 secure :-1 s=READY pgs=1040 cs=0 l=1 rev1=1 crypto rx=0x7fea2c009ca0 tx=0x7fea2c02c180 comp rx=0 tx=0).stop 2024-04-21T08:42:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.003+0000 7fea39e81640 1 -- 172.21.15.86:0/843284080 shutdown_connections 2024-04-21T08:42:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.003+0000 7fea39e81640 1 --2- 172.21.15.86:0/843284080 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea3406b120 0x7fea3410edf0 unknown :-1 s=CLOSED pgs=1040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.003+0000 7fea39e81640 1 --2- 172.21.15.86:0/843284080 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea3406a780 0x7fea3406abe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.003+0000 7fea39e81640 1 --2- 172.21.15.86:0/843284080 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea341080e0 0x7fea341084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.003+0000 7fea39e81640 1 -- 172.21.15.86:0/843284080 >> 172.21.15.86:0/843284080 conn(0x7fea34076c50 msgr2=0x7fea34077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:11.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.003+0000 7fea39e81640 1 -- 172.21.15.86:0/843284080 shutdown_connections 2024-04-21T08:42:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.003+0000 7fea39e81640 1 -- 172.21.15.86:0/843284080 wait complete. 2024-04-21T08:42:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.004+0000 7fea39e81640 1 Processor -- start 2024-04-21T08:42:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.004+0000 7fea39e81640 1 -- start start 2024-04-21T08:42:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.004+0000 7fea39e81640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea3406a780 0x7fea341985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.004+0000 7fea39e81640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea3406b120 0x7fea34198b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.004+0000 7fea39e81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea341080e0 0x7fea3419ce90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.004+0000 7fea39e81640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea341121b0 con 0x7fea341080e0 2024-04-21T08:42:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.004+0000 7fea39e81640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fea34112030 con 0x7fea3406a780 2024-04-21T08:42:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.004+0000 7fea39e81640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fea34112330 con 0x7fea3406b120 2024-04-21T08:42:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.004+0000 7fea33fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea341080e0 0x7fea3419ce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.004+0000 7fea33fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea341080e0 0x7fea3419ce90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53798/0 (socket says 172.21.15.86:53798) 2024-04-21T08:42:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.004+0000 7fea33fff640 1 -- 172.21.15.86:0/985977470 learned_addr learned my addr 172.21.15.86:0/985977470 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.005+0000 7fea337fe640 1 --2- 172.21.15.86:0/985977470 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea3406a780 0x7fea341985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:11.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.005+0000 7fea32ffd640 1 --2- 172.21.15.86:0/985977470 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea3406b120 0x7fea34198b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.005+0000 7fea33fff640 1 -- 172.21.15.86:0/985977470 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea3406b120 msgr2=0x7fea34198b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.005+0000 7fea33fff640 1 --2- 172.21.15.86:0/985977470 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea3406b120 0x7fea34198b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.005+0000 7fea33fff640 1 -- 172.21.15.86:0/985977470 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea3406a780 msgr2=0x7fea341985d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.005+0000 7fea33fff640 1 --2- 172.21.15.86:0/985977470 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea3406a780 0x7fea341985d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.005+0000 7fea33fff640 1 -- 172.21.15.86:0/985977470 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fea3419d5c0 con 0x7fea341080e0 2024-04-21T08:42:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.005+0000 7fea33fff640 1 --2- 172.21.15.86:0/985977470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea341080e0 0x7fea3419ce90 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fea2c02c790 tx=0x7fea2c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.005+0000 7fea30ff9640 1 -- 172.21.15.86:0/985977470 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c039040 con 0x7fea341080e0 2024-04-21T08:42:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.005+0000 7fea30ff9640 1 -- 172.21.15.86:0/985977470 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea2c044070 con 0x7fea341080e0 2024-04-21T08:42:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.005+0000 7fea39e81640 1 -- 172.21.15.86:0/985977470 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fea3419d850 con 0x7fea341080e0 2024-04-21T08:42:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.005+0000 7fea30ff9640 1 -- 172.21.15.86:0/985977470 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c032980 con 0x7fea341080e0 2024-04-21T08:42:11.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.005+0000 7fea39e81640 1 -- 172.21.15.86:0/985977470 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fea341a5140 con 0x7fea341080e0 2024-04-21T08:42:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.007+0000 7fea30ff9640 1 -- 172.21.15.86:0/985977470 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fea2c032b20 con 0x7fea341080e0 2024-04-21T08:42:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.008+0000 7fea3ca74740 1 Processor -- start 2024-04-21T08:42:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.008+0000 7fea30ff9640 1 --2- 172.21.15.86:0/985977470 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fea1c0a3dc0 0x7fea1c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.008+0000 7fea3ca74740 1 -- start start 2024-04-21T08:42:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.008+0000 7fea32ffd640 1 --2- 172.21.15.86:0/985977470 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fea1c0a3dc0 0x7fea1c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.008+0000 7fea3ca74740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e802c648d0 0x55e802c63620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.008+0000 7fea3ca74740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e802c63b60 0x55e802c63fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.008+0000 7fea3ca74740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802c6d0e0 0x55e802c6f4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:11.016 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.008+0000 7fea30ff9640 1 -- 172.21.15.86:0/985977470 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fea2c0f3050 con 0x7fea341080e0 2024-04-21T08:42:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.008+0000 7fea3ca74740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e802bc88d0 con 0x55e802c6d0e0 2024-04-21T08:42:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.008+0000 7fea3ca74740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55e802b65e80 con 0x55e802c63b60 2024-04-21T08:42:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.008+0000 7fea3ca74740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e802bc4890 con 0x55e802c648d0 2024-04-21T08:42:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.008+0000 7fea33fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802c6d0e0 0x55e802c6f4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.008+0000 7fea33fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802c6d0e0 0x55e802c6f4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53802/0 (socket says 172.21.15.86:53802) 2024-04-21T08:42:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.008+0000 7fea33fff640 1 -- 172.21.15.86:0/950011517 learned_addr learned my addr 172.21.15.86:0/950011517 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.008+0000 7fea337fe640 1 --2- 172.21.15.86:0/950011517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e802c648d0 0x55e802c63620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.008+0000 7fea32ffd640 1 --2- 172.21.15.86:0/950011517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e802c63b60 0x55e802c63fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.008+0000 7fea33fff640 1 -- 172.21.15.86:0/950011517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e802c648d0 msgr2=0x55e802c63620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.008+0000 7fea33fff640 1 --2- 172.21.15.86:0/950011517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e802c648d0 0x55e802c63620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.008+0000 7fea33fff640 1 -- 172.21.15.86:0/950011517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e802c63b60 msgr2=0x55e802c63fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.008+0000 7fea32ffd640 1 --2- 172.21.15.86:0/985977470 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fea1c0a3dc0 0x7fea1c0a6280 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fea24005fd0 tx=0x7fea24009ec0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:11.017 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.008+0000 7fea33fff640 1 --2- 172.21.15.86:0/950011517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e802c63b60 0x55e802c63fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.008+0000 7fea33fff640 1 -- 172.21.15.86:0/950011517 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e802b87320 con 0x55e802c6d0e0 2024-04-21T08:42:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.009+0000 7fea33fff640 1 --2- 172.21.15.86:0/950011517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802c6d0e0 0x55e802c6f4d0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fea2c0bb760 tx=0x7fea2c0bec20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5212543f1a69f1fe server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.009+0000 7fea19ffb640 1 -- 172.21.15.86:0/950011517 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c0bf490 con 0x55e802c6d0e0 2024-04-21T08:42:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.009+0000 7fea19ffb640 1 -- 172.21.15.86:0/950011517 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea2c0bed90 con 0x55e802c6d0e0 2024-04-21T08:42:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.009+0000 7fea19ffb640 1 -- 172.21.15.86:0/950011517 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2c0bb400 con 0x55e802c6d0e0 2024-04-21T08:42:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.009+0000 7fea3ca74740 1 -- 172.21.15.86:0/950011517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802c6d0e0 msgr2=0x55e802c6f4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.009+0000 7fea3ca74740 1 --2- 172.21.15.86:0/950011517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802c6d0e0 0x55e802c6f4d0 secure :-1 s=READY pgs=1042 cs=0 l=1 rev1=1 crypto rx=0x7fea2c0bb760 tx=0x7fea2c0bec20 comp rx=0 tx=0).stop 2024-04-21T08:42:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.010+0000 7fea3ca74740 1 -- 172.21.15.86:0/950011517 shutdown_connections 2024-04-21T08:42:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.010+0000 7fea3ca74740 1 --2- 172.21.15.86:0/950011517 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802c6d0e0 0x55e802c6f4d0 unknown :-1 s=CLOSED pgs=1042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.010+0000 7fea3ca74740 1 --2- 172.21.15.86:0/950011517 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e802c63b60 0x55e802c63fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.010+0000 7fea3ca74740 1 --2- 172.21.15.86:0/950011517 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e802c648d0 0x55e802c63620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.010+0000 7fea3ca74740 1 -- 172.21.15.86:0/950011517 >> 172.21.15.86:0/950011517 conn(0x55e802c2ac90 msgr2=0x55e802c5e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.010+0000 7fea3ca74740 1 -- 172.21.15.86:0/950011517 shutdown_connections 2024-04-21T08:42:11.018 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.010+0000 7fea3ca74740 1 -- 172.21.15.86:0/950011517 wait complete. 2024-04-21T08:42:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.010+0000 7fea3ca74740 1 -- start start 2024-04-21T08:42:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.010+0000 7fea3ca74740 1 Processor -- start 2024-04-21T08:42:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.010+0000 7fea3ca74740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802c63b60 0x55e802c6ca10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.010+0000 7fea3ca74740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e802c648d0 0x55e802c69da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.010+0000 7fea3ca74740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e802c6d0e0 0x55e802c6a2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.010+0000 7fea3ca74740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e802c6fbd0 con 0x55e802c63b60 2024-04-21T08:42:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.010+0000 7fea3ca74740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55e802b50eb0 con 0x55e802c6d0e0 2024-04-21T08:42:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.010+0000 7fea3ca74740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e802c6fd50 con 0x55e802c648d0 2024-04-21T08:42:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.010+0000 7fea3ca74740 10 client.0 _ll_register_callbacks cb 0x55e802b7db40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.011+0000 7fea337fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802c63b60 0x55e802c6ca10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.011+0000 7fea337fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802c63b60 0x55e802c6ca10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53806/0 (socket says 172.21.15.86:53806) 2024-04-21T08:42:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.011+0000 7fea337fe640 1 -- 172.21.15.86:0/4038486121 learned_addr learned my addr 172.21.15.86:0/4038486121 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:11.019 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.011+0000 7fea32ffd640 1 --2- 172.21.15.86:0/4038486121 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e802c648d0 0x55e802c69da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.011+0000 7fea33fff640 1 --2- 172.21.15.86:0/4038486121 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e802c6d0e0 0x55e802c6a2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.011+0000 7fea337fe640 1 -- 172.21.15.86:0/4038486121 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e802c648d0 msgr2=0x55e802c69da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.011+0000 7fea337fe640 1 --2- 172.21.15.86:0/4038486121 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e802c648d0 0x55e802c69da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.011+0000 7fea337fe640 1 -- 172.21.15.86:0/4038486121 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e802c6d0e0 msgr2=0x55e802c6a2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.011+0000 7fea337fe640 1 --2- 172.21.15.86:0/4038486121 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e802c6d0e0 0x55e802c6a2e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.011+0000 7fea337fe640 1 -- 172.21.15.86:0/4038486121 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e8028db210 con 0x55e802c63b60 2024-04-21T08:42:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.011+0000 7fea33fff640 1 --2- 172.21.15.86:0/4038486121 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e802c6d0e0 0x55e802c6a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.011+0000 7fea337fe640 1 --2- 172.21.15.86:0/4038486121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802c63b60 0x55e802c6ca10 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fea2800bf40 tx=0x7fea280128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a60a75a9596198e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.011+0000 7fea12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.011+0000 7fea12ffd640 1 -- 172.21.15.86:0/4038486121 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2801f060 con 0x55e802c63b60 2024-04-21T08:42:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.011+0000 7fea12ffd640 1 -- 172.21.15.86:0/4038486121 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fea280108a0 con 0x55e802c63b60 2024-04-21T08:42:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.011+0000 7fea3ca74740 1 -- 172.21.15.86:0/4038486121 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea2c0bed90 con 0x55e802c63b60 2024-04-21T08:42:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.011+0000 7fea12ffd640 1 -- 172.21.15.86:0/4038486121 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fea28026050 con 0x55e802c63b60 2024-04-21T08:42:11.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.011+0000 7fea3ca74740 10 client.17298 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:11.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.011+0000 7fea3ca74740 1 -- 172.21.15.86:0/4038486121 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e8029a7620 con 0x55e802c63b60 2024-04-21T08:42:11.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.012+0000 7fea12ffd640 1 -- 172.21.15.86:0/4038486121 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea2c0bed90 con 0x55e802c63b60 2024-04-21T08:42:11.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.012+0000 7fea12ffd640 1 -- 172.21.15.86:0/4038486121 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fea28019030 con 0x55e802c63b60 2024-04-21T08:42:11.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.012+0000 7fea3ca74740 10 client.17298 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:11.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.012+0000 7fea3ca74740 10 client.17298 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:11.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.012+0000 7fea3ca74740 20 client.17298 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:11.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.012+0000 7fea3ca74740 1 --2- 172.21.15.86:0/4038486121 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55e802c7ce30 0x55e802c7f240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:11.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.012+0000 7fea3ca74740 4 client.17298 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:11.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.012+0000 7fea3ca74740 1 -- 172.21.15.86:0/4038486121 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e802bdc880 con 0x55e802c7ce30 2024-04-21T08:42:11.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.012+0000 7fea32ffd640 1 --2- 172.21.15.86:0/4038486121 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55e802c7ce30 0x55e802c7f240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:11.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.013+0000 7fea32ffd640 1 --2- 172.21.15.86:0/4038486121 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55e802c7ce30 0x55e802c7f240 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ce819300ffb1659a server_cookie=33d5b134d57410ca in_seq=0 out_seq=0 2024-04-21T08:42:11.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.013+0000 7fea12ffd640 10 client.17298 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:11.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.013+0000 7fea12ffd640 1 -- 172.21.15.86:0/4038486121 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e802bdc880 con 0x55e802c7ce30 2024-04-21T08:42:11.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.013+0000 7fea12ffd640 10 client.17298 handle_command_reply: tid=0 2024-04-21T08:42:11.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.013+0000 7fea3ca74740 1 client.17298 shutdown 2024-04-21T08:42:11.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.013+0000 7fea3ca74740 1 -- 172.21.15.86:0/4038486121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802c63b60 msgr2=0x55e802c6ca10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:11.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.013+0000 7fea3ca74740 1 --2- 172.21.15.86:0/4038486121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802c63b60 0x55e802c6ca10 secure :-1 s=READY pgs=1043 cs=0 l=1 rev1=1 crypto rx=0x7fea2800bf40 tx=0x7fea280128e0 comp rx=0 tx=0).stop 2024-04-21T08:42:11.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.013+0000 7fea3ca74740 1 -- 172.21.15.86:0/4038486121 shutdown_connections 2024-04-21T08:42:11.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.013+0000 7fea3ca74740 1 --2- 172.21.15.86:0/4038486121 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55e802c7ce30 0x55e802c7f240 crc :-1 s=READY pgs=135 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:11.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.014+0000 7fea3ca74740 1 --2- 172.21.15.86:0/4038486121 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e802c6d0e0 0x55e802c6a2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:11.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.014+0000 7fea12ffd640 0 client.17298 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:11.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.014+0000 7fea3ca74740 1 --2- 172.21.15.86:0/4038486121 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e802c648d0 0x55e802c69da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:11.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.014+0000 7fea3ca74740 1 --2- 172.21.15.86:0/4038486121 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802c63b60 0x55e802c6ca10 unknown :-1 s=CLOSED pgs=1043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:11.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.014+0000 7fea3ca74740 1 -- 172.21.15.86:0/4038486121 >> 172.21.15.86:0/4038486121 conn(0x55e802c2ac90 msgr2=0x55e802c605a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:11.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.014+0000 7fea3ca74740 1 -- 172.21.15.86:0/4038486121 shutdown_connections 2024-04-21T08:42:11.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.014+0000 7fea3ca74740 1 -- 172.21.15.86:0/4038486121 wait complete. 2024-04-21T08:42:11.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.014+0000 7fea3ca74740 20 client.17298 trim_cache size 0 max 16384 2024-04-21T08:42:11.030 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.028+0000 7fea3ca74740 1 Processor -- start 2024-04-21T08:42:11.030 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.029+0000 7fea3ca74740 1 -- start start 2024-04-21T08:42:11.031 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.029+0000 7fea3ca74740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e802c2ac90 0x55e802c78a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:11.031 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.029+0000 7fea3ca74740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802c63b60 0x55e802c59760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:11.031 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.029+0000 7fea3ca74740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e802c648d0 0x55e802c6c4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:11.031 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.029+0000 7fea3ca74740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea341af190 con 0x55e802c63b60 2024-04-21T08:42:11.031 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.029+0000 7fea3ca74740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fea341b0cd0 con 0x55e802c2ac90 2024-04-21T08:42:11.031 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.029+0000 7fea3ca74740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fea341af860 con 0x55e802c648d0 2024-04-21T08:42:11.031 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.029+0000 7fea32ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802c63b60 0x55e802c59760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:11.031 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.029+0000 7fea32ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802c63b60 0x55e802c59760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53814/0 (socket says 172.21.15.86:53814) 2024-04-21T08:42:11.031 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.029+0000 7fea32ffd640 1 -- 172.21.15.86:0/701618126 learned_addr learned my addr 172.21.15.86:0/701618126 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:11.031 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.029+0000 7fea32ffd640 1 -- 172.21.15.86:0/701618126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e802c648d0 msgr2=0x55e802c6c4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:11.031 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.029+0000 7fea32ffd640 1 --2- 172.21.15.86:0/701618126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e802c648d0 0x55e802c6c4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:11.031 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.029+0000 7fea32ffd640 1 -- 172.21.15.86:0/701618126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e802c2ac90 msgr2=0x55e802c78a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:11.031 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.029+0000 7fea32ffd640 1 --2- 172.21.15.86:0/701618126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e802c2ac90 0x55e802c78a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:11.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.029+0000 7fea32ffd640 1 -- 172.21.15.86:0/701618126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e802c59f30 con 0x55e802c63b60 2024-04-21T08:42:11.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.029+0000 7fea32ffd640 1 --2- 172.21.15.86:0/701618126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802c63b60 0x55e802c59760 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x55e802c65030 tx=0x7fea24017000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f3c95fdc84475770 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:11.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.029+0000 7fea197fa640 1 -- 172.21.15.86:0/701618126 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea240285a0 con 0x55e802c63b60 2024-04-21T08:42:11.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.029+0000 7fea197fa640 1 -- 172.21.15.86:0/701618126 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea2402d070 con 0x55e802c63b60 2024-04-21T08:42:11.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.029+0000 7fea197fa640 1 -- 172.21.15.86:0/701618126 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2401e440 con 0x55e802c63b60 2024-04-21T08:42:11.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.030+0000 7fea3ca74740 1 -- 172.21.15.86:0/701618126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802c63b60 msgr2=0x55e802c59760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:11.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.030+0000 7fea3ca74740 1 --2- 172.21.15.86:0/701618126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802c63b60 0x55e802c59760 secure :-1 s=READY pgs=1044 cs=0 l=1 rev1=1 crypto rx=0x55e802c65030 tx=0x7fea24017000 comp rx=0 tx=0).stop 2024-04-21T08:42:11.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.030+0000 7fea3ca74740 1 -- 172.21.15.86:0/701618126 shutdown_connections 2024-04-21T08:42:11.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.030+0000 7fea3ca74740 1 --2- 172.21.15.86:0/701618126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e802c648d0 0x55e802c6c4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:11.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.030+0000 7fea3ca74740 1 --2- 172.21.15.86:0/701618126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802c63b60 0x55e802c59760 unknown :-1 s=CLOSED pgs=1044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:11.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.030+0000 7fea3ca74740 1 --2- 172.21.15.86:0/701618126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e802c2ac90 0x55e802c78a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:11.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.030+0000 7fea3ca74740 1 -- 172.21.15.86:0/701618126 >> 172.21.15.86:0/701618126 conn(0x55e802c6a440 msgr2=0x55e802c68670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:11.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.030+0000 7fea3ca74740 1 -- 172.21.15.86:0/701618126 shutdown_connections 2024-04-21T08:42:11.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.030+0000 7fea3ca74740 1 -- 172.21.15.86:0/701618126 wait complete. 2024-04-21T08:42:11.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.030+0000 7fea3ca74740 1 -- start start 2024-04-21T08:42:11.032 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.031+0000 7fea3ca74740 1 Processor -- start 2024-04-21T08:42:11.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.031+0000 7fea3ca74740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802c2ac90 0x55e802c75430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:11.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.031+0000 7fea3ca74740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e802c63b60 0x55e802c66880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.031+0000 7fea3ca74740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e802c648d0 0x55e802c66e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.031+0000 7fea3ca74740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fea341b0600 con 0x55e802c2ac90 2024-04-21T08:42:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.031+0000 7fea3ca74740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fea341aff30 con 0x55e802c648d0 2024-04-21T08:42:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.031+0000 7fea3ca74740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fea341ad150 con 0x55e802c63b60 2024-04-21T08:42:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.031+0000 7fea3ca74740 10 client.0 _ll_register_callbacks cb 0x7fea2800bb50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.031+0000 7fea337fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802c2ac90 0x55e802c75430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.031+0000 7fea337fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802c2ac90 0x55e802c75430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:53818/0 (socket says 172.21.15.86:53818) 2024-04-21T08:42:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.031+0000 7fea337fe640 1 -- 172.21.15.86:0/3528089678 learned_addr learned my addr 172.21.15.86:0/3528089678 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.031+0000 7fea32ffd640 1 --2- 172.21.15.86:0/3528089678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e802c63b60 0x55e802c66880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.031+0000 7fea33fff640 1 --2- 172.21.15.86:0/3528089678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e802c648d0 0x55e802c66e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.031+0000 7fea337fe640 1 -- 172.21.15.86:0/3528089678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e802c63b60 msgr2=0x55e802c66880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:11.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.031+0000 7fea337fe640 1 --2- 172.21.15.86:0/3528089678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e802c63b60 0x55e802c66880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.031+0000 7fea337fe640 1 -- 172.21.15.86:0/3528089678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e802c648d0 msgr2=0x55e802c66e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.031+0000 7fea337fe640 1 --2- 172.21.15.86:0/3528089678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e802c648d0 0x55e802c66e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.031+0000 7fea337fe640 1 -- 172.21.15.86:0/3528089678 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e802c676c0 con 0x55e802c2ac90 2024-04-21T08:42:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.031+0000 7fea337fe640 1 --2- 172.21.15.86:0/3528089678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802c2ac90 0x55e802c75430 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fea2800bc60 tx=0x7fea28005970 comp rx=0 tx=0).ready entity=mon.0 client_cookie=400a36703cd054a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.031+0000 7fea12ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.032+0000 7fea12ffd640 1 -- 172.21.15.86:0/3528089678 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fea2802f060 con 0x55e802c2ac90 2024-04-21T08:42:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.032+0000 7fea12ffd640 1 -- 172.21.15.86:0/3528089678 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fea28005510 con 0x55e802c2ac90 2024-04-21T08:42:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.032+0000 7fea3ca74740 1 -- 172.21.15.86:0/3528089678 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fea2402d070 con 0x55e802c2ac90 2024-04-21T08:42:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.032+0000 7fea12ffd640 1 -- 172.21.15.86:0/3528089678 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fea2801f020 con 0x55e802c2ac90 2024-04-21T08:42:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.032+0000 7fea3ca74740 10 client.17304 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.032+0000 7fea3ca74740 1 -- 172.21.15.86:0/3528089678 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e802ce3530 con 0x55e802c2ac90 2024-04-21T08:42:11.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.032+0000 7fea12ffd640 1 -- 172.21.15.86:0/3528089678 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fea2402d070 con 0x55e802c2ac90 2024-04-21T08:42:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.032+0000 7fea12ffd640 1 -- 172.21.15.86:0/3528089678 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fea28019070 con 0x55e802c2ac90 2024-04-21T08:42:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.032+0000 7fea3ca74740 10 client.17304 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.032+0000 7fea3ca74740 10 client.17304 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.032+0000 7fea3ca74740 20 client.17304 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.032+0000 7fea3ca74740 1 --2- 172.21.15.86:0/3528089678 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55e802c6d0e0 0x55e802ce5d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.032+0000 7fea3ca74740 4 client.17304 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:42:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.032+0000 7fea3ca74740 1 -- 172.21.15.86:0/3528089678 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55e802b683d0 con 0x55e802c6d0e0 2024-04-21T08:42:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.032+0000 7fea32ffd640 1 --2- 172.21.15.86:0/3528089678 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55e802c6d0e0 0x55e802ce5d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.033+0000 7fea12ffd640 10 client.17304 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:11.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.033+0000 7fea32ffd640 1 --2- 172.21.15.86:0/3528089678 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55e802c6d0e0 0x55e802ce5d20 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=34e0f99668c0e28b server_cookie=17f8253068c2ebc2 in_seq=0 out_seq=0 2024-04-21T08:42:11.047 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.046+0000 7fea12ffd640 1 -- 172.21.15.86:0/3528089678 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55e802b683d0 con 0x55e802c6d0e0 2024-04-21T08:42:11.047 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.046+0000 7fea12ffd640 10 client.17304 handle_command_reply: tid=0 2024-04-21T08:42:11.047 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.046+0000 7fea3ca74740 1 client.17304 shutdown 2024-04-21T08:42:11.047 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.046+0000 7fea3ca74740 1 -- 172.21.15.86:0/3528089678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802c2ac90 msgr2=0x55e802c75430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:11.047 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.046+0000 7fea3ca74740 1 --2- 172.21.15.86:0/3528089678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802c2ac90 0x55e802c75430 secure :-1 s=READY pgs=1045 cs=0 l=1 rev1=1 crypto rx=0x7fea2800bc60 tx=0x7fea28005970 comp rx=0 tx=0).stop 2024-04-21T08:42:11.047 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.047+0000 7fea3ca74740 1 -- 172.21.15.86:0/3528089678 shutdown_connections 2024-04-21T08:42:11.048 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.047+0000 7fea3ca74740 1 --2- 172.21.15.86:0/3528089678 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55e802c6d0e0 0x55e802ce5d20 crc :-1 s=READY pgs=136 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:11.048 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.047+0000 7fea3ca74740 1 --2- 172.21.15.86:0/3528089678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e802c648d0 0x55e802c66e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:11.048 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.047+0000 7fea3ca74740 1 --2- 172.21.15.86:0/3528089678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e802c63b60 0x55e802c66880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:11.048 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.047+0000 7fea3ca74740 1 --2- 172.21.15.86:0/3528089678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e802c2ac90 0x55e802c75430 unknown :-1 s=CLOSED pgs=1045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:11.048 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.047+0000 7fea12ffd640 0 client.17304 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:11.048 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.047+0000 7fea3ca74740 1 -- 172.21.15.86:0/3528089678 >> 172.21.15.86:0/3528089678 conn(0x55e802c6a440 msgr2=0x55e802c5d040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:11.048 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.047+0000 7fea3ca74740 1 -- 172.21.15.86:0/3528089678 shutdown_connections 2024-04-21T08:42:11.048 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.047+0000 7fea3ca74740 1 -- 172.21.15.86:0/3528089678 wait complete. 2024-04-21T08:42:11.048 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.047+0000 7fea3ca74740 20 client.17304 trim_cache size 0 max 16384 2024-04-21T08:42:11.048 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:42:11.049 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:42:11.049 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:42:11.049 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:42:11.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.047+0000 7fea39e81640 1 -- 172.21.15.86:0/985977470 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fea1c0a3dc0 msgr2=0x7fea1c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:11.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.047+0000 7fea39e81640 1 --2- 172.21.15.86:0/985977470 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fea1c0a3dc0 0x7fea1c0a6280 secure :-1 s=READY pgs=159 cs=0 l=1 rev1=1 crypto rx=0x7fea24005fd0 tx=0x7fea24009ec0 comp rx=0 tx=0).stop 2024-04-21T08:42:11.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.047+0000 7fea39e81640 1 -- 172.21.15.86:0/985977470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea341080e0 msgr2=0x7fea3419ce90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:11.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.047+0000 7fea39e81640 1 --2- 172.21.15.86:0/985977470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea341080e0 0x7fea3419ce90 secure :-1 s=READY pgs=1041 cs=0 l=1 rev1=1 crypto rx=0x7fea2c02c790 tx=0x7fea2c003c20 comp rx=0 tx=0).stop 2024-04-21T08:42:11.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.047+0000 7fea39e81640 1 -- 172.21.15.86:0/985977470 shutdown_connections 2024-04-21T08:42:11.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.047+0000 7fea39e81640 1 --2- 172.21.15.86:0/985977470 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fea1c0a3dc0 0x7fea1c0a6280 unknown :-1 s=CLOSED pgs=159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:11.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.047+0000 7fea39e81640 1 --2- 172.21.15.86:0/985977470 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fea341080e0 0x7fea3419ce90 unknown :-1 s=CLOSED pgs=1041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:11.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.047+0000 7fea39e81640 1 --2- 172.21.15.86:0/985977470 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fea3406b120 0x7fea34198b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:11.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.047+0000 7fea39e81640 1 --2- 172.21.15.86:0/985977470 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fea3406a780 0x7fea341985d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:11.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.047+0000 7fea39e81640 1 -- 172.21.15.86:0/985977470 >> 172.21.15.86:0/985977470 conn(0x7fea34076c50 msgr2=0x7fea340ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:11.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.047+0000 7fea39e81640 1 -- 172.21.15.86:0/985977470 shutdown_connections 2024-04-21T08:42:11.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:11.047+0000 7fea39e81640 1 -- 172.21.15.86:0/985977470 wait complete. 2024-04-21T08:42:12.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:12 smithi086 ceph-mon[27939]: pgmap v105: 129 pgs: 129 active+clean; 3.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 51 KiB/s wr, 30 op/s 2024-04-21T08:42:12.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:12 smithi088 ceph-mon[31259]: pgmap v105: 129 pgs: 129 active+clean; 3.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 51 KiB/s wr, 30 op/s 2024-04-21T08:42:12.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:12 smithi179 ceph-mon[31829]: pgmap v105: 129 pgs: 129 active+clean; 3.5 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 51 KiB/s wr, 30 op/s 2024-04-21T08:42:14.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:14 smithi088 ceph-mon[31259]: pgmap v106: 129 pgs: 129 active+clean; 3.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 68 KiB/s wr, 41 op/s 2024-04-21T08:42:14.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:14 smithi179 ceph-mon[31829]: pgmap v106: 129 pgs: 129 active+clean; 3.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 68 KiB/s wr, 41 op/s 2024-04-21T08:42:14.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:14 smithi086 ceph-mon[27939]: pgmap v106: 129 pgs: 129 active+clean; 3.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 68 KiB/s wr, 41 op/s 2024-04-21T08:42:16.668 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:16 smithi086 ceph-mon[27939]: pgmap v107: 129 pgs: 129 active+clean; 3.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 KiB/s wr, 17 op/s 2024-04-21T08:42:16.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:16 smithi088 ceph-mon[31259]: pgmap v107: 129 pgs: 129 active+clean; 3.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 KiB/s wr, 17 op/s 2024-04-21T08:42:16.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:16 smithi179 ceph-mon[31829]: pgmap v107: 129 pgs: 129 active+clean; 3.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 KiB/s wr, 17 op/s 2024-04-21T08:42:18.716 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:18 smithi088 ceph-mon[31259]: pgmap v108: 129 pgs: 129 active+clean; 3.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 KiB/s wr, 17 op/s 2024-04-21T08:42:18.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:18 smithi179 ceph-mon[31829]: pgmap v108: 129 pgs: 129 active+clean; 3.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 KiB/s wr, 17 op/s 2024-04-21T08:42:18.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:18 smithi086 ceph-mon[27939]: pgmap v108: 129 pgs: 129 active+clean; 3.6 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 33 KiB/s rd, 31 KiB/s wr, 17 op/s 2024-04-21T08:42:20.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:20 smithi088 ceph-mon[31259]: pgmap v109: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 41 KiB/s wr, 23 op/s 2024-04-21T08:42:20.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:20 smithi179 ceph-mon[31829]: pgmap v109: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 41 KiB/s wr, 23 op/s 2024-04-21T08:42:20.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:20 smithi086 ceph-mon[27939]: pgmap v109: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 47 KiB/s rd, 41 KiB/s wr, 23 op/s 2024-04-21T08:42:22.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:22 smithi088 ceph-mon[31259]: pgmap v110: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 16 op/s 2024-04-21T08:42:22.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:22 smithi179 ceph-mon[31829]: pgmap v110: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 16 op/s 2024-04-21T08:42:22.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:22 smithi086 ceph-mon[27939]: pgmap v110: 129 pgs: 129 active+clean; 3.8 MiB data, 397 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 27 KiB/s wr, 16 op/s 2024-04-21T08:42:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.143+0000 7fa0c1a74640 1 -- 172.21.15.86:0/3505743007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0bc1021c0 msgr2=0x7fa0bc1025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.143+0000 7fa0c1a74640 1 --2- 172.21.15.86:0/3505743007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0bc1021c0 0x7fa0bc1025a0 secure :-1 s=READY pgs=1046 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac009900 tx=0x7fa0ac02bc30 comp rx=0 tx=0).stop 2024-04-21T08:42:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.143+0000 7fa0c1a74640 1 -- 172.21.15.86:0/3505743007 shutdown_connections 2024-04-21T08:42:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.143+0000 7fa0c1a74640 1 --2- 172.21.15.86:0/3505743007 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0bc107290 0x7fa0bc10ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.143+0000 7fa0c1a74640 1 --2- 172.21.15.86:0/3505743007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0bc102b70 0x7fa0bc106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:24.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.143+0000 7fa0c1a74640 1 --2- 172.21.15.86:0/3505743007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0bc1021c0 0x7fa0bc1025a0 unknown :-1 s=CLOSED pgs=1046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.143+0000 7fa0c1a74640 1 -- 172.21.15.86:0/3505743007 >> 172.21.15.86:0/3505743007 conn(0x7fa0bc0fda20 msgr2=0x7fa0bc0ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.144+0000 7fa0c1a74640 1 -- 172.21.15.86:0/3505743007 shutdown_connections 2024-04-21T08:42:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.144+0000 7fa0c1a74640 1 -- 172.21.15.86:0/3505743007 wait complete. 2024-04-21T08:42:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.144+0000 7fa0c1a74640 1 Processor -- start 2024-04-21T08:42:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.144+0000 7fa0c1a74640 1 -- start start 2024-04-21T08:42:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.145+0000 7fa0c1a74640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0bc1021c0 0x7fa0bc074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.145+0000 7fa0c1a74640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0bc107290 0x7fa0bc074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.145+0000 7fa0c1a74640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0bc074cd0 0x7fa0bc1a6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.145+0000 7fa0c1a74640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0bc1121d0 con 0x7fa0bc107290 2024-04-21T08:42:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.145+0000 7fa0c1a74640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa0bc112050 con 0x7fa0bc1021c0 2024-04-21T08:42:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.145+0000 7fa0c1a74640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa0bc112350 con 0x7fa0bc074cd0 2024-04-21T08:42:24.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.145+0000 7fa0baffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0bc107290 0x7fa0bc074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.145+0000 7fa0baffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0bc107290 0x7fa0bc074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56202/0 (socket says 172.21.15.86:56202) 2024-04-21T08:42:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.145+0000 7fa0baffd640 1 -- 172.21.15.86:0/4133718043 learned_addr learned my addr 172.21.15.86:0/4133718043 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.145+0000 7fa0bbfff640 1 --2- 172.21.15.86:0/4133718043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0bc074cd0 0x7fa0bc1a6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.145+0000 7fa0bb7fe640 1 --2- 172.21.15.86:0/4133718043 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0bc1021c0 0x7fa0bc074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.145+0000 7fa0baffd640 1 -- 172.21.15.86:0/4133718043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0bc074cd0 msgr2=0x7fa0bc1a6900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.145+0000 7fa0baffd640 1 --2- 172.21.15.86:0/4133718043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0bc074cd0 0x7fa0bc1a6900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.145+0000 7fa0baffd640 1 -- 172.21.15.86:0/4133718043 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0bc1021c0 msgr2=0x7fa0bc074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.145+0000 7fa0baffd640 1 --2- 172.21.15.86:0/4133718043 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0bc1021c0 0x7fa0bc074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.145+0000 7fa0baffd640 1 -- 172.21.15.86:0/4133718043 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0bc1a6fa0 con 0x7fa0bc107290 2024-04-21T08:42:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.145+0000 7fa0bb7fe640 1 --2- 172.21.15.86:0/4133718043 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0bc1021c0 0x7fa0bc074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.145+0000 7fa0baffd640 1 --2- 172.21.15.86:0/4133718043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0bc107290 0x7fa0bc074790 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fa0b000d770 tx=0x7fa0b000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.145+0000 7fa0b8ff9640 1 -- 172.21.15.86:0/4133718043 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0017070 con 0x7fa0bc107290 2024-04-21T08:42:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.145+0000 7fa0b8ff9640 1 -- 172.21.15.86:0/4133718043 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0b0009c30 con 0x7fa0bc107290 2024-04-21T08:42:24.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.145+0000 7fa0c1a74640 1 -- 172.21.15.86:0/4133718043 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0bc1a7290 con 0x7fa0bc107290 2024-04-21T08:42:24.148 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.145+0000 7fa0b8ff9640 1 -- 172.21.15.86:0/4133718043 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b000f620 con 0x7fa0bc107290 2024-04-21T08:42:24.148 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.145+0000 7fa0c1a74640 1 -- 172.21.15.86:0/4133718043 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0bc1a77e0 con 0x7fa0bc107290 2024-04-21T08:42:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.148+0000 7fa0b8ff9640 1 -- 172.21.15.86:0/4133718043 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa0b001a070 con 0x7fa0bc107290 2024-04-21T08:42:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.148+0000 7fa0c466d740 1 Processor -- start 2024-04-21T08:42:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.148+0000 7fa0c466d740 1 -- start start 2024-04-21T08:42:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.148+0000 7fa0c466d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d9605688d0 0x55d960567620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.148+0000 7fa0b8ff9640 1 --2- 172.21.15.86:0/4133718043 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa0a40a3dc0 0x7fa0a40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.148+0000 7fa0c466d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d960567b60 0x55d960567fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.148+0000 7fa0c466d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9605710e0 0x55d9605734d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.148+0000 7fa0c466d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d9604cc8d0 con 0x55d960567b60 2024-04-21T08:42:24.154 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.148+0000 7fa0c466d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d960469e80 con 0x55d9605688d0 2024-04-21T08:42:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.148+0000 7fa0c466d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d9604c8890 con 0x55d9605710e0 2024-04-21T08:42:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.148+0000 7fa0baffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d960567b60 0x55d960567fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.148+0000 7fa0bb7fe640 1 --2- 172.21.15.86:0/4133718043 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa0a40a3dc0 0x7fa0a40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.148+0000 7fa0bbfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d9605688d0 0x55d960567620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.148+0000 7fa0baffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d960567b60 0x55d960567fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56208/0 (socket says 172.21.15.86:56208) 2024-04-21T08:42:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.148+0000 7fa0bbfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d9605688d0 0x55d960567620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.86:52804/0 (socket says 172.21.15.86:52804) 2024-04-21T08:42:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.148+0000 7fa0baffd640 1 -- 172.21.15.86:0/389416485 learned_addr learned my addr 172.21.15.86:0/389416485 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.149+0000 7fa0b8ff9640 1 -- 172.21.15.86:0/4133718043 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fa0b00d0790 con 0x7fa0bc107290 2024-04-21T08:42:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.149+0000 7fa0bbfff640 1 --2- 172.21.15.86:0/389416485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9605710e0 0x55d9605734d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.149+0000 7fa0bbfff640 1 -- 172.21.15.86:0/389416485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9605710e0 msgr2=0x55d9605734d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.149+0000 7fa0bbfff640 1 --2- 172.21.15.86:0/389416485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9605710e0 0x55d9605734d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.149+0000 7fa0bbfff640 1 -- 172.21.15.86:0/389416485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d960567b60 msgr2=0x55d960567fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:24.155 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.149+0000 7fa0bbfff640 1 --2- 172.21.15.86:0/389416485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d960567b60 0x55d960567fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.149+0000 7fa0bbfff640 1 -- 172.21.15.86:0/389416485 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d96048b320 con 0x55d9605688d0 2024-04-21T08:42:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.149+0000 7fa0baffd640 1 --2- 172.21.15.86:0/389416485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d960567b60 0x55d960567fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:42:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.149+0000 7fa0bb7fe640 1 --2- 172.21.15.86:0/4133718043 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa0a40a3dc0 0x7fa0a40a6280 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac036040 tx=0x7fa0ac000bc0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.149+0000 7fa0bbfff640 1 --2- 172.21.15.86:0/389416485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d9605688d0 0x55d960567620 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa0b40125d0 tx=0x7fa0b4007700 comp rx=0 tx=0).ready entity=mon.2 client_cookie=54aee64ffa6ae096 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.149+0000 7fa0a1ffb640 1 -- 172.21.15.86:0/389416485 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4003b80 con 0x55d9605688d0 2024-04-21T08:42:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.149+0000 7fa0a1ffb640 1 -- 172.21.15.86:0/389416485 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0b4005020 con 0x55d9605688d0 2024-04-21T08:42:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.149+0000 7fa0a1ffb640 1 -- 172.21.15.86:0/389416485 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b4010710 con 0x55d9605688d0 2024-04-21T08:42:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.150+0000 7fa0c466d740 1 -- 172.21.15.86:0/389416485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d9605688d0 msgr2=0x55d960567620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.150+0000 7fa0c466d740 1 --2- 172.21.15.86:0/389416485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d9605688d0 0x55d960567620 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fa0b40125d0 tx=0x7fa0b4007700 comp rx=0 tx=0).stop 2024-04-21T08:42:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.150+0000 7fa0c466d740 1 -- 172.21.15.86:0/389416485 shutdown_connections 2024-04-21T08:42:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.150+0000 7fa0c466d740 1 --2- 172.21.15.86:0/389416485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9605710e0 0x55d9605734d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.150+0000 7fa0c466d740 1 --2- 172.21.15.86:0/389416485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d960567b60 0x55d960567fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.150+0000 7fa0c466d740 1 --2- 172.21.15.86:0/389416485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d9605688d0 0x55d960567620 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:24.156 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.150+0000 7fa0c466d740 1 -- 172.21.15.86:0/389416485 >> 172.21.15.86:0/389416485 conn(0x55d96052ec90 msgr2=0x55d960562850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.150+0000 7fa0c466d740 1 -- 172.21.15.86:0/389416485 shutdown_connections 2024-04-21T08:42:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.150+0000 7fa0c466d740 1 -- 172.21.15.86:0/389416485 wait complete. 2024-04-21T08:42:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.150+0000 7fa0c466d740 1 -- start start 2024-04-21T08:42:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.151+0000 7fa0c466d740 1 Processor -- start 2024-04-21T08:42:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.151+0000 7fa0c466d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d960567b60 0x55d960570a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.151+0000 7fa0c466d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9605688d0 0x55d96056dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.151+0000 7fa0c466d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9605710e0 0x55d96056e340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.151+0000 7fa0c466d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d960573e10 con 0x55d9605688d0 2024-04-21T08:42:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.151+0000 7fa0c466d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d960454eb0 con 0x55d960567b60 2024-04-21T08:42:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.151+0000 7fa0c466d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d960573f90 con 0x55d9605710e0 2024-04-21T08:42:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.151+0000 7fa0c466d740 10 client.0 _ll_register_callbacks cb 0x55d960481b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:24.157 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.151+0000 7fa0baffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9605688d0 0x55d96056dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.151+0000 7fa0baffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9605688d0 0x55d96056dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56210/0 (socket says 172.21.15.86:56210) 2024-04-21T08:42:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.151+0000 7fa0baffd640 1 -- 172.21.15.86:0/81621159 learned_addr learned my addr 172.21.15.86:0/81621159 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.151+0000 7fa0bb7fe640 1 --2- 172.21.15.86:0/81621159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d960567b60 0x55d960570a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.151+0000 7fa0bbfff640 1 --2- 172.21.15.86:0/81621159 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9605710e0 0x55d96056e340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.151+0000 7fa0baffd640 1 -- 172.21.15.86:0/81621159 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9605710e0 msgr2=0x55d96056e340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.151+0000 7fa0baffd640 1 --2- 172.21.15.86:0/81621159 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9605710e0 0x55d96056e340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.151+0000 7fa0baffd640 1 -- 172.21.15.86:0/81621159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d960567b60 msgr2=0x55d960570a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.151+0000 7fa0baffd640 1 --2- 172.21.15.86:0/81621159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d960567b60 0x55d960570a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.151+0000 7fa0baffd640 1 -- 172.21.15.86:0/81621159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9601df210 con 0x55d9605688d0 2024-04-21T08:42:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.151+0000 7fa0baffd640 1 --2- 172.21.15.86:0/81621159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9605688d0 0x55d96056dde0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0098820 tx=0x7fa0b006bca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=861010ebf94ffd47 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.151+0000 7fa09affd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.152+0000 7fa09affd640 1 -- 172.21.15.86:0/81621159 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0009de0 con 0x55d9605688d0 2024-04-21T08:42:24.158 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.152+0000 7fa09affd640 1 -- 172.21.15.86:0/81621159 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fa0b009aef0 con 0x55d9605688d0 2024-04-21T08:42:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.152+0000 7fa0c466d740 1 -- 172.21.15.86:0/81621159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0b4005020 con 0x55d9605688d0 2024-04-21T08:42:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.152+0000 7fa09affd640 1 -- 172.21.15.86:0/81621159 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa0b00b0e60 con 0x55d9605688d0 2024-04-21T08:42:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.152+0000 7fa0c466d740 10 client.17316 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.152+0000 7fa0c466d740 1 -- 172.21.15.86:0/81621159 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d9602ab620 con 0x55d9605688d0 2024-04-21T08:42:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.152+0000 7fa09affd640 1 -- 172.21.15.86:0/81621159 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0b4005020 con 0x55d9605688d0 2024-04-21T08:42:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.152+0000 7fa09affd640 1 -- 172.21.15.86:0/81621159 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa0b009a980 con 0x55d9605688d0 2024-04-21T08:42:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.152+0000 7fa0c466d740 10 client.17316 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.152+0000 7fa0c466d740 10 client.17316 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.152+0000 7fa0c466d740 20 client.17316 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.152+0000 7fa0c466d740 1 --2- 172.21.15.86:0/81621159 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d9605810e0 0x55d9605834d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.152+0000 7fa0c466d740 4 client.17316 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.152+0000 7fa0c466d740 1 -- 172.21.15.86:0/81621159 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9604e0880 con 0x55d9605810e0 2024-04-21T08:42:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.152+0000 7fa0bb7fe640 1 --2- 172.21.15.86:0/81621159 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d9605810e0 0x55d9605834d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:24.159 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.153+0000 7fa0bb7fe640 1 --2- 172.21.15.86:0/81621159 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d9605810e0 0x55d9605834d0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2184768bf8f44c80 server_cookie=d277f38a384fadea in_seq=0 out_seq=0 2024-04-21T08:42:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.153+0000 7fa09affd640 10 client.17316 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.154+0000 7fa09affd640 1 -- 172.21.15.86:0/81621159 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d9604e0880 con 0x55d9605810e0 2024-04-21T08:42:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.154+0000 7fa09affd640 10 client.17316 handle_command_reply: tid=0 2024-04-21T08:42:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.154+0000 7fa0c466d740 1 client.17316 shutdown 2024-04-21T08:42:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.154+0000 7fa0c466d740 1 -- 172.21.15.86:0/81621159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9605688d0 msgr2=0x55d96056dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.154+0000 7fa0c466d740 1 --2- 172.21.15.86:0/81621159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9605688d0 0x55d96056dde0 secure :-1 s=READY pgs=1048 cs=0 l=1 rev1=1 crypto rx=0x7fa0b0098820 tx=0x7fa0b006bca0 comp rx=0 tx=0).stop 2024-04-21T08:42:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.154+0000 7fa0c466d740 1 -- 172.21.15.86:0/81621159 shutdown_connections 2024-04-21T08:42:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.154+0000 7fa0c466d740 1 --2- 172.21.15.86:0/81621159 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d9605810e0 0x55d9605834d0 crc :-1 s=READY pgs=137 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.154+0000 7fa0c466d740 1 --2- 172.21.15.86:0/81621159 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9605710e0 0x55d96056e340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.154+0000 7fa0c466d740 1 --2- 172.21.15.86:0/81621159 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9605688d0 0x55d96056dde0 unknown :-1 s=CLOSED pgs=1048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.154+0000 7fa09affd640 0 client.17316 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.154+0000 7fa0c466d740 1 --2- 172.21.15.86:0/81621159 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d960567b60 0x55d960570a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.154+0000 7fa0c466d740 1 -- 172.21.15.86:0/81621159 >> 172.21.15.86:0/81621159 conn(0x55d96052ec90 msgr2=0x55d9605645a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:24.160 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.154+0000 7fa0c466d740 1 -- 172.21.15.86:0/81621159 shutdown_connections 2024-04-21T08:42:24.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.154+0000 7fa0c466d740 1 -- 172.21.15.86:0/81621159 wait complete. 2024-04-21T08:42:24.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.154+0000 7fa0c466d740 20 client.17316 trim_cache size 0 max 16384 2024-04-21T08:42:24.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.169+0000 7fa0c466d740 1 Processor -- start 2024-04-21T08:42:24.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.169+0000 7fa0c466d740 1 -- start start 2024-04-21T08:42:24.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.169+0000 7fa0c466d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d960567b60 0x55d96057cd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:24.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.169+0000 7fa0c466d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9605688d0 0x55d960570510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:24.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.169+0000 7fa0c466d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d9605710e0 0x55d960570a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:24.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.169+0000 7fa0c466d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0bc1b1440 con 0x55d960567b60 2024-04-21T08:42:24.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.169+0000 7fa0c466d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa0bc1b2f80 con 0x55d9605710e0 2024-04-21T08:42:24.169 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.169+0000 7fa0c466d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa0bc1b1b10 con 0x55d9605688d0 2024-04-21T08:42:24.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.169+0000 7fa0bb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d960567b60 0x55d96057cd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:24.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.169+0000 7fa0bb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d960567b60 0x55d96057cd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56226/0 (socket says 172.21.15.86:56226) 2024-04-21T08:42:24.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.169+0000 7fa0bb7fe640 1 -- 172.21.15.86:0/3506661254 learned_addr learned my addr 172.21.15.86:0/3506661254 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:24.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.169+0000 7fa0baffd640 1 --2- 172.21.15.86:0/3506661254 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9605688d0 0x55d960570510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:24.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.169+0000 7fa0bbfff640 1 --2- 172.21.15.86:0/3506661254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d9605710e0 0x55d960570a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:24.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.169+0000 7fa0bb7fe640 1 -- 172.21.15.86:0/3506661254 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9605688d0 msgr2=0x55d960570510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:24.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.169+0000 7fa0bb7fe640 1 --2- 172.21.15.86:0/3506661254 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9605688d0 0x55d960570510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:24.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.169+0000 7fa0bb7fe640 1 -- 172.21.15.86:0/3506661254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d9605710e0 msgr2=0x55d960570a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:24.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.169+0000 7fa0bb7fe640 1 --2- 172.21.15.86:0/3506661254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d9605710e0 0x55d960570a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:24.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.169+0000 7fa0bb7fe640 1 -- 172.21.15.86:0/3506661254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d960583b50 con 0x55d960567b60 2024-04-21T08:42:24.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.170+0000 7fa0bb7fe640 1 --2- 172.21.15.86:0/3506661254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d960567b60 0x55d96057cd40 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac009900 tx=0x7fa0ac03c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bf080921d501daf8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:24.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.170+0000 7fa0a17fa640 1 -- 172.21.15.86:0/3506661254 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac055070 con 0x55d960567b60 2024-04-21T08:42:24.170 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.170+0000 7fa0a17fa640 1 -- 172.21.15.86:0/3506661254 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0ac0054f0 con 0x55d960567b60 2024-04-21T08:42:24.171 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.170+0000 7fa0a17fa640 1 -- 172.21.15.86:0/3506661254 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0ac046040 con 0x55d960567b60 2024-04-21T08:42:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.170+0000 7fa0c466d740 1 -- 172.21.15.86:0/3506661254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d960567b60 msgr2=0x55d96057cd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:24.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.170+0000 7fa0c466d740 1 --2- 172.21.15.86:0/3506661254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d960567b60 0x55d96057cd40 secure :-1 s=READY pgs=1049 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac009900 tx=0x7fa0ac03c000 comp rx=0 tx=0).stop 2024-04-21T08:42:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.171+0000 7fa0c466d740 1 -- 172.21.15.86:0/3506661254 shutdown_connections 2024-04-21T08:42:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.171+0000 7fa0c466d740 1 --2- 172.21.15.86:0/3506661254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d9605710e0 0x55d960570a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.171+0000 7fa0c466d740 1 --2- 172.21.15.86:0/3506661254 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9605688d0 0x55d960570510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.171+0000 7fa0c466d740 1 --2- 172.21.15.86:0/3506661254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d960567b60 0x55d96057cd40 unknown :-1 s=CLOSED pgs=1049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.171+0000 7fa0c466d740 1 -- 172.21.15.86:0/3506661254 >> 172.21.15.86:0/3506661254 conn(0x55d96052ec90 msgr2=0x55d960568fa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.171+0000 7fa0c466d740 1 -- 172.21.15.86:0/3506661254 shutdown_connections 2024-04-21T08:42:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.171+0000 7fa0c466d740 1 -- 172.21.15.86:0/3506661254 wait complete. 2024-04-21T08:42:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.171+0000 7fa0c466d740 1 -- start start 2024-04-21T08:42:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.171+0000 7fa0c466d740 1 Processor -- start 2024-04-21T08:42:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.171+0000 7fa0c466d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d960567b60 0x55d960579540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.171+0000 7fa0c466d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9605688d0 0x55d96056a790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:24.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.171+0000 7fa0c466d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d9605710e0 0x55d96056acd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.171+0000 7fa0c466d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0bc1b28b0 con 0x55d9605688d0 2024-04-21T08:42:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.171+0000 7fa0c466d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa0bc1b21e0 con 0x55d9605710e0 2024-04-21T08:42:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.171+0000 7fa0c466d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa0bc1af400 con 0x55d960567b60 2024-04-21T08:42:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.171+0000 7fa0c466d740 10 client.0 _ll_register_callbacks cb 0x7fa0b00cdb10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.171+0000 7fa0baffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9605688d0 0x55d96056a790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.172+0000 7fa0baffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9605688d0 0x55d96056a790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56232/0 (socket says 172.21.15.86:56232) 2024-04-21T08:42:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.172+0000 7fa0baffd640 1 -- 172.21.15.86:0/266024516 learned_addr learned my addr 172.21.15.86:0/266024516 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.172+0000 7fa0bbfff640 1 --2- 172.21.15.86:0/266024516 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d9605710e0 0x55d96056acd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.172+0000 7fa0bb7fe640 1 --2- 172.21.15.86:0/266024516 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d960567b60 0x55d960579540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.172+0000 7fa0baffd640 1 -- 172.21.15.86:0/266024516 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d960567b60 msgr2=0x55d960579540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.172+0000 7fa0baffd640 1 --2- 172.21.15.86:0/266024516 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d960567b60 0x55d960579540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.172+0000 7fa0baffd640 1 -- 172.21.15.86:0/266024516 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d9605710e0 msgr2=0x55d96056acd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.172+0000 7fa0baffd640 1 --2- 172.21.15.86:0/266024516 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d9605710e0 0x55d96056acd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.172+0000 7fa0baffd640 1 -- 172.21.15.86:0/266024516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d960566e30 con 0x55d9605688d0 2024-04-21T08:42:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.172+0000 7fa0baffd640 1 --2- 172.21.15.86:0/266024516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9605688d0 0x55d96056a790 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fa0b00cde10 tx=0x7fa0b009b120 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe558ed84cd7eb1a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:24.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.172+0000 7fa09affd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.172+0000 7fa09affd640 1 -- 172.21.15.86:0/266024516 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0b0021070 con 0x55d9605688d0 2024-04-21T08:42:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.172+0000 7fa09affd640 1 -- 172.21.15.86:0/266024516 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fa0b00a4ba0 con 0x55d9605688d0 2024-04-21T08:42:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.172+0000 7fa0c466d740 1 -- 172.21.15.86:0/266024516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa0ac0054f0 con 0x55d9605688d0 2024-04-21T08:42:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.172+0000 7fa09affd640 1 -- 172.21.15.86:0/266024516 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa0b0015050 con 0x55d9605688d0 2024-04-21T08:42:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.172+0000 7fa0c466d740 10 client.17322 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.172+0000 7fa0c466d740 1 -- 172.21.15.86:0/266024516 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d9605e7400 con 0x55d9605688d0 2024-04-21T08:42:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.172+0000 7fa09affd640 1 -- 172.21.15.86:0/266024516 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa0ac0054f0 con 0x55d9605688d0 2024-04-21T08:42:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.173+0000 7fa09affd640 1 -- 172.21.15.86:0/266024516 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa0b0098110 con 0x55d9605688d0 2024-04-21T08:42:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.173+0000 7fa0c466d740 10 client.17322 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.173+0000 7fa0c466d740 10 client.17322 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.173+0000 7fa0c466d740 20 client.17322 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.173+0000 7fa0c466d740 1 --2- 172.21.15.86:0/266024516 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d9605e7af0 0x55d9605e9f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.173+0000 7fa0c466d740 4 client.17322 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:42:24.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.173+0000 7fa0c466d740 1 -- 172.21.15.86:0/266024516 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d96046c3d0 con 0x55d9605e7af0 2024-04-21T08:42:24.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.173+0000 7fa0bb7fe640 1 --2- 172.21.15.86:0/266024516 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d9605e7af0 0x55d9605e9f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:24.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.173+0000 7fa09affd640 10 client.17322 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:24.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.173+0000 7fa0bb7fe640 1 --2- 172.21.15.86:0/266024516 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d9605e7af0 0x55d9605e9f10 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2e776f860c9881b2 server_cookie=c7035d083f9758f0 in_seq=0 out_seq=0 2024-04-21T08:42:24.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.188+0000 7fa09affd640 1 -- 172.21.15.86:0/266024516 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d96046c3d0 con 0x55d9605e7af0 2024-04-21T08:42:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.188+0000 7fa09affd640 10 client.17322 handle_command_reply: tid=0 2024-04-21T08:42:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.188+0000 7fa0c466d740 1 client.17322 shutdown 2024-04-21T08:42:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.189+0000 7fa0c466d740 1 -- 172.21.15.86:0/266024516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9605688d0 msgr2=0x55d96056a790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.189+0000 7fa0c466d740 1 --2- 172.21.15.86:0/266024516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9605688d0 0x55d96056a790 secure :-1 s=READY pgs=1050 cs=0 l=1 rev1=1 crypto rx=0x7fa0b00cde10 tx=0x7fa0b009b120 comp rx=0 tx=0).stop 2024-04-21T08:42:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.189+0000 7fa0c466d740 1 -- 172.21.15.86:0/266024516 shutdown_connections 2024-04-21T08:42:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.189+0000 7fa0c466d740 1 --2- 172.21.15.86:0/266024516 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d9605e7af0 0x55d9605e9f10 crc :-1 s=READY pgs=138 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.189+0000 7fa0c466d740 1 --2- 172.21.15.86:0/266024516 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d9605710e0 0x55d96056acd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.189+0000 7fa0c466d740 1 --2- 172.21.15.86:0/266024516 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9605688d0 0x55d96056a790 unknown :-1 s=CLOSED pgs=1050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.189+0000 7fa0c466d740 1 --2- 172.21.15.86:0/266024516 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d960567b60 0x55d960579540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.189+0000 7fa09affd640 0 client.17322 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.189+0000 7fa0c466d740 1 -- 172.21.15.86:0/266024516 >> 172.21.15.86:0/266024516 conn(0x55d96052ec90 msgr2=0x55d96056c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:24.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.189+0000 7fa0c466d740 1 -- 172.21.15.86:0/266024516 shutdown_connections 2024-04-21T08:42:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.189+0000 7fa0c466d740 1 -- 172.21.15.86:0/266024516 wait complete. 2024-04-21T08:42:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.189+0000 7fa0c466d740 20 client.17322 trim_cache size 0 max 16384 2024-04-21T08:42:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:42:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:42:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:42:24.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:42:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.189+0000 7fa0c1a74640 1 -- 172.21.15.86:0/4133718043 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa0a40a3dc0 msgr2=0x7fa0a40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.189+0000 7fa0c1a74640 1 --2- 172.21.15.86:0/4133718043 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa0a40a3dc0 0x7fa0a40a6280 secure :-1 s=READY pgs=160 cs=0 l=1 rev1=1 crypto rx=0x7fa0ac036040 tx=0x7fa0ac000bc0 comp rx=0 tx=0).stop 2024-04-21T08:42:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.189+0000 7fa0c1a74640 1 -- 172.21.15.86:0/4133718043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0bc107290 msgr2=0x7fa0bc074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:24.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.189+0000 7fa0c1a74640 1 --2- 172.21.15.86:0/4133718043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0bc107290 0x7fa0bc074790 secure :-1 s=READY pgs=1047 cs=0 l=1 rev1=1 crypto rx=0x7fa0b000d770 tx=0x7fa0b000dc30 comp rx=0 tx=0).stop 2024-04-21T08:42:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.189+0000 7fa0c1a74640 1 -- 172.21.15.86:0/4133718043 shutdown_connections 2024-04-21T08:42:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.189+0000 7fa0c1a74640 1 --2- 172.21.15.86:0/4133718043 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa0a40a3dc0 0x7fa0a40a6280 unknown :-1 s=CLOSED pgs=160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.189+0000 7fa0c1a74640 1 --2- 172.21.15.86:0/4133718043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0bc074cd0 0x7fa0bc1a6900 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.189+0000 7fa0c1a74640 1 --2- 172.21.15.86:0/4133718043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0bc107290 0x7fa0bc074790 unknown :-1 s=CLOSED pgs=1047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.189+0000 7fa0c1a74640 1 --2- 172.21.15.86:0/4133718043 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0bc1021c0 0x7fa0bc074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.189+0000 7fa0c1a74640 1 -- 172.21.15.86:0/4133718043 >> 172.21.15.86:0/4133718043 conn(0x7fa0bc0fda20 msgr2=0x7fa0bc0ffd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.190+0000 7fa0c1a74640 1 -- 172.21.15.86:0/4133718043 shutdown_connections 2024-04-21T08:42:24.192 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:24.190+0000 7fa0c1a74640 1 -- 172.21.15.86:0/4133718043 wait complete. 2024-04-21T08:42:24.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:24 smithi088 ceph-mon[31259]: pgmap v111: 129 pgs: 129 active+clean; 3.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 40 KiB/s wr, 23 op/s 2024-04-21T08:42:24.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:24 smithi179 ceph-mon[31829]: pgmap v111: 129 pgs: 129 active+clean; 3.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 40 KiB/s wr, 23 op/s 2024-04-21T08:42:24.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:24 smithi086 ceph-mon[27939]: pgmap v111: 129 pgs: 129 active+clean; 3.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 40 KiB/s wr, 23 op/s 2024-04-21T08:42:26.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:26 smithi088 ceph-mon[31259]: pgmap v112: 129 pgs: 129 active+clean; 3.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-04-21T08:42:26.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:26 smithi179 ceph-mon[31829]: pgmap v112: 129 pgs: 129 active+clean; 3.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-04-21T08:42:26.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:26 smithi086 ceph-mon[27939]: pgmap v112: 129 pgs: 129 active+clean; 3.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-04-21T08:42:28.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:28 smithi088 ceph-mon[31259]: pgmap v113: 129 pgs: 129 active+clean; 3.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-04-21T08:42:28.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:28 smithi179 ceph-mon[31829]: pgmap v113: 129 pgs: 129 active+clean; 3.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-04-21T08:42:28.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:28 smithi086 ceph-mon[27939]: pgmap v113: 129 pgs: 129 active+clean; 3.9 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 28 KiB/s rd, 22 KiB/s wr, 12 op/s 2024-04-21T08:42:30.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:30 smithi088 ceph-mon[31259]: pgmap v114: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 40 KiB/s wr, 21 op/s 2024-04-21T08:42:30.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:30 smithi179 ceph-mon[31829]: pgmap v114: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 40 KiB/s wr, 21 op/s 2024-04-21T08:42:30.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:30 smithi086 ceph-mon[27939]: pgmap v114: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 40 KiB/s wr, 21 op/s 2024-04-21T08:42:32.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:32 smithi088 ceph-mon[31259]: pgmap v115: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-21T08:42:32.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:32 smithi179 ceph-mon[31829]: pgmap v115: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-21T08:42:32.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:32 smithi086 ceph-mon[27939]: pgmap v115: 129 pgs: 129 active+clean; 4.1 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-21T08:42:34.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:34 smithi088 ceph-mon[31259]: pgmap v116: 129 pgs: 129 active+clean; 4.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 43 KiB/s wr, 22 op/s 2024-04-21T08:42:34.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:34 smithi179 ceph-mon[31829]: pgmap v116: 129 pgs: 129 active+clean; 4.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 43 KiB/s wr, 22 op/s 2024-04-21T08:42:34.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:34 smithi086 ceph-mon[27939]: pgmap v116: 129 pgs: 129 active+clean; 4.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 43 KiB/s wr, 22 op/s 2024-04-21T08:42:35.816 INFO:tasks.workunit.client.0.smithi086.stdout:64/600: open fd = 3 2024-04-21T08:42:35.817 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.817 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.817 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.817 INFO:tasks.workunit.client.0.smithi086.stdout:65/600: open fd = 3 2024-04-21T08:42:35.817 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.817 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.817 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.817 INFO:tasks.workunit.client.0.smithi086.stdout:66/600: open fd = 3 2024-04-21T08:42:35.817 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.817 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.817 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.817 INFO:tasks.workunit.client.0.smithi086.stdout:67/600: open fd = 3 2024-04-21T08:42:35.817 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.818 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.818 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.818 INFO:tasks.workunit.client.0.smithi086.stdout:68/600: open fd = 3 2024-04-21T08:42:35.818 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.818 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.818 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.818 INFO:tasks.workunit.client.0.smithi086.stdout:69/600: open fd = 3 2024-04-21T08:42:35.818 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.818 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.818 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.818 INFO:tasks.workunit.client.0.smithi086.stdout:70/600: open fd = 3 2024-04-21T08:42:35.818 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.818 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.818 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.819 INFO:tasks.workunit.client.0.smithi086.stdout:71/600: open fd = 3 2024-04-21T08:42:35.819 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.819 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.819 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.819 INFO:tasks.workunit.client.0.smithi086.stdout:72/600: open fd = 3 2024-04-21T08:42:35.819 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.819 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.819 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.819 INFO:tasks.workunit.client.0.smithi086.stdout:73/600: open fd = 3 2024-04-21T08:42:35.819 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.819 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.819 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.819 INFO:tasks.workunit.client.0.smithi086.stdout:74/600: open fd = 3 2024-04-21T08:42:35.819 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.819 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.820 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.820 INFO:tasks.workunit.client.0.smithi086.stdout:75/600: open fd = 3 2024-04-21T08:42:35.820 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.820 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.820 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.820 INFO:tasks.workunit.client.0.smithi086.stdout:76/600: open fd = 3 2024-04-21T08:42:35.820 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.820 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.820 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.820 INFO:tasks.workunit.client.0.smithi086.stdout:77/600: open fd = 3 2024-04-21T08:42:35.820 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.820 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.820 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.820 INFO:tasks.workunit.client.0.smithi086.stdout:78/600: open fd = 3 2024-04-21T08:42:35.820 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.820 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.821 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.821 INFO:tasks.workunit.client.0.smithi086.stdout:79/600: open fd = 3 2024-04-21T08:42:35.821 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.821 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.821 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.821 INFO:tasks.workunit.client.0.smithi086.stdout:80/600: open fd = 3 2024-04-21T08:42:35.821 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.821 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.821 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.821 INFO:tasks.workunit.client.0.smithi086.stdout:81/600: open fd = 3 2024-04-21T08:42:35.821 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.821 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.821 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.821 INFO:tasks.workunit.client.0.smithi086.stdout:82/600: open fd = 3 2024-04-21T08:42:35.821 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.822 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.822 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.822 INFO:tasks.workunit.client.0.smithi086.stdout:83/600: open fd = 3 2024-04-21T08:42:35.822 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.822 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.822 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.822 INFO:tasks.workunit.client.0.smithi086.stdout:84/600: open fd = 3 2024-04-21T08:42:35.822 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.822 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.822 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.822 INFO:tasks.workunit.client.0.smithi086.stdout:85/600: open fd = 3 2024-04-21T08:42:35.822 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.822 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.822 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.822 INFO:tasks.workunit.client.0.smithi086.stdout:86/600: open fd = 3 2024-04-21T08:42:35.823 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.823 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.823 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.823 INFO:tasks.workunit.client.0.smithi086.stdout:87/600: open fd = 3 2024-04-21T08:42:35.823 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.823 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.823 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.823 INFO:tasks.workunit.client.0.smithi086.stdout:88/600: open fd = 3 2024-04-21T08:42:35.823 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.823 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.823 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.823 INFO:tasks.workunit.client.0.smithi086.stdout:89/600: open fd = 3 2024-04-21T08:42:35.823 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.823 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.823 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.824 INFO:tasks.workunit.client.0.smithi086.stdout:90/600: open fd = 3 2024-04-21T08:42:35.824 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.824 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.824 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.824 INFO:tasks.workunit.client.0.smithi086.stdout:91/600: open fd = 3 2024-04-21T08:42:35.824 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.824 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.824 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.824 INFO:tasks.workunit.client.0.smithi086.stdout:92/600: open fd = 3 2024-04-21T08:42:35.824 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.824 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.824 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.824 INFO:tasks.workunit.client.0.smithi086.stdout:93/600: open fd = 3 2024-04-21T08:42:35.824 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.824 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.825 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.825 INFO:tasks.workunit.client.0.smithi086.stdout:94/600: open fd = 3 2024-04-21T08:42:35.825 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.825 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.825 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.825 INFO:tasks.workunit.client.0.smithi086.stdout:95/600: open fd = 3 2024-04-21T08:42:35.825 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.825 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.825 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.825 INFO:tasks.workunit.client.0.smithi086.stdout:96/600: open fd = 3 2024-04-21T08:42:35.825 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.825 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.825 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.825 INFO:tasks.workunit.client.0.smithi086.stdout:97/600: open fd = 3 2024-04-21T08:42:35.825 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.826 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.826 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.826 INFO:tasks.workunit.client.0.smithi086.stdout:98/600: open fd = 3 2024-04-21T08:42:35.826 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.826 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.826 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.826 INFO:tasks.workunit.client.0.smithi086.stdout:99/600: open fd = 3 2024-04-21T08:42:35.826 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.826 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.826 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.826 INFO:tasks.workunit.client.0.smithi086.stdout:100/600: open fd = 3 2024-04-21T08:42:35.826 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.826 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.826 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.826 INFO:tasks.workunit.client.0.smithi086.stdout:101/600: open fd = 3 2024-04-21T08:42:35.827 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.827 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.827 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.827 INFO:tasks.workunit.client.0.smithi086.stdout:102/600: open fd = 3 2024-04-21T08:42:35.827 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.827 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.827 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.827 INFO:tasks.workunit.client.0.smithi086.stdout:103/600: open fd = 3 2024-04-21T08:42:35.827 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.827 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.827 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.827 INFO:tasks.workunit.client.0.smithi086.stdout:104/600: open fd = 3 2024-04-21T08:42:35.827 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.828 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.828 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.828 INFO:tasks.workunit.client.0.smithi086.stdout:105/600: open fd = 3 2024-04-21T08:42:35.828 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.828 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.828 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.828 INFO:tasks.workunit.client.0.smithi086.stdout:106/600: open fd = 3 2024-04-21T08:42:35.828 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.828 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.828 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.828 INFO:tasks.workunit.client.0.smithi086.stdout:107/600: open fd = 3 2024-04-21T08:42:35.828 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.828 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.828 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.828 INFO:tasks.workunit.client.0.smithi086.stdout:108/600: open fd = 3 2024-04-21T08:42:35.828 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.829 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.829 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.829 INFO:tasks.workunit.client.0.smithi086.stdout:109/600: open fd = 3 2024-04-21T08:42:35.829 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.829 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.829 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.829 INFO:tasks.workunit.client.0.smithi086.stdout:110/600: open fd = 3 2024-04-21T08:42:35.829 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.829 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.829 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.829 INFO:tasks.workunit.client.0.smithi086.stdout:111/600: open fd = 3 2024-04-21T08:42:35.829 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.829 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.829 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.830 INFO:tasks.workunit.client.0.smithi086.stdout:112/600: open fd = 3 2024-04-21T08:42:35.830 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.830 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.830 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.830 INFO:tasks.workunit.client.0.smithi086.stdout:113/600: open fd = 3 2024-04-21T08:42:35.830 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.830 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.830 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.830 INFO:tasks.workunit.client.0.smithi086.stdout:114/600: open fd = 3 2024-04-21T08:42:35.830 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.830 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.830 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.830 INFO:tasks.workunit.client.0.smithi086.stdout:115/600: open fd = 3 2024-04-21T08:42:35.830 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.830 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.831 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.831 INFO:tasks.workunit.client.0.smithi086.stdout:116/600: open fd = 3 2024-04-21T08:42:35.831 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.831 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.831 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.831 INFO:tasks.workunit.client.0.smithi086.stdout:117/600: open fd = 3 2024-04-21T08:42:35.831 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.831 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.831 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.831 INFO:tasks.workunit.client.0.smithi086.stdout:118/600: open fd = 3 2024-04-21T08:42:35.831 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.831 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.831 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.831 INFO:tasks.workunit.client.0.smithi086.stdout:119/600: open fd = 3 2024-04-21T08:42:35.831 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.832 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.832 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.832 INFO:tasks.workunit.client.0.smithi086.stdout:120/600: open fd = 3 2024-04-21T08:42:35.832 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.832 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.832 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.832 INFO:tasks.workunit.client.0.smithi086.stdout:121/600: open fd = 3 2024-04-21T08:42:35.832 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.832 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.832 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.832 INFO:tasks.workunit.client.0.smithi086.stdout:122/600: open fd = 3 2024-04-21T08:42:35.832 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.832 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.832 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.832 INFO:tasks.workunit.client.0.smithi086.stdout:123/600: open fd = 3 2024-04-21T08:42:35.833 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.833 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.833 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.833 INFO:tasks.workunit.client.0.smithi086.stdout:124/600: open fd = 3 2024-04-21T08:42:35.833 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.833 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.833 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.833 INFO:tasks.workunit.client.0.smithi086.stdout:125/600: open fd = 3 2024-04-21T08:42:35.833 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.833 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:35.833 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:42:35.833 INFO:tasks.workunit.client.0.smithi086.stdout:126/600: open fd = 3 2024-04-21T08:42:35.833 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:42:36.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:36 smithi088 ceph-mon[31259]: pgmap v117: 129 pgs: 129 active+clean; 4.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-21T08:42:36.766 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:36 smithi179 ceph-mon[31829]: pgmap v117: 129 pgs: 129 active+clean; 4.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-21T08:42:36.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:36 smithi086 ceph-mon[27939]: pgmap v117: 129 pgs: 129 active+clean; 4.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-21T08:42:37.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.285+0000 7f8cc0bc7640 1 -- 172.21.15.86:0/2021915866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cbc106170 msgr2=0x7f8cbc10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.285+0000 7f8cc0bc7640 1 --2- 172.21.15.86:0/2021915866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cbc106170 0x7f8cbc10dd40 secure :-1 s=READY pgs=1051 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4009ca0 tx=0x7f8cb402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:42:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.285+0000 7f8cc0bc7640 1 -- 172.21.15.86:0/2021915866 shutdown_connections 2024-04-21T08:42:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.285+0000 7f8cc0bc7640 1 --2- 172.21.15.86:0/2021915866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cbc106170 0x7f8cbc10dd40 unknown :-1 s=CLOSED pgs=1051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.285+0000 7f8cc0bc7640 1 --2- 172.21.15.86:0/2021915866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cbc101aa0 0x7f8cbc105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.285+0000 7f8cc0bc7640 1 --2- 172.21.15.86:0/2021915866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cbc1010f0 0x7f8cbc1014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.285+0000 7f8cc0bc7640 1 -- 172.21.15.86:0/2021915866 >> 172.21.15.86:0/2021915866 conn(0x7f8cbc077fd0 msgr2=0x7f8cbc0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.285+0000 7f8cc0bc7640 1 -- 172.21.15.86:0/2021915866 shutdown_connections 2024-04-21T08:42:37.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.285+0000 7f8cc0bc7640 1 -- 172.21.15.86:0/2021915866 wait complete. 2024-04-21T08:42:37.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.286+0000 7f8cc0bc7640 1 Processor -- start 2024-04-21T08:42:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.286+0000 7f8cc0bc7640 1 -- start start 2024-04-21T08:42:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.286+0000 7f8cc0bc7640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cbc1010f0 0x7f8cbc1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.286+0000 7f8cc0bc7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cbc106170 0x7f8cbc198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.286+0000 7f8cc0bc7640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cbc19cf40 0x7f8cbc19d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.286+0000 7f8cc0bc7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc1121f0 con 0x7f8cbc106170 2024-04-21T08:42:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.286+0000 7f8cc0bc7640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc112070 con 0x7f8cbc1010f0 2024-04-21T08:42:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.286+0000 7f8cc0bc7640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc112370 con 0x7f8cbc19cf40 2024-04-21T08:42:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.286+0000 7f8cb3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cbc106170 0x7f8cbc198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.286+0000 7f8cb3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cbc106170 0x7f8cbc198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33154/0 (socket says 172.21.15.86:33154) 2024-04-21T08:42:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.286+0000 7f8cb3fff640 1 -- 172.21.15.86:0/2390930264 learned_addr learned my addr 172.21.15.86:0/2390930264 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.286+0000 7f8cba59c640 1 --2- 172.21.15.86:0/2390930264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cbc1010f0 0x7f8cbc1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.286+0000 7f8cbad9d640 1 --2- 172.21.15.86:0/2390930264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cbc19cf40 0x7f8cbc19d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:37.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.287+0000 7f8cb3fff640 1 -- 172.21.15.86:0/2390930264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cbc19cf40 msgr2=0x7f8cbc19d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.287+0000 7f8cb3fff640 1 --2- 172.21.15.86:0/2390930264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cbc19cf40 0x7f8cbc19d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.287+0000 7f8cb3fff640 1 -- 172.21.15.86:0/2390930264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cbc1010f0 msgr2=0x7f8cbc1986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.287+0000 7f8cb3fff640 1 --2- 172.21.15.86:0/2390930264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cbc1010f0 0x7f8cbc1986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.287+0000 7f8cb3fff640 1 -- 172.21.15.86:0/2390930264 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8cbc19da30 con 0x7f8cbc106170 2024-04-21T08:42:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.287+0000 7f8cb3fff640 1 --2- 172.21.15.86:0/2390930264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cbc106170 0x7f8cbc198c10 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f8ca400e9f0 tx=0x7f8ca400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.287+0000 7f8cb37fe640 1 -- 172.21.15.86:0/2390930264 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca400ccc0 con 0x7f8cbc106170 2024-04-21T08:42:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.287+0000 7f8cb37fe640 1 -- 172.21.15.86:0/2390930264 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ca40054d0 con 0x7f8cbc106170 2024-04-21T08:42:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.287+0000 7f8cc0bc7640 1 -- 172.21.15.86:0/2390930264 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8cbc19dd20 con 0x7f8cbc106170 2024-04-21T08:42:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.287+0000 7f8cb37fe640 1 -- 172.21.15.86:0/2390930264 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca4010430 con 0x7f8cbc106170 2024-04-21T08:42:37.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.287+0000 7f8cc0bc7640 1 -- 172.21.15.86:0/2390930264 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8cbc1a54c0 con 0x7f8cbc106170 2024-04-21T08:42:37.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.289+0000 7f8cb37fe640 1 -- 172.21.15.86:0/2390930264 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8ca40046c0 con 0x7f8cbc106170 2024-04-21T08:42:37.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.289+0000 7f8cc37b8740 1 Processor -- start 2024-04-21T08:42:37.296 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.290+0000 7f8cc37b8740 1 -- start start 2024-04-21T08:42:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.290+0000 7f8cc37b8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a43c078d0 0x564a43c06620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.290+0000 7f8cc37b8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a43c06b60 0x564a43c06fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.290+0000 7f8cc37b8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a43c100e0 0x564a43c124d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.290+0000 7f8cc37b8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564a43b6b8d0 con 0x564a43c078d0 2024-04-21T08:42:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.290+0000 7f8cc37b8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564a43b08e80 con 0x564a43c100e0 2024-04-21T08:42:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.290+0000 7f8cc37b8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564a43b67890 con 0x564a43c06b60 2024-04-21T08:42:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.290+0000 7f8cb37fe640 1 --2- 172.21.15.86:0/2390930264 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8ca80a3dc0 0x7f8ca80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.290+0000 7f8cbad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a43c078d0 0x564a43c06620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.290+0000 7f8cbad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a43c078d0 0x564a43c06620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33164/0 (socket says 172.21.15.86:33164) 2024-04-21T08:42:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.290+0000 7f8cbad9d640 1 -- 172.21.15.86:0/1520337544 learned_addr learned my addr 172.21.15.86:0/1520337544 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.290+0000 7f8cbad9d640 1 --2- 172.21.15.86:0/2390930264 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8ca80a3dc0 0x7f8ca80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.290+0000 7f8cba59c640 1 --2- 172.21.15.86:0/1520337544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a43c06b60 0x564a43c06fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:37.297 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.290+0000 7f8cb37fe640 1 -- 172.21.15.86:0/2390930264 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f8ca40d0ea0 con 0x7f8cbc106170 2024-04-21T08:42:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.290+0000 7f8cb3fff640 1 --2- 172.21.15.86:0/1520337544 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a43c100e0 0x564a43c124d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.290+0000 7f8cbad9d640 1 -- 172.21.15.86:0/1520337544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a43c06b60 msgr2=0x564a43c06fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.290+0000 7f8cbad9d640 1 --2- 172.21.15.86:0/1520337544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a43c06b60 0x564a43c06fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.290+0000 7f8cbad9d640 1 -- 172.21.15.86:0/1520337544 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a43c100e0 msgr2=0x564a43c124d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.290+0000 7f8cbad9d640 1 --2- 172.21.15.86:0/1520337544 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a43c100e0 0x564a43c124d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.290+0000 7f8cbad9d640 1 -- 172.21.15.86:0/1520337544 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a43b2a320 con 0x564a43c078d0 2024-04-21T08:42:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.290+0000 7f8cba59c640 1 --2- 172.21.15.86:0/1520337544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a43c06b60 0x564a43c06fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.291+0000 7f8cbad9d640 1 --2- 172.21.15.86:0/1520337544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a43c078d0 0x564a43c06620 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4037040 tx=0x7f8cb4032f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d8790b4283cf417 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.291+0000 7f8cbad9d640 1 --2- 172.21.15.86:0/2390930264 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8ca80a3dc0 0x7f8ca80a6280 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4009ca0 tx=0x7f8cb4042000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.291+0000 7f8cb0ff9640 1 -- 172.21.15.86:0/1520337544 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb4051070 con 0x564a43c078d0 2024-04-21T08:42:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.291+0000 7f8cb0ff9640 1 -- 172.21.15.86:0/1520337544 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cb40327c0 con 0x564a43c078d0 2024-04-21T08:42:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.291+0000 7f8cb0ff9640 1 -- 172.21.15.86:0/1520337544 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cb4039040 con 0x564a43c078d0 2024-04-21T08:42:37.298 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.292+0000 7f8cc37b8740 1 -- 172.21.15.86:0/1520337544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a43c078d0 msgr2=0x564a43c06620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.292+0000 7f8cc37b8740 1 --2- 172.21.15.86:0/1520337544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a43c078d0 0x564a43c06620 secure :-1 s=READY pgs=1053 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4037040 tx=0x7f8cb4032f70 comp rx=0 tx=0).stop 2024-04-21T08:42:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.292+0000 7f8cc37b8740 1 -- 172.21.15.86:0/1520337544 shutdown_connections 2024-04-21T08:42:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.292+0000 7f8cc37b8740 1 --2- 172.21.15.86:0/1520337544 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a43c100e0 0x564a43c124d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.292+0000 7f8cc37b8740 1 --2- 172.21.15.86:0/1520337544 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a43c06b60 0x564a43c06fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.292+0000 7f8cc37b8740 1 --2- 172.21.15.86:0/1520337544 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a43c078d0 0x564a43c06620 unknown :-1 s=CLOSED pgs=1053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.292+0000 7f8cc37b8740 1 -- 172.21.15.86:0/1520337544 >> 172.21.15.86:0/1520337544 conn(0x564a43bcdc90 msgr2=0x564a43c01850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.292+0000 7f8cc37b8740 1 -- 172.21.15.86:0/1520337544 shutdown_connections 2024-04-21T08:42:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.292+0000 7f8cc37b8740 1 -- 172.21.15.86:0/1520337544 wait complete. 2024-04-21T08:42:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.292+0000 7f8cc37b8740 1 -- start start 2024-04-21T08:42:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.292+0000 7f8cc37b8740 1 Processor -- start 2024-04-21T08:42:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.292+0000 7f8cc37b8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a43c06b60 0x564a43c0fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.292+0000 7f8cc37b8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a43c078d0 0x564a43c0ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.292+0000 7f8cc37b8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a43c100e0 0x564a43c0d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:37.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.292+0000 7f8cc37b8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564a43c12db0 con 0x564a43c078d0 2024-04-21T08:42:37.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.292+0000 7f8cc37b8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564a43af3eb0 con 0x564a43c100e0 2024-04-21T08:42:37.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.292+0000 7f8cc37b8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564a43c12f30 con 0x564a43c06b60 2024-04-21T08:42:37.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.292+0000 7f8cc37b8740 10 client.0 _ll_register_callbacks cb 0x564a43b20b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:37.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.293+0000 7f8cb3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a43c078d0 0x564a43c0ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:37.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.293+0000 7f8cb3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a43c078d0 0x564a43c0ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33170/0 (socket says 172.21.15.86:33170) 2024-04-21T08:42:37.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.293+0000 7f8cb3fff640 1 -- 172.21.15.86:0/2626957236 learned_addr learned my addr 172.21.15.86:0/2626957236 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:37.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.293+0000 7f8cba59c640 1 --2- 172.21.15.86:0/2626957236 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a43c06b60 0x564a43c0fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:37.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.293+0000 7f8cbad9d640 1 --2- 172.21.15.86:0/2626957236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a43c100e0 0x564a43c0d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:37.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.293+0000 7f8cb3fff640 1 -- 172.21.15.86:0/2626957236 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a43c06b60 msgr2=0x564a43c0fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:37.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.293+0000 7f8cb3fff640 1 --2- 172.21.15.86:0/2626957236 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a43c06b60 0x564a43c0fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:37.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.293+0000 7f8cb3fff640 1 -- 172.21.15.86:0/2626957236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a43c100e0 msgr2=0x564a43c0d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:37.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.293+0000 7f8cb3fff640 1 --2- 172.21.15.86:0/2626957236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a43c100e0 0x564a43c0d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:37.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.293+0000 7f8cb3fff640 1 -- 172.21.15.86:0/2626957236 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a4387e210 con 0x564a43c078d0 2024-04-21T08:42:37.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.293+0000 7f8cb3fff640 1 --2- 172.21.15.86:0/2626957236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a43c078d0 0x564a43c0ce70 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f8ca409c7d0 tx=0x7f8ca409c4f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61ae085e4a98319c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:37.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.293+0000 7f8ca1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:37.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.293+0000 7f8ca1ffb640 1 -- 172.21.15.86:0/2626957236 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca40a0e80 con 0x564a43c078d0 2024-04-21T08:42:37.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.293+0000 7f8ca1ffb640 1 -- 172.21.15.86:0/2626957236 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f8ca40ce500 con 0x564a43c078d0 2024-04-21T08:42:37.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.293+0000 7f8cc37b8740 1 -- 172.21.15.86:0/2626957236 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cb40327c0 con 0x564a43c078d0 2024-04-21T08:42:37.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.293+0000 7f8ca1ffb640 1 -- 172.21.15.86:0/2626957236 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8ca40ce700 con 0x564a43c078d0 2024-04-21T08:42:37.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.293+0000 7f8cc37b8740 10 client.17334 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:37.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.293+0000 7f8cc37b8740 1 -- 172.21.15.86:0/2626957236 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564a4394a620 con 0x564a43c078d0 2024-04-21T08:42:37.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.293+0000 7f8ca1ffb640 1 -- 172.21.15.86:0/2626957236 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cb40327c0 con 0x564a43c078d0 2024-04-21T08:42:37.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.294+0000 7f8ca1ffb640 1 -- 172.21.15.86:0/2626957236 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8ca409bfb0 con 0x564a43c078d0 2024-04-21T08:42:37.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.294+0000 7f8cc37b8740 10 client.17334 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:37.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.294+0000 7f8cc37b8740 10 client.17334 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:37.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.294+0000 7f8cc37b8740 20 client.17334 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:37.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.294+0000 7f8cc37b8740 1 --2- 172.21.15.86:0/2626957236 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564a43c20130 0x564a43c22520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.294+0000 7f8cc37b8740 4 client.17334 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.294+0000 7f8cc37b8740 1 -- 172.21.15.86:0/2626957236 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a43b7f880 con 0x564a43c20130 2024-04-21T08:42:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.294+0000 7f8cba59c640 1 --2- 172.21.15.86:0/2626957236 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564a43c20130 0x564a43c22520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.294+0000 7f8cba59c640 1 --2- 172.21.15.86:0/2626957236 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564a43c20130 0x564a43c22520 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd7b334390dc1155 server_cookie=6f09b559909cf283 in_seq=0 out_seq=0 2024-04-21T08:42:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.294+0000 7f8ca1ffb640 10 client.17334 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.295+0000 7f8ca1ffb640 1 -- 172.21.15.86:0/2626957236 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564a43b7f880 con 0x564a43c20130 2024-04-21T08:42:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.295+0000 7f8ca1ffb640 10 client.17334 handle_command_reply: tid=0 2024-04-21T08:42:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.295+0000 7f8cc37b8740 1 client.17334 shutdown 2024-04-21T08:42:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.295+0000 7f8cc37b8740 1 -- 172.21.15.86:0/2626957236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a43c078d0 msgr2=0x564a43c0ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.295+0000 7f8cc37b8740 1 --2- 172.21.15.86:0/2626957236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a43c078d0 0x564a43c0ce70 secure :-1 s=READY pgs=1054 cs=0 l=1 rev1=1 crypto rx=0x7f8ca409c7d0 tx=0x7f8ca409c4f0 comp rx=0 tx=0).stop 2024-04-21T08:42:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.295+0000 7f8cc37b8740 1 -- 172.21.15.86:0/2626957236 shutdown_connections 2024-04-21T08:42:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.295+0000 7f8cc37b8740 1 --2- 172.21.15.86:0/2626957236 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564a43c20130 0x564a43c22520 crc :-1 s=READY pgs=139 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.295+0000 7f8cc37b8740 1 --2- 172.21.15.86:0/2626957236 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a43c100e0 0x564a43c0d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:37.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.295+0000 7f8cc37b8740 1 --2- 172.21.15.86:0/2626957236 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a43c078d0 0x564a43c0ce70 unknown :-1 s=CLOSED pgs=1054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.295+0000 7f8cc37b8740 1 --2- 172.21.15.86:0/2626957236 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a43c06b60 0x564a43c0fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.295+0000 7f8ca1ffb640 0 client.17334 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.295+0000 7f8cc37b8740 1 -- 172.21.15.86:0/2626957236 >> 172.21.15.86:0/2626957236 conn(0x564a43bcdc90 msgr2=0x564a43c035a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.295+0000 7f8cc37b8740 1 -- 172.21.15.86:0/2626957236 shutdown_connections 2024-04-21T08:42:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.295+0000 7f8cc37b8740 1 -- 172.21.15.86:0/2626957236 wait complete. 2024-04-21T08:42:37.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.296+0000 7f8cc37b8740 20 client.17334 trim_cache size 0 max 16384 2024-04-21T08:42:37.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.311+0000 7f8cc37b8740 1 Processor -- start 2024-04-21T08:42:37.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.311+0000 7f8cc37b8740 1 -- start start 2024-04-21T08:42:37.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.311+0000 7f8cc37b8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a43c06b60 0x564a43c1bcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:37.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.311+0000 7f8cc37b8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a43c078d0 0x564a43c0f5a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:37.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.311+0000 7f8cc37b8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a43c100e0 0x564a43c0fae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:37.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.311+0000 7f8cc37b8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc1af1d0 con 0x564a43c06b60 2024-04-21T08:42:37.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.311+0000 7f8cc37b8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc1b0d10 con 0x564a43c078d0 2024-04-21T08:42:37.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.311+0000 7f8cc37b8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc1af8a0 con 0x564a43c100e0 2024-04-21T08:42:37.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.311+0000 7f8cba59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a43c06b60 0x564a43c1bcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:37.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.311+0000 7f8cba59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a43c06b60 0x564a43c1bcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33182/0 (socket says 172.21.15.86:33182) 2024-04-21T08:42:37.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.311+0000 7f8cba59c640 1 -- 172.21.15.86:0/2982200664 learned_addr learned my addr 172.21.15.86:0/2982200664 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:37.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.311+0000 7f8cbad9d640 1 --2- 172.21.15.86:0/2982200664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a43c100e0 0x564a43c0fae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:37.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.311+0000 7f8cb3fff640 1 --2- 172.21.15.86:0/2982200664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a43c078d0 0x564a43c0f5a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:37.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.311+0000 7f8cba59c640 1 -- 172.21.15.86:0/2982200664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a43c100e0 msgr2=0x564a43c0fae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:37.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.311+0000 7f8cba59c640 1 --2- 172.21.15.86:0/2982200664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a43c100e0 0x564a43c0fae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:37.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.311+0000 7f8cba59c640 1 -- 172.21.15.86:0/2982200664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a43c078d0 msgr2=0x564a43c0f5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:37.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.311+0000 7f8cba59c640 1 --2- 172.21.15.86:0/2982200664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a43c078d0 0x564a43c0f5a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:37.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.311+0000 7f8cba59c640 1 -- 172.21.15.86:0/2982200664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a43c22b00 con 0x564a43c06b60 2024-04-21T08:42:37.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.311+0000 7f8cb3fff640 1 --2- 172.21.15.86:0/2982200664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a43c078d0 0x564a43c0f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:37.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.311+0000 7f8cba59c640 1 --2- 172.21.15.86:0/2982200664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a43c06b60 0x564a43c1bcb0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f8cac000c00 tx=0x7f8cac011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3a3e7ea2d107caf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:37.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.311+0000 7f8ca3fff640 1 -- 172.21.15.86:0/2982200664 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cac021030 con 0x564a43c06b60 2024-04-21T08:42:37.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.312+0000 7f8ca3fff640 1 -- 172.21.15.86:0/2982200664 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cac01d420 con 0x564a43c06b60 2024-04-21T08:42:37.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.312+0000 7f8ca3fff640 1 -- 172.21.15.86:0/2982200664 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8cac003950 con 0x564a43c06b60 2024-04-21T08:42:37.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.312+0000 7f8cc37b8740 1 -- 172.21.15.86:0/2982200664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a43c06b60 msgr2=0x564a43c1bcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:37.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.312+0000 7f8cc37b8740 1 --2- 172.21.15.86:0/2982200664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a43c06b60 0x564a43c1bcb0 secure :-1 s=READY pgs=1055 cs=0 l=1 rev1=1 crypto rx=0x7f8cac000c00 tx=0x7f8cac011810 comp rx=0 tx=0).stop 2024-04-21T08:42:37.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.312+0000 7f8cc37b8740 1 -- 172.21.15.86:0/2982200664 shutdown_connections 2024-04-21T08:42:37.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.312+0000 7f8cc37b8740 1 --2- 172.21.15.86:0/2982200664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a43c100e0 0x564a43c0fae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:37.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.312+0000 7f8cc37b8740 1 --2- 172.21.15.86:0/2982200664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a43c078d0 0x564a43c0f5a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.312+0000 7f8cc37b8740 1 --2- 172.21.15.86:0/2982200664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a43c06b60 0x564a43c1bcb0 unknown :-1 s=CLOSED pgs=1055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.312+0000 7f8cc37b8740 1 -- 172.21.15.86:0/2982200664 >> 172.21.15.86:0/2982200664 conn(0x564a43bcdc90 msgr2=0x564a43c0b740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.312+0000 7f8cc37b8740 1 -- 172.21.15.86:0/2982200664 shutdown_connections 2024-04-21T08:42:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.312+0000 7f8cc37b8740 1 -- 172.21.15.86:0/2982200664 wait complete. 2024-04-21T08:42:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.312+0000 7f8cc37b8740 1 -- start start 2024-04-21T08:42:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.312+0000 7f8cc37b8740 1 Processor -- start 2024-04-21T08:42:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.313+0000 7f8cc37b8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a43c06b60 0x564a43c181c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.313+0000 7f8cc37b8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a43c078d0 0x564a43c18700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.313+0000 7f8cc37b8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a43c100e0 0x564a43c112c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.313+0000 7f8cc37b8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc1b0640 con 0x564a43c078d0 2024-04-21T08:42:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.313+0000 7f8cc37b8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc1aff70 con 0x564a43c100e0 2024-04-21T08:42:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.313+0000 7f8cc37b8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8cbc1ad190 con 0x564a43c06b60 2024-04-21T08:42:37.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.313+0000 7f8cc37b8740 10 client.0 _ll_register_callbacks cb 0x7f8ca4098e00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.313+0000 7f8cb3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a43c078d0 0x564a43c18700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.313+0000 7f8cb3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a43c078d0 0x564a43c18700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33192/0 (socket says 172.21.15.86:33192) 2024-04-21T08:42:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.313+0000 7f8cb3fff640 1 -- 172.21.15.86:0/1083831058 learned_addr learned my addr 172.21.15.86:0/1083831058 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.313+0000 7f8cba59c640 1 --2- 172.21.15.86:0/1083831058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a43c06b60 0x564a43c181c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.313+0000 7f8cbad9d640 1 --2- 172.21.15.86:0/1083831058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a43c100e0 0x564a43c112c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.313+0000 7f8cb3fff640 1 -- 172.21.15.86:0/1083831058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a43c06b60 msgr2=0x564a43c181c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.313+0000 7f8cb3fff640 1 --2- 172.21.15.86:0/1083831058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a43c06b60 0x564a43c181c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.313+0000 7f8cb3fff640 1 -- 172.21.15.86:0/1083831058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a43c100e0 msgr2=0x564a43c112c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.313+0000 7f8cb3fff640 1 --2- 172.21.15.86:0/1083831058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a43c100e0 0x564a43c112c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.313+0000 7f8cb3fff640 1 -- 172.21.15.86:0/1083831058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a43c05e30 con 0x564a43c078d0 2024-04-21T08:42:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.313+0000 7f8cbad9d640 1 --2- 172.21.15.86:0/1083831058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a43c100e0 0x564a43c112c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.313+0000 7f8cb3fff640 1 --2- 172.21.15.86:0/1083831058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a43c078d0 0x564a43c18700 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f8ca409c370 tx=0x7f8ca4004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b499ead5f7ac2a4d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.313+0000 7f8ca1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:37.319 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.313+0000 7f8ca1ffb640 1 -- 172.21.15.86:0/1083831058 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ca409c610 con 0x564a43c078d0 2024-04-21T08:42:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.313+0000 7f8ca1ffb640 1 -- 172.21.15.86:0/1083831058 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f8ca409bf20 con 0x564a43c078d0 2024-04-21T08:42:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.313+0000 7f8cc37b8740 1 -- 172.21.15.86:0/1083831058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8cac01d420 con 0x564a43c078d0 2024-04-21T08:42:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.314+0000 7f8ca1ffb640 1 -- 172.21.15.86:0/1083831058 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8ca40a7e60 con 0x564a43c078d0 2024-04-21T08:42:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.314+0000 7f8cc37b8740 10 client.17340 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.314+0000 7f8cc37b8740 1 -- 172.21.15.86:0/1083831058 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564a43c86bd0 con 0x564a43c078d0 2024-04-21T08:42:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.314+0000 7f8ca1ffb640 1 -- 172.21.15.86:0/1083831058 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8cac01d420 con 0x564a43c078d0 2024-04-21T08:42:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.314+0000 7f8ca1ffb640 1 -- 172.21.15.86:0/1083831058 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8ca409b9e0 con 0x564a43c078d0 2024-04-21T08:42:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.314+0000 7f8cc37b8740 10 client.17340 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.314+0000 7f8cc37b8740 10 client.17340 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.314+0000 7f8cc37b8740 20 client.17340 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.314+0000 7f8cc37b8740 1 --2- 172.21.15.86:0/1083831058 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564a43c87130 0x564a43c89550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.314+0000 7f8cc37b8740 4 client.17340 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:42:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.314+0000 7f8cc37b8740 1 -- 172.21.15.86:0/1083831058 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564a43b0b3d0 con 0x564a43c87130 2024-04-21T08:42:37.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.314+0000 7f8cba59c640 1 --2- 172.21.15.86:0/1083831058 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564a43c87130 0x564a43c89550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.315+0000 7f8ca1ffb640 10 client.17340 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:37.321 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.315+0000 7f8cba59c640 1 --2- 172.21.15.86:0/1083831058 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564a43c87130 0x564a43c89550 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bac46bf680092585 server_cookie=9dbf28cf6b2667d5 in_seq=0 out_seq=0 2024-04-21T08:42:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.330+0000 7f8ca1ffb640 1 -- 172.21.15.86:0/1083831058 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564a43b0b3d0 con 0x564a43c87130 2024-04-21T08:42:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.330+0000 7f8ca1ffb640 10 client.17340 handle_command_reply: tid=0 2024-04-21T08:42:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.330+0000 7f8cc37b8740 1 client.17340 shutdown 2024-04-21T08:42:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.330+0000 7f8cc37b8740 1 -- 172.21.15.86:0/1083831058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a43c078d0 msgr2=0x564a43c18700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.330+0000 7f8cc37b8740 1 --2- 172.21.15.86:0/1083831058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a43c078d0 0x564a43c18700 secure :-1 s=READY pgs=1056 cs=0 l=1 rev1=1 crypto rx=0x7f8ca409c370 tx=0x7f8ca4004a60 comp rx=0 tx=0).stop 2024-04-21T08:42:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.330+0000 7f8cc37b8740 1 -- 172.21.15.86:0/1083831058 shutdown_connections 2024-04-21T08:42:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.330+0000 7f8cc37b8740 1 --2- 172.21.15.86:0/1083831058 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564a43c87130 0x564a43c89550 crc :-1 s=READY pgs=140 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.330+0000 7f8cc37b8740 1 --2- 172.21.15.86:0/1083831058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a43c100e0 0x564a43c112c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:37.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.330+0000 7f8cc37b8740 1 --2- 172.21.15.86:0/1083831058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a43c078d0 0x564a43c18700 unknown :-1 s=CLOSED pgs=1056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.330+0000 7f8cc37b8740 1 --2- 172.21.15.86:0/1083831058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a43c06b60 0x564a43c181c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.330+0000 7f8ca1ffb640 0 client.17340 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.330+0000 7f8cc37b8740 1 -- 172.21.15.86:0/1083831058 >> 172.21.15.86:0/1083831058 conn(0x564a43bcdc90 msgr2=0x564a43c00040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.330+0000 7f8cc37b8740 1 -- 172.21.15.86:0/1083831058 shutdown_connections 2024-04-21T08:42:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.330+0000 7f8cc37b8740 1 -- 172.21.15.86:0/1083831058 wait complete. 2024-04-21T08:42:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.331+0000 7f8cc37b8740 20 client.17340 trim_cache size 0 max 16384 2024-04-21T08:42:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:42:37.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:42:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:42:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:42:37.332 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.331+0000 7f8cc0bc7640 1 -- 172.21.15.86:0/2390930264 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8ca80a3dc0 msgr2=0x7f8ca80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.331+0000 7f8cc0bc7640 1 --2- 172.21.15.86:0/2390930264 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8ca80a3dc0 0x7f8ca80a6280 secure :-1 s=READY pgs=161 cs=0 l=1 rev1=1 crypto rx=0x7f8cb4009ca0 tx=0x7f8cb4042000 comp rx=0 tx=0).stop 2024-04-21T08:42:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.331+0000 7f8cc0bc7640 1 -- 172.21.15.86:0/2390930264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cbc106170 msgr2=0x7f8cbc198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.331+0000 7f8cc0bc7640 1 --2- 172.21.15.86:0/2390930264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cbc106170 0x7f8cbc198c10 secure :-1 s=READY pgs=1052 cs=0 l=1 rev1=1 crypto rx=0x7f8ca400e9f0 tx=0x7f8ca400eeb0 comp rx=0 tx=0).stop 2024-04-21T08:42:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.331+0000 7f8cc0bc7640 1 -- 172.21.15.86:0/2390930264 shutdown_connections 2024-04-21T08:42:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.331+0000 7f8cc0bc7640 1 --2- 172.21.15.86:0/2390930264 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8ca80a3dc0 0x7f8ca80a6280 unknown :-1 s=CLOSED pgs=161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.331+0000 7f8cc0bc7640 1 --2- 172.21.15.86:0/2390930264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8cbc19cf40 0x7f8cbc19d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.331+0000 7f8cc0bc7640 1 --2- 172.21.15.86:0/2390930264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8cbc106170 0x7f8cbc198c10 unknown :-1 s=CLOSED pgs=1052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.331+0000 7f8cc0bc7640 1 --2- 172.21.15.86:0/2390930264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8cbc1010f0 0x7f8cbc1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.331+0000 7f8cc0bc7640 1 -- 172.21.15.86:0/2390930264 >> 172.21.15.86:0/2390930264 conn(0x7f8cbc077fd0 msgr2=0x7f8cbc0ff010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.331+0000 7f8cc0bc7640 1 -- 172.21.15.86:0/2390930264 shutdown_connections 2024-04-21T08:42:37.333 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:37.331+0000 7f8cc0bc7640 1 -- 172.21.15.86:0/2390930264 wait complete. 2024-04-21T08:42:37.959 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:42:37.959 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:42:38.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.071+0000 7f6b79811640 1 -- 172.21.15.86:0/3274631541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b74103ae0 msgr2=0x7f6b7410bfc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.071+0000 7f6b79811640 1 --2- 172.21.15.86:0/3274631541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b74103ae0 0x7f6b7410bfc0 secure :-1 s=READY pgs=1057 cs=0 l=1 rev1=1 crypto rx=0x7f6b68009900 tx=0x7f6b6802bd60 comp rx=0 tx=0).stop 2024-04-21T08:42:38.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.072+0000 7f6b79811640 1 -- 172.21.15.86:0/3274631541 shutdown_connections 2024-04-21T08:42:38.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.072+0000 7f6b79811640 1 --2- 172.21.15.86:0/3274631541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b7410c500 0x7f6b7410e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.072+0000 7f6b79811640 1 --2- 172.21.15.86:0/3274631541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b74103ae0 0x7f6b7410bfc0 unknown :-1 s=CLOSED pgs=1057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.072+0000 7f6b79811640 1 --2- 172.21.15.86:0/3274631541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b741031c0 0x7f6b741035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.072+0000 7f6b79811640 1 -- 172.21.15.86:0/3274631541 >> 172.21.15.86:0/3274631541 conn(0x7f6b74078000 msgr2=0x7f6b74078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:38.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.072+0000 7f6b79811640 1 -- 172.21.15.86:0/3274631541 shutdown_connections 2024-04-21T08:42:38.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.072+0000 7f6b79811640 1 -- 172.21.15.86:0/3274631541 wait complete. 2024-04-21T08:42:38.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.072+0000 7f6b79811640 1 Processor -- start 2024-04-21T08:42:38.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.072+0000 7f6b79811640 1 -- start start 2024-04-21T08:42:38.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.073+0000 7f6b79811640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b741031c0 0x7f6b7419e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.073+0000 7f6b79811640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b74103ae0 0x7f6b7419e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.073+0000 7f6b79811640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b7410c500 0x7f6b741985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.073+0000 7f6b79811640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6b74112170 con 0x7f6b741031c0 2024-04-21T08:42:38.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.073+0000 7f6b79811640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6b74111ff0 con 0x7f6b7410c500 2024-04-21T08:42:38.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.073+0000 7f6b79811640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6b741122f0 con 0x7f6b74103ae0 2024-04-21T08:42:38.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.073+0000 7f6b72ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b741031c0 0x7f6b7419e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.073+0000 7f6b72ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b741031c0 0x7f6b7419e410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33210/0 (socket says 172.21.15.86:33210) 2024-04-21T08:42:38.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.073+0000 7f6b72ffd640 1 -- 172.21.15.86:0/586785581 learned_addr learned my addr 172.21.15.86:0/586785581 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:38.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.073+0000 7f6b737fe640 1 --2- 172.21.15.86:0/586785581 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b7410c500 0x7f6b741985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.073+0000 7f6b72ffd640 1 -- 172.21.15.86:0/586785581 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b74103ae0 msgr2=0x7f6b7419e950 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T08:42:38.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.073+0000 7f6b72ffd640 1 --2- 172.21.15.86:0/586785581 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b74103ae0 0x7f6b7419e950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.073+0000 7f6b72ffd640 1 -- 172.21.15.86:0/586785581 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b7410c500 msgr2=0x7f6b741985c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.073+0000 7f6b72ffd640 1 --2- 172.21.15.86:0/586785581 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b7410c500 0x7f6b741985c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.073+0000 7f6b72ffd640 1 -- 172.21.15.86:0/586785581 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6b74198d40 con 0x7f6b741031c0 2024-04-21T08:42:38.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.073+0000 7f6b737fe640 1 --2- 172.21.15.86:0/586785581 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b7410c500 0x7f6b741985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:38.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.073+0000 7f6b72ffd640 1 --2- 172.21.15.86:0/586785581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b741031c0 0x7f6b7419e410 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f6b6400ebc0 tx=0x7f6b6400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:38.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.073+0000 7f6b7880f640 1 -- 172.21.15.86:0/586785581 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b64010630 con 0x7f6b741031c0 2024-04-21T08:42:38.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.073+0000 7f6b7880f640 1 -- 172.21.15.86:0/586785581 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b64010c80 con 0x7f6b741031c0 2024-04-21T08:42:38.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.073+0000 7f6b79811640 1 -- 172.21.15.86:0/586785581 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6b74199030 con 0x7f6b741031c0 2024-04-21T08:42:38.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.073+0000 7f6b7880f640 1 -- 172.21.15.86:0/586785581 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b64018b60 con 0x7f6b741031c0 2024-04-21T08:42:38.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.073+0000 7f6b79811640 1 -- 172.21.15.86:0/586785581 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6b741a52d0 con 0x7f6b741031c0 2024-04-21T08:42:38.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.075+0000 7f6b7880f640 1 -- 172.21.15.86:0/586785581 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6b64018d00 con 0x7f6b741031c0 2024-04-21T08:42:38.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.076+0000 7f6b7c400740 1 Processor -- start 2024-04-21T08:42:38.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.076+0000 7f6b7c400740 1 -- start start 2024-04-21T08:42:38.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.076+0000 7f6b7880f640 1 --2- 172.21.15.86:0/586785581 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f6b5c0a3d20 0x7f6b5c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.076+0000 7f6b7c400740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55847f957230 0x55847f957610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.076+0000 7f6b7c400740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55847f957b50 0x55847f964950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.076+0000 7f6b7c400740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55847f964e90 0x55847f967280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.076+0000 7f6b7c400740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55847f8bc1e0 con 0x55847f957230 2024-04-21T08:42:38.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.076+0000 7f6b7c400740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55847f859390 con 0x55847f964e90 2024-04-21T08:42:38.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.076+0000 7f6b7c400740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55847f8b81a0 con 0x55847f957b50 2024-04-21T08:42:38.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.076+0000 7f6b72ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55847f957230 0x55847f957610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.076+0000 7f6b737fe640 1 --2- 172.21.15.86:0/586785581 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f6b5c0a3d20 0x7f6b5c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.076+0000 7f6b72ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55847f957230 0x55847f957610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33214/0 (socket says 172.21.15.86:33214) 2024-04-21T08:42:38.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.076+0000 7f6b72ffd640 1 -- 172.21.15.86:0/2885098782 learned_addr learned my addr 172.21.15.86:0/2885098782 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:38.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.076+0000 7f6b7880f640 1 -- 172.21.15.86:0/586785581 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f6b640d12b0 con 0x7f6b741031c0 2024-04-21T08:42:38.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.076+0000 7f6b72ffd640 1 -- 172.21.15.86:0/2885098782 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55847f957b50 msgr2=0x55847f964950 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:42:38.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.076+0000 7f6b72ffd640 1 --2- 172.21.15.86:0/2885098782 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55847f957b50 0x55847f964950 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.076+0000 7f6b737fe640 1 --2- 172.21.15.86:0/2885098782 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55847f964e90 0x55847f967280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.076+0000 7f6b72ffd640 1 -- 172.21.15.86:0/2885098782 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55847f964e90 msgr2=0x55847f967280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:38.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.077+0000 7f6b72ffd640 1 --2- 172.21.15.86:0/2885098782 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55847f964e90 0x55847f967280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.077+0000 7f6b72ffd640 1 -- 172.21.15.86:0/2885098782 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55847f6987d0 con 0x55847f957230 2024-04-21T08:42:38.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.077+0000 7f6b737fe640 1 --2- 172.21.15.86:0/2885098782 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55847f964e90 0x55847f967280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:38.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.077+0000 7f6b737fe640 1 --2- 172.21.15.86:0/586785581 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f6b5c0a3d20 0x7f6b5c0a61e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c005d80 tx=0x7f6b6c009510 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:38.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.077+0000 7f6b72ffd640 1 --2- 172.21.15.86:0/2885098782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55847f957230 0x55847f957610 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f6b640ce6f0 tx=0x7f6b6406c7c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=579c8c16eef1a41d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:38.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.077+0000 7f6b59ffb640 1 -- 172.21.15.86:0/2885098782 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6409a530 con 0x55847f957230 2024-04-21T08:42:38.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.077+0000 7f6b59ffb640 1 -- 172.21.15.86:0/2885098782 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b64018420 con 0x55847f957230 2024-04-21T08:42:38.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.077+0000 7f6b59ffb640 1 -- 172.21.15.86:0/2885098782 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b640a4850 con 0x55847f957230 2024-04-21T08:42:38.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.077+0000 7f6b7c400740 1 -- 172.21.15.86:0/2885098782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55847f957230 msgr2=0x55847f957610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.077+0000 7f6b7c400740 1 --2- 172.21.15.86:0/2885098782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55847f957230 0x55847f957610 secure :-1 s=READY pgs=1059 cs=0 l=1 rev1=1 crypto rx=0x7f6b640ce6f0 tx=0x7f6b6406c7c0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.078+0000 7f6b7c400740 1 -- 172.21.15.86:0/2885098782 shutdown_connections 2024-04-21T08:42:38.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.078+0000 7f6b7c400740 1 --2- 172.21.15.86:0/2885098782 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55847f964e90 0x55847f967280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.078+0000 7f6b7c400740 1 --2- 172.21.15.86:0/2885098782 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55847f957b50 0x55847f964950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.078+0000 7f6b7c400740 1 --2- 172.21.15.86:0/2885098782 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55847f957230 0x55847f957610 unknown :-1 s=CLOSED pgs=1059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.078+0000 7f6b7c400740 1 -- 172.21.15.86:0/2885098782 >> 172.21.15.86:0/2885098782 conn(0x55847f951cb0 msgr2=0x55847f952090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:38.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.078+0000 7f6b7c400740 1 -- 172.21.15.86:0/2885098782 shutdown_connections 2024-04-21T08:42:38.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.078+0000 7f6b7c400740 1 -- 172.21.15.86:0/2885098782 wait complete. 2024-04-21T08:42:38.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.078+0000 7f6b7c400740 1 -- start start 2024-04-21T08:42:38.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.078+0000 7f6b7c400740 1 Processor -- start 2024-04-21T08:42:38.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.078+0000 7f6b7c400740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55847f957230 0x55847f961e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.078+0000 7f6b7c400740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55847f957b50 0x55847f9623d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.078+0000 7f6b7c400740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55847f964e90 0x55847f962910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.078+0000 7f6b7c400740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55847f967cd0 con 0x55847f957b50 2024-04-21T08:42:38.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.078+0000 7f6b7c400740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55847f8752f0 con 0x55847f964e90 2024-04-21T08:42:38.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.078+0000 7f6b7c400740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55847f967e50 con 0x55847f957230 2024-04-21T08:42:38.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.078+0000 7f6b7c400740 10 client.0 _ll_register_callbacks cb 0x55847f882580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:38.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.078+0000 7f6b727fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55847f957b50 0x55847f9623d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.079+0000 7f6b727fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55847f957b50 0x55847f9623d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33216/0 (socket says 172.21.15.86:33216) 2024-04-21T08:42:38.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.079+0000 7f6b727fc640 1 -- 172.21.15.86:0/962868707 learned_addr learned my addr 172.21.15.86:0/962868707 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:38.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.079+0000 7f6b737fe640 1 --2- 172.21.15.86:0/962868707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55847f964e90 0x55847f962910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.079+0000 7f6b727fc640 1 -- 172.21.15.86:0/962868707 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55847f957230 msgr2=0x55847f961e90 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:42:38.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.079+0000 7f6b727fc640 1 --2- 172.21.15.86:0/962868707 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55847f957230 0x55847f961e90 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.079+0000 7f6b727fc640 1 -- 172.21.15.86:0/962868707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55847f964e90 msgr2=0x55847f962910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:38.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.079+0000 7f6b727fc640 1 --2- 172.21.15.86:0/962868707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55847f964e90 0x55847f962910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.079+0000 7f6b727fc640 1 -- 172.21.15.86:0/962868707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55847f5ce3f0 con 0x55847f957b50 2024-04-21T08:42:38.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.079+0000 7f6b737fe640 1 --2- 172.21.15.86:0/962868707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55847f964e90 0x55847f962910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:38.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.079+0000 7f6b727fc640 1 --2- 172.21.15.86:0/962868707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55847f957b50 0x55847f9623d0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f6b68005770 tx=0x7f6b6802cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d3a94ba4d0bc33a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:38.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.079+0000 7f6b52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:38.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.079+0000 7f6b52ffd640 1 -- 172.21.15.86:0/962868707 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b68041020 con 0x55847f957b50 2024-04-21T08:42:38.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.079+0000 7f6b52ffd640 1 -- 172.21.15.86:0/962868707 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f6b68005250 con 0x55847f957b50 2024-04-21T08:42:38.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.079+0000 7f6b7c400740 1 -- 172.21.15.86:0/962868707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55847f8c9c00 con 0x55847f957b50 2024-04-21T08:42:38.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.079+0000 7f6b52ffd640 1 -- 172.21.15.86:0/962868707 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6b6804c050 con 0x55847f957b50 2024-04-21T08:42:38.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.079+0000 7f6b7c400740 10 client.17352 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:38.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.079+0000 7f6b7c400740 1 -- 172.21.15.86:0/962868707 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55847f58e0f0 con 0x55847f957b50 2024-04-21T08:42:38.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.079+0000 7f6b52ffd640 1 -- 172.21.15.86:0/962868707 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55847f8c9c00 con 0x55847f957b50 2024-04-21T08:42:38.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.080+0000 7f6b52ffd640 1 -- 172.21.15.86:0/962868707 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6b68004b70 con 0x55847f957b50 2024-04-21T08:42:38.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.080+0000 7f6b7c400740 10 client.17352 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:38.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.080+0000 7f6b7c400740 10 client.17352 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:38.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.080+0000 7f6b7c400740 20 client.17352 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:38.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.080+0000 7f6b7c400740 1 --2- 172.21.15.86:0/962868707 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55847f974f40 0x55847f9773e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.080+0000 7f6b7c400740 4 client.17352 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:38.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.080+0000 7f6b7c400740 1 -- 172.21.15.86:0/962868707 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55847f8c3a70 con 0x55847f974f40 2024-04-21T08:42:38.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.080+0000 7f6b72ffd640 1 --2- 172.21.15.86:0/962868707 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55847f974f40 0x55847f9773e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.080+0000 7f6b52ffd640 10 client.17352 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:38.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.080+0000 7f6b72ffd640 1 --2- 172.21.15.86:0/962868707 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55847f974f40 0x55847f9773e0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f9e5ae5bc4a14d10 server_cookie=31af632a9c72200c in_seq=0 out_seq=0 2024-04-21T08:42:38.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.081+0000 7f6b52ffd640 1 -- 172.21.15.86:0/962868707 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55847f8c3a70 con 0x55847f974f40 2024-04-21T08:42:38.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.081+0000 7f6b52ffd640 10 client.17352 handle_command_reply: tid=0 2024-04-21T08:42:38.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.081+0000 7f6b7c400740 1 client.17352 shutdown 2024-04-21T08:42:38.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.081+0000 7f6b7c400740 1 -- 172.21.15.86:0/962868707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55847f957b50 msgr2=0x55847f9623d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.081+0000 7f6b7c400740 1 --2- 172.21.15.86:0/962868707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55847f957b50 0x55847f9623d0 secure :-1 s=READY pgs=1060 cs=0 l=1 rev1=1 crypto rx=0x7f6b68005770 tx=0x7f6b6802cb20 comp rx=0 tx=0).stop 2024-04-21T08:42:38.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.081+0000 7f6b7c400740 1 -- 172.21.15.86:0/962868707 shutdown_connections 2024-04-21T08:42:38.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.081+0000 7f6b7c400740 1 --2- 172.21.15.86:0/962868707 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55847f974f40 0x55847f9773e0 crc :-1 s=READY pgs=141 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.081+0000 7f6b7c400740 1 --2- 172.21.15.86:0/962868707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55847f964e90 0x55847f962910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.081+0000 7f6b7c400740 1 --2- 172.21.15.86:0/962868707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55847f957b50 0x55847f9623d0 unknown :-1 s=CLOSED pgs=1060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.081+0000 7f6b52ffd640 0 client.17352 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:38.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.081+0000 7f6b7c400740 1 --2- 172.21.15.86:0/962868707 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55847f957230 0x55847f961e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.081+0000 7f6b7c400740 1 -- 172.21.15.86:0/962868707 >> 172.21.15.86:0/962868707 conn(0x55847f951cb0 msgr2=0x55847f953a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:38.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.081+0000 7f6b7c400740 1 -- 172.21.15.86:0/962868707 shutdown_connections 2024-04-21T08:42:38.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.081+0000 7f6b7c400740 1 -- 172.21.15.86:0/962868707 wait complete. 2024-04-21T08:42:38.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.082+0000 7f6b7c400740 20 client.17352 trim_cache size 0 max 16384 2024-04-21T08:42:38.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.099+0000 7f6b7c400740 1 Processor -- start 2024-04-21T08:42:38.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.099+0000 7f6b7c400740 1 -- start start 2024-04-21T08:42:38.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.099+0000 7f6b7c400740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55847f957230 0x55847f9645c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.099+0000 7f6b7c400740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55847f957b50 0x55847f970c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.099+0000 7f6b7c400740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55847f964e90 0x55847f97ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.099+0000 7f6b7c400740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6b741ad1b0 con 0x55847f957b50 2024-04-21T08:42:38.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.099+0000 7f6b7c400740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6b741aecf0 con 0x55847f964e90 2024-04-21T08:42:38.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.099+0000 7f6b7c400740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6b741ad880 con 0x55847f957230 2024-04-21T08:42:38.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.099+0000 7f6b727fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55847f957b50 0x55847f970c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.099+0000 7f6b727fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55847f957b50 0x55847f970c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33222/0 (socket says 172.21.15.86:33222) 2024-04-21T08:42:38.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.099+0000 7f6b727fc640 1 -- 172.21.15.86:0/2206558451 learned_addr learned my addr 172.21.15.86:0/2206558451 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:38.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.099+0000 7f6b737fe640 1 --2- 172.21.15.86:0/2206558451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55847f964e90 0x55847f97ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.100+0000 7f6b727fc640 1 -- 172.21.15.86:0/2206558451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55847f957230 msgr2=0x55847f9645c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:42:38.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.100+0000 7f6b727fc640 1 --2- 172.21.15.86:0/2206558451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55847f957230 0x55847f9645c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.100+0000 7f6b727fc640 1 -- 172.21.15.86:0/2206558451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55847f964e90 msgr2=0x55847f97ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:38.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.100+0000 7f6b727fc640 1 --2- 172.21.15.86:0/2206558451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55847f964e90 0x55847f97ccb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.100+0000 7f6b727fc640 1 -- 172.21.15.86:0/2206558451 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55847f95cc60 con 0x55847f957b50 2024-04-21T08:42:38.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.100+0000 7f6b727fc640 1 --2- 172.21.15.86:0/2206558451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55847f957b50 0x55847f970c30 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f6b6802bda0 tx=0x7f6b680038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=23c0ab8bed7aa0cb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:38.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.100+0000 7f6b597fa640 1 -- 172.21.15.86:0/2206558451 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b68056070 con 0x55847f957b50 2024-04-21T08:42:38.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.100+0000 7f6b597fa640 1 -- 172.21.15.86:0/2206558451 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b680054b0 con 0x55847f957b50 2024-04-21T08:42:38.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.100+0000 7f6b597fa640 1 -- 172.21.15.86:0/2206558451 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b68041070 con 0x55847f957b50 2024-04-21T08:42:38.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.100+0000 7f6b7c400740 1 -- 172.21.15.86:0/2206558451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55847f957b50 msgr2=0x55847f970c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.100+0000 7f6b7c400740 1 --2- 172.21.15.86:0/2206558451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55847f957b50 0x55847f970c30 secure :-1 s=READY pgs=1061 cs=0 l=1 rev1=1 crypto rx=0x7f6b6802bda0 tx=0x7f6b680038a0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.101+0000 7f6b7c400740 1 -- 172.21.15.86:0/2206558451 shutdown_connections 2024-04-21T08:42:38.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.101+0000 7f6b7c400740 1 --2- 172.21.15.86:0/2206558451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55847f964e90 0x55847f97ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.101+0000 7f6b7c400740 1 --2- 172.21.15.86:0/2206558451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55847f957b50 0x55847f970c30 unknown :-1 s=CLOSED pgs=1061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.101+0000 7f6b7c400740 1 --2- 172.21.15.86:0/2206558451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55847f957230 0x55847f9645c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.101+0000 7f6b7c400740 1 -- 172.21.15.86:0/2206558451 >> 172.21.15.86:0/2206558451 conn(0x55847f951cb0 msgr2=0x55847f960760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:38.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.101+0000 7f6b7c400740 1 -- 172.21.15.86:0/2206558451 shutdown_connections 2024-04-21T08:42:38.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.101+0000 7f6b7c400740 1 -- 172.21.15.86:0/2206558451 wait complete. 2024-04-21T08:42:38.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.101+0000 7f6b7c400740 1 -- start start 2024-04-21T08:42:38.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.101+0000 7f6b7c400740 1 Processor -- start 2024-04-21T08:42:38.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.101+0000 7f6b7c400740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55847f957230 0x55847f96d330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.101+0000 7f6b7c400740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55847f957b50 0x55847f9686a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.101+0000 7f6b7c400740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55847f964e90 0x55847f968be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.101+0000 7f6b7c400740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6b741ae620 con 0x55847f964e90 2024-04-21T08:42:38.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.101+0000 7f6b7c400740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6b741adf50 con 0x55847f957b50 2024-04-21T08:42:38.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.101+0000 7f6b7c400740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6b741ab170 con 0x55847f957230 2024-04-21T08:42:38.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.101+0000 7f6b7c400740 10 client.0 _ll_register_callbacks cb 0x7f6b640093a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:38.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.101+0000 7f6b737fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55847f964e90 0x55847f968be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.101+0000 7f6b737fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55847f964e90 0x55847f968be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33230/0 (socket says 172.21.15.86:33230) 2024-04-21T08:42:38.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.101+0000 7f6b737fe640 1 -- 172.21.15.86:0/292566316 learned_addr learned my addr 172.21.15.86:0/292566316 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:38.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.102+0000 7f6b727fc640 1 --2- 172.21.15.86:0/292566316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55847f957b50 0x55847f9686a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.102+0000 7f6b737fe640 1 -- 172.21.15.86:0/292566316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55847f957230 msgr2=0x55847f96d330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:38.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.102+0000 7f6b737fe640 1 --2- 172.21.15.86:0/292566316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55847f957230 0x55847f96d330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.102+0000 7f6b737fe640 1 -- 172.21.15.86:0/292566316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55847f957b50 msgr2=0x55847f9686a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:38.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.102+0000 7f6b737fe640 1 --2- 172.21.15.86:0/292566316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55847f957b50 0x55847f9686a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.102+0000 7f6b737fe640 1 -- 172.21.15.86:0/292566316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55847f969220 con 0x55847f964e90 2024-04-21T08:42:38.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.102+0000 7f6b737fe640 1 --2- 172.21.15.86:0/292566316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55847f964e90 0x55847f968be0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c012420 tx=0x7f6b6c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1625281b6a46101 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:38.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.102+0000 7f6b52ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:38.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.102+0000 7f6b52ffd640 1 -- 172.21.15.86:0/292566316 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6b6c020840 con 0x55847f964e90 2024-04-21T08:42:38.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.102+0000 7f6b52ffd640 1 -- 172.21.15.86:0/292566316 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f6b6c0209e0 con 0x55847f964e90 2024-04-21T08:42:38.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.102+0000 7f6b7c400740 1 -- 172.21.15.86:0/292566316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6b680054b0 con 0x55847f964e90 2024-04-21T08:42:38.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.102+0000 7f6b52ffd640 1 -- 172.21.15.86:0/292566316 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6b6c032050 con 0x55847f964e90 2024-04-21T08:42:38.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.102+0000 7f6b7c400740 10 client.17358 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:38.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.102+0000 7f6b7c400740 1 -- 172.21.15.86:0/292566316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55847f9db3c0 con 0x55847f964e90 2024-04-21T08:42:38.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.102+0000 7f6b52ffd640 1 -- 172.21.15.86:0/292566316 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6b680054b0 con 0x55847f964e90 2024-04-21T08:42:38.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.102+0000 7f6b52ffd640 1 -- 172.21.15.86:0/292566316 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6b6c01fc80 con 0x55847f964e90 2024-04-21T08:42:38.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.103+0000 7f6b7c400740 10 client.17358 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:38.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.103+0000 7f6b7c400740 10 client.17358 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:38.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.103+0000 7f6b7c400740 20 client.17358 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:38.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.103+0000 7f6b7c400740 1 --2- 172.21.15.86:0/292566316 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55847f9dbb40 0x55847f9ddf60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.103+0000 7f6b7c400740 4 client.17358 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:42:38.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.103+0000 7f6b7c400740 1 -- 172.21.15.86:0/292566316 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55847f83f590 con 0x55847f9dbb40 2024-04-21T08:42:38.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.103+0000 7f6b72ffd640 1 --2- 172.21.15.86:0/292566316 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55847f9dbb40 0x55847f9ddf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.103+0000 7f6b52ffd640 10 client.17358 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:38.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.103+0000 7f6b72ffd640 1 --2- 172.21.15.86:0/292566316 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55847f9dbb40 0x55847f9ddf60 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=195fad42cb51ba0c server_cookie=cdefa011a6309d49 in_seq=0 out_seq=0 2024-04-21T08:42:38.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.104+0000 7f6b52ffd640 1 -- 172.21.15.86:0/292566316 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55847f83f590 con 0x55847f9dbb40 2024-04-21T08:42:38.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.104+0000 7f6b52ffd640 10 client.17358 handle_command_reply: tid=0 2024-04-21T08:42:38.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.104+0000 7f6b7c400740 1 client.17358 shutdown 2024-04-21T08:42:38.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.104+0000 7f6b7c400740 1 -- 172.21.15.86:0/292566316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55847f964e90 msgr2=0x55847f968be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.104+0000 7f6b7c400740 1 --2- 172.21.15.86:0/292566316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55847f964e90 0x55847f968be0 secure :-1 s=READY pgs=1062 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c012420 tx=0x7f6b6c004730 comp rx=0 tx=0).stop 2024-04-21T08:42:38.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.104+0000 7f6b7c400740 1 -- 172.21.15.86:0/292566316 shutdown_connections 2024-04-21T08:42:38.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.104+0000 7f6b7c400740 1 --2- 172.21.15.86:0/292566316 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55847f9dbb40 0x55847f9ddf60 crc :-1 s=READY pgs=142 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.104+0000 7f6b7c400740 1 --2- 172.21.15.86:0/292566316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55847f964e90 0x55847f968be0 unknown :-1 s=CLOSED pgs=1062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.104+0000 7f6b52ffd640 0 client.17358 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:38.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.104+0000 7f6b7c400740 1 --2- 172.21.15.86:0/292566316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55847f957b50 0x55847f9686a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.104+0000 7f6b7c400740 1 --2- 172.21.15.86:0/292566316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55847f957230 0x55847f96d330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.104+0000 7f6b7c400740 1 -- 172.21.15.86:0/292566316 >> 172.21.15.86:0/292566316 conn(0x55847f951cb0 msgr2=0x55847f961e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:38.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.104+0000 7f6b7c400740 1 -- 172.21.15.86:0/292566316 shutdown_connections 2024-04-21T08:42:38.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.104+0000 7f6b7c400740 1 -- 172.21.15.86:0/292566316 wait complete. 2024-04-21T08:42:38.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.104+0000 7f6b7c400740 20 client.17358 trim_cache size 0 max 16384 2024-04-21T08:42:38.110 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:42:38.110 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-21T08:42:38.110 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-21T08:42:38.110 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:42:38.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.104+0000 7f6b79811640 1 -- 172.21.15.86:0/586785581 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f6b5c0a3d20 msgr2=0x7f6b5c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.104+0000 7f6b79811640 1 --2- 172.21.15.86:0/586785581 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f6b5c0a3d20 0x7f6b5c0a61e0 secure :-1 s=READY pgs=162 cs=0 l=1 rev1=1 crypto rx=0x7f6b6c005d80 tx=0x7f6b6c009510 comp rx=0 tx=0).stop 2024-04-21T08:42:38.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.104+0000 7f6b79811640 1 -- 172.21.15.86:0/586785581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b741031c0 msgr2=0x7f6b7419e410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.104+0000 7f6b79811640 1 --2- 172.21.15.86:0/586785581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b741031c0 0x7f6b7419e410 secure :-1 s=READY pgs=1058 cs=0 l=1 rev1=1 crypto rx=0x7f6b6400ebc0 tx=0x7f6b6400c6a0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.104+0000 7f6b79811640 1 -- 172.21.15.86:0/586785581 shutdown_connections 2024-04-21T08:42:38.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.104+0000 7f6b79811640 1 --2- 172.21.15.86:0/586785581 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f6b5c0a3d20 0x7f6b5c0a61e0 unknown :-1 s=CLOSED pgs=162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.105+0000 7f6b79811640 1 --2- 172.21.15.86:0/586785581 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6b7410c500 0x7f6b741985c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.105+0000 7f6b79811640 1 --2- 172.21.15.86:0/586785581 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6b74103ae0 0x7f6b7419e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.105+0000 7f6b79811640 1 --2- 172.21.15.86:0/586785581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6b741031c0 0x7f6b7419e410 unknown :-1 s=CLOSED pgs=1058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.105+0000 7f6b79811640 1 -- 172.21.15.86:0/586785581 >> 172.21.15.86:0/586785581 conn(0x7f6b74078000 msgr2=0x7f6b740ff310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:38.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.105+0000 7f6b79811640 1 -- 172.21.15.86:0/586785581 shutdown_connections 2024-04-21T08:42:38.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.105+0000 7f6b79811640 1 -- 172.21.15.86:0/586785581 wait complete. 2024-04-21T08:42:38.114 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:42:38.115 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:42:38.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.213+0000 7f4ee6932640 1 -- 172.21.15.86:0/3087959290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee0102b50 msgr2=0x7f4ee0106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.213+0000 7f4ee6932640 1 --2- 172.21.15.86:0/3087959290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee0102b50 0x7f4ee0106b90 secure :-1 s=READY pgs=1063 cs=0 l=1 rev1=1 crypto rx=0x7f4ed4009900 tx=0x7f4ed402bc50 comp rx=0 tx=0).stop 2024-04-21T08:42:38.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.213+0000 7f4ee6932640 1 -- 172.21.15.86:0/3087959290 shutdown_connections 2024-04-21T08:42:38.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.213+0000 7f4ee6932640 1 --2- 172.21.15.86:0/3087959290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ee0107270 0x7f4ee010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.213+0000 7f4ee6932640 1 --2- 172.21.15.86:0/3087959290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee0102b50 0x7f4ee0106b90 unknown :-1 s=CLOSED pgs=1063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.213+0000 7f4ee6932640 1 --2- 172.21.15.86:0/3087959290 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ee01021a0 0x7f4ee0102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.213+0000 7f4ee6932640 1 -- 172.21.15.86:0/3087959290 >> 172.21.15.86:0/3087959290 conn(0x7f4ee00793a0 msgr2=0x7f4ee00797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:38.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.213+0000 7f4ee6932640 1 -- 172.21.15.86:0/3087959290 shutdown_connections 2024-04-21T08:42:38.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.213+0000 7f4ee6932640 1 -- 172.21.15.86:0/3087959290 wait complete. 2024-04-21T08:42:38.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.213+0000 7f4ee6932640 1 Processor -- start 2024-04-21T08:42:38.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.214+0000 7f4ee6932640 1 -- start start 2024-04-21T08:42:38.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.214+0000 7f4ee6932640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ee01021a0 0x7f4ee019e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.214+0000 7f4ee6932640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ee0107270 0x7f4ee019e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.214+0000 7f4ee6932640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee01985b0 0x7f4ee0198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.214+0000 7f4ee6932640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee0112230 con 0x7f4ee01985b0 2024-04-21T08:42:38.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.214+0000 7f4ee6932640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee01120b0 con 0x7f4ee0107270 2024-04-21T08:42:38.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.214+0000 7f4ee6932640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4ee01123b0 con 0x7f4ee01021a0 2024-04-21T08:42:38.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.214+0000 7f4ee4ecf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee01985b0 0x7f4ee0198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.214+0000 7f4ee4ecf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee01985b0 0x7f4ee0198a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33254/0 (socket says 172.21.15.86:33254) 2024-04-21T08:42:38.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.214+0000 7f4ee4ecf640 1 -- 172.21.15.86:0/431114796 learned_addr learned my addr 172.21.15.86:0/431114796 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:38.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.214+0000 7f4edf7fe640 1 --2- 172.21.15.86:0/431114796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ee0107270 0x7f4ee019e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.214+0000 7f4ee4ecf640 1 -- 172.21.15.86:0/431114796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ee01021a0 msgr2=0x7f4ee019e400 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T08:42:38.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.214+0000 7f4ee4ecf640 1 --2- 172.21.15.86:0/431114796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ee01021a0 0x7f4ee019e400 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.214+0000 7f4ee4ecf640 1 -- 172.21.15.86:0/431114796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ee0107270 msgr2=0x7f4ee019e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.214+0000 7f4ee4ecf640 1 --2- 172.21.15.86:0/431114796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ee0107270 0x7f4ee019e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.214+0000 7f4ee4ecf640 1 -- 172.21.15.86:0/431114796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4ee01990d0 con 0x7f4ee01985b0 2024-04-21T08:42:38.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.214+0000 7f4edf7fe640 1 --2- 172.21.15.86:0/431114796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ee0107270 0x7f4ee019e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:38.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.214+0000 7f4ee4ecf640 1 --2- 172.21.15.86:0/431114796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee01985b0 0x7f4ee0198a60 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f4ed800b990 tx=0x7f4ed800be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:38.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.215+0000 7f4edd7fa640 1 -- 172.21.15.86:0/431114796 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed8019070 con 0x7f4ee01985b0 2024-04-21T08:42:38.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.215+0000 7f4edd7fa640 1 -- 172.21.15.86:0/431114796 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ed800cba0 con 0x7f4ee01985b0 2024-04-21T08:42:38.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.215+0000 7f4ee6932640 1 -- 172.21.15.86:0/431114796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4ee01993c0 con 0x7f4ee01985b0 2024-04-21T08:42:38.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.215+0000 7f4edd7fa640 1 -- 172.21.15.86:0/431114796 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ed80075d0 con 0x7f4ee01985b0 2024-04-21T08:42:38.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.215+0000 7f4ee6932640 1 -- 172.21.15.86:0/431114796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4ee01a5610 con 0x7f4ee01985b0 2024-04-21T08:42:38.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.216+0000 7f4ee6932640 1 -- 172.21.15.86:0/431114796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4ee0069230 con 0x7f4ee01985b0 2024-04-21T08:42:38.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.217+0000 7f4edd7fa640 1 -- 172.21.15.86:0/431114796 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4ed801c070 con 0x7f4ee01985b0 2024-04-21T08:42:38.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.217+0000 7f4edd7fa640 1 --2- 172.21.15.86:0/431114796 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4ec80a3da0 0x7f4ec80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.217+0000 7f4edffff640 1 --2- 172.21.15.86:0/431114796 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4ec80a3da0 0x7f4ec80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.217+0000 7f4edd7fa640 1 -- 172.21.15.86:0/431114796 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f4ed80d0530 con 0x7f4ee01985b0 2024-04-21T08:42:38.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.218+0000 7f4edffff640 1 --2- 172.21.15.86:0/431114796 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4ec80a3da0 0x7f4ec80a6260 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4ed000b990 tx=0x7f4ed000b370 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:38.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.221+0000 7f4edd7fa640 1 -- 172.21.15.86:0/431114796 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4ed809c970 con 0x7f4ee01985b0 2024-04-21T08:42:38.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.437+0000 7f4ee6932640 1 -- 172.21.15.86:0/431114796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4ee0199ae0 con 0x7f4ee01985b0 2024-04-21T08:42:38.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.438+0000 7f4edd7fa640 1 -- 172.21.15.86:0/431114796 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7f4ed809c270 con 0x7f4ee01985b0 2024-04-21T08:42:38.437 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:42:38.437 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:38:45.202126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24398,24394,24455,24472]},"id":1}]} 2024-04-21T08:42:38.437 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-21T08:42:38.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.440+0000 7f4ee6932640 1 -- 172.21.15.86:0/431114796 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4ec80a3da0 msgr2=0x7f4ec80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.440+0000 7f4ee6932640 1 --2- 172.21.15.86:0/431114796 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4ec80a3da0 0x7f4ec80a6260 secure :-1 s=READY pgs=163 cs=0 l=1 rev1=1 crypto rx=0x7f4ed000b990 tx=0x7f4ed000b370 comp rx=0 tx=0).stop 2024-04-21T08:42:38.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.440+0000 7f4ee6932640 1 -- 172.21.15.86:0/431114796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee01985b0 msgr2=0x7f4ee0198a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.440+0000 7f4ee6932640 1 --2- 172.21.15.86:0/431114796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee01985b0 0x7f4ee0198a60 secure :-1 s=READY pgs=1064 cs=0 l=1 rev1=1 crypto rx=0x7f4ed800b990 tx=0x7f4ed800be50 comp rx=0 tx=0).stop 2024-04-21T08:42:38.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.440+0000 7f4ee6932640 1 -- 172.21.15.86:0/431114796 shutdown_connections 2024-04-21T08:42:38.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.440+0000 7f4ee6932640 1 --2- 172.21.15.86:0/431114796 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4ec80a3da0 0x7f4ec80a6260 unknown :-1 s=CLOSED pgs=163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.440+0000 7f4ee6932640 1 --2- 172.21.15.86:0/431114796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4ee01985b0 0x7f4ee0198a60 unknown :-1 s=CLOSED pgs=1064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.440+0000 7f4ee6932640 1 --2- 172.21.15.86:0/431114796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4ee0107270 0x7f4ee019e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.440+0000 7f4ee6932640 1 --2- 172.21.15.86:0/431114796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4ee01021a0 0x7f4ee019e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.440+0000 7f4ee6932640 1 -- 172.21.15.86:0/431114796 >> 172.21.15.86:0/431114796 conn(0x7f4ee00793a0 msgr2=0x7f4ee00fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:38.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.440+0000 7f4ee6932640 1 -- 172.21.15.86:0/431114796 shutdown_connections 2024-04-21T08:42:38.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.440+0000 7f4ee6932640 1 -- 172.21.15.86:0/431114796 wait complete. 2024-04-21T08:42:38.450 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:42:38.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.551+0000 7f4a5f23f640 1 -- 172.21.15.86:0/1892336664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a58101a40 msgr2=0x7f4a58105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.551+0000 7f4a5f23f640 1 --2- 172.21.15.86:0/1892336664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a58101a40 0x7f4a58105a80 secure :-1 s=READY pgs=1065 cs=0 l=1 rev1=1 crypto rx=0x7f4a44009900 tx=0x7f4a4402bc50 comp rx=0 tx=0).stop 2024-04-21T08:42:38.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.551+0000 7f4a5f23f640 1 -- 172.21.15.86:0/1892336664 shutdown_connections 2024-04-21T08:42:38.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.551+0000 7f4a5f23f640 1 --2- 172.21.15.86:0/1892336664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a58106160 0x7f4a5810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.551+0000 7f4a5f23f640 1 --2- 172.21.15.86:0/1892336664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a58101a40 0x7f4a58105a80 unknown :-1 s=CLOSED pgs=1065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.551+0000 7f4a5f23f640 1 --2- 172.21.15.86:0/1892336664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a58101090 0x7f4a58101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.551+0000 7f4a5f23f640 1 -- 172.21.15.86:0/1892336664 >> 172.21.15.86:0/1892336664 conn(0x7f4a58077f50 msgr2=0x7f4a58078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:38.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.551+0000 7f4a5f23f640 1 -- 172.21.15.86:0/1892336664 shutdown_connections 2024-04-21T08:42:38.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.551+0000 7f4a5f23f640 1 -- 172.21.15.86:0/1892336664 wait complete. 2024-04-21T08:42:38.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.551+0000 7f4a5f23f640 1 Processor -- start 2024-04-21T08:42:38.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.551+0000 7f4a5f23f640 1 -- start start 2024-04-21T08:42:38.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.552+0000 7f4a5f23f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a58101090 0x7f4a5819e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.552+0000 7f4a5f23f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a58106160 0x7f4a5819e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.552+0000 7f4a5f23f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a581985a0 0x7f4a58198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.552+0000 7f4a5f23f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4a58112220 con 0x7f4a58106160 2024-04-21T08:42:38.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.552+0000 7f4a5f23f640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4a581120a0 con 0x7f4a58101090 2024-04-21T08:42:38.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.552+0000 7f4a5f23f640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4a581123a0 con 0x7f4a581985a0 2024-04-21T08:42:38.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.552+0000 7f4a57fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a58106160 0x7f4a5819e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.552+0000 7f4a57fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a58106160 0x7f4a5819e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33270/0 (socket says 172.21.15.86:33270) 2024-04-21T08:42:38.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.552+0000 7f4a57fff640 1 -- 172.21.15.86:0/267833783 learned_addr learned my addr 172.21.15.86:0/267833783 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:38.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.552+0000 7f4a5cfdb640 1 --2- 172.21.15.86:0/267833783 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a58101090 0x7f4a5819e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.552+0000 7f4a57fff640 1 -- 172.21.15.86:0/267833783 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a581985a0 msgr2=0x7f4a58198a50 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T08:42:38.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.552+0000 7f4a57fff640 1 --2- 172.21.15.86:0/267833783 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a581985a0 0x7f4a58198a50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.552+0000 7f4a57fff640 1 -- 172.21.15.86:0/267833783 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a58101090 msgr2=0x7f4a5819e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.552+0000 7f4a57fff640 1 --2- 172.21.15.86:0/267833783 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a58101090 0x7f4a5819e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.552+0000 7f4a57fff640 1 -- 172.21.15.86:0/267833783 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a581990c0 con 0x7f4a58106160 2024-04-21T08:42:38.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.552+0000 7f4a57fff640 1 --2- 172.21.15.86:0/267833783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a58106160 0x7f4a5819e930 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f4a44000c00 tx=0x7f4a4402c800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:38.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.552+0000 7f4a55ffb640 1 -- 172.21.15.86:0/267833783 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a44043020 con 0x7f4a58106160 2024-04-21T08:42:38.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.552+0000 7f4a5f23f640 1 -- 172.21.15.86:0/267833783 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a58199350 con 0x7f4a58106160 2024-04-21T08:42:38.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.552+0000 7f4a55ffb640 1 -- 172.21.15.86:0/267833783 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a440052b0 con 0x7f4a58106160 2024-04-21T08:42:38.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.552+0000 7f4a55ffb640 1 -- 172.21.15.86:0/267833783 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a44003cd0 con 0x7f4a58106160 2024-04-21T08:42:38.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.553+0000 7f4a5f23f640 1 -- 172.21.15.86:0/267833783 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a581a55a0 con 0x7f4a58106160 2024-04-21T08:42:38.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.555+0000 7f4a55ffb640 1 -- 172.21.15.86:0/267833783 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4a44046020 con 0x7f4a58106160 2024-04-21T08:42:38.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.555+0000 7f4a61e2b740 1 Processor -- start 2024-04-21T08:42:38.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.555+0000 7f4a61e2b740 1 -- start start 2024-04-21T08:42:38.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.555+0000 7f4a61e2b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c69d751330 0x55c69d751710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.555+0000 7f4a61e2b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c69d751ce0 0x55c69d75a820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.555+0000 7f4a61e2b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c69d75ae40 0x55c69d75b2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.555+0000 7f4a61e2b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c69d6b61f0 con 0x55c69d751330 2024-04-21T08:42:38.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.555+0000 7f4a61e2b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c69d6534b0 con 0x55c69d751ce0 2024-04-21T08:42:38.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.555+0000 7f4a61e2b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c69d6b21b0 con 0x55c69d75ae40 2024-04-21T08:42:38.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.555+0000 7f4a55ffb640 1 --2- 172.21.15.86:0/267833783 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4a480a3d20 0x7f4a480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.555+0000 7f4a5d7dc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c69d751330 0x55c69d751710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.555+0000 7f4a5d7dc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c69d751330 0x55c69d751710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33282/0 (socket says 172.21.15.86:33282) 2024-04-21T08:42:38.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.555+0000 7f4a5d7dc640 1 -- 172.21.15.86:0/1121375744 learned_addr learned my addr 172.21.15.86:0/1121375744 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:38.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.556+0000 7f4a55ffb640 1 -- 172.21.15.86:0/267833783 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f4a440f2dc0 con 0x7f4a58106160 2024-04-21T08:42:38.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.556+0000 7f4a5d7dc640 1 --2- 172.21.15.86:0/267833783 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4a480a3d20 0x7f4a480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.556+0000 7f4a5cfdb640 1 --2- 172.21.15.86:0/1121375744 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c69d751ce0 0x55c69d75a820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.556+0000 7f4a5d7dc640 1 -- 172.21.15.86:0/1121375744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c69d75ae40 msgr2=0x55c69d75b2c0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:42:38.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.556+0000 7f4a5d7dc640 1 --2- 172.21.15.86:0/1121375744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c69d75ae40 0x55c69d75b2c0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.556+0000 7f4a5d7dc640 1 -- 172.21.15.86:0/1121375744 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c69d751ce0 msgr2=0x55c69d75a820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:38.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.556+0000 7f4a5d7dc640 1 --2- 172.21.15.86:0/1121375744 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c69d751ce0 0x55c69d75a820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.556+0000 7f4a5d7dc640 1 -- 172.21.15.86:0/1121375744 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c69d6451b0 con 0x55c69d751330 2024-04-21T08:42:38.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.556+0000 7f4a5d7dc640 1 --2- 172.21.15.86:0/1121375744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c69d751330 0x55c69d751710 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f4a50012710 tx=0x7f4a50012bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=646620b94845cfb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:38.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.556+0000 7f4a42ffd640 1 -- 172.21.15.86:0/1121375744 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5001b070 con 0x55c69d751330 2024-04-21T08:42:38.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.556+0000 7f4a42ffd640 1 -- 172.21.15.86:0/1121375744 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a50013040 con 0x55c69d751330 2024-04-21T08:42:38.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.556+0000 7f4a42ffd640 1 -- 172.21.15.86:0/1121375744 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a50017400 con 0x55c69d751330 2024-04-21T08:42:38.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.557+0000 7f4a61e2b740 1 -- 172.21.15.86:0/1121375744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c69d751330 msgr2=0x55c69d751710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.557+0000 7f4a5d7dc640 1 --2- 172.21.15.86:0/267833783 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4a480a3d20 0x7f4a480a61e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4a5001d000 tx=0x7f4a5001b000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:38.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.557+0000 7f4a61e2b740 1 --2- 172.21.15.86:0/1121375744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c69d751330 0x55c69d751710 secure :-1 s=READY pgs=1067 cs=0 l=1 rev1=1 crypto rx=0x7f4a50012710 tx=0x7f4a50012bd0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.557+0000 7f4a61e2b740 1 -- 172.21.15.86:0/1121375744 shutdown_connections 2024-04-21T08:42:38.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.557+0000 7f4a61e2b740 1 --2- 172.21.15.86:0/1121375744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c69d75ae40 0x55c69d75b2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.557+0000 7f4a61e2b740 1 --2- 172.21.15.86:0/1121375744 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c69d751ce0 0x55c69d75a820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.557+0000 7f4a61e2b740 1 --2- 172.21.15.86:0/1121375744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c69d751330 0x55c69d751710 unknown :-1 s=CLOSED pgs=1067 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.557+0000 7f4a61e2b740 1 -- 172.21.15.86:0/1121375744 >> 172.21.15.86:0/1121375744 conn(0x55c69d74bba0 msgr2=0x55c69d74bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:38.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.557+0000 7f4a61e2b740 1 -- 172.21.15.86:0/1121375744 shutdown_connections 2024-04-21T08:42:38.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.557+0000 7f4a61e2b740 1 -- 172.21.15.86:0/1121375744 wait complete. 2024-04-21T08:42:38.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.557+0000 7f4a61e2b740 1 -- start start 2024-04-21T08:42:38.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.557+0000 7f4a61e2b740 1 Processor -- start 2024-04-21T08:42:38.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.558+0000 7f4a61e2b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c69d751330 0x55c69d7612a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.558+0000 7f4a61e2b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c69d751ce0 0x55c69d75e630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.558+0000 7f4a61e2b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c69d75ae40 0x55c69d75ec40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.558+0000 7f4a61e2b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c69d761dc0 con 0x55c69d75ae40 2024-04-21T08:42:38.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.558+0000 7f4a61e2b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c69d66f500 con 0x55c69d751330 2024-04-21T08:42:38.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.558+0000 7f4a61e2b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c69d761f40 con 0x55c69d751ce0 2024-04-21T08:42:38.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.558+0000 7f4a61e2b740 10 client.0 _ll_register_callbacks cb 0x55c69d63db80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:38.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.558+0000 7f4a5d7dc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c69d75ae40 0x55c69d75ec40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.558+0000 7f4a5d7dc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c69d75ae40 0x55c69d75ec40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33294/0 (socket says 172.21.15.86:33294) 2024-04-21T08:42:38.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.558+0000 7f4a5d7dc640 1 -- 172.21.15.86:0/2504742077 learned_addr learned my addr 172.21.15.86:0/2504742077 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:38.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.558+0000 7f4a5cfdb640 1 --2- 172.21.15.86:0/2504742077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c69d751330 0x55c69d7612a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.558+0000 7f4a5d7dc640 1 -- 172.21.15.86:0/2504742077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c69d751ce0 msgr2=0x55c69d75e630 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:42:38.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.558+0000 7f4a5d7dc640 1 --2- 172.21.15.86:0/2504742077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c69d751ce0 0x55c69d75e630 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.558+0000 7f4a5d7dc640 1 -- 172.21.15.86:0/2504742077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c69d751330 msgr2=0x55c69d7612a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:38.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.558+0000 7f4a5d7dc640 1 --2- 172.21.15.86:0/2504742077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c69d751330 0x55c69d7612a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.558+0000 7f4a5d7dc640 1 -- 172.21.15.86:0/2504742077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c69d3c83f0 con 0x55c69d75ae40 2024-04-21T08:42:38.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.558+0000 7f4a5d7dc640 1 --2- 172.21.15.86:0/2504742077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c69d75ae40 0x55c69d75ec40 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f4a50012250 tx=0x7f4a50020ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=409f5094344b565 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:38.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.558+0000 7f4a37fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:38.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.558+0000 7f4a37fff640 1 -- 172.21.15.86:0/2504742077 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a5002e020 con 0x55c69d75ae40 2024-04-21T08:42:38.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.558+0000 7f4a37fff640 1 -- 172.21.15.86:0/2504742077 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f4a500175e0 con 0x55c69d75ae40 2024-04-21T08:42:38.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.558+0000 7f4a61e2b740 1 -- 172.21.15.86:0/2504742077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a50013040 con 0x55c69d75ae40 2024-04-21T08:42:38.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.558+0000 7f4a37fff640 1 -- 172.21.15.86:0/2504742077 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4a50039050 con 0x55c69d75ae40 2024-04-21T08:42:38.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.558+0000 7f4a61e2b740 10 client.17376 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:38.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.558+0000 7f4a61e2b740 1 -- 172.21.15.86:0/2504742077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c69d4927d0 con 0x55c69d75ae40 2024-04-21T08:42:38.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.559+0000 7f4a37fff640 1 -- 172.21.15.86:0/2504742077 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a50013040 con 0x55c69d75ae40 2024-04-21T08:42:38.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.559+0000 7f4a37fff640 1 -- 172.21.15.86:0/2504742077 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4a5002a440 con 0x55c69d75ae40 2024-04-21T08:42:38.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.559+0000 7f4a61e2b740 10 client.17376 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:38.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.559+0000 7f4a61e2b740 10 client.17376 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:42:38.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.559+0000 7f4a61e2b740 20 client.17376 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:38.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.559+0000 7f4a61e2b740 1 --2- 172.21.15.86:0/2504742077 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55c69d76f060 0x55c69d771450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.559+0000 7f4a61e2b740 4 client.17376 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:38.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.559+0000 7f4a61e2b740 1 -- 172.21.15.86:0/2504742077 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c69d655930 con 0x55c69d76f060 2024-04-21T08:42:38.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.559+0000 7f4a5cfdb640 1 --2- 172.21.15.86:0/2504742077 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55c69d76f060 0x55c69d771450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.560+0000 7f4a5cfdb640 1 --2- 172.21.15.86:0/2504742077 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55c69d76f060 0x55c69d771450 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=f9646573f6f3817b server_cookie=7a2b7f069294d0f0 in_seq=0 out_seq=0 2024-04-21T08:42:38.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.560+0000 7f4a37fff640 10 client.17376 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:42:38.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.560+0000 7f4a37fff640 1 -- 172.21.15.86:0/2504742077 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c69d655930 con 0x55c69d76f060 2024-04-21T08:42:38.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.560+0000 7f4a37fff640 10 client.17376 handle_command_reply: tid=0 2024-04-21T08:42:38.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.560+0000 7f4a61e2b740 1 client.17376 shutdown 2024-04-21T08:42:38.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.560+0000 7f4a61e2b740 1 -- 172.21.15.86:0/2504742077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c69d75ae40 msgr2=0x55c69d75ec40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.560+0000 7f4a61e2b740 1 --2- 172.21.15.86:0/2504742077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c69d75ae40 0x55c69d75ec40 secure :-1 s=READY pgs=1068 cs=0 l=1 rev1=1 crypto rx=0x7f4a50012250 tx=0x7f4a50020ba0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.560+0000 7f4a61e2b740 1 -- 172.21.15.86:0/2504742077 shutdown_connections 2024-04-21T08:42:38.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.560+0000 7f4a61e2b740 1 --2- 172.21.15.86:0/2504742077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c69d75ae40 0x55c69d75ec40 unknown :-1 s=CLOSED pgs=1068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.560+0000 7f4a61e2b740 1 --2- 172.21.15.86:0/2504742077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c69d751ce0 0x55c69d75e630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.560+0000 7f4a61e2b740 1 --2- 172.21.15.86:0/2504742077 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55c69d76f060 0x55c69d771450 crc :-1 s=READY pgs=46 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.560+0000 7f4a61e2b740 1 --2- 172.21.15.86:0/2504742077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c69d751330 0x55c69d7612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.560+0000 7f4a61e2b740 1 -- 172.21.15.86:0/2504742077 >> 172.21.15.86:0/2504742077 conn(0x55c69d74bba0 msgr2=0x55c69d74d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:38.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.560+0000 7f4a37fff640 0 client.17376 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:42:38.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.561+0000 7f4a61e2b740 1 -- 172.21.15.86:0/2504742077 shutdown_connections 2024-04-21T08:42:38.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.561+0000 7f4a61e2b740 1 -- 172.21.15.86:0/2504742077 wait complete. 2024-04-21T08:42:38.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.561+0000 7f4a61e2b740 20 client.17376 trim_cache size 0 max 16384 2024-04-21T08:42:38.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.574+0000 7f4a61e2b740 1 Processor -- start 2024-04-21T08:42:38.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.574+0000 7f4a61e2b740 1 -- start start 2024-04-21T08:42:38.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.574+0000 7f4a61e2b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c69d751330 0x55c69d76acc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.574+0000 7f4a61e2b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c69d751ce0 0x55c69d760d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.574+0000 7f4a61e2b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c69d75ae40 0x55c69d7612a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.574+0000 7f4a61e2b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4a581ad030 con 0x55c69d751ce0 2024-04-21T08:42:38.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.574+0000 7f4a61e2b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4a581aeb70 con 0x55c69d75ae40 2024-04-21T08:42:38.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.574+0000 7f4a61e2b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4a581ad700 con 0x55c69d751330 2024-04-21T08:42:38.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.574+0000 7f4a57fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c69d751ce0 0x55c69d760d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.574+0000 7f4a57fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c69d751ce0 0x55c69d760d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33300/0 (socket says 172.21.15.86:33300) 2024-04-21T08:42:38.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.574+0000 7f4a57fff640 1 -- 172.21.15.86:0/204522252 learned_addr learned my addr 172.21.15.86:0/204522252 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:38.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.574+0000 7f4a5d7dc640 1 --2- 172.21.15.86:0/204522252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c69d75ae40 0x55c69d7612a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.574+0000 7f4a57fff640 1 -- 172.21.15.86:0/204522252 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c69d751330 msgr2=0x55c69d76acc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:42:38.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.574+0000 7f4a57fff640 1 --2- 172.21.15.86:0/204522252 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c69d751330 0x55c69d76acc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.574+0000 7f4a57fff640 1 -- 172.21.15.86:0/204522252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c69d75ae40 msgr2=0x55c69d7612a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:38.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.574+0000 7f4a57fff640 1 --2- 172.21.15.86:0/204522252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c69d75ae40 0x55c69d7612a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.574+0000 7f4a57fff640 1 -- 172.21.15.86:0/204522252 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c69d76b830 con 0x55c69d751ce0 2024-04-21T08:42:38.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.574+0000 7f4a57fff640 1 --2- 172.21.15.86:0/204522252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c69d751ce0 0x55c69d760d60 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f4a440baf60 tx=0x7f4a440037d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=940afc0eea2aecbc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:38.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.574+0000 7f4a427fc640 1 -- 172.21.15.86:0/204522252 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a440be460 con 0x55c69d751ce0 2024-04-21T08:42:38.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.575+0000 7f4a427fc640 1 -- 172.21.15.86:0/204522252 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a440035f0 con 0x55c69d751ce0 2024-04-21T08:42:38.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.575+0000 7f4a427fc640 1 -- 172.21.15.86:0/204522252 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a440bce90 con 0x55c69d751ce0 2024-04-21T08:42:38.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.575+0000 7f4a61e2b740 1 -- 172.21.15.86:0/204522252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c69d751ce0 msgr2=0x55c69d760d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.575+0000 7f4a61e2b740 1 --2- 172.21.15.86:0/204522252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c69d751ce0 0x55c69d760d60 secure :-1 s=READY pgs=1069 cs=0 l=1 rev1=1 crypto rx=0x7f4a440baf60 tx=0x7f4a440037d0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.575+0000 7f4a61e2b740 1 -- 172.21.15.86:0/204522252 shutdown_connections 2024-04-21T08:42:38.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.575+0000 7f4a61e2b740 1 --2- 172.21.15.86:0/204522252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c69d75ae40 0x55c69d7612a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.575+0000 7f4a61e2b740 1 --2- 172.21.15.86:0/204522252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c69d751ce0 0x55c69d760d60 unknown :-1 s=CLOSED pgs=1069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.575+0000 7f4a61e2b740 1 --2- 172.21.15.86:0/204522252 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c69d751330 0x55c69d76acc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.575+0000 7f4a61e2b740 1 -- 172.21.15.86:0/204522252 >> 172.21.15.86:0/204522252 conn(0x55c69d74bba0 msgr2=0x55c69d75d660 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:38.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.575+0000 7f4a61e2b740 1 -- 172.21.15.86:0/204522252 shutdown_connections 2024-04-21T08:42:38.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.575+0000 7f4a61e2b740 1 -- 172.21.15.86:0/204522252 wait complete. 2024-04-21T08:42:38.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.575+0000 7f4a61e2b740 1 -- start start 2024-04-21T08:42:38.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.575+0000 7f4a61e2b740 1 Processor -- start 2024-04-21T08:42:38.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.576+0000 7f4a61e2b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c69d751330 0x55c69d767660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.576+0000 7f4a61e2b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c69d751ce0 0x55c69d760110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.576+0000 7f4a61e2b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c69d75ae40 0x55c69d760720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.576+0000 7f4a61e2b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4a581ae4a0 con 0x55c69d751ce0 2024-04-21T08:42:38.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.576+0000 7f4a61e2b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4a581addd0 con 0x55c69d75ae40 2024-04-21T08:42:38.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.576+0000 7f4a61e2b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4a581aaff0 con 0x55c69d751330 2024-04-21T08:42:38.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.576+0000 7f4a61e2b740 10 client.0 _ll_register_callbacks cb 0x7f4a5000f570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:38.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.576+0000 7f4a57fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c69d751ce0 0x55c69d760110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.576+0000 7f4a57fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c69d751ce0 0x55c69d760110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33316/0 (socket says 172.21.15.86:33316) 2024-04-21T08:42:38.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.576+0000 7f4a57fff640 1 -- 172.21.15.86:0/3355358511 learned_addr learned my addr 172.21.15.86:0/3355358511 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:38.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.576+0000 7f4a5d7dc640 1 --2- 172.21.15.86:0/3355358511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c69d75ae40 0x55c69d760720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.576+0000 7f4a57fff640 1 -- 172.21.15.86:0/3355358511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c69d751330 msgr2=0x55c69d767660 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:42:38.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.576+0000 7f4a57fff640 1 --2- 172.21.15.86:0/3355358511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c69d751330 0x55c69d767660 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.576+0000 7f4a57fff640 1 -- 172.21.15.86:0/3355358511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c69d75ae40 msgr2=0x55c69d760720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:38.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.576+0000 7f4a57fff640 1 --2- 172.21.15.86:0/3355358511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c69d75ae40 0x55c69d760720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.576+0000 7f4a57fff640 1 -- 172.21.15.86:0/3355358511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c69d760f20 con 0x55c69d751ce0 2024-04-21T08:42:38.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.576+0000 7f4a5d7dc640 1 --2- 172.21.15.86:0/3355358511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c69d75ae40 0x55c69d760720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:38.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.576+0000 7f4a57fff640 1 --2- 172.21.15.86:0/3355358511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c69d751ce0 0x55c69d760110 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f4a440be800 tx=0x7f4a440f0290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=121604257e7cb78b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:38.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.576+0000 7f4a37fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:38.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.576+0000 7f4a37fff640 1 -- 172.21.15.86:0/3355358511 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a440d7070 con 0x55c69d751ce0 2024-04-21T08:42:38.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.576+0000 7f4a37fff640 1 -- 172.21.15.86:0/3355358511 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f4a440f0420 con 0x55c69d751ce0 2024-04-21T08:42:38.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.576+0000 7f4a61e2b740 1 -- 172.21.15.86:0/3355358511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c69d6c3c10 con 0x55c69d751ce0 2024-04-21T08:42:38.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.577+0000 7f4a37fff640 1 -- 172.21.15.86:0/3355358511 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4a440f0620 con 0x55c69d751ce0 2024-04-21T08:42:38.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.577+0000 7f4a61e2b740 10 client.17382 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:38.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.577+0000 7f4a61e2b740 1 -- 172.21.15.86:0/3355358511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c69d7d5420 con 0x55c69d751ce0 2024-04-21T08:42:38.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.577+0000 7f4a37fff640 1 -- 172.21.15.86:0/3355358511 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c69d6c3c10 con 0x55c69d751ce0 2024-04-21T08:42:38.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.577+0000 7f4a37fff640 1 -- 172.21.15.86:0/3355358511 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4a440c72e0 con 0x55c69d751ce0 2024-04-21T08:42:38.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.577+0000 7f4a61e2b740 10 client.17382 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:38.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.577+0000 7f4a61e2b740 10 client.17382 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:42:38.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.577+0000 7f4a61e2b740 20 client.17382 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:38.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.577+0000 7f4a61e2b740 1 --2- 172.21.15.86:0/3355358511 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55c69d7d58d0 0x55c69d7d7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.577+0000 7f4a61e2b740 4 client.17382 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:42:38.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.577+0000 7f4a61e2b740 1 -- 172.21.15.86:0/3355358511 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c69d6bda80 con 0x55c69d7d58d0 2024-04-21T08:42:38.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.579+0000 7f4a5cfdb640 1 --2- 172.21.15.86:0/3355358511 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55c69d7d58d0 0x55c69d7d7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.582+0000 7f4a5cfdb640 1 --2- 172.21.15.86:0/3355358511 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55c69d7d58d0 0x55c69d7d7cf0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=709350656cbae98f server_cookie=d3f7a90b011ae818 in_seq=0 out_seq=0 2024-04-21T08:42:38.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.582+0000 7f4a37fff640 10 client.17382 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:42:38.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.583+0000 7f4a37fff640 1 -- 172.21.15.86:0/3355358511 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c69d6bda80 con 0x55c69d7d58d0 2024-04-21T08:42:38.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.583+0000 7f4a37fff640 10 client.17382 handle_command_reply: tid=0 2024-04-21T08:42:38.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.583+0000 7f4a61e2b740 1 client.17382 shutdown 2024-04-21T08:42:38.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.583+0000 7f4a61e2b740 1 -- 172.21.15.86:0/3355358511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c69d751ce0 msgr2=0x55c69d760110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.583+0000 7f4a61e2b740 1 --2- 172.21.15.86:0/3355358511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c69d751ce0 0x55c69d760110 secure :-1 s=READY pgs=1070 cs=0 l=1 rev1=1 crypto rx=0x7f4a440be800 tx=0x7f4a440f0290 comp rx=0 tx=0).stop 2024-04-21T08:42:38.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.583+0000 7f4a61e2b740 1 -- 172.21.15.86:0/3355358511 shutdown_connections 2024-04-21T08:42:38.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.583+0000 7f4a61e2b740 1 --2- 172.21.15.86:0/3355358511 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55c69d7d58d0 0x55c69d7d7cf0 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.583+0000 7f4a61e2b740 1 --2- 172.21.15.86:0/3355358511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c69d75ae40 0x55c69d760720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.583+0000 7f4a37fff640 0 client.17382 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:42:38.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.583+0000 7f4a61e2b740 1 --2- 172.21.15.86:0/3355358511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c69d751ce0 0x55c69d760110 unknown :-1 s=CLOSED pgs=1070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.583+0000 7f4a61e2b740 1 --2- 172.21.15.86:0/3355358511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c69d751330 0x55c69d767660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.583+0000 7f4a61e2b740 1 -- 172.21.15.86:0/3355358511 >> 172.21.15.86:0/3355358511 conn(0x55c69d74bba0 msgr2=0x55c69d74a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:38.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.583+0000 7f4a61e2b740 1 -- 172.21.15.86:0/3355358511 shutdown_connections 2024-04-21T08:42:38.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.583+0000 7f4a61e2b740 1 -- 172.21.15.86:0/3355358511 wait complete. 2024-04-21T08:42:38.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.583+0000 7f4a61e2b740 20 client.17382 trim_cache size 0 max 16384 2024-04-21T08:42:38.585 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:42:38.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.584+0000 7f4a5f23f640 1 -- 172.21.15.86:0/267833783 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4a480a3d20 msgr2=0x7f4a480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.584+0000 7f4a5f23f640 1 --2- 172.21.15.86:0/267833783 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4a480a3d20 0x7f4a480a61e0 secure :-1 s=READY pgs=164 cs=0 l=1 rev1=1 crypto rx=0x7f4a5001d000 tx=0x7f4a5001b000 comp rx=0 tx=0).stop 2024-04-21T08:42:38.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.584+0000 7f4a5f23f640 1 -- 172.21.15.86:0/267833783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a58106160 msgr2=0x7f4a5819e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.584+0000 7f4a5f23f640 1 --2- 172.21.15.86:0/267833783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a58106160 0x7f4a5819e930 secure :-1 s=READY pgs=1066 cs=0 l=1 rev1=1 crypto rx=0x7f4a44000c00 tx=0x7f4a4402c800 comp rx=0 tx=0).stop 2024-04-21T08:42:38.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.584+0000 7f4a5f23f640 1 -- 172.21.15.86:0/267833783 shutdown_connections 2024-04-21T08:42:38.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.584+0000 7f4a5f23f640 1 --2- 172.21.15.86:0/267833783 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4a480a3d20 0x7f4a480a61e0 unknown :-1 s=CLOSED pgs=164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.584+0000 7f4a5f23f640 1 --2- 172.21.15.86:0/267833783 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a581985a0 0x7f4a58198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.584+0000 7f4a5f23f640 1 --2- 172.21.15.86:0/267833783 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a58106160 0x7f4a5819e930 unknown :-1 s=CLOSED pgs=1066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.584+0000 7f4a5f23f640 1 --2- 172.21.15.86:0/267833783 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a58101090 0x7f4a5819e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.584+0000 7f4a5f23f640 1 -- 172.21.15.86:0/267833783 >> 172.21.15.86:0/267833783 conn(0x7f4a58077f50 msgr2=0x7f4a580ff060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:38.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.584+0000 7f4a5f23f640 1 -- 172.21.15.86:0/267833783 shutdown_connections 2024-04-21T08:42:38.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.584+0000 7f4a5f23f640 1 -- 172.21.15.86:0/267833783 wait complete. 2024-04-21T08:42:38.593 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:42:38.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.694+0000 7f142fb11640 1 -- 172.21.15.86:0/3821865432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1428107270 msgr2=0x7f142810ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.694+0000 7f142fb11640 1 --2- 172.21.15.86:0/3821865432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1428107270 0x7f142810ee40 secure :-1 s=READY pgs=1071 cs=0 l=1 rev1=1 crypto rx=0x7f1420009ca0 tx=0x7f142002c120 comp rx=0 tx=0).stop 2024-04-21T08:42:38.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.694+0000 7f142fb11640 1 -- 172.21.15.86:0/3821865432 shutdown_connections 2024-04-21T08:42:38.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.694+0000 7f142fb11640 1 --2- 172.21.15.86:0/3821865432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1428107270 0x7f142810ee40 unknown :-1 s=CLOSED pgs=1071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.694+0000 7f142fb11640 1 --2- 172.21.15.86:0/3821865432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1428102b50 0x7f1428106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.694+0000 7f142fb11640 1 --2- 172.21.15.86:0/3821865432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14281021a0 0x7f1428102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.694+0000 7f142fb11640 1 -- 172.21.15.86:0/3821865432 >> 172.21.15.86:0/3821865432 conn(0x7f14280fd9c0 msgr2=0x7f14280ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:38.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.694+0000 7f142fb11640 1 -- 172.21.15.86:0/3821865432 shutdown_connections 2024-04-21T08:42:38.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.694+0000 7f142fb11640 1 -- 172.21.15.86:0/3821865432 wait complete. 2024-04-21T08:42:38.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.695+0000 7f142fb11640 1 Processor -- start 2024-04-21T08:42:38.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.695+0000 7f142fb11640 1 -- start start 2024-04-21T08:42:38.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.695+0000 7f142fb11640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14281021a0 0x7f1428198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.695+0000 7f142fb11640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1428107270 0x7f1428198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.695+0000 7f142fb11640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f142819cfc0 0x7f142819d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.695+0000 7f142fb11640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1428112250 con 0x7f1428107270 2024-04-21T08:42:38.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.695+0000 7f142fb11640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f14281120d0 con 0x7f14281021a0 2024-04-21T08:42:38.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.695+0000 7f142fb11640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f14281123d0 con 0x7f142819cfc0 2024-04-21T08:42:38.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.695+0000 7f142d0ac640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1428107270 0x7f1428198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.695+0000 7f142d0ac640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1428107270 0x7f1428198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33336/0 (socket says 172.21.15.86:33336) 2024-04-21T08:42:38.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.695+0000 7f142d0ac640 1 -- 172.21.15.86:0/91772075 learned_addr learned my addr 172.21.15.86:0/91772075 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:38.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.695+0000 7f142d8ad640 1 --2- 172.21.15.86:0/91772075 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14281021a0 0x7f1428198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.695+0000 7f142d0ac640 1 -- 172.21.15.86:0/91772075 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f142819cfc0 msgr2=0x7f142819d470 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T08:42:38.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.695+0000 7f142d0ac640 1 --2- 172.21.15.86:0/91772075 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f142819cfc0 0x7f142819d470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.695+0000 7f142d0ac640 1 -- 172.21.15.86:0/91772075 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14281021a0 msgr2=0x7f1428198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.695+0000 7f142d0ac640 1 --2- 172.21.15.86:0/91772075 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14281021a0 0x7f1428198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.695+0000 7f142d0ac640 1 -- 172.21.15.86:0/91772075 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f142819dab0 con 0x7f1428107270 2024-04-21T08:42:38.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.695+0000 7f142d8ad640 1 --2- 172.21.15.86:0/91772075 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14281021a0 0x7f1428198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:38.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.696+0000 7f142d0ac640 1 --2- 172.21.15.86:0/91772075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1428107270 0x7f1428198c90 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f141800c970 tx=0x7f141800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:38.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.696+0000 7f141effd640 1 -- 172.21.15.86:0/91772075 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1418014070 con 0x7f1428107270 2024-04-21T08:42:38.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.696+0000 7f141effd640 1 -- 172.21.15.86:0/91772075 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1418004440 con 0x7f1428107270 2024-04-21T08:42:38.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.696+0000 7f142fb11640 1 -- 172.21.15.86:0/91772075 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f142819dda0 con 0x7f1428107270 2024-04-21T08:42:38.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.696+0000 7f141effd640 1 -- 172.21.15.86:0/91772075 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1418003e50 con 0x7f1428107270 2024-04-21T08:42:38.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.696+0000 7f142fb11640 1 -- 172.21.15.86:0/91772075 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14281a5590 con 0x7f1428107270 2024-04-21T08:42:38.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.698+0000 7f141effd640 1 -- 172.21.15.86:0/91772075 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1418017070 con 0x7f1428107270 2024-04-21T08:42:38.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.698+0000 7f14327d6740 1 Processor -- start 2024-04-21T08:42:38.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.699+0000 7f14327d6740 1 -- start start 2024-04-21T08:42:38.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.699+0000 7f14327d6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561f02d814d0 0x561f02d818b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.699+0000 7f14327d6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f02d81e80 0x561f02d8a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.699+0000 7f14327d6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f02d8aa00 0x561f02d8ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.699+0000 7f14327d6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561f02ce61f0 con 0x561f02d8aa00 2024-04-21T08:42:38.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.699+0000 7f14327d6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x561f02c834b0 con 0x561f02d814d0 2024-04-21T08:42:38.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.699+0000 7f14327d6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561f02ce21b0 con 0x561f02d81e80 2024-04-21T08:42:38.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.699+0000 7f142d0ac640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f02d8aa00 0x561f02d8ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.699+0000 7f142d0ac640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f02d8aa00 0x561f02d8ce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33338/0 (socket says 172.21.15.86:33338) 2024-04-21T08:42:38.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.699+0000 7f142d0ac640 1 -- 172.21.15.86:0/2156355482 learned_addr learned my addr 172.21.15.86:0/2156355482 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:38.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.699+0000 7f141effd640 1 --2- 172.21.15.86:0/91772075 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f14140a3ec0 0x7f14140a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.699+0000 7f142e0ae640 1 --2- 172.21.15.86:0/2156355482 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561f02d814d0 0x561f02d818b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.699+0000 7f142e0ae640 1 --2- 172.21.15.86:0/91772075 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f14140a3ec0 0x7f14140a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.699+0000 7f142d8ad640 1 --2- 172.21.15.86:0/2156355482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f02d81e80 0x561f02d8a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.699+0000 7f142d0ac640 1 -- 172.21.15.86:0/2156355482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f02d81e80 msgr2=0x561f02d8a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:38.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.699+0000 7f142d0ac640 1 --2- 172.21.15.86:0/2156355482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f02d81e80 0x561f02d8a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.699+0000 7f142d0ac640 1 -- 172.21.15.86:0/2156355482 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561f02d814d0 msgr2=0x561f02d818b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:38.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.699+0000 7f142d0ac640 1 --2- 172.21.15.86:0/2156355482 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561f02d814d0 0x561f02d818b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.699+0000 7f141effd640 1 -- 172.21.15.86:0/91772075 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f14180d07b0 con 0x7f1428107270 2024-04-21T08:42:38.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.699+0000 7f142d0ac640 1 -- 172.21.15.86:0/2156355482 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f02c751b0 con 0x561f02d8aa00 2024-04-21T08:42:38.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.699+0000 7f142d8ad640 1 --2- 172.21.15.86:0/2156355482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f02d81e80 0x561f02d8a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:38.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.699+0000 7f142d0ac640 1 --2- 172.21.15.86:0/2156355482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f02d8aa00 0x561f02d8ce50 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f1418098840 tx=0x7f1418000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6148c3785b3a617 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:38.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.700+0000 7f142e0ae640 1 --2- 172.21.15.86:0/91772075 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f14140a3ec0 0x7f14140a6380 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1420037040 tx=0x7f142002c120 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:38.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.700+0000 7f1413fff640 1 -- 172.21.15.86:0/2156355482 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f141800de20 con 0x561f02d8aa00 2024-04-21T08:42:38.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.700+0000 7f1413fff640 1 -- 172.21.15.86:0/2156355482 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f141809c3e0 con 0x561f02d8aa00 2024-04-21T08:42:38.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.700+0000 7f1413fff640 1 -- 172.21.15.86:0/2156355482 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14180cde10 con 0x561f02d8aa00 2024-04-21T08:42:38.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.700+0000 7f14327d6740 1 -- 172.21.15.86:0/2156355482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f02d8aa00 msgr2=0x561f02d8ce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.700+0000 7f14327d6740 1 --2- 172.21.15.86:0/2156355482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f02d8aa00 0x561f02d8ce50 secure :-1 s=READY pgs=1073 cs=0 l=1 rev1=1 crypto rx=0x7f1418098840 tx=0x7f1418000bc0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.700+0000 7f14327d6740 1 -- 172.21.15.86:0/2156355482 shutdown_connections 2024-04-21T08:42:38.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.700+0000 7f14327d6740 1 --2- 172.21.15.86:0/2156355482 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f02d8aa00 0x561f02d8ce50 unknown :-1 s=CLOSED pgs=1073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.700+0000 7f14327d6740 1 --2- 172.21.15.86:0/2156355482 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f02d81e80 0x561f02d8a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.700+0000 7f14327d6740 1 --2- 172.21.15.86:0/2156355482 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561f02d814d0 0x561f02d818b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.700+0000 7f14327d6740 1 -- 172.21.15.86:0/2156355482 >> 172.21.15.86:0/2156355482 conn(0x561f02d7bba0 msgr2=0x561f02d7bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:38.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.700+0000 7f14327d6740 1 -- 172.21.15.86:0/2156355482 shutdown_connections 2024-04-21T08:42:38.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.701+0000 7f14327d6740 1 -- 172.21.15.86:0/2156355482 wait complete. 2024-04-21T08:42:38.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.701+0000 7f14327d6740 1 -- start start 2024-04-21T08:42:38.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.701+0000 7f14327d6740 1 Processor -- start 2024-04-21T08:42:38.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.701+0000 7f14327d6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f02d814d0 0x561f02d8a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.701+0000 7f14327d6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561f02d81e80 0x561f02d87710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.701+0000 7f14327d6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f02d8aa00 0x561f02d87d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.701+0000 7f14327d6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561f02d8d910 con 0x561f02d814d0 2024-04-21T08:42:38.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.701+0000 7f14327d6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x561f02c9f500 con 0x561f02d81e80 2024-04-21T08:42:38.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.701+0000 7f14327d6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561f02d8da90 con 0x561f02d8aa00 2024-04-21T08:42:38.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.701+0000 7f14327d6740 10 client.0 _ll_register_callbacks cb 0x561f02c6db80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:38.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.701+0000 7f142d8ad640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f02d814d0 0x561f02d8a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.701+0000 7f142d8ad640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f02d814d0 0x561f02d8a380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33340/0 (socket says 172.21.15.86:33340) 2024-04-21T08:42:38.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.701+0000 7f142d8ad640 1 -- 172.21.15.86:0/2550314969 learned_addr learned my addr 172.21.15.86:0/2550314969 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:38.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.701+0000 7f142d0ac640 1 --2- 172.21.15.86:0/2550314969 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561f02d81e80 0x561f02d87710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.702+0000 7f142d8ad640 1 -- 172.21.15.86:0/2550314969 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f02d8aa00 msgr2=0x561f02d87d20 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:42:38.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.702+0000 7f142d8ad640 1 --2- 172.21.15.86:0/2550314969 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f02d8aa00 0x561f02d87d20 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.702+0000 7f142d8ad640 1 -- 172.21.15.86:0/2550314969 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561f02d81e80 msgr2=0x561f02d87710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:38.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.702+0000 7f142d8ad640 1 --2- 172.21.15.86:0/2550314969 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561f02d81e80 0x561f02d87710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.702+0000 7f142d8ad640 1 -- 172.21.15.86:0/2550314969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f029f83f0 con 0x561f02d814d0 2024-04-21T08:42:38.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.702+0000 7f142d8ad640 1 --2- 172.21.15.86:0/2550314969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f02d814d0 0x561f02d8a380 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f142400e240 tx=0x7f142400e700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1252247054646e77 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:38.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.702+0000 7f14117fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:38.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.702+0000 7f14117fa640 1 -- 172.21.15.86:0/2550314969 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1424017020 con 0x561f02d814d0 2024-04-21T08:42:38.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.702+0000 7f14117fa640 1 -- 172.21.15.86:0/2550314969 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f1424013610 con 0x561f02d814d0 2024-04-21T08:42:38.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.702+0000 7f14327d6740 1 -- 172.21.15.86:0/2550314969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f141809c3e0 con 0x561f02d814d0 2024-04-21T08:42:38.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.702+0000 7f14117fa640 1 -- 172.21.15.86:0/2550314969 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1424022050 con 0x561f02d814d0 2024-04-21T08:42:38.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.702+0000 7f14327d6740 10 client.17394 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:38.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.702+0000 7f14327d6740 1 -- 172.21.15.86:0/2550314969 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561f02ac27d0 con 0x561f02d814d0 2024-04-21T08:42:38.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.702+0000 7f14117fa640 1 -- 172.21.15.86:0/2550314969 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f141809c3e0 con 0x561f02d814d0 2024-04-21T08:42:38.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.703+0000 7f14117fa640 1 -- 172.21.15.86:0/2550314969 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1424012050 con 0x561f02d814d0 2024-04-21T08:42:38.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.703+0000 7f14327d6740 10 client.17394 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:38.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.703+0000 7f14327d6740 10 client.17394 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:42:38.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.703+0000 7f14327d6740 20 client.17394 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:38.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.703+0000 7f14327d6740 1 --2- 172.21.15.86:0/2550314969 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x561f02d9ac00 0x561f02d9d090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.703+0000 7f14327d6740 4 client.17394 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:38.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.703+0000 7f14327d6740 1 -- 172.21.15.86:0/2550314969 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f02c85930 con 0x561f02d9ac00 2024-04-21T08:42:38.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.703+0000 7f142d0ac640 1 --2- 172.21.15.86:0/2550314969 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x561f02d9ac00 0x561f02d9d090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.703+0000 7f142d0ac640 1 --2- 172.21.15.86:0/2550314969 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x561f02d9ac00 0x561f02d9d090 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=2f3a9cfd25478be0 server_cookie=c3be1f9ddf26154c in_seq=0 out_seq=0 2024-04-21T08:42:38.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.703+0000 7f14117fa640 10 client.17394 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:42:38.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.704+0000 7f14117fa640 1 -- 172.21.15.86:0/2550314969 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561f02c85930 con 0x561f02d9ac00 2024-04-21T08:42:38.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.704+0000 7f14117fa640 10 client.17394 handle_command_reply: tid=0 2024-04-21T08:42:38.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.704+0000 7f14327d6740 1 client.17394 shutdown 2024-04-21T08:42:38.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.704+0000 7f14327d6740 1 -- 172.21.15.86:0/2550314969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f02d814d0 msgr2=0x561f02d8a380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.704+0000 7f14327d6740 1 --2- 172.21.15.86:0/2550314969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f02d814d0 0x561f02d8a380 secure :-1 s=READY pgs=1074 cs=0 l=1 rev1=1 crypto rx=0x7f142400e240 tx=0x7f142400e700 comp rx=0 tx=0).stop 2024-04-21T08:42:38.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.704+0000 7f14327d6740 1 -- 172.21.15.86:0/2550314969 shutdown_connections 2024-04-21T08:42:38.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.704+0000 7f14327d6740 1 --2- 172.21.15.86:0/2550314969 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x561f02d9ac00 0x561f02d9d090 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.704+0000 7f14327d6740 1 --2- 172.21.15.86:0/2550314969 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f02d8aa00 0x561f02d87d20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.704+0000 7f14327d6740 1 --2- 172.21.15.86:0/2550314969 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561f02d81e80 0x561f02d87710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.704+0000 7f14117fa640 0 client.17394 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:42:38.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.704+0000 7f14327d6740 1 --2- 172.21.15.86:0/2550314969 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f02d814d0 0x561f02d8a380 unknown :-1 s=CLOSED pgs=1074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.704+0000 7f14327d6740 1 -- 172.21.15.86:0/2550314969 >> 172.21.15.86:0/2550314969 conn(0x561f02d7bba0 msgr2=0x561f02d7d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:38.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.704+0000 7f14327d6740 1 -- 172.21.15.86:0/2550314969 shutdown_connections 2024-04-21T08:42:38.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.704+0000 7f14327d6740 1 -- 172.21.15.86:0/2550314969 wait complete. 2024-04-21T08:42:38.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.704+0000 7f14327d6740 20 client.17394 trim_cache size 0 max 16384 2024-04-21T08:42:38.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:38 smithi088 ceph-mon[31259]: pgmap v118: 129 pgs: 129 active+clean; 4.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-21T08:42:38.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.717+0000 7f14327d6740 1 Processor -- start 2024-04-21T08:42:38.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.717+0000 7f14327d6740 1 -- start start 2024-04-21T08:42:38.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.717+0000 7f14327d6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f02d814d0 0x561f02d967d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.717+0000 7f14327d6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561f02d81e80 0x561f02d82ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.717+0000 7f14327d6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f02d8aa00 0x561f02d89e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.717+0000 7f14327d6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14281af290 con 0x561f02d814d0 2024-04-21T08:42:38.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.717+0000 7f14327d6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f14281b0dd0 con 0x561f02d81e80 2024-04-21T08:42:38.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.717+0000 7f14327d6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f14281af960 con 0x561f02d8aa00 2024-04-21T08:42:38.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.718+0000 7f142d8ad640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f02d814d0 0x561f02d967d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.718+0000 7f142d8ad640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f02d814d0 0x561f02d967d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33346/0 (socket says 172.21.15.86:33346) 2024-04-21T08:42:38.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.718+0000 7f142d8ad640 1 -- 172.21.15.86:0/1252509557 learned_addr learned my addr 172.21.15.86:0/1252509557 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:38.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.718+0000 7f142d0ac640 1 --2- 172.21.15.86:0/1252509557 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561f02d81e80 0x561f02d82ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.718+0000 7f142d8ad640 1 -- 172.21.15.86:0/1252509557 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f02d8aa00 msgr2=0x561f02d89e40 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:42:38.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.718+0000 7f142d8ad640 1 --2- 172.21.15.86:0/1252509557 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f02d8aa00 0x561f02d89e40 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.718+0000 7f142d8ad640 1 -- 172.21.15.86:0/1252509557 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561f02d81e80 msgr2=0x561f02d82ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:38.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.718+0000 7f142d8ad640 1 --2- 172.21.15.86:0/1252509557 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561f02d81e80 0x561f02d82ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.718+0000 7f142d8ad640 1 -- 172.21.15.86:0/1252509557 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f02d81090 con 0x561f02d814d0 2024-04-21T08:42:38.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.718+0000 7f142d0ac640 1 --2- 172.21.15.86:0/1252509557 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561f02d81e80 0x561f02d82ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:38.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.718+0000 7f142d8ad640 1 --2- 172.21.15.86:0/1252509557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f02d814d0 0x561f02d967d0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f1424006a50 tx=0x7f1424009080 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ddda5674db78fc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:38.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.718+0000 7f14137fe640 1 -- 172.21.15.86:0/1252509557 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1424017070 con 0x561f02d814d0 2024-04-21T08:42:38.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.718+0000 7f14137fe640 1 -- 172.21.15.86:0/1252509557 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f142402b070 con 0x561f02d814d0 2024-04-21T08:42:38.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.718+0000 7f14137fe640 1 -- 172.21.15.86:0/1252509557 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142400f040 con 0x561f02d814d0 2024-04-21T08:42:38.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.719+0000 7f14327d6740 1 -- 172.21.15.86:0/1252509557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f02d814d0 msgr2=0x561f02d967d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.719+0000 7f14327d6740 1 --2- 172.21.15.86:0/1252509557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f02d814d0 0x561f02d967d0 secure :-1 s=READY pgs=1075 cs=0 l=1 rev1=1 crypto rx=0x7f1424006a50 tx=0x7f1424009080 comp rx=0 tx=0).stop 2024-04-21T08:42:38.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.719+0000 7f14327d6740 1 -- 172.21.15.86:0/1252509557 shutdown_connections 2024-04-21T08:42:38.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.719+0000 7f14327d6740 1 --2- 172.21.15.86:0/1252509557 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f02d8aa00 0x561f02d89e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.719+0000 7f14327d6740 1 --2- 172.21.15.86:0/1252509557 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561f02d81e80 0x561f02d82ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.719+0000 7f14327d6740 1 --2- 172.21.15.86:0/1252509557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f02d814d0 0x561f02d967d0 unknown :-1 s=CLOSED pgs=1075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.719+0000 7f14327d6740 1 -- 172.21.15.86:0/1252509557 >> 172.21.15.86:0/1252509557 conn(0x561f02d7bba0 msgr2=0x561f02d823d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:38.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.719+0000 7f14327d6740 1 -- 172.21.15.86:0/1252509557 shutdown_connections 2024-04-21T08:42:38.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.719+0000 7f14327d6740 1 -- 172.21.15.86:0/1252509557 wait complete. 2024-04-21T08:42:38.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.719+0000 7f14327d6740 1 -- start start 2024-04-21T08:42:38.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.719+0000 7f14327d6740 1 Processor -- start 2024-04-21T08:42:38.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.720+0000 7f14327d6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f02d814d0 0x561f02d93050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.720+0000 7f14327d6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f02d81e80 0x561f02d8e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.720+0000 7f14327d6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561f02d8aa00 0x561f02d8e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.720+0000 7f14327d6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14281b0700 con 0x561f02d81e80 2024-04-21T08:42:38.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.720+0000 7f14327d6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f14281b0030 con 0x561f02d8aa00 2024-04-21T08:42:38.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.720+0000 7f14327d6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f14281ad250 con 0x561f02d814d0 2024-04-21T08:42:38.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.720+0000 7f14327d6740 10 client.0 _ll_register_callbacks cb 0x7f1424009c80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:38.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.720+0000 7f142d0ac640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f02d81e80 0x561f02d8e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.720+0000 7f142d0ac640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f02d81e80 0x561f02d8e3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33350/0 (socket says 172.21.15.86:33350) 2024-04-21T08:42:38.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.720+0000 7f142d0ac640 1 -- 172.21.15.86:0/684245144 learned_addr learned my addr 172.21.15.86:0/684245144 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:38.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.720+0000 7f142d8ad640 1 --2- 172.21.15.86:0/684245144 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f02d814d0 0x561f02d93050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.720+0000 7f142d0ac640 1 -- 172.21.15.86:0/684245144 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f02d814d0 msgr2=0x561f02d93050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:38.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.720+0000 7f142d0ac640 1 --2- 172.21.15.86:0/684245144 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f02d814d0 0x561f02d93050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.720+0000 7f142d0ac640 1 -- 172.21.15.86:0/684245144 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561f02d8aa00 msgr2=0x561f02d8e900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:38.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.720+0000 7f142d0ac640 1 --2- 172.21.15.86:0/684245144 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561f02d8aa00 0x561f02d8e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.720+0000 7f142d0ac640 1 -- 172.21.15.86:0/684245144 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f02d8f130 con 0x561f02d81e80 2024-04-21T08:42:38.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.720+0000 7f142d8ad640 1 --2- 172.21.15.86:0/684245144 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f02d814d0 0x561f02d93050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:38.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.720+0000 7f142d0ac640 1 --2- 172.21.15.86:0/684245144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f02d81e80 0x561f02d8e3c0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f1418098db0 tx=0x7f14180076c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ef29606f4e75132 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:38.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.720+0000 7f14117fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:38.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.720+0000 7f14117fa640 1 -- 172.21.15.86:0/684245144 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1418098680 con 0x561f02d81e80 2024-04-21T08:42:38.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.720+0000 7f14117fa640 1 -- 172.21.15.86:0/684245144 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f14180035f0 con 0x561f02d81e80 2024-04-21T08:42:38.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.720+0000 7f14327d6740 1 -- 172.21.15.86:0/684245144 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f142402b070 con 0x561f02d81e80 2024-04-21T08:42:38.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.720+0000 7f14117fa640 1 -- 172.21.15.86:0/684245144 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f14180a5050 con 0x561f02d81e80 2024-04-21T08:42:38.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.720+0000 7f14327d6740 10 client.17400 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:38.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.720+0000 7f14327d6740 1 -- 172.21.15.86:0/684245144 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561f02e01160 con 0x561f02d81e80 2024-04-21T08:42:38.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.721+0000 7f14117fa640 1 -- 172.21.15.86:0/684245144 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f142402b070 con 0x561f02d81e80 2024-04-21T08:42:38.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.721+0000 7f14117fa640 1 -- 172.21.15.86:0/684245144 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f14180d0040 con 0x561f02d81e80 2024-04-21T08:42:38.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.721+0000 7f14327d6740 10 client.17400 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:38.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.721+0000 7f14327d6740 10 client.17400 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:42:38.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.721+0000 7f14327d6740 20 client.17400 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:38.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.721+0000 7f14327d6740 1 --2- 172.21.15.86:0/684245144 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x561f02e01510 0x561f02e03930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.721+0000 7f14327d6740 4 client.17400 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:42:38.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.721+0000 7f14327d6740 1 -- 172.21.15.86:0/684245144 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561f02cf3c10 con 0x561f02e01510 2024-04-21T08:42:38.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.721+0000 7f142d8ad640 1 --2- 172.21.15.86:0/684245144 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x561f02e01510 0x561f02e03930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.721+0000 7f142d8ad640 1 --2- 172.21.15.86:0/684245144 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x561f02e01510 0x561f02e03930 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f9e547bc90000474 server_cookie=a7af89d74c4c26e in_seq=0 out_seq=0 2024-04-21T08:42:38.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.721+0000 7f14117fa640 10 client.17400 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:42:38.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.721+0000 7f14117fa640 1 -- 172.21.15.86:0/684245144 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561f02cf3c10 con 0x561f02e01510 2024-04-21T08:42:38.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.721+0000 7f14117fa640 10 client.17400 handle_command_reply: tid=0 2024-04-21T08:42:38.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.722+0000 7f14327d6740 1 client.17400 shutdown 2024-04-21T08:42:38.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.722+0000 7f14327d6740 1 -- 172.21.15.86:0/684245144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f02d81e80 msgr2=0x561f02d8e3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.722+0000 7f14327d6740 1 --2- 172.21.15.86:0/684245144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f02d81e80 0x561f02d8e3c0 secure :-1 s=READY pgs=1076 cs=0 l=1 rev1=1 crypto rx=0x7f1418098db0 tx=0x7f14180076c0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.722+0000 7f14327d6740 1 -- 172.21.15.86:0/684245144 shutdown_connections 2024-04-21T08:42:38.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.722+0000 7f14327d6740 1 --2- 172.21.15.86:0/684245144 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x561f02e01510 0x561f02e03930 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.722+0000 7f14327d6740 1 --2- 172.21.15.86:0/684245144 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561f02d8aa00 0x561f02d8e900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.722+0000 7f14327d6740 1 --2- 172.21.15.86:0/684245144 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f02d81e80 0x561f02d8e3c0 unknown :-1 s=CLOSED pgs=1076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.722+0000 7f14117fa640 0 client.17400 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:42:38.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.722+0000 7f14327d6740 1 --2- 172.21.15.86:0/684245144 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f02d814d0 0x561f02d93050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.722+0000 7f14327d6740 1 -- 172.21.15.86:0/684245144 >> 172.21.15.86:0/684245144 conn(0x561f02d7bba0 msgr2=0x561f02d82c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:38.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.722+0000 7f14327d6740 1 -- 172.21.15.86:0/684245144 shutdown_connections 2024-04-21T08:42:38.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.722+0000 7f14327d6740 1 -- 172.21.15.86:0/684245144 wait complete. 2024-04-21T08:42:38.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.722+0000 7f14327d6740 20 client.17400 trim_cache size 0 max 16384 2024-04-21T08:42:38.729 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:42:38.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.722+0000 7f142fb11640 1 -- 172.21.15.86:0/91772075 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f14140a3ec0 msgr2=0x7f14140a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.722+0000 7f142fb11640 1 --2- 172.21.15.86:0/91772075 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f14140a3ec0 0x7f14140a6380 secure :-1 s=READY pgs=165 cs=0 l=1 rev1=1 crypto rx=0x7f1420037040 tx=0x7f142002c120 comp rx=0 tx=0).stop 2024-04-21T08:42:38.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.722+0000 7f142fb11640 1 -- 172.21.15.86:0/91772075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1428107270 msgr2=0x7f1428198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.722+0000 7f142fb11640 1 --2- 172.21.15.86:0/91772075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1428107270 0x7f1428198c90 secure :-1 s=READY pgs=1072 cs=0 l=1 rev1=1 crypto rx=0x7f141800c970 tx=0x7f141800ce30 comp rx=0 tx=0).stop 2024-04-21T08:42:38.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.722+0000 7f142fb11640 1 -- 172.21.15.86:0/91772075 shutdown_connections 2024-04-21T08:42:38.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.722+0000 7f142fb11640 1 --2- 172.21.15.86:0/91772075 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f14140a3ec0 0x7f14140a6380 unknown :-1 s=CLOSED pgs=165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.722+0000 7f142fb11640 1 --2- 172.21.15.86:0/91772075 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f142819cfc0 0x7f142819d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.722+0000 7f142fb11640 1 --2- 172.21.15.86:0/91772075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1428107270 0x7f1428198c90 unknown :-1 s=CLOSED pgs=1072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.722+0000 7f142fb11640 1 --2- 172.21.15.86:0/91772075 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14281021a0 0x7f1428198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.722+0000 7f142fb11640 1 -- 172.21.15.86:0/91772075 >> 172.21.15.86:0/91772075 conn(0x7f14280fd9c0 msgr2=0x7f14280ffcc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:38.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.722+0000 7f142fb11640 1 -- 172.21.15.86:0/91772075 shutdown_connections 2024-04-21T08:42:38.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.722+0000 7f142fb11640 1 -- 172.21.15.86:0/91772075 wait complete. 2024-04-21T08:42:38.731 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:38 smithi086 ceph-mon[27939]: pgmap v118: 129 pgs: 129 active+clean; 4.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-21T08:42:38.733 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:42:38.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:38 smithi179 ceph-mon[31829]: pgmap v118: 129 pgs: 129 active+clean; 4.2 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-21T08:42:38.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.833+0000 7fc3d1225640 1 -- 172.21.15.86:0/1377959058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3cc102190 msgr2=0x7fc3cc102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.833+0000 7fc3d1225640 1 --2- 172.21.15.86:0/1377959058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3cc102190 0x7fc3cc102570 secure :-1 s=READY pgs=1077 cs=0 l=1 rev1=1 crypto rx=0x7fc3bc009900 tx=0x7fc3bc02bc50 comp rx=0 tx=0).stop 2024-04-21T08:42:38.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.833+0000 7fc3d1225640 1 -- 172.21.15.86:0/1377959058 shutdown_connections 2024-04-21T08:42:38.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.833+0000 7fc3d1225640 1 --2- 172.21.15.86:0/1377959058 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3cc107260 0x7fc3cc10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.833+0000 7fc3d1225640 1 --2- 172.21.15.86:0/1377959058 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3cc102b40 0x7fc3cc106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.833+0000 7fc3d1225640 1 --2- 172.21.15.86:0/1377959058 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3cc102190 0x7fc3cc102570 unknown :-1 s=CLOSED pgs=1077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.833+0000 7fc3d1225640 1 -- 172.21.15.86:0/1377959058 >> 172.21.15.86:0/1377959058 conn(0x7fc3cc0793a0 msgr2=0x7fc3cc0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:38.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.833+0000 7fc3d1225640 1 -- 172.21.15.86:0/1377959058 shutdown_connections 2024-04-21T08:42:38.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.833+0000 7fc3d1225640 1 -- 172.21.15.86:0/1377959058 wait complete. 2024-04-21T08:42:38.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.834+0000 7fc3d1225640 1 Processor -- start 2024-04-21T08:42:38.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.834+0000 7fc3d1225640 1 -- start start 2024-04-21T08:42:38.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.834+0000 7fc3d1225640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3cc102190 0x7fc3cc19e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.834+0000 7fc3d1225640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3cc107260 0x7fc3cc19e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.834+0000 7fc3d1225640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3cc198620 0x7fc3cc198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.834+0000 7fc3d1225640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc112240 con 0x7fc3cc198620 2024-04-21T08:42:38.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.834+0000 7fc3d1225640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc1120c0 con 0x7fc3cc107260 2024-04-21T08:42:38.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.834+0000 7fc3d1225640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc1123c0 con 0x7fc3cc102190 2024-04-21T08:42:38.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.834+0000 7fc3cb59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3cc198620 0x7fc3cc198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.834+0000 7fc3cb59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3cc198620 0x7fc3cc198ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33370/0 (socket says 172.21.15.86:33370) 2024-04-21T08:42:38.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.834+0000 7fc3cb59e640 1 -- 172.21.15.86:0/1031745291 learned_addr learned my addr 172.21.15.86:0/1031745291 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:38.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.834+0000 7fc3c3fff640 1 --2- 172.21.15.86:0/1031745291 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3cc107260 0x7fc3cc19e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.834+0000 7fc3cb59e640 1 -- 172.21.15.86:0/1031745291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3cc102190 msgr2=0x7fc3cc19e470 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T08:42:38.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.834+0000 7fc3cb59e640 1 --2- 172.21.15.86:0/1031745291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3cc102190 0x7fc3cc19e470 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.834+0000 7fc3cb59e640 1 -- 172.21.15.86:0/1031745291 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3cc107260 msgr2=0x7fc3cc19e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.834+0000 7fc3cb59e640 1 --2- 172.21.15.86:0/1031745291 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3cc107260 0x7fc3cc19e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.835+0000 7fc3cb59e640 1 -- 172.21.15.86:0/1031745291 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3cc199140 con 0x7fc3cc198620 2024-04-21T08:42:38.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.835+0000 7fc3c3fff640 1 --2- 172.21.15.86:0/1031745291 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3cc107260 0x7fc3cc19e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:38.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.835+0000 7fc3cb59e640 1 --2- 172.21.15.86:0/1031745291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3cc198620 0x7fc3cc198ad0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fc3c400b990 tx=0x7fc3c400be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:38.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.835+0000 7fc3c8d99640 1 -- 172.21.15.86:0/1031745291 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c4019070 con 0x7fc3cc198620 2024-04-21T08:42:38.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.835+0000 7fc3c8d99640 1 -- 172.21.15.86:0/1031745291 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3c400cba0 con 0x7fc3cc198620 2024-04-21T08:42:38.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.835+0000 7fc3d1225640 1 -- 172.21.15.86:0/1031745291 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3cc199430 con 0x7fc3cc198620 2024-04-21T08:42:38.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.835+0000 7fc3c8d99640 1 -- 172.21.15.86:0/1031745291 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c40075d0 con 0x7fc3cc198620 2024-04-21T08:42:38.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.835+0000 7fc3d1225640 1 -- 172.21.15.86:0/1031745291 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3cc1a5680 con 0x7fc3cc198620 2024-04-21T08:42:38.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.837+0000 7fc3c8d99640 1 -- 172.21.15.86:0/1031745291 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc3c401c070 con 0x7fc3cc198620 2024-04-21T08:42:38.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.837+0000 7fc3d3e15740 1 Processor -- start 2024-04-21T08:42:38.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.837+0000 7fc3d3e15740 1 -- start start 2024-04-21T08:42:38.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.838+0000 7fc3d3e15740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556987624d0 0x5556987628b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.838+0000 7fc3d3e15740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555698762e80 0x55569876b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.838+0000 7fc3c8d99640 1 --2- 172.21.15.86:0/1031745291 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc3b80a3d20 0x7fc3b80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.838+0000 7fc3d3e15740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55569876ba00 0x55569876de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.838+0000 7fc3d3e15740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5556986c71f0 con 0x5556987624d0 2024-04-21T08:42:38.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.838+0000 7fc3d3e15740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5556986644b0 con 0x55569876ba00 2024-04-21T08:42:38.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.838+0000 7fc3d3e15740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5556986c31b0 con 0x555698762e80 2024-04-21T08:42:38.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.838+0000 7fc3c3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556987624d0 0x5556987628b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.838+0000 7fc3c3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556987624d0 0x5556987628b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33384/0 (socket says 172.21.15.86:33384) 2024-04-21T08:42:38.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.838+0000 7fc3c8d99640 1 -- 172.21.15.86:0/1031745291 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fc3c40d00a0 con 0x7fc3cc198620 2024-04-21T08:42:38.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.838+0000 7fc3c3fff640 1 -- 172.21.15.86:0/2008799983 learned_addr learned my addr 172.21.15.86:0/2008799983 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:38.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.838+0000 7fc3c3fff640 1 --2- 172.21.15.86:0/1031745291 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc3b80a3d20 0x7fc3b80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.838+0000 7fc3cb59e640 1 --2- 172.21.15.86:0/2008799983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55569876ba00 0x55569876de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.838+0000 7fc3c3fff640 1 -- 172.21.15.86:0/2008799983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555698762e80 msgr2=0x55569876b430 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:42:38.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.838+0000 7fc3c3fff640 1 --2- 172.21.15.86:0/2008799983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555698762e80 0x55569876b430 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.838+0000 7fc3c3fff640 1 -- 172.21.15.86:0/2008799983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55569876ba00 msgr2=0x55569876de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:38.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.838+0000 7fc3c3fff640 1 --2- 172.21.15.86:0/2008799983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55569876ba00 0x55569876de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.838+0000 7fc3c3fff640 1 -- 172.21.15.86:0/2008799983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5556986561b0 con 0x5556987624d0 2024-04-21T08:42:38.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.838+0000 7fc3cb59e640 1 --2- 172.21.15.86:0/2008799983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55569876ba00 0x55569876de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:38.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.838+0000 7fc3c3fff640 1 --2- 172.21.15.86:0/2008799983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556987624d0 0x5556987628b0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4014c70 tx=0x7fc3b40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92c9ea1fc12e6f5d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:38.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.838+0000 7fc3c3fff640 1 --2- 172.21.15.86:0/1031745291 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc3b80a3d20 0x7fc3b80a61e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4011cb0 tx=0x7fc3b4016000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:38.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.838+0000 7fc3c17fa640 1 -- 172.21.15.86:0/2008799983 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4022070 con 0x5556987624d0 2024-04-21T08:42:38.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.838+0000 7fc3c17fa640 1 -- 172.21.15.86:0/2008799983 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3b400b9f0 con 0x5556987624d0 2024-04-21T08:42:38.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.838+0000 7fc3c17fa640 1 -- 172.21.15.86:0/2008799983 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4005020 con 0x5556987624d0 2024-04-21T08:42:38.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.839+0000 7fc3d3e15740 1 -- 172.21.15.86:0/2008799983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556987624d0 msgr2=0x5556987628b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.839+0000 7fc3d3e15740 1 --2- 172.21.15.86:0/2008799983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556987624d0 0x5556987628b0 secure :-1 s=READY pgs=1079 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4014c70 tx=0x7fc3b40117b0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.839+0000 7fc3d3e15740 1 -- 172.21.15.86:0/2008799983 shutdown_connections 2024-04-21T08:42:38.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.839+0000 7fc3d3e15740 1 --2- 172.21.15.86:0/2008799983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55569876ba00 0x55569876de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.839+0000 7fc3d3e15740 1 --2- 172.21.15.86:0/2008799983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555698762e80 0x55569876b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.839+0000 7fc3d3e15740 1 --2- 172.21.15.86:0/2008799983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5556987624d0 0x5556987628b0 unknown :-1 s=CLOSED pgs=1079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.839+0000 7fc3d3e15740 1 -- 172.21.15.86:0/2008799983 >> 172.21.15.86:0/2008799983 conn(0x55569875cba0 msgr2=0x55569875cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:38.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.839+0000 7fc3d3e15740 1 -- 172.21.15.86:0/2008799983 shutdown_connections 2024-04-21T08:42:38.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.839+0000 7fc3d3e15740 1 -- 172.21.15.86:0/2008799983 wait complete. 2024-04-21T08:42:38.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.840+0000 7fc3d3e15740 1 -- start start 2024-04-21T08:42:38.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.840+0000 7fc3d3e15740 1 Processor -- start 2024-04-21T08:42:38.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.840+0000 7fc3d3e15740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5556987624d0 0x55569876b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.840+0000 7fc3d3e15740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555698762e80 0x5556987687a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.840+0000 7fc3d3e15740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55569876ba00 0x555698768e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.840+0000 7fc3d3e15740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55569876e910 con 0x55569876ba00 2024-04-21T08:42:38.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.840+0000 7fc3d3e15740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x555698680500 con 0x555698762e80 2024-04-21T08:42:38.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.840+0000 7fc3d3e15740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55569876ea90 con 0x5556987624d0 2024-04-21T08:42:38.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.840+0000 7fc3d3e15740 10 client.0 _ll_register_callbacks cb 0x55569864eb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:38.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.840+0000 7fc3cb59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55569876ba00 0x555698768e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.840+0000 7fc3cb59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55569876ba00 0x555698768e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33388/0 (socket says 172.21.15.86:33388) 2024-04-21T08:42:38.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.840+0000 7fc3cb59e640 1 -- 172.21.15.86:0/4013392731 learned_addr learned my addr 172.21.15.86:0/4013392731 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:38.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.840+0000 7fc3c3fff640 1 --2- 172.21.15.86:0/4013392731 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555698762e80 0x5556987687a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.840+0000 7fc3cb59e640 1 -- 172.21.15.86:0/4013392731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5556987624d0 msgr2=0x55569876b410 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:42:38.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.840+0000 7fc3cb59e640 1 --2- 172.21.15.86:0/4013392731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5556987624d0 0x55569876b410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.840+0000 7fc3cb59e640 1 -- 172.21.15.86:0/4013392731 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555698762e80 msgr2=0x5556987687a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:38.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.840+0000 7fc3cb59e640 1 --2- 172.21.15.86:0/4013392731 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555698762e80 0x5556987687a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.840+0000 7fc3cb59e640 1 -- 172.21.15.86:0/4013392731 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556983d93f0 con 0x55569876ba00 2024-04-21T08:42:38.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.840+0000 7fc3cb59e640 1 --2- 172.21.15.86:0/4013392731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55569876ba00 0x555698768e90 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fc3c40a4520 tx=0x7fc3c409c5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=921fd0e6051a75ea server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:38.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.840+0000 7fc3b27fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:38.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.840+0000 7fc3b27fc640 1 -- 172.21.15.86:0/4013392731 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c409d110 con 0x55569876ba00 2024-04-21T08:42:38.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.840+0000 7fc3b27fc640 1 -- 172.21.15.86:0/4013392731 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fc3c40cd700 con 0x55569876ba00 2024-04-21T08:42:38.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.840+0000 7fc3d3e15740 1 -- 172.21.15.86:0/4013392731 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x555698666930 con 0x55569876ba00 2024-04-21T08:42:38.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.841+0000 7fc3b27fc640 1 -- 172.21.15.86:0/4013392731 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc3c40cd900 con 0x55569876ba00 2024-04-21T08:42:38.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.841+0000 7fc3d3e15740 10 client.17412 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:38.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.841+0000 7fc3d3e15740 1 -- 172.21.15.86:0/4013392731 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5556984a37d0 con 0x55569876ba00 2024-04-21T08:42:38.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.841+0000 7fc3b27fc640 1 -- 172.21.15.86:0/4013392731 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x555698666930 con 0x55569876ba00 2024-04-21T08:42:38.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.841+0000 7fc3b27fc640 1 -- 172.21.15.86:0/4013392731 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc3c40cd0e0 con 0x55569876ba00 2024-04-21T08:42:38.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.841+0000 7fc3d3e15740 10 client.17412 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:38.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.841+0000 7fc3d3e15740 10 client.17412 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:42:38.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.841+0000 7fc3d3e15740 20 client.17412 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:38.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.841+0000 7fc3d3e15740 1 --2- 172.21.15.86:0/4013392731 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55569877bc30 0x55569877e040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.841+0000 7fc3d3e15740 4 client.17412 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:38.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.841+0000 7fc3d3e15740 1 -- 172.21.15.86:0/4013392731 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5556986d4c10 con 0x55569877bc30 2024-04-21T08:42:38.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.841+0000 7fc3cad9d640 1 --2- 172.21.15.86:0/4013392731 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55569877bc30 0x55569877e040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.842+0000 7fc3b27fc640 10 client.17412 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:42:38.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.842+0000 7fc3cad9d640 1 --2- 172.21.15.86:0/4013392731 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55569877bc30 0x55569877e040 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6b4b160ff00046ed server_cookie=a59596b0e95c978e in_seq=0 out_seq=0 2024-04-21T08:42:38.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.842+0000 7fc3b27fc640 1 -- 172.21.15.86:0/4013392731 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5556986d4c10 con 0x55569877bc30 2024-04-21T08:42:38.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.842+0000 7fc3b27fc640 10 client.17412 handle_command_reply: tid=0 2024-04-21T08:42:38.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.842+0000 7fc3d3e15740 1 client.17412 shutdown 2024-04-21T08:42:38.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.842+0000 7fc3d3e15740 1 -- 172.21.15.86:0/4013392731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55569876ba00 msgr2=0x555698768e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.842+0000 7fc3d3e15740 1 --2- 172.21.15.86:0/4013392731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55569876ba00 0x555698768e90 secure :-1 s=READY pgs=1080 cs=0 l=1 rev1=1 crypto rx=0x7fc3c40a4520 tx=0x7fc3c409c5a0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.842+0000 7fc3d3e15740 1 -- 172.21.15.86:0/4013392731 shutdown_connections 2024-04-21T08:42:38.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.842+0000 7fc3d3e15740 1 --2- 172.21.15.86:0/4013392731 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55569877bc30 0x55569877e040 crc :-1 s=READY pgs=47 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.842+0000 7fc3d3e15740 1 --2- 172.21.15.86:0/4013392731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55569876ba00 0x555698768e90 unknown :-1 s=CLOSED pgs=1080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.842+0000 7fc3b27fc640 0 client.17412 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:42:38.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.842+0000 7fc3d3e15740 1 --2- 172.21.15.86:0/4013392731 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555698762e80 0x5556987687a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.842+0000 7fc3d3e15740 1 --2- 172.21.15.86:0/4013392731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5556987624d0 0x55569876b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.842+0000 7fc3d3e15740 1 -- 172.21.15.86:0/4013392731 >> 172.21.15.86:0/4013392731 conn(0x55569875cba0 msgr2=0x55569875e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:38.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.843+0000 7fc3d3e15740 1 -- 172.21.15.86:0/4013392731 shutdown_connections 2024-04-21T08:42:38.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.843+0000 7fc3d3e15740 1 -- 172.21.15.86:0/4013392731 wait complete. 2024-04-21T08:42:38.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.843+0000 7fc3d3e15740 20 client.17412 trim_cache size 0 max 16384 2024-04-21T08:42:38.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.856+0000 7fc3d3e15740 1 Processor -- start 2024-04-21T08:42:38.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.856+0000 7fc3d3e15740 1 -- start start 2024-04-21T08:42:38.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.856+0000 7fc3d3e15740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5556987624d0 0x5556987777d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.856+0000 7fc3d3e15740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555698762e80 0x55569876aed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.856+0000 7fc3d3e15740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55569876ba00 0x55569876b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.856+0000 7fc3d3e15740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc1af2b0 con 0x55569876ba00 2024-04-21T08:42:38.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.856+0000 7fc3d3e15740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc1b0df0 con 0x555698762e80 2024-04-21T08:42:38.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.856+0000 7fc3d3e15740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc1af980 con 0x5556987624d0 2024-04-21T08:42:38.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.856+0000 7fc3cb59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55569876ba00 0x55569876b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.856+0000 7fc3cb59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55569876ba00 0x55569876b410 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33394/0 (socket says 172.21.15.86:33394) 2024-04-21T08:42:38.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.856+0000 7fc3cb59e640 1 -- 172.21.15.86:0/720788611 learned_addr learned my addr 172.21.15.86:0/720788611 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:38.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.856+0000 7fc3c3fff640 1 --2- 172.21.15.86:0/720788611 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555698762e80 0x55569876aed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.856+0000 7fc3cb59e640 1 -- 172.21.15.86:0/720788611 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5556987624d0 msgr2=0x5556987777d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:42:38.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.856+0000 7fc3cb59e640 1 --2- 172.21.15.86:0/720788611 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5556987624d0 0x5556987777d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.856+0000 7fc3cb59e640 1 -- 172.21.15.86:0/720788611 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555698762e80 msgr2=0x55569876aed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:38.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.856+0000 7fc3cb59e640 1 --2- 172.21.15.86:0/720788611 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555698762e80 0x55569876aed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.856+0000 7fc3cb59e640 1 -- 172.21.15.86:0/720788611 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555698758470 con 0x55569876ba00 2024-04-21T08:42:38.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.856+0000 7fc3cb59e640 1 --2- 172.21.15.86:0/720788611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55569876ba00 0x55569876b410 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4098240 tx=0x7fc3c4004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1039d65ec96ae6c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:38.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.857+0000 7fc3c0ff9640 1 -- 172.21.15.86:0/720788611 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c40a4a50 con 0x55569876ba00 2024-04-21T08:42:38.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.857+0000 7fc3c0ff9640 1 -- 172.21.15.86:0/720788611 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3c40cd0c0 con 0x55569876ba00 2024-04-21T08:42:38.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.857+0000 7fc3c0ff9640 1 -- 172.21.15.86:0/720788611 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3c40cd700 con 0x55569876ba00 2024-04-21T08:42:38.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.857+0000 7fc3d3e15740 1 -- 172.21.15.86:0/720788611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55569876ba00 msgr2=0x55569876b410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.857+0000 7fc3d3e15740 1 --2- 172.21.15.86:0/720788611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55569876ba00 0x55569876b410 secure :-1 s=READY pgs=1081 cs=0 l=1 rev1=1 crypto rx=0x7fc3c4098240 tx=0x7fc3c4004a60 comp rx=0 tx=0).stop 2024-04-21T08:42:38.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.857+0000 7fc3d3e15740 1 -- 172.21.15.86:0/720788611 shutdown_connections 2024-04-21T08:42:38.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.857+0000 7fc3d3e15740 1 --2- 172.21.15.86:0/720788611 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55569876ba00 0x55569876b410 unknown :-1 s=CLOSED pgs=1081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.857+0000 7fc3d3e15740 1 --2- 172.21.15.86:0/720788611 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555698762e80 0x55569876aed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.857+0000 7fc3d3e15740 1 --2- 172.21.15.86:0/720788611 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5556987624d0 0x5556987777d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.857+0000 7fc3d3e15740 1 -- 172.21.15.86:0/720788611 >> 172.21.15.86:0/720788611 conn(0x55569875cba0 msgr2=0x55569875bc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:38.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.857+0000 7fc3d3e15740 1 -- 172.21.15.86:0/720788611 shutdown_connections 2024-04-21T08:42:38.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.857+0000 7fc3d3e15740 1 -- 172.21.15.86:0/720788611 wait complete. 2024-04-21T08:42:38.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.858+0000 7fc3d3e15740 1 -- start start 2024-04-21T08:42:38.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.858+0000 7fc3d3e15740 1 Processor -- start 2024-04-21T08:42:38.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.858+0000 7fc3d3e15740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5556987624d0 0x555698765590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.858+0000 7fc3d3e15740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555698762e80 0x555698765ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.858+0000 7fc3d3e15740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55569876ba00 0x555698766010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.858+0000 7fc3d3e15740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc1b0720 con 0x555698762e80 2024-04-21T08:42:38.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.858+0000 7fc3d3e15740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc1b0050 con 0x55569876ba00 2024-04-21T08:42:38.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.858+0000 7fc3d3e15740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc3cc1ad270 con 0x5556987624d0 2024-04-21T08:42:38.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.858+0000 7fc3d3e15740 10 client.0 _ll_register_callbacks cb 0x7fc3c409cf10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:38.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.858+0000 7fc3c3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555698762e80 0x555698765ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.858+0000 7fc3c3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555698762e80 0x555698765ad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33406/0 (socket says 172.21.15.86:33406) 2024-04-21T08:42:38.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.858+0000 7fc3c3fff640 1 -- 172.21.15.86:0/296255743 learned_addr learned my addr 172.21.15.86:0/296255743 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:38.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.858+0000 7fc3cb59e640 1 --2- 172.21.15.86:0/296255743 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55569876ba00 0x555698766010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.858+0000 7fc3c3fff640 1 -- 172.21.15.86:0/296255743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5556987624d0 msgr2=0x555698765590 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:42:38.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.858+0000 7fc3c3fff640 1 --2- 172.21.15.86:0/296255743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5556987624d0 0x555698765590 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.858+0000 7fc3c3fff640 1 -- 172.21.15.86:0/296255743 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55569876ba00 msgr2=0x555698766010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:38.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.858+0000 7fc3c3fff640 1 --2- 172.21.15.86:0/296255743 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55569876ba00 0x555698766010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.858+0000 7fc3c3fff640 1 -- 172.21.15.86:0/296255743 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5556987e12e0 con 0x555698762e80 2024-04-21T08:42:38.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.858+0000 7fc3cb59e640 1 --2- 172.21.15.86:0/296255743 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55569876ba00 0x555698766010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:38.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.858+0000 7fc3c3fff640 1 --2- 172.21.15.86:0/296255743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555698762e80 0x555698765ad0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4006010 tx=0x7fc3b40143c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ff960ba1a962a55 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:38.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.858+0000 7fc3b27fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:38.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.859+0000 7fc3b27fc640 1 -- 172.21.15.86:0/296255743 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3b4022040 con 0x555698762e80 2024-04-21T08:42:38.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.859+0000 7fc3b27fc640 1 -- 172.21.15.86:0/296255743 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fc3b400cd40 con 0x555698762e80 2024-04-21T08:42:38.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.859+0000 7fc3d3e15740 1 -- 172.21.15.86:0/296255743 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc3c40cd0c0 con 0x555698762e80 2024-04-21T08:42:38.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.859+0000 7fc3b27fc640 1 -- 172.21.15.86:0/296255743 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc3b403d050 con 0x555698762e80 2024-04-21T08:42:38.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.859+0000 7fc3d3e15740 10 client.17418 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:38.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.859+0000 7fc3d3e15740 1 -- 172.21.15.86:0/296255743 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5556987e2560 con 0x555698762e80 2024-04-21T08:42:38.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.859+0000 7fc3b27fc640 1 -- 172.21.15.86:0/296255743 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3c40cd0c0 con 0x555698762e80 2024-04-21T08:42:38.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.859+0000 7fc3b27fc640 1 -- 172.21.15.86:0/296255743 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc3b4026ba0 con 0x555698762e80 2024-04-21T08:42:38.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.859+0000 7fc3d3e15740 10 client.17418 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:38.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.859+0000 7fc3d3e15740 10 client.17418 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:42:38.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.859+0000 7fc3d3e15740 20 client.17418 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:38.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.859+0000 7fc3d3e15740 1 --2- 172.21.15.86:0/296255743 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5556987e2a40 0x5556987e4e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.859+0000 7fc3d3e15740 4 client.17418 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:42:38.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.859+0000 7fc3d3e15740 1 -- 172.21.15.86:0/296255743 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5556986cea80 con 0x5556987e2a40 2024-04-21T08:42:38.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.859+0000 7fc3cad9d640 1 --2- 172.21.15.86:0/296255743 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5556987e2a40 0x5556987e4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.860+0000 7fc3b27fc640 10 client.17418 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:42:38.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.860+0000 7fc3cad9d640 1 --2- 172.21.15.86:0/296255743 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5556987e2a40 0x5556987e4e60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=edc5ea720231e0fc server_cookie=8533517bc38a145f in_seq=0 out_seq=0 2024-04-21T08:42:38.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.860+0000 7fc3b27fc640 1 -- 172.21.15.86:0/296255743 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5556986cea80 con 0x5556987e2a40 2024-04-21T08:42:38.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.860+0000 7fc3b27fc640 10 client.17418 handle_command_reply: tid=0 2024-04-21T08:42:38.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.860+0000 7fc3d3e15740 1 client.17418 shutdown 2024-04-21T08:42:38.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.860+0000 7fc3d3e15740 1 -- 172.21.15.86:0/296255743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555698762e80 msgr2=0x555698765ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.860+0000 7fc3d3e15740 1 --2- 172.21.15.86:0/296255743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555698762e80 0x555698765ad0 secure :-1 s=READY pgs=1082 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4006010 tx=0x7fc3b40143c0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.860+0000 7fc3d3e15740 1 -- 172.21.15.86:0/296255743 shutdown_connections 2024-04-21T08:42:38.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.860+0000 7fc3d3e15740 1 --2- 172.21.15.86:0/296255743 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5556987e2a40 0x5556987e4e60 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.860+0000 7fc3d3e15740 1 --2- 172.21.15.86:0/296255743 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55569876ba00 0x555698766010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.860+0000 7fc3d3e15740 1 --2- 172.21.15.86:0/296255743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555698762e80 0x555698765ad0 unknown :-1 s=CLOSED pgs=1082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.860+0000 7fc3b27fc640 0 client.17418 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:42:38.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.860+0000 7fc3d3e15740 1 --2- 172.21.15.86:0/296255743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5556987624d0 0x555698765590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.860+0000 7fc3d3e15740 1 -- 172.21.15.86:0/296255743 >> 172.21.15.86:0/296255743 conn(0x55569875cba0 msgr2=0x55569875dea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:38.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.860+0000 7fc3d3e15740 1 -- 172.21.15.86:0/296255743 shutdown_connections 2024-04-21T08:42:38.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.860+0000 7fc3d3e15740 1 -- 172.21.15.86:0/296255743 wait complete. 2024-04-21T08:42:38.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.860+0000 7fc3d3e15740 20 client.17418 trim_cache size 0 max 16384 2024-04-21T08:42:38.866 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:42:38.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.861+0000 7fc3d1225640 1 -- 172.21.15.86:0/1031745291 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc3b80a3d20 msgr2=0x7fc3b80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.861+0000 7fc3d1225640 1 --2- 172.21.15.86:0/1031745291 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc3b80a3d20 0x7fc3b80a61e0 secure :-1 s=READY pgs=166 cs=0 l=1 rev1=1 crypto rx=0x7fc3b4011cb0 tx=0x7fc3b4016000 comp rx=0 tx=0).stop 2024-04-21T08:42:38.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.861+0000 7fc3d1225640 1 -- 172.21.15.86:0/1031745291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3cc198620 msgr2=0x7fc3cc198ad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.861+0000 7fc3d1225640 1 --2- 172.21.15.86:0/1031745291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3cc198620 0x7fc3cc198ad0 secure :-1 s=READY pgs=1078 cs=0 l=1 rev1=1 crypto rx=0x7fc3c400b990 tx=0x7fc3c400be50 comp rx=0 tx=0).stop 2024-04-21T08:42:38.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.861+0000 7fc3d1225640 1 -- 172.21.15.86:0/1031745291 shutdown_connections 2024-04-21T08:42:38.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.861+0000 7fc3d1225640 1 --2- 172.21.15.86:0/1031745291 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc3b80a3d20 0x7fc3b80a61e0 unknown :-1 s=CLOSED pgs=166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.861+0000 7fc3d1225640 1 --2- 172.21.15.86:0/1031745291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3cc198620 0x7fc3cc198ad0 unknown :-1 s=CLOSED pgs=1078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.861+0000 7fc3d1225640 1 --2- 172.21.15.86:0/1031745291 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3cc107260 0x7fc3cc19e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.861+0000 7fc3d1225640 1 --2- 172.21.15.86:0/1031745291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3cc102190 0x7fc3cc19e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.861+0000 7fc3d1225640 1 -- 172.21.15.86:0/1031745291 >> 172.21.15.86:0/1031745291 conn(0x7fc3cc0793a0 msgr2=0x7fc3cc1000c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:38.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.861+0000 7fc3d1225640 1 -- 172.21.15.86:0/1031745291 shutdown_connections 2024-04-21T08:42:38.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.861+0000 7fc3d1225640 1 -- 172.21.15.86:0/1031745291 wait complete. 2024-04-21T08:42:38.871 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:42:38.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.971+0000 7fb45bc62640 1 -- 172.21.15.86:0/57070943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb454112910 msgr2=0x7fb454112cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.971+0000 7fb45bc62640 1 --2- 172.21.15.86:0/57070943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb454112910 0x7fb454112cf0 secure :-1 s=READY pgs=1083 cs=0 l=1 rev1=1 crypto rx=0x7fb444009930 tx=0x7fb44402bc50 comp rx=0 tx=0).stop 2024-04-21T08:42:38.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.971+0000 7fb45bc62640 1 -- 172.21.15.86:0/57070943 shutdown_connections 2024-04-21T08:42:38.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.971+0000 7fb45bc62640 1 --2- 172.21.15.86:0/57070943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb45410d2c0 0x7fb4541197d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.971+0000 7fb45bc62640 1 --2- 172.21.15.86:0/57070943 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb45410c900 0x7fb45410cd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.971+0000 7fb45bc62640 1 --2- 172.21.15.86:0/57070943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb454112910 0x7fb454112cf0 unknown :-1 s=CLOSED pgs=1083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.971+0000 7fb45bc62640 1 -- 172.21.15.86:0/57070943 >> 172.21.15.86:0/57070943 conn(0x7fb45406cf80 msgr2=0x7fb45410af60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:38.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.971+0000 7fb45bc62640 1 -- 172.21.15.86:0/57070943 shutdown_connections 2024-04-21T08:42:38.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.971+0000 7fb45bc62640 1 -- 172.21.15.86:0/57070943 wait complete. 2024-04-21T08:42:38.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.972+0000 7fb45bc62640 1 Processor -- start 2024-04-21T08:42:38.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.972+0000 7fb45bc62640 1 -- start start 2024-04-21T08:42:38.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.972+0000 7fb45bc62640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb45410c900 0x7fb4541a32f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.972+0000 7fb45bc62640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb45410d2c0 0x7fb4541a3830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.972+0000 7fb45bc62640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb454112910 0x7fb4541a7ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.972+0000 7fb45bc62640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb45411cbf0 con 0x7fb45410c900 2024-04-21T08:42:38.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.972+0000 7fb45bc62640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb45411ca70 con 0x7fb45410d2c0 2024-04-21T08:42:38.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.972+0000 7fb45bc62640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb45411cd70 con 0x7fb454112910 2024-04-21T08:42:38.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.972+0000 7fb4599fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb45410c900 0x7fb4541a32f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.972+0000 7fb4599fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb45410c900 0x7fb4541a32f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33424/0 (socket says 172.21.15.86:33424) 2024-04-21T08:42:38.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.972+0000 7fb4599fe640 1 -- 172.21.15.86:0/1015848837 learned_addr learned my addr 172.21.15.86:0/1015848837 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:38.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.972+0000 7fb4591fd640 1 --2- 172.21.15.86:0/1015848837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb45410d2c0 0x7fb4541a3830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.972+0000 7fb4599fe640 1 -- 172.21.15.86:0/1015848837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb454112910 msgr2=0x7fb4541a7ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.972+0000 7fb4599fe640 1 --2- 172.21.15.86:0/1015848837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb454112910 0x7fb4541a7ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.972+0000 7fb4599fe640 1 -- 172.21.15.86:0/1015848837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb45410d2c0 msgr2=0x7fb4541a3830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.972+0000 7fb4599fe640 1 --2- 172.21.15.86:0/1015848837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb45410d2c0 0x7fb4541a3830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.972+0000 7fb4599fe640 1 -- 172.21.15.86:0/1015848837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb4541a81f0 con 0x7fb45410c900 2024-04-21T08:42:38.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.972+0000 7fb4599fe640 1 --2- 172.21.15.86:0/1015848837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb45410c900 0x7fb4541a32f0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fb444009230 tx=0x7fb44402c880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:38.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.972+0000 7fb452ffd640 1 -- 172.21.15.86:0/1015848837 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb444043020 con 0x7fb45410c900 2024-04-21T08:42:38.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.973+0000 7fb452ffd640 1 -- 172.21.15.86:0/1015848837 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb4440052b0 con 0x7fb45410c900 2024-04-21T08:42:38.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.973+0000 7fb45bc62640 1 -- 172.21.15.86:0/1015848837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb4541a8480 con 0x7fb45410c900 2024-04-21T08:42:38.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.973+0000 7fb452ffd640 1 -- 172.21.15.86:0/1015848837 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb444003c30 con 0x7fb45410c900 2024-04-21T08:42:38.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.973+0000 7fb45bc62640 1 -- 172.21.15.86:0/1015848837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb4541afd70 con 0x7fb45410c900 2024-04-21T08:42:38.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.975+0000 7fb452ffd640 1 -- 172.21.15.86:0/1015848837 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb444046020 con 0x7fb45410c900 2024-04-21T08:42:38.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.975+0000 7fb45e84b740 1 Processor -- start 2024-04-21T08:42:38.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.975+0000 7fb45e84b740 1 -- start start 2024-04-21T08:42:38.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.976+0000 7fb45e84b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555968b4f4d0 0x555968b4f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.976+0000 7fb45e84b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555968b4fe80 0x555968b58430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.976+0000 7fb45e84b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555968b58a00 0x555968b5ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.976+0000 7fb45e84b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555968ab41f0 con 0x555968b4f4d0 2024-04-21T08:42:38.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.976+0000 7fb45e84b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x555968a514b0 con 0x555968b4fe80 2024-04-21T08:42:38.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.976+0000 7fb45e84b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555968ab01b0 con 0x555968b58a00 2024-04-21T08:42:38.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.976+0000 7fb45a1ff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555968b4f4d0 0x555968b4f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.976+0000 7fb45a1ff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555968b4f4d0 0x555968b4f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33436/0 (socket says 172.21.15.86:33436) 2024-04-21T08:42:38.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.976+0000 7fb45a1ff640 1 -- 172.21.15.86:0/1537021541 learned_addr learned my addr 172.21.15.86:0/1537021541 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:38.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.976+0000 7fb452ffd640 1 --2- 172.21.15.86:0/1015848837 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb43c0a3d20 0x7fb43c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.976+0000 7fb4599fe640 1 --2- 172.21.15.86:0/1537021541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555968b4fe80 0x555968b58430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.976+0000 7fb45a1ff640 1 --2- 172.21.15.86:0/1015848837 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb43c0a3d20 0x7fb43c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.976+0000 7fb45a1ff640 1 -- 172.21.15.86:0/1537021541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555968b58a00 msgr2=0x555968b5ae50 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:42:38.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.976+0000 7fb452ffd640 1 -- 172.21.15.86:0/1015848837 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fb4440f2eb0 con 0x7fb45410c900 2024-04-21T08:42:38.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.976+0000 7fb45a1ff640 1 --2- 172.21.15.86:0/1537021541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555968b58a00 0x555968b5ae50 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.976+0000 7fb45a1ff640 1 -- 172.21.15.86:0/1537021541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555968b4fe80 msgr2=0x555968b58430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:38.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.976+0000 7fb45a1ff640 1 --2- 172.21.15.86:0/1537021541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555968b4fe80 0x555968b58430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.976+0000 7fb45a1ff640 1 -- 172.21.15.86:0/1537021541 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555968a431b0 con 0x555968b4f4d0 2024-04-21T08:42:38.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.976+0000 7fb45a1ff640 1 --2- 172.21.15.86:0/1537021541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555968b4f4d0 0x555968b4f8b0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fb44c009f40 tx=0x7fb44c0126a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19258d257c1558c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:38.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.976+0000 7fb43bfff640 1 -- 172.21.15.86:0/1537021541 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c00f050 con 0x555968b4f4d0 2024-04-21T08:42:38.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.977+0000 7fb43bfff640 1 -- 172.21.15.86:0/1537021541 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb44c010040 con 0x555968b4f4d0 2024-04-21T08:42:38.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.977+0000 7fb43bfff640 1 -- 172.21.15.86:0/1537021541 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c01a030 con 0x555968b4f4d0 2024-04-21T08:42:38.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.977+0000 7fb45e84b740 1 -- 172.21.15.86:0/1537021541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555968b4f4d0 msgr2=0x555968b4f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.977+0000 7fb45a1ff640 1 --2- 172.21.15.86:0/1015848837 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb43c0a3d20 0x7fb43c0a61e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb44c01d000 tx=0x7fb44c019000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:38.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.977+0000 7fb45e84b740 1 --2- 172.21.15.86:0/1537021541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555968b4f4d0 0x555968b4f8b0 secure :-1 s=READY pgs=1085 cs=0 l=1 rev1=1 crypto rx=0x7fb44c009f40 tx=0x7fb44c0126a0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.977+0000 7fb45e84b740 1 -- 172.21.15.86:0/1537021541 shutdown_connections 2024-04-21T08:42:38.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.977+0000 7fb45e84b740 1 --2- 172.21.15.86:0/1537021541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555968b58a00 0x555968b5ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.977+0000 7fb45e84b740 1 --2- 172.21.15.86:0/1537021541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555968b4fe80 0x555968b58430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.977+0000 7fb45e84b740 1 --2- 172.21.15.86:0/1537021541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555968b4f4d0 0x555968b4f8b0 unknown :-1 s=CLOSED pgs=1085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.977+0000 7fb45e84b740 1 -- 172.21.15.86:0/1537021541 >> 172.21.15.86:0/1537021541 conn(0x555968b49ba0 msgr2=0x555968b49f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:38.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.977+0000 7fb45e84b740 1 -- 172.21.15.86:0/1537021541 shutdown_connections 2024-04-21T08:42:38.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.977+0000 7fb45e84b740 1 -- 172.21.15.86:0/1537021541 wait complete. 2024-04-21T08:42:38.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.978+0000 7fb45e84b740 1 -- start start 2024-04-21T08:42:38.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.978+0000 7fb45e84b740 1 Processor -- start 2024-04-21T08:42:38.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.978+0000 7fb45e84b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555968b4f4d0 0x555968b582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.978+0000 7fb45e84b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555968b4fe80 0x555968b55670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.978+0000 7fb45e84b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555968b58a00 0x555968b55bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.978+0000 7fb45e84b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555968b5b970 con 0x555968b4f4d0 2024-04-21T08:42:38.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.978+0000 7fb45e84b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x555968a6d500 con 0x555968b4fe80 2024-04-21T08:42:38.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.978+0000 7fb45e84b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555968b5baf0 con 0x555968b58a00 2024-04-21T08:42:38.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.978+0000 7fb45e84b740 10 client.0 _ll_register_callbacks cb 0x555968a3bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:38.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.978+0000 7fb4599fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555968b4f4d0 0x555968b582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.978+0000 7fb4599fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555968b4f4d0 0x555968b582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33452/0 (socket says 172.21.15.86:33452) 2024-04-21T08:42:38.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.978+0000 7fb4599fe640 1 -- 172.21.15.86:0/3046794446 learned_addr learned my addr 172.21.15.86:0/3046794446 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:38.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.978+0000 7fb4591fd640 1 --2- 172.21.15.86:0/3046794446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555968b4fe80 0x555968b55670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.979+0000 7fb4599fe640 1 -- 172.21.15.86:0/3046794446 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555968b58a00 msgr2=0x555968b55bd0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:42:38.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.979+0000 7fb4599fe640 1 --2- 172.21.15.86:0/3046794446 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555968b58a00 0x555968b55bd0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.979+0000 7fb4599fe640 1 -- 172.21.15.86:0/3046794446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555968b4fe80 msgr2=0x555968b55670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:38.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.979+0000 7fb4599fe640 1 --2- 172.21.15.86:0/3046794446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555968b4fe80 0x555968b55670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.979+0000 7fb4599fe640 1 -- 172.21.15.86:0/3046794446 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5559687c63f0 con 0x555968b4f4d0 2024-04-21T08:42:38.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.979+0000 7fb4591fd640 1 --2- 172.21.15.86:0/3046794446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555968b4fe80 0x555968b55670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:38.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.979+0000 7fb4599fe640 1 --2- 172.21.15.86:0/3046794446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555968b4f4d0 0x555968b582e0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fb4440bb050 tx=0x7fb44408d5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5170dc284b49b758 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:38.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.979+0000 7fb4397fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:38.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.979+0000 7fb4397fa640 1 -- 172.21.15.86:0/3046794446 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb444089700 con 0x555968b4f4d0 2024-04-21T08:42:38.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.979+0000 7fb4397fa640 1 -- 172.21.15.86:0/3046794446 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fb4440df0b0 con 0x555968b4f4d0 2024-04-21T08:42:38.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.979+0000 7fb45e84b740 1 -- 172.21.15.86:0/3046794446 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb44c010040 con 0x555968b4f4d0 2024-04-21T08:42:38.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.979+0000 7fb4397fa640 1 -- 172.21.15.86:0/3046794446 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb4440bf2f0 con 0x555968b4f4d0 2024-04-21T08:42:38.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.979+0000 7fb45e84b740 10 client.17430 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:38.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.979+0000 7fb45e84b740 1 -- 172.21.15.86:0/3046794446 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5559688907d0 con 0x555968b4f4d0 2024-04-21T08:42:38.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.979+0000 7fb4397fa640 1 -- 172.21.15.86:0/3046794446 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb44c010040 con 0x555968b4f4d0 2024-04-21T08:42:38.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.979+0000 7fb4397fa640 1 -- 172.21.15.86:0/3046794446 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb4440f0510 con 0x555968b4f4d0 2024-04-21T08:42:38.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.979+0000 7fb45e84b740 10 client.17430 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:38.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.979+0000 7fb45e84b740 10 client.17430 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:42:38.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.979+0000 7fb45e84b740 20 client.17430 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:38.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.979+0000 7fb45e84b740 1 --2- 172.21.15.86:0/3046794446 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x555968b68b70 0x555968b6afc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:38.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.979+0000 7fb45e84b740 4 client.17430 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:38.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.979+0000 7fb45e84b740 1 -- 172.21.15.86:0/3046794446 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555968a53930 con 0x555968b68b70 2024-04-21T08:42:38.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.981+0000 7fb4591fd640 1 --2- 172.21.15.86:0/3046794446 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x555968b68b70 0x555968b6afc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:38.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.983+0000 7fb4591fd640 1 --2- 172.21.15.86:0/3046794446 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x555968b68b70 0x555968b6afc0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7d393ced7cbaedd8 server_cookie=8b4ba02676d2590e in_seq=0 out_seq=0 2024-04-21T08:42:38.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.983+0000 7fb4397fa640 10 client.17430 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:42:38.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.984+0000 7fb4397fa640 1 -- 172.21.15.86:0/3046794446 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555968a53930 con 0x555968b68b70 2024-04-21T08:42:38.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.984+0000 7fb4397fa640 10 client.17430 handle_command_reply: tid=0 2024-04-21T08:42:38.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.984+0000 7fb45e84b740 1 client.17430 shutdown 2024-04-21T08:42:38.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.985+0000 7fb45e84b740 1 -- 172.21.15.86:0/3046794446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555968b4f4d0 msgr2=0x555968b582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:38.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.985+0000 7fb45e84b740 1 --2- 172.21.15.86:0/3046794446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555968b4f4d0 0x555968b582e0 secure :-1 s=READY pgs=1086 cs=0 l=1 rev1=1 crypto rx=0x7fb4440bb050 tx=0x7fb44408d5a0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.985+0000 7fb45e84b740 1 -- 172.21.15.86:0/3046794446 shutdown_connections 2024-04-21T08:42:38.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.985+0000 7fb45e84b740 1 --2- 172.21.15.86:0/3046794446 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555968b58a00 0x555968b55bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.985+0000 7fb45e84b740 1 --2- 172.21.15.86:0/3046794446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555968b4fe80 0x555968b55670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.985+0000 7fb45e84b740 1 --2- 172.21.15.86:0/3046794446 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x555968b68b70 0x555968b6afc0 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.985+0000 7fb45e84b740 1 --2- 172.21.15.86:0/3046794446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555968b4f4d0 0x555968b582e0 unknown :-1 s=CLOSED pgs=1086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:38.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.985+0000 7fb45e84b740 1 -- 172.21.15.86:0/3046794446 >> 172.21.15.86:0/3046794446 conn(0x555968b49ba0 msgr2=0x555968b4b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:38.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.985+0000 7fb4397fa640 0 client.17430 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:42:38.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.985+0000 7fb45e84b740 1 -- 172.21.15.86:0/3046794446 shutdown_connections 2024-04-21T08:42:38.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.985+0000 7fb45e84b740 1 -- 172.21.15.86:0/3046794446 wait complete. 2024-04-21T08:42:38.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.985+0000 7fb45e84b740 20 client.17430 trim_cache size 0 max 16384 2024-04-21T08:42:39.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.998+0000 7fb45e84b740 1 Processor -- start 2024-04-21T08:42:39.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.998+0000 7fb45e84b740 1 -- start start 2024-04-21T08:42:39.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.998+0000 7fb45e84b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555968b4f4d0 0x555968b64830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:39.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.998+0000 7fb45e84b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555968b4fe80 0x555968b57da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:39.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.998+0000 7fb45e84b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555968b58a00 0x555968b582e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:39.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.998+0000 7fb45e84b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4541b9d30 con 0x555968b58a00 2024-04-21T08:42:39.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.998+0000 7fb45e84b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb4541bb870 con 0x555968b4fe80 2024-04-21T08:42:39.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.998+0000 7fb45e84b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb4541ba400 con 0x555968b4f4d0 2024-04-21T08:42:39.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.998+0000 7fb45a1ff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555968b58a00 0x555968b582e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:39.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.998+0000 7fb45a1ff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555968b58a00 0x555968b582e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33462/0 (socket says 172.21.15.86:33462) 2024-04-21T08:42:39.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.998+0000 7fb45a1ff640 1 -- 172.21.15.86:0/973654615 learned_addr learned my addr 172.21.15.86:0/973654615 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:39.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.998+0000 7fb4591fd640 1 --2- 172.21.15.86:0/973654615 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555968b4fe80 0x555968b57da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:39.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.999+0000 7fb45a1ff640 1 -- 172.21.15.86:0/973654615 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555968b4f4d0 msgr2=0x555968b64830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:39.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.999+0000 7fb4599fe640 1 --2- 172.21.15.86:0/973654615 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555968b4f4d0 0x555968b64830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:39.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.999+0000 7fb45a1ff640 1 --2- 172.21.15.86:0/973654615 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555968b4f4d0 0x555968b64830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.999+0000 7fb45a1ff640 1 -- 172.21.15.86:0/973654615 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555968b4fe80 msgr2=0x555968b57da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:39.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.999+0000 7fb45a1ff640 1 --2- 172.21.15.86:0/973654615 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555968b4fe80 0x555968b57da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.999+0000 7fb45a1ff640 1 -- 172.21.15.86:0/973654615 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555968b55f40 con 0x555968b58a00 2024-04-21T08:42:39.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.999+0000 7fb4591fd640 1 --2- 172.21.15.86:0/973654615 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555968b4fe80 0x555968b57da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:39.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.999+0000 7fb45a1ff640 1 --2- 172.21.15.86:0/973654615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555968b58a00 0x555968b582e0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fb44c00a010 tx=0x7fb44c004790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bee314d76154734 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:39.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.999+0000 7fb43b7fe640 1 -- 172.21.15.86:0/973654615 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c01a030 con 0x555968b58a00 2024-04-21T08:42:39.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.999+0000 7fb43b7fe640 1 -- 172.21.15.86:0/973654615 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb44c023070 con 0x555968b58a00 2024-04-21T08:42:39.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.999+0000 7fb43b7fe640 1 -- 172.21.15.86:0/973654615 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44c00f070 con 0x555968b58a00 2024-04-21T08:42:39.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.999+0000 7fb45e84b740 1 -- 172.21.15.86:0/973654615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555968b58a00 msgr2=0x555968b582e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:39.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.999+0000 7fb45e84b740 1 --2- 172.21.15.86:0/973654615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555968b58a00 0x555968b582e0 secure :-1 s=READY pgs=1087 cs=0 l=1 rev1=1 crypto rx=0x7fb44c00a010 tx=0x7fb44c004790 comp rx=0 tx=0).stop 2024-04-21T08:42:39.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.999+0000 7fb45e84b740 1 -- 172.21.15.86:0/973654615 shutdown_connections 2024-04-21T08:42:39.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.999+0000 7fb45e84b740 1 --2- 172.21.15.86:0/973654615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555968b58a00 0x555968b582e0 unknown :-1 s=CLOSED pgs=1087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.999+0000 7fb45e84b740 1 --2- 172.21.15.86:0/973654615 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555968b4fe80 0x555968b57da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.999+0000 7fb45e84b740 1 --2- 172.21.15.86:0/973654615 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555968b4f4d0 0x555968b64830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.999+0000 7fb45e84b740 1 -- 172.21.15.86:0/973654615 >> 172.21.15.86:0/973654615 conn(0x555968b49ba0 msgr2=0x555968b4b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:39.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.999+0000 7fb45e84b740 1 -- 172.21.15.86:0/973654615 shutdown_connections 2024-04-21T08:42:39.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:38.999+0000 7fb45e84b740 1 -- 172.21.15.86:0/973654615 wait complete. 2024-04-21T08:42:39.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.000+0000 7fb45e84b740 1 -- start start 2024-04-21T08:42:39.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.000+0000 7fb45e84b740 1 Processor -- start 2024-04-21T08:42:39.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.000+0000 7fb45e84b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555968b4f4d0 0x555968b527a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:39.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.000+0000 7fb45e84b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555968b4fe80 0x555968b52ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:39.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.000+0000 7fb45e84b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555968b58a00 0x555968bce140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:39.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.000+0000 7fb45e84b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb4541bb1a0 con 0x555968b4fe80 2024-04-21T08:42:39.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.000+0000 7fb45e84b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb4541baad0 con 0x555968b58a00 2024-04-21T08:42:39.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.000+0000 7fb45e84b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb4541b7cf0 con 0x555968b4f4d0 2024-04-21T08:42:39.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.000+0000 7fb45e84b740 10 client.0 _ll_register_callbacks cb 0x7fb444009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:39.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.000+0000 7fb4591fd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555968b4fe80 0x555968b52ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:39.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.000+0000 7fb4591fd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555968b4fe80 0x555968b52ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33470/0 (socket says 172.21.15.86:33470) 2024-04-21T08:42:39.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.000+0000 7fb4591fd640 1 -- 172.21.15.86:0/1588442552 learned_addr learned my addr 172.21.15.86:0/1588442552 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:39.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.000+0000 7fb45a1ff640 1 --2- 172.21.15.86:0/1588442552 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555968b58a00 0x555968bce140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:39.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.001+0000 7fb4591fd640 1 -- 172.21.15.86:0/1588442552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555968b4f4d0 msgr2=0x555968b527a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:39.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.001+0000 7fb4591fd640 1 --2- 172.21.15.86:0/1588442552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555968b4f4d0 0x555968b527a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.001+0000 7fb4591fd640 1 -- 172.21.15.86:0/1588442552 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555968b58a00 msgr2=0x555968bce140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:39.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.001+0000 7fb4591fd640 1 --2- 172.21.15.86:0/1588442552 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555968b58a00 0x555968bce140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.001+0000 7fb4591fd640 1 -- 172.21.15.86:0/1588442552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555968bce680 con 0x555968b4fe80 2024-04-21T08:42:39.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.001+0000 7fb45a1ff640 1 --2- 172.21.15.86:0/1588442552 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555968b58a00 0x555968bce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:39.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.001+0000 7fb4591fd640 1 --2- 172.21.15.86:0/1588442552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555968b4fe80 0x555968b52ce0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x555968b54ed0 tx=0x7fb448013750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dba735860ebc7c88 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:39.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.001+0000 7fb4397fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:39.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.001+0000 7fb4397fa640 1 -- 172.21.15.86:0/1588442552 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb44801b880 con 0x555968b4fe80 2024-04-21T08:42:39.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.001+0000 7fb4397fa640 1 -- 172.21.15.86:0/1588442552 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fb448011440 con 0x555968b4fe80 2024-04-21T08:42:39.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.001+0000 7fb45e84b740 1 -- 172.21.15.86:0/1588442552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb44c023070 con 0x555968b4fe80 2024-04-21T08:42:39.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.001+0000 7fb4397fa640 1 -- 172.21.15.86:0/1588442552 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb44802b050 con 0x555968b4fe80 2024-04-21T08:42:39.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.001+0000 7fb45e84b740 10 client.17436 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:39.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.001+0000 7fb45e84b740 1 -- 172.21.15.86:0/1588442552 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555968bcf840 con 0x555968b4fe80 2024-04-21T08:42:39.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.001+0000 7fb4397fa640 1 -- 172.21.15.86:0/1588442552 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb44c023070 con 0x555968b4fe80 2024-04-21T08:42:39.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.001+0000 7fb4397fa640 1 -- 172.21.15.86:0/1588442552 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb448010710 con 0x555968b4fe80 2024-04-21T08:42:39.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.001+0000 7fb45e84b740 10 client.17436 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:39.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.001+0000 7fb45e84b740 10 client.17436 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:42:39.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.001+0000 7fb45e84b740 20 client.17436 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:39.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.001+0000 7fb45e84b740 1 --2- 172.21.15.86:0/1588442552 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x555968bcfff0 0x555968bd2410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:39.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.001+0000 7fb45e84b740 4 client.17436 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:42:39.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.001+0000 7fb45e84b740 1 -- 172.21.15.86:0/1588442552 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555968ac1c10 con 0x555968bcfff0 2024-04-21T08:42:39.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.002+0000 7fb4599fe640 1 --2- 172.21.15.86:0/1588442552 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x555968bcfff0 0x555968bd2410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:39.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.002+0000 7fb4599fe640 1 --2- 172.21.15.86:0/1588442552 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x555968bcfff0 0x555968bd2410 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=623845ff301765b4 server_cookie=89fd95eea708bb87 in_seq=0 out_seq=0 2024-04-21T08:42:39.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.002+0000 7fb4397fa640 10 client.17436 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:42:39.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.003+0000 7fb4397fa640 1 -- 172.21.15.86:0/1588442552 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555968ac1c10 con 0x555968bcfff0 2024-04-21T08:42:39.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.003+0000 7fb4397fa640 10 client.17436 handle_command_reply: tid=0 2024-04-21T08:42:39.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.003+0000 7fb45e84b740 1 client.17436 shutdown 2024-04-21T08:42:39.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.003+0000 7fb45e84b740 1 -- 172.21.15.86:0/1588442552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555968b4fe80 msgr2=0x555968b52ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:39.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.003+0000 7fb45e84b740 1 --2- 172.21.15.86:0/1588442552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555968b4fe80 0x555968b52ce0 secure :-1 s=READY pgs=1088 cs=0 l=1 rev1=1 crypto rx=0x555968b54ed0 tx=0x7fb448013750 comp rx=0 tx=0).stop 2024-04-21T08:42:39.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.003+0000 7fb45e84b740 1 -- 172.21.15.86:0/1588442552 shutdown_connections 2024-04-21T08:42:39.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.003+0000 7fb45e84b740 1 --2- 172.21.15.86:0/1588442552 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555968b58a00 0x555968bce140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.003+0000 7fb45e84b740 1 --2- 172.21.15.86:0/1588442552 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x555968bcfff0 0x555968bd2410 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.003+0000 7fb45e84b740 1 --2- 172.21.15.86:0/1588442552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555968b4fe80 0x555968b52ce0 unknown :-1 s=CLOSED pgs=1088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.003+0000 7fb45e84b740 1 --2- 172.21.15.86:0/1588442552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555968b4f4d0 0x555968b527a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.003+0000 7fb4397fa640 0 client.17436 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:42:39.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.003+0000 7fb45e84b740 1 -- 172.21.15.86:0/1588442552 >> 172.21.15.86:0/1588442552 conn(0x555968b49ba0 msgr2=0x555968b4afc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:39.008 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.003+0000 7fb45e84b740 1 -- 172.21.15.86:0/1588442552 shutdown_connections 2024-04-21T08:42:39.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.003+0000 7fb45e84b740 1 -- 172.21.15.86:0/1588442552 wait complete. 2024-04-21T08:42:39.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.003+0000 7fb45e84b740 20 client.17436 trim_cache size 0 max 16384 2024-04-21T08:42:39.009 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:42:39.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.003+0000 7fb45bc62640 1 -- 172.21.15.86:0/1015848837 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb43c0a3d20 msgr2=0x7fb43c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:39.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.003+0000 7fb45bc62640 1 --2- 172.21.15.86:0/1015848837 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb43c0a3d20 0x7fb43c0a61e0 secure :-1 s=READY pgs=167 cs=0 l=1 rev1=1 crypto rx=0x7fb44c01d000 tx=0x7fb44c019000 comp rx=0 tx=0).stop 2024-04-21T08:42:39.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.003+0000 7fb45bc62640 1 -- 172.21.15.86:0/1015848837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb45410c900 msgr2=0x7fb4541a32f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:39.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.003+0000 7fb45bc62640 1 --2- 172.21.15.86:0/1015848837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb45410c900 0x7fb4541a32f0 secure :-1 s=READY pgs=1084 cs=0 l=1 rev1=1 crypto rx=0x7fb444009230 tx=0x7fb44402c880 comp rx=0 tx=0).stop 2024-04-21T08:42:39.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.003+0000 7fb45bc62640 1 -- 172.21.15.86:0/1015848837 shutdown_connections 2024-04-21T08:42:39.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.003+0000 7fb45bc62640 1 --2- 172.21.15.86:0/1015848837 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb43c0a3d20 0x7fb43c0a61e0 unknown :-1 s=CLOSED pgs=167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.003+0000 7fb45bc62640 1 --2- 172.21.15.86:0/1015848837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb454112910 0x7fb4541a7ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.003+0000 7fb45bc62640 1 --2- 172.21.15.86:0/1015848837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb45410d2c0 0x7fb4541a3830 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.003+0000 7fb45bc62640 1 --2- 172.21.15.86:0/1015848837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb45410c900 0x7fb4541a32f0 unknown :-1 s=CLOSED pgs=1084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.003+0000 7fb45bc62640 1 -- 172.21.15.86:0/1015848837 >> 172.21.15.86:0/1015848837 conn(0x7fb45406cf80 msgr2=0x7fb45410fa90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:39.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.004+0000 7fb45bc62640 1 -- 172.21.15.86:0/1015848837 shutdown_connections 2024-04-21T08:42:39.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.004+0000 7fb45bc62640 1 -- 172.21.15.86:0/1015848837 wait complete. 2024-04-21T08:42:39.013 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:42:39.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.113+0000 7f16003b1640 1 -- 172.21.15.86:0/1235212337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15f8101b40 msgr2=0x7f15f810dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:39.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.113+0000 7f16003b1640 1 --2- 172.21.15.86:0/1235212337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15f8101b40 0x7f15f810dc80 secure :-1 s=READY pgs=1089 cs=0 l=1 rev1=1 crypto rx=0x7f15f0009cd0 tx=0x7f15f002c100 comp rx=0 tx=0).stop 2024-04-21T08:42:39.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.113+0000 7f16003b1640 1 -- 172.21.15.86:0/1235212337 shutdown_connections 2024-04-21T08:42:39.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.113+0000 7f16003b1640 1 --2- 172.21.15.86:0/1235212337 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15f8101b40 0x7f15f810dc80 unknown :-1 s=CLOSED pgs=1089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.113+0000 7f16003b1640 1 --2- 172.21.15.86:0/1235212337 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15f81011a0 0x7f15f8101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.113+0000 7f16003b1640 1 --2- 172.21.15.86:0/1235212337 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15f8107030 0x7f15f8107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.113+0000 7f16003b1640 1 -- 172.21.15.86:0/1235212337 >> 172.21.15.86:0/1235212337 conn(0x7f15f8076c50 msgr2=0x7f15f8077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:39.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.113+0000 7f16003b1640 1 -- 172.21.15.86:0/1235212337 shutdown_connections 2024-04-21T08:42:39.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.113+0000 7f16003b1640 1 -- 172.21.15.86:0/1235212337 wait complete. 2024-04-21T08:42:39.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.114+0000 7f16003b1640 1 Processor -- start 2024-04-21T08:42:39.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.114+0000 7f16003b1640 1 -- start start 2024-04-21T08:42:39.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.114+0000 7f16003b1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15f81011a0 0x7f15f819e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:39.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.114+0000 7f16003b1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15f8101b40 0x7f15f819e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:39.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.114+0000 7f16003b1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15f8107030 0x7f15f81985c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:39.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.114+0000 7f16003b1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f15f8112260 con 0x7f15f8107030 2024-04-21T08:42:39.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.114+0000 7f16003b1640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f15f81120e0 con 0x7f15f8101b40 2024-04-21T08:42:39.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.114+0000 7f16003b1640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f15f81123e0 con 0x7f15f81011a0 2024-04-21T08:42:39.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.114+0000 7f15fe94e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15f8107030 0x7f15f81985c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:39.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.114+0000 7f15fe94e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15f8107030 0x7f15f81985c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33496/0 (socket says 172.21.15.86:33496) 2024-04-21T08:42:39.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.114+0000 7f15fe94e640 1 -- 172.21.15.86:0/3364046926 learned_addr learned my addr 172.21.15.86:0/3364046926 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:39.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.114+0000 7f15fd94c640 1 --2- 172.21.15.86:0/3364046926 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15f8101b40 0x7f15f819e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:39.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.114+0000 7f15fe94e640 1 -- 172.21.15.86:0/3364046926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15f81011a0 msgr2=0x7f15f819e410 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T08:42:39.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.114+0000 7f15fe94e640 1 --2- 172.21.15.86:0/3364046926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15f81011a0 0x7f15f819e410 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.114+0000 7f15fe94e640 1 -- 172.21.15.86:0/3364046926 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15f8101b40 msgr2=0x7f15f819e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:39.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.114+0000 7f15fe94e640 1 --2- 172.21.15.86:0/3364046926 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15f8101b40 0x7f15f819e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.115+0000 7f15fe94e640 1 -- 172.21.15.86:0/3364046926 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f15f8198d40 con 0x7f15f8107030 2024-04-21T08:42:39.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.115+0000 7f15fe94e640 1 --2- 172.21.15.86:0/3364046926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15f8107030 0x7f15f81985c0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f15f0033000 tx=0x7f15f0035ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:39.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.115+0000 7f15ef7fe640 1 -- 172.21.15.86:0/3364046926 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f0039040 con 0x7f15f8107030 2024-04-21T08:42:39.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.115+0000 7f15ef7fe640 1 -- 172.21.15.86:0/3364046926 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15f0044070 con 0x7f15f8107030 2024-04-21T08:42:39.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.115+0000 7f16003b1640 1 -- 172.21.15.86:0/3364046926 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f15f8198fd0 con 0x7f15f8107030 2024-04-21T08:42:39.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.115+0000 7f15ef7fe640 1 -- 172.21.15.86:0/3364046926 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f00042a0 con 0x7f15f8107030 2024-04-21T08:42:39.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.115+0000 7f16003b1640 1 -- 172.21.15.86:0/3364046926 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f15f81a52d0 con 0x7f15f8107030 2024-04-21T08:42:39.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.118+0000 7f1602f9c740 1 Processor -- start 2024-04-21T08:42:39.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.118+0000 7f15ef7fe640 1 -- 172.21.15.86:0/3364046926 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f15f0004440 con 0x7f15f8107030 2024-04-21T08:42:39.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.118+0000 7f1602f9c740 1 -- start start 2024-04-21T08:42:39.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.118+0000 7f1602f9c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bed00944d0 0x55bed00948b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:39.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.118+0000 7f1602f9c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bed0094e80 0x55bed009d430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:39.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.118+0000 7f1602f9c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bed009da00 0x55bed009fe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:39.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.118+0000 7f1602f9c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55becfff91f0 con 0x55bed0094e80 2024-04-21T08:42:39.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.118+0000 7f1602f9c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55becff964b0 con 0x55bed009da00 2024-04-21T08:42:39.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.118+0000 7f1602f9c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55becfff51b0 con 0x55bed00944d0 2024-04-21T08:42:39.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.118+0000 7f15fe14d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bed0094e80 0x55bed009d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:39.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.118+0000 7f15fe14d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bed0094e80 0x55bed009d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33510/0 (socket says 172.21.15.86:33510) 2024-04-21T08:42:39.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.118+0000 7f15fe14d640 1 -- 172.21.15.86:0/3131823043 learned_addr learned my addr 172.21.15.86:0/3131823043 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:39.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.119+0000 7f15fd94c640 1 --2- 172.21.15.86:0/3131823043 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bed009da00 0x55bed009fe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:39.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.119+0000 7f15ef7fe640 1 --2- 172.21.15.86:0/3364046926 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f15e40a3d20 0x7f15e40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:39.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.119+0000 7f15fe14d640 1 -- 172.21.15.86:0/3131823043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bed00944d0 msgr2=0x55bed00948b0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:42:39.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.119+0000 7f15fe14d640 1 --2- 172.21.15.86:0/3131823043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bed00944d0 0x55bed00948b0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.119+0000 7f15fe14d640 1 -- 172.21.15.86:0/3131823043 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bed009da00 msgr2=0x55bed009fe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:39.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.119+0000 7f15fe14d640 1 --2- 172.21.15.86:0/3131823043 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bed009da00 0x55bed009fe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.119+0000 7f15fe14d640 1 -- 172.21.15.86:0/3131823043 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55becff881b0 con 0x55bed0094e80 2024-04-21T08:42:39.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.119+0000 7f15fe94e640 1 --2- 172.21.15.86:0/3364046926 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f15e40a3d20 0x7f15e40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:39.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.119+0000 7f15ef7fe640 1 -- 172.21.15.86:0/3364046926 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f15f00f2fe0 con 0x7f15f8107030 2024-04-21T08:42:39.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.119+0000 7f15fe14d640 1 --2- 172.21.15.86:0/3131823043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bed0094e80 0x55bed009d430 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f15e800bb40 tx=0x7f15e800d2e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf86e9400661226 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:39.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.119+0000 7f15ecbfe640 1 -- 172.21.15.86:0/3131823043 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e8014030 con 0x55bed0094e80 2024-04-21T08:42:39.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.119+0000 7f15ecbfe640 1 -- 172.21.15.86:0/3131823043 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15e8004020 con 0x55bed0094e80 2024-04-21T08:42:39.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.119+0000 7f15ecbfe640 1 -- 172.21.15.86:0/3131823043 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e8003da0 con 0x55bed0094e80 2024-04-21T08:42:39.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.120+0000 7f1602f9c740 1 -- 172.21.15.86:0/3131823043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bed0094e80 msgr2=0x55bed009d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:39.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.120+0000 7f15fe94e640 1 --2- 172.21.15.86:0/3364046926 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f15e40a3d20 0x7f15e40a61e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f15f00bb180 tx=0x7f15f0034840 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:39.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.120+0000 7f1602f9c740 1 --2- 172.21.15.86:0/3131823043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bed0094e80 0x55bed009d430 secure :-1 s=READY pgs=1091 cs=0 l=1 rev1=1 crypto rx=0x7f15e800bb40 tx=0x7f15e800d2e0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.120+0000 7f1602f9c740 1 -- 172.21.15.86:0/3131823043 shutdown_connections 2024-04-21T08:42:39.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.120+0000 7f1602f9c740 1 --2- 172.21.15.86:0/3131823043 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bed009da00 0x55bed009fe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.120+0000 7f1602f9c740 1 --2- 172.21.15.86:0/3131823043 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bed0094e80 0x55bed009d430 unknown :-1 s=CLOSED pgs=1091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.120+0000 7f1602f9c740 1 --2- 172.21.15.86:0/3131823043 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bed00944d0 0x55bed00948b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.120+0000 7f1602f9c740 1 -- 172.21.15.86:0/3131823043 >> 172.21.15.86:0/3131823043 conn(0x55bed008eba0 msgr2=0x55bed008ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:39.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.120+0000 7f1602f9c740 1 -- 172.21.15.86:0/3131823043 shutdown_connections 2024-04-21T08:42:39.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.120+0000 7f1602f9c740 1 -- 172.21.15.86:0/3131823043 wait complete. 2024-04-21T08:42:39.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.120+0000 7f1602f9c740 1 -- start start 2024-04-21T08:42:39.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.120+0000 7f1602f9c740 1 Processor -- start 2024-04-21T08:42:39.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.121+0000 7f1602f9c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bed00944d0 0x55bed009d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:39.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.121+0000 7f1602f9c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bed0094e80 0x55bed009a5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:39.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.121+0000 7f1602f9c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bed009da00 0x55bed009ab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:39.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.121+0000 7f1602f9c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bed00a0970 con 0x55bed009da00 2024-04-21T08:42:39.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.121+0000 7f1602f9c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55becffb2500 con 0x55bed00944d0 2024-04-21T08:42:39.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.121+0000 7f1602f9c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55bed00a0af0 con 0x55bed0094e80 2024-04-21T08:42:39.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.121+0000 7f1602f9c740 10 client.0 _ll_register_callbacks cb 0x55becff80b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:39.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.121+0000 7f15fd94c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bed009da00 0x55bed009ab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:39.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.121+0000 7f15fd94c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bed009da00 0x55bed009ab30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33516/0 (socket says 172.21.15.86:33516) 2024-04-21T08:42:39.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.121+0000 7f15fd94c640 1 -- 172.21.15.86:0/1949466718 learned_addr learned my addr 172.21.15.86:0/1949466718 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:39.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.121+0000 7f15fd94c640 1 --2- 172.21.15.86:0/1949466718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bed00944d0 0x55bed009d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:39.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.121+0000 7f15fe14d640 1 --2- 172.21.15.86:0/1949466718 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bed0094e80 0x55bed009a5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:39.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.121+0000 7f15fd94c640 1 -- 172.21.15.86:0/1949466718 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bed0094e80 msgr2=0x55bed009a5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:39.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.121+0000 7f15fd94c640 1 --2- 172.21.15.86:0/1949466718 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bed0094e80 0x55bed009a5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.121+0000 7f15fd94c640 1 -- 172.21.15.86:0/1949466718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bed00944d0 msgr2=0x55bed009d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:39.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.121+0000 7f15fd94c640 1 --2- 172.21.15.86:0/1949466718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bed00944d0 0x55bed009d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.121+0000 7f15fd94c640 1 -- 172.21.15.86:0/1949466718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55becfd0b3f0 con 0x55bed009da00 2024-04-21T08:42:39.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.121+0000 7f15fd94c640 1 --2- 172.21.15.86:0/1949466718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bed009da00 0x55bed009ab30 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f15f400a000 tx=0x7f15f4009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b01ea5be97c889a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:39.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.121+0000 7f15e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:39.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.121+0000 7f15e1ffb640 1 -- 172.21.15.86:0/1949466718 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f4018e30 con 0x55bed009da00 2024-04-21T08:42:39.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.121+0000 7f15e1ffb640 1 -- 172.21.15.86:0/1949466718 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f15f40180d0 con 0x55bed009da00 2024-04-21T08:42:39.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.121+0000 7f1602f9c740 1 -- 172.21.15.86:0/1949466718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15e8004020 con 0x55bed009da00 2024-04-21T08:42:39.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.121+0000 7f15e1ffb640 1 -- 172.21.15.86:0/1949466718 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f15f4028050 con 0x55bed009da00 2024-04-21T08:42:39.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.121+0000 7f1602f9c740 10 client.17448 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:39.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.121+0000 7f1602f9c740 1 -- 172.21.15.86:0/1949466718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55becfdd57d0 con 0x55bed009da00 2024-04-21T08:42:39.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.122+0000 7f15e1ffb640 1 -- 172.21.15.86:0/1949466718 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15e8004020 con 0x55bed009da00 2024-04-21T08:42:39.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.122+0000 7f15e1ffb640 1 -- 172.21.15.86:0/1949466718 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f15f4015420 con 0x55bed009da00 2024-04-21T08:42:39.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.122+0000 7f1602f9c740 10 client.17448 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:39.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.122+0000 7f1602f9c740 10 client.17448 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:39.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.122+0000 7f1602f9c740 20 client.17448 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:39.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.122+0000 7f1602f9c740 1 --2- 172.21.15.86:0/1949466718 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55bed00adc70 0x55bed00b00f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:39.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.122+0000 7f1602f9c740 4 client.17448 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:39.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.122+0000 7f1602f9c740 1 -- 172.21.15.86:0/1949466718 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55becff98930 con 0x55bed00adc70 2024-04-21T08:42:39.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.122+0000 7f15fe14d640 1 --2- 172.21.15.86:0/1949466718 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55bed00adc70 0x55bed00b00f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:39.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.122+0000 7f15e1ffb640 10 client.17448 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:39.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.122+0000 7f15fe14d640 1 --2- 172.21.15.86:0/1949466718 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55bed00adc70 0x55bed00b00f0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aece71c62beb099 server_cookie=eb252aeb7db8ade9 in_seq=0 out_seq=0 2024-04-21T08:42:39.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.123+0000 7f15e1ffb640 1 -- 172.21.15.86:0/1949466718 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55becff98930 con 0x55bed00adc70 2024-04-21T08:42:39.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.123+0000 7f15e1ffb640 10 client.17448 handle_command_reply: tid=0 2024-04-21T08:42:39.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.123+0000 7f1602f9c740 1 client.17448 shutdown 2024-04-21T08:42:39.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.123+0000 7f1602f9c740 1 -- 172.21.15.86:0/1949466718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bed009da00 msgr2=0x55bed009ab30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:39.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.123+0000 7f1602f9c740 1 --2- 172.21.15.86:0/1949466718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bed009da00 0x55bed009ab30 secure :-1 s=READY pgs=1092 cs=0 l=1 rev1=1 crypto rx=0x7f15f400a000 tx=0x7f15f4009740 comp rx=0 tx=0).stop 2024-04-21T08:42:39.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.123+0000 7f1602f9c740 1 -- 172.21.15.86:0/1949466718 shutdown_connections 2024-04-21T08:42:39.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.123+0000 7f1602f9c740 1 --2- 172.21.15.86:0/1949466718 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55bed00adc70 0x55bed00b00f0 crc :-1 s=READY pgs=143 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.123+0000 7f1602f9c740 1 --2- 172.21.15.86:0/1949466718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bed009da00 0x55bed009ab30 unknown :-1 s=CLOSED pgs=1092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.123+0000 7f1602f9c740 1 --2- 172.21.15.86:0/1949466718 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bed0094e80 0x55bed009a5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.123+0000 7f15e1ffb640 0 client.17448 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:39.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.123+0000 7f1602f9c740 1 --2- 172.21.15.86:0/1949466718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bed00944d0 0x55bed009d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.123+0000 7f1602f9c740 1 -- 172.21.15.86:0/1949466718 >> 172.21.15.86:0/1949466718 conn(0x55bed008eba0 msgr2=0x55bed00908f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:39.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.123+0000 7f1602f9c740 1 -- 172.21.15.86:0/1949466718 shutdown_connections 2024-04-21T08:42:39.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.123+0000 7f1602f9c740 1 -- 172.21.15.86:0/1949466718 wait complete. 2024-04-21T08:42:39.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.123+0000 7f1602f9c740 20 client.17448 trim_cache size 0 max 16384 2024-04-21T08:42:39.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.136+0000 7f1602f9c740 1 Processor -- start 2024-04-21T08:42:39.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.136+0000 7f1602f9c740 1 -- start start 2024-04-21T08:42:39.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.137+0000 7f1602f9c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bed008eba0 0x55bed00a9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:39.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.137+0000 7f1602f9c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bed00944d0 0x55bed009cd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:39.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.137+0000 7f1602f9c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bed0094e80 0x55bed009d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:39.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.137+0000 7f1602f9c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f15f81af1f0 con 0x55bed00944d0 2024-04-21T08:42:39.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.137+0000 7f1602f9c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f15f81b0d30 con 0x55bed0094e80 2024-04-21T08:42:39.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.137+0000 7f1602f9c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f15f81af8c0 con 0x55bed008eba0 2024-04-21T08:42:39.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.137+0000 7f15fe14d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bed00944d0 0x55bed009cd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:39.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.137+0000 7f15fe14d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bed00944d0 0x55bed009cd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33532/0 (socket says 172.21.15.86:33532) 2024-04-21T08:42:39.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.137+0000 7f15fe14d640 1 -- 172.21.15.86:0/2210301949 learned_addr learned my addr 172.21.15.86:0/2210301949 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:39.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.137+0000 7f15fe14d640 1 -- 172.21.15.86:0/2210301949 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bed008eba0 msgr2=0x55bed00a9890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:39.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.137+0000 7f15fd94c640 1 --2- 172.21.15.86:0/2210301949 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bed0094e80 0x55bed009d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:39.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.137+0000 7f15fe14d640 1 --2- 172.21.15.86:0/2210301949 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bed008eba0 0x55bed00a9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.137+0000 7f15fe14d640 1 -- 172.21.15.86:0/2210301949 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bed0094e80 msgr2=0x55bed009d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:39.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.137+0000 7f15fe14d640 1 --2- 172.21.15.86:0/2210301949 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bed0094e80 0x55bed009d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.137+0000 7f15fe14d640 1 -- 172.21.15.86:0/2210301949 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bed0094c20 con 0x55bed00944d0 2024-04-21T08:42:39.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.137+0000 7f15fe14d640 1 --2- 172.21.15.86:0/2210301949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bed00944d0 0x55bed009cd20 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f15e800d7e0 tx=0x7f15e800b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d890bb9ca9f4a5e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:39.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.137+0000 7f15e3fff640 1 -- 172.21.15.86:0/2210301949 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e8029020 con 0x55bed00944d0 2024-04-21T08:42:39.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.137+0000 7f15e3fff640 1 -- 172.21.15.86:0/2210301949 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15e800dd10 con 0x55bed00944d0 2024-04-21T08:42:39.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.137+0000 7f15e3fff640 1 -- 172.21.15.86:0/2210301949 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15e8014040 con 0x55bed00944d0 2024-04-21T08:42:39.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.138+0000 7f1602f9c740 1 -- 172.21.15.86:0/2210301949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bed00944d0 msgr2=0x55bed009cd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:39.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.138+0000 7f1602f9c740 1 --2- 172.21.15.86:0/2210301949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bed00944d0 0x55bed009cd20 secure :-1 s=READY pgs=1093 cs=0 l=1 rev1=1 crypto rx=0x7f15e800d7e0 tx=0x7f15e800b840 comp rx=0 tx=0).stop 2024-04-21T08:42:39.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.138+0000 7f1602f9c740 1 -- 172.21.15.86:0/2210301949 shutdown_connections 2024-04-21T08:42:39.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.138+0000 7f1602f9c740 1 --2- 172.21.15.86:0/2210301949 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bed0094e80 0x55bed009d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.138+0000 7f1602f9c740 1 --2- 172.21.15.86:0/2210301949 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bed00944d0 0x55bed009cd20 unknown :-1 s=CLOSED pgs=1093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.138+0000 7f1602f9c740 1 --2- 172.21.15.86:0/2210301949 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bed008eba0 0x55bed00a9890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.138+0000 7f1602f9c740 1 -- 172.21.15.86:0/2210301949 >> 172.21.15.86:0/2210301949 conn(0x55bed009ac90 msgr2=0x55bed0095b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:39.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.138+0000 7f1602f9c740 1 -- 172.21.15.86:0/2210301949 shutdown_connections 2024-04-21T08:42:39.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.138+0000 7f1602f9c740 1 -- 172.21.15.86:0/2210301949 wait complete. 2024-04-21T08:42:39.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.138+0000 7f1602f9c740 1 -- start start 2024-04-21T08:42:39.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.138+0000 7f1602f9c740 1 Processor -- start 2024-04-21T08:42:39.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.138+0000 7f1602f9c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bed008eba0 0x55bed00a16e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:39.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.138+0000 7f1602f9c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bed00944d0 0x55bed00a1c20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:39.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.138+0000 7f1602f9c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bed0094e80 0x55bed00a2160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:39.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.138+0000 7f1602f9c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f15f81b0660 con 0x55bed008eba0 2024-04-21T08:42:39.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.138+0000 7f1602f9c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f15f81aff90 con 0x55bed0094e80 2024-04-21T08:42:39.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.138+0000 7f1602f9c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f15f81ad1b0 con 0x55bed00944d0 2024-04-21T08:42:39.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.138+0000 7f1602f9c740 10 client.0 _ll_register_callbacks cb 0x7f15f4018870 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:39.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.138+0000 7f15fd94c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bed008eba0 0x55bed00a16e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:39.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.138+0000 7f15fd94c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bed008eba0 0x55bed00a16e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33538/0 (socket says 172.21.15.86:33538) 2024-04-21T08:42:39.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.138+0000 7f15fd94c640 1 -- 172.21.15.86:0/2218818007 learned_addr learned my addr 172.21.15.86:0/2218818007 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:39.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.139+0000 7f15fd94c640 1 -- 172.21.15.86:0/2218818007 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bed00944d0 msgr2=0x55bed00a1c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:39.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.139+0000 7f15fd94c640 1 --2- 172.21.15.86:0/2218818007 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bed00944d0 0x55bed00a1c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.139+0000 7f15fd94c640 1 -- 172.21.15.86:0/2218818007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bed0094e80 msgr2=0x55bed00a2160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:39.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.139+0000 7f15fd94c640 1 --2- 172.21.15.86:0/2218818007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bed0094e80 0x55bed00a2160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.139+0000 7f15fd94c640 1 -- 172.21.15.86:0/2218818007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bed0113230 con 0x55bed008eba0 2024-04-21T08:42:39.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.139+0000 7f15fd94c640 1 --2- 172.21.15.86:0/2218818007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bed008eba0 0x55bed00a16e0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f15f40093f0 tx=0x7f15f4009580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6b8b911381bea1e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:39.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.139+0000 7f15e1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:39.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.139+0000 7f15e1ffb640 1 -- 172.21.15.86:0/2218818007 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f15f4031060 con 0x55bed008eba0 2024-04-21T08:42:39.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.139+0000 7f15e1ffb640 1 -- 172.21.15.86:0/2218818007 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f15f40159e0 con 0x55bed008eba0 2024-04-21T08:42:39.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.139+0000 7f1602f9c740 1 -- 172.21.15.86:0/2218818007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f15e800dd10 con 0x55bed008eba0 2024-04-21T08:42:39.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.139+0000 7f15e1ffb640 1 -- 172.21.15.86:0/2218818007 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f15f4038050 con 0x55bed008eba0 2024-04-21T08:42:39.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.139+0000 7f1602f9c740 10 client.17454 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:39.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.139+0000 7f1602f9c740 1 -- 172.21.15.86:0/2218818007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bed0114440 con 0x55bed008eba0 2024-04-21T08:42:39.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.139+0000 7f15e1ffb640 1 -- 172.21.15.86:0/2218818007 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f15e800dd10 con 0x55bed008eba0 2024-04-21T08:42:39.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.139+0000 7f15e1ffb640 1 -- 172.21.15.86:0/2218818007 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f15f4016630 con 0x55bed008eba0 2024-04-21T08:42:39.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.139+0000 7f1602f9c740 10 client.17454 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:39.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.139+0000 7f1602f9c740 10 client.17454 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:39.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.139+0000 7f1602f9c740 20 client.17454 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:39.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.139+0000 7f1602f9c740 1 --2- 172.21.15.86:0/2218818007 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55bed009da00 0x55bed0116c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:39.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.139+0000 7f1602f9c740 4 client.17454 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:42:39.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.139+0000 7f1602f9c740 1 -- 172.21.15.86:0/2218818007 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bed0006c10 con 0x55bed009da00 2024-04-21T08:42:39.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.140+0000 7f15fe14d640 1 --2- 172.21.15.86:0/2218818007 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55bed009da00 0x55bed0116c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:39.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.140+0000 7f15e1ffb640 10 client.17454 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:39.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.140+0000 7f15fe14d640 1 --2- 172.21.15.86:0/2218818007 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55bed009da00 0x55bed0116c30 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=53910f6c62d4572e server_cookie=8f4f9f26ab77132c in_seq=0 out_seq=0 2024-04-21T08:42:39.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.140+0000 7f15e1ffb640 1 -- 172.21.15.86:0/2218818007 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bed0006c10 con 0x55bed009da00 2024-04-21T08:42:39.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.140+0000 7f15e1ffb640 10 client.17454 handle_command_reply: tid=0 2024-04-21T08:42:39.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.140+0000 7f1602f9c740 1 client.17454 shutdown 2024-04-21T08:42:39.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.140+0000 7f1602f9c740 1 -- 172.21.15.86:0/2218818007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bed008eba0 msgr2=0x55bed00a16e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:39.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.140+0000 7f1602f9c740 1 --2- 172.21.15.86:0/2218818007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bed008eba0 0x55bed00a16e0 secure :-1 s=READY pgs=1094 cs=0 l=1 rev1=1 crypto rx=0x7f15f40093f0 tx=0x7f15f4009580 comp rx=0 tx=0).stop 2024-04-21T08:42:39.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.141+0000 7f1602f9c740 1 -- 172.21.15.86:0/2218818007 shutdown_connections 2024-04-21T08:42:39.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.141+0000 7f1602f9c740 1 --2- 172.21.15.86:0/2218818007 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55bed009da00 0x55bed0116c30 crc :-1 s=READY pgs=144 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.141+0000 7f1602f9c740 1 --2- 172.21.15.86:0/2218818007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bed0094e80 0x55bed00a2160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.141+0000 7f1602f9c740 1 --2- 172.21.15.86:0/2218818007 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bed00944d0 0x55bed00a1c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.141+0000 7f15e1ffb640 0 client.17454 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:39.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.141+0000 7f1602f9c740 1 --2- 172.21.15.86:0/2218818007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bed008eba0 0x55bed00a16e0 unknown :-1 s=CLOSED pgs=1094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.141+0000 7f1602f9c740 1 -- 172.21.15.86:0/2218818007 >> 172.21.15.86:0/2218818007 conn(0x55bed009ac90 msgr2=0x55bed008d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:39.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.141+0000 7f1602f9c740 1 -- 172.21.15.86:0/2218818007 shutdown_connections 2024-04-21T08:42:39.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.141+0000 7f1602f9c740 1 -- 172.21.15.86:0/2218818007 wait complete. 2024-04-21T08:42:39.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.141+0000 7f1602f9c740 20 client.17454 trim_cache size 0 max 16384 2024-04-21T08:42:39.147 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:42:39.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.141+0000 7f16003b1640 1 -- 172.21.15.86:0/3364046926 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f15e40a3d20 msgr2=0x7f15e40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:39.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.141+0000 7f16003b1640 1 --2- 172.21.15.86:0/3364046926 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f15e40a3d20 0x7f15e40a61e0 secure :-1 s=READY pgs=168 cs=0 l=1 rev1=1 crypto rx=0x7f15f00bb180 tx=0x7f15f0034840 comp rx=0 tx=0).stop 2024-04-21T08:42:39.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.141+0000 7f16003b1640 1 -- 172.21.15.86:0/3364046926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15f8107030 msgr2=0x7f15f81985c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:39.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.141+0000 7f16003b1640 1 --2- 172.21.15.86:0/3364046926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15f8107030 0x7f15f81985c0 secure :-1 s=READY pgs=1090 cs=0 l=1 rev1=1 crypto rx=0x7f15f0033000 tx=0x7f15f0035ce0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.141+0000 7f16003b1640 1 -- 172.21.15.86:0/3364046926 shutdown_connections 2024-04-21T08:42:39.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.141+0000 7f16003b1640 1 --2- 172.21.15.86:0/3364046926 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f15e40a3d20 0x7f15e40a61e0 unknown :-1 s=CLOSED pgs=168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.141+0000 7f16003b1640 1 --2- 172.21.15.86:0/3364046926 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f15f8107030 0x7f15f81985c0 unknown :-1 s=CLOSED pgs=1090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.141+0000 7f16003b1640 1 --2- 172.21.15.86:0/3364046926 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f15f8101b40 0x7f15f819e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.141+0000 7f16003b1640 1 --2- 172.21.15.86:0/3364046926 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f15f81011a0 0x7f15f819e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:39.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.141+0000 7f16003b1640 1 -- 172.21.15.86:0/3364046926 >> 172.21.15.86:0/3364046926 conn(0x7f15f8076c50 msgr2=0x7f15f80ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:39.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.141+0000 7f16003b1640 1 -- 172.21.15.86:0/3364046926 shutdown_connections 2024-04-21T08:42:39.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:39.141+0000 7f16003b1640 1 -- 172.21.15.86:0/3364046926 wait complete. 2024-04-21T08:42:39.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:39 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/431114796' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:42:39.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:39 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/431114796' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:42:39.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:39 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/431114796' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:42:40.155 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:42:40.155 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:42:40.155 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:42:40.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.269+0000 7fc454bab640 1 -- 172.21.15.86:0/890359608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc450103b00 msgr2=0x7fc45010bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:40.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.269+0000 7fc454bab640 1 --2- 172.21.15.86:0/890359608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc450103b00 0x7fc45010bfe0 secure :-1 s=READY pgs=1095 cs=0 l=1 rev1=1 crypto rx=0x7fc438009900 tx=0x7fc43802bd60 comp rx=0 tx=0).stop 2024-04-21T08:42:40.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.269+0000 7fc454bab640 1 -- 172.21.15.86:0/890359608 shutdown_connections 2024-04-21T08:42:40.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.269+0000 7fc454bab640 1 --2- 172.21.15.86:0/890359608 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc45010c520 0x7fc45010e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.269+0000 7fc454bab640 1 --2- 172.21.15.86:0/890359608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc450103b00 0x7fc45010bfe0 unknown :-1 s=CLOSED pgs=1095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.269+0000 7fc454bab640 1 --2- 172.21.15.86:0/890359608 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4501031e0 0x7fc4501035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.269+0000 7fc454bab640 1 -- 172.21.15.86:0/890359608 >> 172.21.15.86:0/890359608 conn(0x7fc4500771d0 msgr2=0x7fc4500775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:40.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.269+0000 7fc454bab640 1 -- 172.21.15.86:0/890359608 shutdown_connections 2024-04-21T08:42:40.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.269+0000 7fc454bab640 1 -- 172.21.15.86:0/890359608 wait complete. 2024-04-21T08:42:40.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.270+0000 7fc454bab640 1 Processor -- start 2024-04-21T08:42:40.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.270+0000 7fc454bab640 1 -- start start 2024-04-21T08:42:40.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.270+0000 7fc454bab640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4501031e0 0x7fc450198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:40.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.270+0000 7fc454bab640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc450103b00 0x7fc450198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:40.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.270+0000 7fc454bab640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc45010c520 0x7fc45019cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:40.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.270+0000 7fc454bab640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc4501121b0 con 0x7fc450103b00 2024-04-21T08:42:40.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.270+0000 7fc454bab640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc450112030 con 0x7fc45010c520 2024-04-21T08:42:40.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.270+0000 7fc454bab640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc450112330 con 0x7fc4501031e0 2024-04-21T08:42:40.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.270+0000 7fc447fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc450103b00 0x7fc450198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:40.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.270+0000 7fc447fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc450103b00 0x7fc450198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47632/0 (socket says 172.21.15.86:47632) 2024-04-21T08:42:40.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.270+0000 7fc447fff640 1 -- 172.21.15.86:0/344939445 learned_addr learned my addr 172.21.15.86:0/344939445 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:40.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.270+0000 7fc44e59c640 1 --2- 172.21.15.86:0/344939445 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4501031e0 0x7fc450198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:40.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.270+0000 7fc44ed9d640 1 --2- 172.21.15.86:0/344939445 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc45010c520 0x7fc45019cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:40.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.271+0000 7fc447fff640 1 -- 172.21.15.86:0/344939445 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4501031e0 msgr2=0x7fc450198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:40.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.271+0000 7fc447fff640 1 --2- 172.21.15.86:0/344939445 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4501031e0 0x7fc450198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.271+0000 7fc447fff640 1 -- 172.21.15.86:0/344939445 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc45010c520 msgr2=0x7fc45019cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:40.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.271+0000 7fc447fff640 1 --2- 172.21.15.86:0/344939445 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc45010c520 0x7fc45019cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.271+0000 7fc447fff640 1 -- 172.21.15.86:0/344939445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc45019d700 con 0x7fc450103b00 2024-04-21T08:42:40.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.271+0000 7fc447fff640 1 --2- 172.21.15.86:0/344939445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc450103b00 0x7fc450198c50 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fc438005940 tx=0x7fc43802d920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:40.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.271+0000 7fc4477fe640 1 -- 172.21.15.86:0/344939445 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc438043020 con 0x7fc450103b00 2024-04-21T08:42:40.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.271+0000 7fc454bab640 1 -- 172.21.15.86:0/344939445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc45019d990 con 0x7fc450103b00 2024-04-21T08:42:40.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.271+0000 7fc4477fe640 1 -- 172.21.15.86:0/344939445 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc43802dd20 con 0x7fc450103b00 2024-04-21T08:42:40.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.271+0000 7fc4477fe640 1 -- 172.21.15.86:0/344939445 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc438003b80 con 0x7fc450103b00 2024-04-21T08:42:40.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.271+0000 7fc454bab640 1 -- 172.21.15.86:0/344939445 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc4501a5280 con 0x7fc450103b00 2024-04-21T08:42:40.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.273+0000 7fc4477fe640 1 -- 172.21.15.86:0/344939445 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc438046020 con 0x7fc450103b00 2024-04-21T08:42:40.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.273+0000 7fc45779d740 1 Processor -- start 2024-04-21T08:42:40.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.274+0000 7fc45779d740 1 -- start start 2024-04-21T08:42:40.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.274+0000 7fc45779d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f9b807c0 0x5640f9b80ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:40.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.274+0000 7fc45779d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640f9b81170 0x5640f9b89ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:40.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.274+0000 7fc4477fe640 1 --2- 172.21.15.86:0/344939445 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc43c0a3d20 0x7fc43c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:40.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.274+0000 7fc45779d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640f9b8a270 0x5640f9b8a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:40.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.274+0000 7fc45779d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5640f9ae4f00 con 0x5640f9b807c0 2024-04-21T08:42:40.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.274+0000 7fc45779d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5640f9a82590 con 0x5640f9b81170 2024-04-21T08:42:40.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.274+0000 7fc45779d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5640f9ae0ec0 con 0x5640f9b8a270 2024-04-21T08:42:40.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.274+0000 7fc44ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f9b807c0 0x5640f9b80ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:40.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.274+0000 7fc44ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f9b807c0 0x5640f9b80ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47642/0 (socket says 172.21.15.86:47642) 2024-04-21T08:42:40.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.274+0000 7fc44e59c640 1 --2- 172.21.15.86:0/344939445 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc43c0a3d20 0x7fc43c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:40.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.274+0000 7fc44ed9d640 1 -- 172.21.15.86:0/1102816475 learned_addr learned my addr 172.21.15.86:0/1102816475 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:40.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.274+0000 7fc4477fe640 1 -- 172.21.15.86:0/344939445 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fc4380f2c80 con 0x7fc450103b00 2024-04-21T08:42:40.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.274+0000 7fc447fff640 1 --2- 172.21.15.86:0/1102816475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640f9b81170 0x5640f9b89ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:40.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.274+0000 7fc44ed9d640 1 -- 172.21.15.86:0/1102816475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640f9b8a270 msgr2=0x5640f9b8a6d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:42:40.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.274+0000 7fc44ed9d640 1 --2- 172.21.15.86:0/1102816475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640f9b8a270 0x5640f9b8a6d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.274+0000 7fc44ed9d640 1 -- 172.21.15.86:0/1102816475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640f9b81170 msgr2=0x5640f9b89ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:40.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.274+0000 7fc44ed9d640 1 --2- 172.21.15.86:0/1102816475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640f9b81170 0x5640f9b89ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.274+0000 7fc44ed9d640 1 -- 172.21.15.86:0/1102816475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640f98bf7e0 con 0x5640f9b807c0 2024-04-21T08:42:40.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.274+0000 7fc447fff640 1 --2- 172.21.15.86:0/1102816475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640f9b81170 0x5640f9b89ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:40.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.274+0000 7fc44e59c640 1 --2- 172.21.15.86:0/344939445 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc43c0a3d20 0x7fc43c0a61e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc440005fd0 tx=0x7fc44000b390 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:40.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.274+0000 7fc44ed9d640 1 --2- 172.21.15.86:0/1102816475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f9b807c0 0x5640f9b80ba0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fc448007db0 tx=0x7fc44800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e01fabddfe1dbe8e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:40.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.275+0000 7fc444ff9640 1 -- 172.21.15.86:0/1102816475 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc448014070 con 0x5640f9b807c0 2024-04-21T08:42:40.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.275+0000 7fc444ff9640 1 -- 172.21.15.86:0/1102816475 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc44800ee30 con 0x5640f9b807c0 2024-04-21T08:42:40.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.275+0000 7fc444ff9640 1 -- 172.21.15.86:0/1102816475 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc448005280 con 0x5640f9b807c0 2024-04-21T08:42:40.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.275+0000 7fc45779d740 1 -- 172.21.15.86:0/1102816475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f9b807c0 msgr2=0x5640f9b80ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:40.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.275+0000 7fc45779d740 1 --2- 172.21.15.86:0/1102816475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f9b807c0 0x5640f9b80ba0 secure :-1 s=READY pgs=1097 cs=0 l=1 rev1=1 crypto rx=0x7fc448007db0 tx=0x7fc44800e720 comp rx=0 tx=0).stop 2024-04-21T08:42:40.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.275+0000 7fc45779d740 1 -- 172.21.15.86:0/1102816475 shutdown_connections 2024-04-21T08:42:40.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.275+0000 7fc45779d740 1 --2- 172.21.15.86:0/1102816475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640f9b8a270 0x5640f9b8a6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.275+0000 7fc45779d740 1 --2- 172.21.15.86:0/1102816475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640f9b81170 0x5640f9b89ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.275+0000 7fc45779d740 1 --2- 172.21.15.86:0/1102816475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f9b807c0 0x5640f9b80ba0 unknown :-1 s=CLOSED pgs=1097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.275+0000 7fc45779d740 1 -- 172.21.15.86:0/1102816475 >> 172.21.15.86:0/1102816475 conn(0x5640f9b7afb0 msgr2=0x5640f9b7b390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:40.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.276+0000 7fc45779d740 1 -- 172.21.15.86:0/1102816475 shutdown_connections 2024-04-21T08:42:40.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.276+0000 7fc45779d740 1 -- 172.21.15.86:0/1102816475 wait complete. 2024-04-21T08:42:40.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.276+0000 7fc45779d740 1 -- start start 2024-04-21T08:42:40.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.276+0000 7fc45779d740 1 Processor -- start 2024-04-21T08:42:40.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.276+0000 7fc45779d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640f9b807c0 0x5640f9b8dd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:40.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.276+0000 7fc45779d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f9b81170 0x5640f9b8e2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:40.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.276+0000 7fc45779d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640f9b8a270 0x5640f9b8e7f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:40.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.276+0000 7fc45779d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5640f9b911b0 con 0x5640f9b81170 2024-04-21T08:42:40.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.276+0000 7fc45779d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5640f9a9e580 con 0x5640f9b807c0 2024-04-21T08:42:40.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.276+0000 7fc45779d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5640f9b91330 con 0x5640f9b8a270 2024-04-21T08:42:40.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.276+0000 7fc45779d740 10 client.0 _ll_register_callbacks cb 0x5640f9aa4650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:40.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.276+0000 7fc447fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f9b81170 0x5640f9b8e2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:40.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.276+0000 7fc447fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f9b81170 0x5640f9b8e2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47646/0 (socket says 172.21.15.86:47646) 2024-04-21T08:42:40.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.276+0000 7fc447fff640 1 -- 172.21.15.86:0/865769132 learned_addr learned my addr 172.21.15.86:0/865769132 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:40.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.276+0000 7fc44e59c640 1 --2- 172.21.15.86:0/865769132 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640f9b807c0 0x5640f9b8dd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:40.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.277+0000 7fc447fff640 1 -- 172.21.15.86:0/865769132 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640f9b8a270 msgr2=0x5640f9b8e7f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:42:40.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.277+0000 7fc447fff640 1 --2- 172.21.15.86:0/865769132 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640f9b8a270 0x5640f9b8e7f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.277+0000 7fc447fff640 1 -- 172.21.15.86:0/865769132 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640f9b807c0 msgr2=0x5640f9b8dd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:40.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.277+0000 7fc447fff640 1 --2- 172.21.15.86:0/865769132 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640f9b807c0 0x5640f9b8dd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.277+0000 7fc447fff640 1 -- 172.21.15.86:0/865769132 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640f991ff70 con 0x5640f9b81170 2024-04-21T08:42:40.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.277+0000 7fc447fff640 1 --2- 172.21.15.86:0/865769132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f9b81170 0x5640f9b8e2b0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fc4380be6c0 tx=0x7fc4380be5c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9fdac1f4a67edf56 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:40.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.277+0000 7fc435ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:40.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.277+0000 7fc435ffb640 1 -- 172.21.15.86:0/865769132 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4380c2d70 con 0x5640f9b81170 2024-04-21T08:42:40.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.277+0000 7fc435ffb640 1 -- 172.21.15.86:0/865769132 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fc4380da1a0 con 0x5640f9b81170 2024-04-21T08:42:40.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.277+0000 7fc45779d740 1 -- 172.21.15.86:0/865769132 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc44800ee30 con 0x5640f9b81170 2024-04-21T08:42:40.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.277+0000 7fc435ffb640 1 -- 172.21.15.86:0/865769132 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc4380c7630 con 0x5640f9b81170 2024-04-21T08:42:40.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.277+0000 7fc45779d740 10 client.17466 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:40.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.277+0000 7fc45779d740 1 -- 172.21.15.86:0/865769132 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5640f97b7170 con 0x5640f9b81170 2024-04-21T08:42:40.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.277+0000 7fc435ffb640 1 -- 172.21.15.86:0/865769132 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc44800ee30 con 0x5640f9b81170 2024-04-21T08:42:40.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.277+0000 7fc435ffb640 1 -- 172.21.15.86:0/865769132 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc4380c71a0 con 0x5640f9b81170 2024-04-21T08:42:40.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.277+0000 7fc45779d740 10 client.17466 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:40.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.277+0000 7fc45779d740 10 client.17466 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:40.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.277+0000 7fc45779d740 20 client.17466 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:40.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.277+0000 7fc45779d740 1 --2- 172.21.15.86:0/865769132 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5640f9b9e5e0 0x5640f9ba0a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:40.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.277+0000 7fc45779d740 4 client.17466 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:40.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.277+0000 7fc45779d740 1 -- 172.21.15.86:0/865769132 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5640f9af2920 con 0x5640f9b9e5e0 2024-04-21T08:42:40.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.278+0000 7fc44e59c640 1 --2- 172.21.15.86:0/865769132 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5640f9b9e5e0 0x5640f9ba0a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:40.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.278+0000 7fc435ffb640 10 client.17466 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:40.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.278+0000 7fc44e59c640 1 --2- 172.21.15.86:0/865769132 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5640f9b9e5e0 0x5640f9ba0a30 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7d0fafaa2d6ecf50 server_cookie=5c99cfd079cf295a in_seq=0 out_seq=0 2024-04-21T08:42:40.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.279+0000 7fc435ffb640 1 -- 172.21.15.86:0/865769132 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5640f9af2920 con 0x5640f9b9e5e0 2024-04-21T08:42:40.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.279+0000 7fc435ffb640 10 client.17466 handle_command_reply: tid=0 2024-04-21T08:42:40.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.279+0000 7fc45779d740 1 client.17466 shutdown 2024-04-21T08:42:40.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.279+0000 7fc45779d740 1 -- 172.21.15.86:0/865769132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f9b81170 msgr2=0x5640f9b8e2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:40.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.279+0000 7fc45779d740 1 --2- 172.21.15.86:0/865769132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f9b81170 0x5640f9b8e2b0 secure :-1 s=READY pgs=1098 cs=0 l=1 rev1=1 crypto rx=0x7fc4380be6c0 tx=0x7fc4380be5c0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.279+0000 7fc45779d740 1 -- 172.21.15.86:0/865769132 shutdown_connections 2024-04-21T08:42:40.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.279+0000 7fc45779d740 1 --2- 172.21.15.86:0/865769132 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5640f9b9e5e0 0x5640f9ba0a30 crc :-1 s=READY pgs=145 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.279+0000 7fc45779d740 1 --2- 172.21.15.86:0/865769132 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640f9b8a270 0x5640f9b8e7f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.279+0000 7fc435ffb640 0 client.17466 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:40.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.279+0000 7fc45779d740 1 --2- 172.21.15.86:0/865769132 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f9b81170 0x5640f9b8e2b0 unknown :-1 s=CLOSED pgs=1098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.279+0000 7fc45779d740 1 --2- 172.21.15.86:0/865769132 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640f9b807c0 0x5640f9b8dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.279+0000 7fc45779d740 1 -- 172.21.15.86:0/865769132 >> 172.21.15.86:0/865769132 conn(0x5640f9b7afb0 msgr2=0x5640f9b7cd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:40.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.279+0000 7fc45779d740 1 -- 172.21.15.86:0/865769132 shutdown_connections 2024-04-21T08:42:40.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.279+0000 7fc45779d740 1 -- 172.21.15.86:0/865769132 wait complete. 2024-04-21T08:42:40.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.279+0000 7fc45779d740 20 client.17466 trim_cache size 0 max 16384 2024-04-21T08:42:40.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.297+0000 7fc45779d740 1 Processor -- start 2024-04-21T08:42:40.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.297+0000 7fc45779d740 1 -- start start 2024-04-21T08:42:40.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.297+0000 7fc45779d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640f9b807c0 0x5640f9b904a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:40.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.297+0000 7fc45779d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640f9b81170 0x5640f9b8ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:40.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.297+0000 7fc45779d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f9b8a270 0x5640f9b9a0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:40.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.297+0000 7fc45779d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc4501acfb0 con 0x5640f9b8a270 2024-04-21T08:42:40.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.297+0000 7fc45779d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc4501aeaf0 con 0x5640f9b81170 2024-04-21T08:42:40.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.297+0000 7fc45779d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc4501ad680 con 0x5640f9b807c0 2024-04-21T08:42:40.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.297+0000 7fc44ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f9b8a270 0x5640f9b9a0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:40.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.297+0000 7fc44ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f9b8a270 0x5640f9b9a0e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47662/0 (socket says 172.21.15.86:47662) 2024-04-21T08:42:40.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.297+0000 7fc44ed9d640 1 -- 172.21.15.86:0/1282680003 learned_addr learned my addr 172.21.15.86:0/1282680003 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:40.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.297+0000 7fc44e59c640 1 --2- 172.21.15.86:0/1282680003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640f9b807c0 0x5640f9b904a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:40.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.297+0000 7fc447fff640 1 --2- 172.21.15.86:0/1282680003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640f9b81170 0x5640f9b8ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:40.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.297+0000 7fc44ed9d640 1 -- 172.21.15.86:0/1282680003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640f9b807c0 msgr2=0x5640f9b904a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:40.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.297+0000 7fc44ed9d640 1 --2- 172.21.15.86:0/1282680003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640f9b807c0 0x5640f9b904a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.297+0000 7fc44ed9d640 1 -- 172.21.15.86:0/1282680003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640f9b81170 msgr2=0x5640f9b8ccb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:40.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.297+0000 7fc44ed9d640 1 --2- 172.21.15.86:0/1282680003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640f9b81170 0x5640f9b8ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.297+0000 7fc44ed9d640 1 -- 172.21.15.86:0/1282680003 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5640f9b80f10 con 0x5640f9b8a270 2024-04-21T08:42:40.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.297+0000 7fc44ed9d640 1 --2- 172.21.15.86:0/1282680003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f9b8a270 0x5640f9b9a0e0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fc4480075d0 tx=0x7fc448007600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d82857d520c3be3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:40.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.298+0000 7fc437fff640 1 -- 172.21.15.86:0/1282680003 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc44801d070 con 0x5640f9b8a270 2024-04-21T08:42:40.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.298+0000 7fc437fff640 1 -- 172.21.15.86:0/1282680003 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc448003e20 con 0x5640f9b8a270 2024-04-21T08:42:40.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.298+0000 7fc437fff640 1 -- 172.21.15.86:0/1282680003 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc448014040 con 0x5640f9b8a270 2024-04-21T08:42:40.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.298+0000 7fc45779d740 1 -- 172.21.15.86:0/1282680003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f9b8a270 msgr2=0x5640f9b9a0e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:40.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.298+0000 7fc45779d740 1 --2- 172.21.15.86:0/1282680003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f9b8a270 0x5640f9b9a0e0 secure :-1 s=READY pgs=1099 cs=0 l=1 rev1=1 crypto rx=0x7fc4480075d0 tx=0x7fc448007600 comp rx=0 tx=0).stop 2024-04-21T08:42:40.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.298+0000 7fc45779d740 1 -- 172.21.15.86:0/1282680003 shutdown_connections 2024-04-21T08:42:40.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.298+0000 7fc45779d740 1 --2- 172.21.15.86:0/1282680003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f9b8a270 0x5640f9b9a0e0 unknown :-1 s=CLOSED pgs=1099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.298+0000 7fc45779d740 1 --2- 172.21.15.86:0/1282680003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640f9b81170 0x5640f9b8ccb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.298+0000 7fc45779d740 1 --2- 172.21.15.86:0/1282680003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640f9b807c0 0x5640f9b904a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.298+0000 7fc45779d740 1 -- 172.21.15.86:0/1282680003 >> 172.21.15.86:0/1282680003 conn(0x5640f9b7afb0 msgr2=0x5640f9b7cc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:40.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.298+0000 7fc45779d740 1 -- 172.21.15.86:0/1282680003 shutdown_connections 2024-04-21T08:42:40.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.298+0000 7fc45779d740 1 -- 172.21.15.86:0/1282680003 wait complete. 2024-04-21T08:42:40.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.299+0000 7fc45779d740 1 -- start start 2024-04-21T08:42:40.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.299+0000 7fc45779d740 1 Processor -- start 2024-04-21T08:42:40.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.299+0000 7fc45779d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640f9b807c0 0x5640f9b965f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:40.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.299+0000 7fc45779d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f9b81170 0x5640f9b96b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:40.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.299+0000 7fc45779d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640f9b8a270 0x5640f9b93640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:40.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.299+0000 7fc45779d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc4501ae420 con 0x5640f9b81170 2024-04-21T08:42:40.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.299+0000 7fc45779d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc4501add50 con 0x5640f9b8a270 2024-04-21T08:42:40.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.299+0000 7fc45779d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc4501aaf70 con 0x5640f9b807c0 2024-04-21T08:42:40.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.299+0000 7fc45779d740 10 client.0 _ll_register_callbacks cb 0x7fc4380f2f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:40.304 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.299+0000 7fc447fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f9b81170 0x5640f9b96b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:40.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.299+0000 7fc447fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f9b81170 0x5640f9b96b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47676/0 (socket says 172.21.15.86:47676) 2024-04-21T08:42:40.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.299+0000 7fc447fff640 1 -- 172.21.15.86:0/872542478 learned_addr learned my addr 172.21.15.86:0/872542478 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:40.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.299+0000 7fc44ed9d640 1 --2- 172.21.15.86:0/872542478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640f9b8a270 0x5640f9b93640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:40.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.299+0000 7fc447fff640 1 -- 172.21.15.86:0/872542478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640f9b807c0 msgr2=0x5640f9b965f0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:42:40.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.299+0000 7fc447fff640 1 --2- 172.21.15.86:0/872542478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640f9b807c0 0x5640f9b965f0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.299+0000 7fc447fff640 1 -- 172.21.15.86:0/872542478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640f9b8a270 msgr2=0x5640f9b93640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:40.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.299+0000 7fc447fff640 1 --2- 172.21.15.86:0/872542478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640f9b8a270 0x5640f9b93640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.299+0000 7fc447fff640 1 -- 172.21.15.86:0/872542478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5640f9b93e70 con 0x5640f9b81170 2024-04-21T08:42:40.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.299+0000 7fc44ed9d640 1 --2- 172.21.15.86:0/872542478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640f9b8a270 0x5640f9b93640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:40.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.299+0000 7fc447fff640 1 --2- 172.21.15.86:0/872542478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f9b81170 0x5640f9b96b30 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fc4380be950 tx=0x7fc43802bd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e8910d95742131 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:40.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.299+0000 7fc435ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:40.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.299+0000 7fc435ffb640 1 -- 172.21.15.86:0/872542478 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc4380bac60 con 0x5640f9b81170 2024-04-21T08:42:40.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.300+0000 7fc435ffb640 1 -- 172.21.15.86:0/872542478 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fc4380f02e0 con 0x5640f9b81170 2024-04-21T08:42:40.305 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.300+0000 7fc45779d740 1 -- 172.21.15.86:0/872542478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc448003e20 con 0x5640f9b81170 2024-04-21T08:42:40.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.300+0000 7fc435ffb640 1 -- 172.21.15.86:0/872542478 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc4380be790 con 0x5640f9b81170 2024-04-21T08:42:40.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.300+0000 7fc45779d740 10 client.17472 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:40.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.300+0000 7fc45779d740 1 -- 172.21.15.86:0/872542478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5640f9b93e70 con 0x5640f9b81170 2024-04-21T08:42:40.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.300+0000 7fc435ffb640 1 -- 172.21.15.86:0/872542478 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc448003e20 con 0x5640f9b81170 2024-04-21T08:42:40.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.300+0000 7fc435ffb640 1 -- 172.21.15.86:0/872542478 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc4380ba7a0 con 0x5640f9b81170 2024-04-21T08:42:40.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.300+0000 7fc45779d740 10 client.17472 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:40.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.300+0000 7fc45779d740 10 client.17472 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:40.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.300+0000 7fc45779d740 20 client.17472 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:40.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.300+0000 7fc45779d740 1 --2- 172.21.15.86:0/872542478 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5640f9c05b80 0x5640f9c07fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:40.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.300+0000 7fc45779d740 4 client.17472 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:42:40.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.300+0000 7fc45779d740 1 -- 172.21.15.86:0/872542478 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5640f9aec790 con 0x5640f9c05b80 2024-04-21T08:42:40.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.300+0000 7fc44e59c640 1 --2- 172.21.15.86:0/872542478 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5640f9c05b80 0x5640f9c07fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:40.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.301+0000 7fc44e59c640 1 --2- 172.21.15.86:0/872542478 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5640f9c05b80 0x5640f9c07fa0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1f7e5eb4c4fe6147 server_cookie=dc4b3069a785af19 in_seq=0 out_seq=0 2024-04-21T08:42:40.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.301+0000 7fc435ffb640 10 client.17472 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:40.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.302+0000 7fc435ffb640 1 -- 172.21.15.86:0/872542478 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5640f9aec790 con 0x5640f9c05b80 2024-04-21T08:42:40.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.302+0000 7fc435ffb640 10 client.17472 handle_command_reply: tid=0 2024-04-21T08:42:40.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.302+0000 7fc45779d740 1 client.17472 shutdown 2024-04-21T08:42:40.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.302+0000 7fc45779d740 1 -- 172.21.15.86:0/872542478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f9b81170 msgr2=0x5640f9b96b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:40.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.302+0000 7fc45779d740 1 --2- 172.21.15.86:0/872542478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f9b81170 0x5640f9b96b30 secure :-1 s=READY pgs=1100 cs=0 l=1 rev1=1 crypto rx=0x7fc4380be950 tx=0x7fc43802bd60 comp rx=0 tx=0).stop 2024-04-21T08:42:40.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.302+0000 7fc45779d740 1 -- 172.21.15.86:0/872542478 shutdown_connections 2024-04-21T08:42:40.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.302+0000 7fc45779d740 1 --2- 172.21.15.86:0/872542478 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5640f9c05b80 0x5640f9c07fa0 crc :-1 s=READY pgs=146 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.302+0000 7fc45779d740 1 --2- 172.21.15.86:0/872542478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5640f9b8a270 0x5640f9b93640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.302+0000 7fc45779d740 1 --2- 172.21.15.86:0/872542478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5640f9b81170 0x5640f9b96b30 unknown :-1 s=CLOSED pgs=1100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.302+0000 7fc45779d740 1 --2- 172.21.15.86:0/872542478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5640f9b807c0 0x5640f9b965f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.302+0000 7fc45779d740 1 -- 172.21.15.86:0/872542478 >> 172.21.15.86:0/872542478 conn(0x5640f9b7afb0 msgr2=0x5640f9b8dd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:40.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.302+0000 7fc435ffb640 0 client.17472 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:40.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.302+0000 7fc45779d740 1 -- 172.21.15.86:0/872542478 shutdown_connections 2024-04-21T08:42:40.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.302+0000 7fc45779d740 1 -- 172.21.15.86:0/872542478 wait complete. 2024-04-21T08:42:40.308 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.302+0000 7fc45779d740 20 client.17472 trim_cache size 0 max 16384 2024-04-21T08:42:40.308 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:42:40.308 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-21T08:42:40.308 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "154e1373-14d9-44fd-bb0b-f9c826e182fe", 2024-04-21T08:42:40.308 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-21T08:42:40.308 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:42:40.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.302+0000 7fc454bab640 1 -- 172.21.15.86:0/344939445 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc43c0a3d20 msgr2=0x7fc43c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:40.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.302+0000 7fc454bab640 1 --2- 172.21.15.86:0/344939445 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc43c0a3d20 0x7fc43c0a61e0 secure :-1 s=READY pgs=169 cs=0 l=1 rev1=1 crypto rx=0x7fc440005fd0 tx=0x7fc44000b390 comp rx=0 tx=0).stop 2024-04-21T08:42:40.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.302+0000 7fc454bab640 1 -- 172.21.15.86:0/344939445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc450103b00 msgr2=0x7fc450198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:40.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.302+0000 7fc454bab640 1 --2- 172.21.15.86:0/344939445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc450103b00 0x7fc450198c50 secure :-1 s=READY pgs=1096 cs=0 l=1 rev1=1 crypto rx=0x7fc438005940 tx=0x7fc43802d920 comp rx=0 tx=0).stop 2024-04-21T08:42:40.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.303+0000 7fc454bab640 1 -- 172.21.15.86:0/344939445 shutdown_connections 2024-04-21T08:42:40.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.303+0000 7fc454bab640 1 --2- 172.21.15.86:0/344939445 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc43c0a3d20 0x7fc43c0a61e0 unknown :-1 s=CLOSED pgs=169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.303+0000 7fc454bab640 1 --2- 172.21.15.86:0/344939445 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc45010c520 0x7fc45019cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.303+0000 7fc454bab640 1 --2- 172.21.15.86:0/344939445 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc450103b00 0x7fc450198c50 unknown :-1 s=CLOSED pgs=1096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.303+0000 7fc454bab640 1 --2- 172.21.15.86:0/344939445 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc4501031e0 0x7fc450198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.303+0000 7fc454bab640 1 -- 172.21.15.86:0/344939445 >> 172.21.15.86:0/344939445 conn(0x7fc4500771d0 msgr2=0x7fc4500fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:40.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.303+0000 7fc454bab640 1 -- 172.21.15.86:0/344939445 shutdown_connections 2024-04-21T08:42:40.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.303+0000 7fc454bab640 1 -- 172.21.15.86:0/344939445 wait complete. 2024-04-21T08:42:40.312 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:42:40.313 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:42:40.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.414+0000 7fbf0a45c640 1 -- 172.21.15.86:0/2003421625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf041021a0 msgr2=0x7fbf04102580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:40.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.414+0000 7fbf0a45c640 1 --2- 172.21.15.86:0/2003421625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf041021a0 0x7fbf04102580 secure :-1 s=READY pgs=1101 cs=0 l=1 rev1=1 crypto rx=0x7fbef4009900 tx=0x7fbef402bc50 comp rx=0 tx=0).stop 2024-04-21T08:42:40.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.414+0000 7fbf0a45c640 1 -- 172.21.15.86:0/2003421625 shutdown_connections 2024-04-21T08:42:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.414+0000 7fbf0a45c640 1 --2- 172.21.15.86:0/2003421625 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf04107270 0x7fbf0410ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.414+0000 7fbf0a45c640 1 --2- 172.21.15.86:0/2003421625 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf04102b50 0x7fbf04106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.414+0000 7fbf0a45c640 1 --2- 172.21.15.86:0/2003421625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf041021a0 0x7fbf04102580 unknown :-1 s=CLOSED pgs=1101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.414+0000 7fbf0a45c640 1 -- 172.21.15.86:0/2003421625 >> 172.21.15.86:0/2003421625 conn(0x7fbf040793e0 msgr2=0x7fbf040797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.414+0000 7fbf0a45c640 1 -- 172.21.15.86:0/2003421625 shutdown_connections 2024-04-21T08:42:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.414+0000 7fbf0a45c640 1 -- 172.21.15.86:0/2003421625 wait complete. 2024-04-21T08:42:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.415+0000 7fbf0a45c640 1 Processor -- start 2024-04-21T08:42:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.415+0000 7fbf0a45c640 1 -- start start 2024-04-21T08:42:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.415+0000 7fbf0a45c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf041021a0 0x7fbf0419e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.415+0000 7fbf0a45c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf04107270 0x7fbf0419e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.415+0000 7fbf0a45c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf041985d0 0x7fbf04198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.415+0000 7fbf0a45c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbf04112250 con 0x7fbf041985d0 2024-04-21T08:42:40.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.415+0000 7fbf0a45c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041120d0 con 0x7fbf041021a0 2024-04-21T08:42:40.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.415+0000 7fbf0a45c640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041123d0 con 0x7fbf04107270 2024-04-21T08:42:40.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.415+0000 7fbf089f9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf041985d0 0x7fbf04198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:40.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.415+0000 7fbf089f9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf041985d0 0x7fbf04198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47688/0 (socket says 172.21.15.86:47688) 2024-04-21T08:42:40.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.415+0000 7fbf089f9640 1 -- 172.21.15.86:0/3543721982 learned_addr learned my addr 172.21.15.86:0/3543721982 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:40.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.415+0000 7fbf03fff640 1 --2- 172.21.15.86:0/3543721982 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf041021a0 0x7fbf0419e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:40.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.415+0000 7fbf089f9640 1 -- 172.21.15.86:0/3543721982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf04107270 msgr2=0x7fbf0419e960 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T08:42:40.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.415+0000 7fbf089f9640 1 --2- 172.21.15.86:0/3543721982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf04107270 0x7fbf0419e960 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.415+0000 7fbf089f9640 1 -- 172.21.15.86:0/3543721982 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf041021a0 msgr2=0x7fbf0419e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:40.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.415+0000 7fbf089f9640 1 --2- 172.21.15.86:0/3543721982 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf041021a0 0x7fbf0419e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.415+0000 7fbf089f9640 1 -- 172.21.15.86:0/3543721982 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbf041990f0 con 0x7fbf041985d0 2024-04-21T08:42:40.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.416+0000 7fbf089f9640 1 --2- 172.21.15.86:0/3543721982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf041985d0 0x7fbf04198a80 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00b990 tx=0x7fbefc00be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:40.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.416+0000 7fbf017fa640 1 -- 172.21.15.86:0/3543721982 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc019070 con 0x7fbf041985d0 2024-04-21T08:42:40.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.416+0000 7fbf0a45c640 1 -- 172.21.15.86:0/3543721982 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbf041993e0 con 0x7fbf041985d0 2024-04-21T08:42:40.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.416+0000 7fbf017fa640 1 -- 172.21.15.86:0/3543721982 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbefc00cba0 con 0x7fbf041985d0 2024-04-21T08:42:40.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.416+0000 7fbf017fa640 1 -- 172.21.15.86:0/3543721982 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbefc0075d0 con 0x7fbf041985d0 2024-04-21T08:42:40.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.416+0000 7fbf0a45c640 1 -- 172.21.15.86:0/3543721982 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbf041a55e0 con 0x7fbf041985d0 2024-04-21T08:42:40.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.418+0000 7fbf017fa640 1 -- 172.21.15.86:0/3543721982 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbefc01c070 con 0x7fbf041985d0 2024-04-21T08:42:40.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.419+0000 7fbf0d054740 1 Processor -- start 2024-04-21T08:42:40.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.419+0000 7fbf0d054740 1 -- start start 2024-04-21T08:42:40.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.419+0000 7fbf0d054740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f294ff4580 0x55f294ff4960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:40.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.419+0000 7fbf0d054740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f294ff4ea0 0x55f294ffd540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:40.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.419+0000 7fbf0d054740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f294ffda80 0x55f294fffe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:40.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.419+0000 7fbf0d054740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f294f591e0 con 0x55f294ffda80 2024-04-21T08:42:40.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.419+0000 7fbf0d054740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55f294ef6390 con 0x55f294ff4ea0 2024-04-21T08:42:40.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.419+0000 7fbf0d054740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f294f551a0 con 0x55f294ff4580 2024-04-21T08:42:40.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.419+0000 7fbf017fa640 1 --2- 172.21.15.86:0/3543721982 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbeec0a3d20 0x7fbeec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:40.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.419+0000 7fbf037fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f294ffda80 0x55f294fffe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:40.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.419+0000 7fbf037fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f294ffda80 0x55f294fffe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47692/0 (socket says 172.21.15.86:47692) 2024-04-21T08:42:40.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.419+0000 7fbf037fe640 1 -- 172.21.15.86:0/3292666213 learned_addr learned my addr 172.21.15.86:0/3292666213 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:40.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.419+0000 7fbf089f9640 1 --2- 172.21.15.86:0/3543721982 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbeec0a3d20 0x7fbeec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:40.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.419+0000 7fbf03fff640 1 --2- 172.21.15.86:0/3292666213 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f294ff4ea0 0x55f294ffd540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:40.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.419+0000 7fbf017fa640 1 -- 172.21.15.86:0/3543721982 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fbefc0d0050 con 0x7fbf041985d0 2024-04-21T08:42:40.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.419+0000 7fbf037fe640 1 -- 172.21.15.86:0/3292666213 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f294ff4580 msgr2=0x55f294ff4960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:40.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.419+0000 7fbf037fe640 1 --2- 172.21.15.86:0/3292666213 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f294ff4580 0x55f294ff4960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.419+0000 7fbf037fe640 1 -- 172.21.15.86:0/3292666213 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f294ff4ea0 msgr2=0x55f294ffd540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:40.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.419+0000 7fbf037fe640 1 --2- 172.21.15.86:0/3292666213 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f294ff4ea0 0x55f294ffd540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.419+0000 7fbf037fe640 1 -- 172.21.15.86:0/3292666213 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f294d357d0 con 0x55f294ffda80 2024-04-21T08:42:40.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.419+0000 7fbf089f9640 1 --2- 172.21.15.86:0/3543721982 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbeec0a3d20 0x7fbeec0a61e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbefc09bdd0 tx=0x7fbefc06d880 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:40.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.419+0000 7fbf037fe640 1 --2- 172.21.15.86:0/3292666213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f294ffda80 0x55f294fffe70 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fbef8004650 tx=0x7fbef800bf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c072ca82ded2a71d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:40.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.423+0000 7fbeea7fc640 1 -- 172.21.15.86:0/3292666213 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef8016070 con 0x55f294ffda80 2024-04-21T08:42:40.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.423+0000 7fbeea7fc640 1 -- 172.21.15.86:0/3292666213 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbef80054d0 con 0x55f294ffda80 2024-04-21T08:42:40.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.423+0000 7fbeea7fc640 1 -- 172.21.15.86:0/3292666213 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef8004d30 con 0x55f294ffda80 2024-04-21T08:42:40.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.424+0000 7fbf0d054740 1 -- 172.21.15.86:0/3292666213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f294ffda80 msgr2=0x55f294fffe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:40.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.424+0000 7fbf0d054740 1 --2- 172.21.15.86:0/3292666213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f294ffda80 0x55f294fffe70 secure :-1 s=READY pgs=1103 cs=0 l=1 rev1=1 crypto rx=0x7fbef8004650 tx=0x7fbef800bf70 comp rx=0 tx=0).stop 2024-04-21T08:42:40.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.424+0000 7fbf0d054740 1 -- 172.21.15.86:0/3292666213 shutdown_connections 2024-04-21T08:42:40.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.424+0000 7fbf0d054740 1 --2- 172.21.15.86:0/3292666213 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f294ffda80 0x55f294fffe70 unknown :-1 s=CLOSED pgs=1103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.424+0000 7fbf0d054740 1 --2- 172.21.15.86:0/3292666213 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f294ff4ea0 0x55f294ffd540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.424+0000 7fbf0d054740 1 --2- 172.21.15.86:0/3292666213 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f294ff4580 0x55f294ff4960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.424+0000 7fbf0d054740 1 -- 172.21.15.86:0/3292666213 >> 172.21.15.86:0/3292666213 conn(0x55f294feecb0 msgr2=0x55f294fef090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:40.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.424+0000 7fbf0d054740 1 -- 172.21.15.86:0/3292666213 shutdown_connections 2024-04-21T08:42:40.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.424+0000 7fbf0d054740 1 -- 172.21.15.86:0/3292666213 wait complete. 2024-04-21T08:42:40.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.424+0000 7fbf0d054740 1 -- start start 2024-04-21T08:42:40.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.424+0000 7fbf0d054740 1 Processor -- start 2024-04-21T08:42:40.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.425+0000 7fbf0d054740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f294ff4580 0x55f294ffab50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:40.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.425+0000 7fbf0d054740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f294ff4ea0 0x55f294ffb090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:40.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.425+0000 7fbf0d054740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f294ffda80 0x55f294ffb5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:40.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.425+0000 7fbf0d054740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f295000870 con 0x55f294ff4ea0 2024-04-21T08:42:40.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.425+0000 7fbf0d054740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55f294f122f0 con 0x55f294ff4580 2024-04-21T08:42:40.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.425+0000 7fbf0d054740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f2950009f0 con 0x55f294ffda80 2024-04-21T08:42:40.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.425+0000 7fbf0d054740 10 client.0 _ll_register_callbacks cb 0x55f294f1f580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:40.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.425+0000 7fbf03fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f294ff4ea0 0x55f294ffb090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:40.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.425+0000 7fbf03fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f294ff4ea0 0x55f294ffb090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47696/0 (socket says 172.21.15.86:47696) 2024-04-21T08:42:40.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.425+0000 7fbf03fff640 1 -- 172.21.15.86:0/2621933598 learned_addr learned my addr 172.21.15.86:0/2621933598 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:40.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.425+0000 7fbf037fe640 1 --2- 172.21.15.86:0/2621933598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f294ff4580 0x55f294ffab50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:40.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.425+0000 7fbf03fff640 1 -- 172.21.15.86:0/2621933598 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f294ffda80 msgr2=0x55f294ffb5d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:42:40.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.425+0000 7fbf03fff640 1 --2- 172.21.15.86:0/2621933598 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f294ffda80 0x55f294ffb5d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.425+0000 7fbf03fff640 1 -- 172.21.15.86:0/2621933598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f294ff4580 msgr2=0x55f294ffab50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:40.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.425+0000 7fbf03fff640 1 --2- 172.21.15.86:0/2621933598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f294ff4580 0x55f294ffab50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.425+0000 7fbf03fff640 1 -- 172.21.15.86:0/2621933598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f294c6b3f0 con 0x55f294ff4ea0 2024-04-21T08:42:40.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.425+0000 7fbf037fe640 1 --2- 172.21.15.86:0/2621933598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f294ff4580 0x55f294ffab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:40.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.425+0000 7fbf03fff640 1 --2- 172.21.15.86:0/2621933598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f294ff4ea0 0x55f294ffb090 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fbef4009900 tx=0x7fbef402ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=29aac46cede13c8c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:40.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.425+0000 7fbee37fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:40.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.425+0000 7fbee37fe640 1 -- 172.21.15.86:0/2621933598 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef4005030 con 0x55f294ff4ea0 2024-04-21T08:42:40.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.425+0000 7fbee37fe640 1 -- 172.21.15.86:0/2621933598 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fbef40051d0 con 0x55f294ff4ea0 2024-04-21T08:42:40.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.425+0000 7fbf0d054740 1 -- 172.21.15.86:0/2621933598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbef80054d0 con 0x55f294ff4ea0 2024-04-21T08:42:40.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.425+0000 7fbee37fe640 1 -- 172.21.15.86:0/2621933598 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbef40053d0 con 0x55f294ff4ea0 2024-04-21T08:42:40.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.425+0000 7fbf0d054740 10 client.17484 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:40.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.425+0000 7fbf0d054740 1 -- 172.21.15.86:0/2621933598 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f294c2b0f0 con 0x55f294ff4ea0 2024-04-21T08:42:40.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.426+0000 7fbee37fe640 1 -- 172.21.15.86:0/2621933598 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbef80054d0 con 0x55f294ff4ea0 2024-04-21T08:42:40.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.426+0000 7fbee37fe640 1 -- 172.21.15.86:0/2621933598 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbef4003710 con 0x55f294ff4ea0 2024-04-21T08:42:40.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.426+0000 7fbf0d054740 10 client.17484 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:40.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.426+0000 7fbf0d054740 10 client.17484 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:40.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.426+0000 7fbf0d054740 20 client.17484 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:40.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.426+0000 7fbf0d054740 1 --2- 172.21.15.86:0/2621933598 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f29500dba0 0x55f295010020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:40.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.426+0000 7fbf0d054740 4 client.17484 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:40.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.426+0000 7fbf0d054740 1 -- 172.21.15.86:0/2621933598 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f294f66c00 con 0x55f29500dba0 2024-04-21T08:42:40.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.426+0000 7fbf037fe640 1 --2- 172.21.15.86:0/2621933598 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f29500dba0 0x55f295010020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:40.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.426+0000 7fbf037fe640 1 --2- 172.21.15.86:0/2621933598 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f29500dba0 0x55f295010020 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=29a776bf3c2cbe3c server_cookie=6ad686b128f137ff in_seq=0 out_seq=0 2024-04-21T08:42:40.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.426+0000 7fbee37fe640 10 client.17484 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:40.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.427+0000 7fbee37fe640 1 -- 172.21.15.86:0/2621933598 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f294f66c00 con 0x55f29500dba0 2024-04-21T08:42:40.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.427+0000 7fbee37fe640 10 client.17484 handle_command_reply: tid=0 2024-04-21T08:42:40.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.427+0000 7fbf0d054740 1 client.17484 shutdown 2024-04-21T08:42:40.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.427+0000 7fbf0d054740 1 -- 172.21.15.86:0/2621933598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f294ff4ea0 msgr2=0x55f294ffb090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:40.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.427+0000 7fbf0d054740 1 --2- 172.21.15.86:0/2621933598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f294ff4ea0 0x55f294ffb090 secure :-1 s=READY pgs=1104 cs=0 l=1 rev1=1 crypto rx=0x7fbef4009900 tx=0x7fbef402ca10 comp rx=0 tx=0).stop 2024-04-21T08:42:40.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.427+0000 7fbf0d054740 1 -- 172.21.15.86:0/2621933598 shutdown_connections 2024-04-21T08:42:40.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.427+0000 7fbf0d054740 1 --2- 172.21.15.86:0/2621933598 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f29500dba0 0x55f295010020 crc :-1 s=READY pgs=147 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.427+0000 7fbf0d054740 1 --2- 172.21.15.86:0/2621933598 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f294ffda80 0x55f294ffb5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.427+0000 7fbf0d054740 1 --2- 172.21.15.86:0/2621933598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f294ff4ea0 0x55f294ffb090 unknown :-1 s=CLOSED pgs=1104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.427+0000 7fbee37fe640 0 client.17484 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:40.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.427+0000 7fbf0d054740 1 --2- 172.21.15.86:0/2621933598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f294ff4580 0x55f294ffab50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.427+0000 7fbf0d054740 1 -- 172.21.15.86:0/2621933598 >> 172.21.15.86:0/2621933598 conn(0x55f294feecb0 msgr2=0x55f294ff0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:40.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.427+0000 7fbf0d054740 1 -- 172.21.15.86:0/2621933598 shutdown_connections 2024-04-21T08:42:40.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.427+0000 7fbf0d054740 1 -- 172.21.15.86:0/2621933598 wait complete. 2024-04-21T08:42:40.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.428+0000 7fbf0d054740 20 client.17484 trim_cache size 0 max 16384 2024-04-21T08:42:40.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.445+0000 7fbf0d054740 1 Processor -- start 2024-04-21T08:42:40.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.445+0000 7fbf0d054740 1 -- start start 2024-04-21T08:42:40.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.445+0000 7fbf0d054740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f294ff4580 0x55f294ffd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:40.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.445+0000 7fbf0d054740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f294ff4ea0 0x55f295009730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:40.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.445+0000 7fbf0d054740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f294ffda80 0x55f294ffb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:40.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.445+0000 7fbf0d054740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041af310 con 0x55f294ff4ea0 2024-04-21T08:42:40.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.445+0000 7fbf0d054740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041b0e50 con 0x55f294ffda80 2024-04-21T08:42:40.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.445+0000 7fbf0d054740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041af9e0 con 0x55f294ff4580 2024-04-21T08:42:40.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.445+0000 7fbf03fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f294ff4ea0 0x55f295009730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:40.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.445+0000 7fbf03fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f294ff4ea0 0x55f295009730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47704/0 (socket says 172.21.15.86:47704) 2024-04-21T08:42:40.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.445+0000 7fbf03fff640 1 -- 172.21.15.86:0/3295785047 learned_addr learned my addr 172.21.15.86:0/3295785047 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:40.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.445+0000 7fbf03fff640 1 -- 172.21.15.86:0/3295785047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f294ff4580 msgr2=0x55f294ffd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:40.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.445+0000 7fbf03fff640 1 --2- 172.21.15.86:0/3295785047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f294ff4580 0x55f294ffd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.445+0000 7fbf03fff640 1 -- 172.21.15.86:0/3295785047 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f294ffda80 msgr2=0x55f294ffb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:40.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.445+0000 7fbf03fff640 1 --2- 172.21.15.86:0/3295785047 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f294ffda80 0x55f294ffb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.445+0000 7fbf03fff640 1 -- 172.21.15.86:0/3295785047 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f294ff6750 con 0x55f294ff4ea0 2024-04-21T08:42:40.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.445+0000 7fbf03fff640 1 --2- 172.21.15.86:0/3295785047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f294ff4ea0 0x55f295009730 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fbef4003bb0 tx=0x7fbef402cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a462b81c08c241e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:40.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.445+0000 7fbee9ffb640 1 -- 172.21.15.86:0/3295785047 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef4000e60 con 0x55f294ff4ea0 2024-04-21T08:42:40.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.445+0000 7fbee9ffb640 1 -- 172.21.15.86:0/3295785047 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbef4005570 con 0x55f294ff4ea0 2024-04-21T08:42:40.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.445+0000 7fbee9ffb640 1 -- 172.21.15.86:0/3295785047 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef4034b30 con 0x55f294ff4ea0 2024-04-21T08:42:40.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.446+0000 7fbf0d054740 1 -- 172.21.15.86:0/3295785047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f294ff4ea0 msgr2=0x55f295009730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:40.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.446+0000 7fbf0d054740 1 --2- 172.21.15.86:0/3295785047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f294ff4ea0 0x55f295009730 secure :-1 s=READY pgs=1105 cs=0 l=1 rev1=1 crypto rx=0x7fbef4003bb0 tx=0x7fbef402cd90 comp rx=0 tx=0).stop 2024-04-21T08:42:40.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.446+0000 7fbf0d054740 1 -- 172.21.15.86:0/3295785047 shutdown_connections 2024-04-21T08:42:40.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.446+0000 7fbf0d054740 1 --2- 172.21.15.86:0/3295785047 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f294ffda80 0x55f294ffb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.446+0000 7fbf0d054740 1 --2- 172.21.15.86:0/3295785047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f294ff4ea0 0x55f295009730 unknown :-1 s=CLOSED pgs=1105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.446+0000 7fbf0d054740 1 --2- 172.21.15.86:0/3295785047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f294ff4580 0x55f294ffd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.446+0000 7fbf0d054740 1 -- 172.21.15.86:0/3295785047 >> 172.21.15.86:0/3295785047 conn(0x55f294feecb0 msgr2=0x55f294ffab10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:40.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.446+0000 7fbf0d054740 1 -- 172.21.15.86:0/3295785047 shutdown_connections 2024-04-21T08:42:40.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.446+0000 7fbf0d054740 1 -- 172.21.15.86:0/3295785047 wait complete. 2024-04-21T08:42:40.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.446+0000 7fbf0d054740 1 -- start start 2024-04-21T08:42:40.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.446+0000 7fbf0d054740 1 Processor -- start 2024-04-21T08:42:40.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.447+0000 7fbf0d054740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f294ff4580 0x55f294ff7f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:40.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.447+0000 7fbf0d054740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f294ff4ea0 0x55f294ff8440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:40.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.447+0000 7fbf0d054740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f294ffda80 0x55f295077570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:40.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.447+0000 7fbf0d054740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041b0780 con 0x55f294ff4ea0 2024-04-21T08:42:40.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.447+0000 7fbf0d054740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041b00b0 con 0x55f294ffda80 2024-04-21T08:42:40.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.447+0000 7fbf0d054740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbf041ad2d0 con 0x55f294ff4580 2024-04-21T08:42:40.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.447+0000 7fbf0d054740 10 client.0 _ll_register_callbacks cb 0x7fbef402c5f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:40.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.447+0000 7fbf03fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f294ff4ea0 0x55f294ff8440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:40.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.447+0000 7fbf03fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f294ff4ea0 0x55f294ff8440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47716/0 (socket says 172.21.15.86:47716) 2024-04-21T08:42:40.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.447+0000 7fbf03fff640 1 -- 172.21.15.86:0/74669023 learned_addr learned my addr 172.21.15.86:0/74669023 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:40.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.447+0000 7fbf037fe640 1 --2- 172.21.15.86:0/74669023 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f294ffda80 0x55f295077570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:40.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.447+0000 7fbf037fe640 1 --2- 172.21.15.86:0/74669023 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f294ff4580 0x55f294ff7f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:40.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.447+0000 7fbf03fff640 1 -- 172.21.15.86:0/74669023 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f294ff4580 msgr2=0x55f294ff7f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:40.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.447+0000 7fbf03fff640 1 --2- 172.21.15.86:0/74669023 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f294ff4580 0x55f294ff7f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.447+0000 7fbf03fff640 1 -- 172.21.15.86:0/74669023 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f294ffda80 msgr2=0x55f295077570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:40.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.447+0000 7fbf03fff640 1 --2- 172.21.15.86:0/74669023 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f294ffda80 0x55f295077570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.447+0000 7fbf03fff640 1 -- 172.21.15.86:0/74669023 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f295077ab0 con 0x55f294ff4ea0 2024-04-21T08:42:40.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.447+0000 7fbf037fe640 1 --2- 172.21.15.86:0/74669023 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f294ff4580 0x55f294ff7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:40.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.447+0000 7fbf03fff640 1 --2- 172.21.15.86:0/74669023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f294ff4ea0 0x55f294ff8440 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fbef4009900 tx=0x7fbef4034440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2a97edb60ebc267 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:40.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.447+0000 7fbee37fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:40.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.447+0000 7fbee37fe640 1 -- 172.21.15.86:0/74669023 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbef40348b0 con 0x55f294ff4ea0 2024-04-21T08:42:40.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.447+0000 7fbee37fe640 1 -- 172.21.15.86:0/74669023 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fbef4034a50 con 0x55f294ff4ea0 2024-04-21T08:42:40.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.447+0000 7fbf0d054740 1 -- 172.21.15.86:0/74669023 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbef4005570 con 0x55f294ff4ea0 2024-04-21T08:42:40.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.447+0000 7fbee37fe640 1 -- 172.21.15.86:0/74669023 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbef4067050 con 0x55f294ff4ea0 2024-04-21T08:42:40.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.447+0000 7fbf0d054740 10 client.17490 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:40.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.448+0000 7fbf0d054740 1 -- 172.21.15.86:0/74669023 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f295078d30 con 0x55f294ff4ea0 2024-04-21T08:42:40.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.448+0000 7fbee37fe640 1 -- 172.21.15.86:0/74669023 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbef4005570 con 0x55f294ff4ea0 2024-04-21T08:42:40.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.448+0000 7fbee37fe640 1 -- 172.21.15.86:0/74669023 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbef4056bf0 con 0x55f294ff4ea0 2024-04-21T08:42:40.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.448+0000 7fbf0d054740 10 client.17490 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:40.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.448+0000 7fbf0d054740 10 client.17490 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:40.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.448+0000 7fbf0d054740 20 client.17490 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:40.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.448+0000 7fbf0d054740 1 --2- 172.21.15.86:0/74669023 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f2950793f0 0x55f29507b810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:40.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.448+0000 7fbf0d054740 4 client.17490 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:42:40.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.448+0000 7fbf0d054740 1 -- 172.21.15.86:0/74669023 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55f294f60a70 con 0x55f2950793f0 2024-04-21T08:42:40.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.448+0000 7fbf037fe640 1 --2- 172.21.15.86:0/74669023 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f2950793f0 0x55f29507b810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:40.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.448+0000 7fbee37fe640 10 client.17490 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:40.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.448+0000 7fbf037fe640 1 --2- 172.21.15.86:0/74669023 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f2950793f0 0x55f29507b810 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d576077519f27244 server_cookie=5d637c25e74abb60 in_seq=0 out_seq=0 2024-04-21T08:42:40.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.449+0000 7fbee37fe640 1 -- 172.21.15.86:0/74669023 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55f294f60a70 con 0x55f2950793f0 2024-04-21T08:42:40.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.449+0000 7fbee37fe640 10 client.17490 handle_command_reply: tid=0 2024-04-21T08:42:40.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.449+0000 7fbf0d054740 1 client.17490 shutdown 2024-04-21T08:42:40.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.449+0000 7fbf0d054740 1 -- 172.21.15.86:0/74669023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f294ff4ea0 msgr2=0x55f294ff8440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:40.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.449+0000 7fbf0d054740 1 --2- 172.21.15.86:0/74669023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f294ff4ea0 0x55f294ff8440 secure :-1 s=READY pgs=1106 cs=0 l=1 rev1=1 crypto rx=0x7fbef4009900 tx=0x7fbef4034440 comp rx=0 tx=0).stop 2024-04-21T08:42:40.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.449+0000 7fbf0d054740 1 -- 172.21.15.86:0/74669023 shutdown_connections 2024-04-21T08:42:40.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.449+0000 7fbf0d054740 1 --2- 172.21.15.86:0/74669023 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f2950793f0 0x55f29507b810 crc :-1 s=READY pgs=148 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.449+0000 7fbf0d054740 1 --2- 172.21.15.86:0/74669023 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f294ffda80 0x55f295077570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.449+0000 7fbee37fe640 0 client.17490 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:40.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.449+0000 7fbf0d054740 1 --2- 172.21.15.86:0/74669023 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f294ff4ea0 0x55f294ff8440 unknown :-1 s=CLOSED pgs=1106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.449+0000 7fbf0d054740 1 --2- 172.21.15.86:0/74669023 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f294ff4580 0x55f294ff7f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.449+0000 7fbf0d054740 1 -- 172.21.15.86:0/74669023 >> 172.21.15.86:0/74669023 conn(0x55f294feecb0 msgr2=0x55f294ffb380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:40.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.449+0000 7fbf0d054740 1 -- 172.21.15.86:0/74669023 shutdown_connections 2024-04-21T08:42:40.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.449+0000 7fbf0d054740 1 -- 172.21.15.86:0/74669023 wait complete. 2024-04-21T08:42:40.455 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.449+0000 7fbf0d054740 20 client.17490 trim_cache size 0 max 16384 2024-04-21T08:42:40.455 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:42:40.455 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:42:40.455 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-21T08:42:40.455 INFO:teuthology.orchestra.run.smithi086.stdout: "154e1373-14d9-44fd-bb0b-f9c826e182fe": { 2024-04-21T08:42:40.455 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-21T08:42:40.455 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "154e1373-14d9-44fd-bb0b-f9c826e182fe", 2024-04-21T08:42:40.455 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-21T08:42:40.455 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:42:40.456 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:42:40.456 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:42:40.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.449+0000 7fbf0a45c640 1 -- 172.21.15.86:0/3543721982 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbeec0a3d20 msgr2=0x7fbeec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:40.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.449+0000 7fbf0a45c640 1 --2- 172.21.15.86:0/3543721982 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbeec0a3d20 0x7fbeec0a61e0 secure :-1 s=READY pgs=170 cs=0 l=1 rev1=1 crypto rx=0x7fbefc09bdd0 tx=0x7fbefc06d880 comp rx=0 tx=0).stop 2024-04-21T08:42:40.456 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.449+0000 7fbf0a45c640 1 -- 172.21.15.86:0/3543721982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf041985d0 msgr2=0x7fbf04198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:40.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.449+0000 7fbf0a45c640 1 --2- 172.21.15.86:0/3543721982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf041985d0 0x7fbf04198a80 secure :-1 s=READY pgs=1102 cs=0 l=1 rev1=1 crypto rx=0x7fbefc00b990 tx=0x7fbefc00be50 comp rx=0 tx=0).stop 2024-04-21T08:42:40.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.450+0000 7fbf0a45c640 1 -- 172.21.15.86:0/3543721982 shutdown_connections 2024-04-21T08:42:40.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.450+0000 7fbf0a45c640 1 --2- 172.21.15.86:0/3543721982 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbeec0a3d20 0x7fbeec0a61e0 unknown :-1 s=CLOSED pgs=170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.450+0000 7fbf0a45c640 1 --2- 172.21.15.86:0/3543721982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbf041985d0 0x7fbf04198a80 unknown :-1 s=CLOSED pgs=1102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.450+0000 7fbf0a45c640 1 --2- 172.21.15.86:0/3543721982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbf04107270 0x7fbf0419e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.450+0000 7fbf0a45c640 1 --2- 172.21.15.86:0/3543721982 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbf041021a0 0x7fbf0419e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:40.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.450+0000 7fbf0a45c640 1 -- 172.21.15.86:0/3543721982 >> 172.21.15.86:0/3543721982 conn(0x7fbf040793e0 msgr2=0x7fbf040fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:40.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.450+0000 7fbf0a45c640 1 -- 172.21.15.86:0/3543721982 shutdown_connections 2024-04-21T08:42:40.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:42:40.450+0000 7fbf0a45c640 1 -- 172.21.15.86:0/3543721982 wait complete. 2024-04-21T08:42:40.458 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:40 smithi086 ceph-mon[27939]: pgmap v119: 129 pgs: 129 active+clean; 4.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 47 KiB/s wr, 25 op/s 2024-04-21T08:42:40.459 INFO:tasks.cephfs.filesystem:scrub status for tag:154e1373-14d9-44fd-bb0b-f9c826e182fe - {'path': '/', 'tag': '154e1373-14d9-44fd-bb0b-f9c826e182fe', 'options': 'recursive,force'} 2024-04-21T08:42:40.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:40 smithi088 ceph-mon[31259]: pgmap v119: 129 pgs: 129 active+clean; 4.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 47 KiB/s wr, 25 op/s 2024-04-21T08:42:40.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:40 smithi179 ceph-mon[31829]: pgmap v119: 129 pgs: 129 active+clean; 4.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 47 KiB/s wr, 25 op/s 2024-04-21T08:42:41.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:41 smithi088 ceph-mon[31259]: scrub queued for path: / 2024-04-21T08:42:41.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:41 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:42:41.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:41 smithi088 ceph-mon[31259]: scrub summary: active paths [/] 2024-04-21T08:42:41.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:41 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:42:41.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:41 smithi179 ceph-mon[31829]: scrub queued for path: / 2024-04-21T08:42:41.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:41 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:42:41.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:41 smithi179 ceph-mon[31829]: scrub summary: active paths [/] 2024-04-21T08:42:41.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:41 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:42:41.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:41 smithi086 ceph-mon[27939]: scrub queued for path: / 2024-04-21T08:42:41.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:41 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:42:41.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:41 smithi086 ceph-mon[27939]: scrub summary: active paths [/] 2024-04-21T08:42:41.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:41 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:42:42.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:42 smithi088 ceph-mon[31259]: pgmap v120: 129 pgs: 129 active+clean; 4.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-04-21T08:42:42.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:42 smithi179 ceph-mon[31829]: pgmap v120: 129 pgs: 129 active+clean; 4.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-04-21T08:42:42.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:42 smithi086 ceph-mon[27939]: pgmap v120: 129 pgs: 129 active+clean; 4.4 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-04-21T08:42:44.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:44 smithi088 ceph-mon[31259]: pgmap v121: 129 pgs: 129 active+clean; 4.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 23 op/s 2024-04-21T08:42:44.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:44 smithi179 ceph-mon[31829]: pgmap v121: 129 pgs: 129 active+clean; 4.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 23 op/s 2024-04-21T08:42:44.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:44 smithi086 ceph-mon[27939]: pgmap v121: 129 pgs: 129 active+clean; 4.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 23 op/s 2024-04-21T08:42:45.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:45 smithi088 ceph-mon[31259]: scrub summary: idle 2024-04-21T08:42:45.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:45 smithi179 ceph-mon[31829]: scrub summary: idle 2024-04-21T08:42:45.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:45 smithi086 ceph-mon[27939]: scrub summary: idle 2024-04-21T08:42:46.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:46 smithi088 ceph-mon[31259]: pgmap v122: 129 pgs: 129 active+clean; 4.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 17 op/s 2024-04-21T08:42:46.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:46 smithi179 ceph-mon[31829]: pgmap v122: 129 pgs: 129 active+clean; 4.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 17 op/s 2024-04-21T08:42:46.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:46 smithi086 ceph-mon[27939]: pgmap v122: 129 pgs: 129 active+clean; 4.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 17 op/s 2024-04-21T08:42:48.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:48 smithi088 ceph-mon[31259]: pgmap v123: 129 pgs: 129 active+clean; 4.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 17 op/s 2024-04-21T08:42:48.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:48 smithi179 ceph-mon[31829]: pgmap v123: 129 pgs: 129 active+clean; 4.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 17 op/s 2024-04-21T08:42:48.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:48 smithi086 ceph-mon[27939]: pgmap v123: 129 pgs: 129 active+clean; 4.5 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 17 op/s 2024-04-21T08:42:50.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.426+0000 7f5b3e0ba640 1 -- 172.21.15.86:0/1046268626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b381072a0 msgr2=0x7f5b3810ee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:50.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.426+0000 7f5b3e0ba640 1 --2- 172.21.15.86:0/1046268626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b381072a0 0x7f5b3810ee70 secure :-1 s=READY pgs=1107 cs=0 l=1 rev1=1 crypto rx=0x7f5b30009ca0 tx=0x7f5b3002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:42:50.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.426+0000 7f5b3e0ba640 1 -- 172.21.15.86:0/1046268626 shutdown_connections 2024-04-21T08:42:50.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.426+0000 7f5b3e0ba640 1 --2- 172.21.15.86:0/1046268626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b381072a0 0x7f5b3810ee70 unknown :-1 s=CLOSED pgs=1107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:50.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.426+0000 7f5b3e0ba640 1 --2- 172.21.15.86:0/1046268626 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b38102b80 0x7f5b38106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:50.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.426+0000 7f5b3e0ba640 1 --2- 172.21.15.86:0/1046268626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b381021d0 0x7f5b381025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:50.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.426+0000 7f5b3e0ba640 1 -- 172.21.15.86:0/1046268626 >> 172.21.15.86:0/1046268626 conn(0x7f5b380793e0 msgr2=0x7f5b380797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:50.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.426+0000 7f5b3e0ba640 1 -- 172.21.15.86:0/1046268626 shutdown_connections 2024-04-21T08:42:50.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.426+0000 7f5b3e0ba640 1 -- 172.21.15.86:0/1046268626 wait complete. 2024-04-21T08:42:50.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.426+0000 7f5b3e0ba640 1 Processor -- start 2024-04-21T08:42:50.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.427+0000 7f5b3e0ba640 1 -- start start 2024-04-21T08:42:50.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.427+0000 7f5b3e0ba640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b381021d0 0x7f5b38074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:50.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.427+0000 7f5b3e0ba640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b381072a0 0x7f5b38074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:50.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.427+0000 7f5b3e0ba640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b38074cd0 0x7f5b381a6820 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:50.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.427+0000 7f5b3e0ba640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5b381121f0 con 0x7f5b381021d0 2024-04-21T08:42:50.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.427+0000 7f5b3e0ba640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5b38112070 con 0x7f5b38074cd0 2024-04-21T08:42:50.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.427+0000 7f5b3e0ba640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5b38112370 con 0x7f5b381072a0 2024-04-21T08:42:50.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.427+0000 7f5b377fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b381021d0 0x7f5b38074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:50.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.427+0000 7f5b377fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b381021d0 0x7f5b38074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38002/0 (socket says 172.21.15.86:38002) 2024-04-21T08:42:50.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.427+0000 7f5b377fe640 1 -- 172.21.15.86:0/3115648350 learned_addr learned my addr 172.21.15.86:0/3115648350 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:50.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.427+0000 7f5b2ffff640 1 --2- 172.21.15.86:0/3115648350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b381072a0 0x7f5b38074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:50.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.427+0000 7f5b37fff640 1 --2- 172.21.15.86:0/3115648350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b38074cd0 0x7f5b381a6820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:50.430 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.427+0000 7f5b377fe640 1 -- 172.21.15.86:0/3115648350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b381072a0 msgr2=0x7f5b38074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:50.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.427+0000 7f5b377fe640 1 --2- 172.21.15.86:0/3115648350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b381072a0 0x7f5b38074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:50.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.427+0000 7f5b377fe640 1 -- 172.21.15.86:0/3115648350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b38074cd0 msgr2=0x7f5b381a6820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:50.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.427+0000 7f5b377fe640 1 --2- 172.21.15.86:0/3115648350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b38074cd0 0x7f5b381a6820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:50.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.427+0000 7f5b377fe640 1 -- 172.21.15.86:0/3115648350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5b381a6f20 con 0x7f5b381021d0 2024-04-21T08:42:50.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.427+0000 7f5b37fff640 1 --2- 172.21.15.86:0/3115648350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b38074cd0 0x7f5b381a6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:50.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.427+0000 7f5b2ffff640 1 --2- 172.21.15.86:0/3115648350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b381072a0 0x7f5b38074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:50.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.427+0000 7f5b377fe640 1 --2- 172.21.15.86:0/3115648350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b381021d0 0x7f5b38074250 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f5b2800e9f0 tx=0x7f5b2800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:50.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.428+0000 7f5b357fa640 1 -- 172.21.15.86:0/3115648350 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2800cd80 con 0x7f5b381021d0 2024-04-21T08:42:50.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.428+0000 7f5b357fa640 1 -- 172.21.15.86:0/3115648350 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b280108e0 con 0x7f5b381021d0 2024-04-21T08:42:50.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.428+0000 7f5b357fa640 1 -- 172.21.15.86:0/3115648350 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b28018980 con 0x7f5b381021d0 2024-04-21T08:42:50.431 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.428+0000 7f5b3e0ba640 1 -- 172.21.15.86:0/3115648350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5b381a7210 con 0x7f5b381021d0 2024-04-21T08:42:50.432 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.428+0000 7f5b3e0ba640 1 -- 172.21.15.86:0/3115648350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5b381a7760 con 0x7f5b381021d0 2024-04-21T08:42:50.438 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.430+0000 7f5b357fa640 1 -- 172.21.15.86:0/3115648350 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5b28010430 con 0x7f5b381021d0 2024-04-21T08:42:50.438 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.430+0000 7f5b40d87740 1 Processor -- start 2024-04-21T08:42:50.438 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.430+0000 7f5b40d87740 1 -- start start 2024-04-21T08:42:50.438 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.430+0000 7f5b357fa640 1 --2- 172.21.15.86:0/3115648350 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5b240a3df0 0x7f5b240a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:50.438 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.430+0000 7f5b40d87740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555f7c7168d0 0x555f7c715620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:50.438 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.430+0000 7f5b40d87740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7c715b60 0x555f7c715fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:50.438 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.431+0000 7f5b40d87740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f7c71f0e0 0x555f7c7214d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:50.438 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.431+0000 7f5b40d87740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555f7c67a8d0 con 0x555f7c715b60 2024-04-21T08:42:50.438 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.431+0000 7f5b40d87740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x555f7c617e80 con 0x555f7c7168d0 2024-04-21T08:42:50.438 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.431+0000 7f5b37fff640 1 --2- 172.21.15.86:0/3115648350 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5b240a3df0 0x7f5b240a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:50.438 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.431+0000 7f5b40d87740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555f7c676890 con 0x555f7c71f0e0 2024-04-21T08:42:50.438 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.431+0000 7f5b357fa640 1 -- 172.21.15.86:0/3115648350 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f5b280d0fc0 con 0x7f5b381021d0 2024-04-21T08:42:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.431+0000 7f5b2ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7c715b60 0x555f7c715fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.431+0000 7f5b2ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7c715b60 0x555f7c715fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38018/0 (socket says 172.21.15.86:38018) 2024-04-21T08:42:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.431+0000 7f5b2ffff640 1 -- 172.21.15.86:0/84148581 learned_addr learned my addr 172.21.15.86:0/84148581 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.431+0000 7f5b377fe640 1 --2- 172.21.15.86:0/84148581 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555f7c7168d0 0x555f7c715620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.431+0000 7f5b2ffff640 1 -- 172.21.15.86:0/84148581 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f7c71f0e0 msgr2=0x555f7c7214d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.431+0000 7f5b37fff640 1 --2- 172.21.15.86:0/84148581 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f7c71f0e0 0x555f7c7214d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.431+0000 7f5b2ffff640 1 --2- 172.21.15.86:0/84148581 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f7c71f0e0 0x555f7c7214d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.431+0000 7f5b2ffff640 1 -- 172.21.15.86:0/84148581 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555f7c7168d0 msgr2=0x555f7c715620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.431+0000 7f5b2ffff640 1 --2- 172.21.15.86:0/84148581 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555f7c7168d0 0x555f7c715620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.431+0000 7f5b2ffff640 1 -- 172.21.15.86:0/84148581 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f7c639320 con 0x555f7c715b60 2024-04-21T08:42:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.431+0000 7f5b377fe640 1 --2- 172.21.15.86:0/84148581 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555f7c7168d0 0x555f7c715620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.431+0000 7f5b37fff640 1 --2- 172.21.15.86:0/84148581 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f7c71f0e0 0x555f7c7214d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.431+0000 7f5b37fff640 1 --2- 172.21.15.86:0/3115648350 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5b240a3df0 0x7f5b240a62b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5b30032460 tx=0x7f5b30009a00 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.431+0000 7f5b2ffff640 1 --2- 172.21.15.86:0/84148581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7c715b60 0x555f7c715fe0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f5b2000dc00 tx=0x7f5b2000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dddd615fead5b9d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:50.439 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.431+0000 7f5b2dffb640 1 -- 172.21.15.86:0/84148581 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b20005020 con 0x555f7c715b60 2024-04-21T08:42:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.431+0000 7f5b2dffb640 1 -- 172.21.15.86:0/84148581 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b200047a0 con 0x555f7c715b60 2024-04-21T08:42:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.431+0000 7f5b2dffb640 1 -- 172.21.15.86:0/84148581 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b200116a0 con 0x555f7c715b60 2024-04-21T08:42:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.432+0000 7f5b40d87740 1 -- 172.21.15.86:0/84148581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7c715b60 msgr2=0x555f7c715fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.432+0000 7f5b40d87740 1 --2- 172.21.15.86:0/84148581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7c715b60 0x555f7c715fe0 secure :-1 s=READY pgs=1109 cs=0 l=1 rev1=1 crypto rx=0x7f5b2000dc00 tx=0x7f5b2000f730 comp rx=0 tx=0).stop 2024-04-21T08:42:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.432+0000 7f5b40d87740 1 -- 172.21.15.86:0/84148581 shutdown_connections 2024-04-21T08:42:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.432+0000 7f5b40d87740 1 --2- 172.21.15.86:0/84148581 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f7c71f0e0 0x555f7c7214d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.432+0000 7f5b40d87740 1 --2- 172.21.15.86:0/84148581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7c715b60 0x555f7c715fe0 unknown :-1 s=CLOSED pgs=1109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.432+0000 7f5b40d87740 1 --2- 172.21.15.86:0/84148581 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555f7c7168d0 0x555f7c715620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.432+0000 7f5b40d87740 1 -- 172.21.15.86:0/84148581 >> 172.21.15.86:0/84148581 conn(0x555f7c6dcc90 msgr2=0x555f7c710850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.432+0000 7f5b40d87740 1 -- 172.21.15.86:0/84148581 shutdown_connections 2024-04-21T08:42:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.432+0000 7f5b40d87740 1 -- 172.21.15.86:0/84148581 wait complete. 2024-04-21T08:42:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.432+0000 7f5b40d87740 1 -- start start 2024-04-21T08:42:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.433+0000 7f5b40d87740 1 Processor -- start 2024-04-21T08:42:50.440 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.433+0000 7f5b40d87740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555f7c715b60 0x555f7c71e9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.433+0000 7f5b40d87740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7c7168d0 0x555f7c71bde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.433+0000 7f5b40d87740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f7c71f0e0 0x555f7c71c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.433+0000 7f5b40d87740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555f7c721e10 con 0x555f7c7168d0 2024-04-21T08:42:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.433+0000 7f5b40d87740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x555f7c602eb0 con 0x555f7c715b60 2024-04-21T08:42:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.433+0000 7f5b40d87740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555f7c721f90 con 0x555f7c71f0e0 2024-04-21T08:42:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.433+0000 7f5b40d87740 10 client.0 _ll_register_callbacks cb 0x555f7c62fb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.433+0000 7f5b2ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7c7168d0 0x555f7c71bde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.433+0000 7f5b2ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7c7168d0 0x555f7c71bde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38030/0 (socket says 172.21.15.86:38030) 2024-04-21T08:42:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.433+0000 7f5b2ffff640 1 -- 172.21.15.86:0/2368044456 learned_addr learned my addr 172.21.15.86:0/2368044456 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.433+0000 7f5b377fe640 1 --2- 172.21.15.86:0/2368044456 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555f7c715b60 0x555f7c71e9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.433+0000 7f5b37fff640 1 --2- 172.21.15.86:0/2368044456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f7c71f0e0 0x555f7c71c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.433+0000 7f5b2ffff640 1 -- 172.21.15.86:0/2368044456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f7c71f0e0 msgr2=0x555f7c71c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.433+0000 7f5b2ffff640 1 --2- 172.21.15.86:0/2368044456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f7c71f0e0 0x555f7c71c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.433+0000 7f5b2ffff640 1 -- 172.21.15.86:0/2368044456 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555f7c715b60 msgr2=0x555f7c71e9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:50.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.433+0000 7f5b2ffff640 1 --2- 172.21.15.86:0/2368044456 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555f7c715b60 0x555f7c71e9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.433+0000 7f5b2ffff640 1 -- 172.21.15.86:0/2368044456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f7c38d210 con 0x555f7c7168d0 2024-04-21T08:42:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.434+0000 7f5b2ffff640 1 --2- 172.21.15.86:0/2368044456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7c7168d0 0x555f7c71bde0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f5b20000c00 tx=0x7f5b20004b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5ed667cba118ccd9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.434+0000 7f5b16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.434+0000 7f5b16ffd640 1 -- 172.21.15.86:0/2368044456 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2002a060 con 0x555f7c7168d0 2024-04-21T08:42:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.434+0000 7f5b40d87740 1 -- 172.21.15.86:0/2368044456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b200047a0 con 0x555f7c7168d0 2024-04-21T08:42:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.434+0000 7f5b16ffd640 1 -- 172.21.15.86:0/2368044456 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f5b20004250 con 0x555f7c7168d0 2024-04-21T08:42:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.434+0000 7f5b16ffd640 1 -- 172.21.15.86:0/2368044456 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5b20031050 con 0x555f7c7168d0 2024-04-21T08:42:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.434+0000 7f5b40d87740 10 client.17502 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.434+0000 7f5b40d87740 1 -- 172.21.15.86:0/2368044456 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555f7c459620 con 0x555f7c7168d0 2024-04-21T08:42:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.434+0000 7f5b16ffd640 1 -- 172.21.15.86:0/2368044456 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b200047a0 con 0x555f7c7168d0 2024-04-21T08:42:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.434+0000 7f5b16ffd640 1 -- 172.21.15.86:0/2368044456 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5b200035f0 con 0x555f7c7168d0 2024-04-21T08:42:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.434+0000 7f5b40d87740 10 client.17502 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.434+0000 7f5b40d87740 10 client.17502 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:50.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.434+0000 7f5b40d87740 20 client.17502 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.434+0000 7f5b40d87740 1 --2- 172.21.15.86:0/2368044456 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x555f7c72efc0 0x555f7c7313b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.434+0000 7f5b40d87740 4 client.17502 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:42:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.434+0000 7f5b40d87740 1 -- 172.21.15.86:0/2368044456 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555f7c68e880 con 0x555f7c72efc0 2024-04-21T08:42:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.435+0000 7f5b377fe640 1 --2- 172.21.15.86:0/2368044456 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x555f7c72efc0 0x555f7c7313b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.435+0000 7f5b16ffd640 10 client.17502 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.435+0000 7f5b377fe640 1 --2- 172.21.15.86:0/2368044456 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x555f7c72efc0 0x555f7c7313b0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e81eb0b2cd4e2b6 server_cookie=7e7bd5ddee3df49 in_seq=0 out_seq=0 2024-04-21T08:42:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.436+0000 7f5b16ffd640 1 -- 172.21.15.86:0/2368044456 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555f7c68e880 con 0x555f7c72efc0 2024-04-21T08:42:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.436+0000 7f5b16ffd640 10 client.17502 handle_command_reply: tid=0 2024-04-21T08:42:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.436+0000 7f5b40d87740 1 client.17502 shutdown 2024-04-21T08:42:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.436+0000 7f5b40d87740 1 -- 172.21.15.86:0/2368044456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7c7168d0 msgr2=0x555f7c71bde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.436+0000 7f5b40d87740 1 --2- 172.21.15.86:0/2368044456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7c7168d0 0x555f7c71bde0 secure :-1 s=READY pgs=1110 cs=0 l=1 rev1=1 crypto rx=0x7f5b20000c00 tx=0x7f5b20004b80 comp rx=0 tx=0).stop 2024-04-21T08:42:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.436+0000 7f5b40d87740 1 -- 172.21.15.86:0/2368044456 shutdown_connections 2024-04-21T08:42:50.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.436+0000 7f5b40d87740 1 --2- 172.21.15.86:0/2368044456 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x555f7c72efc0 0x555f7c7313b0 crc :-1 s=READY pgs=149 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.436+0000 7f5b40d87740 1 --2- 172.21.15.86:0/2368044456 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f7c71f0e0 0x555f7c71c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.436+0000 7f5b40d87740 1 --2- 172.21.15.86:0/2368044456 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7c7168d0 0x555f7c71bde0 unknown :-1 s=CLOSED pgs=1110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.436+0000 7f5b16ffd640 0 client.17502 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.436+0000 7f5b40d87740 1 --2- 172.21.15.86:0/2368044456 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555f7c715b60 0x555f7c71e9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.436+0000 7f5b40d87740 1 -- 172.21.15.86:0/2368044456 >> 172.21.15.86:0/2368044456 conn(0x555f7c6dcc90 msgr2=0x555f7c7125a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.436+0000 7f5b40d87740 1 -- 172.21.15.86:0/2368044456 shutdown_connections 2024-04-21T08:42:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.436+0000 7f5b40d87740 1 -- 172.21.15.86:0/2368044456 wait complete. 2024-04-21T08:42:50.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.436+0000 7f5b40d87740 20 client.17502 trim_cache size 0 max 16384 2024-04-21T08:42:50.453 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.451+0000 7f5b40d87740 1 Processor -- start 2024-04-21T08:42:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.451+0000 7f5b40d87740 1 -- start start 2024-04-21T08:42:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.452+0000 7f5b40d87740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555f7c715b60 0x555f7c72ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.452+0000 7f5b40d87740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7c7168d0 0x555f7c71b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.452+0000 7f5b40d87740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f7c71f0e0 0x555f7c7369e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.452+0000 7f5b40d87740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5b381b12e0 con 0x555f7c7168d0 2024-04-21T08:42:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.452+0000 7f5b40d87740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5b381b2e20 con 0x555f7c715b60 2024-04-21T08:42:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.452+0000 7f5b40d87740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5b381b19b0 con 0x555f7c71f0e0 2024-04-21T08:42:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.452+0000 7f5b2ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7c7168d0 0x555f7c71b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.452+0000 7f5b2ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7c7168d0 0x555f7c71b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38034/0 (socket says 172.21.15.86:38034) 2024-04-21T08:42:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.452+0000 7f5b2ffff640 1 -- 172.21.15.86:0/3515068977 learned_addr learned my addr 172.21.15.86:0/3515068977 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.452+0000 7f5b37fff640 1 --2- 172.21.15.86:0/3515068977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f7c71f0e0 0x555f7c7369e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.452+0000 7f5b377fe640 1 --2- 172.21.15.86:0/3515068977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555f7c715b60 0x555f7c72ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:50.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.452+0000 7f5b2ffff640 1 -- 172.21.15.86:0/3515068977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f7c71f0e0 msgr2=0x555f7c7369e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.452+0000 7f5b2ffff640 1 --2- 172.21.15.86:0/3515068977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f7c71f0e0 0x555f7c7369e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.452+0000 7f5b2ffff640 1 -- 172.21.15.86:0/3515068977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555f7c715b60 msgr2=0x555f7c72ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.452+0000 7f5b2ffff640 1 --2- 172.21.15.86:0/3515068977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555f7c715b60 0x555f7c72ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.452+0000 7f5b2ffff640 1 -- 172.21.15.86:0/3515068977 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555f7c714e30 con 0x555f7c7168d0 2024-04-21T08:42:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.452+0000 7f5b377fe640 1 --2- 172.21.15.86:0/3515068977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555f7c715b60 0x555f7c72ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:42:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.452+0000 7f5b37fff640 1 --2- 172.21.15.86:0/3515068977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f7c71f0e0 0x555f7c7369e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.452+0000 7f5b2ffff640 1 --2- 172.21.15.86:0/3515068977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7c7168d0 0x555f7c71b360 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f5b20003a40 tx=0x7f5b2000fd60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7208fd1c90207cf4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.452+0000 7f5b2d7fa640 1 -- 172.21.15.86:0/3515068977 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b2002a050 con 0x555f7c7168d0 2024-04-21T08:42:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.452+0000 7f5b2d7fa640 1 -- 172.21.15.86:0/3515068977 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b2000dcf0 con 0x555f7c7168d0 2024-04-21T08:42:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.452+0000 7f5b2d7fa640 1 -- 172.21.15.86:0/3515068977 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b20004250 con 0x555f7c7168d0 2024-04-21T08:42:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.453+0000 7f5b40d87740 1 -- 172.21.15.86:0/3515068977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7c7168d0 msgr2=0x555f7c71b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.453+0000 7f5b40d87740 1 --2- 172.21.15.86:0/3515068977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7c7168d0 0x555f7c71b360 secure :-1 s=READY pgs=1111 cs=0 l=1 rev1=1 crypto rx=0x7f5b20003a40 tx=0x7f5b2000fd60 comp rx=0 tx=0).stop 2024-04-21T08:42:50.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.453+0000 7f5b40d87740 1 -- 172.21.15.86:0/3515068977 shutdown_connections 2024-04-21T08:42:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.453+0000 7f5b40d87740 1 --2- 172.21.15.86:0/3515068977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f7c71f0e0 0x555f7c7369e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.453+0000 7f5b40d87740 1 --2- 172.21.15.86:0/3515068977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7c7168d0 0x555f7c71b360 unknown :-1 s=CLOSED pgs=1111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.453+0000 7f5b40d87740 1 --2- 172.21.15.86:0/3515068977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555f7c715b60 0x555f7c72ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.453+0000 7f5b40d87740 1 -- 172.21.15.86:0/3515068977 >> 172.21.15.86:0/3515068977 conn(0x555f7c6dcc90 msgr2=0x555f7c717030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.453+0000 7f5b40d87740 1 -- 172.21.15.86:0/3515068977 shutdown_connections 2024-04-21T08:42:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.453+0000 7f5b40d87740 1 -- 172.21.15.86:0/3515068977 wait complete. 2024-04-21T08:42:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.453+0000 7f5b40d87740 1 -- start start 2024-04-21T08:42:50.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.454+0000 7f5b40d87740 1 Processor -- start 2024-04-21T08:42:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.454+0000 7f5b40d87740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7c715b60 0x555f7c727450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.454+0000 7f5b40d87740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555f7c7168d0 0x555f7c7190a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.454+0000 7f5b40d87740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f7c71f0e0 0x555f7c7195e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.454+0000 7f5b40d87740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5b381b2750 con 0x555f7c715b60 2024-04-21T08:42:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.454+0000 7f5b40d87740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5b381b2080 con 0x555f7c7168d0 2024-04-21T08:42:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.454+0000 7f5b40d87740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5b381af2a0 con 0x555f7c71f0e0 2024-04-21T08:42:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.454+0000 7f5b40d87740 10 client.0 _ll_register_callbacks cb 0x7f5b20015050 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:42:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.454+0000 7f5b377fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7c715b60 0x555f7c727450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.454+0000 7f5b377fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7c715b60 0x555f7c727450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38042/0 (socket says 172.21.15.86:38042) 2024-04-21T08:42:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.454+0000 7f5b377fe640 1 -- 172.21.15.86:0/1796556852 learned_addr learned my addr 172.21.15.86:0/1796556852 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:42:50.459 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.454+0000 7f5b2ffff640 1 --2- 172.21.15.86:0/1796556852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555f7c7168d0 0x555f7c7190a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.454+0000 7f5b37fff640 1 --2- 172.21.15.86:0/1796556852 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f7c71f0e0 0x555f7c7195e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.454+0000 7f5b377fe640 1 -- 172.21.15.86:0/1796556852 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f7c71f0e0 msgr2=0x555f7c7195e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.454+0000 7f5b377fe640 1 --2- 172.21.15.86:0/1796556852 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f7c71f0e0 0x555f7c7195e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.454+0000 7f5b377fe640 1 -- 172.21.15.86:0/1796556852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555f7c7168d0 msgr2=0x555f7c7190a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:42:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.454+0000 7f5b377fe640 1 --2- 172.21.15.86:0/1796556852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555f7c7168d0 0x555f7c7190a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.454+0000 7f5b377fe640 1 -- 172.21.15.86:0/1796556852 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555f7c719c20 con 0x555f7c715b60 2024-04-21T08:42:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.454+0000 7f5b37fff640 1 --2- 172.21.15.86:0/1796556852 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f7c71f0e0 0x555f7c7195e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:42:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.455+0000 7f5b377fe640 1 --2- 172.21.15.86:0/1796556852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7c715b60 0x555f7c727450 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f5b28000c00 tx=0x7f5b28098f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f199e45d9675cda server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:42:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.455+0000 7f5b16ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:42:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.455+0000 7f5b16ffd640 1 -- 172.21.15.86:0/1796556852 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5b280a5860 con 0x555f7c715b60 2024-04-21T08:42:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.455+0000 7f5b16ffd640 1 -- 172.21.15.86:0/1796556852 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f5b28010a90 con 0x555f7c715b60 2024-04-21T08:42:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.455+0000 7f5b40d87740 1 -- 172.21.15.86:0/1796556852 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5b2000dcf0 con 0x555f7c715b60 2024-04-21T08:42:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.455+0000 7f5b16ffd640 1 -- 172.21.15.86:0/1796556852 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5b28099260 con 0x555f7c715b60 2024-04-21T08:42:50.460 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.455+0000 7f5b40d87740 10 client.17508 fetch_fsmap learned FSMap version 29 2024-04-21T08:42:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.455+0000 7f5b40d87740 1 -- 172.21.15.86:0/1796556852 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555f7c795250 con 0x555f7c715b60 2024-04-21T08:42:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.455+0000 7f5b16ffd640 1 -- 172.21.15.86:0/1796556852 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5b2000dcf0 con 0x555f7c715b60 2024-04-21T08:42:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.455+0000 7f5b16ffd640 1 -- 172.21.15.86:0/1796556852 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5b280ce620 con 0x555f7c715b60 2024-04-21T08:42:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.455+0000 7f5b40d87740 10 client.17508 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:42:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.455+0000 7f5b40d87740 10 client.17508 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:42:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.455+0000 7f5b40d87740 20 client.17508 populate_metadata read hostname 'smithi086' 2024-04-21T08:42:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.455+0000 7f5b40d87740 1 --2- 172.21.15.86:0/1796556852 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x555f7c7957b0 0x555f7c797bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:42:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.455+0000 7f5b40d87740 4 client.17508 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:42:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.455+0000 7f5b40d87740 1 -- 172.21.15.86:0/1796556852 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x555f7c61a3d0 con 0x555f7c7957b0 2024-04-21T08:42:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.456+0000 7f5b2ffff640 1 --2- 172.21.15.86:0/1796556852 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x555f7c7957b0 0x555f7c797bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:42:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.456+0000 7f5b16ffd640 10 client.17508 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:50.461 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.456+0000 7f5b2ffff640 1 --2- 172.21.15.86:0/1796556852 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x555f7c7957b0 0x555f7c797bd0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=99f652ca8d93f781 server_cookie=8ad5d2ad8ca0089b in_seq=0 out_seq=0 2024-04-21T08:42:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.472+0000 7f5b16ffd640 1 -- 172.21.15.86:0/1796556852 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x555f7c61a3d0 con 0x555f7c7957b0 2024-04-21T08:42:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.472+0000 7f5b16ffd640 10 client.17508 handle_command_reply: tid=0 2024-04-21T08:42:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.472+0000 7f5b40d87740 1 client.17508 shutdown 2024-04-21T08:42:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.472+0000 7f5b40d87740 1 -- 172.21.15.86:0/1796556852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7c715b60 msgr2=0x555f7c727450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.472+0000 7f5b40d87740 1 --2- 172.21.15.86:0/1796556852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7c715b60 0x555f7c727450 secure :-1 s=READY pgs=1112 cs=0 l=1 rev1=1 crypto rx=0x7f5b28000c00 tx=0x7f5b28098f40 comp rx=0 tx=0).stop 2024-04-21T08:42:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.472+0000 7f5b40d87740 1 -- 172.21.15.86:0/1796556852 shutdown_connections 2024-04-21T08:42:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.472+0000 7f5b40d87740 1 --2- 172.21.15.86:0/1796556852 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x555f7c7957b0 0x555f7c797bd0 crc :-1 s=READY pgs=150 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.472+0000 7f5b40d87740 1 --2- 172.21.15.86:0/1796556852 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555f7c71f0e0 0x555f7c7195e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:50.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.472+0000 7f5b40d87740 1 --2- 172.21.15.86:0/1796556852 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555f7c7168d0 0x555f7c7190a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.472+0000 7f5b40d87740 1 --2- 172.21.15.86:0/1796556852 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555f7c715b60 0x555f7c727450 unknown :-1 s=CLOSED pgs=1112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.472+0000 7f5b16ffd640 0 client.17508 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:42:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.472+0000 7f5b40d87740 1 -- 172.21.15.86:0/1796556852 >> 172.21.15.86:0/1796556852 conn(0x555f7c6dcc90 msgr2=0x555f7c71a630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.472+0000 7f5b40d87740 1 -- 172.21.15.86:0/1796556852 shutdown_connections 2024-04-21T08:42:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.472+0000 7f5b40d87740 1 -- 172.21.15.86:0/1796556852 wait complete. 2024-04-21T08:42:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.472+0000 7f5b40d87740 20 client.17508 trim_cache size 0 max 16384 2024-04-21T08:42:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:42:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:42:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:42:50.473 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:42:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.472+0000 7f5b3e0ba640 1 -- 172.21.15.86:0/3115648350 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5b240a3df0 msgr2=0x7f5b240a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.472+0000 7f5b3e0ba640 1 --2- 172.21.15.86:0/3115648350 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5b240a3df0 0x7f5b240a62b0 secure :-1 s=READY pgs=171 cs=0 l=1 rev1=1 crypto rx=0x7f5b30032460 tx=0x7f5b30009a00 comp rx=0 tx=0).stop 2024-04-21T08:42:50.474 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.472+0000 7f5b3e0ba640 1 -- 172.21.15.86:0/3115648350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b381021d0 msgr2=0x7f5b38074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:42:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.472+0000 7f5b3e0ba640 1 --2- 172.21.15.86:0/3115648350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b381021d0 0x7f5b38074250 secure :-1 s=READY pgs=1108 cs=0 l=1 rev1=1 crypto rx=0x7f5b2800e9f0 tx=0x7f5b2800eeb0 comp rx=0 tx=0).stop 2024-04-21T08:42:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.472+0000 7f5b3e0ba640 1 -- 172.21.15.86:0/3115648350 shutdown_connections 2024-04-21T08:42:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.472+0000 7f5b3e0ba640 1 --2- 172.21.15.86:0/3115648350 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5b240a3df0 0x7f5b240a62b0 unknown :-1 s=CLOSED pgs=171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.472+0000 7f5b3e0ba640 1 --2- 172.21.15.86:0/3115648350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5b38074cd0 0x7f5b381a6820 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.472+0000 7f5b3e0ba640 1 --2- 172.21.15.86:0/3115648350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5b381072a0 0x7f5b38074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.472+0000 7f5b3e0ba640 1 --2- 172.21.15.86:0/3115648350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5b381021d0 0x7f5b38074250 unknown :-1 s=CLOSED pgs=1108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:42:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.472+0000 7f5b3e0ba640 1 -- 172.21.15.86:0/3115648350 >> 172.21.15.86:0/3115648350 conn(0x7f5b380793e0 msgr2=0x7f5b380ffc20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:42:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.472+0000 7f5b3e0ba640 1 -- 172.21.15.86:0/3115648350 shutdown_connections 2024-04-21T08:42:50.475 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:42:50.472+0000 7f5b3e0ba640 1 -- 172.21.15.86:0/3115648350 wait complete. 2024-04-21T08:42:50.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:50 smithi088 ceph-mon[31259]: pgmap v124: 129 pgs: 129 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 23 op/s 2024-04-21T08:42:50.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:50 smithi179 ceph-mon[31829]: pgmap v124: 129 pgs: 129 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 23 op/s 2024-04-21T08:42:50.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:50 smithi086 ceph-mon[27939]: pgmap v124: 129 pgs: 129 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 23 op/s 2024-04-21T08:42:52.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:52 smithi088 ceph-mon[31259]: pgmap v125: 129 pgs: 129 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 25 KiB/s wr, 14 op/s 2024-04-21T08:42:52.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:52 smithi179 ceph-mon[31829]: pgmap v125: 129 pgs: 129 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 25 KiB/s wr, 14 op/s 2024-04-21T08:42:52.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:52 smithi086 ceph-mon[27939]: pgmap v125: 129 pgs: 129 active+clean; 4.6 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 25 KiB/s wr, 14 op/s 2024-04-21T08:42:54.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:54 smithi088 ceph-mon[31259]: pgmap v126: 129 pgs: 129 active+clean; 4.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 42 KiB/s wr, 23 op/s 2024-04-21T08:42:54.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:54 smithi179 ceph-mon[31829]: pgmap v126: 129 pgs: 129 active+clean; 4.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 42 KiB/s wr, 23 op/s 2024-04-21T08:42:54.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:54 smithi086 ceph-mon[27939]: pgmap v126: 129 pgs: 129 active+clean; 4.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 42 KiB/s wr, 23 op/s 2024-04-21T08:42:55.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:55 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:42:55.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:55 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:42:55.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:55 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:42:56.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:56 smithi088 ceph-mon[31259]: pgmap v127: 129 pgs: 129 active+clean; 4.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-21T08:42:56.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:56 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:42:56.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:56 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:42:56.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:56 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:42:56.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:56 smithi179 ceph-mon[31829]: pgmap v127: 129 pgs: 129 active+clean; 4.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-21T08:42:56.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:56 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:42:56.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:56 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:42:56.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:56 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:42:56.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:56 smithi086 ceph-mon[27939]: pgmap v127: 129 pgs: 129 active+clean; 4.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-21T08:42:56.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:56 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:42:56.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:56 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:42:56.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:56 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:42:58.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:42:58 smithi088 ceph-mon[31259]: pgmap v128: 129 pgs: 129 active+clean; 4.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-21T08:42:58.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:42:58 smithi179 ceph-mon[31829]: pgmap v128: 129 pgs: 129 active+clean; 4.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-21T08:42:58.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:42:58 smithi086 ceph-mon[27939]: pgmap v128: 129 pgs: 129 active+clean; 4.8 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 30 KiB/s wr, 15 op/s 2024-04-21T08:43:00.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:00 smithi088 ceph-mon[31259]: pgmap v129: 129 pgs: 129 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 42 KiB/s wr, 22 op/s 2024-04-21T08:43:00.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:00 smithi179 ceph-mon[31829]: pgmap v129: 129 pgs: 129 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 42 KiB/s wr, 22 op/s 2024-04-21T08:43:00.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:00 smithi086 ceph-mon[27939]: pgmap v129: 129 pgs: 129 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 42 KiB/s wr, 22 op/s 2024-04-21T08:43:02.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:02 smithi179 ceph-mon[31829]: pgmap v130: 129 pgs: 129 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-04-21T08:43:02.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:02 smithi086 ceph-mon[27939]: pgmap v130: 129 pgs: 129 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-04-21T08:43:02.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:02 smithi088 ceph-mon[31259]: pgmap v130: 129 pgs: 129 active+clean; 4.9 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 30 KiB/s rd, 29 KiB/s wr, 15 op/s 2024-04-21T08:43:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.569+0000 7f4b7ec81640 1 -- 172.21.15.86:0/2478021681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b781011a0 msgr2=0x7f4b78101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.569+0000 7f4b7ec81640 1 --2- 172.21.15.86:0/2478021681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b781011a0 0x7f4b78101620 secure :-1 s=READY pgs=1113 cs=0 l=1 rev1=1 crypto rx=0x7f4b64009900 tx=0x7f4b6402bc80 comp rx=0 tx=0).stop 2024-04-21T08:43:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.569+0000 7f4b7ec81640 1 -- 172.21.15.86:0/2478021681 shutdown_connections 2024-04-21T08:43:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.569+0000 7f4b7ec81640 1 --2- 172.21.15.86:0/2478021681 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b78101b60 0x7f4b7810dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.569+0000 7f4b7ec81640 1 --2- 172.21.15.86:0/2478021681 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b781011a0 0x7f4b78101620 unknown :-1 s=CLOSED pgs=1113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.569+0000 7f4b7ec81640 1 --2- 172.21.15.86:0/2478021681 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b78106fb0 0x7f4b78107390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:03.571 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.569+0000 7f4b7ec81640 1 -- 172.21.15.86:0/2478021681 >> 172.21.15.86:0/2478021681 conn(0x7f4b78076c50 msgr2=0x7f4b78077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.569+0000 7f4b7ec81640 1 -- 172.21.15.86:0/2478021681 shutdown_connections 2024-04-21T08:43:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.569+0000 7f4b7ec81640 1 -- 172.21.15.86:0/2478021681 wait complete. 2024-04-21T08:43:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.570+0000 7f4b7ec81640 1 Processor -- start 2024-04-21T08:43:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.570+0000 7f4b7ec81640 1 -- start start 2024-04-21T08:43:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.570+0000 7f4b7ec81640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b781011a0 0x7f4b78198610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.570+0000 7f4b7ec81640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b78101b60 0x7f4b78198b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.570+0000 7f4b7ec81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b78106fb0 0x7f4b7819ced0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.570+0000 7f4b7ec81640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b78111090 con 0x7f4b78106fb0 2024-04-21T08:43:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.570+0000 7f4b7ec81640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4b78110f10 con 0x7f4b78101b60 2024-04-21T08:43:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.570+0000 7f4b7ec81640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4b78111210 con 0x7f4b781011a0 2024-04-21T08:43:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.570+0000 7f4b7d21e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b78106fb0 0x7f4b7819ced0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.570+0000 7f4b7d21e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b78106fb0 0x7f4b7819ced0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32984/0 (socket says 172.21.15.86:32984) 2024-04-21T08:43:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.570+0000 7f4b7d21e640 1 -- 172.21.15.86:0/361411168 learned_addr learned my addr 172.21.15.86:0/361411168 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.571+0000 7f4b7ca1d640 1 --2- 172.21.15.86:0/361411168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b781011a0 0x7f4b78198610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:03.572 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.571+0000 7f4b77fff640 1 --2- 172.21.15.86:0/361411168 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b78101b60 0x7f4b78198b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.571+0000 7f4b7d21e640 1 -- 172.21.15.86:0/361411168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b781011a0 msgr2=0x7f4b78198610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.571+0000 7f4b7d21e640 1 --2- 172.21.15.86:0/361411168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b781011a0 0x7f4b78198610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.571+0000 7f4b7d21e640 1 -- 172.21.15.86:0/361411168 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b78101b60 msgr2=0x7f4b78198b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.571+0000 7f4b7d21e640 1 --2- 172.21.15.86:0/361411168 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b78101b60 0x7f4b78198b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.571+0000 7f4b7d21e640 1 -- 172.21.15.86:0/361411168 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b7819d600 con 0x7f4b78106fb0 2024-04-21T08:43:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.571+0000 7f4b77fff640 1 --2- 172.21.15.86:0/361411168 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b78101b60 0x7f4b78198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.571+0000 7f4b7d21e640 1 --2- 172.21.15.86:0/361411168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b78106fb0 0x7f4b7819ced0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000eaa0 tx=0x7f4b7000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.571+0000 7f4b75ffb640 1 -- 172.21.15.86:0/361411168 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b7000cdb0 con 0x7f4b78106fb0 2024-04-21T08:43:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.571+0000 7f4b7ec81640 1 -- 172.21.15.86:0/361411168 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b7819d8f0 con 0x7f4b78106fb0 2024-04-21T08:43:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.571+0000 7f4b75ffb640 1 -- 172.21.15.86:0/361411168 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b70005020 con 0x7f4b78106fb0 2024-04-21T08:43:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.571+0000 7f4b75ffb640 1 -- 172.21.15.86:0/361411168 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b700116d0 con 0x7f4b78106fb0 2024-04-21T08:43:03.573 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.571+0000 7f4b7ec81640 1 -- 172.21.15.86:0/361411168 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b781a5180 con 0x7f4b78106fb0 2024-04-21T08:43:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.574+0000 7f4b75ffb640 1 -- 172.21.15.86:0/361411168 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4b700054a0 con 0x7f4b78106fb0 2024-04-21T08:43:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.574+0000 7f4b81878740 1 Processor -- start 2024-04-21T08:43:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.575+0000 7f4b81878740 1 -- start start 2024-04-21T08:43:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.575+0000 7f4b75ffb640 1 --2- 172.21.15.86:0/361411168 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4b680a3dc0 0x7f4b680a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.575+0000 7f4b81878740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4b5518d0 0x55ec4b550620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.575+0000 7f4b81878740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4b550b60 0x55ec4b550fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.575+0000 7f4b81878740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec4b55e4f0 0x55ec4b5608e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.575+0000 7f4b81878740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ec4b4b58d0 con 0x55ec4b5518d0 2024-04-21T08:43:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.575+0000 7f4b81878740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55ec4b452e80 con 0x55ec4b550b60 2024-04-21T08:43:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.575+0000 7f4b81878740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ec4b4b1890 con 0x55ec4b55e4f0 2024-04-21T08:43:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.575+0000 7f4b77fff640 1 --2- 172.21.15.86:0/361411168 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4b680a3dc0 0x7f4b680a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.575+0000 7f4b75ffb640 1 -- 172.21.15.86:0/361411168 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f4b700d0df0 con 0x7f4b78106fb0 2024-04-21T08:43:03.579 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.575+0000 7f4b7ca1d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4b5518d0 0x55ec4b550620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.575+0000 7f4b7ca1d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4b5518d0 0x55ec4b550620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32988/0 (socket says 172.21.15.86:32988) 2024-04-21T08:43:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.575+0000 7f4b7ca1d640 1 -- 172.21.15.86:0/3322182375 learned_addr learned my addr 172.21.15.86:0/3322182375 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.575+0000 7f4b7d21e640 1 --2- 172.21.15.86:0/3322182375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec4b55e4f0 0x55ec4b5608e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.575+0000 7f4b7ca1d640 1 -- 172.21.15.86:0/3322182375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec4b55e4f0 msgr2=0x55ec4b5608e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.575+0000 7f4b77fff640 1 --2- 172.21.15.86:0/3322182375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4b550b60 0x55ec4b550fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.575+0000 7f4b7ca1d640 1 --2- 172.21.15.86:0/3322182375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec4b55e4f0 0x55ec4b5608e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.575+0000 7f4b7ca1d640 1 -- 172.21.15.86:0/3322182375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4b550b60 msgr2=0x55ec4b550fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.575+0000 7f4b7ca1d640 1 --2- 172.21.15.86:0/3322182375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4b550b60 0x55ec4b550fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.575+0000 7f4b7ca1d640 1 -- 172.21.15.86:0/3322182375 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec4b474320 con 0x55ec4b5518d0 2024-04-21T08:43:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.575+0000 7f4b77fff640 1 --2- 172.21.15.86:0/3322182375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4b550b60 0x55ec4b550fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.575+0000 7f4b77fff640 1 --2- 172.21.15.86:0/361411168 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4b680a3dc0 0x7f4b680a6280 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4b78105ca0 tx=0x7f4b64005d50 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.575+0000 7f4b7ca1d640 1 --2- 172.21.15.86:0/3322182375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4b5518d0 0x55ec4b550620 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00bcb0 tx=0x7f4b6c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3812da745b3d7ffe server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.576+0000 7f4b62ffd640 1 -- 172.21.15.86:0/3322182375 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c015070 con 0x55ec4b5518d0 2024-04-21T08:43:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.576+0000 7f4b62ffd640 1 -- 172.21.15.86:0/3322182375 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b6c01a910 con 0x55ec4b5518d0 2024-04-21T08:43:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.576+0000 7f4b62ffd640 1 -- 172.21.15.86:0/3322182375 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c011920 con 0x55ec4b5518d0 2024-04-21T08:43:03.580 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.576+0000 7f4b81878740 1 -- 172.21.15.86:0/3322182375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4b5518d0 msgr2=0x55ec4b550620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.576+0000 7f4b81878740 1 --2- 172.21.15.86:0/3322182375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4b5518d0 0x55ec4b550620 secure :-1 s=READY pgs=1115 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c00bcb0 tx=0x7f4b6c00f730 comp rx=0 tx=0).stop 2024-04-21T08:43:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.576+0000 7f4b81878740 1 -- 172.21.15.86:0/3322182375 shutdown_connections 2024-04-21T08:43:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.576+0000 7f4b81878740 1 --2- 172.21.15.86:0/3322182375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec4b55e4f0 0x55ec4b5608e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.576+0000 7f4b81878740 1 --2- 172.21.15.86:0/3322182375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4b550b60 0x55ec4b550fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.576+0000 7f4b81878740 1 --2- 172.21.15.86:0/3322182375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4b5518d0 0x55ec4b550620 unknown :-1 s=CLOSED pgs=1115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.576+0000 7f4b81878740 1 -- 172.21.15.86:0/3322182375 >> 172.21.15.86:0/3322182375 conn(0x55ec4b517c90 msgr2=0x55ec4b54b850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.576+0000 7f4b81878740 1 -- 172.21.15.86:0/3322182375 shutdown_connections 2024-04-21T08:43:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.576+0000 7f4b81878740 1 -- 172.21.15.86:0/3322182375 wait complete. 2024-04-21T08:43:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.577+0000 7f4b81878740 1 -- start start 2024-04-21T08:43:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.577+0000 7f4b81878740 1 Processor -- start 2024-04-21T08:43:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.577+0000 7f4b81878740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4b550b60 0x55ec4b55de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.577+0000 7f4b81878740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4b5518d0 0x55ec4b55b1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.577+0000 7f4b81878740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec4b55e4f0 0x55ec4b55b6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:03.581 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.577+0000 7f4b81878740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ec4b561220 con 0x55ec4b550b60 2024-04-21T08:43:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.577+0000 7f4b81878740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55ec4b43deb0 con 0x55ec4b5518d0 2024-04-21T08:43:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.577+0000 7f4b81878740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ec4b5613a0 con 0x55ec4b55e4f0 2024-04-21T08:43:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.577+0000 7f4b81878740 10 client.0 _ll_register_callbacks cb 0x55ec4b46ab40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.577+0000 7f4b7ca1d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4b550b60 0x55ec4b55de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.577+0000 7f4b7ca1d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4b550b60 0x55ec4b55de20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33002/0 (socket says 172.21.15.86:33002) 2024-04-21T08:43:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.577+0000 7f4b7ca1d640 1 -- 172.21.15.86:0/3785692725 learned_addr learned my addr 172.21.15.86:0/3785692725 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.578+0000 7f4b7d21e640 1 --2- 172.21.15.86:0/3785692725 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec4b55e4f0 0x55ec4b55b6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.578+0000 7f4b77fff640 1 --2- 172.21.15.86:0/3785692725 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4b5518d0 0x55ec4b55b1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.578+0000 7f4b7ca1d640 1 -- 172.21.15.86:0/3785692725 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec4b55e4f0 msgr2=0x55ec4b55b6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.578+0000 7f4b7ca1d640 1 --2- 172.21.15.86:0/3785692725 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec4b55e4f0 0x55ec4b55b6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.578+0000 7f4b7ca1d640 1 -- 172.21.15.86:0/3785692725 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4b5518d0 msgr2=0x55ec4b55b1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.578+0000 7f4b7ca1d640 1 --2- 172.21.15.86:0/3785692725 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4b5518d0 0x55ec4b55b1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.578+0000 7f4b7ca1d640 1 -- 172.21.15.86:0/3785692725 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec4b1c8210 con 0x55ec4b550b60 2024-04-21T08:43:03.582 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.578+0000 7f4b7ca1d640 1 --2- 172.21.15.86:0/3785692725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4b550b60 0x55ec4b55de20 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c011480 tx=0x7f4b6c00b9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=61cadb4c92dd362b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.578+0000 7f4b57fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.578+0000 7f4b57fff640 1 -- 172.21.15.86:0/3785692725 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b6c028060 con 0x55ec4b550b60 2024-04-21T08:43:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.578+0000 7f4b81878740 1 -- 172.21.15.86:0/3785692725 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b6c01a910 con 0x55ec4b550b60 2024-04-21T08:43:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.578+0000 7f4b57fff640 1 -- 172.21.15.86:0/3785692725 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f4b6c01aac0 con 0x55ec4b550b60 2024-04-21T08:43:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.578+0000 7f4b57fff640 1 -- 172.21.15.86:0/3785692725 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4b6c02f050 con 0x55ec4b550b60 2024-04-21T08:43:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.578+0000 7f4b81878740 10 client.17520 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.578+0000 7f4b81878740 1 -- 172.21.15.86:0/3785692725 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ec4b294620 con 0x55ec4b550b60 2024-04-21T08:43:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.578+0000 7f4b57fff640 1 -- 172.21.15.86:0/3785692725 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b6c01a910 con 0x55ec4b550b60 2024-04-21T08:43:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.579+0000 7f4b57fff640 1 -- 172.21.15.86:0/3785692725 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4b6c011970 con 0x55ec4b550b60 2024-04-21T08:43:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.579+0000 7f4b81878740 10 client.17520 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.579+0000 7f4b81878740 10 client.17520 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.579+0000 7f4b81878740 20 client.17520 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.579+0000 7f4b81878740 1 --2- 172.21.15.86:0/3785692725 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ec4b56e380 0x55ec4b570770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:03.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.579+0000 7f4b81878740 4 client.17520 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.579+0000 7f4b81878740 1 -- 172.21.15.86:0/3785692725 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec4b4c9880 con 0x55ec4b56e380 2024-04-21T08:43:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.579+0000 7f4b77fff640 1 --2- 172.21.15.86:0/3785692725 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ec4b56e380 0x55ec4b570770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.579+0000 7f4b57fff640 10 client.17520 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:03.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.579+0000 7f4b77fff640 1 --2- 172.21.15.86:0/3785692725 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ec4b56e380 0x55ec4b570770 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=67d5096e0769b68c server_cookie=fe3d1b34c299d33e in_seq=0 out_seq=0 2024-04-21T08:43:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.580+0000 7f4b57fff640 1 -- 172.21.15.86:0/3785692725 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ec4b4c9880 con 0x55ec4b56e380 2024-04-21T08:43:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.580+0000 7f4b57fff640 10 client.17520 handle_command_reply: tid=0 2024-04-21T08:43:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.580+0000 7f4b81878740 1 client.17520 shutdown 2024-04-21T08:43:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.580+0000 7f4b81878740 1 -- 172.21.15.86:0/3785692725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4b550b60 msgr2=0x55ec4b55de20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.580+0000 7f4b81878740 1 --2- 172.21.15.86:0/3785692725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4b550b60 0x55ec4b55de20 secure :-1 s=READY pgs=1116 cs=0 l=1 rev1=1 crypto rx=0x7f4b6c011480 tx=0x7f4b6c00b9f0 comp rx=0 tx=0).stop 2024-04-21T08:43:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.580+0000 7f4b81878740 1 -- 172.21.15.86:0/3785692725 shutdown_connections 2024-04-21T08:43:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.580+0000 7f4b81878740 1 --2- 172.21.15.86:0/3785692725 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ec4b56e380 0x55ec4b570770 crc :-1 s=READY pgs=151 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.580+0000 7f4b81878740 1 --2- 172.21.15.86:0/3785692725 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec4b55e4f0 0x55ec4b55b6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.580+0000 7f4b81878740 1 --2- 172.21.15.86:0/3785692725 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4b5518d0 0x55ec4b55b1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.580+0000 7f4b57fff640 0 client.17520 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:03.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.580+0000 7f4b81878740 1 --2- 172.21.15.86:0/3785692725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4b550b60 0x55ec4b55de20 unknown :-1 s=CLOSED pgs=1116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.580+0000 7f4b81878740 1 -- 172.21.15.86:0/3785692725 >> 172.21.15.86:0/3785692725 conn(0x55ec4b517c90 msgr2=0x55ec4b54d5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.580+0000 7f4b81878740 1 -- 172.21.15.86:0/3785692725 shutdown_connections 2024-04-21T08:43:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.581+0000 7f4b81878740 1 -- 172.21.15.86:0/3785692725 wait complete. 2024-04-21T08:43:03.586 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.581+0000 7f4b81878740 20 client.17520 trim_cache size 0 max 16384 2024-04-21T08:43:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.595+0000 7f4b81878740 1 Processor -- start 2024-04-21T08:43:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.595+0000 7f4b81878740 1 -- start start 2024-04-21T08:43:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.595+0000 7f4b81878740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec4b550b60 0x55ec4b56a150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:03.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.595+0000 7f4b81878740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4b5518d0 0x55ec4b55d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.595+0000 7f4b81878740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4b55e4f0 0x55ec4b55de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.595+0000 7f4b81878740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b781acfd0 con 0x55ec4b5518d0 2024-04-21T08:43:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.595+0000 7f4b81878740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4b781aeb10 con 0x55ec4b55e4f0 2024-04-21T08:43:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.595+0000 7f4b81878740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4b781ad6a0 con 0x55ec4b550b60 2024-04-21T08:43:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.596+0000 7f4b77fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4b5518d0 0x55ec4b55d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.596+0000 7f4b77fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4b5518d0 0x55ec4b55d8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33012/0 (socket says 172.21.15.86:33012) 2024-04-21T08:43:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.596+0000 7f4b77fff640 1 -- 172.21.15.86:0/3280193109 learned_addr learned my addr 172.21.15.86:0/3280193109 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.596+0000 7f4b7d21e640 1 --2- 172.21.15.86:0/3280193109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4b55e4f0 0x55ec4b55de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.596+0000 7f4b77fff640 1 -- 172.21.15.86:0/3280193109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec4b550b60 msgr2=0x55ec4b56a150 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:43:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.596+0000 7f4b77fff640 1 --2- 172.21.15.86:0/3280193109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec4b550b60 0x55ec4b56a150 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.596+0000 7f4b77fff640 1 -- 172.21.15.86:0/3280193109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4b55e4f0 msgr2=0x55ec4b55de20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.596+0000 7f4b77fff640 1 --2- 172.21.15.86:0/3280193109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4b55e4f0 0x55ec4b55de20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.596+0000 7f4b77fff640 1 -- 172.21.15.86:0/3280193109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec4b54fe30 con 0x55ec4b5518d0 2024-04-21T08:43:03.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.596+0000 7f4b77fff640 1 --2- 172.21.15.86:0/3280193109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4b5518d0 0x55ec4b55d8e0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f4b6402c1c0 tx=0x7f4b64005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85d3bfb08a4f0f56 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.596+0000 7f4b627fc640 1 -- 172.21.15.86:0/3280193109 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b64058070 con 0x55ec4b5518d0 2024-04-21T08:43:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.596+0000 7f4b627fc640 1 -- 172.21.15.86:0/3280193109 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b64004250 con 0x55ec4b5518d0 2024-04-21T08:43:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.596+0000 7f4b627fc640 1 -- 172.21.15.86:0/3280193109 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b640046c0 con 0x55ec4b5518d0 2024-04-21T08:43:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.597+0000 7f4b81878740 1 -- 172.21.15.86:0/3280193109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4b5518d0 msgr2=0x55ec4b55d8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.597+0000 7f4b81878740 1 --2- 172.21.15.86:0/3280193109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4b5518d0 0x55ec4b55d8e0 secure :-1 s=READY pgs=1117 cs=0 l=1 rev1=1 crypto rx=0x7f4b6402c1c0 tx=0x7f4b64005a20 comp rx=0 tx=0).stop 2024-04-21T08:43:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.597+0000 7f4b81878740 1 -- 172.21.15.86:0/3280193109 shutdown_connections 2024-04-21T08:43:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.597+0000 7f4b81878740 1 --2- 172.21.15.86:0/3280193109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4b55e4f0 0x55ec4b55de20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.597+0000 7f4b81878740 1 --2- 172.21.15.86:0/3280193109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4b5518d0 0x55ec4b55d8e0 unknown :-1 s=CLOSED pgs=1117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.597+0000 7f4b81878740 1 --2- 172.21.15.86:0/3280193109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec4b550b60 0x55ec4b56a150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.597+0000 7f4b81878740 1 -- 172.21.15.86:0/3280193109 >> 172.21.15.86:0/3280193109 conn(0x55ec4b517c90 msgr2=0x55ec4b559a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.597+0000 7f4b81878740 1 -- 172.21.15.86:0/3280193109 shutdown_connections 2024-04-21T08:43:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.597+0000 7f4b81878740 1 -- 172.21.15.86:0/3280193109 wait complete. 2024-04-21T08:43:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.597+0000 7f4b81878740 1 -- start start 2024-04-21T08:43:03.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.597+0000 7f4b81878740 1 Processor -- start 2024-04-21T08:43:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.598+0000 7f4b81878740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4b550b60 0x55ec4b5666c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.598+0000 7f4b81878740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4b5518d0 0x55ec4b561a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.598+0000 7f4b81878740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec4b55e4f0 0x55ec4b561f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.598+0000 7f4b81878740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b781ae440 con 0x55ec4b5518d0 2024-04-21T08:43:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.598+0000 7f4b81878740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4b781add70 con 0x55ec4b550b60 2024-04-21T08:43:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.598+0000 7f4b81878740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4b781aaf90 con 0x55ec4b55e4f0 2024-04-21T08:43:03.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.598+0000 7f4b81878740 10 client.0 _ll_register_callbacks cb 0x7f4b7000cf50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.598+0000 7f4b77fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4b5518d0 0x55ec4b561a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.598+0000 7f4b77fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4b5518d0 0x55ec4b561a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33016/0 (socket says 172.21.15.86:33016) 2024-04-21T08:43:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.598+0000 7f4b77fff640 1 -- 172.21.15.86:0/1023624082 learned_addr learned my addr 172.21.15.86:0/1023624082 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.598+0000 7f4b7ca1d640 1 --2- 172.21.15.86:0/1023624082 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4b550b60 0x55ec4b5666c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.598+0000 7f4b77fff640 1 -- 172.21.15.86:0/1023624082 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec4b55e4f0 msgr2=0x55ec4b561f70 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:43:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.598+0000 7f4b77fff640 1 --2- 172.21.15.86:0/1023624082 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec4b55e4f0 0x55ec4b561f70 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.598+0000 7f4b77fff640 1 -- 172.21.15.86:0/1023624082 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4b550b60 msgr2=0x55ec4b5666c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.598+0000 7f4b77fff640 1 --2- 172.21.15.86:0/1023624082 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4b550b60 0x55ec4b5666c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.598+0000 7f4b77fff640 1 -- 172.21.15.86:0/1023624082 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec4b5624b0 con 0x55ec4b5518d0 2024-04-21T08:43:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.598+0000 7f4b77fff640 1 --2- 172.21.15.86:0/1023624082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4b5518d0 0x55ec4b561a30 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f4b64056040 tx=0x7f4b640527c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa3289350ed48417 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.598+0000 7f4b57fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.598+0000 7f4b57fff640 1 -- 172.21.15.86:0/1023624082 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b64058070 con 0x55ec4b5518d0 2024-04-21T08:43:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.598+0000 7f4b57fff640 1 -- 172.21.15.86:0/1023624082 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f4b64005240 con 0x55ec4b5518d0 2024-04-21T08:43:03.602 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.598+0000 7f4b81878740 1 -- 172.21.15.86:0/1023624082 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b64004250 con 0x55ec4b5518d0 2024-04-21T08:43:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.598+0000 7f4b57fff640 1 -- 172.21.15.86:0/1023624082 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4b64005440 con 0x55ec4b5518d0 2024-04-21T08:43:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.599+0000 7f4b81878740 10 client.17526 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.599+0000 7f4b81878740 1 -- 172.21.15.86:0/1023624082 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ec4b5d4270 con 0x55ec4b5518d0 2024-04-21T08:43:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.599+0000 7f4b57fff640 1 -- 172.21.15.86:0/1023624082 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b64004250 con 0x55ec4b5518d0 2024-04-21T08:43:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.599+0000 7f4b57fff640 1 -- 172.21.15.86:0/1023624082 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4b640036f0 con 0x55ec4b5518d0 2024-04-21T08:43:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.599+0000 7f4b81878740 10 client.17526 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.599+0000 7f4b81878740 10 client.17526 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.599+0000 7f4b81878740 20 client.17526 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.599+0000 7f4b81878740 1 --2- 172.21.15.86:0/1023624082 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ec4b5d49d0 0x55ec4b5d6df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.599+0000 7f4b81878740 4 client.17526 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:43:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.599+0000 7f4b81878740 1 -- 172.21.15.86:0/1023624082 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ec4b4553d0 con 0x55ec4b5d49d0 2024-04-21T08:43:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.599+0000 7f4b7ca1d640 1 --2- 172.21.15.86:0/1023624082 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ec4b5d49d0 0x55ec4b5d6df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.599+0000 7f4b57fff640 10 client.17526 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:03.603 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.599+0000 7f4b7ca1d640 1 --2- 172.21.15.86:0/1023624082 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ec4b5d49d0 0x55ec4b5d6df0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=42803857d5e65a9a server_cookie=11245b8e8f5cb5ed in_seq=0 out_seq=0 2024-04-21T08:43:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.616+0000 7f4b57fff640 1 -- 172.21.15.86:0/1023624082 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ec4b4553d0 con 0x55ec4b5d49d0 2024-04-21T08:43:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.616+0000 7f4b57fff640 10 client.17526 handle_command_reply: tid=0 2024-04-21T08:43:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.616+0000 7f4b81878740 1 client.17526 shutdown 2024-04-21T08:43:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.616+0000 7f4b81878740 1 -- 172.21.15.86:0/1023624082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4b5518d0 msgr2=0x55ec4b561a30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.616+0000 7f4b81878740 1 --2- 172.21.15.86:0/1023624082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4b5518d0 0x55ec4b561a30 secure :-1 s=READY pgs=1118 cs=0 l=1 rev1=1 crypto rx=0x7f4b64056040 tx=0x7f4b640527c0 comp rx=0 tx=0).stop 2024-04-21T08:43:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.616+0000 7f4b81878740 1 -- 172.21.15.86:0/1023624082 shutdown_connections 2024-04-21T08:43:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.616+0000 7f4b81878740 1 --2- 172.21.15.86:0/1023624082 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ec4b5d49d0 0x55ec4b5d6df0 crc :-1 s=READY pgs=152 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.616+0000 7f4b81878740 1 --2- 172.21.15.86:0/1023624082 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec4b55e4f0 0x55ec4b561f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.616+0000 7f4b81878740 1 --2- 172.21.15.86:0/1023624082 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4b5518d0 0x55ec4b561a30 unknown :-1 s=CLOSED pgs=1118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.616+0000 7f4b57fff640 0 client.17526 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:03.616 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.616+0000 7f4b81878740 1 --2- 172.21.15.86:0/1023624082 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4b550b60 0x55ec4b5666c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.616+0000 7f4b81878740 1 -- 172.21.15.86:0/1023624082 >> 172.21.15.86:0/1023624082 conn(0x55ec4b517c90 msgr2=0x55ec4b54cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.616+0000 7f4b81878740 1 -- 172.21.15.86:0/1023624082 shutdown_connections 2024-04-21T08:43:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.616+0000 7f4b81878740 1 -- 172.21.15.86:0/1023624082 wait complete. 2024-04-21T08:43:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.616+0000 7f4b81878740 20 client.17526 trim_cache size 0 max 16384 2024-04-21T08:43:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:43:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:43:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:43:03.617 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:43:03.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.617+0000 7f4b7ec81640 1 -- 172.21.15.86:0/361411168 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4b680a3dc0 msgr2=0x7f4b680a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:03.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.617+0000 7f4b7ec81640 1 --2- 172.21.15.86:0/361411168 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4b680a3dc0 0x7f4b680a6280 secure :-1 s=READY pgs=172 cs=0 l=1 rev1=1 crypto rx=0x7f4b78105ca0 tx=0x7f4b64005d50 comp rx=0 tx=0).stop 2024-04-21T08:43:03.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.617+0000 7f4b7ec81640 1 -- 172.21.15.86:0/361411168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b78106fb0 msgr2=0x7f4b7819ced0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:03.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.617+0000 7f4b7ec81640 1 --2- 172.21.15.86:0/361411168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b78106fb0 0x7f4b7819ced0 secure :-1 s=READY pgs=1114 cs=0 l=1 rev1=1 crypto rx=0x7f4b7000eaa0 tx=0x7f4b7000ef60 comp rx=0 tx=0).stop 2024-04-21T08:43:03.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.617+0000 7f4b7ec81640 1 -- 172.21.15.86:0/361411168 shutdown_connections 2024-04-21T08:43:03.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.617+0000 7f4b7ec81640 1 --2- 172.21.15.86:0/361411168 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4b680a3dc0 0x7f4b680a6280 unknown :-1 s=CLOSED pgs=172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:03.618 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.617+0000 7f4b7ec81640 1 --2- 172.21.15.86:0/361411168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b78106fb0 0x7f4b7819ced0 unknown :-1 s=CLOSED pgs=1114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:03.619 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.617+0000 7f4b7ec81640 1 --2- 172.21.15.86:0/361411168 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b78101b60 0x7f4b78198b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:03.619 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.617+0000 7f4b7ec81640 1 --2- 172.21.15.86:0/361411168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b781011a0 0x7f4b78198610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:03.619 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.617+0000 7f4b7ec81640 1 -- 172.21.15.86:0/361411168 >> 172.21.15.86:0/361411168 conn(0x7f4b78076c50 msgr2=0x7f4b78100450 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:03.619 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.617+0000 7f4b7ec81640 1 -- 172.21.15.86:0/361411168 shutdown_connections 2024-04-21T08:43:03.619 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:03.617+0000 7f4b7ec81640 1 -- 172.21.15.86:0/361411168 wait complete. 2024-04-21T08:43:04.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:04 smithi179 ceph-mon[31829]: pgmap v131: 129 pgs: 129 active+clean; 5.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 66 KiB/s wr, 36 op/s 2024-04-21T08:43:04.766 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:04 smithi088 ceph-mon[31259]: pgmap v131: 129 pgs: 129 active+clean; 5.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 66 KiB/s wr, 36 op/s 2024-04-21T08:43:04.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:04 smithi086 ceph-mon[27939]: pgmap v131: 129 pgs: 129 active+clean; 5.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 66 KiB/s wr, 36 op/s 2024-04-21T08:43:06.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:06 smithi179 ceph-mon[31829]: pgmap v132: 129 pgs: 129 active+clean; 5.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 49 KiB/s wr, 27 op/s 2024-04-21T08:43:06.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:06 smithi086 ceph-mon[27939]: pgmap v132: 129 pgs: 129 active+clean; 5.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 49 KiB/s wr, 27 op/s 2024-04-21T08:43:06.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:06 smithi088 ceph-mon[31259]: pgmap v132: 129 pgs: 129 active+clean; 5.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 49 KiB/s wr, 27 op/s 2024-04-21T08:43:08.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:08 smithi179 ceph-mon[31829]: pgmap v133: 129 pgs: 129 active+clean; 5.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 49 KiB/s wr, 27 op/s 2024-04-21T08:43:08.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:08 smithi086 ceph-mon[27939]: pgmap v133: 129 pgs: 129 active+clean; 5.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 49 KiB/s wr, 27 op/s 2024-04-21T08:43:08.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:08 smithi088 ceph-mon[31259]: pgmap v133: 129 pgs: 129 active+clean; 5.1 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 49 KiB/s wr, 27 op/s 2024-04-21T08:43:10.461 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:43:10.461 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:43:10.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.575+0000 7f166c481640 1 -- 172.21.15.86:0/2387736202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f166406af70 msgr2=0x7f166410ede0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:10.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.575+0000 7f166c481640 1 --2- 172.21.15.86:0/2387736202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f166406af70 0x7f166410ede0 secure :-1 s=READY pgs=1119 cs=0 l=1 rev1=1 crypto rx=0x7f165c009930 tx=0x7f165c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:43:10.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.575+0000 7f166c481640 1 -- 172.21.15.86:0/2387736202 shutdown_connections 2024-04-21T08:43:10.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.575+0000 7f166c481640 1 --2- 172.21.15.86:0/2387736202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f166406af70 0x7f166410ede0 unknown :-1 s=CLOSED pgs=1119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.575+0000 7f166c481640 1 --2- 172.21.15.86:0/2387736202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f166406a5b0 0x7f166406aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.575+0000 7f166c481640 1 --2- 172.21.15.86:0/2387736202 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1664108060 0x7f1664108440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.575+0000 7f166c481640 1 -- 172.21.15.86:0/2387736202 >> 172.21.15.86:0/2387736202 conn(0x7f1664077c40 msgr2=0x7f1664078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:10.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.575+0000 7f166c481640 1 -- 172.21.15.86:0/2387736202 shutdown_connections 2024-04-21T08:43:10.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.575+0000 7f166c481640 1 -- 172.21.15.86:0/2387736202 wait complete. 2024-04-21T08:43:10.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.576+0000 7f166c481640 1 Processor -- start 2024-04-21T08:43:10.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.576+0000 7f166c481640 1 -- start start 2024-04-21T08:43:10.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.576+0000 7f166c481640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f166406a5b0 0x7f1664101170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:10.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.576+0000 7f166c481640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f166406af70 0x7f16641016b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:10.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.576+0000 7f166c481640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1664108060 0x7f1664101bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:10.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.576+0000 7f166c481640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1664112180 con 0x7f1664108060 2024-04-21T08:43:10.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.576+0000 7f166c481640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1664112000 con 0x7f166406af70 2024-04-21T08:43:10.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.576+0000 7f166c481640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1664112300 con 0x7f166406a5b0 2024-04-21T08:43:10.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.576+0000 7f166aa1e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1664108060 0x7f1664101bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:10.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.576+0000 7f166aa1e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1664108060 0x7f1664101bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50890/0 (socket says 172.21.15.86:50890) 2024-04-21T08:43:10.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.576+0000 7f166aa1e640 1 -- 172.21.15.86:0/1969879027 learned_addr learned my addr 172.21.15.86:0/1969879027 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:10.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.576+0000 7f166a21d640 1 --2- 172.21.15.86:0/1969879027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f166406a5b0 0x7f1664101170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:10.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.576+0000 7f1669a1c640 1 --2- 172.21.15.86:0/1969879027 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f166406af70 0x7f16641016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:10.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.576+0000 7f166aa1e640 1 -- 172.21.15.86:0/1969879027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f166406a5b0 msgr2=0x7f1664101170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:10.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.576+0000 7f166aa1e640 1 --2- 172.21.15.86:0/1969879027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f166406a5b0 0x7f1664101170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.576+0000 7f166aa1e640 1 -- 172.21.15.86:0/1969879027 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f166406af70 msgr2=0x7f16641016b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:10.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.576+0000 7f166aa1e640 1 --2- 172.21.15.86:0/1969879027 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f166406af70 0x7f16641016b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.576+0000 7f166aa1e640 1 -- 172.21.15.86:0/1969879027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f16641a6a40 con 0x7f1664108060 2024-04-21T08:43:10.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.576+0000 7f1669a1c640 1 --2- 172.21.15.86:0/1969879027 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f166406af70 0x7f16641016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:10.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.577+0000 7f166aa1e640 1 --2- 172.21.15.86:0/1969879027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1664108060 0x7f1664101bf0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f165c005940 tx=0x7f165c0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:10.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.577+0000 7f165b7fe640 1 -- 172.21.15.86:0/1969879027 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165c043020 con 0x7f1664108060 2024-04-21T08:43:10.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.577+0000 7f165b7fe640 1 -- 172.21.15.86:0/1969879027 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f165c02ca70 con 0x7f1664108060 2024-04-21T08:43:10.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.577+0000 7f166c481640 1 -- 172.21.15.86:0/1969879027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f16641a6cd0 con 0x7f1664108060 2024-04-21T08:43:10.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.577+0000 7f165b7fe640 1 -- 172.21.15.86:0/1969879027 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165c003ab0 con 0x7f1664108060 2024-04-21T08:43:10.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.577+0000 7f166c481640 1 -- 172.21.15.86:0/1969879027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f16641a7220 con 0x7f1664108060 2024-04-21T08:43:10.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.579+0000 7f165b7fe640 1 -- 172.21.15.86:0/1969879027 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f165c046020 con 0x7f1664108060 2024-04-21T08:43:10.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.579+0000 7f166f078740 1 Processor -- start 2024-04-21T08:43:10.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.580+0000 7f166f078740 1 -- start start 2024-04-21T08:43:10.586 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.580+0000 7f166f078740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586b9759580 0x5586b9759960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:10.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.580+0000 7f166f078740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586b9759ea0 0x5586b9762540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:10.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.580+0000 7f165b7fe640 1 --2- 172.21.15.86:0/1969879027 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f16500a3d20 0x7f16500a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:10.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.580+0000 7f166f078740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586b9762a80 0x5586b9764e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:10.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.580+0000 7f166f078740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5586b96be1e0 con 0x5586b9759580 2024-04-21T08:43:10.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.580+0000 7f166f078740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5586b965b390 con 0x5586b9759ea0 2024-04-21T08:43:10.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.580+0000 7f166f078740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5586b96ba1a0 con 0x5586b9762a80 2024-04-21T08:43:10.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.580+0000 7f1669a1c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586b9759580 0x5586b9759960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:10.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.580+0000 7f1669a1c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586b9759580 0x5586b9759960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50902/0 (socket says 172.21.15.86:50902) 2024-04-21T08:43:10.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.580+0000 7f1669a1c640 1 -- 172.21.15.86:0/4142156036 learned_addr learned my addr 172.21.15.86:0/4142156036 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:10.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.580+0000 7f165b7fe640 1 -- 172.21.15.86:0/1969879027 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f165c0f2cb0 con 0x7f1664108060 2024-04-21T08:43:10.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.580+0000 7f1669a1c640 1 --2- 172.21.15.86:0/1969879027 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f16500a3d20 0x7f16500a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:10.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.580+0000 7f166a21d640 1 --2- 172.21.15.86:0/4142156036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586b9759ea0 0x5586b9762540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:10.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.580+0000 7f166aa1e640 1 --2- 172.21.15.86:0/4142156036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586b9762a80 0x5586b9764e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:10.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.580+0000 7f1669a1c640 1 -- 172.21.15.86:0/4142156036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586b9762a80 msgr2=0x5586b9764e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:10.587 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.580+0000 7f1669a1c640 1 --2- 172.21.15.86:0/4142156036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586b9762a80 0x5586b9764e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.580+0000 7f1669a1c640 1 -- 172.21.15.86:0/4142156036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586b9759ea0 msgr2=0x5586b9762540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:10.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.580+0000 7f1669a1c640 1 --2- 172.21.15.86:0/4142156036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586b9759ea0 0x5586b9762540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.580+0000 7f1669a1c640 1 -- 172.21.15.86:0/4142156036 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586b949a7d0 con 0x5586b9759580 2024-04-21T08:43:10.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.580+0000 7f166aa1e640 1 --2- 172.21.15.86:0/4142156036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586b9762a80 0x5586b9764e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:10.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.581+0000 7f1669a1c640 1 --2- 172.21.15.86:0/4142156036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586b9759580 0x5586b9759960 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f1654014c10 tx=0x7f16540117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=713bbbadcc5006b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:10.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.581+0000 7f1669a1c640 1 --2- 172.21.15.86:0/1969879027 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f16500a3d20 0x7f16500a61e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1654011cb0 tx=0x7f165401c000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:10.588 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.581+0000 7f1658bfe640 1 -- 172.21.15.86:0/4142156036 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1654014590 con 0x5586b9759580 2024-04-21T08:43:10.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.581+0000 7f1658bfe640 1 -- 172.21.15.86:0/4142156036 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1654009a00 con 0x5586b9759580 2024-04-21T08:43:10.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.581+0000 7f1658bfe640 1 -- 172.21.15.86:0/4142156036 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1654003970 con 0x5586b9759580 2024-04-21T08:43:10.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.582+0000 7f166f078740 1 -- 172.21.15.86:0/4142156036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586b9759580 msgr2=0x5586b9759960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:10.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.582+0000 7f166f078740 1 --2- 172.21.15.86:0/4142156036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586b9759580 0x5586b9759960 secure :-1 s=READY pgs=1121 cs=0 l=1 rev1=1 crypto rx=0x7f1654014c10 tx=0x7f16540117b0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.582+0000 7f166f078740 1 -- 172.21.15.86:0/4142156036 shutdown_connections 2024-04-21T08:43:10.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.582+0000 7f166f078740 1 --2- 172.21.15.86:0/4142156036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586b9762a80 0x5586b9764e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.582+0000 7f166f078740 1 --2- 172.21.15.86:0/4142156036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586b9759ea0 0x5586b9762540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.582+0000 7f166f078740 1 --2- 172.21.15.86:0/4142156036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586b9759580 0x5586b9759960 unknown :-1 s=CLOSED pgs=1121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.582+0000 7f166f078740 1 -- 172.21.15.86:0/4142156036 >> 172.21.15.86:0/4142156036 conn(0x5586b9753cb0 msgr2=0x5586b9754090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:10.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.582+0000 7f166f078740 1 -- 172.21.15.86:0/4142156036 shutdown_connections 2024-04-21T08:43:10.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.582+0000 7f166f078740 1 -- 172.21.15.86:0/4142156036 wait complete. 2024-04-21T08:43:10.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.582+0000 7f166f078740 1 -- start start 2024-04-21T08:43:10.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.582+0000 7f166f078740 1 Processor -- start 2024-04-21T08:43:10.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.582+0000 7f166f078740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586b9759580 0x5586b975fb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:10.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.582+0000 7f166f078740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586b9759ea0 0x5586b9760060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:10.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.582+0000 7f166f078740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586b9762a80 0x5586b97605a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:10.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.582+0000 7f166f078740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5586b9765810 con 0x5586b9762a80 2024-04-21T08:43:10.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.582+0000 7f166f078740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5586b96772f0 con 0x5586b9759580 2024-04-21T08:43:10.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.582+0000 7f166f078740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5586b9765990 con 0x5586b9759ea0 2024-04-21T08:43:10.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.582+0000 7f166f078740 10 client.0 _ll_register_callbacks cb 0x5586b9684580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:10.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.582+0000 7f166aa1e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586b9762a80 0x5586b97605a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:10.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.582+0000 7f166aa1e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586b9762a80 0x5586b97605a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50912/0 (socket says 172.21.15.86:50912) 2024-04-21T08:43:10.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.582+0000 7f166aa1e640 1 -- 172.21.15.86:0/2940811673 learned_addr learned my addr 172.21.15.86:0/2940811673 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:10.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.583+0000 7f166a21d640 1 --2- 172.21.15.86:0/2940811673 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586b9759580 0x5586b975fb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:10.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.583+0000 7f166aa1e640 1 -- 172.21.15.86:0/2940811673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586b9759ea0 msgr2=0x5586b9760060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:10.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.583+0000 7f1669a1c640 1 --2- 172.21.15.86:0/2940811673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586b9759ea0 0x5586b9760060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:10.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.583+0000 7f166aa1e640 1 --2- 172.21.15.86:0/2940811673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586b9759ea0 0x5586b9760060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.583+0000 7f166aa1e640 1 -- 172.21.15.86:0/2940811673 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586b9759580 msgr2=0x5586b975fb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:10.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.583+0000 7f166aa1e640 1 --2- 172.21.15.86:0/2940811673 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586b9759580 0x5586b975fb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.583+0000 7f166aa1e640 1 -- 172.21.15.86:0/2940811673 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586b93d03f0 con 0x5586b9762a80 2024-04-21T08:43:10.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.583+0000 7f1669a1c640 1 --2- 172.21.15.86:0/2940811673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586b9759ea0 0x5586b9760060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:10.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.583+0000 7f166aa1e640 1 --2- 172.21.15.86:0/2940811673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586b9762a80 0x5586b97605a0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f165c0beaa0 tx=0x7f165c0d1f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=638d3fae0e6c20a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:10.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.583+0000 7f164dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:10.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.583+0000 7f164dffb640 1 -- 172.21.15.86:0/2940811673 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f165c0d3040 con 0x5586b9762a80 2024-04-21T08:43:10.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.583+0000 7f164dffb640 1 -- 172.21.15.86:0/2940811673 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f165c005340 con 0x5586b9762a80 2024-04-21T08:43:10.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.583+0000 7f166f078740 1 -- 172.21.15.86:0/2940811673 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5586b96cbc00 con 0x5586b9762a80 2024-04-21T08:43:10.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.583+0000 7f164dffb640 1 -- 172.21.15.86:0/2940811673 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f165c02cc20 con 0x5586b9762a80 2024-04-21T08:43:10.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.583+0000 7f166f078740 10 client.17538 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:10.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.583+0000 7f166f078740 1 -- 172.21.15.86:0/2940811673 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5586b93900f0 con 0x5586b9762a80 2024-04-21T08:43:10.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.583+0000 7f164dffb640 1 -- 172.21.15.86:0/2940811673 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5586b96cbc00 con 0x5586b9762a80 2024-04-21T08:43:10.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.583+0000 7f164dffb640 1 -- 172.21.15.86:0/2940811673 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f165c003c50 con 0x5586b9762a80 2024-04-21T08:43:10.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.584+0000 7f166f078740 10 client.17538 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:10.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.584+0000 7f166f078740 10 client.17538 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:10.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.584+0000 7f166f078740 20 client.17538 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:10.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.584+0000 7f166f078740 1 --2- 172.21.15.86:0/2940811673 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5586b97729c0 0x5586b9774db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:10.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.584+0000 7f166f078740 4 client.17538 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:10.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.584+0000 7f166f078740 1 -- 172.21.15.86:0/2940811673 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5586b96c5a70 con 0x5586b97729c0 2024-04-21T08:43:10.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.584+0000 7f166a21d640 1 --2- 172.21.15.86:0/2940811673 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5586b97729c0 0x5586b9774db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:10.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.584+0000 7f164dffb640 10 client.17538 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:10.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.584+0000 7f166a21d640 1 --2- 172.21.15.86:0/2940811673 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5586b97729c0 0x5586b9774db0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=513315a19cd73f12 server_cookie=95828dfe8668abf8 in_seq=0 out_seq=0 2024-04-21T08:43:10.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.585+0000 7f164dffb640 1 -- 172.21.15.86:0/2940811673 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5586b96c5a70 con 0x5586b97729c0 2024-04-21T08:43:10.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.585+0000 7f164dffb640 10 client.17538 handle_command_reply: tid=0 2024-04-21T08:43:10.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.585+0000 7f166f078740 1 client.17538 shutdown 2024-04-21T08:43:10.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.585+0000 7f166f078740 1 -- 172.21.15.86:0/2940811673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586b9762a80 msgr2=0x5586b97605a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:10.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.585+0000 7f166f078740 1 --2- 172.21.15.86:0/2940811673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586b9762a80 0x5586b97605a0 secure :-1 s=READY pgs=1122 cs=0 l=1 rev1=1 crypto rx=0x7f165c0beaa0 tx=0x7f165c0d1f90 comp rx=0 tx=0).stop 2024-04-21T08:43:10.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.585+0000 7f166f078740 1 -- 172.21.15.86:0/2940811673 shutdown_connections 2024-04-21T08:43:10.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.585+0000 7f166f078740 1 --2- 172.21.15.86:0/2940811673 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5586b97729c0 0x5586b9774db0 crc :-1 s=READY pgs=153 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.585+0000 7f166f078740 1 --2- 172.21.15.86:0/2940811673 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586b9762a80 0x5586b97605a0 unknown :-1 s=CLOSED pgs=1122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.585+0000 7f164dffb640 0 client.17538 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:10.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.585+0000 7f166f078740 1 --2- 172.21.15.86:0/2940811673 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586b9759ea0 0x5586b9760060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.585+0000 7f166f078740 1 --2- 172.21.15.86:0/2940811673 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586b9759580 0x5586b975fb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.585+0000 7f166f078740 1 -- 172.21.15.86:0/2940811673 >> 172.21.15.86:0/2940811673 conn(0x5586b9753cb0 msgr2=0x5586b9755a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:10.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.585+0000 7f166f078740 1 -- 172.21.15.86:0/2940811673 shutdown_connections 2024-04-21T08:43:10.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.585+0000 7f166f078740 1 -- 172.21.15.86:0/2940811673 wait complete. 2024-04-21T08:43:10.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.585+0000 7f166f078740 20 client.17538 trim_cache size 0 max 16384 2024-04-21T08:43:10.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.603+0000 7f166f078740 1 Processor -- start 2024-04-21T08:43:10.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.603+0000 7f166f078740 1 -- start start 2024-04-21T08:43:10.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.603+0000 7f166f078740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586b9759580 0x5586b976e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:10.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.603+0000 7f166f078740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586b9759ea0 0x5586b975ea60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:10.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.603+0000 7f166f078740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586b9762a80 0x5586b97603c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:10.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.603+0000 7f166f078740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f16641b1350 con 0x5586b9759580 2024-04-21T08:43:10.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.603+0000 7f166f078740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f16641b2e90 con 0x5586b9759ea0 2024-04-21T08:43:10.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.603+0000 7f166f078740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f16641b1a20 con 0x5586b9762a80 2024-04-21T08:43:10.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.603+0000 7f166a21d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586b9759580 0x5586b976e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:10.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.603+0000 7f166a21d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586b9759580 0x5586b976e6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50918/0 (socket says 172.21.15.86:50918) 2024-04-21T08:43:10.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.603+0000 7f166a21d640 1 -- 172.21.15.86:0/2439598289 learned_addr learned my addr 172.21.15.86:0/2439598289 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:10.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.603+0000 7f1669a1c640 1 --2- 172.21.15.86:0/2439598289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586b9759ea0 0x5586b975ea60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:10.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.603+0000 7f166a21d640 1 -- 172.21.15.86:0/2439598289 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586b9762a80 msgr2=0x5586b97603c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:10.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.603+0000 7f166a21d640 1 --2- 172.21.15.86:0/2439598289 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586b9762a80 0x5586b97603c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.603+0000 7f166a21d640 1 -- 172.21.15.86:0/2439598289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586b9759ea0 msgr2=0x5586b975ea60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:10.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.603+0000 7f166a21d640 1 --2- 172.21.15.86:0/2439598289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586b9759ea0 0x5586b975ea60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.603+0000 7f166a21d640 1 -- 172.21.15.86:0/2439598289 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5586b9754880 con 0x5586b9759580 2024-04-21T08:43:10.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.603+0000 7f166a21d640 1 --2- 172.21.15.86:0/2439598289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586b9759580 0x5586b976e6d0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f1660004a80 tx=0x7f16600075b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9bd355f9b01833a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:10.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.603+0000 7f164ffff640 1 -- 172.21.15.86:0/2439598289 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166000fae0 con 0x5586b9759580 2024-04-21T08:43:10.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.603+0000 7f164ffff640 1 -- 172.21.15.86:0/2439598289 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1660023020 con 0x5586b9759580 2024-04-21T08:43:10.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.603+0000 7f164ffff640 1 -- 172.21.15.86:0/2439598289 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166001c650 con 0x5586b9759580 2024-04-21T08:43:10.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.604+0000 7f166f078740 1 -- 172.21.15.86:0/2439598289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586b9759580 msgr2=0x5586b976e6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:10.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.604+0000 7f166f078740 1 --2- 172.21.15.86:0/2439598289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586b9759580 0x5586b976e6d0 secure :-1 s=READY pgs=1123 cs=0 l=1 rev1=1 crypto rx=0x7f1660004a80 tx=0x7f16600075b0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.604+0000 7f166f078740 1 -- 172.21.15.86:0/2439598289 shutdown_connections 2024-04-21T08:43:10.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.604+0000 7f166f078740 1 --2- 172.21.15.86:0/2439598289 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586b9762a80 0x5586b97603c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.604+0000 7f166f078740 1 --2- 172.21.15.86:0/2439598289 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586b9759ea0 0x5586b975ea60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.604+0000 7f166f078740 1 --2- 172.21.15.86:0/2439598289 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586b9759580 0x5586b976e6d0 unknown :-1 s=CLOSED pgs=1123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.604+0000 7f166f078740 1 -- 172.21.15.86:0/2439598289 >> 172.21.15.86:0/2439598289 conn(0x5586b9753cb0 msgr2=0x5586b9752d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:10.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.604+0000 7f166f078740 1 -- 172.21.15.86:0/2439598289 shutdown_connections 2024-04-21T08:43:10.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.604+0000 7f166f078740 1 -- 172.21.15.86:0/2439598289 wait complete. 2024-04-21T08:43:10.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.604+0000 7f166f078740 1 -- start start 2024-04-21T08:43:10.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.604+0000 7f166f078740 1 Processor -- start 2024-04-21T08:43:10.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.605+0000 7f166f078740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586b9759580 0x5586b976a8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:10.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.605+0000 7f166f078740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586b9759ea0 0x5586b976ade0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:10.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.605+0000 7f166f078740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586b9762a80 0x5586b9766d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:10.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.605+0000 7f166f078740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f16641b27c0 con 0x5586b9759580 2024-04-21T08:43:10.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.605+0000 7f166f078740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f16641b20f0 con 0x5586b9762a80 2024-04-21T08:43:10.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.605+0000 7f166f078740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f16641af310 con 0x5586b9759ea0 2024-04-21T08:43:10.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.605+0000 7f166f078740 10 client.0 _ll_register_callbacks cb 0x7f165c0056e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:10.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.605+0000 7f166a21d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586b9759580 0x5586b976a8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:10.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.605+0000 7f166a21d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586b9759580 0x5586b976a8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50922/0 (socket says 172.21.15.86:50922) 2024-04-21T08:43:10.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.605+0000 7f166a21d640 1 -- 172.21.15.86:0/1878271330 learned_addr learned my addr 172.21.15.86:0/1878271330 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:10.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.605+0000 7f1669a1c640 1 --2- 172.21.15.86:0/1878271330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586b9759ea0 0x5586b976ade0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:10.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.605+0000 7f166aa1e640 1 --2- 172.21.15.86:0/1878271330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586b9762a80 0x5586b9766d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:10.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.605+0000 7f166a21d640 1 -- 172.21.15.86:0/1878271330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586b9759ea0 msgr2=0x5586b976ade0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:10.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.605+0000 7f166a21d640 1 --2- 172.21.15.86:0/1878271330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586b9759ea0 0x5586b976ade0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.605+0000 7f166a21d640 1 -- 172.21.15.86:0/1878271330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586b9762a80 msgr2=0x5586b9766d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:10.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.605+0000 7f166a21d640 1 --2- 172.21.15.86:0/1878271330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586b9762a80 0x5586b9766d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.605+0000 7f166a21d640 1 -- 172.21.15.86:0/1878271330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5586b9767540 con 0x5586b9759580 2024-04-21T08:43:10.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.605+0000 7f1669a1c640 1 --2- 172.21.15.86:0/1878271330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586b9759ea0 0x5586b976ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:10.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.605+0000 7f166a21d640 1 --2- 172.21.15.86:0/1878271330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586b9759580 0x5586b976a8a0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f16600104a0 tx=0x7f166000fe70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5464a01863042dbf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:10.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.605+0000 7f164dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:10.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.605+0000 7f164dffb640 1 -- 172.21.15.86:0/1878271330 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f166002c070 con 0x5586b9759580 2024-04-21T08:43:10.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.605+0000 7f164dffb640 1 -- 172.21.15.86:0/1878271330 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f1660004e00 con 0x5586b9759580 2024-04-21T08:43:10.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.605+0000 7f166f078740 1 -- 172.21.15.86:0/1878271330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1660023020 con 0x5586b9759580 2024-04-21T08:43:10.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.605+0000 7f164dffb640 1 -- 172.21.15.86:0/1878271330 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1660038050 con 0x5586b9759580 2024-04-21T08:43:10.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.605+0000 7f166f078740 10 client.17544 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:10.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.605+0000 7f166f078740 1 -- 172.21.15.86:0/1878271330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5586b97d9970 con 0x5586b9759580 2024-04-21T08:43:10.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.606+0000 7f164dffb640 1 -- 172.21.15.86:0/1878271330 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1660023020 con 0x5586b9759580 2024-04-21T08:43:10.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.606+0000 7f164dffb640 1 -- 172.21.15.86:0/1878271330 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f16600055c0 con 0x5586b9759580 2024-04-21T08:43:10.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.606+0000 7f166f078740 10 client.17544 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:10.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.606+0000 7f166f078740 10 client.17544 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:10.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.606+0000 7f166f078740 20 client.17544 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:10.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.606+0000 7f166f078740 1 --2- 172.21.15.86:0/1878271330 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5586b97da000 0x5586b97dc420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:10.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.606+0000 7f166f078740 4 client.17544 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:43:10.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.606+0000 7f166f078740 1 -- 172.21.15.86:0/1878271330 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5586b9641590 con 0x5586b97da000 2024-04-21T08:43:10.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.606+0000 7f1669a1c640 1 --2- 172.21.15.86:0/1878271330 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5586b97da000 0x5586b97dc420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:10.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.606+0000 7f1669a1c640 1 --2- 172.21.15.86:0/1878271330 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5586b97da000 0x5586b97dc420 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e7777ab50c7b7c1 server_cookie=5d8f8c838068976 in_seq=0 out_seq=0 2024-04-21T08:43:10.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.606+0000 7f164dffb640 10 client.17544 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:10.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.607+0000 7f164dffb640 1 -- 172.21.15.86:0/1878271330 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5586b9641590 con 0x5586b97da000 2024-04-21T08:43:10.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.607+0000 7f164dffb640 10 client.17544 handle_command_reply: tid=0 2024-04-21T08:43:10.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.607+0000 7f166f078740 1 client.17544 shutdown 2024-04-21T08:43:10.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.607+0000 7f166f078740 1 -- 172.21.15.86:0/1878271330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586b9759580 msgr2=0x5586b976a8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:10.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.607+0000 7f166f078740 1 --2- 172.21.15.86:0/1878271330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586b9759580 0x5586b976a8a0 secure :-1 s=READY pgs=1124 cs=0 l=1 rev1=1 crypto rx=0x7f16600104a0 tx=0x7f166000fe70 comp rx=0 tx=0).stop 2024-04-21T08:43:10.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.607+0000 7f166f078740 1 -- 172.21.15.86:0/1878271330 shutdown_connections 2024-04-21T08:43:10.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.607+0000 7f166f078740 1 --2- 172.21.15.86:0/1878271330 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5586b97da000 0x5586b97dc420 crc :-1 s=READY pgs=154 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.607+0000 7f166f078740 1 --2- 172.21.15.86:0/1878271330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5586b9762a80 0x5586b9766d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.607+0000 7f166f078740 1 --2- 172.21.15.86:0/1878271330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5586b9759ea0 0x5586b976ade0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.607+0000 7f166f078740 1 --2- 172.21.15.86:0/1878271330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5586b9759580 0x5586b976a8a0 unknown :-1 s=CLOSED pgs=1124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.607+0000 7f164dffb640 0 client.17544 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:10.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.607+0000 7f166f078740 1 -- 172.21.15.86:0/1878271330 >> 172.21.15.86:0/1878271330 conn(0x5586b9753cb0 msgr2=0x5586b9754f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:10.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.607+0000 7f166f078740 1 -- 172.21.15.86:0/1878271330 shutdown_connections 2024-04-21T08:43:10.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.607+0000 7f166f078740 1 -- 172.21.15.86:0/1878271330 wait complete. 2024-04-21T08:43:10.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.607+0000 7f166f078740 20 client.17544 trim_cache size 0 max 16384 2024-04-21T08:43:10.614 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:43:10.615 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-21T08:43:10.615 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-21T08:43:10.615 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:43:10.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.607+0000 7f166c481640 1 -- 172.21.15.86:0/1969879027 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f16500a3d20 msgr2=0x7f16500a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:10.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.607+0000 7f166c481640 1 --2- 172.21.15.86:0/1969879027 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f16500a3d20 0x7f16500a61e0 secure :-1 s=READY pgs=173 cs=0 l=1 rev1=1 crypto rx=0x7f1654011cb0 tx=0x7f165401c000 comp rx=0 tx=0).stop 2024-04-21T08:43:10.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.607+0000 7f166c481640 1 -- 172.21.15.86:0/1969879027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1664108060 msgr2=0x7f1664101bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:10.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.607+0000 7f166c481640 1 --2- 172.21.15.86:0/1969879027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1664108060 0x7f1664101bf0 secure :-1 s=READY pgs=1120 cs=0 l=1 rev1=1 crypto rx=0x7f165c005940 tx=0x7f165c0045e0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.607+0000 7f166c481640 1 -- 172.21.15.86:0/1969879027 shutdown_connections 2024-04-21T08:43:10.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.607+0000 7f166c481640 1 --2- 172.21.15.86:0/1969879027 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f16500a3d20 0x7f16500a61e0 unknown :-1 s=CLOSED pgs=173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.607+0000 7f166c481640 1 --2- 172.21.15.86:0/1969879027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1664108060 0x7f1664101bf0 unknown :-1 s=CLOSED pgs=1120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.607+0000 7f166c481640 1 --2- 172.21.15.86:0/1969879027 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f166406af70 0x7f16641016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.607+0000 7f166c481640 1 --2- 172.21.15.86:0/1969879027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f166406a5b0 0x7f1664101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.607+0000 7f166c481640 1 -- 172.21.15.86:0/1969879027 >> 172.21.15.86:0/1969879027 conn(0x7f1664077c40 msgr2=0x7f16640ffa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:10.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.607+0000 7f166c481640 1 -- 172.21.15.86:0/1969879027 shutdown_connections 2024-04-21T08:43:10.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.608+0000 7f166c481640 1 -- 172.21.15.86:0/1969879027 wait complete. 2024-04-21T08:43:10.617 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:43:10.618 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:43:10.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.716+0000 7f20ef92c640 1 -- 172.21.15.86:0/1390887921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20e8101b60 msgr2=0x7f20e810dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:10.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.716+0000 7f20ef92c640 1 --2- 172.21.15.86:0/1390887921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20e8101b60 0x7f20e810dc90 secure :-1 s=READY pgs=1125 cs=0 l=1 rev1=1 crypto rx=0x7f20e0009cd0 tx=0x7f20e002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.716+0000 7f20ef92c640 1 -- 172.21.15.86:0/1390887921 shutdown_connections 2024-04-21T08:43:10.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.716+0000 7f20ef92c640 1 --2- 172.21.15.86:0/1390887921 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20e8101b60 0x7f20e810dc90 unknown :-1 s=CLOSED pgs=1125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.716+0000 7f20ef92c640 1 --2- 172.21.15.86:0/1390887921 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20e81011a0 0x7f20e8101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.716+0000 7f20ef92c640 1 --2- 172.21.15.86:0/1390887921 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20e8105e30 0x7f20e8106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.716+0000 7f20ef92c640 1 -- 172.21.15.86:0/1390887921 >> 172.21.15.86:0/1390887921 conn(0x7f20e8077e30 msgr2=0x7f20e8078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:10.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.716+0000 7f20ef92c640 1 -- 172.21.15.86:0/1390887921 shutdown_connections 2024-04-21T08:43:10.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.716+0000 7f20ef92c640 1 -- 172.21.15.86:0/1390887921 wait complete. 2024-04-21T08:43:10.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.717+0000 7f20ef92c640 1 Processor -- start 2024-04-21T08:43:10.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.717+0000 7f20ef92c640 1 -- start start 2024-04-21T08:43:10.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.717+0000 7f20ef92c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20e81011a0 0x7f20e8197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:10.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.717+0000 7f20ef92c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20e8101b60 0x7f20e8197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:10.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.717+0000 7f20ef92c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20e8105e30 0x7f20e819bf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:10.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.717+0000 7f20ef92c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f20e8111140 con 0x7f20e81011a0 2024-04-21T08:43:10.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.717+0000 7f20ef92c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f20e8110fc0 con 0x7f20e8101b60 2024-04-21T08:43:10.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.717+0000 7f20ef92c640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f20e81112c0 con 0x7f20e8105e30 2024-04-21T08:43:10.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.717+0000 7f20ed6c8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20e81011a0 0x7f20e8197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:10.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.717+0000 7f20ed6c8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20e81011a0 0x7f20e8197640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50940/0 (socket says 172.21.15.86:50940) 2024-04-21T08:43:10.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.717+0000 7f20ed6c8640 1 -- 172.21.15.86:0/227440889 learned_addr learned my addr 172.21.15.86:0/227440889 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:10.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.717+0000 7f20ecec7640 1 --2- 172.21.15.86:0/227440889 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20e8101b60 0x7f20e8197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:10.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.717+0000 7f20edec9640 1 --2- 172.21.15.86:0/227440889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20e8105e30 0x7f20e819bf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:10.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.717+0000 7f20ed6c8640 1 -- 172.21.15.86:0/227440889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20e8105e30 msgr2=0x7f20e819bf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:10.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.717+0000 7f20ed6c8640 1 --2- 172.21.15.86:0/227440889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20e8105e30 0x7f20e819bf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.717+0000 7f20ed6c8640 1 -- 172.21.15.86:0/227440889 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20e8101b60 msgr2=0x7f20e8197b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:10.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.717+0000 7f20ed6c8640 1 --2- 172.21.15.86:0/227440889 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20e8101b60 0x7f20e8197b80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.717+0000 7f20ed6c8640 1 -- 172.21.15.86:0/227440889 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f20e819c630 con 0x7f20e81011a0 2024-04-21T08:43:10.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.717+0000 7f20ed6c8640 1 --2- 172.21.15.86:0/227440889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20e81011a0 0x7f20e8197640 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f20e400b4a0 tx=0x7f20e400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:10.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.718+0000 7f20de7fc640 1 -- 172.21.15.86:0/227440889 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e4014070 con 0x7f20e81011a0 2024-04-21T08:43:10.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.718+0000 7f20de7fc640 1 -- 172.21.15.86:0/227440889 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f20e4010ac0 con 0x7f20e81011a0 2024-04-21T08:43:10.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.718+0000 7f20de7fc640 1 -- 172.21.15.86:0/227440889 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f20e4017c30 con 0x7f20e81011a0 2024-04-21T08:43:10.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.718+0000 7f20ef92c640 1 -- 172.21.15.86:0/227440889 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f20e819c920 con 0x7f20e81011a0 2024-04-21T08:43:10.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.718+0000 7f20ef92c640 1 -- 172.21.15.86:0/227440889 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f20e81a41b0 con 0x7f20e81011a0 2024-04-21T08:43:10.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.718+0000 7f20ef92c640 1 -- 172.21.15.86:0/227440889 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f20e8102570 con 0x7f20e81011a0 2024-04-21T08:43:10.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.720+0000 7f20de7fc640 1 -- 172.21.15.86:0/227440889 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f20e4017070 con 0x7f20e81011a0 2024-04-21T08:43:10.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.720+0000 7f20de7fc640 1 --2- 172.21.15.86:0/227440889 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f20d40a3da0 0x7f20d40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:10.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.720+0000 7f20ecec7640 1 --2- 172.21.15.86:0/227440889 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f20d40a3da0 0x7f20d40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:10.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.720+0000 7f20de7fc640 1 -- 172.21.15.86:0/227440889 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f20e40d0140 con 0x7f20e81011a0 2024-04-21T08:43:10.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.721+0000 7f20ecec7640 1 --2- 172.21.15.86:0/227440889 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f20d40a3da0 0x7f20d40a6260 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f20e8100d00 tx=0x7f20d8005f90 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:10.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.724+0000 7f20de7fc640 1 -- 172.21.15.86:0/227440889 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f20e409c580 con 0x7f20e81011a0 2024-04-21T08:43:10.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:10 smithi179 ceph-mon[31829]: pgmap v134: 129 pgs: 129 active+clean; 5.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 70 KiB/s wr, 37 op/s 2024-04-21T08:43:10.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:10 smithi086 ceph-mon[27939]: pgmap v134: 129 pgs: 129 active+clean; 5.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 70 KiB/s wr, 37 op/s 2024-04-21T08:43:10.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.939+0000 7f20ef92c640 1 -- 172.21.15.86:0/227440889 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f20e8068de0 con 0x7f20e81011a0 2024-04-21T08:43:10.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.940+0000 7f20de7fc640 1 -- 172.21.15.86:0/227440889 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7f20e409be80 con 0x7f20e81011a0 2024-04-21T08:43:10.939 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:43:10.939 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:38:45.202126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24398,24394,24455,24472]},"id":1}]} 2024-04-21T08:43:10.940 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-21T08:43:10.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.942+0000 7f20ef92c640 1 -- 172.21.15.86:0/227440889 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f20d40a3da0 msgr2=0x7f20d40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:10.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.942+0000 7f20ef92c640 1 --2- 172.21.15.86:0/227440889 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f20d40a3da0 0x7f20d40a6260 secure :-1 s=READY pgs=174 cs=0 l=1 rev1=1 crypto rx=0x7f20e8100d00 tx=0x7f20d8005f90 comp rx=0 tx=0).stop 2024-04-21T08:43:10.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.942+0000 7f20ef92c640 1 -- 172.21.15.86:0/227440889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20e81011a0 msgr2=0x7f20e8197640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:10.941 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.942+0000 7f20ef92c640 1 --2- 172.21.15.86:0/227440889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20e81011a0 0x7f20e8197640 secure :-1 s=READY pgs=1126 cs=0 l=1 rev1=1 crypto rx=0x7f20e400b4a0 tx=0x7f20e400b960 comp rx=0 tx=0).stop 2024-04-21T08:43:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.942+0000 7f20ef92c640 1 -- 172.21.15.86:0/227440889 shutdown_connections 2024-04-21T08:43:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.942+0000 7f20ef92c640 1 --2- 172.21.15.86:0/227440889 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f20d40a3da0 0x7f20d40a6260 unknown :-1 s=CLOSED pgs=174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.942+0000 7f20ef92c640 1 --2- 172.21.15.86:0/227440889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f20e8105e30 0x7f20e819bf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.942+0000 7f20ef92c640 1 --2- 172.21.15.86:0/227440889 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f20e8101b60 0x7f20e8197b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.942+0000 7f20ef92c640 1 --2- 172.21.15.86:0/227440889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f20e81011a0 0x7f20e8197640 unknown :-1 s=CLOSED pgs=1126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.942+0000 7f20ef92c640 1 -- 172.21.15.86:0/227440889 >> 172.21.15.86:0/227440889 conn(0x7f20e8077e30 msgr2=0x7f20e80fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.942+0000 7f20ef92c640 1 -- 172.21.15.86:0/227440889 shutdown_connections 2024-04-21T08:43:10.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:10.942+0000 7f20ef92c640 1 -- 172.21.15.86:0/227440889 wait complete. 2024-04-21T08:43:10.952 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:43:10.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:10 smithi088 ceph-mon[31259]: pgmap v134: 129 pgs: 129 active+clean; 5.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 70 KiB/s wr, 37 op/s 2024-04-21T08:43:11.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.052+0000 7fb55e674640 1 -- 172.21.15.86:0/1127656355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb558102b40 msgr2=0x7fb558106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.052+0000 7fb55e674640 1 --2- 172.21.15.86:0/1127656355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb558102b40 0x7fb558106b80 secure :-1 s=READY pgs=1127 cs=0 l=1 rev1=1 crypto rx=0x7fb54c009900 tx=0x7fb54c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:43:11.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.052+0000 7fb55e674640 1 -- 172.21.15.86:0/1127656355 shutdown_connections 2024-04-21T08:43:11.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.052+0000 7fb55e674640 1 --2- 172.21.15.86:0/1127656355 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb558107260 0x7fb55810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.052+0000 7fb55e674640 1 --2- 172.21.15.86:0/1127656355 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb558102b40 0x7fb558106b80 unknown :-1 s=CLOSED pgs=1127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.052+0000 7fb55e674640 1 --2- 172.21.15.86:0/1127656355 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb558102190 0x7fb558102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.052+0000 7fb55e674640 1 -- 172.21.15.86:0/1127656355 >> 172.21.15.86:0/1127656355 conn(0x7fb5580793a0 msgr2=0x7fb5580797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:11.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.052+0000 7fb55e674640 1 -- 172.21.15.86:0/1127656355 shutdown_connections 2024-04-21T08:43:11.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.052+0000 7fb55e674640 1 -- 172.21.15.86:0/1127656355 wait complete. 2024-04-21T08:43:11.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.053+0000 7fb55e674640 1 Processor -- start 2024-04-21T08:43:11.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.053+0000 7fb55e674640 1 -- start start 2024-04-21T08:43:11.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.053+0000 7fb55e674640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb558102190 0x7fb55819e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.053+0000 7fb55e674640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb558107260 0x7fb55819e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.053+0000 7fb55e674640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb558198620 0x7fb558198ad0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.053+0000 7fb55e674640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb5581121a0 con 0x7fb558102190 2024-04-21T08:43:11.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.053+0000 7fb55e674640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb558112020 con 0x7fb558107260 2024-04-21T08:43:11.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.053+0000 7fb55e674640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb558112320 con 0x7fb558198620 2024-04-21T08:43:11.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.053+0000 7fb557fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb558102190 0x7fb55819e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.053+0000 7fb557fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb558102190 0x7fb55819e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50960/0 (socket says 172.21.15.86:50960) 2024-04-21T08:43:11.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.053+0000 7fb557fff640 1 -- 172.21.15.86:0/2088116809 learned_addr learned my addr 172.21.15.86:0/2088116809 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:11.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.053+0000 7fb5577fe640 1 --2- 172.21.15.86:0/2088116809 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb558107260 0x7fb55819e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.053+0000 7fb55cc11640 1 --2- 172.21.15.86:0/2088116809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb558198620 0x7fb558198ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.053+0000 7fb557fff640 1 -- 172.21.15.86:0/2088116809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb558198620 msgr2=0x7fb558198ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.053+0000 7fb557fff640 1 --2- 172.21.15.86:0/2088116809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb558198620 0x7fb558198ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.053+0000 7fb557fff640 1 -- 172.21.15.86:0/2088116809 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb558107260 msgr2=0x7fb55819e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.053+0000 7fb557fff640 1 --2- 172.21.15.86:0/2088116809 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb558107260 0x7fb55819e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.053+0000 7fb557fff640 1 -- 172.21.15.86:0/2088116809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb558199140 con 0x7fb558102190 2024-04-21T08:43:11.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.054+0000 7fb557fff640 1 --2- 172.21.15.86:0/2088116809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb558102190 0x7fb55819e420 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fb54800c890 tx=0x7fb54800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:11.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.054+0000 7fb5557fa640 1 -- 172.21.15.86:0/2088116809 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb548004220 con 0x7fb558102190 2024-04-21T08:43:11.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.054+0000 7fb5557fa640 1 -- 172.21.15.86:0/2088116809 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5480043c0 con 0x7fb558102190 2024-04-21T08:43:11.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.054+0000 7fb55e674640 1 -- 172.21.15.86:0/2088116809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb558199430 con 0x7fb558102190 2024-04-21T08:43:11.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.054+0000 7fb5557fa640 1 -- 172.21.15.86:0/2088116809 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb548004a00 con 0x7fb558102190 2024-04-21T08:43:11.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.054+0000 7fb55e674640 1 -- 172.21.15.86:0/2088116809 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5581a5630 con 0x7fb558102190 2024-04-21T08:43:11.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.056+0000 7fb5557fa640 1 -- 172.21.15.86:0/2088116809 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb5480079b0 con 0x7fb558102190 2024-04-21T08:43:11.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.056+0000 7fb56126b740 1 Processor -- start 2024-04-21T08:43:11.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.056+0000 7fb56126b740 1 -- start start 2024-04-21T08:43:11.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.056+0000 7fb5557fa640 1 --2- 172.21.15.86:0/2088116809 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb5400a3d20 0x7fb5400a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.056+0000 7fb56126b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a62384330 0x560a62384710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.056+0000 7fb56126b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a62384ce0 0x560a6238d820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.056+0000 7fb56126b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560a6238de40 0x560a6238e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.056+0000 7fb56126b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560a622e91f0 con 0x560a62384330 2024-04-21T08:43:11.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.056+0000 7fb56126b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x560a622864b0 con 0x560a6238de40 2024-04-21T08:43:11.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.057+0000 7fb56126b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x560a622e51b0 con 0x560a62384ce0 2024-04-21T08:43:11.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.057+0000 7fb55cc11640 1 --2- 172.21.15.86:0/2088116809 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb5400a3d20 0x7fb5400a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.057+0000 7fb557fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a62384330 0x560a62384710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.057+0000 7fb557fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a62384330 0x560a62384710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50964/0 (socket says 172.21.15.86:50964) 2024-04-21T08:43:11.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.057+0000 7fb557fff640 1 -- 172.21.15.86:0/3024235625 learned_addr learned my addr 172.21.15.86:0/3024235625 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:11.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.057+0000 7fb5557fa640 1 -- 172.21.15.86:0/2088116809 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fb5480d08c0 con 0x7fb558102190 2024-04-21T08:43:11.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.057+0000 7fb55cc11640 1 --2- 172.21.15.86:0/3024235625 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560a6238de40 0x560a6238e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.057+0000 7fb557fff640 1 -- 172.21.15.86:0/3024235625 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a62384ce0 msgr2=0x560a6238d820 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:43:11.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.057+0000 7fb557fff640 1 --2- 172.21.15.86:0/3024235625 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a62384ce0 0x560a6238d820 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.057+0000 7fb557fff640 1 -- 172.21.15.86:0/3024235625 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560a6238de40 msgr2=0x560a6238e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.057+0000 7fb557fff640 1 --2- 172.21.15.86:0/3024235625 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560a6238de40 0x560a6238e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.057+0000 7fb557fff640 1 -- 172.21.15.86:0/3024235625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a622781b0 con 0x560a62384330 2024-04-21T08:43:11.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.057+0000 7fb557fff640 1 --2- 172.21.15.86:0/3024235625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a62384330 0x560a62384710 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fb548004770 tx=0x7fb54809c380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bac82a746b9d8f5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:11.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.057+0000 7fb55cc11640 1 --2- 172.21.15.86:0/2088116809 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb5400a3d20 0x7fb5400a61e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb550005cc0 tx=0x7fb550012960 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:11.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.057+0000 7fb53e7fc640 1 -- 172.21.15.86:0/3024235625 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54801d040 con 0x560a62384330 2024-04-21T08:43:11.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.057+0000 7fb53e7fc640 1 -- 172.21.15.86:0/3024235625 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb548005290 con 0x560a62384330 2024-04-21T08:43:11.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.057+0000 7fb53e7fc640 1 -- 172.21.15.86:0/3024235625 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5480a5160 con 0x560a62384330 2024-04-21T08:43:11.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.058+0000 7fb56126b740 1 -- 172.21.15.86:0/3024235625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a62384330 msgr2=0x560a62384710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.058+0000 7fb56126b740 1 --2- 172.21.15.86:0/3024235625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a62384330 0x560a62384710 secure :-1 s=READY pgs=1129 cs=0 l=1 rev1=1 crypto rx=0x7fb548004770 tx=0x7fb54809c380 comp rx=0 tx=0).stop 2024-04-21T08:43:11.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.058+0000 7fb56126b740 1 -- 172.21.15.86:0/3024235625 shutdown_connections 2024-04-21T08:43:11.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.058+0000 7fb56126b740 1 --2- 172.21.15.86:0/3024235625 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560a6238de40 0x560a6238e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.058+0000 7fb56126b740 1 --2- 172.21.15.86:0/3024235625 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a62384ce0 0x560a6238d820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.058+0000 7fb56126b740 1 --2- 172.21.15.86:0/3024235625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a62384330 0x560a62384710 unknown :-1 s=CLOSED pgs=1129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.058+0000 7fb56126b740 1 -- 172.21.15.86:0/3024235625 >> 172.21.15.86:0/3024235625 conn(0x560a6237eba0 msgr2=0x560a6237ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:11.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.058+0000 7fb56126b740 1 -- 172.21.15.86:0/3024235625 shutdown_connections 2024-04-21T08:43:11.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.058+0000 7fb56126b740 1 -- 172.21.15.86:0/3024235625 wait complete. 2024-04-21T08:43:11.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.058+0000 7fb56126b740 1 -- start start 2024-04-21T08:43:11.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.058+0000 7fb56126b740 1 Processor -- start 2024-04-21T08:43:11.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.059+0000 7fb56126b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a62384330 0x560a62394250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.059+0000 7fb56126b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560a62384ce0 0x560a623915e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.059+0000 7fb56126b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a6238de40 0x560a62391b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.059+0000 7fb56126b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560a62394dc0 con 0x560a62384330 2024-04-21T08:43:11.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.059+0000 7fb56126b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x560a622a2500 con 0x560a62384ce0 2024-04-21T08:43:11.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.059+0000 7fb56126b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x560a62394f40 con 0x560a6238de40 2024-04-21T08:43:11.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.059+0000 7fb557fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a62384330 0x560a62394250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.059+0000 7fb56126b740 10 client.0 _ll_register_callbacks cb 0x560a62270b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:11.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.059+0000 7fb557fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a62384330 0x560a62394250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50966/0 (socket says 172.21.15.86:50966) 2024-04-21T08:43:11.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.059+0000 7fb557fff640 1 -- 172.21.15.86:0/2766368578 learned_addr learned my addr 172.21.15.86:0/2766368578 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:11.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.059+0000 7fb5577fe640 1 --2- 172.21.15.86:0/2766368578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560a62384ce0 0x560a623915e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.059+0000 7fb557fff640 1 -- 172.21.15.86:0/2766368578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a6238de40 msgr2=0x560a62391b60 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:43:11.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.059+0000 7fb557fff640 1 --2- 172.21.15.86:0/2766368578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a6238de40 0x560a62391b60 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.059+0000 7fb557fff640 1 -- 172.21.15.86:0/2766368578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560a62384ce0 msgr2=0x560a623915e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.059+0000 7fb557fff640 1 --2- 172.21.15.86:0/2766368578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560a62384ce0 0x560a623915e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.059+0000 7fb557fff640 1 -- 172.21.15.86:0/2766368578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a61ffb3f0 con 0x560a62384330 2024-04-21T08:43:11.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.059+0000 7fb557fff640 1 --2- 172.21.15.86:0/2766368578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a62384330 0x560a62394250 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fb54809c1f0 tx=0x7fb54806da10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c085b3421180226 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:11.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.059+0000 7fb5377fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:11.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.059+0000 7fb5377fe640 1 -- 172.21.15.86:0/2766368578 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54801d040 con 0x560a62384330 2024-04-21T08:43:11.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.059+0000 7fb5377fe640 1 -- 172.21.15.86:0/2766368578 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fb5480cdf20 con 0x560a62384330 2024-04-21T08:43:11.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.059+0000 7fb56126b740 1 -- 172.21.15.86:0/2766368578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560a62288930 con 0x560a62384330 2024-04-21T08:43:11.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.060+0000 7fb5377fe640 1 -- 172.21.15.86:0/2766368578 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb5480ce120 con 0x560a62384330 2024-04-21T08:43:11.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.060+0000 7fb56126b740 10 client.17562 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:11.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.060+0000 7fb56126b740 1 -- 172.21.15.86:0/2766368578 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560a620c57d0 con 0x560a62384330 2024-04-21T08:43:11.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.060+0000 7fb5377fe640 1 -- 172.21.15.86:0/2766368578 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560a62288930 con 0x560a62384330 2024-04-21T08:43:11.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.060+0000 7fb5377fe640 1 -- 172.21.15.86:0/2766368578 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb5480a4cd0 con 0x560a62384330 2024-04-21T08:43:11.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.060+0000 7fb56126b740 10 client.17562 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:11.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.060+0000 7fb56126b740 10 client.17562 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:43:11.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.060+0000 7fb56126b740 20 client.17562 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:11.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.060+0000 7fb56126b740 1 --2- 172.21.15.86:0/2766368578 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x560a623a1f10 0x560a623a4300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.060+0000 7fb56126b740 4 client.17562 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:11.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.060+0000 7fb56126b740 1 -- 172.21.15.86:0/2766368578 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560a622f6c10 con 0x560a623a1f10 2024-04-21T08:43:11.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.062+0000 7fb5577fe640 1 --2- 172.21.15.86:0/2766368578 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x560a623a1f10 0x560a623a4300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.063+0000 7fb5377fe640 10 client.17562 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:43:11.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.063+0000 7fb5577fe640 1 --2- 172.21.15.86:0/2766368578 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x560a623a1f10 0x560a623a4300 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d758b1f822b908b7 server_cookie=4d1ef7aaa3b12c7d in_seq=0 out_seq=0 2024-04-21T08:43:11.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.064+0000 7fb5377fe640 1 -- 172.21.15.86:0/2766368578 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560a622f6c10 con 0x560a623a1f10 2024-04-21T08:43:11.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.064+0000 7fb5377fe640 10 client.17562 handle_command_reply: tid=0 2024-04-21T08:43:11.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.064+0000 7fb56126b740 1 client.17562 shutdown 2024-04-21T08:43:11.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.064+0000 7fb56126b740 1 -- 172.21.15.86:0/2766368578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a62384330 msgr2=0x560a62394250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.064+0000 7fb56126b740 1 --2- 172.21.15.86:0/2766368578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a62384330 0x560a62394250 secure :-1 s=READY pgs=1130 cs=0 l=1 rev1=1 crypto rx=0x7fb54809c1f0 tx=0x7fb54806da10 comp rx=0 tx=0).stop 2024-04-21T08:43:11.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.064+0000 7fb56126b740 1 -- 172.21.15.86:0/2766368578 shutdown_connections 2024-04-21T08:43:11.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.064+0000 7fb56126b740 1 --2- 172.21.15.86:0/2766368578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a6238de40 0x560a62391b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.064+0000 7fb56126b740 1 --2- 172.21.15.86:0/2766368578 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x560a623a1f10 0x560a623a4300 crc :-1 s=READY pgs=48 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.064+0000 7fb56126b740 1 --2- 172.21.15.86:0/2766368578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560a62384ce0 0x560a623915e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.064+0000 7fb56126b740 1 --2- 172.21.15.86:0/2766368578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a62384330 0x560a62394250 unknown :-1 s=CLOSED pgs=1130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.064+0000 7fb56126b740 1 -- 172.21.15.86:0/2766368578 >> 172.21.15.86:0/2766368578 conn(0x560a6237eba0 msgr2=0x560a623808f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:11.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.064+0000 7fb5377fe640 0 client.17562 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:43:11.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.064+0000 7fb56126b740 1 -- 172.21.15.86:0/2766368578 shutdown_connections 2024-04-21T08:43:11.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.064+0000 7fb56126b740 1 -- 172.21.15.86:0/2766368578 wait complete. 2024-04-21T08:43:11.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.065+0000 7fb56126b740 20 client.17562 trim_cache size 0 max 16384 2024-04-21T08:43:11.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.077+0000 7fb56126b740 1 Processor -- start 2024-04-21T08:43:11.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.077+0000 7fb56126b740 1 -- start start 2024-04-21T08:43:11.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.078+0000 7fb56126b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a62384330 0x560a6238f230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.078+0000 7fb56126b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a62384ce0 0x560a6239d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.078+0000 7fb56126b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560a6238de40 0x560a62393d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.078+0000 7fb56126b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb5581ad160 con 0x560a62384330 2024-04-21T08:43:11.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.078+0000 7fb56126b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb5581aeca0 con 0x560a6238de40 2024-04-21T08:43:11.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.078+0000 7fb56126b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb5581ad830 con 0x560a62384ce0 2024-04-21T08:43:11.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.078+0000 7fb557fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a62384330 0x560a6238f230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.078+0000 7fb557fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a62384330 0x560a6238f230 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50976/0 (socket says 172.21.15.86:50976) 2024-04-21T08:43:11.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.078+0000 7fb557fff640 1 -- 172.21.15.86:0/860429384 learned_addr learned my addr 172.21.15.86:0/860429384 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:11.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.078+0000 7fb5577fe640 1 --2- 172.21.15.86:0/860429384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a62384ce0 0x560a6239d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.078+0000 7fb557fff640 1 -- 172.21.15.86:0/860429384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a62384ce0 msgr2=0x560a6239d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.078+0000 7fb557fff640 1 --2- 172.21.15.86:0/860429384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a62384ce0 0x560a6239d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.078+0000 7fb55cc11640 1 --2- 172.21.15.86:0/860429384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560a6238de40 0x560a62393d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.078+0000 7fb557fff640 1 -- 172.21.15.86:0/860429384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560a6238de40 msgr2=0x560a62393d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.078+0000 7fb557fff640 1 --2- 172.21.15.86:0/860429384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560a6238de40 0x560a62393d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.078+0000 7fb557fff640 1 -- 172.21.15.86:0/860429384 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560a62391ed0 con 0x560a62384330 2024-04-21T08:43:11.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.078+0000 7fb557fff640 1 --2- 172.21.15.86:0/860429384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a62384330 0x560a6238f230 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fb54809a7a0 tx=0x7fb548007730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a6915028823999 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:11.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.078+0000 7fb53dffb640 1 -- 172.21.15.86:0/860429384 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb54809be50 con 0x560a62384330 2024-04-21T08:43:11.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.078+0000 7fb53dffb640 1 -- 172.21.15.86:0/860429384 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb548098040 con 0x560a62384330 2024-04-21T08:43:11.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.078+0000 7fb53dffb640 1 -- 172.21.15.86:0/860429384 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5480a4e30 con 0x560a62384330 2024-04-21T08:43:11.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.079+0000 7fb56126b740 1 -- 172.21.15.86:0/860429384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a62384330 msgr2=0x560a6238f230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.079+0000 7fb56126b740 1 --2- 172.21.15.86:0/860429384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a62384330 0x560a6238f230 secure :-1 s=READY pgs=1131 cs=0 l=1 rev1=1 crypto rx=0x7fb54809a7a0 tx=0x7fb548007730 comp rx=0 tx=0).stop 2024-04-21T08:43:11.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.079+0000 7fb56126b740 1 -- 172.21.15.86:0/860429384 shutdown_connections 2024-04-21T08:43:11.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.079+0000 7fb56126b740 1 --2- 172.21.15.86:0/860429384 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560a6238de40 0x560a62393d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.079+0000 7fb56126b740 1 --2- 172.21.15.86:0/860429384 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a62384ce0 0x560a6239d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.079+0000 7fb56126b740 1 --2- 172.21.15.86:0/860429384 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a62384330 0x560a6238f230 unknown :-1 s=CLOSED pgs=1131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.079+0000 7fb56126b740 1 -- 172.21.15.86:0/860429384 >> 172.21.15.86:0/860429384 conn(0x560a6237eba0 msgr2=0x560a6239d1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:11.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.079+0000 7fb56126b740 1 -- 172.21.15.86:0/860429384 shutdown_connections 2024-04-21T08:43:11.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.079+0000 7fb56126b740 1 -- 172.21.15.86:0/860429384 wait complete. 2024-04-21T08:43:11.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.079+0000 7fb56126b740 1 -- start start 2024-04-21T08:43:11.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.079+0000 7fb56126b740 1 Processor -- start 2024-04-21T08:43:11.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.079+0000 7fb56126b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560a62384330 0x560a623a7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.079+0000 7fb56126b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a62384ce0 0x560a623a4900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.080+0000 7fb56126b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a6238de40 0x560a623a4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.080+0000 7fb56126b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb5581ae5d0 con 0x560a6238de40 2024-04-21T08:43:11.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.080+0000 7fb56126b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb5581adf00 con 0x560a62384330 2024-04-21T08:43:11.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.080+0000 7fb56126b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb5581ab120 con 0x560a62384ce0 2024-04-21T08:43:11.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.080+0000 7fb56126b740 10 client.0 _ll_register_callbacks cb 0x7fb54806c080 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:11.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.080+0000 7fb55cc11640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a6238de40 0x560a623a4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.080+0000 7fb55cc11640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a6238de40 0x560a623a4e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50988/0 (socket says 172.21.15.86:50988) 2024-04-21T08:43:11.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.080+0000 7fb55cc11640 1 -- 172.21.15.86:0/2984491245 learned_addr learned my addr 172.21.15.86:0/2984491245 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:11.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.080+0000 7fb5577fe640 1 --2- 172.21.15.86:0/2984491245 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a62384ce0 0x560a623a4900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.080+0000 7fb557fff640 1 --2- 172.21.15.86:0/2984491245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560a62384330 0x560a623a7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.080+0000 7fb55cc11640 1 -- 172.21.15.86:0/2984491245 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a62384ce0 msgr2=0x560a623a4900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.080+0000 7fb55cc11640 1 --2- 172.21.15.86:0/2984491245 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a62384ce0 0x560a623a4900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.080+0000 7fb55cc11640 1 -- 172.21.15.86:0/2984491245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560a62384330 msgr2=0x560a623a7470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.080+0000 7fb55cc11640 1 --2- 172.21.15.86:0/2984491245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560a62384330 0x560a623a7470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.080+0000 7fb55cc11640 1 -- 172.21.15.86:0/2984491245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560a623a5380 con 0x560a6238de40 2024-04-21T08:43:11.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.080+0000 7fb557fff640 1 --2- 172.21.15.86:0/2984491245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560a62384330 0x560a623a7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:11.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.080+0000 7fb55cc11640 1 --2- 172.21.15.86:0/2984491245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a6238de40 0x560a623a4e40 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fb550010cd0 tx=0x7fb550018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e72cdcfec813e99 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:11.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.080+0000 7fb5377fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:11.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.080+0000 7fb5377fe640 1 -- 172.21.15.86:0/2984491245 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb550023070 con 0x560a6238de40 2024-04-21T08:43:11.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.080+0000 7fb5377fe640 1 -- 172.21.15.86:0/2984491245 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fb550003970 con 0x560a6238de40 2024-04-21T08:43:11.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.080+0000 7fb56126b740 1 -- 172.21.15.86:0/2984491245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560a622f0a80 con 0x560a6238de40 2024-04-21T08:43:11.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.080+0000 7fb5377fe640 1 -- 172.21.15.86:0/2984491245 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb550030050 con 0x560a6238de40 2024-04-21T08:43:11.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.080+0000 7fb56126b740 10 client.17568 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:11.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.080+0000 7fb56126b740 1 -- 172.21.15.86:0/2984491245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560a62407e40 con 0x560a6238de40 2024-04-21T08:43:11.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.081+0000 7fb5377fe640 1 -- 172.21.15.86:0/2984491245 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560a622f0a80 con 0x560a6238de40 2024-04-21T08:43:11.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.081+0000 7fb5377fe640 1 -- 172.21.15.86:0/2984491245 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb55001e380 con 0x560a6238de40 2024-04-21T08:43:11.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.081+0000 7fb56126b740 10 client.17568 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:11.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.081+0000 7fb56126b740 10 client.17568 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:43:11.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.081+0000 7fb56126b740 20 client.17568 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:11.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.081+0000 7fb56126b740 1 --2- 172.21.15.86:0/2984491245 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x560a62408620 0x560a6240aa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.081+0000 7fb56126b740 4 client.17568 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:43:11.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.081+0000 7fb56126b740 1 -- 172.21.15.86:0/2984491245 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560a6226c6e0 con 0x560a62408620 2024-04-21T08:43:11.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.081+0000 7fb557fff640 1 --2- 172.21.15.86:0/2984491245 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x560a62408620 0x560a6240aa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.081+0000 7fb557fff640 1 --2- 172.21.15.86:0/2984491245 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x560a62408620 0x560a6240aa40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=22653b56bea11115 server_cookie=efe0fd10f88ca21b in_seq=0 out_seq=0 2024-04-21T08:43:11.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.081+0000 7fb5377fe640 10 client.17568 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:43:11.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.081+0000 7fb5377fe640 1 -- 172.21.15.86:0/2984491245 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560a6226c6e0 con 0x560a62408620 2024-04-21T08:43:11.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.081+0000 7fb5377fe640 10 client.17568 handle_command_reply: tid=0 2024-04-21T08:43:11.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.081+0000 7fb56126b740 1 client.17568 shutdown 2024-04-21T08:43:11.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.082+0000 7fb56126b740 1 -- 172.21.15.86:0/2984491245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a6238de40 msgr2=0x560a623a4e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.082+0000 7fb56126b740 1 --2- 172.21.15.86:0/2984491245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a6238de40 0x560a623a4e40 secure :-1 s=READY pgs=1132 cs=0 l=1 rev1=1 crypto rx=0x7fb550010cd0 tx=0x7fb550018000 comp rx=0 tx=0).stop 2024-04-21T08:43:11.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.082+0000 7fb56126b740 1 -- 172.21.15.86:0/2984491245 shutdown_connections 2024-04-21T08:43:11.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.082+0000 7fb56126b740 1 --2- 172.21.15.86:0/2984491245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560a6238de40 0x560a623a4e40 unknown :-1 s=CLOSED pgs=1132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.082+0000 7fb56126b740 1 --2- 172.21.15.86:0/2984491245 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560a62384ce0 0x560a623a4900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.082+0000 7fb56126b740 1 --2- 172.21.15.86:0/2984491245 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x560a62408620 0x560a6240aa40 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.082+0000 7fb56126b740 1 --2- 172.21.15.86:0/2984491245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560a62384330 0x560a623a7470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.082+0000 7fb5377fe640 0 client.17568 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:43:11.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.082+0000 7fb56126b740 1 -- 172.21.15.86:0/2984491245 >> 172.21.15.86:0/2984491245 conn(0x560a6237eba0 msgr2=0x560a6239da60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:11.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.082+0000 7fb56126b740 1 -- 172.21.15.86:0/2984491245 shutdown_connections 2024-04-21T08:43:11.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.082+0000 7fb56126b740 1 -- 172.21.15.86:0/2984491245 wait complete. 2024-04-21T08:43:11.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.082+0000 7fb56126b740 20 client.17568 trim_cache size 0 max 16384 2024-04-21T08:43:11.089 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:43:11.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.082+0000 7fb55e674640 1 -- 172.21.15.86:0/2088116809 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb5400a3d20 msgr2=0x7fb5400a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.082+0000 7fb55e674640 1 --2- 172.21.15.86:0/2088116809 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb5400a3d20 0x7fb5400a61e0 secure :-1 s=READY pgs=175 cs=0 l=1 rev1=1 crypto rx=0x7fb550005cc0 tx=0x7fb550012960 comp rx=0 tx=0).stop 2024-04-21T08:43:11.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.082+0000 7fb55e674640 1 -- 172.21.15.86:0/2088116809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb558102190 msgr2=0x7fb55819e420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.082+0000 7fb55e674640 1 --2- 172.21.15.86:0/2088116809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb558102190 0x7fb55819e420 secure :-1 s=READY pgs=1128 cs=0 l=1 rev1=1 crypto rx=0x7fb54800c890 tx=0x7fb54800cd50 comp rx=0 tx=0).stop 2024-04-21T08:43:11.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.082+0000 7fb55e674640 1 -- 172.21.15.86:0/2088116809 shutdown_connections 2024-04-21T08:43:11.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.082+0000 7fb55e674640 1 --2- 172.21.15.86:0/2088116809 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb5400a3d20 0x7fb5400a61e0 unknown :-1 s=CLOSED pgs=175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.082+0000 7fb55e674640 1 --2- 172.21.15.86:0/2088116809 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb558198620 0x7fb558198ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.082+0000 7fb55e674640 1 --2- 172.21.15.86:0/2088116809 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb558107260 0x7fb55819e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.082+0000 7fb55e674640 1 --2- 172.21.15.86:0/2088116809 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb558102190 0x7fb55819e420 unknown :-1 s=CLOSED pgs=1128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.082+0000 7fb55e674640 1 -- 172.21.15.86:0/2088116809 >> 172.21.15.86:0/2088116809 conn(0x7fb5580793a0 msgr2=0x7fb558100080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:11.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.083+0000 7fb55e674640 1 -- 172.21.15.86:0/2088116809 shutdown_connections 2024-04-21T08:43:11.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.083+0000 7fb55e674640 1 -- 172.21.15.86:0/2088116809 wait complete. 2024-04-21T08:43:11.092 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:43:11.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.193+0000 7f11e4306640 1 -- 172.21.15.86:0/594201473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11dc102190 msgr2=0x7f11dc102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.193+0000 7f11e4306640 1 --2- 172.21.15.86:0/594201473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11dc102190 0x7f11dc102570 secure :-1 s=READY pgs=1133 cs=0 l=1 rev1=1 crypto rx=0x7f11d4009ca0 tx=0x7f11d402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.193+0000 7f11e4306640 1 -- 172.21.15.86:0/594201473 shutdown_connections 2024-04-21T08:43:11.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.193+0000 7f11e4306640 1 --2- 172.21.15.86:0/594201473 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11dc107260 0x7f11dc10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.193+0000 7f11e4306640 1 --2- 172.21.15.86:0/594201473 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11dc102b40 0x7f11dc106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.193+0000 7f11e4306640 1 --2- 172.21.15.86:0/594201473 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11dc102190 0x7f11dc102570 unknown :-1 s=CLOSED pgs=1133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.193+0000 7f11e4306640 1 -- 172.21.15.86:0/594201473 >> 172.21.15.86:0/594201473 conn(0x7f11dc0793a0 msgr2=0x7f11dc0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:11.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.193+0000 7f11e4306640 1 -- 172.21.15.86:0/594201473 shutdown_connections 2024-04-21T08:43:11.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.193+0000 7f11e4306640 1 -- 172.21.15.86:0/594201473 wait complete. 2024-04-21T08:43:11.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.193+0000 7f11e4306640 1 Processor -- start 2024-04-21T08:43:11.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.194+0000 7f11e4306640 1 -- start start 2024-04-21T08:43:11.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.194+0000 7f11e4306640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11dc102190 0x7f11dc19e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.194+0000 7f11e4306640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11dc107260 0x7f11dc19e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.194+0000 7f11e4306640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11dc1985a0 0x7f11dc198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.194+0000 7f11e4306640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f11dc1121c0 con 0x7f11dc102190 2024-04-21T08:43:11.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.194+0000 7f11e4306640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f11dc112040 con 0x7f11dc107260 2024-04-21T08:43:11.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.194+0000 7f11e4306640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f11dc112340 con 0x7f11dc1985a0 2024-04-21T08:43:11.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.194+0000 7f11e20a2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11dc102190 0x7f11dc19e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.194+0000 7f11e20a2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11dc102190 0x7f11dc19e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:50996/0 (socket says 172.21.15.86:50996) 2024-04-21T08:43:11.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.194+0000 7f11e20a2640 1 -- 172.21.15.86:0/1405486041 learned_addr learned my addr 172.21.15.86:0/1405486041 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:11.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.194+0000 7f11e18a1640 1 --2- 172.21.15.86:0/1405486041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11dc107260 0x7f11dc19e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.194+0000 7f11e28a3640 1 --2- 172.21.15.86:0/1405486041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11dc1985a0 0x7f11dc198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.194+0000 7f11e20a2640 1 -- 172.21.15.86:0/1405486041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11dc1985a0 msgr2=0x7f11dc198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.194+0000 7f11e20a2640 1 --2- 172.21.15.86:0/1405486041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11dc1985a0 0x7f11dc198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.194+0000 7f11e20a2640 1 -- 172.21.15.86:0/1405486041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11dc107260 msgr2=0x7f11dc19e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.194+0000 7f11e20a2640 1 --2- 172.21.15.86:0/1405486041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11dc107260 0x7f11dc19e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.194+0000 7f11e20a2640 1 -- 172.21.15.86:0/1405486041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11dc1990c0 con 0x7f11dc102190 2024-04-21T08:43:11.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.194+0000 7f11e28a3640 1 --2- 172.21.15.86:0/1405486041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11dc1985a0 0x7f11dc198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:11.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.194+0000 7f11e20a2640 1 --2- 172.21.15.86:0/1405486041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11dc102190 0x7f11dc19e3f0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f11d4005e30 tx=0x7f11d4034780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:11.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.194+0000 7f11d37fe640 1 -- 172.21.15.86:0/1405486041 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d4039040 con 0x7f11dc102190 2024-04-21T08:43:11.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.194+0000 7f11d37fe640 1 -- 172.21.15.86:0/1405486041 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11d4044070 con 0x7f11dc102190 2024-04-21T08:43:11.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.194+0000 7f11e4306640 1 -- 172.21.15.86:0/1405486041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11dc199350 con 0x7f11dc102190 2024-04-21T08:43:11.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.194+0000 7f11d37fe640 1 -- 172.21.15.86:0/1405486041 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d4003e60 con 0x7f11dc102190 2024-04-21T08:43:11.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.195+0000 7f11e4306640 1 -- 172.21.15.86:0/1405486041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11dc1a5540 con 0x7f11dc102190 2024-04-21T08:43:11.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.197+0000 7f11d37fe640 1 -- 172.21.15.86:0/1405486041 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f11d40324c0 con 0x7f11dc102190 2024-04-21T08:43:11.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.197+0000 7f11e6fcb740 1 Processor -- start 2024-04-21T08:43:11.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.197+0000 7f11e6fcb740 1 -- start start 2024-04-21T08:43:11.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.198+0000 7f11e6fcb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a4327d330 0x559a4327d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.198+0000 7f11e6fcb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4327dce0 0x559a43286820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.198+0000 7f11e6fcb740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a43286e40 0x559a432872c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.198+0000 7f11e6fcb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559a431e21f0 con 0x559a4327d330 2024-04-21T08:43:11.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.198+0000 7f11e6fcb740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559a4317f4b0 con 0x559a43286e40 2024-04-21T08:43:11.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.198+0000 7f11e6fcb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559a431de1b0 con 0x559a4327dce0 2024-04-21T08:43:11.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.198+0000 7f11e28a3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a4327d330 0x559a4327d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.198+0000 7f11e28a3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a4327d330 0x559a4327d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51008/0 (socket says 172.21.15.86:51008) 2024-04-21T08:43:11.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.198+0000 7f11e28a3640 1 -- 172.21.15.86:0/3714114288 learned_addr learned my addr 172.21.15.86:0/3714114288 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:11.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.198+0000 7f11e18a1640 1 --2- 172.21.15.86:0/3714114288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a43286e40 0x559a432872c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.198+0000 7f11e20a2640 1 --2- 172.21.15.86:0/3714114288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4327dce0 0x559a43286820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.198+0000 7f11d37fe640 1 --2- 172.21.15.86:0/1405486041 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f11c80a3d20 0x7f11c80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.198+0000 7f11e28a3640 1 -- 172.21.15.86:0/3714114288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4327dce0 msgr2=0x559a43286820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.198+0000 7f11e28a3640 1 --2- 172.21.15.86:0/3714114288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4327dce0 0x559a43286820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.198+0000 7f11e28a3640 1 -- 172.21.15.86:0/3714114288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a43286e40 msgr2=0x559a432872c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.198+0000 7f11e28a3640 1 --2- 172.21.15.86:0/3714114288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a43286e40 0x559a432872c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.198+0000 7f11e28a3640 1 -- 172.21.15.86:0/3714114288 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a431711b0 con 0x559a4327d330 2024-04-21T08:43:11.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.198+0000 7f11e20a2640 1 --2- 172.21.15.86:0/3714114288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4327dce0 0x559a43286820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:11.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.198+0000 7f11e28a3640 1 --2- 172.21.15.86:0/1405486041 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f11c80a3d20 0x7f11c80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.198+0000 7f11d37fe640 1 -- 172.21.15.86:0/1405486041 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f11d40f2da0 con 0x7f11dc102190 2024-04-21T08:43:11.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.198+0000 7f11e28a3640 1 --2- 172.21.15.86:0/3714114288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a4327d330 0x559a4327d710 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f11d800be40 tx=0x7f11d800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ef3dee17d8c0e27a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:11.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.199+0000 7f11d0bfe640 1 -- 172.21.15.86:0/3714114288 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d800fde0 con 0x559a4327d330 2024-04-21T08:43:11.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.199+0000 7f11d0bfe640 1 -- 172.21.15.86:0/3714114288 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11d801cdc0 con 0x559a4327d330 2024-04-21T08:43:11.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.199+0000 7f11d0bfe640 1 -- 172.21.15.86:0/3714114288 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d801bc40 con 0x559a4327d330 2024-04-21T08:43:11.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.199+0000 7f11e6fcb740 1 -- 172.21.15.86:0/3714114288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a4327d330 msgr2=0x559a4327d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.199+0000 7f11e28a3640 1 --2- 172.21.15.86:0/1405486041 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f11c80a3d20 0x7f11c80a61e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f11d8020000 tx=0x7f11d801d000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:11.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.199+0000 7f11e6fcb740 1 --2- 172.21.15.86:0/3714114288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a4327d330 0x559a4327d710 secure :-1 s=READY pgs=1135 cs=0 l=1 rev1=1 crypto rx=0x7f11d800be40 tx=0x7f11d800f730 comp rx=0 tx=0).stop 2024-04-21T08:43:11.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.199+0000 7f11e6fcb740 1 -- 172.21.15.86:0/3714114288 shutdown_connections 2024-04-21T08:43:11.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.199+0000 7f11e6fcb740 1 --2- 172.21.15.86:0/3714114288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a43286e40 0x559a432872c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.199+0000 7f11e6fcb740 1 --2- 172.21.15.86:0/3714114288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4327dce0 0x559a43286820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.199+0000 7f11e6fcb740 1 --2- 172.21.15.86:0/3714114288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a4327d330 0x559a4327d710 unknown :-1 s=CLOSED pgs=1135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.199+0000 7f11e6fcb740 1 -- 172.21.15.86:0/3714114288 >> 172.21.15.86:0/3714114288 conn(0x559a43277ba0 msgr2=0x559a43277f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:11.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.199+0000 7f11e6fcb740 1 -- 172.21.15.86:0/3714114288 shutdown_connections 2024-04-21T08:43:11.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.200+0000 7f11e6fcb740 1 -- 172.21.15.86:0/3714114288 wait complete. 2024-04-21T08:43:11.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.200+0000 7f11e6fcb740 1 -- start start 2024-04-21T08:43:11.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.200+0000 7f11e6fcb740 1 Processor -- start 2024-04-21T08:43:11.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.200+0000 7f11e6fcb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4327d330 0x559a4328d290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.200+0000 7f11e6fcb740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a4327dce0 0x559a4328a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.200+0000 7f11e6fcb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a43286e40 0x559a4328ac30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.200+0000 7f11e6fcb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559a4328ddc0 con 0x559a43286e40 2024-04-21T08:43:11.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.200+0000 7f11e6fcb740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559a4319b500 con 0x559a4327dce0 2024-04-21T08:43:11.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.200+0000 7f11e6fcb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559a4328df40 con 0x559a4327d330 2024-04-21T08:43:11.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.200+0000 7f11e6fcb740 10 client.0 _ll_register_callbacks cb 0x559a43169b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:11.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.200+0000 7f11e28a3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a43286e40 0x559a4328ac30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.200+0000 7f11e28a3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a43286e40 0x559a4328ac30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51024/0 (socket says 172.21.15.86:51024) 2024-04-21T08:43:11.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.200+0000 7f11e28a3640 1 -- 172.21.15.86:0/2260278379 learned_addr learned my addr 172.21.15.86:0/2260278379 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:11.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.200+0000 7f11e18a1640 1 --2- 172.21.15.86:0/2260278379 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a4327dce0 0x559a4328a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.200+0000 7f11e28a3640 1 -- 172.21.15.86:0/2260278379 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4327d330 msgr2=0x559a4328d290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.200+0000 7f11e20a2640 1 --2- 172.21.15.86:0/2260278379 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4327d330 0x559a4328d290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.200+0000 7f11e28a3640 1 --2- 172.21.15.86:0/2260278379 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4327d330 0x559a4328d290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.200+0000 7f11e28a3640 1 -- 172.21.15.86:0/2260278379 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a4327dce0 msgr2=0x559a4328a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.200+0000 7f11e28a3640 1 --2- 172.21.15.86:0/2260278379 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a4327dce0 0x559a4328a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.200+0000 7f11e28a3640 1 -- 172.21.15.86:0/2260278379 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a42ef43f0 con 0x559a43286e40 2024-04-21T08:43:11.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.200+0000 7f11e18a1640 1 --2- 172.21.15.86:0/2260278379 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a4327dce0 0x559a4328a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:11.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.201+0000 7f11e28a3640 1 --2- 172.21.15.86:0/2260278379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a43286e40 0x559a4328ac30 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f11d80058c0 tx=0x7f11d8005cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e83a81ee116a1924 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:11.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.201+0000 7f11c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:11.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.201+0000 7f11c5ffb640 1 -- 172.21.15.86:0/2260278379 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d801b9c0 con 0x559a43286e40 2024-04-21T08:43:11.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.201+0000 7f11c5ffb640 1 -- 172.21.15.86:0/2260278379 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f11d8022070 con 0x559a43286e40 2024-04-21T08:43:11.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.201+0000 7f11e6fcb740 1 -- 172.21.15.86:0/2260278379 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11d801cdc0 con 0x559a43286e40 2024-04-21T08:43:11.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.201+0000 7f11c5ffb640 1 -- 172.21.15.86:0/2260278379 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f11d801bb60 con 0x559a43286e40 2024-04-21T08:43:11.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.201+0000 7f11e6fcb740 10 client.17580 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:11.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.201+0000 7f11e6fcb740 1 -- 172.21.15.86:0/2260278379 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559a42fbe7d0 con 0x559a43286e40 2024-04-21T08:43:11.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.201+0000 7f11c5ffb640 1 -- 172.21.15.86:0/2260278379 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11d801cdc0 con 0x559a43286e40 2024-04-21T08:43:11.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.201+0000 7f11c5ffb640 1 -- 172.21.15.86:0/2260278379 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f11d8026970 con 0x559a43286e40 2024-04-21T08:43:11.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.201+0000 7f11e6fcb740 10 client.17580 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:11.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.201+0000 7f11e6fcb740 10 client.17580 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:43:11.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.201+0000 7f11e6fcb740 20 client.17580 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:11.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.201+0000 7f11e6fcb740 1 --2- 172.21.15.86:0/2260278379 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x559a4329afb0 0x559a4329d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.201+0000 7f11e6fcb740 4 client.17580 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:11.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.201+0000 7f11e6fcb740 1 -- 172.21.15.86:0/2260278379 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559a43181930 con 0x559a4329afb0 2024-04-21T08:43:11.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.202+0000 7f11e20a2640 1 --2- 172.21.15.86:0/2260278379 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x559a4329afb0 0x559a4329d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.202+0000 7f11c5ffb640 10 client.17580 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:43:11.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.202+0000 7f11e20a2640 1 --2- 172.21.15.86:0/2260278379 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x559a4329afb0 0x559a4329d3a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=66f57b1360f1fe58 server_cookie=c2242a54a020d18e in_seq=0 out_seq=0 2024-04-21T08:43:11.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.203+0000 7f11c5ffb640 1 -- 172.21.15.86:0/2260278379 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559a43181930 con 0x559a4329afb0 2024-04-21T08:43:11.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.203+0000 7f11c5ffb640 10 client.17580 handle_command_reply: tid=0 2024-04-21T08:43:11.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.203+0000 7f11e6fcb740 1 client.17580 shutdown 2024-04-21T08:43:11.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.203+0000 7f11e6fcb740 1 -- 172.21.15.86:0/2260278379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a43286e40 msgr2=0x559a4328ac30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.203+0000 7f11e6fcb740 1 --2- 172.21.15.86:0/2260278379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a43286e40 0x559a4328ac30 secure :-1 s=READY pgs=1136 cs=0 l=1 rev1=1 crypto rx=0x7f11d80058c0 tx=0x7f11d8005cf0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.203+0000 7f11e6fcb740 1 -- 172.21.15.86:0/2260278379 shutdown_connections 2024-04-21T08:43:11.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.203+0000 7f11e6fcb740 1 --2- 172.21.15.86:0/2260278379 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x559a4329afb0 0x559a4329d3a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.203+0000 7f11e6fcb740 1 --2- 172.21.15.86:0/2260278379 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a43286e40 0x559a4328ac30 unknown :-1 s=CLOSED pgs=1136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.203+0000 7f11e6fcb740 1 --2- 172.21.15.86:0/2260278379 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a4327dce0 0x559a4328a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.203+0000 7f11c5ffb640 0 client.17580 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:43:11.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.203+0000 7f11e6fcb740 1 --2- 172.21.15.86:0/2260278379 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4327d330 0x559a4328d290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.203+0000 7f11e6fcb740 1 -- 172.21.15.86:0/2260278379 >> 172.21.15.86:0/2260278379 conn(0x559a43277ba0 msgr2=0x559a432798f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:11.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.203+0000 7f11e6fcb740 1 -- 172.21.15.86:0/2260278379 shutdown_connections 2024-04-21T08:43:11.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.203+0000 7f11e6fcb740 1 -- 172.21.15.86:0/2260278379 wait complete. 2024-04-21T08:43:11.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.203+0000 7f11e6fcb740 20 client.17580 trim_cache size 0 max 16384 2024-04-21T08:43:11.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.216+0000 7f11e6fcb740 1 Processor -- start 2024-04-21T08:43:11.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.216+0000 7f11e6fcb740 1 -- start start 2024-04-21T08:43:11.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.216+0000 7f11e6fcb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4327d330 0x559a43296cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.216+0000 7f11e6fcb740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a4327dce0 0x559a43288230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.216+0000 7f11e6fcb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a43286e40 0x559a43289780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.216+0000 7f11e6fcb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f11dc1ad0d0 con 0x559a43286e40 2024-04-21T08:43:11.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.216+0000 7f11e6fcb740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f11dc1aec10 con 0x559a4327dce0 2024-04-21T08:43:11.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.216+0000 7f11e6fcb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f11dc1ad7a0 con 0x559a4327d330 2024-04-21T08:43:11.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.216+0000 7f11e28a3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a43286e40 0x559a43289780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.217+0000 7f11e28a3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a43286e40 0x559a43289780 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51036/0 (socket says 172.21.15.86:51036) 2024-04-21T08:43:11.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.217+0000 7f11e28a3640 1 -- 172.21.15.86:0/1936700376 learned_addr learned my addr 172.21.15.86:0/1936700376 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:11.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.217+0000 7f11e18a1640 1 --2- 172.21.15.86:0/1936700376 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a4327dce0 0x559a43288230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.217+0000 7f11e20a2640 1 --2- 172.21.15.86:0/1936700376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4327d330 0x559a43296cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.217+0000 7f11e28a3640 1 -- 172.21.15.86:0/1936700376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4327d330 msgr2=0x559a43296cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.217+0000 7f11e28a3640 1 --2- 172.21.15.86:0/1936700376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4327d330 0x559a43296cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.217+0000 7f11e28a3640 1 -- 172.21.15.86:0/1936700376 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a4327dce0 msgr2=0x559a43288230 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.217+0000 7f11e28a3640 1 --2- 172.21.15.86:0/1936700376 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a4327dce0 0x559a43288230 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.217+0000 7f11e28a3640 1 -- 172.21.15.86:0/1936700376 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559a4327ce50 con 0x559a43286e40 2024-04-21T08:43:11.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.217+0000 7f11e20a2640 1 --2- 172.21.15.86:0/1936700376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4327d330 0x559a43296cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:11.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.217+0000 7f11e28a3640 1 --2- 172.21.15.86:0/1936700376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a43286e40 0x559a43289780 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f11d801b7f0 tx=0x7f11d800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=20ed1d555a8d9ea4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:11.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.217+0000 7f11c7fff640 1 -- 172.21.15.86:0/1936700376 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d801b890 con 0x559a43286e40 2024-04-21T08:43:11.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.217+0000 7f11c7fff640 1 -- 172.21.15.86:0/1936700376 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11d801ccc0 con 0x559a43286e40 2024-04-21T08:43:11.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.217+0000 7f11c7fff640 1 -- 172.21.15.86:0/1936700376 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d80266d0 con 0x559a43286e40 2024-04-21T08:43:11.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.218+0000 7f11e6fcb740 1 -- 172.21.15.86:0/1936700376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a43286e40 msgr2=0x559a43289780 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.218+0000 7f11e6fcb740 1 --2- 172.21.15.86:0/1936700376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a43286e40 0x559a43289780 secure :-1 s=READY pgs=1137 cs=0 l=1 rev1=1 crypto rx=0x7f11d801b7f0 tx=0x7f11d800f730 comp rx=0 tx=0).stop 2024-04-21T08:43:11.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.218+0000 7f11e6fcb740 1 -- 172.21.15.86:0/1936700376 shutdown_connections 2024-04-21T08:43:11.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.218+0000 7f11e6fcb740 1 --2- 172.21.15.86:0/1936700376 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a43286e40 0x559a43289780 unknown :-1 s=CLOSED pgs=1137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.218+0000 7f11e6fcb740 1 --2- 172.21.15.86:0/1936700376 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a4327dce0 0x559a43288230 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.218+0000 7f11e6fcb740 1 --2- 172.21.15.86:0/1936700376 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4327d330 0x559a43296cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.218+0000 7f11e6fcb740 1 -- 172.21.15.86:0/1936700376 >> 172.21.15.86:0/1936700376 conn(0x559a43277ba0 msgr2=0x559a43276370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:11.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.218+0000 7f11e6fcb740 1 -- 172.21.15.86:0/1936700376 shutdown_connections 2024-04-21T08:43:11.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.218+0000 7f11e6fcb740 1 -- 172.21.15.86:0/1936700376 wait complete. 2024-04-21T08:43:11.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.218+0000 7f11e6fcb740 1 -- start start 2024-04-21T08:43:11.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.218+0000 7f11e6fcb740 1 Processor -- start 2024-04-21T08:43:11.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.218+0000 7f11e6fcb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a4327d330 0x559a432931d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.218+0000 7f11e6fcb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4327dce0 0x559a43293710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.218+0000 7f11e6fcb740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a43286e40 0x559a4328bc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.218+0000 7f11e6fcb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f11dc1ae540 con 0x559a4327d330 2024-04-21T08:43:11.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.218+0000 7f11e6fcb740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f11dc1ade70 con 0x559a43286e40 2024-04-21T08:43:11.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.218+0000 7f11e6fcb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f11dc1ab090 con 0x559a4327dce0 2024-04-21T08:43:11.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.218+0000 7f11e6fcb740 10 client.0 _ll_register_callbacks cb 0x7f11d800bca0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:11.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.218+0000 7f11e20a2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a4327d330 0x559a432931d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.218+0000 7f11e20a2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a4327d330 0x559a432931d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51044/0 (socket says 172.21.15.86:51044) 2024-04-21T08:43:11.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.218+0000 7f11e20a2640 1 -- 172.21.15.86:0/721352708 learned_addr learned my addr 172.21.15.86:0/721352708 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:11.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.218+0000 7f11e28a3640 1 --2- 172.21.15.86:0/721352708 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a43286e40 0x559a4328bc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.218+0000 7f11e18a1640 1 --2- 172.21.15.86:0/721352708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4327dce0 0x559a43293710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.219+0000 7f11e20a2640 1 -- 172.21.15.86:0/721352708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4327dce0 msgr2=0x559a43293710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.219+0000 7f11e20a2640 1 --2- 172.21.15.86:0/721352708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4327dce0 0x559a43293710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.219+0000 7f11e20a2640 1 -- 172.21.15.86:0/721352708 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a43286e40 msgr2=0x559a4328bc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.219+0000 7f11e20a2640 1 --2- 172.21.15.86:0/721352708 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a43286e40 0x559a4328bc80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.219+0000 7f11e20a2640 1 -- 172.21.15.86:0/721352708 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559a4328c460 con 0x559a4327d330 2024-04-21T08:43:11.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.219+0000 7f11e20a2640 1 --2- 172.21.15.86:0/721352708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a4327d330 0x559a432931d0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f11d402c9a0 tx=0x7f11d4005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=464481df5e43f622 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:11.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.219+0000 7f11c5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:11.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.219+0000 7f11c5ffb640 1 -- 172.21.15.86:0/721352708 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11d40beae0 con 0x559a4327d330 2024-04-21T08:43:11.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.219+0000 7f11c5ffb640 1 -- 172.21.15.86:0/721352708 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f11d40df0b0 con 0x559a4327d330 2024-04-21T08:43:11.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.219+0000 7f11e6fcb740 1 -- 172.21.15.86:0/721352708 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11d801ccc0 con 0x559a4327d330 2024-04-21T08:43:11.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.219+0000 7f11c5ffb640 1 -- 172.21.15.86:0/721352708 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f11d40c2e90 con 0x559a4327d330 2024-04-21T08:43:11.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.219+0000 7f11e6fcb740 10 client.17586 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:11.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.219+0000 7f11e6fcb740 1 -- 172.21.15.86:0/721352708 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559a43300ce0 con 0x559a4327d330 2024-04-21T08:43:11.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.219+0000 7f11c5ffb640 1 -- 172.21.15.86:0/721352708 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11d801ccc0 con 0x559a4327d330 2024-04-21T08:43:11.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.219+0000 7f11c5ffb640 1 -- 172.21.15.86:0/721352708 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f11d403c030 con 0x559a4327d330 2024-04-21T08:43:11.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.219+0000 7f11e6fcb740 10 client.17586 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:11.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.219+0000 7f11e6fcb740 10 client.17586 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:43:11.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.219+0000 7f11e6fcb740 20 client.17586 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:11.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.219+0000 7f11e6fcb740 1 --2- 172.21.15.86:0/721352708 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x559a433013c0 0x559a433037e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.219+0000 7f11e6fcb740 4 client.17586 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:43:11.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.219+0000 7f11e6fcb740 1 -- 172.21.15.86:0/721352708 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "damage ls"}) -- 0x559a431efc10 con 0x559a433013c0 2024-04-21T08:43:11.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.220+0000 7f11e18a1640 1 --2- 172.21.15.86:0/721352708 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x559a433013c0 0x559a433037e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.220+0000 7f11e18a1640 1 --2- 172.21.15.86:0/721352708 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x559a433013c0 0x559a433037e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=410f5367261d1a1b server_cookie=45c83d7da35a4cf4 in_seq=0 out_seq=0 2024-04-21T08:43:11.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.220+0000 7f11c5ffb640 10 client.17586 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:43:11.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.220+0000 7f11c5ffb640 1 -- 172.21.15.86:0/721352708 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x559a431efc10 con 0x559a433013c0 2024-04-21T08:43:11.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.220+0000 7f11c5ffb640 10 client.17586 handle_command_reply: tid=0 2024-04-21T08:43:11.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.220+0000 7f11e6fcb740 1 client.17586 shutdown 2024-04-21T08:43:11.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.220+0000 7f11e6fcb740 1 -- 172.21.15.86:0/721352708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a4327d330 msgr2=0x559a432931d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.220+0000 7f11e6fcb740 1 --2- 172.21.15.86:0/721352708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a4327d330 0x559a432931d0 secure :-1 s=READY pgs=1138 cs=0 l=1 rev1=1 crypto rx=0x7f11d402c9a0 tx=0x7f11d4005c30 comp rx=0 tx=0).stop 2024-04-21T08:43:11.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.220+0000 7f11e6fcb740 1 -- 172.21.15.86:0/721352708 shutdown_connections 2024-04-21T08:43:11.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.220+0000 7f11e6fcb740 1 --2- 172.21.15.86:0/721352708 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x559a433013c0 0x559a433037e0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.220+0000 7f11e6fcb740 1 --2- 172.21.15.86:0/721352708 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559a43286e40 0x559a4328bc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.220+0000 7f11e6fcb740 1 --2- 172.21.15.86:0/721352708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559a4327dce0 0x559a43293710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.220+0000 7f11c5ffb640 0 client.17586 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:43:11.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.220+0000 7f11e6fcb740 1 --2- 172.21.15.86:0/721352708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559a4327d330 0x559a432931d0 unknown :-1 s=CLOSED pgs=1138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.220+0000 7f11e6fcb740 1 -- 172.21.15.86:0/721352708 >> 172.21.15.86:0/721352708 conn(0x559a43277ba0 msgr2=0x559a43276c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:11.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.220+0000 7f11e6fcb740 1 -- 172.21.15.86:0/721352708 shutdown_connections 2024-04-21T08:43:11.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.220+0000 7f11e6fcb740 1 -- 172.21.15.86:0/721352708 wait complete. 2024-04-21T08:43:11.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.221+0000 7f11e6fcb740 20 client.17586 trim_cache size 0 max 16384 2024-04-21T08:43:11.228 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:43:11.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.221+0000 7f11e4306640 1 -- 172.21.15.86:0/1405486041 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f11c80a3d20 msgr2=0x7f11c80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.221+0000 7f11e4306640 1 --2- 172.21.15.86:0/1405486041 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f11c80a3d20 0x7f11c80a61e0 secure :-1 s=READY pgs=176 cs=0 l=1 rev1=1 crypto rx=0x7f11d8020000 tx=0x7f11d801d000 comp rx=0 tx=0).stop 2024-04-21T08:43:11.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.221+0000 7f11e4306640 1 -- 172.21.15.86:0/1405486041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11dc102190 msgr2=0x7f11dc19e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.221+0000 7f11e4306640 1 --2- 172.21.15.86:0/1405486041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11dc102190 0x7f11dc19e3f0 secure :-1 s=READY pgs=1134 cs=0 l=1 rev1=1 crypto rx=0x7f11d4005e30 tx=0x7f11d4034780 comp rx=0 tx=0).stop 2024-04-21T08:43:11.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.221+0000 7f11e4306640 1 -- 172.21.15.86:0/1405486041 shutdown_connections 2024-04-21T08:43:11.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.221+0000 7f11e4306640 1 --2- 172.21.15.86:0/1405486041 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f11c80a3d20 0x7f11c80a61e0 unknown :-1 s=CLOSED pgs=176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.221+0000 7f11e4306640 1 --2- 172.21.15.86:0/1405486041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11dc1985a0 0x7f11dc198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.221+0000 7f11e4306640 1 --2- 172.21.15.86:0/1405486041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11dc107260 0x7f11dc19e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.221+0000 7f11e4306640 1 --2- 172.21.15.86:0/1405486041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11dc102190 0x7f11dc19e3f0 unknown :-1 s=CLOSED pgs=1134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.221+0000 7f11e4306640 1 -- 172.21.15.86:0/1405486041 >> 172.21.15.86:0/1405486041 conn(0x7f11dc0793a0 msgr2=0x7f11dc100000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:11.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.221+0000 7f11e4306640 1 -- 172.21.15.86:0/1405486041 shutdown_connections 2024-04-21T08:43:11.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.221+0000 7f11e4306640 1 -- 172.21.15.86:0/1405486041 wait complete. 2024-04-21T08:43:11.231 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:43:11.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.332+0000 7fca6834a640 1 -- 172.21.15.86:0/3828660064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca60103af0 msgr2=0x7fca6010bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.332+0000 7fca6834a640 1 --2- 172.21.15.86:0/3828660064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca60103af0 0x7fca6010bfd0 secure :-1 s=READY pgs=1139 cs=0 l=1 rev1=1 crypto rx=0x7fca5c009900 tx=0x7fca5c02bd60 comp rx=0 tx=0).stop 2024-04-21T08:43:11.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.332+0000 7fca6834a640 1 -- 172.21.15.86:0/3828660064 shutdown_connections 2024-04-21T08:43:11.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.332+0000 7fca6834a640 1 --2- 172.21.15.86:0/3828660064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca6010c510 0x7fca6010e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.332+0000 7fca6834a640 1 --2- 172.21.15.86:0/3828660064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca60103af0 0x7fca6010bfd0 unknown :-1 s=CLOSED pgs=1139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.332+0000 7fca6834a640 1 --2- 172.21.15.86:0/3828660064 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca601031d0 0x7fca601035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.332+0000 7fca6834a640 1 -- 172.21.15.86:0/3828660064 >> 172.21.15.86:0/3828660064 conn(0x7fca60077f50 msgr2=0x7fca60078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:11.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.332+0000 7fca6834a640 1 -- 172.21.15.86:0/3828660064 shutdown_connections 2024-04-21T08:43:11.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.332+0000 7fca6834a640 1 -- 172.21.15.86:0/3828660064 wait complete. 2024-04-21T08:43:11.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.332+0000 7fca6834a640 1 Processor -- start 2024-04-21T08:43:11.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.332+0000 7fca6834a640 1 -- start start 2024-04-21T08:43:11.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.333+0000 7fca6834a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca601031d0 0x7fca6019e440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.333+0000 7fca6834a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca60103af0 0x7fca6019e980 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.333+0000 7fca6834a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca6010c510 0x7fca601985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.333+0000 7fca6834a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fca60112180 con 0x7fca6010c510 2024-04-21T08:43:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.333+0000 7fca6834a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fca60112000 con 0x7fca601031d0 2024-04-21T08:43:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.333+0000 7fca6834a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fca60112300 con 0x7fca60103af0 2024-04-21T08:43:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.333+0000 7fca668e7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca6010c510 0x7fca601985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.333+0000 7fca668e7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca6010c510 0x7fca601985f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51068/0 (socket says 172.21.15.86:51068) 2024-04-21T08:43:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.333+0000 7fca668e7640 1 -- 172.21.15.86:0/2505496390 learned_addr learned my addr 172.21.15.86:0/2505496390 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.333+0000 7fca658e5640 1 --2- 172.21.15.86:0/2505496390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca60103af0 0x7fca6019e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.333+0000 7fca660e6640 1 --2- 172.21.15.86:0/2505496390 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca601031d0 0x7fca6019e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.333+0000 7fca668e7640 1 -- 172.21.15.86:0/2505496390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca60103af0 msgr2=0x7fca6019e980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.333+0000 7fca668e7640 1 --2- 172.21.15.86:0/2505496390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca60103af0 0x7fca6019e980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.333+0000 7fca668e7640 1 -- 172.21.15.86:0/2505496390 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca601031d0 msgr2=0x7fca6019e440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.333+0000 7fca668e7640 1 --2- 172.21.15.86:0/2505496390 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca601031d0 0x7fca6019e440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.333+0000 7fca668e7640 1 -- 172.21.15.86:0/2505496390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca60198d70 con 0x7fca6010c510 2024-04-21T08:43:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.333+0000 7fca660e6640 1 --2- 172.21.15.86:0/2505496390 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca601031d0 0x7fca6019e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.333+0000 7fca668e7640 1 --2- 172.21.15.86:0/2505496390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca6010c510 0x7fca601985f0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fca5800eaa0 tx=0x7fca5800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.333+0000 7fca577fe640 1 -- 172.21.15.86:0/2505496390 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5800cdb0 con 0x7fca6010c510 2024-04-21T08:43:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.333+0000 7fca577fe640 1 -- 172.21.15.86:0/2505496390 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca58005020 con 0x7fca6010c510 2024-04-21T08:43:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.333+0000 7fca6834a640 1 -- 172.21.15.86:0/2505496390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca60199060 con 0x7fca6010c510 2024-04-21T08:43:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.333+0000 7fca577fe640 1 -- 172.21.15.86:0/2505496390 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca580116d0 con 0x7fca6010c510 2024-04-21T08:43:11.336 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.333+0000 7fca6834a640 1 -- 172.21.15.86:0/2505496390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca601a5300 con 0x7fca6010c510 2024-04-21T08:43:11.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.336+0000 7fca577fe640 1 -- 172.21.15.86:0/2505496390 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fca580051d0 con 0x7fca6010c510 2024-04-21T08:43:11.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.336+0000 7fca6b00f740 1 Processor -- start 2024-04-21T08:43:11.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.336+0000 7fca6b00f740 1 -- start start 2024-04-21T08:43:11.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.336+0000 7fca6b00f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b8e8e54d0 0x562b8e8e58b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.336+0000 7fca6b00f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b8e8e5e80 0x562b8e8ee430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.336+0000 7fca6b00f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8e8eea00 0x562b8e8f0e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.336+0000 7fca6b00f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562b8e84a1f0 con 0x562b8e8eea00 2024-04-21T08:43:11.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.336+0000 7fca6b00f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x562b8e7e74b0 con 0x562b8e8e5e80 2024-04-21T08:43:11.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.336+0000 7fca6b00f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562b8e8461b0 con 0x562b8e8e54d0 2024-04-21T08:43:11.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.336+0000 7fca658e5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8e8eea00 0x562b8e8f0e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.336+0000 7fca658e5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8e8eea00 0x562b8e8f0e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51080/0 (socket says 172.21.15.86:51080) 2024-04-21T08:43:11.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.336+0000 7fca658e5640 1 -- 172.21.15.86:0/3773568732 learned_addr learned my addr 172.21.15.86:0/3773568732 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:11.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.337+0000 7fca658e5640 1 --2- 172.21.15.86:0/3773568732 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b8e8e54d0 0x562b8e8e58b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.337+0000 7fca660e6640 1 --2- 172.21.15.86:0/3773568732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b8e8e5e80 0x562b8e8ee430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.337+0000 7fca577fe640 1 --2- 172.21.15.86:0/2505496390 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fca4c0a3d20 0x7fca4c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.337+0000 7fca658e5640 1 -- 172.21.15.86:0/3773568732 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b8e8e54d0 msgr2=0x562b8e8e58b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.337+0000 7fca658e5640 1 --2- 172.21.15.86:0/3773568732 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b8e8e54d0 0x562b8e8e58b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.337+0000 7fca658e5640 1 -- 172.21.15.86:0/3773568732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b8e8e5e80 msgr2=0x562b8e8ee430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.337+0000 7fca658e5640 1 --2- 172.21.15.86:0/3773568732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b8e8e5e80 0x562b8e8ee430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.337+0000 7fca658e5640 1 -- 172.21.15.86:0/3773568732 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b8e7d91b0 con 0x562b8e8eea00 2024-04-21T08:43:11.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.337+0000 7fca668e7640 1 --2- 172.21.15.86:0/2505496390 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fca4c0a3d20 0x7fca4c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.337+0000 7fca577fe640 1 -- 172.21.15.86:0/2505496390 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fca580d0dc0 con 0x7fca6010c510 2024-04-21T08:43:11.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.337+0000 7fca658e5640 1 --2- 172.21.15.86:0/3773568732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8e8eea00 0x562b8e8f0e50 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fca5c005940 tx=0x7fca5c0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1868526cf055f532 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:11.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.337+0000 7fca54bfe640 1 -- 172.21.15.86:0/3773568732 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c049070 con 0x562b8e8eea00 2024-04-21T08:43:11.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.337+0000 7fca54bfe640 1 -- 172.21.15.86:0/3773568732 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca5c0035f0 con 0x562b8e8eea00 2024-04-21T08:43:11.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.337+0000 7fca54bfe640 1 -- 172.21.15.86:0/3773568732 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c043690 con 0x562b8e8eea00 2024-04-21T08:43:11.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.338+0000 7fca6b00f740 1 -- 172.21.15.86:0/3773568732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8e8eea00 msgr2=0x562b8e8f0e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.338+0000 7fca668e7640 1 --2- 172.21.15.86:0/2505496390 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fca4c0a3d20 0x7fca4c0a61e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fca5809c6b0 tx=0x7fca5806e3f0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:11.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.338+0000 7fca6b00f740 1 --2- 172.21.15.86:0/3773568732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8e8eea00 0x562b8e8f0e50 secure :-1 s=READY pgs=1141 cs=0 l=1 rev1=1 crypto rx=0x7fca5c005940 tx=0x7fca5c0058b0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.338+0000 7fca6b00f740 1 -- 172.21.15.86:0/3773568732 shutdown_connections 2024-04-21T08:43:11.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.338+0000 7fca6b00f740 1 --2- 172.21.15.86:0/3773568732 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8e8eea00 0x562b8e8f0e50 unknown :-1 s=CLOSED pgs=1141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.338+0000 7fca6b00f740 1 --2- 172.21.15.86:0/3773568732 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b8e8e5e80 0x562b8e8ee430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.338+0000 7fca6b00f740 1 --2- 172.21.15.86:0/3773568732 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b8e8e54d0 0x562b8e8e58b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.338+0000 7fca6b00f740 1 -- 172.21.15.86:0/3773568732 >> 172.21.15.86:0/3773568732 conn(0x562b8e8dfba0 msgr2=0x562b8e8dff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:11.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.338+0000 7fca6b00f740 1 -- 172.21.15.86:0/3773568732 shutdown_connections 2024-04-21T08:43:11.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.338+0000 7fca6b00f740 1 -- 172.21.15.86:0/3773568732 wait complete. 2024-04-21T08:43:11.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.338+0000 7fca6b00f740 1 -- start start 2024-04-21T08:43:11.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.338+0000 7fca6b00f740 1 Processor -- start 2024-04-21T08:43:11.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.339+0000 7fca6b00f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8e8e54d0 0x562b8e8ee310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.339+0000 7fca6b00f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b8e8e5e80 0x562b8e8eb6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.339+0000 7fca6b00f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b8e8eea00 0x562b8e8ebc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.339+0000 7fca6b00f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562b8e8f1910 con 0x562b8e8e54d0 2024-04-21T08:43:11.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.339+0000 7fca6b00f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x562b8e803500 con 0x562b8e8eea00 2024-04-21T08:43:11.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.339+0000 7fca6b00f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562b8e8f1a90 con 0x562b8e8e5e80 2024-04-21T08:43:11.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.339+0000 7fca658e5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8e8e54d0 0x562b8e8ee310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.339+0000 7fca6b00f740 10 client.0 _ll_register_callbacks cb 0x562b8e7d1b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:11.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.339+0000 7fca658e5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8e8e54d0 0x562b8e8ee310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51082/0 (socket says 172.21.15.86:51082) 2024-04-21T08:43:11.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.339+0000 7fca658e5640 1 -- 172.21.15.86:0/1364888894 learned_addr learned my addr 172.21.15.86:0/1364888894 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:11.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.339+0000 7fca660e6640 1 --2- 172.21.15.86:0/1364888894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b8e8e5e80 0x562b8e8eb6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.339+0000 7fca658e5640 1 -- 172.21.15.86:0/1364888894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b8e8e5e80 msgr2=0x562b8e8eb6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.339+0000 7fca658e5640 1 --2- 172.21.15.86:0/1364888894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b8e8e5e80 0x562b8e8eb6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.339+0000 7fca658e5640 1 -- 172.21.15.86:0/1364888894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b8e8eea00 msgr2=0x562b8e8ebc40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.339+0000 7fca658e5640 1 --2- 172.21.15.86:0/1364888894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b8e8eea00 0x562b8e8ebc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.339+0000 7fca658e5640 1 -- 172.21.15.86:0/1364888894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b8e55c3f0 con 0x562b8e8e54d0 2024-04-21T08:43:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.339+0000 7fca660e6640 1 --2- 172.21.15.86:0/1364888894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b8e8e5e80 0x562b8e8eb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.339+0000 7fca658e5640 1 --2- 172.21.15.86:0/1364888894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8e8e54d0 0x562b8e8ee310 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fca5c001eb0 tx=0x7fca5c02de80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=959ac52af1a4646e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.339+0000 7fca49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.339+0000 7fca49ffb640 1 -- 172.21.15.86:0/1364888894 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c054060 con 0x562b8e8e54d0 2024-04-21T08:43:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.339+0000 7fca49ffb640 1 -- 172.21.15.86:0/1364888894 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fca5c0039a0 con 0x562b8e8e54d0 2024-04-21T08:43:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.339+0000 7fca6b00f740 1 -- 172.21.15.86:0/1364888894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca5c0035f0 con 0x562b8e8e54d0 2024-04-21T08:43:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.340+0000 7fca49ffb640 1 -- 172.21.15.86:0/1364888894 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fca5c049040 con 0x562b8e8e54d0 2024-04-21T08:43:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.340+0000 7fca6b00f740 10 client.17598 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.340+0000 7fca6b00f740 1 -- 172.21.15.86:0/1364888894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562b8e6267d0 con 0x562b8e8e54d0 2024-04-21T08:43:11.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.340+0000 7fca49ffb640 1 -- 172.21.15.86:0/1364888894 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca5c0035f0 con 0x562b8e8e54d0 2024-04-21T08:43:11.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.340+0000 7fca49ffb640 1 -- 172.21.15.86:0/1364888894 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fca5c040040 con 0x562b8e8e54d0 2024-04-21T08:43:11.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.340+0000 7fca6b00f740 10 client.17598 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:11.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.340+0000 7fca6b00f740 10 client.17598 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:43:11.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.340+0000 7fca6b00f740 20 client.17598 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:11.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.340+0000 7fca6b00f740 1 --2- 172.21.15.86:0/1364888894 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x562b8e8feaa0 0x562b8e900e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.340+0000 7fca6b00f740 4 client.17598 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:11.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.340+0000 7fca6b00f740 1 -- 172.21.15.86:0/1364888894 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562b8e7e9930 con 0x562b8e8feaa0 2024-04-21T08:43:11.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.340+0000 7fca660e6640 1 --2- 172.21.15.86:0/1364888894 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x562b8e8feaa0 0x562b8e900e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.341+0000 7fca49ffb640 10 client.17598 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:43:11.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.341+0000 7fca660e6640 1 --2- 172.21.15.86:0/1364888894 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x562b8e8feaa0 0x562b8e900e90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c1f4a3204033deb0 server_cookie=e9b008fd217fd1f9 in_seq=0 out_seq=0 2024-04-21T08:43:11.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.341+0000 7fca49ffb640 1 -- 172.21.15.86:0/1364888894 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562b8e7e9930 con 0x562b8e8feaa0 2024-04-21T08:43:11.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.341+0000 7fca49ffb640 10 client.17598 handle_command_reply: tid=0 2024-04-21T08:43:11.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.341+0000 7fca6b00f740 1 client.17598 shutdown 2024-04-21T08:43:11.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.341+0000 7fca6b00f740 1 -- 172.21.15.86:0/1364888894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8e8e54d0 msgr2=0x562b8e8ee310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.341+0000 7fca6b00f740 1 --2- 172.21.15.86:0/1364888894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8e8e54d0 0x562b8e8ee310 secure :-1 s=READY pgs=1142 cs=0 l=1 rev1=1 crypto rx=0x7fca5c001eb0 tx=0x7fca5c02de80 comp rx=0 tx=0).stop 2024-04-21T08:43:11.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.341+0000 7fca6b00f740 1 -- 172.21.15.86:0/1364888894 shutdown_connections 2024-04-21T08:43:11.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.341+0000 7fca6b00f740 1 --2- 172.21.15.86:0/1364888894 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x562b8e8feaa0 0x562b8e900e90 crc :-1 s=READY pgs=49 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.341+0000 7fca6b00f740 1 --2- 172.21.15.86:0/1364888894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b8e8eea00 0x562b8e8ebc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.341+0000 7fca6b00f740 1 --2- 172.21.15.86:0/1364888894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b8e8e5e80 0x562b8e8eb6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.341+0000 7fca49ffb640 0 client.17598 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:43:11.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.341+0000 7fca6b00f740 1 --2- 172.21.15.86:0/1364888894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8e8e54d0 0x562b8e8ee310 unknown :-1 s=CLOSED pgs=1142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.341+0000 7fca6b00f740 1 -- 172.21.15.86:0/1364888894 >> 172.21.15.86:0/1364888894 conn(0x562b8e8dfba0 msgr2=0x562b8e8e18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:11.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.341+0000 7fca6b00f740 1 -- 172.21.15.86:0/1364888894 shutdown_connections 2024-04-21T08:43:11.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.341+0000 7fca6b00f740 1 -- 172.21.15.86:0/1364888894 wait complete. 2024-04-21T08:43:11.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.342+0000 7fca6b00f740 20 client.17598 trim_cache size 0 max 16384 2024-04-21T08:43:11.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.354+0000 7fca6b00f740 1 Processor -- start 2024-04-21T08:43:11.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.354+0000 7fca6b00f740 1 -- start start 2024-04-21T08:43:11.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.355+0000 7fca6b00f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8e8e54d0 0x562b8e8eddd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.355+0000 7fca6b00f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b8e8e5e80 0x562b8e8ee310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.355+0000 7fca6b00f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b8e8eea00 0x562b8e8e8900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.355+0000 7fca6b00f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fca601af2a0 con 0x562b8e8e54d0 2024-04-21T08:43:11.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.355+0000 7fca6b00f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fca601b0de0 con 0x562b8e8eea00 2024-04-21T08:43:11.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.355+0000 7fca6b00f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fca601af970 con 0x562b8e8e5e80 2024-04-21T08:43:11.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.355+0000 7fca658e5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8e8e54d0 0x562b8e8eddd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.355+0000 7fca658e5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8e8e54d0 0x562b8e8eddd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51094/0 (socket says 172.21.15.86:51094) 2024-04-21T08:43:11.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.355+0000 7fca658e5640 1 -- 172.21.15.86:0/1647323717 learned_addr learned my addr 172.21.15.86:0/1647323717 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:11.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.355+0000 7fca658e5640 1 -- 172.21.15.86:0/1647323717 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b8e8e5e80 msgr2=0x562b8e8ee310 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:43:11.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.355+0000 7fca658e5640 1 --2- 172.21.15.86:0/1647323717 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b8e8e5e80 0x562b8e8ee310 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.355+0000 7fca658e5640 1 -- 172.21.15.86:0/1647323717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b8e8eea00 msgr2=0x562b8e8e8900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.355+0000 7fca658e5640 1 --2- 172.21.15.86:0/1647323717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b8e8eea00 0x562b8e8e8900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.355+0000 7fca658e5640 1 -- 172.21.15.86:0/1647323717 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562b8e8f1850 con 0x562b8e8e54d0 2024-04-21T08:43:11.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.355+0000 7fca658e5640 1 --2- 172.21.15.86:0/1647323717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8e8e54d0 0x562b8e8eddd0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fca5c006010 tx=0x7fca5c02ddd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a675b0e457e1aa server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:11.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.355+0000 7fca4bfff640 1 -- 172.21.15.86:0/1647323717 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c054040 con 0x562b8e8e54d0 2024-04-21T08:43:11.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.355+0000 7fca4bfff640 1 -- 172.21.15.86:0/1647323717 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca5c02c700 con 0x562b8e8e54d0 2024-04-21T08:43:11.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.355+0000 7fca4bfff640 1 -- 172.21.15.86:0/1647323717 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c003850 con 0x562b8e8e54d0 2024-04-21T08:43:11.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.356+0000 7fca6b00f740 1 -- 172.21.15.86:0/1647323717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8e8e54d0 msgr2=0x562b8e8eddd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.356+0000 7fca6b00f740 1 --2- 172.21.15.86:0/1647323717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8e8e54d0 0x562b8e8eddd0 secure :-1 s=READY pgs=1143 cs=0 l=1 rev1=1 crypto rx=0x7fca5c006010 tx=0x7fca5c02ddd0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.356+0000 7fca6b00f740 1 -- 172.21.15.86:0/1647323717 shutdown_connections 2024-04-21T08:43:11.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.356+0000 7fca6b00f740 1 --2- 172.21.15.86:0/1647323717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b8e8eea00 0x562b8e8e8900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.356+0000 7fca6b00f740 1 --2- 172.21.15.86:0/1647323717 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b8e8e5e80 0x562b8e8ee310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.356+0000 7fca6b00f740 1 --2- 172.21.15.86:0/1647323717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8e8e54d0 0x562b8e8eddd0 unknown :-1 s=CLOSED pgs=1143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.356+0000 7fca6b00f740 1 -- 172.21.15.86:0/1647323717 >> 172.21.15.86:0/1647323717 conn(0x562b8e8dfba0 msgr2=0x562b8e8dec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:11.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.356+0000 7fca6b00f740 1 -- 172.21.15.86:0/1647323717 shutdown_connections 2024-04-21T08:43:11.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.356+0000 7fca6b00f740 1 -- 172.21.15.86:0/1647323717 wait complete. 2024-04-21T08:43:11.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.356+0000 7fca6b00f740 1 -- start start 2024-04-21T08:43:11.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.356+0000 7fca6b00f740 1 Processor -- start 2024-04-21T08:43:11.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.356+0000 7fca6b00f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8e8e54d0 0x562b8e901010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.356+0000 7fca6b00f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b8e8e5e80 0x562b8e901550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.356+0000 7fca6b00f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b8e8eea00 0x562b8e962c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.356+0000 7fca6b00f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fca601b0710 con 0x562b8e8e54d0 2024-04-21T08:43:11.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.356+0000 7fca6b00f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fca601b0040 con 0x562b8e8e5e80 2024-04-21T08:43:11.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.356+0000 7fca6b00f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fca601ad260 con 0x562b8e8eea00 2024-04-21T08:43:11.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.357+0000 7fca6b00f740 10 client.0 _ll_register_callbacks cb 0x7fca5c004120 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:11.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.357+0000 7fca658e5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8e8e54d0 0x562b8e901010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.357+0000 7fca658e5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8e8e54d0 0x562b8e901010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51100/0 (socket says 172.21.15.86:51100) 2024-04-21T08:43:11.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.357+0000 7fca658e5640 1 -- 172.21.15.86:0/403081425 learned_addr learned my addr 172.21.15.86:0/403081425 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:11.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.357+0000 7fca658e5640 1 -- 172.21.15.86:0/403081425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b8e8eea00 msgr2=0x562b8e962c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.357+0000 7fca658e5640 1 --2- 172.21.15.86:0/403081425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b8e8eea00 0x562b8e962c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.357+0000 7fca658e5640 1 -- 172.21.15.86:0/403081425 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b8e8e5e80 msgr2=0x562b8e901550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.357+0000 7fca658e5640 1 --2- 172.21.15.86:0/403081425 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b8e8e5e80 0x562b8e901550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.357+0000 7fca658e5640 1 -- 172.21.15.86:0/403081425 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562b8e9631b0 con 0x562b8e8e54d0 2024-04-21T08:43:11.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.357+0000 7fca658e5640 1 --2- 172.21.15.86:0/403081425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8e8e54d0 0x562b8e901010 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fca5c0096c0 tx=0x7fca5c043a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=681aec48deb3197c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:11.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.357+0000 7fca49ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:11.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.357+0000 7fca49ffb640 1 -- 172.21.15.86:0/403081425 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca5c063070 con 0x562b8e8e54d0 2024-04-21T08:43:11.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.357+0000 7fca49ffb640 1 -- 172.21.15.86:0/403081425 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fca5c003c30 con 0x562b8e8e54d0 2024-04-21T08:43:11.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.357+0000 7fca6b00f740 1 -- 172.21.15.86:0/403081425 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca5c02c700 con 0x562b8e8e54d0 2024-04-21T08:43:11.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.357+0000 7fca49ffb640 1 -- 172.21.15.86:0/403081425 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fca5c054040 con 0x562b8e8e54d0 2024-04-21T08:43:11.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.357+0000 7fca6b00f740 10 client.17604 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:11.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.357+0000 7fca6b00f740 1 -- 172.21.15.86:0/403081425 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562b8e9644b0 con 0x562b8e8e54d0 2024-04-21T08:43:11.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.357+0000 7fca49ffb640 1 -- 172.21.15.86:0/403081425 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca5c02c700 con 0x562b8e8e54d0 2024-04-21T08:43:11.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.358+0000 7fca49ffb640 1 -- 172.21.15.86:0/403081425 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fca5c034b50 con 0x562b8e8e54d0 2024-04-21T08:43:11.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.358+0000 7fca6b00f740 10 client.17604 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:11.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.358+0000 7fca6b00f740 10 client.17604 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:43:11.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.358+0000 7fca6b00f740 20 client.17604 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:11.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.358+0000 7fca6b00f740 1 --2- 172.21.15.86:0/403081425 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x562b8e964980 0x562b8e966d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.358+0000 7fca6b00f740 4 client.17604 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:43:11.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.358+0000 7fca6b00f740 1 -- 172.21.15.86:0/403081425 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x562b8e857c10 con 0x562b8e964980 2024-04-21T08:43:11.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.358+0000 7fca660e6640 1 --2- 172.21.15.86:0/403081425 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x562b8e964980 0x562b8e966d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.358+0000 7fca49ffb640 10 client.17604 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:43:11.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.358+0000 7fca660e6640 1 --2- 172.21.15.86:0/403081425 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x562b8e964980 0x562b8e966d70 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=57fbf261dbdd972d server_cookie=eda22d182df816d3 in_seq=0 out_seq=0 2024-04-21T08:43:11.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.358+0000 7fca49ffb640 1 -- 172.21.15.86:0/403081425 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x562b8e857c10 con 0x562b8e964980 2024-04-21T08:43:11.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.358+0000 7fca49ffb640 10 client.17604 handle_command_reply: tid=0 2024-04-21T08:43:11.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.358+0000 7fca6b00f740 1 client.17604 shutdown 2024-04-21T08:43:11.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.359+0000 7fca6b00f740 1 -- 172.21.15.86:0/403081425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8e8e54d0 msgr2=0x562b8e901010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.359+0000 7fca6b00f740 1 --2- 172.21.15.86:0/403081425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8e8e54d0 0x562b8e901010 secure :-1 s=READY pgs=1144 cs=0 l=1 rev1=1 crypto rx=0x7fca5c0096c0 tx=0x7fca5c043a20 comp rx=0 tx=0).stop 2024-04-21T08:43:11.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.359+0000 7fca6b00f740 1 -- 172.21.15.86:0/403081425 shutdown_connections 2024-04-21T08:43:11.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.359+0000 7fca6b00f740 1 --2- 172.21.15.86:0/403081425 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x562b8e964980 0x562b8e966d70 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.359+0000 7fca6b00f740 1 --2- 172.21.15.86:0/403081425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562b8e8eea00 0x562b8e962c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.359+0000 7fca6b00f740 1 --2- 172.21.15.86:0/403081425 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562b8e8e5e80 0x562b8e901550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.359+0000 7fca49ffb640 0 client.17604 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:43:11.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.359+0000 7fca6b00f740 1 --2- 172.21.15.86:0/403081425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562b8e8e54d0 0x562b8e901010 unknown :-1 s=CLOSED pgs=1144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.359+0000 7fca6b00f740 1 -- 172.21.15.86:0/403081425 >> 172.21.15.86:0/403081425 conn(0x562b8e8dfba0 msgr2=0x562b8e8f9d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:11.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.359+0000 7fca6b00f740 1 -- 172.21.15.86:0/403081425 shutdown_connections 2024-04-21T08:43:11.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.359+0000 7fca6b00f740 1 -- 172.21.15.86:0/403081425 wait complete. 2024-04-21T08:43:11.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.359+0000 7fca6b00f740 20 client.17604 trim_cache size 0 max 16384 2024-04-21T08:43:11.365 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:43:11.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.359+0000 7fca6834a640 1 -- 172.21.15.86:0/2505496390 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fca4c0a3d20 msgr2=0x7fca4c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.359+0000 7fca6834a640 1 --2- 172.21.15.86:0/2505496390 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fca4c0a3d20 0x7fca4c0a61e0 secure :-1 s=READY pgs=177 cs=0 l=1 rev1=1 crypto rx=0x7fca5809c6b0 tx=0x7fca5806e3f0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.359+0000 7fca6834a640 1 -- 172.21.15.86:0/2505496390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca6010c510 msgr2=0x7fca601985f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.359+0000 7fca6834a640 1 --2- 172.21.15.86:0/2505496390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca6010c510 0x7fca601985f0 secure :-1 s=READY pgs=1140 cs=0 l=1 rev1=1 crypto rx=0x7fca5800eaa0 tx=0x7fca5800ef60 comp rx=0 tx=0).stop 2024-04-21T08:43:11.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.359+0000 7fca6834a640 1 -- 172.21.15.86:0/2505496390 shutdown_connections 2024-04-21T08:43:11.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.359+0000 7fca6834a640 1 --2- 172.21.15.86:0/2505496390 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fca4c0a3d20 0x7fca4c0a61e0 unknown :-1 s=CLOSED pgs=177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.359+0000 7fca6834a640 1 --2- 172.21.15.86:0/2505496390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca6010c510 0x7fca601985f0 unknown :-1 s=CLOSED pgs=1140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.359+0000 7fca6834a640 1 --2- 172.21.15.86:0/2505496390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca60103af0 0x7fca6019e980 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.359+0000 7fca6834a640 1 --2- 172.21.15.86:0/2505496390 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca601031d0 0x7fca6019e440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.359+0000 7fca6834a640 1 -- 172.21.15.86:0/2505496390 >> 172.21.15.86:0/2505496390 conn(0x7fca60077f50 msgr2=0x7fca600ff380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:11.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.359+0000 7fca6834a640 1 -- 172.21.15.86:0/2505496390 shutdown_connections 2024-04-21T08:43:11.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.359+0000 7fca6834a640 1 -- 172.21.15.86:0/2505496390 wait complete. 2024-04-21T08:43:11.369 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:43:11.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.471+0000 7f84dfe25640 1 -- 172.21.15.86:0/3104031398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84d8102b40 msgr2=0x7f84d8106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.471+0000 7f84dfe25640 1 --2- 172.21.15.86:0/3104031398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84d8102b40 0x7f84d8106b80 secure :-1 s=READY pgs=1145 cs=0 l=1 rev1=1 crypto rx=0x7f84d4009900 tx=0x7f84d402bc30 comp rx=0 tx=0).stop 2024-04-21T08:43:11.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.471+0000 7f84dfe25640 1 -- 172.21.15.86:0/3104031398 shutdown_connections 2024-04-21T08:43:11.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.471+0000 7f84dfe25640 1 --2- 172.21.15.86:0/3104031398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84d8107260 0x7f84d810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.471+0000 7f84dfe25640 1 --2- 172.21.15.86:0/3104031398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84d8102b40 0x7f84d8106b80 unknown :-1 s=CLOSED pgs=1145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.471+0000 7f84dfe25640 1 --2- 172.21.15.86:0/3104031398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84d8102190 0x7f84d8102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.471+0000 7f84dfe25640 1 -- 172.21.15.86:0/3104031398 >> 172.21.15.86:0/3104031398 conn(0x7f84d80793a0 msgr2=0x7f84d80797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:11.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.471+0000 7f84dfe25640 1 -- 172.21.15.86:0/3104031398 shutdown_connections 2024-04-21T08:43:11.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.471+0000 7f84dfe25640 1 -- 172.21.15.86:0/3104031398 wait complete. 2024-04-21T08:43:11.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.471+0000 7f84dfe25640 1 Processor -- start 2024-04-21T08:43:11.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.472+0000 7f84dfe25640 1 -- start start 2024-04-21T08:43:11.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.472+0000 7f84dfe25640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84d8102190 0x7f84d81a0680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.472+0000 7f84dfe25640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84d8107260 0x7f84d81a0bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.472+0000 7f84dfe25640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84d819a7f0 0x7f84d819aca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.472+0000 7f84dfe25640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f84d81121a0 con 0x7f84d819a7f0 2024-04-21T08:43:11.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.472+0000 7f84dfe25640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f84d8112020 con 0x7f84d8102190 2024-04-21T08:43:11.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.472+0000 7f84dfe25640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f84d8112320 con 0x7f84d8107260 2024-04-21T08:43:11.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.472+0000 7f84de3c2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84d819a7f0 0x7f84d819aca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.472+0000 7f84de3c2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84d819a7f0 0x7f84d819aca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51114/0 (socket says 172.21.15.86:51114) 2024-04-21T08:43:11.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.472+0000 7f84de3c2640 1 -- 172.21.15.86:0/119523546 learned_addr learned my addr 172.21.15.86:0/119523546 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:11.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.472+0000 7f84dd3c0640 1 --2- 172.21.15.86:0/119523546 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84d8107260 0x7f84d81a0bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.472+0000 7f84ddbc1640 1 --2- 172.21.15.86:0/119523546 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84d8102190 0x7f84d81a0680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.472+0000 7f84de3c2640 1 -- 172.21.15.86:0/119523546 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84d8107260 msgr2=0x7f84d81a0bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.472+0000 7f84de3c2640 1 --2- 172.21.15.86:0/119523546 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84d8107260 0x7f84d81a0bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.472+0000 7f84de3c2640 1 -- 172.21.15.86:0/119523546 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84d8102190 msgr2=0x7f84d81a0680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.472+0000 7f84de3c2640 1 --2- 172.21.15.86:0/119523546 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84d8102190 0x7f84d81a0680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.472+0000 7f84de3c2640 1 -- 172.21.15.86:0/119523546 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d819b380 con 0x7f84d819a7f0 2024-04-21T08:43:11.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.472+0000 7f84dd3c0640 1 --2- 172.21.15.86:0/119523546 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84d8107260 0x7f84d81a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:11.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.472+0000 7f84de3c2640 1 --2- 172.21.15.86:0/119523546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84d819a7f0 0x7f84d819aca0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f84d000eaa0 tx=0x7f84d000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:11.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.472+0000 7f84ceffd640 1 -- 172.21.15.86:0/119523546 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d000cdb0 con 0x7f84d819a7f0 2024-04-21T08:43:11.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.472+0000 7f84ceffd640 1 -- 172.21.15.86:0/119523546 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84d0005020 con 0x7f84d819a7f0 2024-04-21T08:43:11.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.472+0000 7f84ceffd640 1 -- 172.21.15.86:0/119523546 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d00116d0 con 0x7f84d819a7f0 2024-04-21T08:43:11.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.472+0000 7f84dfe25640 1 -- 172.21.15.86:0/119523546 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84d819b670 con 0x7f84d819a7f0 2024-04-21T08:43:11.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.473+0000 7f84dfe25640 1 -- 172.21.15.86:0/119523546 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84d81a7800 con 0x7f84d819a7f0 2024-04-21T08:43:11.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.475+0000 7f84ceffd640 1 -- 172.21.15.86:0/119523546 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f84d00051d0 con 0x7f84d819a7f0 2024-04-21T08:43:11.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.475+0000 7f84e2a15740 1 Processor -- start 2024-04-21T08:43:11.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.475+0000 7f84e2a15740 1 -- start start 2024-04-21T08:43:11.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.475+0000 7f84e2a15740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564de9b504d0 0x564de9b508b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.475+0000 7f84e2a15740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564de9b50e80 0x564de9b59430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.475+0000 7f84e2a15740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564de9b59a00 0x564de9b5be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.475+0000 7f84e2a15740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564de9ab51f0 con 0x564de9b504d0 2024-04-21T08:43:11.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.475+0000 7f84e2a15740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564de9a524b0 con 0x564de9b59a00 2024-04-21T08:43:11.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.475+0000 7f84e2a15740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564de9ab11b0 con 0x564de9b50e80 2024-04-21T08:43:11.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.475+0000 7f84dd3c0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564de9b504d0 0x564de9b508b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.476+0000 7f84dd3c0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564de9b504d0 0x564de9b508b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51130/0 (socket says 172.21.15.86:51130) 2024-04-21T08:43:11.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.476+0000 7f84dd3c0640 1 -- 172.21.15.86:0/1391320176 learned_addr learned my addr 172.21.15.86:0/1391320176 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:11.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.476+0000 7f84ceffd640 1 --2- 172.21.15.86:0/119523546 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f84c40a3d20 0x7f84c40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.476+0000 7f84ddbc1640 1 --2- 172.21.15.86:0/1391320176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564de9b50e80 0x564de9b59430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.476+0000 7f84dd3c0640 1 -- 172.21.15.86:0/1391320176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564de9b50e80 msgr2=0x564de9b59430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.476+0000 7f84dd3c0640 1 --2- 172.21.15.86:0/1391320176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564de9b50e80 0x564de9b59430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.476+0000 7f84dd3c0640 1 -- 172.21.15.86:0/1391320176 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564de9b59a00 msgr2=0x564de9b5be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.476+0000 7f84dd3c0640 1 --2- 172.21.15.86:0/1391320176 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564de9b59a00 0x564de9b5be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.476+0000 7f84dd3c0640 1 -- 172.21.15.86:0/1391320176 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564de9a441b0 con 0x564de9b504d0 2024-04-21T08:43:11.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.476+0000 7f84de3c2640 1 --2- 172.21.15.86:0/119523546 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f84c40a3d20 0x7f84c40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.476+0000 7f84ceffd640 1 -- 172.21.15.86:0/119523546 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f84d00d0e00 con 0x7f84d819a7f0 2024-04-21T08:43:11.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.476+0000 7f84dd3c0640 1 --2- 172.21.15.86:0/1391320176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564de9b504d0 0x564de9b508b0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f84d4005c40 tx=0x7f84d4005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ebf6dcb1886a995 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:11.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.476+0000 7f84c3fff640 1 -- 172.21.15.86:0/1391320176 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d4043020 con 0x564de9b504d0 2024-04-21T08:43:11.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.476+0000 7f84de3c2640 1 --2- 172.21.15.86:0/119523546 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f84c40a3d20 0x7f84c40a61e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f84d0098e50 tx=0x7f84d0087040 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:11.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.476+0000 7f84c3fff640 1 -- 172.21.15.86:0/1391320176 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84d402cc90 con 0x564de9b504d0 2024-04-21T08:43:11.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.476+0000 7f84c3fff640 1 -- 172.21.15.86:0/1391320176 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d40046c0 con 0x564de9b504d0 2024-04-21T08:43:11.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.477+0000 7f84e2a15740 1 -- 172.21.15.86:0/1391320176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564de9b504d0 msgr2=0x564de9b508b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.477+0000 7f84e2a15740 1 --2- 172.21.15.86:0/1391320176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564de9b504d0 0x564de9b508b0 secure :-1 s=READY pgs=1147 cs=0 l=1 rev1=1 crypto rx=0x7f84d4005c40 tx=0x7f84d4005e80 comp rx=0 tx=0).stop 2024-04-21T08:43:11.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.477+0000 7f84e2a15740 1 -- 172.21.15.86:0/1391320176 shutdown_connections 2024-04-21T08:43:11.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.477+0000 7f84e2a15740 1 --2- 172.21.15.86:0/1391320176 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564de9b59a00 0x564de9b5be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.477+0000 7f84e2a15740 1 --2- 172.21.15.86:0/1391320176 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564de9b50e80 0x564de9b59430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.477+0000 7f84e2a15740 1 --2- 172.21.15.86:0/1391320176 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564de9b504d0 0x564de9b508b0 unknown :-1 s=CLOSED pgs=1147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.477+0000 7f84e2a15740 1 -- 172.21.15.86:0/1391320176 >> 172.21.15.86:0/1391320176 conn(0x564de9b4aba0 msgr2=0x564de9b4af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:11.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.477+0000 7f84e2a15740 1 -- 172.21.15.86:0/1391320176 shutdown_connections 2024-04-21T08:43:11.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.477+0000 7f84e2a15740 1 -- 172.21.15.86:0/1391320176 wait complete. 2024-04-21T08:43:11.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.477+0000 7f84e2a15740 1 -- start start 2024-04-21T08:43:11.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.477+0000 7f84e2a15740 1 Processor -- start 2024-04-21T08:43:11.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.478+0000 7f84e2a15740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564de9b504d0 0x564de9b59340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.478+0000 7f84e2a15740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564de9b50e80 0x564de9b566d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.478+0000 7f84e2a15740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564de9b59a00 0x564de9b56ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.478+0000 7f84e2a15740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564de9b5c910 con 0x564de9b50e80 2024-04-21T08:43:11.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.478+0000 7f84e2a15740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564de9a6e500 con 0x564de9b59a00 2024-04-21T08:43:11.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.478+0000 7f84e2a15740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564de9b5ca90 con 0x564de9b504d0 2024-04-21T08:43:11.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.478+0000 7f84e2a15740 10 client.0 _ll_register_callbacks cb 0x564de9a3cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:11.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.478+0000 7f84ddbc1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564de9b50e80 0x564de9b566d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.478+0000 7f84ddbc1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564de9b50e80 0x564de9b566d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51140/0 (socket says 172.21.15.86:51140) 2024-04-21T08:43:11.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.478+0000 7f84ddbc1640 1 -- 172.21.15.86:0/2854646585 learned_addr learned my addr 172.21.15.86:0/2854646585 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:11.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.478+0000 7f84dd3c0640 1 --2- 172.21.15.86:0/2854646585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564de9b59a00 0x564de9b56ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.478+0000 7f84ddbc1640 1 -- 172.21.15.86:0/2854646585 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564de9b504d0 msgr2=0x564de9b59340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.478+0000 7f84ddbc1640 1 --2- 172.21.15.86:0/2854646585 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564de9b504d0 0x564de9b59340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.478+0000 7f84ddbc1640 1 -- 172.21.15.86:0/2854646585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564de9b59a00 msgr2=0x564de9b56ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.478+0000 7f84ddbc1640 1 --2- 172.21.15.86:0/2854646585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564de9b59a00 0x564de9b56ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.478+0000 7f84ddbc1640 1 -- 172.21.15.86:0/2854646585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564de97c73f0 con 0x564de9b50e80 2024-04-21T08:43:11.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.479+0000 7f84ddbc1640 1 --2- 172.21.15.86:0/2854646585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564de9b50e80 0x564de9b566d0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f84c8010420 tx=0x7f84c80108e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=262dc3e9747bc875 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:11.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.479+0000 7f84c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:11.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.481+0000 7f84c17fa640 1 -- 172.21.15.86:0/2854646585 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c8016070 con 0x564de9b50e80 2024-04-21T08:43:11.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.481+0000 7f84c17fa640 1 -- 172.21.15.86:0/2854646585 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f84c8005220 con 0x564de9b50e80 2024-04-21T08:43:11.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.481+0000 7f84e2a15740 1 -- 172.21.15.86:0/2854646585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84d402cc90 con 0x564de9b50e80 2024-04-21T08:43:11.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.481+0000 7f84c17fa640 1 -- 172.21.15.86:0/2854646585 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f84c8023050 con 0x564de9b50e80 2024-04-21T08:43:11.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.481+0000 7f84e2a15740 10 client.17616 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:11.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.481+0000 7f84e2a15740 1 -- 172.21.15.86:0/2854646585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564de98917d0 con 0x564de9b50e80 2024-04-21T08:43:11.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.481+0000 7f84c17fa640 1 -- 172.21.15.86:0/2854646585 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84d402cc90 con 0x564de9b50e80 2024-04-21T08:43:11.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.482+0000 7f84c17fa640 1 -- 172.21.15.86:0/2854646585 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f84c800f040 con 0x564de9b50e80 2024-04-21T08:43:11.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.482+0000 7f84e2a15740 10 client.17616 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:11.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.482+0000 7f84e2a15740 10 client.17616 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:43:11.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.482+0000 7f84e2a15740 20 client.17616 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:11.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.482+0000 7f84e2a15740 1 --2- 172.21.15.86:0/2854646585 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x564de9b69c80 0x564de9b6c0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.482+0000 7f84e2a15740 4 client.17616 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:11.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.482+0000 7f84e2a15740 1 -- 172.21.15.86:0/2854646585 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564de9a54930 con 0x564de9b69c80 2024-04-21T08:43:11.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.482+0000 7f84dd3c0640 1 --2- 172.21.15.86:0/2854646585 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x564de9b69c80 0x564de9b6c0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.482+0000 7f84c17fa640 10 client.17616 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:43:11.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.482+0000 7f84dd3c0640 1 --2- 172.21.15.86:0/2854646585 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x564de9b69c80 0x564de9b6c0a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=ec11eec8ea63a563 server_cookie=b0f4e575f83878d0 in_seq=0 out_seq=0 2024-04-21T08:43:11.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.483+0000 7f84c17fa640 1 -- 172.21.15.86:0/2854646585 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564de9a54930 con 0x564de9b69c80 2024-04-21T08:43:11.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.483+0000 7f84c17fa640 10 client.17616 handle_command_reply: tid=0 2024-04-21T08:43:11.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.483+0000 7f84e2a15740 1 client.17616 shutdown 2024-04-21T08:43:11.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.484+0000 7f84e2a15740 1 -- 172.21.15.86:0/2854646585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564de9b50e80 msgr2=0x564de9b566d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.484+0000 7f84e2a15740 1 --2- 172.21.15.86:0/2854646585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564de9b50e80 0x564de9b566d0 secure :-1 s=READY pgs=1148 cs=0 l=1 rev1=1 crypto rx=0x7f84c8010420 tx=0x7f84c80108e0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.484+0000 7f84e2a15740 1 -- 172.21.15.86:0/2854646585 shutdown_connections 2024-04-21T08:43:11.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.484+0000 7f84e2a15740 1 --2- 172.21.15.86:0/2854646585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564de9b59a00 0x564de9b56ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.484+0000 7f84e2a15740 1 --2- 172.21.15.86:0/2854646585 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x564de9b69c80 0x564de9b6c0a0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.484+0000 7f84e2a15740 1 --2- 172.21.15.86:0/2854646585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564de9b50e80 0x564de9b566d0 unknown :-1 s=CLOSED pgs=1148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.484+0000 7f84e2a15740 1 --2- 172.21.15.86:0/2854646585 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564de9b504d0 0x564de9b59340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.484+0000 7f84c17fa640 0 client.17616 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:43:11.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.484+0000 7f84e2a15740 1 -- 172.21.15.86:0/2854646585 >> 172.21.15.86:0/2854646585 conn(0x564de9b4aba0 msgr2=0x564de9b4c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:11.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.484+0000 7f84e2a15740 1 -- 172.21.15.86:0/2854646585 shutdown_connections 2024-04-21T08:43:11.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.484+0000 7f84e2a15740 1 -- 172.21.15.86:0/2854646585 wait complete. 2024-04-21T08:43:11.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.484+0000 7f84e2a15740 20 client.17616 trim_cache size 0 max 16384 2024-04-21T08:43:11.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.498+0000 7f84e2a15740 1 Processor -- start 2024-04-21T08:43:11.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.498+0000 7f84e2a15740 1 -- start start 2024-04-21T08:43:11.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.498+0000 7f84e2a15740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564de9b504d0 0x564de9b657d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.498+0000 7f84e2a15740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564de9b50e80 0x564de9b58e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.498+0000 7f84e2a15740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564de9b59a00 0x564de9b59340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.498+0000 7f84e2a15740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f84d81b1500 con 0x564de9b59a00 2024-04-21T08:43:11.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.498+0000 7f84e2a15740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f84d81b3040 con 0x564de9b50e80 2024-04-21T08:43:11.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.498+0000 7f84e2a15740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f84d81b1bd0 con 0x564de9b504d0 2024-04-21T08:43:11.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.498+0000 7f84dd3c0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564de9b59a00 0x564de9b59340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.498+0000 7f84dd3c0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564de9b59a00 0x564de9b59340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51144/0 (socket says 172.21.15.86:51144) 2024-04-21T08:43:11.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.498+0000 7f84dd3c0640 1 -- 172.21.15.86:0/2888495093 learned_addr learned my addr 172.21.15.86:0/2888495093 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:11.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.498+0000 7f84ddbc1640 1 --2- 172.21.15.86:0/2888495093 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564de9b50e80 0x564de9b58e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.498+0000 7f84dd3c0640 1 --2- 172.21.15.86:0/2888495093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564de9b504d0 0x564de9b657d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.498+0000 7f84dd3c0640 1 -- 172.21.15.86:0/2888495093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564de9b504d0 msgr2=0x564de9b657d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.498+0000 7f84dd3c0640 1 --2- 172.21.15.86:0/2888495093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564de9b504d0 0x564de9b657d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.498+0000 7f84dd3c0640 1 -- 172.21.15.86:0/2888495093 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564de9b50e80 msgr2=0x564de9b58e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.498+0000 7f84dd3c0640 1 --2- 172.21.15.86:0/2888495093 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564de9b50e80 0x564de9b58e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.498+0000 7f84dd3c0640 1 -- 172.21.15.86:0/2888495093 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564de9b4f560 con 0x564de9b59a00 2024-04-21T08:43:11.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.499+0000 7f84dd3c0640 1 --2- 172.21.15.86:0/2888495093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564de9b59a00 0x564de9b59340 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f84d4009900 tx=0x7f84d4005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f92b2a521d264512 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:11.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.499+0000 7f84c37fe640 1 -- 172.21.15.86:0/2888495093 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d4043070 con 0x564de9b59a00 2024-04-21T08:43:11.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.499+0000 7f84c37fe640 1 -- 172.21.15.86:0/2888495093 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84d4057070 con 0x564de9b59a00 2024-04-21T08:43:11.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.499+0000 7f84c37fe640 1 -- 172.21.15.86:0/2888495093 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84d402cb60 con 0x564de9b59a00 2024-04-21T08:43:11.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.499+0000 7f84e2a15740 1 -- 172.21.15.86:0/2888495093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564de9b59a00 msgr2=0x564de9b59340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.499+0000 7f84e2a15740 1 --2- 172.21.15.86:0/2888495093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564de9b59a00 0x564de9b59340 secure :-1 s=READY pgs=1149 cs=0 l=1 rev1=1 crypto rx=0x7f84d4009900 tx=0x7f84d4005d50 comp rx=0 tx=0).stop 2024-04-21T08:43:11.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.499+0000 7f84e2a15740 1 -- 172.21.15.86:0/2888495093 shutdown_connections 2024-04-21T08:43:11.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.499+0000 7f84e2a15740 1 --2- 172.21.15.86:0/2888495093 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564de9b59a00 0x564de9b59340 unknown :-1 s=CLOSED pgs=1149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.499+0000 7f84e2a15740 1 --2- 172.21.15.86:0/2888495093 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564de9b50e80 0x564de9b58e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.499+0000 7f84e2a15740 1 --2- 172.21.15.86:0/2888495093 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564de9b504d0 0x564de9b657d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.499+0000 7f84e2a15740 1 -- 172.21.15.86:0/2888495093 >> 172.21.15.86:0/2888495093 conn(0x564de9b4aba0 msgr2=0x564de9b4c860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:11.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.499+0000 7f84e2a15740 1 -- 172.21.15.86:0/2888495093 shutdown_connections 2024-04-21T08:43:11.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.500+0000 7f84e2a15740 1 -- 172.21.15.86:0/2888495093 wait complete. 2024-04-21T08:43:11.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.500+0000 7f84e2a15740 1 -- start start 2024-04-21T08:43:11.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.500+0000 7f84e2a15740 1 Processor -- start 2024-04-21T08:43:11.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.500+0000 7f84e2a15740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564de9b504d0 0x564de9b61f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.500+0000 7f84e2a15740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564de9b50e80 0x564de9b5d320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.500+0000 7f84e2a15740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564de9b59a00 0x564de9b5d860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.500+0000 7f84e2a15740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f84d81b2970 con 0x564de9b50e80 2024-04-21T08:43:11.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.500+0000 7f84e2a15740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f84d81b22a0 con 0x564de9b59a00 2024-04-21T08:43:11.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.500+0000 7f84e2a15740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f84d81af4c0 con 0x564de9b504d0 2024-04-21T08:43:11.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.500+0000 7f84e2a15740 10 client.0 _ll_register_callbacks cb 0x7f84c8009c30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:11.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.500+0000 7f84ddbc1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564de9b50e80 0x564de9b5d320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.500+0000 7f84ddbc1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564de9b50e80 0x564de9b5d320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51148/0 (socket says 172.21.15.86:51148) 2024-04-21T08:43:11.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.500+0000 7f84ddbc1640 1 -- 172.21.15.86:0/1823179787 learned_addr learned my addr 172.21.15.86:0/1823179787 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:11.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.500+0000 7f84dd3c0640 1 --2- 172.21.15.86:0/1823179787 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564de9b59a00 0x564de9b5d860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.501+0000 7f84ddbc1640 1 -- 172.21.15.86:0/1823179787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564de9b504d0 msgr2=0x564de9b61f80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.501+0000 7f84dd3c0640 1 --2- 172.21.15.86:0/1823179787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564de9b504d0 0x564de9b61f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.501+0000 7f84ddbc1640 1 --2- 172.21.15.86:0/1823179787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564de9b504d0 0x564de9b61f80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.501+0000 7f84ddbc1640 1 -- 172.21.15.86:0/1823179787 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564de9b59a00 msgr2=0x564de9b5d860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.501+0000 7f84ddbc1640 1 --2- 172.21.15.86:0/1823179787 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564de9b59a00 0x564de9b5d860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.501+0000 7f84ddbc1640 1 -- 172.21.15.86:0/1823179787 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564de9b5df20 con 0x564de9b50e80 2024-04-21T08:43:11.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.501+0000 7f84dd3c0640 1 --2- 172.21.15.86:0/1823179787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564de9b504d0 0x564de9b61f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:11.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.501+0000 7f84ddbc1640 1 --2- 172.21.15.86:0/1823179787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564de9b50e80 0x564de9b5d320 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f84c8009350 tx=0x7f84c8012790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ecad1982fe40e02 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:11.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.501+0000 7f84c17fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:11.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.501+0000 7f84c17fa640 1 -- 172.21.15.86:0/1823179787 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c8016070 con 0x564de9b50e80 2024-04-21T08:43:11.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.501+0000 7f84c17fa640 1 -- 172.21.15.86:0/1823179787 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f84c8005040 con 0x564de9b50e80 2024-04-21T08:43:11.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.501+0000 7f84e2a15740 1 -- 172.21.15.86:0/1823179787 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f84d4057070 con 0x564de9b50e80 2024-04-21T08:43:11.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.501+0000 7f84c17fa640 1 -- 172.21.15.86:0/1823179787 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f84c8037050 con 0x564de9b50e80 2024-04-21T08:43:11.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.501+0000 7f84e2a15740 10 client.17622 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:11.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.501+0000 7f84e2a15740 1 -- 172.21.15.86:0/1823179787 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564de9bd0070 con 0x564de9b50e80 2024-04-21T08:43:11.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.501+0000 7f84c17fa640 1 -- 172.21.15.86:0/1823179787 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84c8012e50 con 0x564de9b50e80 2024-04-21T08:43:11.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.501+0000 7f84c17fa640 1 -- 172.21.15.86:0/1823179787 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f84c8011530 con 0x564de9b50e80 2024-04-21T08:43:11.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.501+0000 7f84e2a15740 10 client.17622 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:11.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.501+0000 7f84e2a15740 10 client.17622 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:43:11.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.501+0000 7f84e2a15740 20 client.17622 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:11.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.501+0000 7f84e2a15740 1 --2- 172.21.15.86:0/1823179787 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x564de9bd0870 0x564de9bd2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.501+0000 7f84e2a15740 4 client.17622 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:43:11.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.501+0000 7f84e2a15740 1 -- 172.21.15.86:0/1823179787 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564de9ac2c10 con 0x564de9bd0870 2024-04-21T08:43:11.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.502+0000 7f84dd3c0640 1 --2- 172.21.15.86:0/1823179787 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x564de9bd0870 0x564de9bd2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.502+0000 7f84c17fa640 10 client.17622 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:43:11.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.502+0000 7f84dd3c0640 1 --2- 172.21.15.86:0/1823179787 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x564de9bd0870 0x564de9bd2c90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b737f08516425e92 server_cookie=99b0159707f4ebd6 in_seq=0 out_seq=0 2024-04-21T08:43:11.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.502+0000 7f84c17fa640 1 -- 172.21.15.86:0/1823179787 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564de9ac2c10 con 0x564de9bd0870 2024-04-21T08:43:11.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.502+0000 7f84c17fa640 10 client.17622 handle_command_reply: tid=0 2024-04-21T08:43:11.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.502+0000 7f84e2a15740 1 client.17622 shutdown 2024-04-21T08:43:11.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.502+0000 7f84e2a15740 1 -- 172.21.15.86:0/1823179787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564de9b50e80 msgr2=0x564de9b5d320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.502+0000 7f84e2a15740 1 --2- 172.21.15.86:0/1823179787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564de9b50e80 0x564de9b5d320 secure :-1 s=READY pgs=1150 cs=0 l=1 rev1=1 crypto rx=0x7f84c8009350 tx=0x7f84c8012790 comp rx=0 tx=0).stop 2024-04-21T08:43:11.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.503+0000 7f84e2a15740 1 -- 172.21.15.86:0/1823179787 shutdown_connections 2024-04-21T08:43:11.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.503+0000 7f84e2a15740 1 --2- 172.21.15.86:0/1823179787 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564de9b59a00 0x564de9b5d860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.503+0000 7f84e2a15740 1 --2- 172.21.15.86:0/1823179787 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x564de9bd0870 0x564de9bd2c90 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.503+0000 7f84e2a15740 1 --2- 172.21.15.86:0/1823179787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564de9b50e80 0x564de9b5d320 unknown :-1 s=CLOSED pgs=1150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.503+0000 7f84e2a15740 1 --2- 172.21.15.86:0/1823179787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564de9b504d0 0x564de9b61f80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.503+0000 7f84c17fa640 0 client.17622 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:43:11.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.503+0000 7f84e2a15740 1 -- 172.21.15.86:0/1823179787 >> 172.21.15.86:0/1823179787 conn(0x564de9b4aba0 msgr2=0x564de9b49370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:11.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.503+0000 7f84e2a15740 1 -- 172.21.15.86:0/1823179787 shutdown_connections 2024-04-21T08:43:11.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.503+0000 7f84e2a15740 1 -- 172.21.15.86:0/1823179787 wait complete. 2024-04-21T08:43:11.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.503+0000 7f84e2a15740 20 client.17622 trim_cache size 0 max 16384 2024-04-21T08:43:11.510 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:43:11.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.503+0000 7f84dfe25640 1 -- 172.21.15.86:0/119523546 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f84c40a3d20 msgr2=0x7f84c40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.503+0000 7f84dfe25640 1 --2- 172.21.15.86:0/119523546 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f84c40a3d20 0x7f84c40a61e0 secure :-1 s=READY pgs=178 cs=0 l=1 rev1=1 crypto rx=0x7f84d0098e50 tx=0x7f84d0087040 comp rx=0 tx=0).stop 2024-04-21T08:43:11.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.503+0000 7f84dfe25640 1 -- 172.21.15.86:0/119523546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84d819a7f0 msgr2=0x7f84d819aca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.503+0000 7f84dfe25640 1 --2- 172.21.15.86:0/119523546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84d819a7f0 0x7f84d819aca0 secure :-1 s=READY pgs=1146 cs=0 l=1 rev1=1 crypto rx=0x7f84d000eaa0 tx=0x7f84d000ef60 comp rx=0 tx=0).stop 2024-04-21T08:43:11.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.503+0000 7f84dfe25640 1 -- 172.21.15.86:0/119523546 shutdown_connections 2024-04-21T08:43:11.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.503+0000 7f84dfe25640 1 --2- 172.21.15.86:0/119523546 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f84c40a3d20 0x7f84c40a61e0 unknown :-1 s=CLOSED pgs=178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.503+0000 7f84dfe25640 1 --2- 172.21.15.86:0/119523546 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84d819a7f0 0x7f84d819aca0 unknown :-1 s=CLOSED pgs=1146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.503+0000 7f84dfe25640 1 --2- 172.21.15.86:0/119523546 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84d8107260 0x7f84d81a0bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.503+0000 7f84dfe25640 1 --2- 172.21.15.86:0/119523546 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84d8102190 0x7f84d81a0680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.503+0000 7f84dfe25640 1 -- 172.21.15.86:0/119523546 >> 172.21.15.86:0/119523546 conn(0x7f84d80793a0 msgr2=0x7f84d8100180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:11.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.503+0000 7f84dfe25640 1 -- 172.21.15.86:0/119523546 shutdown_connections 2024-04-21T08:43:11.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.503+0000 7f84dfe25640 1 -- 172.21.15.86:0/119523546 wait complete. 2024-04-21T08:43:11.513 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:43:11.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.614+0000 7fcc0b0b3640 1 -- 172.21.15.86:0/3009383357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc04107030 msgr2=0x7fcc04107410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.614+0000 7fcc0b0b3640 1 --2- 172.21.15.86:0/3009383357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc04107030 0x7fcc04107410 secure :-1 s=READY pgs=1151 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8009930 tx=0x7fcbf802bc30 comp rx=0 tx=0).stop 2024-04-21T08:43:11.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.614+0000 7fcc0b0b3640 1 -- 172.21.15.86:0/3009383357 shutdown_connections 2024-04-21T08:43:11.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.614+0000 7fcc0b0b3640 1 --2- 172.21.15.86:0/3009383357 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc04101b40 0x7fcc0410dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.614+0000 7fcc0b0b3640 1 --2- 172.21.15.86:0/3009383357 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc041011a0 0x7fcc04101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.614+0000 7fcc0b0b3640 1 --2- 172.21.15.86:0/3009383357 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc04107030 0x7fcc04107410 unknown :-1 s=CLOSED pgs=1151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.614+0000 7fcc0b0b3640 1 -- 172.21.15.86:0/3009383357 >> 172.21.15.86:0/3009383357 conn(0x7fcc04076c50 msgr2=0x7fcc04077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:11.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.614+0000 7fcc0b0b3640 1 -- 172.21.15.86:0/3009383357 shutdown_connections 2024-04-21T08:43:11.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.614+0000 7fcc0b0b3640 1 -- 172.21.15.86:0/3009383357 wait complete. 2024-04-21T08:43:11.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.614+0000 7fcc0b0b3640 1 Processor -- start 2024-04-21T08:43:11.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.614+0000 7fcc0b0b3640 1 -- start start 2024-04-21T08:43:11.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.615+0000 7fcc0b0b3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc041011a0 0x7fcc04199560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.615+0000 7fcc0b0b3640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc04101b40 0x7fcc04199aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.615+0000 7fcc0b0b3640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc04107030 0x7fcc041a45e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.615+0000 7fcc0b0b3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcc041110e0 con 0x7fcc041011a0 2024-04-21T08:43:11.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.615+0000 7fcc0b0b3640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcc04110f60 con 0x7fcc04101b40 2024-04-21T08:43:11.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.615+0000 7fcc0b0b3640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fcc04111260 con 0x7fcc04107030 2024-04-21T08:43:11.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.615+0000 7fcc08e4f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc041011a0 0x7fcc04199560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.615+0000 7fcc08e4f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc041011a0 0x7fcc04199560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51174/0 (socket says 172.21.15.86:51174) 2024-04-21T08:43:11.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.615+0000 7fcc08e4f640 1 -- 172.21.15.86:0/1113046109 learned_addr learned my addr 172.21.15.86:0/1113046109 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:11.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.615+0000 7fcc03fff640 1 --2- 172.21.15.86:0/1113046109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc04101b40 0x7fcc04199aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.615+0000 7fcc09650640 1 --2- 172.21.15.86:0/1113046109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc04107030 0x7fcc041a45e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.615+0000 7fcc08e4f640 1 -- 172.21.15.86:0/1113046109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc04107030 msgr2=0x7fcc041a45e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.615+0000 7fcc08e4f640 1 --2- 172.21.15.86:0/1113046109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc04107030 0x7fcc041a45e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.615+0000 7fcc08e4f640 1 -- 172.21.15.86:0/1113046109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc04101b40 msgr2=0x7fcc04199aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.615+0000 7fcc08e4f640 1 --2- 172.21.15.86:0/1113046109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc04101b40 0x7fcc04199aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.615+0000 7fcc08e4f640 1 -- 172.21.15.86:0/1113046109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcc041a4b20 con 0x7fcc041011a0 2024-04-21T08:43:11.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.615+0000 7fcc08e4f640 1 --2- 172.21.15.86:0/1113046109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc041011a0 0x7fcc04199560 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8009230 tx=0x7fcbf80045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:11.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.615+0000 7fcc01ffb640 1 -- 172.21.15.86:0/1113046109 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8043020 con 0x7fcc041011a0 2024-04-21T08:43:11.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.615+0000 7fcc01ffb640 1 -- 172.21.15.86:0/1113046109 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbf802cb70 con 0x7fcc041011a0 2024-04-21T08:43:11.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.615+0000 7fcc0b0b3640 1 -- 172.21.15.86:0/1113046109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcc041a4cf0 con 0x7fcc041011a0 2024-04-21T08:43:11.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.615+0000 7fcc01ffb640 1 -- 172.21.15.86:0/1113046109 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf8003ac0 con 0x7fcc041011a0 2024-04-21T08:43:11.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.616+0000 7fcc0b0b3640 1 -- 172.21.15.86:0/1113046109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcc041a5150 con 0x7fcc041011a0 2024-04-21T08:43:11.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.618+0000 7fcc01ffb640 1 -- 172.21.15.86:0/1113046109 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcbf8046020 con 0x7fcc041011a0 2024-04-21T08:43:11.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.618+0000 7fcc0dd7e740 1 Processor -- start 2024-04-21T08:43:11.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.618+0000 7fcc0dd7e740 1 -- start start 2024-04-21T08:43:11.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.619+0000 7fcc0dd7e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556ae2a93330 0x556ae2a93710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.619+0000 7fcc0dd7e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ae2a93ce0 0x556ae2a9c820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.619+0000 7fcc0dd7e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ae2a9ce40 0x556ae2a9d2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.619+0000 7fcc0dd7e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556ae29f81f0 con 0x556ae2a93ce0 2024-04-21T08:43:11.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.619+0000 7fcc0dd7e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x556ae29954b0 con 0x556ae2a93330 2024-04-21T08:43:11.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.619+0000 7fcc0dd7e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556ae29f41b0 con 0x556ae2a9ce40 2024-04-21T08:43:11.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.619+0000 7fcc08e4f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ae2a93ce0 0x556ae2a9c820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.619+0000 7fcc08e4f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ae2a93ce0 0x556ae2a9c820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51182/0 (socket says 172.21.15.86:51182) 2024-04-21T08:43:11.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.619+0000 7fcc08e4f640 1 -- 172.21.15.86:0/796004045 learned_addr learned my addr 172.21.15.86:0/796004045 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:11.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.619+0000 7fcc03fff640 1 --2- 172.21.15.86:0/796004045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ae2a9ce40 0x556ae2a9d2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.619+0000 7fcc09650640 1 --2- 172.21.15.86:0/796004045 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556ae2a93330 0x556ae2a93710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.619+0000 7fcc01ffb640 1 --2- 172.21.15.86:0/1113046109 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fcbf40a3d20 0x7fcbf40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.619+0000 7fcc08e4f640 1 -- 172.21.15.86:0/796004045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ae2a9ce40 msgr2=0x556ae2a9d2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.619+0000 7fcc08e4f640 1 --2- 172.21.15.86:0/796004045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ae2a9ce40 0x556ae2a9d2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.619+0000 7fcc08e4f640 1 -- 172.21.15.86:0/796004045 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556ae2a93330 msgr2=0x556ae2a93710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.619+0000 7fcc08e4f640 1 --2- 172.21.15.86:0/796004045 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556ae2a93330 0x556ae2a93710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.619+0000 7fcc08e4f640 1 -- 172.21.15.86:0/796004045 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ae29871b0 con 0x556ae2a93ce0 2024-04-21T08:43:11.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.619+0000 7fcc09650640 1 --2- 172.21.15.86:0/1113046109 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fcbf40a3d20 0x7fcbf40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.619+0000 7fcc01ffb640 1 -- 172.21.15.86:0/1113046109 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fcbf80f2c60 con 0x7fcc041011a0 2024-04-21T08:43:11.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.619+0000 7fcc08e4f640 1 --2- 172.21.15.86:0/796004045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ae2a93ce0 0x556ae2a9c820 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fcbf80f0090 tx=0x7fcbf80e9ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2dee97575e6a3835 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:11.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.619+0000 7fcbeeffd640 1 -- 172.21.15.86:0/796004045 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf80be4e0 con 0x556ae2a93ce0 2024-04-21T08:43:11.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.619+0000 7fcbeeffd640 1 -- 172.21.15.86:0/796004045 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbf80ed3d0 con 0x556ae2a93ce0 2024-04-21T08:43:11.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.620+0000 7fcbeeffd640 1 -- 172.21.15.86:0/796004045 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf802e040 con 0x556ae2a93ce0 2024-04-21T08:43:11.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.620+0000 7fcc09650640 1 --2- 172.21.15.86:0/1113046109 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fcbf40a3d20 0x7fcbf40a61e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc005fd0 tx=0x7fcbfc004a40 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:11.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.620+0000 7fcc0dd7e740 1 -- 172.21.15.86:0/796004045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ae2a93ce0 msgr2=0x556ae2a9c820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.620+0000 7fcc0dd7e740 1 --2- 172.21.15.86:0/796004045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ae2a93ce0 0x556ae2a9c820 secure :-1 s=READY pgs=1153 cs=0 l=1 rev1=1 crypto rx=0x7fcbf80f0090 tx=0x7fcbf80e9ab0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.620+0000 7fcc0dd7e740 1 -- 172.21.15.86:0/796004045 shutdown_connections 2024-04-21T08:43:11.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.620+0000 7fcc0dd7e740 1 --2- 172.21.15.86:0/796004045 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ae2a9ce40 0x556ae2a9d2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.620+0000 7fcc0dd7e740 1 --2- 172.21.15.86:0/796004045 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ae2a93ce0 0x556ae2a9c820 unknown :-1 s=CLOSED pgs=1153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.620+0000 7fcc0dd7e740 1 --2- 172.21.15.86:0/796004045 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556ae2a93330 0x556ae2a93710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.620+0000 7fcc0dd7e740 1 -- 172.21.15.86:0/796004045 >> 172.21.15.86:0/796004045 conn(0x556ae2a8dba0 msgr2=0x556ae2a8df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:11.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.620+0000 7fcc0dd7e740 1 -- 172.21.15.86:0/796004045 shutdown_connections 2024-04-21T08:43:11.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.620+0000 7fcc0dd7e740 1 -- 172.21.15.86:0/796004045 wait complete. 2024-04-21T08:43:11.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.620+0000 7fcc0dd7e740 1 -- start start 2024-04-21T08:43:11.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.621+0000 7fcc0dd7e740 1 Processor -- start 2024-04-21T08:43:11.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.621+0000 7fcc0dd7e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ae2a93330 0x556ae2aa31e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.621+0000 7fcc0dd7e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ae2a93ce0 0x556ae2aa05c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.621+0000 7fcc0dd7e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556ae2a9ce40 0x556ae2aa0b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.621+0000 7fcc0dd7e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556ae29f81f0 con 0x556ae2a93330 2024-04-21T08:43:11.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.621+0000 7fcc0dd7e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x556ae29b1500 con 0x556ae2a9ce40 2024-04-21T08:43:11.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.621+0000 7fcc0dd7e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556ae2aa3b80 con 0x556ae2a93ce0 2024-04-21T08:43:11.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.621+0000 7fcc0dd7e740 10 client.0 _ll_register_callbacks cb 0x556ae297fb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:11.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.621+0000 7fcc08e4f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ae2a93330 0x556ae2aa31e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.621+0000 7fcc08e4f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ae2a93330 0x556ae2aa31e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51186/0 (socket says 172.21.15.86:51186) 2024-04-21T08:43:11.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.621+0000 7fcc08e4f640 1 -- 172.21.15.86:0/3391126193 learned_addr learned my addr 172.21.15.86:0/3391126193 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:11.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.621+0000 7fcc09650640 1 --2- 172.21.15.86:0/3391126193 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556ae2a9ce40 0x556ae2aa0b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.621+0000 7fcc03fff640 1 --2- 172.21.15.86:0/3391126193 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ae2a93ce0 0x556ae2aa05c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.621+0000 7fcc08e4f640 1 -- 172.21.15.86:0/3391126193 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ae2a93ce0 msgr2=0x556ae2aa05c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.621+0000 7fcc08e4f640 1 --2- 172.21.15.86:0/3391126193 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ae2a93ce0 0x556ae2aa05c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.621+0000 7fcc08e4f640 1 -- 172.21.15.86:0/3391126193 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556ae2a9ce40 msgr2=0x556ae2aa0b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.621+0000 7fcc08e4f640 1 --2- 172.21.15.86:0/3391126193 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556ae2a9ce40 0x556ae2aa0b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.621+0000 7fcc08e4f640 1 -- 172.21.15.86:0/3391126193 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ae270a3f0 con 0x556ae2a93330 2024-04-21T08:43:11.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.621+0000 7fcc09650640 1 --2- 172.21.15.86:0/3391126193 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556ae2a9ce40 0x556ae2aa0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:11.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.621+0000 7fcc03fff640 1 --2- 172.21.15.86:0/3391126193 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ae2a93ce0 0x556ae2aa05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:11.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.622+0000 7fcc08e4f640 1 --2- 172.21.15.86:0/3391126193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ae2a93330 0x556ae2aa31e0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fcbf80f0090 tx=0x7fcbf8009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2654469e34ac8d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:11.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.622+0000 7fcbe3fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:11.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.622+0000 7fcbe3fff640 1 -- 172.21.15.86:0/3391126193 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbf80be300 con 0x556ae2a93330 2024-04-21T08:43:11.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.622+0000 7fcbe3fff640 1 -- 172.21.15.86:0/3391126193 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fcbf80bb010 con 0x556ae2a93330 2024-04-21T08:43:11.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.622+0000 7fcc0dd7e740 1 -- 172.21.15.86:0/3391126193 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x556ae2997930 con 0x556ae2a93330 2024-04-21T08:43:11.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.622+0000 7fcbe3fff640 1 -- 172.21.15.86:0/3391126193 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcbf80be4e0 con 0x556ae2a93330 2024-04-21T08:43:11.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.622+0000 7fcc0dd7e740 10 client.17634 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:11.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.622+0000 7fcc0dd7e740 1 -- 172.21.15.86:0/3391126193 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556ae27d47d0 con 0x556ae2a93330 2024-04-21T08:43:11.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.622+0000 7fcbe3fff640 1 -- 172.21.15.86:0/3391126193 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x556ae2997930 con 0x556ae2a93330 2024-04-21T08:43:11.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.622+0000 7fcbe3fff640 1 -- 172.21.15.86:0/3391126193 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcbf80f8020 con 0x556ae2a93330 2024-04-21T08:43:11.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.622+0000 7fcc0dd7e740 10 client.17634 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:11.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.622+0000 7fcc0dd7e740 10 client.17634 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:11.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.622+0000 7fcc0dd7e740 20 client.17634 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:11.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.622+0000 7fcc0dd7e740 1 --2- 172.21.15.86:0/3391126193 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x556ae2ab0ba0 0x556ae2ab2fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.622+0000 7fcc0dd7e740 4 client.17634 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:11.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.622+0000 7fcc0dd7e740 1 -- 172.21.15.86:0/3391126193 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556ae2a05c10 con 0x556ae2ab0ba0 2024-04-21T08:43:11.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.623+0000 7fcc03fff640 1 --2- 172.21.15.86:0/3391126193 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x556ae2ab0ba0 0x556ae2ab2fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.623+0000 7fcbe3fff640 10 client.17634 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:11.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.623+0000 7fcc03fff640 1 --2- 172.21.15.86:0/3391126193 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x556ae2ab0ba0 0x556ae2ab2fd0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=126751b9b0afe0a6 server_cookie=c7d0369dce25bf79 in_seq=0 out_seq=0 2024-04-21T08:43:11.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.624+0000 7fcbe3fff640 1 -- 172.21.15.86:0/3391126193 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556ae2a05c10 con 0x556ae2ab0ba0 2024-04-21T08:43:11.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.624+0000 7fcbe3fff640 10 client.17634 handle_command_reply: tid=0 2024-04-21T08:43:11.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.624+0000 7fcc0dd7e740 1 client.17634 shutdown 2024-04-21T08:43:11.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.624+0000 7fcc0dd7e740 1 -- 172.21.15.86:0/3391126193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ae2a93330 msgr2=0x556ae2aa31e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.624+0000 7fcc0dd7e740 1 --2- 172.21.15.86:0/3391126193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ae2a93330 0x556ae2aa31e0 secure :-1 s=READY pgs=1154 cs=0 l=1 rev1=1 crypto rx=0x7fcbf80f0090 tx=0x7fcbf8009260 comp rx=0 tx=0).stop 2024-04-21T08:43:11.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.624+0000 7fcc0dd7e740 1 -- 172.21.15.86:0/3391126193 shutdown_connections 2024-04-21T08:43:11.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.624+0000 7fcc0dd7e740 1 --2- 172.21.15.86:0/3391126193 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x556ae2ab0ba0 0x556ae2ab2fd0 crc :-1 s=READY pgs=155 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.624+0000 7fcc0dd7e740 1 --2- 172.21.15.86:0/3391126193 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556ae2a9ce40 0x556ae2aa0b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.624+0000 7fcbe3fff640 0 client.17634 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:11.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.624+0000 7fcc0dd7e740 1 --2- 172.21.15.86:0/3391126193 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ae2a93ce0 0x556ae2aa05c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.624+0000 7fcc0dd7e740 1 --2- 172.21.15.86:0/3391126193 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ae2a93330 0x556ae2aa31e0 unknown :-1 s=CLOSED pgs=1154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.624+0000 7fcc0dd7e740 1 -- 172.21.15.86:0/3391126193 >> 172.21.15.86:0/3391126193 conn(0x556ae2a8dba0 msgr2=0x556ae2a8f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:11.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.624+0000 7fcc0dd7e740 1 -- 172.21.15.86:0/3391126193 shutdown_connections 2024-04-21T08:43:11.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.624+0000 7fcc0dd7e740 1 -- 172.21.15.86:0/3391126193 wait complete. 2024-04-21T08:43:11.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.624+0000 7fcc0dd7e740 20 client.17634 trim_cache size 0 max 16384 2024-04-21T08:43:11.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.637+0000 7fcc0dd7e740 1 Processor -- start 2024-04-21T08:43:11.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.637+0000 7fcc0dd7e740 1 -- start start 2024-04-21T08:43:11.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.637+0000 7fcc0dd7e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556ae2a8dba0 0x556ae2aac900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.637+0000 7fcc0dd7e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ae2a93330 0x556ae2a9c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.637+0000 7fcc0dd7e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ae2a93ce0 0x556ae2a9f720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.637+0000 7fcc0dd7e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcc041ad010 con 0x556ae2a93ce0 2024-04-21T08:43:11.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.637+0000 7fcc0dd7e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcc041aeb50 con 0x556ae2a8dba0 2024-04-21T08:43:11.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.637+0000 7fcc0dd7e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fcc041ad6e0 con 0x556ae2a93330 2024-04-21T08:43:11.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.637+0000 7fcc09650640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ae2a93ce0 0x556ae2a9f720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.637+0000 7fcc09650640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ae2a93ce0 0x556ae2a9f720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51190/0 (socket says 172.21.15.86:51190) 2024-04-21T08:43:11.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.637+0000 7fcc09650640 1 -- 172.21.15.86:0/1285306689 learned_addr learned my addr 172.21.15.86:0/1285306689 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:11.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.637+0000 7fcc08e4f640 1 --2- 172.21.15.86:0/1285306689 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556ae2a8dba0 0x556ae2aac900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.638+0000 7fcc03fff640 1 --2- 172.21.15.86:0/1285306689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ae2a93330 0x556ae2a9c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.638+0000 7fcc09650640 1 -- 172.21.15.86:0/1285306689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ae2a93330 msgr2=0x556ae2a9c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.638+0000 7fcc09650640 1 --2- 172.21.15.86:0/1285306689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ae2a93330 0x556ae2a9c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.638+0000 7fcc09650640 1 -- 172.21.15.86:0/1285306689 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556ae2a8dba0 msgr2=0x556ae2aac900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.638+0000 7fcc09650640 1 --2- 172.21.15.86:0/1285306689 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556ae2a8dba0 0x556ae2aac900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.638+0000 7fcc09650640 1 -- 172.21.15.86:0/1285306689 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556ae2a92560 con 0x556ae2a93ce0 2024-04-21T08:43:11.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.638+0000 7fcc08e4f640 1 --2- 172.21.15.86:0/1285306689 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556ae2a8dba0 0x556ae2aac900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:43:11.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.638+0000 7fcc09650640 1 --2- 172.21.15.86:0/1285306689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ae2a93ce0 0x556ae2a9f720 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc01e710 tx=0x7fcbfc014000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb8390c887224795 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:11.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.638+0000 7fcbee7fc640 1 -- 172.21.15.86:0/1285306689 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc0127b0 con 0x556ae2a93ce0 2024-04-21T08:43:11.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.638+0000 7fcbee7fc640 1 -- 172.21.15.86:0/1285306689 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbfc016070 con 0x556ae2a93ce0 2024-04-21T08:43:11.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.638+0000 7fcbee7fc640 1 -- 172.21.15.86:0/1285306689 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc027d60 con 0x556ae2a93ce0 2024-04-21T08:43:11.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.639+0000 7fcc0dd7e740 1 -- 172.21.15.86:0/1285306689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ae2a93ce0 msgr2=0x556ae2a9f720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.639+0000 7fcc0dd7e740 1 --2- 172.21.15.86:0/1285306689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ae2a93ce0 0x556ae2a9f720 secure :-1 s=READY pgs=1155 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc01e710 tx=0x7fcbfc014000 comp rx=0 tx=0).stop 2024-04-21T08:43:11.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.639+0000 7fcc0dd7e740 1 -- 172.21.15.86:0/1285306689 shutdown_connections 2024-04-21T08:43:11.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.639+0000 7fcc0dd7e740 1 --2- 172.21.15.86:0/1285306689 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ae2a93ce0 0x556ae2a9f720 unknown :-1 s=CLOSED pgs=1155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.639+0000 7fcc0dd7e740 1 --2- 172.21.15.86:0/1285306689 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ae2a93330 0x556ae2a9c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.639+0000 7fcc0dd7e740 1 --2- 172.21.15.86:0/1285306689 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556ae2a8dba0 0x556ae2aac900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.639+0000 7fcc0dd7e740 1 -- 172.21.15.86:0/1285306689 >> 172.21.15.86:0/1285306689 conn(0x556ae2aa0c60 msgr2=0x556ae2a8c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:11.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.639+0000 7fcc0dd7e740 1 -- 172.21.15.86:0/1285306689 shutdown_connections 2024-04-21T08:43:11.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.639+0000 7fcc0dd7e740 1 -- 172.21.15.86:0/1285306689 wait complete. 2024-04-21T08:43:11.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.639+0000 7fcc0dd7e740 1 -- start start 2024-04-21T08:43:11.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.639+0000 7fcc0dd7e740 1 Processor -- start 2024-04-21T08:43:11.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.639+0000 7fcc0dd7e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ae2a8dba0 0x556ae2aa2470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.639+0000 7fcc0dd7e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556ae2a93330 0x556ae2aa29b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.639+0000 7fcc0dd7e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ae2a93ce0 0x556ae2b16110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.639+0000 7fcc0dd7e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcc041ae480 con 0x556ae2a93ce0 2024-04-21T08:43:11.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.639+0000 7fcc0dd7e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcc041addb0 con 0x556ae2a93330 2024-04-21T08:43:11.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.639+0000 7fcc0dd7e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fcc041aafd0 con 0x556ae2a8dba0 2024-04-21T08:43:11.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.639+0000 7fcc0dd7e740 10 client.0 _ll_register_callbacks cb 0x7fcbf80ef0d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:11.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.640+0000 7fcc09650640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ae2a93ce0 0x556ae2b16110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.640+0000 7fcc09650640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ae2a93ce0 0x556ae2b16110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51194/0 (socket says 172.21.15.86:51194) 2024-04-21T08:43:11.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.640+0000 7fcc09650640 1 -- 172.21.15.86:0/1566178563 learned_addr learned my addr 172.21.15.86:0/1566178563 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:11.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.640+0000 7fcc03fff640 1 --2- 172.21.15.86:0/1566178563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556ae2a93330 0x556ae2aa29b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.640+0000 7fcc08e4f640 1 --2- 172.21.15.86:0/1566178563 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ae2a8dba0 0x556ae2aa2470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.640+0000 7fcc09650640 1 -- 172.21.15.86:0/1566178563 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ae2a8dba0 msgr2=0x556ae2aa2470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.640+0000 7fcc09650640 1 --2- 172.21.15.86:0/1566178563 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ae2a8dba0 0x556ae2aa2470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.640+0000 7fcc09650640 1 -- 172.21.15.86:0/1566178563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556ae2a93330 msgr2=0x556ae2aa29b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:11.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.640+0000 7fcc09650640 1 --2- 172.21.15.86:0/1566178563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556ae2a93330 0x556ae2aa29b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.640+0000 7fcc09650640 1 -- 172.21.15.86:0/1566178563 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556ae2aa3240 con 0x556ae2a93ce0 2024-04-21T08:43:11.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.640+0000 7fcc03fff640 1 --2- 172.21.15.86:0/1566178563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556ae2a93330 0x556ae2aa29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:11.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.640+0000 7fcc09650640 1 --2- 172.21.15.86:0/1566178563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ae2a93ce0 0x556ae2b16110 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc01e6e0 tx=0x7fcbfc027720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93a55cae4c3c78ef server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:11.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.640+0000 7fcbe3fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:11.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.640+0000 7fcbe3fff640 1 -- 172.21.15.86:0/1566178563 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcbfc0125b0 con 0x556ae2a93ce0 2024-04-21T08:43:11.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.640+0000 7fcc0dd7e740 1 -- 172.21.15.86:0/1566178563 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcbfc016070 con 0x556ae2a93ce0 2024-04-21T08:43:11.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.640+0000 7fcbe3fff640 1 -- 172.21.15.86:0/1566178563 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fcbfc012750 con 0x556ae2a93ce0 2024-04-21T08:43:11.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.640+0000 7fcbe3fff640 1 -- 172.21.15.86:0/1566178563 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcbfc012950 con 0x556ae2a93ce0 2024-04-21T08:43:11.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.640+0000 7fcc0dd7e740 10 client.17640 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:11.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.640+0000 7fcc0dd7e740 1 -- 172.21.15.86:0/1566178563 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556ae2b17720 con 0x556ae2a93ce0 2024-04-21T08:43:11.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.640+0000 7fcbe3fff640 1 -- 172.21.15.86:0/1566178563 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcbfc016070 con 0x556ae2a93ce0 2024-04-21T08:43:11.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.641+0000 7fcbe3fff640 1 -- 172.21.15.86:0/1566178563 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcbfc02e070 con 0x556ae2a93ce0 2024-04-21T08:43:11.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.641+0000 7fcc0dd7e740 10 client.17640 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:11.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.641+0000 7fcc0dd7e740 10 client.17640 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.641+0000 7fcc0dd7e740 20 client.17640 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.641+0000 7fcc0dd7e740 1 --2- 172.21.15.86:0/1566178563 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x556ae2a9ce40 0x556ae2b19d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.641+0000 7fcc0dd7e740 4 client.17640 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:43:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.641+0000 7fcc0dd7e740 1 -- 172.21.15.86:0/1566178563 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556ae29ffa80 con 0x556ae2a9ce40 2024-04-21T08:43:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.641+0000 7fcc08e4f640 1 --2- 172.21.15.86:0/1566178563 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x556ae2a9ce40 0x556ae2b19d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.641+0000 7fcbe3fff640 10 client.17640 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.641+0000 7fcc08e4f640 1 --2- 172.21.15.86:0/1566178563 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x556ae2a9ce40 0x556ae2b19d90 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e506d68bb3c723e2 server_cookie=5858a134e41e198a in_seq=0 out_seq=0 2024-04-21T08:43:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.641+0000 7fcbe3fff640 1 -- 172.21.15.86:0/1566178563 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556ae29ffa80 con 0x556ae2a9ce40 2024-04-21T08:43:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.641+0000 7fcbe3fff640 10 client.17640 handle_command_reply: tid=0 2024-04-21T08:43:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.641+0000 7fcc0dd7e740 1 client.17640 shutdown 2024-04-21T08:43:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.641+0000 7fcc0dd7e740 1 -- 172.21.15.86:0/1566178563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ae2a93ce0 msgr2=0x556ae2b16110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.641+0000 7fcc0dd7e740 1 --2- 172.21.15.86:0/1566178563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ae2a93ce0 0x556ae2b16110 secure :-1 s=READY pgs=1156 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc01e6e0 tx=0x7fcbfc027720 comp rx=0 tx=0).stop 2024-04-21T08:43:11.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.642+0000 7fcc0dd7e740 1 -- 172.21.15.86:0/1566178563 shutdown_connections 2024-04-21T08:43:11.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.642+0000 7fcc0dd7e740 1 --2- 172.21.15.86:0/1566178563 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x556ae2a9ce40 0x556ae2b19d90 crc :-1 s=READY pgs=156 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.642+0000 7fcc0dd7e740 1 --2- 172.21.15.86:0/1566178563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556ae2a93ce0 0x556ae2b16110 unknown :-1 s=CLOSED pgs=1156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.642+0000 7fcc0dd7e740 1 --2- 172.21.15.86:0/1566178563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556ae2a93330 0x556ae2aa29b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.642+0000 7fcc0dd7e740 1 --2- 172.21.15.86:0/1566178563 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556ae2a8dba0 0x556ae2aa2470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.642+0000 7fcbe3fff640 0 client.17640 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:11.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.642+0000 7fcc0dd7e740 1 -- 172.21.15.86:0/1566178563 >> 172.21.15.86:0/1566178563 conn(0x556ae2aa0c60 msgr2=0x556ae2a8cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:11.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.642+0000 7fcc0dd7e740 1 -- 172.21.15.86:0/1566178563 shutdown_connections 2024-04-21T08:43:11.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.642+0000 7fcc0dd7e740 1 -- 172.21.15.86:0/1566178563 wait complete. 2024-04-21T08:43:11.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.642+0000 7fcc0dd7e740 20 client.17640 trim_cache size 0 max 16384 2024-04-21T08:43:11.648 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:43:11.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.642+0000 7fcc0b0b3640 1 -- 172.21.15.86:0/1113046109 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fcbf40a3d20 msgr2=0x7fcbf40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.642+0000 7fcc0b0b3640 1 --2- 172.21.15.86:0/1113046109 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fcbf40a3d20 0x7fcbf40a61e0 secure :-1 s=READY pgs=179 cs=0 l=1 rev1=1 crypto rx=0x7fcbfc005fd0 tx=0x7fcbfc004a40 comp rx=0 tx=0).stop 2024-04-21T08:43:11.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.642+0000 7fcc0b0b3640 1 -- 172.21.15.86:0/1113046109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc041011a0 msgr2=0x7fcc04199560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:11.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.642+0000 7fcc0b0b3640 1 --2- 172.21.15.86:0/1113046109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc041011a0 0x7fcc04199560 secure :-1 s=READY pgs=1152 cs=0 l=1 rev1=1 crypto rx=0x7fcbf8009230 tx=0x7fcbf80045e0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.642+0000 7fcc0b0b3640 1 -- 172.21.15.86:0/1113046109 shutdown_connections 2024-04-21T08:43:11.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.642+0000 7fcc0b0b3640 1 --2- 172.21.15.86:0/1113046109 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fcbf40a3d20 0x7fcbf40a61e0 unknown :-1 s=CLOSED pgs=179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.642+0000 7fcc0b0b3640 1 --2- 172.21.15.86:0/1113046109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcc04107030 0x7fcc041a45e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.642+0000 7fcc0b0b3640 1 --2- 172.21.15.86:0/1113046109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcc04101b40 0x7fcc04199aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.642+0000 7fcc0b0b3640 1 --2- 172.21.15.86:0/1113046109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcc041011a0 0x7fcc04199560 unknown :-1 s=CLOSED pgs=1152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:11.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.642+0000 7fcc0b0b3640 1 -- 172.21.15.86:0/1113046109 >> 172.21.15.86:0/1113046109 conn(0x7fcc04076c50 msgr2=0x7fcc040ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:11.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.642+0000 7fcc0b0b3640 1 -- 172.21.15.86:0/1113046109 shutdown_connections 2024-04-21T08:43:11.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:11.642+0000 7fcc0b0b3640 1 -- 172.21.15.86:0/1113046109 wait complete. 2024-04-21T08:43:11.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:11 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/227440889' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:43:11.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:11 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/227440889' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:43:11.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:11 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/227440889' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:43:12.653 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:43:12.653 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:43:12.654 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:43:12.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:12 smithi179 ceph-mon[31829]: pgmap v135: 129 pgs: 129 active+clean; 5.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 58 KiB/s wr, 31 op/s 2024-04-21T08:43:12.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.769+0000 7fbbb9ae0640 1 -- 172.21.15.86:0/699530342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb4102b40 msgr2=0x7fbbb4106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:12.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.769+0000 7fbbb9ae0640 1 --2- 172.21.15.86:0/699530342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb4102b40 0x7fbbb4106b80 secure :-1 s=READY pgs=1157 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c009900 tx=0x7fbb9c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:43:12.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.769+0000 7fbbb9ae0640 1 -- 172.21.15.86:0/699530342 shutdown_connections 2024-04-21T08:43:12.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.769+0000 7fbbb9ae0640 1 --2- 172.21.15.86:0/699530342 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbb4107260 0x7fbbb410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.769+0000 7fbbb9ae0640 1 --2- 172.21.15.86:0/699530342 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb4102b40 0x7fbbb4106b80 unknown :-1 s=CLOSED pgs=1157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.769+0000 7fbbb9ae0640 1 --2- 172.21.15.86:0/699530342 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbb4102190 0x7fbbb4102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.769+0000 7fbbb9ae0640 1 -- 172.21.15.86:0/699530342 >> 172.21.15.86:0/699530342 conn(0x7fbbb40793a0 msgr2=0x7fbbb40797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:12.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.769+0000 7fbbb9ae0640 1 -- 172.21.15.86:0/699530342 shutdown_connections 2024-04-21T08:43:12.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.769+0000 7fbbb9ae0640 1 -- 172.21.15.86:0/699530342 wait complete. 2024-04-21T08:43:12.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.770+0000 7fbbb9ae0640 1 Processor -- start 2024-04-21T08:43:12.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.770+0000 7fbbb9ae0640 1 -- start start 2024-04-21T08:43:12.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.770+0000 7fbbb9ae0640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbb4102190 0x7fbbb419e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:12.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.770+0000 7fbbb9ae0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb4107260 0x7fbbb419e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:12.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.770+0000 7fbbb9ae0640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbb41985a0 0x7fbbb4198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:12.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.770+0000 7fbbb9ae0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb41121a0 con 0x7fbbb4107260 2024-04-21T08:43:12.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.770+0000 7fbbb9ae0640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb4112020 con 0x7fbbb4102190 2024-04-21T08:43:12.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.770+0000 7fbbb9ae0640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb4112320 con 0x7fbbb41985a0 2024-04-21T08:43:12.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.770+0000 7fbbb2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb4107260 0x7fbbb419e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:12.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.770+0000 7fbbb2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb4107260 0x7fbbb419e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51216/0 (socket says 172.21.15.86:51216) 2024-04-21T08:43:12.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.770+0000 7fbbb2ffd640 1 -- 172.21.15.86:0/2425569345 learned_addr learned my addr 172.21.15.86:0/2425569345 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:12.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.770+0000 7fbbb37fe640 1 --2- 172.21.15.86:0/2425569345 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbb4102190 0x7fbbb419e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:12.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.771+0000 7fbbb3fff640 1 --2- 172.21.15.86:0/2425569345 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbb41985a0 0x7fbbb4198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:12.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.771+0000 7fbbb2ffd640 1 -- 172.21.15.86:0/2425569345 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbb41985a0 msgr2=0x7fbbb4198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:12.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.771+0000 7fbbb2ffd640 1 --2- 172.21.15.86:0/2425569345 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbb41985a0 0x7fbbb4198a50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.771+0000 7fbbb2ffd640 1 -- 172.21.15.86:0/2425569345 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbb4102190 msgr2=0x7fbbb419e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:12.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.771+0000 7fbbb2ffd640 1 --2- 172.21.15.86:0/2425569345 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbb4102190 0x7fbbb419e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.771+0000 7fbbb2ffd640 1 -- 172.21.15.86:0/2425569345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbb41990c0 con 0x7fbbb4107260 2024-04-21T08:43:12.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.771+0000 7fbbb2ffd640 1 --2- 172.21.15.86:0/2425569345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb4107260 0x7fbbb419e930 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c036040 tx=0x7fbb9c02c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:12.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.771+0000 7fbbb0ff9640 1 -- 172.21.15.86:0/2425569345 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c043020 con 0x7fbbb4107260 2024-04-21T08:43:12.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.771+0000 7fbbb0ff9640 1 -- 172.21.15.86:0/2425569345 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb9c02cbe0 con 0x7fbbb4107260 2024-04-21T08:43:12.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.771+0000 7fbbb9ae0640 1 -- 172.21.15.86:0/2425569345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbb4199350 con 0x7fbbb4107260 2024-04-21T08:43:12.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.771+0000 7fbbb0ff9640 1 -- 172.21.15.86:0/2425569345 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c003bf0 con 0x7fbbb4107260 2024-04-21T08:43:12.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.771+0000 7fbbb9ae0640 1 -- 172.21.15.86:0/2425569345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbb41a55a0 con 0x7fbbb4107260 2024-04-21T08:43:12.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.773+0000 7fbbb0ff9640 1 -- 172.21.15.86:0/2425569345 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbb9c046020 con 0x7fbbb4107260 2024-04-21T08:43:12.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.773+0000 7fbbbc7ae740 1 Processor -- start 2024-04-21T08:43:12.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.774+0000 7fbbbc7ae740 1 -- start start 2024-04-21T08:43:12.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.774+0000 7fbbb0ff9640 1 --2- 172.21.15.86:0/2425569345 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbba00a3d20 0x7fbba00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:12.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.774+0000 7fbbbc7ae740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636445a48e0 0x5636445a4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:12.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.774+0000 7fbbbc7ae740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636445a5290 0x5636445ad840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:12.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.774+0000 7fbbbc7ae740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636445ade10 0x5636445b0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:12.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.774+0000 7fbbbc7ae740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563644508f00 con 0x5636445a5290 2024-04-21T08:43:12.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.774+0000 7fbbbc7ae740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5636444a6590 con 0x5636445a48e0 2024-04-21T08:43:12.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.774+0000 7fbbbc7ae740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563644504ec0 con 0x5636445ade10 2024-04-21T08:43:12.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.774+0000 7fbbb2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636445a5290 0x5636445ad840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:12.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.774+0000 7fbbb3fff640 1 --2- 172.21.15.86:0/2425569345 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbba00a3d20 0x7fbba00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:12.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.774+0000 7fbbb2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636445a5290 0x5636445ad840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51232/0 (socket says 172.21.15.86:51232) 2024-04-21T08:43:12.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.774+0000 7fbbb2ffd640 1 -- 172.21.15.86:0/405186769 learned_addr learned my addr 172.21.15.86:0/405186769 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:12.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.774+0000 7fbbb0ff9640 1 -- 172.21.15.86:0/2425569345 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fbb9c0f2d00 con 0x7fbbb4107260 2024-04-21T08:43:12.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.774+0000 7fbbb3fff640 1 --2- 172.21.15.86:0/405186769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636445ade10 0x5636445b0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:12.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.774+0000 7fbbb37fe640 1 --2- 172.21.15.86:0/405186769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636445a48e0 0x5636445a4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:12.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.774+0000 7fbbb2ffd640 1 -- 172.21.15.86:0/405186769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636445ade10 msgr2=0x5636445b0260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:12.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.774+0000 7fbbb2ffd640 1 --2- 172.21.15.86:0/405186769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636445ade10 0x5636445b0260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.774+0000 7fbbb2ffd640 1 -- 172.21.15.86:0/405186769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636445a48e0 msgr2=0x5636445a4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:12.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.774+0000 7fbbb2ffd640 1 --2- 172.21.15.86:0/405186769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636445a48e0 0x5636445a4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.774+0000 7fbbb2ffd640 1 -- 172.21.15.86:0/405186769 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636442e37e0 con 0x5636445a5290 2024-04-21T08:43:12.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.774+0000 7fbbb37fe640 1 --2- 172.21.15.86:0/405186769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636445a48e0 0x5636445a4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:12.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.774+0000 7fbbb2ffd640 1 --2- 172.21.15.86:0/405186769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636445a5290 0x5636445ad840 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0f0250 tx=0x7fbb9c03ac10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a07f32917baa134 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:12.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.774+0000 7fbbb3fff640 1 --2- 172.21.15.86:0/2425569345 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbba00a3d20 0x7fbba00a61e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009fd0 tx=0x7fbbac009c60 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:12.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.775+0000 7fbba9ffb640 1 -- 172.21.15.86:0/405186769 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c0be580 con 0x5636445a5290 2024-04-21T08:43:12.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.775+0000 7fbba9ffb640 1 -- 172.21.15.86:0/405186769 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb9c0bf140 con 0x5636445a5290 2024-04-21T08:43:12.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.775+0000 7fbba9ffb640 1 -- 172.21.15.86:0/405186769 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb9c0df380 con 0x5636445a5290 2024-04-21T08:43:12.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.775+0000 7fbbbc7ae740 1 -- 172.21.15.86:0/405186769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636445a5290 msgr2=0x5636445ad840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:12.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.775+0000 7fbbbc7ae740 1 --2- 172.21.15.86:0/405186769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636445a5290 0x5636445ad840 secure :-1 s=READY pgs=1159 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c0f0250 tx=0x7fbb9c03ac10 comp rx=0 tx=0).stop 2024-04-21T08:43:12.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.775+0000 7fbbbc7ae740 1 -- 172.21.15.86:0/405186769 shutdown_connections 2024-04-21T08:43:12.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.775+0000 7fbbbc7ae740 1 --2- 172.21.15.86:0/405186769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636445ade10 0x5636445b0260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.775+0000 7fbbbc7ae740 1 --2- 172.21.15.86:0/405186769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636445a5290 0x5636445ad840 unknown :-1 s=CLOSED pgs=1159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.775+0000 7fbbbc7ae740 1 --2- 172.21.15.86:0/405186769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636445a48e0 0x5636445a4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.775+0000 7fbbbc7ae740 1 -- 172.21.15.86:0/405186769 >> 172.21.15.86:0/405186769 conn(0x56364459efb0 msgr2=0x56364459f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:12.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.775+0000 7fbbbc7ae740 1 -- 172.21.15.86:0/405186769 shutdown_connections 2024-04-21T08:43:12.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.775+0000 7fbbbc7ae740 1 -- 172.21.15.86:0/405186769 wait complete. 2024-04-21T08:43:12.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.776+0000 7fbbbc7ae740 1 -- start start 2024-04-21T08:43:12.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.776+0000 7fbbbc7ae740 1 Processor -- start 2024-04-21T08:43:12.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.776+0000 7fbbbc7ae740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636445a48e0 0x5636445aae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:12.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.776+0000 7fbbbc7ae740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636445a5290 0x5636445ab360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:12.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.776+0000 7fbbbc7ae740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636445ade10 0x5636445ab8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:12.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.776+0000 7fbbbc7ae740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5636445b0b40 con 0x5636445ade10 2024-04-21T08:43:12.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.776+0000 7fbbbc7ae740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5636444c2580 con 0x5636445a48e0 2024-04-21T08:43:12.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.776+0000 7fbbbc7ae740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5636445b0cc0 con 0x5636445a5290 2024-04-21T08:43:12.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.776+0000 7fbbbc7ae740 10 client.0 _ll_register_callbacks cb 0x5636444c8650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:12.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.776+0000 7fbbb3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636445ade10 0x5636445ab8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:12.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.776+0000 7fbbb3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636445ade10 0x5636445ab8a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51242/0 (socket says 172.21.15.86:51242) 2024-04-21T08:43:12.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.776+0000 7fbbb3fff640 1 -- 172.21.15.86:0/490529065 learned_addr learned my addr 172.21.15.86:0/490529065 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:12.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.776+0000 7fbbb37fe640 1 --2- 172.21.15.86:0/490529065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636445a48e0 0x5636445aae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:12.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.776+0000 7fbbb2ffd640 1 --2- 172.21.15.86:0/490529065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636445a5290 0x5636445ab360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:12.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.776+0000 7fbbb3fff640 1 -- 172.21.15.86:0/490529065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636445a5290 msgr2=0x5636445ab360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:12.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.777+0000 7fbbb3fff640 1 --2- 172.21.15.86:0/490529065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636445a5290 0x5636445ab360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.777+0000 7fbbb3fff640 1 -- 172.21.15.86:0/490529065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636445a48e0 msgr2=0x5636445aae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:12.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.777+0000 7fbbb3fff640 1 --2- 172.21.15.86:0/490529065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636445a48e0 0x5636445aae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.777+0000 7fbbb3fff640 1 -- 172.21.15.86:0/490529065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563644343f70 con 0x5636445ade10 2024-04-21T08:43:12.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.777+0000 7fbbb2ffd640 1 --2- 172.21.15.86:0/490529065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636445a5290 0x5636445ab360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:12.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.777+0000 7fbbb3fff640 1 --2- 172.21.15.86:0/490529065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636445ade10 0x5636445ab8a0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fbbac014340 tx=0x7fbbac004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=72f28f37d968cc57 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:12.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.777+0000 7fbb92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:12.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.777+0000 7fbb92ffd640 1 -- 172.21.15.86:0/490529065 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac012b70 con 0x5636445ade10 2024-04-21T08:43:12.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.777+0000 7fbb92ffd640 1 -- 172.21.15.86:0/490529065 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fbbac012d10 con 0x5636445ade10 2024-04-21T08:43:12.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.777+0000 7fbbbc7ae740 1 -- 172.21.15.86:0/490529065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb9c0bf140 con 0x5636445ade10 2024-04-21T08:43:12.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.777+0000 7fbb92ffd640 1 -- 172.21.15.86:0/490529065 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbbac032050 con 0x5636445ade10 2024-04-21T08:43:12.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.777+0000 7fbbbc7ae740 10 client.17652 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:12.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.777+0000 7fbbbc7ae740 1 -- 172.21.15.86:0/490529065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5636441db170 con 0x5636445ade10 2024-04-21T08:43:12.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.777+0000 7fbb92ffd640 1 -- 172.21.15.86:0/490529065 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb9c0bf140 con 0x5636445ade10 2024-04-21T08:43:12.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.777+0000 7fbb92ffd640 1 -- 172.21.15.86:0/490529065 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbbac0213f0 con 0x5636445ade10 2024-04-21T08:43:12.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.777+0000 7fbbbc7ae740 10 client.17652 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:12.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.777+0000 7fbbbc7ae740 10 client.17652 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:12.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.777+0000 7fbbbc7ae740 20 client.17652 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:12.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.777+0000 7fbbbc7ae740 1 --2- 172.21.15.86:0/490529065 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5636445be140 0x5636445c0560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:12.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.777+0000 7fbbbc7ae740 4 client.17652 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:12.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.778+0000 7fbbbc7ae740 1 -- 172.21.15.86:0/490529065 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563644516920 con 0x5636445be140 2024-04-21T08:43:12.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.778+0000 7fbbb37fe640 1 --2- 172.21.15.86:0/490529065 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5636445be140 0x5636445c0560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:12.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.778+0000 7fbb92ffd640 10 client.17652 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:12.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.778+0000 7fbbb37fe640 1 --2- 172.21.15.86:0/490529065 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5636445be140 0x5636445c0560 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bb25215198cb64dc server_cookie=1ed89528beef7a25 in_seq=0 out_seq=0 2024-04-21T08:43:12.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.779+0000 7fbb92ffd640 1 -- 172.21.15.86:0/490529065 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563644516920 con 0x5636445be140 2024-04-21T08:43:12.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.779+0000 7fbb92ffd640 10 client.17652 handle_command_reply: tid=0 2024-04-21T08:43:12.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.779+0000 7fbbbc7ae740 1 client.17652 shutdown 2024-04-21T08:43:12.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.779+0000 7fbbbc7ae740 1 -- 172.21.15.86:0/490529065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636445ade10 msgr2=0x5636445ab8a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:12.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.779+0000 7fbbbc7ae740 1 --2- 172.21.15.86:0/490529065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636445ade10 0x5636445ab8a0 secure :-1 s=READY pgs=1160 cs=0 l=1 rev1=1 crypto rx=0x7fbbac014340 tx=0x7fbbac004ab0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.779+0000 7fbbbc7ae740 1 -- 172.21.15.86:0/490529065 shutdown_connections 2024-04-21T08:43:12.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.779+0000 7fbbbc7ae740 1 --2- 172.21.15.86:0/490529065 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5636445be140 0x5636445c0560 crc :-1 s=READY pgs=157 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.779+0000 7fbbbc7ae740 1 --2- 172.21.15.86:0/490529065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636445ade10 0x5636445ab8a0 unknown :-1 s=CLOSED pgs=1160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.779+0000 7fbb92ffd640 0 client.17652 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:12.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.779+0000 7fbbbc7ae740 1 --2- 172.21.15.86:0/490529065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636445a5290 0x5636445ab360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.779+0000 7fbbbc7ae740 1 --2- 172.21.15.86:0/490529065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636445a48e0 0x5636445aae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.779+0000 7fbbbc7ae740 1 -- 172.21.15.86:0/490529065 >> 172.21.15.86:0/490529065 conn(0x56364459efb0 msgr2=0x5636445a0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:12.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.779+0000 7fbbbc7ae740 1 -- 172.21.15.86:0/490529065 shutdown_connections 2024-04-21T08:43:12.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.779+0000 7fbbbc7ae740 1 -- 172.21.15.86:0/490529065 wait complete. 2024-04-21T08:43:12.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.779+0000 7fbbbc7ae740 20 client.17652 trim_cache size 0 max 16384 2024-04-21T08:43:12.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.796+0000 7fbbbc7ae740 1 Processor -- start 2024-04-21T08:43:12.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.797+0000 7fbbbc7ae740 1 -- start start 2024-04-21T08:43:12.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.797+0000 7fbbbc7ae740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636445a48e0 0x5636445b9970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:12.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.797+0000 7fbbbc7ae740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636445a5290 0x5636445c5e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:12.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.797+0000 7fbbbc7ae740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636445ade10 0x5636445c6370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:12.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.797+0000 7fbbbc7ae740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb41af2a0 con 0x5636445ade10 2024-04-21T08:43:12.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.797+0000 7fbbbc7ae740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb41b0de0 con 0x5636445a5290 2024-04-21T08:43:12.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.797+0000 7fbbbc7ae740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb41af970 con 0x5636445a48e0 2024-04-21T08:43:12.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.797+0000 7fbbb3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636445ade10 0x5636445c6370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:12.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.797+0000 7fbbb3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636445ade10 0x5636445c6370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51248/0 (socket says 172.21.15.86:51248) 2024-04-21T08:43:12.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.797+0000 7fbbb3fff640 1 -- 172.21.15.86:0/3065160947 learned_addr learned my addr 172.21.15.86:0/3065160947 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:12.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.797+0000 7fbbb37fe640 1 --2- 172.21.15.86:0/3065160947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636445a48e0 0x5636445b9970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:12.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.797+0000 7fbbb3fff640 1 -- 172.21.15.86:0/3065160947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636445a48e0 msgr2=0x5636445b9970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:12.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.797+0000 7fbbb2ffd640 1 --2- 172.21.15.86:0/3065160947 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636445a5290 0x5636445c5e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:12.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.797+0000 7fbbb3fff640 1 --2- 172.21.15.86:0/3065160947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636445a48e0 0x5636445b9970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.797+0000 7fbbb3fff640 1 -- 172.21.15.86:0/3065160947 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636445a5290 msgr2=0x5636445c5e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:12.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.797+0000 7fbbb3fff640 1 --2- 172.21.15.86:0/3065160947 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636445a5290 0x5636445c5e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.797+0000 7fbbb3fff640 1 -- 172.21.15.86:0/3065160947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5636445a5030 con 0x5636445ade10 2024-04-21T08:43:12.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.797+0000 7fbbb37fe640 1 --2- 172.21.15.86:0/3065160947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636445a48e0 0x5636445b9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:12.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.797+0000 7fbbb3fff640 1 --2- 172.21.15.86:0/3065160947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636445ade10 0x5636445c6370 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009490 tx=0x7fbbac000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b9cca6476e4aeff server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:12.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.797+0000 7fbba97fa640 1 -- 172.21.15.86:0/3065160947 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac03a070 con 0x5636445ade10 2024-04-21T08:43:12.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.797+0000 7fbba97fa640 1 -- 172.21.15.86:0/3065160947 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbac014370 con 0x5636445ade10 2024-04-21T08:43:12.803 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.797+0000 7fbba97fa640 1 -- 172.21.15.86:0/3065160947 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbac012e60 con 0x5636445ade10 2024-04-21T08:43:12.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.798+0000 7fbbbc7ae740 1 -- 172.21.15.86:0/3065160947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636445ade10 msgr2=0x5636445c6370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:12.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.798+0000 7fbbbc7ae740 1 --2- 172.21.15.86:0/3065160947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636445ade10 0x5636445c6370 secure :-1 s=READY pgs=1161 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009490 tx=0x7fbbac000f40 comp rx=0 tx=0).stop 2024-04-21T08:43:12.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.798+0000 7fbbbc7ae740 1 -- 172.21.15.86:0/3065160947 shutdown_connections 2024-04-21T08:43:12.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.798+0000 7fbbbc7ae740 1 --2- 172.21.15.86:0/3065160947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636445ade10 0x5636445c6370 unknown :-1 s=CLOSED pgs=1161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.798+0000 7fbbbc7ae740 1 --2- 172.21.15.86:0/3065160947 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636445a5290 0x5636445c5e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.798+0000 7fbbbc7ae740 1 --2- 172.21.15.86:0/3065160947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636445a48e0 0x5636445b9970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.798+0000 7fbbbc7ae740 1 -- 172.21.15.86:0/3065160947 >> 172.21.15.86:0/3065160947 conn(0x56364459efb0 msgr2=0x5636445acee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:12.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.798+0000 7fbbbc7ae740 1 -- 172.21.15.86:0/3065160947 shutdown_connections 2024-04-21T08:43:12.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.798+0000 7fbbbc7ae740 1 -- 172.21.15.86:0/3065160947 wait complete. 2024-04-21T08:43:12.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.798+0000 7fbbbc7ae740 1 -- start start 2024-04-21T08:43:12.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.799+0000 7fbbbc7ae740 1 Processor -- start 2024-04-21T08:43:12.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.799+0000 7fbbbc7ae740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636445a48e0 0x5636445b6000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:12.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.799+0000 7fbbbc7ae740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636445a5290 0x5636445b2380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:12.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.799+0000 7fbbbc7ae740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636445ade10 0x5636445b28c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.799+0000 7fbbbc7ae740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb41b0710 con 0x5636445a48e0 2024-04-21T08:43:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.799+0000 7fbbbc7ae740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb41b0040 con 0x5636445a5290 2024-04-21T08:43:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.799+0000 7fbbbc7ae740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbbb41ad260 con 0x5636445ade10 2024-04-21T08:43:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.799+0000 7fbbbc7ae740 10 client.0 _ll_register_callbacks cb 0x7fbbac0093c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.799+0000 7fbbb37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636445a48e0 0x5636445b6000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.799+0000 7fbbb37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636445a48e0 0x5636445b6000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51260/0 (socket says 172.21.15.86:51260) 2024-04-21T08:43:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.799+0000 7fbbb37fe640 1 -- 172.21.15.86:0/3968298554 learned_addr learned my addr 172.21.15.86:0/3968298554 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.799+0000 7fbbb3fff640 1 --2- 172.21.15.86:0/3968298554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636445ade10 0x5636445b28c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.799+0000 7fbbb2ffd640 1 --2- 172.21.15.86:0/3968298554 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636445a5290 0x5636445b2380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.799+0000 7fbbb37fe640 1 -- 172.21.15.86:0/3968298554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636445ade10 msgr2=0x5636445b28c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.799+0000 7fbbb37fe640 1 --2- 172.21.15.86:0/3968298554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636445ade10 0x5636445b28c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.799+0000 7fbbb37fe640 1 -- 172.21.15.86:0/3968298554 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636445a5290 msgr2=0x5636445b2380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:12.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.799+0000 7fbbb37fe640 1 --2- 172.21.15.86:0/3968298554 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636445a5290 0x5636445b2380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.799+0000 7fbbb37fe640 1 -- 172.21.15.86:0/3968298554 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5636445b2e80 con 0x5636445a48e0 2024-04-21T08:43:12.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.799+0000 7fbbb2ffd640 1 --2- 172.21.15.86:0/3968298554 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636445a5290 0x5636445b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:12.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.799+0000 7fbbb37fe640 1 --2- 172.21.15.86:0/3968298554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636445a48e0 0x5636445b6000 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fbba400b620 tx=0x7fbba40035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec636c11350db486 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:12.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.799+0000 7fbb92ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:12.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.800+0000 7fbb92ffd640 1 -- 172.21.15.86:0/3968298554 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbba401d5b0 con 0x5636445a48e0 2024-04-21T08:43:12.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.800+0000 7fbb92ffd640 1 -- 172.21.15.86:0/3968298554 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fbba401d750 con 0x5636445a48e0 2024-04-21T08:43:12.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.800+0000 7fbbbc7ae740 1 -- 172.21.15.86:0/3968298554 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbbac014370 con 0x5636445a48e0 2024-04-21T08:43:12.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.800+0000 7fbb92ffd640 1 -- 172.21.15.86:0/3968298554 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbba402c050 con 0x5636445a48e0 2024-04-21T08:43:12.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.800+0000 7fbbbc7ae740 10 client.17658 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:12.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.800+0000 7fbbbc7ae740 1 -- 172.21.15.86:0/3968298554 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563644625470 con 0x5636445a48e0 2024-04-21T08:43:12.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.800+0000 7fbb92ffd640 1 -- 172.21.15.86:0/3968298554 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbac014370 con 0x5636445a48e0 2024-04-21T08:43:12.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.800+0000 7fbb92ffd640 1 -- 172.21.15.86:0/3968298554 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbba4004b70 con 0x5636445a48e0 2024-04-21T08:43:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.800+0000 7fbbbc7ae740 10 client.17658 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.800+0000 7fbbbc7ae740 10 client.17658 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.800+0000 7fbbbc7ae740 20 client.17658 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.800+0000 7fbbbc7ae740 1 --2- 172.21.15.86:0/3968298554 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5636445be140 0x563644627c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.800+0000 7fbbbc7ae740 4 client.17658 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:43:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.800+0000 7fbbbc7ae740 1 -- 172.21.15.86:0/3968298554 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x563644510790 con 0x5636445be140 2024-04-21T08:43:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.800+0000 7fbbb2ffd640 1 --2- 172.21.15.86:0/3968298554 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5636445be140 0x563644627c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.801+0000 7fbbb2ffd640 1 --2- 172.21.15.86:0/3968298554 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5636445be140 0x563644627c30 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de72d10a0e2c4134 server_cookie=f1c78551fce949d8 in_seq=0 out_seq=0 2024-04-21T08:43:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.801+0000 7fbb92ffd640 10 client.17658 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.802+0000 7fbb92ffd640 1 -- 172.21.15.86:0/3968298554 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x563644510790 con 0x5636445be140 2024-04-21T08:43:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.802+0000 7fbb92ffd640 10 client.17658 handle_command_reply: tid=0 2024-04-21T08:43:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.802+0000 7fbbbc7ae740 1 client.17658 shutdown 2024-04-21T08:43:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.802+0000 7fbbbc7ae740 1 -- 172.21.15.86:0/3968298554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636445a48e0 msgr2=0x5636445b6000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.802+0000 7fbbbc7ae740 1 --2- 172.21.15.86:0/3968298554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636445a48e0 0x5636445b6000 secure :-1 s=READY pgs=1162 cs=0 l=1 rev1=1 crypto rx=0x7fbba400b620 tx=0x7fbba40035f0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.802+0000 7fbbbc7ae740 1 -- 172.21.15.86:0/3968298554 shutdown_connections 2024-04-21T08:43:12.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.802+0000 7fbbbc7ae740 1 --2- 172.21.15.86:0/3968298554 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5636445be140 0x563644627c30 crc :-1 s=READY pgs=158 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.802+0000 7fbbbc7ae740 1 --2- 172.21.15.86:0/3968298554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5636445ade10 0x5636445b28c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.802+0000 7fbbbc7ae740 1 --2- 172.21.15.86:0/3968298554 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5636445a5290 0x5636445b2380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.802+0000 7fbb92ffd640 0 client.17658 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:12.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.802+0000 7fbbbc7ae740 1 --2- 172.21.15.86:0/3968298554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5636445a48e0 0x5636445b6000 unknown :-1 s=CLOSED pgs=1162 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.802+0000 7fbbbc7ae740 1 -- 172.21.15.86:0/3968298554 >> 172.21.15.86:0/3968298554 conn(0x56364459efb0 msgr2=0x56364459d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:12.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.802+0000 7fbbbc7ae740 1 -- 172.21.15.86:0/3968298554 shutdown_connections 2024-04-21T08:43:12.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.802+0000 7fbbbc7ae740 1 -- 172.21.15.86:0/3968298554 wait complete. 2024-04-21T08:43:12.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.802+0000 7fbbbc7ae740 20 client.17658 trim_cache size 0 max 16384 2024-04-21T08:43:12.808 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:43:12.808 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-21T08:43:12.808 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "95ef26af-0c67-4fa5-ba7b-acc08e492709", 2024-04-21T08:43:12.809 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-21T08:43:12.809 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:43:12.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.803+0000 7fbbb9ae0640 1 -- 172.21.15.86:0/2425569345 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbba00a3d20 msgr2=0x7fbba00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:12.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.803+0000 7fbbb9ae0640 1 --2- 172.21.15.86:0/2425569345 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbba00a3d20 0x7fbba00a61e0 secure :-1 s=READY pgs=180 cs=0 l=1 rev1=1 crypto rx=0x7fbbac009fd0 tx=0x7fbbac009c60 comp rx=0 tx=0).stop 2024-04-21T08:43:12.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.803+0000 7fbbb9ae0640 1 -- 172.21.15.86:0/2425569345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb4107260 msgr2=0x7fbbb419e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:12.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.803+0000 7fbbb9ae0640 1 --2- 172.21.15.86:0/2425569345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb4107260 0x7fbbb419e930 secure :-1 s=READY pgs=1158 cs=0 l=1 rev1=1 crypto rx=0x7fbb9c036040 tx=0x7fbb9c02c7e0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.803+0000 7fbbb9ae0640 1 -- 172.21.15.86:0/2425569345 shutdown_connections 2024-04-21T08:43:12.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.803+0000 7fbbb9ae0640 1 --2- 172.21.15.86:0/2425569345 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbba00a3d20 0x7fbba00a61e0 unknown :-1 s=CLOSED pgs=180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.803+0000 7fbbb9ae0640 1 --2- 172.21.15.86:0/2425569345 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbb41985a0 0x7fbbb4198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.803+0000 7fbbb9ae0640 1 --2- 172.21.15.86:0/2425569345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbb4107260 0x7fbbb419e930 unknown :-1 s=CLOSED pgs=1158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.803+0000 7fbbb9ae0640 1 --2- 172.21.15.86:0/2425569345 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbb4102190 0x7fbbb419e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.803+0000 7fbbb9ae0640 1 -- 172.21.15.86:0/2425569345 >> 172.21.15.86:0/2425569345 conn(0x7fbbb40793a0 msgr2=0x7fbbb4100000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:12.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.803+0000 7fbbb9ae0640 1 -- 172.21.15.86:0/2425569345 shutdown_connections 2024-04-21T08:43:12.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.803+0000 7fbbb9ae0640 1 -- 172.21.15.86:0/2425569345 wait complete. 2024-04-21T08:43:12.812 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:12 smithi086 ceph-mon[27939]: pgmap v135: 129 pgs: 129 active+clean; 5.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 58 KiB/s wr, 31 op/s 2024-04-21T08:43:12.813 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:43:12.813 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:43:12.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.914+0000 7f93ee3fb640 1 -- 172.21.15.86:0/1659671812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93e8102b50 msgr2=0x7f93e8106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:12.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.914+0000 7f93ee3fb640 1 --2- 172.21.15.86:0/1659671812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93e8102b50 0x7f93e8106b90 secure :-1 s=READY pgs=1163 cs=0 l=1 rev1=1 crypto rx=0x7f93d0009900 tx=0x7f93d002bc30 comp rx=0 tx=0).stop 2024-04-21T08:43:12.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.914+0000 7f93ee3fb640 1 -- 172.21.15.86:0/1659671812 shutdown_connections 2024-04-21T08:43:12.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.914+0000 7f93ee3fb640 1 --2- 172.21.15.86:0/1659671812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93e8107270 0x7f93e810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.914+0000 7f93ee3fb640 1 --2- 172.21.15.86:0/1659671812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93e8102b50 0x7f93e8106b90 unknown :-1 s=CLOSED pgs=1163 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.914+0000 7f93ee3fb640 1 --2- 172.21.15.86:0/1659671812 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93e81021a0 0x7f93e8102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.914+0000 7f93ee3fb640 1 -- 172.21.15.86:0/1659671812 >> 172.21.15.86:0/1659671812 conn(0x7f93e80fda40 msgr2=0x7f93e80ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:12.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.914+0000 7f93ee3fb640 1 -- 172.21.15.86:0/1659671812 shutdown_connections 2024-04-21T08:43:12.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.914+0000 7f93ee3fb640 1 -- 172.21.15.86:0/1659671812 wait complete. 2024-04-21T08:43:12.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.914+0000 7f93ee3fb640 1 Processor -- start 2024-04-21T08:43:12.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.914+0000 7f93ee3fb640 1 -- start start 2024-04-21T08:43:12.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.915+0000 7f93ee3fb640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93e81021a0 0x7f93e819e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:12.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.915+0000 7f93ee3fb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93e8107270 0x7f93e819e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:12.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.915+0000 7f93ee3fb640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93e8198630 0x7f93e8198ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:12.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.915+0000 7f93ee3fb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93e81121b0 con 0x7f93e8107270 2024-04-21T08:43:12.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.915+0000 7f93ee3fb640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f93e8112030 con 0x7f93e81021a0 2024-04-21T08:43:12.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.915+0000 7f93ee3fb640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f93e8112330 con 0x7f93e8198630 2024-04-21T08:43:12.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.915+0000 7f93e77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93e8107270 0x7f93e819e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:12.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.915+0000 7f93e77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93e8107270 0x7f93e819e970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51276/0 (socket says 172.21.15.86:51276) 2024-04-21T08:43:12.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.915+0000 7f93e77fe640 1 -- 172.21.15.86:0/1785305728 learned_addr learned my addr 172.21.15.86:0/1785305728 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:12.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.915+0000 7f93ec998640 1 --2- 172.21.15.86:0/1785305728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93e8198630 0x7f93e8198ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:12.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.915+0000 7f93e7fff640 1 --2- 172.21.15.86:0/1785305728 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93e81021a0 0x7f93e819e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:12.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.915+0000 7f93e77fe640 1 -- 172.21.15.86:0/1785305728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93e8198630 msgr2=0x7f93e8198ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:12.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.915+0000 7f93e77fe640 1 --2- 172.21.15.86:0/1785305728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93e8198630 0x7f93e8198ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.915+0000 7f93e77fe640 1 -- 172.21.15.86:0/1785305728 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93e81021a0 msgr2=0x7f93e819e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:12.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.915+0000 7f93e77fe640 1 --2- 172.21.15.86:0/1785305728 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93e81021a0 0x7f93e819e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.915+0000 7f93e77fe640 1 -- 172.21.15.86:0/1785305728 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f93e8199150 con 0x7f93e8107270 2024-04-21T08:43:12.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.915+0000 7f93e77fe640 1 --2- 172.21.15.86:0/1785305728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93e8107270 0x7f93e819e970 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f93d0036040 tx=0x7f93d002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:12.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.916+0000 7f93e57fa640 1 -- 172.21.15.86:0/1785305728 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d0043020 con 0x7f93e8107270 2024-04-21T08:43:12.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.916+0000 7f93e57fa640 1 -- 172.21.15.86:0/1785305728 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93d002cbe0 con 0x7f93e8107270 2024-04-21T08:43:12.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.916+0000 7f93ee3fb640 1 -- 172.21.15.86:0/1785305728 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f93e81993e0 con 0x7f93e8107270 2024-04-21T08:43:12.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.916+0000 7f93e57fa640 1 -- 172.21.15.86:0/1785305728 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d0003bf0 con 0x7f93e8107270 2024-04-21T08:43:12.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.916+0000 7f93ee3fb640 1 -- 172.21.15.86:0/1785305728 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93e81a5630 con 0x7f93e8107270 2024-04-21T08:43:12.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.918+0000 7f93e57fa640 1 -- 172.21.15.86:0/1785305728 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f93d0046020 con 0x7f93e8107270 2024-04-21T08:43:12.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.918+0000 7f93f0fe1740 1 Processor -- start 2024-04-21T08:43:12.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.918+0000 7f93f0fe1740 1 -- start start 2024-04-21T08:43:12.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.918+0000 7f93f0fe1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae36104390 0x55ae36104770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:12.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.918+0000 7f93f0fe1740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae36104d40 0x55ae36111950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:12.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.918+0000 7f93f0fe1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36111e90 0x55ae36114280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:12.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.918+0000 7f93f0fe1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ae360691e0 con 0x55ae36104390 2024-04-21T08:43:12.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.918+0000 7f93f0fe1740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55ae36006390 con 0x55ae36104d40 2024-04-21T08:43:12.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.918+0000 7f93f0fe1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ae360651a0 con 0x55ae36111e90 2024-04-21T08:43:12.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.918+0000 7f93e57fa640 1 --2- 172.21.15.86:0/1785305728 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f93d40a3da0 0x7f93d40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:12.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.918+0000 7f93ec998640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae36104390 0x55ae36104770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:12.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.918+0000 7f93ec998640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae36104390 0x55ae36104770 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51282/0 (socket says 172.21.15.86:51282) 2024-04-21T08:43:12.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.918+0000 7f93ec998640 1 -- 172.21.15.86:0/3827730625 learned_addr learned my addr 172.21.15.86:0/3827730625 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:12.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.918+0000 7f93ec998640 1 --2- 172.21.15.86:0/1785305728 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f93d40a3da0 0x7f93d40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:12.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.919+0000 7f93e7fff640 1 --2- 172.21.15.86:0/3827730625 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae36104d40 0x55ae36111950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:12.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.919+0000 7f93e57fa640 1 -- 172.21.15.86:0/1785305728 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f93d00f2ba0 con 0x7f93e8107270 2024-04-21T08:43:12.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.919+0000 7f93e77fe640 1 --2- 172.21.15.86:0/3827730625 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36111e90 0x55ae36114280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:12.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.919+0000 7f93ec998640 1 -- 172.21.15.86:0/3827730625 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36111e90 msgr2=0x55ae36114280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:12.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.919+0000 7f93ec998640 1 --2- 172.21.15.86:0/3827730625 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36111e90 0x55ae36114280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.919+0000 7f93ec998640 1 -- 172.21.15.86:0/3827730625 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae36104d40 msgr2=0x55ae36111950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:12.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.919+0000 7f93ec998640 1 --2- 172.21.15.86:0/3827730625 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae36104d40 0x55ae36111950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.919+0000 7f93ec998640 1 -- 172.21.15.86:0/3827730625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae35e457d0 con 0x55ae36104390 2024-04-21T08:43:12.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.919+0000 7f93ec998640 1 --2- 172.21.15.86:0/3827730625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae36104390 0x55ae36104770 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f93e0012d00 tx=0x7f93e0010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=591e6f21d5abe7d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:12.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.919+0000 7f93ec998640 1 --2- 172.21.15.86:0/1785305728 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f93d40a3da0 0x7f93d40a6260 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f93e0010c40 tx=0x7f93e000c000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:12.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.919+0000 7f93de7fc640 1 -- 172.21.15.86:0/3827730625 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e0024070 con 0x55ae36104390 2024-04-21T08:43:12.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.919+0000 7f93de7fc640 1 -- 172.21.15.86:0/3827730625 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93e0009d20 con 0x55ae36104390 2024-04-21T08:43:12.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.919+0000 7f93de7fc640 1 -- 172.21.15.86:0/3827730625 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e0005280 con 0x55ae36104390 2024-04-21T08:43:12.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.920+0000 7f93f0fe1740 1 -- 172.21.15.86:0/3827730625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae36104390 msgr2=0x55ae36104770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:12.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.920+0000 7f93f0fe1740 1 --2- 172.21.15.86:0/3827730625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae36104390 0x55ae36104770 secure :-1 s=READY pgs=1165 cs=0 l=1 rev1=1 crypto rx=0x7f93e0012d00 tx=0x7f93e0010740 comp rx=0 tx=0).stop 2024-04-21T08:43:12.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.920+0000 7f93f0fe1740 1 -- 172.21.15.86:0/3827730625 shutdown_connections 2024-04-21T08:43:12.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.920+0000 7f93f0fe1740 1 --2- 172.21.15.86:0/3827730625 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36111e90 0x55ae36114280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.920+0000 7f93f0fe1740 1 --2- 172.21.15.86:0/3827730625 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae36104d40 0x55ae36111950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.920+0000 7f93f0fe1740 1 --2- 172.21.15.86:0/3827730625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae36104390 0x55ae36104770 unknown :-1 s=CLOSED pgs=1165 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.920+0000 7f93f0fe1740 1 -- 172.21.15.86:0/3827730625 >> 172.21.15.86:0/3827730625 conn(0x55ae360fecb0 msgr2=0x55ae360ff090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:12.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.920+0000 7f93f0fe1740 1 -- 172.21.15.86:0/3827730625 shutdown_connections 2024-04-21T08:43:12.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.920+0000 7f93f0fe1740 1 -- 172.21.15.86:0/3827730625 wait complete. 2024-04-21T08:43:12.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.920+0000 7f93f0fe1740 1 -- start start 2024-04-21T08:43:12.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.920+0000 7f93f0fe1740 1 Processor -- start 2024-04-21T08:43:12.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.921+0000 7f93f0fe1740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae36104390 0x55ae3610efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:12.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.921+0000 7f93f0fe1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36104d40 0x55ae3610f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:12.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.921+0000 7f93f0fe1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae36111e90 0x55ae3610fa30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:12.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.921+0000 7f93f0fe1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ae36114bf0 con 0x55ae36111e90 2024-04-21T08:43:12.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.921+0000 7f93f0fe1740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55ae360222f0 con 0x55ae36104390 2024-04-21T08:43:12.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.921+0000 7f93f0fe1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ae36114d70 con 0x55ae36104d40 2024-04-21T08:43:12.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.921+0000 7f93f0fe1740 10 client.0 _ll_register_callbacks cb 0x55ae3602f580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:12.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.921+0000 7f93ec998640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae36111e90 0x55ae3610fa30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:12.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.921+0000 7f93ec998640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae36111e90 0x55ae3610fa30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51294/0 (socket says 172.21.15.86:51294) 2024-04-21T08:43:12.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.921+0000 7f93ec998640 1 -- 172.21.15.86:0/4009915392 learned_addr learned my addr 172.21.15.86:0/4009915392 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:12.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.921+0000 7f93e7fff640 1 --2- 172.21.15.86:0/4009915392 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae36104390 0x55ae3610efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:12.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.921+0000 7f93e77fe640 1 --2- 172.21.15.86:0/4009915392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36104d40 0x55ae3610f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:12.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.921+0000 7f93ec998640 1 -- 172.21.15.86:0/4009915392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36104d40 msgr2=0x55ae3610f4f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:12.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.921+0000 7f93ec998640 1 --2- 172.21.15.86:0/4009915392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36104d40 0x55ae3610f4f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.921+0000 7f93ec998640 1 -- 172.21.15.86:0/4009915392 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae36104390 msgr2=0x55ae3610efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:12.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.921+0000 7f93ec998640 1 --2- 172.21.15.86:0/4009915392 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae36104390 0x55ae3610efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.921+0000 7f93ec998640 1 -- 172.21.15.86:0/4009915392 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae35d7b3f0 con 0x55ae36111e90 2024-04-21T08:43:12.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.921+0000 7f93ec998640 1 --2- 172.21.15.86:0/4009915392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae36111e90 0x55ae3610fa30 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009a40 tx=0x7f93e000cbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bdce73c702ef1c68 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:12.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.921+0000 7f93c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:12.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.921+0000 7f93c77fe640 1 -- 172.21.15.86:0/4009915392 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e0024040 con 0x55ae36111e90 2024-04-21T08:43:12.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.921+0000 7f93c77fe640 1 -- 172.21.15.86:0/4009915392 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f93e0005790 con 0x55ae36111e90 2024-04-21T08:43:12.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.921+0000 7f93f0fe1740 1 -- 172.21.15.86:0/4009915392 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55ae36076c00 con 0x55ae36111e90 2024-04-21T08:43:12.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.922+0000 7f93c77fe640 1 -- 172.21.15.86:0/4009915392 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f93e003a050 con 0x55ae36111e90 2024-04-21T08:43:12.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.922+0000 7f93f0fe1740 10 client.17670 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:12.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.922+0000 7f93f0fe1740 1 -- 172.21.15.86:0/4009915392 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ae35d3b0f0 con 0x55ae36111e90 2024-04-21T08:43:12.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.922+0000 7f93c77fe640 1 -- 172.21.15.86:0/4009915392 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55ae36076c00 con 0x55ae36111e90 2024-04-21T08:43:12.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.922+0000 7f93c77fe640 1 -- 172.21.15.86:0/4009915392 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f93e000f040 con 0x55ae36111e90 2024-04-21T08:43:12.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.922+0000 7f93f0fe1740 10 client.17670 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:12.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.922+0000 7f93f0fe1740 10 client.17670 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:12.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.922+0000 7f93f0fe1740 20 client.17670 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:12.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.922+0000 7f93f0fe1740 1 --2- 172.21.15.86:0/4009915392 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ae36121e80 0x55ae36124270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:12.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.922+0000 7f93f0fe1740 4 client.17670 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:12.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.922+0000 7f93f0fe1740 1 -- 172.21.15.86:0/4009915392 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ae36070a70 con 0x55ae36121e80 2024-04-21T08:43:12.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.922+0000 7f93e7fff640 1 --2- 172.21.15.86:0/4009915392 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ae36121e80 0x55ae36124270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:12.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.922+0000 7f93c77fe640 10 client.17670 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:12.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.922+0000 7f93e7fff640 1 --2- 172.21.15.86:0/4009915392 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ae36121e80 0x55ae36124270 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2b3a6c6e64609315 server_cookie=97f3e6e83bac201e in_seq=0 out_seq=0 2024-04-21T08:43:12.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.923+0000 7f93c77fe640 1 -- 172.21.15.86:0/4009915392 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ae36070a70 con 0x55ae36121e80 2024-04-21T08:43:12.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.923+0000 7f93c77fe640 10 client.17670 handle_command_reply: tid=0 2024-04-21T08:43:12.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.923+0000 7f93f0fe1740 1 client.17670 shutdown 2024-04-21T08:43:12.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.923+0000 7f93f0fe1740 1 -- 172.21.15.86:0/4009915392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae36111e90 msgr2=0x55ae3610fa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:12.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.923+0000 7f93f0fe1740 1 --2- 172.21.15.86:0/4009915392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae36111e90 0x55ae3610fa30 secure :-1 s=READY pgs=1166 cs=0 l=1 rev1=1 crypto rx=0x7f93e0009a40 tx=0x7f93e000cbc0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.923+0000 7f93f0fe1740 1 -- 172.21.15.86:0/4009915392 shutdown_connections 2024-04-21T08:43:12.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.923+0000 7f93f0fe1740 1 --2- 172.21.15.86:0/4009915392 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ae36121e80 0x55ae36124270 crc :-1 s=READY pgs=159 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.923+0000 7f93f0fe1740 1 --2- 172.21.15.86:0/4009915392 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae36111e90 0x55ae3610fa30 unknown :-1 s=CLOSED pgs=1166 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.923+0000 7f93c77fe640 0 client.17670 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:12.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.923+0000 7f93f0fe1740 1 --2- 172.21.15.86:0/4009915392 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36104d40 0x55ae3610f4f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.923+0000 7f93f0fe1740 1 --2- 172.21.15.86:0/4009915392 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae36104390 0x55ae3610efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.923+0000 7f93f0fe1740 1 -- 172.21.15.86:0/4009915392 >> 172.21.15.86:0/4009915392 conn(0x55ae360fecb0 msgr2=0x55ae36100a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:12.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.923+0000 7f93f0fe1740 1 -- 172.21.15.86:0/4009915392 shutdown_connections 2024-04-21T08:43:12.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.923+0000 7f93f0fe1740 1 -- 172.21.15.86:0/4009915392 wait complete. 2024-04-21T08:43:12.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.923+0000 7f93f0fe1740 20 client.17670 trim_cache size 0 max 16384 2024-04-21T08:43:12.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.943+0000 7f93f0fe1740 1 Processor -- start 2024-04-21T08:43:12.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.943+0000 7f93f0fe1740 1 -- start start 2024-04-21T08:43:12.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.943+0000 7f93f0fe1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36104390 0x55ae3611dab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:12.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.943+0000 7f93f0fe1740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae36104d40 0x55ae36111670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:12.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.943+0000 7f93f0fe1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae36111e90 0x55ae36129870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:12.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.943+0000 7f93f0fe1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93e81ad0d0 con 0x55ae36111e90 2024-04-21T08:43:12.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.943+0000 7f93f0fe1740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f93e81aec10 con 0x55ae36104d40 2024-04-21T08:43:12.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.943+0000 7f93f0fe1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f93e81ad7a0 con 0x55ae36104390 2024-04-21T08:43:12.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.943+0000 7f93ec998640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae36111e90 0x55ae36129870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:12.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.943+0000 7f93ec998640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae36111e90 0x55ae36129870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51296/0 (socket says 172.21.15.86:51296) 2024-04-21T08:43:12.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.943+0000 7f93ec998640 1 -- 172.21.15.86:0/1555984359 learned_addr learned my addr 172.21.15.86:0/1555984359 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:12.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.943+0000 7f93e77fe640 1 --2- 172.21.15.86:0/1555984359 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae36104d40 0x55ae36111670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:12.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.943+0000 7f93ec998640 1 -- 172.21.15.86:0/1555984359 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36104390 msgr2=0x55ae3611dab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:12.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.943+0000 7f93ec998640 1 --2- 172.21.15.86:0/1555984359 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36104390 0x55ae3611dab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.943+0000 7f93ec998640 1 -- 172.21.15.86:0/1555984359 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae36104d40 msgr2=0x55ae36111670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:12.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.943+0000 7f93ec998640 1 --2- 172.21.15.86:0/1555984359 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae36104d40 0x55ae36111670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.943+0000 7f93ec998640 1 -- 172.21.15.86:0/1555984359 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ae3610d540 con 0x55ae36111e90 2024-04-21T08:43:12.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.943+0000 7f93e77fe640 1 --2- 172.21.15.86:0/1555984359 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae36104d40 0x55ae36111670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:12.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.944+0000 7f93ec998640 1 --2- 172.21.15.86:0/1555984359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae36111e90 0x55ae36129870 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f93e0002230 tx=0x7f93e0012520 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec36a7ff3fc7d124 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:12.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.944+0000 7f93ddffb640 1 -- 172.21.15.86:0/1555984359 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e0024050 con 0x55ae36111e90 2024-04-21T08:43:12.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.944+0000 7f93ddffb640 1 -- 172.21.15.86:0/1555984359 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93e00207a0 con 0x55ae36111e90 2024-04-21T08:43:12.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.944+0000 7f93ddffb640 1 -- 172.21.15.86:0/1555984359 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93e00120b0 con 0x55ae36111e90 2024-04-21T08:43:12.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.944+0000 7f93f0fe1740 1 -- 172.21.15.86:0/1555984359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae36111e90 msgr2=0x55ae36129870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:12.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.944+0000 7f93f0fe1740 1 --2- 172.21.15.86:0/1555984359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae36111e90 0x55ae36129870 secure :-1 s=READY pgs=1167 cs=0 l=1 rev1=1 crypto rx=0x7f93e0002230 tx=0x7f93e0012520 comp rx=0 tx=0).stop 2024-04-21T08:43:12.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.944+0000 7f93f0fe1740 1 -- 172.21.15.86:0/1555984359 shutdown_connections 2024-04-21T08:43:12.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.944+0000 7f93f0fe1740 1 --2- 172.21.15.86:0/1555984359 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae36111e90 0x55ae36129870 unknown :-1 s=CLOSED pgs=1167 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.944+0000 7f93f0fe1740 1 --2- 172.21.15.86:0/1555984359 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae36104d40 0x55ae36111670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.944+0000 7f93f0fe1740 1 --2- 172.21.15.86:0/1555984359 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36104390 0x55ae3611dab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.944+0000 7f93f0fe1740 1 -- 172.21.15.86:0/1555984359 >> 172.21.15.86:0/1555984359 conn(0x55ae360fecb0 msgr2=0x55ae3610f800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:12.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.944+0000 7f93f0fe1740 1 -- 172.21.15.86:0/1555984359 shutdown_connections 2024-04-21T08:43:12.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.944+0000 7f93f0fe1740 1 -- 172.21.15.86:0/1555984359 wait complete. 2024-04-21T08:43:12.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.945+0000 7f93f0fe1740 1 -- start start 2024-04-21T08:43:12.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.945+0000 7f93f0fe1740 1 Processor -- start 2024-04-21T08:43:12.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.945+0000 7f93f0fe1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae36104390 0x55ae3610bc40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:12.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.945+0000 7f93f0fe1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36104d40 0x55ae3610c180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:12.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.945+0000 7f93f0fe1740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae36111e90 0x55ae3610c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:12.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.945+0000 7f93f0fe1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93e81ae540 con 0x55ae36104390 2024-04-21T08:43:12.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.945+0000 7f93f0fe1740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f93e81ade70 con 0x55ae36111e90 2024-04-21T08:43:12.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.945+0000 7f93f0fe1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f93e81ab090 con 0x55ae36104d40 2024-04-21T08:43:12.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.945+0000 7f93f0fe1740 10 client.0 _ll_register_callbacks cb 0x7f93d800b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:12.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.945+0000 7f93e7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae36104390 0x55ae3610bc40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:12.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.945+0000 7f93e7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae36104390 0x55ae3610bc40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51312/0 (socket says 172.21.15.86:51312) 2024-04-21T08:43:12.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.945+0000 7f93e7fff640 1 -- 172.21.15.86:0/1160790791 learned_addr learned my addr 172.21.15.86:0/1160790791 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:12.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.945+0000 7f93ec998640 1 --2- 172.21.15.86:0/1160790791 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae36111e90 0x55ae3610c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:12.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.945+0000 7f93e77fe640 1 --2- 172.21.15.86:0/1160790791 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36104d40 0x55ae3610c180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:12.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.946+0000 7f93e7fff640 1 -- 172.21.15.86:0/1160790791 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36104d40 msgr2=0x55ae3610c180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:12.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.946+0000 7f93e7fff640 1 --2- 172.21.15.86:0/1160790791 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36104d40 0x55ae3610c180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.946+0000 7f93e7fff640 1 -- 172.21.15.86:0/1160790791 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae36111e90 msgr2=0x55ae3610c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:12.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.946+0000 7f93e7fff640 1 --2- 172.21.15.86:0/1160790791 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae36111e90 0x55ae3610c6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.946+0000 7f93e7fff640 1 -- 172.21.15.86:0/1160790791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ae36187110 con 0x55ae36104390 2024-04-21T08:43:12.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.946+0000 7f93e7fff640 1 --2- 172.21.15.86:0/1160790791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae36104390 0x55ae3610bc40 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x55ae36124810 tx=0x7f93d800bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc0fc26e0d0fdf9a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:12.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.946+0000 7f93c77fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:12.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.946+0000 7f93c77fe640 1 -- 172.21.15.86:0/1160790791 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f93d80201b0 con 0x55ae36104390 2024-04-21T08:43:12.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.946+0000 7f93c77fe640 1 -- 172.21.15.86:0/1160790791 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f93d8020350 con 0x55ae36104390 2024-04-21T08:43:12.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.946+0000 7f93f0fe1740 1 -- 172.21.15.86:0/1160790791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f93e00207a0 con 0x55ae36104390 2024-04-21T08:43:12.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.946+0000 7f93c77fe640 1 -- 172.21.15.86:0/1160790791 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f93d802f050 con 0x55ae36104390 2024-04-21T08:43:12.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.946+0000 7f93f0fe1740 10 client.17676 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:12.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.946+0000 7f93f0fe1740 1 -- 172.21.15.86:0/1160790791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ae361883d0 con 0x55ae36104390 2024-04-21T08:43:12.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.946+0000 7f93c77fe640 1 -- 172.21.15.86:0/1160790791 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f93e00207a0 con 0x55ae36104390 2024-04-21T08:43:12.952 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.946+0000 7f93c77fe640 1 -- 172.21.15.86:0/1160790791 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f93d8011b60 con 0x55ae36104390 2024-04-21T08:43:12.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.946+0000 7f93f0fe1740 10 client.17676 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:12.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.946+0000 7f93f0fe1740 10 client.17676 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:12.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.946+0000 7f93f0fe1740 20 client.17676 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:12.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.946+0000 7f93f0fe1740 1 --2- 172.21.15.86:0/1160790791 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ae36188b20 0x55ae3618af40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:12.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.946+0000 7f93f0fe1740 4 client.17676 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:43:12.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.946+0000 7f93f0fe1740 1 -- 172.21.15.86:0/1160790791 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ae35fec590 con 0x55ae36188b20 2024-04-21T08:43:12.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.947+0000 7f93e77fe640 1 --2- 172.21.15.86:0/1160790791 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ae36188b20 0x55ae3618af40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:12.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.947+0000 7f93e77fe640 1 --2- 172.21.15.86:0/1160790791 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ae36188b20 0x55ae3618af40 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b39ee28a39f4770 server_cookie=f26a471bf51face3 in_seq=0 out_seq=0 2024-04-21T08:43:12.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.947+0000 7f93c77fe640 10 client.17676 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:12.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.947+0000 7f93c77fe640 1 -- 172.21.15.86:0/1160790791 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ae35fec590 con 0x55ae36188b20 2024-04-21T08:43:12.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.947+0000 7f93c77fe640 10 client.17676 handle_command_reply: tid=0 2024-04-21T08:43:12.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.947+0000 7f93f0fe1740 1 client.17676 shutdown 2024-04-21T08:43:12.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.947+0000 7f93f0fe1740 1 -- 172.21.15.86:0/1160790791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae36104390 msgr2=0x55ae3610bc40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:12.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.947+0000 7f93f0fe1740 1 --2- 172.21.15.86:0/1160790791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae36104390 0x55ae3610bc40 secure :-1 s=READY pgs=1168 cs=0 l=1 rev1=1 crypto rx=0x55ae36124810 tx=0x7f93d800bee0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.947+0000 7f93f0fe1740 1 -- 172.21.15.86:0/1160790791 shutdown_connections 2024-04-21T08:43:12.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.947+0000 7f93f0fe1740 1 --2- 172.21.15.86:0/1160790791 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ae36188b20 0x55ae3618af40 crc :-1 s=READY pgs=160 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.947+0000 7f93f0fe1740 1 --2- 172.21.15.86:0/1160790791 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ae36111e90 0x55ae3610c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.947+0000 7f93f0fe1740 1 --2- 172.21.15.86:0/1160790791 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ae36104d40 0x55ae3610c180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.947+0000 7f93c77fe640 0 client.17676 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:12.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.947+0000 7f93f0fe1740 1 --2- 172.21.15.86:0/1160790791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ae36104390 0x55ae3610bc40 unknown :-1 s=CLOSED pgs=1168 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.948+0000 7f93f0fe1740 1 -- 172.21.15.86:0/1160790791 >> 172.21.15.86:0/1160790791 conn(0x55ae360fecb0 msgr2=0x55ae360ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:12.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.948+0000 7f93f0fe1740 1 -- 172.21.15.86:0/1160790791 shutdown_connections 2024-04-21T08:43:12.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.948+0000 7f93f0fe1740 1 -- 172.21.15.86:0/1160790791 wait complete. 2024-04-21T08:43:12.954 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.948+0000 7f93f0fe1740 20 client.17676 trim_cache size 0 max 16384 2024-04-21T08:43:12.955 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:43:12.955 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:43:12.955 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-21T08:43:12.955 INFO:teuthology.orchestra.run.smithi086.stdout: "95ef26af-0c67-4fa5-ba7b-acc08e492709": { 2024-04-21T08:43:12.955 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-21T08:43:12.955 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "95ef26af-0c67-4fa5-ba7b-acc08e492709", 2024-04-21T08:43:12.955 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-21T08:43:12.955 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:43:12.955 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:43:12.955 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:43:12.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.948+0000 7f93ee3fb640 1 -- 172.21.15.86:0/1785305728 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f93d40a3da0 msgr2=0x7f93d40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:12.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.948+0000 7f93ee3fb640 1 --2- 172.21.15.86:0/1785305728 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f93d40a3da0 0x7f93d40a6260 secure :-1 s=READY pgs=181 cs=0 l=1 rev1=1 crypto rx=0x7f93e0010c40 tx=0x7f93e000c000 comp rx=0 tx=0).stop 2024-04-21T08:43:12.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.948+0000 7f93ee3fb640 1 -- 172.21.15.86:0/1785305728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93e8107270 msgr2=0x7f93e819e970 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:12.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.948+0000 7f93ee3fb640 1 --2- 172.21.15.86:0/1785305728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93e8107270 0x7f93e819e970 secure :-1 s=READY pgs=1164 cs=0 l=1 rev1=1 crypto rx=0x7f93d0036040 tx=0x7f93d002c7e0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.948+0000 7f93ee3fb640 1 -- 172.21.15.86:0/1785305728 shutdown_connections 2024-04-21T08:43:12.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.948+0000 7f93ee3fb640 1 --2- 172.21.15.86:0/1785305728 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f93d40a3da0 0x7f93d40a6260 unknown :-1 s=CLOSED pgs=181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.948+0000 7f93ee3fb640 1 --2- 172.21.15.86:0/1785305728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93e8198630 0x7f93e8198ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.948+0000 7f93ee3fb640 1 --2- 172.21.15.86:0/1785305728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f93e8107270 0x7f93e819e970 unknown :-1 s=CLOSED pgs=1164 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.948+0000 7f93ee3fb640 1 --2- 172.21.15.86:0/1785305728 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f93e81021a0 0x7f93e819e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:12.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.948+0000 7f93ee3fb640 1 -- 172.21.15.86:0/1785305728 >> 172.21.15.86:0/1785305728 conn(0x7f93e80fda40 msgr2=0x7f93e80fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:12.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.948+0000 7f93ee3fb640 1 -- 172.21.15.86:0/1785305728 shutdown_connections 2024-04-21T08:43:12.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:12.948+0000 7f93ee3fb640 1 -- 172.21.15.86:0/1785305728 wait complete. 2024-04-21T08:43:12.958 INFO:tasks.cephfs.filesystem:scrub status for tag:95ef26af-0c67-4fa5-ba7b-acc08e492709 - {'path': '/', 'tag': '95ef26af-0c67-4fa5-ba7b-acc08e492709', 'options': 'recursive,force'} 2024-04-21T08:43:12.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:12 smithi088 ceph-mon[31259]: pgmap v135: 129 pgs: 129 active+clean; 5.3 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 58 KiB/s wr, 31 op/s 2024-04-21T08:43:14.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:14 smithi179 ceph-mon[31829]: pgmap v136: 129 pgs: 129 active+clean; 5.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 73 KiB/s wr, 38 op/s 2024-04-21T08:43:14.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:14 smithi179 ceph-mon[31829]: scrub queued for path: / 2024-04-21T08:43:14.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:14 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:43:14.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:14 smithi179 ceph-mon[31829]: scrub summary: active paths [/] 2024-04-21T08:43:14.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:14 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:43:14.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:14 smithi086 ceph-mon[27939]: pgmap v136: 129 pgs: 129 active+clean; 5.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 73 KiB/s wr, 38 op/s 2024-04-21T08:43:14.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:14 smithi086 ceph-mon[27939]: scrub queued for path: / 2024-04-21T08:43:14.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:14 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:43:14.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:14 smithi086 ceph-mon[27939]: scrub summary: active paths [/] 2024-04-21T08:43:14.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:14 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:43:14.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:14 smithi088 ceph-mon[31259]: pgmap v136: 129 pgs: 129 active+clean; 5.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 73 KiB/s wr, 38 op/s 2024-04-21T08:43:14.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:14 smithi088 ceph-mon[31259]: scrub queued for path: / 2024-04-21T08:43:14.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:14 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:43:14.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:14 smithi088 ceph-mon[31259]: scrub summary: active paths [/] 2024-04-21T08:43:14.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:14 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:43:15.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:15 smithi086 ceph-mon[27939]: scrub summary: idle 2024-04-21T08:43:15.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:15 smithi088 ceph-mon[31259]: scrub summary: idle 2024-04-21T08:43:15.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:15 smithi179 ceph-mon[31829]: scrub summary: idle 2024-04-21T08:43:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.712+0000 7fa35984a640 1 -- 172.21.15.86:0/4164215536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa354100f80 msgr2=0x7fa35410d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.712+0000 7fa35984a640 1 --2- 172.21.15.86:0/4164215536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa354100f80 0x7fa35410d490 secure :-1 s=READY pgs=1169 cs=0 l=1 rev1=1 crypto rx=0x7fa34c009cd0 tx=0x7fa34c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:43:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.712+0000 7fa35984a640 1 -- 172.21.15.86:0/4164215536 shutdown_connections 2024-04-21T08:43:16.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.712+0000 7fa35984a640 1 --2- 172.21.15.86:0/4164215536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa354100f80 0x7fa35410d490 unknown :-1 s=CLOSED pgs=1169 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.712+0000 7fa35984a640 1 --2- 172.21.15.86:0/4164215536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3541005c0 0x7fa354100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.712+0000 7fa35984a640 1 --2- 172.21.15.86:0/4164215536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa354106660 0x7fa354106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.712+0000 7fa35984a640 1 -- 172.21.15.86:0/4164215536 >> 172.21.15.86:0/4164215536 conn(0x7fa354077370 msgr2=0x7fa3540fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.713+0000 7fa35984a640 1 -- 172.21.15.86:0/4164215536 shutdown_connections 2024-04-21T08:43:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.713+0000 7fa35984a640 1 -- 172.21.15.86:0/4164215536 wait complete. 2024-04-21T08:43:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.713+0000 7fa35984a640 1 Processor -- start 2024-04-21T08:43:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.713+0000 7fa35984a640 1 -- start start 2024-04-21T08:43:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.713+0000 7fa35984a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3541005c0 0x7fa354198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.713+0000 7fa35984a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa354100f80 0x7fa354198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.713+0000 7fa35984a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa354106660 0x7fa35419cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.713+0000 7fa35984a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa354110890 con 0x7fa354100f80 2024-04-21T08:43:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.713+0000 7fa35984a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa354110710 con 0x7fa3541005c0 2024-04-21T08:43:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.713+0000 7fa35984a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa354110a10 con 0x7fa354106660 2024-04-21T08:43:16.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.713+0000 7fa34bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa354100f80 0x7fa354198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.714+0000 7fa34bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa354100f80 0x7fa354198bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51318/0 (socket says 172.21.15.86:51318) 2024-04-21T08:43:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.714+0000 7fa34bfff640 1 -- 172.21.15.86:0/3750412048 learned_addr learned my addr 172.21.15.86:0/3750412048 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.714+0000 7fa352ffd640 1 --2- 172.21.15.86:0/3750412048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3541005c0 0x7fa354198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.714+0000 7fa3537fe640 1 --2- 172.21.15.86:0/3750412048 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa354106660 0x7fa35419cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.714+0000 7fa34bfff640 1 -- 172.21.15.86:0/3750412048 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa354106660 msgr2=0x7fa35419cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.714+0000 7fa34bfff640 1 --2- 172.21.15.86:0/3750412048 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa354106660 0x7fa35419cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.714+0000 7fa34bfff640 1 -- 172.21.15.86:0/3750412048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3541005c0 msgr2=0x7fa354198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.714+0000 7fa34bfff640 1 --2- 172.21.15.86:0/3750412048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3541005c0 0x7fa354198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.714+0000 7fa34bfff640 1 -- 172.21.15.86:0/3750412048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa35419d670 con 0x7fa354100f80 2024-04-21T08:43:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.714+0000 7fa34bfff640 1 --2- 172.21.15.86:0/3750412048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa354100f80 0x7fa354198bc0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fa33c00c970 tx=0x7fa33c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.714+0000 7fa358848640 1 -- 172.21.15.86:0/3750412048 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c007bb0 con 0x7fa354100f80 2024-04-21T08:43:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.715+0000 7fa358848640 1 -- 172.21.15.86:0/3750412048 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa33c007d50 con 0x7fa354100f80 2024-04-21T08:43:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.715+0000 7fa35984a640 1 -- 172.21.15.86:0/3750412048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa35419d960 con 0x7fa354100f80 2024-04-21T08:43:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.715+0000 7fa358848640 1 -- 172.21.15.86:0/3750412048 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c00f450 con 0x7fa354100f80 2024-04-21T08:43:16.716 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.715+0000 7fa35984a640 1 -- 172.21.15.86:0/3750412048 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3541a51f0 con 0x7fa354100f80 2024-04-21T08:43:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.717+0000 7fa358848640 1 -- 172.21.15.86:0/3750412048 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa33c004f20 con 0x7fa354100f80 2024-04-21T08:43:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.718+0000 7fa35c43e740 1 Processor -- start 2024-04-21T08:43:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.718+0000 7fa35c43e740 1 -- start start 2024-04-21T08:43:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.718+0000 7fa35c43e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562831ef88d0 0x562831ef7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.718+0000 7fa35c43e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562831ef7b60 0x562831ef7fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.718+0000 7fa35c43e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562831f054f0 0x562831f078e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.718+0000 7fa35c43e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562831e5c8d0 con 0x562831ef7b60 2024-04-21T08:43:16.723 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.718+0000 7fa35c43e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x562831df9e80 con 0x562831f054f0 2024-04-21T08:43:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.718+0000 7fa35c43e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562831e58890 con 0x562831ef88d0 2024-04-21T08:43:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.718+0000 7fa358848640 1 --2- 172.21.15.86:0/3750412048 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa3400a3e40 0x7fa3400a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.718+0000 7fa352ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562831ef7b60 0x562831ef7fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.718+0000 7fa352ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562831ef7b60 0x562831ef7fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51320/0 (socket says 172.21.15.86:51320) 2024-04-21T08:43:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.718+0000 7fa352ffd640 1 -- 172.21.15.86:0/338295827 learned_addr learned my addr 172.21.15.86:0/338295827 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.718+0000 7fa3537fe640 1 --2- 172.21.15.86:0/338295827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562831ef88d0 0x562831ef7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.718+0000 7fa3537fe640 1 --2- 172.21.15.86:0/3750412048 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa3400a3e40 0x7fa3400a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.718+0000 7fa34bfff640 1 --2- 172.21.15.86:0/338295827 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562831f054f0 0x562831f078e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.718+0000 7fa358848640 1 -- 172.21.15.86:0/3750412048 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fa33c0d0810 con 0x7fa354100f80 2024-04-21T08:43:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.718+0000 7fa352ffd640 1 -- 172.21.15.86:0/338295827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562831ef88d0 msgr2=0x562831ef7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.718+0000 7fa352ffd640 1 --2- 172.21.15.86:0/338295827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562831ef88d0 0x562831ef7620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.718+0000 7fa352ffd640 1 -- 172.21.15.86:0/338295827 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562831f054f0 msgr2=0x562831f078e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:16.724 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.718+0000 7fa352ffd640 1 --2- 172.21.15.86:0/338295827 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562831f054f0 0x562831f078e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.718+0000 7fa352ffd640 1 -- 172.21.15.86:0/338295827 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562831e1b320 con 0x562831ef7b60 2024-04-21T08:43:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.718+0000 7fa34bfff640 1 --2- 172.21.15.86:0/338295827 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562831f054f0 0x562831f078e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.719+0000 7fa352ffd640 1 --2- 172.21.15.86:0/338295827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562831ef7b60 0x562831ef7fe0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fa34400abf0 tx=0x7fa34400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38309cd09e4cc2b4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.719+0000 7fa3537fe640 1 --2- 172.21.15.86:0/3750412048 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa3400a3e40 0x7fa3400a6300 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa34c006010 tx=0x7fa34c02c0f0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.719+0000 7fa349ffb640 1 -- 172.21.15.86:0/338295827 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa344005220 con 0x562831ef7b60 2024-04-21T08:43:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.719+0000 7fa349ffb640 1 -- 172.21.15.86:0/338295827 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3440047a0 con 0x562831ef7b60 2024-04-21T08:43:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.719+0000 7fa349ffb640 1 -- 172.21.15.86:0/338295827 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3440186c0 con 0x562831ef7b60 2024-04-21T08:43:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.720+0000 7fa35c43e740 1 -- 172.21.15.86:0/338295827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562831ef7b60 msgr2=0x562831ef7fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.720+0000 7fa35c43e740 1 --2- 172.21.15.86:0/338295827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562831ef7b60 0x562831ef7fe0 secure :-1 s=READY pgs=1171 cs=0 l=1 rev1=1 crypto rx=0x7fa34400abf0 tx=0x7fa34400f790 comp rx=0 tx=0).stop 2024-04-21T08:43:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.720+0000 7fa35c43e740 1 -- 172.21.15.86:0/338295827 shutdown_connections 2024-04-21T08:43:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.720+0000 7fa35c43e740 1 --2- 172.21.15.86:0/338295827 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562831f054f0 0x562831f078e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.720+0000 7fa35c43e740 1 --2- 172.21.15.86:0/338295827 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562831ef7b60 0x562831ef7fe0 unknown :-1 s=CLOSED pgs=1171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.720+0000 7fa35c43e740 1 --2- 172.21.15.86:0/338295827 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562831ef88d0 0x562831ef7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.720+0000 7fa35c43e740 1 -- 172.21.15.86:0/338295827 >> 172.21.15.86:0/338295827 conn(0x562831ebec90 msgr2=0x562831ef2850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.720+0000 7fa35c43e740 1 -- 172.21.15.86:0/338295827 shutdown_connections 2024-04-21T08:43:16.725 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.720+0000 7fa35c43e740 1 -- 172.21.15.86:0/338295827 wait complete. 2024-04-21T08:43:16.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.720+0000 7fa35c43e740 1 -- start start 2024-04-21T08:43:16.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.720+0000 7fa35c43e740 1 Processor -- start 2024-04-21T08:43:16.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.720+0000 7fa35c43e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562831ef7b60 0x562831f04df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:16.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.720+0000 7fa35c43e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562831ef88d0 0x562831f02180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:16.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.720+0000 7fa35c43e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562831f054f0 0x562831f026c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:16.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.721+0000 7fa35c43e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562831f08220 con 0x562831ef88d0 2024-04-21T08:43:16.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.721+0000 7fa35c43e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x562831de4eb0 con 0x562831f054f0 2024-04-21T08:43:16.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.721+0000 7fa35c43e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562831f083a0 con 0x562831ef7b60 2024-04-21T08:43:16.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.721+0000 7fa35c43e740 10 client.0 _ll_register_callbacks cb 0x562831e11b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:16.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.721+0000 7fa34bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562831ef88d0 0x562831f02180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:16.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.721+0000 7fa34bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562831ef88d0 0x562831f02180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51326/0 (socket says 172.21.15.86:51326) 2024-04-21T08:43:16.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.721+0000 7fa34bfff640 1 -- 172.21.15.86:0/965260539 learned_addr learned my addr 172.21.15.86:0/965260539 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:16.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.721+0000 7fa3537fe640 1 --2- 172.21.15.86:0/965260539 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562831f054f0 0x562831f026c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:16.726 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.721+0000 7fa352ffd640 1 --2- 172.21.15.86:0/965260539 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562831ef7b60 0x562831f04df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:16.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.721+0000 7fa34bfff640 1 -- 172.21.15.86:0/965260539 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562831ef7b60 msgr2=0x562831f04df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:16.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.721+0000 7fa34bfff640 1 --2- 172.21.15.86:0/965260539 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562831ef7b60 0x562831f04df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:16.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.721+0000 7fa34bfff640 1 -- 172.21.15.86:0/965260539 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562831f054f0 msgr2=0x562831f026c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:16.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.721+0000 7fa34bfff640 1 --2- 172.21.15.86:0/965260539 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562831f054f0 0x562831f026c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:16.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.721+0000 7fa34bfff640 1 -- 172.21.15.86:0/965260539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562831b6f210 con 0x562831ef88d0 2024-04-21T08:43:16.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.721+0000 7fa34bfff640 1 --2- 172.21.15.86:0/965260539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562831ef88d0 0x562831f02180 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fa33c0cdc50 tx=0x7fa33c008e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f09567a07f3adecf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:16.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.721+0000 7fa332ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:16.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.721+0000 7fa332ffd640 1 -- 172.21.15.86:0/965260539 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c0ce070 con 0x562831ef88d0 2024-04-21T08:43:16.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.721+0000 7fa332ffd640 1 -- 172.21.15.86:0/965260539 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fa33c005210 con 0x562831ef88d0 2024-04-21T08:43:16.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.721+0000 7fa35c43e740 1 -- 172.21.15.86:0/965260539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3440047a0 con 0x562831ef88d0 2024-04-21T08:43:16.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.721+0000 7fa332ffd640 1 -- 172.21.15.86:0/965260539 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa33c005410 con 0x562831ef88d0 2024-04-21T08:43:16.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.721+0000 7fa35c43e740 10 client.17688 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:16.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.721+0000 7fa35c43e740 1 -- 172.21.15.86:0/965260539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562831c3b620 con 0x562831ef88d0 2024-04-21T08:43:16.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.722+0000 7fa332ffd640 1 -- 172.21.15.86:0/965260539 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3440047a0 con 0x562831ef88d0 2024-04-21T08:43:16.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.722+0000 7fa332ffd640 1 -- 172.21.15.86:0/965260539 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa33c0d03c0 con 0x562831ef88d0 2024-04-21T08:43:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.722+0000 7fa35c43e740 10 client.17688 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.722+0000 7fa35c43e740 10 client.17688 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.722+0000 7fa35c43e740 20 client.17688 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.722+0000 7fa35c43e740 1 --2- 172.21.15.86:0/965260539 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x562831f15480 0x562831f17870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.722+0000 7fa35c43e740 4 client.17688 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.722+0000 7fa35c43e740 1 -- 172.21.15.86:0/965260539 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562831e70880 con 0x562831f15480 2024-04-21T08:43:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.722+0000 7fa352ffd640 1 --2- 172.21.15.86:0/965260539 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x562831f15480 0x562831f17870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.723+0000 7fa332ffd640 10 client.17688 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.723+0000 7fa352ffd640 1 --2- 172.21.15.86:0/965260539 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x562831f15480 0x562831f17870 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6ebb8f487c8b3e49 server_cookie=57612f847d69ea07 in_seq=0 out_seq=0 2024-04-21T08:43:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.723+0000 7fa332ffd640 1 -- 172.21.15.86:0/965260539 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562831e70880 con 0x562831f15480 2024-04-21T08:43:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.723+0000 7fa332ffd640 10 client.17688 handle_command_reply: tid=0 2024-04-21T08:43:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.723+0000 7fa35c43e740 1 client.17688 shutdown 2024-04-21T08:43:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.723+0000 7fa35c43e740 1 -- 172.21.15.86:0/965260539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562831ef88d0 msgr2=0x562831f02180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.723+0000 7fa35c43e740 1 --2- 172.21.15.86:0/965260539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562831ef88d0 0x562831f02180 secure :-1 s=READY pgs=1172 cs=0 l=1 rev1=1 crypto rx=0x7fa33c0cdc50 tx=0x7fa33c008e80 comp rx=0 tx=0).stop 2024-04-21T08:43:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.723+0000 7fa35c43e740 1 -- 172.21.15.86:0/965260539 shutdown_connections 2024-04-21T08:43:16.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.723+0000 7fa35c43e740 1 --2- 172.21.15.86:0/965260539 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x562831f15480 0x562831f17870 crc :-1 s=READY pgs=161 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:16.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.723+0000 7fa35c43e740 1 --2- 172.21.15.86:0/965260539 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562831f054f0 0x562831f026c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:16.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.723+0000 7fa35c43e740 1 --2- 172.21.15.86:0/965260539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562831ef88d0 0x562831f02180 unknown :-1 s=CLOSED pgs=1172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:16.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.723+0000 7fa332ffd640 0 client.17688 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:16.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.723+0000 7fa35c43e740 1 --2- 172.21.15.86:0/965260539 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562831ef7b60 0x562831f04df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:16.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.723+0000 7fa35c43e740 1 -- 172.21.15.86:0/965260539 >> 172.21.15.86:0/965260539 conn(0x562831ebec90 msgr2=0x562831ef45a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:16.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.724+0000 7fa35c43e740 1 -- 172.21.15.86:0/965260539 shutdown_connections 2024-04-21T08:43:16.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.724+0000 7fa35c43e740 1 -- 172.21.15.86:0/965260539 wait complete. 2024-04-21T08:43:16.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.724+0000 7fa35c43e740 20 client.17688 trim_cache size 0 max 16384 2024-04-21T08:43:16.738 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.738+0000 7fa35c43e740 1 Processor -- start 2024-04-21T08:43:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.738+0000 7fa35c43e740 1 -- start start 2024-04-21T08:43:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.738+0000 7fa35c43e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562831ef7b60 0x562831f11120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.738+0000 7fa35c43e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562831ef88d0 0x562831ef7170 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.738+0000 7fa35c43e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562831f054f0 0x562831f048b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.738+0000 7fa35c43e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa3541ad030 con 0x562831f054f0 2024-04-21T08:43:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.738+0000 7fa35c43e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa3541aeb70 con 0x562831ef7b60 2024-04-21T08:43:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.738+0000 7fa35c43e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa3541ad700 con 0x562831ef88d0 2024-04-21T08:43:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.739+0000 7fa3537fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562831f054f0 0x562831f048b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.739+0000 7fa3537fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562831f054f0 0x562831f048b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51332/0 (socket says 172.21.15.86:51332) 2024-04-21T08:43:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.739+0000 7fa3537fe640 1 -- 172.21.15.86:0/2280704502 learned_addr learned my addr 172.21.15.86:0/2280704502 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.739+0000 7fa352ffd640 1 --2- 172.21.15.86:0/2280704502 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562831ef7b60 0x562831f11120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.739+0000 7fa3537fe640 1 -- 172.21.15.86:0/2280704502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562831ef88d0 msgr2=0x562831ef7170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:16.739 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.739+0000 7fa3537fe640 1 --2- 172.21.15.86:0/2280704502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562831ef88d0 0x562831ef7170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:16.740 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.739+0000 7fa3537fe640 1 -- 172.21.15.86:0/2280704502 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562831ef7b60 msgr2=0x562831f11120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:16.740 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.739+0000 7fa3537fe640 1 --2- 172.21.15.86:0/2280704502 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562831ef7b60 0x562831f11120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:16.740 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.739+0000 7fa3537fe640 1 -- 172.21.15.86:0/2280704502 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562831f14020 con 0x562831f054f0 2024-04-21T08:43:16.740 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.739+0000 7fa3537fe640 1 --2- 172.21.15.86:0/2280704502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562831f054f0 0x562831f048b0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fa34c009cd0 tx=0x7fa34c041000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1631f6adfbd4704 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:16.740 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.739+0000 7fa3497fa640 1 -- 172.21.15.86:0/2280704502 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c039070 con 0x562831f054f0 2024-04-21T08:43:16.740 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.739+0000 7fa3497fa640 1 -- 172.21.15.86:0/2280704502 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa34c055070 con 0x562831f054f0 2024-04-21T08:43:16.740 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.739+0000 7fa3497fa640 1 -- 172.21.15.86:0/2280704502 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa34c004570 con 0x562831f054f0 2024-04-21T08:43:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.740+0000 7fa35c43e740 1 -- 172.21.15.86:0/2280704502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562831f054f0 msgr2=0x562831f048b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.740+0000 7fa35c43e740 1 --2- 172.21.15.86:0/2280704502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562831f054f0 0x562831f048b0 secure :-1 s=READY pgs=1173 cs=0 l=1 rev1=1 crypto rx=0x7fa34c009cd0 tx=0x7fa34c041000 comp rx=0 tx=0).stop 2024-04-21T08:43:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.740+0000 7fa35c43e740 1 -- 172.21.15.86:0/2280704502 shutdown_connections 2024-04-21T08:43:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.740+0000 7fa35c43e740 1 --2- 172.21.15.86:0/2280704502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562831f054f0 0x562831f048b0 unknown :-1 s=CLOSED pgs=1173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:16.744 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.740+0000 7fa35c43e740 1 --2- 172.21.15.86:0/2280704502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562831ef88d0 0x562831ef7170 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.740+0000 7fa35c43e740 1 --2- 172.21.15.86:0/2280704502 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562831ef7b60 0x562831f11120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.740+0000 7fa35c43e740 1 -- 172.21.15.86:0/2280704502 >> 172.21.15.86:0/2280704502 conn(0x562831ebec90 msgr2=0x562831f00a50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.740+0000 7fa35c43e740 1 -- 172.21.15.86:0/2280704502 shutdown_connections 2024-04-21T08:43:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.740+0000 7fa35c43e740 1 -- 172.21.15.86:0/2280704502 wait complete. 2024-04-21T08:43:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.740+0000 7fa35c43e740 1 -- start start 2024-04-21T08:43:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.741+0000 7fa35c43e740 1 Processor -- start 2024-04-21T08:43:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.741+0000 7fa35c43e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562831ef7b60 0x562831f06020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.741+0000 7fa35c43e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562831ef88d0 0x562831f06560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.741+0000 7fa35c43e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562831f054f0 0x562831f06aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.741+0000 7fa35c43e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa3541ae4a0 con 0x562831ef88d0 2024-04-21T08:43:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.741+0000 7fa35c43e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa3541addd0 con 0x562831f054f0 2024-04-21T08:43:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.741+0000 7fa35c43e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa3541aaff0 con 0x562831ef7b60 2024-04-21T08:43:16.745 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.741+0000 7fa35c43e740 10 client.0 _ll_register_callbacks cb 0x7fa33c00c8c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.741+0000 7fa34bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562831ef88d0 0x562831f06560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.741+0000 7fa34bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562831ef88d0 0x562831f06560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51344/0 (socket says 172.21.15.86:51344) 2024-04-21T08:43:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.741+0000 7fa34bfff640 1 -- 172.21.15.86:0/1666738403 learned_addr learned my addr 172.21.15.86:0/1666738403 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.741+0000 7fa352ffd640 1 --2- 172.21.15.86:0/1666738403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562831ef7b60 0x562831f06020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.741+0000 7fa3537fe640 1 --2- 172.21.15.86:0/1666738403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562831f054f0 0x562831f06aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.741+0000 7fa34bfff640 1 -- 172.21.15.86:0/1666738403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562831ef7b60 msgr2=0x562831f06020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.741+0000 7fa34bfff640 1 --2- 172.21.15.86:0/1666738403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562831ef7b60 0x562831f06020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.741+0000 7fa34bfff640 1 -- 172.21.15.86:0/1666738403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562831f054f0 msgr2=0x562831f06aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.741+0000 7fa34bfff640 1 --2- 172.21.15.86:0/1666738403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562831f054f0 0x562831f06aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.741+0000 7fa34bfff640 1 -- 172.21.15.86:0/1666738403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562831ef6e30 con 0x562831ef88d0 2024-04-21T08:43:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.741+0000 7fa352ffd640 1 --2- 172.21.15.86:0/1666738403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562831ef7b60 0x562831f06020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.741+0000 7fa34bfff640 1 --2- 172.21.15.86:0/1666738403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562831ef88d0 0x562831f06560 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fa33c09c3d0 tx=0x7fa33c09bcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53b28c698f77ef72 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.741+0000 7fa332ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:16.746 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.741+0000 7fa332ffd640 1 -- 172.21.15.86:0/1666738403 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa33c0ba060 con 0x562831ef88d0 2024-04-21T08:43:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.741+0000 7fa332ffd640 1 -- 172.21.15.86:0/1666738403 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fa33c0b5bc0 con 0x562831ef88d0 2024-04-21T08:43:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.741+0000 7fa35c43e740 1 -- 172.21.15.86:0/1666738403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa34c055070 con 0x562831ef88d0 2024-04-21T08:43:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.742+0000 7fa332ffd640 1 -- 172.21.15.86:0/1666738403 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa33c09bda0 con 0x562831ef88d0 2024-04-21T08:43:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.742+0000 7fa35c43e740 10 client.17694 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.742+0000 7fa35c43e740 1 -- 172.21.15.86:0/1666738403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562831f7bb10 con 0x562831ef88d0 2024-04-21T08:43:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.742+0000 7fa332ffd640 1 -- 172.21.15.86:0/1666738403 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa34c055070 con 0x562831ef88d0 2024-04-21T08:43:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.742+0000 7fa332ffd640 1 -- 172.21.15.86:0/1666738403 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa33c0bb250 con 0x562831ef88d0 2024-04-21T08:43:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.742+0000 7fa35c43e740 10 client.17694 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.742+0000 7fa35c43e740 10 client.17694 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.742+0000 7fa35c43e740 20 client.17694 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.742+0000 7fa35c43e740 1 --2- 172.21.15.86:0/1666738403 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x562831f7c0b0 0x562831f7e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.742+0000 7fa35c43e740 4 client.17694 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:43:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.742+0000 7fa35c43e740 1 -- 172.21.15.86:0/1666738403 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562831dfc3d0 con 0x562831f7c0b0 2024-04-21T08:43:16.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.742+0000 7fa352ffd640 1 --2- 172.21.15.86:0/1666738403 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x562831f7c0b0 0x562831f7e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:16.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.742+0000 7fa352ffd640 1 --2- 172.21.15.86:0/1666738403 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x562831f7c0b0 0x562831f7e4d0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=38b6bba5e6d970d server_cookie=999f5981f2dfceb3 in_seq=0 out_seq=0 2024-04-21T08:43:16.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.742+0000 7fa332ffd640 10 client.17694 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:16.748 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:16 smithi179 ceph-mon[31829]: pgmap v137: 129 pgs: 129 active+clean; 5.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 37 KiB/s wr, 16 op/s 2024-04-21T08:43:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.756+0000 7fa332ffd640 1 -- 172.21.15.86:0/1666738403 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562831dfc3d0 con 0x562831f7c0b0 2024-04-21T08:43:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.756+0000 7fa332ffd640 10 client.17694 handle_command_reply: tid=0 2024-04-21T08:43:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.756+0000 7fa35c43e740 1 client.17694 shutdown 2024-04-21T08:43:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.756+0000 7fa35c43e740 1 -- 172.21.15.86:0/1666738403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562831ef88d0 msgr2=0x562831f06560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.756+0000 7fa35c43e740 1 --2- 172.21.15.86:0/1666738403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562831ef88d0 0x562831f06560 secure :-1 s=READY pgs=1174 cs=0 l=1 rev1=1 crypto rx=0x7fa33c09c3d0 tx=0x7fa33c09bcd0 comp rx=0 tx=0).stop 2024-04-21T08:43:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.756+0000 7fa35c43e740 1 -- 172.21.15.86:0/1666738403 shutdown_connections 2024-04-21T08:43:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.756+0000 7fa35c43e740 1 --2- 172.21.15.86:0/1666738403 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x562831f7c0b0 0x562831f7e4d0 crc :-1 s=READY pgs=162 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.756+0000 7fa35c43e740 1 --2- 172.21.15.86:0/1666738403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562831f054f0 0x562831f06aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.756+0000 7fa35c43e740 1 --2- 172.21.15.86:0/1666738403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562831ef88d0 0x562831f06560 unknown :-1 s=CLOSED pgs=1174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.756+0000 7fa332ffd640 0 client.17694 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.756+0000 7fa35c43e740 1 --2- 172.21.15.86:0/1666738403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562831ef7b60 0x562831f06020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.756+0000 7fa35c43e740 1 -- 172.21.15.86:0/1666738403 >> 172.21.15.86:0/1666738403 conn(0x562831ebec90 msgr2=0x562831ef3c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.756+0000 7fa35c43e740 1 -- 172.21.15.86:0/1666738403 shutdown_connections 2024-04-21T08:43:16.756 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.756+0000 7fa35c43e740 1 -- 172.21.15.86:0/1666738403 wait complete. 2024-04-21T08:43:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.756+0000 7fa35c43e740 20 client.17694 trim_cache size 0 max 16384 2024-04-21T08:43:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:43:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:43:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:43:16.757 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:43:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.757+0000 7fa35984a640 1 -- 172.21.15.86:0/3750412048 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa3400a3e40 msgr2=0x7fa3400a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.757+0000 7fa35984a640 1 --2- 172.21.15.86:0/3750412048 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa3400a3e40 0x7fa3400a6300 secure :-1 s=READY pgs=182 cs=0 l=1 rev1=1 crypto rx=0x7fa34c006010 tx=0x7fa34c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:43:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.757+0000 7fa35984a640 1 -- 172.21.15.86:0/3750412048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa354100f80 msgr2=0x7fa354198bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.757+0000 7fa35984a640 1 --2- 172.21.15.86:0/3750412048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa354100f80 0x7fa354198bc0 secure :-1 s=READY pgs=1170 cs=0 l=1 rev1=1 crypto rx=0x7fa33c00c970 tx=0x7fa33c00ce30 comp rx=0 tx=0).stop 2024-04-21T08:43:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.757+0000 7fa35984a640 1 -- 172.21.15.86:0/3750412048 shutdown_connections 2024-04-21T08:43:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.757+0000 7fa35984a640 1 --2- 172.21.15.86:0/3750412048 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa3400a3e40 0x7fa3400a6300 unknown :-1 s=CLOSED pgs=182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.757+0000 7fa35984a640 1 --2- 172.21.15.86:0/3750412048 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa354106660 0x7fa35419cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.757+0000 7fa35984a640 1 --2- 172.21.15.86:0/3750412048 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa354100f80 0x7fa354198bc0 unknown :-1 s=CLOSED pgs=1170 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.757+0000 7fa35984a640 1 --2- 172.21.15.86:0/3750412048 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3541005c0 0x7fa354198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.757+0000 7fa35984a640 1 -- 172.21.15.86:0/3750412048 >> 172.21.15.86:0/3750412048 conn(0x7fa354077370 msgr2=0x7fa3541043a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.757+0000 7fa35984a640 1 -- 172.21.15.86:0/3750412048 shutdown_connections 2024-04-21T08:43:16.758 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:16.757+0000 7fa35984a640 1 -- 172.21.15.86:0/3750412048 wait complete. 2024-04-21T08:43:16.767 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:16 smithi086 ceph-mon[27939]: pgmap v137: 129 pgs: 129 active+clean; 5.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 37 KiB/s wr, 16 op/s 2024-04-21T08:43:16.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:16 smithi088 ceph-mon[31259]: pgmap v137: 129 pgs: 129 active+clean; 5.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 37 KiB/s wr, 16 op/s 2024-04-21T08:43:18.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:18 smithi086 ceph-mon[27939]: pgmap v138: 129 pgs: 129 active+clean; 5.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 37 KiB/s wr, 16 op/s 2024-04-21T08:43:18.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:18 smithi088 ceph-mon[31259]: pgmap v138: 129 pgs: 129 active+clean; 5.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 37 KiB/s wr, 16 op/s 2024-04-21T08:43:18.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:18 smithi179 ceph-mon[31829]: pgmap v138: 129 pgs: 129 active+clean; 5.5 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 37 KiB/s wr, 16 op/s 2024-04-21T08:43:20.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:20 smithi086 ceph-mon[27939]: pgmap v139: 129 pgs: 129 active+clean; 1.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 57 KiB/s wr, 27 op/s 2024-04-21T08:43:20.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:20 smithi088 ceph-mon[31259]: pgmap v139: 129 pgs: 129 active+clean; 1.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 57 KiB/s wr, 27 op/s 2024-04-21T08:43:20.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:20 smithi179 ceph-mon[31829]: pgmap v139: 129 pgs: 129 active+clean; 1.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 57 KiB/s wr, 27 op/s 2024-04-21T08:43:22.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:22 smithi086 ceph-mon[27939]: pgmap v140: 129 pgs: 129 active+clean; 1.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 18 op/s 2024-04-21T08:43:22.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:22 smithi088 ceph-mon[31259]: pgmap v140: 129 pgs: 129 active+clean; 1.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 18 op/s 2024-04-21T08:43:22.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:22 smithi179 ceph-mon[31829]: pgmap v140: 129 pgs: 129 active+clean; 1.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 36 KiB/s wr, 18 op/s 2024-04-21T08:43:24.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:24 smithi086 ceph-mon[27939]: pgmap v141: 129 pgs: 129 active+clean; 1.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 51 KiB/s wr, 24 op/s 2024-04-21T08:43:24.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:24 smithi088 ceph-mon[31259]: pgmap v141: 129 pgs: 129 active+clean; 1.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 51 KiB/s wr, 24 op/s 2024-04-21T08:43:24.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:24 smithi179 ceph-mon[31829]: pgmap v141: 129 pgs: 129 active+clean; 1.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 51 KiB/s wr, 24 op/s 2024-04-21T08:43:26.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:26 smithi086 ceph-mon[27939]: pgmap v142: 129 pgs: 129 active+clean; 1.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 17 op/s 2024-04-21T08:43:26.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:26 smithi088 ceph-mon[31259]: pgmap v142: 129 pgs: 129 active+clean; 1.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 17 op/s 2024-04-21T08:43:26.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:26 smithi179 ceph-mon[31829]: pgmap v142: 129 pgs: 129 active+clean; 1.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 17 op/s 2024-04-21T08:43:28.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:28 smithi086 ceph-mon[27939]: pgmap v143: 129 pgs: 129 active+clean; 1.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 17 op/s 2024-04-21T08:43:28.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:28 smithi088 ceph-mon[31259]: pgmap v143: 129 pgs: 129 active+clean; 1.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 17 op/s 2024-04-21T08:43:28.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:28 smithi179 ceph-mon[31829]: pgmap v143: 129 pgs: 129 active+clean; 1.8 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 35 KiB/s wr, 17 op/s 2024-04-21T08:43:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.852+0000 7f4b5d74b640 1 -- 172.21.15.86:0/870355433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b581031e0 msgr2=0x7f4b581035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:29.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.852+0000 7f4b5d74b640 1 --2- 172.21.15.86:0/870355433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b581031e0 0x7f4b581035c0 secure :-1 s=READY pgs=1175 cs=0 l=1 rev1=1 crypto rx=0x7f4b48009ca0 tx=0x7f4b4802c230 comp rx=0 tx=0).stop 2024-04-21T08:43:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.853+0000 7f4b5d74b640 1 -- 172.21.15.86:0/870355433 shutdown_connections 2024-04-21T08:43:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.853+0000 7f4b5d74b640 1 --2- 172.21.15.86:0/870355433 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b5810c520 0x7f4b5810e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.853+0000 7f4b5d74b640 1 --2- 172.21.15.86:0/870355433 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b58103b00 0x7f4b5810bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.853+0000 7f4b5d74b640 1 --2- 172.21.15.86:0/870355433 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b581031e0 0x7f4b581035c0 unknown :-1 s=CLOSED pgs=1175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.853+0000 7f4b5d74b640 1 -- 172.21.15.86:0/870355433 >> 172.21.15.86:0/870355433 conn(0x7f4b58077fd0 msgr2=0x7f4b580783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.853+0000 7f4b5d74b640 1 -- 172.21.15.86:0/870355433 shutdown_connections 2024-04-21T08:43:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.853+0000 7f4b5d74b640 1 -- 172.21.15.86:0/870355433 wait complete. 2024-04-21T08:43:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.853+0000 7f4b5d74b640 1 Processor -- start 2024-04-21T08:43:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.853+0000 7f4b5d74b640 1 -- start start 2024-04-21T08:43:29.855 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.854+0000 7f4b5d74b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b581031e0 0x7f4b58198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.854+0000 7f4b5d74b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b58103b00 0x7f4b58198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.854+0000 7f4b5d74b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b5810c520 0x7f4b5819cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.854+0000 7f4b5d74b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b58112210 con 0x7f4b58103b00 2024-04-21T08:43:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.854+0000 7f4b5d74b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4b58112090 con 0x7f4b5810c520 2024-04-21T08:43:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.854+0000 7f4b5d74b640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4b58112390 con 0x7f4b581031e0 2024-04-21T08:43:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.854+0000 7f4b567fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b58103b00 0x7f4b58198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.854+0000 7f4b567fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b58103b00 0x7f4b58198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33276/0 (socket says 172.21.15.86:33276) 2024-04-21T08:43:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.854+0000 7f4b567fc640 1 -- 172.21.15.86:0/1236338036 learned_addr learned my addr 172.21.15.86:0/1236338036 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.854+0000 7f4b56ffd640 1 --2- 172.21.15.86:0/1236338036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b581031e0 0x7f4b58198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.854+0000 7f4b577fe640 1 --2- 172.21.15.86:0/1236338036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b5810c520 0x7f4b5819cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.854+0000 7f4b567fc640 1 -- 172.21.15.86:0/1236338036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b581031e0 msgr2=0x7f4b58198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:29.856 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.854+0000 7f4b567fc640 1 --2- 172.21.15.86:0/1236338036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b581031e0 0x7f4b58198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.854+0000 7f4b567fc640 1 -- 172.21.15.86:0/1236338036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b5810c520 msgr2=0x7f4b5819cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.854+0000 7f4b567fc640 1 --2- 172.21.15.86:0/1236338036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b5810c520 0x7f4b5819cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.854+0000 7f4b567fc640 1 -- 172.21.15.86:0/1236338036 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b5819d6f0 con 0x7f4b58103b00 2024-04-21T08:43:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.854+0000 7f4b577fe640 1 --2- 172.21.15.86:0/1236338036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b5810c520 0x7f4b5819cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.854+0000 7f4b567fc640 1 --2- 172.21.15.86:0/1236338036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b58103b00 0x7f4b58198c90 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00b4a0 tx=0x7f4b4c00b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.854+0000 7f4b3ffff640 1 -- 172.21.15.86:0/1236338036 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c014070 con 0x7f4b58103b00 2024-04-21T08:43:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.854+0000 7f4b3ffff640 1 -- 172.21.15.86:0/1236338036 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b4c010ac0 con 0x7f4b58103b00 2024-04-21T08:43:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.854+0000 7f4b5d74b640 1 -- 172.21.15.86:0/1236338036 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b5819d9e0 con 0x7f4b58103b00 2024-04-21T08:43:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.855+0000 7f4b3ffff640 1 -- 172.21.15.86:0/1236338036 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c017c30 con 0x7f4b58103b00 2024-04-21T08:43:29.857 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.855+0000 7f4b5d74b640 1 -- 172.21.15.86:0/1236338036 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b581a5270 con 0x7f4b58103b00 2024-04-21T08:43:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.857+0000 7f4b3ffff640 1 -- 172.21.15.86:0/1236338036 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4b4c017070 con 0x7f4b58103b00 2024-04-21T08:43:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.857+0000 7f4b60415740 1 Processor -- start 2024-04-21T08:43:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.857+0000 7f4b60415740 1 -- start start 2024-04-21T08:43:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.857+0000 7f4b3ffff640 1 --2- 172.21.15.86:0/1236338036 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4b400a3dc0 0x7f4b400a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.857+0000 7f4b60415740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b63fb58d0 0x563b63fb4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.857+0000 7f4b60415740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b63fb4b60 0x563b63fb4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.857+0000 7f4b60415740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b63fbe0e0 0x563b63fc04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:29.865 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.857+0000 7f4b60415740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563b63f198d0 con 0x563b63fbe0e0 2024-04-21T08:43:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.857+0000 7f4b60415740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x563b63eb6e80 con 0x563b63fb4b60 2024-04-21T08:43:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.857+0000 7f4b60415740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563b63f15890 con 0x563b63fb58d0 2024-04-21T08:43:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.857+0000 7f4b577fe640 1 --2- 172.21.15.86:0/1236338036 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4b400a3dc0 0x7f4b400a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.857+0000 7f4b577fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b63fbe0e0 0x563b63fc04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.857+0000 7f4b3ffff640 1 -- 172.21.15.86:0/1236338036 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f4b4c0cfea0 con 0x7f4b58103b00 2024-04-21T08:43:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.857+0000 7f4b577fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b63fbe0e0 0x563b63fc04d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33284/0 (socket says 172.21.15.86:33284) 2024-04-21T08:43:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.857+0000 7f4b577fe640 1 -- 172.21.15.86:0/2564400463 learned_addr learned my addr 172.21.15.86:0/2564400463 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.858+0000 7f4b567fc640 1 --2- 172.21.15.86:0/2564400463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b63fb4b60 0x563b63fb4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.858+0000 7f4b56ffd640 1 --2- 172.21.15.86:0/2564400463 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b63fb58d0 0x563b63fb4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.858+0000 7f4b577fe640 1 -- 172.21.15.86:0/2564400463 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b63fb58d0 msgr2=0x563b63fb4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:29.866 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.858+0000 7f4b577fe640 1 --2- 172.21.15.86:0/2564400463 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b63fb58d0 0x563b63fb4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.858+0000 7f4b577fe640 1 -- 172.21.15.86:0/2564400463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b63fb4b60 msgr2=0x563b63fb4fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.858+0000 7f4b577fe640 1 --2- 172.21.15.86:0/2564400463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b63fb4b60 0x563b63fb4fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.858+0000 7f4b577fe640 1 -- 172.21.15.86:0/2564400463 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b63ed8320 con 0x563b63fbe0e0 2024-04-21T08:43:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.858+0000 7f4b577fe640 1 --2- 172.21.15.86:0/2564400463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b63fbe0e0 0x563b63fc04d0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f4b50012b10 tx=0x7f4b5000f360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a9047104213c23c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.858+0000 7f4b577fe640 1 --2- 172.21.15.86:0/1236338036 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4b400a3dc0 0x7f4b400a6280 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4b50012fd0 tx=0x7f4b5001a000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.858+0000 7f4b3d7fa640 1 -- 172.21.15.86:0/2564400463 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b50020070 con 0x563b63fbe0e0 2024-04-21T08:43:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.858+0000 7f4b3d7fa640 1 -- 172.21.15.86:0/2564400463 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b50012510 con 0x563b63fbe0e0 2024-04-21T08:43:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.858+0000 7f4b3d7fa640 1 -- 172.21.15.86:0/2564400463 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b5001cb40 con 0x563b63fbe0e0 2024-04-21T08:43:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.859+0000 7f4b60415740 1 -- 172.21.15.86:0/2564400463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b63fbe0e0 msgr2=0x563b63fc04d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.859+0000 7f4b60415740 1 --2- 172.21.15.86:0/2564400463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b63fbe0e0 0x563b63fc04d0 secure :-1 s=READY pgs=1177 cs=0 l=1 rev1=1 crypto rx=0x7f4b50012b10 tx=0x7f4b5000f360 comp rx=0 tx=0).stop 2024-04-21T08:43:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.859+0000 7f4b60415740 1 -- 172.21.15.86:0/2564400463 shutdown_connections 2024-04-21T08:43:29.867 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.859+0000 7f4b60415740 1 --2- 172.21.15.86:0/2564400463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b63fbe0e0 0x563b63fc04d0 unknown :-1 s=CLOSED pgs=1177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.859+0000 7f4b60415740 1 --2- 172.21.15.86:0/2564400463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b63fb4b60 0x563b63fb4fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.859+0000 7f4b60415740 1 --2- 172.21.15.86:0/2564400463 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b63fb58d0 0x563b63fb4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.859+0000 7f4b60415740 1 -- 172.21.15.86:0/2564400463 >> 172.21.15.86:0/2564400463 conn(0x563b63f7bc90 msgr2=0x563b63faf850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.859+0000 7f4b60415740 1 -- 172.21.15.86:0/2564400463 shutdown_connections 2024-04-21T08:43:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.859+0000 7f4b60415740 1 -- 172.21.15.86:0/2564400463 wait complete. 2024-04-21T08:43:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.859+0000 7f4b60415740 1 -- start start 2024-04-21T08:43:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.859+0000 7f4b60415740 1 Processor -- start 2024-04-21T08:43:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.860+0000 7f4b60415740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b63fb4b60 0x563b63fbdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.860+0000 7f4b60415740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b63fb58d0 0x563b63fbae90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:29.868 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.860+0000 7f4b60415740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b63fbe0e0 0x563b63fbb4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.860+0000 7f4b60415740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563b63fc0db0 con 0x563b63fb4b60 2024-04-21T08:43:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.860+0000 7f4b60415740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x563b63ea1eb0 con 0x563b63fbe0e0 2024-04-21T08:43:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.860+0000 7f4b60415740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563b63fc0f30 con 0x563b63fb58d0 2024-04-21T08:43:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.860+0000 7f4b60415740 10 client.0 _ll_register_callbacks cb 0x563b63eceb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.860+0000 7f4b56ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b63fb4b60 0x563b63fbdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.860+0000 7f4b56ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b63fb4b60 0x563b63fbdb00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33288/0 (socket says 172.21.15.86:33288) 2024-04-21T08:43:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.860+0000 7f4b56ffd640 1 -- 172.21.15.86:0/1606591696 learned_addr learned my addr 172.21.15.86:0/1606591696 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.860+0000 7f4b567fc640 1 --2- 172.21.15.86:0/1606591696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b63fb58d0 0x563b63fbae90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.860+0000 7f4b577fe640 1 --2- 172.21.15.86:0/1606591696 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b63fbe0e0 0x563b63fbb4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.860+0000 7f4b56ffd640 1 -- 172.21.15.86:0/1606591696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b63fb58d0 msgr2=0x563b63fbae90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.860+0000 7f4b56ffd640 1 --2- 172.21.15.86:0/1606591696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b63fb58d0 0x563b63fbae90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:29.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.860+0000 7f4b56ffd640 1 -- 172.21.15.86:0/1606591696 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b63fbe0e0 msgr2=0x563b63fbb4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.860+0000 7f4b56ffd640 1 --2- 172.21.15.86:0/1606591696 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b63fbe0e0 0x563b63fbb4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.860+0000 7f4b56ffd640 1 -- 172.21.15.86:0/1606591696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b63c2c210 con 0x563b63fb4b60 2024-04-21T08:43:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.860+0000 7f4b577fe640 1 --2- 172.21.15.86:0/1606591696 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b63fbe0e0 0x563b63fbb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.860+0000 7f4b56ffd640 1 --2- 172.21.15.86:0/1606591696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b63fb4b60 0x563b63fbdb00 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f4b48037040 tx=0x7f4b48009c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6321a5a45f7e7b55 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.860+0000 7f4b3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.860+0000 7f4b3aff5640 1 -- 172.21.15.86:0/1606591696 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b48044070 con 0x563b63fb4b60 2024-04-21T08:43:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.861+0000 7f4b60415740 1 -- 172.21.15.86:0/1606591696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b50012510 con 0x563b63fb4b60 2024-04-21T08:43:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.861+0000 7f4b3aff5640 1 -- 172.21.15.86:0/1606591696 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f4b48004040 con 0x563b63fb4b60 2024-04-21T08:43:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.861+0000 7f4b3aff5640 1 -- 172.21.15.86:0/1606591696 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4b48050050 con 0x563b63fb4b60 2024-04-21T08:43:29.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.861+0000 7f4b60415740 10 client.17706 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.861+0000 7f4b60415740 1 -- 172.21.15.86:0/1606591696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563b63cf8620 con 0x563b63fb4b60 2024-04-21T08:43:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.861+0000 7f4b3aff5640 1 -- 172.21.15.86:0/1606591696 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b50012510 con 0x563b63fb4b60 2024-04-21T08:43:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.861+0000 7f4b3aff5640 1 -- 172.21.15.86:0/1606591696 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4b48032ba0 con 0x563b63fb4b60 2024-04-21T08:43:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.861+0000 7f4b60415740 10 client.17706 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.861+0000 7f4b60415740 10 client.17706 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.861+0000 7f4b60415740 20 client.17706 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.861+0000 7f4b60415740 1 --2- 172.21.15.86:0/1606591696 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563b63fce0c0 0x563b63fd04b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.861+0000 7f4b60415740 4 client.17706 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.861+0000 7f4b60415740 1 -- 172.21.15.86:0/1606591696 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563b63f2d880 con 0x563b63fce0c0 2024-04-21T08:43:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.861+0000 7f4b567fc640 1 --2- 172.21.15.86:0/1606591696 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563b63fce0c0 0x563b63fd04b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.862+0000 7f4b567fc640 1 --2- 172.21.15.86:0/1606591696 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563b63fce0c0 0x563b63fd04b0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a848f94dfcb0c83 server_cookie=29f1397e90270863 in_seq=0 out_seq=0 2024-04-21T08:43:29.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.862+0000 7f4b3aff5640 10 client.17706 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:29.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.862+0000 7f4b3aff5640 1 -- 172.21.15.86:0/1606591696 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563b63f2d880 con 0x563b63fce0c0 2024-04-21T08:43:29.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.862+0000 7f4b3aff5640 10 client.17706 handle_command_reply: tid=0 2024-04-21T08:43:29.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.863+0000 7f4b60415740 1 client.17706 shutdown 2024-04-21T08:43:29.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.863+0000 7f4b60415740 1 -- 172.21.15.86:0/1606591696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b63fb4b60 msgr2=0x563b63fbdb00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:29.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.863+0000 7f4b60415740 1 --2- 172.21.15.86:0/1606591696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b63fb4b60 0x563b63fbdb00 secure :-1 s=READY pgs=1178 cs=0 l=1 rev1=1 crypto rx=0x7f4b48037040 tx=0x7f4b48009c20 comp rx=0 tx=0).stop 2024-04-21T08:43:29.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.863+0000 7f4b60415740 1 -- 172.21.15.86:0/1606591696 shutdown_connections 2024-04-21T08:43:29.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.863+0000 7f4b60415740 1 --2- 172.21.15.86:0/1606591696 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563b63fce0c0 0x563b63fd04b0 crc :-1 s=READY pgs=163 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:29.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.863+0000 7f4b60415740 1 --2- 172.21.15.86:0/1606591696 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b63fbe0e0 0x563b63fbb4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:29.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.863+0000 7f4b60415740 1 --2- 172.21.15.86:0/1606591696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b63fb58d0 0x563b63fbae90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:29.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.863+0000 7f4b60415740 1 --2- 172.21.15.86:0/1606591696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b63fb4b60 0x563b63fbdb00 unknown :-1 s=CLOSED pgs=1178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:29.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.863+0000 7f4b3aff5640 0 client.17706 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:29.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.863+0000 7f4b60415740 1 -- 172.21.15.86:0/1606591696 >> 172.21.15.86:0/1606591696 conn(0x563b63f7bc90 msgr2=0x563b63fb15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:29.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.863+0000 7f4b60415740 1 -- 172.21.15.86:0/1606591696 shutdown_connections 2024-04-21T08:43:29.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.863+0000 7f4b60415740 1 -- 172.21.15.86:0/1606591696 wait complete. 2024-04-21T08:43:29.873 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.863+0000 7f4b60415740 20 client.17706 trim_cache size 0 max 16384 2024-04-21T08:43:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.878+0000 7f4b60415740 1 Processor -- start 2024-04-21T08:43:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.878+0000 7f4b60415740 1 -- start start 2024-04-21T08:43:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.878+0000 7f4b60415740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b63fb4b60 0x563b63fc9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.878+0000 7f4b60415740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b63fb58d0 0x563b63fbd5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.878+0000 7f4b60415740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b63fbe0e0 0x563b63fbdb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:29.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.878+0000 7f4b60415740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b581af2e0 con 0x563b63fb58d0 2024-04-21T08:43:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.878+0000 7f4b60415740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4b581b0e20 con 0x563b63fb4b60 2024-04-21T08:43:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.878+0000 7f4b60415740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4b581af9b0 con 0x563b63fbe0e0 2024-04-21T08:43:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.878+0000 7f4b567fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b63fb58d0 0x563b63fbd5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.878+0000 7f4b567fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b63fb58d0 0x563b63fbd5c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33298/0 (socket says 172.21.15.86:33298) 2024-04-21T08:43:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.878+0000 7f4b567fc640 1 -- 172.21.15.86:0/2294608354 learned_addr learned my addr 172.21.15.86:0/2294608354 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.878+0000 7f4b567fc640 1 -- 172.21.15.86:0/2294608354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b63fbe0e0 msgr2=0x563b63fbdb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.878+0000 7f4b567fc640 1 --2- 172.21.15.86:0/2294608354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b63fbe0e0 0x563b63fbdb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.878+0000 7f4b56ffd640 1 --2- 172.21.15.86:0/2294608354 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b63fb4b60 0x563b63fc9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.878+0000 7f4b567fc640 1 -- 172.21.15.86:0/2294608354 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b63fb4b60 msgr2=0x563b63fc9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.878+0000 7f4b567fc640 1 --2- 172.21.15.86:0/2294608354 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b63fb4b60 0x563b63fc9cb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.878+0000 7f4b567fc640 1 -- 172.21.15.86:0/2294608354 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563b63fb3e30 con 0x563b63fb58d0 2024-04-21T08:43:29.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.878+0000 7f4b567fc640 1 --2- 172.21.15.86:0/2294608354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b63fb58d0 0x563b63fbd5c0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0cd3f0 tx=0x7f4b4c017650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=53a40b631f405002 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.878+0000 7f4b3cff9640 1 -- 172.21.15.86:0/2294608354 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c01e040 con 0x563b63fb58d0 2024-04-21T08:43:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.878+0000 7f4b3cff9640 1 -- 172.21.15.86:0/2294608354 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b4c0bada0 con 0x563b63fb58d0 2024-04-21T08:43:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.878+0000 7f4b3cff9640 1 -- 172.21.15.86:0/2294608354 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b4c010d90 con 0x563b63fb58d0 2024-04-21T08:43:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.879+0000 7f4b60415740 1 -- 172.21.15.86:0/2294608354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b63fb58d0 msgr2=0x563b63fbd5c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.879+0000 7f4b60415740 1 --2- 172.21.15.86:0/2294608354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b63fb58d0 0x563b63fbd5c0 secure :-1 s=READY pgs=1179 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c0cd3f0 tx=0x7f4b4c017650 comp rx=0 tx=0).stop 2024-04-21T08:43:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.879+0000 7f4b60415740 1 -- 172.21.15.86:0/2294608354 shutdown_connections 2024-04-21T08:43:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.879+0000 7f4b60415740 1 --2- 172.21.15.86:0/2294608354 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b63fbe0e0 0x563b63fbdb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.879+0000 7f4b60415740 1 --2- 172.21.15.86:0/2294608354 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b63fb58d0 0x563b63fbd5c0 unknown :-1 s=CLOSED pgs=1179 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.879+0000 7f4b60415740 1 --2- 172.21.15.86:0/2294608354 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b63fb4b60 0x563b63fc9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.879+0000 7f4b60415740 1 -- 172.21.15.86:0/2294608354 >> 172.21.15.86:0/2294608354 conn(0x563b63f7bc90 msgr2=0x563b63fb97a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:29.884 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.879+0000 7f4b60415740 1 -- 172.21.15.86:0/2294608354 shutdown_connections 2024-04-21T08:43:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.879+0000 7f4b60415740 1 -- 172.21.15.86:0/2294608354 wait complete. 2024-04-21T08:43:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.879+0000 7f4b60415740 1 -- start start 2024-04-21T08:43:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.879+0000 7f4b60415740 1 Processor -- start 2024-04-21T08:43:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.880+0000 7f4b60415740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b63fb4b60 0x563b63fc6690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.880+0000 7f4b60415740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b63fb58d0 0x563b63fb84a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.880+0000 7f4b60415740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b63fbe0e0 0x563b63fb8ad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.880+0000 7f4b60415740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b581b0750 con 0x563b63fb4b60 2024-04-21T08:43:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.880+0000 7f4b60415740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4b581b0080 con 0x563b63fb58d0 2024-04-21T08:43:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.880+0000 7f4b60415740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4b581ad2a0 con 0x563b63fbe0e0 2024-04-21T08:43:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.880+0000 7f4b60415740 10 client.0 _ll_register_callbacks cb 0x7f4b48009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.880+0000 7f4b56ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b63fb4b60 0x563b63fc6690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:29.885 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.880+0000 7f4b56ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b63fb4b60 0x563b63fc6690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33312/0 (socket says 172.21.15.86:33312) 2024-04-21T08:43:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.880+0000 7f4b56ffd640 1 -- 172.21.15.86:0/695626481 learned_addr learned my addr 172.21.15.86:0/695626481 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.880+0000 7f4b577fe640 1 --2- 172.21.15.86:0/695626481 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b63fbe0e0 0x563b63fb8ad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.880+0000 7f4b567fc640 1 --2- 172.21.15.86:0/695626481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b63fb58d0 0x563b63fb84a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.880+0000 7f4b56ffd640 1 -- 172.21.15.86:0/695626481 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b63fbe0e0 msgr2=0x563b63fb8ad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.880+0000 7f4b56ffd640 1 --2- 172.21.15.86:0/695626481 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b63fbe0e0 0x563b63fb8ad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.880+0000 7f4b56ffd640 1 -- 172.21.15.86:0/695626481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b63fb58d0 msgr2=0x563b63fb84a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.880+0000 7f4b56ffd640 1 --2- 172.21.15.86:0/695626481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b63fb58d0 0x563b63fb84a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.880+0000 7f4b56ffd640 1 -- 172.21.15.86:0/695626481 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563b63fbbf90 con 0x563b63fb4b60 2024-04-21T08:43:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.880+0000 7f4b567fc640 1 --2- 172.21.15.86:0/695626481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b63fb58d0 0x563b63fb84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.880+0000 7f4b56ffd640 1 --2- 172.21.15.86:0/695626481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b63fb4b60 0x563b63fc6690 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f4b48009bf0 tx=0x7f4b48040530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f4de54508d8911 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.880+0000 7f4b3aff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:29.886 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.880+0000 7f4b3aff5640 1 -- 172.21.15.86:0/695626481 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b48044040 con 0x563b63fb4b60 2024-04-21T08:43:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.880+0000 7f4b3aff5640 1 -- 172.21.15.86:0/695626481 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f4b48040bc0 con 0x563b63fb4b60 2024-04-21T08:43:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.880+0000 7f4b60415740 1 -- 172.21.15.86:0/695626481 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b4c0bada0 con 0x563b63fb4b60 2024-04-21T08:43:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.880+0000 7f4b3aff5640 1 -- 172.21.15.86:0/695626481 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4b48039040 con 0x563b63fb4b60 2024-04-21T08:43:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.880+0000 7f4b60415740 10 client.17712 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.880+0000 7f4b60415740 1 -- 172.21.15.86:0/695626481 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563b640348a0 con 0x563b63fb4b60 2024-04-21T08:43:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.881+0000 7f4b3aff5640 1 -- 172.21.15.86:0/695626481 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b4c0bada0 con 0x563b63fb4b60 2024-04-21T08:43:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.881+0000 7f4b3aff5640 1 -- 172.21.15.86:0/695626481 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4b48034440 con 0x563b63fb4b60 2024-04-21T08:43:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.881+0000 7f4b60415740 10 client.17712 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.881+0000 7f4b60415740 10 client.17712 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.881+0000 7f4b60415740 20 client.17712 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.881+0000 7f4b60415740 1 --2- 172.21.15.86:0/695626481 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563b640350a0 0x563b640374c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:29.887 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.881+0000 7f4b60415740 4 client.17712 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:43:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.881+0000 7f4b60415740 1 -- 172.21.15.86:0/695626481 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563b63eb93d0 con 0x563b640350a0 2024-04-21T08:43:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.881+0000 7f4b567fc640 1 --2- 172.21.15.86:0/695626481 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563b640350a0 0x563b640374c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:29.888 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.881+0000 7f4b567fc640 1 --2- 172.21.15.86:0/695626481 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563b640350a0 0x563b640374c0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=221b92a2780230c2 server_cookie=e9f101af738b0d55 in_seq=0 out_seq=0 2024-04-21T08:43:29.891 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.881+0000 7f4b3aff5640 10 client.17712 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.896+0000 7f4b3aff5640 1 -- 172.21.15.86:0/695626481 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563b63eb93d0 con 0x563b640350a0 2024-04-21T08:43:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.896+0000 7f4b3aff5640 10 client.17712 handle_command_reply: tid=0 2024-04-21T08:43:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.896+0000 7f4b60415740 1 client.17712 shutdown 2024-04-21T08:43:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.896+0000 7f4b60415740 1 -- 172.21.15.86:0/695626481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b63fb4b60 msgr2=0x563b63fc6690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.896+0000 7f4b60415740 1 --2- 172.21.15.86:0/695626481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b63fb4b60 0x563b63fc6690 secure :-1 s=READY pgs=1180 cs=0 l=1 rev1=1 crypto rx=0x7f4b48009bf0 tx=0x7f4b48040530 comp rx=0 tx=0).stop 2024-04-21T08:43:29.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.896+0000 7f4b60415740 1 -- 172.21.15.86:0/695626481 shutdown_connections 2024-04-21T08:43:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.896+0000 7f4b60415740 1 --2- 172.21.15.86:0/695626481 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563b640350a0 0x563b640374c0 crc :-1 s=READY pgs=164 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.896+0000 7f4b60415740 1 --2- 172.21.15.86:0/695626481 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563b63fbe0e0 0x563b63fb8ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.896+0000 7f4b60415740 1 --2- 172.21.15.86:0/695626481 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563b63fb58d0 0x563b63fb84a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.896+0000 7f4b3aff5640 0 client.17712 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.896+0000 7f4b60415740 1 --2- 172.21.15.86:0/695626481 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563b63fb4b60 0x563b63fc6690 unknown :-1 s=CLOSED pgs=1180 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.896+0000 7f4b60415740 1 -- 172.21.15.86:0/695626481 >> 172.21.15.86:0/695626481 conn(0x563b63f7bc90 msgr2=0x563b63fb5ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.896+0000 7f4b60415740 1 -- 172.21.15.86:0/695626481 shutdown_connections 2024-04-21T08:43:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.896+0000 7f4b60415740 1 -- 172.21.15.86:0/695626481 wait complete. 2024-04-21T08:43:29.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.897+0000 7f4b60415740 20 client.17712 trim_cache size 0 max 16384 2024-04-21T08:43:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:43:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:43:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:43:29.898 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:43:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.897+0000 7f4b5d74b640 1 -- 172.21.15.86:0/1236338036 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4b400a3dc0 msgr2=0x7f4b400a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.897+0000 7f4b5d74b640 1 --2- 172.21.15.86:0/1236338036 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4b400a3dc0 0x7f4b400a6280 secure :-1 s=READY pgs=183 cs=0 l=1 rev1=1 crypto rx=0x7f4b50012fd0 tx=0x7f4b5001a000 comp rx=0 tx=0).stop 2024-04-21T08:43:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.897+0000 7f4b5d74b640 1 -- 172.21.15.86:0/1236338036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b58103b00 msgr2=0x7f4b58198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.897+0000 7f4b5d74b640 1 --2- 172.21.15.86:0/1236338036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b58103b00 0x7f4b58198c90 secure :-1 s=READY pgs=1176 cs=0 l=1 rev1=1 crypto rx=0x7f4b4c00b4a0 tx=0x7f4b4c00b960 comp rx=0 tx=0).stop 2024-04-21T08:43:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.897+0000 7f4b5d74b640 1 -- 172.21.15.86:0/1236338036 shutdown_connections 2024-04-21T08:43:29.899 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.897+0000 7f4b5d74b640 1 --2- 172.21.15.86:0/1236338036 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4b400a3dc0 0x7f4b400a6280 unknown :-1 s=CLOSED pgs=183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.897+0000 7f4b5d74b640 1 --2- 172.21.15.86:0/1236338036 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b5810c520 0x7f4b5819cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.897+0000 7f4b5d74b640 1 --2- 172.21.15.86:0/1236338036 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b58103b00 0x7f4b58198c90 unknown :-1 s=CLOSED pgs=1176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.897+0000 7f4b5d74b640 1 --2- 172.21.15.86:0/1236338036 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b581031e0 0x7f4b58198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.897+0000 7f4b5d74b640 1 -- 172.21.15.86:0/1236338036 >> 172.21.15.86:0/1236338036 conn(0x7f4b58077fd0 msgr2=0x7f4b580fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.897+0000 7f4b5d74b640 1 -- 172.21.15.86:0/1236338036 shutdown_connections 2024-04-21T08:43:29.900 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:29.897+0000 7f4b5d74b640 1 -- 172.21.15.86:0/1236338036 wait complete. 2024-04-21T08:43:30.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:30 smithi086 ceph-mon[27939]: pgmap v144: 129 pgs: 129 active+clean; 2.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 51 KiB/s wr, 23 op/s 2024-04-21T08:43:30.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:30 smithi088 ceph-mon[31259]: pgmap v144: 129 pgs: 129 active+clean; 2.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 51 KiB/s wr, 23 op/s 2024-04-21T08:43:30.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:30 smithi179 ceph-mon[31829]: pgmap v144: 129 pgs: 129 active+clean; 2.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 51 KiB/s wr, 23 op/s 2024-04-21T08:43:32.798 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:32 smithi179 ceph-mon[31829]: pgmap v145: 129 pgs: 129 active+clean; 2.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 12 op/s 2024-04-21T08:43:32.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:32 smithi086 ceph-mon[27939]: pgmap v145: 129 pgs: 129 active+clean; 2.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 12 op/s 2024-04-21T08:43:32.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:32 smithi088 ceph-mon[31259]: pgmap v145: 129 pgs: 129 active+clean; 2.0 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 12 op/s 2024-04-21T08:43:34.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:34 smithi086 ceph-mon[27939]: pgmap v146: 129 pgs: 129 active+clean; 2.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 22 op/s 2024-04-21T08:43:34.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:34 smithi088 ceph-mon[31259]: pgmap v146: 129 pgs: 129 active+clean; 2.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 22 op/s 2024-04-21T08:43:34.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:34 smithi179 ceph-mon[31829]: pgmap v146: 129 pgs: 129 active+clean; 2.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 22 op/s 2024-04-21T08:43:36.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:36 smithi086 ceph-mon[27939]: pgmap v147: 129 pgs: 129 active+clean; 2.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-04-21T08:43:36.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:36 smithi088 ceph-mon[31259]: pgmap v147: 129 pgs: 129 active+clean; 2.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-04-21T08:43:36.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:36 smithi179 ceph-mon[31829]: pgmap v147: 129 pgs: 129 active+clean; 2.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-04-21T08:43:38.337 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.337 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.338 INFO:tasks.workunit.client.0.smithi086.stdout:127/600: open fd = 3 2024-04-21T08:43:38.338 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.338 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.338 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.338 INFO:tasks.workunit.client.0.smithi086.stdout:128/600: open fd = 3 2024-04-21T08:43:38.338 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.338 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.338 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.338 INFO:tasks.workunit.client.0.smithi086.stdout:129/600: open fd = 3 2024-04-21T08:43:38.338 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.338 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.338 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.338 INFO:tasks.workunit.client.0.smithi086.stdout:130/600: open fd = 3 2024-04-21T08:43:38.339 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.339 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.339 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.339 INFO:tasks.workunit.client.0.smithi086.stdout:131/600: open fd = 3 2024-04-21T08:43:38.339 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.339 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.339 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.339 INFO:tasks.workunit.client.0.smithi086.stdout:132/600: open fd = 3 2024-04-21T08:43:38.339 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.339 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.339 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.339 INFO:tasks.workunit.client.0.smithi086.stdout:133/600: open fd = 3 2024-04-21T08:43:38.339 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.340 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.340 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.340 INFO:tasks.workunit.client.0.smithi086.stdout:134/600: open fd = 3 2024-04-21T08:43:38.340 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.340 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.340 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.340 INFO:tasks.workunit.client.0.smithi086.stdout:135/600: open fd = 3 2024-04-21T08:43:38.340 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.340 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.340 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.340 INFO:tasks.workunit.client.0.smithi086.stdout:136/600: open fd = 3 2024-04-21T08:43:38.340 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.340 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.341 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.341 INFO:tasks.workunit.client.0.smithi086.stdout:137/600: open fd = 3 2024-04-21T08:43:38.341 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.341 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.341 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.341 INFO:tasks.workunit.client.0.smithi086.stdout:138/600: open fd = 3 2024-04-21T08:43:38.341 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.341 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.341 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.341 INFO:tasks.workunit.client.0.smithi086.stdout:139/600: open fd = 3 2024-04-21T08:43:38.341 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.341 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.341 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.341 INFO:tasks.workunit.client.0.smithi086.stdout:140/600: open fd = 3 2024-04-21T08:43:38.342 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.342 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.342 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.342 INFO:tasks.workunit.client.0.smithi086.stdout:141/600: open fd = 3 2024-04-21T08:43:38.342 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.342 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.342 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.342 INFO:tasks.workunit.client.0.smithi086.stdout:142/600: open fd = 3 2024-04-21T08:43:38.342 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.342 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.342 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.342 INFO:tasks.workunit.client.0.smithi086.stdout:143/600: open fd = 3 2024-04-21T08:43:38.342 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.343 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.343 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.343 INFO:tasks.workunit.client.0.smithi086.stdout:144/600: open fd = 3 2024-04-21T08:43:38.343 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.343 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.343 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.343 INFO:tasks.workunit.client.0.smithi086.stdout:145/600: open fd = 3 2024-04-21T08:43:38.343 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.343 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.343 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.343 INFO:tasks.workunit.client.0.smithi086.stdout:146/600: open fd = 3 2024-04-21T08:43:38.343 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.343 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.343 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.344 INFO:tasks.workunit.client.0.smithi086.stdout:147/600: open fd = 3 2024-04-21T08:43:38.344 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.344 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.344 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.344 INFO:tasks.workunit.client.0.smithi086.stdout:148/600: open fd = 3 2024-04-21T08:43:38.344 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.344 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.344 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.344 INFO:tasks.workunit.client.0.smithi086.stdout:149/600: open fd = 3 2024-04-21T08:43:38.344 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.344 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.344 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.344 INFO:tasks.workunit.client.0.smithi086.stdout:150/600: open fd = 3 2024-04-21T08:43:38.344 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.345 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.345 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.345 INFO:tasks.workunit.client.0.smithi086.stdout:151/600: open fd = 3 2024-04-21T08:43:38.345 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.345 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.345 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.345 INFO:tasks.workunit.client.0.smithi086.stdout:152/600: open fd = 3 2024-04-21T08:43:38.345 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.345 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.345 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.345 INFO:tasks.workunit.client.0.smithi086.stdout:153/600: open fd = 3 2024-04-21T08:43:38.345 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.345 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.346 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.346 INFO:tasks.workunit.client.0.smithi086.stdout:154/600: open fd = 3 2024-04-21T08:43:38.346 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.346 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.346 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.346 INFO:tasks.workunit.client.0.smithi086.stdout:155/600: open fd = 3 2024-04-21T08:43:38.346 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.346 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.346 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.346 INFO:tasks.workunit.client.0.smithi086.stdout:156/600: open fd = 3 2024-04-21T08:43:38.346 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.346 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.346 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.346 INFO:tasks.workunit.client.0.smithi086.stdout:157/600: open fd = 3 2024-04-21T08:43:38.347 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.347 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.347 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.347 INFO:tasks.workunit.client.0.smithi086.stdout:158/600: open fd = 3 2024-04-21T08:43:38.347 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.347 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.347 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.347 INFO:tasks.workunit.client.0.smithi086.stdout:159/600: open fd = 3 2024-04-21T08:43:38.347 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.347 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.347 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.347 INFO:tasks.workunit.client.0.smithi086.stdout:160/600: open fd = 3 2024-04-21T08:43:38.347 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.348 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.348 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.348 INFO:tasks.workunit.client.0.smithi086.stdout:161/600: open fd = 3 2024-04-21T08:43:38.348 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.348 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.348 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.348 INFO:tasks.workunit.client.0.smithi086.stdout:162/600: open fd = 3 2024-04-21T08:43:38.348 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.348 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.348 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.348 INFO:tasks.workunit.client.0.smithi086.stdout:163/600: open fd = 3 2024-04-21T08:43:38.348 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.348 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.348 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.349 INFO:tasks.workunit.client.0.smithi086.stdout:164/600: open fd = 3 2024-04-21T08:43:38.349 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.349 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.349 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.349 INFO:tasks.workunit.client.0.smithi086.stdout:165/600: open fd = 3 2024-04-21T08:43:38.349 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.349 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.349 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.349 INFO:tasks.workunit.client.0.smithi086.stdout:166/600: open fd = 3 2024-04-21T08:43:38.349 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.349 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.349 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.349 INFO:tasks.workunit.client.0.smithi086.stdout:167/600: open fd = 3 2024-04-21T08:43:38.349 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.350 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.350 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.350 INFO:tasks.workunit.client.0.smithi086.stdout:168/600: open fd = 3 2024-04-21T08:43:38.350 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.350 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.350 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.350 INFO:tasks.workunit.client.0.smithi086.stdout:169/600: open fd = 3 2024-04-21T08:43:38.350 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.350 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.350 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.350 INFO:tasks.workunit.client.0.smithi086.stdout:170/600: open fd = 3 2024-04-21T08:43:38.350 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.350 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.350 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.351 INFO:tasks.workunit.client.0.smithi086.stdout:171/600: open fd = 3 2024-04-21T08:43:38.351 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.351 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.351 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.351 INFO:tasks.workunit.client.0.smithi086.stdout:172/600: open fd = 3 2024-04-21T08:43:38.351 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.351 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.351 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.351 INFO:tasks.workunit.client.0.smithi086.stdout:173/600: open fd = 3 2024-04-21T08:43:38.351 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.351 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.351 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.351 INFO:tasks.workunit.client.0.smithi086.stdout:174/600: open fd = 3 2024-04-21T08:43:38.351 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.352 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.352 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.352 INFO:tasks.workunit.client.0.smithi086.stdout:175/600: open fd = 3 2024-04-21T08:43:38.352 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.352 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.352 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.352 INFO:tasks.workunit.client.0.smithi086.stdout:176/600: open fd = 3 2024-04-21T08:43:38.352 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.352 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.352 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.352 INFO:tasks.workunit.client.0.smithi086.stdout:177/600: open fd = 3 2024-04-21T08:43:38.352 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.352 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.352 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.353 INFO:tasks.workunit.client.0.smithi086.stdout:178/600: open fd = 3 2024-04-21T08:43:38.353 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.353 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.353 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.353 INFO:tasks.workunit.client.0.smithi086.stdout:179/600: open fd = 3 2024-04-21T08:43:38.353 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.353 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.353 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.353 INFO:tasks.workunit.client.0.smithi086.stdout:180/600: open fd = 3 2024-04-21T08:43:38.353 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.354 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.354 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.354 INFO:tasks.workunit.client.0.smithi086.stdout:181/600: open fd = 3 2024-04-21T08:43:38.354 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.354 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.354 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.354 INFO:tasks.workunit.client.0.smithi086.stdout:182/600: open fd = 3 2024-04-21T08:43:38.354 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.354 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.354 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.354 INFO:tasks.workunit.client.0.smithi086.stdout:183/600: open fd = 3 2024-04-21T08:43:38.354 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.354 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.354 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.355 INFO:tasks.workunit.client.0.smithi086.stdout:184/600: open fd = 3 2024-04-21T08:43:38.355 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.355 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.355 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.355 INFO:tasks.workunit.client.0.smithi086.stdout:185/600: open fd = 3 2024-04-21T08:43:38.355 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.355 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.355 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.355 INFO:tasks.workunit.client.0.smithi086.stdout:186/600: open fd = 3 2024-04-21T08:43:38.355 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.355 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.355 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.355 INFO:tasks.workunit.client.0.smithi086.stdout:187/600: open fd = 3 2024-04-21T08:43:38.355 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.355 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.356 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:43:38.356 INFO:tasks.workunit.client.0.smithi086.stdout:188/600: open fd = 3 2024-04-21T08:43:38.356 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.356 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:43:38.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:38 smithi086 ceph-mon[27939]: pgmap v148: 129 pgs: 129 active+clean; 2.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-04-21T08:43:38.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:38 smithi088 ceph-mon[31259]: pgmap v148: 129 pgs: 129 active+clean; 2.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-04-21T08:43:38.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:38 smithi179 ceph-mon[31829]: pgmap v148: 129 pgs: 129 active+clean; 2.2 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 36 KiB/s wr, 15 op/s 2024-04-21T08:43:40.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:40 smithi086 ceph-mon[27939]: pgmap v149: 129 pgs: 129 active+clean; 2.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 51 KiB/s wr, 22 op/s 2024-04-21T08:43:40.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:40 smithi088 ceph-mon[31259]: pgmap v149: 129 pgs: 129 active+clean; 2.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 51 KiB/s wr, 22 op/s 2024-04-21T08:43:40.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:40 smithi179 ceph-mon[31829]: pgmap v149: 129 pgs: 129 active+clean; 2.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 51 KiB/s wr, 22 op/s 2024-04-21T08:43:42.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:42 smithi086 ceph-mon[27939]: pgmap v150: 129 pgs: 129 active+clean; 2.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-21T08:43:42.959 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:43:42.959 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:43:42.963 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:42 smithi088 ceph-mon[31259]: pgmap v150: 129 pgs: 129 active+clean; 2.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-21T08:43:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.993+0000 7fcab6559640 1 -- 172.21.15.86:0/3682635102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcab0101aa0 msgr2=0x7fcab0105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.993+0000 7fcab6559640 1 --2- 172.21.15.86:0/3682635102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcab0101aa0 0x7fcab0105a90 secure :-1 s=READY pgs=1181 cs=0 l=1 rev1=1 crypto rx=0x7fcaa4009900 tx=0x7fcaa402bc30 comp rx=0 tx=0).stop 2024-04-21T08:43:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.993+0000 7fcab6559640 1 -- 172.21.15.86:0/3682635102 shutdown_connections 2024-04-21T08:43:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.993+0000 7fcab6559640 1 --2- 172.21.15.86:0/3682635102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcab0106170 0x7fcab010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.993+0000 7fcab6559640 1 --2- 172.21.15.86:0/3682635102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcab0101aa0 0x7fcab0105a90 unknown :-1 s=CLOSED pgs=1181 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.993+0000 7fcab6559640 1 --2- 172.21.15.86:0/3682635102 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcab01010f0 0x7fcab01014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:42.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.993+0000 7fcab6559640 1 -- 172.21.15.86:0/3682635102 >> 172.21.15.86:0/3682635102 conn(0x7fcab0077fd0 msgr2=0x7fcab00783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.993+0000 7fcab6559640 1 -- 172.21.15.86:0/3682635102 shutdown_connections 2024-04-21T08:43:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.993+0000 7fcab6559640 1 -- 172.21.15.86:0/3682635102 wait complete. 2024-04-21T08:43:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.993+0000 7fcab6559640 1 Processor -- start 2024-04-21T08:43:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.993+0000 7fcab6559640 1 -- start start 2024-04-21T08:43:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.994+0000 7fcab6559640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcab01010f0 0x7fcab0197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.994+0000 7fcab6559640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcab0106170 0x7fcab0197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.994+0000 7fcab6559640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcab019bf10 0x7fcab019c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.994+0000 7fcab6559640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcab01110b0 con 0x7fcab01010f0 2024-04-21T08:43:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.994+0000 7fcab6559640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcab0110f30 con 0x7fcab0106170 2024-04-21T08:43:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.994+0000 7fcab6559640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fcab0111230 con 0x7fcab019bf10 2024-04-21T08:43:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.994+0000 7fcaaffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcab01010f0 0x7fcab0197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.994+0000 7fcaaffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcab01010f0 0x7fcab0197650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37384/0 (socket says 172.21.15.86:37384) 2024-04-21T08:43:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.994+0000 7fcaaffff640 1 -- 172.21.15.86:0/3235469922 learned_addr learned my addr 172.21.15.86:0/3235469922 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:42.997 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.994+0000 7fcab4af6640 1 --2- 172.21.15.86:0/3235469922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcab019bf10 0x7fcab019c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.994+0000 7fcaaf7fe640 1 --2- 172.21.15.86:0/3235469922 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcab0106170 0x7fcab0197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.994+0000 7fcaaffff640 1 -- 172.21.15.86:0/3235469922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcab019bf10 msgr2=0x7fcab019c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.994+0000 7fcaaffff640 1 --2- 172.21.15.86:0/3235469922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcab019bf10 0x7fcab019c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.994+0000 7fcaaffff640 1 -- 172.21.15.86:0/3235469922 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcab0106170 msgr2=0x7fcab0197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.994+0000 7fcaaffff640 1 --2- 172.21.15.86:0/3235469922 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcab0106170 0x7fcab0197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.994+0000 7fcaaffff640 1 -- 172.21.15.86:0/3235469922 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcab019ca00 con 0x7fcab01010f0 2024-04-21T08:43:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.995+0000 7fcaaffff640 1 --2- 172.21.15.86:0/3235469922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcab01010f0 0x7fcab0197650 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fcaa000d950 tx=0x7fcaa000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.995+0000 7fcaad7fa640 1 -- 172.21.15.86:0/3235469922 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0017070 con 0x7fcab01010f0 2024-04-21T08:43:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.995+0000 7fcab6559640 1 -- 172.21.15.86:0/3235469922 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcab019ccf0 con 0x7fcab01010f0 2024-04-21T08:43:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.995+0000 7fcaad7fa640 1 -- 172.21.15.86:0/3235469922 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaa0009c30 con 0x7fcab01010f0 2024-04-21T08:43:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.995+0000 7fcaad7fa640 1 -- 172.21.15.86:0/3235469922 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa000f620 con 0x7fcab01010f0 2024-04-21T08:43:42.998 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.995+0000 7fcab6559640 1 -- 172.21.15.86:0/3235469922 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcab01a44b0 con 0x7fcab01010f0 2024-04-21T08:43:42.999 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:42 smithi179 ceph-mon[31829]: pgmap v150: 129 pgs: 129 active+clean; 2.4 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-21T08:43:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.997+0000 7fcaad7fa640 1 -- 172.21.15.86:0/3235469922 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcaa0010460 con 0x7fcab01010f0 2024-04-21T08:43:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.997+0000 7fcab922f740 1 Processor -- start 2024-04-21T08:43:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.998+0000 7fcab922f740 1 -- start start 2024-04-21T08:43:43.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.998+0000 7fcab922f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2735be8d0 0x55b2735bd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.998+0000 7fcab922f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2735bdb60 0x55b2735bdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.998+0000 7fcab922f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2735c92f0 0x55b2735cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.998+0000 7fcab922f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b2735228d0 con 0x55b2735c92f0 2024-04-21T08:43:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.998+0000 7fcab922f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b2734bfe80 con 0x55b2735bdb60 2024-04-21T08:43:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.998+0000 7fcab922f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b27351e890 con 0x55b2735be8d0 2024-04-21T08:43:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.998+0000 7fcaaf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2735c92f0 0x55b2735cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.998+0000 7fcaaf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2735c92f0 0x55b2735cb6e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37396/0 (socket says 172.21.15.86:37396) 2024-04-21T08:43:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.998+0000 7fcaaf7fe640 1 -- 172.21.15.86:0/3109424458 learned_addr learned my addr 172.21.15.86:0/3109424458 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.998+0000 7fcaaffff640 1 --2- 172.21.15.86:0/3109424458 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2735bdb60 0x55b2735bdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.998+0000 7fcaad7fa640 1 --2- 172.21.15.86:0/3235469922 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fca980a3dc0 0x7fca980a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.998+0000 7fcab4af6640 1 --2- 172.21.15.86:0/3109424458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2735be8d0 0x55b2735bd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.998+0000 7fcab4af6640 1 --2- 172.21.15.86:0/3235469922 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fca980a3dc0 0x7fca980a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.998+0000 7fcaaf7fe640 1 -- 172.21.15.86:0/3109424458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2735be8d0 msgr2=0x55b2735bd620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.998+0000 7fcaad7fa640 1 -- 172.21.15.86:0/3235469922 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fcaa00d07b0 con 0x7fcab01010f0 2024-04-21T08:43:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.998+0000 7fcaaf7fe640 1 --2- 172.21.15.86:0/3109424458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2735be8d0 0x55b2735bd620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.998+0000 7fcaaf7fe640 1 -- 172.21.15.86:0/3109424458 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2735bdb60 msgr2=0x55b2735bdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.998+0000 7fcaaf7fe640 1 --2- 172.21.15.86:0/3109424458 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2735bdb60 0x55b2735bdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.999+0000 7fcaaf7fe640 1 -- 172.21.15.86:0/3109424458 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2734e1320 con 0x55b2735c92f0 2024-04-21T08:43:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.999+0000 7fcaaffff640 1 --2- 172.21.15.86:0/3109424458 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2735bdb60 0x55b2735bdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:43:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.999+0000 7fcab4af6640 1 --2- 172.21.15.86:0/3109424458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2735be8d0 0x55b2735bd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:43:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.999+0000 7fcaaf7fe640 1 --2- 172.21.15.86:0/3109424458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2735c92f0 0x55b2735cb6e0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fcab006b730 tx=0x7fcaa4005e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44d189f41e2b09b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.999+0000 7fcab4af6640 1 --2- 172.21.15.86:0/3235469922 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fca980a3dc0 0x7fca980a6280 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8005fd0 tx=0x7fcaa8004ab0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.999+0000 7fca967fc640 1 -- 172.21.15.86:0/3109424458 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4044020 con 0x55b2735c92f0 2024-04-21T08:43:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.999+0000 7fca967fc640 1 -- 172.21.15.86:0/3109424458 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaa402cc90 con 0x55b2735c92f0 2024-04-21T08:43:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:42.999+0000 7fca967fc640 1 -- 172.21.15.86:0/3109424458 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa40046c0 con 0x55b2735c92f0 2024-04-21T08:43:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.000+0000 7fcab922f740 1 -- 172.21.15.86:0/3109424458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2735c92f0 msgr2=0x55b2735cb6e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.000+0000 7fcab922f740 1 --2- 172.21.15.86:0/3109424458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2735c92f0 0x55b2735cb6e0 secure :-1 s=READY pgs=1183 cs=0 l=1 rev1=1 crypto rx=0x7fcab006b730 tx=0x7fcaa4005e70 comp rx=0 tx=0).stop 2024-04-21T08:43:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.000+0000 7fcab922f740 1 -- 172.21.15.86:0/3109424458 shutdown_connections 2024-04-21T08:43:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.000+0000 7fcab922f740 1 --2- 172.21.15.86:0/3109424458 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2735c92f0 0x55b2735cb6e0 unknown :-1 s=CLOSED pgs=1183 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.000+0000 7fcab922f740 1 --2- 172.21.15.86:0/3109424458 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2735bdb60 0x55b2735bdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.000+0000 7fcab922f740 1 --2- 172.21.15.86:0/3109424458 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2735be8d0 0x55b2735bd620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.000+0000 7fcab922f740 1 -- 172.21.15.86:0/3109424458 >> 172.21.15.86:0/3109424458 conn(0x55b273584c90 msgr2=0x55b2735b8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.000+0000 7fcab922f740 1 -- 172.21.15.86:0/3109424458 shutdown_connections 2024-04-21T08:43:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.000+0000 7fcab922f740 1 -- 172.21.15.86:0/3109424458 wait complete. 2024-04-21T08:43:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.000+0000 7fcab922f740 1 -- start start 2024-04-21T08:43:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.001+0000 7fcab922f740 1 Processor -- start 2024-04-21T08:43:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.001+0000 7fcab922f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2735bdb60 0x55b2735c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.001+0000 7fcab922f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2735be8d0 0x55b2735c5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.001+0000 7fcab922f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2735c92f0 0x55b2735c6530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.001+0000 7fcab922f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b2735cbfc0 con 0x55b2735bdb60 2024-04-21T08:43:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.001+0000 7fcab922f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b2734aaeb0 con 0x55b2735be8d0 2024-04-21T08:43:43.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.001+0000 7fcab922f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b2735cc140 con 0x55b2735c92f0 2024-04-21T08:43:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.001+0000 7fcab922f740 10 client.0 _ll_register_callbacks cb 0x55b2734d7b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.001+0000 7fcaaffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2735bdb60 0x55b2735c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.001+0000 7fcaaffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2735bdb60 0x55b2735c8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37410/0 (socket says 172.21.15.86:37410) 2024-04-21T08:43:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.001+0000 7fcaaffff640 1 -- 172.21.15.86:0/4234301493 learned_addr learned my addr 172.21.15.86:0/4234301493 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.001+0000 7fcaaf7fe640 1 --2- 172.21.15.86:0/4234301493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2735be8d0 0x55b2735c5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.001+0000 7fcab4af6640 1 --2- 172.21.15.86:0/4234301493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2735c92f0 0x55b2735c6530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.001+0000 7fcaaffff640 1 -- 172.21.15.86:0/4234301493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2735c92f0 msgr2=0x55b2735c6530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.001+0000 7fcaaffff640 1 --2- 172.21.15.86:0/4234301493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2735c92f0 0x55b2735c6530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.001+0000 7fcaaffff640 1 -- 172.21.15.86:0/4234301493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2735be8d0 msgr2=0x55b2735c5fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.001+0000 7fcaaffff640 1 --2- 172.21.15.86:0/4234301493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2735be8d0 0x55b2735c5fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.001+0000 7fcaaffff640 1 -- 172.21.15.86:0/4234301493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b273235210 con 0x55b2735bdb60 2024-04-21T08:43:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.002+0000 7fcaaffff640 1 --2- 172.21.15.86:0/4234301493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2735bdb60 0x55b2735c8c40 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fcaa00cdbf0 tx=0x7fcaa000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60bbe56b9cb7ff0b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.002+0000 7fca8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:43.009 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.002+0000 7fca8f7fe640 1 -- 172.21.15.86:0/4234301493 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0020040 con 0x55b2735bdb60 2024-04-21T08:43:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.002+0000 7fca8f7fe640 1 -- 172.21.15.86:0/4234301493 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fcaa00cde10 con 0x55b2735bdb60 2024-04-21T08:43:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.002+0000 7fcab922f740 1 -- 172.21.15.86:0/4234301493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcaa402cc90 con 0x55b2735bdb60 2024-04-21T08:43:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.002+0000 7fca8f7fe640 1 -- 172.21.15.86:0/4234301493 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcaa00ce010 con 0x55b2735bdb60 2024-04-21T08:43:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.002+0000 7fcab922f740 10 client.17724 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.002+0000 7fcab922f740 1 -- 172.21.15.86:0/4234301493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b273301620 con 0x55b2735bdb60 2024-04-21T08:43:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.002+0000 7fca8f7fe640 1 -- 172.21.15.86:0/4234301493 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaa402cc90 con 0x55b2735bdb60 2024-04-21T08:43:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.002+0000 7fca8f7fe640 1 -- 172.21.15.86:0/4234301493 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcaa00d03e0 con 0x55b2735bdb60 2024-04-21T08:43:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.002+0000 7fcab922f740 10 client.17724 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.002+0000 7fcab922f740 10 client.17724 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.002+0000 7fcab922f740 20 client.17724 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.002+0000 7fcab922f740 1 --2- 172.21.15.86:0/4234301493 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b2735d9260 0x55b2735db6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.002+0000 7fcab922f740 4 client.17724 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.002+0000 7fcab922f740 1 -- 172.21.15.86:0/4234301493 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b273536880 con 0x55b2735d9260 2024-04-21T08:43:43.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.003+0000 7fcaaf7fe640 1 --2- 172.21.15.86:0/4234301493 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b2735d9260 0x55b2735db6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.003+0000 7fca8f7fe640 10 client.17724 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.003+0000 7fcaaf7fe640 1 --2- 172.21.15.86:0/4234301493 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b2735d9260 0x55b2735db6b0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=218675047ebfc67d server_cookie=c39782289169b253 in_seq=0 out_seq=0 2024-04-21T08:43:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.004+0000 7fca8f7fe640 1 -- 172.21.15.86:0/4234301493 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b273536880 con 0x55b2735d9260 2024-04-21T08:43:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.004+0000 7fca8f7fe640 10 client.17724 handle_command_reply: tid=0 2024-04-21T08:43:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.004+0000 7fcab922f740 1 client.17724 shutdown 2024-04-21T08:43:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.004+0000 7fcab922f740 1 -- 172.21.15.86:0/4234301493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2735bdb60 msgr2=0x55b2735c8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.004+0000 7fcab922f740 1 --2- 172.21.15.86:0/4234301493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2735bdb60 0x55b2735c8c40 secure :-1 s=READY pgs=1184 cs=0 l=1 rev1=1 crypto rx=0x7fcaa00cdbf0 tx=0x7fcaa000bf60 comp rx=0 tx=0).stop 2024-04-21T08:43:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.004+0000 7fcab922f740 1 -- 172.21.15.86:0/4234301493 shutdown_connections 2024-04-21T08:43:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.004+0000 7fcab922f740 1 --2- 172.21.15.86:0/4234301493 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b2735d9260 0x55b2735db6b0 crc :-1 s=READY pgs=165 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.004+0000 7fcab922f740 1 --2- 172.21.15.86:0/4234301493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2735c92f0 0x55b2735c6530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.004+0000 7fcab922f740 1 --2- 172.21.15.86:0/4234301493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2735be8d0 0x55b2735c5fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.004+0000 7fca8f7fe640 0 client.17724 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.004+0000 7fcab922f740 1 --2- 172.21.15.86:0/4234301493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2735bdb60 0x55b2735c8c40 unknown :-1 s=CLOSED pgs=1184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.004+0000 7fcab922f740 1 -- 172.21.15.86:0/4234301493 >> 172.21.15.86:0/4234301493 conn(0x55b273584c90 msgr2=0x55b2735ba5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.004+0000 7fcab922f740 1 -- 172.21.15.86:0/4234301493 shutdown_connections 2024-04-21T08:43:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.004+0000 7fcab922f740 1 -- 172.21.15.86:0/4234301493 wait complete. 2024-04-21T08:43:43.012 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.004+0000 7fcab922f740 20 client.17724 trim_cache size 0 max 16384 2024-04-21T08:43:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.019+0000 7fcab922f740 1 Processor -- start 2024-04-21T08:43:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.019+0000 7fcab922f740 1 -- start start 2024-04-21T08:43:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.019+0000 7fcab922f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2735bdb60 0x55b2735d4ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.019+0000 7fcab922f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2735be8d0 0x55b2735c8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.019+0000 7fcab922f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2735c92f0 0x55b2735c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.019+0000 7fcab922f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcab01ad070 con 0x55b2735be8d0 2024-04-21T08:43:43.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.019+0000 7fcab922f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcab01aebb0 con 0x55b2735c92f0 2024-04-21T08:43:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.019+0000 7fcab922f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fcab01ad740 con 0x55b2735bdb60 2024-04-21T08:43:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.019+0000 7fcaaf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2735be8d0 0x55b2735c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.019+0000 7fcaaf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2735be8d0 0x55b2735c8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37416/0 (socket says 172.21.15.86:37416) 2024-04-21T08:43:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.019+0000 7fcaaf7fe640 1 -- 172.21.15.86:0/2465054330 learned_addr learned my addr 172.21.15.86:0/2465054330 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.019+0000 7fcaaffff640 1 --2- 172.21.15.86:0/2465054330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2735bdb60 0x55b2735d4ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.019+0000 7fcab4af6640 1 --2- 172.21.15.86:0/2465054330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2735c92f0 0x55b2735c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.019+0000 7fcaaf7fe640 1 -- 172.21.15.86:0/2465054330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2735bdb60 msgr2=0x55b2735d4ec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.019+0000 7fcaaf7fe640 1 --2- 172.21.15.86:0/2465054330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2735bdb60 0x55b2735d4ec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.019+0000 7fcaaf7fe640 1 -- 172.21.15.86:0/2465054330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2735c92f0 msgr2=0x55b2735c8c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.019+0000 7fcaaf7fe640 1 --2- 172.21.15.86:0/2465054330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2735c92f0 0x55b2735c8c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.019+0000 7fcaaf7fe640 1 -- 172.21.15.86:0/2465054330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b2735bce30 con 0x55b2735be8d0 2024-04-21T08:43:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.019+0000 7fcaaffff640 1 --2- 172.21.15.86:0/2465054330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2735bdb60 0x55b2735d4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:43.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.020+0000 7fcaaf7fe640 1 --2- 172.21.15.86:0/2465054330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2735be8d0 0x55b2735c8700 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fcab006b730 tx=0x7fcaa4005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b3320bf8689d368 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:43.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.020+0000 7fca95ffb640 1 -- 172.21.15.86:0/2465054330 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4044040 con 0x55b2735be8d0 2024-04-21T08:43:43.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.020+0000 7fca95ffb640 1 -- 172.21.15.86:0/2465054330 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaa402c8e0 con 0x55b2735be8d0 2024-04-21T08:43:43.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.020+0000 7fca95ffb640 1 -- 172.21.15.86:0/2465054330 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa4004040 con 0x55b2735be8d0 2024-04-21T08:43:43.025 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.020+0000 7fcab922f740 1 -- 172.21.15.86:0/2465054330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2735be8d0 msgr2=0x55b2735c8700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.025 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.020+0000 7fcab922f740 1 --2- 172.21.15.86:0/2465054330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2735be8d0 0x55b2735c8700 secure :-1 s=READY pgs=1185 cs=0 l=1 rev1=1 crypto rx=0x7fcab006b730 tx=0x7fcaa4005d50 comp rx=0 tx=0).stop 2024-04-21T08:43:43.025 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.021+0000 7fcab922f740 1 -- 172.21.15.86:0/2465054330 shutdown_connections 2024-04-21T08:43:43.025 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.021+0000 7fcab922f740 1 --2- 172.21.15.86:0/2465054330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2735c92f0 0x55b2735c8c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.025 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.021+0000 7fcab922f740 1 --2- 172.21.15.86:0/2465054330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2735be8d0 0x55b2735c8700 unknown :-1 s=CLOSED pgs=1185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.025 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.021+0000 7fcab922f740 1 --2- 172.21.15.86:0/2465054330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2735bdb60 0x55b2735d4ec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.021+0000 7fcab922f740 1 -- 172.21.15.86:0/2465054330 >> 172.21.15.86:0/2465054330 conn(0x55b273584c90 msgr2=0x55b2735b8380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.021+0000 7fcab922f740 1 -- 172.21.15.86:0/2465054330 shutdown_connections 2024-04-21T08:43:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.021+0000 7fcab922f740 1 -- 172.21.15.86:0/2465054330 wait complete. 2024-04-21T08:43:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.021+0000 7fcab922f740 1 -- start start 2024-04-21T08:43:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.021+0000 7fcab922f740 1 Processor -- start 2024-04-21T08:43:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.021+0000 7fcab922f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2735bdb60 0x55b2735d13d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.021+0000 7fcab922f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2735be8d0 0x55b2735d1910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.021+0000 7fcab922f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2735c92f0 0x55b2735c2240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.021+0000 7fcab922f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcab01ae4e0 con 0x55b2735bdb60 2024-04-21T08:43:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.021+0000 7fcab922f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcab01ade10 con 0x55b2735be8d0 2024-04-21T08:43:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.021+0000 7fcab922f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fcab01ab030 con 0x55b2735c92f0 2024-04-21T08:43:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.021+0000 7fcab922f740 10 client.0 _ll_register_callbacks cb 0x7fcaa0096f50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:43.026 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.022+0000 7fcaaffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2735bdb60 0x55b2735d13d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.027 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.022+0000 7fcaaffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2735bdb60 0x55b2735d13d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37432/0 (socket says 172.21.15.86:37432) 2024-04-21T08:43:43.027 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.022+0000 7fcaaffff640 1 -- 172.21.15.86:0/3585089203 learned_addr learned my addr 172.21.15.86:0/3585089203 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:43.027 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.022+0000 7fcaaf7fe640 1 --2- 172.21.15.86:0/3585089203 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2735be8d0 0x55b2735d1910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.027 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.022+0000 7fcaaffff640 1 -- 172.21.15.86:0/3585089203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2735c92f0 msgr2=0x55b2735c2240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.027 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.022+0000 7fcaaffff640 1 --2- 172.21.15.86:0/3585089203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2735c92f0 0x55b2735c2240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.027 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.022+0000 7fcaaffff640 1 -- 172.21.15.86:0/3585089203 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2735be8d0 msgr2=0x55b2735d1910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.027 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.022+0000 7fcaaffff640 1 --2- 172.21.15.86:0/3585089203 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2735be8d0 0x55b2735d1910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.027 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.022+0000 7fcaaffff640 1 -- 172.21.15.86:0/3585089203 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b2735c2a40 con 0x55b2735bdb60 2024-04-21T08:43:43.027 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.022+0000 7fcaaf7fe640 1 --2- 172.21.15.86:0/3585089203 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2735be8d0 0x55b2735d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:43.027 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.022+0000 7fcaaffff640 1 --2- 172.21.15.86:0/3585089203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2735bdb60 0x55b2735d13d0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fcaa009a690 tx=0x7fcaa00d0610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11ed32458f4a6ca1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:43.027 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.022+0000 7fca8f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:43.027 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.022+0000 7fca8f7fe640 1 -- 172.21.15.86:0/3585089203 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcaa0014070 con 0x55b2735bdb60 2024-04-21T08:43:43.027 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.022+0000 7fca8f7fe640 1 -- 172.21.15.86:0/3585089203 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fcaa00cde10 con 0x55b2735bdb60 2024-04-21T08:43:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.022+0000 7fcab922f740 1 -- 172.21.15.86:0/3585089203 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fcaa402c8e0 con 0x55b2735bdb60 2024-04-21T08:43:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.022+0000 7fca8f7fe640 1 -- 172.21.15.86:0/3585089203 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fcaa00ce010 con 0x55b2735bdb60 2024-04-21T08:43:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.022+0000 7fcab922f740 10 client.17730 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.022+0000 7fcab922f740 1 -- 172.21.15.86:0/3585089203 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b2735c3d50 con 0x55b2735bdb60 2024-04-21T08:43:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.022+0000 7fca8f7fe640 1 -- 172.21.15.86:0/3585089203 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcaa402c8e0 con 0x55b2735bdb60 2024-04-21T08:43:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.023+0000 7fca8f7fe640 1 -- 172.21.15.86:0/3585089203 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fcaa00b5b60 con 0x55b2735bdb60 2024-04-21T08:43:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.023+0000 7fcab922f740 10 client.17730 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.023+0000 7fcab922f740 10 client.17730 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.023+0000 7fcab922f740 20 client.17730 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.023+0000 7fcab922f740 1 --2- 172.21.15.86:0/3585089203 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b273640850 0x55b273642c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.023+0000 7fcab922f740 4 client.17730 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:43:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.023+0000 7fcab922f740 1 -- 172.21.15.86:0/3585089203 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b2734c23d0 con 0x55b273640850 2024-04-21T08:43:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.023+0000 7fcaaf7fe640 1 --2- 172.21.15.86:0/3585089203 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b273640850 0x55b273642c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.028 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.023+0000 7fca8f7fe640 10 client.17730 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:43.029 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.023+0000 7fcaaf7fe640 1 --2- 172.21.15.86:0/3585089203 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b273640850 0x55b273642c40 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6625232087b779ea server_cookie=89e897286297c436 in_seq=0 out_seq=0 2024-04-21T08:43:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.038+0000 7fca8f7fe640 1 -- 172.21.15.86:0/3585089203 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b2734c23d0 con 0x55b273640850 2024-04-21T08:43:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.038+0000 7fca8f7fe640 10 client.17730 handle_command_reply: tid=0 2024-04-21T08:43:43.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.038+0000 7fcab922f740 1 client.17730 shutdown 2024-04-21T08:43:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.038+0000 7fcab922f740 1 -- 172.21.15.86:0/3585089203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2735bdb60 msgr2=0x55b2735d13d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.038+0000 7fcab922f740 1 --2- 172.21.15.86:0/3585089203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2735bdb60 0x55b2735d13d0 secure :-1 s=READY pgs=1186 cs=0 l=1 rev1=1 crypto rx=0x7fcaa009a690 tx=0x7fcaa00d0610 comp rx=0 tx=0).stop 2024-04-21T08:43:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.038+0000 7fcab922f740 1 -- 172.21.15.86:0/3585089203 shutdown_connections 2024-04-21T08:43:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.038+0000 7fcab922f740 1 --2- 172.21.15.86:0/3585089203 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b273640850 0x55b273642c40 crc :-1 s=READY pgs=166 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.039+0000 7fca8f7fe640 0 client.17730 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.038+0000 7fcab922f740 1 --2- 172.21.15.86:0/3585089203 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b2735c92f0 0x55b2735c2240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.039+0000 7fcab922f740 1 --2- 172.21.15.86:0/3585089203 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b2735be8d0 0x55b2735d1910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.039+0000 7fcab922f740 1 --2- 172.21.15.86:0/3585089203 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b2735bdb60 0x55b2735d13d0 unknown :-1 s=CLOSED pgs=1186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.039+0000 7fcab922f740 1 -- 172.21.15.86:0/3585089203 >> 172.21.15.86:0/3585089203 conn(0x55b273584c90 msgr2=0x55b2735b7040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.039+0000 7fcab922f740 1 -- 172.21.15.86:0/3585089203 shutdown_connections 2024-04-21T08:43:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.039+0000 7fcab922f740 1 -- 172.21.15.86:0/3585089203 wait complete. 2024-04-21T08:43:43.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.039+0000 7fcab922f740 20 client.17730 trim_cache size 0 max 16384 2024-04-21T08:43:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:43:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:43:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:43:43.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:43:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.039+0000 7fcab6559640 1 -- 172.21.15.86:0/3235469922 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fca980a3dc0 msgr2=0x7fca980a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.039+0000 7fcab6559640 1 --2- 172.21.15.86:0/3235469922 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fca980a3dc0 0x7fca980a6280 secure :-1 s=READY pgs=184 cs=0 l=1 rev1=1 crypto rx=0x7fcaa8005fd0 tx=0x7fcaa8004ab0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.039+0000 7fcab6559640 1 -- 172.21.15.86:0/3235469922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcab01010f0 msgr2=0x7fcab0197650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.039+0000 7fcab6559640 1 --2- 172.21.15.86:0/3235469922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcab01010f0 0x7fcab0197650 secure :-1 s=READY pgs=1182 cs=0 l=1 rev1=1 crypto rx=0x7fcaa000d950 tx=0x7fcaa000de10 comp rx=0 tx=0).stop 2024-04-21T08:43:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.039+0000 7fcab6559640 1 -- 172.21.15.86:0/3235469922 shutdown_connections 2024-04-21T08:43:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.039+0000 7fcab6559640 1 --2- 172.21.15.86:0/3235469922 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fca980a3dc0 0x7fca980a6280 unknown :-1 s=CLOSED pgs=184 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.039+0000 7fcab6559640 1 --2- 172.21.15.86:0/3235469922 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcab019bf10 0x7fcab019c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.039+0000 7fcab6559640 1 --2- 172.21.15.86:0/3235469922 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcab0106170 0x7fcab0197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.039+0000 7fcab6559640 1 --2- 172.21.15.86:0/3235469922 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcab01010f0 0x7fcab0197650 unknown :-1 s=CLOSED pgs=1182 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.039+0000 7fcab6559640 1 -- 172.21.15.86:0/3235469922 >> 172.21.15.86:0/3235469922 conn(0x7fcab0077fd0 msgr2=0x7fcab00ff1f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.039+0000 7fcab6559640 1 -- 172.21.15.86:0/3235469922 shutdown_connections 2024-04-21T08:43:43.042 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:43.039+0000 7fcab6559640 1 -- 172.21.15.86:0/3235469922 wait complete. 2024-04-21T08:43:43.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.071+0000 7fabae811640 1 -- 172.21.15.86:0/1339212997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faba81003e0 msgr2=0x7faba8100820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.071+0000 7fabae811640 1 --2- 172.21.15.86:0/1339212997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faba81003e0 0x7faba8100820 secure :-1 s=READY pgs=1187 cs=0 l=1 rev1=1 crypto rx=0x7fab9c009930 tx=0x7fab9c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:43:43.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.071+0000 7fabae811640 1 -- 172.21.15.86:0/1339212997 shutdown_connections 2024-04-21T08:43:43.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.071+0000 7fabae811640 1 --2- 172.21.15.86:0/1339212997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faba8100d60 0x7faba810ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.071+0000 7fabae811640 1 --2- 172.21.15.86:0/1339212997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faba81003e0 0x7faba8100820 unknown :-1 s=CLOSED pgs=1187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.071+0000 7fabae811640 1 --2- 172.21.15.86:0/1339212997 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faba8107fd0 0x7faba81083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.071+0000 7fabae811640 1 -- 172.21.15.86:0/1339212997 >> 172.21.15.86:0/1339212997 conn(0x7faba8077360 msgr2=0x7faba80fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:43.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.071+0000 7fabae811640 1 -- 172.21.15.86:0/1339212997 shutdown_connections 2024-04-21T08:43:43.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.071+0000 7fabae811640 1 -- 172.21.15.86:0/1339212997 wait complete. 2024-04-21T08:43:43.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.072+0000 7fabae811640 1 Processor -- start 2024-04-21T08:43:43.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.072+0000 7fabae811640 1 -- start start 2024-04-21T08:43:43.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.072+0000 7fabae811640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faba81003e0 0x7faba81015c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.072+0000 7fabae811640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faba8100d60 0x7faba8101b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.072+0000 7fabae811640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faba8107fd0 0x7faba8102040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.072+0000 7fabae811640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faba81121a0 con 0x7faba8107fd0 2024-04-21T08:43:43.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.072+0000 7fabae811640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faba8112020 con 0x7faba81003e0 2024-04-21T08:43:43.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.072+0000 7fabae811640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faba8112320 con 0x7faba8100d60 2024-04-21T08:43:43.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.072+0000 7fabacdae640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faba8107fd0 0x7faba8102040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.072+0000 7fabacdae640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faba8107fd0 0x7faba8102040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37450/0 (socket says 172.21.15.86:37450) 2024-04-21T08:43:43.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.072+0000 7fabacdae640 1 -- 172.21.15.86:0/613134866 learned_addr learned my addr 172.21.15.86:0/613134866 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:43.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.072+0000 7faba7fff640 1 --2- 172.21.15.86:0/613134866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faba81003e0 0x7faba81015c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.072+0000 7faba77fe640 1 --2- 172.21.15.86:0/613134866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faba8100d60 0x7faba8101b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.072+0000 7fabacdae640 1 -- 172.21.15.86:0/613134866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faba8100d60 msgr2=0x7faba8101b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.072+0000 7fabacdae640 1 --2- 172.21.15.86:0/613134866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faba8100d60 0x7faba8101b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.072+0000 7fabacdae640 1 -- 172.21.15.86:0/613134866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faba81003e0 msgr2=0x7faba81015c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.072+0000 7fabacdae640 1 --2- 172.21.15.86:0/613134866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faba81003e0 0x7faba81015c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.072+0000 7fabacdae640 1 -- 172.21.15.86:0/613134866 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faba81a6d10 con 0x7faba8107fd0 2024-04-21T08:43:43.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.073+0000 7fabacdae640 1 --2- 172.21.15.86:0/613134866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faba8107fd0 0x7faba8102040 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7faba000eb10 tx=0x7faba000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:43.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.073+0000 7faba57fa640 1 -- 172.21.15.86:0/613134866 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba0004b20 con 0x7faba8107fd0 2024-04-21T08:43:43.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.073+0000 7faba57fa640 1 -- 172.21.15.86:0/613134866 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faba0004cc0 con 0x7faba8107fd0 2024-04-21T08:43:43.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.073+0000 7fabae811640 1 -- 172.21.15.86:0/613134866 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faba81a7000 con 0x7faba8107fd0 2024-04-21T08:43:43.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.073+0000 7faba57fa640 1 -- 172.21.15.86:0/613134866 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faba0010670 con 0x7faba8107fd0 2024-04-21T08:43:43.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.073+0000 7fabae811640 1 -- 172.21.15.86:0/613134866 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faba81a74d0 con 0x7faba8107fd0 2024-04-21T08:43:43.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.075+0000 7faba57fa640 1 -- 172.21.15.86:0/613134866 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faba0005020 con 0x7faba8107fd0 2024-04-21T08:43:43.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.075+0000 7fabb1404740 1 Processor -- start 2024-04-21T08:43:43.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.075+0000 7fabb1404740 1 -- start start 2024-04-21T08:43:43.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.075+0000 7fabb1404740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f9273d580 0x561f9273d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.076+0000 7fabb1404740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f9273dea0 0x561f92746540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.076+0000 7fabb1404740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561f92746a80 0x561f92748e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.076+0000 7fabb1404740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561f926a21e0 con 0x561f9273d580 2024-04-21T08:43:43.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.076+0000 7fabb1404740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x561f9263f390 con 0x561f92746a80 2024-04-21T08:43:43.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.076+0000 7fabb1404740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561f9269e1a0 con 0x561f9273dea0 2024-04-21T08:43:43.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.076+0000 7faba77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f9273d580 0x561f9273d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.076+0000 7faba77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f9273d580 0x561f9273d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37454/0 (socket says 172.21.15.86:37454) 2024-04-21T08:43:43.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.076+0000 7faba77fe640 1 -- 172.21.15.86:0/1471120079 learned_addr learned my addr 172.21.15.86:0/1471120079 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:43.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.076+0000 7faba7fff640 1 --2- 172.21.15.86:0/1471120079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f9273dea0 0x561f92746540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.076+0000 7faba57fa640 1 --2- 172.21.15.86:0/613134866 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fab900a3d20 0x7fab900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.076+0000 7fabacdae640 1 --2- 172.21.15.86:0/613134866 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fab900a3d20 0x7fab900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.083 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.076+0000 7faba77fe640 1 -- 172.21.15.86:0/1471120079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f9273dea0 msgr2=0x561f92746540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.076+0000 7faba77fe640 1 --2- 172.21.15.86:0/1471120079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f9273dea0 0x561f92746540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.076+0000 7faba77fe640 1 -- 172.21.15.86:0/1471120079 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561f92746a80 msgr2=0x561f92748e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.076+0000 7faba77fe640 1 --2- 172.21.15.86:0/1471120079 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561f92746a80 0x561f92748e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.076+0000 7faba77fe640 1 -- 172.21.15.86:0/1471120079 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f9247e7d0 con 0x561f9273d580 2024-04-21T08:43:43.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.076+0000 7faba57fa640 1 -- 172.21.15.86:0/613134866 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7faba00d10d0 con 0x7faba8107fd0 2024-04-21T08:43:43.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.076+0000 7faba77fe640 1 --2- 172.21.15.86:0/1471120079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f9273d580 0x561f9273d960 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fab9c009900 tx=0x7fab9c02c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=381ddef481ba4d09 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:43.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.076+0000 7fab8e7fc640 1 -- 172.21.15.86:0/1471120079 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c044020 con 0x561f9273d580 2024-04-21T08:43:43.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.076+0000 7fabacdae640 1 --2- 172.21.15.86:0/613134866 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fab900a3d20 0x7fab900a61e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7faba0099690 tx=0x7faba000efd0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:43.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.076+0000 7fab8e7fc640 1 -- 172.21.15.86:0/1471120079 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab9c02c7e0 con 0x561f9273d580 2024-04-21T08:43:43.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.076+0000 7fab8e7fc640 1 -- 172.21.15.86:0/1471120079 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c0054c0 con 0x561f9273d580 2024-04-21T08:43:43.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.077+0000 7fabb1404740 1 -- 172.21.15.86:0/1471120079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f9273d580 msgr2=0x561f9273d960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.077+0000 7fabb1404740 1 --2- 172.21.15.86:0/1471120079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f9273d580 0x561f9273d960 secure :-1 s=READY pgs=1189 cs=0 l=1 rev1=1 crypto rx=0x7fab9c009900 tx=0x7fab9c02c5d0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.077+0000 7fabb1404740 1 -- 172.21.15.86:0/1471120079 shutdown_connections 2024-04-21T08:43:43.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.077+0000 7fabb1404740 1 --2- 172.21.15.86:0/1471120079 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561f92746a80 0x561f92748e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.077+0000 7fabb1404740 1 --2- 172.21.15.86:0/1471120079 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f9273dea0 0x561f92746540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.077+0000 7fabb1404740 1 --2- 172.21.15.86:0/1471120079 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f9273d580 0x561f9273d960 unknown :-1 s=CLOSED pgs=1189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.077+0000 7fabb1404740 1 -- 172.21.15.86:0/1471120079 >> 172.21.15.86:0/1471120079 conn(0x561f92737cb0 msgr2=0x561f92738090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:43.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.077+0000 7fabb1404740 1 -- 172.21.15.86:0/1471120079 shutdown_connections 2024-04-21T08:43:43.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.077+0000 7fabb1404740 1 -- 172.21.15.86:0/1471120079 wait complete. 2024-04-21T08:43:43.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.077+0000 7fabb1404740 1 -- start start 2024-04-21T08:43:43.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.078+0000 7fabb1404740 1 Processor -- start 2024-04-21T08:43:43.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.078+0000 7fabb1404740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f9273d580 0x561f92743b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.078+0000 7fabb1404740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f9273dea0 0x561f92744050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.078+0000 7fabb1404740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561f92746a80 0x561f92744590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.078+0000 7fabb1404740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561f92749810 con 0x561f9273d580 2024-04-21T08:43:43.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.078+0000 7fabb1404740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x561f9265b2f0 con 0x561f92746a80 2024-04-21T08:43:43.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.078+0000 7fabb1404740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561f92749990 con 0x561f9273dea0 2024-04-21T08:43:43.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.078+0000 7fabb1404740 10 client.0 _ll_register_callbacks cb 0x561f92668580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:43.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.078+0000 7faba77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f9273d580 0x561f92743b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.078+0000 7faba77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f9273d580 0x561f92743b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37468/0 (socket says 172.21.15.86:37468) 2024-04-21T08:43:43.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.078+0000 7faba77fe640 1 -- 172.21.15.86:0/2032159276 learned_addr learned my addr 172.21.15.86:0/2032159276 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:43.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.078+0000 7faba7fff640 1 --2- 172.21.15.86:0/2032159276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f9273dea0 0x561f92744050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.078+0000 7faba77fe640 1 --2- 172.21.15.86:0/2032159276 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561f92746a80 0x561f92744590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.078+0000 7faba77fe640 1 -- 172.21.15.86:0/2032159276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f9273dea0 msgr2=0x561f92744050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.078+0000 7faba77fe640 1 --2- 172.21.15.86:0/2032159276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f9273dea0 0x561f92744050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.078+0000 7faba77fe640 1 -- 172.21.15.86:0/2032159276 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561f92746a80 msgr2=0x561f92744590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.078+0000 7faba77fe640 1 --2- 172.21.15.86:0/2032159276 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561f92746a80 0x561f92744590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.078+0000 7faba77fe640 1 -- 172.21.15.86:0/2032159276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f923b43f0 con 0x561f9273d580 2024-04-21T08:43:43.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.078+0000 7faba7fff640 1 --2- 172.21.15.86:0/2032159276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f9273dea0 0x561f92744050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:43.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.078+0000 7faba77fe640 1 --2- 172.21.15.86:0/2032159276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f9273d580 0x561f92743b10 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fab9c02c170 tx=0x7fab9c005180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a73dc8dd75a82144 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:43.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.078+0000 7fab877fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:43.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.079+0000 7fab877fe640 1 -- 172.21.15.86:0/2032159276 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c044030 con 0x561f9273d580 2024-04-21T08:43:43.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.079+0000 7fab877fe640 1 -- 172.21.15.86:0/2032159276 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fab9c004460 con 0x561f9273d580 2024-04-21T08:43:43.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.079+0000 7fabb1404740 1 -- 172.21.15.86:0/2032159276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab9c02c7e0 con 0x561f9273d580 2024-04-21T08:43:43.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.079+0000 7fab877fe640 1 -- 172.21.15.86:0/2032159276 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fab9c05b050 con 0x561f9273d580 2024-04-21T08:43:43.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.079+0000 7fabb1404740 10 client.17742 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:43.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.079+0000 7fabb1404740 1 -- 172.21.15.86:0/2032159276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561f923740f0 con 0x561f9273d580 2024-04-21T08:43:43.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.079+0000 7fab877fe640 1 -- 172.21.15.86:0/2032159276 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab9c02c7e0 con 0x561f9273d580 2024-04-21T08:43:43.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.079+0000 7fab877fe640 1 -- 172.21.15.86:0/2032159276 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fab9c003850 con 0x561f9273d580 2024-04-21T08:43:43.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.079+0000 7fabb1404740 10 client.17742 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:43.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.079+0000 7fabb1404740 10 client.17742 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:43.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.079+0000 7fabb1404740 20 client.17742 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:43.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.079+0000 7fabb1404740 1 --2- 172.21.15.86:0/2032159276 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x561f92756920 0x561f92758d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.079+0000 7fabb1404740 4 client.17742 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:43.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.079+0000 7fabb1404740 1 -- 172.21.15.86:0/2032159276 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561f926afc00 con 0x561f92756920 2024-04-21T08:43:43.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.079+0000 7faba7fff640 1 --2- 172.21.15.86:0/2032159276 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x561f92756920 0x561f92758d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.080+0000 7fab877fe640 10 client.17742 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:43.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.080+0000 7faba7fff640 1 --2- 172.21.15.86:0/2032159276 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x561f92756920 0x561f92758d10 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a304be9a0cea9aec server_cookie=b5125962bb0e468b in_seq=0 out_seq=0 2024-04-21T08:43:43.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.081+0000 7fab877fe640 1 -- 172.21.15.86:0/2032159276 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561f926afc00 con 0x561f92756920 2024-04-21T08:43:43.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.081+0000 7fab877fe640 10 client.17742 handle_command_reply: tid=0 2024-04-21T08:43:43.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.081+0000 7fabb1404740 1 client.17742 shutdown 2024-04-21T08:43:43.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.081+0000 7fabb1404740 1 -- 172.21.15.86:0/2032159276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f9273d580 msgr2=0x561f92743b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.081+0000 7fabb1404740 1 --2- 172.21.15.86:0/2032159276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f9273d580 0x561f92743b10 secure :-1 s=READY pgs=1190 cs=0 l=1 rev1=1 crypto rx=0x7fab9c02c170 tx=0x7fab9c005180 comp rx=0 tx=0).stop 2024-04-21T08:43:43.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.081+0000 7fabb1404740 1 -- 172.21.15.86:0/2032159276 shutdown_connections 2024-04-21T08:43:43.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.081+0000 7fabb1404740 1 --2- 172.21.15.86:0/2032159276 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x561f92756920 0x561f92758d10 crc :-1 s=READY pgs=167 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.081+0000 7fabb1404740 1 --2- 172.21.15.86:0/2032159276 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561f92746a80 0x561f92744590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.081+0000 7fab877fe640 0 client.17742 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:43.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.081+0000 7fabb1404740 1 --2- 172.21.15.86:0/2032159276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f9273dea0 0x561f92744050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.081+0000 7fabb1404740 1 --2- 172.21.15.86:0/2032159276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f9273d580 0x561f92743b10 unknown :-1 s=CLOSED pgs=1190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.081+0000 7fabb1404740 1 -- 172.21.15.86:0/2032159276 >> 172.21.15.86:0/2032159276 conn(0x561f92737cb0 msgr2=0x561f92739a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:43.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.081+0000 7fabb1404740 1 -- 172.21.15.86:0/2032159276 shutdown_connections 2024-04-21T08:43:43.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.081+0000 7fabb1404740 1 -- 172.21.15.86:0/2032159276 wait complete. 2024-04-21T08:43:43.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.081+0000 7fabb1404740 20 client.17742 trim_cache size 0 max 16384 2024-04-21T08:43:43.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.098+0000 7fabb1404740 1 Processor -- start 2024-04-21T08:43:43.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.098+0000 7fabb1404740 1 -- start start 2024-04-21T08:43:43.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.098+0000 7fabb1404740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561f9273d580 0x561f92746240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.099+0000 7fabb1404740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f9273dea0 0x561f92752700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.099+0000 7fabb1404740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f92746a80 0x561f92742c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.099+0000 7fabb1404740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faba81b1490 con 0x561f92746a80 2024-04-21T08:43:43.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.099+0000 7fabb1404740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faba81b2fd0 con 0x561f9273d580 2024-04-21T08:43:43.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.099+0000 7fabb1404740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faba81b1b60 con 0x561f9273dea0 2024-04-21T08:43:43.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.099+0000 7faba77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f92746a80 0x561f92742c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.099+0000 7faba77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f92746a80 0x561f92742c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37470/0 (socket says 172.21.15.86:37470) 2024-04-21T08:43:43.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.099+0000 7faba77fe640 1 -- 172.21.15.86:0/438877369 learned_addr learned my addr 172.21.15.86:0/438877369 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:43.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.099+0000 7faba7fff640 1 --2- 172.21.15.86:0/438877369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f9273dea0 0x561f92752700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.099+0000 7faba77fe640 1 --2- 172.21.15.86:0/438877369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561f9273d580 0x561f92746240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.099+0000 7faba77fe640 1 -- 172.21.15.86:0/438877369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f9273dea0 msgr2=0x561f92752700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.099+0000 7faba77fe640 1 --2- 172.21.15.86:0/438877369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f9273dea0 0x561f92752700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.099+0000 7faba77fe640 1 -- 172.21.15.86:0/438877369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561f9273d580 msgr2=0x561f92746240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.099+0000 7faba77fe640 1 --2- 172.21.15.86:0/438877369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561f9273d580 0x561f92746240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.099+0000 7faba77fe640 1 -- 172.21.15.86:0/438877369 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561f9273dcd0 con 0x561f92746a80 2024-04-21T08:43:43.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.099+0000 7faba77fe640 1 --2- 172.21.15.86:0/438877369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f92746a80 0x561f92742c70 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fab9c02bc70 tx=0x7fab9c0057a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93c6fdd18b6b802 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:43.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.099+0000 7fab8dffb640 1 -- 172.21.15.86:0/438877369 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c044070 con 0x561f92746a80 2024-04-21T08:43:43.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.099+0000 7fab8dffb640 1 -- 172.21.15.86:0/438877369 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab9c052070 con 0x561f92746a80 2024-04-21T08:43:43.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.099+0000 7fab8dffb640 1 -- 172.21.15.86:0/438877369 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab9c000e60 con 0x561f92746a80 2024-04-21T08:43:43.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.100+0000 7fabb1404740 1 -- 172.21.15.86:0/438877369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f92746a80 msgr2=0x561f92742c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.100+0000 7fabb1404740 1 --2- 172.21.15.86:0/438877369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f92746a80 0x561f92742c70 secure :-1 s=READY pgs=1191 cs=0 l=1 rev1=1 crypto rx=0x7fab9c02bc70 tx=0x7fab9c0057a0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.100+0000 7fabb1404740 1 -- 172.21.15.86:0/438877369 shutdown_connections 2024-04-21T08:43:43.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.100+0000 7fabb1404740 1 --2- 172.21.15.86:0/438877369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f92746a80 0x561f92742c70 unknown :-1 s=CLOSED pgs=1191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.100+0000 7fabb1404740 1 --2- 172.21.15.86:0/438877369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f9273dea0 0x561f92752700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.100+0000 7fabb1404740 1 --2- 172.21.15.86:0/438877369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561f9273d580 0x561f92746240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.100+0000 7fabb1404740 1 -- 172.21.15.86:0/438877369 >> 172.21.15.86:0/438877369 conn(0x561f92737cb0 msgr2=0x561f927364e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:43.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.100+0000 7fabb1404740 1 -- 172.21.15.86:0/438877369 shutdown_connections 2024-04-21T08:43:43.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.100+0000 7fabb1404740 1 -- 172.21.15.86:0/438877369 wait complete. 2024-04-21T08:43:43.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.100+0000 7fabb1404740 1 -- start start 2024-04-21T08:43:43.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.101+0000 7fabb1404740 1 Processor -- start 2024-04-21T08:43:43.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.101+0000 7fabb1404740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f9273d580 0x561f92749e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.101+0000 7fabb1404740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f9273dea0 0x561f9274a3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.101+0000 7fabb1404740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561f92746a80 0x561f927b4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.101+0000 7fabb1404740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faba81b2900 con 0x561f9273dea0 2024-04-21T08:43:43.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.101+0000 7fabb1404740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faba81b2230 con 0x561f92746a80 2024-04-21T08:43:43.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.101+0000 7fabb1404740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faba81af450 con 0x561f9273d580 2024-04-21T08:43:43.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.101+0000 7fabb1404740 10 client.0 _ll_register_callbacks cb 0x7fab9800b730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:43.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.101+0000 7faba7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f9273dea0 0x561f9274a3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.101+0000 7faba7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f9273dea0 0x561f9274a3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37482/0 (socket says 172.21.15.86:37482) 2024-04-21T08:43:43.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.101+0000 7faba7fff640 1 -- 172.21.15.86:0/62979002 learned_addr learned my addr 172.21.15.86:0/62979002 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:43.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.101+0000 7faba77fe640 1 --2- 172.21.15.86:0/62979002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f9273d580 0x561f92749e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.101+0000 7faba7fff640 1 -- 172.21.15.86:0/62979002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f9273d580 msgr2=0x561f92749e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.101+0000 7faba7fff640 1 --2- 172.21.15.86:0/62979002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f9273d580 0x561f92749e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.101+0000 7faba7fff640 1 -- 172.21.15.86:0/62979002 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561f92746a80 msgr2=0x561f927b4ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.101+0000 7faba7fff640 1 --2- 172.21.15.86:0/62979002 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561f92746a80 0x561f927b4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.101+0000 7faba7fff640 1 -- 172.21.15.86:0/62979002 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561f927b50d0 con 0x561f9273dea0 2024-04-21T08:43:43.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.101+0000 7faba7fff640 1 --2- 172.21.15.86:0/62979002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f9273dea0 0x561f9274a3c0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x561f9273d050 tx=0x7fab98011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af699208ba415113 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:43.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.101+0000 7fab877fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:43.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.101+0000 7fab877fe640 1 -- 172.21.15.86:0/62979002 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fab98004220 con 0x561f9273dea0 2024-04-21T08:43:43.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.101+0000 7fab877fe640 1 -- 172.21.15.86:0/62979002 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fab980043c0 con 0x561f9273dea0 2024-04-21T08:43:43.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.101+0000 7fabb1404740 1 -- 172.21.15.86:0/62979002 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fab9c052070 con 0x561f9273dea0 2024-04-21T08:43:43.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.102+0000 7fab877fe640 1 -- 172.21.15.86:0/62979002 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fab9802f050 con 0x561f9273dea0 2024-04-21T08:43:43.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.102+0000 7fabb1404740 10 client.17748 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:43.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.102+0000 7fabb1404740 1 -- 172.21.15.86:0/62979002 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561f927bc510 con 0x561f9273dea0 2024-04-21T08:43:43.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.102+0000 7fab877fe640 1 -- 172.21.15.86:0/62979002 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fab9c052070 con 0x561f9273dea0 2024-04-21T08:43:43.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.102+0000 7fab877fe640 1 -- 172.21.15.86:0/62979002 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fab980047a0 con 0x561f9273dea0 2024-04-21T08:43:43.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.102+0000 7fabb1404740 10 client.17748 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:43.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.102+0000 7fabb1404740 10 client.17748 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:43.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.102+0000 7fabb1404740 20 client.17748 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:43.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.102+0000 7fabb1404740 1 --2- 172.21.15.86:0/62979002 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x561f927bccc0 0x561f927bf0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.102+0000 7fabb1404740 4 client.17748 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:43:43.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.102+0000 7fabb1404740 1 -- 172.21.15.86:0/62979002 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x561f926a9a70 con 0x561f927bccc0 2024-04-21T08:43:43.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.102+0000 7faba77fe640 1 --2- 172.21.15.86:0/62979002 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x561f927bccc0 0x561f927bf0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.102+0000 7fab877fe640 10 client.17748 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:43.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.102+0000 7faba77fe640 1 --2- 172.21.15.86:0/62979002 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x561f927bccc0 0x561f927bf0e0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8f956c3ef7c1b2c server_cookie=a2119417d7df88b9 in_seq=0 out_seq=0 2024-04-21T08:43:43.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.103+0000 7fab877fe640 1 -- 172.21.15.86:0/62979002 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x561f926a9a70 con 0x561f927bccc0 2024-04-21T08:43:43.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.103+0000 7fab877fe640 10 client.17748 handle_command_reply: tid=0 2024-04-21T08:43:43.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.103+0000 7fabb1404740 1 client.17748 shutdown 2024-04-21T08:43:43.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.103+0000 7fabb1404740 1 -- 172.21.15.86:0/62979002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f9273dea0 msgr2=0x561f9274a3c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.103+0000 7fabb1404740 1 --2- 172.21.15.86:0/62979002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f9273dea0 0x561f9274a3c0 secure :-1 s=READY pgs=1192 cs=0 l=1 rev1=1 crypto rx=0x561f9273d050 tx=0x7fab98011a30 comp rx=0 tx=0).stop 2024-04-21T08:43:43.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.103+0000 7fabb1404740 1 -- 172.21.15.86:0/62979002 shutdown_connections 2024-04-21T08:43:43.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.103+0000 7fabb1404740 1 --2- 172.21.15.86:0/62979002 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x561f927bccc0 0x561f927bf0e0 crc :-1 s=READY pgs=168 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.103+0000 7fabb1404740 1 --2- 172.21.15.86:0/62979002 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561f92746a80 0x561f927b4ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.103+0000 7fabb1404740 1 --2- 172.21.15.86:0/62979002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561f9273dea0 0x561f9274a3c0 unknown :-1 s=CLOSED pgs=1192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.103+0000 7fab877fe640 0 client.17748 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:43.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.103+0000 7fabb1404740 1 --2- 172.21.15.86:0/62979002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561f9273d580 0x561f92749e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.103+0000 7fabb1404740 1 -- 172.21.15.86:0/62979002 >> 172.21.15.86:0/62979002 conn(0x561f92737cb0 msgr2=0x561f92744360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:43.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.103+0000 7fabb1404740 1 -- 172.21.15.86:0/62979002 shutdown_connections 2024-04-21T08:43:43.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.103+0000 7fabb1404740 1 -- 172.21.15.86:0/62979002 wait complete. 2024-04-21T08:43:43.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.103+0000 7fabb1404740 20 client.17748 trim_cache size 0 max 16384 2024-04-21T08:43:43.110 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:43:43.111 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-21T08:43:43.111 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-21T08:43:43.111 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:43:43.111 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.103+0000 7fabae811640 1 -- 172.21.15.86:0/613134866 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fab900a3d20 msgr2=0x7fab900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.103+0000 7fabae811640 1 --2- 172.21.15.86:0/613134866 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fab900a3d20 0x7fab900a61e0 secure :-1 s=READY pgs=185 cs=0 l=1 rev1=1 crypto rx=0x7faba0099690 tx=0x7faba000efd0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.103+0000 7fabae811640 1 -- 172.21.15.86:0/613134866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faba8107fd0 msgr2=0x7faba8102040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.103+0000 7fabae811640 1 --2- 172.21.15.86:0/613134866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faba8107fd0 0x7faba8102040 secure :-1 s=READY pgs=1188 cs=0 l=1 rev1=1 crypto rx=0x7faba000eb10 tx=0x7faba000c2d0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.103+0000 7fabae811640 1 -- 172.21.15.86:0/613134866 shutdown_connections 2024-04-21T08:43:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.103+0000 7fabae811640 1 --2- 172.21.15.86:0/613134866 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fab900a3d20 0x7fab900a61e0 unknown :-1 s=CLOSED pgs=185 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.103+0000 7fabae811640 1 --2- 172.21.15.86:0/613134866 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faba8107fd0 0x7faba8102040 unknown :-1 s=CLOSED pgs=1188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.103+0000 7fabae811640 1 --2- 172.21.15.86:0/613134866 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faba8100d60 0x7faba8101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.104+0000 7fabae811640 1 --2- 172.21.15.86:0/613134866 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faba81003e0 0x7faba81015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.104+0000 7fabae811640 1 -- 172.21.15.86:0/613134866 >> 172.21.15.86:0/613134866 conn(0x7faba8077360 msgr2=0x7faba80fec90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.104+0000 7fabae811640 1 -- 172.21.15.86:0/613134866 shutdown_connections 2024-04-21T08:43:43.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.104+0000 7fabae811640 1 -- 172.21.15.86:0/613134866 wait complete. 2024-04-21T08:43:43.114 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:43:43.115 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:43:43.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.253+0000 7f69a98c8640 1 -- 172.21.15.86:0/1783302740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69a4102190 msgr2=0x7f69a4102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.253+0000 7f69a98c8640 1 --2- 172.21.15.86:0/1783302740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69a4102190 0x7f69a4102570 secure :-1 s=READY pgs=1193 cs=0 l=1 rev1=1 crypto rx=0x7f6994009900 tx=0x7f699402bc30 comp rx=0 tx=0).stop 2024-04-21T08:43:43.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.253+0000 7f69a98c8640 1 -- 172.21.15.86:0/1783302740 shutdown_connections 2024-04-21T08:43:43.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.253+0000 7f69a98c8640 1 --2- 172.21.15.86:0/1783302740 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69a4107260 0x7f69a410ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.253+0000 7f69a98c8640 1 --2- 172.21.15.86:0/1783302740 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f69a4102b40 0x7f69a4106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.253+0000 7f69a98c8640 1 --2- 172.21.15.86:0/1783302740 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69a4102190 0x7f69a4102570 unknown :-1 s=CLOSED pgs=1193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.253+0000 7f69a98c8640 1 -- 172.21.15.86:0/1783302740 >> 172.21.15.86:0/1783302740 conn(0x7f69a40793e0 msgr2=0x7f69a40797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:43.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.253+0000 7f69a98c8640 1 -- 172.21.15.86:0/1783302740 shutdown_connections 2024-04-21T08:43:43.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.253+0000 7f69a98c8640 1 -- 172.21.15.86:0/1783302740 wait complete. 2024-04-21T08:43:43.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.254+0000 7f69a98c8640 1 Processor -- start 2024-04-21T08:43:43.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.254+0000 7f69a98c8640 1 -- start start 2024-04-21T08:43:43.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.254+0000 7f69a98c8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69a4102190 0x7f69a41a0630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.254+0000 7f69a98c8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69a4107260 0x7f69a41a0b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.254+0000 7f69a98c8640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f69a419a7a0 0x7f69a419ac50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.254+0000 7f69a98c8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f69a41121c0 con 0x7f69a4102190 2024-04-21T08:43:43.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.254+0000 7f69a98c8640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f69a4112040 con 0x7f69a4107260 2024-04-21T08:43:43.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.254+0000 7f69a98c8640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f69a4112340 con 0x7f69a419a7a0 2024-04-21T08:43:43.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.254+0000 7f69a2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69a4102190 0x7f69a41a0630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.254+0000 7f69a2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69a4102190 0x7f69a41a0630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37500/0 (socket says 172.21.15.86:37500) 2024-04-21T08:43:43.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.254+0000 7f69a2ffd640 1 -- 172.21.15.86:0/2351292169 learned_addr learned my addr 172.21.15.86:0/2351292169 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:43.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.254+0000 7f69a37fe640 1 --2- 172.21.15.86:0/2351292169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f69a419a7a0 0x7f69a419ac50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.254+0000 7f699bfff640 1 --2- 172.21.15.86:0/2351292169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69a4107260 0x7f69a41a0b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.254+0000 7f69a2ffd640 1 -- 172.21.15.86:0/2351292169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f69a419a7a0 msgr2=0x7f69a419ac50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.254+0000 7f69a2ffd640 1 --2- 172.21.15.86:0/2351292169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f69a419a7a0 0x7f69a419ac50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.254+0000 7f69a2ffd640 1 -- 172.21.15.86:0/2351292169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69a4107260 msgr2=0x7f69a41a0b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.254+0000 7f69a2ffd640 1 --2- 172.21.15.86:0/2351292169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69a4107260 0x7f69a41a0b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.254+0000 7f69a2ffd640 1 -- 172.21.15.86:0/2351292169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f69a419b330 con 0x7f69a4102190 2024-04-21T08:43:43.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.254+0000 7f699bfff640 1 --2- 172.21.15.86:0/2351292169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69a4107260 0x7f69a41a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:43.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.254+0000 7f69a2ffd640 1 --2- 172.21.15.86:0/2351292169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69a4102190 0x7f69a41a0630 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f6994005940 tx=0x7f69940045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:43.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.255+0000 7f69a88c6640 1 -- 172.21.15.86:0/2351292169 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6994043020 con 0x7f69a4102190 2024-04-21T08:43:43.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.255+0000 7f69a88c6640 1 -- 172.21.15.86:0/2351292169 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f699402caf0 con 0x7f69a4102190 2024-04-21T08:43:43.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.255+0000 7f69a98c8640 1 -- 172.21.15.86:0/2351292169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f69a419b5c0 con 0x7f69a4102190 2024-04-21T08:43:43.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.255+0000 7f69a88c6640 1 -- 172.21.15.86:0/2351292169 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6994003af0 con 0x7f69a4102190 2024-04-21T08:43:43.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.255+0000 7f69a98c8640 1 -- 172.21.15.86:0/2351292169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f69a41a77a0 con 0x7f69a4102190 2024-04-21T08:43:43.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.256+0000 7f69a98c8640 1 -- 172.21.15.86:0/2351292169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f69a4103660 con 0x7f69a4102190 2024-04-21T08:43:43.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.257+0000 7f69a88c6640 1 -- 172.21.15.86:0/2351292169 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6994046020 con 0x7f69a4102190 2024-04-21T08:43:43.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.257+0000 7f69a88c6640 1 --2- 172.21.15.86:0/2351292169 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f69900a3d20 0x7f69900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.258+0000 7f699bfff640 1 --2- 172.21.15.86:0/2351292169 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f69900a3d20 0x7f69900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.258+0000 7f69a88c6640 1 -- 172.21.15.86:0/2351292169 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f69940f31e0 con 0x7f69a4102190 2024-04-21T08:43:43.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.258+0000 7f699bfff640 1 --2- 172.21.15.86:0/2351292169 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f69900a3d20 0x7f69900a61e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f698c005fd0 tx=0x7f698c005ea0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:43.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.262+0000 7f69a88c6640 1 -- 172.21.15.86:0/2351292169 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f699404d020 con 0x7f69a4102190 2024-04-21T08:43:43.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.476+0000 7f69a98c8640 1 -- 172.21.15.86:0/2351292169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f69a4069230 con 0x7f69a4102190 2024-04-21T08:43:43.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.477+0000 7f69a88c6640 1 -- 172.21.15.86:0/2351292169 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7f69940bf620 con 0x7f69a4102190 2024-04-21T08:43:43.477 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:43:43.477 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:38:45.202126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24398,24394,24455,24472]},"id":1}]} 2024-04-21T08:43:43.477 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-21T08:43:43.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.479+0000 7f69a98c8640 1 -- 172.21.15.86:0/2351292169 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f69900a3d20 msgr2=0x7f69900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.479+0000 7f69a98c8640 1 --2- 172.21.15.86:0/2351292169 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f69900a3d20 0x7f69900a61e0 secure :-1 s=READY pgs=186 cs=0 l=1 rev1=1 crypto rx=0x7f698c005fd0 tx=0x7f698c005ea0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.479+0000 7f69a98c8640 1 -- 172.21.15.86:0/2351292169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69a4102190 msgr2=0x7f69a41a0630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.479+0000 7f69a98c8640 1 --2- 172.21.15.86:0/2351292169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69a4102190 0x7f69a41a0630 secure :-1 s=READY pgs=1194 cs=0 l=1 rev1=1 crypto rx=0x7f6994005940 tx=0x7f69940045e0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.479+0000 7f69a98c8640 1 -- 172.21.15.86:0/2351292169 shutdown_connections 2024-04-21T08:43:43.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.479+0000 7f69a98c8640 1 --2- 172.21.15.86:0/2351292169 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f69900a3d20 0x7f69900a61e0 unknown :-1 s=CLOSED pgs=186 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.479+0000 7f69a98c8640 1 --2- 172.21.15.86:0/2351292169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f69a419a7a0 0x7f69a419ac50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.479+0000 7f69a98c8640 1 --2- 172.21.15.86:0/2351292169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f69a4107260 0x7f69a41a0b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.479+0000 7f69a98c8640 1 --2- 172.21.15.86:0/2351292169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f69a4102190 0x7f69a41a0630 unknown :-1 s=CLOSED pgs=1194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.479+0000 7f69a98c8640 1 -- 172.21.15.86:0/2351292169 >> 172.21.15.86:0/2351292169 conn(0x7f69a40793e0 msgr2=0x7f69a4100110 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:43.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.479+0000 7f69a98c8640 1 -- 172.21.15.86:0/2351292169 shutdown_connections 2024-04-21T08:43:43.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.479+0000 7f69a98c8640 1 -- 172.21.15.86:0/2351292169 wait complete. 2024-04-21T08:43:43.490 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:43:43.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.594+0000 7fa057da4640 1 -- 172.21.15.86:0/2864834715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa050101090 msgr2=0x7fa050101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.594+0000 7fa057da4640 1 --2- 172.21.15.86:0/2864834715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa050101090 0x7fa050101470 secure :-1 s=READY pgs=1195 cs=0 l=1 rev1=1 crypto rx=0x7fa040009900 tx=0x7fa04002bc30 comp rx=0 tx=0).stop 2024-04-21T08:43:43.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.594+0000 7fa057da4640 1 -- 172.21.15.86:0/2864834715 shutdown_connections 2024-04-21T08:43:43.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.594+0000 7fa057da4640 1 --2- 172.21.15.86:0/2864834715 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa050106160 0x7fa05010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.594+0000 7fa057da4640 1 --2- 172.21.15.86:0/2864834715 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa050101a40 0x7fa050105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.594+0000 7fa057da4640 1 --2- 172.21.15.86:0/2864834715 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa050101090 0x7fa050101470 unknown :-1 s=CLOSED pgs=1195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.594+0000 7fa057da4640 1 -- 172.21.15.86:0/2864834715 >> 172.21.15.86:0/2864834715 conn(0x7fa050077f50 msgr2=0x7fa050078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:43.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.594+0000 7fa057da4640 1 -- 172.21.15.86:0/2864834715 shutdown_connections 2024-04-21T08:43:43.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.594+0000 7fa057da4640 1 -- 172.21.15.86:0/2864834715 wait complete. 2024-04-21T08:43:43.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.595+0000 7fa057da4640 1 Processor -- start 2024-04-21T08:43:43.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.595+0000 7fa057da4640 1 -- start start 2024-04-21T08:43:43.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.595+0000 7fa057da4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa050101090 0x7fa05019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.595+0000 7fa057da4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa050106160 0x7fa05019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.595+0000 7fa057da4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa050198590 0x7fa050198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.595+0000 7fa057da4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0501121d0 con 0x7fa050106160 2024-04-21T08:43:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.595+0000 7fa057da4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa050112050 con 0x7fa050101090 2024-04-21T08:43:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.595+0000 7fa057da4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa050112350 con 0x7fa050198590 2024-04-21T08:43:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.595+0000 7fa05533f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa050106160 0x7fa05019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.595+0000 7fa05533f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa050106160 0x7fa05019e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37506/0 (socket says 172.21.15.86:37506) 2024-04-21T08:43:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.595+0000 7fa05533f640 1 -- 172.21.15.86:0/2198289870 learned_addr learned my addr 172.21.15.86:0/2198289870 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.595+0000 7fa056341640 1 --2- 172.21.15.86:0/2198289870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa050198590 0x7fa050198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.595+0000 7fa055b40640 1 --2- 172.21.15.86:0/2198289870 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa050101090 0x7fa05019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.595+0000 7fa05533f640 1 -- 172.21.15.86:0/2198289870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa050198590 msgr2=0x7fa050198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.595+0000 7fa05533f640 1 --2- 172.21.15.86:0/2198289870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa050198590 0x7fa050198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.595+0000 7fa05533f640 1 -- 172.21.15.86:0/2198289870 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa050101090 msgr2=0x7fa05019e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.595+0000 7fa05533f640 1 --2- 172.21.15.86:0/2198289870 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa050101090 0x7fa05019e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.595+0000 7fa05533f640 1 -- 172.21.15.86:0/2198289870 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0501990b0 con 0x7fa050106160 2024-04-21T08:43:43.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.595+0000 7fa055b40640 1 --2- 172.21.15.86:0/2198289870 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa050101090 0x7fa05019e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:43.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.595+0000 7fa05533f640 1 --2- 172.21.15.86:0/2198289870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa050106160 0x7fa05019e920 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00d740 tx=0x7fa04c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:43.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.595+0000 7fa046ffd640 1 -- 172.21.15.86:0/2198289870 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04c017070 con 0x7fa050106160 2024-04-21T08:43:43.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.595+0000 7fa046ffd640 1 -- 172.21.15.86:0/2198289870 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa04c00de50 con 0x7fa050106160 2024-04-21T08:43:43.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.595+0000 7fa057da4640 1 -- 172.21.15.86:0/2198289870 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0501993a0 con 0x7fa050106160 2024-04-21T08:43:43.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.595+0000 7fa046ffd640 1 -- 172.21.15.86:0/2198289870 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04c004ce0 con 0x7fa050106160 2024-04-21T08:43:43.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.596+0000 7fa057da4640 1 -- 172.21.15.86:0/2198289870 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0501a55a0 con 0x7fa050106160 2024-04-21T08:43:43.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.598+0000 7fa046ffd640 1 -- 172.21.15.86:0/2198289870 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa04c004e80 con 0x7fa050106160 2024-04-21T08:43:43.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.598+0000 7fa05a997740 1 Processor -- start 2024-04-21T08:43:43.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.598+0000 7fa05a997740 1 -- start start 2024-04-21T08:43:43.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.598+0000 7fa05a997740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b827f4d0 0x5654b827f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.598+0000 7fa05a997740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654b827fe80 0x5654b8288430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.598+0000 7fa05a997740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654b8288a00 0x5654b828ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.598+0000 7fa05a997740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5654b81e41f0 con 0x5654b827f4d0 2024-04-21T08:43:43.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.598+0000 7fa05a997740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5654b81814b0 con 0x5654b827fe80 2024-04-21T08:43:43.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.598+0000 7fa05a997740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5654b81e01b0 con 0x5654b8288a00 2024-04-21T08:43:43.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.598+0000 7fa056341640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b827f4d0 0x5654b827f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.598+0000 7fa056341640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b827f4d0 0x5654b827f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37510/0 (socket says 172.21.15.86:37510) 2024-04-21T08:43:43.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.598+0000 7fa056341640 1 -- 172.21.15.86:0/3540745316 learned_addr learned my addr 172.21.15.86:0/3540745316 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:43.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.599+0000 7fa055b40640 1 --2- 172.21.15.86:0/3540745316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654b827fe80 0x5654b8288430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.599+0000 7fa046ffd640 1 --2- 172.21.15.86:0/2198289870 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa03c0a3d20 0x7fa03c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.599+0000 7fa05533f640 1 --2- 172.21.15.86:0/3540745316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654b8288a00 0x5654b828ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.599+0000 7fa056341640 1 -- 172.21.15.86:0/3540745316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654b8288a00 msgr2=0x5654b828ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.599+0000 7fa056341640 1 --2- 172.21.15.86:0/3540745316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654b8288a00 0x5654b828ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.599+0000 7fa056341640 1 -- 172.21.15.86:0/3540745316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654b827fe80 msgr2=0x5654b8288430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.599+0000 7fa056341640 1 --2- 172.21.15.86:0/3540745316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654b827fe80 0x5654b8288430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.599+0000 7fa056341640 1 -- 172.21.15.86:0/3540745316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654b81731b0 con 0x5654b827f4d0 2024-04-21T08:43:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.599+0000 7fa056341640 1 --2- 172.21.15.86:0/2198289870 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa03c0a3d20 0x7fa03c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.599+0000 7fa046ffd640 1 -- 172.21.15.86:0/2198289870 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fa04c0d0480 con 0x7fa050106160 2024-04-21T08:43:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.599+0000 7fa056341640 1 --2- 172.21.15.86:0/3540745316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b827f4d0 0x5654b827f8b0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fa048009990 tx=0x7fa048011a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=765bf497c4c03d97 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.599+0000 7fa03bfff640 1 -- 172.21.15.86:0/3540745316 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa048018c00 con 0x5654b827f4d0 2024-04-21T08:43:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.599+0000 7fa03bfff640 1 -- 172.21.15.86:0/3540745316 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa048018da0 con 0x5654b827f4d0 2024-04-21T08:43:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.599+0000 7fa03bfff640 1 -- 172.21.15.86:0/3540745316 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04801f370 con 0x5654b827f4d0 2024-04-21T08:43:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.600+0000 7fa05a997740 1 -- 172.21.15.86:0/3540745316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b827f4d0 msgr2=0x5654b827f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.600+0000 7fa056341640 1 --2- 172.21.15.86:0/2198289870 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa03c0a3d20 0x7fa03c0a61e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa048026000 tx=0x7fa048022030 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.600+0000 7fa05a997740 1 --2- 172.21.15.86:0/3540745316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b827f4d0 0x5654b827f8b0 secure :-1 s=READY pgs=1197 cs=0 l=1 rev1=1 crypto rx=0x7fa048009990 tx=0x7fa048011a20 comp rx=0 tx=0).stop 2024-04-21T08:43:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.600+0000 7fa05a997740 1 -- 172.21.15.86:0/3540745316 shutdown_connections 2024-04-21T08:43:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.600+0000 7fa05a997740 1 --2- 172.21.15.86:0/3540745316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654b8288a00 0x5654b828ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.600+0000 7fa05a997740 1 --2- 172.21.15.86:0/3540745316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654b827fe80 0x5654b8288430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.600+0000 7fa05a997740 1 --2- 172.21.15.86:0/3540745316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b827f4d0 0x5654b827f8b0 unknown :-1 s=CLOSED pgs=1197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.600+0000 7fa05a997740 1 -- 172.21.15.86:0/3540745316 >> 172.21.15.86:0/3540745316 conn(0x5654b8279ba0 msgr2=0x5654b8279f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.600+0000 7fa05a997740 1 -- 172.21.15.86:0/3540745316 shutdown_connections 2024-04-21T08:43:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.600+0000 7fa05a997740 1 -- 172.21.15.86:0/3540745316 wait complete. 2024-04-21T08:43:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.600+0000 7fa05a997740 1 -- start start 2024-04-21T08:43:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.600+0000 7fa05a997740 1 Processor -- start 2024-04-21T08:43:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.601+0000 7fa05a997740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b827f4d0 0x5654b8288370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.601+0000 7fa05a997740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654b827fe80 0x5654b8285700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.601+0000 7fa05a997740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654b8288a00 0x5654b8285d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.601+0000 7fa05a997740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5654b828b970 con 0x5654b827f4d0 2024-04-21T08:43:43.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.601+0000 7fa05a997740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5654b819d500 con 0x5654b827fe80 2024-04-21T08:43:43.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.601+0000 7fa05a997740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5654b828baf0 con 0x5654b8288a00 2024-04-21T08:43:43.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.601+0000 7fa05a997740 10 client.0 _ll_register_callbacks cb 0x5654b816bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:43.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.601+0000 7fa055b40640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b827f4d0 0x5654b8288370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.601+0000 7fa055b40640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b827f4d0 0x5654b8288370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37512/0 (socket says 172.21.15.86:37512) 2024-04-21T08:43:43.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.601+0000 7fa055b40640 1 -- 172.21.15.86:0/2834385624 learned_addr learned my addr 172.21.15.86:0/2834385624 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:43.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.601+0000 7fa05533f640 1 --2- 172.21.15.86:0/2834385624 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654b827fe80 0x5654b8285700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.601+0000 7fa056341640 1 --2- 172.21.15.86:0/2834385624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654b8288a00 0x5654b8285d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.601+0000 7fa055b40640 1 -- 172.21.15.86:0/2834385624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654b8288a00 msgr2=0x5654b8285d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.601+0000 7fa055b40640 1 --2- 172.21.15.86:0/2834385624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654b8288a00 0x5654b8285d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.601+0000 7fa055b40640 1 -- 172.21.15.86:0/2834385624 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654b827fe80 msgr2=0x5654b8285700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.601+0000 7fa055b40640 1 --2- 172.21.15.86:0/2834385624 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654b827fe80 0x5654b8285700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.601+0000 7fa055b40640 1 -- 172.21.15.86:0/2834385624 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654b7ef63f0 con 0x5654b827f4d0 2024-04-21T08:43:43.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.601+0000 7fa056341640 1 --2- 172.21.15.86:0/2834385624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654b8288a00 0x5654b8285d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.601+0000 7fa055b40640 1 --2- 172.21.15.86:0/2834385624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b827f4d0 0x5654b8288370 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fa040009900 tx=0x7fa04002ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=edf9725eefa572b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.601+0000 7fa0397fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.602+0000 7fa0397fa640 1 -- 172.21.15.86:0/2834385624 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa040005450 con 0x5654b827f4d0 2024-04-21T08:43:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.602+0000 7fa0397fa640 1 -- 172.21.15.86:0/2834385624 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fa0400046c0 con 0x5654b827f4d0 2024-04-21T08:43:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.602+0000 7fa05a997740 1 -- 172.21.15.86:0/2834385624 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa048018da0 con 0x5654b827f4d0 2024-04-21T08:43:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.602+0000 7fa0397fa640 1 -- 172.21.15.86:0/2834385624 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa0400055f0 con 0x5654b827f4d0 2024-04-21T08:43:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.602+0000 7fa05a997740 10 client.17766 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.602+0000 7fa05a997740 1 -- 172.21.15.86:0/2834385624 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5654b7fc07d0 con 0x5654b827f4d0 2024-04-21T08:43:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.602+0000 7fa0397fa640 1 -- 172.21.15.86:0/2834385624 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa048018da0 con 0x5654b827f4d0 2024-04-21T08:43:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.602+0000 7fa0397fa640 1 -- 172.21.15.86:0/2834385624 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa040003680 con 0x5654b827f4d0 2024-04-21T08:43:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.602+0000 7fa05a997740 10 client.17766 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.602+0000 7fa05a997740 10 client.17766 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:43:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.602+0000 7fa05a997740 20 client.17766 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.602+0000 7fa05a997740 1 --2- 172.21.15.86:0/2834385624 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x5654b8298c50 0x5654b829b0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.602+0000 7fa05a997740 4 client.17766 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.602+0000 7fa05a997740 1 -- 172.21.15.86:0/2834385624 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5654b8183930 con 0x5654b8298c50 2024-04-21T08:43:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.602+0000 7fa05533f640 1 --2- 172.21.15.86:0/2834385624 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x5654b8298c50 0x5654b829b0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.603+0000 7fa0397fa640 10 client.17766 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:43:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.603+0000 7fa05533f640 1 --2- 172.21.15.86:0/2834385624 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x5654b8298c50 0x5654b829b0c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=9d8d277b4604095a server_cookie=90181b76fd0834bf in_seq=0 out_seq=0 2024-04-21T08:43:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.603+0000 7fa0397fa640 1 -- 172.21.15.86:0/2834385624 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5654b8183930 con 0x5654b8298c50 2024-04-21T08:43:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.603+0000 7fa0397fa640 10 client.17766 handle_command_reply: tid=0 2024-04-21T08:43:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.603+0000 7fa05a997740 1 client.17766 shutdown 2024-04-21T08:43:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.604+0000 7fa05a997740 1 -- 172.21.15.86:0/2834385624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b827f4d0 msgr2=0x5654b8288370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.604+0000 7fa05a997740 1 --2- 172.21.15.86:0/2834385624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b827f4d0 0x5654b8288370 secure :-1 s=READY pgs=1198 cs=0 l=1 rev1=1 crypto rx=0x7fa040009900 tx=0x7fa04002ca90 comp rx=0 tx=0).stop 2024-04-21T08:43:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.604+0000 7fa05a997740 1 -- 172.21.15.86:0/2834385624 shutdown_connections 2024-04-21T08:43:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.604+0000 7fa05a997740 1 --2- 172.21.15.86:0/2834385624 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654b8288a00 0x5654b8285d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.604+0000 7fa05a997740 1 --2- 172.21.15.86:0/2834385624 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x5654b8298c50 0x5654b829b0c0 crc :-1 s=READY pgs=50 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.604+0000 7fa05a997740 1 --2- 172.21.15.86:0/2834385624 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654b827fe80 0x5654b8285700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.604+0000 7fa05a997740 1 --2- 172.21.15.86:0/2834385624 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b827f4d0 0x5654b8288370 unknown :-1 s=CLOSED pgs=1198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.604+0000 7fa0397fa640 0 client.17766 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:43:43.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.604+0000 7fa05a997740 1 -- 172.21.15.86:0/2834385624 >> 172.21.15.86:0/2834385624 conn(0x5654b8279ba0 msgr2=0x5654b827b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:43.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.604+0000 7fa05a997740 1 -- 172.21.15.86:0/2834385624 shutdown_connections 2024-04-21T08:43:43.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.604+0000 7fa05a997740 1 -- 172.21.15.86:0/2834385624 wait complete. 2024-04-21T08:43:43.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.604+0000 7fa05a997740 20 client.17766 trim_cache size 0 max 16384 2024-04-21T08:43:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.617+0000 7fa05a997740 1 Processor -- start 2024-04-21T08:43:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.617+0000 7fa05a997740 1 -- start start 2024-04-21T08:43:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.617+0000 7fa05a997740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654b827f4d0 0x5654b8294830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.617+0000 7fa05a997740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b827fe80 0x5654b8287e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.617+0000 7fa05a997740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654b8288a00 0x5654b8288370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.617+0000 7fa05a997740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0501ad080 con 0x5654b827fe80 2024-04-21T08:43:43.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.617+0000 7fa05a997740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa0501aebc0 con 0x5654b827f4d0 2024-04-21T08:43:43.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.617+0000 7fa05a997740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa0501ad750 con 0x5654b8288a00 2024-04-21T08:43:43.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.617+0000 7fa05533f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b827fe80 0x5654b8287e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.617+0000 7fa05533f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b827fe80 0x5654b8287e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37528/0 (socket says 172.21.15.86:37528) 2024-04-21T08:43:43.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.617+0000 7fa05533f640 1 -- 172.21.15.86:0/196359642 learned_addr learned my addr 172.21.15.86:0/196359642 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:43.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.617+0000 7fa055b40640 1 --2- 172.21.15.86:0/196359642 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654b827f4d0 0x5654b8294830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.617+0000 7fa056341640 1 --2- 172.21.15.86:0/196359642 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654b8288a00 0x5654b8288370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.617+0000 7fa05533f640 1 -- 172.21.15.86:0/196359642 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654b8288a00 msgr2=0x5654b8288370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.617+0000 7fa05533f640 1 --2- 172.21.15.86:0/196359642 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654b8288a00 0x5654b8288370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.617+0000 7fa05533f640 1 -- 172.21.15.86:0/196359642 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654b827f4d0 msgr2=0x5654b8294830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.617+0000 7fa05533f640 1 --2- 172.21.15.86:0/196359642 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654b827f4d0 0x5654b8294830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.617+0000 7fa05533f640 1 -- 172.21.15.86:0/196359642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5654b827fc20 con 0x5654b827fe80 2024-04-21T08:43:43.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.617+0000 7fa055b40640 1 --2- 172.21.15.86:0/196359642 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654b827f4d0 0x5654b8294830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:43.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.617+0000 7fa05533f640 1 --2- 172.21.15.86:0/196359642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b827fe80 0x5654b8287e30 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fa04c0985b0 tx=0x7fa04c0d29e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3d6251c28e432da2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:43.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.618+0000 7fa03b7fe640 1 -- 172.21.15.86:0/196359642 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04c020040 con 0x5654b827fe80 2024-04-21T08:43:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.618+0000 7fa03b7fe640 1 -- 172.21.15.86:0/196359642 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa04c0a4dc0 con 0x5654b827fe80 2024-04-21T08:43:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.618+0000 7fa03b7fe640 1 -- 172.21.15.86:0/196359642 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04c0a0150 con 0x5654b827fe80 2024-04-21T08:43:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.618+0000 7fa05a997740 1 -- 172.21.15.86:0/196359642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b827fe80 msgr2=0x5654b8287e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.618+0000 7fa05a997740 1 --2- 172.21.15.86:0/196359642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b827fe80 0x5654b8287e30 secure :-1 s=READY pgs=1199 cs=0 l=1 rev1=1 crypto rx=0x7fa04c0985b0 tx=0x7fa04c0d29e0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.618+0000 7fa05a997740 1 -- 172.21.15.86:0/196359642 shutdown_connections 2024-04-21T08:43:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.618+0000 7fa05a997740 1 --2- 172.21.15.86:0/196359642 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654b8288a00 0x5654b8288370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.618+0000 7fa05a997740 1 --2- 172.21.15.86:0/196359642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b827fe80 0x5654b8287e30 unknown :-1 s=CLOSED pgs=1199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.618+0000 7fa05a997740 1 --2- 172.21.15.86:0/196359642 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654b827f4d0 0x5654b8294830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.618+0000 7fa05a997740 1 -- 172.21.15.86:0/196359642 >> 172.21.15.86:0/196359642 conn(0x5654b8279ba0 msgr2=0x5654b8278c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.618+0000 7fa05a997740 1 -- 172.21.15.86:0/196359642 shutdown_connections 2024-04-21T08:43:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.618+0000 7fa05a997740 1 -- 172.21.15.86:0/196359642 wait complete. 2024-04-21T08:43:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.619+0000 7fa05a997740 1 -- start start 2024-04-21T08:43:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.619+0000 7fa05a997740 1 Processor -- start 2024-04-21T08:43:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.619+0000 7fa05a997740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654b827f4d0 0x5654b828cf90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.619+0000 7fa05a997740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b827fe80 0x5654b828d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.619+0000 7fa05a997740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654b8288a00 0x5654b82fe200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.619+0000 7fa05a997740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0501ae4f0 con 0x5654b827fe80 2024-04-21T08:43:43.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.619+0000 7fa05a997740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa0501ade20 con 0x5654b8288a00 2024-04-21T08:43:43.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.619+0000 7fa05a997740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa0501ab040 con 0x5654b827f4d0 2024-04-21T08:43:43.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.619+0000 7fa05a997740 10 client.0 _ll_register_callbacks cb 0x7fa040004490 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:43.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.619+0000 7fa05533f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b827fe80 0x5654b828d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.619+0000 7fa05533f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b827fe80 0x5654b828d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37532/0 (socket says 172.21.15.86:37532) 2024-04-21T08:43:43.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.619+0000 7fa05533f640 1 -- 172.21.15.86:0/3825709422 learned_addr learned my addr 172.21.15.86:0/3825709422 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:43.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.619+0000 7fa055b40640 1 --2- 172.21.15.86:0/3825709422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654b827f4d0 0x5654b828cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.619+0000 7fa056341640 1 --2- 172.21.15.86:0/3825709422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654b8288a00 0x5654b82fe200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.619+0000 7fa05533f640 1 -- 172.21.15.86:0/3825709422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654b827f4d0 msgr2=0x5654b828cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.619+0000 7fa05533f640 1 --2- 172.21.15.86:0/3825709422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654b827f4d0 0x5654b828cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.619+0000 7fa05533f640 1 -- 172.21.15.86:0/3825709422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654b8288a00 msgr2=0x5654b82fe200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.619+0000 7fa05533f640 1 --2- 172.21.15.86:0/3825709422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654b8288a00 0x5654b82fe200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.619+0000 7fa05533f640 1 -- 172.21.15.86:0/3825709422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5654b82fe7f0 con 0x5654b827fe80 2024-04-21T08:43:43.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.620+0000 7fa05533f640 1 --2- 172.21.15.86:0/3825709422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b827fe80 0x5654b828d4d0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fa04c0047e0 tx=0x7fa04c0097b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=167994893627a4f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:43.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.620+0000 7fa0397fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:43.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.620+0000 7fa0397fa640 1 -- 172.21.15.86:0/3825709422 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa04c0b4020 con 0x5654b827fe80 2024-04-21T08:43:43.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.620+0000 7fa0397fa640 1 -- 172.21.15.86:0/3825709422 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fa04c0a0150 con 0x5654b827fe80 2024-04-21T08:43:43.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.620+0000 7fa05a997740 1 -- 172.21.15.86:0/3825709422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa04c0a4dc0 con 0x5654b827fe80 2024-04-21T08:43:43.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.620+0000 7fa0397fa640 1 -- 172.21.15.86:0/3825709422 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa04c020040 con 0x5654b827fe80 2024-04-21T08:43:43.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.620+0000 7fa05a997740 10 client.17772 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:43.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.620+0000 7fa05a997740 1 -- 172.21.15.86:0/3825709422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5654b82ffad0 con 0x5654b827fe80 2024-04-21T08:43:43.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.620+0000 7fa0397fa640 1 -- 172.21.15.86:0/3825709422 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa04c0a4dc0 con 0x5654b827fe80 2024-04-21T08:43:43.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.620+0000 7fa0397fa640 1 -- 172.21.15.86:0/3825709422 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa04c097f30 con 0x5654b827fe80 2024-04-21T08:43:43.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.620+0000 7fa05a997740 10 client.17772 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:43.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.620+0000 7fa05a997740 10 client.17772 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:43:43.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.620+0000 7fa05a997740 20 client.17772 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:43.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.620+0000 7fa05a997740 1 --2- 172.21.15.86:0/3825709422 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x5654b8300190 0x5654b83025b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.620+0000 7fa05a997740 4 client.17772 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:43:43.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.620+0000 7fa05a997740 1 -- 172.21.15.86:0/3825709422 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5654b81f1c10 con 0x5654b8300190 2024-04-21T08:43:43.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.620+0000 7fa055b40640 1 --2- 172.21.15.86:0/3825709422 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x5654b8300190 0x5654b83025b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.621+0000 7fa055b40640 1 --2- 172.21.15.86:0/3825709422 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x5654b8300190 0x5654b83025b0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1af8f3ed30c96879 server_cookie=5857f27c8a832eb4 in_seq=0 out_seq=0 2024-04-21T08:43:43.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.621+0000 7fa0397fa640 10 client.17772 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:43:43.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.621+0000 7fa0397fa640 1 -- 172.21.15.86:0/3825709422 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5654b81f1c10 con 0x5654b8300190 2024-04-21T08:43:43.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.621+0000 7fa0397fa640 10 client.17772 handle_command_reply: tid=0 2024-04-21T08:43:43.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.621+0000 7fa05a997740 1 client.17772 shutdown 2024-04-21T08:43:43.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.621+0000 7fa05a997740 1 -- 172.21.15.86:0/3825709422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b827fe80 msgr2=0x5654b828d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.621+0000 7fa05a997740 1 --2- 172.21.15.86:0/3825709422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b827fe80 0x5654b828d4d0 secure :-1 s=READY pgs=1200 cs=0 l=1 rev1=1 crypto rx=0x7fa04c0047e0 tx=0x7fa04c0097b0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.621+0000 7fa05a997740 1 -- 172.21.15.86:0/3825709422 shutdown_connections 2024-04-21T08:43:43.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.621+0000 7fa05a997740 1 --2- 172.21.15.86:0/3825709422 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x5654b8300190 0x5654b83025b0 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.621+0000 7fa05a997740 1 --2- 172.21.15.86:0/3825709422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5654b8288a00 0x5654b82fe200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.621+0000 7fa05a997740 1 --2- 172.21.15.86:0/3825709422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5654b827fe80 0x5654b828d4d0 unknown :-1 s=CLOSED pgs=1200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.621+0000 7fa05a997740 1 --2- 172.21.15.86:0/3825709422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5654b827f4d0 0x5654b828cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.621+0000 7fa0397fa640 0 client.17772 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:43:43.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.621+0000 7fa05a997740 1 -- 172.21.15.86:0/3825709422 >> 172.21.15.86:0/3825709422 conn(0x5654b8279ba0 msgr2=0x5654b827af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:43.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.621+0000 7fa05a997740 1 -- 172.21.15.86:0/3825709422 shutdown_connections 2024-04-21T08:43:43.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.621+0000 7fa05a997740 1 -- 172.21.15.86:0/3825709422 wait complete. 2024-04-21T08:43:43.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.621+0000 7fa05a997740 20 client.17772 trim_cache size 0 max 16384 2024-04-21T08:43:43.632 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:43:43.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.622+0000 7fa057da4640 1 -- 172.21.15.86:0/2198289870 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa03c0a3d20 msgr2=0x7fa03c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.622+0000 7fa057da4640 1 --2- 172.21.15.86:0/2198289870 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa03c0a3d20 0x7fa03c0a61e0 secure :-1 s=READY pgs=187 cs=0 l=1 rev1=1 crypto rx=0x7fa048026000 tx=0x7fa048022030 comp rx=0 tx=0).stop 2024-04-21T08:43:43.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.622+0000 7fa057da4640 1 -- 172.21.15.86:0/2198289870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa050106160 msgr2=0x7fa05019e920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.622+0000 7fa057da4640 1 --2- 172.21.15.86:0/2198289870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa050106160 0x7fa05019e920 secure :-1 s=READY pgs=1196 cs=0 l=1 rev1=1 crypto rx=0x7fa04c00d740 tx=0x7fa04c00dc00 comp rx=0 tx=0).stop 2024-04-21T08:43:43.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.622+0000 7fa057da4640 1 -- 172.21.15.86:0/2198289870 shutdown_connections 2024-04-21T08:43:43.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.622+0000 7fa057da4640 1 --2- 172.21.15.86:0/2198289870 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa03c0a3d20 0x7fa03c0a61e0 unknown :-1 s=CLOSED pgs=187 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.622+0000 7fa057da4640 1 --2- 172.21.15.86:0/2198289870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa050198590 0x7fa050198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.622+0000 7fa057da4640 1 --2- 172.21.15.86:0/2198289870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa050106160 0x7fa05019e920 unknown :-1 s=CLOSED pgs=1196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.622+0000 7fa057da4640 1 --2- 172.21.15.86:0/2198289870 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa050101090 0x7fa05019e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.622+0000 7fa057da4640 1 -- 172.21.15.86:0/2198289870 >> 172.21.15.86:0/2198289870 conn(0x7fa050077f50 msgr2=0x7fa0500ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:43.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.622+0000 7fa057da4640 1 -- 172.21.15.86:0/2198289870 shutdown_connections 2024-04-21T08:43:43.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.622+0000 7fa057da4640 1 -- 172.21.15.86:0/2198289870 wait complete. 2024-04-21T08:43:43.636 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:43:43.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.736+0000 7f10181c7640 1 -- 172.21.15.86:0/2657948447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1010101a40 msgr2=0x7f1010105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.736+0000 7f10181c7640 1 --2- 172.21.15.86:0/2657948447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1010101a40 0x7f1010105a80 secure :-1 s=READY pgs=1201 cs=0 l=1 rev1=1 crypto rx=0x7f1000009900 tx=0x7f100002bc30 comp rx=0 tx=0).stop 2024-04-21T08:43:43.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.736+0000 7f10181c7640 1 -- 172.21.15.86:0/2657948447 shutdown_connections 2024-04-21T08:43:43.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.736+0000 7f10181c7640 1 --2- 172.21.15.86:0/2657948447 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1010106160 0x7f101010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.736+0000 7f10181c7640 1 --2- 172.21.15.86:0/2657948447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1010101a40 0x7f1010105a80 unknown :-1 s=CLOSED pgs=1201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.736+0000 7f10181c7640 1 --2- 172.21.15.86:0/2657948447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1010101090 0x7f1010101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.736+0000 7f10181c7640 1 -- 172.21.15.86:0/2657948447 >> 172.21.15.86:0/2657948447 conn(0x7f1010077f50 msgr2=0x7f1010078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:43.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.736+0000 7f10181c7640 1 -- 172.21.15.86:0/2657948447 shutdown_connections 2024-04-21T08:43:43.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.736+0000 7f10181c7640 1 -- 172.21.15.86:0/2657948447 wait complete. 2024-04-21T08:43:43.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.736+0000 7f10181c7640 1 Processor -- start 2024-04-21T08:43:43.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.736+0000 7f10181c7640 1 -- start start 2024-04-21T08:43:43.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.736+0000 7f10181c7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1010101090 0x7f1010197370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.736+0000 7f10181c7640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1010106160 0x7f10101978b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.736+0000 7f10181c7640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1010197df0 0x7f10101a69f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.736+0000 7f10181c7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1010111080 con 0x7f1010101090 2024-04-21T08:43:43.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.736+0000 7f10181c7640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1010110f00 con 0x7f1010197df0 2024-04-21T08:43:43.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.736+0000 7f10181c7640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1010111200 con 0x7f1010106160 2024-04-21T08:43:43.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.737+0000 7f1015f63640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1010101090 0x7f1010197370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.737+0000 7f1015f63640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1010101090 0x7f1010197370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37552/0 (socket says 172.21.15.86:37552) 2024-04-21T08:43:43.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.737+0000 7f1015f63640 1 -- 172.21.15.86:0/1349437115 learned_addr learned my addr 172.21.15.86:0/1349437115 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:43.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.737+0000 7f1015762640 1 --2- 172.21.15.86:0/1349437115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1010106160 0x7f10101978b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.737+0000 7f1016764640 1 --2- 172.21.15.86:0/1349437115 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1010197df0 0x7f10101a69f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.737+0000 7f1015f63640 1 -- 172.21.15.86:0/1349437115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1010106160 msgr2=0x7f10101978b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.737+0000 7f1015f63640 1 --2- 172.21.15.86:0/1349437115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1010106160 0x7f10101978b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.737+0000 7f1015f63640 1 -- 172.21.15.86:0/1349437115 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1010197df0 msgr2=0x7f10101a69f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.737+0000 7f1015f63640 1 --2- 172.21.15.86:0/1349437115 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1010197df0 0x7f10101a69f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.737+0000 7f1015f63640 1 -- 172.21.15.86:0/1349437115 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10101a6fb0 con 0x7f1010101090 2024-04-21T08:43:43.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.737+0000 7f1016764640 1 --2- 172.21.15.86:0/1349437115 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1010197df0 0x7f10101a69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:43.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.737+0000 7f1015f63640 1 --2- 172.21.15.86:0/1349437115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1010101090 0x7f1010197370 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f100c00d980 tx=0x7f100c00de40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:43.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.737+0000 7f1006ffd640 1 -- 172.21.15.86:0/1349437115 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100c017030 con 0x7f1010101090 2024-04-21T08:43:43.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.737+0000 7f1006ffd640 1 -- 172.21.15.86:0/1349437115 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f100c0099d0 con 0x7f1010101090 2024-04-21T08:43:43.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.737+0000 7f10181c7640 1 -- 172.21.15.86:0/1349437115 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10101a72a0 con 0x7f1010101090 2024-04-21T08:43:43.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.737+0000 7f1006ffd640 1 -- 172.21.15.86:0/1349437115 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100c0055c0 con 0x7f1010101090 2024-04-21T08:43:43.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.737+0000 7f10181c7640 1 -- 172.21.15.86:0/1349437115 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10101a7810 con 0x7f1010101090 2024-04-21T08:43:43.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.739+0000 7f1006ffd640 1 -- 172.21.15.86:0/1349437115 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f100c004020 con 0x7f1010101090 2024-04-21T08:43:43.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.740+0000 7f101adba740 1 Processor -- start 2024-04-21T08:43:43.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.740+0000 7f101adba740 1 -- start start 2024-04-21T08:43:43.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.740+0000 7f101adba740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c03d73e4d0 0x55c03d73e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.740+0000 7f101adba740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c03d73ee80 0x55c03d747430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.740+0000 7f101adba740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c03d747a00 0x55c03d749e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.740+0000 7f1006ffd640 1 --2- 172.21.15.86:0/1349437115 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0ffc0a3da0 0x7f0ffc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.740+0000 7f101adba740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c03d6a31f0 con 0x55c03d747a00 2024-04-21T08:43:43.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.740+0000 7f101adba740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c03d6404b0 con 0x55c03d73ee80 2024-04-21T08:43:43.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.740+0000 7f101adba740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c03d69f1b0 con 0x55c03d73e4d0 2024-04-21T08:43:43.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.740+0000 7f1015762640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c03d747a00 0x55c03d749e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.740+0000 7f1015762640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c03d747a00 0x55c03d749e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37556/0 (socket says 172.21.15.86:37556) 2024-04-21T08:43:43.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.740+0000 7f1016764640 1 --2- 172.21.15.86:0/1349437115 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0ffc0a3da0 0x7f0ffc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.740+0000 7f1015762640 1 -- 172.21.15.86:0/3014251545 learned_addr learned my addr 172.21.15.86:0/3014251545 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:43.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.740+0000 7f1015f63640 1 --2- 172.21.15.86:0/3014251545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c03d73ee80 0x55c03d747430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.740+0000 7f1006ffd640 1 -- 172.21.15.86:0/1349437115 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f100c0d0470 con 0x7f1010101090 2024-04-21T08:43:43.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.740+0000 7f1016764640 1 --2- 172.21.15.86:0/3014251545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c03d73e4d0 0x55c03d73e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.740+0000 7f1015762640 1 -- 172.21.15.86:0/3014251545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c03d73e4d0 msgr2=0x55c03d73e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.740+0000 7f1015762640 1 --2- 172.21.15.86:0/3014251545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c03d73e4d0 0x55c03d73e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.740+0000 7f1015762640 1 -- 172.21.15.86:0/3014251545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c03d73ee80 msgr2=0x55c03d747430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.740+0000 7f1015762640 1 --2- 172.21.15.86:0/3014251545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c03d73ee80 0x55c03d747430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.740+0000 7f1015762640 1 -- 172.21.15.86:0/3014251545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c03d6321b0 con 0x55c03d747a00 2024-04-21T08:43:43.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.741+0000 7f1016764640 1 --2- 172.21.15.86:0/1349437115 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0ffc0a3da0 0x7f0ffc0a6260 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1008005c40 tx=0x7f10080094a0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:43.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.741+0000 7f1015762640 1 --2- 172.21.15.86:0/3014251545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c03d747a00 0x55c03d749e50 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f1000009900 tx=0x7f100002c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb47c679bd53d27 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:43.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.741+0000 7f0ffbfff640 1 -- 172.21.15.86:0/3014251545 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1000045020 con 0x55c03d747a00 2024-04-21T08:43:43.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.741+0000 7f0ffbfff640 1 -- 172.21.15.86:0/3014251545 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f100002c7e0 con 0x55c03d747a00 2024-04-21T08:43:43.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.741+0000 7f0ffbfff640 1 -- 172.21.15.86:0/3014251545 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10000046c0 con 0x55c03d747a00 2024-04-21T08:43:43.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.742+0000 7f101adba740 1 -- 172.21.15.86:0/3014251545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c03d747a00 msgr2=0x55c03d749e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.742+0000 7f101adba740 1 --2- 172.21.15.86:0/3014251545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c03d747a00 0x55c03d749e50 secure :-1 s=READY pgs=1203 cs=0 l=1 rev1=1 crypto rx=0x7f1000009900 tx=0x7f100002c610 comp rx=0 tx=0).stop 2024-04-21T08:43:43.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.742+0000 7f101adba740 1 -- 172.21.15.86:0/3014251545 shutdown_connections 2024-04-21T08:43:43.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.742+0000 7f101adba740 1 --2- 172.21.15.86:0/3014251545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c03d747a00 0x55c03d749e50 unknown :-1 s=CLOSED pgs=1203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.742+0000 7f101adba740 1 --2- 172.21.15.86:0/3014251545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c03d73ee80 0x55c03d747430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.742+0000 7f101adba740 1 --2- 172.21.15.86:0/3014251545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c03d73e4d0 0x55c03d73e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.742+0000 7f101adba740 1 -- 172.21.15.86:0/3014251545 >> 172.21.15.86:0/3014251545 conn(0x55c03d738ba0 msgr2=0x55c03d738f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:43.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.742+0000 7f101adba740 1 -- 172.21.15.86:0/3014251545 shutdown_connections 2024-04-21T08:43:43.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.742+0000 7f101adba740 1 -- 172.21.15.86:0/3014251545 wait complete. 2024-04-21T08:43:43.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.742+0000 7f101adba740 1 -- start start 2024-04-21T08:43:43.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.742+0000 7f101adba740 1 Processor -- start 2024-04-21T08:43:43.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.742+0000 7f101adba740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c03d73e4d0 0x55c03d7472f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.742+0000 7f101adba740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c03d73ee80 0x55c03d744680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.742+0000 7f101adba740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c03d747a00 0x55c03d744c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.742+0000 7f101adba740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c03d74a910 con 0x55c03d73e4d0 2024-04-21T08:43:43.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.742+0000 7f101adba740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c03d65c500 con 0x55c03d73ee80 2024-04-21T08:43:43.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.742+0000 7f101adba740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c03d74aa90 con 0x55c03d747a00 2024-04-21T08:43:43.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.742+0000 7f101adba740 10 client.0 _ll_register_callbacks cb 0x55c03d62ab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:43.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.743+0000 7f1015f63640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c03d73e4d0 0x55c03d7472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.743+0000 7f1015f63640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c03d73e4d0 0x55c03d7472f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37570/0 (socket says 172.21.15.86:37570) 2024-04-21T08:43:43.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.743+0000 7f1015f63640 1 -- 172.21.15.86:0/554323919 learned_addr learned my addr 172.21.15.86:0/554323919 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:43.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.743+0000 7f1015762640 1 --2- 172.21.15.86:0/554323919 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c03d73ee80 0x55c03d744680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.743+0000 7f1016764640 1 --2- 172.21.15.86:0/554323919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c03d747a00 0x55c03d744c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.743+0000 7f1015f63640 1 -- 172.21.15.86:0/554323919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c03d747a00 msgr2=0x55c03d744c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.743+0000 7f1015f63640 1 --2- 172.21.15.86:0/554323919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c03d747a00 0x55c03d744c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.743+0000 7f1015f63640 1 -- 172.21.15.86:0/554323919 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c03d73ee80 msgr2=0x55c03d744680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.743+0000 7f1015f63640 1 --2- 172.21.15.86:0/554323919 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c03d73ee80 0x55c03d744680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.743+0000 7f1015f63640 1 -- 172.21.15.86:0/554323919 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c03d3b53f0 con 0x55c03d73e4d0 2024-04-21T08:43:43.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.743+0000 7f1016764640 1 --2- 172.21.15.86:0/554323919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c03d747a00 0x55c03d744c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:43.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.743+0000 7f1015762640 1 --2- 172.21.15.86:0/554323919 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c03d73ee80 0x55c03d744680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:43:43.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.743+0000 7f1015f63640 1 --2- 172.21.15.86:0/554323919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c03d73e4d0 0x55c03d7472f0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f100c00d950 tx=0x7f100c021c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a9400d6cb4c5bd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:43.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.743+0000 7f0ff97fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:43.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.743+0000 7f0ff97fa640 1 -- 172.21.15.86:0/554323919 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f100c0b0020 con 0x55c03d73e4d0 2024-04-21T08:43:43.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.743+0000 7f0ff97fa640 1 -- 172.21.15.86:0/554323919 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f100c004310 con 0x55c03d73e4d0 2024-04-21T08:43:43.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.743+0000 7f101adba740 1 -- 172.21.15.86:0/554323919 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f100002c7e0 con 0x55c03d73e4d0 2024-04-21T08:43:43.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.743+0000 7f0ff97fa640 1 -- 172.21.15.86:0/554323919 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f100c09c7a0 con 0x55c03d73e4d0 2024-04-21T08:43:43.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.743+0000 7f101adba740 10 client.17784 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:43.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.743+0000 7f101adba740 1 -- 172.21.15.86:0/554323919 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c03d47f7d0 con 0x55c03d73e4d0 2024-04-21T08:43:43.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.743+0000 7f0ff97fa640 1 -- 172.21.15.86:0/554323919 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f100002c7e0 con 0x55c03d73e4d0 2024-04-21T08:43:43.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.744+0000 7f0ff97fa640 1 -- 172.21.15.86:0/554323919 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f100c021c80 con 0x55c03d73e4d0 2024-04-21T08:43:43.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.744+0000 7f101adba740 10 client.17784 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:43.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.744+0000 7f101adba740 10 client.17784 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:43:43.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.744+0000 7f101adba740 20 client.17784 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:43.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.744+0000 7f101adba740 1 --2- 172.21.15.86:0/554323919 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55c03d757b90 0x55c03d75a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.744+0000 7f101adba740 4 client.17784 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:43.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.744+0000 7f101adba740 1 -- 172.21.15.86:0/554323919 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c03d642930 con 0x55c03d757b90 2024-04-21T08:43:43.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.744+0000 7f1015762640 1 --2- 172.21.15.86:0/554323919 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55c03d757b90 0x55c03d75a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.744+0000 7f0ff97fa640 10 client.17784 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:43:43.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.744+0000 7f1015762640 1 --2- 172.21.15.86:0/554323919 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55c03d757b90 0x55c03d75a000 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b276aacccfb70a82 server_cookie=5c3a5a9b053ce701 in_seq=0 out_seq=0 2024-04-21T08:43:43.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.745+0000 7f0ff97fa640 1 -- 172.21.15.86:0/554323919 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c03d642930 con 0x55c03d757b90 2024-04-21T08:43:43.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.745+0000 7f0ff97fa640 10 client.17784 handle_command_reply: tid=0 2024-04-21T08:43:43.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.745+0000 7f101adba740 1 client.17784 shutdown 2024-04-21T08:43:43.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.745+0000 7f101adba740 1 -- 172.21.15.86:0/554323919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c03d73e4d0 msgr2=0x55c03d7472f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.745+0000 7f101adba740 1 --2- 172.21.15.86:0/554323919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c03d73e4d0 0x55c03d7472f0 secure :-1 s=READY pgs=1204 cs=0 l=1 rev1=1 crypto rx=0x7f100c00d950 tx=0x7f100c021c10 comp rx=0 tx=0).stop 2024-04-21T08:43:43.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.745+0000 7f101adba740 1 -- 172.21.15.86:0/554323919 shutdown_connections 2024-04-21T08:43:43.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.745+0000 7f101adba740 1 --2- 172.21.15.86:0/554323919 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55c03d757b90 0x55c03d75a000 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.745+0000 7f101adba740 1 --2- 172.21.15.86:0/554323919 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c03d747a00 0x55c03d744c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.745+0000 7f101adba740 1 --2- 172.21.15.86:0/554323919 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c03d73ee80 0x55c03d744680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.745+0000 7f101adba740 1 --2- 172.21.15.86:0/554323919 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c03d73e4d0 0x55c03d7472f0 unknown :-1 s=CLOSED pgs=1204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.745+0000 7f0ff97fa640 0 client.17784 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:43:43.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.745+0000 7f101adba740 1 -- 172.21.15.86:0/554323919 >> 172.21.15.86:0/554323919 conn(0x55c03d738ba0 msgr2=0x55c03d73a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:43.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.745+0000 7f101adba740 1 -- 172.21.15.86:0/554323919 shutdown_connections 2024-04-21T08:43:43.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.745+0000 7f101adba740 1 -- 172.21.15.86:0/554323919 wait complete. 2024-04-21T08:43:43.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.745+0000 7f101adba740 20 client.17784 trim_cache size 0 max 16384 2024-04-21T08:43:43.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.758+0000 7f101adba740 1 Processor -- start 2024-04-21T08:43:43.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.758+0000 7f101adba740 1 -- start start 2024-04-21T08:43:43.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.758+0000 7f101adba740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c03d73e4d0 0x55c03d7537d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.758+0000 7f101adba740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c03d73ee80 0x55c03d746db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.758+0000 7f101adba740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c03d747a00 0x55c03d7472f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.758+0000 7f101adba740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f10101b1450 con 0x55c03d73ee80 2024-04-21T08:43:43.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.758+0000 7f101adba740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f10101b2f90 con 0x55c03d73e4d0 2024-04-21T08:43:43.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.758+0000 7f101adba740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f10101b1b20 con 0x55c03d747a00 2024-04-21T08:43:43.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.759+0000 7f1015762640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c03d73ee80 0x55c03d746db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.759+0000 7f1015762640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c03d73ee80 0x55c03d746db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37578/0 (socket says 172.21.15.86:37578) 2024-04-21T08:43:43.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.759+0000 7f1015762640 1 -- 172.21.15.86:0/1322765332 learned_addr learned my addr 172.21.15.86:0/1322765332 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:43.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.759+0000 7f1015762640 1 -- 172.21.15.86:0/1322765332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c03d747a00 msgr2=0x55c03d7472f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.759+0000 7f1015762640 1 --2- 172.21.15.86:0/1322765332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c03d747a00 0x55c03d7472f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.759+0000 7f1015762640 1 -- 172.21.15.86:0/1322765332 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c03d73e4d0 msgr2=0x55c03d7537d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.759+0000 7f1015762640 1 --2- 172.21.15.86:0/1322765332 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c03d73e4d0 0x55c03d7537d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.759+0000 7f1015762640 1 -- 172.21.15.86:0/1322765332 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c03d742bd0 con 0x55c03d73ee80 2024-04-21T08:43:43.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.759+0000 7f1015762640 1 --2- 172.21.15.86:0/1322765332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c03d73ee80 0x55c03d746db0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f1000005310 tx=0x7f1000005340 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3e24f13d5e20f752 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:43.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.759+0000 7f0ffb7fe640 1 -- 172.21.15.86:0/1322765332 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1000045040 con 0x55c03d73ee80 2024-04-21T08:43:43.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.759+0000 7f0ffb7fe640 1 -- 172.21.15.86:0/1322765332 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f100002c7e0 con 0x55c03d73ee80 2024-04-21T08:43:43.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.759+0000 7f0ffb7fe640 1 -- 172.21.15.86:0/1322765332 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1000003d00 con 0x55c03d73ee80 2024-04-21T08:43:43.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.760+0000 7f101adba740 1 -- 172.21.15.86:0/1322765332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c03d73ee80 msgr2=0x55c03d746db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.760+0000 7f101adba740 1 --2- 172.21.15.86:0/1322765332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c03d73ee80 0x55c03d746db0 secure :-1 s=READY pgs=1205 cs=0 l=1 rev1=1 crypto rx=0x7f1000005310 tx=0x7f1000005340 comp rx=0 tx=0).stop 2024-04-21T08:43:43.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.760+0000 7f101adba740 1 -- 172.21.15.86:0/1322765332 shutdown_connections 2024-04-21T08:43:43.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.760+0000 7f101adba740 1 --2- 172.21.15.86:0/1322765332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c03d747a00 0x55c03d7472f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.760+0000 7f101adba740 1 --2- 172.21.15.86:0/1322765332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c03d73ee80 0x55c03d746db0 unknown :-1 s=CLOSED pgs=1205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.760+0000 7f101adba740 1 --2- 172.21.15.86:0/1322765332 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c03d73e4d0 0x55c03d7537d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.760+0000 7f101adba740 1 -- 172.21.15.86:0/1322765332 >> 172.21.15.86:0/1322765332 conn(0x55c03d738ba0 msgr2=0x55c03d737c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:43.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.760+0000 7f101adba740 1 -- 172.21.15.86:0/1322765332 shutdown_connections 2024-04-21T08:43:43.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.760+0000 7f101adba740 1 -- 172.21.15.86:0/1322765332 wait complete. 2024-04-21T08:43:43.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.760+0000 7f101adba740 1 -- start start 2024-04-21T08:43:43.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.760+0000 7f101adba740 1 Processor -- start 2024-04-21T08:43:43.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.760+0000 7f101adba740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c03d73e4d0 0x55c03d7413f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.760+0000 7f101adba740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c03d73ee80 0x55c03d741930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.760+0000 7f101adba740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c03d747a00 0x55c03d741e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.760+0000 7f101adba740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f10101b28c0 con 0x55c03d73ee80 2024-04-21T08:43:43.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.760+0000 7f101adba740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f10101b21f0 con 0x55c03d747a00 2024-04-21T08:43:43.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.760+0000 7f101adba740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f10101af410 con 0x55c03d73e4d0 2024-04-21T08:43:43.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.760+0000 7f101adba740 10 client.0 _ll_register_callbacks cb 0x7f100c09be70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:43.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.761+0000 7f1015762640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c03d73ee80 0x55c03d741930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.761+0000 7f1015762640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c03d73ee80 0x55c03d741930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37586/0 (socket says 172.21.15.86:37586) 2024-04-21T08:43:43.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.761+0000 7f1015762640 1 -- 172.21.15.86:0/2782721318 learned_addr learned my addr 172.21.15.86:0/2782721318 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:43.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.761+0000 7f1015f63640 1 --2- 172.21.15.86:0/2782721318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c03d73e4d0 0x55c03d7413f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.761+0000 7f1015762640 1 -- 172.21.15.86:0/2782721318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c03d73e4d0 msgr2=0x55c03d7413f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.761+0000 7f1015762640 1 --2- 172.21.15.86:0/2782721318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c03d73e4d0 0x55c03d7413f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.761+0000 7f1015762640 1 -- 172.21.15.86:0/2782721318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c03d747a00 msgr2=0x55c03d741e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.761+0000 7f1016764640 1 --2- 172.21.15.86:0/2782721318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c03d747a00 0x55c03d741e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.761+0000 7f1015762640 1 --2- 172.21.15.86:0/2782721318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c03d747a00 0x55c03d741e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.761+0000 7f1015762640 1 -- 172.21.15.86:0/2782721318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c03d7bd1f0 con 0x55c03d73ee80 2024-04-21T08:43:43.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.761+0000 7f1015f63640 1 --2- 172.21.15.86:0/2782721318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c03d73e4d0 0x55c03d7413f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:43.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.761+0000 7f1015762640 1 --2- 172.21.15.86:0/2782721318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c03d73ee80 0x55c03d741930 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f100002c170 tx=0x7f1000004540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab3b1385ba13b1eb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:43.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.761+0000 7f0ff97fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:43.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.761+0000 7f0ff97fa640 1 -- 172.21.15.86:0/2782721318 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1000045040 con 0x55c03d73ee80 2024-04-21T08:43:43.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.761+0000 7f0ff97fa640 1 -- 172.21.15.86:0/2782721318 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f10000037f0 con 0x55c03d73ee80 2024-04-21T08:43:43.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.761+0000 7f101adba740 1 -- 172.21.15.86:0/2782721318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f100002c7e0 con 0x55c03d73ee80 2024-04-21T08:43:43.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.761+0000 7f0ff97fa640 1 -- 172.21.15.86:0/2782721318 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1000064050 con 0x55c03d73ee80 2024-04-21T08:43:43.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.761+0000 7f101adba740 10 client.17790 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:43.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.761+0000 7f101adba740 1 -- 172.21.15.86:0/2782721318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c03d7be4d0 con 0x55c03d73ee80 2024-04-21T08:43:43.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.761+0000 7f0ff97fa640 1 -- 172.21.15.86:0/2782721318 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f100002c7e0 con 0x55c03d73ee80 2024-04-21T08:43:43.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.762+0000 7f0ff97fa640 1 -- 172.21.15.86:0/2782721318 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1000041ad0 con 0x55c03d73ee80 2024-04-21T08:43:43.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.762+0000 7f101adba740 10 client.17790 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:43.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.762+0000 7f101adba740 10 client.17790 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:43:43.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.762+0000 7f101adba740 20 client.17790 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:43.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.762+0000 7f101adba740 1 --2- 172.21.15.86:0/2782721318 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55c03d7be760 0x55c03d7c0b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.762+0000 7f101adba740 4 client.17790 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:43:43.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.762+0000 7f101adba740 1 -- 172.21.15.86:0/2782721318 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c03d6b0c10 con 0x55c03d7be760 2024-04-21T08:43:43.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.762+0000 7f1015f63640 1 --2- 172.21.15.86:0/2782721318 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55c03d7be760 0x55c03d7c0b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.762+0000 7f0ff97fa640 10 client.17790 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:43:43.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.762+0000 7f1015f63640 1 --2- 172.21.15.86:0/2782721318 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55c03d7be760 0x55c03d7c0b80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=6e88e612014f70d4 server_cookie=ce4366fd1d32a032 in_seq=0 out_seq=0 2024-04-21T08:43:43.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.763+0000 7f0ff97fa640 1 -- 172.21.15.86:0/2782721318 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c03d6b0c10 con 0x55c03d7be760 2024-04-21T08:43:43.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.763+0000 7f0ff97fa640 10 client.17790 handle_command_reply: tid=0 2024-04-21T08:43:43.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.763+0000 7f101adba740 1 client.17790 shutdown 2024-04-21T08:43:43.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.763+0000 7f101adba740 1 -- 172.21.15.86:0/2782721318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c03d73ee80 msgr2=0x55c03d741930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.763+0000 7f101adba740 1 --2- 172.21.15.86:0/2782721318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c03d73ee80 0x55c03d741930 secure :-1 s=READY pgs=1206 cs=0 l=1 rev1=1 crypto rx=0x7f100002c170 tx=0x7f1000004540 comp rx=0 tx=0).stop 2024-04-21T08:43:43.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.763+0000 7f101adba740 1 -- 172.21.15.86:0/2782721318 shutdown_connections 2024-04-21T08:43:43.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.763+0000 7f101adba740 1 --2- 172.21.15.86:0/2782721318 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55c03d7be760 0x55c03d7c0b80 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.763+0000 7f101adba740 1 --2- 172.21.15.86:0/2782721318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c03d747a00 0x55c03d741e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.763+0000 7f101adba740 1 --2- 172.21.15.86:0/2782721318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c03d73ee80 0x55c03d741930 unknown :-1 s=CLOSED pgs=1206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.763+0000 7f101adba740 1 --2- 172.21.15.86:0/2782721318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c03d73e4d0 0x55c03d7413f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.763+0000 7f0ff97fa640 0 client.17790 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:43:43.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.763+0000 7f101adba740 1 -- 172.21.15.86:0/2782721318 >> 172.21.15.86:0/2782721318 conn(0x55c03d738ba0 msgr2=0x55c03d73f380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:43.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.763+0000 7f101adba740 1 -- 172.21.15.86:0/2782721318 shutdown_connections 2024-04-21T08:43:43.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.763+0000 7f101adba740 1 -- 172.21.15.86:0/2782721318 wait complete. 2024-04-21T08:43:43.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.763+0000 7f101adba740 20 client.17790 trim_cache size 0 max 16384 2024-04-21T08:43:43.771 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:43:43.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.763+0000 7f10181c7640 1 -- 172.21.15.86:0/1349437115 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0ffc0a3da0 msgr2=0x7f0ffc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.763+0000 7f10181c7640 1 --2- 172.21.15.86:0/1349437115 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0ffc0a3da0 0x7f0ffc0a6260 secure :-1 s=READY pgs=188 cs=0 l=1 rev1=1 crypto rx=0x7f1008005c40 tx=0x7f10080094a0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.763+0000 7f10181c7640 1 -- 172.21.15.86:0/1349437115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1010101090 msgr2=0x7f1010197370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.763+0000 7f10181c7640 1 --2- 172.21.15.86:0/1349437115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1010101090 0x7f1010197370 secure :-1 s=READY pgs=1202 cs=0 l=1 rev1=1 crypto rx=0x7f100c00d980 tx=0x7f100c00de40 comp rx=0 tx=0).stop 2024-04-21T08:43:43.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.763+0000 7f10181c7640 1 -- 172.21.15.86:0/1349437115 shutdown_connections 2024-04-21T08:43:43.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.763+0000 7f10181c7640 1 --2- 172.21.15.86:0/1349437115 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0ffc0a3da0 0x7f0ffc0a6260 unknown :-1 s=CLOSED pgs=188 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.763+0000 7f10181c7640 1 --2- 172.21.15.86:0/1349437115 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1010197df0 0x7f10101a69f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.763+0000 7f10181c7640 1 --2- 172.21.15.86:0/1349437115 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1010106160 0x7f10101978b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.763+0000 7f10181c7640 1 --2- 172.21.15.86:0/1349437115 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1010101090 0x7f1010197370 unknown :-1 s=CLOSED pgs=1202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.763+0000 7f10181c7640 1 -- 172.21.15.86:0/1349437115 >> 172.21.15.86:0/1349437115 conn(0x7f1010077f50 msgr2=0x7f10100ff0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:43.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.763+0000 7f10181c7640 1 -- 172.21.15.86:0/1349437115 shutdown_connections 2024-04-21T08:43:43.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.763+0000 7f10181c7640 1 -- 172.21.15.86:0/1349437115 wait complete. 2024-04-21T08:43:43.773 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:43 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2351292169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:43:43.774 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:43:43.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.874+0000 7f579e59f640 1 -- 172.21.15.86:0/1548724726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f579810c3d0 msgr2=0x7f579810e870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.874+0000 7f579e59f640 1 --2- 172.21.15.86:0/1548724726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f579810c3d0 0x7f579810e870 secure :-1 s=READY pgs=1207 cs=0 l=1 rev1=1 crypto rx=0x7f5790009ca0 tx=0x7f579002c180 comp rx=0 tx=0).stop 2024-04-21T08:43:43.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.874+0000 7f579e59f640 1 -- 172.21.15.86:0/1548724726 shutdown_connections 2024-04-21T08:43:43.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.874+0000 7f579e59f640 1 --2- 172.21.15.86:0/1548724726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f579810c3d0 0x7f579810e870 unknown :-1 s=CLOSED pgs=1207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.874+0000 7f579e59f640 1 --2- 172.21.15.86:0/1548724726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57981038d0 0x7f579810bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.874+0000 7f579e59f640 1 --2- 172.21.15.86:0/1548724726 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5798102fb0 0x7f5798103390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.874+0000 7f579e59f640 1 -- 172.21.15.86:0/1548724726 >> 172.21.15.86:0/1548724726 conn(0x7f57980fc7b0 msgr2=0x7f57980febd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:43.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.874+0000 7f579e59f640 1 -- 172.21.15.86:0/1548724726 shutdown_connections 2024-04-21T08:43:43.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.874+0000 7f579e59f640 1 -- 172.21.15.86:0/1548724726 wait complete. 2024-04-21T08:43:43.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.875+0000 7f579e59f640 1 Processor -- start 2024-04-21T08:43:43.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.875+0000 7f579e59f640 1 -- start start 2024-04-21T08:43:43.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.875+0000 7f579e59f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5798102fb0 0x7f5798198520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.875+0000 7f579e59f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57981038d0 0x7f5798198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.875+0000 7f579e59f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f579810c3d0 0x7f579819cd90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.875+0000 7f579e59f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5798112040 con 0x7f5798102fb0 2024-04-21T08:43:43.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.875+0000 7f579e59f640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5798111ec0 con 0x7f57981038d0 2024-04-21T08:43:43.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.875+0000 7f579e59f640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f57981121c0 con 0x7f579810c3d0 2024-04-21T08:43:43.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.875+0000 7f579d59d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5798102fb0 0x7f5798198520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.875+0000 7f579d59d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5798102fb0 0x7f5798198520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37608/0 (socket says 172.21.15.86:37608) 2024-04-21T08:43:43.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.875+0000 7f579d59d640 1 -- 172.21.15.86:0/2821425408 learned_addr learned my addr 172.21.15.86:0/2821425408 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:43.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.875+0000 7f579cd9c640 1 --2- 172.21.15.86:0/2821425408 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57981038d0 0x7f5798198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.875+0000 7f579dd9e640 1 --2- 172.21.15.86:0/2821425408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f579810c3d0 0x7f579819cd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.875+0000 7f579d59d640 1 -- 172.21.15.86:0/2821425408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f579810c3d0 msgr2=0x7f579819cd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.875+0000 7f579d59d640 1 --2- 172.21.15.86:0/2821425408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f579810c3d0 0x7f579819cd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.875+0000 7f579d59d640 1 -- 172.21.15.86:0/2821425408 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57981038d0 msgr2=0x7f5798198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.875+0000 7f579d59d640 1 --2- 172.21.15.86:0/2821425408 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57981038d0 0x7f5798198a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.875+0000 7f579d59d640 1 -- 172.21.15.86:0/2821425408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f579819d4c0 con 0x7f5798102fb0 2024-04-21T08:43:43.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.876+0000 7f579d59d640 1 --2- 172.21.15.86:0/2821425408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5798102fb0 0x7f5798198520 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f579400d950 tx=0x7f579400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:43.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.876+0000 7f578e7fc640 1 -- 172.21.15.86:0/2821425408 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5794017070 con 0x7f5798102fb0 2024-04-21T08:43:43.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.876+0000 7f578e7fc640 1 -- 172.21.15.86:0/2821425408 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5794009c30 con 0x7f5798102fb0 2024-04-21T08:43:43.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.876+0000 7f579e59f640 1 -- 172.21.15.86:0/2821425408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f579819d7b0 con 0x7f5798102fb0 2024-04-21T08:43:43.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.876+0000 7f578e7fc640 1 -- 172.21.15.86:0/2821425408 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f579400f620 con 0x7f5798102fb0 2024-04-21T08:43:43.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.876+0000 7f579e59f640 1 -- 172.21.15.86:0/2821425408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57981a5040 con 0x7f5798102fb0 2024-04-21T08:43:43.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.878+0000 7f578e7fc640 1 -- 172.21.15.86:0/2821425408 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5794010460 con 0x7f5798102fb0 2024-04-21T08:43:43.888 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.878+0000 7f57a23f0740 1 Processor -- start 2024-04-21T08:43:43.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.878+0000 7f57a23f0740 1 -- start start 2024-04-21T08:43:43.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.878+0000 7f57a23f0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bdc0ab96d0 0x55bdc0ab9ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.879+0000 7f57a23f0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdc0aba080 0x55bdc0ac2630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.879+0000 7f57a23f0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdc0ac2c00 0x55bdc0ac5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.879+0000 7f57a23f0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bdc0a1e1f0 con 0x55bdc0aba080 2024-04-21T08:43:43.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.879+0000 7f57a23f0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55bdc09bb4b0 con 0x55bdc0ab96d0 2024-04-21T08:43:43.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.879+0000 7f57a23f0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55bdc0a1a1b0 con 0x55bdc0ac2c00 2024-04-21T08:43:43.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.879+0000 7f579d59d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdc0aba080 0x55bdc0ac2630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.879+0000 7f579d59d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdc0aba080 0x55bdc0ac2630 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37622/0 (socket says 172.21.15.86:37622) 2024-04-21T08:43:43.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.879+0000 7f579d59d640 1 -- 172.21.15.86:0/436605848 learned_addr learned my addr 172.21.15.86:0/436605848 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:43.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.879+0000 7f578e7fc640 1 --2- 172.21.15.86:0/2821425408 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f57840a3d20 0x7f57840a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.879+0000 7f579dd9e640 1 --2- 172.21.15.86:0/2821425408 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f57840a3d20 0x7f57840a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.889 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.879+0000 7f579cd9c640 1 --2- 172.21.15.86:0/436605848 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdc0ac2c00 0x55bdc0ac5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.879+0000 7f579dd9e640 1 --2- 172.21.15.86:0/436605848 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bdc0ab96d0 0x55bdc0ab9ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.879+0000 7f578e7fc640 1 -- 172.21.15.86:0/2821425408 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f57940d07d0 con 0x7f5798102fb0 2024-04-21T08:43:43.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.879+0000 7f579d59d640 1 -- 172.21.15.86:0/436605848 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdc0ac2c00 msgr2=0x55bdc0ac5050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.879+0000 7f579d59d640 1 --2- 172.21.15.86:0/436605848 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdc0ac2c00 0x55bdc0ac5050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.879+0000 7f579d59d640 1 -- 172.21.15.86:0/436605848 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bdc0ab96d0 msgr2=0x55bdc0ab9ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.879+0000 7f579d59d640 1 --2- 172.21.15.86:0/436605848 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bdc0ab96d0 0x55bdc0ab9ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.879+0000 7f579d59d640 1 -- 172.21.15.86:0/436605848 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdc09ad1b0 con 0x55bdc0aba080 2024-04-21T08:43:43.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.879+0000 7f579cd9c640 1 --2- 172.21.15.86:0/436605848 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdc0ac2c00 0x55bdc0ac5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:43.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.879+0000 7f579dd9e640 1 --2- 172.21.15.86:0/436605848 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bdc0ab96d0 0x55bdc0ab9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:43.890 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.879+0000 7f579d59d640 1 --2- 172.21.15.86:0/436605848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdc0aba080 0x55bdc0ac2630 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f57940cdb20 tx=0x7f5794010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f5e3a698956b89 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:43.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.879+0000 7f579dd9e640 1 --2- 172.21.15.86:0/2821425408 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f57840a3d20 0x7f57840a61e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5790009430 tx=0x7f579002e040 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:43.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.879+0000 7f57837fe640 1 -- 172.21.15.86:0/436605848 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5794020070 con 0x55bdc0aba080 2024-04-21T08:43:43.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.879+0000 7f57837fe640 1 -- 172.21.15.86:0/436605848 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f579409cb00 con 0x55bdc0aba080 2024-04-21T08:43:43.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.879+0000 7f57837fe640 1 -- 172.21.15.86:0/436605848 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5794098a70 con 0x55bdc0aba080 2024-04-21T08:43:43.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.880+0000 7f57a23f0740 1 -- 172.21.15.86:0/436605848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdc0aba080 msgr2=0x55bdc0ac2630 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.880+0000 7f57a23f0740 1 --2- 172.21.15.86:0/436605848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdc0aba080 0x55bdc0ac2630 secure :-1 s=READY pgs=1209 cs=0 l=1 rev1=1 crypto rx=0x7f57940cdb20 tx=0x7f5794010cb0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.880+0000 7f57a23f0740 1 -- 172.21.15.86:0/436605848 shutdown_connections 2024-04-21T08:43:43.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.880+0000 7f57a23f0740 1 --2- 172.21.15.86:0/436605848 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdc0ac2c00 0x55bdc0ac5050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.880+0000 7f57a23f0740 1 --2- 172.21.15.86:0/436605848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdc0aba080 0x55bdc0ac2630 unknown :-1 s=CLOSED pgs=1209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.880+0000 7f57a23f0740 1 --2- 172.21.15.86:0/436605848 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bdc0ab96d0 0x55bdc0ab9ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.880+0000 7f57a23f0740 1 -- 172.21.15.86:0/436605848 >> 172.21.15.86:0/436605848 conn(0x55bdc0ab3da0 msgr2=0x55bdc0ab4180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:43.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.880+0000 7f57a23f0740 1 -- 172.21.15.86:0/436605848 shutdown_connections 2024-04-21T08:43:43.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.880+0000 7f57a23f0740 1 -- 172.21.15.86:0/436605848 wait complete. 2024-04-21T08:43:43.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.880+0000 7f57a23f0740 1 -- start start 2024-04-21T08:43:43.891 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.881+0000 7f57a23f0740 1 Processor -- start 2024-04-21T08:43:43.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.881+0000 7f57a23f0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdc0ab96d0 0x55bdc0ac2490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.881+0000 7f57a23f0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bdc0aba080 0x55bdc0abf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.881+0000 7f57a23f0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdc0ac2c00 0x55bdc0abfd60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.881+0000 7f57a23f0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55bdc0ac5b10 con 0x55bdc0ac2c00 2024-04-21T08:43:43.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.881+0000 7f57a23f0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55bdc09d7500 con 0x55bdc0aba080 2024-04-21T08:43:43.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.881+0000 7f57a23f0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55bdc0ac5c90 con 0x55bdc0ab96d0 2024-04-21T08:43:43.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.881+0000 7f57a23f0740 10 client.0 _ll_register_callbacks cb 0x55bdc09a5b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:43.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.881+0000 7f579dd9e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdc0ac2c00 0x55bdc0abfd60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.881+0000 7f579dd9e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdc0ac2c00 0x55bdc0abfd60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37636/0 (socket says 172.21.15.86:37636) 2024-04-21T08:43:43.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.881+0000 7f579dd9e640 1 -- 172.21.15.86:0/1464696241 learned_addr learned my addr 172.21.15.86:0/1464696241 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:43.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.881+0000 7f579d59d640 1 --2- 172.21.15.86:0/1464696241 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdc0ab96d0 0x55bdc0ac2490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.881+0000 7f579cd9c640 1 --2- 172.21.15.86:0/1464696241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bdc0aba080 0x55bdc0abf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.881+0000 7f579dd9e640 1 -- 172.21.15.86:0/1464696241 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdc0ab96d0 msgr2=0x55bdc0ac2490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.881+0000 7f579dd9e640 1 --2- 172.21.15.86:0/1464696241 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdc0ab96d0 0x55bdc0ac2490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.881+0000 7f579dd9e640 1 -- 172.21.15.86:0/1464696241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bdc0aba080 msgr2=0x55bdc0abf820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.881+0000 7f579dd9e640 1 --2- 172.21.15.86:0/1464696241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bdc0aba080 0x55bdc0abf820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.881+0000 7f579dd9e640 1 -- 172.21.15.86:0/1464696241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdc07303f0 con 0x55bdc0ac2c00 2024-04-21T08:43:43.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.882+0000 7f579dd9e640 1 --2- 172.21.15.86:0/1464696241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdc0ac2c00 0x55bdc0abfd60 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f5790036000 tx=0x7f57900095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d65ae5be79ed095 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:43.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.882+0000 7f5780ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:43.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.882+0000 7f5780ff9640 1 -- 172.21.15.86:0/1464696241 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5790051020 con 0x55bdc0ac2c00 2024-04-21T08:43:43.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.882+0000 7f5780ff9640 1 -- 172.21.15.86:0/1464696241 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f57900041a0 con 0x55bdc0ac2c00 2024-04-21T08:43:43.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.882+0000 7f57a23f0740 1 -- 172.21.15.86:0/1464696241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f579409cb00 con 0x55bdc0ac2c00 2024-04-21T08:43:43.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.882+0000 7f5780ff9640 1 -- 172.21.15.86:0/1464696241 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5790049040 con 0x55bdc0ac2c00 2024-04-21T08:43:43.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.882+0000 7f57a23f0740 10 client.17802 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:43.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.882+0000 7f57a23f0740 1 -- 172.21.15.86:0/1464696241 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bdc07fa7d0 con 0x55bdc0ac2c00 2024-04-21T08:43:43.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.882+0000 7f5780ff9640 1 -- 172.21.15.86:0/1464696241 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f579409cb00 con 0x55bdc0ac2c00 2024-04-21T08:43:43.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.882+0000 7f5780ff9640 1 -- 172.21.15.86:0/1464696241 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f579004c030 con 0x55bdc0ac2c00 2024-04-21T08:43:43.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.882+0000 7f57a23f0740 10 client.17802 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:43.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.882+0000 7f57a23f0740 10 client.17802 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:43:43.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.882+0000 7f57a23f0740 20 client.17802 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:43.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.882+0000 7f57a23f0740 1 --2- 172.21.15.86:0/1464696241 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55bdc0ad2df0 0x55bdc0ad5210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.882+0000 7f57a23f0740 4 client.17802 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:43.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.882+0000 7f57a23f0740 1 -- 172.21.15.86:0/1464696241 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55bdc09bd930 con 0x55bdc0ad2df0 2024-04-21T08:43:43.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.883+0000 7f579d59d640 1 --2- 172.21.15.86:0/1464696241 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55bdc0ad2df0 0x55bdc0ad5210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.883+0000 7f5780ff9640 10 client.17802 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:43:43.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.883+0000 7f579d59d640 1 --2- 172.21.15.86:0/1464696241 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55bdc0ad2df0 0x55bdc0ad5210 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=f1005c63d965aa40 server_cookie=a86d30950f6f0406 in_seq=0 out_seq=0 2024-04-21T08:43:43.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.883+0000 7f5780ff9640 1 -- 172.21.15.86:0/1464696241 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55bdc09bd930 con 0x55bdc0ad2df0 2024-04-21T08:43:43.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.883+0000 7f5780ff9640 10 client.17802 handle_command_reply: tid=0 2024-04-21T08:43:43.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.884+0000 7f57a23f0740 1 client.17802 shutdown 2024-04-21T08:43:43.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.884+0000 7f57a23f0740 1 -- 172.21.15.86:0/1464696241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdc0ac2c00 msgr2=0x55bdc0abfd60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.884+0000 7f57a23f0740 1 --2- 172.21.15.86:0/1464696241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdc0ac2c00 0x55bdc0abfd60 secure :-1 s=READY pgs=1210 cs=0 l=1 rev1=1 crypto rx=0x7f5790036000 tx=0x7f57900095c0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.884+0000 7f57a23f0740 1 -- 172.21.15.86:0/1464696241 shutdown_connections 2024-04-21T08:43:43.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.884+0000 7f57a23f0740 1 --2- 172.21.15.86:0/1464696241 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55bdc0ad2df0 0x55bdc0ad5210 crc :-1 s=READY pgs=51 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.884+0000 7f57a23f0740 1 --2- 172.21.15.86:0/1464696241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdc0ac2c00 0x55bdc0abfd60 unknown :-1 s=CLOSED pgs=1210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.884+0000 7f57a23f0740 1 --2- 172.21.15.86:0/1464696241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bdc0aba080 0x55bdc0abf820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.884+0000 7f5780ff9640 0 client.17802 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:43:43.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.884+0000 7f57a23f0740 1 --2- 172.21.15.86:0/1464696241 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdc0ab96d0 0x55bdc0ac2490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.884+0000 7f57a23f0740 1 -- 172.21.15.86:0/1464696241 >> 172.21.15.86:0/1464696241 conn(0x55bdc0ab3da0 msgr2=0x55bdc0ab5af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:43.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.884+0000 7f57a23f0740 1 -- 172.21.15.86:0/1464696241 shutdown_connections 2024-04-21T08:43:43.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.884+0000 7f57a23f0740 1 -- 172.21.15.86:0/1464696241 wait complete. 2024-04-21T08:43:43.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.884+0000 7f57a23f0740 20 client.17802 trim_cache size 0 max 16384 2024-04-21T08:43:43.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.897+0000 7f57a23f0740 1 Processor -- start 2024-04-21T08:43:43.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.897+0000 7f57a23f0740 1 -- start start 2024-04-21T08:43:43.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.897+0000 7f57a23f0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdc0ab3da0 0x55bdc0acea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.897+0000 7f57a23f0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bdc0ab96d0 0x55bdc0acb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.897+0000 7f57a23f0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdc0aba080 0x55bdc0ab2ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.897+0000 7f57a23f0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57981af0b0 con 0x55bdc0aba080 2024-04-21T08:43:43.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.897+0000 7f57a23f0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f57981b0bf0 con 0x55bdc0ab96d0 2024-04-21T08:43:43.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.897+0000 7f57a23f0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f57981af780 con 0x55bdc0ab3da0 2024-04-21T08:43:43.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.897+0000 7f579dd9e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdc0aba080 0x55bdc0ab2ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.897+0000 7f579dd9e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdc0aba080 0x55bdc0ab2ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37652/0 (socket says 172.21.15.86:37652) 2024-04-21T08:43:43.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.897+0000 7f579dd9e640 1 -- 172.21.15.86:0/2739737838 learned_addr learned my addr 172.21.15.86:0/2739737838 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:43.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.898+0000 7f579cd9c640 1 --2- 172.21.15.86:0/2739737838 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bdc0ab96d0 0x55bdc0acb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.898+0000 7f579d59d640 1 --2- 172.21.15.86:0/2739737838 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdc0ab3da0 0x55bdc0acea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.898+0000 7f579dd9e640 1 -- 172.21.15.86:0/2739737838 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdc0ab3da0 msgr2=0x55bdc0acea00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.898+0000 7f579dd9e640 1 --2- 172.21.15.86:0/2739737838 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdc0ab3da0 0x55bdc0acea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.898+0000 7f579dd9e640 1 -- 172.21.15.86:0/2739737838 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bdc0ab96d0 msgr2=0x55bdc0acb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.898+0000 7f579dd9e640 1 --2- 172.21.15.86:0/2739737838 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bdc0ab96d0 0x55bdc0acb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.898+0000 7f579dd9e640 1 -- 172.21.15.86:0/2739737838 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55bdc0ac0360 con 0x55bdc0aba080 2024-04-21T08:43:43.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.898+0000 7f579cd9c640 1 --2- 172.21.15.86:0/2739737838 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bdc0ab96d0 0x55bdc0acb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:43.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.898+0000 7f579dd9e640 1 --2- 172.21.15.86:0/2739737838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdc0aba080 0x55bdc0ab2ca0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f579002c6c0 tx=0x7f57900434e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a51e5d6e460942 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:43.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.898+0000 7f5782ffd640 1 -- 172.21.15.86:0/2739737838 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5790065070 con 0x55bdc0aba080 2024-04-21T08:43:43.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.898+0000 7f5782ffd640 1 -- 172.21.15.86:0/2739737838 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f579003a030 con 0x55bdc0aba080 2024-04-21T08:43:43.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.898+0000 7f5782ffd640 1 -- 172.21.15.86:0/2739737838 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5790051070 con 0x55bdc0aba080 2024-04-21T08:43:43.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.898+0000 7f57a23f0740 1 -- 172.21.15.86:0/2739737838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdc0aba080 msgr2=0x55bdc0ab2ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.898+0000 7f57a23f0740 1 --2- 172.21.15.86:0/2739737838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdc0aba080 0x55bdc0ab2ca0 secure :-1 s=READY pgs=1211 cs=0 l=1 rev1=1 crypto rx=0x7f579002c6c0 tx=0x7f57900434e0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.899+0000 7f57a23f0740 1 -- 172.21.15.86:0/2739737838 shutdown_connections 2024-04-21T08:43:43.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.899+0000 7f57a23f0740 1 --2- 172.21.15.86:0/2739737838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdc0aba080 0x55bdc0ab2ca0 unknown :-1 s=CLOSED pgs=1211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.899+0000 7f57a23f0740 1 --2- 172.21.15.86:0/2739737838 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bdc0ab96d0 0x55bdc0acb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.899+0000 7f57a23f0740 1 --2- 172.21.15.86:0/2739737838 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdc0ab3da0 0x55bdc0acea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.899+0000 7f57a23f0740 1 -- 172.21.15.86:0/2739737838 >> 172.21.15.86:0/2739737838 conn(0x55bdc0abfec0 msgr2=0x55bdc0ab2590 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:43.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.899+0000 7f57a23f0740 1 -- 172.21.15.86:0/2739737838 shutdown_connections 2024-04-21T08:43:43.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.899+0000 7f57a23f0740 1 -- 172.21.15.86:0/2739737838 wait complete. 2024-04-21T08:43:43.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.899+0000 7f57a23f0740 1 -- start start 2024-04-21T08:43:43.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.899+0000 7f57a23f0740 1 Processor -- start 2024-04-21T08:43:43.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.899+0000 7f57a23f0740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdc0ab3da0 0x55bdc0b312f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.899+0000 7f57a23f0740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bdc0ab96d0 0x55bdc0b31830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.899+0000 7f57a23f0740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdc0aba080 0x55bdc0b31d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.899+0000 7f57a23f0740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57981b0520 con 0x55bdc0aba080 2024-04-21T08:43:43.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.899+0000 7f57a23f0740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f57981afe50 con 0x55bdc0ab96d0 2024-04-21T08:43:43.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.899+0000 7f57a23f0740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f57981ad070 con 0x55bdc0ab3da0 2024-04-21T08:43:43.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.899+0000 7f57a23f0740 10 client.0 _ll_register_callbacks cb 0x7f5790032890 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:43.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.899+0000 7f579dd9e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdc0aba080 0x55bdc0b31d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.899+0000 7f579dd9e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdc0aba080 0x55bdc0b31d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37656/0 (socket says 172.21.15.86:37656) 2024-04-21T08:43:43.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.899+0000 7f579dd9e640 1 -- 172.21.15.86:0/645755009 learned_addr learned my addr 172.21.15.86:0/645755009 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:43.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.900+0000 7f579cd9c640 1 --2- 172.21.15.86:0/645755009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bdc0ab96d0 0x55bdc0b31830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.900+0000 7f579d59d640 1 --2- 172.21.15.86:0/645755009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdc0ab3da0 0x55bdc0b312f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.900+0000 7f579dd9e640 1 -- 172.21.15.86:0/645755009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdc0ab3da0 msgr2=0x55bdc0b312f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.900+0000 7f579dd9e640 1 --2- 172.21.15.86:0/645755009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdc0ab3da0 0x55bdc0b312f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.900+0000 7f579dd9e640 1 -- 172.21.15.86:0/645755009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bdc0ab96d0 msgr2=0x55bdc0b31830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:43.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.900+0000 7f579dd9e640 1 --2- 172.21.15.86:0/645755009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bdc0ab96d0 0x55bdc0b31830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.900+0000 7f579dd9e640 1 -- 172.21.15.86:0/645755009 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55bdc0b3c4d0 con 0x55bdc0aba080 2024-04-21T08:43:43.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.900+0000 7f579cd9c640 1 --2- 172.21.15.86:0/645755009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bdc0ab96d0 0x55bdc0b31830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:43.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.900+0000 7f579d59d640 1 --2- 172.21.15.86:0/645755009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdc0ab3da0 0x55bdc0b312f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:43.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.900+0000 7f579dd9e640 1 --2- 172.21.15.86:0/645755009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdc0aba080 0x55bdc0b31d70 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f579002c790 tx=0x7f5790009940 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a26f9c481141cc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:43.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.900+0000 7f5780ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:43.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.900+0000 7f5780ff9640 1 -- 172.21.15.86:0/645755009 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5790065070 con 0x55bdc0aba080 2024-04-21T08:43:43.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.900+0000 7f5780ff9640 1 -- 172.21.15.86:0/645755009 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f5790034440 con 0x55bdc0aba080 2024-04-21T08:43:43.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.900+0000 7f57a23f0740 1 -- 172.21.15.86:0/645755009 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f579003a030 con 0x55bdc0aba080 2024-04-21T08:43:43.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.900+0000 7f5780ff9640 1 -- 172.21.15.86:0/645755009 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5790051040 con 0x55bdc0aba080 2024-04-21T08:43:43.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.900+0000 7f57a23f0740 10 client.17808 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:43.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.900+0000 7f57a23f0740 1 -- 172.21.15.86:0/645755009 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55bdc0b3d740 con 0x55bdc0aba080 2024-04-21T08:43:43.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.900+0000 7f5780ff9640 1 -- 172.21.15.86:0/645755009 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f579003a030 con 0x55bdc0aba080 2024-04-21T08:43:43.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.900+0000 7f5780ff9640 1 -- 172.21.15.86:0/645755009 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f579004c040 con 0x55bdc0aba080 2024-04-21T08:43:43.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.901+0000 7f57a23f0740 10 client.17808 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:43.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.901+0000 7f57a23f0740 10 client.17808 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:43:43.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.901+0000 7f57a23f0740 20 client.17808 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:43.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.901+0000 7f57a23f0740 1 --2- 172.21.15.86:0/645755009 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55bdc0ac2c00 0x55bdc0b3fe60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:43.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.901+0000 7f57a23f0740 4 client.17808 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:43:43.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.901+0000 7f57a23f0740 1 -- 172.21.15.86:0/645755009 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55bdc0a2bc10 con 0x55bdc0ac2c00 2024-04-21T08:43:43.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.901+0000 7f579d59d640 1 --2- 172.21.15.86:0/645755009 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55bdc0ac2c00 0x55bdc0b3fe60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:43.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.901+0000 7f5780ff9640 10 client.17808 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:43:43.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.901+0000 7f579d59d640 1 --2- 172.21.15.86:0/645755009 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55bdc0ac2c00 0x55bdc0b3fe60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=dce9d0ac81620a74 server_cookie=7dd791acafdd6c56 in_seq=0 out_seq=0 2024-04-21T08:43:43.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.901+0000 7f5780ff9640 1 -- 172.21.15.86:0/645755009 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55bdc0a2bc10 con 0x55bdc0ac2c00 2024-04-21T08:43:43.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.901+0000 7f5780ff9640 10 client.17808 handle_command_reply: tid=0 2024-04-21T08:43:43.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.901+0000 7f57a23f0740 1 client.17808 shutdown 2024-04-21T08:43:43.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.901+0000 7f57a23f0740 1 -- 172.21.15.86:0/645755009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdc0aba080 msgr2=0x55bdc0b31d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.901+0000 7f57a23f0740 1 --2- 172.21.15.86:0/645755009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdc0aba080 0x55bdc0b31d70 secure :-1 s=READY pgs=1212 cs=0 l=1 rev1=1 crypto rx=0x7f579002c790 tx=0x7f5790009940 comp rx=0 tx=0).stop 2024-04-21T08:43:43.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.901+0000 7f57a23f0740 1 -- 172.21.15.86:0/645755009 shutdown_connections 2024-04-21T08:43:43.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.901+0000 7f57a23f0740 1 --2- 172.21.15.86:0/645755009 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55bdc0ac2c00 0x55bdc0b3fe60 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.901+0000 7f57a23f0740 1 --2- 172.21.15.86:0/645755009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55bdc0aba080 0x55bdc0b31d70 unknown :-1 s=CLOSED pgs=1212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.901+0000 7f57a23f0740 1 --2- 172.21.15.86:0/645755009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55bdc0ab96d0 0x55bdc0b31830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.901+0000 7f5780ff9640 0 client.17808 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:43:43.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.901+0000 7f57a23f0740 1 --2- 172.21.15.86:0/645755009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55bdc0ab3da0 0x55bdc0b312f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.901+0000 7f57a23f0740 1 -- 172.21.15.86:0/645755009 >> 172.21.15.86:0/645755009 conn(0x55bdc0abfec0 msgr2=0x55bdc0ab2e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:43.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.902+0000 7f57a23f0740 1 -- 172.21.15.86:0/645755009 shutdown_connections 2024-04-21T08:43:43.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.902+0000 7f57a23f0740 1 -- 172.21.15.86:0/645755009 wait complete. 2024-04-21T08:43:43.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.902+0000 7f57a23f0740 20 client.17808 trim_cache size 0 max 16384 2024-04-21T08:43:43.916 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:43:43.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.902+0000 7f579e59f640 1 -- 172.21.15.86:0/2821425408 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f57840a3d20 msgr2=0x7f57840a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.902+0000 7f579e59f640 1 --2- 172.21.15.86:0/2821425408 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f57840a3d20 0x7f57840a61e0 secure :-1 s=READY pgs=189 cs=0 l=1 rev1=1 crypto rx=0x7f5790009430 tx=0x7f579002e040 comp rx=0 tx=0).stop 2024-04-21T08:43:43.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.902+0000 7f579e59f640 1 -- 172.21.15.86:0/2821425408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5798102fb0 msgr2=0x7f5798198520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:43.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.902+0000 7f579e59f640 1 --2- 172.21.15.86:0/2821425408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5798102fb0 0x7f5798198520 secure :-1 s=READY pgs=1208 cs=0 l=1 rev1=1 crypto rx=0x7f579400d950 tx=0x7f579400de10 comp rx=0 tx=0).stop 2024-04-21T08:43:43.918 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.902+0000 7f579e59f640 1 -- 172.21.15.86:0/2821425408 shutdown_connections 2024-04-21T08:43:43.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.902+0000 7f579e59f640 1 --2- 172.21.15.86:0/2821425408 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f57840a3d20 0x7f57840a61e0 unknown :-1 s=CLOSED pgs=189 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.902+0000 7f579e59f640 1 --2- 172.21.15.86:0/2821425408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f579810c3d0 0x7f579819cd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.902+0000 7f579e59f640 1 --2- 172.21.15.86:0/2821425408 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57981038d0 0x7f5798198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.902+0000 7f579e59f640 1 --2- 172.21.15.86:0/2821425408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5798102fb0 0x7f5798198520 unknown :-1 s=CLOSED pgs=1208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:43.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.902+0000 7f579e59f640 1 -- 172.21.15.86:0/2821425408 >> 172.21.15.86:0/2821425408 conn(0x7f57980fc7b0 msgr2=0x7f57980fded0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:43.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.902+0000 7f579e59f640 1 -- 172.21.15.86:0/2821425408 shutdown_connections 2024-04-21T08:43:43.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:43.902+0000 7f579e59f640 1 -- 172.21.15.86:0/2821425408 wait complete. 2024-04-21T08:43:43.919 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:43:43.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:43 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2351292169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:43:43.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:43 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2351292169' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:43:44.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.019+0000 7f33a0e31640 1 -- 172.21.15.86:0/3166141153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f339c107260 msgr2=0x7f339c10ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:44.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.019+0000 7f33a0e31640 1 --2- 172.21.15.86:0/3166141153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f339c107260 0x7f339c10ee30 secure :-1 s=READY pgs=1213 cs=0 l=1 rev1=1 crypto rx=0x7f3394009ca0 tx=0x7f339402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.019+0000 7f33a0e31640 1 -- 172.21.15.86:0/3166141153 shutdown_connections 2024-04-21T08:43:44.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.019+0000 7f33a0e31640 1 --2- 172.21.15.86:0/3166141153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f339c107260 0x7f339c10ee30 unknown :-1 s=CLOSED pgs=1213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.019+0000 7f33a0e31640 1 --2- 172.21.15.86:0/3166141153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f339c102b40 0x7f339c106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.019+0000 7f33a0e31640 1 --2- 172.21.15.86:0/3166141153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f339c102190 0x7f339c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.019+0000 7f33a0e31640 1 -- 172.21.15.86:0/3166141153 >> 172.21.15.86:0/3166141153 conn(0x7f339c0793a0 msgr2=0x7f339c0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:44.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.019+0000 7f33a0e31640 1 -- 172.21.15.86:0/3166141153 shutdown_connections 2024-04-21T08:43:44.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.019+0000 7f33a0e31640 1 -- 172.21.15.86:0/3166141153 wait complete. 2024-04-21T08:43:44.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.020+0000 7f33a0e31640 1 Processor -- start 2024-04-21T08:43:44.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.020+0000 7f33a0e31640 1 -- start start 2024-04-21T08:43:44.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.020+0000 7f33a0e31640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f339c102190 0x7f339c198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.020+0000 7f33a0e31640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f339c107260 0x7f339c198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.020+0000 7f33a0e31640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f339c19cfa0 0x7f339c19d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.020+0000 7f33a0e31640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f339c1121d0 con 0x7f339c19cfa0 2024-04-21T08:43:44.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.020+0000 7f33a0e31640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f339c112050 con 0x7f339c102190 2024-04-21T08:43:44.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.020+0000 7f33a0e31640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f339c112350 con 0x7f339c107260 2024-04-21T08:43:44.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.020+0000 7f339ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f339c19cfa0 0x7f339c19d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.020+0000 7f339ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f339c19cfa0 0x7f339c19d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37684/0 (socket says 172.21.15.86:37684) 2024-04-21T08:43:44.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.020+0000 7f339ad9d640 1 -- 172.21.15.86:0/3636632918 learned_addr learned my addr 172.21.15.86:0/3636632918 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:44.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.020+0000 7f339a59c640 1 --2- 172.21.15.86:0/3636632918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f339c102190 0x7f339c198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.020+0000 7f3399d9b640 1 --2- 172.21.15.86:0/3636632918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f339c107260 0x7f339c198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.020+0000 7f339ad9d640 1 -- 172.21.15.86:0/3636632918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f339c107260 msgr2=0x7f339c198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:44.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.020+0000 7f339ad9d640 1 --2- 172.21.15.86:0/3636632918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f339c107260 0x7f339c198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.020+0000 7f339ad9d640 1 -- 172.21.15.86:0/3636632918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f339c102190 msgr2=0x7f339c198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:44.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.020+0000 7f339ad9d640 1 --2- 172.21.15.86:0/3636632918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f339c102190 0x7f339c198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.020+0000 7f339ad9d640 1 -- 172.21.15.86:0/3636632918 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f339c19da90 con 0x7f339c19cfa0 2024-04-21T08:43:44.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.020+0000 7f339a59c640 1 --2- 172.21.15.86:0/3636632918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f339c102190 0x7f339c198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:44.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.020+0000 7f339ad9d640 1 --2- 172.21.15.86:0/3636632918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f339c19cfa0 0x7f339c19d450 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f3394005e30 tx=0x7f3394034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:44.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.021+0000 7f33937fe640 1 -- 172.21.15.86:0/3636632918 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3394039040 con 0x7f339c19cfa0 2024-04-21T08:43:44.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.021+0000 7f33a0e31640 1 -- 172.21.15.86:0/3636632918 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f339c19dd20 con 0x7f339c19cfa0 2024-04-21T08:43:44.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.021+0000 7f33937fe640 1 -- 172.21.15.86:0/3636632918 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3394044070 con 0x7f339c19cfa0 2024-04-21T08:43:44.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.021+0000 7f33937fe640 1 -- 172.21.15.86:0/3636632918 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3394003e60 con 0x7f339c19cfa0 2024-04-21T08:43:44.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.021+0000 7f33a0e31640 1 -- 172.21.15.86:0/3636632918 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f339c1a5540 con 0x7f339c19cfa0 2024-04-21T08:43:44.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.023+0000 7f33937fe640 1 -- 172.21.15.86:0/3636632918 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f33940324c0 con 0x7f339c19cfa0 2024-04-21T08:43:44.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.023+0000 7f33a3a24740 1 Processor -- start 2024-04-21T08:43:44.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.023+0000 7f33a3a24740 1 -- start start 2024-04-21T08:43:44.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.023+0000 7f33a3a24740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e61e64b4d0 0x55e61e64b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.030 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.023+0000 7f33a3a24740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e61e64be80 0x55e61e654430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.023+0000 7f33a3a24740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61e654a00 0x55e61e656e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.023+0000 7f33a3a24740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e61e5b01f0 con 0x55e61e654a00 2024-04-21T08:43:44.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.023+0000 7f33a3a24740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55e61e54d4b0 con 0x55e61e64be80 2024-04-21T08:43:44.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.023+0000 7f33a3a24740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e61e5ac1b0 con 0x55e61e64b4d0 2024-04-21T08:43:44.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.024+0000 7f3399d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61e654a00 0x55e61e656e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.024+0000 7f3399d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61e654a00 0x55e61e656e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37688/0 (socket says 172.21.15.86:37688) 2024-04-21T08:43:44.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.024+0000 7f3399d9b640 1 -- 172.21.15.86:0/538915233 learned_addr learned my addr 172.21.15.86:0/538915233 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:44.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.024+0000 7f33937fe640 1 --2- 172.21.15.86:0/3636632918 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f33880a3d20 0x7f33880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.024+0000 7f3399d9b640 1 --2- 172.21.15.86:0/538915233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e61e64b4d0 0x55e61e64b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.024+0000 7f339a59c640 1 --2- 172.21.15.86:0/538915233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e61e64be80 0x55e61e654430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.024+0000 7f339ad9d640 1 --2- 172.21.15.86:0/3636632918 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f33880a3d20 0x7f33880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.024+0000 7f3399d9b640 1 -- 172.21.15.86:0/538915233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e61e64b4d0 msgr2=0x55e61e64b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:44.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.024+0000 7f3399d9b640 1 --2- 172.21.15.86:0/538915233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e61e64b4d0 0x55e61e64b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.031 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.024+0000 7f3399d9b640 1 -- 172.21.15.86:0/538915233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e61e64be80 msgr2=0x55e61e654430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:44.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.024+0000 7f3399d9b640 1 --2- 172.21.15.86:0/538915233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e61e64be80 0x55e61e654430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.024+0000 7f3399d9b640 1 -- 172.21.15.86:0/538915233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e61e53f1b0 con 0x55e61e654a00 2024-04-21T08:43:44.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.024+0000 7f33937fe640 1 -- 172.21.15.86:0/3636632918 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f33940f2df0 con 0x7f339c19cfa0 2024-04-21T08:43:44.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.024+0000 7f3399d9b640 1 --2- 172.21.15.86:0/538915233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61e654a00 0x55e61e656e50 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f338c013910 tx=0x7f338c00e430 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2e428e8f6e5d30a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:44.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.024+0000 7f3390ff9640 1 -- 172.21.15.86:0/538915233 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338c004020 con 0x55e61e654a00 2024-04-21T08:43:44.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.024+0000 7f339ad9d640 1 --2- 172.21.15.86:0/3636632918 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f33880a3d20 0x7f33880a61e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f33940c7270 tx=0x7f3394091620 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:44.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.024+0000 7f3390ff9640 1 -- 172.21.15.86:0/538915233 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f338c00bdc0 con 0x55e61e654a00 2024-04-21T08:43:44.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.024+0000 7f3390ff9640 1 -- 172.21.15.86:0/538915233 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338c003850 con 0x55e61e654a00 2024-04-21T08:43:44.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.025+0000 7f33a3a24740 1 -- 172.21.15.86:0/538915233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61e654a00 msgr2=0x55e61e656e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:44.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.025+0000 7f33a3a24740 1 --2- 172.21.15.86:0/538915233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61e654a00 0x55e61e656e50 secure :-1 s=READY pgs=1215 cs=0 l=1 rev1=1 crypto rx=0x7f338c013910 tx=0x7f338c00e430 comp rx=0 tx=0).stop 2024-04-21T08:43:44.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.025+0000 7f33a3a24740 1 -- 172.21.15.86:0/538915233 shutdown_connections 2024-04-21T08:43:44.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.025+0000 7f33a3a24740 1 --2- 172.21.15.86:0/538915233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61e654a00 0x55e61e656e50 unknown :-1 s=CLOSED pgs=1215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.025+0000 7f33a3a24740 1 --2- 172.21.15.86:0/538915233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e61e64be80 0x55e61e654430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.025+0000 7f33a3a24740 1 --2- 172.21.15.86:0/538915233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e61e64b4d0 0x55e61e64b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.025+0000 7f33a3a24740 1 -- 172.21.15.86:0/538915233 >> 172.21.15.86:0/538915233 conn(0x55e61e645ba0 msgr2=0x55e61e645f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:44.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.025+0000 7f33a3a24740 1 -- 172.21.15.86:0/538915233 shutdown_connections 2024-04-21T08:43:44.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.025+0000 7f33a3a24740 1 -- 172.21.15.86:0/538915233 wait complete. 2024-04-21T08:43:44.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.025+0000 7f33a3a24740 1 -- start start 2024-04-21T08:43:44.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.026+0000 7f33a3a24740 1 Processor -- start 2024-04-21T08:43:44.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.026+0000 7f33a3a24740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61e64b4d0 0x55e61e654290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.026+0000 7f33a3a24740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e61e64be80 0x55e61e651620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.026+0000 7f33a3a24740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e61e654a00 0x55e61e651b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.026+0000 7f33a3a24740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e61e657970 con 0x55e61e64b4d0 2024-04-21T08:43:44.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.026+0000 7f33a3a24740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55e61e569500 con 0x55e61e654a00 2024-04-21T08:43:44.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.026+0000 7f33a3a24740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e61e657af0 con 0x55e61e64be80 2024-04-21T08:43:44.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.026+0000 7f33a3a24740 10 client.0 _ll_register_callbacks cb 0x55e61e537b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:44.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.026+0000 7f3399d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61e64b4d0 0x55e61e654290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.026+0000 7f3399d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61e64b4d0 0x55e61e654290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37694/0 (socket says 172.21.15.86:37694) 2024-04-21T08:43:44.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.026+0000 7f3399d9b640 1 -- 172.21.15.86:0/1574415766 learned_addr learned my addr 172.21.15.86:0/1574415766 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:44.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.026+0000 7f339a59c640 1 --2- 172.21.15.86:0/1574415766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e61e64be80 0x55e61e651620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.026+0000 7f3399d9b640 1 --2- 172.21.15.86:0/1574415766 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e61e654a00 0x55e61e651b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.026+0000 7f3399d9b640 1 -- 172.21.15.86:0/1574415766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e61e64be80 msgr2=0x55e61e651620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:44.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.026+0000 7f3399d9b640 1 --2- 172.21.15.86:0/1574415766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e61e64be80 0x55e61e651620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.026+0000 7f3399d9b640 1 -- 172.21.15.86:0/1574415766 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e61e654a00 msgr2=0x55e61e651b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:44.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.026+0000 7f3399d9b640 1 --2- 172.21.15.86:0/1574415766 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e61e654a00 0x55e61e651b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.026+0000 7f3399d9b640 1 -- 172.21.15.86:0/1574415766 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e61e2c23f0 con 0x55e61e64b4d0 2024-04-21T08:43:44.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.026+0000 7f339a59c640 1 --2- 172.21.15.86:0/1574415766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e61e64be80 0x55e61e651620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:44.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.026+0000 7f3399d9b640 1 --2- 172.21.15.86:0/1574415766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61e64b4d0 0x55e61e654290 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f338c004500 tx=0x7f338c001e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a94d5ba88646730f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:44.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.026+0000 7f3381ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:44.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.026+0000 7f3381ffb640 1 -- 172.21.15.86:0/1574415766 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338c00bb10 con 0x55e61e64b4d0 2024-04-21T08:43:44.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.027+0000 7f3381ffb640 1 -- 172.21.15.86:0/1574415766 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f338c011440 con 0x55e61e64b4d0 2024-04-21T08:43:44.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.027+0000 7f33a3a24740 1 -- 172.21.15.86:0/1574415766 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f338c00bdc0 con 0x55e61e64b4d0 2024-04-21T08:43:44.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.027+0000 7f3381ffb640 1 -- 172.21.15.86:0/1574415766 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f338c033050 con 0x55e61e64b4d0 2024-04-21T08:43:44.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.027+0000 7f33a3a24740 10 client.17820 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:44.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.027+0000 7f33a3a24740 1 -- 172.21.15.86:0/1574415766 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e61e38c7d0 con 0x55e61e64b4d0 2024-04-21T08:43:44.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.027+0000 7f3381ffb640 1 -- 172.21.15.86:0/1574415766 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f338c00bdc0 con 0x55e61e64b4d0 2024-04-21T08:43:44.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.027+0000 7f3381ffb640 1 -- 172.21.15.86:0/1574415766 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f338c01e040 con 0x55e61e64b4d0 2024-04-21T08:43:44.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.027+0000 7f33a3a24740 10 client.17820 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:44.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.027+0000 7f33a3a24740 10 client.17820 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:43:44.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.027+0000 7f33a3a24740 20 client.17820 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:44.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.027+0000 7f33a3a24740 1 --2- 172.21.15.86:0/1574415766 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55e61e664af0 0x55e61e666ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.027+0000 7f33a3a24740 4 client.17820 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:44.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.027+0000 7f33a3a24740 1 -- 172.21.15.86:0/1574415766 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e61e54f930 con 0x55e61e664af0 2024-04-21T08:43:44.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.027+0000 7f339a59c640 1 --2- 172.21.15.86:0/1574415766 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55e61e664af0 0x55e61e666ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.028+0000 7f339a59c640 1 --2- 172.21.15.86:0/1574415766 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55e61e664af0 0x55e61e666ee0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eded742d491aa3d8 server_cookie=aa58376de1e00bb8 in_seq=0 out_seq=0 2024-04-21T08:43:44.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.028+0000 7f3381ffb640 10 client.17820 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:43:44.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.029+0000 7f3381ffb640 1 -- 172.21.15.86:0/1574415766 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e61e54f930 con 0x55e61e664af0 2024-04-21T08:43:44.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.029+0000 7f3381ffb640 10 client.17820 handle_command_reply: tid=0 2024-04-21T08:43:44.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.029+0000 7f33a3a24740 1 client.17820 shutdown 2024-04-21T08:43:44.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.029+0000 7f33a3a24740 1 -- 172.21.15.86:0/1574415766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61e64b4d0 msgr2=0x55e61e654290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:44.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.029+0000 7f33a3a24740 1 --2- 172.21.15.86:0/1574415766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61e64b4d0 0x55e61e654290 secure :-1 s=READY pgs=1216 cs=0 l=1 rev1=1 crypto rx=0x7f338c004500 tx=0x7f338c001e50 comp rx=0 tx=0).stop 2024-04-21T08:43:44.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.029+0000 7f33a3a24740 1 -- 172.21.15.86:0/1574415766 shutdown_connections 2024-04-21T08:43:44.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.029+0000 7f33a3a24740 1 --2- 172.21.15.86:0/1574415766 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e61e654a00 0x55e61e651b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.029+0000 7f33a3a24740 1 --2- 172.21.15.86:0/1574415766 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e61e64be80 0x55e61e651620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.029+0000 7f33a3a24740 1 --2- 172.21.15.86:0/1574415766 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55e61e664af0 0x55e61e666ee0 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.029+0000 7f33a3a24740 1 --2- 172.21.15.86:0/1574415766 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61e64b4d0 0x55e61e654290 unknown :-1 s=CLOSED pgs=1216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.029+0000 7f3381ffb640 0 client.17820 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:43:44.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.029+0000 7f33a3a24740 1 -- 172.21.15.86:0/1574415766 >> 172.21.15.86:0/1574415766 conn(0x55e61e645ba0 msgr2=0x55e61e6478f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:44.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.029+0000 7f33a3a24740 1 -- 172.21.15.86:0/1574415766 shutdown_connections 2024-04-21T08:43:44.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.029+0000 7f33a3a24740 1 -- 172.21.15.86:0/1574415766 wait complete. 2024-04-21T08:43:44.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.029+0000 7f33a3a24740 20 client.17820 trim_cache size 0 max 16384 2024-04-21T08:43:44.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.042+0000 7f33a3a24740 1 Processor -- start 2024-04-21T08:43:44.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.042+0000 7f33a3a24740 1 -- start start 2024-04-21T08:43:44.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.042+0000 7f33a3a24740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e61e64b4d0 0x55e61e660860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.042+0000 7f33a3a24740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61e64be80 0x55e61e65d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.042+0000 7f33a3a24740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e61e654a00 0x55e61e650650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.042+0000 7f33a3a24740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f339c1af2e0 con 0x55e61e64be80 2024-04-21T08:43:44.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.042+0000 7f33a3a24740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f339c1b0e20 con 0x55e61e654a00 2024-04-21T08:43:44.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.042+0000 7f33a3a24740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f339c1af9b0 con 0x55e61e64b4d0 2024-04-21T08:43:44.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.042+0000 7f339a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61e64be80 0x55e61e65d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.042+0000 7f339a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61e64be80 0x55e61e65d150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37708/0 (socket says 172.21.15.86:37708) 2024-04-21T08:43:44.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.042+0000 7f339a59c640 1 -- 172.21.15.86:0/3444001369 learned_addr learned my addr 172.21.15.86:0/3444001369 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:44.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.042+0000 7f3399d9b640 1 --2- 172.21.15.86:0/3444001369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e61e64b4d0 0x55e61e660860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.042+0000 7f3399d9b640 1 --2- 172.21.15.86:0/3444001369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e61e654a00 0x55e61e650650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.042+0000 7f339a59c640 1 -- 172.21.15.86:0/3444001369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e61e64b4d0 msgr2=0x55e61e660860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:44.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.042+0000 7f339a59c640 1 --2- 172.21.15.86:0/3444001369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e61e64b4d0 0x55e61e660860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.042+0000 7f339a59c640 1 -- 172.21.15.86:0/3444001369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e61e654a00 msgr2=0x55e61e650650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:44.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.042+0000 7f339a59c640 1 --2- 172.21.15.86:0/3444001369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e61e654a00 0x55e61e650650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.042+0000 7f339a59c640 1 -- 172.21.15.86:0/3444001369 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e61e647e90 con 0x55e61e64be80 2024-04-21T08:43:44.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.042+0000 7f3399d9b640 1 --2- 172.21.15.86:0/3444001369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e61e654a00 0x55e61e650650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:44.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.042+0000 7f339a59c640 1 --2- 172.21.15.86:0/3444001369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61e64be80 0x55e61e65d150 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x55e61e64c280 tx=0x7f3384013be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca4c61951006d411 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:44.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.043+0000 7f3383fff640 1 -- 172.21.15.86:0/3444001369 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338401d430 con 0x55e61e64be80 2024-04-21T08:43:44.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.043+0000 7f3383fff640 1 -- 172.21.15.86:0/3444001369 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3384005020 con 0x55e61e64be80 2024-04-21T08:43:44.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.043+0000 7f3383fff640 1 -- 172.21.15.86:0/3444001369 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3384010d80 con 0x55e61e64be80 2024-04-21T08:43:44.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.043+0000 7f33a3a24740 1 -- 172.21.15.86:0/3444001369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61e64be80 msgr2=0x55e61e65d150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:44.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.043+0000 7f33a3a24740 1 --2- 172.21.15.86:0/3444001369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61e64be80 0x55e61e65d150 secure :-1 s=READY pgs=1217 cs=0 l=1 rev1=1 crypto rx=0x55e61e64c280 tx=0x7f3384013be0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.043+0000 7f33a3a24740 1 -- 172.21.15.86:0/3444001369 shutdown_connections 2024-04-21T08:43:44.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.043+0000 7f33a3a24740 1 --2- 172.21.15.86:0/3444001369 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e61e654a00 0x55e61e650650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.043+0000 7f33a3a24740 1 --2- 172.21.15.86:0/3444001369 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61e64be80 0x55e61e65d150 unknown :-1 s=CLOSED pgs=1217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.043+0000 7f33a3a24740 1 --2- 172.21.15.86:0/3444001369 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e61e64b4d0 0x55e61e660860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.043+0000 7f33a3a24740 1 -- 172.21.15.86:0/3444001369 >> 172.21.15.86:0/3444001369 conn(0x55e61e645ba0 msgr2=0x55e61e644c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:44.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.043+0000 7f33a3a24740 1 -- 172.21.15.86:0/3444001369 shutdown_connections 2024-04-21T08:43:44.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.043+0000 7f33a3a24740 1 -- 172.21.15.86:0/3444001369 wait complete. 2024-04-21T08:43:44.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.043+0000 7f33a3a24740 1 -- start start 2024-04-21T08:43:44.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.044+0000 7f33a3a24740 1 Processor -- start 2024-04-21T08:43:44.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.044+0000 7f33a3a24740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61e64b4d0 0x55e61e6c71e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.044+0000 7f33a3a24740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e61e64be80 0x55e61e6c7720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.044+0000 7f33a3a24740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e61e654a00 0x55e61e650900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.044+0000 7f33a3a24740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f339c1b0750 con 0x55e61e64b4d0 2024-04-21T08:43:44.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.044+0000 7f33a3a24740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f339c1b0080 con 0x55e61e654a00 2024-04-21T08:43:44.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.044+0000 7f33a3a24740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f339c1ad2a0 con 0x55e61e64be80 2024-04-21T08:43:44.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.044+0000 7f33a3a24740 10 client.0 _ll_register_callbacks cb 0x7f338c013810 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:44.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.044+0000 7f3399d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61e64b4d0 0x55e61e6c71e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.044+0000 7f3399d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61e64b4d0 0x55e61e6c71e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37720/0 (socket says 172.21.15.86:37720) 2024-04-21T08:43:44.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.044+0000 7f3399d9b640 1 -- 172.21.15.86:0/312285371 learned_addr learned my addr 172.21.15.86:0/312285371 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:44.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.044+0000 7f339a59c640 1 --2- 172.21.15.86:0/312285371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e61e64be80 0x55e61e6c7720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.044+0000 7f3399d9b640 1 --2- 172.21.15.86:0/312285371 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e61e654a00 0x55e61e650900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.044+0000 7f3399d9b640 1 -- 172.21.15.86:0/312285371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e61e64be80 msgr2=0x55e61e6c7720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:44.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.044+0000 7f3399d9b640 1 --2- 172.21.15.86:0/312285371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e61e64be80 0x55e61e6c7720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.044+0000 7f3399d9b640 1 -- 172.21.15.86:0/312285371 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e61e654a00 msgr2=0x55e61e650900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:44.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.044+0000 7f3399d9b640 1 --2- 172.21.15.86:0/312285371 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e61e654a00 0x55e61e650900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.044+0000 7f3399d9b640 1 -- 172.21.15.86:0/312285371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e61e650ff0 con 0x55e61e64b4d0 2024-04-21T08:43:44.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.044+0000 7f339a59c640 1 --2- 172.21.15.86:0/312285371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e61e64be80 0x55e61e6c7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:44.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.044+0000 7f3399d9b640 1 --2- 172.21.15.86:0/312285371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61e64b4d0 0x55e61e6c71e0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f338c01f8b0 tx=0x7f338c025040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a3770d872c7d1d0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:44.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.044+0000 7f3381ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:44.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.044+0000 7f3381ffb640 1 -- 172.21.15.86:0/312285371 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f338c00bd20 con 0x55e61e64b4d0 2024-04-21T08:43:44.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.044+0000 7f3381ffb640 1 -- 172.21.15.86:0/312285371 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f338c005070 con 0x55e61e64b4d0 2024-04-21T08:43:44.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.044+0000 7f33a3a24740 1 -- 172.21.15.86:0/312285371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3384005020 con 0x55e61e64b4d0 2024-04-21T08:43:44.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.045+0000 7f3381ffb640 1 -- 172.21.15.86:0/312285371 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f338c00ee10 con 0x55e61e64b4d0 2024-04-21T08:43:44.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.045+0000 7f33a3a24740 10 client.17826 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:44.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.045+0000 7f33a3a24740 1 -- 172.21.15.86:0/312285371 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e61e6ce900 con 0x55e61e64b4d0 2024-04-21T08:43:44.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.045+0000 7f3381ffb640 1 -- 172.21.15.86:0/312285371 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3384005020 con 0x55e61e64b4d0 2024-04-21T08:43:44.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.045+0000 7f3381ffb640 1 -- 172.21.15.86:0/312285371 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f338c004980 con 0x55e61e64b4d0 2024-04-21T08:43:44.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.045+0000 7f33a3a24740 10 client.17826 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:44.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.045+0000 7f33a3a24740 10 client.17826 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:43:44.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.045+0000 7f33a3a24740 20 client.17826 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:44.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.045+0000 7f33a3a24740 1 --2- 172.21.15.86:0/312285371 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55e61e6cf030 0x55e61e6d1420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.045+0000 7f33a3a24740 4 client.17826 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:43:44.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.045+0000 7f33a3a24740 1 -- 172.21.15.86:0/312285371 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e61e5bdc10 con 0x55e61e6cf030 2024-04-21T08:43:44.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.045+0000 7f339a59c640 1 --2- 172.21.15.86:0/312285371 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55e61e6cf030 0x55e61e6d1420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.046+0000 7f3381ffb640 10 client.17826 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:43:44.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.046+0000 7f339a59c640 1 --2- 172.21.15.86:0/312285371 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55e61e6cf030 0x55e61e6d1420 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=6a43c71c667cd8fc server_cookie=cd3c3158a0a9f017 in_seq=0 out_seq=0 2024-04-21T08:43:44.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.046+0000 7f3381ffb640 1 -- 172.21.15.86:0/312285371 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e61e5bdc10 con 0x55e61e6cf030 2024-04-21T08:43:44.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.046+0000 7f3381ffb640 10 client.17826 handle_command_reply: tid=0 2024-04-21T08:43:44.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.046+0000 7f33a3a24740 1 client.17826 shutdown 2024-04-21T08:43:44.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.046+0000 7f33a3a24740 1 -- 172.21.15.86:0/312285371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61e64b4d0 msgr2=0x55e61e6c71e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:44.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.046+0000 7f33a3a24740 1 --2- 172.21.15.86:0/312285371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61e64b4d0 0x55e61e6c71e0 secure :-1 s=READY pgs=1218 cs=0 l=1 rev1=1 crypto rx=0x7f338c01f8b0 tx=0x7f338c025040 comp rx=0 tx=0).stop 2024-04-21T08:43:44.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.046+0000 7f33a3a24740 1 -- 172.21.15.86:0/312285371 shutdown_connections 2024-04-21T08:43:44.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.046+0000 7f33a3a24740 1 --2- 172.21.15.86:0/312285371 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e61e654a00 0x55e61e650900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.046+0000 7f33a3a24740 1 --2- 172.21.15.86:0/312285371 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e61e64be80 0x55e61e6c7720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.046+0000 7f33a3a24740 1 --2- 172.21.15.86:0/312285371 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55e61e6cf030 0x55e61e6d1420 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.046+0000 7f33a3a24740 1 --2- 172.21.15.86:0/312285371 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e61e64b4d0 0x55e61e6c71e0 unknown :-1 s=CLOSED pgs=1218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.046+0000 7f33a3a24740 1 -- 172.21.15.86:0/312285371 >> 172.21.15.86:0/312285371 conn(0x55e61e645ba0 msgr2=0x55e61e647710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:44.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.046+0000 7f3381ffb640 0 client.17826 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:43:44.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.046+0000 7f33a3a24740 1 -- 172.21.15.86:0/312285371 shutdown_connections 2024-04-21T08:43:44.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.046+0000 7f33a3a24740 1 -- 172.21.15.86:0/312285371 wait complete. 2024-04-21T08:43:44.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.046+0000 7f33a3a24740 20 client.17826 trim_cache size 0 max 16384 2024-04-21T08:43:44.054 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:43:44.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.046+0000 7f33a0e31640 1 -- 172.21.15.86:0/3636632918 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f33880a3d20 msgr2=0x7f33880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:44.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.046+0000 7f33a0e31640 1 --2- 172.21.15.86:0/3636632918 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f33880a3d20 0x7f33880a61e0 secure :-1 s=READY pgs=190 cs=0 l=1 rev1=1 crypto rx=0x7f33940c7270 tx=0x7f3394091620 comp rx=0 tx=0).stop 2024-04-21T08:43:44.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.046+0000 7f33a0e31640 1 -- 172.21.15.86:0/3636632918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f339c19cfa0 msgr2=0x7f339c19d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:44.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.046+0000 7f33a0e31640 1 --2- 172.21.15.86:0/3636632918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f339c19cfa0 0x7f339c19d450 secure :-1 s=READY pgs=1214 cs=0 l=1 rev1=1 crypto rx=0x7f3394005e30 tx=0x7f3394034760 comp rx=0 tx=0).stop 2024-04-21T08:43:44.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.047+0000 7f33a0e31640 1 -- 172.21.15.86:0/3636632918 shutdown_connections 2024-04-21T08:43:44.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.047+0000 7f33a0e31640 1 --2- 172.21.15.86:0/3636632918 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f33880a3d20 0x7f33880a61e0 unknown :-1 s=CLOSED pgs=190 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.047+0000 7f33a0e31640 1 --2- 172.21.15.86:0/3636632918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f339c19cfa0 0x7f339c19d450 unknown :-1 s=CLOSED pgs=1214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.047+0000 7f33a0e31640 1 --2- 172.21.15.86:0/3636632918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f339c107260 0x7f339c198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.047+0000 7f33a0e31640 1 --2- 172.21.15.86:0/3636632918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f339c102190 0x7f339c198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.047+0000 7f33a0e31640 1 -- 172.21.15.86:0/3636632918 >> 172.21.15.86:0/3636632918 conn(0x7f339c0793a0 msgr2=0x7f339c0ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:44.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.047+0000 7f33a0e31640 1 -- 172.21.15.86:0/3636632918 shutdown_connections 2024-04-21T08:43:44.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.047+0000 7f33a0e31640 1 -- 172.21.15.86:0/3636632918 wait complete. 2024-04-21T08:43:44.057 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:43:44.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.157+0000 7f14ce706640 1 -- 172.21.15.86:0/1064605253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14c806a570 msgr2=0x7f14c806a9f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:44.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.157+0000 7f14ce706640 1 --2- 172.21.15.86:0/1064605253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14c806a570 0x7f14c806a9f0 secure :-1 s=READY pgs=1219 cs=0 l=1 rev1=1 crypto rx=0x7f14b8009930 tx=0x7f14b802bc30 comp rx=0 tx=0).stop 2024-04-21T08:43:44.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.157+0000 7f14ce706640 1 -- 172.21.15.86:0/1064605253 shutdown_connections 2024-04-21T08:43:44.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.157+0000 7f14ce706640 1 --2- 172.21.15.86:0/1064605253 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14c806af30 0x7f14c810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.157+0000 7f14ce706640 1 --2- 172.21.15.86:0/1064605253 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14c806a570 0x7f14c806a9f0 unknown :-1 s=CLOSED pgs=1219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.157+0000 7f14ce706640 1 --2- 172.21.15.86:0/1064605253 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14c8106f90 0x7f14c8107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.157+0000 7f14ce706640 1 -- 172.21.15.86:0/1064605253 >> 172.21.15.86:0/1064605253 conn(0x7f14c8077e30 msgr2=0x7f14c8078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:44.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.157+0000 7f14ce706640 1 -- 172.21.15.86:0/1064605253 shutdown_connections 2024-04-21T08:43:44.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.157+0000 7f14ce706640 1 -- 172.21.15.86:0/1064605253 wait complete. 2024-04-21T08:43:44.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.157+0000 7f14ce706640 1 Processor -- start 2024-04-21T08:43:44.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.157+0000 7f14ce706640 1 -- start start 2024-04-21T08:43:44.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.158+0000 7f14ce706640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14c806a570 0x7f14c819e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.158+0000 7f14ce706640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14c806af30 0x7f14c819e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.158+0000 7f14ce706640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14c8106f90 0x7f14c81985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.158+0000 7f14ce706640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14c8112210 con 0x7f14c8106f90 2024-04-21T08:43:44.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.158+0000 7f14ce706640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f14c8112090 con 0x7f14c806a570 2024-04-21T08:43:44.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.158+0000 7f14ce706640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f14c8112390 con 0x7f14c806af30 2024-04-21T08:43:44.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.158+0000 7f14ccca3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14c8106f90 0x7f14c81985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.158+0000 7f14ccca3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14c8106f90 0x7f14c81985b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37742/0 (socket says 172.21.15.86:37742) 2024-04-21T08:43:44.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.158+0000 7f14ccca3640 1 -- 172.21.15.86:0/3538057999 learned_addr learned my addr 172.21.15.86:0/3538057999 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:44.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.158+0000 7f14c77fe640 1 --2- 172.21.15.86:0/3538057999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14c806af30 0x7f14c819e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.158+0000 7f14c7fff640 1 --2- 172.21.15.86:0/3538057999 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14c806a570 0x7f14c819e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.158+0000 7f14ccca3640 1 -- 172.21.15.86:0/3538057999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14c806af30 msgr2=0x7f14c819e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:44.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.158+0000 7f14ccca3640 1 --2- 172.21.15.86:0/3538057999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14c806af30 0x7f14c819e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.158+0000 7f14ccca3640 1 -- 172.21.15.86:0/3538057999 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14c806a570 msgr2=0x7f14c819e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:44.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.158+0000 7f14ccca3640 1 --2- 172.21.15.86:0/3538057999 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14c806a570 0x7f14c819e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.158+0000 7f14ccca3640 1 -- 172.21.15.86:0/3538057999 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14c8198d30 con 0x7f14c8106f90 2024-04-21T08:43:44.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.158+0000 7f14c77fe640 1 --2- 172.21.15.86:0/3538057999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14c806af30 0x7f14c819e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:44.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.158+0000 7f14c7fff640 1 --2- 172.21.15.86:0/3538057999 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14c806a570 0x7f14c819e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:44.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.158+0000 7f14ccca3640 1 --2- 172.21.15.86:0/3538057999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14c8106f90 0x7f14c81985b0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f14c000eaa0 tx=0x7f14c000ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:44.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.158+0000 7f14c57fa640 1 -- 172.21.15.86:0/3538057999 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c000cdb0 con 0x7f14c8106f90 2024-04-21T08:43:44.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.158+0000 7f14c57fa640 1 -- 172.21.15.86:0/3538057999 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14c0005020 con 0x7f14c8106f90 2024-04-21T08:43:44.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.158+0000 7f14ce706640 1 -- 172.21.15.86:0/3538057999 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f14c8199020 con 0x7f14c8106f90 2024-04-21T08:43:44.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.158+0000 7f14c57fa640 1 -- 172.21.15.86:0/3538057999 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c00116d0 con 0x7f14c8106f90 2024-04-21T08:43:44.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.158+0000 7f14ce706640 1 -- 172.21.15.86:0/3538057999 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14c81a52c0 con 0x7f14c8106f90 2024-04-21T08:43:44.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.161+0000 7f14c57fa640 1 -- 172.21.15.86:0/3538057999 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f14c00051d0 con 0x7f14c8106f90 2024-04-21T08:43:44.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.161+0000 7f14d13cb740 1 Processor -- start 2024-04-21T08:43:44.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.161+0000 7f14d13cb740 1 -- start start 2024-04-21T08:43:44.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.161+0000 7f14c57fa640 1 --2- 172.21.15.86:0/3538057999 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f14b00a3d20 0x7f14b00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.161+0000 7f14d13cb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55743d0ab4d0 0x55743d0ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.161+0000 7f14d13cb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743d0abe80 0x55743d0b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.161+0000 7f14d13cb740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55743d0b4a00 0x55743d0b6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.161+0000 7f14d13cb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55743d0101f0 con 0x55743d0abe80 2024-04-21T08:43:44.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.161+0000 7f14d13cb740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55743cfad4b0 con 0x55743d0b4a00 2024-04-21T08:43:44.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.161+0000 7f14d13cb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55743d00c1b0 con 0x55743d0ab4d0 2024-04-21T08:43:44.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.161+0000 7f14c77fe640 1 --2- 172.21.15.86:0/3538057999 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f14b00a3d20 0x7f14b00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.161+0000 7f14c57fa640 1 -- 172.21.15.86:0/3538057999 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f14c00d0e30 con 0x7f14c8106f90 2024-04-21T08:43:44.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.161+0000 7f14c77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743d0abe80 0x55743d0b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.161+0000 7f14c77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743d0abe80 0x55743d0b4430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37754/0 (socket says 172.21.15.86:37754) 2024-04-21T08:43:44.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.161+0000 7f14c77fe640 1 -- 172.21.15.86:0/1328376020 learned_addr learned my addr 172.21.15.86:0/1328376020 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:44.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.161+0000 7f14c7fff640 1 --2- 172.21.15.86:0/1328376020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55743d0ab4d0 0x55743d0ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.162+0000 7f14ccca3640 1 --2- 172.21.15.86:0/1328376020 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55743d0b4a00 0x55743d0b6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.162+0000 7f14c77fe640 1 -- 172.21.15.86:0/1328376020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55743d0ab4d0 msgr2=0x55743d0ab8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:44.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.162+0000 7f14c77fe640 1 --2- 172.21.15.86:0/1328376020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55743d0ab4d0 0x55743d0ab8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.162+0000 7f14c77fe640 1 -- 172.21.15.86:0/1328376020 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55743d0b4a00 msgr2=0x55743d0b6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:44.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.162+0000 7f14c77fe640 1 --2- 172.21.15.86:0/1328376020 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55743d0b4a00 0x55743d0b6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.162+0000 7f14c77fe640 1 -- 172.21.15.86:0/1328376020 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55743cf9f1b0 con 0x55743d0abe80 2024-04-21T08:43:44.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.162+0000 7f14c77fe640 1 --2- 172.21.15.86:0/3538057999 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f14b00a3d20 0x7f14b00a61e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f14b802c780 tx=0x7f14b802c650 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:44.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.162+0000 7f14c77fe640 1 --2- 172.21.15.86:0/1328376020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743d0abe80 0x55743d0b4430 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f14b8005ec0 tx=0x7f14b8045000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db966d037347bf3e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:44.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.162+0000 7f14ae7fc640 1 -- 172.21.15.86:0/1328376020 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b8052070 con 0x55743d0abe80 2024-04-21T08:43:44.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.162+0000 7f14ae7fc640 1 -- 172.21.15.86:0/1328376020 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14b80054e0 con 0x55743d0abe80 2024-04-21T08:43:44.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.162+0000 7f14ae7fc640 1 -- 172.21.15.86:0/1328376020 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14b80036b0 con 0x55743d0abe80 2024-04-21T08:43:44.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.163+0000 7f14d13cb740 1 -- 172.21.15.86:0/1328376020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743d0abe80 msgr2=0x55743d0b4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:44.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.163+0000 7f14d13cb740 1 --2- 172.21.15.86:0/1328376020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743d0abe80 0x55743d0b4430 secure :-1 s=READY pgs=1221 cs=0 l=1 rev1=1 crypto rx=0x7f14b8005ec0 tx=0x7f14b8045000 comp rx=0 tx=0).stop 2024-04-21T08:43:44.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.163+0000 7f14d13cb740 1 -- 172.21.15.86:0/1328376020 shutdown_connections 2024-04-21T08:43:44.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.163+0000 7f14d13cb740 1 --2- 172.21.15.86:0/1328376020 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55743d0b4a00 0x55743d0b6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.163+0000 7f14d13cb740 1 --2- 172.21.15.86:0/1328376020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743d0abe80 0x55743d0b4430 unknown :-1 s=CLOSED pgs=1221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.163+0000 7f14d13cb740 1 --2- 172.21.15.86:0/1328376020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55743d0ab4d0 0x55743d0ab8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.163+0000 7f14d13cb740 1 -- 172.21.15.86:0/1328376020 >> 172.21.15.86:0/1328376020 conn(0x55743d0a5ba0 msgr2=0x55743d0a5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:44.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.163+0000 7f14d13cb740 1 -- 172.21.15.86:0/1328376020 shutdown_connections 2024-04-21T08:43:44.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.163+0000 7f14d13cb740 1 -- 172.21.15.86:0/1328376020 wait complete. 2024-04-21T08:43:44.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.163+0000 7f14d13cb740 1 -- start start 2024-04-21T08:43:44.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.163+0000 7f14d13cb740 1 Processor -- start 2024-04-21T08:43:44.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.164+0000 7f14d13cb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55743d0ab4d0 0x55743d0b4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.164+0000 7f14d13cb740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55743d0abe80 0x55743d0b16a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.164+0000 7f14d13cb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743d0b4a00 0x55743d0b1c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.164+0000 7f14d13cb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55743d0b7910 con 0x55743d0b4a00 2024-04-21T08:43:44.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.164+0000 7f14d13cb740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55743cfc9500 con 0x55743d0abe80 2024-04-21T08:43:44.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.164+0000 7f14d13cb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55743d0b7a90 con 0x55743d0ab4d0 2024-04-21T08:43:44.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.164+0000 7f14d13cb740 10 client.0 _ll_register_callbacks cb 0x55743cf97b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:44.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.164+0000 7f14ccca3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743d0b4a00 0x55743d0b1c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.164+0000 7f14ccca3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743d0b4a00 0x55743d0b1c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37758/0 (socket says 172.21.15.86:37758) 2024-04-21T08:43:44.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.164+0000 7f14ccca3640 1 -- 172.21.15.86:0/3489805492 learned_addr learned my addr 172.21.15.86:0/3489805492 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:44.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.164+0000 7f14c7fff640 1 --2- 172.21.15.86:0/3489805492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55743d0ab4d0 0x55743d0b4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.164+0000 7f14c77fe640 1 --2- 172.21.15.86:0/3489805492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55743d0abe80 0x55743d0b16a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.164+0000 7f14ccca3640 1 -- 172.21.15.86:0/3489805492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55743d0ab4d0 msgr2=0x55743d0b4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:44.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.164+0000 7f14ccca3640 1 --2- 172.21.15.86:0/3489805492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55743d0ab4d0 0x55743d0b4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.164+0000 7f14ccca3640 1 -- 172.21.15.86:0/3489805492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55743d0abe80 msgr2=0x55743d0b16a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:44.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.164+0000 7f14ccca3640 1 --2- 172.21.15.86:0/3489805492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55743d0abe80 0x55743d0b16a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.164+0000 7f14ccca3640 1 -- 172.21.15.86:0/3489805492 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55743cd223f0 con 0x55743d0b4a00 2024-04-21T08:43:44.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.164+0000 7f14c77fe640 1 --2- 172.21.15.86:0/3489805492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55743d0abe80 0x55743d0b16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:44.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.164+0000 7f14ccca3640 1 --2- 172.21.15.86:0/3489805492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743d0b4a00 0x55743d0b1c40 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f14c0098ec0 tx=0x7f14c00a0f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=615bfc2bb6a2d32f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:44.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.164+0000 7f14a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:44.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.164+0000 7f14a77fe640 1 -- 172.21.15.86:0/3489805492 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c001f070 con 0x55743d0b4a00 2024-04-21T08:43:44.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.164+0000 7f14a77fe640 1 -- 172.21.15.86:0/3489805492 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f14c00ce490 con 0x55743d0b4a00 2024-04-21T08:43:44.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.164+0000 7f14d13cb740 1 -- 172.21.15.86:0/3489805492 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14b80054e0 con 0x55743d0b4a00 2024-04-21T08:43:44.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.164+0000 7f14a77fe640 1 -- 172.21.15.86:0/3489805492 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f14c00ce690 con 0x55743d0b4a00 2024-04-21T08:43:44.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.164+0000 7f14d13cb740 10 client.17838 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:44.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.164+0000 7f14d13cb740 1 -- 172.21.15.86:0/3489805492 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55743cdec7d0 con 0x55743d0b4a00 2024-04-21T08:43:44.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.165+0000 7f14a77fe640 1 -- 172.21.15.86:0/3489805492 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14b80054e0 con 0x55743d0b4a00 2024-04-21T08:43:44.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.165+0000 7f14a77fe640 1 -- 172.21.15.86:0/3489805492 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f14c000f040 con 0x55743d0b4a00 2024-04-21T08:43:44.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.165+0000 7f14d13cb740 10 client.17838 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:44.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.165+0000 7f14d13cb740 10 client.17838 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:44.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.165+0000 7f14d13cb740 20 client.17838 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:44.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.165+0000 7f14d13cb740 1 --2- 172.21.15.86:0/3489805492 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55743d0c4b60 0x55743d0c6f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.165+0000 7f14d13cb740 4 client.17838 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:44.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.165+0000 7f14d13cb740 1 -- 172.21.15.86:0/3489805492 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55743cfaf930 con 0x55743d0c4b60 2024-04-21T08:43:44.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.165+0000 7f14c7fff640 1 --2- 172.21.15.86:0/3489805492 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55743d0c4b60 0x55743d0c6f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.166+0000 7f14c7fff640 1 --2- 172.21.15.86:0/3489805492 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55743d0c4b60 0x55743d0c6f70 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80f4267536434ca3 server_cookie=da87445a6317824e in_seq=0 out_seq=0 2024-04-21T08:43:44.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.166+0000 7f14a77fe640 10 client.17838 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:44.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.166+0000 7f14a77fe640 1 -- 172.21.15.86:0/3489805492 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55743cfaf930 con 0x55743d0c4b60 2024-04-21T08:43:44.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.166+0000 7f14a77fe640 10 client.17838 handle_command_reply: tid=0 2024-04-21T08:43:44.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.166+0000 7f14d13cb740 1 client.17838 shutdown 2024-04-21T08:43:44.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.166+0000 7f14d13cb740 1 -- 172.21.15.86:0/3489805492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743d0b4a00 msgr2=0x55743d0b1c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:44.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.166+0000 7f14d13cb740 1 --2- 172.21.15.86:0/3489805492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743d0b4a00 0x55743d0b1c40 secure :-1 s=READY pgs=1222 cs=0 l=1 rev1=1 crypto rx=0x7f14c0098ec0 tx=0x7f14c00a0f50 comp rx=0 tx=0).stop 2024-04-21T08:43:44.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.166+0000 7f14d13cb740 1 -- 172.21.15.86:0/3489805492 shutdown_connections 2024-04-21T08:43:44.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.166+0000 7f14d13cb740 1 --2- 172.21.15.86:0/3489805492 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55743d0c4b60 0x55743d0c6f70 crc :-1 s=READY pgs=169 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.166+0000 7f14d13cb740 1 --2- 172.21.15.86:0/3489805492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743d0b4a00 0x55743d0b1c40 unknown :-1 s=CLOSED pgs=1222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.166+0000 7f14d13cb740 1 --2- 172.21.15.86:0/3489805492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55743d0abe80 0x55743d0b16a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.166+0000 7f14a77fe640 0 client.17838 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:44.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.166+0000 7f14d13cb740 1 --2- 172.21.15.86:0/3489805492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55743d0ab4d0 0x55743d0b4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.166+0000 7f14d13cb740 1 -- 172.21.15.86:0/3489805492 >> 172.21.15.86:0/3489805492 conn(0x55743d0a5ba0 msgr2=0x55743d0a78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:44.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.167+0000 7f14d13cb740 1 -- 172.21.15.86:0/3489805492 shutdown_connections 2024-04-21T08:43:44.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.167+0000 7f14d13cb740 1 -- 172.21.15.86:0/3489805492 wait complete. 2024-04-21T08:43:44.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.167+0000 7f14d13cb740 20 client.17838 trim_cache size 0 max 16384 2024-04-21T08:43:44.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.180+0000 7f14d13cb740 1 Processor -- start 2024-04-21T08:43:44.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.180+0000 7f14d13cb740 1 -- start start 2024-04-21T08:43:44.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.180+0000 7f14d13cb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743d0ab4d0 0x55743d0c0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.180+0000 7f14d13cb740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55743d0abe80 0x55743d0b3dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.180+0000 7f14d13cb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55743d0b4a00 0x55743d0b4310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.180+0000 7f14d13cb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14c81af2d0 con 0x55743d0ab4d0 2024-04-21T08:43:44.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.180+0000 7f14d13cb740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f14c81b0e10 con 0x55743d0abe80 2024-04-21T08:43:44.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.180+0000 7f14d13cb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f14c81af9a0 con 0x55743d0b4a00 2024-04-21T08:43:44.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.180+0000 7f14c7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743d0ab4d0 0x55743d0c0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.180+0000 7f14c7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743d0ab4d0 0x55743d0c0830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37774/0 (socket says 172.21.15.86:37774) 2024-04-21T08:43:44.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.180+0000 7f14c7fff640 1 -- 172.21.15.86:0/3440659532 learned_addr learned my addr 172.21.15.86:0/3440659532 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:44.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.180+0000 7f14ccca3640 1 --2- 172.21.15.86:0/3440659532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55743d0b4a00 0x55743d0b4310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.180+0000 7f14c77fe640 1 --2- 172.21.15.86:0/3440659532 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55743d0abe80 0x55743d0b3dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.180+0000 7f14c7fff640 1 -- 172.21.15.86:0/3440659532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55743d0b4a00 msgr2=0x55743d0b4310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:44.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.180+0000 7f14c7fff640 1 --2- 172.21.15.86:0/3440659532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55743d0b4a00 0x55743d0b4310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.180+0000 7f14c7fff640 1 -- 172.21.15.86:0/3440659532 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55743d0abe80 msgr2=0x55743d0b3dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:44.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.180+0000 7f14c7fff640 1 --2- 172.21.15.86:0/3440659532 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55743d0abe80 0x55743d0b3dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.180+0000 7f14c7fff640 1 -- 172.21.15.86:0/3440659532 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55743d0c13a0 con 0x55743d0ab4d0 2024-04-21T08:43:44.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.180+0000 7f14c7fff640 1 --2- 172.21.15.86:0/3440659532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743d0ab4d0 0x55743d0c0830 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x55743d0c7780 tx=0x7f14bc005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43edb0c4442ab0ee server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:44.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.180+0000 7f14adffb640 1 -- 172.21.15.86:0/3440659532 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14bc01e050 con 0x55743d0ab4d0 2024-04-21T08:43:44.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.180+0000 7f14adffb640 1 -- 172.21.15.86:0/3440659532 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14bc01e6a0 con 0x55743d0ab4d0 2024-04-21T08:43:44.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.180+0000 7f14adffb640 1 -- 172.21.15.86:0/3440659532 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14bc012bb0 con 0x55743d0ab4d0 2024-04-21T08:43:44.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.181+0000 7f14d13cb740 1 -- 172.21.15.86:0/3440659532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743d0ab4d0 msgr2=0x55743d0c0830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:44.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.181+0000 7f14d13cb740 1 --2- 172.21.15.86:0/3440659532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743d0ab4d0 0x55743d0c0830 secure :-1 s=READY pgs=1223 cs=0 l=1 rev1=1 crypto rx=0x55743d0c7780 tx=0x7f14bc005900 comp rx=0 tx=0).stop 2024-04-21T08:43:44.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.181+0000 7f14d13cb740 1 -- 172.21.15.86:0/3440659532 shutdown_connections 2024-04-21T08:43:44.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.181+0000 7f14d13cb740 1 --2- 172.21.15.86:0/3440659532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55743d0b4a00 0x55743d0b4310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.181+0000 7f14d13cb740 1 --2- 172.21.15.86:0/3440659532 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55743d0abe80 0x55743d0b3dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.181+0000 7f14d13cb740 1 --2- 172.21.15.86:0/3440659532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743d0ab4d0 0x55743d0c0830 unknown :-1 s=CLOSED pgs=1223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.181+0000 7f14d13cb740 1 -- 172.21.15.86:0/3440659532 >> 172.21.15.86:0/3440659532 conn(0x55743d0a5ba0 msgr2=0x55743d0a4c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:44.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.181+0000 7f14d13cb740 1 -- 172.21.15.86:0/3440659532 shutdown_connections 2024-04-21T08:43:44.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.181+0000 7f14d13cb740 1 -- 172.21.15.86:0/3440659532 wait complete. 2024-04-21T08:43:44.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.181+0000 7f14d13cb740 1 -- start start 2024-04-21T08:43:44.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.182+0000 7f14d13cb740 1 Processor -- start 2024-04-21T08:43:44.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.182+0000 7f14d13cb740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55743d0ab4d0 0x55743d0b87c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.182+0000 7f14d13cb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55743d0abe80 0x55743d0b8d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.182+0000 7f14d13cb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743d0b4a00 0x55743d0b9240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.182+0000 7f14d13cb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14c81b0740 con 0x55743d0b4a00 2024-04-21T08:43:44.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.182+0000 7f14d13cb740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f14c81b0070 con 0x55743d0ab4d0 2024-04-21T08:43:44.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.182+0000 7f14d13cb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f14c81ad290 con 0x55743d0abe80 2024-04-21T08:43:44.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.182+0000 7f14d13cb740 10 client.0 _ll_register_callbacks cb 0x7f14c009d210 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:44.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.182+0000 7f14ccca3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743d0b4a00 0x55743d0b9240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.182+0000 7f14ccca3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743d0b4a00 0x55743d0b9240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37778/0 (socket says 172.21.15.86:37778) 2024-04-21T08:43:44.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.182+0000 7f14ccca3640 1 -- 172.21.15.86:0/3753755218 learned_addr learned my addr 172.21.15.86:0/3753755218 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:44.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.182+0000 7f14c7fff640 1 --2- 172.21.15.86:0/3753755218 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55743d0ab4d0 0x55743d0b87c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.182+0000 7f14c77fe640 1 --2- 172.21.15.86:0/3753755218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55743d0abe80 0x55743d0b8d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.182+0000 7f14ccca3640 1 -- 172.21.15.86:0/3753755218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55743d0abe80 msgr2=0x55743d0b8d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:44.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.182+0000 7f14ccca3640 1 --2- 172.21.15.86:0/3753755218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55743d0abe80 0x55743d0b8d00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.182+0000 7f14ccca3640 1 -- 172.21.15.86:0/3753755218 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55743d0ab4d0 msgr2=0x55743d0b87c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:44.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.182+0000 7f14ccca3640 1 --2- 172.21.15.86:0/3753755218 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55743d0ab4d0 0x55743d0b87c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.182+0000 7f14ccca3640 1 -- 172.21.15.86:0/3753755218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55743d12a150 con 0x55743d0b4a00 2024-04-21T08:43:44.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.182+0000 7f14ccca3640 1 --2- 172.21.15.86:0/3753755218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743d0b4a00 0x55743d0b9240 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f14c009b970 tx=0x7f14c00ce6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e05fd73766d9132f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:44.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.182+0000 7f14a77fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:44.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.182+0000 7f14a77fe640 1 -- 172.21.15.86:0/3753755218 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14c009c3c0 con 0x55743d0b4a00 2024-04-21T08:43:44.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.182+0000 7f14a77fe640 1 -- 172.21.15.86:0/3753755218 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f14c009bbf0 con 0x55743d0b4a00 2024-04-21T08:43:44.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.182+0000 7f14d13cb740 1 -- 172.21.15.86:0/3753755218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14bc01e6a0 con 0x55743d0b4a00 2024-04-21T08:43:44.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.183+0000 7f14a77fe640 1 -- 172.21.15.86:0/3753755218 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f14c0087440 con 0x55743d0b4a00 2024-04-21T08:43:44.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.183+0000 7f14d13cb740 10 client.17844 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:44.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.183+0000 7f14d13cb740 1 -- 172.21.15.86:0/3753755218 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55743d12b450 con 0x55743d0b4a00 2024-04-21T08:43:44.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.183+0000 7f14a77fe640 1 -- 172.21.15.86:0/3753755218 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14bc01e6a0 con 0x55743d0b4a00 2024-04-21T08:43:44.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.183+0000 7f14a77fe640 1 -- 172.21.15.86:0/3753755218 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f14c009bfd0 con 0x55743d0b4a00 2024-04-21T08:43:44.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.183+0000 7f14d13cb740 10 client.17844 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:44.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.183+0000 7f14d13cb740 10 client.17844 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:44.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.183+0000 7f14d13cb740 20 client.17844 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:44.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.183+0000 7f14d13cb740 1 --2- 172.21.15.86:0/3753755218 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55743d12ba00 0x55743d12de20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:44.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.183+0000 7f14d13cb740 4 client.17844 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:43:44.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.183+0000 7f14d13cb740 1 -- 172.21.15.86:0/3753755218 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55743d01dc10 con 0x55743d12ba00 2024-04-21T08:43:44.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.183+0000 7f14c7fff640 1 --2- 172.21.15.86:0/3753755218 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55743d12ba00 0x55743d12de20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:44.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.184+0000 7f14a77fe640 10 client.17844 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:44.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.184+0000 7f14c7fff640 1 --2- 172.21.15.86:0/3753755218 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55743d12ba00 0x55743d12de20 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=144799c4806256c1 server_cookie=b3fca6b2d0588e8e in_seq=0 out_seq=0 2024-04-21T08:43:44.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.184+0000 7f14a77fe640 1 -- 172.21.15.86:0/3753755218 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55743d01dc10 con 0x55743d12ba00 2024-04-21T08:43:44.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.184+0000 7f14a77fe640 10 client.17844 handle_command_reply: tid=0 2024-04-21T08:43:44.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.184+0000 7f14d13cb740 1 client.17844 shutdown 2024-04-21T08:43:44.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.184+0000 7f14d13cb740 1 -- 172.21.15.86:0/3753755218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743d0b4a00 msgr2=0x55743d0b9240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:44.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.184+0000 7f14d13cb740 1 --2- 172.21.15.86:0/3753755218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743d0b4a00 0x55743d0b9240 secure :-1 s=READY pgs=1224 cs=0 l=1 rev1=1 crypto rx=0x7f14c009b970 tx=0x7f14c00ce6c0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.184+0000 7f14d13cb740 1 -- 172.21.15.86:0/3753755218 shutdown_connections 2024-04-21T08:43:44.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.184+0000 7f14d13cb740 1 --2- 172.21.15.86:0/3753755218 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55743d12ba00 0x55743d12de20 crc :-1 s=READY pgs=170 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.184+0000 7f14d13cb740 1 --2- 172.21.15.86:0/3753755218 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55743d0b4a00 0x55743d0b9240 unknown :-1 s=CLOSED pgs=1224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.184+0000 7f14d13cb740 1 --2- 172.21.15.86:0/3753755218 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55743d0abe80 0x55743d0b8d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.184+0000 7f14a77fe640 0 client.17844 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:44.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.184+0000 7f14d13cb740 1 --2- 172.21.15.86:0/3753755218 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55743d0ab4d0 0x55743d0b87c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.184+0000 7f14d13cb740 1 -- 172.21.15.86:0/3753755218 >> 172.21.15.86:0/3753755218 conn(0x55743d0a5ba0 msgr2=0x55743d0b0f00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:44.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.184+0000 7f14d13cb740 1 -- 172.21.15.86:0/3753755218 shutdown_connections 2024-04-21T08:43:44.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.184+0000 7f14d13cb740 1 -- 172.21.15.86:0/3753755218 wait complete. 2024-04-21T08:43:44.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.184+0000 7f14d13cb740 20 client.17844 trim_cache size 0 max 16384 2024-04-21T08:43:44.192 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:43:44.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.185+0000 7f14ce706640 1 -- 172.21.15.86:0/3538057999 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f14b00a3d20 msgr2=0x7f14b00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:44.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.185+0000 7f14ce706640 1 --2- 172.21.15.86:0/3538057999 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f14b00a3d20 0x7f14b00a61e0 secure :-1 s=READY pgs=191 cs=0 l=1 rev1=1 crypto rx=0x7f14b802c780 tx=0x7f14b802c650 comp rx=0 tx=0).stop 2024-04-21T08:43:44.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.185+0000 7f14ce706640 1 -- 172.21.15.86:0/3538057999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14c8106f90 msgr2=0x7f14c81985b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:44.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.185+0000 7f14ce706640 1 --2- 172.21.15.86:0/3538057999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14c8106f90 0x7f14c81985b0 secure :-1 s=READY pgs=1220 cs=0 l=1 rev1=1 crypto rx=0x7f14c000eaa0 tx=0x7f14c000ef60 comp rx=0 tx=0).stop 2024-04-21T08:43:44.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.185+0000 7f14ce706640 1 -- 172.21.15.86:0/3538057999 shutdown_connections 2024-04-21T08:43:44.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.185+0000 7f14ce706640 1 --2- 172.21.15.86:0/3538057999 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f14b00a3d20 0x7f14b00a61e0 unknown :-1 s=CLOSED pgs=191 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.185+0000 7f14ce706640 1 --2- 172.21.15.86:0/3538057999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f14c8106f90 0x7f14c81985b0 unknown :-1 s=CLOSED pgs=1220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.185+0000 7f14ce706640 1 --2- 172.21.15.86:0/3538057999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f14c806af30 0x7f14c819e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.185+0000 7f14ce706640 1 --2- 172.21.15.86:0/3538057999 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14c806a570 0x7f14c819e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:44.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.185+0000 7f14ce706640 1 -- 172.21.15.86:0/3538057999 >> 172.21.15.86:0/3538057999 conn(0x7f14c8077e30 msgr2=0x7f14c80feec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:44.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.185+0000 7f14ce706640 1 -- 172.21.15.86:0/3538057999 shutdown_connections 2024-04-21T08:43:44.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:44.185+0000 7f14ce706640 1 -- 172.21.15.86:0/3538057999 wait complete. 2024-04-21T08:43:44.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:44 smithi086 ceph-mon[27939]: pgmap v151: 129 pgs: 129 active+clean; 2.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 51 KiB/s wr, 22 op/s 2024-04-21T08:43:44.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:44 smithi088 ceph-mon[31259]: pgmap v151: 129 pgs: 129 active+clean; 2.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 51 KiB/s wr, 22 op/s 2024-04-21T08:43:44.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:44 smithi179 ceph-mon[31829]: pgmap v151: 129 pgs: 129 active+clean; 2.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 51 KiB/s wr, 22 op/s 2024-04-21T08:43:45.195 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:43:45.196 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:43:45.196 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:43:45.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.310+0000 7f8e7c069640 1 -- 172.21.15.86:0/713488903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e74101a40 msgr2=0x7f8e74105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:45.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.310+0000 7f8e7c069640 1 --2- 172.21.15.86:0/713488903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e74101a40 0x7f8e74105a80 secure :-1 s=READY pgs=1225 cs=0 l=1 rev1=1 crypto rx=0x7f8e64009900 tx=0x7f8e6402bc30 comp rx=0 tx=0).stop 2024-04-21T08:43:45.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.310+0000 7f8e7c069640 1 -- 172.21.15.86:0/713488903 shutdown_connections 2024-04-21T08:43:45.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.310+0000 7f8e7c069640 1 --2- 172.21.15.86:0/713488903 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8e74106160 0x7f8e7410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.310+0000 7f8e7c069640 1 --2- 172.21.15.86:0/713488903 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e74101a40 0x7f8e74105a80 unknown :-1 s=CLOSED pgs=1225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.310+0000 7f8e7c069640 1 --2- 172.21.15.86:0/713488903 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e74101090 0x7f8e74101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.310+0000 7f8e7c069640 1 -- 172.21.15.86:0/713488903 >> 172.21.15.86:0/713488903 conn(0x7f8e74077f50 msgr2=0x7f8e74078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:45.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.310+0000 7f8e7c069640 1 -- 172.21.15.86:0/713488903 shutdown_connections 2024-04-21T08:43:45.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.310+0000 7f8e7c069640 1 -- 172.21.15.86:0/713488903 wait complete. 2024-04-21T08:43:45.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.311+0000 7f8e7c069640 1 Processor -- start 2024-04-21T08:43:45.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.311+0000 7f8e7c069640 1 -- start start 2024-04-21T08:43:45.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.311+0000 7f8e7c069640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8e74101090 0x7f8e74197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:45.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.311+0000 7f8e7c069640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e74106160 0x7f8e74197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:45.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.311+0000 7f8e7c069640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e7419bf20 0x7f8e7419c3d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:45.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.311+0000 7f8e7c069640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8e741110a0 con 0x7f8e74106160 2024-04-21T08:43:45.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.311+0000 7f8e7c069640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8e74110f20 con 0x7f8e74101090 2024-04-21T08:43:45.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.311+0000 7f8e7c069640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8e74111220 con 0x7f8e7419bf20 2024-04-21T08:43:45.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.311+0000 7f8e79604640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e74106160 0x7f8e74197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:45.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.311+0000 7f8e79604640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e74106160 0x7f8e74197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37792/0 (socket says 172.21.15.86:37792) 2024-04-21T08:43:45.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.311+0000 7f8e79604640 1 -- 172.21.15.86:0/2409986186 learned_addr learned my addr 172.21.15.86:0/2409986186 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:45.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.311+0000 7f8e79e05640 1 --2- 172.21.15.86:0/2409986186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8e74101090 0x7f8e74197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:45.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.311+0000 7f8e7a606640 1 --2- 172.21.15.86:0/2409986186 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e7419bf20 0x7f8e7419c3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:45.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.311+0000 7f8e79604640 1 -- 172.21.15.86:0/2409986186 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e7419bf20 msgr2=0x7f8e7419c3d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:45.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.311+0000 7f8e79604640 1 --2- 172.21.15.86:0/2409986186 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e7419bf20 0x7f8e7419c3d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.311+0000 7f8e79604640 1 -- 172.21.15.86:0/2409986186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8e74101090 msgr2=0x7f8e74197660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:45.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.311+0000 7f8e79604640 1 --2- 172.21.15.86:0/2409986186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8e74101090 0x7f8e74197660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.311+0000 7f8e79604640 1 -- 172.21.15.86:0/2409986186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e7419ca10 con 0x7f8e74106160 2024-04-21T08:43:45.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.311+0000 7f8e79604640 1 --2- 172.21.15.86:0/2409986186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e74106160 0x7f8e74197ba0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f8e64036040 tx=0x7f8e640045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:45.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.312+0000 7f8e6affd640 1 -- 172.21.15.86:0/2409986186 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e64043020 con 0x7f8e74106160 2024-04-21T08:43:45.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.312+0000 7f8e6affd640 1 -- 172.21.15.86:0/2409986186 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e6402ca70 con 0x7f8e74106160 2024-04-21T08:43:45.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.312+0000 7f8e7c069640 1 -- 172.21.15.86:0/2409986186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e7419cca0 con 0x7f8e74106160 2024-04-21T08:43:45.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.312+0000 7f8e6affd640 1 -- 172.21.15.86:0/2409986186 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e64003ac0 con 0x7f8e74106160 2024-04-21T08:43:45.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.312+0000 7f8e7c069640 1 -- 172.21.15.86:0/2409986186 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e741a44c0 con 0x7f8e74106160 2024-04-21T08:43:45.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.314+0000 7f8e6affd640 1 -- 172.21.15.86:0/2409986186 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8e64046020 con 0x7f8e74106160 2024-04-21T08:43:45.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.314+0000 7f8e7ec56740 1 Processor -- start 2024-04-21T08:43:45.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.314+0000 7f8e7ec56740 1 -- start start 2024-04-21T08:43:45.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.314+0000 7f8e6affd640 1 --2- 172.21.15.86:0/2409986186 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8e600a3d20 0x7f8e600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:45.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.314+0000 7f8e7ec56740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558706ada780 0x558706adab60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:45.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.314+0000 7f8e7ec56740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558706adb130 0x558706ae5a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:45.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.315+0000 7f8e7ec56740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558706ae6020 0x558706ae8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:45.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.315+0000 7f8e7ec56740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558706a3ef00 con 0x558706ada780 2024-04-21T08:43:45.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.315+0000 7f8e7ec56740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5587069dc590 con 0x558706adb130 2024-04-21T08:43:45.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.315+0000 7f8e7ec56740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558706a3aec0 con 0x558706ae6020 2024-04-21T08:43:45.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.315+0000 7f8e79e05640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558706ada780 0x558706adab60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:45.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.315+0000 7f8e79e05640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558706ada780 0x558706adab60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37806/0 (socket says 172.21.15.86:37806) 2024-04-21T08:43:45.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.315+0000 7f8e7a606640 1 --2- 172.21.15.86:0/2409986186 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8e600a3d20 0x7f8e600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:45.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.315+0000 7f8e79e05640 1 -- 172.21.15.86:0/1902589360 learned_addr learned my addr 172.21.15.86:0/1902589360 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:45.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.315+0000 7f8e6affd640 1 -- 172.21.15.86:0/2409986186 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f8e640f2b90 con 0x7f8e74106160 2024-04-21T08:43:45.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.315+0000 7f8e79604640 1 --2- 172.21.15.86:0/1902589360 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558706adb130 0x558706ae5a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:45.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.315+0000 7f8e79e05640 1 -- 172.21.15.86:0/1902589360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558706ae6020 msgr2=0x558706ae8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:45.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.315+0000 7f8e79e05640 1 --2- 172.21.15.86:0/1902589360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558706ae6020 0x558706ae8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.315+0000 7f8e79e05640 1 -- 172.21.15.86:0/1902589360 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558706adb130 msgr2=0x558706ae5a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:45.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.315+0000 7f8e79e05640 1 --2- 172.21.15.86:0/1902589360 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558706adb130 0x558706ae5a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.315+0000 7f8e79e05640 1 -- 172.21.15.86:0/1902589360 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5587068197e0 con 0x558706ada780 2024-04-21T08:43:45.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.315+0000 7f8e7a606640 1 --2- 172.21.15.86:0/2409986186 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8e600a3d20 0x7f8e600a61e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c009c10 tx=0x7f8e6c009750 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:45.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.315+0000 7f8e79e05640 1 --2- 172.21.15.86:0/1902589360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558706ada780 0x558706adab60 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f8e700114f0 tx=0x7f8e700119b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43c5e6dfdc5a6738 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:45.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.315+0000 7f8e5ffff640 1 -- 172.21.15.86:0/1902589360 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e70004d20 con 0x558706ada780 2024-04-21T08:43:45.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.315+0000 7f8e5ffff640 1 -- 172.21.15.86:0/1902589360 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e70004050 con 0x558706ada780 2024-04-21T08:43:45.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.315+0000 7f8e5ffff640 1 -- 172.21.15.86:0/1902589360 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e70005280 con 0x558706ada780 2024-04-21T08:43:45.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.316+0000 7f8e7ec56740 1 -- 172.21.15.86:0/1902589360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558706ada780 msgr2=0x558706adab60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:45.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.316+0000 7f8e7ec56740 1 --2- 172.21.15.86:0/1902589360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558706ada780 0x558706adab60 secure :-1 s=READY pgs=1227 cs=0 l=1 rev1=1 crypto rx=0x7f8e700114f0 tx=0x7f8e700119b0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.316+0000 7f8e7ec56740 1 -- 172.21.15.86:0/1902589360 shutdown_connections 2024-04-21T08:43:45.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.316+0000 7f8e7ec56740 1 --2- 172.21.15.86:0/1902589360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558706ae6020 0x558706ae8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.316+0000 7f8e7ec56740 1 --2- 172.21.15.86:0/1902589360 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558706adb130 0x558706ae5a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.316+0000 7f8e7ec56740 1 --2- 172.21.15.86:0/1902589360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558706ada780 0x558706adab60 unknown :-1 s=CLOSED pgs=1227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.316+0000 7f8e7ec56740 1 -- 172.21.15.86:0/1902589360 >> 172.21.15.86:0/1902589360 conn(0x558706ad4fb0 msgr2=0x558706ad5390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:45.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.316+0000 7f8e7ec56740 1 -- 172.21.15.86:0/1902589360 shutdown_connections 2024-04-21T08:43:45.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.316+0000 7f8e7ec56740 1 -- 172.21.15.86:0/1902589360 wait complete. 2024-04-21T08:43:45.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.317+0000 7f8e7ec56740 1 -- start start 2024-04-21T08:43:45.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.317+0000 7f8e7ec56740 1 Processor -- start 2024-04-21T08:43:45.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.317+0000 7f8e7ec56740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558706ada780 0x558706ae2f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:45.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.317+0000 7f8e7ec56740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558706adb130 0x558706ae3460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:45.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.317+0000 7f8e7ec56740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558706ae6020 0x558706ae39a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:45.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.317+0000 7f8e7ec56740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558706ae8f50 con 0x558706ada780 2024-04-21T08:43:45.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.317+0000 7f8e7ec56740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5587069f8580 con 0x558706adb130 2024-04-21T08:43:45.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.317+0000 7f8e7ec56740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558706ae90d0 con 0x558706ae6020 2024-04-21T08:43:45.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.317+0000 7f8e7ec56740 10 client.0 _ll_register_callbacks cb 0x5587069fe650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:45.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.317+0000 7f8e79e05640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558706ada780 0x558706ae2f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:45.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.317+0000 7f8e79e05640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558706ada780 0x558706ae2f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37810/0 (socket says 172.21.15.86:37810) 2024-04-21T08:43:45.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.317+0000 7f8e79e05640 1 -- 172.21.15.86:0/1839172475 learned_addr learned my addr 172.21.15.86:0/1839172475 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:45.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.317+0000 7f8e79604640 1 --2- 172.21.15.86:0/1839172475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558706adb130 0x558706ae3460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:45.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.317+0000 7f8e7a606640 1 --2- 172.21.15.86:0/1839172475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558706ae6020 0x558706ae39a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:45.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.317+0000 7f8e79e05640 1 -- 172.21.15.86:0/1839172475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558706ae6020 msgr2=0x558706ae39a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:45.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.317+0000 7f8e79e05640 1 --2- 172.21.15.86:0/1839172475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558706ae6020 0x558706ae39a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.317+0000 7f8e79e05640 1 -- 172.21.15.86:0/1839172475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558706adb130 msgr2=0x558706ae3460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:45.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.317+0000 7f8e79e05640 1 --2- 172.21.15.86:0/1839172475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558706adb130 0x558706ae3460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.317+0000 7f8e79e05640 1 -- 172.21.15.86:0/1839172475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558706879f70 con 0x558706ada780 2024-04-21T08:43:45.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.317+0000 7f8e79604640 1 --2- 172.21.15.86:0/1839172475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558706adb130 0x558706ae3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:45.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.317+0000 7f8e7a606640 1 --2- 172.21.15.86:0/1839172475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558706ae6020 0x558706ae39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:45.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.318+0000 7f8e79e05640 1 --2- 172.21.15.86:0/1839172475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558706ada780 0x558706ae2f20 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f8e70004950 tx=0x7f8e70004cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b5852c619e5ebc5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:45.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.318+0000 7f8e5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:45.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.318+0000 7f8e5d7fa640 1 -- 172.21.15.86:0/1839172475 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e70028020 con 0x558706ada780 2024-04-21T08:43:45.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.318+0000 7f8e5d7fa640 1 -- 172.21.15.86:0/1839172475 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f8e70004410 con 0x558706ada780 2024-04-21T08:43:45.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.318+0000 7f8e7ec56740 1 -- 172.21.15.86:0/1839172475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e70004050 con 0x558706ada780 2024-04-21T08:43:45.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.318+0000 7f8e5d7fa640 1 -- 172.21.15.86:0/1839172475 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8e70033050 con 0x558706ada780 2024-04-21T08:43:45.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.318+0000 7f8e7ec56740 10 client.17856 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:45.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.318+0000 7f8e7ec56740 1 -- 172.21.15.86:0/1839172475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558706711170 con 0x558706ada780 2024-04-21T08:43:45.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.318+0000 7f8e5d7fa640 1 -- 172.21.15.86:0/1839172475 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e70004050 con 0x558706ada780 2024-04-21T08:43:45.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.318+0000 7f8e5d7fa640 1 -- 172.21.15.86:0/1839172475 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8e7000f4e0 con 0x558706ada780 2024-04-21T08:43:45.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.318+0000 7f8e7ec56740 10 client.17856 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:45.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.318+0000 7f8e7ec56740 10 client.17856 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:45.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.318+0000 7f8e7ec56740 20 client.17856 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:45.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.318+0000 7f8e7ec56740 1 --2- 172.21.15.86:0/1839172475 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x558706af6420 0x558706af8810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:45.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.318+0000 7f8e7ec56740 4 client.17856 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:45.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.318+0000 7f8e7ec56740 1 -- 172.21.15.86:0/1839172475 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558706a4c920 con 0x558706af6420 2024-04-21T08:43:45.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.319+0000 7f8e79604640 1 --2- 172.21.15.86:0/1839172475 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x558706af6420 0x558706af8810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:45.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.319+0000 7f8e5d7fa640 10 client.17856 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:45.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.319+0000 7f8e79604640 1 --2- 172.21.15.86:0/1839172475 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x558706af6420 0x558706af8810 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b8874d30139eaa58 server_cookie=65fd0868f673da36 in_seq=0 out_seq=0 2024-04-21T08:43:45.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.320+0000 7f8e5d7fa640 1 -- 172.21.15.86:0/1839172475 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558706a4c920 con 0x558706af6420 2024-04-21T08:43:45.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.320+0000 7f8e5d7fa640 10 client.17856 handle_command_reply: tid=0 2024-04-21T08:43:45.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.320+0000 7f8e7ec56740 1 client.17856 shutdown 2024-04-21T08:43:45.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.320+0000 7f8e7ec56740 1 -- 172.21.15.86:0/1839172475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558706ada780 msgr2=0x558706ae2f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:45.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.320+0000 7f8e7ec56740 1 --2- 172.21.15.86:0/1839172475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558706ada780 0x558706ae2f20 secure :-1 s=READY pgs=1228 cs=0 l=1 rev1=1 crypto rx=0x7f8e70004950 tx=0x7f8e70004cf0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.320+0000 7f8e7ec56740 1 -- 172.21.15.86:0/1839172475 shutdown_connections 2024-04-21T08:43:45.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.320+0000 7f8e7ec56740 1 --2- 172.21.15.86:0/1839172475 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x558706af6420 0x558706af8810 crc :-1 s=READY pgs=171 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.320+0000 7f8e7ec56740 1 --2- 172.21.15.86:0/1839172475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558706ae6020 0x558706ae39a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.320+0000 7f8e7ec56740 1 --2- 172.21.15.86:0/1839172475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558706adb130 0x558706ae3460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.320+0000 7f8e5d7fa640 0 client.17856 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:45.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.320+0000 7f8e7ec56740 1 --2- 172.21.15.86:0/1839172475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558706ada780 0x558706ae2f20 unknown :-1 s=CLOSED pgs=1228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.320+0000 7f8e7ec56740 1 -- 172.21.15.86:0/1839172475 >> 172.21.15.86:0/1839172475 conn(0x558706ad4fb0 msgr2=0x558706ad6d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:45.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.320+0000 7f8e7ec56740 1 -- 172.21.15.86:0/1839172475 shutdown_connections 2024-04-21T08:43:45.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.320+0000 7f8e7ec56740 1 -- 172.21.15.86:0/1839172475 wait complete. 2024-04-21T08:43:45.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.320+0000 7f8e7ec56740 20 client.17856 trim_cache size 0 max 16384 2024-04-21T08:43:45.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.337+0000 7f8e7ec56740 1 Processor -- start 2024-04-21T08:43:45.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.338+0000 7f8e7ec56740 1 -- start start 2024-04-21T08:43:45.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.338+0000 7f8e7ec56740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558706ada780 0x558706af1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:45.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.338+0000 7f8e7ec56740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558706adb130 0x558706ad3df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:45.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.338+0000 7f8e7ec56740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558706ae6020 0x558706ae2800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:45.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.338+0000 7f8e7ec56740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8e741ad100 con 0x558706ada780 2024-04-21T08:43:45.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.338+0000 7f8e7ec56740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8e741aec40 con 0x558706ae6020 2024-04-21T08:43:45.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.338+0000 7f8e7ec56740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8e741ad7d0 con 0x558706adb130 2024-04-21T08:43:45.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.338+0000 7f8e79e05640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558706ada780 0x558706af1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:45.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.338+0000 7f8e79e05640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558706ada780 0x558706af1e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37816/0 (socket says 172.21.15.86:37816) 2024-04-21T08:43:45.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.338+0000 7f8e79e05640 1 -- 172.21.15.86:0/3941377091 learned_addr learned my addr 172.21.15.86:0/3941377091 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:45.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.338+0000 7f8e7a606640 1 --2- 172.21.15.86:0/3941377091 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558706ae6020 0x558706ae2800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:45.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.338+0000 7f8e79604640 1 --2- 172.21.15.86:0/3941377091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558706adb130 0x558706ad3df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:45.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.338+0000 7f8e79e05640 1 -- 172.21.15.86:0/3941377091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558706adb130 msgr2=0x558706ad3df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:45.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.338+0000 7f8e79e05640 1 --2- 172.21.15.86:0/3941377091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558706adb130 0x558706ad3df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.338+0000 7f8e79e05640 1 -- 172.21.15.86:0/3941377091 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558706ae6020 msgr2=0x558706ae2800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:45.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.338+0000 7f8e79e05640 1 --2- 172.21.15.86:0/3941377091 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558706ae6020 0x558706ae2800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.338+0000 7f8e79e05640 1 -- 172.21.15.86:0/3941377091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558706ad9970 con 0x558706ada780 2024-04-21T08:43:45.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.338+0000 7f8e7a606640 1 --2- 172.21.15.86:0/3941377091 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558706ae6020 0x558706ae2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:45.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.338+0000 7f8e79e05640 1 --2- 172.21.15.86:0/3941377091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558706ada780 0x558706af1e50 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f8e7000faf0 tx=0x7f8e7000fdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e405591ea643bf3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:45.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.338+0000 7f8e5f7fe640 1 -- 172.21.15.86:0/3941377091 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e70028040 con 0x558706ada780 2024-04-21T08:43:45.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.338+0000 7f8e5f7fe640 1 -- 172.21.15.86:0/3941377091 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e7000f6e0 con 0x558706ada780 2024-04-21T08:43:45.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.338+0000 7f8e5f7fe640 1 -- 172.21.15.86:0/3941377091 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e70037690 con 0x558706ada780 2024-04-21T08:43:45.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.339+0000 7f8e7ec56740 1 -- 172.21.15.86:0/3941377091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558706ada780 msgr2=0x558706af1e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:45.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.339+0000 7f8e7ec56740 1 --2- 172.21.15.86:0/3941377091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558706ada780 0x558706af1e50 secure :-1 s=READY pgs=1229 cs=0 l=1 rev1=1 crypto rx=0x7f8e7000faf0 tx=0x7f8e7000fdd0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.339+0000 7f8e7ec56740 1 -- 172.21.15.86:0/3941377091 shutdown_connections 2024-04-21T08:43:45.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.339+0000 7f8e7ec56740 1 --2- 172.21.15.86:0/3941377091 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558706ae6020 0x558706ae2800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.339+0000 7f8e7ec56740 1 --2- 172.21.15.86:0/3941377091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558706adb130 0x558706ad3df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.339+0000 7f8e7ec56740 1 --2- 172.21.15.86:0/3941377091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558706ada780 0x558706af1e50 unknown :-1 s=CLOSED pgs=1229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.339+0000 7f8e7ec56740 1 -- 172.21.15.86:0/3941377091 >> 172.21.15.86:0/3941377091 conn(0x558706ad4fb0 msgr2=0x558706ae3770 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:45.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.339+0000 7f8e7ec56740 1 -- 172.21.15.86:0/3941377091 shutdown_connections 2024-04-21T08:43:45.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.339+0000 7f8e7ec56740 1 -- 172.21.15.86:0/3941377091 wait complete. 2024-04-21T08:43:45.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.339+0000 7f8e7ec56740 1 -- start start 2024-04-21T08:43:45.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.339+0000 7f8e7ec56740 1 Processor -- start 2024-04-21T08:43:45.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.340+0000 7f8e7ec56740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558706ada780 0x558706ae9d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:45.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.340+0000 7f8e7ec56740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558706adb130 0x558706aea2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:45.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.340+0000 7f8e7ec56740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558706ae6020 0x558706aea7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:45.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.340+0000 7f8e7ec56740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8e741ae570 con 0x558706ada780 2024-04-21T08:43:45.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.340+0000 7f8e7ec56740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8e741adea0 con 0x558706ae6020 2024-04-21T08:43:45.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.340+0000 7f8e7ec56740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8e741ab0c0 con 0x558706adb130 2024-04-21T08:43:45.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.340+0000 7f8e7ec56740 10 client.0 _ll_register_callbacks cb 0x7f8e70009ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:45.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.340+0000 7f8e79e05640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558706ada780 0x558706ae9d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:45.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.340+0000 7f8e79e05640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558706ada780 0x558706ae9d60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37826/0 (socket says 172.21.15.86:37826) 2024-04-21T08:43:45.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.340+0000 7f8e79e05640 1 -- 172.21.15.86:0/816416014 learned_addr learned my addr 172.21.15.86:0/816416014 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:45.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.340+0000 7f8e79604640 1 --2- 172.21.15.86:0/816416014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558706adb130 0x558706aea2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:45.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.340+0000 7f8e79e05640 1 -- 172.21.15.86:0/816416014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558706adb130 msgr2=0x558706aea2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:45.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.340+0000 7f8e79e05640 1 --2- 172.21.15.86:0/816416014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558706adb130 0x558706aea2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.340+0000 7f8e79e05640 1 -- 172.21.15.86:0/816416014 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558706ae6020 msgr2=0x558706aea7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:45.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.340+0000 7f8e7a606640 1 --2- 172.21.15.86:0/816416014 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558706ae6020 0x558706aea7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:45.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.340+0000 7f8e79e05640 1 --2- 172.21.15.86:0/816416014 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558706ae6020 0x558706aea7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.340+0000 7f8e79e05640 1 -- 172.21.15.86:0/816416014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558706b5b880 con 0x558706ada780 2024-04-21T08:43:45.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.340+0000 7f8e7a606640 1 --2- 172.21.15.86:0/816416014 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558706ae6020 0x558706aea7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:45.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.340+0000 7f8e79e05640 1 --2- 172.21.15.86:0/816416014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558706ada780 0x558706ae9d60 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f8e70006010 tx=0x7f8e7000fba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c64326852c228d4d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:45.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.340+0000 7f8e5d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:45.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.340+0000 7f8e5d7fa640 1 -- 172.21.15.86:0/816416014 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e70003d40 con 0x558706ada780 2024-04-21T08:43:45.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.340+0000 7f8e5d7fa640 1 -- 172.21.15.86:0/816416014 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f8e70003ee0 con 0x558706ada780 2024-04-21T08:43:45.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.340+0000 7f8e7ec56740 1 -- 172.21.15.86:0/816416014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e7000f6e0 con 0x558706ada780 2024-04-21T08:43:45.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.341+0000 7f8e5d7fa640 1 -- 172.21.15.86:0/816416014 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8e70048050 con 0x558706ada780 2024-04-21T08:43:45.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.341+0000 7f8e7ec56740 10 client.17862 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:45.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.341+0000 7f8e7ec56740 1 -- 172.21.15.86:0/816416014 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558706b5cb00 con 0x558706ada780 2024-04-21T08:43:45.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.341+0000 7f8e5d7fa640 1 -- 172.21.15.86:0/816416014 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e7000f6e0 con 0x558706ada780 2024-04-21T08:43:45.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.341+0000 7f8e5d7fa640 1 -- 172.21.15.86:0/816416014 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8e7003f070 con 0x558706ada780 2024-04-21T08:43:45.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.341+0000 7f8e7ec56740 10 client.17862 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:45.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.341+0000 7f8e7ec56740 10 client.17862 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:45.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.341+0000 7f8e7ec56740 20 client.17862 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:45.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.341+0000 7f8e7ec56740 1 --2- 172.21.15.86:0/816416014 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x558706b5d110 0x558706b5f530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:45.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.341+0000 7f8e7ec56740 4 client.17862 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:43:45.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.341+0000 7f8e7ec56740 1 -- 172.21.15.86:0/816416014 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558706a46790 con 0x558706b5d110 2024-04-21T08:43:45.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.341+0000 7f8e79604640 1 --2- 172.21.15.86:0/816416014 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x558706b5d110 0x558706b5f530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:45.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.342+0000 7f8e79604640 1 --2- 172.21.15.86:0/816416014 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x558706b5d110 0x558706b5f530 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8912133d5f14606c server_cookie=970cfce6cf2db642 in_seq=0 out_seq=0 2024-04-21T08:43:45.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.342+0000 7f8e5d7fa640 10 client.17862 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:45.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.343+0000 7f8e5d7fa640 1 -- 172.21.15.86:0/816416014 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558706a46790 con 0x558706b5d110 2024-04-21T08:43:45.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.343+0000 7f8e5d7fa640 10 client.17862 handle_command_reply: tid=0 2024-04-21T08:43:45.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.343+0000 7f8e7ec56740 1 client.17862 shutdown 2024-04-21T08:43:45.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.343+0000 7f8e7ec56740 1 -- 172.21.15.86:0/816416014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558706ada780 msgr2=0x558706ae9d60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:45.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.343+0000 7f8e7ec56740 1 --2- 172.21.15.86:0/816416014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558706ada780 0x558706ae9d60 secure :-1 s=READY pgs=1230 cs=0 l=1 rev1=1 crypto rx=0x7f8e70006010 tx=0x7f8e7000fba0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.343+0000 7f8e7ec56740 1 -- 172.21.15.86:0/816416014 shutdown_connections 2024-04-21T08:43:45.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.343+0000 7f8e7ec56740 1 --2- 172.21.15.86:0/816416014 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x558706b5d110 0x558706b5f530 crc :-1 s=READY pgs=172 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.343+0000 7f8e5d7fa640 0 client.17862 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:45.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.343+0000 7f8e7ec56740 1 --2- 172.21.15.86:0/816416014 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558706ae6020 0x558706aea7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.343+0000 7f8e7ec56740 1 --2- 172.21.15.86:0/816416014 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558706adb130 0x558706aea2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.343+0000 7f8e7ec56740 1 --2- 172.21.15.86:0/816416014 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558706ada780 0x558706ae9d60 unknown :-1 s=CLOSED pgs=1230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.343+0000 7f8e7ec56740 1 -- 172.21.15.86:0/816416014 >> 172.21.15.86:0/816416014 conn(0x558706ad4fb0 msgr2=0x558706ad5fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:45.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.343+0000 7f8e7ec56740 1 -- 172.21.15.86:0/816416014 shutdown_connections 2024-04-21T08:43:45.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.343+0000 7f8e7ec56740 1 -- 172.21.15.86:0/816416014 wait complete. 2024-04-21T08:43:45.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.343+0000 7f8e7ec56740 20 client.17862 trim_cache size 0 max 16384 2024-04-21T08:43:45.353 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:43:45.353 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-21T08:43:45.353 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "18507780-feef-47e5-b927-9d3c22dfb928", 2024-04-21T08:43:45.354 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-21T08:43:45.354 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:43:45.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.344+0000 7f8e7c069640 1 -- 172.21.15.86:0/2409986186 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8e600a3d20 msgr2=0x7f8e600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:45.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.344+0000 7f8e7c069640 1 --2- 172.21.15.86:0/2409986186 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8e600a3d20 0x7f8e600a61e0 secure :-1 s=READY pgs=192 cs=0 l=1 rev1=1 crypto rx=0x7f8e6c009c10 tx=0x7f8e6c009750 comp rx=0 tx=0).stop 2024-04-21T08:43:45.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.344+0000 7f8e7c069640 1 -- 172.21.15.86:0/2409986186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e74106160 msgr2=0x7f8e74197ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:45.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.344+0000 7f8e7c069640 1 --2- 172.21.15.86:0/2409986186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e74106160 0x7f8e74197ba0 secure :-1 s=READY pgs=1226 cs=0 l=1 rev1=1 crypto rx=0x7f8e64036040 tx=0x7f8e640045e0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.344+0000 7f8e7c069640 1 -- 172.21.15.86:0/2409986186 shutdown_connections 2024-04-21T08:43:45.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.344+0000 7f8e7c069640 1 --2- 172.21.15.86:0/2409986186 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8e600a3d20 0x7f8e600a61e0 unknown :-1 s=CLOSED pgs=192 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.344+0000 7f8e7c069640 1 --2- 172.21.15.86:0/2409986186 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e7419bf20 0x7f8e7419c3d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.344+0000 7f8e7c069640 1 --2- 172.21.15.86:0/2409986186 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e74106160 0x7f8e74197ba0 unknown :-1 s=CLOSED pgs=1226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.344+0000 7f8e7c069640 1 --2- 172.21.15.86:0/2409986186 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8e74101090 0x7f8e74197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.344+0000 7f8e7c069640 1 -- 172.21.15.86:0/2409986186 >> 172.21.15.86:0/2409986186 conn(0x7f8e74077f50 msgr2=0x7f8e740ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:45.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.344+0000 7f8e7c069640 1 -- 172.21.15.86:0/2409986186 shutdown_connections 2024-04-21T08:43:45.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.344+0000 7f8e7c069640 1 -- 172.21.15.86:0/2409986186 wait complete. 2024-04-21T08:43:45.356 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:43:45.356 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:43:45.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.456+0000 7f1f0a69a640 1 -- 172.21.15.86:0/353327003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f04100f80 msgr2=0x7f1f0410d490 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:45.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.456+0000 7f1f0a69a640 1 --2- 172.21.15.86:0/353327003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f04100f80 0x7f1f0410d490 secure :-1 s=READY pgs=1231 cs=0 l=1 rev1=1 crypto rx=0x7f1efc009cd0 tx=0x7f1efc02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.456+0000 7f1f0a69a640 1 -- 172.21.15.86:0/353327003 shutdown_connections 2024-04-21T08:43:45.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.456+0000 7f1f0a69a640 1 --2- 172.21.15.86:0/353327003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f04100f80 0x7f1f0410d490 unknown :-1 s=CLOSED pgs=1231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.456+0000 7f1f0a69a640 1 --2- 172.21.15.86:0/353327003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f041005c0 0x7f1f04100a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.456+0000 7f1f0a69a640 1 --2- 172.21.15.86:0/353327003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f04106660 0x7f1f04106a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.456+0000 7f1f0a69a640 1 -- 172.21.15.86:0/353327003 >> 172.21.15.86:0/353327003 conn(0x7f1f040773a0 msgr2=0x7f1f040fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:45.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.456+0000 7f1f0a69a640 1 -- 172.21.15.86:0/353327003 shutdown_connections 2024-04-21T08:43:45.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.456+0000 7f1f0a69a640 1 -- 172.21.15.86:0/353327003 wait complete. 2024-04-21T08:43:45.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.457+0000 7f1f0a69a640 1 Processor -- start 2024-04-21T08:43:45.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.457+0000 7f1f0a69a640 1 -- start start 2024-04-21T08:43:45.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.457+0000 7f1f0a69a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f041005c0 0x7f1f041994d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:45.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.457+0000 7f1f0a69a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f04100f80 0x7f1f04199a10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:45.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.457+0000 7f1f0a69a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f04106660 0x7f1f041a4550 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:45.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.457+0000 7f1f0a69a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f04110890 con 0x7f1f041005c0 2024-04-21T08:43:45.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.457+0000 7f1f0a69a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1f04110710 con 0x7f1f04106660 2024-04-21T08:43:45.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.457+0000 7f1f0a69a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1f04110a10 con 0x7f1f04100f80 2024-04-21T08:43:45.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.457+0000 7f1f03fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f041005c0 0x7f1f041994d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:45.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.457+0000 7f1f03fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f041005c0 0x7f1f041994d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37836/0 (socket says 172.21.15.86:37836) 2024-04-21T08:43:45.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.457+0000 7f1f03fff640 1 -- 172.21.15.86:0/780709009 learned_addr learned my addr 172.21.15.86:0/780709009 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:45.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.457+0000 7f1f08c37640 1 --2- 172.21.15.86:0/780709009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f04106660 0x7f1f041a4550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:45.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.457+0000 7f1f03fff640 1 -- 172.21.15.86:0/780709009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f04100f80 msgr2=0x7f1f04199a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:45.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.457+0000 7f1f03fff640 1 --2- 172.21.15.86:0/780709009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f04100f80 0x7f1f04199a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.457+0000 7f1f03fff640 1 -- 172.21.15.86:0/780709009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f04106660 msgr2=0x7f1f041a4550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:45.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.457+0000 7f1f03fff640 1 --2- 172.21.15.86:0/780709009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f04106660 0x7f1f041a4550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.457+0000 7f1f03fff640 1 -- 172.21.15.86:0/780709009 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f041a4a90 con 0x7f1f041005c0 2024-04-21T08:43:45.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.457+0000 7f1f08c37640 1 --2- 172.21.15.86:0/780709009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f04106660 0x7f1f041a4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:45.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.458+0000 7f1f03fff640 1 --2- 172.21.15.86:0/780709009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f041005c0 0x7f1f041994d0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f1ef400e9c0 tx=0x7f1ef400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:45.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.458+0000 7f1f017fa640 1 -- 172.21.15.86:0/780709009 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef400cd80 con 0x7f1f041005c0 2024-04-21T08:43:45.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.458+0000 7f1f017fa640 1 -- 172.21.15.86:0/780709009 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ef40035f0 con 0x7f1f041005c0 2024-04-21T08:43:45.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.458+0000 7f1f0a69a640 1 -- 172.21.15.86:0/780709009 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f041a4c60 con 0x7f1f041005c0 2024-04-21T08:43:45.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.458+0000 7f1f017fa640 1 -- 172.21.15.86:0/780709009 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef4010690 con 0x7f1f041005c0 2024-04-21T08:43:45.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.458+0000 7f1f0a69a640 1 -- 172.21.15.86:0/780709009 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f041a5160 con 0x7f1f041005c0 2024-04-21T08:43:45.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.460+0000 7f1f017fa640 1 -- 172.21.15.86:0/780709009 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1ef4004000 con 0x7f1f041005c0 2024-04-21T08:43:45.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.460+0000 7f1f0d36a740 1 Processor -- start 2024-04-21T08:43:45.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.460+0000 7f1f0d36a740 1 -- start start 2024-04-21T08:43:45.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.460+0000 7f1f017fa640 1 --2- 172.21.15.86:0/780709009 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1eec0a3d20 0x7f1eec0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:45.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.460+0000 7f1f0d36a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba88c71580 0x55ba88c71960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:45.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.460+0000 7f1f0d36a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba88c71ea0 0x55ba88c7a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:45.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.460+0000 7f1f0d36a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba88c7aa80 0x55ba88c7ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:45.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.460+0000 7f1f0d36a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ba88bd61e0 con 0x55ba88c71ea0 2024-04-21T08:43:45.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.460+0000 7f1f0d36a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55ba88b73390 con 0x55ba88c71580 2024-04-21T08:43:45.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.460+0000 7f1f0d36a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ba88bd21a0 con 0x55ba88c7aa80 2024-04-21T08:43:45.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.460+0000 7f1f037fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba88c71ea0 0x55ba88c7a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:45.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.460+0000 7f1f08c37640 1 --2- 172.21.15.86:0/780709009 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1eec0a3d20 0x7f1eec0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:45.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.460+0000 7f1f037fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba88c71ea0 0x55ba88c7a540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37838/0 (socket says 172.21.15.86:37838) 2024-04-21T08:43:45.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.460+0000 7f1f037fe640 1 -- 172.21.15.86:0/185268268 learned_addr learned my addr 172.21.15.86:0/185268268 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:45.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.460+0000 7f1f017fa640 1 -- 172.21.15.86:0/780709009 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f1ef40d1170 con 0x7f1f041005c0 2024-04-21T08:43:45.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.461+0000 7f1f03fff640 1 --2- 172.21.15.86:0/185268268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba88c71580 0x55ba88c71960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:45.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.461+0000 7f1f037fe640 1 -- 172.21.15.86:0/185268268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba88c7aa80 msgr2=0x55ba88c7ce70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:45.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.461+0000 7f1f037fe640 1 --2- 172.21.15.86:0/185268268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba88c7aa80 0x55ba88c7ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.461+0000 7f1f037fe640 1 -- 172.21.15.86:0/185268268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba88c71580 msgr2=0x55ba88c71960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:45.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.461+0000 7f1f037fe640 1 --2- 172.21.15.86:0/185268268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba88c71580 0x55ba88c71960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.461+0000 7f1f037fe640 1 -- 172.21.15.86:0/185268268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba889b27d0 con 0x55ba88c71ea0 2024-04-21T08:43:45.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.461+0000 7f1f08c37640 1 --2- 172.21.15.86:0/780709009 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1eec0a3d20 0x7f1eec0a61e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1efc006010 tx=0x7f1efc0093f0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:45.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.461+0000 7f1f037fe640 1 --2- 172.21.15.86:0/185268268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba88c71ea0 0x55ba88c7a540 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8009be0 tx=0x7f1ef800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d7ca49d2e454276 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:45.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.461+0000 7f1eea7fc640 1 -- 172.21.15.86:0/185268268 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8005020 con 0x55ba88c71ea0 2024-04-21T08:43:45.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.461+0000 7f1eea7fc640 1 -- 172.21.15.86:0/185268268 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ef80047a0 con 0x55ba88c71ea0 2024-04-21T08:43:45.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.461+0000 7f1eea7fc640 1 -- 172.21.15.86:0/185268268 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8010690 con 0x55ba88c71ea0 2024-04-21T08:43:45.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.462+0000 7f1f0d36a740 1 -- 172.21.15.86:0/185268268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba88c71ea0 msgr2=0x55ba88c7a540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:45.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.462+0000 7f1f0d36a740 1 --2- 172.21.15.86:0/185268268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba88c71ea0 0x55ba88c7a540 secure :-1 s=READY pgs=1233 cs=0 l=1 rev1=1 crypto rx=0x7f1ef8009be0 tx=0x7f1ef800e720 comp rx=0 tx=0).stop 2024-04-21T08:43:45.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.462+0000 7f1f0d36a740 1 -- 172.21.15.86:0/185268268 shutdown_connections 2024-04-21T08:43:45.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.462+0000 7f1f0d36a740 1 --2- 172.21.15.86:0/185268268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba88c7aa80 0x55ba88c7ce70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.462+0000 7f1f0d36a740 1 --2- 172.21.15.86:0/185268268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba88c71ea0 0x55ba88c7a540 unknown :-1 s=CLOSED pgs=1233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.462+0000 7f1f0d36a740 1 --2- 172.21.15.86:0/185268268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba88c71580 0x55ba88c71960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.462+0000 7f1f0d36a740 1 -- 172.21.15.86:0/185268268 >> 172.21.15.86:0/185268268 conn(0x55ba88c6bcb0 msgr2=0x55ba88c6c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:45.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.462+0000 7f1f0d36a740 1 -- 172.21.15.86:0/185268268 shutdown_connections 2024-04-21T08:43:45.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.462+0000 7f1f0d36a740 1 -- 172.21.15.86:0/185268268 wait complete. 2024-04-21T08:43:45.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.462+0000 7f1f0d36a740 1 -- start start 2024-04-21T08:43:45.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.462+0000 7f1f0d36a740 1 Processor -- start 2024-04-21T08:43:45.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.463+0000 7f1f0d36a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba88c71580 0x55ba88c77ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:45.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.463+0000 7f1f0d36a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba88c71ea0 0x55ba88c78020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:45.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.463+0000 7f1f0d36a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba88c7aa80 0x55ba88c78560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:45.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.463+0000 7f1f0d36a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ba88c7d870 con 0x55ba88c7aa80 2024-04-21T08:43:45.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.463+0000 7f1f0d36a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55ba88b8f2f0 con 0x55ba88c71580 2024-04-21T08:43:45.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.463+0000 7f1f0d36a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ba88c7d9f0 con 0x55ba88c71ea0 2024-04-21T08:43:45.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.463+0000 7f1f0d36a740 10 client.0 _ll_register_callbacks cb 0x55ba88b9c580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:45.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.463+0000 7f1f08c37640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba88c7aa80 0x55ba88c78560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:45.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.463+0000 7f1f08c37640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba88c7aa80 0x55ba88c78560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37848/0 (socket says 172.21.15.86:37848) 2024-04-21T08:43:45.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.463+0000 7f1f08c37640 1 -- 172.21.15.86:0/3819519884 learned_addr learned my addr 172.21.15.86:0/3819519884 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:45.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.463+0000 7f1f03fff640 1 --2- 172.21.15.86:0/3819519884 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba88c71580 0x55ba88c77ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:45.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.463+0000 7f1f037fe640 1 --2- 172.21.15.86:0/3819519884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba88c71ea0 0x55ba88c78020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:45.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.463+0000 7f1f08c37640 1 -- 172.21.15.86:0/3819519884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba88c71ea0 msgr2=0x55ba88c78020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:45.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.463+0000 7f1f08c37640 1 --2- 172.21.15.86:0/3819519884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba88c71ea0 0x55ba88c78020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.463+0000 7f1f08c37640 1 -- 172.21.15.86:0/3819519884 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba88c71580 msgr2=0x55ba88c77ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:45.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.463+0000 7f1f08c37640 1 --2- 172.21.15.86:0/3819519884 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba88c71580 0x55ba88c77ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.463+0000 7f1f08c37640 1 -- 172.21.15.86:0/3819519884 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba888e83f0 con 0x55ba88c7aa80 2024-04-21T08:43:45.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.463+0000 7f1f08c37640 1 --2- 172.21.15.86:0/3819519884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba88c7aa80 0x55ba88c78560 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f1efc036000 tx=0x7f1efc0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2e2b9de12a6678b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:45.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.463+0000 7f1ee37fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:45.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.463+0000 7f1ee37fe640 1 -- 172.21.15.86:0/3819519884 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1efc039040 con 0x55ba88c7aa80 2024-04-21T08:43:45.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.463+0000 7f1f0d36a740 1 -- 172.21.15.86:0/3819519884 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ef80047a0 con 0x55ba88c7aa80 2024-04-21T08:43:45.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.463+0000 7f1ee37fe640 1 -- 172.21.15.86:0/3819519884 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f1efc0047d0 con 0x55ba88c7aa80 2024-04-21T08:43:45.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.463+0000 7f1ee37fe640 1 -- 172.21.15.86:0/3819519884 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1efc05a050 con 0x55ba88c7aa80 2024-04-21T08:43:45.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.463+0000 7f1f0d36a740 10 client.17874 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:45.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.463+0000 7f1f0d36a740 1 -- 172.21.15.86:0/3819519884 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ba888a80f0 con 0x55ba88c7aa80 2024-04-21T08:43:45.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.464+0000 7f1ee37fe640 1 -- 172.21.15.86:0/3819519884 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ef80047a0 con 0x55ba88c7aa80 2024-04-21T08:43:45.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.464+0000 7f1ee37fe640 1 -- 172.21.15.86:0/3819519884 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1efc03f040 con 0x55ba88c7aa80 2024-04-21T08:43:45.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.464+0000 7f1f0d36a740 10 client.17874 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:45.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.464+0000 7f1f0d36a740 10 client.17874 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:45.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.464+0000 7f1f0d36a740 20 client.17874 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:45.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.464+0000 7f1f0d36a740 1 --2- 172.21.15.86:0/3819519884 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ba88c8ab60 0x55ba88c8cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:45.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.464+0000 7f1f0d36a740 4 client.17874 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:45.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.464+0000 7f1f0d36a740 1 -- 172.21.15.86:0/3819519884 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba88be3c00 con 0x55ba88c8ab60 2024-04-21T08:43:45.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.464+0000 7f1f03fff640 1 --2- 172.21.15.86:0/3819519884 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ba88c8ab60 0x55ba88c8cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:45.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.464+0000 7f1f03fff640 1 --2- 172.21.15.86:0/3819519884 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ba88c8ab60 0x55ba88c8cfe0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7e9d258b83a506f1 server_cookie=31b3ef00f1514f92 in_seq=0 out_seq=0 2024-04-21T08:43:45.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.464+0000 7f1ee37fe640 10 client.17874 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:45.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.465+0000 7f1ee37fe640 1 -- 172.21.15.86:0/3819519884 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ba88be3c00 con 0x55ba88c8ab60 2024-04-21T08:43:45.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.465+0000 7f1ee37fe640 10 client.17874 handle_command_reply: tid=0 2024-04-21T08:43:45.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.465+0000 7f1f0d36a740 1 client.17874 shutdown 2024-04-21T08:43:45.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.465+0000 7f1f0d36a740 1 -- 172.21.15.86:0/3819519884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba88c7aa80 msgr2=0x55ba88c78560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:45.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.465+0000 7f1f0d36a740 1 --2- 172.21.15.86:0/3819519884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba88c7aa80 0x55ba88c78560 secure :-1 s=READY pgs=1234 cs=0 l=1 rev1=1 crypto rx=0x7f1efc036000 tx=0x7f1efc0095c0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.465+0000 7f1f0d36a740 1 -- 172.21.15.86:0/3819519884 shutdown_connections 2024-04-21T08:43:45.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.465+0000 7f1f0d36a740 1 --2- 172.21.15.86:0/3819519884 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ba88c8ab60 0x55ba88c8cfe0 crc :-1 s=READY pgs=173 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.465+0000 7f1f0d36a740 1 --2- 172.21.15.86:0/3819519884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba88c7aa80 0x55ba88c78560 unknown :-1 s=CLOSED pgs=1234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.465+0000 7f1f0d36a740 1 --2- 172.21.15.86:0/3819519884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba88c71ea0 0x55ba88c78020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.465+0000 7f1ee37fe640 0 client.17874 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:45.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.465+0000 7f1f0d36a740 1 --2- 172.21.15.86:0/3819519884 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba88c71580 0x55ba88c77ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.465+0000 7f1f0d36a740 1 -- 172.21.15.86:0/3819519884 >> 172.21.15.86:0/3819519884 conn(0x55ba88c6bcb0 msgr2=0x55ba88c6da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:45.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.465+0000 7f1f0d36a740 1 -- 172.21.15.86:0/3819519884 shutdown_connections 2024-04-21T08:43:45.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.465+0000 7f1f0d36a740 1 -- 172.21.15.86:0/3819519884 wait complete. 2024-04-21T08:43:45.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.465+0000 7f1f0d36a740 20 client.17874 trim_cache size 0 max 16384 2024-04-21T08:43:45.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.483+0000 7f1f0d36a740 1 Processor -- start 2024-04-21T08:43:45.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.483+0000 7f1f0d36a740 1 -- start start 2024-04-21T08:43:45.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.483+0000 7f1f0d36a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba88c71580 0x55ba88c86760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:45.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.483+0000 7f1f0d36a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba88c71ea0 0x55ba88c773c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:45.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.483+0000 7f1f0d36a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba88c7aa80 0x55ba88c928b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:45.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.483+0000 7f1f0d36a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f041af310 con 0x55ba88c71ea0 2024-04-21T08:43:45.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.483+0000 7f1f0d36a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1f041b0e50 con 0x55ba88c7aa80 2024-04-21T08:43:45.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.483+0000 7f1f0d36a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1f041af9e0 con 0x55ba88c71580 2024-04-21T08:43:45.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.483+0000 7f1f037fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba88c71ea0 0x55ba88c773c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:45.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.483+0000 7f1f037fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba88c71ea0 0x55ba88c773c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37852/0 (socket says 172.21.15.86:37852) 2024-04-21T08:43:45.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.483+0000 7f1f037fe640 1 -- 172.21.15.86:0/51760200 learned_addr learned my addr 172.21.15.86:0/51760200 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:45.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.483+0000 7f1f037fe640 1 -- 172.21.15.86:0/51760200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba88c71580 msgr2=0x55ba88c86760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:45.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.483+0000 7f1f03fff640 1 --2- 172.21.15.86:0/51760200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba88c71580 0x55ba88c86760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:45.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.483+0000 7f1f037fe640 1 --2- 172.21.15.86:0/51760200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba88c71580 0x55ba88c86760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.483+0000 7f1f037fe640 1 -- 172.21.15.86:0/51760200 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba88c7aa80 msgr2=0x55ba88c928b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:45.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.483+0000 7f1f08c37640 1 --2- 172.21.15.86:0/51760200 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba88c7aa80 0x55ba88c928b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:45.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.483+0000 7f1f037fe640 1 --2- 172.21.15.86:0/51760200 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba88c7aa80 0x55ba88c928b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.483+0000 7f1f037fe640 1 -- 172.21.15.86:0/51760200 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba88c6b510 con 0x55ba88c71ea0 2024-04-21T08:43:45.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.483+0000 7f1f03fff640 1 --2- 172.21.15.86:0/51760200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba88c71580 0x55ba88c86760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:43:45.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.483+0000 7f1f037fe640 1 --2- 172.21.15.86:0/51760200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba88c71ea0 0x55ba88c773c0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f1ef80046b0 tx=0x7f1ef8009530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be416c7e7970e869 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:45.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.484+0000 7f1ee9ffb640 1 -- 172.21.15.86:0/51760200 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8025070 con 0x55ba88c71ea0 2024-04-21T08:43:45.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.484+0000 7f1ee9ffb640 1 -- 172.21.15.86:0/51760200 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ef8009730 con 0x55ba88c71ea0 2024-04-21T08:43:45.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.484+0000 7f1ee9ffb640 1 -- 172.21.15.86:0/51760200 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef8004c40 con 0x55ba88c71ea0 2024-04-21T08:43:45.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.484+0000 7f1f0d36a740 1 -- 172.21.15.86:0/51760200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba88c71ea0 msgr2=0x55ba88c773c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:45.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.484+0000 7f1f0d36a740 1 --2- 172.21.15.86:0/51760200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba88c71ea0 0x55ba88c773c0 secure :-1 s=READY pgs=1235 cs=0 l=1 rev1=1 crypto rx=0x7f1ef80046b0 tx=0x7f1ef8009530 comp rx=0 tx=0).stop 2024-04-21T08:43:45.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.484+0000 7f1f0d36a740 1 -- 172.21.15.86:0/51760200 shutdown_connections 2024-04-21T08:43:45.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.484+0000 7f1f0d36a740 1 --2- 172.21.15.86:0/51760200 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba88c7aa80 0x55ba88c928b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.484+0000 7f1f0d36a740 1 --2- 172.21.15.86:0/51760200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba88c71ea0 0x55ba88c773c0 unknown :-1 s=CLOSED pgs=1235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.484+0000 7f1f0d36a740 1 --2- 172.21.15.86:0/51760200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba88c71580 0x55ba88c86760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.484+0000 7f1f0d36a740 1 -- 172.21.15.86:0/51760200 >> 172.21.15.86:0/51760200 conn(0x55ba88c6bcb0 msgr2=0x55ba88c79a90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:45.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.484+0000 7f1f0d36a740 1 -- 172.21.15.86:0/51760200 shutdown_connections 2024-04-21T08:43:45.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.484+0000 7f1f0d36a740 1 -- 172.21.15.86:0/51760200 wait complete. 2024-04-21T08:43:45.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.485+0000 7f1f0d36a740 1 -- start start 2024-04-21T08:43:45.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.485+0000 7f1f0d36a740 1 Processor -- start 2024-04-21T08:43:45.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.485+0000 7f1f0d36a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba88c71580 0x55ba88c7e900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:45.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.485+0000 7f1f0d36a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba88c71ea0 0x55ba88c7ee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:45.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.485+0000 7f1f0d36a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba88c7aa80 0x55ba88c7f380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:45.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.485+0000 7f1f0d36a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f041b0780 con 0x55ba88c71580 2024-04-21T08:43:45.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.485+0000 7f1f0d36a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1f041b00b0 con 0x55ba88c71ea0 2024-04-21T08:43:45.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.485+0000 7f1f0d36a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1f041ad2d0 con 0x55ba88c7aa80 2024-04-21T08:43:45.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.485+0000 7f1f0d36a740 10 client.0 _ll_register_callbacks cb 0x7f1efc037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:45.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.485+0000 7f1f03fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba88c71580 0x55ba88c7e900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:45.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.485+0000 7f1f03fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba88c71580 0x55ba88c7e900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37856/0 (socket says 172.21.15.86:37856) 2024-04-21T08:43:45.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.485+0000 7f1f03fff640 1 -- 172.21.15.86:0/1664321670 learned_addr learned my addr 172.21.15.86:0/1664321670 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:45.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.485+0000 7f1f037fe640 1 --2- 172.21.15.86:0/1664321670 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba88c71ea0 0x55ba88c7ee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:45.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.485+0000 7f1f08c37640 1 --2- 172.21.15.86:0/1664321670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba88c7aa80 0x55ba88c7f380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:45.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.485+0000 7f1f03fff640 1 -- 172.21.15.86:0/1664321670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba88c7aa80 msgr2=0x55ba88c7f380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:45.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.485+0000 7f1f03fff640 1 --2- 172.21.15.86:0/1664321670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba88c7aa80 0x55ba88c7f380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.485+0000 7f1f03fff640 1 -- 172.21.15.86:0/1664321670 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba88c71ea0 msgr2=0x55ba88c7ee40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:45.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.485+0000 7f1f03fff640 1 --2- 172.21.15.86:0/1664321670 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba88c71ea0 0x55ba88c7ee40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.485+0000 7f1f03fff640 1 -- 172.21.15.86:0/1664321670 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba88c6b320 con 0x55ba88c71580 2024-04-21T08:43:45.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.486+0000 7f1f03fff640 1 --2- 172.21.15.86:0/1664321670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba88c71580 0x55ba88c7e900 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009fd0 tx=0x7f1ef4005f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=41cd6aec2318f650 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:45.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.486+0000 7f1ee37fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:45.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.486+0000 7f1ee37fe640 1 -- 172.21.15.86:0/1664321670 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1ef40a0070 con 0x55ba88c71580 2024-04-21T08:43:45.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.486+0000 7f1ee37fe640 1 -- 172.21.15.86:0/1664321670 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f1ef4003aa0 con 0x55ba88c71580 2024-04-21T08:43:45.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.486+0000 7f1f0d36a740 1 -- 172.21.15.86:0/1664321670 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1ef8009730 con 0x55ba88c71580 2024-04-21T08:43:45.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.486+0000 7f1ee37fe640 1 -- 172.21.15.86:0/1664321670 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1ef40a5a10 con 0x55ba88c71580 2024-04-21T08:43:45.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.486+0000 7f1f0d36a740 10 client.17880 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:45.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.486+0000 7f1f0d36a740 1 -- 172.21.15.86:0/1664321670 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ba88c7ce70 con 0x55ba88c71580 2024-04-21T08:43:45.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.486+0000 7f1ee37fe640 1 -- 172.21.15.86:0/1664321670 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1ef8009730 con 0x55ba88c71580 2024-04-21T08:43:45.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.486+0000 7f1ee37fe640 1 -- 172.21.15.86:0/1664321670 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1ef40042f0 con 0x55ba88c71580 2024-04-21T08:43:45.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.486+0000 7f1f0d36a740 10 client.17880 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:45.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.486+0000 7f1f0d36a740 10 client.17880 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:45.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.486+0000 7f1f0d36a740 20 client.17880 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:45.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.486+0000 7f1f0d36a740 1 --2- 172.21.15.86:0/1664321670 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ba88cf5db0 0x55ba88cf81a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:45.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.486+0000 7f1f0d36a740 4 client.17880 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:43:45.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.486+0000 7f1f0d36a740 1 -- 172.21.15.86:0/1664321670 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55ba88bdda70 con 0x55ba88cf5db0 2024-04-21T08:43:45.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.486+0000 7f1f037fe640 1 --2- 172.21.15.86:0/1664321670 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ba88cf5db0 0x55ba88cf81a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:45.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.487+0000 7f1ee37fe640 10 client.17880 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:45.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.487+0000 7f1f037fe640 1 --2- 172.21.15.86:0/1664321670 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ba88cf5db0 0x55ba88cf81a0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a9f24c861d3fae3 server_cookie=617365123816e50 in_seq=0 out_seq=0 2024-04-21T08:43:45.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.487+0000 7f1ee37fe640 1 -- 172.21.15.86:0/1664321670 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55ba88bdda70 con 0x55ba88cf5db0 2024-04-21T08:43:45.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.487+0000 7f1ee37fe640 10 client.17880 handle_command_reply: tid=0 2024-04-21T08:43:45.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.487+0000 7f1f0d36a740 1 client.17880 shutdown 2024-04-21T08:43:45.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.487+0000 7f1f0d36a740 1 -- 172.21.15.86:0/1664321670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba88c71580 msgr2=0x55ba88c7e900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:45.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.487+0000 7f1f0d36a740 1 --2- 172.21.15.86:0/1664321670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba88c71580 0x55ba88c7e900 secure :-1 s=READY pgs=1236 cs=0 l=1 rev1=1 crypto rx=0x7f1ef4009fd0 tx=0x7f1ef4005f10 comp rx=0 tx=0).stop 2024-04-21T08:43:45.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.487+0000 7f1f0d36a740 1 -- 172.21.15.86:0/1664321670 shutdown_connections 2024-04-21T08:43:45.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.487+0000 7f1f0d36a740 1 --2- 172.21.15.86:0/1664321670 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ba88cf5db0 0x55ba88cf81a0 crc :-1 s=READY pgs=174 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.487+0000 7f1f0d36a740 1 --2- 172.21.15.86:0/1664321670 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba88c7aa80 0x55ba88c7f380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.487+0000 7f1f0d36a740 1 --2- 172.21.15.86:0/1664321670 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba88c71ea0 0x55ba88c7ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.487+0000 7f1f0d36a740 1 --2- 172.21.15.86:0/1664321670 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba88c71580 0x55ba88c7e900 unknown :-1 s=CLOSED pgs=1236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.487+0000 7f1ee37fe640 0 client.17880 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:45.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.487+0000 7f1f0d36a740 1 -- 172.21.15.86:0/1664321670 >> 172.21.15.86:0/1664321670 conn(0x55ba88c6bcb0 msgr2=0x55ba88c6ce70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:45.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.487+0000 7f1f0d36a740 1 -- 172.21.15.86:0/1664321670 shutdown_connections 2024-04-21T08:43:45.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.487+0000 7f1f0d36a740 1 -- 172.21.15.86:0/1664321670 wait complete. 2024-04-21T08:43:45.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.487+0000 7f1f0d36a740 20 client.17880 trim_cache size 0 max 16384 2024-04-21T08:43:45.495 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:43:45.495 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:43:45.495 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-21T08:43:45.495 INFO:teuthology.orchestra.run.smithi086.stdout: "18507780-feef-47e5-b927-9d3c22dfb928": { 2024-04-21T08:43:45.495 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-21T08:43:45.495 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "18507780-feef-47e5-b927-9d3c22dfb928", 2024-04-21T08:43:45.495 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-21T08:43:45.495 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:43:45.495 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:43:45.495 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:43:45.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.488+0000 7f1f0a69a640 1 -- 172.21.15.86:0/780709009 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1eec0a3d20 msgr2=0x7f1eec0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:45.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.488+0000 7f1f0a69a640 1 --2- 172.21.15.86:0/780709009 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1eec0a3d20 0x7f1eec0a61e0 secure :-1 s=READY pgs=193 cs=0 l=1 rev1=1 crypto rx=0x7f1efc006010 tx=0x7f1efc0093f0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.488+0000 7f1f0a69a640 1 -- 172.21.15.86:0/780709009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f041005c0 msgr2=0x7f1f041994d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:45.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.488+0000 7f1f0a69a640 1 --2- 172.21.15.86:0/780709009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f041005c0 0x7f1f041994d0 secure :-1 s=READY pgs=1232 cs=0 l=1 rev1=1 crypto rx=0x7f1ef400e9c0 tx=0x7f1ef400ee80 comp rx=0 tx=0).stop 2024-04-21T08:43:45.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.488+0000 7f1f0a69a640 1 -- 172.21.15.86:0/780709009 shutdown_connections 2024-04-21T08:43:45.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.488+0000 7f1f0a69a640 1 --2- 172.21.15.86:0/780709009 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1eec0a3d20 0x7f1eec0a61e0 unknown :-1 s=CLOSED pgs=193 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.488+0000 7f1f0a69a640 1 --2- 172.21.15.86:0/780709009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f04106660 0x7f1f041a4550 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.488+0000 7f1f0a69a640 1 --2- 172.21.15.86:0/780709009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f04100f80 0x7f1f04199a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.488+0000 7f1f0a69a640 1 --2- 172.21.15.86:0/780709009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f041005c0 0x7f1f041994d0 unknown :-1 s=CLOSED pgs=1232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:45.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.488+0000 7f1f0a69a640 1 -- 172.21.15.86:0/780709009 >> 172.21.15.86:0/780709009 conn(0x7f1f040773a0 msgr2=0x7f1f04102190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:45.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.488+0000 7f1f0a69a640 1 -- 172.21.15.86:0/780709009 shutdown_connections 2024-04-21T08:43:45.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:43:45.488+0000 7f1f0a69a640 1 -- 172.21.15.86:0/780709009 wait complete. 2024-04-21T08:43:45.499 INFO:tasks.cephfs.filesystem:scrub status for tag:18507780-feef-47e5-b927-9d3c22dfb928 - {'path': '/', 'tag': '18507780-feef-47e5-b927-9d3c22dfb928', 'options': 'recursive,force'} 2024-04-21T08:43:45.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:45 smithi086 ceph-mon[27939]: pgmap v152: 129 pgs: 129 active+clean; 2.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-04-21T08:43:45.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:45 smithi088 ceph-mon[31259]: pgmap v152: 129 pgs: 129 active+clean; 2.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-04-21T08:43:45.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:45 smithi179 ceph-mon[31829]: pgmap v152: 129 pgs: 129 active+clean; 2.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-04-21T08:43:46.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:46 smithi086 ceph-mon[27939]: scrub queued for path: / 2024-04-21T08:43:46.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:46 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:43:46.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:46 smithi086 ceph-mon[27939]: scrub summary: active paths [/] 2024-04-21T08:43:46.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:46 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:43:46.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:46 smithi088 ceph-mon[31259]: scrub queued for path: / 2024-04-21T08:43:46.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:46 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:43:46.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:46 smithi088 ceph-mon[31259]: scrub summary: active paths [/] 2024-04-21T08:43:46.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:46 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:43:46.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:46 smithi179 ceph-mon[31829]: scrub queued for path: / 2024-04-21T08:43:46.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:46 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:43:46.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:46 smithi179 ceph-mon[31829]: scrub summary: active paths [/] 2024-04-21T08:43:46.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:46 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:43:47.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:47 smithi086 ceph-mon[27939]: pgmap v153: 129 pgs: 129 active+clean; 2.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-04-21T08:43:47.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:47 smithi088 ceph-mon[31259]: pgmap v153: 129 pgs: 129 active+clean; 2.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-04-21T08:43:47.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:47 smithi179 ceph-mon[31829]: pgmap v153: 129 pgs: 129 active+clean; 2.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 30 KiB/s wr, 13 op/s 2024-04-21T08:43:50.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:49 smithi086 ceph-mon[27939]: pgmap v154: 129 pgs: 129 active+clean; 2.7 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 52 KiB/s wr, 23 op/s 2024-04-21T08:43:50.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:49 smithi086 ceph-mon[27939]: scrub summary: idle 2024-04-21T08:43:50.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:49 smithi088 ceph-mon[31259]: pgmap v154: 129 pgs: 129 active+clean; 2.7 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 52 KiB/s wr, 23 op/s 2024-04-21T08:43:50.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:49 smithi088 ceph-mon[31259]: scrub summary: idle 2024-04-21T08:43:50.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:49 smithi179 ceph-mon[31829]: pgmap v154: 129 pgs: 129 active+clean; 2.7 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 52 KiB/s wr, 23 op/s 2024-04-21T08:43:50.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:49 smithi179 ceph-mon[31829]: scrub summary: idle 2024-04-21T08:43:52.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:51 smithi086 ceph-mon[27939]: pgmap v155: 129 pgs: 129 active+clean; 2.7 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 37 KiB/s wr, 17 op/s 2024-04-21T08:43:52.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:51 smithi088 ceph-mon[31259]: pgmap v155: 129 pgs: 129 active+clean; 2.7 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 37 KiB/s wr, 17 op/s 2024-04-21T08:43:52.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:51 smithi179 ceph-mon[31829]: pgmap v155: 129 pgs: 129 active+clean; 2.7 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 37 KiB/s wr, 17 op/s 2024-04-21T08:43:54.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:53 smithi086 ceph-mon[27939]: pgmap v156: 129 pgs: 129 active+clean; 2.9 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 23 op/s 2024-04-21T08:43:54.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:53 smithi088 ceph-mon[31259]: pgmap v156: 129 pgs: 129 active+clean; 2.9 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 23 op/s 2024-04-21T08:43:54.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:53 smithi179 ceph-mon[31829]: pgmap v156: 129 pgs: 129 active+clean; 2.9 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 23 op/s 2024-04-21T08:43:56.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:55 smithi086 ceph-mon[27939]: pgmap v157: 129 pgs: 129 active+clean; 2.9 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 KiB/s wr, 17 op/s 2024-04-21T08:43:56.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:55 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:43:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.129+0000 7f39c85e6640 1 -- 172.21.15.86:0/171867418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39c0101120 msgr2=0x7f39c0101500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.129+0000 7f39c85e6640 1 --2- 172.21.15.86:0/171867418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39c0101120 0x7f39c0101500 secure :-1 s=READY pgs=1237 cs=0 l=1 rev1=1 crypto rx=0x7f39b0009900 tx=0x7f39b002bc30 comp rx=0 tx=0).stop 2024-04-21T08:43:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.129+0000 7f39c85e6640 1 -- 172.21.15.86:0/171867418 shutdown_connections 2024-04-21T08:43:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.129+0000 7f39c85e6640 1 --2- 172.21.15.86:0/171867418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39c01061a0 0x7f39c010dd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.129+0000 7f39c85e6640 1 --2- 172.21.15.86:0/171867418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39c0101ad0 0x7f39c0105ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.129+0000 7f39c85e6640 1 --2- 172.21.15.86:0/171867418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39c0101120 0x7f39c0101500 unknown :-1 s=CLOSED pgs=1237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.129+0000 7f39c85e6640 1 -- 172.21.15.86:0/171867418 >> 172.21.15.86:0/171867418 conn(0x7f39c0077fa0 msgr2=0x7f39c00783b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.129+0000 7f39c85e6640 1 -- 172.21.15.86:0/171867418 shutdown_connections 2024-04-21T08:43:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.129+0000 7f39c85e6640 1 -- 172.21.15.86:0/171867418 wait complete. 2024-04-21T08:43:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.130+0000 7f39c85e6640 1 Processor -- start 2024-04-21T08:43:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.130+0000 7f39c85e6640 1 -- start start 2024-04-21T08:43:56.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.130+0000 7f39c85e6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39c0101120 0x7f39c0199480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.130+0000 7f39c85e6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39c01061a0 0x7f39c01999c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.130+0000 7f39c85e6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39c0199f50 0x7f39c01a4500 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.130+0000 7f39c85e6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f39c01110e0 con 0x7f39c01061a0 2024-04-21T08:43:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.130+0000 7f39c85e6640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f39c0110f60 con 0x7f39c0101120 2024-04-21T08:43:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.130+0000 7f39c85e6640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f39c0111260 con 0x7f39c0199f50 2024-04-21T08:43:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.130+0000 7f39c5b81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39c01061a0 0x7f39c01999c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.130+0000 7f39c5b81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39c01061a0 0x7f39c01999c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42878/0 (socket says 172.21.15.86:42878) 2024-04-21T08:43:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.130+0000 7f39c5b81640 1 -- 172.21.15.86:0/2044985447 learned_addr learned my addr 172.21.15.86:0/2044985447 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.130+0000 7f39c6b83640 1 --2- 172.21.15.86:0/2044985447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39c0199f50 0x7f39c01a4500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.130+0000 7f39c6382640 1 --2- 172.21.15.86:0/2044985447 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39c0101120 0x7f39c0199480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.130+0000 7f39c5b81640 1 -- 172.21.15.86:0/2044985447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39c0199f50 msgr2=0x7f39c01a4500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.130+0000 7f39c5b81640 1 --2- 172.21.15.86:0/2044985447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39c0199f50 0x7f39c01a4500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.130+0000 7f39c5b81640 1 -- 172.21.15.86:0/2044985447 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39c0101120 msgr2=0x7f39c0199480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:56.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.130+0000 7f39c5b81640 1 --2- 172.21.15.86:0/2044985447 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39c0101120 0x7f39c0199480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.130+0000 7f39c5b81640 1 -- 172.21.15.86:0/2044985447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f39c01a4d00 con 0x7f39c01061a0 2024-04-21T08:43:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.130+0000 7f39c5b81640 1 --2- 172.21.15.86:0/2044985447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39c01061a0 0x7f39c01999c0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f39bc00c970 tx=0x7f39bc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.131+0000 7f39b77fe640 1 -- 172.21.15.86:0/2044985447 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc014070 con 0x7f39c01061a0 2024-04-21T08:43:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.131+0000 7f39c85e6640 1 -- 172.21.15.86:0/2044985447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f39c01a4ff0 con 0x7f39c01061a0 2024-04-21T08:43:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.131+0000 7f39b77fe640 1 -- 172.21.15.86:0/2044985447 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39bc004440 con 0x7f39c01061a0 2024-04-21T08:43:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.131+0000 7f39b77fe640 1 -- 172.21.15.86:0/2044985447 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc003e50 con 0x7f39c01061a0 2024-04-21T08:43:56.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.131+0000 7f39c85e6640 1 -- 172.21.15.86:0/2044985447 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f39c01a54b0 con 0x7f39c01061a0 2024-04-21T08:43:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.133+0000 7f39b77fe640 1 -- 172.21.15.86:0/2044985447 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f39bc017070 con 0x7f39c01061a0 2024-04-21T08:43:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.134+0000 7f39b77fe640 1 --2- 172.21.15.86:0/2044985447 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f39ac0a3f60 0x7f39ac0a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.134+0000 7f39c6382640 1 --2- 172.21.15.86:0/2044985447 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f39ac0a3f60 0x7f39ac0a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.134+0000 7f39b77fe640 1 -- 172.21.15.86:0/2044985447 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f39bc0d0700 con 0x7f39c01061a0 2024-04-21T08:43:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.134+0000 7f39cb1d2740 1 Processor -- start 2024-04-21T08:43:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.135+0000 7f39cb1d2740 1 -- start start 2024-04-21T08:43:56.140 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.135+0000 7f39c6382640 1 --2- 172.21.15.86:0/2044985447 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f39ac0a3f60 0x7f39ac0a6420 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f39b0000c00 tx=0x7f39b0004040 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.135+0000 7f39cb1d2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c7ebd298d0 0x55c7ebd28620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.135+0000 7f39cb1d2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7ebd28b60 0x55c7ebd28fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.135+0000 7f39cb1d2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7ebd364f0 0x55c7ebd388e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.135+0000 7f39cb1d2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c7ebc8d8d0 con 0x55c7ebd28b60 2024-04-21T08:43:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.135+0000 7f39cb1d2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c7ebc2ae80 con 0x55c7ebd298d0 2024-04-21T08:43:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.135+0000 7f39cb1d2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c7ebc89890 con 0x55c7ebd364f0 2024-04-21T08:43:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.135+0000 7f39c5b81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7ebd28b60 0x55c7ebd28fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.135+0000 7f39c5b81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7ebd28b60 0x55c7ebd28fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42880/0 (socket says 172.21.15.86:42880) 2024-04-21T08:43:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.135+0000 7f39c5b81640 1 -- 172.21.15.86:0/2428825348 learned_addr learned my addr 172.21.15.86:0/2428825348 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.135+0000 7f39c6382640 1 --2- 172.21.15.86:0/2428825348 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c7ebd298d0 0x55c7ebd28620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.135+0000 7f39c6b83640 1 --2- 172.21.15.86:0/2428825348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7ebd364f0 0x55c7ebd388e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.135+0000 7f39c5b81640 1 -- 172.21.15.86:0/2428825348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7ebd364f0 msgr2=0x55c7ebd388e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.135+0000 7f39c5b81640 1 --2- 172.21.15.86:0/2428825348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7ebd364f0 0x55c7ebd388e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:56.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.135+0000 7f39c5b81640 1 -- 172.21.15.86:0/2428825348 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c7ebd298d0 msgr2=0x55c7ebd28620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.135+0000 7f39c5b81640 1 --2- 172.21.15.86:0/2428825348 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c7ebd298d0 0x55c7ebd28620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.135+0000 7f39c5b81640 1 -- 172.21.15.86:0/2428825348 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7ebc4c320 con 0x55c7ebd28b60 2024-04-21T08:43:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.135+0000 7f39c6b83640 1 --2- 172.21.15.86:0/2428825348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7ebd364f0 0x55c7ebd388e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.136+0000 7f39c5b81640 1 --2- 172.21.15.86:0/2428825348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7ebd28b60 0x55c7ebd28fe0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f39bc00afd0 tx=0x7f39bc09bd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=84eb9c897d37f022 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.136+0000 7f39b4bfe640 1 -- 172.21.15.86:0/2428825348 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc0b5070 con 0x55c7ebd28b60 2024-04-21T08:43:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.136+0000 7f39b4bfe640 1 -- 172.21.15.86:0/2428825348 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39bc0a4fa0 con 0x55c7ebd28b60 2024-04-21T08:43:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.136+0000 7f39b4bfe640 1 -- 172.21.15.86:0/2428825348 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc0d0510 con 0x55c7ebd28b60 2024-04-21T08:43:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.136+0000 7f39cb1d2740 1 -- 172.21.15.86:0/2428825348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7ebd28b60 msgr2=0x55c7ebd28fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.136+0000 7f39cb1d2740 1 --2- 172.21.15.86:0/2428825348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7ebd28b60 0x55c7ebd28fe0 secure :-1 s=READY pgs=1239 cs=0 l=1 rev1=1 crypto rx=0x7f39bc00afd0 tx=0x7f39bc09bd90 comp rx=0 tx=0).stop 2024-04-21T08:43:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.137+0000 7f39cb1d2740 1 -- 172.21.15.86:0/2428825348 shutdown_connections 2024-04-21T08:43:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.137+0000 7f39cb1d2740 1 --2- 172.21.15.86:0/2428825348 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7ebd364f0 0x55c7ebd388e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.137+0000 7f39cb1d2740 1 --2- 172.21.15.86:0/2428825348 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7ebd28b60 0x55c7ebd28fe0 unknown :-1 s=CLOSED pgs=1239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:56.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.137+0000 7f39cb1d2740 1 --2- 172.21.15.86:0/2428825348 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c7ebd298d0 0x55c7ebd28620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.137+0000 7f39cb1d2740 1 -- 172.21.15.86:0/2428825348 >> 172.21.15.86:0/2428825348 conn(0x55c7ebcefc90 msgr2=0x55c7ebd23850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.137+0000 7f39cb1d2740 1 -- 172.21.15.86:0/2428825348 shutdown_connections 2024-04-21T08:43:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.137+0000 7f39cb1d2740 1 -- 172.21.15.86:0/2428825348 wait complete. 2024-04-21T08:43:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.137+0000 7f39cb1d2740 1 -- start start 2024-04-21T08:43:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.137+0000 7f39cb1d2740 1 Processor -- start 2024-04-21T08:43:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.137+0000 7f39cb1d2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c7ebd28b60 0x55c7ebd35f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.137+0000 7f39cb1d2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7ebd298d0 0x55c7ebd33300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.137+0000 7f39cb1d2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7ebd364f0 0x55c7ebd33a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:56.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.137+0000 7f39cb1d2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c7ebd39220 con 0x55c7ebd364f0 2024-04-21T08:43:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.137+0000 7f39cb1d2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c7ebc15eb0 con 0x55c7ebd28b60 2024-04-21T08:43:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.137+0000 7f39cb1d2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c7ebd393a0 con 0x55c7ebd298d0 2024-04-21T08:43:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.137+0000 7f39cb1d2740 10 client.0 _ll_register_callbacks cb 0x55c7ebc42b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.137+0000 7f39c6b83640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7ebd364f0 0x55c7ebd33a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.137+0000 7f39c6382640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c7ebd28b60 0x55c7ebd35f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.137+0000 7f39c6b83640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7ebd364f0 0x55c7ebd33a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42886/0 (socket says 172.21.15.86:42886) 2024-04-21T08:43:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.137+0000 7f39c6b83640 1 -- 172.21.15.86:0/2040476512 learned_addr learned my addr 172.21.15.86:0/2040476512 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.138+0000 7f39c5b81640 1 --2- 172.21.15.86:0/2040476512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7ebd298d0 0x55c7ebd33300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.138+0000 7f39c6382640 1 -- 172.21.15.86:0/2040476512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7ebd298d0 msgr2=0x55c7ebd33300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.138+0000 7f39c6382640 1 --2- 172.21.15.86:0/2040476512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7ebd298d0 0x55c7ebd33300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.138+0000 7f39c6382640 1 -- 172.21.15.86:0/2040476512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7ebd364f0 msgr2=0x55c7ebd33a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.138+0000 7f39c6382640 1 --2- 172.21.15.86:0/2040476512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7ebd364f0 0x55c7ebd33a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:56.144 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.138+0000 7f39c6382640 1 -- 172.21.15.86:0/2040476512 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7eb9a0210 con 0x55c7ebd28b60 2024-04-21T08:43:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.138+0000 7f39c5b81640 1 --2- 172.21.15.86:0/2040476512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7ebd298d0 0x55c7ebd33300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.138+0000 7f39c6b83640 1 --2- 172.21.15.86:0/2040476512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7ebd364f0 0x55c7ebd33a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:43:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.138+0000 7f39c6382640 1 --2- 172.21.15.86:0/2040476512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c7ebd28b60 0x55c7ebd35f70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f39b0009230 tx=0x7f39b0009260 comp rx=0 tx=0).ready entity=mon.2 client_cookie=bdba08670a163439 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.138+0000 7f39a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-04-21T08:43:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.138+0000 7f39a9ffb640 1 -- 172.21.15.86:0/2040476512 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b004f030 con 0x55c7ebd28b60 2024-04-21T08:43:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.138+0000 7f39cb1d2740 1 -- 172.21.15.86:0/2040476512 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39bc0a4fa0 con 0x55c7ebd28b60 2024-04-21T08:43:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.138+0000 7f39a9ffb640 1 -- 172.21.15.86:0/2040476512 <== mon.2 v2:172.21.15.88:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f39b00046c0 con 0x55c7ebd28b60 2024-04-21T08:43:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.138+0000 7f39a9ffb640 1 -- 172.21.15.86:0/2040476512 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f39b005c050 con 0x55c7ebd28b60 2024-04-21T08:43:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.138+0000 7f39cb1d2740 10 client.26684 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.138+0000 7f39cb1d2740 1 -- 172.21.15.86:0/2040476512 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c7eba6c620 con 0x55c7ebd28b60 2024-04-21T08:43:56.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.138+0000 7f39a9ffb640 1 -- 172.21.15.86:0/2040476512 <== mon.2 v2:172.21.15.88:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39bc0a4fa0 con 0x55c7ebd28b60 2024-04-21T08:43:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.139+0000 7f39a9ffb640 1 -- 172.21.15.86:0/2040476512 <== mon.2 v2:172.21.15.88:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f39b0003b90 con 0x55c7ebd28b60 2024-04-21T08:43:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.139+0000 7f39cb1d2740 10 client.26684 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.139+0000 7f39cb1d2740 10 client.26684 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.139+0000 7f39cb1d2740 20 client.26684 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.139+0000 7f39cb1d2740 1 --2- 172.21.15.86:0/2040476512 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c7ebd46630 0x55c7ebd48a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.139+0000 7f39cb1d2740 4 client.26684 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:43:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.139+0000 7f39cb1d2740 1 -- 172.21.15.86:0/2040476512 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c7ebca1880 con 0x55c7ebd46630 2024-04-21T08:43:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.139+0000 7f39c5b81640 1 --2- 172.21.15.86:0/2040476512 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c7ebd46630 0x55c7ebd48a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.139+0000 7f39a9ffb640 10 client.26684 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.139+0000 7f39c5b81640 1 --2- 172.21.15.86:0/2040476512 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c7ebd46630 0x55c7ebd48a20 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2620e14658c89aa4 server_cookie=4f8bd8a18e489b60 in_seq=0 out_seq=0 2024-04-21T08:43:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.140+0000 7f39a9ffb640 1 -- 172.21.15.86:0/2040476512 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c7ebca1880 con 0x55c7ebd46630 2024-04-21T08:43:56.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.140+0000 7f39a9ffb640 10 client.26684 handle_command_reply: tid=0 2024-04-21T08:43:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.140+0000 7f39cb1d2740 1 client.26684 shutdown 2024-04-21T08:43:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.140+0000 7f39cb1d2740 1 -- 172.21.15.86:0/2040476512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c7ebd28b60 msgr2=0x55c7ebd35f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.140+0000 7f39cb1d2740 1 --2- 172.21.15.86:0/2040476512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c7ebd28b60 0x55c7ebd35f70 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f39b0009230 tx=0x7f39b0009260 comp rx=0 tx=0).stop 2024-04-21T08:43:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.140+0000 7f39cb1d2740 1 -- 172.21.15.86:0/2040476512 shutdown_connections 2024-04-21T08:43:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.140+0000 7f39cb1d2740 1 --2- 172.21.15.86:0/2040476512 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c7ebd46630 0x55c7ebd48a20 crc :-1 s=READY pgs=175 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.140+0000 7f39cb1d2740 1 --2- 172.21.15.86:0/2040476512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7ebd364f0 0x55c7ebd33a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.140+0000 7f39cb1d2740 1 --2- 172.21.15.86:0/2040476512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7ebd298d0 0x55c7ebd33300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.140+0000 7f39cb1d2740 1 --2- 172.21.15.86:0/2040476512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c7ebd28b60 0x55c7ebd35f70 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.140+0000 7f39a9ffb640 0 client.26684 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.140+0000 7f39cb1d2740 1 -- 172.21.15.86:0/2040476512 >> 172.21.15.86:0/2040476512 conn(0x55c7ebcefc90 msgr2=0x55c7ebd255a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.140+0000 7f39cb1d2740 1 -- 172.21.15.86:0/2040476512 shutdown_connections 2024-04-21T08:43:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.140+0000 7f39cb1d2740 1 -- 172.21.15.86:0/2040476512 wait complete. 2024-04-21T08:43:56.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.140+0000 7f39cb1d2740 20 client.26684 trim_cache size 0 max 16384 2024-04-21T08:43:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.157+0000 7f39cb1d2740 1 Processor -- start 2024-04-21T08:43:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.157+0000 7f39cb1d2740 1 -- start start 2024-04-21T08:43:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.157+0000 7f39cb1d2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c7ebd28b60 0x55c7ebd42120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:56.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.157+0000 7f39cb1d2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7ebd298d0 0x55c7ebd1ea20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.157+0000 7f39cb1d2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7ebd364f0 0x55c7ebd322e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.157+0000 7f39cb1d2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f39c01ad090 con 0x55c7ebd298d0 2024-04-21T08:43:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.157+0000 7f39cb1d2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f39c01aebd0 con 0x55c7ebd28b60 2024-04-21T08:43:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.157+0000 7f39cb1d2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f39c01ad760 con 0x55c7ebd364f0 2024-04-21T08:43:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.157+0000 7f39c5b81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7ebd298d0 0x55c7ebd1ea20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.157+0000 7f39c5b81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7ebd298d0 0x55c7ebd1ea20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42896/0 (socket says 172.21.15.86:42896) 2024-04-21T08:43:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.157+0000 7f39c5b81640 1 -- 172.21.15.86:0/3704633417 learned_addr learned my addr 172.21.15.86:0/3704633417 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.157+0000 7f39c6b83640 1 --2- 172.21.15.86:0/3704633417 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7ebd364f0 0x55c7ebd322e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.157+0000 7f39c6382640 1 --2- 172.21.15.86:0/3704633417 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c7ebd28b60 0x55c7ebd42120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.157+0000 7f39c5b81640 1 -- 172.21.15.86:0/3704633417 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7ebd364f0 msgr2=0x55c7ebd322e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.157+0000 7f39c5b81640 1 --2- 172.21.15.86:0/3704633417 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7ebd364f0 0x55c7ebd322e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.157+0000 7f39c5b81640 1 -- 172.21.15.86:0/3704633417 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c7ebd28b60 msgr2=0x55c7ebd42120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:56.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.157+0000 7f39c5b81640 1 --2- 172.21.15.86:0/3704633417 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c7ebd28b60 0x55c7ebd42120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.157+0000 7f39c5b81640 1 -- 172.21.15.86:0/3704633417 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c7ebd48d90 con 0x55c7ebd298d0 2024-04-21T08:43:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.158+0000 7f39c5b81640 1 --2- 172.21.15.86:0/3704633417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7ebd298d0 0x55c7ebd1ea20 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f39bc09a800 tx=0x7f39bc000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd89b16eff9c8ad1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.158+0000 7f39abfff640 1 -- 172.21.15.86:0/3704633417 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc0b5040 con 0x55c7ebd298d0 2024-04-21T08:43:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.158+0000 7f39abfff640 1 -- 172.21.15.86:0/3704633417 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39bc0989a0 con 0x55c7ebd298d0 2024-04-21T08:43:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.158+0000 7f39abfff640 1 -- 172.21.15.86:0/3704633417 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39bc0a0170 con 0x55c7ebd298d0 2024-04-21T08:43:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.159+0000 7f39cb1d2740 1 -- 172.21.15.86:0/3704633417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7ebd298d0 msgr2=0x55c7ebd1ea20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.159+0000 7f39cb1d2740 1 --2- 172.21.15.86:0/3704633417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7ebd298d0 0x55c7ebd1ea20 secure :-1 s=READY pgs=1240 cs=0 l=1 rev1=1 crypto rx=0x7f39bc09a800 tx=0x7f39bc000bc0 comp rx=0 tx=0).stop 2024-04-21T08:43:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.159+0000 7f39cb1d2740 1 -- 172.21.15.86:0/3704633417 shutdown_connections 2024-04-21T08:43:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.159+0000 7f39cb1d2740 1 --2- 172.21.15.86:0/3704633417 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7ebd364f0 0x55c7ebd322e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.159+0000 7f39cb1d2740 1 --2- 172.21.15.86:0/3704633417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7ebd298d0 0x55c7ebd1ea20 unknown :-1 s=CLOSED pgs=1240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.159+0000 7f39cb1d2740 1 --2- 172.21.15.86:0/3704633417 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c7ebd28b60 0x55c7ebd42120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.159+0000 7f39cb1d2740 1 -- 172.21.15.86:0/3704633417 >> 172.21.15.86:0/3704633417 conn(0x55c7ebcefc90 msgr2=0x55c7ebd31bd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.159+0000 7f39cb1d2740 1 -- 172.21.15.86:0/3704633417 shutdown_connections 2024-04-21T08:43:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.159+0000 7f39cb1d2740 1 -- 172.21.15.86:0/3704633417 wait complete. 2024-04-21T08:43:56.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.159+0000 7f39cb1d2740 1 -- start start 2024-04-21T08:43:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.159+0000 7f39cb1d2740 1 Processor -- start 2024-04-21T08:43:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.159+0000 7f39cb1d2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7ebd28b60 0x55c7ebd36ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.159+0000 7f39cb1d2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c7ebd298d0 0x55c7ebd37410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.159+0000 7f39cb1d2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7ebd364f0 0x55c7ebd3d630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.159+0000 7f39cb1d2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f39c01ae500 con 0x55c7ebd28b60 2024-04-21T08:43:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.159+0000 7f39cb1d2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f39c01ade30 con 0x55c7ebd298d0 2024-04-21T08:43:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.159+0000 7f39cb1d2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f39c01ab050 con 0x55c7ebd364f0 2024-04-21T08:43:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.159+0000 7f39cb1d2740 10 client.0 _ll_register_callbacks cb 0x7f39b002c7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:43:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.159+0000 7f39c6382640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7ebd28b60 0x55c7ebd36ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.159+0000 7f39c6382640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7ebd28b60 0x55c7ebd36ed0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42904/0 (socket says 172.21.15.86:42904) 2024-04-21T08:43:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.159+0000 7f39c6382640 1 -- 172.21.15.86:0/3434917320 learned_addr learned my addr 172.21.15.86:0/3434917320 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:43:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.160+0000 7f39c5b81640 1 --2- 172.21.15.86:0/3434917320 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c7ebd298d0 0x55c7ebd37410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.160+0000 7f39c6b83640 1 --2- 172.21.15.86:0/3434917320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7ebd364f0 0x55c7ebd3d630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:56.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.160+0000 7f39c6382640 1 -- 172.21.15.86:0/3434917320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7ebd364f0 msgr2=0x55c7ebd3d630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.160+0000 7f39c6382640 1 --2- 172.21.15.86:0/3434917320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7ebd364f0 0x55c7ebd3d630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.160+0000 7f39c6382640 1 -- 172.21.15.86:0/3434917320 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c7ebd298d0 msgr2=0x55c7ebd37410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:43:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.160+0000 7f39c6382640 1 --2- 172.21.15.86:0/3434917320 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c7ebd298d0 0x55c7ebd37410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.160+0000 7f39c6382640 1 -- 172.21.15.86:0/3434917320 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c7ebd3db70 con 0x55c7ebd28b60 2024-04-21T08:43:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.160+0000 7f39c6b83640 1 --2- 172.21.15.86:0/3434917320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7ebd364f0 0x55c7ebd3d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:43:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.160+0000 7f39c6382640 1 --2- 172.21.15.86:0/3434917320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7ebd28b60 0x55c7ebd36ed0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f39b002c170 tx=0x7f39b00038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8291f9d1710a7ae3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:43:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.160+0000 7f39a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:43:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.160+0000 7f39a9ffb640 1 -- 172.21.15.86:0/3434917320 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f39b004f040 con 0x55c7ebd28b60 2024-04-21T08:43:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.160+0000 7f39a9ffb640 1 -- 172.21.15.86:0/3434917320 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f39b0003c90 con 0x55c7ebd28b60 2024-04-21T08:43:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.160+0000 7f39cb1d2740 1 -- 172.21.15.86:0/3434917320 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f39bc0989a0 con 0x55c7ebd28b60 2024-04-21T08:43:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.160+0000 7f39a9ffb640 1 -- 172.21.15.86:0/3434917320 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f39b00046c0 con 0x55c7ebd28b60 2024-04-21T08:43:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.160+0000 7f39cb1d2740 10 client.17898 fetch_fsmap learned FSMap version 29 2024-04-21T08:43:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.160+0000 7f39cb1d2740 1 -- 172.21.15.86:0/3434917320 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c7ebd3ee80 con 0x55c7ebd28b60 2024-04-21T08:43:56.165 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.160+0000 7f39a9ffb640 1 -- 172.21.15.86:0/3434917320 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f39bc0989a0 con 0x55c7ebd28b60 2024-04-21T08:43:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.161+0000 7f39a9ffb640 1 -- 172.21.15.86:0/3434917320 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f39b0040750 con 0x55c7ebd28b60 2024-04-21T08:43:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.161+0000 7f39cb1d2740 10 client.17898 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:43:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.161+0000 7f39cb1d2740 10 client.17898 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:43:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.161+0000 7f39cb1d2740 20 client.17898 populate_metadata read hostname 'smithi086' 2024-04-21T08:43:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.161+0000 7f39cb1d2740 1 --2- 172.21.15.86:0/3434917320 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c7ebdacfc0 0x55c7ebdaf3e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:43:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.161+0000 7f39cb1d2740 4 client.17898 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:43:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.161+0000 7f39cb1d2740 1 -- 172.21.15.86:0/3434917320 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c7ebd289b0 con 0x55c7ebdacfc0 2024-04-21T08:43:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.161+0000 7f39c5b81640 1 --2- 172.21.15.86:0/3434917320 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c7ebdacfc0 0x55c7ebdaf3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:43:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.161+0000 7f39c5b81640 1 --2- 172.21.15.86:0/3434917320 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c7ebdacfc0 0x55c7ebdaf3e0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=310411f4405e03c1 server_cookie=cc140466bc0d1af3 in_seq=0 out_seq=0 2024-04-21T08:43:56.166 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.161+0000 7f39a9ffb640 10 client.17898 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.172+0000 7f39a9ffb640 1 -- 172.21.15.86:0/3434917320 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c7ebd289b0 con 0x55c7ebdacfc0 2024-04-21T08:43:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.172+0000 7f39a9ffb640 10 client.17898 handle_command_reply: tid=0 2024-04-21T08:43:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.172+0000 7f39cb1d2740 1 client.17898 shutdown 2024-04-21T08:43:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.172+0000 7f39cb1d2740 1 -- 172.21.15.86:0/3434917320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7ebd28b60 msgr2=0x55c7ebd36ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.172+0000 7f39cb1d2740 1 --2- 172.21.15.86:0/3434917320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7ebd28b60 0x55c7ebd36ed0 secure :-1 s=READY pgs=1241 cs=0 l=1 rev1=1 crypto rx=0x7f39b002c170 tx=0x7f39b00038a0 comp rx=0 tx=0).stop 2024-04-21T08:43:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.172+0000 7f39cb1d2740 1 -- 172.21.15.86:0/3434917320 shutdown_connections 2024-04-21T08:43:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.172+0000 7f39cb1d2740 1 --2- 172.21.15.86:0/3434917320 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c7ebdacfc0 0x55c7ebdaf3e0 crc :-1 s=READY pgs=176 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.172+0000 7f39cb1d2740 1 --2- 172.21.15.86:0/3434917320 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c7ebd364f0 0x55c7ebd3d630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:56.172 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.172+0000 7f39cb1d2740 1 --2- 172.21.15.86:0/3434917320 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c7ebd298d0 0x55c7ebd37410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.172+0000 7f39a9ffb640 0 client.17898 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:43:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.172+0000 7f39cb1d2740 1 --2- 172.21.15.86:0/3434917320 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c7ebd28b60 0x55c7ebd36ed0 unknown :-1 s=CLOSED pgs=1241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.172+0000 7f39cb1d2740 1 -- 172.21.15.86:0/3434917320 >> 172.21.15.86:0/3434917320 conn(0x55c7ebcefc90 msgr2=0x55c7ebd32460 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.172+0000 7f39cb1d2740 1 -- 172.21.15.86:0/3434917320 shutdown_connections 2024-04-21T08:43:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.172+0000 7f39cb1d2740 1 -- 172.21.15.86:0/3434917320 wait complete. 2024-04-21T08:43:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.172+0000 7f39cb1d2740 20 client.17898 trim_cache size 0 max 16384 2024-04-21T08:43:56.173 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:43:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:43:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:43:56.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:43:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.173+0000 7f39c85e6640 1 -- 172.21.15.86:0/2044985447 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f39ac0a3f60 msgr2=0x7f39ac0a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.173+0000 7f39c85e6640 1 --2- 172.21.15.86:0/2044985447 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f39ac0a3f60 0x7f39ac0a6420 secure :-1 s=READY pgs=194 cs=0 l=1 rev1=1 crypto rx=0x7f39b0000c00 tx=0x7f39b0004040 comp rx=0 tx=0).stop 2024-04-21T08:43:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.173+0000 7f39c85e6640 1 -- 172.21.15.86:0/2044985447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39c01061a0 msgr2=0x7f39c01999c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:43:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.173+0000 7f39c85e6640 1 --2- 172.21.15.86:0/2044985447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39c01061a0 0x7f39c01999c0 secure :-1 s=READY pgs=1238 cs=0 l=1 rev1=1 crypto rx=0x7f39bc00c970 tx=0x7f39bc00ce30 comp rx=0 tx=0).stop 2024-04-21T08:43:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.173+0000 7f39c85e6640 1 -- 172.21.15.86:0/2044985447 shutdown_connections 2024-04-21T08:43:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.173+0000 7f39c85e6640 1 --2- 172.21.15.86:0/2044985447 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f39ac0a3f60 0x7f39ac0a6420 unknown :-1 s=CLOSED pgs=194 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.173+0000 7f39c85e6640 1 --2- 172.21.15.86:0/2044985447 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f39c0199f50 0x7f39c01a4500 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.173+0000 7f39c85e6640 1 --2- 172.21.15.86:0/2044985447 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f39c01061a0 0x7f39c01999c0 unknown :-1 s=CLOSED pgs=1238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.173+0000 7f39c85e6640 1 --2- 172.21.15.86:0/2044985447 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f39c0101120 0x7f39c0199480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:43:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.173+0000 7f39c85e6640 1 -- 172.21.15.86:0/2044985447 >> 172.21.15.86:0/2044985447 conn(0x7f39c0077fa0 msgr2=0x7f39c00fedd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:43:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.173+0000 7f39c85e6640 1 -- 172.21.15.86:0/2044985447 shutdown_connections 2024-04-21T08:43:56.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:43:56.173+0000 7f39c85e6640 1 -- 172.21.15.86:0/2044985447 wait complete. 2024-04-21T08:43:56.213 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:55 smithi088 ceph-mon[31259]: pgmap v157: 129 pgs: 129 active+clean; 2.9 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 KiB/s wr, 17 op/s 2024-04-21T08:43:56.213 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:55 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:43:56.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:55 smithi179 ceph-mon[31829]: pgmap v157: 129 pgs: 129 active+clean; 2.9 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 KiB/s wr, 17 op/s 2024-04-21T08:43:56.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:55 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:43:57.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:56 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:43:57.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:56 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:43:57.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:56 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:43:57.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:56 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:43:57.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:56 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:43:57.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:56 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:43:57.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:56 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:43:57.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:56 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:43:57.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:56 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:43:58.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:43:58 smithi086 ceph-mon[27939]: pgmap v158: 129 pgs: 129 active+clean; 2.9 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 KiB/s wr, 17 op/s 2024-04-21T08:43:58.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:43:58 smithi088 ceph-mon[31259]: pgmap v158: 129 pgs: 129 active+clean; 2.9 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 KiB/s wr, 17 op/s 2024-04-21T08:43:58.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:43:58 smithi179 ceph-mon[31829]: pgmap v158: 129 pgs: 129 active+clean; 2.9 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 36 KiB/s wr, 17 op/s 2024-04-21T08:44:00.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:00 smithi086 ceph-mon[27939]: pgmap v159: 129 pgs: 129 active+clean; 3.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 57 KiB/s wr, 26 op/s 2024-04-21T08:44:00.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:00 smithi088 ceph-mon[31259]: pgmap v159: 129 pgs: 129 active+clean; 3.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 57 KiB/s wr, 26 op/s 2024-04-21T08:44:00.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:00 smithi179 ceph-mon[31829]: pgmap v159: 129 pgs: 129 active+clean; 3.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 54 KiB/s rd, 57 KiB/s wr, 26 op/s 2024-04-21T08:44:02.548 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:02 smithi179 ceph-mon[31829]: pgmap v160: 129 pgs: 129 active+clean; 3.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-21T08:44:02.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:02 smithi086 ceph-mon[27939]: pgmap v160: 129 pgs: 129 active+clean; 3.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-21T08:44:02.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:02 smithi088 ceph-mon[31259]: pgmap v160: 129 pgs: 129 active+clean; 3.1 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 35 KiB/s wr, 15 op/s 2024-04-21T08:44:04.516 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:04 smithi088 ceph-mon[31259]: pgmap v161: 129 pgs: 129 active+clean; 3.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 73 KiB/s wr, 36 op/s 2024-04-21T08:44:04.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:04 smithi086 ceph-mon[27939]: pgmap v161: 129 pgs: 129 active+clean; 3.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 73 KiB/s wr, 36 op/s 2024-04-21T08:44:04.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:04 smithi179 ceph-mon[31829]: pgmap v161: 129 pgs: 129 active+clean; 3.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 73 KiB/s wr, 36 op/s 2024-04-21T08:44:06.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:06 smithi086 ceph-mon[27939]: pgmap v162: 129 pgs: 129 active+clean; 3.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 58 KiB/s wr, 30 op/s 2024-04-21T08:44:06.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:06 smithi088 ceph-mon[31259]: pgmap v162: 129 pgs: 129 active+clean; 3.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 58 KiB/s wr, 30 op/s 2024-04-21T08:44:06.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:06 smithi179 ceph-mon[31829]: pgmap v162: 129 pgs: 129 active+clean; 3.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 58 KiB/s wr, 30 op/s 2024-04-21T08:44:08.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:08 smithi086 ceph-mon[27939]: pgmap v163: 129 pgs: 129 active+clean; 3.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 58 KiB/s wr, 30 op/s 2024-04-21T08:44:08.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:08 smithi088 ceph-mon[31259]: pgmap v163: 129 pgs: 129 active+clean; 3.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 58 KiB/s wr, 30 op/s 2024-04-21T08:44:08.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:08 smithi179 ceph-mon[31829]: pgmap v163: 129 pgs: 129 active+clean; 3.3 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 58 KiB/s wr, 30 op/s 2024-04-21T08:44:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.269+0000 7fed4f77e640 1 -- 172.21.15.86:0/724901214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed48101aa0 msgr2=0x7fed48105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.269+0000 7fed4f77e640 1 --2- 172.21.15.86:0/724901214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed48101aa0 0x7fed48105a90 secure :-1 s=READY pgs=1242 cs=0 l=1 rev1=1 crypto rx=0x7fed44009900 tx=0x7fed4402bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.269+0000 7fed4f77e640 1 -- 172.21.15.86:0/724901214 shutdown_connections 2024-04-21T08:44:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.269+0000 7fed4f77e640 1 --2- 172.21.15.86:0/724901214 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed48106170 0x7fed4810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.269+0000 7fed4f77e640 1 --2- 172.21.15.86:0/724901214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed48101aa0 0x7fed48105a90 unknown :-1 s=CLOSED pgs=1242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.269+0000 7fed4f77e640 1 --2- 172.21.15.86:0/724901214 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed481010f0 0x7fed481014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.269+0000 7fed4f77e640 1 -- 172.21.15.86:0/724901214 >> 172.21.15.86:0/724901214 conn(0x7fed48077fd0 msgr2=0x7fed480783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.269+0000 7fed4f77e640 1 -- 172.21.15.86:0/724901214 shutdown_connections 2024-04-21T08:44:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.269+0000 7fed4f77e640 1 -- 172.21.15.86:0/724901214 wait complete. 2024-04-21T08:44:09.271 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.270+0000 7fed4f77e640 1 Processor -- start 2024-04-21T08:44:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.270+0000 7fed4f77e640 1 -- start start 2024-04-21T08:44:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.270+0000 7fed4f77e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed481010f0 0x7fed481994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.270+0000 7fed4f77e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed48106170 0x7fed481999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.270+0000 7fed4f77e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed48199fd0 0x7fed481a4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.270+0000 7fed4f77e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fed481110b0 con 0x7fed48199fd0 2024-04-21T08:44:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.270+0000 7fed4f77e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fed48110f30 con 0x7fed48106170 2024-04-21T08:44:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.270+0000 7fed4f77e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fed48111230 con 0x7fed481010f0 2024-04-21T08:44:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.270+0000 7fed4dd1b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed48199fd0 0x7fed481a4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.270+0000 7fed4dd1b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed48199fd0 0x7fed481a4530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36022/0 (socket says 172.21.15.86:36022) 2024-04-21T08:44:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.270+0000 7fed4dd1b640 1 -- 172.21.15.86:0/3978922323 learned_addr learned my addr 172.21.15.86:0/3978922323 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.270+0000 7fed4cd19640 1 --2- 172.21.15.86:0/3978922323 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed48106170 0x7fed481999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.270+0000 7fed4d51a640 1 --2- 172.21.15.86:0/3978922323 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed481010f0 0x7fed481994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.270+0000 7fed4dd1b640 1 -- 172.21.15.86:0/3978922323 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed481010f0 msgr2=0x7fed481994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:09.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.270+0000 7fed4dd1b640 1 --2- 172.21.15.86:0/3978922323 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed481010f0 0x7fed481994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.270+0000 7fed4dd1b640 1 -- 172.21.15.86:0/3978922323 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed48106170 msgr2=0x7fed481999f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.270+0000 7fed4dd1b640 1 --2- 172.21.15.86:0/3978922323 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed48106170 0x7fed481999f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.270+0000 7fed4dd1b640 1 -- 172.21.15.86:0/3978922323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fed481a4d50 con 0x7fed48199fd0 2024-04-21T08:44:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.271+0000 7fed4dd1b640 1 --2- 172.21.15.86:0/3978922323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed48199fd0 0x7fed481a4530 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fed4000eb10 tx=0x7fed4000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.271+0000 7fed3e7fc640 1 -- 172.21.15.86:0/3978922323 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed40004b20 con 0x7fed48199fd0 2024-04-21T08:44:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.271+0000 7fed3e7fc640 1 -- 172.21.15.86:0/3978922323 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed40004cc0 con 0x7fed48199fd0 2024-04-21T08:44:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.271+0000 7fed4f77e640 1 -- 172.21.15.86:0/3978922323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fed481a5040 con 0x7fed48199fd0 2024-04-21T08:44:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.271+0000 7fed3e7fc640 1 -- 172.21.15.86:0/3978922323 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed40010670 con 0x7fed48199fd0 2024-04-21T08:44:09.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.271+0000 7fed4f77e640 1 -- 172.21.15.86:0/3978922323 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fed481a5510 con 0x7fed48199fd0 2024-04-21T08:44:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.274+0000 7fed3e7fc640 1 -- 172.21.15.86:0/3978922323 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fed40005020 con 0x7fed48199fd0 2024-04-21T08:44:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.274+0000 7fed5236c740 1 Processor -- start 2024-04-21T08:44:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.274+0000 7fed5236c740 1 -- start start 2024-04-21T08:44:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.274+0000 7fed5236c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fe58d1b8d0 0x55fe58d1a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.274+0000 7fed5236c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe58d1ab60 0x55fe58d1afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.274+0000 7fed5236c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe58d240e0 0x55fe58d264d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.274+0000 7fed5236c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fe58c7f8d0 con 0x55fe58d240e0 2024-04-21T08:44:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.274+0000 7fed5236c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55fe58c1ce80 con 0x55fe58d1b8d0 2024-04-21T08:44:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.274+0000 7fed5236c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55fe58c7b890 con 0x55fe58d1ab60 2024-04-21T08:44:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.274+0000 7fed3e7fc640 1 --2- 172.21.15.86:0/3978922323 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fed340a3dc0 0x7fed340a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.275+0000 7fed4cd19640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe58d240e0 0x55fe58d264d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:09.280 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.275+0000 7fed4dd1b640 1 --2- 172.21.15.86:0/3978922323 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fed340a3dc0 0x7fed340a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.275+0000 7fed4cd19640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe58d240e0 0x55fe58d264d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36032/0 (socket says 172.21.15.86:36032) 2024-04-21T08:44:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.275+0000 7fed4cd19640 1 -- 172.21.15.86:0/574720550 learned_addr learned my addr 172.21.15.86:0/574720550 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.275+0000 7fed4d51a640 1 --2- 172.21.15.86:0/574720550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe58d1ab60 0x55fe58d1afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.275+0000 7fed4cd19640 1 --2- 172.21.15.86:0/574720550 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fe58d1b8d0 0x55fe58d1a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.275+0000 7fed3e7fc640 1 -- 172.21.15.86:0/3978922323 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fed400d1050 con 0x7fed48199fd0 2024-04-21T08:44:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.275+0000 7fed4cd19640 1 -- 172.21.15.86:0/574720550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe58d1ab60 msgr2=0x55fe58d1afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.275+0000 7fed4cd19640 1 --2- 172.21.15.86:0/574720550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe58d1ab60 0x55fe58d1afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.275+0000 7fed4cd19640 1 -- 172.21.15.86:0/574720550 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fe58d1b8d0 msgr2=0x55fe58d1a620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.275+0000 7fed4cd19640 1 --2- 172.21.15.86:0/574720550 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fe58d1b8d0 0x55fe58d1a620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.275+0000 7fed4cd19640 1 -- 172.21.15.86:0/574720550 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe58c3e320 con 0x55fe58d240e0 2024-04-21T08:44:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.275+0000 7fed4dd1b640 1 --2- 172.21.15.86:0/3978922323 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fed340a3dc0 0x7fed340a6280 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fed4009cc10 tx=0x7fed4006f770 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.275+0000 7fed4cd19640 1 --2- 172.21.15.86:0/574720550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe58d240e0 0x55fe58d264d0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fed44009900 tx=0x7fed44005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d848bad5bd0b03 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.275+0000 7fed337fe640 1 -- 172.21.15.86:0/574720550 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed44047020 con 0x55fe58d240e0 2024-04-21T08:44:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.275+0000 7fed337fe640 1 -- 172.21.15.86:0/574720550 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed4402cc90 con 0x55fe58d240e0 2024-04-21T08:44:09.281 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.275+0000 7fed337fe640 1 -- 172.21.15.86:0/574720550 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed440046c0 con 0x55fe58d240e0 2024-04-21T08:44:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.276+0000 7fed5236c740 1 -- 172.21.15.86:0/574720550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe58d240e0 msgr2=0x55fe58d264d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.276+0000 7fed5236c740 1 --2- 172.21.15.86:0/574720550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe58d240e0 0x55fe58d264d0 secure :-1 s=READY pgs=1244 cs=0 l=1 rev1=1 crypto rx=0x7fed44009900 tx=0x7fed44005e80 comp rx=0 tx=0).stop 2024-04-21T08:44:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.276+0000 7fed5236c740 1 -- 172.21.15.86:0/574720550 shutdown_connections 2024-04-21T08:44:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.276+0000 7fed5236c740 1 --2- 172.21.15.86:0/574720550 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe58d240e0 0x55fe58d264d0 unknown :-1 s=CLOSED pgs=1244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.276+0000 7fed5236c740 1 --2- 172.21.15.86:0/574720550 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe58d1ab60 0x55fe58d1afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.276+0000 7fed5236c740 1 --2- 172.21.15.86:0/574720550 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fe58d1b8d0 0x55fe58d1a620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.276+0000 7fed5236c740 1 -- 172.21.15.86:0/574720550 >> 172.21.15.86:0/574720550 conn(0x55fe58ce1c90 msgr2=0x55fe58d15850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.276+0000 7fed5236c740 1 -- 172.21.15.86:0/574720550 shutdown_connections 2024-04-21T08:44:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.276+0000 7fed5236c740 1 -- 172.21.15.86:0/574720550 wait complete. 2024-04-21T08:44:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.276+0000 7fed5236c740 1 -- start start 2024-04-21T08:44:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.277+0000 7fed5236c740 1 Processor -- start 2024-04-21T08:44:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.277+0000 7fed5236c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fe58d1ab60 0x55fe58d23ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:09.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.277+0000 7fed5236c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe58d1b8d0 0x55fe58d20e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.277+0000 7fed5236c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe58d240e0 0x55fe58d21460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.277+0000 7fed5236c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fe58d26db0 con 0x55fe58d240e0 2024-04-21T08:44:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.277+0000 7fed5236c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55fe58c07eb0 con 0x55fe58d1ab60 2024-04-21T08:44:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.277+0000 7fed5236c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55fe58d26f30 con 0x55fe58d1b8d0 2024-04-21T08:44:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.277+0000 7fed5236c740 10 client.0 _ll_register_callbacks cb 0x55fe58c34b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.277+0000 7fed4cd19640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe58d240e0 0x55fe58d21460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.277+0000 7fed4cd19640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe58d240e0 0x55fe58d21460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36044/0 (socket says 172.21.15.86:36044) 2024-04-21T08:44:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.277+0000 7fed4cd19640 1 -- 172.21.15.86:0/3959199778 learned_addr learned my addr 172.21.15.86:0/3959199778 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.277+0000 7fed4cd19640 1 --2- 172.21.15.86:0/3959199778 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fe58d1ab60 0x55fe58d23ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.277+0000 7fed4d51a640 1 --2- 172.21.15.86:0/3959199778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe58d1b8d0 0x55fe58d20e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.277+0000 7fed4cd19640 1 -- 172.21.15.86:0/3959199778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe58d1b8d0 msgr2=0x55fe58d20e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.277+0000 7fed4cd19640 1 --2- 172.21.15.86:0/3959199778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe58d1b8d0 0x55fe58d20e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.277+0000 7fed4cd19640 1 -- 172.21.15.86:0/3959199778 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fe58d1ab60 msgr2=0x55fe58d23ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.277+0000 7fed4cd19640 1 --2- 172.21.15.86:0/3959199778 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fe58d1ab60 0x55fe58d23ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.277+0000 7fed4cd19640 1 -- 172.21.15.86:0/3959199778 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe58992210 con 0x55fe58d240e0 2024-04-21T08:44:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.278+0000 7fed4cd19640 1 --2- 172.21.15.86:0/3959199778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe58d240e0 0x55fe58d21460 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fed4402c170 tx=0x7fed44005360 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f40f2d99d0bcc8ff server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.278+0000 7fed30ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.278+0000 7fed30ff9640 1 -- 172.21.15.86:0/3959199778 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed44051020 con 0x55fe58d240e0 2024-04-21T08:44:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.278+0000 7fed30ff9640 1 -- 172.21.15.86:0/3959199778 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fed440048c0 con 0x55fe58d240e0 2024-04-21T08:44:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.278+0000 7fed5236c740 1 -- 172.21.15.86:0/3959199778 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed4402cc90 con 0x55fe58d240e0 2024-04-21T08:44:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.278+0000 7fed30ff9640 1 -- 172.21.15.86:0/3959199778 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fed4405f050 con 0x55fe58d240e0 2024-04-21T08:44:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.278+0000 7fed5236c740 10 client.17910 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.278+0000 7fed5236c740 1 -- 172.21.15.86:0/3959199778 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fe58a5e620 con 0x55fe58d240e0 2024-04-21T08:44:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.278+0000 7fed30ff9640 1 -- 172.21.15.86:0/3959199778 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed4402cc90 con 0x55fe58d240e0 2024-04-21T08:44:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.278+0000 7fed30ff9640 1 -- 172.21.15.86:0/3959199778 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fed44003a30 con 0x55fe58d240e0 2024-04-21T08:44:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.278+0000 7fed5236c740 10 client.17910 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.278+0000 7fed5236c740 10 client.17910 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.278+0000 7fed5236c740 20 client.17910 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.278+0000 7fed5236c740 1 --2- 172.21.15.86:0/3959199778 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55fe58d33f90 0x55fe58d36380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.278+0000 7fed5236c740 4 client.17910 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:09.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.278+0000 7fed5236c740 1 -- 172.21.15.86:0/3959199778 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fe58c93880 con 0x55fe58d33f90 2024-04-21T08:44:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.279+0000 7fed4d51a640 1 --2- 172.21.15.86:0/3959199778 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55fe58d33f90 0x55fe58d36380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.279+0000 7fed30ff9640 10 client.17910 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.279+0000 7fed4d51a640 1 --2- 172.21.15.86:0/3959199778 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55fe58d33f90 0x55fe58d36380 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6834384257e3bc35 server_cookie=c2fc549dfc40a82b in_seq=0 out_seq=0 2024-04-21T08:44:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.280+0000 7fed30ff9640 1 -- 172.21.15.86:0/3959199778 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fe58c93880 con 0x55fe58d33f90 2024-04-21T08:44:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.280+0000 7fed30ff9640 10 client.17910 handle_command_reply: tid=0 2024-04-21T08:44:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.280+0000 7fed5236c740 1 client.17910 shutdown 2024-04-21T08:44:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.280+0000 7fed5236c740 1 -- 172.21.15.86:0/3959199778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe58d240e0 msgr2=0x55fe58d21460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.280+0000 7fed5236c740 1 --2- 172.21.15.86:0/3959199778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe58d240e0 0x55fe58d21460 secure :-1 s=READY pgs=1245 cs=0 l=1 rev1=1 crypto rx=0x7fed4402c170 tx=0x7fed44005360 comp rx=0 tx=0).stop 2024-04-21T08:44:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.280+0000 7fed5236c740 1 -- 172.21.15.86:0/3959199778 shutdown_connections 2024-04-21T08:44:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.280+0000 7fed5236c740 1 --2- 172.21.15.86:0/3959199778 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55fe58d33f90 0x55fe58d36380 crc :-1 s=READY pgs=177 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.280+0000 7fed5236c740 1 --2- 172.21.15.86:0/3959199778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe58d240e0 0x55fe58d21460 unknown :-1 s=CLOSED pgs=1245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.280+0000 7fed5236c740 1 --2- 172.21.15.86:0/3959199778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe58d1b8d0 0x55fe58d20e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.280+0000 7fed5236c740 1 --2- 172.21.15.86:0/3959199778 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fe58d1ab60 0x55fe58d23ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.280+0000 7fed30ff9640 0 client.17910 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:09.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.280+0000 7fed5236c740 1 -- 172.21.15.86:0/3959199778 >> 172.21.15.86:0/3959199778 conn(0x55fe58ce1c90 msgr2=0x55fe58d175a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.280+0000 7fed5236c740 1 -- 172.21.15.86:0/3959199778 shutdown_connections 2024-04-21T08:44:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.280+0000 7fed5236c740 1 -- 172.21.15.86:0/3959199778 wait complete. 2024-04-21T08:44:09.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.280+0000 7fed5236c740 20 client.17910 trim_cache size 0 max 16384 2024-04-21T08:44:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.295+0000 7fed5236c740 1 Processor -- start 2024-04-21T08:44:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.295+0000 7fed5236c740 1 -- start start 2024-04-21T08:44:09.299 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.295+0000 7fed5236c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fe58d1ab60 0x55fe58d2fce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.295+0000 7fed5236c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe58d1b8d0 0x55fe58d1fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.295+0000 7fed5236c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe58d240e0 0x55fe58d23580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.295+0000 7fed5236c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fed481af140 con 0x55fe58d240e0 2024-04-21T08:44:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.295+0000 7fed5236c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fed481b0c80 con 0x55fe58d1ab60 2024-04-21T08:44:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.295+0000 7fed5236c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fed481af810 con 0x55fe58d1b8d0 2024-04-21T08:44:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.295+0000 7fed4cd19640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe58d240e0 0x55fe58d23580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.296+0000 7fed4cd19640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe58d240e0 0x55fe58d23580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36050/0 (socket says 172.21.15.86:36050) 2024-04-21T08:44:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.296+0000 7fed4cd19640 1 -- 172.21.15.86:0/3341729793 learned_addr learned my addr 172.21.15.86:0/3341729793 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.296+0000 7fed4cd19640 1 --2- 172.21.15.86:0/3341729793 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fe58d1ab60 0x55fe58d2fce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.296+0000 7fed4d51a640 1 --2- 172.21.15.86:0/3341729793 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe58d1b8d0 0x55fe58d1fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.296+0000 7fed4cd19640 1 -- 172.21.15.86:0/3341729793 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe58d1b8d0 msgr2=0x55fe58d1fe30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.296+0000 7fed4cd19640 1 --2- 172.21.15.86:0/3341729793 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe58d1b8d0 0x55fe58d1fe30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.296+0000 7fed4cd19640 1 -- 172.21.15.86:0/3341729793 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fe58d1ab60 msgr2=0x55fe58d2fce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:09.300 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.296+0000 7fed4cd19640 1 --2- 172.21.15.86:0/3341729793 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fe58d1ab60 0x55fe58d2fce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.296+0000 7fed4cd19640 1 -- 172.21.15.86:0/3341729793 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fe58d19e30 con 0x55fe58d240e0 2024-04-21T08:44:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.296+0000 7fed4d51a640 1 --2- 172.21.15.86:0/3341729793 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe58d1b8d0 0x55fe58d1fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.296+0000 7fed4cd19640 1 --2- 172.21.15.86:0/3341729793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe58d240e0 0x55fe58d23580 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fed44000c00 tx=0x7fed440040d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=91e2db6fc1e01f90 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.296+0000 7fed32ffd640 1 -- 172.21.15.86:0/3341729793 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed44051040 con 0x55fe58d240e0 2024-04-21T08:44:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.296+0000 7fed32ffd640 1 -- 172.21.15.86:0/3341729793 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed44041d50 con 0x55fe58d240e0 2024-04-21T08:44:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.296+0000 7fed32ffd640 1 -- 172.21.15.86:0/3341729793 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed44047070 con 0x55fe58d240e0 2024-04-21T08:44:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.297+0000 7fed5236c740 1 -- 172.21.15.86:0/3341729793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe58d240e0 msgr2=0x55fe58d23580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.297+0000 7fed5236c740 1 --2- 172.21.15.86:0/3341729793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe58d240e0 0x55fe58d23580 secure :-1 s=READY pgs=1246 cs=0 l=1 rev1=1 crypto rx=0x7fed44000c00 tx=0x7fed440040d0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.297+0000 7fed5236c740 1 -- 172.21.15.86:0/3341729793 shutdown_connections 2024-04-21T08:44:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.297+0000 7fed5236c740 1 --2- 172.21.15.86:0/3341729793 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe58d240e0 0x55fe58d23580 unknown :-1 s=CLOSED pgs=1246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.297+0000 7fed5236c740 1 --2- 172.21.15.86:0/3341729793 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe58d1b8d0 0x55fe58d1fe30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.297+0000 7fed5236c740 1 --2- 172.21.15.86:0/3341729793 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fe58d1ab60 0x55fe58d2fce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.297+0000 7fed5236c740 1 -- 172.21.15.86:0/3341729793 >> 172.21.15.86:0/3341729793 conn(0x55fe58ce1c90 msgr2=0x55fe58d1f720 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.297+0000 7fed5236c740 1 -- 172.21.15.86:0/3341729793 shutdown_connections 2024-04-21T08:44:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.297+0000 7fed5236c740 1 -- 172.21.15.86:0/3341729793 wait complete. 2024-04-21T08:44:09.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.297+0000 7fed5236c740 1 -- start start 2024-04-21T08:44:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.297+0000 7fed5236c740 1 Processor -- start 2024-04-21T08:44:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.297+0000 7fed5236c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe58d1ab60 0x55fe58d24bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.298+0000 7fed5236c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe58d1b8d0 0x55fe58d250f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.298+0000 7fed5236c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fe58d240e0 0x55fe58d25630 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.298+0000 7fed5236c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fed481b05b0 con 0x55fe58d1b8d0 2024-04-21T08:44:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.298+0000 7fed5236c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fed481afee0 con 0x55fe58d240e0 2024-04-21T08:44:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.298+0000 7fed5236c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fed481ad100 con 0x55fe58d1ab60 2024-04-21T08:44:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.298+0000 7fed5236c740 10 client.0 _ll_register_callbacks cb 0x7fed38009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.298+0000 7fed4d51a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe58d1b8d0 0x55fe58d250f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.298+0000 7fed4d51a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe58d1b8d0 0x55fe58d250f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36062/0 (socket says 172.21.15.86:36062) 2024-04-21T08:44:09.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.298+0000 7fed4d51a640 1 -- 172.21.15.86:0/1324671319 learned_addr learned my addr 172.21.15.86:0/1324671319 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.298+0000 7fed4cd19640 1 --2- 172.21.15.86:0/1324671319 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fe58d240e0 0x55fe58d25630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.298+0000 7fed4cd19640 1 --2- 172.21.15.86:0/1324671319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe58d1ab60 0x55fe58d24bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.298+0000 7fed4d51a640 1 -- 172.21.15.86:0/1324671319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe58d1ab60 msgr2=0x55fe58d24bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.298+0000 7fed4d51a640 1 --2- 172.21.15.86:0/1324671319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe58d1ab60 0x55fe58d24bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.298+0000 7fed4d51a640 1 -- 172.21.15.86:0/1324671319 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fe58d240e0 msgr2=0x55fe58d25630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.298+0000 7fed4d51a640 1 --2- 172.21.15.86:0/1324671319 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fe58d240e0 0x55fe58d25630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.298+0000 7fed4d51a640 1 -- 172.21.15.86:0/1324671319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fe58d2a2c0 con 0x55fe58d1b8d0 2024-04-21T08:44:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.298+0000 7fed4cd19640 1 --2- 172.21.15.86:0/1324671319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe58d1ab60 0x55fe58d24bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.298+0000 7fed4d51a640 1 --2- 172.21.15.86:0/1324671319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe58d1b8d0 0x55fe58d250f0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x55fe58d206b0 tx=0x7fed38004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7224b5b1c7cb60e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.298+0000 7fed30ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.298+0000 7fed30ff9640 1 -- 172.21.15.86:0/1324671319 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fed38026060 con 0x55fe58d1b8d0 2024-04-21T08:44:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.298+0000 7fed30ff9640 1 -- 172.21.15.86:0/1324671319 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fed38005050 con 0x55fe58d1b8d0 2024-04-21T08:44:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.298+0000 7fed5236c740 1 -- 172.21.15.86:0/1324671319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fed44041d50 con 0x55fe58d1b8d0 2024-04-21T08:44:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.298+0000 7fed30ff9640 1 -- 172.21.15.86:0/1324671319 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fed3802d050 con 0x55fe58d1b8d0 2024-04-21T08:44:09.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.298+0000 7fed5236c740 10 client.17916 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.299+0000 7fed5236c740 1 -- 172.21.15.86:0/1324671319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fe58d9a3d0 con 0x55fe58d1b8d0 2024-04-21T08:44:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.299+0000 7fed30ff9640 1 -- 172.21.15.86:0/1324671319 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fed44041d50 con 0x55fe58d1b8d0 2024-04-21T08:44:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.299+0000 7fed30ff9640 1 -- 172.21.15.86:0/1324671319 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fed38020110 con 0x55fe58d1b8d0 2024-04-21T08:44:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.299+0000 7fed5236c740 10 client.17916 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.299+0000 7fed5236c740 10 client.17916 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.299+0000 7fed5236c740 20 client.17916 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.299+0000 7fed5236c740 1 --2- 172.21.15.86:0/1324671319 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55fe58d9ab30 0x55fe58d9cf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.299+0000 7fed5236c740 4 client.17916 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:44:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.299+0000 7fed5236c740 1 -- 172.21.15.86:0/1324671319 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55fe58c1f3d0 con 0x55fe58d9ab30 2024-04-21T08:44:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.299+0000 7fed4cd19640 1 --2- 172.21.15.86:0/1324671319 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55fe58d9ab30 0x55fe58d9cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.300+0000 7fed30ff9640 10 client.17916 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:09.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.300+0000 7fed4cd19640 1 --2- 172.21.15.86:0/1324671319 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55fe58d9ab30 0x55fe58d9cf50 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fce7afb3a9e5bb96 server_cookie=49b4db4cda81a0d in_seq=0 out_seq=0 2024-04-21T08:44:09.314 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.315+0000 7fed30ff9640 1 -- 172.21.15.86:0/1324671319 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55fe58c1f3d0 con 0x55fe58d9ab30 2024-04-21T08:44:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.315+0000 7fed30ff9640 10 client.17916 handle_command_reply: tid=0 2024-04-21T08:44:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.315+0000 7fed5236c740 1 client.17916 shutdown 2024-04-21T08:44:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.315+0000 7fed5236c740 1 -- 172.21.15.86:0/1324671319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe58d1b8d0 msgr2=0x55fe58d250f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.315+0000 7fed5236c740 1 --2- 172.21.15.86:0/1324671319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe58d1b8d0 0x55fe58d250f0 secure :-1 s=READY pgs=1247 cs=0 l=1 rev1=1 crypto rx=0x55fe58d206b0 tx=0x7fed38004410 comp rx=0 tx=0).stop 2024-04-21T08:44:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.315+0000 7fed5236c740 1 -- 172.21.15.86:0/1324671319 shutdown_connections 2024-04-21T08:44:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.315+0000 7fed5236c740 1 --2- 172.21.15.86:0/1324671319 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55fe58d9ab30 0x55fe58d9cf50 crc :-1 s=READY pgs=178 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.315+0000 7fed5236c740 1 --2- 172.21.15.86:0/1324671319 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fe58d240e0 0x55fe58d25630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.315+0000 7fed5236c740 1 --2- 172.21.15.86:0/1324671319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fe58d1b8d0 0x55fe58d250f0 unknown :-1 s=CLOSED pgs=1247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.315+0000 7fed30ff9640 0 client.17916 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.315+0000 7fed5236c740 1 --2- 172.21.15.86:0/1324671319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fe58d1ab60 0x55fe58d24bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.315+0000 7fed5236c740 1 -- 172.21.15.86:0/1324671319 >> 172.21.15.86:0/1324671319 conn(0x55fe58ce1c90 msgr2=0x55fe58d16c30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.315+0000 7fed5236c740 1 -- 172.21.15.86:0/1324671319 shutdown_connections 2024-04-21T08:44:09.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.315+0000 7fed5236c740 1 -- 172.21.15.86:0/1324671319 wait complete. 2024-04-21T08:44:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.315+0000 7fed5236c740 20 client.17916 trim_cache size 0 max 16384 2024-04-21T08:44:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:44:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:44:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:44:09.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:44:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.315+0000 7fed4f77e640 1 -- 172.21.15.86:0/3978922323 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fed340a3dc0 msgr2=0x7fed340a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.315+0000 7fed4f77e640 1 --2- 172.21.15.86:0/3978922323 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fed340a3dc0 0x7fed340a6280 secure :-1 s=READY pgs=195 cs=0 l=1 rev1=1 crypto rx=0x7fed4009cc10 tx=0x7fed4006f770 comp rx=0 tx=0).stop 2024-04-21T08:44:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.315+0000 7fed4f77e640 1 -- 172.21.15.86:0/3978922323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed48199fd0 msgr2=0x7fed481a4530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.315+0000 7fed4f77e640 1 --2- 172.21.15.86:0/3978922323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed48199fd0 0x7fed481a4530 secure :-1 s=READY pgs=1243 cs=0 l=1 rev1=1 crypto rx=0x7fed4000eb10 tx=0x7fed4000c2d0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.316+0000 7fed4f77e640 1 -- 172.21.15.86:0/3978922323 shutdown_connections 2024-04-21T08:44:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.316+0000 7fed4f77e640 1 --2- 172.21.15.86:0/3978922323 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fed340a3dc0 0x7fed340a6280 unknown :-1 s=CLOSED pgs=195 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.316+0000 7fed4f77e640 1 --2- 172.21.15.86:0/3978922323 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fed48199fd0 0x7fed481a4530 unknown :-1 s=CLOSED pgs=1243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.316+0000 7fed4f77e640 1 --2- 172.21.15.86:0/3978922323 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fed48106170 0x7fed481999f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.316+0000 7fed4f77e640 1 --2- 172.21.15.86:0/3978922323 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fed481010f0 0x7fed481994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.316+0000 7fed4f77e640 1 -- 172.21.15.86:0/3978922323 >> 172.21.15.86:0/3978922323 conn(0x7fed48077fd0 msgr2=0x7fed480ff000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.316+0000 7fed4f77e640 1 -- 172.21.15.86:0/3978922323 shutdown_connections 2024-04-21T08:44:09.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:09.316+0000 7fed4f77e640 1 -- 172.21.15.86:0/3978922323 wait complete. 2024-04-21T08:44:10.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:10 smithi086 ceph-mon[27939]: pgmap v164: 129 pgs: 129 active+clean; 3.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 75 KiB/s wr, 37 op/s 2024-04-21T08:44:10.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:10 smithi088 ceph-mon[31259]: pgmap v164: 129 pgs: 129 active+clean; 3.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 75 KiB/s wr, 37 op/s 2024-04-21T08:44:10.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:10 smithi179 ceph-mon[31829]: pgmap v164: 129 pgs: 129 active+clean; 3.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 75 KiB/s wr, 37 op/s 2024-04-21T08:44:12.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:12 smithi086 ceph-mon[27939]: pgmap v165: 129 pgs: 129 active+clean; 3.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 55 KiB/s wr, 27 op/s 2024-04-21T08:44:12.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:12 smithi088 ceph-mon[31259]: pgmap v165: 129 pgs: 129 active+clean; 3.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 55 KiB/s wr, 27 op/s 2024-04-21T08:44:12.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:12 smithi179 ceph-mon[31829]: pgmap v165: 129 pgs: 129 active+clean; 3.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 55 KiB/s wr, 27 op/s 2024-04-21T08:44:14.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:14 smithi086 ceph-mon[27939]: pgmap v166: 129 pgs: 129 active+clean; 3.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 79 KiB/s wr, 37 op/s 2024-04-21T08:44:14.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:14 smithi088 ceph-mon[31259]: pgmap v166: 129 pgs: 129 active+clean; 3.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 79 KiB/s wr, 37 op/s 2024-04-21T08:44:14.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:14 smithi179 ceph-mon[31829]: pgmap v166: 129 pgs: 129 active+clean; 3.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 79 KiB/s wr, 37 op/s 2024-04-21T08:44:15.500 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:44:15.500 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:44:15.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.614+0000 7f1a22406640 1 -- 172.21.15.86:0/486368332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a1c108060 msgr2=0x7f1a1c108440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:15.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.614+0000 7f1a22406640 1 --2- 172.21.15.86:0/486368332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a1c108060 0x7f1a1c108440 secure :-1 s=READY pgs=1248 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c009930 tx=0x7f1a0c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:15.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.614+0000 7f1a22406640 1 -- 172.21.15.86:0/486368332 shutdown_connections 2024-04-21T08:44:15.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.614+0000 7f1a22406640 1 --2- 172.21.15.86:0/486368332 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a1c06af70 0x7f1a1c10ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.614+0000 7f1a22406640 1 --2- 172.21.15.86:0/486368332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a1c06a5b0 0x7f1a1c06aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.614+0000 7f1a22406640 1 --2- 172.21.15.86:0/486368332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a1c108060 0x7f1a1c108440 unknown :-1 s=CLOSED pgs=1248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.614+0000 7f1a22406640 1 -- 172.21.15.86:0/486368332 >> 172.21.15.86:0/486368332 conn(0x7f1a1c077c40 msgr2=0x7f1a1c078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:15.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.614+0000 7f1a22406640 1 -- 172.21.15.86:0/486368332 shutdown_connections 2024-04-21T08:44:15.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.614+0000 7f1a22406640 1 -- 172.21.15.86:0/486368332 wait complete. 2024-04-21T08:44:15.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.614+0000 7f1a22406640 1 Processor -- start 2024-04-21T08:44:15.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.614+0000 7f1a22406640 1 -- start start 2024-04-21T08:44:15.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.615+0000 7f1a22406640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a1c06a5b0 0x7f1a1c101170 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:15.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.615+0000 7f1a22406640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a1c06af70 0x7f1a1c1016b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:15.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.615+0000 7f1a22406640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a1c108060 0x7f1a1c101bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:15.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.615+0000 7f1a22406640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1c1121a0 con 0x7f1a1c108060 2024-04-21T08:44:15.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.615+0000 7f1a22406640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1c112020 con 0x7f1a1c06af70 2024-04-21T08:44:15.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.615+0000 7f1a22406640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1c112320 con 0x7f1a1c06a5b0 2024-04-21T08:44:15.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.615+0000 7f1a209a3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a1c108060 0x7f1a1c101bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:15.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.615+0000 7f1a209a3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a1c108060 0x7f1a1c101bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34640/0 (socket says 172.21.15.86:34640) 2024-04-21T08:44:15.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.615+0000 7f1a209a3640 1 -- 172.21.15.86:0/1348289832 learned_addr learned my addr 172.21.15.86:0/1348289832 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:15.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.615+0000 7f1a1bfff640 1 --2- 172.21.15.86:0/1348289832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a1c06a5b0 0x7f1a1c101170 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:15.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.615+0000 7f1a1b7fe640 1 --2- 172.21.15.86:0/1348289832 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a1c06af70 0x7f1a1c1016b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:15.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.615+0000 7f1a209a3640 1 -- 172.21.15.86:0/1348289832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a1c06a5b0 msgr2=0x7f1a1c101170 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:15.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.615+0000 7f1a209a3640 1 --2- 172.21.15.86:0/1348289832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a1c06a5b0 0x7f1a1c101170 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.615+0000 7f1a209a3640 1 -- 172.21.15.86:0/1348289832 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a1c06af70 msgr2=0x7f1a1c1016b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:15.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.615+0000 7f1a209a3640 1 --2- 172.21.15.86:0/1348289832 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a1c06af70 0x7f1a1c1016b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.615+0000 7f1a209a3640 1 -- 172.21.15.86:0/1348289832 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1a1c1a6ba0 con 0x7f1a1c108060 2024-04-21T08:44:15.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.615+0000 7f1a1b7fe640 1 --2- 172.21.15.86:0/1348289832 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a1c06af70 0x7f1a1c1016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:15.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.615+0000 7f1a209a3640 1 --2- 172.21.15.86:0/1348289832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a1c108060 0x7f1a1c101bf0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f1a1400cad0 tx=0x7f1a1400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:15.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.615+0000 7f1a197fa640 1 -- 172.21.15.86:0/1348289832 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1400ae20 con 0x7f1a1c108060 2024-04-21T08:44:15.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.616+0000 7f1a197fa640 1 -- 172.21.15.86:0/1348289832 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a14005020 con 0x7f1a1c108060 2024-04-21T08:44:15.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.616+0000 7f1a22406640 1 -- 172.21.15.86:0/1348289832 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1a1c1a6e90 con 0x7f1a1c108060 2024-04-21T08:44:15.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.616+0000 7f1a197fa640 1 -- 172.21.15.86:0/1348289832 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1400f6b0 con 0x7f1a1c108060 2024-04-21T08:44:15.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.616+0000 7f1a22406640 1 -- 172.21.15.86:0/1348289832 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1a1c1a7360 con 0x7f1a1c108060 2024-04-21T08:44:15.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.618+0000 7f1a197fa640 1 -- 172.21.15.86:0/1348289832 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1a140051d0 con 0x7f1a1c108060 2024-04-21T08:44:15.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.618+0000 7f1a24ff6740 1 Processor -- start 2024-04-21T08:44:15.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.618+0000 7f1a24ff6740 1 -- start start 2024-04-21T08:44:15.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.618+0000 7f1a24ff6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635db5a9580 0x5635db5a9960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:15.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.618+0000 7f1a197fa640 1 --2- 172.21.15.86:0/1348289832 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1a040a3d50 0x7f1a040a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:15.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.618+0000 7f1a24ff6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635db5a9ea0 0x5635db5b2540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:15.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.618+0000 7f1a24ff6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635db5b2a80 0x5635db5b4e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:15.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.618+0000 7f1a24ff6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5635db50e1e0 con 0x5635db5a9580 2024-04-21T08:44:15.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.618+0000 7f1a24ff6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5635db4ab390 con 0x5635db5b2a80 2024-04-21T08:44:15.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.618+0000 7f1a24ff6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5635db50a1a0 con 0x5635db5a9ea0 2024-04-21T08:44:15.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.618+0000 7f1a1b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635db5a9580 0x5635db5a9960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:15.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.618+0000 7f1a1bfff640 1 --2- 172.21.15.86:0/1348289832 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1a040a3d50 0x7f1a040a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:15.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.618+0000 7f1a1b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635db5a9580 0x5635db5a9960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34656/0 (socket says 172.21.15.86:34656) 2024-04-21T08:44:15.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.618+0000 7f1a1b7fe640 1 -- 172.21.15.86:0/4109490200 learned_addr learned my addr 172.21.15.86:0/4109490200 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:15.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.618+0000 7f1a197fa640 1 -- 172.21.15.86:0/1348289832 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f1a140d0b50 con 0x7f1a1c108060 2024-04-21T08:44:15.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.619+0000 7f1a209a3640 1 --2- 172.21.15.86:0/4109490200 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635db5b2a80 0x5635db5b4e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:15.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.619+0000 7f1a1b7fe640 1 --2- 172.21.15.86:0/4109490200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635db5a9ea0 0x5635db5b2540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:15.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.619+0000 7f1a1b7fe640 1 -- 172.21.15.86:0/4109490200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635db5a9ea0 msgr2=0x5635db5b2540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:15.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.619+0000 7f1a1b7fe640 1 --2- 172.21.15.86:0/4109490200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635db5a9ea0 0x5635db5b2540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.619+0000 7f1a1b7fe640 1 -- 172.21.15.86:0/4109490200 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635db5b2a80 msgr2=0x5635db5b4e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:15.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.619+0000 7f1a1b7fe640 1 --2- 172.21.15.86:0/4109490200 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635db5b2a80 0x5635db5b4e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.619+0000 7f1a1b7fe640 1 -- 172.21.15.86:0/4109490200 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635db2ea7d0 con 0x5635db5a9580 2024-04-21T08:44:15.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.619+0000 7f1a209a3640 1 --2- 172.21.15.86:0/4109490200 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635db5b2a80 0x5635db5b4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:15.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.619+0000 7f1a1b7fe640 1 --2- 172.21.15.86:0/4109490200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635db5a9580 0x5635db5a9960 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f1a100144c0 tx=0x7f1a1000b530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=82dd828aadd6ed50 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:15.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.619+0000 7f1a1bfff640 1 --2- 172.21.15.86:0/1348289832 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1a040a3d50 0x7f1a040a6210 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c005ab0 tx=0x7f1a0c005fd0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:15.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.619+0000 7f1a027fc640 1 -- 172.21.15.86:0/4109490200 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a10012710 con 0x5635db5a9580 2024-04-21T08:44:15.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.619+0000 7f1a027fc640 1 -- 172.21.15.86:0/4109490200 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a1001a070 con 0x5635db5a9580 2024-04-21T08:44:15.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.619+0000 7f1a027fc640 1 -- 172.21.15.86:0/4109490200 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1000e9d0 con 0x5635db5a9580 2024-04-21T08:44:15.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.620+0000 7f1a24ff6740 1 -- 172.21.15.86:0/4109490200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635db5a9580 msgr2=0x5635db5a9960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:15.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.620+0000 7f1a24ff6740 1 --2- 172.21.15.86:0/4109490200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635db5a9580 0x5635db5a9960 secure :-1 s=READY pgs=1250 cs=0 l=1 rev1=1 crypto rx=0x7f1a100144c0 tx=0x7f1a1000b530 comp rx=0 tx=0).stop 2024-04-21T08:44:15.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.620+0000 7f1a24ff6740 1 -- 172.21.15.86:0/4109490200 shutdown_connections 2024-04-21T08:44:15.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.620+0000 7f1a24ff6740 1 --2- 172.21.15.86:0/4109490200 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635db5b2a80 0x5635db5b4e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.620+0000 7f1a24ff6740 1 --2- 172.21.15.86:0/4109490200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635db5a9ea0 0x5635db5b2540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.620+0000 7f1a24ff6740 1 --2- 172.21.15.86:0/4109490200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635db5a9580 0x5635db5a9960 unknown :-1 s=CLOSED pgs=1250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.620+0000 7f1a24ff6740 1 -- 172.21.15.86:0/4109490200 >> 172.21.15.86:0/4109490200 conn(0x5635db5a3cb0 msgr2=0x5635db5a4090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:15.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.620+0000 7f1a24ff6740 1 -- 172.21.15.86:0/4109490200 shutdown_connections 2024-04-21T08:44:15.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.620+0000 7f1a24ff6740 1 -- 172.21.15.86:0/4109490200 wait complete. 2024-04-21T08:44:15.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.620+0000 7f1a24ff6740 1 -- start start 2024-04-21T08:44:15.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.620+0000 7f1a24ff6740 1 Processor -- start 2024-04-21T08:44:15.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.621+0000 7f1a24ff6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635db5a9580 0x5635db5afab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:15.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.621+0000 7f1a24ff6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635db5a9ea0 0x5635db5afff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:15.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.621+0000 7f1a24ff6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635db5b2a80 0x5635db5b0530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:15.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.621+0000 7f1a24ff6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5635db5b5870 con 0x5635db5b2a80 2024-04-21T08:44:15.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.621+0000 7f1a24ff6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5635db4c72f0 con 0x5635db5a9580 2024-04-21T08:44:15.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.621+0000 7f1a24ff6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5635db5b59f0 con 0x5635db5a9ea0 2024-04-21T08:44:15.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.621+0000 7f1a24ff6740 10 client.0 _ll_register_callbacks cb 0x5635db4d4580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:15.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.621+0000 7f1a209a3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635db5b2a80 0x5635db5b0530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:15.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.621+0000 7f1a209a3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635db5b2a80 0x5635db5b0530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34664/0 (socket says 172.21.15.86:34664) 2024-04-21T08:44:15.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.621+0000 7f1a209a3640 1 -- 172.21.15.86:0/1359734956 learned_addr learned my addr 172.21.15.86:0/1359734956 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:15.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.621+0000 7f1a1b7fe640 1 --2- 172.21.15.86:0/1359734956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635db5a9ea0 0x5635db5afff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:15.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.621+0000 7f1a1bfff640 1 --2- 172.21.15.86:0/1359734956 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635db5a9580 0x5635db5afab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:15.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.621+0000 7f1a209a3640 1 -- 172.21.15.86:0/1359734956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635db5a9ea0 msgr2=0x5635db5afff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:15.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.621+0000 7f1a209a3640 1 --2- 172.21.15.86:0/1359734956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635db5a9ea0 0x5635db5afff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.621+0000 7f1a209a3640 1 -- 172.21.15.86:0/1359734956 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635db5a9580 msgr2=0x5635db5afab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:15.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.621+0000 7f1a209a3640 1 --2- 172.21.15.86:0/1359734956 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635db5a9580 0x5635db5afab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.621+0000 7f1a209a3640 1 -- 172.21.15.86:0/1359734956 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635db2203f0 con 0x5635db5b2a80 2024-04-21T08:44:15.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.621+0000 7f1a1bfff640 1 --2- 172.21.15.86:0/1359734956 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635db5a9580 0x5635db5afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:15.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.621+0000 7f1a209a3640 1 --2- 172.21.15.86:0/1359734956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635db5b2a80 0x5635db5b0530 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f1a1409c710 tx=0x7f1a140afae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45ec2b599074a2b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:15.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.621+0000 7f19fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:15.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.621+0000 7f19fb7fe640 1 -- 172.21.15.86:0/1359734956 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1401d070 con 0x5635db5b2a80 2024-04-21T08:44:15.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.621+0000 7f19fb7fe640 1 -- 172.21.15.86:0/1359734956 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f1a140054f0 con 0x5635db5b2a80 2024-04-21T08:44:15.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.621+0000 7f1a24ff6740 1 -- 172.21.15.86:0/1359734956 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a1001a070 con 0x5635db5b2a80 2024-04-21T08:44:15.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.622+0000 7f19fb7fe640 1 -- 172.21.15.86:0/1359734956 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1a1409c780 con 0x5635db5b2a80 2024-04-21T08:44:15.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.622+0000 7f1a24ff6740 10 client.17928 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:15.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.622+0000 7f1a24ff6740 1 -- 172.21.15.86:0/1359734956 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5635db1e00f0 con 0x5635db5b2a80 2024-04-21T08:44:15.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.622+0000 7f19fb7fe640 1 -- 172.21.15.86:0/1359734956 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a1001a070 con 0x5635db5b2a80 2024-04-21T08:44:15.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.622+0000 7f19fb7fe640 1 -- 172.21.15.86:0/1359734956 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1a14098040 con 0x5635db5b2a80 2024-04-21T08:44:15.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.622+0000 7f1a24ff6740 10 client.17928 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:15.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.622+0000 7f1a24ff6740 10 client.17928 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:15.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.622+0000 7f1a24ff6740 20 client.17928 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:15.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.622+0000 7f1a24ff6740 1 --2- 172.21.15.86:0/1359734956 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5635db5c2a30 0x5635db5c4e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:15.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.622+0000 7f1a24ff6740 4 client.17928 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:15.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.622+0000 7f1a24ff6740 1 -- 172.21.15.86:0/1359734956 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5635db51bc00 con 0x5635db5c2a30 2024-04-21T08:44:15.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.622+0000 7f1a1bfff640 1 --2- 172.21.15.86:0/1359734956 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5635db5c2a30 0x5635db5c4e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:15.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.623+0000 7f1a1bfff640 1 --2- 172.21.15.86:0/1359734956 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5635db5c2a30 0x5635db5c4e40 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d176962ebb24ddef server_cookie=2fb890f53d87fa11 in_seq=0 out_seq=0 2024-04-21T08:44:15.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.623+0000 7f19fb7fe640 10 client.17928 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:15.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.623+0000 7f19fb7fe640 1 -- 172.21.15.86:0/1359734956 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5635db51bc00 con 0x5635db5c2a30 2024-04-21T08:44:15.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.623+0000 7f19fb7fe640 10 client.17928 handle_command_reply: tid=0 2024-04-21T08:44:15.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.624+0000 7f1a24ff6740 1 client.17928 shutdown 2024-04-21T08:44:15.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.624+0000 7f1a24ff6740 1 -- 172.21.15.86:0/1359734956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635db5b2a80 msgr2=0x5635db5b0530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:15.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.624+0000 7f1a24ff6740 1 --2- 172.21.15.86:0/1359734956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635db5b2a80 0x5635db5b0530 secure :-1 s=READY pgs=1251 cs=0 l=1 rev1=1 crypto rx=0x7f1a1409c710 tx=0x7f1a140afae0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.624+0000 7f1a24ff6740 1 -- 172.21.15.86:0/1359734956 shutdown_connections 2024-04-21T08:44:15.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.624+0000 7f1a24ff6740 1 --2- 172.21.15.86:0/1359734956 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5635db5c2a30 0x5635db5c4e40 crc :-1 s=READY pgs=179 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.624+0000 7f1a24ff6740 1 --2- 172.21.15.86:0/1359734956 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635db5b2a80 0x5635db5b0530 unknown :-1 s=CLOSED pgs=1251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.624+0000 7f1a24ff6740 1 --2- 172.21.15.86:0/1359734956 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635db5a9ea0 0x5635db5afff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.624+0000 7f1a24ff6740 1 --2- 172.21.15.86:0/1359734956 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635db5a9580 0x5635db5afab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.624+0000 7f19fb7fe640 0 client.17928 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:15.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.624+0000 7f1a24ff6740 1 -- 172.21.15.86:0/1359734956 >> 172.21.15.86:0/1359734956 conn(0x5635db5a3cb0 msgr2=0x5635db5a5a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:15.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.624+0000 7f1a24ff6740 1 -- 172.21.15.86:0/1359734956 shutdown_connections 2024-04-21T08:44:15.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.624+0000 7f1a24ff6740 1 -- 172.21.15.86:0/1359734956 wait complete. 2024-04-21T08:44:15.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.624+0000 7f1a24ff6740 20 client.17928 trim_cache size 0 max 16384 2024-04-21T08:44:15.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.641+0000 7f1a24ff6740 1 Processor -- start 2024-04-21T08:44:15.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.641+0000 7f1a24ff6740 1 -- start start 2024-04-21T08:44:15.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.641+0000 7f1a24ff6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635db5a9580 0x5635db5b21e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:15.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.641+0000 7f1a24ff6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635db5a9ea0 0x5635db5be730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.641+0000 7f1a24ff6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635db5b2a80 0x5635db5a2bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.641+0000 7f1a24ff6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1c1b1530 con 0x5635db5b2a80 2024-04-21T08:44:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.641+0000 7f1a24ff6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1c1b3070 con 0x5635db5a9ea0 2024-04-21T08:44:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.641+0000 7f1a24ff6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1c1b1c00 con 0x5635db5a9580 2024-04-21T08:44:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.641+0000 7f1a209a3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635db5b2a80 0x5635db5a2bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.641+0000 7f1a209a3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635db5b2a80 0x5635db5a2bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34674/0 (socket says 172.21.15.86:34674) 2024-04-21T08:44:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.641+0000 7f1a209a3640 1 -- 172.21.15.86:0/2039582951 learned_addr learned my addr 172.21.15.86:0/2039582951 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.641+0000 7f1a1b7fe640 1 --2- 172.21.15.86:0/2039582951 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635db5a9ea0 0x5635db5be730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.642+0000 7f1a1bfff640 1 --2- 172.21.15.86:0/2039582951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635db5a9580 0x5635db5b21e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.642+0000 7f1a209a3640 1 -- 172.21.15.86:0/2039582951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635db5a9580 msgr2=0x5635db5b21e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.642+0000 7f1a209a3640 1 --2- 172.21.15.86:0/2039582951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635db5a9580 0x5635db5b21e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.642+0000 7f1a209a3640 1 -- 172.21.15.86:0/2039582951 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635db5a9ea0 msgr2=0x5635db5be730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.642+0000 7f1a209a3640 1 --2- 172.21.15.86:0/2039582951 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635db5a9ea0 0x5635db5be730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.642+0000 7f1a209a3640 1 -- 172.21.15.86:0/2039582951 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5635db5a9cd0 con 0x5635db5b2a80 2024-04-21T08:44:15.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.642+0000 7f1a209a3640 1 --2- 172.21.15.86:0/2039582951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635db5b2a80 0x5635db5a2bf0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f1a140a4ec0 tx=0x7f1a140077e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52e2328ab54118cc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:15.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.642+0000 7f1a01ffb640 1 -- 172.21.15.86:0/2039582951 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1401d070 con 0x5635db5b2a80 2024-04-21T08:44:15.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.642+0000 7f1a01ffb640 1 -- 172.21.15.86:0/2039582951 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a140a53f0 con 0x5635db5b2a80 2024-04-21T08:44:15.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.642+0000 7f1a01ffb640 1 -- 172.21.15.86:0/2039582951 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a140986e0 con 0x5635db5b2a80 2024-04-21T08:44:15.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.643+0000 7f1a24ff6740 1 -- 172.21.15.86:0/2039582951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635db5b2a80 msgr2=0x5635db5a2bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:15.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.643+0000 7f1a24ff6740 1 --2- 172.21.15.86:0/2039582951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635db5b2a80 0x5635db5a2bf0 secure :-1 s=READY pgs=1252 cs=0 l=1 rev1=1 crypto rx=0x7f1a140a4ec0 tx=0x7f1a140077e0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.643+0000 7f1a24ff6740 1 -- 172.21.15.86:0/2039582951 shutdown_connections 2024-04-21T08:44:15.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.643+0000 7f1a24ff6740 1 --2- 172.21.15.86:0/2039582951 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635db5b2a80 0x5635db5a2bf0 unknown :-1 s=CLOSED pgs=1252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.643+0000 7f1a24ff6740 1 --2- 172.21.15.86:0/2039582951 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635db5a9ea0 0x5635db5be730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.643+0000 7f1a24ff6740 1 --2- 172.21.15.86:0/2039582951 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635db5a9580 0x5635db5b21e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.643+0000 7f1a24ff6740 1 -- 172.21.15.86:0/2039582951 >> 172.21.15.86:0/2039582951 conn(0x5635db5a3cb0 msgr2=0x5635db5a24e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:15.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.643+0000 7f1a24ff6740 1 -- 172.21.15.86:0/2039582951 shutdown_connections 2024-04-21T08:44:15.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.643+0000 7f1a24ff6740 1 -- 172.21.15.86:0/2039582951 wait complete. 2024-04-21T08:44:15.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.643+0000 7f1a24ff6740 1 -- start start 2024-04-21T08:44:15.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.643+0000 7f1a24ff6740 1 Processor -- start 2024-04-21T08:44:15.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.643+0000 7f1a24ff6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635db5a9580 0x5635db5bb010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:15.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.643+0000 7f1a24ff6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635db5a9ea0 0x5635db5b6330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:15.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.643+0000 7f1a24ff6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635db5b2a80 0x5635db5b6870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:15.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.643+0000 7f1a24ff6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1c1b29a0 con 0x5635db5b2a80 2024-04-21T08:44:15.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.643+0000 7f1a24ff6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1c1b22d0 con 0x5635db5a9580 2024-04-21T08:44:15.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.643+0000 7f1a24ff6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1a1c1af4f0 con 0x5635db5a9ea0 2024-04-21T08:44:15.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.643+0000 7f1a24ff6740 10 client.0 _ll_register_callbacks cb 0x7f1a1409c150 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:15.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.644+0000 7f1a209a3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635db5b2a80 0x5635db5b6870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:15.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.644+0000 7f1a209a3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635db5b2a80 0x5635db5b6870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34678/0 (socket says 172.21.15.86:34678) 2024-04-21T08:44:15.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.644+0000 7f1a209a3640 1 -- 172.21.15.86:0/3268884843 learned_addr learned my addr 172.21.15.86:0/3268884843 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:15.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.644+0000 7f1a1bfff640 1 --2- 172.21.15.86:0/3268884843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635db5a9580 0x5635db5bb010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:15.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.644+0000 7f1a1b7fe640 1 --2- 172.21.15.86:0/3268884843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635db5a9ea0 0x5635db5b6330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:15.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.644+0000 7f1a209a3640 1 -- 172.21.15.86:0/3268884843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635db5a9ea0 msgr2=0x5635db5b6330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:15.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.644+0000 7f1a209a3640 1 --2- 172.21.15.86:0/3268884843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635db5a9ea0 0x5635db5b6330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.644+0000 7f1a209a3640 1 -- 172.21.15.86:0/3268884843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635db5a9580 msgr2=0x5635db5bb010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:15.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.644+0000 7f1a209a3640 1 --2- 172.21.15.86:0/3268884843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635db5a9580 0x5635db5bb010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.644+0000 7f1a209a3640 1 -- 172.21.15.86:0/3268884843 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5635db5b70a0 con 0x5635db5b2a80 2024-04-21T08:44:15.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.644+0000 7f1a209a3640 1 --2- 172.21.15.86:0/3268884843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635db5b2a80 0x5635db5b6870 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f1a1409c940 tx=0x7f1a14005800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4d377b79b63a336c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:15.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.644+0000 7f19fb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:15.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.644+0000 7f19fb7fe640 1 -- 172.21.15.86:0/3268884843 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1a1401d070 con 0x5635db5b2a80 2024-04-21T08:44:15.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.644+0000 7f19fb7fe640 1 -- 172.21.15.86:0/3268884843 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f1a1409bbd0 con 0x5635db5b2a80 2024-04-21T08:44:15.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.644+0000 7f1a24ff6740 1 -- 172.21.15.86:0/3268884843 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1a140a53f0 con 0x5635db5b2a80 2024-04-21T08:44:15.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.644+0000 7f19fb7fe640 1 -- 172.21.15.86:0/3268884843 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1a140bb050 con 0x5635db5b2a80 2024-04-21T08:44:15.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.644+0000 7f1a24ff6740 10 client.17934 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:15.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.644+0000 7f1a24ff6740 1 -- 172.21.15.86:0/3268884843 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5635db628f80 con 0x5635db5b2a80 2024-04-21T08:44:15.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.644+0000 7f19fb7fe640 1 -- 172.21.15.86:0/3268884843 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1a140a53f0 con 0x5635db5b2a80 2024-04-21T08:44:15.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.645+0000 7f19fb7fe640 1 -- 172.21.15.86:0/3268884843 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1a1409b640 con 0x5635db5b2a80 2024-04-21T08:44:15.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.645+0000 7f1a24ff6740 10 client.17934 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:15.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.645+0000 7f1a24ff6740 10 client.17934 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:15.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.645+0000 7f1a24ff6740 20 client.17934 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:15.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.645+0000 7f1a24ff6740 1 --2- 172.21.15.86:0/3268884843 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5635db629320 0x5635db62b740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:15.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.645+0000 7f1a24ff6740 4 client.17934 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:44:15.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.645+0000 7f1a24ff6740 1 -- 172.21.15.86:0/3268884843 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5635db515a70 con 0x5635db629320 2024-04-21T08:44:15.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.645+0000 7f1a1bfff640 1 --2- 172.21.15.86:0/3268884843 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5635db629320 0x5635db62b740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:15.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.645+0000 7f19fb7fe640 10 client.17934 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:15.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.645+0000 7f1a1bfff640 1 --2- 172.21.15.86:0/3268884843 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5635db629320 0x5635db62b740 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c41ca43dd8564bed server_cookie=78c10f8b32c4a4ef in_seq=0 out_seq=0 2024-04-21T08:44:15.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.645+0000 7f19fb7fe640 1 -- 172.21.15.86:0/3268884843 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5635db515a70 con 0x5635db629320 2024-04-21T08:44:15.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.645+0000 7f19fb7fe640 10 client.17934 handle_command_reply: tid=0 2024-04-21T08:44:15.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.645+0000 7f1a24ff6740 1 client.17934 shutdown 2024-04-21T08:44:15.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.646+0000 7f1a24ff6740 1 -- 172.21.15.86:0/3268884843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635db5b2a80 msgr2=0x5635db5b6870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:15.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.646+0000 7f1a24ff6740 1 --2- 172.21.15.86:0/3268884843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635db5b2a80 0x5635db5b6870 secure :-1 s=READY pgs=1253 cs=0 l=1 rev1=1 crypto rx=0x7f1a1409c940 tx=0x7f1a14005800 comp rx=0 tx=0).stop 2024-04-21T08:44:15.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.646+0000 7f1a24ff6740 1 -- 172.21.15.86:0/3268884843 shutdown_connections 2024-04-21T08:44:15.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.646+0000 7f1a24ff6740 1 --2- 172.21.15.86:0/3268884843 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5635db629320 0x5635db62b740 crc :-1 s=READY pgs=180 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.646+0000 7f1a24ff6740 1 --2- 172.21.15.86:0/3268884843 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5635db5b2a80 0x5635db5b6870 unknown :-1 s=CLOSED pgs=1253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.646+0000 7f1a24ff6740 1 --2- 172.21.15.86:0/3268884843 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5635db5a9ea0 0x5635db5b6330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.646+0000 7f19fb7fe640 0 client.17934 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:15.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.646+0000 7f1a24ff6740 1 --2- 172.21.15.86:0/3268884843 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5635db5a9580 0x5635db5bb010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.646+0000 7f1a24ff6740 1 -- 172.21.15.86:0/3268884843 >> 172.21.15.86:0/3268884843 conn(0x5635db5a3cb0 msgr2=0x5635db5b02e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:15.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.646+0000 7f1a24ff6740 1 -- 172.21.15.86:0/3268884843 shutdown_connections 2024-04-21T08:44:15.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.646+0000 7f1a24ff6740 1 -- 172.21.15.86:0/3268884843 wait complete. 2024-04-21T08:44:15.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.646+0000 7f1a24ff6740 20 client.17934 trim_cache size 0 max 16384 2024-04-21T08:44:15.652 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:44:15.652 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-21T08:44:15.652 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-21T08:44:15.652 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:44:15.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.646+0000 7f1a22406640 1 -- 172.21.15.86:0/1348289832 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1a040a3d50 msgr2=0x7f1a040a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:15.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.646+0000 7f1a22406640 1 --2- 172.21.15.86:0/1348289832 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1a040a3d50 0x7f1a040a6210 secure :-1 s=READY pgs=196 cs=0 l=1 rev1=1 crypto rx=0x7f1a0c005ab0 tx=0x7f1a0c005fd0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.646+0000 7f1a22406640 1 -- 172.21.15.86:0/1348289832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a1c108060 msgr2=0x7f1a1c101bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:15.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.646+0000 7f1a22406640 1 --2- 172.21.15.86:0/1348289832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a1c108060 0x7f1a1c101bf0 secure :-1 s=READY pgs=1249 cs=0 l=1 rev1=1 crypto rx=0x7f1a1400cad0 tx=0x7f1a1400cf90 comp rx=0 tx=0).stop 2024-04-21T08:44:15.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.646+0000 7f1a22406640 1 -- 172.21.15.86:0/1348289832 shutdown_connections 2024-04-21T08:44:15.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.646+0000 7f1a22406640 1 --2- 172.21.15.86:0/1348289832 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1a040a3d50 0x7f1a040a6210 unknown :-1 s=CLOSED pgs=196 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.646+0000 7f1a22406640 1 --2- 172.21.15.86:0/1348289832 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1a1c108060 0x7f1a1c101bf0 unknown :-1 s=CLOSED pgs=1249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.646+0000 7f1a22406640 1 --2- 172.21.15.86:0/1348289832 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1a1c06af70 0x7f1a1c1016b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.646+0000 7f1a22406640 1 --2- 172.21.15.86:0/1348289832 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1a1c06a5b0 0x7f1a1c101170 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.646+0000 7f1a22406640 1 -- 172.21.15.86:0/1348289832 >> 172.21.15.86:0/1348289832 conn(0x7f1a1c077c40 msgr2=0x7f1a1c0ffeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:15.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.646+0000 7f1a22406640 1 -- 172.21.15.86:0/1348289832 shutdown_connections 2024-04-21T08:44:15.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.646+0000 7f1a22406640 1 -- 172.21.15.86:0/1348289832 wait complete. 2024-04-21T08:44:15.657 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:44:15.657 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:44:15.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.755+0000 7f56ee9b1640 1 -- 172.21.15.86:0/1009535733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56e8106f10 msgr2=0x7f56e81072f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:15.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.755+0000 7f56ee9b1640 1 --2- 172.21.15.86:0/1009535733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56e8106f10 0x7f56e81072f0 secure :-1 s=READY pgs=1254 cs=0 l=1 rev1=1 crypto rx=0x7f56d8009900 tx=0x7f56d802bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:15.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.755+0000 7f56ee9b1640 1 -- 172.21.15.86:0/1009535733 shutdown_connections 2024-04-21T08:44:15.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.755+0000 7f56ee9b1640 1 --2- 172.21.15.86:0/1009535733 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56e8101ad0 0x7f56e810dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.755+0000 7f56ee9b1640 1 --2- 172.21.15.86:0/1009535733 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56e8101130 0x7f56e8101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.755+0000 7f56ee9b1640 1 --2- 172.21.15.86:0/1009535733 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56e8106f10 0x7f56e81072f0 unknown :-1 s=CLOSED pgs=1254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.755+0000 7f56ee9b1640 1 -- 172.21.15.86:0/1009535733 >> 172.21.15.86:0/1009535733 conn(0x7f56e8077e30 msgr2=0x7f56e8078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:15.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.755+0000 7f56ee9b1640 1 -- 172.21.15.86:0/1009535733 shutdown_connections 2024-04-21T08:44:15.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.755+0000 7f56ee9b1640 1 -- 172.21.15.86:0/1009535733 wait complete. 2024-04-21T08:44:15.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.756+0000 7f56ee9b1640 1 Processor -- start 2024-04-21T08:44:15.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.756+0000 7f56ee9b1640 1 -- start start 2024-04-21T08:44:15.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.756+0000 7f56ee9b1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56e8101130 0x7f56e819e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:15.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.756+0000 7f56ee9b1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56e8101ad0 0x7f56e819e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:15.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.756+0000 7f56ee9b1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56e8106f10 0x7f56e8198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:15.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.756+0000 7f56ee9b1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f56e8112260 con 0x7f56e8106f10 2024-04-21T08:44:15.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.756+0000 7f56ee9b1640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f56e81120e0 con 0x7f56e8101ad0 2024-04-21T08:44:15.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.756+0000 7f56ee9b1640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f56e81123e0 con 0x7f56e8101130 2024-04-21T08:44:15.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.756+0000 7f56ecf4e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56e8106f10 0x7f56e8198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:15.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.756+0000 7f56ecf4e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56e8106f10 0x7f56e8198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34702/0 (socket says 172.21.15.86:34702) 2024-04-21T08:44:15.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.756+0000 7f56ecf4e640 1 -- 172.21.15.86:0/622025932 learned_addr learned my addr 172.21.15.86:0/622025932 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:15.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.756+0000 7f56e77fe640 1 --2- 172.21.15.86:0/622025932 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56e8101ad0 0x7f56e819e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:15.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.756+0000 7f56e7fff640 1 --2- 172.21.15.86:0/622025932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56e8101130 0x7f56e819e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:15.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.756+0000 7f56ecf4e640 1 -- 172.21.15.86:0/622025932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56e8101130 msgr2=0x7f56e819e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:15.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.756+0000 7f56ecf4e640 1 --2- 172.21.15.86:0/622025932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56e8101130 0x7f56e819e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.756+0000 7f56ecf4e640 1 -- 172.21.15.86:0/622025932 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56e8101ad0 msgr2=0x7f56e819e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:15.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.756+0000 7f56ecf4e640 1 --2- 172.21.15.86:0/622025932 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56e8101ad0 0x7f56e819e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.756+0000 7f56ecf4e640 1 -- 172.21.15.86:0/622025932 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f56e8198da0 con 0x7f56e8106f10 2024-04-21T08:44:15.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.757+0000 7f56ecf4e640 1 --2- 172.21.15.86:0/622025932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56e8106f10 0x7f56e8198620 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f56e000eb10 tx=0x7f56e000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:15.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.757+0000 7f56e57fa640 1 -- 172.21.15.86:0/622025932 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e0004b20 con 0x7f56e8106f10 2024-04-21T08:44:15.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.757+0000 7f56e57fa640 1 -- 172.21.15.86:0/622025932 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f56e0004cc0 con 0x7f56e8106f10 2024-04-21T08:44:15.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.757+0000 7f56ee9b1640 1 -- 172.21.15.86:0/622025932 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f56e8199090 con 0x7f56e8106f10 2024-04-21T08:44:15.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.757+0000 7f56e57fa640 1 -- 172.21.15.86:0/622025932 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f56e0010670 con 0x7f56e8106f10 2024-04-21T08:44:15.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.757+0000 7f56ee9b1640 1 -- 172.21.15.86:0/622025932 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f56e81a5330 con 0x7f56e8106f10 2024-04-21T08:44:15.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.758+0000 7f56ee9b1640 1 -- 172.21.15.86:0/622025932 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f56e8069230 con 0x7f56e8106f10 2024-04-21T08:44:15.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.759+0000 7f56e57fa640 1 -- 172.21.15.86:0/622025932 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f56e0005020 con 0x7f56e8106f10 2024-04-21T08:44:15.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.759+0000 7f56e57fa640 1 --2- 172.21.15.86:0/622025932 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f56d00a3d20 0x7f56d00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:15.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.759+0000 7f56e7fff640 1 --2- 172.21.15.86:0/622025932 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f56d00a3d20 0x7f56d00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:15.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.759+0000 7f56e57fa640 1 -- 172.21.15.86:0/622025932 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f56e00d1050 con 0x7f56e8106f10 2024-04-21T08:44:15.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.760+0000 7f56e7fff640 1 --2- 172.21.15.86:0/622025932 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f56d00a3d20 0x7f56d00a61e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f56d8006010 tx=0x7f56d8005e40 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:15.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.764+0000 7f56e57fa640 1 -- 172.21.15.86:0/622025932 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f56e009d380 con 0x7f56e8106f10 2024-04-21T08:44:15.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.979+0000 7f56ee9b1640 1 -- 172.21.15.86:0/622025932 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f56e8109350 con 0x7f56e8106f10 2024-04-21T08:44:15.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.979+0000 7f56e57fa640 1 -- 172.21.15.86:0/622025932 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7f56e009cc80 con 0x7f56e8106f10 2024-04-21T08:44:15.979 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:44:15.979 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:38:45.202126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24398,24394,24455,24472]},"id":1}]} 2024-04-21T08:44:15.979 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-21T08:44:15.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.981+0000 7f56ee9b1640 1 -- 172.21.15.86:0/622025932 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f56d00a3d20 msgr2=0x7f56d00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:15.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.981+0000 7f56ee9b1640 1 --2- 172.21.15.86:0/622025932 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f56d00a3d20 0x7f56d00a61e0 secure :-1 s=READY pgs=197 cs=0 l=1 rev1=1 crypto rx=0x7f56d8006010 tx=0x7f56d8005e40 comp rx=0 tx=0).stop 2024-04-21T08:44:15.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.981+0000 7f56ee9b1640 1 -- 172.21.15.86:0/622025932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56e8106f10 msgr2=0x7f56e8198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:15.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.981+0000 7f56ee9b1640 1 --2- 172.21.15.86:0/622025932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56e8106f10 0x7f56e8198620 secure :-1 s=READY pgs=1255 cs=0 l=1 rev1=1 crypto rx=0x7f56e000eb10 tx=0x7f56e000c2d0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.981+0000 7f56ee9b1640 1 -- 172.21.15.86:0/622025932 shutdown_connections 2024-04-21T08:44:15.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.981+0000 7f56ee9b1640 1 --2- 172.21.15.86:0/622025932 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f56d00a3d20 0x7f56d00a61e0 unknown :-1 s=CLOSED pgs=197 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.981+0000 7f56ee9b1640 1 --2- 172.21.15.86:0/622025932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f56e8106f10 0x7f56e8198620 unknown :-1 s=CLOSED pgs=1255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.981+0000 7f56ee9b1640 1 --2- 172.21.15.86:0/622025932 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f56e8101ad0 0x7f56e819e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.981+0000 7f56ee9b1640 1 --2- 172.21.15.86:0/622025932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f56e8101130 0x7f56e819e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:15.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.981+0000 7f56ee9b1640 1 -- 172.21.15.86:0/622025932 >> 172.21.15.86:0/622025932 conn(0x7f56e8077e30 msgr2=0x7f56e80ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:15.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.981+0000 7f56ee9b1640 1 -- 172.21.15.86:0/622025932 shutdown_connections 2024-04-21T08:44:15.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:15.981+0000 7f56ee9b1640 1 -- 172.21.15.86:0/622025932 wait complete. 2024-04-21T08:44:15.992 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:44:16.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.092+0000 7faa33304640 1 -- 172.21.15.86:0/3526360084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa2c101ad0 msgr2=0x7faa2c10dcf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.092+0000 7faa33304640 1 --2- 172.21.15.86:0/3526360084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa2c101ad0 0x7faa2c10dcf0 secure :-1 s=READY pgs=1256 cs=0 l=1 rev1=1 crypto rx=0x7faa24009ca0 tx=0x7faa2402c140 comp rx=0 tx=0).stop 2024-04-21T08:44:16.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.092+0000 7faa33304640 1 -- 172.21.15.86:0/3526360084 shutdown_connections 2024-04-21T08:44:16.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.092+0000 7faa33304640 1 --2- 172.21.15.86:0/3526360084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa2c101ad0 0x7faa2c10dcf0 unknown :-1 s=CLOSED pgs=1256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.092+0000 7faa33304640 1 --2- 172.21.15.86:0/3526360084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa2c101130 0x7faa2c101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.092+0000 7faa33304640 1 --2- 172.21.15.86:0/3526360084 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa2c105ea0 0x7faa2c106280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.092+0000 7faa33304640 1 -- 172.21.15.86:0/3526360084 >> 172.21.15.86:0/3526360084 conn(0x7faa2c077e30 msgr2=0x7faa2c078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:16.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.092+0000 7faa33304640 1 -- 172.21.15.86:0/3526360084 shutdown_connections 2024-04-21T08:44:16.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.092+0000 7faa33304640 1 -- 172.21.15.86:0/3526360084 wait complete. 2024-04-21T08:44:16.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.093+0000 7faa33304640 1 Processor -- start 2024-04-21T08:44:16.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.093+0000 7faa33304640 1 -- start start 2024-04-21T08:44:16.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.093+0000 7faa33304640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa2c101130 0x7faa2c197630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.093+0000 7faa33304640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa2c101ad0 0x7faa2c197b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.093+0000 7faa33304640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa2c105ea0 0x7faa2c19bef0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.093+0000 7faa33304640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa2c1111d0 con 0x7faa2c101ad0 2024-04-21T08:44:16.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.093+0000 7faa33304640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faa2c111050 con 0x7faa2c105ea0 2024-04-21T08:44:16.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.093+0000 7faa33304640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faa2c111350 con 0x7faa2c101130 2024-04-21T08:44:16.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.093+0000 7faa3089f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa2c101ad0 0x7faa2c197b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.093+0000 7faa3089f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa2c101ad0 0x7faa2c197b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34716/0 (socket says 172.21.15.86:34716) 2024-04-21T08:44:16.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.093+0000 7faa3089f640 1 -- 172.21.15.86:0/1864123024 learned_addr learned my addr 172.21.15.86:0/1864123024 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:16.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.093+0000 7faa318a1640 1 --2- 172.21.15.86:0/1864123024 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa2c105ea0 0x7faa2c19bef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.093+0000 7faa310a0640 1 --2- 172.21.15.86:0/1864123024 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa2c101130 0x7faa2c197630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.094+0000 7faa3089f640 1 -- 172.21.15.86:0/1864123024 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa2c101130 msgr2=0x7faa2c197630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.094+0000 7faa3089f640 1 --2- 172.21.15.86:0/1864123024 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa2c101130 0x7faa2c197630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.094+0000 7faa3089f640 1 -- 172.21.15.86:0/1864123024 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa2c105ea0 msgr2=0x7faa2c19bef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.094+0000 7faa3089f640 1 --2- 172.21.15.86:0/1864123024 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa2c105ea0 0x7faa2c19bef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.094+0000 7faa3089f640 1 -- 172.21.15.86:0/1864123024 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa2c19c620 con 0x7faa2c101ad0 2024-04-21T08:44:16.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.094+0000 7faa3089f640 1 --2- 172.21.15.86:0/1864123024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa2c101ad0 0x7faa2c197b70 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7faa1c00c920 tx=0x7faa1c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:16.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.094+0000 7faa227fc640 1 -- 172.21.15.86:0/1864123024 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c007c70 con 0x7faa2c101ad0 2024-04-21T08:44:16.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.094+0000 7faa227fc640 1 -- 172.21.15.86:0/1864123024 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa1c007e10 con 0x7faa2c101ad0 2024-04-21T08:44:16.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.094+0000 7faa33304640 1 -- 172.21.15.86:0/1864123024 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa2c19c910 con 0x7faa2c101ad0 2024-04-21T08:44:16.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.094+0000 7faa227fc640 1 -- 172.21.15.86:0/1864123024 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c018820 con 0x7faa2c101ad0 2024-04-21T08:44:16.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.094+0000 7faa33304640 1 -- 172.21.15.86:0/1864123024 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa2c1a41a0 con 0x7faa2c101ad0 2024-04-21T08:44:16.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.096+0000 7faa227fc640 1 -- 172.21.15.86:0/1864123024 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faa1c00f450 con 0x7faa2c101ad0 2024-04-21T08:44:16.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.096+0000 7faa35fd2740 1 Processor -- start 2024-04-21T08:44:16.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.096+0000 7faa35fd2740 1 -- start start 2024-04-21T08:44:16.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.097+0000 7faa35fd2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631412f04d0 0x5631412f08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.097+0000 7faa35fd2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631412f0e80 0x5631412f9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.097+0000 7faa35fd2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5631412f9a00 0x5631412fbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.097+0000 7faa35fd2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5631412551f0 con 0x5631412f04d0 2024-04-21T08:44:16.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.097+0000 7faa35fd2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5631411f24b0 con 0x5631412f9a00 2024-04-21T08:44:16.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.097+0000 7faa35fd2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5631412511b0 con 0x5631412f0e80 2024-04-21T08:44:16.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.097+0000 7faa318a1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631412f04d0 0x5631412f08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.097+0000 7faa227fc640 1 --2- 172.21.15.86:0/1864123024 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7faa180a3d20 0x7faa180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.097+0000 7faa318a1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631412f04d0 0x5631412f08b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34720/0 (socket says 172.21.15.86:34720) 2024-04-21T08:44:16.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.097+0000 7faa318a1640 1 -- 172.21.15.86:0/1235051151 learned_addr learned my addr 172.21.15.86:0/1235051151 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:16.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.097+0000 7faa318a1640 1 --2- 172.21.15.86:0/1864123024 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7faa180a3d20 0x7faa180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.097+0000 7faa310a0640 1 --2- 172.21.15.86:0/1235051151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631412f0e80 0x5631412f9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.097+0000 7faa3089f640 1 --2- 172.21.15.86:0/1235051151 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5631412f9a00 0x5631412fbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.097+0000 7faa318a1640 1 -- 172.21.15.86:0/1235051151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631412f0e80 msgr2=0x5631412f9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.097+0000 7faa318a1640 1 --2- 172.21.15.86:0/1235051151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631412f0e80 0x5631412f9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.097+0000 7faa318a1640 1 -- 172.21.15.86:0/1235051151 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5631412f9a00 msgr2=0x5631412fbe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.097+0000 7faa318a1640 1 --2- 172.21.15.86:0/1235051151 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5631412f9a00 0x5631412fbe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.097+0000 7faa318a1640 1 -- 172.21.15.86:0/1235051151 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631411e41b0 con 0x5631412f04d0 2024-04-21T08:44:16.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.097+0000 7faa227fc640 1 -- 172.21.15.86:0/1864123024 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7faa1c0d0ca0 con 0x7faa2c101ad0 2024-04-21T08:44:16.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.097+0000 7faa3089f640 1 --2- 172.21.15.86:0/1235051151 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5631412f9a00 0x5631412fbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:16.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.097+0000 7faa318a1640 1 --2- 172.21.15.86:0/1235051151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631412f04d0 0x5631412f08b0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7faa24009ca0 tx=0x7faa240324b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9b996f987eb1280 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:16.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.098+0000 7faa177fe640 1 -- 172.21.15.86:0/1235051151 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa24050070 con 0x5631412f04d0 2024-04-21T08:44:16.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.098+0000 7faa177fe640 1 -- 172.21.15.86:0/1235051151 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa240044f0 con 0x5631412f04d0 2024-04-21T08:44:16.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.098+0000 7faa177fe640 1 -- 172.21.15.86:0/1235051151 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa24039040 con 0x5631412f04d0 2024-04-21T08:44:16.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.098+0000 7faa318a1640 1 --2- 172.21.15.86:0/1864123024 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7faa180a3d20 0x7faa180a61e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7faa2404d000 tx=0x7faa24048000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:16.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.098+0000 7faa35fd2740 1 -- 172.21.15.86:0/1235051151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631412f04d0 msgr2=0x5631412f08b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.098+0000 7faa35fd2740 1 --2- 172.21.15.86:0/1235051151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631412f04d0 0x5631412f08b0 secure :-1 s=READY pgs=1258 cs=0 l=1 rev1=1 crypto rx=0x7faa24009ca0 tx=0x7faa240324b0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.098+0000 7faa35fd2740 1 -- 172.21.15.86:0/1235051151 shutdown_connections 2024-04-21T08:44:16.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.098+0000 7faa35fd2740 1 --2- 172.21.15.86:0/1235051151 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5631412f9a00 0x5631412fbe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.098+0000 7faa35fd2740 1 --2- 172.21.15.86:0/1235051151 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631412f0e80 0x5631412f9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.098+0000 7faa35fd2740 1 --2- 172.21.15.86:0/1235051151 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631412f04d0 0x5631412f08b0 unknown :-1 s=CLOSED pgs=1258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.098+0000 7faa35fd2740 1 -- 172.21.15.86:0/1235051151 >> 172.21.15.86:0/1235051151 conn(0x5631412eaba0 msgr2=0x5631412eaf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:16.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.098+0000 7faa35fd2740 1 -- 172.21.15.86:0/1235051151 shutdown_connections 2024-04-21T08:44:16.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.098+0000 7faa35fd2740 1 -- 172.21.15.86:0/1235051151 wait complete. 2024-04-21T08:44:16.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.099+0000 7faa35fd2740 1 -- start start 2024-04-21T08:44:16.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.099+0000 7faa35fd2740 1 Processor -- start 2024-04-21T08:44:16.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.099+0000 7faa35fd2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631412f04d0 0x5631412f9310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.099+0000 7faa35fd2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5631412f0e80 0x5631412f66a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.099+0000 7faa35fd2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631412f9a00 0x5631412f6c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.099+0000 7faa35fd2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5631412fc970 con 0x5631412f04d0 2024-04-21T08:44:16.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.099+0000 7faa35fd2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x56314120e500 con 0x5631412f0e80 2024-04-21T08:44:16.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.099+0000 7faa35fd2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5631412fcaf0 con 0x5631412f9a00 2024-04-21T08:44:16.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.099+0000 7faa35fd2740 10 client.0 _ll_register_callbacks cb 0x5631411dcb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:16.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.099+0000 7faa310a0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631412f04d0 0x5631412f9310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.099+0000 7faa310a0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631412f04d0 0x5631412f9310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34722/0 (socket says 172.21.15.86:34722) 2024-04-21T08:44:16.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.099+0000 7faa310a0640 1 -- 172.21.15.86:0/3317050062 learned_addr learned my addr 172.21.15.86:0/3317050062 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:16.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.099+0000 7faa3089f640 1 --2- 172.21.15.86:0/3317050062 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5631412f0e80 0x5631412f66a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.099+0000 7faa318a1640 1 --2- 172.21.15.86:0/3317050062 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631412f9a00 0x5631412f6c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.099+0000 7faa310a0640 1 -- 172.21.15.86:0/3317050062 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631412f9a00 msgr2=0x5631412f6c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.099+0000 7faa310a0640 1 --2- 172.21.15.86:0/3317050062 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631412f9a00 0x5631412f6c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.099+0000 7faa310a0640 1 -- 172.21.15.86:0/3317050062 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5631412f0e80 msgr2=0x5631412f66a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.099+0000 7faa310a0640 1 --2- 172.21.15.86:0/3317050062 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5631412f0e80 0x5631412f66a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.099+0000 7faa310a0640 1 -- 172.21.15.86:0/3317050062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563140f673f0 con 0x5631412f04d0 2024-04-21T08:44:16.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.100+0000 7faa310a0640 1 --2- 172.21.15.86:0/3317050062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631412f04d0 0x5631412f9310 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7faa2800be50 tx=0x7faa280117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb7244e5084cdcd7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:16.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.100+0000 7faa14ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:16.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.100+0000 7faa14ff9640 1 -- 172.21.15.86:0/3317050062 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa28004230 con 0x5631412f04d0 2024-04-21T08:44:16.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.100+0000 7faa14ff9640 1 -- 172.21.15.86:0/3317050062 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7faa280043d0 con 0x5631412f04d0 2024-04-21T08:44:16.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.100+0000 7faa35fd2740 1 -- 172.21.15.86:0/3317050062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa240044f0 con 0x5631412f04d0 2024-04-21T08:44:16.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.100+0000 7faa14ff9640 1 -- 172.21.15.86:0/3317050062 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faa280035f0 con 0x5631412f04d0 2024-04-21T08:44:16.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.100+0000 7faa35fd2740 10 client.17952 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:16.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.100+0000 7faa35fd2740 1 -- 172.21.15.86:0/3317050062 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5631410317d0 con 0x5631412f04d0 2024-04-21T08:44:16.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.100+0000 7faa14ff9640 1 -- 172.21.15.86:0/3317050062 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa240044f0 con 0x5631412f04d0 2024-04-21T08:44:16.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.100+0000 7faa14ff9640 1 -- 172.21.15.86:0/3317050062 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7faa2801b880 con 0x5631412f04d0 2024-04-21T08:44:16.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.100+0000 7faa35fd2740 10 client.17952 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:16.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.100+0000 7faa35fd2740 10 client.17952 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:44:16.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.100+0000 7faa35fd2740 20 client.17952 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:16.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.100+0000 7faa35fd2740 1 --2- 172.21.15.86:0/3317050062 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x563141309c10 0x56314130c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.100+0000 7faa35fd2740 4 client.17952 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:16.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.100+0000 7faa35fd2740 1 -- 172.21.15.86:0/3317050062 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5631411f4930 con 0x563141309c10 2024-04-21T08:44:16.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.101+0000 7faa3089f640 1 --2- 172.21.15.86:0/3317050062 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x563141309c10 0x56314130c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.101+0000 7faa14ff9640 10 client.17952 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:44:16.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.101+0000 7faa3089f640 1 --2- 172.21.15.86:0/3317050062 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x563141309c10 0x56314130c060 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a997b274262dacae server_cookie=4a7c62232e2f241 in_seq=0 out_seq=0 2024-04-21T08:44:16.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.102+0000 7faa14ff9640 1 -- 172.21.15.86:0/3317050062 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5631411f4930 con 0x563141309c10 2024-04-21T08:44:16.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.102+0000 7faa14ff9640 10 client.17952 handle_command_reply: tid=0 2024-04-21T08:44:16.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.102+0000 7faa35fd2740 1 client.17952 shutdown 2024-04-21T08:44:16.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.102+0000 7faa35fd2740 1 -- 172.21.15.86:0/3317050062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631412f04d0 msgr2=0x5631412f9310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.102+0000 7faa35fd2740 1 --2- 172.21.15.86:0/3317050062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631412f04d0 0x5631412f9310 secure :-1 s=READY pgs=1259 cs=0 l=1 rev1=1 crypto rx=0x7faa2800be50 tx=0x7faa280117b0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.102+0000 7faa35fd2740 1 -- 172.21.15.86:0/3317050062 shutdown_connections 2024-04-21T08:44:16.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.102+0000 7faa35fd2740 1 --2- 172.21.15.86:0/3317050062 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631412f9a00 0x5631412f6c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.102+0000 7faa35fd2740 1 --2- 172.21.15.86:0/3317050062 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x563141309c10 0x56314130c060 crc :-1 s=READY pgs=52 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.102+0000 7faa35fd2740 1 --2- 172.21.15.86:0/3317050062 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5631412f0e80 0x5631412f66a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.102+0000 7faa14ff9640 0 client.17952 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:44:16.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.102+0000 7faa35fd2740 1 --2- 172.21.15.86:0/3317050062 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631412f04d0 0x5631412f9310 unknown :-1 s=CLOSED pgs=1259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.102+0000 7faa35fd2740 1 -- 172.21.15.86:0/3317050062 >> 172.21.15.86:0/3317050062 conn(0x5631412eaba0 msgr2=0x5631412ec8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:16.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.102+0000 7faa35fd2740 1 -- 172.21.15.86:0/3317050062 shutdown_connections 2024-04-21T08:44:16.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.102+0000 7faa35fd2740 1 -- 172.21.15.86:0/3317050062 wait complete. 2024-04-21T08:44:16.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.102+0000 7faa35fd2740 20 client.17952 trim_cache size 0 max 16384 2024-04-21T08:44:16.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.115+0000 7faa35fd2740 1 Processor -- start 2024-04-21T08:44:16.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.115+0000 7faa35fd2740 1 -- start start 2024-04-21T08:44:16.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.115+0000 7faa35fd2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631412f04d0 0x563141305860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.115+0000 7faa35fd2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631412f0e80 0x5631412e9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.115+0000 7faa35fd2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5631412f9a00 0x5631412f8dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.115+0000 7faa35fd2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa2c1ae000 con 0x5631412f0e80 2024-04-21T08:44:16.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.115+0000 7faa35fd2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faa2c1afb40 con 0x5631412f9a00 2024-04-21T08:44:16.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.115+0000 7faa35fd2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faa2c1ae6d0 con 0x5631412f04d0 2024-04-21T08:44:16.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.115+0000 7faa3089f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631412f0e80 0x5631412e9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.115+0000 7faa3089f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631412f0e80 0x5631412e9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34734/0 (socket says 172.21.15.86:34734) 2024-04-21T08:44:16.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.115+0000 7faa3089f640 1 -- 172.21.15.86:0/3891548877 learned_addr learned my addr 172.21.15.86:0/3891548877 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:16.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.116+0000 7faa310a0640 1 --2- 172.21.15.86:0/3891548877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631412f04d0 0x563141305860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.116+0000 7faa3089f640 1 -- 172.21.15.86:0/3891548877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631412f04d0 msgr2=0x563141305860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.116+0000 7faa318a1640 1 --2- 172.21.15.86:0/3891548877 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5631412f9a00 0x5631412f8dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.116+0000 7faa3089f640 1 --2- 172.21.15.86:0/3891548877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631412f04d0 0x563141305860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.116+0000 7faa3089f640 1 -- 172.21.15.86:0/3891548877 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5631412f9a00 msgr2=0x5631412f8dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.116+0000 7faa3089f640 1 --2- 172.21.15.86:0/3891548877 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5631412f9a00 0x5631412f8dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.116+0000 7faa3089f640 1 -- 172.21.15.86:0/3891548877 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5631412f5650 con 0x5631412f0e80 2024-04-21T08:44:16.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.116+0000 7faa318a1640 1 --2- 172.21.15.86:0/3891548877 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5631412f9a00 0x5631412f8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:16.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.116+0000 7faa3089f640 1 --2- 172.21.15.86:0/3891548877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631412f0e80 0x5631412e9a80 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7faa1c0992a0 tx=0x7faa1c013b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e06fe1bf20dfb690 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:16.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.116+0000 7faa16ffd640 1 -- 172.21.15.86:0/3891548877 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c0a0c80 con 0x5631412f0e80 2024-04-21T08:44:16.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.116+0000 7faa16ffd640 1 -- 172.21.15.86:0/3891548877 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa1c0a5540 con 0x5631412f0e80 2024-04-21T08:44:16.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.116+0000 7faa16ffd640 1 -- 172.21.15.86:0/3891548877 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c098f40 con 0x5631412f0e80 2024-04-21T08:44:16.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.117+0000 7faa35fd2740 1 -- 172.21.15.86:0/3891548877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631412f0e80 msgr2=0x5631412e9a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.117+0000 7faa35fd2740 1 --2- 172.21.15.86:0/3891548877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631412f0e80 0x5631412e9a80 secure :-1 s=READY pgs=1260 cs=0 l=1 rev1=1 crypto rx=0x7faa1c0992a0 tx=0x7faa1c013b80 comp rx=0 tx=0).stop 2024-04-21T08:44:16.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.117+0000 7faa35fd2740 1 -- 172.21.15.86:0/3891548877 shutdown_connections 2024-04-21T08:44:16.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.117+0000 7faa35fd2740 1 --2- 172.21.15.86:0/3891548877 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5631412f9a00 0x5631412f8dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.117+0000 7faa35fd2740 1 --2- 172.21.15.86:0/3891548877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631412f0e80 0x5631412e9a80 unknown :-1 s=CLOSED pgs=1260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.117+0000 7faa35fd2740 1 --2- 172.21.15.86:0/3891548877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631412f04d0 0x563141305860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.117+0000 7faa35fd2740 1 -- 172.21.15.86:0/3891548877 >> 172.21.15.86:0/3891548877 conn(0x5631412eaba0 msgr2=0x5631412e9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:16.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.117+0000 7faa35fd2740 1 -- 172.21.15.86:0/3891548877 shutdown_connections 2024-04-21T08:44:16.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.117+0000 7faa35fd2740 1 -- 172.21.15.86:0/3891548877 wait complete. 2024-04-21T08:44:16.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.117+0000 7faa35fd2740 1 -- start start 2024-04-21T08:44:16.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.117+0000 7faa35fd2740 1 Processor -- start 2024-04-21T08:44:16.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.117+0000 7faa35fd2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5631412f04d0 0x563141302300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.117+0000 7faa35fd2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631412f0e80 0x5631412ff690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.117+0000 7faa35fd2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631412f9a00 0x5631412ffdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.117+0000 7faa35fd2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa2c1af470 con 0x5631412f0e80 2024-04-21T08:44:16.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.117+0000 7faa35fd2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faa2c1aeda0 con 0x5631412f04d0 2024-04-21T08:44:16.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.117+0000 7faa35fd2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faa2c1abfc0 con 0x5631412f9a00 2024-04-21T08:44:16.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.118+0000 7faa35fd2740 10 client.0 _ll_register_callbacks cb 0x7faa2800b9e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:16.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.118+0000 7faa3089f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631412f0e80 0x5631412ff690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.118+0000 7faa3089f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631412f0e80 0x5631412ff690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34744/0 (socket says 172.21.15.86:34744) 2024-04-21T08:44:16.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.118+0000 7faa3089f640 1 -- 172.21.15.86:0/1523048872 learned_addr learned my addr 172.21.15.86:0/1523048872 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:16.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.118+0000 7faa318a1640 1 --2- 172.21.15.86:0/1523048872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631412f9a00 0x5631412ffdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.118+0000 7faa310a0640 1 --2- 172.21.15.86:0/1523048872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5631412f04d0 0x563141302300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.118+0000 7faa3089f640 1 -- 172.21.15.86:0/1523048872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631412f9a00 msgr2=0x5631412ffdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.118+0000 7faa3089f640 1 --2- 172.21.15.86:0/1523048872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631412f9a00 0x5631412ffdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.118+0000 7faa3089f640 1 -- 172.21.15.86:0/1523048872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5631412f04d0 msgr2=0x563141302300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.118+0000 7faa3089f640 1 --2- 172.21.15.86:0/1523048872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5631412f04d0 0x563141302300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.118+0000 7faa3089f640 1 -- 172.21.15.86:0/1523048872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5631412f6fb0 con 0x5631412f0e80 2024-04-21T08:44:16.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.118+0000 7faa3089f640 1 --2- 172.21.15.86:0/1523048872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631412f0e80 0x5631412ff690 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7faa1c007590 tx=0x7faa1c09abd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=576934c09a692167 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:16.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.118+0000 7faa14ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:16.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.118+0000 7faa14ff9640 1 -- 172.21.15.86:0/1523048872 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa1c066060 con 0x5631412f0e80 2024-04-21T08:44:16.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.118+0000 7faa14ff9640 1 -- 172.21.15.86:0/1523048872 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7faa1c00f740 con 0x5631412f0e80 2024-04-21T08:44:16.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.118+0000 7faa35fd2740 1 -- 172.21.15.86:0/1523048872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa1c0a5540 con 0x5631412f0e80 2024-04-21T08:44:16.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.118+0000 7faa14ff9640 1 -- 172.21.15.86:0/1523048872 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faa1c0ba030 con 0x5631412f0e80 2024-04-21T08:44:16.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.118+0000 7faa35fd2740 10 client.17958 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:16.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.118+0000 7faa35fd2740 1 -- 172.21.15.86:0/1523048872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563141372120 con 0x5631412f0e80 2024-04-21T08:44:16.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.118+0000 7faa14ff9640 1 -- 172.21.15.86:0/1523048872 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa1c0a5540 con 0x5631412f0e80 2024-04-21T08:44:16.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.118+0000 7faa14ff9640 1 -- 172.21.15.86:0/1523048872 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7faa1c0ce320 con 0x5631412f0e80 2024-04-21T08:44:16.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.118+0000 7faa35fd2740 10 client.17958 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:16.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.118+0000 7faa35fd2740 10 client.17958 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:44:16.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.118+0000 7faa35fd2740 20 client.17958 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:16.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.119+0000 7faa35fd2740 1 --2- 172.21.15.86:0/1523048872 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x563141372790 0x563141374bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.119+0000 7faa35fd2740 4 client.17958 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:44:16.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.119+0000 7faa35fd2740 1 -- 172.21.15.86:0/1523048872 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563141262c10 con 0x563141372790 2024-04-21T08:44:16.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.119+0000 7faa310a0640 1 --2- 172.21.15.86:0/1523048872 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x563141372790 0x563141374bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.119+0000 7faa14ff9640 10 client.17958 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:44:16.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.119+0000 7faa310a0640 1 --2- 172.21.15.86:0/1523048872 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x563141372790 0x563141374bb0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ffb307bf06bb1447 server_cookie=c18423f88e9de652 in_seq=0 out_seq=0 2024-04-21T08:44:16.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.120+0000 7faa14ff9640 1 -- 172.21.15.86:0/1523048872 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563141262c10 con 0x563141372790 2024-04-21T08:44:16.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.120+0000 7faa14ff9640 10 client.17958 handle_command_reply: tid=0 2024-04-21T08:44:16.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.120+0000 7faa35fd2740 1 client.17958 shutdown 2024-04-21T08:44:16.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.120+0000 7faa35fd2740 1 -- 172.21.15.86:0/1523048872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631412f0e80 msgr2=0x5631412ff690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.120+0000 7faa35fd2740 1 --2- 172.21.15.86:0/1523048872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631412f0e80 0x5631412ff690 secure :-1 s=READY pgs=1261 cs=0 l=1 rev1=1 crypto rx=0x7faa1c007590 tx=0x7faa1c09abd0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.120+0000 7faa35fd2740 1 -- 172.21.15.86:0/1523048872 shutdown_connections 2024-04-21T08:44:16.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.120+0000 7faa35fd2740 1 --2- 172.21.15.86:0/1523048872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5631412f9a00 0x5631412ffdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.120+0000 7faa35fd2740 1 --2- 172.21.15.86:0/1523048872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5631412f0e80 0x5631412ff690 unknown :-1 s=CLOSED pgs=1261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.120+0000 7faa35fd2740 1 --2- 172.21.15.86:0/1523048872 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x563141372790 0x563141374bb0 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.120+0000 7faa35fd2740 1 --2- 172.21.15.86:0/1523048872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5631412f04d0 0x563141302300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.120+0000 7faa14ff9640 0 client.17958 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:44:16.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.120+0000 7faa35fd2740 1 -- 172.21.15.86:0/1523048872 >> 172.21.15.86:0/1523048872 conn(0x5631412eaba0 msgr2=0x5631412e9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:16.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.120+0000 7faa35fd2740 1 -- 172.21.15.86:0/1523048872 shutdown_connections 2024-04-21T08:44:16.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.120+0000 7faa35fd2740 1 -- 172.21.15.86:0/1523048872 wait complete. 2024-04-21T08:44:16.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.120+0000 7faa35fd2740 20 client.17958 trim_cache size 0 max 16384 2024-04-21T08:44:16.126 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:44:16.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.120+0000 7faa33304640 1 -- 172.21.15.86:0/1864123024 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7faa180a3d20 msgr2=0x7faa180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.120+0000 7faa33304640 1 --2- 172.21.15.86:0/1864123024 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7faa180a3d20 0x7faa180a61e0 secure :-1 s=READY pgs=198 cs=0 l=1 rev1=1 crypto rx=0x7faa2404d000 tx=0x7faa24048000 comp rx=0 tx=0).stop 2024-04-21T08:44:16.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.120+0000 7faa33304640 1 -- 172.21.15.86:0/1864123024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa2c101ad0 msgr2=0x7faa2c197b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.120+0000 7faa33304640 1 --2- 172.21.15.86:0/1864123024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa2c101ad0 0x7faa2c197b70 secure :-1 s=READY pgs=1257 cs=0 l=1 rev1=1 crypto rx=0x7faa1c00c920 tx=0x7faa1c00cde0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.120+0000 7faa33304640 1 -- 172.21.15.86:0/1864123024 shutdown_connections 2024-04-21T08:44:16.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.120+0000 7faa33304640 1 --2- 172.21.15.86:0/1864123024 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7faa180a3d20 0x7faa180a61e0 unknown :-1 s=CLOSED pgs=198 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.120+0000 7faa33304640 1 --2- 172.21.15.86:0/1864123024 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa2c105ea0 0x7faa2c19bef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.120+0000 7faa33304640 1 --2- 172.21.15.86:0/1864123024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa2c101ad0 0x7faa2c197b70 unknown :-1 s=CLOSED pgs=1257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.120+0000 7faa33304640 1 --2- 172.21.15.86:0/1864123024 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa2c101130 0x7faa2c197630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.120+0000 7faa33304640 1 -- 172.21.15.86:0/1864123024 >> 172.21.15.86:0/1864123024 conn(0x7faa2c077e30 msgr2=0x7faa2c0fec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:16.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.121+0000 7faa33304640 1 -- 172.21.15.86:0/1864123024 shutdown_connections 2024-04-21T08:44:16.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.121+0000 7faa33304640 1 -- 172.21.15.86:0/1864123024 wait complete. 2024-04-21T08:44:16.130 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:44:16.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.231+0000 7f28d517e640 1 -- 172.21.15.86:0/3104284403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28d0102190 msgr2=0x7f28d0102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.231+0000 7f28d517e640 1 --2- 172.21.15.86:0/3104284403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28d0102190 0x7f28d0102570 secure :-1 s=READY pgs=1262 cs=0 l=1 rev1=1 crypto rx=0x7f28c0009900 tx=0x7f28c002bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:16.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.231+0000 7f28d517e640 1 -- 172.21.15.86:0/3104284403 shutdown_connections 2024-04-21T08:44:16.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.231+0000 7f28d517e640 1 --2- 172.21.15.86:0/3104284403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28d0107260 0x7f28d010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.231+0000 7f28d517e640 1 --2- 172.21.15.86:0/3104284403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28d0102b40 0x7f28d0106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.231+0000 7f28d517e640 1 --2- 172.21.15.86:0/3104284403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28d0102190 0x7f28d0102570 unknown :-1 s=CLOSED pgs=1262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.231+0000 7f28d517e640 1 -- 172.21.15.86:0/3104284403 >> 172.21.15.86:0/3104284403 conn(0x7f28d00fd990 msgr2=0x7f28d00ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:16.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.231+0000 7f28d517e640 1 -- 172.21.15.86:0/3104284403 shutdown_connections 2024-04-21T08:44:16.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.231+0000 7f28d517e640 1 -- 172.21.15.86:0/3104284403 wait complete. 2024-04-21T08:44:16.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.232+0000 7f28d517e640 1 Processor -- start 2024-04-21T08:44:16.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.232+0000 7f28d517e640 1 -- start start 2024-04-21T08:44:16.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.232+0000 7f28d517e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28d0102190 0x7f28d0198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.232+0000 7f28d517e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28d0107260 0x7f28d0198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.232+0000 7f28d517e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28d019cfe0 0x7f28d019d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.232+0000 7f28d517e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f28d01121a0 con 0x7f28d0107260 2024-04-21T08:44:16.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.232+0000 7f28d517e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f28d0112020 con 0x7f28d019cfe0 2024-04-21T08:44:16.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.232+0000 7f28d517e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f28d0112320 con 0x7f28d0102190 2024-04-21T08:44:16.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.232+0000 7f28c7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28d0107260 0x7f28d0198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.232+0000 7f28c7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28d0107260 0x7f28d0198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34754/0 (socket says 172.21.15.86:34754) 2024-04-21T08:44:16.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.232+0000 7f28c7fff640 1 -- 172.21.15.86:0/2533497138 learned_addr learned my addr 172.21.15.86:0/2533497138 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:16.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.232+0000 7f28cf59e640 1 --2- 172.21.15.86:0/2533497138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28d019cfe0 0x7f28d019d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.232+0000 7f28ced9d640 1 --2- 172.21.15.86:0/2533497138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28d0102190 0x7f28d0198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.233+0000 7f28c7fff640 1 -- 172.21.15.86:0/2533497138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28d0102190 msgr2=0x7f28d0198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.233+0000 7f28c7fff640 1 --2- 172.21.15.86:0/2533497138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28d0102190 0x7f28d0198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.233+0000 7f28c7fff640 1 -- 172.21.15.86:0/2533497138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28d019cfe0 msgr2=0x7f28d019d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.233+0000 7f28c7fff640 1 --2- 172.21.15.86:0/2533497138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28d019cfe0 0x7f28d019d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.233+0000 7f28c7fff640 1 -- 172.21.15.86:0/2533497138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f28d019dad0 con 0x7f28d0107260 2024-04-21T08:44:16.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.233+0000 7f28ced9d640 1 --2- 172.21.15.86:0/2533497138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28d0102190 0x7f28d0198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:16.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.233+0000 7f28cf59e640 1 --2- 172.21.15.86:0/2533497138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28d019cfe0 0x7f28d019d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:16.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.233+0000 7f28c7fff640 1 --2- 172.21.15.86:0/2533497138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28d0107260 0x7f28d0198c60 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f28b8005fd0 tx=0x7f28b800dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:16.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.233+0000 7f28ccd99640 1 -- 172.21.15.86:0/2533497138 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b8017070 con 0x7f28d0107260 2024-04-21T08:44:16.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.233+0000 7f28ccd99640 1 -- 172.21.15.86:0/2533497138 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28b8009c30 con 0x7f28d0107260 2024-04-21T08:44:16.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.233+0000 7f28d517e640 1 -- 172.21.15.86:0/2533497138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f28d019ddc0 con 0x7f28d0107260 2024-04-21T08:44:16.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.233+0000 7f28ccd99640 1 -- 172.21.15.86:0/2533497138 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28b8003d30 con 0x7f28d0107260 2024-04-21T08:44:16.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.233+0000 7f28d517e640 1 -- 172.21.15.86:0/2533497138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f28d01a55b0 con 0x7f28d0107260 2024-04-21T08:44:16.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.235+0000 7f28ccd99640 1 -- 172.21.15.86:0/2533497138 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f28b8003ed0 con 0x7f28d0107260 2024-04-21T08:44:16.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.235+0000 7f28d7d6c740 1 Processor -- start 2024-04-21T08:44:16.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.235+0000 7f28d7d6c740 1 -- start start 2024-04-21T08:44:16.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.236+0000 7f28d7d6c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649b653d4d0 0x5649b653d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.236+0000 7f28d7d6c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649b653de80 0x5649b6546430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.236+0000 7f28d7d6c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5649b6546a00 0x5649b6548e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.236+0000 7f28d7d6c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5649b64a21f0 con 0x5649b653de80 2024-04-21T08:44:16.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.236+0000 7f28d7d6c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5649b643f4b0 con 0x5649b6546a00 2024-04-21T08:44:16.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.236+0000 7f28d7d6c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5649b649e1b0 con 0x5649b653d4d0 2024-04-21T08:44:16.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.236+0000 7f28ced9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649b653de80 0x5649b6546430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.236+0000 7f28ced9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649b653de80 0x5649b6546430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34770/0 (socket says 172.21.15.86:34770) 2024-04-21T08:44:16.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.236+0000 7f28ced9d640 1 -- 172.21.15.86:0/2162889287 learned_addr learned my addr 172.21.15.86:0/2162889287 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:16.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.236+0000 7f28ccd99640 1 --2- 172.21.15.86:0/2533497138 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f28bc0a3d20 0x7f28bc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.236+0000 7f28cf59e640 1 --2- 172.21.15.86:0/2162889287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649b653d4d0 0x5649b653d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.236+0000 7f28c7fff640 1 --2- 172.21.15.86:0/2162889287 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5649b6546a00 0x5649b6548e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.236+0000 7f28cf59e640 1 --2- 172.21.15.86:0/2533497138 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f28bc0a3d20 0x7f28bc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.236+0000 7f28ced9d640 1 -- 172.21.15.86:0/2162889287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649b653d4d0 msgr2=0x5649b653d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.236+0000 7f28ced9d640 1 --2- 172.21.15.86:0/2162889287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649b653d4d0 0x5649b653d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.236+0000 7f28ced9d640 1 -- 172.21.15.86:0/2162889287 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5649b6546a00 msgr2=0x5649b6548e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.236+0000 7f28ced9d640 1 --2- 172.21.15.86:0/2162889287 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5649b6546a00 0x5649b6548e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.236+0000 7f28ced9d640 1 -- 172.21.15.86:0/2162889287 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649b64311b0 con 0x5649b653de80 2024-04-21T08:44:16.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.236+0000 7f28cf59e640 1 --2- 172.21.15.86:0/2162889287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649b653d4d0 0x5649b653d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:16.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.236+0000 7f28ccd99640 1 -- 172.21.15.86:0/2533497138 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f28b80d08b0 con 0x7f28d0107260 2024-04-21T08:44:16.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.237+0000 7f28ced9d640 1 --2- 172.21.15.86:0/2162889287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649b653de80 0x5649b6546430 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f28c0036040 tx=0x7f28c002c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c215aa83309d34f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:16.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.237+0000 7f28cf59e640 1 --2- 172.21.15.86:0/2533497138 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f28bc0a3d20 0x7f28bc0a61e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f28c8005c40 tx=0x7f28c8009280 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:16.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.237+0000 7f28c57fa640 1 -- 172.21.15.86:0/2162889287 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c0043020 con 0x5649b653de80 2024-04-21T08:44:16.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.237+0000 7f28c57fa640 1 -- 172.21.15.86:0/2162889287 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28c002c7e0 con 0x5649b653de80 2024-04-21T08:44:16.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.237+0000 7f28c57fa640 1 -- 172.21.15.86:0/2162889287 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c0005590 con 0x5649b653de80 2024-04-21T08:44:16.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.237+0000 7f28d7d6c740 1 -- 172.21.15.86:0/2162889287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649b653de80 msgr2=0x5649b6546430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.237+0000 7f28d7d6c740 1 --2- 172.21.15.86:0/2162889287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649b653de80 0x5649b6546430 secure :-1 s=READY pgs=1264 cs=0 l=1 rev1=1 crypto rx=0x7f28c0036040 tx=0x7f28c002c680 comp rx=0 tx=0).stop 2024-04-21T08:44:16.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.238+0000 7f28d7d6c740 1 -- 172.21.15.86:0/2162889287 shutdown_connections 2024-04-21T08:44:16.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.238+0000 7f28d7d6c740 1 --2- 172.21.15.86:0/2162889287 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5649b6546a00 0x5649b6548e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.238+0000 7f28d7d6c740 1 --2- 172.21.15.86:0/2162889287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649b653de80 0x5649b6546430 unknown :-1 s=CLOSED pgs=1264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.238+0000 7f28d7d6c740 1 --2- 172.21.15.86:0/2162889287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649b653d4d0 0x5649b653d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.238+0000 7f28d7d6c740 1 -- 172.21.15.86:0/2162889287 >> 172.21.15.86:0/2162889287 conn(0x5649b6537ba0 msgr2=0x5649b6537f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:16.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.238+0000 7f28d7d6c740 1 -- 172.21.15.86:0/2162889287 shutdown_connections 2024-04-21T08:44:16.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.238+0000 7f28d7d6c740 1 -- 172.21.15.86:0/2162889287 wait complete. 2024-04-21T08:44:16.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.238+0000 7f28d7d6c740 1 -- start start 2024-04-21T08:44:16.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.238+0000 7f28d7d6c740 1 Processor -- start 2024-04-21T08:44:16.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.238+0000 7f28d7d6c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649b653d4d0 0x5649b65462c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.238+0000 7f28d7d6c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5649b653de80 0x5649b6543650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.238+0000 7f28d7d6c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649b6546a00 0x5649b6543bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.238+0000 7f28d7d6c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5649b6549910 con 0x5649b653d4d0 2024-04-21T08:44:16.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.238+0000 7f28d7d6c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5649b645b500 con 0x5649b653de80 2024-04-21T08:44:16.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.238+0000 7f28d7d6c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5649b6549a90 con 0x5649b6546a00 2024-04-21T08:44:16.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.238+0000 7f28d7d6c740 10 client.0 _ll_register_callbacks cb 0x5649b6429b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:16.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.238+0000 7f28ced9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649b653d4d0 0x5649b65462c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.238+0000 7f28ced9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649b653d4d0 0x5649b65462c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34776/0 (socket says 172.21.15.86:34776) 2024-04-21T08:44:16.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.238+0000 7f28ced9d640 1 -- 172.21.15.86:0/2811066252 learned_addr learned my addr 172.21.15.86:0/2811066252 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:16.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.238+0000 7f28c7fff640 1 --2- 172.21.15.86:0/2811066252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5649b653de80 0x5649b6543650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.238+0000 7f28cf59e640 1 --2- 172.21.15.86:0/2811066252 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649b6546a00 0x5649b6543bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.239+0000 7f28ced9d640 1 -- 172.21.15.86:0/2811066252 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649b6546a00 msgr2=0x5649b6543bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.239+0000 7f28ced9d640 1 --2- 172.21.15.86:0/2811066252 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649b6546a00 0x5649b6543bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.239+0000 7f28ced9d640 1 -- 172.21.15.86:0/2811066252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5649b653de80 msgr2=0x5649b6543650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.239+0000 7f28ced9d640 1 --2- 172.21.15.86:0/2811066252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5649b653de80 0x5649b6543650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.239+0000 7f28ced9d640 1 -- 172.21.15.86:0/2811066252 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649b61b43f0 con 0x5649b653d4d0 2024-04-21T08:44:16.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.239+0000 7f28cf59e640 1 --2- 172.21.15.86:0/2811066252 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649b6546a00 0x5649b6543bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:16.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.239+0000 7f28ced9d640 1 --2- 172.21.15.86:0/2811066252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649b653d4d0 0x5649b65462c0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f28c0005fd0 tx=0x7f28c0005150 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3557b771ce3b16f8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:16.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.239+0000 7f28b67fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:16.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.239+0000 7f28b67fc640 1 -- 172.21.15.86:0/2811066252 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c004c070 con 0x5649b653d4d0 2024-04-21T08:44:16.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.239+0000 7f28b67fc640 1 -- 172.21.15.86:0/2811066252 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f28c00035f0 con 0x5649b653d4d0 2024-04-21T08:44:16.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.239+0000 7f28d7d6c740 1 -- 172.21.15.86:0/2811066252 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28c002c7e0 con 0x5649b653d4d0 2024-04-21T08:44:16.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.239+0000 7f28b67fc640 1 -- 172.21.15.86:0/2811066252 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f28c0058050 con 0x5649b653d4d0 2024-04-21T08:44:16.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.239+0000 7f28d7d6c740 10 client.17970 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:16.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.239+0000 7f28d7d6c740 1 -- 172.21.15.86:0/2811066252 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5649b627e7d0 con 0x5649b653d4d0 2024-04-21T08:44:16.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.239+0000 7f28b67fc640 1 -- 172.21.15.86:0/2811066252 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28c002c7e0 con 0x5649b653d4d0 2024-04-21T08:44:16.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.239+0000 7f28b67fc640 1 -- 172.21.15.86:0/2811066252 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f28c0004bf0 con 0x5649b653d4d0 2024-04-21T08:44:16.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.239+0000 7f28d7d6c740 10 client.17970 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:16.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.239+0000 7f28d7d6c740 10 client.17970 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:44:16.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.239+0000 7f28d7d6c740 20 client.17970 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:16.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.239+0000 7f28d7d6c740 1 --2- 172.21.15.86:0/2811066252 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x5649b6556a30 0x5649b6558e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.239+0000 7f28d7d6c740 4 client.17970 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:16.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.239+0000 7f28d7d6c740 1 -- 172.21.15.86:0/2811066252 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5649b6441930 con 0x5649b6556a30 2024-04-21T08:44:16.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.240+0000 7f28c7fff640 1 --2- 172.21.15.86:0/2811066252 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x5649b6556a30 0x5649b6558e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.240+0000 7f28b67fc640 10 client.17970 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:44:16.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.240+0000 7f28c7fff640 1 --2- 172.21.15.86:0/2811066252 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x5649b6556a30 0x5649b6558e20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d28547c90e460388 server_cookie=1ea030d9534dcc70 in_seq=0 out_seq=0 2024-04-21T08:44:16.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.241+0000 7f28b67fc640 1 -- 172.21.15.86:0/2811066252 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5649b6441930 con 0x5649b6556a30 2024-04-21T08:44:16.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.241+0000 7f28b67fc640 10 client.17970 handle_command_reply: tid=0 2024-04-21T08:44:16.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.241+0000 7f28d7d6c740 1 client.17970 shutdown 2024-04-21T08:44:16.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.241+0000 7f28d7d6c740 1 -- 172.21.15.86:0/2811066252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649b653d4d0 msgr2=0x5649b65462c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.241+0000 7f28d7d6c740 1 --2- 172.21.15.86:0/2811066252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649b653d4d0 0x5649b65462c0 secure :-1 s=READY pgs=1265 cs=0 l=1 rev1=1 crypto rx=0x7f28c0005fd0 tx=0x7f28c0005150 comp rx=0 tx=0).stop 2024-04-21T08:44:16.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.241+0000 7f28d7d6c740 1 -- 172.21.15.86:0/2811066252 shutdown_connections 2024-04-21T08:44:16.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.241+0000 7f28d7d6c740 1 --2- 172.21.15.86:0/2811066252 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x5649b6556a30 0x5649b6558e20 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.241+0000 7f28d7d6c740 1 --2- 172.21.15.86:0/2811066252 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649b6546a00 0x5649b6543bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.241+0000 7f28d7d6c740 1 --2- 172.21.15.86:0/2811066252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5649b653de80 0x5649b6543650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.241+0000 7f28b67fc640 0 client.17970 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:44:16.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.241+0000 7f28d7d6c740 1 --2- 172.21.15.86:0/2811066252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649b653d4d0 0x5649b65462c0 unknown :-1 s=CLOSED pgs=1265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.241+0000 7f28d7d6c740 1 -- 172.21.15.86:0/2811066252 >> 172.21.15.86:0/2811066252 conn(0x5649b6537ba0 msgr2=0x5649b65398f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:16.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.241+0000 7f28d7d6c740 1 -- 172.21.15.86:0/2811066252 shutdown_connections 2024-04-21T08:44:16.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.241+0000 7f28d7d6c740 1 -- 172.21.15.86:0/2811066252 wait complete. 2024-04-21T08:44:16.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.241+0000 7f28d7d6c740 20 client.17970 trim_cache size 0 max 16384 2024-04-21T08:44:16.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.254+0000 7f28d7d6c740 1 Processor -- start 2024-04-21T08:44:16.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.254+0000 7f28d7d6c740 1 -- start start 2024-04-21T08:44:16.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.255+0000 7f28d7d6c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649b653d4d0 0x5649b6536e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.255+0000 7f28d7d6c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649b653de80 0x5649b6552800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.255+0000 7f28d7d6c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5649b6546a00 0x5649b6545d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.255+0000 7f28d7d6c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f28d01af330 con 0x5649b653d4d0 2024-04-21T08:44:16.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.255+0000 7f28d7d6c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f28d01b0e70 con 0x5649b6546a00 2024-04-21T08:44:16.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.255+0000 7f28d7d6c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f28d01afa00 con 0x5649b653de80 2024-04-21T08:44:16.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.255+0000 7f28ced9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649b653d4d0 0x5649b6536e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.255+0000 7f28ced9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649b653d4d0 0x5649b6536e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34792/0 (socket says 172.21.15.86:34792) 2024-04-21T08:44:16.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.255+0000 7f28ced9d640 1 -- 172.21.15.86:0/3138210554 learned_addr learned my addr 172.21.15.86:0/3138210554 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:16.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.255+0000 7f28cf59e640 1 --2- 172.21.15.86:0/3138210554 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5649b6546a00 0x5649b6545d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.255+0000 7f28c7fff640 1 --2- 172.21.15.86:0/3138210554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649b653de80 0x5649b6552800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.255+0000 7f28ced9d640 1 -- 172.21.15.86:0/3138210554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649b653de80 msgr2=0x5649b6552800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.255+0000 7f28ced9d640 1 --2- 172.21.15.86:0/3138210554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649b653de80 0x5649b6552800 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.255+0000 7f28ced9d640 1 -- 172.21.15.86:0/3138210554 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5649b6546a00 msgr2=0x5649b6545d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.255+0000 7f28ced9d640 1 --2- 172.21.15.86:0/3138210554 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5649b6546a00 0x5649b6545d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.255+0000 7f28ced9d640 1 -- 172.21.15.86:0/3138210554 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5649b653dc20 con 0x5649b653d4d0 2024-04-21T08:44:16.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.255+0000 7f28cf59e640 1 --2- 172.21.15.86:0/3138210554 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5649b6546a00 0x5649b6545d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:16.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.255+0000 7f28c7fff640 1 --2- 172.21.15.86:0/3138210554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649b653de80 0x5649b6552800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:16.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.255+0000 7f28ced9d640 1 --2- 172.21.15.86:0/3138210554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649b653d4d0 0x5649b6536e40 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f28c00096a0 tx=0x7f28c00050e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b1f4b403a8ed427 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:16.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.255+0000 7f28c4ff9640 1 -- 172.21.15.86:0/3138210554 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c004c040 con 0x5649b653d4d0 2024-04-21T08:44:16.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.255+0000 7f28c4ff9640 1 -- 172.21.15.86:0/3138210554 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28c0055030 con 0x5649b653d4d0 2024-04-21T08:44:16.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.255+0000 7f28c4ff9640 1 -- 172.21.15.86:0/3138210554 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c0043040 con 0x5649b653d4d0 2024-04-21T08:44:16.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.256+0000 7f28d7d6c740 1 -- 172.21.15.86:0/3138210554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649b653d4d0 msgr2=0x5649b6536e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.256+0000 7f28d7d6c740 1 --2- 172.21.15.86:0/3138210554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649b653d4d0 0x5649b6536e40 secure :-1 s=READY pgs=1266 cs=0 l=1 rev1=1 crypto rx=0x7f28c00096a0 tx=0x7f28c00050e0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.256+0000 7f28d7d6c740 1 -- 172.21.15.86:0/3138210554 shutdown_connections 2024-04-21T08:44:16.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.256+0000 7f28d7d6c740 1 --2- 172.21.15.86:0/3138210554 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5649b6546a00 0x5649b6545d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.256+0000 7f28d7d6c740 1 --2- 172.21.15.86:0/3138210554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649b653de80 0x5649b6552800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.256+0000 7f28d7d6c740 1 --2- 172.21.15.86:0/3138210554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649b653d4d0 0x5649b6536e40 unknown :-1 s=CLOSED pgs=1266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.256+0000 7f28d7d6c740 1 -- 172.21.15.86:0/3138210554 >> 172.21.15.86:0/3138210554 conn(0x5649b6537ba0 msgr2=0x5649b6542680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:16.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.256+0000 7f28d7d6c740 1 -- 172.21.15.86:0/3138210554 shutdown_connections 2024-04-21T08:44:16.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.256+0000 7f28d7d6c740 1 -- 172.21.15.86:0/3138210554 wait complete. 2024-04-21T08:44:16.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.256+0000 7f28d7d6c740 1 -- start start 2024-04-21T08:44:16.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.257+0000 7f28d7d6c740 1 Processor -- start 2024-04-21T08:44:16.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.257+0000 7f28d7d6c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649b653d4d0 0x5649b654ec70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.257+0000 7f28d7d6c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649b653de80 0x5649b654f1b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.257+0000 7f28d7d6c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5649b6546a00 0x5649b654a140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.257+0000 7f28d7d6c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f28d01b07a0 con 0x5649b653d4d0 2024-04-21T08:44:16.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.257+0000 7f28d7d6c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f28d01b00d0 con 0x5649b6546a00 2024-04-21T08:44:16.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.257+0000 7f28d7d6c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f28d01ad2f0 con 0x5649b653de80 2024-04-21T08:44:16.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.257+0000 7f28d7d6c740 10 client.0 _ll_register_callbacks cb 0x7f28c002c990 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:16.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.257+0000 7f28ced9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649b653d4d0 0x5649b654ec70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.257+0000 7f28ced9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649b653d4d0 0x5649b654ec70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34806/0 (socket says 172.21.15.86:34806) 2024-04-21T08:44:16.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.257+0000 7f28ced9d640 1 -- 172.21.15.86:0/3313400702 learned_addr learned my addr 172.21.15.86:0/3313400702 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:16.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.257+0000 7f28c7fff640 1 --2- 172.21.15.86:0/3313400702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649b653de80 0x5649b654f1b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.257+0000 7f28cf59e640 1 --2- 172.21.15.86:0/3313400702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5649b6546a00 0x5649b654a140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.257+0000 7f28ced9d640 1 -- 172.21.15.86:0/3313400702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649b653de80 msgr2=0x5649b654f1b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.257+0000 7f28ced9d640 1 --2- 172.21.15.86:0/3313400702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649b653de80 0x5649b654f1b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.257+0000 7f28ced9d640 1 -- 172.21.15.86:0/3313400702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5649b6546a00 msgr2=0x5649b654a140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.257+0000 7f28ced9d640 1 --2- 172.21.15.86:0/3313400702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5649b6546a00 0x5649b654a140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.257+0000 7f28ced9d640 1 -- 172.21.15.86:0/3313400702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5649b654a970 con 0x5649b653d4d0 2024-04-21T08:44:16.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.257+0000 7f28cf59e640 1 --2- 172.21.15.86:0/3313400702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5649b6546a00 0x5649b654a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:16.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.257+0000 7f28ced9d640 1 --2- 172.21.15.86:0/3313400702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649b653d4d0 0x5649b654ec70 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f28c0004600 tx=0x7f28c002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a287dddf81d23503 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:16.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.257+0000 7f28b67fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:16.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.257+0000 7f28b67fc640 1 -- 172.21.15.86:0/3313400702 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f28c0043070 con 0x5649b653d4d0 2024-04-21T08:44:16.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.257+0000 7f28b67fc640 1 -- 172.21.15.86:0/3313400702 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f28c0004ad0 con 0x5649b653d4d0 2024-04-21T08:44:16.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.257+0000 7f28d7d6c740 1 -- 172.21.15.86:0/3313400702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f28c0055030 con 0x5649b653d4d0 2024-04-21T08:44:16.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.258+0000 7f28b67fc640 1 -- 172.21.15.86:0/3313400702 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f28c006a050 con 0x5649b653d4d0 2024-04-21T08:44:16.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.258+0000 7f28d7d6c740 10 client.17976 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:16.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.258+0000 7f28d7d6c740 1 -- 172.21.15.86:0/3313400702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5649b65bc810 con 0x5649b653d4d0 2024-04-21T08:44:16.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.258+0000 7f28b67fc640 1 -- 172.21.15.86:0/3313400702 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f28c0055030 con 0x5649b653d4d0 2024-04-21T08:44:16.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.258+0000 7f28b67fc640 1 -- 172.21.15.86:0/3313400702 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f28c00484a0 con 0x5649b653d4d0 2024-04-21T08:44:16.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.258+0000 7f28d7d6c740 10 client.17976 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:16.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.258+0000 7f28d7d6c740 10 client.17976 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:44:16.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.258+0000 7f28d7d6c740 20 client.17976 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:16.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.258+0000 7f28d7d6c740 1 --2- 172.21.15.86:0/3313400702 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x5649b65bcfb0 0x5649b65bf3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.258+0000 7f28d7d6c740 4 client.17976 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:44:16.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.258+0000 7f28d7d6c740 1 -- 172.21.15.86:0/3313400702 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5649b64afc10 con 0x5649b65bcfb0 2024-04-21T08:44:16.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.258+0000 7f28c7fff640 1 --2- 172.21.15.86:0/3313400702 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x5649b65bcfb0 0x5649b65bf3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.258+0000 7f28b67fc640 10 client.17976 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:44:16.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.258+0000 7f28c7fff640 1 --2- 172.21.15.86:0/3313400702 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x5649b65bcfb0 0x5649b65bf3d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5c31f2b5b5a4afc7 server_cookie=9995f7c9a328677a in_seq=0 out_seq=0 2024-04-21T08:44:16.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.259+0000 7f28b67fc640 1 -- 172.21.15.86:0/3313400702 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5649b64afc10 con 0x5649b65bcfb0 2024-04-21T08:44:16.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.259+0000 7f28b67fc640 10 client.17976 handle_command_reply: tid=0 2024-04-21T08:44:16.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.259+0000 7f28d7d6c740 1 client.17976 shutdown 2024-04-21T08:44:16.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.259+0000 7f28d7d6c740 1 -- 172.21.15.86:0/3313400702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649b653d4d0 msgr2=0x5649b654ec70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.259+0000 7f28d7d6c740 1 --2- 172.21.15.86:0/3313400702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649b653d4d0 0x5649b654ec70 secure :-1 s=READY pgs=1267 cs=0 l=1 rev1=1 crypto rx=0x7f28c0004600 tx=0x7f28c002bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:16.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.259+0000 7f28d7d6c740 1 -- 172.21.15.86:0/3313400702 shutdown_connections 2024-04-21T08:44:16.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.259+0000 7f28d7d6c740 1 --2- 172.21.15.86:0/3313400702 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x5649b65bcfb0 0x5649b65bf3d0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.259+0000 7f28d7d6c740 1 --2- 172.21.15.86:0/3313400702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5649b6546a00 0x5649b654a140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.259+0000 7f28d7d6c740 1 --2- 172.21.15.86:0/3313400702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5649b653de80 0x5649b654f1b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.259+0000 7f28d7d6c740 1 --2- 172.21.15.86:0/3313400702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5649b653d4d0 0x5649b654ec70 unknown :-1 s=CLOSED pgs=1267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.266 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.259+0000 7f28b67fc640 0 client.17976 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:44:16.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.259+0000 7f28d7d6c740 1 -- 172.21.15.86:0/3313400702 >> 172.21.15.86:0/3313400702 conn(0x5649b6537ba0 msgr2=0x5649b6541df0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:16.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.259+0000 7f28d7d6c740 1 -- 172.21.15.86:0/3313400702 shutdown_connections 2024-04-21T08:44:16.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.259+0000 7f28d7d6c740 1 -- 172.21.15.86:0/3313400702 wait complete. 2024-04-21T08:44:16.267 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.259+0000 7f28d7d6c740 20 client.17976 trim_cache size 0 max 16384 2024-04-21T08:44:16.267 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:44:16.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.259+0000 7f28d517e640 1 -- 172.21.15.86:0/2533497138 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f28bc0a3d20 msgr2=0x7f28bc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.259+0000 7f28d517e640 1 --2- 172.21.15.86:0/2533497138 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f28bc0a3d20 0x7f28bc0a61e0 secure :-1 s=READY pgs=199 cs=0 l=1 rev1=1 crypto rx=0x7f28c8005c40 tx=0x7f28c8009280 comp rx=0 tx=0).stop 2024-04-21T08:44:16.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.259+0000 7f28d517e640 1 -- 172.21.15.86:0/2533497138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28d0107260 msgr2=0x7f28d0198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.259+0000 7f28d517e640 1 --2- 172.21.15.86:0/2533497138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28d0107260 0x7f28d0198c60 secure :-1 s=READY pgs=1263 cs=0 l=1 rev1=1 crypto rx=0x7f28b8005fd0 tx=0x7f28b800dde0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.259+0000 7f28d517e640 1 -- 172.21.15.86:0/2533497138 shutdown_connections 2024-04-21T08:44:16.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.259+0000 7f28d517e640 1 --2- 172.21.15.86:0/2533497138 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f28bc0a3d20 0x7f28bc0a61e0 unknown :-1 s=CLOSED pgs=199 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.259+0000 7f28d517e640 1 --2- 172.21.15.86:0/2533497138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f28d019cfe0 0x7f28d019d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.259+0000 7f28d517e640 1 --2- 172.21.15.86:0/2533497138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f28d0107260 0x7f28d0198c60 unknown :-1 s=CLOSED pgs=1263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.259+0000 7f28d517e640 1 --2- 172.21.15.86:0/2533497138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f28d0102190 0x7f28d0198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.260+0000 7f28d517e640 1 -- 172.21.15.86:0/2533497138 >> 172.21.15.86:0/2533497138 conn(0x7f28d00fd990 msgr2=0x7f28d00fed40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:16.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.260+0000 7f28d517e640 1 -- 172.21.15.86:0/2533497138 shutdown_connections 2024-04-21T08:44:16.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.260+0000 7f28d517e640 1 -- 172.21.15.86:0/2533497138 wait complete. 2024-04-21T08:44:16.270 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:44:16.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.372+0000 7f077d6fd640 1 -- 172.21.15.86:0/1642078893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f077810c510 msgr2=0x7f077810e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.372+0000 7f077d6fd640 1 --2- 172.21.15.86:0/1642078893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f077810c510 0x7f077810e9b0 secure :-1 s=READY pgs=1268 cs=0 l=1 rev1=1 crypto rx=0x7f0770009900 tx=0x7f077002bd60 comp rx=0 tx=0).stop 2024-04-21T08:44:16.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.372+0000 7f077d6fd640 1 -- 172.21.15.86:0/1642078893 shutdown_connections 2024-04-21T08:44:16.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.372+0000 7f077d6fd640 1 --2- 172.21.15.86:0/1642078893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f077810c510 0x7f077810e9b0 unknown :-1 s=CLOSED pgs=1268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.372+0000 7f077d6fd640 1 --2- 172.21.15.86:0/1642078893 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0778103af0 0x7f077810bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.372+0000 7f077d6fd640 1 --2- 172.21.15.86:0/1642078893 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f07781031d0 0x7f07781035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.372+0000 7f077d6fd640 1 -- 172.21.15.86:0/1642078893 >> 172.21.15.86:0/1642078893 conn(0x7f0778077f50 msgr2=0x7f0778078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:16.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.372+0000 7f077d6fd640 1 -- 172.21.15.86:0/1642078893 shutdown_connections 2024-04-21T08:44:16.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.372+0000 7f077d6fd640 1 -- 172.21.15.86:0/1642078893 wait complete. 2024-04-21T08:44:16.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.373+0000 7f077d6fd640 1 Processor -- start 2024-04-21T08:44:16.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.373+0000 7f077d6fd640 1 -- start start 2024-04-21T08:44:16.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.373+0000 7f077d6fd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07781031d0 0x7f07781a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.373+0000 7f077d6fd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0778103af0 0x7f07781a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.373+0000 7f077d6fd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f077810c510 0x7f077819a710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.373+0000 7f077d6fd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f07781121a0 con 0x7f077810c510 2024-04-21T08:44:16.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.373+0000 7f077d6fd640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0778112020 con 0x7f07781031d0 2024-04-21T08:44:16.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.373+0000 7f077d6fd640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0778112320 con 0x7f0778103af0 2024-04-21T08:44:16.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.374+0000 7f07777fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f077810c510 0x7f077819a710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.374+0000 7f07777fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f077810c510 0x7f077819a710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34822/0 (socket says 172.21.15.86:34822) 2024-04-21T08:44:16.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.374+0000 7f07777fe640 1 -- 172.21.15.86:0/3084761441 learned_addr learned my addr 172.21.15.86:0/3084761441 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:16.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.374+0000 7f07767fc640 1 --2- 172.21.15.86:0/3084761441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0778103af0 0x7f07781a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.374+0000 7f0776ffd640 1 --2- 172.21.15.86:0/3084761441 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07781031d0 0x7f07781a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.374+0000 7f07777fe640 1 -- 172.21.15.86:0/3084761441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0778103af0 msgr2=0x7f07781a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.374+0000 7f07777fe640 1 --2- 172.21.15.86:0/3084761441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0778103af0 0x7f07781a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.374+0000 7f07777fe640 1 -- 172.21.15.86:0/3084761441 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07781031d0 msgr2=0x7f07781a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.374+0000 7f07777fe640 1 --2- 172.21.15.86:0/3084761441 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07781031d0 0x7f07781a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.374+0000 7f07777fe640 1 -- 172.21.15.86:0/3084761441 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f077819afd0 con 0x7f077810c510 2024-04-21T08:44:16.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.374+0000 7f0776ffd640 1 --2- 172.21.15.86:0/3084761441 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07781031d0 0x7f07781a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:16.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.374+0000 7f07777fe640 1 --2- 172.21.15.86:0/3084761441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f077810c510 0x7f077819a710 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f0770009230 tx=0x7f07700045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:16.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.374+0000 7f075ffff640 1 -- 172.21.15.86:0/3084761441 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0770043020 con 0x7f077810c510 2024-04-21T08:44:16.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.374+0000 7f075ffff640 1 -- 172.21.15.86:0/3084761441 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f077002dcb0 con 0x7f077810c510 2024-04-21T08:44:16.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.374+0000 7f077d6fd640 1 -- 172.21.15.86:0/3084761441 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f077819b260 con 0x7f077810c510 2024-04-21T08:44:16.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.374+0000 7f075ffff640 1 -- 172.21.15.86:0/3084761441 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0770003a90 con 0x7f077810c510 2024-04-21T08:44:16.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.374+0000 7f077d6fd640 1 -- 172.21.15.86:0/3084761441 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f07781a75d0 con 0x7f077810c510 2024-04-21T08:44:16.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.376+0000 7f075ffff640 1 -- 172.21.15.86:0/3084761441 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0770046020 con 0x7f077810c510 2024-04-21T08:44:16.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.377+0000 7f07803ca740 1 Processor -- start 2024-04-21T08:44:16.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.377+0000 7f07803ca740 1 -- start start 2024-04-21T08:44:16.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.377+0000 7f07803ca740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561bfef6d4d0 0x561bfef6d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.377+0000 7f07803ca740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bfef6de80 0x561bfef76430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.377+0000 7f07803ca740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561bfef76a00 0x561bfef78e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.377+0000 7f07803ca740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561bfeed21f0 con 0x561bfef6de80 2024-04-21T08:44:16.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.377+0000 7f07803ca740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x561bfee6f4b0 con 0x561bfef76a00 2024-04-21T08:44:16.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.377+0000 7f07803ca740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561bfeece1b0 con 0x561bfef6d4d0 2024-04-21T08:44:16.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.377+0000 7f075ffff640 1 --2- 172.21.15.86:0/3084761441 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f07600a3d20 0x7f07600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.377+0000 7f0776ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bfef6de80 0x561bfef76430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.377+0000 7f0776ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bfef6de80 0x561bfef76430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34824/0 (socket says 172.21.15.86:34824) 2024-04-21T08:44:16.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.377+0000 7f0776ffd640 1 -- 172.21.15.86:0/4259225106 learned_addr learned my addr 172.21.15.86:0/4259225106 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:16.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.377+0000 7f07767fc640 1 --2- 172.21.15.86:0/4259225106 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561bfef6d4d0 0x561bfef6d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.377+0000 7f07777fe640 1 --2- 172.21.15.86:0/3084761441 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f07600a3d20 0x7f07600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.377+0000 7f07767fc640 1 --2- 172.21.15.86:0/4259225106 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561bfef76a00 0x561bfef78e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.377+0000 7f075ffff640 1 -- 172.21.15.86:0/3084761441 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f07700f2c60 con 0x7f077810c510 2024-04-21T08:44:16.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.377+0000 7f0776ffd640 1 -- 172.21.15.86:0/4259225106 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561bfef6d4d0 msgr2=0x561bfef6d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.377+0000 7f0776ffd640 1 --2- 172.21.15.86:0/4259225106 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561bfef6d4d0 0x561bfef6d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.390 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.377+0000 7f0776ffd640 1 -- 172.21.15.86:0/4259225106 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561bfef76a00 msgr2=0x561bfef78e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.377+0000 7f0776ffd640 1 --2- 172.21.15.86:0/4259225106 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561bfef76a00 0x561bfef78e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.377+0000 7f0776ffd640 1 -- 172.21.15.86:0/4259225106 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bfee611b0 con 0x561bfef6de80 2024-04-21T08:44:16.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.377+0000 7f07767fc640 1 --2- 172.21.15.86:0/4259225106 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561bfef6d4d0 0x561bfef6d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:44:16.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.378+0000 7f07777fe640 1 --2- 172.21.15.86:0/3084761441 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f07600a3d20 0x7f07600a61e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f07700bb370 tx=0x7f077008d350 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:16.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.378+0000 7f0776ffd640 1 --2- 172.21.15.86:0/4259225106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bfef6de80 0x561bfef76430 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f076800af40 tx=0x7f076800f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac913ff535b99a05 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:16.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.378+0000 7f075d7fa640 1 -- 172.21.15.86:0/4259225106 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0768004d20 con 0x561bfef6de80 2024-04-21T08:44:16.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.378+0000 7f075d7fa640 1 -- 172.21.15.86:0/4259225106 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0768011b00 con 0x561bfef6de80 2024-04-21T08:44:16.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.378+0000 7f075d7fa640 1 -- 172.21.15.86:0/4259225106 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07680055f0 con 0x561bfef6de80 2024-04-21T08:44:16.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.378+0000 7f07803ca740 1 -- 172.21.15.86:0/4259225106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bfef6de80 msgr2=0x561bfef76430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.378+0000 7f07803ca740 1 --2- 172.21.15.86:0/4259225106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bfef6de80 0x561bfef76430 secure :-1 s=READY pgs=1270 cs=0 l=1 rev1=1 crypto rx=0x7f076800af40 tx=0x7f076800f7f0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.378+0000 7f07803ca740 1 -- 172.21.15.86:0/4259225106 shutdown_connections 2024-04-21T08:44:16.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.379+0000 7f07803ca740 1 --2- 172.21.15.86:0/4259225106 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561bfef76a00 0x561bfef78e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.379+0000 7f07803ca740 1 --2- 172.21.15.86:0/4259225106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bfef6de80 0x561bfef76430 unknown :-1 s=CLOSED pgs=1270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.379+0000 7f07803ca740 1 --2- 172.21.15.86:0/4259225106 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561bfef6d4d0 0x561bfef6d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.379+0000 7f07803ca740 1 -- 172.21.15.86:0/4259225106 >> 172.21.15.86:0/4259225106 conn(0x561bfef67ba0 msgr2=0x561bfef67f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:16.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.379+0000 7f07803ca740 1 -- 172.21.15.86:0/4259225106 shutdown_connections 2024-04-21T08:44:16.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.379+0000 7f07803ca740 1 -- 172.21.15.86:0/4259225106 wait complete. 2024-04-21T08:44:16.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.379+0000 7f07803ca740 1 -- start start 2024-04-21T08:44:16.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.379+0000 7f07803ca740 1 Processor -- start 2024-04-21T08:44:16.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.379+0000 7f07803ca740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561bfef6d4d0 0x561bfef76350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.379+0000 7f07803ca740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561bfef6de80 0x561bfef736e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.379+0000 7f07803ca740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bfef76a00 0x561bfef73cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.379+0000 7f07803ca740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561bfef79970 con 0x561bfef76a00 2024-04-21T08:44:16.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.379+0000 7f07803ca740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x561bfee8b500 con 0x561bfef6de80 2024-04-21T08:44:16.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.379+0000 7f07803ca740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561bfef79af0 con 0x561bfef6d4d0 2024-04-21T08:44:16.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.379+0000 7f07803ca740 10 client.0 _ll_register_callbacks cb 0x561bfee59b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:16.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.379+0000 7f07767fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bfef76a00 0x561bfef73cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.379+0000 7f07767fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bfef76a00 0x561bfef73cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34836/0 (socket says 172.21.15.86:34836) 2024-04-21T08:44:16.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.379+0000 7f07767fc640 1 -- 172.21.15.86:0/3168530468 learned_addr learned my addr 172.21.15.86:0/3168530468 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:16.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.379+0000 7f0776ffd640 1 --2- 172.21.15.86:0/3168530468 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561bfef6de80 0x561bfef736e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.379+0000 7f07767fc640 1 --2- 172.21.15.86:0/3168530468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561bfef6d4d0 0x561bfef76350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.380+0000 7f07767fc640 1 -- 172.21.15.86:0/3168530468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561bfef6d4d0 msgr2=0x561bfef76350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.380+0000 7f07767fc640 1 --2- 172.21.15.86:0/3168530468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561bfef6d4d0 0x561bfef76350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.380+0000 7f07767fc640 1 -- 172.21.15.86:0/3168530468 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561bfef6de80 msgr2=0x561bfef736e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.380+0000 7f07767fc640 1 --2- 172.21.15.86:0/3168530468 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561bfef6de80 0x561bfef736e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.380+0000 7f07767fc640 1 -- 172.21.15.86:0/3168530468 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bfebe43f0 con 0x561bfef76a00 2024-04-21T08:44:16.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.380+0000 7f07767fc640 1 --2- 172.21.15.86:0/3168530468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bfef76a00 0x561bfef73cf0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f076c005890 tx=0x7f076c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be8c4a5cde9de687 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:16.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.380+0000 7f075aff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:16.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.380+0000 7f075aff5640 1 -- 172.21.15.86:0/3168530468 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c00dce0 con 0x561bfef76a00 2024-04-21T08:44:16.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.380+0000 7f075aff5640 1 -- 172.21.15.86:0/3168530468 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f076c004020 con 0x561bfef76a00 2024-04-21T08:44:16.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.380+0000 7f07803ca740 1 -- 172.21.15.86:0/3168530468 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0768011b00 con 0x561bfef76a00 2024-04-21T08:44:16.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.380+0000 7f075aff5640 1 -- 172.21.15.86:0/3168530468 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f076c028050 con 0x561bfef76a00 2024-04-21T08:44:16.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.380+0000 7f07803ca740 10 client.17988 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:16.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.380+0000 7f07803ca740 1 -- 172.21.15.86:0/3168530468 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561bfecae7d0 con 0x561bfef76a00 2024-04-21T08:44:16.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.380+0000 7f075aff5640 1 -- 172.21.15.86:0/3168530468 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0768011b00 con 0x561bfef76a00 2024-04-21T08:44:16.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.380+0000 7f075aff5640 1 -- 172.21.15.86:0/3168530468 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f076c0047a0 con 0x561bfef76a00 2024-04-21T08:44:16.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.380+0000 7f07803ca740 10 client.17988 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:16.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.380+0000 7f07803ca740 10 client.17988 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:44:16.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.380+0000 7f07803ca740 20 client.17988 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:16.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.380+0000 7f07803ca740 1 --2- 172.21.15.86:0/3168530468 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x561bfef86c80 0x561bfef89070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.380+0000 7f07803ca740 4 client.17988 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:16.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.380+0000 7f07803ca740 1 -- 172.21.15.86:0/3168530468 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561bfee71930 con 0x561bfef86c80 2024-04-21T08:44:16.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.381+0000 7f0776ffd640 1 --2- 172.21.15.86:0/3168530468 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x561bfef86c80 0x561bfef89070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.381+0000 7f075aff5640 10 client.17988 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:44:16.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.381+0000 7f0776ffd640 1 --2- 172.21.15.86:0/3168530468 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x561bfef86c80 0x561bfef89070 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=28409a5e31ab65c2 server_cookie=95272924a761f696 in_seq=0 out_seq=0 2024-04-21T08:44:16.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.382+0000 7f075aff5640 1 -- 172.21.15.86:0/3168530468 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561bfee71930 con 0x561bfef86c80 2024-04-21T08:44:16.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.382+0000 7f075aff5640 10 client.17988 handle_command_reply: tid=0 2024-04-21T08:44:16.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.382+0000 7f07803ca740 1 client.17988 shutdown 2024-04-21T08:44:16.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.382+0000 7f07803ca740 1 -- 172.21.15.86:0/3168530468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bfef76a00 msgr2=0x561bfef73cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.382+0000 7f07803ca740 1 --2- 172.21.15.86:0/3168530468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bfef76a00 0x561bfef73cf0 secure :-1 s=READY pgs=1271 cs=0 l=1 rev1=1 crypto rx=0x7f076c005890 tx=0x7f076c0046c0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.382+0000 7f07803ca740 1 -- 172.21.15.86:0/3168530468 shutdown_connections 2024-04-21T08:44:16.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.382+0000 7f07803ca740 1 --2- 172.21.15.86:0/3168530468 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x561bfef86c80 0x561bfef89070 crc :-1 s=READY pgs=53 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.382+0000 7f07803ca740 1 --2- 172.21.15.86:0/3168530468 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bfef76a00 0x561bfef73cf0 unknown :-1 s=CLOSED pgs=1271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.382+0000 7f075aff5640 0 client.17988 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:44:16.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.382+0000 7f07803ca740 1 --2- 172.21.15.86:0/3168530468 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561bfef6de80 0x561bfef736e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.382+0000 7f07803ca740 1 --2- 172.21.15.86:0/3168530468 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561bfef6d4d0 0x561bfef76350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.382+0000 7f07803ca740 1 -- 172.21.15.86:0/3168530468 >> 172.21.15.86:0/3168530468 conn(0x561bfef67ba0 msgr2=0x561bfef698f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:16.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.382+0000 7f07803ca740 1 -- 172.21.15.86:0/3168530468 shutdown_connections 2024-04-21T08:44:16.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.382+0000 7f07803ca740 1 -- 172.21.15.86:0/3168530468 wait complete. 2024-04-21T08:44:16.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.382+0000 7f07803ca740 20 client.17988 trim_cache size 0 max 16384 2024-04-21T08:44:16.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.395+0000 7f07803ca740 1 Processor -- start 2024-04-21T08:44:16.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.395+0000 7f07803ca740 1 -- start start 2024-04-21T08:44:16.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.395+0000 7f07803ca740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561bfef6d4d0 0x561bfef82860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.395+0000 7f07803ca740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bfef6de80 0x561bfef66a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.395+0000 7f07803ca740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561bfef76a00 0x561bfef75e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.395+0000 7f07803ca740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f07781b14c0 con 0x561bfef6de80 2024-04-21T08:44:16.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.395+0000 7f07803ca740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f07781b3000 con 0x561bfef76a00 2024-04-21T08:44:16.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.395+0000 7f07803ca740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f07781b1b90 con 0x561bfef6d4d0 2024-04-21T08:44:16.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.396+0000 7f0776ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bfef6de80 0x561bfef66a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.396+0000 7f0776ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bfef6de80 0x561bfef66a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34848/0 (socket says 172.21.15.86:34848) 2024-04-21T08:44:16.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.396+0000 7f0776ffd640 1 -- 172.21.15.86:0/3964815039 learned_addr learned my addr 172.21.15.86:0/3964815039 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:16.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.396+0000 7f07767fc640 1 --2- 172.21.15.86:0/3964815039 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561bfef76a00 0x561bfef75e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.405 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.396+0000 7f0776ffd640 1 -- 172.21.15.86:0/3964815039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561bfef6d4d0 msgr2=0x561bfef82860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.396+0000 7f0776ffd640 1 --2- 172.21.15.86:0/3964815039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561bfef6d4d0 0x561bfef82860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.396+0000 7f0776ffd640 1 -- 172.21.15.86:0/3964815039 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561bfef76a00 msgr2=0x561bfef75e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.396+0000 7f0776ffd640 1 --2- 172.21.15.86:0/3964815039 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561bfef76a00 0x561bfef75e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.396+0000 7f0776ffd640 1 -- 172.21.15.86:0/3964815039 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561bfef62040 con 0x561bfef6de80 2024-04-21T08:44:16.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.396+0000 7f07767fc640 1 --2- 172.21.15.86:0/3964815039 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561bfef76a00 0x561bfef75e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:16.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.396+0000 7f0776ffd640 1 --2- 172.21.15.86:0/3964815039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bfef6de80 0x561bfef66a80 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f0768000f80 tx=0x7f0768005270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11d4efb21e35b365 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:16.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.396+0000 7f075cff9640 1 -- 172.21.15.86:0/3964815039 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f07680116d0 con 0x561bfef6de80 2024-04-21T08:44:16.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.396+0000 7f075cff9640 1 -- 172.21.15.86:0/3964815039 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0768003e20 con 0x561bfef6de80 2024-04-21T08:44:16.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.396+0000 7f075cff9640 1 -- 172.21.15.86:0/3964815039 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0768004ef0 con 0x561bfef6de80 2024-04-21T08:44:16.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.397+0000 7f07803ca740 1 -- 172.21.15.86:0/3964815039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bfef6de80 msgr2=0x561bfef66a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.397+0000 7f07803ca740 1 --2- 172.21.15.86:0/3964815039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bfef6de80 0x561bfef66a80 secure :-1 s=READY pgs=1272 cs=0 l=1 rev1=1 crypto rx=0x7f0768000f80 tx=0x7f0768005270 comp rx=0 tx=0).stop 2024-04-21T08:44:16.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.397+0000 7f07803ca740 1 -- 172.21.15.86:0/3964815039 shutdown_connections 2024-04-21T08:44:16.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.397+0000 7f07803ca740 1 --2- 172.21.15.86:0/3964815039 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561bfef76a00 0x561bfef75e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.397+0000 7f07803ca740 1 --2- 172.21.15.86:0/3964815039 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bfef6de80 0x561bfef66a80 unknown :-1 s=CLOSED pgs=1272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.397+0000 7f07803ca740 1 --2- 172.21.15.86:0/3964815039 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561bfef6d4d0 0x561bfef82860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.397+0000 7f07803ca740 1 -- 172.21.15.86:0/3964815039 >> 172.21.15.86:0/3964815039 conn(0x561bfef67ba0 msgr2=0x561bfef66370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:16.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.397+0000 7f07803ca740 1 -- 172.21.15.86:0/3964815039 shutdown_connections 2024-04-21T08:44:16.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.397+0000 7f07803ca740 1 -- 172.21.15.86:0/3964815039 wait complete. 2024-04-21T08:44:16.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.397+0000 7f07803ca740 1 -- start start 2024-04-21T08:44:16.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.397+0000 7f07803ca740 1 Processor -- start 2024-04-21T08:44:16.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.397+0000 7f07803ca740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bfef6d4d0 0x561bfef7ef60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.397+0000 7f07803ca740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561bfef6de80 0x561bfef6ff50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.397+0000 7f07803ca740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561bfef76a00 0x561bfef70490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.397+0000 7f07803ca740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f07781b2930 con 0x561bfef6d4d0 2024-04-21T08:44:16.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.397+0000 7f07803ca740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f07781b2260 con 0x561bfef6de80 2024-04-21T08:44:16.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.397+0000 7f07803ca740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f07781af480 con 0x561bfef76a00 2024-04-21T08:44:16.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.397+0000 7f07803ca740 10 client.0 _ll_register_callbacks cb 0x7f076c00bcc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:16.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.397+0000 7f07767fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bfef6d4d0 0x561bfef7ef60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.397+0000 7f07767fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bfef6d4d0 0x561bfef7ef60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34850/0 (socket says 172.21.15.86:34850) 2024-04-21T08:44:16.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.397+0000 7f07767fc640 1 -- 172.21.15.86:0/3694729009 learned_addr learned my addr 172.21.15.86:0/3694729009 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:16.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.397+0000 7f0776ffd640 1 --2- 172.21.15.86:0/3694729009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561bfef6de80 0x561bfef6ff50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.398+0000 7f07767fc640 1 -- 172.21.15.86:0/3694729009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561bfef76a00 msgr2=0x561bfef70490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.398+0000 7f07767fc640 1 --2- 172.21.15.86:0/3694729009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561bfef76a00 0x561bfef70490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.398+0000 7f07767fc640 1 -- 172.21.15.86:0/3694729009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561bfef6de80 msgr2=0x561bfef6ff50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.398+0000 7f07767fc640 1 --2- 172.21.15.86:0/3694729009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561bfef6de80 0x561bfef6ff50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.398+0000 7f07767fc640 1 -- 172.21.15.86:0/3694729009 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561bfef709d0 con 0x561bfef6d4d0 2024-04-21T08:44:16.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.398+0000 7f07767fc640 1 --2- 172.21.15.86:0/3694729009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bfef6d4d0 0x561bfef7ef60 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f076c005c30 tx=0x7f076c00b410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8ef9a4ad13a69b1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:16.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.398+0000 7f075aff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:16.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.398+0000 7f075aff5640 1 -- 172.21.15.86:0/3694729009 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f076c031060 con 0x561bfef6d4d0 2024-04-21T08:44:16.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.398+0000 7f075aff5640 1 -- 172.21.15.86:0/3694729009 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f076c00dce0 con 0x561bfef6d4d0 2024-04-21T08:44:16.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.398+0000 7f07803ca740 1 -- 172.21.15.86:0/3694729009 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0768003e20 con 0x561bfef6d4d0 2024-04-21T08:44:16.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.398+0000 7f075aff5640 1 -- 172.21.15.86:0/3694729009 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f076c00be90 con 0x561bfef6d4d0 2024-04-21T08:44:16.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.398+0000 7f07803ca740 10 client.17994 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:16.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.398+0000 7f07803ca740 1 -- 172.21.15.86:0/3694729009 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561bfefecd40 con 0x561bfef6d4d0 2024-04-21T08:44:16.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.398+0000 7f075aff5640 1 -- 172.21.15.86:0/3694729009 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0768003e20 con 0x561bfef6d4d0 2024-04-21T08:44:16.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.398+0000 7f075aff5640 1 -- 172.21.15.86:0/3694729009 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f076c005310 con 0x561bfef6d4d0 2024-04-21T08:44:16.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.398+0000 7f07803ca740 10 client.17994 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:16.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.398+0000 7f07803ca740 10 client.17994 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:44:16.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.398+0000 7f07803ca740 20 client.17994 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:16.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.398+0000 7f07803ca740 1 --2- 172.21.15.86:0/3694729009 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x561bfef86c80 0x561bfefef410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.398+0000 7f07803ca740 4 client.17994 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:44:16.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.398+0000 7f07803ca740 1 -- 172.21.15.86:0/3694729009 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561bfeedfc10 con 0x561bfef86c80 2024-04-21T08:44:16.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.399+0000 7f0776ffd640 1 --2- 172.21.15.86:0/3694729009 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x561bfef86c80 0x561bfefef410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.399+0000 7f075aff5640 10 client.17994 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:44:16.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.399+0000 7f0776ffd640 1 --2- 172.21.15.86:0/3694729009 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x561bfef86c80 0x561bfefef410 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=449de69522a4330d server_cookie=1a24292c8f8608d4 in_seq=0 out_seq=0 2024-04-21T08:44:16.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.399+0000 7f075aff5640 1 -- 172.21.15.86:0/3694729009 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561bfeedfc10 con 0x561bfef86c80 2024-04-21T08:44:16.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.399+0000 7f075aff5640 10 client.17994 handle_command_reply: tid=0 2024-04-21T08:44:16.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.399+0000 7f07803ca740 1 client.17994 shutdown 2024-04-21T08:44:16.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.399+0000 7f07803ca740 1 -- 172.21.15.86:0/3694729009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bfef6d4d0 msgr2=0x561bfef7ef60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.399+0000 7f07803ca740 1 --2- 172.21.15.86:0/3694729009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bfef6d4d0 0x561bfef7ef60 secure :-1 s=READY pgs=1273 cs=0 l=1 rev1=1 crypto rx=0x7f076c005c30 tx=0x7f076c00b410 comp rx=0 tx=0).stop 2024-04-21T08:44:16.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.399+0000 7f07803ca740 1 -- 172.21.15.86:0/3694729009 shutdown_connections 2024-04-21T08:44:16.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.399+0000 7f07803ca740 1 --2- 172.21.15.86:0/3694729009 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x561bfef86c80 0x561bfefef410 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.399+0000 7f07803ca740 1 --2- 172.21.15.86:0/3694729009 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561bfef76a00 0x561bfef70490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.399+0000 7f07803ca740 1 --2- 172.21.15.86:0/3694729009 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561bfef6de80 0x561bfef6ff50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.399+0000 7f075aff5640 0 client.17994 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:44:16.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.399+0000 7f07803ca740 1 --2- 172.21.15.86:0/3694729009 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561bfef6d4d0 0x561bfef7ef60 unknown :-1 s=CLOSED pgs=1273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.399+0000 7f07803ca740 1 -- 172.21.15.86:0/3694729009 >> 172.21.15.86:0/3694729009 conn(0x561bfef67ba0 msgr2=0x561bfef6eaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:16.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.399+0000 7f07803ca740 1 -- 172.21.15.86:0/3694729009 shutdown_connections 2024-04-21T08:44:16.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.399+0000 7f07803ca740 1 -- 172.21.15.86:0/3694729009 wait complete. 2024-04-21T08:44:16.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.400+0000 7f07803ca740 20 client.17994 trim_cache size 0 max 16384 2024-04-21T08:44:16.411 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:44:16.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.400+0000 7f077d6fd640 1 -- 172.21.15.86:0/3084761441 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f07600a3d20 msgr2=0x7f07600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.400+0000 7f077d6fd640 1 --2- 172.21.15.86:0/3084761441 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f07600a3d20 0x7f07600a61e0 secure :-1 s=READY pgs=200 cs=0 l=1 rev1=1 crypto rx=0x7f07700bb370 tx=0x7f077008d350 comp rx=0 tx=0).stop 2024-04-21T08:44:16.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.400+0000 7f077d6fd640 1 -- 172.21.15.86:0/3084761441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f077810c510 msgr2=0x7f077819a710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.400+0000 7f077d6fd640 1 --2- 172.21.15.86:0/3084761441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f077810c510 0x7f077819a710 secure :-1 s=READY pgs=1269 cs=0 l=1 rev1=1 crypto rx=0x7f0770009230 tx=0x7f07700045e0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.400+0000 7f077d6fd640 1 -- 172.21.15.86:0/3084761441 shutdown_connections 2024-04-21T08:44:16.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.400+0000 7f077d6fd640 1 --2- 172.21.15.86:0/3084761441 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f07600a3d20 0x7f07600a61e0 unknown :-1 s=CLOSED pgs=200 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.400+0000 7f077d6fd640 1 --2- 172.21.15.86:0/3084761441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f077810c510 0x7f077819a710 unknown :-1 s=CLOSED pgs=1269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.400+0000 7f077d6fd640 1 --2- 172.21.15.86:0/3084761441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0778103af0 0x7f07781a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.400+0000 7f077d6fd640 1 --2- 172.21.15.86:0/3084761441 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f07781031d0 0x7f07781a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.400+0000 7f077d6fd640 1 -- 172.21.15.86:0/3084761441 >> 172.21.15.86:0/3084761441 conn(0x7f0778077f50 msgr2=0x7f0778108420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:16.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.400+0000 7f077d6fd640 1 -- 172.21.15.86:0/3084761441 shutdown_connections 2024-04-21T08:44:16.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.400+0000 7f077d6fd640 1 -- 172.21.15.86:0/3084761441 wait complete. 2024-04-21T08:44:16.414 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:44:16.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.516+0000 7f016cac8640 1 -- 172.21.15.86:0/2725575671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0168107260 msgr2=0x7f016810ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.516+0000 7f016cac8640 1 --2- 172.21.15.86:0/2725575671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0168107260 0x7f016810ee30 secure :-1 s=READY pgs=1274 cs=0 l=1 rev1=1 crypto rx=0x7f0160009ca0 tx=0x7f016002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.516+0000 7f016cac8640 1 -- 172.21.15.86:0/2725575671 shutdown_connections 2024-04-21T08:44:16.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.516+0000 7f016cac8640 1 --2- 172.21.15.86:0/2725575671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0168107260 0x7f016810ee30 unknown :-1 s=CLOSED pgs=1274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.516+0000 7f016cac8640 1 --2- 172.21.15.86:0/2725575671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0168102b40 0x7f0168106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.516+0000 7f016cac8640 1 --2- 172.21.15.86:0/2725575671 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0168102190 0x7f0168102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.516+0000 7f016cac8640 1 -- 172.21.15.86:0/2725575671 >> 172.21.15.86:0/2725575671 conn(0x7f01680793a0 msgr2=0x7f01680797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:16.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.516+0000 7f016cac8640 1 -- 172.21.15.86:0/2725575671 shutdown_connections 2024-04-21T08:44:16.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.516+0000 7f016cac8640 1 -- 172.21.15.86:0/2725575671 wait complete. 2024-04-21T08:44:16.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.517+0000 7f016cac8640 1 Processor -- start 2024-04-21T08:44:16.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.517+0000 7f016cac8640 1 -- start start 2024-04-21T08:44:16.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.517+0000 7f016cac8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0168102190 0x7f0168198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.517+0000 7f016cac8640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0168107260 0x7f0168198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.517+0000 7f016cac8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f016819cff0 0x7f016819d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.517+0000 7f016cac8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f01681121d0 con 0x7f0168102190 2024-04-21T08:44:16.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.517+0000 7f016cac8640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0168112050 con 0x7f016819cff0 2024-04-21T08:44:16.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.517+0000 7f016cac8640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0168112350 con 0x7f0168107260 2024-04-21T08:44:16.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.517+0000 7f016659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0168102190 0x7f0168198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.517+0000 7f016659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0168102190 0x7f0168198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34866/0 (socket says 172.21.15.86:34866) 2024-04-21T08:44:16.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.517+0000 7f016659c640 1 -- 172.21.15.86:0/1163266636 learned_addr learned my addr 172.21.15.86:0/1163266636 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:16.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.517+0000 7f0166d9d640 1 --2- 172.21.15.86:0/1163266636 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f016819cff0 0x7f016819d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.517+0000 7f0165d9b640 1 --2- 172.21.15.86:0/1163266636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0168107260 0x7f0168198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.517+0000 7f016659c640 1 -- 172.21.15.86:0/1163266636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0168107260 msgr2=0x7f0168198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.517+0000 7f016659c640 1 --2- 172.21.15.86:0/1163266636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0168107260 0x7f0168198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.517+0000 7f016659c640 1 -- 172.21.15.86:0/1163266636 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f016819cff0 msgr2=0x7f016819d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.517+0000 7f016659c640 1 --2- 172.21.15.86:0/1163266636 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f016819cff0 0x7f016819d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.517+0000 7f016659c640 1 -- 172.21.15.86:0/1163266636 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f016819dae0 con 0x7f0168102190 2024-04-21T08:44:16.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.517+0000 7f0165d9b640 1 --2- 172.21.15.86:0/1163266636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0168107260 0x7f0168198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:16.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.517+0000 7f016659c640 1 --2- 172.21.15.86:0/1163266636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0168102190 0x7f0168198730 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f015000e9f0 tx=0x7f015000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:16.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.517+0000 7f015f7fe640 1 -- 172.21.15.86:0/1163266636 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015000cd80 con 0x7f0168102190 2024-04-21T08:44:16.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.518+0000 7f016cac8640 1 -- 172.21.15.86:0/1163266636 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f016819ddd0 con 0x7f0168102190 2024-04-21T08:44:16.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.518+0000 7f015f7fe640 1 -- 172.21.15.86:0/1163266636 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01500108e0 con 0x7f0168102190 2024-04-21T08:44:16.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.518+0000 7f015f7fe640 1 -- 172.21.15.86:0/1163266636 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0150018980 con 0x7f0168102190 2024-04-21T08:44:16.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.518+0000 7f016cac8640 1 -- 172.21.15.86:0/1163266636 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f01681a5610 con 0x7f0168102190 2024-04-21T08:44:16.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.520+0000 7f015f7fe640 1 -- 172.21.15.86:0/1163266636 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0150010430 con 0x7f0168102190 2024-04-21T08:44:16.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.520+0000 7f016f79a740 1 Processor -- start 2024-04-21T08:44:16.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.520+0000 7f016f79a740 1 -- start start 2024-04-21T08:44:16.527 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.520+0000 7f016f79a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8219624d0 0x55a8219628b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.521+0000 7f016f79a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a821962e80 0x55a82196b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.521+0000 7f016f79a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a82196ba00 0x55a82196de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.521+0000 7f016f79a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a8218c71f0 con 0x55a8219624d0 2024-04-21T08:44:16.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.521+0000 7f016f79a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55a8218644b0 con 0x55a821962e80 2024-04-21T08:44:16.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.521+0000 7f016f79a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a8218c31b0 con 0x55a82196ba00 2024-04-21T08:44:16.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.521+0000 7f015f7fe640 1 --2- 172.21.15.86:0/1163266636 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f01540a3d20 0x7f01540a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.521+0000 7f0166d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8219624d0 0x55a8219628b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.521+0000 7f0166d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8219624d0 0x55a8219628b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34882/0 (socket says 172.21.15.86:34882) 2024-04-21T08:44:16.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.521+0000 7f0166d9d640 1 -- 172.21.15.86:0/1586243259 learned_addr learned my addr 172.21.15.86:0/1586243259 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:16.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.521+0000 7f0166d9d640 1 --2- 172.21.15.86:0/1163266636 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f01540a3d20 0x7f01540a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.528 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.521+0000 7f016659c640 1 --2- 172.21.15.86:0/1586243259 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a821962e80 0x55a82196b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.521+0000 7f0165d9b640 1 --2- 172.21.15.86:0/1586243259 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a82196ba00 0x55a82196de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.521+0000 7f015f7fe640 1 -- 172.21.15.86:0/1163266636 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f01500d0f00 con 0x7f0168102190 2024-04-21T08:44:16.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.521+0000 7f0166d9d640 1 -- 172.21.15.86:0/1586243259 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a82196ba00 msgr2=0x55a82196de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.521+0000 7f0166d9d640 1 --2- 172.21.15.86:0/1586243259 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a82196ba00 0x55a82196de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.521+0000 7f0166d9d640 1 -- 172.21.15.86:0/1586243259 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a821962e80 msgr2=0x55a82196b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.521+0000 7f0166d9d640 1 --2- 172.21.15.86:0/1586243259 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a821962e80 0x55a82196b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.521+0000 7f0166d9d640 1 -- 172.21.15.86:0/1586243259 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a8218561b0 con 0x55a8219624d0 2024-04-21T08:44:16.529 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.521+0000 7f0166d9d640 1 --2- 172.21.15.86:0/1586243259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8219624d0 0x55a8219628b0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f0160037040 tx=0x7f0160032f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a69d4095a183e3cf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:16.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.521+0000 7f0166d9d640 1 --2- 172.21.15.86:0/1163266636 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f01540a3d20 0x7f01540a61e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0160009ca0 tx=0x7f0160042000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:16.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.521+0000 7f015cff9640 1 -- 172.21.15.86:0/1586243259 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0160051070 con 0x55a8219624d0 2024-04-21T08:44:16.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.521+0000 7f015cff9640 1 -- 172.21.15.86:0/1586243259 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01600327c0 con 0x55a8219624d0 2024-04-21T08:44:16.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.521+0000 7f015cff9640 1 -- 172.21.15.86:0/1586243259 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0160039040 con 0x55a8219624d0 2024-04-21T08:44:16.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.522+0000 7f016f79a740 1 -- 172.21.15.86:0/1586243259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8219624d0 msgr2=0x55a8219628b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.522+0000 7f016f79a740 1 --2- 172.21.15.86:0/1586243259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8219624d0 0x55a8219628b0 secure :-1 s=READY pgs=1276 cs=0 l=1 rev1=1 crypto rx=0x7f0160037040 tx=0x7f0160032f70 comp rx=0 tx=0).stop 2024-04-21T08:44:16.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.522+0000 7f016f79a740 1 -- 172.21.15.86:0/1586243259 shutdown_connections 2024-04-21T08:44:16.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.522+0000 7f016f79a740 1 --2- 172.21.15.86:0/1586243259 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a82196ba00 0x55a82196de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.522+0000 7f016f79a740 1 --2- 172.21.15.86:0/1586243259 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a821962e80 0x55a82196b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.522+0000 7f016f79a740 1 --2- 172.21.15.86:0/1586243259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8219624d0 0x55a8219628b0 unknown :-1 s=CLOSED pgs=1276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.522+0000 7f016f79a740 1 -- 172.21.15.86:0/1586243259 >> 172.21.15.86:0/1586243259 conn(0x55a82195cba0 msgr2=0x55a82195cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:16.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.522+0000 7f016f79a740 1 -- 172.21.15.86:0/1586243259 shutdown_connections 2024-04-21T08:44:16.530 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.522+0000 7f016f79a740 1 -- 172.21.15.86:0/1586243259 wait complete. 2024-04-21T08:44:16.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.523+0000 7f016f79a740 1 -- start start 2024-04-21T08:44:16.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.523+0000 7f016f79a740 1 Processor -- start 2024-04-21T08:44:16.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.523+0000 7f016f79a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8219624d0 0x55a82196b370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.523+0000 7f016f79a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a821962e80 0x55a821968700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.523+0000 7f016f79a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a82196ba00 0x55a821968d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.523+0000 7f016f79a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a82196e910 con 0x55a821962e80 2024-04-21T08:44:16.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.523+0000 7f016f79a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55a821880500 con 0x55a82196ba00 2024-04-21T08:44:16.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.523+0000 7f016f79a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a82196ea90 con 0x55a8219624d0 2024-04-21T08:44:16.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.523+0000 7f016f79a740 10 client.0 _ll_register_callbacks cb 0x55a82184eb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:16.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.523+0000 7f0165d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a821962e80 0x55a821968700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.523+0000 7f0165d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a821962e80 0x55a821968700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34894/0 (socket says 172.21.15.86:34894) 2024-04-21T08:44:16.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.523+0000 7f0165d9b640 1 -- 172.21.15.86:0/2235077831 learned_addr learned my addr 172.21.15.86:0/2235077831 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:16.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.523+0000 7f0166d9d640 1 --2- 172.21.15.86:0/2235077831 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a82196ba00 0x55a821968d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.531 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.523+0000 7f016659c640 1 --2- 172.21.15.86:0/2235077831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8219624d0 0x55a82196b370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.523+0000 7f0165d9b640 1 -- 172.21.15.86:0/2235077831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8219624d0 msgr2=0x55a82196b370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.523+0000 7f0165d9b640 1 --2- 172.21.15.86:0/2235077831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8219624d0 0x55a82196b370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.523+0000 7f0165d9b640 1 -- 172.21.15.86:0/2235077831 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a82196ba00 msgr2=0x55a821968d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.523+0000 7f0165d9b640 1 --2- 172.21.15.86:0/2235077831 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a82196ba00 0x55a821968d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.523+0000 7f0165d9b640 1 -- 172.21.15.86:0/2235077831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8215d93f0 con 0x55a821962e80 2024-04-21T08:44:16.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.523+0000 7f0166d9d640 1 --2- 172.21.15.86:0/2235077831 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a82196ba00 0x55a821968d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:44:16.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.523+0000 7f0165d9b640 1 --2- 172.21.15.86:0/2235077831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a821962e80 0x55a821968700 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f0158012470 tx=0x7f0158012930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9856c4a66ff3f1e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:16.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.523+0000 7f014dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:16.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.524+0000 7f014dffb640 1 -- 172.21.15.86:0/2235077831 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0158004460 con 0x55a821962e80 2024-04-21T08:44:16.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.524+0000 7f014dffb640 1 -- 172.21.15.86:0/2235077831 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f0158005020 con 0x55a821962e80 2024-04-21T08:44:16.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.524+0000 7f016f79a740 1 -- 172.21.15.86:0/2235077831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01600327c0 con 0x55a821962e80 2024-04-21T08:44:16.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.524+0000 7f014dffb640 1 -- 172.21.15.86:0/2235077831 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0158025050 con 0x55a821962e80 2024-04-21T08:44:16.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.524+0000 7f016f79a740 10 client.18006 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:16.532 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.524+0000 7f016f79a740 1 -- 172.21.15.86:0/2235077831 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a8216a37d0 con 0x55a821962e80 2024-04-21T08:44:16.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.524+0000 7f014dffb640 1 -- 172.21.15.86:0/2235077831 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01600327c0 con 0x55a821962e80 2024-04-21T08:44:16.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.524+0000 7f014dffb640 1 -- 172.21.15.86:0/2235077831 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0158019030 con 0x55a821962e80 2024-04-21T08:44:16.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.524+0000 7f016f79a740 10 client.18006 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:16.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.524+0000 7f016f79a740 10 client.18006 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:44:16.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.524+0000 7f016f79a740 20 client.18006 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:16.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.524+0000 7f016f79a740 1 --2- 172.21.15.86:0/2235077831 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55a82197bcf0 0x55a82197e110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.524+0000 7f016f79a740 4 client.18006 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:16.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.524+0000 7f016f79a740 1 -- 172.21.15.86:0/2235077831 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a821866930 con 0x55a82197bcf0 2024-04-21T08:44:16.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.524+0000 7f016659c640 1 --2- 172.21.15.86:0/2235077831 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55a82197bcf0 0x55a82197e110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.525+0000 7f016659c640 1 --2- 172.21.15.86:0/2235077831 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55a82197bcf0 0x55a82197e110 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8901330bb779fbe6 server_cookie=5e1ad0796c6fd6bf in_seq=0 out_seq=0 2024-04-21T08:44:16.533 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.525+0000 7f014dffb640 10 client.18006 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:44:16.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.525+0000 7f014dffb640 1 -- 172.21.15.86:0/2235077831 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a821866930 con 0x55a82197bcf0 2024-04-21T08:44:16.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.525+0000 7f014dffb640 10 client.18006 handle_command_reply: tid=0 2024-04-21T08:44:16.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.526+0000 7f016f79a740 1 client.18006 shutdown 2024-04-21T08:44:16.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.526+0000 7f016f79a740 1 -- 172.21.15.86:0/2235077831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a821962e80 msgr2=0x55a821968700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.526+0000 7f016f79a740 1 --2- 172.21.15.86:0/2235077831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a821962e80 0x55a821968700 secure :-1 s=READY pgs=1277 cs=0 l=1 rev1=1 crypto rx=0x7f0158012470 tx=0x7f0158012930 comp rx=0 tx=0).stop 2024-04-21T08:44:16.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.526+0000 7f016f79a740 1 -- 172.21.15.86:0/2235077831 shutdown_connections 2024-04-21T08:44:16.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.526+0000 7f016f79a740 1 --2- 172.21.15.86:0/2235077831 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a82196ba00 0x55a821968d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.526+0000 7f016f79a740 1 --2- 172.21.15.86:0/2235077831 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55a82197bcf0 0x55a82197e110 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.526+0000 7f016f79a740 1 --2- 172.21.15.86:0/2235077831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a821962e80 0x55a821968700 unknown :-1 s=CLOSED pgs=1277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.526+0000 7f016f79a740 1 --2- 172.21.15.86:0/2235077831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a8219624d0 0x55a82196b370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.526+0000 7f014dffb640 0 client.18006 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:44:16.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.526+0000 7f016f79a740 1 -- 172.21.15.86:0/2235077831 >> 172.21.15.86:0/2235077831 conn(0x55a82195cba0 msgr2=0x55a82195e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:16.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.526+0000 7f016f79a740 1 -- 172.21.15.86:0/2235077831 shutdown_connections 2024-04-21T08:44:16.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.526+0000 7f016f79a740 1 -- 172.21.15.86:0/2235077831 wait complete. 2024-04-21T08:44:16.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.526+0000 7f016f79a740 20 client.18006 trim_cache size 0 max 16384 2024-04-21T08:44:16.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.539+0000 7f016f79a740 1 Processor -- start 2024-04-21T08:44:16.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.539+0000 7f016f79a740 1 -- start start 2024-04-21T08:44:16.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.539+0000 7f016f79a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8219624d0 0x55a8219777d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.539+0000 7f016f79a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a821962e80 0x55a821967860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.539+0000 7f016f79a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a82196ba00 0x55a82195ba80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.539+0000 7f016f79a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f01681af3f0 con 0x55a8219624d0 2024-04-21T08:44:16.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.539+0000 7f016f79a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f01681b0f30 con 0x55a82196ba00 2024-04-21T08:44:16.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.539+0000 7f016f79a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f01681afac0 con 0x55a821962e80 2024-04-21T08:44:16.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.539+0000 7f016659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8219624d0 0x55a8219777d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.539+0000 7f016659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8219624d0 0x55a8219777d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34898/0 (socket says 172.21.15.86:34898) 2024-04-21T08:44:16.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.539+0000 7f016659c640 1 -- 172.21.15.86:0/193486327 learned_addr learned my addr 172.21.15.86:0/193486327 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:16.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.539+0000 7f0165d9b640 1 --2- 172.21.15.86:0/193486327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a821962e80 0x55a821967860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.539+0000 7f0166d9d640 1 --2- 172.21.15.86:0/193486327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a82196ba00 0x55a82195ba80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.539+0000 7f016659c640 1 -- 172.21.15.86:0/193486327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a821962e80 msgr2=0x55a821967860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.539+0000 7f016659c640 1 --2- 172.21.15.86:0/193486327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a821962e80 0x55a821967860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.539+0000 7f016659c640 1 -- 172.21.15.86:0/193486327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a82196ba00 msgr2=0x55a82195ba80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.539+0000 7f016659c640 1 --2- 172.21.15.86:0/193486327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a82196ba00 0x55a82195ba80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.539+0000 7f016659c640 1 -- 172.21.15.86:0/193486327 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a821969080 con 0x55a8219624d0 2024-04-21T08:44:16.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.539+0000 7f0166d9d640 1 --2- 172.21.15.86:0/193486327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a82196ba00 0x55a82195ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:16.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.539+0000 7f016659c640 1 --2- 172.21.15.86:0/193486327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8219624d0 0x55a8219777d0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f0150099500 tx=0x7f01500d0040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d692503ae57da1b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:16.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.540+0000 7f014ffff640 1 -- 172.21.15.86:0/193486327 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015009de60 con 0x55a8219624d0 2024-04-21T08:44:16.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.540+0000 7f014ffff640 1 -- 172.21.15.86:0/193486327 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01500a0ee0 con 0x55a8219624d0 2024-04-21T08:44:16.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.540+0000 7f014ffff640 1 -- 172.21.15.86:0/193486327 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0150026040 con 0x55a8219624d0 2024-04-21T08:44:16.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.540+0000 7f016f79a740 1 -- 172.21.15.86:0/193486327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8219624d0 msgr2=0x55a8219777d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.540+0000 7f016f79a740 1 --2- 172.21.15.86:0/193486327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8219624d0 0x55a8219777d0 secure :-1 s=READY pgs=1278 cs=0 l=1 rev1=1 crypto rx=0x7f0150099500 tx=0x7f01500d0040 comp rx=0 tx=0).stop 2024-04-21T08:44:16.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.540+0000 7f016f79a740 1 -- 172.21.15.86:0/193486327 shutdown_connections 2024-04-21T08:44:16.545 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.540+0000 7f016f79a740 1 --2- 172.21.15.86:0/193486327 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a82196ba00 0x55a82195ba80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.540+0000 7f016f79a740 1 --2- 172.21.15.86:0/193486327 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a821962e80 0x55a821967860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.540+0000 7f016f79a740 1 --2- 172.21.15.86:0/193486327 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8219624d0 0x55a8219777d0 unknown :-1 s=CLOSED pgs=1278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.540+0000 7f016f79a740 1 -- 172.21.15.86:0/193486327 >> 172.21.15.86:0/193486327 conn(0x55a82195cba0 msgr2=0x55a82195b370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:16.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.540+0000 7f016f79a740 1 -- 172.21.15.86:0/193486327 shutdown_connections 2024-04-21T08:44:16.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.540+0000 7f016f79a740 1 -- 172.21.15.86:0/193486327 wait complete. 2024-04-21T08:44:16.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.540+0000 7f016f79a740 1 -- start start 2024-04-21T08:44:16.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.541+0000 7f016f79a740 1 Processor -- start 2024-04-21T08:44:16.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.541+0000 7f016f79a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8219624d0 0x55a82196f870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.541+0000 7f016f79a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a821962e80 0x55a82196fdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.541+0000 7f016f79a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a82196ba00 0x55a8219702f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.541+0000 7f016f79a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f01681b0860 con 0x55a8219624d0 2024-04-21T08:44:16.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.541+0000 7f016f79a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f01681b0190 con 0x55a82196ba00 2024-04-21T08:44:16.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.541+0000 7f016f79a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f01681ad3b0 con 0x55a821962e80 2024-04-21T08:44:16.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.541+0000 7f016f79a740 10 client.0 _ll_register_callbacks cb 0x7f0158009c50 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:16.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.541+0000 7f016659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8219624d0 0x55a82196f870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.541+0000 7f016659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8219624d0 0x55a82196f870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34900/0 (socket says 172.21.15.86:34900) 2024-04-21T08:44:16.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.541+0000 7f016659c640 1 -- 172.21.15.86:0/1819526687 learned_addr learned my addr 172.21.15.86:0/1819526687 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:16.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.541+0000 7f0165d9b640 1 --2- 172.21.15.86:0/1819526687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a821962e80 0x55a82196fdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.541+0000 7f0166d9d640 1 --2- 172.21.15.86:0/1819526687 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a82196ba00 0x55a8219702f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.541+0000 7f016659c640 1 -- 172.21.15.86:0/1819526687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a821962e80 msgr2=0x55a82196fdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.541+0000 7f016659c640 1 --2- 172.21.15.86:0/1819526687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a821962e80 0x55a82196fdb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.541+0000 7f016659c640 1 -- 172.21.15.86:0/1819526687 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a82196ba00 msgr2=0x55a8219702f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.541+0000 7f016659c640 1 --2- 172.21.15.86:0/1819526687 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a82196ba00 0x55a8219702f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.541+0000 7f016659c640 1 -- 172.21.15.86:0/1819526687 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a8219e1400 con 0x55a8219624d0 2024-04-21T08:44:16.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.541+0000 7f016659c640 1 --2- 172.21.15.86:0/1819526687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8219624d0 0x55a82196f870 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f015009cac0 tx=0x7f015009c900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59e2a9f6f87fe932 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:16.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.541+0000 7f014dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:16.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.541+0000 7f014dffb640 1 -- 172.21.15.86:0/1819526687 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f015009de60 con 0x55a8219624d0 2024-04-21T08:44:16.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.541+0000 7f014dffb640 1 -- 172.21.15.86:0/1819526687 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f0150010a90 con 0x55a8219624d0 2024-04-21T08:44:16.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.541+0000 7f016f79a740 1 -- 172.21.15.86:0/1819526687 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f01500a0ee0 con 0x55a8219624d0 2024-04-21T08:44:16.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.542+0000 7f014dffb640 1 -- 172.21.15.86:0/1819526687 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f01500a57a0 con 0x55a8219624d0 2024-04-21T08:44:16.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.542+0000 7f016f79a740 10 client.18012 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:16.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.542+0000 7f016f79a740 1 -- 172.21.15.86:0/1819526687 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a8219e2630 con 0x55a8219624d0 2024-04-21T08:44:16.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.542+0000 7f014dffb640 1 -- 172.21.15.86:0/1819526687 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f01500a0ee0 con 0x55a8219624d0 2024-04-21T08:44:16.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.542+0000 7f014dffb640 1 -- 172.21.15.86:0/1819526687 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0150018b20 con 0x55a8219624d0 2024-04-21T08:44:16.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.542+0000 7f016f79a740 10 client.18012 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:16.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.542+0000 7f016f79a740 10 client.18012 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:44:16.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.542+0000 7f016f79a740 20 client.18012 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:16.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.542+0000 7f016f79a740 1 --2- 172.21.15.86:0/1819526687 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55a8219e2cf0 0x55a8219e50e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.542+0000 7f016f79a740 4 client.18012 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:44:16.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.542+0000 7f016f79a740 1 -- 172.21.15.86:0/1819526687 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a8218d4c10 con 0x55a8219e2cf0 2024-04-21T08:44:16.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.542+0000 7f0165d9b640 1 --2- 172.21.15.86:0/1819526687 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55a8219e2cf0 0x55a8219e50e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.543+0000 7f0165d9b640 1 --2- 172.21.15.86:0/1819526687 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55a8219e2cf0 0x55a8219e50e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4efb4aed6bfc88c4 server_cookie=421b41080e292eb8 in_seq=0 out_seq=0 2024-04-21T08:44:16.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.543+0000 7f014dffb640 10 client.18012 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:44:16.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.543+0000 7f014dffb640 1 -- 172.21.15.86:0/1819526687 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a8218d4c10 con 0x55a8219e2cf0 2024-04-21T08:44:16.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.543+0000 7f014dffb640 10 client.18012 handle_command_reply: tid=0 2024-04-21T08:44:16.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.543+0000 7f016f79a740 1 client.18012 shutdown 2024-04-21T08:44:16.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.543+0000 7f016f79a740 1 -- 172.21.15.86:0/1819526687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8219624d0 msgr2=0x55a82196f870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.543+0000 7f016f79a740 1 --2- 172.21.15.86:0/1819526687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8219624d0 0x55a82196f870 secure :-1 s=READY pgs=1279 cs=0 l=1 rev1=1 crypto rx=0x7f015009cac0 tx=0x7f015009c900 comp rx=0 tx=0).stop 2024-04-21T08:44:16.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.543+0000 7f016f79a740 1 -- 172.21.15.86:0/1819526687 shutdown_connections 2024-04-21T08:44:16.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.543+0000 7f016f79a740 1 --2- 172.21.15.86:0/1819526687 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a82196ba00 0x55a8219702f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.543+0000 7f016f79a740 1 --2- 172.21.15.86:0/1819526687 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a821962e80 0x55a82196fdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.543+0000 7f016f79a740 1 --2- 172.21.15.86:0/1819526687 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55a8219e2cf0 0x55a8219e50e0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.543+0000 7f016f79a740 1 --2- 172.21.15.86:0/1819526687 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a8219624d0 0x55a82196f870 unknown :-1 s=CLOSED pgs=1279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.543+0000 7f014dffb640 0 client.18012 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:44:16.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.543+0000 7f016f79a740 1 -- 172.21.15.86:0/1819526687 >> 172.21.15.86:0/1819526687 conn(0x55a82195cba0 msgr2=0x55a82195e860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:16.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.543+0000 7f016f79a740 1 -- 172.21.15.86:0/1819526687 shutdown_connections 2024-04-21T08:44:16.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.543+0000 7f016f79a740 1 -- 172.21.15.86:0/1819526687 wait complete. 2024-04-21T08:44:16.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.543+0000 7f016f79a740 20 client.18012 trim_cache size 0 max 16384 2024-04-21T08:44:16.557 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:44:16.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.543+0000 7f016cac8640 1 -- 172.21.15.86:0/1163266636 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f01540a3d20 msgr2=0x7f01540a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.543+0000 7f016cac8640 1 --2- 172.21.15.86:0/1163266636 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f01540a3d20 0x7f01540a61e0 secure :-1 s=READY pgs=201 cs=0 l=1 rev1=1 crypto rx=0x7f0160009ca0 tx=0x7f0160042000 comp rx=0 tx=0).stop 2024-04-21T08:44:16.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.543+0000 7f016cac8640 1 -- 172.21.15.86:0/1163266636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0168102190 msgr2=0x7f0168198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.543+0000 7f016cac8640 1 --2- 172.21.15.86:0/1163266636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0168102190 0x7f0168198730 secure :-1 s=READY pgs=1275 cs=0 l=1 rev1=1 crypto rx=0x7f015000e9f0 tx=0x7f015000eeb0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.544+0000 7f016cac8640 1 -- 172.21.15.86:0/1163266636 shutdown_connections 2024-04-21T08:44:16.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.544+0000 7f016cac8640 1 --2- 172.21.15.86:0/1163266636 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f01540a3d20 0x7f01540a61e0 unknown :-1 s=CLOSED pgs=201 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.544+0000 7f016cac8640 1 --2- 172.21.15.86:0/1163266636 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f016819cff0 0x7f016819d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.544+0000 7f016cac8640 1 --2- 172.21.15.86:0/1163266636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0168107260 0x7f0168198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.544+0000 7f016cac8640 1 --2- 172.21.15.86:0/1163266636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0168102190 0x7f0168198730 unknown :-1 s=CLOSED pgs=1275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.544+0000 7f016cac8640 1 -- 172.21.15.86:0/1163266636 >> 172.21.15.86:0/1163266636 conn(0x7f01680793a0 msgr2=0x7f01680fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:16.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.544+0000 7f016cac8640 1 -- 172.21.15.86:0/1163266636 shutdown_connections 2024-04-21T08:44:16.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.544+0000 7f016cac8640 1 -- 172.21.15.86:0/1163266636 wait complete. 2024-04-21T08:44:16.559 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:16 smithi086 ceph-mon[27939]: pgmap v167: 129 pgs: 129 active+clean; 3.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-04-21T08:44:16.559 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:16 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/622025932' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:16.560 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:44:16.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.661+0000 7f8e2df6d640 1 -- 172.21.15.86:0/1669107726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e2810c510 msgr2=0x7f8e2810e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.661+0000 7f8e2df6d640 1 --2- 172.21.15.86:0/1669107726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e2810c510 0x7f8e2810e9b0 secure :-1 s=READY pgs=1280 cs=0 l=1 rev1=1 crypto rx=0x7f8e20009ca0 tx=0x7f8e2002c230 comp rx=0 tx=0).stop 2024-04-21T08:44:16.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.661+0000 7f8e2df6d640 1 -- 172.21.15.86:0/1669107726 shutdown_connections 2024-04-21T08:44:16.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.661+0000 7f8e2df6d640 1 --2- 172.21.15.86:0/1669107726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e2810c510 0x7f8e2810e9b0 unknown :-1 s=CLOSED pgs=1280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.661+0000 7f8e2df6d640 1 --2- 172.21.15.86:0/1669107726 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8e28103af0 0x7f8e2810bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.661+0000 7f8e2df6d640 1 --2- 172.21.15.86:0/1669107726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e281031d0 0x7f8e281035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.661+0000 7f8e2df6d640 1 -- 172.21.15.86:0/1669107726 >> 172.21.15.86:0/1669107726 conn(0x7f8e28077f50 msgr2=0x7f8e28078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:16.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.661+0000 7f8e2df6d640 1 -- 172.21.15.86:0/1669107726 shutdown_connections 2024-04-21T08:44:16.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.661+0000 7f8e2df6d640 1 -- 172.21.15.86:0/1669107726 wait complete. 2024-04-21T08:44:16.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.661+0000 7f8e2df6d640 1 Processor -- start 2024-04-21T08:44:16.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.661+0000 7f8e2df6d640 1 -- start start 2024-04-21T08:44:16.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.661+0000 7f8e2df6d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e281031d0 0x7f8e28198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.661+0000 7f8e2df6d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e28103af0 0x7f8e28198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.661+0000 7f8e2df6d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8e2810c510 0x7f8e2819cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.661+0000 7f8e2df6d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8e281121a0 con 0x7f8e281031d0 2024-04-21T08:44:16.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.661+0000 7f8e2df6d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8e28112020 con 0x7f8e2810c510 2024-04-21T08:44:16.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.661+0000 7f8e2df6d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8e28112320 con 0x7f8e28103af0 2024-04-21T08:44:16.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.661+0000 7f8e277fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e281031d0 0x7f8e28198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.662+0000 7f8e277fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e281031d0 0x7f8e28198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34920/0 (socket says 172.21.15.86:34920) 2024-04-21T08:44:16.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.662+0000 7f8e277fe640 1 -- 172.21.15.86:0/3576819927 learned_addr learned my addr 172.21.15.86:0/3576819927 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:16.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.662+0000 7f8e27fff640 1 --2- 172.21.15.86:0/3576819927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8e2810c510 0x7f8e2819cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.662+0000 7f8e1ffff640 1 --2- 172.21.15.86:0/3576819927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e28103af0 0x7f8e28198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.662+0000 7f8e277fe640 1 -- 172.21.15.86:0/3576819927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e28103af0 msgr2=0x7f8e28198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.662+0000 7f8e277fe640 1 --2- 172.21.15.86:0/3576819927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e28103af0 0x7f8e28198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.662+0000 7f8e277fe640 1 -- 172.21.15.86:0/3576819927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8e2810c510 msgr2=0x7f8e2819cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.662+0000 7f8e277fe640 1 --2- 172.21.15.86:0/3576819927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8e2810c510 0x7f8e2819cfc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.662+0000 7f8e277fe640 1 -- 172.21.15.86:0/3576819927 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e2819d6f0 con 0x7f8e281031d0 2024-04-21T08:44:16.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.662+0000 7f8e277fe640 1 --2- 172.21.15.86:0/3576819927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e281031d0 0x7f8e28198700 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f8e1800e9f0 tx=0x7f8e1800edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:16.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.662+0000 7f8e257fa640 1 -- 172.21.15.86:0/3576819927 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1800cd80 con 0x7f8e281031d0 2024-04-21T08:44:16.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.662+0000 7f8e257fa640 1 -- 172.21.15.86:0/3576819927 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e180054d0 con 0x7f8e281031d0 2024-04-21T08:44:16.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.662+0000 7f8e2df6d640 1 -- 172.21.15.86:0/3576819927 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e2819d9e0 con 0x7f8e281031d0 2024-04-21T08:44:16.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.662+0000 7f8e257fa640 1 -- 172.21.15.86:0/3576819927 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e18010690 con 0x7f8e281031d0 2024-04-21T08:44:16.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.662+0000 7f8e2df6d640 1 -- 172.21.15.86:0/3576819927 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e281a5270 con 0x7f8e281031d0 2024-04-21T08:44:16.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.665+0000 7f8e30c35740 1 Processor -- start 2024-04-21T08:44:16.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.665+0000 7f8e257fa640 1 -- 172.21.15.86:0/3576819927 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8e18004e90 con 0x7f8e281031d0 2024-04-21T08:44:16.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.665+0000 7f8e30c35740 1 -- start start 2024-04-21T08:44:16.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.665+0000 7f8e30c35740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba032cf4d0 0x55ba032cf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.665+0000 7f8e30c35740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba032cfe80 0x55ba032d8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.665+0000 7f8e30c35740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba032d8a00 0x55ba032dae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.665+0000 7f8e30c35740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ba032341f0 con 0x55ba032d8a00 2024-04-21T08:44:16.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.665+0000 7f8e30c35740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55ba031d14b0 con 0x55ba032cf4d0 2024-04-21T08:44:16.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.665+0000 7f8e30c35740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ba032301b0 con 0x55ba032cfe80 2024-04-21T08:44:16.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.665+0000 7f8e1ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba032d8a00 0x55ba032dae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.665+0000 7f8e1ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba032d8a00 0x55ba032dae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34924/0 (socket says 172.21.15.86:34924) 2024-04-21T08:44:16.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.665+0000 7f8e1ffff640 1 -- 172.21.15.86:0/4111152705 learned_addr learned my addr 172.21.15.86:0/4111152705 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:16.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.665+0000 7f8e27fff640 1 --2- 172.21.15.86:0/4111152705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba032cf4d0 0x55ba032cf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.666+0000 7f8e1ffff640 1 -- 172.21.15.86:0/4111152705 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba032cfe80 msgr2=0x55ba032d8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.666+0000 7f8e277fe640 1 --2- 172.21.15.86:0/4111152705 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba032cfe80 0x55ba032d8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.666+0000 7f8e1ffff640 1 --2- 172.21.15.86:0/4111152705 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba032cfe80 0x55ba032d8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.666+0000 7f8e1ffff640 1 -- 172.21.15.86:0/4111152705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba032cf4d0 msgr2=0x55ba032cf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.666+0000 7f8e1ffff640 1 --2- 172.21.15.86:0/4111152705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba032cf4d0 0x55ba032cf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.666+0000 7f8e257fa640 1 --2- 172.21.15.86:0/3576819927 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8e140a3d20 0x7f8e140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.666+0000 7f8e1ffff640 1 -- 172.21.15.86:0/4111152705 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba031c31b0 con 0x55ba032d8a00 2024-04-21T08:44:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.666+0000 7f8e277fe640 1 --2- 172.21.15.86:0/4111152705 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba032cfe80 0x55ba032d8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.666+0000 7f8e27fff640 1 --2- 172.21.15.86:0/4111152705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba032cf4d0 0x55ba032cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.666+0000 7f8e277fe640 1 --2- 172.21.15.86:0/3576819927 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8e140a3d20 0x7f8e140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.666+0000 7f8e1ffff640 1 --2- 172.21.15.86:0/4111152705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba032d8a00 0x55ba032dae50 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f8e10009c10 tx=0x7f8e10011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e10dbdd1163b016 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.666+0000 7f8e257fa640 1 -- 172.21.15.86:0/3576819927 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f8e180d0f60 con 0x7f8e281031d0 2024-04-21T08:44:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.666+0000 7f8e1dffb640 1 -- 172.21.15.86:0/4111152705 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e10004230 con 0x55ba032d8a00 2024-04-21T08:44:16.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.666+0000 7f8e1dffb640 1 -- 172.21.15.86:0/4111152705 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e100043d0 con 0x55ba032d8a00 2024-04-21T08:44:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.666+0000 7f8e1dffb640 1 -- 172.21.15.86:0/4111152705 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e10004d30 con 0x55ba032d8a00 2024-04-21T08:44:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.667+0000 7f8e30c35740 1 -- 172.21.15.86:0/4111152705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba032d8a00 msgr2=0x55ba032dae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.667+0000 7f8e277fe640 1 --2- 172.21.15.86:0/3576819927 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8e140a3d20 0x7f8e140a61e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x55ba032cefa0 tx=0x7f8e18004a90 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.667+0000 7f8e30c35740 1 --2- 172.21.15.86:0/4111152705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba032d8a00 0x55ba032dae50 secure :-1 s=READY pgs=1282 cs=0 l=1 rev1=1 crypto rx=0x7f8e10009c10 tx=0x7f8e10011810 comp rx=0 tx=0).stop 2024-04-21T08:44:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.667+0000 7f8e30c35740 1 -- 172.21.15.86:0/4111152705 shutdown_connections 2024-04-21T08:44:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.667+0000 7f8e30c35740 1 --2- 172.21.15.86:0/4111152705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba032d8a00 0x55ba032dae50 unknown :-1 s=CLOSED pgs=1282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.667+0000 7f8e30c35740 1 --2- 172.21.15.86:0/4111152705 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba032cfe80 0x55ba032d8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.667+0000 7f8e30c35740 1 --2- 172.21.15.86:0/4111152705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba032cf4d0 0x55ba032cf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.667+0000 7f8e30c35740 1 -- 172.21.15.86:0/4111152705 >> 172.21.15.86:0/4111152705 conn(0x55ba032c9ba0 msgr2=0x55ba032c9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.667+0000 7f8e30c35740 1 -- 172.21.15.86:0/4111152705 shutdown_connections 2024-04-21T08:44:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.667+0000 7f8e30c35740 1 -- 172.21.15.86:0/4111152705 wait complete. 2024-04-21T08:44:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.667+0000 7f8e30c35740 1 -- start start 2024-04-21T08:44:16.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.667+0000 7f8e30c35740 1 Processor -- start 2024-04-21T08:44:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.667+0000 7f8e30c35740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba032cf4d0 0x55ba032d8290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.667+0000 7f8e30c35740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba032cfe80 0x55ba032d5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.667+0000 7f8e30c35740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba032d8a00 0x55ba032d5b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.668+0000 7f8e30c35740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ba032db970 con 0x55ba032d8a00 2024-04-21T08:44:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.668+0000 7f8e30c35740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55ba031ed500 con 0x55ba032cf4d0 2024-04-21T08:44:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.668+0000 7f8e30c35740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ba032dbaf0 con 0x55ba032cfe80 2024-04-21T08:44:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.668+0000 7f8e30c35740 10 client.0 _ll_register_callbacks cb 0x55ba031bbb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.668+0000 7f8e27fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba032d8a00 0x55ba032d5b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.668+0000 7f8e27fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba032d8a00 0x55ba032d5b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34934/0 (socket says 172.21.15.86:34934) 2024-04-21T08:44:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.668+0000 7f8e27fff640 1 -- 172.21.15.86:0/2597735418 learned_addr learned my addr 172.21.15.86:0/2597735418 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.668+0000 7f8e27fff640 1 --2- 172.21.15.86:0/2597735418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba032cf4d0 0x55ba032d8290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.668+0000 7f8e1ffff640 1 --2- 172.21.15.86:0/2597735418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba032cfe80 0x55ba032d5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.668+0000 7f8e27fff640 1 -- 172.21.15.86:0/2597735418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba032cfe80 msgr2=0x55ba032d5620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.668+0000 7f8e27fff640 1 --2- 172.21.15.86:0/2597735418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba032cfe80 0x55ba032d5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.668+0000 7f8e27fff640 1 -- 172.21.15.86:0/2597735418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba032cf4d0 msgr2=0x55ba032d8290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.668+0000 7f8e27fff640 1 --2- 172.21.15.86:0/2597735418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba032cf4d0 0x55ba032d8290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.668+0000 7f8e27fff640 1 -- 172.21.15.86:0/2597735418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba02f463f0 con 0x55ba032d8a00 2024-04-21T08:44:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.668+0000 7f8e27fff640 1 --2- 172.21.15.86:0/2597735418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba032d8a00 0x55ba032d5b60 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f8e20037040 tx=0x7f8e20005c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fecdcc08f6f059a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.668+0000 7f8e06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.668+0000 7f8e06ffd640 1 -- 172.21.15.86:0/2597735418 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2004a060 con 0x55ba032d8a00 2024-04-21T08:44:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.668+0000 7f8e06ffd640 1 -- 172.21.15.86:0/2597735418 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f8e20004450 con 0x55ba032d8a00 2024-04-21T08:44:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.668+0000 7f8e30c35740 1 -- 172.21.15.86:0/2597735418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e100043d0 con 0x55ba032d8a00 2024-04-21T08:44:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.668+0000 7f8e06ffd640 1 -- 172.21.15.86:0/2597735418 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8e20051050 con 0x55ba032d8a00 2024-04-21T08:44:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.668+0000 7f8e30c35740 10 client.18024 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.668+0000 7f8e30c35740 1 -- 172.21.15.86:0/2597735418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ba030107d0 con 0x55ba032d8a00 2024-04-21T08:44:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.669+0000 7f8e06ffd640 1 -- 172.21.15.86:0/2597735418 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e100043d0 con 0x55ba032d8a00 2024-04-21T08:44:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.669+0000 7f8e06ffd640 1 -- 172.21.15.86:0/2597735418 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8e20034440 con 0x55ba032d8a00 2024-04-21T08:44:16.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.669+0000 7f8e30c35740 10 client.18024 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:16.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.669+0000 7f8e30c35740 10 client.18024 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:16.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.669+0000 7f8e30c35740 20 client.18024 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:16.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.669+0000 7f8e30c35740 1 --2- 172.21.15.86:0/2597735418 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ba032e8af0 0x55ba032eaf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.669+0000 7f8e30c35740 4 client.18024 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:16.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.669+0000 7f8e30c35740 1 -- 172.21.15.86:0/2597735418 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ba031d3930 con 0x55ba032e8af0 2024-04-21T08:44:16.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.669+0000 7f8e1ffff640 1 --2- 172.21.15.86:0/2597735418 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ba032e8af0 0x55ba032eaf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.670+0000 7f8e06ffd640 10 client.18024 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:16.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.670+0000 7f8e1ffff640 1 --2- 172.21.15.86:0/2597735418 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ba032e8af0 0x55ba032eaf40 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bc666c19deef5bf5 server_cookie=8bd9aa194ef85138 in_seq=0 out_seq=0 2024-04-21T08:44:16.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.670+0000 7f8e06ffd640 1 -- 172.21.15.86:0/2597735418 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ba031d3930 con 0x55ba032e8af0 2024-04-21T08:44:16.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.670+0000 7f8e06ffd640 10 client.18024 handle_command_reply: tid=0 2024-04-21T08:44:16.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.670+0000 7f8e30c35740 1 client.18024 shutdown 2024-04-21T08:44:16.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.670+0000 7f8e30c35740 1 -- 172.21.15.86:0/2597735418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba032d8a00 msgr2=0x55ba032d5b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.670+0000 7f8e30c35740 1 --2- 172.21.15.86:0/2597735418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba032d8a00 0x55ba032d5b60 secure :-1 s=READY pgs=1283 cs=0 l=1 rev1=1 crypto rx=0x7f8e20037040 tx=0x7f8e20005c50 comp rx=0 tx=0).stop 2024-04-21T08:44:16.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.670+0000 7f8e30c35740 1 -- 172.21.15.86:0/2597735418 shutdown_connections 2024-04-21T08:44:16.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.670+0000 7f8e30c35740 1 --2- 172.21.15.86:0/2597735418 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ba032e8af0 0x55ba032eaf40 crc :-1 s=READY pgs=181 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.670+0000 7f8e30c35740 1 --2- 172.21.15.86:0/2597735418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba032d8a00 0x55ba032d5b60 unknown :-1 s=CLOSED pgs=1283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.670+0000 7f8e30c35740 1 --2- 172.21.15.86:0/2597735418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba032cfe80 0x55ba032d5620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.670+0000 7f8e06ffd640 0 client.18024 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:16.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.670+0000 7f8e30c35740 1 --2- 172.21.15.86:0/2597735418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba032cf4d0 0x55ba032d8290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.670+0000 7f8e30c35740 1 -- 172.21.15.86:0/2597735418 >> 172.21.15.86:0/2597735418 conn(0x55ba032c9ba0 msgr2=0x55ba032cb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:16.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.670+0000 7f8e30c35740 1 -- 172.21.15.86:0/2597735418 shutdown_connections 2024-04-21T08:44:16.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.671+0000 7f8e30c35740 1 -- 172.21.15.86:0/2597735418 wait complete. 2024-04-21T08:44:16.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.671+0000 7f8e30c35740 20 client.18024 trim_cache size 0 max 16384 2024-04-21T08:44:16.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.684+0000 7f8e30c35740 1 Processor -- start 2024-04-21T08:44:16.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.684+0000 7f8e30c35740 1 -- start start 2024-04-21T08:44:16.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.684+0000 7f8e30c35740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba032cf4d0 0x55ba032e4830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.684+0000 7f8e30c35740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba032cfe80 0x55ba032cb6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.684+0000 7f8e30c35740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba032d8a00 0x55ba032d7d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.684+0000 7f8e30c35740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8e281af320 con 0x55ba032d8a00 2024-04-21T08:44:16.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.684+0000 7f8e30c35740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8e281b0e60 con 0x55ba032cfe80 2024-04-21T08:44:16.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.684+0000 7f8e30c35740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8e281af9f0 con 0x55ba032cf4d0 2024-04-21T08:44:16.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.684+0000 7f8e27fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba032d8a00 0x55ba032d7d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.684+0000 7f8e27fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba032d8a00 0x55ba032d7d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34950/0 (socket says 172.21.15.86:34950) 2024-04-21T08:44:16.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.684+0000 7f8e27fff640 1 -- 172.21.15.86:0/871135682 learned_addr learned my addr 172.21.15.86:0/871135682 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:16.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.684+0000 7f8e27fff640 1 --2- 172.21.15.86:0/871135682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba032cf4d0 0x55ba032e4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.684+0000 7f8e27fff640 1 -- 172.21.15.86:0/871135682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba032cf4d0 msgr2=0x55ba032e4830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.684+0000 7f8e27fff640 1 --2- 172.21.15.86:0/871135682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba032cf4d0 0x55ba032e4830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.684+0000 7f8e27fff640 1 -- 172.21.15.86:0/871135682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba032cfe80 msgr2=0x55ba032cb6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.684+0000 7f8e27fff640 1 --2- 172.21.15.86:0/871135682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba032cfe80 0x55ba032cb6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.684+0000 7f8e27fff640 1 -- 172.21.15.86:0/871135682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ba032cf090 con 0x55ba032d8a00 2024-04-21T08:44:16.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.684+0000 7f8e27fff640 1 --2- 172.21.15.86:0/871135682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba032d8a00 0x55ba032d7d50 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f8e20002230 tx=0x7f8e2002c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c6109f74c26a10d4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:16.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.684+0000 7f8e1d7fa640 1 -- 172.21.15.86:0/871135682 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e2005a070 con 0x55ba032d8a00 2024-04-21T08:44:16.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.684+0000 7f8e1d7fa640 1 -- 172.21.15.86:0/871135682 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e20004700 con 0x55ba032d8a00 2024-04-21T08:44:16.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.684+0000 7f8e1d7fa640 1 -- 172.21.15.86:0/871135682 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e20034e60 con 0x55ba032d8a00 2024-04-21T08:44:16.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.685+0000 7f8e30c35740 1 -- 172.21.15.86:0/871135682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba032d8a00 msgr2=0x55ba032d7d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.685+0000 7f8e30c35740 1 --2- 172.21.15.86:0/871135682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba032d8a00 0x55ba032d7d50 secure :-1 s=READY pgs=1284 cs=0 l=1 rev1=1 crypto rx=0x7f8e20002230 tx=0x7f8e2002c230 comp rx=0 tx=0).stop 2024-04-21T08:44:16.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.685+0000 7f8e30c35740 1 -- 172.21.15.86:0/871135682 shutdown_connections 2024-04-21T08:44:16.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.685+0000 7f8e30c35740 1 --2- 172.21.15.86:0/871135682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba032d8a00 0x55ba032d7d50 unknown :-1 s=CLOSED pgs=1284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.685+0000 7f8e30c35740 1 --2- 172.21.15.86:0/871135682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba032cfe80 0x55ba032cb6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.685+0000 7f8e30c35740 1 --2- 172.21.15.86:0/871135682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba032cf4d0 0x55ba032e4830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.685+0000 7f8e30c35740 1 -- 172.21.15.86:0/871135682 >> 172.21.15.86:0/871135682 conn(0x55ba032c9ba0 msgr2=0x55ba032cafd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:16.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.685+0000 7f8e30c35740 1 -- 172.21.15.86:0/871135682 shutdown_connections 2024-04-21T08:44:16.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.685+0000 7f8e30c35740 1 -- 172.21.15.86:0/871135682 wait complete. 2024-04-21T08:44:16.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.685+0000 7f8e30c35740 1 -- start start 2024-04-21T08:44:16.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.686+0000 7f8e30c35740 1 Processor -- start 2024-04-21T08:44:16.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.686+0000 7f8e30c35740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba032cf4d0 0x55ba032e1130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.686+0000 7f8e30c35740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba032cfe80 0x55ba032dc4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.686+0000 7f8e30c35740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba032d8a00 0x55ba032dc9e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.686+0000 7f8e30c35740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8e281b0790 con 0x55ba032cfe80 2024-04-21T08:44:16.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.686+0000 7f8e30c35740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8e281b00c0 con 0x55ba032cf4d0 2024-04-21T08:44:16.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.686+0000 7f8e30c35740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8e281ad2e0 con 0x55ba032d8a00 2024-04-21T08:44:16.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.686+0000 7f8e30c35740 10 client.0 _ll_register_callbacks cb 0x7f8e20009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:16.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.686+0000 7f8e1ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba032cfe80 0x55ba032dc4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.686+0000 7f8e1ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba032cfe80 0x55ba032dc4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34952/0 (socket says 172.21.15.86:34952) 2024-04-21T08:44:16.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.686+0000 7f8e1ffff640 1 -- 172.21.15.86:0/1970519584 learned_addr learned my addr 172.21.15.86:0/1970519584 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:16.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.686+0000 7f8e27fff640 1 --2- 172.21.15.86:0/1970519584 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba032d8a00 0x55ba032dc9e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.686+0000 7f8e1ffff640 1 -- 172.21.15.86:0/1970519584 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba032d8a00 msgr2=0x55ba032dc9e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.686+0000 7f8e1ffff640 1 --2- 172.21.15.86:0/1970519584 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba032d8a00 0x55ba032dc9e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.686+0000 7f8e1ffff640 1 -- 172.21.15.86:0/1970519584 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba032cf4d0 msgr2=0x55ba032e1130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:16.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.686+0000 7f8e27fff640 1 --2- 172.21.15.86:0/1970519584 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba032cf4d0 0x55ba032e1130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.686+0000 7f8e1ffff640 1 --2- 172.21.15.86:0/1970519584 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba032cf4d0 0x55ba032e1130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.686+0000 7f8e1ffff640 1 -- 172.21.15.86:0/1970519584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ba032dd1e0 con 0x55ba032cfe80 2024-04-21T08:44:16.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.686+0000 7f8e27fff640 1 --2- 172.21.15.86:0/1970519584 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba032cf4d0 0x55ba032e1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:16.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.686+0000 7f8e1ffff640 1 --2- 172.21.15.86:0/1970519584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba032cfe80 0x55ba032dc4a0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f8e100098f0 tx=0x7f8e10005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e20a81e1b783a9ae server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:16.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.686+0000 7f8e06ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:16.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.686+0000 7f8e06ffd640 1 -- 172.21.15.86:0/1970519584 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e1002a1d0 con 0x55ba032cfe80 2024-04-21T08:44:16.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.686+0000 7f8e06ffd640 1 -- 172.21.15.86:0/1970519584 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f8e1001e040 con 0x55ba032cfe80 2024-04-21T08:44:16.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.686+0000 7f8e30c35740 1 -- 172.21.15.86:0/1970519584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e20004700 con 0x55ba032cfe80 2024-04-21T08:44:16.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.687+0000 7f8e06ffd640 1 -- 172.21.15.86:0/1970519584 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8e10037050 con 0x55ba032cfe80 2024-04-21T08:44:16.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.687+0000 7f8e30c35740 10 client.18030 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:16.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.687+0000 7f8e30c35740 1 -- 172.21.15.86:0/1970519584 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ba033534c0 con 0x55ba032cfe80 2024-04-21T08:44:16.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.687+0000 7f8e06ffd640 1 -- 172.21.15.86:0/1970519584 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e20004700 con 0x55ba032cfe80 2024-04-21T08:44:16.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.687+0000 7f8e06ffd640 1 -- 172.21.15.86:0/1970519584 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8e10018070 con 0x55ba032cfe80 2024-04-21T08:44:16.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.687+0000 7f8e30c35740 10 client.18030 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:16.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.687+0000 7f8e30c35740 10 client.18030 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:16.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.687+0000 7f8e30c35740 20 client.18030 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:16.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.687+0000 7f8e30c35740 1 --2- 172.21.15.86:0/1970519584 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ba03353a80 0x55ba03355ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:16.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.687+0000 7f8e30c35740 4 client.18030 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:44:16.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.687+0000 7f8e30c35740 1 -- 172.21.15.86:0/1970519584 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ba03241c10 con 0x55ba03353a80 2024-04-21T08:44:16.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.687+0000 7f8e27fff640 1 --2- 172.21.15.86:0/1970519584 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ba03353a80 0x55ba03355ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:16.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.688+0000 7f8e06ffd640 10 client.18030 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:16.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.687+0000 7f8e27fff640 1 --2- 172.21.15.86:0/1970519584 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ba03353a80 0x55ba03355ea0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec381936fd223356 server_cookie=3dd7c5c7ca95468c in_seq=0 out_seq=0 2024-04-21T08:44:16.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.688+0000 7f8e06ffd640 1 -- 172.21.15.86:0/1970519584 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ba03241c10 con 0x55ba03353a80 2024-04-21T08:44:16.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.688+0000 7f8e06ffd640 10 client.18030 handle_command_reply: tid=0 2024-04-21T08:44:16.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.688+0000 7f8e30c35740 1 client.18030 shutdown 2024-04-21T08:44:16.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.688+0000 7f8e30c35740 1 -- 172.21.15.86:0/1970519584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba032cfe80 msgr2=0x55ba032dc4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.688+0000 7f8e30c35740 1 --2- 172.21.15.86:0/1970519584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba032cfe80 0x55ba032dc4a0 secure :-1 s=READY pgs=1285 cs=0 l=1 rev1=1 crypto rx=0x7f8e100098f0 tx=0x7f8e10005da0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.688+0000 7f8e30c35740 1 -- 172.21.15.86:0/1970519584 shutdown_connections 2024-04-21T08:44:16.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.688+0000 7f8e30c35740 1 --2- 172.21.15.86:0/1970519584 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ba03353a80 0x55ba03355ea0 crc :-1 s=READY pgs=182 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.688+0000 7f8e30c35740 1 --2- 172.21.15.86:0/1970519584 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ba032d8a00 0x55ba032dc9e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.688+0000 7f8e06ffd640 0 client.18030 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:16.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.688+0000 7f8e30c35740 1 --2- 172.21.15.86:0/1970519584 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ba032cfe80 0x55ba032dc4a0 unknown :-1 s=CLOSED pgs=1285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.688+0000 7f8e30c35740 1 --2- 172.21.15.86:0/1970519584 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ba032cf4d0 0x55ba032e1130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.688+0000 7f8e30c35740 1 -- 172.21.15.86:0/1970519584 >> 172.21.15.86:0/1970519584 conn(0x55ba032c9ba0 msgr2=0x55ba032d0af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:16.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.688+0000 7f8e30c35740 1 -- 172.21.15.86:0/1970519584 shutdown_connections 2024-04-21T08:44:16.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.688+0000 7f8e30c35740 1 -- 172.21.15.86:0/1970519584 wait complete. 2024-04-21T08:44:16.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.688+0000 7f8e30c35740 20 client.18030 trim_cache size 0 max 16384 2024-04-21T08:44:16.695 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:44:16.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.688+0000 7f8e2df6d640 1 -- 172.21.15.86:0/3576819927 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8e140a3d20 msgr2=0x7f8e140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.688+0000 7f8e2df6d640 1 --2- 172.21.15.86:0/3576819927 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8e140a3d20 0x7f8e140a61e0 secure :-1 s=READY pgs=202 cs=0 l=1 rev1=1 crypto rx=0x55ba032cefa0 tx=0x7f8e18004a90 comp rx=0 tx=0).stop 2024-04-21T08:44:16.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.688+0000 7f8e2df6d640 1 -- 172.21.15.86:0/3576819927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e281031d0 msgr2=0x7f8e28198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:16.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.688+0000 7f8e2df6d640 1 --2- 172.21.15.86:0/3576819927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e281031d0 0x7f8e28198700 secure :-1 s=READY pgs=1281 cs=0 l=1 rev1=1 crypto rx=0x7f8e1800e9f0 tx=0x7f8e1800edf0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.688+0000 7f8e2df6d640 1 -- 172.21.15.86:0/3576819927 shutdown_connections 2024-04-21T08:44:16.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.688+0000 7f8e2df6d640 1 --2- 172.21.15.86:0/3576819927 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8e140a3d20 0x7f8e140a61e0 unknown :-1 s=CLOSED pgs=202 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.689+0000 7f8e2df6d640 1 --2- 172.21.15.86:0/3576819927 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8e2810c510 0x7f8e2819cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.689+0000 7f8e2df6d640 1 --2- 172.21.15.86:0/3576819927 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e28103af0 0x7f8e28198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.689+0000 7f8e2df6d640 1 --2- 172.21.15.86:0/3576819927 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e281031d0 0x7f8e28198700 unknown :-1 s=CLOSED pgs=1281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:16.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.689+0000 7f8e2df6d640 1 -- 172.21.15.86:0/3576819927 >> 172.21.15.86:0/3576819927 conn(0x7f8e28077f50 msgr2=0x7f8e280fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:16.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.689+0000 7f8e2df6d640 1 -- 172.21.15.86:0/3576819927 shutdown_connections 2024-04-21T08:44:16.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:16.689+0000 7f8e2df6d640 1 -- 172.21.15.86:0/3576819927 wait complete. 2024-04-21T08:44:16.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:16 smithi088 ceph-mon[31259]: pgmap v167: 129 pgs: 129 active+clean; 3.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-04-21T08:44:16.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:16 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/622025932' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:16.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:16 smithi179 ceph-mon[31829]: pgmap v167: 129 pgs: 129 active+clean; 3.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-04-21T08:44:16.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:16 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/622025932' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:17.699 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:44:17.699 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:44:17.699 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:44:17.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.814+0000 7f1777d96640 1 -- 172.21.15.86:0/3956207183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1770102190 msgr2=0x7f1770102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:17.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.814+0000 7f1777d96640 1 --2- 172.21.15.86:0/3956207183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1770102190 0x7f1770102570 secure :-1 s=READY pgs=1286 cs=0 l=1 rev1=1 crypto rx=0x7f176c009900 tx=0x7f176c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:17.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.814+0000 7f1777d96640 1 -- 172.21.15.86:0/3956207183 shutdown_connections 2024-04-21T08:44:17.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.814+0000 7f1777d96640 1 --2- 172.21.15.86:0/3956207183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1770107260 0x7f177010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.814+0000 7f1777d96640 1 --2- 172.21.15.86:0/3956207183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1770102b40 0x7f1770106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.814+0000 7f1777d96640 1 --2- 172.21.15.86:0/3956207183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1770102190 0x7f1770102570 unknown :-1 s=CLOSED pgs=1286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.814+0000 7f1777d96640 1 -- 172.21.15.86:0/3956207183 >> 172.21.15.86:0/3956207183 conn(0x7f17700793a0 msgr2=0x7f17700797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:17.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.815+0000 7f1777d96640 1 -- 172.21.15.86:0/3956207183 shutdown_connections 2024-04-21T08:44:17.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.815+0000 7f1777d96640 1 -- 172.21.15.86:0/3956207183 wait complete. 2024-04-21T08:44:17.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.815+0000 7f1777d96640 1 Processor -- start 2024-04-21T08:44:17.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.815+0000 7f1777d96640 1 -- start start 2024-04-21T08:44:17.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.815+0000 7f1777d96640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1770102190 0x7f1770074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.815+0000 7f1777d96640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1770107260 0x7f1770074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.815+0000 7f1777d96640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1770074cd0 0x7f17701a68f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.815+0000 7f1777d96640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f17701121c0 con 0x7f1770074cd0 2024-04-21T08:44:17.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.815+0000 7f1777d96640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1770112040 con 0x7f1770102190 2024-04-21T08:44:17.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.815+0000 7f1777d96640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1770112340 con 0x7f1770107260 2024-04-21T08:44:17.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.815+0000 7f1776333640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1770074cd0 0x7f17701a68f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.815+0000 7f1776333640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1770074cd0 0x7f17701a68f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34974/0 (socket says 172.21.15.86:34974) 2024-04-21T08:44:17.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.815+0000 7f1776333640 1 -- 172.21.15.86:0/244288975 learned_addr learned my addr 172.21.15.86:0/244288975 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:17.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.816+0000 7f1775331640 1 --2- 172.21.15.86:0/244288975 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1770107260 0x7f1770074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.816+0000 7f1775b32640 1 --2- 172.21.15.86:0/244288975 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1770102190 0x7f1770074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.816+0000 7f1776333640 1 -- 172.21.15.86:0/244288975 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1770107260 msgr2=0x7f1770074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:17.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.816+0000 7f1776333640 1 --2- 172.21.15.86:0/244288975 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1770107260 0x7f1770074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.816+0000 7f1776333640 1 -- 172.21.15.86:0/244288975 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1770102190 msgr2=0x7f1770074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:17.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.816+0000 7f1776333640 1 --2- 172.21.15.86:0/244288975 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1770102190 0x7f1770074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.816+0000 7f1776333640 1 -- 172.21.15.86:0/244288975 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f17701a6ff0 con 0x7f1770074cd0 2024-04-21T08:44:17.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.816+0000 7f1776333640 1 --2- 172.21.15.86:0/244288975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1770074cd0 0x7f17701a68f0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f176800cce0 tx=0x7f176800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:17.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.816+0000 7f1766ffd640 1 -- 172.21.15.86:0/244288975 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176800f450 con 0x7f1770074cd0 2024-04-21T08:44:17.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.816+0000 7f1777d96640 1 -- 172.21.15.86:0/244288975 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f17701a72e0 con 0x7f1770074cd0 2024-04-21T08:44:17.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.816+0000 7f1766ffd640 1 -- 172.21.15.86:0/244288975 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f176800faa0 con 0x7f1770074cd0 2024-04-21T08:44:17.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.816+0000 7f1766ffd640 1 -- 172.21.15.86:0/244288975 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1768018b70 con 0x7f1770074cd0 2024-04-21T08:44:17.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.816+0000 7f1777d96640 1 -- 172.21.15.86:0/244288975 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f17701a7830 con 0x7f1770074cd0 2024-04-21T08:44:17.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.819+0000 7f1766ffd640 1 -- 172.21.15.86:0/244288975 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1768018d10 con 0x7f1770074cd0 2024-04-21T08:44:17.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.819+0000 7f177a985740 1 Processor -- start 2024-04-21T08:44:17.825 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.819+0000 7f177a985740 1 -- start start 2024-04-21T08:44:17.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.819+0000 7f177a985740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dcfe668e0 0x559dcfe66cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.819+0000 7f177a985740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcfe67290 0x559dcfe6f840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.819+0000 7f177a985740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559dcfe6fe10 0x559dcfe72260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.819+0000 7f177a985740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559dcfdcaf00 con 0x559dcfe67290 2024-04-21T08:44:17.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.819+0000 7f177a985740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559dcfd68590 con 0x559dcfe6fe10 2024-04-21T08:44:17.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.819+0000 7f177a985740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559dcfdc6ec0 con 0x559dcfe668e0 2024-04-21T08:44:17.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.819+0000 7f1766ffd640 1 --2- 172.21.15.86:0/244288975 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f175c0a3d50 0x7f175c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.819+0000 7f1775b32640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcfe67290 0x559dcfe6f840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.819+0000 7f1775b32640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcfe67290 0x559dcfe6f840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34976/0 (socket says 172.21.15.86:34976) 2024-04-21T08:44:17.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.819+0000 7f1775b32640 1 -- 172.21.15.86:0/1896906237 learned_addr learned my addr 172.21.15.86:0/1896906237 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:17.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.819+0000 7f1775331640 1 --2- 172.21.15.86:0/1896906237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dcfe668e0 0x559dcfe66cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.819+0000 7f1775331640 1 --2- 172.21.15.86:0/1896906237 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559dcfe6fe10 0x559dcfe72260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.820+0000 7f1776333640 1 --2- 172.21.15.86:0/244288975 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f175c0a3d50 0x7f175c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.820+0000 7f1766ffd640 1 -- 172.21.15.86:0/244288975 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f17680d0e60 con 0x7f1770074cd0 2024-04-21T08:44:17.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.820+0000 7f1775b32640 1 -- 172.21.15.86:0/1896906237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dcfe668e0 msgr2=0x559dcfe66cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:17.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.820+0000 7f1775b32640 1 --2- 172.21.15.86:0/1896906237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dcfe668e0 0x559dcfe66cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.820+0000 7f1775b32640 1 -- 172.21.15.86:0/1896906237 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559dcfe6fe10 msgr2=0x559dcfe72260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:17.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.820+0000 7f1775b32640 1 --2- 172.21.15.86:0/1896906237 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559dcfe6fe10 0x559dcfe72260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.820+0000 7f1775b32640 1 -- 172.21.15.86:0/1896906237 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dcfba57e0 con 0x559dcfe67290 2024-04-21T08:44:17.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.820+0000 7f1775331640 1 --2- 172.21.15.86:0/1896906237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dcfe668e0 0x559dcfe66cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:44:17.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.820+0000 7f1775b32640 1 --2- 172.21.15.86:0/1896906237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcfe67290 0x559dcfe6f840 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f176c009900 tx=0x7f176c004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6156b1e8001ad127 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:17.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.820+0000 7f1776333640 1 --2- 172.21.15.86:0/244288975 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f175c0a3d50 0x7f175c0a6210 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f176809c790 tx=0x7f176806e4d0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:17.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.820+0000 7f175bfff640 1 -- 172.21.15.86:0/1896906237 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c045020 con 0x559dcfe67290 2024-04-21T08:44:17.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.820+0000 7f175bfff640 1 -- 172.21.15.86:0/1896906237 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f176c02cc90 con 0x559dcfe67290 2024-04-21T08:44:17.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.820+0000 7f175bfff640 1 -- 172.21.15.86:0/1896906237 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c0046c0 con 0x559dcfe67290 2024-04-21T08:44:17.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.821+0000 7f177a985740 1 -- 172.21.15.86:0/1896906237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcfe67290 msgr2=0x559dcfe6f840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:17.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.821+0000 7f177a985740 1 --2- 172.21.15.86:0/1896906237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcfe67290 0x559dcfe6f840 secure :-1 s=READY pgs=1288 cs=0 l=1 rev1=1 crypto rx=0x7f176c009900 tx=0x7f176c004040 comp rx=0 tx=0).stop 2024-04-21T08:44:17.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.821+0000 7f177a985740 1 -- 172.21.15.86:0/1896906237 shutdown_connections 2024-04-21T08:44:17.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.821+0000 7f177a985740 1 --2- 172.21.15.86:0/1896906237 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559dcfe6fe10 0x559dcfe72260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.821+0000 7f177a985740 1 --2- 172.21.15.86:0/1896906237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcfe67290 0x559dcfe6f840 unknown :-1 s=CLOSED pgs=1288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.821+0000 7f177a985740 1 --2- 172.21.15.86:0/1896906237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dcfe668e0 0x559dcfe66cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.821+0000 7f177a985740 1 -- 172.21.15.86:0/1896906237 >> 172.21.15.86:0/1896906237 conn(0x559dcfe60fb0 msgr2=0x559dcfe61390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:17.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.821+0000 7f177a985740 1 -- 172.21.15.86:0/1896906237 shutdown_connections 2024-04-21T08:44:17.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.821+0000 7f177a985740 1 -- 172.21.15.86:0/1896906237 wait complete. 2024-04-21T08:44:17.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.821+0000 7f177a985740 1 -- start start 2024-04-21T08:44:17.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.821+0000 7f177a985740 1 Processor -- start 2024-04-21T08:44:17.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.822+0000 7f177a985740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcfe668e0 0x559dcfe6ce40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.822+0000 7f177a985740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dcfe67290 0x559dcfe6d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.822+0000 7f177a985740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559dcfe6fe10 0x559dcfe6d8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.822+0000 7f177a985740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559dcfe72d20 con 0x559dcfe668e0 2024-04-21T08:44:17.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.822+0000 7f177a985740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559dcfd84580 con 0x559dcfe6fe10 2024-04-21T08:44:17.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.822+0000 7f177a985740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559dcfe72ea0 con 0x559dcfe67290 2024-04-21T08:44:17.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.822+0000 7f177a985740 10 client.0 _ll_register_callbacks cb 0x559dcfd8a650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:17.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.822+0000 7f1775331640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcfe668e0 0x559dcfe6ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.822+0000 7f1775331640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcfe668e0 0x559dcfe6ce40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34988/0 (socket says 172.21.15.86:34988) 2024-04-21T08:44:17.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.822+0000 7f1775331640 1 -- 172.21.15.86:0/103236472 learned_addr learned my addr 172.21.15.86:0/103236472 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:17.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.822+0000 7f1775331640 1 --2- 172.21.15.86:0/103236472 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559dcfe6fe10 0x559dcfe6d8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.822+0000 7f1775b32640 1 --2- 172.21.15.86:0/103236472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dcfe67290 0x559dcfe6d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.822+0000 7f1775331640 1 -- 172.21.15.86:0/103236472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dcfe67290 msgr2=0x559dcfe6d380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:17.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.822+0000 7f1775331640 1 --2- 172.21.15.86:0/103236472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dcfe67290 0x559dcfe6d380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.822+0000 7f1775331640 1 -- 172.21.15.86:0/103236472 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559dcfe6fe10 msgr2=0x559dcfe6d8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:17.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.822+0000 7f1775331640 1 --2- 172.21.15.86:0/103236472 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559dcfe6fe10 0x559dcfe6d8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.822+0000 7f1775331640 1 -- 172.21.15.86:0/103236472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dcfc05f70 con 0x559dcfe668e0 2024-04-21T08:44:17.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.822+0000 7f1775b32640 1 --2- 172.21.15.86:0/103236472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dcfe67290 0x559dcfe6d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:17.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.822+0000 7f1775331640 1 --2- 172.21.15.86:0/103236472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcfe668e0 0x559dcfe6ce40 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f1760004450 tx=0x7f17600046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a0f0f2b451daf64e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:17.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.822+0000 7f17597fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:17.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.823+0000 7f17597fa640 1 -- 172.21.15.86:0/103236472 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1760005220 con 0x559dcfe668e0 2024-04-21T08:44:17.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.823+0000 7f17597fa640 1 -- 172.21.15.86:0/103236472 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f17600053c0 con 0x559dcfe668e0 2024-04-21T08:44:17.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.823+0000 7f177a985740 1 -- 172.21.15.86:0/103236472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f176c02cc90 con 0x559dcfe668e0 2024-04-21T08:44:17.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.823+0000 7f17597fa640 1 -- 172.21.15.86:0/103236472 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f17600055c0 con 0x559dcfe668e0 2024-04-21T08:44:17.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.823+0000 7f177a985740 10 client.18042 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:17.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.823+0000 7f177a985740 1 -- 172.21.15.86:0/103236472 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559dcfa9d170 con 0x559dcfe668e0 2024-04-21T08:44:17.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.823+0000 7f17597fa640 1 -- 172.21.15.86:0/103236472 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f176c02cc90 con 0x559dcfe668e0 2024-04-21T08:44:17.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.823+0000 7f17597fa640 1 -- 172.21.15.86:0/103236472 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f176001e030 con 0x559dcfe668e0 2024-04-21T08:44:17.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.823+0000 7f177a985740 10 client.18042 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:17.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.823+0000 7f177a985740 10 client.18042 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:17.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.823+0000 7f177a985740 20 client.18042 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:17.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.823+0000 7f177a985740 1 --2- 172.21.15.86:0/103236472 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559dcfe80330 0x559dcfe82780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.823+0000 7f177a985740 4 client.18042 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:17.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.823+0000 7f177a985740 1 -- 172.21.15.86:0/103236472 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559dcfdd8920 con 0x559dcfe80330 2024-04-21T08:44:17.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.823+0000 7f1775b32640 1 --2- 172.21.15.86:0/103236472 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559dcfe80330 0x559dcfe82780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.824+0000 7f17597fa640 10 client.18042 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:17.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.824+0000 7f1775b32640 1 --2- 172.21.15.86:0/103236472 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559dcfe80330 0x559dcfe82780 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b2f357d1404a5bcf server_cookie=163345b6a77f2453 in_seq=0 out_seq=0 2024-04-21T08:44:17.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.825+0000 7f17597fa640 1 -- 172.21.15.86:0/103236472 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559dcfdd8920 con 0x559dcfe80330 2024-04-21T08:44:17.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.825+0000 7f17597fa640 10 client.18042 handle_command_reply: tid=0 2024-04-21T08:44:17.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.825+0000 7f177a985740 1 client.18042 shutdown 2024-04-21T08:44:17.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.825+0000 7f177a985740 1 -- 172.21.15.86:0/103236472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcfe668e0 msgr2=0x559dcfe6ce40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:17.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.825+0000 7f177a985740 1 --2- 172.21.15.86:0/103236472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcfe668e0 0x559dcfe6ce40 secure :-1 s=READY pgs=1289 cs=0 l=1 rev1=1 crypto rx=0x7f1760004450 tx=0x7f17600046c0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.825+0000 7f177a985740 1 -- 172.21.15.86:0/103236472 shutdown_connections 2024-04-21T08:44:17.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.825+0000 7f177a985740 1 --2- 172.21.15.86:0/103236472 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559dcfe80330 0x559dcfe82780 crc :-1 s=READY pgs=183 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.825+0000 7f177a985740 1 --2- 172.21.15.86:0/103236472 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559dcfe6fe10 0x559dcfe6d8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.825+0000 7f177a985740 1 --2- 172.21.15.86:0/103236472 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dcfe67290 0x559dcfe6d380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.825+0000 7f17597fa640 0 client.18042 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:17.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.825+0000 7f177a985740 1 --2- 172.21.15.86:0/103236472 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcfe668e0 0x559dcfe6ce40 unknown :-1 s=CLOSED pgs=1289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.825+0000 7f177a985740 1 -- 172.21.15.86:0/103236472 >> 172.21.15.86:0/103236472 conn(0x559dcfe60fb0 msgr2=0x559dcfe62d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:17.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.825+0000 7f177a985740 1 -- 172.21.15.86:0/103236472 shutdown_connections 2024-04-21T08:44:17.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.825+0000 7f177a985740 1 -- 172.21.15.86:0/103236472 wait complete. 2024-04-21T08:44:17.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.825+0000 7f177a985740 20 client.18042 trim_cache size 0 max 16384 2024-04-21T08:44:17.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.842+0000 7f177a985740 1 Processor -- start 2024-04-21T08:44:17.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.842+0000 7f177a985740 1 -- start start 2024-04-21T08:44:17.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.842+0000 7f177a985740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559dcfe668e0 0x559dcfe7bbe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.842+0000 7f177a985740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dcfe67290 0x559dcfe6c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.842+0000 7f177a985740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcfe6fe10 0x559dcfe5fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.842+0000 7f177a985740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f17701b1420 con 0x559dcfe6fe10 2024-04-21T08:44:17.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.842+0000 7f177a985740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f17701b2f60 con 0x559dcfe668e0 2024-04-21T08:44:17.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.842+0000 7f177a985740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f17701b1af0 con 0x559dcfe67290 2024-04-21T08:44:17.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.842+0000 7f1775331640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcfe6fe10 0x559dcfe5fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.843+0000 7f1775331640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcfe6fe10 0x559dcfe5fe90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35004/0 (socket says 172.21.15.86:35004) 2024-04-21T08:44:17.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.843+0000 7f1775331640 1 -- 172.21.15.86:0/1953806220 learned_addr learned my addr 172.21.15.86:0/1953806220 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:17.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.843+0000 7f1775b32640 1 --2- 172.21.15.86:0/1953806220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dcfe67290 0x559dcfe6c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.843+0000 7f1775331640 1 --2- 172.21.15.86:0/1953806220 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559dcfe668e0 0x559dcfe7bbe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.843+0000 7f1775331640 1 -- 172.21.15.86:0/1953806220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dcfe67290 msgr2=0x559dcfe6c5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:17.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.843+0000 7f1775331640 1 --2- 172.21.15.86:0/1953806220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dcfe67290 0x559dcfe6c5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.843+0000 7f1775331640 1 -- 172.21.15.86:0/1953806220 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559dcfe668e0 msgr2=0x559dcfe7bbe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:17.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.843+0000 7f1775331640 1 --2- 172.21.15.86:0/1953806220 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559dcfe668e0 0x559dcfe7bbe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.843+0000 7f1775331640 1 -- 172.21.15.86:0/1953806220 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559dcfe67030 con 0x559dcfe6fe10 2024-04-21T08:44:17.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.843+0000 7f1775331640 1 --2- 172.21.15.86:0/1953806220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcfe6fe10 0x559dcfe5fe90 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f1760001eb0 tx=0x7f1760005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e7ba3349443ba6e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:17.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.843+0000 7f175b7fe640 1 -- 172.21.15.86:0/1953806220 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1760004120 con 0x559dcfe6fe10 2024-04-21T08:44:17.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.843+0000 7f175b7fe640 1 -- 172.21.15.86:0/1953806220 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f176000fbb0 con 0x559dcfe6fe10 2024-04-21T08:44:17.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.843+0000 7f175b7fe640 1 -- 172.21.15.86:0/1953806220 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176002de10 con 0x559dcfe6fe10 2024-04-21T08:44:17.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.844+0000 7f177a985740 1 -- 172.21.15.86:0/1953806220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcfe6fe10 msgr2=0x559dcfe5fe90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:17.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.844+0000 7f177a985740 1 --2- 172.21.15.86:0/1953806220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcfe6fe10 0x559dcfe5fe90 secure :-1 s=READY pgs=1290 cs=0 l=1 rev1=1 crypto rx=0x7f1760001eb0 tx=0x7f1760005f30 comp rx=0 tx=0).stop 2024-04-21T08:44:17.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.844+0000 7f177a985740 1 -- 172.21.15.86:0/1953806220 shutdown_connections 2024-04-21T08:44:17.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.844+0000 7f177a985740 1 --2- 172.21.15.86:0/1953806220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcfe6fe10 0x559dcfe5fe90 unknown :-1 s=CLOSED pgs=1290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.844+0000 7f177a985740 1 --2- 172.21.15.86:0/1953806220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dcfe67290 0x559dcfe6c5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.844+0000 7f177a985740 1 --2- 172.21.15.86:0/1953806220 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559dcfe668e0 0x559dcfe7bbe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.844+0000 7f177a985740 1 -- 172.21.15.86:0/1953806220 >> 172.21.15.86:0/1953806220 conn(0x559dcfe60fb0 msgr2=0x559dcfe5f780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:17.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.844+0000 7f177a985740 1 -- 172.21.15.86:0/1953806220 shutdown_connections 2024-04-21T08:44:17.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.844+0000 7f177a985740 1 -- 172.21.15.86:0/1953806220 wait complete. 2024-04-21T08:44:17.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.844+0000 7f177a985740 1 -- start start 2024-04-21T08:44:17.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.844+0000 7f177a985740 1 Processor -- start 2024-04-21T08:44:17.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.844+0000 7f177a985740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dcfe668e0 0x559dcfe73db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.844+0000 7f177a985740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcfe67290 0x559dcfe742f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.844+0000 7f177a985740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559dcfe6fe10 0x559dcfe74830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.844+0000 7f177a985740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f17701b2890 con 0x559dcfe67290 2024-04-21T08:44:17.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.844+0000 7f177a985740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f17701b21c0 con 0x559dcfe6fe10 2024-04-21T08:44:17.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.844+0000 7f177a985740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f17701af3e0 con 0x559dcfe668e0 2024-04-21T08:44:17.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.845+0000 7f177a985740 10 client.0 _ll_register_callbacks cb 0x7f176000bd20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:17.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.845+0000 7f1775b32640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcfe67290 0x559dcfe742f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.845+0000 7f1775b32640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcfe67290 0x559dcfe742f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35008/0 (socket says 172.21.15.86:35008) 2024-04-21T08:44:17.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.845+0000 7f1775b32640 1 -- 172.21.15.86:0/2806610141 learned_addr learned my addr 172.21.15.86:0/2806610141 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:17.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.845+0000 7f1775b32640 1 -- 172.21.15.86:0/2806610141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dcfe668e0 msgr2=0x559dcfe73db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:17.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.845+0000 7f1775331640 1 --2- 172.21.15.86:0/2806610141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dcfe668e0 0x559dcfe73db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.845+0000 7f1775b32640 1 --2- 172.21.15.86:0/2806610141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dcfe668e0 0x559dcfe73db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.845+0000 7f1775331640 1 --2- 172.21.15.86:0/2806610141 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559dcfe6fe10 0x559dcfe74830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.845+0000 7f1775b32640 1 -- 172.21.15.86:0/2806610141 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559dcfe6fe10 msgr2=0x559dcfe74830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:17.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.845+0000 7f1775b32640 1 --2- 172.21.15.86:0/2806610141 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559dcfe6fe10 0x559dcfe74830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.845+0000 7f1775b32640 1 -- 172.21.15.86:0/2806610141 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559dcfee5b60 con 0x559dcfe67290 2024-04-21T08:44:17.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.845+0000 7f1775331640 1 --2- 172.21.15.86:0/2806610141 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559dcfe6fe10 0x559dcfe74830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:17.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.845+0000 7f1775b32640 1 --2- 172.21.15.86:0/2806610141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcfe67290 0x559dcfe742f0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f176c005360 tx=0x7f176c0052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4b1a4aaaa9d0286b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:17.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.845+0000 7f17597fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:17.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.845+0000 7f17597fa640 1 -- 172.21.15.86:0/2806610141 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f176c045040 con 0x559dcfe67290 2024-04-21T08:44:17.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.845+0000 7f17597fa640 1 -- 172.21.15.86:0/2806610141 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f176c004430 con 0x559dcfe67290 2024-04-21T08:44:17.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.845+0000 7f177a985740 1 -- 172.21.15.86:0/2806610141 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f176000fbb0 con 0x559dcfe67290 2024-04-21T08:44:17.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.845+0000 7f17597fa640 1 -- 172.21.15.86:0/2806610141 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f176c005400 con 0x559dcfe67290 2024-04-21T08:44:17.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.845+0000 7f177a985740 10 client.18048 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:17.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.845+0000 7f177a985740 1 -- 172.21.15.86:0/2806610141 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559dcfee6ea0 con 0x559dcfe67290 2024-04-21T08:44:17.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.846+0000 7f17597fa640 1 -- 172.21.15.86:0/2806610141 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f176000fbb0 con 0x559dcfe67290 2024-04-21T08:44:17.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.846+0000 7f17597fa640 1 -- 172.21.15.86:0/2806610141 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f176c051280 con 0x559dcfe67290 2024-04-21T08:44:17.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.846+0000 7f177a985740 10 client.18048 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:17.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.846+0000 7f177a985740 10 client.18048 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:17.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.846+0000 7f177a985740 20 client.18048 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:17.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.846+0000 7f177a985740 1 --2- 172.21.15.86:0/2806610141 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559dcfee7120 0x559dcfee9510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.846+0000 7f177a985740 4 client.18048 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:44:17.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.846+0000 7f177a985740 1 -- 172.21.15.86:0/2806610141 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559dcfdd2790 con 0x559dcfee7120 2024-04-21T08:44:17.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.846+0000 7f1775331640 1 --2- 172.21.15.86:0/2806610141 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559dcfee7120 0x559dcfee9510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.847+0000 7f17597fa640 10 client.18048 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:17.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.847+0000 7f1775331640 1 --2- 172.21.15.86:0/2806610141 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559dcfee7120 0x559dcfee9510 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3d9af6141ab861eb server_cookie=3b40321bfa33902b in_seq=0 out_seq=0 2024-04-21T08:44:17.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.848+0000 7f17597fa640 1 -- 172.21.15.86:0/2806610141 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559dcfdd2790 con 0x559dcfee7120 2024-04-21T08:44:17.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.848+0000 7f17597fa640 10 client.18048 handle_command_reply: tid=0 2024-04-21T08:44:17.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.848+0000 7f177a985740 1 client.18048 shutdown 2024-04-21T08:44:17.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.848+0000 7f177a985740 1 -- 172.21.15.86:0/2806610141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcfe67290 msgr2=0x559dcfe742f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:17.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.848+0000 7f177a985740 1 --2- 172.21.15.86:0/2806610141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcfe67290 0x559dcfe742f0 secure :-1 s=READY pgs=1291 cs=0 l=1 rev1=1 crypto rx=0x7f176c005360 tx=0x7f176c0052f0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.848+0000 7f177a985740 1 -- 172.21.15.86:0/2806610141 shutdown_connections 2024-04-21T08:44:17.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.848+0000 7f177a985740 1 --2- 172.21.15.86:0/2806610141 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559dcfee7120 0x559dcfee9510 crc :-1 s=READY pgs=184 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.848+0000 7f177a985740 1 --2- 172.21.15.86:0/2806610141 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559dcfe6fe10 0x559dcfe74830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.848+0000 7f17597fa640 0 client.18048 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:17.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.848+0000 7f177a985740 1 --2- 172.21.15.86:0/2806610141 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559dcfe67290 0x559dcfe742f0 unknown :-1 s=CLOSED pgs=1291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.848+0000 7f177a985740 1 --2- 172.21.15.86:0/2806610141 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559dcfe668e0 0x559dcfe73db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.848+0000 7f177a985740 1 -- 172.21.15.86:0/2806610141 >> 172.21.15.86:0/2806610141 conn(0x559dcfe60fb0 msgr2=0x559dcfe62b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:17.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.848+0000 7f177a985740 1 -- 172.21.15.86:0/2806610141 shutdown_connections 2024-04-21T08:44:17.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.848+0000 7f177a985740 1 -- 172.21.15.86:0/2806610141 wait complete. 2024-04-21T08:44:17.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.848+0000 7f177a985740 20 client.18048 trim_cache size 0 max 16384 2024-04-21T08:44:17.854 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:44:17.854 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-21T08:44:17.854 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "9aad138f-d980-4312-882a-b44aa5f6da9b", 2024-04-21T08:44:17.854 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-21T08:44:17.854 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:44:17.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.848+0000 7f1777d96640 1 -- 172.21.15.86:0/244288975 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f175c0a3d50 msgr2=0x7f175c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:17.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.848+0000 7f1777d96640 1 --2- 172.21.15.86:0/244288975 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f175c0a3d50 0x7f175c0a6210 secure :-1 s=READY pgs=203 cs=0 l=1 rev1=1 crypto rx=0x7f176809c790 tx=0x7f176806e4d0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.849+0000 7f1777d96640 1 -- 172.21.15.86:0/244288975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1770074cd0 msgr2=0x7f17701a68f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:17.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.849+0000 7f1777d96640 1 --2- 172.21.15.86:0/244288975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1770074cd0 0x7f17701a68f0 secure :-1 s=READY pgs=1287 cs=0 l=1 rev1=1 crypto rx=0x7f176800cce0 tx=0x7f176800a620 comp rx=0 tx=0).stop 2024-04-21T08:44:17.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.849+0000 7f1777d96640 1 -- 172.21.15.86:0/244288975 shutdown_connections 2024-04-21T08:44:17.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.849+0000 7f1777d96640 1 --2- 172.21.15.86:0/244288975 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f175c0a3d50 0x7f175c0a6210 unknown :-1 s=CLOSED pgs=203 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.849+0000 7f1777d96640 1 --2- 172.21.15.86:0/244288975 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1770074cd0 0x7f17701a68f0 unknown :-1 s=CLOSED pgs=1287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.849+0000 7f1777d96640 1 --2- 172.21.15.86:0/244288975 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1770107260 0x7f1770074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.849+0000 7f1777d96640 1 --2- 172.21.15.86:0/244288975 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1770102190 0x7f1770074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.849+0000 7f1777d96640 1 -- 172.21.15.86:0/244288975 >> 172.21.15.86:0/244288975 conn(0x7f17700793a0 msgr2=0x7f17700fff30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:17.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.849+0000 7f1777d96640 1 -- 172.21.15.86:0/244288975 shutdown_connections 2024-04-21T08:44:17.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.849+0000 7f1777d96640 1 -- 172.21.15.86:0/244288975 wait complete. 2024-04-21T08:44:17.859 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:44:17.859 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:44:17.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.959+0000 7fb045752640 1 -- 172.21.15.86:0/1985954332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb040108000 msgr2=0x7fb0401083e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:17.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.959+0000 7fb045752640 1 --2- 172.21.15.86:0/1985954332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb040108000 0x7fb0401083e0 secure :-1 s=READY pgs=1292 cs=0 l=1 rev1=1 crypto rx=0x7fb030009900 tx=0x7fb03002bcd0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.960+0000 7fb045752640 1 -- 172.21.15.86:0/1985954332 shutdown_connections 2024-04-21T08:44:17.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.960+0000 7fb045752640 1 --2- 172.21.15.86:0/1985954332 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb04006ace0 0x7fb04010ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.960+0000 7fb045752640 1 --2- 172.21.15.86:0/1985954332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb040108920 0x7fb04006a5d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.960+0000 7fb045752640 1 --2- 172.21.15.86:0/1985954332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb040108000 0x7fb0401083e0 unknown :-1 s=CLOSED pgs=1292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.960+0000 7fb045752640 1 -- 172.21.15.86:0/1985954332 >> 172.21.15.86:0/1985954332 conn(0x7fb040076c50 msgr2=0x7fb040077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:17.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.960+0000 7fb045752640 1 -- 172.21.15.86:0/1985954332 shutdown_connections 2024-04-21T08:44:17.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.960+0000 7fb045752640 1 -- 172.21.15.86:0/1985954332 wait complete. 2024-04-21T08:44:17.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.960+0000 7fb045752640 1 Processor -- start 2024-04-21T08:44:17.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.960+0000 7fb045752640 1 -- start start 2024-04-21T08:44:17.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.960+0000 7fb045752640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb04006ace0 0x7fb0401986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.960+0000 7fb045752640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb040108000 0x7fb040198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.960+0000 7fb045752640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb040108920 0x7fb04019cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.960+0000 7fb045752640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb040112180 con 0x7fb04006ace0 2024-04-21T08:44:17.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.960+0000 7fb045752640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb040112000 con 0x7fb040108000 2024-04-21T08:44:17.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.961+0000 7fb045752640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb040112300 con 0x7fb040108920 2024-04-21T08:44:17.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.961+0000 7fb03effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb04006ace0 0x7fb0401986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.961+0000 7fb03effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb04006ace0 0x7fb0401986c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35032/0 (socket says 172.21.15.86:35032) 2024-04-21T08:44:17.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.961+0000 7fb03effd640 1 -- 172.21.15.86:0/825216429 learned_addr learned my addr 172.21.15.86:0/825216429 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:17.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.961+0000 7fb03effd640 1 -- 172.21.15.86:0/825216429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb040108920 msgr2=0x7fb04019cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:17.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.961+0000 7fb03effd640 1 --2- 172.21.15.86:0/825216429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb040108920 0x7fb04019cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.961+0000 7fb03effd640 1 -- 172.21.15.86:0/825216429 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb040108000 msgr2=0x7fb040198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:17.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.961+0000 7fb03effd640 1 --2- 172.21.15.86:0/825216429 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb040108000 0x7fb040198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.961+0000 7fb03effd640 1 -- 172.21.15.86:0/825216429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb04019d6b0 con 0x7fb04006ace0 2024-04-21T08:44:17.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.961+0000 7fb03effd640 1 --2- 172.21.15.86:0/825216429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb04006ace0 0x7fb0401986c0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fb030005940 tx=0x7fb03002d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:17.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.961+0000 7fb027fff640 1 -- 172.21.15.86:0/825216429 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb030043020 con 0x7fb04006ace0 2024-04-21T08:44:17.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.961+0000 7fb027fff640 1 -- 172.21.15.86:0/825216429 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb03002dc90 con 0x7fb04006ace0 2024-04-21T08:44:17.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.961+0000 7fb045752640 1 -- 172.21.15.86:0/825216429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb04019d940 con 0x7fb04006ace0 2024-04-21T08:44:17.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.961+0000 7fb027fff640 1 -- 172.21.15.86:0/825216429 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb030003ab0 con 0x7fb04006ace0 2024-04-21T08:44:17.963 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.961+0000 7fb045752640 1 -- 172.21.15.86:0/825216429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0401a5230 con 0x7fb04006ace0 2024-04-21T08:44:17.969 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.964+0000 7fb027fff640 1 -- 172.21.15.86:0/825216429 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb030046020 con 0x7fb04006ace0 2024-04-21T08:44:17.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.964+0000 7fb048428740 1 Processor -- start 2024-04-21T08:44:17.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.964+0000 7fb048428740 1 -- start start 2024-04-21T08:44:17.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.964+0000 7fb048428740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639f4972580 0x5639f4972960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.964+0000 7fb048428740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639f4972ea0 0x5639f497b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.964+0000 7fb048428740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639f497ba80 0x5639f497de70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.964+0000 7fb048428740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5639f48d71e0 con 0x5639f4972580 2024-04-21T08:44:17.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.964+0000 7fb048428740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5639f4874390 con 0x5639f4972ea0 2024-04-21T08:44:17.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.964+0000 7fb048428740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5639f48d31a0 con 0x5639f497ba80 2024-04-21T08:44:17.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.964+0000 7fb03f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639f4972580 0x5639f4972960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.964+0000 7fb027fff640 1 --2- 172.21.15.86:0/825216429 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb0280a3d20 0x7fb0280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.964+0000 7fb03f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639f4972580 0x5639f4972960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35044/0 (socket says 172.21.15.86:35044) 2024-04-21T08:44:17.970 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.964+0000 7fb03f7fe640 1 -- 172.21.15.86:0/2644646221 learned_addr learned my addr 172.21.15.86:0/2644646221 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:17.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.965+0000 7fb03f7fe640 1 --2- 172.21.15.86:0/825216429 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb0280a3d20 0x7fb0280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.965+0000 7fb03e7fc640 1 --2- 172.21.15.86:0/2644646221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639f497ba80 0x5639f497de70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.965+0000 7fb027fff640 1 -- 172.21.15.86:0/825216429 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fb0300f2c90 con 0x7fb04006ace0 2024-04-21T08:44:17.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.965+0000 7fb03effd640 1 --2- 172.21.15.86:0/2644646221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639f4972ea0 0x5639f497b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.965+0000 7fb03f7fe640 1 -- 172.21.15.86:0/2644646221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639f497ba80 msgr2=0x5639f497de70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:17.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.965+0000 7fb03f7fe640 1 --2- 172.21.15.86:0/2644646221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639f497ba80 0x5639f497de70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.965+0000 7fb03f7fe640 1 -- 172.21.15.86:0/2644646221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639f4972ea0 msgr2=0x5639f497b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:17.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.965+0000 7fb03f7fe640 1 --2- 172.21.15.86:0/2644646221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639f4972ea0 0x5639f497b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.965+0000 7fb03f7fe640 1 -- 172.21.15.86:0/2644646221 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639f46b37d0 con 0x5639f4972580 2024-04-21T08:44:17.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.965+0000 7fb03f7fe640 1 --2- 172.21.15.86:0/2644646221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639f4972580 0x5639f4972960 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fb038013870 tx=0x7fb038013d30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5b2f2d217ff74dd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:17.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.965+0000 7fb0257fa640 1 -- 172.21.15.86:0/2644646221 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb038010d20 con 0x5639f4972580 2024-04-21T08:44:17.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.965+0000 7fb0257fa640 1 -- 172.21.15.86:0/2644646221 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0380044e0 con 0x5639f4972580 2024-04-21T08:44:17.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.965+0000 7fb0257fa640 1 -- 172.21.15.86:0/2644646221 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb03801c7b0 con 0x5639f4972580 2024-04-21T08:44:17.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.966+0000 7fb03f7fe640 1 --2- 172.21.15.86:0/825216429 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb0280a3d20 0x7fb0280a61e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb03801d000 tx=0x7fb038018000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:17.971 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.966+0000 7fb048428740 1 -- 172.21.15.86:0/2644646221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639f4972580 msgr2=0x5639f4972960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:17.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.966+0000 7fb048428740 1 --2- 172.21.15.86:0/2644646221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639f4972580 0x5639f4972960 secure :-1 s=READY pgs=1294 cs=0 l=1 rev1=1 crypto rx=0x7fb038013870 tx=0x7fb038013d30 comp rx=0 tx=0).stop 2024-04-21T08:44:17.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.966+0000 7fb048428740 1 -- 172.21.15.86:0/2644646221 shutdown_connections 2024-04-21T08:44:17.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.966+0000 7fb048428740 1 --2- 172.21.15.86:0/2644646221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639f497ba80 0x5639f497de70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.966+0000 7fb048428740 1 --2- 172.21.15.86:0/2644646221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639f4972ea0 0x5639f497b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.966+0000 7fb048428740 1 --2- 172.21.15.86:0/2644646221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639f4972580 0x5639f4972960 unknown :-1 s=CLOSED pgs=1294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.966+0000 7fb048428740 1 -- 172.21.15.86:0/2644646221 >> 172.21.15.86:0/2644646221 conn(0x5639f496ccb0 msgr2=0x5639f496d090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:17.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.966+0000 7fb048428740 1 -- 172.21.15.86:0/2644646221 shutdown_connections 2024-04-21T08:44:17.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.966+0000 7fb048428740 1 -- 172.21.15.86:0/2644646221 wait complete. 2024-04-21T08:44:17.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.966+0000 7fb048428740 1 -- start start 2024-04-21T08:44:17.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.966+0000 7fb048428740 1 Processor -- start 2024-04-21T08:44:17.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.966+0000 7fb048428740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639f4972580 0x5639f4978ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.966+0000 7fb048428740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639f4972ea0 0x5639f4979000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.967+0000 7fb048428740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639f497ba80 0x5639f4979540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.967+0000 7fb048428740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5639f497e870 con 0x5639f497ba80 2024-04-21T08:44:17.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.967+0000 7fb048428740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5639f48902f0 con 0x5639f4972580 2024-04-21T08:44:17.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.967+0000 7fb048428740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5639f497e9f0 con 0x5639f4972ea0 2024-04-21T08:44:17.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.967+0000 7fb048428740 10 client.0 _ll_register_callbacks cb 0x5639f489d580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:17.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.967+0000 7fb03f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639f497ba80 0x5639f4979540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.967+0000 7fb03f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639f497ba80 0x5639f4979540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35058/0 (socket says 172.21.15.86:35058) 2024-04-21T08:44:17.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.967+0000 7fb03f7fe640 1 -- 172.21.15.86:0/3433662084 learned_addr learned my addr 172.21.15.86:0/3433662084 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:17.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.967+0000 7fb03effd640 1 --2- 172.21.15.86:0/3433662084 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639f4972580 0x5639f4978ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.967+0000 7fb03e7fc640 1 --2- 172.21.15.86:0/3433662084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639f4972ea0 0x5639f4979000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.967+0000 7fb03f7fe640 1 -- 172.21.15.86:0/3433662084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639f4972ea0 msgr2=0x5639f4979000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:17.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.967+0000 7fb03f7fe640 1 --2- 172.21.15.86:0/3433662084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639f4972ea0 0x5639f4979000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.967+0000 7fb03f7fe640 1 -- 172.21.15.86:0/3433662084 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639f4972580 msgr2=0x5639f4978ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:17.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.967+0000 7fb03f7fe640 1 --2- 172.21.15.86:0/3433662084 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639f4972580 0x5639f4978ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.967+0000 7fb03f7fe640 1 -- 172.21.15.86:0/3433662084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639f45e93f0 con 0x5639f497ba80 2024-04-21T08:44:17.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.967+0000 7fb03f7fe640 1 --2- 172.21.15.86:0/3433662084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639f497ba80 0x5639f4979540 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fb0380108c0 tx=0x7fb0380048c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf7288cd131fd789 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:17.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.967+0000 7fb022ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:17.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.967+0000 7fb022ff5640 1 -- 172.21.15.86:0/3433662084 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb038034060 con 0x5639f497ba80 2024-04-21T08:44:17.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.967+0000 7fb022ff5640 1 -- 172.21.15.86:0/3433662084 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fb03801c590 con 0x5639f497ba80 2024-04-21T08:44:17.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.967+0000 7fb048428740 1 -- 172.21.15.86:0/3433662084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0380044e0 con 0x5639f497ba80 2024-04-21T08:44:17.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.967+0000 7fb022ff5640 1 -- 172.21.15.86:0/3433662084 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb03803b050 con 0x5639f497ba80 2024-04-21T08:44:17.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.967+0000 7fb048428740 10 client.18060 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:17.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.967+0000 7fb048428740 1 -- 172.21.15.86:0/3433662084 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5639f45a90f0 con 0x5639f497ba80 2024-04-21T08:44:17.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.968+0000 7fb022ff5640 1 -- 172.21.15.86:0/3433662084 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0380044e0 con 0x5639f497ba80 2024-04-21T08:44:17.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.968+0000 7fb022ff5640 1 -- 172.21.15.86:0/3433662084 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb03801a070 con 0x5639f497ba80 2024-04-21T08:44:17.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.968+0000 7fb048428740 10 client.18060 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:17.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.968+0000 7fb048428740 10 client.18060 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:17.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.968+0000 7fb048428740 20 client.18060 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:17.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.968+0000 7fb048428740 1 --2- 172.21.15.86:0/3433662084 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5639f498ba90 0x5639f498de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.968+0000 7fb048428740 4 client.18060 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:17.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.968+0000 7fb048428740 1 -- 172.21.15.86:0/3433662084 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5639f48e4c00 con 0x5639f498ba90 2024-04-21T08:44:17.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.968+0000 7fb03effd640 1 --2- 172.21.15.86:0/3433662084 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5639f498ba90 0x5639f498de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.968+0000 7fb022ff5640 10 client.18060 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:17.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.968+0000 7fb03effd640 1 --2- 172.21.15.86:0/3433662084 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5639f498ba90 0x5639f498de80 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=598fd4de7d82c648 server_cookie=7a9621f1c2c66bf1 in_seq=0 out_seq=0 2024-04-21T08:44:17.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.969+0000 7fb022ff5640 1 -- 172.21.15.86:0/3433662084 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5639f48e4c00 con 0x5639f498ba90 2024-04-21T08:44:17.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.969+0000 7fb022ff5640 10 client.18060 handle_command_reply: tid=0 2024-04-21T08:44:17.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.969+0000 7fb048428740 1 client.18060 shutdown 2024-04-21T08:44:17.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.969+0000 7fb048428740 1 -- 172.21.15.86:0/3433662084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639f497ba80 msgr2=0x5639f4979540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:17.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.969+0000 7fb048428740 1 --2- 172.21.15.86:0/3433662084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639f497ba80 0x5639f4979540 secure :-1 s=READY pgs=1295 cs=0 l=1 rev1=1 crypto rx=0x7fb0380108c0 tx=0x7fb0380048c0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.969+0000 7fb048428740 1 -- 172.21.15.86:0/3433662084 shutdown_connections 2024-04-21T08:44:17.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.969+0000 7fb048428740 1 --2- 172.21.15.86:0/3433662084 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5639f498ba90 0x5639f498de80 crc :-1 s=READY pgs=185 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.969+0000 7fb048428740 1 --2- 172.21.15.86:0/3433662084 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639f497ba80 0x5639f4979540 unknown :-1 s=CLOSED pgs=1295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.969+0000 7fb048428740 1 --2- 172.21.15.86:0/3433662084 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639f4972ea0 0x5639f4979000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.969+0000 7fb022ff5640 0 client.18060 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:17.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.969+0000 7fb048428740 1 --2- 172.21.15.86:0/3433662084 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639f4972580 0x5639f4978ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.969+0000 7fb048428740 1 -- 172.21.15.86:0/3433662084 >> 172.21.15.86:0/3433662084 conn(0x5639f496ccb0 msgr2=0x5639f496ea00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:17.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.969+0000 7fb048428740 1 -- 172.21.15.86:0/3433662084 shutdown_connections 2024-04-21T08:44:17.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.969+0000 7fb048428740 1 -- 172.21.15.86:0/3433662084 wait complete. 2024-04-21T08:44:17.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.969+0000 7fb048428740 20 client.18060 trim_cache size 0 max 16384 2024-04-21T08:44:17.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.986+0000 7fb048428740 1 Processor -- start 2024-04-21T08:44:17.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.986+0000 7fb048428740 1 -- start start 2024-04-21T08:44:17.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.986+0000 7fb048428740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639f4972580 0x5639f497b1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.986+0000 7fb048428740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639f4972ea0 0x5639f4987730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.986+0000 7fb048428740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639f497ba80 0x5639f4979190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.987+0000 7fb048428740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb0401af220 con 0x5639f497ba80 2024-04-21T08:44:17.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.987+0000 7fb048428740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb0401b0d60 con 0x5639f4972ea0 2024-04-21T08:44:17.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.987+0000 7fb048428740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb0401af8f0 con 0x5639f4972580 2024-04-21T08:44:17.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.987+0000 7fb03f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639f497ba80 0x5639f4979190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.987+0000 7fb03f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639f497ba80 0x5639f4979190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35064/0 (socket says 172.21.15.86:35064) 2024-04-21T08:44:17.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.987+0000 7fb03f7fe640 1 -- 172.21.15.86:0/2721437216 learned_addr learned my addr 172.21.15.86:0/2721437216 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:17.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.987+0000 7fb03effd640 1 --2- 172.21.15.86:0/2721437216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639f4972580 0x5639f497b1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.987+0000 7fb03f7fe640 1 -- 172.21.15.86:0/2721437216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639f4972580 msgr2=0x5639f497b1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:17.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.987+0000 7fb03f7fe640 1 --2- 172.21.15.86:0/2721437216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639f4972580 0x5639f497b1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.987+0000 7fb03e7fc640 1 --2- 172.21.15.86:0/2721437216 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639f4972ea0 0x5639f4987730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.987+0000 7fb03f7fe640 1 -- 172.21.15.86:0/2721437216 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639f4972ea0 msgr2=0x5639f4987730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:17.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.987+0000 7fb03f7fe640 1 --2- 172.21.15.86:0/2721437216 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639f4972ea0 0x5639f4987730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.987+0000 7fb03f7fe640 1 -- 172.21.15.86:0/2721437216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5639f496efa0 con 0x5639f497ba80 2024-04-21T08:44:17.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.987+0000 7fb03e7fc640 1 --2- 172.21.15.86:0/2721437216 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639f4972ea0 0x5639f4987730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:17.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.987+0000 7fb03f7fe640 1 --2- 172.21.15.86:0/2721437216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639f497ba80 0x5639f4979190 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fb038001220 tx=0x7fb0380137c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b455e1ea93711374 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:17.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.987+0000 7fb024ff9640 1 -- 172.21.15.86:0/2721437216 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb038034040 con 0x5639f497ba80 2024-04-21T08:44:17.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.987+0000 7fb024ff9640 1 -- 172.21.15.86:0/2721437216 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0380040f0 con 0x5639f497ba80 2024-04-21T08:44:17.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.987+0000 7fb024ff9640 1 -- 172.21.15.86:0/2721437216 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0380132a0 con 0x5639f497ba80 2024-04-21T08:44:17.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.988+0000 7fb048428740 1 -- 172.21.15.86:0/2721437216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639f497ba80 msgr2=0x5639f4979190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:17.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.988+0000 7fb048428740 1 --2- 172.21.15.86:0/2721437216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639f497ba80 0x5639f4979190 secure :-1 s=READY pgs=1296 cs=0 l=1 rev1=1 crypto rx=0x7fb038001220 tx=0x7fb0380137c0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.988+0000 7fb048428740 1 -- 172.21.15.86:0/2721437216 shutdown_connections 2024-04-21T08:44:17.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.988+0000 7fb048428740 1 --2- 172.21.15.86:0/2721437216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639f497ba80 0x5639f4979190 unknown :-1 s=CLOSED pgs=1296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.988+0000 7fb048428740 1 --2- 172.21.15.86:0/2721437216 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639f4972ea0 0x5639f4987730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.988+0000 7fb048428740 1 --2- 172.21.15.86:0/2721437216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639f4972580 0x5639f497b1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.988+0000 7fb048428740 1 -- 172.21.15.86:0/2721437216 >> 172.21.15.86:0/2721437216 conn(0x5639f496ccb0 msgr2=0x5639f4978a80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:17.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.988+0000 7fb048428740 1 -- 172.21.15.86:0/2721437216 shutdown_connections 2024-04-21T08:44:17.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.988+0000 7fb048428740 1 -- 172.21.15.86:0/2721437216 wait complete. 2024-04-21T08:44:17.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.988+0000 7fb048428740 1 -- start start 2024-04-21T08:44:17.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.988+0000 7fb048428740 1 Processor -- start 2024-04-21T08:44:17.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.988+0000 7fb048428740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639f4972580 0x5639f497f950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.988+0000 7fb048428740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639f4972ea0 0x5639f497fe90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.988+0000 7fb048428740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639f497ba80 0x5639f4980410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.988+0000 7fb048428740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb0401b0690 con 0x5639f4972580 2024-04-21T08:44:17.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.988+0000 7fb048428740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb0401affc0 con 0x5639f497ba80 2024-04-21T08:44:17.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.988+0000 7fb048428740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb0401ad1e0 con 0x5639f4972ea0 2024-04-21T08:44:17.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.988+0000 7fb048428740 10 client.0 _ll_register_callbacks cb 0x7fb038013640 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:17.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.988+0000 7fb03effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639f4972580 0x5639f497f950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.988+0000 7fb03effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639f4972580 0x5639f497f950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35080/0 (socket says 172.21.15.86:35080) 2024-04-21T08:44:17.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.988+0000 7fb03effd640 1 -- 172.21.15.86:0/2496274233 learned_addr learned my addr 172.21.15.86:0/2496274233 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:17.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.989+0000 7fb03e7fc640 1 --2- 172.21.15.86:0/2496274233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639f4972ea0 0x5639f497fe90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.989+0000 7fb03f7fe640 1 --2- 172.21.15.86:0/2496274233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639f497ba80 0x5639f4980410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.989+0000 7fb03effd640 1 -- 172.21.15.86:0/2496274233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639f4972ea0 msgr2=0x5639f497fe90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:17.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.989+0000 7fb03effd640 1 --2- 172.21.15.86:0/2496274233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639f4972ea0 0x5639f497fe90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.989+0000 7fb03effd640 1 -- 172.21.15.86:0/2496274233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639f497ba80 msgr2=0x5639f4980410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:17.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.989+0000 7fb03effd640 1 --2- 172.21.15.86:0/2496274233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639f497ba80 0x5639f4980410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.989+0000 7fb03effd640 1 -- 172.21.15.86:0/2496274233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5639f49f1230 con 0x5639f4972580 2024-04-21T08:44:17.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.989+0000 7fb03f7fe640 1 --2- 172.21.15.86:0/2496274233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639f497ba80 0x5639f4980410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:17.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.989+0000 7fb03effd640 1 --2- 172.21.15.86:0/2496274233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639f4972580 0x5639f497f950 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fb0300be690 tx=0x7fb0300df040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9284bec347ff90d9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:17.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.989+0000 7fb022ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:17.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.989+0000 7fb022ff5640 1 -- 172.21.15.86:0/2496274233 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0300d7070 con 0x5639f4972580 2024-04-21T08:44:17.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.989+0000 7fb022ff5640 1 -- 172.21.15.86:0/2496274233 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fb030005340 con 0x5639f4972580 2024-04-21T08:44:17.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.989+0000 7fb048428740 1 -- 172.21.15.86:0/2496274233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb0380040f0 con 0x5639f4972580 2024-04-21T08:44:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.989+0000 7fb022ff5640 1 -- 172.21.15.86:0/2496274233 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb0300c7600 con 0x5639f4972580 2024-04-21T08:44:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.989+0000 7fb048428740 10 client.18066 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.989+0000 7fb048428740 1 -- 172.21.15.86:0/2496274233 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5639f49f2510 con 0x5639f4972580 2024-04-21T08:44:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.989+0000 7fb022ff5640 1 -- 172.21.15.86:0/2496274233 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0380040f0 con 0x5639f4972580 2024-04-21T08:44:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.989+0000 7fb022ff5640 1 -- 172.21.15.86:0/2496274233 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb0300f02b0 con 0x5639f4972580 2024-04-21T08:44:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.989+0000 7fb048428740 10 client.18066 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.989+0000 7fb048428740 10 client.18066 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.989+0000 7fb048428740 20 client.18066 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.989+0000 7fb048428740 1 --2- 172.21.15.86:0/2496274233 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5639f49f2c30 0x5639f49f5050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.989+0000 7fb048428740 4 client.18066 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:44:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.989+0000 7fb048428740 1 -- 172.21.15.86:0/2496274233 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5639f48dea70 con 0x5639f49f2c30 2024-04-21T08:44:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.990+0000 7fb03e7fc640 1 --2- 172.21.15.86:0/2496274233 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5639f49f2c30 0x5639f49f5050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:17.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.990+0000 7fb03e7fc640 1 --2- 172.21.15.86:0/2496274233 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5639f49f2c30 0x5639f49f5050 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b775c4c04bbefcb server_cookie=9a7b0fe1a0b5a755 in_seq=0 out_seq=0 2024-04-21T08:44:17.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.990+0000 7fb022ff5640 10 client.18066 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:17.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.990+0000 7fb022ff5640 1 -- 172.21.15.86:0/2496274233 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5639f48dea70 con 0x5639f49f2c30 2024-04-21T08:44:17.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.990+0000 7fb022ff5640 10 client.18066 handle_command_reply: tid=0 2024-04-21T08:44:17.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.990+0000 7fb048428740 1 client.18066 shutdown 2024-04-21T08:44:17.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.991+0000 7fb048428740 1 -- 172.21.15.86:0/2496274233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639f4972580 msgr2=0x5639f497f950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:17.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.991+0000 7fb048428740 1 --2- 172.21.15.86:0/2496274233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639f4972580 0x5639f497f950 secure :-1 s=READY pgs=1297 cs=0 l=1 rev1=1 crypto rx=0x7fb0300be690 tx=0x7fb0300df040 comp rx=0 tx=0).stop 2024-04-21T08:44:17.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.991+0000 7fb048428740 1 -- 172.21.15.86:0/2496274233 shutdown_connections 2024-04-21T08:44:17.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.991+0000 7fb048428740 1 --2- 172.21.15.86:0/2496274233 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5639f49f2c30 0x5639f49f5050 crc :-1 s=READY pgs=186 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.991+0000 7fb048428740 1 --2- 172.21.15.86:0/2496274233 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5639f497ba80 0x5639f4980410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.991+0000 7fb022ff5640 0 client.18066 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:17.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.991+0000 7fb048428740 1 --2- 172.21.15.86:0/2496274233 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5639f4972ea0 0x5639f497fe90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.991+0000 7fb048428740 1 --2- 172.21.15.86:0/2496274233 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5639f4972580 0x5639f497f950 unknown :-1 s=CLOSED pgs=1297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:17.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.991+0000 7fb048428740 1 -- 172.21.15.86:0/2496274233 >> 172.21.15.86:0/2496274233 conn(0x5639f496ccb0 msgr2=0x5639f496e820 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:17.997 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.991+0000 7fb048428740 1 -- 172.21.15.86:0/2496274233 shutdown_connections 2024-04-21T08:44:17.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.991+0000 7fb048428740 1 -- 172.21.15.86:0/2496274233 wait complete. 2024-04-21T08:44:17.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.991+0000 7fb048428740 20 client.18066 trim_cache size 0 max 16384 2024-04-21T08:44:17.998 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:44:17.998 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:44:17.998 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-21T08:44:17.998 INFO:teuthology.orchestra.run.smithi086.stdout: "9aad138f-d980-4312-882a-b44aa5f6da9b": { 2024-04-21T08:44:17.998 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-21T08:44:17.998 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "9aad138f-d980-4312-882a-b44aa5f6da9b", 2024-04-21T08:44:17.998 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-21T08:44:17.998 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:44:17.998 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:44:17.998 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:44:17.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.991+0000 7fb045752640 1 -- 172.21.15.86:0/825216429 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb0280a3d20 msgr2=0x7fb0280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:17.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.991+0000 7fb045752640 1 --2- 172.21.15.86:0/825216429 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb0280a3d20 0x7fb0280a61e0 secure :-1 s=READY pgs=204 cs=0 l=1 rev1=1 crypto rx=0x7fb03801d000 tx=0x7fb038018000 comp rx=0 tx=0).stop 2024-04-21T08:44:17.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.991+0000 7fb045752640 1 -- 172.21.15.86:0/825216429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb04006ace0 msgr2=0x7fb0401986c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:18.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.991+0000 7fb045752640 1 --2- 172.21.15.86:0/825216429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb04006ace0 0x7fb0401986c0 secure :-1 s=READY pgs=1293 cs=0 l=1 rev1=1 crypto rx=0x7fb030005940 tx=0x7fb03002d890 comp rx=0 tx=0).stop 2024-04-21T08:44:18.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.991+0000 7fb045752640 1 -- 172.21.15.86:0/825216429 shutdown_connections 2024-04-21T08:44:18.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.991+0000 7fb045752640 1 --2- 172.21.15.86:0/825216429 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb0280a3d20 0x7fb0280a61e0 unknown :-1 s=CLOSED pgs=204 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:18.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.991+0000 7fb045752640 1 --2- 172.21.15.86:0/825216429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb040108920 0x7fb04019cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:18.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.991+0000 7fb045752640 1 --2- 172.21.15.86:0/825216429 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb040108000 0x7fb040198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:18.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.991+0000 7fb045752640 1 --2- 172.21.15.86:0/825216429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb04006ace0 0x7fb0401986c0 unknown :-1 s=CLOSED pgs=1293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:18.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.991+0000 7fb045752640 1 -- 172.21.15.86:0/825216429 >> 172.21.15.86:0/825216429 conn(0x7fb040076c50 msgr2=0x7fb0400fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:18.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.991+0000 7fb045752640 1 -- 172.21.15.86:0/825216429 shutdown_connections 2024-04-21T08:44:18.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:17.991+0000 7fb045752640 1 -- 172.21.15.86:0/825216429 wait complete. 2024-04-21T08:44:18.001 INFO:tasks.cephfs.filesystem:scrub status for tag:9aad138f-d980-4312-882a-b44aa5f6da9b - {'path': '/', 'tag': '9aad138f-d980-4312-882a-b44aa5f6da9b', 'options': 'recursive,force'} 2024-04-21T08:44:18.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:18 smithi086 ceph-mon[27939]: pgmap v168: 129 pgs: 129 active+clean; 3.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-04-21T08:44:18.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:18 smithi088 ceph-mon[31259]: pgmap v168: 129 pgs: 129 active+clean; 3.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-04-21T08:44:18.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:18 smithi179 ceph-mon[31829]: pgmap v168: 129 pgs: 129 active+clean; 3.7 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 16 op/s 2024-04-21T08:44:19.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:19 smithi086 ceph-mon[27939]: scrub queued for path: / 2024-04-21T08:44:19.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:19 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:44:19.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:19 smithi086 ceph-mon[27939]: scrub summary: active paths [/] 2024-04-21T08:44:19.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:19 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:44:19.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:19 smithi088 ceph-mon[31259]: scrub queued for path: / 2024-04-21T08:44:19.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:19 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:44:19.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:19 smithi088 ceph-mon[31259]: scrub summary: active paths [/] 2024-04-21T08:44:19.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:19 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:44:19.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:19 smithi179 ceph-mon[31829]: scrub queued for path: / 2024-04-21T08:44:19.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:19 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:44:19.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:19 smithi179 ceph-mon[31829]: scrub summary: active paths [/] 2024-04-21T08:44:19.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:19 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:44:20.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:20 smithi086 ceph-mon[27939]: pgmap v169: 129 pgs: 129 active+clean; 3.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-04-21T08:44:20.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:20 smithi086 ceph-mon[27939]: scrub summary: idle 2024-04-21T08:44:20.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:20 smithi088 ceph-mon[31259]: pgmap v169: 129 pgs: 129 active+clean; 3.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-04-21T08:44:20.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:20 smithi088 ceph-mon[31259]: scrub summary: idle 2024-04-21T08:44:20.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:20 smithi179 ceph-mon[31829]: pgmap v169: 129 pgs: 129 active+clean; 3.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 57 KiB/s wr, 22 op/s 2024-04-21T08:44:20.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:20 smithi179 ceph-mon[31829]: scrub summary: idle 2024-04-21T08:44:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.411+0000 7f9ab8d96640 1 -- 172.21.15.86:0/76200577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ab4101130 msgr2=0x7f9ab41015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.411+0000 7f9ab8d96640 1 --2- 172.21.15.86:0/76200577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ab4101130 0x7f9ab41015b0 secure :-1 s=READY pgs=1298 cs=0 l=1 rev1=1 crypto rx=0x7f9a9c009930 tx=0x7f9a9c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:22.411 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.411+0000 7f9ab8d96640 1 -- 172.21.15.86:0/76200577 shutdown_connections 2024-04-21T08:44:22.412 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.411+0000 7f9ab8d96640 1 --2- 172.21.15.86:0/76200577 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ab4101af0 0x7f9ab410dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:22.412 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.411+0000 7f9ab8d96640 1 --2- 172.21.15.86:0/76200577 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ab4101130 0x7f9ab41015b0 unknown :-1 s=CLOSED pgs=1298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:22.412 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.411+0000 7f9ab8d96640 1 --2- 172.21.15.86:0/76200577 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ab4106f70 0x7f9ab4107350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:22.412 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.411+0000 7f9ab8d96640 1 -- 172.21.15.86:0/76200577 >> 172.21.15.86:0/76200577 conn(0x7f9ab4077ba0 msgr2=0x7f9ab4077fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:22.412 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.411+0000 7f9ab8d96640 1 -- 172.21.15.86:0/76200577 shutdown_connections 2024-04-21T08:44:22.412 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.411+0000 7f9ab8d96640 1 -- 172.21.15.86:0/76200577 wait complete. 2024-04-21T08:44:22.412 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.412+0000 7f9ab8d96640 1 Processor -- start 2024-04-21T08:44:22.412 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.412+0000 7f9ab8d96640 1 -- start start 2024-04-21T08:44:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.412+0000 7f9ab8d96640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ab4101130 0x7f9ab41972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.412+0000 7f9ab8d96640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ab4101af0 0x7f9ab4197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.413+0000 7f9ab8d96640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ab4106f70 0x7f9ab4197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.413+0000 7f9ab8d96640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab4111080 con 0x7f9ab4106f70 2024-04-21T08:44:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.413+0000 7f9ab8d96640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab4110f00 con 0x7f9ab4101af0 2024-04-21T08:44:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.413+0000 7f9ab8d96640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab4111200 con 0x7f9ab4101130 2024-04-21T08:44:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.413+0000 7f9ab2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ab4106f70 0x7f9ab4197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:22.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.413+0000 7f9ab2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ab4106f70 0x7f9ab4197d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46508/0 (socket says 172.21.15.86:46508) 2024-04-21T08:44:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.413+0000 7f9ab2d9d640 1 -- 172.21.15.86:0/2055266260 learned_addr learned my addr 172.21.15.86:0/2055266260 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.413+0000 7f9ab259c640 1 --2- 172.21.15.86:0/2055266260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ab4101130 0x7f9ab41972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.413+0000 7f9aabfff640 1 --2- 172.21.15.86:0/2055266260 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ab4101af0 0x7f9ab4197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.413+0000 7f9ab2d9d640 1 -- 172.21.15.86:0/2055266260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ab4101130 msgr2=0x7f9ab41972d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.413+0000 7f9ab2d9d640 1 --2- 172.21.15.86:0/2055266260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ab4101130 0x7f9ab41972d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.413+0000 7f9ab2d9d640 1 -- 172.21.15.86:0/2055266260 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ab4101af0 msgr2=0x7f9ab4197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.413+0000 7f9ab2d9d640 1 --2- 172.21.15.86:0/2055266260 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ab4101af0 0x7f9ab4197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.413+0000 7f9ab2d9d640 1 -- 172.21.15.86:0/2055266260 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9ab41a6ab0 con 0x7f9ab4106f70 2024-04-21T08:44:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.413+0000 7f9ab2d9d640 1 --2- 172.21.15.86:0/2055266260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ab4106f70 0x7f9ab4197d50 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f9aac00db20 tx=0x7f9aac00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.413+0000 7f9aab7fe640 1 -- 172.21.15.86:0/2055266260 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aac017070 con 0x7f9ab4106f70 2024-04-21T08:44:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.413+0000 7f9aab7fe640 1 -- 172.21.15.86:0/2055266260 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9aac003e20 con 0x7f9ab4106f70 2024-04-21T08:44:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.413+0000 7f9ab8d96640 1 -- 172.21.15.86:0/2055266260 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9ab41a6da0 con 0x7f9ab4106f70 2024-04-21T08:44:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.413+0000 7f9aab7fe640 1 -- 172.21.15.86:0/2055266260 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aac010460 con 0x7f9ab4106f70 2024-04-21T08:44:22.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.413+0000 7f9ab8d96640 1 -- 172.21.15.86:0/2055266260 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9ab41a7270 con 0x7f9ab4106f70 2024-04-21T08:44:22.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.415+0000 7f9aab7fe640 1 -- 172.21.15.86:0/2055266260 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9aac003970 con 0x7f9ab4106f70 2024-04-21T08:44:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.416+0000 7f9abb985740 1 Processor -- start 2024-04-21T08:44:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.416+0000 7f9abb985740 1 -- start start 2024-04-21T08:44:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.416+0000 7f9abb985740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ef5a58d0 0x55c4ef5a4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.417+0000 7f9abb985740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ef5a4b60 0x55c4ef5a4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.417+0000 7f9abb985740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4ef5ae0e0 0x55c4ef5b04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.417+0000 7f9abb985740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c4ef5098d0 con 0x55c4ef5a4b60 2024-04-21T08:44:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.417+0000 7f9abb985740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c4ef4a6e80 con 0x55c4ef5ae0e0 2024-04-21T08:44:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.417+0000 7f9abb985740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c4ef505890 con 0x55c4ef5a58d0 2024-04-21T08:44:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.417+0000 7f9aab7fe640 1 --2- 172.21.15.86:0/2055266260 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f9aa00a3dc0 0x7f9aa00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.417+0000 7f9ab259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ef5a4b60 0x55c4ef5a4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.417+0000 7f9ab259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ef5a4b60 0x55c4ef5a4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46516/0 (socket says 172.21.15.86:46516) 2024-04-21T08:44:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.417+0000 7f9ab259c640 1 -- 172.21.15.86:0/2287699372 learned_addr learned my addr 172.21.15.86:0/2287699372 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:22.423 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.417+0000 7f9ab2d9d640 1 --2- 172.21.15.86:0/2055266260 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f9aa00a3dc0 0x7f9aa00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.417+0000 7f9aabfff640 1 --2- 172.21.15.86:0/2287699372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ef5a58d0 0x55c4ef5a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.417+0000 7f9aabfff640 1 --2- 172.21.15.86:0/2287699372 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4ef5ae0e0 0x55c4ef5b04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.417+0000 7f9ab259c640 1 -- 172.21.15.86:0/2287699372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ef5a58d0 msgr2=0x55c4ef5a4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.417+0000 7f9ab259c640 1 --2- 172.21.15.86:0/2287699372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ef5a58d0 0x55c4ef5a4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.417+0000 7f9ab259c640 1 -- 172.21.15.86:0/2287699372 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4ef5ae0e0 msgr2=0x55c4ef5b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.417+0000 7f9aab7fe640 1 -- 172.21.15.86:0/2055266260 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f9aac0d0910 con 0x7f9ab4106f70 2024-04-21T08:44:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.417+0000 7f9ab259c640 1 --2- 172.21.15.86:0/2287699372 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4ef5ae0e0 0x55c4ef5b04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.417+0000 7f9ab259c640 1 -- 172.21.15.86:0/2287699372 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4ef4c8320 con 0x55c4ef5a4b60 2024-04-21T08:44:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.417+0000 7f9aabfff640 1 --2- 172.21.15.86:0/2287699372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ef5a58d0 0x55c4ef5a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.417+0000 7f9ab259c640 1 --2- 172.21.15.86:0/2287699372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ef5a4b60 0x55c4ef5a4fe0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f9aa400bc90 tx=0x7f9aa40107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=79f5ed081963a34e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.417+0000 7f9ab2d9d640 1 --2- 172.21.15.86:0/2055266260 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f9aa00a3dc0 0x7f9aa00a6280 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9aac09c2b0 tx=0x7f9aac0a0040 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.417+0000 7f9aa8ff9640 1 -- 172.21.15.86:0/2287699372 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa401c480 con 0x55c4ef5a4b60 2024-04-21T08:44:22.424 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.417+0000 7f9aa8ff9640 1 -- 172.21.15.86:0/2287699372 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9aa401cad0 con 0x55c4ef5a4b60 2024-04-21T08:44:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.417+0000 7f9aa8ff9640 1 -- 172.21.15.86:0/2287699372 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4011ae0 con 0x55c4ef5a4b60 2024-04-21T08:44:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.418+0000 7f9abb985740 1 -- 172.21.15.86:0/2287699372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ef5a4b60 msgr2=0x55c4ef5a4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.418+0000 7f9abb985740 1 --2- 172.21.15.86:0/2287699372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ef5a4b60 0x55c4ef5a4fe0 secure :-1 s=READY pgs=1300 cs=0 l=1 rev1=1 crypto rx=0x7f9aa400bc90 tx=0x7f9aa40107a0 comp rx=0 tx=0).stop 2024-04-21T08:44:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.418+0000 7f9abb985740 1 -- 172.21.15.86:0/2287699372 shutdown_connections 2024-04-21T08:44:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.418+0000 7f9abb985740 1 --2- 172.21.15.86:0/2287699372 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4ef5ae0e0 0x55c4ef5b04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.418+0000 7f9abb985740 1 --2- 172.21.15.86:0/2287699372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ef5a4b60 0x55c4ef5a4fe0 unknown :-1 s=CLOSED pgs=1300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.418+0000 7f9abb985740 1 --2- 172.21.15.86:0/2287699372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ef5a58d0 0x55c4ef5a4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.418+0000 7f9abb985740 1 -- 172.21.15.86:0/2287699372 >> 172.21.15.86:0/2287699372 conn(0x55c4ef56bc90 msgr2=0x55c4ef59f850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.418+0000 7f9abb985740 1 -- 172.21.15.86:0/2287699372 shutdown_connections 2024-04-21T08:44:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.418+0000 7f9abb985740 1 -- 172.21.15.86:0/2287699372 wait complete. 2024-04-21T08:44:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.419+0000 7f9abb985740 1 -- start start 2024-04-21T08:44:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.419+0000 7f9abb985740 1 Processor -- start 2024-04-21T08:44:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.419+0000 7f9abb985740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4ef5a4b60 0x55c4ef5ada00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:22.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.419+0000 7f9abb985740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ef5a58d0 0x55c4ef5aae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.419+0000 7f9abb985740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ef5ae0e0 0x55c4ef5ab380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.419+0000 7f9abb985740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c4ef5b0e10 con 0x55c4ef5a58d0 2024-04-21T08:44:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.419+0000 7f9abb985740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c4ef491eb0 con 0x55c4ef5a4b60 2024-04-21T08:44:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.419+0000 7f9abb985740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c4ef5b0f90 con 0x55c4ef5ae0e0 2024-04-21T08:44:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.419+0000 7f9abb985740 10 client.0 _ll_register_callbacks cb 0x55c4ef4beb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.419+0000 7f9ab259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ef5a58d0 0x55c4ef5aae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.419+0000 7f9ab259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ef5a58d0 0x55c4ef5aae00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46518/0 (socket says 172.21.15.86:46518) 2024-04-21T08:44:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.419+0000 7f9aabfff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4ef5a4b60 0x55c4ef5ada00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.419+0000 7f9ab259c640 1 -- 172.21.15.86:0/3413542993 learned_addr learned my addr 172.21.15.86:0/3413542993 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.419+0000 7f9aabfff640 1 --2- 172.21.15.86:0/3413542993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ef5ae0e0 0x55c4ef5ab380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.419+0000 7f9ab259c640 1 -- 172.21.15.86:0/3413542993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ef5ae0e0 msgr2=0x55c4ef5ab380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.419+0000 7f9ab259c640 1 --2- 172.21.15.86:0/3413542993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ef5ae0e0 0x55c4ef5ab380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:22.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.419+0000 7f9ab259c640 1 -- 172.21.15.86:0/3413542993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4ef5a4b60 msgr2=0x55c4ef5ada00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.419+0000 7f9ab259c640 1 --2- 172.21.15.86:0/3413542993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4ef5a4b60 0x55c4ef5ada00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.419+0000 7f9ab259c640 1 -- 172.21.15.86:0/3413542993 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4ef21c210 con 0x55c4ef5a58d0 2024-04-21T08:44:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.420+0000 7f9ab259c640 1 --2- 172.21.15.86:0/3413542993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ef5a58d0 0x55c4ef5aae00 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4011700 tx=0x7f9aa40058e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=10f2e26eb014527a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.420+0000 7f9a99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.420+0000 7f9a99ffb640 1 -- 172.21.15.86:0/3413542993 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa401c680 con 0x55c4ef5a58d0 2024-04-21T08:44:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.420+0000 7f9a99ffb640 1 -- 172.21.15.86:0/3413542993 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f9aa401c820 con 0x55c4ef5a58d0 2024-04-21T08:44:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.420+0000 7f9abb985740 1 -- 172.21.15.86:0/3413542993 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9aa401cad0 con 0x55c4ef5a58d0 2024-04-21T08:44:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.420+0000 7f9a99ffb640 1 -- 172.21.15.86:0/3413542993 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9aa4011860 con 0x55c4ef5a58d0 2024-04-21T08:44:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.420+0000 7f9abb985740 10 client.18078 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.420+0000 7f9abb985740 1 -- 172.21.15.86:0/3413542993 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c4ef2e8620 con 0x55c4ef5a58d0 2024-04-21T08:44:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.420+0000 7f9a99ffb640 1 -- 172.21.15.86:0/3413542993 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9aa401cad0 con 0x55c4ef5a58d0 2024-04-21T08:44:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.420+0000 7f9a99ffb640 1 -- 172.21.15.86:0/3413542993 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9aa40046c0 con 0x55c4ef5a58d0 2024-04-21T08:44:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.420+0000 7f9abb985740 10 client.18078 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.420+0000 7f9abb985740 10 client.18078 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:22.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.420+0000 7f9abb985740 20 client.18078 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.420+0000 7f9abb985740 1 --2- 172.21.15.86:0/3413542993 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c4ef5be010 0x55c4ef5c0400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.420+0000 7f9abb985740 4 client.18078 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.420+0000 7f9abb985740 1 -- 172.21.15.86:0/3413542993 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c4ef51d880 con 0x55c4ef5be010 2024-04-21T08:44:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.421+0000 7f9aabfff640 1 --2- 172.21.15.86:0/3413542993 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c4ef5be010 0x55c4ef5c0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.421+0000 7f9a99ffb640 10 client.18078 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.421+0000 7f9aabfff640 1 --2- 172.21.15.86:0/3413542993 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c4ef5be010 0x55c4ef5c0400 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f907c694f6e177a6 server_cookie=2c7240234fdb314 in_seq=0 out_seq=0 2024-04-21T08:44:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.422+0000 7f9a99ffb640 1 -- 172.21.15.86:0/3413542993 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c4ef51d880 con 0x55c4ef5be010 2024-04-21T08:44:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.422+0000 7f9a99ffb640 10 client.18078 handle_command_reply: tid=0 2024-04-21T08:44:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.422+0000 7f9abb985740 1 client.18078 shutdown 2024-04-21T08:44:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.422+0000 7f9abb985740 1 -- 172.21.15.86:0/3413542993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ef5a58d0 msgr2=0x55c4ef5aae00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.422+0000 7f9abb985740 1 --2- 172.21.15.86:0/3413542993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ef5a58d0 0x55c4ef5aae00 secure :-1 s=READY pgs=1301 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4011700 tx=0x7f9aa40058e0 comp rx=0 tx=0).stop 2024-04-21T08:44:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.422+0000 7f9abb985740 1 -- 172.21.15.86:0/3413542993 shutdown_connections 2024-04-21T08:44:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.422+0000 7f9abb985740 1 --2- 172.21.15.86:0/3413542993 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c4ef5be010 0x55c4ef5c0400 crc :-1 s=READY pgs=187 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:22.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.422+0000 7f9abb985740 1 --2- 172.21.15.86:0/3413542993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ef5ae0e0 0x55c4ef5ab380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.422+0000 7f9abb985740 1 --2- 172.21.15.86:0/3413542993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ef5a58d0 0x55c4ef5aae00 unknown :-1 s=CLOSED pgs=1301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.422+0000 7f9a99ffb640 0 client.18078 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.422+0000 7f9abb985740 1 --2- 172.21.15.86:0/3413542993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4ef5a4b60 0x55c4ef5ada00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.422+0000 7f9abb985740 1 -- 172.21.15.86:0/3413542993 >> 172.21.15.86:0/3413542993 conn(0x55c4ef56bc90 msgr2=0x55c4ef5a15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.422+0000 7f9abb985740 1 -- 172.21.15.86:0/3413542993 shutdown_connections 2024-04-21T08:44:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.422+0000 7f9abb985740 1 -- 172.21.15.86:0/3413542993 wait complete. 2024-04-21T08:44:22.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.422+0000 7f9abb985740 20 client.18078 trim_cache size 0 max 16384 2024-04-21T08:44:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.437+0000 7f9abb985740 1 Processor -- start 2024-04-21T08:44:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.437+0000 7f9abb985740 1 -- start start 2024-04-21T08:44:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.437+0000 7f9abb985740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ef5a4b60 0x55c4ef5b9d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.437+0000 7f9abb985740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ef5a58d0 0x55c4ef5aa5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.437+0000 7f9abb985740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4ef5ae0e0 0x55c4ef5c59d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.437+0000 7f9abb985740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab41b13f0 con 0x55c4ef5a58d0 2024-04-21T08:44:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.437+0000 7f9abb985740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab41b2f30 con 0x55c4ef5ae0e0 2024-04-21T08:44:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.437+0000 7f9abb985740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab41b1ac0 con 0x55c4ef5a4b60 2024-04-21T08:44:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.437+0000 7f9ab259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ef5a58d0 0x55c4ef5aa5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.437+0000 7f9ab259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ef5a58d0 0x55c4ef5aa5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46520/0 (socket says 172.21.15.86:46520) 2024-04-21T08:44:22.441 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.437+0000 7f9ab259c640 1 -- 172.21.15.86:0/2115484407 learned_addr learned my addr 172.21.15.86:0/2115484407 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.437+0000 7f9aabfff640 1 --2- 172.21.15.86:0/2115484407 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ef5a4b60 0x55c4ef5b9d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.437+0000 7f9ab259c640 1 -- 172.21.15.86:0/2115484407 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ef5a4b60 msgr2=0x55c4ef5b9d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.437+0000 7f9ab259c640 1 --2- 172.21.15.86:0/2115484407 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ef5a4b60 0x55c4ef5b9d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.437+0000 7f9ab259c640 1 -- 172.21.15.86:0/2115484407 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4ef5ae0e0 msgr2=0x55c4ef5c59d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.437+0000 7f9ab259c640 1 --2- 172.21.15.86:0/2115484407 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4ef5ae0e0 0x55c4ef5c59d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.437+0000 7f9ab259c640 1 -- 172.21.15.86:0/2115484407 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c4ef5a3e30 con 0x55c4ef5a58d0 2024-04-21T08:44:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.437+0000 7f9aabfff640 1 --2- 172.21.15.86:0/2115484407 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ef5a4b60 0x55c4ef5b9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.437+0000 7f9ab259c640 1 --2- 172.21.15.86:0/2115484407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ef5a58d0 0x55c4ef5aa5d0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4011730 tx=0x7f9aa401cf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7dbf151ca10bba9d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.438+0000 7f9a9bfff640 1 -- 172.21.15.86:0/2115484407 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4004cb0 con 0x55c4ef5a58d0 2024-04-21T08:44:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.438+0000 7f9a9bfff640 1 -- 172.21.15.86:0/2115484407 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9aa4004fe0 con 0x55c4ef5a58d0 2024-04-21T08:44:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.438+0000 7f9a9bfff640 1 -- 172.21.15.86:0/2115484407 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa401c6e0 con 0x55c4ef5a58d0 2024-04-21T08:44:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.438+0000 7f9abb985740 1 -- 172.21.15.86:0/2115484407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ef5a58d0 msgr2=0x55c4ef5aa5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:22.442 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.438+0000 7f9abb985740 1 --2- 172.21.15.86:0/2115484407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ef5a58d0 0x55c4ef5aa5d0 secure :-1 s=READY pgs=1302 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4011730 tx=0x7f9aa401cf10 comp rx=0 tx=0).stop 2024-04-21T08:44:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.438+0000 7f9abb985740 1 -- 172.21.15.86:0/2115484407 shutdown_connections 2024-04-21T08:44:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.438+0000 7f9abb985740 1 --2- 172.21.15.86:0/2115484407 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4ef5ae0e0 0x55c4ef5c59d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.438+0000 7f9abb985740 1 --2- 172.21.15.86:0/2115484407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ef5a58d0 0x55c4ef5aa5d0 unknown :-1 s=CLOSED pgs=1302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.438+0000 7f9abb985740 1 --2- 172.21.15.86:0/2115484407 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ef5a4b60 0x55c4ef5b9d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.438+0000 7f9abb985740 1 -- 172.21.15.86:0/2115484407 >> 172.21.15.86:0/2115484407 conn(0x55c4ef56bc90 msgr2=0x55c4ef5a6010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.438+0000 7f9abb985740 1 -- 172.21.15.86:0/2115484407 shutdown_connections 2024-04-21T08:44:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.438+0000 7f9abb985740 1 -- 172.21.15.86:0/2115484407 wait complete. 2024-04-21T08:44:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.439+0000 7f9abb985740 1 -- start start 2024-04-21T08:44:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.439+0000 7f9abb985740 1 Processor -- start 2024-04-21T08:44:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.439+0000 7f9abb985740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ef5a4b60 0x55c4ef5a7270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.439+0000 7f9abb985740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ef5a58d0 0x55c4ef5a77b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.439+0000 7f9abb985740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4ef5ae0e0 0x55c4ef5a7cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.439+0000 7f9abb985740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab41b2860 con 0x55c4ef5a58d0 2024-04-21T08:44:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.439+0000 7f9abb985740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab41b2190 con 0x55c4ef5ae0e0 2024-04-21T08:44:22.443 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.439+0000 7f9abb985740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9ab41af3b0 con 0x55c4ef5a4b60 2024-04-21T08:44:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.439+0000 7f9abb985740 10 client.0 _ll_register_callbacks cb 0x7f9aa400bae0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.439+0000 7f9ab259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ef5a58d0 0x55c4ef5a77b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.439+0000 7f9ab259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ef5a58d0 0x55c4ef5a77b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46526/0 (socket says 172.21.15.86:46526) 2024-04-21T08:44:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.439+0000 7f9ab259c640 1 -- 172.21.15.86:0/1322257167 learned_addr learned my addr 172.21.15.86:0/1322257167 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.439+0000 7f9aabfff640 1 --2- 172.21.15.86:0/1322257167 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ef5a4b60 0x55c4ef5a7270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.439+0000 7f9aabfff640 1 --2- 172.21.15.86:0/1322257167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4ef5ae0e0 0x55c4ef5a7cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.439+0000 7f9ab259c640 1 -- 172.21.15.86:0/1322257167 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ef5a4b60 msgr2=0x55c4ef5a7270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.439+0000 7f9ab259c640 1 --2- 172.21.15.86:0/1322257167 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ef5a4b60 0x55c4ef5a7270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.439+0000 7f9ab259c640 1 -- 172.21.15.86:0/1322257167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4ef5ae0e0 msgr2=0x55c4ef5a7cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.439+0000 7f9ab259c640 1 --2- 172.21.15.86:0/1322257167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4ef5ae0e0 0x55c4ef5a7cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.439+0000 7f9ab259c640 1 -- 172.21.15.86:0/1322257167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c4ef59af30 con 0x55c4ef5a58d0 2024-04-21T08:44:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.439+0000 7f9aabfff640 1 --2- 172.21.15.86:0/1322257167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4ef5ae0e0 0x55c4ef5a7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:22.444 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.440+0000 7f9ab259c640 1 --2- 172.21.15.86:0/1322257167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ef5a58d0 0x55c4ef5a77b0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4004000 tx=0x7f9aa4005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=818adc1980c6fc4b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.440+0000 7f9a99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.440+0000 7f9a99ffb640 1 -- 172.21.15.86:0/1322257167 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9aa4005420 con 0x55c4ef5a58d0 2024-04-21T08:44:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.440+0000 7f9a99ffb640 1 -- 172.21.15.86:0/1322257167 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f9aa4011a00 con 0x55c4ef5a58d0 2024-04-21T08:44:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.440+0000 7f9abb985740 1 -- 172.21.15.86:0/1322257167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9aa4004fe0 con 0x55c4ef5a58d0 2024-04-21T08:44:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.440+0000 7f9a99ffb640 1 -- 172.21.15.86:0/1322257167 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9aa4011c00 con 0x55c4ef5a58d0 2024-04-21T08:44:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.440+0000 7f9abb985740 10 client.18084 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.440+0000 7f9abb985740 1 -- 172.21.15.86:0/1322257167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c4ef5b0940 con 0x55c4ef5a58d0 2024-04-21T08:44:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.440+0000 7f9a99ffb640 1 -- 172.21.15.86:0/1322257167 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9aa4004fe0 con 0x55c4ef5a58d0 2024-04-21T08:44:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.440+0000 7f9a99ffb640 1 -- 172.21.15.86:0/1322257167 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9aa4003860 con 0x55c4ef5a58d0 2024-04-21T08:44:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.440+0000 7f9abb985740 10 client.18084 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.440+0000 7f9abb985740 10 client.18084 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.440+0000 7f9abb985740 20 client.18084 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.440+0000 7f9abb985740 1 --2- 172.21.15.86:0/1322257167 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c4ef5b6860 0x55c4ef626250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:22.445 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.440+0000 7f9abb985740 4 client.18084 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:44:22.446 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.440+0000 7f9abb985740 1 -- 172.21.15.86:0/1322257167 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c4ef4a93d0 con 0x55c4ef5b6860 2024-04-21T08:44:22.446 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.441+0000 7f9aabfff640 1 --2- 172.21.15.86:0/1322257167 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c4ef5b6860 0x55c4ef626250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:22.446 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.441+0000 7f9a99ffb640 10 client.18084 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:22.446 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.441+0000 7f9aabfff640 1 --2- 172.21.15.86:0/1322257167 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c4ef5b6860 0x55c4ef626250 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35a3d550d784d6bc server_cookie=53d3721aee842963 in_seq=0 out_seq=0 2024-04-21T08:44:22.454 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.455+0000 7f9a99ffb640 1 -- 172.21.15.86:0/1322257167 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c4ef4a93d0 con 0x55c4ef5b6860 2024-04-21T08:44:22.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.455+0000 7f9a99ffb640 10 client.18084 handle_command_reply: tid=0 2024-04-21T08:44:22.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.455+0000 7f9abb985740 1 client.18084 shutdown 2024-04-21T08:44:22.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.455+0000 7f9abb985740 1 -- 172.21.15.86:0/1322257167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ef5a58d0 msgr2=0x55c4ef5a77b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:22.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.455+0000 7f9abb985740 1 --2- 172.21.15.86:0/1322257167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ef5a58d0 0x55c4ef5a77b0 secure :-1 s=READY pgs=1303 cs=0 l=1 rev1=1 crypto rx=0x7f9aa4004000 tx=0x7f9aa4005850 comp rx=0 tx=0).stop 2024-04-21T08:44:22.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.455+0000 7f9abb985740 1 -- 172.21.15.86:0/1322257167 shutdown_connections 2024-04-21T08:44:22.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.455+0000 7f9abb985740 1 --2- 172.21.15.86:0/1322257167 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c4ef5b6860 0x55c4ef626250 crc :-1 s=READY pgs=188 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:22.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.455+0000 7f9abb985740 1 --2- 172.21.15.86:0/1322257167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c4ef5ae0e0 0x55c4ef5a7cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:22.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.455+0000 7f9abb985740 1 --2- 172.21.15.86:0/1322257167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c4ef5a58d0 0x55c4ef5a77b0 unknown :-1 s=CLOSED pgs=1303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:22.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.455+0000 7f9a99ffb640 0 client.18084 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:22.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.455+0000 7f9abb985740 1 --2- 172.21.15.86:0/1322257167 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c4ef5a4b60 0x55c4ef5a7270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:22.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.455+0000 7f9abb985740 1 -- 172.21.15.86:0/1322257167 >> 172.21.15.86:0/1322257167 conn(0x55c4ef56bc90 msgr2=0x55c4ef59e8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:22.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.455+0000 7f9abb985740 1 -- 172.21.15.86:0/1322257167 shutdown_connections 2024-04-21T08:44:22.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.455+0000 7f9abb985740 1 -- 172.21.15.86:0/1322257167 wait complete. 2024-04-21T08:44:22.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.455+0000 7f9abb985740 20 client.18084 trim_cache size 0 max 16384 2024-04-21T08:44:22.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:44:22.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:44:22.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:44:22.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:44:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.455+0000 7f9ab8d96640 1 -- 172.21.15.86:0/2055266260 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f9aa00a3dc0 msgr2=0x7f9aa00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.455+0000 7f9ab8d96640 1 --2- 172.21.15.86:0/2055266260 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f9aa00a3dc0 0x7f9aa00a6280 secure :-1 s=READY pgs=205 cs=0 l=1 rev1=1 crypto rx=0x7f9aac09c2b0 tx=0x7f9aac0a0040 comp rx=0 tx=0).stop 2024-04-21T08:44:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.455+0000 7f9ab8d96640 1 -- 172.21.15.86:0/2055266260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ab4106f70 msgr2=0x7f9ab4197d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.455+0000 7f9ab8d96640 1 --2- 172.21.15.86:0/2055266260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ab4106f70 0x7f9ab4197d50 secure :-1 s=READY pgs=1299 cs=0 l=1 rev1=1 crypto rx=0x7f9aac00db20 tx=0x7f9aac00b260 comp rx=0 tx=0).stop 2024-04-21T08:44:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.456+0000 7f9ab8d96640 1 -- 172.21.15.86:0/2055266260 shutdown_connections 2024-04-21T08:44:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.456+0000 7f9ab8d96640 1 --2- 172.21.15.86:0/2055266260 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f9aa00a3dc0 0x7f9aa00a6280 unknown :-1 s=CLOSED pgs=205 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.456+0000 7f9ab8d96640 1 --2- 172.21.15.86:0/2055266260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9ab4106f70 0x7f9ab4197d50 unknown :-1 s=CLOSED pgs=1299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.456+0000 7f9ab8d96640 1 --2- 172.21.15.86:0/2055266260 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9ab4101af0 0x7f9ab4197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:22.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.456+0000 7f9ab8d96640 1 --2- 172.21.15.86:0/2055266260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9ab4101130 0x7f9ab41972d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.456+0000 7f9ab8d96640 1 -- 172.21.15.86:0/2055266260 >> 172.21.15.86:0/2055266260 conn(0x7f9ab4077ba0 msgr2=0x7f9ab40ffcb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.456+0000 7f9ab8d96640 1 -- 172.21.15.86:0/2055266260 shutdown_connections 2024-04-21T08:44:22.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:22.456+0000 7f9ab8d96640 1 -- 172.21.15.86:0/2055266260 wait complete. 2024-04-21T08:44:22.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:22 smithi086 ceph-mon[27939]: pgmap v170: 129 pgs: 129 active+clean; 3.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-21T08:44:22.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:22 smithi088 ceph-mon[31259]: pgmap v170: 129 pgs: 129 active+clean; 3.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-21T08:44:22.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:22 smithi179 ceph-mon[31829]: pgmap v170: 129 pgs: 129 active+clean; 3.9 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-21T08:44:24.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:24 smithi086 ceph-mon[27939]: pgmap v171: 129 pgs: 129 active+clean; 4.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 25 op/s 2024-04-21T08:44:24.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:24 smithi088 ceph-mon[31259]: pgmap v171: 129 pgs: 129 active+clean; 4.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 25 op/s 2024-04-21T08:44:24.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:24 smithi179 ceph-mon[31829]: pgmap v171: 129 pgs: 129 active+clean; 4.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 58 KiB/s wr, 25 op/s 2024-04-21T08:44:26.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:26 smithi086 ceph-mon[27939]: pgmap v172: 129 pgs: 129 active+clean; 4.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 16 op/s 2024-04-21T08:44:26.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:26 smithi088 ceph-mon[31259]: pgmap v172: 129 pgs: 129 active+clean; 4.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 16 op/s 2024-04-21T08:44:26.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:26 smithi179 ceph-mon[31829]: pgmap v172: 129 pgs: 129 active+clean; 4.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 16 op/s 2024-04-21T08:44:28.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:28 smithi086 ceph-mon[27939]: pgmap v173: 129 pgs: 129 active+clean; 4.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 16 op/s 2024-04-21T08:44:28.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:28 smithi088 ceph-mon[31259]: pgmap v173: 129 pgs: 129 active+clean; 4.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 16 op/s 2024-04-21T08:44:28.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:28 smithi179 ceph-mon[31829]: pgmap v173: 129 pgs: 129 active+clean; 4.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 35 KiB/s wr, 16 op/s 2024-04-21T08:44:30.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:30 smithi086 ceph-mon[27939]: pgmap v174: 129 pgs: 129 active+clean; 4.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 50 KiB/s wr, 22 op/s 2024-04-21T08:44:30.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:30 smithi088 ceph-mon[31259]: pgmap v174: 129 pgs: 129 active+clean; 4.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 50 KiB/s wr, 22 op/s 2024-04-21T08:44:30.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:30 smithi179 ceph-mon[31829]: pgmap v174: 129 pgs: 129 active+clean; 4.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 50 KiB/s wr, 22 op/s 2024-04-21T08:44:32.798 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:32 smithi179 ceph-mon[31829]: pgmap v175: 129 pgs: 129 active+clean; 4.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 34 KiB/s wr, 16 op/s 2024-04-21T08:44:32.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:32 smithi086 ceph-mon[27939]: pgmap v175: 129 pgs: 129 active+clean; 4.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 34 KiB/s wr, 16 op/s 2024-04-21T08:44:32.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:32 smithi088 ceph-mon[31259]: pgmap v175: 129 pgs: 129 active+clean; 4.2 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 34 KiB/s wr, 16 op/s 2024-04-21T08:44:34.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:34 smithi086 ceph-mon[27939]: pgmap v176: 129 pgs: 129 active+clean; 4.4 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 51 KiB/s wr, 22 op/s 2024-04-21T08:44:34.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:34 smithi088 ceph-mon[31259]: pgmap v176: 129 pgs: 129 active+clean; 4.4 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 51 KiB/s wr, 22 op/s 2024-04-21T08:44:34.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:34 smithi179 ceph-mon[31829]: pgmap v176: 129 pgs: 129 active+clean; 4.4 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 51 KiB/s wr, 22 op/s 2024-04-21T08:44:35.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.550+0000 7f4a3850d640 1 -- 172.21.15.86:0/682586781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a30103b20 msgr2=0x7f4a3010c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:35.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.550+0000 7f4a3850d640 1 --2- 172.21.15.86:0/682586781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a30103b20 0x7f4a3010c000 secure :-1 s=READY pgs=1304 cs=0 l=1 rev1=1 crypto rx=0x7f4a20009900 tx=0x7f4a2002bcd0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.551+0000 7f4a3850d640 1 -- 172.21.15.86:0/682586781 shutdown_connections 2024-04-21T08:44:35.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.551+0000 7f4a3850d640 1 --2- 172.21.15.86:0/682586781 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a3010c540 0x7f4a3010e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.551+0000 7f4a3850d640 1 --2- 172.21.15.86:0/682586781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a30103b20 0x7f4a3010c000 unknown :-1 s=CLOSED pgs=1304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.551+0000 7f4a3850d640 1 --2- 172.21.15.86:0/682586781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a30103200 0x7f4a301035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.551+0000 7f4a3850d640 1 -- 172.21.15.86:0/682586781 >> 172.21.15.86:0/682586781 conn(0x7f4a30077fd0 msgr2=0x7f4a300783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:35.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.551+0000 7f4a3850d640 1 -- 172.21.15.86:0/682586781 shutdown_connections 2024-04-21T08:44:35.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.551+0000 7f4a3850d640 1 -- 172.21.15.86:0/682586781 wait complete. 2024-04-21T08:44:35.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.551+0000 7f4a3850d640 1 Processor -- start 2024-04-21T08:44:35.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.551+0000 7f4a3850d640 1 -- start start 2024-04-21T08:44:35.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.551+0000 7f4a3850d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a30103200 0x7f4a301986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.552+0000 7f4a3850d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a30103b20 0x7f4a30198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.552+0000 7f4a3850d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a3010c540 0x7f4a3019cf80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.552+0000 7f4a3850d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4a30112180 con 0x7f4a30103b20 2024-04-21T08:44:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.552+0000 7f4a3850d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4a30112000 con 0x7f4a3010c540 2024-04-21T08:44:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.552+0000 7f4a3850d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4a30112300 con 0x7f4a30103200 2024-04-21T08:44:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.552+0000 7f4a35aa8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a30103b20 0x7f4a30198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.552+0000 7f4a35aa8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a30103b20 0x7f4a30198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39928/0 (socket says 172.21.15.86:39928) 2024-04-21T08:44:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.552+0000 7f4a35aa8640 1 -- 172.21.15.86:0/2254736825 learned_addr learned my addr 172.21.15.86:0/2254736825 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.552+0000 7f4a362a9640 1 --2- 172.21.15.86:0/2254736825 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a30103200 0x7f4a301986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.552+0000 7f4a35aa8640 1 -- 172.21.15.86:0/2254736825 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a30103200 msgr2=0x7f4a301986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.552+0000 7f4a35aa8640 1 --2- 172.21.15.86:0/2254736825 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a30103200 0x7f4a301986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.552+0000 7f4a36aaa640 1 --2- 172.21.15.86:0/2254736825 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a3010c540 0x7f4a3019cf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:35.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.552+0000 7f4a35aa8640 1 -- 172.21.15.86:0/2254736825 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a3010c540 msgr2=0x7f4a3019cf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.552+0000 7f4a35aa8640 1 --2- 172.21.15.86:0/2254736825 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a3010c540 0x7f4a3019cf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.552+0000 7f4a35aa8640 1 -- 172.21.15.86:0/2254736825 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4a3019d6b0 con 0x7f4a30103b20 2024-04-21T08:44:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.552+0000 7f4a35aa8640 1 --2- 172.21.15.86:0/2254736825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a30103b20 0x7f4a30198c00 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f4a20005940 tx=0x7f4a2002d890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.552+0000 7f4a277fe640 1 -- 172.21.15.86:0/2254736825 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a20043020 con 0x7f4a30103b20 2024-04-21T08:44:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.552+0000 7f4a277fe640 1 -- 172.21.15.86:0/2254736825 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a2002dc90 con 0x7f4a30103b20 2024-04-21T08:44:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.552+0000 7f4a277fe640 1 -- 172.21.15.86:0/2254736825 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a20003ab0 con 0x7f4a30103b20 2024-04-21T08:44:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.552+0000 7f4a3850d640 1 -- 172.21.15.86:0/2254736825 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4a3019d940 con 0x7f4a30103b20 2024-04-21T08:44:35.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.552+0000 7f4a3850d640 1 -- 172.21.15.86:0/2254736825 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4a301a5230 con 0x7f4a30103b20 2024-04-21T08:44:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.554+0000 7f4a277fe640 1 -- 172.21.15.86:0/2254736825 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4a20046020 con 0x7f4a30103b20 2024-04-21T08:44:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.555+0000 7f4a3b1cf740 1 Processor -- start 2024-04-21T08:44:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.555+0000 7f4a3b1cf740 1 -- start start 2024-04-21T08:44:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.556+0000 7f4a3b1cf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5566af8638d0 0x5566af862620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:35.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.556+0000 7f4a277fe640 1 --2- 172.21.15.86:0/2254736825 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4a1c0a3dc0 0x7f4a1c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.556+0000 7f4a3b1cf740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5566af862b60 0x5566af862fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.556+0000 7f4a3b1cf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566af86c0e0 0x5566af86e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.556+0000 7f4a3b1cf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5566af7c78d0 con 0x5566af86c0e0 2024-04-21T08:44:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.556+0000 7f4a3b1cf740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5566af764e80 con 0x5566af862b60 2024-04-21T08:44:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.556+0000 7f4a3b1cf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5566af7c3890 con 0x5566af8638d0 2024-04-21T08:44:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.556+0000 7f4a362a9640 1 --2- 172.21.15.86:0/2254736825 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4a1c0a3dc0 0x7f4a1c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.556+0000 7f4a36aaa640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566af86c0e0 0x5566af86e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.556+0000 7f4a277fe640 1 -- 172.21.15.86:0/2254736825 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f4a200f2ba0 con 0x7f4a30103b20 2024-04-21T08:44:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.556+0000 7f4a36aaa640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566af86c0e0 0x5566af86e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39932/0 (socket says 172.21.15.86:39932) 2024-04-21T08:44:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.556+0000 7f4a36aaa640 1 -- 172.21.15.86:0/3366007330 learned_addr learned my addr 172.21.15.86:0/3366007330 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.556+0000 7f4a36aaa640 1 --2- 172.21.15.86:0/3366007330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5566af8638d0 0x5566af862620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:35.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.556+0000 7f4a35aa8640 1 --2- 172.21.15.86:0/3366007330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5566af862b60 0x5566af862fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.556+0000 7f4a36aaa640 1 -- 172.21.15.86:0/3366007330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5566af8638d0 msgr2=0x5566af862620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.556+0000 7f4a36aaa640 1 --2- 172.21.15.86:0/3366007330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5566af8638d0 0x5566af862620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.556+0000 7f4a36aaa640 1 -- 172.21.15.86:0/3366007330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5566af862b60 msgr2=0x5566af862fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.556+0000 7f4a36aaa640 1 --2- 172.21.15.86:0/3366007330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5566af862b60 0x5566af862fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.556+0000 7f4a36aaa640 1 -- 172.21.15.86:0/3366007330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566af786320 con 0x5566af86c0e0 2024-04-21T08:44:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.556+0000 7f4a362a9640 1 --2- 172.21.15.86:0/2254736825 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4a1c0a3dc0 0x7f4a1c0a6280 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c005d00 tx=0x7f4a2c00b410 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.556+0000 7f4a36aaa640 1 --2- 172.21.15.86:0/3366007330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566af86c0e0 0x5566af86e4d0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f4a28014a20 tx=0x7f4a28009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6dfeaf8d1f759ed4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.556+0000 7f4a24bfe640 1 -- 172.21.15.86:0/3366007330 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a28014270 con 0x5566af86c0e0 2024-04-21T08:44:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.557+0000 7f4a24bfe640 1 -- 172.21.15.86:0/3366007330 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a2800f700 con 0x5566af86c0e0 2024-04-21T08:44:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.557+0000 7f4a24bfe640 1 -- 172.21.15.86:0/3366007330 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a28005220 con 0x5566af86c0e0 2024-04-21T08:44:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.557+0000 7f4a3b1cf740 1 -- 172.21.15.86:0/3366007330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566af86c0e0 msgr2=0x5566af86e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.557+0000 7f4a3b1cf740 1 --2- 172.21.15.86:0/3366007330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566af86c0e0 0x5566af86e4d0 secure :-1 s=READY pgs=1306 cs=0 l=1 rev1=1 crypto rx=0x7f4a28014a20 tx=0x7f4a28009600 comp rx=0 tx=0).stop 2024-04-21T08:44:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.557+0000 7f4a3b1cf740 1 -- 172.21.15.86:0/3366007330 shutdown_connections 2024-04-21T08:44:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.557+0000 7f4a3b1cf740 1 --2- 172.21.15.86:0/3366007330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566af86c0e0 0x5566af86e4d0 unknown :-1 s=CLOSED pgs=1306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.564 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.557+0000 7f4a3b1cf740 1 --2- 172.21.15.86:0/3366007330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5566af862b60 0x5566af862fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.557+0000 7f4a3b1cf740 1 --2- 172.21.15.86:0/3366007330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5566af8638d0 0x5566af862620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.557+0000 7f4a3b1cf740 1 -- 172.21.15.86:0/3366007330 >> 172.21.15.86:0/3366007330 conn(0x5566af829c90 msgr2=0x5566af85d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.557+0000 7f4a3b1cf740 1 -- 172.21.15.86:0/3366007330 shutdown_connections 2024-04-21T08:44:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.557+0000 7f4a3b1cf740 1 -- 172.21.15.86:0/3366007330 wait complete. 2024-04-21T08:44:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.558+0000 7f4a3b1cf740 1 -- start start 2024-04-21T08:44:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.558+0000 7f4a3b1cf740 1 Processor -- start 2024-04-21T08:44:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.558+0000 7f4a3b1cf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566af862b60 0x5566af86baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.558+0000 7f4a3b1cf740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5566af8638d0 0x5566af868e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.558+0000 7f4a3b1cf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5566af86c0e0 0x5566af869440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.558+0000 7f4a3b1cf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5566af86ee10 con 0x5566af862b60 2024-04-21T08:44:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.558+0000 7f4a3b1cf740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5566af74feb0 con 0x5566af8638d0 2024-04-21T08:44:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.558+0000 7f4a3b1cf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5566af86ef90 con 0x5566af86c0e0 2024-04-21T08:44:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.558+0000 7f4a3b1cf740 10 client.0 _ll_register_callbacks cb 0x5566af77cb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.558+0000 7f4a362a9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566af862b60 0x5566af86baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:35.565 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.558+0000 7f4a362a9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566af862b60 0x5566af86baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39936/0 (socket says 172.21.15.86:39936) 2024-04-21T08:44:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.558+0000 7f4a362a9640 1 -- 172.21.15.86:0/3257773557 learned_addr learned my addr 172.21.15.86:0/3257773557 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.558+0000 7f4a35aa8640 1 --2- 172.21.15.86:0/3257773557 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5566af8638d0 0x5566af868e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.558+0000 7f4a36aaa640 1 --2- 172.21.15.86:0/3257773557 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5566af86c0e0 0x5566af869440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.558+0000 7f4a362a9640 1 -- 172.21.15.86:0/3257773557 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5566af86c0e0 msgr2=0x5566af869440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.558+0000 7f4a362a9640 1 --2- 172.21.15.86:0/3257773557 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5566af86c0e0 0x5566af869440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.558+0000 7f4a362a9640 1 -- 172.21.15.86:0/3257773557 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5566af8638d0 msgr2=0x5566af868e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.558+0000 7f4a362a9640 1 --2- 172.21.15.86:0/3257773557 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5566af8638d0 0x5566af868e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.558+0000 7f4a362a9640 1 -- 172.21.15.86:0/3257773557 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566af4da210 con 0x5566af862b60 2024-04-21T08:44:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.559+0000 7f4a35aa8640 1 --2- 172.21.15.86:0/3257773557 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5566af8638d0 0x5566af868e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.559+0000 7f4a362a9640 1 --2- 172.21.15.86:0/3257773557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566af862b60 0x5566af86baa0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c0116c0 tx=0x7f4a2c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e97322dfe6247659 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.559+0000 7f4a19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.559+0000 7f4a19ffb640 1 -- 172.21.15.86:0/3257773557 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2c01e8f0 con 0x5566af862b60 2024-04-21T08:44:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.559+0000 7f4a19ffb640 1 -- 172.21.15.86:0/3257773557 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f4a2c01ea90 con 0x5566af862b60 2024-04-21T08:44:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.559+0000 7f4a3b1cf740 1 -- 172.21.15.86:0/3257773557 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a2800f700 con 0x5566af862b60 2024-04-21T08:44:35.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.559+0000 7f4a19ffb640 1 -- 172.21.15.86:0/3257773557 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4a2c02f050 con 0x5566af862b60 2024-04-21T08:44:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.559+0000 7f4a3b1cf740 10 client.18096 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.559+0000 7f4a3b1cf740 1 -- 172.21.15.86:0/3257773557 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5566af5a6620 con 0x5566af862b60 2024-04-21T08:44:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.559+0000 7f4a19ffb640 1 -- 172.21.15.86:0/3257773557 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a2800f700 con 0x5566af862b60 2024-04-21T08:44:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.559+0000 7f4a19ffb640 1 -- 172.21.15.86:0/3257773557 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4a2c01d3b0 con 0x5566af862b60 2024-04-21T08:44:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.559+0000 7f4a3b1cf740 10 client.18096 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.559+0000 7f4a3b1cf740 10 client.18096 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.559+0000 7f4a3b1cf740 20 client.18096 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.559+0000 7f4a3b1cf740 1 --2- 172.21.15.86:0/3257773557 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5566af87c110 0x5566af87e500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.559+0000 7f4a3b1cf740 4 client.18096 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.559+0000 7f4a3b1cf740 1 -- 172.21.15.86:0/3257773557 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5566af7db880 con 0x5566af87c110 2024-04-21T08:44:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.560+0000 7f4a35aa8640 1 --2- 172.21.15.86:0/3257773557 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5566af87c110 0x5566af87e500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.560+0000 7f4a35aa8640 1 --2- 172.21.15.86:0/3257773557 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5566af87c110 0x5566af87e500 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4fd64ac9bc38133c server_cookie=50f915a7de169438 in_seq=0 out_seq=0 2024-04-21T08:44:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.560+0000 7f4a19ffb640 10 client.18096 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.561+0000 7f4a19ffb640 1 -- 172.21.15.86:0/3257773557 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5566af7db880 con 0x5566af87c110 2024-04-21T08:44:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.561+0000 7f4a19ffb640 10 client.18096 handle_command_reply: tid=0 2024-04-21T08:44:35.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.561+0000 7f4a3b1cf740 1 client.18096 shutdown 2024-04-21T08:44:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.561+0000 7f4a3b1cf740 1 -- 172.21.15.86:0/3257773557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566af862b60 msgr2=0x5566af86baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.561+0000 7f4a3b1cf740 1 --2- 172.21.15.86:0/3257773557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566af862b60 0x5566af86baa0 secure :-1 s=READY pgs=1307 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c0116c0 tx=0x7f4a2c004730 comp rx=0 tx=0).stop 2024-04-21T08:44:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.561+0000 7f4a3b1cf740 1 -- 172.21.15.86:0/3257773557 shutdown_connections 2024-04-21T08:44:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.561+0000 7f4a3b1cf740 1 --2- 172.21.15.86:0/3257773557 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5566af87c110 0x5566af87e500 crc :-1 s=READY pgs=189 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.561+0000 7f4a3b1cf740 1 --2- 172.21.15.86:0/3257773557 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5566af86c0e0 0x5566af869440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.561+0000 7f4a3b1cf740 1 --2- 172.21.15.86:0/3257773557 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5566af8638d0 0x5566af868e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.561+0000 7f4a3b1cf740 1 --2- 172.21.15.86:0/3257773557 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566af862b60 0x5566af86baa0 unknown :-1 s=CLOSED pgs=1307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.561+0000 7f4a3b1cf740 1 -- 172.21.15.86:0/3257773557 >> 172.21.15.86:0/3257773557 conn(0x5566af829c90 msgr2=0x5566af85f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.561+0000 7f4a19ffb640 0 client.18096 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.561+0000 7f4a3b1cf740 1 -- 172.21.15.86:0/3257773557 shutdown_connections 2024-04-21T08:44:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.561+0000 7f4a3b1cf740 1 -- 172.21.15.86:0/3257773557 wait complete. 2024-04-21T08:44:35.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.561+0000 7f4a3b1cf740 20 client.18096 trim_cache size 0 max 16384 2024-04-21T08:44:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.576+0000 7f4a3b1cf740 1 Processor -- start 2024-04-21T08:44:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.576+0000 7f4a3b1cf740 1 -- start start 2024-04-21T08:44:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.576+0000 7f4a3b1cf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5566af862b60 0x5566af877d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.576+0000 7f4a3b1cf740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5566af8638d0 0x5566af867e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.576+0000 7f4a3b1cf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566af86c0e0 0x5566af86b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.576+0000 7f4a3b1cf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4a301af310 con 0x5566af86c0e0 2024-04-21T08:44:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.576+0000 7f4a3b1cf740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4a301b0e50 con 0x5566af8638d0 2024-04-21T08:44:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.576+0000 7f4a3b1cf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4a301af9e0 con 0x5566af862b60 2024-04-21T08:44:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.576+0000 7f4a36aaa640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566af86c0e0 0x5566af86b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.576+0000 7f4a36aaa640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566af86c0e0 0x5566af86b560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39952/0 (socket says 172.21.15.86:39952) 2024-04-21T08:44:35.583 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.576+0000 7f4a36aaa640 1 -- 172.21.15.86:0/2331650332 learned_addr learned my addr 172.21.15.86:0/2331650332 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.576+0000 7f4a35aa8640 1 --2- 172.21.15.86:0/2331650332 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5566af8638d0 0x5566af867e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.576+0000 7f4a362a9640 1 --2- 172.21.15.86:0/2331650332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5566af862b60 0x5566af877d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.576+0000 7f4a36aaa640 1 -- 172.21.15.86:0/2331650332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5566af862b60 msgr2=0x5566af877d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.576+0000 7f4a36aaa640 1 --2- 172.21.15.86:0/2331650332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5566af862b60 0x5566af877d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.576+0000 7f4a36aaa640 1 -- 172.21.15.86:0/2331650332 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5566af8638d0 msgr2=0x5566af867e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.576+0000 7f4a36aaa640 1 --2- 172.21.15.86:0/2331650332 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5566af8638d0 0x5566af867e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.576+0000 7f4a36aaa640 1 -- 172.21.15.86:0/2331650332 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5566af864720 con 0x5566af86c0e0 2024-04-21T08:44:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.576+0000 7f4a362a9640 1 --2- 172.21.15.86:0/2331650332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5566af862b60 0x5566af877d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.577+0000 7f4a36aaa640 1 --2- 172.21.15.86:0/2331650332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566af86c0e0 0x5566af86b560 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f4a28009bc0 tx=0x7f4a28009540 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7ebf56fe8d00c39 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.577+0000 7f4a1bfff640 1 -- 172.21.15.86:0/2331650332 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a280140d0 con 0x5566af86c0e0 2024-04-21T08:44:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.577+0000 7f4a1bfff640 1 -- 172.21.15.86:0/2331650332 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a28003e20 con 0x5566af86c0e0 2024-04-21T08:44:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.577+0000 7f4a1bfff640 1 -- 172.21.15.86:0/2331650332 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a2800f6d0 con 0x5566af86c0e0 2024-04-21T08:44:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.577+0000 7f4a3b1cf740 1 -- 172.21.15.86:0/2331650332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566af86c0e0 msgr2=0x5566af86b560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:35.584 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.577+0000 7f4a3b1cf740 1 --2- 172.21.15.86:0/2331650332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566af86c0e0 0x5566af86b560 secure :-1 s=READY pgs=1308 cs=0 l=1 rev1=1 crypto rx=0x7f4a28009bc0 tx=0x7f4a28009540 comp rx=0 tx=0).stop 2024-04-21T08:44:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.577+0000 7f4a3b1cf740 1 -- 172.21.15.86:0/2331650332 shutdown_connections 2024-04-21T08:44:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.577+0000 7f4a3b1cf740 1 --2- 172.21.15.86:0/2331650332 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566af86c0e0 0x5566af86b560 unknown :-1 s=CLOSED pgs=1308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.577+0000 7f4a3b1cf740 1 --2- 172.21.15.86:0/2331650332 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5566af8638d0 0x5566af867e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.577+0000 7f4a3b1cf740 1 --2- 172.21.15.86:0/2331650332 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5566af862b60 0x5566af877d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.577+0000 7f4a3b1cf740 1 -- 172.21.15.86:0/2331650332 >> 172.21.15.86:0/2331650332 conn(0x5566af829c90 msgr2=0x5566af867700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.577+0000 7f4a3b1cf740 1 -- 172.21.15.86:0/2331650332 shutdown_connections 2024-04-21T08:44:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.577+0000 7f4a3b1cf740 1 -- 172.21.15.86:0/2331650332 wait complete. 2024-04-21T08:44:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.578+0000 7f4a3b1cf740 1 -- start start 2024-04-21T08:44:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.578+0000 7f4a3b1cf740 1 Processor -- start 2024-04-21T08:44:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.578+0000 7f4a3b1cf740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5566af862b60 0x5566af8744f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.578+0000 7f4a3b1cf740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5566af8638d0 0x5566af866190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.578+0000 7f4a3b1cf740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566af86c0e0 0x5566af8666d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:35.585 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.578+0000 7f4a3b1cf740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4a301b0780 con 0x5566af86c0e0 2024-04-21T08:44:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.578+0000 7f4a3b1cf740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4a301b00b0 con 0x5566af862b60 2024-04-21T08:44:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.578+0000 7f4a3b1cf740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4a301ad2d0 con 0x5566af8638d0 2024-04-21T08:44:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.578+0000 7f4a3b1cf740 10 client.0 _ll_register_callbacks cb 0x7f4a2c00b950 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.578+0000 7f4a36aaa640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566af86c0e0 0x5566af8666d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.578+0000 7f4a36aaa640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566af86c0e0 0x5566af8666d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39964/0 (socket says 172.21.15.86:39964) 2024-04-21T08:44:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.578+0000 7f4a36aaa640 1 -- 172.21.15.86:0/1671905537 learned_addr learned my addr 172.21.15.86:0/1671905537 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.578+0000 7f4a35aa8640 1 --2- 172.21.15.86:0/1671905537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5566af8638d0 0x5566af866190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.578+0000 7f4a362a9640 1 --2- 172.21.15.86:0/1671905537 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5566af862b60 0x5566af8744f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.578+0000 7f4a36aaa640 1 -- 172.21.15.86:0/1671905537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5566af8638d0 msgr2=0x5566af866190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.578+0000 7f4a36aaa640 1 --2- 172.21.15.86:0/1671905537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5566af8638d0 0x5566af866190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.578+0000 7f4a36aaa640 1 -- 172.21.15.86:0/1671905537 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5566af862b60 msgr2=0x5566af8744f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.578+0000 7f4a36aaa640 1 --2- 172.21.15.86:0/1671905537 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5566af862b60 0x5566af8744f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.578+0000 7f4a36aaa640 1 -- 172.21.15.86:0/1671905537 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5566af8697b0 con 0x5566af86c0e0 2024-04-21T08:44:35.586 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.579+0000 7f4a36aaa640 1 --2- 172.21.15.86:0/1671905537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566af86c0e0 0x5566af8666d0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f4a28014380 tx=0x7f4a28005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d5e407f1841fe43 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.579+0000 7f4a19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.579+0000 7f4a19ffb640 1 -- 172.21.15.86:0/1671905537 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4a28009ca0 con 0x5566af86c0e0 2024-04-21T08:44:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.579+0000 7f4a19ffb640 1 -- 172.21.15.86:0/1671905537 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f4a28003970 con 0x5566af86c0e0 2024-04-21T08:44:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.579+0000 7f4a3b1cf740 1 -- 172.21.15.86:0/1671905537 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4a28003e20 con 0x5566af86c0e0 2024-04-21T08:44:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.579+0000 7f4a19ffb640 1 -- 172.21.15.86:0/1671905537 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4a2803c050 con 0x5566af86c0e0 2024-04-21T08:44:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.579+0000 7f4a3b1cf740 10 client.18102 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.579+0000 7f4a3b1cf740 1 -- 172.21.15.86:0/1671905537 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5566af86d870 con 0x5566af86c0e0 2024-04-21T08:44:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.579+0000 7f4a19ffb640 1 -- 172.21.15.86:0/1671905537 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4a28003e20 con 0x5566af86c0e0 2024-04-21T08:44:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.579+0000 7f4a19ffb640 1 -- 172.21.15.86:0/1671905537 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4a2800c070 con 0x5566af86c0e0 2024-04-21T08:44:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.579+0000 7f4a3b1cf740 10 client.18102 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.579+0000 7f4a3b1cf740 10 client.18102 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.579+0000 7f4a3b1cf740 20 client.18102 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:35.587 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.579+0000 7f4a3b1cf740 1 --2- 172.21.15.86:0/1671905537 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5566af8e2970 0x5566af8e4d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.579+0000 7f4a3b1cf740 4 client.18102 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:44:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.579+0000 7f4a3b1cf740 1 -- 172.21.15.86:0/1671905537 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5566af7673d0 con 0x5566af8e2970 2024-04-21T08:44:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.580+0000 7f4a362a9640 1 --2- 172.21.15.86:0/1671905537 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5566af8e2970 0x5566af8e4d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.580+0000 7f4a19ffb640 10 client.18102 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:35.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.580+0000 7f4a362a9640 1 --2- 172.21.15.86:0/1671905537 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5566af8e2970 0x5566af8e4d60 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3ab71e572d764b52 server_cookie=f76696c782d37c13 in_seq=0 out_seq=0 2024-04-21T08:44:35.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.597+0000 7f4a19ffb640 1 -- 172.21.15.86:0/1671905537 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5566af7673d0 con 0x5566af8e2970 2024-04-21T08:44:35.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.597+0000 7f4a19ffb640 10 client.18102 handle_command_reply: tid=0 2024-04-21T08:44:35.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.597+0000 7f4a3b1cf740 1 client.18102 shutdown 2024-04-21T08:44:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.597+0000 7f4a3b1cf740 1 -- 172.21.15.86:0/1671905537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566af86c0e0 msgr2=0x5566af8666d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.597+0000 7f4a3b1cf740 1 --2- 172.21.15.86:0/1671905537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566af86c0e0 0x5566af8666d0 secure :-1 s=READY pgs=1309 cs=0 l=1 rev1=1 crypto rx=0x7f4a28014380 tx=0x7f4a28005bd0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.597+0000 7f4a3b1cf740 1 -- 172.21.15.86:0/1671905537 shutdown_connections 2024-04-21T08:44:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.597+0000 7f4a3b1cf740 1 --2- 172.21.15.86:0/1671905537 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5566af8e2970 0x5566af8e4d60 crc :-1 s=READY pgs=190 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.597+0000 7f4a3b1cf740 1 --2- 172.21.15.86:0/1671905537 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5566af86c0e0 0x5566af8666d0 unknown :-1 s=CLOSED pgs=1309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.597+0000 7f4a3b1cf740 1 --2- 172.21.15.86:0/1671905537 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5566af8638d0 0x5566af866190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.597+0000 7f4a3b1cf740 1 --2- 172.21.15.86:0/1671905537 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5566af862b60 0x5566af8744f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.597+0000 7f4a19ffb640 0 client.18102 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.597+0000 7f4a3b1cf740 1 -- 172.21.15.86:0/1671905537 >> 172.21.15.86:0/1671905537 conn(0x5566af829c90 msgr2=0x5566af867f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.597+0000 7f4a3b1cf740 1 -- 172.21.15.86:0/1671905537 shutdown_connections 2024-04-21T08:44:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.597+0000 7f4a3b1cf740 1 -- 172.21.15.86:0/1671905537 wait complete. 2024-04-21T08:44:35.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.597+0000 7f4a3b1cf740 20 client.18102 trim_cache size 0 max 16384 2024-04-21T08:44:35.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:44:35.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:44:35.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:44:35.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:44:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.597+0000 7f4a3850d640 1 -- 172.21.15.86:0/2254736825 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4a1c0a3dc0 msgr2=0x7f4a1c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.597+0000 7f4a3850d640 1 --2- 172.21.15.86:0/2254736825 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4a1c0a3dc0 0x7f4a1c0a6280 secure :-1 s=READY pgs=206 cs=0 l=1 rev1=1 crypto rx=0x7f4a2c005d00 tx=0x7f4a2c00b410 comp rx=0 tx=0).stop 2024-04-21T08:44:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.597+0000 7f4a3850d640 1 -- 172.21.15.86:0/2254736825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a30103b20 msgr2=0x7f4a30198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.597+0000 7f4a3850d640 1 --2- 172.21.15.86:0/2254736825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a30103b20 0x7f4a30198c00 secure :-1 s=READY pgs=1305 cs=0 l=1 rev1=1 crypto rx=0x7f4a20005940 tx=0x7f4a2002d890 comp rx=0 tx=0).stop 2024-04-21T08:44:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.597+0000 7f4a3850d640 1 -- 172.21.15.86:0/2254736825 shutdown_connections 2024-04-21T08:44:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.597+0000 7f4a3850d640 1 --2- 172.21.15.86:0/2254736825 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4a1c0a3dc0 0x7f4a1c0a6280 unknown :-1 s=CLOSED pgs=206 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.597+0000 7f4a3850d640 1 --2- 172.21.15.86:0/2254736825 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4a3010c540 0x7f4a3019cf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.597+0000 7f4a3850d640 1 --2- 172.21.15.86:0/2254736825 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4a30103b20 0x7f4a30198c00 unknown :-1 s=CLOSED pgs=1305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.600 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.597+0000 7f4a3850d640 1 --2- 172.21.15.86:0/2254736825 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4a30103200 0x7f4a301986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.597+0000 7f4a3850d640 1 -- 172.21.15.86:0/2254736825 >> 172.21.15.86:0/2254736825 conn(0x7f4a30077fd0 msgr2=0x7f4a300fdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.597+0000 7f4a3850d640 1 -- 172.21.15.86:0/2254736825 shutdown_connections 2024-04-21T08:44:35.601 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:35.597+0000 7f4a3850d640 1 -- 172.21.15.86:0/2254736825 wait complete. 2024-04-21T08:44:36.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:36 smithi086 ceph-mon[27939]: pgmap v177: 129 pgs: 129 active+clean; 4.4 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-21T08:44:36.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:36 smithi088 ceph-mon[31259]: pgmap v177: 129 pgs: 129 active+clean; 4.4 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-21T08:44:36.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:36 smithi179 ceph-mon[31829]: pgmap v177: 129 pgs: 129 active+clean; 4.4 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-21T08:44:38.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:38 smithi086 ceph-mon[27939]: pgmap v178: 129 pgs: 129 active+clean; 4.4 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-21T08:44:38.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:38 smithi088 ceph-mon[31259]: pgmap v178: 129 pgs: 129 active+clean; 4.4 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-21T08:44:38.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:38 smithi179 ceph-mon[31829]: pgmap v178: 129 pgs: 129 active+clean; 4.4 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-21T08:44:40.844 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.844 INFO:tasks.workunit.client.0.smithi086.stdout:189/600: open fd = 3 2024-04-21T08:44:40.844 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.844 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.844 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.845 INFO:tasks.workunit.client.0.smithi086.stdout:190/600: open fd = 3 2024-04-21T08:44:40.845 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.845 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.845 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.845 INFO:tasks.workunit.client.0.smithi086.stdout:191/600: open fd = 3 2024-04-21T08:44:40.845 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.845 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.845 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.845 INFO:tasks.workunit.client.0.smithi086.stdout:192/600: open fd = 3 2024-04-21T08:44:40.845 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.845 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.845 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.845 INFO:tasks.workunit.client.0.smithi086.stdout:193/600: open fd = 3 2024-04-21T08:44:40.845 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.845 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.846 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.846 INFO:tasks.workunit.client.0.smithi086.stdout:194/600: open fd = 3 2024-04-21T08:44:40.846 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.846 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.846 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.846 INFO:tasks.workunit.client.0.smithi086.stdout:195/600: open fd = 3 2024-04-21T08:44:40.846 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.846 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.846 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.846 INFO:tasks.workunit.client.0.smithi086.stdout:196/600: open fd = 3 2024-04-21T08:44:40.846 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.846 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.846 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.847 INFO:tasks.workunit.client.0.smithi086.stdout:197/600: open fd = 3 2024-04-21T08:44:40.847 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.847 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.847 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.847 INFO:tasks.workunit.client.0.smithi086.stdout:198/600: open fd = 3 2024-04-21T08:44:40.847 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.847 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.847 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.847 INFO:tasks.workunit.client.0.smithi086.stdout:199/600: open fd = 3 2024-04-21T08:44:40.847 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.847 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.847 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.847 INFO:tasks.workunit.client.0.smithi086.stdout:200/600: open fd = 3 2024-04-21T08:44:40.847 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.847 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.847 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.848 INFO:tasks.workunit.client.0.smithi086.stdout:201/600: open fd = 3 2024-04-21T08:44:40.848 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.848 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.848 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.848 INFO:tasks.workunit.client.0.smithi086.stdout:202/600: open fd = 3 2024-04-21T08:44:40.848 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.848 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.848 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.848 INFO:tasks.workunit.client.0.smithi086.stdout:203/600: open fd = 3 2024-04-21T08:44:40.848 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.848 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.848 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.848 INFO:tasks.workunit.client.0.smithi086.stdout:204/600: open fd = 3 2024-04-21T08:44:40.848 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.849 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.849 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.849 INFO:tasks.workunit.client.0.smithi086.stdout:205/600: open fd = 3 2024-04-21T08:44:40.849 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.849 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.849 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.849 INFO:tasks.workunit.client.0.smithi086.stdout:206/600: open fd = 3 2024-04-21T08:44:40.849 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.849 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.849 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.849 INFO:tasks.workunit.client.0.smithi086.stdout:207/600: open fd = 3 2024-04-21T08:44:40.849 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.849 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.849 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.850 INFO:tasks.workunit.client.0.smithi086.stdout:208/600: open fd = 3 2024-04-21T08:44:40.850 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.850 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.850 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.850 INFO:tasks.workunit.client.0.smithi086.stdout:209/600: open fd = 3 2024-04-21T08:44:40.850 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.850 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.850 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.850 INFO:tasks.workunit.client.0.smithi086.stdout:210/600: open fd = 3 2024-04-21T08:44:40.850 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.850 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.850 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.850 INFO:tasks.workunit.client.0.smithi086.stdout:211/600: open fd = 3 2024-04-21T08:44:40.851 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.851 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.851 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.851 INFO:tasks.workunit.client.0.smithi086.stdout:212/600: open fd = 3 2024-04-21T08:44:40.851 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.851 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.851 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.851 INFO:tasks.workunit.client.0.smithi086.stdout:213/600: open fd = 3 2024-04-21T08:44:40.851 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.851 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.851 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.851 INFO:tasks.workunit.client.0.smithi086.stdout:214/600: open fd = 3 2024-04-21T08:44:40.851 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.851 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.851 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.852 INFO:tasks.workunit.client.0.smithi086.stdout:215/600: open fd = 3 2024-04-21T08:44:40.852 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.852 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.852 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.852 INFO:tasks.workunit.client.0.smithi086.stdout:216/600: open fd = 3 2024-04-21T08:44:40.852 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.852 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.852 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.852 INFO:tasks.workunit.client.0.smithi086.stdout:217/600: open fd = 3 2024-04-21T08:44:40.852 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.852 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.852 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.852 INFO:tasks.workunit.client.0.smithi086.stdout:218/600: open fd = 3 2024-04-21T08:44:40.852 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.852 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.853 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.853 INFO:tasks.workunit.client.0.smithi086.stdout:219/600: open fd = 3 2024-04-21T08:44:40.853 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.853 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.853 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.853 INFO:tasks.workunit.client.0.smithi086.stdout:220/600: open fd = 3 2024-04-21T08:44:40.853 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.853 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.853 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.853 INFO:tasks.workunit.client.0.smithi086.stdout:221/600: open fd = 3 2024-04-21T08:44:40.853 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.853 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.853 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.853 INFO:tasks.workunit.client.0.smithi086.stdout:222/600: open fd = 3 2024-04-21T08:44:40.853 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.854 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.854 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.854 INFO:tasks.workunit.client.0.smithi086.stdout:223/600: open fd = 3 2024-04-21T08:44:40.854 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.854 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.854 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.854 INFO:tasks.workunit.client.0.smithi086.stdout:224/600: open fd = 3 2024-04-21T08:44:40.854 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.854 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.854 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.854 INFO:tasks.workunit.client.0.smithi086.stdout:225/600: open fd = 3 2024-04-21T08:44:40.854 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.854 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.854 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.855 INFO:tasks.workunit.client.0.smithi086.stdout:226/600: open fd = 3 2024-04-21T08:44:40.855 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.855 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.855 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.855 INFO:tasks.workunit.client.0.smithi086.stdout:227/600: open fd = 3 2024-04-21T08:44:40.855 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.855 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.855 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.855 INFO:tasks.workunit.client.0.smithi086.stdout:228/600: open fd = 3 2024-04-21T08:44:40.855 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.855 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.855 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.855 INFO:tasks.workunit.client.0.smithi086.stdout:229/600: open fd = 3 2024-04-21T08:44:40.855 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.856 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.856 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.856 INFO:tasks.workunit.client.0.smithi086.stdout:230/600: open fd = 3 2024-04-21T08:44:40.856 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.856 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.856 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.856 INFO:tasks.workunit.client.0.smithi086.stdout:231/600: open fd = 3 2024-04-21T08:44:40.856 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.856 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.856 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.856 INFO:tasks.workunit.client.0.smithi086.stdout:232/600: open fd = 3 2024-04-21T08:44:40.856 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.856 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.856 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.856 INFO:tasks.workunit.client.0.smithi086.stdout:233/600: open fd = 3 2024-04-21T08:44:40.857 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.857 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.857 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.857 INFO:tasks.workunit.client.0.smithi086.stdout:234/600: open fd = 3 2024-04-21T08:44:40.857 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.857 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.857 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.857 INFO:tasks.workunit.client.0.smithi086.stdout:235/600: open fd = 3 2024-04-21T08:44:40.857 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.857 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.857 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.857 INFO:tasks.workunit.client.0.smithi086.stdout:236/600: open fd = 3 2024-04-21T08:44:40.857 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.857 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.857 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.858 INFO:tasks.workunit.client.0.smithi086.stdout:237/600: open fd = 3 2024-04-21T08:44:40.858 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.858 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.858 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.858 INFO:tasks.workunit.client.0.smithi086.stdout:238/600: open fd = 3 2024-04-21T08:44:40.858 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.858 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.858 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.858 INFO:tasks.workunit.client.0.smithi086.stdout:239/600: open fd = 3 2024-04-21T08:44:40.858 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.858 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.858 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.858 INFO:tasks.workunit.client.0.smithi086.stdout:240/600: open fd = 3 2024-04-21T08:44:40.858 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.858 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.859 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.859 INFO:tasks.workunit.client.0.smithi086.stdout:241/600: open fd = 3 2024-04-21T08:44:40.859 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.859 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.859 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.859 INFO:tasks.workunit.client.0.smithi086.stdout:242/600: open fd = 3 2024-04-21T08:44:40.859 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.859 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.859 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.859 INFO:tasks.workunit.client.0.smithi086.stdout:243/600: open fd = 3 2024-04-21T08:44:40.859 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.859 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.859 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.859 INFO:tasks.workunit.client.0.smithi086.stdout:244/600: open fd = 3 2024-04-21T08:44:40.859 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.860 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.860 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.860 INFO:tasks.workunit.client.0.smithi086.stdout:245/600: open fd = 3 2024-04-21T08:44:40.860 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.860 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.860 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.860 INFO:tasks.workunit.client.0.smithi086.stdout:246/600: open fd = 3 2024-04-21T08:44:40.860 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.860 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.860 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.860 INFO:tasks.workunit.client.0.smithi086.stdout:247/600: open fd = 3 2024-04-21T08:44:40.860 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.860 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.860 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.860 INFO:tasks.workunit.client.0.smithi086.stdout:248/600: open fd = 3 2024-04-21T08:44:40.861 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.861 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.861 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.861 INFO:tasks.workunit.client.0.smithi086.stdout:249/600: open fd = 3 2024-04-21T08:44:40.861 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.861 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.861 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:44:40.861 INFO:tasks.workunit.client.0.smithi086.stdout:250/600: open fd = 3 2024-04-21T08:44:40.861 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.861 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:44:40.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:40 smithi086 ceph-mon[27939]: pgmap v179: 129 pgs: 129 active+clean; 4.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 21 op/s 2024-04-21T08:44:40.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:40 smithi088 ceph-mon[31259]: pgmap v179: 129 pgs: 129 active+clean; 4.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 21 op/s 2024-04-21T08:44:40.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:40 smithi179 ceph-mon[31829]: pgmap v179: 129 pgs: 129 active+clean; 4.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 55 KiB/s wr, 21 op/s 2024-04-21T08:44:41.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:41 smithi086 ceph-mon[27939]: pgmap v180: 129 pgs: 129 active+clean; 4.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-21T08:44:41.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:41 smithi088 ceph-mon[31259]: pgmap v180: 129 pgs: 129 active+clean; 4.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-21T08:44:41.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:41 smithi179 ceph-mon[31829]: pgmap v180: 129 pgs: 129 active+clean; 4.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-21T08:44:44.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:43 smithi086 ceph-mon[27939]: pgmap v181: 129 pgs: 129 active+clean; 4.8 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-04-21T08:44:44.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:43 smithi088 ceph-mon[31259]: pgmap v181: 129 pgs: 129 active+clean; 4.8 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-04-21T08:44:44.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:43 smithi179 ceph-mon[31829]: pgmap v181: 129 pgs: 129 active+clean; 4.8 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-04-21T08:44:46.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:45 smithi086 ceph-mon[27939]: pgmap v182: 129 pgs: 129 active+clean; 4.8 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-21T08:44:46.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:45 smithi088 ceph-mon[31259]: pgmap v182: 129 pgs: 129 active+clean; 4.8 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-21T08:44:46.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:45 smithi179 ceph-mon[31829]: pgmap v182: 129 pgs: 129 active+clean; 4.8 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-21T08:44:48.002 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:44:48.002 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:44:48.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.115+0000 7f64070fd640 1 -- 172.21.15.86:0/1333880636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6400107fd0 msgr2=0x7f64001083b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.115+0000 7f64070fd640 1 --2- 172.21.15.86:0/1333880636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6400107fd0 0x7f64001083b0 secure :-1 s=READY pgs=1310 cs=0 l=1 rev1=1 crypto rx=0x7f63f4009930 tx=0x7f63f402bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:48.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.115+0000 7f64070fd640 1 -- 172.21.15.86:0/1333880636 shutdown_connections 2024-04-21T08:44:48.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.115+0000 7f64070fd640 1 --2- 172.21.15.86:0/1333880636 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6400100d60 0x7f640010ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.115+0000 7f64070fd640 1 --2- 172.21.15.86:0/1333880636 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64001003e0 0x7f6400100820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.115+0000 7f64070fd640 1 --2- 172.21.15.86:0/1333880636 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6400107fd0 0x7f64001083b0 unknown :-1 s=CLOSED pgs=1310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.115+0000 7f64070fd640 1 -- 172.21.15.86:0/1333880636 >> 172.21.15.86:0/1333880636 conn(0x7f6400077360 msgr2=0x7f64000fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:48.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.115+0000 7f64070fd640 1 -- 172.21.15.86:0/1333880636 shutdown_connections 2024-04-21T08:44:48.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.115+0000 7f64070fd640 1 -- 172.21.15.86:0/1333880636 wait complete. 2024-04-21T08:44:48.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.116+0000 7f64070fd640 1 Processor -- start 2024-04-21T08:44:48.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.116+0000 7f64070fd640 1 -- start start 2024-04-21T08:44:48.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.116+0000 7f64070fd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64001003e0 0x7f64001015c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.116+0000 7f64070fd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6400100d60 0x7f6400101b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.116+0000 7f64070fd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6400107fd0 0x7f6400102040 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.116+0000 7f64070fd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f64001121b0 con 0x7f6400107fd0 2024-04-21T08:44:48.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.116+0000 7f64070fd640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6400112030 con 0x7f64001003e0 2024-04-21T08:44:48.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.116+0000 7f64070fd640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6400112330 con 0x7f6400100d60 2024-04-21T08:44:48.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.116+0000 7f640569a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6400107fd0 0x7f6400102040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.116+0000 7f640569a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6400107fd0 0x7f6400102040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42238/0 (socket says 172.21.15.86:42238) 2024-04-21T08:44:48.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.116+0000 7f640569a640 1 -- 172.21.15.86:0/479501306 learned_addr learned my addr 172.21.15.86:0/479501306 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:48.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.116+0000 7f6404e99640 1 --2- 172.21.15.86:0/479501306 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64001003e0 0x7f64001015c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.116+0000 7f63fffff640 1 --2- 172.21.15.86:0/479501306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6400100d60 0x7f6400101b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.116+0000 7f640569a640 1 -- 172.21.15.86:0/479501306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6400100d60 msgr2=0x7f6400101b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.116+0000 7f640569a640 1 --2- 172.21.15.86:0/479501306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6400100d60 0x7f6400101b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.116+0000 7f640569a640 1 -- 172.21.15.86:0/479501306 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64001003e0 msgr2=0x7f64001015c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.116+0000 7f640569a640 1 --2- 172.21.15.86:0/479501306 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64001003e0 0x7f64001015c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.116+0000 7f640569a640 1 -- 172.21.15.86:0/479501306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64001a6cb0 con 0x7f6400107fd0 2024-04-21T08:44:48.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.117+0000 7f640569a640 1 --2- 172.21.15.86:0/479501306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6400107fd0 0x7f6400102040 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f63f800cad0 tx=0x7f63f800cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.117+0000 7f63fdffb640 1 -- 172.21.15.86:0/479501306 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f800ae20 con 0x7f6400107fd0 2024-04-21T08:44:48.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.117+0000 7f63fdffb640 1 -- 172.21.15.86:0/479501306 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63f8005020 con 0x7f6400107fd0 2024-04-21T08:44:48.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.117+0000 7f63fdffb640 1 -- 172.21.15.86:0/479501306 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f800f6b0 con 0x7f6400107fd0 2024-04-21T08:44:48.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.117+0000 7f64070fd640 1 -- 172.21.15.86:0/479501306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64001a6fa0 con 0x7f6400107fd0 2024-04-21T08:44:48.119 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:47 smithi086 ceph-mon[27939]: pgmap v183: 129 pgs: 129 active+clean; 4.8 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-21T08:44:48.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.117+0000 7f64070fd640 1 -- 172.21.15.86:0/479501306 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64001a7410 con 0x7f6400107fd0 2024-04-21T08:44:48.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.119+0000 7f63fdffb640 1 -- 172.21.15.86:0/479501306 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f63f80051d0 con 0x7f6400107fd0 2024-04-21T08:44:48.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.120+0000 7f6409dc6740 1 Processor -- start 2024-04-21T08:44:48.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.120+0000 7f6409dc6740 1 -- start start 2024-04-21T08:44:48.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.120+0000 7f6409dc6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c92aa07580 0x55c92aa07960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.120+0000 7f6409dc6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c92aa07ea0 0x55c92aa10540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.120+0000 7f63fdffb640 1 --2- 172.21.15.86:0/479501306 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f63f00a3d20 0x7f63f00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.120+0000 7f6409dc6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c92aa10a80 0x55c92aa12e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.120+0000 7f6409dc6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c92a96c1e0 con 0x55c92aa07580 2024-04-21T08:44:48.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.120+0000 7f6409dc6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c92a909390 con 0x55c92aa10a80 2024-04-21T08:44:48.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.120+0000 7f6409dc6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c92a9681a0 con 0x55c92aa07ea0 2024-04-21T08:44:48.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.120+0000 7f63fffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c92aa07580 0x55c92aa07960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.120+0000 7f63fffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c92aa07580 0x55c92aa07960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42252/0 (socket says 172.21.15.86:42252) 2024-04-21T08:44:48.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.120+0000 7f63fffff640 1 -- 172.21.15.86:0/1401026906 learned_addr learned my addr 172.21.15.86:0/1401026906 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:48.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.120+0000 7f63fffff640 1 --2- 172.21.15.86:0/479501306 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f63f00a3d20 0x7f63f00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.120+0000 7f63fdffb640 1 -- 172.21.15.86:0/479501306 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f63f80d0a10 con 0x7f6400107fd0 2024-04-21T08:44:48.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.120+0000 7f6404e99640 1 --2- 172.21.15.86:0/1401026906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c92aa07ea0 0x55c92aa10540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.120+0000 7f640569a640 1 --2- 172.21.15.86:0/1401026906 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c92aa10a80 0x55c92aa12e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.120+0000 7f63fffff640 1 -- 172.21.15.86:0/1401026906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c92aa07ea0 msgr2=0x55c92aa10540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.120+0000 7f63fffff640 1 --2- 172.21.15.86:0/1401026906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c92aa07ea0 0x55c92aa10540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.120+0000 7f63fffff640 1 -- 172.21.15.86:0/1401026906 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c92aa10a80 msgr2=0x55c92aa12e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.120+0000 7f63fffff640 1 --2- 172.21.15.86:0/1401026906 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c92aa10a80 0x55c92aa12e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.120+0000 7f63fffff640 1 -- 172.21.15.86:0/1401026906 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c92a7487d0 con 0x55c92aa07580 2024-04-21T08:44:48.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.121+0000 7f63fffff640 1 --2- 172.21.15.86:0/1401026906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c92aa07580 0x55c92aa07960 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f63ec011ab0 tx=0x7f63ec011f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2a4d416fc30dead server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.121+0000 7f63fffff640 1 --2- 172.21.15.86:0/479501306 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f63f00a3d20 0x7f63f00a61e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f63ec00f850 tx=0x7f63ec01e000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.121+0000 7f63eaffd640 1 -- 172.21.15.86:0/1401026906 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ec011050 con 0x55c92aa07580 2024-04-21T08:44:48.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.121+0000 7f63eaffd640 1 -- 172.21.15.86:0/1401026906 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63ec0111f0 con 0x55c92aa07580 2024-04-21T08:44:48.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.121+0000 7f63eaffd640 1 -- 172.21.15.86:0/1401026906 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ec01d820 con 0x55c92aa07580 2024-04-21T08:44:48.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.122+0000 7f6409dc6740 1 -- 172.21.15.86:0/1401026906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c92aa07580 msgr2=0x55c92aa07960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.122+0000 7f6409dc6740 1 --2- 172.21.15.86:0/1401026906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c92aa07580 0x55c92aa07960 secure :-1 s=READY pgs=1312 cs=0 l=1 rev1=1 crypto rx=0x7f63ec011ab0 tx=0x7f63ec011f70 comp rx=0 tx=0).stop 2024-04-21T08:44:48.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.122+0000 7f6409dc6740 1 -- 172.21.15.86:0/1401026906 shutdown_connections 2024-04-21T08:44:48.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.122+0000 7f6409dc6740 1 --2- 172.21.15.86:0/1401026906 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c92aa10a80 0x55c92aa12e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.122+0000 7f6409dc6740 1 --2- 172.21.15.86:0/1401026906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c92aa07ea0 0x55c92aa10540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.122+0000 7f6409dc6740 1 --2- 172.21.15.86:0/1401026906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c92aa07580 0x55c92aa07960 unknown :-1 s=CLOSED pgs=1312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.122+0000 7f6409dc6740 1 -- 172.21.15.86:0/1401026906 >> 172.21.15.86:0/1401026906 conn(0x55c92aa01cb0 msgr2=0x55c92aa02090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:48.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.122+0000 7f6409dc6740 1 -- 172.21.15.86:0/1401026906 shutdown_connections 2024-04-21T08:44:48.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.122+0000 7f6409dc6740 1 -- 172.21.15.86:0/1401026906 wait complete. 2024-04-21T08:44:48.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.122+0000 7f6409dc6740 1 -- start start 2024-04-21T08:44:48.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.122+0000 7f6409dc6740 1 Processor -- start 2024-04-21T08:44:48.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.122+0000 7f6409dc6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c92aa07580 0x55c92aa0db10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.122+0000 7f6409dc6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c92aa07ea0 0x55c92aa0e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.122+0000 7f6409dc6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c92aa10a80 0x55c92aa0e590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.122+0000 7f6409dc6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c92aa13810 con 0x55c92aa07ea0 2024-04-21T08:44:48.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.122+0000 7f6409dc6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c92a9252f0 con 0x55c92aa07580 2024-04-21T08:44:48.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.122+0000 7f6409dc6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c92aa13990 con 0x55c92aa10a80 2024-04-21T08:44:48.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.122+0000 7f6409dc6740 10 client.0 _ll_register_callbacks cb 0x55c92a932580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:48.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.123+0000 7f63fffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c92aa07ea0 0x55c92aa0e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.123+0000 7f63fffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c92aa07ea0 0x55c92aa0e050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42258/0 (socket says 172.21.15.86:42258) 2024-04-21T08:44:48.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.123+0000 7f63fffff640 1 -- 172.21.15.86:0/2530613199 learned_addr learned my addr 172.21.15.86:0/2530613199 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:48.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.123+0000 7f6404e99640 1 --2- 172.21.15.86:0/2530613199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c92aa07580 0x55c92aa0db10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.123+0000 7f640569a640 1 --2- 172.21.15.86:0/2530613199 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c92aa10a80 0x55c92aa0e590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.123+0000 7f63fffff640 1 -- 172.21.15.86:0/2530613199 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c92aa10a80 msgr2=0x55c92aa0e590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.123+0000 7f63fffff640 1 --2- 172.21.15.86:0/2530613199 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c92aa10a80 0x55c92aa0e590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.123+0000 7f63fffff640 1 -- 172.21.15.86:0/2530613199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c92aa07580 msgr2=0x55c92aa0db10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.123+0000 7f63fffff640 1 --2- 172.21.15.86:0/2530613199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c92aa07580 0x55c92aa0db10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.123+0000 7f63fffff640 1 -- 172.21.15.86:0/2530613199 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c92a67e3f0 con 0x55c92aa07ea0 2024-04-21T08:44:48.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.123+0000 7f6404e99640 1 --2- 172.21.15.86:0/2530613199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c92aa07580 0x55c92aa0db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:48.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.123+0000 7f640569a640 1 --2- 172.21.15.86:0/2530613199 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c92aa10a80 0x55c92aa0e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:48.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.123+0000 7f63fffff640 1 --2- 172.21.15.86:0/2530613199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c92aa07ea0 0x55c92aa0e050 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f63ec0113d0 tx=0x7f63ec011400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=36e11fa510737a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.123+0000 7f63dffff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:48.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.123+0000 7f63dffff640 1 -- 172.21.15.86:0/2530613199 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63ec035060 con 0x55c92aa07ea0 2024-04-21T08:44:48.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.123+0000 7f6409dc6740 1 -- 172.21.15.86:0/2530613199 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63ec0111f0 con 0x55c92aa07ea0 2024-04-21T08:44:48.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.123+0000 7f63dffff640 1 -- 172.21.15.86:0/2530613199 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f63ec0115e0 con 0x55c92aa07ea0 2024-04-21T08:44:48.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.123+0000 7f63dffff640 1 -- 172.21.15.86:0/2530613199 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f63ec03c050 con 0x55c92aa07ea0 2024-04-21T08:44:48.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.123+0000 7f6409dc6740 10 client.18114 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:48.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.123+0000 7f6409dc6740 1 -- 172.21.15.86:0/2530613199 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c92a63e0f0 con 0x55c92aa07ea0 2024-04-21T08:44:48.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.124+0000 7f63dffff640 1 -- 172.21.15.86:0/2530613199 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63ec0111f0 con 0x55c92aa07ea0 2024-04-21T08:44:48.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.124+0000 7f63dffff640 1 -- 172.21.15.86:0/2530613199 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f63ec004e30 con 0x55c92aa07ea0 2024-04-21T08:44:48.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.124+0000 7f6409dc6740 10 client.18114 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:48.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.124+0000 7f6409dc6740 10 client.18114 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:48.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.124+0000 7f6409dc6740 20 client.18114 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:48.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.124+0000 7f6409dc6740 1 --2- 172.21.15.86:0/2530613199 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c92aa20a70 0x55c92aa22e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.124+0000 7f6409dc6740 4 client.18114 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:48.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.124+0000 7f6409dc6740 1 -- 172.21.15.86:0/2530613199 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c92a979c00 con 0x55c92aa20a70 2024-04-21T08:44:48.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.124+0000 7f6404e99640 1 --2- 172.21.15.86:0/2530613199 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c92aa20a70 0x55c92aa22e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.124+0000 7f6404e99640 1 --2- 172.21.15.86:0/2530613199 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c92aa20a70 0x55c92aa22e60 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4b3f258d10b65de server_cookie=8ac2839ca5957a9d in_seq=0 out_seq=0 2024-04-21T08:44:48.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.124+0000 7f63dffff640 10 client.18114 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:48.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.125+0000 7f63dffff640 1 -- 172.21.15.86:0/2530613199 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c92a979c00 con 0x55c92aa20a70 2024-04-21T08:44:48.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.125+0000 7f63dffff640 10 client.18114 handle_command_reply: tid=0 2024-04-21T08:44:48.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.125+0000 7f6409dc6740 1 client.18114 shutdown 2024-04-21T08:44:48.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.125+0000 7f6409dc6740 1 -- 172.21.15.86:0/2530613199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c92aa07ea0 msgr2=0x55c92aa0e050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.125+0000 7f6409dc6740 1 --2- 172.21.15.86:0/2530613199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c92aa07ea0 0x55c92aa0e050 secure :-1 s=READY pgs=1313 cs=0 l=1 rev1=1 crypto rx=0x7f63ec0113d0 tx=0x7f63ec011400 comp rx=0 tx=0).stop 2024-04-21T08:44:48.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.125+0000 7f6409dc6740 1 -- 172.21.15.86:0/2530613199 shutdown_connections 2024-04-21T08:44:48.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.125+0000 7f6409dc6740 1 --2- 172.21.15.86:0/2530613199 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c92aa20a70 0x55c92aa22e60 crc :-1 s=READY pgs=191 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.125+0000 7f6409dc6740 1 --2- 172.21.15.86:0/2530613199 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c92aa10a80 0x55c92aa0e590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.125+0000 7f6409dc6740 1 --2- 172.21.15.86:0/2530613199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c92aa07ea0 0x55c92aa0e050 unknown :-1 s=CLOSED pgs=1313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.125+0000 7f63dffff640 0 client.18114 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:48.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.125+0000 7f6409dc6740 1 --2- 172.21.15.86:0/2530613199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c92aa07580 0x55c92aa0db10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.125+0000 7f6409dc6740 1 -- 172.21.15.86:0/2530613199 >> 172.21.15.86:0/2530613199 conn(0x55c92aa01cb0 msgr2=0x55c92aa03a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:48.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.125+0000 7f6409dc6740 1 -- 172.21.15.86:0/2530613199 shutdown_connections 2024-04-21T08:44:48.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.125+0000 7f6409dc6740 1 -- 172.21.15.86:0/2530613199 wait complete. 2024-04-21T08:44:48.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.125+0000 7f6409dc6740 20 client.18114 trim_cache size 0 max 16384 2024-04-21T08:44:48.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.143+0000 7f6409dc6740 1 Processor -- start 2024-04-21T08:44:48.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.143+0000 7f6409dc6740 1 -- start start 2024-04-21T08:44:48.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.143+0000 7f6409dc6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c92aa07580 0x55c92aa10240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.143+0000 7f6409dc6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c92aa07ea0 0x55c92aa1c6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.143+0000 7f6409dc6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c92aa10a80 0x55c92aa00d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.143+0000 7f6409dc6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f64001b1520 con 0x55c92aa10a80 2024-04-21T08:44:48.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.143+0000 7f6409dc6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f64001b3060 con 0x55c92aa07580 2024-04-21T08:44:48.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.143+0000 7f6409dc6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f64001b1bf0 con 0x55c92aa07ea0 2024-04-21T08:44:48.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.143+0000 7f640569a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c92aa10a80 0x55c92aa00d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.143+0000 7f640569a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c92aa10a80 0x55c92aa00d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42262/0 (socket says 172.21.15.86:42262) 2024-04-21T08:44:48.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.143+0000 7f640569a640 1 -- 172.21.15.86:0/2250062091 learned_addr learned my addr 172.21.15.86:0/2250062091 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:48.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.143+0000 7f63fffff640 1 --2- 172.21.15.86:0/2250062091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c92aa07ea0 0x55c92aa1c6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.143+0000 7f6404e99640 1 --2- 172.21.15.86:0/2250062091 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c92aa07580 0x55c92aa10240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.143+0000 7f640569a640 1 -- 172.21.15.86:0/2250062091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c92aa07ea0 msgr2=0x55c92aa1c6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.143+0000 7f640569a640 1 --2- 172.21.15.86:0/2250062091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c92aa07ea0 0x55c92aa1c6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.143+0000 7f640569a640 1 -- 172.21.15.86:0/2250062091 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c92aa07580 msgr2=0x55c92aa10240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.143+0000 7f640569a640 1 --2- 172.21.15.86:0/2250062091 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c92aa07580 0x55c92aa10240 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.143+0000 7f640569a640 1 -- 172.21.15.86:0/2250062091 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c92aa231d0 con 0x55c92aa10a80 2024-04-21T08:44:48.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.143+0000 7f6404e99640 1 --2- 172.21.15.86:0/2250062091 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c92aa07580 0x55c92aa10240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:48.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.143+0000 7f640569a640 1 --2- 172.21.15.86:0/2250062091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c92aa10a80 0x55c92aa00d70 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f63f809c3f0 tx=0x7f63f809c830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b13f45599cc99b5f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.143+0000 7f63ea7fc640 1 -- 172.21.15.86:0/2250062091 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f801d040 con 0x55c92aa10a80 2024-04-21T08:44:48.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.144+0000 7f63ea7fc640 1 -- 172.21.15.86:0/2250062091 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63f80a38a0 con 0x55c92aa10a80 2024-04-21T08:44:48.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.144+0000 7f63ea7fc640 1 -- 172.21.15.86:0/2250062091 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f8098ef0 con 0x55c92aa10a80 2024-04-21T08:44:48.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.144+0000 7f6409dc6740 1 -- 172.21.15.86:0/2250062091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c92aa10a80 msgr2=0x55c92aa00d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.144+0000 7f6409dc6740 1 --2- 172.21.15.86:0/2250062091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c92aa10a80 0x55c92aa00d70 secure :-1 s=READY pgs=1314 cs=0 l=1 rev1=1 crypto rx=0x7f63f809c3f0 tx=0x7f63f809c830 comp rx=0 tx=0).stop 2024-04-21T08:44:48.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.144+0000 7f6409dc6740 1 -- 172.21.15.86:0/2250062091 shutdown_connections 2024-04-21T08:44:48.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.144+0000 7f6409dc6740 1 --2- 172.21.15.86:0/2250062091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c92aa10a80 0x55c92aa00d70 unknown :-1 s=CLOSED pgs=1314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.144+0000 7f6409dc6740 1 --2- 172.21.15.86:0/2250062091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c92aa07ea0 0x55c92aa1c6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.144+0000 7f6409dc6740 1 --2- 172.21.15.86:0/2250062091 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c92aa07580 0x55c92aa10240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.144+0000 7f6409dc6740 1 -- 172.21.15.86:0/2250062091 >> 172.21.15.86:0/2250062091 conn(0x55c92aa01cb0 msgr2=0x55c92aa0ca50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:48.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.144+0000 7f6409dc6740 1 -- 172.21.15.86:0/2250062091 shutdown_connections 2024-04-21T08:44:48.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.144+0000 7f6409dc6740 1 -- 172.21.15.86:0/2250062091 wait complete. 2024-04-21T08:44:48.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.144+0000 7f6409dc6740 1 -- start start 2024-04-21T08:44:48.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.145+0000 7f6409dc6740 1 Processor -- start 2024-04-21T08:44:48.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.145+0000 7f6409dc6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c92aa07580 0x55c92aa14990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.145+0000 7f6409dc6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c92aa07ea0 0x55c92aa14ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.145+0000 7f6409dc6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c92aa10a80 0x55c92aa85f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.145+0000 7f6409dc6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f64001b2990 con 0x55c92aa07580 2024-04-21T08:44:48.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.145+0000 7f6409dc6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f64001b22c0 con 0x55c92aa07ea0 2024-04-21T08:44:48.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.145+0000 7f6409dc6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f64001af4e0 con 0x55c92aa10a80 2024-04-21T08:44:48.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.145+0000 7f6409dc6740 10 client.0 _ll_register_callbacks cb 0x7f63f8098350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:48.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.145+0000 7f6404e99640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c92aa07580 0x55c92aa14990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.145+0000 7f6404e99640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c92aa07580 0x55c92aa14990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42274/0 (socket says 172.21.15.86:42274) 2024-04-21T08:44:48.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.145+0000 7f6404e99640 1 -- 172.21.15.86:0/1396746264 learned_addr learned my addr 172.21.15.86:0/1396746264 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:48.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.145+0000 7f63fffff640 1 --2- 172.21.15.86:0/1396746264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c92aa07ea0 0x55c92aa14ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.145+0000 7f6404e99640 1 -- 172.21.15.86:0/1396746264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c92aa10a80 msgr2=0x55c92aa85f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.145+0000 7f6404e99640 1 --2- 172.21.15.86:0/1396746264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c92aa10a80 0x55c92aa85f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.145+0000 7f6404e99640 1 -- 172.21.15.86:0/1396746264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c92aa07ea0 msgr2=0x55c92aa14ed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.145+0000 7f6404e99640 1 --2- 172.21.15.86:0/1396746264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c92aa07ea0 0x55c92aa14ed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.145+0000 7f6404e99640 1 -- 172.21.15.86:0/1396746264 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c92aa86470 con 0x55c92aa07580 2024-04-21T08:44:48.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.145+0000 7f6404e99640 1 --2- 172.21.15.86:0/1396746264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c92aa07580 0x55c92aa14990 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f63f4005ab0 tx=0x7f63f40042e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a2c54a84022bca4f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.145+0000 7f63dffff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:48.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.146+0000 7f63dffff640 1 -- 172.21.15.86:0/1396746264 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f63f404abd0 con 0x55c92aa07580 2024-04-21T08:44:48.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.146+0000 7f63dffff640 1 -- 172.21.15.86:0/1396746264 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f63f403a040 con 0x55c92aa07580 2024-04-21T08:44:48.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.146+0000 7f6409dc6740 1 -- 172.21.15.86:0/1396746264 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f63f80a38a0 con 0x55c92aa07580 2024-04-21T08:44:48.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.146+0000 7f63dffff640 1 -- 172.21.15.86:0/1396746264 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f63f4057050 con 0x55c92aa07580 2024-04-21T08:44:48.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.146+0000 7f6409dc6740 10 client.18120 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:48.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.146+0000 7f6409dc6740 1 -- 172.21.15.86:0/1396746264 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c92aa87520 con 0x55c92aa07580 2024-04-21T08:44:48.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.146+0000 7f63dffff640 1 -- 172.21.15.86:0/1396746264 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f63f80a38a0 con 0x55c92aa07580 2024-04-21T08:44:48.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.146+0000 7f63dffff640 1 -- 172.21.15.86:0/1396746264 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f63f4005040 con 0x55c92aa07580 2024-04-21T08:44:48.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.146+0000 7f6409dc6740 10 client.18120 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:48.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.146+0000 7f6409dc6740 10 client.18120 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:48.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.146+0000 7f6409dc6740 20 client.18120 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:48.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.146+0000 7f6409dc6740 1 --2- 172.21.15.86:0/1396746264 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c92aa87ca0 0x55c92aa8a0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.146+0000 7f6409dc6740 4 client.18120 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:44:48.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.146+0000 7f6409dc6740 1 -- 172.21.15.86:0/1396746264 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c92aa09710 con 0x55c92aa87ca0 2024-04-21T08:44:48.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.147+0000 7f63fffff640 1 --2- 172.21.15.86:0/1396746264 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c92aa87ca0 0x55c92aa8a0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.147+0000 7f63fffff640 1 --2- 172.21.15.86:0/1396746264 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c92aa87ca0 0x55c92aa8a0c0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ed612527d41bfb31 server_cookie=4f8a12cf258ab505 in_seq=0 out_seq=0 2024-04-21T08:44:48.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.147+0000 7f63dffff640 10 client.18120 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:48.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.147+0000 7f63dffff640 1 -- 172.21.15.86:0/1396746264 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c92aa09710 con 0x55c92aa87ca0 2024-04-21T08:44:48.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.147+0000 7f63dffff640 10 client.18120 handle_command_reply: tid=0 2024-04-21T08:44:48.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.147+0000 7f6409dc6740 1 client.18120 shutdown 2024-04-21T08:44:48.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.147+0000 7f6409dc6740 1 -- 172.21.15.86:0/1396746264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c92aa07580 msgr2=0x55c92aa14990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.147+0000 7f6409dc6740 1 --2- 172.21.15.86:0/1396746264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c92aa07580 0x55c92aa14990 secure :-1 s=READY pgs=1315 cs=0 l=1 rev1=1 crypto rx=0x7f63f4005ab0 tx=0x7f63f40042e0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.147+0000 7f6409dc6740 1 -- 172.21.15.86:0/1396746264 shutdown_connections 2024-04-21T08:44:48.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.147+0000 7f6409dc6740 1 --2- 172.21.15.86:0/1396746264 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c92aa87ca0 0x55c92aa8a0c0 crc :-1 s=READY pgs=192 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.147+0000 7f6409dc6740 1 --2- 172.21.15.86:0/1396746264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c92aa10a80 0x55c92aa85f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.147+0000 7f63dffff640 0 client.18120 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:48.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.147+0000 7f6409dc6740 1 --2- 172.21.15.86:0/1396746264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c92aa07ea0 0x55c92aa14ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.147+0000 7f6409dc6740 1 --2- 172.21.15.86:0/1396746264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c92aa07580 0x55c92aa14990 unknown :-1 s=CLOSED pgs=1315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.147+0000 7f6409dc6740 1 -- 172.21.15.86:0/1396746264 >> 172.21.15.86:0/1396746264 conn(0x55c92aa01cb0 msgr2=0x55c92aa03840 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:48.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.147+0000 7f6409dc6740 1 -- 172.21.15.86:0/1396746264 shutdown_connections 2024-04-21T08:44:48.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.147+0000 7f6409dc6740 1 -- 172.21.15.86:0/1396746264 wait complete. 2024-04-21T08:44:48.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.147+0000 7f6409dc6740 20 client.18120 trim_cache size 0 max 16384 2024-04-21T08:44:48.153 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:44:48.153 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-21T08:44:48.153 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-21T08:44:48.154 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:44:48.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.148+0000 7f64070fd640 1 -- 172.21.15.86:0/479501306 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f63f00a3d20 msgr2=0x7f63f00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.148+0000 7f64070fd640 1 --2- 172.21.15.86:0/479501306 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f63f00a3d20 0x7f63f00a61e0 secure :-1 s=READY pgs=207 cs=0 l=1 rev1=1 crypto rx=0x7f63ec00f850 tx=0x7f63ec01e000 comp rx=0 tx=0).stop 2024-04-21T08:44:48.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.148+0000 7f64070fd640 1 -- 172.21.15.86:0/479501306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6400107fd0 msgr2=0x7f6400102040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.148+0000 7f64070fd640 1 --2- 172.21.15.86:0/479501306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6400107fd0 0x7f6400102040 secure :-1 s=READY pgs=1311 cs=0 l=1 rev1=1 crypto rx=0x7f63f800cad0 tx=0x7f63f800cf90 comp rx=0 tx=0).stop 2024-04-21T08:44:48.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.148+0000 7f64070fd640 1 -- 172.21.15.86:0/479501306 shutdown_connections 2024-04-21T08:44:48.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.148+0000 7f64070fd640 1 --2- 172.21.15.86:0/479501306 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f63f00a3d20 0x7f63f00a61e0 unknown :-1 s=CLOSED pgs=207 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.148+0000 7f64070fd640 1 --2- 172.21.15.86:0/479501306 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6400107fd0 0x7f6400102040 unknown :-1 s=CLOSED pgs=1311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.148+0000 7f64070fd640 1 --2- 172.21.15.86:0/479501306 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6400100d60 0x7f6400101b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.148+0000 7f64070fd640 1 --2- 172.21.15.86:0/479501306 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64001003e0 0x7f64001015c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.148+0000 7f64070fd640 1 -- 172.21.15.86:0/479501306 >> 172.21.15.86:0/479501306 conn(0x7f6400077360 msgr2=0x7f64000fec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:48.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.148+0000 7f64070fd640 1 -- 172.21.15.86:0/479501306 shutdown_connections 2024-04-21T08:44:48.155 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.148+0000 7f64070fd640 1 -- 172.21.15.86:0/479501306 wait complete. 2024-04-21T08:44:48.158 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:44:48.158 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:44:48.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:47 smithi088 ceph-mon[31259]: pgmap v183: 129 pgs: 129 active+clean; 4.8 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-21T08:44:48.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:47 smithi179 ceph-mon[31829]: pgmap v183: 129 pgs: 129 active+clean; 4.8 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 39 KiB/s wr, 15 op/s 2024-04-21T08:44:48.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.256+0000 7f1e4a9e2640 1 -- 172.21.15.86:0/3386889953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e44101ad0 msgr2=0x7f1e4410dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.256+0000 7f1e4a9e2640 1 --2- 172.21.15.86:0/3386889953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e44101ad0 0x7f1e4410dcb0 secure :-1 s=READY pgs=1316 cs=0 l=1 rev1=1 crypto rx=0x7f1e3c009ca0 tx=0x7f1e3c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.257+0000 7f1e4a9e2640 1 -- 172.21.15.86:0/3386889953 shutdown_connections 2024-04-21T08:44:48.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.257+0000 7f1e4a9e2640 1 --2- 172.21.15.86:0/3386889953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e44101ad0 0x7f1e4410dcb0 unknown :-1 s=CLOSED pgs=1316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.257+0000 7f1e4a9e2640 1 --2- 172.21.15.86:0/3386889953 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e44101130 0x7f1e44101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.257+0000 7f1e4a9e2640 1 --2- 172.21.15.86:0/3386889953 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e44106f10 0x7f1e441072f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.257+0000 7f1e4a9e2640 1 -- 172.21.15.86:0/3386889953 >> 172.21.15.86:0/3386889953 conn(0x7f1e44077e30 msgr2=0x7f1e44078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:48.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.257+0000 7f1e4a9e2640 1 -- 172.21.15.86:0/3386889953 shutdown_connections 2024-04-21T08:44:48.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.257+0000 7f1e4a9e2640 1 -- 172.21.15.86:0/3386889953 wait complete. 2024-04-21T08:44:48.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.257+0000 7f1e4a9e2640 1 Processor -- start 2024-04-21T08:44:48.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.257+0000 7f1e4a9e2640 1 -- start start 2024-04-21T08:44:48.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.257+0000 7f1e4a9e2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e44101130 0x7f1e44198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.257+0000 7f1e4a9e2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e44101ad0 0x7f1e44198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.257+0000 7f1e4a9e2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e44106f10 0x7f1e4419cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.257+0000 7f1e4a9e2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1e441110d0 con 0x7f1e44101ad0 2024-04-21T08:44:48.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.257+0000 7f1e4a9e2640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1e44110f50 con 0x7f1e44101130 2024-04-21T08:44:48.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.257+0000 7f1e4a9e2640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1e44111250 con 0x7f1e44106f10 2024-04-21T08:44:48.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.258+0000 7f1e437fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e44101ad0 0x7f1e44198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.258+0000 7f1e437fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e44101ad0 0x7f1e44198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42292/0 (socket says 172.21.15.86:42292) 2024-04-21T08:44:48.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.258+0000 7f1e437fe640 1 -- 172.21.15.86:0/1903634077 learned_addr learned my addr 172.21.15.86:0/1903634077 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:48.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.258+0000 7f1e43fff640 1 --2- 172.21.15.86:0/1903634077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e44101130 0x7f1e44198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.258+0000 7f1e437fe640 1 -- 172.21.15.86:0/1903634077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e44106f10 msgr2=0x7f1e4419cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.258+0000 7f1e437fe640 1 --2- 172.21.15.86:0/1903634077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e44106f10 0x7f1e4419cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.258+0000 7f1e437fe640 1 -- 172.21.15.86:0/1903634077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e44101130 msgr2=0x7f1e44198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.258+0000 7f1e437fe640 1 --2- 172.21.15.86:0/1903634077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e44101130 0x7f1e44198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.258+0000 7f1e437fe640 1 -- 172.21.15.86:0/1903634077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1e4419d6d0 con 0x7f1e44101ad0 2024-04-21T08:44:48.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.258+0000 7f1e437fe640 1 --2- 172.21.15.86:0/1903634077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e44101ad0 0x7f1e44198c70 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800d950 tx=0x7f1e3800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.258+0000 7f1e417fa640 1 -- 172.21.15.86:0/1903634077 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e38017070 con 0x7f1e44101ad0 2024-04-21T08:44:48.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.258+0000 7f1e417fa640 1 -- 172.21.15.86:0/1903634077 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1e38009c30 con 0x7f1e44101ad0 2024-04-21T08:44:48.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.258+0000 7f1e4a9e2640 1 -- 172.21.15.86:0/1903634077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1e4419d9c0 con 0x7f1e44101ad0 2024-04-21T08:44:48.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.258+0000 7f1e417fa640 1 -- 172.21.15.86:0/1903634077 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1e3800f620 con 0x7f1e44101ad0 2024-04-21T08:44:48.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.258+0000 7f1e4a9e2640 1 -- 172.21.15.86:0/1903634077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1e441a5250 con 0x7f1e44101ad0 2024-04-21T08:44:48.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.259+0000 7f1e4a9e2640 1 -- 172.21.15.86:0/1903634077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1e44069230 con 0x7f1e44101ad0 2024-04-21T08:44:48.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.260+0000 7f1e417fa640 1 -- 172.21.15.86:0/1903634077 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1e38010460 con 0x7f1e44101ad0 2024-04-21T08:44:48.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.261+0000 7f1e417fa640 1 --2- 172.21.15.86:0/1903634077 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1e2c0a3d20 0x7f1e2c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.261+0000 7f1e43fff640 1 --2- 172.21.15.86:0/1903634077 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1e2c0a3d20 0x7f1e2c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.261+0000 7f1e417fa640 1 -- 172.21.15.86:0/1903634077 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f1e380d0880 con 0x7f1e44101ad0 2024-04-21T08:44:48.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.261+0000 7f1e43fff640 1 --2- 172.21.15.86:0/1903634077 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1e2c0a3d20 0x7f1e2c0a61e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1e34005e00 tx=0x7f1e34005c50 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.266+0000 7f1e417fa640 1 -- 172.21.15.86:0/1903634077 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1e3809cbb0 con 0x7f1e44101ad0 2024-04-21T08:44:48.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.482+0000 7f1e4a9e2640 1 -- 172.21.15.86:0/1903634077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1e441a54f0 con 0x7f1e44101ad0 2024-04-21T08:44:48.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.483+0000 7f1e417fa640 1 -- 172.21.15.86:0/1903634077 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7f1e38009de0 con 0x7f1e44101ad0 2024-04-21T08:44:48.482 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:44:48.482 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:38:45.202126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24398,24394,24455,24472]},"id":1}]} 2024-04-21T08:44:48.482 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-21T08:44:48.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.485+0000 7f1e4a9e2640 1 -- 172.21.15.86:0/1903634077 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1e2c0a3d20 msgr2=0x7f1e2c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.485+0000 7f1e4a9e2640 1 --2- 172.21.15.86:0/1903634077 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1e2c0a3d20 0x7f1e2c0a61e0 secure :-1 s=READY pgs=208 cs=0 l=1 rev1=1 crypto rx=0x7f1e34005e00 tx=0x7f1e34005c50 comp rx=0 tx=0).stop 2024-04-21T08:44:48.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.485+0000 7f1e4a9e2640 1 -- 172.21.15.86:0/1903634077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e44101ad0 msgr2=0x7f1e44198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.485+0000 7f1e4a9e2640 1 --2- 172.21.15.86:0/1903634077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e44101ad0 0x7f1e44198c70 secure :-1 s=READY pgs=1317 cs=0 l=1 rev1=1 crypto rx=0x7f1e3800d950 tx=0x7f1e3800de10 comp rx=0 tx=0).stop 2024-04-21T08:44:48.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.485+0000 7f1e4a9e2640 1 -- 172.21.15.86:0/1903634077 shutdown_connections 2024-04-21T08:44:48.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.485+0000 7f1e4a9e2640 1 --2- 172.21.15.86:0/1903634077 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1e2c0a3d20 0x7f1e2c0a61e0 unknown :-1 s=CLOSED pgs=208 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.485+0000 7f1e4a9e2640 1 --2- 172.21.15.86:0/1903634077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1e44106f10 0x7f1e4419cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.485+0000 7f1e4a9e2640 1 --2- 172.21.15.86:0/1903634077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1e44101ad0 0x7f1e44198c70 unknown :-1 s=CLOSED pgs=1317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.485+0000 7f1e4a9e2640 1 --2- 172.21.15.86:0/1903634077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1e44101130 0x7f1e44198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.485+0000 7f1e4a9e2640 1 -- 172.21.15.86:0/1903634077 >> 172.21.15.86:0/1903634077 conn(0x7f1e44077e30 msgr2=0x7f1e4406a7f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:48.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.485+0000 7f1e4a9e2640 1 -- 172.21.15.86:0/1903634077 shutdown_connections 2024-04-21T08:44:48.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.485+0000 7f1e4a9e2640 1 -- 172.21.15.86:0/1903634077 wait complete. 2024-04-21T08:44:48.495 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:44:48.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.596+0000 7f4b4d71e640 1 -- 172.21.15.86:0/3420192707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b48105ea0 msgr2=0x7f4b48106280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.596+0000 7f4b4d71e640 1 --2- 172.21.15.86:0/3420192707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b48105ea0 0x7f4b48106280 secure :-1 s=READY pgs=1318 cs=0 l=1 rev1=1 crypto rx=0x7f4b40009ca0 tx=0x7f4b4002c140 comp rx=0 tx=0).stop 2024-04-21T08:44:48.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.596+0000 7f4b4d71e640 1 -- 172.21.15.86:0/3420192707 shutdown_connections 2024-04-21T08:44:48.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.596+0000 7f4b4d71e640 1 --2- 172.21.15.86:0/3420192707 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b48101ad0 0x7f4b4810dcf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.596+0000 7f4b4d71e640 1 --2- 172.21.15.86:0/3420192707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b48101130 0x7f4b48101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.596+0000 7f4b4d71e640 1 --2- 172.21.15.86:0/3420192707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b48105ea0 0x7f4b48106280 unknown :-1 s=CLOSED pgs=1318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.596+0000 7f4b4d71e640 1 -- 172.21.15.86:0/3420192707 >> 172.21.15.86:0/3420192707 conn(0x7f4b48077e30 msgr2=0x7f4b48078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:48.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.596+0000 7f4b4d71e640 1 -- 172.21.15.86:0/3420192707 shutdown_connections 2024-04-21T08:44:48.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.596+0000 7f4b4d71e640 1 -- 172.21.15.86:0/3420192707 wait complete. 2024-04-21T08:44:48.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.596+0000 7f4b4d71e640 1 Processor -- start 2024-04-21T08:44:48.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.597+0000 7f4b4d71e640 1 -- start start 2024-04-21T08:44:48.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.597+0000 7f4b4d71e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b48101130 0x7f4b4819e460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.597+0000 7f4b4d71e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b48101ad0 0x7f4b4819e9a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.597+0000 7f4b4d71e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b48105ea0 0x7f4b48198660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.597+0000 7f4b4d71e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b481111d0 con 0x7f4b48101130 2024-04-21T08:44:48.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.597+0000 7f4b4d71e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4b48111050 con 0x7f4b48105ea0 2024-04-21T08:44:48.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.597+0000 7f4b4d71e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4b48111350 con 0x7f4b48101ad0 2024-04-21T08:44:48.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.597+0000 7f4b46ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b48101130 0x7f4b4819e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.597+0000 7f4b46ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b48101130 0x7f4b4819e460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42320/0 (socket says 172.21.15.86:42320) 2024-04-21T08:44:48.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.597+0000 7f4b46ffd640 1 -- 172.21.15.86:0/2260442632 learned_addr learned my addr 172.21.15.86:0/2260442632 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:48.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.597+0000 7f4b477fe640 1 --2- 172.21.15.86:0/2260442632 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b48105ea0 0x7f4b48198660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.597+0000 7f4b467fc640 1 --2- 172.21.15.86:0/2260442632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b48101ad0 0x7f4b4819e9a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.597+0000 7f4b46ffd640 1 -- 172.21.15.86:0/2260442632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b48101ad0 msgr2=0x7f4b4819e9a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.597+0000 7f4b46ffd640 1 --2- 172.21.15.86:0/2260442632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b48101ad0 0x7f4b4819e9a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.597+0000 7f4b46ffd640 1 -- 172.21.15.86:0/2260442632 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b48105ea0 msgr2=0x7f4b48198660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.597+0000 7f4b46ffd640 1 --2- 172.21.15.86:0/2260442632 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b48105ea0 0x7f4b48198660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.597+0000 7f4b46ffd640 1 -- 172.21.15.86:0/2260442632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b48198de0 con 0x7f4b48101130 2024-04-21T08:44:48.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.597+0000 7f4b477fe640 1 --2- 172.21.15.86:0/2260442632 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b48105ea0 0x7f4b48198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:48.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.597+0000 7f4b46ffd640 1 --2- 172.21.15.86:0/2260442632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b48101130 0x7f4b4819e460 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f4b40005e30 tx=0x7f4b40035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.597+0000 7f4b2ffff640 1 -- 172.21.15.86:0/2260442632 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b40039040 con 0x7f4b48101130 2024-04-21T08:44:48.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.598+0000 7f4b2ffff640 1 -- 172.21.15.86:0/2260442632 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b40044070 con 0x7f4b48101130 2024-04-21T08:44:48.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.598+0000 7f4b4d71e640 1 -- 172.21.15.86:0/2260442632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b48199070 con 0x7f4b48101130 2024-04-21T08:44:48.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.598+0000 7f4b2ffff640 1 -- 172.21.15.86:0/2260442632 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b40004330 con 0x7f4b48101130 2024-04-21T08:44:48.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.598+0000 7f4b4d71e640 1 -- 172.21.15.86:0/2260442632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b48102ae0 con 0x7f4b48101130 2024-04-21T08:44:48.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.600+0000 7f4b2ffff640 1 -- 172.21.15.86:0/2260442632 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4b40003970 con 0x7f4b48101130 2024-04-21T08:44:48.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.600+0000 7f4b503df740 1 Processor -- start 2024-04-21T08:44:48.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.600+0000 7f4b2ffff640 1 --2- 172.21.15.86:0/2260442632 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4b300a3d20 0x7f4b300a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.600+0000 7f4b503df740 1 -- start start 2024-04-21T08:44:48.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.600+0000 7f4b477fe640 1 --2- 172.21.15.86:0/2260442632 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4b300a3d20 0x7f4b300a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.600+0000 7f4b503df740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558d454bc4d0 0x558d454bc8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.600+0000 7f4b2ffff640 1 -- 172.21.15.86:0/2260442632 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f4b400f3160 con 0x7f4b48101130 2024-04-21T08:44:48.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.600+0000 7f4b503df740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d454bce80 0x558d454c5430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.600+0000 7f4b503df740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558d454c5a00 0x558d454c7e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.600+0000 7f4b503df740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558d454211f0 con 0x558d454bce80 2024-04-21T08:44:48.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.600+0000 7f4b503df740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x558d453be4b0 con 0x558d454bc4d0 2024-04-21T08:44:48.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.600+0000 7f4b503df740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558d4541d1b0 con 0x558d454c5a00 2024-04-21T08:44:48.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.600+0000 7f4b467fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d454bce80 0x558d454c5430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.600+0000 7f4b467fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d454bce80 0x558d454c5430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42336/0 (socket says 172.21.15.86:42336) 2024-04-21T08:44:48.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.600+0000 7f4b467fc640 1 -- 172.21.15.86:0/1525054822 learned_addr learned my addr 172.21.15.86:0/1525054822 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:48.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.601+0000 7f4b46ffd640 1 --2- 172.21.15.86:0/1525054822 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558d454bc4d0 0x558d454bc8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.601+0000 7f4b477fe640 1 --2- 172.21.15.86:0/1525054822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558d454c5a00 0x558d454c7e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.601+0000 7f4b467fc640 1 -- 172.21.15.86:0/1525054822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558d454c5a00 msgr2=0x558d454c7e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.601+0000 7f4b467fc640 1 --2- 172.21.15.86:0/1525054822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558d454c5a00 0x558d454c7e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.601+0000 7f4b467fc640 1 -- 172.21.15.86:0/1525054822 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558d454bc4d0 msgr2=0x558d454bc8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.601+0000 7f4b467fc640 1 --2- 172.21.15.86:0/1525054822 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558d454bc4d0 0x558d454bc8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.601+0000 7f4b467fc640 1 -- 172.21.15.86:0/1525054822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d453b01b0 con 0x558d454bce80 2024-04-21T08:44:48.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.601+0000 7f4b477fe640 1 --2- 172.21.15.86:0/1525054822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558d454c5a00 0x558d454c7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:48.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.601+0000 7f4b477fe640 1 --2- 172.21.15.86:0/2260442632 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4b300a3d20 0x7f4b300a61e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4b38005fd0 tx=0x7f4b3800f360 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.601+0000 7f4b467fc640 1 --2- 172.21.15.86:0/1525054822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d454bce80 0x558d454c5430 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c00cef0 tx=0x7f4b3c007590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=58fc03ff2ee6483b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.601+0000 7f4b2d7fa640 1 -- 172.21.15.86:0/1525054822 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c005220 con 0x558d454bce80 2024-04-21T08:44:48.611 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.601+0000 7f4b2d7fa640 1 -- 172.21.15.86:0/1525054822 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b3c004c50 con 0x558d454bce80 2024-04-21T08:44:48.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.601+0000 7f4b2d7fa640 1 -- 172.21.15.86:0/1525054822 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c018950 con 0x558d454bce80 2024-04-21T08:44:48.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.602+0000 7f4b503df740 1 -- 172.21.15.86:0/1525054822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d454bce80 msgr2=0x558d454c5430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.602+0000 7f4b503df740 1 --2- 172.21.15.86:0/1525054822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d454bce80 0x558d454c5430 secure :-1 s=READY pgs=1320 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c00cef0 tx=0x7f4b3c007590 comp rx=0 tx=0).stop 2024-04-21T08:44:48.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.602+0000 7f4b503df740 1 -- 172.21.15.86:0/1525054822 shutdown_connections 2024-04-21T08:44:48.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.602+0000 7f4b503df740 1 --2- 172.21.15.86:0/1525054822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558d454c5a00 0x558d454c7e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.602+0000 7f4b503df740 1 --2- 172.21.15.86:0/1525054822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d454bce80 0x558d454c5430 unknown :-1 s=CLOSED pgs=1320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.602+0000 7f4b503df740 1 --2- 172.21.15.86:0/1525054822 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558d454bc4d0 0x558d454bc8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.602+0000 7f4b503df740 1 -- 172.21.15.86:0/1525054822 >> 172.21.15.86:0/1525054822 conn(0x558d454b6ba0 msgr2=0x558d454b6f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:48.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.602+0000 7f4b503df740 1 -- 172.21.15.86:0/1525054822 shutdown_connections 2024-04-21T08:44:48.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.602+0000 7f4b503df740 1 -- 172.21.15.86:0/1525054822 wait complete. 2024-04-21T08:44:48.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.602+0000 7f4b503df740 1 -- start start 2024-04-21T08:44:48.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.602+0000 7f4b503df740 1 Processor -- start 2024-04-21T08:44:48.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.602+0000 7f4b503df740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558d454bc4d0 0x558d454c5300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.602+0000 7f4b503df740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d454bce80 0x558d454c2690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.602+0000 7f4b503df740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558d454c5a00 0x558d454c2c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.602+0000 7f4b503df740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558d454c8970 con 0x558d454bce80 2024-04-21T08:44:48.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.602+0000 7f4b503df740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x558d453da500 con 0x558d454c5a00 2024-04-21T08:44:48.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.602+0000 7f4b503df740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558d454c8af0 con 0x558d454bc4d0 2024-04-21T08:44:48.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.602+0000 7f4b503df740 10 client.0 _ll_register_callbacks cb 0x558d453a8b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:48.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.602+0000 7f4b467fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d454bce80 0x558d454c2690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.603+0000 7f4b467fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d454bce80 0x558d454c2690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42344/0 (socket says 172.21.15.86:42344) 2024-04-21T08:44:48.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.603+0000 7f4b467fc640 1 -- 172.21.15.86:0/1880300362 learned_addr learned my addr 172.21.15.86:0/1880300362 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:48.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.603+0000 7f4b46ffd640 1 --2- 172.21.15.86:0/1880300362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558d454bc4d0 0x558d454c5300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.603+0000 7f4b477fe640 1 --2- 172.21.15.86:0/1880300362 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558d454c5a00 0x558d454c2c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.603+0000 7f4b467fc640 1 -- 172.21.15.86:0/1880300362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558d454bc4d0 msgr2=0x558d454c5300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.603+0000 7f4b467fc640 1 --2- 172.21.15.86:0/1880300362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558d454bc4d0 0x558d454c5300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.603+0000 7f4b467fc640 1 -- 172.21.15.86:0/1880300362 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558d454c5a00 msgr2=0x558d454c2c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.603+0000 7f4b467fc640 1 --2- 172.21.15.86:0/1880300362 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558d454c5a00 0x558d454c2c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.603+0000 7f4b467fc640 1 -- 172.21.15.86:0/1880300362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d451333f0 con 0x558d454bce80 2024-04-21T08:44:48.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.603+0000 7f4b477fe640 1 --2- 172.21.15.86:0/1880300362 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558d454c5a00 0x558d454c2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:48.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.603+0000 7f4b467fc640 1 --2- 172.21.15.86:0/1880300362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d454bce80 0x558d454c2690 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c01a040 tx=0x7f4b3c018980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f5d14ced28e19370 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.603+0000 7f4b2aff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:48.614 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.603+0000 7f4b2aff5640 1 -- 172.21.15.86:0/1880300362 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b3c018d80 con 0x558d454bce80 2024-04-21T08:44:48.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.603+0000 7f4b2aff5640 1 -- 172.21.15.86:0/1880300362 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f4b3c004050 con 0x558d454bce80 2024-04-21T08:44:48.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.603+0000 7f4b503df740 1 -- 172.21.15.86:0/1880300362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b3c004c50 con 0x558d454bce80 2024-04-21T08:44:48.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.603+0000 7f4b2aff5640 1 -- 172.21.15.86:0/1880300362 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4b3c02f050 con 0x558d454bce80 2024-04-21T08:44:48.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.603+0000 7f4b503df740 10 client.18138 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:48.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.603+0000 7f4b503df740 1 -- 172.21.15.86:0/1880300362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558d451fd7d0 con 0x558d454bce80 2024-04-21T08:44:48.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.603+0000 7f4b2aff5640 1 -- 172.21.15.86:0/1880300362 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b3c004c50 con 0x558d454bce80 2024-04-21T08:44:48.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.604+0000 7f4b2aff5640 1 -- 172.21.15.86:0/1880300362 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4b3c017070 con 0x558d454bce80 2024-04-21T08:44:48.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.604+0000 7f4b503df740 10 client.18138 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:48.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.604+0000 7f4b503df740 10 client.18138 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:44:48.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.604+0000 7f4b503df740 20 client.18138 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:48.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.604+0000 7f4b503df740 1 --2- 172.21.15.86:0/1880300362 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x558d454d5b00 0x558d454d7ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.604+0000 7f4b503df740 4 client.18138 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:48.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.604+0000 7f4b503df740 1 -- 172.21.15.86:0/1880300362 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558d453c0930 con 0x558d454d5b00 2024-04-21T08:44:48.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.604+0000 7f4b46ffd640 1 --2- 172.21.15.86:0/1880300362 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x558d454d5b00 0x558d454d7ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.604+0000 7f4b46ffd640 1 --2- 172.21.15.86:0/1880300362 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x558d454d5b00 0x558d454d7ef0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=36f5432ee4473442 server_cookie=ada01f18a03545de in_seq=0 out_seq=0 2024-04-21T08:44:48.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.604+0000 7f4b2aff5640 10 client.18138 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:44:48.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.605+0000 7f4b2aff5640 1 -- 172.21.15.86:0/1880300362 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558d453c0930 con 0x558d454d5b00 2024-04-21T08:44:48.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.605+0000 7f4b2aff5640 10 client.18138 handle_command_reply: tid=0 2024-04-21T08:44:48.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.605+0000 7f4b503df740 1 client.18138 shutdown 2024-04-21T08:44:48.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.605+0000 7f4b503df740 1 -- 172.21.15.86:0/1880300362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d454bce80 msgr2=0x558d454c2690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.605+0000 7f4b503df740 1 --2- 172.21.15.86:0/1880300362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d454bce80 0x558d454c2690 secure :-1 s=READY pgs=1321 cs=0 l=1 rev1=1 crypto rx=0x7f4b3c01a040 tx=0x7f4b3c018980 comp rx=0 tx=0).stop 2024-04-21T08:44:48.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.605+0000 7f4b503df740 1 -- 172.21.15.86:0/1880300362 shutdown_connections 2024-04-21T08:44:48.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.605+0000 7f4b503df740 1 --2- 172.21.15.86:0/1880300362 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x558d454d5b00 0x558d454d7ef0 crc :-1 s=READY pgs=54 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.605+0000 7f4b503df740 1 --2- 172.21.15.86:0/1880300362 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558d454c5a00 0x558d454c2c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.605+0000 7f4b503df740 1 --2- 172.21.15.86:0/1880300362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d454bce80 0x558d454c2690 unknown :-1 s=CLOSED pgs=1321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.605+0000 7f4b2aff5640 0 client.18138 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:44:48.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.605+0000 7f4b503df740 1 --2- 172.21.15.86:0/1880300362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558d454bc4d0 0x558d454c5300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.605+0000 7f4b503df740 1 -- 172.21.15.86:0/1880300362 >> 172.21.15.86:0/1880300362 conn(0x558d454b6ba0 msgr2=0x558d454b88f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:48.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.605+0000 7f4b503df740 1 -- 172.21.15.86:0/1880300362 shutdown_connections 2024-04-21T08:44:48.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.605+0000 7f4b503df740 1 -- 172.21.15.86:0/1880300362 wait complete. 2024-04-21T08:44:48.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.605+0000 7f4b503df740 20 client.18138 trim_cache size 0 max 16384 2024-04-21T08:44:48.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.618+0000 7f4b503df740 1 Processor -- start 2024-04-21T08:44:48.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.618+0000 7f4b503df740 1 -- start start 2024-04-21T08:44:48.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.619+0000 7f4b503df740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d454bc4d0 0x558d454d1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.619+0000 7f4b503df740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558d454bce80 0x558d454b8aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.619+0000 7f4b503df740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558d454c5a00 0x558d454c4dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.619+0000 7f4b503df740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b481acf30 con 0x558d454bc4d0 2024-04-21T08:44:48.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.619+0000 7f4b503df740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4b481aea70 con 0x558d454c5a00 2024-04-21T08:44:48.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.619+0000 7f4b503df740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4b481ad600 con 0x558d454bce80 2024-04-21T08:44:48.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.619+0000 7f4b46ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d454bc4d0 0x558d454d1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.619+0000 7f4b46ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d454bc4d0 0x558d454d1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42352/0 (socket says 172.21.15.86:42352) 2024-04-21T08:44:48.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.619+0000 7f4b46ffd640 1 -- 172.21.15.86:0/2735836408 learned_addr learned my addr 172.21.15.86:0/2735836408 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:48.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.619+0000 7f4b477fe640 1 --2- 172.21.15.86:0/2735836408 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558d454c5a00 0x558d454c4dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.619+0000 7f4b467fc640 1 --2- 172.21.15.86:0/2735836408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558d454bce80 0x558d454b8aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.619+0000 7f4b46ffd640 1 -- 172.21.15.86:0/2735836408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558d454bce80 msgr2=0x558d454b8aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.619+0000 7f4b46ffd640 1 --2- 172.21.15.86:0/2735836408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558d454bce80 0x558d454b8aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.619+0000 7f4b46ffd640 1 -- 172.21.15.86:0/2735836408 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558d454c5a00 msgr2=0x558d454c4dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.619+0000 7f4b46ffd640 1 --2- 172.21.15.86:0/2735836408 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558d454c5a00 0x558d454c4dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.619+0000 7f4b46ffd640 1 -- 172.21.15.86:0/2735836408 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558d454bcc20 con 0x558d454bc4d0 2024-04-21T08:44:48.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.619+0000 7f4b46ffd640 1 --2- 172.21.15.86:0/2735836408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d454bc4d0 0x558d454d1830 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f4b400bb870 tx=0x7f4b400f2040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2fa5d14fa00ebc7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.619+0000 7f4b2cff9640 1 -- 172.21.15.86:0/2735836408 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b400c7b10 con 0x558d454bc4d0 2024-04-21T08:44:48.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.619+0000 7f4b2cff9640 1 -- 172.21.15.86:0/2735836408 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b400d7070 con 0x558d454bc4d0 2024-04-21T08:44:48.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.619+0000 7f4b2cff9640 1 -- 172.21.15.86:0/2735836408 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b400f07c0 con 0x558d454bc4d0 2024-04-21T08:44:48.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.620+0000 7f4b503df740 1 -- 172.21.15.86:0/2735836408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d454bc4d0 msgr2=0x558d454d1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.620+0000 7f4b503df740 1 --2- 172.21.15.86:0/2735836408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d454bc4d0 0x558d454d1830 secure :-1 s=READY pgs=1322 cs=0 l=1 rev1=1 crypto rx=0x7f4b400bb870 tx=0x7f4b400f2040 comp rx=0 tx=0).stop 2024-04-21T08:44:48.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.620+0000 7f4b503df740 1 -- 172.21.15.86:0/2735836408 shutdown_connections 2024-04-21T08:44:48.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.620+0000 7f4b503df740 1 --2- 172.21.15.86:0/2735836408 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558d454c5a00 0x558d454c4dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.620+0000 7f4b503df740 1 --2- 172.21.15.86:0/2735836408 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558d454bce80 0x558d454b8aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.620+0000 7f4b503df740 1 --2- 172.21.15.86:0/2735836408 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d454bc4d0 0x558d454d1830 unknown :-1 s=CLOSED pgs=1322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.620+0000 7f4b503df740 1 -- 172.21.15.86:0/2735836408 >> 172.21.15.86:0/2735836408 conn(0x558d454b6ba0 msgr2=0x558d454c16c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:48.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.620+0000 7f4b503df740 1 -- 172.21.15.86:0/2735836408 shutdown_connections 2024-04-21T08:44:48.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.620+0000 7f4b503df740 1 -- 172.21.15.86:0/2735836408 wait complete. 2024-04-21T08:44:48.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.620+0000 7f4b503df740 1 -- start start 2024-04-21T08:44:48.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.620+0000 7f4b503df740 1 Processor -- start 2024-04-21T08:44:48.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.621+0000 7f4b503df740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558d454bc4d0 0x558d454cdf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.621+0000 7f4b503df740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558d454bce80 0x558d454c92c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.621+0000 7f4b503df740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d454c5a00 0x558d454c9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.621+0000 7f4b503df740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b481ae3a0 con 0x558d454c5a00 2024-04-21T08:44:48.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.621+0000 7f4b503df740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4b481adcd0 con 0x558d454bce80 2024-04-21T08:44:48.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.621+0000 7f4b503df740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4b481aaef0 con 0x558d454bc4d0 2024-04-21T08:44:48.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.621+0000 7f4b503df740 10 client.0 _ll_register_callbacks cb 0x7f4b3c004a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:48.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.621+0000 7f4b477fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d454c5a00 0x558d454c9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.621+0000 7f4b467fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558d454bce80 0x558d454c92c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.621+0000 7f4b477fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d454c5a00 0x558d454c9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42364/0 (socket says 172.21.15.86:42364) 2024-04-21T08:44:48.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.621+0000 7f4b477fe640 1 -- 172.21.15.86:0/537955667 learned_addr learned my addr 172.21.15.86:0/537955667 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:48.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.621+0000 7f4b46ffd640 1 --2- 172.21.15.86:0/537955667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558d454bc4d0 0x558d454cdf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.621+0000 7f4b477fe640 1 -- 172.21.15.86:0/537955667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558d454bc4d0 msgr2=0x558d454cdf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.621+0000 7f4b477fe640 1 --2- 172.21.15.86:0/537955667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558d454bc4d0 0x558d454cdf50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.621+0000 7f4b477fe640 1 -- 172.21.15.86:0/537955667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558d454bce80 msgr2=0x558d454c92c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.621+0000 7f4b477fe640 1 --2- 172.21.15.86:0/537955667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558d454bce80 0x558d454c92c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.621+0000 7f4b477fe640 1 -- 172.21.15.86:0/537955667 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558d454c9e40 con 0x558d454c5a00 2024-04-21T08:44:48.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.621+0000 7f4b467fc640 1 --2- 172.21.15.86:0/537955667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558d454bce80 0x558d454c92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:44:48.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.621+0000 7f4b477fe640 1 --2- 172.21.15.86:0/537955667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d454c5a00 0x558d454c9800 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f4b3800bfd0 tx=0x7f4b38004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=55c79e3971ac87b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.621+0000 7f4b2aff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:48.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.621+0000 7f4b2aff5640 1 -- 172.21.15.86:0/537955667 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b38004220 con 0x558d454c5a00 2024-04-21T08:44:48.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.621+0000 7f4b2aff5640 1 -- 172.21.15.86:0/537955667 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f4b380043c0 con 0x558d454c5a00 2024-04-21T08:44:48.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.621+0000 7f4b503df740 1 -- 172.21.15.86:0/537955667 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4b400d7070 con 0x558d454c5a00 2024-04-21T08:44:48.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.622+0000 7f4b2aff5640 1 -- 172.21.15.86:0/537955667 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4b38031050 con 0x558d454c5a00 2024-04-21T08:44:48.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.622+0000 7f4b503df740 10 client.18144 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:48.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.622+0000 7f4b503df740 1 -- 172.21.15.86:0/537955667 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558d4553bcd0 con 0x558d454c5a00 2024-04-21T08:44:48.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.622+0000 7f4b2aff5640 1 -- 172.21.15.86:0/537955667 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b400d7070 con 0x558d454c5a00 2024-04-21T08:44:48.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.622+0000 7f4b2aff5640 1 -- 172.21.15.86:0/537955667 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4b38003b10 con 0x558d454c5a00 2024-04-21T08:44:48.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.622+0000 7f4b503df740 10 client.18144 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:48.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.622+0000 7f4b503df740 10 client.18144 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:44:48.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.622+0000 7f4b503df740 20 client.18144 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:48.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.622+0000 7f4b503df740 1 --2- 172.21.15.86:0/537955667 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x558d4553c480 0x558d4553e8a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.622+0000 7f4b503df740 4 client.18144 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:44:48.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.622+0000 7f4b503df740 1 -- 172.21.15.86:0/537955667 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558d4542ec10 con 0x558d4553c480 2024-04-21T08:44:48.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.622+0000 7f4b46ffd640 1 --2- 172.21.15.86:0/537955667 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x558d4553c480 0x558d4553e8a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.623+0000 7f4b2aff5640 10 client.18144 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:44:48.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.623+0000 7f4b46ffd640 1 --2- 172.21.15.86:0/537955667 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x558d4553c480 0x558d4553e8a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1004f4fbaacbd548 server_cookie=b23241db3a3d1fed in_seq=0 out_seq=0 2024-04-21T08:44:48.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.623+0000 7f4b2aff5640 1 -- 172.21.15.86:0/537955667 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558d4542ec10 con 0x558d4553c480 2024-04-21T08:44:48.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.623+0000 7f4b2aff5640 10 client.18144 handle_command_reply: tid=0 2024-04-21T08:44:48.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.623+0000 7f4b503df740 1 client.18144 shutdown 2024-04-21T08:44:48.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.623+0000 7f4b503df740 1 -- 172.21.15.86:0/537955667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d454c5a00 msgr2=0x558d454c9800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.623+0000 7f4b503df740 1 --2- 172.21.15.86:0/537955667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d454c5a00 0x558d454c9800 secure :-1 s=READY pgs=1323 cs=0 l=1 rev1=1 crypto rx=0x7f4b3800bfd0 tx=0x7f4b38004730 comp rx=0 tx=0).stop 2024-04-21T08:44:48.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.623+0000 7f4b503df740 1 -- 172.21.15.86:0/537955667 shutdown_connections 2024-04-21T08:44:48.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.623+0000 7f4b503df740 1 --2- 172.21.15.86:0/537955667 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558d454c5a00 0x558d454c9800 unknown :-1 s=CLOSED pgs=1323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.623+0000 7f4b503df740 1 --2- 172.21.15.86:0/537955667 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x558d4553c480 0x558d4553e8a0 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.623+0000 7f4b503df740 1 --2- 172.21.15.86:0/537955667 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558d454bce80 0x558d454c92c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.623+0000 7f4b503df740 1 --2- 172.21.15.86:0/537955667 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558d454bc4d0 0x558d454cdf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.623+0000 7f4b2aff5640 0 client.18144 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:44:48.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.623+0000 7f4b503df740 1 -- 172.21.15.86:0/537955667 >> 172.21.15.86:0/537955667 conn(0x558d454b6ba0 msgr2=0x558d454b5370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:48.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.623+0000 7f4b503df740 1 -- 172.21.15.86:0/537955667 shutdown_connections 2024-04-21T08:44:48.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.623+0000 7f4b503df740 1 -- 172.21.15.86:0/537955667 wait complete. 2024-04-21T08:44:48.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.623+0000 7f4b503df740 20 client.18144 trim_cache size 0 max 16384 2024-04-21T08:44:48.632 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:44:48.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.624+0000 7f4b4d71e640 1 -- 172.21.15.86:0/2260442632 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4b300a3d20 msgr2=0x7f4b300a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.624+0000 7f4b4d71e640 1 --2- 172.21.15.86:0/2260442632 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4b300a3d20 0x7f4b300a61e0 secure :-1 s=READY pgs=209 cs=0 l=1 rev1=1 crypto rx=0x7f4b38005fd0 tx=0x7f4b3800f360 comp rx=0 tx=0).stop 2024-04-21T08:44:48.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.624+0000 7f4b4d71e640 1 -- 172.21.15.86:0/2260442632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b48101130 msgr2=0x7f4b4819e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.624+0000 7f4b4d71e640 1 --2- 172.21.15.86:0/2260442632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b48101130 0x7f4b4819e460 secure :-1 s=READY pgs=1319 cs=0 l=1 rev1=1 crypto rx=0x7f4b40005e30 tx=0x7f4b40035d60 comp rx=0 tx=0).stop 2024-04-21T08:44:48.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.624+0000 7f4b4d71e640 1 -- 172.21.15.86:0/2260442632 shutdown_connections 2024-04-21T08:44:48.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.624+0000 7f4b4d71e640 1 --2- 172.21.15.86:0/2260442632 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4b300a3d20 0x7f4b300a61e0 unknown :-1 s=CLOSED pgs=209 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.624+0000 7f4b4d71e640 1 --2- 172.21.15.86:0/2260442632 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b48105ea0 0x7f4b48198660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.624+0000 7f4b4d71e640 1 --2- 172.21.15.86:0/2260442632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b48101ad0 0x7f4b4819e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.624+0000 7f4b4d71e640 1 --2- 172.21.15.86:0/2260442632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b48101130 0x7f4b4819e460 unknown :-1 s=CLOSED pgs=1319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.624+0000 7f4b4d71e640 1 -- 172.21.15.86:0/2260442632 >> 172.21.15.86:0/2260442632 conn(0x7f4b48077e30 msgr2=0x7f4b48104e30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:48.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.624+0000 7f4b4d71e640 1 -- 172.21.15.86:0/2260442632 shutdown_connections 2024-04-21T08:44:48.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.624+0000 7f4b4d71e640 1 -- 172.21.15.86:0/2260442632 wait complete. 2024-04-21T08:44:48.635 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:44:48.689 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.688+0000 7fe56bfbd640 1 -- 172.21.15.86:0/3893784731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5641010f0 msgr2=0x7fe5641014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.688+0000 7fe56bfbd640 1 --2- 172.21.15.86:0/3893784731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5641010f0 0x7fe5641014d0 secure :-1 s=READY pgs=1324 cs=0 l=1 rev1=1 crypto rx=0x7fe55c009900 tx=0x7fe55c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.688+0000 7fe56bfbd640 1 -- 172.21.15.86:0/3893784731 shutdown_connections 2024-04-21T08:44:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.688+0000 7fe56bfbd640 1 --2- 172.21.15.86:0/3893784731 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe564106170 0x7fe56410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.688+0000 7fe56bfbd640 1 --2- 172.21.15.86:0/3893784731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe564101aa0 0x7fe564105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.688+0000 7fe56bfbd640 1 --2- 172.21.15.86:0/3893784731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe5641010f0 0x7fe5641014d0 unknown :-1 s=CLOSED pgs=1324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.688+0000 7fe56bfbd640 1 -- 172.21.15.86:0/3893784731 >> 172.21.15.86:0/3893784731 conn(0x7fe564077fd0 msgr2=0x7fe5640783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.688+0000 7fe56bfbd640 1 -- 172.21.15.86:0/3893784731 shutdown_connections 2024-04-21T08:44:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.688+0000 7fe56bfbd640 1 -- 172.21.15.86:0/3893784731 wait complete. 2024-04-21T08:44:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.689+0000 7fe56bfbd640 1 Processor -- start 2024-04-21T08:44:48.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.689+0000 7fe56bfbd640 1 -- start start 2024-04-21T08:44:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.689+0000 7fe56bfbd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5641010f0 0x7fe564102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.689+0000 7fe56bfbd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe564106170 0x7fe564103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.689+0000 7fe56bfbd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5641035b0 0x7fe5641a6910 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.689+0000 7fe56bfbd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe5641110d0 con 0x7fe564106170 2024-04-21T08:44:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.689+0000 7fe56bfbd640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe564110f50 con 0x7fe5641035b0 2024-04-21T08:44:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.689+0000 7fe56bfbd640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe564111250 con 0x7fe5641010f0 2024-04-21T08:44:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.689+0000 7fe569558640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe564106170 0x7fe564103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.689+0000 7fe569558640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe564106170 0x7fe564103070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42384/0 (socket says 172.21.15.86:42384) 2024-04-21T08:44:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.689+0000 7fe569558640 1 -- 172.21.15.86:0/2535132820 learned_addr learned my addr 172.21.15.86:0/2535132820 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.689+0000 7fe569d59640 1 --2- 172.21.15.86:0/2535132820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5641010f0 0x7fe564102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.689+0000 7fe56a55a640 1 --2- 172.21.15.86:0/2535132820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5641035b0 0x7fe5641a6910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.690+0000 7fe569558640 1 -- 172.21.15.86:0/2535132820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5641010f0 msgr2=0x7fe564102b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.690+0000 7fe569558640 1 --2- 172.21.15.86:0/2535132820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5641010f0 0x7fe564102b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.690+0000 7fe569558640 1 -- 172.21.15.86:0/2535132820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5641035b0 msgr2=0x7fe5641a6910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.690+0000 7fe569558640 1 --2- 172.21.15.86:0/2535132820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5641035b0 0x7fe5641a6910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.690+0000 7fe569558640 1 -- 172.21.15.86:0/2535132820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe5641a7010 con 0x7fe564106170 2024-04-21T08:44:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.690+0000 7fe569558640 1 --2- 172.21.15.86:0/2535132820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe564106170 0x7fe564103070 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fe55400d740 tx=0x7fe55400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.690+0000 7fe55affd640 1 -- 172.21.15.86:0/2535132820 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe554017070 con 0x7fe564106170 2024-04-21T08:44:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.690+0000 7fe56bfbd640 1 -- 172.21.15.86:0/2535132820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe5641a7300 con 0x7fe564106170 2024-04-21T08:44:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.690+0000 7fe55affd640 1 -- 172.21.15.86:0/2535132820 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe55400de50 con 0x7fe564106170 2024-04-21T08:44:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.690+0000 7fe55affd640 1 -- 172.21.15.86:0/2535132820 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe554004ce0 con 0x7fe564106170 2024-04-21T08:44:48.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.690+0000 7fe56bfbd640 1 -- 172.21.15.86:0/2535132820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe5641a7800 con 0x7fe564106170 2024-04-21T08:44:48.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.692+0000 7fe56ebb2740 1 Processor -- start 2024-04-21T08:44:48.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.693+0000 7fe56ebb2740 1 -- start start 2024-04-21T08:44:48.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.693+0000 7fe55affd640 1 -- 172.21.15.86:0/2535132820 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe554004e80 con 0x7fe564106170 2024-04-21T08:44:48.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.693+0000 7fe56ebb2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582a23bc8d0 0x5582a23bb620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.693+0000 7fe56ebb2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582a23bbb60 0x5582a23bbfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.693+0000 7fe56ebb2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5582a23c50e0 0x5582a23c74d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.693+0000 7fe56ebb2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5582a23208d0 con 0x5582a23bc8d0 2024-04-21T08:44:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.693+0000 7fe56ebb2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5582a22bde80 con 0x5582a23c50e0 2024-04-21T08:44:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.693+0000 7fe56ebb2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5582a231c890 con 0x5582a23bbb60 2024-04-21T08:44:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.693+0000 7fe56a55a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582a23bc8d0 0x5582a23bb620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.693+0000 7fe56a55a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582a23bc8d0 0x5582a23bb620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42394/0 (socket says 172.21.15.86:42394) 2024-04-21T08:44:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.693+0000 7fe56a55a640 1 -- 172.21.15.86:0/404732572 learned_addr learned my addr 172.21.15.86:0/404732572 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.693+0000 7fe569d59640 1 --2- 172.21.15.86:0/404732572 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582a23bbb60 0x5582a23bbfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.693+0000 7fe569558640 1 --2- 172.21.15.86:0/404732572 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5582a23c50e0 0x5582a23c74d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.693+0000 7fe56a55a640 1 -- 172.21.15.86:0/404732572 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582a23bbb60 msgr2=0x5582a23bbfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.693+0000 7fe56a55a640 1 --2- 172.21.15.86:0/404732572 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582a23bbb60 0x5582a23bbfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.693+0000 7fe56a55a640 1 -- 172.21.15.86:0/404732572 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5582a23c50e0 msgr2=0x5582a23c74d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.693+0000 7fe56a55a640 1 --2- 172.21.15.86:0/404732572 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5582a23c50e0 0x5582a23c74d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.701 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.693+0000 7fe56a55a640 1 -- 172.21.15.86:0/404732572 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582a22df320 con 0x5582a23bc8d0 2024-04-21T08:44:48.701 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.693+0000 7fe569558640 1 --2- 172.21.15.86:0/404732572 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5582a23c50e0 0x5582a23c74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:48.701 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.693+0000 7fe569d59640 1 --2- 172.21.15.86:0/404732572 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582a23bbb60 0x5582a23bbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:48.701 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.693+0000 7fe55affd640 1 --2- 172.21.15.86:0/2535132820 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe5500a3df0 0x7fe5500a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.701 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.693+0000 7fe56a55a640 1 --2- 172.21.15.86:0/404732572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582a23bc8d0 0x5582a23bb620 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fe560009f20 tx=0x7fe560010860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5af43719430510b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.701 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.694+0000 7fe569d59640 1 --2- 172.21.15.86:0/2535132820 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe5500a3df0 0x7fe5500a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.701 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.694+0000 7fe55affd640 1 -- 172.21.15.86:0/2535132820 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fe5540d04b0 con 0x7fe564106170 2024-04-21T08:44:48.701 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.694+0000 7fe54ffff640 1 -- 172.21.15.86:0/404732572 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe560011700 con 0x5582a23bc8d0 2024-04-21T08:44:48.701 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.694+0000 7fe54ffff640 1 -- 172.21.15.86:0/404732572 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe560011d50 con 0x5582a23bc8d0 2024-04-21T08:44:48.701 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.694+0000 7fe54ffff640 1 -- 172.21.15.86:0/404732572 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56001cdf0 con 0x5582a23bc8d0 2024-04-21T08:44:48.701 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.694+0000 7fe56ebb2740 1 -- 172.21.15.86:0/404732572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582a23bc8d0 msgr2=0x5582a23bb620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.694+0000 7fe569d59640 1 --2- 172.21.15.86:0/2535132820 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe5500a3df0 0x7fe5500a62b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe55c006010 tx=0x7fe55c005e40 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.694+0000 7fe56ebb2740 1 --2- 172.21.15.86:0/404732572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582a23bc8d0 0x5582a23bb620 secure :-1 s=READY pgs=1326 cs=0 l=1 rev1=1 crypto rx=0x7fe560009f20 tx=0x7fe560010860 comp rx=0 tx=0).stop 2024-04-21T08:44:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.695+0000 7fe56ebb2740 1 -- 172.21.15.86:0/404732572 shutdown_connections 2024-04-21T08:44:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.695+0000 7fe56ebb2740 1 --2- 172.21.15.86:0/404732572 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5582a23c50e0 0x5582a23c74d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.695+0000 7fe56ebb2740 1 --2- 172.21.15.86:0/404732572 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582a23bbb60 0x5582a23bbfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.695+0000 7fe56ebb2740 1 --2- 172.21.15.86:0/404732572 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582a23bc8d0 0x5582a23bb620 unknown :-1 s=CLOSED pgs=1326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.695+0000 7fe56ebb2740 1 -- 172.21.15.86:0/404732572 >> 172.21.15.86:0/404732572 conn(0x5582a2382c90 msgr2=0x5582a23b6850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.695+0000 7fe56ebb2740 1 -- 172.21.15.86:0/404732572 shutdown_connections 2024-04-21T08:44:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.695+0000 7fe56ebb2740 1 -- 172.21.15.86:0/404732572 wait complete. 2024-04-21T08:44:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.695+0000 7fe56ebb2740 1 -- start start 2024-04-21T08:44:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.695+0000 7fe56ebb2740 1 Processor -- start 2024-04-21T08:44:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.695+0000 7fe56ebb2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582a23bbb60 0x5582a23c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.702 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.695+0000 7fe56ebb2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5582a23bc8d0 0x5582a23c1e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.695+0000 7fe56ebb2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582a23c50e0 0x5582a23c2450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.695+0000 7fe56ebb2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5582a23c7e10 con 0x5582a23c50e0 2024-04-21T08:44:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.695+0000 7fe56ebb2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5582a22a8eb0 con 0x5582a23bc8d0 2024-04-21T08:44:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.695+0000 7fe56ebb2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5582a23c7f90 con 0x5582a23bbb60 2024-04-21T08:44:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.695+0000 7fe56ebb2740 10 client.0 _ll_register_callbacks cb 0x5582a22d5b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.696+0000 7fe56a55a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582a23c50e0 0x5582a23c2450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.696+0000 7fe56a55a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582a23c50e0 0x5582a23c2450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42398/0 (socket says 172.21.15.86:42398) 2024-04-21T08:44:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.696+0000 7fe56a55a640 1 -- 172.21.15.86:0/2008439573 learned_addr learned my addr 172.21.15.86:0/2008439573 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.696+0000 7fe569558640 1 --2- 172.21.15.86:0/2008439573 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5582a23bc8d0 0x5582a23c1e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.696+0000 7fe569d59640 1 --2- 172.21.15.86:0/2008439573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582a23bbb60 0x5582a23c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.696+0000 7fe56a55a640 1 -- 172.21.15.86:0/2008439573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582a23bbb60 msgr2=0x5582a23c4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.696+0000 7fe56a55a640 1 --2- 172.21.15.86:0/2008439573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582a23bbb60 0x5582a23c4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.696+0000 7fe56a55a640 1 -- 172.21.15.86:0/2008439573 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5582a23bc8d0 msgr2=0x5582a23c1e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.696+0000 7fe56a55a640 1 --2- 172.21.15.86:0/2008439573 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5582a23bc8d0 0x5582a23c1e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.703 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.696+0000 7fe56a55a640 1 -- 172.21.15.86:0/2008439573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582a2033210 con 0x5582a23c50e0 2024-04-21T08:44:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.696+0000 7fe56a55a640 1 --2- 172.21.15.86:0/2008439573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582a23c50e0 0x5582a23c2450 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fe5600099a0 tx=0x7fe56001cd40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb74ae7f6509836c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.696+0000 7fe54d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.696+0000 7fe54d7fa640 1 -- 172.21.15.86:0/2008439573 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56002b060 con 0x5582a23c50e0 2024-04-21T08:44:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.696+0000 7fe54d7fa640 1 -- 172.21.15.86:0/2008439573 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fe560004c50 con 0x5582a23c50e0 2024-04-21T08:44:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.696+0000 7fe56ebb2740 1 -- 172.21.15.86:0/2008439573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe560011d50 con 0x5582a23c50e0 2024-04-21T08:44:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.696+0000 7fe54d7fa640 1 -- 172.21.15.86:0/2008439573 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe560032050 con 0x5582a23c50e0 2024-04-21T08:44:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.696+0000 7fe56ebb2740 10 client.18156 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.696+0000 7fe56ebb2740 1 -- 172.21.15.86:0/2008439573 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5582a20ff620 con 0x5582a23c50e0 2024-04-21T08:44:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.697+0000 7fe54d7fa640 1 -- 172.21.15.86:0/2008439573 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe560011d50 con 0x5582a23c50e0 2024-04-21T08:44:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.697+0000 7fe54d7fa640 1 -- 172.21.15.86:0/2008439573 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe560005200 con 0x5582a23c50e0 2024-04-21T08:44:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.697+0000 7fe56ebb2740 10 client.18156 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.697+0000 7fe56ebb2740 10 client.18156 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.697+0000 7fe56ebb2740 20 client.18156 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.697+0000 7fe56ebb2740 1 --2- 172.21.15.86:0/2008439573 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5582a23d5020 0x5582a23d7410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.704 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.697+0000 7fe56ebb2740 4 client.18156 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:48.705 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.697+0000 7fe56ebb2740 1 -- 172.21.15.86:0/2008439573 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5582a2334880 con 0x5582a23d5020 2024-04-21T08:44:48.705 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.697+0000 7fe569d59640 1 --2- 172.21.15.86:0/2008439573 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5582a23d5020 0x5582a23d7410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.705 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.698+0000 7fe569d59640 1 --2- 172.21.15.86:0/2008439573 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5582a23d5020 0x5582a23d7410 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4dc874b60fac34fe server_cookie=5ff23401aca17798 in_seq=0 out_seq=0 2024-04-21T08:44:48.705 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.698+0000 7fe54d7fa640 10 client.18156 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:48.705 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.698+0000 7fe54d7fa640 1 -- 172.21.15.86:0/2008439573 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5582a2334880 con 0x5582a23d5020 2024-04-21T08:44:48.705 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.698+0000 7fe54d7fa640 10 client.18156 handle_command_reply: tid=0 2024-04-21T08:44:48.705 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.698+0000 7fe56ebb2740 1 client.18156 shutdown 2024-04-21T08:44:48.705 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.698+0000 7fe56ebb2740 1 -- 172.21.15.86:0/2008439573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582a23c50e0 msgr2=0x5582a23c2450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.705 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.698+0000 7fe56ebb2740 1 --2- 172.21.15.86:0/2008439573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582a23c50e0 0x5582a23c2450 secure :-1 s=READY pgs=1327 cs=0 l=1 rev1=1 crypto rx=0x7fe5600099a0 tx=0x7fe56001cd40 comp rx=0 tx=0).stop 2024-04-21T08:44:48.705 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.699+0000 7fe56ebb2740 1 -- 172.21.15.86:0/2008439573 shutdown_connections 2024-04-21T08:44:48.705 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.699+0000 7fe56ebb2740 1 --2- 172.21.15.86:0/2008439573 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5582a23d5020 0x5582a23d7410 crc :-1 s=READY pgs=193 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.705 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.699+0000 7fe56ebb2740 1 --2- 172.21.15.86:0/2008439573 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582a23c50e0 0x5582a23c2450 unknown :-1 s=CLOSED pgs=1327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.705 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.699+0000 7fe54d7fa640 0 client.18156 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:48.705 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.699+0000 7fe56ebb2740 1 --2- 172.21.15.86:0/2008439573 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5582a23bc8d0 0x5582a23c1e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.705 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.699+0000 7fe56ebb2740 1 --2- 172.21.15.86:0/2008439573 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582a23bbb60 0x5582a23c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.699+0000 7fe56ebb2740 1 -- 172.21.15.86:0/2008439573 >> 172.21.15.86:0/2008439573 conn(0x5582a2382c90 msgr2=0x5582a23b85a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:48.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.699+0000 7fe56ebb2740 1 -- 172.21.15.86:0/2008439573 shutdown_connections 2024-04-21T08:44:48.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.699+0000 7fe56ebb2740 1 -- 172.21.15.86:0/2008439573 wait complete. 2024-04-21T08:44:48.706 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.699+0000 7fe56ebb2740 20 client.18156 trim_cache size 0 max 16384 2024-04-21T08:44:48.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.727+0000 7fe56ebb2740 1 Processor -- start 2024-04-21T08:44:48.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.727+0000 7fe56ebb2740 1 -- start start 2024-04-21T08:44:48.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.728+0000 7fe56ebb2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582a23bbb60 0x5582a23c4570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.728+0000 7fe56ebb2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5582a23bc8d0 0x5582a23c4ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.728+0000 7fe56ebb2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582a23c50e0 0x5582a23da430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.728+0000 7fe56ebb2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe5641b1440 con 0x5582a23c50e0 2024-04-21T08:44:48.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.728+0000 7fe56ebb2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe5641b2f80 con 0x5582a23bc8d0 2024-04-21T08:44:48.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.728+0000 7fe56ebb2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe5641b1b10 con 0x5582a23bbb60 2024-04-21T08:44:48.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.728+0000 7fe56a55a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582a23c50e0 0x5582a23da430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.728+0000 7fe56a55a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582a23c50e0 0x5582a23da430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42402/0 (socket says 172.21.15.86:42402) 2024-04-21T08:44:48.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.728+0000 7fe56a55a640 1 -- 172.21.15.86:0/3919110877 learned_addr learned my addr 172.21.15.86:0/3919110877 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:48.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.728+0000 7fe569d59640 1 --2- 172.21.15.86:0/3919110877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582a23bbb60 0x5582a23c4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.728+0000 7fe569558640 1 --2- 172.21.15.86:0/3919110877 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5582a23bc8d0 0x5582a23c4ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.728+0000 7fe56a55a640 1 -- 172.21.15.86:0/3919110877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582a23bbb60 msgr2=0x5582a23c4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.728+0000 7fe56a55a640 1 --2- 172.21.15.86:0/3919110877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582a23bbb60 0x5582a23c4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.728+0000 7fe56a55a640 1 -- 172.21.15.86:0/3919110877 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5582a23bc8d0 msgr2=0x5582a23c4ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.728+0000 7fe56a55a640 1 --2- 172.21.15.86:0/3919110877 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5582a23bc8d0 0x5582a23c4ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.728+0000 7fe56a55a640 1 -- 172.21.15.86:0/3919110877 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5582a23bae30 con 0x5582a23c50e0 2024-04-21T08:44:48.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.728+0000 7fe56a55a640 1 --2- 172.21.15.86:0/3919110877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582a23c50e0 0x5582a23da430 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fe560011fd0 tx=0x7fe5600092a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ebad57add3fd262e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.729+0000 7fe54f7fe640 1 -- 172.21.15.86:0/3919110877 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe56002b040 con 0x5582a23c50e0 2024-04-21T08:44:48.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.729+0000 7fe54f7fe640 1 -- 172.21.15.86:0/3919110877 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe560005940 con 0x5582a23c50e0 2024-04-21T08:44:48.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.729+0000 7fe54f7fe640 1 -- 172.21.15.86:0/3919110877 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe560011500 con 0x5582a23c50e0 2024-04-21T08:44:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.729+0000 7fe56ebb2740 1 -- 172.21.15.86:0/3919110877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582a23c50e0 msgr2=0x5582a23da430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.729+0000 7fe56ebb2740 1 --2- 172.21.15.86:0/3919110877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582a23c50e0 0x5582a23da430 secure :-1 s=READY pgs=1328 cs=0 l=1 rev1=1 crypto rx=0x7fe560011fd0 tx=0x7fe5600092a0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.729+0000 7fe56ebb2740 1 -- 172.21.15.86:0/3919110877 shutdown_connections 2024-04-21T08:44:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.729+0000 7fe56ebb2740 1 --2- 172.21.15.86:0/3919110877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582a23c50e0 0x5582a23da430 unknown :-1 s=CLOSED pgs=1328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.729+0000 7fe56ebb2740 1 --2- 172.21.15.86:0/3919110877 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5582a23bc8d0 0x5582a23c4ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.729+0000 7fe56ebb2740 1 --2- 172.21.15.86:0/3919110877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582a23bbb60 0x5582a23c4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.729+0000 7fe56ebb2740 1 -- 172.21.15.86:0/3919110877 >> 172.21.15.86:0/3919110877 conn(0x5582a2382c90 msgr2=0x5582a23b6380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.729+0000 7fe56ebb2740 1 -- 172.21.15.86:0/3919110877 shutdown_connections 2024-04-21T08:44:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.729+0000 7fe56ebb2740 1 -- 172.21.15.86:0/3919110877 wait complete. 2024-04-21T08:44:48.733 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.730+0000 7fe56ebb2740 1 -- start start 2024-04-21T08:44:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.730+0000 7fe56ebb2740 1 Processor -- start 2024-04-21T08:44:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.730+0000 7fe56ebb2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582a23bbb60 0x5582a23d8110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.730+0000 7fe56ebb2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582a23bc8d0 0x5582a23d8650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.730+0000 7fe56ebb2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5582a23c50e0 0x5582a23d8b90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.730+0000 7fe56ebb2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe5641b28b0 con 0x5582a23bbb60 2024-04-21T08:44:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.730+0000 7fe56ebb2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe5641b21e0 con 0x5582a23c50e0 2024-04-21T08:44:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.730+0000 7fe56ebb2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe5641af400 con 0x5582a23bc8d0 2024-04-21T08:44:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.730+0000 7fe56ebb2740 10 client.0 _ll_register_callbacks cb 0x7fe560009e20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.730+0000 7fe569d59640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582a23bbb60 0x5582a23d8110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.730+0000 7fe569d59640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582a23bbb60 0x5582a23d8110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42408/0 (socket says 172.21.15.86:42408) 2024-04-21T08:44:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.730+0000 7fe569d59640 1 -- 172.21.15.86:0/4056170685 learned_addr learned my addr 172.21.15.86:0/4056170685 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.730+0000 7fe56a55a640 1 --2- 172.21.15.86:0/4056170685 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5582a23c50e0 0x5582a23d8b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.730+0000 7fe569558640 1 --2- 172.21.15.86:0/4056170685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582a23bc8d0 0x5582a23d8650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.734 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.731+0000 7fe569d59640 1 -- 172.21.15.86:0/4056170685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582a23bc8d0 msgr2=0x5582a23d8650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.731+0000 7fe569d59640 1 --2- 172.21.15.86:0/4056170685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582a23bc8d0 0x5582a23d8650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.731+0000 7fe569d59640 1 -- 172.21.15.86:0/4056170685 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5582a23c50e0 msgr2=0x5582a23d8b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.731+0000 7fe569d59640 1 --2- 172.21.15.86:0/4056170685 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5582a23c50e0 0x5582a23d8b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.731+0000 7fe569d59640 1 -- 172.21.15.86:0/4056170685 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5582a243a4a0 con 0x5582a23bbb60 2024-04-21T08:44:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.731+0000 7fe569d59640 1 --2- 172.21.15.86:0/4056170685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582a23bbb60 0x5582a23d8110 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fe55c004460 tx=0x7fe55c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b862c9d459d8df40 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.731+0000 7fe54d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.731+0000 7fe54d7fa640 1 -- 172.21.15.86:0/4056170685 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe55c02cab0 con 0x5582a23bbb60 2024-04-21T08:44:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.731+0000 7fe54d7fa640 1 -- 172.21.15.86:0/4056170685 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fe55c005040 con 0x5582a23bbb60 2024-04-21T08:44:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.731+0000 7fe56ebb2740 1 -- 172.21.15.86:0/4056170685 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe560005940 con 0x5582a23bbb60 2024-04-21T08:44:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.731+0000 7fe54d7fa640 1 -- 172.21.15.86:0/4056170685 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe55c02cc50 con 0x5582a23bbb60 2024-04-21T08:44:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.731+0000 7fe56ebb2740 10 client.18162 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.731+0000 7fe56ebb2740 1 -- 172.21.15.86:0/4056170685 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5582a243b810 con 0x5582a23bbb60 2024-04-21T08:44:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.731+0000 7fe54d7fa640 1 -- 172.21.15.86:0/4056170685 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe560005940 con 0x5582a23bbb60 2024-04-21T08:44:48.735 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.731+0000 7fe54d7fa640 1 -- 172.21.15.86:0/4056170685 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe55c05a070 con 0x5582a23bbb60 2024-04-21T08:44:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.731+0000 7fe56ebb2740 10 client.18162 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.731+0000 7fe56ebb2740 10 client.18162 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.731+0000 7fe56ebb2740 20 client.18162 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.732+0000 7fe56ebb2740 1 --2- 172.21.15.86:0/4056170685 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5582a243bee0 0x5582a243e2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.732+0000 7fe56ebb2740 4 client.18162 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:44:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.732+0000 7fe56ebb2740 1 -- 172.21.15.86:0/4056170685 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5582a22c03d0 con 0x5582a243bee0 2024-04-21T08:44:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.732+0000 7fe569558640 1 --2- 172.21.15.86:0/4056170685 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5582a243bee0 0x5582a243e2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.732+0000 7fe54d7fa640 10 client.18162 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:48.736 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.732+0000 7fe569558640 1 --2- 172.21.15.86:0/4056170685 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5582a243bee0 0x5582a243e2f0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5f766fb133658654 server_cookie=d40f35e994ff7c01 in_seq=0 out_seq=0 2024-04-21T08:44:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.747+0000 7fe54d7fa640 1 -- 172.21.15.86:0/4056170685 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5582a22c03d0 con 0x5582a243bee0 2024-04-21T08:44:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.747+0000 7fe54d7fa640 10 client.18162 handle_command_reply: tid=0 2024-04-21T08:44:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.747+0000 7fe56ebb2740 1 client.18162 shutdown 2024-04-21T08:44:48.747 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.747+0000 7fe56ebb2740 1 -- 172.21.15.86:0/4056170685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582a23bbb60 msgr2=0x5582a23d8110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.747+0000 7fe56ebb2740 1 --2- 172.21.15.86:0/4056170685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582a23bbb60 0x5582a23d8110 secure :-1 s=READY pgs=1329 cs=0 l=1 rev1=1 crypto rx=0x7fe55c004460 tx=0x7fe55c009260 comp rx=0 tx=0).stop 2024-04-21T08:44:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.748+0000 7fe56ebb2740 1 -- 172.21.15.86:0/4056170685 shutdown_connections 2024-04-21T08:44:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.748+0000 7fe56ebb2740 1 --2- 172.21.15.86:0/4056170685 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5582a243bee0 0x5582a243e2f0 crc :-1 s=READY pgs=194 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.748+0000 7fe56ebb2740 1 --2- 172.21.15.86:0/4056170685 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5582a23c50e0 0x5582a23d8b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.748+0000 7fe56ebb2740 1 --2- 172.21.15.86:0/4056170685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5582a23bc8d0 0x5582a23d8650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.748+0000 7fe54d7fa640 0 client.18162 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.748+0000 7fe56ebb2740 1 --2- 172.21.15.86:0/4056170685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5582a23bbb60 0x5582a23d8110 unknown :-1 s=CLOSED pgs=1329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.748+0000 7fe56ebb2740 1 -- 172.21.15.86:0/4056170685 >> 172.21.15.86:0/4056170685 conn(0x5582a2382c90 msgr2=0x5582a23b5040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.748+0000 7fe56ebb2740 1 -- 172.21.15.86:0/4056170685 shutdown_connections 2024-04-21T08:44:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.748+0000 7fe56ebb2740 1 -- 172.21.15.86:0/4056170685 wait complete. 2024-04-21T08:44:48.748 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.748+0000 7fe56ebb2740 20 client.18162 trim_cache size 0 max 16384 2024-04-21T08:44:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:44:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:44:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:44:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:44:48.749 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.748+0000 7fe56bfbd640 1 -- 172.21.15.86:0/2535132820 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe5500a3df0 msgr2=0x7fe5500a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.748+0000 7fe56bfbd640 1 --2- 172.21.15.86:0/2535132820 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe5500a3df0 0x7fe5500a62b0 secure :-1 s=READY pgs=210 cs=0 l=1 rev1=1 crypto rx=0x7fe55c006010 tx=0x7fe55c005e40 comp rx=0 tx=0).stop 2024-04-21T08:44:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.748+0000 7fe56bfbd640 1 -- 172.21.15.86:0/2535132820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe564106170 msgr2=0x7fe564103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.748+0000 7fe56bfbd640 1 --2- 172.21.15.86:0/2535132820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe564106170 0x7fe564103070 secure :-1 s=READY pgs=1325 cs=0 l=1 rev1=1 crypto rx=0x7fe55400d740 tx=0x7fe55400dc00 comp rx=0 tx=0).stop 2024-04-21T08:44:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.748+0000 7fe56bfbd640 1 -- 172.21.15.86:0/2535132820 shutdown_connections 2024-04-21T08:44:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.748+0000 7fe56bfbd640 1 --2- 172.21.15.86:0/2535132820 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe5500a3df0 0x7fe5500a62b0 unknown :-1 s=CLOSED pgs=210 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.748+0000 7fe56bfbd640 1 --2- 172.21.15.86:0/2535132820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe5641035b0 0x7fe5641a6910 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.748+0000 7fe56bfbd640 1 --2- 172.21.15.86:0/2535132820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe564106170 0x7fe564103070 unknown :-1 s=CLOSED pgs=1325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.748+0000 7fe56bfbd640 1 --2- 172.21.15.86:0/2535132820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe5641010f0 0x7fe564102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.748+0000 7fe56bfbd640 1 -- 172.21.15.86:0/2535132820 >> 172.21.15.86:0/2535132820 conn(0x7fe564077fd0 msgr2=0x7fe5640699a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.748+0000 7fe56bfbd640 1 -- 172.21.15.86:0/2535132820 shutdown_connections 2024-04-21T08:44:48.750 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:44:48.748+0000 7fe56bfbd640 1 -- 172.21.15.86:0/2535132820 wait complete. 2024-04-21T08:44:48.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.753+0000 7fee2f606640 1 -- 172.21.15.86:0/1091659831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee28101a40 msgr2=0x7fee28105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.753+0000 7fee2f606640 1 --2- 172.21.15.86:0/1091659831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee28101a40 0x7fee28105a80 secure :-1 s=READY pgs=1330 cs=0 l=1 rev1=1 crypto rx=0x7fee18009900 tx=0x7fee1802bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:48.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.753+0000 7fee2f606640 1 -- 172.21.15.86:0/1091659831 shutdown_connections 2024-04-21T08:44:48.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.753+0000 7fee2f606640 1 --2- 172.21.15.86:0/1091659831 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee28106160 0x7fee2810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.753+0000 7fee2f606640 1 --2- 172.21.15.86:0/1091659831 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee28101a40 0x7fee28105a80 unknown :-1 s=CLOSED pgs=1330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.753+0000 7fee2f606640 1 --2- 172.21.15.86:0/1091659831 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee28101090 0x7fee28101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.753+0000 7fee2f606640 1 -- 172.21.15.86:0/1091659831 >> 172.21.15.86:0/1091659831 conn(0x7fee28077f50 msgr2=0x7fee28078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:48.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.753+0000 7fee2f606640 1 -- 172.21.15.86:0/1091659831 shutdown_connections 2024-04-21T08:44:48.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.753+0000 7fee2f606640 1 -- 172.21.15.86:0/1091659831 wait complete. 2024-04-21T08:44:48.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.754+0000 7fee2f606640 1 Processor -- start 2024-04-21T08:44:48.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.754+0000 7fee2f606640 1 -- start start 2024-04-21T08:44:48.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.754+0000 7fee2f606640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee28101090 0x7fee28199520 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.754+0000 7fee2f606640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee28106160 0x7fee28199a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.754+0000 7fee2f606640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee28199ff0 0x7fee281a45a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.754+0000 7fee2f606640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee28111080 con 0x7fee28106160 2024-04-21T08:44:48.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.754+0000 7fee2f606640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fee28110f00 con 0x7fee28101090 2024-04-21T08:44:48.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.754+0000 7fee2f606640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fee28111200 con 0x7fee28199ff0 2024-04-21T08:44:48.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.754+0000 7fee2cba1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee28106160 0x7fee28199a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.754+0000 7fee2cba1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee28106160 0x7fee28199a60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42418/0 (socket says 172.21.15.86:42418) 2024-04-21T08:44:48.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.754+0000 7fee2cba1640 1 -- 172.21.15.86:0/1137484771 learned_addr learned my addr 172.21.15.86:0/1137484771 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:48.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.754+0000 7fee2d3a2640 1 --2- 172.21.15.86:0/1137484771 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee28101090 0x7fee28199520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.754+0000 7fee2dba3640 1 --2- 172.21.15.86:0/1137484771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee28199ff0 0x7fee281a45a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.754+0000 7fee2cba1640 1 -- 172.21.15.86:0/1137484771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee28199ff0 msgr2=0x7fee281a45a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.754+0000 7fee2cba1640 1 --2- 172.21.15.86:0/1137484771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee28199ff0 0x7fee281a45a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.754+0000 7fee2cba1640 1 -- 172.21.15.86:0/1137484771 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee28101090 msgr2=0x7fee28199520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.754+0000 7fee2cba1640 1 --2- 172.21.15.86:0/1137484771 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee28101090 0x7fee28199520 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.754+0000 7fee2cba1640 1 -- 172.21.15.86:0/1137484771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fee281a4da0 con 0x7fee28106160 2024-04-21T08:44:48.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.754+0000 7fee2d3a2640 1 --2- 172.21.15.86:0/1137484771 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee28101090 0x7fee28199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:48.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.755+0000 7fee2cba1640 1 --2- 172.21.15.86:0/1137484771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee28106160 0x7fee28199a60 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fee18036040 tx=0x7fee1802c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.755+0000 7fee1e7fc640 1 -- 172.21.15.86:0/1137484771 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee18043020 con 0x7fee28106160 2024-04-21T08:44:48.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.755+0000 7fee2f606640 1 -- 172.21.15.86:0/1137484771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fee281a5030 con 0x7fee28106160 2024-04-21T08:44:48.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.755+0000 7fee1e7fc640 1 -- 172.21.15.86:0/1137484771 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee1802cbe0 con 0x7fee28106160 2024-04-21T08:44:48.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.755+0000 7fee1e7fc640 1 -- 172.21.15.86:0/1137484771 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee18003bf0 con 0x7fee28106160 2024-04-21T08:44:48.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.755+0000 7fee2f606640 1 -- 172.21.15.86:0/1137484771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fee281a55d0 con 0x7fee28106160 2024-04-21T08:44:48.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.757+0000 7fee1e7fc640 1 -- 172.21.15.86:0/1137484771 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fee18046020 con 0x7fee28106160 2024-04-21T08:44:48.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.758+0000 7fee321f3740 1 Processor -- start 2024-04-21T08:44:48.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.758+0000 7fee321f3740 1 -- start start 2024-04-21T08:44:48.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.758+0000 7fee1e7fc640 1 --2- 172.21.15.86:0/1137484771 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fee140a3d20 0x7fee140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.758+0000 7fee321f3740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da696a5330 0x55da696a5710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.758+0000 7fee321f3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da696a5ce0 0x55da696ae820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.758+0000 7fee321f3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da696aee40 0x55da696af2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.758+0000 7fee321f3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55da6960a1f0 con 0x55da696a5ce0 2024-04-21T08:44:48.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.758+0000 7fee321f3740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55da695a74b0 con 0x55da696a5330 2024-04-21T08:44:48.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.758+0000 7fee321f3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55da696061b0 con 0x55da696aee40 2024-04-21T08:44:48.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.758+0000 7fee2dba3640 1 --2- 172.21.15.86:0/1137484771 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fee140a3d20 0x7fee140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.758+0000 7fee1e7fc640 1 -- 172.21.15.86:0/1137484771 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fee180f2d90 con 0x7fee28106160 2024-04-21T08:44:48.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.758+0000 7fee2cba1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da696a5ce0 0x55da696ae820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.758+0000 7fee2d3a2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da696a5330 0x55da696a5710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.758+0000 7fee2cba1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da696a5ce0 0x55da696ae820 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42432/0 (socket says 172.21.15.86:42432) 2024-04-21T08:44:48.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.758+0000 7fee2cba1640 1 -- 172.21.15.86:0/4227811439 learned_addr learned my addr 172.21.15.86:0/4227811439 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:48.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.758+0000 7fee2dba3640 1 --2- 172.21.15.86:0/4227811439 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da696aee40 0x55da696af2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.758+0000 7fee2cba1640 1 -- 172.21.15.86:0/4227811439 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da696aee40 msgr2=0x55da696af2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.759+0000 7fee2cba1640 1 --2- 172.21.15.86:0/4227811439 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da696aee40 0x55da696af2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.759+0000 7fee2cba1640 1 -- 172.21.15.86:0/4227811439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da696a5330 msgr2=0x55da696a5710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.759+0000 7fee2cba1640 1 --2- 172.21.15.86:0/4227811439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da696a5330 0x55da696a5710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.759+0000 7fee2dba3640 1 --2- 172.21.15.86:0/1137484771 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fee140a3d20 0x7fee140a61e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fee200092f0 tx=0x7fee20010740 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.759+0000 7fee2cba1640 1 -- 172.21.15.86:0/4227811439 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da695991b0 con 0x55da696a5ce0 2024-04-21T08:44:48.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.759+0000 7fee2cba1640 1 --2- 172.21.15.86:0/4227811439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da696a5ce0 0x55da696ae820 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fee180bb4a0 tx=0x7fee180be6d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=763d2584f7e1a8f9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.759+0000 7fee137fe640 1 -- 172.21.15.86:0/4227811439 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee180df110 con 0x55da696a5ce0 2024-04-21T08:44:48.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.759+0000 7fee137fe640 1 -- 172.21.15.86:0/4227811439 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee180bead0 con 0x55da696a5ce0 2024-04-21T08:44:48.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.759+0000 7fee137fe640 1 -- 172.21.15.86:0/4227811439 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee180054d0 con 0x55da696a5ce0 2024-04-21T08:44:48.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.760+0000 7fee321f3740 1 -- 172.21.15.86:0/4227811439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da696a5ce0 msgr2=0x55da696ae820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.760+0000 7fee321f3740 1 --2- 172.21.15.86:0/4227811439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da696a5ce0 0x55da696ae820 secure :-1 s=READY pgs=1332 cs=0 l=1 rev1=1 crypto rx=0x7fee180bb4a0 tx=0x7fee180be6d0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.760+0000 7fee321f3740 1 -- 172.21.15.86:0/4227811439 shutdown_connections 2024-04-21T08:44:48.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.760+0000 7fee321f3740 1 --2- 172.21.15.86:0/4227811439 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da696aee40 0x55da696af2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.760+0000 7fee321f3740 1 --2- 172.21.15.86:0/4227811439 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da696a5ce0 0x55da696ae820 unknown :-1 s=CLOSED pgs=1332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.760+0000 7fee321f3740 1 --2- 172.21.15.86:0/4227811439 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da696a5330 0x55da696a5710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.760+0000 7fee321f3740 1 -- 172.21.15.86:0/4227811439 >> 172.21.15.86:0/4227811439 conn(0x55da6969fba0 msgr2=0x55da6969ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:48.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.760+0000 7fee321f3740 1 -- 172.21.15.86:0/4227811439 shutdown_connections 2024-04-21T08:44:48.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.760+0000 7fee321f3740 1 -- 172.21.15.86:0/4227811439 wait complete. 2024-04-21T08:44:48.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.760+0000 7fee321f3740 1 -- start start 2024-04-21T08:44:48.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.760+0000 7fee321f3740 1 Processor -- start 2024-04-21T08:44:48.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.760+0000 7fee321f3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da696a5330 0x55da696b51a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.761+0000 7fee321f3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da696a5ce0 0x55da696b2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.761+0000 7fee321f3740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da696aee40 0x55da696b2a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.761+0000 7fee321f3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55da696b5b80 con 0x55da696a5330 2024-04-21T08:44:48.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.761+0000 7fee321f3740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55da695c3500 con 0x55da696aee40 2024-04-21T08:44:48.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.761+0000 7fee321f3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55da696b5d00 con 0x55da696a5ce0 2024-04-21T08:44:48.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.761+0000 7fee321f3740 10 client.0 _ll_register_callbacks cb 0x55da69591b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:48.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.761+0000 7fee2d3a2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da696a5330 0x55da696b51a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.761+0000 7fee2d3a2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da696a5330 0x55da696b51a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42438/0 (socket says 172.21.15.86:42438) 2024-04-21T08:44:48.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.761+0000 7fee2d3a2640 1 -- 172.21.15.86:0/1075029227 learned_addr learned my addr 172.21.15.86:0/1075029227 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:48.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.761+0000 7fee2dba3640 1 --2- 172.21.15.86:0/1075029227 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da696aee40 0x55da696b2a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.761+0000 7fee2cba1640 1 --2- 172.21.15.86:0/1075029227 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da696a5ce0 0x55da696b2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.761+0000 7fee2d3a2640 1 -- 172.21.15.86:0/1075029227 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da696a5ce0 msgr2=0x55da696b2530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.761+0000 7fee2d3a2640 1 --2- 172.21.15.86:0/1075029227 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da696a5ce0 0x55da696b2530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.761+0000 7fee2d3a2640 1 -- 172.21.15.86:0/1075029227 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da696aee40 msgr2=0x55da696b2a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.761+0000 7fee2d3a2640 1 --2- 172.21.15.86:0/1075029227 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da696aee40 0x55da696b2a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.761+0000 7fee2d3a2640 1 -- 172.21.15.86:0/1075029227 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da6931c3f0 con 0x55da696a5330 2024-04-21T08:44:48.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.761+0000 7fee2d3a2640 1 --2- 172.21.15.86:0/1075029227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da696a5330 0x55da696b51a0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fee24011a30 tx=0x7fee24011ef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67969a8802a0c80 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.761+0000 7fee10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:48.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.761+0000 7fee10ff9640 1 -- 172.21.15.86:0/1075029227 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2400fdf0 con 0x55da696a5330 2024-04-21T08:44:48.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.761+0000 7fee10ff9640 1 -- 172.21.15.86:0/1075029227 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fee24005020 con 0x55da696a5330 2024-04-21T08:44:48.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.761+0000 7fee321f3740 1 -- 172.21.15.86:0/1075029227 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee180bead0 con 0x55da696a5330 2024-04-21T08:44:48.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.761+0000 7fee10ff9640 1 -- 172.21.15.86:0/1075029227 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fee24024050 con 0x55da696a5330 2024-04-21T08:44:48.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.761+0000 7fee321f3740 10 client.18174 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:48.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.761+0000 7fee321f3740 1 -- 172.21.15.86:0/1075029227 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55da693e67d0 con 0x55da696a5330 2024-04-21T08:44:48.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.762+0000 7fee10ff9640 1 -- 172.21.15.86:0/1075029227 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee180bead0 con 0x55da696a5330 2024-04-21T08:44:48.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.762+0000 7fee10ff9640 1 -- 172.21.15.86:0/1075029227 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fee24018030 con 0x55da696a5330 2024-04-21T08:44:48.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.762+0000 7fee321f3740 10 client.18174 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:48.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.762+0000 7fee321f3740 10 client.18174 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:44:48.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.762+0000 7fee321f3740 20 client.18174 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:48.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.762+0000 7fee321f3740 1 --2- 172.21.15.86:0/1075029227 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55da696c2d30 0x55da696c5180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.762+0000 7fee321f3740 4 client.18174 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:48.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.762+0000 7fee321f3740 1 -- 172.21.15.86:0/1075029227 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55da695a9930 con 0x55da696c2d30 2024-04-21T08:44:48.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.762+0000 7fee2cba1640 1 --2- 172.21.15.86:0/1075029227 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55da696c2d30 0x55da696c5180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.762+0000 7fee10ff9640 10 client.18174 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:44:48.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.762+0000 7fee2cba1640 1 --2- 172.21.15.86:0/1075029227 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55da696c2d30 0x55da696c5180 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fecb6a00be0efec8 server_cookie=9a907cd59da801f3 in_seq=0 out_seq=0 2024-04-21T08:44:48.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.763+0000 7fee10ff9640 1 -- 172.21.15.86:0/1075029227 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55da695a9930 con 0x55da696c2d30 2024-04-21T08:44:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.763+0000 7fee10ff9640 10 client.18174 handle_command_reply: tid=0 2024-04-21T08:44:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.763+0000 7fee321f3740 1 client.18174 shutdown 2024-04-21T08:44:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.763+0000 7fee321f3740 1 -- 172.21.15.86:0/1075029227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da696a5330 msgr2=0x55da696b51a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.763+0000 7fee321f3740 1 --2- 172.21.15.86:0/1075029227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da696a5330 0x55da696b51a0 secure :-1 s=READY pgs=1333 cs=0 l=1 rev1=1 crypto rx=0x7fee24011a30 tx=0x7fee24011ef0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.763+0000 7fee321f3740 1 -- 172.21.15.86:0/1075029227 shutdown_connections 2024-04-21T08:44:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.763+0000 7fee321f3740 1 --2- 172.21.15.86:0/1075029227 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55da696c2d30 0x55da696c5180 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.763+0000 7fee321f3740 1 --2- 172.21.15.86:0/1075029227 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da696aee40 0x55da696b2a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.763+0000 7fee321f3740 1 --2- 172.21.15.86:0/1075029227 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da696a5ce0 0x55da696b2530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.763+0000 7fee321f3740 1 --2- 172.21.15.86:0/1075029227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da696a5330 0x55da696b51a0 unknown :-1 s=CLOSED pgs=1333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.763+0000 7fee10ff9640 0 client.18174 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:44:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.763+0000 7fee321f3740 1 -- 172.21.15.86:0/1075029227 >> 172.21.15.86:0/1075029227 conn(0x55da6969fba0 msgr2=0x55da696a18f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.763+0000 7fee321f3740 1 -- 172.21.15.86:0/1075029227 shutdown_connections 2024-04-21T08:44:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.763+0000 7fee321f3740 1 -- 172.21.15.86:0/1075029227 wait complete. 2024-04-21T08:44:48.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.763+0000 7fee321f3740 20 client.18174 trim_cache size 0 max 16384 2024-04-21T08:44:48.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.777+0000 7fee321f3740 1 Processor -- start 2024-04-21T08:44:48.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.777+0000 7fee321f3740 1 -- start start 2024-04-21T08:44:48.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.777+0000 7fee321f3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da696a5330 0x55da696be8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.777+0000 7fee321f3740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da696a5ce0 0x55da696b1510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.777+0000 7fee321f3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da696aee40 0x55da696b4c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.777+0000 7fee321f3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee281ad0c0 con 0x55da696aee40 2024-04-21T08:44:48.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.777+0000 7fee321f3740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fee281aec00 con 0x55da696a5ce0 2024-04-21T08:44:48.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.777+0000 7fee321f3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fee281ad790 con 0x55da696a5330 2024-04-21T08:44:48.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.777+0000 7fee2dba3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da696aee40 0x55da696b4c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.777+0000 7fee2dba3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da696aee40 0x55da696b4c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42450/0 (socket says 172.21.15.86:42450) 2024-04-21T08:44:48.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.777+0000 7fee2dba3640 1 -- 172.21.15.86:0/4038040837 learned_addr learned my addr 172.21.15.86:0/4038040837 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:48.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.777+0000 7fee2d3a2640 1 --2- 172.21.15.86:0/4038040837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da696a5330 0x55da696be8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.777+0000 7fee2cba1640 1 --2- 172.21.15.86:0/4038040837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da696a5ce0 0x55da696b1510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.777+0000 7fee2dba3640 1 -- 172.21.15.86:0/4038040837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da696a5330 msgr2=0x55da696be8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.777+0000 7fee2dba3640 1 --2- 172.21.15.86:0/4038040837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da696a5330 0x55da696be8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.777+0000 7fee2dba3640 1 -- 172.21.15.86:0/4038040837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da696a5ce0 msgr2=0x55da696b1510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.777+0000 7fee2dba3640 1 --2- 172.21.15.86:0/4038040837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da696a5ce0 0x55da696b1510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.777+0000 7fee2dba3640 1 -- 172.21.15.86:0/4038040837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55da696a4560 con 0x55da696aee40 2024-04-21T08:44:48.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.777+0000 7fee2cba1640 1 --2- 172.21.15.86:0/4038040837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da696a5ce0 0x55da696b1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:48.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.777+0000 7fee2dba3640 1 --2- 172.21.15.86:0/4038040837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da696aee40 0x55da696b4c60 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fee20012750 tx=0x7fee20015000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a11759f303ca0bac server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.777+0000 7fee12ffd640 1 -- 172.21.15.86:0/4038040837 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee2001f850 con 0x55da696aee40 2024-04-21T08:44:48.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.777+0000 7fee12ffd640 1 -- 172.21.15.86:0/4038040837 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee200273f0 con 0x55da696aee40 2024-04-21T08:44:48.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.777+0000 7fee12ffd640 1 -- 172.21.15.86:0/4038040837 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee20011640 con 0x55da696aee40 2024-04-21T08:44:48.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.778+0000 7fee321f3740 1 -- 172.21.15.86:0/4038040837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da696aee40 msgr2=0x55da696b4c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.778+0000 7fee321f3740 1 --2- 172.21.15.86:0/4038040837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da696aee40 0x55da696b4c60 secure :-1 s=READY pgs=1334 cs=0 l=1 rev1=1 crypto rx=0x7fee20012750 tx=0x7fee20015000 comp rx=0 tx=0).stop 2024-04-21T08:44:48.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.778+0000 7fee321f3740 1 -- 172.21.15.86:0/4038040837 shutdown_connections 2024-04-21T08:44:48.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.778+0000 7fee321f3740 1 --2- 172.21.15.86:0/4038040837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da696aee40 0x55da696b4c60 unknown :-1 s=CLOSED pgs=1334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.778+0000 7fee321f3740 1 --2- 172.21.15.86:0/4038040837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da696a5ce0 0x55da696b1510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.778+0000 7fee321f3740 1 --2- 172.21.15.86:0/4038040837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da696a5330 0x55da696be8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.778+0000 7fee321f3740 1 -- 172.21.15.86:0/4038040837 >> 172.21.15.86:0/4038040837 conn(0x55da6969fba0 msgr2=0x55da696b0e00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:48.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.778+0000 7fee321f3740 1 -- 172.21.15.86:0/4038040837 shutdown_connections 2024-04-21T08:44:48.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.778+0000 7fee321f3740 1 -- 172.21.15.86:0/4038040837 wait complete. 2024-04-21T08:44:48.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.778+0000 7fee321f3740 1 -- start start 2024-04-21T08:44:48.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.778+0000 7fee321f3740 1 Processor -- start 2024-04-21T08:44:48.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.779+0000 7fee321f3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da696a5330 0x55da696badd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.779+0000 7fee321f3740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da696a5ce0 0x55da696bb310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.779+0000 7fee321f3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da696aee40 0x55da696b3b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.779+0000 7fee321f3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fee281ae530 con 0x55da696aee40 2024-04-21T08:44:48.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.779+0000 7fee321f3740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fee281ade60 con 0x55da696a5ce0 2024-04-21T08:44:48.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.779+0000 7fee321f3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fee281ab080 con 0x55da696a5330 2024-04-21T08:44:48.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.779+0000 7fee321f3740 10 client.0 _ll_register_callbacks cb 0x55da69591b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:48.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.779+0000 7fee2dba3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da696aee40 0x55da696b3b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.779+0000 7fee2dba3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da696aee40 0x55da696b3b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42456/0 (socket says 172.21.15.86:42456) 2024-04-21T08:44:48.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.779+0000 7fee2dba3640 1 -- 172.21.15.86:0/1359881292 learned_addr learned my addr 172.21.15.86:0/1359881292 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:48.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.779+0000 7fee2d3a2640 1 --2- 172.21.15.86:0/1359881292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da696a5330 0x55da696badd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.779+0000 7fee2cba1640 1 --2- 172.21.15.86:0/1359881292 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da696a5ce0 0x55da696bb310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.779+0000 7fee2dba3640 1 -- 172.21.15.86:0/1359881292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da696a5330 msgr2=0x55da696badd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.779+0000 7fee2dba3640 1 --2- 172.21.15.86:0/1359881292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da696a5330 0x55da696badd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.779+0000 7fee2dba3640 1 -- 172.21.15.86:0/1359881292 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da696a5ce0 msgr2=0x55da696bb310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.779+0000 7fee2dba3640 1 --2- 172.21.15.86:0/1359881292 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da696a5ce0 0x55da696bb310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.779+0000 7fee2dba3640 1 -- 172.21.15.86:0/1359881292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55da696b4330 con 0x55da696aee40 2024-04-21T08:44:48.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.779+0000 7fee2cba1640 1 --2- 172.21.15.86:0/1359881292 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da696a5ce0 0x55da696bb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:48.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.779+0000 7fee2dba3640 1 --2- 172.21.15.86:0/1359881292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da696aee40 0x55da696b3b00 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fee200096a0 tx=0x7fee20009280 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ffea5b0e665a9655 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.779+0000 7fee10ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:48.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.779+0000 7fee10ff9640 1 -- 172.21.15.86:0/1359881292 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fee20035020 con 0x55da696aee40 2024-04-21T08:44:48.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.779+0000 7fee10ff9640 1 -- 172.21.15.86:0/1359881292 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fee20004c40 con 0x55da696aee40 2024-04-21T08:44:48.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.779+0000 7fee321f3740 1 -- 172.21.15.86:0/1359881292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fee200273f0 con 0x55da696aee40 2024-04-21T08:44:48.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.780+0000 7fee10ff9640 1 -- 172.21.15.86:0/1359881292 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fee20004e40 con 0x55da696aee40 2024-04-21T08:44:48.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.780+0000 7fee321f3740 10 client.18180 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:48.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.780+0000 7fee321f3740 1 -- 172.21.15.86:0/1359881292 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55da69728d70 con 0x55da696aee40 2024-04-21T08:44:48.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.780+0000 7fee10ff9640 1 -- 172.21.15.86:0/1359881292 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fee200273f0 con 0x55da696aee40 2024-04-21T08:44:48.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.780+0000 7fee10ff9640 1 -- 172.21.15.86:0/1359881292 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fee20026030 con 0x55da696aee40 2024-04-21T08:44:48.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.780+0000 7fee321f3740 10 client.18180 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:48.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.780+0000 7fee321f3740 10 client.18180 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:44:48.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.780+0000 7fee321f3740 20 client.18180 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:48.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.780+0000 7fee321f3740 1 --2- 172.21.15.86:0/1359881292 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55da697294b0 0x55da6972b8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.780+0000 7fee321f3740 4 client.18180 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:44:48.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.780+0000 7fee321f3740 1 -- 172.21.15.86:0/1359881292 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55da69617c10 con 0x55da697294b0 2024-04-21T08:44:48.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.780+0000 7fee2d3a2640 1 --2- 172.21.15.86:0/1359881292 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55da697294b0 0x55da6972b8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.781+0000 7fee10ff9640 10 client.18180 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:44:48.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.781+0000 7fee2d3a2640 1 --2- 172.21.15.86:0/1359881292 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55da697294b0 0x55da6972b8d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=7fbfef6024fb5eae server_cookie=1ca09f4b77f20d57 in_seq=0 out_seq=0 2024-04-21T08:44:48.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.781+0000 7fee10ff9640 1 -- 172.21.15.86:0/1359881292 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55da69617c10 con 0x55da697294b0 2024-04-21T08:44:48.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.781+0000 7fee10ff9640 10 client.18180 handle_command_reply: tid=0 2024-04-21T08:44:48.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.781+0000 7fee321f3740 1 client.18180 shutdown 2024-04-21T08:44:48.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.781+0000 7fee321f3740 1 -- 172.21.15.86:0/1359881292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da696aee40 msgr2=0x55da696b3b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.781+0000 7fee321f3740 1 --2- 172.21.15.86:0/1359881292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da696aee40 0x55da696b3b00 secure :-1 s=READY pgs=1335 cs=0 l=1 rev1=1 crypto rx=0x7fee200096a0 tx=0x7fee20009280 comp rx=0 tx=0).stop 2024-04-21T08:44:48.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.781+0000 7fee321f3740 1 -- 172.21.15.86:0/1359881292 shutdown_connections 2024-04-21T08:44:48.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.781+0000 7fee321f3740 1 --2- 172.21.15.86:0/1359881292 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55da697294b0 0x55da6972b8d0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.781+0000 7fee321f3740 1 --2- 172.21.15.86:0/1359881292 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55da696aee40 0x55da696b3b00 unknown :-1 s=CLOSED pgs=1335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.781+0000 7fee10ff9640 0 client.18180 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:44:48.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.781+0000 7fee321f3740 1 --2- 172.21.15.86:0/1359881292 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55da696a5ce0 0x55da696bb310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.781+0000 7fee321f3740 1 --2- 172.21.15.86:0/1359881292 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55da696a5330 0x55da696badd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.781+0000 7fee321f3740 1 -- 172.21.15.86:0/1359881292 >> 172.21.15.86:0/1359881292 conn(0x55da6969fba0 msgr2=0x55da696b1690 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:48.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.781+0000 7fee321f3740 1 -- 172.21.15.86:0/1359881292 shutdown_connections 2024-04-21T08:44:48.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.781+0000 7fee321f3740 1 -- 172.21.15.86:0/1359881292 wait complete. 2024-04-21T08:44:48.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.781+0000 7fee321f3740 20 client.18180 trim_cache size 0 max 16384 2024-04-21T08:44:48.789 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:44:48.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.782+0000 7fee2f606640 1 -- 172.21.15.86:0/1137484771 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fee140a3d20 msgr2=0x7fee140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.782+0000 7fee2f606640 1 --2- 172.21.15.86:0/1137484771 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fee140a3d20 0x7fee140a61e0 secure :-1 s=READY pgs=211 cs=0 l=1 rev1=1 crypto rx=0x7fee200092f0 tx=0x7fee20010740 comp rx=0 tx=0).stop 2024-04-21T08:44:48.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.782+0000 7fee2f606640 1 -- 172.21.15.86:0/1137484771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee28106160 msgr2=0x7fee28199a60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.782+0000 7fee2f606640 1 --2- 172.21.15.86:0/1137484771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee28106160 0x7fee28199a60 secure :-1 s=READY pgs=1331 cs=0 l=1 rev1=1 crypto rx=0x7fee18036040 tx=0x7fee1802c7e0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.782+0000 7fee2f606640 1 -- 172.21.15.86:0/1137484771 shutdown_connections 2024-04-21T08:44:48.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.782+0000 7fee2f606640 1 --2- 172.21.15.86:0/1137484771 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fee140a3d20 0x7fee140a61e0 unknown :-1 s=CLOSED pgs=211 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.782+0000 7fee2f606640 1 --2- 172.21.15.86:0/1137484771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fee28199ff0 0x7fee281a45a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.782+0000 7fee2f606640 1 --2- 172.21.15.86:0/1137484771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fee28106160 0x7fee28199a60 unknown :-1 s=CLOSED pgs=1331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.782+0000 7fee2f606640 1 --2- 172.21.15.86:0/1137484771 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fee28101090 0x7fee28199520 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.782+0000 7fee2f606640 1 -- 172.21.15.86:0/1137484771 >> 172.21.15.86:0/1137484771 conn(0x7fee28077f50 msgr2=0x7fee280feff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:48.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.782+0000 7fee2f606640 1 -- 172.21.15.86:0/1137484771 shutdown_connections 2024-04-21T08:44:48.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.782+0000 7fee2f606640 1 -- 172.21.15.86:0/1137484771 wait complete. 2024-04-21T08:44:48.792 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:44:48.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.893+0000 7f832b198640 1 -- 172.21.15.86:0/4085147796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8324107030 msgr2=0x7f8324107410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.893+0000 7f832b198640 1 --2- 172.21.15.86:0/4085147796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8324107030 0x7f8324107410 secure :-1 s=READY pgs=1336 cs=0 l=1 rev1=1 crypto rx=0x7f8318009930 tx=0x7f831802bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:48.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.893+0000 7f832b198640 1 -- 172.21.15.86:0/4085147796 shutdown_connections 2024-04-21T08:44:48.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.893+0000 7f832b198640 1 --2- 172.21.15.86:0/4085147796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8324101b40 0x7f832410dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.893+0000 7f832b198640 1 --2- 172.21.15.86:0/4085147796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f83241011a0 0x7f8324101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.893+0000 7f832b198640 1 --2- 172.21.15.86:0/4085147796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8324107030 0x7f8324107410 unknown :-1 s=CLOSED pgs=1336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.893+0000 7f832b198640 1 -- 172.21.15.86:0/4085147796 >> 172.21.15.86:0/4085147796 conn(0x7f8324076c50 msgr2=0x7f8324077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:48.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.893+0000 7f832b198640 1 -- 172.21.15.86:0/4085147796 shutdown_connections 2024-04-21T08:44:48.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.893+0000 7f832b198640 1 -- 172.21.15.86:0/4085147796 wait complete. 2024-04-21T08:44:48.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.893+0000 7f832b198640 1 Processor -- start 2024-04-21T08:44:48.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.893+0000 7f832b198640 1 -- start start 2024-04-21T08:44:48.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.894+0000 7f832b198640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83241011a0 0x7f83241995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.894+0000 7f832b198640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8324101b40 0x7f8324199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.894+0000 7f832b198640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8324107030 0x7f83241a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.894+0000 7f832b198640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f83241110e0 con 0x7f8324107030 2024-04-21T08:44:48.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.894+0000 7f832b198640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8324110f60 con 0x7f8324101b40 2024-04-21T08:44:48.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.894+0000 7f832b198640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8324111260 con 0x7f83241011a0 2024-04-21T08:44:48.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.894+0000 7f8329735640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8324107030 0x7f83241a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.894+0000 7f8329735640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8324107030 0x7f83241a4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42474/0 (socket says 172.21.15.86:42474) 2024-04-21T08:44:48.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.894+0000 7f8329735640 1 -- 172.21.15.86:0/642637340 learned_addr learned my addr 172.21.15.86:0/642637340 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:48.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.894+0000 7f8323fff640 1 --2- 172.21.15.86:0/642637340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8324101b40 0x7f8324199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.894+0000 7f8328f34640 1 --2- 172.21.15.86:0/642637340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83241011a0 0x7f83241995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.894+0000 7f8329735640 1 -- 172.21.15.86:0/642637340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83241011a0 msgr2=0x7f83241995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.894+0000 7f8329735640 1 --2- 172.21.15.86:0/642637340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83241011a0 0x7f83241995a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.894+0000 7f8329735640 1 -- 172.21.15.86:0/642637340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8324101b40 msgr2=0x7f8324199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.894+0000 7f8329735640 1 --2- 172.21.15.86:0/642637340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8324101b40 0x7f8324199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.894+0000 7f8329735640 1 -- 172.21.15.86:0/642637340 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f83241a4b60 con 0x7f8324107030 2024-04-21T08:44:48.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.894+0000 7f8323fff640 1 --2- 172.21.15.86:0/642637340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8324101b40 0x7f8324199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:48.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.894+0000 7f8329735640 1 --2- 172.21.15.86:0/642637340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8324107030 0x7f83241a4620 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f831c00eaa0 tx=0x7f831c00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.894+0000 7f8321ffb640 1 -- 172.21.15.86:0/642637340 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c00cdb0 con 0x7f8324107030 2024-04-21T08:44:48.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.894+0000 7f8321ffb640 1 -- 172.21.15.86:0/642637340 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f831c005020 con 0x7f8324107030 2024-04-21T08:44:48.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.894+0000 7f832b198640 1 -- 172.21.15.86:0/642637340 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f83241a4d30 con 0x7f8324107030 2024-04-21T08:44:48.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.894+0000 7f8321ffb640 1 -- 172.21.15.86:0/642637340 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c0116d0 con 0x7f8324107030 2024-04-21T08:44:48.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.895+0000 7f832b198640 1 -- 172.21.15.86:0/642637340 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f83241a5250 con 0x7f8324107030 2024-04-21T08:44:48.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.897+0000 7f8321ffb640 1 -- 172.21.15.86:0/642637340 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f831c0051d0 con 0x7f8324107030 2024-04-21T08:44:48.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.897+0000 7f832dd81740 1 Processor -- start 2024-04-21T08:44:48.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.897+0000 7f832dd81740 1 -- start start 2024-04-21T08:44:48.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.897+0000 7f8321ffb640 1 --2- 172.21.15.86:0/642637340 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f83140a3d20 0x7f83140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.897+0000 7f832dd81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f3ba14d0 0x5629f3ba18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.897+0000 7f832dd81740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629f3ba1e80 0x5629f3baa430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.897+0000 7f832dd81740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629f3baaa00 0x5629f3bace50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.897+0000 7f832dd81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5629f3b061f0 con 0x5629f3ba14d0 2024-04-21T08:44:48.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.897+0000 7f832dd81740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5629f3aa34b0 con 0x5629f3ba1e80 2024-04-21T08:44:48.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.897+0000 7f832dd81740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5629f3b021b0 con 0x5629f3baaa00 2024-04-21T08:44:48.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.897+0000 7f8323fff640 1 --2- 172.21.15.86:0/642637340 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f83140a3d20 0x7f83140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.897+0000 7f8328f34640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f3ba14d0 0x5629f3ba18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.898+0000 7f8328f34640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f3ba14d0 0x5629f3ba18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42490/0 (socket says 172.21.15.86:42490) 2024-04-21T08:44:48.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.898+0000 7f8328f34640 1 -- 172.21.15.86:0/357201248 learned_addr learned my addr 172.21.15.86:0/357201248 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:48.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.898+0000 7f8321ffb640 1 -- 172.21.15.86:0/642637340 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f831c0d0d90 con 0x7f8324107030 2024-04-21T08:44:48.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.898+0000 7f8323fff640 1 --2- 172.21.15.86:0/357201248 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629f3ba1e80 0x5629f3baa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.898+0000 7f8329735640 1 --2- 172.21.15.86:0/357201248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629f3baaa00 0x5629f3bace50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.898+0000 7f8328f34640 1 -- 172.21.15.86:0/357201248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629f3baaa00 msgr2=0x5629f3bace50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.898+0000 7f8328f34640 1 --2- 172.21.15.86:0/357201248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629f3baaa00 0x5629f3bace50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.898+0000 7f8328f34640 1 -- 172.21.15.86:0/357201248 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629f3ba1e80 msgr2=0x5629f3baa430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.898+0000 7f8328f34640 1 --2- 172.21.15.86:0/357201248 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629f3ba1e80 0x5629f3baa430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.898+0000 7f8328f34640 1 -- 172.21.15.86:0/357201248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629f3a951b0 con 0x5629f3ba14d0 2024-04-21T08:44:48.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.898+0000 7f8329735640 1 --2- 172.21.15.86:0/357201248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629f3baaa00 0x5629f3bace50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:48.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.898+0000 7f8328f34640 1 --2- 172.21.15.86:0/357201248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f3ba14d0 0x5629f3ba18b0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f8318006010 tx=0x7f8318005dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cff52e1c937b4f8a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.898+0000 7f8323fff640 1 --2- 172.21.15.86:0/642637340 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f83140a3d20 0x7f83140a61e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f832406b0a0 tx=0x7f8310013820 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.898+0000 7f830effd640 1 -- 172.21.15.86:0/357201248 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8318043020 con 0x5629f3ba14d0 2024-04-21T08:44:48.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.898+0000 7f830effd640 1 -- 172.21.15.86:0/357201248 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f831802c7e0 con 0x5629f3ba14d0 2024-04-21T08:44:48.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.898+0000 7f830effd640 1 -- 172.21.15.86:0/357201248 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8318005410 con 0x5629f3ba14d0 2024-04-21T08:44:48.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.899+0000 7f832dd81740 1 -- 172.21.15.86:0/357201248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f3ba14d0 msgr2=0x5629f3ba18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.899+0000 7f832dd81740 1 --2- 172.21.15.86:0/357201248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f3ba14d0 0x5629f3ba18b0 secure :-1 s=READY pgs=1338 cs=0 l=1 rev1=1 crypto rx=0x7f8318006010 tx=0x7f8318005dd0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.899+0000 7f832dd81740 1 -- 172.21.15.86:0/357201248 shutdown_connections 2024-04-21T08:44:48.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.899+0000 7f832dd81740 1 --2- 172.21.15.86:0/357201248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629f3baaa00 0x5629f3bace50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.899+0000 7f832dd81740 1 --2- 172.21.15.86:0/357201248 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629f3ba1e80 0x5629f3baa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.899+0000 7f832dd81740 1 --2- 172.21.15.86:0/357201248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f3ba14d0 0x5629f3ba18b0 unknown :-1 s=CLOSED pgs=1338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.899+0000 7f832dd81740 1 -- 172.21.15.86:0/357201248 >> 172.21.15.86:0/357201248 conn(0x5629f3b9bba0 msgr2=0x5629f3b9bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:48.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.899+0000 7f832dd81740 1 -- 172.21.15.86:0/357201248 shutdown_connections 2024-04-21T08:44:48.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.899+0000 7f832dd81740 1 -- 172.21.15.86:0/357201248 wait complete. 2024-04-21T08:44:48.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.899+0000 7f832dd81740 1 -- start start 2024-04-21T08:44:48.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.899+0000 7f832dd81740 1 Processor -- start 2024-04-21T08:44:48.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.900+0000 7f832dd81740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629f3ba14d0 0x5629f3baa2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.900+0000 7f832dd81740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629f3ba1e80 0x5629f3ba7670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.900+0000 7f832dd81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f3baaa00 0x5629f3ba7bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.900+0000 7f832dd81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5629f3bad910 con 0x5629f3baaa00 2024-04-21T08:44:48.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.900+0000 7f832dd81740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5629f3abf500 con 0x5629f3ba1e80 2024-04-21T08:44:48.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.900+0000 7f832dd81740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5629f3bada90 con 0x5629f3ba14d0 2024-04-21T08:44:48.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.900+0000 7f832dd81740 10 client.0 _ll_register_callbacks cb 0x5629f3a8db80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:48.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.900+0000 7f8329735640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f3baaa00 0x5629f3ba7bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.900+0000 7f8329735640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f3baaa00 0x5629f3ba7bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42496/0 (socket says 172.21.15.86:42496) 2024-04-21T08:44:48.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.900+0000 7f8329735640 1 -- 172.21.15.86:0/607866877 learned_addr learned my addr 172.21.15.86:0/607866877 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:48.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.900+0000 7f8323fff640 1 --2- 172.21.15.86:0/607866877 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629f3ba1e80 0x5629f3ba7670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.900+0000 7f8329735640 1 -- 172.21.15.86:0/607866877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629f3ba14d0 msgr2=0x5629f3baa2e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.900+0000 7f8328f34640 1 --2- 172.21.15.86:0/607866877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629f3ba14d0 0x5629f3baa2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.900+0000 7f8329735640 1 --2- 172.21.15.86:0/607866877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629f3ba14d0 0x5629f3baa2e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.900+0000 7f8329735640 1 -- 172.21.15.86:0/607866877 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629f3ba1e80 msgr2=0x5629f3ba7670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.900+0000 7f8329735640 1 --2- 172.21.15.86:0/607866877 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629f3ba1e80 0x5629f3ba7670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.900+0000 7f8329735640 1 -- 172.21.15.86:0/607866877 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629f38183f0 con 0x5629f3baaa00 2024-04-21T08:44:48.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.900+0000 7f8323fff640 1 --2- 172.21.15.86:0/607866877 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629f3ba1e80 0x5629f3ba7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:48.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.900+0000 7f8329735640 1 --2- 172.21.15.86:0/607866877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f3baaa00 0x5629f3ba7bd0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f831c09c950 tx=0x7f831c09c9c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2821b8b15b42a2a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.900+0000 7f8303fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:48.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.900+0000 7f8303fff640 1 -- 172.21.15.86:0/607866877 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c01f070 con 0x5629f3baaa00 2024-04-21T08:44:48.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.900+0000 7f8303fff640 1 -- 172.21.15.86:0/607866877 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f831c0ce3f0 con 0x5629f3baaa00 2024-04-21T08:44:48.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.900+0000 7f832dd81740 1 -- 172.21.15.86:0/607866877 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f831802c7e0 con 0x5629f3baaa00 2024-04-21T08:44:48.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.900+0000 7f8303fff640 1 -- 172.21.15.86:0/607866877 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f831c0ce5f0 con 0x5629f3baaa00 2024-04-21T08:44:48.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.900+0000 7f832dd81740 10 client.18192 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:48.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.900+0000 7f832dd81740 1 -- 172.21.15.86:0/607866877 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5629f38e27d0 con 0x5629f3baaa00 2024-04-21T08:44:48.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.900+0000 7f8303fff640 1 -- 172.21.15.86:0/607866877 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f831802c7e0 con 0x5629f3baaa00 2024-04-21T08:44:48.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.901+0000 7f8303fff640 1 -- 172.21.15.86:0/607866877 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f831c00f040 con 0x5629f3baaa00 2024-04-21T08:44:48.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.901+0000 7f832dd81740 10 client.18192 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:48.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.901+0000 7f832dd81740 10 client.18192 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:44:48.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.901+0000 7f832dd81740 20 client.18192 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:48.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.901+0000 7f832dd81740 1 --2- 172.21.15.86:0/607866877 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5629f3bbaa60 0x5629f3bbce90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.901+0000 7f832dd81740 4 client.18192 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:48.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.901+0000 7f832dd81740 1 -- 172.21.15.86:0/607866877 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5629f3aa5930 con 0x5629f3bbaa60 2024-04-21T08:44:48.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.901+0000 7f8328f34640 1 --2- 172.21.15.86:0/607866877 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5629f3bbaa60 0x5629f3bbce90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.901+0000 7f8303fff640 10 client.18192 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:44:48.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.901+0000 7f8328f34640 1 --2- 172.21.15.86:0/607866877 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5629f3bbaa60 0x5629f3bbce90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=68e76856bd4a205a server_cookie=173cc505d0635f39 in_seq=0 out_seq=0 2024-04-21T08:44:48.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.902+0000 7f8303fff640 1 -- 172.21.15.86:0/607866877 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5629f3aa5930 con 0x5629f3bbaa60 2024-04-21T08:44:48.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.902+0000 7f8303fff640 10 client.18192 handle_command_reply: tid=0 2024-04-21T08:44:48.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.902+0000 7f832dd81740 1 client.18192 shutdown 2024-04-21T08:44:48.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.902+0000 7f832dd81740 1 -- 172.21.15.86:0/607866877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f3baaa00 msgr2=0x5629f3ba7bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.902+0000 7f832dd81740 1 --2- 172.21.15.86:0/607866877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f3baaa00 0x5629f3ba7bd0 secure :-1 s=READY pgs=1339 cs=0 l=1 rev1=1 crypto rx=0x7f831c09c950 tx=0x7f831c09c9c0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.902+0000 7f832dd81740 1 -- 172.21.15.86:0/607866877 shutdown_connections 2024-04-21T08:44:48.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.902+0000 7f832dd81740 1 --2- 172.21.15.86:0/607866877 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5629f3bbaa60 0x5629f3bbce90 crc :-1 s=READY pgs=55 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.902+0000 7f832dd81740 1 --2- 172.21.15.86:0/607866877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f3baaa00 0x5629f3ba7bd0 unknown :-1 s=CLOSED pgs=1339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.902+0000 7f832dd81740 1 --2- 172.21.15.86:0/607866877 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629f3ba1e80 0x5629f3ba7670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.902+0000 7f8303fff640 0 client.18192 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:44:48.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.902+0000 7f832dd81740 1 --2- 172.21.15.86:0/607866877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629f3ba14d0 0x5629f3baa2e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.902+0000 7f832dd81740 1 -- 172.21.15.86:0/607866877 >> 172.21.15.86:0/607866877 conn(0x5629f3b9bba0 msgr2=0x5629f3b9d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:48.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.902+0000 7f832dd81740 1 -- 172.21.15.86:0/607866877 shutdown_connections 2024-04-21T08:44:48.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.902+0000 7f832dd81740 1 -- 172.21.15.86:0/607866877 wait complete. 2024-04-21T08:44:48.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.903+0000 7f832dd81740 20 client.18192 trim_cache size 0 max 16384 2024-04-21T08:44:48.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.916+0000 7f832dd81740 1 Processor -- start 2024-04-21T08:44:48.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.916+0000 7f832dd81740 1 -- start start 2024-04-21T08:44:48.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.916+0000 7f832dd81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f3ba14d0 0x5629f3bb6800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.916+0000 7f832dd81740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629f3ba1e80 0x5629f3b9aa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.916+0000 7f832dd81740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629f3baaa00 0x5629f3ba9da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.916+0000 7f832dd81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f83241af350 con 0x5629f3ba14d0 2024-04-21T08:44:48.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.916+0000 7f832dd81740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f83241b0e90 con 0x5629f3baaa00 2024-04-21T08:44:48.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.916+0000 7f832dd81740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f83241afa20 con 0x5629f3ba1e80 2024-04-21T08:44:48.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.916+0000 7f8328f34640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f3ba14d0 0x5629f3bb6800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.916+0000 7f8328f34640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f3ba14d0 0x5629f3bb6800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42510/0 (socket says 172.21.15.86:42510) 2024-04-21T08:44:48.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.916+0000 7f8328f34640 1 -- 172.21.15.86:0/3536531334 learned_addr learned my addr 172.21.15.86:0/3536531334 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:48.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.916+0000 7f8323fff640 1 --2- 172.21.15.86:0/3536531334 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629f3ba1e80 0x5629f3b9aa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.916+0000 7f8329735640 1 --2- 172.21.15.86:0/3536531334 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629f3baaa00 0x5629f3ba9da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.916+0000 7f8328f34640 1 -- 172.21.15.86:0/3536531334 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629f3ba1e80 msgr2=0x5629f3b9aa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.916+0000 7f8328f34640 1 --2- 172.21.15.86:0/3536531334 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629f3ba1e80 0x5629f3b9aa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.916+0000 7f8328f34640 1 -- 172.21.15.86:0/3536531334 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629f3baaa00 msgr2=0x5629f3ba9da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.916+0000 7f8328f34640 1 --2- 172.21.15.86:0/3536531334 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629f3baaa00 0x5629f3ba9da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.916+0000 7f8328f34640 1 -- 172.21.15.86:0/3536531334 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5629f3ba7f40 con 0x5629f3ba14d0 2024-04-21T08:44:48.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.916+0000 7f8329735640 1 --2- 172.21.15.86:0/3536531334 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629f3baaa00 0x5629f3ba9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:48.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.916+0000 7f8328f34640 1 --2- 172.21.15.86:0/3536531334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f3ba14d0 0x5629f3bb6800 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f831802c170 tx=0x7f8318009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b38c81e4cd6e28 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.916+0000 7f830e7fc640 1 -- 172.21.15.86:0/3536531334 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8318052070 con 0x5629f3ba14d0 2024-04-21T08:44:48.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.916+0000 7f830e7fc640 1 -- 172.21.15.86:0/3536531334 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f831802c7e0 con 0x5629f3ba14d0 2024-04-21T08:44:48.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.916+0000 7f830e7fc640 1 -- 172.21.15.86:0/3536531334 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8318043040 con 0x5629f3ba14d0 2024-04-21T08:44:48.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.917+0000 7f832dd81740 1 -- 172.21.15.86:0/3536531334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f3ba14d0 msgr2=0x5629f3bb6800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.917+0000 7f832dd81740 1 --2- 172.21.15.86:0/3536531334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f3ba14d0 0x5629f3bb6800 secure :-1 s=READY pgs=1340 cs=0 l=1 rev1=1 crypto rx=0x7f831802c170 tx=0x7f8318009560 comp rx=0 tx=0).stop 2024-04-21T08:44:48.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.917+0000 7f832dd81740 1 -- 172.21.15.86:0/3536531334 shutdown_connections 2024-04-21T08:44:48.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.917+0000 7f832dd81740 1 --2- 172.21.15.86:0/3536531334 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629f3baaa00 0x5629f3ba9da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.917+0000 7f832dd81740 1 --2- 172.21.15.86:0/3536531334 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629f3ba1e80 0x5629f3b9aa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.917+0000 7f832dd81740 1 --2- 172.21.15.86:0/3536531334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f3ba14d0 0x5629f3bb6800 unknown :-1 s=CLOSED pgs=1340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.917+0000 7f832dd81740 1 -- 172.21.15.86:0/3536531334 >> 172.21.15.86:0/3536531334 conn(0x5629f3b9bba0 msgr2=0x5629f3b9a370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:48.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.917+0000 7f832dd81740 1 -- 172.21.15.86:0/3536531334 shutdown_connections 2024-04-21T08:44:48.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.917+0000 7f832dd81740 1 -- 172.21.15.86:0/3536531334 wait complete. 2024-04-21T08:44:48.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.917+0000 7f832dd81740 1 -- start start 2024-04-21T08:44:48.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.917+0000 7f832dd81740 1 Processor -- start 2024-04-21T08:44:48.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.918+0000 7f832dd81740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629f3ba14d0 0x5629f3bb30a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.918+0000 7f832dd81740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629f3ba1e80 0x5629f3bae410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.918+0000 7f832dd81740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f3baaa00 0x5629f3bae950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.918+0000 7f832dd81740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f83241b07c0 con 0x5629f3baaa00 2024-04-21T08:44:48.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.918+0000 7f832dd81740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f83241b00f0 con 0x5629f3ba1e80 2024-04-21T08:44:48.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.918+0000 7f832dd81740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f83241ad310 con 0x5629f3ba14d0 2024-04-21T08:44:48.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.918+0000 7f832dd81740 10 client.0 _ll_register_callbacks cb 0x7f831c09c390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:48.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.918+0000 7f8329735640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f3baaa00 0x5629f3bae950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.918+0000 7f8329735640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f3baaa00 0x5629f3bae950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42514/0 (socket says 172.21.15.86:42514) 2024-04-21T08:44:48.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.918+0000 7f8329735640 1 -- 172.21.15.86:0/2659555018 learned_addr learned my addr 172.21.15.86:0/2659555018 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:48.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.918+0000 7f8328f34640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629f3ba14d0 0x5629f3bb30a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.918+0000 7f8323fff640 1 --2- 172.21.15.86:0/2659555018 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629f3ba1e80 0x5629f3bae410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.918+0000 7f8329735640 1 -- 172.21.15.86:0/2659555018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629f3ba14d0 msgr2=0x5629f3bb30a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.918+0000 7f8329735640 1 --2- 172.21.15.86:0/2659555018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629f3ba14d0 0x5629f3bb30a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.918+0000 7f8329735640 1 -- 172.21.15.86:0/2659555018 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629f3ba1e80 msgr2=0x5629f3bae410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:48.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.918+0000 7f8329735640 1 --2- 172.21.15.86:0/2659555018 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629f3ba1e80 0x5629f3bae410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.918+0000 7f8329735640 1 -- 172.21.15.86:0/2659555018 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5629f3baf160 con 0x5629f3baaa00 2024-04-21T08:44:48.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.918+0000 7f8329735640 1 --2- 172.21.15.86:0/2659555018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f3baaa00 0x5629f3bae950 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f831c09c6c0 tx=0x7f831c000f40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9f6bfa255639ee69 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:48.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.918+0000 7f8303fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:48.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.918+0000 7f8303fff640 1 -- 172.21.15.86:0/2659555018 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f831c084ba0 con 0x5629f3baaa00 2024-04-21T08:44:48.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.918+0000 7f8303fff640 1 -- 172.21.15.86:0/2659555018 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f831c09be10 con 0x5629f3baaa00 2024-04-21T08:44:48.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.918+0000 7f832dd81740 1 -- 172.21.15.86:0/2659555018 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f831802c7e0 con 0x5629f3baaa00 2024-04-21T08:44:48.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.919+0000 7f8303fff640 1 -- 172.21.15.86:0/2659555018 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f831c0a5630 con 0x5629f3baaa00 2024-04-21T08:44:48.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.919+0000 7f832dd81740 10 client.18198 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:48.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.919+0000 7f832dd81740 1 -- 172.21.15.86:0/2659555018 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5629f3c20fa0 con 0x5629f3baaa00 2024-04-21T08:44:48.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.919+0000 7f8303fff640 1 -- 172.21.15.86:0/2659555018 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f831802c7e0 con 0x5629f3baaa00 2024-04-21T08:44:48.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.919+0000 7f8303fff640 1 -- 172.21.15.86:0/2659555018 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f831c09b880 con 0x5629f3baaa00 2024-04-21T08:44:48.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.919+0000 7f832dd81740 10 client.18198 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:48.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.919+0000 7f832dd81740 10 client.18198 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:44:48.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.919+0000 7f832dd81740 20 client.18198 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:48.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.919+0000 7f832dd81740 1 --2- 172.21.15.86:0/2659555018 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5629f3c216a0 0x5629f3c23ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:48.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.919+0000 7f832dd81740 4 client.18198 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:44:48.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.919+0000 7f832dd81740 1 -- 172.21.15.86:0/2659555018 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5629f3b13c10 con 0x5629f3c216a0 2024-04-21T08:44:48.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.919+0000 7f8328f34640 1 --2- 172.21.15.86:0/2659555018 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5629f3c216a0 0x5629f3c23ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:48.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.919+0000 7f8303fff640 10 client.18198 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:44:48.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.919+0000 7f8328f34640 1 --2- 172.21.15.86:0/2659555018 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5629f3c216a0 0x5629f3c23ac0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=714743ceb64e01a1 server_cookie=2017cc89748bded1 in_seq=0 out_seq=0 2024-04-21T08:44:48.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.920+0000 7f8303fff640 1 -- 172.21.15.86:0/2659555018 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5629f3b13c10 con 0x5629f3c216a0 2024-04-21T08:44:48.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.920+0000 7f8303fff640 10 client.18198 handle_command_reply: tid=0 2024-04-21T08:44:48.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.920+0000 7f832dd81740 1 client.18198 shutdown 2024-04-21T08:44:48.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.920+0000 7f832dd81740 1 -- 172.21.15.86:0/2659555018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f3baaa00 msgr2=0x5629f3bae950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.920+0000 7f832dd81740 1 --2- 172.21.15.86:0/2659555018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f3baaa00 0x5629f3bae950 secure :-1 s=READY pgs=1341 cs=0 l=1 rev1=1 crypto rx=0x7f831c09c6c0 tx=0x7f831c000f40 comp rx=0 tx=0).stop 2024-04-21T08:44:48.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.920+0000 7f832dd81740 1 -- 172.21.15.86:0/2659555018 shutdown_connections 2024-04-21T08:44:48.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.920+0000 7f832dd81740 1 --2- 172.21.15.86:0/2659555018 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5629f3c216a0 0x5629f3c23ac0 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.920+0000 7f832dd81740 1 --2- 172.21.15.86:0/2659555018 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5629f3baaa00 0x5629f3bae950 unknown :-1 s=CLOSED pgs=1341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.920+0000 7f832dd81740 1 --2- 172.21.15.86:0/2659555018 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5629f3ba1e80 0x5629f3bae410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.920+0000 7f8303fff640 0 client.18198 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:44:48.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.920+0000 7f832dd81740 1 --2- 172.21.15.86:0/2659555018 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5629f3ba14d0 0x5629f3bb30a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.920+0000 7f832dd81740 1 -- 172.21.15.86:0/2659555018 >> 172.21.15.86:0/2659555018 conn(0x5629f3b9bba0 msgr2=0x5629f3ba5b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:48.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.920+0000 7f832dd81740 1 -- 172.21.15.86:0/2659555018 shutdown_connections 2024-04-21T08:44:48.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.920+0000 7f832dd81740 1 -- 172.21.15.86:0/2659555018 wait complete. 2024-04-21T08:44:48.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.920+0000 7f832dd81740 20 client.18198 trim_cache size 0 max 16384 2024-04-21T08:44:48.928 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:44:48.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.921+0000 7f832b198640 1 -- 172.21.15.86:0/642637340 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f83140a3d20 msgr2=0x7f83140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.921+0000 7f832b198640 1 --2- 172.21.15.86:0/642637340 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f83140a3d20 0x7f83140a61e0 secure :-1 s=READY pgs=212 cs=0 l=1 rev1=1 crypto rx=0x7f832406b0a0 tx=0x7f8310013820 comp rx=0 tx=0).stop 2024-04-21T08:44:48.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.921+0000 7f832b198640 1 -- 172.21.15.86:0/642637340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8324107030 msgr2=0x7f83241a4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:48.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.921+0000 7f832b198640 1 --2- 172.21.15.86:0/642637340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8324107030 0x7f83241a4620 secure :-1 s=READY pgs=1337 cs=0 l=1 rev1=1 crypto rx=0x7f831c00eaa0 tx=0x7f831c00ef60 comp rx=0 tx=0).stop 2024-04-21T08:44:48.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.921+0000 7f832b198640 1 -- 172.21.15.86:0/642637340 shutdown_connections 2024-04-21T08:44:48.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.921+0000 7f832b198640 1 --2- 172.21.15.86:0/642637340 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f83140a3d20 0x7f83140a61e0 unknown :-1 s=CLOSED pgs=212 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.921+0000 7f832b198640 1 --2- 172.21.15.86:0/642637340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8324107030 0x7f83241a4620 unknown :-1 s=CLOSED pgs=1337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.921+0000 7f832b198640 1 --2- 172.21.15.86:0/642637340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8324101b40 0x7f8324199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.921+0000 7f832b198640 1 --2- 172.21.15.86:0/642637340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f83241011a0 0x7f83241995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:48.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.921+0000 7f832b198640 1 -- 172.21.15.86:0/642637340 >> 172.21.15.86:0/642637340 conn(0x7f8324076c50 msgr2=0x7f83240ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:48.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.921+0000 7f832b198640 1 -- 172.21.15.86:0/642637340 shutdown_connections 2024-04-21T08:44:48.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:48.921+0000 7f832b198640 1 -- 172.21.15.86:0/642637340 wait complete. 2024-04-21T08:44:48.931 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:44:49.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.033+0000 7fa660732640 1 -- 172.21.15.86:0/4226568241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa658103af0 msgr2=0x7fa65810bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:49.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.033+0000 7fa660732640 1 --2- 172.21.15.86:0/4226568241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa658103af0 0x7fa65810bfd0 secure :-1 s=READY pgs=1342 cs=0 l=1 rev1=1 crypto rx=0x7fa648009900 tx=0x7fa64802bd60 comp rx=0 tx=0).stop 2024-04-21T08:44:49.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.034+0000 7fa660732640 1 -- 172.21.15.86:0/4226568241 shutdown_connections 2024-04-21T08:44:49.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.034+0000 7fa660732640 1 --2- 172.21.15.86:0/4226568241 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa65810c510 0x7fa65810e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.034+0000 7fa660732640 1 --2- 172.21.15.86:0/4226568241 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa658103af0 0x7fa65810bfd0 unknown :-1 s=CLOSED pgs=1342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.034+0000 7fa660732640 1 --2- 172.21.15.86:0/4226568241 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6581031d0 0x7fa6581035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.034+0000 7fa660732640 1 -- 172.21.15.86:0/4226568241 >> 172.21.15.86:0/4226568241 conn(0x7fa658077f50 msgr2=0x7fa658078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:49.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.034+0000 7fa660732640 1 -- 172.21.15.86:0/4226568241 shutdown_connections 2024-04-21T08:44:49.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.034+0000 7fa660732640 1 -- 172.21.15.86:0/4226568241 wait complete. 2024-04-21T08:44:49.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.034+0000 7fa660732640 1 Processor -- start 2024-04-21T08:44:49.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.034+0000 7fa660732640 1 -- start start 2024-04-21T08:44:49.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.034+0000 7fa660732640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6581031d0 0x7fa658198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.034+0000 7fa660732640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa658103af0 0x7fa658198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.034+0000 7fa660732640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa65810c510 0x7fa65819cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.034+0000 7fa660732640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa658112200 con 0x7fa65810c510 2024-04-21T08:44:49.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.034+0000 7fa660732640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa658112080 con 0x7fa658103af0 2024-04-21T08:44:49.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.034+0000 7fa660732640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa658112380 con 0x7fa6581031d0 2024-04-21T08:44:49.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.034+0000 7fa65eccf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa65810c510 0x7fa65819cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:49.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.034+0000 7fa65eccf640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa65810c510 0x7fa65819cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42536/0 (socket says 172.21.15.86:42536) 2024-04-21T08:44:49.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.034+0000 7fa65eccf640 1 -- 172.21.15.86:0/924308521 learned_addr learned my addr 172.21.15.86:0/924308521 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:49.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.035+0000 7fa65dccd640 1 --2- 172.21.15.86:0/924308521 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa658103af0 0x7fa658198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:49.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.035+0000 7fa65eccf640 1 -- 172.21.15.86:0/924308521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6581031d0 msgr2=0x7fa658198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:49.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.035+0000 7fa65eccf640 1 --2- 172.21.15.86:0/924308521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6581031d0 0x7fa658198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.035+0000 7fa65eccf640 1 -- 172.21.15.86:0/924308521 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa658103af0 msgr2=0x7fa658198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:49.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.035+0000 7fa65eccf640 1 --2- 172.21.15.86:0/924308521 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa658103af0 0x7fa658198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.035+0000 7fa65eccf640 1 -- 172.21.15.86:0/924308521 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa65819d710 con 0x7fa65810c510 2024-04-21T08:44:49.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.035+0000 7fa65dccd640 1 --2- 172.21.15.86:0/924308521 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa658103af0 0x7fa658198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:49.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.035+0000 7fa65eccf640 1 --2- 172.21.15.86:0/924308521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa65810c510 0x7fa65819cfe0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fa65000b970 tx=0x7fa65000be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:49.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.035+0000 7fa64f7fe640 1 -- 172.21.15.86:0/924308521 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa650019070 con 0x7fa65810c510 2024-04-21T08:44:49.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.035+0000 7fa660732640 1 -- 172.21.15.86:0/924308521 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa65819da00 con 0x7fa65810c510 2024-04-21T08:44:49.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.035+0000 7fa64f7fe640 1 -- 172.21.15.86:0/924308521 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa650014440 con 0x7fa65810c510 2024-04-21T08:44:49.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.035+0000 7fa64f7fe640 1 -- 172.21.15.86:0/924308521 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6500075d0 con 0x7fa65810c510 2024-04-21T08:44:49.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.035+0000 7fa660732640 1 -- 172.21.15.86:0/924308521 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6581a5290 con 0x7fa65810c510 2024-04-21T08:44:49.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.038+0000 7fa663401740 1 Processor -- start 2024-04-21T08:44:49.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.038+0000 7fa64f7fe640 1 -- 172.21.15.86:0/924308521 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa65001c070 con 0x7fa65810c510 2024-04-21T08:44:49.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.038+0000 7fa663401740 1 -- start start 2024-04-21T08:44:49.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.038+0000 7fa663401740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646fc23a4d0 0x5646fc23a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.038+0000 7fa663401740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5646fc23ae80 0x5646fc243430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.038+0000 7fa663401740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646fc243a00 0x5646fc245e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.038+0000 7fa663401740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5646fc19f1f0 con 0x5646fc23a4d0 2024-04-21T08:44:49.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.038+0000 7fa663401740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5646fc13c4b0 con 0x5646fc23ae80 2024-04-21T08:44:49.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.038+0000 7fa663401740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5646fc19b1b0 con 0x5646fc243a00 2024-04-21T08:44:49.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.038+0000 7fa65dccd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646fc23a4d0 0x5646fc23a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:49.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.038+0000 7fa65dccd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646fc23a4d0 0x5646fc23a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42548/0 (socket says 172.21.15.86:42548) 2024-04-21T08:44:49.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.038+0000 7fa65dccd640 1 -- 172.21.15.86:0/3909702634 learned_addr learned my addr 172.21.15.86:0/3909702634 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:49.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.039+0000 7fa65e4ce640 1 --2- 172.21.15.86:0/3909702634 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5646fc23ae80 0x5646fc243430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.039+0000 7fa65dccd640 1 -- 172.21.15.86:0/3909702634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646fc243a00 msgr2=0x5646fc245e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.039+0000 7fa65dccd640 1 --2- 172.21.15.86:0/3909702634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646fc243a00 0x5646fc245e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.039+0000 7fa65dccd640 1 -- 172.21.15.86:0/3909702634 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5646fc23ae80 msgr2=0x5646fc243430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.039+0000 7fa65dccd640 1 --2- 172.21.15.86:0/3909702634 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5646fc23ae80 0x5646fc243430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.039+0000 7fa64f7fe640 1 --2- 172.21.15.86:0/924308521 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa6440a3d20 0x7fa6440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.039+0000 7fa65dccd640 1 -- 172.21.15.86:0/3909702634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646fc12e1b0 con 0x5646fc23a4d0 2024-04-21T08:44:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.039+0000 7fa65e4ce640 1 --2- 172.21.15.86:0/3909702634 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5646fc23ae80 0x5646fc243430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.039+0000 7fa65eccf640 1 --2- 172.21.15.86:0/924308521 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa6440a3d20 0x7fa6440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.039+0000 7fa65dccd640 1 --2- 172.21.15.86:0/3909702634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646fc23a4d0 0x5646fc23a8b0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fa648036040 tx=0x7fa64802c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa9aec3537ab7b29 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.039+0000 7fa64f7fe640 1 -- 172.21.15.86:0/924308521 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fa6500d0070 con 0x7fa65810c510 2024-04-21T08:44:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.039+0000 7fa64cff9640 1 -- 172.21.15.86:0/3909702634 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa648042020 con 0x5646fc23a4d0 2024-04-21T08:44:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.039+0000 7fa64cff9640 1 -- 172.21.15.86:0/3909702634 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa64802cdc0 con 0x5646fc23a4d0 2024-04-21T08:44:49.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.039+0000 7fa64cff9640 1 -- 172.21.15.86:0/3909702634 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6480046c0 con 0x5646fc23a4d0 2024-04-21T08:44:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.040+0000 7fa663401740 1 -- 172.21.15.86:0/3909702634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646fc23a4d0 msgr2=0x5646fc23a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.040+0000 7fa663401740 1 --2- 172.21.15.86:0/3909702634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646fc23a4d0 0x5646fc23a8b0 secure :-1 s=READY pgs=1344 cs=0 l=1 rev1=1 crypto rx=0x7fa648036040 tx=0x7fa64802c810 comp rx=0 tx=0).stop 2024-04-21T08:44:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.040+0000 7fa65eccf640 1 --2- 172.21.15.86:0/924308521 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa6440a3d20 0x7fa6440a61e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa65009bd40 tx=0x7fa65006d7f0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.040+0000 7fa663401740 1 -- 172.21.15.86:0/3909702634 shutdown_connections 2024-04-21T08:44:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.040+0000 7fa663401740 1 --2- 172.21.15.86:0/3909702634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646fc243a00 0x5646fc245e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.040+0000 7fa663401740 1 --2- 172.21.15.86:0/3909702634 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5646fc23ae80 0x5646fc243430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.040+0000 7fa663401740 1 --2- 172.21.15.86:0/3909702634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646fc23a4d0 0x5646fc23a8b0 unknown :-1 s=CLOSED pgs=1344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.040+0000 7fa663401740 1 -- 172.21.15.86:0/3909702634 >> 172.21.15.86:0/3909702634 conn(0x5646fc234ba0 msgr2=0x5646fc234f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.040+0000 7fa663401740 1 -- 172.21.15.86:0/3909702634 shutdown_connections 2024-04-21T08:44:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.040+0000 7fa663401740 1 -- 172.21.15.86:0/3909702634 wait complete. 2024-04-21T08:44:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.040+0000 7fa663401740 1 -- start start 2024-04-21T08:44:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.040+0000 7fa663401740 1 Processor -- start 2024-04-21T08:44:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.041+0000 7fa663401740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646fc23a4d0 0x5646fc243330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.041+0000 7fa663401740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646fc23ae80 0x5646fc2406c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.041+0000 7fa663401740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5646fc243a00 0x5646fc240cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.041+0000 7fa663401740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5646fc246910 con 0x5646fc23a4d0 2024-04-21T08:44:49.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.041+0000 7fa663401740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5646fc158500 con 0x5646fc243a00 2024-04-21T08:44:49.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.041+0000 7fa663401740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5646fc246a90 con 0x5646fc23ae80 2024-04-21T08:44:49.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.041+0000 7fa663401740 10 client.0 _ll_register_callbacks cb 0x5646fc126b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:49.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.041+0000 7fa65dccd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646fc23a4d0 0x5646fc243330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:49.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.041+0000 7fa65dccd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646fc23a4d0 0x5646fc243330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42554/0 (socket says 172.21.15.86:42554) 2024-04-21T08:44:49.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.041+0000 7fa65dccd640 1 -- 172.21.15.86:0/822649890 learned_addr learned my addr 172.21.15.86:0/822649890 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:49.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.041+0000 7fa65dccd640 1 --2- 172.21.15.86:0/822649890 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5646fc243a00 0x5646fc240cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:49.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.041+0000 7fa65dccd640 1 -- 172.21.15.86:0/822649890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646fc23ae80 msgr2=0x5646fc2406c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:49.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.041+0000 7fa65dccd640 1 --2- 172.21.15.86:0/822649890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646fc23ae80 0x5646fc2406c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.041+0000 7fa65dccd640 1 -- 172.21.15.86:0/822649890 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5646fc243a00 msgr2=0x5646fc240cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:49.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.041+0000 7fa65dccd640 1 --2- 172.21.15.86:0/822649890 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5646fc243a00 0x5646fc240cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.041+0000 7fa65dccd640 1 -- 172.21.15.86:0/822649890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646fbeb13f0 con 0x5646fc23a4d0 2024-04-21T08:44:49.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.041+0000 7fa65dccd640 1 --2- 172.21.15.86:0/822649890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646fc23a4d0 0x5646fc243330 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fa648009900 tx=0x7fa6480052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea741b58af91f3b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:49.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.041+0000 7fa641ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:49.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.041+0000 7fa641ffb640 1 -- 172.21.15.86:0/822649890 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa648042070 con 0x5646fc23a4d0 2024-04-21T08:44:49.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.041+0000 7fa641ffb640 1 -- 172.21.15.86:0/822649890 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fa648003a00 con 0x5646fc23a4d0 2024-04-21T08:44:49.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.041+0000 7fa663401740 1 -- 172.21.15.86:0/822649890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa64802cdc0 con 0x5646fc23a4d0 2024-04-21T08:44:49.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.041+0000 7fa641ffb640 1 -- 172.21.15.86:0/822649890 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa648003c00 con 0x5646fc23a4d0 2024-04-21T08:44:49.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.041+0000 7fa663401740 10 client.18210 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:49.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.041+0000 7fa663401740 1 -- 172.21.15.86:0/822649890 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5646fbf7b7d0 con 0x5646fc23a4d0 2024-04-21T08:44:49.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.041+0000 7fa641ffb640 1 -- 172.21.15.86:0/822649890 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa64802cdc0 con 0x5646fc23a4d0 2024-04-21T08:44:49.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.042+0000 7fa641ffb640 1 -- 172.21.15.86:0/822649890 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa64803e450 con 0x5646fc23a4d0 2024-04-21T08:44:49.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.042+0000 7fa663401740 10 client.18210 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:49.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.042+0000 7fa663401740 10 client.18210 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:44:49.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.042+0000 7fa663401740 20 client.18210 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:49.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.042+0000 7fa663401740 1 --2- 172.21.15.86:0/822649890 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5646fc253a90 0x5646fc255e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.042+0000 7fa663401740 4 client.18210 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.042+0000 7fa663401740 1 -- 172.21.15.86:0/822649890 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5646fc13e930 con 0x5646fc253a90 2024-04-21T08:44:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.042+0000 7fa65e4ce640 1 --2- 172.21.15.86:0/822649890 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5646fc253a90 0x5646fc255e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.043+0000 7fa641ffb640 10 client.18210 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:44:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.043+0000 7fa65e4ce640 1 --2- 172.21.15.86:0/822649890 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5646fc253a90 0x5646fc255e80 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=fa2ec7d841cd60d8 server_cookie=c8e62677583961f5 in_seq=0 out_seq=0 2024-04-21T08:44:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.043+0000 7fa641ffb640 1 -- 172.21.15.86:0/822649890 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5646fc13e930 con 0x5646fc253a90 2024-04-21T08:44:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.043+0000 7fa641ffb640 10 client.18210 handle_command_reply: tid=0 2024-04-21T08:44:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.044+0000 7fa663401740 1 client.18210 shutdown 2024-04-21T08:44:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.044+0000 7fa663401740 1 -- 172.21.15.86:0/822649890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646fc23a4d0 msgr2=0x5646fc243330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.044+0000 7fa663401740 1 --2- 172.21.15.86:0/822649890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646fc23a4d0 0x5646fc243330 secure :-1 s=READY pgs=1345 cs=0 l=1 rev1=1 crypto rx=0x7fa648009900 tx=0x7fa6480052f0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.044+0000 7fa663401740 1 -- 172.21.15.86:0/822649890 shutdown_connections 2024-04-21T08:44:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.044+0000 7fa663401740 1 --2- 172.21.15.86:0/822649890 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5646fc243a00 0x5646fc240cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.044+0000 7fa663401740 1 --2- 172.21.15.86:0/822649890 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646fc23ae80 0x5646fc2406c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.044+0000 7fa663401740 1 --2- 172.21.15.86:0/822649890 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5646fc253a90 0x5646fc255e80 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.044+0000 7fa663401740 1 --2- 172.21.15.86:0/822649890 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646fc23a4d0 0x5646fc243330 unknown :-1 s=CLOSED pgs=1345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.044+0000 7fa641ffb640 0 client.18210 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:44:49.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.044+0000 7fa663401740 1 -- 172.21.15.86:0/822649890 >> 172.21.15.86:0/822649890 conn(0x5646fc234ba0 msgr2=0x5646fc2368f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:49.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.044+0000 7fa663401740 1 -- 172.21.15.86:0/822649890 shutdown_connections 2024-04-21T08:44:49.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.044+0000 7fa663401740 1 -- 172.21.15.86:0/822649890 wait complete. 2024-04-21T08:44:49.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.044+0000 7fa663401740 20 client.18210 trim_cache size 0 max 16384 2024-04-21T08:44:49.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.057+0000 7fa663401740 1 Processor -- start 2024-04-21T08:44:49.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.057+0000 7fa663401740 1 -- start start 2024-04-21T08:44:49.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.057+0000 7fa663401740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646fc23a4d0 0x5646fc242df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.057+0000 7fa663401740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646fc23ae80 0x5646fc243330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.057+0000 7fa663401740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5646fc243a00 0x5646fc258ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.057+0000 7fa663401740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa6581af2a0 con 0x5646fc23a4d0 2024-04-21T08:44:49.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.057+0000 7fa663401740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa6581b0de0 con 0x5646fc243a00 2024-04-21T08:44:49.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.057+0000 7fa663401740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa6581af970 con 0x5646fc23ae80 2024-04-21T08:44:49.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.057+0000 7fa65dccd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646fc23a4d0 0x5646fc242df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:49.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.057+0000 7fa65dccd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646fc23a4d0 0x5646fc242df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42570/0 (socket says 172.21.15.86:42570) 2024-04-21T08:44:49.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.057+0000 7fa65dccd640 1 -- 172.21.15.86:0/4075755146 learned_addr learned my addr 172.21.15.86:0/4075755146 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:49.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.057+0000 7fa65e4ce640 1 --2- 172.21.15.86:0/4075755146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646fc23ae80 0x5646fc243330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.057+0000 7fa65dccd640 1 -- 172.21.15.86:0/4075755146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646fc23ae80 msgr2=0x5646fc243330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.057+0000 7fa65dccd640 1 --2- 172.21.15.86:0/4075755146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646fc23ae80 0x5646fc243330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.057+0000 7fa65dccd640 1 -- 172.21.15.86:0/4075755146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5646fc243a00 msgr2=0x5646fc258ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.057+0000 7fa65dccd640 1 --2- 172.21.15.86:0/4075755146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5646fc243a00 0x5646fc258ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.057+0000 7fa65dccd640 1 -- 172.21.15.86:0/4075755146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646fc23ac20 con 0x5646fc23a4d0 2024-04-21T08:44:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.057+0000 7fa65e4ce640 1 --2- 172.21.15.86:0/4075755146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646fc23ae80 0x5646fc243330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.057+0000 7fa65dccd640 1 --2- 172.21.15.86:0/4075755146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646fc23a4d0 0x5646fc242df0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fa648006010 tx=0x7fa648040f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=298d6198dbd02f26 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.058+0000 7fa643fff640 1 -- 172.21.15.86:0/4075755146 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa648042040 con 0x5646fc23a4d0 2024-04-21T08:44:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.058+0000 7fa643fff640 1 -- 172.21.15.86:0/4075755146 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa648009520 con 0x5646fc23a4d0 2024-04-21T08:44:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.058+0000 7fa643fff640 1 -- 172.21.15.86:0/4075755146 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa64802cb90 con 0x5646fc23a4d0 2024-04-21T08:44:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.058+0000 7fa663401740 1 -- 172.21.15.86:0/4075755146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646fc23a4d0 msgr2=0x5646fc242df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.058+0000 7fa663401740 1 --2- 172.21.15.86:0/4075755146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646fc23a4d0 0x5646fc242df0 secure :-1 s=READY pgs=1346 cs=0 l=1 rev1=1 crypto rx=0x7fa648006010 tx=0x7fa648040f80 comp rx=0 tx=0).stop 2024-04-21T08:44:49.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.058+0000 7fa663401740 1 -- 172.21.15.86:0/4075755146 shutdown_connections 2024-04-21T08:44:49.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.058+0000 7fa663401740 1 --2- 172.21.15.86:0/4075755146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5646fc243a00 0x5646fc258ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.058+0000 7fa663401740 1 --2- 172.21.15.86:0/4075755146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646fc23ae80 0x5646fc243330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.058+0000 7fa663401740 1 --2- 172.21.15.86:0/4075755146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646fc23a4d0 0x5646fc242df0 unknown :-1 s=CLOSED pgs=1346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.058+0000 7fa663401740 1 -- 172.21.15.86:0/4075755146 >> 172.21.15.86:0/4075755146 conn(0x5646fc234ba0 msgr2=0x5646fc233c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:49.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.058+0000 7fa663401740 1 -- 172.21.15.86:0/4075755146 shutdown_connections 2024-04-21T08:44:49.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.058+0000 7fa663401740 1 -- 172.21.15.86:0/4075755146 wait complete. 2024-04-21T08:44:49.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.058+0000 7fa663401740 1 -- start start 2024-04-21T08:44:49.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.058+0000 7fa663401740 1 Processor -- start 2024-04-21T08:44:49.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.059+0000 7fa663401740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646fc23a4d0 0x5646fc2591c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.059+0000 7fa663401740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5646fc23ae80 0x5646fc256550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.059+0000 7fa663401740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646fc243a00 0x5646fc256a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.059+0000 7fa663401740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa6581b0710 con 0x5646fc23a4d0 2024-04-21T08:44:49.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.059+0000 7fa663401740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa6581b0040 con 0x5646fc23ae80 2024-04-21T08:44:49.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.059+0000 7fa663401740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa6581ad260 con 0x5646fc243a00 2024-04-21T08:44:49.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.059+0000 7fa663401740 10 client.0 _ll_register_callbacks cb 0x7fa648005690 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:49.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.059+0000 7fa65dccd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646fc23a4d0 0x5646fc2591c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:49.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.059+0000 7fa65dccd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646fc23a4d0 0x5646fc2591c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42582/0 (socket says 172.21.15.86:42582) 2024-04-21T08:44:49.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.059+0000 7fa65dccd640 1 -- 172.21.15.86:0/4089991523 learned_addr learned my addr 172.21.15.86:0/4089991523 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:49.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.059+0000 7fa65dccd640 1 -- 172.21.15.86:0/4089991523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646fc243a00 msgr2=0x5646fc256a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:49.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.059+0000 7fa65dccd640 1 --2- 172.21.15.86:0/4089991523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646fc243a00 0x5646fc256a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.059+0000 7fa65dccd640 1 -- 172.21.15.86:0/4089991523 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5646fc23ae80 msgr2=0x5646fc256550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:49.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.059+0000 7fa65dccd640 1 --2- 172.21.15.86:0/4089991523 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5646fc23ae80 0x5646fc256550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.059+0000 7fa65dccd640 1 -- 172.21.15.86:0/4089991523 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646fc257290 con 0x5646fc23a4d0 2024-04-21T08:44:49.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.059+0000 7fa65dccd640 1 --2- 172.21.15.86:0/4089991523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646fc23a4d0 0x5646fc2591c0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fa64802c950 tx=0x7fa64803eae0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46468b4d69ae3b96 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:49.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.059+0000 7fa641ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:49.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.059+0000 7fa641ffb640 1 -- 172.21.15.86:0/4089991523 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa648042040 con 0x5646fc23a4d0 2024-04-21T08:44:49.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.059+0000 7fa641ffb640 1 -- 172.21.15.86:0/4089991523 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fa648003da0 con 0x5646fc23a4d0 2024-04-21T08:44:49.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.059+0000 7fa663401740 1 -- 172.21.15.86:0/4089991523 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa648009520 con 0x5646fc23a4d0 2024-04-21T08:44:49.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.059+0000 7fa641ffb640 1 -- 172.21.15.86:0/4089991523 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa64802ce50 con 0x5646fc23a4d0 2024-04-21T08:44:49.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.059+0000 7fa663401740 10 client.18216 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:49.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.059+0000 7fa663401740 1 -- 172.21.15.86:0/4089991523 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5646fc2b9b30 con 0x5646fc23a4d0 2024-04-21T08:44:49.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.060+0000 7fa641ffb640 1 -- 172.21.15.86:0/4089991523 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa648009520 con 0x5646fc23a4d0 2024-04-21T08:44:49.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.060+0000 7fa641ffb640 1 -- 172.21.15.86:0/4089991523 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa648049440 con 0x5646fc23a4d0 2024-04-21T08:44:49.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.060+0000 7fa663401740 10 client.18216 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:49.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.060+0000 7fa663401740 10 client.18216 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:44:49.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.060+0000 7fa663401740 20 client.18216 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:49.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.060+0000 7fa663401740 1 --2- 172.21.15.86:0/4089991523 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5646fc2b9f70 0x5646fc2bc390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.060+0000 7fa663401740 4 client.18216 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:44:49.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.060+0000 7fa663401740 1 -- 172.21.15.86:0/4089991523 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5646fc1acc10 con 0x5646fc2b9f70 2024-04-21T08:44:49.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.060+0000 7fa65e4ce640 1 --2- 172.21.15.86:0/4089991523 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5646fc2b9f70 0x5646fc2bc390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:49.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.060+0000 7fa65e4ce640 1 --2- 172.21.15.86:0/4089991523 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5646fc2b9f70 0x5646fc2bc390 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=3b69377f03d0c7f3 server_cookie=762d44eb535d36b in_seq=0 out_seq=0 2024-04-21T08:44:49.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.060+0000 7fa641ffb640 10 client.18216 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:44:49.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.061+0000 7fa641ffb640 1 -- 172.21.15.86:0/4089991523 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5646fc1acc10 con 0x5646fc2b9f70 2024-04-21T08:44:49.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.061+0000 7fa641ffb640 10 client.18216 handle_command_reply: tid=0 2024-04-21T08:44:49.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.061+0000 7fa663401740 1 client.18216 shutdown 2024-04-21T08:44:49.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.061+0000 7fa663401740 1 -- 172.21.15.86:0/4089991523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646fc23a4d0 msgr2=0x5646fc2591c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:49.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.061+0000 7fa663401740 1 --2- 172.21.15.86:0/4089991523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646fc23a4d0 0x5646fc2591c0 secure :-1 s=READY pgs=1347 cs=0 l=1 rev1=1 crypto rx=0x7fa64802c950 tx=0x7fa64803eae0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.061+0000 7fa663401740 1 -- 172.21.15.86:0/4089991523 shutdown_connections 2024-04-21T08:44:49.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.061+0000 7fa663401740 1 --2- 172.21.15.86:0/4089991523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646fc243a00 0x5646fc256a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.061+0000 7fa663401740 1 --2- 172.21.15.86:0/4089991523 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5646fc23ae80 0x5646fc256550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.061+0000 7fa663401740 1 --2- 172.21.15.86:0/4089991523 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5646fc2b9f70 0x5646fc2bc390 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.061+0000 7fa663401740 1 --2- 172.21.15.86:0/4089991523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646fc23a4d0 0x5646fc2591c0 unknown :-1 s=CLOSED pgs=1347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.061+0000 7fa641ffb640 0 client.18216 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:44:49.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.061+0000 7fa663401740 1 -- 172.21.15.86:0/4089991523 >> 172.21.15.86:0/4089991523 conn(0x5646fc234ba0 msgr2=0x5646fc24ed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:49.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.061+0000 7fa663401740 1 -- 172.21.15.86:0/4089991523 shutdown_connections 2024-04-21T08:44:49.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.061+0000 7fa663401740 1 -- 172.21.15.86:0/4089991523 wait complete. 2024-04-21T08:44:49.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.061+0000 7fa663401740 20 client.18216 trim_cache size 0 max 16384 2024-04-21T08:44:49.070 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:44:49.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.061+0000 7fa660732640 1 -- 172.21.15.86:0/924308521 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa6440a3d20 msgr2=0x7fa6440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:49.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.061+0000 7fa660732640 1 --2- 172.21.15.86:0/924308521 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa6440a3d20 0x7fa6440a61e0 secure :-1 s=READY pgs=213 cs=0 l=1 rev1=1 crypto rx=0x7fa65009bd40 tx=0x7fa65006d7f0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.061+0000 7fa660732640 1 -- 172.21.15.86:0/924308521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa65810c510 msgr2=0x7fa65819cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:49.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.061+0000 7fa660732640 1 --2- 172.21.15.86:0/924308521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa65810c510 0x7fa65819cfe0 secure :-1 s=READY pgs=1343 cs=0 l=1 rev1=1 crypto rx=0x7fa65000b970 tx=0x7fa65000be30 comp rx=0 tx=0).stop 2024-04-21T08:44:49.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.061+0000 7fa660732640 1 -- 172.21.15.86:0/924308521 shutdown_connections 2024-04-21T08:44:49.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.061+0000 7fa660732640 1 --2- 172.21.15.86:0/924308521 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa6440a3d20 0x7fa6440a61e0 unknown :-1 s=CLOSED pgs=213 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.061+0000 7fa660732640 1 --2- 172.21.15.86:0/924308521 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa65810c510 0x7fa65819cfe0 unknown :-1 s=CLOSED pgs=1343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.061+0000 7fa660732640 1 --2- 172.21.15.86:0/924308521 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa658103af0 0x7fa658198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.061+0000 7fa660732640 1 --2- 172.21.15.86:0/924308521 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6581031d0 0x7fa658198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.061+0000 7fa660732640 1 -- 172.21.15.86:0/924308521 >> 172.21.15.86:0/924308521 conn(0x7fa658077f50 msgr2=0x7fa6580fdf30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:49.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.061+0000 7fa660732640 1 -- 172.21.15.86:0/924308521 shutdown_connections 2024-04-21T08:44:49.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.061+0000 7fa660732640 1 -- 172.21.15.86:0/924308521 wait complete. 2024-04-21T08:44:49.072 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:48 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1903634077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:49.073 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:44:49.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.173+0000 7fdf50ed9640 1 -- 172.21.15.86:0/2477465535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf4c06a7c0 msgr2=0x7fdf4c06aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:49.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.173+0000 7fdf50ed9640 1 --2- 172.21.15.86:0/2477465535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf4c06a7c0 0x7fdf4c06aba0 secure :-1 s=READY pgs=1348 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009900 tx=0x7fdf3c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:49.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.174+0000 7fdf50ed9640 1 -- 172.21.15.86:0/2477465535 shutdown_connections 2024-04-21T08:44:49.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.174+0000 7fdf50ed9640 1 --2- 172.21.15.86:0/2477465535 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf4c107280 0x7fdf4c10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.174+0000 7fdf50ed9640 1 --2- 172.21.15.86:0/2477465535 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf4c06b170 0x7fdf4c106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.174+0000 7fdf50ed9640 1 --2- 172.21.15.86:0/2477465535 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf4c06a7c0 0x7fdf4c06aba0 unknown :-1 s=CLOSED pgs=1348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.174+0000 7fdf50ed9640 1 -- 172.21.15.86:0/2477465535 >> 172.21.15.86:0/2477465535 conn(0x7fdf4c076c50 msgr2=0x7fdf4c077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:49.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.174+0000 7fdf50ed9640 1 -- 172.21.15.86:0/2477465535 shutdown_connections 2024-04-21T08:44:49.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.174+0000 7fdf50ed9640 1 -- 172.21.15.86:0/2477465535 wait complete. 2024-04-21T08:44:49.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.174+0000 7fdf50ed9640 1 Processor -- start 2024-04-21T08:44:49.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.174+0000 7fdf50ed9640 1 -- start start 2024-04-21T08:44:49.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.174+0000 7fdf50ed9640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf4c06a7c0 0x7fdf4c101270 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.174+0000 7fdf50ed9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf4c107280 0x7fdf4c1017b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.174+0000 7fdf50ed9640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf4c101cf0 0x7fdf4c1a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.174+0000 7fdf50ed9640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4c1121f0 con 0x7fdf4c107280 2024-04-21T08:44:49.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.174+0000 7fdf50ed9640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4c112070 con 0x7fdf4c101cf0 2024-04-21T08:44:49.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.174+0000 7fdf50ed9640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4c112370 con 0x7fdf4c06a7c0 2024-04-21T08:44:49.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.175+0000 7fdf49d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf4c107280 0x7fdf4c1017b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.175+0000 7fdf49d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf4c107280 0x7fdf4c1017b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42600/0 (socket says 172.21.15.86:42600) 2024-04-21T08:44:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.175+0000 7fdf49d9b640 1 -- 172.21.15.86:0/2721957268 learned_addr learned my addr 172.21.15.86:0/2721957268 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.175+0000 7fdf4a59c640 1 --2- 172.21.15.86:0/2721957268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf4c06a7c0 0x7fdf4c101270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.175+0000 7fdf4ad9d640 1 --2- 172.21.15.86:0/2721957268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf4c101cf0 0x7fdf4c1a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.175+0000 7fdf49d9b640 1 -- 172.21.15.86:0/2721957268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf4c06a7c0 msgr2=0x7fdf4c101270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.175+0000 7fdf49d9b640 1 --2- 172.21.15.86:0/2721957268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf4c06a7c0 0x7fdf4c101270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.175+0000 7fdf49d9b640 1 -- 172.21.15.86:0/2721957268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf4c101cf0 msgr2=0x7fdf4c1a68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.175+0000 7fdf49d9b640 1 --2- 172.21.15.86:0/2721957268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf4c101cf0 0x7fdf4c1a68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.175+0000 7fdf49d9b640 1 -- 172.21.15.86:0/2721957268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf4c1a6fe0 con 0x7fdf4c107280 2024-04-21T08:44:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.175+0000 7fdf4ad9d640 1 --2- 172.21.15.86:0/2721957268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf4c101cf0 0x7fdf4c1a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.175+0000 7fdf49d9b640 1 --2- 172.21.15.86:0/2721957268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf4c107280 0x7fdf4c1017b0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fdf4000e970 tx=0x7fdf4000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:49.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.175+0000 7fdf3b7fe640 1 -- 172.21.15.86:0/2721957268 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf4000cd80 con 0x7fdf4c107280 2024-04-21T08:44:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.175+0000 7fdf3b7fe640 1 -- 172.21.15.86:0/2721957268 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf400035f0 con 0x7fdf4c107280 2024-04-21T08:44:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.175+0000 7fdf50ed9640 1 -- 172.21.15.86:0/2721957268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf4c1a72d0 con 0x7fdf4c107280 2024-04-21T08:44:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.175+0000 7fdf3b7fe640 1 -- 172.21.15.86:0/2721957268 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf40010690 con 0x7fdf4c107280 2024-04-21T08:44:49.178 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.175+0000 7fdf50ed9640 1 -- 172.21.15.86:0/2721957268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf4c1a7820 con 0x7fdf4c107280 2024-04-21T08:44:49.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.178+0000 7fdf3b7fe640 1 -- 172.21.15.86:0/2721957268 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdf40003aa0 con 0x7fdf4c107280 2024-04-21T08:44:49.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.178+0000 7fdf53ba9740 1 Processor -- start 2024-04-21T08:44:49.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.178+0000 7fdf53ba9740 1 -- start start 2024-04-21T08:44:49.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.178+0000 7fdf53ba9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c796e864d0 0x55c796e868b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.178+0000 7fdf53ba9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c796e86e80 0x55c796e8f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.178+0000 7fdf53ba9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c796e8fa00 0x55c796e91e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.178+0000 7fdf53ba9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c796deb1f0 con 0x55c796e86e80 2024-04-21T08:44:49.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.178+0000 7fdf53ba9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c796d884b0 con 0x55c796e8fa00 2024-04-21T08:44:49.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.178+0000 7fdf53ba9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c796de71b0 con 0x55c796e864d0 2024-04-21T08:44:49.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.179+0000 7fdf4a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c796e86e80 0x55c796e8f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:49.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.179+0000 7fdf4a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c796e86e80 0x55c796e8f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42608/0 (socket says 172.21.15.86:42608) 2024-04-21T08:44:49.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.179+0000 7fdf4a59c640 1 -- 172.21.15.86:0/680064269 learned_addr learned my addr 172.21.15.86:0/680064269 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:49.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.179+0000 7fdf4ad9d640 1 --2- 172.21.15.86:0/680064269 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c796e864d0 0x55c796e868b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:49.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.179+0000 7fdf3b7fe640 1 --2- 172.21.15.86:0/2721957268 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdf340a3d50 0x7fdf340a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.179+0000 7fdf49d9b640 1 --2- 172.21.15.86:0/680064269 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c796e8fa00 0x55c796e91e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:49.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.179+0000 7fdf4a59c640 1 -- 172.21.15.86:0/680064269 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c796e864d0 msgr2=0x55c796e868b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:49.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.179+0000 7fdf4a59c640 1 --2- 172.21.15.86:0/680064269 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c796e864d0 0x55c796e868b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.179+0000 7fdf4a59c640 1 -- 172.21.15.86:0/680064269 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c796e8fa00 msgr2=0x55c796e91e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:49.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.179+0000 7fdf4a59c640 1 --2- 172.21.15.86:0/680064269 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c796e8fa00 0x55c796e91e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.179+0000 7fdf4a59c640 1 -- 172.21.15.86:0/680064269 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c796d7a1b0 con 0x55c796e86e80 2024-04-21T08:44:49.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.179+0000 7fdf4ad9d640 1 --2- 172.21.15.86:0/680064269 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c796e864d0 0x55c796e868b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:49.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.179+0000 7fdf4ad9d640 1 --2- 172.21.15.86:0/2721957268 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdf340a3d50 0x7fdf340a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:49.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.179+0000 7fdf3b7fe640 1 -- 172.21.15.86:0/2721957268 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fdf400d0f20 con 0x7fdf4c107280 2024-04-21T08:44:49.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.179+0000 7fdf4a59c640 1 --2- 172.21.15.86:0/680064269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c796e86e80 0x55c796e8f430 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009900 tx=0x7fdf3c005dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67f5cde5556b6613 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:49.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.179+0000 7fdf38ff9640 1 -- 172.21.15.86:0/680064269 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c044020 con 0x55c796e86e80 2024-04-21T08:44:49.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.179+0000 7fdf4ad9d640 1 --2- 172.21.15.86:0/2721957268 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdf340a3d50 0x7fdf340a6210 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdf44005fd0 tx=0x7fdf44004a40 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:49.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.179+0000 7fdf38ff9640 1 -- 172.21.15.86:0/680064269 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf3c02cc90 con 0x55c796e86e80 2024-04-21T08:44:49.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.179+0000 7fdf38ff9640 1 -- 172.21.15.86:0/680064269 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c0046c0 con 0x55c796e86e80 2024-04-21T08:44:49.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.180+0000 7fdf53ba9740 1 -- 172.21.15.86:0/680064269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c796e86e80 msgr2=0x55c796e8f430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:49.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.180+0000 7fdf53ba9740 1 --2- 172.21.15.86:0/680064269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c796e86e80 0x55c796e8f430 secure :-1 s=READY pgs=1350 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c009900 tx=0x7fdf3c005dc0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.180+0000 7fdf53ba9740 1 -- 172.21.15.86:0/680064269 shutdown_connections 2024-04-21T08:44:49.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.180+0000 7fdf53ba9740 1 --2- 172.21.15.86:0/680064269 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c796e8fa00 0x55c796e91e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.180+0000 7fdf53ba9740 1 --2- 172.21.15.86:0/680064269 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c796e86e80 0x55c796e8f430 unknown :-1 s=CLOSED pgs=1350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.180+0000 7fdf53ba9740 1 --2- 172.21.15.86:0/680064269 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c796e864d0 0x55c796e868b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.180+0000 7fdf53ba9740 1 -- 172.21.15.86:0/680064269 >> 172.21.15.86:0/680064269 conn(0x55c796e80ba0 msgr2=0x55c796e80f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.180+0000 7fdf53ba9740 1 -- 172.21.15.86:0/680064269 shutdown_connections 2024-04-21T08:44:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.180+0000 7fdf53ba9740 1 -- 172.21.15.86:0/680064269 wait complete. 2024-04-21T08:44:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.181+0000 7fdf53ba9740 1 -- start start 2024-04-21T08:44:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.181+0000 7fdf53ba9740 1 Processor -- start 2024-04-21T08:44:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.181+0000 7fdf53ba9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c796e864d0 0x55c796e8f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.181+0000 7fdf53ba9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c796e86e80 0x55c796e8c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.181+0000 7fdf53ba9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c796e8fa00 0x55c796e8cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.181+0000 7fdf53ba9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c796e92910 con 0x55c796e864d0 2024-04-21T08:44:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.181+0000 7fdf53ba9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c796da4500 con 0x55c796e8fa00 2024-04-21T08:44:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.181+0000 7fdf53ba9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c796e92a90 con 0x55c796e86e80 2024-04-21T08:44:49.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.181+0000 7fdf4a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c796e864d0 0x55c796e8f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:49.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.181+0000 7fdf53ba9740 10 client.0 _ll_register_callbacks cb 0x55c796d72b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:49.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.181+0000 7fdf4a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c796e864d0 0x55c796e8f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42614/0 (socket says 172.21.15.86:42614) 2024-04-21T08:44:49.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.181+0000 7fdf4a59c640 1 -- 172.21.15.86:0/1721932912 learned_addr learned my addr 172.21.15.86:0/1721932912 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:49.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.181+0000 7fdf49d9b640 1 --2- 172.21.15.86:0/1721932912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c796e86e80 0x55c796e8c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:49.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.181+0000 7fdf4ad9d640 1 --2- 172.21.15.86:0/1721932912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c796e8fa00 0x55c796e8cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:49.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.181+0000 7fdf4a59c640 1 -- 172.21.15.86:0/1721932912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c796e86e80 msgr2=0x55c796e8c650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:49.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.181+0000 7fdf4a59c640 1 --2- 172.21.15.86:0/1721932912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c796e86e80 0x55c796e8c650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.181+0000 7fdf4a59c640 1 -- 172.21.15.86:0/1721932912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c796e8fa00 msgr2=0x55c796e8cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:49.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.181+0000 7fdf4a59c640 1 --2- 172.21.15.86:0/1721932912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c796e8fa00 0x55c796e8cbb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.181+0000 7fdf4a59c640 1 -- 172.21.15.86:0/1721932912 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c796afd3f0 con 0x55c796e864d0 2024-04-21T08:44:49.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.181+0000 7fdf4a59c640 1 --2- 172.21.15.86:0/1721932912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c796e864d0 0x55c796e8f2c0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c02c170 tx=0x7fdf3c02bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c81b6891dead4a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:49.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.181+0000 7fdf31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:49.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.181+0000 7fdf31ffb640 1 -- 172.21.15.86:0/1721932912 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf3c044030 con 0x55c796e864d0 2024-04-21T08:44:49.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.182+0000 7fdf31ffb640 1 -- 172.21.15.86:0/1721932912 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fdf3c004450 con 0x55c796e864d0 2024-04-21T08:44:49.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.182+0000 7fdf53ba9740 1 -- 172.21.15.86:0/1721932912 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf3c02cc90 con 0x55c796e864d0 2024-04-21T08:44:49.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.182+0000 7fdf31ffb640 1 -- 172.21.15.86:0/1721932912 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdf3c059050 con 0x55c796e864d0 2024-04-21T08:44:49.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.182+0000 7fdf53ba9740 10 client.18228 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:49.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.182+0000 7fdf53ba9740 1 -- 172.21.15.86:0/1721932912 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c796bc77d0 con 0x55c796e864d0 2024-04-21T08:44:49.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.182+0000 7fdf31ffb640 1 -- 172.21.15.86:0/1721932912 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf3c02cc90 con 0x55c796e864d0 2024-04-21T08:44:49.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.182+0000 7fdf31ffb640 1 -- 172.21.15.86:0/1721932912 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdf3c004850 con 0x55c796e864d0 2024-04-21T08:44:49.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.182+0000 7fdf53ba9740 10 client.18228 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:49.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.182+0000 7fdf53ba9740 10 client.18228 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:49.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.182+0000 7fdf53ba9740 20 client.18228 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:49.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.182+0000 7fdf53ba9740 1 --2- 172.21.15.86:0/1721932912 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c796e9fa30 0x55c796ea1e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.182+0000 7fdf53ba9740 4 client.18228 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:49.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.182+0000 7fdf53ba9740 1 -- 172.21.15.86:0/1721932912 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c796d8a930 con 0x55c796e9fa30 2024-04-21T08:44:49.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.182+0000 7fdf49d9b640 1 --2- 172.21.15.86:0/1721932912 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c796e9fa30 0x55c796ea1e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:49.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.183+0000 7fdf49d9b640 1 --2- 172.21.15.86:0/1721932912 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c796e9fa30 0x55c796ea1e20 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=80c8e61087cf41ea server_cookie=d6da29b2be6a139e in_seq=0 out_seq=0 2024-04-21T08:44:49.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.183+0000 7fdf31ffb640 10 client.18228 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:49.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.183+0000 7fdf31ffb640 1 -- 172.21.15.86:0/1721932912 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c796d8a930 con 0x55c796e9fa30 2024-04-21T08:44:49.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.183+0000 7fdf31ffb640 10 client.18228 handle_command_reply: tid=0 2024-04-21T08:44:49.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.184+0000 7fdf53ba9740 1 client.18228 shutdown 2024-04-21T08:44:49.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.184+0000 7fdf53ba9740 1 -- 172.21.15.86:0/1721932912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c796e864d0 msgr2=0x55c796e8f2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:49.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.184+0000 7fdf53ba9740 1 --2- 172.21.15.86:0/1721932912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c796e864d0 0x55c796e8f2c0 secure :-1 s=READY pgs=1351 cs=0 l=1 rev1=1 crypto rx=0x7fdf3c02c170 tx=0x7fdf3c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:49.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.184+0000 7fdf53ba9740 1 -- 172.21.15.86:0/1721932912 shutdown_connections 2024-04-21T08:44:49.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.184+0000 7fdf53ba9740 1 --2- 172.21.15.86:0/1721932912 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c796e9fa30 0x55c796ea1e20 crc :-1 s=READY pgs=195 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.184+0000 7fdf53ba9740 1 --2- 172.21.15.86:0/1721932912 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c796e8fa00 0x55c796e8cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.184+0000 7fdf53ba9740 1 --2- 172.21.15.86:0/1721932912 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c796e86e80 0x55c796e8c650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.184+0000 7fdf31ffb640 0 client.18228 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:49.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.184+0000 7fdf53ba9740 1 --2- 172.21.15.86:0/1721932912 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c796e864d0 0x55c796e8f2c0 unknown :-1 s=CLOSED pgs=1351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.184+0000 7fdf53ba9740 1 -- 172.21.15.86:0/1721932912 >> 172.21.15.86:0/1721932912 conn(0x55c796e80ba0 msgr2=0x55c796e828f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:49.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.184+0000 7fdf53ba9740 1 -- 172.21.15.86:0/1721932912 shutdown_connections 2024-04-21T08:44:49.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.184+0000 7fdf53ba9740 1 -- 172.21.15.86:0/1721932912 wait complete. 2024-04-21T08:44:49.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.184+0000 7fdf53ba9740 20 client.18228 trim_cache size 0 max 16384 2024-04-21T08:44:49.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.197+0000 7fdf53ba9740 1 Processor -- start 2024-04-21T08:44:49.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.197+0000 7fdf53ba9740 1 -- start start 2024-04-21T08:44:49.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.197+0000 7fdf53ba9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c796e864d0 0x55c796e9b800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.197+0000 7fdf53ba9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c796e86e80 0x55c796e7fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.197+0000 7fdf53ba9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c796e8fa00 0x55c796e8ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.197+0000 7fdf53ba9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4c1b1580 con 0x55c796e86e80 2024-04-21T08:44:49.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.197+0000 7fdf53ba9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4c1b30c0 con 0x55c796e8fa00 2024-04-21T08:44:49.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.197+0000 7fdf53ba9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4c1b1c50 con 0x55c796e864d0 2024-04-21T08:44:49.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.197+0000 7fdf49d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c796e86e80 0x55c796e7fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:49.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.197+0000 7fdf49d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c796e86e80 0x55c796e7fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42628/0 (socket says 172.21.15.86:42628) 2024-04-21T08:44:49.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.197+0000 7fdf49d9b640 1 -- 172.21.15.86:0/3131746820 learned_addr learned my addr 172.21.15.86:0/3131746820 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:49.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.197+0000 7fdf4a59c640 1 --2- 172.21.15.86:0/3131746820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c796e864d0 0x55c796e9b800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:49.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.197+0000 7fdf49d9b640 1 -- 172.21.15.86:0/3131746820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c796e864d0 msgr2=0x55c796e9b800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:49.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.197+0000 7fdf49d9b640 1 --2- 172.21.15.86:0/3131746820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c796e864d0 0x55c796e9b800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.197+0000 7fdf49d9b640 1 -- 172.21.15.86:0/3131746820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c796e8fa00 msgr2=0x55c796e8ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:49.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.197+0000 7fdf49d9b640 1 --2- 172.21.15.86:0/3131746820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c796e8fa00 0x55c796e8ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.197+0000 7fdf49d9b640 1 -- 172.21.15.86:0/3131746820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c796e922b0 con 0x55c796e86e80 2024-04-21T08:44:49.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.198+0000 7fdf49d9b640 1 --2- 172.21.15.86:0/3131746820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c796e86e80 0x55c796e7fa80 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fdf400ce360 tx=0x7fdf4009c6f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a18b9f79a4f9f884 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:49.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.198+0000 7fdf33fff640 1 -- 172.21.15.86:0/3131746820 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf400a0f00 con 0x55c796e86e80 2024-04-21T08:44:49.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.198+0000 7fdf33fff640 1 -- 172.21.15.86:0/3131746820 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf400a57c0 con 0x55c796e86e80 2024-04-21T08:44:49.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.198+0000 7fdf33fff640 1 -- 172.21.15.86:0/3131746820 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf40099350 con 0x55c796e86e80 2024-04-21T08:44:49.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.198+0000 7fdf53ba9740 1 -- 172.21.15.86:0/3131746820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c796e86e80 msgr2=0x55c796e7fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:49.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.198+0000 7fdf53ba9740 1 --2- 172.21.15.86:0/3131746820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c796e86e80 0x55c796e7fa80 secure :-1 s=READY pgs=1352 cs=0 l=1 rev1=1 crypto rx=0x7fdf400ce360 tx=0x7fdf4009c6f0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.198+0000 7fdf53ba9740 1 -- 172.21.15.86:0/3131746820 shutdown_connections 2024-04-21T08:44:49.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.198+0000 7fdf53ba9740 1 --2- 172.21.15.86:0/3131746820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c796e8fa00 0x55c796e8ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.198+0000 7fdf53ba9740 1 --2- 172.21.15.86:0/3131746820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c796e86e80 0x55c796e7fa80 unknown :-1 s=CLOSED pgs=1352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.198+0000 7fdf53ba9740 1 --2- 172.21.15.86:0/3131746820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c796e864d0 0x55c796e9b800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.198+0000 7fdf53ba9740 1 -- 172.21.15.86:0/3131746820 >> 172.21.15.86:0/3131746820 conn(0x55c796e80ba0 msgr2=0x55c796e7f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:49.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.198+0000 7fdf53ba9740 1 -- 172.21.15.86:0/3131746820 shutdown_connections 2024-04-21T08:44:49.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.198+0000 7fdf53ba9740 1 -- 172.21.15.86:0/3131746820 wait complete. 2024-04-21T08:44:49.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.199+0000 7fdf53ba9740 1 -- start start 2024-04-21T08:44:49.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.199+0000 7fdf53ba9740 1 Processor -- start 2024-04-21T08:44:49.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.199+0000 7fdf53ba9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c796e864d0 0x55c796e97f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.199+0000 7fdf53ba9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c796e86e80 0x55c796e932d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.199+0000 7fdf53ba9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c796e8fa00 0x55c796e93810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.199+0000 7fdf53ba9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4c1b29f0 con 0x55c796e8fa00 2024-04-21T08:44:49.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.199+0000 7fdf53ba9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4c1b2320 con 0x55c796e864d0 2024-04-21T08:44:49.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.199+0000 7fdf53ba9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdf4c1af540 con 0x55c796e86e80 2024-04-21T08:44:49.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.199+0000 7fdf53ba9740 10 client.0 _ll_register_callbacks cb 0x7fdf3c02c6d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:49.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.199+0000 7fdf4ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c796e8fa00 0x55c796e93810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:49.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.199+0000 7fdf4ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c796e8fa00 0x55c796e93810 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42630/0 (socket says 172.21.15.86:42630) 2024-04-21T08:44:49.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.199+0000 7fdf4ad9d640 1 -- 172.21.15.86:0/2799477059 learned_addr learned my addr 172.21.15.86:0/2799477059 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:49.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.199+0000 7fdf49d9b640 1 --2- 172.21.15.86:0/2799477059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c796e86e80 0x55c796e932d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:49.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.199+0000 7fdf4a59c640 1 --2- 172.21.15.86:0/2799477059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c796e864d0 0x55c796e97f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:49.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.199+0000 7fdf4ad9d640 1 -- 172.21.15.86:0/2799477059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c796e86e80 msgr2=0x55c796e932d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:49.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.199+0000 7fdf4ad9d640 1 --2- 172.21.15.86:0/2799477059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c796e86e80 0x55c796e932d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.199+0000 7fdf4ad9d640 1 -- 172.21.15.86:0/2799477059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c796e864d0 msgr2=0x55c796e97f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:49.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.199+0000 7fdf4ad9d640 1 --2- 172.21.15.86:0/2799477059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c796e864d0 0x55c796e97f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.199+0000 7fdf4ad9d640 1 -- 172.21.15.86:0/2799477059 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c796e93e50 con 0x55c796e8fa00 2024-04-21T08:44:49.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.199+0000 7fdf4ad9d640 1 --2- 172.21.15.86:0/2799477059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c796e8fa00 0x55c796e93810 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fdf440129f0 tx=0x7fdf44009500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b2ea75b3b229dac server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:49.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.199+0000 7fdf31ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:49.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.200+0000 7fdf31ffb640 1 -- 172.21.15.86:0/2799477059 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf44003e40 con 0x55c796e8fa00 2024-04-21T08:44:49.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.200+0000 7fdf53ba9740 1 -- 172.21.15.86:0/2799477059 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf400a57c0 con 0x55c796e8fa00 2024-04-21T08:44:49.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.200+0000 7fdf31ffb640 1 -- 172.21.15.86:0/2799477059 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fdf44010950 con 0x55c796e8fa00 2024-04-21T08:44:49.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.200+0000 7fdf31ffb640 1 -- 172.21.15.86:0/2799477059 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdf44010b50 con 0x55c796e8fa00 2024-04-21T08:44:49.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.200+0000 7fdf53ba9740 10 client.18234 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:49.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.200+0000 7fdf53ba9740 1 -- 172.21.15.86:0/2799477059 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c796f05be0 con 0x55c796e8fa00 2024-04-21T08:44:49.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.200+0000 7fdf31ffb640 1 -- 172.21.15.86:0/2799477059 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf400a57c0 con 0x55c796e8fa00 2024-04-21T08:44:49.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.200+0000 7fdf31ffb640 1 -- 172.21.15.86:0/2799477059 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdf4401f550 con 0x55c796e8fa00 2024-04-21T08:44:49.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.200+0000 7fdf53ba9740 10 client.18234 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:49.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.200+0000 7fdf53ba9740 10 client.18234 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:49.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.200+0000 7fdf53ba9740 20 client.18234 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:49.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.200+0000 7fdf53ba9740 1 --2- 172.21.15.86:0/2799477059 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c796f06390 0x55c796f087b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:49.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.200+0000 7fdf53ba9740 4 client.18234 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:44:49.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.200+0000 7fdf53ba9740 1 -- 172.21.15.86:0/2799477059 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c796df8c10 con 0x55c796f06390 2024-04-21T08:44:49.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.200+0000 7fdf4a59c640 1 --2- 172.21.15.86:0/2799477059 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c796f06390 0x55c796f087b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:49.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.201+0000 7fdf4a59c640 1 --2- 172.21.15.86:0/2799477059 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c796f06390 0x55c796f087b0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15772874ae25e87e server_cookie=e7d4ea44f770c6e2 in_seq=0 out_seq=0 2024-04-21T08:44:49.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.201+0000 7fdf31ffb640 10 client.18234 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:49.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.201+0000 7fdf31ffb640 1 -- 172.21.15.86:0/2799477059 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c796df8c10 con 0x55c796f06390 2024-04-21T08:44:49.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.201+0000 7fdf31ffb640 10 client.18234 handle_command_reply: tid=0 2024-04-21T08:44:49.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.201+0000 7fdf53ba9740 1 client.18234 shutdown 2024-04-21T08:44:49.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.201+0000 7fdf53ba9740 1 -- 172.21.15.86:0/2799477059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c796e8fa00 msgr2=0x55c796e93810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:49.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.201+0000 7fdf53ba9740 1 --2- 172.21.15.86:0/2799477059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c796e8fa00 0x55c796e93810 secure :-1 s=READY pgs=1353 cs=0 l=1 rev1=1 crypto rx=0x7fdf440129f0 tx=0x7fdf44009500 comp rx=0 tx=0).stop 2024-04-21T08:44:49.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.201+0000 7fdf53ba9740 1 -- 172.21.15.86:0/2799477059 shutdown_connections 2024-04-21T08:44:49.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.201+0000 7fdf53ba9740 1 --2- 172.21.15.86:0/2799477059 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c796f06390 0x55c796f087b0 crc :-1 s=READY pgs=196 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.201+0000 7fdf53ba9740 1 --2- 172.21.15.86:0/2799477059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c796e8fa00 0x55c796e93810 unknown :-1 s=CLOSED pgs=1353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.201+0000 7fdf31ffb640 0 client.18234 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:49.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.201+0000 7fdf53ba9740 1 --2- 172.21.15.86:0/2799477059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c796e86e80 0x55c796e932d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.201+0000 7fdf53ba9740 1 --2- 172.21.15.86:0/2799477059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c796e864d0 0x55c796e97f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.201+0000 7fdf53ba9740 1 -- 172.21.15.86:0/2799477059 >> 172.21.15.86:0/2799477059 conn(0x55c796e80ba0 msgr2=0x55c796e7fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:49.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.201+0000 7fdf53ba9740 1 -- 172.21.15.86:0/2799477059 shutdown_connections 2024-04-21T08:44:49.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.201+0000 7fdf53ba9740 1 -- 172.21.15.86:0/2799477059 wait complete. 2024-04-21T08:44:49.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.201+0000 7fdf53ba9740 20 client.18234 trim_cache size 0 max 16384 2024-04-21T08:44:49.208 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:44:49.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.202+0000 7fdf50ed9640 1 -- 172.21.15.86:0/2721957268 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdf340a3d50 msgr2=0x7fdf340a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:49.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.202+0000 7fdf50ed9640 1 --2- 172.21.15.86:0/2721957268 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdf340a3d50 0x7fdf340a6210 secure :-1 s=READY pgs=214 cs=0 l=1 rev1=1 crypto rx=0x7fdf44005fd0 tx=0x7fdf44004a40 comp rx=0 tx=0).stop 2024-04-21T08:44:49.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.202+0000 7fdf50ed9640 1 -- 172.21.15.86:0/2721957268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf4c107280 msgr2=0x7fdf4c1017b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:49.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.202+0000 7fdf50ed9640 1 --2- 172.21.15.86:0/2721957268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf4c107280 0x7fdf4c1017b0 secure :-1 s=READY pgs=1349 cs=0 l=1 rev1=1 crypto rx=0x7fdf4000e970 tx=0x7fdf4000ee30 comp rx=0 tx=0).stop 2024-04-21T08:44:49.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.202+0000 7fdf50ed9640 1 -- 172.21.15.86:0/2721957268 shutdown_connections 2024-04-21T08:44:49.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.202+0000 7fdf50ed9640 1 --2- 172.21.15.86:0/2721957268 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdf340a3d50 0x7fdf340a6210 unknown :-1 s=CLOSED pgs=214 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.202+0000 7fdf50ed9640 1 --2- 172.21.15.86:0/2721957268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf4c101cf0 0x7fdf4c1a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.202+0000 7fdf50ed9640 1 --2- 172.21.15.86:0/2721957268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf4c107280 0x7fdf4c1017b0 unknown :-1 s=CLOSED pgs=1349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.202+0000 7fdf50ed9640 1 --2- 172.21.15.86:0/2721957268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf4c06a7c0 0x7fdf4c101270 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:49.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.202+0000 7fdf50ed9640 1 -- 172.21.15.86:0/2721957268 >> 172.21.15.86:0/2721957268 conn(0x7fdf4c076c50 msgr2=0x7fdf4c0ff4a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:49.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.202+0000 7fdf50ed9640 1 -- 172.21.15.86:0/2721957268 shutdown_connections 2024-04-21T08:44:49.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:49.202+0000 7fdf50ed9640 1 -- 172.21.15.86:0/2721957268 wait complete. 2024-04-21T08:44:49.213 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:48 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1903634077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:49.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:48 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1903634077' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:44:50.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:49 smithi086 ceph-mon[27939]: pgmap v184: 129 pgs: 129 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-04-21T08:44:50.213 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:44:50.213 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:44:50.213 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:44:50.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:49 smithi088 ceph-mon[31259]: pgmap v184: 129 pgs: 129 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-04-21T08:44:50.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:49 smithi179 ceph-mon[31829]: pgmap v184: 129 pgs: 129 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 55 KiB/s wr, 22 op/s 2024-04-21T08:44:50.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.329+0000 7f4f10b46640 1 -- 172.21.15.86:0/4064681032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0c1010a0 msgr2=0x7f4f0c101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:50.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.329+0000 7f4f10b46640 1 --2- 172.21.15.86:0/4064681032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0c1010a0 0x7f4f0c101480 secure :-1 s=READY pgs=1354 cs=0 l=1 rev1=1 crypto rx=0x7f4efc009900 tx=0x7f4efc02bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:50.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.329+0000 7f4f10b46640 1 -- 172.21.15.86:0/4064681032 shutdown_connections 2024-04-21T08:44:50.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.329+0000 7f4f10b46640 1 --2- 172.21.15.86:0/4064681032 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f0c106170 0x7f4f0c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.329+0000 7f4f10b46640 1 --2- 172.21.15.86:0/4064681032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f0c101a50 0x7f4f0c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.329+0000 7f4f10b46640 1 --2- 172.21.15.86:0/4064681032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0c1010a0 0x7f4f0c101480 unknown :-1 s=CLOSED pgs=1354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.329+0000 7f4f10b46640 1 -- 172.21.15.86:0/4064681032 >> 172.21.15.86:0/4064681032 conn(0x7f4f0c077eb0 msgr2=0x7f4f0c0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:50.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.329+0000 7f4f10b46640 1 -- 172.21.15.86:0/4064681032 shutdown_connections 2024-04-21T08:44:50.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.329+0000 7f4f10b46640 1 -- 172.21.15.86:0/4064681032 wait complete. 2024-04-21T08:44:50.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.330+0000 7f4f10b46640 1 Processor -- start 2024-04-21T08:44:50.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.330+0000 7f4f10b46640 1 -- start start 2024-04-21T08:44:50.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.330+0000 7f4f10b46640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f0c1010a0 0x7f4f0c1986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:50.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.330+0000 7f4f10b46640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f0c106170 0x7f4f0c198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:50.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.330+0000 7f4f10b46640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0c19cfa0 0x7f4f0c19d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:50.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.330+0000 7f4f10b46640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c1110b0 con 0x7f4f0c19cfa0 2024-04-21T08:44:50.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.330+0000 7f4f10b46640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c110f30 con 0x7f4f0c106170 2024-04-21T08:44:50.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.330+0000 7f4f10b46640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c111230 con 0x7f4f0c1010a0 2024-04-21T08:44:50.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.330+0000 7f4f0ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0c19cfa0 0x7f4f0c19d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:50.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.330+0000 7f4f0ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0c19cfa0 0x7f4f0c19d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32850/0 (socket says 172.21.15.86:32850) 2024-04-21T08:44:50.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.330+0000 7f4f0ad9d640 1 -- 172.21.15.86:0/1058154571 learned_addr learned my addr 172.21.15.86:0/1058154571 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:50.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.330+0000 7f4f03fff640 1 --2- 172.21.15.86:0/1058154571 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f0c106170 0x7f4f0c198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:50.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.330+0000 7f4f0ad9d640 1 -- 172.21.15.86:0/1058154571 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f0c1010a0 msgr2=0x7f4f0c1986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:50.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.330+0000 7f4f0ad9d640 1 --2- 172.21.15.86:0/1058154571 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f0c1010a0 0x7f4f0c1986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.330+0000 7f4f0ad9d640 1 -- 172.21.15.86:0/1058154571 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f0c106170 msgr2=0x7f4f0c198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:50.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.330+0000 7f4f0ad9d640 1 --2- 172.21.15.86:0/1058154571 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f0c106170 0x7f4f0c198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.330+0000 7f4f0ad9d640 1 -- 172.21.15.86:0/1058154571 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4f0c19da90 con 0x7f4f0c19cfa0 2024-04-21T08:44:50.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.330+0000 7f4f0ad9d640 1 --2- 172.21.15.86:0/1058154571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0c19cfa0 0x7f4f0c19d450 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f4f0400eb10 tx=0x7f4f0400ef10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:50.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.330+0000 7f4f037fe640 1 -- 172.21.15.86:0/1058154571 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f0400ce20 con 0x7f4f0c19cfa0 2024-04-21T08:44:50.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.331+0000 7f4f10b46640 1 -- 172.21.15.86:0/1058154571 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4f0c19dd80 con 0x7f4f0c19cfa0 2024-04-21T08:44:50.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.331+0000 7f4f037fe640 1 -- 172.21.15.86:0/1058154571 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f04005020 con 0x7f4f0c19cfa0 2024-04-21T08:44:50.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.331+0000 7f4f037fe640 1 -- 172.21.15.86:0/1058154571 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f04010690 con 0x7f4f0c19cfa0 2024-04-21T08:44:50.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.331+0000 7f4f10b46640 1 -- 172.21.15.86:0/1058154571 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4f0c1a5570 con 0x7f4f0c19cfa0 2024-04-21T08:44:50.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.333+0000 7f4f037fe640 1 -- 172.21.15.86:0/1058154571 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4f04005510 con 0x7f4f0c19cfa0 2024-04-21T08:44:50.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.334+0000 7f4f1380b740 1 Processor -- start 2024-04-21T08:44:50.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.334+0000 7f4f1380b740 1 -- start start 2024-04-21T08:44:50.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.334+0000 7f4f1380b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5604453818e0 0x560445381cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:50.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.334+0000 7f4f037fe640 1 --2- 172.21.15.86:0/1058154571 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4ef80a3d20 0x7f4ef80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:50.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.334+0000 7f4f1380b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560445382290 0x56044538a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:50.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.334+0000 7f4f1380b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56044538ae10 0x56044538d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:50.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.334+0000 7f4f1380b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5604452e5f00 con 0x560445382290 2024-04-21T08:44:50.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.334+0000 7f4f1380b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x560445283590 con 0x56044538ae10 2024-04-21T08:44:50.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.334+0000 7f4f1380b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5604452e1ec0 con 0x5604453818e0 2024-04-21T08:44:50.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.334+0000 7f4f0a59c640 1 --2- 172.21.15.86:0/1058154571 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4ef80a3d20 0x7f4ef80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:50.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.334+0000 7f4f03fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560445382290 0x56044538a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:50.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.334+0000 7f4f03fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560445382290 0x56044538a840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32854/0 (socket says 172.21.15.86:32854) 2024-04-21T08:44:50.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.334+0000 7f4f03fff640 1 -- 172.21.15.86:0/477204987 learned_addr learned my addr 172.21.15.86:0/477204987 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:50.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.334+0000 7f4f0ad9d640 1 --2- 172.21.15.86:0/477204987 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56044538ae10 0x56044538d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:50.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.335+0000 7f4f037fe640 1 -- 172.21.15.86:0/1058154571 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f4f040d1120 con 0x7f4f0c19cfa0 2024-04-21T08:44:50.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.335+0000 7f4f03fff640 1 --2- 172.21.15.86:0/477204987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5604453818e0 0x560445381cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:50.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.335+0000 7f4f03fff640 1 -- 172.21.15.86:0/477204987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5604453818e0 msgr2=0x560445381cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:50.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.335+0000 7f4f03fff640 1 --2- 172.21.15.86:0/477204987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5604453818e0 0x560445381cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.335+0000 7f4f03fff640 1 -- 172.21.15.86:0/477204987 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56044538ae10 msgr2=0x56044538d260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:50.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.335+0000 7f4f03fff640 1 --2- 172.21.15.86:0/477204987 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56044538ae10 0x56044538d260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.335+0000 7f4f03fff640 1 -- 172.21.15.86:0/477204987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5604450c07e0 con 0x560445382290 2024-04-21T08:44:50.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.335+0000 7f4f0ad9d640 1 --2- 172.21.15.86:0/477204987 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56044538ae10 0x56044538d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:44:50.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.335+0000 7f4f0a59c640 1 --2- 172.21.15.86:0/1058154571 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4ef80a3d20 0x7f4ef80a61e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4efc005770 tx=0x7f4efc000bc0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:50.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.335+0000 7f4f03fff640 1 --2- 172.21.15.86:0/477204987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560445382290 0x56044538a840 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f4ef400afd0 tx=0x7f4ef400a7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e8a7a6bd0ae60da server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:50.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.335+0000 7f4f00ff9640 1 -- 172.21.15.86:0/477204987 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef4005220 con 0x560445382290 2024-04-21T08:44:50.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.335+0000 7f4f00ff9640 1 -- 172.21.15.86:0/477204987 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ef4004c50 con 0x560445382290 2024-04-21T08:44:50.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.335+0000 7f4f00ff9640 1 -- 172.21.15.86:0/477204987 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ef400faa0 con 0x560445382290 2024-04-21T08:44:50.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.336+0000 7f4f1380b740 1 -- 172.21.15.86:0/477204987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560445382290 msgr2=0x56044538a840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:50.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.336+0000 7f4f1380b740 1 --2- 172.21.15.86:0/477204987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560445382290 0x56044538a840 secure :-1 s=READY pgs=1356 cs=0 l=1 rev1=1 crypto rx=0x7f4ef400afd0 tx=0x7f4ef400a7d0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.336+0000 7f4f1380b740 1 -- 172.21.15.86:0/477204987 shutdown_connections 2024-04-21T08:44:50.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.336+0000 7f4f1380b740 1 --2- 172.21.15.86:0/477204987 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56044538ae10 0x56044538d260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.336+0000 7f4f1380b740 1 --2- 172.21.15.86:0/477204987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560445382290 0x56044538a840 unknown :-1 s=CLOSED pgs=1356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.336+0000 7f4f1380b740 1 --2- 172.21.15.86:0/477204987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5604453818e0 0x560445381cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.336+0000 7f4f1380b740 1 -- 172.21.15.86:0/477204987 >> 172.21.15.86:0/477204987 conn(0x56044537bfb0 msgr2=0x56044537c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:50.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.336+0000 7f4f1380b740 1 -- 172.21.15.86:0/477204987 shutdown_connections 2024-04-21T08:44:50.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.336+0000 7f4f1380b740 1 -- 172.21.15.86:0/477204987 wait complete. 2024-04-21T08:44:50.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.336+0000 7f4f1380b740 1 -- start start 2024-04-21T08:44:50.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.336+0000 7f4f1380b740 1 Processor -- start 2024-04-21T08:44:50.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.337+0000 7f4f1380b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5604453818e0 0x560445387e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:50.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.337+0000 7f4f1380b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560445382290 0x560445388390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:50.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.337+0000 7f4f1380b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56044538ae10 0x5604453888d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:50.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.337+0000 7f4f1380b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56044538dd80 con 0x56044538ae10 2024-04-21T08:44:50.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.337+0000 7f4f1380b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x56044529f580 con 0x560445382290 2024-04-21T08:44:50.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.337+0000 7f4f1380b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56044538df00 con 0x5604453818e0 2024-04-21T08:44:50.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.337+0000 7f4f1380b740 10 client.0 _ll_register_callbacks cb 0x5604452a5650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:50.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.337+0000 7f4f0ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56044538ae10 0x5604453888d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:50.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.337+0000 7f4f0ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56044538ae10 0x5604453888d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32868/0 (socket says 172.21.15.86:32868) 2024-04-21T08:44:50.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.337+0000 7f4f0ad9d640 1 -- 172.21.15.86:0/2576134958 learned_addr learned my addr 172.21.15.86:0/2576134958 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:50.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.337+0000 7f4f0a59c640 1 --2- 172.21.15.86:0/2576134958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5604453818e0 0x560445387e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:50.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.337+0000 7f4f03fff640 1 --2- 172.21.15.86:0/2576134958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560445382290 0x560445388390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:50.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.337+0000 7f4f0ad9d640 1 -- 172.21.15.86:0/2576134958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5604453818e0 msgr2=0x560445387e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:50.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.337+0000 7f4f0ad9d640 1 --2- 172.21.15.86:0/2576134958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5604453818e0 0x560445387e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.337+0000 7f4f0ad9d640 1 -- 172.21.15.86:0/2576134958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560445382290 msgr2=0x560445388390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:50.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.337+0000 7f4f0ad9d640 1 --2- 172.21.15.86:0/2576134958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560445382290 0x560445388390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.337+0000 7f4f0ad9d640 1 -- 172.21.15.86:0/2576134958 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560445120f70 con 0x56044538ae10 2024-04-21T08:44:50.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.337+0000 7f4f0ad9d640 1 --2- 172.21.15.86:0/2576134958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56044538ae10 0x5604453888d0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f4f0409ca50 tx=0x7f4f0409ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb9ef182b18af893 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:50.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.337+0000 7f4ef1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:50.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.337+0000 7f4ef1ffb640 1 -- 172.21.15.86:0/2576134958 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f040a59c0 con 0x56044538ae10 2024-04-21T08:44:50.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.337+0000 7f4ef1ffb640 1 -- 172.21.15.86:0/2576134958 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f4f04016dd0 con 0x56044538ae10 2024-04-21T08:44:50.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.337+0000 7f4f1380b740 1 -- 172.21.15.86:0/2576134958 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ef4004c50 con 0x56044538ae10 2024-04-21T08:44:50.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.337+0000 7f4ef1ffb640 1 -- 172.21.15.86:0/2576134958 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4f040ce780 con 0x56044538ae10 2024-04-21T08:44:50.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.337+0000 7f4f1380b740 10 client.18246 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:50.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.337+0000 7f4f1380b740 1 -- 172.21.15.86:0/2576134958 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560444fb8170 con 0x56044538ae10 2024-04-21T08:44:50.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.338+0000 7f4ef1ffb640 1 -- 172.21.15.86:0/2576134958 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ef4004c50 con 0x56044538ae10 2024-04-21T08:44:50.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.338+0000 7f4ef1ffb640 1 -- 172.21.15.86:0/2576134958 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4f04098aa0 con 0x56044538ae10 2024-04-21T08:44:50.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.338+0000 7f4f1380b740 10 client.18246 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:50.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.338+0000 7f4f1380b740 10 client.18246 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:50.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.338+0000 7f4f1380b740 20 client.18246 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:50.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.338+0000 7f4f1380b740 1 --2- 172.21.15.86:0/2576134958 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56044539b350 0x56044539d740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:50.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.338+0000 7f4f1380b740 4 client.18246 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:50.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.338+0000 7f4f1380b740 1 -- 172.21.15.86:0/2576134958 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5604452f3920 con 0x56044539b350 2024-04-21T08:44:50.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.338+0000 7f4f0a59c640 1 --2- 172.21.15.86:0/2576134958 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56044539b350 0x56044539d740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:50.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.338+0000 7f4ef1ffb640 10 client.18246 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:50.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.338+0000 7f4f0a59c640 1 --2- 172.21.15.86:0/2576134958 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56044539b350 0x56044539d740 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14742d12dca61720 server_cookie=ab528250be8b99b1 in_seq=0 out_seq=0 2024-04-21T08:44:50.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.339+0000 7f4ef1ffb640 1 -- 172.21.15.86:0/2576134958 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5604452f3920 con 0x56044539b350 2024-04-21T08:44:50.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.339+0000 7f4ef1ffb640 10 client.18246 handle_command_reply: tid=0 2024-04-21T08:44:50.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.339+0000 7f4f1380b740 1 client.18246 shutdown 2024-04-21T08:44:50.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.339+0000 7f4f1380b740 1 -- 172.21.15.86:0/2576134958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56044538ae10 msgr2=0x5604453888d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:50.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.339+0000 7f4f1380b740 1 --2- 172.21.15.86:0/2576134958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56044538ae10 0x5604453888d0 secure :-1 s=READY pgs=1357 cs=0 l=1 rev1=1 crypto rx=0x7f4f0409ca50 tx=0x7f4f0409ce10 comp rx=0 tx=0).stop 2024-04-21T08:44:50.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.339+0000 7f4f1380b740 1 -- 172.21.15.86:0/2576134958 shutdown_connections 2024-04-21T08:44:50.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.339+0000 7f4f1380b740 1 --2- 172.21.15.86:0/2576134958 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56044539b350 0x56044539d740 crc :-1 s=READY pgs=197 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.339+0000 7f4f1380b740 1 --2- 172.21.15.86:0/2576134958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56044538ae10 0x5604453888d0 unknown :-1 s=CLOSED pgs=1357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.339+0000 7f4f1380b740 1 --2- 172.21.15.86:0/2576134958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560445382290 0x560445388390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.339+0000 7f4ef1ffb640 0 client.18246 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:50.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.339+0000 7f4f1380b740 1 --2- 172.21.15.86:0/2576134958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5604453818e0 0x560445387e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.339+0000 7f4f1380b740 1 -- 172.21.15.86:0/2576134958 >> 172.21.15.86:0/2576134958 conn(0x56044537bfb0 msgr2=0x56044537dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:50.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.339+0000 7f4f1380b740 1 -- 172.21.15.86:0/2576134958 shutdown_connections 2024-04-21T08:44:50.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.339+0000 7f4f1380b740 1 -- 172.21.15.86:0/2576134958 wait complete. 2024-04-21T08:44:50.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.340+0000 7f4f1380b740 20 client.18246 trim_cache size 0 max 16384 2024-04-21T08:44:50.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.357+0000 7f4f1380b740 1 Processor -- start 2024-04-21T08:44:50.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.357+0000 7f4f1380b740 1 -- start start 2024-04-21T08:44:50.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.357+0000 7f4f1380b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5604453818e0 0x56044538a580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:50.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.357+0000 7f4f1380b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560445382290 0x560445396c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:50.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.357+0000 7f4f1380b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56044538ae10 0x5604453a3060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:50.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.357+0000 7f4f1380b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c1af290 con 0x56044538ae10 2024-04-21T08:44:50.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.357+0000 7f4f1380b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c1b0dd0 con 0x560445382290 2024-04-21T08:44:50.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.357+0000 7f4f1380b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c1af960 con 0x5604453818e0 2024-04-21T08:44:50.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.357+0000 7f4f0ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56044538ae10 0x5604453a3060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:50.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.357+0000 7f4f0ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56044538ae10 0x5604453a3060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32872/0 (socket says 172.21.15.86:32872) 2024-04-21T08:44:50.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.357+0000 7f4f0ad9d640 1 -- 172.21.15.86:0/3105836349 learned_addr learned my addr 172.21.15.86:0/3105836349 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:50.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.357+0000 7f4f0a59c640 1 --2- 172.21.15.86:0/3105836349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5604453818e0 0x56044538a580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:50.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.357+0000 7f4f03fff640 1 --2- 172.21.15.86:0/3105836349 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560445382290 0x560445396c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:50.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.357+0000 7f4f0ad9d640 1 -- 172.21.15.86:0/3105836349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5604453818e0 msgr2=0x56044538a580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:50.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.357+0000 7f4f0ad9d640 1 --2- 172.21.15.86:0/3105836349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5604453818e0 0x56044538a580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.357+0000 7f4f0ad9d640 1 -- 172.21.15.86:0/3105836349 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560445382290 msgr2=0x560445396c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:50.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.357+0000 7f4f0ad9d640 1 --2- 172.21.15.86:0/3105836349 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560445382290 0x560445396c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.357+0000 7f4f0ad9d640 1 -- 172.21.15.86:0/3105836349 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560445388db0 con 0x56044538ae10 2024-04-21T08:44:50.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.357+0000 7f4f03fff640 1 --2- 172.21.15.86:0/3105836349 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560445382290 0x560445396c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:50.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.358+0000 7f4f0ad9d640 1 --2- 172.21.15.86:0/3105836349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56044538ae10 0x5604453a3060 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f4f0400c760 tx=0x7f4f0409b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d59cc127762a7785 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:50.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.358+0000 7f4ef3fff640 1 -- 172.21.15.86:0/3105836349 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f04065070 con 0x56044538ae10 2024-04-21T08:44:50.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.358+0000 7f4ef3fff640 1 -- 172.21.15.86:0/3105836349 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4f040ce920 con 0x56044538ae10 2024-04-21T08:44:50.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.358+0000 7f4ef3fff640 1 -- 172.21.15.86:0/3105836349 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f040a5530 con 0x56044538ae10 2024-04-21T08:44:50.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.358+0000 7f4f1380b740 1 -- 172.21.15.86:0/3105836349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56044538ae10 msgr2=0x5604453a3060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:50.363 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.358+0000 7f4f1380b740 1 --2- 172.21.15.86:0/3105836349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56044538ae10 0x5604453a3060 secure :-1 s=READY pgs=1358 cs=0 l=1 rev1=1 crypto rx=0x7f4f0400c760 tx=0x7f4f0409b260 comp rx=0 tx=0).stop 2024-04-21T08:44:50.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.358+0000 7f4f1380b740 1 -- 172.21.15.86:0/3105836349 shutdown_connections 2024-04-21T08:44:50.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.358+0000 7f4f1380b740 1 --2- 172.21.15.86:0/3105836349 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56044538ae10 0x5604453a3060 unknown :-1 s=CLOSED pgs=1358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.358+0000 7f4f1380b740 1 --2- 172.21.15.86:0/3105836349 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560445382290 0x560445396c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.358+0000 7f4f1380b740 1 --2- 172.21.15.86:0/3105836349 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5604453818e0 0x56044538a580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.358+0000 7f4f1380b740 1 -- 172.21.15.86:0/3105836349 >> 172.21.15.86:0/3105836349 conn(0x56044537bfb0 msgr2=0x5604453886a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:50.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.358+0000 7f4f1380b740 1 -- 172.21.15.86:0/3105836349 shutdown_connections 2024-04-21T08:44:50.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.358+0000 7f4f1380b740 1 -- 172.21.15.86:0/3105836349 wait complete. 2024-04-21T08:44:50.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.359+0000 7f4f1380b740 1 -- start start 2024-04-21T08:44:50.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.359+0000 7f4f1380b740 1 Processor -- start 2024-04-21T08:44:50.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.359+0000 7f4f1380b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5604453818e0 0x56044538f730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:50.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.359+0000 7f4f1380b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560445382290 0x56044538fc70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:50.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.359+0000 7f4f1380b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56044538ae10 0x5604453f9890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:50.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.359+0000 7f4f1380b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c1b0700 con 0x56044538ae10 2024-04-21T08:44:50.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.359+0000 7f4f1380b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c1b0030 con 0x5604453818e0 2024-04-21T08:44:50.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.359+0000 7f4f1380b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4f0c1ad250 con 0x560445382290 2024-04-21T08:44:50.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.359+0000 7f4f1380b740 10 client.0 _ll_register_callbacks cb 0x7f4f0409d450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:50.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.359+0000 7f4f0ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56044538ae10 0x5604453f9890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:50.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.359+0000 7f4f0ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56044538ae10 0x5604453f9890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32878/0 (socket says 172.21.15.86:32878) 2024-04-21T08:44:50.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.359+0000 7f4f0ad9d640 1 -- 172.21.15.86:0/3945165480 learned_addr learned my addr 172.21.15.86:0/3945165480 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:50.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.359+0000 7f4f03fff640 1 --2- 172.21.15.86:0/3945165480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560445382290 0x56044538fc70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:50.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.359+0000 7f4f0a59c640 1 --2- 172.21.15.86:0/3945165480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5604453818e0 0x56044538f730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:50.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.359+0000 7f4f0ad9d640 1 -- 172.21.15.86:0/3945165480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560445382290 msgr2=0x56044538fc70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:50.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.359+0000 7f4f0ad9d640 1 --2- 172.21.15.86:0/3945165480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560445382290 0x56044538fc70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.359+0000 7f4f0ad9d640 1 -- 172.21.15.86:0/3945165480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5604453818e0 msgr2=0x56044538f730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:50.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.359+0000 7f4f0ad9d640 1 --2- 172.21.15.86:0/3945165480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5604453818e0 0x56044538f730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.359+0000 7f4f0ad9d640 1 -- 172.21.15.86:0/3945165480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5604453fa090 con 0x56044538ae10 2024-04-21T08:44:50.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.359+0000 7f4f0ad9d640 1 --2- 172.21.15.86:0/3945165480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56044538ae10 0x5604453f9890 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f4f04098f20 tx=0x7f4f0407af90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=52447900d1ba68d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:50.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.359+0000 7f4ef1ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:50.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.360+0000 7f4ef1ffb640 1 -- 172.21.15.86:0/3945165480 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4f04065070 con 0x56044538ae10 2024-04-21T08:44:50.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.360+0000 7f4f1380b740 1 -- 172.21.15.86:0/3945165480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5604452ed790 con 0x56044538ae10 2024-04-21T08:44:50.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.360+0000 7f4ef1ffb640 1 -- 172.21.15.86:0/3945165480 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f4f0407bb70 con 0x56044538ae10 2024-04-21T08:44:50.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.360+0000 7f4ef1ffb640 1 -- 172.21.15.86:0/3945165480 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4f04098ff0 con 0x56044538ae10 2024-04-21T08:44:50.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.360+0000 7f4f1380b740 10 client.18252 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:50.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.360+0000 7f4f1380b740 1 -- 172.21.15.86:0/3945165480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560445402480 con 0x56044538ae10 2024-04-21T08:44:50.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.360+0000 7f4ef1ffb640 1 -- 172.21.15.86:0/3945165480 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5604452ed790 con 0x56044538ae10 2024-04-21T08:44:50.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.360+0000 7f4ef1ffb640 1 -- 172.21.15.86:0/3945165480 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4f0409bdd0 con 0x56044538ae10 2024-04-21T08:44:50.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.360+0000 7f4f1380b740 10 client.18252 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:50.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.360+0000 7f4f1380b740 10 client.18252 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:50.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.360+0000 7f4f1380b740 20 client.18252 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:50.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.360+0000 7f4f1380b740 1 --2- 172.21.15.86:0/3945165480 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x560445402950 0x560445404d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:50.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.360+0000 7f4f1380b740 4 client.18252 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:44:50.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.360+0000 7f4f1380b740 1 -- 172.21.15.86:0/3945165480 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x560445269c50 con 0x560445402950 2024-04-21T08:44:50.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.360+0000 7f4f0a59c640 1 --2- 172.21.15.86:0/3945165480 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x560445402950 0x560445404d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:50.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.361+0000 7f4f0a59c640 1 --2- 172.21.15.86:0/3945165480 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x560445402950 0x560445404d40 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88d8fe2118c6b074 server_cookie=45c78b0147d3900 in_seq=0 out_seq=0 2024-04-21T08:44:50.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.361+0000 7f4ef1ffb640 10 client.18252 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:50.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.362+0000 7f4ef1ffb640 1 -- 172.21.15.86:0/3945165480 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x560445269c50 con 0x560445402950 2024-04-21T08:44:50.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.362+0000 7f4ef1ffb640 10 client.18252 handle_command_reply: tid=0 2024-04-21T08:44:50.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.362+0000 7f4f1380b740 1 client.18252 shutdown 2024-04-21T08:44:50.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.362+0000 7f4f1380b740 1 -- 172.21.15.86:0/3945165480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56044538ae10 msgr2=0x5604453f9890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:50.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.362+0000 7f4f1380b740 1 --2- 172.21.15.86:0/3945165480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56044538ae10 0x5604453f9890 secure :-1 s=READY pgs=1359 cs=0 l=1 rev1=1 crypto rx=0x7f4f04098f20 tx=0x7f4f0407af90 comp rx=0 tx=0).stop 2024-04-21T08:44:50.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.362+0000 7f4f1380b740 1 -- 172.21.15.86:0/3945165480 shutdown_connections 2024-04-21T08:44:50.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.362+0000 7f4f1380b740 1 --2- 172.21.15.86:0/3945165480 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x560445402950 0x560445404d40 crc :-1 s=READY pgs=198 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.362+0000 7f4ef1ffb640 0 client.18252 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:50.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.362+0000 7f4f1380b740 1 --2- 172.21.15.86:0/3945165480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56044538ae10 0x5604453f9890 unknown :-1 s=CLOSED pgs=1359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.362+0000 7f4f1380b740 1 --2- 172.21.15.86:0/3945165480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560445382290 0x56044538fc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.362+0000 7f4f1380b740 1 --2- 172.21.15.86:0/3945165480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5604453818e0 0x56044538f730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.362+0000 7f4f1380b740 1 -- 172.21.15.86:0/3945165480 >> 172.21.15.86:0/3945165480 conn(0x56044537bfb0 msgr2=0x56044537a780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:50.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.362+0000 7f4f1380b740 1 -- 172.21.15.86:0/3945165480 shutdown_connections 2024-04-21T08:44:50.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.362+0000 7f4f1380b740 1 -- 172.21.15.86:0/3945165480 wait complete. 2024-04-21T08:44:50.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.362+0000 7f4f1380b740 20 client.18252 trim_cache size 0 max 16384 2024-04-21T08:44:50.368 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:44:50.368 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-21T08:44:50.368 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "2b835aff-748b-4ace-b316-68f328cad4a0", 2024-04-21T08:44:50.368 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-21T08:44:50.368 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:44:50.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.363+0000 7f4f10b46640 1 -- 172.21.15.86:0/1058154571 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4ef80a3d20 msgr2=0x7f4ef80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:50.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.363+0000 7f4f10b46640 1 --2- 172.21.15.86:0/1058154571 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4ef80a3d20 0x7f4ef80a61e0 secure :-1 s=READY pgs=215 cs=0 l=1 rev1=1 crypto rx=0x7f4efc005770 tx=0x7f4efc000bc0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.363+0000 7f4f10b46640 1 -- 172.21.15.86:0/1058154571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0c19cfa0 msgr2=0x7f4f0c19d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:50.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.363+0000 7f4f10b46640 1 --2- 172.21.15.86:0/1058154571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0c19cfa0 0x7f4f0c19d450 secure :-1 s=READY pgs=1355 cs=0 l=1 rev1=1 crypto rx=0x7f4f0400eb10 tx=0x7f4f0400ef10 comp rx=0 tx=0).stop 2024-04-21T08:44:50.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.363+0000 7f4f10b46640 1 -- 172.21.15.86:0/1058154571 shutdown_connections 2024-04-21T08:44:50.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.363+0000 7f4f10b46640 1 --2- 172.21.15.86:0/1058154571 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4ef80a3d20 0x7f4ef80a61e0 unknown :-1 s=CLOSED pgs=215 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.363+0000 7f4f10b46640 1 --2- 172.21.15.86:0/1058154571 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4f0c19cfa0 0x7f4f0c19d450 unknown :-1 s=CLOSED pgs=1355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.363+0000 7f4f10b46640 1 --2- 172.21.15.86:0/1058154571 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4f0c106170 0x7f4f0c198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.363+0000 7f4f10b46640 1 --2- 172.21.15.86:0/1058154571 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4f0c1010a0 0x7f4f0c1986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.363+0000 7f4f10b46640 1 -- 172.21.15.86:0/1058154571 >> 172.21.15.86:0/1058154571 conn(0x7f4f0c077eb0 msgr2=0x7f4f0c100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:50.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.363+0000 7f4f10b46640 1 -- 172.21.15.86:0/1058154571 shutdown_connections 2024-04-21T08:44:50.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.363+0000 7f4f10b46640 1 -- 172.21.15.86:0/1058154571 wait complete. 2024-04-21T08:44:50.373 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:44:50.373 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:44:50.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.473+0000 7fd9cbce4640 1 -- 172.21.15.86:0/3200762693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9c4100610 msgr2=0x7fd9c4100a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:50.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.473+0000 7fd9cbce4640 1 --2- 172.21.15.86:0/3200762693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9c4100610 0x7fd9c4100a90 secure :-1 s=READY pgs=1360 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0009900 tx=0x7fd9c002bc30 comp rx=0 tx=0).stop 2024-04-21T08:44:50.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.473+0000 7fd9cbce4640 1 -- 172.21.15.86:0/3200762693 shutdown_connections 2024-04-21T08:44:50.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.473+0000 7fd9cbce4640 1 --2- 172.21.15.86:0/3200762693 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9c4100fd0 0x7fd9c410d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.473+0000 7fd9cbce4640 1 --2- 172.21.15.86:0/3200762693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9c4100610 0x7fd9c4100a90 unknown :-1 s=CLOSED pgs=1360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.473+0000 7fd9cbce4640 1 --2- 172.21.15.86:0/3200762693 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9c4106680 0x7fd9c4106a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.473+0000 7fd9cbce4640 1 -- 172.21.15.86:0/3200762693 >> 172.21.15.86:0/3200762693 conn(0x7fd9c4077360 msgr2=0x7fd9c40fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:50.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.473+0000 7fd9cbce4640 1 -- 172.21.15.86:0/3200762693 shutdown_connections 2024-04-21T08:44:50.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.473+0000 7fd9cbce4640 1 -- 172.21.15.86:0/3200762693 wait complete. 2024-04-21T08:44:50.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.474+0000 7fd9cbce4640 1 Processor -- start 2024-04-21T08:44:50.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.474+0000 7fd9cbce4640 1 -- start start 2024-04-21T08:44:50.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.474+0000 7fd9cbce4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9c4100610 0x7fd9c4198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:50.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.474+0000 7fd9cbce4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9c4100fd0 0x7fd9c4198b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:50.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.474+0000 7fd9cbce4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9c4106680 0x7fd9c419cee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:50.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.474+0000 7fd9cbce4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c41110a0 con 0x7fd9c4100fd0 2024-04-21T08:44:50.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.474+0000 7fd9cbce4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c4110f20 con 0x7fd9c4106680 2024-04-21T08:44:50.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.474+0000 7fd9cbce4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c4111220 con 0x7fd9c4100610 2024-04-21T08:44:50.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.474+0000 7fd9c927f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9c4100fd0 0x7fd9c4198b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:50.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.474+0000 7fd9c927f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9c4100fd0 0x7fd9c4198b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32892/0 (socket says 172.21.15.86:32892) 2024-04-21T08:44:50.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.474+0000 7fd9c927f640 1 -- 172.21.15.86:0/3441232350 learned_addr learned my addr 172.21.15.86:0/3441232350 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:50.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.474+0000 7fd9c9a80640 1 --2- 172.21.15.86:0/3441232350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9c4100610 0x7fd9c4198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:50.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.474+0000 7fd9ca281640 1 --2- 172.21.15.86:0/3441232350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9c4106680 0x7fd9c419cee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:50.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.474+0000 7fd9c927f640 1 -- 172.21.15.86:0/3441232350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9c4100610 msgr2=0x7fd9c4198620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:50.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.474+0000 7fd9c927f640 1 --2- 172.21.15.86:0/3441232350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9c4100610 0x7fd9c4198620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.474+0000 7fd9c927f640 1 -- 172.21.15.86:0/3441232350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9c4106680 msgr2=0x7fd9c419cee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:50.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.474+0000 7fd9c927f640 1 --2- 172.21.15.86:0/3441232350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9c4106680 0x7fd9c419cee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.474+0000 7fd9c927f640 1 -- 172.21.15.86:0/3441232350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd9c419d610 con 0x7fd9c4100fd0 2024-04-21T08:44:50.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.474+0000 7fd9ca281640 1 --2- 172.21.15.86:0/3441232350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9c4106680 0x7fd9c419cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:44:50.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.474+0000 7fd9c927f640 1 --2- 172.21.15.86:0/3441232350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9c4100fd0 0x7fd9c4198b60 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0006010 tx=0x7fd9c0004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:50.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.475+0000 7fd9baffd640 1 -- 172.21.15.86:0/3441232350 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c0043020 con 0x7fd9c4100fd0 2024-04-21T08:44:50.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.475+0000 7fd9baffd640 1 -- 172.21.15.86:0/3441232350 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9c0004440 con 0x7fd9c4100fd0 2024-04-21T08:44:50.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.475+0000 7fd9cbce4640 1 -- 172.21.15.86:0/3441232350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd9c419d8a0 con 0x7fd9c4100fd0 2024-04-21T08:44:50.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.475+0000 7fd9baffd640 1 -- 172.21.15.86:0/3441232350 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c0003b50 con 0x7fd9c4100fd0 2024-04-21T08:44:50.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.475+0000 7fd9cbce4640 1 -- 172.21.15.86:0/3441232350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd9c41a5190 con 0x7fd9c4100fd0 2024-04-21T08:44:50.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.477+0000 7fd9baffd640 1 -- 172.21.15.86:0/3441232350 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd9c0046020 con 0x7fd9c4100fd0 2024-04-21T08:44:50.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.477+0000 7fd9ce9b3740 1 Processor -- start 2024-04-21T08:44:50.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.477+0000 7fd9ce9b3740 1 -- start start 2024-04-21T08:44:50.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.477+0000 7fd9baffd640 1 --2- 172.21.15.86:0/3441232350 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd9b00a3d20 0x7fd9b00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:50.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.477+0000 7fd9ce9b3740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5651276652b0 0x565127665690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:50.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.477+0000 7fd9ce9b3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565127665bd0 0x565127672950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:50.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.477+0000 7fd9ce9b3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565127672e90 0x565127675280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:50.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.477+0000 7fd9ce9b3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5651275ca1e0 con 0x565127672e90 2024-04-21T08:44:50.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.477+0000 7fd9ce9b3740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x565127567390 con 0x5651276652b0 2024-04-21T08:44:50.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.477+0000 7fd9ce9b3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5651275c61a0 con 0x565127665bd0 2024-04-21T08:44:50.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.478+0000 7fd9ca281640 1 --2- 172.21.15.86:0/3441232350 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd9b00a3d20 0x7fd9b00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:50.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.478+0000 7fd9baffd640 1 -- 172.21.15.86:0/3441232350 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fd9c00f25f0 con 0x7fd9c4100fd0 2024-04-21T08:44:50.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.478+0000 7fd9c927f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565127665bd0 0x565127672950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:50.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.478+0000 7fd9ca281640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565127672e90 0x565127675280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:50.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.478+0000 7fd9c927f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565127665bd0 0x565127672950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.86:54356/0 (socket says 172.21.15.86:54356) 2024-04-21T08:44:50.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.478+0000 7fd9c927f640 1 -- 172.21.15.86:0/1251379960 learned_addr learned my addr 172.21.15.86:0/1251379960 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:50.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.478+0000 7fd9c9a80640 1 --2- 172.21.15.86:0/1251379960 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5651276652b0 0x565127665690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:50.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.478+0000 7fd9ca281640 1 -- 172.21.15.86:0/1251379960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565127665bd0 msgr2=0x565127672950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:50.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.478+0000 7fd9ca281640 1 --2- 172.21.15.86:0/1251379960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565127665bd0 0x565127672950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.478+0000 7fd9ca281640 1 -- 172.21.15.86:0/1251379960 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5651276652b0 msgr2=0x565127665690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:50.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.478+0000 7fd9ca281640 1 --2- 172.21.15.86:0/1251379960 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5651276652b0 0x565127665690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.478+0000 7fd9ca281640 1 -- 172.21.15.86:0/1251379960 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651273a67d0 con 0x565127672e90 2024-04-21T08:44:50.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.478+0000 7fd9ca281640 1 --2- 172.21.15.86:0/3441232350 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd9b00a3d20 0x7fd9b00a61e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc005c40 tx=0x7fd9bc009db0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:50.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.478+0000 7fd9ca281640 1 --2- 172.21.15.86:0/1251379960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565127672e90 0x565127675280 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc012fd0 tx=0x7fd9bc01c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b1e3685fca280f69 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:50.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.478+0000 7fd9affff640 1 -- 172.21.15.86:0/1251379960 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc004b20 con 0x565127672e90 2024-04-21T08:44:50.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.478+0000 7fd9affff640 1 -- 172.21.15.86:0/1251379960 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9bc004cc0 con 0x565127672e90 2024-04-21T08:44:50.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.478+0000 7fd9affff640 1 -- 172.21.15.86:0/1251379960 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc005320 con 0x565127672e90 2024-04-21T08:44:50.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.479+0000 7fd9ce9b3740 1 -- 172.21.15.86:0/1251379960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565127672e90 msgr2=0x565127675280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:50.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.479+0000 7fd9ce9b3740 1 --2- 172.21.15.86:0/1251379960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565127672e90 0x565127675280 secure :-1 s=READY pgs=1362 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc012fd0 tx=0x7fd9bc01c000 comp rx=0 tx=0).stop 2024-04-21T08:44:50.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.479+0000 7fd9ce9b3740 1 -- 172.21.15.86:0/1251379960 shutdown_connections 2024-04-21T08:44:50.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.479+0000 7fd9ce9b3740 1 --2- 172.21.15.86:0/1251379960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565127672e90 0x565127675280 unknown :-1 s=CLOSED pgs=1362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.479+0000 7fd9ce9b3740 1 --2- 172.21.15.86:0/1251379960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565127665bd0 0x565127672950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.479+0000 7fd9ce9b3740 1 --2- 172.21.15.86:0/1251379960 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5651276652b0 0x565127665690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.479+0000 7fd9ce9b3740 1 -- 172.21.15.86:0/1251379960 >> 172.21.15.86:0/1251379960 conn(0x56512765fcb0 msgr2=0x565127660090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:50.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.479+0000 7fd9ce9b3740 1 -- 172.21.15.86:0/1251379960 shutdown_connections 2024-04-21T08:44:50.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.479+0000 7fd9ce9b3740 1 -- 172.21.15.86:0/1251379960 wait complete. 2024-04-21T08:44:50.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.480+0000 7fd9ce9b3740 1 -- start start 2024-04-21T08:44:50.486 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.480+0000 7fd9ce9b3740 1 Processor -- start 2024-04-21T08:44:50.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.480+0000 7fd9ce9b3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651276652b0 0x56512766fef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:50.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.480+0000 7fd9ce9b3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565127665bd0 0x565127670430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:50.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.480+0000 7fd9ce9b3740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x565127672e90 0x565127670970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:50.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.480+0000 7fd9ce9b3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x565127675bf0 con 0x565127665bd0 2024-04-21T08:44:50.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.480+0000 7fd9ce9b3740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5651275832f0 con 0x565127672e90 2024-04-21T08:44:50.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.480+0000 7fd9ce9b3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x565127675d70 con 0x5651276652b0 2024-04-21T08:44:50.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.480+0000 7fd9ce9b3740 10 client.0 _ll_register_callbacks cb 0x565127590580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:50.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.480+0000 7fd9c927f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565127665bd0 0x565127670430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:50.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.480+0000 7fd9c927f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565127665bd0 0x565127670430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32916/0 (socket says 172.21.15.86:32916) 2024-04-21T08:44:50.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.480+0000 7fd9c927f640 1 -- 172.21.15.86:0/2236257628 learned_addr learned my addr 172.21.15.86:0/2236257628 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:50.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.480+0000 7fd9ca281640 1 --2- 172.21.15.86:0/2236257628 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x565127672e90 0x565127670970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:50.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.480+0000 7fd9c927f640 1 -- 172.21.15.86:0/2236257628 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651276652b0 msgr2=0x56512766fef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:50.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.480+0000 7fd9c9a80640 1 --2- 172.21.15.86:0/2236257628 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651276652b0 0x56512766fef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:50.487 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.480+0000 7fd9c927f640 1 --2- 172.21.15.86:0/2236257628 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651276652b0 0x56512766fef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.480+0000 7fd9c927f640 1 -- 172.21.15.86:0/2236257628 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x565127672e90 msgr2=0x565127670970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:50.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.480+0000 7fd9c927f640 1 --2- 172.21.15.86:0/2236257628 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x565127672e90 0x565127670970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.480+0000 7fd9c927f640 1 -- 172.21.15.86:0/2236257628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651272dc3f0 con 0x565127665bd0 2024-04-21T08:44:50.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.480+0000 7fd9ca281640 1 --2- 172.21.15.86:0/2236257628 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x565127672e90 0x565127670970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:50.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.481+0000 7fd9c927f640 1 --2- 172.21.15.86:0/2236257628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565127665bd0 0x565127670430 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fd9c00ba010 tx=0x7fd9c00be230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5f3dfdb440f8450 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:50.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.481+0000 7fd9ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:50.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.481+0000 7fd9ad7fa640 1 -- 172.21.15.86:0/2236257628 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9c00d7ab0 con 0x565127665bd0 2024-04-21T08:44:50.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.481+0000 7fd9ad7fa640 1 -- 172.21.15.86:0/2236257628 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fd9c00d7e70 con 0x565127665bd0 2024-04-21T08:44:50.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.481+0000 7fd9ce9b3740 1 -- 172.21.15.86:0/2236257628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9bc004cc0 con 0x565127665bd0 2024-04-21T08:44:50.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.481+0000 7fd9ad7fa640 1 -- 172.21.15.86:0/2236257628 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd9c00bea30 con 0x565127665bd0 2024-04-21T08:44:50.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.481+0000 7fd9ce9b3740 10 client.18264 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:50.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.481+0000 7fd9ce9b3740 1 -- 172.21.15.86:0/2236257628 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56512729c0f0 con 0x565127665bd0 2024-04-21T08:44:50.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.481+0000 7fd9ad7fa640 1 -- 172.21.15.86:0/2236257628 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9bc004cc0 con 0x565127665bd0 2024-04-21T08:44:50.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.481+0000 7fd9ad7fa640 1 -- 172.21.15.86:0/2236257628 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd9c00efc50 con 0x565127665bd0 2024-04-21T08:44:50.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.481+0000 7fd9ce9b3740 10 client.18264 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:50.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.481+0000 7fd9ce9b3740 10 client.18264 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:50.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.481+0000 7fd9ce9b3740 20 client.18264 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:50.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.481+0000 7fd9ce9b3740 1 --2- 172.21.15.86:0/2236257628 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x565127682de0 0x565127685230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:50.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.481+0000 7fd9ce9b3740 4 client.18264 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:44:50.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.481+0000 7fd9ce9b3740 1 -- 172.21.15.86:0/2236257628 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651275d7c00 con 0x565127682de0 2024-04-21T08:44:50.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.481+0000 7fd9c9a80640 1 --2- 172.21.15.86:0/2236257628 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x565127682de0 0x565127685230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:50.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.482+0000 7fd9ad7fa640 10 client.18264 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:50.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.482+0000 7fd9c9a80640 1 --2- 172.21.15.86:0/2236257628 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x565127682de0 0x565127685230 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=35ecefe0d53e34a7 server_cookie=8f56c6e0d6dcfe9d in_seq=0 out_seq=0 2024-04-21T08:44:50.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.482+0000 7fd9ad7fa640 1 -- 172.21.15.86:0/2236257628 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5651275d7c00 con 0x565127682de0 2024-04-21T08:44:50.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.482+0000 7fd9ad7fa640 10 client.18264 handle_command_reply: tid=0 2024-04-21T08:44:50.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.482+0000 7fd9ce9b3740 1 client.18264 shutdown 2024-04-21T08:44:50.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.483+0000 7fd9ce9b3740 1 -- 172.21.15.86:0/2236257628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565127665bd0 msgr2=0x565127670430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:50.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.483+0000 7fd9ce9b3740 1 --2- 172.21.15.86:0/2236257628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565127665bd0 0x565127670430 secure :-1 s=READY pgs=1363 cs=0 l=1 rev1=1 crypto rx=0x7fd9c00ba010 tx=0x7fd9c00be230 comp rx=0 tx=0).stop 2024-04-21T08:44:50.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.483+0000 7fd9ce9b3740 1 -- 172.21.15.86:0/2236257628 shutdown_connections 2024-04-21T08:44:50.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.483+0000 7fd9ce9b3740 1 --2- 172.21.15.86:0/2236257628 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x565127682de0 0x565127685230 crc :-1 s=READY pgs=199 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.483+0000 7fd9ce9b3740 1 --2- 172.21.15.86:0/2236257628 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x565127672e90 0x565127670970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.483+0000 7fd9ad7fa640 0 client.18264 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:50.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.483+0000 7fd9ce9b3740 1 --2- 172.21.15.86:0/2236257628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565127665bd0 0x565127670430 unknown :-1 s=CLOSED pgs=1363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.483+0000 7fd9ce9b3740 1 --2- 172.21.15.86:0/2236257628 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651276652b0 0x56512766fef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.483+0000 7fd9ce9b3740 1 -- 172.21.15.86:0/2236257628 >> 172.21.15.86:0/2236257628 conn(0x56512765fcb0 msgr2=0x565127661a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:50.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.483+0000 7fd9ce9b3740 1 -- 172.21.15.86:0/2236257628 shutdown_connections 2024-04-21T08:44:50.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.483+0000 7fd9ce9b3740 1 -- 172.21.15.86:0/2236257628 wait complete. 2024-04-21T08:44:50.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.483+0000 7fd9ce9b3740 20 client.18264 trim_cache size 0 max 16384 2024-04-21T08:44:50.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.500+0000 7fd9ce9b3740 1 Processor -- start 2024-04-21T08:44:50.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.500+0000 7fd9ce9b3740 1 -- start start 2024-04-21T08:44:50.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.500+0000 7fd9ce9b3740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5651276652b0 0x565127672620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:50.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.500+0000 7fd9ce9b3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565127665bd0 0x56512766ee30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:50.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.500+0000 7fd9ce9b3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565127672e90 0x56512767eb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:50.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.500+0000 7fd9ce9b3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c41acf80 con 0x565127672e90 2024-04-21T08:44:50.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.500+0000 7fd9ce9b3740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c41aeac0 con 0x5651276652b0 2024-04-21T08:44:50.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.500+0000 7fd9ce9b3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c41ad650 con 0x565127665bd0 2024-04-21T08:44:50.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.500+0000 7fd9ca281640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565127672e90 0x56512767eb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:50.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.500+0000 7fd9ca281640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565127672e90 0x56512767eb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32918/0 (socket says 172.21.15.86:32918) 2024-04-21T08:44:50.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.500+0000 7fd9ca281640 1 -- 172.21.15.86:0/3038360745 learned_addr learned my addr 172.21.15.86:0/3038360745 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:50.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.500+0000 7fd9c9a80640 1 --2- 172.21.15.86:0/3038360745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5651276652b0 0x565127672620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:50.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.500+0000 7fd9ca281640 1 -- 172.21.15.86:0/3038360745 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565127665bd0 msgr2=0x56512766ee30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:50.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.500+0000 7fd9c927f640 1 --2- 172.21.15.86:0/3038360745 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565127665bd0 0x56512766ee30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:50.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.500+0000 7fd9ca281640 1 --2- 172.21.15.86:0/3038360745 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565127665bd0 0x56512766ee30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.500+0000 7fd9ca281640 1 -- 172.21.15.86:0/3038360745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5651276652b0 msgr2=0x565127672620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:50.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.500+0000 7fd9ca281640 1 --2- 172.21.15.86:0/3038360745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5651276652b0 0x565127672620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.500+0000 7fd9ca281640 1 -- 172.21.15.86:0/3038360745 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651276858b0 con 0x565127672e90 2024-04-21T08:44:50.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.500+0000 7fd9c9a80640 1 --2- 172.21.15.86:0/3038360745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5651276652b0 0x565127672620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:50.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.500+0000 7fd9c927f640 1 --2- 172.21.15.86:0/3038360745 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565127665bd0 0x56512766ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:50.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.501+0000 7fd9ca281640 1 --2- 172.21.15.86:0/3038360745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565127672e90 0x56512767eb40 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc000f80 tx=0x7fd9bc0094c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=69d9fba924cbeeac server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:50.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.501+0000 7fd9af7fe640 1 -- 172.21.15.86:0/3038360745 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc02d020 con 0x565127672e90 2024-04-21T08:44:50.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.501+0000 7fd9af7fe640 1 -- 172.21.15.86:0/3038360745 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9bc004db0 con 0x565127672e90 2024-04-21T08:44:50.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.501+0000 7fd9af7fe640 1 -- 172.21.15.86:0/3038360745 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc0053c0 con 0x565127672e90 2024-04-21T08:44:50.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.501+0000 7fd9ce9b3740 1 -- 172.21.15.86:0/3038360745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565127672e90 msgr2=0x56512767eb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:50.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.501+0000 7fd9ce9b3740 1 --2- 172.21.15.86:0/3038360745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565127672e90 0x56512767eb40 secure :-1 s=READY pgs=1364 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc000f80 tx=0x7fd9bc0094c0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.501+0000 7fd9ce9b3740 1 -- 172.21.15.86:0/3038360745 shutdown_connections 2024-04-21T08:44:50.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.501+0000 7fd9ce9b3740 1 --2- 172.21.15.86:0/3038360745 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565127672e90 0x56512767eb40 unknown :-1 s=CLOSED pgs=1364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.501+0000 7fd9ce9b3740 1 --2- 172.21.15.86:0/3038360745 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x565127665bd0 0x56512766ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.501+0000 7fd9ce9b3740 1 --2- 172.21.15.86:0/3038360745 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5651276652b0 0x565127672620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.501+0000 7fd9ce9b3740 1 -- 172.21.15.86:0/3038360745 >> 172.21.15.86:0/3038360745 conn(0x56512765fcb0 msgr2=0x565127661990 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:50.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.501+0000 7fd9ce9b3740 1 -- 172.21.15.86:0/3038360745 shutdown_connections 2024-04-21T08:44:50.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.501+0000 7fd9ce9b3740 1 -- 172.21.15.86:0/3038360745 wait complete. 2024-04-21T08:44:50.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.502+0000 7fd9ce9b3740 1 -- start start 2024-04-21T08:44:50.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.502+0000 7fd9ce9b3740 1 Processor -- start 2024-04-21T08:44:50.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.502+0000 7fd9ce9b3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651276652b0 0x565127677250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:50.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.502+0000 7fd9ce9b3740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x565127665bd0 0x565127677790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:50.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.502+0000 7fd9ce9b3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565127672e90 0x5651276e83b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:50.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.502+0000 7fd9ce9b3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c41ae3f0 con 0x565127672e90 2024-04-21T08:44:50.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.502+0000 7fd9ce9b3740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c41add20 con 0x565127665bd0 2024-04-21T08:44:50.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.502+0000 7fd9ce9b3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd9c41aaf40 con 0x5651276652b0 2024-04-21T08:44:50.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.502+0000 7fd9ce9b3740 10 client.0 _ll_register_callbacks cb 0x7fd9c00ba6a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:44:50.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.502+0000 7fd9ca281640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565127672e90 0x5651276e83b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:50.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.502+0000 7fd9ca281640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565127672e90 0x5651276e83b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32920/0 (socket says 172.21.15.86:32920) 2024-04-21T08:44:50.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.502+0000 7fd9ca281640 1 -- 172.21.15.86:0/3815925197 learned_addr learned my addr 172.21.15.86:0/3815925197 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:44:50.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.502+0000 7fd9c927f640 1 --2- 172.21.15.86:0/3815925197 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x565127665bd0 0x565127677790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:50.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.502+0000 7fd9ca281640 1 -- 172.21.15.86:0/3815925197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651276652b0 msgr2=0x565127677250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:50.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.502+0000 7fd9ca281640 1 --2- 172.21.15.86:0/3815925197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651276652b0 0x565127677250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.502+0000 7fd9ca281640 1 -- 172.21.15.86:0/3815925197 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x565127665bd0 msgr2=0x565127677790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:44:50.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.502+0000 7fd9ca281640 1 --2- 172.21.15.86:0/3815925197 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x565127665bd0 0x565127677790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.502+0000 7fd9ca281640 1 -- 172.21.15.86:0/3815925197 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x565127664ef0 con 0x565127672e90 2024-04-21T08:44:50.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.502+0000 7fd9c927f640 1 --2- 172.21.15.86:0/3815925197 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x565127665bd0 0x565127677790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:44:50.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.502+0000 7fd9ca281640 1 --2- 172.21.15.86:0/3815925197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565127672e90 0x5651276e83b0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc012820 tx=0x7fd9bc012850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11cf032d94778792 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:44:50.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.502+0000 7fd9ad7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:44:50.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.503+0000 7fd9ad7fa640 1 -- 172.21.15.86:0/3815925197 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd9bc02d070 con 0x565127672e90 2024-04-21T08:44:50.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.503+0000 7fd9ce9b3740 1 -- 172.21.15.86:0/3815925197 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd9bc004db0 con 0x565127672e90 2024-04-21T08:44:50.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.503+0000 7fd9ad7fa640 1 -- 172.21.15.86:0/3815925197 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7fd9bc00c8a0 con 0x565127672e90 2024-04-21T08:44:50.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.503+0000 7fd9ad7fa640 1 -- 172.21.15.86:0/3815925197 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd9bc00cae0 con 0x565127672e90 2024-04-21T08:44:50.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.503+0000 7fd9ce9b3740 10 client.18270 fetch_fsmap learned FSMap version 29 2024-04-21T08:44:50.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.503+0000 7fd9ce9b3740 1 -- 172.21.15.86:0/3815925197 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5651276e9a30 con 0x565127672e90 2024-04-21T08:44:50.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.503+0000 7fd9ad7fa640 1 -- 172.21.15.86:0/3815925197 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd9bc004db0 con 0x565127672e90 2024-04-21T08:44:50.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.503+0000 7fd9ad7fa640 1 -- 172.21.15.86:0/3815925197 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd9bc01f070 con 0x565127672e90 2024-04-21T08:44:50.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.503+0000 7fd9ce9b3740 10 client.18270 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:44:50.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.503+0000 7fd9ce9b3740 10 client.18270 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:44:50.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.503+0000 7fd9ce9b3740 20 client.18270 populate_metadata read hostname 'smithi086' 2024-04-21T08:44:50.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.503+0000 7fd9ce9b3740 1 --2- 172.21.15.86:0/3815925197 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5651276e9f50 0x5651276ec340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:44:50.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.503+0000 7fd9ce9b3740 4 client.18270 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:44:50.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.503+0000 7fd9ce9b3740 1 -- 172.21.15.86:0/3815925197 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5651275d1a70 con 0x5651276e9f50 2024-04-21T08:44:50.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.503+0000 7fd9c9a80640 1 --2- 172.21.15.86:0/3815925197 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5651276e9f50 0x5651276ec340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:44:50.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.504+0000 7fd9ad7fa640 10 client.18270 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:50.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.504+0000 7fd9c9a80640 1 --2- 172.21.15.86:0/3815925197 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5651276e9f50 0x5651276ec340 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a5044080e061bbb server_cookie=d7bf5a71c89be8bf in_seq=0 out_seq=0 2024-04-21T08:44:50.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.504+0000 7fd9ad7fa640 1 -- 172.21.15.86:0/3815925197 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5651275d1a70 con 0x5651276e9f50 2024-04-21T08:44:50.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.504+0000 7fd9ad7fa640 10 client.18270 handle_command_reply: tid=0 2024-04-21T08:44:50.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.504+0000 7fd9ce9b3740 1 client.18270 shutdown 2024-04-21T08:44:50.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.504+0000 7fd9ce9b3740 1 -- 172.21.15.86:0/3815925197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565127672e90 msgr2=0x5651276e83b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:50.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.504+0000 7fd9ce9b3740 1 --2- 172.21.15.86:0/3815925197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565127672e90 0x5651276e83b0 secure :-1 s=READY pgs=1365 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc012820 tx=0x7fd9bc012850 comp rx=0 tx=0).stop 2024-04-21T08:44:50.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.504+0000 7fd9ce9b3740 1 -- 172.21.15.86:0/3815925197 shutdown_connections 2024-04-21T08:44:50.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.504+0000 7fd9ce9b3740 1 --2- 172.21.15.86:0/3815925197 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5651276e9f50 0x5651276ec340 crc :-1 s=READY pgs=200 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.504+0000 7fd9ce9b3740 1 --2- 172.21.15.86:0/3815925197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x565127672e90 0x5651276e83b0 unknown :-1 s=CLOSED pgs=1365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.504+0000 7fd9ce9b3740 1 --2- 172.21.15.86:0/3815925197 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x565127665bd0 0x565127677790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.504+0000 7fd9ce9b3740 1 --2- 172.21.15.86:0/3815925197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651276652b0 0x565127677250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.504+0000 7fd9ad7fa640 0 client.18270 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:44:50.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.504+0000 7fd9ce9b3740 1 -- 172.21.15.86:0/3815925197 >> 172.21.15.86:0/3815925197 conn(0x56512765fcb0 msgr2=0x56512766feb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:50.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.504+0000 7fd9ce9b3740 1 -- 172.21.15.86:0/3815925197 shutdown_connections 2024-04-21T08:44:50.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.504+0000 7fd9ce9b3740 1 -- 172.21.15.86:0/3815925197 wait complete. 2024-04-21T08:44:50.512 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.504+0000 7fd9ce9b3740 20 client.18270 trim_cache size 0 max 16384 2024-04-21T08:44:50.512 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:44:50.512 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:44:50.512 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-21T08:44:50.512 INFO:teuthology.orchestra.run.smithi086.stdout: "2b835aff-748b-4ace-b316-68f328cad4a0": { 2024-04-21T08:44:50.512 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-21T08:44:50.512 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "2b835aff-748b-4ace-b316-68f328cad4a0", 2024-04-21T08:44:50.512 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-21T08:44:50.512 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:44:50.513 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:44:50.513 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:44:50.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.505+0000 7fd9cbce4640 1 -- 172.21.15.86:0/3441232350 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd9b00a3d20 msgr2=0x7fd9b00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:50.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.505+0000 7fd9cbce4640 1 --2- 172.21.15.86:0/3441232350 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd9b00a3d20 0x7fd9b00a61e0 secure :-1 s=READY pgs=216 cs=0 l=1 rev1=1 crypto rx=0x7fd9bc005c40 tx=0x7fd9bc009db0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.513 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.505+0000 7fd9cbce4640 1 -- 172.21.15.86:0/3441232350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9c4100fd0 msgr2=0x7fd9c4198b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:44:50.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.505+0000 7fd9cbce4640 1 --2- 172.21.15.86:0/3441232350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9c4100fd0 0x7fd9c4198b60 secure :-1 s=READY pgs=1361 cs=0 l=1 rev1=1 crypto rx=0x7fd9c0006010 tx=0x7fd9c0004040 comp rx=0 tx=0).stop 2024-04-21T08:44:50.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.505+0000 7fd9cbce4640 1 -- 172.21.15.86:0/3441232350 shutdown_connections 2024-04-21T08:44:50.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.505+0000 7fd9cbce4640 1 --2- 172.21.15.86:0/3441232350 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd9b00a3d20 0x7fd9b00a61e0 unknown :-1 s=CLOSED pgs=216 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.505+0000 7fd9cbce4640 1 --2- 172.21.15.86:0/3441232350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd9c4106680 0x7fd9c419cee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.505+0000 7fd9cbce4640 1 --2- 172.21.15.86:0/3441232350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd9c4100fd0 0x7fd9c4198b60 unknown :-1 s=CLOSED pgs=1361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.505+0000 7fd9cbce4640 1 --2- 172.21.15.86:0/3441232350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd9c4100610 0x7fd9c4198620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:44:50.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.505+0000 7fd9cbce4640 1 -- 172.21.15.86:0/3441232350 >> 172.21.15.86:0/3441232350 conn(0x7fd9c4077360 msgr2=0x7fd9c410dfe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:44:50.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.505+0000 7fd9cbce4640 1 -- 172.21.15.86:0/3441232350 shutdown_connections 2024-04-21T08:44:50.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:44:50.505+0000 7fd9cbce4640 1 -- 172.21.15.86:0/3441232350 wait complete. 2024-04-21T08:44:50.515 INFO:tasks.cephfs.filesystem:scrub status for tag:2b835aff-748b-4ace-b316-68f328cad4a0 - {'path': '/', 'tag': '2b835aff-748b-4ace-b316-68f328cad4a0', 'options': 'recursive,force'} 2024-04-21T08:44:51.118 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:50 smithi086 ceph-mon[27939]: scrub queued for path: / 2024-04-21T08:44:51.118 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:50 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:44:51.118 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:50 smithi086 ceph-mon[27939]: scrub summary: active paths [/] 2024-04-21T08:44:51.118 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:50 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:44:51.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:50 smithi088 ceph-mon[31259]: scrub queued for path: / 2024-04-21T08:44:51.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:50 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:44:51.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:50 smithi088 ceph-mon[31259]: scrub summary: active paths [/] 2024-04-21T08:44:51.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:50 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:44:51.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:50 smithi179 ceph-mon[31829]: scrub queued for path: / 2024-04-21T08:44:51.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:50 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:44:51.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:50 smithi179 ceph-mon[31829]: scrub summary: active paths [/] 2024-04-21T08:44:51.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:50 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:44:52.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:51 smithi086 ceph-mon[27939]: pgmap v185: 129 pgs: 129 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-21T08:44:52.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:51 smithi088 ceph-mon[31259]: pgmap v185: 129 pgs: 129 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-21T08:44:52.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:51 smithi179 ceph-mon[31829]: pgmap v185: 129 pgs: 129 active+clean; 5.0 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 32 KiB/s wr, 12 op/s 2024-04-21T08:44:54.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:53 smithi086 ceph-mon[27939]: pgmap v186: 129 pgs: 129 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 23 op/s 2024-04-21T08:44:54.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:53 smithi088 ceph-mon[31259]: pgmap v186: 129 pgs: 129 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 23 op/s 2024-04-21T08:44:54.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:53 smithi179 ceph-mon[31829]: pgmap v186: 129 pgs: 129 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 23 op/s 2024-04-21T08:44:55.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:54 smithi086 ceph-mon[27939]: scrub summary: idle 2024-04-21T08:44:55.213 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:54 smithi088 ceph-mon[31259]: scrub summary: idle 2024-04-21T08:44:55.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:54 smithi179 ceph-mon[31829]: scrub summary: idle 2024-04-21T08:44:56.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:55 smithi086 ceph-mon[27939]: pgmap v187: 129 pgs: 129 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 16 op/s 2024-04-21T08:44:56.213 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:55 smithi088 ceph-mon[31259]: pgmap v187: 129 pgs: 129 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 16 op/s 2024-04-21T08:44:56.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:55 smithi179 ceph-mon[31829]: pgmap v187: 129 pgs: 129 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 16 op/s 2024-04-21T08:44:57.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:56 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:44:57.213 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:56 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:44:57.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:56 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:44:58.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:57 smithi086 ceph-mon[27939]: pgmap v188: 129 pgs: 129 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 16 op/s 2024-04-21T08:44:58.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:57 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:58.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:57 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:44:58.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:57 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:44:58.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:57 smithi088 ceph-mon[31259]: pgmap v188: 129 pgs: 129 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 16 op/s 2024-04-21T08:44:58.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:57 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:58.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:57 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:44:58.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:57 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:44:58.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:57 smithi179 ceph-mon[31829]: pgmap v188: 129 pgs: 129 active+clean; 5.2 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 40 KiB/s wr, 16 op/s 2024-04-21T08:44:58.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:57 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:44:58.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:57 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:44:58.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:57 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:45:00.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:44:59 smithi086 ceph-mon[27939]: pgmap v189: 129 pgs: 129 active+clean; 5.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 24 op/s 2024-04-21T08:45:00.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:44:59 smithi088 ceph-mon[31259]: pgmap v189: 129 pgs: 129 active+clean; 5.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 24 op/s 2024-04-21T08:45:00.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:44:59 smithi179 ceph-mon[31829]: pgmap v189: 129 pgs: 129 active+clean; 5.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 55 KiB/s wr, 24 op/s 2024-04-21T08:45:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.846+0000 7f9305081640 1 -- 172.21.15.86:0/2932052341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9300101b60 msgr2=0x7f930010dc70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.846+0000 7f9305081640 1 --2- 172.21.15.86:0/2932052341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9300101b60 0x7f930010dc70 secure :-1 s=READY pgs=1366 cs=0 l=1 rev1=1 crypto rx=0x7f92f8009ca0 tx=0x7f92f802c140 comp rx=0 tx=0).stop 2024-04-21T08:45:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.846+0000 7f9305081640 1 -- 172.21.15.86:0/2932052341 shutdown_connections 2024-04-21T08:45:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.846+0000 7f9305081640 1 --2- 172.21.15.86:0/2932052341 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9300101b60 0x7f930010dc70 unknown :-1 s=CLOSED pgs=1366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.846+0000 7f9305081640 1 --2- 172.21.15.86:0/2932052341 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93001011a0 0x7f9300101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.846+0000 7f9305081640 1 --2- 172.21.15.86:0/2932052341 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9300106fb0 0x7f9300107390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.846+0000 7f9305081640 1 -- 172.21.15.86:0/2932052341 >> 172.21.15.86:0/2932052341 conn(0x7f9300076c50 msgr2=0x7f9300077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.846+0000 7f9305081640 1 -- 172.21.15.86:0/2932052341 shutdown_connections 2024-04-21T08:45:01.846 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.846+0000 7f9305081640 1 -- 172.21.15.86:0/2932052341 wait complete. 2024-04-21T08:45:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.847+0000 7f9305081640 1 Processor -- start 2024-04-21T08:45:01.848 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.847+0000 7f9305081640 1 -- start start 2024-04-21T08:45:01.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.847+0000 7f9305081640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93001011a0 0x7f9300198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.847+0000 7f9305081640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9300101b60 0x7f9300198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.847+0000 7f9305081640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9300106fb0 0x7f930019ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.847+0000 7f9305081640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93001121e0 con 0x7f9300101b60 2024-04-21T08:45:01.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.847+0000 7f9305081640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9300112060 con 0x7f9300106fb0 2024-04-21T08:45:01.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.847+0000 7f9305081640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9300112360 con 0x7f93001011a0 2024-04-21T08:45:01.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.847+0000 7f92fe59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9300101b60 0x7f9300198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.847+0000 7f92fe59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9300101b60 0x7f9300198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35256/0 (socket says 172.21.15.86:35256) 2024-04-21T08:45:01.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.847+0000 7f92fe59c640 1 -- 172.21.15.86:0/2653284604 learned_addr learned my addr 172.21.15.86:0/2653284604 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:01.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.848+0000 7f92ff59e640 1 --2- 172.21.15.86:0/2653284604 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9300106fb0 0x7f930019ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.848+0000 7f92fed9d640 1 --2- 172.21.15.86:0/2653284604 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93001011a0 0x7f9300198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.848+0000 7f92fe59c640 1 -- 172.21.15.86:0/2653284604 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93001011a0 msgr2=0x7f9300198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.849 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.848+0000 7f92fe59c640 1 --2- 172.21.15.86:0/2653284604 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93001011a0 0x7f9300198560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.848+0000 7f92fe59c640 1 -- 172.21.15.86:0/2653284604 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9300106fb0 msgr2=0x7f930019ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.848+0000 7f92fe59c640 1 --2- 172.21.15.86:0/2653284604 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9300106fb0 0x7f930019ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.848+0000 7f92fe59c640 1 -- 172.21.15.86:0/2653284604 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f930019d550 con 0x7f9300101b60 2024-04-21T08:45:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.848+0000 7f92ff59e640 1 --2- 172.21.15.86:0/2653284604 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9300106fb0 0x7f930019ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.848+0000 7f92fe59c640 1 --2- 172.21.15.86:0/2653284604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9300101b60 0x7f9300198aa0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f92f400d950 tx=0x7f92f400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.848+0000 7f92e7fff640 1 -- 172.21.15.86:0/2653284604 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f4017030 con 0x7f9300101b60 2024-04-21T08:45:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.848+0000 7f9305081640 1 -- 172.21.15.86:0/2653284604 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f930019d840 con 0x7f9300101b60 2024-04-21T08:45:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.848+0000 7f92e7fff640 1 -- 172.21.15.86:0/2653284604 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92f4004020 con 0x7f9300101b60 2024-04-21T08:45:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.848+0000 7f92e7fff640 1 -- 172.21.15.86:0/2653284604 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f40047a0 con 0x7f9300101b60 2024-04-21T08:45:01.850 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.848+0000 7f9305081640 1 -- 172.21.15.86:0/2653284604 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f93001a50d0 con 0x7f9300101b60 2024-04-21T08:45:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.851+0000 7f92e7fff640 1 -- 172.21.15.86:0/2653284604 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f92f4009a30 con 0x7f9300101b60 2024-04-21T08:45:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.851+0000 7f9307c79740 1 Processor -- start 2024-04-21T08:45:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.851+0000 7f9307c79740 1 -- start start 2024-04-21T08:45:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.851+0000 7f92e7fff640 1 --2- 172.21.15.86:0/2653284604 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f92e80a3dc0 0x7f92e80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.851+0000 7f9307c79740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593646568d0 0x559364655620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.851+0000 7f9307c79740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559364655b60 0x559364655fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.852 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.851+0000 7f92ff59e640 1 --2- 172.21.15.86:0/2653284604 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f92e80a3dc0 0x7f92e80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.851+0000 7f9307c79740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55936465f0e0 0x5593646614d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.851+0000 7f9307c79740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5593645ba8d0 con 0x55936465f0e0 2024-04-21T08:45:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.851+0000 7f9307c79740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559364557e80 con 0x559364655b60 2024-04-21T08:45:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.851+0000 7f9307c79740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5593645b6890 con 0x5593646568d0 2024-04-21T08:45:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.851+0000 7f92e7fff640 1 -- 172.21.15.86:0/2653284604 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f92f40d0720 con 0x7f9300101b60 2024-04-21T08:45:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.852+0000 7f92fed9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593646568d0 0x559364655620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.852+0000 7f92ff59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55936465f0e0 0x5593646614d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.852+0000 7f92fed9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593646568d0 0x559364655620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.86:40366/0 (socket says 172.21.15.86:40366) 2024-04-21T08:45:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.852+0000 7f92fed9d640 1 -- 172.21.15.86:0/304447631 learned_addr learned my addr 172.21.15.86:0/304447631 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.852+0000 7f92fe59c640 1 --2- 172.21.15.86:0/304447631 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559364655b60 0x559364655fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.852+0000 7f92fe59c640 1 -- 172.21.15.86:0/304447631 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593646568d0 msgr2=0x559364655620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.852+0000 7f92fe59c640 1 --2- 172.21.15.86:0/304447631 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593646568d0 0x559364655620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.852+0000 7f92fe59c640 1 -- 172.21.15.86:0/304447631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55936465f0e0 msgr2=0x5593646614d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:01.853 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.852+0000 7f92fe59c640 1 --2- 172.21.15.86:0/304447631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55936465f0e0 0x5593646614d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.852+0000 7f92fe59c640 1 -- 172.21.15.86:0/304447631 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559364579320 con 0x559364655b60 2024-04-21T08:45:01.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.852+0000 7f92ff59e640 1 --2- 172.21.15.86:0/2653284604 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f92e80a3dc0 0x7f92e80a6280 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f92f8005e30 tx=0x7f92f8032ec0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:01.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.852+0000 7f92fed9d640 1 --2- 172.21.15.86:0/304447631 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593646568d0 0x559364655620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:45:01.854 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.852+0000 7f92fe59c640 1 --2- 172.21.15.86:0/304447631 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559364655b60 0x559364655fe0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f92f40cdb00 tx=0x7f92f409cb90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=da309556ee7a2072 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:01.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.855+0000 7f92e57fa640 1 -- 172.21.15.86:0/304447631 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f40b5070 con 0x559364655b60 2024-04-21T08:45:01.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.855+0000 7f92e57fa640 1 -- 172.21.15.86:0/304447631 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92f40a4fc0 con 0x559364655b60 2024-04-21T08:45:01.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.855+0000 7f92e57fa640 1 -- 172.21.15.86:0/304447631 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f40989c0 con 0x559364655b60 2024-04-21T08:45:01.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.855+0000 7f9307c79740 1 -- 172.21.15.86:0/304447631 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559364655b60 msgr2=0x559364655fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.855+0000 7f9307c79740 1 --2- 172.21.15.86:0/304447631 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559364655b60 0x559364655fe0 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f92f40cdb00 tx=0x7f92f409cb90 comp rx=0 tx=0).stop 2024-04-21T08:45:01.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.856+0000 7f9307c79740 1 -- 172.21.15.86:0/304447631 shutdown_connections 2024-04-21T08:45:01.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.856+0000 7f9307c79740 1 --2- 172.21.15.86:0/304447631 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55936465f0e0 0x5593646614d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.856+0000 7f9307c79740 1 --2- 172.21.15.86:0/304447631 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559364655b60 0x559364655fe0 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.856+0000 7f9307c79740 1 --2- 172.21.15.86:0/304447631 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593646568d0 0x559364655620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.856+0000 7f9307c79740 1 -- 172.21.15.86:0/304447631 >> 172.21.15.86:0/304447631 conn(0x55936461cc90 msgr2=0x559364650850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:01.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.856+0000 7f9307c79740 1 -- 172.21.15.86:0/304447631 shutdown_connections 2024-04-21T08:45:01.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.856+0000 7f9307c79740 1 -- 172.21.15.86:0/304447631 wait complete. 2024-04-21T08:45:01.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.856+0000 7f9307c79740 1 -- start start 2024-04-21T08:45:01.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.856+0000 7f9307c79740 1 Processor -- start 2024-04-21T08:45:01.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.856+0000 7f9307c79740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559364655b60 0x55936465eb20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.856+0000 7f9307c79740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593646568d0 0x55936465beb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.856+0000 7f9307c79740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55936465f0e0 0x55936465c5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.856+0000 7f9307c79740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559364661e10 con 0x55936465f0e0 2024-04-21T08:45:01.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.856+0000 7f9307c79740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559364542eb0 con 0x559364655b60 2024-04-21T08:45:01.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.856+0000 7f9307c79740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559364661f90 con 0x5593646568d0 2024-04-21T08:45:01.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.856+0000 7f9307c79740 10 client.0 _ll_register_callbacks cb 0x55936456fb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:01.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.856+0000 7f92ff59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55936465f0e0 0x55936465c5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.856+0000 7f92ff59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55936465f0e0 0x55936465c5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35280/0 (socket says 172.21.15.86:35280) 2024-04-21T08:45:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.856+0000 7f92ff59e640 1 -- 172.21.15.86:0/2960468146 learned_addr learned my addr 172.21.15.86:0/2960468146 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.856+0000 7f92fed9d640 1 --2- 172.21.15.86:0/2960468146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559364655b60 0x55936465eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.856+0000 7f92fe59c640 1 --2- 172.21.15.86:0/2960468146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593646568d0 0x55936465beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.857+0000 7f92ff59e640 1 -- 172.21.15.86:0/2960468146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593646568d0 msgr2=0x55936465beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.857+0000 7f92ff59e640 1 --2- 172.21.15.86:0/2960468146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593646568d0 0x55936465beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.857+0000 7f92ff59e640 1 -- 172.21.15.86:0/2960468146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559364655b60 msgr2=0x55936465eb20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.857+0000 7f92ff59e640 1 --2- 172.21.15.86:0/2960468146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559364655b60 0x55936465eb20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.857+0000 7f92ff59e640 1 -- 172.21.15.86:0/2960468146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593642cd210 con 0x55936465f0e0 2024-04-21T08:45:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.857+0000 7f92ff59e640 1 --2- 172.21.15.86:0/2960468146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55936465f0e0 0x55936465c5f0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f92f8009ca0 tx=0x7f92f802c140 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f04a69db09b172eb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.857+0000 7f92e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.857+0000 7f92e2ff5640 1 -- 172.21.15.86:0/2960468146 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f8050070 con 0x55936465f0e0 2024-04-21T08:45:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.857+0000 7f92e2ff5640 1 -- 172.21.15.86:0/2960468146 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f92f8004240 con 0x55936465f0e0 2024-04-21T08:45:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.857+0000 7f9307c79740 1 -- 172.21.15.86:0/2960468146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92f40a4fc0 con 0x55936465f0e0 2024-04-21T08:45:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.857+0000 7f92e2ff5640 1 -- 172.21.15.86:0/2960468146 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f92f805c050 con 0x55936465f0e0 2024-04-21T08:45:01.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.857+0000 7f9307c79740 10 client.18282 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.857+0000 7f9307c79740 1 -- 172.21.15.86:0/2960468146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559364399620 con 0x55936465f0e0 2024-04-21T08:45:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.857+0000 7f92e2ff5640 1 -- 172.21.15.86:0/2960468146 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92f40a4fc0 con 0x55936465f0e0 2024-04-21T08:45:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.858+0000 7f92e2ff5640 1 -- 172.21.15.86:0/2960468146 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f92f8034a10 con 0x55936465f0e0 2024-04-21T08:45:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.858+0000 7f9307c79740 10 client.18282 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.858+0000 7f9307c79740 10 client.18282 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.858+0000 7f9307c79740 20 client.18282 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.858+0000 7f9307c79740 1 --2- 172.21.15.86:0/2960468146 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55936466f170 0x559364671560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.858+0000 7f9307c79740 4 client.18282 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.858+0000 7f9307c79740 1 -- 172.21.15.86:0/2960468146 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5593645ce880 con 0x55936466f170 2024-04-21T08:45:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.858+0000 7f92fed9d640 1 --2- 172.21.15.86:0/2960468146 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55936466f170 0x559364671560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.858+0000 7f92e2ff5640 10 client.18282 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:01.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.858+0000 7f92fed9d640 1 --2- 172.21.15.86:0/2960468146 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55936466f170 0x559364671560 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f4fdd4cdb548bafc server_cookie=a011bd9ed92b9cb2 in_seq=0 out_seq=0 2024-04-21T08:45:01.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.859+0000 7f92e2ff5640 1 -- 172.21.15.86:0/2960468146 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5593645ce880 con 0x55936466f170 2024-04-21T08:45:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.859+0000 7f92e2ff5640 10 client.18282 handle_command_reply: tid=0 2024-04-21T08:45:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.859+0000 7f9307c79740 1 client.18282 shutdown 2024-04-21T08:45:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.859+0000 7f9307c79740 1 -- 172.21.15.86:0/2960468146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55936465f0e0 msgr2=0x55936465c5f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.859+0000 7f9307c79740 1 --2- 172.21.15.86:0/2960468146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55936465f0e0 0x55936465c5f0 secure :-1 s=READY pgs=1368 cs=0 l=1 rev1=1 crypto rx=0x7f92f8009ca0 tx=0x7f92f802c140 comp rx=0 tx=0).stop 2024-04-21T08:45:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.859+0000 7f9307c79740 1 -- 172.21.15.86:0/2960468146 shutdown_connections 2024-04-21T08:45:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.859+0000 7f9307c79740 1 --2- 172.21.15.86:0/2960468146 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55936466f170 0x559364671560 crc :-1 s=READY pgs=201 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.859+0000 7f9307c79740 1 --2- 172.21.15.86:0/2960468146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55936465f0e0 0x55936465c5f0 unknown :-1 s=CLOSED pgs=1368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.859+0000 7f9307c79740 1 --2- 172.21.15.86:0/2960468146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593646568d0 0x55936465beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.859+0000 7f92e2ff5640 0 client.18282 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.859+0000 7f9307c79740 1 --2- 172.21.15.86:0/2960468146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559364655b60 0x55936465eb20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.859+0000 7f9307c79740 1 -- 172.21.15.86:0/2960468146 >> 172.21.15.86:0/2960468146 conn(0x55936461cc90 msgr2=0x5593646525a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.859+0000 7f9307c79740 1 -- 172.21.15.86:0/2960468146 shutdown_connections 2024-04-21T08:45:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.859+0000 7f9307c79740 1 -- 172.21.15.86:0/2960468146 wait complete. 2024-04-21T08:45:01.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.860+0000 7f9307c79740 20 client.18282 trim_cache size 0 max 16384 2024-04-21T08:45:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.874+0000 7f9307c79740 1 Processor -- start 2024-04-21T08:45:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.874+0000 7f9307c79740 1 -- start start 2024-04-21T08:45:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.874+0000 7f9307c79740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55936461cc90 0x55936466ad10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.874+0000 7f9307c79740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559364655b60 0x559364667600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.874+0000 7f9307c79740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593646568d0 0x559364676e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.878 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.874+0000 7f9307c79740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93001af1e0 con 0x5593646568d0 2024-04-21T08:45:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.874+0000 7f9307c79740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f93001b0d20 con 0x55936461cc90 2024-04-21T08:45:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.874+0000 7f9307c79740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f93001af8b0 con 0x559364655b60 2024-04-21T08:45:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.875+0000 7f92ff59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593646568d0 0x559364676e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.875+0000 7f92ff59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593646568d0 0x559364676e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35282/0 (socket says 172.21.15.86:35282) 2024-04-21T08:45:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.875+0000 7f92ff59e640 1 -- 172.21.15.86:0/3716664641 learned_addr learned my addr 172.21.15.86:0/3716664641 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.875+0000 7f92fed9d640 1 --2- 172.21.15.86:0/3716664641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55936461cc90 0x55936466ad10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.875+0000 7f92ff59e640 1 -- 172.21.15.86:0/3716664641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559364655b60 msgr2=0x559364667600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.875+0000 7f92fe59c640 1 --2- 172.21.15.86:0/3716664641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559364655b60 0x559364667600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.875+0000 7f92ff59e640 1 --2- 172.21.15.86:0/3716664641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559364655b60 0x559364667600 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.875+0000 7f92ff59e640 1 -- 172.21.15.86:0/3716664641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55936461cc90 msgr2=0x55936466ad10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.875+0000 7f92ff59e640 1 --2- 172.21.15.86:0/3716664641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55936461cc90 0x55936466ad10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.875+0000 7f92ff59e640 1 -- 172.21.15.86:0/3716664641 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559364671b90 con 0x5593646568d0 2024-04-21T08:45:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.875+0000 7f92fed9d640 1 --2- 172.21.15.86:0/3716664641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55936461cc90 0x55936466ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:01.879 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.875+0000 7f92ff59e640 1 --2- 172.21.15.86:0/3716664641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593646568d0 0x559364676e20 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f92f8036000 tx=0x7f92f8005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8b806842087a43b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.875+0000 7f92e4ff9640 1 -- 172.21.15.86:0/3716664641 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f8065070 con 0x5593646568d0 2024-04-21T08:45:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.875+0000 7f92e4ff9640 1 -- 172.21.15.86:0/3716664641 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92f804c3e0 con 0x5593646568d0 2024-04-21T08:45:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.875+0000 7f92e4ff9640 1 -- 172.21.15.86:0/3716664641 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f8050040 con 0x5593646568d0 2024-04-21T08:45:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.876+0000 7f9307c79740 1 -- 172.21.15.86:0/3716664641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593646568d0 msgr2=0x559364676e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.876+0000 7f9307c79740 1 --2- 172.21.15.86:0/3716664641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593646568d0 0x559364676e20 secure :-1 s=READY pgs=1369 cs=0 l=1 rev1=1 crypto rx=0x7f92f8036000 tx=0x7f92f8005d80 comp rx=0 tx=0).stop 2024-04-21T08:45:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.876+0000 7f9307c79740 1 -- 172.21.15.86:0/3716664641 shutdown_connections 2024-04-21T08:45:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.876+0000 7f9307c79740 1 --2- 172.21.15.86:0/3716664641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5593646568d0 0x559364676e20 unknown :-1 s=CLOSED pgs=1369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.876+0000 7f9307c79740 1 --2- 172.21.15.86:0/3716664641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559364655b60 0x559364667600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.876+0000 7f9307c79740 1 --2- 172.21.15.86:0/3716664641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55936461cc90 0x55936466ad10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.876+0000 7f9307c79740 1 -- 172.21.15.86:0/3716664641 >> 172.21.15.86:0/3716664641 conn(0x5593646616b0 msgr2=0x559364657060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.876+0000 7f9307c79740 1 -- 172.21.15.86:0/3716664641 shutdown_connections 2024-04-21T08:45:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.876+0000 7f9307c79740 1 -- 172.21.15.86:0/3716664641 wait complete. 2024-04-21T08:45:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.876+0000 7f9307c79740 1 -- start start 2024-04-21T08:45:01.880 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.876+0000 7f9307c79740 1 Processor -- start 2024-04-21T08:45:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.876+0000 7f9307c79740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55936461cc90 0x559364660940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.876+0000 7f9307c79740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559364655b60 0x559364659150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.876+0000 7f9307c79740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593646568d0 0x559364659690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.876+0000 7f9307c79740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f93001b0650 con 0x55936461cc90 2024-04-21T08:45:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.876+0000 7f9307c79740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f93001aff80 con 0x559364655b60 2024-04-21T08:45:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.876+0000 7f9307c79740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f93001ad1a0 con 0x5593646568d0 2024-04-21T08:45:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.876+0000 7f9307c79740 10 client.0 _ll_register_callbacks cb 0x7f92f8037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.876+0000 7f92fed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55936461cc90 0x559364660940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.876+0000 7f92fed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55936461cc90 0x559364660940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35292/0 (socket says 172.21.15.86:35292) 2024-04-21T08:45:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.876+0000 7f92fed9d640 1 -- 172.21.15.86:0/567913330 learned_addr learned my addr 172.21.15.86:0/567913330 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.877+0000 7f92fe59c640 1 --2- 172.21.15.86:0/567913330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559364655b60 0x559364659150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.877+0000 7f92fed9d640 1 -- 172.21.15.86:0/567913330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593646568d0 msgr2=0x559364659690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.877+0000 7f92fed9d640 1 --2- 172.21.15.86:0/567913330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593646568d0 0x559364659690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.881 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.877+0000 7f92fed9d640 1 -- 172.21.15.86:0/567913330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559364655b60 msgr2=0x559364659150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.877+0000 7f92fed9d640 1 --2- 172.21.15.86:0/567913330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559364655b60 0x559364659150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.877+0000 7f92fed9d640 1 -- 172.21.15.86:0/567913330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559364654e30 con 0x55936461cc90 2024-04-21T08:45:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.877+0000 7f92fe59c640 1 --2- 172.21.15.86:0/567913330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559364655b60 0x559364659150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.877+0000 7f92fed9d640 1 --2- 172.21.15.86:0/567913330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55936461cc90 0x559364660940 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f92f0000c00 tx=0x7f92f0012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2224056ecd31c3c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.877+0000 7f92e2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.877+0000 7f92e2ff5640 1 -- 172.21.15.86:0/567913330 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f92f001e580 con 0x55936461cc90 2024-04-21T08:45:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.877+0000 7f92e2ff5640 1 -- 172.21.15.86:0/567913330 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(94..94 src has 1..94) ==== 9408+0+0 (secure 0 0 0) 0x7f92f001e720 con 0x55936461cc90 2024-04-21T08:45:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.877+0000 7f9307c79740 1 -- 172.21.15.86:0/567913330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f92f804c3e0 con 0x55936461cc90 2024-04-21T08:45:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.877+0000 7f92e2ff5640 1 -- 172.21.15.86:0/567913330 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f92f002e050 con 0x55936461cc90 2024-04-21T08:45:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.877+0000 7f9307c79740 10 client.18288 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.877+0000 7f9307c79740 1 -- 172.21.15.86:0/567913330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559364660050 con 0x55936461cc90 2024-04-21T08:45:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.877+0000 7f92e2ff5640 1 -- 172.21.15.86:0/567913330 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f92f804c3e0 con 0x55936461cc90 2024-04-21T08:45:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.878+0000 7f92e2ff5640 1 -- 172.21.15.86:0/567913330 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f92f000f040 con 0x55936461cc90 2024-04-21T08:45:01.882 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.878+0000 7f9307c79740 10 client.18288 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.878+0000 7f9307c79740 10 client.18288 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.878+0000 7f9307c79740 20 client.18288 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.878+0000 7f9307c79740 1 --2- 172.21.15.86:0/567913330 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55936465f0e0 0x5593646dcab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.878+0000 7f9307c79740 4 client.18288 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:45:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.878+0000 7f9307c79740 1 -- 172.21.15.86:0/567913330 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55936455a3d0 con 0x55936465f0e0 2024-04-21T08:45:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.878+0000 7f92fe59c640 1 --2- 172.21.15.86:0/567913330 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55936465f0e0 0x5593646dcab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.878+0000 7f92fe59c640 1 --2- 172.21.15.86:0/567913330 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55936465f0e0 0x5593646dcab0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=151357c27d803cfe server_cookie=2f38f09fc99bb92 in_seq=0 out_seq=0 2024-04-21T08:45:01.883 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.878+0000 7f92e2ff5640 10 client.18288 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.894+0000 7f92e2ff5640 1 -- 172.21.15.86:0/567913330 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55936455a3d0 con 0x55936465f0e0 2024-04-21T08:45:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.894+0000 7f92e2ff5640 10 client.18288 handle_command_reply: tid=0 2024-04-21T08:45:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.894+0000 7f9307c79740 1 client.18288 shutdown 2024-04-21T08:45:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.894+0000 7f9307c79740 1 -- 172.21.15.86:0/567913330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55936461cc90 msgr2=0x559364660940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.894+0000 7f9307c79740 1 --2- 172.21.15.86:0/567913330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55936461cc90 0x559364660940 secure :-1 s=READY pgs=1370 cs=0 l=1 rev1=1 crypto rx=0x7f92f0000c00 tx=0x7f92f0012820 comp rx=0 tx=0).stop 2024-04-21T08:45:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.894+0000 7f9307c79740 1 -- 172.21.15.86:0/567913330 shutdown_connections 2024-04-21T08:45:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.894+0000 7f9307c79740 1 --2- 172.21.15.86:0/567913330 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55936465f0e0 0x5593646dcab0 crc :-1 s=READY pgs=202 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.894 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.894+0000 7f9307c79740 1 --2- 172.21.15.86:0/567913330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5593646568d0 0x559364659690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.894+0000 7f9307c79740 1 --2- 172.21.15.86:0/567913330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559364655b60 0x559364659150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.894+0000 7f92e2ff5640 0 client.18288 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.894+0000 7f9307c79740 1 --2- 172.21.15.86:0/567913330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55936461cc90 0x559364660940 unknown :-1 s=CLOSED pgs=1370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.894+0000 7f9307c79740 1 -- 172.21.15.86:0/567913330 >> 172.21.15.86:0/567913330 conn(0x5593646616b0 msgr2=0x559364651b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.894+0000 7f9307c79740 1 -- 172.21.15.86:0/567913330 shutdown_connections 2024-04-21T08:45:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.894+0000 7f9307c79740 1 -- 172.21.15.86:0/567913330 wait complete. 2024-04-21T08:45:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.894+0000 7f9307c79740 20 client.18288 trim_cache size 0 max 16384 2024-04-21T08:45:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:45:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:45:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:45:01.895 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:45:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.895+0000 7f9305081640 1 -- 172.21.15.86:0/2653284604 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f92e80a3dc0 msgr2=0x7f92e80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.895+0000 7f9305081640 1 --2- 172.21.15.86:0/2653284604 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f92e80a3dc0 0x7f92e80a6280 secure :-1 s=READY pgs=217 cs=0 l=1 rev1=1 crypto rx=0x7f92f8005e30 tx=0x7f92f8032ec0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.895+0000 7f9305081640 1 -- 172.21.15.86:0/2653284604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9300101b60 msgr2=0x7f9300198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.895+0000 7f9305081640 1 --2- 172.21.15.86:0/2653284604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9300101b60 0x7f9300198aa0 secure :-1 s=READY pgs=1367 cs=0 l=1 rev1=1 crypto rx=0x7f92f400d950 tx=0x7f92f400de10 comp rx=0 tx=0).stop 2024-04-21T08:45:01.896 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.895+0000 7f9305081640 1 -- 172.21.15.86:0/2653284604 shutdown_connections 2024-04-21T08:45:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.895+0000 7f9305081640 1 --2- 172.21.15.86:0/2653284604 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f92e80a3dc0 0x7f92e80a6280 unknown :-1 s=CLOSED pgs=217 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.895+0000 7f9305081640 1 --2- 172.21.15.86:0/2653284604 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9300106fb0 0x7f930019ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.895+0000 7f9305081640 1 --2- 172.21.15.86:0/2653284604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9300101b60 0x7f9300198aa0 unknown :-1 s=CLOSED pgs=1367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.895+0000 7f9305081640 1 --2- 172.21.15.86:0/2653284604 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f93001011a0 0x7f9300198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.895+0000 7f9305081640 1 -- 172.21.15.86:0/2653284604 >> 172.21.15.86:0/2653284604 conn(0x7f9300076c50 msgr2=0x7f93000ffc50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.895+0000 7f9305081640 1 -- 172.21.15.86:0/2653284604 shutdown_connections 2024-04-21T08:45:01.897 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:01.895+0000 7f9305081640 1 -- 172.21.15.86:0/2653284604 wait complete. 2024-04-21T08:45:02.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:01 smithi086 ceph-mon[27939]: pgmap v190: 129 pgs: 129 active+clean; 5.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 17 op/s 2024-04-21T08:45:02.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:01 smithi088 ceph-mon[31259]: pgmap v190: 129 pgs: 129 active+clean; 5.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 17 op/s 2024-04-21T08:45:02.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:01 smithi179 ceph-mon[31829]: pgmap v190: 129 pgs: 129 active+clean; 5.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 39 KiB/s wr, 17 op/s 2024-04-21T08:45:04.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:03 smithi086 ceph-mon[27939]: pgmap v191: 129 pgs: 129 active+clean; 1.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 109 KiB/s wr, 44 op/s 2024-04-21T08:45:04.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:03 smithi088 ceph-mon[31259]: pgmap v191: 129 pgs: 129 active+clean; 1.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 109 KiB/s wr, 44 op/s 2024-04-21T08:45:04.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:03 smithi179 ceph-mon[31829]: pgmap v191: 129 pgs: 129 active+clean; 1.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 75 KiB/s rd, 109 KiB/s wr, 44 op/s 2024-04-21T08:45:06.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:05 smithi086 ceph-mon[27939]: pgmap v192: 129 pgs: 129 active+clean; 1.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 85 KiB/s wr, 33 op/s 2024-04-21T08:45:06.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:05 smithi086 ceph-mon[27939]: osdmap e95: 12 total, 12 up, 12 in 2024-04-21T08:45:06.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:05 smithi088 ceph-mon[31259]: pgmap v192: 129 pgs: 129 active+clean; 1.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 85 KiB/s wr, 33 op/s 2024-04-21T08:45:06.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:05 smithi088 ceph-mon[31259]: osdmap e95: 12 total, 12 up, 12 in 2024-04-21T08:45:06.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:05 smithi179 ceph-mon[31829]: pgmap v192: 129 pgs: 129 active+clean; 1.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 85 KiB/s wr, 33 op/s 2024-04-21T08:45:06.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:05 smithi179 ceph-mon[31829]: osdmap e95: 12 total, 12 up, 12 in 2024-04-21T08:45:08.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:07 smithi086 ceph-mon[27939]: pgmap v194: 129 pgs: 129 active+clean; 1.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 40 op/s 2024-04-21T08:45:08.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:07 smithi088 ceph-mon[31259]: pgmap v194: 129 pgs: 129 active+clean; 1.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 40 op/s 2024-04-21T08:45:08.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:07 smithi179 ceph-mon[31829]: pgmap v194: 129 pgs: 129 active+clean; 1.7 MiB data, 407 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 40 op/s 2024-04-21T08:45:10.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:09 smithi088 ceph-mon[31259]: pgmap v195: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 39 op/s 2024-04-21T08:45:10.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:09 smithi179 ceph-mon[31829]: pgmap v195: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 39 op/s 2024-04-21T08:45:10.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:09 smithi086 ceph-mon[27939]: pgmap v195: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 103 KiB/s wr, 39 op/s 2024-04-21T08:45:11.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:11 smithi086 ceph-mon[27939]: osdmap e96: 12 total, 12 up, 12 in 2024-04-21T08:45:11.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:11 smithi088 ceph-mon[31259]: osdmap e96: 12 total, 12 up, 12 in 2024-04-21T08:45:11.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:11 smithi179 ceph-mon[31829]: osdmap e96: 12 total, 12 up, 12 in 2024-04-21T08:45:12.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:12 smithi086 ceph-mon[27939]: pgmap v197: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 9 op/s 2024-04-21T08:45:12.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:12 smithi088 ceph-mon[31259]: pgmap v197: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 9 op/s 2024-04-21T08:45:12.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:12 smithi179 ceph-mon[31829]: pgmap v197: 129 pgs: 129 active+clean; 1.8 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 25 KiB/s wr, 9 op/s 2024-04-21T08:45:14.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:14 smithi086 ceph-mon[27939]: pgmap v198: 129 pgs: 129 active+clean; 2.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 19 op/s 2024-04-21T08:45:14.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:14 smithi088 ceph-mon[31259]: pgmap v198: 129 pgs: 129 active+clean; 2.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 19 op/s 2024-04-21T08:45:14.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.989+0000 7f8dca0e4640 1 -- 172.21.15.86:0/1894850336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dc406b3e0 msgr2=0x7f8dc4106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:14.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.989+0000 7f8dca0e4640 1 --2- 172.21.15.86:0/1894850336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dc406b3e0 0x7f8dc4106bb0 secure :-1 s=READY pgs=1371 cs=0 l=1 rev1=1 crypto rx=0x7f8db8009900 tx=0x7f8db802bc30 comp rx=0 tx=0).stop 2024-04-21T08:45:14.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.989+0000 7f8dca0e4640 1 -- 172.21.15.86:0/1894850336 shutdown_connections 2024-04-21T08:45:14.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.989+0000 7f8dca0e4640 1 --2- 172.21.15.86:0/1894850336 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8dc41072e0 0x7f8dc410ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:14.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.989+0000 7f8dca0e4640 1 --2- 172.21.15.86:0/1894850336 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dc406b3e0 0x7f8dc4106bb0 unknown :-1 s=CLOSED pgs=1371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:14.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.989+0000 7f8dca0e4640 1 --2- 172.21.15.86:0/1894850336 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8dc406aa30 0x7f8dc406ae10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.989+0000 7f8dca0e4640 1 -- 172.21.15.86:0/1894850336 >> 172.21.15.86:0/1894850336 conn(0x7f8dc4077ba0 msgr2=0x7f8dc4077fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.989+0000 7f8dca0e4640 1 -- 172.21.15.86:0/1894850336 shutdown_connections 2024-04-21T08:45:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.989+0000 7f8dca0e4640 1 -- 172.21.15.86:0/1894850336 wait complete. 2024-04-21T08:45:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.990+0000 7f8dca0e4640 1 Processor -- start 2024-04-21T08:45:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.990+0000 7f8dca0e4640 1 -- start start 2024-04-21T08:45:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.990+0000 7f8dca0e4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8dc406aa30 0x7f8dc4198630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.990+0000 7f8dca0e4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8dc41072e0 0x7f8dc4198b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.990+0000 7f8dca0e4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dc419cef0 0x7f8dc419d3a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.990+0000 7f8dca0e4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc41121b0 con 0x7f8dc419cef0 2024-04-21T08:45:14.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.990+0000 7f8dca0e4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc4112030 con 0x7f8dc41072e0 2024-04-21T08:45:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.990+0000 7f8dca0e4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc4112330 con 0x7f8dc406aa30 2024-04-21T08:45:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.991+0000 7f8dc3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dc419cef0 0x7f8dc419d3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.991+0000 7f8dc3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dc419cef0 0x7f8dc419d3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34860/0 (socket says 172.21.15.86:34860) 2024-04-21T08:45:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.991+0000 7f8dc3fff640 1 -- 172.21.15.86:0/2434035982 learned_addr learned my addr 172.21.15.86:0/2434035982 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.991+0000 7f8dc37fe640 1 --2- 172.21.15.86:0/2434035982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8dc406aa30 0x7f8dc4198630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.991+0000 7f8dc2ffd640 1 --2- 172.21.15.86:0/2434035982 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8dc41072e0 0x7f8dc4198b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.991+0000 7f8dc3fff640 1 -- 172.21.15.86:0/2434035982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8dc406aa30 msgr2=0x7f8dc4198630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.991+0000 7f8dc3fff640 1 --2- 172.21.15.86:0/2434035982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8dc406aa30 0x7f8dc4198630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.991+0000 7f8dc3fff640 1 -- 172.21.15.86:0/2434035982 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8dc41072e0 msgr2=0x7f8dc4198b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.991+0000 7f8dc3fff640 1 --2- 172.21.15.86:0/2434035982 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8dc41072e0 0x7f8dc4198b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.991+0000 7f8dc3fff640 1 -- 172.21.15.86:0/2434035982 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8dc419d9e0 con 0x7f8dc419cef0 2024-04-21T08:45:14.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.991+0000 7f8dc2ffd640 1 --2- 172.21.15.86:0/2434035982 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8dc41072e0 0x7f8dc4198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.991+0000 7f8dc3fff640 1 --2- 172.21.15.86:0/2434035982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dc419cef0 0x7f8dc419d3a0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc00cad0 tx=0x7f8dbc00ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.991+0000 7f8dc0ff9640 1 -- 172.21.15.86:0/2434035982 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc00ad60 con 0x7f8dc419cef0 2024-04-21T08:45:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.991+0000 7f8dca0e4640 1 -- 172.21.15.86:0/2434035982 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8dc419dcd0 con 0x7f8dc419cef0 2024-04-21T08:45:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.991+0000 7f8dc0ff9640 1 -- 172.21.15.86:0/2434035982 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8dbc005020 con 0x7f8dc419cef0 2024-04-21T08:45:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.991+0000 7f8dc0ff9640 1 -- 172.21.15.86:0/2434035982 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8dbc00f6b0 con 0x7f8dc419cef0 2024-04-21T08:45:14.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.991+0000 7f8dca0e4640 1 -- 172.21.15.86:0/2434035982 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8dc41a54c0 con 0x7f8dc419cef0 2024-04-21T08:45:14.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:14 smithi179 ceph-mon[31829]: pgmap v198: 129 pgs: 129 active+clean; 2.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 51 KiB/s wr, 19 op/s 2024-04-21T08:45:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.994+0000 7f8dc0ff9640 1 -- 172.21.15.86:0/2434035982 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8dbc0051d0 con 0x7f8dc419cef0 2024-04-21T08:45:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.994+0000 7f8dccdb2740 1 Processor -- start 2024-04-21T08:45:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.994+0000 7f8dccdb2740 1 -- start start 2024-04-21T08:45:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.994+0000 7f8dc0ff9640 1 --2- 172.21.15.86:0/2434035982 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8dac0a3dc0 0x7f8dac0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.994+0000 7f8dccdb2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d90f6038d0 0x55d90f602620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.994+0000 7f8dccdb2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d90f602b60 0x55d90f602fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.994+0000 7f8dccdb2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d90f60c0e0 0x55d90f60e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:15.004 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.994+0000 7f8dccdb2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d90f5678d0 con 0x55d90f6038d0 2024-04-21T08:45:15.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.994+0000 7f8dccdb2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d90f504e80 con 0x55d90f602b60 2024-04-21T08:45:15.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.994+0000 7f8dccdb2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d90f563890 con 0x55d90f60c0e0 2024-04-21T08:45:15.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.995+0000 7f8dc2ffd640 1 --2- 172.21.15.86:0/2434035982 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8dac0a3dc0 0x7f8dac0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:15.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.995+0000 7f8dc37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d90f6038d0 0x55d90f602620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:15.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.995+0000 7f8dc0ff9640 1 -- 172.21.15.86:0/2434035982 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f8dbc0d0c10 con 0x7f8dc419cef0 2024-04-21T08:45:15.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.995+0000 7f8dc37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d90f6038d0 0x55d90f602620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34870/0 (socket says 172.21.15.86:34870) 2024-04-21T08:45:15.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.995+0000 7f8dc37fe640 1 -- 172.21.15.86:0/3126989642 learned_addr learned my addr 172.21.15.86:0/3126989642 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:15.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.995+0000 7f8dc2ffd640 1 --2- 172.21.15.86:0/3126989642 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d90f602b60 0x55d90f602fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:15.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.995+0000 7f8dc37fe640 1 -- 172.21.15.86:0/3126989642 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d90f60c0e0 msgr2=0x55d90f60e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:15.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.995+0000 7f8dc37fe640 1 --2- 172.21.15.86:0/3126989642 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d90f60c0e0 0x55d90f60e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:15.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.995+0000 7f8dc37fe640 1 -- 172.21.15.86:0/3126989642 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d90f602b60 msgr2=0x55d90f602fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:15.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.995+0000 7f8dc37fe640 1 --2- 172.21.15.86:0/3126989642 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d90f602b60 0x55d90f602fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.995+0000 7f8dc37fe640 1 -- 172.21.15.86:0/3126989642 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d90f526320 con 0x55d90f6038d0 2024-04-21T08:45:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.995+0000 7f8dc37fe640 1 --2- 172.21.15.86:0/3126989642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d90f6038d0 0x55d90f602620 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f8db400ad00 tx=0x7f8db400f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bec5018974be9f1e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.995+0000 7f8dc2ffd640 1 --2- 172.21.15.86:0/2434035982 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8dac0a3dc0 0x7f8dac0a6280 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8db8006010 tx=0x7f8db8004450 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.995+0000 7f8da9ffb640 1 -- 172.21.15.86:0/3126989642 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db401d070 con 0x55d90f6038d0 2024-04-21T08:45:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.995+0000 7f8da9ffb640 1 -- 172.21.15.86:0/3126989642 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8db4018af0 con 0x55d90f6038d0 2024-04-21T08:45:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.995+0000 7f8da9ffb640 1 -- 172.21.15.86:0/3126989642 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db4010aa0 con 0x55d90f6038d0 2024-04-21T08:45:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.996+0000 7f8dccdb2740 1 -- 172.21.15.86:0/3126989642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d90f6038d0 msgr2=0x55d90f602620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.996+0000 7f8dccdb2740 1 --2- 172.21.15.86:0/3126989642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d90f6038d0 0x55d90f602620 secure :-1 s=READY pgs=1373 cs=0 l=1 rev1=1 crypto rx=0x7f8db400ad00 tx=0x7f8db400f790 comp rx=0 tx=0).stop 2024-04-21T08:45:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.996+0000 7f8dccdb2740 1 -- 172.21.15.86:0/3126989642 shutdown_connections 2024-04-21T08:45:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.996+0000 7f8dccdb2740 1 --2- 172.21.15.86:0/3126989642 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d90f60c0e0 0x55d90f60e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:15.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.996+0000 7f8dccdb2740 1 --2- 172.21.15.86:0/3126989642 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d90f602b60 0x55d90f602fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:15.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.996+0000 7f8dccdb2740 1 --2- 172.21.15.86:0/3126989642 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d90f6038d0 0x55d90f602620 unknown :-1 s=CLOSED pgs=1373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:15.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.996+0000 7f8dccdb2740 1 -- 172.21.15.86:0/3126989642 >> 172.21.15.86:0/3126989642 conn(0x55d90f5c9c90 msgr2=0x55d90f5fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:15.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.996+0000 7f8dccdb2740 1 -- 172.21.15.86:0/3126989642 shutdown_connections 2024-04-21T08:45:15.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.996+0000 7f8dccdb2740 1 -- 172.21.15.86:0/3126989642 wait complete. 2024-04-21T08:45:15.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.996+0000 7f8dccdb2740 1 -- start start 2024-04-21T08:45:15.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.997+0000 7f8dccdb2740 1 Processor -- start 2024-04-21T08:45:15.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.997+0000 7f8dccdb2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d90f602b60 0x55d90f60baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:15.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.997+0000 7f8dccdb2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d90f6038d0 0x55d90f608e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:15.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.997+0000 7f8dccdb2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d90f60c0e0 0x55d90f609440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:15.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.997+0000 7f8dccdb2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d90f60ee10 con 0x55d90f602b60 2024-04-21T08:45:15.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.997+0000 7f8dccdb2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d90f4efeb0 con 0x55d90f6038d0 2024-04-21T08:45:15.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.997+0000 7f8dccdb2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d90f60ef90 con 0x55d90f60c0e0 2024-04-21T08:45:15.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.997+0000 7f8dccdb2740 10 client.0 _ll_register_callbacks cb 0x55d90f51cb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:15.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.997+0000 7f8dc37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d90f602b60 0x55d90f60baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:15.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.997+0000 7f8dc37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d90f602b60 0x55d90f60baa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34880/0 (socket says 172.21.15.86:34880) 2024-04-21T08:45:15.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.997+0000 7f8dc37fe640 1 -- 172.21.15.86:0/1309668192 learned_addr learned my addr 172.21.15.86:0/1309668192 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:15.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.997+0000 7f8dc3fff640 1 --2- 172.21.15.86:0/1309668192 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d90f60c0e0 0x55d90f609440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:15.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.997+0000 7f8dc2ffd640 1 --2- 172.21.15.86:0/1309668192 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d90f6038d0 0x55d90f608e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:15.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.997+0000 7f8dc37fe640 1 -- 172.21.15.86:0/1309668192 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d90f60c0e0 msgr2=0x55d90f609440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:15.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.997+0000 7f8dc37fe640 1 --2- 172.21.15.86:0/1309668192 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d90f60c0e0 0x55d90f609440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:15.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.997+0000 7f8dc37fe640 1 -- 172.21.15.86:0/1309668192 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d90f6038d0 msgr2=0x55d90f608e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:15.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.997+0000 7f8dc37fe640 1 --2- 172.21.15.86:0/1309668192 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d90f6038d0 0x55d90f608e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:15.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.997+0000 7f8dc37fe640 1 -- 172.21.15.86:0/1309668192 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d90f27a210 con 0x55d90f602b60 2024-04-21T08:45:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.997+0000 7f8dc3fff640 1 --2- 172.21.15.86:0/1309668192 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d90f60c0e0 0x55d90f609440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.997+0000 7f8dc2ffd640 1 --2- 172.21.15.86:0/1309668192 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d90f6038d0 0x55d90f608e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.997+0000 7f8dc37fe640 1 --2- 172.21.15.86:0/1309668192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d90f602b60 0x55d90f60baa0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f8db401b040 tx=0x7f8db4018440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a29abc452f146eed server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.997+0000 7f8da2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.998+0000 7f8da2ffd640 1 -- 172.21.15.86:0/1309668192 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db401d040 con 0x55d90f602b60 2024-04-21T08:45:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.998+0000 7f8da2ffd640 1 -- 172.21.15.86:0/1309668192 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f8db4018860 con 0x55d90f602b60 2024-04-21T08:45:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.998+0000 7f8dccdb2740 1 -- 172.21.15.86:0/1309668192 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8db4018af0 con 0x55d90f602b60 2024-04-21T08:45:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.998+0000 7f8da2ffd640 1 -- 172.21.15.86:0/1309668192 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8db4031050 con 0x55d90f602b60 2024-04-21T08:45:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.998+0000 7f8dccdb2740 10 client.18300 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.998+0000 7f8dccdb2740 1 -- 172.21.15.86:0/1309668192 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d90f346620 con 0x55d90f602b60 2024-04-21T08:45:15.009 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.998+0000 7f8da2ffd640 1 -- 172.21.15.86:0/1309668192 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8db4018af0 con 0x55d90f602b60 2024-04-21T08:45:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.998+0000 7f8da2ffd640 1 -- 172.21.15.86:0/1309668192 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8db4020030 con 0x55d90f602b60 2024-04-21T08:45:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.998+0000 7f8dccdb2740 10 client.18300 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.998+0000 7f8dccdb2740 10 client.18300 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.998+0000 7f8dccdb2740 20 client.18300 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.998+0000 7f8dccdb2740 1 --2- 172.21.15.86:0/1309668192 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d90f61c050 0x55d90f61e440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.998+0000 7f8dccdb2740 4 client.18300 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.998+0000 7f8dccdb2740 1 -- 172.21.15.86:0/1309668192 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d90f57b880 con 0x55d90f61c050 2024-04-21T08:45:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.998+0000 7f8dc2ffd640 1 --2- 172.21.15.86:0/1309668192 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d90f61c050 0x55d90f61e440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.999+0000 7f8da2ffd640 10 client.18300 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:14.999+0000 7f8dc2ffd640 1 --2- 172.21.15.86:0/1309668192 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d90f61c050 0x55d90f61e440 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9cdaca814ce2c45 server_cookie=d74bb55a79b328fd in_seq=0 out_seq=0 2024-04-21T08:45:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.000+0000 7f8da2ffd640 1 -- 172.21.15.86:0/1309668192 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d90f57b880 con 0x55d90f61c050 2024-04-21T08:45:15.010 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.000+0000 7f8da2ffd640 10 client.18300 handle_command_reply: tid=0 2024-04-21T08:45:15.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.000+0000 7f8dccdb2740 1 client.18300 shutdown 2024-04-21T08:45:15.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.000+0000 7f8dccdb2740 1 -- 172.21.15.86:0/1309668192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d90f602b60 msgr2=0x55d90f60baa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:15.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.000+0000 7f8dccdb2740 1 --2- 172.21.15.86:0/1309668192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d90f602b60 0x55d90f60baa0 secure :-1 s=READY pgs=1374 cs=0 l=1 rev1=1 crypto rx=0x7f8db401b040 tx=0x7f8db4018440 comp rx=0 tx=0).stop 2024-04-21T08:45:15.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.000+0000 7f8dccdb2740 1 -- 172.21.15.86:0/1309668192 shutdown_connections 2024-04-21T08:45:15.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.000+0000 7f8dccdb2740 1 --2- 172.21.15.86:0/1309668192 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d90f61c050 0x55d90f61e440 crc :-1 s=READY pgs=203 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:15.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.000+0000 7f8dccdb2740 1 --2- 172.21.15.86:0/1309668192 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d90f60c0e0 0x55d90f609440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:15.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.000+0000 7f8dccdb2740 1 --2- 172.21.15.86:0/1309668192 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d90f6038d0 0x55d90f608e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:15.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.000+0000 7f8da2ffd640 0 client.18300 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:15.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.000+0000 7f8dccdb2740 1 --2- 172.21.15.86:0/1309668192 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d90f602b60 0x55d90f60baa0 unknown :-1 s=CLOSED pgs=1374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:15.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.000+0000 7f8dccdb2740 1 -- 172.21.15.86:0/1309668192 >> 172.21.15.86:0/1309668192 conn(0x55d90f5c9c90 msgr2=0x55d90f5ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:15.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.000+0000 7f8dccdb2740 1 -- 172.21.15.86:0/1309668192 shutdown_connections 2024-04-21T08:45:15.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.000+0000 7f8dccdb2740 1 -- 172.21.15.86:0/1309668192 wait complete. 2024-04-21T08:45:15.011 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.000+0000 7f8dccdb2740 20 client.18300 trim_cache size 0 max 16384 2024-04-21T08:45:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.014+0000 7f8dccdb2740 1 Processor -- start 2024-04-21T08:45:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.015+0000 7f8dccdb2740 1 -- start start 2024-04-21T08:45:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.015+0000 7f8dccdb2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d90f5c9c90 0x55d90f608710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.015+0000 7f8dccdb2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d90f602b60 0x55d90f5fc750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.015+0000 7f8dccdb2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d90f6038d0 0x55d90f60b560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.015+0000 7f8dccdb2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc41af3a0 con 0x55d90f602b60 2024-04-21T08:45:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.015+0000 7f8dccdb2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc41b0ee0 con 0x55d90f6038d0 2024-04-21T08:45:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.015+0000 7f8dccdb2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc41afa70 con 0x55d90f5c9c90 2024-04-21T08:45:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.015+0000 7f8dc2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d90f602b60 0x55d90f5fc750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.015+0000 7f8dc2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d90f602b60 0x55d90f5fc750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34884/0 (socket says 172.21.15.86:34884) 2024-04-21T08:45:15.020 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.015+0000 7f8dc2ffd640 1 -- 172.21.15.86:0/2972758910 learned_addr learned my addr 172.21.15.86:0/2972758910 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:15.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.015+0000 7f8dc3fff640 1 --2- 172.21.15.86:0/2972758910 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d90f6038d0 0x55d90f60b560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:15.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.015+0000 7f8dc37fe640 1 --2- 172.21.15.86:0/2972758910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d90f5c9c90 0x55d90f608710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:15.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.015+0000 7f8dc2ffd640 1 -- 172.21.15.86:0/2972758910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d90f5c9c90 msgr2=0x55d90f608710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:15.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.015+0000 7f8dc2ffd640 1 --2- 172.21.15.86:0/2972758910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d90f5c9c90 0x55d90f608710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:15.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.015+0000 7f8dc2ffd640 1 -- 172.21.15.86:0/2972758910 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d90f6038d0 msgr2=0x55d90f60b560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:15.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.015+0000 7f8dc2ffd640 1 --2- 172.21.15.86:0/2972758910 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d90f6038d0 0x55d90f60b560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:15.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.015+0000 7f8dc2ffd640 1 -- 172.21.15.86:0/2972758910 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d90f6097b0 con 0x55d90f602b60 2024-04-21T08:45:15.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.015+0000 7f8dc3fff640 1 --2- 172.21.15.86:0/2972758910 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d90f6038d0 0x55d90f60b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:15.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.015+0000 7f8dc2ffd640 1 --2- 172.21.15.86:0/2972758910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d90f602b60 0x55d90f5fc750 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4079070 tx=0x7f8db8047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=49cee4cc34e3d946 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:15.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.015+0000 7f8da97fa640 1 -- 172.21.15.86:0/2972758910 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db8005510 con 0x55d90f602b60 2024-04-21T08:45:15.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.015+0000 7f8da97fa640 1 -- 172.21.15.86:0/2972758910 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8db8051900 con 0x55d90f602b60 2024-04-21T08:45:15.021 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.016+0000 7f8da97fa640 1 -- 172.21.15.86:0/2972758910 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db8004990 con 0x55d90f602b60 2024-04-21T08:45:15.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.016+0000 7f8dccdb2740 1 -- 172.21.15.86:0/2972758910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d90f602b60 msgr2=0x55d90f5fc750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:15.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.016+0000 7f8dccdb2740 1 --2- 172.21.15.86:0/2972758910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d90f602b60 0x55d90f5fc750 secure :-1 s=READY pgs=1375 cs=0 l=1 rev1=1 crypto rx=0x7f8dc4079070 tx=0x7f8db8047000 comp rx=0 tx=0).stop 2024-04-21T08:45:15.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.016+0000 7f8dccdb2740 1 -- 172.21.15.86:0/2972758910 shutdown_connections 2024-04-21T08:45:15.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.016+0000 7f8dccdb2740 1 --2- 172.21.15.86:0/2972758910 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d90f6038d0 0x55d90f60b560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:15.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.016+0000 7f8dccdb2740 1 --2- 172.21.15.86:0/2972758910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d90f602b60 0x55d90f5fc750 unknown :-1 s=CLOSED pgs=1375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:15.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.016+0000 7f8dccdb2740 1 --2- 172.21.15.86:0/2972758910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d90f5c9c90 0x55d90f608710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:15.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.016+0000 7f8dccdb2740 1 -- 172.21.15.86:0/2972758910 >> 172.21.15.86:0/2972758910 conn(0x55d90f604800 msgr2=0x55d90f5fc040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:15.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.016+0000 7f8dccdb2740 1 -- 172.21.15.86:0/2972758910 shutdown_connections 2024-04-21T08:45:15.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.016+0000 7f8dccdb2740 1 -- 172.21.15.86:0/2972758910 wait complete. 2024-04-21T08:45:15.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.016+0000 7f8dccdb2740 1 -- start start 2024-04-21T08:45:15.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.017+0000 7f8dccdb2740 1 Processor -- start 2024-04-21T08:45:15.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.017+0000 7f8dccdb2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d90f5c9c90 0x55d90f614250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:15.022 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.017+0000 7f8dccdb2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d90f602b60 0x55d90f614790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:15.023 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.017+0000 7f8dccdb2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d90f6038d0 0x55d90f60f970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:15.023 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.017+0000 7f8dccdb2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc41b0810 con 0x55d90f602b60 2024-04-21T08:45:15.023 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.017+0000 7f8dccdb2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc41b0140 con 0x55d90f6038d0 2024-04-21T08:45:15.023 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.017+0000 7f8dccdb2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8dc41ad3a0 con 0x55d90f5c9c90 2024-04-21T08:45:15.023 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.017+0000 7f8dccdb2740 10 client.0 _ll_register_callbacks cb 0x7f8db401b070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:15.023 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.017+0000 7f8dc2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d90f602b60 0x55d90f614790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:15.023 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.017+0000 7f8dc2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d90f602b60 0x55d90f614790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34902/0 (socket says 172.21.15.86:34902) 2024-04-21T08:45:15.023 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.017+0000 7f8dc2ffd640 1 -- 172.21.15.86:0/3671055722 learned_addr learned my addr 172.21.15.86:0/3671055722 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:15.023 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.017+0000 7f8dc37fe640 1 --2- 172.21.15.86:0/3671055722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d90f5c9c90 0x55d90f614250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:15.023 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.017+0000 7f8dc3fff640 1 --2- 172.21.15.86:0/3671055722 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d90f6038d0 0x55d90f60f970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:15.023 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.017+0000 7f8dc2ffd640 1 -- 172.21.15.86:0/3671055722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d90f5c9c90 msgr2=0x55d90f614250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:15.023 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.017+0000 7f8dc2ffd640 1 --2- 172.21.15.86:0/3671055722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d90f5c9c90 0x55d90f614250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:15.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.017+0000 7f8dc2ffd640 1 -- 172.21.15.86:0/3671055722 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d90f6038d0 msgr2=0x55d90f60f970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:15.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.017+0000 7f8dc2ffd640 1 --2- 172.21.15.86:0/3671055722 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d90f6038d0 0x55d90f60f970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:15.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.017+0000 7f8dc2ffd640 1 -- 172.21.15.86:0/3671055722 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d90f6101a0 con 0x55d90f602b60 2024-04-21T08:45:15.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.017+0000 7f8dc2ffd640 1 --2- 172.21.15.86:0/3671055722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d90f602b60 0x55d90f614790 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f8db8009850 tx=0x7f8db8005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d7762d6fc16e69d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:15.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.018+0000 7f8da2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:15.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.018+0000 7f8da2ffd640 1 -- 172.21.15.86:0/3671055722 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8db802cb10 con 0x55d90f602b60 2024-04-21T08:45:15.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.018+0000 7f8da2ffd640 1 -- 172.21.15.86:0/3671055722 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f8db8051450 con 0x55d90f602b60 2024-04-21T08:45:15.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.018+0000 7f8dccdb2740 1 -- 172.21.15.86:0/3671055722 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8db8051900 con 0x55d90f602b60 2024-04-21T08:45:15.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.018+0000 7f8da2ffd640 1 -- 172.21.15.86:0/3671055722 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8db806e050 con 0x55d90f602b60 2024-04-21T08:45:15.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.018+0000 7f8dccdb2740 10 client.18306 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:15.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.018+0000 7f8dccdb2740 1 -- 172.21.15.86:0/3671055722 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d90f682000 con 0x55d90f602b60 2024-04-21T08:45:15.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.018+0000 7f8da2ffd640 1 -- 172.21.15.86:0/3671055722 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8db8051900 con 0x55d90f602b60 2024-04-21T08:45:15.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.018+0000 7f8da2ffd640 1 -- 172.21.15.86:0/3671055722 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8db8051ce0 con 0x55d90f602b60 2024-04-21T08:45:15.024 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.018+0000 7f8dccdb2740 10 client.18306 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.018+0000 7f8dccdb2740 10 client.18306 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.018+0000 7f8dccdb2740 20 client.18306 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.018+0000 7f8dccdb2740 1 --2- 172.21.15.86:0/3671055722 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d90f60c0e0 0x55d90f684810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.018+0000 7f8dccdb2740 4 client.18306 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:45:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.018+0000 7f8dccdb2740 1 -- 172.21.15.86:0/3671055722 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d90f5073d0 con 0x55d90f60c0e0 2024-04-21T08:45:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.019+0000 7f8dc37fe640 1 --2- 172.21.15.86:0/3671055722 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d90f60c0e0 0x55d90f684810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.019+0000 7f8dc37fe640 1 --2- 172.21.15.86:0/3671055722 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d90f60c0e0 0x55d90f684810 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e7d54d6f2a4bef06 server_cookie=36fb20f2e6749af0 in_seq=0 out_seq=0 2024-04-21T08:45:15.025 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.019+0000 7f8da2ffd640 10 client.18306 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:15.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.034+0000 7f8da2ffd640 1 -- 172.21.15.86:0/3671055722 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d90f5073d0 con 0x55d90f60c0e0 2024-04-21T08:45:15.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.034+0000 7f8da2ffd640 10 client.18306 handle_command_reply: tid=0 2024-04-21T08:45:15.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.034+0000 7f8dccdb2740 1 client.18306 shutdown 2024-04-21T08:45:15.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.034+0000 7f8dccdb2740 1 -- 172.21.15.86:0/3671055722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d90f602b60 msgr2=0x55d90f614790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:15.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.034+0000 7f8dccdb2740 1 --2- 172.21.15.86:0/3671055722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d90f602b60 0x55d90f614790 secure :-1 s=READY pgs=1376 cs=0 l=1 rev1=1 crypto rx=0x7f8db8009850 tx=0x7f8db8005620 comp rx=0 tx=0).stop 2024-04-21T08:45:15.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.034+0000 7f8dccdb2740 1 -- 172.21.15.86:0/3671055722 shutdown_connections 2024-04-21T08:45:15.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.034+0000 7f8dccdb2740 1 --2- 172.21.15.86:0/3671055722 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d90f60c0e0 0x55d90f684810 crc :-1 s=READY pgs=204 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:15.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.034+0000 7f8dccdb2740 1 --2- 172.21.15.86:0/3671055722 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d90f6038d0 0x55d90f60f970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:15.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.034+0000 7f8dccdb2740 1 --2- 172.21.15.86:0/3671055722 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d90f602b60 0x55d90f614790 unknown :-1 s=CLOSED pgs=1376 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:15.035 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.034+0000 7f8dccdb2740 1 --2- 172.21.15.86:0/3671055722 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d90f5c9c90 0x55d90f614250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.034+0000 7f8da2ffd640 0 client.18306 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.034+0000 7f8dccdb2740 1 -- 172.21.15.86:0/3671055722 >> 172.21.15.86:0/3671055722 conn(0x55d90f604800 msgr2=0x55d90f5fc8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.034+0000 7f8dccdb2740 1 -- 172.21.15.86:0/3671055722 shutdown_connections 2024-04-21T08:45:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.034+0000 7f8dccdb2740 1 -- 172.21.15.86:0/3671055722 wait complete. 2024-04-21T08:45:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.034+0000 7f8dccdb2740 20 client.18306 trim_cache size 0 max 16384 2024-04-21T08:45:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:45:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:45:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:45:15.036 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:45:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.035+0000 7f8dca0e4640 1 -- 172.21.15.86:0/2434035982 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8dac0a3dc0 msgr2=0x7f8dac0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.035+0000 7f8dca0e4640 1 --2- 172.21.15.86:0/2434035982 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8dac0a3dc0 0x7f8dac0a6280 secure :-1 s=READY pgs=218 cs=0 l=1 rev1=1 crypto rx=0x7f8db8006010 tx=0x7f8db8004450 comp rx=0 tx=0).stop 2024-04-21T08:45:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.035+0000 7f8dca0e4640 1 -- 172.21.15.86:0/2434035982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dc419cef0 msgr2=0x7f8dc419d3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.035+0000 7f8dca0e4640 1 --2- 172.21.15.86:0/2434035982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dc419cef0 0x7f8dc419d3a0 secure :-1 s=READY pgs=1372 cs=0 l=1 rev1=1 crypto rx=0x7f8dbc00cad0 tx=0x7f8dbc00ced0 comp rx=0 tx=0).stop 2024-04-21T08:45:15.037 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.035+0000 7f8dca0e4640 1 -- 172.21.15.86:0/2434035982 shutdown_connections 2024-04-21T08:45:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.035+0000 7f8dca0e4640 1 --2- 172.21.15.86:0/2434035982 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8dac0a3dc0 0x7f8dac0a6280 unknown :-1 s=CLOSED pgs=218 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.035+0000 7f8dca0e4640 1 --2- 172.21.15.86:0/2434035982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8dc419cef0 0x7f8dc419d3a0 unknown :-1 s=CLOSED pgs=1372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.035+0000 7f8dca0e4640 1 --2- 172.21.15.86:0/2434035982 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8dc41072e0 0x7f8dc4198b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.035+0000 7f8dca0e4640 1 --2- 172.21.15.86:0/2434035982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8dc406aa30 0x7f8dc4198630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.035+0000 7f8dca0e4640 1 -- 172.21.15.86:0/2434035982 >> 172.21.15.86:0/2434035982 conn(0x7f8dc4077ba0 msgr2=0x7f8dc40ff270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.035+0000 7f8dca0e4640 1 -- 172.21.15.86:0/2434035982 shutdown_connections 2024-04-21T08:45:15.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:15.035+0000 7f8dca0e4640 1 -- 172.21.15.86:0/2434035982 wait complete. 2024-04-21T08:45:16.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:16 smithi086 ceph-mon[27939]: pgmap v199: 129 pgs: 129 active+clean; 2.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:45:16.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:16 smithi088 ceph-mon[31259]: pgmap v199: 129 pgs: 129 active+clean; 2.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:45:16.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:16 smithi179 ceph-mon[31829]: pgmap v199: 129 pgs: 129 active+clean; 2.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 45 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:45:18.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:18 smithi086 ceph-mon[27939]: pgmap v200: 129 pgs: 129 active+clean; 2.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:45:18.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:18 smithi088 ceph-mon[31259]: pgmap v200: 129 pgs: 129 active+clean; 2.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:45:18.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:18 smithi179 ceph-mon[31829]: pgmap v200: 129 pgs: 129 active+clean; 2.0 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:45:19.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:19 smithi086 ceph-mon[27939]: pgmap v201: 129 pgs: 129 active+clean; 2.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-04-21T08:45:19.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:19 smithi088 ceph-mon[31259]: pgmap v201: 129 pgs: 129 active+clean; 2.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-04-21T08:45:19.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:19 smithi179 ceph-mon[31829]: pgmap v201: 129 pgs: 129 active+clean; 2.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-04-21T08:45:20.516 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:45:20.517 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:45:20.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.629+0000 7f9752752640 1 -- 172.21.15.86:0/1620818429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f974c101650 msgr2=0x7f974c10e3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:20.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.629+0000 7f9752752640 1 --2- 172.21.15.86:0/1620818429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f974c101650 0x7f974c10e3a0 secure :-1 s=READY pgs=1377 cs=0 l=1 rev1=1 crypto rx=0x7f9740009900 tx=0x7f974002bc30 comp rx=0 tx=0).stop 2024-04-21T08:45:20.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.629+0000 7f9752752640 1 -- 172.21.15.86:0/1620818429 shutdown_connections 2024-04-21T08:45:20.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.629+0000 7f9752752640 1 --2- 172.21.15.86:0/1620818429 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f974c10e8e0 0x7f974c110cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.629+0000 7f9752752640 1 --2- 172.21.15.86:0/1620818429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f974c101650 0x7f974c10e3a0 unknown :-1 s=CLOSED pgs=1377 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.629+0000 7f9752752640 1 --2- 172.21.15.86:0/1620818429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f974c100d30 0x7f974c101110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.629+0000 7f9752752640 1 -- 172.21.15.86:0/1620818429 >> 172.21.15.86:0/1620818429 conn(0x7f974c0771d0 msgr2=0x7f974c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:20.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.629+0000 7f9752752640 1 -- 172.21.15.86:0/1620818429 shutdown_connections 2024-04-21T08:45:20.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.629+0000 7f9752752640 1 -- 172.21.15.86:0/1620818429 wait complete. 2024-04-21T08:45:20.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.630+0000 7f9752752640 1 Processor -- start 2024-04-21T08:45:20.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.630+0000 7f9752752640 1 -- start start 2024-04-21T08:45:20.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.630+0000 7f9752752640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f974c100d30 0x7f974c1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:20.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.630+0000 7f9752752640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f974c101650 0x7f974c1a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:20.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.630+0000 7f9752752640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f974c10e8e0 0x7f974c19a7e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:20.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.630+0000 7f9752752640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f974c114400 con 0x7f974c101650 2024-04-21T08:45:20.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.630+0000 7f9752752640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f974c114280 con 0x7f974c10e8e0 2024-04-21T08:45:20.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.630+0000 7f9752752640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f974c114580 con 0x7f974c100d30 2024-04-21T08:45:20.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.630+0000 7f974b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f974c101650 0x7f974c1a0bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:20.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.630+0000 7f974b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f974c101650 0x7f974c1a0bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47648/0 (socket says 172.21.15.86:47648) 2024-04-21T08:45:20.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.630+0000 7f974b7fe640 1 -- 172.21.15.86:0/1617016414 learned_addr learned my addr 172.21.15.86:0/1617016414 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:20.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.630+0000 7f9750cef640 1 --2- 172.21.15.86:0/1617016414 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f974c10e8e0 0x7f974c19a7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:20.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.630+0000 7f974bfff640 1 --2- 172.21.15.86:0/1617016414 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f974c100d30 0x7f974c1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:20.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.630+0000 7f974b7fe640 1 -- 172.21.15.86:0/1617016414 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f974c100d30 msgr2=0x7f974c1a0670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:20.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.631+0000 7f974b7fe640 1 --2- 172.21.15.86:0/1617016414 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f974c100d30 0x7f974c1a0670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.631+0000 7f974b7fe640 1 -- 172.21.15.86:0/1617016414 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f974c10e8e0 msgr2=0x7f974c19a7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:20.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.631+0000 7f974b7fe640 1 --2- 172.21.15.86:0/1617016414 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f974c10e8e0 0x7f974c19a7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.631+0000 7f974b7fe640 1 -- 172.21.15.86:0/1617016414 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f974c19afb0 con 0x7f974c101650 2024-04-21T08:45:20.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.631+0000 7f9750cef640 1 --2- 172.21.15.86:0/1617016414 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f974c10e8e0 0x7f974c19a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:20.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.631+0000 7f974b7fe640 1 --2- 172.21.15.86:0/1617016414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f974c101650 0x7f974c1a0bb0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f9740005940 tx=0x7f974002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:20.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.631+0000 7f97497fa640 1 -- 172.21.15.86:0/1617016414 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9740043020 con 0x7f974c101650 2024-04-21T08:45:20.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.631+0000 7f9752752640 1 -- 172.21.15.86:0/1617016414 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f974c19b240 con 0x7f974c101650 2024-04-21T08:45:20.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.631+0000 7f97497fa640 1 -- 172.21.15.86:0/1617016414 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f974002cbe0 con 0x7f974c101650 2024-04-21T08:45:20.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.631+0000 7f97497fa640 1 -- 172.21.15.86:0/1617016414 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9740003bf0 con 0x7f974c101650 2024-04-21T08:45:20.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.631+0000 7f9752752640 1 -- 172.21.15.86:0/1617016414 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f974c1a74c0 con 0x7f974c101650 2024-04-21T08:45:20.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.633+0000 7f97497fa640 1 -- 172.21.15.86:0/1617016414 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9740046020 con 0x7f974c101650 2024-04-21T08:45:20.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.633+0000 7f9755429740 1 Processor -- start 2024-04-21T08:45:20.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.633+0000 7f9755429740 1 -- start start 2024-04-21T08:45:20.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.634+0000 7f9755429740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e143a580 0x55c6e143a960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:20.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.634+0000 7f9755429740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6e143aea0 0x55c6e1443540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:20.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.634+0000 7f9755429740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e1443a80 0x55c6e1445e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:20.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.634+0000 7f9755429740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c6e139f1e0 con 0x55c6e143a580 2024-04-21T08:45:20.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.634+0000 7f9755429740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c6e133c390 con 0x55c6e143aea0 2024-04-21T08:45:20.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.634+0000 7f9755429740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c6e139b1a0 con 0x55c6e1443a80 2024-04-21T08:45:20.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.634+0000 7f97497fa640 1 --2- 172.21.15.86:0/1617016414 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f97340a3d20 0x7f97340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:20.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.634+0000 7f9750cef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e143a580 0x55c6e143a960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:20.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.634+0000 7f9750cef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e143a580 0x55c6e143a960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47664/0 (socket says 172.21.15.86:47664) 2024-04-21T08:45:20.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.634+0000 7f9750cef640 1 -- 172.21.15.86:0/2380550864 learned_addr learned my addr 172.21.15.86:0/2380550864 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:20.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.634+0000 7f9750cef640 1 --2- 172.21.15.86:0/1617016414 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f97340a3d20 0x7f97340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:20.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.634+0000 7f974b7fe640 1 --2- 172.21.15.86:0/2380550864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e1443a80 0x55c6e1445e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:20.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.634+0000 7f974bfff640 1 --2- 172.21.15.86:0/2380550864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6e143aea0 0x55c6e1443540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:20.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.634+0000 7f97497fa640 1 -- 172.21.15.86:0/1617016414 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f97400f2db0 con 0x7f974c101650 2024-04-21T08:45:20.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.634+0000 7f9750cef640 1 -- 172.21.15.86:0/2380550864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e1443a80 msgr2=0x55c6e1445e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:20.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.634+0000 7f9750cef640 1 --2- 172.21.15.86:0/2380550864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e1443a80 0x55c6e1445e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.634+0000 7f9750cef640 1 -- 172.21.15.86:0/2380550864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6e143aea0 msgr2=0x55c6e1443540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:20.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.634+0000 7f9750cef640 1 --2- 172.21.15.86:0/2380550864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6e143aea0 0x55c6e1443540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.634+0000 7f9750cef640 1 -- 172.21.15.86:0/2380550864 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6e117b7d0 con 0x55c6e143a580 2024-04-21T08:45:20.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.634+0000 7f974b7fe640 1 --2- 172.21.15.86:0/2380550864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e1443a80 0x55c6e1445e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:20.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.634+0000 7f9750cef640 1 --2- 172.21.15.86:0/2380550864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e143a580 0x55c6e143a960 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f9744014ef0 tx=0x7f97440117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60a6e86dd930e389 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:20.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.634+0000 7f9750cef640 1 --2- 172.21.15.86:0/1617016414 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f97340a3d20 0x7f97340a61e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9744011cb0 tx=0x7f9744016000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:20.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.634+0000 7f97327fc640 1 -- 172.21.15.86:0/2380550864 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9744024070 con 0x55c6e143a580 2024-04-21T08:45:20.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.635+0000 7f97327fc640 1 -- 172.21.15.86:0/2380550864 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9744009c80 con 0x55c6e143a580 2024-04-21T08:45:20.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.635+0000 7f97327fc640 1 -- 172.21.15.86:0/2380550864 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9744004020 con 0x55c6e143a580 2024-04-21T08:45:20.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.635+0000 7f9755429740 1 -- 172.21.15.86:0/2380550864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e143a580 msgr2=0x55c6e143a960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:20.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.635+0000 7f9755429740 1 --2- 172.21.15.86:0/2380550864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e143a580 0x55c6e143a960 secure :-1 s=READY pgs=1379 cs=0 l=1 rev1=1 crypto rx=0x7f9744014ef0 tx=0x7f97440117b0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.635+0000 7f9755429740 1 -- 172.21.15.86:0/2380550864 shutdown_connections 2024-04-21T08:45:20.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.635+0000 7f9755429740 1 --2- 172.21.15.86:0/2380550864 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e1443a80 0x55c6e1445e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.635+0000 7f9755429740 1 --2- 172.21.15.86:0/2380550864 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6e143aea0 0x55c6e1443540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.635+0000 7f9755429740 1 --2- 172.21.15.86:0/2380550864 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e143a580 0x55c6e143a960 unknown :-1 s=CLOSED pgs=1379 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.635+0000 7f9755429740 1 -- 172.21.15.86:0/2380550864 >> 172.21.15.86:0/2380550864 conn(0x55c6e1434cb0 msgr2=0x55c6e1435090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:20.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.635+0000 7f9755429740 1 -- 172.21.15.86:0/2380550864 shutdown_connections 2024-04-21T08:45:20.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.635+0000 7f9755429740 1 -- 172.21.15.86:0/2380550864 wait complete. 2024-04-21T08:45:20.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.635+0000 7f9755429740 1 -- start start 2024-04-21T08:45:20.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.636+0000 7f9755429740 1 Processor -- start 2024-04-21T08:45:20.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.636+0000 7f9755429740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6e143a580 0x55c6e1440b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:20.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.636+0000 7f9755429740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e143aea0 0x55c6e1441060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:20.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.636+0000 7f9755429740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e1443a80 0x55c6e14415a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:20.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.636+0000 7f9755429740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c6e1446810 con 0x55c6e1443a80 2024-04-21T08:45:20.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.636+0000 7f9755429740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c6e13582f0 con 0x55c6e143a580 2024-04-21T08:45:20.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.636+0000 7f9755429740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c6e1446990 con 0x55c6e143aea0 2024-04-21T08:45:20.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.636+0000 7f9755429740 10 client.0 _ll_register_callbacks cb 0x55c6e1365580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:20.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.636+0000 7f9750cef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e1443a80 0x55c6e14415a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:20.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.636+0000 7f9750cef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e1443a80 0x55c6e14415a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47666/0 (socket says 172.21.15.86:47666) 2024-04-21T08:45:20.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.636+0000 7f9750cef640 1 -- 172.21.15.86:0/2754633512 learned_addr learned my addr 172.21.15.86:0/2754633512 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:20.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.636+0000 7f974bfff640 1 --2- 172.21.15.86:0/2754633512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6e143a580 0x55c6e1440b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:20.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.636+0000 7f974b7fe640 1 --2- 172.21.15.86:0/2754633512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e143aea0 0x55c6e1441060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:20.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.636+0000 7f9750cef640 1 -- 172.21.15.86:0/2754633512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e143aea0 msgr2=0x55c6e1441060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:20.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.636+0000 7f9750cef640 1 --2- 172.21.15.86:0/2754633512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e143aea0 0x55c6e1441060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.636+0000 7f9750cef640 1 -- 172.21.15.86:0/2754633512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6e143a580 msgr2=0x55c6e1440b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:20.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.636+0000 7f9750cef640 1 --2- 172.21.15.86:0/2754633512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6e143a580 0x55c6e1440b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.636+0000 7f9750cef640 1 -- 172.21.15.86:0/2754633512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6e10b13f0 con 0x55c6e1443a80 2024-04-21T08:45:20.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.636+0000 7f974bfff640 1 --2- 172.21.15.86:0/2754633512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6e143a580 0x55c6e1440b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:20.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.637+0000 7f9750cef640 1 --2- 172.21.15.86:0/2754633512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e1443a80 0x55c6e14415a0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f9744006010 tx=0x7f9744005a70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c1ea75b7081ea19 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:20.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.637+0000 7f972b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:20.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.637+0000 7f972b7fe640 1 -- 172.21.15.86:0/2754633512 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9744024040 con 0x55c6e1443a80 2024-04-21T08:45:20.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.637+0000 7f972b7fe640 1 -- 172.21.15.86:0/2754633512 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f974400f650 con 0x55c6e1443a80 2024-04-21T08:45:20.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.637+0000 7f9755429740 1 -- 172.21.15.86:0/2754633512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55c6e13acc00 con 0x55c6e1443a80 2024-04-21T08:45:20.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.637+0000 7f972b7fe640 1 -- 172.21.15.86:0/2754633512 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f974400f850 con 0x55c6e1443a80 2024-04-21T08:45:20.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.637+0000 7f9755429740 10 client.18318 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:20.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.637+0000 7f9755429740 1 -- 172.21.15.86:0/2754633512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c6e10710f0 con 0x55c6e1443a80 2024-04-21T08:45:20.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.637+0000 7f972b7fe640 1 -- 172.21.15.86:0/2754633512 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55c6e13acc00 con 0x55c6e1443a80 2024-04-21T08:45:20.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.637+0000 7f972b7fe640 1 -- 172.21.15.86:0/2754633512 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f97440209d0 con 0x55c6e1443a80 2024-04-21T08:45:20.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.637+0000 7f9755429740 10 client.18318 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:20.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.637+0000 7f9755429740 10 client.18318 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:20.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.637+0000 7f9755429740 20 client.18318 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:20.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.637+0000 7f9755429740 1 --2- 172.21.15.86:0/2754633512 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c6e1453aa0 0x55c6e1455e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:20.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.637+0000 7f9755429740 4 client.18318 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:20.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.637+0000 7f9755429740 1 -- 172.21.15.86:0/2754633512 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c6e13a6a70 con 0x55c6e1453aa0 2024-04-21T08:45:20.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.638+0000 7f974bfff640 1 --2- 172.21.15.86:0/2754633512 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c6e1453aa0 0x55c6e1455e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:20.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.638+0000 7f972b7fe640 10 client.18318 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:20.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.638+0000 7f974bfff640 1 --2- 172.21.15.86:0/2754633512 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c6e1453aa0 0x55c6e1455e90 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fe3884b26846ab0 server_cookie=4f854e93b3bf894d in_seq=0 out_seq=0 2024-04-21T08:45:20.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.639+0000 7f972b7fe640 1 -- 172.21.15.86:0/2754633512 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c6e13a6a70 con 0x55c6e1453aa0 2024-04-21T08:45:20.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.639+0000 7f972b7fe640 10 client.18318 handle_command_reply: tid=0 2024-04-21T08:45:20.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.639+0000 7f9755429740 1 client.18318 shutdown 2024-04-21T08:45:20.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.639+0000 7f9755429740 1 -- 172.21.15.86:0/2754633512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e1443a80 msgr2=0x55c6e14415a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:20.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.639+0000 7f9755429740 1 --2- 172.21.15.86:0/2754633512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e1443a80 0x55c6e14415a0 secure :-1 s=READY pgs=1380 cs=0 l=1 rev1=1 crypto rx=0x7f9744006010 tx=0x7f9744005a70 comp rx=0 tx=0).stop 2024-04-21T08:45:20.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.639+0000 7f9755429740 1 -- 172.21.15.86:0/2754633512 shutdown_connections 2024-04-21T08:45:20.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.639+0000 7f9755429740 1 --2- 172.21.15.86:0/2754633512 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c6e1453aa0 0x55c6e1455e90 crc :-1 s=READY pgs=205 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.639+0000 7f9755429740 1 --2- 172.21.15.86:0/2754633512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e1443a80 0x55c6e14415a0 unknown :-1 s=CLOSED pgs=1380 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.639+0000 7f9755429740 1 --2- 172.21.15.86:0/2754633512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e143aea0 0x55c6e1441060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.639+0000 7f972b7fe640 0 client.18318 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:20.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.639+0000 7f9755429740 1 --2- 172.21.15.86:0/2754633512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6e143a580 0x55c6e1440b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.639+0000 7f9755429740 1 -- 172.21.15.86:0/2754633512 >> 172.21.15.86:0/2754633512 conn(0x55c6e1434cb0 msgr2=0x55c6e1436a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:20.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.639+0000 7f9755429740 1 -- 172.21.15.86:0/2754633512 shutdown_connections 2024-04-21T08:45:20.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.639+0000 7f9755429740 1 -- 172.21.15.86:0/2754633512 wait complete. 2024-04-21T08:45:20.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.639+0000 7f9755429740 20 client.18318 trim_cache size 0 max 16384 2024-04-21T08:45:20.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.656+0000 7f9755429740 1 Processor -- start 2024-04-21T08:45:20.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.656+0000 7f9755429740 1 -- start start 2024-04-21T08:45:20.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.657+0000 7f9755429740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e143a580 0x55c6e1443250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:20.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.657+0000 7f9755429740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6e143aea0 0x55c6e144f700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:20.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.657+0000 7f9755429740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e1443a80 0x55c6e1433bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:20.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.657+0000 7f9755429740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f974c1b1590 con 0x55c6e143a580 2024-04-21T08:45:20.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.657+0000 7f9755429740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f974c1b30d0 con 0x55c6e143aea0 2024-04-21T08:45:20.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.657+0000 7f9755429740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f974c1b1c60 con 0x55c6e1443a80 2024-04-21T08:45:20.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.657+0000 7f974bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e143a580 0x55c6e1443250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:20.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.657+0000 7f974bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e143a580 0x55c6e1443250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47670/0 (socket says 172.21.15.86:47670) 2024-04-21T08:45:20.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.657+0000 7f974bfff640 1 -- 172.21.15.86:0/1686894407 learned_addr learned my addr 172.21.15.86:0/1686894407 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:20.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.657+0000 7f974bfff640 1 -- 172.21.15.86:0/1686894407 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e1443a80 msgr2=0x55c6e1433bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:20.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.657+0000 7f974bfff640 1 --2- 172.21.15.86:0/1686894407 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e1443a80 0x55c6e1433bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.657+0000 7f974bfff640 1 -- 172.21.15.86:0/1686894407 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6e143aea0 msgr2=0x55c6e144f700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:20.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.657+0000 7f974bfff640 1 --2- 172.21.15.86:0/1686894407 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6e143aea0 0x55c6e144f700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.657+0000 7f974bfff640 1 -- 172.21.15.86:0/1686894407 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c6e1456200 con 0x55c6e143a580 2024-04-21T08:45:20.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.657+0000 7f974bfff640 1 --2- 172.21.15.86:0/1686894407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e143a580 0x55c6e1443250 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x55c6e1456400 tx=0x7f973c005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f496ae54cec7b098 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:20.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.657+0000 7f9731ffb640 1 -- 172.21.15.86:0/1686894407 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c01f070 con 0x55c6e143a580 2024-04-21T08:45:20.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.657+0000 7f9731ffb640 1 -- 172.21.15.86:0/1686894407 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f973c01bdc0 con 0x55c6e143a580 2024-04-21T08:45:20.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.657+0000 7f9731ffb640 1 -- 172.21.15.86:0/1686894407 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f973c011b70 con 0x55c6e143a580 2024-04-21T08:45:20.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.658+0000 7f9755429740 1 -- 172.21.15.86:0/1686894407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e143a580 msgr2=0x55c6e1443250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:20.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.658+0000 7f9755429740 1 --2- 172.21.15.86:0/1686894407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e143a580 0x55c6e1443250 secure :-1 s=READY pgs=1381 cs=0 l=1 rev1=1 crypto rx=0x55c6e1456400 tx=0x7f973c005f30 comp rx=0 tx=0).stop 2024-04-21T08:45:20.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.658+0000 7f9755429740 1 -- 172.21.15.86:0/1686894407 shutdown_connections 2024-04-21T08:45:20.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.658+0000 7f9755429740 1 --2- 172.21.15.86:0/1686894407 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e1443a80 0x55c6e1433bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.658+0000 7f9755429740 1 --2- 172.21.15.86:0/1686894407 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6e143aea0 0x55c6e144f700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.658+0000 7f9755429740 1 --2- 172.21.15.86:0/1686894407 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e143a580 0x55c6e1443250 unknown :-1 s=CLOSED pgs=1381 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.658+0000 7f9755429740 1 -- 172.21.15.86:0/1686894407 >> 172.21.15.86:0/1686894407 conn(0x55c6e1434cb0 msgr2=0x55c6e14334e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:20.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.658+0000 7f9755429740 1 -- 172.21.15.86:0/1686894407 shutdown_connections 2024-04-21T08:45:20.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.658+0000 7f9755429740 1 -- 172.21.15.86:0/1686894407 wait complete. 2024-04-21T08:45:20.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.658+0000 7f9755429740 1 -- start start 2024-04-21T08:45:20.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.659+0000 7f9755429740 1 Processor -- start 2024-04-21T08:45:20.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.659+0000 7f9755429740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6e143a580 0x55c6e143d650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:20.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.659+0000 7f9755429740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e143aea0 0x55c6e143db90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:20.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.659+0000 7f9755429740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e1443a80 0x55c6e143e0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:20.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.659+0000 7f9755429740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f974c1b2a00 con 0x55c6e1443a80 2024-04-21T08:45:20.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.659+0000 7f9755429740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f974c1b2330 con 0x55c6e143a580 2024-04-21T08:45:20.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.659+0000 7f9755429740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f974c1af590 con 0x55c6e143aea0 2024-04-21T08:45:20.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.659+0000 7f9755429740 10 client.0 _ll_register_callbacks cb 0x7f9744014cc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:20.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.659+0000 7f9750cef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e1443a80 0x55c6e143e0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:20.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.659+0000 7f9750cef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e1443a80 0x55c6e143e0d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47672/0 (socket says 172.21.15.86:47672) 2024-04-21T08:45:20.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.659+0000 7f9750cef640 1 -- 172.21.15.86:0/3641610520 learned_addr learned my addr 172.21.15.86:0/3641610520 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:20.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.659+0000 7f974bfff640 1 --2- 172.21.15.86:0/3641610520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6e143a580 0x55c6e143d650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:20.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.659+0000 7f974b7fe640 1 --2- 172.21.15.86:0/3641610520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e143aea0 0x55c6e143db90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:20.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.659+0000 7f9750cef640 1 -- 172.21.15.86:0/3641610520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e143aea0 msgr2=0x55c6e143db90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:20.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.659+0000 7f9750cef640 1 --2- 172.21.15.86:0/3641610520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e143aea0 0x55c6e143db90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.659+0000 7f9750cef640 1 -- 172.21.15.86:0/3641610520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6e143a580 msgr2=0x55c6e143d650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:20.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.659+0000 7f9750cef640 1 --2- 172.21.15.86:0/3641610520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6e143a580 0x55c6e143d650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.659+0000 7f9750cef640 1 -- 172.21.15.86:0/3641610520 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c6e14b8d00 con 0x55c6e1443a80 2024-04-21T08:45:20.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.659+0000 7f9750cef640 1 --2- 172.21.15.86:0/3641610520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e1443a80 0x55c6e143e0d0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f9744002230 tx=0x7f974400f6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e594c48f7d9c8e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:20.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.659+0000 7f972b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:20.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.660+0000 7f972b7fe640 1 -- 172.21.15.86:0/3641610520 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9744024070 con 0x55c6e1443a80 2024-04-21T08:45:20.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.660+0000 7f972b7fe640 1 -- 172.21.15.86:0/3641610520 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f974400fe00 con 0x55c6e1443a80 2024-04-21T08:45:20.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.660+0000 7f9755429740 1 -- 172.21.15.86:0/3641610520 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f973c01bdc0 con 0x55c6e1443a80 2024-04-21T08:45:20.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.660+0000 7f972b7fe640 1 -- 172.21.15.86:0/3641610520 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f974404a050 con 0x55c6e1443a80 2024-04-21T08:45:20.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.660+0000 7f9755429740 10 client.18324 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:20.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.660+0000 7f9755429740 1 -- 172.21.15.86:0/3641610520 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c6e14b9e50 con 0x55c6e1443a80 2024-04-21T08:45:20.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.660+0000 7f972b7fe640 1 -- 172.21.15.86:0/3641610520 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f973c01bdc0 con 0x55c6e1443a80 2024-04-21T08:45:20.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.660+0000 7f972b7fe640 1 -- 172.21.15.86:0/3641610520 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9744028840 con 0x55c6e1443a80 2024-04-21T08:45:20.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.660+0000 7f9755429740 10 client.18324 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:20.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.660+0000 7f9755429740 10 client.18324 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:20.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.660+0000 7f9755429740 20 client.18324 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:20.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.660+0000 7f9755429740 1 --2- 172.21.15.86:0/3641610520 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c6e14ba500 0x55c6e14bc920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:20.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.660+0000 7f9755429740 4 client.18324 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:45:20.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.660+0000 7f9755429740 1 -- 172.21.15.86:0/3641610520 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55c6e1322590 con 0x55c6e14ba500 2024-04-21T08:45:20.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.660+0000 7f974bfff640 1 --2- 172.21.15.86:0/3641610520 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c6e14ba500 0x55c6e14bc920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:20.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.661+0000 7f972b7fe640 10 client.18324 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:20.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.661+0000 7f974bfff640 1 --2- 172.21.15.86:0/3641610520 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c6e14ba500 0x55c6e14bc920 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=57de56f42cc5a0e8 server_cookie=b1c0a59cf671f9e7 in_seq=0 out_seq=0 2024-04-21T08:45:20.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.661+0000 7f972b7fe640 1 -- 172.21.15.86:0/3641610520 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55c6e1322590 con 0x55c6e14ba500 2024-04-21T08:45:20.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.661+0000 7f972b7fe640 10 client.18324 handle_command_reply: tid=0 2024-04-21T08:45:20.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.661+0000 7f9755429740 1 client.18324 shutdown 2024-04-21T08:45:20.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.661+0000 7f9755429740 1 -- 172.21.15.86:0/3641610520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e1443a80 msgr2=0x55c6e143e0d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:20.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.661+0000 7f9755429740 1 --2- 172.21.15.86:0/3641610520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e1443a80 0x55c6e143e0d0 secure :-1 s=READY pgs=1382 cs=0 l=1 rev1=1 crypto rx=0x7f9744002230 tx=0x7f974400f6e0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.661+0000 7f9755429740 1 -- 172.21.15.86:0/3641610520 shutdown_connections 2024-04-21T08:45:20.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.661+0000 7f9755429740 1 --2- 172.21.15.86:0/3641610520 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c6e14ba500 0x55c6e14bc920 crc :-1 s=READY pgs=206 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.661+0000 7f9755429740 1 --2- 172.21.15.86:0/3641610520 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c6e1443a80 0x55c6e143e0d0 unknown :-1 s=CLOSED pgs=1382 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.661+0000 7f972b7fe640 0 client.18324 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:20.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.661+0000 7f9755429740 1 --2- 172.21.15.86:0/3641610520 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c6e143aea0 0x55c6e143db90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.661+0000 7f9755429740 1 --2- 172.21.15.86:0/3641610520 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c6e143a580 0x55c6e143d650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.661+0000 7f9755429740 1 -- 172.21.15.86:0/3641610520 >> 172.21.15.86:0/3641610520 conn(0x55c6e1434cb0 msgr2=0x55c6e1433d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:20.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.661+0000 7f9755429740 1 -- 172.21.15.86:0/3641610520 shutdown_connections 2024-04-21T08:45:20.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.661+0000 7f9755429740 1 -- 172.21.15.86:0/3641610520 wait complete. 2024-04-21T08:45:20.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.661+0000 7f9755429740 20 client.18324 trim_cache size 0 max 16384 2024-04-21T08:45:20.670 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:45:20.670 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-21T08:45:20.670 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-21T08:45:20.670 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:45:20.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.662+0000 7f9752752640 1 -- 172.21.15.86:0/1617016414 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f97340a3d20 msgr2=0x7f97340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:20.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.662+0000 7f9752752640 1 --2- 172.21.15.86:0/1617016414 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f97340a3d20 0x7f97340a61e0 secure :-1 s=READY pgs=219 cs=0 l=1 rev1=1 crypto rx=0x7f9744011cb0 tx=0x7f9744016000 comp rx=0 tx=0).stop 2024-04-21T08:45:20.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.662+0000 7f9752752640 1 -- 172.21.15.86:0/1617016414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f974c101650 msgr2=0x7f974c1a0bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:20.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.662+0000 7f9752752640 1 --2- 172.21.15.86:0/1617016414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f974c101650 0x7f974c1a0bb0 secure :-1 s=READY pgs=1378 cs=0 l=1 rev1=1 crypto rx=0x7f9740005940 tx=0x7f974002c7e0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.662+0000 7f9752752640 1 -- 172.21.15.86:0/1617016414 shutdown_connections 2024-04-21T08:45:20.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.662+0000 7f9752752640 1 --2- 172.21.15.86:0/1617016414 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f97340a3d20 0x7f97340a61e0 unknown :-1 s=CLOSED pgs=219 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.662+0000 7f9752752640 1 --2- 172.21.15.86:0/1617016414 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f974c10e8e0 0x7f974c19a7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.662+0000 7f9752752640 1 --2- 172.21.15.86:0/1617016414 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f974c101650 0x7f974c1a0bb0 unknown :-1 s=CLOSED pgs=1378 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.662+0000 7f9752752640 1 --2- 172.21.15.86:0/1617016414 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f974c100d30 0x7f974c1a0670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.662+0000 7f9752752640 1 -- 172.21.15.86:0/1617016414 >> 172.21.15.86:0/1617016414 conn(0x7f974c0771d0 msgr2=0x7f974c10ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:20.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.662+0000 7f9752752640 1 -- 172.21.15.86:0/1617016414 shutdown_connections 2024-04-21T08:45:20.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.662+0000 7f9752752640 1 -- 172.21.15.86:0/1617016414 wait complete. 2024-04-21T08:45:20.673 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:45:20.673 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:45:20.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.771+0000 7fe8ae396640 1 -- 172.21.15.86:0/3293851808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a8101b60 msgr2=0x7fe8a810dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:20.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.771+0000 7fe8ae396640 1 --2- 172.21.15.86:0/3293851808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a8101b60 0x7fe8a810dc90 secure :-1 s=READY pgs=1383 cs=0 l=1 rev1=1 crypto rx=0x7fe8a0009cd0 tx=0x7fe8a002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.771+0000 7fe8ae396640 1 -- 172.21.15.86:0/3293851808 shutdown_connections 2024-04-21T08:45:20.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.771+0000 7fe8ae396640 1 --2- 172.21.15.86:0/3293851808 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a8101b60 0x7fe8a810dc90 unknown :-1 s=CLOSED pgs=1383 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.771+0000 7fe8ae396640 1 --2- 172.21.15.86:0/3293851808 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8a81011a0 0x7fe8a8101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.771+0000 7fe8ae396640 1 --2- 172.21.15.86:0/3293851808 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8a8105e30 0x7fe8a8106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.771+0000 7fe8ae396640 1 -- 172.21.15.86:0/3293851808 >> 172.21.15.86:0/3293851808 conn(0x7fe8a8077e30 msgr2=0x7fe8a8078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:20.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.771+0000 7fe8ae396640 1 -- 172.21.15.86:0/3293851808 shutdown_connections 2024-04-21T08:45:20.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.771+0000 7fe8ae396640 1 -- 172.21.15.86:0/3293851808 wait complete. 2024-04-21T08:45:20.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.771+0000 7fe8ae396640 1 Processor -- start 2024-04-21T08:45:20.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.771+0000 7fe8ae396640 1 -- start start 2024-04-21T08:45:20.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.772+0000 7fe8ae396640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a81011a0 0x7fe8a81975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:20.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.772+0000 7fe8ae396640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8a8101b60 0x7fe8a8197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:20.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.772+0000 7fe8ae396640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8a8105e30 0x7fe8a819be80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:20.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.772+0000 7fe8ae396640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a81110c0 con 0x7fe8a81011a0 2024-04-21T08:45:20.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.772+0000 7fe8ae396640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a8110f40 con 0x7fe8a8101b60 2024-04-21T08:45:20.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.772+0000 7fe8ae396640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe8a8111240 con 0x7fe8a8105e30 2024-04-21T08:45:20.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.772+0000 7fe8a7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a81011a0 0x7fe8a81975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:20.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.772+0000 7fe8a7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a81011a0 0x7fe8a81975c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47690/0 (socket says 172.21.15.86:47690) 2024-04-21T08:45:20.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.772+0000 7fe8a7fff640 1 -- 172.21.15.86:0/719136826 learned_addr learned my addr 172.21.15.86:0/719136826 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:20.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.772+0000 7fe8ac933640 1 --2- 172.21.15.86:0/719136826 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8a8105e30 0x7fe8a819be80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:20.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.772+0000 7fe8a77fe640 1 --2- 172.21.15.86:0/719136826 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8a8101b60 0x7fe8a8197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:20.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.772+0000 7fe8a7fff640 1 -- 172.21.15.86:0/719136826 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8a8105e30 msgr2=0x7fe8a819be80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:20.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.772+0000 7fe8a7fff640 1 --2- 172.21.15.86:0/719136826 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8a8105e30 0x7fe8a819be80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.772+0000 7fe8a7fff640 1 -- 172.21.15.86:0/719136826 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8a8101b60 msgr2=0x7fe8a8197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:20.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.772+0000 7fe8a7fff640 1 --2- 172.21.15.86:0/719136826 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8a8101b60 0x7fe8a8197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:20.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.772+0000 7fe8a7fff640 1 -- 172.21.15.86:0/719136826 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe8a819c5b0 con 0x7fe8a81011a0 2024-04-21T08:45:20.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.772+0000 7fe8a77fe640 1 --2- 172.21.15.86:0/719136826 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8a8101b60 0x7fe8a8197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:20.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.772+0000 7fe8a7fff640 1 --2- 172.21.15.86:0/719136826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a81011a0 0x7fe8a81975c0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fe89800d950 tx=0x7fe89800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:20.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.773+0000 7fe8a57fa640 1 -- 172.21.15.86:0/719136826 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe898017070 con 0x7fe8a81011a0 2024-04-21T08:45:20.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.773+0000 7fe8ae396640 1 -- 172.21.15.86:0/719136826 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe8a819c8a0 con 0x7fe8a81011a0 2024-04-21T08:45:20.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.773+0000 7fe8a57fa640 1 -- 172.21.15.86:0/719136826 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe898009c30 con 0x7fe8a81011a0 2024-04-21T08:45:20.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.773+0000 7fe8a57fa640 1 -- 172.21.15.86:0/719136826 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe89800f620 con 0x7fe8a81011a0 2024-04-21T08:45:20.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.773+0000 7fe8ae396640 1 -- 172.21.15.86:0/719136826 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe8a81a4130 con 0x7fe8a81011a0 2024-04-21T08:45:20.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.774+0000 7fe8ae396640 1 -- 172.21.15.86:0/719136826 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe8a8068de0 con 0x7fe8a81011a0 2024-04-21T08:45:20.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.775+0000 7fe8a57fa640 1 -- 172.21.15.86:0/719136826 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe898010460 con 0x7fe8a81011a0 2024-04-21T08:45:20.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.775+0000 7fe8a57fa640 1 --2- 172.21.15.86:0/719136826 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe8900a3d20 0x7fe8900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:20.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.775+0000 7fe8a77fe640 1 --2- 172.21.15.86:0/719136826 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe8900a3d20 0x7fe8900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:20.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.775+0000 7fe8a57fa640 1 -- 172.21.15.86:0/719136826 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7fe8980d0830 con 0x7fe8a81011a0 2024-04-21T08:45:20.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.776+0000 7fe8a77fe640 1 --2- 172.21.15.86:0/719136826 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe8900a3d20 0x7fe8900a61e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8100d00 tx=0x7fe89c005ec0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:20.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.780+0000 7fe8a57fa640 1 -- 172.21.15.86:0/719136826 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe89809cb00 con 0x7fe8a81011a0 2024-04-21T08:45:20.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.997+0000 7fe8ae396640 1 -- 172.21.15.86:0/719136826 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fe8a8198910 con 0x7fe8a81011a0 2024-04-21T08:45:20.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:20.998+0000 7fe8a57fa640 1 -- 172.21.15.86:0/719136826 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7fe898009de0 con 0x7fe8a81011a0 2024-04-21T08:45:20.999 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:45:20.999 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:38:45.202126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24398,24394,24455,24472]},"id":1}]} 2024-04-21T08:45:20.999 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-21T08:45:21.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.001+0000 7fe8ae396640 1 -- 172.21.15.86:0/719136826 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe8900a3d20 msgr2=0x7fe8900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.001+0000 7fe8ae396640 1 --2- 172.21.15.86:0/719136826 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe8900a3d20 0x7fe8900a61e0 secure :-1 s=READY pgs=220 cs=0 l=1 rev1=1 crypto rx=0x7fe8a8100d00 tx=0x7fe89c005ec0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.001+0000 7fe8ae396640 1 -- 172.21.15.86:0/719136826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a81011a0 msgr2=0x7fe8a81975c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.001+0000 7fe8ae396640 1 --2- 172.21.15.86:0/719136826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a81011a0 0x7fe8a81975c0 secure :-1 s=READY pgs=1384 cs=0 l=1 rev1=1 crypto rx=0x7fe89800d950 tx=0x7fe89800de10 comp rx=0 tx=0).stop 2024-04-21T08:45:21.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.001+0000 7fe8ae396640 1 -- 172.21.15.86:0/719136826 shutdown_connections 2024-04-21T08:45:21.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.001+0000 7fe8ae396640 1 --2- 172.21.15.86:0/719136826 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe8900a3d20 0x7fe8900a61e0 unknown :-1 s=CLOSED pgs=220 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.001+0000 7fe8ae396640 1 --2- 172.21.15.86:0/719136826 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe8a8105e30 0x7fe8a819be80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.001+0000 7fe8ae396640 1 --2- 172.21.15.86:0/719136826 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe8a8101b60 0x7fe8a8197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.001+0000 7fe8ae396640 1 --2- 172.21.15.86:0/719136826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe8a81011a0 0x7fe8a81975c0 unknown :-1 s=CLOSED pgs=1384 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.001+0000 7fe8ae396640 1 -- 172.21.15.86:0/719136826 >> 172.21.15.86:0/719136826 conn(0x7fe8a8077e30 msgr2=0x7fe8a80fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:21.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.001+0000 7fe8ae396640 1 -- 172.21.15.86:0/719136826 shutdown_connections 2024-04-21T08:45:21.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.001+0000 7fe8ae396640 1 -- 172.21.15.86:0/719136826 wait complete. 2024-04-21T08:45:21.013 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:45:21.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.112+0000 7fad22323640 1 -- 172.21.15.86:0/3282174648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad1c1072b0 msgr2=0x7fad1c10eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.112+0000 7fad22323640 1 --2- 172.21.15.86:0/3282174648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad1c1072b0 0x7fad1c10eec0 secure :-1 s=READY pgs=1385 cs=0 l=1 rev1=1 crypto rx=0x7fad14009ca0 tx=0x7fad1402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.112+0000 7fad22323640 1 -- 172.21.15.86:0/3282174648 shutdown_connections 2024-04-21T08:45:21.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.112+0000 7fad22323640 1 --2- 172.21.15.86:0/3282174648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad1c1072b0 0x7fad1c10eec0 unknown :-1 s=CLOSED pgs=1385 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.112+0000 7fad22323640 1 --2- 172.21.15.86:0/3282174648 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad1c1014a0 0x7fad1c106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.112+0000 7fad22323640 1 --2- 172.21.15.86:0/3282174648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad1c100af0 0x7fad1c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.112+0000 7fad22323640 1 -- 172.21.15.86:0/3282174648 >> 172.21.15.86:0/3282174648 conn(0x7fad1c0760e0 msgr2=0x7fad1c0764f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:21.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.112+0000 7fad22323640 1 -- 172.21.15.86:0/3282174648 shutdown_connections 2024-04-21T08:45:21.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.112+0000 7fad22323640 1 -- 172.21.15.86:0/3282174648 wait complete. 2024-04-21T08:45:21.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.113+0000 7fad22323640 1 Processor -- start 2024-04-21T08:45:21.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.113+0000 7fad22323640 1 -- start start 2024-04-21T08:45:21.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.113+0000 7fad22323640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad1c100af0 0x7fad1c198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.113+0000 7fad22323640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad1c1072b0 0x7fad1c198bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.113+0000 7fad22323640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad1c19cf30 0x7fad1c19d3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.113+0000 7fad22323640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fad1c112230 con 0x7fad1c1072b0 2024-04-21T08:45:21.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.113+0000 7fad22323640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fad1c1120b0 con 0x7fad1c19cf30 2024-04-21T08:45:21.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.113+0000 7fad22323640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fad1c1123b0 con 0x7fad1c100af0 2024-04-21T08:45:21.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.113+0000 7fad1b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad1c1072b0 0x7fad1c198bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.113+0000 7fad1b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad1c1072b0 0x7fad1c198bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47706/0 (socket says 172.21.15.86:47706) 2024-04-21T08:45:21.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.113+0000 7fad1b7fe640 1 -- 172.21.15.86:0/1411070798 learned_addr learned my addr 172.21.15.86:0/1411070798 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:21.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.113+0000 7fad1bfff640 1 --2- 172.21.15.86:0/1411070798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad1c100af0 0x7fad1c198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.113+0000 7fad208c0640 1 --2- 172.21.15.86:0/1411070798 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad1c19cf30 0x7fad1c19d3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.114+0000 7fad1b7fe640 1 -- 172.21.15.86:0/1411070798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad1c100af0 msgr2=0x7fad1c198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.114+0000 7fad1b7fe640 1 --2- 172.21.15.86:0/1411070798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad1c100af0 0x7fad1c198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.114+0000 7fad1b7fe640 1 -- 172.21.15.86:0/1411070798 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad1c19cf30 msgr2=0x7fad1c19d3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.114+0000 7fad1b7fe640 1 --2- 172.21.15.86:0/1411070798 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad1c19cf30 0x7fad1c19d3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.114+0000 7fad1b7fe640 1 -- 172.21.15.86:0/1411070798 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad1c19da20 con 0x7fad1c1072b0 2024-04-21T08:45:21.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.114+0000 7fad1b7fe640 1 --2- 172.21.15.86:0/1411070798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad1c1072b0 0x7fad1c198bb0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fad1000c970 tx=0x7fad1000cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:21.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.114+0000 7fad197fa640 1 -- 172.21.15.86:0/1411070798 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad10014070 con 0x7fad1c1072b0 2024-04-21T08:45:21.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.114+0000 7fad22323640 1 -- 172.21.15.86:0/1411070798 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad1c19dd10 con 0x7fad1c1072b0 2024-04-21T08:45:21.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.114+0000 7fad197fa640 1 -- 172.21.15.86:0/1411070798 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad10004440 con 0x7fad1c1072b0 2024-04-21T08:45:21.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.114+0000 7fad197fa640 1 -- 172.21.15.86:0/1411070798 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad10003d30 con 0x7fad1c1072b0 2024-04-21T08:45:21.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.114+0000 7fad22323640 1 -- 172.21.15.86:0/1411070798 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad1c1a5550 con 0x7fad1c1072b0 2024-04-21T08:45:21.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.116+0000 7fad197fa640 1 -- 172.21.15.86:0/1411070798 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fad10017070 con 0x7fad1c1072b0 2024-04-21T08:45:21.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.116+0000 7fad24fee740 1 Processor -- start 2024-04-21T08:45:21.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.116+0000 7fad24fee740 1 -- start start 2024-04-21T08:45:21.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.116+0000 7fad24fee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a52cd94d0 0x561a52cd98b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.116+0000 7fad24fee740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561a52cd9e80 0x561a52ce2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.116+0000 7fad24fee740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a52ce2a00 0x561a52ce4e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.116+0000 7fad24fee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561a52c3e1f0 con 0x561a52cd94d0 2024-04-21T08:45:21.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.116+0000 7fad24fee740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x561a52bdb4b0 con 0x561a52cd9e80 2024-04-21T08:45:21.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.116+0000 7fad24fee740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561a52c3a1b0 con 0x561a52ce2a00 2024-04-21T08:45:21.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.116+0000 7fad197fa640 1 --2- 172.21.15.86:0/1411070798 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fad040a3d20 0x7fad040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.116+0000 7fad208c0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a52cd94d0 0x561a52cd98b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.117+0000 7fad208c0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a52cd94d0 0x561a52cd98b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47708/0 (socket says 172.21.15.86:47708) 2024-04-21T08:45:21.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.117+0000 7fad208c0640 1 -- 172.21.15.86:0/3722300400 learned_addr learned my addr 172.21.15.86:0/3722300400 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:21.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.117+0000 7fad208c0640 1 --2- 172.21.15.86:0/1411070798 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fad040a3d20 0x7fad040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.117+0000 7fad1bfff640 1 --2- 172.21.15.86:0/3722300400 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561a52cd9e80 0x561a52ce2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.117+0000 7fad197fa640 1 -- 172.21.15.86:0/1411070798 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7fad100d08d0 con 0x7fad1c1072b0 2024-04-21T08:45:21.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.117+0000 7fad1b7fe640 1 --2- 172.21.15.86:0/3722300400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a52ce2a00 0x561a52ce4e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.117+0000 7fad208c0640 1 -- 172.21.15.86:0/3722300400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a52ce2a00 msgr2=0x561a52ce4e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.117+0000 7fad208c0640 1 --2- 172.21.15.86:0/3722300400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a52ce2a00 0x561a52ce4e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.117+0000 7fad208c0640 1 -- 172.21.15.86:0/3722300400 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561a52cd9e80 msgr2=0x561a52ce2430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.117+0000 7fad208c0640 1 --2- 172.21.15.86:0/3722300400 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561a52cd9e80 0x561a52ce2430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.117+0000 7fad208c0640 1 -- 172.21.15.86:0/3722300400 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a52bcd1b0 con 0x561a52cd94d0 2024-04-21T08:45:21.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.117+0000 7fad1b7fe640 1 --2- 172.21.15.86:0/3722300400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a52ce2a00 0x561a52ce4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:21.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.117+0000 7fad208c0640 1 --2- 172.21.15.86:0/3722300400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a52cd94d0 0x561a52cd98b0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fad14009ca0 tx=0x7fad14032d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3a588f032ec8e0a9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:21.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.117+0000 7fad027fc640 1 -- 172.21.15.86:0/3722300400 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad14050070 con 0x561a52cd94d0 2024-04-21T08:45:21.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.117+0000 7fad027fc640 1 -- 172.21.15.86:0/3722300400 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad14003c30 con 0x561a52cd94d0 2024-04-21T08:45:21.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.117+0000 7fad027fc640 1 -- 172.21.15.86:0/3722300400 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad14039040 con 0x561a52cd94d0 2024-04-21T08:45:21.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.118+0000 7fad24fee740 1 -- 172.21.15.86:0/3722300400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a52cd94d0 msgr2=0x561a52cd98b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.118+0000 7fad208c0640 1 --2- 172.21.15.86:0/1411070798 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fad040a3d20 0x7fad040a61e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fad1404d000 tx=0x7fad14048000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:21.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.118+0000 7fad24fee740 1 --2- 172.21.15.86:0/3722300400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a52cd94d0 0x561a52cd98b0 secure :-1 s=READY pgs=1387 cs=0 l=1 rev1=1 crypto rx=0x7fad14009ca0 tx=0x7fad14032d70 comp rx=0 tx=0).stop 2024-04-21T08:45:21.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.118+0000 7fad24fee740 1 -- 172.21.15.86:0/3722300400 shutdown_connections 2024-04-21T08:45:21.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.118+0000 7fad24fee740 1 --2- 172.21.15.86:0/3722300400 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a52ce2a00 0x561a52ce4e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.118+0000 7fad24fee740 1 --2- 172.21.15.86:0/3722300400 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561a52cd9e80 0x561a52ce2430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.118+0000 7fad24fee740 1 --2- 172.21.15.86:0/3722300400 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a52cd94d0 0x561a52cd98b0 unknown :-1 s=CLOSED pgs=1387 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.118+0000 7fad24fee740 1 -- 172.21.15.86:0/3722300400 >> 172.21.15.86:0/3722300400 conn(0x561a52cd3ba0 msgr2=0x561a52cd3f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:21.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.118+0000 7fad24fee740 1 -- 172.21.15.86:0/3722300400 shutdown_connections 2024-04-21T08:45:21.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.118+0000 7fad24fee740 1 -- 172.21.15.86:0/3722300400 wait complete. 2024-04-21T08:45:21.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.118+0000 7fad24fee740 1 -- start start 2024-04-21T08:45:21.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.118+0000 7fad24fee740 1 Processor -- start 2024-04-21T08:45:21.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.119+0000 7fad24fee740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561a52cd94d0 0x561a52ce2330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.119+0000 7fad24fee740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a52cd9e80 0x561a52cdf6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.119+0000 7fad24fee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a52ce2a00 0x561a52cdfcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.119+0000 7fad24fee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561a52ce5970 con 0x561a52ce2a00 2024-04-21T08:45:21.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.119+0000 7fad24fee740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x561a52bf7500 con 0x561a52cd94d0 2024-04-21T08:45:21.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.119+0000 7fad24fee740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561a52ce5af0 con 0x561a52cd9e80 2024-04-21T08:45:21.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.119+0000 7fad24fee740 10 client.0 _ll_register_callbacks cb 0x561a52bc5b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:21.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.119+0000 7fad208c0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a52ce2a00 0x561a52cdfcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.119+0000 7fad208c0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a52ce2a00 0x561a52cdfcd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47716/0 (socket says 172.21.15.86:47716) 2024-04-21T08:45:21.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.119+0000 7fad208c0640 1 -- 172.21.15.86:0/1630936110 learned_addr learned my addr 172.21.15.86:0/1630936110 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:21.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.119+0000 7fad1bfff640 1 --2- 172.21.15.86:0/1630936110 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561a52cd94d0 0x561a52ce2330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.119+0000 7fad1b7fe640 1 --2- 172.21.15.86:0/1630936110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a52cd9e80 0x561a52cdf6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.119+0000 7fad208c0640 1 -- 172.21.15.86:0/1630936110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a52cd9e80 msgr2=0x561a52cdf6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.119+0000 7fad208c0640 1 --2- 172.21.15.86:0/1630936110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a52cd9e80 0x561a52cdf6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.119+0000 7fad208c0640 1 -- 172.21.15.86:0/1630936110 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561a52cd94d0 msgr2=0x561a52ce2330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.119+0000 7fad208c0640 1 --2- 172.21.15.86:0/1630936110 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561a52cd94d0 0x561a52ce2330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.119+0000 7fad208c0640 1 -- 172.21.15.86:0/1630936110 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a529503f0 con 0x561a52ce2a00 2024-04-21T08:45:21.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.119+0000 7fad1b7fe640 1 --2- 172.21.15.86:0/1630936110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a52cd9e80 0x561a52cdf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:21.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.119+0000 7fad208c0640 1 --2- 172.21.15.86:0/1630936110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a52ce2a00 0x561a52cdfcd0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fad140047f0 tx=0x7fad14005c30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3db0fa876587579 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:21.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.119+0000 7facfb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:21.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.119+0000 7facfb7fe640 1 -- 172.21.15.86:0/1630936110 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad14050030 con 0x561a52ce2a00 2024-04-21T08:45:21.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.119+0000 7facfb7fe640 1 -- 172.21.15.86:0/1630936110 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7fad14034440 con 0x561a52ce2a00 2024-04-21T08:45:21.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.119+0000 7fad24fee740 1 -- 172.21.15.86:0/1630936110 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad14003c30 con 0x561a52ce2a00 2024-04-21T08:45:21.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.119+0000 7facfb7fe640 1 -- 172.21.15.86:0/1630936110 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fad14062050 con 0x561a52ce2a00 2024-04-21T08:45:21.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.120+0000 7fad24fee740 10 client.18342 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:21.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.120+0000 7fad24fee740 1 -- 172.21.15.86:0/1630936110 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561a52a1a7d0 con 0x561a52ce2a00 2024-04-21T08:45:21.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.120+0000 7facfb7fe640 1 -- 172.21.15.86:0/1630936110 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad14003c30 con 0x561a52ce2a00 2024-04-21T08:45:21.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.120+0000 7facfb7fe640 1 -- 172.21.15.86:0/1630936110 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fad1403f070 con 0x561a52ce2a00 2024-04-21T08:45:21.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.120+0000 7fad24fee740 10 client.18342 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:21.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.120+0000 7fad24fee740 10 client.18342 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:45:21.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.120+0000 7fad24fee740 20 client.18342 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:21.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.120+0000 7fad24fee740 1 --2- 172.21.15.86:0/1630936110 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x561a52cf2a30 0x561a52cf4e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.120+0000 7fad24fee740 4 client.18342 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.120+0000 7fad24fee740 1 -- 172.21.15.86:0/1630936110 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561a52bdd930 con 0x561a52cf2a30 2024-04-21T08:45:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.120+0000 7fad1bfff640 1 --2- 172.21.15.86:0/1630936110 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x561a52cf2a30 0x561a52cf4e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.121+0000 7facfb7fe640 10 client.18342 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:45:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.121+0000 7fad1bfff640 1 --2- 172.21.15.86:0/1630936110 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x561a52cf2a30 0x561a52cf4e20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=43fcbccffbae5076 server_cookie=3e45e65645139bae in_seq=0 out_seq=0 2024-04-21T08:45:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.121+0000 7facfb7fe640 1 -- 172.21.15.86:0/1630936110 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561a52bdd930 con 0x561a52cf2a30 2024-04-21T08:45:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.121+0000 7facfb7fe640 10 client.18342 handle_command_reply: tid=0 2024-04-21T08:45:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.121+0000 7fad24fee740 1 client.18342 shutdown 2024-04-21T08:45:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.121+0000 7fad24fee740 1 -- 172.21.15.86:0/1630936110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a52ce2a00 msgr2=0x561a52cdfcd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.121+0000 7fad24fee740 1 --2- 172.21.15.86:0/1630936110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a52ce2a00 0x561a52cdfcd0 secure :-1 s=READY pgs=1388 cs=0 l=1 rev1=1 crypto rx=0x7fad140047f0 tx=0x7fad14005c30 comp rx=0 tx=0).stop 2024-04-21T08:45:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.122+0000 7fad24fee740 1 -- 172.21.15.86:0/1630936110 shutdown_connections 2024-04-21T08:45:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.122+0000 7fad24fee740 1 --2- 172.21.15.86:0/1630936110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a52ce2a00 0x561a52cdfcd0 unknown :-1 s=CLOSED pgs=1388 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.122+0000 7fad24fee740 1 --2- 172.21.15.86:0/1630936110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a52cd9e80 0x561a52cdf6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.122+0000 7fad24fee740 1 --2- 172.21.15.86:0/1630936110 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x561a52cf2a30 0x561a52cf4e20 crc :-1 s=READY pgs=56 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.122+0000 7fad24fee740 1 --2- 172.21.15.86:0/1630936110 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561a52cd94d0 0x561a52ce2330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.122+0000 7fad24fee740 1 -- 172.21.15.86:0/1630936110 >> 172.21.15.86:0/1630936110 conn(0x561a52cd3ba0 msgr2=0x561a52cd58f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:21.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.122+0000 7facfb7fe640 0 client.18342 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:45:21.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.122+0000 7fad24fee740 1 -- 172.21.15.86:0/1630936110 shutdown_connections 2024-04-21T08:45:21.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.122+0000 7fad24fee740 1 -- 172.21.15.86:0/1630936110 wait complete. 2024-04-21T08:45:21.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.122+0000 7fad24fee740 20 client.18342 trim_cache size 0 max 16384 2024-04-21T08:45:21.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.134+0000 7fad24fee740 1 Processor -- start 2024-04-21T08:45:21.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.135+0000 7fad24fee740 1 -- start start 2024-04-21T08:45:21.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.135+0000 7fad24fee740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a52cd94d0 0x561a52cee830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.135+0000 7fad24fee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a52cd9e80 0x561a52cdde60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.135+0000 7fad24fee740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561a52ce2a00 0x561a52ce1df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.135+0000 7fad24fee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fad1c1af240 con 0x561a52cd9e80 2024-04-21T08:45:21.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.135+0000 7fad24fee740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fad1c1b0d80 con 0x561a52ce2a00 2024-04-21T08:45:21.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.135+0000 7fad24fee740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fad1c1af910 con 0x561a52cd94d0 2024-04-21T08:45:21.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.135+0000 7fad1b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a52cd9e80 0x561a52cdde60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.135+0000 7fad1b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a52cd9e80 0x561a52cdde60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47732/0 (socket says 172.21.15.86:47732) 2024-04-21T08:45:21.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.135+0000 7fad1b7fe640 1 -- 172.21.15.86:0/687218255 learned_addr learned my addr 172.21.15.86:0/687218255 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:21.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.135+0000 7fad1b7fe640 1 -- 172.21.15.86:0/687218255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a52cd94d0 msgr2=0x561a52cee830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.135+0000 7fad1b7fe640 1 --2- 172.21.15.86:0/687218255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a52cd94d0 0x561a52cee830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.135+0000 7fad1b7fe640 1 -- 172.21.15.86:0/687218255 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561a52ce2a00 msgr2=0x561a52ce1df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.135+0000 7fad1b7fe640 1 --2- 172.21.15.86:0/687218255 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561a52ce2a00 0x561a52ce1df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.135+0000 7fad1b7fe640 1 -- 172.21.15.86:0/687218255 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561a52ce0630 con 0x561a52cd9e80 2024-04-21T08:45:21.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.135+0000 7fad1b7fe640 1 --2- 172.21.15.86:0/687218255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a52cd9e80 0x561a52cdde60 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fad100cdcb0 tx=0x7fad1009bf10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=62288d52200b7ca2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:21.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.135+0000 7fad01ffb640 1 -- 172.21.15.86:0/687218255 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad100b4020 con 0x561a52cd9e80 2024-04-21T08:45:21.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.135+0000 7fad01ffb640 1 -- 172.21.15.86:0/687218255 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad1009cba0 con 0x561a52cd9e80 2024-04-21T08:45:21.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.135+0000 7fad01ffb640 1 -- 172.21.15.86:0/687218255 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad1009b8f0 con 0x561a52cd9e80 2024-04-21T08:45:21.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.136+0000 7fad24fee740 1 -- 172.21.15.86:0/687218255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a52cd9e80 msgr2=0x561a52cdde60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.136+0000 7fad24fee740 1 --2- 172.21.15.86:0/687218255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a52cd9e80 0x561a52cdde60 secure :-1 s=READY pgs=1389 cs=0 l=1 rev1=1 crypto rx=0x7fad100cdcb0 tx=0x7fad1009bf10 comp rx=0 tx=0).stop 2024-04-21T08:45:21.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.136+0000 7fad24fee740 1 -- 172.21.15.86:0/687218255 shutdown_connections 2024-04-21T08:45:21.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.136+0000 7fad24fee740 1 --2- 172.21.15.86:0/687218255 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561a52ce2a00 0x561a52ce1df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.136+0000 7fad24fee740 1 --2- 172.21.15.86:0/687218255 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a52cd9e80 0x561a52cdde60 unknown :-1 s=CLOSED pgs=1389 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.136+0000 7fad24fee740 1 --2- 172.21.15.86:0/687218255 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a52cd94d0 0x561a52cee830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.136+0000 7fad24fee740 1 -- 172.21.15.86:0/687218255 >> 172.21.15.86:0/687218255 conn(0x561a52cd3ba0 msgr2=0x561a52cdac60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:21.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.136+0000 7fad24fee740 1 -- 172.21.15.86:0/687218255 shutdown_connections 2024-04-21T08:45:21.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.136+0000 7fad24fee740 1 -- 172.21.15.86:0/687218255 wait complete. 2024-04-21T08:45:21.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.136+0000 7fad24fee740 1 -- start start 2024-04-21T08:45:21.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.136+0000 7fad24fee740 1 Processor -- start 2024-04-21T08:45:21.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.137+0000 7fad24fee740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561a52cd94d0 0x561a52ceb1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.137+0000 7fad24fee740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a52cd9e80 0x561a52ce65a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.137+0000 7fad24fee740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a52ce2a00 0x561a52ce6bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.137+0000 7fad24fee740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fad1c1b06b0 con 0x561a52cd9e80 2024-04-21T08:45:21.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.137+0000 7fad24fee740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fad1c1affe0 con 0x561a52cd94d0 2024-04-21T08:45:21.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.137+0000 7fad24fee740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fad1c1ad240 con 0x561a52ce2a00 2024-04-21T08:45:21.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.137+0000 7fad24fee740 10 client.0 _ll_register_callbacks cb 0x7fad140043b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:21.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.137+0000 7fad1b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a52cd9e80 0x561a52ce65a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.137+0000 7fad1b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a52cd9e80 0x561a52ce65a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47740/0 (socket says 172.21.15.86:47740) 2024-04-21T08:45:21.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.137+0000 7fad1b7fe640 1 -- 172.21.15.86:0/878554875 learned_addr learned my addr 172.21.15.86:0/878554875 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:21.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.137+0000 7fad208c0640 1 --2- 172.21.15.86:0/878554875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a52ce2a00 0x561a52ce6bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.137+0000 7fad1bfff640 1 --2- 172.21.15.86:0/878554875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561a52cd94d0 0x561a52ceb1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.137+0000 7fad1b7fe640 1 -- 172.21.15.86:0/878554875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a52ce2a00 msgr2=0x561a52ce6bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.137+0000 7fad1b7fe640 1 --2- 172.21.15.86:0/878554875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a52ce2a00 0x561a52ce6bb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.137+0000 7fad1b7fe640 1 -- 172.21.15.86:0/878554875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561a52cd94d0 msgr2=0x561a52ceb1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.137+0000 7fad1b7fe640 1 --2- 172.21.15.86:0/878554875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561a52cd94d0 0x561a52ceb1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.137+0000 7fad1b7fe640 1 -- 172.21.15.86:0/878554875 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561a52ce7380 con 0x561a52cd9e80 2024-04-21T08:45:21.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.137+0000 7fad1bfff640 1 --2- 172.21.15.86:0/878554875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561a52cd94d0 0x561a52ceb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:21.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.137+0000 7fad1b7fe640 1 --2- 172.21.15.86:0/878554875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a52cd9e80 0x561a52ce65a0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fad100ce250 tx=0x7fad100a5180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1b817f9ab7bb91ac server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:21.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.137+0000 7facfb7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:21.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.137+0000 7facfb7fe640 1 -- 172.21.15.86:0/878554875 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad10066060 con 0x561a52cd9e80 2024-04-21T08:45:21.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.137+0000 7facfb7fe640 1 -- 172.21.15.86:0/878554875 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7fad100d0420 con 0x561a52cd9e80 2024-04-21T08:45:21.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.137+0000 7fad24fee740 1 -- 172.21.15.86:0/878554875 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad1009cba0 con 0x561a52cd9e80 2024-04-21T08:45:21.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.137+0000 7facfb7fe640 1 -- 172.21.15.86:0/878554875 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fad100bb070 con 0x561a52cd9e80 2024-04-21T08:45:21.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.138+0000 7fad24fee740 10 client.18348 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:21.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.138+0000 7fad24fee740 1 -- 172.21.15.86:0/878554875 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561a52d615d0 con 0x561a52cd9e80 2024-04-21T08:45:21.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.138+0000 7facfb7fe640 1 -- 172.21.15.86:0/878554875 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad1009cba0 con 0x561a52cd9e80 2024-04-21T08:45:21.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.138+0000 7facfb7fe640 1 -- 172.21.15.86:0/878554875 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fad100982f0 con 0x561a52cd9e80 2024-04-21T08:45:21.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.138+0000 7fad24fee740 10 client.18348 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:21.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.138+0000 7fad24fee740 10 client.18348 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:45:21.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.138+0000 7fad24fee740 20 client.18348 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:21.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.138+0000 7fad24fee740 1 --2- 172.21.15.86:0/878554875 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x561a52d61c90 0x561a52d64080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.138+0000 7fad24fee740 4 client.18348 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:45:21.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.138+0000 7fad24fee740 1 -- 172.21.15.86:0/878554875 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x561a52c4bc10 con 0x561a52d61c90 2024-04-21T08:45:21.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.138+0000 7fad1bfff640 1 --2- 172.21.15.86:0/878554875 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x561a52d61c90 0x561a52d64080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.138+0000 7fad1bfff640 1 --2- 172.21.15.86:0/878554875 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x561a52d61c90 0x561a52d64080 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1d9642c3f199f8a8 server_cookie=279d52f2f4ecff2d in_seq=0 out_seq=0 2024-04-21T08:45:21.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.138+0000 7facfb7fe640 10 client.18348 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:45:21.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.139+0000 7facfb7fe640 1 -- 172.21.15.86:0/878554875 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x561a52c4bc10 con 0x561a52d61c90 2024-04-21T08:45:21.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.139+0000 7facfb7fe640 10 client.18348 handle_command_reply: tid=0 2024-04-21T08:45:21.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.139+0000 7fad24fee740 1 client.18348 shutdown 2024-04-21T08:45:21.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.139+0000 7fad24fee740 1 -- 172.21.15.86:0/878554875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a52cd9e80 msgr2=0x561a52ce65a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.139+0000 7fad24fee740 1 --2- 172.21.15.86:0/878554875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a52cd9e80 0x561a52ce65a0 secure :-1 s=READY pgs=1390 cs=0 l=1 rev1=1 crypto rx=0x7fad100ce250 tx=0x7fad100a5180 comp rx=0 tx=0).stop 2024-04-21T08:45:21.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.139+0000 7fad24fee740 1 -- 172.21.15.86:0/878554875 shutdown_connections 2024-04-21T08:45:21.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.139+0000 7fad24fee740 1 --2- 172.21.15.86:0/878554875 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561a52ce2a00 0x561a52ce6bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.139+0000 7fad24fee740 1 --2- 172.21.15.86:0/878554875 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561a52cd9e80 0x561a52ce65a0 unknown :-1 s=CLOSED pgs=1390 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.139+0000 7fad24fee740 1 --2- 172.21.15.86:0/878554875 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x561a52d61c90 0x561a52d64080 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.139+0000 7fad24fee740 1 --2- 172.21.15.86:0/878554875 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561a52cd94d0 0x561a52ceb1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.139+0000 7facfb7fe640 0 client.18348 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:45:21.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.139+0000 7fad24fee740 1 -- 172.21.15.86:0/878554875 >> 172.21.15.86:0/878554875 conn(0x561a52cd3ba0 msgr2=0x561a52cd2370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:21.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.139+0000 7fad24fee740 1 -- 172.21.15.86:0/878554875 shutdown_connections 2024-04-21T08:45:21.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.139+0000 7fad24fee740 1 -- 172.21.15.86:0/878554875 wait complete. 2024-04-21T08:45:21.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.139+0000 7fad24fee740 20 client.18348 trim_cache size 0 max 16384 2024-04-21T08:45:21.154 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:45:21.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.139+0000 7fad22323640 1 -- 172.21.15.86:0/1411070798 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fad040a3d20 msgr2=0x7fad040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.139+0000 7fad22323640 1 --2- 172.21.15.86:0/1411070798 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fad040a3d20 0x7fad040a61e0 secure :-1 s=READY pgs=221 cs=0 l=1 rev1=1 crypto rx=0x7fad1404d000 tx=0x7fad14048000 comp rx=0 tx=0).stop 2024-04-21T08:45:21.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.139+0000 7fad22323640 1 -- 172.21.15.86:0/1411070798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad1c1072b0 msgr2=0x7fad1c198bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.139+0000 7fad22323640 1 --2- 172.21.15.86:0/1411070798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad1c1072b0 0x7fad1c198bb0 secure :-1 s=READY pgs=1386 cs=0 l=1 rev1=1 crypto rx=0x7fad1000c970 tx=0x7fad1000cd70 comp rx=0 tx=0).stop 2024-04-21T08:45:21.156 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.139+0000 7fad22323640 1 -- 172.21.15.86:0/1411070798 shutdown_connections 2024-04-21T08:45:21.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.139+0000 7fad22323640 1 --2- 172.21.15.86:0/1411070798 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fad040a3d20 0x7fad040a61e0 unknown :-1 s=CLOSED pgs=221 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.139+0000 7fad22323640 1 --2- 172.21.15.86:0/1411070798 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad1c19cf30 0x7fad1c19d3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.139+0000 7fad22323640 1 --2- 172.21.15.86:0/1411070798 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad1c1072b0 0x7fad1c198bb0 unknown :-1 s=CLOSED pgs=1386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.139+0000 7fad22323640 1 --2- 172.21.15.86:0/1411070798 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad1c100af0 0x7fad1c198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.140+0000 7fad22323640 1 -- 172.21.15.86:0/1411070798 >> 172.21.15.86:0/1411070798 conn(0x7fad1c0760e0 msgr2=0x7fad1c0fe830 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:21.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.140+0000 7fad22323640 1 -- 172.21.15.86:0/1411070798 shutdown_connections 2024-04-21T08:45:21.157 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.140+0000 7fad22323640 1 -- 172.21.15.86:0/1411070798 wait complete. 2024-04-21T08:45:21.157 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:45:21.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.258+0000 7fdc67211640 1 -- 172.21.15.86:0/2795064743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc60101a40 msgr2=0x7fdc60105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.258+0000 7fdc67211640 1 --2- 172.21.15.86:0/2795064743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc60101a40 0x7fdc60105a80 secure :-1 s=READY pgs=1391 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c009900 tx=0x7fdc4c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:45:21.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.258+0000 7fdc67211640 1 -- 172.21.15.86:0/2795064743 shutdown_connections 2024-04-21T08:45:21.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.258+0000 7fdc67211640 1 --2- 172.21.15.86:0/2795064743 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc60106160 0x7fdc6010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.258+0000 7fdc67211640 1 --2- 172.21.15.86:0/2795064743 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc60101a40 0x7fdc60105a80 unknown :-1 s=CLOSED pgs=1391 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.258+0000 7fdc67211640 1 --2- 172.21.15.86:0/2795064743 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc60101090 0x7fdc60101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.258+0000 7fdc67211640 1 -- 172.21.15.86:0/2795064743 >> 172.21.15.86:0/2795064743 conn(0x7fdc60077f50 msgr2=0x7fdc60078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:21.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.258+0000 7fdc67211640 1 -- 172.21.15.86:0/2795064743 shutdown_connections 2024-04-21T08:45:21.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.258+0000 7fdc67211640 1 -- 172.21.15.86:0/2795064743 wait complete. 2024-04-21T08:45:21.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.258+0000 7fdc67211640 1 Processor -- start 2024-04-21T08:45:21.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.258+0000 7fdc67211640 1 -- start start 2024-04-21T08:45:21.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.258+0000 7fdc67211640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc60101090 0x7fdc601995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.259+0000 7fdc67211640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc60106160 0x7fdc60199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.259+0000 7fdc67211640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc6019a090 0x7fdc601a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.259+0000 7fdc67211640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdc601110a0 con 0x7fdc60106160 2024-04-21T08:45:21.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.259+0000 7fdc67211640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdc60110f20 con 0x7fdc60101090 2024-04-21T08:45:21.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.259+0000 7fdc67211640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdc60111220 con 0x7fdc6019a090 2024-04-21T08:45:21.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.259+0000 7fdc5ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc60106160 0x7fdc60199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.259+0000 7fdc5ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc60106160 0x7fdc60199b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47758/0 (socket says 172.21.15.86:47758) 2024-04-21T08:45:21.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.259+0000 7fdc5ffff640 1 -- 172.21.15.86:0/3582156181 learned_addr learned my addr 172.21.15.86:0/3582156181 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:21.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.259+0000 7fdc64fad640 1 --2- 172.21.15.86:0/3582156181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc60101090 0x7fdc601995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.259+0000 7fdc657ae640 1 --2- 172.21.15.86:0/3582156181 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc6019a090 0x7fdc601a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.259+0000 7fdc5ffff640 1 -- 172.21.15.86:0/3582156181 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc6019a090 msgr2=0x7fdc601a4640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.259+0000 7fdc5ffff640 1 --2- 172.21.15.86:0/3582156181 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc6019a090 0x7fdc601a4640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.259+0000 7fdc5ffff640 1 -- 172.21.15.86:0/3582156181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc60101090 msgr2=0x7fdc601995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.259+0000 7fdc5ffff640 1 --2- 172.21.15.86:0/3582156181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc60101090 0x7fdc601995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.259+0000 7fdc5ffff640 1 -- 172.21.15.86:0/3582156181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdc601a4e10 con 0x7fdc60106160 2024-04-21T08:45:21.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.259+0000 7fdc657ae640 1 --2- 172.21.15.86:0/3582156181 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc6019a090 0x7fdc601a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:21.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.259+0000 7fdc5ffff640 1 --2- 172.21.15.86:0/3582156181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc60106160 0x7fdc60199b00 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c036040 tx=0x7fdc4c02c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:21.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.259+0000 7fdc5dffb640 1 -- 172.21.15.86:0/3582156181 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c043020 con 0x7fdc60106160 2024-04-21T08:45:21.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.259+0000 7fdc5dffb640 1 -- 172.21.15.86:0/3582156181 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc4c02cbe0 con 0x7fdc60106160 2024-04-21T08:45:21.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.259+0000 7fdc67211640 1 -- 172.21.15.86:0/3582156181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdc601a50a0 con 0x7fdc60106160 2024-04-21T08:45:21.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.259+0000 7fdc5dffb640 1 -- 172.21.15.86:0/3582156181 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c003bf0 con 0x7fdc60106160 2024-04-21T08:45:21.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.259+0000 7fdc67211640 1 -- 172.21.15.86:0/3582156181 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdc601a5640 con 0x7fdc60106160 2024-04-21T08:45:21.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.262+0000 7fdc5dffb640 1 -- 172.21.15.86:0/3582156181 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdc4c046020 con 0x7fdc60106160 2024-04-21T08:45:21.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.262+0000 7fdc69dfe740 1 Processor -- start 2024-04-21T08:45:21.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.262+0000 7fdc69dfe740 1 -- start start 2024-04-21T08:45:21.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.262+0000 7fdc5dffb640 1 --2- 172.21.15.86:0/3582156181 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdc500a3da0 0x7fdc500a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.262+0000 7fdc69dfe740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56086fb514d0 0x56086fb518b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.262+0000 7fdc69dfe740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56086fb51e80 0x56086fb5a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.262+0000 7fdc69dfe740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56086fb5aa00 0x56086fb5ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.262+0000 7fdc69dfe740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56086fab61f0 con 0x56086fb51e80 2024-04-21T08:45:21.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.262+0000 7fdc69dfe740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x56086fa534b0 con 0x56086fb514d0 2024-04-21T08:45:21.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.262+0000 7fdc69dfe740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56086fab21b0 con 0x56086fb5aa00 2024-04-21T08:45:21.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.262+0000 7fdc657ae640 1 --2- 172.21.15.86:0/3582156181 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdc500a3da0 0x7fdc500a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.262+0000 7fdc5dffb640 1 -- 172.21.15.86:0/3582156181 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7fdc4c0f2cb0 con 0x7fdc60106160 2024-04-21T08:45:21.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.262+0000 7fdc5ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56086fb51e80 0x56086fb5a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.262+0000 7fdc5ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56086fb51e80 0x56086fb5a430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47764/0 (socket says 172.21.15.86:47764) 2024-04-21T08:45:21.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.262+0000 7fdc5ffff640 1 -- 172.21.15.86:0/4121005028 learned_addr learned my addr 172.21.15.86:0/4121005028 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.263+0000 7fdc64fad640 1 --2- 172.21.15.86:0/4121005028 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56086fb514d0 0x56086fb518b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.263+0000 7fdc5ffff640 1 -- 172.21.15.86:0/4121005028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56086fb5aa00 msgr2=0x56086fb5ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.263+0000 7fdc5ffff640 1 --2- 172.21.15.86:0/4121005028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56086fb5aa00 0x56086fb5ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.263+0000 7fdc5ffff640 1 -- 172.21.15.86:0/4121005028 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56086fb514d0 msgr2=0x56086fb518b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.263+0000 7fdc5ffff640 1 --2- 172.21.15.86:0/4121005028 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56086fb514d0 0x56086fb518b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.263+0000 7fdc5ffff640 1 -- 172.21.15.86:0/4121005028 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56086fa451b0 con 0x56086fb51e80 2024-04-21T08:45:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.263+0000 7fdc657ae640 1 --2- 172.21.15.86:0/3582156181 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdc500a3da0 0x7fdc500a6260 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdc580092f0 tx=0x7fdc58010740 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.263+0000 7fdc5ffff640 1 --2- 172.21.15.86:0/4121005028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56086fb51e80 0x56086fb5a430 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c0c70d0 tx=0x7fdc4c000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a6a2bbef75108f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.263+0000 7fdc4affd640 1 -- 172.21.15.86:0/4121005028 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c0be4d0 con 0x56086fb51e80 2024-04-21T08:45:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.263+0000 7fdc4affd640 1 -- 172.21.15.86:0/4121005028 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc4c0bf090 con 0x56086fb51e80 2024-04-21T08:45:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.263+0000 7fdc4affd640 1 -- 172.21.15.86:0/4121005028 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c0be2f0 con 0x56086fb51e80 2024-04-21T08:45:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.264+0000 7fdc69dfe740 1 -- 172.21.15.86:0/4121005028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56086fb51e80 msgr2=0x56086fb5a430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.264+0000 7fdc69dfe740 1 --2- 172.21.15.86:0/4121005028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56086fb51e80 0x56086fb5a430 secure :-1 s=READY pgs=1393 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c0c70d0 tx=0x7fdc4c000bc0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.264+0000 7fdc69dfe740 1 -- 172.21.15.86:0/4121005028 shutdown_connections 2024-04-21T08:45:21.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.264+0000 7fdc69dfe740 1 --2- 172.21.15.86:0/4121005028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56086fb5aa00 0x56086fb5ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.264+0000 7fdc69dfe740 1 --2- 172.21.15.86:0/4121005028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56086fb51e80 0x56086fb5a430 unknown :-1 s=CLOSED pgs=1393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.264+0000 7fdc69dfe740 1 --2- 172.21.15.86:0/4121005028 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56086fb514d0 0x56086fb518b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.264+0000 7fdc69dfe740 1 -- 172.21.15.86:0/4121005028 >> 172.21.15.86:0/4121005028 conn(0x56086fb4bba0 msgr2=0x56086fb4bf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:21.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.264+0000 7fdc69dfe740 1 -- 172.21.15.86:0/4121005028 shutdown_connections 2024-04-21T08:45:21.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.264+0000 7fdc69dfe740 1 -- 172.21.15.86:0/4121005028 wait complete. 2024-04-21T08:45:21.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.264+0000 7fdc69dfe740 1 -- start start 2024-04-21T08:45:21.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.264+0000 7fdc69dfe740 1 Processor -- start 2024-04-21T08:45:21.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.264+0000 7fdc69dfe740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56086fb514d0 0x56086fb5a290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.264+0000 7fdc69dfe740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56086fb51e80 0x56086fb57580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.264+0000 7fdc69dfe740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56086fb5aa00 0x56086fb57ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.264+0000 7fdc69dfe740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56086fb5d730 con 0x56086fb51e80 2024-04-21T08:45:21.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.264+0000 7fdc69dfe740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x56086fa6f500 con 0x56086fb514d0 2024-04-21T08:45:21.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.264+0000 7fdc69dfe740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56086fb5d8b0 con 0x56086fb5aa00 2024-04-21T08:45:21.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.265+0000 7fdc69dfe740 10 client.0 _ll_register_callbacks cb 0x56086fa3db80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:21.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.265+0000 7fdc5ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56086fb51e80 0x56086fb57580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.265+0000 7fdc5ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56086fb51e80 0x56086fb57580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47774/0 (socket says 172.21.15.86:47774) 2024-04-21T08:45:21.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.265+0000 7fdc5ffff640 1 -- 172.21.15.86:0/2991957340 learned_addr learned my addr 172.21.15.86:0/2991957340 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:21.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.265+0000 7fdc64fad640 1 --2- 172.21.15.86:0/2991957340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56086fb514d0 0x56086fb5a290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.265+0000 7fdc657ae640 1 --2- 172.21.15.86:0/2991957340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56086fb5aa00 0x56086fb57ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.265+0000 7fdc5ffff640 1 -- 172.21.15.86:0/2991957340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56086fb5aa00 msgr2=0x56086fb57ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.265+0000 7fdc5ffff640 1 --2- 172.21.15.86:0/2991957340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56086fb5aa00 0x56086fb57ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.265+0000 7fdc5ffff640 1 -- 172.21.15.86:0/2991957340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56086fb514d0 msgr2=0x56086fb5a290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.265+0000 7fdc5ffff640 1 --2- 172.21.15.86:0/2991957340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56086fb514d0 0x56086fb5a290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.265+0000 7fdc5ffff640 1 -- 172.21.15.86:0/2991957340 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56086f7c83f0 con 0x56086fb51e80 2024-04-21T08:45:21.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.265+0000 7fdc64fad640 1 --2- 172.21.15.86:0/2991957340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56086fb514d0 0x56086fb5a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:21.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.265+0000 7fdc5ffff640 1 --2- 172.21.15.86:0/2991957340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56086fb51e80 0x56086fb57580 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c0f01f0 tx=0x7fdc4c07b2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=50a07671eaffec41 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:21.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.265+0000 7fdc3ffff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:21.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.265+0000 7fdc3ffff640 1 -- 172.21.15.86:0/2991957340 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc4c0be990 con 0x56086fb51e80 2024-04-21T08:45:21.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.265+0000 7fdc3ffff640 1 -- 172.21.15.86:0/2991957340 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7fdc4c0bb000 con 0x56086fb51e80 2024-04-21T08:45:21.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.265+0000 7fdc69dfe740 1 -- 172.21.15.86:0/2991957340 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdc4c0bf090 con 0x56086fb51e80 2024-04-21T08:45:21.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.265+0000 7fdc3ffff640 1 -- 172.21.15.86:0/2991957340 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdc4c0c2d40 con 0x56086fb51e80 2024-04-21T08:45:21.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.265+0000 7fdc69dfe740 10 client.18360 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:21.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.265+0000 7fdc69dfe740 1 -- 172.21.15.86:0/2991957340 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56086f8927d0 con 0x56086fb51e80 2024-04-21T08:45:21.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.265+0000 7fdc3ffff640 1 -- 172.21.15.86:0/2991957340 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc4c0bf090 con 0x56086fb51e80 2024-04-21T08:45:21.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.266+0000 7fdc3ffff640 1 -- 172.21.15.86:0/2991957340 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdc4c0a90b0 con 0x56086fb51e80 2024-04-21T08:45:21.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.266+0000 7fdc69dfe740 10 client.18360 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:21.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.266+0000 7fdc69dfe740 10 client.18360 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:45:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.266+0000 7fdc69dfe740 20 client.18360 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.266+0000 7fdc69dfe740 1 --2- 172.21.15.86:0/2991957340 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x56086fb6a740 0x56086fb6cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.266+0000 7fdc69dfe740 4 client.18360 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.266+0000 7fdc69dfe740 1 -- 172.21.15.86:0/2991957340 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56086fa55930 con 0x56086fb6a740 2024-04-21T08:45:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.266+0000 7fdc64fad640 1 --2- 172.21.15.86:0/2991957340 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x56086fb6a740 0x56086fb6cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.266+0000 7fdc3ffff640 10 client.18360 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:45:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.266+0000 7fdc64fad640 1 --2- 172.21.15.86:0/2991957340 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x56086fb6a740 0x56086fb6cb30 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e324d9fa0435ab65 server_cookie=2423a3302ecf643e in_seq=0 out_seq=0 2024-04-21T08:45:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.267+0000 7fdc3ffff640 1 -- 172.21.15.86:0/2991957340 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56086fa55930 con 0x56086fb6a740 2024-04-21T08:45:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.267+0000 7fdc3ffff640 10 client.18360 handle_command_reply: tid=0 2024-04-21T08:45:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.267+0000 7fdc69dfe740 1 client.18360 shutdown 2024-04-21T08:45:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.267+0000 7fdc69dfe740 1 -- 172.21.15.86:0/2991957340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56086fb51e80 msgr2=0x56086fb57580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.267+0000 7fdc69dfe740 1 --2- 172.21.15.86:0/2991957340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56086fb51e80 0x56086fb57580 secure :-1 s=READY pgs=1394 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c0f01f0 tx=0x7fdc4c07b2d0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.267+0000 7fdc69dfe740 1 -- 172.21.15.86:0/2991957340 shutdown_connections 2024-04-21T08:45:21.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.267+0000 7fdc69dfe740 1 --2- 172.21.15.86:0/2991957340 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x56086fb6a740 0x56086fb6cb30 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.267+0000 7fdc69dfe740 1 --2- 172.21.15.86:0/2991957340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56086fb5aa00 0x56086fb57ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.267+0000 7fdc69dfe740 1 --2- 172.21.15.86:0/2991957340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56086fb51e80 0x56086fb57580 unknown :-1 s=CLOSED pgs=1394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.267+0000 7fdc69dfe740 1 --2- 172.21.15.86:0/2991957340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56086fb514d0 0x56086fb5a290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.267+0000 7fdc3ffff640 0 client.18360 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:45:21.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.267+0000 7fdc69dfe740 1 -- 172.21.15.86:0/2991957340 >> 172.21.15.86:0/2991957340 conn(0x56086fb4bba0 msgr2=0x56086fb4d8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:21.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.267+0000 7fdc69dfe740 1 -- 172.21.15.86:0/2991957340 shutdown_connections 2024-04-21T08:45:21.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.267+0000 7fdc69dfe740 1 -- 172.21.15.86:0/2991957340 wait complete. 2024-04-21T08:45:21.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.267+0000 7fdc69dfe740 20 client.18360 trim_cache size 0 max 16384 2024-04-21T08:45:21.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.280+0000 7fdc69dfe740 1 Processor -- start 2024-04-21T08:45:21.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.280+0000 7fdc69dfe740 1 -- start start 2024-04-21T08:45:21.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.280+0000 7fdc69dfe740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56086fb514d0 0x56086fb66520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.280+0000 7fdc69dfe740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56086fb51e80 0x56086fb59dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.280+0000 7fdc69dfe740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56086fb5aa00 0x56086fb72320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.280+0000 7fdc69dfe740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdc601af310 con 0x56086fb514d0 2024-04-21T08:45:21.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.280+0000 7fdc69dfe740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdc601b0e50 con 0x56086fb51e80 2024-04-21T08:45:21.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.280+0000 7fdc69dfe740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdc601af9e0 con 0x56086fb5aa00 2024-04-21T08:45:21.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.281+0000 7fdc64fad640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56086fb514d0 0x56086fb66520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.281+0000 7fdc64fad640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56086fb514d0 0x56086fb66520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47788/0 (socket says 172.21.15.86:47788) 2024-04-21T08:45:21.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.281+0000 7fdc64fad640 1 -- 172.21.15.86:0/1278305362 learned_addr learned my addr 172.21.15.86:0/1278305362 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:21.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.281+0000 7fdc5ffff640 1 --2- 172.21.15.86:0/1278305362 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56086fb51e80 0x56086fb59dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.281+0000 7fdc64fad640 1 -- 172.21.15.86:0/1278305362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56086fb5aa00 msgr2=0x56086fb72320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.281+0000 7fdc64fad640 1 --2- 172.21.15.86:0/1278305362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56086fb5aa00 0x56086fb72320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.281+0000 7fdc64fad640 1 -- 172.21.15.86:0/1278305362 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56086fb51e80 msgr2=0x56086fb59dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.281+0000 7fdc64fad640 1 --2- 172.21.15.86:0/1278305362 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56086fb51e80 0x56086fb59dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.281+0000 7fdc64fad640 1 -- 172.21.15.86:0/1278305362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56086fb4b3f0 con 0x56086fb514d0 2024-04-21T08:45:21.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.281+0000 7fdc5ffff640 1 --2- 172.21.15.86:0/1278305362 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56086fb51e80 0x56086fb59dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:21.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.281+0000 7fdc64fad640 1 --2- 172.21.15.86:0/1278305362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56086fb514d0 0x56086fb66520 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x56086fb6d2c0 tx=0x7fdc54013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16271c436ac50e63 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:21.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.281+0000 7fdc4a7fc640 1 -- 172.21.15.86:0/1278305362 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc5400bca0 con 0x56086fb514d0 2024-04-21T08:45:21.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.281+0000 7fdc4a7fc640 1 -- 172.21.15.86:0/1278305362 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdc5400be40 con 0x56086fb514d0 2024-04-21T08:45:21.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.281+0000 7fdc4a7fc640 1 -- 172.21.15.86:0/1278305362 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc54011c00 con 0x56086fb514d0 2024-04-21T08:45:21.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.282+0000 7fdc69dfe740 1 -- 172.21.15.86:0/1278305362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56086fb514d0 msgr2=0x56086fb66520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.282+0000 7fdc69dfe740 1 --2- 172.21.15.86:0/1278305362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56086fb514d0 0x56086fb66520 secure :-1 s=READY pgs=1395 cs=0 l=1 rev1=1 crypto rx=0x56086fb6d2c0 tx=0x7fdc54013830 comp rx=0 tx=0).stop 2024-04-21T08:45:21.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.282+0000 7fdc69dfe740 1 -- 172.21.15.86:0/1278305362 shutdown_connections 2024-04-21T08:45:21.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.282+0000 7fdc69dfe740 1 --2- 172.21.15.86:0/1278305362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56086fb5aa00 0x56086fb72320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.282+0000 7fdc69dfe740 1 --2- 172.21.15.86:0/1278305362 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56086fb51e80 0x56086fb59dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.282+0000 7fdc69dfe740 1 --2- 172.21.15.86:0/1278305362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56086fb514d0 0x56086fb66520 unknown :-1 s=CLOSED pgs=1395 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.282+0000 7fdc69dfe740 1 -- 172.21.15.86:0/1278305362 >> 172.21.15.86:0/1278305362 conn(0x56086fb4bba0 msgr2=0x56086fb4ac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:21.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.282+0000 7fdc69dfe740 1 -- 172.21.15.86:0/1278305362 shutdown_connections 2024-04-21T08:45:21.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.282+0000 7fdc69dfe740 1 -- 172.21.15.86:0/1278305362 wait complete. 2024-04-21T08:45:21.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.282+0000 7fdc69dfe740 1 -- start start 2024-04-21T08:45:21.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.282+0000 7fdc69dfe740 1 Processor -- start 2024-04-21T08:45:21.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.282+0000 7fdc69dfe740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56086fb514d0 0x56086fb62b00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.282+0000 7fdc69dfe740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56086fb51e80 0x56086fb5dff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.282+0000 7fdc69dfe740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56086fb5aa00 0x56086fb5e530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.282+0000 7fdc69dfe740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdc601b0780 con 0x56086fb514d0 2024-04-21T08:45:21.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.282+0000 7fdc69dfe740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdc601b00b0 con 0x56086fb51e80 2024-04-21T08:45:21.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.282+0000 7fdc69dfe740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdc601ad310 con 0x56086fb5aa00 2024-04-21T08:45:21.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.282+0000 7fdc69dfe740 10 client.0 _ll_register_callbacks cb 0x7fdc4c07b200 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:21.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.282+0000 7fdc64fad640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56086fb514d0 0x56086fb62b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.293 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.282+0000 7fdc64fad640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56086fb514d0 0x56086fb62b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47802/0 (socket says 172.21.15.86:47802) 2024-04-21T08:45:21.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.282+0000 7fdc64fad640 1 -- 172.21.15.86:0/2495468916 learned_addr learned my addr 172.21.15.86:0/2495468916 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:21.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.283+0000 7fdc657ae640 1 --2- 172.21.15.86:0/2495468916 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56086fb5aa00 0x56086fb5e530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.283+0000 7fdc5ffff640 1 --2- 172.21.15.86:0/2495468916 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56086fb51e80 0x56086fb5dff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.283+0000 7fdc64fad640 1 -- 172.21.15.86:0/2495468916 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56086fb5aa00 msgr2=0x56086fb5e530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.283+0000 7fdc64fad640 1 --2- 172.21.15.86:0/2495468916 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56086fb5aa00 0x56086fb5e530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.283+0000 7fdc64fad640 1 -- 172.21.15.86:0/2495468916 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56086fb51e80 msgr2=0x56086fb5dff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.283+0000 7fdc64fad640 1 --2- 172.21.15.86:0/2495468916 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56086fb51e80 0x56086fb5dff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.283+0000 7fdc64fad640 1 -- 172.21.15.86:0/2495468916 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56086fb5ea70 con 0x56086fb514d0 2024-04-21T08:45:21.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.283+0000 7fdc657ae640 1 --2- 172.21.15.86:0/2495468916 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56086fb5aa00 0x56086fb5e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:21.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.283+0000 7fdc5ffff640 1 --2- 172.21.15.86:0/2495468916 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56086fb51e80 0x56086fb5dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:21.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.283+0000 7fdc64fad640 1 --2- 172.21.15.86:0/2495468916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56086fb514d0 0x56086fb62b00 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fdc54005f70 tx=0x7fdc540131d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aaa5a28c16115cc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:21.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.283+0000 7fdc3ffff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:21.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.283+0000 7fdc3ffff640 1 -- 172.21.15.86:0/2495468916 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdc54011b80 con 0x56086fb514d0 2024-04-21T08:45:21.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.283+0000 7fdc3ffff640 1 -- 172.21.15.86:0/2495468916 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7fdc54011d20 con 0x56086fb514d0 2024-04-21T08:45:21.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.283+0000 7fdc69dfe740 1 -- 172.21.15.86:0/2495468916 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x56086fac3c10 con 0x56086fb514d0 2024-04-21T08:45:21.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.283+0000 7fdc3ffff640 1 -- 172.21.15.86:0/2495468916 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdc54038050 con 0x56086fb514d0 2024-04-21T08:45:21.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.283+0000 7fdc69dfe740 10 client.18366 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:21.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.283+0000 7fdc69dfe740 1 -- 172.21.15.86:0/2495468916 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56086fbd0810 con 0x56086fb514d0 2024-04-21T08:45:21.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.283+0000 7fdc3ffff640 1 -- 172.21.15.86:0/2495468916 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x56086fac3c10 con 0x56086fb514d0 2024-04-21T08:45:21.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.284+0000 7fdc3ffff640 1 -- 172.21.15.86:0/2495468916 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdc5402f940 con 0x56086fb514d0 2024-04-21T08:45:21.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.284+0000 7fdc69dfe740 10 client.18366 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:21.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.284+0000 7fdc69dfe740 10 client.18366 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:45:21.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.284+0000 7fdc69dfe740 20 client.18366 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:21.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.284+0000 7fdc69dfe740 1 --2- 172.21.15.86:0/2495468916 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x56086fbd0f30 0x56086fbd3350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.284+0000 7fdc69dfe740 4 client.18366 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:45:21.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.284+0000 7fdc69dfe740 1 -- 172.21.15.86:0/2495468916 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56086fabda80 con 0x56086fbd0f30 2024-04-21T08:45:21.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.284+0000 7fdc5ffff640 1 --2- 172.21.15.86:0/2495468916 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x56086fbd0f30 0x56086fbd3350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.284+0000 7fdc3ffff640 10 client.18366 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:45:21.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.284+0000 7fdc5ffff640 1 --2- 172.21.15.86:0/2495468916 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x56086fbd0f30 0x56086fbd3350 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5172637293f5b0bd server_cookie=772fd6aaeeaf50ee in_seq=0 out_seq=0 2024-04-21T08:45:21.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.284+0000 7fdc3ffff640 1 -- 172.21.15.86:0/2495468916 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56086fabda80 con 0x56086fbd0f30 2024-04-21T08:45:21.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.284+0000 7fdc3ffff640 10 client.18366 handle_command_reply: tid=0 2024-04-21T08:45:21.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.284+0000 7fdc69dfe740 1 client.18366 shutdown 2024-04-21T08:45:21.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.284+0000 7fdc69dfe740 1 -- 172.21.15.86:0/2495468916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56086fb514d0 msgr2=0x56086fb62b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.284+0000 7fdc69dfe740 1 --2- 172.21.15.86:0/2495468916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56086fb514d0 0x56086fb62b00 secure :-1 s=READY pgs=1396 cs=0 l=1 rev1=1 crypto rx=0x7fdc54005f70 tx=0x7fdc540131d0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.284+0000 7fdc69dfe740 1 -- 172.21.15.86:0/2495468916 shutdown_connections 2024-04-21T08:45:21.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.285+0000 7fdc69dfe740 1 --2- 172.21.15.86:0/2495468916 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x56086fbd0f30 0x56086fbd3350 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.285+0000 7fdc69dfe740 1 --2- 172.21.15.86:0/2495468916 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56086fb5aa00 0x56086fb5e530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.285+0000 7fdc69dfe740 1 --2- 172.21.15.86:0/2495468916 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56086fb51e80 0x56086fb5dff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.285+0000 7fdc69dfe740 1 --2- 172.21.15.86:0/2495468916 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56086fb514d0 0x56086fb62b00 unknown :-1 s=CLOSED pgs=1396 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.285+0000 7fdc3ffff640 0 client.18366 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:45:21.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.285+0000 7fdc69dfe740 1 -- 172.21.15.86:0/2495468916 >> 172.21.15.86:0/2495468916 conn(0x56086fb4bba0 msgr2=0x56086fb52260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:21.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.285+0000 7fdc69dfe740 1 -- 172.21.15.86:0/2495468916 shutdown_connections 2024-04-21T08:45:21.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.285+0000 7fdc69dfe740 1 -- 172.21.15.86:0/2495468916 wait complete. 2024-04-21T08:45:21.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.285+0000 7fdc69dfe740 20 client.18366 trim_cache size 0 max 16384 2024-04-21T08:45:21.297 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:45:21.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.285+0000 7fdc67211640 1 -- 172.21.15.86:0/3582156181 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdc500a3da0 msgr2=0x7fdc500a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.285+0000 7fdc67211640 1 --2- 172.21.15.86:0/3582156181 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdc500a3da0 0x7fdc500a6260 secure :-1 s=READY pgs=222 cs=0 l=1 rev1=1 crypto rx=0x7fdc580092f0 tx=0x7fdc58010740 comp rx=0 tx=0).stop 2024-04-21T08:45:21.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.285+0000 7fdc67211640 1 -- 172.21.15.86:0/3582156181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc60106160 msgr2=0x7fdc60199b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.285+0000 7fdc67211640 1 --2- 172.21.15.86:0/3582156181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc60106160 0x7fdc60199b00 secure :-1 s=READY pgs=1392 cs=0 l=1 rev1=1 crypto rx=0x7fdc4c036040 tx=0x7fdc4c02c7e0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.285+0000 7fdc67211640 1 -- 172.21.15.86:0/3582156181 shutdown_connections 2024-04-21T08:45:21.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.285+0000 7fdc67211640 1 --2- 172.21.15.86:0/3582156181 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdc500a3da0 0x7fdc500a6260 unknown :-1 s=CLOSED pgs=222 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.285+0000 7fdc67211640 1 --2- 172.21.15.86:0/3582156181 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdc6019a090 0x7fdc601a4640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.285+0000 7fdc67211640 1 --2- 172.21.15.86:0/3582156181 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdc60106160 0x7fdc60199b00 unknown :-1 s=CLOSED pgs=1392 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.285+0000 7fdc67211640 1 --2- 172.21.15.86:0/3582156181 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdc60101090 0x7fdc601995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.285+0000 7fdc67211640 1 -- 172.21.15.86:0/3582156181 >> 172.21.15.86:0/3582156181 conn(0x7fdc60077f50 msgr2=0x7fdc600ff120 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:21.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.285+0000 7fdc67211640 1 -- 172.21.15.86:0/3582156181 shutdown_connections 2024-04-21T08:45:21.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.285+0000 7fdc67211640 1 -- 172.21.15.86:0/3582156181 wait complete. 2024-04-21T08:45:21.300 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:45:21.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.399+0000 7fb812a81640 1 -- 172.21.15.86:0/1992964989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb80c101990 msgr2=0x7fb80c101e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.399+0000 7fb812a81640 1 --2- 172.21.15.86:0/1992964989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb80c101990 0x7fb80c101e10 secure :-1 s=READY pgs=1397 cs=0 l=1 rev1=1 crypto rx=0x7fb800009930 tx=0x7fb80002bc30 comp rx=0 tx=0).stop 2024-04-21T08:45:21.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.399+0000 7fb812a81640 1 -- 172.21.15.86:0/1992964989 shutdown_connections 2024-04-21T08:45:21.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.399+0000 7fb812a81640 1 --2- 172.21.15.86:0/1992964989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb80c102350 0x7fb80c10dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.399+0000 7fb812a81640 1 --2- 172.21.15.86:0/1992964989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb80c101990 0x7fb80c101e10 unknown :-1 s=CLOSED pgs=1397 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.399+0000 7fb812a81640 1 --2- 172.21.15.86:0/1992964989 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb80c105e20 0x7fb80c106200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.399+0000 7fb812a81640 1 -- 172.21.15.86:0/1992964989 >> 172.21.15.86:0/1992964989 conn(0x7fb80c0760e0 msgr2=0x7fb80c0764f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:21.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.399+0000 7fb812a81640 1 -- 172.21.15.86:0/1992964989 shutdown_connections 2024-04-21T08:45:21.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.399+0000 7fb812a81640 1 -- 172.21.15.86:0/1992964989 wait complete. 2024-04-21T08:45:21.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.400+0000 7fb812a81640 1 Processor -- start 2024-04-21T08:45:21.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.400+0000 7fb812a81640 1 -- start start 2024-04-21T08:45:21.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.400+0000 7fb812a81640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb80c101990 0x7fb80c198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.400+0000 7fb812a81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb80c102350 0x7fb80c198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.400+0000 7fb812a81640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb80c105e20 0x7fb80c19d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.400+0000 7fb812a81640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c1121c0 con 0x7fb80c102350 2024-04-21T08:45:21.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.400+0000 7fb812a81640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c112040 con 0x7fb80c105e20 2024-04-21T08:45:21.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.400+0000 7fb812a81640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c112340 con 0x7fb80c101990 2024-04-21T08:45:21.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.400+0000 7fb80bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb80c102350 0x7fb80c198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.400+0000 7fb80bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb80c102350 0x7fb80c198c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47830/0 (socket says 172.21.15.86:47830) 2024-04-21T08:45:21.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.400+0000 7fb80bfff640 1 -- 172.21.15.86:0/2893795640 learned_addr learned my addr 172.21.15.86:0/2893795640 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:21.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.400+0000 7fb81101e640 1 --2- 172.21.15.86:0/2893795640 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb80c105e20 0x7fb80c19d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.400+0000 7fb81081d640 1 --2- 172.21.15.86:0/2893795640 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb80c101990 0x7fb80c198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.401+0000 7fb80bfff640 1 -- 172.21.15.86:0/2893795640 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb80c101990 msgr2=0x7fb80c198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.401+0000 7fb80bfff640 1 --2- 172.21.15.86:0/2893795640 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb80c101990 0x7fb80c198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.401+0000 7fb80bfff640 1 -- 172.21.15.86:0/2893795640 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb80c105e20 msgr2=0x7fb80c19d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.401+0000 7fb80bfff640 1 --2- 172.21.15.86:0/2893795640 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb80c105e20 0x7fb80c19d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.401+0000 7fb80bfff640 1 -- 172.21.15.86:0/2893795640 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb80c19d730 con 0x7fb80c102350 2024-04-21T08:45:21.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.401+0000 7fb80bfff640 1 --2- 172.21.15.86:0/2893795640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb80c102350 0x7fb80c198c80 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fb800005940 tx=0x7fb8000040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:21.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.401+0000 7fb809ffb640 1 -- 172.21.15.86:0/2893795640 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb800043020 con 0x7fb80c102350 2024-04-21T08:45:21.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.401+0000 7fb812a81640 1 -- 172.21.15.86:0/2893795640 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb80c19d9c0 con 0x7fb80c102350 2024-04-21T08:45:21.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.401+0000 7fb809ffb640 1 -- 172.21.15.86:0/2893795640 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8000044c0 con 0x7fb80c102350 2024-04-21T08:45:21.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.401+0000 7fb809ffb640 1 -- 172.21.15.86:0/2893795640 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb800003bf0 con 0x7fb80c102350 2024-04-21T08:45:21.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.401+0000 7fb812a81640 1 -- 172.21.15.86:0/2893795640 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb80c1a52b0 con 0x7fb80c102350 2024-04-21T08:45:21.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.403+0000 7fb809ffb640 1 -- 172.21.15.86:0/2893795640 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb800046020 con 0x7fb80c102350 2024-04-21T08:45:21.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.404+0000 7fb815676740 1 Processor -- start 2024-04-21T08:45:21.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.404+0000 7fb815676740 1 -- start start 2024-04-21T08:45:21.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.404+0000 7fb815676740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d1ff803b0 0x564d1ff80790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.404+0000 7fb815676740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d1ff80d60 0x564d1ff89890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.404+0000 7fb809ffb640 1 --2- 172.21.15.86:0/2893795640 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb7f40a3d20 0x7fb7f40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.404+0000 7fb815676740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d1ff89e60 0x564d1ff8a2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.404+0000 7fb815676740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564d1fee51f0 con 0x564d1ff803b0 2024-04-21T08:45:21.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.404+0000 7fb815676740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564d1fe824b0 con 0x564d1ff80d60 2024-04-21T08:45:21.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.404+0000 7fb815676740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564d1fee11b0 con 0x564d1ff89e60 2024-04-21T08:45:21.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.404+0000 7fb81101e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d1ff803b0 0x564d1ff80790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.404+0000 7fb81081d640 1 --2- 172.21.15.86:0/2893795640 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb7f40a3d20 0x7fb7f40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.404+0000 7fb81101e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d1ff803b0 0x564d1ff80790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47838/0 (socket says 172.21.15.86:47838) 2024-04-21T08:45:21.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.404+0000 7fb81101e640 1 -- 172.21.15.86:0/2710564465 learned_addr learned my addr 172.21.15.86:0/2710564465 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:21.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.404+0000 7fb809ffb640 1 -- 172.21.15.86:0/2893795640 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7fb8000f2690 con 0x7fb80c102350 2024-04-21T08:45:21.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.404+0000 7fb80bfff640 1 --2- 172.21.15.86:0/2710564465 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d1ff80d60 0x564d1ff89890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.404+0000 7fb81101e640 1 -- 172.21.15.86:0/2710564465 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d1ff89e60 msgr2=0x564d1ff8a2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.404+0000 7fb81101e640 1 --2- 172.21.15.86:0/2710564465 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d1ff89e60 0x564d1ff8a2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.404+0000 7fb81101e640 1 -- 172.21.15.86:0/2710564465 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d1ff80d60 msgr2=0x564d1ff89890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.404+0000 7fb81101e640 1 --2- 172.21.15.86:0/2710564465 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d1ff80d60 0x564d1ff89890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.404+0000 7fb81101e640 1 -- 172.21.15.86:0/2710564465 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d1fe741b0 con 0x564d1ff803b0 2024-04-21T08:45:21.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.404+0000 7fb81081d640 1 --2- 172.21.15.86:0/2893795640 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb7f40a3d20 0x7fb7f40a61e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc005fd0 tx=0x7fb7fc00b510 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:21.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.404+0000 7fb81101e640 1 --2- 172.21.15.86:0/2710564465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d1ff803b0 0x564d1ff80790 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fb80400af40 tx=0x7fb804011870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f96eae27bd511a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:21.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.405+0000 7fb7f2ffd640 1 -- 172.21.15.86:0/2710564465 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb804003b70 con 0x564d1ff803b0 2024-04-21T08:45:21.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.405+0000 7fb7f2ffd640 1 -- 172.21.15.86:0/2710564465 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb804003d10 con 0x564d1ff803b0 2024-04-21T08:45:21.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.405+0000 7fb7f2ffd640 1 -- 172.21.15.86:0/2710564465 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb804012890 con 0x564d1ff803b0 2024-04-21T08:45:21.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.405+0000 7fb815676740 1 -- 172.21.15.86:0/2710564465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d1ff803b0 msgr2=0x564d1ff80790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.405+0000 7fb815676740 1 --2- 172.21.15.86:0/2710564465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d1ff803b0 0x564d1ff80790 secure :-1 s=READY pgs=1399 cs=0 l=1 rev1=1 crypto rx=0x7fb80400af40 tx=0x7fb804011870 comp rx=0 tx=0).stop 2024-04-21T08:45:21.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.405+0000 7fb815676740 1 -- 172.21.15.86:0/2710564465 shutdown_connections 2024-04-21T08:45:21.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.405+0000 7fb815676740 1 --2- 172.21.15.86:0/2710564465 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d1ff89e60 0x564d1ff8a2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.405+0000 7fb815676740 1 --2- 172.21.15.86:0/2710564465 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d1ff80d60 0x564d1ff89890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.405+0000 7fb815676740 1 --2- 172.21.15.86:0/2710564465 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d1ff803b0 0x564d1ff80790 unknown :-1 s=CLOSED pgs=1399 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.405+0000 7fb815676740 1 -- 172.21.15.86:0/2710564465 >> 172.21.15.86:0/2710564465 conn(0x564d1ff7aba0 msgr2=0x564d1ff7af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:21.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.405+0000 7fb815676740 1 -- 172.21.15.86:0/2710564465 shutdown_connections 2024-04-21T08:45:21.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.405+0000 7fb815676740 1 -- 172.21.15.86:0/2710564465 wait complete. 2024-04-21T08:45:21.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.406+0000 7fb815676740 1 -- start start 2024-04-21T08:45:21.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.406+0000 7fb815676740 1 Processor -- start 2024-04-21T08:45:21.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.406+0000 7fb815676740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d1ff803b0 0x564d1ff902e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.406+0000 7fb815676740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d1ff80d60 0x564d1ff8d670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.406+0000 7fb815676740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d1ff89e60 0x564d1ff8dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.406+0000 7fb815676740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564d1ff90da0 con 0x564d1ff80d60 2024-04-21T08:45:21.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.406+0000 7fb815676740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564d1fe9e500 con 0x564d1ff89e60 2024-04-21T08:45:21.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.406+0000 7fb815676740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564d1ff90f20 con 0x564d1ff803b0 2024-04-21T08:45:21.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.406+0000 7fb815676740 10 client.0 _ll_register_callbacks cb 0x564d1fe6cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:21.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.406+0000 7fb80bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d1ff80d60 0x564d1ff8d670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.406+0000 7fb80bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d1ff80d60 0x564d1ff8d670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47846/0 (socket says 172.21.15.86:47846) 2024-04-21T08:45:21.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.406+0000 7fb80bfff640 1 -- 172.21.15.86:0/249696060 learned_addr learned my addr 172.21.15.86:0/249696060 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:21.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.406+0000 7fb81081d640 1 --2- 172.21.15.86:0/249696060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d1ff803b0 0x564d1ff902e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.406+0000 7fb81101e640 1 --2- 172.21.15.86:0/249696060 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d1ff89e60 0x564d1ff8dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.407+0000 7fb80bfff640 1 -- 172.21.15.86:0/249696060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d1ff803b0 msgr2=0x564d1ff902e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.407+0000 7fb80bfff640 1 --2- 172.21.15.86:0/249696060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d1ff803b0 0x564d1ff902e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.407+0000 7fb80bfff640 1 -- 172.21.15.86:0/249696060 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d1ff89e60 msgr2=0x564d1ff8dc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.407+0000 7fb80bfff640 1 --2- 172.21.15.86:0/249696060 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d1ff89e60 0x564d1ff8dc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.407+0000 7fb80bfff640 1 -- 172.21.15.86:0/249696060 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d1fbf73f0 con 0x564d1ff80d60 2024-04-21T08:45:21.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.407+0000 7fb80bfff640 1 --2- 172.21.15.86:0/249696060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d1ff80d60 0x564d1ff8d670 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fb8000be070 tx=0x7fb8000bdcd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=469a59a3e47de2c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:21.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.407+0000 7fb7ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:21.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.407+0000 7fb7ebfff640 1 -- 172.21.15.86:0/249696060 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8000be370 con 0x564d1ff80d60 2024-04-21T08:45:21.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.407+0000 7fb7ebfff640 1 -- 172.21.15.86:0/249696060 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7fb8000ba9e0 con 0x564d1ff80d60 2024-04-21T08:45:21.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.407+0000 7fb815676740 1 -- 172.21.15.86:0/249696060 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb804003d10 con 0x564d1ff80d60 2024-04-21T08:45:21.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.407+0000 7fb7ebfff640 1 -- 172.21.15.86:0/249696060 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb8000c2720 con 0x564d1ff80d60 2024-04-21T08:45:21.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.407+0000 7fb815676740 10 client.18378 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:21.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.407+0000 7fb815676740 1 -- 172.21.15.86:0/249696060 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564d1fcc17d0 con 0x564d1ff80d60 2024-04-21T08:45:21.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.407+0000 7fb7ebfff640 1 -- 172.21.15.86:0/249696060 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb804003d10 con 0x564d1ff80d60 2024-04-21T08:45:21.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.407+0000 7fb7ebfff640 1 -- 172.21.15.86:0/249696060 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb800084050 con 0x564d1ff80d60 2024-04-21T08:45:21.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.407+0000 7fb815676740 10 client.18378 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:21.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.407+0000 7fb815676740 10 client.18378 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:45:21.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.407+0000 7fb815676740 20 client.18378 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:21.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.407+0000 7fb815676740 1 --2- 172.21.15.86:0/249696060 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x564d1ff9df90 0x564d1ffa03a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.407+0000 7fb815676740 4 client.18378 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:21.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.407+0000 7fb815676740 1 -- 172.21.15.86:0/249696060 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d1fe84930 con 0x564d1ff9df90 2024-04-21T08:45:21.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.408+0000 7fb81081d640 1 --2- 172.21.15.86:0/249696060 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x564d1ff9df90 0x564d1ffa03a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.408+0000 7fb81081d640 1 --2- 172.21.15.86:0/249696060 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x564d1ff9df90 0x564d1ffa03a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c32a946db44648d5 server_cookie=6ecf9fd8c6d1f9a3 in_seq=0 out_seq=0 2024-04-21T08:45:21.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.408+0000 7fb7ebfff640 10 client.18378 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:45:21.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.409+0000 7fb7ebfff640 1 -- 172.21.15.86:0/249696060 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564d1fe84930 con 0x564d1ff9df90 2024-04-21T08:45:21.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.409+0000 7fb7ebfff640 10 client.18378 handle_command_reply: tid=0 2024-04-21T08:45:21.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.409+0000 7fb815676740 1 client.18378 shutdown 2024-04-21T08:45:21.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.409+0000 7fb815676740 1 -- 172.21.15.86:0/249696060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d1ff80d60 msgr2=0x564d1ff8d670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.409+0000 7fb815676740 1 --2- 172.21.15.86:0/249696060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d1ff80d60 0x564d1ff8d670 secure :-1 s=READY pgs=1400 cs=0 l=1 rev1=1 crypto rx=0x7fb8000be070 tx=0x7fb8000bdcd0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.409+0000 7fb815676740 1 -- 172.21.15.86:0/249696060 shutdown_connections 2024-04-21T08:45:21.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.409+0000 7fb815676740 1 --2- 172.21.15.86:0/249696060 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x564d1ff9df90 0x564d1ffa03a0 crc :-1 s=READY pgs=57 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.409+0000 7fb815676740 1 --2- 172.21.15.86:0/249696060 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d1ff89e60 0x564d1ff8dc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.409+0000 7fb815676740 1 --2- 172.21.15.86:0/249696060 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d1ff80d60 0x564d1ff8d670 unknown :-1 s=CLOSED pgs=1400 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.409+0000 7fb7ebfff640 0 client.18378 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:45:21.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.409+0000 7fb815676740 1 --2- 172.21.15.86:0/249696060 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d1ff803b0 0x564d1ff902e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.409+0000 7fb815676740 1 -- 172.21.15.86:0/249696060 >> 172.21.15.86:0/249696060 conn(0x564d1ff7aba0 msgr2=0x564d1ff7c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:21.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.409+0000 7fb815676740 1 -- 172.21.15.86:0/249696060 shutdown_connections 2024-04-21T08:45:21.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.409+0000 7fb815676740 1 -- 172.21.15.86:0/249696060 wait complete. 2024-04-21T08:45:21.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.409+0000 7fb815676740 20 client.18378 trim_cache size 0 max 16384 2024-04-21T08:45:21.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.422+0000 7fb815676740 1 Processor -- start 2024-04-21T08:45:21.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.422+0000 7fb815676740 1 -- start start 2024-04-21T08:45:21.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.422+0000 7fb815676740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d1ff803b0 0x564d1ff99ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.422+0000 7fb815676740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d1ff80d60 0x564d1ff8b250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.422+0000 7fb815676740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d1ff89e60 0x564d1ff7bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.422+0000 7fb815676740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c1ad1e0 con 0x564d1ff89e60 2024-04-21T08:45:21.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.422+0000 7fb815676740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c1aed20 con 0x564d1ff803b0 2024-04-21T08:45:21.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.422+0000 7fb815676740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c1ad8b0 con 0x564d1ff80d60 2024-04-21T08:45:21.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.423+0000 7fb81101e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d1ff89e60 0x564d1ff7bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.423+0000 7fb81101e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d1ff89e60 0x564d1ff7bf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47850/0 (socket says 172.21.15.86:47850) 2024-04-21T08:45:21.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.423+0000 7fb81101e640 1 -- 172.21.15.86:0/716182398 learned_addr learned my addr 172.21.15.86:0/716182398 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:21.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.423+0000 7fb81081d640 1 --2- 172.21.15.86:0/716182398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d1ff803b0 0x564d1ff99ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.423+0000 7fb80bfff640 1 --2- 172.21.15.86:0/716182398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d1ff80d60 0x564d1ff8b250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.423+0000 7fb81101e640 1 -- 172.21.15.86:0/716182398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d1ff80d60 msgr2=0x564d1ff8b250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.423+0000 7fb81101e640 1 --2- 172.21.15.86:0/716182398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d1ff80d60 0x564d1ff8b250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.423+0000 7fb81101e640 1 -- 172.21.15.86:0/716182398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d1ff803b0 msgr2=0x564d1ff99ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.423+0000 7fb81101e640 1 --2- 172.21.15.86:0/716182398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d1ff803b0 0x564d1ff99ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.423+0000 7fb81101e640 1 -- 172.21.15.86:0/716182398 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d1ffa0a20 con 0x564d1ff89e60 2024-04-21T08:45:21.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.423+0000 7fb81081d640 1 --2- 172.21.15.86:0/716182398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d1ff803b0 0x564d1ff99ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:21.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.423+0000 7fb81101e640 1 --2- 172.21.15.86:0/716182398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d1ff89e60 0x564d1ff7bf80 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fb80400af40 tx=0x7fb80400ac80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8abd0122d2395553 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:21.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.423+0000 7fb7f27fc640 1 -- 172.21.15.86:0/716182398 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb80402a070 con 0x564d1ff89e60 2024-04-21T08:45:21.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.423+0000 7fb7f27fc640 1 -- 172.21.15.86:0/716182398 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8040044b0 con 0x564d1ff89e60 2024-04-21T08:45:21.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.423+0000 7fb7f27fc640 1 -- 172.21.15.86:0/716182398 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb804025700 con 0x564d1ff89e60 2024-04-21T08:45:21.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.424+0000 7fb815676740 1 -- 172.21.15.86:0/716182398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d1ff89e60 msgr2=0x564d1ff7bf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.424+0000 7fb815676740 1 --2- 172.21.15.86:0/716182398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d1ff89e60 0x564d1ff7bf80 secure :-1 s=READY pgs=1401 cs=0 l=1 rev1=1 crypto rx=0x7fb80400af40 tx=0x7fb80400ac80 comp rx=0 tx=0).stop 2024-04-21T08:45:21.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.424+0000 7fb815676740 1 -- 172.21.15.86:0/716182398 shutdown_connections 2024-04-21T08:45:21.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.424+0000 7fb815676740 1 --2- 172.21.15.86:0/716182398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d1ff89e60 0x564d1ff7bf80 unknown :-1 s=CLOSED pgs=1401 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.424+0000 7fb815676740 1 --2- 172.21.15.86:0/716182398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d1ff80d60 0x564d1ff8b250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.424+0000 7fb815676740 1 --2- 172.21.15.86:0/716182398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d1ff803b0 0x564d1ff99ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.424+0000 7fb815676740 1 -- 172.21.15.86:0/716182398 >> 172.21.15.86:0/716182398 conn(0x564d1ff7aba0 msgr2=0x564d1ff8bf60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:21.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.424+0000 7fb815676740 1 -- 172.21.15.86:0/716182398 shutdown_connections 2024-04-21T08:45:21.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.424+0000 7fb815676740 1 -- 172.21.15.86:0/716182398 wait complete. 2024-04-21T08:45:21.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.424+0000 7fb815676740 1 -- start start 2024-04-21T08:45:21.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.424+0000 7fb815676740 1 Processor -- start 2024-04-21T08:45:21.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.424+0000 7fb815676740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d1ff803b0 0x564d1ffa5340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.424+0000 7fb815676740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d1ff80d60 0x564d1ffa5880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.424+0000 7fb815676740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d1ff89e60 0x564d1ffa5dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.424+0000 7fb815676740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c1ae650 con 0x564d1ff803b0 2024-04-21T08:45:21.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.424+0000 7fb815676740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c1adf80 con 0x564d1ff89e60 2024-04-21T08:45:21.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.424+0000 7fb815676740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb80c1ab1e0 con 0x564d1ff80d60 2024-04-21T08:45:21.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.425+0000 7fb815676740 10 client.0 _ll_register_callbacks cb 0x7fb8000ba630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:21.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.425+0000 7fb81081d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d1ff803b0 0x564d1ffa5340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.425+0000 7fb81081d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d1ff803b0 0x564d1ffa5340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47854/0 (socket says 172.21.15.86:47854) 2024-04-21T08:45:21.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.425+0000 7fb81081d640 1 -- 172.21.15.86:0/2151448942 learned_addr learned my addr 172.21.15.86:0/2151448942 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:21.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.425+0000 7fb81081d640 1 -- 172.21.15.86:0/2151448942 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d1ff80d60 msgr2=0x564d1ffa5880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.425+0000 7fb80bfff640 1 --2- 172.21.15.86:0/2151448942 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d1ff80d60 0x564d1ffa5880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.425+0000 7fb81081d640 1 --2- 172.21.15.86:0/2151448942 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d1ff80d60 0x564d1ffa5880 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.425+0000 7fb81081d640 1 -- 172.21.15.86:0/2151448942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d1ff89e60 msgr2=0x564d1ffa5dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.425+0000 7fb81081d640 1 --2- 172.21.15.86:0/2151448942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d1ff89e60 0x564d1ffa5dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.425+0000 7fb81081d640 1 -- 172.21.15.86:0/2151448942 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d200036f0 con 0x564d1ff803b0 2024-04-21T08:45:21.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.425+0000 7fb80bfff640 1 --2- 172.21.15.86:0/2151448942 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d1ff80d60 0x564d1ffa5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:21.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.425+0000 7fb81081d640 1 --2- 172.21.15.86:0/2151448942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d1ff803b0 0x564d1ffa5340 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x564d1ffa0c20 tx=0x7fb7fc004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e961a53b6dcaf73a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:21.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.425+0000 7fb7ebfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:21.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.425+0000 7fb7ebfff640 1 -- 172.21.15.86:0/2151448942 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb7fc02c070 con 0x564d1ff803b0 2024-04-21T08:45:21.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.425+0000 7fb7ebfff640 1 -- 172.21.15.86:0/2151448942 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7fb7fc028530 con 0x564d1ff803b0 2024-04-21T08:45:21.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.425+0000 7fb815676740 1 -- 172.21.15.86:0/2151448942 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb8040044b0 con 0x564d1ff803b0 2024-04-21T08:45:21.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.425+0000 7fb7ebfff640 1 -- 172.21.15.86:0/2151448942 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb7fc038050 con 0x564d1ff803b0 2024-04-21T08:45:21.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.425+0000 7fb815676740 10 client.18384 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:21.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.425+0000 7fb815676740 1 -- 172.21.15.86:0/2151448942 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564d20004980 con 0x564d1ff803b0 2024-04-21T08:45:21.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.425+0000 7fb7ebfff640 1 -- 172.21.15.86:0/2151448942 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb8040044b0 con 0x564d1ff803b0 2024-04-21T08:45:21.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.426+0000 7fb7ebfff640 1 -- 172.21.15.86:0/2151448942 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb7fc01e420 con 0x564d1ff803b0 2024-04-21T08:45:21.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.426+0000 7fb815676740 10 client.18384 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:21.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.426+0000 7fb815676740 10 client.18384 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:45:21.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.426+0000 7fb815676740 20 client.18384 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:21.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.426+0000 7fb815676740 1 --2- 172.21.15.86:0/2151448942 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x564d200050b0 0x564d20007530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.426+0000 7fb815676740 4 client.18384 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:45:21.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.426+0000 7fb815676740 1 -- 172.21.15.86:0/2151448942 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564d1fef2c10 con 0x564d200050b0 2024-04-21T08:45:21.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.426+0000 7fb80bfff640 1 --2- 172.21.15.86:0/2151448942 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x564d200050b0 0x564d20007530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.426+0000 7fb80bfff640 1 --2- 172.21.15.86:0/2151448942 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x564d200050b0 0x564d20007530 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=27b8c28806f47f5b server_cookie=1709356d5f00d36f in_seq=0 out_seq=0 2024-04-21T08:45:21.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.426+0000 7fb7ebfff640 10 client.18384 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:45:21.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.426+0000 7fb7ebfff640 1 -- 172.21.15.86:0/2151448942 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564d1fef2c10 con 0x564d200050b0 2024-04-21T08:45:21.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.426+0000 7fb7ebfff640 10 client.18384 handle_command_reply: tid=0 2024-04-21T08:45:21.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.426+0000 7fb815676740 1 client.18384 shutdown 2024-04-21T08:45:21.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.426+0000 7fb815676740 1 -- 172.21.15.86:0/2151448942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d1ff803b0 msgr2=0x564d1ffa5340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.426+0000 7fb815676740 1 --2- 172.21.15.86:0/2151448942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d1ff803b0 0x564d1ffa5340 secure :-1 s=READY pgs=1402 cs=0 l=1 rev1=1 crypto rx=0x564d1ffa0c20 tx=0x7fb7fc004730 comp rx=0 tx=0).stop 2024-04-21T08:45:21.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.427+0000 7fb815676740 1 -- 172.21.15.86:0/2151448942 shutdown_connections 2024-04-21T08:45:21.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.427+0000 7fb815676740 1 --2- 172.21.15.86:0/2151448942 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x564d200050b0 0x564d20007530 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.427+0000 7fb815676740 1 --2- 172.21.15.86:0/2151448942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d1ff89e60 0x564d1ffa5dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.427+0000 7fb7ebfff640 0 client.18384 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:45:21.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.427+0000 7fb815676740 1 --2- 172.21.15.86:0/2151448942 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d1ff80d60 0x564d1ffa5880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.427+0000 7fb815676740 1 --2- 172.21.15.86:0/2151448942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d1ff803b0 0x564d1ffa5340 unknown :-1 s=CLOSED pgs=1402 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.427+0000 7fb815676740 1 -- 172.21.15.86:0/2151448942 >> 172.21.15.86:0/2151448942 conn(0x564d1ff7aba0 msgr2=0x564d1ff8c3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:21.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.427+0000 7fb815676740 1 -- 172.21.15.86:0/2151448942 shutdown_connections 2024-04-21T08:45:21.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.427+0000 7fb815676740 1 -- 172.21.15.86:0/2151448942 wait complete. 2024-04-21T08:45:21.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.427+0000 7fb815676740 20 client.18384 trim_cache size 0 max 16384 2024-04-21T08:45:21.434 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:45:21.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.427+0000 7fb812a81640 1 -- 172.21.15.86:0/2893795640 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb7f40a3d20 msgr2=0x7fb7f40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.427+0000 7fb812a81640 1 --2- 172.21.15.86:0/2893795640 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb7f40a3d20 0x7fb7f40a61e0 secure :-1 s=READY pgs=223 cs=0 l=1 rev1=1 crypto rx=0x7fb7fc005fd0 tx=0x7fb7fc00b510 comp rx=0 tx=0).stop 2024-04-21T08:45:21.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.427+0000 7fb812a81640 1 -- 172.21.15.86:0/2893795640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb80c102350 msgr2=0x7fb80c198c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.427+0000 7fb812a81640 1 --2- 172.21.15.86:0/2893795640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb80c102350 0x7fb80c198c80 secure :-1 s=READY pgs=1398 cs=0 l=1 rev1=1 crypto rx=0x7fb800005940 tx=0x7fb8000040c0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.427+0000 7fb812a81640 1 -- 172.21.15.86:0/2893795640 shutdown_connections 2024-04-21T08:45:21.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.427+0000 7fb812a81640 1 --2- 172.21.15.86:0/2893795640 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb7f40a3d20 0x7fb7f40a61e0 unknown :-1 s=CLOSED pgs=223 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.427+0000 7fb812a81640 1 --2- 172.21.15.86:0/2893795640 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb80c105e20 0x7fb80c19d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.427+0000 7fb812a81640 1 --2- 172.21.15.86:0/2893795640 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb80c102350 0x7fb80c198c80 unknown :-1 s=CLOSED pgs=1398 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.427+0000 7fb812a81640 1 --2- 172.21.15.86:0/2893795640 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb80c101990 0x7fb80c198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.427+0000 7fb812a81640 1 -- 172.21.15.86:0/2893795640 >> 172.21.15.86:0/2893795640 conn(0x7fb80c0760e0 msgr2=0x7fb80c0fea30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:21.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.427+0000 7fb812a81640 1 -- 172.21.15.86:0/2893795640 shutdown_connections 2024-04-21T08:45:21.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.427+0000 7fb812a81640 1 -- 172.21.15.86:0/2893795640 wait complete. 2024-04-21T08:45:21.438 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:45:21.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.539+0000 7f10b5359640 1 -- 172.21.15.86:0/1597133128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10b006b170 msgr2=0x7f10b0106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.539+0000 7f10b5359640 1 --2- 172.21.15.86:0/1597133128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10b006b170 0x7f10b0106ba0 secure :-1 s=READY pgs=1403 cs=0 l=1 rev1=1 crypto rx=0x7f1098009900 tx=0x7f109802bc30 comp rx=0 tx=0).stop 2024-04-21T08:45:21.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.539+0000 7f10b5359640 1 -- 172.21.15.86:0/1597133128 shutdown_connections 2024-04-21T08:45:21.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.539+0000 7f10b5359640 1 --2- 172.21.15.86:0/1597133128 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10b0107280 0x7f10b010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.539+0000 7f10b5359640 1 --2- 172.21.15.86:0/1597133128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10b006b170 0x7f10b0106ba0 unknown :-1 s=CLOSED pgs=1403 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.539+0000 7f10b5359640 1 --2- 172.21.15.86:0/1597133128 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10b006a7c0 0x7f10b006aba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.539+0000 7f10b5359640 1 -- 172.21.15.86:0/1597133128 >> 172.21.15.86:0/1597133128 conn(0x7f10b0076c50 msgr2=0x7f10b0077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:21.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.539+0000 7f10b5359640 1 -- 172.21.15.86:0/1597133128 shutdown_connections 2024-04-21T08:45:21.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.539+0000 7f10b5359640 1 -- 172.21.15.86:0/1597133128 wait complete. 2024-04-21T08:45:21.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.540+0000 7f10b5359640 1 Processor -- start 2024-04-21T08:45:21.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.540+0000 7f10b5359640 1 -- start start 2024-04-21T08:45:21.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.540+0000 7f10b5359640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10b006a7c0 0x7f10b01a0650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.540+0000 7f10b5359640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10b0107280 0x7f10b01a0b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.540+0000 7f10b5359640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10b019a710 0x7f10b019ab90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.540+0000 7f10b5359640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f10b01121a0 con 0x7f10b019a710 2024-04-21T08:45:21.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.540+0000 7f10b5359640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f10b0112020 con 0x7f10b0107280 2024-04-21T08:45:21.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.540+0000 7f10b5359640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f10b0112320 con 0x7f10b006a7c0 2024-04-21T08:45:21.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.540+0000 7f10af7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10b019a710 0x7f10b019ab90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.540+0000 7f10af7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10b019a710 0x7f10b019ab90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47872/0 (socket says 172.21.15.86:47872) 2024-04-21T08:45:21.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.540+0000 7f10af7fe640 1 -- 172.21.15.86:0/2560502169 learned_addr learned my addr 172.21.15.86:0/2560502169 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:21.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.540+0000 7f10a67fc640 1 --2- 172.21.15.86:0/2560502169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10b0107280 0x7f10b01a0b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.540+0000 7f10aeffd640 1 --2- 172.21.15.86:0/2560502169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10b006a7c0 0x7f10b01a0650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.540+0000 7f10af7fe640 1 -- 172.21.15.86:0/2560502169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10b006a7c0 msgr2=0x7f10b01a0650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.540+0000 7f10af7fe640 1 --2- 172.21.15.86:0/2560502169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10b006a7c0 0x7f10b01a0650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.540+0000 7f10af7fe640 1 -- 172.21.15.86:0/2560502169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10b0107280 msgr2=0x7f10b01a0b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.540+0000 7f10af7fe640 1 --2- 172.21.15.86:0/2560502169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10b0107280 0x7f10b01a0b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.543 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.540+0000 7f10af7fe640 1 -- 172.21.15.86:0/2560502169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f10b019b310 con 0x7f10b019a710 2024-04-21T08:45:21.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.540+0000 7f10aeffd640 1 --2- 172.21.15.86:0/2560502169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10b006a7c0 0x7f10b01a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:21.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.541+0000 7f10af7fe640 1 --2- 172.21.15.86:0/2560502169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10b019a710 0x7f10b019ab90 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f10a800eaa0 tx=0x7f10a800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:21.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.541+0000 7f10acff9640 1 -- 172.21.15.86:0/2560502169 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a800cdb0 con 0x7f10b019a710 2024-04-21T08:45:21.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.541+0000 7f10acff9640 1 -- 172.21.15.86:0/2560502169 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10a8005020 con 0x7f10b019a710 2024-04-21T08:45:21.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.541+0000 7f10acff9640 1 -- 172.21.15.86:0/2560502169 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a80116d0 con 0x7f10b019a710 2024-04-21T08:45:21.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.541+0000 7f10b5359640 1 -- 172.21.15.86:0/2560502169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f10b01a73f0 con 0x7f10b019a710 2024-04-21T08:45:21.544 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.541+0000 7f10b5359640 1 -- 172.21.15.86:0/2560502169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f10b01a77a0 con 0x7f10b019a710 2024-04-21T08:45:21.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.543+0000 7f10acff9640 1 -- 172.21.15.86:0/2560502169 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f10a80051d0 con 0x7f10b019a710 2024-04-21T08:45:21.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.543+0000 7f10b802b740 1 Processor -- start 2024-04-21T08:45:21.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.543+0000 7f10b802b740 1 -- start start 2024-04-21T08:45:21.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.544+0000 7f10b802b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630096ab4d0 0x5630096ab8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.544+0000 7f10b802b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5630096abe80 0x5630096b4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.544+0000 7f10b802b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630096b4a00 0x5630096b6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.544+0000 7f10b802b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5630096101f0 con 0x5630096ab4d0 2024-04-21T08:45:21.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.544+0000 7f10b802b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5630095ad4b0 con 0x5630096abe80 2024-04-21T08:45:21.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.544+0000 7f10b802b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56300960c1b0 con 0x5630096b4a00 2024-04-21T08:45:21.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.544+0000 7f10a67fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630096ab4d0 0x5630096ab8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.544+0000 7f10a67fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630096ab4d0 0x5630096ab8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47886/0 (socket says 172.21.15.86:47886) 2024-04-21T08:45:21.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.544+0000 7f10a67fc640 1 -- 172.21.15.86:0/880888616 learned_addr learned my addr 172.21.15.86:0/880888616 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:21.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.544+0000 7f10acff9640 1 --2- 172.21.15.86:0/2560502169 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f109c0a3d20 0x7f109c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.544+0000 7f10aeffd640 1 --2- 172.21.15.86:0/880888616 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5630096abe80 0x5630096b4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.544+0000 7f10a67fc640 1 --2- 172.21.15.86:0/880888616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630096b4a00 0x5630096b6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.544+0000 7f10af7fe640 1 --2- 172.21.15.86:0/2560502169 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f109c0a3d20 0x7f109c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.544+0000 7f10a67fc640 1 -- 172.21.15.86:0/880888616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630096b4a00 msgr2=0x5630096b6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.544+0000 7f10a67fc640 1 --2- 172.21.15.86:0/880888616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630096b4a00 0x5630096b6e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.544+0000 7f10a67fc640 1 -- 172.21.15.86:0/880888616 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5630096abe80 msgr2=0x5630096b4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.544+0000 7f10a67fc640 1 --2- 172.21.15.86:0/880888616 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5630096abe80 0x5630096b4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.544+0000 7f10a67fc640 1 -- 172.21.15.86:0/880888616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56300959f1b0 con 0x5630096ab4d0 2024-04-21T08:45:21.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.544+0000 7f10acff9640 1 -- 172.21.15.86:0/2560502169 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f10a80d0df0 con 0x7f10b019a710 2024-04-21T08:45:21.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.544+0000 7f10a67fc640 1 --2- 172.21.15.86:0/880888616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630096ab4d0 0x5630096ab8b0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f1098009230 tx=0x7f1098005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45351969ad5ba295 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:21.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.544+0000 7f10af7fe640 1 --2- 172.21.15.86:0/2560502169 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f109c0a3d20 0x7f109c0a61e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f10a80a50a0 tx=0x7f10a806f450 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:21.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.544+0000 7f10a57fa640 1 -- 172.21.15.86:0/880888616 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1098047020 con 0x5630096ab4d0 2024-04-21T08:45:21.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.545+0000 7f10a57fa640 1 -- 172.21.15.86:0/880888616 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f109802cc90 con 0x5630096ab4d0 2024-04-21T08:45:21.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.545+0000 7f10a57fa640 1 -- 172.21.15.86:0/880888616 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1098004900 con 0x5630096ab4d0 2024-04-21T08:45:21.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.545+0000 7f10b802b740 1 -- 172.21.15.86:0/880888616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630096ab4d0 msgr2=0x5630096ab8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.545+0000 7f10b802b740 1 --2- 172.21.15.86:0/880888616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630096ab4d0 0x5630096ab8b0 secure :-1 s=READY pgs=1405 cs=0 l=1 rev1=1 crypto rx=0x7f1098009230 tx=0x7f1098005b70 comp rx=0 tx=0).stop 2024-04-21T08:45:21.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.545+0000 7f10b802b740 1 -- 172.21.15.86:0/880888616 shutdown_connections 2024-04-21T08:45:21.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.545+0000 7f10b802b740 1 --2- 172.21.15.86:0/880888616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630096b4a00 0x5630096b6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.545+0000 7f10b802b740 1 --2- 172.21.15.86:0/880888616 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5630096abe80 0x5630096b4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.545+0000 7f10b802b740 1 --2- 172.21.15.86:0/880888616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630096ab4d0 0x5630096ab8b0 unknown :-1 s=CLOSED pgs=1405 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.545+0000 7f10b802b740 1 -- 172.21.15.86:0/880888616 >> 172.21.15.86:0/880888616 conn(0x5630096a5ba0 msgr2=0x5630096a5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:21.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.545+0000 7f10b802b740 1 -- 172.21.15.86:0/880888616 shutdown_connections 2024-04-21T08:45:21.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.545+0000 7f10b802b740 1 -- 172.21.15.86:0/880888616 wait complete. 2024-04-21T08:45:21.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.546+0000 7f10b802b740 1 -- start start 2024-04-21T08:45:21.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.546+0000 7f10b802b740 1 Processor -- start 2024-04-21T08:45:21.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.546+0000 7f10b802b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630096ab4d0 0x5630096b42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.546+0000 7f10b802b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630096abe80 0x5630096b1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.546+0000 7f10b802b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5630096b4a00 0x5630096b1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.546+0000 7f10b802b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5630096b7910 con 0x5630096abe80 2024-04-21T08:45:21.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.546+0000 7f10b802b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5630095c9500 con 0x5630096b4a00 2024-04-21T08:45:21.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.546+0000 7f10b802b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5630096b7a90 con 0x5630096ab4d0 2024-04-21T08:45:21.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.546+0000 7f10b802b740 10 client.0 _ll_register_callbacks cb 0x563009597b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:21.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.546+0000 7f10aeffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630096abe80 0x5630096b1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.546+0000 7f10aeffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630096abe80 0x5630096b1660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47890/0 (socket says 172.21.15.86:47890) 2024-04-21T08:45:21.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.546+0000 7f10aeffd640 1 -- 172.21.15.86:0/2001760556 learned_addr learned my addr 172.21.15.86:0/2001760556 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:21.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.546+0000 7f10a67fc640 1 --2- 172.21.15.86:0/2001760556 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5630096b4a00 0x5630096b1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.546+0000 7f10a67fc640 1 --2- 172.21.15.86:0/2001760556 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630096ab4d0 0x5630096b42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.546+0000 7f10aeffd640 1 -- 172.21.15.86:0/2001760556 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630096ab4d0 msgr2=0x5630096b42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.546+0000 7f10aeffd640 1 --2- 172.21.15.86:0/2001760556 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630096ab4d0 0x5630096b42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.546+0000 7f10aeffd640 1 -- 172.21.15.86:0/2001760556 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5630096b4a00 msgr2=0x5630096b1bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.546+0000 7f10aeffd640 1 --2- 172.21.15.86:0/2001760556 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5630096b4a00 0x5630096b1bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.546+0000 7f10aeffd640 1 -- 172.21.15.86:0/2001760556 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630093223f0 con 0x5630096abe80 2024-04-21T08:45:21.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.547+0000 7f10aeffd640 1 --2- 172.21.15.86:0/2001760556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630096abe80 0x5630096b1660 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f10a0009f60 tx=0x7f10a0009f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7b10e45937bd94da server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:21.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.547+0000 7f10967fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:21.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.547+0000 7f10967fc640 1 -- 172.21.15.86:0/2001760556 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a0004220 con 0x5630096abe80 2024-04-21T08:45:21.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.547+0000 7f10967fc640 1 -- 172.21.15.86:0/2001760556 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f10a00043c0 con 0x5630096abe80 2024-04-21T08:45:21.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.547+0000 7f10b802b740 1 -- 172.21.15.86:0/2001760556 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f109802cc90 con 0x5630096abe80 2024-04-21T08:45:21.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.547+0000 7f10967fc640 1 -- 172.21.15.86:0/2001760556 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f10a0012720 con 0x5630096abe80 2024-04-21T08:45:21.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.547+0000 7f10b802b740 10 client.18396 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:21.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.547+0000 7f10b802b740 1 -- 172.21.15.86:0/2001760556 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5630093ec7d0 con 0x5630096abe80 2024-04-21T08:45:21.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.547+0000 7f10967fc640 1 -- 172.21.15.86:0/2001760556 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f109802cc90 con 0x5630096abe80 2024-04-21T08:45:21.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.547+0000 7f10967fc640 1 -- 172.21.15.86:0/2001760556 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f10a000f040 con 0x5630096abe80 2024-04-21T08:45:21.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.547+0000 7f10b802b740 10 client.18396 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:21.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.547+0000 7f10b802b740 10 client.18396 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:45:21.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.547+0000 7f10b802b740 20 client.18396 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:21.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.547+0000 7f10b802b740 1 --2- 172.21.15.86:0/2001760556 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5630096c4c70 0x5630096c7090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.547+0000 7f10b802b740 4 client.18396 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:21.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.547+0000 7f10b802b740 1 -- 172.21.15.86:0/2001760556 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5630095af930 con 0x5630096c4c70 2024-04-21T08:45:21.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.548+0000 7f10a67fc640 1 --2- 172.21.15.86:0/2001760556 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5630096c4c70 0x5630096c7090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.548+0000 7f10967fc640 10 client.18396 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:45:21.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.548+0000 7f10a67fc640 1 --2- 172.21.15.86:0/2001760556 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5630096c4c70 0x5630096c7090 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=f2cdb94bf54d3f29 server_cookie=1378c3766cee7961 in_seq=0 out_seq=0 2024-04-21T08:45:21.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.549+0000 7f10967fc640 1 -- 172.21.15.86:0/2001760556 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5630095af930 con 0x5630096c4c70 2024-04-21T08:45:21.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.549+0000 7f10967fc640 10 client.18396 handle_command_reply: tid=0 2024-04-21T08:45:21.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.549+0000 7f10b802b740 1 client.18396 shutdown 2024-04-21T08:45:21.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.549+0000 7f10b802b740 1 -- 172.21.15.86:0/2001760556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630096abe80 msgr2=0x5630096b1660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.549+0000 7f10b802b740 1 --2- 172.21.15.86:0/2001760556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630096abe80 0x5630096b1660 secure :-1 s=READY pgs=1406 cs=0 l=1 rev1=1 crypto rx=0x7f10a0009f60 tx=0x7f10a0009f90 comp rx=0 tx=0).stop 2024-04-21T08:45:21.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.549+0000 7f10b802b740 1 -- 172.21.15.86:0/2001760556 shutdown_connections 2024-04-21T08:45:21.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.549+0000 7f10b802b740 1 --2- 172.21.15.86:0/2001760556 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5630096b4a00 0x5630096b1bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.549+0000 7f10b802b740 1 --2- 172.21.15.86:0/2001760556 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5630096c4c70 0x5630096c7090 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.549+0000 7f10b802b740 1 --2- 172.21.15.86:0/2001760556 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630096abe80 0x5630096b1660 unknown :-1 s=CLOSED pgs=1406 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.549+0000 7f10b802b740 1 --2- 172.21.15.86:0/2001760556 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630096ab4d0 0x5630096b42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.549+0000 7f10967fc640 0 client.18396 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:45:21.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.549+0000 7f10b802b740 1 -- 172.21.15.86:0/2001760556 >> 172.21.15.86:0/2001760556 conn(0x5630096a5ba0 msgr2=0x5630096a78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:21.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.549+0000 7f10b802b740 1 -- 172.21.15.86:0/2001760556 shutdown_connections 2024-04-21T08:45:21.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.549+0000 7f10b802b740 1 -- 172.21.15.86:0/2001760556 wait complete. 2024-04-21T08:45:21.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.549+0000 7f10b802b740 20 client.18396 trim_cache size 0 max 16384 2024-04-21T08:45:21.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.562+0000 7f10b802b740 1 Processor -- start 2024-04-21T08:45:21.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.562+0000 7f10b802b740 1 -- start start 2024-04-21T08:45:21.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.562+0000 7f10b802b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5630096ab4d0 0x5630096c07d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.563+0000 7f10b802b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630096abe80 0x5630096a4a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.563+0000 7f10b802b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630096b4a00 0x5630096b3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.563+0000 7f10b802b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f10b01b1520 con 0x5630096abe80 2024-04-21T08:45:21.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.563+0000 7f10b802b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f10b01b3060 con 0x5630096ab4d0 2024-04-21T08:45:21.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.563+0000 7f10b802b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f10b01b1bf0 con 0x5630096b4a00 2024-04-21T08:45:21.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.563+0000 7f10aeffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630096abe80 0x5630096a4a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.563+0000 7f10aeffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630096abe80 0x5630096a4a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47892/0 (socket says 172.21.15.86:47892) 2024-04-21T08:45:21.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.563+0000 7f10aeffd640 1 -- 172.21.15.86:0/3306285201 learned_addr learned my addr 172.21.15.86:0/3306285201 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:21.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.563+0000 7f10a67fc640 1 --2- 172.21.15.86:0/3306285201 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5630096ab4d0 0x5630096c07d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.563+0000 7f10aeffd640 1 -- 172.21.15.86:0/3306285201 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630096b4a00 msgr2=0x5630096b3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.563+0000 7f10aeffd640 1 --2- 172.21.15.86:0/3306285201 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630096b4a00 0x5630096b3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.563+0000 7f10aeffd640 1 -- 172.21.15.86:0/3306285201 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5630096ab4d0 msgr2=0x5630096c07d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.563+0000 7f10aeffd640 1 --2- 172.21.15.86:0/3306285201 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5630096ab4d0 0x5630096c07d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.563+0000 7f10aeffd640 1 -- 172.21.15.86:0/3306285201 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5630096c1990 con 0x5630096abe80 2024-04-21T08:45:21.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.563+0000 7f10aeffd640 1 --2- 172.21.15.86:0/3306285201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630096abe80 0x5630096a4a80 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f10a0000c00 tx=0x7f10a0009870 comp rx=0 tx=0).ready entity=mon.0 client_cookie=74a15e86ca0627e7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:21.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.563+0000 7f10a4ff9640 1 -- 172.21.15.86:0/3306285201 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a00037f0 con 0x5630096abe80 2024-04-21T08:45:21.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.563+0000 7f10a4ff9640 1 -- 172.21.15.86:0/3306285201 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10a0011d00 con 0x5630096abe80 2024-04-21T08:45:21.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.563+0000 7f10a4ff9640 1 -- 172.21.15.86:0/3306285201 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f10a0012df0 con 0x5630096abe80 2024-04-21T08:45:21.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.564+0000 7f10b802b740 1 -- 172.21.15.86:0/3306285201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630096abe80 msgr2=0x5630096a4a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.564+0000 7f10b802b740 1 --2- 172.21.15.86:0/3306285201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630096abe80 0x5630096a4a80 secure :-1 s=READY pgs=1407 cs=0 l=1 rev1=1 crypto rx=0x7f10a0000c00 tx=0x7f10a0009870 comp rx=0 tx=0).stop 2024-04-21T08:45:21.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.564+0000 7f10b802b740 1 -- 172.21.15.86:0/3306285201 shutdown_connections 2024-04-21T08:45:21.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.564+0000 7f10b802b740 1 --2- 172.21.15.86:0/3306285201 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630096b4a00 0x5630096b3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.564+0000 7f10b802b740 1 --2- 172.21.15.86:0/3306285201 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630096abe80 0x5630096a4a80 unknown :-1 s=CLOSED pgs=1407 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.564+0000 7f10b802b740 1 --2- 172.21.15.86:0/3306285201 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5630096ab4d0 0x5630096c07d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.564+0000 7f10b802b740 1 -- 172.21.15.86:0/3306285201 >> 172.21.15.86:0/3306285201 conn(0x5630096a5ba0 msgr2=0x5630096a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:21.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.564+0000 7f10b802b740 1 -- 172.21.15.86:0/3306285201 shutdown_connections 2024-04-21T08:45:21.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.564+0000 7f10b802b740 1 -- 172.21.15.86:0/3306285201 wait complete. 2024-04-21T08:45:21.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.564+0000 7f10b802b740 1 -- start start 2024-04-21T08:45:21.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.564+0000 7f10b802b740 1 Processor -- start 2024-04-21T08:45:21.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.565+0000 7f10b802b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630096ab4d0 0x5630096bcf50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.565+0000 7f10b802b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5630096abe80 0x5630096b82c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.565+0000 7f10b802b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630096b4a00 0x5630096b8800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.565+0000 7f10b802b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f10b01b2990 con 0x5630096b4a00 2024-04-21T08:45:21.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.565+0000 7f10b802b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f10b01b22c0 con 0x5630096abe80 2024-04-21T08:45:21.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.565+0000 7f10b802b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f10b01af520 con 0x5630096ab4d0 2024-04-21T08:45:21.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.565+0000 7f10b802b740 10 client.0 _ll_register_callbacks cb 0x7f10a0009b70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:21.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.565+0000 7f10a67fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630096b4a00 0x5630096b8800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.565+0000 7f10a67fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630096b4a00 0x5630096b8800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47894/0 (socket says 172.21.15.86:47894) 2024-04-21T08:45:21.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.565+0000 7f10a67fc640 1 -- 172.21.15.86:0/4089105158 learned_addr learned my addr 172.21.15.86:0/4089105158 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:21.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.565+0000 7f10a67fc640 1 -- 172.21.15.86:0/4089105158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630096ab4d0 msgr2=0x5630096bcf50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.565+0000 7f10a67fc640 1 --2- 172.21.15.86:0/4089105158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630096ab4d0 0x5630096bcf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.565+0000 7f10a67fc640 1 -- 172.21.15.86:0/4089105158 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5630096abe80 msgr2=0x5630096b82c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.565+0000 7f10aeffd640 1 --2- 172.21.15.86:0/4089105158 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5630096abe80 0x5630096b82c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.565+0000 7f10a67fc640 1 --2- 172.21.15.86:0/4089105158 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5630096abe80 0x5630096b82c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.565+0000 7f10a67fc640 1 -- 172.21.15.86:0/4089105158 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5630096b8ea0 con 0x5630096b4a00 2024-04-21T08:45:21.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.565+0000 7f10aeffd640 1 --2- 172.21.15.86:0/4089105158 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5630096abe80 0x5630096b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:21.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.565+0000 7f10a67fc640 1 --2- 172.21.15.86:0/4089105158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630096b4a00 0x5630096b8800 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f10980057c0 tx=0x7f109802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f5debdfc0db4795 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:21.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.565+0000 7f10967fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:21.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.565+0000 7f10967fc640 1 -- 172.21.15.86:0/4089105158 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f109805e060 con 0x5630096b4a00 2024-04-21T08:45:21.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.565+0000 7f10967fc640 1 -- 172.21.15.86:0/4089105158 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f109802c9a0 con 0x5630096b4a00 2024-04-21T08:45:21.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.565+0000 7f10b802b740 1 -- 172.21.15.86:0/4089105158 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f10a0011d00 con 0x5630096b4a00 2024-04-21T08:45:21.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.565+0000 7f10967fc640 1 -- 172.21.15.86:0/4089105158 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1098047040 con 0x5630096b4a00 2024-04-21T08:45:21.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.565+0000 7f10b802b740 10 client.18402 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:21.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.565+0000 7f10b802b740 1 -- 172.21.15.86:0/4089105158 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56300972b060 con 0x5630096b4a00 2024-04-21T08:45:21.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.565+0000 7f10967fc640 1 -- 172.21.15.86:0/4089105158 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f10a0011d00 con 0x5630096b4a00 2024-04-21T08:45:21.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.566+0000 7f10967fc640 1 -- 172.21.15.86:0/4089105158 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1098040040 con 0x5630096b4a00 2024-04-21T08:45:21.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.566+0000 7f10b802b740 10 client.18402 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:21.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.566+0000 7f10b802b740 10 client.18402 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:45:21.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.566+0000 7f10b802b740 20 client.18402 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:21.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.566+0000 7f10b802b740 1 --2- 172.21.15.86:0/4089105158 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x56300972b3f0 0x56300972d7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.566+0000 7f10b802b740 4 client.18402 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:45:21.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.566+0000 7f10b802b740 1 -- 172.21.15.86:0/4089105158 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56300961dc10 con 0x56300972b3f0 2024-04-21T08:45:21.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.566+0000 7f10aeffd640 1 --2- 172.21.15.86:0/4089105158 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x56300972b3f0 0x56300972d7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.566+0000 7f10967fc640 10 client.18402 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:45:21.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.566+0000 7f10aeffd640 1 --2- 172.21.15.86:0/4089105158 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x56300972b3f0 0x56300972d7e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4b867a8ba5976b3 server_cookie=d79276a643133c2 in_seq=0 out_seq=0 2024-04-21T08:45:21.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.566+0000 7f10967fc640 1 -- 172.21.15.86:0/4089105158 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56300961dc10 con 0x56300972b3f0 2024-04-21T08:45:21.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.566+0000 7f10967fc640 10 client.18402 handle_command_reply: tid=0 2024-04-21T08:45:21.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.567+0000 7f10b802b740 1 client.18402 shutdown 2024-04-21T08:45:21.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.567+0000 7f10b802b740 1 -- 172.21.15.86:0/4089105158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630096b4a00 msgr2=0x5630096b8800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.567+0000 7f10b802b740 1 --2- 172.21.15.86:0/4089105158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630096b4a00 0x5630096b8800 secure :-1 s=READY pgs=1408 cs=0 l=1 rev1=1 crypto rx=0x7f10980057c0 tx=0x7f109802bc30 comp rx=0 tx=0).stop 2024-04-21T08:45:21.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.567+0000 7f10b802b740 1 -- 172.21.15.86:0/4089105158 shutdown_connections 2024-04-21T08:45:21.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.567+0000 7f10b802b740 1 --2- 172.21.15.86:0/4089105158 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x56300972b3f0 0x56300972d7e0 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.567+0000 7f10b802b740 1 --2- 172.21.15.86:0/4089105158 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5630096b4a00 0x5630096b8800 unknown :-1 s=CLOSED pgs=1408 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.567+0000 7f10967fc640 0 client.18402 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:45:21.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.567+0000 7f10b802b740 1 --2- 172.21.15.86:0/4089105158 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5630096abe80 0x5630096b82c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.567+0000 7f10b802b740 1 --2- 172.21.15.86:0/4089105158 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5630096ab4d0 0x5630096bcf50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.567+0000 7f10b802b740 1 -- 172.21.15.86:0/4089105158 >> 172.21.15.86:0/4089105158 conn(0x5630096a5ba0 msgr2=0x5630096a4370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:21.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.567+0000 7f10b802b740 1 -- 172.21.15.86:0/4089105158 shutdown_connections 2024-04-21T08:45:21.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.567+0000 7f10b802b740 1 -- 172.21.15.86:0/4089105158 wait complete. 2024-04-21T08:45:21.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.567+0000 7f10b802b740 20 client.18402 trim_cache size 0 max 16384 2024-04-21T08:45:21.574 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:45:21.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.567+0000 7f10b5359640 1 -- 172.21.15.86:0/2560502169 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f109c0a3d20 msgr2=0x7f109c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.567+0000 7f10b5359640 1 --2- 172.21.15.86:0/2560502169 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f109c0a3d20 0x7f109c0a61e0 secure :-1 s=READY pgs=224 cs=0 l=1 rev1=1 crypto rx=0x7f10a80a50a0 tx=0x7f10a806f450 comp rx=0 tx=0).stop 2024-04-21T08:45:21.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.567+0000 7f10b5359640 1 -- 172.21.15.86:0/2560502169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10b019a710 msgr2=0x7f10b019ab90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.567+0000 7f10b5359640 1 --2- 172.21.15.86:0/2560502169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10b019a710 0x7f10b019ab90 secure :-1 s=READY pgs=1404 cs=0 l=1 rev1=1 crypto rx=0x7f10a800eaa0 tx=0x7f10a800ef60 comp rx=0 tx=0).stop 2024-04-21T08:45:21.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.567+0000 7f10b5359640 1 -- 172.21.15.86:0/2560502169 shutdown_connections 2024-04-21T08:45:21.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.567+0000 7f10b5359640 1 --2- 172.21.15.86:0/2560502169 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f109c0a3d20 0x7f109c0a61e0 unknown :-1 s=CLOSED pgs=224 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.567+0000 7f10b5359640 1 --2- 172.21.15.86:0/2560502169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f10b019a710 0x7f10b019ab90 unknown :-1 s=CLOSED pgs=1404 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.567+0000 7f10b5359640 1 --2- 172.21.15.86:0/2560502169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f10b0107280 0x7f10b01a0b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.567+0000 7f10b5359640 1 --2- 172.21.15.86:0/2560502169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f10b006a7c0 0x7f10b01a0650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.567+0000 7f10b5359640 1 -- 172.21.15.86:0/2560502169 >> 172.21.15.86:0/2560502169 conn(0x7f10b0076c50 msgr2=0x7f10b010d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:21.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.567+0000 7f10b5359640 1 -- 172.21.15.86:0/2560502169 shutdown_connections 2024-04-21T08:45:21.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.567+0000 7f10b5359640 1 -- 172.21.15.86:0/2560502169 wait complete. 2024-04-21T08:45:21.578 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:45:21.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.677+0000 7fa3c8b54640 1 -- 172.21.15.86:0/470479982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3c4100af0 msgr2=0x7fa3c4100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.677+0000 7fa3c8b54640 1 --2- 172.21.15.86:0/470479982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3c4100af0 0x7fa3c4100ed0 secure :-1 s=READY pgs=1409 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4009900 tx=0x7fa3b402bc30 comp rx=0 tx=0).stop 2024-04-21T08:45:21.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.677+0000 7fa3c8b54640 1 -- 172.21.15.86:0/470479982 shutdown_connections 2024-04-21T08:45:21.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.677+0000 7fa3c8b54640 1 --2- 172.21.15.86:0/470479982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3c41072b0 0x7fa3c410eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.677+0000 7fa3c8b54640 1 --2- 172.21.15.86:0/470479982 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3c41014a0 0x7fa3c4106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.677+0000 7fa3c8b54640 1 --2- 172.21.15.86:0/470479982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3c4100af0 0x7fa3c4100ed0 unknown :-1 s=CLOSED pgs=1409 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.677+0000 7fa3c8b54640 1 -- 172.21.15.86:0/470479982 >> 172.21.15.86:0/470479982 conn(0x7fa3c40760e0 msgr2=0x7fa3c40764f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:21.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.677+0000 7fa3c8b54640 1 -- 172.21.15.86:0/470479982 shutdown_connections 2024-04-21T08:45:21.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.677+0000 7fa3c8b54640 1 -- 172.21.15.86:0/470479982 wait complete. 2024-04-21T08:45:21.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.678+0000 7fa3c8b54640 1 Processor -- start 2024-04-21T08:45:21.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.678+0000 7fa3c8b54640 1 -- start start 2024-04-21T08:45:21.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.678+0000 7fa3c8b54640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3c4100af0 0x7fa3c4198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.678+0000 7fa3c8b54640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3c41072b0 0x7fa3c4198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.678+0000 7fa3c8b54640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3c419d000 0x7fa3c419d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.678+0000 7fa3c8b54640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c4112260 con 0x7fa3c419d000 2024-04-21T08:45:21.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.678+0000 7fa3c8b54640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c41120e0 con 0x7fa3c4100af0 2024-04-21T08:45:21.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.678+0000 7fa3c8b54640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c41123e0 con 0x7fa3c41072b0 2024-04-21T08:45:21.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.678+0000 7fa3c2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3c419d000 0x7fa3c419d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.679+0000 7fa3c2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3c419d000 0x7fa3c419d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47910/0 (socket says 172.21.15.86:47910) 2024-04-21T08:45:21.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.679+0000 7fa3c2d9d640 1 -- 172.21.15.86:0/1781058977 learned_addr learned my addr 172.21.15.86:0/1781058977 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:21.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.679+0000 7fa3c259c640 1 --2- 172.21.15.86:0/1781058977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3c4100af0 0x7fa3c4198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.679+0000 7fa3c1d9b640 1 --2- 172.21.15.86:0/1781058977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3c41072b0 0x7fa3c4198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.679+0000 7fa3c2d9d640 1 -- 172.21.15.86:0/1781058977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3c41072b0 msgr2=0x7fa3c4198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.679+0000 7fa3c2d9d640 1 --2- 172.21.15.86:0/1781058977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3c41072b0 0x7fa3c4198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.679+0000 7fa3c2d9d640 1 -- 172.21.15.86:0/1781058977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3c4100af0 msgr2=0x7fa3c4198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.679+0000 7fa3c2d9d640 1 --2- 172.21.15.86:0/1781058977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3c4100af0 0x7fa3c4198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.679+0000 7fa3c2d9d640 1 -- 172.21.15.86:0/1781058977 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3c419daf0 con 0x7fa3c419d000 2024-04-21T08:45:21.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.679+0000 7fa3c2d9d640 1 --2- 172.21.15.86:0/1781058977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3c419d000 0x7fa3c419d4b0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00eaa0 tx=0x7fa3bc00ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:21.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.679+0000 7fa3b37fe640 1 -- 172.21.15.86:0/1781058977 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc00cdb0 con 0x7fa3c419d000 2024-04-21T08:45:21.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.679+0000 7fa3b37fe640 1 -- 172.21.15.86:0/1781058977 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3bc005020 con 0x7fa3c419d000 2024-04-21T08:45:21.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.679+0000 7fa3c8b54640 1 -- 172.21.15.86:0/1781058977 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3c419dde0 con 0x7fa3c419d000 2024-04-21T08:45:21.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.679+0000 7fa3b37fe640 1 -- 172.21.15.86:0/1781058977 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3bc0116d0 con 0x7fa3c419d000 2024-04-21T08:45:21.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.679+0000 7fa3c8b54640 1 -- 172.21.15.86:0/1781058977 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3c41a55d0 con 0x7fa3c419d000 2024-04-21T08:45:21.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.681+0000 7fa3b37fe640 1 -- 172.21.15.86:0/1781058977 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa3bc0051d0 con 0x7fa3c419d000 2024-04-21T08:45:21.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.682+0000 7fa3cb816740 1 Processor -- start 2024-04-21T08:45:21.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.682+0000 7fa3cb816740 1 -- start start 2024-04-21T08:45:21.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.682+0000 7fa3cb816740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577cbca44d0 0x5577cbca48b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.682+0000 7fa3cb816740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cbca4e80 0x5577cbcad430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.682+0000 7fa3cb816740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577cbcada00 0x5577cbcafe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.682+0000 7fa3cb816740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5577cbc091f0 con 0x5577cbca4e80 2024-04-21T08:45:21.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.682+0000 7fa3cb816740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5577cbba64b0 con 0x5577cbca44d0 2024-04-21T08:45:21.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.682+0000 7fa3cb816740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5577cbc051b0 con 0x5577cbcada00 2024-04-21T08:45:21.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.682+0000 7fa3c259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cbca4e80 0x5577cbcad430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.682+0000 7fa3c259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cbca4e80 0x5577cbcad430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47922/0 (socket says 172.21.15.86:47922) 2024-04-21T08:45:21.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.682+0000 7fa3c259c640 1 -- 172.21.15.86:0/3938037118 learned_addr learned my addr 172.21.15.86:0/3938037118 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:21.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.682+0000 7fa3b37fe640 1 --2- 172.21.15.86:0/1781058977 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa3ac0a3d20 0x7fa3ac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.682+0000 7fa3c1d9b640 1 --2- 172.21.15.86:0/3938037118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577cbca44d0 0x5577cbca48b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.683+0000 7fa3c1d9b640 1 --2- 172.21.15.86:0/3938037118 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577cbcada00 0x5577cbcafe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.683+0000 7fa3c2d9d640 1 --2- 172.21.15.86:0/1781058977 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa3ac0a3d20 0x7fa3ac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.683+0000 7fa3c259c640 1 -- 172.21.15.86:0/3938037118 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577cbcada00 msgr2=0x5577cbcafe50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.683+0000 7fa3c259c640 1 --2- 172.21.15.86:0/3938037118 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577cbcada00 0x5577cbcafe50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.683+0000 7fa3c259c640 1 -- 172.21.15.86:0/3938037118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577cbca44d0 msgr2=0x5577cbca48b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.683+0000 7fa3c259c640 1 --2- 172.21.15.86:0/3938037118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577cbca44d0 0x5577cbca48b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.683+0000 7fa3c259c640 1 -- 172.21.15.86:0/3938037118 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577cbb981b0 con 0x5577cbca4e80 2024-04-21T08:45:21.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.683+0000 7fa3b37fe640 1 -- 172.21.15.86:0/1781058977 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7fa3bc0d0e20 con 0x7fa3c419d000 2024-04-21T08:45:21.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.683+0000 7fa3c259c640 1 --2- 172.21.15.86:0/3938037118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cbca4e80 0x5577cbcad430 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4009900 tx=0x7fa3b402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bbcfa6498c322d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:21.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.683+0000 7fa3c2d9d640 1 --2- 172.21.15.86:0/1781058977 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa3ac0a3d20 0x7fa3ac0a61e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc09c690 tx=0x7fa3bc0bd040 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:21.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.683+0000 7fa3b0ff9640 1 -- 172.21.15.86:0/3938037118 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b4044020 con 0x5577cbca4e80 2024-04-21T08:45:21.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.683+0000 7fa3b0ff9640 1 -- 172.21.15.86:0/3938037118 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3b402cc90 con 0x5577cbca4e80 2024-04-21T08:45:21.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.683+0000 7fa3b0ff9640 1 -- 172.21.15.86:0/3938037118 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b40046c0 con 0x5577cbca4e80 2024-04-21T08:45:21.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.684+0000 7fa3cb816740 1 -- 172.21.15.86:0/3938037118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cbca4e80 msgr2=0x5577cbcad430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.684+0000 7fa3cb816740 1 --2- 172.21.15.86:0/3938037118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cbca4e80 0x5577cbcad430 secure :-1 s=READY pgs=1411 cs=0 l=1 rev1=1 crypto rx=0x7fa3b4009900 tx=0x7fa3b402c650 comp rx=0 tx=0).stop 2024-04-21T08:45:21.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.684+0000 7fa3cb816740 1 -- 172.21.15.86:0/3938037118 shutdown_connections 2024-04-21T08:45:21.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.684+0000 7fa3cb816740 1 --2- 172.21.15.86:0/3938037118 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577cbcada00 0x5577cbcafe50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.684+0000 7fa3cb816740 1 --2- 172.21.15.86:0/3938037118 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cbca4e80 0x5577cbcad430 unknown :-1 s=CLOSED pgs=1411 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.684+0000 7fa3cb816740 1 --2- 172.21.15.86:0/3938037118 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577cbca44d0 0x5577cbca48b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.684+0000 7fa3cb816740 1 -- 172.21.15.86:0/3938037118 >> 172.21.15.86:0/3938037118 conn(0x5577cbc9eba0 msgr2=0x5577cbc9ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:21.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.684+0000 7fa3cb816740 1 -- 172.21.15.86:0/3938037118 shutdown_connections 2024-04-21T08:45:21.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.684+0000 7fa3cb816740 1 -- 172.21.15.86:0/3938037118 wait complete. 2024-04-21T08:45:21.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.684+0000 7fa3cb816740 1 -- start start 2024-04-21T08:45:21.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.684+0000 7fa3cb816740 1 Processor -- start 2024-04-21T08:45:21.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.684+0000 7fa3cb816740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cbca44d0 0x5577cbcad260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.684+0000 7fa3cb816740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577cbca4e80 0x5577cbcaa5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.684+0000 7fa3cb816740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577cbcada00 0x5577cbcaab30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.685+0000 7fa3cb816740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5577cbcb0910 con 0x5577cbca44d0 2024-04-21T08:45:21.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.685+0000 7fa3cb816740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5577cbbc2500 con 0x5577cbca4e80 2024-04-21T08:45:21.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.685+0000 7fa3cb816740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5577cbcb0a90 con 0x5577cbcada00 2024-04-21T08:45:21.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.685+0000 7fa3c1d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cbca44d0 0x5577cbcad260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.685+0000 7fa3cb816740 10 client.0 _ll_register_callbacks cb 0x5577cbb90b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:21.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.685+0000 7fa3c1d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cbca44d0 0x5577cbcad260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47936/0 (socket says 172.21.15.86:47936) 2024-04-21T08:45:21.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.685+0000 7fa3c1d9b640 1 -- 172.21.15.86:0/4125966258 learned_addr learned my addr 172.21.15.86:0/4125966258 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:21.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.685+0000 7fa3c1d9b640 1 -- 172.21.15.86:0/4125966258 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577cbcada00 msgr2=0x5577cbcaab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.685+0000 7fa3c259c640 1 --2- 172.21.15.86:0/4125966258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577cbca4e80 0x5577cbcaa5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.685+0000 7fa3c1d9b640 1 --2- 172.21.15.86:0/4125966258 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577cbcada00 0x5577cbcaab30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.685+0000 7fa3c1d9b640 1 -- 172.21.15.86:0/4125966258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577cbca4e80 msgr2=0x5577cbcaa5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.685+0000 7fa3c1d9b640 1 --2- 172.21.15.86:0/4125966258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577cbca4e80 0x5577cbcaa5f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.685+0000 7fa3c1d9b640 1 -- 172.21.15.86:0/4125966258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577cb91b3f0 con 0x5577cbca44d0 2024-04-21T08:45:21.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.685+0000 7fa3c1d9b640 1 --2- 172.21.15.86:0/4125966258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cbca44d0 0x5577cbcad260 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80057b0 tx=0x7fa3b8010800 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6c7abcce2a406e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:21.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.685+0000 7fa3a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:21.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.685+0000 7fa3a9ffb640 1 -- 172.21.15.86:0/4125966258 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b801c670 con 0x5577cbca44d0 2024-04-21T08:45:21.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.685+0000 7fa3a9ffb640 1 -- 172.21.15.86:0/4125966258 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7fa3b801c810 con 0x5577cbca44d0 2024-04-21T08:45:21.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.685+0000 7fa3cb816740 1 -- 172.21.15.86:0/4125966258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3b402cc90 con 0x5577cbca44d0 2024-04-21T08:45:21.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.685+0000 7fa3a9ffb640 1 -- 172.21.15.86:0/4125966258 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa3b8025050 con 0x5577cbca44d0 2024-04-21T08:45:21.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.685+0000 7fa3cb816740 10 client.18414 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:21.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.685+0000 7fa3cb816740 1 -- 172.21.15.86:0/4125966258 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5577cb9e57d0 con 0x5577cbca44d0 2024-04-21T08:45:21.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.685+0000 7fa3a9ffb640 1 -- 172.21.15.86:0/4125966258 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3b402cc90 con 0x5577cbca44d0 2024-04-21T08:45:21.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.686+0000 7fa3a9ffb640 1 -- 172.21.15.86:0/4125966258 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa3b800f990 con 0x5577cbca44d0 2024-04-21T08:45:21.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.686+0000 7fa3cb816740 10 client.18414 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:21.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.686+0000 7fa3cb816740 10 client.18414 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:21.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.686+0000 7fa3cb816740 20 client.18414 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:21.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.686+0000 7fa3cb816740 1 --2- 172.21.15.86:0/4125966258 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5577cbcbdba0 0x5577cbcc0010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.686+0000 7fa3cb816740 4 client.18414 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:21.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.686+0000 7fa3cb816740 1 -- 172.21.15.86:0/4125966258 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5577cbba8930 con 0x5577cbcbdba0 2024-04-21T08:45:21.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.686+0000 7fa3c259c640 1 --2- 172.21.15.86:0/4125966258 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5577cbcbdba0 0x5577cbcc0010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.686+0000 7fa3a9ffb640 10 client.18414 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:21.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.686+0000 7fa3c259c640 1 --2- 172.21.15.86:0/4125966258 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5577cbcbdba0 0x5577cbcc0010 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93ed091f8e3224e2 server_cookie=67fb1f248f8860f4 in_seq=0 out_seq=0 2024-04-21T08:45:21.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.687+0000 7fa3a9ffb640 1 -- 172.21.15.86:0/4125966258 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5577cbba8930 con 0x5577cbcbdba0 2024-04-21T08:45:21.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.687+0000 7fa3a9ffb640 10 client.18414 handle_command_reply: tid=0 2024-04-21T08:45:21.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.687+0000 7fa3cb816740 1 client.18414 shutdown 2024-04-21T08:45:21.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.687+0000 7fa3cb816740 1 -- 172.21.15.86:0/4125966258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cbca44d0 msgr2=0x5577cbcad260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.687+0000 7fa3cb816740 1 --2- 172.21.15.86:0/4125966258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cbca44d0 0x5577cbcad260 secure :-1 s=READY pgs=1412 cs=0 l=1 rev1=1 crypto rx=0x7fa3b80057b0 tx=0x7fa3b8010800 comp rx=0 tx=0).stop 2024-04-21T08:45:21.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.687+0000 7fa3cb816740 1 -- 172.21.15.86:0/4125966258 shutdown_connections 2024-04-21T08:45:21.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.687+0000 7fa3cb816740 1 --2- 172.21.15.86:0/4125966258 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5577cbcbdba0 0x5577cbcc0010 crc :-1 s=READY pgs=207 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.687+0000 7fa3cb816740 1 --2- 172.21.15.86:0/4125966258 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577cbcada00 0x5577cbcaab30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.687+0000 7fa3a9ffb640 0 client.18414 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:21.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.687+0000 7fa3cb816740 1 --2- 172.21.15.86:0/4125966258 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577cbca4e80 0x5577cbcaa5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.687+0000 7fa3cb816740 1 --2- 172.21.15.86:0/4125966258 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cbca44d0 0x5577cbcad260 unknown :-1 s=CLOSED pgs=1412 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.687+0000 7fa3cb816740 1 -- 172.21.15.86:0/4125966258 >> 172.21.15.86:0/4125966258 conn(0x5577cbc9eba0 msgr2=0x5577cbca08f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:21.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.687+0000 7fa3cb816740 1 -- 172.21.15.86:0/4125966258 shutdown_connections 2024-04-21T08:45:21.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.687+0000 7fa3cb816740 1 -- 172.21.15.86:0/4125966258 wait complete. 2024-04-21T08:45:21.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.687+0000 7fa3cb816740 20 client.18414 trim_cache size 0 max 16384 2024-04-21T08:45:21.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.700+0000 7fa3cb816740 1 Processor -- start 2024-04-21T08:45:21.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.700+0000 7fa3cb816740 1 -- start start 2024-04-21T08:45:21.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.701+0000 7fa3cb816740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577cbca44d0 0x5577cbcb9800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.701+0000 7fa3cb816740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577cbca4e80 0x5577cbca5b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.701+0000 7fa3cb816740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cbcada00 0x5577cbcacd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.701+0000 7fa3cb816740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c41af3c0 con 0x5577cbcada00 2024-04-21T08:45:21.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.701+0000 7fa3cb816740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c41b0f00 con 0x5577cbca4e80 2024-04-21T08:45:21.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.701+0000 7fa3cb816740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c41afa90 con 0x5577cbca44d0 2024-04-21T08:45:21.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.701+0000 7fa3c1d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cbcada00 0x5577cbcacd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.701+0000 7fa3c1d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cbcada00 0x5577cbcacd20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47952/0 (socket says 172.21.15.86:47952) 2024-04-21T08:45:21.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.701+0000 7fa3c1d9b640 1 -- 172.21.15.86:0/3644458025 learned_addr learned my addr 172.21.15.86:0/3644458025 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:21.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.701+0000 7fa3c1d9b640 1 --2- 172.21.15.86:0/3644458025 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577cbca44d0 0x5577cbcb9800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.701+0000 7fa3c259c640 1 --2- 172.21.15.86:0/3644458025 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577cbca4e80 0x5577cbca5b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.701+0000 7fa3c1d9b640 1 -- 172.21.15.86:0/3644458025 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577cbca44d0 msgr2=0x5577cbcb9800 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.701+0000 7fa3c1d9b640 1 --2- 172.21.15.86:0/3644458025 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577cbca44d0 0x5577cbcb9800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.701+0000 7fa3c1d9b640 1 -- 172.21.15.86:0/3644458025 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577cbca4e80 msgr2=0x5577cbca5b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.701+0000 7fa3c1d9b640 1 --2- 172.21.15.86:0/3644458025 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577cbca4e80 0x5577cbca5b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.701+0000 7fa3c1d9b640 1 -- 172.21.15.86:0/3644458025 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5577cbcaae90 con 0x5577cbcada00 2024-04-21T08:45:21.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.701+0000 7fa3c259c640 1 --2- 172.21.15.86:0/3644458025 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577cbca4e80 0x5577cbca5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:21.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.701+0000 7fa3c1d9b640 1 --2- 172.21.15.86:0/3644458025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cbcada00 0x5577cbcacd20 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fa3b800fd60 tx=0x7fa3b8001e70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1b346dcfb1845f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:21.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.701+0000 7fa3abfff640 1 -- 172.21.15.86:0/3644458025 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b801a370 con 0x5577cbcada00 2024-04-21T08:45:21.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.701+0000 7fa3abfff640 1 -- 172.21.15.86:0/3644458025 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3b8017070 con 0x5577cbcada00 2024-04-21T08:45:21.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.701+0000 7fa3abfff640 1 -- 172.21.15.86:0/3644458025 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b800f740 con 0x5577cbcada00 2024-04-21T08:45:21.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.702+0000 7fa3cb816740 1 -- 172.21.15.86:0/3644458025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cbcada00 msgr2=0x5577cbcacd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.702+0000 7fa3cb816740 1 --2- 172.21.15.86:0/3644458025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cbcada00 0x5577cbcacd20 secure :-1 s=READY pgs=1413 cs=0 l=1 rev1=1 crypto rx=0x7fa3b800fd60 tx=0x7fa3b8001e70 comp rx=0 tx=0).stop 2024-04-21T08:45:21.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.702+0000 7fa3cb816740 1 -- 172.21.15.86:0/3644458025 shutdown_connections 2024-04-21T08:45:21.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.702+0000 7fa3cb816740 1 --2- 172.21.15.86:0/3644458025 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cbcada00 0x5577cbcacd20 unknown :-1 s=CLOSED pgs=1413 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.702+0000 7fa3cb816740 1 --2- 172.21.15.86:0/3644458025 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577cbca4e80 0x5577cbca5b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.702+0000 7fa3cb816740 1 --2- 172.21.15.86:0/3644458025 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577cbca44d0 0x5577cbcb9800 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.702+0000 7fa3cb816740 1 -- 172.21.15.86:0/3644458025 >> 172.21.15.86:0/3644458025 conn(0x5577cbc9eba0 msgr2=0x5577cbc9dc10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:21.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.702+0000 7fa3cb816740 1 -- 172.21.15.86:0/3644458025 shutdown_connections 2024-04-21T08:45:21.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.702+0000 7fa3cb816740 1 -- 172.21.15.86:0/3644458025 wait complete. 2024-04-21T08:45:21.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.702+0000 7fa3cb816740 1 -- start start 2024-04-21T08:45:21.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.702+0000 7fa3cb816740 1 Processor -- start 2024-04-21T08:45:21.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.702+0000 7fa3cb816740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577cbca44d0 0x5577cbcb60d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.702+0000 7fa3cb816740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577cbca4e80 0x5577cbcb1440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.702+0000 7fa3cb816740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cbcada00 0x5577cbcb1980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.702+0000 7fa3cb816740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c41b0830 con 0x5577cbcada00 2024-04-21T08:45:21.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.702+0000 7fa3cb816740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c41b0160 con 0x5577cbca4e80 2024-04-21T08:45:21.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.702+0000 7fa3cb816740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa3c41ad3c0 con 0x5577cbca44d0 2024-04-21T08:45:21.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.703+0000 7fa3cb816740 10 client.0 _ll_register_callbacks cb 0x7fa3b8011b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:21.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.703+0000 7fa3c1d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cbcada00 0x5577cbcb1980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.703+0000 7fa3c1d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cbcada00 0x5577cbcb1980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47960/0 (socket says 172.21.15.86:47960) 2024-04-21T08:45:21.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.703+0000 7fa3c1d9b640 1 -- 172.21.15.86:0/4117074490 learned_addr learned my addr 172.21.15.86:0/4117074490 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:21.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.703+0000 7fa3c259c640 1 --2- 172.21.15.86:0/4117074490 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577cbca4e80 0x5577cbcb1440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.703+0000 7fa3c1d9b640 1 -- 172.21.15.86:0/4117074490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577cbca44d0 msgr2=0x5577cbcb60d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.703+0000 7fa3c1d9b640 1 --2- 172.21.15.86:0/4117074490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577cbca44d0 0x5577cbcb60d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.703+0000 7fa3c1d9b640 1 -- 172.21.15.86:0/4117074490 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577cbca4e80 msgr2=0x5577cbcb1440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:21.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.703+0000 7fa3c1d9b640 1 --2- 172.21.15.86:0/4117074490 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577cbca4e80 0x5577cbcb1440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.703+0000 7fa3c1d9b640 1 -- 172.21.15.86:0/4117074490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5577cbcb2160 con 0x5577cbcada00 2024-04-21T08:45:21.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.703+0000 7fa3c1d9b640 1 --2- 172.21.15.86:0/4117074490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cbcada00 0x5577cbcb1980 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8005ee0 tx=0x7fa3b80118e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=68690736ed0eee3e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:21.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.703+0000 7fa3a9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:21.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.703+0000 7fa3a9ffb640 1 -- 172.21.15.86:0/4117074490 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3b801a970 con 0x5577cbcada00 2024-04-21T08:45:21.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.703+0000 7fa3a9ffb640 1 -- 172.21.15.86:0/4117074490 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7fa3b801ab10 con 0x5577cbcada00 2024-04-21T08:45:21.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.703+0000 7fa3cb816740 1 -- 172.21.15.86:0/4117074490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa3b8017070 con 0x5577cbcada00 2024-04-21T08:45:21.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.703+0000 7fa3a9ffb640 1 -- 172.21.15.86:0/4117074490 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa3b800f3c0 con 0x5577cbcada00 2024-04-21T08:45:21.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.703+0000 7fa3cb816740 10 client.18420 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:21.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.703+0000 7fa3cb816740 1 -- 172.21.15.86:0/4117074490 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5577cbd284e0 con 0x5577cbcada00 2024-04-21T08:45:21.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.703+0000 7fa3a9ffb640 1 -- 172.21.15.86:0/4117074490 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3b8017070 con 0x5577cbcada00 2024-04-21T08:45:21.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.704+0000 7fa3a9ffb640 1 -- 172.21.15.86:0/4117074490 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa3b801ca20 con 0x5577cbcada00 2024-04-21T08:45:21.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.704+0000 7fa3cb816740 10 client.18420 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:21.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.704+0000 7fa3cb816740 10 client.18420 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:21.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.704+0000 7fa3cb816740 20 client.18420 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:21.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.704+0000 7fa3cb816740 1 --2- 172.21.15.86:0/4117074490 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5577cbd28b40 0x5577cbd2af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:21.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.704+0000 7fa3cb816740 4 client.18420 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:45:21.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.704+0000 7fa3cb816740 1 -- 172.21.15.86:0/4117074490 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5577cbc16c10 con 0x5577cbd28b40 2024-04-21T08:45:21.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.704+0000 7fa3c259c640 1 --2- 172.21.15.86:0/4117074490 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5577cbd28b40 0x5577cbd2af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:21.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.704+0000 7fa3a9ffb640 10 client.18420 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:21.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.704+0000 7fa3c259c640 1 --2- 172.21.15.86:0/4117074490 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5577cbd28b40 0x5577cbd2af60 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d62cb7444ab8737b server_cookie=503e28da80e0f791 in_seq=0 out_seq=0 2024-04-21T08:45:21.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.704+0000 7fa3a9ffb640 1 -- 172.21.15.86:0/4117074490 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5577cbc16c10 con 0x5577cbd28b40 2024-04-21T08:45:21.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.704+0000 7fa3a9ffb640 10 client.18420 handle_command_reply: tid=0 2024-04-21T08:45:21.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.704+0000 7fa3cb816740 1 client.18420 shutdown 2024-04-21T08:45:21.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.704+0000 7fa3cb816740 1 -- 172.21.15.86:0/4117074490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cbcada00 msgr2=0x5577cbcb1980 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.704+0000 7fa3cb816740 1 --2- 172.21.15.86:0/4117074490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cbcada00 0x5577cbcb1980 secure :-1 s=READY pgs=1414 cs=0 l=1 rev1=1 crypto rx=0x7fa3b8005ee0 tx=0x7fa3b80118e0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.704+0000 7fa3cb816740 1 -- 172.21.15.86:0/4117074490 shutdown_connections 2024-04-21T08:45:21.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.704+0000 7fa3cb816740 1 --2- 172.21.15.86:0/4117074490 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5577cbd28b40 0x5577cbd2af60 crc :-1 s=READY pgs=208 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.704+0000 7fa3cb816740 1 --2- 172.21.15.86:0/4117074490 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5577cbcada00 0x5577cbcb1980 unknown :-1 s=CLOSED pgs=1414 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.704+0000 7fa3a9ffb640 0 client.18420 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:21.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.704+0000 7fa3cb816740 1 --2- 172.21.15.86:0/4117074490 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5577cbca4e80 0x5577cbcb1440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.704+0000 7fa3cb816740 1 --2- 172.21.15.86:0/4117074490 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5577cbca44d0 0x5577cbcb60d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.704+0000 7fa3cb816740 1 -- 172.21.15.86:0/4117074490 >> 172.21.15.86:0/4117074490 conn(0x5577cbc9eba0 msgr2=0x5577cbcaa710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:21.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.705+0000 7fa3cb816740 1 -- 172.21.15.86:0/4117074490 shutdown_connections 2024-04-21T08:45:21.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.705+0000 7fa3cb816740 1 -- 172.21.15.86:0/4117074490 wait complete. 2024-04-21T08:45:21.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.705+0000 7fa3cb816740 20 client.18420 trim_cache size 0 max 16384 2024-04-21T08:45:21.712 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:45:21.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.705+0000 7fa3c8b54640 1 -- 172.21.15.86:0/1781058977 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa3ac0a3d20 msgr2=0x7fa3ac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.705+0000 7fa3c8b54640 1 --2- 172.21.15.86:0/1781058977 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa3ac0a3d20 0x7fa3ac0a61e0 secure :-1 s=READY pgs=225 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc09c690 tx=0x7fa3bc0bd040 comp rx=0 tx=0).stop 2024-04-21T08:45:21.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.705+0000 7fa3c8b54640 1 -- 172.21.15.86:0/1781058977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3c419d000 msgr2=0x7fa3c419d4b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:21.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.705+0000 7fa3c8b54640 1 --2- 172.21.15.86:0/1781058977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3c419d000 0x7fa3c419d4b0 secure :-1 s=READY pgs=1410 cs=0 l=1 rev1=1 crypto rx=0x7fa3bc00eaa0 tx=0x7fa3bc00ef60 comp rx=0 tx=0).stop 2024-04-21T08:45:21.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.705+0000 7fa3c8b54640 1 -- 172.21.15.86:0/1781058977 shutdown_connections 2024-04-21T08:45:21.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.705+0000 7fa3c8b54640 1 --2- 172.21.15.86:0/1781058977 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa3ac0a3d20 0x7fa3ac0a61e0 unknown :-1 s=CLOSED pgs=225 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.705+0000 7fa3c8b54640 1 --2- 172.21.15.86:0/1781058977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3c419d000 0x7fa3c419d4b0 unknown :-1 s=CLOSED pgs=1410 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.705+0000 7fa3c8b54640 1 --2- 172.21.15.86:0/1781058977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3c41072b0 0x7fa3c4198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.705+0000 7fa3c8b54640 1 --2- 172.21.15.86:0/1781058977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3c4100af0 0x7fa3c4198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:21.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.705+0000 7fa3c8b54640 1 -- 172.21.15.86:0/1781058977 >> 172.21.15.86:0/1781058977 conn(0x7fa3c40760e0 msgr2=0x7fa3c40feaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:21.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.705+0000 7fa3c8b54640 1 -- 172.21.15.86:0/1781058977 shutdown_connections 2024-04-21T08:45:21.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:21.705+0000 7fa3c8b54640 1 -- 172.21.15.86:0/1781058977 wait complete. 2024-04-21T08:45:21.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:21 smithi086 ceph-mon[27939]: pgmap v202: 129 pgs: 129 active+clean; 2.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-21T08:45:21.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:21 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/719136826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:45:22.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:21 smithi088 ceph-mon[31259]: pgmap v202: 129 pgs: 129 active+clean; 2.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-21T08:45:22.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:21 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/719136826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:45:22.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:21 smithi179 ceph-mon[31829]: pgmap v202: 129 pgs: 129 active+clean; 2.3 MiB data, 401 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-21T08:45:22.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:21 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/719136826' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:45:22.716 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:45:22.717 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:45:22.717 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:45:22.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.832+0000 7f557dfbd640 1 -- 172.21.15.86:0/1103390238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f557806a860 msgr2=0x7f557806ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:22.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.832+0000 7f557dfbd640 1 --2- 172.21.15.86:0/1103390238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f557806a860 0x7f557806ac40 secure :-1 s=READY pgs=1415 cs=0 l=1 rev1=1 crypto rx=0x7f556c009900 tx=0x7f556c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:45:22.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.832+0000 7f557dfbd640 1 -- 172.21.15.86:0/1103390238 shutdown_connections 2024-04-21T08:45:22.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.832+0000 7f557dfbd640 1 --2- 172.21.15.86:0/1103390238 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5578107270 0x7f557810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.832+0000 7f557dfbd640 1 --2- 172.21.15.86:0/1103390238 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f557806b210 0x7f5578106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.832+0000 7f557dfbd640 1 --2- 172.21.15.86:0/1103390238 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f557806a860 0x7f557806ac40 unknown :-1 s=CLOSED pgs=1415 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.832+0000 7f557dfbd640 1 -- 172.21.15.86:0/1103390238 >> 172.21.15.86:0/1103390238 conn(0x7f5578077eb0 msgr2=0x7f55780782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:22.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.832+0000 7f557dfbd640 1 -- 172.21.15.86:0/1103390238 shutdown_connections 2024-04-21T08:45:22.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.832+0000 7f557dfbd640 1 -- 172.21.15.86:0/1103390238 wait complete. 2024-04-21T08:45:22.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.832+0000 7f557dfbd640 1 Processor -- start 2024-04-21T08:45:22.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.833+0000 7f557dfbd640 1 -- start start 2024-04-21T08:45:22.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.833+0000 7f557dfbd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f557806a860 0x7f5578198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:22.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.833+0000 7f557dfbd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5578107270 0x7f5578198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:22.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.833+0000 7f557dfbd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f557819d020 0x7f557819d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:22.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.833+0000 7f557dfbd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5578112280 con 0x7f557806a860 2024-04-21T08:45:22.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.833+0000 7f557dfbd640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5578112100 con 0x7f5578107270 2024-04-21T08:45:22.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.833+0000 7f557dfbd640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5578112400 con 0x7f557819d020 2024-04-21T08:45:22.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.833+0000 7f55777fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f557806a860 0x7f5578198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:22.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.833+0000 7f55777fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f557806a860 0x7f5578198760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47978/0 (socket says 172.21.15.86:47978) 2024-04-21T08:45:22.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.833+0000 7f55777fe640 1 -- 172.21.15.86:0/1529802690 learned_addr learned my addr 172.21.15.86:0/1529802690 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:22.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.833+0000 7f5577fff640 1 --2- 172.21.15.86:0/1529802690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f557819d020 0x7f557819d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:22.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.833+0000 7f5576ffd640 1 --2- 172.21.15.86:0/1529802690 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5578107270 0x7f5578198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:22.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.833+0000 7f55777fe640 1 -- 172.21.15.86:0/1529802690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f557819d020 msgr2=0x7f557819d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:22.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.833+0000 7f55777fe640 1 --2- 172.21.15.86:0/1529802690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f557819d020 0x7f557819d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.833+0000 7f55777fe640 1 -- 172.21.15.86:0/1529802690 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5578107270 msgr2=0x7f5578198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:22.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.833+0000 7f55777fe640 1 --2- 172.21.15.86:0/1529802690 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5578107270 0x7f5578198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.833+0000 7f55777fe640 1 -- 172.21.15.86:0/1529802690 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f557819db10 con 0x7f557806a860 2024-04-21T08:45:22.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.833+0000 7f5576ffd640 1 --2- 172.21.15.86:0/1529802690 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5578107270 0x7f5578198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:22.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.833+0000 7f5577fff640 1 --2- 172.21.15.86:0/1529802690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f557819d020 0x7f557819d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:22.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.833+0000 7f55777fe640 1 --2- 172.21.15.86:0/1529802690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f557806a860 0x7f5578198760 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f556c005ab0 tx=0x7f556c02ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:22.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.834+0000 7f5574ff9640 1 -- 172.21.15.86:0/1529802690 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556c043020 con 0x7f557806a860 2024-04-21T08:45:22.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.834+0000 7f557dfbd640 1 -- 172.21.15.86:0/1529802690 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f557819dda0 con 0x7f557806a860 2024-04-21T08:45:22.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.834+0000 7f5574ff9640 1 -- 172.21.15.86:0/1529802690 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f556c02cc80 con 0x7f557806a860 2024-04-21T08:45:22.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.834+0000 7f5574ff9640 1 -- 172.21.15.86:0/1529802690 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556c003d40 con 0x7f557806a860 2024-04-21T08:45:22.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.834+0000 7f557dfbd640 1 -- 172.21.15.86:0/1529802690 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f55781a55c0 con 0x7f557806a860 2024-04-21T08:45:22.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.836+0000 7f5574ff9640 1 -- 172.21.15.86:0/1529802690 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f556c046020 con 0x7f557806a860 2024-04-21T08:45:22.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.836+0000 7f5580bb5740 1 Processor -- start 2024-04-21T08:45:22.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.836+0000 7f5580bb5740 1 -- start start 2024-04-21T08:45:22.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.836+0000 7f5574ff9640 1 --2- 172.21.15.86:0/1529802690 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f55600a3d20 0x7f55600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:22.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.836+0000 7f5580bb5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588276c4740 0x5588276c4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:22.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.836+0000 7f5580bb5740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588276c50f0 0x5588276cdc30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:22.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.836+0000 7f5580bb5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588276ce250 0x5588276ce6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:22.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.836+0000 7f5580bb5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558827628f00 con 0x5588276ce250 2024-04-21T08:45:22.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.836+0000 7f5580bb5740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5588275c6590 con 0x5588276c50f0 2024-04-21T08:45:22.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.836+0000 7f5580bb5740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558827624ec0 con 0x5588276c4740 2024-04-21T08:45:22.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.836+0000 7f5574ff9640 1 -- 172.21.15.86:0/1529802690 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f556c0f2f00 con 0x7f557806a860 2024-04-21T08:45:22.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.836+0000 7f5577fff640 1 --2- 172.21.15.86:0/1529802690 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f55600a3d20 0x7f55600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:22.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.837+0000 7f5577fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588276ce250 0x5588276ce6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:22.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.837+0000 7f5577fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588276ce250 0x5588276ce6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47982/0 (socket says 172.21.15.86:47982) 2024-04-21T08:45:22.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.837+0000 7f5577fff640 1 -- 172.21.15.86:0/96805083 learned_addr learned my addr 172.21.15.86:0/96805083 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:22.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.837+0000 7f55777fe640 1 --2- 172.21.15.86:0/96805083 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588276c4740 0x5588276c4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:22.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.837+0000 7f5576ffd640 1 --2- 172.21.15.86:0/96805083 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588276c50f0 0x5588276cdc30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:22.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.837+0000 7f5577fff640 1 -- 172.21.15.86:0/96805083 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588276c4740 msgr2=0x5588276c4b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:22.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.837+0000 7f5577fff640 1 --2- 172.21.15.86:0/96805083 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588276c4740 0x5588276c4b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.837+0000 7f5577fff640 1 -- 172.21.15.86:0/96805083 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588276c50f0 msgr2=0x5588276cdc30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:22.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.837+0000 7f5577fff640 1 --2- 172.21.15.86:0/96805083 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588276c50f0 0x5588276cdc30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.837+0000 7f5577fff640 1 -- 172.21.15.86:0/96805083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588274037e0 con 0x5588276ce250 2024-04-21T08:45:22.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.837+0000 7f5577fff640 1 --2- 172.21.15.86:0/1529802690 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f55600a3d20 0x7f55600a61e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5570013c50 tx=0x7f5570007e30 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:22.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.837+0000 7f5577fff640 1 --2- 172.21.15.86:0/96805083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588276ce250 0x5588276ce6d0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f5570010c30 tx=0x7f557001a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2f3e83a6a5a89b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:22.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.837+0000 7f555dffb640 1 -- 172.21.15.86:0/96805083 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5570021070 con 0x5588276ce250 2024-04-21T08:45:22.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.837+0000 7f555dffb640 1 -- 172.21.15.86:0/96805083 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5570013270 con 0x5588276ce250 2024-04-21T08:45:22.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.837+0000 7f555dffb640 1 -- 172.21.15.86:0/96805083 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5570005600 con 0x5588276ce250 2024-04-21T08:45:22.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.838+0000 7f5580bb5740 1 -- 172.21.15.86:0/96805083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588276ce250 msgr2=0x5588276ce6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:22.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.838+0000 7f5580bb5740 1 --2- 172.21.15.86:0/96805083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588276ce250 0x5588276ce6d0 secure :-1 s=READY pgs=1417 cs=0 l=1 rev1=1 crypto rx=0x7f5570010c30 tx=0x7f557001a000 comp rx=0 tx=0).stop 2024-04-21T08:45:22.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.838+0000 7f5580bb5740 1 -- 172.21.15.86:0/96805083 shutdown_connections 2024-04-21T08:45:22.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.838+0000 7f5580bb5740 1 --2- 172.21.15.86:0/96805083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588276ce250 0x5588276ce6d0 unknown :-1 s=CLOSED pgs=1417 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.838+0000 7f5580bb5740 1 --2- 172.21.15.86:0/96805083 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588276c50f0 0x5588276cdc30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.838+0000 7f5580bb5740 1 --2- 172.21.15.86:0/96805083 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588276c4740 0x5588276c4b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.838+0000 7f5580bb5740 1 -- 172.21.15.86:0/96805083 >> 172.21.15.86:0/96805083 conn(0x5588276befb0 msgr2=0x5588276bf390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:22.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.838+0000 7f5580bb5740 1 -- 172.21.15.86:0/96805083 shutdown_connections 2024-04-21T08:45:22.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.838+0000 7f5580bb5740 1 -- 172.21.15.86:0/96805083 wait complete. 2024-04-21T08:45:22.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.838+0000 7f5580bb5740 1 -- start start 2024-04-21T08:45:22.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.839+0000 7f5580bb5740 1 Processor -- start 2024-04-21T08:45:22.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.839+0000 7f5580bb5740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588276c4740 0x5588276d1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:22.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.839+0000 7f5580bb5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588276c50f0 0x5588276d2270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:22.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.839+0000 7f5580bb5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588276ce250 0x5588276d27b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:22.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.839+0000 7f5580bb5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5588276d5170 con 0x5588276c50f0 2024-04-21T08:45:22.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.839+0000 7f5580bb5740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5588275e2580 con 0x5588276c4740 2024-04-21T08:45:22.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.839+0000 7f5580bb5740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5588276d52f0 con 0x5588276ce250 2024-04-21T08:45:22.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.839+0000 7f5580bb5740 10 client.0 _ll_register_callbacks cb 0x5588275e8650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:22.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.839+0000 7f5576ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588276c50f0 0x5588276d2270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:22.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.839+0000 7f5576ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588276c50f0 0x5588276d2270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47984/0 (socket says 172.21.15.86:47984) 2024-04-21T08:45:22.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.839+0000 7f5576ffd640 1 -- 172.21.15.86:0/2378258146 learned_addr learned my addr 172.21.15.86:0/2378258146 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:22.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.839+0000 7f55777fe640 1 --2- 172.21.15.86:0/2378258146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588276c4740 0x5588276d1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:22.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.839+0000 7f5577fff640 1 --2- 172.21.15.86:0/2378258146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588276ce250 0x5588276d27b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:22.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.839+0000 7f5576ffd640 1 -- 172.21.15.86:0/2378258146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588276ce250 msgr2=0x5588276d27b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:22.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.839+0000 7f5576ffd640 1 --2- 172.21.15.86:0/2378258146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588276ce250 0x5588276d27b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.839+0000 7f5576ffd640 1 -- 172.21.15.86:0/2378258146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588276c4740 msgr2=0x5588276d1d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:22.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.839+0000 7f5576ffd640 1 --2- 172.21.15.86:0/2378258146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588276c4740 0x5588276d1d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.839+0000 7f5576ffd640 1 -- 172.21.15.86:0/2378258146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558827463f70 con 0x5588276c50f0 2024-04-21T08:45:22.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.839+0000 7f5576ffd640 1 --2- 172.21.15.86:0/2378258146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588276c50f0 0x5588276d2270 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f556800f520 tx=0x7f556800f9e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a35e344e6e01bd4f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:22.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.839+0000 7f5556ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:22.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.840+0000 7f5556ffd640 1 -- 172.21.15.86:0/2378258146 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5568013850 con 0x5588276c50f0 2024-04-21T08:45:22.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.840+0000 7f5556ffd640 1 -- 172.21.15.86:0/2378258146 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f55680139f0 con 0x5588276c50f0 2024-04-21T08:45:22.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.840+0000 7f5580bb5740 1 -- 172.21.15.86:0/2378258146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5570013270 con 0x5588276c50f0 2024-04-21T08:45:22.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.840+0000 7f5556ffd640 1 -- 172.21.15.86:0/2378258146 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5568013bf0 con 0x5588276c50f0 2024-04-21T08:45:22.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.840+0000 7f5580bb5740 10 client.18432 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:22.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.840+0000 7f5580bb5740 1 -- 172.21.15.86:0/2378258146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5588272fb170 con 0x5588276c50f0 2024-04-21T08:45:22.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.840+0000 7f5556ffd640 1 -- 172.21.15.86:0/2378258146 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5570013270 con 0x5588276c50f0 2024-04-21T08:45:22.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.840+0000 7f5556ffd640 1 -- 172.21.15.86:0/2378258146 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f55680113e0 con 0x5588276c50f0 2024-04-21T08:45:22.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.840+0000 7f5580bb5740 10 client.18432 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:22.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.840+0000 7f5580bb5740 10 client.18432 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:22.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.840+0000 7f5580bb5740 20 client.18432 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:22.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.840+0000 7f5580bb5740 1 --2- 172.21.15.86:0/2378258146 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5588276e2890 0x5588276e4ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:22.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.840+0000 7f5580bb5740 4 client.18432 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:22.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.840+0000 7f5580bb5740 1 -- 172.21.15.86:0/2378258146 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558827636920 con 0x5588276e2890 2024-04-21T08:45:22.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.840+0000 7f55777fe640 1 --2- 172.21.15.86:0/2378258146 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5588276e2890 0x5588276e4ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:22.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.841+0000 7f55777fe640 1 --2- 172.21.15.86:0/2378258146 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5588276e2890 0x5588276e4ce0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4ca68d6562812510 server_cookie=aa660cc2fa0a3558 in_seq=0 out_seq=0 2024-04-21T08:45:22.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.841+0000 7f5556ffd640 10 client.18432 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:22.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.841+0000 7f5556ffd640 1 -- 172.21.15.86:0/2378258146 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558827636920 con 0x5588276e2890 2024-04-21T08:45:22.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.841+0000 7f5556ffd640 10 client.18432 handle_command_reply: tid=0 2024-04-21T08:45:22.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.841+0000 7f5580bb5740 1 client.18432 shutdown 2024-04-21T08:45:22.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.842+0000 7f5580bb5740 1 -- 172.21.15.86:0/2378258146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588276c50f0 msgr2=0x5588276d2270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:22.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.842+0000 7f5580bb5740 1 --2- 172.21.15.86:0/2378258146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588276c50f0 0x5588276d2270 secure :-1 s=READY pgs=1418 cs=0 l=1 rev1=1 crypto rx=0x7f556800f520 tx=0x7f556800f9e0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.842+0000 7f5580bb5740 1 -- 172.21.15.86:0/2378258146 shutdown_connections 2024-04-21T08:45:22.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.842+0000 7f5580bb5740 1 --2- 172.21.15.86:0/2378258146 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5588276e2890 0x5588276e4ce0 crc :-1 s=READY pgs=209 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.842+0000 7f5580bb5740 1 --2- 172.21.15.86:0/2378258146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588276ce250 0x5588276d27b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.842+0000 7f5580bb5740 1 --2- 172.21.15.86:0/2378258146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588276c50f0 0x5588276d2270 unknown :-1 s=CLOSED pgs=1418 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.842+0000 7f5580bb5740 1 --2- 172.21.15.86:0/2378258146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588276c4740 0x5588276d1d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.842+0000 7f5556ffd640 0 client.18432 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:22.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.842+0000 7f5580bb5740 1 -- 172.21.15.86:0/2378258146 >> 172.21.15.86:0/2378258146 conn(0x5588276befb0 msgr2=0x5588276c0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:22.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.842+0000 7f5580bb5740 1 -- 172.21.15.86:0/2378258146 shutdown_connections 2024-04-21T08:45:22.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.842+0000 7f5580bb5740 1 -- 172.21.15.86:0/2378258146 wait complete. 2024-04-21T08:45:22.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.842+0000 7f5580bb5740 20 client.18432 trim_cache size 0 max 16384 2024-04-21T08:45:22.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.859+0000 7f5580bb5740 1 Processor -- start 2024-04-21T08:45:22.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.859+0000 7f5580bb5740 1 -- start start 2024-04-21T08:45:22.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.859+0000 7f5580bb5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588276befb0 0x5588276d4460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:22.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.859+0000 7f5580bb5740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588276c4740 0x5588276de0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:22.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.859+0000 7f5580bb5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588276c50f0 0x5588276cf6a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:22.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.859+0000 7f5580bb5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f55781ad0f0 con 0x5588276befb0 2024-04-21T08:45:22.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.859+0000 7f5580bb5740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f55781aec30 con 0x5588276c4740 2024-04-21T08:45:22.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.859+0000 7f5580bb5740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f55781ad7c0 con 0x5588276c50f0 2024-04-21T08:45:22.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.859+0000 7f55777fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588276befb0 0x5588276d4460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:22.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.859+0000 7f55777fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588276befb0 0x5588276d4460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48000/0 (socket says 172.21.15.86:48000) 2024-04-21T08:45:22.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.859+0000 7f55777fe640 1 -- 172.21.15.86:0/772307780 learned_addr learned my addr 172.21.15.86:0/772307780 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:22.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.860+0000 7f5576ffd640 1 --2- 172.21.15.86:0/772307780 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588276c4740 0x5588276de0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:22.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.860+0000 7f5577fff640 1 --2- 172.21.15.86:0/772307780 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588276c50f0 0x5588276cf6a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:22.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.860+0000 7f55777fe640 1 -- 172.21.15.86:0/772307780 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588276c50f0 msgr2=0x5588276cf6a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:22.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.860+0000 7f55777fe640 1 --2- 172.21.15.86:0/772307780 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588276c50f0 0x5588276cf6a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.860+0000 7f55777fe640 1 -- 172.21.15.86:0/772307780 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588276c4740 msgr2=0x5588276de0f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:22.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.860+0000 7f55777fe640 1 --2- 172.21.15.86:0/772307780 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588276c4740 0x5588276de0f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.860+0000 7f55777fe640 1 -- 172.21.15.86:0/772307780 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5588276be790 con 0x5588276befb0 2024-04-21T08:45:22.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.860+0000 7f5577fff640 1 --2- 172.21.15.86:0/772307780 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588276c50f0 0x5588276cf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:22.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.860+0000 7f55777fe640 1 --2- 172.21.15.86:0/772307780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588276befb0 0x5588276d4460 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f556c0bb040 tx=0x7f556c003840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1ed77f26b05c1790 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:22.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.860+0000 7f555d7fa640 1 -- 172.21.15.86:0/772307780 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556c0bebe0 con 0x5588276befb0 2024-04-21T08:45:22.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.860+0000 7f555d7fa640 1 -- 172.21.15.86:0/772307780 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f556c0c2f90 con 0x5588276befb0 2024-04-21T08:45:22.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.860+0000 7f555d7fa640 1 -- 172.21.15.86:0/772307780 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556c0bcf70 con 0x5588276befb0 2024-04-21T08:45:22.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.861+0000 7f5580bb5740 1 -- 172.21.15.86:0/772307780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588276befb0 msgr2=0x5588276d4460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:22.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.861+0000 7f5580bb5740 1 --2- 172.21.15.86:0/772307780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588276befb0 0x5588276d4460 secure :-1 s=READY pgs=1419 cs=0 l=1 rev1=1 crypto rx=0x7f556c0bb040 tx=0x7f556c003840 comp rx=0 tx=0).stop 2024-04-21T08:45:22.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.861+0000 7f5580bb5740 1 -- 172.21.15.86:0/772307780 shutdown_connections 2024-04-21T08:45:22.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.861+0000 7f5580bb5740 1 --2- 172.21.15.86:0/772307780 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588276c50f0 0x5588276cf6a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.861+0000 7f5580bb5740 1 --2- 172.21.15.86:0/772307780 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588276c4740 0x5588276de0f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.861+0000 7f5580bb5740 1 --2- 172.21.15.86:0/772307780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588276befb0 0x5588276d4460 unknown :-1 s=CLOSED pgs=1419 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.861+0000 7f5580bb5740 1 -- 172.21.15.86:0/772307780 >> 172.21.15.86:0/772307780 conn(0x5588276cd1d0 msgr2=0x5588276d2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:22.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.861+0000 7f5580bb5740 1 -- 172.21.15.86:0/772307780 shutdown_connections 2024-04-21T08:45:22.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.861+0000 7f5580bb5740 1 -- 172.21.15.86:0/772307780 wait complete. 2024-04-21T08:45:22.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.861+0000 7f5580bb5740 1 -- start start 2024-04-21T08:45:22.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.861+0000 7f5580bb5740 1 Processor -- start 2024-04-21T08:45:22.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.861+0000 7f5580bb5740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588276befb0 0x5588276da8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:22.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.861+0000 7f5580bb5740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588276c4740 0x5588276d75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:22.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.861+0000 7f5580bb5740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588276c50f0 0x5588276d7b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:22.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.861+0000 7f5580bb5740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f55781ae560 con 0x5588276c4740 2024-04-21T08:45:22.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.861+0000 7f5580bb5740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f55781ade90 con 0x5588276c50f0 2024-04-21T08:45:22.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.861+0000 7f5580bb5740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f55781ab0f0 con 0x5588276befb0 2024-04-21T08:45:22.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.861+0000 7f5580bb5740 10 client.0 _ll_register_callbacks cb 0x7f556800a510 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:22.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.861+0000 7f5576ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588276c4740 0x5588276d75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:22.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.861+0000 7f5576ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588276c4740 0x5588276d75d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48002/0 (socket says 172.21.15.86:48002) 2024-04-21T08:45:22.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.861+0000 7f5576ffd640 1 -- 172.21.15.86:0/252891224 learned_addr learned my addr 172.21.15.86:0/252891224 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:22.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.862+0000 7f55777fe640 1 --2- 172.21.15.86:0/252891224 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588276befb0 0x5588276da8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:22.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.862+0000 7f5576ffd640 1 -- 172.21.15.86:0/252891224 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588276befb0 msgr2=0x5588276da8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:22.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.862+0000 7f5576ffd640 1 --2- 172.21.15.86:0/252891224 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588276befb0 0x5588276da8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.862+0000 7f5576ffd640 1 -- 172.21.15.86:0/252891224 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588276c50f0 msgr2=0x5588276d7b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:22.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.862+0000 7f5577fff640 1 --2- 172.21.15.86:0/252891224 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588276c50f0 0x5588276d7b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:22.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.862+0000 7f5576ffd640 1 --2- 172.21.15.86:0/252891224 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588276c50f0 0x5588276d7b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.862+0000 7f5576ffd640 1 -- 172.21.15.86:0/252891224 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5588276d81d0 con 0x5588276c4740 2024-04-21T08:45:22.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.862+0000 7f5576ffd640 1 --2- 172.21.15.86:0/252891224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588276c4740 0x5588276d75d0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f5568004810 tx=0x7f556800fee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=efd80f20e4360ab2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:22.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.862+0000 7f5556ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:22.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.862+0000 7f5556ffd640 1 -- 172.21.15.86:0/252891224 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f556802e020 con 0x5588276c4740 2024-04-21T08:45:22.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.862+0000 7f5556ffd640 1 -- 172.21.15.86:0/252891224 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f5568011a40 con 0x5588276c4740 2024-04-21T08:45:22.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.862+0000 7f5580bb5740 1 -- 172.21.15.86:0/252891224 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f556c0c2f90 con 0x5588276c4740 2024-04-21T08:45:22.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.862+0000 7f5556ffd640 1 -- 172.21.15.86:0/252891224 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5568039050 con 0x5588276c4740 2024-04-21T08:45:22.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.862+0000 7f5580bb5740 10 client.18438 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:22.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.862+0000 7f5580bb5740 1 -- 172.21.15.86:0/252891224 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558827749d10 con 0x5588276c4740 2024-04-21T08:45:22.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.862+0000 7f5556ffd640 1 -- 172.21.15.86:0/252891224 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f556c0c2f90 con 0x5588276c4740 2024-04-21T08:45:22.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.862+0000 7f5556ffd640 1 -- 172.21.15.86:0/252891224 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5568006160 con 0x5588276c4740 2024-04-21T08:45:22.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.863+0000 7f5580bb5740 10 client.18438 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:22.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.863+0000 7f5580bb5740 10 client.18438 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:22.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.863+0000 7f5580bb5740 20 client.18438 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:22.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.863+0000 7f5580bb5740 1 --2- 172.21.15.86:0/252891224 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5588276ce250 0x55882774c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:22.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.863+0000 7f5580bb5740 4 client.18438 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:45:22.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.863+0000 7f5580bb5740 1 -- 172.21.15.86:0/252891224 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558827630790 con 0x5588276ce250 2024-04-21T08:45:22.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.863+0000 7f55777fe640 1 --2- 172.21.15.86:0/252891224 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5588276ce250 0x55882774c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:22.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.863+0000 7f5556ffd640 10 client.18438 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:22.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.863+0000 7f55777fe640 1 --2- 172.21.15.86:0/252891224 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5588276ce250 0x55882774c520 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae50c478b7e53453 server_cookie=e3d4242d69d937d6 in_seq=0 out_seq=0 2024-04-21T08:45:22.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.864+0000 7f5556ffd640 1 -- 172.21.15.86:0/252891224 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558827630790 con 0x5588276ce250 2024-04-21T08:45:22.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.864+0000 7f5556ffd640 10 client.18438 handle_command_reply: tid=0 2024-04-21T08:45:22.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.864+0000 7f5580bb5740 1 client.18438 shutdown 2024-04-21T08:45:22.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.865+0000 7f5580bb5740 1 -- 172.21.15.86:0/252891224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588276c4740 msgr2=0x5588276d75d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:22.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.865+0000 7f5580bb5740 1 --2- 172.21.15.86:0/252891224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588276c4740 0x5588276d75d0 secure :-1 s=READY pgs=1420 cs=0 l=1 rev1=1 crypto rx=0x7f5568004810 tx=0x7f556800fee0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.865+0000 7f5580bb5740 1 -- 172.21.15.86:0/252891224 shutdown_connections 2024-04-21T08:45:22.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.865+0000 7f5580bb5740 1 --2- 172.21.15.86:0/252891224 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5588276ce250 0x55882774c520 crc :-1 s=READY pgs=210 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.865+0000 7f5580bb5740 1 --2- 172.21.15.86:0/252891224 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5588276c50f0 0x5588276d7b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.865+0000 7f5580bb5740 1 --2- 172.21.15.86:0/252891224 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5588276c4740 0x5588276d75d0 unknown :-1 s=CLOSED pgs=1420 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.865+0000 7f5580bb5740 1 --2- 172.21.15.86:0/252891224 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5588276befb0 0x5588276da8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.865+0000 7f5556ffd640 0 client.18438 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:22.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.865+0000 7f5580bb5740 1 -- 172.21.15.86:0/252891224 >> 172.21.15.86:0/252891224 conn(0x5588276cd1d0 msgr2=0x5588276d1cd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:22.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.865+0000 7f5580bb5740 1 -- 172.21.15.86:0/252891224 shutdown_connections 2024-04-21T08:45:22.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.865+0000 7f5580bb5740 1 -- 172.21.15.86:0/252891224 wait complete. 2024-04-21T08:45:22.871 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.865+0000 7f5580bb5740 20 client.18438 trim_cache size 0 max 16384 2024-04-21T08:45:22.871 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:45:22.871 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-21T08:45:22.871 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "c5e73be7-e8ad-41ea-a0ee-2ee7e8f6c988", 2024-04-21T08:45:22.871 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-21T08:45:22.871 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:45:22.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.865+0000 7f557dfbd640 1 -- 172.21.15.86:0/1529802690 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f55600a3d20 msgr2=0x7f55600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:22.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.865+0000 7f557dfbd640 1 --2- 172.21.15.86:0/1529802690 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f55600a3d20 0x7f55600a61e0 secure :-1 s=READY pgs=226 cs=0 l=1 rev1=1 crypto rx=0x7f5570013c50 tx=0x7f5570007e30 comp rx=0 tx=0).stop 2024-04-21T08:45:22.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.865+0000 7f557dfbd640 1 -- 172.21.15.86:0/1529802690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f557806a860 msgr2=0x7f5578198760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:22.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.865+0000 7f557dfbd640 1 --2- 172.21.15.86:0/1529802690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f557806a860 0x7f5578198760 secure :-1 s=READY pgs=1416 cs=0 l=1 rev1=1 crypto rx=0x7f556c005ab0 tx=0x7f556c02ca90 comp rx=0 tx=0).stop 2024-04-21T08:45:22.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.865+0000 7f557dfbd640 1 -- 172.21.15.86:0/1529802690 shutdown_connections 2024-04-21T08:45:22.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.865+0000 7f557dfbd640 1 --2- 172.21.15.86:0/1529802690 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f55600a3d20 0x7f55600a61e0 unknown :-1 s=CLOSED pgs=226 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.865+0000 7f557dfbd640 1 --2- 172.21.15.86:0/1529802690 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f557819d020 0x7f557819d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.865+0000 7f557dfbd640 1 --2- 172.21.15.86:0/1529802690 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5578107270 0x7f5578198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.865+0000 7f557dfbd640 1 --2- 172.21.15.86:0/1529802690 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f557806a860 0x7f5578198760 unknown :-1 s=CLOSED pgs=1416 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.865+0000 7f557dfbd640 1 -- 172.21.15.86:0/1529802690 >> 172.21.15.86:0/1529802690 conn(0x7f5578077eb0 msgr2=0x7f5578100530 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:22.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.865+0000 7f557dfbd640 1 -- 172.21.15.86:0/1529802690 shutdown_connections 2024-04-21T08:45:22.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.865+0000 7f557dfbd640 1 -- 172.21.15.86:0/1529802690 wait complete. 2024-04-21T08:45:22.876 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:45:22.876 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:45:22.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.977+0000 7f78c1069640 1 -- 172.21.15.86:0/3443734110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78bc10c590 msgr2=0x7f78bc10e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:22.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.977+0000 7f78c1069640 1 --2- 172.21.15.86:0/3443734110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78bc10c590 0x7f78bc10e9c0 secure :-1 s=READY pgs=1421 cs=0 l=1 rev1=1 crypto rx=0x7f78b4009ca0 tx=0x7f78b402c180 comp rx=0 tx=0).stop 2024-04-21T08:45:22.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.977+0000 7f78c1069640 1 -- 172.21.15.86:0/3443734110 shutdown_connections 2024-04-21T08:45:22.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.977+0000 7f78c1069640 1 --2- 172.21.15.86:0/3443734110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78bc10c590 0x7f78bc10e9c0 unknown :-1 s=CLOSED pgs=1421 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.977+0000 7f78c1069640 1 --2- 172.21.15.86:0/3443734110 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78bc103ae0 0x7f78bc10bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.977+0000 7f78c1069640 1 --2- 172.21.15.86:0/3443734110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78bc1031c0 0x7f78bc1035a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.977+0000 7f78c1069640 1 -- 172.21.15.86:0/3443734110 >> 172.21.15.86:0/3443734110 conn(0x7f78bc077f70 msgr2=0x7f78bc078380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:22.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.977+0000 7f78c1069640 1 -- 172.21.15.86:0/3443734110 shutdown_connections 2024-04-21T08:45:22.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.977+0000 7f78c1069640 1 -- 172.21.15.86:0/3443734110 wait complete. 2024-04-21T08:45:22.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.978+0000 7f78c1069640 1 Processor -- start 2024-04-21T08:45:22.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.978+0000 7f78c1069640 1 -- start start 2024-04-21T08:45:22.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.978+0000 7f78c1069640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78bc1031c0 0x7f78bc1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:22.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.978+0000 7f78c1069640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78bc103ae0 0x7f78bc198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:22.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.978+0000 7f78c1069640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78bc10c590 0x7f78bc19cfb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:22.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.978+0000 7f78c1069640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc112190 con 0x7f78bc103ae0 2024-04-21T08:45:22.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.978+0000 7f78c1069640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc112010 con 0x7f78bc1031c0 2024-04-21T08:45:22.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.978+0000 7f78c1069640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc112310 con 0x7f78bc10c590 2024-04-21T08:45:22.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.978+0000 7f78b3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78bc103ae0 0x7f78bc198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:22.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.978+0000 7f78b3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78bc103ae0 0x7f78bc198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48016/0 (socket says 172.21.15.86:48016) 2024-04-21T08:45:22.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.978+0000 7f78b3fff640 1 -- 172.21.15.86:0/776329271 learned_addr learned my addr 172.21.15.86:0/776329271 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:22.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.978+0000 7f78bb59e640 1 --2- 172.21.15.86:0/776329271 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78bc10c590 0x7f78bc19cfb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:22.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.979+0000 7f78bad9d640 1 --2- 172.21.15.86:0/776329271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78bc1031c0 0x7f78bc1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:22.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.979+0000 7f78b3fff640 1 -- 172.21.15.86:0/776329271 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78bc10c590 msgr2=0x7f78bc19cfb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:22.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.979+0000 7f78b3fff640 1 --2- 172.21.15.86:0/776329271 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78bc10c590 0x7f78bc19cfb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.979+0000 7f78b3fff640 1 -- 172.21.15.86:0/776329271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78bc1031c0 msgr2=0x7f78bc1986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:22.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.979+0000 7f78b3fff640 1 --2- 172.21.15.86:0/776329271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78bc1031c0 0x7f78bc1986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.979+0000 7f78b3fff640 1 -- 172.21.15.86:0/776329271 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f78bc19d6e0 con 0x7f78bc103ae0 2024-04-21T08:45:22.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.979+0000 7f78b3fff640 1 --2- 172.21.15.86:0/776329271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78bc103ae0 0x7f78bc198c30 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f78a400c890 tx=0x7f78a400cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:22.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.979+0000 7f78b8d99640 1 -- 172.21.15.86:0/776329271 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a4004220 con 0x7f78bc103ae0 2024-04-21T08:45:22.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.979+0000 7f78b8d99640 1 -- 172.21.15.86:0/776329271 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78a40043c0 con 0x7f78bc103ae0 2024-04-21T08:45:22.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.979+0000 7f78c1069640 1 -- 172.21.15.86:0/776329271 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f78bc19d9d0 con 0x7f78bc103ae0 2024-04-21T08:45:22.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.979+0000 7f78b8d99640 1 -- 172.21.15.86:0/776329271 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a4004a00 con 0x7f78bc103ae0 2024-04-21T08:45:22.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.979+0000 7f78c1069640 1 -- 172.21.15.86:0/776329271 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f78bc1a5260 con 0x7f78bc103ae0 2024-04-21T08:45:22.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.981+0000 7f78b8d99640 1 -- 172.21.15.86:0/776329271 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f78a40079b0 con 0x7f78bc103ae0 2024-04-21T08:45:22.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.981+0000 7f78c3c57740 1 Processor -- start 2024-04-21T08:45:22.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.981+0000 7f78c3c57740 1 -- start start 2024-04-21T08:45:22.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.982+0000 7f78c3c57740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559354b2d580 0x559354b2d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:22.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.982+0000 7f78c3c57740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559354b2dea0 0x559354b36540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:22.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.982+0000 7f78c3c57740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559354b36a80 0x559354b38e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:22.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.982+0000 7f78c3c57740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559354a921e0 con 0x559354b2dea0 2024-04-21T08:45:22.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.982+0000 7f78c3c57740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559354a2f390 con 0x559354b2d580 2024-04-21T08:45:22.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.982+0000 7f78c3c57740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559354a8e1a0 con 0x559354b36a80 2024-04-21T08:45:22.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.982+0000 7f78bad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559354b2dea0 0x559354b36540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:22.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.982+0000 7f78bad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559354b2dea0 0x559354b36540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48030/0 (socket says 172.21.15.86:48030) 2024-04-21T08:45:22.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.982+0000 7f78bad9d640 1 -- 172.21.15.86:0/3167434005 learned_addr learned my addr 172.21.15.86:0/3167434005 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:22.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.982+0000 7f78b8d99640 1 --2- 172.21.15.86:0/776329271 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f78a80a3d20 0x7f78a80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:22.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.982+0000 7f78b3fff640 1 --2- 172.21.15.86:0/3167434005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559354b36a80 0x559354b38e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:22.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.982+0000 7f78bb59e640 1 --2- 172.21.15.86:0/3167434005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559354b2d580 0x559354b2d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:22.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.982+0000 7f78bb59e640 1 --2- 172.21.15.86:0/776329271 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f78a80a3d20 0x7f78a80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:22.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.982+0000 7f78bad9d640 1 -- 172.21.15.86:0/3167434005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559354b36a80 msgr2=0x559354b38e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:22.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.982+0000 7f78bad9d640 1 --2- 172.21.15.86:0/3167434005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559354b36a80 0x559354b38e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.982+0000 7f78bad9d640 1 -- 172.21.15.86:0/3167434005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559354b2d580 msgr2=0x559354b2d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:22.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.982+0000 7f78bad9d640 1 --2- 172.21.15.86:0/3167434005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559354b2d580 0x559354b2d960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.982+0000 7f78bad9d640 1 -- 172.21.15.86:0/3167434005 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55935486e7d0 con 0x559354b2dea0 2024-04-21T08:45:22.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.982+0000 7f78b8d99640 1 -- 172.21.15.86:0/776329271 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f78a40d0800 con 0x7f78bc103ae0 2024-04-21T08:45:22.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.982+0000 7f78bad9d640 1 --2- 172.21.15.86:0/3167434005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559354b2dea0 0x559354b36540 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f78ac00bd10 tx=0x7f78ac0107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9d1bb2ac96a8b59 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:22.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.983+0000 7f78bb59e640 1 --2- 172.21.15.86:0/776329271 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f78a80a3d20 0x7f78a80a61e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f78b4005cc0 tx=0x7f78b4005c50 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:22.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.983+0000 7f78b17fa640 1 -- 172.21.15.86:0/3167434005 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ac004030 con 0x559354b2dea0 2024-04-21T08:45:22.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.983+0000 7f78b17fa640 1 -- 172.21.15.86:0/3167434005 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78ac0035f0 con 0x559354b2dea0 2024-04-21T08:45:22.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.983+0000 7f78b17fa640 1 -- 172.21.15.86:0/3167434005 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78ac0117b0 con 0x559354b2dea0 2024-04-21T08:45:22.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.983+0000 7f78c3c57740 1 -- 172.21.15.86:0/3167434005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559354b2dea0 msgr2=0x559354b36540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:22.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.983+0000 7f78c3c57740 1 --2- 172.21.15.86:0/3167434005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559354b2dea0 0x559354b36540 secure :-1 s=READY pgs=1423 cs=0 l=1 rev1=1 crypto rx=0x7f78ac00bd10 tx=0x7f78ac0107a0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.983+0000 7f78c3c57740 1 -- 172.21.15.86:0/3167434005 shutdown_connections 2024-04-21T08:45:22.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.983+0000 7f78c3c57740 1 --2- 172.21.15.86:0/3167434005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559354b36a80 0x559354b38e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.983+0000 7f78c3c57740 1 --2- 172.21.15.86:0/3167434005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559354b2dea0 0x559354b36540 unknown :-1 s=CLOSED pgs=1423 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.983+0000 7f78c3c57740 1 --2- 172.21.15.86:0/3167434005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559354b2d580 0x559354b2d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.983+0000 7f78c3c57740 1 -- 172.21.15.86:0/3167434005 >> 172.21.15.86:0/3167434005 conn(0x559354b27cb0 msgr2=0x559354b28090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:22.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.983+0000 7f78c3c57740 1 -- 172.21.15.86:0/3167434005 shutdown_connections 2024-04-21T08:45:22.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.983+0000 7f78c3c57740 1 -- 172.21.15.86:0/3167434005 wait complete. 2024-04-21T08:45:22.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.984+0000 7f78c3c57740 1 -- start start 2024-04-21T08:45:22.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.984+0000 7f78c3c57740 1 Processor -- start 2024-04-21T08:45:22.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.984+0000 7f78c3c57740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559354b2d580 0x559354b33a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:22.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.984+0000 7f78c3c57740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559354b2dea0 0x559354b33f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:22.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.984+0000 7f78c3c57740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559354b36a80 0x559354b34490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:22.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.984+0000 7f78c3c57740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559354b39870 con 0x559354b2dea0 2024-04-21T08:45:22.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.984+0000 7f78c3c57740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559354a4b2f0 con 0x559354b2d580 2024-04-21T08:45:22.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.984+0000 7f78c3c57740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559354b399f0 con 0x559354b36a80 2024-04-21T08:45:22.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.984+0000 7f78c3c57740 10 client.0 _ll_register_callbacks cb 0x559354a58580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:22.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.984+0000 7f78b3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559354b2dea0 0x559354b33f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:22.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.984+0000 7f78b3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559354b2dea0 0x559354b33f50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48032/0 (socket says 172.21.15.86:48032) 2024-04-21T08:45:22.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.984+0000 7f78b3fff640 1 -- 172.21.15.86:0/742809352 learned_addr learned my addr 172.21.15.86:0/742809352 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:22.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.984+0000 7f78bad9d640 1 --2- 172.21.15.86:0/742809352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559354b2d580 0x559354b33a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:22.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.984+0000 7f78bb59e640 1 --2- 172.21.15.86:0/742809352 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559354b36a80 0x559354b34490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:22.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.984+0000 7f78b3fff640 1 -- 172.21.15.86:0/742809352 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559354b36a80 msgr2=0x559354b34490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:22.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.984+0000 7f78b3fff640 1 --2- 172.21.15.86:0/742809352 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559354b36a80 0x559354b34490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.984+0000 7f78b3fff640 1 -- 172.21.15.86:0/742809352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559354b2d580 msgr2=0x559354b33a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:22.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.984+0000 7f78b3fff640 1 --2- 172.21.15.86:0/742809352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559354b2d580 0x559354b33a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.984+0000 7f78b3fff640 1 -- 172.21.15.86:0/742809352 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5593547a43f0 con 0x559354b2dea0 2024-04-21T08:45:22.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.985+0000 7f78b3fff640 1 --2- 172.21.15.86:0/742809352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559354b2dea0 0x559354b33f50 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f78a40cdbe0 tx=0x7f78a409c310 comp rx=0 tx=0).ready entity=mon.0 client_cookie=284bbda3e7f06a0a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:22.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.985+0000 7f78a27fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:22.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.985+0000 7f78a27fc640 1 -- 172.21.15.86:0/742809352 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a401d040 con 0x559354b2dea0 2024-04-21T08:45:22.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.985+0000 7f78a27fc640 1 -- 172.21.15.86:0/742809352 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f78a4098af0 con 0x559354b2dea0 2024-04-21T08:45:22.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.985+0000 7f78c3c57740 1 -- 172.21.15.86:0/742809352 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78ac0035f0 con 0x559354b2dea0 2024-04-21T08:45:22.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.985+0000 7f78a27fc640 1 -- 172.21.15.86:0/742809352 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f78a40ba050 con 0x559354b2dea0 2024-04-21T08:45:22.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.985+0000 7f78c3c57740 10 client.18450 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:22.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.985+0000 7f78c3c57740 1 -- 172.21.15.86:0/742809352 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5593547640f0 con 0x559354b2dea0 2024-04-21T08:45:22.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.985+0000 7f78a27fc640 1 -- 172.21.15.86:0/742809352 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78ac0035f0 con 0x559354b2dea0 2024-04-21T08:45:22.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.985+0000 7f78a27fc640 1 -- 172.21.15.86:0/742809352 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f78a40d03d0 con 0x559354b2dea0 2024-04-21T08:45:22.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.985+0000 7f78c3c57740 10 client.18450 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:22.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.985+0000 7f78c3c57740 10 client.18450 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:22.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.985+0000 7f78c3c57740 20 client.18450 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:22.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.985+0000 7f78c3c57740 1 --2- 172.21.15.86:0/742809352 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559354b469d0 0x559354b48e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:22.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.985+0000 7f78c3c57740 4 client.18450 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:22.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.985+0000 7f78c3c57740 1 -- 172.21.15.86:0/742809352 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559354a9fc00 con 0x559354b469d0 2024-04-21T08:45:22.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.985+0000 7f78bad9d640 1 --2- 172.21.15.86:0/742809352 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559354b469d0 0x559354b48e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:22.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.986+0000 7f78a27fc640 10 client.18450 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:22.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.986+0000 7f78bad9d640 1 --2- 172.21.15.86:0/742809352 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559354b469d0 0x559354b48e50 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5812a89281a912c9 server_cookie=636424e579d5c909 in_seq=0 out_seq=0 2024-04-21T08:45:22.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.987+0000 7f78a27fc640 1 -- 172.21.15.86:0/742809352 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559354a9fc00 con 0x559354b469d0 2024-04-21T08:45:22.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.987+0000 7f78a27fc640 10 client.18450 handle_command_reply: tid=0 2024-04-21T08:45:22.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.987+0000 7f78c3c57740 1 client.18450 shutdown 2024-04-21T08:45:22.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.987+0000 7f78c3c57740 1 -- 172.21.15.86:0/742809352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559354b2dea0 msgr2=0x559354b33f50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:22.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.987+0000 7f78c3c57740 1 --2- 172.21.15.86:0/742809352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559354b2dea0 0x559354b33f50 secure :-1 s=READY pgs=1424 cs=0 l=1 rev1=1 crypto rx=0x7f78a40cdbe0 tx=0x7f78a409c310 comp rx=0 tx=0).stop 2024-04-21T08:45:22.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.987+0000 7f78c3c57740 1 -- 172.21.15.86:0/742809352 shutdown_connections 2024-04-21T08:45:22.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.987+0000 7f78c3c57740 1 --2- 172.21.15.86:0/742809352 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559354b469d0 0x559354b48e50 crc :-1 s=READY pgs=211 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.987+0000 7f78c3c57740 1 --2- 172.21.15.86:0/742809352 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559354b36a80 0x559354b34490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.987+0000 7f78a27fc640 0 client.18450 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:22.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.987+0000 7f78c3c57740 1 --2- 172.21.15.86:0/742809352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559354b2dea0 0x559354b33f50 unknown :-1 s=CLOSED pgs=1424 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.987+0000 7f78c3c57740 1 --2- 172.21.15.86:0/742809352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559354b2d580 0x559354b33a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:22.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.987+0000 7f78c3c57740 1 -- 172.21.15.86:0/742809352 >> 172.21.15.86:0/742809352 conn(0x559354b27cb0 msgr2=0x559354b29a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:22.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.987+0000 7f78c3c57740 1 -- 172.21.15.86:0/742809352 shutdown_connections 2024-04-21T08:45:22.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.987+0000 7f78c3c57740 1 -- 172.21.15.86:0/742809352 wait complete. 2024-04-21T08:45:22.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:22.987+0000 7f78c3c57740 20 client.18450 trim_cache size 0 max 16384 2024-04-21T08:45:23.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.004+0000 7f78c3c57740 1 Processor -- start 2024-04-21T08:45:23.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.004+0000 7f78c3c57740 1 -- start start 2024-04-21T08:45:23.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.005+0000 7f78c3c57740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559354b2d580 0x559354b36140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:23.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.005+0000 7f78c3c57740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559354b2dea0 0x559354b42730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:23.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.005+0000 7f78c3c57740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559354b36a80 0x559354b4e720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:23.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.005+0000 7f78c3c57740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc1af4d0 con 0x559354b2dea0 2024-04-21T08:45:23.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.005+0000 7f78c3c57740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc1b1010 con 0x559354b2d580 2024-04-21T08:45:23.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.005+0000 7f78c3c57740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc1afba0 con 0x559354b36a80 2024-04-21T08:45:23.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.005+0000 7f78b3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559354b2dea0 0x559354b42730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:23.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.005+0000 7f78b3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559354b2dea0 0x559354b42730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48048/0 (socket says 172.21.15.86:48048) 2024-04-21T08:45:23.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.005+0000 7f78b3fff640 1 -- 172.21.15.86:0/362858901 learned_addr learned my addr 172.21.15.86:0/362858901 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:23.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.005+0000 7f78bad9d640 1 --2- 172.21.15.86:0/362858901 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559354b2d580 0x559354b36140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:23.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.005+0000 7f78bb59e640 1 --2- 172.21.15.86:0/362858901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559354b36a80 0x559354b4e720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:23.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.005+0000 7f78b3fff640 1 -- 172.21.15.86:0/362858901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559354b36a80 msgr2=0x559354b4e720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:23.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.005+0000 7f78b3fff640 1 --2- 172.21.15.86:0/362858901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559354b36a80 0x559354b4e720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.005+0000 7f78b3fff640 1 -- 172.21.15.86:0/362858901 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559354b2d580 msgr2=0x559354b36140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:23.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.005+0000 7f78b3fff640 1 --2- 172.21.15.86:0/362858901 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559354b2d580 0x559354b36140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.005+0000 7f78b3fff640 1 -- 172.21.15.86:0/362858901 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559354b437f0 con 0x559354b2dea0 2024-04-21T08:45:23.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.005+0000 7f78b3fff640 1 --2- 172.21.15.86:0/362858901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559354b2dea0 0x559354b42730 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f78a409c180 tx=0x7f78a409a990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=241cab551bf6ade9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:23.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.005+0000 7f78b0ff9640 1 -- 172.21.15.86:0/362858901 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a401d070 con 0x559354b2dea0 2024-04-21T08:45:23.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.005+0000 7f78b0ff9640 1 -- 172.21.15.86:0/362858901 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78a40d0b00 con 0x559354b2dea0 2024-04-21T08:45:23.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.005+0000 7f78b0ff9640 1 -- 172.21.15.86:0/362858901 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78a40a0780 con 0x559354b2dea0 2024-04-21T08:45:23.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.006+0000 7f78c3c57740 1 -- 172.21.15.86:0/362858901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559354b2dea0 msgr2=0x559354b42730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:23.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.006+0000 7f78c3c57740 1 --2- 172.21.15.86:0/362858901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559354b2dea0 0x559354b42730 secure :-1 s=READY pgs=1425 cs=0 l=1 rev1=1 crypto rx=0x7f78a409c180 tx=0x7f78a409a990 comp rx=0 tx=0).stop 2024-04-21T08:45:23.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.006+0000 7f78c3c57740 1 -- 172.21.15.86:0/362858901 shutdown_connections 2024-04-21T08:45:23.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.006+0000 7f78c3c57740 1 --2- 172.21.15.86:0/362858901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559354b36a80 0x559354b4e720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.006+0000 7f78c3c57740 1 --2- 172.21.15.86:0/362858901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559354b2dea0 0x559354b42730 unknown :-1 s=CLOSED pgs=1425 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.006+0000 7f78c3c57740 1 --2- 172.21.15.86:0/362858901 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559354b2d580 0x559354b36140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.006+0000 7f78c3c57740 1 -- 172.21.15.86:0/362858901 >> 172.21.15.86:0/362858901 conn(0x559354b27cb0 msgr2=0x559354b2e280 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:23.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.006+0000 7f78c3c57740 1 -- 172.21.15.86:0/362858901 shutdown_connections 2024-04-21T08:45:23.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.006+0000 7f78c3c57740 1 -- 172.21.15.86:0/362858901 wait complete. 2024-04-21T08:45:23.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.006+0000 7f78c3c57740 1 -- start start 2024-04-21T08:45:23.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.006+0000 7f78c3c57740 1 Processor -- start 2024-04-21T08:45:23.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.007+0000 7f78c3c57740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559354b2d580 0x559354b3ecc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:23.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.007+0000 7f78c3c57740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559354b2dea0 0x559354b3a220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:23.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.007+0000 7f78c3c57740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559354b36a80 0x559354b3a760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:23.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.007+0000 7f78c3c57740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc1b0940 con 0x559354b36a80 2024-04-21T08:45:23.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.007+0000 7f78c3c57740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc1b0270 con 0x559354b2dea0 2024-04-21T08:45:23.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.007+0000 7f78c3c57740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f78bc1ad4d0 con 0x559354b2d580 2024-04-21T08:45:23.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.007+0000 7f78c3c57740 10 client.0 _ll_register_callbacks cb 0x7f78a40980e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:23.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.007+0000 7f78bb59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559354b36a80 0x559354b3a760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:23.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.007+0000 7f78bb59e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559354b36a80 0x559354b3a760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48062/0 (socket says 172.21.15.86:48062) 2024-04-21T08:45:23.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.007+0000 7f78bb59e640 1 -- 172.21.15.86:0/526965525 learned_addr learned my addr 172.21.15.86:0/526965525 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:23.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.007+0000 7f78b3fff640 1 --2- 172.21.15.86:0/526965525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559354b2dea0 0x559354b3a220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:23.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.007+0000 7f78bad9d640 1 --2- 172.21.15.86:0/526965525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559354b2d580 0x559354b3ecc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:23.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.007+0000 7f78bb59e640 1 -- 172.21.15.86:0/526965525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559354b2d580 msgr2=0x559354b3ecc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:23.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.007+0000 7f78bb59e640 1 --2- 172.21.15.86:0/526965525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559354b2d580 0x559354b3ecc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.007+0000 7f78bb59e640 1 -- 172.21.15.86:0/526965525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559354b2dea0 msgr2=0x559354b3a220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:23.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.007+0000 7f78bb59e640 1 --2- 172.21.15.86:0/526965525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559354b2dea0 0x559354b3a220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.007+0000 7f78bb59e640 1 -- 172.21.15.86:0/526965525 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559354b3aca0 con 0x559354b36a80 2024-04-21T08:45:23.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.007+0000 7f78b3fff640 1 --2- 172.21.15.86:0/526965525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559354b2dea0 0x559354b3a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:23.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.007+0000 7f78bb59e640 1 --2- 172.21.15.86:0/526965525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559354b36a80 0x559354b3a760 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f78b4009ca0 tx=0x7f78b402c180 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8648e4791dfe4b43 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:23.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.007+0000 7f78a27fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:23.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.007+0000 7f78a27fc640 1 -- 172.21.15.86:0/526965525 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f78b4039040 con 0x559354b36a80 2024-04-21T08:45:23.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.007+0000 7f78a27fc640 1 -- 172.21.15.86:0/526965525 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f78b4034da0 con 0x559354b36a80 2024-04-21T08:45:23.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.007+0000 7f78c3c57740 1 -- 172.21.15.86:0/526965525 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f78a40d0b00 con 0x559354b36a80 2024-04-21T08:45:23.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.007+0000 7f78a27fc640 1 -- 172.21.15.86:0/526965525 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f78b4032e50 con 0x559354b36a80 2024-04-21T08:45:23.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.007+0000 7f78c3c57740 10 client.18456 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:23.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.007+0000 7f78c3c57740 1 -- 172.21.15.86:0/526965525 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559354bacc50 con 0x559354b36a80 2024-04-21T08:45:23.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.008+0000 7f78a27fc640 1 -- 172.21.15.86:0/526965525 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f78a40d0b00 con 0x559354b36a80 2024-04-21T08:45:23.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.008+0000 7f78a27fc640 1 -- 172.21.15.86:0/526965525 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f78b403f040 con 0x559354b36a80 2024-04-21T08:45:23.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.008+0000 7f78c3c57740 10 client.18456 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:23.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.008+0000 7f78c3c57740 10 client.18456 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:23.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.008+0000 7f78c3c57740 20 client.18456 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:23.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.008+0000 7f78c3c57740 1 --2- 172.21.15.86:0/526965525 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559354bad400 0x559354baf820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:23.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.008+0000 7f78c3c57740 4 client.18456 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:45:23.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.008+0000 7f78c3c57740 1 -- 172.21.15.86:0/526965525 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x559354a99a70 con 0x559354bad400 2024-04-21T08:45:23.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.008+0000 7f78bad9d640 1 --2- 172.21.15.86:0/526965525 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559354bad400 0x559354baf820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:23.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.008+0000 7f78a27fc640 10 client.18456 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:23.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.008+0000 7f78bad9d640 1 --2- 172.21.15.86:0/526965525 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559354bad400 0x559354baf820 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a1333a3dcf9f00d1 server_cookie=f5a0e3b2d50a38d5 in_seq=0 out_seq=0 2024-04-21T08:45:23.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.009+0000 7f78a27fc640 1 -- 172.21.15.86:0/526965525 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x559354a99a70 con 0x559354bad400 2024-04-21T08:45:23.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.009+0000 7f78a27fc640 10 client.18456 handle_command_reply: tid=0 2024-04-21T08:45:23.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.009+0000 7f78c3c57740 1 client.18456 shutdown 2024-04-21T08:45:23.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.009+0000 7f78c3c57740 1 -- 172.21.15.86:0/526965525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559354b36a80 msgr2=0x559354b3a760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:23.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.009+0000 7f78c3c57740 1 --2- 172.21.15.86:0/526965525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559354b36a80 0x559354b3a760 secure :-1 s=READY pgs=1426 cs=0 l=1 rev1=1 crypto rx=0x7f78b4009ca0 tx=0x7f78b402c180 comp rx=0 tx=0).stop 2024-04-21T08:45:23.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.009+0000 7f78c3c57740 1 -- 172.21.15.86:0/526965525 shutdown_connections 2024-04-21T08:45:23.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.009+0000 7f78c3c57740 1 --2- 172.21.15.86:0/526965525 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559354bad400 0x559354baf820 crc :-1 s=READY pgs=212 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.009+0000 7f78c3c57740 1 --2- 172.21.15.86:0/526965525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559354b36a80 0x559354b3a760 unknown :-1 s=CLOSED pgs=1426 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.009+0000 7f78a27fc640 0 client.18456 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:23.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.009+0000 7f78c3c57740 1 --2- 172.21.15.86:0/526965525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559354b2dea0 0x559354b3a220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.009+0000 7f78c3c57740 1 --2- 172.21.15.86:0/526965525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559354b2d580 0x559354b3ecc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.009+0000 7f78c3c57740 1 -- 172.21.15.86:0/526965525 >> 172.21.15.86:0/526965525 conn(0x559354b27cb0 msgr2=0x559354b32220 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:23.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.009+0000 7f78c3c57740 1 -- 172.21.15.86:0/526965525 shutdown_connections 2024-04-21T08:45:23.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.009+0000 7f78c3c57740 1 -- 172.21.15.86:0/526965525 wait complete. 2024-04-21T08:45:23.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.009+0000 7f78c3c57740 20 client.18456 trim_cache size 0 max 16384 2024-04-21T08:45:23.016 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:45:23.016 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:45:23.016 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-21T08:45:23.017 INFO:teuthology.orchestra.run.smithi086.stdout: "c5e73be7-e8ad-41ea-a0ee-2ee7e8f6c988": { 2024-04-21T08:45:23.017 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-21T08:45:23.017 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "c5e73be7-e8ad-41ea-a0ee-2ee7e8f6c988", 2024-04-21T08:45:23.017 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-21T08:45:23.017 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:45:23.017 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:45:23.017 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:45:23.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.009+0000 7f78c1069640 1 -- 172.21.15.86:0/776329271 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f78a80a3d20 msgr2=0x7f78a80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:23.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.009+0000 7f78c1069640 1 --2- 172.21.15.86:0/776329271 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f78a80a3d20 0x7f78a80a61e0 secure :-1 s=READY pgs=227 cs=0 l=1 rev1=1 crypto rx=0x7f78b4005cc0 tx=0x7f78b4005c50 comp rx=0 tx=0).stop 2024-04-21T08:45:23.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.009+0000 7f78c1069640 1 -- 172.21.15.86:0/776329271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78bc103ae0 msgr2=0x7f78bc198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:23.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.009+0000 7f78c1069640 1 --2- 172.21.15.86:0/776329271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78bc103ae0 0x7f78bc198c30 secure :-1 s=READY pgs=1422 cs=0 l=1 rev1=1 crypto rx=0x7f78a400c890 tx=0x7f78a400cd50 comp rx=0 tx=0).stop 2024-04-21T08:45:23.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.009+0000 7f78c1069640 1 -- 172.21.15.86:0/776329271 shutdown_connections 2024-04-21T08:45:23.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.009+0000 7f78c1069640 1 --2- 172.21.15.86:0/776329271 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f78a80a3d20 0x7f78a80a61e0 unknown :-1 s=CLOSED pgs=227 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.009+0000 7f78c1069640 1 --2- 172.21.15.86:0/776329271 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f78bc10c590 0x7f78bc19cfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.009+0000 7f78c1069640 1 --2- 172.21.15.86:0/776329271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f78bc103ae0 0x7f78bc198c30 unknown :-1 s=CLOSED pgs=1422 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.009+0000 7f78c1069640 1 --2- 172.21.15.86:0/776329271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f78bc1031c0 0x7f78bc1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:23.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.009+0000 7f78c1069640 1 -- 172.21.15.86:0/776329271 >> 172.21.15.86:0/776329271 conn(0x7f78bc077f70 msgr2=0x7f78bc0fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:23.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.010+0000 7f78c1069640 1 -- 172.21.15.86:0/776329271 shutdown_connections 2024-04-21T08:45:23.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:23.010+0000 7f78c1069640 1 -- 172.21.15.86:0/776329271 wait complete. 2024-04-21T08:45:23.020 INFO:tasks.cephfs.filesystem:scrub status for tag:c5e73be7-e8ad-41ea-a0ee-2ee7e8f6c988 - {'path': '/', 'tag': 'c5e73be7-e8ad-41ea-a0ee-2ee7e8f6c988', 'options': 'recursive,force'} 2024-04-21T08:45:24.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:23 smithi086 ceph-mon[27939]: pgmap v203: 129 pgs: 129 active+clean; 2.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-21T08:45:24.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:23 smithi086 ceph-mon[27939]: scrub queued for path: / 2024-04-21T08:45:24.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:23 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:45:24.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:23 smithi086 ceph-mon[27939]: scrub summary: active paths [/] 2024-04-21T08:45:24.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:23 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:45:24.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:23 smithi088 ceph-mon[31259]: pgmap v203: 129 pgs: 129 active+clean; 2.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-21T08:45:24.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:23 smithi088 ceph-mon[31259]: scrub queued for path: / 2024-04-21T08:45:24.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:23 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:45:24.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:23 smithi088 ceph-mon[31259]: scrub summary: active paths [/] 2024-04-21T08:45:24.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:23 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:45:24.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:23 smithi179 ceph-mon[31829]: pgmap v203: 129 pgs: 129 active+clean; 2.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-21T08:45:24.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:23 smithi179 ceph-mon[31829]: scrub queued for path: / 2024-04-21T08:45:24.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:23 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:45:24.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:23 smithi179 ceph-mon[31829]: scrub summary: active paths [/] 2024-04-21T08:45:24.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:23 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:45:25.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:24 smithi086 ceph-mon[27939]: scrub summary: idle 2024-04-21T08:45:25.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:24 smithi088 ceph-mon[31259]: scrub summary: idle 2024-04-21T08:45:25.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:24 smithi179 ceph-mon[31829]: scrub summary: idle 2024-04-21T08:45:26.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:25 smithi086 ceph-mon[27939]: pgmap v204: 129 pgs: 129 active+clean; 2.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:45:26.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:25 smithi088 ceph-mon[31259]: pgmap v204: 129 pgs: 129 active+clean; 2.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:45:26.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:25 smithi179 ceph-mon[31829]: pgmap v204: 129 pgs: 129 active+clean; 2.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:45:28.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:27 smithi086 ceph-mon[27939]: pgmap v205: 129 pgs: 129 active+clean; 2.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:45:28.130 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.131+0000 7f689093d640 1 -- 172.21.15.86:0/2765431184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f688c1031e0 msgr2=0x7f688c1035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.131+0000 7f689093d640 1 --2- 172.21.15.86:0/2765431184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f688c1031e0 0x7f688c1035c0 secure :-1 s=READY pgs=1427 cs=0 l=1 rev1=1 crypto rx=0x7f6874009900 tx=0x7f687402bd60 comp rx=0 tx=0).stop 2024-04-21T08:45:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.131+0000 7f689093d640 1 -- 172.21.15.86:0/2765431184 shutdown_connections 2024-04-21T08:45:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.131+0000 7f689093d640 1 --2- 172.21.15.86:0/2765431184 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f688c10c520 0x7f688c10e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.131+0000 7f689093d640 1 --2- 172.21.15.86:0/2765431184 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f688c103b00 0x7f688c10bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.131+0000 7f689093d640 1 --2- 172.21.15.86:0/2765431184 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f688c1031e0 0x7f688c1035c0 unknown :-1 s=CLOSED pgs=1427 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.131+0000 7f689093d640 1 -- 172.21.15.86:0/2765431184 >> 172.21.15.86:0/2765431184 conn(0x7f688c077fd0 msgr2=0x7f688c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.131+0000 7f689093d640 1 -- 172.21.15.86:0/2765431184 shutdown_connections 2024-04-21T08:45:28.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.131+0000 7f689093d640 1 -- 172.21.15.86:0/2765431184 wait complete. 2024-04-21T08:45:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.132+0000 7f689093d640 1 Processor -- start 2024-04-21T08:45:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.132+0000 7f689093d640 1 -- start start 2024-04-21T08:45:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.132+0000 7f689093d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f688c1031e0 0x7f688c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.132+0000 7f689093d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f688c103b00 0x7f688c198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.132+0000 7f689093d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f688c10c520 0x7f688c19cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.132+0000 7f689093d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f688c112210 con 0x7f688c103b00 2024-04-21T08:45:28.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.132+0000 7f689093d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f688c112090 con 0x7f688c1031e0 2024-04-21T08:45:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.132+0000 7f689093d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f688c112390 con 0x7f688c10c520 2024-04-21T08:45:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.132+0000 7f6889d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f688c103b00 0x7f688c198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.132+0000 7f6889d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f688c103b00 0x7f688c198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48084/0 (socket says 172.21.15.86:48084) 2024-04-21T08:45:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.132+0000 7f6889d9b640 1 -- 172.21.15.86:0/3280702729 learned_addr learned my addr 172.21.15.86:0/3280702729 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.132+0000 7f688a59c640 1 --2- 172.21.15.86:0/3280702729 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f688c1031e0 0x7f688c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.132+0000 7f688ad9d640 1 --2- 172.21.15.86:0/3280702729 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f688c10c520 0x7f688c19cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.133+0000 7f6889d9b640 1 -- 172.21.15.86:0/3280702729 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f688c10c520 msgr2=0x7f688c19cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.133+0000 7f6889d9b640 1 --2- 172.21.15.86:0/3280702729 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f688c10c520 0x7f688c19cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.133+0000 7f6889d9b640 1 -- 172.21.15.86:0/3280702729 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f688c1031e0 msgr2=0x7f688c1986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.133+0000 7f6889d9b640 1 --2- 172.21.15.86:0/3280702729 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f688c1031e0 0x7f688c1986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.133+0000 7f6889d9b640 1 -- 172.21.15.86:0/3280702729 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f688c19d690 con 0x7f688c103b00 2024-04-21T08:45:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.133+0000 7f6889d9b640 1 --2- 172.21.15.86:0/3280702729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f688c103b00 0x7f688c198c30 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f687c00c920 tx=0x7f687c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:28.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.133+0000 7f68837fe640 1 -- 172.21.15.86:0/3280702729 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c007c70 con 0x7f688c103b00 2024-04-21T08:45:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.133+0000 7f68837fe640 1 -- 172.21.15.86:0/3280702729 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f687c007e10 con 0x7f688c103b00 2024-04-21T08:45:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.133+0000 7f689093d640 1 -- 172.21.15.86:0/3280702729 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f688c19d980 con 0x7f688c103b00 2024-04-21T08:45:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.133+0000 7f68837fe640 1 -- 172.21.15.86:0/3280702729 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c018820 con 0x7f688c103b00 2024-04-21T08:45:28.135 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.133+0000 7f689093d640 1 -- 172.21.15.86:0/3280702729 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f688c1a5210 con 0x7f688c103b00 2024-04-21T08:45:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.135+0000 7f68837fe640 1 -- 172.21.15.86:0/3280702729 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f687c00f450 con 0x7f688c103b00 2024-04-21T08:45:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.135+0000 7f689360a740 1 Processor -- start 2024-04-21T08:45:28.141 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.136+0000 7f689360a740 1 -- start start 2024-04-21T08:45:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.136+0000 7f689360a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b974d738d0 0x55b974d72620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.136+0000 7f689360a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b974d72b60 0x55b974d72fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.136+0000 7f68837fe640 1 --2- 172.21.15.86:0/3280702729 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f68780a3dc0 0x7f68780a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.136+0000 7f689360a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b974d804f0 0x55b974d828e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.136+0000 7f689360a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b974cd78d0 con 0x55b974d738d0 2024-04-21T08:45:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.136+0000 7f689360a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b974c74e80 con 0x55b974d72b60 2024-04-21T08:45:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.136+0000 7f689360a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b974cd3890 con 0x55b974d804f0 2024-04-21T08:45:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.136+0000 7f688ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b974d738d0 0x55b974d72620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.136+0000 7f688ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b974d738d0 0x55b974d72620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48092/0 (socket says 172.21.15.86:48092) 2024-04-21T08:45:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.136+0000 7f688ad9d640 1 -- 172.21.15.86:0/2543456237 learned_addr learned my addr 172.21.15.86:0/2543456237 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.136+0000 7f688a59c640 1 --2- 172.21.15.86:0/3280702729 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f68780a3dc0 0x7f68780a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.136+0000 7f68837fe640 1 -- 172.21.15.86:0/3280702729 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f687c0d0d00 con 0x7f688c103b00 2024-04-21T08:45:28.142 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.136+0000 7f6889d9b640 1 --2- 172.21.15.86:0/2543456237 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b974d72b60 0x55b974d72fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.136+0000 7f688ad9d640 1 --2- 172.21.15.86:0/2543456237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b974d804f0 0x55b974d828e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.136+0000 7f688ad9d640 1 -- 172.21.15.86:0/2543456237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b974d804f0 msgr2=0x55b974d828e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.136+0000 7f688ad9d640 1 --2- 172.21.15.86:0/2543456237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b974d804f0 0x55b974d828e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.136+0000 7f688ad9d640 1 -- 172.21.15.86:0/2543456237 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b974d72b60 msgr2=0x55b974d72fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.136+0000 7f688ad9d640 1 --2- 172.21.15.86:0/2543456237 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b974d72b60 0x55b974d72fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.136+0000 7f688ad9d640 1 -- 172.21.15.86:0/2543456237 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b974c96320 con 0x55b974d738d0 2024-04-21T08:45:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.137+0000 7f688a59c640 1 --2- 172.21.15.86:0/3280702729 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f68780a3dc0 0x7f68780a6280 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6874009230 tx=0x7f6874005fd0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.137+0000 7f688ad9d640 1 --2- 172.21.15.86:0/2543456237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b974d738d0 0x55b974d72620 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f68840070b0 tx=0x7f68840099c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=76fab3562e3edccb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.137+0000 7f6880ff9640 1 -- 172.21.15.86:0/2543456237 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688400c2d0 con 0x55b974d738d0 2024-04-21T08:45:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.137+0000 7f6880ff9640 1 -- 172.21.15.86:0/2543456237 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6884010030 con 0x55b974d738d0 2024-04-21T08:45:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.137+0000 7f6880ff9640 1 -- 172.21.15.86:0/2543456237 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688400aa10 con 0x55b974d738d0 2024-04-21T08:45:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.138+0000 7f689360a740 1 -- 172.21.15.86:0/2543456237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b974d738d0 msgr2=0x55b974d72620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.138+0000 7f689360a740 1 --2- 172.21.15.86:0/2543456237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b974d738d0 0x55b974d72620 secure :-1 s=READY pgs=1429 cs=0 l=1 rev1=1 crypto rx=0x7f68840070b0 tx=0x7f68840099c0 comp rx=0 tx=0).stop 2024-04-21T08:45:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.138+0000 7f689360a740 1 -- 172.21.15.86:0/2543456237 shutdown_connections 2024-04-21T08:45:28.143 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.138+0000 7f689360a740 1 --2- 172.21.15.86:0/2543456237 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b974d804f0 0x55b974d828e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.138+0000 7f689360a740 1 --2- 172.21.15.86:0/2543456237 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b974d72b60 0x55b974d72fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.138+0000 7f689360a740 1 --2- 172.21.15.86:0/2543456237 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b974d738d0 0x55b974d72620 unknown :-1 s=CLOSED pgs=1429 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.138+0000 7f689360a740 1 -- 172.21.15.86:0/2543456237 >> 172.21.15.86:0/2543456237 conn(0x55b974d39c90 msgr2=0x55b974d6d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.138+0000 7f689360a740 1 -- 172.21.15.86:0/2543456237 shutdown_connections 2024-04-21T08:45:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.138+0000 7f689360a740 1 -- 172.21.15.86:0/2543456237 wait complete. 2024-04-21T08:45:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.138+0000 7f689360a740 1 -- start start 2024-04-21T08:45:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.138+0000 7f689360a740 1 Processor -- start 2024-04-21T08:45:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.138+0000 7f689360a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b974d72b60 0x55b974d7fdf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.138+0000 7f689360a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b974d738d0 0x55b974d7d1f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.138+0000 7f689360a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b974d804f0 0x55b974d7d750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.138+0000 7f689360a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b974d83220 con 0x55b974d804f0 2024-04-21T08:45:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.138+0000 7f689360a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b974c5feb0 con 0x55b974d738d0 2024-04-21T08:45:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.138+0000 7f689360a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b974d833a0 con 0x55b974d72b60 2024-04-21T08:45:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.138+0000 7f689360a740 10 client.0 _ll_register_callbacks cb 0x55b974c8cb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:28.144 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.138+0000 7f688ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b974d804f0 0x55b974d7d750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.139+0000 7f688ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b974d804f0 0x55b974d7d750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48108/0 (socket says 172.21.15.86:48108) 2024-04-21T08:45:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.139+0000 7f688ad9d640 1 -- 172.21.15.86:0/3864841512 learned_addr learned my addr 172.21.15.86:0/3864841512 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.139+0000 7f6889d9b640 1 --2- 172.21.15.86:0/3864841512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b974d738d0 0x55b974d7d1f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.139+0000 7f688a59c640 1 --2- 172.21.15.86:0/3864841512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b974d72b60 0x55b974d7fdf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.139+0000 7f688ad9d640 1 -- 172.21.15.86:0/3864841512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b974d72b60 msgr2=0x55b974d7fdf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.139+0000 7f688ad9d640 1 --2- 172.21.15.86:0/3864841512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b974d72b60 0x55b974d7fdf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.139+0000 7f688ad9d640 1 -- 172.21.15.86:0/3864841512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b974d738d0 msgr2=0x55b974d7d1f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.139+0000 7f688ad9d640 1 --2- 172.21.15.86:0/3864841512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b974d738d0 0x55b974d7d1f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.139+0000 7f688ad9d640 1 -- 172.21.15.86:0/3864841512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9749ea210 con 0x55b974d804f0 2024-04-21T08:45:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.139+0000 7f6889d9b640 1 --2- 172.21.15.86:0/3864841512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b974d738d0 0x55b974d7d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.139+0000 7f688ad9d640 1 --2- 172.21.15.86:0/3864841512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b974d804f0 0x55b974d7d750 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f6884004a80 tx=0x7f688400c5a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ab91ddd33ccc0a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.139+0000 7f6871ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.139+0000 7f6871ffb640 1 -- 172.21.15.86:0/3864841512 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f688400a9f0 con 0x55b974d804f0 2024-04-21T08:45:28.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.139+0000 7f6871ffb640 1 -- 172.21.15.86:0/3864841512 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f688400ab90 con 0x55b974d804f0 2024-04-21T08:45:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.139+0000 7f689360a740 1 -- 172.21.15.86:0/3864841512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6884010030 con 0x55b974d804f0 2024-04-21T08:45:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.139+0000 7f6871ffb640 1 -- 172.21.15.86:0/3864841512 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f688400ad90 con 0x55b974d804f0 2024-04-21T08:45:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.139+0000 7f689360a740 10 client.18468 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.139+0000 7f689360a740 1 -- 172.21.15.86:0/3864841512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b974ab6620 con 0x55b974d804f0 2024-04-21T08:45:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.139+0000 7f6871ffb640 1 -- 172.21.15.86:0/3864841512 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6884010030 con 0x55b974d804f0 2024-04-21T08:45:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.140+0000 7f6871ffb640 1 -- 172.21.15.86:0/3864841512 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f688401b570 con 0x55b974d804f0 2024-04-21T08:45:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.140+0000 7f689360a740 10 client.18468 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.140+0000 7f689360a740 10 client.18468 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.140+0000 7f689360a740 20 client.18468 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.140+0000 7f689360a740 1 --2- 172.21.15.86:0/3864841512 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b974d90410 0x55b974d92800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.140+0000 7f689360a740 4 client.18468 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.140+0000 7f689360a740 1 -- 172.21.15.86:0/3864841512 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b974ceb880 con 0x55b974d90410 2024-04-21T08:45:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.140+0000 7f688a59c640 1 --2- 172.21.15.86:0/3864841512 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b974d90410 0x55b974d92800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:28.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.140+0000 7f6871ffb640 10 client.18468 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.140+0000 7f688a59c640 1 --2- 172.21.15.86:0/3864841512 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b974d90410 0x55b974d92800 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3b17a5942ecf67a server_cookie=59a9e7e5dcc91d0e in_seq=0 out_seq=0 2024-04-21T08:45:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.141+0000 7f6871ffb640 1 -- 172.21.15.86:0/3864841512 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b974ceb880 con 0x55b974d90410 2024-04-21T08:45:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.141+0000 7f6871ffb640 10 client.18468 handle_command_reply: tid=0 2024-04-21T08:45:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.141+0000 7f689360a740 1 client.18468 shutdown 2024-04-21T08:45:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.141+0000 7f689360a740 1 -- 172.21.15.86:0/3864841512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b974d804f0 msgr2=0x55b974d7d750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.141+0000 7f689360a740 1 --2- 172.21.15.86:0/3864841512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b974d804f0 0x55b974d7d750 secure :-1 s=READY pgs=1430 cs=0 l=1 rev1=1 crypto rx=0x7f6884004a80 tx=0x7f688400c5a0 comp rx=0 tx=0).stop 2024-04-21T08:45:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.141+0000 7f689360a740 1 -- 172.21.15.86:0/3864841512 shutdown_connections 2024-04-21T08:45:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.141+0000 7f689360a740 1 --2- 172.21.15.86:0/3864841512 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b974d90410 0x55b974d92800 crc :-1 s=READY pgs=213 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.141+0000 7f689360a740 1 --2- 172.21.15.86:0/3864841512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b974d804f0 0x55b974d7d750 unknown :-1 s=CLOSED pgs=1430 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.141+0000 7f689360a740 1 --2- 172.21.15.86:0/3864841512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b974d738d0 0x55b974d7d1f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.141+0000 7f6871ffb640 0 client.18468 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.141+0000 7f689360a740 1 --2- 172.21.15.86:0/3864841512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b974d72b60 0x55b974d7fdf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.141+0000 7f689360a740 1 -- 172.21.15.86:0/3864841512 >> 172.21.15.86:0/3864841512 conn(0x55b974d39c90 msgr2=0x55b974d6f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.141+0000 7f689360a740 1 -- 172.21.15.86:0/3864841512 shutdown_connections 2024-04-21T08:45:28.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.141+0000 7f689360a740 1 -- 172.21.15.86:0/3864841512 wait complete. 2024-04-21T08:45:28.148 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.141+0000 7f689360a740 20 client.18468 trim_cache size 0 max 16384 2024-04-21T08:45:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.156+0000 7f689360a740 1 Processor -- start 2024-04-21T08:45:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.156+0000 7f689360a740 1 -- start start 2024-04-21T08:45:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.156+0000 7f689360a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b974d72b60 0x55b974d8c120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.156+0000 7f689360a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b974d738d0 0x55b974d7cb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.156+0000 7f689360a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b974d804f0 0x55b974d72080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.156+0000 7f689360a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f688c1ad170 con 0x55b974d72b60 2024-04-21T08:45:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.156+0000 7f689360a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f688c1aecb0 con 0x55b974d804f0 2024-04-21T08:45:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.156+0000 7f689360a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f688c1ad840 con 0x55b974d738d0 2024-04-21T08:45:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.156+0000 7f688a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b974d72b60 0x55b974d8c120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.156+0000 7f688a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b974d72b60 0x55b974d8c120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48118/0 (socket says 172.21.15.86:48118) 2024-04-21T08:45:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.156+0000 7f688a59c640 1 -- 172.21.15.86:0/1804392033 learned_addr learned my addr 172.21.15.86:0/1804392033 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:28.160 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.157+0000 7f688ad9d640 1 --2- 172.21.15.86:0/1804392033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b974d804f0 0x55b974d72080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.157+0000 7f688a59c640 1 -- 172.21.15.86:0/1804392033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b974d738d0 msgr2=0x55b974d7cb30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.157+0000 7f688a59c640 1 --2- 172.21.15.86:0/1804392033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b974d738d0 0x55b974d7cb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.157+0000 7f688a59c640 1 -- 172.21.15.86:0/1804392033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b974d804f0 msgr2=0x55b974d72080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.157+0000 7f688a59c640 1 --2- 172.21.15.86:0/1804392033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b974d804f0 0x55b974d72080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.157+0000 7f688a59c640 1 -- 172.21.15.86:0/1804392033 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b974d71e30 con 0x55b974d72b60 2024-04-21T08:45:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.157+0000 7f688ad9d640 1 --2- 172.21.15.86:0/1804392033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b974d804f0 0x55b974d72080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.157+0000 7f688a59c640 1 --2- 172.21.15.86:0/1804392033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b974d72b60 0x55b974d8c120 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f687402c2a0 tx=0x7f687403e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=43ee8debb595d4da server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.157+0000 7f6873fff640 1 -- 172.21.15.86:0/1804392033 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6874047070 con 0x55b974d72b60 2024-04-21T08:45:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.157+0000 7f6873fff640 1 -- 172.21.15.86:0/1804392033 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6874056070 con 0x55b974d72b60 2024-04-21T08:45:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.157+0000 7f6873fff640 1 -- 172.21.15.86:0/1804392033 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6874004cd0 con 0x55b974d72b60 2024-04-21T08:45:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.157+0000 7f689360a740 1 -- 172.21.15.86:0/1804392033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b974d72b60 msgr2=0x55b974d8c120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.157+0000 7f689360a740 1 --2- 172.21.15.86:0/1804392033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b974d72b60 0x55b974d8c120 secure :-1 s=READY pgs=1431 cs=0 l=1 rev1=1 crypto rx=0x7f687402c2a0 tx=0x7f687403e000 comp rx=0 tx=0).stop 2024-04-21T08:45:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.157+0000 7f689360a740 1 -- 172.21.15.86:0/1804392033 shutdown_connections 2024-04-21T08:45:28.161 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.158+0000 7f689360a740 1 --2- 172.21.15.86:0/1804392033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b974d804f0 0x55b974d72080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.158+0000 7f689360a740 1 --2- 172.21.15.86:0/1804392033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b974d738d0 0x55b974d7cb30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.158+0000 7f689360a740 1 --2- 172.21.15.86:0/1804392033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b974d72b60 0x55b974d8c120 unknown :-1 s=CLOSED pgs=1431 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.158+0000 7f689360a740 1 -- 172.21.15.86:0/1804392033 >> 172.21.15.86:0/1804392033 conn(0x55b974d39c90 msgr2=0x55b974d6ec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.158+0000 7f689360a740 1 -- 172.21.15.86:0/1804392033 shutdown_connections 2024-04-21T08:45:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.158+0000 7f689360a740 1 -- 172.21.15.86:0/1804392033 wait complete. 2024-04-21T08:45:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.158+0000 7f689360a740 1 -- start start 2024-04-21T08:45:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.158+0000 7f689360a740 1 Processor -- start 2024-04-21T08:45:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.158+0000 7f689360a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b974d72b60 0x55b974d88820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.158+0000 7f689360a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b974d738d0 0x55b974d79ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.158+0000 7f689360a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b974d804f0 0x55b974d7a000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.158+0000 7f689360a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f688c1ae5e0 con 0x55b974d738d0 2024-04-21T08:45:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.158+0000 7f689360a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f688c1adf10 con 0x55b974d72b60 2024-04-21T08:45:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.158+0000 7f689360a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f688c1ab170 con 0x55b974d804f0 2024-04-21T08:45:28.162 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.158+0000 7f689360a740 10 client.0 _ll_register_callbacks cb 0x7f687c00a560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.158+0000 7f6889d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b974d738d0 0x55b974d79ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.158+0000 7f6889d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b974d738d0 0x55b974d79ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48124/0 (socket says 172.21.15.86:48124) 2024-04-21T08:45:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.158+0000 7f6889d9b640 1 -- 172.21.15.86:0/1371016100 learned_addr learned my addr 172.21.15.86:0/1371016100 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.158+0000 7f688a59c640 1 --2- 172.21.15.86:0/1371016100 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b974d72b60 0x55b974d88820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.158+0000 7f688ad9d640 1 --2- 172.21.15.86:0/1371016100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b974d804f0 0x55b974d7a000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.158+0000 7f6889d9b640 1 -- 172.21.15.86:0/1371016100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b974d804f0 msgr2=0x55b974d7a000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.158+0000 7f6889d9b640 1 --2- 172.21.15.86:0/1371016100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b974d804f0 0x55b974d7a000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.158+0000 7f6889d9b640 1 -- 172.21.15.86:0/1371016100 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b974d72b60 msgr2=0x55b974d88820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.158+0000 7f6889d9b640 1 --2- 172.21.15.86:0/1371016100 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b974d72b60 0x55b974d88820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.159+0000 7f6889d9b640 1 -- 172.21.15.86:0/1371016100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b974d68f30 con 0x55b974d738d0 2024-04-21T08:45:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.159+0000 7f6889d9b640 1 --2- 172.21.15.86:0/1371016100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b974d738d0 0x55b974d79ac0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f687c09c5d0 tx=0x7f687c00fcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d003ae200341152 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.159+0000 7f6871ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.159+0000 7f6871ffb640 1 -- 172.21.15.86:0/1371016100 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f687c09cfd0 con 0x55b974d738d0 2024-04-21T08:45:28.163 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.159+0000 7f6871ffb640 1 -- 172.21.15.86:0/1371016100 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f687c09a970 con 0x55b974d738d0 2024-04-21T08:45:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.159+0000 7f689360a740 1 -- 172.21.15.86:0/1371016100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6874056070 con 0x55b974d738d0 2024-04-21T08:45:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.159+0000 7f6871ffb640 1 -- 172.21.15.86:0/1371016100 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f687c09c8d0 con 0x55b974d738d0 2024-04-21T08:45:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.159+0000 7f689360a740 10 client.18474 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.159+0000 7f689360a740 1 -- 172.21.15.86:0/1371016100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b974df6420 con 0x55b974d738d0 2024-04-21T08:45:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.159+0000 7f6871ffb640 1 -- 172.21.15.86:0/1371016100 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6874056070 con 0x55b974d738d0 2024-04-21T08:45:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.159+0000 7f6871ffb640 1 -- 172.21.15.86:0/1371016100 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f687c0870b0 con 0x55b974d738d0 2024-04-21T08:45:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.159+0000 7f689360a740 10 client.18474 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.159+0000 7f689360a740 10 client.18474 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.159+0000 7f689360a740 20 client.18474 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.159+0000 7f689360a740 1 --2- 172.21.15.86:0/1371016100 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b974df6ba0 0x55b974df8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.159+0000 7f689360a740 4 client.18474 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:45:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.159+0000 7f689360a740 1 -- 172.21.15.86:0/1371016100 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b974c773d0 con 0x55b974df6ba0 2024-04-21T08:45:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.160+0000 7f688a59c640 1 --2- 172.21.15.86:0/1371016100 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b974df6ba0 0x55b974df8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.160+0000 7f6871ffb640 10 client.18474 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:28.164 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.160+0000 7f688a59c640 1 --2- 172.21.15.86:0/1371016100 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b974df6ba0 0x55b974df8fc0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d3abf2dcacfe72d9 server_cookie=736d8b90ec17498f in_seq=0 out_seq=0 2024-04-21T08:45:28.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.175+0000 7f6871ffb640 1 -- 172.21.15.86:0/1371016100 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b974c773d0 con 0x55b974df6ba0 2024-04-21T08:45:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.175+0000 7f6871ffb640 10 client.18474 handle_command_reply: tid=0 2024-04-21T08:45:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.175+0000 7f689360a740 1 client.18474 shutdown 2024-04-21T08:45:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.175+0000 7f689360a740 1 -- 172.21.15.86:0/1371016100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b974d738d0 msgr2=0x55b974d79ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.175+0000 7f689360a740 1 --2- 172.21.15.86:0/1371016100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b974d738d0 0x55b974d79ac0 secure :-1 s=READY pgs=1432 cs=0 l=1 rev1=1 crypto rx=0x7f687c09c5d0 tx=0x7f687c00fcc0 comp rx=0 tx=0).stop 2024-04-21T08:45:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.175+0000 7f689360a740 1 -- 172.21.15.86:0/1371016100 shutdown_connections 2024-04-21T08:45:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.175+0000 7f689360a740 1 --2- 172.21.15.86:0/1371016100 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b974df6ba0 0x55b974df8fc0 crc :-1 s=READY pgs=214 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.175+0000 7f689360a740 1 --2- 172.21.15.86:0/1371016100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b974d804f0 0x55b974d7a000 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.175+0000 7f689360a740 1 --2- 172.21.15.86:0/1371016100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b974d738d0 0x55b974d79ac0 unknown :-1 s=CLOSED pgs=1432 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.175+0000 7f689360a740 1 --2- 172.21.15.86:0/1371016100 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b974d72b60 0x55b974d88820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.175+0000 7f6871ffb640 0 client.18474 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.175+0000 7f689360a740 1 -- 172.21.15.86:0/1371016100 >> 172.21.15.86:0/1371016100 conn(0x55b974d39c90 msgr2=0x55b974d6c040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.175+0000 7f689360a740 1 -- 172.21.15.86:0/1371016100 shutdown_connections 2024-04-21T08:45:28.175 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.175+0000 7f689360a740 1 -- 172.21.15.86:0/1371016100 wait complete. 2024-04-21T08:45:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.175+0000 7f689360a740 20 client.18474 trim_cache size 0 max 16384 2024-04-21T08:45:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:45:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:45:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:45:28.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:45:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.175+0000 7f689093d640 1 -- 172.21.15.86:0/3280702729 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f68780a3dc0 msgr2=0x7f68780a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.175+0000 7f689093d640 1 --2- 172.21.15.86:0/3280702729 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f68780a3dc0 0x7f68780a6280 secure :-1 s=READY pgs=228 cs=0 l=1 rev1=1 crypto rx=0x7f6874009230 tx=0x7f6874005fd0 comp rx=0 tx=0).stop 2024-04-21T08:45:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.175+0000 7f689093d640 1 -- 172.21.15.86:0/3280702729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f688c103b00 msgr2=0x7f688c198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.175+0000 7f689093d640 1 --2- 172.21.15.86:0/3280702729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f688c103b00 0x7f688c198c30 secure :-1 s=READY pgs=1428 cs=0 l=1 rev1=1 crypto rx=0x7f687c00c920 tx=0x7f687c00cde0 comp rx=0 tx=0).stop 2024-04-21T08:45:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.176+0000 7f689093d640 1 -- 172.21.15.86:0/3280702729 shutdown_connections 2024-04-21T08:45:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.176+0000 7f689093d640 1 --2- 172.21.15.86:0/3280702729 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f68780a3dc0 0x7f68780a6280 unknown :-1 s=CLOSED pgs=228 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.176+0000 7f689093d640 1 --2- 172.21.15.86:0/3280702729 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f688c10c520 0x7f688c19cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.176+0000 7f689093d640 1 --2- 172.21.15.86:0/3280702729 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f688c103b00 0x7f688c198c30 unknown :-1 s=CLOSED pgs=1428 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.176+0000 7f689093d640 1 --2- 172.21.15.86:0/3280702729 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f688c1031e0 0x7f688c1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.176+0000 7f689093d640 1 -- 172.21.15.86:0/3280702729 >> 172.21.15.86:0/3280702729 conn(0x7f688c077fd0 msgr2=0x7f688c0fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:28.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.176+0000 7f689093d640 1 -- 172.21.15.86:0/3280702729 shutdown_connections 2024-04-21T08:45:28.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:28.176+0000 7f689093d640 1 -- 172.21.15.86:0/3280702729 wait complete. 2024-04-21T08:45:28.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:27 smithi088 ceph-mon[31259]: pgmap v205: 129 pgs: 129 active+clean; 2.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:45:28.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:27 smithi179 ceph-mon[31829]: pgmap v205: 129 pgs: 129 active+clean; 2.5 MiB data, 402 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:45:30.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:29 smithi086 ceph-mon[27939]: pgmap v206: 129 pgs: 129 active+clean; 2.7 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-21T08:45:30.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:29 smithi088 ceph-mon[31259]: pgmap v206: 129 pgs: 129 active+clean; 2.7 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-21T08:45:30.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:29 smithi179 ceph-mon[31829]: pgmap v206: 129 pgs: 129 active+clean; 2.7 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-21T08:45:32.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:31 smithi086 ceph-mon[27939]: pgmap v207: 129 pgs: 129 active+clean; 2.7 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 14 op/s 2024-04-21T08:45:32.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:31 smithi088 ceph-mon[31259]: pgmap v207: 129 pgs: 129 active+clean; 2.7 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 14 op/s 2024-04-21T08:45:32.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:31 smithi179 ceph-mon[31829]: pgmap v207: 129 pgs: 129 active+clean; 2.7 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 14 op/s 2024-04-21T08:45:34.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:33 smithi086 ceph-mon[27939]: pgmap v208: 129 pgs: 129 active+clean; 2.9 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-21T08:45:34.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:33 smithi088 ceph-mon[31259]: pgmap v208: 129 pgs: 129 active+clean; 2.9 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-21T08:45:34.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:33 smithi179 ceph-mon[31829]: pgmap v208: 129 pgs: 129 active+clean; 2.9 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-21T08:45:36.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:35 smithi086 ceph-mon[27939]: pgmap v209: 129 pgs: 129 active+clean; 2.9 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:45:36.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:35 smithi088 ceph-mon[31259]: pgmap v209: 129 pgs: 129 active+clean; 2.9 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:45:36.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:35 smithi179 ceph-mon[31829]: pgmap v209: 129 pgs: 129 active+clean; 2.9 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:45:38.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:37 smithi086 ceph-mon[27939]: pgmap v210: 129 pgs: 129 active+clean; 2.9 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:45:38.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:37 smithi088 ceph-mon[31259]: pgmap v210: 129 pgs: 129 active+clean; 2.9 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:45:38.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:37 smithi179 ceph-mon[31829]: pgmap v210: 129 pgs: 129 active+clean; 2.9 MiB data, 403 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:45:40.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:39 smithi086 ceph-mon[27939]: pgmap v211: 129 pgs: 129 active+clean; 3.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-21T08:45:40.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:39 smithi088 ceph-mon[31259]: pgmap v211: 129 pgs: 129 active+clean; 3.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-21T08:45:40.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:39 smithi179 ceph-mon[31829]: pgmap v211: 129 pgs: 129 active+clean; 3.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-21T08:45:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.272+0000 7f777ba5c640 1 -- 172.21.15.86:0/4198550143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7774113010 msgr2=0x7f7774115400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.272+0000 7f777ba5c640 1 --2- 172.21.15.86:0/4198550143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7774113010 0x7f7774115400 secure :-1 s=READY pgs=1433 cs=0 l=1 rev1=1 crypto rx=0x7f776c009ca0 tx=0x7f776c02c180 comp rx=0 tx=0).stop 2024-04-21T08:45:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.272+0000 7f777ba5c640 1 -- 172.21.15.86:0/4198550143 shutdown_connections 2024-04-21T08:45:41.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.272+0000 7f777ba5c640 1 --2- 172.21.15.86:0/4198550143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7774113010 0x7f7774115400 unknown :-1 s=CLOSED pgs=1433 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.272+0000 7f777ba5c640 1 --2- 172.21.15.86:0/4198550143 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7774074b70 0x7f7774112ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.272+0000 7f777ba5c640 1 --2- 172.21.15.86:0/4198550143 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7774074250 0x7f7774074630 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.272+0000 7f777ba5c640 1 -- 172.21.15.86:0/4198550143 >> 172.21.15.86:0/4198550143 conn(0x7f77740793e0 msgr2=0x7f77740797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.272+0000 7f777ba5c640 1 -- 172.21.15.86:0/4198550143 shutdown_connections 2024-04-21T08:45:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.272+0000 7f777ba5c640 1 -- 172.21.15.86:0/4198550143 wait complete. 2024-04-21T08:45:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.272+0000 7f777ba5c640 1 Processor -- start 2024-04-21T08:45:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.272+0000 7f777ba5c640 1 -- start start 2024-04-21T08:45:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.273+0000 7f777ba5c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7774074250 0x7f77741a4860 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.273+0000 7f777ba5c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7774074b70 0x7f77741a4da0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.273+0000 7f777ba5c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7774113010 0x7f777419ea10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:41.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.273+0000 7f777ba5c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f77741187f0 con 0x7f7774074b70 2024-04-21T08:45:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.273+0000 7f777ba5c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7774118670 con 0x7f7774113010 2024-04-21T08:45:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.273+0000 7f777ba5c640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7774118970 con 0x7f7774074250 2024-04-21T08:45:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.273+0000 7f7778ff7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7774074b70 0x7f77741a4da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.273+0000 7f7778ff7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7774074b70 0x7f77741a4da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55666/0 (socket says 172.21.15.86:55666) 2024-04-21T08:45:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.273+0000 7f7778ff7640 1 -- 172.21.15.86:0/3191706086 learned_addr learned my addr 172.21.15.86:0/3191706086 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.273+0000 7f7779ff9640 1 --2- 172.21.15.86:0/3191706086 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7774113010 0x7f777419ea10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.273+0000 7f77797f8640 1 --2- 172.21.15.86:0/3191706086 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7774074250 0x7f77741a4860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.273+0000 7f7778ff7640 1 -- 172.21.15.86:0/3191706086 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7774074250 msgr2=0x7f77741a4860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.273+0000 7f7778ff7640 1 --2- 172.21.15.86:0/3191706086 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7774074250 0x7f77741a4860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.273+0000 7f7778ff7640 1 -- 172.21.15.86:0/3191706086 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7774113010 msgr2=0x7f777419ea10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.273+0000 7f7778ff7640 1 --2- 172.21.15.86:0/3191706086 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7774113010 0x7f777419ea10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.273+0000 7f7778ff7640 1 -- 172.21.15.86:0/3191706086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f777419f190 con 0x7f7774074b70 2024-04-21T08:45:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.273+0000 7f7778ff7640 1 --2- 172.21.15.86:0/3191706086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7774074b70 0x7f77741a4da0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f776400c920 tx=0x7f776400cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:41.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.273+0000 7f776a7fc640 1 -- 172.21.15.86:0/3191706086 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7764007c70 con 0x7f7774074b70 2024-04-21T08:45:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.273+0000 7f776a7fc640 1 -- 172.21.15.86:0/3191706086 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7764007e10 con 0x7f7774074b70 2024-04-21T08:45:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.273+0000 7f777ba5c640 1 -- 172.21.15.86:0/3191706086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f777419f480 con 0x7f7774074b70 2024-04-21T08:45:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.273+0000 7f776a7fc640 1 -- 172.21.15.86:0/3191706086 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7764018820 con 0x7f7774074b70 2024-04-21T08:45:41.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.273+0000 7f777ba5c640 1 -- 172.21.15.86:0/3191706086 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77741ab720 con 0x7f7774074b70 2024-04-21T08:45:41.282 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.276+0000 7f776a7fc640 1 -- 172.21.15.86:0/3191706086 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f776400f450 con 0x7f7774074b70 2024-04-21T08:45:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.276+0000 7f777e650740 1 Processor -- start 2024-04-21T08:45:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.276+0000 7f777e650740 1 -- start start 2024-04-21T08:45:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.276+0000 7f777e650740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8d4008d0 0x55ac8d3ff620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.276+0000 7f777e650740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac8d3ffb60 0x55ac8d3fffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.276+0000 7f777e650740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac8d4090e0 0x55ac8d40b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.276+0000 7f777e650740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ac8d3648d0 con 0x55ac8d4008d0 2024-04-21T08:45:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.276+0000 7f776a7fc640 1 --2- 172.21.15.86:0/3191706086 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f77600a3dc0 0x7f77600a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.276+0000 7f777e650740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55ac8d301e80 con 0x55ac8d4090e0 2024-04-21T08:45:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.276+0000 7f777e650740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ac8d360890 con 0x55ac8d3ffb60 2024-04-21T08:45:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.277+0000 7f7779ff9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8d4008d0 0x55ac8d3ff620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.277+0000 7f7779ff9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8d4008d0 0x55ac8d3ff620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55680/0 (socket says 172.21.15.86:55680) 2024-04-21T08:45:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.277+0000 7f7779ff9640 1 -- 172.21.15.86:0/837063682 learned_addr learned my addr 172.21.15.86:0/837063682 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:41.283 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.277+0000 7f7779ff9640 1 --2- 172.21.15.86:0/3191706086 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f77600a3dc0 0x7f77600a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.277+0000 7f77797f8640 1 --2- 172.21.15.86:0/837063682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac8d3ffb60 0x55ac8d3fffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.277+0000 7f776a7fc640 1 -- 172.21.15.86:0/3191706086 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f77640d0c80 con 0x7f7774074b70 2024-04-21T08:45:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.277+0000 7f7778ff7640 1 --2- 172.21.15.86:0/837063682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac8d4090e0 0x55ac8d40b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.277+0000 7f7779ff9640 1 -- 172.21.15.86:0/837063682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac8d3ffb60 msgr2=0x55ac8d3fffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.277+0000 7f7779ff9640 1 --2- 172.21.15.86:0/837063682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac8d3ffb60 0x55ac8d3fffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.277+0000 7f7779ff9640 1 -- 172.21.15.86:0/837063682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac8d4090e0 msgr2=0x55ac8d40b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.277+0000 7f7779ff9640 1 --2- 172.21.15.86:0/837063682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac8d4090e0 0x55ac8d40b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.277+0000 7f7779ff9640 1 -- 172.21.15.86:0/837063682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac8d323320 con 0x55ac8d4008d0 2024-04-21T08:45:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.277+0000 7f7779ff9640 1 --2- 172.21.15.86:0/837063682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8d4008d0 0x55ac8d3ff620 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f777406b630 tx=0x7f776c0040c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=674237e0bbbdba8e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.277+0000 7f7779ff9640 1 --2- 172.21.15.86:0/3191706086 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f77600a3dc0 0x7f77600a6280 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f776c009ca0 tx=0x7f776c042000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.277+0000 7f775f7fe640 1 -- 172.21.15.86:0/837063682 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c051070 con 0x55ac8d4008d0 2024-04-21T08:45:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.277+0000 7f775f7fe640 1 -- 172.21.15.86:0/837063682 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f776c0047a0 con 0x55ac8d4008d0 2024-04-21T08:45:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.277+0000 7f775f7fe640 1 -- 172.21.15.86:0/837063682 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c039040 con 0x55ac8d4008d0 2024-04-21T08:45:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.278+0000 7f777e650740 1 -- 172.21.15.86:0/837063682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8d4008d0 msgr2=0x55ac8d3ff620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:41.284 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.278+0000 7f777e650740 1 --2- 172.21.15.86:0/837063682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8d4008d0 0x55ac8d3ff620 secure :-1 s=READY pgs=1435 cs=0 l=1 rev1=1 crypto rx=0x7f777406b630 tx=0x7f776c0040c0 comp rx=0 tx=0).stop 2024-04-21T08:45:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.278+0000 7f777e650740 1 -- 172.21.15.86:0/837063682 shutdown_connections 2024-04-21T08:45:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.278+0000 7f777e650740 1 --2- 172.21.15.86:0/837063682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac8d4090e0 0x55ac8d40b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.278+0000 7f777e650740 1 --2- 172.21.15.86:0/837063682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac8d3ffb60 0x55ac8d3fffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.278+0000 7f777e650740 1 --2- 172.21.15.86:0/837063682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8d4008d0 0x55ac8d3ff620 unknown :-1 s=CLOSED pgs=1435 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.278+0000 7f777e650740 1 -- 172.21.15.86:0/837063682 >> 172.21.15.86:0/837063682 conn(0x55ac8d3c6c90 msgr2=0x55ac8d3fa850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.278+0000 7f777e650740 1 -- 172.21.15.86:0/837063682 shutdown_connections 2024-04-21T08:45:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.278+0000 7f777e650740 1 -- 172.21.15.86:0/837063682 wait complete. 2024-04-21T08:45:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.279+0000 7f777e650740 1 -- start start 2024-04-21T08:45:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.279+0000 7f777e650740 1 Processor -- start 2024-04-21T08:45:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.279+0000 7f777e650740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac8d3ffb60 0x55ac8d408ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.279+0000 7f777e650740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac8d4008d0 0x55ac8d405e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.279+0000 7f777e650740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8d4090e0 0x55ac8d406450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.279+0000 7f777e650740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ac8d40bdb0 con 0x55ac8d4090e0 2024-04-21T08:45:41.285 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.279+0000 7f777e650740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55ac8d2eceb0 con 0x55ac8d3ffb60 2024-04-21T08:45:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.279+0000 7f777e650740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ac8d40bf30 con 0x55ac8d4008d0 2024-04-21T08:45:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.279+0000 7f777e650740 10 client.0 _ll_register_callbacks cb 0x55ac8d319b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.279+0000 7f7779ff9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8d4090e0 0x55ac8d406450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.279+0000 7f7779ff9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8d4090e0 0x55ac8d406450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55688/0 (socket says 172.21.15.86:55688) 2024-04-21T08:45:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.279+0000 7f7779ff9640 1 -- 172.21.15.86:0/719241528 learned_addr learned my addr 172.21.15.86:0/719241528 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.279+0000 7f77797f8640 1 --2- 172.21.15.86:0/719241528 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac8d3ffb60 0x55ac8d408ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.279+0000 7f7778ff7640 1 --2- 172.21.15.86:0/719241528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac8d4008d0 0x55ac8d405e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.279+0000 7f7779ff9640 1 -- 172.21.15.86:0/719241528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac8d4008d0 msgr2=0x55ac8d405e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.279+0000 7f7779ff9640 1 --2- 172.21.15.86:0/719241528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac8d4008d0 0x55ac8d405e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.279+0000 7f7779ff9640 1 -- 172.21.15.86:0/719241528 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac8d3ffb60 msgr2=0x55ac8d408ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.279+0000 7f7779ff9640 1 --2- 172.21.15.86:0/719241528 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac8d3ffb60 0x55ac8d408ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.279+0000 7f7779ff9640 1 -- 172.21.15.86:0/719241528 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac8d077210 con 0x55ac8d4090e0 2024-04-21T08:45:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.280+0000 7f7779ff9640 1 --2- 172.21.15.86:0/719241528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8d4090e0 0x55ac8d406450 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f776c02c790 tx=0x7f776c032f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd2b04dcf1f3061 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:41.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.280+0000 7f775cff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.280+0000 7f775cff9640 1 -- 172.21.15.86:0/719241528 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776c039070 con 0x55ac8d4090e0 2024-04-21T08:45:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.280+0000 7f775cff9640 1 -- 172.21.15.86:0/719241528 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f776c004b40 con 0x55ac8d4090e0 2024-04-21T08:45:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.280+0000 7f777e650740 1 -- 172.21.15.86:0/719241528 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f776c0047a0 con 0x55ac8d4090e0 2024-04-21T08:45:41.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.280+0000 7f775cff9640 1 -- 172.21.15.86:0/719241528 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f776c051040 con 0x55ac8d4090e0 2024-04-21T08:45:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.280+0000 7f777e650740 10 client.18486 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.280+0000 7f777e650740 1 -- 172.21.15.86:0/719241528 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ac8d143620 con 0x55ac8d4090e0 2024-04-21T08:45:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.280+0000 7f775cff9640 1 -- 172.21.15.86:0/719241528 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f776c0047a0 con 0x55ac8d4090e0 2024-04-21T08:45:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.280+0000 7f775cff9640 1 -- 172.21.15.86:0/719241528 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f776c04b030 con 0x55ac8d4090e0 2024-04-21T08:45:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.280+0000 7f777e650740 10 client.18486 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.280+0000 7f777e650740 10 client.18486 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.280+0000 7f777e650740 20 client.18486 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.280+0000 7f777e650740 1 --2- 172.21.15.86:0/719241528 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ac8d418f40 0x55ac8d41b330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.280+0000 7f777e650740 4 client.18486 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:41.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.280+0000 7f777e650740 1 -- 172.21.15.86:0/719241528 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac8d378880 con 0x55ac8d418f40 2024-04-21T08:45:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.281+0000 7f77797f8640 1 --2- 172.21.15.86:0/719241528 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ac8d418f40 0x55ac8d41b330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.281+0000 7f775cff9640 10 client.18486 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.281+0000 7f77797f8640 1 --2- 172.21.15.86:0/719241528 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ac8d418f40 0x55ac8d41b330 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2b0fca9d8771db0 server_cookie=df4ab52c465199e5 in_seq=0 out_seq=0 2024-04-21T08:45:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.282+0000 7f775cff9640 1 -- 172.21.15.86:0/719241528 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ac8d378880 con 0x55ac8d418f40 2024-04-21T08:45:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.282+0000 7f775cff9640 10 client.18486 handle_command_reply: tid=0 2024-04-21T08:45:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.282+0000 7f777e650740 1 client.18486 shutdown 2024-04-21T08:45:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.282+0000 7f777e650740 1 -- 172.21.15.86:0/719241528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8d4090e0 msgr2=0x55ac8d406450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.282+0000 7f777e650740 1 --2- 172.21.15.86:0/719241528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8d4090e0 0x55ac8d406450 secure :-1 s=READY pgs=1436 cs=0 l=1 rev1=1 crypto rx=0x7f776c02c790 tx=0x7f776c032f10 comp rx=0 tx=0).stop 2024-04-21T08:45:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.282+0000 7f777e650740 1 -- 172.21.15.86:0/719241528 shutdown_connections 2024-04-21T08:45:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.282+0000 7f777e650740 1 --2- 172.21.15.86:0/719241528 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ac8d418f40 0x55ac8d41b330 crc :-1 s=READY pgs=215 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:41.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.282+0000 7f777e650740 1 --2- 172.21.15.86:0/719241528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8d4090e0 0x55ac8d406450 unknown :-1 s=CLOSED pgs=1436 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:41.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.282+0000 7f777e650740 1 --2- 172.21.15.86:0/719241528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac8d4008d0 0x55ac8d405e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:41.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.282+0000 7f775cff9640 0 client.18486 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:41.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.282+0000 7f777e650740 1 --2- 172.21.15.86:0/719241528 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac8d3ffb60 0x55ac8d408ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:41.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.282+0000 7f777e650740 1 -- 172.21.15.86:0/719241528 >> 172.21.15.86:0/719241528 conn(0x55ac8d3c6c90 msgr2=0x55ac8d3fc5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:41.290 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.282+0000 7f777e650740 1 -- 172.21.15.86:0/719241528 shutdown_connections 2024-04-21T08:45:41.291 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.282+0000 7f777e650740 1 -- 172.21.15.86:0/719241528 wait complete. 2024-04-21T08:45:41.291 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.282+0000 7f777e650740 20 client.18486 trim_cache size 0 max 16384 2024-04-21T08:45:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.297+0000 7f777e650740 1 Processor -- start 2024-04-21T08:45:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.297+0000 7f777e650740 1 -- start start 2024-04-21T08:45:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.297+0000 7f777e650740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac8d3ffb60 0x55ac8d414cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.297+0000 7f777e650740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8d4008d0 0x55ac8d4050b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.297+0000 7f777e650740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac8d4090e0 0x55ac8d408570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.297+0000 7f777e650740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f77741b5830 con 0x55ac8d4008d0 2024-04-21T08:45:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.297+0000 7f777e650740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f77741b7370 con 0x55ac8d3ffb60 2024-04-21T08:45:41.301 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.297+0000 7f777e650740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f77741b5f00 con 0x55ac8d4090e0 2024-04-21T08:45:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.297+0000 7f7778ff7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8d4008d0 0x55ac8d4050b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.297+0000 7f7778ff7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8d4008d0 0x55ac8d4050b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55694/0 (socket says 172.21.15.86:55694) 2024-04-21T08:45:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.297+0000 7f7778ff7640 1 -- 172.21.15.86:0/3747107047 learned_addr learned my addr 172.21.15.86:0/3747107047 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.297+0000 7f77797f8640 1 --2- 172.21.15.86:0/3747107047 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac8d3ffb60 0x55ac8d414cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.297+0000 7f7779ff9640 1 --2- 172.21.15.86:0/3747107047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac8d4090e0 0x55ac8d408570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.298+0000 7f7778ff7640 1 -- 172.21.15.86:0/3747107047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac8d4090e0 msgr2=0x55ac8d408570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.298+0000 7f7778ff7640 1 --2- 172.21.15.86:0/3747107047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac8d4090e0 0x55ac8d408570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.298+0000 7f7778ff7640 1 -- 172.21.15.86:0/3747107047 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac8d3ffb60 msgr2=0x55ac8d414cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.298+0000 7f7778ff7640 1 --2- 172.21.15.86:0/3747107047 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac8d3ffb60 0x55ac8d414cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.298+0000 7f7778ff7640 1 -- 172.21.15.86:0/3747107047 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac8d3fee30 con 0x55ac8d4008d0 2024-04-21T08:45:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.298+0000 7f77797f8640 1 --2- 172.21.15.86:0/3747107047 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac8d3ffb60 0x55ac8d414cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.298+0000 7f7779ff9640 1 --2- 172.21.15.86:0/3747107047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac8d4090e0 0x55ac8d408570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:41.302 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.298+0000 7f7778ff7640 1 --2- 172.21.15.86:0/3747107047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8d4008d0 0x55ac8d4050b0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f77640ce600 tx=0x7f7764087040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d857a0756073f0de server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.298+0000 7f775effd640 1 -- 172.21.15.86:0/3747107047 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f776409c850 con 0x55ac8d4008d0 2024-04-21T08:45:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.298+0000 7f775effd640 1 -- 172.21.15.86:0/3747107047 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77640a0c00 con 0x55ac8d4008d0 2024-04-21T08:45:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.298+0000 7f775effd640 1 -- 172.21.15.86:0/3747107047 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7764098ec0 con 0x55ac8d4008d0 2024-04-21T08:45:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.298+0000 7f777e650740 1 -- 172.21.15.86:0/3747107047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8d4008d0 msgr2=0x55ac8d4050b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.298+0000 7f777e650740 1 --2- 172.21.15.86:0/3747107047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8d4008d0 0x55ac8d4050b0 secure :-1 s=READY pgs=1437 cs=0 l=1 rev1=1 crypto rx=0x7f77640ce600 tx=0x7f7764087040 comp rx=0 tx=0).stop 2024-04-21T08:45:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.299+0000 7f777e650740 1 -- 172.21.15.86:0/3747107047 shutdown_connections 2024-04-21T08:45:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.299+0000 7f777e650740 1 --2- 172.21.15.86:0/3747107047 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac8d4090e0 0x55ac8d408570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.299+0000 7f777e650740 1 --2- 172.21.15.86:0/3747107047 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8d4008d0 0x55ac8d4050b0 unknown :-1 s=CLOSED pgs=1437 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.299+0000 7f777e650740 1 --2- 172.21.15.86:0/3747107047 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac8d3ffb60 0x55ac8d414cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.299+0000 7f777e650740 1 -- 172.21.15.86:0/3747107047 >> 172.21.15.86:0/3747107047 conn(0x55ac8d3c6c90 msgr2=0x55ac8d3fc4c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.299+0000 7f777e650740 1 -- 172.21.15.86:0/3747107047 shutdown_connections 2024-04-21T08:45:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.299+0000 7f777e650740 1 -- 172.21.15.86:0/3747107047 wait complete. 2024-04-21T08:45:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.299+0000 7f777e650740 1 -- start start 2024-04-21T08:45:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.299+0000 7f777e650740 1 Processor -- start 2024-04-21T08:45:41.303 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.299+0000 7f777e650740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac8d3ffb60 0x55ac8d4116a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.299+0000 7f777e650740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8d4008d0 0x55ac8d4030e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.299+0000 7f777e650740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac8d4090e0 0x55ac8d403710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.299+0000 7f777e650740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f77741b6ca0 con 0x55ac8d4008d0 2024-04-21T08:45:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.299+0000 7f777e650740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f77741b65d0 con 0x55ac8d3ffb60 2024-04-21T08:45:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.299+0000 7f777e650740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f77741b3880 con 0x55ac8d4090e0 2024-04-21T08:45:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.299+0000 7f777e650740 10 client.0 _ll_register_callbacks cb 0x7f776c04f070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.299+0000 7f7778ff7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8d4008d0 0x55ac8d4030e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.299+0000 7f7778ff7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8d4008d0 0x55ac8d4030e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55708/0 (socket says 172.21.15.86:55708) 2024-04-21T08:45:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.300+0000 7f7778ff7640 1 -- 172.21.15.86:0/1260127044 learned_addr learned my addr 172.21.15.86:0/1260127044 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.300+0000 7f7779ff9640 1 --2- 172.21.15.86:0/1260127044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac8d4090e0 0x55ac8d403710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.300+0000 7f77797f8640 1 --2- 172.21.15.86:0/1260127044 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac8d3ffb60 0x55ac8d4116a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.300+0000 7f7778ff7640 1 -- 172.21.15.86:0/1260127044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac8d4090e0 msgr2=0x55ac8d403710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.300+0000 7f7778ff7640 1 --2- 172.21.15.86:0/1260127044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac8d4090e0 0x55ac8d403710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:41.304 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.300+0000 7f7778ff7640 1 -- 172.21.15.86:0/1260127044 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac8d3ffb60 msgr2=0x55ac8d4116a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.300+0000 7f7778ff7640 1 --2- 172.21.15.86:0/1260127044 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac8d3ffb60 0x55ac8d4116a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.300+0000 7f7778ff7640 1 -- 172.21.15.86:0/1260127044 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac8d4020b0 con 0x55ac8d4008d0 2024-04-21T08:45:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.300+0000 7f77797f8640 1 --2- 172.21.15.86:0/1260127044 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac8d3ffb60 0x55ac8d4116a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.300+0000 7f7778ff7640 1 --2- 172.21.15.86:0/1260127044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8d4008d0 0x55ac8d4030e0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f776409c7e0 tx=0x7f776400fcc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c30805807b25b130 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.300+0000 7f775cff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.300+0000 7f775cff9640 1 -- 172.21.15.86:0/1260127044 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77640bb070 con 0x55ac8d4008d0 2024-04-21T08:45:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.300+0000 7f775cff9640 1 -- 172.21.15.86:0/1260127044 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f77640870b0 con 0x55ac8d4008d0 2024-04-21T08:45:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.300+0000 7f777e650740 1 -- 172.21.15.86:0/1260127044 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77640a0c00 con 0x55ac8d4008d0 2024-04-21T08:45:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.300+0000 7f775cff9640 1 -- 172.21.15.86:0/1260127044 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f77640672b0 con 0x55ac8d4008d0 2024-04-21T08:45:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.300+0000 7f777e650740 10 client.18492 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.300+0000 7f777e650740 1 -- 172.21.15.86:0/1260127044 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ac8d47f290 con 0x55ac8d4008d0 2024-04-21T08:45:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.300+0000 7f775cff9640 1 -- 172.21.15.86:0/1260127044 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77640a0c00 con 0x55ac8d4008d0 2024-04-21T08:45:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.301+0000 7f775cff9640 1 -- 172.21.15.86:0/1260127044 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f77640b2070 con 0x55ac8d4008d0 2024-04-21T08:45:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.301+0000 7f777e650740 10 client.18492 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:41.305 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.301+0000 7f777e650740 10 client.18492 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.301+0000 7f777e650740 20 client.18492 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.301+0000 7f777e650740 1 --2- 172.21.15.86:0/1260127044 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ac8d47f8d0 0x55ac8d481cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.301+0000 7f777e650740 4 client.18492 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:45:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.301+0000 7f777e650740 1 -- 172.21.15.86:0/1260127044 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ac8d3043d0 con 0x55ac8d47f8d0 2024-04-21T08:45:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.301+0000 7f77797f8640 1 --2- 172.21.15.86:0/1260127044 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ac8d47f8d0 0x55ac8d481cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.301+0000 7f77797f8640 1 --2- 172.21.15.86:0/1260127044 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ac8d47f8d0 0x55ac8d481cf0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95a50b5ab6873e59 server_cookie=772bcb2e5ba3496d in_seq=0 out_seq=0 2024-04-21T08:45:41.306 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.301+0000 7f775cff9640 10 client.18492 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.317+0000 7f775cff9640 1 -- 172.21.15.86:0/1260127044 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ac8d3043d0 con 0x55ac8d47f8d0 2024-04-21T08:45:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.317+0000 7f775cff9640 10 client.18492 handle_command_reply: tid=0 2024-04-21T08:45:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.317+0000 7f777e650740 1 client.18492 shutdown 2024-04-21T08:45:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.317+0000 7f777e650740 1 -- 172.21.15.86:0/1260127044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8d4008d0 msgr2=0x55ac8d4030e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.317+0000 7f777e650740 1 --2- 172.21.15.86:0/1260127044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8d4008d0 0x55ac8d4030e0 secure :-1 s=READY pgs=1438 cs=0 l=1 rev1=1 crypto rx=0x7f776409c7e0 tx=0x7f776400fcc0 comp rx=0 tx=0).stop 2024-04-21T08:45:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.317+0000 7f777e650740 1 -- 172.21.15.86:0/1260127044 shutdown_connections 2024-04-21T08:45:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.317+0000 7f777e650740 1 --2- 172.21.15.86:0/1260127044 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ac8d47f8d0 0x55ac8d481cf0 crc :-1 s=READY pgs=216 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.317+0000 7f777e650740 1 --2- 172.21.15.86:0/1260127044 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac8d4090e0 0x55ac8d403710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.317+0000 7f777e650740 1 --2- 172.21.15.86:0/1260127044 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac8d4008d0 0x55ac8d4030e0 unknown :-1 s=CLOSED pgs=1438 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.317+0000 7f775cff9640 0 client.18492 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.317+0000 7f777e650740 1 --2- 172.21.15.86:0/1260127044 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac8d3ffb60 0x55ac8d4116a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.317+0000 7f777e650740 1 -- 172.21.15.86:0/1260127044 >> 172.21.15.86:0/1260127044 conn(0x55ac8d3c6c90 msgr2=0x55ac8d3f9040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:41.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.317+0000 7f777e650740 1 -- 172.21.15.86:0/1260127044 shutdown_connections 2024-04-21T08:45:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.317+0000 7f777e650740 1 -- 172.21.15.86:0/1260127044 wait complete. 2024-04-21T08:45:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.317+0000 7f777e650740 20 client.18492 trim_cache size 0 max 16384 2024-04-21T08:45:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:45:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:45:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:45:41.318 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:45:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.318+0000 7f777ba5c640 1 -- 172.21.15.86:0/3191706086 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f77600a3dc0 msgr2=0x7f77600a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.318+0000 7f777ba5c640 1 --2- 172.21.15.86:0/3191706086 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f77600a3dc0 0x7f77600a6280 secure :-1 s=READY pgs=229 cs=0 l=1 rev1=1 crypto rx=0x7f776c009ca0 tx=0x7f776c042000 comp rx=0 tx=0).stop 2024-04-21T08:45:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.318+0000 7f777ba5c640 1 -- 172.21.15.86:0/3191706086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7774074b70 msgr2=0x7f77741a4da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.318+0000 7f777ba5c640 1 --2- 172.21.15.86:0/3191706086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7774074b70 0x7f77741a4da0 secure :-1 s=READY pgs=1434 cs=0 l=1 rev1=1 crypto rx=0x7f776400c920 tx=0x7f776400cde0 comp rx=0 tx=0).stop 2024-04-21T08:45:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.318+0000 7f777ba5c640 1 -- 172.21.15.86:0/3191706086 shutdown_connections 2024-04-21T08:45:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.318+0000 7f777ba5c640 1 --2- 172.21.15.86:0/3191706086 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f77600a3dc0 0x7f77600a6280 unknown :-1 s=CLOSED pgs=229 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.318+0000 7f777ba5c640 1 --2- 172.21.15.86:0/3191706086 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7774113010 0x7f777419ea10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.318+0000 7f777ba5c640 1 --2- 172.21.15.86:0/3191706086 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7774074b70 0x7f77741a4da0 unknown :-1 s=CLOSED pgs=1434 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.318+0000 7f777ba5c640 1 --2- 172.21.15.86:0/3191706086 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7774074250 0x7f77741a4860 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.318+0000 7f777ba5c640 1 -- 172.21.15.86:0/3191706086 >> 172.21.15.86:0/3191706086 conn(0x7f77740793e0 msgr2=0x7f77740ffd80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.318+0000 7f777ba5c640 1 -- 172.21.15.86:0/3191706086 shutdown_connections 2024-04-21T08:45:41.320 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:41.318+0000 7f777ba5c640 1 -- 172.21.15.86:0/3191706086 wait complete. 2024-04-21T08:45:42.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:41 smithi086 ceph-mon[27939]: pgmap v212: 129 pgs: 129 active+clean; 3.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:45:42.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:41 smithi088 ceph-mon[31259]: pgmap v212: 129 pgs: 129 active+clean; 3.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:45:42.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:41 smithi179 ceph-mon[31829]: pgmap v212: 129 pgs: 129 active+clean; 3.1 MiB data, 404 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:45:43.358 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.358 INFO:tasks.workunit.client.0.smithi086.stdout:251/600: open fd = 3 2024-04-21T08:45:43.358 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.358 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.359 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.359 INFO:tasks.workunit.client.0.smithi086.stdout:252/600: open fd = 3 2024-04-21T08:45:43.359 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.359 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.359 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.359 INFO:tasks.workunit.client.0.smithi086.stdout:253/600: open fd = 3 2024-04-21T08:45:43.359 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.359 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.360 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.360 INFO:tasks.workunit.client.0.smithi086.stdout:254/600: open fd = 3 2024-04-21T08:45:43.360 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.360 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.360 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.360 INFO:tasks.workunit.client.0.smithi086.stdout:255/600: open fd = 3 2024-04-21T08:45:43.360 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.360 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.361 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.361 INFO:tasks.workunit.client.0.smithi086.stdout:256/600: open fd = 3 2024-04-21T08:45:43.361 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.361 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.361 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.361 INFO:tasks.workunit.client.0.smithi086.stdout:257/600: open fd = 3 2024-04-21T08:45:43.361 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.361 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.361 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.362 INFO:tasks.workunit.client.0.smithi086.stdout:258/600: open fd = 3 2024-04-21T08:45:43.362 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.362 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.362 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.362 INFO:tasks.workunit.client.0.smithi086.stdout:259/600: open fd = 3 2024-04-21T08:45:43.362 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.362 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.362 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.363 INFO:tasks.workunit.client.0.smithi086.stdout:260/600: open fd = 3 2024-04-21T08:45:43.363 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.363 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.363 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.363 INFO:tasks.workunit.client.0.smithi086.stdout:261/600: open fd = 3 2024-04-21T08:45:43.363 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.363 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.363 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.363 INFO:tasks.workunit.client.0.smithi086.stdout:262/600: open fd = 3 2024-04-21T08:45:43.364 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.364 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.364 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.364 INFO:tasks.workunit.client.0.smithi086.stdout:263/600: open fd = 3 2024-04-21T08:45:43.364 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.364 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.364 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.364 INFO:tasks.workunit.client.0.smithi086.stdout:264/600: open fd = 3 2024-04-21T08:45:43.365 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.365 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.365 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.365 INFO:tasks.workunit.client.0.smithi086.stdout:265/600: open fd = 3 2024-04-21T08:45:43.365 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.365 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.365 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.366 INFO:tasks.workunit.client.0.smithi086.stdout:266/600: open fd = 3 2024-04-21T08:45:43.366 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.366 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.366 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.366 INFO:tasks.workunit.client.0.smithi086.stdout:267/600: open fd = 3 2024-04-21T08:45:43.366 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.367 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.367 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.367 INFO:tasks.workunit.client.0.smithi086.stdout:268/600: open fd = 3 2024-04-21T08:45:43.367 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.367 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.367 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.367 INFO:tasks.workunit.client.0.smithi086.stdout:269/600: open fd = 3 2024-04-21T08:45:43.367 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.368 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.368 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.368 INFO:tasks.workunit.client.0.smithi086.stdout:270/600: open fd = 3 2024-04-21T08:45:43.368 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.368 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.368 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.368 INFO:tasks.workunit.client.0.smithi086.stdout:271/600: open fd = 3 2024-04-21T08:45:43.368 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.369 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.369 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.369 INFO:tasks.workunit.client.0.smithi086.stdout:272/600: open fd = 3 2024-04-21T08:45:43.369 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.369 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.369 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.369 INFO:tasks.workunit.client.0.smithi086.stdout:273/600: open fd = 3 2024-04-21T08:45:43.369 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.369 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.370 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.370 INFO:tasks.workunit.client.0.smithi086.stdout:274/600: open fd = 3 2024-04-21T08:45:43.370 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.370 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.370 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.370 INFO:tasks.workunit.client.0.smithi086.stdout:275/600: open fd = 3 2024-04-21T08:45:43.370 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.371 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.371 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.371 INFO:tasks.workunit.client.0.smithi086.stdout:276/600: open fd = 3 2024-04-21T08:45:43.371 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.371 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.371 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.371 INFO:tasks.workunit.client.0.smithi086.stdout:277/600: open fd = 3 2024-04-21T08:45:43.371 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.372 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.372 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.372 INFO:tasks.workunit.client.0.smithi086.stdout:278/600: open fd = 3 2024-04-21T08:45:43.372 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.372 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.372 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.372 INFO:tasks.workunit.client.0.smithi086.stdout:279/600: open fd = 3 2024-04-21T08:45:43.372 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.373 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.373 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.373 INFO:tasks.workunit.client.0.smithi086.stdout:280/600: open fd = 3 2024-04-21T08:45:43.373 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.373 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.373 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.373 INFO:tasks.workunit.client.0.smithi086.stdout:281/600: open fd = 3 2024-04-21T08:45:43.373 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.374 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.374 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.374 INFO:tasks.workunit.client.0.smithi086.stdout:282/600: open fd = 3 2024-04-21T08:45:43.374 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.374 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.374 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.374 INFO:tasks.workunit.client.0.smithi086.stdout:283/600: open fd = 3 2024-04-21T08:45:43.374 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.375 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.375 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.375 INFO:tasks.workunit.client.0.smithi086.stdout:284/600: open fd = 3 2024-04-21T08:45:43.375 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.375 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.375 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.375 INFO:tasks.workunit.client.0.smithi086.stdout:285/600: open fd = 3 2024-04-21T08:45:43.375 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.376 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.376 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.376 INFO:tasks.workunit.client.0.smithi086.stdout:286/600: open fd = 3 2024-04-21T08:45:43.376 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.376 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.376 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.376 INFO:tasks.workunit.client.0.smithi086.stdout:287/600: open fd = 3 2024-04-21T08:45:43.377 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.377 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.377 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.377 INFO:tasks.workunit.client.0.smithi086.stdout:288/600: open fd = 3 2024-04-21T08:45:43.377 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.377 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.377 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.377 INFO:tasks.workunit.client.0.smithi086.stdout:289/600: open fd = 3 2024-04-21T08:45:43.378 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.378 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.378 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.378 INFO:tasks.workunit.client.0.smithi086.stdout:290/600: open fd = 3 2024-04-21T08:45:43.378 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.378 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.378 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.379 INFO:tasks.workunit.client.0.smithi086.stdout:291/600: open fd = 3 2024-04-21T08:45:43.379 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.379 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.379 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.379 INFO:tasks.workunit.client.0.smithi086.stdout:292/600: open fd = 3 2024-04-21T08:45:43.379 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.379 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.379 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.380 INFO:tasks.workunit.client.0.smithi086.stdout:293/600: open fd = 3 2024-04-21T08:45:43.380 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.380 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.380 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.380 INFO:tasks.workunit.client.0.smithi086.stdout:294/600: open fd = 3 2024-04-21T08:45:43.380 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.380 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.380 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.381 INFO:tasks.workunit.client.0.smithi086.stdout:295/600: open fd = 3 2024-04-21T08:45:43.381 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.381 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.381 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.381 INFO:tasks.workunit.client.0.smithi086.stdout:296/600: open fd = 3 2024-04-21T08:45:43.381 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.381 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.381 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.382 INFO:tasks.workunit.client.0.smithi086.stdout:297/600: open fd = 3 2024-04-21T08:45:43.382 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.382 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.382 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.382 INFO:tasks.workunit.client.0.smithi086.stdout:298/600: open fd = 3 2024-04-21T08:45:43.382 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.382 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.382 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.383 INFO:tasks.workunit.client.0.smithi086.stdout:299/600: open fd = 3 2024-04-21T08:45:43.383 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.383 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.383 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.383 INFO:tasks.workunit.client.0.smithi086.stdout:300/600: open fd = 3 2024-04-21T08:45:43.383 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.383 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.383 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.384 INFO:tasks.workunit.client.0.smithi086.stdout:301/600: open fd = 3 2024-04-21T08:45:43.384 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.384 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.384 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.384 INFO:tasks.workunit.client.0.smithi086.stdout:302/600: open fd = 3 2024-04-21T08:45:43.384 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.384 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.385 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.385 INFO:tasks.workunit.client.0.smithi086.stdout:303/600: open fd = 3 2024-04-21T08:45:43.385 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.385 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.385 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.385 INFO:tasks.workunit.client.0.smithi086.stdout:304/600: open fd = 3 2024-04-21T08:45:43.385 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.386 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.386 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.386 INFO:tasks.workunit.client.0.smithi086.stdout:305/600: open fd = 3 2024-04-21T08:45:43.386 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.386 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.386 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.386 INFO:tasks.workunit.client.0.smithi086.stdout:306/600: open fd = 3 2024-04-21T08:45:43.386 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.387 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.387 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.387 INFO:tasks.workunit.client.0.smithi086.stdout:307/600: open fd = 3 2024-04-21T08:45:43.387 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.387 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.387 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.387 INFO:tasks.workunit.client.0.smithi086.stdout:308/600: open fd = 3 2024-04-21T08:45:43.387 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.387 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.387 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.387 INFO:tasks.workunit.client.0.smithi086.stdout:309/600: open fd = 3 2024-04-21T08:45:43.387 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.388 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.388 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.388 INFO:tasks.workunit.client.0.smithi086.stdout:310/600: open fd = 3 2024-04-21T08:45:43.388 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.388 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.388 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.388 INFO:tasks.workunit.client.0.smithi086.stdout:311/600: open fd = 3 2024-04-21T08:45:43.388 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.388 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.388 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:45:43.388 INFO:tasks.workunit.client.0.smithi086.stdout:312/600: open fd = 3 2024-04-21T08:45:43.388 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:43.388 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:45:44.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:43 smithi086 ceph-mon[27939]: pgmap v213: 129 pgs: 129 active+clean; 3.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 66 KiB/s wr, 24 op/s 2024-04-21T08:45:44.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:43 smithi088 ceph-mon[31259]: pgmap v213: 129 pgs: 129 active+clean; 3.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 66 KiB/s wr, 24 op/s 2024-04-21T08:45:44.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:43 smithi179 ceph-mon[31829]: pgmap v213: 129 pgs: 129 active+clean; 3.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 66 KiB/s wr, 24 op/s 2024-04-21T08:45:46.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:45 smithi086 ceph-mon[27939]: pgmap v214: 129 pgs: 129 active+clean; 3.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:45:46.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:45 smithi088 ceph-mon[31259]: pgmap v214: 129 pgs: 129 active+clean; 3.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:45:46.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:45 smithi179 ceph-mon[31829]: pgmap v214: 129 pgs: 129 active+clean; 3.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:45:48.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:47 smithi086 ceph-mon[27939]: pgmap v215: 129 pgs: 129 active+clean; 3.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:45:48.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:47 smithi088 ceph-mon[31259]: pgmap v215: 129 pgs: 129 active+clean; 3.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:45:48.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:47 smithi179 ceph-mon[31829]: pgmap v215: 129 pgs: 129 active+clean; 3.3 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:45:50.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:49 smithi086 ceph-mon[27939]: pgmap v216: 129 pgs: 129 active+clean; 3.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-04-21T08:45:50.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:49 smithi088 ceph-mon[31259]: pgmap v216: 129 pgs: 129 active+clean; 3.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-04-21T08:45:50.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:49 smithi179 ceph-mon[31829]: pgmap v216: 129 pgs: 129 active+clean; 3.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-04-21T08:45:52.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:51 smithi088 ceph-mon[31259]: pgmap v217: 129 pgs: 129 active+clean; 3.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:45:52.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:51 smithi179 ceph-mon[31829]: pgmap v217: 129 pgs: 129 active+clean; 3.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:45:52.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:51 smithi086 ceph-mon[27939]: pgmap v217: 129 pgs: 129 active+clean; 3.5 MiB data, 405 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:45:53.022 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:45:53.022 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:45:53.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.135+0000 7f97600a5640 1 -- 172.21.15.86:0/843969279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9758102170 msgr2=0x7f9758102550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.135+0000 7f97600a5640 1 --2- 172.21.15.86:0/843969279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9758102170 0x7f9758102550 secure :-1 s=READY pgs=1439 cs=0 l=1 rev1=1 crypto rx=0x7f9754009900 tx=0x7f975402bc50 comp rx=0 tx=0).stop 2024-04-21T08:45:53.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.135+0000 7f97600a5640 1 -- 172.21.15.86:0/843969279 shutdown_connections 2024-04-21T08:45:53.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.135+0000 7f97600a5640 1 --2- 172.21.15.86:0/843969279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9758107240 0x7f975810ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.135+0000 7f97600a5640 1 --2- 172.21.15.86:0/843969279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9758102b20 0x7f9758106b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.135+0000 7f97600a5640 1 --2- 172.21.15.86:0/843969279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9758102170 0x7f9758102550 unknown :-1 s=CLOSED pgs=1439 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.135+0000 7f97600a5640 1 -- 172.21.15.86:0/843969279 >> 172.21.15.86:0/843969279 conn(0x7f97580793a0 msgr2=0x7f97580797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:53.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.135+0000 7f97600a5640 1 -- 172.21.15.86:0/843969279 shutdown_connections 2024-04-21T08:45:53.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.135+0000 7f97600a5640 1 -- 172.21.15.86:0/843969279 wait complete. 2024-04-21T08:45:53.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.136+0000 7f97600a5640 1 Processor -- start 2024-04-21T08:45:53.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.136+0000 7f97600a5640 1 -- start start 2024-04-21T08:45:53.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.136+0000 7f97600a5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9758102170 0x7f9758074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.136+0000 7f97600a5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9758107240 0x7f9758074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.136+0000 7f97600a5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9758074cd0 0x7f97581a6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.136+0000 7f97600a5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9758112220 con 0x7f9758074cd0 2024-04-21T08:45:53.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.136+0000 7f97600a5640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f97581120a0 con 0x7f9758107240 2024-04-21T08:45:53.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.136+0000 7f97600a5640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f97581123a0 con 0x7f9758102170 2024-04-21T08:45:53.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.136+0000 7f975e642640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9758074cd0 0x7f97581a6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.136+0000 7f975e642640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9758074cd0 0x7f97581a6950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38294/0 (socket says 172.21.15.86:38294) 2024-04-21T08:45:53.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.136+0000 7f975e642640 1 -- 172.21.15.86:0/3682266438 learned_addr learned my addr 172.21.15.86:0/3682266438 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:53.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.136+0000 7f975d640640 1 --2- 172.21.15.86:0/3682266438 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9758107240 0x7f9758074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.136+0000 7f975de41640 1 --2- 172.21.15.86:0/3682266438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9758102170 0x7f9758074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.136+0000 7f975e642640 1 -- 172.21.15.86:0/3682266438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9758102170 msgr2=0x7f9758074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.136+0000 7f975e642640 1 --2- 172.21.15.86:0/3682266438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9758102170 0x7f9758074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.136+0000 7f975e642640 1 -- 172.21.15.86:0/3682266438 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9758107240 msgr2=0x7f9758074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.136+0000 7f975e642640 1 --2- 172.21.15.86:0/3682266438 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9758107240 0x7f9758074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.136+0000 7f975e642640 1 -- 172.21.15.86:0/3682266438 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f97581a7050 con 0x7f9758074cd0 2024-04-21T08:45:53.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.137+0000 7f975e642640 1 --2- 172.21.15.86:0/3682266438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9758074cd0 0x7f97581a6950 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f975000b990 tx=0x7f975000be50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:53.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.137+0000 7f974affd640 1 -- 172.21.15.86:0/3682266438 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9750019070 con 0x7f9758074cd0 2024-04-21T08:45:53.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.137+0000 7f974affd640 1 -- 172.21.15.86:0/3682266438 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f975000cba0 con 0x7f9758074cd0 2024-04-21T08:45:53.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.137+0000 7f97600a5640 1 -- 172.21.15.86:0/3682266438 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f97581a7340 con 0x7f9758074cd0 2024-04-21T08:45:53.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.137+0000 7f974affd640 1 -- 172.21.15.86:0/3682266438 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f97500075d0 con 0x7f9758074cd0 2024-04-21T08:45:53.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.137+0000 7f97600a5640 1 -- 172.21.15.86:0/3682266438 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f97581a78e0 con 0x7f9758074cd0 2024-04-21T08:45:53.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.139+0000 7f974affd640 1 -- 172.21.15.86:0/3682266438 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f975001c070 con 0x7f9758074cd0 2024-04-21T08:45:53.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.139+0000 7f9762d75740 1 Processor -- start 2024-04-21T08:45:53.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.140+0000 7f9762d75740 1 -- start start 2024-04-21T08:45:53.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.140+0000 7f974affd640 1 --2- 172.21.15.86:0/3682266438 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f97440a3dd0 0x7f97440a6290 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.140+0000 7f9762d75740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560b4b111580 0x560b4b111960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.140+0000 7f9762d75740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b4b111ea0 0x560b4b11a540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.140+0000 7f9762d75740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b4b11aa80 0x560b4b11ce70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.140+0000 7f9762d75740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560b4b0761e0 con 0x560b4b11aa80 2024-04-21T08:45:53.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.140+0000 7f9762d75740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x560b4b013390 con 0x560b4b111580 2024-04-21T08:45:53.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.140+0000 7f9762d75740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x560b4b0721a0 con 0x560b4b111ea0 2024-04-21T08:45:53.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.140+0000 7f975e642640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b4b11aa80 0x560b4b11ce70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.140+0000 7f975d640640 1 --2- 172.21.15.86:0/3682266438 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f97440a3dd0 0x7f97440a6290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.140+0000 7f975e642640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b4b11aa80 0x560b4b11ce70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38310/0 (socket says 172.21.15.86:38310) 2024-04-21T08:45:53.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.140+0000 7f975e642640 1 -- 172.21.15.86:0/414238512 learned_addr learned my addr 172.21.15.86:0/414238512 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.140+0000 7f974affd640 1 -- 172.21.15.86:0/3682266438 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f97500cff50 con 0x7f9758074cd0 2024-04-21T08:45:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.140+0000 7f975de41640 1 --2- 172.21.15.86:0/414238512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560b4b111580 0x560b4b111960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.140+0000 7f975d640640 1 --2- 172.21.15.86:0/414238512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b4b111ea0 0x560b4b11a540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.140+0000 7f975e642640 1 -- 172.21.15.86:0/414238512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b4b111ea0 msgr2=0x560b4b11a540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.140+0000 7f975e642640 1 --2- 172.21.15.86:0/414238512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b4b111ea0 0x560b4b11a540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.140+0000 7f975e642640 1 -- 172.21.15.86:0/414238512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560b4b111580 msgr2=0x560b4b111960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.140+0000 7f975e642640 1 --2- 172.21.15.86:0/414238512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560b4b111580 0x560b4b111960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.140+0000 7f975e642640 1 -- 172.21.15.86:0/414238512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b4ae527d0 con 0x560b4b11aa80 2024-04-21T08:45:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.140+0000 7f975de41640 1 --2- 172.21.15.86:0/414238512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560b4b111580 0x560b4b111960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.140+0000 7f975d640640 1 --2- 172.21.15.86:0/3682266438 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f97440a3dd0 0x7f97440a6290 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9758075ce0 tx=0x7f974c013880 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.141+0000 7f975e642640 1 --2- 172.21.15.86:0/414238512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b4b11aa80 0x560b4b11ce70 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f975000b960 tx=0x7f975001bf40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b33ae95cf49a41a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.141+0000 7f9743fff640 1 -- 172.21.15.86:0/414238512 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975009b820 con 0x560b4b11aa80 2024-04-21T08:45:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.141+0000 7f9743fff640 1 -- 172.21.15.86:0/414238512 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9750018170 con 0x560b4b11aa80 2024-04-21T08:45:53.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.141+0000 7f9743fff640 1 -- 172.21.15.86:0/414238512 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975009b640 con 0x560b4b11aa80 2024-04-21T08:45:53.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.141+0000 7f9762d75740 1 -- 172.21.15.86:0/414238512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b4b11aa80 msgr2=0x560b4b11ce70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.141+0000 7f9762d75740 1 --2- 172.21.15.86:0/414238512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b4b11aa80 0x560b4b11ce70 secure :-1 s=READY pgs=1441 cs=0 l=1 rev1=1 crypto rx=0x7f975000b960 tx=0x7f975001bf40 comp rx=0 tx=0).stop 2024-04-21T08:45:53.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.141+0000 7f9762d75740 1 -- 172.21.15.86:0/414238512 shutdown_connections 2024-04-21T08:45:53.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.141+0000 7f9762d75740 1 --2- 172.21.15.86:0/414238512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b4b11aa80 0x560b4b11ce70 unknown :-1 s=CLOSED pgs=1441 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.141+0000 7f9762d75740 1 --2- 172.21.15.86:0/414238512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b4b111ea0 0x560b4b11a540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.141+0000 7f9762d75740 1 --2- 172.21.15.86:0/414238512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560b4b111580 0x560b4b111960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.141+0000 7f9762d75740 1 -- 172.21.15.86:0/414238512 >> 172.21.15.86:0/414238512 conn(0x560b4b10bcb0 msgr2=0x560b4b10c090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:53.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.142+0000 7f9762d75740 1 -- 172.21.15.86:0/414238512 shutdown_connections 2024-04-21T08:45:53.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.142+0000 7f9762d75740 1 -- 172.21.15.86:0/414238512 wait complete. 2024-04-21T08:45:53.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.142+0000 7f9762d75740 1 -- start start 2024-04-21T08:45:53.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.142+0000 7f9762d75740 1 Processor -- start 2024-04-21T08:45:53.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.142+0000 7f9762d75740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b4b111580 0x560b4b1179e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.142+0000 7f9762d75740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560b4b111ea0 0x560b4b117f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.142+0000 7f9762d75740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b4b11aa80 0x560b4b118460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.142+0000 7f9762d75740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560b4b11d870 con 0x560b4b111580 2024-04-21T08:45:53.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.142+0000 7f9762d75740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x560b4b02f2f0 con 0x560b4b111ea0 2024-04-21T08:45:53.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.142+0000 7f9762d75740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x560b4b11d9f0 con 0x560b4b11aa80 2024-04-21T08:45:53.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.142+0000 7f9762d75740 10 client.0 _ll_register_callbacks cb 0x560b4b03c580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:53.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.143+0000 7f975de41640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b4b111580 0x560b4b1179e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.143+0000 7f975de41640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b4b111580 0x560b4b1179e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38324/0 (socket says 172.21.15.86:38324) 2024-04-21T08:45:53.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.143+0000 7f975de41640 1 -- 172.21.15.86:0/2176888275 learned_addr learned my addr 172.21.15.86:0/2176888275 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:53.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.143+0000 7f975d640640 1 --2- 172.21.15.86:0/2176888275 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560b4b111ea0 0x560b4b117f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.143+0000 7f975de41640 1 -- 172.21.15.86:0/2176888275 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b4b11aa80 msgr2=0x560b4b118460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:53.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.143+0000 7f975de41640 1 --2- 172.21.15.86:0/2176888275 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b4b11aa80 0x560b4b118460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.143+0000 7f975de41640 1 -- 172.21.15.86:0/2176888275 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560b4b111ea0 msgr2=0x560b4b117f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:53.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.143+0000 7f975de41640 1 --2- 172.21.15.86:0/2176888275 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560b4b111ea0 0x560b4b117f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.143+0000 7f975de41640 1 -- 172.21.15.86:0/2176888275 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b4ad883f0 con 0x560b4b111580 2024-04-21T08:45:53.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.143+0000 7f975de41640 1 --2- 172.21.15.86:0/2176888275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b4b111580 0x560b4b1179e0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f9754035000 tx=0x7f975402ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e56f30a2380af40 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:53.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.143+0000 7f97417fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:53.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.143+0000 7f97417fa640 1 -- 172.21.15.86:0/2176888275 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9754005450 con 0x560b4b111580 2024-04-21T08:45:53.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.143+0000 7f97417fa640 1 -- 172.21.15.86:0/2176888275 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f97540046c0 con 0x560b4b111580 2024-04-21T08:45:53.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.143+0000 7f9762d75740 1 -- 172.21.15.86:0/2176888275 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x560b4b083c00 con 0x560b4b111580 2024-04-21T08:45:53.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.143+0000 7f97417fa640 1 -- 172.21.15.86:0/2176888275 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f975402ccb0 con 0x560b4b111580 2024-04-21T08:45:53.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.143+0000 7f9762d75740 10 client.18504 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:53.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.143+0000 7f9762d75740 1 -- 172.21.15.86:0/2176888275 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560b4ad480f0 con 0x560b4b111580 2024-04-21T08:45:53.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.143+0000 7f97417fa640 1 -- 172.21.15.86:0/2176888275 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x560b4b083c00 con 0x560b4b111580 2024-04-21T08:45:53.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.144+0000 7f97417fa640 1 -- 172.21.15.86:0/2176888275 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9754003960 con 0x560b4b111580 2024-04-21T08:45:53.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.144+0000 7f9762d75740 10 client.18504 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:53.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.144+0000 7f9762d75740 10 client.18504 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:53.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.144+0000 7f9762d75740 20 client.18504 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:53.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.144+0000 7f9762d75740 1 --2- 172.21.15.86:0/2176888275 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x560b4b12a9f0 0x560b4b12ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.144+0000 7f9762d75740 4 client.18504 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:53.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.144+0000 7f9762d75740 1 -- 172.21.15.86:0/2176888275 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560b4b07da70 con 0x560b4b12a9f0 2024-04-21T08:45:53.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.144+0000 7f975d640640 1 --2- 172.21.15.86:0/2176888275 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x560b4b12a9f0 0x560b4b12ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.144+0000 7f975d640640 1 --2- 172.21.15.86:0/2176888275 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x560b4b12a9f0 0x560b4b12ce50 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3845cc5c7bc12542 server_cookie=6ab702747f2a08e2 in_seq=0 out_seq=0 2024-04-21T08:45:53.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.144+0000 7f97417fa640 10 client.18504 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:53.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.145+0000 7f97417fa640 1 -- 172.21.15.86:0/2176888275 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560b4b07da70 con 0x560b4b12a9f0 2024-04-21T08:45:53.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.145+0000 7f97417fa640 10 client.18504 handle_command_reply: tid=0 2024-04-21T08:45:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.145+0000 7f9762d75740 1 client.18504 shutdown 2024-04-21T08:45:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.145+0000 7f9762d75740 1 -- 172.21.15.86:0/2176888275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b4b111580 msgr2=0x560b4b1179e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.145+0000 7f9762d75740 1 --2- 172.21.15.86:0/2176888275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b4b111580 0x560b4b1179e0 secure :-1 s=READY pgs=1442 cs=0 l=1 rev1=1 crypto rx=0x7f9754035000 tx=0x7f975402ca90 comp rx=0 tx=0).stop 2024-04-21T08:45:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.145+0000 7f9762d75740 1 -- 172.21.15.86:0/2176888275 shutdown_connections 2024-04-21T08:45:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.145+0000 7f9762d75740 1 --2- 172.21.15.86:0/2176888275 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x560b4b12a9f0 0x560b4b12ce50 crc :-1 s=READY pgs=217 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.145+0000 7f9762d75740 1 --2- 172.21.15.86:0/2176888275 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b4b11aa80 0x560b4b118460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.145+0000 7f9762d75740 1 --2- 172.21.15.86:0/2176888275 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560b4b111ea0 0x560b4b117f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.145+0000 7f9762d75740 1 --2- 172.21.15.86:0/2176888275 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b4b111580 0x560b4b1179e0 unknown :-1 s=CLOSED pgs=1442 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.145+0000 7f97417fa640 0 client.18504 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.145+0000 7f9762d75740 1 -- 172.21.15.86:0/2176888275 >> 172.21.15.86:0/2176888275 conn(0x560b4b10bcb0 msgr2=0x560b4b10da00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.146+0000 7f9762d75740 1 -- 172.21.15.86:0/2176888275 shutdown_connections 2024-04-21T08:45:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.146+0000 7f9762d75740 1 -- 172.21.15.86:0/2176888275 wait complete. 2024-04-21T08:45:53.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.146+0000 7f9762d75740 20 client.18504 trim_cache size 0 max 16384 2024-04-21T08:45:53.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.163+0000 7f9762d75740 1 Processor -- start 2024-04-21T08:45:53.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.163+0000 7f9762d75740 1 -- start start 2024-04-21T08:45:53.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.163+0000 7f9762d75740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b4b111580 0x560b4b11a110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.163+0000 7f9762d75740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560b4b111ea0 0x560b4b126730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.163+0000 7f9762d75740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b4b11aa80 0x560b4b116920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.163+0000 7f9762d75740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f97581b1610 con 0x560b4b111580 2024-04-21T08:45:53.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.163+0000 7f9762d75740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f97581b3150 con 0x560b4b111ea0 2024-04-21T08:45:53.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.163+0000 7f9762d75740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f97581b1ce0 con 0x560b4b11aa80 2024-04-21T08:45:53.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.163+0000 7f975de41640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b4b111580 0x560b4b11a110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.163+0000 7f975de41640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b4b111580 0x560b4b11a110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38332/0 (socket says 172.21.15.86:38332) 2024-04-21T08:45:53.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.163+0000 7f975de41640 1 -- 172.21.15.86:0/2415334650 learned_addr learned my addr 172.21.15.86:0/2415334650 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:53.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.163+0000 7f975d640640 1 --2- 172.21.15.86:0/2415334650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560b4b111ea0 0x560b4b126730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.163+0000 7f975de41640 1 -- 172.21.15.86:0/2415334650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b4b11aa80 msgr2=0x560b4b116920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:53.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.163+0000 7f975de41640 1 --2- 172.21.15.86:0/2415334650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b4b11aa80 0x560b4b116920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.163+0000 7f975de41640 1 -- 172.21.15.86:0/2415334650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560b4b111ea0 msgr2=0x560b4b126730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:53.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.163+0000 7f975de41640 1 --2- 172.21.15.86:0/2415334650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560b4b111ea0 0x560b4b126730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.163+0000 7f975de41640 1 -- 172.21.15.86:0/2415334650 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560b4b10bae0 con 0x560b4b111580 2024-04-21T08:45:53.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.163+0000 7f975d640640 1 --2- 172.21.15.86:0/2415334650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560b4b111ea0 0x560b4b126730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:53.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.163+0000 7f975de41640 1 --2- 172.21.15.86:0/2415334650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b4b111580 0x560b4b11a110 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f9754009900 tx=0x7f975402bc50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a479af33cfdc17f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:53.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.164+0000 7f97437fe640 1 -- 172.21.15.86:0/2415334650 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9754034440 con 0x560b4b111580 2024-04-21T08:45:53.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.164+0000 7f97437fe640 1 -- 172.21.15.86:0/2415334650 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9754003aa0 con 0x560b4b111580 2024-04-21T08:45:53.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.164+0000 7f97437fe640 1 -- 172.21.15.86:0/2415334650 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9754034ba0 con 0x560b4b111580 2024-04-21T08:45:53.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.164+0000 7f9762d75740 1 -- 172.21.15.86:0/2415334650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b4b111580 msgr2=0x560b4b11a110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.164+0000 7f9762d75740 1 --2- 172.21.15.86:0/2415334650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b4b111580 0x560b4b11a110 secure :-1 s=READY pgs=1443 cs=0 l=1 rev1=1 crypto rx=0x7f9754009900 tx=0x7f975402bc50 comp rx=0 tx=0).stop 2024-04-21T08:45:53.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.164+0000 7f9762d75740 1 -- 172.21.15.86:0/2415334650 shutdown_connections 2024-04-21T08:45:53.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.164+0000 7f9762d75740 1 --2- 172.21.15.86:0/2415334650 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b4b11aa80 0x560b4b116920 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.164+0000 7f9762d75740 1 --2- 172.21.15.86:0/2415334650 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560b4b111ea0 0x560b4b126730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.164+0000 7f9762d75740 1 --2- 172.21.15.86:0/2415334650 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b4b111580 0x560b4b11a110 unknown :-1 s=CLOSED pgs=1443 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.164+0000 7f9762d75740 1 -- 172.21.15.86:0/2415334650 >> 172.21.15.86:0/2415334650 conn(0x560b4b10bcb0 msgr2=0x560b4b10ad70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:53.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.164+0000 7f9762d75740 1 -- 172.21.15.86:0/2415334650 shutdown_connections 2024-04-21T08:45:53.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.164+0000 7f9762d75740 1 -- 172.21.15.86:0/2415334650 wait complete. 2024-04-21T08:45:53.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.164+0000 7f9762d75740 1 -- start start 2024-04-21T08:45:53.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.165+0000 7f9762d75740 1 Processor -- start 2024-04-21T08:45:53.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.165+0000 7f9762d75740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b4b111580 0x560b4b122950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.165+0000 7f9762d75740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560b4b111ea0 0x560b4b122e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.165+0000 7f9762d75740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b4b11aa80 0x560b4b11ee20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.165+0000 7f9762d75740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f97581b2a80 con 0x560b4b11aa80 2024-04-21T08:45:53.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.165+0000 7f9762d75740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f97581b23b0 con 0x560b4b111ea0 2024-04-21T08:45:53.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.165+0000 7f9762d75740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f97581af610 con 0x560b4b111580 2024-04-21T08:45:53.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.165+0000 7f9762d75740 10 client.0 _ll_register_callbacks cb 0x7f9754004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:53.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.165+0000 7f975e642640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b4b11aa80 0x560b4b11ee20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.165+0000 7f975e642640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b4b11aa80 0x560b4b11ee20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38344/0 (socket says 172.21.15.86:38344) 2024-04-21T08:45:53.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.165+0000 7f975e642640 1 -- 172.21.15.86:0/1524252172 learned_addr learned my addr 172.21.15.86:0/1524252172 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:53.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.165+0000 7f975de41640 1 --2- 172.21.15.86:0/1524252172 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b4b111580 0x560b4b122950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.165+0000 7f975d640640 1 --2- 172.21.15.86:0/1524252172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560b4b111ea0 0x560b4b122e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.165+0000 7f975e642640 1 -- 172.21.15.86:0/1524252172 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b4b111580 msgr2=0x560b4b122950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:53.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.165+0000 7f975e642640 1 --2- 172.21.15.86:0/1524252172 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b4b111580 0x560b4b122950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.165+0000 7f975e642640 1 -- 172.21.15.86:0/1524252172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560b4b111ea0 msgr2=0x560b4b122e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:53.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.165+0000 7f975e642640 1 --2- 172.21.15.86:0/1524252172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560b4b111ea0 0x560b4b122e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.165+0000 7f975e642640 1 -- 172.21.15.86:0/1524252172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560b4b11f5b0 con 0x560b4b11aa80 2024-04-21T08:45:53.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.165+0000 7f975d640640 1 --2- 172.21.15.86:0/1524252172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560b4b111ea0 0x560b4b122e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:53.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.165+0000 7f975e642640 1 --2- 172.21.15.86:0/1524252172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b4b11aa80 0x560b4b11ee20 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f97500cd8d0 tx=0x7f9750014bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6a3bcd034f34c96 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:53.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.165+0000 7f97417fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:53.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.166+0000 7f97417fa640 1 -- 172.21.15.86:0/1524252172 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f975009ffe0 con 0x560b4b11aa80 2024-04-21T08:45:53.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.166+0000 7f97417fa640 1 -- 172.21.15.86:0/1524252172 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f97500bc080 con 0x560b4b11aa80 2024-04-21T08:45:53.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.166+0000 7f9762d75740 1 -- 172.21.15.86:0/1524252172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9754003aa0 con 0x560b4b11aa80 2024-04-21T08:45:53.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.166+0000 7f97417fa640 1 -- 172.21.15.86:0/1524252172 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f97500bc280 con 0x560b4b11aa80 2024-04-21T08:45:53.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.166+0000 7f9762d75740 10 client.18510 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:53.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.166+0000 7f9762d75740 1 -- 172.21.15.86:0/1524252172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560b4b191960 con 0x560b4b11aa80 2024-04-21T08:45:53.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.166+0000 7f97417fa640 1 -- 172.21.15.86:0/1524252172 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9754003aa0 con 0x560b4b11aa80 2024-04-21T08:45:53.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.166+0000 7f97417fa640 1 -- 172.21.15.86:0/1524252172 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f97500a4410 con 0x560b4b11aa80 2024-04-21T08:45:53.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.166+0000 7f9762d75740 10 client.18510 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:53.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.166+0000 7f9762d75740 10 client.18510 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:53.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.166+0000 7f9762d75740 20 client.18510 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:53.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.166+0000 7f9762d75740 1 --2- 172.21.15.86:0/1524252172 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x560b4b191c40 0x560b4b194030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.166+0000 7f9762d75740 4 client.18510 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:45:53.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.166+0000 7f9762d75740 1 -- 172.21.15.86:0/1524252172 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x560b4aff9590 con 0x560b4b191c40 2024-04-21T08:45:53.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.166+0000 7f975de41640 1 --2- 172.21.15.86:0/1524252172 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x560b4b191c40 0x560b4b194030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.167+0000 7f97417fa640 10 client.18510 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:53.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.167+0000 7f975de41640 1 --2- 172.21.15.86:0/1524252172 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x560b4b191c40 0x560b4b194030 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3b15b2e5aed22864 server_cookie=a9204cbd6a9de1f3 in_seq=0 out_seq=0 2024-04-21T08:45:53.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.167+0000 7f97417fa640 1 -- 172.21.15.86:0/1524252172 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x560b4aff9590 con 0x560b4b191c40 2024-04-21T08:45:53.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.167+0000 7f97417fa640 10 client.18510 handle_command_reply: tid=0 2024-04-21T08:45:53.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.167+0000 7f9762d75740 1 client.18510 shutdown 2024-04-21T08:45:53.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.167+0000 7f9762d75740 1 -- 172.21.15.86:0/1524252172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b4b11aa80 msgr2=0x560b4b11ee20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.167+0000 7f9762d75740 1 --2- 172.21.15.86:0/1524252172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b4b11aa80 0x560b4b11ee20 secure :-1 s=READY pgs=1444 cs=0 l=1 rev1=1 crypto rx=0x7f97500cd8d0 tx=0x7f9750014bb0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.167+0000 7f9762d75740 1 -- 172.21.15.86:0/1524252172 shutdown_connections 2024-04-21T08:45:53.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.167+0000 7f9762d75740 1 --2- 172.21.15.86:0/1524252172 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x560b4b191c40 0x560b4b194030 crc :-1 s=READY pgs=218 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.167+0000 7f9762d75740 1 --2- 172.21.15.86:0/1524252172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560b4b11aa80 0x560b4b11ee20 unknown :-1 s=CLOSED pgs=1444 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.167+0000 7f97417fa640 0 client.18510 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:53.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.167+0000 7f9762d75740 1 --2- 172.21.15.86:0/1524252172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560b4b111ea0 0x560b4b122e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.167+0000 7f9762d75740 1 --2- 172.21.15.86:0/1524252172 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560b4b111580 0x560b4b122950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.167+0000 7f9762d75740 1 -- 172.21.15.86:0/1524252172 >> 172.21.15.86:0/1524252172 conn(0x560b4b10bcb0 msgr2=0x560b4b117900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:53.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.167+0000 7f9762d75740 1 -- 172.21.15.86:0/1524252172 shutdown_connections 2024-04-21T08:45:53.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.167+0000 7f9762d75740 1 -- 172.21.15.86:0/1524252172 wait complete. 2024-04-21T08:45:53.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.167+0000 7f9762d75740 20 client.18510 trim_cache size 0 max 16384 2024-04-21T08:45:53.175 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:45:53.175 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-21T08:45:53.175 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-21T08:45:53.175 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:45:53.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.168+0000 7f97600a5640 1 -- 172.21.15.86:0/3682266438 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f97440a3dd0 msgr2=0x7f97440a6290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.168+0000 7f97600a5640 1 --2- 172.21.15.86:0/3682266438 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f97440a3dd0 0x7f97440a6290 secure :-1 s=READY pgs=230 cs=0 l=1 rev1=1 crypto rx=0x7f9758075ce0 tx=0x7f974c013880 comp rx=0 tx=0).stop 2024-04-21T08:45:53.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.168+0000 7f97600a5640 1 -- 172.21.15.86:0/3682266438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9758074cd0 msgr2=0x7f97581a6950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.168+0000 7f97600a5640 1 --2- 172.21.15.86:0/3682266438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9758074cd0 0x7f97581a6950 secure :-1 s=READY pgs=1440 cs=0 l=1 rev1=1 crypto rx=0x7f975000b990 tx=0x7f975000be50 comp rx=0 tx=0).stop 2024-04-21T08:45:53.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.168+0000 7f97600a5640 1 -- 172.21.15.86:0/3682266438 shutdown_connections 2024-04-21T08:45:53.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.168+0000 7f97600a5640 1 --2- 172.21.15.86:0/3682266438 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f97440a3dd0 0x7f97440a6290 unknown :-1 s=CLOSED pgs=230 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.168+0000 7f97600a5640 1 --2- 172.21.15.86:0/3682266438 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9758074cd0 0x7f97581a6950 unknown :-1 s=CLOSED pgs=1440 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.176 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.168+0000 7f97600a5640 1 --2- 172.21.15.86:0/3682266438 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9758107240 0x7f9758074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.168+0000 7f97600a5640 1 --2- 172.21.15.86:0/3682266438 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9758102170 0x7f9758074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.168+0000 7f97600a5640 1 -- 172.21.15.86:0/3682266438 >> 172.21.15.86:0/3682266438 conn(0x7f97580793a0 msgr2=0x7f97580fffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:53.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.168+0000 7f97600a5640 1 -- 172.21.15.86:0/3682266438 shutdown_connections 2024-04-21T08:45:53.177 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.168+0000 7f97600a5640 1 -- 172.21.15.86:0/3682266438 wait complete. 2024-04-21T08:45:53.178 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:45:53.178 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:45:53.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.277+0000 7efd94ac0640 1 -- 172.21.15.86:0/2844421769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd90101b40 msgr2=0x7efd9010dcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.277+0000 7efd94ac0640 1 --2- 172.21.15.86:0/2844421769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd90101b40 0x7efd9010dcb0 secure :-1 s=READY pgs=1445 cs=0 l=1 rev1=1 crypto rx=0x7efd80009900 tx=0x7efd8002bc30 comp rx=0 tx=0).stop 2024-04-21T08:45:53.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.277+0000 7efd94ac0640 1 -- 172.21.15.86:0/2844421769 shutdown_connections 2024-04-21T08:45:53.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.277+0000 7efd94ac0640 1 --2- 172.21.15.86:0/2844421769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd90101b40 0x7efd9010dcb0 unknown :-1 s=CLOSED pgs=1445 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.277+0000 7efd94ac0640 1 --2- 172.21.15.86:0/2844421769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd901011a0 0x7efd90101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.277+0000 7efd94ac0640 1 --2- 172.21.15.86:0/2844421769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd90105ea0 0x7efd90106280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.277+0000 7efd94ac0640 1 -- 172.21.15.86:0/2844421769 >> 172.21.15.86:0/2844421769 conn(0x7efd90077e30 msgr2=0x7efd90078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:53.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.277+0000 7efd94ac0640 1 -- 172.21.15.86:0/2844421769 shutdown_connections 2024-04-21T08:45:53.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.277+0000 7efd94ac0640 1 -- 172.21.15.86:0/2844421769 wait complete. 2024-04-21T08:45:53.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.277+0000 7efd94ac0640 1 Processor -- start 2024-04-21T08:45:53.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.277+0000 7efd94ac0640 1 -- start start 2024-04-21T08:45:53.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.278+0000 7efd94ac0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd901011a0 0x7efd90199510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.278+0000 7efd94ac0640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd90101b40 0x7efd90199a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.278+0000 7efd94ac0640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd90105ea0 0x7efd901a4590 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.278+0000 7efd94ac0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efd901110f0 con 0x7efd901011a0 2024-04-21T08:45:53.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.278+0000 7efd94ac0640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efd90110f70 con 0x7efd90101b40 2024-04-21T08:45:53.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.278+0000 7efd94ac0640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7efd90111270 con 0x7efd90105ea0 2024-04-21T08:45:53.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.278+0000 7efd8e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd901011a0 0x7efd90199510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.278+0000 7efd8e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd901011a0 0x7efd90199510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38370/0 (socket says 172.21.15.86:38370) 2024-04-21T08:45:53.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.278+0000 7efd8e59c640 1 -- 172.21.15.86:0/96047811 learned_addr learned my addr 172.21.15.86:0/96047811 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:53.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.278+0000 7efd8dd9b640 1 --2- 172.21.15.86:0/96047811 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd90101b40 0x7efd90199a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.278+0000 7efd8ed9d640 1 --2- 172.21.15.86:0/96047811 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd90105ea0 0x7efd901a4590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.278+0000 7efd8e59c640 1 -- 172.21.15.86:0/96047811 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd90105ea0 msgr2=0x7efd901a4590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.278+0000 7efd8e59c640 1 --2- 172.21.15.86:0/96047811 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd90105ea0 0x7efd901a4590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.278+0000 7efd8e59c640 1 -- 172.21.15.86:0/96047811 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd90101b40 msgr2=0x7efd90199a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.278+0000 7efd8e59c640 1 --2- 172.21.15.86:0/96047811 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd90101b40 0x7efd90199a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.278+0000 7efd8e59c640 1 -- 172.21.15.86:0/96047811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efd901a4ad0 con 0x7efd901011a0 2024-04-21T08:45:53.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.278+0000 7efd8ed9d640 1 --2- 172.21.15.86:0/96047811 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd90105ea0 0x7efd901a4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:53.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.278+0000 7efd8e59c640 1 --2- 172.21.15.86:0/96047811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd901011a0 0x7efd90199510 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7efd8800d770 tx=0x7efd8800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:53.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.278+0000 7efd7f7fe640 1 -- 172.21.15.86:0/96047811 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd88017070 con 0x7efd901011a0 2024-04-21T08:45:53.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.278+0000 7efd94ac0640 1 -- 172.21.15.86:0/96047811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efd901a4ca0 con 0x7efd901011a0 2024-04-21T08:45:53.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.278+0000 7efd7f7fe640 1 -- 172.21.15.86:0/96047811 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efd880054d0 con 0x7efd901011a0 2024-04-21T08:45:53.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.278+0000 7efd7f7fe640 1 -- 172.21.15.86:0/96047811 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efd88004c30 con 0x7efd901011a0 2024-04-21T08:45:53.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.278+0000 7efd94ac0640 1 -- 172.21.15.86:0/96047811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efd901a5190 con 0x7efd901011a0 2024-04-21T08:45:53.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.279+0000 7efd94ac0640 1 -- 172.21.15.86:0/96047811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efd90102590 con 0x7efd901011a0 2024-04-21T08:45:53.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.281+0000 7efd7f7fe640 1 -- 172.21.15.86:0/96047811 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efd88004e10 con 0x7efd901011a0 2024-04-21T08:45:53.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.281+0000 7efd7f7fe640 1 --2- 172.21.15.86:0/96047811 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efd780a3d20 0x7efd780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.281+0000 7efd8dd9b640 1 --2- 172.21.15.86:0/96047811 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efd780a3d20 0x7efd780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.281+0000 7efd7f7fe640 1 -- 172.21.15.86:0/96047811 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7efd880d05d0 con 0x7efd901011a0 2024-04-21T08:45:53.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.282+0000 7efd8dd9b640 1 --2- 172.21.15.86:0/96047811 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efd780a3d20 0x7efd780a61e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efd9006b1a0 tx=0x7efd84009760 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:53.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.286+0000 7efd7f7fe640 1 -- 172.21.15.86:0/96047811 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efd8809c8a0 con 0x7efd901011a0 2024-04-21T08:45:53.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.504+0000 7efd94ac0640 1 -- 172.21.15.86:0/96047811 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efd90069230 con 0x7efd901011a0 2024-04-21T08:45:53.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.505+0000 7efd7f7fe640 1 -- 172.21.15.86:0/96047811 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7efd8809c1a0 con 0x7efd901011a0 2024-04-21T08:45:53.505 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:45:53.505 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:38:45.202126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24398,24394,24455,24472]},"id":1}]} 2024-04-21T08:45:53.505 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-21T08:45:53.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.507+0000 7efd94ac0640 1 -- 172.21.15.86:0/96047811 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efd780a3d20 msgr2=0x7efd780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.507+0000 7efd94ac0640 1 --2- 172.21.15.86:0/96047811 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efd780a3d20 0x7efd780a61e0 secure :-1 s=READY pgs=231 cs=0 l=1 rev1=1 crypto rx=0x7efd9006b1a0 tx=0x7efd84009760 comp rx=0 tx=0).stop 2024-04-21T08:45:53.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.507+0000 7efd94ac0640 1 -- 172.21.15.86:0/96047811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd901011a0 msgr2=0x7efd90199510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.507+0000 7efd94ac0640 1 --2- 172.21.15.86:0/96047811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd901011a0 0x7efd90199510 secure :-1 s=READY pgs=1446 cs=0 l=1 rev1=1 crypto rx=0x7efd8800d770 tx=0x7efd8800dc30 comp rx=0 tx=0).stop 2024-04-21T08:45:53.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.507+0000 7efd94ac0640 1 -- 172.21.15.86:0/96047811 shutdown_connections 2024-04-21T08:45:53.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.507+0000 7efd94ac0640 1 --2- 172.21.15.86:0/96047811 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efd780a3d20 0x7efd780a61e0 unknown :-1 s=CLOSED pgs=231 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.507+0000 7efd94ac0640 1 --2- 172.21.15.86:0/96047811 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efd90105ea0 0x7efd901a4590 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.507+0000 7efd94ac0640 1 --2- 172.21.15.86:0/96047811 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efd90101b40 0x7efd90199a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.507+0000 7efd94ac0640 1 --2- 172.21.15.86:0/96047811 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efd901011a0 0x7efd90199510 unknown :-1 s=CLOSED pgs=1446 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.507+0000 7efd94ac0640 1 -- 172.21.15.86:0/96047811 >> 172.21.15.86:0/96047811 conn(0x7efd90077e30 msgr2=0x7efd900fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:53.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.507+0000 7efd94ac0640 1 -- 172.21.15.86:0/96047811 shutdown_connections 2024-04-21T08:45:53.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.507+0000 7efd94ac0640 1 -- 172.21.15.86:0/96047811 wait complete. 2024-04-21T08:45:53.517 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:45:53.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.618+0000 7f5c891e2640 1 -- 172.21.15.86:0/4079788543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c84101b40 msgr2=0x7f5c8410dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.618+0000 7f5c891e2640 1 --2- 172.21.15.86:0/4079788543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c84101b40 0x7f5c8410dc80 secure :-1 s=READY pgs=1447 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c009cd0 tx=0x7f5c7c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.618+0000 7f5c891e2640 1 -- 172.21.15.86:0/4079788543 shutdown_connections 2024-04-21T08:45:53.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.618+0000 7f5c891e2640 1 --2- 172.21.15.86:0/4079788543 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c84101b40 0x7f5c8410dc80 unknown :-1 s=CLOSED pgs=1447 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.618+0000 7f5c891e2640 1 --2- 172.21.15.86:0/4079788543 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c841011a0 0x7f5c84101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.618+0000 7f5c891e2640 1 --2- 172.21.15.86:0/4079788543 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c84107030 0x7f5c84107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.618+0000 7f5c891e2640 1 -- 172.21.15.86:0/4079788543 >> 172.21.15.86:0/4079788543 conn(0x7f5c84076c50 msgr2=0x7f5c84077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:53.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.618+0000 7f5c891e2640 1 -- 172.21.15.86:0/4079788543 shutdown_connections 2024-04-21T08:45:53.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.618+0000 7f5c891e2640 1 -- 172.21.15.86:0/4079788543 wait complete. 2024-04-21T08:45:53.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.618+0000 7f5c891e2640 1 Processor -- start 2024-04-21T08:45:53.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.619+0000 7f5c891e2640 1 -- start start 2024-04-21T08:45:53.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.619+0000 7f5c891e2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c841011a0 0x7f5c84197580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.619+0000 7f5c891e2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c84101b40 0x7f5c84197ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.619+0000 7f5c891e2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c84107030 0x7f5c8419be40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.619+0000 7f5c891e2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5c841110e0 con 0x7f5c84107030 2024-04-21T08:45:53.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.619+0000 7f5c891e2640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5c84110f60 con 0x7f5c84101b40 2024-04-21T08:45:53.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.619+0000 7f5c891e2640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5c84111260 con 0x7f5c841011a0 2024-04-21T08:45:53.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.619+0000 7f5c8359e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c84107030 0x7f5c8419be40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.619+0000 7f5c8359e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c84107030 0x7f5c8419be40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38394/0 (socket says 172.21.15.86:38394) 2024-04-21T08:45:53.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.619+0000 7f5c8359e640 1 -- 172.21.15.86:0/3170281999 learned_addr learned my addr 172.21.15.86:0/3170281999 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:53.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.619+0000 7f5c7bfff640 1 --2- 172.21.15.86:0/3170281999 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c84101b40 0x7f5c84197ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.619+0000 7f5c82d9d640 1 --2- 172.21.15.86:0/3170281999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c841011a0 0x7f5c84197580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.619+0000 7f5c8359e640 1 -- 172.21.15.86:0/3170281999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c841011a0 msgr2=0x7f5c84197580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.619+0000 7f5c8359e640 1 --2- 172.21.15.86:0/3170281999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c841011a0 0x7f5c84197580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.619+0000 7f5c8359e640 1 -- 172.21.15.86:0/3170281999 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c84101b40 msgr2=0x7f5c84197ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.619+0000 7f5c8359e640 1 --2- 172.21.15.86:0/3170281999 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c84101b40 0x7f5c84197ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.619+0000 7f5c8359e640 1 -- 172.21.15.86:0/3170281999 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5c8419c570 con 0x7f5c84107030 2024-04-21T08:45:53.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.619+0000 7f5c82d9d640 1 --2- 172.21.15.86:0/3170281999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c841011a0 0x7f5c84197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:53.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.619+0000 7f5c8359e640 1 --2- 172.21.15.86:0/3170281999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c84107030 0x7f5c8419be40 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c037040 tx=0x7f5c7c034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:53.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.619+0000 7f5c80d99640 1 -- 172.21.15.86:0/3170281999 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c034ca0 con 0x7f5c84107030 2024-04-21T08:45:53.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.619+0000 7f5c80d99640 1 -- 172.21.15.86:0/3170281999 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c7c032970 con 0x7f5c84107030 2024-04-21T08:45:53.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.619+0000 7f5c80d99640 1 -- 172.21.15.86:0/3170281999 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7c0042a0 con 0x7f5c84107030 2024-04-21T08:45:53.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.619+0000 7f5c891e2640 1 -- 172.21.15.86:0/3170281999 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5c8419c800 con 0x7f5c84107030 2024-04-21T08:45:53.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.620+0000 7f5c891e2640 1 -- 172.21.15.86:0/3170281999 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5c8406a550 con 0x7f5c84107030 2024-04-21T08:45:53.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.622+0000 7f5c80d99640 1 -- 172.21.15.86:0/3170281999 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5c7c0324c0 con 0x7f5c84107030 2024-04-21T08:45:53.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.622+0000 7f5c8bdcb740 1 Processor -- start 2024-04-21T08:45:53.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.622+0000 7f5c8bdcb740 1 -- start start 2024-04-21T08:45:53.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.622+0000 7f5c8bdcb740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f839e74d0 0x564f839e78b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.622+0000 7f5c8bdcb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f839e7e80 0x564f839f0430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.622+0000 7f5c8bdcb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f839f0a00 0x564f839f2e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.622+0000 7f5c8bdcb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564f8394c1f0 con 0x564f839e7e80 2024-04-21T08:45:53.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.622+0000 7f5c8bdcb740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564f838e94b0 con 0x564f839e74d0 2024-04-21T08:45:53.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.622+0000 7f5c8bdcb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564f839481b0 con 0x564f839f0a00 2024-04-21T08:45:53.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.622+0000 7f5c82d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f839e7e80 0x564f839f0430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.623+0000 7f5c80d99640 1 --2- 172.21.15.86:0/3170281999 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5c700a3d20 0x7f5c700a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.623+0000 7f5c82d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f839e7e80 0x564f839f0430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38400/0 (socket says 172.21.15.86:38400) 2024-04-21T08:45:53.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.623+0000 7f5c82d9d640 1 -- 172.21.15.86:0/1992771073 learned_addr learned my addr 172.21.15.86:0/1992771073 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:53.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.623+0000 7f5c7bfff640 1 --2- 172.21.15.86:0/1992771073 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f839e74d0 0x564f839e78b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.623+0000 7f5c8359e640 1 --2- 172.21.15.86:0/3170281999 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5c700a3d20 0x7f5c700a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.623+0000 7f5c7bfff640 1 --2- 172.21.15.86:0/1992771073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f839f0a00 0x564f839f2e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.623+0000 7f5c82d9d640 1 -- 172.21.15.86:0/1992771073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f839f0a00 msgr2=0x564f839f2e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:53.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.623+0000 7f5c82d9d640 1 --2- 172.21.15.86:0/1992771073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f839f0a00 0x564f839f2e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.623+0000 7f5c80d99640 1 -- 172.21.15.86:0/3170281999 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f5c7c0f48d0 con 0x7f5c84107030 2024-04-21T08:45:53.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.623+0000 7f5c82d9d640 1 -- 172.21.15.86:0/1992771073 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f839e74d0 msgr2=0x564f839e78b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:53.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.623+0000 7f5c82d9d640 1 --2- 172.21.15.86:0/1992771073 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f839e74d0 0x564f839e78b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.623+0000 7f5c82d9d640 1 -- 172.21.15.86:0/1992771073 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f838db1b0 con 0x564f839e7e80 2024-04-21T08:45:53.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.623+0000 7f5c82d9d640 1 --2- 172.21.15.86:0/1992771073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f839e7e80 0x564f839f0430 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f5c7400bb70 tx=0x7f5c7400d710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c85252de0836ec24 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:53.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.623+0000 7f5c8359e640 1 --2- 172.21.15.86:0/3170281999 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5c700a3d20 0x7f5c700a61e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0f1e10 tx=0x7f5c7c08ef60 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:53.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.623+0000 7f5c797fa640 1 -- 172.21.15.86:0/1992771073 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c74017620 con 0x564f839e7e80 2024-04-21T08:45:53.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.623+0000 7f5c797fa640 1 -- 172.21.15.86:0/1992771073 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c74017c70 con 0x564f839e7e80 2024-04-21T08:45:53.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.623+0000 7f5c797fa640 1 -- 172.21.15.86:0/1992771073 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c7400fb40 con 0x564f839e7e80 2024-04-21T08:45:53.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.624+0000 7f5c8bdcb740 1 -- 172.21.15.86:0/1992771073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f839e7e80 msgr2=0x564f839f0430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.624+0000 7f5c8bdcb740 1 --2- 172.21.15.86:0/1992771073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f839e7e80 0x564f839f0430 secure :-1 s=READY pgs=1449 cs=0 l=1 rev1=1 crypto rx=0x7f5c7400bb70 tx=0x7f5c7400d710 comp rx=0 tx=0).stop 2024-04-21T08:45:53.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.624+0000 7f5c8bdcb740 1 -- 172.21.15.86:0/1992771073 shutdown_connections 2024-04-21T08:45:53.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.624+0000 7f5c8bdcb740 1 --2- 172.21.15.86:0/1992771073 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f839f0a00 0x564f839f2e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.624+0000 7f5c8bdcb740 1 --2- 172.21.15.86:0/1992771073 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f839e7e80 0x564f839f0430 unknown :-1 s=CLOSED pgs=1449 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.624+0000 7f5c8bdcb740 1 --2- 172.21.15.86:0/1992771073 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f839e74d0 0x564f839e78b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.624+0000 7f5c8bdcb740 1 -- 172.21.15.86:0/1992771073 >> 172.21.15.86:0/1992771073 conn(0x564f839e1ba0 msgr2=0x564f839e1f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:53.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.624+0000 7f5c8bdcb740 1 -- 172.21.15.86:0/1992771073 shutdown_connections 2024-04-21T08:45:53.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.624+0000 7f5c8bdcb740 1 -- 172.21.15.86:0/1992771073 wait complete. 2024-04-21T08:45:53.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.624+0000 7f5c8bdcb740 1 -- start start 2024-04-21T08:45:53.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.624+0000 7f5c8bdcb740 1 Processor -- start 2024-04-21T08:45:53.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.625+0000 7f5c8bdcb740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f839e74d0 0x564f839f0320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.625+0000 7f5c8bdcb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f839e7e80 0x564f839ed6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.625+0000 7f5c8bdcb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f839f0a00 0x564f839edc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.625+0000 7f5c8bdcb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564f839f3970 con 0x564f839e7e80 2024-04-21T08:45:53.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.625+0000 7f5c8bdcb740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564f83905500 con 0x564f839e74d0 2024-04-21T08:45:53.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.625+0000 7f5c8bdcb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564f839f3af0 con 0x564f839f0a00 2024-04-21T08:45:53.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.625+0000 7f5c8bdcb740 10 client.0 _ll_register_callbacks cb 0x564f838d3b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:53.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.625+0000 7f5c82d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f839e7e80 0x564f839ed6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.625+0000 7f5c82d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f839e7e80 0x564f839ed6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38414/0 (socket says 172.21.15.86:38414) 2024-04-21T08:45:53.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.625+0000 7f5c82d9d640 1 -- 172.21.15.86:0/4073949287 learned_addr learned my addr 172.21.15.86:0/4073949287 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:53.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.625+0000 7f5c7bfff640 1 --2- 172.21.15.86:0/4073949287 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f839e74d0 0x564f839f0320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.625+0000 7f5c7bfff640 1 --2- 172.21.15.86:0/4073949287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f839f0a00 0x564f839edc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.625+0000 7f5c82d9d640 1 -- 172.21.15.86:0/4073949287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f839f0a00 msgr2=0x564f839edc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:53.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.625+0000 7f5c82d9d640 1 --2- 172.21.15.86:0/4073949287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f839f0a00 0x564f839edc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.625+0000 7f5c82d9d640 1 -- 172.21.15.86:0/4073949287 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f839e74d0 msgr2=0x564f839f0320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:53.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.625+0000 7f5c82d9d640 1 --2- 172.21.15.86:0/4073949287 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f839e74d0 0x564f839f0320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.625+0000 7f5c82d9d640 1 -- 172.21.15.86:0/4073949287 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f8365e3f0 con 0x564f839e7e80 2024-04-21T08:45:53.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.625+0000 7f5c7bfff640 1 --2- 172.21.15.86:0/4073949287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f839f0a00 0x564f839edc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:53.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.625+0000 7f5c82d9d640 1 --2- 172.21.15.86:0/4073949287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f839e7e80 0x564f839ed6b0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f5c74019040 tx=0x7f5c74005bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc14cbb4a243a180 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:53.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.625+0000 7f5c6a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:53.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.625+0000 7f5c6a7fc640 1 -- 172.21.15.86:0/4073949287 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c74026070 con 0x564f839e7e80 2024-04-21T08:45:53.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.625+0000 7f5c6a7fc640 1 -- 172.21.15.86:0/4073949287 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f5c74017830 con 0x564f839e7e80 2024-04-21T08:45:53.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.625+0000 7f5c8bdcb740 1 -- 172.21.15.86:0/4073949287 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c74017c70 con 0x564f839e7e80 2024-04-21T08:45:53.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.626+0000 7f5c6a7fc640 1 -- 172.21.15.86:0/4073949287 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5c74032050 con 0x564f839e7e80 2024-04-21T08:45:53.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.626+0000 7f5c8bdcb740 10 client.18528 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:53.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.626+0000 7f5c8bdcb740 1 -- 172.21.15.86:0/4073949287 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564f837287d0 con 0x564f839e7e80 2024-04-21T08:45:53.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.626+0000 7f5c6a7fc640 1 -- 172.21.15.86:0/4073949287 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c74017c70 con 0x564f839e7e80 2024-04-21T08:45:53.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.626+0000 7f5c6a7fc640 1 -- 172.21.15.86:0/4073949287 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5c740228f0 con 0x564f839e7e80 2024-04-21T08:45:53.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.626+0000 7f5c8bdcb740 10 client.18528 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:53.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.626+0000 7f5c8bdcb740 10 client.18528 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:45:53.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.626+0000 7f5c8bdcb740 20 client.18528 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:53.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.626+0000 7f5c8bdcb740 1 --2- 172.21.15.86:0/4073949287 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x564f83a00b00 0x564f83a02ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.626+0000 7f5c8bdcb740 4 client.18528 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:53.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.626+0000 7f5c8bdcb740 1 -- 172.21.15.86:0/4073949287 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564f838eb930 con 0x564f83a00b00 2024-04-21T08:45:53.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.626+0000 7f5c7bfff640 1 --2- 172.21.15.86:0/4073949287 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x564f83a00b00 0x564f83a02ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.627+0000 7f5c6a7fc640 10 client.18528 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:45:53.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.627+0000 7f5c7bfff640 1 --2- 172.21.15.86:0/4073949287 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x564f83a00b00 0x564f83a02ef0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=68cbf4dfa944dad8 server_cookie=72721772f6fc5de1 in_seq=0 out_seq=0 2024-04-21T08:45:53.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.627+0000 7f5c6a7fc640 1 -- 172.21.15.86:0/4073949287 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564f838eb930 con 0x564f83a00b00 2024-04-21T08:45:53.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.627+0000 7f5c6a7fc640 10 client.18528 handle_command_reply: tid=0 2024-04-21T08:45:53.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.627+0000 7f5c8bdcb740 1 client.18528 shutdown 2024-04-21T08:45:53.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.627+0000 7f5c8bdcb740 1 -- 172.21.15.86:0/4073949287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f839e7e80 msgr2=0x564f839ed6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.627+0000 7f5c8bdcb740 1 --2- 172.21.15.86:0/4073949287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f839e7e80 0x564f839ed6b0 secure :-1 s=READY pgs=1450 cs=0 l=1 rev1=1 crypto rx=0x7f5c74019040 tx=0x7f5c74005bf0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.627+0000 7f5c8bdcb740 1 -- 172.21.15.86:0/4073949287 shutdown_connections 2024-04-21T08:45:53.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.627+0000 7f5c8bdcb740 1 --2- 172.21.15.86:0/4073949287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f839f0a00 0x564f839edc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.627+0000 7f5c8bdcb740 1 --2- 172.21.15.86:0/4073949287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f839e7e80 0x564f839ed6b0 unknown :-1 s=CLOSED pgs=1450 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.627+0000 7f5c8bdcb740 1 --2- 172.21.15.86:0/4073949287 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x564f83a00b00 0x564f83a02ef0 crc :-1 s=READY pgs=58 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.627+0000 7f5c8bdcb740 1 --2- 172.21.15.86:0/4073949287 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f839e74d0 0x564f839f0320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.627+0000 7f5c6a7fc640 0 client.18528 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:45:53.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.627+0000 7f5c8bdcb740 1 -- 172.21.15.86:0/4073949287 >> 172.21.15.86:0/4073949287 conn(0x564f839e1ba0 msgr2=0x564f839e38f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:53.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.628+0000 7f5c8bdcb740 1 -- 172.21.15.86:0/4073949287 shutdown_connections 2024-04-21T08:45:53.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.628+0000 7f5c8bdcb740 1 -- 172.21.15.86:0/4073949287 wait complete. 2024-04-21T08:45:53.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.628+0000 7f5c8bdcb740 20 client.18528 trim_cache size 0 max 16384 2024-04-21T08:45:53.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.641+0000 7f5c8bdcb740 1 Processor -- start 2024-04-21T08:45:53.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.641+0000 7f5c8bdcb740 1 -- start start 2024-04-21T08:45:53.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.641+0000 7f5c8bdcb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f839e74d0 0x564f839fc830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.641+0000 7f5c8bdcb740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f839e7e80 0x564f839ec810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.641+0000 7f5c8bdcb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f839f0a00 0x564f839e3700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.641+0000 7f5c8bdcb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5c841ad110 con 0x564f839e74d0 2024-04-21T08:45:53.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.641+0000 7f5c8bdcb740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5c841aec50 con 0x564f839e7e80 2024-04-21T08:45:53.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.641+0000 7f5c8bdcb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5c841ad7e0 con 0x564f839f0a00 2024-04-21T08:45:53.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.641+0000 7f5c7bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f839e74d0 0x564f839fc830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.641+0000 7f5c7bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f839e74d0 0x564f839fc830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38416/0 (socket says 172.21.15.86:38416) 2024-04-21T08:45:53.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.641+0000 7f5c7bfff640 1 -- 172.21.15.86:0/3212669754 learned_addr learned my addr 172.21.15.86:0/3212669754 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:53.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.641+0000 7f5c82d9d640 1 --2- 172.21.15.86:0/3212669754 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f839e7e80 0x564f839ec810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.641+0000 7f5c7bfff640 1 -- 172.21.15.86:0/3212669754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f839f0a00 msgr2=0x564f839e3700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:53.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.641+0000 7f5c7bfff640 1 --2- 172.21.15.86:0/3212669754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f839f0a00 0x564f839e3700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.641+0000 7f5c7bfff640 1 -- 172.21.15.86:0/3212669754 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f839e7e80 msgr2=0x564f839ec810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:53.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.641+0000 7f5c7bfff640 1 --2- 172.21.15.86:0/3212669754 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f839e7e80 0x564f839ec810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.641+0000 7f5c7bfff640 1 -- 172.21.15.86:0/3212669754 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564f839fd3a0 con 0x564f839e74d0 2024-04-21T08:45:53.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.641+0000 7f5c82d9d640 1 --2- 172.21.15.86:0/3212669754 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f839e7e80 0x564f839ec810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:53.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.641+0000 7f5c7bfff640 1 --2- 172.21.15.86:0/3212669754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f839e74d0 0x564f839fc830 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c0104f0 tx=0x7f5c6c0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a1dfecb661032f4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:53.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.642+0000 7f5c78ff9640 1 -- 172.21.15.86:0/3212669754 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c020530 con 0x564f839e74d0 2024-04-21T08:45:53.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.642+0000 7f5c78ff9640 1 -- 172.21.15.86:0/3212669754 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c6c0054d0 con 0x564f839e74d0 2024-04-21T08:45:53.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.642+0000 7f5c78ff9640 1 -- 172.21.15.86:0/3212669754 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c004c70 con 0x564f839e74d0 2024-04-21T08:45:53.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.642+0000 7f5c8bdcb740 1 -- 172.21.15.86:0/3212669754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f839e74d0 msgr2=0x564f839fc830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.642+0000 7f5c8bdcb740 1 --2- 172.21.15.86:0/3212669754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f839e74d0 0x564f839fc830 secure :-1 s=READY pgs=1451 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c0104f0 tx=0x7f5c6c0046c0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.642+0000 7f5c8bdcb740 1 -- 172.21.15.86:0/3212669754 shutdown_connections 2024-04-21T08:45:53.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.642+0000 7f5c8bdcb740 1 --2- 172.21.15.86:0/3212669754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f839f0a00 0x564f839e3700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.642+0000 7f5c8bdcb740 1 --2- 172.21.15.86:0/3212669754 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f839e7e80 0x564f839ec810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.642+0000 7f5c8bdcb740 1 --2- 172.21.15.86:0/3212669754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f839e74d0 0x564f839fc830 unknown :-1 s=CLOSED pgs=1451 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.642+0000 7f5c8bdcb740 1 -- 172.21.15.86:0/3212669754 >> 172.21.15.86:0/3212669754 conn(0x564f839e1ba0 msgr2=0x564f839e2ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:53.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.642+0000 7f5c8bdcb740 1 -- 172.21.15.86:0/3212669754 shutdown_connections 2024-04-21T08:45:53.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.642+0000 7f5c8bdcb740 1 -- 172.21.15.86:0/3212669754 wait complete. 2024-04-21T08:45:53.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.643+0000 7f5c8bdcb740 1 -- start start 2024-04-21T08:45:53.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.643+0000 7f5c8bdcb740 1 Processor -- start 2024-04-21T08:45:53.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.643+0000 7f5c8bdcb740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f839e74d0 0x564f839f8d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.643+0000 7f5c8bdcb740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f839e7e80 0x564f839f9280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.643+0000 7f5c8bdcb740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f839f0a00 0x564f839e9a40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.643+0000 7f5c8bdcb740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5c841ae580 con 0x564f839f0a00 2024-04-21T08:45:53.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.643+0000 7f5c8bdcb740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5c841adeb0 con 0x564f839e7e80 2024-04-21T08:45:53.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.643+0000 7f5c8bdcb740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5c841ab110 con 0x564f839e74d0 2024-04-21T08:45:53.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.643+0000 7f5c8bdcb740 10 client.0 _ll_register_callbacks cb 0x7f5c7400fcd0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:53.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.643+0000 7f5c7bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f839f0a00 0x564f839e9a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.643+0000 7f5c7bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f839f0a00 0x564f839e9a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38422/0 (socket says 172.21.15.86:38422) 2024-04-21T08:45:53.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.643+0000 7f5c7bfff640 1 -- 172.21.15.86:0/522543558 learned_addr learned my addr 172.21.15.86:0/522543558 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:53.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.643+0000 7f5c82d9d640 1 --2- 172.21.15.86:0/522543558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f839e7e80 0x564f839f9280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.643+0000 7f5c7bfff640 1 --2- 172.21.15.86:0/522543558 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f839e74d0 0x564f839f8d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.643+0000 7f5c7bfff640 1 -- 172.21.15.86:0/522543558 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f839e74d0 msgr2=0x564f839f8d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:53.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.643+0000 7f5c7bfff640 1 --2- 172.21.15.86:0/522543558 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f839e74d0 0x564f839f8d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.643+0000 7f5c7bfff640 1 -- 172.21.15.86:0/522543558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f839e7e80 msgr2=0x564f839f9280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:53.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.643+0000 7f5c7bfff640 1 --2- 172.21.15.86:0/522543558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f839e7e80 0x564f839f9280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.643+0000 7f5c7bfff640 1 -- 172.21.15.86:0/522543558 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564f839ea180 con 0x564f839f0a00 2024-04-21T08:45:53.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.643+0000 7f5c7bfff640 1 --2- 172.21.15.86:0/522543558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f839f0a00 0x564f839e9a40 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c000c00 tx=0x7f5c6c005850 comp rx=0 tx=0).ready entity=mon.0 client_cookie=31042847cbd13264 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:53.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.643+0000 7f5c6a7fc640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:53.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.644+0000 7f5c6a7fc640 1 -- 172.21.15.86:0/522543558 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5c6c0208c0 con 0x564f839f0a00 2024-04-21T08:45:53.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.644+0000 7f5c6a7fc640 1 -- 172.21.15.86:0/522543558 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f5c6c019040 con 0x564f839f0a00 2024-04-21T08:45:53.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.644+0000 7f5c8bdcb740 1 -- 172.21.15.86:0/522543558 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5c6c0054d0 con 0x564f839f0a00 2024-04-21T08:45:53.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.644+0000 7f5c6a7fc640 1 -- 172.21.15.86:0/522543558 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5c6c039050 con 0x564f839f0a00 2024-04-21T08:45:53.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.644+0000 7f5c8bdcb740 10 client.18534 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:53.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.644+0000 7f5c8bdcb740 1 -- 172.21.15.86:0/522543558 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564f83a6ac60 con 0x564f839f0a00 2024-04-21T08:45:53.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.644+0000 7f5c6a7fc640 1 -- 172.21.15.86:0/522543558 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5c6c0054d0 con 0x564f839f0a00 2024-04-21T08:45:53.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.644+0000 7f5c6a7fc640 1 -- 172.21.15.86:0/522543558 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5c6c02ea40 con 0x564f839f0a00 2024-04-21T08:45:53.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.644+0000 7f5c8bdcb740 10 client.18534 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:53.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.644+0000 7f5c8bdcb740 10 client.18534 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:45:53.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.644+0000 7f5c8bdcb740 20 client.18534 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:53.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.644+0000 7f5c8bdcb740 1 --2- 172.21.15.86:0/522543558 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x564f83a6b350 0x564f83a6d770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.644+0000 7f5c8bdcb740 4 client.18534 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:45:53.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.644+0000 7f5c8bdcb740 1 -- 172.21.15.86:0/522543558 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564f83959c10 con 0x564f83a6b350 2024-04-21T08:45:53.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.644+0000 7f5c82d9d640 1 --2- 172.21.15.86:0/522543558 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x564f83a6b350 0x564f83a6d770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.645+0000 7f5c6a7fc640 10 client.18534 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:45:53.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.645+0000 7f5c82d9d640 1 --2- 172.21.15.86:0/522543558 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x564f83a6b350 0x564f83a6d770 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=e2c39a41a47dd62c server_cookie=95f11fd63b005cda in_seq=0 out_seq=0 2024-04-21T08:45:53.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.645+0000 7f5c6a7fc640 1 -- 172.21.15.86:0/522543558 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564f83959c10 con 0x564f83a6b350 2024-04-21T08:45:53.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.645+0000 7f5c6a7fc640 10 client.18534 handle_command_reply: tid=0 2024-04-21T08:45:53.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.645+0000 7f5c8bdcb740 1 client.18534 shutdown 2024-04-21T08:45:53.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.645+0000 7f5c8bdcb740 1 -- 172.21.15.86:0/522543558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f839f0a00 msgr2=0x564f839e9a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.645+0000 7f5c8bdcb740 1 --2- 172.21.15.86:0/522543558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f839f0a00 0x564f839e9a40 secure :-1 s=READY pgs=1452 cs=0 l=1 rev1=1 crypto rx=0x7f5c6c000c00 tx=0x7f5c6c005850 comp rx=0 tx=0).stop 2024-04-21T08:45:53.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.645+0000 7f5c8bdcb740 1 -- 172.21.15.86:0/522543558 shutdown_connections 2024-04-21T08:45:53.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.645+0000 7f5c8bdcb740 1 --2- 172.21.15.86:0/522543558 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564f839f0a00 0x564f839e9a40 unknown :-1 s=CLOSED pgs=1452 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.645+0000 7f5c8bdcb740 1 --2- 172.21.15.86:0/522543558 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x564f83a6b350 0x564f83a6d770 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.645+0000 7f5c8bdcb740 1 --2- 172.21.15.86:0/522543558 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564f839e7e80 0x564f839f9280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.645+0000 7f5c8bdcb740 1 --2- 172.21.15.86:0/522543558 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564f839e74d0 0x564f839f8d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.645+0000 7f5c6a7fc640 0 client.18534 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:45:53.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.645+0000 7f5c8bdcb740 1 -- 172.21.15.86:0/522543558 >> 172.21.15.86:0/522543558 conn(0x564f839e1ba0 msgr2=0x564f839e0c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:53.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.645+0000 7f5c8bdcb740 1 -- 172.21.15.86:0/522543558 shutdown_connections 2024-04-21T08:45:53.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.645+0000 7f5c8bdcb740 1 -- 172.21.15.86:0/522543558 wait complete. 2024-04-21T08:45:53.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.645+0000 7f5c8bdcb740 20 client.18534 trim_cache size 0 max 16384 2024-04-21T08:45:53.652 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:45:53.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.646+0000 7f5c891e2640 1 -- 172.21.15.86:0/3170281999 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5c700a3d20 msgr2=0x7f5c700a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.646+0000 7f5c891e2640 1 --2- 172.21.15.86:0/3170281999 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5c700a3d20 0x7f5c700a61e0 secure :-1 s=READY pgs=232 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c0f1e10 tx=0x7f5c7c08ef60 comp rx=0 tx=0).stop 2024-04-21T08:45:53.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.646+0000 7f5c891e2640 1 -- 172.21.15.86:0/3170281999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c84107030 msgr2=0x7f5c8419be40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.646+0000 7f5c891e2640 1 --2- 172.21.15.86:0/3170281999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c84107030 0x7f5c8419be40 secure :-1 s=READY pgs=1448 cs=0 l=1 rev1=1 crypto rx=0x7f5c7c037040 tx=0x7f5c7c034670 comp rx=0 tx=0).stop 2024-04-21T08:45:53.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.646+0000 7f5c891e2640 1 -- 172.21.15.86:0/3170281999 shutdown_connections 2024-04-21T08:45:53.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.646+0000 7f5c891e2640 1 --2- 172.21.15.86:0/3170281999 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5c700a3d20 0x7f5c700a61e0 unknown :-1 s=CLOSED pgs=232 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.646+0000 7f5c891e2640 1 --2- 172.21.15.86:0/3170281999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5c84107030 0x7f5c8419be40 unknown :-1 s=CLOSED pgs=1448 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.646+0000 7f5c891e2640 1 --2- 172.21.15.86:0/3170281999 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5c84101b40 0x7f5c84197ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.646+0000 7f5c891e2640 1 --2- 172.21.15.86:0/3170281999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5c841011a0 0x7f5c84197580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.646+0000 7f5c891e2640 1 -- 172.21.15.86:0/3170281999 >> 172.21.15.86:0/3170281999 conn(0x7f5c84076c50 msgr2=0x7f5c840ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:53.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.646+0000 7f5c891e2640 1 -- 172.21.15.86:0/3170281999 shutdown_connections 2024-04-21T08:45:53.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.646+0000 7f5c891e2640 1 -- 172.21.15.86:0/3170281999 wait complete. 2024-04-21T08:45:53.656 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:45:53.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.756+0000 7f4ce426d640 1 -- 172.21.15.86:0/3200266188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cdc06b210 msgr2=0x7f4cdc106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.756+0000 7f4ce426d640 1 --2- 172.21.15.86:0/3200266188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cdc06b210 0x7f4cdc106b90 secure :-1 s=READY pgs=1453 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc009900 tx=0x7f4ccc02bc30 comp rx=0 tx=0).stop 2024-04-21T08:45:53.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.756+0000 7f4ce426d640 1 -- 172.21.15.86:0/3200266188 shutdown_connections 2024-04-21T08:45:53.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.756+0000 7f4ce426d640 1 --2- 172.21.15.86:0/3200266188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cdc107270 0x7f4cdc10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.756+0000 7f4ce426d640 1 --2- 172.21.15.86:0/3200266188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cdc06b210 0x7f4cdc106b90 unknown :-1 s=CLOSED pgs=1453 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.756+0000 7f4ce426d640 1 --2- 172.21.15.86:0/3200266188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4cdc06a860 0x7f4cdc06ac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.756+0000 7f4ce426d640 1 -- 172.21.15.86:0/3200266188 >> 172.21.15.86:0/3200266188 conn(0x7f4cdc077eb0 msgr2=0x7f4cdc0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:53.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.756+0000 7f4ce426d640 1 -- 172.21.15.86:0/3200266188 shutdown_connections 2024-04-21T08:45:53.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.756+0000 7f4ce426d640 1 -- 172.21.15.86:0/3200266188 wait complete. 2024-04-21T08:45:53.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.756+0000 7f4ce426d640 1 Processor -- start 2024-04-21T08:45:53.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.757+0000 7f4ce426d640 1 -- start start 2024-04-21T08:45:53.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.757+0000 7f4ce426d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4cdc06a860 0x7f4cdc1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.757+0000 7f4ce426d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cdc107270 0x7f4cdc198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.757+0000 7f4ce426d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cdc19cf70 0x7f4cdc19d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.757+0000 7f4ce426d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc1121b0 con 0x7f4cdc19cf70 2024-04-21T08:45:53.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.757+0000 7f4ce426d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc112030 con 0x7f4cdc107270 2024-04-21T08:45:53.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.757+0000 7f4ce426d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc112330 con 0x7f4cdc06a860 2024-04-21T08:45:53.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.757+0000 7f4ce280a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cdc19cf70 0x7f4cdc19d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.757+0000 7f4ce280a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cdc19cf70 0x7f4cdc19d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38432/0 (socket says 172.21.15.86:38432) 2024-04-21T08:45:53.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.757+0000 7f4ce280a640 1 -- 172.21.15.86:0/1631080286 learned_addr learned my addr 172.21.15.86:0/1631080286 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:53.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.757+0000 7f4ce2009640 1 --2- 172.21.15.86:0/1631080286 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4cdc06a860 0x7f4cdc1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.757+0000 7f4ce1808640 1 --2- 172.21.15.86:0/1631080286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cdc107270 0x7f4cdc198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.757+0000 7f4ce280a640 1 -- 172.21.15.86:0/1631080286 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4cdc06a860 msgr2=0x7f4cdc1986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.757+0000 7f4ce280a640 1 --2- 172.21.15.86:0/1631080286 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4cdc06a860 0x7f4cdc1986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.757+0000 7f4ce280a640 1 -- 172.21.15.86:0/1631080286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cdc107270 msgr2=0x7f4cdc198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.757+0000 7f4ce280a640 1 --2- 172.21.15.86:0/1631080286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cdc107270 0x7f4cdc198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.757+0000 7f4ce280a640 1 -- 172.21.15.86:0/1631080286 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4cdc19da60 con 0x7f4cdc19cf70 2024-04-21T08:45:53.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.757+0000 7f4ce1808640 1 --2- 172.21.15.86:0/1631080286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cdc107270 0x7f4cdc198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:53.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.757+0000 7f4ce280a640 1 --2- 172.21.15.86:0/1631080286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cdc19cf70 0x7f4cdc19d420 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f4cd800cce0 tx=0x7f4cd800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:53.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.757+0000 7f4cd37fe640 1 -- 172.21.15.86:0/1631080286 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd800f450 con 0x7f4cdc19cf70 2024-04-21T08:45:53.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.757+0000 7f4cd37fe640 1 -- 172.21.15.86:0/1631080286 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cd800faa0 con 0x7f4cdc19cf70 2024-04-21T08:45:53.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.757+0000 7f4ce426d640 1 -- 172.21.15.86:0/1631080286 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4cdc19dd50 con 0x7f4cdc19cf70 2024-04-21T08:45:53.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.758+0000 7f4cd37fe640 1 -- 172.21.15.86:0/1631080286 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd8018b70 con 0x7f4cdc19cf70 2024-04-21T08:45:53.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.758+0000 7f4ce426d640 1 -- 172.21.15.86:0/1631080286 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4cdc1a54c0 con 0x7f4cdc19cf70 2024-04-21T08:45:53.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.760+0000 7f4cd37fe640 1 -- 172.21.15.86:0/1631080286 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4cd8018d10 con 0x7f4cdc19cf70 2024-04-21T08:45:53.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.760+0000 7f4ce6e65740 1 Processor -- start 2024-04-21T08:45:53.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.760+0000 7f4ce6e65740 1 -- start start 2024-04-21T08:45:53.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.761+0000 7f4ce6e65740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff3e7c4d0 0x55cff3e7c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.761+0000 7f4ce6e65740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff3e7ce80 0x55cff3e85430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.761+0000 7f4ce6e65740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff3e85a00 0x55cff3e87e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.761+0000 7f4ce6e65740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cff3de11f0 con 0x55cff3e7ce80 2024-04-21T08:45:53.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.761+0000 7f4ce6e65740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55cff3d7e4b0 con 0x55cff3e7c4d0 2024-04-21T08:45:53.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.761+0000 7f4ce6e65740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55cff3ddd1b0 con 0x55cff3e85a00 2024-04-21T08:45:53.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.761+0000 7f4cd37fe640 1 --2- 172.21.15.86:0/1631080286 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4cc80a3d20 0x7f4cc80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.761+0000 7f4ce2009640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff3e7ce80 0x55cff3e85430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.761+0000 7f4ce280a640 1 --2- 172.21.15.86:0/1631080286 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4cc80a3d20 0x7f4cc80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.761+0000 7f4ce2009640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff3e7ce80 0x55cff3e85430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38442/0 (socket says 172.21.15.86:38442) 2024-04-21T08:45:53.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.761+0000 7f4ce2009640 1 -- 172.21.15.86:0/3101798787 learned_addr learned my addr 172.21.15.86:0/3101798787 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:53.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.761+0000 7f4ce1808640 1 --2- 172.21.15.86:0/3101798787 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff3e7c4d0 0x55cff3e7c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.761+0000 7f4ce1808640 1 --2- 172.21.15.86:0/3101798787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff3e85a00 0x55cff3e87e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.761+0000 7f4cd37fe640 1 -- 172.21.15.86:0/1631080286 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f4cd80d1020 con 0x7f4cdc19cf70 2024-04-21T08:45:53.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.761+0000 7f4ce1808640 1 -- 172.21.15.86:0/3101798787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff3e85a00 msgr2=0x55cff3e87e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:53.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.761+0000 7f4ce1808640 1 --2- 172.21.15.86:0/3101798787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff3e85a00 0x55cff3e87e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.761+0000 7f4ce1808640 1 -- 172.21.15.86:0/3101798787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff3e7ce80 msgr2=0x55cff3e85430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:53.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.761+0000 7f4ce1808640 1 --2- 172.21.15.86:0/3101798787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff3e7ce80 0x55cff3e85430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.761+0000 7f4ce1808640 1 -- 172.21.15.86:0/3101798787 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cff3d701b0 con 0x55cff3e7c4d0 2024-04-21T08:45:53.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.761+0000 7f4ce2009640 1 --2- 172.21.15.86:0/3101798787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff3e7ce80 0x55cff3e85430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:45:53.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.761+0000 7f4ce280a640 1 --2- 172.21.15.86:0/1631080286 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4cc80a3d20 0x7f4cc80a61e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4cd80ce9a0 tx=0x7f4cd806b5a0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:53.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.761+0000 7f4ce1808640 1 --2- 172.21.15.86:0/3101798787 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff3e7c4d0 0x55cff3e7c8b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc036040 tx=0x7f4ccc02bc30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=5e2e953e0fd786c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:53.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.762+0000 7f4cd0bfe640 1 -- 172.21.15.86:0/3101798787 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc02c9e0 con 0x55cff3e7c4d0 2024-04-21T08:45:53.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.762+0000 7f4cd0bfe640 1 -- 172.21.15.86:0/3101798787 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ccc02cb80 con 0x55cff3e7c4d0 2024-04-21T08:45:53.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.762+0000 7f4cd0bfe640 1 -- 172.21.15.86:0/3101798787 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc043690 con 0x55cff3e7c4d0 2024-04-21T08:45:53.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.762+0000 7f4ce6e65740 1 -- 172.21.15.86:0/3101798787 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff3e7c4d0 msgr2=0x55cff3e7c8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.762+0000 7f4ce6e65740 1 --2- 172.21.15.86:0/3101798787 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff3e7c4d0 0x55cff3e7c8b0 secure :-1 s=READY pgs=74 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc036040 tx=0x7f4ccc02bc30 comp rx=0 tx=0).stop 2024-04-21T08:45:53.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.762+0000 7f4ce6e65740 1 -- 172.21.15.86:0/3101798787 shutdown_connections 2024-04-21T08:45:53.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.762+0000 7f4ce6e65740 1 --2- 172.21.15.86:0/3101798787 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff3e85a00 0x55cff3e87e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.762+0000 7f4ce6e65740 1 --2- 172.21.15.86:0/3101798787 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff3e7ce80 0x55cff3e85430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.762+0000 7f4ce6e65740 1 --2- 172.21.15.86:0/3101798787 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff3e7c4d0 0x55cff3e7c8b0 unknown :-1 s=CLOSED pgs=74 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.762+0000 7f4ce6e65740 1 -- 172.21.15.86:0/3101798787 >> 172.21.15.86:0/3101798787 conn(0x55cff3e76ba0 msgr2=0x55cff3e76f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:53.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.762+0000 7f4ce6e65740 1 -- 172.21.15.86:0/3101798787 shutdown_connections 2024-04-21T08:45:53.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.762+0000 7f4ce6e65740 1 -- 172.21.15.86:0/3101798787 wait complete. 2024-04-21T08:45:53.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.763+0000 7f4ce6e65740 1 -- start start 2024-04-21T08:45:53.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.763+0000 7f4ce6e65740 1 Processor -- start 2024-04-21T08:45:53.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.763+0000 7f4ce6e65740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff3e7c4d0 0x55cff3e85390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.763+0000 7f4ce6e65740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff3e7ce80 0x55cff3e82720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.763+0000 7f4ce6e65740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff3e85a00 0x55cff3e82d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.763+0000 7f4ce6e65740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cff3e88910 con 0x55cff3e7c4d0 2024-04-21T08:45:53.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.763+0000 7f4ce6e65740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55cff3d9a500 con 0x55cff3e7ce80 2024-04-21T08:45:53.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.763+0000 7f4ce6e65740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55cff3e88a90 con 0x55cff3e85a00 2024-04-21T08:45:53.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.763+0000 7f4ce1808640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff3e7c4d0 0x55cff3e85390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.763+0000 7f4ce6e65740 10 client.0 _ll_register_callbacks cb 0x55cff3d68b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:53.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.763+0000 7f4ce1808640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff3e7c4d0 0x55cff3e85390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38456/0 (socket says 172.21.15.86:38456) 2024-04-21T08:45:53.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.763+0000 7f4ce1808640 1 -- 172.21.15.86:0/1347199295 learned_addr learned my addr 172.21.15.86:0/1347199295 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:53.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.763+0000 7f4ce2009640 1 --2- 172.21.15.86:0/1347199295 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff3e7ce80 0x55cff3e82720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.763+0000 7f4ce1808640 1 --2- 172.21.15.86:0/1347199295 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff3e85a00 0x55cff3e82d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.763+0000 7f4ce1808640 1 -- 172.21.15.86:0/1347199295 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff3e85a00 msgr2=0x55cff3e82d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:53.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.763+0000 7f4ce1808640 1 --2- 172.21.15.86:0/1347199295 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff3e85a00 0x55cff3e82d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.763+0000 7f4ce1808640 1 -- 172.21.15.86:0/1347199295 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff3e7ce80 msgr2=0x55cff3e82720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:53.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.763+0000 7f4ce1808640 1 --2- 172.21.15.86:0/1347199295 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff3e7ce80 0x55cff3e82720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.763+0000 7f4ce1808640 1 -- 172.21.15.86:0/1347199295 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cff3af33f0 con 0x55cff3e7c4d0 2024-04-21T08:45:53.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.764+0000 7f4ce1808640 1 --2- 172.21.15.86:0/1347199295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff3e7c4d0 0x55cff3e85390 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc02c320 tx=0x7f4ccc02c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=219fc332f911e9e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:53.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.764+0000 7f4cc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:53.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.764+0000 7f4cc5ffb640 1 -- 172.21.15.86:0/1347199295 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc005250 con 0x55cff3e7c4d0 2024-04-21T08:45:53.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.764+0000 7f4ce6e65740 1 -- 172.21.15.86:0/1347199295 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4ccc02cb80 con 0x55cff3e7c4d0 2024-04-21T08:45:53.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.764+0000 7f4cc5ffb640 1 -- 172.21.15.86:0/1347199295 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f4ccc0053f0 con 0x55cff3e7c4d0 2024-04-21T08:45:53.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.764+0000 7f4cc5ffb640 1 -- 172.21.15.86:0/1347199295 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4ccc060050 con 0x55cff3e7c4d0 2024-04-21T08:45:53.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.764+0000 7f4ce6e65740 10 client.18546 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:53.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.764+0000 7f4ce6e65740 1 -- 172.21.15.86:0/1347199295 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cff3bbd7d0 con 0x55cff3e7c4d0 2024-04-21T08:45:53.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.764+0000 7f4cc5ffb640 1 -- 172.21.15.86:0/1347199295 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4ccc02cb80 con 0x55cff3e7c4d0 2024-04-21T08:45:53.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.764+0000 7f4cc5ffb640 1 -- 172.21.15.86:0/1347199295 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4ccc040040 con 0x55cff3e7c4d0 2024-04-21T08:45:53.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.764+0000 7f4ce6e65740 10 client.18546 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:53.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.764+0000 7f4ce6e65740 10 client.18546 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:45:53.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.764+0000 7f4ce6e65740 20 client.18546 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:53.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.764+0000 7f4ce6e65740 1 --2- 172.21.15.86:0/1347199295 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55cff3e95ac0 0x55cff3e97eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.764+0000 7f4ce6e65740 4 client.18546 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:53.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.764+0000 7f4ce6e65740 1 -- 172.21.15.86:0/1347199295 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cff3d80930 con 0x55cff3e95ac0 2024-04-21T08:45:53.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.765+0000 7f4ce2009640 1 --2- 172.21.15.86:0/1347199295 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55cff3e95ac0 0x55cff3e97eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.765+0000 7f4cc5ffb640 10 client.18546 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:45:53.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.765+0000 7f4ce2009640 1 --2- 172.21.15.86:0/1347199295 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55cff3e95ac0 0x55cff3e97eb0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f9825fbdd6b1b8fe server_cookie=cbdef36505026bce in_seq=0 out_seq=0 2024-04-21T08:45:53.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.766+0000 7f4cc5ffb640 1 -- 172.21.15.86:0/1347199295 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cff3d80930 con 0x55cff3e95ac0 2024-04-21T08:45:53.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.766+0000 7f4cc5ffb640 10 client.18546 handle_command_reply: tid=0 2024-04-21T08:45:53.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.766+0000 7f4ce6e65740 1 client.18546 shutdown 2024-04-21T08:45:53.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.766+0000 7f4ce6e65740 1 -- 172.21.15.86:0/1347199295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff3e7c4d0 msgr2=0x55cff3e85390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.766+0000 7f4ce6e65740 1 --2- 172.21.15.86:0/1347199295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff3e7c4d0 0x55cff3e85390 secure :-1 s=READY pgs=1455 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc02c320 tx=0x7f4ccc02c2b0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.766+0000 7f4ce6e65740 1 -- 172.21.15.86:0/1347199295 shutdown_connections 2024-04-21T08:45:53.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.766+0000 7f4ce6e65740 1 --2- 172.21.15.86:0/1347199295 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55cff3e95ac0 0x55cff3e97eb0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.766+0000 7f4ce6e65740 1 --2- 172.21.15.86:0/1347199295 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff3e85a00 0x55cff3e82d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.766+0000 7f4ce6e65740 1 --2- 172.21.15.86:0/1347199295 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff3e7ce80 0x55cff3e82720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.766+0000 7f4ce6e65740 1 --2- 172.21.15.86:0/1347199295 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff3e7c4d0 0x55cff3e85390 unknown :-1 s=CLOSED pgs=1455 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.766+0000 7f4cc5ffb640 0 client.18546 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:45:53.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.766+0000 7f4ce6e65740 1 -- 172.21.15.86:0/1347199295 >> 172.21.15.86:0/1347199295 conn(0x55cff3e76ba0 msgr2=0x55cff3e788f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:53.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.766+0000 7f4ce6e65740 1 -- 172.21.15.86:0/1347199295 shutdown_connections 2024-04-21T08:45:53.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.766+0000 7f4ce6e65740 1 -- 172.21.15.86:0/1347199295 wait complete. 2024-04-21T08:45:53.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.766+0000 7f4ce6e65740 20 client.18546 trim_cache size 0 max 16384 2024-04-21T08:45:53.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.779+0000 7f4ce6e65740 1 Processor -- start 2024-04-21T08:45:53.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.779+0000 7f4ce6e65740 1 -- start start 2024-04-21T08:45:53.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.780+0000 7f4ce6e65740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff3e7c4d0 0x55cff3e917b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.780+0000 7f4ce6e65740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff3e7ce80 0x55cff3e84e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.780+0000 7f4ce6e65740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff3e85a00 0x55cff3e85390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.780+0000 7f4ce6e65740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc1ad000 con 0x55cff3e7ce80 2024-04-21T08:45:53.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.780+0000 7f4ce6e65740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc1aeb40 con 0x55cff3e85a00 2024-04-21T08:45:53.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.780+0000 7f4ce6e65740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc1ad6d0 con 0x55cff3e7c4d0 2024-04-21T08:45:53.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.780+0000 7f4ce2009640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff3e7ce80 0x55cff3e84e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.780+0000 7f4ce2009640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff3e7ce80 0x55cff3e84e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38464/0 (socket says 172.21.15.86:38464) 2024-04-21T08:45:53.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.780+0000 7f4ce2009640 1 -- 172.21.15.86:0/1296631940 learned_addr learned my addr 172.21.15.86:0/1296631940 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:53.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.780+0000 7f4ce1808640 1 --2- 172.21.15.86:0/1296631940 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff3e85a00 0x55cff3e85390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.780+0000 7f4ce2009640 1 -- 172.21.15.86:0/1296631940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff3e7c4d0 msgr2=0x55cff3e917b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:53.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.780+0000 7f4ce2009640 1 --2- 172.21.15.86:0/1296631940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff3e7c4d0 0x55cff3e917b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.780+0000 7f4ce2009640 1 -- 172.21.15.86:0/1296631940 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff3e85a00 msgr2=0x55cff3e85390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:53.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.780+0000 7f4ce2009640 1 --2- 172.21.15.86:0/1296631940 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff3e85a00 0x55cff3e85390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.780+0000 7f4ce2009640 1 -- 172.21.15.86:0/1296631940 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cff3e882b0 con 0x55cff3e7ce80 2024-04-21T08:45:53.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.780+0000 7f4ce1808640 1 --2- 172.21.15.86:0/1296631940 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff3e85a00 0x55cff3e85390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:53.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.780+0000 7f4ce2009640 1 --2- 172.21.15.86:0/1296631940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff3e7ce80 0x55cff3e84e50 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4005d00 tx=0x7f4cd4009cf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca9e2d3d02e2aaf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:53.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.780+0000 7f4cc7fff640 1 -- 172.21.15.86:0/1296631940 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd4004250 con 0x55cff3e7ce80 2024-04-21T08:45:53.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.780+0000 7f4cc7fff640 1 -- 172.21.15.86:0/1296631940 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cd40043f0 con 0x55cff3e7ce80 2024-04-21T08:45:53.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.780+0000 7f4cc7fff640 1 -- 172.21.15.86:0/1296631940 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4cd40150d0 con 0x55cff3e7ce80 2024-04-21T08:45:53.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.781+0000 7f4ce6e65740 1 -- 172.21.15.86:0/1296631940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff3e7ce80 msgr2=0x55cff3e84e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.781+0000 7f4ce6e65740 1 --2- 172.21.15.86:0/1296631940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff3e7ce80 0x55cff3e84e50 secure :-1 s=READY pgs=1456 cs=0 l=1 rev1=1 crypto rx=0x7f4cd4005d00 tx=0x7f4cd4009cf0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.781+0000 7f4ce6e65740 1 -- 172.21.15.86:0/1296631940 shutdown_connections 2024-04-21T08:45:53.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.781+0000 7f4ce6e65740 1 --2- 172.21.15.86:0/1296631940 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff3e85a00 0x55cff3e85390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.781+0000 7f4ce6e65740 1 --2- 172.21.15.86:0/1296631940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff3e7ce80 0x55cff3e84e50 unknown :-1 s=CLOSED pgs=1456 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.781+0000 7f4ce6e65740 1 --2- 172.21.15.86:0/1296631940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff3e7c4d0 0x55cff3e917b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.781+0000 7f4ce6e65740 1 -- 172.21.15.86:0/1296631940 >> 172.21.15.86:0/1296631940 conn(0x55cff3e76ba0 msgr2=0x55cff3e81750 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:53.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.781+0000 7f4ce6e65740 1 -- 172.21.15.86:0/1296631940 shutdown_connections 2024-04-21T08:45:53.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.781+0000 7f4ce6e65740 1 -- 172.21.15.86:0/1296631940 wait complete. 2024-04-21T08:45:53.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.781+0000 7f4ce6e65740 1 -- start start 2024-04-21T08:45:53.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.781+0000 7f4ce6e65740 1 Processor -- start 2024-04-21T08:45:53.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.781+0000 7f4ce6e65740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff3e7c4d0 0x55cff3e98a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.781+0000 7f4ce6e65740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff3e7ce80 0x55cff3e98fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.781+0000 7f4ce6e65740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff3e85a00 0x55cff3e99500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.781+0000 7f4ce6e65740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc1ae470 con 0x55cff3e7c4d0 2024-04-21T08:45:53.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.781+0000 7f4ce6e65740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc1adda0 con 0x55cff3e85a00 2024-04-21T08:45:53.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.781+0000 7f4ce6e65740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4cdc1ab000 con 0x55cff3e7ce80 2024-04-21T08:45:53.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.781+0000 7f4ce6e65740 10 client.0 _ll_register_callbacks cb 0x7f4ccc004560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:53.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.781+0000 7f4ce1808640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff3e7c4d0 0x55cff3e98a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.781+0000 7f4ce1808640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff3e7c4d0 0x55cff3e98a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38474/0 (socket says 172.21.15.86:38474) 2024-04-21T08:45:53.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.782+0000 7f4ce1808640 1 -- 172.21.15.86:0/3260521085 learned_addr learned my addr 172.21.15.86:0/3260521085 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:53.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.782+0000 7f4ce2009640 1 --2- 172.21.15.86:0/3260521085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff3e7ce80 0x55cff3e98fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.782+0000 7f4ce1808640 1 -- 172.21.15.86:0/3260521085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff3e7ce80 msgr2=0x55cff3e98fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:53.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.782+0000 7f4ce1808640 1 --2- 172.21.15.86:0/3260521085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff3e7ce80 0x55cff3e98fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.782+0000 7f4ce1808640 1 -- 172.21.15.86:0/3260521085 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff3e85a00 msgr2=0x55cff3e99500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:53.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.782+0000 7f4ce1808640 1 --2- 172.21.15.86:0/3260521085 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff3e85a00 0x55cff3e99500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.782+0000 7f4ce1808640 1 -- 172.21.15.86:0/3260521085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cff3efada0 con 0x55cff3e7c4d0 2024-04-21T08:45:53.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.782+0000 7f4ce2009640 1 --2- 172.21.15.86:0/3260521085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff3e7ce80 0x55cff3e98fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:53.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.782+0000 7f4ce1808640 1 --2- 172.21.15.86:0/3260521085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff3e7c4d0 0x55cff3e98a80 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc004c60 tx=0x7f4ccc004f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8d9c1ae050e30332 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:53.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.782+0000 7f4cc5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:53.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.782+0000 7f4cc5ffb640 1 -- 172.21.15.86:0/3260521085 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4ccc053020 con 0x55cff3e7c4d0 2024-04-21T08:45:53.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.782+0000 7f4cc5ffb640 1 -- 172.21.15.86:0/3260521085 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f4ccc003f80 con 0x55cff3e7c4d0 2024-04-21T08:45:53.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.782+0000 7f4ce6e65740 1 -- 172.21.15.86:0/3260521085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4cd40043f0 con 0x55cff3e7c4d0 2024-04-21T08:45:53.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.782+0000 7f4cc5ffb640 1 -- 172.21.15.86:0/3260521085 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4ccc06b050 con 0x55cff3e7c4d0 2024-04-21T08:45:53.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.782+0000 7f4ce6e65740 10 client.18552 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:53.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.782+0000 7f4ce6e65740 1 -- 172.21.15.86:0/3260521085 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cff3efc010 con 0x55cff3e7c4d0 2024-04-21T08:45:53.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.782+0000 7f4cc5ffb640 1 -- 172.21.15.86:0/3260521085 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4cd40043f0 con 0x55cff3e7c4d0 2024-04-21T08:45:53.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.782+0000 7f4cc5ffb640 1 -- 172.21.15.86:0/3260521085 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4ccc003680 con 0x55cff3e7c4d0 2024-04-21T08:45:53.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.782+0000 7f4ce6e65740 10 client.18552 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:53.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.782+0000 7f4ce6e65740 10 client.18552 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:45:53.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.782+0000 7f4ce6e65740 20 client.18552 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:53.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.783+0000 7f4ce6e65740 1 --2- 172.21.15.86:0/3260521085 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55cff3efc680 0x55cff3efea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.783+0000 7f4ce6e65740 4 client.18552 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:45:53.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.783+0000 7f4ce6e65740 1 -- 172.21.15.86:0/3260521085 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cff3deec10 con 0x55cff3efc680 2024-04-21T08:45:53.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.783+0000 7f4ce2009640 1 --2- 172.21.15.86:0/3260521085 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55cff3efc680 0x55cff3efea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.783+0000 7f4cc5ffb640 10 client.18552 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:45:53.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.783+0000 7f4ce2009640 1 --2- 172.21.15.86:0/3260521085 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55cff3efc680 0x55cff3efea70 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=b9c416a00a4535fd server_cookie=b6594f72697b56d5 in_seq=0 out_seq=0 2024-04-21T08:45:53.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.783+0000 7f4cc5ffb640 1 -- 172.21.15.86:0/3260521085 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cff3deec10 con 0x55cff3efc680 2024-04-21T08:45:53.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.783+0000 7f4cc5ffb640 10 client.18552 handle_command_reply: tid=0 2024-04-21T08:45:53.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.783+0000 7f4ce6e65740 1 client.18552 shutdown 2024-04-21T08:45:53.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.784+0000 7f4ce6e65740 1 -- 172.21.15.86:0/3260521085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff3e7c4d0 msgr2=0x55cff3e98a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.784+0000 7f4ce6e65740 1 --2- 172.21.15.86:0/3260521085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff3e7c4d0 0x55cff3e98a80 secure :-1 s=READY pgs=1457 cs=0 l=1 rev1=1 crypto rx=0x7f4ccc004c60 tx=0x7f4ccc004f10 comp rx=0 tx=0).stop 2024-04-21T08:45:53.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.784+0000 7f4ce6e65740 1 -- 172.21.15.86:0/3260521085 shutdown_connections 2024-04-21T08:45:53.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.784+0000 7f4ce6e65740 1 --2- 172.21.15.86:0/3260521085 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55cff3efc680 0x55cff3efea70 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.784+0000 7f4ce6e65740 1 --2- 172.21.15.86:0/3260521085 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff3e85a00 0x55cff3e99500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.784+0000 7f4cc5ffb640 0 client.18552 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:45:53.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.784+0000 7f4ce6e65740 1 --2- 172.21.15.86:0/3260521085 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff3e7ce80 0x55cff3e98fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.784+0000 7f4ce6e65740 1 --2- 172.21.15.86:0/3260521085 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff3e7c4d0 0x55cff3e98a80 unknown :-1 s=CLOSED pgs=1457 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.784+0000 7f4ce6e65740 1 -- 172.21.15.86:0/3260521085 >> 172.21.15.86:0/3260521085 conn(0x55cff3e76ba0 msgr2=0x55cff3e75370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:53.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.784+0000 7f4ce6e65740 1 -- 172.21.15.86:0/3260521085 shutdown_connections 2024-04-21T08:45:53.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.784+0000 7f4ce6e65740 1 -- 172.21.15.86:0/3260521085 wait complete. 2024-04-21T08:45:53.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.784+0000 7f4ce6e65740 20 client.18552 trim_cache size 0 max 16384 2024-04-21T08:45:53.791 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:45:53.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.784+0000 7f4ce426d640 1 -- 172.21.15.86:0/1631080286 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4cc80a3d20 msgr2=0x7f4cc80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.784+0000 7f4ce426d640 1 --2- 172.21.15.86:0/1631080286 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4cc80a3d20 0x7f4cc80a61e0 secure :-1 s=READY pgs=233 cs=0 l=1 rev1=1 crypto rx=0x7f4cd80ce9a0 tx=0x7f4cd806b5a0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.784+0000 7f4ce426d640 1 -- 172.21.15.86:0/1631080286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cdc19cf70 msgr2=0x7f4cdc19d420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.784+0000 7f4ce426d640 1 --2- 172.21.15.86:0/1631080286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cdc19cf70 0x7f4cdc19d420 secure :-1 s=READY pgs=1454 cs=0 l=1 rev1=1 crypto rx=0x7f4cd800cce0 tx=0x7f4cd800a620 comp rx=0 tx=0).stop 2024-04-21T08:45:53.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.784+0000 7f4ce426d640 1 -- 172.21.15.86:0/1631080286 shutdown_connections 2024-04-21T08:45:53.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.784+0000 7f4ce426d640 1 --2- 172.21.15.86:0/1631080286 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4cc80a3d20 0x7f4cc80a61e0 unknown :-1 s=CLOSED pgs=233 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.784+0000 7f4ce426d640 1 --2- 172.21.15.86:0/1631080286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4cdc19cf70 0x7f4cdc19d420 unknown :-1 s=CLOSED pgs=1454 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.784+0000 7f4ce426d640 1 --2- 172.21.15.86:0/1631080286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4cdc107270 0x7f4cdc198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.784+0000 7f4ce426d640 1 --2- 172.21.15.86:0/1631080286 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4cdc06a860 0x7f4cdc1986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.784+0000 7f4ce426d640 1 -- 172.21.15.86:0/1631080286 >> 172.21.15.86:0/1631080286 conn(0x7f4cdc077eb0 msgr2=0x7f4cdc100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:53.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.784+0000 7f4ce426d640 1 -- 172.21.15.86:0/1631080286 shutdown_connections 2024-04-21T08:45:53.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.784+0000 7f4ce426d640 1 -- 172.21.15.86:0/1631080286 wait complete. 2024-04-21T08:45:53.794 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:45:53.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.895+0000 7f70346d3640 1 -- 172.21.15.86:0/3835801185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f702c103af0 msgr2=0x7f702c10bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.895+0000 7f70346d3640 1 --2- 172.21.15.86:0/3835801185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f702c103af0 0x7f702c10bfd0 secure :-1 s=READY pgs=1458 cs=0 l=1 rev1=1 crypto rx=0x7f701c009900 tx=0x7f701c02bd60 comp rx=0 tx=0).stop 2024-04-21T08:45:53.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.896+0000 7f70346d3640 1 -- 172.21.15.86:0/3835801185 shutdown_connections 2024-04-21T08:45:53.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.896+0000 7f70346d3640 1 --2- 172.21.15.86:0/3835801185 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f702c10c510 0x7f702c10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.896+0000 7f70346d3640 1 --2- 172.21.15.86:0/3835801185 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f702c103af0 0x7f702c10bfd0 unknown :-1 s=CLOSED pgs=1458 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.896+0000 7f70346d3640 1 --2- 172.21.15.86:0/3835801185 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f702c1031d0 0x7f702c1035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.896+0000 7f70346d3640 1 -- 172.21.15.86:0/3835801185 >> 172.21.15.86:0/3835801185 conn(0x7f702c077f50 msgr2=0x7f702c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:53.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.896+0000 7f70346d3640 1 -- 172.21.15.86:0/3835801185 shutdown_connections 2024-04-21T08:45:53.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.896+0000 7f70346d3640 1 -- 172.21.15.86:0/3835801185 wait complete. 2024-04-21T08:45:53.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.896+0000 7f70346d3640 1 Processor -- start 2024-04-21T08:45:53.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.896+0000 7f70346d3640 1 -- start start 2024-04-21T08:45:53.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.896+0000 7f70346d3640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f702c1031d0 0x7f702c198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.896+0000 7f70346d3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f702c103af0 0x7f702c198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.896+0000 7f70346d3640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f702c10c510 0x7f702c19cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.896+0000 7f70346d3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f702c1121a0 con 0x7f702c103af0 2024-04-21T08:45:53.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.896+0000 7f70346d3640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f702c112020 con 0x7f702c1031d0 2024-04-21T08:45:53.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.896+0000 7f70346d3640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f702c112320 con 0x7f702c10c510 2024-04-21T08:45:53.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.896+0000 7f7031c6e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f702c103af0 0x7f702c198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.896+0000 7f7031c6e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f702c103af0 0x7f702c198c60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38494/0 (socket says 172.21.15.86:38494) 2024-04-21T08:45:53.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.896+0000 7f7031c6e640 1 -- 172.21.15.86:0/2053760410 learned_addr learned my addr 172.21.15.86:0/2053760410 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:53.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.897+0000 7f7032c70640 1 --2- 172.21.15.86:0/2053760410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f702c10c510 0x7f702c19cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.897+0000 7f7031c6e640 1 -- 172.21.15.86:0/2053760410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f702c10c510 msgr2=0x7f702c19cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.897+0000 7f7031c6e640 1 --2- 172.21.15.86:0/2053760410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f702c10c510 0x7f702c19cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.897+0000 7f703246f640 1 --2- 172.21.15.86:0/2053760410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f702c1031d0 0x7f702c198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.897+0000 7f7031c6e640 1 -- 172.21.15.86:0/2053760410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f702c1031d0 msgr2=0x7f702c198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.897+0000 7f7031c6e640 1 --2- 172.21.15.86:0/2053760410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f702c1031d0 0x7f702c198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.897+0000 7f7031c6e640 1 -- 172.21.15.86:0/2053760410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f702c19d710 con 0x7f702c103af0 2024-04-21T08:45:53.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.897+0000 7f7032c70640 1 --2- 172.21.15.86:0/2053760410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f702c10c510 0x7f702c19cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:53.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.897+0000 7f7031c6e640 1 --2- 172.21.15.86:0/2053760410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f702c103af0 0x7f702c198c60 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f701c005940 tx=0x7f701c02d920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:53.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.897+0000 7f70237fe640 1 -- 172.21.15.86:0/2053760410 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c043020 con 0x7f702c103af0 2024-04-21T08:45:53.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.897+0000 7f70237fe640 1 -- 172.21.15.86:0/2053760410 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f701c02dd20 con 0x7f702c103af0 2024-04-21T08:45:53.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.897+0000 7f70346d3640 1 -- 172.21.15.86:0/2053760410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f702c19d9a0 con 0x7f702c103af0 2024-04-21T08:45:53.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.897+0000 7f70237fe640 1 -- 172.21.15.86:0/2053760410 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c003b80 con 0x7f702c103af0 2024-04-21T08:45:53.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.897+0000 7f70346d3640 1 -- 172.21.15.86:0/2053760410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f702c1a5290 con 0x7f702c103af0 2024-04-21T08:45:53.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.899+0000 7f70237fe640 1 -- 172.21.15.86:0/2053760410 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f701c046020 con 0x7f702c103af0 2024-04-21T08:45:53.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.899+0000 7f703739d740 1 Processor -- start 2024-04-21T08:45:53.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.900+0000 7f703739d740 1 -- start start 2024-04-21T08:45:53.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.900+0000 7f703739d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55790509f4d0 0x55790509f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.900+0000 7f703739d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55790509fe80 0x5579050a8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.900+0000 7f703739d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579050a8a00 0x5579050aae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.900+0000 7f703739d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5579050041f0 con 0x55790509fe80 2024-04-21T08:45:53.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.900+0000 7f703739d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x557904fa14b0 con 0x55790509f4d0 2024-04-21T08:45:53.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.900+0000 7f703739d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5579050001b0 con 0x5579050a8a00 2024-04-21T08:45:53.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.900+0000 7f703246f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55790509fe80 0x5579050a8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.900+0000 7f703246f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55790509fe80 0x5579050a8430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38504/0 (socket says 172.21.15.86:38504) 2024-04-21T08:45:53.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.900+0000 7f703246f640 1 -- 172.21.15.86:0/4283635820 learned_addr learned my addr 172.21.15.86:0/4283635820 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:53.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.900+0000 7f70237fe640 1 --2- 172.21.15.86:0/2053760410 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f70180a3d20 0x7f70180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.900+0000 7f7032c70640 1 --2- 172.21.15.86:0/4283635820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55790509f4d0 0x55790509f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.900+0000 7f7031c6e640 1 --2- 172.21.15.86:0/4283635820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579050a8a00 0x5579050aae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.900+0000 7f703246f640 1 -- 172.21.15.86:0/4283635820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579050a8a00 msgr2=0x5579050aae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:53.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.900+0000 7f7032c70640 1 --2- 172.21.15.86:0/2053760410 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f70180a3d20 0x7f70180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.900+0000 7f703246f640 1 --2- 172.21.15.86:0/4283635820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579050a8a00 0x5579050aae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.900+0000 7f703246f640 1 -- 172.21.15.86:0/4283635820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55790509f4d0 msgr2=0x55790509f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:53.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.900+0000 7f703246f640 1 --2- 172.21.15.86:0/4283635820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55790509f4d0 0x55790509f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.900+0000 7f70237fe640 1 -- 172.21.15.86:0/2053760410 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f701c0f2de0 con 0x7f702c103af0 2024-04-21T08:45:53.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.900+0000 7f703246f640 1 -- 172.21.15.86:0/4283635820 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557904f931b0 con 0x55790509fe80 2024-04-21T08:45:53.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.900+0000 7f7031c6e640 1 --2- 172.21.15.86:0/4283635820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579050a8a00 0x5579050aae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:53.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.901+0000 7f703246f640 1 --2- 172.21.15.86:0/4283635820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55790509fe80 0x5579050a8430 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f7028009bc0 tx=0x7f702800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cfeadb65484524d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:53.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.901+0000 7f7032c70640 1 --2- 172.21.15.86:0/2053760410 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f70180a3d20 0x7f70180a61e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7024005c40 tx=0x7f7024009280 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:53.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.901+0000 7f7020ff9640 1 -- 172.21.15.86:0/4283635820 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7028016070 con 0x55790509fe80 2024-04-21T08:45:53.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.901+0000 7f7020ff9640 1 -- 172.21.15.86:0/4283635820 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70280054d0 con 0x55790509fe80 2024-04-21T08:45:53.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.901+0000 7f7020ff9640 1 -- 172.21.15.86:0/4283635820 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7028011470 con 0x55790509fe80 2024-04-21T08:45:53.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.901+0000 7f703739d740 1 -- 172.21.15.86:0/4283635820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55790509fe80 msgr2=0x5579050a8430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.901+0000 7f703739d740 1 --2- 172.21.15.86:0/4283635820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55790509fe80 0x5579050a8430 secure :-1 s=READY pgs=1460 cs=0 l=1 rev1=1 crypto rx=0x7f7028009bc0 tx=0x7f702800e6c0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.901+0000 7f703739d740 1 -- 172.21.15.86:0/4283635820 shutdown_connections 2024-04-21T08:45:53.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.901+0000 7f703739d740 1 --2- 172.21.15.86:0/4283635820 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579050a8a00 0x5579050aae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.901+0000 7f703739d740 1 --2- 172.21.15.86:0/4283635820 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55790509fe80 0x5579050a8430 unknown :-1 s=CLOSED pgs=1460 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.901+0000 7f703739d740 1 --2- 172.21.15.86:0/4283635820 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55790509f4d0 0x55790509f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.901+0000 7f703739d740 1 -- 172.21.15.86:0/4283635820 >> 172.21.15.86:0/4283635820 conn(0x557905099ba0 msgr2=0x557905099f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:53.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.902+0000 7f703739d740 1 -- 172.21.15.86:0/4283635820 shutdown_connections 2024-04-21T08:45:53.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.902+0000 7f703739d740 1 -- 172.21.15.86:0/4283635820 wait complete. 2024-04-21T08:45:53.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.902+0000 7f703739d740 1 -- start start 2024-04-21T08:45:53.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.902+0000 7f703739d740 1 Processor -- start 2024-04-21T08:45:53.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.902+0000 7f703739d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55790509f4d0 0x5579050b2a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.902+0000 7f703739d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55790509fe80 0x5579050b2f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.902+0000 7f703739d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579050a8a00 0x5579050ace40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.902+0000 7f703739d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5579050ab970 con 0x55790509fe80 2024-04-21T08:45:53.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.902+0000 7f703739d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x557904fbd500 con 0x5579050a8a00 2024-04-21T08:45:53.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.902+0000 7f703739d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5579050abaf0 con 0x55790509f4d0 2024-04-21T08:45:53.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.902+0000 7f703739d740 10 client.0 _ll_register_callbacks cb 0x557904f8bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:53.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.902+0000 7f7031c6e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55790509fe80 0x5579050b2f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.902+0000 7f7031c6e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55790509fe80 0x5579050b2f70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38510/0 (socket says 172.21.15.86:38510) 2024-04-21T08:45:53.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.902+0000 7f7031c6e640 1 -- 172.21.15.86:0/1391672717 learned_addr learned my addr 172.21.15.86:0/1391672717 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:53.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.902+0000 7f7032c70640 1 --2- 172.21.15.86:0/1391672717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579050a8a00 0x5579050ace40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.902+0000 7f703246f640 1 --2- 172.21.15.86:0/1391672717 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55790509f4d0 0x5579050b2a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.902+0000 7f7031c6e640 1 -- 172.21.15.86:0/1391672717 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55790509f4d0 msgr2=0x5579050b2a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:53.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.902+0000 7f7031c6e640 1 --2- 172.21.15.86:0/1391672717 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55790509f4d0 0x5579050b2a30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.902+0000 7f7031c6e640 1 -- 172.21.15.86:0/1391672717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579050a8a00 msgr2=0x5579050ace40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:53.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.902+0000 7f7031c6e640 1 --2- 172.21.15.86:0/1391672717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579050a8a00 0x5579050ace40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.903+0000 7f7031c6e640 1 -- 172.21.15.86:0/1391672717 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557904d163f0 con 0x55790509fe80 2024-04-21T08:45:53.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.903+0000 7f7031c6e640 1 --2- 172.21.15.86:0/1391672717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55790509fe80 0x5579050b2f70 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f701c006010 tx=0x7f701c003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b6aad09e3627041 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:53.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.903+0000 7f7015ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:53.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.903+0000 7f7015ffb640 1 -- 172.21.15.86:0/1391672717 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c0df0b0 con 0x55790509fe80 2024-04-21T08:45:53.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.903+0000 7f7015ffb640 1 -- 172.21.15.86:0/1391672717 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f701c0bb0d0 con 0x55790509fe80 2024-04-21T08:45:53.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.903+0000 7f703739d740 1 -- 172.21.15.86:0/1391672717 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f70280054d0 con 0x55790509fe80 2024-04-21T08:45:53.910 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.903+0000 7f7015ffb640 1 -- 172.21.15.86:0/1391672717 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f701c03c350 con 0x55790509fe80 2024-04-21T08:45:53.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.903+0000 7f703739d740 10 client.18564 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:53.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.903+0000 7f703739d740 1 -- 172.21.15.86:0/1391672717 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557904de07d0 con 0x55790509fe80 2024-04-21T08:45:53.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.903+0000 7f7015ffb640 1 -- 172.21.15.86:0/1391672717 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f70280054d0 con 0x55790509fe80 2024-04-21T08:45:53.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.903+0000 7f7015ffb640 1 -- 172.21.15.86:0/1391672717 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f701c03ac80 con 0x55790509fe80 2024-04-21T08:45:53.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.903+0000 7f703739d740 10 client.18564 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:53.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.903+0000 7f703739d740 10 client.18564 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:45:53.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.903+0000 7f703739d740 20 client.18564 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:53.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.903+0000 7f703739d740 1 --2- 172.21.15.86:0/1391672717 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5579050bce40 0x5579050bf230 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.903+0000 7f703739d740 4 client.18564 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:53.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.903+0000 7f703739d740 1 -- 172.21.15.86:0/1391672717 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557904fa3930 con 0x5579050bce40 2024-04-21T08:45:53.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.904+0000 7f703246f640 1 --2- 172.21.15.86:0/1391672717 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5579050bce40 0x5579050bf230 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.904+0000 7f7015ffb640 10 client.18564 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:45:53.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.904+0000 7f703246f640 1 --2- 172.21.15.86:0/1391672717 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5579050bce40 0x5579050bf230 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=a233f7627a9f9d8b server_cookie=79be2892ffbcb89d in_seq=0 out_seq=0 2024-04-21T08:45:53.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.905+0000 7f7015ffb640 1 -- 172.21.15.86:0/1391672717 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557904fa3930 con 0x5579050bce40 2024-04-21T08:45:53.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.905+0000 7f7015ffb640 10 client.18564 handle_command_reply: tid=0 2024-04-21T08:45:53.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.905+0000 7f703739d740 1 client.18564 shutdown 2024-04-21T08:45:53.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.905+0000 7f703739d740 1 -- 172.21.15.86:0/1391672717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55790509fe80 msgr2=0x5579050b2f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.905+0000 7f703739d740 1 --2- 172.21.15.86:0/1391672717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55790509fe80 0x5579050b2f70 secure :-1 s=READY pgs=1461 cs=0 l=1 rev1=1 crypto rx=0x7f701c006010 tx=0x7f701c003680 comp rx=0 tx=0).stop 2024-04-21T08:45:53.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.905+0000 7f703739d740 1 -- 172.21.15.86:0/1391672717 shutdown_connections 2024-04-21T08:45:53.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.905+0000 7f703739d740 1 --2- 172.21.15.86:0/1391672717 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5579050bce40 0x5579050bf230 crc :-1 s=READY pgs=59 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.905+0000 7f703739d740 1 --2- 172.21.15.86:0/1391672717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579050a8a00 0x5579050ace40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.905+0000 7f703739d740 1 --2- 172.21.15.86:0/1391672717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55790509fe80 0x5579050b2f70 unknown :-1 s=CLOSED pgs=1461 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.905+0000 7f7015ffb640 0 client.18564 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:45:53.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.905+0000 7f703739d740 1 --2- 172.21.15.86:0/1391672717 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55790509f4d0 0x5579050b2a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.905+0000 7f703739d740 1 -- 172.21.15.86:0/1391672717 >> 172.21.15.86:0/1391672717 conn(0x557905099ba0 msgr2=0x55790509b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:53.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.905+0000 7f703739d740 1 -- 172.21.15.86:0/1391672717 shutdown_connections 2024-04-21T08:45:53.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.905+0000 7f703739d740 1 -- 172.21.15.86:0/1391672717 wait complete. 2024-04-21T08:45:53.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.905+0000 7f703739d740 20 client.18564 trim_cache size 0 max 16384 2024-04-21T08:45:53.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.918+0000 7f703739d740 1 Processor -- start 2024-04-21T08:45:53.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.918+0000 7f703739d740 1 -- start start 2024-04-21T08:45:53.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.918+0000 7f703739d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55790509f4d0 0x5579050b2c90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.918+0000 7f703739d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55790509fe80 0x5579050c4b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.918+0000 7f703739d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579050a8a00 0x5579050c5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.918+0000 7f703739d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f702c1ad160 con 0x55790509fe80 2024-04-21T08:45:53.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.918+0000 7f703739d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f702c1aeca0 con 0x5579050a8a00 2024-04-21T08:45:53.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.918+0000 7f703739d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f702c1ad830 con 0x55790509f4d0 2024-04-21T08:45:53.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.918+0000 7f7031c6e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55790509fe80 0x5579050c4b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.918+0000 7f7031c6e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55790509fe80 0x5579050c4b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38514/0 (socket says 172.21.15.86:38514) 2024-04-21T08:45:53.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.918+0000 7f7031c6e640 1 -- 172.21.15.86:0/2448878221 learned_addr learned my addr 172.21.15.86:0/2448878221 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:53.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.918+0000 7f7031c6e640 1 -- 172.21.15.86:0/2448878221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55790509f4d0 msgr2=0x5579050b2c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:53.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.918+0000 7f7031c6e640 1 --2- 172.21.15.86:0/2448878221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55790509f4d0 0x5579050b2c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.918+0000 7f7031c6e640 1 -- 172.21.15.86:0/2448878221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579050a8a00 msgr2=0x5579050c5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:53.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.918+0000 7f7032c70640 1 --2- 172.21.15.86:0/2448878221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579050a8a00 0x5579050c5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.918+0000 7f7031c6e640 1 --2- 172.21.15.86:0/2448878221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579050a8a00 0x5579050c5060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.918+0000 7f7031c6e640 1 -- 172.21.15.86:0/2448878221 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579050a8750 con 0x55790509fe80 2024-04-21T08:45:53.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.919+0000 7f7031c6e640 1 --2- 172.21.15.86:0/2448878221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55790509fe80 0x5579050c4b20 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f701c0be760 tx=0x7f701c003f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed11a8e4908b84dc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:53.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.919+0000 7f7017fff640 1 -- 172.21.15.86:0/2448878221 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c0b8030 con 0x55790509fe80 2024-04-21T08:45:53.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.919+0000 7f7017fff640 1 -- 172.21.15.86:0/2448878221 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f701c0df0b0 con 0x55790509fe80 2024-04-21T08:45:53.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.919+0000 7f7017fff640 1 -- 172.21.15.86:0/2448878221 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f701c0c76d0 con 0x55790509fe80 2024-04-21T08:45:53.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.919+0000 7f703739d740 1 -- 172.21.15.86:0/2448878221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55790509fe80 msgr2=0x5579050c4b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.919+0000 7f703739d740 1 --2- 172.21.15.86:0/2448878221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55790509fe80 0x5579050c4b20 secure :-1 s=READY pgs=1462 cs=0 l=1 rev1=1 crypto rx=0x7f701c0be760 tx=0x7f701c003f30 comp rx=0 tx=0).stop 2024-04-21T08:45:53.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.919+0000 7f703739d740 1 -- 172.21.15.86:0/2448878221 shutdown_connections 2024-04-21T08:45:53.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.919+0000 7f703739d740 1 --2- 172.21.15.86:0/2448878221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579050a8a00 0x5579050c5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.919+0000 7f703739d740 1 --2- 172.21.15.86:0/2448878221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55790509fe80 0x5579050c4b20 unknown :-1 s=CLOSED pgs=1462 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.919+0000 7f703739d740 1 --2- 172.21.15.86:0/2448878221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55790509f4d0 0x5579050b2c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.919+0000 7f703739d740 1 -- 172.21.15.86:0/2448878221 >> 172.21.15.86:0/2448878221 conn(0x557905099ba0 msgr2=0x5579050b2580 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:53.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.919+0000 7f703739d740 1 -- 172.21.15.86:0/2448878221 shutdown_connections 2024-04-21T08:45:53.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.919+0000 7f703739d740 1 -- 172.21.15.86:0/2448878221 wait complete. 2024-04-21T08:45:53.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.920+0000 7f703739d740 1 -- start start 2024-04-21T08:45:53.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.920+0000 7f703739d740 1 Processor -- start 2024-04-21T08:45:53.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.920+0000 7f703739d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55790509f4d0 0x5579050b8fc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.920+0000 7f703739d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55790509fe80 0x5579050b9500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.920+0000 7f703739d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579050a8a00 0x5579050b6500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.920+0000 7f703739d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f702c1ae5d0 con 0x5579050a8a00 2024-04-21T08:45:53.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.920+0000 7f703739d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f702c1adf00 con 0x55790509fe80 2024-04-21T08:45:53.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.920+0000 7f703739d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f702c1ab160 con 0x55790509f4d0 2024-04-21T08:45:53.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.920+0000 7f703739d740 10 client.0 _ll_register_callbacks cb 0x7f701c0bad20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:53.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.920+0000 7f7032c70640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579050a8a00 0x5579050b6500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.920+0000 7f7032c70640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579050a8a00 0x5579050b6500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38518/0 (socket says 172.21.15.86:38518) 2024-04-21T08:45:53.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.920+0000 7f7032c70640 1 -- 172.21.15.86:0/1468651492 learned_addr learned my addr 172.21.15.86:0/1468651492 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:53.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.920+0000 7f703246f640 1 --2- 172.21.15.86:0/1468651492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55790509f4d0 0x5579050b8fc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.920+0000 7f7031c6e640 1 --2- 172.21.15.86:0/1468651492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55790509fe80 0x5579050b9500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.920+0000 7f7032c70640 1 -- 172.21.15.86:0/1468651492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55790509f4d0 msgr2=0x5579050b8fc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:53.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.920+0000 7f7032c70640 1 --2- 172.21.15.86:0/1468651492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55790509f4d0 0x5579050b8fc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.920+0000 7f7032c70640 1 -- 172.21.15.86:0/1468651492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55790509fe80 msgr2=0x5579050b9500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:53.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.920+0000 7f7032c70640 1 --2- 172.21.15.86:0/1468651492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55790509fe80 0x5579050b9500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.920+0000 7f7032c70640 1 -- 172.21.15.86:0/1468651492 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579050b6d00 con 0x5579050a8a00 2024-04-21T08:45:53.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.921+0000 7f7032c70640 1 --2- 172.21.15.86:0/1468651492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579050a8a00 0x5579050b6500 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f7024005fd0 tx=0x7f7024004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e23d42d6bf077c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:53.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.921+0000 7f7015ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:53.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.921+0000 7f7015ffb640 1 -- 172.21.15.86:0/1468651492 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7024004b20 con 0x5579050a8a00 2024-04-21T08:45:53.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.921+0000 7f7015ffb640 1 -- 172.21.15.86:0/1468651492 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f7024019070 con 0x5579050a8a00 2024-04-21T08:45:53.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.921+0000 7f703739d740 1 -- 172.21.15.86:0/1468651492 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f701c0df0b0 con 0x5579050a8a00 2024-04-21T08:45:53.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.921+0000 7f7015ffb640 1 -- 172.21.15.86:0/1468651492 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7024034050 con 0x5579050a8a00 2024-04-21T08:45:53.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.921+0000 7f703739d740 10 client.18570 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:53.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.921+0000 7f703739d740 1 -- 172.21.15.86:0/1468651492 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557905122fe0 con 0x5579050a8a00 2024-04-21T08:45:53.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.921+0000 7f7015ffb640 1 -- 172.21.15.86:0/1468651492 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f701c0df0b0 con 0x5579050a8a00 2024-04-21T08:45:53.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.921+0000 7f7015ffb640 1 -- 172.21.15.86:0/1468651492 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f70240053f0 con 0x5579050a8a00 2024-04-21T08:45:53.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.921+0000 7f703739d740 10 client.18570 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:53.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.921+0000 7f703739d740 10 client.18570 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:45:53.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.921+0000 7f703739d740 20 client.18570 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:53.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.921+0000 7f703739d740 1 --2- 172.21.15.86:0/1468651492 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5579051237e0 0x557905125c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:53.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.921+0000 7f703739d740 4 client.18570 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:45:53.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.921+0000 7f703739d740 1 -- 172.21.15.86:0/1468651492 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557905011c10 con 0x5579051237e0 2024-04-21T08:45:53.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.921+0000 7f703246f640 1 --2- 172.21.15.86:0/1468651492 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5579051237e0 0x557905125c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:53.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.922+0000 7f703246f640 1 --2- 172.21.15.86:0/1468651492 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5579051237e0 0x557905125c00 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=df319ed968a4accd server_cookie=2450a96d4604690e in_seq=0 out_seq=0 2024-04-21T08:45:53.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.922+0000 7f7015ffb640 10 client.18570 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:45:53.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.922+0000 7f7015ffb640 1 -- 172.21.15.86:0/1468651492 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557905011c10 con 0x5579051237e0 2024-04-21T08:45:53.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.922+0000 7f7015ffb640 10 client.18570 handle_command_reply: tid=0 2024-04-21T08:45:53.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.922+0000 7f703739d740 1 client.18570 shutdown 2024-04-21T08:45:53.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.922+0000 7f703739d740 1 -- 172.21.15.86:0/1468651492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579050a8a00 msgr2=0x5579050b6500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.922+0000 7f703739d740 1 --2- 172.21.15.86:0/1468651492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579050a8a00 0x5579050b6500 secure :-1 s=READY pgs=1463 cs=0 l=1 rev1=1 crypto rx=0x7f7024005fd0 tx=0x7f7024004a40 comp rx=0 tx=0).stop 2024-04-21T08:45:53.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.922+0000 7f703739d740 1 -- 172.21.15.86:0/1468651492 shutdown_connections 2024-04-21T08:45:53.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.922+0000 7f703739d740 1 --2- 172.21.15.86:0/1468651492 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5579051237e0 0x557905125c00 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.922+0000 7f703739d740 1 --2- 172.21.15.86:0/1468651492 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579050a8a00 0x5579050b6500 unknown :-1 s=CLOSED pgs=1463 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.922+0000 7f703739d740 1 --2- 172.21.15.86:0/1468651492 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55790509fe80 0x5579050b9500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.922+0000 7f7015ffb640 0 client.18570 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:45:53.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.922+0000 7f703739d740 1 --2- 172.21.15.86:0/1468651492 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55790509f4d0 0x5579050b8fc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.922+0000 7f703739d740 1 -- 172.21.15.86:0/1468651492 >> 172.21.15.86:0/1468651492 conn(0x557905099ba0 msgr2=0x5579050acd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:53.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.922+0000 7f703739d740 1 -- 172.21.15.86:0/1468651492 shutdown_connections 2024-04-21T08:45:53.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.922+0000 7f703739d740 1 -- 172.21.15.86:0/1468651492 wait complete. 2024-04-21T08:45:53.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.922+0000 7f703739d740 20 client.18570 trim_cache size 0 max 16384 2024-04-21T08:45:53.928 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:45:53.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.923+0000 7f70346d3640 1 -- 172.21.15.86:0/2053760410 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f70180a3d20 msgr2=0x7f70180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.923+0000 7f70346d3640 1 --2- 172.21.15.86:0/2053760410 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f70180a3d20 0x7f70180a61e0 secure :-1 s=READY pgs=234 cs=0 l=1 rev1=1 crypto rx=0x7f7024005c40 tx=0x7f7024009280 comp rx=0 tx=0).stop 2024-04-21T08:45:53.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.923+0000 7f70346d3640 1 -- 172.21.15.86:0/2053760410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f702c103af0 msgr2=0x7f702c198c60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:53.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.923+0000 7f70346d3640 1 --2- 172.21.15.86:0/2053760410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f702c103af0 0x7f702c198c60 secure :-1 s=READY pgs=1459 cs=0 l=1 rev1=1 crypto rx=0x7f701c005940 tx=0x7f701c02d920 comp rx=0 tx=0).stop 2024-04-21T08:45:53.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.923+0000 7f70346d3640 1 -- 172.21.15.86:0/2053760410 shutdown_connections 2024-04-21T08:45:53.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.923+0000 7f70346d3640 1 --2- 172.21.15.86:0/2053760410 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f70180a3d20 0x7f70180a61e0 unknown :-1 s=CLOSED pgs=234 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.923+0000 7f70346d3640 1 --2- 172.21.15.86:0/2053760410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f702c10c510 0x7f702c19cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.923+0000 7f70346d3640 1 --2- 172.21.15.86:0/2053760410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f702c103af0 0x7f702c198c60 unknown :-1 s=CLOSED pgs=1459 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.923+0000 7f70346d3640 1 --2- 172.21.15.86:0/2053760410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f702c1031d0 0x7f702c198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:53.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.923+0000 7f70346d3640 1 -- 172.21.15.86:0/2053760410 >> 172.21.15.86:0/2053760410 conn(0x7f702c077f50 msgr2=0x7f702c0fdfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:53.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.923+0000 7f70346d3640 1 -- 172.21.15.86:0/2053760410 shutdown_connections 2024-04-21T08:45:53.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:53.923+0000 7f70346d3640 1 -- 172.21.15.86:0/2053760410 wait complete. 2024-04-21T08:45:53.932 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:45:54.035 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.033+0000 7f55424a5640 1 -- 172.21.15.86:0/2565599173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f553c06a550 msgr2=0x7f553c06a9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:54.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.033+0000 7f55424a5640 1 --2- 172.21.15.86:0/2565599173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f553c06a550 0x7f553c06a9d0 secure :-1 s=READY pgs=1464 cs=0 l=1 rev1=1 crypto rx=0x7f552c009930 tx=0x7f552c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:45:54.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.033+0000 7f55424a5640 1 -- 172.21.15.86:0/2565599173 shutdown_connections 2024-04-21T08:45:54.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.033+0000 7f55424a5640 1 --2- 172.21.15.86:0/2565599173 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f553c06af10 0x7f553c10ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.033+0000 7f55424a5640 1 --2- 172.21.15.86:0/2565599173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f553c06a550 0x7f553c06a9d0 unknown :-1 s=CLOSED pgs=1464 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.033+0000 7f55424a5640 1 --2- 172.21.15.86:0/2565599173 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f553c108000 0x7f553c1083e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.033+0000 7f55424a5640 1 -- 172.21.15.86:0/2565599173 >> 172.21.15.86:0/2565599173 conn(0x7f553c076c50 msgr2=0x7f553c077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:54.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.033+0000 7f55424a5640 1 -- 172.21.15.86:0/2565599173 shutdown_connections 2024-04-21T08:45:54.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.033+0000 7f55424a5640 1 -- 172.21.15.86:0/2565599173 wait complete. 2024-04-21T08:45:54.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.033+0000 7f55424a5640 1 Processor -- start 2024-04-21T08:45:54.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.033+0000 7f55424a5640 1 -- start start 2024-04-21T08:45:54.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.034+0000 7f55424a5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f553c06a550 0x7f553c198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.034+0000 7f55424a5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f553c06af10 0x7f553c198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.036 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.034+0000 7f55424a5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f553c108000 0x7f553c19d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.034+0000 7f55424a5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f553c112190 con 0x7f553c06a550 2024-04-21T08:45:54.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.034+0000 7f55424a5640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f553c112010 con 0x7f553c108000 2024-04-21T08:45:54.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.034+0000 7f55424a5640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f553c112310 con 0x7f553c06af10 2024-04-21T08:45:54.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.034+0000 7f553bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f553c06a550 0x7f553c198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.034+0000 7f553bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f553c06a550 0x7f553c198750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38528/0 (socket says 172.21.15.86:38528) 2024-04-21T08:45:54.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.034+0000 7f553bfff640 1 -- 172.21.15.86:0/3159291683 learned_addr learned my addr 172.21.15.86:0/3159291683 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:54.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.034+0000 7f553b7fe640 1 --2- 172.21.15.86:0/3159291683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f553c06af10 0x7f553c198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.034+0000 7f5540a42640 1 --2- 172.21.15.86:0/3159291683 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f553c108000 0x7f553c19d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.034+0000 7f553bfff640 1 -- 172.21.15.86:0/3159291683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f553c06af10 msgr2=0x7f553c198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:54.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.034+0000 7f553bfff640 1 --2- 172.21.15.86:0/3159291683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f553c06af10 0x7f553c198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.034+0000 7f553bfff640 1 -- 172.21.15.86:0/3159291683 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f553c108000 msgr2=0x7f553c19d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:54.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.034+0000 7f553bfff640 1 --2- 172.21.15.86:0/3159291683 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f553c108000 0x7f553c19d010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.034+0000 7f553bfff640 1 -- 172.21.15.86:0/3159291683 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f553c19d740 con 0x7f553c06a550 2024-04-21T08:45:54.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.034+0000 7f5540a42640 1 --2- 172.21.15.86:0/3159291683 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f553c108000 0x7f553c19d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:54.037 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.034+0000 7f553b7fe640 1 --2- 172.21.15.86:0/3159291683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f553c06af10 0x7f553c198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:54.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.034+0000 7f553bfff640 1 --2- 172.21.15.86:0/3159291683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f553c06a550 0x7f553c198750 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f553000e9f0 tx=0x7f553000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:54.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.034+0000 7f55397fa640 1 -- 172.21.15.86:0/3159291683 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f553000cd80 con 0x7f553c06a550 2024-04-21T08:45:54.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.035+0000 7f55424a5640 1 -- 172.21.15.86:0/3159291683 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f553c19da30 con 0x7f553c06a550 2024-04-21T08:45:54.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.035+0000 7f55397fa640 1 -- 172.21.15.86:0/3159291683 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f55300108e0 con 0x7f553c06a550 2024-04-21T08:45:54.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.035+0000 7f55397fa640 1 -- 172.21.15.86:0/3159291683 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5530018980 con 0x7f553c06a550 2024-04-21T08:45:54.038 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.035+0000 7f55424a5640 1 -- 172.21.15.86:0/3159291683 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f553c1a52c0 con 0x7f553c06a550 2024-04-21T08:45:54.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.037+0000 7f55397fa640 1 -- 172.21.15.86:0/3159291683 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5530010430 con 0x7f553c06a550 2024-04-21T08:45:54.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.037+0000 7f5545173740 1 Processor -- start 2024-04-21T08:45:54.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.037+0000 7f5545173740 1 -- start start 2024-04-21T08:45:54.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.037+0000 7f5545173740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5196b64d0 0x55a5196b68b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.037+0000 7f5545173740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5196b6e80 0x55a5196bf430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.037+0000 7f5545173740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5196bfa00 0x55a5196c1e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.037+0000 7f5545173740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a51961b1f0 con 0x55a5196b64d0 2024-04-21T08:45:54.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.037+0000 7f5545173740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55a5195b84b0 con 0x55a5196bfa00 2024-04-21T08:45:54.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.037+0000 7f5545173740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a5196171b0 con 0x55a5196b6e80 2024-04-21T08:45:54.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.038+0000 7f5540a42640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5196b64d0 0x55a5196b68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.038+0000 7f5540a42640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5196b64d0 0x55a5196b68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38530/0 (socket says 172.21.15.86:38530) 2024-04-21T08:45:54.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.038+0000 7f5540a42640 1 -- 172.21.15.86:0/45930146 learned_addr learned my addr 172.21.15.86:0/45930146 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:54.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.038+0000 7f55397fa640 1 --2- 172.21.15.86:0/3159291683 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f55240a3d20 0x7f55240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.038+0000 7f5540a42640 1 --2- 172.21.15.86:0/3159291683 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f55240a3d20 0x7f55240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.038+0000 7f553b7fe640 1 --2- 172.21.15.86:0/45930146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5196bfa00 0x55a5196c1e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.038+0000 7f553bfff640 1 --2- 172.21.15.86:0/45930146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5196b6e80 0x55a5196bf430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.038+0000 7f55397fa640 1 -- 172.21.15.86:0/3159291683 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f55300d0fe0 con 0x7f553c06a550 2024-04-21T08:45:54.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.038+0000 7f5540a42640 1 -- 172.21.15.86:0/45930146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5196b6e80 msgr2=0x55a5196bf430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:54.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.038+0000 7f5540a42640 1 --2- 172.21.15.86:0/45930146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5196b6e80 0x55a5196bf430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.038+0000 7f5540a42640 1 -- 172.21.15.86:0/45930146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5196bfa00 msgr2=0x55a5196c1e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:54.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.038+0000 7f5540a42640 1 --2- 172.21.15.86:0/45930146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5196bfa00 0x55a5196c1e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.038+0000 7f5540a42640 1 -- 172.21.15.86:0/45930146 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5195aa1b0 con 0x55a5196b64d0 2024-04-21T08:45:54.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.038+0000 7f553bfff640 1 --2- 172.21.15.86:0/45930146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5196b6e80 0x55a5196bf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:54.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.038+0000 7f5540a42640 1 --2- 172.21.15.86:0/45930146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5196b64d0 0x55a5196b68b0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f5534013950 tx=0x7f5534013e10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=155aeee33487cfd2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:54.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.038+0000 7f55227fc640 1 -- 172.21.15.86:0/45930146 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5534013420 con 0x55a5196b64d0 2024-04-21T08:45:54.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.038+0000 7f55227fc640 1 -- 172.21.15.86:0/45930146 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5534011c40 con 0x55a5196b64d0 2024-04-21T08:45:54.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.038+0000 7f55227fc640 1 -- 172.21.15.86:0/45930146 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5534005020 con 0x55a5196b64d0 2024-04-21T08:45:54.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.039+0000 7f5545173740 1 -- 172.21.15.86:0/45930146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5196b64d0 msgr2=0x55a5196b68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:54.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.039+0000 7f5545173740 1 --2- 172.21.15.86:0/45930146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5196b64d0 0x55a5196b68b0 secure :-1 s=READY pgs=1466 cs=0 l=1 rev1=1 crypto rx=0x7f5534013950 tx=0x7f5534013e10 comp rx=0 tx=0).stop 2024-04-21T08:45:54.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.039+0000 7f5540a42640 1 --2- 172.21.15.86:0/3159291683 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f55240a3d20 0x7f55240a61e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5534020000 tx=0x7f553401f000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:54.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.039+0000 7f5545173740 1 -- 172.21.15.86:0/45930146 shutdown_connections 2024-04-21T08:45:54.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.039+0000 7f5545173740 1 --2- 172.21.15.86:0/45930146 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5196bfa00 0x55a5196c1e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.039+0000 7f5545173740 1 --2- 172.21.15.86:0/45930146 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5196b6e80 0x55a5196bf430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.039+0000 7f5545173740 1 --2- 172.21.15.86:0/45930146 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5196b64d0 0x55a5196b68b0 unknown :-1 s=CLOSED pgs=1466 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.039+0000 7f5545173740 1 -- 172.21.15.86:0/45930146 >> 172.21.15.86:0/45930146 conn(0x55a5196b0ba0 msgr2=0x55a5196b0f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:54.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.039+0000 7f5545173740 1 -- 172.21.15.86:0/45930146 shutdown_connections 2024-04-21T08:45:54.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.039+0000 7f5545173740 1 -- 172.21.15.86:0/45930146 wait complete. 2024-04-21T08:45:54.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.039+0000 7f5545173740 1 -- start start 2024-04-21T08:45:54.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.040+0000 7f5545173740 1 Processor -- start 2024-04-21T08:45:54.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.040+0000 7f5545173740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5196b64d0 0x55a5196bf200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.040+0000 7f5545173740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5196b6e80 0x55a5196bc590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.040+0000 7f5545173740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5196bfa00 0x55a5196bcad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.040+0000 7f5545173740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a5196c2970 con 0x55a5196bfa00 2024-04-21T08:45:54.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.040+0000 7f5545173740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55a5195d4500 con 0x55a5196b64d0 2024-04-21T08:45:54.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.040+0000 7f5545173740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a5196c2af0 con 0x55a5196b6e80 2024-04-21T08:45:54.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.040+0000 7f5545173740 10 client.0 _ll_register_callbacks cb 0x55a5195a2b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:54.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.040+0000 7f5540a42640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5196bfa00 0x55a5196bcad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.040+0000 7f5540a42640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5196bfa00 0x55a5196bcad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38546/0 (socket says 172.21.15.86:38546) 2024-04-21T08:45:54.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.040+0000 7f5540a42640 1 -- 172.21.15.86:0/404415001 learned_addr learned my addr 172.21.15.86:0/404415001 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:54.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.040+0000 7f553bfff640 1 --2- 172.21.15.86:0/404415001 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5196b64d0 0x55a5196bf200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.040+0000 7f553b7fe640 1 --2- 172.21.15.86:0/404415001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5196b6e80 0x55a5196bc590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.040+0000 7f5540a42640 1 -- 172.21.15.86:0/404415001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5196b6e80 msgr2=0x55a5196bc590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:54.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.040+0000 7f5540a42640 1 --2- 172.21.15.86:0/404415001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5196b6e80 0x55a5196bc590 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.040+0000 7f5540a42640 1 -- 172.21.15.86:0/404415001 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5196b64d0 msgr2=0x55a5196bf200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:54.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.040+0000 7f5540a42640 1 --2- 172.21.15.86:0/404415001 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5196b64d0 0x55a5196bf200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.040+0000 7f5540a42640 1 -- 172.21.15.86:0/404415001 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a51932d3f0 con 0x55a5196bfa00 2024-04-21T08:45:54.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.040+0000 7f553bfff640 1 --2- 172.21.15.86:0/404415001 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5196b64d0 0x55a5196bf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:45:54.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.040+0000 7f5540a42640 1 --2- 172.21.15.86:0/404415001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5196bfa00 0x55a5196bcad0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f553401dbc0 tx=0x7f5534004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec7dd31e3f6b2563 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:54.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.040+0000 7f551b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:54.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.041+0000 7f551b7fe640 1 -- 172.21.15.86:0/404415001 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5534011df0 con 0x55a5196bfa00 2024-04-21T08:45:54.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.041+0000 7f551b7fe640 1 -- 172.21.15.86:0/404415001 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f553401d440 con 0x55a5196bfa00 2024-04-21T08:45:54.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.041+0000 7f5545173740 1 -- 172.21.15.86:0/404415001 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5534011c40 con 0x55a5196bfa00 2024-04-21T08:45:54.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.041+0000 7f551b7fe640 1 -- 172.21.15.86:0/404415001 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f553403c050 con 0x55a5196bfa00 2024-04-21T08:45:54.048 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.041+0000 7f5545173740 10 client.18582 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:54.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.041+0000 7f5545173740 1 -- 172.21.15.86:0/404415001 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a5193f77d0 con 0x55a5196bfa00 2024-04-21T08:45:54.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.041+0000 7f551b7fe640 1 -- 172.21.15.86:0/404415001 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5534011c40 con 0x55a5196bfa00 2024-04-21T08:45:54.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.041+0000 7f551b7fe640 1 -- 172.21.15.86:0/404415001 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5534037030 con 0x55a5196bfa00 2024-04-21T08:45:54.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.041+0000 7f5545173740 10 client.18582 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:54.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.041+0000 7f5545173740 10 client.18582 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:45:54.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.041+0000 7f5545173740 20 client.18582 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:54.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.041+0000 7f5545173740 1 --2- 172.21.15.86:0/404415001 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55a5196cfa20 0x55a5196d1e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.041+0000 7f5545173740 4 client.18582 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:54.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.041+0000 7f5545173740 1 -- 172.21.15.86:0/404415001 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5195ba930 con 0x55a5196cfa20 2024-04-21T08:45:54.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.041+0000 7f553bfff640 1 --2- 172.21.15.86:0/404415001 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55a5196cfa20 0x55a5196d1e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.042+0000 7f553bfff640 1 --2- 172.21.15.86:0/404415001 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55a5196cfa20 0x55a5196d1e10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=977addea29158c36 server_cookie=29d980a9e5b6dd5e in_seq=0 out_seq=0 2024-04-21T08:45:54.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.042+0000 7f551b7fe640 10 client.18582 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:45:54.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.043+0000 7f551b7fe640 1 -- 172.21.15.86:0/404415001 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a5195ba930 con 0x55a5196cfa20 2024-04-21T08:45:54.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.043+0000 7f551b7fe640 10 client.18582 handle_command_reply: tid=0 2024-04-21T08:45:54.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.043+0000 7f5545173740 1 client.18582 shutdown 2024-04-21T08:45:54.049 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.043+0000 7f5545173740 1 -- 172.21.15.86:0/404415001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5196bfa00 msgr2=0x55a5196bcad0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:54.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.043+0000 7f5545173740 1 --2- 172.21.15.86:0/404415001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5196bfa00 0x55a5196bcad0 secure :-1 s=READY pgs=1467 cs=0 l=1 rev1=1 crypto rx=0x7f553401dbc0 tx=0x7f5534004a60 comp rx=0 tx=0).stop 2024-04-21T08:45:54.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.043+0000 7f5545173740 1 -- 172.21.15.86:0/404415001 shutdown_connections 2024-04-21T08:45:54.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.043+0000 7f5545173740 1 --2- 172.21.15.86:0/404415001 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55a5196cfa20 0x55a5196d1e10 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.043+0000 7f5545173740 1 --2- 172.21.15.86:0/404415001 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5196bfa00 0x55a5196bcad0 unknown :-1 s=CLOSED pgs=1467 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.043+0000 7f5545173740 1 --2- 172.21.15.86:0/404415001 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5196b6e80 0x55a5196bc590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.043+0000 7f5545173740 1 --2- 172.21.15.86:0/404415001 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5196b64d0 0x55a5196bf200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.043+0000 7f551b7fe640 0 client.18582 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:45:54.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.043+0000 7f5545173740 1 -- 172.21.15.86:0/404415001 >> 172.21.15.86:0/404415001 conn(0x55a5196b0ba0 msgr2=0x55a5196b28f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:54.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.043+0000 7f5545173740 1 -- 172.21.15.86:0/404415001 shutdown_connections 2024-04-21T08:45:54.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.043+0000 7f5545173740 1 -- 172.21.15.86:0/404415001 wait complete. 2024-04-21T08:45:54.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.043+0000 7f5545173740 20 client.18582 trim_cache size 0 max 16384 2024-04-21T08:45:54.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.056+0000 7f5545173740 1 Processor -- start 2024-04-21T08:45:54.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.056+0000 7f5545173740 1 -- start start 2024-04-21T08:45:54.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.056+0000 7f5545173740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5196b64d0 0x55a5196b7260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.056+0000 7f5545173740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5196b6e80 0x55a5196c8150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.056+0000 7f5545173740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5196bfa00 0x55a5196bbd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.056+0000 7f5545173740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f553c1af550 con 0x55a5196b6e80 2024-04-21T08:45:54.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.056+0000 7f5545173740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f553c1b1090 con 0x55a5196b64d0 2024-04-21T08:45:54.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.056+0000 7f5545173740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f553c1afc20 con 0x55a5196bfa00 2024-04-21T08:45:54.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.057+0000 7f553b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5196b6e80 0x55a5196c8150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.057+0000 7f553b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5196b6e80 0x55a5196c8150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38548/0 (socket says 172.21.15.86:38548) 2024-04-21T08:45:54.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.057+0000 7f553b7fe640 1 -- 172.21.15.86:0/2267537697 learned_addr learned my addr 172.21.15.86:0/2267537697 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:54.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.057+0000 7f5540a42640 1 --2- 172.21.15.86:0/2267537697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5196bfa00 0x55a5196bbd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.057+0000 7f553bfff640 1 --2- 172.21.15.86:0/2267537697 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5196b64d0 0x55a5196b7260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.057+0000 7f553b7fe640 1 -- 172.21.15.86:0/2267537697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5196bfa00 msgr2=0x55a5196bbd40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:54.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.057+0000 7f553b7fe640 1 --2- 172.21.15.86:0/2267537697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5196bfa00 0x55a5196bbd40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.057+0000 7f553b7fe640 1 -- 172.21.15.86:0/2267537697 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5196b64d0 msgr2=0x55a5196b7260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:54.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.057+0000 7f553b7fe640 1 --2- 172.21.15.86:0/2267537697 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5196b64d0 0x55a5196b7260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.057+0000 7f553b7fe640 1 -- 172.21.15.86:0/2267537697 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5196b2e90 con 0x55a5196b6e80 2024-04-21T08:45:54.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.057+0000 7f553b7fe640 1 --2- 172.21.15.86:0/2267537697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5196b6e80 0x55a5196c8150 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f552c009930 tx=0x7f552c02c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86dec4f2fb2f9d21 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:54.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.057+0000 7f5521ffb640 1 -- 172.21.15.86:0/2267537697 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c0042c0 con 0x55a5196b6e80 2024-04-21T08:45:54.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.057+0000 7f5521ffb640 1 -- 172.21.15.86:0/2267537697 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f552c02cc90 con 0x55a5196b6e80 2024-04-21T08:45:54.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.057+0000 7f5521ffb640 1 -- 172.21.15.86:0/2267537697 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f552c004920 con 0x55a5196b6e80 2024-04-21T08:45:54.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.058+0000 7f5545173740 1 -- 172.21.15.86:0/2267537697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5196b6e80 msgr2=0x55a5196c8150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:54.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.058+0000 7f5545173740 1 --2- 172.21.15.86:0/2267537697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5196b6e80 0x55a5196c8150 secure :-1 s=READY pgs=1468 cs=0 l=1 rev1=1 crypto rx=0x7f552c009930 tx=0x7f552c02c5d0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.058+0000 7f5545173740 1 -- 172.21.15.86:0/2267537697 shutdown_connections 2024-04-21T08:45:54.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.058+0000 7f5545173740 1 --2- 172.21.15.86:0/2267537697 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5196bfa00 0x55a5196bbd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.058+0000 7f5545173740 1 --2- 172.21.15.86:0/2267537697 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5196b6e80 0x55a5196c8150 unknown :-1 s=CLOSED pgs=1468 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.058+0000 7f5545173740 1 --2- 172.21.15.86:0/2267537697 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5196b64d0 0x55a5196b7260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.058+0000 7f5545173740 1 -- 172.21.15.86:0/2267537697 >> 172.21.15.86:0/2267537697 conn(0x55a5196b0ba0 msgr2=0x55a5196b2780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:54.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.058+0000 7f5545173740 1 -- 172.21.15.86:0/2267537697 shutdown_connections 2024-04-21T08:45:54.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.058+0000 7f5545173740 1 -- 172.21.15.86:0/2267537697 wait complete. 2024-04-21T08:45:54.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.058+0000 7f5545173740 1 -- start start 2024-04-21T08:45:54.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.058+0000 7f5545173740 1 Processor -- start 2024-04-21T08:45:54.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.058+0000 7f5545173740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5196b64d0 0x55a51972dcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.058+0000 7f5545173740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5196b6e80 0x55a51972e210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.059+0000 7f5545173740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5196bfa00 0x55a51972e750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.059+0000 7f5545173740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f553c1b09c0 con 0x55a5196bfa00 2024-04-21T08:45:54.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.059+0000 7f5545173740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f553c1b02f0 con 0x55a5196b64d0 2024-04-21T08:45:54.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.059+0000 7f5545173740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f553c1ad550 con 0x55a5196b6e80 2024-04-21T08:45:54.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.059+0000 7f5545173740 10 client.0 _ll_register_callbacks cb 0x7f55340097f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:54.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.059+0000 7f5540a42640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5196bfa00 0x55a51972e750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.059+0000 7f5540a42640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5196bfa00 0x55a51972e750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38552/0 (socket says 172.21.15.86:38552) 2024-04-21T08:45:54.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.059+0000 7f5540a42640 1 -- 172.21.15.86:0/88870346 learned_addr learned my addr 172.21.15.86:0/88870346 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:54.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.059+0000 7f553bfff640 1 --2- 172.21.15.86:0/88870346 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5196b64d0 0x55a51972dcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.059+0000 7f553b7fe640 1 --2- 172.21.15.86:0/88870346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5196b6e80 0x55a51972e210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.059+0000 7f5540a42640 1 -- 172.21.15.86:0/88870346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5196b6e80 msgr2=0x55a51972e210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:54.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.059+0000 7f5540a42640 1 --2- 172.21.15.86:0/88870346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5196b6e80 0x55a51972e210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.059+0000 7f5540a42640 1 -- 172.21.15.86:0/88870346 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5196b64d0 msgr2=0x55a51972dcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:54.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.059+0000 7f5540a42640 1 --2- 172.21.15.86:0/88870346 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5196b64d0 0x55a51972dcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.059+0000 7f5540a42640 1 -- 172.21.15.86:0/88870346 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a519738e80 con 0x55a5196bfa00 2024-04-21T08:45:54.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.059+0000 7f5540a42640 1 --2- 172.21.15.86:0/88870346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5196bfa00 0x55a51972e750 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f553401db90 tx=0x7f5534026470 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7d90f22477d07518 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:54.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.059+0000 7f551b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:54.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.059+0000 7f551b7fe640 1 -- 172.21.15.86:0/88870346 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5534044020 con 0x55a5196bfa00 2024-04-21T08:45:54.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.059+0000 7f551b7fe640 1 -- 172.21.15.86:0/88870346 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f553401d440 con 0x55a5196bfa00 2024-04-21T08:45:54.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.059+0000 7f5545173740 1 -- 172.21.15.86:0/88870346 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f552c02cc90 con 0x55a5196bfa00 2024-04-21T08:45:54.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.059+0000 7f551b7fe640 1 -- 172.21.15.86:0/88870346 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f553404f050 con 0x55a5196bfa00 2024-04-21T08:45:54.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.059+0000 7f5545173740 10 client.18588 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:54.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.059+0000 7f5545173740 1 -- 172.21.15.86:0/88870346 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a51973a180 con 0x55a5196bfa00 2024-04-21T08:45:54.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.060+0000 7f551b7fe640 1 -- 172.21.15.86:0/88870346 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f552c02cc90 con 0x55a5196bfa00 2024-04-21T08:45:54.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.060+0000 7f551b7fe640 1 -- 172.21.15.86:0/88870346 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5534037050 con 0x55a5196bfa00 2024-04-21T08:45:54.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.060+0000 7f5545173740 10 client.18588 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:54.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.060+0000 7f5545173740 10 client.18588 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:45:54.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.060+0000 7f5545173740 20 client.18588 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:54.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.060+0000 7f5545173740 1 --2- 172.21.15.86:0/88870346 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55a51973a890 0x55a51973ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.060+0000 7f5545173740 4 client.18588 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:45:54.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.060+0000 7f5545173740 1 -- 172.21.15.86:0/88870346 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a519628c10 con 0x55a51973a890 2024-04-21T08:45:54.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.060+0000 7f553bfff640 1 --2- 172.21.15.86:0/88870346 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55a51973a890 0x55a51973ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.060+0000 7f551b7fe640 10 client.18588 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:45:54.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.060+0000 7f553bfff640 1 --2- 172.21.15.86:0/88870346 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55a51973a890 0x55a51973ccb0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=15b90b27f833097f server_cookie=d5ad22e5dceb3427 in_seq=0 out_seq=0 2024-04-21T08:45:54.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.061+0000 7f551b7fe640 1 -- 172.21.15.86:0/88870346 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a519628c10 con 0x55a51973a890 2024-04-21T08:45:54.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.061+0000 7f551b7fe640 10 client.18588 handle_command_reply: tid=0 2024-04-21T08:45:54.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.061+0000 7f5545173740 1 client.18588 shutdown 2024-04-21T08:45:54.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.061+0000 7f5545173740 1 -- 172.21.15.86:0/88870346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5196bfa00 msgr2=0x55a51972e750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:54.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.061+0000 7f5545173740 1 --2- 172.21.15.86:0/88870346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5196bfa00 0x55a51972e750 secure :-1 s=READY pgs=1469 cs=0 l=1 rev1=1 crypto rx=0x7f553401db90 tx=0x7f5534026470 comp rx=0 tx=0).stop 2024-04-21T08:45:54.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.061+0000 7f5545173740 1 -- 172.21.15.86:0/88870346 shutdown_connections 2024-04-21T08:45:54.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.061+0000 7f5545173740 1 --2- 172.21.15.86:0/88870346 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55a51973a890 0x55a51973ccb0 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.061+0000 7f5545173740 1 --2- 172.21.15.86:0/88870346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5196bfa00 0x55a51972e750 unknown :-1 s=CLOSED pgs=1469 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.061+0000 7f551b7fe640 0 client.18588 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:45:54.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.061+0000 7f5545173740 1 --2- 172.21.15.86:0/88870346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5196b6e80 0x55a51972e210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.061+0000 7f5545173740 1 --2- 172.21.15.86:0/88870346 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5196b64d0 0x55a51972dcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.061+0000 7f5545173740 1 -- 172.21.15.86:0/88870346 >> 172.21.15.86:0/88870346 conn(0x55a5196b0ba0 msgr2=0x55a5196af370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:54.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.061+0000 7f5545173740 1 -- 172.21.15.86:0/88870346 shutdown_connections 2024-04-21T08:45:54.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.061+0000 7f5545173740 1 -- 172.21.15.86:0/88870346 wait complete. 2024-04-21T08:45:54.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.061+0000 7f5545173740 20 client.18588 trim_cache size 0 max 16384 2024-04-21T08:45:54.066 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:45:54.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.061+0000 7f55424a5640 1 -- 172.21.15.86:0/3159291683 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f55240a3d20 msgr2=0x7f55240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:54.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.061+0000 7f55424a5640 1 --2- 172.21.15.86:0/3159291683 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f55240a3d20 0x7f55240a61e0 secure :-1 s=READY pgs=235 cs=0 l=1 rev1=1 crypto rx=0x7f5534020000 tx=0x7f553401f000 comp rx=0 tx=0).stop 2024-04-21T08:45:54.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.061+0000 7f55424a5640 1 -- 172.21.15.86:0/3159291683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f553c06a550 msgr2=0x7f553c198750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:54.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.061+0000 7f55424a5640 1 --2- 172.21.15.86:0/3159291683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f553c06a550 0x7f553c198750 secure :-1 s=READY pgs=1465 cs=0 l=1 rev1=1 crypto rx=0x7f553000e9f0 tx=0x7f553000eeb0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.061+0000 7f55424a5640 1 -- 172.21.15.86:0/3159291683 shutdown_connections 2024-04-21T08:45:54.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.061+0000 7f55424a5640 1 --2- 172.21.15.86:0/3159291683 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f55240a3d20 0x7f55240a61e0 unknown :-1 s=CLOSED pgs=235 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.061+0000 7f55424a5640 1 --2- 172.21.15.86:0/3159291683 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f553c108000 0x7f553c19d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.061+0000 7f55424a5640 1 --2- 172.21.15.86:0/3159291683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f553c06af10 0x7f553c198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.061+0000 7f55424a5640 1 --2- 172.21.15.86:0/3159291683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f553c06a550 0x7f553c198750 unknown :-1 s=CLOSED pgs=1465 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.061+0000 7f55424a5640 1 -- 172.21.15.86:0/3159291683 >> 172.21.15.86:0/3159291683 conn(0x7f553c076c50 msgr2=0x7f553c100050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:54.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.062+0000 7f55424a5640 1 -- 172.21.15.86:0/3159291683 shutdown_connections 2024-04-21T08:45:54.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.062+0000 7f55424a5640 1 -- 172.21.15.86:0/3159291683 wait complete. 2024-04-21T08:45:54.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:53 smithi086 ceph-mon[27939]: pgmap v218: 129 pgs: 129 active+clean; 3.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-21T08:45:54.070 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:53 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/96047811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:45:54.071 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:45:54.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.171+0000 7fe76ef15640 1 -- 172.21.15.86:0/353930242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe768107260 msgr2=0x7fe76810ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:54.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.171+0000 7fe76ef15640 1 --2- 172.21.15.86:0/353930242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe768107260 0x7fe76810ee30 secure :-1 s=READY pgs=1470 cs=0 l=1 rev1=1 crypto rx=0x7fe760009ca0 tx=0x7fe76002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.171+0000 7fe76ef15640 1 -- 172.21.15.86:0/353930242 shutdown_connections 2024-04-21T08:45:54.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.171+0000 7fe76ef15640 1 --2- 172.21.15.86:0/353930242 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe768107260 0x7fe76810ee30 unknown :-1 s=CLOSED pgs=1470 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.171+0000 7fe76ef15640 1 --2- 172.21.15.86:0/353930242 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe768102b40 0x7fe768106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.171+0000 7fe76ef15640 1 --2- 172.21.15.86:0/353930242 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe768102190 0x7fe768102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.171+0000 7fe76ef15640 1 -- 172.21.15.86:0/353930242 >> 172.21.15.86:0/353930242 conn(0x7fe7680793a0 msgr2=0x7fe7680797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:54.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.171+0000 7fe76ef15640 1 -- 172.21.15.86:0/353930242 shutdown_connections 2024-04-21T08:45:54.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.171+0000 7fe76ef15640 1 -- 172.21.15.86:0/353930242 wait complete. 2024-04-21T08:45:54.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.172+0000 7fe76ef15640 1 Processor -- start 2024-04-21T08:45:54.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.172+0000 7fe76ef15640 1 -- start start 2024-04-21T08:45:54.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.172+0000 7fe76ef15640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe768102190 0x7fe768198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.172+0000 7fe76ef15640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe768107260 0x7fe768198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.172+0000 7fe76ef15640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe76819cff0 0x7fe76819d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.172+0000 7fe76ef15640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe7681121d0 con 0x7fe768107260 2024-04-21T08:45:54.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.172+0000 7fe76ef15640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe768112050 con 0x7fe768102190 2024-04-21T08:45:54.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.172+0000 7fe76ef15640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe768112350 con 0x7fe76819cff0 2024-04-21T08:45:54.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.172+0000 7fe767fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe768107260 0x7fe768198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.172+0000 7fe767fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe768107260 0x7fe768198c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38576/0 (socket says 172.21.15.86:38576) 2024-04-21T08:45:54.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.172+0000 7fe767fff640 1 -- 172.21.15.86:0/540605013 learned_addr learned my addr 172.21.15.86:0/540605013 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:54.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.172+0000 7fe76ccb1640 1 --2- 172.21.15.86:0/540605013 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe768102190 0x7fe768198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.172+0000 7fe767fff640 1 -- 172.21.15.86:0/540605013 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe76819cff0 msgr2=0x7fe76819d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:54.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.172+0000 7fe76d4b2640 1 --2- 172.21.15.86:0/540605013 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe76819cff0 0x7fe76819d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.172+0000 7fe767fff640 1 --2- 172.21.15.86:0/540605013 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe76819cff0 0x7fe76819d4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.172+0000 7fe767fff640 1 -- 172.21.15.86:0/540605013 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe768102190 msgr2=0x7fe768198730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:54.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.172+0000 7fe767fff640 1 --2- 172.21.15.86:0/540605013 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe768102190 0x7fe768198730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.172+0000 7fe767fff640 1 -- 172.21.15.86:0/540605013 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe76819dae0 con 0x7fe768107260 2024-04-21T08:45:54.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.172+0000 7fe76ccb1640 1 --2- 172.21.15.86:0/540605013 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe768102190 0x7fe768198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:54.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.172+0000 7fe767fff640 1 --2- 172.21.15.86:0/540605013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe768107260 0x7fe768198c70 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fe75400d740 tx=0x7fe75400dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:54.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.172+0000 7fe765ffb640 1 -- 172.21.15.86:0/540605013 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe754017070 con 0x7fe768107260 2024-04-21T08:45:54.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.172+0000 7fe765ffb640 1 -- 172.21.15.86:0/540605013 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe75400de50 con 0x7fe768107260 2024-04-21T08:45:54.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.173+0000 7fe76ef15640 1 -- 172.21.15.86:0/540605013 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe76819ddd0 con 0x7fe768107260 2024-04-21T08:45:54.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.173+0000 7fe765ffb640 1 -- 172.21.15.86:0/540605013 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe754004ce0 con 0x7fe768107260 2024-04-21T08:45:54.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.173+0000 7fe76ef15640 1 -- 172.21.15.86:0/540605013 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe7681a5570 con 0x7fe768107260 2024-04-21T08:45:54.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.175+0000 7fe765ffb640 1 -- 172.21.15.86:0/540605013 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe754004e80 con 0x7fe768107260 2024-04-21T08:45:54.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.175+0000 7fe771bdd740 1 Processor -- start 2024-04-21T08:45:54.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.175+0000 7fe771bdd740 1 -- start start 2024-04-21T08:45:54.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.175+0000 7fe771bdd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5c20d34d0 0x55a5c20d38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.175+0000 7fe771bdd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c20d3e80 0x55a5c20dc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.175+0000 7fe771bdd740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5c20dca00 0x55a5c20dee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.175+0000 7fe771bdd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a5c20381f0 con 0x55a5c20d34d0 2024-04-21T08:45:54.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.175+0000 7fe765ffb640 1 --2- 172.21.15.86:0/540605013 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe7580a3d20 0x7fe7580a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.175+0000 7fe771bdd740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55a5c1fd54b0 con 0x55a5c20dca00 2024-04-21T08:45:54.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.175+0000 7fe771bdd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a5c20341b0 con 0x55a5c20d3e80 2024-04-21T08:45:54.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.175+0000 7fe76d4b2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5c20d34d0 0x55a5c20d38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.175+0000 7fe76d4b2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5c20d34d0 0x55a5c20d38b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38592/0 (socket says 172.21.15.86:38592) 2024-04-21T08:45:54.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.175+0000 7fe76d4b2640 1 -- 172.21.15.86:0/4155982116 learned_addr learned my addr 172.21.15.86:0/4155982116 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:54.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.176+0000 7fe76d4b2640 1 --2- 172.21.15.86:0/540605013 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe7580a3d20 0x7fe7580a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.176+0000 7fe765ffb640 1 -- 172.21.15.86:0/540605013 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7fe7540d05a0 con 0x7fe768107260 2024-04-21T08:45:54.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.176+0000 7fe76ccb1640 1 --2- 172.21.15.86:0/4155982116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c20d3e80 0x55a5c20dc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.176+0000 7fe767fff640 1 --2- 172.21.15.86:0/4155982116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5c20dca00 0x55a5c20dee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.176+0000 7fe76d4b2640 1 -- 172.21.15.86:0/4155982116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c20d3e80 msgr2=0x55a5c20dc430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:54.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.176+0000 7fe76d4b2640 1 --2- 172.21.15.86:0/4155982116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c20d3e80 0x55a5c20dc430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.176+0000 7fe76d4b2640 1 -- 172.21.15.86:0/4155982116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5c20dca00 msgr2=0x55a5c20dee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:54.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.176+0000 7fe76d4b2640 1 --2- 172.21.15.86:0/4155982116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5c20dca00 0x55a5c20dee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.176+0000 7fe76d4b2640 1 -- 172.21.15.86:0/4155982116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5c1fc71b0 con 0x55a5c20d34d0 2024-04-21T08:45:54.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.176+0000 7fe767fff640 1 --2- 172.21.15.86:0/4155982116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5c20dca00 0x55a5c20dee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:54.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.176+0000 7fe76d4b2640 1 --2- 172.21.15.86:0/4155982116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5c20d34d0 0x55a5c20d38b0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fe760009430 tx=0x7fe7600326f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e2ce57052649d47 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:54.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.176+0000 7fe76d4b2640 1 --2- 172.21.15.86:0/540605013 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe7580a3d20 0x7fe7580a61e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe76002c630 tx=0x7fe760040000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:54.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.176+0000 7fe752ffd640 1 -- 172.21.15.86:0/4155982116 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe760039040 con 0x55a5c20d34d0 2024-04-21T08:45:54.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.176+0000 7fe752ffd640 1 -- 172.21.15.86:0/4155982116 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe760050070 con 0x55a5c20d34d0 2024-04-21T08:45:54.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.176+0000 7fe752ffd640 1 -- 172.21.15.86:0/4155982116 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe760034440 con 0x55a5c20d34d0 2024-04-21T08:45:54.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.177+0000 7fe771bdd740 1 -- 172.21.15.86:0/4155982116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5c20d34d0 msgr2=0x55a5c20d38b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:54.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.177+0000 7fe771bdd740 1 --2- 172.21.15.86:0/4155982116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5c20d34d0 0x55a5c20d38b0 secure :-1 s=READY pgs=1472 cs=0 l=1 rev1=1 crypto rx=0x7fe760009430 tx=0x7fe7600326f0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.177+0000 7fe771bdd740 1 -- 172.21.15.86:0/4155982116 shutdown_connections 2024-04-21T08:45:54.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.177+0000 7fe771bdd740 1 --2- 172.21.15.86:0/4155982116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5c20dca00 0x55a5c20dee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.177+0000 7fe771bdd740 1 --2- 172.21.15.86:0/4155982116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c20d3e80 0x55a5c20dc430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.177+0000 7fe771bdd740 1 --2- 172.21.15.86:0/4155982116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5c20d34d0 0x55a5c20d38b0 unknown :-1 s=CLOSED pgs=1472 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.177+0000 7fe771bdd740 1 -- 172.21.15.86:0/4155982116 >> 172.21.15.86:0/4155982116 conn(0x55a5c20cdba0 msgr2=0x55a5c20cdf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:54.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.177+0000 7fe771bdd740 1 -- 172.21.15.86:0/4155982116 shutdown_connections 2024-04-21T08:45:54.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.177+0000 7fe771bdd740 1 -- 172.21.15.86:0/4155982116 wait complete. 2024-04-21T08:45:54.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.177+0000 7fe771bdd740 1 -- start start 2024-04-21T08:45:54.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.177+0000 7fe771bdd740 1 Processor -- start 2024-04-21T08:45:54.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.178+0000 7fe771bdd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5c20d34d0 0x55a5c20dc3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.178+0000 7fe771bdd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c20d3e80 0x55a5c20d9730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.178+0000 7fe771bdd740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5c20dca00 0x55a5c20d9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.178+0000 7fe771bdd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a5c20df910 con 0x55a5c20d34d0 2024-04-21T08:45:54.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.178+0000 7fe771bdd740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55a5c1ff1500 con 0x55a5c20dca00 2024-04-21T08:45:54.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.178+0000 7fe771bdd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a5c20dfa90 con 0x55a5c20d3e80 2024-04-21T08:45:54.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.178+0000 7fe771bdd740 10 client.0 _ll_register_callbacks cb 0x55a5c1fbfb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:54.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.178+0000 7fe76ccb1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5c20d34d0 0x55a5c20dc3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.178+0000 7fe76ccb1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5c20d34d0 0x55a5c20dc3a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38600/0 (socket says 172.21.15.86:38600) 2024-04-21T08:45:54.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.178+0000 7fe76ccb1640 1 -- 172.21.15.86:0/1138509918 learned_addr learned my addr 172.21.15.86:0/1138509918 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:54.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.178+0000 7fe767fff640 1 --2- 172.21.15.86:0/1138509918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c20d3e80 0x55a5c20d9730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.178+0000 7fe76d4b2640 1 --2- 172.21.15.86:0/1138509918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5c20dca00 0x55a5c20d9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.178+0000 7fe76ccb1640 1 -- 172.21.15.86:0/1138509918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c20d3e80 msgr2=0x55a5c20d9730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:54.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.178+0000 7fe76ccb1640 1 --2- 172.21.15.86:0/1138509918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c20d3e80 0x55a5c20d9730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.178+0000 7fe76ccb1640 1 -- 172.21.15.86:0/1138509918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5c20dca00 msgr2=0x55a5c20d9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:54.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.178+0000 7fe76ccb1640 1 --2- 172.21.15.86:0/1138509918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5c20dca00 0x55a5c20d9e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.178+0000 7fe76ccb1640 1 -- 172.21.15.86:0/1138509918 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5c1d4a3f0 con 0x55a5c20d34d0 2024-04-21T08:45:54.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.178+0000 7fe767fff640 1 --2- 172.21.15.86:0/1138509918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c20d3e80 0x55a5c20d9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:54.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.178+0000 7fe76ccb1640 1 --2- 172.21.15.86:0/1138509918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5c20d34d0 0x55a5c20dc3a0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00bf30 tx=0x7fe75c0128e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ef640b9364f6c99 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:54.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.178+0000 7fe747fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:54.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.178+0000 7fe747fff640 1 -- 172.21.15.86:0/1138509918 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe75c0035f0 con 0x55a5c20d34d0 2024-04-21T08:45:54.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.178+0000 7fe747fff640 1 -- 172.21.15.86:0/1138509918 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7fe75c003790 con 0x55a5c20d34d0 2024-04-21T08:45:54.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.178+0000 7fe771bdd740 1 -- 172.21.15.86:0/1138509918 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe760050070 con 0x55a5c20d34d0 2024-04-21T08:45:54.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.179+0000 7fe747fff640 1 -- 172.21.15.86:0/1138509918 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe75c003990 con 0x55a5c20d34d0 2024-04-21T08:45:54.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.179+0000 7fe771bdd740 10 client.18600 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:54.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.179+0000 7fe771bdd740 1 -- 172.21.15.86:0/1138509918 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a5c1e147d0 con 0x55a5c20d34d0 2024-04-21T08:45:54.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.179+0000 7fe747fff640 1 -- 172.21.15.86:0/1138509918 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe760050070 con 0x55a5c20d34d0 2024-04-21T08:45:54.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.179+0000 7fe747fff640 1 -- 172.21.15.86:0/1138509918 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe75c019070 con 0x55a5c20d34d0 2024-04-21T08:45:54.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.179+0000 7fe771bdd740 10 client.18600 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:54.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.179+0000 7fe771bdd740 10 client.18600 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:54.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.179+0000 7fe771bdd740 20 client.18600 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:54.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.179+0000 7fe771bdd740 1 --2- 172.21.15.86:0/1138509918 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a5c20ecc90 0x55a5c20ef120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.179+0000 7fe771bdd740 4 client.18600 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:54.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.179+0000 7fe771bdd740 1 -- 172.21.15.86:0/1138509918 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a5c1fd7930 con 0x55a5c20ecc90 2024-04-21T08:45:54.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.179+0000 7fe767fff640 1 --2- 172.21.15.86:0/1138509918 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a5c20ecc90 0x55a5c20ef120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.180+0000 7fe747fff640 10 client.18600 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:54.187 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.180+0000 7fe767fff640 1 --2- 172.21.15.86:0/1138509918 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a5c20ecc90 0x55a5c20ef120 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=882765c6047a1470 server_cookie=be107c21e04c17fa in_seq=0 out_seq=0 2024-04-21T08:45:54.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.180+0000 7fe747fff640 1 -- 172.21.15.86:0/1138509918 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a5c1fd7930 con 0x55a5c20ecc90 2024-04-21T08:45:54.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.180+0000 7fe747fff640 10 client.18600 handle_command_reply: tid=0 2024-04-21T08:45:54.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.180+0000 7fe771bdd740 1 client.18600 shutdown 2024-04-21T08:45:54.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.181+0000 7fe771bdd740 1 -- 172.21.15.86:0/1138509918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5c20d34d0 msgr2=0x55a5c20dc3a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:54.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.181+0000 7fe771bdd740 1 --2- 172.21.15.86:0/1138509918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5c20d34d0 0x55a5c20dc3a0 secure :-1 s=READY pgs=1473 cs=0 l=1 rev1=1 crypto rx=0x7fe75c00bf30 tx=0x7fe75c0128e0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.181+0000 7fe771bdd740 1 -- 172.21.15.86:0/1138509918 shutdown_connections 2024-04-21T08:45:54.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.181+0000 7fe771bdd740 1 --2- 172.21.15.86:0/1138509918 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a5c20ecc90 0x55a5c20ef120 crc :-1 s=READY pgs=219 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.181+0000 7fe771bdd740 1 --2- 172.21.15.86:0/1138509918 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5c20dca00 0x55a5c20d9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.181+0000 7fe747fff640 0 client.18600 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:54.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.181+0000 7fe771bdd740 1 --2- 172.21.15.86:0/1138509918 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c20d3e80 0x55a5c20d9730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.181+0000 7fe771bdd740 1 --2- 172.21.15.86:0/1138509918 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5c20d34d0 0x55a5c20dc3a0 unknown :-1 s=CLOSED pgs=1473 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.181+0000 7fe771bdd740 1 -- 172.21.15.86:0/1138509918 >> 172.21.15.86:0/1138509918 conn(0x55a5c20cdba0 msgr2=0x55a5c20cf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:54.188 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.181+0000 7fe771bdd740 1 -- 172.21.15.86:0/1138509918 shutdown_connections 2024-04-21T08:45:54.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.181+0000 7fe771bdd740 1 -- 172.21.15.86:0/1138509918 wait complete. 2024-04-21T08:45:54.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.181+0000 7fe771bdd740 20 client.18600 trim_cache size 0 max 16384 2024-04-21T08:45:54.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.194+0000 7fe771bdd740 1 Processor -- start 2024-04-21T08:45:54.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.194+0000 7fe771bdd740 1 -- start start 2024-04-21T08:45:54.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.194+0000 7fe771bdd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c20d34d0 0x55a5c20e87d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.194+0000 7fe771bdd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5c20d3e80 0x55a5c20e50c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.194+0000 7fe771bdd740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5c20dca00 0x55a5c20cf680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.194+0000 7fe771bdd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe7681af2e0 con 0x55a5c20d3e80 2024-04-21T08:45:54.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.194+0000 7fe771bdd740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe7681b0e20 con 0x55a5c20dca00 2024-04-21T08:45:54.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.194+0000 7fe771bdd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe7681af9b0 con 0x55a5c20d34d0 2024-04-21T08:45:54.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.194+0000 7fe767fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5c20d3e80 0x55a5c20e50c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.194+0000 7fe767fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5c20d3e80 0x55a5c20e50c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38610/0 (socket says 172.21.15.86:38610) 2024-04-21T08:45:54.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.194+0000 7fe767fff640 1 -- 172.21.15.86:0/3986970119 learned_addr learned my addr 172.21.15.86:0/3986970119 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:54.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.194+0000 7fe76d4b2640 1 --2- 172.21.15.86:0/3986970119 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5c20dca00 0x55a5c20cf680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.194+0000 7fe767fff640 1 -- 172.21.15.86:0/3986970119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c20d34d0 msgr2=0x55a5c20e87d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:54.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.194+0000 7fe76ccb1640 1 --2- 172.21.15.86:0/3986970119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c20d34d0 0x55a5c20e87d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.194+0000 7fe767fff640 1 --2- 172.21.15.86:0/3986970119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c20d34d0 0x55a5c20e87d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.194+0000 7fe767fff640 1 -- 172.21.15.86:0/3986970119 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5c20dca00 msgr2=0x55a5c20cf680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:54.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.194+0000 7fe767fff640 1 --2- 172.21.15.86:0/3986970119 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5c20dca00 0x55a5c20cf680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.194+0000 7fe767fff640 1 -- 172.21.15.86:0/3986970119 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a5c20dfa30 con 0x55a5c20d3e80 2024-04-21T08:45:54.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.194+0000 7fe76d4b2640 1 --2- 172.21.15.86:0/3986970119 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5c20dca00 0x55a5c20cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:54.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.194+0000 7fe767fff640 1 --2- 172.21.15.86:0/3986970119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5c20d3e80 0x55a5c20e50c0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fe7540cd980 tx=0x7fe75401d040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75fe053d981dbd8d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:54.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.194+0000 7fe7527fc640 1 -- 172.21.15.86:0/3986970119 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe754020040 con 0x55a5c20d3e80 2024-04-21T08:45:54.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.195+0000 7fe7527fc640 1 -- 172.21.15.86:0/3986970119 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe75409a210 con 0x55a5c20d3e80 2024-04-21T08:45:54.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.195+0000 7fe7527fc640 1 -- 172.21.15.86:0/3986970119 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe7540a53b0 con 0x55a5c20d3e80 2024-04-21T08:45:54.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.195+0000 7fe771bdd740 1 -- 172.21.15.86:0/3986970119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5c20d3e80 msgr2=0x55a5c20e50c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:54.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.195+0000 7fe771bdd740 1 --2- 172.21.15.86:0/3986970119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5c20d3e80 0x55a5c20e50c0 secure :-1 s=READY pgs=1474 cs=0 l=1 rev1=1 crypto rx=0x7fe7540cd980 tx=0x7fe75401d040 comp rx=0 tx=0).stop 2024-04-21T08:45:54.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.195+0000 7fe771bdd740 1 -- 172.21.15.86:0/3986970119 shutdown_connections 2024-04-21T08:45:54.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.195+0000 7fe771bdd740 1 --2- 172.21.15.86:0/3986970119 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5c20dca00 0x55a5c20cf680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.195+0000 7fe771bdd740 1 --2- 172.21.15.86:0/3986970119 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5c20d3e80 0x55a5c20e50c0 unknown :-1 s=CLOSED pgs=1474 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.195+0000 7fe771bdd740 1 --2- 172.21.15.86:0/3986970119 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c20d34d0 0x55a5c20e87d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.195+0000 7fe771bdd740 1 -- 172.21.15.86:0/3986970119 >> 172.21.15.86:0/3986970119 conn(0x55a5c20cdba0 msgr2=0x55a5c20cef70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:54.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.195+0000 7fe771bdd740 1 -- 172.21.15.86:0/3986970119 shutdown_connections 2024-04-21T08:45:54.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.195+0000 7fe771bdd740 1 -- 172.21.15.86:0/3986970119 wait complete. 2024-04-21T08:45:54.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.195+0000 7fe771bdd740 1 -- start start 2024-04-21T08:45:54.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.195+0000 7fe771bdd740 1 Processor -- start 2024-04-21T08:45:54.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.196+0000 7fe771bdd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c20d34d0 0x55a5c214b410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.196+0000 7fe771bdd740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5c20dca00 0x55a5c214b950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.196+0000 7fe771bdd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5c21502e0 0x55a5c2150760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.196+0000 7fe771bdd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe7681b0750 con 0x55a5c21502e0 2024-04-21T08:45:54.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.196+0000 7fe771bdd740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe7681b0080 con 0x55a5c20dca00 2024-04-21T08:45:54.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.196+0000 7fe771bdd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe7681ad2e0 con 0x55a5c20d34d0 2024-04-21T08:45:54.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.196+0000 7fe771bdd740 10 client.0 _ll_register_callbacks cb 0x7fe75c00bb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:54.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.196+0000 7fe76d4b2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5c21502e0 0x55a5c2150760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.196+0000 7fe76d4b2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5c21502e0 0x55a5c2150760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38616/0 (socket says 172.21.15.86:38616) 2024-04-21T08:45:54.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.196+0000 7fe76d4b2640 1 -- 172.21.15.86:0/743449948 learned_addr learned my addr 172.21.15.86:0/743449948 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:54.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.196+0000 7fe76ccb1640 1 --2- 172.21.15.86:0/743449948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c20d34d0 0x55a5c214b410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.196+0000 7fe767fff640 1 --2- 172.21.15.86:0/743449948 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5c20dca00 0x55a5c214b950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.196+0000 7fe76d4b2640 1 -- 172.21.15.86:0/743449948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c20d34d0 msgr2=0x55a5c214b410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:54.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.196+0000 7fe76d4b2640 1 --2- 172.21.15.86:0/743449948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c20d34d0 0x55a5c214b410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.196+0000 7fe76d4b2640 1 -- 172.21.15.86:0/743449948 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5c20dca00 msgr2=0x55a5c214b950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:54.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.196+0000 7fe76d4b2640 1 --2- 172.21.15.86:0/743449948 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5c20dca00 0x55a5c214b950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.196+0000 7fe76d4b2640 1 -- 172.21.15.86:0/743449948 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a5c2150ca0 con 0x55a5c21502e0 2024-04-21T08:45:54.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.196+0000 7fe76ccb1640 1 --2- 172.21.15.86:0/743449948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c20d34d0 0x55a5c214b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:54.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.196+0000 7fe76d4b2640 1 --2- 172.21.15.86:0/743449948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5c21502e0 0x55a5c2150760 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fe760005e00 tx=0x7fe760032810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5147022d54a5439f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:54.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.196+0000 7fe747fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:54.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.196+0000 7fe747fff640 1 -- 172.21.15.86:0/743449948 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe760039040 con 0x55a5c21502e0 2024-04-21T08:45:54.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.196+0000 7fe747fff640 1 -- 172.21.15.86:0/743449948 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7fe7600346f0 con 0x55a5c21502e0 2024-04-21T08:45:54.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.196+0000 7fe771bdd740 1 -- 172.21.15.86:0/743449948 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe75409a210 con 0x55a5c21502e0 2024-04-21T08:45:54.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.197+0000 7fe747fff640 1 -- 172.21.15.86:0/743449948 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe76005f050 con 0x55a5c21502e0 2024-04-21T08:45:54.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.197+0000 7fe771bdd740 10 client.18606 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:54.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.197+0000 7fe771bdd740 1 -- 172.21.15.86:0/743449948 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a5c2157f90 con 0x55a5c21502e0 2024-04-21T08:45:54.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.197+0000 7fe747fff640 1 -- 172.21.15.86:0/743449948 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe75409a210 con 0x55a5c21502e0 2024-04-21T08:45:54.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.197+0000 7fe747fff640 1 -- 172.21.15.86:0/743449948 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe760004540 con 0x55a5c21502e0 2024-04-21T08:45:54.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.197+0000 7fe771bdd740 10 client.18606 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:54.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.197+0000 7fe771bdd740 10 client.18606 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:54.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.197+0000 7fe771bdd740 20 client.18606 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:54.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.197+0000 7fe771bdd740 1 --2- 172.21.15.86:0/743449948 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a5c21582c0 0x55a5c215a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.197+0000 7fe771bdd740 4 client.18606 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:45:54.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.197+0000 7fe771bdd740 1 -- 172.21.15.86:0/743449948 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55a5c2045c10 con 0x55a5c21582c0 2024-04-21T08:45:54.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.197+0000 7fe76ccb1640 1 --2- 172.21.15.86:0/743449948 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a5c21582c0 0x55a5c215a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.197+0000 7fe747fff640 10 client.18606 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:54.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.197+0000 7fe76ccb1640 1 --2- 172.21.15.86:0/743449948 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a5c21582c0 0x55a5c215a6b0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6953f4f3689f1d2c server_cookie=fe29dd85e4797194 in_seq=0 out_seq=0 2024-04-21T08:45:54.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.198+0000 7fe747fff640 1 -- 172.21.15.86:0/743449948 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55a5c2045c10 con 0x55a5c21582c0 2024-04-21T08:45:54.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.198+0000 7fe747fff640 10 client.18606 handle_command_reply: tid=0 2024-04-21T08:45:54.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.198+0000 7fe771bdd740 1 client.18606 shutdown 2024-04-21T08:45:54.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.198+0000 7fe771bdd740 1 -- 172.21.15.86:0/743449948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5c21502e0 msgr2=0x55a5c2150760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:54.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.198+0000 7fe771bdd740 1 --2- 172.21.15.86:0/743449948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5c21502e0 0x55a5c2150760 secure :-1 s=READY pgs=1475 cs=0 l=1 rev1=1 crypto rx=0x7fe760005e00 tx=0x7fe760032810 comp rx=0 tx=0).stop 2024-04-21T08:45:54.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.198+0000 7fe771bdd740 1 -- 172.21.15.86:0/743449948 shutdown_connections 2024-04-21T08:45:54.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.198+0000 7fe771bdd740 1 --2- 172.21.15.86:0/743449948 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a5c21582c0 0x55a5c215a6b0 crc :-1 s=READY pgs=220 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.198+0000 7fe771bdd740 1 --2- 172.21.15.86:0/743449948 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a5c21502e0 0x55a5c2150760 unknown :-1 s=CLOSED pgs=1475 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.198+0000 7fe771bdd740 1 --2- 172.21.15.86:0/743449948 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a5c20dca00 0x55a5c214b950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.198+0000 7fe747fff640 0 client.18606 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:54.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.198+0000 7fe771bdd740 1 --2- 172.21.15.86:0/743449948 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a5c20d34d0 0x55a5c214b410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.198+0000 7fe771bdd740 1 -- 172.21.15.86:0/743449948 >> 172.21.15.86:0/743449948 conn(0x55a5c20cdba0 msgr2=0x55a5c20d4c40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:54.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.198+0000 7fe771bdd740 1 -- 172.21.15.86:0/743449948 shutdown_connections 2024-04-21T08:45:54.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.198+0000 7fe771bdd740 1 -- 172.21.15.86:0/743449948 wait complete. 2024-04-21T08:45:54.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.198+0000 7fe771bdd740 20 client.18606 trim_cache size 0 max 16384 2024-04-21T08:45:54.204 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:45:54.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.199+0000 7fe76ef15640 1 -- 172.21.15.86:0/540605013 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe7580a3d20 msgr2=0x7fe7580a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:54.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.199+0000 7fe76ef15640 1 --2- 172.21.15.86:0/540605013 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe7580a3d20 0x7fe7580a61e0 secure :-1 s=READY pgs=236 cs=0 l=1 rev1=1 crypto rx=0x7fe76002c630 tx=0x7fe760040000 comp rx=0 tx=0).stop 2024-04-21T08:45:54.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.199+0000 7fe76ef15640 1 -- 172.21.15.86:0/540605013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe768107260 msgr2=0x7fe768198c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:54.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.199+0000 7fe76ef15640 1 --2- 172.21.15.86:0/540605013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe768107260 0x7fe768198c70 secure :-1 s=READY pgs=1471 cs=0 l=1 rev1=1 crypto rx=0x7fe75400d740 tx=0x7fe75400dc00 comp rx=0 tx=0).stop 2024-04-21T08:45:54.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.199+0000 7fe76ef15640 1 -- 172.21.15.86:0/540605013 shutdown_connections 2024-04-21T08:45:54.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.199+0000 7fe76ef15640 1 --2- 172.21.15.86:0/540605013 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe7580a3d20 0x7fe7580a61e0 unknown :-1 s=CLOSED pgs=236 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.199+0000 7fe76ef15640 1 --2- 172.21.15.86:0/540605013 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe76819cff0 0x7fe76819d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.199+0000 7fe76ef15640 1 --2- 172.21.15.86:0/540605013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe768107260 0x7fe768198c70 unknown :-1 s=CLOSED pgs=1471 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.199+0000 7fe76ef15640 1 --2- 172.21.15.86:0/540605013 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe768102190 0x7fe768198730 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.199+0000 7fe76ef15640 1 -- 172.21.15.86:0/540605013 >> 172.21.15.86:0/540605013 conn(0x7fe7680793a0 msgr2=0x7fe7680ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:54.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.199+0000 7fe76ef15640 1 -- 172.21.15.86:0/540605013 shutdown_connections 2024-04-21T08:45:54.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:54.199+0000 7fe76ef15640 1 -- 172.21.15.86:0/540605013 wait complete. 2024-04-21T08:45:54.213 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:53 smithi088 ceph-mon[31259]: pgmap v218: 129 pgs: 129 active+clean; 3.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-21T08:45:54.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:53 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/96047811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:45:54.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:53 smithi179 ceph-mon[31829]: pgmap v218: 129 pgs: 129 active+clean; 3.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-21T08:45:54.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:53 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/96047811' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:45:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.404+0000 7ff7a9504640 1 -- 172.21.15.86:0/1173731906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7a4106170 msgr2=0x7ff7a410dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.404+0000 7ff7a9504640 1 --2- 172.21.15.86:0/1173731906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7a4106170 0x7ff7a410dd40 secure :-1 s=READY pgs=1476 cs=0 l=1 rev1=1 crypto rx=0x7ff79c009900 tx=0x7ff79c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:45:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.404+0000 7ff7a9504640 1 -- 172.21.15.86:0/1173731906 shutdown_connections 2024-04-21T08:45:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.404+0000 7ff7a9504640 1 --2- 172.21.15.86:0/1173731906 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7a4106170 0x7ff7a410dd40 unknown :-1 s=CLOSED pgs=1476 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.404+0000 7ff7a9504640 1 --2- 172.21.15.86:0/1173731906 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7a4101aa0 0x7ff7a4105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.406 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.404+0000 7ff7a9504640 1 --2- 172.21.15.86:0/1173731906 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7a41010f0 0x7ff7a41014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.404+0000 7ff7a9504640 1 -- 172.21.15.86:0/1173731906 >> 172.21.15.86:0/1173731906 conn(0x7ff7a4077fd0 msgr2=0x7ff7a40783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.404+0000 7ff7a9504640 1 -- 172.21.15.86:0/1173731906 shutdown_connections 2024-04-21T08:45:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.404+0000 7ff7a9504640 1 -- 172.21.15.86:0/1173731906 wait complete. 2024-04-21T08:45:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.405+0000 7ff7a9504640 1 Processor -- start 2024-04-21T08:45:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.405+0000 7ff7a9504640 1 -- start start 2024-04-21T08:45:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.405+0000 7ff7a9504640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7a41010f0 0x7ff7a4197540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.405+0000 7ff7a9504640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7a4106170 0x7ff7a4197a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.405+0000 7ff7a9504640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7a419bdb0 0x7ff7a419c260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.405+0000 7ff7a9504640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a41110e0 con 0x7ff7a4106170 2024-04-21T08:45:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.405+0000 7ff7a9504640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a4110f60 con 0x7ff7a41010f0 2024-04-21T08:45:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.405+0000 7ff7a9504640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a4111260 con 0x7ff7a419bdb0 2024-04-21T08:45:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.405+0000 7ff7a27fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7a4106170 0x7ff7a4197a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.407 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.405+0000 7ff7a27fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7a4106170 0x7ff7a4197a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38622/0 (socket says 172.21.15.86:38622) 2024-04-21T08:45:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.406+0000 7ff7a27fc640 1 -- 172.21.15.86:0/3205140422 learned_addr learned my addr 172.21.15.86:0/3205140422 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.406+0000 7ff7a37fe640 1 --2- 172.21.15.86:0/3205140422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7a419bdb0 0x7ff7a419c260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.406+0000 7ff7a2ffd640 1 --2- 172.21.15.86:0/3205140422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7a41010f0 0x7ff7a4197540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.406+0000 7ff7a27fc640 1 -- 172.21.15.86:0/3205140422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7a419bdb0 msgr2=0x7ff7a419c260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.406+0000 7ff7a27fc640 1 --2- 172.21.15.86:0/3205140422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7a419bdb0 0x7ff7a419c260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.406+0000 7ff7a27fc640 1 -- 172.21.15.86:0/3205140422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7a41010f0 msgr2=0x7ff7a4197540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.406+0000 7ff7a27fc640 1 --2- 172.21.15.86:0/3205140422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7a41010f0 0x7ff7a4197540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.406+0000 7ff7a27fc640 1 -- 172.21.15.86:0/3205140422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff7a419c8a0 con 0x7ff7a4106170 2024-04-21T08:45:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.406+0000 7ff7a2ffd640 1 --2- 172.21.15.86:0/3205140422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7a41010f0 0x7ff7a4197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.406+0000 7ff7a27fc640 1 --2- 172.21.15.86:0/3205140422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7a4106170 0x7ff7a4197a80 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7ff79800c970 tx=0x7ff79800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.406+0000 7ff78bfff640 1 -- 172.21.15.86:0/3205140422 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff798014070 con 0x7ff7a4106170 2024-04-21T08:45:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.406+0000 7ff78bfff640 1 -- 172.21.15.86:0/3205140422 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff798004440 con 0x7ff7a4106170 2024-04-21T08:45:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.406+0000 7ff7a9504640 1 -- 172.21.15.86:0/3205140422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff7a419cb90 con 0x7ff7a4106170 2024-04-21T08:45:54.408 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.406+0000 7ff78bfff640 1 -- 172.21.15.86:0/3205140422 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff798003e50 con 0x7ff7a4106170 2024-04-21T08:45:54.409 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.406+0000 7ff7a9504640 1 -- 172.21.15.86:0/3205140422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7a41a4380 con 0x7ff7a4106170 2024-04-21T08:45:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.408+0000 7ff78bfff640 1 -- 172.21.15.86:0/3205140422 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff798017070 con 0x7ff7a4106170 2024-04-21T08:45:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.409+0000 7ff7ac1d2740 1 Processor -- start 2024-04-21T08:45:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.409+0000 7ff7ac1d2740 1 -- start start 2024-04-21T08:45:54.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.409+0000 7ff78bfff640 1 --2- 172.21.15.86:0/3205140422 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff78c0a3f60 0x7ff78c0a6420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.409+0000 7ff7ac1d2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564287fa08d0 0x564287f9f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.409+0000 7ff7ac1d2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564287f9fb60 0x564287f9ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.409+0000 7ff7ac1d2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564287fab2f0 0x564287fad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.409+0000 7ff7ac1d2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564287f048d0 con 0x564287fab2f0 2024-04-21T08:45:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.409+0000 7ff7ac1d2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564287ea1e80 con 0x564287f9fb60 2024-04-21T08:45:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.409+0000 7ff7ac1d2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564287f00890 con 0x564287fa08d0 2024-04-21T08:45:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.409+0000 7ff7a37fe640 1 --2- 172.21.15.86:0/3205140422 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff78c0a3f60 0x7ff78c0a6420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.409+0000 7ff78bfff640 1 -- 172.21.15.86:0/3205140422 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7ff7980d06d0 con 0x7ff7a4106170 2024-04-21T08:45:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.409+0000 7ff7a2ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564287fa08d0 0x564287f9f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.409+0000 7ff7a37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564287fab2f0 0x564287fad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.409+0000 7ff7a2ffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564287fa08d0 0x564287f9f620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.86:51656/0 (socket says 172.21.15.86:51656) 2024-04-21T08:45:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.409+0000 7ff7a2ffd640 1 -- 172.21.15.86:0/1869028847 learned_addr learned my addr 172.21.15.86:0/1869028847 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:54.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.409+0000 7ff7a27fc640 1 --2- 172.21.15.86:0/1869028847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564287f9fb60 0x564287f9ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.409+0000 7ff7a2ffd640 1 -- 172.21.15.86:0/1869028847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564287f9fb60 msgr2=0x564287f9ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.409+0000 7ff7a2ffd640 1 --2- 172.21.15.86:0/1869028847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564287f9fb60 0x564287f9ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.409+0000 7ff7a2ffd640 1 -- 172.21.15.86:0/1869028847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564287fab2f0 msgr2=0x564287fad6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.409+0000 7ff7a2ffd640 1 --2- 172.21.15.86:0/1869028847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564287fab2f0 0x564287fad6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.409+0000 7ff7a2ffd640 1 -- 172.21.15.86:0/1869028847 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564287ec3320 con 0x564287fa08d0 2024-04-21T08:45:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.409+0000 7ff7a37fe640 1 --2- 172.21.15.86:0/1869028847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564287fab2f0 0x564287fad6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:45:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.409+0000 7ff7a37fe640 1 --2- 172.21.15.86:0/3205140422 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff78c0a3f60 0x7ff78c0a6420 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff79c005ab0 tx=0x7ff79c02c700 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.410+0000 7ff7a2ffd640 1 --2- 172.21.15.86:0/1869028847 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564287fa08d0 0x564287f9f620 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff7940103a0 tx=0x7ff794010860 comp rx=0 tx=0).ready entity=mon.1 client_cookie=22622e302e049071 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.410+0000 7ff7897fa640 1 -- 172.21.15.86:0/1869028847 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff794016070 con 0x564287fa08d0 2024-04-21T08:45:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.410+0000 7ff7897fa640 1 -- 172.21.15.86:0/1869028847 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff794012bc0 con 0x564287fa08d0 2024-04-21T08:45:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.410+0000 7ff7897fa640 1 -- 172.21.15.86:0/1869028847 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff794011b90 con 0x564287fa08d0 2024-04-21T08:45:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.410+0000 7ff7ac1d2740 1 -- 172.21.15.86:0/1869028847 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564287fa08d0 msgr2=0x564287f9f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.410+0000 7ff7ac1d2740 1 --2- 172.21.15.86:0/1869028847 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564287fa08d0 0x564287f9f620 secure :-1 s=READY pgs=68 cs=0 l=1 rev1=1 crypto rx=0x7ff7940103a0 tx=0x7ff794010860 comp rx=0 tx=0).stop 2024-04-21T08:45:54.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.411+0000 7ff7ac1d2740 1 -- 172.21.15.86:0/1869028847 shutdown_connections 2024-04-21T08:45:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.411+0000 7ff7ac1d2740 1 --2- 172.21.15.86:0/1869028847 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564287fab2f0 0x564287fad6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.411+0000 7ff7ac1d2740 1 --2- 172.21.15.86:0/1869028847 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564287f9fb60 0x564287f9ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.411+0000 7ff7ac1d2740 1 --2- 172.21.15.86:0/1869028847 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564287fa08d0 0x564287f9f620 unknown :-1 s=CLOSED pgs=68 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.411+0000 7ff7ac1d2740 1 -- 172.21.15.86:0/1869028847 >> 172.21.15.86:0/1869028847 conn(0x564287f66c90 msgr2=0x564287f9a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.411+0000 7ff7ac1d2740 1 -- 172.21.15.86:0/1869028847 shutdown_connections 2024-04-21T08:45:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.411+0000 7ff7ac1d2740 1 -- 172.21.15.86:0/1869028847 wait complete. 2024-04-21T08:45:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.411+0000 7ff7ac1d2740 1 -- start start 2024-04-21T08:45:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.411+0000 7ff7ac1d2740 1 Processor -- start 2024-04-21T08:45:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.411+0000 7ff7ac1d2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564287f9fb60 0x564287faaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.411+0000 7ff7ac1d2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564287fa08d0 0x564287fa8030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.411+0000 7ff7ac1d2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564287fab2f0 0x564287fa85d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.411+0000 7ff7ac1d2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564287fae020 con 0x564287fab2f0 2024-04-21T08:45:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.411+0000 7ff7ac1d2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564287e8ceb0 con 0x564287f9fb60 2024-04-21T08:45:54.418 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.411+0000 7ff7ac1d2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564287fae1a0 con 0x564287fa08d0 2024-04-21T08:45:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.411+0000 7ff7ac1d2740 10 client.0 _ll_register_callbacks cb 0x564287eb9b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.411+0000 7ff7a37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564287fab2f0 0x564287fa85d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.411+0000 7ff7a37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564287fab2f0 0x564287fa85d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38646/0 (socket says 172.21.15.86:38646) 2024-04-21T08:45:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.411+0000 7ff7a37fe640 1 -- 172.21.15.86:0/1279072475 learned_addr learned my addr 172.21.15.86:0/1279072475 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.411+0000 7ff7a2ffd640 1 --2- 172.21.15.86:0/1279072475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564287f9fb60 0x564287faaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.411+0000 7ff7a27fc640 1 --2- 172.21.15.86:0/1279072475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564287fa08d0 0x564287fa8030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.412+0000 7ff7a37fe640 1 -- 172.21.15.86:0/1279072475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564287fa08d0 msgr2=0x564287fa8030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.412+0000 7ff7a37fe640 1 --2- 172.21.15.86:0/1279072475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564287fa08d0 0x564287fa8030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.412+0000 7ff7a37fe640 1 -- 172.21.15.86:0/1279072475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564287f9fb60 msgr2=0x564287faaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.412+0000 7ff7a37fe640 1 --2- 172.21.15.86:0/1279072475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564287f9fb60 0x564287faaca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.412+0000 7ff7a37fe640 1 -- 172.21.15.86:0/1279072475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564287c17210 con 0x564287fab2f0 2024-04-21T08:45:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.412+0000 7ff7a37fe640 1 --2- 172.21.15.86:0/1279072475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564287fab2f0 0x564287fa85d0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7ff79c009900 tx=0x7ff79c0058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3ca720188dbcd3be server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.412+0000 7ff786ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:54.419 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.412+0000 7ff786ff5640 1 -- 172.21.15.86:0/1279072475 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79c004d20 con 0x564287fab2f0 2024-04-21T08:45:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.412+0000 7ff786ff5640 1 -- 172.21.15.86:0/1279072475 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7ff79c0035f0 con 0x564287fab2f0 2024-04-21T08:45:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.412+0000 7ff7ac1d2740 1 -- 172.21.15.86:0/1279072475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff794012bc0 con 0x564287fab2f0 2024-04-21T08:45:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.412+0000 7ff786ff5640 1 -- 172.21.15.86:0/1279072475 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff79c0037f0 con 0x564287fab2f0 2024-04-21T08:45:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.412+0000 7ff7ac1d2740 10 client.18618 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.412+0000 7ff7ac1d2740 1 -- 172.21.15.86:0/1279072475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564287ce3620 con 0x564287fab2f0 2024-04-21T08:45:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.412+0000 7ff786ff5640 1 -- 172.21.15.86:0/1279072475 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff794012bc0 con 0x564287fab2f0 2024-04-21T08:45:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.412+0000 7ff786ff5640 1 -- 172.21.15.86:0/1279072475 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff79c043750 con 0x564287fab2f0 2024-04-21T08:45:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.412+0000 7ff7ac1d2740 10 client.18618 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.412+0000 7ff7ac1d2740 10 client.18618 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.412+0000 7ff7ac1d2740 20 client.18618 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.412+0000 7ff7ac1d2740 1 --2- 172.21.15.86:0/1279072475 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564287fbb300 0x564287fbd710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.412+0000 7ff7ac1d2740 4 client.18618 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.412+0000 7ff7ac1d2740 1 -- 172.21.15.86:0/1279072475 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564287f18880 con 0x564287fbb300 2024-04-21T08:45:54.420 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.413+0000 7ff7a2ffd640 1 --2- 172.21.15.86:0/1279072475 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564287fbb300 0x564287fbd710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.413+0000 7ff786ff5640 10 client.18618 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.413+0000 7ff7a2ffd640 1 --2- 172.21.15.86:0/1279072475 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564287fbb300 0x564287fbd710 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=70b721be4b35f86e server_cookie=411b2279347d2b5b in_seq=0 out_seq=0 2024-04-21T08:45:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.414+0000 7ff786ff5640 1 -- 172.21.15.86:0/1279072475 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564287f18880 con 0x564287fbb300 2024-04-21T08:45:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.414+0000 7ff786ff5640 10 client.18618 handle_command_reply: tid=0 2024-04-21T08:45:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.414+0000 7ff7ac1d2740 1 client.18618 shutdown 2024-04-21T08:45:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.414+0000 7ff7ac1d2740 1 -- 172.21.15.86:0/1279072475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564287fab2f0 msgr2=0x564287fa85d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.414+0000 7ff7ac1d2740 1 --2- 172.21.15.86:0/1279072475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564287fab2f0 0x564287fa85d0 secure :-1 s=READY pgs=1478 cs=0 l=1 rev1=1 crypto rx=0x7ff79c009900 tx=0x7ff79c0058d0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.414+0000 7ff7ac1d2740 1 -- 172.21.15.86:0/1279072475 shutdown_connections 2024-04-21T08:45:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.414+0000 7ff7ac1d2740 1 --2- 172.21.15.86:0/1279072475 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564287fbb300 0x564287fbd710 crc :-1 s=READY pgs=221 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.414+0000 7ff7ac1d2740 1 --2- 172.21.15.86:0/1279072475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564287fab2f0 0x564287fa85d0 unknown :-1 s=CLOSED pgs=1478 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.414+0000 7ff7ac1d2740 1 --2- 172.21.15.86:0/1279072475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564287fa08d0 0x564287fa8030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.414+0000 7ff786ff5640 0 client.18618 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.414+0000 7ff7ac1d2740 1 --2- 172.21.15.86:0/1279072475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564287f9fb60 0x564287faaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.414+0000 7ff7ac1d2740 1 -- 172.21.15.86:0/1279072475 >> 172.21.15.86:0/1279072475 conn(0x564287f66c90 msgr2=0x564287f9c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:54.421 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.414+0000 7ff7ac1d2740 1 -- 172.21.15.86:0/1279072475 shutdown_connections 2024-04-21T08:45:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.414+0000 7ff7ac1d2740 1 -- 172.21.15.86:0/1279072475 wait complete. 2024-04-21T08:45:54.422 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.414+0000 7ff7ac1d2740 20 client.18618 trim_cache size 0 max 16384 2024-04-21T08:45:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.428+0000 7ff7ac1d2740 1 Processor -- start 2024-04-21T08:45:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.428+0000 7ff7ac1d2740 1 -- start start 2024-04-21T08:45:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.428+0000 7ff7ac1d2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564287f9fb60 0x564287fb6f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.432 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.429+0000 7ff7ac1d2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564287fa08d0 0x564287faa760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.429+0000 7ff7ac1d2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564287fab2f0 0x564287faaca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.429+0000 7ff7ac1d2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a41ad120 con 0x564287f9fb60 2024-04-21T08:45:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.429+0000 7ff7ac1d2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a41aec60 con 0x564287fa08d0 2024-04-21T08:45:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.429+0000 7ff7ac1d2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a41ad7f0 con 0x564287fab2f0 2024-04-21T08:45:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.429+0000 7ff7a2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564287f9fb60 0x564287fb6f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.429+0000 7ff7a2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564287f9fb60 0x564287fb6f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38650/0 (socket says 172.21.15.86:38650) 2024-04-21T08:45:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.429+0000 7ff7a2ffd640 1 -- 172.21.15.86:0/534924763 learned_addr learned my addr 172.21.15.86:0/534924763 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.429+0000 7ff7a27fc640 1 --2- 172.21.15.86:0/534924763 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564287fa08d0 0x564287faa760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.429+0000 7ff7a2ffd640 1 -- 172.21.15.86:0/534924763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564287fab2f0 msgr2=0x564287faaca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.429+0000 7ff7a2ffd640 1 --2- 172.21.15.86:0/534924763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564287fab2f0 0x564287faaca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.429+0000 7ff7a2ffd640 1 -- 172.21.15.86:0/534924763 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564287fa08d0 msgr2=0x564287faa760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.429+0000 7ff7a2ffd640 1 --2- 172.21.15.86:0/534924763 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564287fa08d0 0x564287faa760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.433 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.429+0000 7ff7a2ffd640 1 -- 172.21.15.86:0/534924763 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564287f9ee30 con 0x564287f9fb60 2024-04-21T08:45:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.429+0000 7ff7a2ffd640 1 --2- 172.21.15.86:0/534924763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564287f9fb60 0x564287fb6f20 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7ff794010370 tx=0x7ff794005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8104d163c28cc125 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.429+0000 7ff788ff9640 1 -- 172.21.15.86:0/534924763 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff79402e070 con 0x564287f9fb60 2024-04-21T08:45:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.429+0000 7ff788ff9640 1 -- 172.21.15.86:0/534924763 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff794011d00 con 0x564287f9fb60 2024-04-21T08:45:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.429+0000 7ff788ff9640 1 -- 172.21.15.86:0/534924763 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff794016040 con 0x564287f9fb60 2024-04-21T08:45:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.430+0000 7ff7ac1d2740 1 -- 172.21.15.86:0/534924763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564287f9fb60 msgr2=0x564287fb6f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.430+0000 7ff7ac1d2740 1 --2- 172.21.15.86:0/534924763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564287f9fb60 0x564287fb6f20 secure :-1 s=READY pgs=1479 cs=0 l=1 rev1=1 crypto rx=0x7ff794010370 tx=0x7ff794005bd0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.430+0000 7ff7ac1d2740 1 -- 172.21.15.86:0/534924763 shutdown_connections 2024-04-21T08:45:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.430+0000 7ff7ac1d2740 1 --2- 172.21.15.86:0/534924763 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564287fab2f0 0x564287faaca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.430+0000 7ff7ac1d2740 1 --2- 172.21.15.86:0/534924763 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564287fa08d0 0x564287faa760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.430+0000 7ff7ac1d2740 1 --2- 172.21.15.86:0/534924763 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564287f9fb60 0x564287fb6f20 unknown :-1 s=CLOSED pgs=1479 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.430+0000 7ff7ac1d2740 1 -- 172.21.15.86:0/534924763 >> 172.21.15.86:0/534924763 conn(0x564287f66c90 msgr2=0x564287fa6900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.430+0000 7ff7ac1d2740 1 -- 172.21.15.86:0/534924763 shutdown_connections 2024-04-21T08:45:54.434 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.430+0000 7ff7ac1d2740 1 -- 172.21.15.86:0/534924763 wait complete. 2024-04-21T08:45:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.430+0000 7ff7ac1d2740 1 -- start start 2024-04-21T08:45:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.430+0000 7ff7ac1d2740 1 Processor -- start 2024-04-21T08:45:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.430+0000 7ff7ac1d2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564287f9fb60 0x564287fb3740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.430+0000 7ff7ac1d2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564287fa08d0 0x564287faeab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.430+0000 7ff7ac1d2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564287fab2f0 0x564287faeff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.430+0000 7ff7ac1d2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a41ae590 con 0x564287f9fb60 2024-04-21T08:45:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.430+0000 7ff7ac1d2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a41adec0 con 0x564287fab2f0 2024-04-21T08:45:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.430+0000 7ff7ac1d2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff7a41ab120 con 0x564287fa08d0 2024-04-21T08:45:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.430+0000 7ff7ac1d2740 10 client.0 _ll_register_callbacks cb 0x7ff79c005420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.430+0000 7ff7a2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564287f9fb60 0x564287fb3740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.430+0000 7ff7a2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564287f9fb60 0x564287fb3740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38664/0 (socket says 172.21.15.86:38664) 2024-04-21T08:45:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.430+0000 7ff7a2ffd640 1 -- 172.21.15.86:0/2354219519 learned_addr learned my addr 172.21.15.86:0/2354219519 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.430+0000 7ff7a27fc640 1 --2- 172.21.15.86:0/2354219519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564287fa08d0 0x564287faeab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.431+0000 7ff7a37fe640 1 --2- 172.21.15.86:0/2354219519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564287fab2f0 0x564287faeff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.435 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.431+0000 7ff7a2ffd640 1 -- 172.21.15.86:0/2354219519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564287fa08d0 msgr2=0x564287faeab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.431+0000 7ff7a2ffd640 1 --2- 172.21.15.86:0/2354219519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564287fa08d0 0x564287faeab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.431+0000 7ff7a2ffd640 1 -- 172.21.15.86:0/2354219519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564287fab2f0 msgr2=0x564287faeff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.431+0000 7ff7a2ffd640 1 --2- 172.21.15.86:0/2354219519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564287fab2f0 0x564287faeff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.431+0000 7ff7a2ffd640 1 -- 172.21.15.86:0/2354219519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564287faf7f0 con 0x564287f9fb60 2024-04-21T08:45:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.431+0000 7ff7a37fe640 1 --2- 172.21.15.86:0/2354219519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564287fab2f0 0x564287faeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.431+0000 7ff7a2ffd640 1 --2- 172.21.15.86:0/2354219519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564287f9fb60 0x564287fb3740 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7ff794004fd0 tx=0x7ff794009cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eebf44fee39c35be server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.431+0000 7ff786ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.431+0000 7ff786ff5640 1 -- 172.21.15.86:0/2354219519 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff794037060 con 0x564287f9fb60 2024-04-21T08:45:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.431+0000 7ff786ff5640 1 -- 172.21.15.86:0/2354219519 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7ff7940117b0 con 0x564287f9fb60 2024-04-21T08:45:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.431+0000 7ff7ac1d2740 1 -- 172.21.15.86:0/2354219519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff794011d00 con 0x564287f9fb60 2024-04-21T08:45:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.431+0000 7ff786ff5640 1 -- 172.21.15.86:0/2354219519 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff79403e050 con 0x564287f9fb60 2024-04-21T08:45:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.431+0000 7ff7ac1d2740 10 client.18624 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.431+0000 7ff7ac1d2740 1 -- 172.21.15.86:0/2354219519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5642880217f0 con 0x564287f9fb60 2024-04-21T08:45:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.431+0000 7ff786ff5640 1 -- 172.21.15.86:0/2354219519 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff794011d00 con 0x564287f9fb60 2024-04-21T08:45:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.431+0000 7ff786ff5640 1 -- 172.21.15.86:0/2354219519 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff79400f040 con 0x564287f9fb60 2024-04-21T08:45:54.436 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.431+0000 7ff7ac1d2740 10 client.18624 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.431+0000 7ff7ac1d2740 10 client.18624 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.431+0000 7ff7ac1d2740 20 client.18624 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.431+0000 7ff7ac1d2740 1 --2- 172.21.15.86:0/2354219519 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564288021cb0 0x5642880240a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.431+0000 7ff7ac1d2740 4 client.18624 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:45:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.431+0000 7ff7ac1d2740 1 -- 172.21.15.86:0/2354219519 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x564287ea43d0 con 0x564288021cb0 2024-04-21T08:45:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.432+0000 7ff7a27fc640 1 --2- 172.21.15.86:0/2354219519 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564288021cb0 0x5642880240a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.432+0000 7ff786ff5640 10 client.18624 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:54.437 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.432+0000 7ff7a27fc640 1 --2- 172.21.15.86:0/2354219519 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564288021cb0 0x5642880240a0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5dc11b276ab876af server_cookie=5da8237e9cc5a661 in_seq=0 out_seq=0 2024-04-21T08:45:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.448+0000 7ff786ff5640 1 -- 172.21.15.86:0/2354219519 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x564287ea43d0 con 0x564288021cb0 2024-04-21T08:45:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.448+0000 7ff786ff5640 10 client.18624 handle_command_reply: tid=0 2024-04-21T08:45:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.448+0000 7ff7ac1d2740 1 client.18624 shutdown 2024-04-21T08:45:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.448+0000 7ff7ac1d2740 1 -- 172.21.15.86:0/2354219519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564287f9fb60 msgr2=0x564287fb3740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.448+0000 7ff7ac1d2740 1 --2- 172.21.15.86:0/2354219519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564287f9fb60 0x564287fb3740 secure :-1 s=READY pgs=1480 cs=0 l=1 rev1=1 crypto rx=0x7ff794004fd0 tx=0x7ff794009cc0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.448+0000 7ff7ac1d2740 1 -- 172.21.15.86:0/2354219519 shutdown_connections 2024-04-21T08:45:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.448+0000 7ff7ac1d2740 1 --2- 172.21.15.86:0/2354219519 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564288021cb0 0x5642880240a0 crc :-1 s=READY pgs=222 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.448+0000 7ff7ac1d2740 1 --2- 172.21.15.86:0/2354219519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564287fab2f0 0x564287faeff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.448+0000 7ff7ac1d2740 1 --2- 172.21.15.86:0/2354219519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564287fa08d0 0x564287faeab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.448+0000 7ff7ac1d2740 1 --2- 172.21.15.86:0/2354219519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564287f9fb60 0x564287fb3740 unknown :-1 s=CLOSED pgs=1480 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.448+0000 7ff786ff5640 0 client.18624 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:54.448 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.448+0000 7ff7ac1d2740 1 -- 172.21.15.86:0/2354219519 >> 172.21.15.86:0/2354219519 conn(0x564287f66c90 msgr2=0x564287fa6d60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.448+0000 7ff7ac1d2740 1 -- 172.21.15.86:0/2354219519 shutdown_connections 2024-04-21T08:45:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.448+0000 7ff7ac1d2740 1 -- 172.21.15.86:0/2354219519 wait complete. 2024-04-21T08:45:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.448+0000 7ff7ac1d2740 20 client.18624 trim_cache size 0 max 16384 2024-04-21T08:45:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:45:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:45:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:45:54.449 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:45:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.448+0000 7ff7a9504640 1 -- 172.21.15.86:0/3205140422 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff78c0a3f60 msgr2=0x7ff78c0a6420 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.448+0000 7ff7a9504640 1 --2- 172.21.15.86:0/3205140422 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff78c0a3f60 0x7ff78c0a6420 secure :-1 s=READY pgs=237 cs=0 l=1 rev1=1 crypto rx=0x7ff79c005ab0 tx=0x7ff79c02c700 comp rx=0 tx=0).stop 2024-04-21T08:45:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.448+0000 7ff7a9504640 1 -- 172.21.15.86:0/3205140422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7a4106170 msgr2=0x7ff7a4197a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.448+0000 7ff7a9504640 1 --2- 172.21.15.86:0/3205140422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7a4106170 0x7ff7a4197a80 secure :-1 s=READY pgs=1477 cs=0 l=1 rev1=1 crypto rx=0x7ff79800c970 tx=0x7ff79800ce30 comp rx=0 tx=0).stop 2024-04-21T08:45:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.448+0000 7ff7a9504640 1 -- 172.21.15.86:0/3205140422 shutdown_connections 2024-04-21T08:45:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.448+0000 7ff7a9504640 1 --2- 172.21.15.86:0/3205140422 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff78c0a3f60 0x7ff78c0a6420 unknown :-1 s=CLOSED pgs=237 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.448+0000 7ff7a9504640 1 --2- 172.21.15.86:0/3205140422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff7a419bdb0 0x7ff7a419c260 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.448+0000 7ff7a9504640 1 --2- 172.21.15.86:0/3205140422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff7a4106170 0x7ff7a4197a80 unknown :-1 s=CLOSED pgs=1477 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.450 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.448+0000 7ff7a9504640 1 --2- 172.21.15.86:0/3205140422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7a41010f0 0x7ff7a4197540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.448+0000 7ff7a9504640 1 -- 172.21.15.86:0/3205140422 >> 172.21.15.86:0/3205140422 conn(0x7ff7a4077fd0 msgr2=0x7ff7a40fed10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.449+0000 7ff7a9504640 1 -- 172.21.15.86:0/3205140422 shutdown_connections 2024-04-21T08:45:54.451 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:45:54.449+0000 7ff7a9504640 1 -- 172.21.15.86:0/3205140422 wait complete. 2024-04-21T08:45:55.209 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:45:55.209 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:45:55.209 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:45:55.319 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.317+0000 7fde8fdb2640 1 -- 172.21.15.86:0/417540064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde88074b40 msgr2=0x7fde88112ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:55.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.317+0000 7fde8fdb2640 1 --2- 172.21.15.86:0/417540064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde88074b40 0x7fde88112ac0 secure :-1 s=READY pgs=1481 cs=0 l=1 rev1=1 crypto rx=0x7fde78009930 tx=0x7fde7802bcd0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.318+0000 7fde8fdb2640 1 -- 172.21.15.86:0/417540064 shutdown_connections 2024-04-21T08:45:55.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.318+0000 7fde8fdb2640 1 --2- 172.21.15.86:0/417540064 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde88113000 0x7fde881153f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.318+0000 7fde8fdb2640 1 --2- 172.21.15.86:0/417540064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde88074b40 0x7fde88112ac0 unknown :-1 s=CLOSED pgs=1481 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.318+0000 7fde8fdb2640 1 --2- 172.21.15.86:0/417540064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde88074220 0x7fde88074600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.318+0000 7fde8fdb2640 1 -- 172.21.15.86:0/417540064 >> 172.21.15.86:0/417540064 conn(0x7fde880793a0 msgr2=0x7fde880797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:55.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.318+0000 7fde8fdb2640 1 -- 172.21.15.86:0/417540064 shutdown_connections 2024-04-21T08:45:55.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.318+0000 7fde8fdb2640 1 -- 172.21.15.86:0/417540064 wait complete. 2024-04-21T08:45:55.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.318+0000 7fde8fdb2640 1 Processor -- start 2024-04-21T08:45:55.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.318+0000 7fde8fdb2640 1 -- start start 2024-04-21T08:45:55.320 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.318+0000 7fde8fdb2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde88074220 0x7fde8819ee00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.318+0000 7fde8fdb2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde88074b40 0x7fde8819f340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.318+0000 7fde8fdb2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde88113000 0x7fde881a36c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.318+0000 7fde8fdb2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fde881187e0 con 0x7fde88113000 2024-04-21T08:45:55.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.318+0000 7fde8fdb2640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fde88118660 con 0x7fde88074220 2024-04-21T08:45:55.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.318+0000 7fde8fdb2640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fde88118960 con 0x7fde88074b40 2024-04-21T08:45:55.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.318+0000 7fde8e34f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde88113000 0x7fde881a36c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:55.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.318+0000 7fde8e34f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde88113000 0x7fde881a36c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38682/0 (socket says 172.21.15.86:38682) 2024-04-21T08:45:55.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.318+0000 7fde8e34f640 1 -- 172.21.15.86:0/1782043989 learned_addr learned my addr 172.21.15.86:0/1782043989 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:55.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.319+0000 7fde8d34d640 1 --2- 172.21.15.86:0/1782043989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde88074b40 0x7fde8819f340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:55.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.319+0000 7fde8db4e640 1 --2- 172.21.15.86:0/1782043989 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde88074220 0x7fde8819ee00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:55.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.319+0000 7fde8e34f640 1 -- 172.21.15.86:0/1782043989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde88074b40 msgr2=0x7fde8819f340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:55.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.319+0000 7fde8e34f640 1 --2- 172.21.15.86:0/1782043989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde88074b40 0x7fde8819f340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.321 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.319+0000 7fde8e34f640 1 -- 172.21.15.86:0/1782043989 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde88074220 msgr2=0x7fde8819ee00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:55.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.319+0000 7fde8e34f640 1 --2- 172.21.15.86:0/1782043989 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde88074220 0x7fde8819ee00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.319+0000 7fde8e34f640 1 -- 172.21.15.86:0/1782043989 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde881a3df0 con 0x7fde88113000 2024-04-21T08:45:55.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.319+0000 7fde8d34d640 1 --2- 172.21.15.86:0/1782043989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde88074b40 0x7fde8819f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:55.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.319+0000 7fde8e34f640 1 --2- 172.21.15.86:0/1782043989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde88113000 0x7fde881a36c0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fde8000eb40 tx=0x7fde8000c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:55.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.319+0000 7fde7effd640 1 -- 172.21.15.86:0/1782043989 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde80004b20 con 0x7fde88113000 2024-04-21T08:45:55.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.319+0000 7fde7effd640 1 -- 172.21.15.86:0/1782043989 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde80004cc0 con 0x7fde88113000 2024-04-21T08:45:55.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.319+0000 7fde8fdb2640 1 -- 172.21.15.86:0/1782043989 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde881a40e0 con 0x7fde88113000 2024-04-21T08:45:55.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.319+0000 7fde7effd640 1 -- 172.21.15.86:0/1782043989 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde80010720 con 0x7fde88113000 2024-04-21T08:45:55.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.319+0000 7fde8fdb2640 1 -- 172.21.15.86:0/1782043989 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde881ab970 con 0x7fde88113000 2024-04-21T08:45:55.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.322+0000 7fde7effd640 1 -- 172.21.15.86:0/1782043989 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fde80005560 con 0x7fde88113000 2024-04-21T08:45:55.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.322+0000 7fde929a2740 1 Processor -- start 2024-04-21T08:45:55.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.322+0000 7fde929a2740 1 -- start start 2024-04-21T08:45:55.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.322+0000 7fde929a2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f70e248e0 0x559f70e24cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.322+0000 7fde929a2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559f70e25290 0x559f70e2d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.323+0000 7fde929a2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f70e2de10 0x559f70e30260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.323+0000 7fde929a2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559f70d88f00 con 0x559f70e248e0 2024-04-21T08:45:55.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.323+0000 7fde929a2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559f70d26590 con 0x559f70e25290 2024-04-21T08:45:55.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.323+0000 7fde929a2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559f70d84ec0 con 0x559f70e2de10 2024-04-21T08:45:55.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.323+0000 7fde8d34d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f70e248e0 0x559f70e24cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:55.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.323+0000 7fde8d34d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f70e248e0 0x559f70e24cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38696/0 (socket says 172.21.15.86:38696) 2024-04-21T08:45:55.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.323+0000 7fde8d34d640 1 -- 172.21.15.86:0/2185710374 learned_addr learned my addr 172.21.15.86:0/2185710374 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:55.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.323+0000 7fde7effd640 1 --2- 172.21.15.86:0/1782043989 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fde740a3d20 0x7fde740a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.323+0000 7fde8db4e640 1 --2- 172.21.15.86:0/2185710374 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559f70e25290 0x559f70e2d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:55.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.323+0000 7fde8d34d640 1 --2- 172.21.15.86:0/2185710374 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f70e2de10 0x559f70e30260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:55.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.323+0000 7fde8e34f640 1 --2- 172.21.15.86:0/1782043989 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fde740a3d20 0x7fde740a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:55.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.323+0000 7fde8d34d640 1 -- 172.21.15.86:0/2185710374 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f70e2de10 msgr2=0x559f70e30260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:55.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.323+0000 7fde8d34d640 1 --2- 172.21.15.86:0/2185710374 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f70e2de10 0x559f70e30260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.323+0000 7fde8d34d640 1 -- 172.21.15.86:0/2185710374 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559f70e25290 msgr2=0x559f70e2d840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:55.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.323+0000 7fde8d34d640 1 --2- 172.21.15.86:0/2185710374 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559f70e25290 0x559f70e2d840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.323+0000 7fde8d34d640 1 -- 172.21.15.86:0/2185710374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f70b637e0 con 0x559f70e248e0 2024-04-21T08:45:55.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.323+0000 7fde7effd640 1 -- 172.21.15.86:0/1782043989 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7fde800d1430 con 0x7fde88113000 2024-04-21T08:45:55.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.323+0000 7fde8d34d640 1 --2- 172.21.15.86:0/2185710374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f70e248e0 0x559f70e24cc0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fde78005c40 tx=0x7fde78005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=970a0e56ce41109f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:55.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.323+0000 7fde8e34f640 1 --2- 172.21.15.86:0/1782043989 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fde740a3d20 0x7fde740a61e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fde800999d0 tx=0x7fde8006ea40 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:55.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.323+0000 7fde73fff640 1 -- 172.21.15.86:0/2185710374 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde78048070 con 0x559f70e248e0 2024-04-21T08:45:55.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.323+0000 7fde73fff640 1 -- 172.21.15.86:0/2185710374 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde780054f0 con 0x559f70e248e0 2024-04-21T08:45:55.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.323+0000 7fde73fff640 1 -- 172.21.15.86:0/2185710374 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde780035f0 con 0x559f70e248e0 2024-04-21T08:45:55.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.324+0000 7fde929a2740 1 -- 172.21.15.86:0/2185710374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f70e248e0 msgr2=0x559f70e24cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:55.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.324+0000 7fde929a2740 1 --2- 172.21.15.86:0/2185710374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f70e248e0 0x559f70e24cc0 secure :-1 s=READY pgs=1483 cs=0 l=1 rev1=1 crypto rx=0x7fde78005c40 tx=0x7fde78005bd0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.324+0000 7fde929a2740 1 -- 172.21.15.86:0/2185710374 shutdown_connections 2024-04-21T08:45:55.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.324+0000 7fde929a2740 1 --2- 172.21.15.86:0/2185710374 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f70e2de10 0x559f70e30260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.324+0000 7fde929a2740 1 --2- 172.21.15.86:0/2185710374 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559f70e25290 0x559f70e2d840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.324+0000 7fde929a2740 1 --2- 172.21.15.86:0/2185710374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f70e248e0 0x559f70e24cc0 unknown :-1 s=CLOSED pgs=1483 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.324+0000 7fde929a2740 1 -- 172.21.15.86:0/2185710374 >> 172.21.15.86:0/2185710374 conn(0x559f70e1efb0 msgr2=0x559f70e1f390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:55.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.324+0000 7fde929a2740 1 -- 172.21.15.86:0/2185710374 shutdown_connections 2024-04-21T08:45:55.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.324+0000 7fde929a2740 1 -- 172.21.15.86:0/2185710374 wait complete. 2024-04-21T08:45:55.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.325+0000 7fde929a2740 1 -- start start 2024-04-21T08:45:55.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.325+0000 7fde929a2740 1 Processor -- start 2024-04-21T08:45:55.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.325+0000 7fde929a2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f70e248e0 0x559f70e2ae60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.325+0000 7fde929a2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559f70e25290 0x559f70e2b3a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.325+0000 7fde929a2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f70e2de10 0x559f70e2b8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.325+0000 7fde929a2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559f70e30d20 con 0x559f70e2de10 2024-04-21T08:45:55.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.325+0000 7fde929a2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559f70d42580 con 0x559f70e25290 2024-04-21T08:45:55.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.325+0000 7fde929a2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x559f70e30ea0 con 0x559f70e248e0 2024-04-21T08:45:55.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.325+0000 7fde929a2740 10 client.0 _ll_register_callbacks cb 0x559f70d48650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:55.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.325+0000 7fde8d34d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f70e2de10 0x559f70e2b8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:55.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.325+0000 7fde8d34d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f70e2de10 0x559f70e2b8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38706/0 (socket says 172.21.15.86:38706) 2024-04-21T08:45:55.332 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.325+0000 7fde8d34d640 1 -- 172.21.15.86:0/2323704698 learned_addr learned my addr 172.21.15.86:0/2323704698 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:55.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.325+0000 7fde8db4e640 1 --2- 172.21.15.86:0/2323704698 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559f70e25290 0x559f70e2b3a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:55.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.325+0000 7fde8d34d640 1 -- 172.21.15.86:0/2323704698 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f70e248e0 msgr2=0x559f70e2ae60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:55.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.325+0000 7fde8d34d640 1 --2- 172.21.15.86:0/2323704698 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f70e248e0 0x559f70e2ae60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.325+0000 7fde8d34d640 1 -- 172.21.15.86:0/2323704698 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559f70e25290 msgr2=0x559f70e2b3a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:55.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.325+0000 7fde8d34d640 1 --2- 172.21.15.86:0/2323704698 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559f70e25290 0x559f70e2b3a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.325+0000 7fde8d34d640 1 -- 172.21.15.86:0/2323704698 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f70bc3f70 con 0x559f70e2de10 2024-04-21T08:45:55.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.326+0000 7fde8d34d640 1 --2- 172.21.15.86:0/2323704698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f70e2de10 0x559f70e2b8e0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fde7802c3c0 tx=0x7fde78004320 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daf8f1b1afa7648e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:55.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.326+0000 7fde717fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:55.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.326+0000 7fde717fa640 1 -- 172.21.15.86:0/2323704698 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde78048070 con 0x559f70e2de10 2024-04-21T08:45:55.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.326+0000 7fde717fa640 1 -- 172.21.15.86:0/2323704698 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7fde780046c0 con 0x559f70e2de10 2024-04-21T08:45:55.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.326+0000 7fde929a2740 1 -- 172.21.15.86:0/2323704698 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde780054f0 con 0x559f70e2de10 2024-04-21T08:45:55.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.326+0000 7fde717fa640 1 -- 172.21.15.86:0/2323704698 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fde780048c0 con 0x559f70e2de10 2024-04-21T08:45:55.333 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.326+0000 7fde929a2740 10 client.18636 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:55.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.326+0000 7fde929a2740 1 -- 172.21.15.86:0/2323704698 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559f70a5b170 con 0x559f70e2de10 2024-04-21T08:45:55.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.326+0000 7fde717fa640 1 -- 172.21.15.86:0/2323704698 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde780054f0 con 0x559f70e2de10 2024-04-21T08:45:55.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.326+0000 7fde717fa640 1 -- 172.21.15.86:0/2323704698 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fde7804c070 con 0x559f70e2de10 2024-04-21T08:45:55.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.326+0000 7fde929a2740 10 client.18636 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:55.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.326+0000 7fde929a2740 10 client.18636 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:55.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.326+0000 7fde929a2740 20 client.18636 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:55.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.326+0000 7fde929a2740 1 --2- 172.21.15.86:0/2323704698 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559f70e3e230 0x559f70e40620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.326+0000 7fde929a2740 4 client.18636 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:55.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.326+0000 7fde929a2740 1 -- 172.21.15.86:0/2323704698 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x559f70d96920 con 0x559f70e3e230 2024-04-21T08:45:55.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.327+0000 7fde8db4e640 1 --2- 172.21.15.86:0/2323704698 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559f70e3e230 0x559f70e40620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:55.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.327+0000 7fde8db4e640 1 --2- 172.21.15.86:0/2323704698 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559f70e3e230 0x559f70e40620 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=51d49f8af51b99b7 server_cookie=635c43d2ef620082 in_seq=0 out_seq=0 2024-04-21T08:45:55.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.327+0000 7fde717fa640 10 client.18636 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:55.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.328+0000 7fde717fa640 1 -- 172.21.15.86:0/2323704698 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x559f70d96920 con 0x559f70e3e230 2024-04-21T08:45:55.334 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.328+0000 7fde717fa640 10 client.18636 handle_command_reply: tid=0 2024-04-21T08:45:55.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.328+0000 7fde929a2740 1 client.18636 shutdown 2024-04-21T08:45:55.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.328+0000 7fde929a2740 1 -- 172.21.15.86:0/2323704698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f70e2de10 msgr2=0x559f70e2b8e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:55.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.328+0000 7fde929a2740 1 --2- 172.21.15.86:0/2323704698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f70e2de10 0x559f70e2b8e0 secure :-1 s=READY pgs=1484 cs=0 l=1 rev1=1 crypto rx=0x7fde7802c3c0 tx=0x7fde78004320 comp rx=0 tx=0).stop 2024-04-21T08:45:55.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.328+0000 7fde929a2740 1 -- 172.21.15.86:0/2323704698 shutdown_connections 2024-04-21T08:45:55.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.328+0000 7fde929a2740 1 --2- 172.21.15.86:0/2323704698 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559f70e3e230 0x559f70e40620 crc :-1 s=READY pgs=223 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.328+0000 7fde929a2740 1 --2- 172.21.15.86:0/2323704698 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f70e2de10 0x559f70e2b8e0 unknown :-1 s=CLOSED pgs=1484 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.328+0000 7fde929a2740 1 --2- 172.21.15.86:0/2323704698 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559f70e25290 0x559f70e2b3a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.328+0000 7fde929a2740 1 --2- 172.21.15.86:0/2323704698 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f70e248e0 0x559f70e2ae60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.328+0000 7fde717fa640 0 client.18636 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:55.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.328+0000 7fde929a2740 1 -- 172.21.15.86:0/2323704698 >> 172.21.15.86:0/2323704698 conn(0x559f70e1efb0 msgr2=0x559f70e20d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:55.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.328+0000 7fde929a2740 1 -- 172.21.15.86:0/2323704698 shutdown_connections 2024-04-21T08:45:55.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.328+0000 7fde929a2740 1 -- 172.21.15.86:0/2323704698 wait complete. 2024-04-21T08:45:55.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.328+0000 7fde929a2740 20 client.18636 trim_cache size 0 max 16384 2024-04-21T08:45:55.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.346+0000 7fde929a2740 1 Processor -- start 2024-04-21T08:45:55.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.346+0000 7fde929a2740 1 -- start start 2024-04-21T08:45:55.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.346+0000 7fde929a2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559f70e248e0 0x559f70e2d590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.346+0000 7fde929a2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f70e25290 0x559f70e1de90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.346+0000 7fde929a2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f70e2de10 0x559f70e43640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.346+0000 7fde929a2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fde881b59d0 con 0x559f70e2de10 2024-04-21T08:45:55.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.346+0000 7fde929a2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fde881b7510 con 0x559f70e248e0 2024-04-21T08:45:55.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.346+0000 7fde929a2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fde881b60a0 con 0x559f70e25290 2024-04-21T08:45:55.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.346+0000 7fde8d34d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f70e2de10 0x559f70e43640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:55.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.346+0000 7fde8d34d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f70e2de10 0x559f70e43640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38710/0 (socket says 172.21.15.86:38710) 2024-04-21T08:45:55.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.346+0000 7fde8d34d640 1 -- 172.21.15.86:0/3236669625 learned_addr learned my addr 172.21.15.86:0/3236669625 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:55.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.346+0000 7fde8d34d640 1 --2- 172.21.15.86:0/3236669625 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559f70e248e0 0x559f70e2d590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:55.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.346+0000 7fde8d34d640 1 -- 172.21.15.86:0/3236669625 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f70e25290 msgr2=0x559f70e1de90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:55.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.346+0000 7fde8d34d640 1 --2- 172.21.15.86:0/3236669625 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f70e25290 0x559f70e1de90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.346+0000 7fde8d34d640 1 -- 172.21.15.86:0/3236669625 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559f70e248e0 msgr2=0x559f70e2d590 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:55.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.346+0000 7fde8d34d640 1 --2- 172.21.15.86:0/3236669625 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559f70e248e0 0x559f70e2d590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.346+0000 7fde8d34d640 1 -- 172.21.15.86:0/3236669625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559f70e30c60 con 0x559f70e2de10 2024-04-21T08:45:55.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.346+0000 7fde8d34d640 1 --2- 172.21.15.86:0/3236669625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f70e2de10 0x559f70e43640 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fde78003820 tx=0x7fde78004cd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ad9576dda92e9c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:55.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.346+0000 7fde737fe640 1 -- 172.21.15.86:0/3236669625 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde78048040 con 0x559f70e2de10 2024-04-21T08:45:55.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.346+0000 7fde737fe640 1 -- 172.21.15.86:0/3236669625 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde78049040 con 0x559f70e2de10 2024-04-21T08:45:55.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.346+0000 7fde737fe640 1 -- 172.21.15.86:0/3236669625 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde78004390 con 0x559f70e2de10 2024-04-21T08:45:55.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.347+0000 7fde929a2740 1 -- 172.21.15.86:0/3236669625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f70e2de10 msgr2=0x559f70e43640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:55.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.347+0000 7fde929a2740 1 --2- 172.21.15.86:0/3236669625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f70e2de10 0x559f70e43640 secure :-1 s=READY pgs=1485 cs=0 l=1 rev1=1 crypto rx=0x7fde78003820 tx=0x7fde78004cd0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.347+0000 7fde929a2740 1 -- 172.21.15.86:0/3236669625 shutdown_connections 2024-04-21T08:45:55.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.347+0000 7fde929a2740 1 --2- 172.21.15.86:0/3236669625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f70e2de10 0x559f70e43640 unknown :-1 s=CLOSED pgs=1485 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.347+0000 7fde929a2740 1 --2- 172.21.15.86:0/3236669625 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f70e25290 0x559f70e1de90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.347+0000 7fde929a2740 1 --2- 172.21.15.86:0/3236669625 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559f70e248e0 0x559f70e2d590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.347+0000 7fde929a2740 1 -- 172.21.15.86:0/3236669625 >> 172.21.15.86:0/3236669625 conn(0x559f70e1efb0 msgr2=0x559f70e1d780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:55.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.347+0000 7fde929a2740 1 -- 172.21.15.86:0/3236669625 shutdown_connections 2024-04-21T08:45:55.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.347+0000 7fde929a2740 1 -- 172.21.15.86:0/3236669625 wait complete. 2024-04-21T08:45:55.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.347+0000 7fde929a2740 1 -- start start 2024-04-21T08:45:55.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.347+0000 7fde929a2740 1 Processor -- start 2024-04-21T08:45:55.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.348+0000 7fde929a2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559f70e248e0 0x559f70e41880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.348+0000 7fde929a2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f70e25290 0x559f70e41dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.348+0000 7fde929a2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f70e2de10 0x559f70ea3410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.348+0000 7fde929a2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fde881b6e40 con 0x559f70e25290 2024-04-21T08:45:55.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.348+0000 7fde929a2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fde881b6770 con 0x559f70e248e0 2024-04-21T08:45:55.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.348+0000 7fde929a2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fde881b39d0 con 0x559f70e2de10 2024-04-21T08:45:55.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.348+0000 7fde929a2740 10 client.0 _ll_register_callbacks cb 0x7fde78046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:55.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.348+0000 7fde8db4e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f70e25290 0x559f70e41dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:55.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.348+0000 7fde8db4e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f70e25290 0x559f70e41dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38712/0 (socket says 172.21.15.86:38712) 2024-04-21T08:45:55.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.348+0000 7fde8db4e640 1 -- 172.21.15.86:0/3227501582 learned_addr learned my addr 172.21.15.86:0/3227501582 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:55.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.348+0000 7fde8d34d640 1 --2- 172.21.15.86:0/3227501582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f70e2de10 0x559f70ea3410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:55.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.348+0000 7fde8db4e640 1 -- 172.21.15.86:0/3227501582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f70e2de10 msgr2=0x559f70ea3410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:55.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.348+0000 7fde8db4e640 1 --2- 172.21.15.86:0/3227501582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f70e2de10 0x559f70ea3410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.348+0000 7fde8db4e640 1 -- 172.21.15.86:0/3227501582 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559f70e248e0 msgr2=0x559f70e41880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:55.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.348+0000 7fde8db4e640 1 --2- 172.21.15.86:0/3227501582 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559f70e248e0 0x559f70e41880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.348+0000 7fde8db4e640 1 -- 172.21.15.86:0/3227501582 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559f70ea3a20 con 0x559f70e25290 2024-04-21T08:45:55.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.348+0000 7fde8d34d640 1 --2- 172.21.15.86:0/3227501582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f70e2de10 0x559f70ea3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:45:55.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.348+0000 7fde8db4e640 1 --2- 172.21.15.86:0/3227501582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f70e25290 0x559f70e41dc0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x559f70e25690 tx=0x7fde84014840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9408ada93b9c477 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:55.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.348+0000 7fde717fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:55.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.348+0000 7fde717fa640 1 -- 172.21.15.86:0/3227501582 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde84005220 con 0x559f70e25290 2024-04-21T08:45:55.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.348+0000 7fde717fa640 1 -- 172.21.15.86:0/3227501582 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7fde840053c0 con 0x559f70e25290 2024-04-21T08:45:55.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.348+0000 7fde929a2740 1 -- 172.21.15.86:0/3227501582 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde78049040 con 0x559f70e25290 2024-04-21T08:45:55.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.349+0000 7fde717fa640 1 -- 172.21.15.86:0/3227501582 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fde8402d050 con 0x559f70e25290 2024-04-21T08:45:55.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.349+0000 7fde929a2740 10 client.18642 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:55.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.349+0000 7fde929a2740 1 -- 172.21.15.86:0/3227501582 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559f70ea4df0 con 0x559f70e25290 2024-04-21T08:45:55.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.349+0000 7fde717fa640 1 -- 172.21.15.86:0/3227501582 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde78049040 con 0x559f70e25290 2024-04-21T08:45:55.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.349+0000 7fde717fa640 1 -- 172.21.15.86:0/3227501582 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fde8401e0e0 con 0x559f70e25290 2024-04-21T08:45:55.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.349+0000 7fde929a2740 10 client.18642 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:55.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.349+0000 7fde929a2740 10 client.18642 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:55.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.349+0000 7fde929a2740 20 client.18642 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:55.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.349+0000 7fde929a2740 1 --2- 172.21.15.86:0/3227501582 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559f70ea55a0 0x559f70ea79c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.349+0000 7fde929a2740 4 client.18642 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:45:55.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.349+0000 7fde929a2740 1 -- 172.21.15.86:0/3227501582 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559f70d90790 con 0x559f70ea55a0 2024-04-21T08:45:55.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.349+0000 7fde8d34d640 1 --2- 172.21.15.86:0/3227501582 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559f70ea55a0 0x559f70ea79c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:55.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.350+0000 7fde717fa640 10 client.18642 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:55.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.350+0000 7fde8d34d640 1 --2- 172.21.15.86:0/3227501582 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559f70ea55a0 0x559f70ea79c0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c936eda94582936 server_cookie=1f1f62818e7b6128 in_seq=0 out_seq=0 2024-04-21T08:45:55.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.351+0000 7fde717fa640 1 -- 172.21.15.86:0/3227501582 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559f70d90790 con 0x559f70ea55a0 2024-04-21T08:45:55.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.351+0000 7fde717fa640 10 client.18642 handle_command_reply: tid=0 2024-04-21T08:45:55.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.351+0000 7fde929a2740 1 client.18642 shutdown 2024-04-21T08:45:55.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.351+0000 7fde929a2740 1 -- 172.21.15.86:0/3227501582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f70e25290 msgr2=0x559f70e41dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:55.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.351+0000 7fde929a2740 1 --2- 172.21.15.86:0/3227501582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f70e25290 0x559f70e41dc0 secure :-1 s=READY pgs=1486 cs=0 l=1 rev1=1 crypto rx=0x559f70e25690 tx=0x7fde84014840 comp rx=0 tx=0).stop 2024-04-21T08:45:55.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.351+0000 7fde929a2740 1 -- 172.21.15.86:0/3227501582 shutdown_connections 2024-04-21T08:45:55.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.351+0000 7fde929a2740 1 --2- 172.21.15.86:0/3227501582 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x559f70ea55a0 0x559f70ea79c0 crc :-1 s=READY pgs=224 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.351+0000 7fde929a2740 1 --2- 172.21.15.86:0/3227501582 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559f70e2de10 0x559f70ea3410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.351+0000 7fde929a2740 1 --2- 172.21.15.86:0/3227501582 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559f70e25290 0x559f70e41dc0 unknown :-1 s=CLOSED pgs=1486 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.351+0000 7fde717fa640 0 client.18642 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:55.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.351+0000 7fde929a2740 1 --2- 172.21.15.86:0/3227501582 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559f70e248e0 0x559f70e41880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.351+0000 7fde929a2740 1 -- 172.21.15.86:0/3227501582 >> 172.21.15.86:0/3227501582 conn(0x559f70e1efb0 msgr2=0x559f70e399b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:55.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.351+0000 7fde929a2740 1 -- 172.21.15.86:0/3227501582 shutdown_connections 2024-04-21T08:45:55.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.351+0000 7fde929a2740 1 -- 172.21.15.86:0/3227501582 wait complete. 2024-04-21T08:45:55.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.351+0000 7fde929a2740 20 client.18642 trim_cache size 0 max 16384 2024-04-21T08:45:55.356 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:45:55.357 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-21T08:45:55.357 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "669ce31b-1ab6-497d-a00e-59a6472b37d2", 2024-04-21T08:45:55.357 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-21T08:45:55.357 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:45:55.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.351+0000 7fde8fdb2640 1 -- 172.21.15.86:0/1782043989 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fde740a3d20 msgr2=0x7fde740a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:55.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.351+0000 7fde8fdb2640 1 --2- 172.21.15.86:0/1782043989 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fde740a3d20 0x7fde740a61e0 secure :-1 s=READY pgs=238 cs=0 l=1 rev1=1 crypto rx=0x7fde800999d0 tx=0x7fde8006ea40 comp rx=0 tx=0).stop 2024-04-21T08:45:55.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.351+0000 7fde8fdb2640 1 -- 172.21.15.86:0/1782043989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde88113000 msgr2=0x7fde881a36c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:55.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.351+0000 7fde8fdb2640 1 --2- 172.21.15.86:0/1782043989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde88113000 0x7fde881a36c0 secure :-1 s=READY pgs=1482 cs=0 l=1 rev1=1 crypto rx=0x7fde8000eb40 tx=0x7fde8000c2d0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.351+0000 7fde8fdb2640 1 -- 172.21.15.86:0/1782043989 shutdown_connections 2024-04-21T08:45:55.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.352+0000 7fde8fdb2640 1 --2- 172.21.15.86:0/1782043989 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fde740a3d20 0x7fde740a61e0 unknown :-1 s=CLOSED pgs=238 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.352+0000 7fde8fdb2640 1 --2- 172.21.15.86:0/1782043989 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde88113000 0x7fde881a36c0 unknown :-1 s=CLOSED pgs=1482 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.352+0000 7fde8fdb2640 1 --2- 172.21.15.86:0/1782043989 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde88074b40 0x7fde8819f340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.352+0000 7fde8fdb2640 1 --2- 172.21.15.86:0/1782043989 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde88074220 0x7fde8819ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.352+0000 7fde8fdb2640 1 -- 172.21.15.86:0/1782043989 >> 172.21.15.86:0/1782043989 conn(0x7fde880793a0 msgr2=0x7fde880fef60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:55.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.352+0000 7fde8fdb2640 1 -- 172.21.15.86:0/1782043989 shutdown_connections 2024-04-21T08:45:55.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.352+0000 7fde8fdb2640 1 -- 172.21.15.86:0/1782043989 wait complete. 2024-04-21T08:45:55.361 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:45:55.362 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:45:55.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.462+0000 7f02d94f2640 1 -- 172.21.15.86:0/662885923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02d41031c0 msgr2=0x7f02d41035a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:55.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.462+0000 7f02d94f2640 1 --2- 172.21.15.86:0/662885923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02d41031c0 0x7f02d41035a0 secure :-1 s=READY pgs=1487 cs=0 l=1 rev1=1 crypto rx=0x7f02cc009ca0 tx=0x7f02cc02c230 comp rx=0 tx=0).stop 2024-04-21T08:45:55.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.463+0000 7f02d94f2640 1 -- 172.21.15.86:0/662885923 shutdown_connections 2024-04-21T08:45:55.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.463+0000 7f02d94f2640 1 --2- 172.21.15.86:0/662885923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02d410c500 0x7f02d410e9a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.463+0000 7f02d94f2640 1 --2- 172.21.15.86:0/662885923 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02d4103ae0 0x7f02d410bfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.463+0000 7f02d94f2640 1 --2- 172.21.15.86:0/662885923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02d41031c0 0x7f02d41035a0 unknown :-1 s=CLOSED pgs=1487 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.463+0000 7f02d94f2640 1 -- 172.21.15.86:0/662885923 >> 172.21.15.86:0/662885923 conn(0x7f02d4078000 msgr2=0x7f02d4078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:55.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.463+0000 7f02d94f2640 1 -- 172.21.15.86:0/662885923 shutdown_connections 2024-04-21T08:45:55.464 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.463+0000 7f02d94f2640 1 -- 172.21.15.86:0/662885923 wait complete. 2024-04-21T08:45:55.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.463+0000 7f02d94f2640 1 Processor -- start 2024-04-21T08:45:55.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.463+0000 7f02d94f2640 1 -- start start 2024-04-21T08:45:55.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.463+0000 7f02d94f2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02d41031c0 0x7f02d4198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.463+0000 7f02d94f2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02d4103ae0 0x7f02d4198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.464+0000 7f02d94f2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02d410c500 0x7f02d419cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.464+0000 7f02d94f2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f02d4112190 con 0x7f02d410c500 2024-04-21T08:45:55.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.464+0000 7f02d94f2640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f02d4112010 con 0x7f02d41031c0 2024-04-21T08:45:55.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.464+0000 7f02d94f2640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f02d4112310 con 0x7f02d4103ae0 2024-04-21T08:45:55.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.464+0000 7f02d37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02d410c500 0x7f02d419cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:55.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.464+0000 7f02d37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02d410c500 0x7f02d419cfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38728/0 (socket says 172.21.15.86:38728) 2024-04-21T08:45:55.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.464+0000 7f02d2ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02d41031c0 0x7f02d4198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:55.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.464+0000 7f02d37fe640 1 -- 172.21.15.86:0/2689271737 learned_addr learned my addr 172.21.15.86:0/2689271737 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:55.465 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.464+0000 7f02d27fc640 1 --2- 172.21.15.86:0/2689271737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02d4103ae0 0x7f02d4198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:55.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.464+0000 7f02d37fe640 1 -- 172.21.15.86:0/2689271737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02d4103ae0 msgr2=0x7f02d4198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:55.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.464+0000 7f02d37fe640 1 --2- 172.21.15.86:0/2689271737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02d4103ae0 0x7f02d4198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.464+0000 7f02d37fe640 1 -- 172.21.15.86:0/2689271737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02d41031c0 msgr2=0x7f02d4198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:55.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.464+0000 7f02d37fe640 1 --2- 172.21.15.86:0/2689271737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02d41031c0 0x7f02d4198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.464+0000 7f02d37fe640 1 -- 172.21.15.86:0/2689271737 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f02d419d700 con 0x7f02d410c500 2024-04-21T08:45:55.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.464+0000 7f02d37fe640 1 --2- 172.21.15.86:0/2689271737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02d410c500 0x7f02d419cfd0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f02c800e970 tx=0x7f02c800ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:55.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.464+0000 7f02bbfff640 1 -- 172.21.15.86:0/2689271737 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c800ccc0 con 0x7f02d410c500 2024-04-21T08:45:55.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.464+0000 7f02d94f2640 1 -- 172.21.15.86:0/2689271737 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f02d419d9f0 con 0x7f02d410c500 2024-04-21T08:45:55.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.464+0000 7f02bbfff640 1 -- 172.21.15.86:0/2689271737 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02c800ce60 con 0x7f02d410c500 2024-04-21T08:45:55.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.464+0000 7f02bbfff640 1 -- 172.21.15.86:0/2689271737 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c8010430 con 0x7f02d410c500 2024-04-21T08:45:55.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.464+0000 7f02d94f2640 1 -- 172.21.15.86:0/2689271737 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f02d41a5280 con 0x7f02d410c500 2024-04-21T08:45:55.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.466+0000 7f02bbfff640 1 -- 172.21.15.86:0/2689271737 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f02c8004ff0 con 0x7f02d410c500 2024-04-21T08:45:55.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.467+0000 7f02dc1c4740 1 Processor -- start 2024-04-21T08:45:55.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.467+0000 7f02dc1c4740 1 -- start start 2024-04-21T08:45:55.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.467+0000 7f02bbfff640 1 --2- 172.21.15.86:0/2689271737 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f02bc0a3d20 0x7f02bc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.467+0000 7f02dc1c4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b6daca4230 0x55b6daca4610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.467+0000 7f02dc1c4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6daca4b50 0x55b6dacb1950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.467+0000 7f02dc1c4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6dacb1e90 0x55b6dacb4280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.467+0000 7f02dc1c4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b6dac091e0 con 0x55b6dacb1e90 2024-04-21T08:45:55.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.467+0000 7f02dc1c4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b6daba6390 con 0x55b6daca4230 2024-04-21T08:45:55.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.467+0000 7f02dc1c4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b6dac051a0 con 0x55b6daca4b50 2024-04-21T08:45:55.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.467+0000 7f02d27fc640 1 --2- 172.21.15.86:0/2689271737 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f02bc0a3d20 0x7f02bc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:55.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.467+0000 7f02d37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6dacb1e90 0x55b6dacb4280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:55.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.467+0000 7f02bbfff640 1 -- 172.21.15.86:0/2689271737 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f02c80d0f80 con 0x7f02d410c500 2024-04-21T08:45:55.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.467+0000 7f02d37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6dacb1e90 0x55b6dacb4280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38742/0 (socket says 172.21.15.86:38742) 2024-04-21T08:45:55.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.467+0000 7f02d37fe640 1 -- 172.21.15.86:0/1957507979 learned_addr learned my addr 172.21.15.86:0/1957507979 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:55.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.467+0000 7f02d27fc640 1 --2- 172.21.15.86:0/1957507979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6daca4b50 0x55b6dacb1950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:55.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.467+0000 7f02d2ffd640 1 --2- 172.21.15.86:0/1957507979 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b6daca4230 0x55b6daca4610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:55.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.467+0000 7f02d37fe640 1 -- 172.21.15.86:0/1957507979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6daca4b50 msgr2=0x55b6dacb1950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:55.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.467+0000 7f02d37fe640 1 --2- 172.21.15.86:0/1957507979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6daca4b50 0x55b6dacb1950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.467+0000 7f02d37fe640 1 -- 172.21.15.86:0/1957507979 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b6daca4230 msgr2=0x55b6daca4610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:55.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.467+0000 7f02d37fe640 1 --2- 172.21.15.86:0/1957507979 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b6daca4230 0x55b6daca4610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.467+0000 7f02d37fe640 1 -- 172.21.15.86:0/1957507979 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6da9e57d0 con 0x55b6dacb1e90 2024-04-21T08:45:55.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.467+0000 7f02d27fc640 1 --2- 172.21.15.86:0/2689271737 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f02bc0a3d20 0x7f02bc0a61e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f02c4009fd0 tx=0x7f02c4009b80 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:55.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.467+0000 7f02d37fe640 1 --2- 172.21.15.86:0/1957507979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6dacb1e90 0x55b6dacb4280 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f02c809cae0 tx=0x7f02c809c690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5d1cc124a15f113 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:55.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.468+0000 7f02b97fa640 1 -- 172.21.15.86:0/1957507979 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c8049580 con 0x55b6dacb1e90 2024-04-21T08:45:55.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.468+0000 7f02b97fa640 1 -- 172.21.15.86:0/1957507979 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02c80055f0 con 0x55b6dacb1e90 2024-04-21T08:45:55.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.468+0000 7f02b97fa640 1 -- 172.21.15.86:0/1957507979 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c809ae50 con 0x55b6dacb1e90 2024-04-21T08:45:55.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.468+0000 7f02dc1c4740 1 -- 172.21.15.86:0/1957507979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6dacb1e90 msgr2=0x55b6dacb4280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:55.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.468+0000 7f02dc1c4740 1 --2- 172.21.15.86:0/1957507979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6dacb1e90 0x55b6dacb4280 secure :-1 s=READY pgs=1489 cs=0 l=1 rev1=1 crypto rx=0x7f02c809cae0 tx=0x7f02c809c690 comp rx=0 tx=0).stop 2024-04-21T08:45:55.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.468+0000 7f02dc1c4740 1 -- 172.21.15.86:0/1957507979 shutdown_connections 2024-04-21T08:45:55.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.468+0000 7f02dc1c4740 1 --2- 172.21.15.86:0/1957507979 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6dacb1e90 0x55b6dacb4280 unknown :-1 s=CLOSED pgs=1489 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.468+0000 7f02dc1c4740 1 --2- 172.21.15.86:0/1957507979 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6daca4b50 0x55b6dacb1950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.468+0000 7f02dc1c4740 1 --2- 172.21.15.86:0/1957507979 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b6daca4230 0x55b6daca4610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.468+0000 7f02dc1c4740 1 -- 172.21.15.86:0/1957507979 >> 172.21.15.86:0/1957507979 conn(0x55b6dac9ecb0 msgr2=0x55b6dac9f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:55.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.468+0000 7f02dc1c4740 1 -- 172.21.15.86:0/1957507979 shutdown_connections 2024-04-21T08:45:55.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.468+0000 7f02dc1c4740 1 -- 172.21.15.86:0/1957507979 wait complete. 2024-04-21T08:45:55.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.469+0000 7f02dc1c4740 1 -- start start 2024-04-21T08:45:55.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.469+0000 7f02dc1c4740 1 Processor -- start 2024-04-21T08:45:55.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.469+0000 7f02dc1c4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6daca4230 0x55b6dacaee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.469+0000 7f02dc1c4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b6daca4b50 0x55b6dacaf3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.469+0000 7f02dc1c4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6dacb1e90 0x55b6dacaf8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.469+0000 7f02dc1c4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b6dacb4c10 con 0x55b6daca4230 2024-04-21T08:45:55.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.469+0000 7f02dc1c4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b6dabc22f0 con 0x55b6daca4b50 2024-04-21T08:45:55.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.469+0000 7f02dc1c4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b6dacb4d90 con 0x55b6dacb1e90 2024-04-21T08:45:55.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.469+0000 7f02dc1c4740 10 client.0 _ll_register_callbacks cb 0x55b6dabcf580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:55.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.469+0000 7f02d2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6daca4230 0x55b6dacaee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:55.476 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.469+0000 7f02d2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6daca4230 0x55b6dacaee70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38748/0 (socket says 172.21.15.86:38748) 2024-04-21T08:45:55.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.469+0000 7f02d2ffd640 1 -- 172.21.15.86:0/4090585643 learned_addr learned my addr 172.21.15.86:0/4090585643 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:55.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.469+0000 7f02d27fc640 1 --2- 172.21.15.86:0/4090585643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b6daca4b50 0x55b6dacaf3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:55.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.469+0000 7f02d37fe640 1 --2- 172.21.15.86:0/4090585643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6dacb1e90 0x55b6dacaf8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:55.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.469+0000 7f02d2ffd640 1 -- 172.21.15.86:0/4090585643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6dacb1e90 msgr2=0x55b6dacaf8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:55.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.469+0000 7f02d2ffd640 1 --2- 172.21.15.86:0/4090585643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6dacb1e90 0x55b6dacaf8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.469+0000 7f02d2ffd640 1 -- 172.21.15.86:0/4090585643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b6daca4b50 msgr2=0x55b6dacaf3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:55.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.469+0000 7f02d2ffd640 1 --2- 172.21.15.86:0/4090585643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b6daca4b50 0x55b6dacaf3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.469+0000 7f02d2ffd640 1 -- 172.21.15.86:0/4090585643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6da91b3f0 con 0x55b6daca4230 2024-04-21T08:45:55.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.469+0000 7f02d37fe640 1 --2- 172.21.15.86:0/4090585643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6dacb1e90 0x55b6dacaf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:55.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.470+0000 7f02d2ffd640 1 --2- 172.21.15.86:0/4090585643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6daca4230 0x55b6dacaee70 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f02cc02c770 tx=0x7f02cc003ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c01af7aad48b2c53 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:55.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.470+0000 7f02b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:55.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.470+0000 7f02b6ff5640 1 -- 172.21.15.86:0/4090585643 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02cc044070 con 0x55b6daca4230 2024-04-21T08:45:55.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.470+0000 7f02b6ff5640 1 -- 172.21.15.86:0/4090585643 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f02cc004240 con 0x55b6daca4230 2024-04-21T08:45:55.477 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.470+0000 7f02dc1c4740 1 -- 172.21.15.86:0/4090585643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55b6dac16c00 con 0x55b6daca4230 2024-04-21T08:45:55.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.470+0000 7f02b6ff5640 1 -- 172.21.15.86:0/4090585643 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f02cc051050 con 0x55b6daca4230 2024-04-21T08:45:55.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.470+0000 7f02dc1c4740 10 client.18654 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:55.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.470+0000 7f02dc1c4740 1 -- 172.21.15.86:0/4090585643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b6da8db0f0 con 0x55b6daca4230 2024-04-21T08:45:55.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.470+0000 7f02b6ff5640 1 -- 172.21.15.86:0/4090585643 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55b6dac16c00 con 0x55b6daca4230 2024-04-21T08:45:55.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.470+0000 7f02b6ff5640 1 -- 172.21.15.86:0/4090585643 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f02cc032cd0 con 0x55b6daca4230 2024-04-21T08:45:55.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.470+0000 7f02dc1c4740 10 client.18654 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:55.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.470+0000 7f02dc1c4740 10 client.18654 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:55.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.470+0000 7f02dc1c4740 20 client.18654 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:55.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.470+0000 7f02dc1c4740 1 --2- 172.21.15.86:0/4090585643 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b6dacc1e30 0x55b6dacc42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.470+0000 7f02dc1c4740 4 client.18654 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:45:55.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.470+0000 7f02dc1c4740 1 -- 172.21.15.86:0/4090585643 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6dac10a70 con 0x55b6dacc1e30 2024-04-21T08:45:55.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.470+0000 7f02d27fc640 1 --2- 172.21.15.86:0/4090585643 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b6dacc1e30 0x55b6dacc42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:55.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.471+0000 7f02b6ff5640 10 client.18654 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:55.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.471+0000 7f02d27fc640 1 --2- 172.21.15.86:0/4090585643 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b6dacc1e30 0x55b6dacc42d0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=637184ec55d6de77 server_cookie=ea97e15f083e853b in_seq=0 out_seq=0 2024-04-21T08:45:55.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.471+0000 7f02b6ff5640 1 -- 172.21.15.86:0/4090585643 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b6dac10a70 con 0x55b6dacc1e30 2024-04-21T08:45:55.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.471+0000 7f02b6ff5640 10 client.18654 handle_command_reply: tid=0 2024-04-21T08:45:55.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.471+0000 7f02dc1c4740 1 client.18654 shutdown 2024-04-21T08:45:55.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.472+0000 7f02dc1c4740 1 -- 172.21.15.86:0/4090585643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6daca4230 msgr2=0x55b6dacaee70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:55.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.472+0000 7f02dc1c4740 1 --2- 172.21.15.86:0/4090585643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6daca4230 0x55b6dacaee70 secure :-1 s=READY pgs=1490 cs=0 l=1 rev1=1 crypto rx=0x7f02cc02c770 tx=0x7f02cc003ee0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.472+0000 7f02dc1c4740 1 -- 172.21.15.86:0/4090585643 shutdown_connections 2024-04-21T08:45:55.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.472+0000 7f02dc1c4740 1 --2- 172.21.15.86:0/4090585643 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b6dacc1e30 0x55b6dacc42d0 crc :-1 s=READY pgs=225 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.472+0000 7f02dc1c4740 1 --2- 172.21.15.86:0/4090585643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6dacb1e90 0x55b6dacaf8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.472+0000 7f02dc1c4740 1 --2- 172.21.15.86:0/4090585643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b6daca4b50 0x55b6dacaf3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.472+0000 7f02b6ff5640 0 client.18654 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:55.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.472+0000 7f02dc1c4740 1 --2- 172.21.15.86:0/4090585643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6daca4230 0x55b6dacaee70 unknown :-1 s=CLOSED pgs=1490 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.472+0000 7f02dc1c4740 1 -- 172.21.15.86:0/4090585643 >> 172.21.15.86:0/4090585643 conn(0x55b6dac9ecb0 msgr2=0x55b6daca0a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:55.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.472+0000 7f02dc1c4740 1 -- 172.21.15.86:0/4090585643 shutdown_connections 2024-04-21T08:45:55.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.472+0000 7f02dc1c4740 1 -- 172.21.15.86:0/4090585643 wait complete. 2024-04-21T08:45:55.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.472+0000 7f02dc1c4740 20 client.18654 trim_cache size 0 max 16384 2024-04-21T08:45:55.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.489+0000 7f02dc1c4740 1 Processor -- start 2024-04-21T08:45:55.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.489+0000 7f02dc1c4740 1 -- start start 2024-04-21T08:45:55.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.489+0000 7f02dc1c4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6daca4230 0x55b6dacb15a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.489+0000 7f02dc1c4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6daca4b50 0x55b6dacbdb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.489+0000 7f02dc1c4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b6dacb1e90 0x55b6daca0990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.489+0000 7f02dc1c4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f02d41ad1c0 con 0x55b6daca4b50 2024-04-21T08:45:55.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.489+0000 7f02dc1c4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f02d41aed00 con 0x55b6dacb1e90 2024-04-21T08:45:55.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.489+0000 7f02dc1c4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f02d41ad890 con 0x55b6daca4230 2024-04-21T08:45:55.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.489+0000 7f02d27fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6daca4b50 0x55b6dacbdb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:55.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.489+0000 7f02d27fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6daca4b50 0x55b6dacbdb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38756/0 (socket says 172.21.15.86:38756) 2024-04-21T08:45:55.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.489+0000 7f02d27fc640 1 -- 172.21.15.86:0/3555603739 learned_addr learned my addr 172.21.15.86:0/3555603739 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:55.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.489+0000 7f02d37fe640 1 --2- 172.21.15.86:0/3555603739 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b6dacb1e90 0x55b6daca0990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:55.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.489+0000 7f02d2ffd640 1 --2- 172.21.15.86:0/3555603739 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6daca4230 0x55b6dacb15a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:55.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.489+0000 7f02d27fc640 1 -- 172.21.15.86:0/3555603739 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6daca4230 msgr2=0x55b6dacb15a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:55.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.489+0000 7f02d27fc640 1 --2- 172.21.15.86:0/3555603739 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6daca4230 0x55b6dacb15a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.489+0000 7f02d27fc640 1 -- 172.21.15.86:0/3555603739 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b6dacb1e90 msgr2=0x55b6daca0990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:55.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.489+0000 7f02d27fc640 1 --2- 172.21.15.86:0/3555603739 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b6dacb1e90 0x55b6daca0990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.489+0000 7f02d27fc640 1 -- 172.21.15.86:0/3555603739 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b6daca4980 con 0x55b6daca4b50 2024-04-21T08:45:55.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.490+0000 7f02d27fc640 1 --2- 172.21.15.86:0/3555603739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6daca4b50 0x55b6dacbdb40 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f02c4009710 tx=0x7f02c4021000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5401bae12b4c0fe server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:55.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.490+0000 7f02b8ff9640 1 -- 172.21.15.86:0/3555603739 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c40044c0 con 0x55b6daca4b50 2024-04-21T08:45:55.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.490+0000 7f02b8ff9640 1 -- 172.21.15.86:0/3555603739 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02c400fe60 con 0x55b6daca4b50 2024-04-21T08:45:55.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.490+0000 7f02b8ff9640 1 -- 172.21.15.86:0/3555603739 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c40052f0 con 0x55b6daca4b50 2024-04-21T08:45:55.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.490+0000 7f02dc1c4740 1 -- 172.21.15.86:0/3555603739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6daca4b50 msgr2=0x55b6dacbdb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:55.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.490+0000 7f02dc1c4740 1 --2- 172.21.15.86:0/3555603739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6daca4b50 0x55b6dacbdb40 secure :-1 s=READY pgs=1491 cs=0 l=1 rev1=1 crypto rx=0x7f02c4009710 tx=0x7f02c4021000 comp rx=0 tx=0).stop 2024-04-21T08:45:55.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.490+0000 7f02dc1c4740 1 -- 172.21.15.86:0/3555603739 shutdown_connections 2024-04-21T08:45:55.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.490+0000 7f02dc1c4740 1 --2- 172.21.15.86:0/3555603739 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b6dacb1e90 0x55b6daca0990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.490+0000 7f02dc1c4740 1 --2- 172.21.15.86:0/3555603739 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6daca4b50 0x55b6dacbdb40 unknown :-1 s=CLOSED pgs=1491 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.490+0000 7f02dc1c4740 1 --2- 172.21.15.86:0/3555603739 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6daca4230 0x55b6dacb15a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.490+0000 7f02dc1c4740 1 -- 172.21.15.86:0/3555603739 >> 172.21.15.86:0/3555603739 conn(0x55b6dac9ecb0 msgr2=0x55b6dacaddb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:55.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.490+0000 7f02dc1c4740 1 -- 172.21.15.86:0/3555603739 shutdown_connections 2024-04-21T08:45:55.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.490+0000 7f02dc1c4740 1 -- 172.21.15.86:0/3555603739 wait complete. 2024-04-21T08:45:55.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.491+0000 7f02dc1c4740 1 -- start start 2024-04-21T08:45:55.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.491+0000 7f02dc1c4740 1 Processor -- start 2024-04-21T08:45:55.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.491+0000 7f02dc1c4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b6daca4230 0x55b6dacb5a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.491+0000 7f02dc1c4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6daca4b50 0x55b6dacb5fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.491+0000 7f02dc1c4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6dacb1e90 0x55b6dacb64f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.491+0000 7f02dc1c4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f02d41ae630 con 0x55b6dacb1e90 2024-04-21T08:45:55.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.491+0000 7f02dc1c4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f02d41adf60 con 0x55b6daca4230 2024-04-21T08:45:55.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.491+0000 7f02dc1c4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f02d41ab1c0 con 0x55b6daca4b50 2024-04-21T08:45:55.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.491+0000 7f02dc1c4740 10 client.0 _ll_register_callbacks cb 0x7f02c4005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:45:55.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.491+0000 7f02d37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6dacb1e90 0x55b6dacb64f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:55.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.491+0000 7f02d37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6dacb1e90 0x55b6dacb64f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38770/0 (socket says 172.21.15.86:38770) 2024-04-21T08:45:55.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.491+0000 7f02d37fe640 1 -- 172.21.15.86:0/3206840103 learned_addr learned my addr 172.21.15.86:0/3206840103 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:45:55.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.491+0000 7f02d2ffd640 1 --2- 172.21.15.86:0/3206840103 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b6daca4230 0x55b6dacb5a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:55.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.491+0000 7f02d27fc640 1 --2- 172.21.15.86:0/3206840103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6daca4b50 0x55b6dacb5fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:55.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.491+0000 7f02d37fe640 1 -- 172.21.15.86:0/3206840103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6daca4b50 msgr2=0x55b6dacb5fb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:55.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.491+0000 7f02d37fe640 1 --2- 172.21.15.86:0/3206840103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6daca4b50 0x55b6dacb5fb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.491+0000 7f02d37fe640 1 -- 172.21.15.86:0/3206840103 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b6daca4230 msgr2=0x55b6dacb5a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:45:55.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.491+0000 7f02d37fe640 1 --2- 172.21.15.86:0/3206840103 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b6daca4230 0x55b6dacb5a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.491+0000 7f02d37fe640 1 -- 172.21.15.86:0/3206840103 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6dad27410 con 0x55b6dacb1e90 2024-04-21T08:45:55.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.491+0000 7f02d2ffd640 1 --2- 172.21.15.86:0/3206840103 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b6daca4230 0x55b6dacb5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:55.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.491+0000 7f02d27fc640 1 --2- 172.21.15.86:0/3206840103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6daca4b50 0x55b6dacb5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:45:55.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.491+0000 7f02d37fe640 1 --2- 172.21.15.86:0/3206840103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6dacb1e90 0x55b6dacb64f0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f02c80ce900 tx=0x7f02c80495f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b85f03346a1d94fc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:45:55.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.492+0000 7f02b6ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:45:55.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.492+0000 7f02b6ff5640 1 -- 172.21.15.86:0/3206840103 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f02c809d250 con 0x55b6dacb1e90 2024-04-21T08:45:55.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.492+0000 7f02b6ff5640 1 -- 172.21.15.86:0/3206840103 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(96..96 src has 1..96) ==== 9496+0+0 (secure 0 0 0) 0x7f02c80991c0 con 0x55b6dacb1e90 2024-04-21T08:45:55.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.492+0000 7f02dc1c4740 1 -- 172.21.15.86:0/3206840103 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02c400fe60 con 0x55b6dacb1e90 2024-04-21T08:45:55.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.492+0000 7f02b6ff5640 1 -- 172.21.15.86:0/3206840103 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f02c8080050 con 0x55b6dacb1e90 2024-04-21T08:45:55.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.492+0000 7f02dc1c4740 10 client.18660 fetch_fsmap learned FSMap version 29 2024-04-21T08:45:55.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.492+0000 7f02dc1c4740 1 -- 172.21.15.86:0/3206840103 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b6dad28700 con 0x55b6dacb1e90 2024-04-21T08:45:55.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.492+0000 7f02b6ff5640 1 -- 172.21.15.86:0/3206840103 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02c400fe60 con 0x55b6dacb1e90 2024-04-21T08:45:55.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.492+0000 7f02b6ff5640 1 -- 172.21.15.86:0/3206840103 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f02c80a5330 con 0x55b6dacb1e90 2024-04-21T08:45:55.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.492+0000 7f02dc1c4740 10 client.18660 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:45:55.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.492+0000 7f02dc1c4740 10 client.18660 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:45:55.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.492+0000 7f02dc1c4740 20 client.18660 populate_metadata read hostname 'smithi086' 2024-04-21T08:45:55.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.492+0000 7f02dc1c4740 1 --2- 172.21.15.86:0/3206840103 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b6dad28a60 0x55b6dad2ae80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:45:55.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.492+0000 7f02dc1c4740 4 client.18660 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:45:55.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.492+0000 7f02dc1c4740 1 -- 172.21.15.86:0/3206840103 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b6dab8c590 con 0x55b6dad28a60 2024-04-21T08:45:55.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.492+0000 7f02d2ffd640 1 --2- 172.21.15.86:0/3206840103 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b6dad28a60 0x55b6dad2ae80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:45:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.493+0000 7f02d2ffd640 1 --2- 172.21.15.86:0/3206840103 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b6dad28a60 0x55b6dad2ae80 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=64e2417cf0eb94dc server_cookie=66479f2837272ab8 in_seq=0 out_seq=0 2024-04-21T08:45:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.493+0000 7f02b6ff5640 10 client.18660 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.493+0000 7f02b6ff5640 1 -- 172.21.15.86:0/3206840103 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b6dab8c590 con 0x55b6dad28a60 2024-04-21T08:45:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.493+0000 7f02b6ff5640 10 client.18660 handle_command_reply: tid=0 2024-04-21T08:45:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.493+0000 7f02dc1c4740 1 client.18660 shutdown 2024-04-21T08:45:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.493+0000 7f02dc1c4740 1 -- 172.21.15.86:0/3206840103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6dacb1e90 msgr2=0x55b6dacb64f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.493+0000 7f02dc1c4740 1 --2- 172.21.15.86:0/3206840103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6dacb1e90 0x55b6dacb64f0 secure :-1 s=READY pgs=1492 cs=0 l=1 rev1=1 crypto rx=0x7f02c80ce900 tx=0x7f02c80495f0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.493+0000 7f02dc1c4740 1 -- 172.21.15.86:0/3206840103 shutdown_connections 2024-04-21T08:45:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.493+0000 7f02dc1c4740 1 --2- 172.21.15.86:0/3206840103 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b6dad28a60 0x55b6dad2ae80 crc :-1 s=READY pgs=226 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.493+0000 7f02dc1c4740 1 --2- 172.21.15.86:0/3206840103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6dacb1e90 0x55b6dacb64f0 unknown :-1 s=CLOSED pgs=1492 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.493+0000 7f02b6ff5640 0 client.18660 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:45:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.493+0000 7f02dc1c4740 1 --2- 172.21.15.86:0/3206840103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6daca4b50 0x55b6dacb5fb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.493+0000 7f02dc1c4740 1 --2- 172.21.15.86:0/3206840103 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b6daca4230 0x55b6dacb5a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.493+0000 7f02dc1c4740 1 -- 172.21.15.86:0/3206840103 >> 172.21.15.86:0/3206840103 conn(0x55b6dac9ecb0 msgr2=0x55b6dac9dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:55.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.493+0000 7f02dc1c4740 1 -- 172.21.15.86:0/3206840103 shutdown_connections 2024-04-21T08:45:55.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.493+0000 7f02dc1c4740 1 -- 172.21.15.86:0/3206840103 wait complete. 2024-04-21T08:45:55.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.493+0000 7f02dc1c4740 20 client.18660 trim_cache size 0 max 16384 2024-04-21T08:45:55.501 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:45:55.501 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:45:55.501 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-21T08:45:55.501 INFO:teuthology.orchestra.run.smithi086.stdout: "669ce31b-1ab6-497d-a00e-59a6472b37d2": { 2024-04-21T08:45:55.501 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-21T08:45:55.501 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "669ce31b-1ab6-497d-a00e-59a6472b37d2", 2024-04-21T08:45:55.501 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-21T08:45:55.502 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:45:55.502 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:45:55.502 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:45:55.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.494+0000 7f02d94f2640 1 -- 172.21.15.86:0/2689271737 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f02bc0a3d20 msgr2=0x7f02bc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:55.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.494+0000 7f02d94f2640 1 --2- 172.21.15.86:0/2689271737 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f02bc0a3d20 0x7f02bc0a61e0 secure :-1 s=READY pgs=239 cs=0 l=1 rev1=1 crypto rx=0x7f02c4009fd0 tx=0x7f02c4009b80 comp rx=0 tx=0).stop 2024-04-21T08:45:55.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.494+0000 7f02d94f2640 1 -- 172.21.15.86:0/2689271737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02d410c500 msgr2=0x7f02d419cfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:45:55.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.494+0000 7f02d94f2640 1 --2- 172.21.15.86:0/2689271737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02d410c500 0x7f02d419cfd0 secure :-1 s=READY pgs=1488 cs=0 l=1 rev1=1 crypto rx=0x7f02c800e970 tx=0x7f02c800ed70 comp rx=0 tx=0).stop 2024-04-21T08:45:55.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.494+0000 7f02d94f2640 1 -- 172.21.15.86:0/2689271737 shutdown_connections 2024-04-21T08:45:55.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.494+0000 7f02d94f2640 1 --2- 172.21.15.86:0/2689271737 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f02bc0a3d20 0x7f02bc0a61e0 unknown :-1 s=CLOSED pgs=239 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.494+0000 7f02d94f2640 1 --2- 172.21.15.86:0/2689271737 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f02d410c500 0x7f02d419cfd0 unknown :-1 s=CLOSED pgs=1488 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.494+0000 7f02d94f2640 1 --2- 172.21.15.86:0/2689271737 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f02d4103ae0 0x7f02d4198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.494+0000 7f02d94f2640 1 --2- 172.21.15.86:0/2689271737 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f02d41031c0 0x7f02d4198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:45:55.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.494+0000 7f02d94f2640 1 -- 172.21.15.86:0/2689271737 >> 172.21.15.86:0/2689271737 conn(0x7f02d4078000 msgr2=0x7f02d40fdfa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:45:55.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.494+0000 7f02d94f2640 1 -- 172.21.15.86:0/2689271737 shutdown_connections 2024-04-21T08:45:55.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:45:55.494+0000 7f02d94f2640 1 -- 172.21.15.86:0/2689271737 wait complete. 2024-04-21T08:45:55.504 INFO:tasks.cephfs.filesystem:scrub status for tag:669ce31b-1ab6-497d-a00e-59a6472b37d2 - {'path': '/', 'tag': '669ce31b-1ab6-497d-a00e-59a6472b37d2', 'options': 'recursive,force'} 2024-04-21T08:45:56.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:55 smithi088 ceph-mon[31259]: pgmap v219: 129 pgs: 129 active+clean; 3.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-21T08:45:56.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:55 smithi088 ceph-mon[31259]: scrub queued for path: / 2024-04-21T08:45:56.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:55 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:45:56.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:55 smithi088 ceph-mon[31259]: scrub summary: active paths [/] 2024-04-21T08:45:56.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:55 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:45:56.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:55 smithi179 ceph-mon[31829]: pgmap v219: 129 pgs: 129 active+clean; 3.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-21T08:45:56.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:55 smithi179 ceph-mon[31829]: scrub queued for path: / 2024-04-21T08:45:56.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:55 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:45:56.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:55 smithi179 ceph-mon[31829]: scrub summary: active paths [/] 2024-04-21T08:45:56.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:55 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:45:56.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:55 smithi086 ceph-mon[27939]: pgmap v219: 129 pgs: 129 active+clean; 3.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-21T08:45:56.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:55 smithi086 ceph-mon[27939]: scrub queued for path: / 2024-04-21T08:45:56.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:55 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:45:56.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:55 smithi086 ceph-mon[27939]: scrub summary: active paths [/] 2024-04-21T08:45:56.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:55 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:45:58.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:57 smithi088 ceph-mon[31259]: pgmap v220: 129 pgs: 129 active+clean; 3.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-21T08:45:58.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:57 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:45:58.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:57 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:45:58.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:57 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:45:58.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:57 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:45:58.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:57 smithi179 ceph-mon[31829]: pgmap v220: 129 pgs: 129 active+clean; 3.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-21T08:45:58.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:57 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:45:58.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:57 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:45:58.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:57 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:45:58.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:57 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:45:58.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:57 smithi086 ceph-mon[27939]: pgmap v220: 129 pgs: 129 active+clean; 3.7 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-21T08:45:58.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:57 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:45:58.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:57 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:45:58.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:57 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:45:58.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:57 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:46:00.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:59 smithi088 ceph-mon[31259]: pgmap v221: 129 pgs: 129 active+clean; 4.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-04-21T08:46:00.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:45:59 smithi088 ceph-mon[31259]: scrub summary: idle 2024-04-21T08:46:00.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:59 smithi179 ceph-mon[31829]: pgmap v221: 129 pgs: 129 active+clean; 4.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-04-21T08:46:00.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:45:59 smithi179 ceph-mon[31829]: scrub summary: idle 2024-04-21T08:46:00.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:59 smithi086 ceph-mon[27939]: pgmap v221: 129 pgs: 129 active+clean; 4.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-04-21T08:46:00.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:45:59 smithi086 ceph-mon[27939]: scrub summary: idle 2024-04-21T08:46:02.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:01 smithi088 ceph-mon[31259]: pgmap v222: 129 pgs: 129 active+clean; 4.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 KiB/s wr, 16 op/s 2024-04-21T08:46:02.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:01 smithi179 ceph-mon[31829]: pgmap v222: 129 pgs: 129 active+clean; 4.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 KiB/s wr, 16 op/s 2024-04-21T08:46:02.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:01 smithi086 ceph-mon[27939]: pgmap v222: 129 pgs: 129 active+clean; 4.0 MiB data, 406 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 43 KiB/s wr, 16 op/s 2024-04-21T08:46:04.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:03 smithi088 ceph-mon[31259]: pgmap v223: 129 pgs: 129 active+clean; 4.2 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 106 KiB/s wr, 40 op/s 2024-04-21T08:46:04.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:03 smithi179 ceph-mon[31829]: pgmap v223: 129 pgs: 129 active+clean; 4.2 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 106 KiB/s wr, 40 op/s 2024-04-21T08:46:04.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:03 smithi086 ceph-mon[27939]: pgmap v223: 129 pgs: 129 active+clean; 4.2 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 106 KiB/s wr, 40 op/s 2024-04-21T08:46:06.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:05 smithi088 ceph-mon[31259]: pgmap v224: 129 pgs: 129 active+clean; 4.2 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 88 KiB/s wr, 33 op/s 2024-04-21T08:46:06.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:05 smithi088 ceph-mon[31259]: osdmap e97: 12 total, 12 up, 12 in 2024-04-21T08:46:06.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:05 smithi179 ceph-mon[31829]: pgmap v224: 129 pgs: 129 active+clean; 4.2 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 88 KiB/s wr, 33 op/s 2024-04-21T08:46:06.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:05 smithi179 ceph-mon[31829]: osdmap e97: 12 total, 12 up, 12 in 2024-04-21T08:46:06.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:05 smithi086 ceph-mon[27939]: pgmap v224: 129 pgs: 129 active+clean; 4.2 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 88 KiB/s wr, 33 op/s 2024-04-21T08:46:06.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:05 smithi086 ceph-mon[27939]: osdmap e97: 12 total, 12 up, 12 in 2024-04-21T08:46:07.545 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.543+0000 7f7d58766640 1 -- 172.21.15.86:0/3874042701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d50101130 msgr2=0x7f7d501015b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.546 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.543+0000 7f7d58766640 1 --2- 172.21.15.86:0/3874042701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d50101130 0x7f7d501015b0 secure :-1 s=READY pgs=1493 cs=0 l=1 rev1=1 crypto rx=0x7f7d40009930 tx=0x7f7d4002bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:07.546 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.543+0000 7f7d58766640 1 -- 172.21.15.86:0/3874042701 shutdown_connections 2024-04-21T08:46:07.546 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.543+0000 7f7d58766640 1 --2- 172.21.15.86:0/3874042701 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d50101af0 0x7f7d5010dc60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.546 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.543+0000 7f7d58766640 1 --2- 172.21.15.86:0/3874042701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d50101130 0x7f7d501015b0 unknown :-1 s=CLOSED pgs=1493 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.546 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.543+0000 7f7d58766640 1 --2- 172.21.15.86:0/3874042701 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d50106f70 0x7f7d50107350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.546 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.543+0000 7f7d58766640 1 -- 172.21.15.86:0/3874042701 >> 172.21.15.86:0/3874042701 conn(0x7f7d50077ba0 msgr2=0x7f7d50077fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:07.546 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.543+0000 7f7d58766640 1 -- 172.21.15.86:0/3874042701 shutdown_connections 2024-04-21T08:46:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.543+0000 7f7d58766640 1 -- 172.21.15.86:0/3874042701 wait complete. 2024-04-21T08:46:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.543+0000 7f7d58766640 1 Processor -- start 2024-04-21T08:46:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.543+0000 7f7d58766640 1 -- start start 2024-04-21T08:46:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.544+0000 7f7d58766640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d50101130 0x7f7d50199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.544+0000 7f7d58766640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d50101af0 0x7f7d50199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.544+0000 7f7d58766640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d50106f70 0x7f7d501a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.544+0000 7f7d58766640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7d50111080 con 0x7f7d50101130 2024-04-21T08:46:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.544+0000 7f7d58766640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7d50110f00 con 0x7f7d50101af0 2024-04-21T08:46:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.544+0000 7f7d58766640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7d50111200 con 0x7f7d50106f70 2024-04-21T08:46:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.544+0000 7f7d56502640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d50101130 0x7f7d50199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.544+0000 7f7d56502640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d50101130 0x7f7d50199580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55800/0 (socket says 172.21.15.86:55800) 2024-04-21T08:46:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.544+0000 7f7d56502640 1 -- 172.21.15.86:0/2666271032 learned_addr learned my addr 172.21.15.86:0/2666271032 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.544+0000 7f7d56d03640 1 --2- 172.21.15.86:0/2666271032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d50106f70 0x7f7d501a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.547 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.544+0000 7f7d55d01640 1 --2- 172.21.15.86:0/2666271032 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d50101af0 0x7f7d50199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.544+0000 7f7d56502640 1 -- 172.21.15.86:0/2666271032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d50106f70 msgr2=0x7f7d501a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.544+0000 7f7d56502640 1 --2- 172.21.15.86:0/2666271032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d50106f70 0x7f7d501a4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.544+0000 7f7d56502640 1 -- 172.21.15.86:0/2666271032 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d50101af0 msgr2=0x7f7d50199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.544+0000 7f7d56502640 1 --2- 172.21.15.86:0/2666271032 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d50101af0 0x7f7d50199ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.544+0000 7f7d56502640 1 -- 172.21.15.86:0/2666271032 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f7d501a4b40 con 0x7f7d50101130 2024-04-21T08:46:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.544+0000 7f7d56502640 1 --2- 172.21.15.86:0/2666271032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d50101130 0x7f7d50199580 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c00cce0 tx=0x7f7d4c00a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.544+0000 7f7d477fe640 1 -- 172.21.15.86:0/2666271032 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c004b20 con 0x7f7d50101130 2024-04-21T08:46:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.544+0000 7f7d58766640 1 -- 172.21.15.86:0/2666271032 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f7d501a4d10 con 0x7f7d50101130 2024-04-21T08:46:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.544+0000 7f7d477fe640 1 -- 172.21.15.86:0/2666271032 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d4c004cc0 con 0x7f7d50101130 2024-04-21T08:46:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.545+0000 7f7d477fe640 1 -- 172.21.15.86:0/2666271032 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c00f6b0 con 0x7f7d50101130 2024-04-21T08:46:07.548 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.545+0000 7f7d58766640 1 -- 172.21.15.86:0/2666271032 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f7d501a51d0 con 0x7f7d50101130 2024-04-21T08:46:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.547+0000 7f7d477fe640 1 -- 172.21.15.86:0/2666271032 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7d4c00f850 con 0x7f7d50101130 2024-04-21T08:46:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.547+0000 7f7d5b431740 1 Processor -- start 2024-04-21T08:46:07.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.547+0000 7f7d5b431740 1 -- start start 2024-04-21T08:46:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.547+0000 7f7d477fe640 1 --2- 172.21.15.86:0/2666271032 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7d3c0a3dc0 0x7f7d3c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.547+0000 7f7d5b431740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8c37ab8d0 0x55f8c37aa620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.548+0000 7f7d5b431740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c37aab60 0x55f8c37aafe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.548+0000 7f7d5b431740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8c37b40e0 0x55f8c37b64d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.548+0000 7f7d5b431740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f8c370f8d0 con 0x55f8c37aab60 2024-04-21T08:46:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.548+0000 7f7d5b431740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55f8c36ace80 con 0x55f8c37b40e0 2024-04-21T08:46:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.548+0000 7f7d5b431740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f8c370b890 con 0x55f8c37ab8d0 2024-04-21T08:46:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.548+0000 7f7d56d03640 1 --2- 172.21.15.86:0/2666271032 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7d3c0a3dc0 0x7f7d3c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.548+0000 7f7d55d01640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c37aab60 0x55f8c37aafe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.548+0000 7f7d55d01640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c37aab60 0x55f8c37aafe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55812/0 (socket says 172.21.15.86:55812) 2024-04-21T08:46:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.548+0000 7f7d55d01640 1 -- 172.21.15.86:0/3682614664 learned_addr learned my addr 172.21.15.86:0/3682614664 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.548+0000 7f7d477fe640 1 -- 172.21.15.86:0/2666271032 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(97..97 src has 1..97) ==== 9584+0+0 (secure 0 0 0) 0x7f7d4c0d0d20 con 0x7f7d50101130 2024-04-21T08:46:07.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.548+0000 7f7d56502640 1 --2- 172.21.15.86:0/3682614664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8c37ab8d0 0x55f8c37aa620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.548+0000 7f7d56d03640 1 --2- 172.21.15.86:0/3682614664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8c37b40e0 0x55f8c37b64d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.548+0000 7f7d55d01640 1 -- 172.21.15.86:0/3682614664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8c37ab8d0 msgr2=0x55f8c37aa620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.548+0000 7f7d55d01640 1 --2- 172.21.15.86:0/3682614664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8c37ab8d0 0x55f8c37aa620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.548+0000 7f7d55d01640 1 -- 172.21.15.86:0/3682614664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8c37b40e0 msgr2=0x55f8c37b64d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.548+0000 7f7d55d01640 1 --2- 172.21.15.86:0/3682614664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8c37b40e0 0x55f8c37b64d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.548+0000 7f7d55d01640 1 -- 172.21.15.86:0/3682614664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8c36ce320 con 0x55f8c37aab60 2024-04-21T08:46:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.548+0000 7f7d56502640 1 --2- 172.21.15.86:0/3682614664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8c37ab8d0 0x55f8c37aa620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.548+0000 7f7d56d03640 1 --2- 172.21.15.86:0/2666271032 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7d3c0a3dc0 0x7f7d3c0a6280 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7d48013400 tx=0x7f7d48013800 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.548+0000 7f7d55d01640 1 --2- 172.21.15.86:0/3682614664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c37aab60 0x55f8c37aafe0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f7d40035000 tx=0x7f7d40009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=447f15f5d4fa4652 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:07.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.548+0000 7f7d44ff9640 1 -- 172.21.15.86:0/3682614664 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4002c780 con 0x55f8c37aab60 2024-04-21T08:46:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.549+0000 7f7d44ff9640 1 -- 172.21.15.86:0/3682614664 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d4002c920 con 0x55f8c37aab60 2024-04-21T08:46:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.549+0000 7f7d44ff9640 1 -- 172.21.15.86:0/3682614664 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d400046c0 con 0x55f8c37aab60 2024-04-21T08:46:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.549+0000 7f7d5b431740 1 -- 172.21.15.86:0/3682614664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c37aab60 msgr2=0x55f8c37aafe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.549+0000 7f7d5b431740 1 --2- 172.21.15.86:0/3682614664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c37aab60 0x55f8c37aafe0 secure :-1 s=READY pgs=1495 cs=0 l=1 rev1=1 crypto rx=0x7f7d40035000 tx=0x7f7d40009680 comp rx=0 tx=0).stop 2024-04-21T08:46:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.549+0000 7f7d5b431740 1 -- 172.21.15.86:0/3682614664 shutdown_connections 2024-04-21T08:46:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.549+0000 7f7d5b431740 1 --2- 172.21.15.86:0/3682614664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8c37b40e0 0x55f8c37b64d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.549+0000 7f7d5b431740 1 --2- 172.21.15.86:0/3682614664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c37aab60 0x55f8c37aafe0 unknown :-1 s=CLOSED pgs=1495 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.549+0000 7f7d5b431740 1 --2- 172.21.15.86:0/3682614664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8c37ab8d0 0x55f8c37aa620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.549+0000 7f7d5b431740 1 -- 172.21.15.86:0/3682614664 >> 172.21.15.86:0/3682614664 conn(0x55f8c3771c90 msgr2=0x55f8c37a5850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.549+0000 7f7d5b431740 1 -- 172.21.15.86:0/3682614664 shutdown_connections 2024-04-21T08:46:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.549+0000 7f7d5b431740 1 -- 172.21.15.86:0/3682614664 wait complete. 2024-04-21T08:46:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.550+0000 7f7d5b431740 1 -- start start 2024-04-21T08:46:07.558 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.550+0000 7f7d5b431740 1 Processor -- start 2024-04-21T08:46:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.550+0000 7f7d5b431740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8c37aab60 0x55f8c37b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.550+0000 7f7d5b431740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8c37ab8d0 0x55f8c37b0e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.550+0000 7f7d5b431740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c37b40e0 0x55f8c37b13c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.550+0000 7f7d5b431740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f8c37b6db0 con 0x55f8c37b40e0 2024-04-21T08:46:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.550+0000 7f7d5b431740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55f8c3697eb0 con 0x55f8c37aab60 2024-04-21T08:46:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.550+0000 7f7d5b431740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f8c37b6f30 con 0x55f8c37ab8d0 2024-04-21T08:46:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.550+0000 7f7d5b431740 10 client.0 _ll_register_callbacks cb 0x55f8c36c4b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.550+0000 7f7d56d03640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c37b40e0 0x55f8c37b13c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.550+0000 7f7d56d03640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c37b40e0 0x55f8c37b13c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55814/0 (socket says 172.21.15.86:55814) 2024-04-21T08:46:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.550+0000 7f7d56d03640 1 -- 172.21.15.86:0/286473297 learned_addr learned my addr 172.21.15.86:0/286473297 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:07.559 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.550+0000 7f7d56502640 1 --2- 172.21.15.86:0/286473297 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8c37aab60 0x55f8c37b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.550+0000 7f7d56d03640 1 -- 172.21.15.86:0/286473297 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8c37ab8d0 msgr2=0x55f8c37b0e20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.550+0000 7f7d56d03640 1 --2- 172.21.15.86:0/286473297 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8c37ab8d0 0x55f8c37b0e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.550+0000 7f7d56d03640 1 -- 172.21.15.86:0/286473297 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8c37aab60 msgr2=0x55f8c37b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.550+0000 7f7d56d03640 1 --2- 172.21.15.86:0/286473297 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8c37aab60 0x55f8c37b3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.550+0000 7f7d56d03640 1 -- 172.21.15.86:0/286473297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8c3422210 con 0x55f8c37b40e0 2024-04-21T08:46:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.551+0000 7f7d56d03640 1 --2- 172.21.15.86:0/286473297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c37b40e0 0x55f8c37b13c0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f7d48011660 tx=0x7f7d48004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3591925779f7ce0a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.551+0000 7f7d39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:46:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.551+0000 7f7d39ffb640 1 -- 172.21.15.86:0/286473297 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d48024030 con 0x55f8c37b40e0 2024-04-21T08:46:07.560 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.551+0000 7f7d39ffb640 1 -- 172.21.15.86:0/286473297 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9584+0+0 (secure 0 0 0) 0x7f7d4801f710 con 0x55f8c37b40e0 2024-04-21T08:46:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.551+0000 7f7d5b431740 1 -- 172.21.15.86:0/286473297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d4002c920 con 0x55f8c37b40e0 2024-04-21T08:46:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.551+0000 7f7d39ffb640 1 -- 172.21.15.86:0/286473297 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7d48031050 con 0x55f8c37b40e0 2024-04-21T08:46:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.551+0000 7f7d5b431740 10 client.18672 fetch_fsmap learned FSMap version 29 2024-04-21T08:46:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.551+0000 7f7d5b431740 1 -- 172.21.15.86:0/286473297 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f8c34ee620 con 0x55f8c37b40e0 2024-04-21T08:46:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.551+0000 7f7d39ffb640 1 -- 172.21.15.86:0/286473297 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d4002c920 con 0x55f8c37b40e0 2024-04-21T08:46:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.551+0000 7f7d39ffb640 1 -- 172.21.15.86:0/286473297 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7d480209f0 con 0x55f8c37b40e0 2024-04-21T08:46:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.551+0000 7f7d5b431740 10 client.18672 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:46:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.551+0000 7f7d5b431740 10 client.18672 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.551+0000 7f7d5b431740 20 client.18672 populate_metadata read hostname 'smithi086' 2024-04-21T08:46:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.551+0000 7f7d5b431740 1 --2- 172.21.15.86:0/286473297 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f8c37c4100 0x55f8c37c6550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.551+0000 7f7d5b431740 4 client.18672 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.551+0000 7f7d5b431740 1 -- 172.21.15.86:0/286473297 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f8c3723880 con 0x55f8c37c4100 2024-04-21T08:46:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.552+0000 7f7d56502640 1 --2- 172.21.15.86:0/286473297 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f8c37c4100 0x55f8c37c6550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.552+0000 7f7d39ffb640 10 client.18672 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:07.561 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.552+0000 7f7d56502640 1 --2- 172.21.15.86:0/286473297 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f8c37c4100 0x55f8c37c6550 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=679cd275b308a8fd server_cookie=165074e99f15a089 in_seq=0 out_seq=0 2024-04-21T08:46:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.553+0000 7f7d39ffb640 1 -- 172.21.15.86:0/286473297 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f8c3723880 con 0x55f8c37c4100 2024-04-21T08:46:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.553+0000 7f7d39ffb640 10 client.18672 handle_command_reply: tid=0 2024-04-21T08:46:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.553+0000 7f7d5b431740 1 client.18672 shutdown 2024-04-21T08:46:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.553+0000 7f7d5b431740 1 -- 172.21.15.86:0/286473297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c37b40e0 msgr2=0x55f8c37b13c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.553+0000 7f7d5b431740 1 --2- 172.21.15.86:0/286473297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c37b40e0 0x55f8c37b13c0 secure :-1 s=READY pgs=1496 cs=0 l=1 rev1=1 crypto rx=0x7f7d48011660 tx=0x7f7d48004730 comp rx=0 tx=0).stop 2024-04-21T08:46:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.553+0000 7f7d5b431740 1 -- 172.21.15.86:0/286473297 shutdown_connections 2024-04-21T08:46:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.553+0000 7f7d5b431740 1 --2- 172.21.15.86:0/286473297 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f8c37c4100 0x55f8c37c6550 crc :-1 s=READY pgs=227 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.553+0000 7f7d5b431740 1 --2- 172.21.15.86:0/286473297 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c37b40e0 0x55f8c37b13c0 unknown :-1 s=CLOSED pgs=1496 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.553+0000 7f7d5b431740 1 --2- 172.21.15.86:0/286473297 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8c37ab8d0 0x55f8c37b0e20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.553+0000 7f7d39ffb640 0 client.18672 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.553+0000 7f7d5b431740 1 --2- 172.21.15.86:0/286473297 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8c37aab60 0x55f8c37b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.553+0000 7f7d5b431740 1 -- 172.21.15.86:0/286473297 >> 172.21.15.86:0/286473297 conn(0x55f8c3771c90 msgr2=0x55f8c37a75a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:07.562 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.553+0000 7f7d5b431740 1 -- 172.21.15.86:0/286473297 shutdown_connections 2024-04-21T08:46:07.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.553+0000 7f7d5b431740 1 -- 172.21.15.86:0/286473297 wait complete. 2024-04-21T08:46:07.563 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.553+0000 7f7d5b431740 20 client.18672 trim_cache size 0 max 16384 2024-04-21T08:46:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.568+0000 7f7d5b431740 1 Processor -- start 2024-04-21T08:46:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.568+0000 7f7d5b431740 1 -- start start 2024-04-21T08:46:07.573 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.568+0000 7f7d5b431740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c37aab60 0x55f8c37bfcb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.568+0000 7f7d5b431740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8c37ab8d0 0x55f8c37b3550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.568+0000 7f7d5b431740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8c37b40e0 0x55f8c37b3a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.568+0000 7f7d5b431740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7d501af550 con 0x55f8c37aab60 2024-04-21T08:46:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.568+0000 7f7d5b431740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7d501b1090 con 0x55f8c37ab8d0 2024-04-21T08:46:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.568+0000 7f7d5b431740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7d501afc20 con 0x55f8c37b40e0 2024-04-21T08:46:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.568+0000 7f7d56502640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c37aab60 0x55f8c37bfcb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.568+0000 7f7d56502640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c37aab60 0x55f8c37bfcb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55828/0 (socket says 172.21.15.86:55828) 2024-04-21T08:46:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.568+0000 7f7d56502640 1 -- 172.21.15.86:0/1519073017 learned_addr learned my addr 172.21.15.86:0/1519073017 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.568+0000 7f7d55d01640 1 --2- 172.21.15.86:0/1519073017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8c37ab8d0 0x55f8c37b3550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.568+0000 7f7d56d03640 1 --2- 172.21.15.86:0/1519073017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8c37b40e0 0x55f8c37b3a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.569+0000 7f7d56502640 1 -- 172.21.15.86:0/1519073017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8c37b40e0 msgr2=0x55f8c37b3a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.574 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.569+0000 7f7d56502640 1 --2- 172.21.15.86:0/1519073017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8c37b40e0 0x55f8c37b3a90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.569+0000 7f7d56502640 1 -- 172.21.15.86:0/1519073017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8c37ab8d0 msgr2=0x55f8c37b3550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.569+0000 7f7d56502640 1 --2- 172.21.15.86:0/1519073017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8c37ab8d0 0x55f8c37b3550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.569+0000 7f7d56502640 1 -- 172.21.15.86:0/1519073017 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f8c37a9e30 con 0x55f8c37aab60 2024-04-21T08:46:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.569+0000 7f7d56d03640 1 --2- 172.21.15.86:0/1519073017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8c37b40e0 0x55f8c37b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.569+0000 7f7d56502640 1 --2- 172.21.15.86:0/1519073017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c37aab60 0x55f8c37bfcb0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c099270 tx=0x7f7d4c087040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=633e783a27961ccf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.569+0000 7f7d3bfff640 1 -- 172.21.15.86:0/1519073017 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c01d040 con 0x55f8c37aab60 2024-04-21T08:46:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.569+0000 7f7d3bfff640 1 -- 172.21.15.86:0/1519073017 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d4c0a0c50 con 0x55f8c37aab60 2024-04-21T08:46:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.569+0000 7f7d3bfff640 1 -- 172.21.15.86:0/1519073017 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c09ac30 con 0x55f8c37aab60 2024-04-21T08:46:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.570+0000 7f7d5b431740 1 -- 172.21.15.86:0/1519073017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c37aab60 msgr2=0x55f8c37bfcb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.570+0000 7f7d5b431740 1 --2- 172.21.15.86:0/1519073017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c37aab60 0x55f8c37bfcb0 secure :-1 s=READY pgs=1497 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c099270 tx=0x7f7d4c087040 comp rx=0 tx=0).stop 2024-04-21T08:46:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.570+0000 7f7d5b431740 1 -- 172.21.15.86:0/1519073017 shutdown_connections 2024-04-21T08:46:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.570+0000 7f7d5b431740 1 --2- 172.21.15.86:0/1519073017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8c37b40e0 0x55f8c37b3a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.570+0000 7f7d5b431740 1 --2- 172.21.15.86:0/1519073017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8c37ab8d0 0x55f8c37b3550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.575 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.570+0000 7f7d5b431740 1 --2- 172.21.15.86:0/1519073017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c37aab60 0x55f8c37bfcb0 unknown :-1 s=CLOSED pgs=1497 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.570+0000 7f7d5b431740 1 -- 172.21.15.86:0/1519073017 >> 172.21.15.86:0/1519073017 conn(0x55f8c3771c90 msgr2=0x55f8c37a5380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.570+0000 7f7d5b431740 1 -- 172.21.15.86:0/1519073017 shutdown_connections 2024-04-21T08:46:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.570+0000 7f7d5b431740 1 -- 172.21.15.86:0/1519073017 wait complete. 2024-04-21T08:46:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.570+0000 7f7d5b431740 1 -- start start 2024-04-21T08:46:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.570+0000 7f7d5b431740 1 Processor -- start 2024-04-21T08:46:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.570+0000 7f7d5b431740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c37aab60 0x55f8c37b4dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.570+0000 7f7d5b431740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8c37ab8d0 0x55f8c37b5310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.570+0000 7f7d5b431740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8c37b40e0 0x55f8c37b5850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.570+0000 7f7d5b431740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7d501b09c0 con 0x55f8c37aab60 2024-04-21T08:46:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.570+0000 7f7d5b431740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7d501b02f0 con 0x55f8c37ab8d0 2024-04-21T08:46:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.570+0000 7f7d5b431740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7d501ad530 con 0x55f8c37b40e0 2024-04-21T08:46:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.570+0000 7f7d5b431740 10 client.0 _ll_register_callbacks cb 0x7f7d48011570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.570+0000 7f7d56502640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c37aab60 0x55f8c37b4dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.571+0000 7f7d56502640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c37aab60 0x55f8c37b4dd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55840/0 (socket says 172.21.15.86:55840) 2024-04-21T08:46:07.576 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.571+0000 7f7d56502640 1 -- 172.21.15.86:0/1867433910 learned_addr learned my addr 172.21.15.86:0/1867433910 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.571+0000 7f7d56d03640 1 --2- 172.21.15.86:0/1867433910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8c37b40e0 0x55f8c37b5850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.571+0000 7f7d55d01640 1 --2- 172.21.15.86:0/1867433910 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8c37ab8d0 0x55f8c37b5310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.571+0000 7f7d56502640 1 -- 172.21.15.86:0/1867433910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8c37b40e0 msgr2=0x55f8c37b5850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.571+0000 7f7d56502640 1 --2- 172.21.15.86:0/1867433910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8c37b40e0 0x55f8c37b5850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.571+0000 7f7d56502640 1 -- 172.21.15.86:0/1867433910 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8c37ab8d0 msgr2=0x55f8c37b5310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.571+0000 7f7d56502640 1 --2- 172.21.15.86:0/1867433910 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8c37ab8d0 0x55f8c37b5310 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.571+0000 7f7d56502640 1 -- 172.21.15.86:0/1867433910 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f8c37ba580 con 0x55f8c37aab60 2024-04-21T08:46:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.571+0000 7f7d56502640 1 --2- 172.21.15.86:0/1867433910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c37aab60 0x55f8c37b4dd0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c004f90 tx=0x7f7d4c09c9d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=40576f1e5723c144 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.571+0000 7f7d39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:46:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.571+0000 7f7d39ffb640 1 -- 172.21.15.86:0/1867433910 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7d4c0ba030 con 0x55f8c37aab60 2024-04-21T08:46:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.571+0000 7f7d39ffb640 1 -- 172.21.15.86:0/1867433910 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(97..97 src has 1..97) ==== 9584+0+0 (secure 0 0 0) 0x7f7d4c0870b0 con 0x55f8c37aab60 2024-04-21T08:46:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.571+0000 7f7d5b431740 1 -- 172.21.15.86:0/1867433910 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7d4c0a0c50 con 0x55f8c37aab60 2024-04-21T08:46:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.571+0000 7f7d39ffb640 1 -- 172.21.15.86:0/1867433910 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f7d4c01d040 con 0x55f8c37aab60 2024-04-21T08:46:07.577 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.571+0000 7f7d5b431740 10 client.18678 fetch_fsmap learned FSMap version 29 2024-04-21T08:46:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.571+0000 7f7d5b431740 1 -- 172.21.15.86:0/1867433910 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f8c382aab0 con 0x55f8c37aab60 2024-04-21T08:46:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.571+0000 7f7d39ffb640 1 -- 172.21.15.86:0/1867433910 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7d4c0a0c50 con 0x55f8c37aab60 2024-04-21T08:46:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.572+0000 7f7d39ffb640 1 -- 172.21.15.86:0/1867433910 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7d4c00fb40 con 0x55f8c37aab60 2024-04-21T08:46:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.572+0000 7f7d5b431740 10 client.18678 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:46:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.572+0000 7f7d5b431740 10 client.18678 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.572+0000 7f7d5b431740 20 client.18678 populate_metadata read hostname 'smithi086' 2024-04-21T08:46:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.572+0000 7f7d5b431740 1 --2- 172.21.15.86:0/1867433910 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f8c382b110 0x55f8c382d530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.572+0000 7f7d5b431740 4 client.18678 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:46:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.572+0000 7f7d5b431740 1 -- 172.21.15.86:0/1867433910 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f8c36af3d0 con 0x55f8c382b110 2024-04-21T08:46:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.572+0000 7f7d55d01640 1 --2- 172.21.15.86:0/1867433910 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f8c382b110 0x55f8c382d530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.572+0000 7f7d55d01640 1 --2- 172.21.15.86:0/1867433910 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f8c382b110 0x55f8c382d530 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=93dff6c95a08d5f9 server_cookie=9ce2af2dd92d99c4 in_seq=0 out_seq=0 2024-04-21T08:46:07.578 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.572+0000 7f7d39ffb640 10 client.18678 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:07.588 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.587+0000 7f7d39ffb640 1 -- 172.21.15.86:0/1867433910 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f8c36af3d0 con 0x55f8c382b110 2024-04-21T08:46:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.587+0000 7f7d39ffb640 10 client.18678 handle_command_reply: tid=0 2024-04-21T08:46:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.587+0000 7f7d5b431740 1 client.18678 shutdown 2024-04-21T08:46:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.588+0000 7f7d5b431740 1 -- 172.21.15.86:0/1867433910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c37aab60 msgr2=0x55f8c37b4dd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.588+0000 7f7d5b431740 1 --2- 172.21.15.86:0/1867433910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c37aab60 0x55f8c37b4dd0 secure :-1 s=READY pgs=1498 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c004f90 tx=0x7f7d4c09c9d0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.588+0000 7f7d5b431740 1 -- 172.21.15.86:0/1867433910 shutdown_connections 2024-04-21T08:46:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.588+0000 7f7d5b431740 1 --2- 172.21.15.86:0/1867433910 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f8c382b110 0x55f8c382d530 crc :-1 s=READY pgs=228 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.588+0000 7f7d5b431740 1 --2- 172.21.15.86:0/1867433910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8c37b40e0 0x55f8c37b5850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.588+0000 7f7d39ffb640 0 client.18678 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.588+0000 7f7d5b431740 1 --2- 172.21.15.86:0/1867433910 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8c37ab8d0 0x55f8c37b5310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.588+0000 7f7d5b431740 1 --2- 172.21.15.86:0/1867433910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8c37aab60 0x55f8c37b4dd0 unknown :-1 s=CLOSED pgs=1498 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.588+0000 7f7d5b431740 1 -- 172.21.15.86:0/1867433910 >> 172.21.15.86:0/1867433910 conn(0x55f8c3771c90 msgr2=0x55f8c37af630 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.588+0000 7f7d5b431740 1 -- 172.21.15.86:0/1867433910 shutdown_connections 2024-04-21T08:46:07.589 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.588+0000 7f7d5b431740 1 -- 172.21.15.86:0/1867433910 wait complete. 2024-04-21T08:46:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.588+0000 7f7d5b431740 20 client.18678 trim_cache size 0 max 16384 2024-04-21T08:46:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:46:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:46:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:46:07.590 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:46:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.588+0000 7f7d58766640 1 -- 172.21.15.86:0/2666271032 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7d3c0a3dc0 msgr2=0x7f7d3c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.588+0000 7f7d58766640 1 --2- 172.21.15.86:0/2666271032 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7d3c0a3dc0 0x7f7d3c0a6280 secure :-1 s=READY pgs=240 cs=0 l=1 rev1=1 crypto rx=0x7f7d48013400 tx=0x7f7d48013800 comp rx=0 tx=0).stop 2024-04-21T08:46:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.588+0000 7f7d58766640 1 -- 172.21.15.86:0/2666271032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d50101130 msgr2=0x7f7d50199580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.588+0000 7f7d58766640 1 --2- 172.21.15.86:0/2666271032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d50101130 0x7f7d50199580 secure :-1 s=READY pgs=1494 cs=0 l=1 rev1=1 crypto rx=0x7f7d4c00cce0 tx=0x7f7d4c00a620 comp rx=0 tx=0).stop 2024-04-21T08:46:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.589+0000 7f7d58766640 1 -- 172.21.15.86:0/2666271032 shutdown_connections 2024-04-21T08:46:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.589+0000 7f7d58766640 1 --2- 172.21.15.86:0/2666271032 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f7d3c0a3dc0 0x7f7d3c0a6280 unknown :-1 s=CLOSED pgs=240 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.589+0000 7f7d58766640 1 --2- 172.21.15.86:0/2666271032 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7d50106f70 0x7f7d501a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.591 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.589+0000 7f7d58766640 1 --2- 172.21.15.86:0/2666271032 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7d50101af0 0x7f7d50199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.589+0000 7f7d58766640 1 --2- 172.21.15.86:0/2666271032 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7d50101130 0x7f7d50199580 unknown :-1 s=CLOSED pgs=1494 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.589+0000 7f7d58766640 1 -- 172.21.15.86:0/2666271032 >> 172.21.15.86:0/2666271032 conn(0x7f7d50077ba0 msgr2=0x7f7d500ffce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.589+0000 7f7d58766640 1 -- 172.21.15.86:0/2666271032 shutdown_connections 2024-04-21T08:46:07.592 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:07.589+0000 7f7d58766640 1 -- 172.21.15.86:0/2666271032 wait complete. 2024-04-21T08:46:08.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:07 smithi088 ceph-mon[31259]: pgmap v226: 129 pgs: 129 active+clean; 4.2 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 106 KiB/s wr, 40 op/s 2024-04-21T08:46:08.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:07 smithi179 ceph-mon[31829]: pgmap v226: 129 pgs: 129 active+clean; 4.2 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 106 KiB/s wr, 40 op/s 2024-04-21T08:46:08.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:07 smithi086 ceph-mon[27939]: pgmap v226: 129 pgs: 129 active+clean; 4.2 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 106 KiB/s wr, 40 op/s 2024-04-21T08:46:10.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:09 smithi088 ceph-mon[31259]: pgmap v227: 129 pgs: 129 active+clean; 4.4 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-21T08:46:10.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:09 smithi179 ceph-mon[31829]: pgmap v227: 129 pgs: 129 active+clean; 4.4 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-21T08:46:10.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:09 smithi086 ceph-mon[27939]: pgmap v227: 129 pgs: 129 active+clean; 4.4 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-21T08:46:11.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:11 smithi086 ceph-mon[27939]: osdmap e98: 12 total, 12 up, 12 in 2024-04-21T08:46:11.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:11 smithi088 ceph-mon[31259]: osdmap e98: 12 total, 12 up, 12 in 2024-04-21T08:46:11.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:11 smithi179 ceph-mon[31829]: osdmap e98: 12 total, 12 up, 12 in 2024-04-21T08:46:12.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:12 smithi086 ceph-mon[27939]: pgmap v229: 129 pgs: 129 active+clean; 4.4 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 38 KiB/s wr, 13 op/s 2024-04-21T08:46:12.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:12 smithi088 ceph-mon[31259]: pgmap v229: 129 pgs: 129 active+clean; 4.4 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 38 KiB/s wr, 13 op/s 2024-04-21T08:46:12.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:12 smithi179 ceph-mon[31829]: pgmap v229: 129 pgs: 129 active+clean; 4.4 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 38 KiB/s wr, 13 op/s 2024-04-21T08:46:14.870 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:14 smithi086 ceph-mon[27939]: pgmap v230: 129 pgs: 129 active+clean; 4.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 62 KiB/s wr, 22 op/s 2024-04-21T08:46:14.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:14 smithi088 ceph-mon[31259]: pgmap v230: 129 pgs: 129 active+clean; 4.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 62 KiB/s wr, 22 op/s 2024-04-21T08:46:14.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:14 smithi179 ceph-mon[31829]: pgmap v230: 129 pgs: 129 active+clean; 4.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 62 KiB/s wr, 22 op/s 2024-04-21T08:46:16.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:16 smithi086 ceph-mon[27939]: pgmap v231: 129 pgs: 129 active+clean; 4.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-04-21T08:46:16.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:16 smithi088 ceph-mon[31259]: pgmap v231: 129 pgs: 129 active+clean; 4.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-04-21T08:46:16.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:16 smithi179 ceph-mon[31829]: pgmap v231: 129 pgs: 129 active+clean; 4.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-04-21T08:46:18.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:18 smithi086 ceph-mon[27939]: pgmap v232: 129 pgs: 129 active+clean; 4.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-21T08:46:18.966 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:18 smithi088 ceph-mon[31259]: pgmap v232: 129 pgs: 129 active+clean; 4.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-21T08:46:18.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:18 smithi179 ceph-mon[31829]: pgmap v232: 129 pgs: 129 active+clean; 4.6 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 50 KiB/s wr, 18 op/s 2024-04-21T08:46:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.683+0000 7fefb1b2e640 1 -- 172.21.15.86:0/4269557837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefac102b80 msgr2=0x7fefac106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.683+0000 7fefb1b2e640 1 --2- 172.21.15.86:0/4269557837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefac102b80 0x7fefac106bc0 secure :-1 s=READY pgs=1499 cs=0 l=1 rev1=1 crypto rx=0x7fefa0009900 tx=0x7fefa002bc50 comp rx=0 tx=0).stop 2024-04-21T08:46:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.683+0000 7fefb1b2e640 1 -- 172.21.15.86:0/4269557837 shutdown_connections 2024-04-21T08:46:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.683+0000 7fefb1b2e640 1 --2- 172.21.15.86:0/4269557837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fefac1072a0 0x7fefac10ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.684 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.683+0000 7fefb1b2e640 1 --2- 172.21.15.86:0/4269557837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefac102b80 0x7fefac106bc0 unknown :-1 s=CLOSED pgs=1499 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.683+0000 7fefb1b2e640 1 --2- 172.21.15.86:0/4269557837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fefac1021d0 0x7fefac1025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.683+0000 7fefb1b2e640 1 -- 172.21.15.86:0/4269557837 >> 172.21.15.86:0/4269557837 conn(0x7fefac0793e0 msgr2=0x7fefac0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.683+0000 7fefb1b2e640 1 -- 172.21.15.86:0/4269557837 shutdown_connections 2024-04-21T08:46:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.683+0000 7fefb1b2e640 1 -- 172.21.15.86:0/4269557837 wait complete. 2024-04-21T08:46:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.683+0000 7fefb1b2e640 1 Processor -- start 2024-04-21T08:46:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.683+0000 7fefb1b2e640 1 -- start start 2024-04-21T08:46:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.684+0000 7fefb1b2e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefac1021d0 0x7fefac198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.684+0000 7fefb1b2e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fefac1072a0 0x7fefac198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.684+0000 7fefb1b2e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fefac19cfe0 0x7fefac19d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.684+0000 7fefb1b2e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fefac112260 con 0x7fefac1021d0 2024-04-21T08:46:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.684+0000 7fefb1b2e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fefac1120e0 con 0x7fefac1072a0 2024-04-21T08:46:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.684+0000 7fefb1b2e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fefac1123e0 con 0x7fefac19cfe0 2024-04-21T08:46:20.685 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.684+0000 7fefab7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefac1021d0 0x7fefac198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.684+0000 7fefab7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefac1021d0 0x7fefac198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47352/0 (socket says 172.21.15.86:47352) 2024-04-21T08:46:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.684+0000 7fefab7fe640 1 -- 172.21.15.86:0/130128996 learned_addr learned my addr 172.21.15.86:0/130128996 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.684+0000 7fefaaffd640 1 --2- 172.21.15.86:0/130128996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fefac1072a0 0x7fefac198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.684+0000 7fefabfff640 1 --2- 172.21.15.86:0/130128996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fefac19cfe0 0x7fefac19d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.684+0000 7fefab7fe640 1 -- 172.21.15.86:0/130128996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fefac19cfe0 msgr2=0x7fefac19d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.684+0000 7fefab7fe640 1 --2- 172.21.15.86:0/130128996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fefac19cfe0 0x7fefac19d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.684+0000 7fefab7fe640 1 -- 172.21.15.86:0/130128996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fefac1072a0 msgr2=0x7fefac198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.684+0000 7fefab7fe640 1 --2- 172.21.15.86:0/130128996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fefac1072a0 0x7fefac198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.684+0000 7fefab7fe640 1 -- 172.21.15.86:0/130128996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fefac19dad0 con 0x7fefac1021d0 2024-04-21T08:46:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.684+0000 7fefab7fe640 1 --2- 172.21.15.86:0/130128996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefac1021d0 0x7fefac198720 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00b450 tx=0x7fef9c00b910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.684+0000 7fefa8ff9640 1 -- 172.21.15.86:0/130128996 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c015020 con 0x7fefac1021d0 2024-04-21T08:46:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.684+0000 7fefa8ff9640 1 -- 172.21.15.86:0/130128996 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef9c011d40 con 0x7fefac1021d0 2024-04-21T08:46:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.684+0000 7fefb1b2e640 1 -- 172.21.15.86:0/130128996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fefac19ddc0 con 0x7fefac1021d0 2024-04-21T08:46:20.686 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.684+0000 7fefa8ff9640 1 -- 172.21.15.86:0/130128996 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c00f340 con 0x7fefac1021d0 2024-04-21T08:46:20.687 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.684+0000 7fefb1b2e640 1 -- 172.21.15.86:0/130128996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fefac1a5580 con 0x7fefac1021d0 2024-04-21T08:46:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.687+0000 7fefa8ff9640 1 -- 172.21.15.86:0/130128996 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fef9c018020 con 0x7fefac1021d0 2024-04-21T08:46:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.687+0000 7fefb47f4740 1 Processor -- start 2024-04-21T08:46:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.687+0000 7fefb47f4740 1 -- start start 2024-04-21T08:46:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.687+0000 7fefb47f4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a5e9ce8d0 0x563a5e9cd620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.687+0000 7fefb47f4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5e9cdb60 0x563a5e9cdfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.687+0000 7fefb47f4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5e9d70e0 0x563a5e9d94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.687+0000 7fefb47f4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563a5e9328d0 con 0x563a5e9cdb60 2024-04-21T08:46:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.687+0000 7fefb47f4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x563a5e8cfe80 con 0x563a5e9ce8d0 2024-04-21T08:46:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.687+0000 7fefb47f4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563a5e92e890 con 0x563a5e9d70e0 2024-04-21T08:46:20.694 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.687+0000 7fefa8ff9640 1 --2- 172.21.15.86:0/130128996 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fef940a3dc0 0x7fef940a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:20.695 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.687+0000 7fefab7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5e9cdb60 0x563a5e9cdfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:20.695 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.687+0000 7fefab7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5e9cdb60 0x563a5e9cdfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47356/0 (socket says 172.21.15.86:47356) 2024-04-21T08:46:20.695 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.687+0000 7fefab7fe640 1 -- 172.21.15.86:0/2023430851 learned_addr learned my addr 172.21.15.86:0/2023430851 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:20.695 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.687+0000 7fefabfff640 1 --2- 172.21.15.86:0/2023430851 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a5e9ce8d0 0x563a5e9cd620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:20.695 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.687+0000 7fefabfff640 1 --2- 172.21.15.86:0/130128996 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fef940a3dc0 0x7fef940a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:20.695 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.687+0000 7fefa8ff9640 1 -- 172.21.15.86:0/130128996 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fef9c0d04d0 con 0x7fefac1021d0 2024-04-21T08:46:20.695 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.688+0000 7fefabfff640 1 -- 172.21.15.86:0/2023430851 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5e9d70e0 msgr2=0x563a5e9d94d0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:46:20.695 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.688+0000 7fefabfff640 1 --2- 172.21.15.86:0/2023430851 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5e9d70e0 0x563a5e9d94d0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.695 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.688+0000 7fefabfff640 1 -- 172.21.15.86:0/2023430851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5e9cdb60 msgr2=0x563a5e9cdfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:20.695 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.688+0000 7fefabfff640 1 --2- 172.21.15.86:0/2023430851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5e9cdb60 0x563a5e9cdfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.695 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.688+0000 7fefabfff640 1 -- 172.21.15.86:0/2023430851 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a5e8f1320 con 0x563a5e9ce8d0 2024-04-21T08:46:20.695 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.688+0000 7fefab7fe640 1 --2- 172.21.15.86:0/2023430851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5e9cdb60 0x563a5e9cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:46:20.695 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.688+0000 7fefabfff640 1 --2- 172.21.15.86:0/130128996 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fef940a3dc0 0x7fef940a6280 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fefa4005c40 tx=0x7fefa401d000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:20.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.688+0000 7fefabfff640 1 --2- 172.21.15.86:0/2023430851 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a5e9ce8d0 0x563a5e9cd620 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fefa4012cb0 tx=0x7fefa40107a0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e50038c2cf1facc1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:20.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.688+0000 7fef91ffb640 1 -- 172.21.15.86:0/2023430851 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa40124d0 con 0x563a5e9ce8d0 2024-04-21T08:46:20.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.688+0000 7fef91ffb640 1 -- 172.21.15.86:0/2023430851 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefa40127c0 con 0x563a5e9ce8d0 2024-04-21T08:46:20.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.688+0000 7fef91ffb640 1 -- 172.21.15.86:0/2023430851 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa4003bd0 con 0x563a5e9ce8d0 2024-04-21T08:46:20.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.689+0000 7fefb47f4740 1 -- 172.21.15.86:0/2023430851 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a5e9ce8d0 msgr2=0x563a5e9cd620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:20.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.689+0000 7fefb47f4740 1 --2- 172.21.15.86:0/2023430851 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a5e9ce8d0 0x563a5e9cd620 secure :-1 s=READY pgs=75 cs=0 l=1 rev1=1 crypto rx=0x7fefa4012cb0 tx=0x7fefa40107a0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.689+0000 7fefb47f4740 1 -- 172.21.15.86:0/2023430851 shutdown_connections 2024-04-21T08:46:20.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.689+0000 7fefb47f4740 1 --2- 172.21.15.86:0/2023430851 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5e9d70e0 0x563a5e9d94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.689+0000 7fefb47f4740 1 --2- 172.21.15.86:0/2023430851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5e9cdb60 0x563a5e9cdfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.689+0000 7fefb47f4740 1 --2- 172.21.15.86:0/2023430851 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a5e9ce8d0 0x563a5e9cd620 unknown :-1 s=CLOSED pgs=75 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.689+0000 7fefb47f4740 1 -- 172.21.15.86:0/2023430851 >> 172.21.15.86:0/2023430851 conn(0x563a5e994c90 msgr2=0x563a5e9c8850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:20.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.689+0000 7fefb47f4740 1 -- 172.21.15.86:0/2023430851 shutdown_connections 2024-04-21T08:46:20.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.689+0000 7fefb47f4740 1 -- 172.21.15.86:0/2023430851 wait complete. 2024-04-21T08:46:20.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.689+0000 7fefb47f4740 1 -- start start 2024-04-21T08:46:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.689+0000 7fefb47f4740 1 Processor -- start 2024-04-21T08:46:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.690+0000 7fefb47f4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5e9cdb60 0x563a5e9d6aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.690+0000 7fefb47f4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5e9ce8d0 0x563a5e9d3e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.690+0000 7fefb47f4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a5e9d70e0 0x563a5e9d4440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.690+0000 7fefb47f4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563a5e9d9d50 con 0x563a5e9cdb60 2024-04-21T08:46:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.690+0000 7fefb47f4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x563a5e8baeb0 con 0x563a5e9d70e0 2024-04-21T08:46:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.690+0000 7fefb47f4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563a5e9d9ed0 con 0x563a5e9ce8d0 2024-04-21T08:46:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.690+0000 7fefb47f4740 10 client.0 _ll_register_callbacks cb 0x563a5e8e7b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.690+0000 7fefab7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5e9cdb60 0x563a5e9d6aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:20.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.690+0000 7fefab7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5e9cdb60 0x563a5e9d6aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47358/0 (socket says 172.21.15.86:47358) 2024-04-21T08:46:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.690+0000 7fefab7fe640 1 -- 172.21.15.86:0/1284226641 learned_addr learned my addr 172.21.15.86:0/1284226641 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.690+0000 7fefabfff640 1 --2- 172.21.15.86:0/1284226641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a5e9d70e0 0x563a5e9d4440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.690+0000 7fefaaffd640 1 --2- 172.21.15.86:0/1284226641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5e9ce8d0 0x563a5e9d3e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.690+0000 7fefab7fe640 1 -- 172.21.15.86:0/1284226641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5e9ce8d0 msgr2=0x563a5e9d3e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.690+0000 7fefab7fe640 1 --2- 172.21.15.86:0/1284226641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5e9ce8d0 0x563a5e9d3e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.690+0000 7fefab7fe640 1 -- 172.21.15.86:0/1284226641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a5e9d70e0 msgr2=0x563a5e9d4440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.690+0000 7fefab7fe640 1 --2- 172.21.15.86:0/1284226641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a5e9d70e0 0x563a5e9d4440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.690+0000 7fefab7fe640 1 -- 172.21.15.86:0/1284226641 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a5e645210 con 0x563a5e9cdb60 2024-04-21T08:46:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.690+0000 7fefab7fe640 1 --2- 172.21.15.86:0/1284226641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5e9cdb60 0x563a5e9d6aa0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fef9c000c00 tx=0x7fef9c009370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8afd3615bfe0fcd3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.690+0000 7fef8affd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:46:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.690+0000 7fef8affd640 1 -- 172.21.15.86:0/1284226641 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c0a3bc0 con 0x563a5e9cdb60 2024-04-21T08:46:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.690+0000 7fef8affd640 1 -- 172.21.15.86:0/1284226641 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fef9c0cdb70 con 0x563a5e9cdb60 2024-04-21T08:46:20.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.691+0000 7fefb47f4740 1 -- 172.21.15.86:0/1284226641 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fefa40127c0 con 0x563a5e9cdb60 2024-04-21T08:46:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.691+0000 7fef8affd640 1 -- 172.21.15.86:0/1284226641 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fef9c09c950 con 0x563a5e9cdb60 2024-04-21T08:46:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.691+0000 7fefb47f4740 10 client.18690 fetch_fsmap learned FSMap version 29 2024-04-21T08:46:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.691+0000 7fefb47f4740 1 -- 172.21.15.86:0/1284226641 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563a5e711620 con 0x563a5e9cdb60 2024-04-21T08:46:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.691+0000 7fef8affd640 1 -- 172.21.15.86:0/1284226641 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fefa40127c0 con 0x563a5e9cdb60 2024-04-21T08:46:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.691+0000 7fef8affd640 1 -- 172.21.15.86:0/1284226641 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fef9c0988c0 con 0x563a5e9cdb60 2024-04-21T08:46:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.691+0000 7fefb47f4740 10 client.18690 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:46:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.691+0000 7fefb47f4740 10 client.18690 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.691+0000 7fefb47f4740 20 client.18690 populate_metadata read hostname 'smithi086' 2024-04-21T08:46:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.691+0000 7fefb47f4740 1 --2- 172.21.15.86:0/1284226641 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563a5e9e70b0 0x563a5e9e94a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.691+0000 7fefb47f4740 4 client.18690 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.691+0000 7fefb47f4740 1 -- 172.21.15.86:0/1284226641 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563a5e946880 con 0x563a5e9e70b0 2024-04-21T08:46:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.691+0000 7fefaaffd640 1 --2- 172.21.15.86:0/1284226641 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563a5e9e70b0 0x563a5e9e94a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.692+0000 7fefaaffd640 1 --2- 172.21.15.86:0/1284226641 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563a5e9e70b0 0x563a5e9e94a0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e6bcda215639af5a server_cookie=ccc966464190c4c8 in_seq=0 out_seq=0 2024-04-21T08:46:20.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.692+0000 7fef8affd640 10 client.18690 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.692+0000 7fef8affd640 1 -- 172.21.15.86:0/1284226641 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563a5e946880 con 0x563a5e9e70b0 2024-04-21T08:46:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.692+0000 7fef8affd640 10 client.18690 handle_command_reply: tid=0 2024-04-21T08:46:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.692+0000 7fefb47f4740 1 client.18690 shutdown 2024-04-21T08:46:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.693+0000 7fefb47f4740 1 -- 172.21.15.86:0/1284226641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5e9cdb60 msgr2=0x563a5e9d6aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.693+0000 7fefb47f4740 1 --2- 172.21.15.86:0/1284226641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5e9cdb60 0x563a5e9d6aa0 secure :-1 s=READY pgs=1501 cs=0 l=1 rev1=1 crypto rx=0x7fef9c000c00 tx=0x7fef9c009370 comp rx=0 tx=0).stop 2024-04-21T08:46:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.693+0000 7fefb47f4740 1 -- 172.21.15.86:0/1284226641 shutdown_connections 2024-04-21T08:46:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.693+0000 7fefb47f4740 1 --2- 172.21.15.86:0/1284226641 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563a5e9e70b0 0x563a5e9e94a0 crc :-1 s=READY pgs=229 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.693+0000 7fefb47f4740 1 --2- 172.21.15.86:0/1284226641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a5e9d70e0 0x563a5e9d4440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.693+0000 7fef8affd640 0 client.18690 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.693+0000 7fefb47f4740 1 --2- 172.21.15.86:0/1284226641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5e9ce8d0 0x563a5e9d3e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.693+0000 7fefb47f4740 1 --2- 172.21.15.86:0/1284226641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5e9cdb60 0x563a5e9d6aa0 unknown :-1 s=CLOSED pgs=1501 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.693+0000 7fefb47f4740 1 -- 172.21.15.86:0/1284226641 >> 172.21.15.86:0/1284226641 conn(0x563a5e994c90 msgr2=0x563a5e9ca5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.693+0000 7fefb47f4740 1 -- 172.21.15.86:0/1284226641 shutdown_connections 2024-04-21T08:46:20.700 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.693+0000 7fefb47f4740 1 -- 172.21.15.86:0/1284226641 wait complete. 2024-04-21T08:46:20.701 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.693+0000 7fefb47f4740 20 client.18690 trim_cache size 0 max 16384 2024-04-21T08:46:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.707+0000 7fefb47f4740 1 Processor -- start 2024-04-21T08:46:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.707+0000 7fefb47f4740 1 -- start start 2024-04-21T08:46:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.708+0000 7fefb47f4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5e9cdb60 0x563a5e9e2c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.708+0000 7fefb47f4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5e9ce8d0 0x563a5e9d2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:20.711 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.708+0000 7fefb47f4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a5e9d70e0 0x563a5e9d6560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.708+0000 7fefb47f4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fefac1af300 con 0x563a5e9cdb60 2024-04-21T08:46:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.708+0000 7fefb47f4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fefac1b0e40 con 0x563a5e9d70e0 2024-04-21T08:46:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.708+0000 7fefb47f4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fefac1af9d0 con 0x563a5e9ce8d0 2024-04-21T08:46:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.708+0000 7fefab7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5e9cdb60 0x563a5e9e2c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.708+0000 7fefab7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5e9cdb60 0x563a5e9e2c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47364/0 (socket says 172.21.15.86:47364) 2024-04-21T08:46:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.708+0000 7fefab7fe640 1 -- 172.21.15.86:0/2766382097 learned_addr learned my addr 172.21.15.86:0/2766382097 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.708+0000 7fefabfff640 1 --2- 172.21.15.86:0/2766382097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a5e9d70e0 0x563a5e9d6560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.708+0000 7fefaaffd640 1 --2- 172.21.15.86:0/2766382097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5e9ce8d0 0x563a5e9d2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.708+0000 7fefab7fe640 1 -- 172.21.15.86:0/2766382097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5e9ce8d0 msgr2=0x563a5e9d2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.708+0000 7fefab7fe640 1 --2- 172.21.15.86:0/2766382097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5e9ce8d0 0x563a5e9d2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.708+0000 7fefab7fe640 1 -- 172.21.15.86:0/2766382097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a5e9d70e0 msgr2=0x563a5e9d6560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.708+0000 7fefab7fe640 1 --2- 172.21.15.86:0/2766382097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a5e9d70e0 0x563a5e9d6560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.708+0000 7fefab7fe640 1 -- 172.21.15.86:0/2766382097 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563a5e9cd990 con 0x563a5e9cdb60 2024-04-21T08:46:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.708+0000 7fefaaffd640 1 --2- 172.21.15.86:0/2766382097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5e9ce8d0 0x563a5e9d2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.708+0000 7fefab7fe640 1 --2- 172.21.15.86:0/2766382097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5e9cdb60 0x563a5e9e2c50 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00be10 tx=0x7fef9c0bb2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=94e862c7f5bb4a19 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.708+0000 7fef917fa640 1 -- 172.21.15.86:0/2766382097 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c0a0600 con 0x563a5e9cdb60 2024-04-21T08:46:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.708+0000 7fef917fa640 1 -- 172.21.15.86:0/2766382097 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef9c0a4ec0 con 0x563a5e9cdb60 2024-04-21T08:46:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.708+0000 7fef917fa640 1 -- 172.21.15.86:0/2766382097 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fef9c0cdae0 con 0x563a5e9cdb60 2024-04-21T08:46:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.709+0000 7fefb47f4740 1 -- 172.21.15.86:0/2766382097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5e9cdb60 msgr2=0x563a5e9e2c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.709+0000 7fefb47f4740 1 --2- 172.21.15.86:0/2766382097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5e9cdb60 0x563a5e9e2c50 secure :-1 s=READY pgs=1502 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00be10 tx=0x7fef9c0bb2b0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.709+0000 7fefb47f4740 1 -- 172.21.15.86:0/2766382097 shutdown_connections 2024-04-21T08:46:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.709+0000 7fefb47f4740 1 --2- 172.21.15.86:0/2766382097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a5e9d70e0 0x563a5e9d6560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.709+0000 7fefb47f4740 1 --2- 172.21.15.86:0/2766382097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5e9ce8d0 0x563a5e9d2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.709+0000 7fefb47f4740 1 --2- 172.21.15.86:0/2766382097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5e9cdb60 0x563a5e9e2c50 unknown :-1 s=CLOSED pgs=1502 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.709+0000 7fefb47f4740 1 -- 172.21.15.86:0/2766382097 >> 172.21.15.86:0/2766382097 conn(0x563a5e994c90 msgr2=0x563a5e9d2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:20.713 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.709+0000 7fefb47f4740 1 -- 172.21.15.86:0/2766382097 shutdown_connections 2024-04-21T08:46:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.709+0000 7fefb47f4740 1 -- 172.21.15.86:0/2766382097 wait complete. 2024-04-21T08:46:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.709+0000 7fefb47f4740 1 -- start start 2024-04-21T08:46:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.710+0000 7fefb47f4740 1 Processor -- start 2024-04-21T08:46:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.710+0000 7fefb47f4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5e9cdb60 0x563a5e9df6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.710+0000 7fefb47f4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a5e9ce8d0 0x563a5e9d09a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.710+0000 7fefb47f4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5e9d70e0 0x563a5e9d10e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.710+0000 7fefb47f4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fefac1b0770 con 0x563a5e9d70e0 2024-04-21T08:46:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.710+0000 7fefb47f4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fefac1b00a0 con 0x563a5e9ce8d0 2024-04-21T08:46:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.710+0000 7fefb47f4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fefac1ad300 con 0x563a5e9cdb60 2024-04-21T08:46:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.710+0000 7fefb47f4740 10 client.0 _ll_register_callbacks cb 0x7fef9c0d0840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.710+0000 7fefabfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5e9d70e0 0x563a5e9d10e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.710+0000 7fefab7fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5e9cdb60 0x563a5e9df6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:20.714 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.710+0000 7fefabfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5e9d70e0 0x563a5e9d10e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47378/0 (socket says 172.21.15.86:47378) 2024-04-21T08:46:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.710+0000 7fefabfff640 1 -- 172.21.15.86:0/396478591 learned_addr learned my addr 172.21.15.86:0/396478591 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.710+0000 7fefaaffd640 1 --2- 172.21.15.86:0/396478591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a5e9ce8d0 0x563a5e9d09a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.710+0000 7fefabfff640 1 -- 172.21.15.86:0/396478591 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5e9cdb60 msgr2=0x563a5e9df6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.710+0000 7fefabfff640 1 --2- 172.21.15.86:0/396478591 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5e9cdb60 0x563a5e9df6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.710+0000 7fefabfff640 1 -- 172.21.15.86:0/396478591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a5e9ce8d0 msgr2=0x563a5e9d09a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.710+0000 7fefabfff640 1 --2- 172.21.15.86:0/396478591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a5e9ce8d0 0x563a5e9d09a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.710+0000 7fefabfff640 1 -- 172.21.15.86:0/396478591 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563a5ea4c5d0 con 0x563a5e9d70e0 2024-04-21T08:46:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.710+0000 7fefabfff640 1 --2- 172.21.15.86:0/396478591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5e9d70e0 0x563a5e9d10e0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fefa4027040 tx=0x7fefa4012c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5adaf98450fc5198 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.710+0000 7fef8affd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:46:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.711+0000 7fef8affd640 1 -- 172.21.15.86:0/396478591 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fefa401b040 con 0x563a5e9d70e0 2024-04-21T08:46:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.711+0000 7fef8affd640 1 -- 172.21.15.86:0/396478591 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fefa400ddb0 con 0x563a5e9d70e0 2024-04-21T08:46:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.711+0000 7fefb47f4740 1 -- 172.21.15.86:0/396478591 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fef9c0a4ec0 con 0x563a5e9d70e0 2024-04-21T08:46:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.711+0000 7fef8affd640 1 -- 172.21.15.86:0/396478591 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fefa4003e40 con 0x563a5e9d70e0 2024-04-21T08:46:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.711+0000 7fefb47f4740 10 client.18696 fetch_fsmap learned FSMap version 29 2024-04-21T08:46:20.715 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.711+0000 7fefb47f4740 1 -- 172.21.15.86:0/396478591 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563a5ea4d6f0 con 0x563a5e9d70e0 2024-04-21T08:46:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.711+0000 7fef8affd640 1 -- 172.21.15.86:0/396478591 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fef9c0a4ec0 con 0x563a5e9d70e0 2024-04-21T08:46:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.711+0000 7fef8affd640 1 -- 172.21.15.86:0/396478591 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fefa4024030 con 0x563a5e9d70e0 2024-04-21T08:46:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.711+0000 7fefb47f4740 10 client.18696 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:46:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.711+0000 7fefb47f4740 10 client.18696 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.711+0000 7fefb47f4740 20 client.18696 populate_metadata read hostname 'smithi086' 2024-04-21T08:46:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.711+0000 7fefb47f4740 1 --2- 172.21.15.86:0/396478591 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563a5ea4dc40 0x563a5ea50060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.711+0000 7fefb47f4740 4 client.18696 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:46:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.711+0000 7fefb47f4740 1 -- 172.21.15.86:0/396478591 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563a5e8d23d0 con 0x563a5ea4dc40 2024-04-21T08:46:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.711+0000 7fefab7fe640 1 --2- 172.21.15.86:0/396478591 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563a5ea4dc40 0x563a5ea50060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.712+0000 7fefab7fe640 1 --2- 172.21.15.86:0/396478591 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563a5ea4dc40 0x563a5ea50060 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c2b432aa7e077eb server_cookie=bc53e79d5fccafe in_seq=0 out_seq=0 2024-04-21T08:46:20.716 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.712+0000 7fef8affd640 10 client.18696 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.727+0000 7fef8affd640 1 -- 172.21.15.86:0/396478591 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563a5e8d23d0 con 0x563a5ea4dc40 2024-04-21T08:46:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.727+0000 7fef8affd640 10 client.18696 handle_command_reply: tid=0 2024-04-21T08:46:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.727+0000 7fefb47f4740 1 client.18696 shutdown 2024-04-21T08:46:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.727+0000 7fefb47f4740 1 -- 172.21.15.86:0/396478591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5e9d70e0 msgr2=0x563a5e9d10e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:20.727 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.727+0000 7fefb47f4740 1 --2- 172.21.15.86:0/396478591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5e9d70e0 0x563a5e9d10e0 secure :-1 s=READY pgs=1503 cs=0 l=1 rev1=1 crypto rx=0x7fefa4027040 tx=0x7fefa4012c10 comp rx=0 tx=0).stop 2024-04-21T08:46:20.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.727+0000 7fefb47f4740 1 -- 172.21.15.86:0/396478591 shutdown_connections 2024-04-21T08:46:20.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.727+0000 7fefb47f4740 1 --2- 172.21.15.86:0/396478591 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563a5ea4dc40 0x563a5ea50060 crc :-1 s=READY pgs=230 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.727+0000 7fefb47f4740 1 --2- 172.21.15.86:0/396478591 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563a5e9d70e0 0x563a5e9d10e0 unknown :-1 s=CLOSED pgs=1503 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.727+0000 7fef8affd640 0 client.18696 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:20.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.727+0000 7fefb47f4740 1 --2- 172.21.15.86:0/396478591 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563a5e9ce8d0 0x563a5e9d09a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.727+0000 7fefb47f4740 1 --2- 172.21.15.86:0/396478591 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563a5e9cdb60 0x563a5e9df6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.727+0000 7fefb47f4740 1 -- 172.21.15.86:0/396478591 >> 172.21.15.86:0/396478591 conn(0x563a5e994c90 msgr2=0x563a5e9d2f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:20.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.728+0000 7fefb47f4740 1 -- 172.21.15.86:0/396478591 shutdown_connections 2024-04-21T08:46:20.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.728+0000 7fefb47f4740 1 -- 172.21.15.86:0/396478591 wait complete. 2024-04-21T08:46:20.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.728+0000 7fefb47f4740 20 client.18696 trim_cache size 0 max 16384 2024-04-21T08:46:20.728 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:46:20.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:46:20.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:46:20.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:46:20.729 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.728+0000 7fefb1b2e640 1 -- 172.21.15.86:0/130128996 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fef940a3dc0 msgr2=0x7fef940a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.728+0000 7fefb1b2e640 1 --2- 172.21.15.86:0/130128996 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fef940a3dc0 0x7fef940a6280 secure :-1 s=READY pgs=241 cs=0 l=1 rev1=1 crypto rx=0x7fefa4005c40 tx=0x7fefa401d000 comp rx=0 tx=0).stop 2024-04-21T08:46:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.728+0000 7fefb1b2e640 1 -- 172.21.15.86:0/130128996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefac1021d0 msgr2=0x7fefac198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.728+0000 7fefb1b2e640 1 --2- 172.21.15.86:0/130128996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefac1021d0 0x7fefac198720 secure :-1 s=READY pgs=1500 cs=0 l=1 rev1=1 crypto rx=0x7fef9c00b450 tx=0x7fef9c00b910 comp rx=0 tx=0).stop 2024-04-21T08:46:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.728+0000 7fefb1b2e640 1 -- 172.21.15.86:0/130128996 shutdown_connections 2024-04-21T08:46:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.728+0000 7fefb1b2e640 1 --2- 172.21.15.86:0/130128996 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fef940a3dc0 0x7fef940a6280 unknown :-1 s=CLOSED pgs=241 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.728+0000 7fefb1b2e640 1 --2- 172.21.15.86:0/130128996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fefac19cfe0 0x7fefac19d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.728+0000 7fefb1b2e640 1 --2- 172.21.15.86:0/130128996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fefac1072a0 0x7fefac198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.728+0000 7fefb1b2e640 1 --2- 172.21.15.86:0/130128996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fefac1021d0 0x7fefac198720 unknown :-1 s=CLOSED pgs=1500 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.728+0000 7fefb1b2e640 1 -- 172.21.15.86:0/130128996 >> 172.21.15.86:0/130128996 conn(0x7fefac0793e0 msgr2=0x7fefac0ffd40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.728+0000 7fefb1b2e640 1 -- 172.21.15.86:0/130128996 shutdown_connections 2024-04-21T08:46:20.730 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:20.728+0000 7fefb1b2e640 1 -- 172.21.15.86:0/130128996 wait complete. 2024-04-21T08:46:20.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:20 smithi086 ceph-mon[27939]: pgmap v233: 129 pgs: 129 active+clean; 4.8 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:46:20.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:20 smithi088 ceph-mon[31259]: pgmap v233: 129 pgs: 129 active+clean; 4.8 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:46:20.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:20 smithi179 ceph-mon[31829]: pgmap v233: 129 pgs: 129 active+clean; 4.8 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:46:21.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:21 smithi086 ceph-mon[27939]: pgmap v234: 129 pgs: 129 active+clean; 4.8 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:46:21.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:21 smithi088 ceph-mon[31259]: pgmap v234: 129 pgs: 129 active+clean; 4.8 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:46:21.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:21 smithi179 ceph-mon[31829]: pgmap v234: 129 pgs: 129 active+clean; 4.8 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:46:24.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:23 smithi086 ceph-mon[27939]: pgmap v235: 129 pgs: 129 active+clean; 5.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-21T08:46:24.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:23 smithi088 ceph-mon[31259]: pgmap v235: 129 pgs: 129 active+clean; 5.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-21T08:46:24.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:23 smithi179 ceph-mon[31829]: pgmap v235: 129 pgs: 129 active+clean; 5.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-21T08:46:25.507 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:46:25.507 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:46:25.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.620+0000 7f0e82b3d640 1 -- 172.21.15.86:0/1129637488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e7c102b20 msgr2=0x7f0e7c106b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:25.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.620+0000 7f0e82b3d640 1 --2- 172.21.15.86:0/1129637488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e7c102b20 0x7f0e7c106b60 secure :-1 s=READY pgs=1504 cs=0 l=1 rev1=1 crypto rx=0x7f0e70009900 tx=0x7f0e7002bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:25.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.621+0000 7f0e82b3d640 1 -- 172.21.15.86:0/1129637488 shutdown_connections 2024-04-21T08:46:25.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.621+0000 7f0e82b3d640 1 --2- 172.21.15.86:0/1129637488 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e7c107240 0x7f0e7c10ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.621+0000 7f0e82b3d640 1 --2- 172.21.15.86:0/1129637488 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e7c102b20 0x7f0e7c106b60 unknown :-1 s=CLOSED pgs=1504 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.621+0000 7f0e82b3d640 1 --2- 172.21.15.86:0/1129637488 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e7c102170 0x7f0e7c102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.621+0000 7f0e82b3d640 1 -- 172.21.15.86:0/1129637488 >> 172.21.15.86:0/1129637488 conn(0x7f0e7c0793a0 msgr2=0x7f0e7c0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:25.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.621+0000 7f0e82b3d640 1 -- 172.21.15.86:0/1129637488 shutdown_connections 2024-04-21T08:46:25.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.621+0000 7f0e82b3d640 1 -- 172.21.15.86:0/1129637488 wait complete. 2024-04-21T08:46:25.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.621+0000 7f0e82b3d640 1 Processor -- start 2024-04-21T08:46:25.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.621+0000 7f0e82b3d640 1 -- start start 2024-04-21T08:46:25.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.621+0000 7f0e82b3d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e7c102170 0x7f0e7c1986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.621+0000 7f0e82b3d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e7c107240 0x7f0e7c198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.621+0000 7f0e82b3d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e7c19cf60 0x7f0e7c19d410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.621+0000 7f0e82b3d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c112160 con 0x7f0e7c102170 2024-04-21T08:46:25.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.621+0000 7f0e82b3d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c111fe0 con 0x7f0e7c107240 2024-04-21T08:46:25.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.621+0000 7f0e82b3d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c1122e0 con 0x7f0e7c19cf60 2024-04-21T08:46:25.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.621+0000 7f0e808d9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e7c102170 0x7f0e7c1986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.621+0000 7f0e808d9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e7c102170 0x7f0e7c1986a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47400/0 (socket says 172.21.15.86:47400) 2024-04-21T08:46:25.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.621+0000 7f0e808d9640 1 -- 172.21.15.86:0/3232721971 learned_addr learned my addr 172.21.15.86:0/3232721971 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:25.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.622+0000 7f0e7bfff640 1 --2- 172.21.15.86:0/3232721971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e7c107240 0x7f0e7c198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.622+0000 7f0e808d9640 1 -- 172.21.15.86:0/3232721971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e7c19cf60 msgr2=0x7f0e7c19d410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:25.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.622+0000 7f0e808d9640 1 --2- 172.21.15.86:0/3232721971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e7c19cf60 0x7f0e7c19d410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.622+0000 7f0e808d9640 1 -- 172.21.15.86:0/3232721971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e7c107240 msgr2=0x7f0e7c198be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:25.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.622+0000 7f0e808d9640 1 --2- 172.21.15.86:0/3232721971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e7c107240 0x7f0e7c198be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.622+0000 7f0e808d9640 1 -- 172.21.15.86:0/3232721971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0e7c19da50 con 0x7f0e7c102170 2024-04-21T08:46:25.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.622+0000 7f0e7bfff640 1 --2- 172.21.15.86:0/3232721971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e7c107240 0x7f0e7c198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:25.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.622+0000 7f0e808d9640 1 --2- 172.21.15.86:0/3232721971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e7c102170 0x7f0e7c1986a0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00c920 tx=0x7f0e6c00cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:25.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.622+0000 7f0e79ffb640 1 -- 172.21.15.86:0/3232721971 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c007c70 con 0x7f0e7c102170 2024-04-21T08:46:25.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.622+0000 7f0e82b3d640 1 -- 172.21.15.86:0/3232721971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0e7c19dd40 con 0x7f0e7c102170 2024-04-21T08:46:25.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.622+0000 7f0e79ffb640 1 -- 172.21.15.86:0/3232721971 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e6c007e10 con 0x7f0e7c102170 2024-04-21T08:46:25.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.622+0000 7f0e79ffb640 1 -- 172.21.15.86:0/3232721971 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e6c018820 con 0x7f0e7c102170 2024-04-21T08:46:25.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.622+0000 7f0e82b3d640 1 -- 172.21.15.86:0/3232721971 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0e7c1a5580 con 0x7f0e7c102170 2024-04-21T08:46:25.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.625+0000 7f0e79ffb640 1 -- 172.21.15.86:0/3232721971 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0e6c00f450 con 0x7f0e7c102170 2024-04-21T08:46:25.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.625+0000 7f0e8580a740 1 Processor -- start 2024-04-21T08:46:25.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.625+0000 7f0e8580a740 1 -- start start 2024-04-21T08:46:25.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.625+0000 7f0e79ffb640 1 --2- 172.21.15.86:0/3232721971 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0e640a3d20 0x7f0e640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.625+0000 7f0e8580a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8da243230 0x55d8da243610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.625+0000 7f0e8580a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8da243b50 0x55d8da250950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.625+0000 7f0e8580a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8da250e90 0x55d8da253280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.625+0000 7f0e810da640 1 --2- 172.21.15.86:0/3232721971 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0e640a3d20 0x7f0e640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.625+0000 7f0e8580a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d8da1a81e0 con 0x55d8da250e90 2024-04-21T08:46:25.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.625+0000 7f0e8580a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d8da145390 con 0x55d8da243b50 2024-04-21T08:46:25.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.625+0000 7f0e8580a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d8da1a41a0 con 0x55d8da243230 2024-04-21T08:46:25.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.625+0000 7f0e79ffb640 1 -- 172.21.15.86:0/3232721971 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f0e6c0d0da0 con 0x7f0e7c102170 2024-04-21T08:46:25.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.626+0000 7f0e810da640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8da250e90 0x55d8da253280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.626+0000 7f0e810da640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8da250e90 0x55d8da253280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47416/0 (socket says 172.21.15.86:47416) 2024-04-21T08:46:25.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.626+0000 7f0e810da640 1 -- 172.21.15.86:0/3492395109 learned_addr learned my addr 172.21.15.86:0/3492395109 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:25.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.626+0000 7f0e7bfff640 1 --2- 172.21.15.86:0/3492395109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8da243b50 0x55d8da250950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.626+0000 7f0e808d9640 1 --2- 172.21.15.86:0/3492395109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8da243230 0x55d8da243610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.626+0000 7f0e810da640 1 --2- 172.21.15.86:0/3232721971 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0e640a3d20 0x7f0e640a61e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0e740126c0 tx=0x7f0e74012ac0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:25.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.626+0000 7f0e810da640 1 -- 172.21.15.86:0/3492395109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8da243230 msgr2=0x55d8da243610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:25.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.626+0000 7f0e810da640 1 --2- 172.21.15.86:0/3492395109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8da243230 0x55d8da243610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.626+0000 7f0e810da640 1 -- 172.21.15.86:0/3492395109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8da243b50 msgr2=0x55d8da250950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:25.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.626+0000 7f0e810da640 1 --2- 172.21.15.86:0/3492395109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8da243b50 0x55d8da250950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.626+0000 7f0e810da640 1 -- 172.21.15.86:0/3492395109 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8d9f847d0 con 0x55d8da250e90 2024-04-21T08:46:25.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.626+0000 7f0e7bfff640 1 --2- 172.21.15.86:0/3492395109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8da243b50 0x55d8da250950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:46:25.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.626+0000 7f0e808d9640 1 --2- 172.21.15.86:0/3492395109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8da243230 0x55d8da243610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:25.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.626+0000 7f0e810da640 1 --2- 172.21.15.86:0/3492395109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8da250e90 0x55d8da253280 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f0e74010920 tx=0x7f0e7401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96ac27f726399a24 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:25.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.626+0000 7f0e62ffd640 1 -- 172.21.15.86:0/3492395109 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e74004b20 con 0x55d8da250e90 2024-04-21T08:46:25.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.626+0000 7f0e62ffd640 1 -- 172.21.15.86:0/3492395109 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e7401dce0 con 0x55d8da250e90 2024-04-21T08:46:25.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.626+0000 7f0e62ffd640 1 -- 172.21.15.86:0/3492395109 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e740054b0 con 0x55d8da250e90 2024-04-21T08:46:25.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.627+0000 7f0e8580a740 1 -- 172.21.15.86:0/3492395109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8da250e90 msgr2=0x55d8da253280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:25.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.627+0000 7f0e8580a740 1 --2- 172.21.15.86:0/3492395109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8da250e90 0x55d8da253280 secure :-1 s=READY pgs=1506 cs=0 l=1 rev1=1 crypto rx=0x7f0e74010920 tx=0x7f0e7401d000 comp rx=0 tx=0).stop 2024-04-21T08:46:25.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.627+0000 7f0e8580a740 1 -- 172.21.15.86:0/3492395109 shutdown_connections 2024-04-21T08:46:25.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.627+0000 7f0e8580a740 1 --2- 172.21.15.86:0/3492395109 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8da250e90 0x55d8da253280 unknown :-1 s=CLOSED pgs=1506 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.627+0000 7f0e8580a740 1 --2- 172.21.15.86:0/3492395109 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8da243b50 0x55d8da250950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.627+0000 7f0e8580a740 1 --2- 172.21.15.86:0/3492395109 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8da243230 0x55d8da243610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.627+0000 7f0e8580a740 1 -- 172.21.15.86:0/3492395109 >> 172.21.15.86:0/3492395109 conn(0x55d8da23dcb0 msgr2=0x55d8da23e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:25.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.627+0000 7f0e8580a740 1 -- 172.21.15.86:0/3492395109 shutdown_connections 2024-04-21T08:46:25.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.627+0000 7f0e8580a740 1 -- 172.21.15.86:0/3492395109 wait complete. 2024-04-21T08:46:25.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.627+0000 7f0e8580a740 1 -- start start 2024-04-21T08:46:25.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.627+0000 7f0e8580a740 1 Processor -- start 2024-04-21T08:46:25.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.628+0000 7f0e8580a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8da243230 0x55d8da24dec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.628+0000 7f0e8580a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8da243b50 0x55d8da24e400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.628+0000 7f0e8580a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8da250e90 0x55d8da24e940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.628+0000 7f0e8580a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d8da253c70 con 0x55d8da243b50 2024-04-21T08:46:25.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.628+0000 7f0e8580a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d8da1612f0 con 0x55d8da250e90 2024-04-21T08:46:25.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.628+0000 7f0e8580a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d8da253df0 con 0x55d8da243230 2024-04-21T08:46:25.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.628+0000 7f0e8580a740 10 client.0 _ll_register_callbacks cb 0x55d8da16e580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:25.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.628+0000 7f0e7bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8da243b50 0x55d8da24e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.628+0000 7f0e7bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8da243b50 0x55d8da24e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47432/0 (socket says 172.21.15.86:47432) 2024-04-21T08:46:25.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.628+0000 7f0e7bfff640 1 -- 172.21.15.86:0/26306613 learned_addr learned my addr 172.21.15.86:0/26306613 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:25.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.628+0000 7f0e808d9640 1 --2- 172.21.15.86:0/26306613 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8da243230 0x55d8da24dec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.628+0000 7f0e810da640 1 --2- 172.21.15.86:0/26306613 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8da250e90 0x55d8da24e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.628+0000 7f0e7bfff640 1 -- 172.21.15.86:0/26306613 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8da243230 msgr2=0x55d8da24dec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:25.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.628+0000 7f0e7bfff640 1 --2- 172.21.15.86:0/26306613 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8da243230 0x55d8da24dec0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.628+0000 7f0e7bfff640 1 -- 172.21.15.86:0/26306613 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8da250e90 msgr2=0x55d8da24e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:25.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.628+0000 7f0e7bfff640 1 --2- 172.21.15.86:0/26306613 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8da250e90 0x55d8da24e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.628+0000 7f0e7bfff640 1 -- 172.21.15.86:0/26306613 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8d9eba3f0 con 0x55d8da243b50 2024-04-21T08:46:25.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.628+0000 7f0e7bfff640 1 --2- 172.21.15.86:0/26306613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8da243b50 0x55d8da24e400 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f0e70009900 tx=0x7f0e7002c9f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ece1a9fcf33b5f40 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:25.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.628+0000 7f0e5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:46:25.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.628+0000 7f0e5bfff640 1 -- 172.21.15.86:0/26306613 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e7002c5d0 con 0x55d8da243b50 2024-04-21T08:46:25.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.629+0000 7f0e5bfff640 1 -- 172.21.15.86:0/26306613 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f0e70005030 con 0x55d8da243b50 2024-04-21T08:46:25.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.629+0000 7f0e8580a740 1 -- 172.21.15.86:0/26306613 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e7401dce0 con 0x55d8da243b50 2024-04-21T08:46:25.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.629+0000 7f0e5bfff640 1 -- 172.21.15.86:0/26306613 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0e7004f050 con 0x55d8da243b50 2024-04-21T08:46:25.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.629+0000 7f0e8580a740 10 client.18708 fetch_fsmap learned FSMap version 29 2024-04-21T08:46:25.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.629+0000 7f0e8580a740 1 -- 172.21.15.86:0/26306613 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d8d9e7a0f0 con 0x55d8da243b50 2024-04-21T08:46:25.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.629+0000 7f0e5bfff640 1 -- 172.21.15.86:0/26306613 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e7401dce0 con 0x55d8da243b50 2024-04-21T08:46:25.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.629+0000 7f0e5bfff640 1 -- 172.21.15.86:0/26306613 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0e700035f0 con 0x55d8da243b50 2024-04-21T08:46:25.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.629+0000 7f0e8580a740 10 client.18708 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:46:25.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.629+0000 7f0e8580a740 10 client.18708 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:25.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.629+0000 7f0e8580a740 20 client.18708 populate_metadata read hostname 'smithi086' 2024-04-21T08:46:25.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.629+0000 7f0e8580a740 1 --2- 172.21.15.86:0/26306613 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d8da260f30 0x55d8da263370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.629+0000 7f0e8580a740 4 client.18708 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:25.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.629+0000 7f0e8580a740 1 -- 172.21.15.86:0/26306613 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d8da1b5c00 con 0x55d8da260f30 2024-04-21T08:46:25.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.629+0000 7f0e808d9640 1 --2- 172.21.15.86:0/26306613 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d8da260f30 0x55d8da263370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.630+0000 7f0e5bfff640 10 client.18708 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:25.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.630+0000 7f0e808d9640 1 --2- 172.21.15.86:0/26306613 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d8da260f30 0x55d8da263370 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb957b5e575a6622 server_cookie=e2a2968fb60eae96 in_seq=0 out_seq=0 2024-04-21T08:46:25.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.630+0000 7f0e5bfff640 1 -- 172.21.15.86:0/26306613 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d8da1b5c00 con 0x55d8da260f30 2024-04-21T08:46:25.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.630+0000 7f0e5bfff640 10 client.18708 handle_command_reply: tid=0 2024-04-21T08:46:25.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.631+0000 7f0e8580a740 1 client.18708 shutdown 2024-04-21T08:46:25.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.631+0000 7f0e8580a740 1 -- 172.21.15.86:0/26306613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8da243b50 msgr2=0x55d8da24e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:25.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.631+0000 7f0e8580a740 1 --2- 172.21.15.86:0/26306613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8da243b50 0x55d8da24e400 secure :-1 s=READY pgs=1507 cs=0 l=1 rev1=1 crypto rx=0x7f0e70009900 tx=0x7f0e7002c9f0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.631+0000 7f0e8580a740 1 -- 172.21.15.86:0/26306613 shutdown_connections 2024-04-21T08:46:25.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.631+0000 7f0e8580a740 1 --2- 172.21.15.86:0/26306613 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d8da260f30 0x55d8da263370 crc :-1 s=READY pgs=231 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.631+0000 7f0e8580a740 1 --2- 172.21.15.86:0/26306613 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8da250e90 0x55d8da24e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.631+0000 7f0e8580a740 1 --2- 172.21.15.86:0/26306613 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8da243b50 0x55d8da24e400 unknown :-1 s=CLOSED pgs=1507 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.631+0000 7f0e5bfff640 0 client.18708 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:25.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.631+0000 7f0e8580a740 1 --2- 172.21.15.86:0/26306613 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8da243230 0x55d8da24dec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.631+0000 7f0e8580a740 1 -- 172.21.15.86:0/26306613 >> 172.21.15.86:0/26306613 conn(0x55d8da23dcb0 msgr2=0x55d8da23fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:25.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.631+0000 7f0e8580a740 1 -- 172.21.15.86:0/26306613 shutdown_connections 2024-04-21T08:46:25.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.631+0000 7f0e8580a740 1 -- 172.21.15.86:0/26306613 wait complete. 2024-04-21T08:46:25.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.631+0000 7f0e8580a740 20 client.18708 trim_cache size 0 max 16384 2024-04-21T08:46:25.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.648+0000 7f0e8580a740 1 Processor -- start 2024-04-21T08:46:25.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.648+0000 7f0e8580a740 1 -- start start 2024-04-21T08:46:25.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.648+0000 7f0e8580a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8da243230 0x55d8da24ce00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.648+0000 7f0e8580a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8da243b50 0x55d8da25cba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.648+0000 7f0e8580a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8da250e90 0x55d8da24e760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.648+0000 7f0e8580a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c1ad1d0 con 0x55d8da250e90 2024-04-21T08:46:25.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.648+0000 7f0e8580a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c1aed10 con 0x55d8da243b50 2024-04-21T08:46:25.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.648+0000 7f0e8580a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c1ad8a0 con 0x55d8da243230 2024-04-21T08:46:25.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.649+0000 7f0e810da640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8da250e90 0x55d8da24e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.649+0000 7f0e810da640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8da250e90 0x55d8da24e760 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47436/0 (socket says 172.21.15.86:47436) 2024-04-21T08:46:25.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.649+0000 7f0e810da640 1 -- 172.21.15.86:0/457952059 learned_addr learned my addr 172.21.15.86:0/457952059 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:25.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.649+0000 7f0e7bfff640 1 --2- 172.21.15.86:0/457952059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8da243b50 0x55d8da25cba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.649+0000 7f0e808d9640 1 --2- 172.21.15.86:0/457952059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8da243230 0x55d8da24ce00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.649+0000 7f0e810da640 1 -- 172.21.15.86:0/457952059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8da243230 msgr2=0x55d8da24ce00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:25.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.649+0000 7f0e810da640 1 --2- 172.21.15.86:0/457952059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8da243230 0x55d8da24ce00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.649+0000 7f0e810da640 1 -- 172.21.15.86:0/457952059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8da243b50 msgr2=0x55d8da25cba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:25.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.649+0000 7f0e810da640 1 --2- 172.21.15.86:0/457952059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8da243b50 0x55d8da25cba0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.649+0000 7f0e810da640 1 -- 172.21.15.86:0/457952059 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d8da244010 con 0x55d8da250e90 2024-04-21T08:46:25.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.649+0000 7f0e808d9640 1 --2- 172.21.15.86:0/457952059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8da243230 0x55d8da24ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:25.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.649+0000 7f0e810da640 1 --2- 172.21.15.86:0/457952059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8da250e90 0x55d8da24e760 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f0e74013fd0 tx=0x7f0e74005c10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4807c4a7cbe1df66 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:25.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.649+0000 7f0e627fc640 1 -- 172.21.15.86:0/457952059 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e74004b20 con 0x55d8da250e90 2024-04-21T08:46:25.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.649+0000 7f0e627fc640 1 -- 172.21.15.86:0/457952059 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e7401dce0 con 0x55d8da250e90 2024-04-21T08:46:25.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.649+0000 7f0e627fc640 1 -- 172.21.15.86:0/457952059 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e740053c0 con 0x55d8da250e90 2024-04-21T08:46:25.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.650+0000 7f0e8580a740 1 -- 172.21.15.86:0/457952059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8da250e90 msgr2=0x55d8da24e760 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:25.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.650+0000 7f0e8580a740 1 --2- 172.21.15.86:0/457952059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8da250e90 0x55d8da24e760 secure :-1 s=READY pgs=1508 cs=0 l=1 rev1=1 crypto rx=0x7f0e74013fd0 tx=0x7f0e74005c10 comp rx=0 tx=0).stop 2024-04-21T08:46:25.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.650+0000 7f0e8580a740 1 -- 172.21.15.86:0/457952059 shutdown_connections 2024-04-21T08:46:25.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.650+0000 7f0e8580a740 1 --2- 172.21.15.86:0/457952059 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8da250e90 0x55d8da24e760 unknown :-1 s=CLOSED pgs=1508 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.650+0000 7f0e8580a740 1 --2- 172.21.15.86:0/457952059 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8da243b50 0x55d8da25cba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.650+0000 7f0e8580a740 1 --2- 172.21.15.86:0/457952059 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8da243230 0x55d8da24ce00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.650+0000 7f0e8580a740 1 -- 172.21.15.86:0/457952059 >> 172.21.15.86:0/457952059 conn(0x55d8da23dcb0 msgr2=0x55d8da23cd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:25.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.650+0000 7f0e8580a740 1 -- 172.21.15.86:0/457952059 shutdown_connections 2024-04-21T08:46:25.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.650+0000 7f0e8580a740 1 -- 172.21.15.86:0/457952059 wait complete. 2024-04-21T08:46:25.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.650+0000 7f0e8580a740 1 -- start start 2024-04-21T08:46:25.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.650+0000 7f0e8580a740 1 Processor -- start 2024-04-21T08:46:25.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.651+0000 7f0e8580a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8da243230 0x55d8da259440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.651+0000 7f0e8580a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8da243b50 0x55d8da2547b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.651+0000 7f0e8580a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8da250e90 0x55d8da254cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.651+0000 7f0e8580a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c1ae640 con 0x55d8da243b50 2024-04-21T08:46:25.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.651+0000 7f0e8580a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c1adf70 con 0x55d8da243230 2024-04-21T08:46:25.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.651+0000 7f0e8580a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0e7c1ab1d0 con 0x55d8da250e90 2024-04-21T08:46:25.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.651+0000 7f0e8580a740 10 client.0 _ll_register_callbacks cb 0x7f0e70009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:25.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.651+0000 7f0e7bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8da243b50 0x55d8da2547b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.651+0000 7f0e7bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8da243b50 0x55d8da2547b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47444/0 (socket says 172.21.15.86:47444) 2024-04-21T08:46:25.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.651+0000 7f0e7bfff640 1 -- 172.21.15.86:0/1099724997 learned_addr learned my addr 172.21.15.86:0/1099724997 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:25.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.651+0000 7f0e810da640 1 --2- 172.21.15.86:0/1099724997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8da250e90 0x55d8da254cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.651+0000 7f0e808d9640 1 --2- 172.21.15.86:0/1099724997 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8da243230 0x55d8da259440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.651+0000 7f0e7bfff640 1 -- 172.21.15.86:0/1099724997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8da250e90 msgr2=0x55d8da254cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:25.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.651+0000 7f0e7bfff640 1 --2- 172.21.15.86:0/1099724997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8da250e90 0x55d8da254cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.651+0000 7f0e7bfff640 1 -- 172.21.15.86:0/1099724997 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8da243230 msgr2=0x55d8da259440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:25.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.651+0000 7f0e7bfff640 1 --2- 172.21.15.86:0/1099724997 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8da243230 0x55d8da259440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.651+0000 7f0e7bfff640 1 -- 172.21.15.86:0/1099724997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d8da255500 con 0x55d8da243b50 2024-04-21T08:46:25.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.651+0000 7f0e7bfff640 1 --2- 172.21.15.86:0/1099724997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8da243b50 0x55d8da2547b0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f0e70005ec0 tx=0x7f0e700038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70ffff9829cc6617 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:25.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.651+0000 7f0e5bfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:46:25.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.651+0000 7f0e5bfff640 1 -- 172.21.15.86:0/1099724997 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0e70057030 con 0x55d8da243b50 2024-04-21T08:46:25.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.651+0000 7f0e5bfff640 1 -- 172.21.15.86:0/1099724997 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f0e700046c0 con 0x55d8da243b50 2024-04-21T08:46:25.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.651+0000 7f0e8580a740 1 -- 172.21.15.86:0/1099724997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0e7401dce0 con 0x55d8da243b50 2024-04-21T08:46:25.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.651+0000 7f0e5bfff640 1 -- 172.21.15.86:0/1099724997 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0e70061050 con 0x55d8da243b50 2024-04-21T08:46:25.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.651+0000 7f0e8580a740 10 client.18714 fetch_fsmap learned FSMap version 29 2024-04-21T08:46:25.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.651+0000 7f0e8580a740 1 -- 172.21.15.86:0/1099724997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d8da2c7400 con 0x55d8da243b50 2024-04-21T08:46:25.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.652+0000 7f0e5bfff640 1 -- 172.21.15.86:0/1099724997 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0e7401dce0 con 0x55d8da243b50 2024-04-21T08:46:25.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.652+0000 7f0e5bfff640 1 -- 172.21.15.86:0/1099724997 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0e70041c60 con 0x55d8da243b50 2024-04-21T08:46:25.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.652+0000 7f0e8580a740 10 client.18714 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:46:25.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.652+0000 7f0e8580a740 10 client.18714 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:25.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.652+0000 7f0e8580a740 20 client.18714 populate_metadata read hostname 'smithi086' 2024-04-21T08:46:25.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.652+0000 7f0e8580a740 1 --2- 172.21.15.86:0/1099724997 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d8da2c7bd0 0x55d8da2c9ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.652+0000 7f0e8580a740 4 client.18714 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:46:25.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.652+0000 7f0e8580a740 1 -- 172.21.15.86:0/1099724997 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d8da1afa70 con 0x55d8da2c7bd0 2024-04-21T08:46:25.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.652+0000 7f0e808d9640 1 --2- 172.21.15.86:0/1099724997 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d8da2c7bd0 0x55d8da2c9ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.652+0000 7f0e808d9640 1 --2- 172.21.15.86:0/1099724997 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d8da2c7bd0 0x55d8da2c9ff0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5c6b81ffba9af966 server_cookie=ef7861cad2daed74 in_seq=0 out_seq=0 2024-04-21T08:46:25.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.652+0000 7f0e5bfff640 10 client.18714 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:25.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.653+0000 7f0e5bfff640 1 -- 172.21.15.86:0/1099724997 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d8da1afa70 con 0x55d8da2c7bd0 2024-04-21T08:46:25.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.653+0000 7f0e5bfff640 10 client.18714 handle_command_reply: tid=0 2024-04-21T08:46:25.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.653+0000 7f0e8580a740 1 client.18714 shutdown 2024-04-21T08:46:25.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.653+0000 7f0e8580a740 1 -- 172.21.15.86:0/1099724997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8da243b50 msgr2=0x55d8da2547b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:25.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.653+0000 7f0e8580a740 1 --2- 172.21.15.86:0/1099724997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8da243b50 0x55d8da2547b0 secure :-1 s=READY pgs=1509 cs=0 l=1 rev1=1 crypto rx=0x7f0e70005ec0 tx=0x7f0e700038a0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.653+0000 7f0e8580a740 1 -- 172.21.15.86:0/1099724997 shutdown_connections 2024-04-21T08:46:25.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.653+0000 7f0e8580a740 1 --2- 172.21.15.86:0/1099724997 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d8da2c7bd0 0x55d8da2c9ff0 crc :-1 s=READY pgs=232 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.653+0000 7f0e8580a740 1 --2- 172.21.15.86:0/1099724997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d8da250e90 0x55d8da254cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.653+0000 7f0e8580a740 1 --2- 172.21.15.86:0/1099724997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d8da243b50 0x55d8da2547b0 unknown :-1 s=CLOSED pgs=1509 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.653+0000 7f0e5bfff640 0 client.18714 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:25.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.653+0000 7f0e8580a740 1 --2- 172.21.15.86:0/1099724997 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d8da243230 0x55d8da259440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.653+0000 7f0e8580a740 1 -- 172.21.15.86:0/1099724997 >> 172.21.15.86:0/1099724997 conn(0x55d8da23dcb0 msgr2=0x55d8da23f100 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:25.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.653+0000 7f0e8580a740 1 -- 172.21.15.86:0/1099724997 shutdown_connections 2024-04-21T08:46:25.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.653+0000 7f0e8580a740 1 -- 172.21.15.86:0/1099724997 wait complete. 2024-04-21T08:46:25.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.653+0000 7f0e8580a740 20 client.18714 trim_cache size 0 max 16384 2024-04-21T08:46:25.660 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:46:25.660 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-21T08:46:25.660 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-21T08:46:25.660 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:46:25.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.653+0000 7f0e82b3d640 1 -- 172.21.15.86:0/3232721971 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0e640a3d20 msgr2=0x7f0e640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:25.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.653+0000 7f0e82b3d640 1 --2- 172.21.15.86:0/3232721971 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0e640a3d20 0x7f0e640a61e0 secure :-1 s=READY pgs=242 cs=0 l=1 rev1=1 crypto rx=0x7f0e740126c0 tx=0x7f0e74012ac0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.653+0000 7f0e82b3d640 1 -- 172.21.15.86:0/3232721971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e7c102170 msgr2=0x7f0e7c1986a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:25.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.653+0000 7f0e82b3d640 1 --2- 172.21.15.86:0/3232721971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e7c102170 0x7f0e7c1986a0 secure :-1 s=READY pgs=1505 cs=0 l=1 rev1=1 crypto rx=0x7f0e6c00c920 tx=0x7f0e6c00cde0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.654+0000 7f0e82b3d640 1 -- 172.21.15.86:0/3232721971 shutdown_connections 2024-04-21T08:46:25.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.654+0000 7f0e82b3d640 1 --2- 172.21.15.86:0/3232721971 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0e640a3d20 0x7f0e640a61e0 unknown :-1 s=CLOSED pgs=242 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.654+0000 7f0e82b3d640 1 --2- 172.21.15.86:0/3232721971 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0e7c19cf60 0x7f0e7c19d410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.654+0000 7f0e82b3d640 1 --2- 172.21.15.86:0/3232721971 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0e7c107240 0x7f0e7c198be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.654+0000 7f0e82b3d640 1 --2- 172.21.15.86:0/3232721971 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0e7c102170 0x7f0e7c1986a0 unknown :-1 s=CLOSED pgs=1505 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.654+0000 7f0e82b3d640 1 -- 172.21.15.86:0/3232721971 >> 172.21.15.86:0/3232721971 conn(0x7f0e7c0793a0 msgr2=0x7f0e7c0fff60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:25.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.654+0000 7f0e82b3d640 1 -- 172.21.15.86:0/3232721971 shutdown_connections 2024-04-21T08:46:25.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.654+0000 7f0e82b3d640 1 -- 172.21.15.86:0/3232721971 wait complete. 2024-04-21T08:46:25.663 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:46:25.664 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:46:25.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.763+0000 7f84da63a640 1 -- 172.21.15.86:0/1053522909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84d4102b20 msgr2=0x7f84d4106b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:25.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.763+0000 7f84da63a640 1 --2- 172.21.15.86:0/1053522909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84d4102b20 0x7f84d4106b60 secure :-1 s=READY pgs=1510 cs=0 l=1 rev1=1 crypto rx=0x7f84c8009900 tx=0x7f84c802bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:25.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.763+0000 7f84da63a640 1 -- 172.21.15.86:0/1053522909 shutdown_connections 2024-04-21T08:46:25.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.763+0000 7f84da63a640 1 --2- 172.21.15.86:0/1053522909 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84d4107240 0x7f84d410ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.763+0000 7f84da63a640 1 --2- 172.21.15.86:0/1053522909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84d4102b20 0x7f84d4106b60 unknown :-1 s=CLOSED pgs=1510 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.763+0000 7f84da63a640 1 --2- 172.21.15.86:0/1053522909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84d4102170 0x7f84d4102550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.763+0000 7f84da63a640 1 -- 172.21.15.86:0/1053522909 >> 172.21.15.86:0/1053522909 conn(0x7f84d40793a0 msgr2=0x7f84d40797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:25.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.763+0000 7f84da63a640 1 -- 172.21.15.86:0/1053522909 shutdown_connections 2024-04-21T08:46:25.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.763+0000 7f84da63a640 1 -- 172.21.15.86:0/1053522909 wait complete. 2024-04-21T08:46:25.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.763+0000 7f84da63a640 1 Processor -- start 2024-04-21T08:46:25.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.763+0000 7f84da63a640 1 -- start start 2024-04-21T08:46:25.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.764+0000 7f84da63a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84d4102170 0x7f84d4198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.764+0000 7f84da63a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84d4107240 0x7f84d4198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.764+0000 7f84da63a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84d419cfc0 0x7f84d419d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.764+0000 7f84da63a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f84d4112180 con 0x7f84d4102170 2024-04-21T08:46:25.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.764+0000 7f84da63a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f84d4112000 con 0x7f84d419cfc0 2024-04-21T08:46:25.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.764+0000 7f84da63a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f84d4112300 con 0x7f84d4107240 2024-04-21T08:46:25.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.764+0000 7f84d3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84d4102170 0x7f84d4198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.764+0000 7f84d3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84d4102170 0x7f84d4198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47458/0 (socket says 172.21.15.86:47458) 2024-04-21T08:46:25.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.764+0000 7f84d3fff640 1 -- 172.21.15.86:0/3304419958 learned_addr learned my addr 172.21.15.86:0/3304419958 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:25.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.764+0000 7f84d8bd7640 1 --2- 172.21.15.86:0/3304419958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84d419cfc0 0x7f84d419d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.764+0000 7f84d37fe640 1 --2- 172.21.15.86:0/3304419958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84d4107240 0x7f84d4198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.764+0000 7f84d3fff640 1 -- 172.21.15.86:0/3304419958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84d4107240 msgr2=0x7f84d4198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:25.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.764+0000 7f84d3fff640 1 --2- 172.21.15.86:0/3304419958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84d4107240 0x7f84d4198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.764+0000 7f84d3fff640 1 -- 172.21.15.86:0/3304419958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84d419cfc0 msgr2=0x7f84d419d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:25.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.764+0000 7f84d3fff640 1 --2- 172.21.15.86:0/3304419958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84d419cfc0 0x7f84d419d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.764+0000 7f84d3fff640 1 -- 172.21.15.86:0/3304419958 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f84d419dab0 con 0x7f84d4102170 2024-04-21T08:46:25.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.764+0000 7f84d3fff640 1 --2- 172.21.15.86:0/3304419958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84d4102170 0x7f84d4198700 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f84c400c970 tx=0x7f84c400ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:25.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.765+0000 7f84d17fa640 1 -- 172.21.15.86:0/3304419958 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c4007bb0 con 0x7f84d4102170 2024-04-21T08:46:25.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.765+0000 7f84d17fa640 1 -- 172.21.15.86:0/3304419958 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f84c4007d50 con 0x7f84d4102170 2024-04-21T08:46:25.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.765+0000 7f84da63a640 1 -- 172.21.15.86:0/3304419958 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f84d419dda0 con 0x7f84d4102170 2024-04-21T08:46:25.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.765+0000 7f84d17fa640 1 -- 172.21.15.86:0/3304419958 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f84c400f450 con 0x7f84d4102170 2024-04-21T08:46:25.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.765+0000 7f84da63a640 1 -- 172.21.15.86:0/3304419958 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f84d41a5510 con 0x7f84d4102170 2024-04-21T08:46:25.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.766+0000 7f84da63a640 1 -- 172.21.15.86:0/3304419958 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f84d4069230 con 0x7f84d4102170 2024-04-21T08:46:25.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.767+0000 7f84d17fa640 1 -- 172.21.15.86:0/3304419958 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f84c4004f20 con 0x7f84d4102170 2024-04-21T08:46:25.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.767+0000 7f84d17fa640 1 --2- 172.21.15.86:0/3304419958 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f84bc0a3d20 0x7f84bc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:25.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.767+0000 7f84d37fe640 1 --2- 172.21.15.86:0/3304419958 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f84bc0a3d20 0x7f84bc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:25.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.767+0000 7f84d17fa640 1 -- 172.21.15.86:0/3304419958 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f84c40d0950 con 0x7f84d4102170 2024-04-21T08:46:25.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.768+0000 7f84d37fe640 1 --2- 172.21.15.86:0/3304419958 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f84bc0a3d20 0x7f84bc0a61e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f84c8006010 tx=0x7f84c8004040 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:25.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.772+0000 7f84d17fa640 1 -- 172.21.15.86:0/3304419958 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f84c4014030 con 0x7f84d4102170 2024-04-21T08:46:25.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.990+0000 7f84da63a640 1 -- 172.21.15.86:0/3304419958 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f84d410a490 con 0x7f84d4102170 2024-04-21T08:46:25.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.991+0000 7f84d17fa640 1 -- 172.21.15.86:0/3304419958 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7f84c409cc20 con 0x7f84d4102170 2024-04-21T08:46:25.990 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:46:25.991 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:38:45.202126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24398,24394,24455,24472]},"id":1}]} 2024-04-21T08:46:25.991 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-21T08:46:25.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.993+0000 7f84da63a640 1 -- 172.21.15.86:0/3304419958 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f84bc0a3d20 msgr2=0x7f84bc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:25.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.993+0000 7f84da63a640 1 --2- 172.21.15.86:0/3304419958 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f84bc0a3d20 0x7f84bc0a61e0 secure :-1 s=READY pgs=243 cs=0 l=1 rev1=1 crypto rx=0x7f84c8006010 tx=0x7f84c8004040 comp rx=0 tx=0).stop 2024-04-21T08:46:25.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.993+0000 7f84da63a640 1 -- 172.21.15.86:0/3304419958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84d4102170 msgr2=0x7f84d4198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:25.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.993+0000 7f84da63a640 1 --2- 172.21.15.86:0/3304419958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84d4102170 0x7f84d4198700 secure :-1 s=READY pgs=1511 cs=0 l=1 rev1=1 crypto rx=0x7f84c400c970 tx=0x7f84c400ce30 comp rx=0 tx=0).stop 2024-04-21T08:46:25.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.993+0000 7f84da63a640 1 -- 172.21.15.86:0/3304419958 shutdown_connections 2024-04-21T08:46:25.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.993+0000 7f84da63a640 1 --2- 172.21.15.86:0/3304419958 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f84bc0a3d20 0x7f84bc0a61e0 unknown :-1 s=CLOSED pgs=243 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.993+0000 7f84da63a640 1 --2- 172.21.15.86:0/3304419958 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f84d419cfc0 0x7f84d419d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.993+0000 7f84da63a640 1 --2- 172.21.15.86:0/3304419958 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f84d4107240 0x7f84d4198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.993+0000 7f84da63a640 1 --2- 172.21.15.86:0/3304419958 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f84d4102170 0x7f84d4198700 unknown :-1 s=CLOSED pgs=1511 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:25.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.993+0000 7f84da63a640 1 -- 172.21.15.86:0/3304419958 >> 172.21.15.86:0/3304419958 conn(0x7f84d40793a0 msgr2=0x7f84d40fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:25.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.993+0000 7f84da63a640 1 -- 172.21.15.86:0/3304419958 shutdown_connections 2024-04-21T08:46:25.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:25.993+0000 7f84da63a640 1 -- 172.21.15.86:0/3304419958 wait complete. 2024-04-21T08:46:26.004 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:46:26.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.105+0000 7fb34162e640 1 -- 172.21.15.86:0/3736958308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb33c10e900 msgr2=0x7fb33c110cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.105+0000 7fb34162e640 1 --2- 172.21.15.86:0/3736958308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb33c10e900 0x7fb33c110cf0 secure :-1 s=READY pgs=1512 cs=0 l=1 rev1=1 crypto rx=0x7fb334009900 tx=0x7fb33402bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:26.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.105+0000 7fb34162e640 1 -- 172.21.15.86:0/3736958308 shutdown_connections 2024-04-21T08:46:26.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.105+0000 7fb34162e640 1 --2- 172.21.15.86:0/3736958308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb33c10e900 0x7fb33c110cf0 unknown :-1 s=CLOSED pgs=1512 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.105+0000 7fb34162e640 1 --2- 172.21.15.86:0/3736958308 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb33c101670 0x7fb33c10e3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.105+0000 7fb34162e640 1 --2- 172.21.15.86:0/3736958308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb33c100d50 0x7fb33c101130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.105+0000 7fb34162e640 1 -- 172.21.15.86:0/3736958308 >> 172.21.15.86:0/3736958308 conn(0x7fb33c077f50 msgr2=0x7fb33c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.105+0000 7fb34162e640 1 -- 172.21.15.86:0/3736958308 shutdown_connections 2024-04-21T08:46:26.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.105+0000 7fb34162e640 1 -- 172.21.15.86:0/3736958308 wait complete. 2024-04-21T08:46:26.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.106+0000 7fb34162e640 1 Processor -- start 2024-04-21T08:46:26.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.106+0000 7fb34162e640 1 -- start start 2024-04-21T08:46:26.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.106+0000 7fb34162e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb33c100d50 0x7fb33c1a0670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.106+0000 7fb34162e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb33c101670 0x7fb33c1a0bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.106+0000 7fb34162e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb33c10e900 0x7fb33c19a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.106+0000 7fb34162e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb33c1143c0 con 0x7fb33c100d50 2024-04-21T08:46:26.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.106+0000 7fb34162e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb33c114240 con 0x7fb33c10e900 2024-04-21T08:46:26.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.106+0000 7fb34162e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb33c114540 con 0x7fb33c101670 2024-04-21T08:46:26.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.106+0000 7fb33affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb33c100d50 0x7fb33c1a0670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.106+0000 7fb33affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb33c100d50 0x7fb33c1a0670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47476/0 (socket says 172.21.15.86:47476) 2024-04-21T08:46:26.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.106+0000 7fb33affd640 1 -- 172.21.15.86:0/2697370600 learned_addr learned my addr 172.21.15.86:0/2697370600 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:26.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.107+0000 7fb33b7fe640 1 --2- 172.21.15.86:0/2697370600 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb33c10e900 0x7fb33c19a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.107+0000 7fb33affd640 1 -- 172.21.15.86:0/2697370600 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb33c101670 msgr2=0x7fb33c1a0bb0 unknown :-1 s=STATE_CONNECTING_RE l=1).mark_down 2024-04-21T08:46:26.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.107+0000 7fb33affd640 1 --2- 172.21.15.86:0/2697370600 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb33c101670 0x7fb33c1a0bb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.107+0000 7fb33affd640 1 -- 172.21.15.86:0/2697370600 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb33c10e900 msgr2=0x7fb33c19a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.107+0000 7fb33affd640 1 --2- 172.21.15.86:0/2697370600 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb33c10e900 0x7fb33c19a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.107+0000 7fb33affd640 1 -- 172.21.15.86:0/2697370600 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb33c19b040 con 0x7fb33c100d50 2024-04-21T08:46:26.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.107+0000 7fb33b7fe640 1 --2- 172.21.15.86:0/2697370600 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb33c10e900 0x7fb33c19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:26.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.107+0000 7fb33affd640 1 --2- 172.21.15.86:0/2697370600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb33c100d50 0x7fb33c1a0670 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00eb90 tx=0x7fb32c00c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.107+0000 7fb323fff640 1 -- 172.21.15.86:0/2697370600 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32c004d20 con 0x7fb33c100d50 2024-04-21T08:46:26.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.107+0000 7fb323fff640 1 -- 172.21.15.86:0/2697370600 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb32c00ee90 con 0x7fb33c100d50 2024-04-21T08:46:26.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.107+0000 7fb34162e640 1 -- 172.21.15.86:0/2697370600 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb33c19b330 con 0x7fb33c100d50 2024-04-21T08:46:26.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.107+0000 7fb323fff640 1 -- 172.21.15.86:0/2697370600 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb32c010670 con 0x7fb33c100d50 2024-04-21T08:46:26.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.107+0000 7fb34162e640 1 -- 172.21.15.86:0/2697370600 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb33c1a75f0 con 0x7fb33c100d50 2024-04-21T08:46:26.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.109+0000 7fb323fff640 1 -- 172.21.15.86:0/2697370600 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb32c005020 con 0x7fb33c100d50 2024-04-21T08:46:26.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.110+0000 7fb344217740 1 Processor -- start 2024-04-21T08:46:26.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.110+0000 7fb344217740 1 -- start start 2024-04-21T08:46:26.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.110+0000 7fb344217740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a4903b4d0 0x557a4903b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.110+0000 7fb344217740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557a4903be80 0x557a49044430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.110+0000 7fb344217740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a49044a00 0x557a49046e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.110+0000 7fb344217740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557a48fa01f0 con 0x557a49044a00 2024-04-21T08:46:26.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.110+0000 7fb344217740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x557a48f3d4b0 con 0x557a4903b4d0 2024-04-21T08:46:26.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.110+0000 7fb344217740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x557a48f9c1b0 con 0x557a4903be80 2024-04-21T08:46:26.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.110+0000 7fb33a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a49044a00 0x557a49046e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.110+0000 7fb33a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a49044a00 0x557a49046e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47478/0 (socket says 172.21.15.86:47478) 2024-04-21T08:46:26.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.110+0000 7fb33a7fc640 1 -- 172.21.15.86:0/1897960618 learned_addr learned my addr 172.21.15.86:0/1897960618 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:26.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.110+0000 7fb33b7fe640 1 --2- 172.21.15.86:0/1897960618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a4903b4d0 0x557a4903b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.110+0000 7fb323fff640 1 --2- 172.21.15.86:0/2697370600 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb3240a3d20 0x7fb3240a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.110+0000 7fb33affd640 1 --2- 172.21.15.86:0/1897960618 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557a4903be80 0x557a49044430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.110+0000 7fb33b7fe640 1 --2- 172.21.15.86:0/2697370600 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb3240a3d20 0x7fb3240a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.110+0000 7fb33a7fc640 1 -- 172.21.15.86:0/1897960618 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557a4903be80 msgr2=0x557a49044430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.110+0000 7fb33a7fc640 1 --2- 172.21.15.86:0/1897960618 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557a4903be80 0x557a49044430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.110+0000 7fb33a7fc640 1 -- 172.21.15.86:0/1897960618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a4903b4d0 msgr2=0x557a4903b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.110+0000 7fb33a7fc640 1 --2- 172.21.15.86:0/1897960618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a4903b4d0 0x557a4903b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.110+0000 7fb33a7fc640 1 -- 172.21.15.86:0/1897960618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a48f2f1b0 con 0x557a49044a00 2024-04-21T08:46:26.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.110+0000 7fb323fff640 1 -- 172.21.15.86:0/2697370600 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fb32c0d1360 con 0x7fb33c100d50 2024-04-21T08:46:26.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.110+0000 7fb33a7fc640 1 --2- 172.21.15.86:0/1897960618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a49044a00 0x557a49046e50 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fb330009af0 tx=0x7fb33000e350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4b1fd03afefaa3b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.111+0000 7fb3217fa640 1 -- 172.21.15.86:0/1897960618 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33000ed80 con 0x557a49044a00 2024-04-21T08:46:26.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.111+0000 7fb33b7fe640 1 --2- 172.21.15.86:0/2697370600 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb3240a3d20 0x7fb3240a61e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb334005c20 tx=0x7fb33402bc30 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.111+0000 7fb3217fa640 1 -- 172.21.15.86:0/1897960618 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3300054d0 con 0x557a49044a00 2024-04-21T08:46:26.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.111+0000 7fb3217fa640 1 -- 172.21.15.86:0/1897960618 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb330010690 con 0x557a49044a00 2024-04-21T08:46:26.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.111+0000 7fb344217740 1 -- 172.21.15.86:0/1897960618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a49044a00 msgr2=0x557a49046e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.111+0000 7fb344217740 1 --2- 172.21.15.86:0/1897960618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a49044a00 0x557a49046e50 secure :-1 s=READY pgs=1514 cs=0 l=1 rev1=1 crypto rx=0x7fb330009af0 tx=0x7fb33000e350 comp rx=0 tx=0).stop 2024-04-21T08:46:26.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.111+0000 7fb344217740 1 -- 172.21.15.86:0/1897960618 shutdown_connections 2024-04-21T08:46:26.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.111+0000 7fb344217740 1 --2- 172.21.15.86:0/1897960618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a49044a00 0x557a49046e50 unknown :-1 s=CLOSED pgs=1514 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.111+0000 7fb344217740 1 --2- 172.21.15.86:0/1897960618 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557a4903be80 0x557a49044430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.111+0000 7fb344217740 1 --2- 172.21.15.86:0/1897960618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a4903b4d0 0x557a4903b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.111+0000 7fb344217740 1 -- 172.21.15.86:0/1897960618 >> 172.21.15.86:0/1897960618 conn(0x557a49035ba0 msgr2=0x557a49035f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.111+0000 7fb344217740 1 -- 172.21.15.86:0/1897960618 shutdown_connections 2024-04-21T08:46:26.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.111+0000 7fb344217740 1 -- 172.21.15.86:0/1897960618 wait complete. 2024-04-21T08:46:26.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.112+0000 7fb344217740 1 -- start start 2024-04-21T08:46:26.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.112+0000 7fb344217740 1 Processor -- start 2024-04-21T08:46:26.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.112+0000 7fb344217740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557a4903b4d0 0x557a49044290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.112+0000 7fb344217740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a4903be80 0x557a49041620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.112+0000 7fb344217740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a49044a00 0x557a49041b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.112+0000 7fb344217740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557a49047970 con 0x557a4903be80 2024-04-21T08:46:26.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.112+0000 7fb344217740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x557a48f59500 con 0x557a49044a00 2024-04-21T08:46:26.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.112+0000 7fb344217740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x557a49047af0 con 0x557a4903b4d0 2024-04-21T08:46:26.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.112+0000 7fb344217740 10 client.0 _ll_register_callbacks cb 0x557a48f27b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:26.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.112+0000 7fb33a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a4903be80 0x557a49041620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.112+0000 7fb33a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a4903be80 0x557a49041620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47482/0 (socket says 172.21.15.86:47482) 2024-04-21T08:46:26.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.112+0000 7fb33a7fc640 1 -- 172.21.15.86:0/1072341171 learned_addr learned my addr 172.21.15.86:0/1072341171 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:26.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.112+0000 7fb33affd640 1 --2- 172.21.15.86:0/1072341171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557a4903b4d0 0x557a49044290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.112+0000 7fb33b7fe640 1 --2- 172.21.15.86:0/1072341171 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a49044a00 0x557a49041b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.112+0000 7fb33a7fc640 1 -- 172.21.15.86:0/1072341171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557a4903b4d0 msgr2=0x557a49044290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.112+0000 7fb33a7fc640 1 --2- 172.21.15.86:0/1072341171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557a4903b4d0 0x557a49044290 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.112+0000 7fb33a7fc640 1 -- 172.21.15.86:0/1072341171 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a49044a00 msgr2=0x557a49041b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.112+0000 7fb33a7fc640 1 --2- 172.21.15.86:0/1072341171 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a49044a00 0x557a49041b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.112+0000 7fb33a7fc640 1 -- 172.21.15.86:0/1072341171 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a48cb23f0 con 0x557a4903be80 2024-04-21T08:46:26.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.112+0000 7fb33b7fe640 1 --2- 172.21.15.86:0/1072341171 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a49044a00 0x557a49041b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:26.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.112+0000 7fb33a7fc640 1 --2- 172.21.15.86:0/1072341171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a4903be80 0x557a49041620 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fb330000c00 tx=0x7fb33000ee30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d8d08797e1ad68d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.112+0000 7fb31eff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:46:26.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.113+0000 7fb31eff5640 1 -- 172.21.15.86:0/1072341171 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb33001d040 con 0x557a4903be80 2024-04-21T08:46:26.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.113+0000 7fb31eff5640 1 -- 172.21.15.86:0/1072341171 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fb3300046c0 con 0x557a4903be80 2024-04-21T08:46:26.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.113+0000 7fb344217740 1 -- 172.21.15.86:0/1072341171 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3300054d0 con 0x557a4903be80 2024-04-21T08:46:26.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.113+0000 7fb31eff5640 1 -- 172.21.15.86:0/1072341171 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb33002f050 con 0x557a4903be80 2024-04-21T08:46:26.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.113+0000 7fb344217740 10 client.18732 fetch_fsmap learned FSMap version 29 2024-04-21T08:46:26.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.113+0000 7fb344217740 1 -- 172.21.15.86:0/1072341171 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557a48d7c7d0 con 0x557a4903be80 2024-04-21T08:46:26.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.113+0000 7fb31eff5640 1 -- 172.21.15.86:0/1072341171 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3300054d0 con 0x557a4903be80 2024-04-21T08:46:26.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.113+0000 7fb31eff5640 1 -- 172.21.15.86:0/1072341171 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb3300035f0 con 0x557a4903be80 2024-04-21T08:46:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.113+0000 7fb344217740 10 client.18732 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:46:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.113+0000 7fb344217740 10 client.18732 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:46:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.113+0000 7fb344217740 20 client.18732 populate_metadata read hostname 'smithi086' 2024-04-21T08:46:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.113+0000 7fb344217740 1 --2- 172.21.15.86:0/1072341171 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x557a49054ac0 0x557a49056eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.113+0000 7fb344217740 4 client.18732 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.113+0000 7fb344217740 1 -- 172.21.15.86:0/1072341171 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557a48f3f930 con 0x557a49054ac0 2024-04-21T08:46:26.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.114+0000 7fb33affd640 1 --2- 172.21.15.86:0/1072341171 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x557a49054ac0 0x557a49056eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.117+0000 7fb33affd640 1 --2- 172.21.15.86:0/1072341171 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x557a49054ac0 0x557a49056eb0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=a0321a6e72049f4e server_cookie=90ec78df6637af4e in_seq=0 out_seq=0 2024-04-21T08:46:26.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.117+0000 7fb31eff5640 10 client.18732 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:46:26.129 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:25 smithi086 ceph-mon[27939]: pgmap v236: 129 pgs: 129 active+clean; 5.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-21T08:46:26.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.118+0000 7fb31eff5640 1 -- 172.21.15.86:0/1072341171 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557a48f3f930 con 0x557a49054ac0 2024-04-21T08:46:26.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.118+0000 7fb31eff5640 10 client.18732 handle_command_reply: tid=0 2024-04-21T08:46:26.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.118+0000 7fb344217740 1 client.18732 shutdown 2024-04-21T08:46:26.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.118+0000 7fb344217740 1 -- 172.21.15.86:0/1072341171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a4903be80 msgr2=0x557a49041620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.118+0000 7fb344217740 1 --2- 172.21.15.86:0/1072341171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a4903be80 0x557a49041620 secure :-1 s=READY pgs=1515 cs=0 l=1 rev1=1 crypto rx=0x7fb330000c00 tx=0x7fb33000ee30 comp rx=0 tx=0).stop 2024-04-21T08:46:26.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.118+0000 7fb344217740 1 -- 172.21.15.86:0/1072341171 shutdown_connections 2024-04-21T08:46:26.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.118+0000 7fb344217740 1 --2- 172.21.15.86:0/1072341171 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x557a49054ac0 0x557a49056eb0 crc :-1 s=READY pgs=60 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.118+0000 7fb344217740 1 --2- 172.21.15.86:0/1072341171 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a49044a00 0x557a49041b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.118+0000 7fb344217740 1 --2- 172.21.15.86:0/1072341171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a4903be80 0x557a49041620 unknown :-1 s=CLOSED pgs=1515 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.118+0000 7fb31eff5640 0 client.18732 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:46:26.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.118+0000 7fb344217740 1 --2- 172.21.15.86:0/1072341171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557a4903b4d0 0x557a49044290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.118+0000 7fb344217740 1 -- 172.21.15.86:0/1072341171 >> 172.21.15.86:0/1072341171 conn(0x557a49035ba0 msgr2=0x557a490378f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.118+0000 7fb344217740 1 -- 172.21.15.86:0/1072341171 shutdown_connections 2024-04-21T08:46:26.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.118+0000 7fb344217740 1 -- 172.21.15.86:0/1072341171 wait complete. 2024-04-21T08:46:26.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.118+0000 7fb344217740 20 client.18732 trim_cache size 0 max 16384 2024-04-21T08:46:26.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.131+0000 7fb344217740 1 Processor -- start 2024-04-21T08:46:26.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.131+0000 7fb344217740 1 -- start start 2024-04-21T08:46:26.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.131+0000 7fb344217740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557a4903b4d0 0x557a49050830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.131+0000 7fb344217740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a4903be80 0x557a49043d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.131+0000 7fb344217740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a49044a00 0x557a49044290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.131+0000 7fb344217740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb33c1b15c0 con 0x557a49044a00 2024-04-21T08:46:26.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.131+0000 7fb344217740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb33c1b3100 con 0x557a4903be80 2024-04-21T08:46:26.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.131+0000 7fb344217740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb33c1b1c90 con 0x557a4903b4d0 2024-04-21T08:46:26.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.131+0000 7fb33b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a49044a00 0x557a49044290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.131+0000 7fb33b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a49044a00 0x557a49044290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47492/0 (socket says 172.21.15.86:47492) 2024-04-21T08:46:26.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.131+0000 7fb33b7fe640 1 -- 172.21.15.86:0/3232485977 learned_addr learned my addr 172.21.15.86:0/3232485977 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:26.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.132+0000 7fb33a7fc640 1 --2- 172.21.15.86:0/3232485977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a4903be80 0x557a49043d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.132+0000 7fb33affd640 1 --2- 172.21.15.86:0/3232485977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557a4903b4d0 0x557a49050830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.132+0000 7fb33b7fe640 1 -- 172.21.15.86:0/3232485977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557a4903b4d0 msgr2=0x557a49050830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.132+0000 7fb33b7fe640 1 --2- 172.21.15.86:0/3232485977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557a4903b4d0 0x557a49050830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.132+0000 7fb33b7fe640 1 -- 172.21.15.86:0/3232485977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a4903be80 msgr2=0x557a49043d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.132+0000 7fb33b7fe640 1 --2- 172.21.15.86:0/3232485977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a4903be80 0x557a49043d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.132+0000 7fb33b7fe640 1 -- 172.21.15.86:0/3232485977 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557a4903bc20 con 0x557a49044a00 2024-04-21T08:46:26.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.132+0000 7fb33a7fc640 1 --2- 172.21.15.86:0/3232485977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a4903be80 0x557a49043d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:26.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.132+0000 7fb33b7fe640 1 --2- 172.21.15.86:0/3232485977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a49044a00 0x557a49044290 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fb334035000 tx=0x7fb334005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38d5cfb9571ab5b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.132+0000 7fb320ff9640 1 -- 172.21.15.86:0/3232485977 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb334045040 con 0x557a49044a00 2024-04-21T08:46:26.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.132+0000 7fb320ff9640 1 -- 172.21.15.86:0/3232485977 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3340054f0 con 0x557a49044a00 2024-04-21T08:46:26.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.132+0000 7fb320ff9640 1 -- 172.21.15.86:0/3232485977 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3340035f0 con 0x557a49044a00 2024-04-21T08:46:26.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.132+0000 7fb344217740 1 -- 172.21.15.86:0/3232485977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a49044a00 msgr2=0x557a49044290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.132+0000 7fb344217740 1 --2- 172.21.15.86:0/3232485977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a49044a00 0x557a49044290 secure :-1 s=READY pgs=1516 cs=0 l=1 rev1=1 crypto rx=0x7fb334035000 tx=0x7fb334005b70 comp rx=0 tx=0).stop 2024-04-21T08:46:26.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.132+0000 7fb344217740 1 -- 172.21.15.86:0/3232485977 shutdown_connections 2024-04-21T08:46:26.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.132+0000 7fb344217740 1 --2- 172.21.15.86:0/3232485977 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a49044a00 0x557a49044290 unknown :-1 s=CLOSED pgs=1516 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.133+0000 7fb344217740 1 --2- 172.21.15.86:0/3232485977 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a4903be80 0x557a49043d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.133+0000 7fb344217740 1 --2- 172.21.15.86:0/3232485977 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557a4903b4d0 0x557a49050830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.133+0000 7fb344217740 1 -- 172.21.15.86:0/3232485977 >> 172.21.15.86:0/3232485977 conn(0x557a49035ba0 msgr2=0x557a4903cc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.133+0000 7fb344217740 1 -- 172.21.15.86:0/3232485977 shutdown_connections 2024-04-21T08:46:26.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.133+0000 7fb344217740 1 -- 172.21.15.86:0/3232485977 wait complete. 2024-04-21T08:46:26.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.133+0000 7fb344217740 1 -- start start 2024-04-21T08:46:26.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.133+0000 7fb344217740 1 Processor -- start 2024-04-21T08:46:26.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.133+0000 7fb344217740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a4903b4d0 0x557a4904cd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.133+0000 7fb344217740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a4903be80 0x557a4904d240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.133+0000 7fb344217740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557a49044a00 0x557a49048310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.133+0000 7fb344217740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb33c1b2a30 con 0x557a4903be80 2024-04-21T08:46:26.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.133+0000 7fb344217740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb33c1b2360 con 0x557a4903b4d0 2024-04-21T08:46:26.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.133+0000 7fb344217740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb33c1af5c0 con 0x557a49044a00 2024-04-21T08:46:26.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.133+0000 7fb344217740 10 client.0 _ll_register_callbacks cb 0x7fb330005d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:26.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.133+0000 7fb33a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a4903be80 0x557a4904d240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.133+0000 7fb33a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a4903be80 0x557a4904d240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47494/0 (socket says 172.21.15.86:47494) 2024-04-21T08:46:26.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.133+0000 7fb33a7fc640 1 -- 172.21.15.86:0/170568536 learned_addr learned my addr 172.21.15.86:0/170568536 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:26.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.133+0000 7fb33affd640 1 --2- 172.21.15.86:0/170568536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a4903b4d0 0x557a4904cd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.133+0000 7fb33a7fc640 1 -- 172.21.15.86:0/170568536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557a49044a00 msgr2=0x557a49048310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.133+0000 7fb33a7fc640 1 --2- 172.21.15.86:0/170568536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557a49044a00 0x557a49048310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.133+0000 7fb33a7fc640 1 -- 172.21.15.86:0/170568536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a4903b4d0 msgr2=0x557a4904cd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.134+0000 7fb33a7fc640 1 --2- 172.21.15.86:0/170568536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a4903b4d0 0x557a4904cd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.134+0000 7fb33a7fc640 1 -- 172.21.15.86:0/170568536 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557a49048b10 con 0x557a4903be80 2024-04-21T08:46:26.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.134+0000 7fb33a7fc640 1 --2- 172.21.15.86:0/170568536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a4903be80 0x557a4904d240 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fb3300184d0 tx=0x7fb330005400 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f8cdead74abeb23 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.134+0000 7fb31eff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:46:26.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.134+0000 7fb31eff5640 1 -- 172.21.15.86:0/170568536 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb3300035f0 con 0x557a4903be80 2024-04-21T08:46:26.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.134+0000 7fb31eff5640 1 -- 172.21.15.86:0/170568536 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fb330027070 con 0x557a4903be80 2024-04-21T08:46:26.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.134+0000 7fb344217740 1 -- 172.21.15.86:0/170568536 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb3340054f0 con 0x557a4903be80 2024-04-21T08:46:26.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.134+0000 7fb31eff5640 1 -- 172.21.15.86:0/170568536 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb330003790 con 0x557a4903be80 2024-04-21T08:46:26.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.134+0000 7fb344217740 10 client.18738 fetch_fsmap learned FSMap version 29 2024-04-21T08:46:26.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.134+0000 7fb344217740 1 -- 172.21.15.86:0/170568536 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557a490ba990 con 0x557a4903be80 2024-04-21T08:46:26.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.134+0000 7fb31eff5640 1 -- 172.21.15.86:0/170568536 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb3340054f0 con 0x557a4903be80 2024-04-21T08:46:26.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.134+0000 7fb31eff5640 1 -- 172.21.15.86:0/170568536 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb330018a00 con 0x557a4903be80 2024-04-21T08:46:26.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.134+0000 7fb344217740 10 client.18738 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:46:26.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.134+0000 7fb344217740 10 client.18738 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:46:26.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.134+0000 7fb344217740 20 client.18738 populate_metadata read hostname 'smithi086' 2024-04-21T08:46:26.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.134+0000 7fb344217740 1 --2- 172.21.15.86:0/170568536 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x557a490bb070 0x557a490bd490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.134+0000 7fb344217740 4 client.18738 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:46:26.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.134+0000 7fb344217740 1 -- 172.21.15.86:0/170568536 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557a48fadc10 con 0x557a490bb070 2024-04-21T08:46:26.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.135+0000 7fb33affd640 1 --2- 172.21.15.86:0/170568536 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x557a490bb070 0x557a490bd490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.135+0000 7fb31eff5640 10 client.18738 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:46:26.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.135+0000 7fb33affd640 1 --2- 172.21.15.86:0/170568536 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x557a490bb070 0x557a490bd490 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=d55e8eff90857fb5 server_cookie=71714cdac51604ea in_seq=0 out_seq=0 2024-04-21T08:46:26.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.135+0000 7fb31eff5640 1 -- 172.21.15.86:0/170568536 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557a48fadc10 con 0x557a490bb070 2024-04-21T08:46:26.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.135+0000 7fb31eff5640 10 client.18738 handle_command_reply: tid=0 2024-04-21T08:46:26.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.135+0000 7fb344217740 1 client.18738 shutdown 2024-04-21T08:46:26.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.135+0000 7fb344217740 1 -- 172.21.15.86:0/170568536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a4903be80 msgr2=0x557a4904d240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.135+0000 7fb344217740 1 --2- 172.21.15.86:0/170568536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a4903be80 0x557a4904d240 secure :-1 s=READY pgs=1517 cs=0 l=1 rev1=1 crypto rx=0x7fb3300184d0 tx=0x7fb330005400 comp rx=0 tx=0).stop 2024-04-21T08:46:26.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.135+0000 7fb344217740 1 -- 172.21.15.86:0/170568536 shutdown_connections 2024-04-21T08:46:26.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.135+0000 7fb344217740 1 --2- 172.21.15.86:0/170568536 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557a49044a00 0x557a49048310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.135+0000 7fb344217740 1 --2- 172.21.15.86:0/170568536 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557a4903be80 0x557a4904d240 unknown :-1 s=CLOSED pgs=1517 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.135+0000 7fb344217740 1 --2- 172.21.15.86:0/170568536 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x557a490bb070 0x557a490bd490 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.135+0000 7fb344217740 1 --2- 172.21.15.86:0/170568536 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557a4903b4d0 0x557a4904cd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.135+0000 7fb31eff5640 0 client.18738 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:46:26.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.135+0000 7fb344217740 1 -- 172.21.15.86:0/170568536 >> 172.21.15.86:0/170568536 conn(0x557a49035ba0 msgr2=0x557a49036f70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.135+0000 7fb344217740 1 -- 172.21.15.86:0/170568536 shutdown_connections 2024-04-21T08:46:26.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.135+0000 7fb344217740 1 -- 172.21.15.86:0/170568536 wait complete. 2024-04-21T08:46:26.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.136+0000 7fb344217740 20 client.18738 trim_cache size 0 max 16384 2024-04-21T08:46:26.145 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:46:26.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.136+0000 7fb34162e640 1 -- 172.21.15.86:0/2697370600 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb3240a3d20 msgr2=0x7fb3240a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.136+0000 7fb34162e640 1 --2- 172.21.15.86:0/2697370600 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb3240a3d20 0x7fb3240a61e0 secure :-1 s=READY pgs=244 cs=0 l=1 rev1=1 crypto rx=0x7fb334005c20 tx=0x7fb33402bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:26.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.136+0000 7fb34162e640 1 -- 172.21.15.86:0/2697370600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb33c100d50 msgr2=0x7fb33c1a0670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.136+0000 7fb34162e640 1 --2- 172.21.15.86:0/2697370600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb33c100d50 0x7fb33c1a0670 secure :-1 s=READY pgs=1513 cs=0 l=1 rev1=1 crypto rx=0x7fb32c00eb90 tx=0x7fb32c00c6a0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.136+0000 7fb34162e640 1 -- 172.21.15.86:0/2697370600 shutdown_connections 2024-04-21T08:46:26.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.136+0000 7fb34162e640 1 --2- 172.21.15.86:0/2697370600 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb3240a3d20 0x7fb3240a61e0 unknown :-1 s=CLOSED pgs=244 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.136+0000 7fb34162e640 1 --2- 172.21.15.86:0/2697370600 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb33c10e900 0x7fb33c19a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.136+0000 7fb34162e640 1 --2- 172.21.15.86:0/2697370600 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb33c101670 0x7fb33c1a0bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.136+0000 7fb34162e640 1 --2- 172.21.15.86:0/2697370600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb33c100d50 0x7fb33c1a0670 unknown :-1 s=CLOSED pgs=1513 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.136+0000 7fb34162e640 1 -- 172.21.15.86:0/2697370600 >> 172.21.15.86:0/2697370600 conn(0x7fb33c077f50 msgr2=0x7fb33c10ffe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.136+0000 7fb34162e640 1 -- 172.21.15.86:0/2697370600 shutdown_connections 2024-04-21T08:46:26.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.136+0000 7fb34162e640 1 -- 172.21.15.86:0/2697370600 wait complete. 2024-04-21T08:46:26.147 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:46:26.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:25 smithi088 ceph-mon[31259]: pgmap v236: 129 pgs: 129 active+clean; 5.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-21T08:46:26.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:25 smithi179 ceph-mon[31829]: pgmap v236: 129 pgs: 129 active+clean; 5.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-21T08:46:26.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.248+0000 7fde6865c640 1 -- 172.21.15.86:0/3884312533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde601072e0 msgr2=0x7fde6010eeb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.248+0000 7fde6865c640 1 --2- 172.21.15.86:0/3884312533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde601072e0 0x7fde6010eeb0 secure :-1 s=READY pgs=1518 cs=0 l=1 rev1=1 crypto rx=0x7fde50009900 tx=0x7fde5002bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:26.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.248+0000 7fde6865c640 1 -- 172.21.15.86:0/3884312533 shutdown_connections 2024-04-21T08:46:26.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.248+0000 7fde6865c640 1 --2- 172.21.15.86:0/3884312533 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde601072e0 0x7fde6010eeb0 unknown :-1 s=CLOSED pgs=1518 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.248+0000 7fde6865c640 1 --2- 172.21.15.86:0/3884312533 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde60102bc0 0x7fde60106c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.248+0000 7fde6865c640 1 --2- 172.21.15.86:0/3884312533 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde60102210 0x7fde601025f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.248+0000 7fde6865c640 1 -- 172.21.15.86:0/3884312533 >> 172.21.15.86:0/3884312533 conn(0x7fde600fdab0 msgr2=0x7fde600ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.248+0000 7fde6865c640 1 -- 172.21.15.86:0/3884312533 shutdown_connections 2024-04-21T08:46:26.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.248+0000 7fde6865c640 1 -- 172.21.15.86:0/3884312533 wait complete. 2024-04-21T08:46:26.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.248+0000 7fde6865c640 1 Processor -- start 2024-04-21T08:46:26.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.249+0000 7fde6865c640 1 -- start start 2024-04-21T08:46:26.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.249+0000 7fde6865c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde60102210 0x7fde6019e450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.249+0000 7fde6865c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde601072e0 0x7fde6019e990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.249+0000 7fde6865c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde60198650 0x7fde60198b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.249+0000 7fde6865c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fde60112250 con 0x7fde601072e0 2024-04-21T08:46:26.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.249+0000 7fde6865c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fde601120d0 con 0x7fde60198650 2024-04-21T08:46:26.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.249+0000 7fde6865c640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fde601123d0 con 0x7fde60102210 2024-04-21T08:46:26.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.249+0000 7fde65bf7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde601072e0 0x7fde6019e990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.249+0000 7fde65bf7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde601072e0 0x7fde6019e990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47510/0 (socket says 172.21.15.86:47510) 2024-04-21T08:46:26.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.249+0000 7fde65bf7640 1 -- 172.21.15.86:0/3681717657 learned_addr learned my addr 172.21.15.86:0/3681717657 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:26.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.249+0000 7fde663f8640 1 --2- 172.21.15.86:0/3681717657 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde60102210 0x7fde6019e450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.249+0000 7fde65bf7640 1 -- 172.21.15.86:0/3681717657 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde60102210 msgr2=0x7fde6019e450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.249+0000 7fde65bf7640 1 --2- 172.21.15.86:0/3681717657 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde60102210 0x7fde6019e450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.249+0000 7fde66bf9640 1 --2- 172.21.15.86:0/3681717657 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde60198650 0x7fde60198b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.249+0000 7fde65bf7640 1 -- 172.21.15.86:0/3681717657 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde60198650 msgr2=0x7fde60198b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.249+0000 7fde65bf7640 1 --2- 172.21.15.86:0/3681717657 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde60198650 0x7fde60198b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.249+0000 7fde65bf7640 1 -- 172.21.15.86:0/3681717657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fde60199170 con 0x7fde601072e0 2024-04-21T08:46:26.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.249+0000 7fde65bf7640 1 --2- 172.21.15.86:0/3681717657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde601072e0 0x7fde6019e990 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fde5400e9f0 tx=0x7fde5400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.249+0000 7fde5f7fe640 1 -- 172.21.15.86:0/3681717657 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5400ccc0 con 0x7fde601072e0 2024-04-21T08:46:26.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.249+0000 7fde5f7fe640 1 -- 172.21.15.86:0/3681717657 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde540054d0 con 0x7fde601072e0 2024-04-21T08:46:26.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.249+0000 7fde6865c640 1 -- 172.21.15.86:0/3681717657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fde60199460 con 0x7fde601072e0 2024-04-21T08:46:26.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.249+0000 7fde5f7fe640 1 -- 172.21.15.86:0/3681717657 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde54010430 con 0x7fde601072e0 2024-04-21T08:46:26.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.249+0000 7fde6865c640 1 -- 172.21.15.86:0/3681717657 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fde601a5660 con 0x7fde601072e0 2024-04-21T08:46:26.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.252+0000 7fde5f7fe640 1 -- 172.21.15.86:0/3681717657 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fde540046c0 con 0x7fde601072e0 2024-04-21T08:46:26.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.252+0000 7fde6b252740 1 Processor -- start 2024-04-21T08:46:26.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.252+0000 7fde6b252740 1 -- start start 2024-04-21T08:46:26.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.252+0000 7fde6b252740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f77e81b4d0 0x55f77e81b8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.252+0000 7fde6b252740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77e81be80 0x55f77e824430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.252+0000 7fde6b252740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f77e824a00 0x55f77e826e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.252+0000 7fde6b252740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f77e7801f0 con 0x55f77e81be80 2024-04-21T08:46:26.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.252+0000 7fde6b252740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55f77e71d4b0 con 0x55f77e81b4d0 2024-04-21T08:46:26.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.252+0000 7fde6b252740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f77e77c1b0 con 0x55f77e824a00 2024-04-21T08:46:26.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.252+0000 7fde663f8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77e81be80 0x55f77e824430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.252+0000 7fde5f7fe640 1 --2- 172.21.15.86:0/3681717657 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fde480a3da0 0x7fde480a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.253+0000 7fde663f8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77e81be80 0x55f77e824430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47514/0 (socket says 172.21.15.86:47514) 2024-04-21T08:46:26.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.253+0000 7fde663f8640 1 -- 172.21.15.86:0/3139038340 learned_addr learned my addr 172.21.15.86:0/3139038340 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:26.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.253+0000 7fde66bf9640 1 --2- 172.21.15.86:0/3681717657 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fde480a3da0 0x7fde480a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.253+0000 7fde66bf9640 1 --2- 172.21.15.86:0/3139038340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f77e81b4d0 0x55f77e81b8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.253+0000 7fde5f7fe640 1 -- 172.21.15.86:0/3681717657 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fde540d0f90 con 0x7fde601072e0 2024-04-21T08:46:26.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.253+0000 7fde65bf7640 1 --2- 172.21.15.86:0/3139038340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f77e824a00 0x55f77e826e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.253+0000 7fde663f8640 1 -- 172.21.15.86:0/3139038340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f77e824a00 msgr2=0x55f77e826e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.253+0000 7fde663f8640 1 --2- 172.21.15.86:0/3139038340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f77e824a00 0x55f77e826e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.253+0000 7fde663f8640 1 -- 172.21.15.86:0/3139038340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f77e81b4d0 msgr2=0x55f77e81b8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.253+0000 7fde663f8640 1 --2- 172.21.15.86:0/3139038340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f77e81b4d0 0x55f77e81b8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.253+0000 7fde663f8640 1 -- 172.21.15.86:0/3139038340 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f77e70f1b0 con 0x55f77e81be80 2024-04-21T08:46:26.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.253+0000 7fde66bf9640 1 --2- 172.21.15.86:0/3139038340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f77e81b4d0 0x55f77e81b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:26.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.253+0000 7fde663f8640 1 --2- 172.21.15.86:0/3139038340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77e81be80 0x55f77e824430 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fde58009e70 tx=0x7fde5800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c37a726aa9f4eb49 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.253+0000 7fde66bf9640 1 --2- 172.21.15.86:0/3681717657 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fde480a3da0 0x7fde480a6260 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fde500096c0 tx=0x7fde5002e040 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.253+0000 7fde5cbfe640 1 -- 172.21.15.86:0/3139038340 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde5801a680 con 0x55f77e81be80 2024-04-21T08:46:26.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.253+0000 7fde5cbfe640 1 -- 172.21.15.86:0/3139038340 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde5801acd0 con 0x55f77e81be80 2024-04-21T08:46:26.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.253+0000 7fde5cbfe640 1 -- 172.21.15.86:0/3139038340 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde58011ca0 con 0x55f77e81be80 2024-04-21T08:46:26.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.254+0000 7fde6b252740 1 -- 172.21.15.86:0/3139038340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77e81be80 msgr2=0x55f77e824430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.254+0000 7fde6b252740 1 --2- 172.21.15.86:0/3139038340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77e81be80 0x55f77e824430 secure :-1 s=READY pgs=1520 cs=0 l=1 rev1=1 crypto rx=0x7fde58009e70 tx=0x7fde5800f730 comp rx=0 tx=0).stop 2024-04-21T08:46:26.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.254+0000 7fde6b252740 1 -- 172.21.15.86:0/3139038340 shutdown_connections 2024-04-21T08:46:26.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.254+0000 7fde6b252740 1 --2- 172.21.15.86:0/3139038340 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f77e824a00 0x55f77e826e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.254+0000 7fde6b252740 1 --2- 172.21.15.86:0/3139038340 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77e81be80 0x55f77e824430 unknown :-1 s=CLOSED pgs=1520 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.254+0000 7fde6b252740 1 --2- 172.21.15.86:0/3139038340 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f77e81b4d0 0x55f77e81b8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.254+0000 7fde6b252740 1 -- 172.21.15.86:0/3139038340 >> 172.21.15.86:0/3139038340 conn(0x55f77e815ba0 msgr2=0x55f77e815f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.254+0000 7fde6b252740 1 -- 172.21.15.86:0/3139038340 shutdown_connections 2024-04-21T08:46:26.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.254+0000 7fde6b252740 1 -- 172.21.15.86:0/3139038340 wait complete. 2024-04-21T08:46:26.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.254+0000 7fde6b252740 1 -- start start 2024-04-21T08:46:26.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.254+0000 7fde6b252740 1 Processor -- start 2024-04-21T08:46:26.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.255+0000 7fde6b252740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f77e81b4d0 0x55f77e824320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.255+0000 7fde6b252740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f77e81be80 0x55f77e8216b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.255+0000 7fde6b252740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77e824a00 0x55f77e821c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.255+0000 7fde6b252740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f77e827970 con 0x55f77e824a00 2024-04-21T08:46:26.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.255+0000 7fde6b252740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55f77e739500 con 0x55f77e81b4d0 2024-04-21T08:46:26.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.255+0000 7fde6b252740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f77e827af0 con 0x55f77e81be80 2024-04-21T08:46:26.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.255+0000 7fde6b252740 10 client.0 _ll_register_callbacks cb 0x55f77e707b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:26.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.255+0000 7fde66bf9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77e824a00 0x55f77e821c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.255+0000 7fde66bf9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77e824a00 0x55f77e821c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47516/0 (socket says 172.21.15.86:47516) 2024-04-21T08:46:26.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.255+0000 7fde66bf9640 1 -- 172.21.15.86:0/2075765671 learned_addr learned my addr 172.21.15.86:0/2075765671 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:26.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.255+0000 7fde663f8640 1 --2- 172.21.15.86:0/2075765671 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f77e81b4d0 0x55f77e824320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.255+0000 7fde65bf7640 1 --2- 172.21.15.86:0/2075765671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f77e81be80 0x55f77e8216b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.255+0000 7fde66bf9640 1 -- 172.21.15.86:0/2075765671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f77e81be80 msgr2=0x55f77e8216b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.255+0000 7fde66bf9640 1 --2- 172.21.15.86:0/2075765671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f77e81be80 0x55f77e8216b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.255+0000 7fde66bf9640 1 -- 172.21.15.86:0/2075765671 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f77e81b4d0 msgr2=0x55f77e824320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.255+0000 7fde66bf9640 1 --2- 172.21.15.86:0/2075765671 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f77e81b4d0 0x55f77e824320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.255+0000 7fde66bf9640 1 -- 172.21.15.86:0/2075765671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f77e4923f0 con 0x55f77e824a00 2024-04-21T08:46:26.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.255+0000 7fde66bf9640 1 --2- 172.21.15.86:0/2075765671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77e824a00 0x55f77e821c50 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fde5002c780 tx=0x7fde50009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3696f62e879db03c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.255+0000 7fde45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:46:26.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.255+0000 7fde45ffb640 1 -- 172.21.15.86:0/2075765671 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde50005260 con 0x55f77e824a00 2024-04-21T08:46:26.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.255+0000 7fde45ffb640 1 -- 172.21.15.86:0/2075765671 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fde50005400 con 0x55f77e824a00 2024-04-21T08:46:26.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.255+0000 7fde6b252740 1 -- 172.21.15.86:0/2075765671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde5801acd0 con 0x55f77e824a00 2024-04-21T08:46:26.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.256+0000 7fde45ffb640 1 -- 172.21.15.86:0/2075765671 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fde50059050 con 0x55f77e824a00 2024-04-21T08:46:26.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.256+0000 7fde6b252740 10 client.18750 fetch_fsmap learned FSMap version 29 2024-04-21T08:46:26.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.256+0000 7fde6b252740 1 -- 172.21.15.86:0/2075765671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f77e55c7d0 con 0x55f77e824a00 2024-04-21T08:46:26.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.256+0000 7fde45ffb640 1 -- 172.21.15.86:0/2075765671 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde5801acd0 con 0x55f77e824a00 2024-04-21T08:46:26.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.256+0000 7fde45ffb640 1 -- 172.21.15.86:0/2075765671 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fde50003820 con 0x55f77e824a00 2024-04-21T08:46:26.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.256+0000 7fde6b252740 10 client.18750 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:46:26.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.256+0000 7fde6b252740 10 client.18750 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:46:26.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.256+0000 7fde6b252740 20 client.18750 populate_metadata read hostname 'smithi086' 2024-04-21T08:46:26.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.256+0000 7fde6b252740 1 --2- 172.21.15.86:0/2075765671 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55f77e834c60 0x55f77e837070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.256+0000 7fde6b252740 4 client.18750 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:26.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.256+0000 7fde6b252740 1 -- 172.21.15.86:0/2075765671 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f77e71f930 con 0x55f77e834c60 2024-04-21T08:46:26.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.256+0000 7fde663f8640 1 --2- 172.21.15.86:0/2075765671 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55f77e834c60 0x55f77e837070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.257+0000 7fde45ffb640 10 client.18750 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:46:26.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.257+0000 7fde663f8640 1 --2- 172.21.15.86:0/2075765671 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55f77e834c60 0x55f77e837070 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9dca7b9ab9ca7934 server_cookie=5a961ef6813e7a45 in_seq=0 out_seq=0 2024-04-21T08:46:26.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.257+0000 7fde45ffb640 1 -- 172.21.15.86:0/2075765671 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f77e71f930 con 0x55f77e834c60 2024-04-21T08:46:26.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.257+0000 7fde45ffb640 10 client.18750 handle_command_reply: tid=0 2024-04-21T08:46:26.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.257+0000 7fde6b252740 1 client.18750 shutdown 2024-04-21T08:46:26.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.257+0000 7fde6b252740 1 -- 172.21.15.86:0/2075765671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77e824a00 msgr2=0x55f77e821c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.257+0000 7fde6b252740 1 --2- 172.21.15.86:0/2075765671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77e824a00 0x55f77e821c50 secure :-1 s=READY pgs=1521 cs=0 l=1 rev1=1 crypto rx=0x7fde5002c780 tx=0x7fde50009260 comp rx=0 tx=0).stop 2024-04-21T08:46:26.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.257+0000 7fde6b252740 1 -- 172.21.15.86:0/2075765671 shutdown_connections 2024-04-21T08:46:26.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.257+0000 7fde6b252740 1 --2- 172.21.15.86:0/2075765671 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55f77e834c60 0x55f77e837070 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.257+0000 7fde6b252740 1 --2- 172.21.15.86:0/2075765671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77e824a00 0x55f77e821c50 unknown :-1 s=CLOSED pgs=1521 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.257+0000 7fde45ffb640 0 client.18750 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:46:26.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.257+0000 7fde6b252740 1 --2- 172.21.15.86:0/2075765671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f77e81be80 0x55f77e8216b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.257+0000 7fde6b252740 1 --2- 172.21.15.86:0/2075765671 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f77e81b4d0 0x55f77e824320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.257+0000 7fde6b252740 1 -- 172.21.15.86:0/2075765671 >> 172.21.15.86:0/2075765671 conn(0x55f77e815ba0 msgr2=0x55f77e8178f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.257+0000 7fde6b252740 1 -- 172.21.15.86:0/2075765671 shutdown_connections 2024-04-21T08:46:26.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.258+0000 7fde6b252740 1 -- 172.21.15.86:0/2075765671 wait complete. 2024-04-21T08:46:26.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.258+0000 7fde6b252740 20 client.18750 trim_cache size 0 max 16384 2024-04-21T08:46:26.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.270+0000 7fde6b252740 1 Processor -- start 2024-04-21T08:46:26.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.271+0000 7fde6b252740 1 -- start start 2024-04-21T08:46:26.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.271+0000 7fde6b252740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77e81b4d0 0x55f77e830830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.271+0000 7fde6b252740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f77e81be80 0x55f77e823de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.271+0000 7fde6b252740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f77e824a00 0x55f77e824320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.271+0000 7fde6b252740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fde601af440 con 0x55f77e81b4d0 2024-04-21T08:46:26.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.271+0000 7fde6b252740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fde601b0f80 con 0x55f77e824a00 2024-04-21T08:46:26.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.271+0000 7fde6b252740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fde601afb10 con 0x55f77e81be80 2024-04-21T08:46:26.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.271+0000 7fde663f8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77e81b4d0 0x55f77e830830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.271+0000 7fde663f8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77e81b4d0 0x55f77e830830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47520/0 (socket says 172.21.15.86:47520) 2024-04-21T08:46:26.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.271+0000 7fde663f8640 1 -- 172.21.15.86:0/2943187280 learned_addr learned my addr 172.21.15.86:0/2943187280 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:26.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.271+0000 7fde663f8640 1 -- 172.21.15.86:0/2943187280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f77e81be80 msgr2=0x55f77e823de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.271+0000 7fde663f8640 1 --2- 172.21.15.86:0/2943187280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f77e81be80 0x55f77e823de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.271+0000 7fde663f8640 1 -- 172.21.15.86:0/2943187280 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f77e824a00 msgr2=0x55f77e824320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.271+0000 7fde66bf9640 1 --2- 172.21.15.86:0/2943187280 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f77e824a00 0x55f77e824320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.271+0000 7fde663f8640 1 --2- 172.21.15.86:0/2943187280 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f77e824a00 0x55f77e824320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.271+0000 7fde663f8640 1 -- 172.21.15.86:0/2943187280 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f77e81a790 con 0x55f77e81b4d0 2024-04-21T08:46:26.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.271+0000 7fde66bf9640 1 --2- 172.21.15.86:0/2943187280 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f77e824a00 0x55f77e824320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:26.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.271+0000 7fde663f8640 1 --2- 172.21.15.86:0/2943187280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77e81b4d0 0x55f77e830830 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fde580118b0 tx=0x7fde58009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8efbfe104805484 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.271+0000 7fde47fff640 1 -- 172.21.15.86:0/2943187280 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde580049a0 con 0x55f77e81b4d0 2024-04-21T08:46:26.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.271+0000 7fde47fff640 1 -- 172.21.15.86:0/2943187280 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde58011920 con 0x55f77e81b4d0 2024-04-21T08:46:26.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.271+0000 7fde47fff640 1 -- 172.21.15.86:0/2943187280 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde580273f0 con 0x55f77e81b4d0 2024-04-21T08:46:26.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.272+0000 7fde6b252740 1 -- 172.21.15.86:0/2943187280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77e81b4d0 msgr2=0x55f77e830830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.272+0000 7fde6b252740 1 --2- 172.21.15.86:0/2943187280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77e81b4d0 0x55f77e830830 secure :-1 s=READY pgs=1522 cs=0 l=1 rev1=1 crypto rx=0x7fde580118b0 tx=0x7fde58009740 comp rx=0 tx=0).stop 2024-04-21T08:46:26.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.272+0000 7fde6b252740 1 -- 172.21.15.86:0/2943187280 shutdown_connections 2024-04-21T08:46:26.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.272+0000 7fde6b252740 1 --2- 172.21.15.86:0/2943187280 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f77e824a00 0x55f77e824320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.272+0000 7fde6b252740 1 --2- 172.21.15.86:0/2943187280 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f77e81be80 0x55f77e823de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.272+0000 7fde6b252740 1 --2- 172.21.15.86:0/2943187280 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77e81b4d0 0x55f77e830830 unknown :-1 s=CLOSED pgs=1522 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.272+0000 7fde6b252740 1 -- 172.21.15.86:0/2943187280 >> 172.21.15.86:0/2943187280 conn(0x55f77e815ba0 msgr2=0x55f77e814c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.272+0000 7fde6b252740 1 -- 172.21.15.86:0/2943187280 shutdown_connections 2024-04-21T08:46:26.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.272+0000 7fde6b252740 1 -- 172.21.15.86:0/2943187280 wait complete. 2024-04-21T08:46:26.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.272+0000 7fde6b252740 1 -- start start 2024-04-21T08:46:26.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.272+0000 7fde6b252740 1 Processor -- start 2024-04-21T08:46:26.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.272+0000 7fde6b252740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f77e81b4d0 0x55f77e82d030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.272+0000 7fde6b252740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77e81be80 0x55f77e81e000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.272+0000 7fde6b252740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f77e824a00 0x55f77e81e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.272+0000 7fde6b252740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fde601b08b0 con 0x55f77e81be80 2024-04-21T08:46:26.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.272+0000 7fde6b252740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fde601b01e0 con 0x55f77e81b4d0 2024-04-21T08:46:26.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.272+0000 7fde6b252740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fde601ad440 con 0x55f77e824a00 2024-04-21T08:46:26.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.272+0000 7fde6b252740 10 client.0 _ll_register_callbacks cb 0x7fde50009580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:26.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.273+0000 7fde65bf7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77e81be80 0x55f77e81e000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.273+0000 7fde65bf7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77e81be80 0x55f77e81e000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47536/0 (socket says 172.21.15.86:47536) 2024-04-21T08:46:26.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.273+0000 7fde65bf7640 1 -- 172.21.15.86:0/3203609570 learned_addr learned my addr 172.21.15.86:0/3203609570 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:26.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.273+0000 7fde66bf9640 1 --2- 172.21.15.86:0/3203609570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f77e824a00 0x55f77e81e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.273+0000 7fde663f8640 1 --2- 172.21.15.86:0/3203609570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f77e81b4d0 0x55f77e82d030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.273+0000 7fde65bf7640 1 -- 172.21.15.86:0/3203609570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f77e824a00 msgr2=0x55f77e81e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.273+0000 7fde65bf7640 1 --2- 172.21.15.86:0/3203609570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f77e824a00 0x55f77e81e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.273+0000 7fde65bf7640 1 -- 172.21.15.86:0/3203609570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f77e81b4d0 msgr2=0x55f77e82d030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.273+0000 7fde65bf7640 1 --2- 172.21.15.86:0/3203609570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f77e81b4d0 0x55f77e82d030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.273+0000 7fde65bf7640 1 -- 172.21.15.86:0/3203609570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f77e81ec80 con 0x55f77e81be80 2024-04-21T08:46:26.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.273+0000 7fde663f8640 1 --2- 172.21.15.86:0/3203609570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f77e81b4d0 0x55f77e82d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:26.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.273+0000 7fde65bf7640 1 --2- 172.21.15.86:0/3203609570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77e81be80 0x55f77e81e000 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fde54099530 tx=0x7fde5401d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e30e9eac1d73f8d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.273+0000 7fde45ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:46:26.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.273+0000 7fde45ffb640 1 -- 172.21.15.86:0/3203609570 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fde540b5020 con 0x55f77e81be80 2024-04-21T08:46:26.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.273+0000 7fde45ffb640 1 -- 172.21.15.86:0/3203609570 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fde540bb990 con 0x55f77e81be80 2024-04-21T08:46:26.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.273+0000 7fde6b252740 1 -- 172.21.15.86:0/3203609570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fde58011920 con 0x55f77e81be80 2024-04-21T08:46:26.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.273+0000 7fde45ffb640 1 -- 172.21.15.86:0/3203609570 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fde54080050 con 0x55f77e81be80 2024-04-21T08:46:26.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.273+0000 7fde6b252740 10 client.18756 fetch_fsmap learned FSMap version 29 2024-04-21T08:46:26.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.273+0000 7fde6b252740 1 -- 172.21.15.86:0/3203609570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f77e89f500 con 0x55f77e81be80 2024-04-21T08:46:26.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.273+0000 7fde45ffb640 1 -- 172.21.15.86:0/3203609570 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fde58011920 con 0x55f77e81be80 2024-04-21T08:46:26.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.274+0000 7fde45ffb640 1 -- 172.21.15.86:0/3203609570 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fde540991d0 con 0x55f77e81be80 2024-04-21T08:46:26.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.274+0000 7fde6b252740 10 client.18756 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:46:26.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.274+0000 7fde6b252740 10 client.18756 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:46:26.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.274+0000 7fde6b252740 20 client.18756 populate_metadata read hostname 'smithi086' 2024-04-21T08:46:26.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.274+0000 7fde6b252740 1 --2- 172.21.15.86:0/3203609570 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55f77e89fc80 0x55f77e8a20a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.274+0000 7fde6b252740 4 client.18756 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:46:26.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.274+0000 7fde6b252740 1 -- 172.21.15.86:0/3203609570 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f77e78dc10 con 0x55f77e89fc80 2024-04-21T08:46:26.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.274+0000 7fde663f8640 1 --2- 172.21.15.86:0/3203609570 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55f77e89fc80 0x55f77e8a20a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.274+0000 7fde45ffb640 10 client.18756 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:46:26.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.274+0000 7fde663f8640 1 --2- 172.21.15.86:0/3203609570 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55f77e89fc80 0x55f77e8a20a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d480bfe7cf6e9b87 server_cookie=3e254ad10c11981b in_seq=0 out_seq=0 2024-04-21T08:46:26.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.274+0000 7fde45ffb640 1 -- 172.21.15.86:0/3203609570 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f77e78dc10 con 0x55f77e89fc80 2024-04-21T08:46:26.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.274+0000 7fde45ffb640 10 client.18756 handle_command_reply: tid=0 2024-04-21T08:46:26.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.274+0000 7fde6b252740 1 client.18756 shutdown 2024-04-21T08:46:26.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.274+0000 7fde6b252740 1 -- 172.21.15.86:0/3203609570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77e81be80 msgr2=0x55f77e81e000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.274+0000 7fde6b252740 1 --2- 172.21.15.86:0/3203609570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77e81be80 0x55f77e81e000 secure :-1 s=READY pgs=1523 cs=0 l=1 rev1=1 crypto rx=0x7fde54099530 tx=0x7fde5401d000 comp rx=0 tx=0).stop 2024-04-21T08:46:26.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.274+0000 7fde6b252740 1 -- 172.21.15.86:0/3203609570 shutdown_connections 2024-04-21T08:46:26.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.274+0000 7fde6b252740 1 --2- 172.21.15.86:0/3203609570 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55f77e89fc80 0x55f77e8a20a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.274+0000 7fde6b252740 1 --2- 172.21.15.86:0/3203609570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f77e824a00 0x55f77e81e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.274+0000 7fde6b252740 1 --2- 172.21.15.86:0/3203609570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f77e81be80 0x55f77e81e000 unknown :-1 s=CLOSED pgs=1523 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.274+0000 7fde45ffb640 0 client.18756 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:46:26.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.274+0000 7fde6b252740 1 --2- 172.21.15.86:0/3203609570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f77e81b4d0 0x55f77e82d030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.274+0000 7fde6b252740 1 -- 172.21.15.86:0/3203609570 >> 172.21.15.86:0/3203609570 conn(0x55f77e815ba0 msgr2=0x55f77e814c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.275+0000 7fde6b252740 1 -- 172.21.15.86:0/3203609570 shutdown_connections 2024-04-21T08:46:26.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.275+0000 7fde6b252740 1 -- 172.21.15.86:0/3203609570 wait complete. 2024-04-21T08:46:26.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.275+0000 7fde6b252740 20 client.18756 trim_cache size 0 max 16384 2024-04-21T08:46:26.282 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:46:26.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.275+0000 7fde6865c640 1 -- 172.21.15.86:0/3681717657 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fde480a3da0 msgr2=0x7fde480a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.275+0000 7fde6865c640 1 --2- 172.21.15.86:0/3681717657 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fde480a3da0 0x7fde480a6260 secure :-1 s=READY pgs=245 cs=0 l=1 rev1=1 crypto rx=0x7fde500096c0 tx=0x7fde5002e040 comp rx=0 tx=0).stop 2024-04-21T08:46:26.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.275+0000 7fde6865c640 1 -- 172.21.15.86:0/3681717657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde601072e0 msgr2=0x7fde6019e990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.275+0000 7fde6865c640 1 --2- 172.21.15.86:0/3681717657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde601072e0 0x7fde6019e990 secure :-1 s=READY pgs=1519 cs=0 l=1 rev1=1 crypto rx=0x7fde5400e9f0 tx=0x7fde5400eeb0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.275+0000 7fde6865c640 1 -- 172.21.15.86:0/3681717657 shutdown_connections 2024-04-21T08:46:26.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.275+0000 7fde6865c640 1 --2- 172.21.15.86:0/3681717657 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fde480a3da0 0x7fde480a6260 unknown :-1 s=CLOSED pgs=245 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.275+0000 7fde6865c640 1 --2- 172.21.15.86:0/3681717657 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fde60198650 0x7fde60198b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.275+0000 7fde6865c640 1 --2- 172.21.15.86:0/3681717657 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fde601072e0 0x7fde6019e990 unknown :-1 s=CLOSED pgs=1519 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.275+0000 7fde6865c640 1 --2- 172.21.15.86:0/3681717657 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fde60102210 0x7fde6019e450 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.275+0000 7fde6865c640 1 -- 172.21.15.86:0/3681717657 >> 172.21.15.86:0/3681717657 conn(0x7fde600fdab0 msgr2=0x7fde600feea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.275+0000 7fde6865c640 1 -- 172.21.15.86:0/3681717657 shutdown_connections 2024-04-21T08:46:26.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.275+0000 7fde6865c640 1 -- 172.21.15.86:0/3681717657 wait complete. 2024-04-21T08:46:26.285 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:46:26.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.385+0000 7fa6afd10640 1 -- 172.21.15.86:0/166641361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6a8106160 msgr2=0x7fa6a810dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.385+0000 7fa6afd10640 1 --2- 172.21.15.86:0/166641361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6a8106160 0x7fa6a810dd30 secure :-1 s=READY pgs=1524 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0009ca0 tx=0x7fa6a002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.385+0000 7fa6afd10640 1 -- 172.21.15.86:0/166641361 shutdown_connections 2024-04-21T08:46:26.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.385+0000 7fa6afd10640 1 --2- 172.21.15.86:0/166641361 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6a8106160 0x7fa6a810dd30 unknown :-1 s=CLOSED pgs=1524 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.385+0000 7fa6afd10640 1 --2- 172.21.15.86:0/166641361 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6a8101a40 0x7fa6a8105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.385+0000 7fa6afd10640 1 --2- 172.21.15.86:0/166641361 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6a8101090 0x7fa6a8101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.385+0000 7fa6afd10640 1 -- 172.21.15.86:0/166641361 >> 172.21.15.86:0/166641361 conn(0x7fa6a8077f50 msgr2=0x7fa6a8078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.385+0000 7fa6afd10640 1 -- 172.21.15.86:0/166641361 shutdown_connections 2024-04-21T08:46:26.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.385+0000 7fa6afd10640 1 -- 172.21.15.86:0/166641361 wait complete. 2024-04-21T08:46:26.387 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.386+0000 7fa6afd10640 1 Processor -- start 2024-04-21T08:46:26.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.386+0000 7fa6afd10640 1 -- start start 2024-04-21T08:46:26.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.386+0000 7fa6afd10640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6a8101090 0x7fa6a8197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.386+0000 7fa6afd10640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6a8106160 0x7fa6a8197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.386+0000 7fa6afd10640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6a819bec0 0x7fa6a819c370 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.386+0000 7fa6afd10640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a81110d0 con 0x7fa6a819bec0 2024-04-21T08:46:26.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.386+0000 7fa6afd10640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a8110f50 con 0x7fa6a8106160 2024-04-21T08:46:26.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.386+0000 7fa6afd10640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a8111250 con 0x7fa6a8101090 2024-04-21T08:46:26.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.386+0000 7fa6ae2ad640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6a819bec0 0x7fa6a819c370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.386+0000 7fa6ae2ad640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6a819bec0 0x7fa6a819c370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47552/0 (socket says 172.21.15.86:47552) 2024-04-21T08:46:26.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.386+0000 7fa6ae2ad640 1 -- 172.21.15.86:0/1007118173 learned_addr learned my addr 172.21.15.86:0/1007118173 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:26.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.386+0000 7fa6adaac640 1 --2- 172.21.15.86:0/1007118173 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6a8101090 0x7fa6a8197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.386+0000 7fa6ad2ab640 1 --2- 172.21.15.86:0/1007118173 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6a8106160 0x7fa6a8197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.386+0000 7fa6ae2ad640 1 -- 172.21.15.86:0/1007118173 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6a8101090 msgr2=0x7fa6a8197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.386+0000 7fa6ae2ad640 1 --2- 172.21.15.86:0/1007118173 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6a8101090 0x7fa6a8197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.388 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.386+0000 7fa6ae2ad640 1 -- 172.21.15.86:0/1007118173 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6a8106160 msgr2=0x7fa6a8197b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.386+0000 7fa6ae2ad640 1 --2- 172.21.15.86:0/1007118173 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6a8106160 0x7fa6a8197b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.386+0000 7fa6ae2ad640 1 -- 172.21.15.86:0/1007118173 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa6a819c9b0 con 0x7fa6a819bec0 2024-04-21T08:46:26.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.386+0000 7fa6ad2ab640 1 --2- 172.21.15.86:0/1007118173 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6a8106160 0x7fa6a8197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:26.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.386+0000 7fa6ae2ad640 1 --2- 172.21.15.86:0/1007118173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6a819bec0 0x7fa6a819c370 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0005e30 tx=0x7fa6a0034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.386+0000 7fa69effd640 1 -- 172.21.15.86:0/1007118173 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0039040 con 0x7fa6a819bec0 2024-04-21T08:46:26.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.387+0000 7fa69effd640 1 -- 172.21.15.86:0/1007118173 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6a0044070 con 0x7fa6a819bec0 2024-04-21T08:46:26.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.387+0000 7fa69effd640 1 -- 172.21.15.86:0/1007118173 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a0003e60 con 0x7fa6a819bec0 2024-04-21T08:46:26.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.387+0000 7fa6afd10640 1 -- 172.21.15.86:0/1007118173 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa6a819cc40 con 0x7fa6a819bec0 2024-04-21T08:46:26.389 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.387+0000 7fa6afd10640 1 -- 172.21.15.86:0/1007118173 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa6a81a4400 con 0x7fa6a819bec0 2024-04-21T08:46:26.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.389+0000 7fa69effd640 1 -- 172.21.15.86:0/1007118173 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa6a00324c0 con 0x7fa6a819bec0 2024-04-21T08:46:26.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.389+0000 7fa6b29d8740 1 Processor -- start 2024-04-21T08:46:26.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.389+0000 7fa6b29d8740 1 -- start start 2024-04-21T08:46:26.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.389+0000 7fa6b29d8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576b05ca4d0 0x5576b05ca8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.389+0000 7fa6b29d8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576b05cae80 0x5576b05d3430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.389+0000 7fa6b29d8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576b05d3a00 0x5576b05d5e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.390+0000 7fa6b29d8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5576b052f1f0 con 0x5576b05ca4d0 2024-04-21T08:46:26.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.390+0000 7fa6b29d8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5576b04cc4b0 con 0x5576b05d3a00 2024-04-21T08:46:26.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.390+0000 7fa69effd640 1 --2- 172.21.15.86:0/1007118173 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa6940a3d20 0x7fa6940a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.390+0000 7fa6b29d8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5576b052b1b0 con 0x5576b05cae80 2024-04-21T08:46:26.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.390+0000 7fa6ad2ab640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576b05ca4d0 0x5576b05ca8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.390+0000 7fa6ae2ad640 1 --2- 172.21.15.86:0/1007118173 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa6940a3d20 0x7fa6940a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.390+0000 7fa6ad2ab640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576b05ca4d0 0x5576b05ca8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47564/0 (socket says 172.21.15.86:47564) 2024-04-21T08:46:26.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.390+0000 7fa6ad2ab640 1 -- 172.21.15.86:0/4085070429 learned_addr learned my addr 172.21.15.86:0/4085070429 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:26.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.390+0000 7fa69effd640 1 -- 172.21.15.86:0/1007118173 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fa6a00f2e50 con 0x7fa6a819bec0 2024-04-21T08:46:26.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.390+0000 7fa6adaac640 1 --2- 172.21.15.86:0/4085070429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576b05cae80 0x5576b05d3430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.390+0000 7fa6ad2ab640 1 -- 172.21.15.86:0/4085070429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576b05cae80 msgr2=0x5576b05d3430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.390+0000 7fa6ad2ab640 1 --2- 172.21.15.86:0/4085070429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576b05cae80 0x5576b05d3430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.396 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.390+0000 7fa6ad2ab640 1 -- 172.21.15.86:0/4085070429 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576b05d3a00 msgr2=0x5576b05d5e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.390+0000 7fa6ad2ab640 1 --2- 172.21.15.86:0/4085070429 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576b05d3a00 0x5576b05d5e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.390+0000 7fa6ad2ab640 1 -- 172.21.15.86:0/4085070429 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576b04be1b0 con 0x5576b05ca4d0 2024-04-21T08:46:26.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.390+0000 7fa6adaac640 1 --2- 172.21.15.86:0/4085070429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576b05cae80 0x5576b05d3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:26.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.390+0000 7fa6ad2ab640 1 --2- 172.21.15.86:0/4085070429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576b05ca4d0 0x5576b05ca8b0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fa698009c10 tx=0x7fa6980106e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a0b82cdeb4395ea server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.390+0000 7fa6ae2ad640 1 --2- 172.21.15.86:0/1007118173 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa6940a3d20 0x7fa6940a61e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa6a00f0390 tx=0x7fa6a00b3fc0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.390+0000 7fa693fff640 1 -- 172.21.15.86:0/4085070429 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa698016070 con 0x5576b05ca4d0 2024-04-21T08:46:26.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.390+0000 7fa693fff640 1 -- 172.21.15.86:0/4085070429 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6980128d0 con 0x5576b05ca4d0 2024-04-21T08:46:26.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.390+0000 7fa693fff640 1 -- 172.21.15.86:0/4085070429 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa698011760 con 0x5576b05ca4d0 2024-04-21T08:46:26.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.391+0000 7fa6b29d8740 1 -- 172.21.15.86:0/4085070429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576b05ca4d0 msgr2=0x5576b05ca8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.391+0000 7fa6b29d8740 1 --2- 172.21.15.86:0/4085070429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576b05ca4d0 0x5576b05ca8b0 secure :-1 s=READY pgs=1526 cs=0 l=1 rev1=1 crypto rx=0x7fa698009c10 tx=0x7fa6980106e0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.391+0000 7fa6b29d8740 1 -- 172.21.15.86:0/4085070429 shutdown_connections 2024-04-21T08:46:26.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.391+0000 7fa6b29d8740 1 --2- 172.21.15.86:0/4085070429 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576b05d3a00 0x5576b05d5e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.391+0000 7fa6b29d8740 1 --2- 172.21.15.86:0/4085070429 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576b05cae80 0x5576b05d3430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.391+0000 7fa6b29d8740 1 --2- 172.21.15.86:0/4085070429 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576b05ca4d0 0x5576b05ca8b0 unknown :-1 s=CLOSED pgs=1526 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.391+0000 7fa6b29d8740 1 -- 172.21.15.86:0/4085070429 >> 172.21.15.86:0/4085070429 conn(0x5576b05c4ba0 msgr2=0x5576b05c4f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.391+0000 7fa6b29d8740 1 -- 172.21.15.86:0/4085070429 shutdown_connections 2024-04-21T08:46:26.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.391+0000 7fa6b29d8740 1 -- 172.21.15.86:0/4085070429 wait complete. 2024-04-21T08:46:26.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.391+0000 7fa6b29d8740 1 -- start start 2024-04-21T08:46:26.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.391+0000 7fa6b29d8740 1 Processor -- start 2024-04-21T08:46:26.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.392+0000 7fa6b29d8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576b05ca4d0 0x5576b05d3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.392+0000 7fa6b29d8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576b05cae80 0x5576b05d0600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.392+0000 7fa6b29d8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576b05d3a00 0x5576b05d0b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.392+0000 7fa6b29d8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5576b05d6970 con 0x5576b05ca4d0 2024-04-21T08:46:26.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.392+0000 7fa6b29d8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5576b04e8500 con 0x5576b05d3a00 2024-04-21T08:46:26.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.392+0000 7fa6b29d8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5576b05d6af0 con 0x5576b05cae80 2024-04-21T08:46:26.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.392+0000 7fa6b29d8740 10 client.0 _ll_register_callbacks cb 0x5576b04b6b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:26.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.392+0000 7fa6ad2ab640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576b05ca4d0 0x5576b05d3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.392+0000 7fa6ad2ab640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576b05ca4d0 0x5576b05d3270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47572/0 (socket says 172.21.15.86:47572) 2024-04-21T08:46:26.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.392+0000 7fa6ad2ab640 1 -- 172.21.15.86:0/869108872 learned_addr learned my addr 172.21.15.86:0/869108872 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:26.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.392+0000 7fa6ad2ab640 1 --2- 172.21.15.86:0/869108872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576b05d3a00 0x5576b05d0b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.392+0000 7fa6ad2ab640 1 -- 172.21.15.86:0/869108872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576b05cae80 msgr2=0x5576b05d0600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.392+0000 7fa6ad2ab640 1 --2- 172.21.15.86:0/869108872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576b05cae80 0x5576b05d0600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.392+0000 7fa6ad2ab640 1 -- 172.21.15.86:0/869108872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576b05d3a00 msgr2=0x5576b05d0b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.392+0000 7fa6ad2ab640 1 --2- 172.21.15.86:0/869108872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576b05d3a00 0x5576b05d0b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.392+0000 7fa6ad2ab640 1 -- 172.21.15.86:0/869108872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576b02413f0 con 0x5576b05ca4d0 2024-04-21T08:46:26.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.392+0000 7fa6ad2ab640 1 --2- 172.21.15.86:0/869108872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576b05ca4d0 0x5576b05d3270 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fa6980113c0 tx=0x7fa6980094f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92ecf3fab95a59cf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.392+0000 7fa6917fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:46:26.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.393+0000 7fa6917fa640 1 -- 172.21.15.86:0/869108872 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa698016050 con 0x5576b05ca4d0 2024-04-21T08:46:26.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.393+0000 7fa6b29d8740 1 -- 172.21.15.86:0/869108872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6980128d0 con 0x5576b05ca4d0 2024-04-21T08:46:26.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.393+0000 7fa6917fa640 1 -- 172.21.15.86:0/869108872 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fa69801d660 con 0x5576b05ca4d0 2024-04-21T08:46:26.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.393+0000 7fa6917fa640 1 -- 172.21.15.86:0/869108872 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa69801d860 con 0x5576b05ca4d0 2024-04-21T08:46:26.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.393+0000 7fa6b29d8740 10 client.18768 fetch_fsmap learned FSMap version 29 2024-04-21T08:46:26.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.393+0000 7fa6b29d8740 1 -- 172.21.15.86:0/869108872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5576b030b7d0 con 0x5576b05ca4d0 2024-04-21T08:46:26.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.393+0000 7fa6917fa640 1 -- 172.21.15.86:0/869108872 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6980128d0 con 0x5576b05ca4d0 2024-04-21T08:46:26.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.393+0000 7fa6917fa640 1 -- 172.21.15.86:0/869108872 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa69802a020 con 0x5576b05ca4d0 2024-04-21T08:46:26.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.393+0000 7fa6b29d8740 10 client.18768 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:46:26.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.393+0000 7fa6b29d8740 10 client.18768 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:46:26.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.393+0000 7fa6b29d8740 20 client.18768 populate_metadata read hostname 'smithi086' 2024-04-21T08:46:26.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.393+0000 7fa6b29d8740 1 --2- 172.21.15.86:0/869108872 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5576b05e3b10 0x5576b05e5f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.393+0000 7fa6b29d8740 4 client.18768 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:26.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.393+0000 7fa6b29d8740 1 -- 172.21.15.86:0/869108872 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576b04ce930 con 0x5576b05e3b10 2024-04-21T08:46:26.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.393+0000 7fa6adaac640 1 --2- 172.21.15.86:0/869108872 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5576b05e3b10 0x5576b05e5f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.394+0000 7fa6917fa640 10 client.18768 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:46:26.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.394+0000 7fa6adaac640 1 --2- 172.21.15.86:0/869108872 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5576b05e3b10 0x5576b05e5f00 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=d0d8bdab47b8fcca server_cookie=137aa0cee1bcbe78 in_seq=0 out_seq=0 2024-04-21T08:46:26.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.394+0000 7fa6917fa640 1 -- 172.21.15.86:0/869108872 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5576b04ce930 con 0x5576b05e3b10 2024-04-21T08:46:26.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.394+0000 7fa6917fa640 10 client.18768 handle_command_reply: tid=0 2024-04-21T08:46:26.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.394+0000 7fa6b29d8740 1 client.18768 shutdown 2024-04-21T08:46:26.400 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.394+0000 7fa6b29d8740 1 -- 172.21.15.86:0/869108872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576b05ca4d0 msgr2=0x5576b05d3270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.394+0000 7fa6b29d8740 1 --2- 172.21.15.86:0/869108872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576b05ca4d0 0x5576b05d3270 secure :-1 s=READY pgs=1527 cs=0 l=1 rev1=1 crypto rx=0x7fa6980113c0 tx=0x7fa6980094f0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.395+0000 7fa6b29d8740 1 -- 172.21.15.86:0/869108872 shutdown_connections 2024-04-21T08:46:26.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.395+0000 7fa6b29d8740 1 --2- 172.21.15.86:0/869108872 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5576b05e3b10 0x5576b05e5f00 crc :-1 s=READY pgs=61 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.395+0000 7fa6b29d8740 1 --2- 172.21.15.86:0/869108872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576b05d3a00 0x5576b05d0b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.395+0000 7fa6b29d8740 1 --2- 172.21.15.86:0/869108872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576b05cae80 0x5576b05d0600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.395+0000 7fa6b29d8740 1 --2- 172.21.15.86:0/869108872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576b05ca4d0 0x5576b05d3270 unknown :-1 s=CLOSED pgs=1527 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.395+0000 7fa6917fa640 0 client.18768 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:46:26.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.395+0000 7fa6b29d8740 1 -- 172.21.15.86:0/869108872 >> 172.21.15.86:0/869108872 conn(0x5576b05c4ba0 msgr2=0x5576b05c68f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.395+0000 7fa6b29d8740 1 -- 172.21.15.86:0/869108872 shutdown_connections 2024-04-21T08:46:26.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.395+0000 7fa6b29d8740 1 -- 172.21.15.86:0/869108872 wait complete. 2024-04-21T08:46:26.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.395+0000 7fa6b29d8740 20 client.18768 trim_cache size 0 max 16384 2024-04-21T08:46:26.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.408+0000 7fa6b29d8740 1 Processor -- start 2024-04-21T08:46:26.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.408+0000 7fa6b29d8740 1 -- start start 2024-04-21T08:46:26.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.408+0000 7fa6b29d8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576b05ca4d0 0x5576b05df860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.408+0000 7fa6b29d8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576b05cae80 0x5576b05d2d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.408+0000 7fa6b29d8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576b05d3a00 0x5576b05d3270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.408+0000 7fa6b29d8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a81ad0b0 con 0x5576b05cae80 2024-04-21T08:46:26.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.408+0000 7fa6b29d8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a81aebf0 con 0x5576b05ca4d0 2024-04-21T08:46:26.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.408+0000 7fa6b29d8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a81ad780 con 0x5576b05d3a00 2024-04-21T08:46:26.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.408+0000 7fa6adaac640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576b05cae80 0x5576b05d2d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.408+0000 7fa6adaac640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576b05cae80 0x5576b05d2d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47580/0 (socket says 172.21.15.86:47580) 2024-04-21T08:46:26.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.408+0000 7fa6adaac640 1 -- 172.21.15.86:0/1490501126 learned_addr learned my addr 172.21.15.86:0/1490501126 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:26.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.408+0000 7fa6adaac640 1 -- 172.21.15.86:0/1490501126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576b05d3a00 msgr2=0x5576b05d3270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.408+0000 7fa6adaac640 1 --2- 172.21.15.86:0/1490501126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576b05d3a00 0x5576b05d3270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.408+0000 7fa6adaac640 1 -- 172.21.15.86:0/1490501126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576b05ca4d0 msgr2=0x5576b05df860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.408+0000 7fa6ad2ab640 1 --2- 172.21.15.86:0/1490501126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576b05ca4d0 0x5576b05df860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.408+0000 7fa6adaac640 1 --2- 172.21.15.86:0/1490501126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576b05ca4d0 0x5576b05df860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.408+0000 7fa6adaac640 1 -- 172.21.15.86:0/1490501126 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576b05d0ea0 con 0x5576b05cae80 2024-04-21T08:46:26.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.408+0000 7fa6ad2ab640 1 --2- 172.21.15.86:0/1490501126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576b05ca4d0 0x5576b05df860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:26.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.408+0000 7fa6adaac640 1 --2- 172.21.15.86:0/1490501126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576b05cae80 0x5576b05d2d30 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fa6a400baf0 tx=0x7fa6a400b2c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a8ba177dffd2feaf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.409+0000 7fa6937fe640 1 -- 172.21.15.86:0/1490501126 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a401d070 con 0x5576b05cae80 2024-04-21T08:46:26.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.409+0000 7fa6937fe640 1 -- 172.21.15.86:0/1490501126 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6a4019d50 con 0x5576b05cae80 2024-04-21T08:46:26.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.409+0000 7fa6937fe640 1 -- 172.21.15.86:0/1490501126 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa6a4004e40 con 0x5576b05cae80 2024-04-21T08:46:26.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.409+0000 7fa6b29d8740 1 -- 172.21.15.86:0/1490501126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576b05cae80 msgr2=0x5576b05d2d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.409+0000 7fa6b29d8740 1 --2- 172.21.15.86:0/1490501126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576b05cae80 0x5576b05d2d30 secure :-1 s=READY pgs=1528 cs=0 l=1 rev1=1 crypto rx=0x7fa6a400baf0 tx=0x7fa6a400b2c0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.409+0000 7fa6b29d8740 1 -- 172.21.15.86:0/1490501126 shutdown_connections 2024-04-21T08:46:26.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.409+0000 7fa6b29d8740 1 --2- 172.21.15.86:0/1490501126 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576b05d3a00 0x5576b05d3270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.409+0000 7fa6b29d8740 1 --2- 172.21.15.86:0/1490501126 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576b05cae80 0x5576b05d2d30 unknown :-1 s=CLOSED pgs=1528 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.409+0000 7fa6b29d8740 1 --2- 172.21.15.86:0/1490501126 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576b05ca4d0 0x5576b05df860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.409+0000 7fa6b29d8740 1 -- 172.21.15.86:0/1490501126 >> 172.21.15.86:0/1490501126 conn(0x5576b05c4ba0 msgr2=0x5576b05c6730 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.409+0000 7fa6b29d8740 1 -- 172.21.15.86:0/1490501126 shutdown_connections 2024-04-21T08:46:26.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.409+0000 7fa6b29d8740 1 -- 172.21.15.86:0/1490501126 wait complete. 2024-04-21T08:46:26.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.410+0000 7fa6b29d8740 1 -- start start 2024-04-21T08:46:26.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.410+0000 7fa6b29d8740 1 Processor -- start 2024-04-21T08:46:26.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.410+0000 7fa6b29d8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576b05ca4d0 0x5576b05dc080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.410+0000 7fa6b29d8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576b05cae80 0x5576b05d73f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.410+0000 7fa6b29d8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576b05d3a00 0x5576b05d7930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.410+0000 7fa6b29d8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a81ae520 con 0x5576b05d3a00 2024-04-21T08:46:26.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.410+0000 7fa6b29d8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a81ade50 con 0x5576b05ca4d0 2024-04-21T08:46:26.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.410+0000 7fa6b29d8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa6a81ab0b0 con 0x5576b05cae80 2024-04-21T08:46:26.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.410+0000 7fa6b29d8740 10 client.0 _ll_register_callbacks cb 0x7fa698012420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:26.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.410+0000 7fa6ad2ab640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576b05d3a00 0x5576b05d7930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.410+0000 7fa6ad2ab640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576b05d3a00 0x5576b05d7930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47586/0 (socket says 172.21.15.86:47586) 2024-04-21T08:46:26.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.410+0000 7fa6ad2ab640 1 -- 172.21.15.86:0/2693057374 learned_addr learned my addr 172.21.15.86:0/2693057374 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:26.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.410+0000 7fa6ad2ab640 1 --2- 172.21.15.86:0/2693057374 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576b05ca4d0 0x5576b05dc080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.410+0000 7fa6adaac640 1 --2- 172.21.15.86:0/2693057374 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576b05cae80 0x5576b05d73f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.410+0000 7fa6ad2ab640 1 -- 172.21.15.86:0/2693057374 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576b05cae80 msgr2=0x5576b05d73f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.410+0000 7fa6ad2ab640 1 --2- 172.21.15.86:0/2693057374 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576b05cae80 0x5576b05d73f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.410+0000 7fa6ad2ab640 1 -- 172.21.15.86:0/2693057374 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576b05ca4d0 msgr2=0x5576b05dc080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.410+0000 7fa6ad2ab640 1 --2- 172.21.15.86:0/2693057374 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576b05ca4d0 0x5576b05dc080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.410+0000 7fa6ad2ab640 1 -- 172.21.15.86:0/2693057374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576b05d8130 con 0x5576b05d3a00 2024-04-21T08:46:26.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.411+0000 7fa6ad2ab640 1 --2- 172.21.15.86:0/2693057374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576b05d3a00 0x5576b05d7930 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fa698009610 tx=0x7fa6980106e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=32316d20cd222553 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.411+0000 7fa6917fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:46:26.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.411+0000 7fa6917fa640 1 -- 172.21.15.86:0/2693057374 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa698016040 con 0x5576b05d3a00 2024-04-21T08:46:26.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.411+0000 7fa6917fa640 1 -- 172.21.15.86:0/2693057374 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fa69801dc00 con 0x5576b05d3a00 2024-04-21T08:46:26.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.411+0000 7fa6b29d8740 1 -- 172.21.15.86:0/2693057374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa6a4019d50 con 0x5576b05d3a00 2024-04-21T08:46:26.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.411+0000 7fa6917fa640 1 -- 172.21.15.86:0/2693057374 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa698011eb0 con 0x5576b05d3a00 2024-04-21T08:46:26.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.411+0000 7fa6b29d8740 10 client.18774 fetch_fsmap learned FSMap version 29 2024-04-21T08:46:26.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.411+0000 7fa6b29d8740 1 -- 172.21.15.86:0/2693057374 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5576b0649c60 con 0x5576b05d3a00 2024-04-21T08:46:26.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.411+0000 7fa6917fa640 1 -- 172.21.15.86:0/2693057374 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa6a4019d50 con 0x5576b05d3a00 2024-04-21T08:46:26.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.411+0000 7fa6917fa640 1 -- 172.21.15.86:0/2693057374 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa69802a070 con 0x5576b05d3a00 2024-04-21T08:46:26.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.411+0000 7fa6b29d8740 10 client.18774 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:46:26.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.411+0000 7fa6b29d8740 10 client.18774 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:46:26.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.411+0000 7fa6b29d8740 20 client.18774 populate_metadata read hostname 'smithi086' 2024-04-21T08:46:26.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.411+0000 7fa6b29d8740 1 --2- 172.21.15.86:0/2693057374 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5576b064a380 0x5576b064c7a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.411+0000 7fa6b29d8740 4 client.18774 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:46:26.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.411+0000 7fa6b29d8740 1 -- 172.21.15.86:0/2693057374 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5576b053cc10 con 0x5576b064a380 2024-04-21T08:46:26.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.411+0000 7fa6adaac640 1 --2- 172.21.15.86:0/2693057374 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5576b064a380 0x5576b064c7a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.412+0000 7fa6917fa640 10 client.18774 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:46:26.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.412+0000 7fa6adaac640 1 --2- 172.21.15.86:0/2693057374 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5576b064a380 0x5576b064c7a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=7364cfcef17cbd72 server_cookie=3a0a261d7870dbdc in_seq=0 out_seq=0 2024-04-21T08:46:26.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.412+0000 7fa6917fa640 1 -- 172.21.15.86:0/2693057374 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5576b053cc10 con 0x5576b064a380 2024-04-21T08:46:26.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.412+0000 7fa6917fa640 10 client.18774 handle_command_reply: tid=0 2024-04-21T08:46:26.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.412+0000 7fa6b29d8740 1 client.18774 shutdown 2024-04-21T08:46:26.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.412+0000 7fa6b29d8740 1 -- 172.21.15.86:0/2693057374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576b05d3a00 msgr2=0x5576b05d7930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.412+0000 7fa6b29d8740 1 --2- 172.21.15.86:0/2693057374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576b05d3a00 0x5576b05d7930 secure :-1 s=READY pgs=1529 cs=0 l=1 rev1=1 crypto rx=0x7fa698009610 tx=0x7fa6980106e0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.412+0000 7fa6b29d8740 1 -- 172.21.15.86:0/2693057374 shutdown_connections 2024-04-21T08:46:26.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.412+0000 7fa6b29d8740 1 --2- 172.21.15.86:0/2693057374 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5576b064a380 0x5576b064c7a0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.412+0000 7fa6b29d8740 1 --2- 172.21.15.86:0/2693057374 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576b05d3a00 0x5576b05d7930 unknown :-1 s=CLOSED pgs=1529 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.412+0000 7fa6b29d8740 1 --2- 172.21.15.86:0/2693057374 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576b05cae80 0x5576b05d73f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.412+0000 7fa6b29d8740 1 --2- 172.21.15.86:0/2693057374 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576b05ca4d0 0x5576b05dc080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.412+0000 7fa6917fa640 0 client.18774 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:46:26.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.412+0000 7fa6b29d8740 1 -- 172.21.15.86:0/2693057374 >> 172.21.15.86:0/2693057374 conn(0x5576b05c4ba0 msgr2=0x5576b05c5e80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.412+0000 7fa6b29d8740 1 -- 172.21.15.86:0/2693057374 shutdown_connections 2024-04-21T08:46:26.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.412+0000 7fa6b29d8740 1 -- 172.21.15.86:0/2693057374 wait complete. 2024-04-21T08:46:26.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.412+0000 7fa6b29d8740 20 client.18774 trim_cache size 0 max 16384 2024-04-21T08:46:26.419 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:46:26.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.413+0000 7fa6afd10640 1 -- 172.21.15.86:0/1007118173 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa6940a3d20 msgr2=0x7fa6940a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.413+0000 7fa6afd10640 1 --2- 172.21.15.86:0/1007118173 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa6940a3d20 0x7fa6940a61e0 secure :-1 s=READY pgs=246 cs=0 l=1 rev1=1 crypto rx=0x7fa6a00f0390 tx=0x7fa6a00b3fc0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.413+0000 7fa6afd10640 1 -- 172.21.15.86:0/1007118173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6a819bec0 msgr2=0x7fa6a819c370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.413+0000 7fa6afd10640 1 --2- 172.21.15.86:0/1007118173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6a819bec0 0x7fa6a819c370 secure :-1 s=READY pgs=1525 cs=0 l=1 rev1=1 crypto rx=0x7fa6a0005e30 tx=0x7fa6a0034760 comp rx=0 tx=0).stop 2024-04-21T08:46:26.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.413+0000 7fa6afd10640 1 -- 172.21.15.86:0/1007118173 shutdown_connections 2024-04-21T08:46:26.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.413+0000 7fa6afd10640 1 --2- 172.21.15.86:0/1007118173 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa6940a3d20 0x7fa6940a61e0 unknown :-1 s=CLOSED pgs=246 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.413+0000 7fa6afd10640 1 --2- 172.21.15.86:0/1007118173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa6a819bec0 0x7fa6a819c370 unknown :-1 s=CLOSED pgs=1525 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.413+0000 7fa6afd10640 1 --2- 172.21.15.86:0/1007118173 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa6a8106160 0x7fa6a8197b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.413+0000 7fa6afd10640 1 --2- 172.21.15.86:0/1007118173 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa6a8101090 0x7fa6a8197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.413+0000 7fa6afd10640 1 -- 172.21.15.86:0/1007118173 >> 172.21.15.86:0/1007118173 conn(0x7fa6a8077f50 msgr2=0x7fa6a80fefd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.413+0000 7fa6afd10640 1 -- 172.21.15.86:0/1007118173 shutdown_connections 2024-04-21T08:46:26.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.413+0000 7fa6afd10640 1 -- 172.21.15.86:0/1007118173 wait complete. 2024-04-21T08:46:26.423 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:46:26.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.522+0000 7fabd1d60640 1 -- 172.21.15.86:0/3450432013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabcc101090 msgr2=0x7fabcc101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.522+0000 7fabd1d60640 1 --2- 172.21.15.86:0/3450432013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabcc101090 0x7fabcc101470 secure :-1 s=READY pgs=1530 cs=0 l=1 rev1=1 crypto rx=0x7fabbc009900 tx=0x7fabbc02bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:26.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.522+0000 7fabd1d60640 1 -- 172.21.15.86:0/3450432013 shutdown_connections 2024-04-21T08:46:26.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.522+0000 7fabd1d60640 1 --2- 172.21.15.86:0/3450432013 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fabcc106160 0x7fabcc10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.522+0000 7fabd1d60640 1 --2- 172.21.15.86:0/3450432013 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabcc101a40 0x7fabcc105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.523+0000 7fabd1d60640 1 --2- 172.21.15.86:0/3450432013 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabcc101090 0x7fabcc101470 unknown :-1 s=CLOSED pgs=1530 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.523+0000 7fabd1d60640 1 -- 172.21.15.86:0/3450432013 >> 172.21.15.86:0/3450432013 conn(0x7fabcc077f50 msgr2=0x7fabcc078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.523+0000 7fabd1d60640 1 -- 172.21.15.86:0/3450432013 shutdown_connections 2024-04-21T08:46:26.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.523+0000 7fabd1d60640 1 -- 172.21.15.86:0/3450432013 wait complete. 2024-04-21T08:46:26.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.523+0000 7fabd1d60640 1 Processor -- start 2024-04-21T08:46:26.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.523+0000 7fabd1d60640 1 -- start start 2024-04-21T08:46:26.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.523+0000 7fabd1d60640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabcc101090 0x7fabcc197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.523+0000 7fabd1d60640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabcc106160 0x7fabcc197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.523+0000 7fabd1d60640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fabcc19bf40 0x7fabcc19c3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.523+0000 7fabd1d60640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc1110c0 con 0x7fabcc101090 2024-04-21T08:46:26.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.523+0000 7fabd1d60640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc110f40 con 0x7fabcc19bf40 2024-04-21T08:46:26.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.523+0000 7fabd1d60640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc111240 con 0x7fabcc106160 2024-04-21T08:46:26.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.524+0000 7fabcb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabcc101090 0x7fabcc197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.524+0000 7fabcb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabcc101090 0x7fabcc197680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47594/0 (socket says 172.21.15.86:47594) 2024-04-21T08:46:26.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.524+0000 7fabcb7fe640 1 -- 172.21.15.86:0/2875198985 learned_addr learned my addr 172.21.15.86:0/2875198985 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:26.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.524+0000 7fabcbfff640 1 --2- 172.21.15.86:0/2875198985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fabcc19bf40 0x7fabcc19c3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.524+0000 7fabcb7fe640 1 -- 172.21.15.86:0/2875198985 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabcc106160 msgr2=0x7fabcc197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.524+0000 7fabcb7fe640 1 --2- 172.21.15.86:0/2875198985 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabcc106160 0x7fabcc197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.524+0000 7fabcb7fe640 1 -- 172.21.15.86:0/2875198985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fabcc19bf40 msgr2=0x7fabcc19c3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.524+0000 7fabcb7fe640 1 --2- 172.21.15.86:0/2875198985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fabcc19bf40 0x7fabcc19c3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.524+0000 7fabcb7fe640 1 -- 172.21.15.86:0/2875198985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fabcc19ca30 con 0x7fabcc101090 2024-04-21T08:46:26.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.524+0000 7fabcb7fe640 1 --2- 172.21.15.86:0/2875198985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabcc101090 0x7fabcc197680 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fabbc005940 tx=0x7fabbc0045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.524+0000 7fabc8ff9640 1 -- 172.21.15.86:0/2875198985 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc043020 con 0x7fabcc101090 2024-04-21T08:46:26.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.524+0000 7fabc8ff9640 1 -- 172.21.15.86:0/2875198985 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabbc02caf0 con 0x7fabcc101090 2024-04-21T08:46:26.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.524+0000 7fabd1d60640 1 -- 172.21.15.86:0/2875198985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fabcc19ccc0 con 0x7fabcc101090 2024-04-21T08:46:26.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.524+0000 7fabc8ff9640 1 -- 172.21.15.86:0/2875198985 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabbc003af0 con 0x7fabcc101090 2024-04-21T08:46:26.526 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.524+0000 7fabd1d60640 1 -- 172.21.15.86:0/2875198985 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fabcc0693d0 con 0x7fabcc101090 2024-04-21T08:46:26.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.526+0000 7fabc8ff9640 1 -- 172.21.15.86:0/2875198985 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fabbc046020 con 0x7fabcc101090 2024-04-21T08:46:26.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.526+0000 7fabd4a32740 1 Processor -- start 2024-04-21T08:46:26.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.527+0000 7fabd4a32740 1 -- start start 2024-04-21T08:46:26.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.527+0000 7fabd4a32740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e52280e4d0 0x55e52280e8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.527+0000 7fabd4a32740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e52280ee80 0x55e522817430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.527+0000 7fabd4a32740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e522817a00 0x55e522819e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.527+0000 7fabd4a32740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e5227731f0 con 0x55e522817a00 2024-04-21T08:46:26.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.527+0000 7fabd4a32740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55e5227104b0 con 0x55e52280ee80 2024-04-21T08:46:26.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.527+0000 7fabd4a32740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e52276f1b0 con 0x55e52280e4d0 2024-04-21T08:46:26.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.527+0000 7fabcaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e522817a00 0x55e522819e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.527+0000 7fabcaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e522817a00 0x55e522819e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47604/0 (socket says 172.21.15.86:47604) 2024-04-21T08:46:26.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.527+0000 7fabcaffd640 1 -- 172.21.15.86:0/2801566893 learned_addr learned my addr 172.21.15.86:0/2801566893 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:26.536 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.527+0000 7fabc8ff9640 1 --2- 172.21.15.86:0/2875198985 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fabb40a3d20 0x7fabb40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.527+0000 7fabcb7fe640 1 --2- 172.21.15.86:0/2801566893 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e52280ee80 0x55e522817430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.527+0000 7fabcbfff640 1 --2- 172.21.15.86:0/2801566893 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e52280e4d0 0x55e52280e8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.527+0000 7fabcbfff640 1 --2- 172.21.15.86:0/2875198985 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fabb40a3d20 0x7fabb40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.527+0000 7fabcaffd640 1 -- 172.21.15.86:0/2801566893 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e52280e4d0 msgr2=0x55e52280e8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.527+0000 7fabcaffd640 1 --2- 172.21.15.86:0/2801566893 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e52280e4d0 0x55e52280e8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.527+0000 7fabcaffd640 1 -- 172.21.15.86:0/2801566893 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e52280ee80 msgr2=0x55e522817430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.527+0000 7fabcaffd640 1 --2- 172.21.15.86:0/2801566893 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e52280ee80 0x55e522817430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.527+0000 7fabcaffd640 1 -- 172.21.15.86:0/2801566893 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e5227021b0 con 0x55e522817a00 2024-04-21T08:46:26.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.527+0000 7fabc8ff9640 1 -- 172.21.15.86:0/2875198985 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fabbc0f2cb0 con 0x7fabcc101090 2024-04-21T08:46:26.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.527+0000 7fabcaffd640 1 --2- 172.21.15.86:0/2801566893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e522817a00 0x55e522819e50 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fabc000baa0 tx=0x7fabc000bf60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5aa8aa5783ed6762 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.528+0000 7fabcbfff640 1 --2- 172.21.15.86:0/2875198985 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fabb40a3d20 0x7fabb40a61e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fabc4005fd0 tx=0x7fabc4009500 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.528+0000 7fabb1ffb640 1 -- 172.21.15.86:0/2801566893 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc0016070 con 0x55e522817a00 2024-04-21T08:46:26.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.528+0000 7fabb1ffb640 1 -- 172.21.15.86:0/2801566893 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabc00054d0 con 0x55e522817a00 2024-04-21T08:46:26.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.528+0000 7fabb1ffb640 1 -- 172.21.15.86:0/2801566893 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc0004e40 con 0x55e522817a00 2024-04-21T08:46:26.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.528+0000 7fabd4a32740 1 -- 172.21.15.86:0/2801566893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e522817a00 msgr2=0x55e522819e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.528+0000 7fabd4a32740 1 --2- 172.21.15.86:0/2801566893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e522817a00 0x55e522819e50 secure :-1 s=READY pgs=1532 cs=0 l=1 rev1=1 crypto rx=0x7fabc000baa0 tx=0x7fabc000bf60 comp rx=0 tx=0).stop 2024-04-21T08:46:26.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.528+0000 7fabd4a32740 1 -- 172.21.15.86:0/2801566893 shutdown_connections 2024-04-21T08:46:26.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.528+0000 7fabd4a32740 1 --2- 172.21.15.86:0/2801566893 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e522817a00 0x55e522819e50 unknown :-1 s=CLOSED pgs=1532 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.528+0000 7fabd4a32740 1 --2- 172.21.15.86:0/2801566893 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e52280ee80 0x55e522817430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.528+0000 7fabd4a32740 1 --2- 172.21.15.86:0/2801566893 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e52280e4d0 0x55e52280e8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.528+0000 7fabd4a32740 1 -- 172.21.15.86:0/2801566893 >> 172.21.15.86:0/2801566893 conn(0x55e522808ba0 msgr2=0x55e522808f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.528+0000 7fabd4a32740 1 -- 172.21.15.86:0/2801566893 shutdown_connections 2024-04-21T08:46:26.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.528+0000 7fabd4a32740 1 -- 172.21.15.86:0/2801566893 wait complete. 2024-04-21T08:46:26.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.529+0000 7fabd4a32740 1 -- start start 2024-04-21T08:46:26.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.529+0000 7fabd4a32740 1 Processor -- start 2024-04-21T08:46:26.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.529+0000 7fabd4a32740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e52280e4d0 0x55e5228172f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.529+0000 7fabd4a32740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e52280ee80 0x55e522814680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.529+0000 7fabd4a32740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e522817a00 0x55e522814c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.529+0000 7fabd4a32740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e52281a970 con 0x55e52280ee80 2024-04-21T08:46:26.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.529+0000 7fabd4a32740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55e52272c500 con 0x55e52280e4d0 2024-04-21T08:46:26.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.529+0000 7fabd4a32740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e52281aaf0 con 0x55e522817a00 2024-04-21T08:46:26.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.529+0000 7fabd4a32740 10 client.0 _ll_register_callbacks cb 0x55e5226fab80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:26.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.529+0000 7fabcaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e52280ee80 0x55e522814680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.529+0000 7fabcaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e52280ee80 0x55e522814680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47620/0 (socket says 172.21.15.86:47620) 2024-04-21T08:46:26.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.529+0000 7fabcaffd640 1 -- 172.21.15.86:0/2845399802 learned_addr learned my addr 172.21.15.86:0/2845399802 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:26.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.529+0000 7fabcb7fe640 1 --2- 172.21.15.86:0/2845399802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e52280e4d0 0x55e5228172f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.529+0000 7fabcbfff640 1 --2- 172.21.15.86:0/2845399802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e522817a00 0x55e522814c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.529+0000 7fabcaffd640 1 -- 172.21.15.86:0/2845399802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e522817a00 msgr2=0x55e522814c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.529+0000 7fabcaffd640 1 --2- 172.21.15.86:0/2845399802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e522817a00 0x55e522814c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.529+0000 7fabcaffd640 1 -- 172.21.15.86:0/2845399802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e52280e4d0 msgr2=0x55e5228172f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.529+0000 7fabcaffd640 1 --2- 172.21.15.86:0/2845399802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e52280e4d0 0x55e5228172f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.529+0000 7fabcaffd640 1 -- 172.21.15.86:0/2845399802 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e5224853f0 con 0x55e52280ee80 2024-04-21T08:46:26.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.530+0000 7fabcaffd640 1 --2- 172.21.15.86:0/2845399802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e52280ee80 0x55e522814680 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fabc0004a40 tx=0x7fabc000b760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d0c80d0638813c61 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.539 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.530+0000 7fabaaffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:46:26.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.530+0000 7fabaaffd640 1 -- 172.21.15.86:0/2845399802 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc0016030 con 0x55e52280ee80 2024-04-21T08:46:26.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.530+0000 7fabaaffd640 1 -- 172.21.15.86:0/2845399802 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fabc000dc70 con 0x55e52280ee80 2024-04-21T08:46:26.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.530+0000 7fabd4a32740 1 -- 172.21.15.86:0/2845399802 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabc00054d0 con 0x55e52280ee80 2024-04-21T08:46:26.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.530+0000 7fabaaffd640 1 -- 172.21.15.86:0/2845399802 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fabc000de70 con 0x55e52280ee80 2024-04-21T08:46:26.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.530+0000 7fabd4a32740 10 client.18786 fetch_fsmap learned FSMap version 29 2024-04-21T08:46:26.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.530+0000 7fabd4a32740 1 -- 172.21.15.86:0/2845399802 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e52254f7d0 con 0x55e52280ee80 2024-04-21T08:46:26.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.530+0000 7fabaaffd640 1 -- 172.21.15.86:0/2845399802 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabc00054d0 con 0x55e52280ee80 2024-04-21T08:46:26.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.530+0000 7fabaaffd640 1 -- 172.21.15.86:0/2845399802 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fabc0022070 con 0x55e52280ee80 2024-04-21T08:46:26.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.530+0000 7fabd4a32740 10 client.18786 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:46:26.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.530+0000 7fabd4a32740 10 client.18786 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:46:26.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.530+0000 7fabd4a32740 20 client.18786 populate_metadata read hostname 'smithi086' 2024-04-21T08:46:26.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.530+0000 7fabd4a32740 1 --2- 172.21.15.86:0/2845399802 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55e522827b40 0x55e522829f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.530+0000 7fabd4a32740 4 client.18786 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:26.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.530+0000 7fabd4a32740 1 -- 172.21.15.86:0/2845399802 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e522712930 con 0x55e522827b40 2024-04-21T08:46:26.540 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.531+0000 7fabcb7fe640 1 --2- 172.21.15.86:0/2845399802 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55e522827b40 0x55e522829f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.531+0000 7fabaaffd640 10 client.18786 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:46:26.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.531+0000 7fabcb7fe640 1 --2- 172.21.15.86:0/2845399802 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55e522827b40 0x55e522829f30 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=290abc055b5195d7 server_cookie=aaa9f5724dbd0533 in_seq=0 out_seq=0 2024-04-21T08:46:26.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.532+0000 7fabaaffd640 1 -- 172.21.15.86:0/2845399802 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e522712930 con 0x55e522827b40 2024-04-21T08:46:26.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.532+0000 7fabaaffd640 10 client.18786 handle_command_reply: tid=0 2024-04-21T08:46:26.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.532+0000 7fabd4a32740 1 client.18786 shutdown 2024-04-21T08:46:26.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.532+0000 7fabd4a32740 1 -- 172.21.15.86:0/2845399802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e52280ee80 msgr2=0x55e522814680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.532+0000 7fabd4a32740 1 --2- 172.21.15.86:0/2845399802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e52280ee80 0x55e522814680 secure :-1 s=READY pgs=1533 cs=0 l=1 rev1=1 crypto rx=0x7fabc0004a40 tx=0x7fabc000b760 comp rx=0 tx=0).stop 2024-04-21T08:46:26.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.532+0000 7fabd4a32740 1 -- 172.21.15.86:0/2845399802 shutdown_connections 2024-04-21T08:46:26.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.532+0000 7fabd4a32740 1 --2- 172.21.15.86:0/2845399802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e522817a00 0x55e522814c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.532+0000 7fabd4a32740 1 --2- 172.21.15.86:0/2845399802 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55e522827b40 0x55e522829f30 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.532+0000 7fabd4a32740 1 --2- 172.21.15.86:0/2845399802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e52280ee80 0x55e522814680 unknown :-1 s=CLOSED pgs=1533 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.532+0000 7fabd4a32740 1 --2- 172.21.15.86:0/2845399802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e52280e4d0 0x55e5228172f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.532+0000 7fabaaffd640 0 client.18786 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:46:26.541 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.532+0000 7fabd4a32740 1 -- 172.21.15.86:0/2845399802 >> 172.21.15.86:0/2845399802 conn(0x55e522808ba0 msgr2=0x55e52280a8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.532+0000 7fabd4a32740 1 -- 172.21.15.86:0/2845399802 shutdown_connections 2024-04-21T08:46:26.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.532+0000 7fabd4a32740 1 -- 172.21.15.86:0/2845399802 wait complete. 2024-04-21T08:46:26.542 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.532+0000 7fabd4a32740 20 client.18786 trim_cache size 0 max 16384 2024-04-21T08:46:26.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.545+0000 7fabd4a32740 1 Processor -- start 2024-04-21T08:46:26.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.545+0000 7fabd4a32740 1 -- start start 2024-04-21T08:46:26.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.545+0000 7fabd4a32740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e52280e4d0 0x55e522823830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.545+0000 7fabd4a32740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e52280ee80 0x55e522820120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.545+0000 7fabd4a32740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e522817a00 0x55e522816db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.545+0000 7fabd4a32740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc1af330 con 0x55e522817a00 2024-04-21T08:46:26.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.545+0000 7fabd4a32740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc1b0e70 con 0x55e52280ee80 2024-04-21T08:46:26.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.545+0000 7fabd4a32740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc1afa00 con 0x55e52280e4d0 2024-04-21T08:46:26.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.545+0000 7fabcbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e522817a00 0x55e522816db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.545+0000 7fabcbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e522817a00 0x55e522816db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47636/0 (socket says 172.21.15.86:47636) 2024-04-21T08:46:26.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.545+0000 7fabcbfff640 1 -- 172.21.15.86:0/1926459884 learned_addr learned my addr 172.21.15.86:0/1926459884 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:26.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.546+0000 7fabcbfff640 1 -- 172.21.15.86:0/1926459884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e52280e4d0 msgr2=0x55e522823830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.546+0000 7fabcbfff640 1 --2- 172.21.15.86:0/1926459884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e52280e4d0 0x55e522823830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.546+0000 7fabcbfff640 1 -- 172.21.15.86:0/1926459884 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e52280ee80 msgr2=0x55e522820120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.546+0000 7fabcbfff640 1 --2- 172.21.15.86:0/1926459884 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e52280ee80 0x55e522820120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.546+0000 7fabcbfff640 1 -- 172.21.15.86:0/1926459884 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e522814f70 con 0x55e522817a00 2024-04-21T08:46:26.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.546+0000 7fabcbfff640 1 --2- 172.21.15.86:0/1926459884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e522817a00 0x55e522816db0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fabc4010890 tx=0x7fabc4018000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9bb132fcae89c1d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.546+0000 7fabb17fa640 1 -- 172.21.15.86:0/1926459884 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc4029630 con 0x55e522817a00 2024-04-21T08:46:26.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.546+0000 7fabb17fa640 1 -- 172.21.15.86:0/1926459884 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabc401f8c0 con 0x55e522817a00 2024-04-21T08:46:26.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.546+0000 7fabb17fa640 1 -- 172.21.15.86:0/1926459884 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc401e8f0 con 0x55e522817a00 2024-04-21T08:46:26.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.546+0000 7fabd4a32740 1 -- 172.21.15.86:0/1926459884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e522817a00 msgr2=0x55e522816db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.546+0000 7fabd4a32740 1 --2- 172.21.15.86:0/1926459884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e522817a00 0x55e522816db0 secure :-1 s=READY pgs=1534 cs=0 l=1 rev1=1 crypto rx=0x7fabc4010890 tx=0x7fabc4018000 comp rx=0 tx=0).stop 2024-04-21T08:46:26.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.546+0000 7fabd4a32740 1 -- 172.21.15.86:0/1926459884 shutdown_connections 2024-04-21T08:46:26.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.546+0000 7fabd4a32740 1 --2- 172.21.15.86:0/1926459884 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e522817a00 0x55e522816db0 unknown :-1 s=CLOSED pgs=1534 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.546+0000 7fabd4a32740 1 --2- 172.21.15.86:0/1926459884 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e52280ee80 0x55e522820120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.546+0000 7fabd4a32740 1 --2- 172.21.15.86:0/1926459884 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e52280e4d0 0x55e522823830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.547+0000 7fabd4a32740 1 -- 172.21.15.86:0/1926459884 >> 172.21.15.86:0/1926459884 conn(0x55e522808ba0 msgr2=0x55e522807c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.547+0000 7fabd4a32740 1 -- 172.21.15.86:0/1926459884 shutdown_connections 2024-04-21T08:46:26.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.547+0000 7fabd4a32740 1 -- 172.21.15.86:0/1926459884 wait complete. 2024-04-21T08:46:26.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.547+0000 7fabd4a32740 1 -- start start 2024-04-21T08:46:26.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.547+0000 7fabd4a32740 1 Processor -- start 2024-04-21T08:46:26.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.547+0000 7fabd4a32740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e52280e4d0 0x55e5228167d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.547+0000 7fabd4a32740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e52280ee80 0x55e522816d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.547+0000 7fabd4a32740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e522817a00 0x55e522817250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.547+0000 7fabd4a32740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc1b07a0 con 0x55e52280ee80 2024-04-21T08:46:26.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.547+0000 7fabd4a32740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc1b00d0 con 0x55e522817a00 2024-04-21T08:46:26.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.547+0000 7fabd4a32740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fabcc1ad330 con 0x55e52280e4d0 2024-04-21T08:46:26.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.547+0000 7fabd4a32740 10 client.0 _ll_register_callbacks cb 0x7fabc0005710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:26.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.547+0000 7fabcaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e52280ee80 0x55e522816d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.547+0000 7fabcaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e52280ee80 0x55e522816d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47644/0 (socket says 172.21.15.86:47644) 2024-04-21T08:46:26.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.547+0000 7fabcaffd640 1 -- 172.21.15.86:0/4134564569 learned_addr learned my addr 172.21.15.86:0/4134564569 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:26.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.547+0000 7fabcb7fe640 1 --2- 172.21.15.86:0/4134564569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e52280e4d0 0x55e5228167d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.548+0000 7fabcbfff640 1 --2- 172.21.15.86:0/4134564569 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e522817a00 0x55e522817250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.548+0000 7fabcaffd640 1 -- 172.21.15.86:0/4134564569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e52280e4d0 msgr2=0x55e5228167d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.548+0000 7fabcaffd640 1 --2- 172.21.15.86:0/4134564569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e52280e4d0 0x55e5228167d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.548+0000 7fabcaffd640 1 -- 172.21.15.86:0/4134564569 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e522817a00 msgr2=0x55e522817250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.548+0000 7fabcaffd640 1 --2- 172.21.15.86:0/4134564569 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e522817a00 0x55e522817250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.548+0000 7fabcaffd640 1 -- 172.21.15.86:0/4134564569 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e522891030 con 0x55e52280ee80 2024-04-21T08:46:26.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.548+0000 7fabcbfff640 1 --2- 172.21.15.86:0/4134564569 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e522817a00 0x55e522817250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:26.553 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.548+0000 7fabcaffd640 1 --2- 172.21.15.86:0/4134564569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e52280ee80 0x55e522816d10 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fabc0001eb0 tx=0x7fabc000d880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a75e7fa93ca71ab8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.548+0000 7fabaaffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:46:26.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.548+0000 7fabaaffd640 1 -- 172.21.15.86:0/4134564569 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fabc0016040 con 0x55e52280ee80 2024-04-21T08:46:26.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.548+0000 7fabaaffd640 1 -- 172.21.15.86:0/4134564569 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fabc0005090 con 0x55e52280ee80 2024-04-21T08:46:26.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.548+0000 7fabd4a32740 1 -- 172.21.15.86:0/4134564569 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fabc401f8c0 con 0x55e52280ee80 2024-04-21T08:46:26.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.548+0000 7fabaaffd640 1 -- 172.21.15.86:0/4134564569 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fabc000dc80 con 0x55e52280ee80 2024-04-21T08:46:26.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.548+0000 7fabd4a32740 10 client.18792 fetch_fsmap learned FSMap version 29 2024-04-21T08:46:26.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.548+0000 7fabd4a32740 1 -- 172.21.15.86:0/4134564569 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e522892300 con 0x55e52280ee80 2024-04-21T08:46:26.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.548+0000 7fabaaffd640 1 -- 172.21.15.86:0/4134564569 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fabc401f8c0 con 0x55e52280ee80 2024-04-21T08:46:26.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.548+0000 7fabaaffd640 1 -- 172.21.15.86:0/4134564569 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fabc0022070 con 0x55e52280ee80 2024-04-21T08:46:26.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.548+0000 7fabd4a32740 10 client.18792 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:46:26.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.548+0000 7fabd4a32740 10 client.18792 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:46:26.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.548+0000 7fabd4a32740 20 client.18792 populate_metadata read hostname 'smithi086' 2024-04-21T08:46:26.554 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.548+0000 7fabd4a32740 1 --2- 172.21.15.86:0/4134564569 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55e522892a50 0x55e522894e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.549+0000 7fabd4a32740 4 client.18792 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:46:26.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.549+0000 7fabd4a32740 1 -- 172.21.15.86:0/4134564569 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e522780c10 con 0x55e522892a50 2024-04-21T08:46:26.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.549+0000 7fabcb7fe640 1 --2- 172.21.15.86:0/4134564569 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55e522892a50 0x55e522894e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.549+0000 7fabcb7fe640 1 --2- 172.21.15.86:0/4134564569 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55e522892a50 0x55e522894e40 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=cda08c72184a525d server_cookie=534ab68f4f059004 in_seq=0 out_seq=0 2024-04-21T08:46:26.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.549+0000 7fabaaffd640 10 client.18792 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:46:26.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.549+0000 7fabaaffd640 1 -- 172.21.15.86:0/4134564569 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e522780c10 con 0x55e522892a50 2024-04-21T08:46:26.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.549+0000 7fabaaffd640 10 client.18792 handle_command_reply: tid=0 2024-04-21T08:46:26.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.550+0000 7fabd4a32740 1 client.18792 shutdown 2024-04-21T08:46:26.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.550+0000 7fabd4a32740 1 -- 172.21.15.86:0/4134564569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e52280ee80 msgr2=0x55e522816d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.550+0000 7fabd4a32740 1 --2- 172.21.15.86:0/4134564569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e52280ee80 0x55e522816d10 secure :-1 s=READY pgs=1535 cs=0 l=1 rev1=1 crypto rx=0x7fabc0001eb0 tx=0x7fabc000d880 comp rx=0 tx=0).stop 2024-04-21T08:46:26.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.550+0000 7fabd4a32740 1 -- 172.21.15.86:0/4134564569 shutdown_connections 2024-04-21T08:46:26.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.550+0000 7fabd4a32740 1 --2- 172.21.15.86:0/4134564569 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e522817a00 0x55e522817250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.550+0000 7fabd4a32740 1 --2- 172.21.15.86:0/4134564569 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55e522892a50 0x55e522894e40 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.550+0000 7fabd4a32740 1 --2- 172.21.15.86:0/4134564569 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e52280ee80 0x55e522816d10 unknown :-1 s=CLOSED pgs=1535 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.555 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.550+0000 7fabd4a32740 1 --2- 172.21.15.86:0/4134564569 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e52280e4d0 0x55e5228167d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.550+0000 7fabaaffd640 0 client.18792 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:46:26.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.550+0000 7fabd4a32740 1 -- 172.21.15.86:0/4134564569 >> 172.21.15.86:0/4134564569 conn(0x55e522808ba0 msgr2=0x55e52280f260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.550+0000 7fabd4a32740 1 -- 172.21.15.86:0/4134564569 shutdown_connections 2024-04-21T08:46:26.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.550+0000 7fabd4a32740 1 -- 172.21.15.86:0/4134564569 wait complete. 2024-04-21T08:46:26.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.550+0000 7fabd4a32740 20 client.18792 trim_cache size 0 max 16384 2024-04-21T08:46:26.556 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:46:26.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.550+0000 7fabd1d60640 1 -- 172.21.15.86:0/2875198985 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fabb40a3d20 msgr2=0x7fabb40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.550+0000 7fabd1d60640 1 --2- 172.21.15.86:0/2875198985 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fabb40a3d20 0x7fabb40a61e0 secure :-1 s=READY pgs=247 cs=0 l=1 rev1=1 crypto rx=0x7fabc4005fd0 tx=0x7fabc4009500 comp rx=0 tx=0).stop 2024-04-21T08:46:26.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.550+0000 7fabd1d60640 1 -- 172.21.15.86:0/2875198985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabcc101090 msgr2=0x7fabcc197680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.550+0000 7fabd1d60640 1 --2- 172.21.15.86:0/2875198985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabcc101090 0x7fabcc197680 secure :-1 s=READY pgs=1531 cs=0 l=1 rev1=1 crypto rx=0x7fabbc005940 tx=0x7fabbc0045e0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.550+0000 7fabd1d60640 1 -- 172.21.15.86:0/2875198985 shutdown_connections 2024-04-21T08:46:26.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.550+0000 7fabd1d60640 1 --2- 172.21.15.86:0/2875198985 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fabb40a3d20 0x7fabb40a61e0 unknown :-1 s=CLOSED pgs=247 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.550+0000 7fabd1d60640 1 --2- 172.21.15.86:0/2875198985 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fabcc19bf40 0x7fabcc19c3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.550+0000 7fabd1d60640 1 --2- 172.21.15.86:0/2875198985 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fabcc106160 0x7fabcc197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.550+0000 7fabd1d60640 1 --2- 172.21.15.86:0/2875198985 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fabcc101090 0x7fabcc197680 unknown :-1 s=CLOSED pgs=1531 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.550+0000 7fabd1d60640 1 -- 172.21.15.86:0/2875198985 >> 172.21.15.86:0/2875198985 conn(0x7fabcc077f50 msgr2=0x7fabcc0ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.550+0000 7fabd1d60640 1 -- 172.21.15.86:0/2875198985 shutdown_connections 2024-04-21T08:46:26.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.550+0000 7fabd1d60640 1 -- 172.21.15.86:0/2875198985 wait complete. 2024-04-21T08:46:26.560 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:46:26.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.661+0000 7f741cc06640 1 -- 172.21.15.86:0/3017006010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7418102b90 msgr2=0x7f7418106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.661+0000 7f741cc06640 1 --2- 172.21.15.86:0/3017006010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7418102b90 0x7f7418106bd0 secure :-1 s=READY pgs=1536 cs=0 l=1 rev1=1 crypto rx=0x7f7400009900 tx=0x7f740002bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:26.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.661+0000 7f741cc06640 1 -- 172.21.15.86:0/3017006010 shutdown_connections 2024-04-21T08:46:26.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.661+0000 7f741cc06640 1 --2- 172.21.15.86:0/3017006010 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f74181072b0 0x7f741810ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.661+0000 7f741cc06640 1 --2- 172.21.15.86:0/3017006010 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7418102b90 0x7f7418106bd0 unknown :-1 s=CLOSED pgs=1536 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.661+0000 7f741cc06640 1 --2- 172.21.15.86:0/3017006010 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74181021e0 0x7f74181025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.661+0000 7f741cc06640 1 -- 172.21.15.86:0/3017006010 >> 172.21.15.86:0/3017006010 conn(0x7f74180fda60 msgr2=0x7f74180ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.661+0000 7f741cc06640 1 -- 172.21.15.86:0/3017006010 shutdown_connections 2024-04-21T08:46:26.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.661+0000 7f741cc06640 1 -- 172.21.15.86:0/3017006010 wait complete. 2024-04-21T08:46:26.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.662+0000 7f741cc06640 1 Processor -- start 2024-04-21T08:46:26.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.662+0000 7f741cc06640 1 -- start start 2024-04-21T08:46:26.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.662+0000 7f741cc06640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74181021e0 0x7f74181987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.662+0000 7f741cc06640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74181072b0 0x7f7418198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.662+0000 7f741cc06640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f741819d020 0x7f741819d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.662+0000 7f741cc06640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f74181121f0 con 0x7f74181072b0 2024-04-21T08:46:26.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.662+0000 7f741cc06640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7418112070 con 0x7f741819d020 2024-04-21T08:46:26.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.662+0000 7f741cc06640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7418112370 con 0x7f74181021e0 2024-04-21T08:46:26.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.662+0000 7f740ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74181072b0 0x7f7418198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.662+0000 7f740ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74181072b0 0x7f7418198cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47660/0 (socket says 172.21.15.86:47660) 2024-04-21T08:46:26.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.662+0000 7f740ffff640 1 -- 172.21.15.86:0/4115269751 learned_addr learned my addr 172.21.15.86:0/4115269751 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:26.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.663+0000 7f7416d9d640 1 --2- 172.21.15.86:0/4115269751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f741819d020 0x7f741819d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.663+0000 7f741659c640 1 --2- 172.21.15.86:0/4115269751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74181021e0 0x7f74181987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.663+0000 7f740ffff640 1 -- 172.21.15.86:0/4115269751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74181021e0 msgr2=0x7f74181987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.663+0000 7f740ffff640 1 --2- 172.21.15.86:0/4115269751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74181021e0 0x7f74181987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.663+0000 7f740ffff640 1 -- 172.21.15.86:0/4115269751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f741819d020 msgr2=0x7f741819d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.663+0000 7f740ffff640 1 --2- 172.21.15.86:0/4115269751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f741819d020 0x7f741819d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.663+0000 7f740ffff640 1 -- 172.21.15.86:0/4115269751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f741819db10 con 0x7f74181072b0 2024-04-21T08:46:26.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.663+0000 7f741659c640 1 --2- 172.21.15.86:0/4115269751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74181021e0 0x7f74181987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:26.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.663+0000 7f7416d9d640 1 --2- 172.21.15.86:0/4115269751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f741819d020 0x7f741819d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:26.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.663+0000 7f740ffff640 1 --2- 172.21.15.86:0/4115269751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74181072b0 0x7f7418198cf0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f7400005ec0 tx=0x7f740002ca10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.663+0000 7f740f7fe640 1 -- 172.21.15.86:0/4115269751 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7400043020 con 0x7f74181072b0 2024-04-21T08:46:26.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.663+0000 7f740f7fe640 1 -- 172.21.15.86:0/4115269751 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74000052b0 con 0x7f74181072b0 2024-04-21T08:46:26.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.663+0000 7f741cc06640 1 -- 172.21.15.86:0/4115269751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f741819dda0 con 0x7f74181072b0 2024-04-21T08:46:26.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.663+0000 7f740f7fe640 1 -- 172.21.15.86:0/4115269751 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7400003d40 con 0x7f74181072b0 2024-04-21T08:46:26.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.663+0000 7f741cc06640 1 -- 172.21.15.86:0/4115269751 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f74181a55c0 con 0x7f74181072b0 2024-04-21T08:46:26.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.665+0000 7f740f7fe640 1 -- 172.21.15.86:0/4115269751 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7400046020 con 0x7f74181072b0 2024-04-21T08:46:26.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.665+0000 7f741f7f6740 1 Processor -- start 2024-04-21T08:46:26.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.666+0000 7f741f7f6740 1 -- start start 2024-04-21T08:46:26.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.666+0000 7f741f7f6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643c6686330 0x5643c6686710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.666+0000 7f741f7f6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643c6686ce0 0x5643c668f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.666+0000 7f741f7f6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643c668fe40 0x5643c66902c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.666+0000 7f741f7f6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5643c65eb1f0 con 0x5643c668fe40 2024-04-21T08:46:26.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.666+0000 7f741f7f6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5643c65884b0 con 0x5643c6686330 2024-04-21T08:46:26.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.666+0000 7f741f7f6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5643c65e71b0 con 0x5643c6686ce0 2024-04-21T08:46:26.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.666+0000 7f740f7fe640 1 --2- 172.21.15.86:0/4115269751 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f74040a3d20 0x7f74040a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.666+0000 7f740ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643c668fe40 0x5643c66902c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.666+0000 7f740ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643c668fe40 0x5643c66902c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47662/0 (socket says 172.21.15.86:47662) 2024-04-21T08:46:26.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.666+0000 7f740ffff640 1 -- 172.21.15.86:0/3447524996 learned_addr learned my addr 172.21.15.86:0/3447524996 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:26.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.666+0000 7f7416d9d640 1 --2- 172.21.15.86:0/4115269751 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f74040a3d20 0x7f74040a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.666+0000 7f741659c640 1 --2- 172.21.15.86:0/3447524996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643c6686ce0 0x5643c668f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.666+0000 7f7416d9d640 1 --2- 172.21.15.86:0/3447524996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643c6686330 0x5643c6686710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.666+0000 7f740f7fe640 1 -- 172.21.15.86:0/4115269751 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f74000f2ee0 con 0x7f74181072b0 2024-04-21T08:46:26.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.666+0000 7f740ffff640 1 -- 172.21.15.86:0/3447524996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643c6686ce0 msgr2=0x5643c668f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.666+0000 7f740ffff640 1 --2- 172.21.15.86:0/3447524996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643c6686ce0 0x5643c668f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.666+0000 7f740ffff640 1 -- 172.21.15.86:0/3447524996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643c6686330 msgr2=0x5643c6686710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.666+0000 7f740ffff640 1 --2- 172.21.15.86:0/3447524996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643c6686330 0x5643c6686710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.666+0000 7f740ffff640 1 -- 172.21.15.86:0/3447524996 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643c657a1b0 con 0x5643c668fe40 2024-04-21T08:46:26.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.666+0000 7f741659c640 1 --2- 172.21.15.86:0/3447524996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643c6686ce0 0x5643c668f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:26.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.667+0000 7f7416d9d640 1 --2- 172.21.15.86:0/4115269751 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f74040a3d20 0x7f74040a61e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7410007fd0 tx=0x7f7410007d00 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.667+0000 7f740ffff640 1 --2- 172.21.15.86:0/3447524996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643c668fe40 0x5643c66902c0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f74000f02b0 tx=0x7f7400003840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e3ceceeaadc93d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.667+0000 7f740cff9640 1 -- 172.21.15.86:0/3447524996 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74000be700 con 0x5643c668fe40 2024-04-21T08:46:26.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.667+0000 7f740cff9640 1 -- 172.21.15.86:0/3447524996 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74000bf2c0 con 0x5643c668fe40 2024-04-21T08:46:26.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.667+0000 7f740cff9640 1 -- 172.21.15.86:0/3447524996 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74000be520 con 0x5643c668fe40 2024-04-21T08:46:26.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.667+0000 7f741f7f6740 1 -- 172.21.15.86:0/3447524996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643c668fe40 msgr2=0x5643c66902c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.667+0000 7f741f7f6740 1 --2- 172.21.15.86:0/3447524996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643c668fe40 0x5643c66902c0 secure :-1 s=READY pgs=1538 cs=0 l=1 rev1=1 crypto rx=0x7f74000f02b0 tx=0x7f7400003840 comp rx=0 tx=0).stop 2024-04-21T08:46:26.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.668+0000 7f741f7f6740 1 -- 172.21.15.86:0/3447524996 shutdown_connections 2024-04-21T08:46:26.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.668+0000 7f741f7f6740 1 --2- 172.21.15.86:0/3447524996 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643c668fe40 0x5643c66902c0 unknown :-1 s=CLOSED pgs=1538 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.668+0000 7f741f7f6740 1 --2- 172.21.15.86:0/3447524996 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643c6686ce0 0x5643c668f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.668+0000 7f741f7f6740 1 --2- 172.21.15.86:0/3447524996 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643c6686330 0x5643c6686710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.668+0000 7f741f7f6740 1 -- 172.21.15.86:0/3447524996 >> 172.21.15.86:0/3447524996 conn(0x5643c6680ba0 msgr2=0x5643c6680f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.668+0000 7f741f7f6740 1 -- 172.21.15.86:0/3447524996 shutdown_connections 2024-04-21T08:46:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.668+0000 7f741f7f6740 1 -- 172.21.15.86:0/3447524996 wait complete. 2024-04-21T08:46:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.668+0000 7f741f7f6740 1 -- start start 2024-04-21T08:46:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.668+0000 7f741f7f6740 1 Processor -- start 2024-04-21T08:46:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.668+0000 7f741f7f6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643c6686330 0x5643c66961e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.668+0000 7f741f7f6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643c6686ce0 0x5643c6693570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.668+0000 7f741f7f6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643c668fe40 0x5643c6693ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.668+0000 7f741f7f6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5643c6696b80 con 0x5643c6686ce0 2024-04-21T08:46:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.668+0000 7f741f7f6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5643c65a4500 con 0x5643c668fe40 2024-04-21T08:46:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.668+0000 7f741f7f6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5643c6696d00 con 0x5643c6686330 2024-04-21T08:46:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.668+0000 7f741f7f6740 10 client.0 _ll_register_callbacks cb 0x5643c6572b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.668+0000 7f740ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643c6686ce0 0x5643c6693570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.668+0000 7f740ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643c6686ce0 0x5643c6693570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47678/0 (socket says 172.21.15.86:47678) 2024-04-21T08:46:26.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.668+0000 7f740ffff640 1 -- 172.21.15.86:0/2289973685 learned_addr learned my addr 172.21.15.86:0/2289973685 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.668+0000 7f7416d9d640 1 --2- 172.21.15.86:0/2289973685 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643c668fe40 0x5643c6693ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.669+0000 7f741659c640 1 --2- 172.21.15.86:0/2289973685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643c6686330 0x5643c66961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.669+0000 7f740ffff640 1 -- 172.21.15.86:0/2289973685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643c6686330 msgr2=0x5643c66961e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.669+0000 7f740ffff640 1 --2- 172.21.15.86:0/2289973685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643c6686330 0x5643c66961e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.669+0000 7f740ffff640 1 -- 172.21.15.86:0/2289973685 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643c668fe40 msgr2=0x5643c6693ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.669+0000 7f740ffff640 1 --2- 172.21.15.86:0/2289973685 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643c668fe40 0x5643c6693ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.669+0000 7f740ffff640 1 -- 172.21.15.86:0/2289973685 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643c62fd3f0 con 0x5643c6686ce0 2024-04-21T08:46:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.669+0000 7f741659c640 1 --2- 172.21.15.86:0/2289973685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643c6686330 0x5643c66961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.669+0000 7f740ffff640 1 --2- 172.21.15.86:0/2289973685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643c6686ce0 0x5643c6693570 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f74000bb590 tx=0x7f74000bebc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4abda1a6a4c05cc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.669+0000 7f73fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:46:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.669+0000 7f73fdffb640 1 -- 172.21.15.86:0/2289973685 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74000c2f70 con 0x5643c6686ce0 2024-04-21T08:46:26.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.669+0000 7f741f7f6740 1 -- 172.21.15.86:0/2289973685 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f74000bf2c0 con 0x5643c6686ce0 2024-04-21T08:46:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.669+0000 7f73fdffb640 1 -- 172.21.15.86:0/2289973685 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f74000bb230 con 0x5643c6686ce0 2024-04-21T08:46:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.669+0000 7f73fdffb640 1 -- 172.21.15.86:0/2289973685 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f74000c7830 con 0x5643c6686ce0 2024-04-21T08:46:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.669+0000 7f741f7f6740 10 client.18804 fetch_fsmap learned FSMap version 29 2024-04-21T08:46:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.669+0000 7f741f7f6740 1 -- 172.21.15.86:0/2289973685 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5643c63c77d0 con 0x5643c6686ce0 2024-04-21T08:46:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.669+0000 7f73fdffb640 1 -- 172.21.15.86:0/2289973685 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f74000bf2c0 con 0x5643c6686ce0 2024-04-21T08:46:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.669+0000 7f73fdffb640 1 -- 172.21.15.86:0/2289973685 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7400086030 con 0x5643c6686ce0 2024-04-21T08:46:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.669+0000 7f741f7f6740 10 client.18804 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:46:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.669+0000 7f741f7f6740 10 client.18804 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.669+0000 7f741f7f6740 20 client.18804 populate_metadata read hostname 'smithi086' 2024-04-21T08:46:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.669+0000 7f741f7f6740 1 --2- 172.21.15.86:0/2289973685 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5643c66a3be0 0x5643c66a5fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.670+0000 7f741f7f6740 4 client.18804 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.670+0000 7f741f7f6740 1 -- 172.21.15.86:0/2289973685 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643c658a930 con 0x5643c66a3be0 2024-04-21T08:46:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.670+0000 7f741659c640 1 --2- 172.21.15.86:0/2289973685 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5643c66a3be0 0x5643c66a5fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.670+0000 7f73fdffb640 10 client.18804 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:26.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.670+0000 7f741659c640 1 --2- 172.21.15.86:0/2289973685 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5643c66a3be0 0x5643c66a5fd0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=decfbca2b5976528 server_cookie=afc06a90442980bd in_seq=0 out_seq=0 2024-04-21T08:46:26.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.671+0000 7f73fdffb640 1 -- 172.21.15.86:0/2289973685 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5643c658a930 con 0x5643c66a3be0 2024-04-21T08:46:26.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.671+0000 7f73fdffb640 10 client.18804 handle_command_reply: tid=0 2024-04-21T08:46:26.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.671+0000 7f741f7f6740 1 client.18804 shutdown 2024-04-21T08:46:26.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.671+0000 7f741f7f6740 1 -- 172.21.15.86:0/2289973685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643c6686ce0 msgr2=0x5643c6693570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.671+0000 7f741f7f6740 1 --2- 172.21.15.86:0/2289973685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643c6686ce0 0x5643c6693570 secure :-1 s=READY pgs=1539 cs=0 l=1 rev1=1 crypto rx=0x7f74000bb590 tx=0x7f74000bebc0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.671+0000 7f741f7f6740 1 -- 172.21.15.86:0/2289973685 shutdown_connections 2024-04-21T08:46:26.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.671+0000 7f741f7f6740 1 --2- 172.21.15.86:0/2289973685 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5643c66a3be0 0x5643c66a5fd0 crc :-1 s=READY pgs=233 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.671+0000 7f741f7f6740 1 --2- 172.21.15.86:0/2289973685 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643c668fe40 0x5643c6693ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.671+0000 7f741f7f6740 1 --2- 172.21.15.86:0/2289973685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643c6686ce0 0x5643c6693570 unknown :-1 s=CLOSED pgs=1539 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.671+0000 7f741f7f6740 1 --2- 172.21.15.86:0/2289973685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643c6686330 0x5643c66961e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.671+0000 7f73fdffb640 0 client.18804 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:26.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.671+0000 7f741f7f6740 1 -- 172.21.15.86:0/2289973685 >> 172.21.15.86:0/2289973685 conn(0x5643c6680ba0 msgr2=0x5643c66828f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.671+0000 7f741f7f6740 1 -- 172.21.15.86:0/2289973685 shutdown_connections 2024-04-21T08:46:26.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.671+0000 7f741f7f6740 1 -- 172.21.15.86:0/2289973685 wait complete. 2024-04-21T08:46:26.681 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.671+0000 7f741f7f6740 20 client.18804 trim_cache size 0 max 16384 2024-04-21T08:46:26.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.684+0000 7f741f7f6740 1 Processor -- start 2024-04-21T08:46:26.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.684+0000 7f741f7f6740 1 -- start start 2024-04-21T08:46:26.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.684+0000 7f741f7f6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643c6686330 0x5643c669fa40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.684+0000 7f741f7f6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643c6686ce0 0x5643c6695ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.684+0000 7f741f7f6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643c668fe40 0x5643c66961e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.684+0000 7f741f7f6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f74181ad0b0 con 0x5643c668fe40 2024-04-21T08:46:26.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.684+0000 7f741f7f6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f74181aebf0 con 0x5643c6686330 2024-04-21T08:46:26.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.684+0000 7f741f7f6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f74181ad780 con 0x5643c6686ce0 2024-04-21T08:46:26.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.684+0000 7f7416d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643c668fe40 0x5643c66961e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.684+0000 7f7416d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643c668fe40 0x5643c66961e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47680/0 (socket says 172.21.15.86:47680) 2024-04-21T08:46:26.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.684+0000 7f7416d9d640 1 -- 172.21.15.86:0/1800623252 learned_addr learned my addr 172.21.15.86:0/1800623252 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:26.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.684+0000 7f741659c640 1 --2- 172.21.15.86:0/1800623252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643c6686330 0x5643c669fa40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.685+0000 7f740ffff640 1 --2- 172.21.15.86:0/1800623252 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643c6686ce0 0x5643c6695ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.685+0000 7f7416d9d640 1 -- 172.21.15.86:0/1800623252 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643c6686ce0 msgr2=0x5643c6695ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.685+0000 7f7416d9d640 1 --2- 172.21.15.86:0/1800623252 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643c6686ce0 0x5643c6695ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.685+0000 7f7416d9d640 1 -- 172.21.15.86:0/1800623252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643c6686330 msgr2=0x5643c669fa40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.685+0000 7f7416d9d640 1 --2- 172.21.15.86:0/1800623252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643c6686330 0x5643c669fa40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.685+0000 7f7416d9d640 1 -- 172.21.15.86:0/1800623252 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643c668fb30 con 0x5643c668fe40 2024-04-21T08:46:26.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.685+0000 7f7416d9d640 1 --2- 172.21.15.86:0/1800623252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643c668fe40 0x5643c66961e0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f7410007bb0 tx=0x7f7410004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=974699f65b340b78 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.685+0000 7f73fffff640 1 -- 172.21.15.86:0/1800623252 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74100132b0 con 0x5643c668fe40 2024-04-21T08:46:26.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.685+0000 7f73fffff640 1 -- 172.21.15.86:0/1800623252 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7410020430 con 0x5643c668fe40 2024-04-21T08:46:26.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.685+0000 7f73fffff640 1 -- 172.21.15.86:0/1800623252 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f741001f690 con 0x5643c668fe40 2024-04-21T08:46:26.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.686+0000 7f741f7f6740 1 -- 172.21.15.86:0/1800623252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643c668fe40 msgr2=0x5643c66961e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.686+0000 7f741f7f6740 1 --2- 172.21.15.86:0/1800623252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643c668fe40 0x5643c66961e0 secure :-1 s=READY pgs=1540 cs=0 l=1 rev1=1 crypto rx=0x7f7410007bb0 tx=0x7f7410004ab0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.686+0000 7f741f7f6740 1 -- 172.21.15.86:0/1800623252 shutdown_connections 2024-04-21T08:46:26.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.686+0000 7f741f7f6740 1 --2- 172.21.15.86:0/1800623252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643c668fe40 0x5643c66961e0 unknown :-1 s=CLOSED pgs=1540 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.686+0000 7f741f7f6740 1 --2- 172.21.15.86:0/1800623252 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643c6686ce0 0x5643c6695ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.686+0000 7f741f7f6740 1 --2- 172.21.15.86:0/1800623252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643c6686330 0x5643c669fa40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.686+0000 7f741f7f6740 1 -- 172.21.15.86:0/1800623252 >> 172.21.15.86:0/1800623252 conn(0x5643c6680ba0 msgr2=0x5643c6691e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.686+0000 7f741f7f6740 1 -- 172.21.15.86:0/1800623252 shutdown_connections 2024-04-21T08:46:26.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.686+0000 7f741f7f6740 1 -- 172.21.15.86:0/1800623252 wait complete. 2024-04-21T08:46:26.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.686+0000 7f741f7f6740 1 -- start start 2024-04-21T08:46:26.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.686+0000 7f741f7f6740 1 Processor -- start 2024-04-21T08:46:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.686+0000 7f741f7f6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643c6686330 0x5643c669c340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.686+0000 7f741f7f6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643c6686ce0 0x5643c6694fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.686+0000 7f741f7f6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643c668fe40 0x5643c6695510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.686+0000 7f741f7f6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f74181ae520 con 0x5643c6686ce0 2024-04-21T08:46:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.686+0000 7f741f7f6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f74181ade50 con 0x5643c668fe40 2024-04-21T08:46:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.686+0000 7f741f7f6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f74181ab0b0 con 0x5643c6686330 2024-04-21T08:46:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.686+0000 7f741f7f6740 10 client.0 _ll_register_callbacks cb 0x7f74000be580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.686+0000 7f740ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643c6686ce0 0x5643c6694fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.686+0000 7f740ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643c6686ce0 0x5643c6694fd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47690/0 (socket says 172.21.15.86:47690) 2024-04-21T08:46:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.686+0000 7f740ffff640 1 -- 172.21.15.86:0/2030867943 learned_addr learned my addr 172.21.15.86:0/2030867943 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.687+0000 7f7416d9d640 1 --2- 172.21.15.86:0/2030867943 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643c668fe40 0x5643c6695510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.687+0000 7f740ffff640 1 -- 172.21.15.86:0/2030867943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643c6686330 msgr2=0x5643c669c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.687+0000 7f740ffff640 1 --2- 172.21.15.86:0/2030867943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643c6686330 0x5643c669c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.687+0000 7f740ffff640 1 -- 172.21.15.86:0/2030867943 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643c668fe40 msgr2=0x5643c6695510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:26.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.687+0000 7f740ffff640 1 --2- 172.21.15.86:0/2030867943 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643c668fe40 0x5643c6695510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.687+0000 7f740ffff640 1 -- 172.21.15.86:0/2030867943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643c6695cf0 con 0x5643c6686ce0 2024-04-21T08:46:26.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.687+0000 7f740ffff640 1 --2- 172.21.15.86:0/2030867943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643c6686ce0 0x5643c6694fd0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f74000bce70 tx=0x7f740002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb4f36daadfbc039 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:26.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.687+0000 7f73fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:46:26.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.687+0000 7f73fdffb640 1 -- 172.21.15.86:0/2030867943 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f74000f0620 con 0x5643c6686ce0 2024-04-21T08:46:26.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.687+0000 7f73fdffb640 1 -- 172.21.15.86:0/2030867943 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f74000b9730 con 0x5643c6686ce0 2024-04-21T08:46:26.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.687+0000 7f741f7f6740 1 -- 172.21.15.86:0/2030867943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7410020430 con 0x5643c6686ce0 2024-04-21T08:46:26.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.687+0000 7f73fdffb640 1 -- 172.21.15.86:0/2030867943 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f74000bae60 con 0x5643c6686ce0 2024-04-21T08:46:26.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.687+0000 7f741f7f6740 10 client.18810 fetch_fsmap learned FSMap version 29 2024-04-21T08:46:26.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.687+0000 7f741f7f6740 1 -- 172.21.15.86:0/2030867943 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5643c6695cf0 con 0x5643c6686ce0 2024-04-21T08:46:26.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.687+0000 7f73fdffb640 1 -- 172.21.15.86:0/2030867943 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7410020430 con 0x5643c6686ce0 2024-04-21T08:46:26.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.687+0000 7f73fdffb640 1 -- 172.21.15.86:0/2030867943 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f7400086070 con 0x5643c6686ce0 2024-04-21T08:46:26.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.687+0000 7f741f7f6740 10 client.18810 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:46:26.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.687+0000 7f741f7f6740 10 client.18810 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:26.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.687+0000 7f741f7f6740 20 client.18810 populate_metadata read hostname 'smithi086' 2024-04-21T08:46:26.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.687+0000 7f741f7f6740 1 --2- 172.21.15.86:0/2030867943 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5643c670a490 0x5643c670c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:26.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.687+0000 7f741f7f6740 4 client.18810 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:46:26.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.687+0000 7f741f7f6740 1 -- 172.21.15.86:0/2030867943 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5643c65f8c10 con 0x5643c670a490 2024-04-21T08:46:26.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.688+0000 7f741659c640 1 --2- 172.21.15.86:0/2030867943 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5643c670a490 0x5643c670c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:26.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.688+0000 7f73fdffb640 10 client.18810 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:26.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.688+0000 7f741659c640 1 --2- 172.21.15.86:0/2030867943 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5643c670a490 0x5643c670c8b0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=75abd31470b0425f server_cookie=26aa203cbbacf1f2 in_seq=0 out_seq=0 2024-04-21T08:46:26.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.688+0000 7f73fdffb640 1 -- 172.21.15.86:0/2030867943 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5643c65f8c10 con 0x5643c670a490 2024-04-21T08:46:26.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.688+0000 7f73fdffb640 10 client.18810 handle_command_reply: tid=0 2024-04-21T08:46:26.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.688+0000 7f741f7f6740 1 client.18810 shutdown 2024-04-21T08:46:26.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.689+0000 7f741f7f6740 1 -- 172.21.15.86:0/2030867943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643c6686ce0 msgr2=0x5643c6694fd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.689+0000 7f741f7f6740 1 --2- 172.21.15.86:0/2030867943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643c6686ce0 0x5643c6694fd0 secure :-1 s=READY pgs=1541 cs=0 l=1 rev1=1 crypto rx=0x7f74000bce70 tx=0x7f740002bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.689+0000 7f741f7f6740 1 -- 172.21.15.86:0/2030867943 shutdown_connections 2024-04-21T08:46:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.689+0000 7f741f7f6740 1 --2- 172.21.15.86:0/2030867943 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5643c670a490 0x5643c670c8b0 crc :-1 s=READY pgs=234 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.689+0000 7f741f7f6740 1 --2- 172.21.15.86:0/2030867943 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643c668fe40 0x5643c6695510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.689+0000 7f741f7f6740 1 --2- 172.21.15.86:0/2030867943 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643c6686ce0 0x5643c6694fd0 unknown :-1 s=CLOSED pgs=1541 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.689+0000 7f73fdffb640 0 client.18810 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.689+0000 7f741f7f6740 1 --2- 172.21.15.86:0/2030867943 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643c6686330 0x5643c669c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.689+0000 7f741f7f6740 1 -- 172.21.15.86:0/2030867943 >> 172.21.15.86:0/2030867943 conn(0x5643c6680ba0 msgr2=0x5643c667f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.689+0000 7f741f7f6740 1 -- 172.21.15.86:0/2030867943 shutdown_connections 2024-04-21T08:46:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.689+0000 7f741f7f6740 1 -- 172.21.15.86:0/2030867943 wait complete. 2024-04-21T08:46:26.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.689+0000 7f741f7f6740 20 client.18810 trim_cache size 0 max 16384 2024-04-21T08:46:26.697 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:46:26.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.689+0000 7f741cc06640 1 -- 172.21.15.86:0/4115269751 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f74040a3d20 msgr2=0x7f74040a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.689+0000 7f741cc06640 1 --2- 172.21.15.86:0/4115269751 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f74040a3d20 0x7f74040a61e0 secure :-1 s=READY pgs=248 cs=0 l=1 rev1=1 crypto rx=0x7f7410007fd0 tx=0x7f7410007d00 comp rx=0 tx=0).stop 2024-04-21T08:46:26.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.689+0000 7f741cc06640 1 -- 172.21.15.86:0/4115269751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74181072b0 msgr2=0x7f7418198cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:26.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.689+0000 7f741cc06640 1 --2- 172.21.15.86:0/4115269751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74181072b0 0x7f7418198cf0 secure :-1 s=READY pgs=1537 cs=0 l=1 rev1=1 crypto rx=0x7f7400005ec0 tx=0x7f740002ca10 comp rx=0 tx=0).stop 2024-04-21T08:46:26.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.689+0000 7f741cc06640 1 -- 172.21.15.86:0/4115269751 shutdown_connections 2024-04-21T08:46:26.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.689+0000 7f741cc06640 1 --2- 172.21.15.86:0/4115269751 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f74040a3d20 0x7f74040a61e0 unknown :-1 s=CLOSED pgs=248 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.689+0000 7f741cc06640 1 --2- 172.21.15.86:0/4115269751 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f741819d020 0x7f741819d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.689+0000 7f741cc06640 1 --2- 172.21.15.86:0/4115269751 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f74181072b0 0x7f7418198cf0 unknown :-1 s=CLOSED pgs=1537 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.689+0000 7f741cc06640 1 --2- 172.21.15.86:0/4115269751 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f74181021e0 0x7f74181987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:26.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.689+0000 7f741cc06640 1 -- 172.21.15.86:0/4115269751 >> 172.21.15.86:0/4115269751 conn(0x7f74180fda60 msgr2=0x7f74180fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:26.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.689+0000 7f741cc06640 1 -- 172.21.15.86:0/4115269751 shutdown_connections 2024-04-21T08:46:26.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:26.689+0000 7f741cc06640 1 -- 172.21.15.86:0/4115269751 wait complete. 2024-04-21T08:46:27.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:26 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3304419958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:46:27.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:26 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3304419958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:46:27.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:26 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3304419958' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:46:27.700 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:46:27.700 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:46:27.700 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:46:27.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.815+0000 7f57de77e640 1 -- 172.21.15.86:0/1424136432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57d8103af0 msgr2=0x7f57d810bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:27.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.815+0000 7f57de77e640 1 --2- 172.21.15.86:0/1424136432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57d8103af0 0x7f57d810bfd0 secure :-1 s=READY pgs=1542 cs=0 l=1 rev1=1 crypto rx=0x7f57cc009900 tx=0x7f57cc02bd60 comp rx=0 tx=0).stop 2024-04-21T08:46:27.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.815+0000 7f57de77e640 1 -- 172.21.15.86:0/1424136432 shutdown_connections 2024-04-21T08:46:27.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.815+0000 7f57de77e640 1 --2- 172.21.15.86:0/1424136432 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57d810c510 0x7f57d810e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.815+0000 7f57de77e640 1 --2- 172.21.15.86:0/1424136432 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57d8103af0 0x7f57d810bfd0 unknown :-1 s=CLOSED pgs=1542 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.815+0000 7f57de77e640 1 --2- 172.21.15.86:0/1424136432 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57d81031d0 0x7f57d81035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.815+0000 7f57de77e640 1 -- 172.21.15.86:0/1424136432 >> 172.21.15.86:0/1424136432 conn(0x7f57d8077f50 msgr2=0x7f57d8078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:27.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.815+0000 7f57de77e640 1 -- 172.21.15.86:0/1424136432 shutdown_connections 2024-04-21T08:46:27.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.815+0000 7f57de77e640 1 -- 172.21.15.86:0/1424136432 wait complete. 2024-04-21T08:46:27.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.815+0000 7f57de77e640 1 Processor -- start 2024-04-21T08:46:27.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.815+0000 7f57de77e640 1 -- start start 2024-04-21T08:46:27.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.816+0000 7f57de77e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57d81031d0 0x7f57d819e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:27.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.816+0000 7f57de77e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57d8103af0 0x7f57d819e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:27.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.816+0000 7f57de77e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57d810c510 0x7f57d81985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:27.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.816+0000 7f57de77e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57d81121f0 con 0x7f57d810c510 2024-04-21T08:46:27.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.816+0000 7f57de77e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f57d8112070 con 0x7f57d8103af0 2024-04-21T08:46:27.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.816+0000 7f57de77e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f57d8112370 con 0x7f57d81031d0 2024-04-21T08:46:27.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.816+0000 7f57dcd1b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57d810c510 0x7f57d81985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:27.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.816+0000 7f57dcd1b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57d810c510 0x7f57d81985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47704/0 (socket says 172.21.15.86:47704) 2024-04-21T08:46:27.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.816+0000 7f57dcd1b640 1 -- 172.21.15.86:0/37394351 learned_addr learned my addr 172.21.15.86:0/37394351 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:27.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.816+0000 7f57d7fff640 1 --2- 172.21.15.86:0/37394351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57d81031d0 0x7f57d819e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:27.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.816+0000 7f57d77fe640 1 --2- 172.21.15.86:0/37394351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57d8103af0 0x7f57d819e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:27.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.816+0000 7f57dcd1b640 1 -- 172.21.15.86:0/37394351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57d81031d0 msgr2=0x7f57d819e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:27.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.816+0000 7f57dcd1b640 1 --2- 172.21.15.86:0/37394351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57d81031d0 0x7f57d819e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.816+0000 7f57dcd1b640 1 -- 172.21.15.86:0/37394351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57d8103af0 msgr2=0x7f57d819e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:27.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.816+0000 7f57dcd1b640 1 --2- 172.21.15.86:0/37394351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57d8103af0 0x7f57d819e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.816+0000 7f57dcd1b640 1 -- 172.21.15.86:0/37394351 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f57d8198d50 con 0x7f57d810c510 2024-04-21T08:46:27.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.816+0000 7f57dcd1b640 1 --2- 172.21.15.86:0/37394351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57d810c510 0x7f57d81985d0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f57d000cad0 tx=0x7f57d000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:27.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.816+0000 7f57d57fa640 1 -- 172.21.15.86:0/37394351 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d000ae20 con 0x7f57d810c510 2024-04-21T08:46:27.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.817+0000 7f57de77e640 1 -- 172.21.15.86:0/37394351 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f57d8199040 con 0x7f57d810c510 2024-04-21T08:46:27.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.817+0000 7f57d57fa640 1 -- 172.21.15.86:0/37394351 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57d0005020 con 0x7f57d810c510 2024-04-21T08:46:27.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.817+0000 7f57d57fa640 1 -- 172.21.15.86:0/37394351 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57d000f6b0 con 0x7f57d810c510 2024-04-21T08:46:27.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.817+0000 7f57de77e640 1 -- 172.21.15.86:0/37394351 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f57d81a52e0 con 0x7f57d810c510 2024-04-21T08:46:27.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.819+0000 7f57d57fa640 1 -- 172.21.15.86:0/37394351 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f57d00051d0 con 0x7f57d810c510 2024-04-21T08:46:27.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.819+0000 7f57e136f740 1 Processor -- start 2024-04-21T08:46:27.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.819+0000 7f57e136f740 1 -- start start 2024-04-21T08:46:27.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.819+0000 7f57e136f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a7439d48e0 0x55a7439d4cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:27.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.819+0000 7f57e136f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a7439d5290 0x55a7439dd840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:27.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.819+0000 7f57e136f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7439dde10 0x55a7439e0260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:27.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.819+0000 7f57e136f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a743938f00 con 0x55a7439dde10 2024-04-21T08:46:27.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.819+0000 7f57e136f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55a7438d6590 con 0x55a7439d5290 2024-04-21T08:46:27.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.819+0000 7f57e136f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a743934ec0 con 0x55a7439d48e0 2024-04-21T08:46:27.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.820+0000 7f57d57fa640 1 --2- 172.21.15.86:0/37394351 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f57c00a3da0 0x7f57c00a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:27.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.820+0000 7f57d77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7439dde10 0x55a7439e0260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:27.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.820+0000 7f57d77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7439dde10 0x55a7439e0260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47710/0 (socket says 172.21.15.86:47710) 2024-04-21T08:46:27.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.820+0000 7f57d77fe640 1 -- 172.21.15.86:0/568640965 learned_addr learned my addr 172.21.15.86:0/568640965 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:27.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.820+0000 7f57dcd1b640 1 --2- 172.21.15.86:0/37394351 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f57c00a3da0 0x7f57c00a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:27.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.820+0000 7f57d7fff640 1 --2- 172.21.15.86:0/568640965 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a7439d5290 0x55a7439dd840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:27.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.820+0000 7f57d77fe640 1 --2- 172.21.15.86:0/568640965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a7439d48e0 0x55a7439d4cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:27.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.820+0000 7f57d57fa640 1 -- 172.21.15.86:0/37394351 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f57d00d0be0 con 0x7f57d810c510 2024-04-21T08:46:27.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.820+0000 7f57d77fe640 1 -- 172.21.15.86:0/568640965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a7439d48e0 msgr2=0x55a7439d4cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:27.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.820+0000 7f57d77fe640 1 --2- 172.21.15.86:0/568640965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a7439d48e0 0x55a7439d4cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.820+0000 7f57d77fe640 1 -- 172.21.15.86:0/568640965 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a7439d5290 msgr2=0x55a7439dd840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:27.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.820+0000 7f57d77fe640 1 --2- 172.21.15.86:0/568640965 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a7439d5290 0x55a7439dd840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.820+0000 7f57d77fe640 1 -- 172.21.15.86:0/568640965 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7437137e0 con 0x55a7439dde10 2024-04-21T08:46:27.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.820+0000 7f57d77fe640 1 --2- 172.21.15.86:0/568640965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7439dde10 0x55a7439e0260 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f57cc009230 tx=0x7f57cc005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa697a20bbfeaae2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:27.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.820+0000 7f57dcd1b640 1 --2- 172.21.15.86:0/37394351 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f57c00a3da0 0x7f57c00a6260 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f57d000a6e0 tx=0x7f57d0087040 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:27.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.820+0000 7f57be7fc640 1 -- 172.21.15.86:0/568640965 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc02ca80 con 0x55a7439dde10 2024-04-21T08:46:27.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.820+0000 7f57be7fc640 1 -- 172.21.15.86:0/568640965 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57cc005500 con 0x55a7439dde10 2024-04-21T08:46:27.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.820+0000 7f57be7fc640 1 -- 172.21.15.86:0/568640965 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc004b20 con 0x55a7439dde10 2024-04-21T08:46:27.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.821+0000 7f57e136f740 1 -- 172.21.15.86:0/568640965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7439dde10 msgr2=0x55a7439e0260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:27.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.821+0000 7f57e136f740 1 --2- 172.21.15.86:0/568640965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7439dde10 0x55a7439e0260 secure :-1 s=READY pgs=1544 cs=0 l=1 rev1=1 crypto rx=0x7f57cc009230 tx=0x7f57cc005c00 comp rx=0 tx=0).stop 2024-04-21T08:46:27.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.821+0000 7f57e136f740 1 -- 172.21.15.86:0/568640965 shutdown_connections 2024-04-21T08:46:27.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.821+0000 7f57e136f740 1 --2- 172.21.15.86:0/568640965 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7439dde10 0x55a7439e0260 unknown :-1 s=CLOSED pgs=1544 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.821+0000 7f57e136f740 1 --2- 172.21.15.86:0/568640965 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a7439d5290 0x55a7439dd840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.821+0000 7f57e136f740 1 --2- 172.21.15.86:0/568640965 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a7439d48e0 0x55a7439d4cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.821+0000 7f57e136f740 1 -- 172.21.15.86:0/568640965 >> 172.21.15.86:0/568640965 conn(0x55a7439cefb0 msgr2=0x55a7439cf390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:27.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.821+0000 7f57e136f740 1 -- 172.21.15.86:0/568640965 shutdown_connections 2024-04-21T08:46:27.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.821+0000 7f57e136f740 1 -- 172.21.15.86:0/568640965 wait complete. 2024-04-21T08:46:27.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.821+0000 7f57e136f740 1 -- start start 2024-04-21T08:46:27.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.822+0000 7f57e136f740 1 Processor -- start 2024-04-21T08:46:27.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.822+0000 7f57e136f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a7439d48e0 0x55a7439dadb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:27.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.822+0000 7f57e136f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7439d5290 0x55a7439db2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:27.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.822+0000 7f57e136f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a7439dde10 0x55a7439db830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:27.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.822+0000 7f57e136f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a7439e0cc0 con 0x55a7439d5290 2024-04-21T08:46:27.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.822+0000 7f57e136f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55a7438f2580 con 0x55a7439dde10 2024-04-21T08:46:27.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.822+0000 7f57e136f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a7439e0e40 con 0x55a7439d48e0 2024-04-21T08:46:27.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.822+0000 7f57e136f740 10 client.0 _ll_register_callbacks cb 0x55a7438f8650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:27.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.822+0000 7f57d7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7439d5290 0x55a7439db2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:27.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.822+0000 7f57d7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7439d5290 0x55a7439db2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47726/0 (socket says 172.21.15.86:47726) 2024-04-21T08:46:27.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.822+0000 7f57d7fff640 1 -- 172.21.15.86:0/2803118249 learned_addr learned my addr 172.21.15.86:0/2803118249 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:27.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.822+0000 7f57d77fe640 1 --2- 172.21.15.86:0/2803118249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a7439d48e0 0x55a7439dadb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:27.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.822+0000 7f57d77fe640 1 --2- 172.21.15.86:0/2803118249 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a7439dde10 0x55a7439db830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:27.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.822+0000 7f57d7fff640 1 -- 172.21.15.86:0/2803118249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a7439d48e0 msgr2=0x55a7439dadb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:27.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.822+0000 7f57d7fff640 1 --2- 172.21.15.86:0/2803118249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a7439d48e0 0x55a7439dadb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.822+0000 7f57d7fff640 1 -- 172.21.15.86:0/2803118249 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a7439dde10 msgr2=0x55a7439db830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:27.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.822+0000 7f57d7fff640 1 --2- 172.21.15.86:0/2803118249 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a7439dde10 0x55a7439db830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.822+0000 7f57d7fff640 1 -- 172.21.15.86:0/2803118249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a743773f70 con 0x55a7439d5290 2024-04-21T08:46:27.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.822+0000 7f57d77fe640 1 --2- 172.21.15.86:0/2803118249 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a7439dde10 0x55a7439db830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:27.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.822+0000 7f57d7fff640 1 --2- 172.21.15.86:0/2803118249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7439d5290 0x55a7439db2f0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f57c8010850 tx=0x7f57c8010d10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8206e1bb1bb237 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:27.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.822+0000 7f57b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:46:27.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.823+0000 7f57b77fe640 1 -- 172.21.15.86:0/2803118249 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57c800fb10 con 0x55a7439d5290 2024-04-21T08:46:27.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.823+0000 7f57b77fe640 1 -- 172.21.15.86:0/2803118249 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f57c800fcb0 con 0x55a7439d5290 2024-04-21T08:46:27.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.823+0000 7f57e136f740 1 -- 172.21.15.86:0/2803118249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57cc005500 con 0x55a7439d5290 2024-04-21T08:46:27.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.823+0000 7f57b77fe640 1 -- 172.21.15.86:0/2803118249 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f57c8004020 con 0x55a7439d5290 2024-04-21T08:46:27.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.823+0000 7f57e136f740 10 client.18822 fetch_fsmap learned FSMap version 29 2024-04-21T08:46:27.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.823+0000 7f57e136f740 1 -- 172.21.15.86:0/2803118249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a74360b170 con 0x55a7439d5290 2024-04-21T08:46:27.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.823+0000 7f57b77fe640 1 -- 172.21.15.86:0/2803118249 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57cc005500 con 0x55a7439d5290 2024-04-21T08:46:27.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.823+0000 7f57b77fe640 1 -- 172.21.15.86:0/2803118249 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f57c80048c0 con 0x55a7439d5290 2024-04-21T08:46:27.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.823+0000 7f57e136f740 10 client.18822 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:46:27.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.823+0000 7f57e136f740 10 client.18822 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:27.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.823+0000 7f57e136f740 20 client.18822 populate_metadata read hostname 'smithi086' 2024-04-21T08:46:27.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.823+0000 7f57e136f740 1 --2- 172.21.15.86:0/2803118249 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a7439ee450 0x55a7439f0890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:27.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.823+0000 7f57e136f740 4 client.18822 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:27.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.823+0000 7f57e136f740 1 -- 172.21.15.86:0/2803118249 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a743946920 con 0x55a7439ee450 2024-04-21T08:46:27.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.823+0000 7f57d77fe640 1 --2- 172.21.15.86:0/2803118249 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a7439ee450 0x55a7439f0890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:27.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.824+0000 7f57b77fe640 10 client.18822 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:27.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.824+0000 7f57d77fe640 1 --2- 172.21.15.86:0/2803118249 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a7439ee450 0x55a7439f0890 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1254337762ad5b7f server_cookie=d0c39bf8715fee4 in_seq=0 out_seq=0 2024-04-21T08:46:27.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.825+0000 7f57b77fe640 1 -- 172.21.15.86:0/2803118249 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a743946920 con 0x55a7439ee450 2024-04-21T08:46:27.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.825+0000 7f57b77fe640 10 client.18822 handle_command_reply: tid=0 2024-04-21T08:46:27.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.825+0000 7f57e136f740 1 client.18822 shutdown 2024-04-21T08:46:27.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.825+0000 7f57e136f740 1 -- 172.21.15.86:0/2803118249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7439d5290 msgr2=0x55a7439db2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:27.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.825+0000 7f57e136f740 1 --2- 172.21.15.86:0/2803118249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7439d5290 0x55a7439db2f0 secure :-1 s=READY pgs=1545 cs=0 l=1 rev1=1 crypto rx=0x7f57c8010850 tx=0x7f57c8010d10 comp rx=0 tx=0).stop 2024-04-21T08:46:27.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.825+0000 7f57e136f740 1 -- 172.21.15.86:0/2803118249 shutdown_connections 2024-04-21T08:46:27.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.825+0000 7f57e136f740 1 --2- 172.21.15.86:0/2803118249 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a7439ee450 0x55a7439f0890 crc :-1 s=READY pgs=235 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.825+0000 7f57e136f740 1 --2- 172.21.15.86:0/2803118249 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a7439dde10 0x55a7439db830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.825+0000 7f57e136f740 1 --2- 172.21.15.86:0/2803118249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7439d5290 0x55a7439db2f0 unknown :-1 s=CLOSED pgs=1545 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.825+0000 7f57b77fe640 0 client.18822 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:27.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.825+0000 7f57e136f740 1 --2- 172.21.15.86:0/2803118249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a7439d48e0 0x55a7439dadb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.825+0000 7f57e136f740 1 -- 172.21.15.86:0/2803118249 >> 172.21.15.86:0/2803118249 conn(0x55a7439cefb0 msgr2=0x55a7439d0d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:27.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.825+0000 7f57e136f740 1 -- 172.21.15.86:0/2803118249 shutdown_connections 2024-04-21T08:46:27.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.825+0000 7f57e136f740 1 -- 172.21.15.86:0/2803118249 wait complete. 2024-04-21T08:46:27.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.825+0000 7f57e136f740 20 client.18822 trim_cache size 0 max 16384 2024-04-21T08:46:27.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.842+0000 7f57e136f740 1 Processor -- start 2024-04-21T08:46:27.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.842+0000 7f57e136f740 1 -- start start 2024-04-21T08:46:27.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.842+0000 7f57e136f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7439d48e0 0x55a7439dd4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:27.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.842+0000 7f57e136f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a7439d5290 0x55a7439e9be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:27.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.842+0000 7f57e136f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a7439dde10 0x55a7439f61a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:27.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.842+0000 7f57e136f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57d81af320 con 0x55a7439d48e0 2024-04-21T08:46:27.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.842+0000 7f57e136f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f57d81b0e60 con 0x55a7439d5290 2024-04-21T08:46:27.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.842+0000 7f57e136f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f57d81af9f0 con 0x55a7439dde10 2024-04-21T08:46:27.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.842+0000 7f57d77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7439d48e0 0x55a7439dd4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:27.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.842+0000 7f57d77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7439d48e0 0x55a7439dd4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47738/0 (socket says 172.21.15.86:47738) 2024-04-21T08:46:27.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.842+0000 7f57d77fe640 1 -- 172.21.15.86:0/2904133508 learned_addr learned my addr 172.21.15.86:0/2904133508 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:27.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.843+0000 7f57d77fe640 1 -- 172.21.15.86:0/2904133508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a7439dde10 msgr2=0x55a7439f61a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:27.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.843+0000 7f57d77fe640 1 --2- 172.21.15.86:0/2904133508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a7439dde10 0x55a7439f61a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.843+0000 7f57d77fe640 1 -- 172.21.15.86:0/2904133508 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a7439d5290 msgr2=0x55a7439e9be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:27.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.843+0000 7f57d77fe640 1 --2- 172.21.15.86:0/2904133508 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a7439d5290 0x55a7439e9be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.843+0000 7f57d77fe640 1 -- 172.21.15.86:0/2904133508 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a7439d9300 con 0x55a7439d48e0 2024-04-21T08:46:27.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.843+0000 7f57d77fe640 1 --2- 172.21.15.86:0/2904133508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7439d48e0 0x55a7439dd4e0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f57cc000c00 tx=0x7f57cc005c00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1e6fcc4801da45af server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:27.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.843+0000 7f57bdffb640 1 -- 172.21.15.86:0/2904133508 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc02cd80 con 0x55a7439d48e0 2024-04-21T08:46:27.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.843+0000 7f57bdffb640 1 -- 172.21.15.86:0/2904133508 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57cc003aa0 con 0x55a7439d48e0 2024-04-21T08:46:27.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.843+0000 7f57bdffb640 1 -- 172.21.15.86:0/2904133508 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc051440 con 0x55a7439d48e0 2024-04-21T08:46:27.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.844+0000 7f57e136f740 1 -- 172.21.15.86:0/2904133508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7439d48e0 msgr2=0x55a7439dd4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:27.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.844+0000 7f57e136f740 1 --2- 172.21.15.86:0/2904133508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7439d48e0 0x55a7439dd4e0 secure :-1 s=READY pgs=1546 cs=0 l=1 rev1=1 crypto rx=0x7f57cc000c00 tx=0x7f57cc005c00 comp rx=0 tx=0).stop 2024-04-21T08:46:27.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.844+0000 7f57e136f740 1 -- 172.21.15.86:0/2904133508 shutdown_connections 2024-04-21T08:46:27.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.844+0000 7f57e136f740 1 --2- 172.21.15.86:0/2904133508 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a7439dde10 0x55a7439f61a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.844+0000 7f57e136f740 1 --2- 172.21.15.86:0/2904133508 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a7439d5290 0x55a7439e9be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.844+0000 7f57e136f740 1 --2- 172.21.15.86:0/2904133508 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7439d48e0 0x55a7439dd4e0 unknown :-1 s=CLOSED pgs=1546 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.844+0000 7f57e136f740 1 -- 172.21.15.86:0/2904133508 >> 172.21.15.86:0/2904133508 conn(0x55a7439cefb0 msgr2=0x55a7439d9680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:27.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.844+0000 7f57e136f740 1 -- 172.21.15.86:0/2904133508 shutdown_connections 2024-04-21T08:46:27.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.844+0000 7f57e136f740 1 -- 172.21.15.86:0/2904133508 wait complete. 2024-04-21T08:46:27.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.844+0000 7f57e136f740 1 -- start start 2024-04-21T08:46:27.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.844+0000 7f57e136f740 1 Processor -- start 2024-04-21T08:46:27.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.844+0000 7f57e136f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7439d48e0 0x55a7439e1ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:27.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.844+0000 7f57e136f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a7439d5290 0x55a7439e20e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:27.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.844+0000 7f57e136f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a7439dde10 0x55a7439e2620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:27.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.844+0000 7f57e136f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f57d81b0790 con 0x55a7439d48e0 2024-04-21T08:46:27.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.844+0000 7f57e136f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f57d81b00c0 con 0x55a7439dde10 2024-04-21T08:46:27.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.844+0000 7f57e136f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f57d81ad320 con 0x55a7439d5290 2024-04-21T08:46:27.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.844+0000 7f57e136f740 10 client.0 _ll_register_callbacks cb 0x7f57c800a6c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:27.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.845+0000 7f57d77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7439d48e0 0x55a7439e1ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:27.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.845+0000 7f57d77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7439d48e0 0x55a7439e1ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47754/0 (socket says 172.21.15.86:47754) 2024-04-21T08:46:27.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.845+0000 7f57d77fe640 1 -- 172.21.15.86:0/3646585221 learned_addr learned my addr 172.21.15.86:0/3646585221 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:27.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.845+0000 7f57d7fff640 1 --2- 172.21.15.86:0/3646585221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a7439d5290 0x55a7439e20e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:27.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.845+0000 7f57d77fe640 1 --2- 172.21.15.86:0/3646585221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a7439dde10 0x55a7439e2620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:27.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.845+0000 7f57d77fe640 1 -- 172.21.15.86:0/3646585221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a7439d5290 msgr2=0x55a7439e20e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:27.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.845+0000 7f57d77fe640 1 --2- 172.21.15.86:0/3646585221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a7439d5290 0x55a7439e20e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.845+0000 7f57d77fe640 1 -- 172.21.15.86:0/3646585221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a7439dde10 msgr2=0x55a7439e2620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:27.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.845+0000 7f57d77fe640 1 --2- 172.21.15.86:0/3646585221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a7439dde10 0x55a7439e2620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.845+0000 7f57d77fe640 1 -- 172.21.15.86:0/3646585221 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a743a53ac0 con 0x55a7439d48e0 2024-04-21T08:46:27.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.845+0000 7f57d77fe640 1 --2- 172.21.15.86:0/3646585221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7439d48e0 0x55a7439e1ba0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f57cc005bd0 tx=0x7f57cc045040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39aa6b95e05199cf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:27.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.845+0000 7f57b77fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:46:27.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.848+0000 7f57b77fe640 1 -- 172.21.15.86:0/3646585221 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f57cc059060 con 0x55a7439d48e0 2024-04-21T08:46:27.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.848+0000 7f57b77fe640 1 -- 172.21.15.86:0/3646585221 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f57cc003e50 con 0x55a7439d48e0 2024-04-21T08:46:27.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.848+0000 7f57e136f740 1 -- 172.21.15.86:0/3646585221 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f57cc003aa0 con 0x55a7439d48e0 2024-04-21T08:46:27.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.848+0000 7f57b77fe640 1 -- 172.21.15.86:0/3646585221 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f57cc051120 con 0x55a7439d48e0 2024-04-21T08:46:27.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.848+0000 7f57e136f740 10 client.18828 fetch_fsmap learned FSMap version 29 2024-04-21T08:46:27.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.848+0000 7f57e136f740 1 -- 172.21.15.86:0/3646585221 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a743a54dc0 con 0x55a7439d48e0 2024-04-21T08:46:27.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.848+0000 7f57b77fe640 1 -- 172.21.15.86:0/3646585221 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f57cc003aa0 con 0x55a7439d48e0 2024-04-21T08:46:27.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.848+0000 7f57b77fe640 1 -- 172.21.15.86:0/3646585221 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f57cc0415f0 con 0x55a7439d48e0 2024-04-21T08:46:27.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.848+0000 7f57e136f740 10 client.18828 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:46:27.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.848+0000 7f57e136f740 10 client.18828 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:27.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.848+0000 7f57e136f740 20 client.18828 populate_metadata read hostname 'smithi086' 2024-04-21T08:46:27.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.848+0000 7f57e136f740 1 --2- 172.21.15.86:0/3646585221 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a743a54ff0 0x55a743a573e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:27.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.848+0000 7f57e136f740 4 client.18828 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:46:27.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.848+0000 7f57e136f740 1 -- 172.21.15.86:0/3646585221 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55a743940790 con 0x55a743a54ff0 2024-04-21T08:46:27.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.849+0000 7f57d7fff640 1 --2- 172.21.15.86:0/3646585221 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a743a54ff0 0x55a743a573e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:27.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.849+0000 7f57b77fe640 10 client.18828 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:27.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.849+0000 7f57d7fff640 1 --2- 172.21.15.86:0/3646585221 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a743a54ff0 0x55a743a573e0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3eb0cee499394c7 server_cookie=5845d82641cd8fc6 in_seq=0 out_seq=0 2024-04-21T08:46:27.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.850+0000 7f57b77fe640 1 -- 172.21.15.86:0/3646585221 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55a743940790 con 0x55a743a54ff0 2024-04-21T08:46:27.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.850+0000 7f57b77fe640 10 client.18828 handle_command_reply: tid=0 2024-04-21T08:46:27.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.850+0000 7f57e136f740 1 client.18828 shutdown 2024-04-21T08:46:27.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.851+0000 7f57e136f740 1 -- 172.21.15.86:0/3646585221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7439d48e0 msgr2=0x55a7439e1ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:27.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.851+0000 7f57e136f740 1 --2- 172.21.15.86:0/3646585221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7439d48e0 0x55a7439e1ba0 secure :-1 s=READY pgs=1547 cs=0 l=1 rev1=1 crypto rx=0x7f57cc005bd0 tx=0x7f57cc045040 comp rx=0 tx=0).stop 2024-04-21T08:46:27.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.851+0000 7f57e136f740 1 -- 172.21.15.86:0/3646585221 shutdown_connections 2024-04-21T08:46:27.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.851+0000 7f57e136f740 1 --2- 172.21.15.86:0/3646585221 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a743a54ff0 0x55a743a573e0 crc :-1 s=READY pgs=236 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.851+0000 7f57e136f740 1 --2- 172.21.15.86:0/3646585221 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a7439dde10 0x55a7439e2620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.851+0000 7f57e136f740 1 --2- 172.21.15.86:0/3646585221 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a7439d5290 0x55a7439e20e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.851+0000 7f57e136f740 1 --2- 172.21.15.86:0/3646585221 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a7439d48e0 0x55a7439e1ba0 unknown :-1 s=CLOSED pgs=1547 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.851+0000 7f57e136f740 1 -- 172.21.15.86:0/3646585221 >> 172.21.15.86:0/3646585221 conn(0x55a7439cefb0 msgr2=0x55a7439dad50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:27.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.851+0000 7f57b77fe640 0 client.18828 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:27.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.851+0000 7f57e136f740 1 -- 172.21.15.86:0/3646585221 shutdown_connections 2024-04-21T08:46:27.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.851+0000 7f57e136f740 1 -- 172.21.15.86:0/3646585221 wait complete. 2024-04-21T08:46:27.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.851+0000 7f57e136f740 20 client.18828 trim_cache size 0 max 16384 2024-04-21T08:46:27.861 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:46:27.861 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-21T08:46:27.861 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "cfc13b84-b562-4019-92e6-ac86f19f6a7c", 2024-04-21T08:46:27.861 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-21T08:46:27.861 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:46:27.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.851+0000 7f57de77e640 1 -- 172.21.15.86:0/37394351 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f57c00a3da0 msgr2=0x7f57c00a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:27.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.851+0000 7f57de77e640 1 --2- 172.21.15.86:0/37394351 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f57c00a3da0 0x7f57c00a6260 secure :-1 s=READY pgs=249 cs=0 l=1 rev1=1 crypto rx=0x7f57d000a6e0 tx=0x7f57d0087040 comp rx=0 tx=0).stop 2024-04-21T08:46:27.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.851+0000 7f57de77e640 1 -- 172.21.15.86:0/37394351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57d810c510 msgr2=0x7f57d81985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:27.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.851+0000 7f57de77e640 1 --2- 172.21.15.86:0/37394351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57d810c510 0x7f57d81985d0 secure :-1 s=READY pgs=1543 cs=0 l=1 rev1=1 crypto rx=0x7f57d000cad0 tx=0x7f57d000cf90 comp rx=0 tx=0).stop 2024-04-21T08:46:27.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.851+0000 7f57de77e640 1 -- 172.21.15.86:0/37394351 shutdown_connections 2024-04-21T08:46:27.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.851+0000 7f57de77e640 1 --2- 172.21.15.86:0/37394351 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f57c00a3da0 0x7f57c00a6260 unknown :-1 s=CLOSED pgs=249 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.851+0000 7f57de77e640 1 --2- 172.21.15.86:0/37394351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f57d810c510 0x7f57d81985d0 unknown :-1 s=CLOSED pgs=1543 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.851+0000 7f57de77e640 1 --2- 172.21.15.86:0/37394351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f57d8103af0 0x7f57d819e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.851+0000 7f57de77e640 1 --2- 172.21.15.86:0/37394351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f57d81031d0 0x7f57d819e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.851+0000 7f57de77e640 1 -- 172.21.15.86:0/37394351 >> 172.21.15.86:0/37394351 conn(0x7f57d8077f50 msgr2=0x7f57d80fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:27.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.852+0000 7f57de77e640 1 -- 172.21.15.86:0/37394351 shutdown_connections 2024-04-21T08:46:27.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.852+0000 7f57de77e640 1 -- 172.21.15.86:0/37394351 wait complete. 2024-04-21T08:46:27.864 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:46:27.864 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:46:27.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.964+0000 7fad15b7e640 1 -- 172.21.15.86:0/3769083617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad1006a5b0 msgr2=0x7fad1006aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:27.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.964+0000 7fad15b7e640 1 --2- 172.21.15.86:0/3769083617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad1006a5b0 0x7fad1006aa30 secure :-1 s=READY pgs=1548 cs=0 l=1 rev1=1 crypto rx=0x7fad00009930 tx=0x7fad0002bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:27.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.964+0000 7fad15b7e640 1 -- 172.21.15.86:0/3769083617 shutdown_connections 2024-04-21T08:46:27.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.964+0000 7fad15b7e640 1 --2- 172.21.15.86:0/3769083617 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad1006af70 0x7fad1010ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.964+0000 7fad15b7e640 1 --2- 172.21.15.86:0/3769083617 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad1006a5b0 0x7fad1006aa30 unknown :-1 s=CLOSED pgs=1548 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.964+0000 7fad15b7e640 1 --2- 172.21.15.86:0/3769083617 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad10108060 0x7fad10108440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.964+0000 7fad15b7e640 1 -- 172.21.15.86:0/3769083617 >> 172.21.15.86:0/3769083617 conn(0x7fad10077c40 msgr2=0x7fad10078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:27.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.964+0000 7fad15b7e640 1 -- 172.21.15.86:0/3769083617 shutdown_connections 2024-04-21T08:46:27.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.964+0000 7fad15b7e640 1 -- 172.21.15.86:0/3769083617 wait complete. 2024-04-21T08:46:27.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.964+0000 7fad15b7e640 1 Processor -- start 2024-04-21T08:46:27.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.965+0000 7fad15b7e640 1 -- start start 2024-04-21T08:46:27.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.965+0000 7fad15b7e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad1006a5b0 0x7fad10198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:27.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.965+0000 7fad15b7e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad1006af70 0x7fad10198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:27.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.965+0000 7fad15b7e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad10108060 0x7fad1019cf70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:27.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.965+0000 7fad15b7e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fad101121a0 con 0x7fad10108060 2024-04-21T08:46:27.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.965+0000 7fad15b7e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fad10112020 con 0x7fad1006a5b0 2024-04-21T08:46:27.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.965+0000 7fad15b7e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fad10112320 con 0x7fad1006af70 2024-04-21T08:46:27.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.965+0000 7fad0ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad10108060 0x7fad1019cf70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:27.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.965+0000 7fad0ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad10108060 0x7fad1019cf70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47764/0 (socket says 172.21.15.86:47764) 2024-04-21T08:46:27.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.965+0000 7fad0ffff640 1 -- 172.21.15.86:0/3679973214 learned_addr learned my addr 172.21.15.86:0/3679973214 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:27.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.965+0000 7fad0effd640 1 --2- 172.21.15.86:0/3679973214 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad1006af70 0x7fad10198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:27.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.965+0000 7fad0f7fe640 1 --2- 172.21.15.86:0/3679973214 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad1006a5b0 0x7fad10198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:27.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.965+0000 7fad0ffff640 1 -- 172.21.15.86:0/3679973214 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad1006af70 msgr2=0x7fad10198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:27.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.965+0000 7fad0ffff640 1 --2- 172.21.15.86:0/3679973214 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad1006af70 0x7fad10198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.965+0000 7fad0ffff640 1 -- 172.21.15.86:0/3679973214 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad1006a5b0 msgr2=0x7fad10198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:27.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.965+0000 7fad0ffff640 1 --2- 172.21.15.86:0/3679973214 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad1006a5b0 0x7fad10198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.965+0000 7fad0ffff640 1 -- 172.21.15.86:0/3679973214 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fad1019d6a0 con 0x7fad10108060 2024-04-21T08:46:27.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.965+0000 7fad0ffff640 1 --2- 172.21.15.86:0/3679973214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad10108060 0x7fad1019cf70 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fad0800eaa0 tx=0x7fad0800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:27.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.966+0000 7fad0cff9640 1 -- 172.21.15.86:0/3679973214 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0800cdb0 con 0x7fad10108060 2024-04-21T08:46:27.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.966+0000 7fad15b7e640 1 -- 172.21.15.86:0/3679973214 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fad1019d990 con 0x7fad10108060 2024-04-21T08:46:27.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.966+0000 7fad0cff9640 1 -- 172.21.15.86:0/3679973214 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad08005020 con 0x7fad10108060 2024-04-21T08:46:27.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.966+0000 7fad0cff9640 1 -- 172.21.15.86:0/3679973214 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad080116d0 con 0x7fad10108060 2024-04-21T08:46:27.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.966+0000 7fad15b7e640 1 -- 172.21.15.86:0/3679973214 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fad101a5220 con 0x7fad10108060 2024-04-21T08:46:27.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.969+0000 7fad0cff9640 1 -- 172.21.15.86:0/3679973214 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fad080054a0 con 0x7fad10108060 2024-04-21T08:46:27.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.969+0000 7fad1876f740 1 Processor -- start 2024-04-21T08:46:27.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.969+0000 7fad1876f740 1 -- start start 2024-04-21T08:46:27.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.969+0000 7fad1876f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c37c6230 0x5574c37c6610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:27.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.969+0000 7fad1876f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5574c37c6b50 0x5574c37d3950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:27.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.969+0000 7fad1876f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574c37d3e90 0x5574c37d6280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:27.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.969+0000 7fad1876f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5574c372b1e0 con 0x5574c37c6230 2024-04-21T08:46:27.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.969+0000 7fad1876f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5574c36c8390 con 0x5574c37c6b50 2024-04-21T08:46:27.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.969+0000 7fad1876f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5574c37271a0 con 0x5574c37d3e90 2024-04-21T08:46:27.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.969+0000 7fad0effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c37c6230 0x5574c37c6610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:27.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.969+0000 7fad0effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c37c6230 0x5574c37c6610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47776/0 (socket says 172.21.15.86:47776) 2024-04-21T08:46:27.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.969+0000 7fad0effd640 1 -- 172.21.15.86:0/4239964882 learned_addr learned my addr 172.21.15.86:0/4239964882 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:27.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.969+0000 7fad0f7fe640 1 --2- 172.21.15.86:0/4239964882 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5574c37c6b50 0x5574c37d3950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:27.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.969+0000 7fad0cff9640 1 --2- 172.21.15.86:0/3679973214 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7facf80a3d20 0x7facf80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:27.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.969+0000 7fad0effd640 1 -- 172.21.15.86:0/4239964882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574c37d3e90 msgr2=0x5574c37d6280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:27.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.969+0000 7fad0effd640 1 --2- 172.21.15.86:0/4239964882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574c37d3e90 0x5574c37d6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.969+0000 7fad0effd640 1 -- 172.21.15.86:0/4239964882 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5574c37c6b50 msgr2=0x5574c37d3950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:27.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.969+0000 7fad0effd640 1 --2- 172.21.15.86:0/4239964882 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5574c37c6b50 0x5574c37d3950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.969+0000 7fad0ffff640 1 --2- 172.21.15.86:0/3679973214 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7facf80a3d20 0x7facf80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:27.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.969+0000 7fad0effd640 1 -- 172.21.15.86:0/4239964882 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574c35077d0 con 0x5574c37c6230 2024-04-21T08:46:27.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.970+0000 7fad0cff9640 1 -- 172.21.15.86:0/3679973214 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fad080d0e10 con 0x7fad10108060 2024-04-21T08:46:27.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.970+0000 7fad0effd640 1 --2- 172.21.15.86:0/4239964882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c37c6230 0x5574c37c6610 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fad00000c00 tx=0x7fad00005e50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44a226ead0cc57e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:27.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.970+0000 7facf5ffb640 1 -- 172.21.15.86:0/4239964882 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad00043020 con 0x5574c37c6230 2024-04-21T08:46:27.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.970+0000 7fad0ffff640 1 --2- 172.21.15.86:0/3679973214 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7facf80a3d20 0x7facf80a61e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fad080993b0 tx=0x7fad0806e420 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:27.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.970+0000 7facf5ffb640 1 -- 172.21.15.86:0/4239964882 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad0002cc90 con 0x5574c37c6230 2024-04-21T08:46:27.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.970+0000 7facf5ffb640 1 -- 172.21.15.86:0/4239964882 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad000046c0 con 0x5574c37c6230 2024-04-21T08:46:27.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.970+0000 7fad1876f740 1 -- 172.21.15.86:0/4239964882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c37c6230 msgr2=0x5574c37c6610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:27.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.970+0000 7fad1876f740 1 --2- 172.21.15.86:0/4239964882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c37c6230 0x5574c37c6610 secure :-1 s=READY pgs=1550 cs=0 l=1 rev1=1 crypto rx=0x7fad00000c00 tx=0x7fad00005e50 comp rx=0 tx=0).stop 2024-04-21T08:46:27.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.971+0000 7fad1876f740 1 -- 172.21.15.86:0/4239964882 shutdown_connections 2024-04-21T08:46:27.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.971+0000 7fad1876f740 1 --2- 172.21.15.86:0/4239964882 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574c37d3e90 0x5574c37d6280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.971+0000 7fad1876f740 1 --2- 172.21.15.86:0/4239964882 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5574c37c6b50 0x5574c37d3950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.971+0000 7fad1876f740 1 --2- 172.21.15.86:0/4239964882 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c37c6230 0x5574c37c6610 unknown :-1 s=CLOSED pgs=1550 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.971+0000 7fad1876f740 1 -- 172.21.15.86:0/4239964882 >> 172.21.15.86:0/4239964882 conn(0x5574c37c0cb0 msgr2=0x5574c37c1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:27.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.971+0000 7fad1876f740 1 -- 172.21.15.86:0/4239964882 shutdown_connections 2024-04-21T08:46:27.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.971+0000 7fad1876f740 1 -- 172.21.15.86:0/4239964882 wait complete. 2024-04-21T08:46:27.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.971+0000 7fad1876f740 1 -- start start 2024-04-21T08:46:27.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.971+0000 7fad1876f740 1 Processor -- start 2024-04-21T08:46:27.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.971+0000 7fad1876f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574c37c6230 0x5574c37d0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:27.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.971+0000 7fad1876f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5574c37c6b50 0x5574c37d1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:27.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.971+0000 7fad1876f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c37d3e90 0x5574c37d1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:27.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.971+0000 7fad1876f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5574c37d6c70 con 0x5574c37d3e90 2024-04-21T08:46:27.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.971+0000 7fad1876f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5574c36e42f0 con 0x5574c37c6b50 2024-04-21T08:46:27.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.971+0000 7fad1876f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5574c37d6df0 con 0x5574c37c6230 2024-04-21T08:46:27.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.971+0000 7fad1876f740 10 client.0 _ll_register_callbacks cb 0x5574c36f1580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:27.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.971+0000 7fad0effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c37d3e90 0x5574c37d1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:27.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.971+0000 7fad0effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c37d3e90 0x5574c37d1870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47790/0 (socket says 172.21.15.86:47790) 2024-04-21T08:46:27.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.972+0000 7fad0effd640 1 -- 172.21.15.86:0/1582297634 learned_addr learned my addr 172.21.15.86:0/1582297634 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:27.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.972+0000 7fad0f7fe640 1 --2- 172.21.15.86:0/1582297634 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5574c37c6b50 0x5574c37d1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:27.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.972+0000 7fad0effd640 1 --2- 172.21.15.86:0/1582297634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574c37c6230 0x5574c37d0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:27.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.972+0000 7fad0effd640 1 -- 172.21.15.86:0/1582297634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574c37c6230 msgr2=0x5574c37d0df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:27.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.972+0000 7fad0effd640 1 --2- 172.21.15.86:0/1582297634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574c37c6230 0x5574c37d0df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.972+0000 7fad0effd640 1 -- 172.21.15.86:0/1582297634 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5574c37c6b50 msgr2=0x5574c37d1330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:27.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.972+0000 7fad0effd640 1 --2- 172.21.15.86:0/1582297634 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5574c37c6b50 0x5574c37d1330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.972+0000 7fad0effd640 1 -- 172.21.15.86:0/1582297634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574c343d3f0 con 0x5574c37d3e90 2024-04-21T08:46:27.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.972+0000 7fad0f7fe640 1 --2- 172.21.15.86:0/1582297634 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5574c37c6b50 0x5574c37d1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:46:27.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.972+0000 7fad0effd640 1 --2- 172.21.15.86:0/1582297634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c37d3e90 0x5574c37d1870 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fad0002cc50 tx=0x7fad0002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ec7df0d4ceb15b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:27.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.972+0000 7faceeffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:46:27.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.972+0000 7faceeffd640 1 -- 172.21.15.86:0/1582297634 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad00050020 con 0x5574c37d3e90 2024-04-21T08:46:27.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.972+0000 7faceeffd640 1 -- 172.21.15.86:0/1582297634 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fad000046c0 con 0x5574c37d3e90 2024-04-21T08:46:27.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.972+0000 7fad1876f740 1 -- 172.21.15.86:0/1582297634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad0002cc90 con 0x5574c37d3e90 2024-04-21T08:46:27.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.972+0000 7faceeffd640 1 -- 172.21.15.86:0/1582297634 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fad00043030 con 0x5574c37d3e90 2024-04-21T08:46:27.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.972+0000 7fad1876f740 10 client.18840 fetch_fsmap learned FSMap version 29 2024-04-21T08:46:27.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.972+0000 7fad1876f740 1 -- 172.21.15.86:0/1582297634 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5574c33fd0f0 con 0x5574c37d3e90 2024-04-21T08:46:27.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.972+0000 7faceeffd640 1 -- 172.21.15.86:0/1582297634 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad0002cc90 con 0x5574c37d3e90 2024-04-21T08:46:27.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.973+0000 7faceeffd640 1 -- 172.21.15.86:0/1582297634 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fad00003c40 con 0x5574c37d3e90 2024-04-21T08:46:27.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.973+0000 7fad1876f740 10 client.18840 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:46:27.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.973+0000 7fad1876f740 10 client.18840 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:27.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.973+0000 7fad1876f740 20 client.18840 populate_metadata read hostname 'smithi086' 2024-04-21T08:46:27.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.973+0000 7fad1876f740 1 --2- 172.21.15.86:0/1582297634 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5574c37e3cd0 0x5574c37e60c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:27.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.973+0000 7fad1876f740 4 client.18840 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:27.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.973+0000 7fad1876f740 1 -- 172.21.15.86:0/1582297634 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5574c3738c00 con 0x5574c37e3cd0 2024-04-21T08:46:27.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.973+0000 7fad0f7fe640 1 --2- 172.21.15.86:0/1582297634 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5574c37e3cd0 0x5574c37e60c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:27.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.973+0000 7faceeffd640 10 client.18840 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:27.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.973+0000 7fad0f7fe640 1 --2- 172.21.15.86:0/1582297634 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5574c37e3cd0 0x5574c37e60c0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4d25f275b80b8c32 server_cookie=f815fd7a7714c585 in_seq=0 out_seq=0 2024-04-21T08:46:27.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.974+0000 7faceeffd640 1 -- 172.21.15.86:0/1582297634 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5574c3738c00 con 0x5574c37e3cd0 2024-04-21T08:46:27.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.974+0000 7faceeffd640 10 client.18840 handle_command_reply: tid=0 2024-04-21T08:46:27.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.974+0000 7fad1876f740 1 client.18840 shutdown 2024-04-21T08:46:27.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.974+0000 7fad1876f740 1 -- 172.21.15.86:0/1582297634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c37d3e90 msgr2=0x5574c37d1870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:27.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.974+0000 7fad1876f740 1 --2- 172.21.15.86:0/1582297634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c37d3e90 0x5574c37d1870 secure :-1 s=READY pgs=1551 cs=0 l=1 rev1=1 crypto rx=0x7fad0002cc50 tx=0x7fad0002bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:27.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.974+0000 7fad1876f740 1 -- 172.21.15.86:0/1582297634 shutdown_connections 2024-04-21T08:46:27.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.974+0000 7fad1876f740 1 --2- 172.21.15.86:0/1582297634 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5574c37e3cd0 0x5574c37e60c0 crc :-1 s=READY pgs=237 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.974+0000 7fad1876f740 1 --2- 172.21.15.86:0/1582297634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c37d3e90 0x5574c37d1870 unknown :-1 s=CLOSED pgs=1551 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.974+0000 7fad1876f740 1 --2- 172.21.15.86:0/1582297634 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5574c37c6b50 0x5574c37d1330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.974+0000 7faceeffd640 0 client.18840 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:27.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.974+0000 7fad1876f740 1 --2- 172.21.15.86:0/1582297634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574c37c6230 0x5574c37d0df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:27.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.974+0000 7fad1876f740 1 -- 172.21.15.86:0/1582297634 >> 172.21.15.86:0/1582297634 conn(0x5574c37c0cb0 msgr2=0x5574c37c2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:27.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.974+0000 7fad1876f740 1 -- 172.21.15.86:0/1582297634 shutdown_connections 2024-04-21T08:46:27.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.974+0000 7fad1876f740 1 -- 172.21.15.86:0/1582297634 wait complete. 2024-04-21T08:46:27.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.974+0000 7fad1876f740 20 client.18840 trim_cache size 0 max 16384 2024-04-21T08:46:27.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.991+0000 7fad1876f740 1 Processor -- start 2024-04-21T08:46:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.991+0000 7fad1876f740 1 -- start start 2024-04-21T08:46:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.992+0000 7fad1876f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5574c37c6230 0x5574c37d3520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.992+0000 7fad1876f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c37c6b50 0x5574c37bfbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.992+0000 7fad1876f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574c37d3e90 0x5574c37e90e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.992+0000 7fad1876f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fad101ad140 con 0x5574c37c6b50 2024-04-21T08:46:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.992+0000 7fad1876f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fad101aec80 con 0x5574c37c6230 2024-04-21T08:46:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.992+0000 7fad1876f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fad101ad810 con 0x5574c37d3e90 2024-04-21T08:46:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.992+0000 7fad0f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c37c6b50 0x5574c37bfbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.992+0000 7fad0f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c37c6b50 0x5574c37bfbf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47806/0 (socket says 172.21.15.86:47806) 2024-04-21T08:46:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.992+0000 7fad0f7fe640 1 -- 172.21.15.86:0/3750710897 learned_addr learned my addr 172.21.15.86:0/3750710897 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:28.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.992+0000 7fad0effd640 1 --2- 172.21.15.86:0/3750710897 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5574c37c6230 0x5574c37d3520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:28.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.992+0000 7fad0f7fe640 1 -- 172.21.15.86:0/3750710897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574c37d3e90 msgr2=0x5574c37e90e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:28.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.992+0000 7fad0f7fe640 1 --2- 172.21.15.86:0/3750710897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574c37d3e90 0x5574c37e90e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:28.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.992+0000 7fad0f7fe640 1 -- 172.21.15.86:0/3750710897 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5574c37c6230 msgr2=0x5574c37d3520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:28.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.992+0000 7fad0f7fe640 1 --2- 172.21.15.86:0/3750710897 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5574c37c6230 0x5574c37d3520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:28.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.992+0000 7fad0f7fe640 1 -- 172.21.15.86:0/3750710897 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5574c37c0ae0 con 0x5574c37c6b50 2024-04-21T08:46:28.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.992+0000 7fad0f7fe640 1 --2- 172.21.15.86:0/3750710897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c37c6b50 0x5574c37bfbf0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x5574c37d6eb0 tx=0x7fad04011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7d09f05c3b0816a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:28.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.992+0000 7facf57fa640 1 -- 172.21.15.86:0/3750710897 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04004230 con 0x5574c37c6b50 2024-04-21T08:46:28.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.992+0000 7facf57fa640 1 -- 172.21.15.86:0/3750710897 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad04022070 con 0x5574c37c6b50 2024-04-21T08:46:28.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.992+0000 7facf57fa640 1 -- 172.21.15.86:0/3750710897 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad04003ca0 con 0x5574c37c6b50 2024-04-21T08:46:28.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.993+0000 7fad1876f740 1 -- 172.21.15.86:0/3750710897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c37c6b50 msgr2=0x5574c37bfbf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:28.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.993+0000 7fad1876f740 1 --2- 172.21.15.86:0/3750710897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c37c6b50 0x5574c37bfbf0 secure :-1 s=READY pgs=1552 cs=0 l=1 rev1=1 crypto rx=0x5574c37d6eb0 tx=0x7fad04011810 comp rx=0 tx=0).stop 2024-04-21T08:46:28.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.993+0000 7fad1876f740 1 -- 172.21.15.86:0/3750710897 shutdown_connections 2024-04-21T08:46:28.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.993+0000 7fad1876f740 1 --2- 172.21.15.86:0/3750710897 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574c37d3e90 0x5574c37e90e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:28.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.993+0000 7fad1876f740 1 --2- 172.21.15.86:0/3750710897 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c37c6b50 0x5574c37bfbf0 unknown :-1 s=CLOSED pgs=1552 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:28.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.993+0000 7fad1876f740 1 --2- 172.21.15.86:0/3750710897 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5574c37c6230 0x5574c37d3520 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:28.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.993+0000 7fad1876f740 1 -- 172.21.15.86:0/3750710897 >> 172.21.15.86:0/3750710897 conn(0x5574c37c0cb0 msgr2=0x5574c37bf4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:28.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.993+0000 7fad1876f740 1 -- 172.21.15.86:0/3750710897 shutdown_connections 2024-04-21T08:46:28.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.993+0000 7fad1876f740 1 -- 172.21.15.86:0/3750710897 wait complete. 2024-04-21T08:46:28.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.993+0000 7fad1876f740 1 -- start start 2024-04-21T08:46:28.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.993+0000 7fad1876f740 1 Processor -- start 2024-04-21T08:46:28.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.994+0000 7fad1876f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5574c37c6230 0x5574c37e9400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:28.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.994+0000 7fad1876f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c37c6b50 0x5574c37e6790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:28.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.994+0000 7fad1876f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574c37d3e90 0x5574c37e6cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:28.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.994+0000 7fad1876f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fad101ae5b0 con 0x5574c37c6b50 2024-04-21T08:46:28.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.994+0000 7fad1876f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fad101adee0 con 0x5574c37c6230 2024-04-21T08:46:28.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.994+0000 7fad1876f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fad101ab140 con 0x5574c37d3e90 2024-04-21T08:46:28.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.994+0000 7fad1876f740 10 client.0 _ll_register_callbacks cb 0x7fad0400b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:28.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.994+0000 7fad0f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c37c6b50 0x5574c37e6790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:28.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.994+0000 7fad0f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c37c6b50 0x5574c37e6790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47808/0 (socket says 172.21.15.86:47808) 2024-04-21T08:46:28.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.994+0000 7fad0f7fe640 1 -- 172.21.15.86:0/2902417983 learned_addr learned my addr 172.21.15.86:0/2902417983 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:28.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.994+0000 7fad0effd640 1 --2- 172.21.15.86:0/2902417983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5574c37c6230 0x5574c37e9400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:28.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.994+0000 7fad0f7fe640 1 -- 172.21.15.86:0/2902417983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574c37d3e90 msgr2=0x5574c37e6cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:28.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.994+0000 7fad0f7fe640 1 --2- 172.21.15.86:0/2902417983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574c37d3e90 0x5574c37e6cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:28.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.994+0000 7fad0f7fe640 1 -- 172.21.15.86:0/2902417983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5574c37c6230 msgr2=0x5574c37e9400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:28.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.994+0000 7fad0f7fe640 1 --2- 172.21.15.86:0/2902417983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5574c37c6230 0x5574c37e9400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:28.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.994+0000 7fad0f7fe640 1 -- 172.21.15.86:0/2902417983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5574c37d6280 con 0x5574c37c6b50 2024-04-21T08:46:28.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.994+0000 7fad0f7fe640 1 --2- 172.21.15.86:0/2902417983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c37c6b50 0x5574c37e6790 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fad0400b370 tx=0x7fad040037d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb61f5cc2b1c348c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:28.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.994+0000 7faceeffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:46:28.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.994+0000 7faceeffd640 1 -- 172.21.15.86:0/2902417983 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fad0401d8f0 con 0x5574c37c6b50 2024-04-21T08:46:28.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.994+0000 7faceeffd640 1 -- 172.21.15.86:0/2902417983 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fad04030420 con 0x5574c37c6b50 2024-04-21T08:46:28.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.994+0000 7fad1876f740 1 -- 172.21.15.86:0/2902417983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fad04022070 con 0x5574c37c6b50 2024-04-21T08:46:28.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.994+0000 7faceeffd640 1 -- 172.21.15.86:0/2902417983 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fad04038050 con 0x5574c37c6b50 2024-04-21T08:46:28.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.994+0000 7fad1876f740 10 client.18846 fetch_fsmap learned FSMap version 29 2024-04-21T08:46:28.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.994+0000 7fad1876f740 1 -- 172.21.15.86:0/2902417983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5574c3849c70 con 0x5574c37c6b50 2024-04-21T08:46:28.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.995+0000 7faceeffd640 1 -- 172.21.15.86:0/2902417983 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fad04022070 con 0x5574c37c6b50 2024-04-21T08:46:28.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.995+0000 7faceeffd640 1 -- 172.21.15.86:0/2902417983 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fad04032700 con 0x5574c37c6b50 2024-04-21T08:46:28.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.995+0000 7fad1876f740 10 client.18846 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:46:28.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.995+0000 7fad1876f740 10 client.18846 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:28.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.995+0000 7fad1876f740 20 client.18846 populate_metadata read hostname 'smithi086' 2024-04-21T08:46:28.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.995+0000 7fad1876f740 1 --2- 172.21.15.86:0/2902417983 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5574c384a450 0x5574c384c870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:28.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.995+0000 7fad1876f740 4 client.18846 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:46:28.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.995+0000 7fad1876f740 1 -- 172.21.15.86:0/2902417983 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5574c3732a70 con 0x5574c384a450 2024-04-21T08:46:28.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.995+0000 7fad0effd640 1 --2- 172.21.15.86:0/2902417983 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5574c384a450 0x5574c384c870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:28.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.996+0000 7faceeffd640 10 client.18846 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:28.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.996+0000 7fad0effd640 1 --2- 172.21.15.86:0/2902417983 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5574c384a450 0x5574c384c870 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=86a57ced1ac5b92a server_cookie=42ea5e11c5e370cb in_seq=0 out_seq=0 2024-04-21T08:46:28.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.996+0000 7faceeffd640 1 -- 172.21.15.86:0/2902417983 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5574c3732a70 con 0x5574c384a450 2024-04-21T08:46:28.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.996+0000 7faceeffd640 10 client.18846 handle_command_reply: tid=0 2024-04-21T08:46:28.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.996+0000 7fad1876f740 1 client.18846 shutdown 2024-04-21T08:46:28.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.996+0000 7fad1876f740 1 -- 172.21.15.86:0/2902417983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c37c6b50 msgr2=0x5574c37e6790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:28.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.996+0000 7fad1876f740 1 --2- 172.21.15.86:0/2902417983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c37c6b50 0x5574c37e6790 secure :-1 s=READY pgs=1553 cs=0 l=1 rev1=1 crypto rx=0x7fad0400b370 tx=0x7fad040037d0 comp rx=0 tx=0).stop 2024-04-21T08:46:28.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.996+0000 7fad1876f740 1 -- 172.21.15.86:0/2902417983 shutdown_connections 2024-04-21T08:46:28.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.996+0000 7fad1876f740 1 --2- 172.21.15.86:0/2902417983 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5574c384a450 0x5574c384c870 crc :-1 s=READY pgs=238 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:28.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.996+0000 7fad1876f740 1 --2- 172.21.15.86:0/2902417983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5574c37d3e90 0x5574c37e6cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:28.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.996+0000 7fad1876f740 1 --2- 172.21.15.86:0/2902417983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5574c37c6b50 0x5574c37e6790 unknown :-1 s=CLOSED pgs=1553 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:28.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.996+0000 7fad1876f740 1 --2- 172.21.15.86:0/2902417983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5574c37c6230 0x5574c37e9400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:28.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.996+0000 7fad1876f740 1 -- 172.21.15.86:0/2902417983 >> 172.21.15.86:0/2902417983 conn(0x5574c37c0cb0 msgr2=0x5574c37bfd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:28.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.996+0000 7faceeffd640 0 client.18846 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:28.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.996+0000 7fad1876f740 1 -- 172.21.15.86:0/2902417983 shutdown_connections 2024-04-21T08:46:28.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.996+0000 7fad1876f740 1 -- 172.21.15.86:0/2902417983 wait complete. 2024-04-21T08:46:28.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.996+0000 7fad1876f740 20 client.18846 trim_cache size 0 max 16384 2024-04-21T08:46:28.007 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:46:28.008 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:46:28.008 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-21T08:46:28.008 INFO:teuthology.orchestra.run.smithi086.stdout: "cfc13b84-b562-4019-92e6-ac86f19f6a7c": { 2024-04-21T08:46:28.008 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-21T08:46:28.008 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "cfc13b84-b562-4019-92e6-ac86f19f6a7c", 2024-04-21T08:46:28.008 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-21T08:46:28.008 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:46:28.008 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:46:28.008 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:46:28.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.996+0000 7fad15b7e640 1 -- 172.21.15.86:0/3679973214 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7facf80a3d20 msgr2=0x7facf80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:28.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.997+0000 7fad15b7e640 1 --2- 172.21.15.86:0/3679973214 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7facf80a3d20 0x7facf80a61e0 secure :-1 s=READY pgs=250 cs=0 l=1 rev1=1 crypto rx=0x7fad080993b0 tx=0x7fad0806e420 comp rx=0 tx=0).stop 2024-04-21T08:46:28.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.997+0000 7fad15b7e640 1 -- 172.21.15.86:0/3679973214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad10108060 msgr2=0x7fad1019cf70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:28.009 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.997+0000 7fad15b7e640 1 --2- 172.21.15.86:0/3679973214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad10108060 0x7fad1019cf70 secure :-1 s=READY pgs=1549 cs=0 l=1 rev1=1 crypto rx=0x7fad0800eaa0 tx=0x7fad0800ef60 comp rx=0 tx=0).stop 2024-04-21T08:46:28.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.997+0000 7fad15b7e640 1 -- 172.21.15.86:0/3679973214 shutdown_connections 2024-04-21T08:46:28.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.997+0000 7fad15b7e640 1 --2- 172.21.15.86:0/3679973214 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7facf80a3d20 0x7facf80a61e0 unknown :-1 s=CLOSED pgs=250 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:28.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.997+0000 7fad15b7e640 1 --2- 172.21.15.86:0/3679973214 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fad10108060 0x7fad1019cf70 unknown :-1 s=CLOSED pgs=1549 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:28.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.997+0000 7fad15b7e640 1 --2- 172.21.15.86:0/3679973214 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fad1006af70 0x7fad10198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:28.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.997+0000 7fad15b7e640 1 --2- 172.21.15.86:0/3679973214 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fad1006a5b0 0x7fad10198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:28.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.997+0000 7fad15b7e640 1 -- 172.21.15.86:0/3679973214 >> 172.21.15.86:0/3679973214 conn(0x7fad10077c40 msgr2=0x7fad100ffe20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:28.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.997+0000 7fad15b7e640 1 -- 172.21.15.86:0/3679973214 shutdown_connections 2024-04-21T08:46:28.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:27.997+0000 7fad15b7e640 1 -- 172.21.15.86:0/3679973214 wait complete. 2024-04-21T08:46:28.011 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:27 smithi086 ceph-mon[27939]: pgmap v237: 129 pgs: 129 active+clean; 5.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-21T08:46:28.012 INFO:tasks.cephfs.filesystem:scrub status for tag:cfc13b84-b562-4019-92e6-ac86f19f6a7c - {'path': '/', 'tag': 'cfc13b84-b562-4019-92e6-ac86f19f6a7c', 'options': 'recursive,force'} 2024-04-21T08:46:28.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:27 smithi088 ceph-mon[31259]: pgmap v237: 129 pgs: 129 active+clean; 5.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-21T08:46:28.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:27 smithi179 ceph-mon[31829]: pgmap v237: 129 pgs: 129 active+clean; 5.1 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-21T08:46:29.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:28 smithi086 ceph-mon[27939]: scrub queued for path: / 2024-04-21T08:46:29.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:28 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:46:29.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:28 smithi086 ceph-mon[27939]: scrub summary: active paths [/] 2024-04-21T08:46:29.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:28 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:46:29.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:28 smithi088 ceph-mon[31259]: scrub queued for path: / 2024-04-21T08:46:29.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:28 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:46:29.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:28 smithi088 ceph-mon[31259]: scrub summary: active paths [/] 2024-04-21T08:46:29.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:28 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:46:29.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:28 smithi179 ceph-mon[31829]: scrub queued for path: / 2024-04-21T08:46:29.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:28 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:46:29.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:28 smithi179 ceph-mon[31829]: scrub summary: active paths [/] 2024-04-21T08:46:29.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:28 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:46:30.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:29 smithi086 ceph-mon[27939]: pgmap v238: 129 pgs: 129 active+clean; 5.2 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-21T08:46:30.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:29 smithi086 ceph-mon[27939]: scrub summary: idle 2024-04-21T08:46:30.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:29 smithi088 ceph-mon[31259]: pgmap v238: 129 pgs: 129 active+clean; 5.2 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-21T08:46:30.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:29 smithi088 ceph-mon[31259]: scrub summary: idle 2024-04-21T08:46:30.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:29 smithi179 ceph-mon[31829]: pgmap v238: 129 pgs: 129 active+clean; 5.2 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-21T08:46:30.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:29 smithi179 ceph-mon[31829]: scrub summary: idle 2024-04-21T08:46:32.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:31 smithi086 ceph-mon[27939]: pgmap v239: 129 pgs: 129 active+clean; 5.2 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-04-21T08:46:32.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:31 smithi088 ceph-mon[31259]: pgmap v239: 129 pgs: 129 active+clean; 5.2 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-04-21T08:46:32.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:31 smithi179 ceph-mon[31829]: pgmap v239: 129 pgs: 129 active+clean; 5.2 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 14 op/s 2024-04-21T08:46:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.825+0000 7ff75cef4640 1 -- 172.21.15.86:0/3707836028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff758106170 msgr2=0x7ff75810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.825+0000 7ff75cef4640 1 --2- 172.21.15.86:0/3707836028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff758106170 0x7ff75810dd40 secure :-1 s=READY pgs=1554 cs=0 l=1 rev1=1 crypto rx=0x7ff750009ca0 tx=0x7ff75002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:46:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.825+0000 7ff75cef4640 1 -- 172.21.15.86:0/3707836028 shutdown_connections 2024-04-21T08:46:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.825+0000 7ff75cef4640 1 --2- 172.21.15.86:0/3707836028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff758106170 0x7ff75810dd40 unknown :-1 s=CLOSED pgs=1554 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.825+0000 7ff75cef4640 1 --2- 172.21.15.86:0/3707836028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff758101aa0 0x7ff758105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.825+0000 7ff75cef4640 1 --2- 172.21.15.86:0/3707836028 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7581010f0 0x7ff7581014d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.825+0000 7ff75cef4640 1 -- 172.21.15.86:0/3707836028 >> 172.21.15.86:0/3707836028 conn(0x7ff758077fd0 msgr2=0x7ff7580783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:33.827 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.825+0000 7ff75cef4640 1 -- 172.21.15.86:0/3707836028 shutdown_connections 2024-04-21T08:46:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.825+0000 7ff75cef4640 1 -- 172.21.15.86:0/3707836028 wait complete. 2024-04-21T08:46:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.826+0000 7ff75cef4640 1 Processor -- start 2024-04-21T08:46:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.826+0000 7ff75cef4640 1 -- start start 2024-04-21T08:46:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.826+0000 7ff75cef4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7581010f0 0x7ff7581986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.826+0000 7ff75cef4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff758106170 0x7ff758198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.826+0000 7ff75cef4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff75819cf80 0x7ff75819d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.826+0000 7ff75cef4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff7581110e0 con 0x7ff75819cf80 2024-04-21T08:46:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.826+0000 7ff75cef4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff758110f60 con 0x7ff7581010f0 2024-04-21T08:46:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.826+0000 7ff75cef4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff758111260 con 0x7ff758106170 2024-04-21T08:46:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.826+0000 7ff756d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff75819cf80 0x7ff75819d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.826+0000 7ff756d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff75819cf80 0x7ff75819d430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46362/0 (socket says 172.21.15.86:46362) 2024-04-21T08:46:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.826+0000 7ff756d9d640 1 -- 172.21.15.86:0/1204461204 learned_addr learned my addr 172.21.15.86:0/1204461204 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.826+0000 7ff75659c640 1 --2- 172.21.15.86:0/1204461204 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7581010f0 0x7ff7581986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:33.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.826+0000 7ff755d9b640 1 --2- 172.21.15.86:0/1204461204 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff758106170 0x7ff758198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.827+0000 7ff756d9d640 1 -- 172.21.15.86:0/1204461204 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff758106170 msgr2=0x7ff758198c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.827+0000 7ff756d9d640 1 --2- 172.21.15.86:0/1204461204 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff758106170 0x7ff758198c00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.827+0000 7ff756d9d640 1 -- 172.21.15.86:0/1204461204 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7581010f0 msgr2=0x7ff7581986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.827+0000 7ff756d9d640 1 --2- 172.21.15.86:0/1204461204 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7581010f0 0x7ff7581986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.827+0000 7ff756d9d640 1 -- 172.21.15.86:0/1204461204 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff75819da70 con 0x7ff75819cf80 2024-04-21T08:46:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.827+0000 7ff75659c640 1 --2- 172.21.15.86:0/1204461204 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7581010f0 0x7ff7581986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.827+0000 7ff756d9d640 1 --2- 172.21.15.86:0/1204461204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff75819cf80 0x7ff75819d430 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7ff750005e30 tx=0x7ff750032420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.827+0000 7ff7477fe640 1 -- 172.21.15.86:0/1204461204 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff750041020 con 0x7ff75819cf80 2024-04-21T08:46:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.827+0000 7ff7477fe640 1 -- 172.21.15.86:0/1204461204 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff750032970 con 0x7ff75819cf80 2024-04-21T08:46:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.827+0000 7ff75cef4640 1 -- 172.21.15.86:0/1204461204 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff75819dd00 con 0x7ff75819cf80 2024-04-21T08:46:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.827+0000 7ff7477fe640 1 -- 172.21.15.86:0/1204461204 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff750039040 con 0x7ff75819cf80 2024-04-21T08:46:33.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.827+0000 7ff75cef4640 1 -- 172.21.15.86:0/1204461204 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff7581a54c0 con 0x7ff75819cf80 2024-04-21T08:46:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.830+0000 7ff7477fe640 1 -- 172.21.15.86:0/1204461204 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff750044020 con 0x7ff75819cf80 2024-04-21T08:46:33.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.830+0000 7ff75fbbd740 1 Processor -- start 2024-04-21T08:46:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.830+0000 7ff75fbbd740 1 -- start start 2024-04-21T08:46:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.830+0000 7ff75fbbd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8230978d0 0x55f823096620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.830+0000 7ff75fbbd740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f823096b60 0x55f823096fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.830+0000 7ff75fbbd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8230a00e0 0x55f8230a24d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.830+0000 7ff75fbbd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f822ffb8d0 con 0x55f8230a00e0 2024-04-21T08:46:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.830+0000 7ff75fbbd740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55f822f98e80 con 0x55f823096b60 2024-04-21T08:46:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.830+0000 7ff75fbbd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f822ff7890 con 0x55f8230978d0 2024-04-21T08:46:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.831+0000 7ff7477fe640 1 --2- 172.21.15.86:0/1204461204 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff7400a3dc0 0x7ff7400a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.831+0000 7ff755d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8230a00e0 0x55f8230a24d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.831+0000 7ff756d9d640 1 --2- 172.21.15.86:0/1204461204 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff7400a3dc0 0x7ff7400a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.831+0000 7ff755d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8230a00e0 0x55f8230a24d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46366/0 (socket says 172.21.15.86:46366) 2024-04-21T08:46:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.831+0000 7ff755d9b640 1 -- 172.21.15.86:0/1537355002 learned_addr learned my addr 172.21.15.86:0/1537355002 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.831+0000 7ff755d9b640 1 --2- 172.21.15.86:0/1537355002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8230978d0 0x55f823096620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:33.837 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.831+0000 7ff75659c640 1 --2- 172.21.15.86:0/1537355002 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f823096b60 0x55f823096fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.831+0000 7ff7477fe640 1 -- 172.21.15.86:0/1204461204 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7ff7500f2890 con 0x7ff75819cf80 2024-04-21T08:46:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.831+0000 7ff755d9b640 1 -- 172.21.15.86:0/1537355002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8230978d0 msgr2=0x55f823096620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.831+0000 7ff755d9b640 1 --2- 172.21.15.86:0/1537355002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8230978d0 0x55f823096620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.831+0000 7ff755d9b640 1 -- 172.21.15.86:0/1537355002 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f823096b60 msgr2=0x55f823096fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.831+0000 7ff755d9b640 1 --2- 172.21.15.86:0/1537355002 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f823096b60 0x55f823096fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.831+0000 7ff755d9b640 1 -- 172.21.15.86:0/1537355002 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f822fba320 con 0x55f8230a00e0 2024-04-21T08:46:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.831+0000 7ff756d9d640 1 --2- 172.21.15.86:0/1204461204 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff7400a3dc0 0x7ff7400a6280 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff7500baf40 tx=0x7ff75008de50 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.831+0000 7ff755d9b640 1 --2- 172.21.15.86:0/1537355002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8230a00e0 0x55f8230a24d0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7ff74c009f00 tx=0x7ff74c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90f6cd5eb271190e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.831+0000 7ff744ff9640 1 -- 172.21.15.86:0/1537355002 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c004050 con 0x55f8230a00e0 2024-04-21T08:46:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.831+0000 7ff744ff9640 1 -- 172.21.15.86:0/1537355002 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff74c0054d0 con 0x55f8230a00e0 2024-04-21T08:46:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.831+0000 7ff744ff9640 1 -- 172.21.15.86:0/1537355002 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c004c20 con 0x55f8230a00e0 2024-04-21T08:46:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.832+0000 7ff75fbbd740 1 -- 172.21.15.86:0/1537355002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8230a00e0 msgr2=0x55f8230a24d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.832+0000 7ff75fbbd740 1 --2- 172.21.15.86:0/1537355002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8230a00e0 0x55f8230a24d0 secure :-1 s=READY pgs=1556 cs=0 l=1 rev1=1 crypto rx=0x7ff74c009f00 tx=0x7ff74c004730 comp rx=0 tx=0).stop 2024-04-21T08:46:33.838 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.832+0000 7ff75fbbd740 1 -- 172.21.15.86:0/1537355002 shutdown_connections 2024-04-21T08:46:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.832+0000 7ff75fbbd740 1 --2- 172.21.15.86:0/1537355002 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8230a00e0 0x55f8230a24d0 unknown :-1 s=CLOSED pgs=1556 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.832+0000 7ff75fbbd740 1 --2- 172.21.15.86:0/1537355002 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f823096b60 0x55f823096fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.832+0000 7ff75fbbd740 1 --2- 172.21.15.86:0/1537355002 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8230978d0 0x55f823096620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.832+0000 7ff75fbbd740 1 -- 172.21.15.86:0/1537355002 >> 172.21.15.86:0/1537355002 conn(0x55f82305dc90 msgr2=0x55f823091850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.832+0000 7ff75fbbd740 1 -- 172.21.15.86:0/1537355002 shutdown_connections 2024-04-21T08:46:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.832+0000 7ff75fbbd740 1 -- 172.21.15.86:0/1537355002 wait complete. 2024-04-21T08:46:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.833+0000 7ff75fbbd740 1 -- start start 2024-04-21T08:46:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.833+0000 7ff75fbbd740 1 Processor -- start 2024-04-21T08:46:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.833+0000 7ff75fbbd740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f823096b60 0x55f82309fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.833+0000 7ff75fbbd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8230978d0 0x55f82309ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.833+0000 7ff75fbbd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8230a00e0 0x55f82309d460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.833+0000 7ff75fbbd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f8230a2e10 con 0x55f8230a00e0 2024-04-21T08:46:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.833+0000 7ff75fbbd740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55f822f83eb0 con 0x55f823096b60 2024-04-21T08:46:33.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.833+0000 7ff75fbbd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f8230a2f90 con 0x55f8230978d0 2024-04-21T08:46:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.833+0000 7ff75fbbd740 10 client.0 _ll_register_callbacks cb 0x55f822fb0b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.833+0000 7ff755d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8230a00e0 0x55f82309d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.833+0000 7ff755d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8230a00e0 0x55f82309d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46372/0 (socket says 172.21.15.86:46372) 2024-04-21T08:46:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.833+0000 7ff755d9b640 1 -- 172.21.15.86:0/746336448 learned_addr learned my addr 172.21.15.86:0/746336448 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.833+0000 7ff755d9b640 1 --2- 172.21.15.86:0/746336448 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f823096b60 0x55f82309fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.833+0000 7ff75659c640 1 --2- 172.21.15.86:0/746336448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8230978d0 0x55f82309ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.833+0000 7ff755d9b640 1 -- 172.21.15.86:0/746336448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8230978d0 msgr2=0x55f82309ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.834+0000 7ff755d9b640 1 --2- 172.21.15.86:0/746336448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8230978d0 0x55f82309ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.834+0000 7ff755d9b640 1 -- 172.21.15.86:0/746336448 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f823096b60 msgr2=0x55f82309fac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.834+0000 7ff755d9b640 1 --2- 172.21.15.86:0/746336448 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f823096b60 0x55f82309fac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.834+0000 7ff755d9b640 1 -- 172.21.15.86:0/746336448 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f822d0e210 con 0x55f8230a00e0 2024-04-21T08:46:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.834+0000 7ff755d9b640 1 --2- 172.21.15.86:0/746336448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8230a00e0 0x55f82309d460 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7ff74c004920 tx=0x7ff74c0047a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cbfab4b0411a32 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.834+0000 7ff73dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:46:33.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.834+0000 7ff73dffb640 1 -- 172.21.15.86:0/746336448 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c028040 con 0x55f8230a00e0 2024-04-21T08:46:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.834+0000 7ff73dffb640 1 -- 172.21.15.86:0/746336448 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7ff74c01e460 con 0x55f8230a00e0 2024-04-21T08:46:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.834+0000 7ff75fbbd740 1 -- 172.21.15.86:0/746336448 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff74c0054d0 con 0x55f8230a00e0 2024-04-21T08:46:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.834+0000 7ff73dffb640 1 -- 172.21.15.86:0/746336448 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff74c034050 con 0x55f8230a00e0 2024-04-21T08:46:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.834+0000 7ff75fbbd740 10 client.18858 fetch_fsmap learned FSMap version 29 2024-04-21T08:46:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.834+0000 7ff75fbbd740 1 -- 172.21.15.86:0/746336448 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f822dda620 con 0x55f8230a00e0 2024-04-21T08:46:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.834+0000 7ff73dffb640 1 -- 172.21.15.86:0/746336448 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff74c0054d0 con 0x55f8230a00e0 2024-04-21T08:46:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.834+0000 7ff73dffb640 1 -- 172.21.15.86:0/746336448 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff74c02f540 con 0x55f8230a00e0 2024-04-21T08:46:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.834+0000 7ff75fbbd740 10 client.18858 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:46:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.834+0000 7ff75fbbd740 10 client.18858 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.834+0000 7ff75fbbd740 20 client.18858 populate_metadata read hostname 'smithi086' 2024-04-21T08:46:33.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.834+0000 7ff75fbbd740 1 --2- 172.21.15.86:0/746336448 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f8230b0040 0x55f8230b2430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.834+0000 7ff75fbbd740 4 client.18858 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.834+0000 7ff75fbbd740 1 -- 172.21.15.86:0/746336448 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f82300f880 con 0x55f8230b0040 2024-04-21T08:46:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.835+0000 7ff75659c640 1 --2- 172.21.15.86:0/746336448 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f8230b0040 0x55f8230b2430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.835+0000 7ff73dffb640 10 client.18858 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.835+0000 7ff75659c640 1 --2- 172.21.15.86:0/746336448 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f8230b0040 0x55f8230b2430 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f63d5c0693d7bc18 server_cookie=17a5b607a20d1039 in_seq=0 out_seq=0 2024-04-21T08:46:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.836+0000 7ff73dffb640 1 -- 172.21.15.86:0/746336448 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f82300f880 con 0x55f8230b0040 2024-04-21T08:46:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.836+0000 7ff73dffb640 10 client.18858 handle_command_reply: tid=0 2024-04-21T08:46:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.836+0000 7ff75fbbd740 1 client.18858 shutdown 2024-04-21T08:46:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.836+0000 7ff75fbbd740 1 -- 172.21.15.86:0/746336448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8230a00e0 msgr2=0x55f82309d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.836+0000 7ff75fbbd740 1 --2- 172.21.15.86:0/746336448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8230a00e0 0x55f82309d460 secure :-1 s=READY pgs=1557 cs=0 l=1 rev1=1 crypto rx=0x7ff74c004920 tx=0x7ff74c0047a0 comp rx=0 tx=0).stop 2024-04-21T08:46:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.836+0000 7ff75fbbd740 1 -- 172.21.15.86:0/746336448 shutdown_connections 2024-04-21T08:46:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.836+0000 7ff75fbbd740 1 --2- 172.21.15.86:0/746336448 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f8230b0040 0x55f8230b2430 crc :-1 s=READY pgs=239 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:33.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.836+0000 7ff75fbbd740 1 --2- 172.21.15.86:0/746336448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8230a00e0 0x55f82309d460 unknown :-1 s=CLOSED pgs=1557 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.836+0000 7ff73dffb640 0 client.18858 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.836+0000 7ff75fbbd740 1 --2- 172.21.15.86:0/746336448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8230978d0 0x55f82309ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.836+0000 7ff75fbbd740 1 --2- 172.21.15.86:0/746336448 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f823096b60 0x55f82309fac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.836+0000 7ff75fbbd740 1 -- 172.21.15.86:0/746336448 >> 172.21.15.86:0/746336448 conn(0x55f82305dc90 msgr2=0x55f8230935a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.836+0000 7ff75fbbd740 1 -- 172.21.15.86:0/746336448 shutdown_connections 2024-04-21T08:46:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.836+0000 7ff75fbbd740 1 -- 172.21.15.86:0/746336448 wait complete. 2024-04-21T08:46:33.843 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.836+0000 7ff75fbbd740 20 client.18858 trim_cache size 0 max 16384 2024-04-21T08:46:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.851+0000 7ff75fbbd740 1 Processor -- start 2024-04-21T08:46:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.851+0000 7ff75fbbd740 1 -- start start 2024-04-21T08:46:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.851+0000 7ff75fbbd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f823096b60 0x55f8230abd40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.851+0000 7ff75fbbd740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8230978d0 0x55f82309bd20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.851+0000 7ff75fbbd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8230a00e0 0x55f82309f580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.851+0000 7ff75fbbd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff7581af280 con 0x55f823096b60 2024-04-21T08:46:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.851+0000 7ff75fbbd740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff7581b0dc0 con 0x55f8230978d0 2024-04-21T08:46:33.858 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.851+0000 7ff75fbbd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff7581af950 con 0x55f8230a00e0 2024-04-21T08:46:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.851+0000 7ff755d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f823096b60 0x55f8230abd40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.851+0000 7ff755d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f823096b60 0x55f8230abd40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46388/0 (socket says 172.21.15.86:46388) 2024-04-21T08:46:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.851+0000 7ff755d9b640 1 -- 172.21.15.86:0/1456986388 learned_addr learned my addr 172.21.15.86:0/1456986388 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.851+0000 7ff75659c640 1 --2- 172.21.15.86:0/1456986388 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8230978d0 0x55f82309bd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.851+0000 7ff755d9b640 1 -- 172.21.15.86:0/1456986388 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8230a00e0 msgr2=0x55f82309f580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.851+0000 7ff755d9b640 1 --2- 172.21.15.86:0/1456986388 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8230a00e0 0x55f82309f580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.851+0000 7ff755d9b640 1 -- 172.21.15.86:0/1456986388 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8230978d0 msgr2=0x55f82309bd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.851+0000 7ff755d9b640 1 --2- 172.21.15.86:0/1456986388 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8230978d0 0x55f82309bd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.851+0000 7ff755d9b640 1 -- 172.21.15.86:0/1456986388 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f823095e30 con 0x55f823096b60 2024-04-21T08:46:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.851+0000 7ff755d9b640 1 --2- 172.21.15.86:0/1456986388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f823096b60 0x55f8230abd40 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7ff74c021040 tx=0x7ff74c004890 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86022a29d835c17f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.852+0000 7ff73ffff640 1 -- 172.21.15.86:0/1456986388 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c023070 con 0x55f823096b60 2024-04-21T08:46:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.852+0000 7ff73ffff640 1 -- 172.21.15.86:0/1456986388 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff74c005450 con 0x55f823096b60 2024-04-21T08:46:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.852+0000 7ff73ffff640 1 -- 172.21.15.86:0/1456986388 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff74c01e460 con 0x55f823096b60 2024-04-21T08:46:33.859 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.852+0000 7ff75fbbd740 1 -- 172.21.15.86:0/1456986388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f823096b60 msgr2=0x55f8230abd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:33.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.852+0000 7ff75fbbd740 1 --2- 172.21.15.86:0/1456986388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f823096b60 0x55f8230abd40 secure :-1 s=READY pgs=1558 cs=0 l=1 rev1=1 crypto rx=0x7ff74c021040 tx=0x7ff74c004890 comp rx=0 tx=0).stop 2024-04-21T08:46:33.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.852+0000 7ff75fbbd740 1 -- 172.21.15.86:0/1456986388 shutdown_connections 2024-04-21T08:46:33.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.852+0000 7ff75fbbd740 1 --2- 172.21.15.86:0/1456986388 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f8230a00e0 0x55f82309f580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:33.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.852+0000 7ff75fbbd740 1 --2- 172.21.15.86:0/1456986388 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8230978d0 0x55f82309bd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:33.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.852+0000 7ff75fbbd740 1 --2- 172.21.15.86:0/1456986388 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f823096b60 0x55f8230abd40 unknown :-1 s=CLOSED pgs=1558 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:33.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.852+0000 7ff75fbbd740 1 -- 172.21.15.86:0/1456986388 >> 172.21.15.86:0/1456986388 conn(0x55f82305dc90 msgr2=0x55f82309b610 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:33.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.852+0000 7ff75fbbd740 1 -- 172.21.15.86:0/1456986388 shutdown_connections 2024-04-21T08:46:33.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.852+0000 7ff75fbbd740 1 -- 172.21.15.86:0/1456986388 wait complete. 2024-04-21T08:46:33.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.853+0000 7ff75fbbd740 1 -- start start 2024-04-21T08:46:33.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.853+0000 7ff75fbbd740 1 Processor -- start 2024-04-21T08:46:33.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.853+0000 7ff75fbbd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f823096b60 0x55f8230a86a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:33.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.853+0000 7ff75fbbd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8230978d0 0x55f82309a0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:33.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.853+0000 7ff75fbbd740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8230a00e0 0x55f82309a690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:33.860 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.853+0000 7ff75fbbd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff7581b06f0 con 0x55f8230978d0 2024-04-21T08:46:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.853+0000 7ff75fbbd740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff7581b0020 con 0x55f8230a00e0 2024-04-21T08:46:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.853+0000 7ff75fbbd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff7581ad280 con 0x55f823096b60 2024-04-21T08:46:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.853+0000 7ff75fbbd740 10 client.0 _ll_register_callbacks cb 0x7ff74800b440 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.853+0000 7ff75659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8230978d0 0x55f82309a0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.853+0000 7ff75659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8230978d0 0x55f82309a0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46400/0 (socket says 172.21.15.86:46400) 2024-04-21T08:46:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.853+0000 7ff75659c640 1 -- 172.21.15.86:0/2825333664 learned_addr learned my addr 172.21.15.86:0/2825333664 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.853+0000 7ff755d9b640 1 --2- 172.21.15.86:0/2825333664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8230a00e0 0x55f82309a690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.853+0000 7ff755d9b640 1 --2- 172.21.15.86:0/2825333664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f823096b60 0x55f8230a86a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.853+0000 7ff75659c640 1 -- 172.21.15.86:0/2825333664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f823096b60 msgr2=0x55f8230a86a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.853+0000 7ff75659c640 1 --2- 172.21.15.86:0/2825333664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f823096b60 0x55f8230a86a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.853+0000 7ff75659c640 1 -- 172.21.15.86:0/2825333664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8230a00e0 msgr2=0x55f82309a690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.853+0000 7ff75659c640 1 --2- 172.21.15.86:0/2825333664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8230a00e0 0x55f82309a690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.853+0000 7ff75659c640 1 -- 172.21.15.86:0/2825333664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f82309aec0 con 0x55f8230978d0 2024-04-21T08:46:33.861 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.853+0000 7ff755d9b640 1 --2- 172.21.15.86:0/2825333664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f823096b60 0x55f8230a86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.853+0000 7ff75659c640 1 --2- 172.21.15.86:0/2825333664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8230978d0 0x55f82309a0f0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x55f823098030 tx=0x7ff748011a30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff0396b90d2e55d7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.853+0000 7ff73dffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:46:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.854+0000 7ff73dffb640 1 -- 172.21.15.86:0/2825333664 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff748004300 con 0x55f8230978d0 2024-04-21T08:46:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.854+0000 7ff73dffb640 1 -- 172.21.15.86:0/2825333664 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7ff74801d470 con 0x55f8230978d0 2024-04-21T08:46:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.854+0000 7ff75fbbd740 1 -- 172.21.15.86:0/2825333664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff74c005450 con 0x55f8230978d0 2024-04-21T08:46:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.854+0000 7ff73dffb640 1 -- 172.21.15.86:0/2825333664 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff74802c050 con 0x55f8230978d0 2024-04-21T08:46:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.854+0000 7ff75fbbd740 10 client.18864 fetch_fsmap learned FSMap version 29 2024-04-21T08:46:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.854+0000 7ff75fbbd740 1 -- 172.21.15.86:0/2825333664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f823116510 con 0x55f8230978d0 2024-04-21T08:46:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.854+0000 7ff73dffb640 1 -- 172.21.15.86:0/2825333664 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff74c005450 con 0x55f8230978d0 2024-04-21T08:46:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.854+0000 7ff73dffb640 1 -- 172.21.15.86:0/2825333664 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff748010430 con 0x55f8230978d0 2024-04-21T08:46:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.854+0000 7ff75fbbd740 10 client.18864 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:46:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.854+0000 7ff75fbbd740 10 client.18864 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:33.862 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.854+0000 7ff75fbbd740 20 client.18864 populate_metadata read hostname 'smithi086' 2024-04-21T08:46:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.854+0000 7ff75fbbd740 1 --2- 172.21.15.86:0/2825333664 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f823116d10 0x55f823119130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.854+0000 7ff75fbbd740 4 client.18864 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:46:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.854+0000 7ff75fbbd740 1 -- 172.21.15.86:0/2825333664 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55f822f9b3d0 con 0x55f823116d10 2024-04-21T08:46:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.854+0000 7ff755d9b640 1 --2- 172.21.15.86:0/2825333664 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f823116d10 0x55f823119130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.855+0000 7ff73dffb640 10 client.18864 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:33.863 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.855+0000 7ff755d9b640 1 --2- 172.21.15.86:0/2825333664 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f823116d10 0x55f823119130 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d000eb8298cb4112 server_cookie=3b6ba93944bae99 in_seq=0 out_seq=0 2024-04-21T08:46:33.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.869+0000 7ff73dffb640 1 -- 172.21.15.86:0/2825333664 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55f822f9b3d0 con 0x55f823116d10 2024-04-21T08:46:33.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.869+0000 7ff73dffb640 10 client.18864 handle_command_reply: tid=0 2024-04-21T08:46:33.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.869+0000 7ff75fbbd740 1 client.18864 shutdown 2024-04-21T08:46:33.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.869+0000 7ff75fbbd740 1 -- 172.21.15.86:0/2825333664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8230978d0 msgr2=0x55f82309a0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:33.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.869+0000 7ff75fbbd740 1 --2- 172.21.15.86:0/2825333664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8230978d0 0x55f82309a0f0 secure :-1 s=READY pgs=1559 cs=0 l=1 rev1=1 crypto rx=0x55f823098030 tx=0x7ff748011a30 comp rx=0 tx=0).stop 2024-04-21T08:46:33.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.869+0000 7ff75fbbd740 1 -- 172.21.15.86:0/2825333664 shutdown_connections 2024-04-21T08:46:33.869 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.869+0000 7ff75fbbd740 1 --2- 172.21.15.86:0/2825333664 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55f823116d10 0x55f823119130 crc :-1 s=READY pgs=240 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.869+0000 7ff75fbbd740 1 --2- 172.21.15.86:0/2825333664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f8230a00e0 0x55f82309a690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.869+0000 7ff75fbbd740 1 --2- 172.21.15.86:0/2825333664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f8230978d0 0x55f82309a0f0 unknown :-1 s=CLOSED pgs=1559 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.869+0000 7ff73dffb640 0 client.18864 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.869+0000 7ff75fbbd740 1 --2- 172.21.15.86:0/2825333664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f823096b60 0x55f8230a86a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.869+0000 7ff75fbbd740 1 -- 172.21.15.86:0/2825333664 >> 172.21.15.86:0/2825333664 conn(0x55f82305dc90 msgr2=0x55f82309bea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.869+0000 7ff75fbbd740 1 -- 172.21.15.86:0/2825333664 shutdown_connections 2024-04-21T08:46:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.869+0000 7ff75fbbd740 1 -- 172.21.15.86:0/2825333664 wait complete. 2024-04-21T08:46:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.869+0000 7ff75fbbd740 20 client.18864 trim_cache size 0 max 16384 2024-04-21T08:46:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:46:33.870 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:46:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:46:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:46:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.870+0000 7ff75cef4640 1 -- 172.21.15.86:0/1204461204 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff7400a3dc0 msgr2=0x7ff7400a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:33.871 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.870+0000 7ff75cef4640 1 --2- 172.21.15.86:0/1204461204 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff7400a3dc0 0x7ff7400a6280 secure :-1 s=READY pgs=251 cs=0 l=1 rev1=1 crypto rx=0x7ff7500baf40 tx=0x7ff75008de50 comp rx=0 tx=0).stop 2024-04-21T08:46:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.870+0000 7ff75cef4640 1 -- 172.21.15.86:0/1204461204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff75819cf80 msgr2=0x7ff75819d430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.870+0000 7ff75cef4640 1 --2- 172.21.15.86:0/1204461204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff75819cf80 0x7ff75819d430 secure :-1 s=READY pgs=1555 cs=0 l=1 rev1=1 crypto rx=0x7ff750005e30 tx=0x7ff750032420 comp rx=0 tx=0).stop 2024-04-21T08:46:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.870+0000 7ff75cef4640 1 -- 172.21.15.86:0/1204461204 shutdown_connections 2024-04-21T08:46:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.870+0000 7ff75cef4640 1 --2- 172.21.15.86:0/1204461204 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff7400a3dc0 0x7ff7400a6280 unknown :-1 s=CLOSED pgs=251 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.870+0000 7ff75cef4640 1 --2- 172.21.15.86:0/1204461204 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff75819cf80 0x7ff75819d430 unknown :-1 s=CLOSED pgs=1555 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.870+0000 7ff75cef4640 1 --2- 172.21.15.86:0/1204461204 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff758106170 0x7ff758198c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.870+0000 7ff75cef4640 1 --2- 172.21.15.86:0/1204461204 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff7581010f0 0x7ff7581986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.870+0000 7ff75cef4640 1 -- 172.21.15.86:0/1204461204 >> 172.21.15.86:0/1204461204 conn(0x7ff758077fd0 msgr2=0x7ff7580698e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.870+0000 7ff75cef4640 1 -- 172.21.15.86:0/1204461204 shutdown_connections 2024-04-21T08:46:33.872 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:33.870+0000 7ff75cef4640 1 -- 172.21.15.86:0/1204461204 wait complete. 2024-04-21T08:46:34.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:33 smithi086 ceph-mon[27939]: pgmap v240: 129 pgs: 129 active+clean; 5.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 56 KiB/s wr, 23 op/s 2024-04-21T08:46:34.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:33 smithi088 ceph-mon[31259]: pgmap v240: 129 pgs: 129 active+clean; 5.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 56 KiB/s wr, 23 op/s 2024-04-21T08:46:34.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:33 smithi179 ceph-mon[31829]: pgmap v240: 129 pgs: 129 active+clean; 5.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 56 KiB/s wr, 23 op/s 2024-04-21T08:46:36.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:35 smithi086 ceph-mon[27939]: pgmap v241: 129 pgs: 129 active+clean; 5.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-04-21T08:46:36.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:35 smithi088 ceph-mon[31259]: pgmap v241: 129 pgs: 129 active+clean; 5.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-04-21T08:46:36.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:35 smithi179 ceph-mon[31829]: pgmap v241: 129 pgs: 129 active+clean; 5.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-04-21T08:46:38.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:37 smithi086 ceph-mon[27939]: pgmap v242: 129 pgs: 129 active+clean; 5.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-04-21T08:46:38.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:37 smithi088 ceph-mon[31259]: pgmap v242: 129 pgs: 129 active+clean; 5.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-04-21T08:46:38.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:37 smithi179 ceph-mon[31829]: pgmap v242: 129 pgs: 129 active+clean; 5.4 MiB data, 419 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 31 KiB/s wr, 14 op/s 2024-04-21T08:46:40.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:39 smithi088 ceph-mon[31259]: pgmap v243: 129 pgs: 129 active+clean; 1.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 55 KiB/s wr, 23 op/s 2024-04-21T08:46:40.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:39 smithi179 ceph-mon[31829]: pgmap v243: 129 pgs: 129 active+clean; 1.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 55 KiB/s wr, 23 op/s 2024-04-21T08:46:40.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:39 smithi086 ceph-mon[27939]: pgmap v243: 129 pgs: 129 active+clean; 1.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 55 KiB/s wr, 23 op/s 2024-04-21T08:46:42.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:41 smithi088 ceph-mon[31259]: pgmap v244: 129 pgs: 129 active+clean; 1.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 17 op/s 2024-04-21T08:46:42.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:41 smithi179 ceph-mon[31829]: pgmap v244: 129 pgs: 129 active+clean; 1.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 17 op/s 2024-04-21T08:46:42.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:41 smithi086 ceph-mon[27939]: pgmap v244: 129 pgs: 129 active+clean; 1.6 MiB data, 408 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 17 op/s 2024-04-21T08:46:44.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:43 smithi088 ceph-mon[31259]: pgmap v245: 129 pgs: 129 active+clean; 1.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-21T08:46:44.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:43 smithi179 ceph-mon[31829]: pgmap v245: 129 pgs: 129 active+clean; 1.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-21T08:46:44.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:43 smithi086 ceph-mon[27939]: pgmap v245: 129 pgs: 129 active+clean; 1.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-21T08:46:45.893 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.893 INFO:tasks.workunit.client.0.smithi086.stdout:313/600: open fd = 3 2024-04-21T08:46:45.893 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.893 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.893 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.893 INFO:tasks.workunit.client.0.smithi086.stdout:314/600: open fd = 3 2024-04-21T08:46:45.894 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.894 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.894 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.894 INFO:tasks.workunit.client.0.smithi086.stdout:315/600: open fd = 3 2024-04-21T08:46:45.894 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.894 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.894 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.894 INFO:tasks.workunit.client.0.smithi086.stdout:316/600: open fd = 3 2024-04-21T08:46:45.894 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.894 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.894 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.894 INFO:tasks.workunit.client.0.smithi086.stdout:317/600: open fd = 3 2024-04-21T08:46:45.894 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.894 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.894 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.895 INFO:tasks.workunit.client.0.smithi086.stdout:318/600: open fd = 3 2024-04-21T08:46:45.895 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.895 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.895 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.895 INFO:tasks.workunit.client.0.smithi086.stdout:319/600: open fd = 3 2024-04-21T08:46:45.895 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.895 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.895 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.895 INFO:tasks.workunit.client.0.smithi086.stdout:320/600: open fd = 3 2024-04-21T08:46:45.895 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.895 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.895 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.895 INFO:tasks.workunit.client.0.smithi086.stdout:321/600: open fd = 3 2024-04-21T08:46:45.895 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.896 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.896 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.896 INFO:tasks.workunit.client.0.smithi086.stdout:322/600: open fd = 3 2024-04-21T08:46:45.896 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.896 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.896 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.896 INFO:tasks.workunit.client.0.smithi086.stdout:323/600: open fd = 3 2024-04-21T08:46:45.896 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.896 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.896 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.896 INFO:tasks.workunit.client.0.smithi086.stdout:324/600: open fd = 3 2024-04-21T08:46:45.896 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.896 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.896 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.896 INFO:tasks.workunit.client.0.smithi086.stdout:325/600: open fd = 3 2024-04-21T08:46:45.896 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.896 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.897 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.897 INFO:tasks.workunit.client.0.smithi086.stdout:326/600: open fd = 3 2024-04-21T08:46:45.897 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.897 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.897 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.897 INFO:tasks.workunit.client.0.smithi086.stdout:327/600: open fd = 3 2024-04-21T08:46:45.897 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.897 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.897 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.897 INFO:tasks.workunit.client.0.smithi086.stdout:328/600: open fd = 3 2024-04-21T08:46:45.897 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.897 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.897 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.897 INFO:tasks.workunit.client.0.smithi086.stdout:329/600: open fd = 3 2024-04-21T08:46:45.897 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.897 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.898 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.898 INFO:tasks.workunit.client.0.smithi086.stdout:330/600: open fd = 3 2024-04-21T08:46:45.898 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.898 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.898 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.898 INFO:tasks.workunit.client.0.smithi086.stdout:331/600: open fd = 3 2024-04-21T08:46:45.898 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.898 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.898 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.898 INFO:tasks.workunit.client.0.smithi086.stdout:332/600: open fd = 3 2024-04-21T08:46:45.898 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.898 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.898 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.898 INFO:tasks.workunit.client.0.smithi086.stdout:333/600: open fd = 3 2024-04-21T08:46:45.898 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.899 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.899 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.899 INFO:tasks.workunit.client.0.smithi086.stdout:334/600: open fd = 3 2024-04-21T08:46:45.899 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.899 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.899 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.899 INFO:tasks.workunit.client.0.smithi086.stdout:335/600: open fd = 3 2024-04-21T08:46:45.899 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.899 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.899 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.899 INFO:tasks.workunit.client.0.smithi086.stdout:336/600: open fd = 3 2024-04-21T08:46:45.899 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.899 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.899 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.899 INFO:tasks.workunit.client.0.smithi086.stdout:337/600: open fd = 3 2024-04-21T08:46:45.899 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.900 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.900 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.900 INFO:tasks.workunit.client.0.smithi086.stdout:338/600: open fd = 3 2024-04-21T08:46:45.900 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.900 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.900 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.900 INFO:tasks.workunit.client.0.smithi086.stdout:339/600: open fd = 3 2024-04-21T08:46:45.900 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.900 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.900 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.900 INFO:tasks.workunit.client.0.smithi086.stdout:340/600: open fd = 3 2024-04-21T08:46:45.900 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.900 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.900 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.900 INFO:tasks.workunit.client.0.smithi086.stdout:341/600: open fd = 3 2024-04-21T08:46:45.901 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.901 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.901 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.901 INFO:tasks.workunit.client.0.smithi086.stdout:342/600: open fd = 3 2024-04-21T08:46:45.901 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.901 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.901 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.901 INFO:tasks.workunit.client.0.smithi086.stdout:343/600: open fd = 3 2024-04-21T08:46:45.901 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.901 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.901 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.901 INFO:tasks.workunit.client.0.smithi086.stdout:344/600: open fd = 3 2024-04-21T08:46:45.901 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.901 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.901 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.901 INFO:tasks.workunit.client.0.smithi086.stdout:345/600: open fd = 3 2024-04-21T08:46:45.902 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.902 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.902 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.902 INFO:tasks.workunit.client.0.smithi086.stdout:346/600: open fd = 3 2024-04-21T08:46:45.902 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.902 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.902 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.902 INFO:tasks.workunit.client.0.smithi086.stdout:347/600: open fd = 3 2024-04-21T08:46:45.902 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.902 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.902 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.902 INFO:tasks.workunit.client.0.smithi086.stdout:348/600: open fd = 3 2024-04-21T08:46:45.902 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.902 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.902 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.902 INFO:tasks.workunit.client.0.smithi086.stdout:349/600: open fd = 3 2024-04-21T08:46:45.903 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.903 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.903 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.903 INFO:tasks.workunit.client.0.smithi086.stdout:350/600: open fd = 3 2024-04-21T08:46:45.903 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.903 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.903 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.903 INFO:tasks.workunit.client.0.smithi086.stdout:351/600: open fd = 3 2024-04-21T08:46:45.903 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.903 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.903 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.903 INFO:tasks.workunit.client.0.smithi086.stdout:352/600: open fd = 3 2024-04-21T08:46:45.903 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.903 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.903 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.903 INFO:tasks.workunit.client.0.smithi086.stdout:353/600: open fd = 3 2024-04-21T08:46:45.903 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.904 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.904 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.904 INFO:tasks.workunit.client.0.smithi086.stdout:354/600: open fd = 3 2024-04-21T08:46:45.904 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.904 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.904 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.904 INFO:tasks.workunit.client.0.smithi086.stdout:355/600: open fd = 3 2024-04-21T08:46:45.904 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.904 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.904 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.904 INFO:tasks.workunit.client.0.smithi086.stdout:356/600: open fd = 3 2024-04-21T08:46:45.904 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.904 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.904 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.905 INFO:tasks.workunit.client.0.smithi086.stdout:357/600: open fd = 3 2024-04-21T08:46:45.905 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.905 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.905 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.905 INFO:tasks.workunit.client.0.smithi086.stdout:358/600: open fd = 3 2024-04-21T08:46:45.905 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.905 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.905 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.905 INFO:tasks.workunit.client.0.smithi086.stdout:359/600: open fd = 3 2024-04-21T08:46:45.905 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.905 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.905 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.905 INFO:tasks.workunit.client.0.smithi086.stdout:360/600: open fd = 3 2024-04-21T08:46:45.905 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.905 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.905 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.906 INFO:tasks.workunit.client.0.smithi086.stdout:361/600: open fd = 3 2024-04-21T08:46:45.906 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.906 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.906 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.906 INFO:tasks.workunit.client.0.smithi086.stdout:362/600: open fd = 3 2024-04-21T08:46:45.906 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.906 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.906 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.906 INFO:tasks.workunit.client.0.smithi086.stdout:363/600: open fd = 3 2024-04-21T08:46:45.906 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.906 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.906 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.906 INFO:tasks.workunit.client.0.smithi086.stdout:364/600: open fd = 3 2024-04-21T08:46:45.906 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.907 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.907 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.907 INFO:tasks.workunit.client.0.smithi086.stdout:365/600: open fd = 3 2024-04-21T08:46:45.907 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.907 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.907 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.907 INFO:tasks.workunit.client.0.smithi086.stdout:366/600: open fd = 3 2024-04-21T08:46:45.907 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.907 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.907 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.907 INFO:tasks.workunit.client.0.smithi086.stdout:367/600: open fd = 3 2024-04-21T08:46:45.907 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.907 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.907 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.907 INFO:tasks.workunit.client.0.smithi086.stdout:368/600: open fd = 3 2024-04-21T08:46:45.907 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.908 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.908 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.908 INFO:tasks.workunit.client.0.smithi086.stdout:369/600: open fd = 3 2024-04-21T08:46:45.908 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.908 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.908 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.908 INFO:tasks.workunit.client.0.smithi086.stdout:370/600: open fd = 3 2024-04-21T08:46:45.908 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.908 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.908 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.908 INFO:tasks.workunit.client.0.smithi086.stdout:371/600: open fd = 3 2024-04-21T08:46:45.908 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.908 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.908 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.908 INFO:tasks.workunit.client.0.smithi086.stdout:372/600: open fd = 3 2024-04-21T08:46:45.908 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.908 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.909 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.909 INFO:tasks.workunit.client.0.smithi086.stdout:373/600: open fd = 3 2024-04-21T08:46:45.909 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.909 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.909 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:45.909 INFO:tasks.workunit.client.0.smithi086.stdout:374/600: open fd = 3 2024-04-21T08:46:45.909 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.909 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:46:45.909 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:46:46.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:46 smithi086 ceph-mon[27939]: pgmap v246: 129 pgs: 129 active+clean; 1.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:46:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.963+0000 7f712ac14640 1 -- 172.21.15.86:0/3091033273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f712410c520 msgr2=0x7f712410e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.963+0000 7f712ac14640 1 --2- 172.21.15.86:0/3091033273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f712410c520 0x7f712410e9c0 secure :-1 s=READY pgs=1560 cs=0 l=1 rev1=1 crypto rx=0x7f711c009ca0 tx=0x7f711c02c230 comp rx=0 tx=0).stop 2024-04-21T08:46:46.965 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.963+0000 7f712ac14640 1 -- 172.21.15.86:0/3091033273 shutdown_connections 2024-04-21T08:46:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.963+0000 7f712ac14640 1 --2- 172.21.15.86:0/3091033273 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f712410c520 0x7f712410e9c0 unknown :-1 s=CLOSED pgs=1560 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.963+0000 7f712ac14640 1 --2- 172.21.15.86:0/3091033273 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7124103b00 0x7f712410bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.963+0000 7f712ac14640 1 --2- 172.21.15.86:0/3091033273 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f71241031e0 0x7f71241035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.963+0000 7f712ac14640 1 -- 172.21.15.86:0/3091033273 >> 172.21.15.86:0/3091033273 conn(0x7f7124077fd0 msgr2=0x7f71240783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.963+0000 7f712ac14640 1 -- 172.21.15.86:0/3091033273 shutdown_connections 2024-04-21T08:46:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.963+0000 7f712ac14640 1 -- 172.21.15.86:0/3091033273 wait complete. 2024-04-21T08:46:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.963+0000 7f712ac14640 1 Processor -- start 2024-04-21T08:46:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.963+0000 7f712ac14640 1 -- start start 2024-04-21T08:46:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.964+0000 7f712ac14640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71241031e0 0x7f71241986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.964+0000 7f712ac14640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7124103b00 0x7f7124198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.964+0000 7f712ac14640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f712410c520 0x7f712419cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:46.966 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.964+0000 7f712ac14640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f7124112190 con 0x7f712410c520 2024-04-21T08:46:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.964+0000 7f712ac14640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7124112010 con 0x7f71241031e0 2024-04-21T08:46:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.964+0000 7f712ac14640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7124112310 con 0x7f7124103b00 2024-04-21T08:46:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.964+0000 7f71291b1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f712410c520 0x7f712419cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.964+0000 7f71291b1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f712410c520 0x7f712419cf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58278/0 (socket says 172.21.15.86:58278) 2024-04-21T08:46:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.964+0000 7f71291b1640 1 -- 172.21.15.86:0/1394838855 learned_addr learned my addr 172.21.15.86:0/1394838855 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.964+0000 7f71289b0640 1 --2- 172.21.15.86:0/1394838855 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71241031e0 0x7f71241986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.964+0000 7f7123fff640 1 --2- 172.21.15.86:0/1394838855 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7124103b00 0x7f7124198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.964+0000 7f71291b1640 1 -- 172.21.15.86:0/1394838855 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7124103b00 msgr2=0x7f7124198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.964+0000 7f71291b1640 1 --2- 172.21.15.86:0/1394838855 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7124103b00 0x7f7124198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.964+0000 7f71291b1640 1 -- 172.21.15.86:0/1394838855 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71241031e0 msgr2=0x7f71241986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.964+0000 7f71291b1640 1 --2- 172.21.15.86:0/1394838855 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71241031e0 0x7f71241986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.964+0000 7f71291b1640 1 -- 172.21.15.86:0/1394838855 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f712419d670 con 0x7f712410c520 2024-04-21T08:46:46.967 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.964+0000 7f71291b1640 1 --2- 172.21.15.86:0/1394838855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f712410c520 0x7f712419cf40 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f711c02c840 tx=0x7f711c003c20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.964+0000 7f7121ffb640 1 -- 172.21.15.86:0/1394838855 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c039040 con 0x7f712410c520 2024-04-21T08:46:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.964+0000 7f7121ffb640 1 -- 172.21.15.86:0/1394838855 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f711c044070 con 0x7f712410c520 2024-04-21T08:46:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.964+0000 7f712ac14640 1 -- 172.21.15.86:0/1394838855 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f712419d900 con 0x7f712410c520 2024-04-21T08:46:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.964+0000 7f7121ffb640 1 -- 172.21.15.86:0/1394838855 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c0329c0 con 0x7f712410c520 2024-04-21T08:46:46.968 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.965+0000 7f712ac14640 1 -- 172.21.15.86:0/1394838855 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71241a51f0 con 0x7f712410c520 2024-04-21T08:46:46.968 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:46 smithi088 ceph-mon[31259]: pgmap v246: 129 pgs: 129 active+clean; 1.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:46:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.967+0000 7f7121ffb640 1 -- 172.21.15.86:0/1394838855 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f711c032b60 con 0x7f712410c520 2024-04-21T08:46:46.974 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.967+0000 7f712d80a740 1 Processor -- start 2024-04-21T08:46:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.967+0000 7f712d80a740 1 -- start start 2024-04-21T08:46:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.967+0000 7f7121ffb640 1 --2- 172.21.15.86:0/1394838855 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f71140a3dc0 0x7f71140a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.967+0000 7f712d80a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad6ac098d0 0x55ad6ac08620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.967+0000 7f712d80a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad6ac08b60 0x55ad6ac08fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.967+0000 7f712d80a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ad6ac120e0 0x55ad6ac144d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.967+0000 7f712d80a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ad6ab6d8d0 con 0x55ad6ac08b60 2024-04-21T08:46:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.967+0000 7f712d80a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55ad6ab0ae80 con 0x55ad6ac120e0 2024-04-21T08:46:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.967+0000 7f712d80a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ad6ab69890 con 0x55ad6ac098d0 2024-04-21T08:46:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.967+0000 7f7123fff640 1 --2- 172.21.15.86:0/1394838855 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f71140a3dc0 0x7f71140a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.967+0000 7f7121ffb640 1 -- 172.21.15.86:0/1394838855 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f711c0f2fe0 con 0x7f712410c520 2024-04-21T08:46:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.967+0000 7f7123fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad6ac08b60 0x55ad6ac08fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.968+0000 7f7123fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad6ac08b60 0x55ad6ac08fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58286/0 (socket says 172.21.15.86:58286) 2024-04-21T08:46:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.968+0000 7f7123fff640 1 -- 172.21.15.86:0/3564066464 learned_addr learned my addr 172.21.15.86:0/3564066464 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.968+0000 7f71289b0640 1 --2- 172.21.15.86:0/3564066464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad6ac098d0 0x55ad6ac08620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:46.975 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.968+0000 7f71291b1640 1 --2- 172.21.15.86:0/3564066464 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ad6ac120e0 0x55ad6ac144d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.968+0000 7f7123fff640 1 -- 172.21.15.86:0/3564066464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad6ac098d0 msgr2=0x55ad6ac08620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.968+0000 7f7123fff640 1 --2- 172.21.15.86:0/3564066464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad6ac098d0 0x55ad6ac08620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.968+0000 7f7123fff640 1 -- 172.21.15.86:0/3564066464 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ad6ac120e0 msgr2=0x55ad6ac144d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.968+0000 7f7123fff640 1 --2- 172.21.15.86:0/3564066464 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ad6ac120e0 0x55ad6ac144d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.968+0000 7f7123fff640 1 -- 172.21.15.86:0/3564066464 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad6ab2c320 con 0x55ad6ac08b60 2024-04-21T08:46:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.968+0000 7f7123fff640 1 --2- 172.21.15.86:0/1394838855 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f71140a3dc0 0x7f71140a6280 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7110014980 tx=0x7f711000cb50 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.968+0000 7f7123fff640 1 --2- 172.21.15.86:0/3564066464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad6ac08b60 0x55ad6ac08fe0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f7110011c20 tx=0x7f711001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e8c8d9f2119996de server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.968+0000 7f710effd640 1 -- 172.21.15.86:0/3564066464 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7110014110 con 0x55ad6ac08b60 2024-04-21T08:46:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.968+0000 7f710effd640 1 -- 172.21.15.86:0/3564066464 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7110024070 con 0x55ad6ac08b60 2024-04-21T08:46:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.968+0000 7f710effd640 1 -- 172.21.15.86:0/3564066464 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71100035f0 con 0x55ad6ac08b60 2024-04-21T08:46:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.969+0000 7f712d80a740 1 -- 172.21.15.86:0/3564066464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad6ac08b60 msgr2=0x55ad6ac08fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.969+0000 7f712d80a740 1 --2- 172.21.15.86:0/3564066464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad6ac08b60 0x55ad6ac08fe0 secure :-1 s=READY pgs=1562 cs=0 l=1 rev1=1 crypto rx=0x7f7110011c20 tx=0x7f711001b000 comp rx=0 tx=0).stop 2024-04-21T08:46:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.969+0000 7f712d80a740 1 -- 172.21.15.86:0/3564066464 shutdown_connections 2024-04-21T08:46:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.969+0000 7f712d80a740 1 --2- 172.21.15.86:0/3564066464 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ad6ac120e0 0x55ad6ac144d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:46.976 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.969+0000 7f712d80a740 1 --2- 172.21.15.86:0/3564066464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad6ac08b60 0x55ad6ac08fe0 unknown :-1 s=CLOSED pgs=1562 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.969+0000 7f712d80a740 1 --2- 172.21.15.86:0/3564066464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad6ac098d0 0x55ad6ac08620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.969+0000 7f712d80a740 1 -- 172.21.15.86:0/3564066464 >> 172.21.15.86:0/3564066464 conn(0x55ad6abcfc90 msgr2=0x55ad6ac03850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.969+0000 7f712d80a740 1 -- 172.21.15.86:0/3564066464 shutdown_connections 2024-04-21T08:46:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.969+0000 7f712d80a740 1 -- 172.21.15.86:0/3564066464 wait complete. 2024-04-21T08:46:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.969+0000 7f712d80a740 1 -- start start 2024-04-21T08:46:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.969+0000 7f712d80a740 1 Processor -- start 2024-04-21T08:46:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.970+0000 7f712d80a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ad6ac08b60 0x55ad6ac11a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.970+0000 7f712d80a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad6ac098d0 0x55ad6ac0edb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.970+0000 7f712d80a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad6ac120e0 0x55ad6ac0f2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.970+0000 7f712d80a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ad6ac14db0 con 0x55ad6ac120e0 2024-04-21T08:46:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.970+0000 7f712d80a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55ad6aaf5eb0 con 0x55ad6ac08b60 2024-04-21T08:46:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.970+0000 7f712d80a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ad6ac14f30 con 0x55ad6ac098d0 2024-04-21T08:46:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.970+0000 7f712d80a740 10 client.0 _ll_register_callbacks cb 0x55ad6ab22b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.970+0000 7f71291b1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad6ac120e0 0x55ad6ac0f2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.970+0000 7f71291b1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad6ac120e0 0x55ad6ac0f2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58294/0 (socket says 172.21.15.86:58294) 2024-04-21T08:46:46.977 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.970+0000 7f71291b1640 1 -- 172.21.15.86:0/1119167460 learned_addr learned my addr 172.21.15.86:0/1119167460 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.970+0000 7f71289b0640 1 --2- 172.21.15.86:0/1119167460 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ad6ac08b60 0x55ad6ac11a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.970+0000 7f71291b1640 1 -- 172.21.15.86:0/1119167460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad6ac098d0 msgr2=0x55ad6ac0edb0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:46:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.970+0000 7f71291b1640 1 --2- 172.21.15.86:0/1119167460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad6ac098d0 0x55ad6ac0edb0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.970+0000 7f71291b1640 1 -- 172.21.15.86:0/1119167460 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ad6ac08b60 msgr2=0x55ad6ac11a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.970+0000 7f71291b1640 1 --2- 172.21.15.86:0/1119167460 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ad6ac08b60 0x55ad6ac11a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.970+0000 7f71291b1640 1 -- 172.21.15.86:0/1119167460 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad6a880210 con 0x55ad6ac120e0 2024-04-21T08:46:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.970+0000 7f71291b1640 1 --2- 172.21.15.86:0/1119167460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad6ac120e0 0x55ad6ac0f2f0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f711c005e30 tx=0x7f711c0be6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a36b808b3fdcfdff server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.970+0000 7f7103fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:46:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.970+0000 7f7103fff640 1 -- 172.21.15.86:0/1119167460 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c0df2e0 con 0x55ad6ac120e0 2024-04-21T08:46:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.970+0000 7f7103fff640 1 -- 172.21.15.86:0/1119167460 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f711c0bef20 con 0x55ad6ac120e0 2024-04-21T08:46:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.970+0000 7f712d80a740 1 -- 172.21.15.86:0/1119167460 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7110024070 con 0x55ad6ac120e0 2024-04-21T08:46:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.971+0000 7f7103fff640 1 -- 172.21.15.86:0/1119167460 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f711c0becc0 con 0x55ad6ac120e0 2024-04-21T08:46:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.971+0000 7f712d80a740 10 client.18876 fetch_fsmap learned FSMap version 29 2024-04-21T08:46:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.971+0000 7f712d80a740 1 -- 172.21.15.86:0/1119167460 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ad6a94c620 con 0x55ad6ac120e0 2024-04-21T08:46:46.978 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.971+0000 7f7103fff640 1 -- 172.21.15.86:0/1119167460 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7110024070 con 0x55ad6ac120e0 2024-04-21T08:46:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.971+0000 7f7103fff640 1 -- 172.21.15.86:0/1119167460 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f711c03c030 con 0x55ad6ac120e0 2024-04-21T08:46:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.971+0000 7f712d80a740 10 client.18876 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:46:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.971+0000 7f712d80a740 10 client.18876 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.971+0000 7f712d80a740 20 client.18876 populate_metadata read hostname 'smithi086' 2024-04-21T08:46:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.971+0000 7f712d80a740 1 --2- 172.21.15.86:0/1119167460 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ad6ac22030 0x55ad6ac244a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.971+0000 7f712d80a740 4 client.18876 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.971+0000 7f712d80a740 1 -- 172.21.15.86:0/1119167460 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ad6ab81880 con 0x55ad6ac22030 2024-04-21T08:46:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.971+0000 7f71289b0640 1 --2- 172.21.15.86:0/1119167460 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ad6ac22030 0x55ad6ac244a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.972+0000 7f7103fff640 10 client.18876 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.972+0000 7f71289b0640 1 --2- 172.21.15.86:0/1119167460 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ad6ac22030 0x55ad6ac244a0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcc7be3b4818c1e server_cookie=8dd9f7a10abb5496 in_seq=0 out_seq=0 2024-04-21T08:46:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.972+0000 7f7103fff640 1 -- 172.21.15.86:0/1119167460 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ad6ab81880 con 0x55ad6ac22030 2024-04-21T08:46:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.972+0000 7f7103fff640 10 client.18876 handle_command_reply: tid=0 2024-04-21T08:46:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.972+0000 7f712d80a740 1 client.18876 shutdown 2024-04-21T08:46:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.972+0000 7f712d80a740 1 -- 172.21.15.86:0/1119167460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad6ac120e0 msgr2=0x55ad6ac0f2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:46.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.972+0000 7f712d80a740 1 --2- 172.21.15.86:0/1119167460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad6ac120e0 0x55ad6ac0f2f0 secure :-1 s=READY pgs=1563 cs=0 l=1 rev1=1 crypto rx=0x7f711c005e30 tx=0x7f711c0be6e0 comp rx=0 tx=0).stop 2024-04-21T08:46:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.972+0000 7f712d80a740 1 -- 172.21.15.86:0/1119167460 shutdown_connections 2024-04-21T08:46:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.972+0000 7f712d80a740 1 --2- 172.21.15.86:0/1119167460 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ad6ac22030 0x55ad6ac244a0 crc :-1 s=READY pgs=241 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.972+0000 7f712d80a740 1 --2- 172.21.15.86:0/1119167460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad6ac120e0 0x55ad6ac0f2f0 unknown :-1 s=CLOSED pgs=1563 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.972+0000 7f712d80a740 1 --2- 172.21.15.86:0/1119167460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad6ac098d0 0x55ad6ac0edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.972+0000 7f712d80a740 1 --2- 172.21.15.86:0/1119167460 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ad6ac08b60 0x55ad6ac11a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.972+0000 7f7103fff640 0 client.18876 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.973+0000 7f712d80a740 1 -- 172.21.15.86:0/1119167460 >> 172.21.15.86:0/1119167460 conn(0x55ad6abcfc90 msgr2=0x55ad6ac055a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.973+0000 7f712d80a740 1 -- 172.21.15.86:0/1119167460 shutdown_connections 2024-04-21T08:46:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.973+0000 7f712d80a740 1 -- 172.21.15.86:0/1119167460 wait complete. 2024-04-21T08:46:46.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.973+0000 7f712d80a740 20 client.18876 trim_cache size 0 max 16384 2024-04-21T08:46:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.987+0000 7f712d80a740 1 Processor -- start 2024-04-21T08:46:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.987+0000 7f712d80a740 1 -- start start 2024-04-21T08:46:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.987+0000 7f712d80a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad6ac08b60 0x55ad6ac1dd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.987+0000 7f712d80a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ad6ac098d0 0x55ad6ac114e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.987+0000 7f712d80a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad6ac120e0 0x55ad6ac11a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.987+0000 7f712d80a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f71241af2a0 con 0x55ad6ac120e0 2024-04-21T08:46:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.987+0000 7f712d80a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f71241b0de0 con 0x55ad6ac098d0 2024-04-21T08:46:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.987+0000 7f712d80a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f71241af970 con 0x55ad6ac08b60 2024-04-21T08:46:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.988+0000 7f71291b1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad6ac120e0 0x55ad6ac11a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.988+0000 7f71291b1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad6ac120e0 0x55ad6ac11a20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58302/0 (socket says 172.21.15.86:58302) 2024-04-21T08:46:46.992 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.988+0000 7f71291b1640 1 -- 172.21.15.86:0/3634157200 learned_addr learned my addr 172.21.15.86:0/3634157200 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.988+0000 7f71289b0640 1 --2- 172.21.15.86:0/3634157200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad6ac08b60 0x55ad6ac1dd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.988+0000 7f7123fff640 1 --2- 172.21.15.86:0/3634157200 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ad6ac098d0 0x55ad6ac114e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.988+0000 7f71291b1640 1 -- 172.21.15.86:0/3634157200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad6ac08b60 msgr2=0x55ad6ac1dd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.988+0000 7f71291b1640 1 --2- 172.21.15.86:0/3634157200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad6ac08b60 0x55ad6ac1dd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.988+0000 7f71291b1640 1 -- 172.21.15.86:0/3634157200 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ad6ac098d0 msgr2=0x55ad6ac114e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.988+0000 7f71291b1640 1 --2- 172.21.15.86:0/3634157200 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ad6ac098d0 0x55ad6ac114e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.988+0000 7f71291b1640 1 -- 172.21.15.86:0/3634157200 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ad6ac14840 con 0x55ad6ac120e0 2024-04-21T08:46:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.988+0000 7f71289b0640 1 --2- 172.21.15.86:0/3634157200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad6ac08b60 0x55ad6ac1dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.988+0000 7f71291b1640 1 --2- 172.21.15.86:0/3634157200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad6ac120e0 0x55ad6ac11a20 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f711c009dc0 tx=0x7f711c0be750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2987a33c69bf1b3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.988+0000 7f710e7fc640 1 -- 172.21.15.86:0/3634157200 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c0be800 con 0x55ad6ac120e0 2024-04-21T08:46:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.988+0000 7f710e7fc640 1 -- 172.21.15.86:0/3634157200 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f711c0c3070 con 0x55ad6ac120e0 2024-04-21T08:46:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.988+0000 7f710e7fc640 1 -- 172.21.15.86:0/3634157200 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c0bef20 con 0x55ad6ac120e0 2024-04-21T08:46:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.989+0000 7f712d80a740 1 -- 172.21.15.86:0/3634157200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad6ac120e0 msgr2=0x55ad6ac11a20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.989+0000 7f712d80a740 1 --2- 172.21.15.86:0/3634157200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad6ac120e0 0x55ad6ac11a20 secure :-1 s=READY pgs=1564 cs=0 l=1 rev1=1 crypto rx=0x7f711c009dc0 tx=0x7f711c0be750 comp rx=0 tx=0).stop 2024-04-21T08:46:46.993 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.989+0000 7f712d80a740 1 -- 172.21.15.86:0/3634157200 shutdown_connections 2024-04-21T08:46:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.989+0000 7f712d80a740 1 --2- 172.21.15.86:0/3634157200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad6ac120e0 0x55ad6ac11a20 unknown :-1 s=CLOSED pgs=1564 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.989+0000 7f712d80a740 1 --2- 172.21.15.86:0/3634157200 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ad6ac098d0 0x55ad6ac114e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.989+0000 7f712d80a740 1 --2- 172.21.15.86:0/3634157200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad6ac08b60 0x55ad6ac1dd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.989+0000 7f712d80a740 1 -- 172.21.15.86:0/3634157200 >> 172.21.15.86:0/3634157200 conn(0x55ad6abcfc90 msgr2=0x55ad6ac0a060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.989+0000 7f712d80a740 1 -- 172.21.15.86:0/3634157200 shutdown_connections 2024-04-21T08:46:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.989+0000 7f712d80a740 1 -- 172.21.15.86:0/3634157200 wait complete. 2024-04-21T08:46:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.989+0000 7f712d80a740 1 -- start start 2024-04-21T08:46:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.989+0000 7f712d80a740 1 Processor -- start 2024-04-21T08:46:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.989+0000 7f712d80a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad6ac08b60 0x55ad6ac0bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.990+0000 7f712d80a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ad6ac098d0 0x55ad6ac0c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.990+0000 7f712d80a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad6ac120e0 0x55ad6ac875f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.990+0000 7f712d80a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f71241b0710 con 0x55ad6ac120e0 2024-04-21T08:46:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.990+0000 7f712d80a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f71241b0040 con 0x55ad6ac098d0 2024-04-21T08:46:46.994 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.990+0000 7f712d80a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f71241ad2a0 con 0x55ad6ac08b60 2024-04-21T08:46:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.990+0000 7f712d80a740 10 client.0 _ll_register_callbacks cb 0x7f711c003f30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.990+0000 7f71291b1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad6ac120e0 0x55ad6ac875f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.990+0000 7f71291b1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad6ac120e0 0x55ad6ac875f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:58316/0 (socket says 172.21.15.86:58316) 2024-04-21T08:46:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.990+0000 7f71291b1640 1 -- 172.21.15.86:0/2079960822 learned_addr learned my addr 172.21.15.86:0/2079960822 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.990+0000 7f7123fff640 1 --2- 172.21.15.86:0/2079960822 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ad6ac098d0 0x55ad6ac0c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.990+0000 7f71289b0640 1 --2- 172.21.15.86:0/2079960822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad6ac08b60 0x55ad6ac0bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.990+0000 7f71291b1640 1 -- 172.21.15.86:0/2079960822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad6ac08b60 msgr2=0x55ad6ac0bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.990+0000 7f71291b1640 1 --2- 172.21.15.86:0/2079960822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad6ac08b60 0x55ad6ac0bdc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.990+0000 7f71291b1640 1 -- 172.21.15.86:0/2079960822 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ad6ac098d0 msgr2=0x55ad6ac0c300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.990+0000 7f71291b1640 1 --2- 172.21.15.86:0/2079960822 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ad6ac098d0 0x55ad6ac0c300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.990+0000 7f71291b1640 1 -- 172.21.15.86:0/2079960822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ad6ac0cbd0 con 0x55ad6ac120e0 2024-04-21T08:46:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.990+0000 7f7123fff640 1 --2- 172.21.15.86:0/2079960822 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ad6ac098d0 0x55ad6ac0c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:46.995 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.990+0000 7f71291b1640 1 --2- 172.21.15.86:0/2079960822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad6ac120e0 0x55ad6ac875f0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f711c005e30 tx=0x7f711c004820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2cf93eef8b417129 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.990+0000 7f7103fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:46:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.990+0000 7f7103fff640 1 -- 172.21.15.86:0/2079960822 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f711c0be800 con 0x55ad6ac120e0 2024-04-21T08:46:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.990+0000 7f712d80a740 1 -- 172.21.15.86:0/2079960822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f711c0c3070 con 0x55ad6ac120e0 2024-04-21T08:46:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.990+0000 7f7103fff640 1 -- 172.21.15.86:0/2079960822 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f711c004b40 con 0x55ad6ac120e0 2024-04-21T08:46:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.990+0000 7f7103fff640 1 -- 172.21.15.86:0/2079960822 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f711c087070 con 0x55ad6ac120e0 2024-04-21T08:46:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.990+0000 7f712d80a740 10 client.18882 fetch_fsmap learned FSMap version 29 2024-04-21T08:46:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.990+0000 7f712d80a740 1 -- 172.21.15.86:0/2079960822 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ad6ac0cbd0 con 0x55ad6ac120e0 2024-04-21T08:46:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.991+0000 7f7103fff640 1 -- 172.21.15.86:0/2079960822 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f711c0c3070 con 0x55ad6ac120e0 2024-04-21T08:46:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.991+0000 7f7103fff640 1 -- 172.21.15.86:0/2079960822 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f711c09db50 con 0x55ad6ac120e0 2024-04-21T08:46:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.991+0000 7f712d80a740 10 client.18882 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:46:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.991+0000 7f712d80a740 10 client.18882 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.991+0000 7f712d80a740 20 client.18882 populate_metadata read hostname 'smithi086' 2024-04-21T08:46:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.991+0000 7f712d80a740 1 --2- 172.21.15.86:0/2079960822 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ad6ac88dd0 0x55ad6ac8b1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.991+0000 7f712d80a740 4 client.18882 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:46:46.996 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.991+0000 7f712d80a740 1 -- 172.21.15.86:0/2079960822 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ad6ab0d3d0 con 0x55ad6ac88dd0 2024-04-21T08:46:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.991+0000 7f71289b0640 1 --2- 172.21.15.86:0/2079960822 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ad6ac88dd0 0x55ad6ac8b1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.992+0000 7f71289b0640 1 --2- 172.21.15.86:0/2079960822 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ad6ac88dd0 0x55ad6ac8b1c0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=237e3cdcdba314f1 server_cookie=11ebf7596dc8a068 in_seq=0 out_seq=0 2024-04-21T08:46:46.997 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:46.992+0000 7f7103fff640 10 client.18882 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:46.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:46 smithi179 ceph-mon[31829]: pgmap v246: 129 pgs: 129 active+clean; 1.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:46:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:47.004+0000 7f7103fff640 1 -- 172.21.15.86:0/2079960822 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ad6ab0d3d0 con 0x55ad6ac88dd0 2024-04-21T08:46:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:47.004+0000 7f7103fff640 10 client.18882 handle_command_reply: tid=0 2024-04-21T08:46:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:47.004+0000 7f712d80a740 1 client.18882 shutdown 2024-04-21T08:46:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:47.004+0000 7f712d80a740 1 -- 172.21.15.86:0/2079960822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad6ac120e0 msgr2=0x55ad6ac875f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:47.004+0000 7f712d80a740 1 --2- 172.21.15.86:0/2079960822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad6ac120e0 0x55ad6ac875f0 secure :-1 s=READY pgs=1565 cs=0 l=1 rev1=1 crypto rx=0x7f711c005e30 tx=0x7f711c004820 comp rx=0 tx=0).stop 2024-04-21T08:46:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:47.004+0000 7f712d80a740 1 -- 172.21.15.86:0/2079960822 shutdown_connections 2024-04-21T08:46:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:47.004+0000 7f712d80a740 1 --2- 172.21.15.86:0/2079960822 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ad6ac88dd0 0x55ad6ac8b1c0 crc :-1 s=READY pgs=242 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:47.004+0000 7f712d80a740 1 --2- 172.21.15.86:0/2079960822 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ad6ac120e0 0x55ad6ac875f0 unknown :-1 s=CLOSED pgs=1565 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:47.004+0000 7f712d80a740 1 --2- 172.21.15.86:0/2079960822 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ad6ac098d0 0x55ad6ac0c300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:47.004+0000 7f7103fff640 0 client.18882 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:47.004+0000 7f712d80a740 1 --2- 172.21.15.86:0/2079960822 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ad6ac08b60 0x55ad6ac0bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:47.004+0000 7f712d80a740 1 -- 172.21.15.86:0/2079960822 >> 172.21.15.86:0/2079960822 conn(0x55ad6abcfc90 msgr2=0x55ad6ac04c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:47.005 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:47.004+0000 7f712d80a740 1 -- 172.21.15.86:0/2079960822 shutdown_connections 2024-04-21T08:46:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:47.004+0000 7f712d80a740 1 -- 172.21.15.86:0/2079960822 wait complete. 2024-04-21T08:46:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:47.004+0000 7f712d80a740 20 client.18882 trim_cache size 0 max 16384 2024-04-21T08:46:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:46:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:46:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:46:47.006 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:46:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:47.005+0000 7f712ac14640 1 -- 172.21.15.86:0/1394838855 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f71140a3dc0 msgr2=0x7f71140a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:47.005+0000 7f712ac14640 1 --2- 172.21.15.86:0/1394838855 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f71140a3dc0 0x7f71140a6280 secure :-1 s=READY pgs=252 cs=0 l=1 rev1=1 crypto rx=0x7f7110014980 tx=0x7f711000cb50 comp rx=0 tx=0).stop 2024-04-21T08:46:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:47.005+0000 7f712ac14640 1 -- 172.21.15.86:0/1394838855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f712410c520 msgr2=0x7f712419cf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:47.005+0000 7f712ac14640 1 --2- 172.21.15.86:0/1394838855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f712410c520 0x7f712419cf40 secure :-1 s=READY pgs=1561 cs=0 l=1 rev1=1 crypto rx=0x7f711c02c840 tx=0x7f711c003c20 comp rx=0 tx=0).stop 2024-04-21T08:46:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:47.005+0000 7f712ac14640 1 -- 172.21.15.86:0/1394838855 shutdown_connections 2024-04-21T08:46:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:47.005+0000 7f712ac14640 1 --2- 172.21.15.86:0/1394838855 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f71140a3dc0 0x7f71140a6280 unknown :-1 s=CLOSED pgs=252 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:47.007 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:47.005+0000 7f712ac14640 1 --2- 172.21.15.86:0/1394838855 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f712410c520 0x7f712419cf40 unknown :-1 s=CLOSED pgs=1561 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:47.005+0000 7f712ac14640 1 --2- 172.21.15.86:0/1394838855 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7124103b00 0x7f7124198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:47.005+0000 7f712ac14640 1 --2- 172.21.15.86:0/1394838855 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f71241031e0 0x7f71241986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:47.005+0000 7f712ac14640 1 -- 172.21.15.86:0/1394838855 >> 172.21.15.86:0/1394838855 conn(0x7f7124077fd0 msgr2=0x7f71240fdf20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:47.005+0000 7f712ac14640 1 -- 172.21.15.86:0/1394838855 shutdown_connections 2024-04-21T08:46:47.008 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:46:47.005+0000 7f712ac14640 1 -- 172.21.15.86:0/1394838855 wait complete. 2024-04-21T08:46:48.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:48 smithi086 ceph-mon[27939]: pgmap v247: 129 pgs: 129 active+clean; 1.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:46:48.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:48 smithi088 ceph-mon[31259]: pgmap v247: 129 pgs: 129 active+clean; 1.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:46:48.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:48 smithi179 ceph-mon[31829]: pgmap v247: 129 pgs: 129 active+clean; 1.8 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:46:50.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:50 smithi086 ceph-mon[27939]: pgmap v248: 129 pgs: 129 active+clean; 2.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 65 KiB/s wr, 24 op/s 2024-04-21T08:46:50.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:50 smithi088 ceph-mon[31259]: pgmap v248: 129 pgs: 129 active+clean; 2.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 65 KiB/s wr, 24 op/s 2024-04-21T08:46:50.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:50 smithi179 ceph-mon[31829]: pgmap v248: 129 pgs: 129 active+clean; 2.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 65 KiB/s wr, 24 op/s 2024-04-21T08:46:52.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:52 smithi086 ceph-mon[27939]: pgmap v249: 129 pgs: 129 active+clean; 2.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:46:52.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:52 smithi088 ceph-mon[31259]: pgmap v249: 129 pgs: 129 active+clean; 2.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:46:52.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:52 smithi179 ceph-mon[31829]: pgmap v249: 129 pgs: 129 active+clean; 2.1 MiB data, 409 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:46:53.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:53 smithi088 ceph-mon[31259]: pgmap v250: 129 pgs: 129 active+clean; 2.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-21T08:46:53.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:53 smithi179 ceph-mon[31829]: pgmap v250: 129 pgs: 129 active+clean; 2.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-21T08:46:54.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:53 smithi086 ceph-mon[27939]: pgmap v250: 129 pgs: 129 active+clean; 2.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 57 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-21T08:46:56.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:55 smithi086 ceph-mon[27939]: pgmap v251: 129 pgs: 129 active+clean; 2.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-21T08:46:56.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:55 smithi088 ceph-mon[31259]: pgmap v251: 129 pgs: 129 active+clean; 2.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-21T08:46:56.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:55 smithi179 ceph-mon[31829]: pgmap v251: 129 pgs: 129 active+clean; 2.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-21T08:46:58.013 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:46:58.016 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:46:58.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:57 smithi086 ceph-mon[27939]: pgmap v252: 129 pgs: 129 active+clean; 2.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-21T08:46:58.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.123+0000 7f710f52b640 1 -- 172.21.15.86:0/982484308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7108106150 msgr2=0x7f710810dd20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.123+0000 7f710f52b640 1 --2- 172.21.15.86:0/982484308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7108106150 0x7f710810dd20 secure :-1 s=READY pgs=1566 cs=0 l=1 rev1=1 crypto rx=0x7f7100009ca0 tx=0x7f710002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.123+0000 7f710f52b640 1 -- 172.21.15.86:0/982484308 shutdown_connections 2024-04-21T08:46:58.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.123+0000 7f710f52b640 1 --2- 172.21.15.86:0/982484308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7108106150 0x7f710810dd20 unknown :-1 s=CLOSED pgs=1566 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.123+0000 7f710f52b640 1 --2- 172.21.15.86:0/982484308 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7108101a30 0x7f7108105a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.123+0000 7f710f52b640 1 --2- 172.21.15.86:0/982484308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7108101080 0x7f7108101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.123+0000 7f710f52b640 1 -- 172.21.15.86:0/982484308 >> 172.21.15.86:0/982484308 conn(0x7f7108078000 msgr2=0x7f7108078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:58.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.123+0000 7f710f52b640 1 -- 172.21.15.86:0/982484308 shutdown_connections 2024-04-21T08:46:58.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.123+0000 7f710f52b640 1 -- 172.21.15.86:0/982484308 wait complete. 2024-04-21T08:46:58.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.123+0000 7f710f52b640 1 Processor -- start 2024-04-21T08:46:58.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.123+0000 7f710f52b640 1 -- start start 2024-04-21T08:46:58.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.123+0000 7f710f52b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7108101080 0x7f7108102b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.124+0000 7f710f52b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7108106150 0x7f7108103050 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.124+0000 7f710f52b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7108103590 0x7f71081a68c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.124+0000 7f710f52b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f71081110c0 con 0x7f7108106150 2024-04-21T08:46:58.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.124+0000 7f710f52b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7108110f40 con 0x7f7108103590 2024-04-21T08:46:58.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.124+0000 7f710f52b640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7108111240 con 0x7f7108101080 2024-04-21T08:46:58.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.124+0000 7f710cac6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7108106150 0x7f7108103050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.124+0000 7f710cac6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7108106150 0x7f7108103050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38280/0 (socket says 172.21.15.86:38280) 2024-04-21T08:46:58.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.124+0000 7f710cac6640 1 -- 172.21.15.86:0/2786266333 learned_addr learned my addr 172.21.15.86:0/2786266333 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:58.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.124+0000 7f710d2c7640 1 --2- 172.21.15.86:0/2786266333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7108101080 0x7f7108102b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.124+0000 7f710dac8640 1 --2- 172.21.15.86:0/2786266333 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7108103590 0x7f71081a68c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.124+0000 7f710cac6640 1 -- 172.21.15.86:0/2786266333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7108101080 msgr2=0x7f7108102b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.124+0000 7f710cac6640 1 --2- 172.21.15.86:0/2786266333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7108101080 0x7f7108102b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.124+0000 7f710cac6640 1 -- 172.21.15.86:0/2786266333 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7108103590 msgr2=0x7f71081a68c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.124+0000 7f710cac6640 1 --2- 172.21.15.86:0/2786266333 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7108103590 0x7f71081a68c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.124+0000 7f710cac6640 1 -- 172.21.15.86:0/2786266333 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f71081a6fc0 con 0x7f7108106150 2024-04-21T08:46:58.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.124+0000 7f710dac8640 1 --2- 172.21.15.86:0/2786266333 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7108103590 0x7f71081a68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:46:58.128 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.124+0000 7f710cac6640 1 --2- 172.21.15.86:0/2786266333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7108106150 0x7f7108103050 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f710400e9c0 tx=0x7f710400ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:58.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.124+0000 7f70fe7fc640 1 -- 172.21.15.86:0/2786266333 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710400cd80 con 0x7f7108106150 2024-04-21T08:46:58.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.124+0000 7f710f52b640 1 -- 172.21.15.86:0/2786266333 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f71081a72b0 con 0x7f7108106150 2024-04-21T08:46:58.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.124+0000 7f70fe7fc640 1 -- 172.21.15.86:0/2786266333 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71040054d0 con 0x7f7108106150 2024-04-21T08:46:58.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.124+0000 7f70fe7fc640 1 -- 172.21.15.86:0/2786266333 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7104010430 con 0x7f7108106150 2024-04-21T08:46:58.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.124+0000 7f710f52b640 1 -- 172.21.15.86:0/2786266333 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f71081a7800 con 0x7f7108106150 2024-04-21T08:46:58.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.127+0000 7f70fe7fc640 1 -- 172.21.15.86:0/2786266333 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7104004f90 con 0x7f7108106150 2024-04-21T08:46:58.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.127+0000 7f71121f8740 1 Processor -- start 2024-04-21T08:46:58.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.127+0000 7f71121f8740 1 -- start start 2024-04-21T08:46:58.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.127+0000 7f71121f8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d8e4e0580 0x564d8e4e0960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.127+0000 7f71121f8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8e4e0ea0 0x564d8e4e9540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.127+0000 7f71121f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d8e4e9a80 0x564d8e4ebe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.127+0000 7f70fe7fc640 1 --2- 172.21.15.86:0/2786266333 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f70f40a3d50 0x7f70f40a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.127+0000 7f71121f8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564d8e4451e0 con 0x564d8e4e9a80 2024-04-21T08:46:58.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.127+0000 7f71121f8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564d8e3e2390 con 0x564d8e4e0580 2024-04-21T08:46:58.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.127+0000 7f71121f8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564d8e4411a0 con 0x564d8e4e0ea0 2024-04-21T08:46:58.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.128+0000 7f710dac8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d8e4e0580 0x564d8e4e0960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.128+0000 7f710d2c7640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8e4e0ea0 0x564d8e4e9540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.128+0000 7f710cac6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d8e4e9a80 0x564d8e4ebe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.128+0000 7f710dac8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d8e4e0580 0x564d8e4e0960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.86:57810/0 (socket says 172.21.15.86:57810) 2024-04-21T08:46:58.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.128+0000 7f710dac8640 1 -- 172.21.15.86:0/303346091 learned_addr learned my addr 172.21.15.86:0/303346091 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:58.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.128+0000 7f710dac8640 1 --2- 172.21.15.86:0/2786266333 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f70f40a3d50 0x7f70f40a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.128+0000 7f70fe7fc640 1 -- 172.21.15.86:0/2786266333 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f71040d0f20 con 0x7f7108106150 2024-04-21T08:46:58.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.128+0000 7f710dac8640 1 -- 172.21.15.86:0/303346091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8e4e0ea0 msgr2=0x564d8e4e9540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:58.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.128+0000 7f710dac8640 1 --2- 172.21.15.86:0/303346091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8e4e0ea0 0x564d8e4e9540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.128+0000 7f710dac8640 1 -- 172.21.15.86:0/303346091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d8e4e9a80 msgr2=0x564d8e4ebe70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:58.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.128+0000 7f710dac8640 1 --2- 172.21.15.86:0/303346091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d8e4e9a80 0x564d8e4ebe70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.128+0000 7f710dac8640 1 -- 172.21.15.86:0/303346091 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d8e2217d0 con 0x564d8e4e0580 2024-04-21T08:46:58.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.128+0000 7f710dac8640 1 --2- 172.21.15.86:0/303346091 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d8e4e0580 0x564d8e4e0960 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7100009430 tx=0x7f71000327f0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e9f81329938788a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:58.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.128+0000 7f710dac8640 1 --2- 172.21.15.86:0/2786266333 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f70f40a3d50 0x7f70f40a6210 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7100006010 tx=0x7f7100042000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:58.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.128+0000 7f70f37fe640 1 -- 172.21.15.86:0/303346091 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7100051070 con 0x564d8e4e0580 2024-04-21T08:46:58.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.128+0000 7f70f37fe640 1 -- 172.21.15.86:0/303346091 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7100004d40 con 0x564d8e4e0580 2024-04-21T08:46:58.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.128+0000 7f70f37fe640 1 -- 172.21.15.86:0/303346091 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7100039040 con 0x564d8e4e0580 2024-04-21T08:46:58.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.129+0000 7f71121f8740 1 -- 172.21.15.86:0/303346091 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d8e4e0580 msgr2=0x564d8e4e0960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.129+0000 7f71121f8740 1 --2- 172.21.15.86:0/303346091 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d8e4e0580 0x564d8e4e0960 secure :-1 s=READY pgs=76 cs=0 l=1 rev1=1 crypto rx=0x7f7100009430 tx=0x7f71000327f0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.129+0000 7f71121f8740 1 -- 172.21.15.86:0/303346091 shutdown_connections 2024-04-21T08:46:58.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.129+0000 7f71121f8740 1 --2- 172.21.15.86:0/303346091 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d8e4e9a80 0x564d8e4ebe70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.129+0000 7f71121f8740 1 --2- 172.21.15.86:0/303346091 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8e4e0ea0 0x564d8e4e9540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.129+0000 7f71121f8740 1 --2- 172.21.15.86:0/303346091 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d8e4e0580 0x564d8e4e0960 unknown :-1 s=CLOSED pgs=76 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.129+0000 7f71121f8740 1 -- 172.21.15.86:0/303346091 >> 172.21.15.86:0/303346091 conn(0x564d8e4dacb0 msgr2=0x564d8e4db090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:58.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.129+0000 7f71121f8740 1 -- 172.21.15.86:0/303346091 shutdown_connections 2024-04-21T08:46:58.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.129+0000 7f71121f8740 1 -- 172.21.15.86:0/303346091 wait complete. 2024-04-21T08:46:58.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.129+0000 7f71121f8740 1 -- start start 2024-04-21T08:46:58.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.130+0000 7f71121f8740 1 Processor -- start 2024-04-21T08:46:58.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.130+0000 7f71121f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d8e4e0580 0x564d8e4e6b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.130+0000 7f71121f8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d8e4e0ea0 0x564d8e4e70b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.130+0000 7f71121f8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8e4e9a80 0x564d8e4e75f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.130+0000 7f71121f8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564d8e4ec810 con 0x564d8e4e0580 2024-04-21T08:46:58.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.130+0000 7f71121f8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564d8e3fe2f0 con 0x564d8e4e0ea0 2024-04-21T08:46:58.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.130+0000 7f71121f8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564d8e4ec990 con 0x564d8e4e9a80 2024-04-21T08:46:58.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.130+0000 7f71121f8740 10 client.0 _ll_register_callbacks cb 0x564d8e40b580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:58.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.130+0000 7f710d2c7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d8e4e0580 0x564d8e4e6b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.130+0000 7f710d2c7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d8e4e0580 0x564d8e4e6b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38310/0 (socket says 172.21.15.86:38310) 2024-04-21T08:46:58.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.130+0000 7f710cac6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d8e4e0ea0 0x564d8e4e70b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.130+0000 7f710d2c7640 1 -- 172.21.15.86:0/3619040527 learned_addr learned my addr 172.21.15.86:0/3619040527 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:58.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.130+0000 7f710dac8640 1 --2- 172.21.15.86:0/3619040527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8e4e9a80 0x564d8e4e75f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.130+0000 7f710cac6640 1 -- 172.21.15.86:0/3619040527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8e4e9a80 msgr2=0x564d8e4e75f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:58.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.130+0000 7f710cac6640 1 --2- 172.21.15.86:0/3619040527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8e4e9a80 0x564d8e4e75f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.130+0000 7f710cac6640 1 -- 172.21.15.86:0/3619040527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d8e4e0580 msgr2=0x564d8e4e6b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:58.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.130+0000 7f710cac6640 1 --2- 172.21.15.86:0/3619040527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d8e4e0580 0x564d8e4e6b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.130+0000 7f710cac6640 1 -- 172.21.15.86:0/3619040527 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d8e1573f0 con 0x564d8e4e0ea0 2024-04-21T08:46:58.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.130+0000 7f710d2c7640 1 --2- 172.21.15.86:0/3619040527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d8e4e0580 0x564d8e4e6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:46:58.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.131+0000 7f710cac6640 1 --2- 172.21.15.86:0/3619040527 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d8e4e0ea0 0x564d8e4e70b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f71040ce2a0 tx=0x7f710409c980 comp rx=0 tx=0).ready entity=mon.2 client_cookie=7272a90141a25c93 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:58.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.131+0000 7f70f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-04-21T08:46:58.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.131+0000 7f70f0ff9640 1 -- 172.21.15.86:0/3619040527 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71040b6450 con 0x564d8e4e0ea0 2024-04-21T08:46:58.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.131+0000 7f71121f8740 1 -- 172.21.15.86:0/3619040527 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7100004d40 con 0x564d8e4e0ea0 2024-04-21T08:46:58.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.131+0000 7f70f0ff9640 1 -- 172.21.15.86:0/3619040527 <== mon.2 v2:172.21.15.88:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f7104099160 con 0x564d8e4e0ea0 2024-04-21T08:46:58.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.131+0000 7f70f0ff9640 1 -- 172.21.15.86:0/3619040527 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f71040b6270 con 0x564d8e4e0ea0 2024-04-21T08:46:58.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.131+0000 7f71121f8740 10 client.27392 fetch_fsmap learned FSMap version 29 2024-04-21T08:46:58.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.131+0000 7f71121f8740 1 -- 172.21.15.86:0/3619040527 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564d8e1170f0 con 0x564d8e4e0ea0 2024-04-21T08:46:58.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.131+0000 7f70f0ff9640 1 -- 172.21.15.86:0/3619040527 <== mon.2 v2:172.21.15.88:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7100004d40 con 0x564d8e4e0ea0 2024-04-21T08:46:58.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.131+0000 7f70f0ff9640 1 -- 172.21.15.86:0/3619040527 <== mon.2 v2:172.21.15.88:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f71040b6630 con 0x564d8e4e0ea0 2024-04-21T08:46:58.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.131+0000 7f71121f8740 10 client.27392 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:46:58.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.131+0000 7f71121f8740 10 client.27392 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:58.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.131+0000 7f71121f8740 20 client.27392 populate_metadata read hostname 'smithi086' 2024-04-21T08:46:58.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.131+0000 7f71121f8740 1 --2- 172.21.15.86:0/3619040527 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564d8e4f9a20 0x564d8e4fbe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.131+0000 7f71121f8740 4 client.27392 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:58.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.131+0000 7f71121f8740 1 -- 172.21.15.86:0/3619040527 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d8e452c00 con 0x564d8e4f9a20 2024-04-21T08:46:58.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.132+0000 7f710d2c7640 1 --2- 172.21.15.86:0/3619040527 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564d8e4f9a20 0x564d8e4fbe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.132+0000 7f710d2c7640 1 --2- 172.21.15.86:0/3619040527 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564d8e4f9a20 0x564d8e4fbe10 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9fb6883d127d49a6 server_cookie=516b431f6c5bdfb8 in_seq=0 out_seq=0 2024-04-21T08:46:58.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.132+0000 7f70f0ff9640 10 client.27392 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:58.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.133+0000 7f70f0ff9640 1 -- 172.21.15.86:0/3619040527 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564d8e452c00 con 0x564d8e4f9a20 2024-04-21T08:46:58.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.133+0000 7f70f0ff9640 10 client.27392 handle_command_reply: tid=0 2024-04-21T08:46:58.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.133+0000 7f71121f8740 1 client.27392 shutdown 2024-04-21T08:46:58.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.133+0000 7f71121f8740 1 -- 172.21.15.86:0/3619040527 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d8e4e0ea0 msgr2=0x564d8e4e70b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.133+0000 7f71121f8740 1 --2- 172.21.15.86:0/3619040527 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d8e4e0ea0 0x564d8e4e70b0 secure :-1 s=READY pgs=77 cs=0 l=1 rev1=1 crypto rx=0x7f71040ce2a0 tx=0x7f710409c980 comp rx=0 tx=0).stop 2024-04-21T08:46:58.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.133+0000 7f71121f8740 1 -- 172.21.15.86:0/3619040527 shutdown_connections 2024-04-21T08:46:58.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.133+0000 7f71121f8740 1 --2- 172.21.15.86:0/3619040527 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564d8e4f9a20 0x564d8e4fbe10 crc :-1 s=READY pgs=243 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.133+0000 7f71121f8740 1 --2- 172.21.15.86:0/3619040527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8e4e9a80 0x564d8e4e75f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.133+0000 7f70f0ff9640 0 client.27392 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:58.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.133+0000 7f71121f8740 1 --2- 172.21.15.86:0/3619040527 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d8e4e0ea0 0x564d8e4e70b0 unknown :-1 s=CLOSED pgs=77 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.133+0000 7f71121f8740 1 --2- 172.21.15.86:0/3619040527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d8e4e0580 0x564d8e4e6b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.133+0000 7f71121f8740 1 -- 172.21.15.86:0/3619040527 >> 172.21.15.86:0/3619040527 conn(0x564d8e4dacb0 msgr2=0x564d8e4dca00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:58.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.133+0000 7f71121f8740 1 -- 172.21.15.86:0/3619040527 shutdown_connections 2024-04-21T08:46:58.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.133+0000 7f71121f8740 1 -- 172.21.15.86:0/3619040527 wait complete. 2024-04-21T08:46:58.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.133+0000 7f71121f8740 20 client.27392 trim_cache size 0 max 16384 2024-04-21T08:46:58.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.150+0000 7f71121f8740 1 Processor -- start 2024-04-21T08:46:58.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.150+0000 7f71121f8740 1 -- start start 2024-04-21T08:46:58.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.151+0000 7f71121f8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d8e4e0580 0x564d8e4e92a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.151+0000 7f71121f8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8e4e0ea0 0x564d8e4e5ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.151+0000 7f71121f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d8e4e9a80 0x564d8e4f56d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.151+0000 7f71121f8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f71081b15b0 con 0x564d8e4e9a80 2024-04-21T08:46:58.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.151+0000 7f71121f8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f71081b30f0 con 0x564d8e4e0580 2024-04-21T08:46:58.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.151+0000 7f71121f8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f71081b1c80 con 0x564d8e4e0ea0 2024-04-21T08:46:58.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.151+0000 7f710dac8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d8e4e9a80 0x564d8e4f56d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.151+0000 7f710d2c7640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d8e4e0580 0x564d8e4e92a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.151+0000 7f710dac8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d8e4e9a80 0x564d8e4f56d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38314/0 (socket says 172.21.15.86:38314) 2024-04-21T08:46:58.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.151+0000 7f710dac8640 1 -- 172.21.15.86:0/229003394 learned_addr learned my addr 172.21.15.86:0/229003394 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:58.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.151+0000 7f710cac6640 1 --2- 172.21.15.86:0/229003394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8e4e0ea0 0x564d8e4e5ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.151+0000 7f710dac8640 1 -- 172.21.15.86:0/229003394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8e4e0ea0 msgr2=0x564d8e4e5ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:58.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.151+0000 7f710dac8640 1 --2- 172.21.15.86:0/229003394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8e4e0ea0 0x564d8e4e5ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.151+0000 7f710dac8640 1 -- 172.21.15.86:0/229003394 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d8e4e0580 msgr2=0x564d8e4e92a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:58.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.151+0000 7f710dac8640 1 --2- 172.21.15.86:0/229003394 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d8e4e0580 0x564d8e4e92a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.151+0000 7f710dac8640 1 -- 172.21.15.86:0/229003394 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d8e4e0cd0 con 0x564d8e4e9a80 2024-04-21T08:46:58.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.151+0000 7f710d2c7640 1 --2- 172.21.15.86:0/229003394 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d8e4e0580 0x564d8e4e92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:46:58.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.151+0000 7f710cac6640 1 --2- 172.21.15.86:0/229003394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8e4e0ea0 0x564d8e4e5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:46:58.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.151+0000 7f710dac8640 1 --2- 172.21.15.86:0/229003394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d8e4e9a80 0x564d8e4f56d0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f7100009430 tx=0x7f710002c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e5fed682e3728746 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:58.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.152+0000 7f70f2ffd640 1 -- 172.21.15.86:0/229003394 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f710005b070 con 0x564d8e4e9a80 2024-04-21T08:46:58.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.152+0000 7f70f2ffd640 1 -- 172.21.15.86:0/229003394 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7100052040 con 0x564d8e4e9a80 2024-04-21T08:46:58.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.152+0000 7f70f2ffd640 1 -- 172.21.15.86:0/229003394 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7100051070 con 0x564d8e4e9a80 2024-04-21T08:46:58.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.152+0000 7f71121f8740 1 -- 172.21.15.86:0/229003394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d8e4e9a80 msgr2=0x564d8e4f56d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.158 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.152+0000 7f71121f8740 1 --2- 172.21.15.86:0/229003394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d8e4e9a80 0x564d8e4f56d0 secure :-1 s=READY pgs=1568 cs=0 l=1 rev1=1 crypto rx=0x7f7100009430 tx=0x7f710002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.152+0000 7f71121f8740 1 -- 172.21.15.86:0/229003394 shutdown_connections 2024-04-21T08:46:58.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.152+0000 7f71121f8740 1 --2- 172.21.15.86:0/229003394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d8e4e9a80 0x564d8e4f56d0 unknown :-1 s=CLOSED pgs=1568 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.152+0000 7f71121f8740 1 --2- 172.21.15.86:0/229003394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8e4e0ea0 0x564d8e4e5ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.152+0000 7f71121f8740 1 --2- 172.21.15.86:0/229003394 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d8e4e0580 0x564d8e4e92a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.152+0000 7f71121f8740 1 -- 172.21.15.86:0/229003394 >> 172.21.15.86:0/229003394 conn(0x564d8e4dacb0 msgr2=0x564d8e4d9d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:58.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.152+0000 7f71121f8740 1 -- 172.21.15.86:0/229003394 shutdown_connections 2024-04-21T08:46:58.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.152+0000 7f71121f8740 1 -- 172.21.15.86:0/229003394 wait complete. 2024-04-21T08:46:58.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.153+0000 7f71121f8740 1 -- start start 2024-04-21T08:46:58.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.153+0000 7f71121f8740 1 Processor -- start 2024-04-21T08:46:58.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.153+0000 7f71121f8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d8e4e0580 0x564d8e4f1be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.153+0000 7f71121f8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d8e4e0ea0 0x564d8e4f2120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.153+0000 7f71121f8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8e4e9a80 0x564d8e4eebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.153+0000 7f71121f8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f71081b2a20 con 0x564d8e4e0580 2024-04-21T08:46:58.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.153+0000 7f71121f8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f71081b2350 con 0x564d8e4e0ea0 2024-04-21T08:46:58.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.153+0000 7f71121f8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f71081af5b0 con 0x564d8e4e9a80 2024-04-21T08:46:58.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.153+0000 7f71121f8740 10 client.0 _ll_register_callbacks cb 0x7f71040048b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:58.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.153+0000 7f710d2c7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d8e4e0580 0x564d8e4f1be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.153+0000 7f710d2c7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d8e4e0580 0x564d8e4f1be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38320/0 (socket says 172.21.15.86:38320) 2024-04-21T08:46:58.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.153+0000 7f710d2c7640 1 -- 172.21.15.86:0/3856699168 learned_addr learned my addr 172.21.15.86:0/3856699168 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:58.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.153+0000 7f710cac6640 1 --2- 172.21.15.86:0/3856699168 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d8e4e0ea0 0x564d8e4f2120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.153+0000 7f710dac8640 1 --2- 172.21.15.86:0/3856699168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8e4e9a80 0x564d8e4eebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.154+0000 7f710d2c7640 1 -- 172.21.15.86:0/3856699168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8e4e9a80 msgr2=0x564d8e4eebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:58.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.154+0000 7f710d2c7640 1 --2- 172.21.15.86:0/3856699168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8e4e9a80 0x564d8e4eebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.154+0000 7f710d2c7640 1 -- 172.21.15.86:0/3856699168 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d8e4e0ea0 msgr2=0x564d8e4f2120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:58.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.154+0000 7f710d2c7640 1 --2- 172.21.15.86:0/3856699168 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d8e4e0ea0 0x564d8e4f2120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.154+0000 7f710d2c7640 1 -- 172.21.15.86:0/3856699168 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d8e4ef3e0 con 0x564d8e4e0580 2024-04-21T08:46:58.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.154+0000 7f710d2c7640 1 --2- 172.21.15.86:0/3856699168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d8e4e0580 0x564d8e4f1be0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x564d8e4fc560 tx=0x7f70f8012b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27faab0dd3fe06b5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:58.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.154+0000 7f70f0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:46:58.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.154+0000 7f70f0ff9640 1 -- 172.21.15.86:0/3856699168 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f70f8020690 con 0x564d8e4e0580 2024-04-21T08:46:58.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.154+0000 7f70f0ff9640 1 -- 172.21.15.86:0/3856699168 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f70f8020830 con 0x564d8e4e0580 2024-04-21T08:46:58.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.154+0000 7f71121f8740 1 -- 172.21.15.86:0/3856699168 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7100052040 con 0x564d8e4e0580 2024-04-21T08:46:58.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.154+0000 7f70f0ff9640 1 -- 172.21.15.86:0/3856699168 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f70f8005020 con 0x564d8e4e0580 2024-04-21T08:46:58.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.154+0000 7f71121f8740 10 client.18900 fetch_fsmap learned FSMap version 29 2024-04-21T08:46:58.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.154+0000 7f71121f8740 1 -- 172.21.15.86:0/3856699168 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564d8e560a30 con 0x564d8e4e0580 2024-04-21T08:46:58.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.154+0000 7f70f0ff9640 1 -- 172.21.15.86:0/3856699168 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7100052040 con 0x564d8e4e0580 2024-04-21T08:46:58.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.154+0000 7f70f0ff9640 1 -- 172.21.15.86:0/3856699168 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f70f8010430 con 0x564d8e4e0580 2024-04-21T08:46:58.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.154+0000 7f71121f8740 10 client.18900 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:46:58.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.154+0000 7f71121f8740 10 client.18900 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:58.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.155+0000 7f71121f8740 20 client.18900 populate_metadata read hostname 'smithi086' 2024-04-21T08:46:58.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.155+0000 7f71121f8740 1 --2- 172.21.15.86:0/3856699168 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564d8e561180 0x564d8e5635a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.155+0000 7f71121f8740 4 client.18900 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:46:58.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.155+0000 7f71121f8740 1 -- 172.21.15.86:0/3856699168 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x564d8e44ca70 con 0x564d8e561180 2024-04-21T08:46:58.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.155+0000 7f710cac6640 1 --2- 172.21.15.86:0/3856699168 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564d8e561180 0x564d8e5635a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.155+0000 7f70f0ff9640 10 client.18900 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:58.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.155+0000 7f710cac6640 1 --2- 172.21.15.86:0/3856699168 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564d8e561180 0x564d8e5635a0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3eaad33707773820 server_cookie=703f24f3422b0445 in_seq=0 out_seq=0 2024-04-21T08:46:58.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.155+0000 7f70f0ff9640 1 -- 172.21.15.86:0/3856699168 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x564d8e44ca70 con 0x564d8e561180 2024-04-21T08:46:58.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.155+0000 7f70f0ff9640 10 client.18900 handle_command_reply: tid=0 2024-04-21T08:46:58.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.155+0000 7f71121f8740 1 client.18900 shutdown 2024-04-21T08:46:58.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.155+0000 7f71121f8740 1 -- 172.21.15.86:0/3856699168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d8e4e0580 msgr2=0x564d8e4f1be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.155+0000 7f71121f8740 1 --2- 172.21.15.86:0/3856699168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d8e4e0580 0x564d8e4f1be0 secure :-1 s=READY pgs=1569 cs=0 l=1 rev1=1 crypto rx=0x564d8e4fc560 tx=0x7f70f8012b20 comp rx=0 tx=0).stop 2024-04-21T08:46:58.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.155+0000 7f71121f8740 1 -- 172.21.15.86:0/3856699168 shutdown_connections 2024-04-21T08:46:58.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.155+0000 7f71121f8740 1 --2- 172.21.15.86:0/3856699168 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x564d8e561180 0x564d8e5635a0 crc :-1 s=READY pgs=244 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.155+0000 7f71121f8740 1 --2- 172.21.15.86:0/3856699168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d8e4e9a80 0x564d8e4eebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.155+0000 7f71121f8740 1 --2- 172.21.15.86:0/3856699168 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d8e4e0ea0 0x564d8e4f2120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.155+0000 7f70f0ff9640 0 client.18900 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:58.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.155+0000 7f71121f8740 1 --2- 172.21.15.86:0/3856699168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d8e4e0580 0x564d8e4f1be0 unknown :-1 s=CLOSED pgs=1569 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.155+0000 7f71121f8740 1 -- 172.21.15.86:0/3856699168 >> 172.21.15.86:0/3856699168 conn(0x564d8e4dacb0 msgr2=0x564d8e4e6b30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:58.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.156+0000 7f71121f8740 1 -- 172.21.15.86:0/3856699168 shutdown_connections 2024-04-21T08:46:58.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.156+0000 7f71121f8740 1 -- 172.21.15.86:0/3856699168 wait complete. 2024-04-21T08:46:58.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.156+0000 7f71121f8740 20 client.18900 trim_cache size 0 max 16384 2024-04-21T08:46:58.163 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:46:58.163 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-21T08:46:58.163 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-21T08:46:58.163 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:46:58.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.156+0000 7f710f52b640 1 -- 172.21.15.86:0/2786266333 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f70f40a3d50 msgr2=0x7f70f40a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.156+0000 7f710f52b640 1 --2- 172.21.15.86:0/2786266333 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f70f40a3d50 0x7f70f40a6210 secure :-1 s=READY pgs=253 cs=0 l=1 rev1=1 crypto rx=0x7f7100006010 tx=0x7f7100042000 comp rx=0 tx=0).stop 2024-04-21T08:46:58.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.156+0000 7f710f52b640 1 -- 172.21.15.86:0/2786266333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7108106150 msgr2=0x7f7108103050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.156+0000 7f710f52b640 1 --2- 172.21.15.86:0/2786266333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7108106150 0x7f7108103050 secure :-1 s=READY pgs=1567 cs=0 l=1 rev1=1 crypto rx=0x7f710400e9c0 tx=0x7f710400ee80 comp rx=0 tx=0).stop 2024-04-21T08:46:58.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.156+0000 7f710f52b640 1 -- 172.21.15.86:0/2786266333 shutdown_connections 2024-04-21T08:46:58.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.156+0000 7f710f52b640 1 --2- 172.21.15.86:0/2786266333 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f70f40a3d50 0x7f70f40a6210 unknown :-1 s=CLOSED pgs=253 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.156+0000 7f710f52b640 1 --2- 172.21.15.86:0/2786266333 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7108103590 0x7f71081a68c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.156+0000 7f710f52b640 1 --2- 172.21.15.86:0/2786266333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7108106150 0x7f7108103050 unknown :-1 s=CLOSED pgs=1567 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.156+0000 7f710f52b640 1 --2- 172.21.15.86:0/2786266333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7108101080 0x7f7108102b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.156+0000 7f710f52b640 1 -- 172.21.15.86:0/2786266333 >> 172.21.15.86:0/2786266333 conn(0x7f7108078000 msgr2=0x7f7108069420 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:58.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.156+0000 7f710f52b640 1 -- 172.21.15.86:0/2786266333 shutdown_connections 2024-04-21T08:46:58.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.156+0000 7f710f52b640 1 -- 172.21.15.86:0/2786266333 wait complete. 2024-04-21T08:46:58.171 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:46:58.171 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:46:58.213 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:57 smithi088 ceph-mon[31259]: pgmap v252: 129 pgs: 129 active+clean; 2.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-21T08:46:58.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:57 smithi179 ceph-mon[31829]: pgmap v252: 129 pgs: 129 active+clean; 2.2 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-21T08:46:58.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.271+0000 7f6a26406640 1 -- 172.21.15.86:0/3932944598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a20105e30 msgr2=0x7f6a20106210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.271+0000 7f6a26406640 1 --2- 172.21.15.86:0/3932944598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a20105e30 0x7f6a20106210 secure :-1 s=READY pgs=1570 cs=0 l=1 rev1=1 crypto rx=0x7f6a10009930 tx=0x7f6a1002bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:58.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.271+0000 7f6a26406640 1 -- 172.21.15.86:0/3932944598 shutdown_connections 2024-04-21T08:46:58.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.271+0000 7f6a26406640 1 --2- 172.21.15.86:0/3932944598 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a20101b60 0x7f6a2010dc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.271+0000 7f6a26406640 1 --2- 172.21.15.86:0/3932944598 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a201011a0 0x7f6a20101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.271+0000 7f6a26406640 1 --2- 172.21.15.86:0/3932944598 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a20105e30 0x7f6a20106210 unknown :-1 s=CLOSED pgs=1570 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.271+0000 7f6a26406640 1 -- 172.21.15.86:0/3932944598 >> 172.21.15.86:0/3932944598 conn(0x7f6a20077e30 msgr2=0x7f6a20078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:58.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.271+0000 7f6a26406640 1 -- 172.21.15.86:0/3932944598 shutdown_connections 2024-04-21T08:46:58.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.271+0000 7f6a26406640 1 -- 172.21.15.86:0/3932944598 wait complete. 2024-04-21T08:46:58.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.272+0000 7f6a26406640 1 Processor -- start 2024-04-21T08:46:58.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.272+0000 7f6a26406640 1 -- start start 2024-04-21T08:46:58.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.272+0000 7f6a26406640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a201011a0 0x7f6a20197620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.272+0000 7f6a26406640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a20101b60 0x7f6a20197b60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.272+0000 7f6a26406640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a20105e30 0x7f6a2019bee0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.272+0000 7f6a26406640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f6a201110e0 con 0x7f6a20101b60 2024-04-21T08:46:58.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.272+0000 7f6a26406640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6a20110f60 con 0x7f6a201011a0 2024-04-21T08:46:58.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.272+0000 7f6a26406640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6a20111260 con 0x7f6a20105e30 2024-04-21T08:46:58.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.272+0000 7f6a1f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a20101b60 0x7f6a20197b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.272+0000 7f6a1f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a20101b60 0x7f6a20197b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38334/0 (socket says 172.21.15.86:38334) 2024-04-21T08:46:58.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.272+0000 7f6a1f7fe640 1 -- 172.21.15.86:0/4037676148 learned_addr learned my addr 172.21.15.86:0/4037676148 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:58.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.272+0000 7f6a1ffff640 1 --2- 172.21.15.86:0/4037676148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a201011a0 0x7f6a20197620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.272+0000 7f6a249a3640 1 --2- 172.21.15.86:0/4037676148 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a20105e30 0x7f6a2019bee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.273+0000 7f6a1f7fe640 1 -- 172.21.15.86:0/4037676148 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a20105e30 msgr2=0x7f6a2019bee0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.273+0000 7f6a1f7fe640 1 --2- 172.21.15.86:0/4037676148 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a20105e30 0x7f6a2019bee0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.273+0000 7f6a1f7fe640 1 -- 172.21.15.86:0/4037676148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a201011a0 msgr2=0x7f6a20197620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.273+0000 7f6a1f7fe640 1 --2- 172.21.15.86:0/4037676148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a201011a0 0x7f6a20197620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.273+0000 7f6a1f7fe640 1 -- 172.21.15.86:0/4037676148 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f6a2019c610 con 0x7f6a20101b60 2024-04-21T08:46:58.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.273+0000 7f6a1f7fe640 1 --2- 172.21.15.86:0/4037676148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a20101b60 0x7f6a20197b60 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400e9f0 tx=0x7f6a1400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:58.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.273+0000 7f6a1d7fa640 1 -- 172.21.15.86:0/4037676148 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a1400cd80 con 0x7f6a20101b60 2024-04-21T08:46:58.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.273+0000 7f6a26406640 1 -- 172.21.15.86:0/4037676148 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f6a2019c900 con 0x7f6a20101b60 2024-04-21T08:46:58.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.273+0000 7f6a1d7fa640 1 -- 172.21.15.86:0/4037676148 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6a140108e0 con 0x7f6a20101b60 2024-04-21T08:46:58.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.273+0000 7f6a26406640 1 -- 172.21.15.86:0/4037676148 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f6a201a4190 con 0x7f6a20101b60 2024-04-21T08:46:58.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.273+0000 7f6a1d7fa640 1 -- 172.21.15.86:0/4037676148 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6a14018920 con 0x7f6a20101b60 2024-04-21T08:46:58.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.274+0000 7f6a26406640 1 -- 172.21.15.86:0/4037676148 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f6a20102570 con 0x7f6a20101b60 2024-04-21T08:46:58.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.275+0000 7f6a1d7fa640 1 -- 172.21.15.86:0/4037676148 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6a14010430 con 0x7f6a20101b60 2024-04-21T08:46:58.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.276+0000 7f6a1d7fa640 1 --2- 172.21.15.86:0/4037676148 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f6a080a5f30 0x7f6a080a83f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.276+0000 7f6a1ffff640 1 --2- 172.21.15.86:0/4037676148 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f6a080a5f30 0x7f6a080a83f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.276+0000 7f6a1d7fa640 1 -- 172.21.15.86:0/4037676148 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f6a140d0e80 con 0x7f6a20101b60 2024-04-21T08:46:58.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.276+0000 7f6a1ffff640 1 --2- 172.21.15.86:0/4037676148 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f6a080a5f30 0x7f6a080a83f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6a10005ab0 tx=0x7f6a10005fd0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:58.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.285+0000 7f6a1d7fa640 1 -- 172.21.15.86:0/4037676148 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f6a1409cb00 con 0x7f6a20101b60 2024-04-21T08:46:58.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.510+0000 7f6a26406640 1 -- 172.21.15.86:0/4037676148 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f6a20108e00 con 0x7f6a20101b60 2024-04-21T08:46:58.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.511+0000 7f6a1d7fa640 1 -- 172.21.15.86:0/4037676148 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7f6a140a0e00 con 0x7f6a20101b60 2024-04-21T08:46:58.511 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:46:58.511 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:38:45.202126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24398,24394,24455,24472]},"id":1}]} 2024-04-21T08:46:58.512 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-21T08:46:58.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.513+0000 7f6a26406640 1 -- 172.21.15.86:0/4037676148 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f6a080a5f30 msgr2=0x7f6a080a83f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.513+0000 7f6a26406640 1 --2- 172.21.15.86:0/4037676148 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f6a080a5f30 0x7f6a080a83f0 secure :-1 s=READY pgs=254 cs=0 l=1 rev1=1 crypto rx=0x7f6a10005ab0 tx=0x7f6a10005fd0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.513+0000 7f6a26406640 1 -- 172.21.15.86:0/4037676148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a20101b60 msgr2=0x7f6a20197b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.513+0000 7f6a26406640 1 --2- 172.21.15.86:0/4037676148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a20101b60 0x7f6a20197b60 secure :-1 s=READY pgs=1571 cs=0 l=1 rev1=1 crypto rx=0x7f6a1400e9f0 tx=0x7f6a1400eeb0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.514+0000 7f6a26406640 1 -- 172.21.15.86:0/4037676148 shutdown_connections 2024-04-21T08:46:58.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.514+0000 7f6a26406640 1 --2- 172.21.15.86:0/4037676148 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f6a080a5f30 0x7f6a080a83f0 unknown :-1 s=CLOSED pgs=254 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.514 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.514+0000 7f6a26406640 1 --2- 172.21.15.86:0/4037676148 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6a20105e30 0x7f6a2019bee0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.514+0000 7f6a26406640 1 --2- 172.21.15.86:0/4037676148 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6a20101b60 0x7f6a20197b60 unknown :-1 s=CLOSED pgs=1571 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.514+0000 7f6a26406640 1 --2- 172.21.15.86:0/4037676148 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6a201011a0 0x7f6a20197620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.514+0000 7f6a26406640 1 -- 172.21.15.86:0/4037676148 >> 172.21.15.86:0/4037676148 conn(0x7f6a20077e30 msgr2=0x7f6a200fef00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:58.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.514+0000 7f6a26406640 1 -- 172.21.15.86:0/4037676148 shutdown_connections 2024-04-21T08:46:58.515 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.514+0000 7f6a26406640 1 -- 172.21.15.86:0/4037676148 wait complete. 2024-04-21T08:46:58.529 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:46:58.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.631+0000 7efeeda43640 1 -- 172.21.15.86:0/827074587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee8101090 msgr2=0x7efee8101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.632+0000 7efeeda43640 1 --2- 172.21.15.86:0/827074587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee8101090 0x7efee8101470 secure :-1 s=READY pgs=1572 cs=0 l=1 rev1=1 crypto rx=0x7efee0009ca0 tx=0x7efee002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.632+0000 7efeeda43640 1 -- 172.21.15.86:0/827074587 shutdown_connections 2024-04-21T08:46:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.632+0000 7efeeda43640 1 --2- 172.21.15.86:0/827074587 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee8106160 0x7efee810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.632+0000 7efeeda43640 1 --2- 172.21.15.86:0/827074587 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efee8101a40 0x7efee8105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.632+0000 7efeeda43640 1 --2- 172.21.15.86:0/827074587 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee8101090 0x7efee8101470 unknown :-1 s=CLOSED pgs=1572 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.632+0000 7efeeda43640 1 -- 172.21.15.86:0/827074587 >> 172.21.15.86:0/827074587 conn(0x7efee8077f50 msgr2=0x7efee8078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.632+0000 7efeeda43640 1 -- 172.21.15.86:0/827074587 shutdown_connections 2024-04-21T08:46:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.632+0000 7efeeda43640 1 -- 172.21.15.86:0/827074587 wait complete. 2024-04-21T08:46:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.632+0000 7efeeda43640 1 Processor -- start 2024-04-21T08:46:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.632+0000 7efeeda43640 1 -- start start 2024-04-21T08:46:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.632+0000 7efeeda43640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efee8101090 0x7efee810fd20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.632+0000 7efeeda43640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee8106160 0x7efee8110260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.632+0000 7efeeda43640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee81107a0 0x7efee81a6950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.632+0000 7efeeda43640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efee81121f0 con 0x7efee8106160 2024-04-21T08:46:58.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.632+0000 7efeeda43640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efee8112070 con 0x7efee81107a0 2024-04-21T08:46:58.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.632+0000 7efeeda43640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7efee8112370 con 0x7efee8101090 2024-04-21T08:46:58.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.633+0000 7efee67fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee8106160 0x7efee8110260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.633+0000 7efee67fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee8106160 0x7efee8110260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38352/0 (socket says 172.21.15.86:38352) 2024-04-21T08:46:58.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.633+0000 7efee67fc640 1 -- 172.21.15.86:0/1122367463 learned_addr learned my addr 172.21.15.86:0/1122367463 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:58.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.633+0000 7efee77fe640 1 --2- 172.21.15.86:0/1122367463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee81107a0 0x7efee81a6950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.633+0000 7efee6ffd640 1 --2- 172.21.15.86:0/1122367463 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efee8101090 0x7efee810fd20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.633+0000 7efee67fc640 1 -- 172.21.15.86:0/1122367463 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efee8101090 msgr2=0x7efee810fd20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.633+0000 7efee67fc640 1 --2- 172.21.15.86:0/1122367463 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efee8101090 0x7efee810fd20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.633+0000 7efee67fc640 1 -- 172.21.15.86:0/1122367463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee81107a0 msgr2=0x7efee81a6950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.633+0000 7efee67fc640 1 --2- 172.21.15.86:0/1122367463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee81107a0 0x7efee81a6950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.633+0000 7efee67fc640 1 -- 172.21.15.86:0/1122367463 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efee81a7050 con 0x7efee8106160 2024-04-21T08:46:58.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.633+0000 7efee77fe640 1 --2- 172.21.15.86:0/1122367463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee81107a0 0x7efee81a6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:58.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.633+0000 7efee6ffd640 1 --2- 172.21.15.86:0/1122367463 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efee8101090 0x7efee810fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:58.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.633+0000 7efee67fc640 1 --2- 172.21.15.86:0/1122367463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee8106160 0x7efee8110260 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7efedc00c970 tx=0x7efedc00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:58.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.633+0000 7efeeca41640 1 -- 172.21.15.86:0/1122367463 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efedc007bb0 con 0x7efee8106160 2024-04-21T08:46:58.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.633+0000 7efeeda43640 1 -- 172.21.15.86:0/1122367463 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efee81a7340 con 0x7efee8106160 2024-04-21T08:46:58.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.633+0000 7efeeca41640 1 -- 172.21.15.86:0/1122367463 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efedc007d50 con 0x7efee8106160 2024-04-21T08:46:58.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.633+0000 7efeeca41640 1 -- 172.21.15.86:0/1122367463 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efedc018730 con 0x7efee8106160 2024-04-21T08:46:58.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.633+0000 7efeeda43640 1 -- 172.21.15.86:0/1122367463 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efee81a7890 con 0x7efee8106160 2024-04-21T08:46:58.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.635+0000 7efeeca41640 1 -- 172.21.15.86:0/1122367463 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efedc00f990 con 0x7efee8106160 2024-04-21T08:46:58.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.636+0000 7efeeca41640 1 --2- 172.21.15.86:0/1122367463 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efed00a3d50 0x7efed00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.636+0000 7efee6ffd640 1 --2- 172.21.15.86:0/1122367463 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efed00a3d50 0x7efed00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.636+0000 7efeeca41640 1 -- 172.21.15.86:0/1122367463 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7efedc0d0c80 con 0x7efee8106160 2024-04-21T08:46:58.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.636+0000 7efef0633740 1 Processor -- start 2024-04-21T08:46:58.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.636+0000 7efef0633740 1 -- start start 2024-04-21T08:46:58.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.636+0000 7efee6ffd640 1 --2- 172.21.15.86:0/1122367463 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efed00a3d50 0x7efed00a6210 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efee0009a60 tx=0x7efee0005d80 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:58.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.637+0000 7efef0633740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de397454d0 0x55de397458b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.637+0000 7efef0633740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de39745e80 0x55de3974e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.637+0000 7efef0633740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de3974ea00 0x55de39750e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.637+0000 7efef0633740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55de396aa1f0 con 0x55de397454d0 2024-04-21T08:46:58.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.637+0000 7efef0633740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55de396474b0 con 0x55de3974ea00 2024-04-21T08:46:58.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.637+0000 7efef0633740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55de396a61b0 con 0x55de39745e80 2024-04-21T08:46:58.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.637+0000 7efee6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de397454d0 0x55de397458b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.637+0000 7efee6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de397454d0 0x55de397458b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38358/0 (socket says 172.21.15.86:38358) 2024-04-21T08:46:58.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.637+0000 7efee6ffd640 1 -- 172.21.15.86:0/1247917626 learned_addr learned my addr 172.21.15.86:0/1247917626 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:58.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.637+0000 7efee77fe640 1 --2- 172.21.15.86:0/1247917626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de3974ea00 0x55de39750e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.637+0000 7efee67fc640 1 --2- 172.21.15.86:0/1247917626 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de39745e80 0x55de3974e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.637+0000 7efee6ffd640 1 -- 172.21.15.86:0/1247917626 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de39745e80 msgr2=0x55de3974e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:58.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.637+0000 7efee6ffd640 1 --2- 172.21.15.86:0/1247917626 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de39745e80 0x55de3974e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.637+0000 7efee6ffd640 1 -- 172.21.15.86:0/1247917626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de3974ea00 msgr2=0x55de39750e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:58.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.637+0000 7efee6ffd640 1 --2- 172.21.15.86:0/1247917626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de3974ea00 0x55de39750e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.637+0000 7efee6ffd640 1 -- 172.21.15.86:0/1247917626 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de396391b0 con 0x55de397454d0 2024-04-21T08:46:58.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.637+0000 7efee6ffd640 1 --2- 172.21.15.86:0/1247917626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de397454d0 0x55de397458b0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7efee0032e60 tx=0x7efee003c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81ca7fa0a3da5692 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:58.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.637+0000 7efecdffb640 1 -- 172.21.15.86:0/1247917626 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee004f070 con 0x55de397454d0 2024-04-21T08:46:58.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.637+0000 7efecdffb640 1 -- 172.21.15.86:0/1247917626 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efee003a040 con 0x55de397454d0 2024-04-21T08:46:58.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.637+0000 7efecdffb640 1 -- 172.21.15.86:0/1247917626 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee0046040 con 0x55de397454d0 2024-04-21T08:46:58.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.638+0000 7efef0633740 1 -- 172.21.15.86:0/1247917626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de397454d0 msgr2=0x55de397458b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.638+0000 7efef0633740 1 --2- 172.21.15.86:0/1247917626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de397454d0 0x55de397458b0 secure :-1 s=READY pgs=1574 cs=0 l=1 rev1=1 crypto rx=0x7efee0032e60 tx=0x7efee003c000 comp rx=0 tx=0).stop 2024-04-21T08:46:58.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.638+0000 7efef0633740 1 -- 172.21.15.86:0/1247917626 shutdown_connections 2024-04-21T08:46:58.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.638+0000 7efef0633740 1 --2- 172.21.15.86:0/1247917626 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de3974ea00 0x55de39750e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.638+0000 7efef0633740 1 --2- 172.21.15.86:0/1247917626 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de39745e80 0x55de3974e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.638+0000 7efef0633740 1 --2- 172.21.15.86:0/1247917626 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de397454d0 0x55de397458b0 unknown :-1 s=CLOSED pgs=1574 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.638+0000 7efef0633740 1 -- 172.21.15.86:0/1247917626 >> 172.21.15.86:0/1247917626 conn(0x55de3973fba0 msgr2=0x55de3973ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:58.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.638+0000 7efef0633740 1 -- 172.21.15.86:0/1247917626 shutdown_connections 2024-04-21T08:46:58.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.638+0000 7efef0633740 1 -- 172.21.15.86:0/1247917626 wait complete. 2024-04-21T08:46:58.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.638+0000 7efef0633740 1 -- start start 2024-04-21T08:46:58.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.639+0000 7efef0633740 1 Processor -- start 2024-04-21T08:46:58.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.639+0000 7efef0633740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de397454d0 0x55de3974e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.639+0000 7efef0633740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de39745e80 0x55de3974b6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.639+0000 7efef0633740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de3974ea00 0x55de3974bcd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.639+0000 7efef0633740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55de39751910 con 0x55de3974ea00 2024-04-21T08:46:58.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.639+0000 7efef0633740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55de39663500 con 0x55de397454d0 2024-04-21T08:46:58.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.639+0000 7efef0633740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55de39751a90 con 0x55de39745e80 2024-04-21T08:46:58.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.639+0000 7efef0633740 10 client.0 _ll_register_callbacks cb 0x55de39631b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:58.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.639+0000 7efee6ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de397454d0 0x55de3974e330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.639+0000 7efee77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de3974ea00 0x55de3974bcd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.639+0000 7efee6ffd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de397454d0 0x55de3974e330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.86:57908/0 (socket says 172.21.15.86:57908) 2024-04-21T08:46:58.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.639+0000 7efee6ffd640 1 -- 172.21.15.86:0/1288007034 learned_addr learned my addr 172.21.15.86:0/1288007034 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:58.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.639+0000 7efee67fc640 1 --2- 172.21.15.86:0/1288007034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de39745e80 0x55de3974b6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.639+0000 7efee6ffd640 1 -- 172.21.15.86:0/1288007034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de39745e80 msgr2=0x55de3974b6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:58.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.639+0000 7efee6ffd640 1 --2- 172.21.15.86:0/1288007034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de39745e80 0x55de3974b6c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.639+0000 7efee6ffd640 1 -- 172.21.15.86:0/1288007034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de3974ea00 msgr2=0x55de3974bcd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:58.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.639+0000 7efee6ffd640 1 --2- 172.21.15.86:0/1288007034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de3974ea00 0x55de3974bcd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.639+0000 7efee6ffd640 1 -- 172.21.15.86:0/1288007034 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de393bc3f0 con 0x55de397454d0 2024-04-21T08:46:58.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.639+0000 7efee67fc640 1 --2- 172.21.15.86:0/1288007034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de39745e80 0x55de3974b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:58.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.639+0000 7efee77fe640 1 --2- 172.21.15.86:0/1288007034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de3974ea00 0x55de3974bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:46:58.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.640+0000 7efee6ffd640 1 --2- 172.21.15.86:0/1288007034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de397454d0 0x55de3974e330 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efee0004af0 tx=0x7efee00347b0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=e1289f5adf4d18c0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:58.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.640+0000 7efec6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-04-21T08:46:58.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.640+0000 7efec6ffd640 1 -- 172.21.15.86:0/1288007034 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efee004f040 con 0x55de397454d0 2024-04-21T08:46:58.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.640+0000 7efec6ffd640 1 -- 172.21.15.86:0/1288007034 <== mon.2 v2:172.21.15.88:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7efee0034ba0 con 0x55de397454d0 2024-04-21T08:46:58.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.640+0000 7efef0633740 1 -- 172.21.15.86:0/1288007034 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efee003a040 con 0x55de397454d0 2024-04-21T08:46:58.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.640+0000 7efec6ffd640 1 -- 172.21.15.86:0/1288007034 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efee0061050 con 0x55de397454d0 2024-04-21T08:46:58.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.640+0000 7efef0633740 10 client.27416 fetch_fsmap learned FSMap version 29 2024-04-21T08:46:58.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.640+0000 7efef0633740 1 -- 172.21.15.86:0/1288007034 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55de394867d0 con 0x55de397454d0 2024-04-21T08:46:58.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.640+0000 7efec6ffd640 1 -- 172.21.15.86:0/1288007034 <== mon.2 v2:172.21.15.88:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efee003a040 con 0x55de397454d0 2024-04-21T08:46:58.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.640+0000 7efec6ffd640 1 -- 172.21.15.86:0/1288007034 <== mon.2 v2:172.21.15.88:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7efee0052070 con 0x55de397454d0 2024-04-21T08:46:58.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.640+0000 7efef0633740 10 client.27416 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:46:58.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.640+0000 7efef0633740 10 client.27416 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:46:58.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.640+0000 7efef0633740 20 client.27416 populate_metadata read hostname 'smithi086' 2024-04-21T08:46:58.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.640+0000 7efef0633740 1 --2- 172.21.15.86:0/1288007034 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55de3975e9d0 0x55de39760dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.640+0000 7efef0633740 4 client.27416 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:58.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.640+0000 7efef0633740 1 -- 172.21.15.86:0/1288007034 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55de39649930 con 0x55de3975e9d0 2024-04-21T08:46:58.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.641+0000 7efee67fc640 1 --2- 172.21.15.86:0/1288007034 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55de3975e9d0 0x55de39760dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.641+0000 7efee67fc640 1 --2- 172.21.15.86:0/1288007034 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55de3975e9d0 0x55de39760dc0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b3c920fd64080762 server_cookie=a0bd3c8fccb3a20c in_seq=0 out_seq=0 2024-04-21T08:46:58.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.641+0000 7efec6ffd640 10 client.27416 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:46:58.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.642+0000 7efec6ffd640 1 -- 172.21.15.86:0/1288007034 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55de39649930 con 0x55de3975e9d0 2024-04-21T08:46:58.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.642+0000 7efec6ffd640 10 client.27416 handle_command_reply: tid=0 2024-04-21T08:46:58.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.642+0000 7efef0633740 1 client.27416 shutdown 2024-04-21T08:46:58.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.642+0000 7efef0633740 1 -- 172.21.15.86:0/1288007034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de397454d0 msgr2=0x55de3974e330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.642+0000 7efef0633740 1 --2- 172.21.15.86:0/1288007034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de397454d0 0x55de3974e330 secure :-1 s=READY pgs=78 cs=0 l=1 rev1=1 crypto rx=0x7efee0004af0 tx=0x7efee00347b0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.642+0000 7efef0633740 1 -- 172.21.15.86:0/1288007034 shutdown_connections 2024-04-21T08:46:58.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.642+0000 7efef0633740 1 --2- 172.21.15.86:0/1288007034 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de3974ea00 0x55de3974bcd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.642+0000 7efef0633740 1 --2- 172.21.15.86:0/1288007034 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de39745e80 0x55de3974b6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.642+0000 7efef0633740 1 --2- 172.21.15.86:0/1288007034 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55de3975e9d0 0x55de39760dc0 crc :-1 s=READY pgs=62 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.642+0000 7efef0633740 1 --2- 172.21.15.86:0/1288007034 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de397454d0 0x55de3974e330 unknown :-1 s=CLOSED pgs=78 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.642+0000 7efec6ffd640 0 client.27416 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:46:58.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.642+0000 7efef0633740 1 -- 172.21.15.86:0/1288007034 >> 172.21.15.86:0/1288007034 conn(0x55de3973fba0 msgr2=0x55de397418f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:58.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.642+0000 7efef0633740 1 -- 172.21.15.86:0/1288007034 shutdown_connections 2024-04-21T08:46:58.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.642+0000 7efef0633740 1 -- 172.21.15.86:0/1288007034 wait complete. 2024-04-21T08:46:58.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.642+0000 7efef0633740 20 client.27416 trim_cache size 0 max 16384 2024-04-21T08:46:58.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.657+0000 7efef0633740 1 Processor -- start 2024-04-21T08:46:58.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.657+0000 7efef0633740 1 -- start start 2024-04-21T08:46:58.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.657+0000 7efef0633740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de397454d0 0x55de3974ddf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.657+0000 7efef0633740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de39745e80 0x55de3974e330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.657+0000 7efef0633740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de3974ea00 0x55de39763de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.657+0000 7efef0633740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efee81b15e0 con 0x55de3974ea00 2024-04-21T08:46:58.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.657+0000 7efef0633740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efee81b3120 con 0x55de397454d0 2024-04-21T08:46:58.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.657+0000 7efef0633740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7efee81b1cb0 con 0x55de39745e80 2024-04-21T08:46:58.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.657+0000 7efee77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de3974ea00 0x55de39763de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.657+0000 7efee77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de3974ea00 0x55de39763de0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38378/0 (socket says 172.21.15.86:38378) 2024-04-21T08:46:58.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.657+0000 7efee77fe640 1 -- 172.21.15.86:0/722766992 learned_addr learned my addr 172.21.15.86:0/722766992 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:58.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.657+0000 7efee6ffd640 1 --2- 172.21.15.86:0/722766992 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de397454d0 0x55de3974ddf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.657+0000 7efee77fe640 1 -- 172.21.15.86:0/722766992 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de39745e80 msgr2=0x55de3974e330 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:46:58.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.657+0000 7efee77fe640 1 --2- 172.21.15.86:0/722766992 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de39745e80 0x55de3974e330 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.657+0000 7efee77fe640 1 -- 172.21.15.86:0/722766992 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de397454d0 msgr2=0x55de3974ddf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:58.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.657+0000 7efee77fe640 1 --2- 172.21.15.86:0/722766992 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de397454d0 0x55de3974ddf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.657+0000 7efee77fe640 1 -- 172.21.15.86:0/722766992 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55de3974c040 con 0x55de3974ea00 2024-04-21T08:46:58.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.657+0000 7efee77fe640 1 --2- 172.21.15.86:0/722766992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de3974ea00 0x55de39763de0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7efed80124b0 tx=0x7efed8005f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f0f6c81aefbf721c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:58.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.658+0000 7efecd7fa640 1 -- 172.21.15.86:0/722766992 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed8005020 con 0x55de3974ea00 2024-04-21T08:46:58.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.658+0000 7efecd7fa640 1 -- 172.21.15.86:0/722766992 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efed80047a0 con 0x55de3974ea00 2024-04-21T08:46:58.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.658+0000 7efecd7fa640 1 -- 172.21.15.86:0/722766992 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed8011850 con 0x55de3974ea00 2024-04-21T08:46:58.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.658+0000 7efef0633740 1 -- 172.21.15.86:0/722766992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de3974ea00 msgr2=0x55de39763de0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.658+0000 7efef0633740 1 --2- 172.21.15.86:0/722766992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de3974ea00 0x55de39763de0 secure :-1 s=READY pgs=1575 cs=0 l=1 rev1=1 crypto rx=0x7efed80124b0 tx=0x7efed8005f90 comp rx=0 tx=0).stop 2024-04-21T08:46:58.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.658+0000 7efef0633740 1 -- 172.21.15.86:0/722766992 shutdown_connections 2024-04-21T08:46:58.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.658+0000 7efef0633740 1 --2- 172.21.15.86:0/722766992 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de3974ea00 0x55de39763de0 unknown :-1 s=CLOSED pgs=1575 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.658+0000 7efef0633740 1 --2- 172.21.15.86:0/722766992 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de39745e80 0x55de3974e330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.658+0000 7efef0633740 1 --2- 172.21.15.86:0/722766992 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de397454d0 0x55de3974ddf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.658+0000 7efef0633740 1 -- 172.21.15.86:0/722766992 >> 172.21.15.86:0/722766992 conn(0x55de3973fba0 msgr2=0x55de39741880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:58.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.658+0000 7efef0633740 1 -- 172.21.15.86:0/722766992 shutdown_connections 2024-04-21T08:46:58.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.658+0000 7efef0633740 1 -- 172.21.15.86:0/722766992 wait complete. 2024-04-21T08:46:58.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.659+0000 7efef0633740 1 -- start start 2024-04-21T08:46:58.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.659+0000 7efef0633740 1 Processor -- start 2024-04-21T08:46:58.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.659+0000 7efef0633740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de397454d0 0x55de39761880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.659+0000 7efef0633740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de39745e80 0x55de39761dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.659+0000 7efef0633740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de3974ea00 0x55de39762300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.659+0000 7efef0633740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efee81b2a50 con 0x55de3974ea00 2024-04-21T08:46:58.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.659+0000 7efef0633740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efee81b2380 con 0x55de39745e80 2024-04-21T08:46:58.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.659+0000 7efef0633740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7efee81af5e0 con 0x55de397454d0 2024-04-21T08:46:58.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.659+0000 7efef0633740 10 client.0 _ll_register_callbacks cb 0x7efed8005e30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:58.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.659+0000 7efee77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de3974ea00 0x55de39762300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.659+0000 7efee77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de3974ea00 0x55de39762300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38384/0 (socket says 172.21.15.86:38384) 2024-04-21T08:46:58.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.659+0000 7efee77fe640 1 -- 172.21.15.86:0/2328019150 learned_addr learned my addr 172.21.15.86:0/2328019150 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:58.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.659+0000 7efee67fc640 1 --2- 172.21.15.86:0/2328019150 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de39745e80 0x55de39761dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.659+0000 7efee77fe640 1 -- 172.21.15.86:0/2328019150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de397454d0 msgr2=0x55de39761880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:58.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.659+0000 7efee77fe640 1 --2- 172.21.15.86:0/2328019150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de397454d0 0x55de39761880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.659+0000 7efee77fe640 1 -- 172.21.15.86:0/2328019150 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de39745e80 msgr2=0x55de39761dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:58.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.659+0000 7efee77fe640 1 --2- 172.21.15.86:0/2328019150 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de39745e80 0x55de39761dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.659+0000 7efee77fe640 1 -- 172.21.15.86:0/2328019150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55de397c3c00 con 0x55de3974ea00 2024-04-21T08:46:58.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.659+0000 7efee77fe640 1 --2- 172.21.15.86:0/2328019150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de3974ea00 0x55de39762300 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7efed80057c0 tx=0x7efed8004b80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d58b2f80f38e29d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:58.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.659+0000 7efec6ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:46:58.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.660+0000 7efec6ffd640 1 -- 172.21.15.86:0/2328019150 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efed802c070 con 0x55de3974ea00 2024-04-21T08:46:58.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.660+0000 7efec6ffd640 1 -- 172.21.15.86:0/2328019150 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7efed8005230 con 0x55de3974ea00 2024-04-21T08:46:58.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.660+0000 7efef0633740 1 -- 172.21.15.86:0/2328019150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efed80047a0 con 0x55de3974ea00 2024-04-21T08:46:58.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.660+0000 7efec6ffd640 1 -- 172.21.15.86:0/2328019150 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efed8038050 con 0x55de3974ea00 2024-04-21T08:46:58.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.660+0000 7efef0633740 10 client.18924 fetch_fsmap learned FSMap version 29 2024-04-21T08:46:58.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.660+0000 7efef0633740 1 -- 172.21.15.86:0/2328019150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55de397c4f00 con 0x55de3974ea00 2024-04-21T08:46:58.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.660+0000 7efec6ffd640 1 -- 172.21.15.86:0/2328019150 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efed80047a0 con 0x55de3974ea00 2024-04-21T08:46:58.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.660+0000 7efec6ffd640 1 -- 172.21.15.86:0/2328019150 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7efed8028440 con 0x55de3974ea00 2024-04-21T08:46:58.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.660+0000 7efef0633740 10 client.18924 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:46:58.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.660+0000 7efef0633740 10 client.18924 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:46:58.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.660+0000 7efef0633740 20 client.18924 populate_metadata read hostname 'smithi086' 2024-04-21T08:46:58.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.660+0000 7efef0633740 1 --2- 172.21.15.86:0/2328019150 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55de397c5580 0x55de397c79a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.660+0000 7efef0633740 4 client.18924 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:46:58.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.660+0000 7efef0633740 1 -- 172.21.15.86:0/2328019150 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55de396b7c10 con 0x55de397c5580 2024-04-21T08:46:58.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.661+0000 7efee6ffd640 1 --2- 172.21.15.86:0/2328019150 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55de397c5580 0x55de397c79a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.661+0000 7efec6ffd640 10 client.18924 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:46:58.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.661+0000 7efee6ffd640 1 --2- 172.21.15.86:0/2328019150 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55de397c5580 0x55de397c79a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=3e452807e37a6d7c server_cookie=bc00880c9dc8c8cb in_seq=0 out_seq=0 2024-04-21T08:46:58.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.661+0000 7efec6ffd640 1 -- 172.21.15.86:0/2328019150 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55de396b7c10 con 0x55de397c5580 2024-04-21T08:46:58.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.661+0000 7efec6ffd640 10 client.18924 handle_command_reply: tid=0 2024-04-21T08:46:58.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.661+0000 7efef0633740 1 client.18924 shutdown 2024-04-21T08:46:58.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.661+0000 7efef0633740 1 -- 172.21.15.86:0/2328019150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de3974ea00 msgr2=0x55de39762300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.661+0000 7efef0633740 1 --2- 172.21.15.86:0/2328019150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de3974ea00 0x55de39762300 secure :-1 s=READY pgs=1576 cs=0 l=1 rev1=1 crypto rx=0x7efed80057c0 tx=0x7efed8004b80 comp rx=0 tx=0).stop 2024-04-21T08:46:58.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.661+0000 7efef0633740 1 -- 172.21.15.86:0/2328019150 shutdown_connections 2024-04-21T08:46:58.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.661+0000 7efef0633740 1 --2- 172.21.15.86:0/2328019150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55de3974ea00 0x55de39762300 unknown :-1 s=CLOSED pgs=1576 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.661+0000 7efef0633740 1 --2- 172.21.15.86:0/2328019150 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55de397c5580 0x55de397c79a0 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.661+0000 7efef0633740 1 --2- 172.21.15.86:0/2328019150 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55de39745e80 0x55de39761dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.661+0000 7efef0633740 1 --2- 172.21.15.86:0/2328019150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55de397454d0 0x55de39761880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.661+0000 7efec6ffd640 0 client.18924 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:46:58.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.661+0000 7efef0633740 1 -- 172.21.15.86:0/2328019150 >> 172.21.15.86:0/2328019150 conn(0x55de3973fba0 msgr2=0x55de39740ff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:58.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.661+0000 7efef0633740 1 -- 172.21.15.86:0/2328019150 shutdown_connections 2024-04-21T08:46:58.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.661+0000 7efef0633740 1 -- 172.21.15.86:0/2328019150 wait complete. 2024-04-21T08:46:58.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.661+0000 7efef0633740 20 client.18924 trim_cache size 0 max 16384 2024-04-21T08:46:58.672 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:46:58.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.662+0000 7efeeda43640 1 -- 172.21.15.86:0/1122367463 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efed00a3d50 msgr2=0x7efed00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.662+0000 7efeeda43640 1 --2- 172.21.15.86:0/1122367463 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efed00a3d50 0x7efed00a6210 secure :-1 s=READY pgs=255 cs=0 l=1 rev1=1 crypto rx=0x7efee0009a60 tx=0x7efee0005d80 comp rx=0 tx=0).stop 2024-04-21T08:46:58.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.662+0000 7efeeda43640 1 -- 172.21.15.86:0/1122367463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee8106160 msgr2=0x7efee8110260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.662+0000 7efeeda43640 1 --2- 172.21.15.86:0/1122367463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee8106160 0x7efee8110260 secure :-1 s=READY pgs=1573 cs=0 l=1 rev1=1 crypto rx=0x7efedc00c970 tx=0x7efedc00ce30 comp rx=0 tx=0).stop 2024-04-21T08:46:58.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.662+0000 7efeeda43640 1 -- 172.21.15.86:0/1122367463 shutdown_connections 2024-04-21T08:46:58.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.662+0000 7efeeda43640 1 --2- 172.21.15.86:0/1122367463 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efed00a3d50 0x7efed00a6210 unknown :-1 s=CLOSED pgs=255 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.662+0000 7efeeda43640 1 --2- 172.21.15.86:0/1122367463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efee81107a0 0x7efee81a6950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.662+0000 7efeeda43640 1 --2- 172.21.15.86:0/1122367463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efee8106160 0x7efee8110260 unknown :-1 s=CLOSED pgs=1573 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.662+0000 7efeeda43640 1 --2- 172.21.15.86:0/1122367463 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efee8101090 0x7efee810fd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.662+0000 7efeeda43640 1 -- 172.21.15.86:0/1122367463 >> 172.21.15.86:0/1122367463 conn(0x7efee8077f50 msgr2=0x7efee80ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:58.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.662+0000 7efeeda43640 1 -- 172.21.15.86:0/1122367463 shutdown_connections 2024-04-21T08:46:58.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.662+0000 7efeeda43640 1 -- 172.21.15.86:0/1122367463 wait complete. 2024-04-21T08:46:58.675 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:46:58.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.776+0000 7f7353f2e640 1 -- 172.21.15.86:0/1635353252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f734c101a40 msgr2=0x7f734c105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.776+0000 7f7353f2e640 1 --2- 172.21.15.86:0/1635353252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f734c101a40 0x7f734c105a80 secure :-1 s=READY pgs=1577 cs=0 l=1 rev1=1 crypto rx=0x7f733c009900 tx=0x7f733c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:46:58.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.776+0000 7f7353f2e640 1 -- 172.21.15.86:0/1635353252 shutdown_connections 2024-04-21T08:46:58.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.776+0000 7f7353f2e640 1 --2- 172.21.15.86:0/1635353252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f734c106160 0x7f734c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.776+0000 7f7353f2e640 1 --2- 172.21.15.86:0/1635353252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f734c101a40 0x7f734c105a80 unknown :-1 s=CLOSED pgs=1577 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.776+0000 7f7353f2e640 1 --2- 172.21.15.86:0/1635353252 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f734c101090 0x7f734c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.776+0000 7f7353f2e640 1 -- 172.21.15.86:0/1635353252 >> 172.21.15.86:0/1635353252 conn(0x7f734c077f50 msgr2=0x7f734c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:58.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.776+0000 7f7353f2e640 1 -- 172.21.15.86:0/1635353252 shutdown_connections 2024-04-21T08:46:58.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.776+0000 7f7353f2e640 1 -- 172.21.15.86:0/1635353252 wait complete. 2024-04-21T08:46:58.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.777+0000 7f7353f2e640 1 Processor -- start 2024-04-21T08:46:58.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.777+0000 7f7353f2e640 1 -- start start 2024-04-21T08:46:58.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.777+0000 7f7353f2e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f734c101090 0x7f734c1995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.777+0000 7f7353f2e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f734c106160 0x7f734c199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.777+0000 7f7353f2e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f734c19a0b0 0x7f734c1a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.777+0000 7f7353f2e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f734c1110a0 con 0x7f734c19a0b0 2024-04-21T08:46:58.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.777+0000 7f7353f2e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f734c110f20 con 0x7f734c106160 2024-04-21T08:46:58.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.777+0000 7f7353f2e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f734c111220 con 0x7f734c101090 2024-04-21T08:46:58.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.777+0000 7f73524cb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f734c19a0b0 0x7f734c1a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.777+0000 7f73524cb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f734c19a0b0 0x7f734c1a4660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38400/0 (socket says 172.21.15.86:38400) 2024-04-21T08:46:58.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.777+0000 7f73524cb640 1 -- 172.21.15.86:0/2939786420 learned_addr learned my addr 172.21.15.86:0/2939786420 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:58.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.777+0000 7f7351cca640 1 --2- 172.21.15.86:0/2939786420 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f734c101090 0x7f734c1995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.777+0000 7f73514c9640 1 --2- 172.21.15.86:0/2939786420 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f734c106160 0x7f734c199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.777+0000 7f73524cb640 1 -- 172.21.15.86:0/2939786420 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f734c101090 msgr2=0x7f734c1995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.777+0000 7f73524cb640 1 --2- 172.21.15.86:0/2939786420 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f734c101090 0x7f734c1995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.777+0000 7f73524cb640 1 -- 172.21.15.86:0/2939786420 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f734c106160 msgr2=0x7f734c199b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.777+0000 7f73524cb640 1 --2- 172.21.15.86:0/2939786420 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f734c106160 0x7f734c199b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.777+0000 7f73524cb640 1 -- 172.21.15.86:0/2939786420 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f734c1a4e60 con 0x7f734c19a0b0 2024-04-21T08:46:58.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.777+0000 7f73514c9640 1 --2- 172.21.15.86:0/2939786420 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f734c106160 0x7f734c199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:58.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.777+0000 7f73524cb640 1 --2- 172.21.15.86:0/2939786420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f734c19a0b0 0x7f734c1a4660 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f734400efd0 tx=0x7f734400c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:58.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.777+0000 7f7342ffd640 1 -- 172.21.15.86:0/2939786420 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7344003b70 con 0x7f734c19a0b0 2024-04-21T08:46:58.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.777+0000 7f7353f2e640 1 -- 172.21.15.86:0/2939786420 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f734c1a5150 con 0x7f734c19a0b0 2024-04-21T08:46:58.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.777+0000 7f7342ffd640 1 -- 172.21.15.86:0/2939786420 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7344003d10 con 0x7f734c19a0b0 2024-04-21T08:46:58.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.777+0000 7f7342ffd640 1 -- 172.21.15.86:0/2939786420 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f73440108a0 con 0x7f734c19a0b0 2024-04-21T08:46:58.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.778+0000 7f7353f2e640 1 -- 172.21.15.86:0/2939786420 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f734c1a5650 con 0x7f734c19a0b0 2024-04-21T08:46:58.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.780+0000 7f7342ffd640 1 -- 172.21.15.86:0/2939786420 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f7344004020 con 0x7f734c19a0b0 2024-04-21T08:46:58.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.780+0000 7f7356bf4740 1 Processor -- start 2024-04-21T08:46:58.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.780+0000 7f7356bf4740 1 -- start start 2024-04-21T08:46:58.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.780+0000 7f7356bf4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f344b643b0 0x55f344b64790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.780+0000 7f7356bf4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f344b64d60 0x55f344b6d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.780+0000 7f7356bf4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f344b6de60 0x55f344b6e2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.780+0000 7f7356bf4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f344ac91f0 con 0x55f344b643b0 2024-04-21T08:46:58.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.780+0000 7f7356bf4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55f344a664b0 con 0x55f344b6de60 2024-04-21T08:46:58.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.780+0000 7f7356bf4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f344ac51b0 con 0x55f344b64d60 2024-04-21T08:46:58.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.780+0000 7f73514c9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f344b643b0 0x55f344b64790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.780+0000 7f73514c9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f344b643b0 0x55f344b64790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38414/0 (socket says 172.21.15.86:38414) 2024-04-21T08:46:58.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.780+0000 7f73514c9640 1 -- 172.21.15.86:0/3379556972 learned_addr learned my addr 172.21.15.86:0/3379556972 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:58.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.780+0000 7f7342ffd640 1 --2- 172.21.15.86:0/2939786420 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f73380a3d20 0x7f73380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.781+0000 7f7351cca640 1 --2- 172.21.15.86:0/3379556972 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f344b64d60 0x55f344b6d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.781+0000 7f73514c9640 1 --2- 172.21.15.86:0/3379556972 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f344b6de60 0x55f344b6e2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.781+0000 7f73524cb640 1 --2- 172.21.15.86:0/2939786420 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f73380a3d20 0x7f73380a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.781+0000 7f7351cca640 1 -- 172.21.15.86:0/3379556972 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f344b6de60 msgr2=0x55f344b6e2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:58.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.781+0000 7f7351cca640 1 --2- 172.21.15.86:0/3379556972 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f344b6de60 0x55f344b6e2c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.781+0000 7f7342ffd640 1 -- 172.21.15.86:0/2939786420 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f73440d1080 con 0x7f734c19a0b0 2024-04-21T08:46:58.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.781+0000 7f7351cca640 1 -- 172.21.15.86:0/3379556972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f344b643b0 msgr2=0x55f344b64790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:58.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.781+0000 7f7351cca640 1 --2- 172.21.15.86:0/3379556972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f344b643b0 0x55f344b64790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.781+0000 7f7351cca640 1 -- 172.21.15.86:0/3379556972 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f344a581b0 con 0x55f344b64d60 2024-04-21T08:46:58.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.781+0000 7f73514c9640 1 --2- 172.21.15.86:0/3379556972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f344b643b0 0x55f344b64790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:46:58.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.781+0000 7f7351cca640 1 --2- 172.21.15.86:0/3379556972 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f344b64d60 0x55f344b6d890 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7348009c00 tx=0x7f7348011810 comp rx=0 tx=0).ready entity=mon.1 client_cookie=f191ff8d09fccfaf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:58.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.781+0000 7f73524cb640 1 --2- 172.21.15.86:0/2939786420 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f73380a3d20 0x7f73380a61e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f734409c8f0 tx=0x7f7344092080 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:58.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.781+0000 7f7337fff640 1 -- 172.21.15.86:0/3379556972 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7348004230 con 0x55f344b64d60 2024-04-21T08:46:58.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.781+0000 7f7337fff640 1 -- 172.21.15.86:0/3379556972 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73480043d0 con 0x55f344b64d60 2024-04-21T08:46:58.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.781+0000 7f7337fff640 1 -- 172.21.15.86:0/3379556972 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7348004b90 con 0x55f344b64d60 2024-04-21T08:46:58.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.782+0000 7f7356bf4740 1 -- 172.21.15.86:0/3379556972 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f344b64d60 msgr2=0x55f344b6d890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.782+0000 7f7356bf4740 1 --2- 172.21.15.86:0/3379556972 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f344b64d60 0x55f344b6d890 secure :-1 s=READY pgs=69 cs=0 l=1 rev1=1 crypto rx=0x7f7348009c00 tx=0x7f7348011810 comp rx=0 tx=0).stop 2024-04-21T08:46:58.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.782+0000 7f7356bf4740 1 -- 172.21.15.86:0/3379556972 shutdown_connections 2024-04-21T08:46:58.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.782+0000 7f7356bf4740 1 --2- 172.21.15.86:0/3379556972 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f344b6de60 0x55f344b6e2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.782+0000 7f7356bf4740 1 --2- 172.21.15.86:0/3379556972 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f344b64d60 0x55f344b6d890 unknown :-1 s=CLOSED pgs=69 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.782+0000 7f7356bf4740 1 --2- 172.21.15.86:0/3379556972 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f344b643b0 0x55f344b64790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.782+0000 7f7356bf4740 1 -- 172.21.15.86:0/3379556972 >> 172.21.15.86:0/3379556972 conn(0x55f344b5eba0 msgr2=0x55f344b5ef80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:58.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.782+0000 7f7356bf4740 1 -- 172.21.15.86:0/3379556972 shutdown_connections 2024-04-21T08:46:58.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.782+0000 7f7356bf4740 1 -- 172.21.15.86:0/3379556972 wait complete. 2024-04-21T08:46:58.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.782+0000 7f7356bf4740 1 -- start start 2024-04-21T08:46:58.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.782+0000 7f7356bf4740 1 Processor -- start 2024-04-21T08:46:58.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.783+0000 7f7356bf4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f344b643b0 0x55f344b74210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.783+0000 7f7356bf4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f344b64d60 0x55f344b715a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.783+0000 7f7356bf4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f344b6de60 0x55f344b71ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.783+0000 7f7356bf4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55f344b74da0 con 0x55f344b6de60 2024-04-21T08:46:58.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.783+0000 7f7356bf4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55f344a82500 con 0x55f344b64d60 2024-04-21T08:46:58.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.783+0000 7f7356bf4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55f344b74f20 con 0x55f344b643b0 2024-04-21T08:46:58.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.783+0000 7f7356bf4740 10 client.0 _ll_register_callbacks cb 0x55f344a50b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:58.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.783+0000 7f73514c9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f344b6de60 0x55f344b71ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.783+0000 7f73514c9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f344b6de60 0x55f344b71ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38424/0 (socket says 172.21.15.86:38424) 2024-04-21T08:46:58.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.783+0000 7f73514c9640 1 -- 172.21.15.86:0/1464819222 learned_addr learned my addr 172.21.15.86:0/1464819222 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:58.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.783+0000 7f73514c9640 1 --2- 172.21.15.86:0/1464819222 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f344b643b0 0x55f344b74210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.783+0000 7f7351cca640 1 --2- 172.21.15.86:0/1464819222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f344b64d60 0x55f344b715a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.783+0000 7f73514c9640 1 -- 172.21.15.86:0/1464819222 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f344b643b0 msgr2=0x55f344b74210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:58.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.783+0000 7f73514c9640 1 --2- 172.21.15.86:0/1464819222 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f344b643b0 0x55f344b74210 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.783+0000 7f73514c9640 1 -- 172.21.15.86:0/1464819222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f344b64d60 msgr2=0x55f344b715a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:58.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.783+0000 7f73514c9640 1 --2- 172.21.15.86:0/1464819222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f344b64d60 0x55f344b715a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.783+0000 7f73514c9640 1 -- 172.21.15.86:0/1464819222 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f3447db3f0 con 0x55f344b6de60 2024-04-21T08:46:58.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.783+0000 7f7351cca640 1 --2- 172.21.15.86:0/1464819222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f344b64d60 0x55f344b715a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:58.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.783+0000 7f73514c9640 1 --2- 172.21.15.86:0/1464819222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f344b6de60 0x55f344b71ae0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f733c006010 tx=0x7f733c005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=90241f85a694cc7c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:58.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.783+0000 7f73357fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:46:58.792 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.784+0000 7f73357fa640 1 -- 172.21.15.86:0/1464819222 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c04a070 con 0x55f344b6de60 2024-04-21T08:46:58.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.784+0000 7f7356bf4740 1 -- 172.21.15.86:0/1464819222 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f73480043d0 con 0x55f344b6de60 2024-04-21T08:46:58.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.784+0000 7f73357fa640 1 -- 172.21.15.86:0/1464819222 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f733c005460 con 0x55f344b6de60 2024-04-21T08:46:58.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.784+0000 7f73357fa640 1 -- 172.21.15.86:0/1464819222 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f733c053050 con 0x55f344b6de60 2024-04-21T08:46:58.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.784+0000 7f7356bf4740 10 client.18936 fetch_fsmap learned FSMap version 29 2024-04-21T08:46:58.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.784+0000 7f7356bf4740 1 -- 172.21.15.86:0/1464819222 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f3448a57d0 con 0x55f344b6de60 2024-04-21T08:46:58.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.784+0000 7f73357fa640 1 -- 172.21.15.86:0/1464819222 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f73480043d0 con 0x55f344b6de60 2024-04-21T08:46:58.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.784+0000 7f73357fa640 1 -- 172.21.15.86:0/1464819222 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f733c04d020 con 0x55f344b6de60 2024-04-21T08:46:58.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.784+0000 7f7356bf4740 10 client.18936 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:46:58.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.784+0000 7f7356bf4740 10 client.18936 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:46:58.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.784+0000 7f7356bf4740 20 client.18936 populate_metadata read hostname 'smithi086' 2024-04-21T08:46:58.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.784+0000 7f7356bf4740 1 --2- 172.21.15.86:0/1464819222 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55f344b81f50 0x55f344b84360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.784+0000 7f7356bf4740 4 client.18936 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:58.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.784+0000 7f7356bf4740 1 -- 172.21.15.86:0/1464819222 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55f344a68930 con 0x55f344b81f50 2024-04-21T08:46:58.793 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.784+0000 7f7351cca640 1 --2- 172.21.15.86:0/1464819222 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55f344b81f50 0x55f344b84360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.785+0000 7f73357fa640 10 client.18936 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:46:58.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.785+0000 7f7351cca640 1 --2- 172.21.15.86:0/1464819222 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55f344b81f50 0x55f344b84360 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f4b9b5c745b76f55 server_cookie=d7507fc26ea8e312 in_seq=0 out_seq=0 2024-04-21T08:46:58.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.785+0000 7f73357fa640 1 -- 172.21.15.86:0/1464819222 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55f344a68930 con 0x55f344b81f50 2024-04-21T08:46:58.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.785+0000 7f73357fa640 10 client.18936 handle_command_reply: tid=0 2024-04-21T08:46:58.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.785+0000 7f7356bf4740 1 client.18936 shutdown 2024-04-21T08:46:58.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.785+0000 7f7356bf4740 1 -- 172.21.15.86:0/1464819222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f344b6de60 msgr2=0x55f344b71ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.785+0000 7f7356bf4740 1 --2- 172.21.15.86:0/1464819222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f344b6de60 0x55f344b71ae0 secure :-1 s=READY pgs=1579 cs=0 l=1 rev1=1 crypto rx=0x7f733c006010 tx=0x7f733c005a20 comp rx=0 tx=0).stop 2024-04-21T08:46:58.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.786+0000 7f7356bf4740 1 -- 172.21.15.86:0/1464819222 shutdown_connections 2024-04-21T08:46:58.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.786+0000 7f7356bf4740 1 --2- 172.21.15.86:0/1464819222 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55f344b81f50 0x55f344b84360 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.786+0000 7f7356bf4740 1 --2- 172.21.15.86:0/1464819222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f344b6de60 0x55f344b71ae0 unknown :-1 s=CLOSED pgs=1579 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.786+0000 7f7356bf4740 1 --2- 172.21.15.86:0/1464819222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f344b64d60 0x55f344b715a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.786+0000 7f73357fa640 0 client.18936 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:46:58.794 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.786+0000 7f7356bf4740 1 --2- 172.21.15.86:0/1464819222 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f344b643b0 0x55f344b74210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.786+0000 7f7356bf4740 1 -- 172.21.15.86:0/1464819222 >> 172.21.15.86:0/1464819222 conn(0x55f344b5eba0 msgr2=0x55f344b608f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:58.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.786+0000 7f7356bf4740 1 -- 172.21.15.86:0/1464819222 shutdown_connections 2024-04-21T08:46:58.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.786+0000 7f7356bf4740 1 -- 172.21.15.86:0/1464819222 wait complete. 2024-04-21T08:46:58.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.786+0000 7f7356bf4740 20 client.18936 trim_cache size 0 max 16384 2024-04-21T08:46:58.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.799+0000 7f7356bf4740 1 Processor -- start 2024-04-21T08:46:58.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.799+0000 7f7356bf4740 1 -- start start 2024-04-21T08:46:58.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.799+0000 7f7356bf4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f344b643b0 0x55f344b7dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.799+0000 7f7356bf4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f344b64d60 0x55f344b73cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.799+0000 7f7356bf4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f344b6de60 0x55f344b74210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.799+0000 7f7356bf4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f734c1ad1c0 con 0x55f344b643b0 2024-04-21T08:46:58.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.799+0000 7f7356bf4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f734c1aed00 con 0x55f344b64d60 2024-04-21T08:46:58.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.799+0000 7f7356bf4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f734c1ad890 con 0x55f344b6de60 2024-04-21T08:46:58.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.799+0000 7f73514c9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f344b643b0 0x55f344b7dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.799+0000 7f73514c9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f344b643b0 0x55f344b7dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38434/0 (socket says 172.21.15.86:38434) 2024-04-21T08:46:58.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.799+0000 7f73514c9640 1 -- 172.21.15.86:0/1852651478 learned_addr learned my addr 172.21.15.86:0/1852651478 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:58.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.799+0000 7f7351cca640 1 --2- 172.21.15.86:0/1852651478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f344b64d60 0x55f344b73cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.799+0000 7f73514c9640 1 -- 172.21.15.86:0/1852651478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f344b6de60 msgr2=0x55f344b74210 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:58.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.799+0000 7f73514c9640 1 --2- 172.21.15.86:0/1852651478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f344b6de60 0x55f344b74210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.799+0000 7f73514c9640 1 -- 172.21.15.86:0/1852651478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f344b64d60 msgr2=0x55f344b73cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:58.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.799+0000 7f73514c9640 1 --2- 172.21.15.86:0/1852651478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f344b64d60 0x55f344b73cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.799+0000 7f73514c9640 1 -- 172.21.15.86:0/1852651478 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55f344b64b00 con 0x55f344b643b0 2024-04-21T08:46:58.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.800+0000 7f73514c9640 1 --2- 172.21.15.86:0/1852651478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f344b643b0 0x55f344b7dca0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f733c0035f0 tx=0x7f733c005b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=265a4bfecafdddb4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:58.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.800+0000 7f73377fe640 1 -- 172.21.15.86:0/1852651478 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c05b020 con 0x55f344b643b0 2024-04-21T08:46:58.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.800+0000 7f73377fe640 1 -- 172.21.15.86:0/1852651478 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f733c003c40 con 0x55f344b643b0 2024-04-21T08:46:58.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.800+0000 7f73377fe640 1 -- 172.21.15.86:0/1852651478 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c04a070 con 0x55f344b643b0 2024-04-21T08:46:58.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.800+0000 7f7356bf4740 1 -- 172.21.15.86:0/1852651478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f344b643b0 msgr2=0x55f344b7dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.800+0000 7f7356bf4740 1 --2- 172.21.15.86:0/1852651478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f344b643b0 0x55f344b7dca0 secure :-1 s=READY pgs=1580 cs=0 l=1 rev1=1 crypto rx=0x7f733c0035f0 tx=0x7f733c005b50 comp rx=0 tx=0).stop 2024-04-21T08:46:58.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.800+0000 7f7356bf4740 1 -- 172.21.15.86:0/1852651478 shutdown_connections 2024-04-21T08:46:58.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.800+0000 7f7356bf4740 1 --2- 172.21.15.86:0/1852651478 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f344b6de60 0x55f344b74210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.800+0000 7f7356bf4740 1 --2- 172.21.15.86:0/1852651478 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f344b64d60 0x55f344b73cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.800+0000 7f7356bf4740 1 --2- 172.21.15.86:0/1852651478 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f344b643b0 0x55f344b7dca0 unknown :-1 s=CLOSED pgs=1580 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.800+0000 7f7356bf4740 1 -- 172.21.15.86:0/1852651478 >> 172.21.15.86:0/1852651478 conn(0x55f344b5eba0 msgr2=0x55f344b6d5f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:58.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.800+0000 7f7356bf4740 1 -- 172.21.15.86:0/1852651478 shutdown_connections 2024-04-21T08:46:58.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.800+0000 7f7356bf4740 1 -- 172.21.15.86:0/1852651478 wait complete. 2024-04-21T08:46:58.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.801+0000 7f7356bf4740 1 -- start start 2024-04-21T08:46:58.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.801+0000 7f7356bf4740 1 Processor -- start 2024-04-21T08:46:58.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.801+0000 7f7356bf4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f344b643b0 0x55f344b72c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.801+0000 7f7356bf4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f344b64d60 0x55f344b73190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.801+0000 7f7356bf4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f344b6de60 0x55f344b736d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.801+0000 7f7356bf4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f734c1ae630 con 0x55f344b643b0 2024-04-21T08:46:58.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.801+0000 7f7356bf4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f734c1adf60 con 0x55f344b6de60 2024-04-21T08:46:58.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.801+0000 7f7356bf4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f734c1ab1c0 con 0x55f344b64d60 2024-04-21T08:46:58.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.801+0000 7f7356bf4740 10 client.0 _ll_register_callbacks cb 0x7f733c02ccc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:58.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.801+0000 7f73514c9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f344b643b0 0x55f344b72c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.801+0000 7f73514c9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f344b643b0 0x55f344b72c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38438/0 (socket says 172.21.15.86:38438) 2024-04-21T08:46:58.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.801+0000 7f73514c9640 1 -- 172.21.15.86:0/3613095175 learned_addr learned my addr 172.21.15.86:0/3613095175 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:58.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.801+0000 7f73514c9640 1 --2- 172.21.15.86:0/3613095175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f344b6de60 0x55f344b736d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.801+0000 7f7351cca640 1 --2- 172.21.15.86:0/3613095175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f344b64d60 0x55f344b73190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.801+0000 7f73514c9640 1 -- 172.21.15.86:0/3613095175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f344b64d60 msgr2=0x55f344b73190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:58.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.801+0000 7f73514c9640 1 --2- 172.21.15.86:0/3613095175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f344b64d60 0x55f344b73190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.802+0000 7f73514c9640 1 -- 172.21.15.86:0/3613095175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f344b6de60 msgr2=0x55f344b736d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:58.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.802+0000 7f73514c9640 1 --2- 172.21.15.86:0/3613095175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f344b6de60 0x55f344b736d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.802+0000 7f73514c9640 1 -- 172.21.15.86:0/3613095175 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55f344b79310 con 0x55f344b643b0 2024-04-21T08:46:58.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.802+0000 7f73514c9640 1 --2- 172.21.15.86:0/3613095175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f344b643b0 0x55f344b72c50 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f733c003760 tx=0x7f733c02cdd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1da5c10463af78dd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:58.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.802+0000 7f73357fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:46:58.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.802+0000 7f73357fa640 1 -- 172.21.15.86:0/3613095175 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f733c047030 con 0x55f344b643b0 2024-04-21T08:46:58.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.802+0000 7f73357fa640 1 -- 172.21.15.86:0/3613095175 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f733c005460 con 0x55f344b643b0 2024-04-21T08:46:58.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.802+0000 7f7356bf4740 1 -- 172.21.15.86:0/3613095175 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f733c003c40 con 0x55f344b643b0 2024-04-21T08:46:58.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.802+0000 7f73357fa640 1 -- 172.21.15.86:0/3613095175 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f733c04a040 con 0x55f344b643b0 2024-04-21T08:46:58.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.802+0000 7f7356bf4740 10 client.18942 fetch_fsmap learned FSMap version 29 2024-04-21T08:46:58.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.802+0000 7f7356bf4740 1 -- 172.21.15.86:0/3613095175 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55f344b7a610 con 0x55f344b643b0 2024-04-21T08:46:58.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.802+0000 7f73357fa640 1 -- 172.21.15.86:0/3613095175 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f733c003c40 con 0x55f344b643b0 2024-04-21T08:46:58.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.802+0000 7f73357fa640 1 -- 172.21.15.86:0/3613095175 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f733c060040 con 0x55f344b643b0 2024-04-21T08:46:58.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.802+0000 7f7356bf4740 10 client.18942 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:46:58.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.802+0000 7f7356bf4740 10 client.18942 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:46:58.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.802+0000 7f7356bf4740 20 client.18942 populate_metadata read hostname 'smithi086' 2024-04-21T08:46:58.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.802+0000 7f7356bf4740 1 --2- 172.21.15.86:0/3613095175 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55f344be84a0 0x55f344bea890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.802+0000 7f7356bf4740 4 client.18942 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:46:58.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.802+0000 7f7356bf4740 1 -- 172.21.15.86:0/3613095175 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55f344ad6c10 con 0x55f344be84a0 2024-04-21T08:46:58.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.803+0000 7f7351cca640 1 --2- 172.21.15.86:0/3613095175 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55f344be84a0 0x55f344bea890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.803+0000 7f73357fa640 10 client.18942 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:46:58.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.803+0000 7f7351cca640 1 --2- 172.21.15.86:0/3613095175 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55f344be84a0 0x55f344bea890 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=3a4a54eae2f18c05 server_cookie=636431c70070e6b8 in_seq=0 out_seq=0 2024-04-21T08:46:58.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.803+0000 7f73357fa640 1 -- 172.21.15.86:0/3613095175 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55f344ad6c10 con 0x55f344be84a0 2024-04-21T08:46:58.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.803+0000 7f73357fa640 10 client.18942 handle_command_reply: tid=0 2024-04-21T08:46:58.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.803+0000 7f7356bf4740 1 client.18942 shutdown 2024-04-21T08:46:58.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.803+0000 7f7356bf4740 1 -- 172.21.15.86:0/3613095175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f344b643b0 msgr2=0x55f344b72c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.803+0000 7f7356bf4740 1 --2- 172.21.15.86:0/3613095175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f344b643b0 0x55f344b72c50 secure :-1 s=READY pgs=1581 cs=0 l=1 rev1=1 crypto rx=0x7f733c003760 tx=0x7f733c02cdd0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.803+0000 7f7356bf4740 1 -- 172.21.15.86:0/3613095175 shutdown_connections 2024-04-21T08:46:58.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.803+0000 7f7356bf4740 1 --2- 172.21.15.86:0/3613095175 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55f344be84a0 0x55f344bea890 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.803+0000 7f7356bf4740 1 --2- 172.21.15.86:0/3613095175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55f344b6de60 0x55f344b736d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.803+0000 7f7356bf4740 1 --2- 172.21.15.86:0/3613095175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55f344b64d60 0x55f344b73190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.803+0000 7f7356bf4740 1 --2- 172.21.15.86:0/3613095175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55f344b643b0 0x55f344b72c50 unknown :-1 s=CLOSED pgs=1581 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.803+0000 7f7356bf4740 1 -- 172.21.15.86:0/3613095175 >> 172.21.15.86:0/3613095175 conn(0x55f344b5eba0 msgr2=0x55f344b5d370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:58.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.803+0000 7f73357fa640 0 client.18942 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:46:58.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.804+0000 7f7356bf4740 1 -- 172.21.15.86:0/3613095175 shutdown_connections 2024-04-21T08:46:58.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.804+0000 7f7356bf4740 1 -- 172.21.15.86:0/3613095175 wait complete. 2024-04-21T08:46:58.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.804+0000 7f7356bf4740 20 client.18942 trim_cache size 0 max 16384 2024-04-21T08:46:58.811 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:46:58.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.804+0000 7f7353f2e640 1 -- 172.21.15.86:0/2939786420 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f73380a3d20 msgr2=0x7f73380a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.804+0000 7f7353f2e640 1 --2- 172.21.15.86:0/2939786420 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f73380a3d20 0x7f73380a61e0 secure :-1 s=READY pgs=256 cs=0 l=1 rev1=1 crypto rx=0x7f734409c8f0 tx=0x7f7344092080 comp rx=0 tx=0).stop 2024-04-21T08:46:58.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.804+0000 7f7353f2e640 1 -- 172.21.15.86:0/2939786420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f734c19a0b0 msgr2=0x7f734c1a4660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.804+0000 7f7353f2e640 1 --2- 172.21.15.86:0/2939786420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f734c19a0b0 0x7f734c1a4660 secure :-1 s=READY pgs=1578 cs=0 l=1 rev1=1 crypto rx=0x7f734400efd0 tx=0x7f734400c6a0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.804+0000 7f7353f2e640 1 -- 172.21.15.86:0/2939786420 shutdown_connections 2024-04-21T08:46:58.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.804+0000 7f7353f2e640 1 --2- 172.21.15.86:0/2939786420 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f73380a3d20 0x7f73380a61e0 unknown :-1 s=CLOSED pgs=256 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.804+0000 7f7353f2e640 1 --2- 172.21.15.86:0/2939786420 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f734c19a0b0 0x7f734c1a4660 unknown :-1 s=CLOSED pgs=1578 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.804+0000 7f7353f2e640 1 --2- 172.21.15.86:0/2939786420 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f734c106160 0x7f734c199b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.804+0000 7f7353f2e640 1 --2- 172.21.15.86:0/2939786420 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f734c101090 0x7f734c1995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.804+0000 7f7353f2e640 1 -- 172.21.15.86:0/2939786420 >> 172.21.15.86:0/2939786420 conn(0x7f734c077f50 msgr2=0x7f734c0ff230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:58.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.804+0000 7f7353f2e640 1 -- 172.21.15.86:0/2939786420 shutdown_connections 2024-04-21T08:46:58.812 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.804+0000 7f7353f2e640 1 -- 172.21.15.86:0/2939786420 wait complete. 2024-04-21T08:46:58.816 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:46:58.919 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.917+0000 7f2aadf15640 1 -- 172.21.15.86:0/1167131315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aa8102190 msgr2=0x7f2aa8102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.917+0000 7f2aadf15640 1 --2- 172.21.15.86:0/1167131315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aa8102190 0x7f2aa8102570 secure :-1 s=READY pgs=1582 cs=0 l=1 rev1=1 crypto rx=0x7f2a98009ca0 tx=0x7f2a9802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.917+0000 7f2aadf15640 1 -- 172.21.15.86:0/1167131315 shutdown_connections 2024-04-21T08:46:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.917+0000 7f2aadf15640 1 --2- 172.21.15.86:0/1167131315 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2aa8107260 0x7f2aa810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.917+0000 7f2aadf15640 1 --2- 172.21.15.86:0/1167131315 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2aa8102b40 0x7f2aa8106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.917+0000 7f2aadf15640 1 --2- 172.21.15.86:0/1167131315 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aa8102190 0x7f2aa8102570 unknown :-1 s=CLOSED pgs=1582 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.917+0000 7f2aadf15640 1 -- 172.21.15.86:0/1167131315 >> 172.21.15.86:0/1167131315 conn(0x7f2aa80fd990 msgr2=0x7f2aa80ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.917+0000 7f2aadf15640 1 -- 172.21.15.86:0/1167131315 shutdown_connections 2024-04-21T08:46:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.917+0000 7f2aadf15640 1 -- 172.21.15.86:0/1167131315 wait complete. 2024-04-21T08:46:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.917+0000 7f2aadf15640 1 Processor -- start 2024-04-21T08:46:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.918+0000 7f2aadf15640 1 -- start start 2024-04-21T08:46:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.918+0000 7f2aadf15640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2aa8102190 0x7f2aa8198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.918+0000 7f2aadf15640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2aa8107260 0x7f2aa8198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.918+0000 7f2aadf15640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aa819d020 0x7f2aa819d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.918+0000 7f2aadf15640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa81121a0 con 0x7f2aa819d020 2024-04-21T08:46:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.918+0000 7f2aadf15640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa8112020 con 0x7f2aa8107260 2024-04-21T08:46:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.918+0000 7f2aadf15640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa8112320 con 0x7f2aa8102190 2024-04-21T08:46:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.918+0000 7f2aa7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aa819d020 0x7f2aa819d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.918+0000 7f2aa7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aa819d020 0x7f2aa819d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38468/0 (socket says 172.21.15.86:38468) 2024-04-21T08:46:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.918+0000 7f2aa7fff640 1 -- 172.21.15.86:0/2249858169 learned_addr learned my addr 172.21.15.86:0/2249858169 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.918+0000 7f2aa6ffd640 1 --2- 172.21.15.86:0/2249858169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2aa8107260 0x7f2aa8198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.918+0000 7f2aa77fe640 1 --2- 172.21.15.86:0/2249858169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2aa8102190 0x7f2aa8198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.918+0000 7f2aa7fff640 1 -- 172.21.15.86:0/2249858169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2aa8102190 msgr2=0x7f2aa8198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.918+0000 7f2aa7fff640 1 --2- 172.21.15.86:0/2249858169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2aa8102190 0x7f2aa8198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.918+0000 7f2aa7fff640 1 -- 172.21.15.86:0/2249858169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2aa8107260 msgr2=0x7f2aa8198ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.918+0000 7f2aa7fff640 1 --2- 172.21.15.86:0/2249858169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2aa8107260 0x7f2aa8198ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.918+0000 7f2aa7fff640 1 -- 172.21.15.86:0/2249858169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2aa819db10 con 0x7f2aa819d020 2024-04-21T08:46:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.918+0000 7f2aa77fe640 1 --2- 172.21.15.86:0/2249858169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2aa8102190 0x7f2aa8198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:58.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.918+0000 7f2aa7fff640 1 --2- 172.21.15.86:0/2249858169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aa819d020 0x7f2aa819d4d0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f2aa000e970 tx=0x7f2aa000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:58.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.919+0000 7f2aa4ff9640 1 -- 172.21.15.86:0/2249858169 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa000ccc0 con 0x7f2aa819d020 2024-04-21T08:46:58.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.919+0000 7f2aadf15640 1 -- 172.21.15.86:0/2249858169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2aa819de00 con 0x7f2aa819d020 2024-04-21T08:46:58.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.919+0000 7f2aa4ff9640 1 -- 172.21.15.86:0/2249858169 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2aa000ce60 con 0x7f2aa819d020 2024-04-21T08:46:58.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.919+0000 7f2aa4ff9640 1 -- 172.21.15.86:0/2249858169 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa0010430 con 0x7f2aa819d020 2024-04-21T08:46:58.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.919+0000 7f2aadf15640 1 -- 172.21.15.86:0/2249858169 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2aa81a55a0 con 0x7f2aa819d020 2024-04-21T08:46:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.921+0000 7f2aa4ff9640 1 -- 172.21.15.86:0/2249858169 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2aa0004ff0 con 0x7f2aa819d020 2024-04-21T08:46:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.921+0000 7f2ab0be8740 1 Processor -- start 2024-04-21T08:46:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.921+0000 7f2ab0be8740 1 -- start start 2024-04-21T08:46:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.921+0000 7f2ab0be8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55681303c4d0 0x55681303c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.921+0000 7f2ab0be8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55681303ce80 0x556813045430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.921+0000 7f2ab0be8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556813045a00 0x556813047e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.921+0000 7f2ab0be8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556812fa11f0 con 0x556813045a00 2024-04-21T08:46:58.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.921+0000 7f2ab0be8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x556812f3e4b0 con 0x55681303c4d0 2024-04-21T08:46:58.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.921+0000 7f2ab0be8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556812f9d1b0 con 0x55681303ce80 2024-04-21T08:46:58.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.921+0000 7f2aa6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556813045a00 0x556813047e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.922+0000 7f2aa6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556813045a00 0x556813047e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38476/0 (socket says 172.21.15.86:38476) 2024-04-21T08:46:58.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.922+0000 7f2aa6ffd640 1 -- 172.21.15.86:0/2235987069 learned_addr learned my addr 172.21.15.86:0/2235987069 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:58.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.922+0000 7f2aa77fe640 1 --2- 172.21.15.86:0/2235987069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55681303ce80 0x556813045430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.922+0000 7f2aa6ffd640 1 --2- 172.21.15.86:0/2235987069 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55681303c4d0 0x55681303c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.922+0000 7f2aa4ff9640 1 --2- 172.21.15.86:0/2249858169 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2a900a3d20 0x7f2a900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.922+0000 7f2aa7fff640 1 --2- 172.21.15.86:0/2249858169 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2a900a3d20 0x7f2a900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.922+0000 7f2aa6ffd640 1 -- 172.21.15.86:0/2235987069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55681303ce80 msgr2=0x556813045430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:58.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.922+0000 7f2aa6ffd640 1 --2- 172.21.15.86:0/2235987069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55681303ce80 0x556813045430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.929 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.922+0000 7f2aa6ffd640 1 -- 172.21.15.86:0/2235987069 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55681303c4d0 msgr2=0x55681303c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.922+0000 7f2aa6ffd640 1 --2- 172.21.15.86:0/2235987069 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55681303c4d0 0x55681303c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.922+0000 7f2aa6ffd640 1 -- 172.21.15.86:0/2235987069 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556812f301b0 con 0x556813045a00 2024-04-21T08:46:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.922+0000 7f2aa77fe640 1 --2- 172.21.15.86:0/2235987069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55681303ce80 0x556813045430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:46:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.922+0000 7f2aa4ff9640 1 -- 172.21.15.86:0/2249858169 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f2aa00d1010 con 0x7f2aa819d020 2024-04-21T08:46:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.922+0000 7f2aa6ffd640 1 --2- 172.21.15.86:0/2235987069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556813045a00 0x556813047e50 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c012750 tx=0x7f2a9c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86733898d346bb09 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.922+0000 7f2aa7fff640 1 --2- 172.21.15.86:0/2249858169 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2a900a3d20 0x7f2a900a61e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2aa00ce440 tx=0x7f2aa006b590 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.922+0000 7f2a8dffb640 1 -- 172.21.15.86:0/2235987069 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c012cd0 con 0x556813045a00 2024-04-21T08:46:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.922+0000 7f2a8dffb640 1 -- 172.21.15.86:0/2235987069 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a9c005020 con 0x556813045a00 2024-04-21T08:46:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.922+0000 7f2a8dffb640 1 -- 172.21.15.86:0/2235987069 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a9c0049e0 con 0x556813045a00 2024-04-21T08:46:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.923+0000 7f2ab0be8740 1 -- 172.21.15.86:0/2235987069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556813045a00 msgr2=0x556813047e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.923+0000 7f2ab0be8740 1 --2- 172.21.15.86:0/2235987069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556813045a00 0x556813047e50 secure :-1 s=READY pgs=1584 cs=0 l=1 rev1=1 crypto rx=0x7f2a9c012750 tx=0x7f2a9c004730 comp rx=0 tx=0).stop 2024-04-21T08:46:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.923+0000 7f2ab0be8740 1 -- 172.21.15.86:0/2235987069 shutdown_connections 2024-04-21T08:46:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.923+0000 7f2ab0be8740 1 --2- 172.21.15.86:0/2235987069 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556813045a00 0x556813047e50 unknown :-1 s=CLOSED pgs=1584 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.923+0000 7f2ab0be8740 1 --2- 172.21.15.86:0/2235987069 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55681303ce80 0x556813045430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.923+0000 7f2ab0be8740 1 --2- 172.21.15.86:0/2235987069 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55681303c4d0 0x55681303c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.923+0000 7f2ab0be8740 1 -- 172.21.15.86:0/2235987069 >> 172.21.15.86:0/2235987069 conn(0x556813036ba0 msgr2=0x556813036f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.923+0000 7f2ab0be8740 1 -- 172.21.15.86:0/2235987069 shutdown_connections 2024-04-21T08:46:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.923+0000 7f2ab0be8740 1 -- 172.21.15.86:0/2235987069 wait complete. 2024-04-21T08:46:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.923+0000 7f2ab0be8740 1 -- start start 2024-04-21T08:46:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.923+0000 7f2ab0be8740 1 Processor -- start 2024-04-21T08:46:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.924+0000 7f2ab0be8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55681303c4d0 0x556813045300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.924+0000 7f2ab0be8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55681303ce80 0x556813042690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.924+0000 7f2ab0be8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556813045a00 0x556813042c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.924+0000 7f2ab0be8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556813048970 con 0x55681303ce80 2024-04-21T08:46:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.924+0000 7f2ab0be8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x556812f5a500 con 0x556813045a00 2024-04-21T08:46:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.924+0000 7f2ab0be8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556813048af0 con 0x55681303c4d0 2024-04-21T08:46:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.924+0000 7f2ab0be8740 10 client.0 _ll_register_callbacks cb 0x556812f28b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.924+0000 7f2aa77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55681303ce80 0x556813042690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.924+0000 7f2aa77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55681303ce80 0x556813042690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38486/0 (socket says 172.21.15.86:38486) 2024-04-21T08:46:58.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.924+0000 7f2aa77fe640 1 -- 172.21.15.86:0/872814100 learned_addr learned my addr 172.21.15.86:0/872814100 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:58.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.924+0000 7f2aa6ffd640 1 --2- 172.21.15.86:0/872814100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55681303c4d0 0x556813045300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.924+0000 7f2aa6ffd640 1 --2- 172.21.15.86:0/872814100 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556813045a00 0x556813042c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.924+0000 7f2aa77fe640 1 -- 172.21.15.86:0/872814100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55681303c4d0 msgr2=0x556813045300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:58.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.924+0000 7f2aa77fe640 1 --2- 172.21.15.86:0/872814100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55681303c4d0 0x556813045300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.924+0000 7f2aa77fe640 1 -- 172.21.15.86:0/872814100 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556813045a00 msgr2=0x556813042c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:58.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.924+0000 7f2aa77fe640 1 --2- 172.21.15.86:0/872814100 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556813045a00 0x556813042c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.924+0000 7f2aa77fe640 1 -- 172.21.15.86:0/872814100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556812cb33f0 con 0x55681303ce80 2024-04-21T08:46:58.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.924+0000 7f2aa77fe640 1 --2- 172.21.15.86:0/872814100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55681303ce80 0x556813042690 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f2a98009ca0 tx=0x7f2a98009960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=767c7e0cf47ed88a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:58.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.924+0000 7f2a86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:46:58.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.924+0000 7f2a86ffd640 1 -- 172.21.15.86:0/872814100 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a98043070 con 0x55681303ce80 2024-04-21T08:46:58.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.925+0000 7f2a86ffd640 1 -- 172.21.15.86:0/872814100 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f2a980324c0 con 0x55681303ce80 2024-04-21T08:46:58.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.925+0000 7f2ab0be8740 1 -- 172.21.15.86:0/872814100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a9c005020 con 0x55681303ce80 2024-04-21T08:46:58.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.925+0000 7f2a86ffd640 1 -- 172.21.15.86:0/872814100 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2a9804f050 con 0x55681303ce80 2024-04-21T08:46:58.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.925+0000 7f2ab0be8740 10 client.18954 fetch_fsmap learned FSMap version 29 2024-04-21T08:46:58.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.925+0000 7f2ab0be8740 1 -- 172.21.15.86:0/872814100 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556812d7d7d0 con 0x55681303ce80 2024-04-21T08:46:58.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.925+0000 7f2a86ffd640 1 -- 172.21.15.86:0/872814100 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a9c005020 con 0x55681303ce80 2024-04-21T08:46:58.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.925+0000 7f2a86ffd640 1 -- 172.21.15.86:0/872814100 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2a980042a0 con 0x55681303ce80 2024-04-21T08:46:58.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.925+0000 7f2ab0be8740 10 client.18954 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:46:58.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.925+0000 7f2ab0be8740 10 client.18954 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:46:58.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.925+0000 7f2ab0be8740 20 client.18954 populate_metadata read hostname 'smithi086' 2024-04-21T08:46:58.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.925+0000 7f2ab0be8740 1 --2- 172.21.15.86:0/872814100 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x556813055b90 0x556813057f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.925+0000 7f2ab0be8740 4 client.18954 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:58.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.925+0000 7f2ab0be8740 1 -- 172.21.15.86:0/872814100 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556812f40930 con 0x556813055b90 2024-04-21T08:46:58.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.925+0000 7f2aa6ffd640 1 --2- 172.21.15.86:0/872814100 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x556813055b90 0x556813057f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.926+0000 7f2aa6ffd640 1 --2- 172.21.15.86:0/872814100 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x556813055b90 0x556813057f80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=96f54f3aaa767e72 server_cookie=120893af9a0dd361 in_seq=0 out_seq=0 2024-04-21T08:46:58.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.926+0000 7f2a86ffd640 10 client.18954 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:46:58.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.926+0000 7f2a86ffd640 1 -- 172.21.15.86:0/872814100 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556812f40930 con 0x556813055b90 2024-04-21T08:46:58.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.926+0000 7f2a86ffd640 10 client.18954 handle_command_reply: tid=0 2024-04-21T08:46:58.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.926+0000 7f2ab0be8740 1 client.18954 shutdown 2024-04-21T08:46:58.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.926+0000 7f2ab0be8740 1 -- 172.21.15.86:0/872814100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55681303ce80 msgr2=0x556813042690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.926+0000 7f2ab0be8740 1 --2- 172.21.15.86:0/872814100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55681303ce80 0x556813042690 secure :-1 s=READY pgs=1585 cs=0 l=1 rev1=1 crypto rx=0x7f2a98009ca0 tx=0x7f2a98009960 comp rx=0 tx=0).stop 2024-04-21T08:46:58.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.926+0000 7f2ab0be8740 1 -- 172.21.15.86:0/872814100 shutdown_connections 2024-04-21T08:46:58.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.926+0000 7f2ab0be8740 1 --2- 172.21.15.86:0/872814100 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x556813055b90 0x556813057f80 crc :-1 s=READY pgs=63 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.926+0000 7f2ab0be8740 1 --2- 172.21.15.86:0/872814100 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556813045a00 0x556813042c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.926+0000 7f2ab0be8740 1 --2- 172.21.15.86:0/872814100 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55681303ce80 0x556813042690 unknown :-1 s=CLOSED pgs=1585 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.926+0000 7f2a86ffd640 0 client.18954 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:46:58.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.926+0000 7f2ab0be8740 1 --2- 172.21.15.86:0/872814100 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55681303c4d0 0x556813045300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.926+0000 7f2ab0be8740 1 -- 172.21.15.86:0/872814100 >> 172.21.15.86:0/872814100 conn(0x556813036ba0 msgr2=0x5568130388f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:58.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.927+0000 7f2ab0be8740 1 -- 172.21.15.86:0/872814100 shutdown_connections 2024-04-21T08:46:58.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.927+0000 7f2ab0be8740 1 -- 172.21.15.86:0/872814100 wait complete. 2024-04-21T08:46:58.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.927+0000 7f2ab0be8740 20 client.18954 trim_cache size 0 max 16384 2024-04-21T08:46:58.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.940+0000 7f2ab0be8740 1 Processor -- start 2024-04-21T08:46:58.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.940+0000 7f2ab0be8740 1 -- start start 2024-04-21T08:46:58.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.940+0000 7f2ab0be8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55681303c4d0 0x556813051830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.940+0000 7f2ab0be8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55681303ce80 0x556813044dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.940+0000 7f2ab0be8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556813045a00 0x556813045300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.940+0000 7f2ab0be8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa81af3c0 con 0x55681303ce80 2024-04-21T08:46:58.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.940+0000 7f2ab0be8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa81b0f00 con 0x55681303c4d0 2024-04-21T08:46:58.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.940+0000 7f2ab0be8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa81afa90 con 0x556813045a00 2024-04-21T08:46:58.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.940+0000 7f2aa77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55681303ce80 0x556813044dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.940+0000 7f2aa77fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55681303ce80 0x556813044dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38488/0 (socket says 172.21.15.86:38488) 2024-04-21T08:46:58.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.940+0000 7f2aa77fe640 1 -- 172.21.15.86:0/1579812041 learned_addr learned my addr 172.21.15.86:0/1579812041 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:58.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.940+0000 7f2aa6ffd640 1 --2- 172.21.15.86:0/1579812041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556813045a00 0x556813045300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.940+0000 7f2aa6ffd640 1 --2- 172.21.15.86:0/1579812041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55681303c4d0 0x556813051830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.940+0000 7f2aa77fe640 1 -- 172.21.15.86:0/1579812041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556813045a00 msgr2=0x556813045300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:58.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.940+0000 7f2aa77fe640 1 --2- 172.21.15.86:0/1579812041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556813045a00 0x556813045300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.940+0000 7f2aa77fe640 1 -- 172.21.15.86:0/1579812041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55681303c4d0 msgr2=0x556813051830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:58.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.940+0000 7f2aa77fe640 1 --2- 172.21.15.86:0/1579812041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55681303c4d0 0x556813051830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.940+0000 7f2aa77fe640 1 -- 172.21.15.86:0/1579812041 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5568130582f0 con 0x55681303ce80 2024-04-21T08:46:58.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.940+0000 7f2aa6ffd640 1 --2- 172.21.15.86:0/1579812041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55681303c4d0 0x556813051830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:58.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.940+0000 7f2aa77fe640 1 --2- 172.21.15.86:0/1579812041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55681303ce80 0x556813044dc0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f2a98006010 tx=0x7f2a980093f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d4ea4303ec05d60b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:58.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.941+0000 7f2a8d7fa640 1 -- 172.21.15.86:0/1579812041 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a98058070 con 0x55681303ce80 2024-04-21T08:46:58.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.941+0000 7f2a8d7fa640 1 -- 172.21.15.86:0/1579812041 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a98009540 con 0x55681303ce80 2024-04-21T08:46:58.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.941+0000 7f2a8d7fa640 1 -- 172.21.15.86:0/1579812041 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a98043050 con 0x55681303ce80 2024-04-21T08:46:58.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.941+0000 7f2ab0be8740 1 -- 172.21.15.86:0/1579812041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55681303ce80 msgr2=0x556813044dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.941+0000 7f2ab0be8740 1 --2- 172.21.15.86:0/1579812041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55681303ce80 0x556813044dc0 secure :-1 s=READY pgs=1586 cs=0 l=1 rev1=1 crypto rx=0x7f2a98006010 tx=0x7f2a980093f0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.941+0000 7f2ab0be8740 1 -- 172.21.15.86:0/1579812041 shutdown_connections 2024-04-21T08:46:58.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.941+0000 7f2ab0be8740 1 --2- 172.21.15.86:0/1579812041 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556813045a00 0x556813045300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.941+0000 7f2ab0be8740 1 --2- 172.21.15.86:0/1579812041 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55681303ce80 0x556813044dc0 unknown :-1 s=CLOSED pgs=1586 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.941+0000 7f2ab0be8740 1 --2- 172.21.15.86:0/1579812041 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55681303c4d0 0x556813051830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.941+0000 7f2ab0be8740 1 -- 172.21.15.86:0/1579812041 >> 172.21.15.86:0/1579812041 conn(0x556813036ba0 msgr2=0x55681303dc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:58.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.941+0000 7f2ab0be8740 1 -- 172.21.15.86:0/1579812041 shutdown_connections 2024-04-21T08:46:58.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.941+0000 7f2ab0be8740 1 -- 172.21.15.86:0/1579812041 wait complete. 2024-04-21T08:46:58.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.941+0000 7f2ab0be8740 1 -- start start 2024-04-21T08:46:58.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.942+0000 7f2ab0be8740 1 Processor -- start 2024-04-21T08:46:58.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.942+0000 7f2ab0be8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55681303c4d0 0x55681304e050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.942+0000 7f2ab0be8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55681303ce80 0x5568130493c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.942+0000 7f2ab0be8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556813045a00 0x556813049900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.942+0000 7f2ab0be8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa81b0830 con 0x556813045a00 2024-04-21T08:46:58.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.942+0000 7f2ab0be8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa81b0160 con 0x55681303c4d0 2024-04-21T08:46:58.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.942+0000 7f2ab0be8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2aa81ad3c0 con 0x55681303ce80 2024-04-21T08:46:58.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.942+0000 7f2ab0be8740 10 client.0 _ll_register_callbacks cb 0x7f2a98009800 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:58.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.942+0000 7f2aa6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556813045a00 0x556813049900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.942+0000 7f2aa77fe640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55681303ce80 0x5568130493c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.942+0000 7f2aa6ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556813045a00 0x556813049900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38504/0 (socket says 172.21.15.86:38504) 2024-04-21T08:46:58.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.942+0000 7f2aa6ffd640 1 -- 172.21.15.86:0/3680284710 learned_addr learned my addr 172.21.15.86:0/3680284710 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:58.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.942+0000 7f2aa6ffd640 1 --2- 172.21.15.86:0/3680284710 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55681303c4d0 0x55681304e050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.942+0000 7f2aa77fe640 1 -- 172.21.15.86:0/3680284710 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55681303c4d0 msgr2=0x55681304e050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:58.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.942+0000 7f2aa77fe640 1 --2- 172.21.15.86:0/3680284710 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55681303c4d0 0x55681304e050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.942+0000 7f2aa77fe640 1 -- 172.21.15.86:0/3680284710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556813045a00 msgr2=0x556813049900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:58.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.942+0000 7f2aa77fe640 1 --2- 172.21.15.86:0/3680284710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556813045a00 0x556813049900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.942+0000 7f2aa77fe640 1 -- 172.21.15.86:0/3680284710 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55681304a100 con 0x55681303ce80 2024-04-21T08:46:58.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.942+0000 7f2aa6ffd640 1 --2- 172.21.15.86:0/3680284710 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55681303c4d0 0x55681304e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:58.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.943+0000 7f2aa77fe640 1 --2- 172.21.15.86:0/3680284710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55681303ce80 0x5568130493c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x5568130438d0 tx=0x7f2a980326f0 comp rx=0 tx=0).ready entity=mon.1 client_cookie=5271cfd81242f79a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:58.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.943+0000 7f2a86ffd640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-04-21T08:46:58.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.943+0000 7f2a86ffd640 1 -- 172.21.15.86:0/3680284710 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2a98058070 con 0x55681303ce80 2024-04-21T08:46:58.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.943+0000 7f2ab0be8740 1 -- 172.21.15.86:0/3680284710 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2a98009540 con 0x55681303ce80 2024-04-21T08:46:58.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.943+0000 7f2a86ffd640 1 -- 172.21.15.86:0/3680284710 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f2a980348d0 con 0x55681303ce80 2024-04-21T08:46:58.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.943+0000 7f2a86ffd640 1 -- 172.21.15.86:0/3680284710 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2a98064050 con 0x55681303ce80 2024-04-21T08:46:58.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.943+0000 7f2ab0be8740 10 client.27082 fetch_fsmap learned FSMap version 29 2024-04-21T08:46:58.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.943+0000 7f2ab0be8740 1 -- 172.21.15.86:0/3680284710 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5568130bbf60 con 0x55681303ce80 2024-04-21T08:46:58.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.943+0000 7f2a86ffd640 1 -- 172.21.15.86:0/3680284710 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2a98009540 con 0x55681303ce80 2024-04-21T08:46:58.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.943+0000 7f2a86ffd640 1 -- 172.21.15.86:0/3680284710 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2a98048820 con 0x55681303ce80 2024-04-21T08:46:58.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.943+0000 7f2ab0be8740 10 client.27082 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:46:58.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.943+0000 7f2ab0be8740 10 client.27082 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:46:58.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.943+0000 7f2ab0be8740 20 client.27082 populate_metadata read hostname 'smithi086' 2024-04-21T08:46:58.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.943+0000 7f2ab0be8740 1 --2- 172.21.15.86:0/3680284710 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5568130bc510 0x5568130be900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:58.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.943+0000 7f2ab0be8740 4 client.27082 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:46:58.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.943+0000 7f2ab0be8740 1 -- 172.21.15.86:0/3680284710 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556812faec10 con 0x5568130bc510 2024-04-21T08:46:58.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.943+0000 7f2aa6ffd640 1 --2- 172.21.15.86:0/3680284710 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5568130bc510 0x5568130be900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:58.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.944+0000 7f2aa6ffd640 1 --2- 172.21.15.86:0/3680284710 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5568130bc510 0x5568130be900 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e99f558490d54793 server_cookie=acefbf8c2c54c10c in_seq=0 out_seq=0 2024-04-21T08:46:58.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.944+0000 7f2a86ffd640 10 client.27082 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:46:58.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.944+0000 7f2a86ffd640 1 -- 172.21.15.86:0/3680284710 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556812faec10 con 0x5568130bc510 2024-04-21T08:46:58.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.944+0000 7f2a86ffd640 10 client.27082 handle_command_reply: tid=0 2024-04-21T08:46:58.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.944+0000 7f2ab0be8740 1 client.27082 shutdown 2024-04-21T08:46:58.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.944+0000 7f2ab0be8740 1 -- 172.21.15.86:0/3680284710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55681303ce80 msgr2=0x5568130493c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.944+0000 7f2ab0be8740 1 --2- 172.21.15.86:0/3680284710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55681303ce80 0x5568130493c0 secure :-1 s=READY pgs=70 cs=0 l=1 rev1=1 crypto rx=0x5568130438d0 tx=0x7f2a980326f0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.944+0000 7f2ab0be8740 1 -- 172.21.15.86:0/3680284710 shutdown_connections 2024-04-21T08:46:58.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.944+0000 7f2ab0be8740 1 --2- 172.21.15.86:0/3680284710 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5568130bc510 0x5568130be900 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.944+0000 7f2ab0be8740 1 --2- 172.21.15.86:0/3680284710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556813045a00 0x556813049900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.944+0000 7f2ab0be8740 1 --2- 172.21.15.86:0/3680284710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55681303ce80 0x5568130493c0 unknown :-1 s=CLOSED pgs=70 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.944+0000 7f2ab0be8740 1 --2- 172.21.15.86:0/3680284710 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55681303c4d0 0x55681304e050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.944+0000 7f2a86ffd640 0 client.27082 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:46:58.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.944+0000 7f2ab0be8740 1 -- 172.21.15.86:0/3680284710 >> 172.21.15.86:0/3680284710 conn(0x556813036ba0 msgr2=0x556813035370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:58.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.944+0000 7f2ab0be8740 1 -- 172.21.15.86:0/3680284710 shutdown_connections 2024-04-21T08:46:58.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.944+0000 7f2ab0be8740 1 -- 172.21.15.86:0/3680284710 wait complete. 2024-04-21T08:46:58.951 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.944+0000 7f2ab0be8740 20 client.27082 trim_cache size 0 max 16384 2024-04-21T08:46:58.952 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:46:58.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.945+0000 7f2aadf15640 1 -- 172.21.15.86:0/2249858169 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2a900a3d20 msgr2=0x7f2a900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.945+0000 7f2aadf15640 1 --2- 172.21.15.86:0/2249858169 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2a900a3d20 0x7f2a900a61e0 secure :-1 s=READY pgs=257 cs=0 l=1 rev1=1 crypto rx=0x7f2aa00ce440 tx=0x7f2aa006b590 comp rx=0 tx=0).stop 2024-04-21T08:46:58.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.945+0000 7f2aadf15640 1 -- 172.21.15.86:0/2249858169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aa819d020 msgr2=0x7f2aa819d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:58.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.945+0000 7f2aadf15640 1 --2- 172.21.15.86:0/2249858169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aa819d020 0x7f2aa819d4d0 secure :-1 s=READY pgs=1583 cs=0 l=1 rev1=1 crypto rx=0x7f2aa000e970 tx=0x7f2aa000ed70 comp rx=0 tx=0).stop 2024-04-21T08:46:58.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.945+0000 7f2aadf15640 1 -- 172.21.15.86:0/2249858169 shutdown_connections 2024-04-21T08:46:58.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.945+0000 7f2aadf15640 1 --2- 172.21.15.86:0/2249858169 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2a900a3d20 0x7f2a900a61e0 unknown :-1 s=CLOSED pgs=257 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.945+0000 7f2aadf15640 1 --2- 172.21.15.86:0/2249858169 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2aa819d020 0x7f2aa819d4d0 unknown :-1 s=CLOSED pgs=1583 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.945+0000 7f2aadf15640 1 --2- 172.21.15.86:0/2249858169 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2aa8107260 0x7f2aa8198ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.945+0000 7f2aadf15640 1 --2- 172.21.15.86:0/2249858169 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2aa8102190 0x7f2aa8198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:58.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.945+0000 7f2aadf15640 1 -- 172.21.15.86:0/2249858169 >> 172.21.15.86:0/2249858169 conn(0x7f2aa80fd990 msgr2=0x7f2aa80fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:58.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.945+0000 7f2aadf15640 1 -- 172.21.15.86:0/2249858169 shutdown_connections 2024-04-21T08:46:58.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:58.945+0000 7f2aadf15640 1 -- 172.21.15.86:0/2249858169 wait complete. 2024-04-21T08:46:58.956 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:46:59.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.056+0000 7fcaa86a5640 1 -- 172.21.15.86:0/445484457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa010c570 msgr2=0x7fcaa010e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:59.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.056+0000 7fcaa86a5640 1 --2- 172.21.15.86:0/445484457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa010c570 0x7fcaa010e9c0 secure :-1 s=READY pgs=1587 cs=0 l=1 rev1=1 crypto rx=0x7fca98009ca0 tx=0x7fca9802c180 comp rx=0 tx=0).stop 2024-04-21T08:46:59.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.056+0000 7fcaa86a5640 1 -- 172.21.15.86:0/445484457 shutdown_connections 2024-04-21T08:46:59.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.056+0000 7fcaa86a5640 1 --2- 172.21.15.86:0/445484457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa010c570 0x7fcaa010e9c0 unknown :-1 s=CLOSED pgs=1587 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.056+0000 7fcaa86a5640 1 --2- 172.21.15.86:0/445484457 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcaa0103970 0x7fcaa010c030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.056+0000 7fcaa86a5640 1 --2- 172.21.15.86:0/445484457 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcaa0103050 0x7fcaa0103430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.056+0000 7fcaa86a5640 1 -- 172.21.15.86:0/445484457 >> 172.21.15.86:0/445484457 conn(0x7fcaa0077fb0 msgr2=0x7fcaa00783c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:59.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.056+0000 7fcaa86a5640 1 -- 172.21.15.86:0/445484457 shutdown_connections 2024-04-21T08:46:59.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.056+0000 7fcaa86a5640 1 -- 172.21.15.86:0/445484457 wait complete. 2024-04-21T08:46:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.056+0000 7fcaa86a5640 1 Processor -- start 2024-04-21T08:46:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.057+0000 7fcaa86a5640 1 -- start start 2024-04-21T08:46:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.057+0000 7fcaa86a5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcaa0103050 0x7fcaa01986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.057+0000 7fcaa86a5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa0103970 0x7fcaa0198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.057+0000 7fcaa86a5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcaa010c570 0x7fcaa019cfa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.057+0000 7fcaa86a5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa01121a0 con 0x7fcaa0103970 2024-04-21T08:46:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.057+0000 7fcaa86a5640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa0112020 con 0x7fcaa0103050 2024-04-21T08:46:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.057+0000 7fcaa86a5640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa0112320 con 0x7fcaa010c570 2024-04-21T08:46:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.057+0000 7fcaa5c40640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa0103970 0x7fcaa0198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.057+0000 7fcaa5c40640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa0103970 0x7fcaa0198c20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38530/0 (socket says 172.21.15.86:38530) 2024-04-21T08:46:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.057+0000 7fcaa5c40640 1 -- 172.21.15.86:0/2331930837 learned_addr learned my addr 172.21.15.86:0/2331930837 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.057+0000 7fcaa6c42640 1 --2- 172.21.15.86:0/2331930837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcaa010c570 0x7fcaa019cfa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.057+0000 7fcaa6441640 1 --2- 172.21.15.86:0/2331930837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcaa0103050 0x7fcaa01986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.057+0000 7fcaa5c40640 1 -- 172.21.15.86:0/2331930837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcaa010c570 msgr2=0x7fcaa019cfa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:59.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.057+0000 7fcaa5c40640 1 --2- 172.21.15.86:0/2331930837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcaa010c570 0x7fcaa019cfa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.057+0000 7fcaa5c40640 1 -- 172.21.15.86:0/2331930837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcaa0103050 msgr2=0x7fcaa01986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:59.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.057+0000 7fcaa5c40640 1 --2- 172.21.15.86:0/2331930837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcaa0103050 0x7fcaa01986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.057+0000 7fcaa5c40640 1 -- 172.21.15.86:0/2331930837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcaa019d6d0 con 0x7fcaa0103970 2024-04-21T08:46:59.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.057+0000 7fcaa6c42640 1 --2- 172.21.15.86:0/2331930837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcaa010c570 0x7fcaa019cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:59.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.057+0000 7fcaa5c40640 1 --2- 172.21.15.86:0/2331930837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa0103970 0x7fcaa0198c20 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fca9000d950 tx=0x7fca9000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:59.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.057+0000 7fca977fe640 1 -- 172.21.15.86:0/2331930837 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca90017070 con 0x7fcaa0103970 2024-04-21T08:46:59.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.057+0000 7fca977fe640 1 -- 172.21.15.86:0/2331930837 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca90009c30 con 0x7fcaa0103970 2024-04-21T08:46:59.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.057+0000 7fcaa86a5640 1 -- 172.21.15.86:0/2331930837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcaa019d9c0 con 0x7fcaa0103970 2024-04-21T08:46:59.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.057+0000 7fca977fe640 1 -- 172.21.15.86:0/2331930837 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9000f620 con 0x7fcaa0103970 2024-04-21T08:46:59.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.058+0000 7fcaa86a5640 1 -- 172.21.15.86:0/2331930837 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcaa01a5250 con 0x7fcaa0103970 2024-04-21T08:46:59.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.059+0000 7fca977fe640 1 -- 172.21.15.86:0/2331930837 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fca90010460 con 0x7fcaa0103970 2024-04-21T08:46:59.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.060+0000 7fcaab372740 1 Processor -- start 2024-04-21T08:46:59.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.060+0000 7fcaab372740 1 -- start start 2024-04-21T08:46:59.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.060+0000 7fcaab372740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5642322bf4d0 0x5642322bf8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.060+0000 7fcaab372740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642322bfe80 0x5642322c8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.060+0000 7fcaab372740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642322c8a00 0x5642322cae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.060+0000 7fcaab372740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5642322241f0 con 0x5642322c8a00 2024-04-21T08:46:59.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.060+0000 7fcaab372740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5642321c14b0 con 0x5642322bf4d0 2024-04-21T08:46:59.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.060+0000 7fcaab372740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5642322201b0 con 0x5642322bfe80 2024-04-21T08:46:59.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.060+0000 7fca977fe640 1 --2- 172.21.15.86:0/2331930837 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fca8c0a3d20 0x7fca8c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.060+0000 7fcaa5c40640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642322c8a00 0x5642322cae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.060+0000 7fcaa6c42640 1 --2- 172.21.15.86:0/2331930837 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fca8c0a3d20 0x7fca8c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.060+0000 7fcaa5c40640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642322c8a00 0x5642322cae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38542/0 (socket says 172.21.15.86:38542) 2024-04-21T08:46:59.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.060+0000 7fcaa6441640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642322bfe80 0x5642322c8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.060+0000 7fcaa5c40640 1 -- 172.21.15.86:0/3718979702 learned_addr learned my addr 172.21.15.86:0/3718979702 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:59.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.060+0000 7fcaa6c42640 1 --2- 172.21.15.86:0/3718979702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5642322bf4d0 0x5642322bf8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.060+0000 7fca977fe640 1 -- 172.21.15.86:0/2331930837 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fca900d0830 con 0x7fcaa0103970 2024-04-21T08:46:59.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.061+0000 7fcaa5c40640 1 -- 172.21.15.86:0/3718979702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642322bfe80 msgr2=0x5642322c8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:59.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.061+0000 7fcaa5c40640 1 --2- 172.21.15.86:0/3718979702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642322bfe80 0x5642322c8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.061+0000 7fcaa5c40640 1 -- 172.21.15.86:0/3718979702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5642322bf4d0 msgr2=0x5642322bf8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:59.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.061+0000 7fcaa5c40640 1 --2- 172.21.15.86:0/3718979702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5642322bf4d0 0x5642322bf8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.061+0000 7fcaa5c40640 1 -- 172.21.15.86:0/3718979702 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642321b31b0 con 0x5642322c8a00 2024-04-21T08:46:59.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.061+0000 7fcaa6c42640 1 --2- 172.21.15.86:0/2331930837 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fca8c0a3d20 0x7fca8c0a61e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fca98009a60 tx=0x7fca98009970 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:59.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.061+0000 7fcaa5c40640 1 --2- 172.21.15.86:0/3718979702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642322c8a00 0x5642322cae50 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fca9009c100 tx=0x7fca9006eef0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d680162b8c77607 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:59.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.061+0000 7fca94ff9640 1 -- 172.21.15.86:0/3718979702 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca90020070 con 0x5642322c8a00 2024-04-21T08:46:59.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.061+0000 7fca94ff9640 1 -- 172.21.15.86:0/3718979702 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca90066f10 con 0x5642322c8a00 2024-04-21T08:46:59.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.061+0000 7fca94ff9640 1 -- 172.21.15.86:0/3718979702 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca90098a70 con 0x5642322c8a00 2024-04-21T08:46:59.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.062+0000 7fcaab372740 1 -- 172.21.15.86:0/3718979702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642322c8a00 msgr2=0x5642322cae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:59.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.062+0000 7fcaab372740 1 --2- 172.21.15.86:0/3718979702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642322c8a00 0x5642322cae50 secure :-1 s=READY pgs=1589 cs=0 l=1 rev1=1 crypto rx=0x7fca9009c100 tx=0x7fca9006eef0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.062+0000 7fcaab372740 1 -- 172.21.15.86:0/3718979702 shutdown_connections 2024-04-21T08:46:59.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.062+0000 7fcaab372740 1 --2- 172.21.15.86:0/3718979702 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642322c8a00 0x5642322cae50 unknown :-1 s=CLOSED pgs=1589 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.062+0000 7fcaab372740 1 --2- 172.21.15.86:0/3718979702 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642322bfe80 0x5642322c8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.062+0000 7fcaab372740 1 --2- 172.21.15.86:0/3718979702 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5642322bf4d0 0x5642322bf8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.062+0000 7fcaab372740 1 -- 172.21.15.86:0/3718979702 >> 172.21.15.86:0/3718979702 conn(0x5642322b9ba0 msgr2=0x5642322b9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:59.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.062+0000 7fcaab372740 1 -- 172.21.15.86:0/3718979702 shutdown_connections 2024-04-21T08:46:59.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.062+0000 7fcaab372740 1 -- 172.21.15.86:0/3718979702 wait complete. 2024-04-21T08:46:59.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.062+0000 7fcaab372740 1 -- start start 2024-04-21T08:46:59.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.062+0000 7fcaab372740 1 Processor -- start 2024-04-21T08:46:59.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.062+0000 7fcaab372740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642322bf4d0 0x5642322c8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.062+0000 7fcaab372740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642322bfe80 0x5642322c5610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.062+0000 7fcaab372740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5642322c8a00 0x5642322c5b50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.062+0000 7fcaab372740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5642322cb8b0 con 0x5642322bf4d0 2024-04-21T08:46:59.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.062+0000 7fcaab372740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5642321dd500 con 0x5642322c8a00 2024-04-21T08:46:59.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.062+0000 7fcaab372740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5642322cba30 con 0x5642322bfe80 2024-04-21T08:46:59.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.062+0000 7fcaab372740 10 client.0 _ll_register_callbacks cb 0x5642321abb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:59.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.062+0000 7fcaa6441640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642322bf4d0 0x5642322c8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.062+0000 7fcaa6441640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642322bf4d0 0x5642322c8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38544/0 (socket says 172.21.15.86:38544) 2024-04-21T08:46:59.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.062+0000 7fcaa6441640 1 -- 172.21.15.86:0/3601891440 learned_addr learned my addr 172.21.15.86:0/3601891440 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:59.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.063+0000 7fcaa5c40640 1 --2- 172.21.15.86:0/3601891440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642322bfe80 0x5642322c5610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.063+0000 7fcaa6c42640 1 --2- 172.21.15.86:0/3601891440 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5642322c8a00 0x5642322c5b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.063+0000 7fcaa6441640 1 -- 172.21.15.86:0/3601891440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642322bfe80 msgr2=0x5642322c5610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:59.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.063+0000 7fcaa6441640 1 --2- 172.21.15.86:0/3601891440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642322bfe80 0x5642322c5610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.063+0000 7fcaa6441640 1 -- 172.21.15.86:0/3601891440 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5642322c8a00 msgr2=0x5642322c5b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:59.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.063+0000 7fcaa6441640 1 --2- 172.21.15.86:0/3601891440 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5642322c8a00 0x5642322c5b50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.063+0000 7fcaa6441640 1 -- 172.21.15.86:0/3601891440 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564231f363f0 con 0x5642322bf4d0 2024-04-21T08:46:59.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.063+0000 7fcaa6c42640 1 --2- 172.21.15.86:0/3601891440 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5642322c8a00 0x5642322c5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:59.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.063+0000 7fcaa5c40640 1 --2- 172.21.15.86:0/3601891440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642322bfe80 0x5642322c5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:59.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.063+0000 7fcaa6441640 1 --2- 172.21.15.86:0/3601891440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642322bf4d0 0x5642322c8280 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fca9c00bf00 tx=0x7fca9c012820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f76968bbf5144a43 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:59.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.063+0000 7fca89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:46:59.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.063+0000 7fca89ffb640 1 -- 172.21.15.86:0/3601891440 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca9c005220 con 0x5642322bf4d0 2024-04-21T08:46:59.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.063+0000 7fca89ffb640 1 -- 172.21.15.86:0/3601891440 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fca9c0053c0 con 0x5642322bf4d0 2024-04-21T08:46:59.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.063+0000 7fcaab372740 1 -- 172.21.15.86:0/3601891440 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca90066f10 con 0x5642322bf4d0 2024-04-21T08:46:59.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.063+0000 7fca89ffb640 1 -- 172.21.15.86:0/3601891440 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fca9c0055c0 con 0x5642322bf4d0 2024-04-21T08:46:59.073 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.063+0000 7fcaab372740 10 client.18972 fetch_fsmap learned FSMap version 29 2024-04-21T08:46:59.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.063+0000 7fcaab372740 1 -- 172.21.15.86:0/3601891440 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5642320007d0 con 0x5642322bf4d0 2024-04-21T08:46:59.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.063+0000 7fca89ffb640 1 -- 172.21.15.86:0/3601891440 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca90066f10 con 0x5642322bf4d0 2024-04-21T08:46:59.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.064+0000 7fca89ffb640 1 -- 172.21.15.86:0/3601891440 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fca9c019070 con 0x5642322bf4d0 2024-04-21T08:46:59.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.064+0000 7fcaab372740 10 client.18972 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:46:59.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.064+0000 7fcaab372740 10 client.18972 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:46:59.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.064+0000 7fcaab372740 20 client.18972 populate_metadata read hostname 'smithi086' 2024-04-21T08:46:59.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.064+0000 7fcaab372740 1 --2- 172.21.15.86:0/3601891440 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5642322d8b20 0x5642322dafa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.064+0000 7fcaab372740 4 client.18972 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:59.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.064+0000 7fcaab372740 1 -- 172.21.15.86:0/3601891440 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5642321c3930 con 0x5642322d8b20 2024-04-21T08:46:59.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.064+0000 7fcaa5c40640 1 --2- 172.21.15.86:0/3601891440 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5642322d8b20 0x5642322dafa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.064+0000 7fca89ffb640 10 client.18972 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:46:59.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.064+0000 7fcaa5c40640 1 --2- 172.21.15.86:0/3601891440 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5642322d8b20 0x5642322dafa0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=33e0147c2942185a server_cookie=5df0918ac2059a23 in_seq=0 out_seq=0 2024-04-21T08:46:59.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.065+0000 7fca89ffb640 1 -- 172.21.15.86:0/3601891440 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5642321c3930 con 0x5642322d8b20 2024-04-21T08:46:59.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.065+0000 7fca89ffb640 10 client.18972 handle_command_reply: tid=0 2024-04-21T08:46:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.065+0000 7fcaab372740 1 client.18972 shutdown 2024-04-21T08:46:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.065+0000 7fcaab372740 1 -- 172.21.15.86:0/3601891440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642322bf4d0 msgr2=0x5642322c8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.065+0000 7fcaab372740 1 --2- 172.21.15.86:0/3601891440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642322bf4d0 0x5642322c8280 secure :-1 s=READY pgs=1590 cs=0 l=1 rev1=1 crypto rx=0x7fca9c00bf00 tx=0x7fca9c012820 comp rx=0 tx=0).stop 2024-04-21T08:46:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.065+0000 7fcaab372740 1 -- 172.21.15.86:0/3601891440 shutdown_connections 2024-04-21T08:46:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.065+0000 7fcaab372740 1 --2- 172.21.15.86:0/3601891440 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5642322c8a00 0x5642322c5b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.065+0000 7fcaab372740 1 --2- 172.21.15.86:0/3601891440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642322bfe80 0x5642322c5610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.065+0000 7fcaab372740 1 --2- 172.21.15.86:0/3601891440 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5642322d8b20 0x5642322dafa0 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.065+0000 7fcaab372740 1 --2- 172.21.15.86:0/3601891440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642322bf4d0 0x5642322c8280 unknown :-1 s=CLOSED pgs=1590 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.065+0000 7fca89ffb640 0 client.18972 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:46:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.065+0000 7fcaab372740 1 -- 172.21.15.86:0/3601891440 >> 172.21.15.86:0/3601891440 conn(0x5642322b9ba0 msgr2=0x5642322bb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.065+0000 7fcaab372740 1 -- 172.21.15.86:0/3601891440 shutdown_connections 2024-04-21T08:46:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.065+0000 7fcaab372740 1 -- 172.21.15.86:0/3601891440 wait complete. 2024-04-21T08:46:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.065+0000 7fcaab372740 20 client.18972 trim_cache size 0 max 16384 2024-04-21T08:46:59.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.078+0000 7fcaab372740 1 Processor -- start 2024-04-21T08:46:59.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.078+0000 7fcaab372740 1 -- start start 2024-04-21T08:46:59.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.078+0000 7fcaab372740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642322bf4d0 0x5642322d4770 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.078+0000 7fcaab372740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5642322bfe80 0x5642322d1060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.078+0000 7fcaab372740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642322c8a00 0x5642322c7d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.078+0000 7fcaab372740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa01af3a0 con 0x5642322c8a00 2024-04-21T08:46:59.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.078+0000 7fcaab372740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa01b0ee0 con 0x5642322bfe80 2024-04-21T08:46:59.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.078+0000 7fcaab372740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa01afa70 con 0x5642322bf4d0 2024-04-21T08:46:59.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.078+0000 7fcaa6c42640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642322c8a00 0x5642322c7d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.078+0000 7fcaa6c42640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642322c8a00 0x5642322c7d40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38556/0 (socket says 172.21.15.86:38556) 2024-04-21T08:46:59.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.078+0000 7fcaa6c42640 1 -- 172.21.15.86:0/3618944102 learned_addr learned my addr 172.21.15.86:0/3618944102 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:59.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.078+0000 7fcaa5c40640 1 --2- 172.21.15.86:0/3618944102 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5642322bfe80 0x5642322d1060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.079+0000 7fcaa6441640 1 --2- 172.21.15.86:0/3618944102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642322bf4d0 0x5642322d4770 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.079+0000 7fcaa6c42640 1 -- 172.21.15.86:0/3618944102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642322bf4d0 msgr2=0x5642322d4770 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:59.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.079+0000 7fcaa6c42640 1 --2- 172.21.15.86:0/3618944102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642322bf4d0 0x5642322d4770 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.079+0000 7fcaa6c42640 1 -- 172.21.15.86:0/3618944102 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5642322bfe80 msgr2=0x5642322d1060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:59.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.079+0000 7fcaa6c42640 1 --2- 172.21.15.86:0/3618944102 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5642322bfe80 0x5642322d1060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.079+0000 7fcaa6c42640 1 -- 172.21.15.86:0/3618944102 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5642322cb950 con 0x5642322c8a00 2024-04-21T08:46:59.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.079+0000 7fcaa6441640 1 --2- 172.21.15.86:0/3618944102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642322bf4d0 0x5642322d4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:59.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.079+0000 7fcaa5c40640 1 --2- 172.21.15.86:0/3618944102 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5642322bfe80 0x5642322d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:46:59.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.079+0000 7fcaa6c42640 1 --2- 172.21.15.86:0/3618944102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642322c8a00 0x5642322c7d40 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fca98009ca0 tx=0x7fca98039000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8dada81c513b8a9c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:59.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.079+0000 7fca8bfff640 1 -- 172.21.15.86:0/3618944102 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca98049050 con 0x5642322c8a00 2024-04-21T08:46:59.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.079+0000 7fca8bfff640 1 -- 172.21.15.86:0/3618944102 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca98053070 con 0x5642322c8a00 2024-04-21T08:46:59.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.079+0000 7fca8bfff640 1 -- 172.21.15.86:0/3618944102 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca98039070 con 0x5642322c8a00 2024-04-21T08:46:59.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.079+0000 7fcaab372740 1 -- 172.21.15.86:0/3618944102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642322c8a00 msgr2=0x5642322c7d40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.079+0000 7fcaab372740 1 --2- 172.21.15.86:0/3618944102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642322c8a00 0x5642322c7d40 secure :-1 s=READY pgs=1591 cs=0 l=1 rev1=1 crypto rx=0x7fca98009ca0 tx=0x7fca98039000 comp rx=0 tx=0).stop 2024-04-21T08:46:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.080+0000 7fcaab372740 1 -- 172.21.15.86:0/3618944102 shutdown_connections 2024-04-21T08:46:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.080+0000 7fcaab372740 1 --2- 172.21.15.86:0/3618944102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642322c8a00 0x5642322c7d40 unknown :-1 s=CLOSED pgs=1591 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.080+0000 7fcaab372740 1 --2- 172.21.15.86:0/3618944102 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5642322bfe80 0x5642322d1060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.080+0000 7fcaab372740 1 --2- 172.21.15.86:0/3618944102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642322bf4d0 0x5642322d4770 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.080+0000 7fcaab372740 1 -- 172.21.15.86:0/3618944102 >> 172.21.15.86:0/3618944102 conn(0x5642322b9ba0 msgr2=0x5642322b8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.080+0000 7fcaab372740 1 -- 172.21.15.86:0/3618944102 shutdown_connections 2024-04-21T08:46:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.080+0000 7fcaab372740 1 -- 172.21.15.86:0/3618944102 wait complete. 2024-04-21T08:46:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.080+0000 7fcaab372740 1 -- start start 2024-04-21T08:46:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.080+0000 7fcaab372740 1 Processor -- start 2024-04-21T08:46:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.080+0000 7fcaab372740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5642322bf4d0 0x56423233baf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.080+0000 7fcaab372740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642322bfe80 0x56423233c030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.080+0000 7fcaab372740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642322c8a00 0x5642322c75d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.080+0000 7fcaab372740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa01b0810 con 0x5642322c8a00 2024-04-21T08:46:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.080+0000 7fcaab372740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa01b0140 con 0x5642322bf4d0 2024-04-21T08:46:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.080+0000 7fcaab372740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fcaa01ad3a0 con 0x5642322bfe80 2024-04-21T08:46:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.080+0000 7fcaab372740 10 client.0 _ll_register_callbacks cb 0x7fca9c00b9d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.080+0000 7fcaa6441640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5642322bf4d0 0x56423233baf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.080+0000 7fcaa5c40640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642322bfe80 0x56423233c030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.080+0000 7fcaa6c42640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642322c8a00 0x5642322c75d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.080+0000 7fcaa5c40640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642322bfe80 0x56423233c030 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.86:54240/0 (socket says 172.21.15.86:54240) 2024-04-21T08:46:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.080+0000 7fcaa6441640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5642322bf4d0 0x56423233baf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.86:58060/0 (socket says 172.21.15.86:58060) 2024-04-21T08:46:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.080+0000 7fcaa5c40640 1 -- 172.21.15.86:0/376032773 learned_addr learned my addr 172.21.15.86:0/376032773 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.081+0000 7fcaa5c40640 1 -- 172.21.15.86:0/376032773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5642322bf4d0 msgr2=0x56423233baf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.081+0000 7fcaa5c40640 1 --2- 172.21.15.86:0/376032773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5642322bf4d0 0x56423233baf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.081+0000 7fcaa5c40640 1 -- 172.21.15.86:0/376032773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642322c8a00 msgr2=0x5642322c75d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:59.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.081+0000 7fcaa5c40640 1 --2- 172.21.15.86:0/376032773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642322c8a00 0x5642322c75d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.081+0000 7fcaa5c40640 1 -- 172.21.15.86:0/376032773 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5642322c7cc0 con 0x5642322bfe80 2024-04-21T08:46:59.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.081+0000 7fcaa6c42640 1 --2- 172.21.15.86:0/376032773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642322c8a00 0x5642322c75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:46:59.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.081+0000 7fcaa6441640 1 --2- 172.21.15.86:0/376032773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5642322bf4d0 0x56423233baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:46:59.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.081+0000 7fcaa5c40640 1 --2- 172.21.15.86:0/376032773 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642322bfe80 0x56423233c030 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fca9009c5c0 tx=0x7fca90005880 comp rx=0 tx=0).ready entity=mon.1 client_cookie=4b4c6c06df6022dc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:59.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.081+0000 7fca89ffb640 10 client.0 ms_handle_connect on v2:172.21.15.179:3300/0 2024-04-21T08:46:59.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.081+0000 7fca89ffb640 1 -- 172.21.15.86:0/376032773 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca900b0070 con 0x5642322bfe80 2024-04-21T08:46:59.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.081+0000 7fcaab372740 1 -- 172.21.15.86:0/376032773 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fca98053070 con 0x5642322bfe80 2024-04-21T08:46:59.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.081+0000 7fca89ffb640 1 -- 172.21.15.86:0/376032773 <== mon.1 v2:172.21.15.179:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fca90098a70 con 0x5642322bfe80 2024-04-21T08:46:59.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.081+0000 7fca89ffb640 1 -- 172.21.15.86:0/376032773 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fca900b5050 con 0x5642322bfe80 2024-04-21T08:46:59.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.081+0000 7fcaab372740 10 client.27094 fetch_fsmap learned FSMap version 29 2024-04-21T08:46:59.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.081+0000 7fcaab372740 1 -- 172.21.15.86:0/376032773 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5642323432c0 con 0x5642322bfe80 2024-04-21T08:46:59.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.081+0000 7fca89ffb640 1 -- 172.21.15.86:0/376032773 <== mon.1 v2:172.21.15.179:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca98053070 con 0x5642322bfe80 2024-04-21T08:46:59.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.081+0000 7fca89ffb640 1 -- 172.21.15.86:0/376032773 <== mon.1 v2:172.21.15.179:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fca9007bce0 con 0x5642322bfe80 2024-04-21T08:46:59.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.081+0000 7fcaab372740 10 client.27094 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:46:59.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.081+0000 7fcaab372740 10 client.27094 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:46:59.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.081+0000 7fcaab372740 20 client.27094 populate_metadata read hostname 'smithi086' 2024-04-21T08:46:59.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.081+0000 7fcaab372740 1 --2- 172.21.15.86:0/376032773 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5642323435b0 0x5642323459a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.081+0000 7fcaab372740 4 client.27094 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:46:59.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.081+0000 7fcaab372740 1 -- 172.21.15.86:0/376032773 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564232231c10 con 0x5642323435b0 2024-04-21T08:46:59.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.082+0000 7fcaa6441640 1 --2- 172.21.15.86:0/376032773 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5642323435b0 0x5642323459a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.082+0000 7fcaa6441640 1 --2- 172.21.15.86:0/376032773 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5642323435b0 0x5642323459a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=4d51c5c3b1d78aeb server_cookie=dde35ffefddcb0cd in_seq=0 out_seq=0 2024-04-21T08:46:59.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.082+0000 7fca89ffb640 10 client.27094 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:46:59.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.082+0000 7fca89ffb640 1 -- 172.21.15.86:0/376032773 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564232231c10 con 0x5642323435b0 2024-04-21T08:46:59.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.082+0000 7fca89ffb640 10 client.27094 handle_command_reply: tid=0 2024-04-21T08:46:59.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.082+0000 7fcaab372740 1 client.27094 shutdown 2024-04-21T08:46:59.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.082+0000 7fcaab372740 1 -- 172.21.15.86:0/376032773 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642322bfe80 msgr2=0x56423233c030 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:59.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.082+0000 7fcaab372740 1 --2- 172.21.15.86:0/376032773 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642322bfe80 0x56423233c030 secure :-1 s=READY pgs=71 cs=0 l=1 rev1=1 crypto rx=0x7fca9009c5c0 tx=0x7fca90005880 comp rx=0 tx=0).stop 2024-04-21T08:46:59.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.082+0000 7fcaab372740 1 -- 172.21.15.86:0/376032773 shutdown_connections 2024-04-21T08:46:59.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.082+0000 7fcaab372740 1 --2- 172.21.15.86:0/376032773 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5642323435b0 0x5642323459a0 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.082+0000 7fcaab372740 1 --2- 172.21.15.86:0/376032773 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5642322c8a00 0x5642322c75d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.082+0000 7fcaab372740 1 --2- 172.21.15.86:0/376032773 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5642322bfe80 0x56423233c030 unknown :-1 s=CLOSED pgs=71 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.082+0000 7fca89ffb640 0 client.27094 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:46:59.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.082+0000 7fcaab372740 1 --2- 172.21.15.86:0/376032773 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5642322bf4d0 0x56423233baf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.082+0000 7fcaab372740 1 -- 172.21.15.86:0/376032773 >> 172.21.15.86:0/376032773 conn(0x5642322b9ba0 msgr2=0x5642322c0260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:59.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.082+0000 7fcaab372740 1 -- 172.21.15.86:0/376032773 shutdown_connections 2024-04-21T08:46:59.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.082+0000 7fcaab372740 1 -- 172.21.15.86:0/376032773 wait complete. 2024-04-21T08:46:59.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.082+0000 7fcaab372740 20 client.27094 trim_cache size 0 max 16384 2024-04-21T08:46:59.090 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:46:59.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.083+0000 7fcaa86a5640 1 -- 172.21.15.86:0/2331930837 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fca8c0a3d20 msgr2=0x7fca8c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:59.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.083+0000 7fcaa86a5640 1 --2- 172.21.15.86:0/2331930837 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fca8c0a3d20 0x7fca8c0a61e0 secure :-1 s=READY pgs=258 cs=0 l=1 rev1=1 crypto rx=0x7fca98009a60 tx=0x7fca98009970 comp rx=0 tx=0).stop 2024-04-21T08:46:59.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.083+0000 7fcaa86a5640 1 -- 172.21.15.86:0/2331930837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa0103970 msgr2=0x7fcaa0198c20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:59.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.083+0000 7fcaa86a5640 1 --2- 172.21.15.86:0/2331930837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa0103970 0x7fcaa0198c20 secure :-1 s=READY pgs=1588 cs=0 l=1 rev1=1 crypto rx=0x7fca9000d950 tx=0x7fca9000de10 comp rx=0 tx=0).stop 2024-04-21T08:46:59.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.083+0000 7fcaa86a5640 1 -- 172.21.15.86:0/2331930837 shutdown_connections 2024-04-21T08:46:59.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.083+0000 7fcaa86a5640 1 --2- 172.21.15.86:0/2331930837 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fca8c0a3d20 0x7fca8c0a61e0 unknown :-1 s=CLOSED pgs=258 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.083+0000 7fcaa86a5640 1 --2- 172.21.15.86:0/2331930837 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcaa010c570 0x7fcaa019cfa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.083+0000 7fcaa86a5640 1 --2- 172.21.15.86:0/2331930837 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcaa0103970 0x7fcaa0198c20 unknown :-1 s=CLOSED pgs=1588 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.083+0000 7fcaa86a5640 1 --2- 172.21.15.86:0/2331930837 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcaa0103050 0x7fcaa01986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.083+0000 7fcaa86a5640 1 -- 172.21.15.86:0/2331930837 >> 172.21.15.86:0/2331930837 conn(0x7fcaa0077fb0 msgr2=0x7fcaa00fdfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:59.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.083+0000 7fcaa86a5640 1 -- 172.21.15.86:0/2331930837 shutdown_connections 2024-04-21T08:46:59.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.083+0000 7fcaa86a5640 1 -- 172.21.15.86:0/2331930837 wait complete. 2024-04-21T08:46:59.093 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:58 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:46:59.093 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:58 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/4037676148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:46:59.093 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:58 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:46:59.093 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:58 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:46:59.093 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:58 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:46:59.094 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:46:59.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.193+0000 7f028187b640 1 -- 172.21.15.86:0/141454629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f027c106160 msgr2=0x7f027c10dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:59.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.193+0000 7f028187b640 1 --2- 172.21.15.86:0/141454629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f027c106160 0x7f027c10dd30 secure :-1 s=READY pgs=1592 cs=0 l=1 rev1=1 crypto rx=0x7f0274009ca0 tx=0x7f027402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.193+0000 7f028187b640 1 -- 172.21.15.86:0/141454629 shutdown_connections 2024-04-21T08:46:59.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.193+0000 7f028187b640 1 --2- 172.21.15.86:0/141454629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f027c106160 0x7f027c10dd30 unknown :-1 s=CLOSED pgs=1592 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.193+0000 7f028187b640 1 --2- 172.21.15.86:0/141454629 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f027c101a40 0x7f027c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.193+0000 7f028187b640 1 --2- 172.21.15.86:0/141454629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f027c101090 0x7f027c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.193+0000 7f028187b640 1 -- 172.21.15.86:0/141454629 >> 172.21.15.86:0/141454629 conn(0x7f027c077f50 msgr2=0x7f027c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:59.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.193+0000 7f028187b640 1 -- 172.21.15.86:0/141454629 shutdown_connections 2024-04-21T08:46:59.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.193+0000 7f028187b640 1 -- 172.21.15.86:0/141454629 wait complete. 2024-04-21T08:46:59.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.194+0000 7f028187b640 1 Processor -- start 2024-04-21T08:46:59.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.194+0000 7f028187b640 1 -- start start 2024-04-21T08:46:59.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.194+0000 7f028187b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f027c101090 0x7f027c198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.194+0000 7f028187b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f027c106160 0x7f027c198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.194+0000 7f028187b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f027c19cf90 0x7f027c19d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.194+0000 7f028187b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f027c1110b0 con 0x7f027c19cf90 2024-04-21T08:46:59.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.194+0000 7f028187b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f027c110f30 con 0x7f027c101090 2024-04-21T08:46:59.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.194+0000 7f028187b640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f027c111230 con 0x7f027c106160 2024-04-21T08:46:59.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.194+0000 7f027b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f027c19cf90 0x7f027c19d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.194+0000 7f027b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f027c19cf90 0x7f027c19d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38574/0 (socket says 172.21.15.86:38574) 2024-04-21T08:46:59.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.194+0000 7f027b7fe640 1 -- 172.21.15.86:0/2025183124 learned_addr learned my addr 172.21.15.86:0/2025183124 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:59.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.194+0000 7f027affd640 1 --2- 172.21.15.86:0/2025183124 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f027c101090 0x7f027c198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.194+0000 7f0273fff640 1 --2- 172.21.15.86:0/2025183124 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f027c106160 0x7f027c198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.194+0000 7f027b7fe640 1 -- 172.21.15.86:0/2025183124 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f027c106160 msgr2=0x7f027c198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:59.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.194+0000 7f027b7fe640 1 --2- 172.21.15.86:0/2025183124 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f027c106160 0x7f027c198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.194+0000 7f027b7fe640 1 -- 172.21.15.86:0/2025183124 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f027c101090 msgr2=0x7f027c198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:59.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.194+0000 7f027b7fe640 1 --2- 172.21.15.86:0/2025183124 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f027c101090 0x7f027c198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.194+0000 7f027b7fe640 1 -- 172.21.15.86:0/2025183124 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f027c19da80 con 0x7f027c19cf90 2024-04-21T08:46:59.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.194+0000 7f027affd640 1 --2- 172.21.15.86:0/2025183124 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f027c101090 0x7f027c198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:46:59.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.195+0000 7f027b7fe640 1 --2- 172.21.15.86:0/2025183124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f027c19cf90 0x7f027c19d440 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f0274005e30 tx=0x7f0274032420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:59.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.195+0000 7f0280879640 1 -- 172.21.15.86:0/2025183124 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0274041020 con 0x7f027c19cf90 2024-04-21T08:46:59.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.195+0000 7f0280879640 1 -- 172.21.15.86:0/2025183124 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0274032970 con 0x7f027c19cf90 2024-04-21T08:46:59.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.195+0000 7f028187b640 1 -- 172.21.15.86:0/2025183124 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f027c19dd10 con 0x7f027c19cf90 2024-04-21T08:46:59.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.195+0000 7f0280879640 1 -- 172.21.15.86:0/2025183124 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0274039040 con 0x7f027c19cf90 2024-04-21T08:46:59.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.195+0000 7f028187b640 1 -- 172.21.15.86:0/2025183124 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f027c1a54d0 con 0x7f027c19cf90 2024-04-21T08:46:59.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.197+0000 7f0280879640 1 -- 172.21.15.86:0/2025183124 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0274044020 con 0x7f027c19cf90 2024-04-21T08:46:59.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.197+0000 7f0284472740 1 Processor -- start 2024-04-21T08:46:59.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.197+0000 7f0284472740 1 -- start start 2024-04-21T08:46:59.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.197+0000 7f0284472740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579ae8804d0 0x5579ae8808b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.197+0000 7f0284472740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579ae880e80 0x5579ae889430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.197+0000 7f0284472740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579ae889a00 0x5579ae88be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.197+0000 7f0284472740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5579ae7e51f0 con 0x5579ae8804d0 2024-04-21T08:46:59.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.197+0000 7f0284472740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5579ae7824b0 con 0x5579ae889a00 2024-04-21T08:46:59.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.197+0000 7f0284472740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5579ae7e11b0 con 0x5579ae880e80 2024-04-21T08:46:59.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.197+0000 7f0273fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579ae8804d0 0x5579ae8808b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.198+0000 7f0273fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579ae8804d0 0x5579ae8808b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38584/0 (socket says 172.21.15.86:38584) 2024-04-21T08:46:59.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.198+0000 7f0280879640 1 --2- 172.21.15.86:0/2025183124 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f02680a3d20 0x7f02680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.198+0000 7f0273fff640 1 -- 172.21.15.86:0/1258986279 learned_addr learned my addr 172.21.15.86:0/1258986279 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:59.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.198+0000 7f027b7fe640 1 --2- 172.21.15.86:0/2025183124 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f02680a3d20 0x7f02680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.198+0000 7f0273fff640 1 --2- 172.21.15.86:0/1258986279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579ae889a00 0x5579ae88be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.198+0000 7f027affd640 1 --2- 172.21.15.86:0/1258986279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579ae880e80 0x5579ae889430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.198+0000 7f0273fff640 1 -- 172.21.15.86:0/1258986279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579ae880e80 msgr2=0x5579ae889430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:59.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.198+0000 7f0273fff640 1 --2- 172.21.15.86:0/1258986279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579ae880e80 0x5579ae889430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.198+0000 7f0273fff640 1 -- 172.21.15.86:0/1258986279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579ae889a00 msgr2=0x5579ae88be50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:59.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.198+0000 7f0273fff640 1 --2- 172.21.15.86:0/1258986279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579ae889a00 0x5579ae88be50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.198+0000 7f0280879640 1 -- 172.21.15.86:0/2025183124 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f02740f2920 con 0x7f027c19cf90 2024-04-21T08:46:59.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.198+0000 7f0273fff640 1 -- 172.21.15.86:0/1258986279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579ae7741b0 con 0x5579ae8804d0 2024-04-21T08:46:59.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.198+0000 7f027affd640 1 --2- 172.21.15.86:0/1258986279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579ae880e80 0x5579ae889430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:59.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.198+0000 7f0273fff640 1 --2- 172.21.15.86:0/1258986279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579ae8804d0 0x5579ae8808b0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f0264009fc0 tx=0x7f0264009700 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3525d2a1b7ec4fb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:59.206 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.198+0000 7f027b7fe640 1 --2- 172.21.15.86:0/2025183124 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f02680a3d20 0x7f02680a61e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f02740910c0 tx=0x7f02740a9040 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:59.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.198+0000 7f0271ffb640 1 -- 172.21.15.86:0/1258986279 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0264004050 con 0x5579ae8804d0 2024-04-21T08:46:59.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.198+0000 7f0271ffb640 1 -- 172.21.15.86:0/1258986279 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02640041f0 con 0x5579ae8804d0 2024-04-21T08:46:59.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.198+0000 7f0271ffb640 1 -- 172.21.15.86:0/1258986279 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0264011470 con 0x5579ae8804d0 2024-04-21T08:46:59.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.199+0000 7f0284472740 1 -- 172.21.15.86:0/1258986279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579ae8804d0 msgr2=0x5579ae8808b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:59.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.199+0000 7f0284472740 1 --2- 172.21.15.86:0/1258986279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579ae8804d0 0x5579ae8808b0 secure :-1 s=READY pgs=1594 cs=0 l=1 rev1=1 crypto rx=0x7f0264009fc0 tx=0x7f0264009700 comp rx=0 tx=0).stop 2024-04-21T08:46:59.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.199+0000 7f0284472740 1 -- 172.21.15.86:0/1258986279 shutdown_connections 2024-04-21T08:46:59.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.199+0000 7f0284472740 1 --2- 172.21.15.86:0/1258986279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579ae889a00 0x5579ae88be50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.199+0000 7f0284472740 1 --2- 172.21.15.86:0/1258986279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579ae880e80 0x5579ae889430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.199+0000 7f0284472740 1 --2- 172.21.15.86:0/1258986279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579ae8804d0 0x5579ae8808b0 unknown :-1 s=CLOSED pgs=1594 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.199+0000 7f0284472740 1 -- 172.21.15.86:0/1258986279 >> 172.21.15.86:0/1258986279 conn(0x5579ae87aba0 msgr2=0x5579ae87af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:59.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.199+0000 7f0284472740 1 -- 172.21.15.86:0/1258986279 shutdown_connections 2024-04-21T08:46:59.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.199+0000 7f0284472740 1 -- 172.21.15.86:0/1258986279 wait complete. 2024-04-21T08:46:59.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.199+0000 7f0284472740 1 -- start start 2024-04-21T08:46:59.207 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.199+0000 7f0284472740 1 Processor -- start 2024-04-21T08:46:59.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.200+0000 7f0284472740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579ae8804d0 0x5579ae889330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.200+0000 7f0284472740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579ae880e80 0x5579ae8866c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.200+0000 7f0284472740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579ae889a00 0x5579ae886cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.200+0000 7f0284472740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5579ae88c970 con 0x5579ae880e80 2024-04-21T08:46:59.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.200+0000 7f0284472740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5579ae79e500 con 0x5579ae8804d0 2024-04-21T08:46:59.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.200+0000 7f0284472740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5579ae88caf0 con 0x5579ae889a00 2024-04-21T08:46:59.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.200+0000 7f0284472740 10 client.0 _ll_register_callbacks cb 0x5579ae76cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:59.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.200+0000 7f027affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579ae880e80 0x5579ae8866c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.200+0000 7f027affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579ae880e80 0x5579ae8866c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38592/0 (socket says 172.21.15.86:38592) 2024-04-21T08:46:59.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.200+0000 7f027affd640 1 -- 172.21.15.86:0/1012990265 learned_addr learned my addr 172.21.15.86:0/1012990265 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:59.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.200+0000 7f0273fff640 1 --2- 172.21.15.86:0/1012990265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579ae889a00 0x5579ae886cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.200+0000 7f0273fff640 1 --2- 172.21.15.86:0/1012990265 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579ae8804d0 0x5579ae889330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.200+0000 7f027affd640 1 -- 172.21.15.86:0/1012990265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579ae889a00 msgr2=0x5579ae886cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:59.208 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.200+0000 7f027affd640 1 --2- 172.21.15.86:0/1012990265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579ae889a00 0x5579ae886cd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.200+0000 7f027affd640 1 -- 172.21.15.86:0/1012990265 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579ae8804d0 msgr2=0x5579ae889330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:59.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.200+0000 7f027affd640 1 --2- 172.21.15.86:0/1012990265 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579ae8804d0 0x5579ae889330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.200+0000 7f027affd640 1 -- 172.21.15.86:0/1012990265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579ae4f73f0 con 0x5579ae880e80 2024-04-21T08:46:59.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.200+0000 7f0273fff640 1 --2- 172.21.15.86:0/1012990265 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579ae8804d0 0x5579ae889330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:46:59.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.200+0000 7f027affd640 1 --2- 172.21.15.86:0/1012990265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579ae880e80 0x5579ae8866c0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f026c0106d0 tx=0x7f026c010b90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f7bda57ca8849d64 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:59.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.200+0000 7f025affd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:46:59.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.200+0000 7f025affd640 1 -- 172.21.15.86:0/1012990265 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026c017070 con 0x5579ae880e80 2024-04-21T08:46:59.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.200+0000 7f025affd640 1 -- 172.21.15.86:0/1012990265 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f026c00eb30 con 0x5579ae880e80 2024-04-21T08:46:59.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.200+0000 7f0284472740 1 -- 172.21.15.86:0/1012990265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f02640041f0 con 0x5579ae880e80 2024-04-21T08:46:59.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.200+0000 7f025affd640 1 -- 172.21.15.86:0/1012990265 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f026c00ed30 con 0x5579ae880e80 2024-04-21T08:46:59.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.200+0000 7f0284472740 10 client.18990 fetch_fsmap learned FSMap version 29 2024-04-21T08:46:59.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.200+0000 7f0284472740 1 -- 172.21.15.86:0/1012990265 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5579ae5c17d0 con 0x5579ae880e80 2024-04-21T08:46:59.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.201+0000 7f025affd640 1 -- 172.21.15.86:0/1012990265 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f02640041f0 con 0x5579ae880e80 2024-04-21T08:46:59.209 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.201+0000 7f025affd640 1 -- 172.21.15.86:0/1012990265 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f026c01b410 con 0x5579ae880e80 2024-04-21T08:46:59.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.201+0000 7f0284472740 10 client.18990 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:46:59.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.201+0000 7f0284472740 10 client.18990 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:59.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.201+0000 7f0284472740 20 client.18990 populate_metadata read hostname 'smithi086' 2024-04-21T08:46:59.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.201+0000 7f0284472740 1 --2- 172.21.15.86:0/1012990265 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5579ae899c30 0x5579ae89c090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.201+0000 7f0284472740 4 client.18990 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:46:59.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.201+0000 7f0284472740 1 -- 172.21.15.86:0/1012990265 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5579ae784930 con 0x5579ae899c30 2024-04-21T08:46:59.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.201+0000 7f0273fff640 1 --2- 172.21.15.86:0/1012990265 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5579ae899c30 0x5579ae89c090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.201+0000 7f0273fff640 1 --2- 172.21.15.86:0/1012990265 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5579ae899c30 0x5579ae89c090 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a7e89ed492871ec server_cookie=53060a1696e9c9c3 in_seq=0 out_seq=0 2024-04-21T08:46:59.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.201+0000 7f025affd640 10 client.18990 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:59.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.202+0000 7f025affd640 1 -- 172.21.15.86:0/1012990265 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5579ae784930 con 0x5579ae899c30 2024-04-21T08:46:59.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.202+0000 7f025affd640 10 client.18990 handle_command_reply: tid=0 2024-04-21T08:46:59.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.202+0000 7f0284472740 1 client.18990 shutdown 2024-04-21T08:46:59.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.202+0000 7f0284472740 1 -- 172.21.15.86:0/1012990265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579ae880e80 msgr2=0x5579ae8866c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:59.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.202+0000 7f0284472740 1 --2- 172.21.15.86:0/1012990265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579ae880e80 0x5579ae8866c0 secure :-1 s=READY pgs=1595 cs=0 l=1 rev1=1 crypto rx=0x7f026c0106d0 tx=0x7f026c010b90 comp rx=0 tx=0).stop 2024-04-21T08:46:59.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.202+0000 7f0284472740 1 -- 172.21.15.86:0/1012990265 shutdown_connections 2024-04-21T08:46:59.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.202+0000 7f0284472740 1 --2- 172.21.15.86:0/1012990265 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5579ae899c30 0x5579ae89c090 crc :-1 s=READY pgs=245 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.202+0000 7f0284472740 1 --2- 172.21.15.86:0/1012990265 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579ae889a00 0x5579ae886cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.202+0000 7f0284472740 1 --2- 172.21.15.86:0/1012990265 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579ae880e80 0x5579ae8866c0 unknown :-1 s=CLOSED pgs=1595 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.202+0000 7f025affd640 0 client.18990 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:59.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.202+0000 7f0284472740 1 --2- 172.21.15.86:0/1012990265 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579ae8804d0 0x5579ae889330 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.202+0000 7f0284472740 1 -- 172.21.15.86:0/1012990265 >> 172.21.15.86:0/1012990265 conn(0x5579ae87aba0 msgr2=0x5579ae87c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:59.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.202+0000 7f0284472740 1 -- 172.21.15.86:0/1012990265 shutdown_connections 2024-04-21T08:46:59.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.203+0000 7f0284472740 1 -- 172.21.15.86:0/1012990265 wait complete. 2024-04-21T08:46:59.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.203+0000 7f0284472740 20 client.18990 trim_cache size 0 max 16384 2024-04-21T08:46:59.213 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:58 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:46:59.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:58 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/4037676148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:46:59.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:58 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:46:59.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:58 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:46:59.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:58 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:46:59.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.215+0000 7f0284472740 1 Processor -- start 2024-04-21T08:46:59.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.215+0000 7f0284472740 1 -- start start 2024-04-21T08:46:59.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.216+0000 7f0284472740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579ae8804d0 0x5579ae895860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.216+0000 7f0284472740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579ae880e80 0x5579ae892150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.216+0000 7f0284472740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579ae889a00 0x5579ae87c6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.216+0000 7f0284472740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f027c1af310 con 0x5579ae8804d0 2024-04-21T08:46:59.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.216+0000 7f0284472740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f027c1b0e50 con 0x5579ae889a00 2024-04-21T08:46:59.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.216+0000 7f0284472740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f027c1af9e0 con 0x5579ae880e80 2024-04-21T08:46:59.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.216+0000 7f0273fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579ae8804d0 0x5579ae895860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.216+0000 7f0273fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579ae8804d0 0x5579ae895860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38602/0 (socket says 172.21.15.86:38602) 2024-04-21T08:46:59.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.216+0000 7f0273fff640 1 -- 172.21.15.86:0/1023713994 learned_addr learned my addr 172.21.15.86:0/1023713994 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:59.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.216+0000 7f027affd640 1 --2- 172.21.15.86:0/1023713994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579ae880e80 0x5579ae892150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.216+0000 7f0273fff640 1 -- 172.21.15.86:0/1023713994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579ae880e80 msgr2=0x5579ae892150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:59.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.216+0000 7f0273fff640 1 --2- 172.21.15.86:0/1023713994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579ae880e80 0x5579ae892150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.216+0000 7f0273fff640 1 -- 172.21.15.86:0/1023713994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579ae889a00 msgr2=0x5579ae87c6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:59.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.216+0000 7f0273fff640 1 --2- 172.21.15.86:0/1023713994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579ae889a00 0x5579ae87c6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.216+0000 7f0273fff640 1 -- 172.21.15.86:0/1023713994 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5579ae887040 con 0x5579ae8804d0 2024-04-21T08:46:59.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.216+0000 7f0273fff640 1 --2- 172.21.15.86:0/1023713994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579ae8804d0 0x5579ae895860 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f0264003c90 tx=0x7f0264003cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c2d3e94ec1b06af server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:59.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.216+0000 7f02717fa640 1 -- 172.21.15.86:0/1023713994 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0264031070 con 0x5579ae8804d0 2024-04-21T08:46:59.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.216+0000 7f02717fa640 1 -- 172.21.15.86:0/1023713994 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f026402c450 con 0x5579ae8804d0 2024-04-21T08:46:59.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.216+0000 7f02717fa640 1 -- 172.21.15.86:0/1023713994 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0264005020 con 0x5579ae8804d0 2024-04-21T08:46:59.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.217+0000 7f0284472740 1 -- 172.21.15.86:0/1023713994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579ae8804d0 msgr2=0x5579ae895860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:59.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.217+0000 7f0284472740 1 --2- 172.21.15.86:0/1023713994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579ae8804d0 0x5579ae895860 secure :-1 s=READY pgs=1596 cs=0 l=1 rev1=1 crypto rx=0x7f0264003c90 tx=0x7f0264003cc0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.217+0000 7f0284472740 1 -- 172.21.15.86:0/1023713994 shutdown_connections 2024-04-21T08:46:59.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.217+0000 7f0284472740 1 --2- 172.21.15.86:0/1023713994 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579ae889a00 0x5579ae87c6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.217+0000 7f0284472740 1 --2- 172.21.15.86:0/1023713994 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579ae880e80 0x5579ae892150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.217+0000 7f0284472740 1 --2- 172.21.15.86:0/1023713994 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579ae8804d0 0x5579ae895860 unknown :-1 s=CLOSED pgs=1596 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.217+0000 7f0284472740 1 -- 172.21.15.86:0/1023713994 >> 172.21.15.86:0/1023713994 conn(0x5579ae87aba0 msgr2=0x5579ae87bfb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:59.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.217+0000 7f0284472740 1 -- 172.21.15.86:0/1023713994 shutdown_connections 2024-04-21T08:46:59.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.217+0000 7f0284472740 1 -- 172.21.15.86:0/1023713994 wait complete. 2024-04-21T08:46:59.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.217+0000 7f0284472740 1 -- start start 2024-04-21T08:46:59.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.217+0000 7f0284472740 1 Processor -- start 2024-04-21T08:46:59.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.218+0000 7f0284472740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579ae8804d0 0x5579ae8f8190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.218+0000 7f0284472740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579ae880e80 0x5579ae8f86d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.218+0000 7f0284472740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579ae889a00 0x5579ae8f8c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.218+0000 7f0284472740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f027c1b0780 con 0x5579ae880e80 2024-04-21T08:46:59.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.218+0000 7f0284472740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f027c1b00b0 con 0x5579ae889a00 2024-04-21T08:46:59.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.218+0000 7f0284472740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f027c1ad310 con 0x5579ae8804d0 2024-04-21T08:46:59.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.218+0000 7f0284472740 10 client.0 _ll_register_callbacks cb 0x7f026c00bd40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:46:59.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.218+0000 7f027affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579ae880e80 0x5579ae8f86d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.218+0000 7f027affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579ae880e80 0x5579ae8f86d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38614/0 (socket says 172.21.15.86:38614) 2024-04-21T08:46:59.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.218+0000 7f027affd640 1 -- 172.21.15.86:0/3781125291 learned_addr learned my addr 172.21.15.86:0/3781125291 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:46:59.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.218+0000 7f0273fff640 1 --2- 172.21.15.86:0/3781125291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579ae8804d0 0x5579ae8f8190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.218+0000 7f027affd640 1 -- 172.21.15.86:0/3781125291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579ae8804d0 msgr2=0x5579ae8f8190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:59.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.218+0000 7f027affd640 1 --2- 172.21.15.86:0/3781125291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579ae8804d0 0x5579ae8f8190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.218+0000 7f027affd640 1 -- 172.21.15.86:0/3781125291 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579ae889a00 msgr2=0x5579ae8f8c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:46:59.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.218+0000 7f027affd640 1 --2- 172.21.15.86:0/3781125291 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579ae889a00 0x5579ae8f8c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.218+0000 7f027affd640 1 -- 172.21.15.86:0/3781125291 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5579ae9034e0 con 0x5579ae880e80 2024-04-21T08:46:59.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.218+0000 7f027affd640 1 --2- 172.21.15.86:0/3781125291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579ae880e80 0x5579ae8f86d0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x5579ae887c90 tx=0x7f026c01b640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=45e705ce6cc1a77e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:46:59.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.218+0000 7f025affd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:46:59.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.218+0000 7f025affd640 1 -- 172.21.15.86:0/3781125291 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f026c02d020 con 0x5579ae880e80 2024-04-21T08:46:59.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.218+0000 7f025affd640 1 -- 172.21.15.86:0/3781125291 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f026c005040 con 0x5579ae880e80 2024-04-21T08:46:59.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.218+0000 7f0284472740 1 -- 172.21.15.86:0/3781125291 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f026402c450 con 0x5579ae880e80 2024-04-21T08:46:59.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.218+0000 7f025affd640 1 -- 172.21.15.86:0/3781125291 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f026c038050 con 0x5579ae880e80 2024-04-21T08:46:59.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.218+0000 7f0284472740 10 client.18996 fetch_fsmap learned FSMap version 29 2024-04-21T08:46:59.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.218+0000 7f0284472740 1 -- 172.21.15.86:0/3781125291 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5579ae9047e0 con 0x5579ae880e80 2024-04-21T08:46:59.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.219+0000 7f025affd640 1 -- 172.21.15.86:0/3781125291 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f026402c450 con 0x5579ae880e80 2024-04-21T08:46:59.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.219+0000 7f025affd640 1 -- 172.21.15.86:0/3781125291 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f026c0038e0 con 0x5579ae880e80 2024-04-21T08:46:59.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.219+0000 7f0284472740 10 client.18996 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:46:59.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.219+0000 7f0284472740 10 client.18996 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:46:59.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.219+0000 7f0284472740 20 client.18996 populate_metadata read hostname 'smithi086' 2024-04-21T08:46:59.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.219+0000 7f0284472740 1 --2- 172.21.15.86:0/3781125291 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5579ae904fb0 0x5579ae9073d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:46:59.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.219+0000 7f0284472740 4 client.18996 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:46:59.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.219+0000 7f0284472740 1 -- 172.21.15.86:0/3781125291 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5579ae7f2c10 con 0x5579ae904fb0 2024-04-21T08:46:59.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.219+0000 7f0273fff640 1 --2- 172.21.15.86:0/3781125291 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5579ae904fb0 0x5579ae9073d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:46:59.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.219+0000 7f025affd640 10 client.18996 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:59.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.219+0000 7f0273fff640 1 --2- 172.21.15.86:0/3781125291 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5579ae904fb0 0x5579ae9073d0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f3d4e47a6c788728 server_cookie=15262f16272235b4 in_seq=0 out_seq=0 2024-04-21T08:46:59.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.219+0000 7f025affd640 1 -- 172.21.15.86:0/3781125291 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5579ae7f2c10 con 0x5579ae904fb0 2024-04-21T08:46:59.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.219+0000 7f025affd640 10 client.18996 handle_command_reply: tid=0 2024-04-21T08:46:59.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.219+0000 7f0284472740 1 client.18996 shutdown 2024-04-21T08:46:59.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.220+0000 7f0284472740 1 -- 172.21.15.86:0/3781125291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579ae880e80 msgr2=0x5579ae8f86d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:59.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.220+0000 7f0284472740 1 --2- 172.21.15.86:0/3781125291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579ae880e80 0x5579ae8f86d0 secure :-1 s=READY pgs=1597 cs=0 l=1 rev1=1 crypto rx=0x5579ae887c90 tx=0x7f026c01b640 comp rx=0 tx=0).stop 2024-04-21T08:46:59.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.220+0000 7f0284472740 1 -- 172.21.15.86:0/3781125291 shutdown_connections 2024-04-21T08:46:59.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.220+0000 7f0284472740 1 --2- 172.21.15.86:0/3781125291 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5579ae904fb0 0x5579ae9073d0 crc :-1 s=READY pgs=246 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.220+0000 7f0284472740 1 --2- 172.21.15.86:0/3781125291 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5579ae889a00 0x5579ae8f8c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.220+0000 7f0284472740 1 --2- 172.21.15.86:0/3781125291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5579ae880e80 0x5579ae8f86d0 unknown :-1 s=CLOSED pgs=1597 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.220+0000 7f025affd640 0 client.18996 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:46:59.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.220+0000 7f0284472740 1 --2- 172.21.15.86:0/3781125291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5579ae8804d0 0x5579ae8f8190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.220+0000 7f0284472740 1 -- 172.21.15.86:0/3781125291 >> 172.21.15.86:0/3781125291 conn(0x5579ae87aba0 msgr2=0x5579ae879c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:59.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.220+0000 7f0284472740 1 -- 172.21.15.86:0/3781125291 shutdown_connections 2024-04-21T08:46:59.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.220+0000 7f0284472740 1 -- 172.21.15.86:0/3781125291 wait complete. 2024-04-21T08:46:59.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.220+0000 7f0284472740 20 client.18996 trim_cache size 0 max 16384 2024-04-21T08:46:59.227 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:46:59.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.220+0000 7f028187b640 1 -- 172.21.15.86:0/2025183124 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f02680a3d20 msgr2=0x7f02680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:59.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.220+0000 7f028187b640 1 --2- 172.21.15.86:0/2025183124 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f02680a3d20 0x7f02680a61e0 secure :-1 s=READY pgs=259 cs=0 l=1 rev1=1 crypto rx=0x7f02740910c0 tx=0x7f02740a9040 comp rx=0 tx=0).stop 2024-04-21T08:46:59.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.220+0000 7f028187b640 1 -- 172.21.15.86:0/2025183124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f027c19cf90 msgr2=0x7f027c19d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:46:59.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.220+0000 7f028187b640 1 --2- 172.21.15.86:0/2025183124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f027c19cf90 0x7f027c19d440 secure :-1 s=READY pgs=1593 cs=0 l=1 rev1=1 crypto rx=0x7f0274005e30 tx=0x7f0274032420 comp rx=0 tx=0).stop 2024-04-21T08:46:59.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.220+0000 7f028187b640 1 -- 172.21.15.86:0/2025183124 shutdown_connections 2024-04-21T08:46:59.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.220+0000 7f028187b640 1 --2- 172.21.15.86:0/2025183124 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f02680a3d20 0x7f02680a61e0 unknown :-1 s=CLOSED pgs=259 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.220+0000 7f028187b640 1 --2- 172.21.15.86:0/2025183124 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f027c19cf90 0x7f027c19d440 unknown :-1 s=CLOSED pgs=1593 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.220+0000 7f028187b640 1 --2- 172.21.15.86:0/2025183124 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f027c106160 0x7f027c198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.220+0000 7f028187b640 1 --2- 172.21.15.86:0/2025183124 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f027c101090 0x7f027c198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:46:59.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.220+0000 7f028187b640 1 -- 172.21.15.86:0/2025183124 >> 172.21.15.86:0/2025183124 conn(0x7f027c077f50 msgr2=0x7f027c0694d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:46:59.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.220+0000 7f028187b640 1 -- 172.21.15.86:0/2025183124 shutdown_connections 2024-04-21T08:46:59.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:46:59.220+0000 7f028187b640 1 -- 172.21.15.86:0/2025183124 wait complete. 2024-04-21T08:46:59.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:58 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:46:59.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:58 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/4037676148' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:46:59.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:58 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:46:59.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:58 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:46:59.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:58 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:47:00.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.101+0000 7ff690e7a640 1 -- 172.21.15.86:0/3487632200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff68c102b80 msgr2=0x7ff68c106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.101+0000 7ff690e7a640 1 --2- 172.21.15.86:0/3487632200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff68c102b80 0x7ff68c106bc0 secure :-1 s=READY pgs=1598 cs=0 l=1 rev1=1 crypto rx=0x7ff674009900 tx=0x7ff67402bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.101+0000 7ff690e7a640 1 -- 172.21.15.86:0/3487632200 shutdown_connections 2024-04-21T08:47:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.101+0000 7ff690e7a640 1 --2- 172.21.15.86:0/3487632200 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff68c1072a0 0x7ff68c10ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.101+0000 7ff690e7a640 1 --2- 172.21.15.86:0/3487632200 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff68c102b80 0x7ff68c106bc0 unknown :-1 s=CLOSED pgs=1598 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.101+0000 7ff690e7a640 1 --2- 172.21.15.86:0/3487632200 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff68c1021d0 0x7ff68c1025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.101+0000 7ff690e7a640 1 -- 172.21.15.86:0/3487632200 >> 172.21.15.86:0/3487632200 conn(0x7ff68c0793e0 msgr2=0x7ff68c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.101+0000 7ff690e7a640 1 -- 172.21.15.86:0/3487632200 shutdown_connections 2024-04-21T08:47:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.101+0000 7ff690e7a640 1 -- 172.21.15.86:0/3487632200 wait complete. 2024-04-21T08:47:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.102+0000 7ff690e7a640 1 Processor -- start 2024-04-21T08:47:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.102+0000 7ff690e7a640 1 -- start start 2024-04-21T08:47:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.102+0000 7ff690e7a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff68c1021d0 0x7ff68c1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.102+0000 7ff690e7a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff68c1072a0 0x7ff68c198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.102+0000 7ff690e7a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff68c19cf80 0x7ff68c19d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.102+0000 7ff690e7a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c1121c0 con 0x7ff68c1072a0 2024-04-21T08:47:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.102+0000 7ff690e7a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c112040 con 0x7ff68c1021d0 2024-04-21T08:47:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.102+0000 7ff690e7a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c112340 con 0x7ff68c19cf80 2024-04-21T08:47:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.102+0000 7ff683fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff68c1072a0 0x7ff68c198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.102+0000 7ff683fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff68c1072a0 0x7ff68c198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37012/0 (socket says 172.21.15.86:37012) 2024-04-21T08:47:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.102+0000 7ff683fff640 1 -- 172.21.15.86:0/1958411880 learned_addr learned my addr 172.21.15.86:0/1958411880 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.102+0000 7ff68a59c640 1 --2- 172.21.15.86:0/1958411880 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff68c1021d0 0x7ff68c1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.102+0000 7ff683fff640 1 -- 172.21.15.86:0/1958411880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff68c19cf80 msgr2=0x7ff68c19d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.102+0000 7ff683fff640 1 --2- 172.21.15.86:0/1958411880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff68c19cf80 0x7ff68c19d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.102+0000 7ff683fff640 1 -- 172.21.15.86:0/1958411880 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff68c1021d0 msgr2=0x7ff68c1986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.102+0000 7ff683fff640 1 --2- 172.21.15.86:0/1958411880 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff68c1021d0 0x7ff68c1986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.102+0000 7ff683fff640 1 -- 172.21.15.86:0/1958411880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff68c19da70 con 0x7ff68c1072a0 2024-04-21T08:47:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.103+0000 7ff683fff640 1 --2- 172.21.15.86:0/1958411880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff68c1072a0 0x7ff68c198c00 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7ff674036040 tx=0x7ff6740045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.103+0000 7ff6837fe640 1 -- 172.21.15.86:0/1958411880 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff674043020 con 0x7ff68c1072a0 2024-04-21T08:47:00.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.103+0000 7ff6837fe640 1 -- 172.21.15.86:0/1958411880 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff67402cb70 con 0x7ff68c1072a0 2024-04-21T08:47:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.103+0000 7ff6837fe640 1 -- 172.21.15.86:0/1958411880 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff674003bf0 con 0x7ff68c1072a0 2024-04-21T08:47:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.103+0000 7ff690e7a640 1 -- 172.21.15.86:0/1958411880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff68c19dd00 con 0x7ff68c1072a0 2024-04-21T08:47:00.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.103+0000 7ff690e7a640 1 -- 172.21.15.86:0/1958411880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff68c1a5520 con 0x7ff68c1072a0 2024-04-21T08:47:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.105+0000 7ff6837fe640 1 -- 172.21.15.86:0/1958411880 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff674046020 con 0x7ff68c1072a0 2024-04-21T08:47:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.106+0000 7ff693a6a740 1 Processor -- start 2024-04-21T08:47:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.106+0000 7ff693a6a740 1 -- start start 2024-04-21T08:47:00.111 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.106+0000 7ff693a6a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc7c9388d0 0x55cc7c937620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.106+0000 7ff693a6a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc7c937b60 0x55cc7c937fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.106+0000 7ff693a6a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc7c9410e0 0x55cc7c9434d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.106+0000 7ff693a6a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cc7c89c8d0 con 0x55cc7c9410e0 2024-04-21T08:47:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.106+0000 7ff693a6a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55cc7c839e80 con 0x55cc7c9388d0 2024-04-21T08:47:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.106+0000 7ff693a6a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55cc7c898890 con 0x55cc7c937b60 2024-04-21T08:47:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.106+0000 7ff683fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc7c9410e0 0x55cc7c9434d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.106+0000 7ff68ad9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc7c9388d0 0x55cc7c937620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.106+0000 7ff683fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc7c9410e0 0x55cc7c9434d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37016/0 (socket says 172.21.15.86:37016) 2024-04-21T08:47:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.106+0000 7ff683fff640 1 -- 172.21.15.86:0/2715677399 learned_addr learned my addr 172.21.15.86:0/2715677399 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.106+0000 7ff6837fe640 1 --2- 172.21.15.86:0/1958411880 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff6780a3dc0 0x7ff6780a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.106+0000 7ff68a59c640 1 --2- 172.21.15.86:0/2715677399 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc7c937b60 0x55cc7c937fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.107+0000 7ff68ad9d640 1 --2- 172.21.15.86:0/1958411880 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff6780a3dc0 0x7ff6780a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.112 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.107+0000 7ff68ad9d640 1 -- 172.21.15.86:0/2715677399 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc7c937b60 msgr2=0x55cc7c937fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.107+0000 7ff68ad9d640 1 --2- 172.21.15.86:0/2715677399 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc7c937b60 0x55cc7c937fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.107+0000 7ff68ad9d640 1 -- 172.21.15.86:0/2715677399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc7c9410e0 msgr2=0x55cc7c9434d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.107+0000 7ff68ad9d640 1 --2- 172.21.15.86:0/2715677399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc7c9410e0 0x55cc7c9434d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.107+0000 7ff68ad9d640 1 -- 172.21.15.86:0/2715677399 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc7c85b320 con 0x55cc7c9388d0 2024-04-21T08:47:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.107+0000 7ff6837fe640 1 -- 172.21.15.86:0/1958411880 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7ff6740f2c80 con 0x7ff68c1072a0 2024-04-21T08:47:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.107+0000 7ff68ad9d640 1 --2- 172.21.15.86:0/2715677399 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc7c9388d0 0x55cc7c937620 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff684015860 tx=0x7ff684015d20 comp rx=0 tx=0).ready entity=mon.2 client_cookie=4b12bc0849a3c5ea server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.107+0000 7ff68ad9d640 1 --2- 172.21.15.86:0/1958411880 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff6780a3dc0 0x7ff6780a6280 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff68400c000 tx=0x7ff68401b000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.107+0000 7ff680ff9640 1 -- 172.21.15.86:0/2715677399 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff684004d20 con 0x55cc7c9388d0 2024-04-21T08:47:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.107+0000 7ff680ff9640 1 -- 172.21.15.86:0/2715677399 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff684010c00 con 0x55cc7c9388d0 2024-04-21T08:47:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.107+0000 7ff680ff9640 1 -- 172.21.15.86:0/2715677399 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff684005690 con 0x55cc7c9388d0 2024-04-21T08:47:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.108+0000 7ff693a6a740 1 -- 172.21.15.86:0/2715677399 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc7c9388d0 msgr2=0x55cc7c937620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.108+0000 7ff693a6a740 1 --2- 172.21.15.86:0/2715677399 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc7c9388d0 0x55cc7c937620 secure :-1 s=READY pgs=79 cs=0 l=1 rev1=1 crypto rx=0x7ff684015860 tx=0x7ff684015d20 comp rx=0 tx=0).stop 2024-04-21T08:47:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.108+0000 7ff693a6a740 1 -- 172.21.15.86:0/2715677399 shutdown_connections 2024-04-21T08:47:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.108+0000 7ff693a6a740 1 --2- 172.21.15.86:0/2715677399 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc7c9410e0 0x55cc7c9434d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.113 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.108+0000 7ff693a6a740 1 --2- 172.21.15.86:0/2715677399 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc7c937b60 0x55cc7c937fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.108+0000 7ff693a6a740 1 --2- 172.21.15.86:0/2715677399 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc7c9388d0 0x55cc7c937620 unknown :-1 s=CLOSED pgs=79 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.108+0000 7ff693a6a740 1 -- 172.21.15.86:0/2715677399 >> 172.21.15.86:0/2715677399 conn(0x55cc7c8fec90 msgr2=0x55cc7c932850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.108+0000 7ff693a6a740 1 -- 172.21.15.86:0/2715677399 shutdown_connections 2024-04-21T08:47:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.108+0000 7ff693a6a740 1 -- 172.21.15.86:0/2715677399 wait complete. 2024-04-21T08:47:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.108+0000 7ff693a6a740 1 -- start start 2024-04-21T08:47:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.109+0000 7ff693a6a740 1 Processor -- start 2024-04-21T08:47:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.109+0000 7ff693a6a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc7c937b60 0x55cc7c940ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.109+0000 7ff693a6a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc7c9388d0 0x55cc7c93de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.109+0000 7ff693a6a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc7c9410e0 0x55cc7c93e460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.109+0000 7ff693a6a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cc7c943db0 con 0x55cc7c9410e0 2024-04-21T08:47:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.109+0000 7ff693a6a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55cc7c824eb0 con 0x55cc7c937b60 2024-04-21T08:47:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.109+0000 7ff693a6a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55cc7c943f30 con 0x55cc7c9388d0 2024-04-21T08:47:00.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.109+0000 7ff693a6a740 10 client.0 _ll_register_callbacks cb 0x55cc7c851b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.109+0000 7ff68a59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc7c937b60 0x55cc7c940ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.109+0000 7ff68ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc7c9410e0 0x55cc7c93e460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.109+0000 7ff68a59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc7c937b60 0x55cc7c940ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.86:38198/0 (socket says 172.21.15.86:38198) 2024-04-21T08:47:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.109+0000 7ff68a59c640 1 -- 172.21.15.86:0/3827564995 learned_addr learned my addr 172.21.15.86:0/3827564995 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.109+0000 7ff683fff640 1 --2- 172.21.15.86:0/3827564995 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc7c9388d0 0x55cc7c93de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.109+0000 7ff68ad9d640 1 -- 172.21.15.86:0/3827564995 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc7c9388d0 msgr2=0x55cc7c93de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.109+0000 7ff68ad9d640 1 --2- 172.21.15.86:0/3827564995 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc7c9388d0 0x55cc7c93de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.109+0000 7ff68ad9d640 1 -- 172.21.15.86:0/3827564995 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc7c937b60 msgr2=0x55cc7c940ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.109+0000 7ff68ad9d640 1 --2- 172.21.15.86:0/3827564995 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc7c937b60 0x55cc7c940ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.109+0000 7ff68ad9d640 1 -- 172.21.15.86:0/3827564995 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc7c5af210 con 0x55cc7c9410e0 2024-04-21T08:47:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.109+0000 7ff68a59c640 1 --2- 172.21.15.86:0/3827564995 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc7c937b60 0x55cc7c940ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:47:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.109+0000 7ff68ad9d640 1 --2- 172.21.15.86:0/3827564995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc7c9410e0 0x55cc7c93e460 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7ff684005fd0 tx=0x7ff684004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6b6f8a2121ee86fc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.109+0000 7ff671ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:47:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.110+0000 7ff671ffb640 1 -- 172.21.15.86:0/3827564995 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68402d070 con 0x55cc7c9410e0 2024-04-21T08:47:00.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.110+0000 7ff671ffb640 1 -- 172.21.15.86:0/3827564995 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7ff684010950 con 0x55cc7c9410e0 2024-04-21T08:47:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.110+0000 7ff693a6a740 1 -- 172.21.15.86:0/3827564995 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff684010c00 con 0x55cc7c9410e0 2024-04-21T08:47:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.110+0000 7ff671ffb640 1 -- 172.21.15.86:0/3827564995 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff684004b20 con 0x55cc7c9410e0 2024-04-21T08:47:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.110+0000 7ff693a6a740 10 client.19008 fetch_fsmap learned FSMap version 29 2024-04-21T08:47:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.110+0000 7ff693a6a740 1 -- 172.21.15.86:0/3827564995 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cc7c67b620 con 0x55cc7c9410e0 2024-04-21T08:47:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.110+0000 7ff671ffb640 1 -- 172.21.15.86:0/3827564995 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff684010c00 con 0x55cc7c9410e0 2024-04-21T08:47:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.110+0000 7ff671ffb640 1 -- 172.21.15.86:0/3827564995 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff68401d440 con 0x55cc7c9410e0 2024-04-21T08:47:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.110+0000 7ff693a6a740 10 client.19008 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:47:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.110+0000 7ff693a6a740 10 client.19008 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.110+0000 7ff693a6a740 20 client.19008 populate_metadata read hostname 'smithi086' 2024-04-21T08:47:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.110+0000 7ff693a6a740 1 --2- 172.21.15.86:0/3827564995 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55cc7c950f60 0x55cc7c953350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.110+0000 7ff693a6a740 4 client.19008 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.110+0000 7ff693a6a740 1 -- 172.21.15.86:0/3827564995 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cc7c8b0880 con 0x55cc7c950f60 2024-04-21T08:47:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.111+0000 7ff68a59c640 1 --2- 172.21.15.86:0/3827564995 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55cc7c950f60 0x55cc7c953350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.111+0000 7ff671ffb640 10 client.19008 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.111+0000 7ff68a59c640 1 --2- 172.21.15.86:0/3827564995 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55cc7c950f60 0x55cc7c953350 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=50f002eb36e0728c server_cookie=7e11848667e2ebf0 in_seq=0 out_seq=0 2024-04-21T08:47:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.112+0000 7ff671ffb640 1 -- 172.21.15.86:0/3827564995 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cc7c8b0880 con 0x55cc7c950f60 2024-04-21T08:47:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.112+0000 7ff671ffb640 10 client.19008 handle_command_reply: tid=0 2024-04-21T08:47:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.112+0000 7ff693a6a740 1 client.19008 shutdown 2024-04-21T08:47:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.112+0000 7ff693a6a740 1 -- 172.21.15.86:0/3827564995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc7c9410e0 msgr2=0x55cc7c93e460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.112+0000 7ff693a6a740 1 --2- 172.21.15.86:0/3827564995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc7c9410e0 0x55cc7c93e460 secure :-1 s=READY pgs=1600 cs=0 l=1 rev1=1 crypto rx=0x7ff684005fd0 tx=0x7ff684004a40 comp rx=0 tx=0).stop 2024-04-21T08:47:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.112+0000 7ff693a6a740 1 -- 172.21.15.86:0/3827564995 shutdown_connections 2024-04-21T08:47:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.112+0000 7ff693a6a740 1 --2- 172.21.15.86:0/3827564995 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55cc7c950f60 0x55cc7c953350 crc :-1 s=READY pgs=247 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.112+0000 7ff693a6a740 1 --2- 172.21.15.86:0/3827564995 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc7c9410e0 0x55cc7c93e460 unknown :-1 s=CLOSED pgs=1600 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.112+0000 7ff693a6a740 1 --2- 172.21.15.86:0/3827564995 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc7c9388d0 0x55cc7c93de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.112+0000 7ff671ffb640 0 client.19008 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.112+0000 7ff693a6a740 1 --2- 172.21.15.86:0/3827564995 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc7c937b60 0x55cc7c940ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.112+0000 7ff693a6a740 1 -- 172.21.15.86:0/3827564995 >> 172.21.15.86:0/3827564995 conn(0x55cc7c8fec90 msgr2=0x55cc7c9345a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:00.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.112+0000 7ff693a6a740 1 -- 172.21.15.86:0/3827564995 shutdown_connections 2024-04-21T08:47:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.112+0000 7ff693a6a740 1 -- 172.21.15.86:0/3827564995 wait complete. 2024-04-21T08:47:00.118 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.112+0000 7ff693a6a740 20 client.19008 trim_cache size 0 max 16384 2024-04-21T08:47:00.118 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:46:59 smithi086 ceph-mon[27939]: pgmap v253: 129 pgs: 129 active+clean; 2.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-21T08:47:00.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.127+0000 7ff693a6a740 1 Processor -- start 2024-04-21T08:47:00.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.127+0000 7ff693a6a740 1 -- start start 2024-04-21T08:47:00.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.127+0000 7ff693a6a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc7c937b60 0x55cc7c94ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.127+0000 7ff693a6a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc7c9388d0 0x55cc7c93ce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.127+0000 7ff693a6a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc7c9410e0 0x55cc7c958ac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.127+0000 7ff693a6a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c1af370 con 0x55cc7c937b60 2024-04-21T08:47:00.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.127+0000 7ff693a6a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c1b0eb0 con 0x55cc7c9410e0 2024-04-21T08:47:00.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.127+0000 7ff693a6a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c1afa40 con 0x55cc7c9388d0 2024-04-21T08:47:00.131 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.127+0000 7ff68a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc7c937b60 0x55cc7c94ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.127+0000 7ff68a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc7c937b60 0x55cc7c94ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37044/0 (socket says 172.21.15.86:37044) 2024-04-21T08:47:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.127+0000 7ff68a59c640 1 -- 172.21.15.86:0/4204230097 learned_addr learned my addr 172.21.15.86:0/4204230097 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.127+0000 7ff683fff640 1 --2- 172.21.15.86:0/4204230097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc7c9388d0 0x55cc7c93ce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.127+0000 7ff68a59c640 1 -- 172.21.15.86:0/4204230097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc7c9388d0 msgr2=0x55cc7c93ce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.127+0000 7ff68a59c640 1 --2- 172.21.15.86:0/4204230097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc7c9388d0 0x55cc7c93ce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.127+0000 7ff68a59c640 1 -- 172.21.15.86:0/4204230097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc7c9410e0 msgr2=0x55cc7c958ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.127+0000 7ff68a59c640 1 --2- 172.21.15.86:0/4204230097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc7c9410e0 0x55cc7c958ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.127+0000 7ff68a59c640 1 -- 172.21.15.86:0/4204230097 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cc7c936e30 con 0x55cc7c937b60 2024-04-21T08:47:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.127+0000 7ff68a59c640 1 --2- 172.21.15.86:0/4204230097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc7c937b60 0x55cc7c94ccb0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x55cc7c953a60 tx=0x7ff67c012ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ad46f15de4079e1a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.127+0000 7ff673fff640 1 -- 172.21.15.86:0/4204230097 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff67c012cd0 con 0x55cc7c937b60 2024-04-21T08:47:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.127+0000 7ff673fff640 1 -- 172.21.15.86:0/4204230097 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff67c024070 con 0x55cc7c937b60 2024-04-21T08:47:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.128+0000 7ff673fff640 1 -- 172.21.15.86:0/4204230097 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff67c01f580 con 0x55cc7c937b60 2024-04-21T08:47:00.132 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.128+0000 7ff693a6a740 1 -- 172.21.15.86:0/4204230097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc7c937b60 msgr2=0x55cc7c94ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.128+0000 7ff693a6a740 1 --2- 172.21.15.86:0/4204230097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc7c937b60 0x55cc7c94ccb0 secure :-1 s=READY pgs=1601 cs=0 l=1 rev1=1 crypto rx=0x55cc7c953a60 tx=0x7ff67c012ba0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.128+0000 7ff693a6a740 1 -- 172.21.15.86:0/4204230097 shutdown_connections 2024-04-21T08:47:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.128+0000 7ff693a6a740 1 --2- 172.21.15.86:0/4204230097 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc7c9410e0 0x55cc7c958ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.128+0000 7ff693a6a740 1 --2- 172.21.15.86:0/4204230097 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc7c9388d0 0x55cc7c93ce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.128+0000 7ff693a6a740 1 --2- 172.21.15.86:0/4204230097 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc7c937b60 0x55cc7c94ccb0 unknown :-1 s=CLOSED pgs=1601 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.128+0000 7ff693a6a740 1 -- 172.21.15.86:0/4204230097 >> 172.21.15.86:0/4204230097 conn(0x55cc7c8fec90 msgr2=0x55cc7c93c740 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.128+0000 7ff693a6a740 1 -- 172.21.15.86:0/4204230097 shutdown_connections 2024-04-21T08:47:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.128+0000 7ff693a6a740 1 -- 172.21.15.86:0/4204230097 wait complete. 2024-04-21T08:47:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.128+0000 7ff693a6a740 1 -- start start 2024-04-21T08:47:00.133 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.128+0000 7ff693a6a740 1 Processor -- start 2024-04-21T08:47:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.129+0000 7ff693a6a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc7c937b60 0x55cc7c93aa20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.129+0000 7ff693a6a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc7c9388d0 0x55cc7c93af60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.129+0000 7ff693a6a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc7c9410e0 0x55cc7c93b4a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.129+0000 7ff693a6a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c1b07e0 con 0x55cc7c9410e0 2024-04-21T08:47:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.129+0000 7ff693a6a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c1b0110 con 0x55cc7c937b60 2024-04-21T08:47:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.129+0000 7ff693a6a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff68c1ad370 con 0x55cc7c9388d0 2024-04-21T08:47:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.129+0000 7ff693a6a740 10 client.0 _ll_register_callbacks cb 0x7ff6740baab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.129+0000 7ff68ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc7c9410e0 0x55cc7c93b4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.129+0000 7ff68ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc7c9410e0 0x55cc7c93b4a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37052/0 (socket says 172.21.15.86:37052) 2024-04-21T08:47:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.129+0000 7ff68ad9d640 1 -- 172.21.15.86:0/3959140711 learned_addr learned my addr 172.21.15.86:0/3959140711 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.129+0000 7ff683fff640 1 --2- 172.21.15.86:0/3959140711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc7c9388d0 0x55cc7c93af60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.134 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.129+0000 7ff68a59c640 1 --2- 172.21.15.86:0/3959140711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc7c937b60 0x55cc7c93aa20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.129+0000 7ff68ad9d640 1 -- 172.21.15.86:0/3959140711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc7c9388d0 msgr2=0x55cc7c93af60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.129+0000 7ff68ad9d640 1 --2- 172.21.15.86:0/3959140711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc7c9388d0 0x55cc7c93af60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.129+0000 7ff68ad9d640 1 -- 172.21.15.86:0/3959140711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc7c937b60 msgr2=0x55cc7c93aa20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.129+0000 7ff68ad9d640 1 --2- 172.21.15.86:0/3959140711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc7c937b60 0x55cc7c93aa20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.129+0000 7ff68ad9d640 1 -- 172.21.15.86:0/3959140711 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cc7c9b6330 con 0x55cc7c9410e0 2024-04-21T08:47:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.129+0000 7ff68a59c640 1 --2- 172.21.15.86:0/3959140711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc7c937b60 0x55cc7c93aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.129+0000 7ff683fff640 1 --2- 172.21.15.86:0/3959140711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc7c9388d0 0x55cc7c93af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.129+0000 7ff68ad9d640 1 --2- 172.21.15.86:0/3959140711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc7c9410e0 0x55cc7c93b4a0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7ff684015860 tx=0x7ff684009f80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a1214dcb1e721026 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.129+0000 7ff671ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:47:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.129+0000 7ff671ffb640 1 -- 172.21.15.86:0/3959140711 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff68402d070 con 0x55cc7c9410e0 2024-04-21T08:47:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.129+0000 7ff671ffb640 1 -- 172.21.15.86:0/3959140711 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7ff6840096d0 con 0x55cc7c9410e0 2024-04-21T08:47:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.129+0000 7ff693a6a740 1 -- 172.21.15.86:0/3959140711 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff67c024070 con 0x55cc7c9410e0 2024-04-21T08:47:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.130+0000 7ff671ffb640 1 -- 172.21.15.86:0/3959140711 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff684043050 con 0x55cc7c9410e0 2024-04-21T08:47:00.135 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.130+0000 7ff693a6a740 10 client.19014 fetch_fsmap learned FSMap version 29 2024-04-21T08:47:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.130+0000 7ff693a6a740 1 -- 172.21.15.86:0/3959140711 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cc7c9b7520 con 0x55cc7c9410e0 2024-04-21T08:47:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.130+0000 7ff671ffb640 1 -- 172.21.15.86:0/3959140711 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff67c024070 con 0x55cc7c9410e0 2024-04-21T08:47:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.130+0000 7ff671ffb640 1 -- 172.21.15.86:0/3959140711 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff68403c060 con 0x55cc7c9410e0 2024-04-21T08:47:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.130+0000 7ff693a6a740 10 client.19014 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:47:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.130+0000 7ff693a6a740 10 client.19014 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.130+0000 7ff693a6a740 20 client.19014 populate_metadata read hostname 'smithi086' 2024-04-21T08:47:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.130+0000 7ff693a6a740 1 --2- 172.21.15.86:0/3959140711 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55cc7c9b7cd0 0x55cc7c9ba0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.130+0000 7ff693a6a740 4 client.19014 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:47:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.130+0000 7ff693a6a740 1 -- 172.21.15.86:0/3959140711 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55cc7c83c3d0 con 0x55cc7c9b7cd0 2024-04-21T08:47:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.130+0000 7ff68a59c640 1 --2- 172.21.15.86:0/3959140711 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55cc7c9b7cd0 0x55cc7c9ba0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.131+0000 7ff671ffb640 10 client.19014 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:00.136 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.131+0000 7ff68a59c640 1 --2- 172.21.15.86:0/3959140711 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55cc7c9b7cd0 0x55cc7c9ba0f0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=db4c4e7ff03da26f server_cookie=5f166ece77311b53 in_seq=0 out_seq=0 2024-04-21T08:47:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.145+0000 7ff671ffb640 1 -- 172.21.15.86:0/3959140711 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55cc7c83c3d0 con 0x55cc7c9b7cd0 2024-04-21T08:47:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.145+0000 7ff671ffb640 10 client.19014 handle_command_reply: tid=0 2024-04-21T08:47:00.145 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.145+0000 7ff693a6a740 1 client.19014 shutdown 2024-04-21T08:47:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.145+0000 7ff693a6a740 1 -- 172.21.15.86:0/3959140711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc7c9410e0 msgr2=0x55cc7c93b4a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.145+0000 7ff693a6a740 1 --2- 172.21.15.86:0/3959140711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc7c9410e0 0x55cc7c93b4a0 secure :-1 s=READY pgs=1602 cs=0 l=1 rev1=1 crypto rx=0x7ff684015860 tx=0x7ff684009f80 comp rx=0 tx=0).stop 2024-04-21T08:47:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.145+0000 7ff693a6a740 1 -- 172.21.15.86:0/3959140711 shutdown_connections 2024-04-21T08:47:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.145+0000 7ff693a6a740 1 --2- 172.21.15.86:0/3959140711 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55cc7c9b7cd0 0x55cc7c9ba0f0 crc :-1 s=READY pgs=248 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.146 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.145+0000 7ff693a6a740 1 --2- 172.21.15.86:0/3959140711 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cc7c9410e0 0x55cc7c93b4a0 unknown :-1 s=CLOSED pgs=1602 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.145+0000 7ff693a6a740 1 --2- 172.21.15.86:0/3959140711 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cc7c9388d0 0x55cc7c93af60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.145+0000 7ff693a6a740 1 --2- 172.21.15.86:0/3959140711 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cc7c937b60 0x55cc7c93aa20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.145+0000 7ff671ffb640 0 client.19014 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.145+0000 7ff693a6a740 1 -- 172.21.15.86:0/3959140711 >> 172.21.15.86:0/3959140711 conn(0x55cc7c8fec90 msgr2=0x55cc7c93cfd0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.145+0000 7ff693a6a740 1 -- 172.21.15.86:0/3959140711 shutdown_connections 2024-04-21T08:47:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.145+0000 7ff693a6a740 1 -- 172.21.15.86:0/3959140711 wait complete. 2024-04-21T08:47:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.145+0000 7ff693a6a740 20 client.19014 trim_cache size 0 max 16384 2024-04-21T08:47:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:47:00.147 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:47:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:47:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:47:00.148 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.146+0000 7ff690e7a640 1 -- 172.21.15.86:0/1958411880 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff6780a3dc0 msgr2=0x7ff6780a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.146+0000 7ff690e7a640 1 --2- 172.21.15.86:0/1958411880 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff6780a3dc0 0x7ff6780a6280 secure :-1 s=READY pgs=260 cs=0 l=1 rev1=1 crypto rx=0x7ff68400c000 tx=0x7ff68401b000 comp rx=0 tx=0).stop 2024-04-21T08:47:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.146+0000 7ff690e7a640 1 -- 172.21.15.86:0/1958411880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff68c1072a0 msgr2=0x7ff68c198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.146+0000 7ff690e7a640 1 --2- 172.21.15.86:0/1958411880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff68c1072a0 0x7ff68c198c00 secure :-1 s=READY pgs=1599 cs=0 l=1 rev1=1 crypto rx=0x7ff674036040 tx=0x7ff6740045e0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.146+0000 7ff690e7a640 1 -- 172.21.15.86:0/1958411880 shutdown_connections 2024-04-21T08:47:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.146+0000 7ff690e7a640 1 --2- 172.21.15.86:0/1958411880 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff6780a3dc0 0x7ff6780a6280 unknown :-1 s=CLOSED pgs=260 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.146+0000 7ff690e7a640 1 --2- 172.21.15.86:0/1958411880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff68c19cf80 0x7ff68c19d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.146+0000 7ff690e7a640 1 --2- 172.21.15.86:0/1958411880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff68c1072a0 0x7ff68c198c00 unknown :-1 s=CLOSED pgs=1599 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.146+0000 7ff690e7a640 1 --2- 172.21.15.86:0/1958411880 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff68c1021d0 0x7ff68c1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.146+0000 7ff690e7a640 1 -- 172.21.15.86:0/1958411880 >> 172.21.15.86:0/1958411880 conn(0x7ff68c0793e0 msgr2=0x7ff68c0ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.146+0000 7ff690e7a640 1 -- 172.21.15.86:0/1958411880 shutdown_connections 2024-04-21T08:47:00.149 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:00.146+0000 7ff690e7a640 1 -- 172.21.15.86:0/1958411880 wait complete. 2024-04-21T08:47:00.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:46:59 smithi088 ceph-mon[31259]: pgmap v253: 129 pgs: 129 active+clean; 2.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-21T08:47:00.232 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:47:00.232 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:47:00.233 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:47:00.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:46:59 smithi179 ceph-mon[31829]: pgmap v253: 129 pgs: 129 active+clean; 2.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 22 op/s 2024-04-21T08:47:00.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.338+0000 7fba0cd52640 1 -- 172.21.15.86:0/4272193425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba08101a40 msgr2=0x7fba08105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:00.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.338+0000 7fba0cd52640 1 --2- 172.21.15.86:0/4272193425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba08101a40 0x7fba08105a80 secure :-1 s=READY pgs=1603 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0009900 tx=0x7fb9f002bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:00.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.338+0000 7fba0cd52640 1 -- 172.21.15.86:0/4272193425 shutdown_connections 2024-04-21T08:47:00.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.338+0000 7fba0cd52640 1 --2- 172.21.15.86:0/4272193425 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba08106160 0x7fba0810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.338+0000 7fba0cd52640 1 --2- 172.21.15.86:0/4272193425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba08101a40 0x7fba08105a80 unknown :-1 s=CLOSED pgs=1603 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.338+0000 7fba0cd52640 1 --2- 172.21.15.86:0/4272193425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fba08101090 0x7fba08101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.338+0000 7fba0cd52640 1 -- 172.21.15.86:0/4272193425 >> 172.21.15.86:0/4272193425 conn(0x7fba080771d0 msgr2=0x7fba080775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:00.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.338+0000 7fba0cd52640 1 -- 172.21.15.86:0/4272193425 shutdown_connections 2024-04-21T08:47:00.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.338+0000 7fba0cd52640 1 -- 172.21.15.86:0/4272193425 wait complete. 2024-04-21T08:47:00.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.338+0000 7fba0cd52640 1 Processor -- start 2024-04-21T08:47:00.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.339+0000 7fba0cd52640 1 -- start start 2024-04-21T08:47:00.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.339+0000 7fba0cd52640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fba08101090 0x7fba08199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.339+0000 7fba0cd52640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba08106160 0x7fba08199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.339+0000 7fba0cd52640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba0819a0d0 0x7fba081a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.339+0000 7fba0cd52640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba08111100 con 0x7fba08106160 2024-04-21T08:47:00.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.339+0000 7fba0cd52640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fba08110f80 con 0x7fba0819a0d0 2024-04-21T08:47:00.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.339+0000 7fba0cd52640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fba08111280 con 0x7fba08101090 2024-04-21T08:47:00.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.339+0000 7fba05d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba08106160 0x7fba08199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.339+0000 7fba05d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba08106160 0x7fba08199b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37078/0 (socket says 172.21.15.86:37078) 2024-04-21T08:47:00.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.339+0000 7fba05d9b640 1 -- 172.21.15.86:0/2335152824 learned_addr learned my addr 172.21.15.86:0/2335152824 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:00.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.339+0000 7fba0659c640 1 --2- 172.21.15.86:0/2335152824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fba08101090 0x7fba08199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.339+0000 7fba06d9d640 1 --2- 172.21.15.86:0/2335152824 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba0819a0d0 0x7fba081a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.339+0000 7fba05d9b640 1 -- 172.21.15.86:0/2335152824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fba08101090 msgr2=0x7fba08199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:00.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.339+0000 7fba05d9b640 1 --2- 172.21.15.86:0/2335152824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fba08101090 0x7fba08199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.339+0000 7fba05d9b640 1 -- 172.21.15.86:0/2335152824 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba0819a0d0 msgr2=0x7fba081a4680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:00.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.339+0000 7fba05d9b640 1 --2- 172.21.15.86:0/2335152824 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba0819a0d0 0x7fba081a4680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.339+0000 7fba05d9b640 1 -- 172.21.15.86:0/2335152824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fba081a4e80 con 0x7fba08106160 2024-04-21T08:47:00.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.339+0000 7fba05d9b640 1 --2- 172.21.15.86:0/2335152824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba08106160 0x7fba08199b40 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0036040 tx=0x7fb9f002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:00.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.339+0000 7fb9ff7fe640 1 -- 172.21.15.86:0/2335152824 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0043020 con 0x7fba08106160 2024-04-21T08:47:00.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.340+0000 7fb9ff7fe640 1 -- 172.21.15.86:0/2335152824 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9f002cbe0 con 0x7fba08106160 2024-04-21T08:47:00.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.340+0000 7fba0cd52640 1 -- 172.21.15.86:0/2335152824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fba081a5110 con 0x7fba08106160 2024-04-21T08:47:00.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.340+0000 7fb9ff7fe640 1 -- 172.21.15.86:0/2335152824 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f0003bf0 con 0x7fba08106160 2024-04-21T08:47:00.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.340+0000 7fba0cd52640 1 -- 172.21.15.86:0/2335152824 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fba081a5660 con 0x7fba08106160 2024-04-21T08:47:00.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.342+0000 7fb9ff7fe640 1 -- 172.21.15.86:0/2335152824 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb9f0046020 con 0x7fba08106160 2024-04-21T08:47:00.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.342+0000 7fba0fa28740 1 Processor -- start 2024-04-21T08:47:00.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.342+0000 7fba0fa28740 1 -- start start 2024-04-21T08:47:00.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.342+0000 7fba0fa28740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b2a5d58e0 0x556b2a5d5cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.342+0000 7fba0fa28740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b2a5d6290 0x556b2a5de840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.342+0000 7fba0fa28740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b2a5dee10 0x556b2a5e1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.342+0000 7fba0fa28740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556b2a539f00 con 0x556b2a5d6290 2024-04-21T08:47:00.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.342+0000 7fba0fa28740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x556b2a4d7590 con 0x556b2a5d58e0 2024-04-21T08:47:00.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.342+0000 7fba0fa28740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556b2a535ec0 con 0x556b2a5dee10 2024-04-21T08:47:00.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.342+0000 7fb9ff7fe640 1 --2- 172.21.15.86:0/2335152824 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb9f40a3d20 0x7fb9f40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.342+0000 7fba0659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b2a5d6290 0x556b2a5de840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.342+0000 7fba0659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b2a5d6290 0x556b2a5de840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37080/0 (socket says 172.21.15.86:37080) 2024-04-21T08:47:00.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.342+0000 7fba0659c640 1 -- 172.21.15.86:0/2919403179 learned_addr learned my addr 172.21.15.86:0/2919403179 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:00.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.343+0000 7fba06d9d640 1 --2- 172.21.15.86:0/2919403179 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b2a5d58e0 0x556b2a5d5cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.343+0000 7fba05d9b640 1 --2- 172.21.15.86:0/2919403179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b2a5dee10 0x556b2a5e1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.343+0000 7fba06d9d640 1 --2- 172.21.15.86:0/2335152824 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb9f40a3d20 0x7fb9f40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.343+0000 7fb9ff7fe640 1 -- 172.21.15.86:0/2335152824 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fb9f00f2d60 con 0x7fba08106160 2024-04-21T08:47:00.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.343+0000 7fba0659c640 1 -- 172.21.15.86:0/2919403179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b2a5dee10 msgr2=0x556b2a5e1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:00.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.343+0000 7fba0659c640 1 --2- 172.21.15.86:0/2919403179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b2a5dee10 0x556b2a5e1260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.343+0000 7fba0659c640 1 -- 172.21.15.86:0/2919403179 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b2a5d58e0 msgr2=0x556b2a5d5cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:00.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.343+0000 7fba0659c640 1 --2- 172.21.15.86:0/2919403179 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b2a5d58e0 0x556b2a5d5cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.343+0000 7fba0659c640 1 -- 172.21.15.86:0/2919403179 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b2a3147e0 con 0x556b2a5d6290 2024-04-21T08:47:00.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.343+0000 7fba0659c640 1 --2- 172.21.15.86:0/2919403179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b2a5d6290 0x556b2a5de840 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800bd10 tx=0x7fb9f80107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f9a750f8c8d16a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:00.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.343+0000 7fba06d9d640 1 --2- 172.21.15.86:0/2335152824 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb9f40a3d20 0x7fb9f40a61e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fba000070b0 tx=0x7fba000093d0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:00.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.343+0000 7fb9fcff9640 1 -- 172.21.15.86:0/2919403179 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f8004030 con 0x556b2a5d6290 2024-04-21T08:47:00.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.343+0000 7fb9fcff9640 1 -- 172.21.15.86:0/2919403179 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9f80035f0 con 0x556b2a5d6290 2024-04-21T08:47:00.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.343+0000 7fb9fcff9640 1 -- 172.21.15.86:0/2919403179 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f80117b0 con 0x556b2a5d6290 2024-04-21T08:47:00.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.344+0000 7fba0fa28740 1 -- 172.21.15.86:0/2919403179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b2a5d6290 msgr2=0x556b2a5de840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:00.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.344+0000 7fba0fa28740 1 --2- 172.21.15.86:0/2919403179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b2a5d6290 0x556b2a5de840 secure :-1 s=READY pgs=1605 cs=0 l=1 rev1=1 crypto rx=0x7fb9f800bd10 tx=0x7fb9f80107a0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.344+0000 7fba0fa28740 1 -- 172.21.15.86:0/2919403179 shutdown_connections 2024-04-21T08:47:00.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.344+0000 7fba0fa28740 1 --2- 172.21.15.86:0/2919403179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b2a5dee10 0x556b2a5e1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.344+0000 7fba0fa28740 1 --2- 172.21.15.86:0/2919403179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b2a5d6290 0x556b2a5de840 unknown :-1 s=CLOSED pgs=1605 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.344+0000 7fba0fa28740 1 --2- 172.21.15.86:0/2919403179 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b2a5d58e0 0x556b2a5d5cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.344+0000 7fba0fa28740 1 -- 172.21.15.86:0/2919403179 >> 172.21.15.86:0/2919403179 conn(0x556b2a5cffb0 msgr2=0x556b2a5d0390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:00.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.344+0000 7fba0fa28740 1 -- 172.21.15.86:0/2919403179 shutdown_connections 2024-04-21T08:47:00.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.344+0000 7fba0fa28740 1 -- 172.21.15.86:0/2919403179 wait complete. 2024-04-21T08:47:00.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.344+0000 7fba0fa28740 1 -- start start 2024-04-21T08:47:00.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.344+0000 7fba0fa28740 1 Processor -- start 2024-04-21T08:47:00.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.345+0000 7fba0fa28740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b2a5d58e0 0x556b2a5dbde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.345+0000 7fba0fa28740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b2a5d6290 0x556b2a5dc320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.345+0000 7fba0fa28740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b2a5dee10 0x556b2a5dc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.345+0000 7fba0fa28740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556b2a5e1d80 con 0x556b2a5dee10 2024-04-21T08:47:00.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.345+0000 7fba0fa28740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x556b2a4f3580 con 0x556b2a5d6290 2024-04-21T08:47:00.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.345+0000 7fba0fa28740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556b2a5e1f00 con 0x556b2a5d58e0 2024-04-21T08:47:00.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.345+0000 7fba0fa28740 10 client.0 _ll_register_callbacks cb 0x556b2a4f9650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:00.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.345+0000 7fba06d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b2a5dee10 0x556b2a5dc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.345+0000 7fba06d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b2a5dee10 0x556b2a5dc860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37090/0 (socket says 172.21.15.86:37090) 2024-04-21T08:47:00.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.345+0000 7fba06d9d640 1 -- 172.21.15.86:0/2521765608 learned_addr learned my addr 172.21.15.86:0/2521765608 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:00.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.345+0000 7fba0659c640 1 --2- 172.21.15.86:0/2521765608 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b2a5d58e0 0x556b2a5dbde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.345+0000 7fba05d9b640 1 --2- 172.21.15.86:0/2521765608 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b2a5d6290 0x556b2a5dc320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.345+0000 7fba06d9d640 1 -- 172.21.15.86:0/2521765608 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b2a5d58e0 msgr2=0x556b2a5dbde0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:00.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.345+0000 7fba06d9d640 1 --2- 172.21.15.86:0/2521765608 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b2a5d58e0 0x556b2a5dbde0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.345+0000 7fba06d9d640 1 -- 172.21.15.86:0/2521765608 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b2a5d6290 msgr2=0x556b2a5dc320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:00.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.345+0000 7fba06d9d640 1 --2- 172.21.15.86:0/2521765608 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b2a5d6290 0x556b2a5dc320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.345+0000 7fba06d9d640 1 -- 172.21.15.86:0/2521765608 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b2a374f70 con 0x556b2a5dee10 2024-04-21T08:47:00.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.345+0000 7fba05d9b640 1 --2- 172.21.15.86:0/2521765608 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b2a5d6290 0x556b2a5dc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:00.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.345+0000 7fba06d9d640 1 --2- 172.21.15.86:0/2521765608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b2a5dee10 0x556b2a5dc860 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fba00009d60 tx=0x7fba00004b50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d07c5475e3509c9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:00.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.345+0000 7fb9edffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:47:00.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.345+0000 7fb9edffb640 1 -- 172.21.15.86:0/2521765608 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0001f660 con 0x556b2a5dee10 2024-04-21T08:47:00.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.345+0000 7fb9edffb640 1 -- 172.21.15.86:0/2521765608 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fba0001f800 con 0x556b2a5dee10 2024-04-21T08:47:00.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.345+0000 7fba0fa28740 1 -- 172.21.15.86:0/2521765608 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9f80035f0 con 0x556b2a5dee10 2024-04-21T08:47:00.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.346+0000 7fb9edffb640 1 -- 172.21.15.86:0/2521765608 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fba0001fa00 con 0x556b2a5dee10 2024-04-21T08:47:00.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.346+0000 7fba0fa28740 10 client.19026 fetch_fsmap learned FSMap version 29 2024-04-21T08:47:00.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.346+0000 7fba0fa28740 1 -- 172.21.15.86:0/2521765608 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556b2a20c170 con 0x556b2a5dee10 2024-04-21T08:47:00.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.346+0000 7fb9edffb640 1 -- 172.21.15.86:0/2521765608 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9f80035f0 con 0x556b2a5dee10 2024-04-21T08:47:00.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.346+0000 7fb9edffb640 1 -- 172.21.15.86:0/2521765608 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fba00028070 con 0x556b2a5dee10 2024-04-21T08:47:00.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.346+0000 7fba0fa28740 10 client.19026 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:47:00.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.346+0000 7fba0fa28740 10 client.19026 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:00.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.346+0000 7fba0fa28740 20 client.19026 populate_metadata read hostname 'smithi086' 2024-04-21T08:47:00.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.346+0000 7fba0fa28740 1 --2- 172.21.15.86:0/2521765608 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x556b2a5ef420 0x556b2a5f1870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.346+0000 7fba0fa28740 4 client.19026 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:00.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.346+0000 7fba0fa28740 1 -- 172.21.15.86:0/2521765608 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556b2a547920 con 0x556b2a5ef420 2024-04-21T08:47:00.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.346+0000 7fba0659c640 1 --2- 172.21.15.86:0/2521765608 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x556b2a5ef420 0x556b2a5f1870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.347+0000 7fba0659c640 1 --2- 172.21.15.86:0/2521765608 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x556b2a5ef420 0x556b2a5f1870 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3a25b4029a0450d0 server_cookie=5b5cca50645b1115 in_seq=0 out_seq=0 2024-04-21T08:47:00.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.347+0000 7fb9edffb640 10 client.19026 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:00.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.347+0000 7fb9edffb640 1 -- 172.21.15.86:0/2521765608 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556b2a547920 con 0x556b2a5ef420 2024-04-21T08:47:00.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.347+0000 7fb9edffb640 10 client.19026 handle_command_reply: tid=0 2024-04-21T08:47:00.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.347+0000 7fba0fa28740 1 client.19026 shutdown 2024-04-21T08:47:00.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.347+0000 7fba0fa28740 1 -- 172.21.15.86:0/2521765608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b2a5dee10 msgr2=0x556b2a5dc860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:00.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.347+0000 7fba0fa28740 1 --2- 172.21.15.86:0/2521765608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b2a5dee10 0x556b2a5dc860 secure :-1 s=READY pgs=1606 cs=0 l=1 rev1=1 crypto rx=0x7fba00009d60 tx=0x7fba00004b50 comp rx=0 tx=0).stop 2024-04-21T08:47:00.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.347+0000 7fba0fa28740 1 -- 172.21.15.86:0/2521765608 shutdown_connections 2024-04-21T08:47:00.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.348+0000 7fba0fa28740 1 --2- 172.21.15.86:0/2521765608 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x556b2a5ef420 0x556b2a5f1870 crc :-1 s=READY pgs=249 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.348+0000 7fba0fa28740 1 --2- 172.21.15.86:0/2521765608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b2a5dee10 0x556b2a5dc860 unknown :-1 s=CLOSED pgs=1606 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.348+0000 7fba0fa28740 1 --2- 172.21.15.86:0/2521765608 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b2a5d6290 0x556b2a5dc320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.348+0000 7fb9edffb640 0 client.19026 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:00.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.348+0000 7fba0fa28740 1 --2- 172.21.15.86:0/2521765608 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b2a5d58e0 0x556b2a5dbde0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.348+0000 7fba0fa28740 1 -- 172.21.15.86:0/2521765608 >> 172.21.15.86:0/2521765608 conn(0x556b2a5cffb0 msgr2=0x556b2a5d1d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:00.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.348+0000 7fba0fa28740 1 -- 172.21.15.86:0/2521765608 shutdown_connections 2024-04-21T08:47:00.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.348+0000 7fba0fa28740 1 -- 172.21.15.86:0/2521765608 wait complete. 2024-04-21T08:47:00.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.348+0000 7fba0fa28740 20 client.19026 trim_cache size 0 max 16384 2024-04-21T08:47:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.365+0000 7fba0fa28740 1 Processor -- start 2024-04-21T08:47:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.365+0000 7fba0fa28740 1 -- start start 2024-04-21T08:47:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.365+0000 7fba0fa28740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b2a5d58e0 0x556b2a5de510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.365+0000 7fba0fa28740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b2a5d6290 0x556b2a5eac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.365+0000 7fba0fa28740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b2a5dee10 0x556b2a5f7140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.365+0000 7fba0fa28740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba081ad1e0 con 0x556b2a5d6290 2024-04-21T08:47:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.365+0000 7fba0fa28740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fba081aed20 con 0x556b2a5dee10 2024-04-21T08:47:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.365+0000 7fba0fa28740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fba081ad8b0 con 0x556b2a5d58e0 2024-04-21T08:47:00.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.365+0000 7fba05d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b2a5d6290 0x556b2a5eac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.365+0000 7fba05d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b2a5d6290 0x556b2a5eac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37106/0 (socket says 172.21.15.86:37106) 2024-04-21T08:47:00.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.365+0000 7fba05d9b640 1 -- 172.21.15.86:0/2576039629 learned_addr learned my addr 172.21.15.86:0/2576039629 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:00.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.365+0000 7fba06d9d640 1 --2- 172.21.15.86:0/2576039629 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b2a5dee10 0x556b2a5f7140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.365+0000 7fba05d9b640 1 -- 172.21.15.86:0/2576039629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b2a5d58e0 msgr2=0x556b2a5de510 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:47:00.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.365+0000 7fba05d9b640 1 --2- 172.21.15.86:0/2576039629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b2a5d58e0 0x556b2a5de510 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.365+0000 7fba05d9b640 1 -- 172.21.15.86:0/2576039629 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b2a5dee10 msgr2=0x556b2a5f7140 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:00.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.365+0000 7fba05d9b640 1 --2- 172.21.15.86:0/2576039629 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b2a5dee10 0x556b2a5f7140 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.365+0000 7fba05d9b640 1 -- 172.21.15.86:0/2576039629 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556b2a5d4ba0 con 0x556b2a5d6290 2024-04-21T08:47:00.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.365+0000 7fba06d9d640 1 --2- 172.21.15.86:0/2576039629 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b2a5dee10 0x556b2a5f7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:00.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.366+0000 7fba05d9b640 1 --2- 172.21.15.86:0/2576039629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b2a5d6290 0x556b2a5eac70 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fb9f00baea0 tx=0x7fb9f0091530 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ab5b148823d84445 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:00.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.366+0000 7fb9effff640 1 -- 172.21.15.86:0/2576039629 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f00be3a0 con 0x556b2a5d6290 2024-04-21T08:47:00.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.366+0000 7fb9effff640 1 -- 172.21.15.86:0/2576039629 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9f00bf140 con 0x556b2a5d6290 2024-04-21T08:47:00.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.366+0000 7fb9effff640 1 -- 172.21.15.86:0/2576039629 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb9f00054d0 con 0x556b2a5d6290 2024-04-21T08:47:00.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.366+0000 7fba0fa28740 1 -- 172.21.15.86:0/2576039629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b2a5d6290 msgr2=0x556b2a5eac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:00.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.366+0000 7fba0fa28740 1 --2- 172.21.15.86:0/2576039629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b2a5d6290 0x556b2a5eac70 secure :-1 s=READY pgs=1607 cs=0 l=1 rev1=1 crypto rx=0x7fb9f00baea0 tx=0x7fb9f0091530 comp rx=0 tx=0).stop 2024-04-21T08:47:00.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.366+0000 7fba0fa28740 1 -- 172.21.15.86:0/2576039629 shutdown_connections 2024-04-21T08:47:00.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.366+0000 7fba0fa28740 1 --2- 172.21.15.86:0/2576039629 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b2a5dee10 0x556b2a5f7140 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.366+0000 7fba0fa28740 1 --2- 172.21.15.86:0/2576039629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b2a5d6290 0x556b2a5eac70 unknown :-1 s=CLOSED pgs=1607 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.366+0000 7fba0fa28740 1 --2- 172.21.15.86:0/2576039629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b2a5d58e0 0x556b2a5de510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.366+0000 7fba0fa28740 1 -- 172.21.15.86:0/2576039629 >> 172.21.15.86:0/2576039629 conn(0x556b2a5cffb0 msgr2=0x556b2a5cf010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:00.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.366+0000 7fba0fa28740 1 -- 172.21.15.86:0/2576039629 shutdown_connections 2024-04-21T08:47:00.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.366+0000 7fba0fa28740 1 -- 172.21.15.86:0/2576039629 wait complete. 2024-04-21T08:47:00.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.367+0000 7fba0fa28740 1 -- start start 2024-04-21T08:47:00.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.367+0000 7fba0fa28740 1 Processor -- start 2024-04-21T08:47:00.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.367+0000 7fba0fa28740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b2a5d58e0 0x556b2a5e7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.367+0000 7fba0fa28740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b2a5d6290 0x556b2a5e76c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.367+0000 7fba0fa28740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b2a5dee10 0x556b2a5e4150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.367+0000 7fba0fa28740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fba081ae650 con 0x556b2a5dee10 2024-04-21T08:47:00.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.367+0000 7fba0fa28740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fba081adf80 con 0x556b2a5d6290 2024-04-21T08:47:00.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.367+0000 7fba0fa28740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fba081ab1e0 con 0x556b2a5d58e0 2024-04-21T08:47:00.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.367+0000 7fba0fa28740 10 client.0 _ll_register_callbacks cb 0x7fba00009c70 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:00.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.367+0000 7fba06d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b2a5dee10 0x556b2a5e4150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.367+0000 7fba06d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b2a5dee10 0x556b2a5e4150 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37116/0 (socket says 172.21.15.86:37116) 2024-04-21T08:47:00.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.367+0000 7fba06d9d640 1 -- 172.21.15.86:0/2843455033 learned_addr learned my addr 172.21.15.86:0/2843455033 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:00.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.367+0000 7fba0659c640 1 --2- 172.21.15.86:0/2843455033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b2a5d58e0 0x556b2a5e7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.367+0000 7fba06d9d640 1 -- 172.21.15.86:0/2843455033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b2a5d58e0 msgr2=0x556b2a5e7180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:00.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.367+0000 7fba06d9d640 1 --2- 172.21.15.86:0/2843455033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b2a5d58e0 0x556b2a5e7180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.367+0000 7fba06d9d640 1 -- 172.21.15.86:0/2843455033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b2a5d6290 msgr2=0x556b2a5e76c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:00.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.367+0000 7fba05d9b640 1 --2- 172.21.15.86:0/2843455033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b2a5d6290 0x556b2a5e76c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.367+0000 7fba06d9d640 1 --2- 172.21.15.86:0/2843455033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b2a5d6290 0x556b2a5e76c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.367+0000 7fba06d9d640 1 -- 172.21.15.86:0/2843455033 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556b2a5e4950 con 0x556b2a5dee10 2024-04-21T08:47:00.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.367+0000 7fba06d9d640 1 --2- 172.21.15.86:0/2843455033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b2a5dee10 0x556b2a5e4150 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fba00009850 tx=0x7fba00004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42381c825ad00e19 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:00.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.367+0000 7fb9edffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:47:00.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.367+0000 7fb9edffb640 1 -- 172.21.15.86:0/2843455033 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fba0002ae60 con 0x556b2a5dee10 2024-04-21T08:47:00.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.368+0000 7fb9edffb640 1 -- 172.21.15.86:0/2843455033 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7fba0002a440 con 0x556b2a5dee10 2024-04-21T08:47:00.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.368+0000 7fba0fa28740 1 -- 172.21.15.86:0/2843455033 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb9f00bf140 con 0x556b2a5dee10 2024-04-21T08:47:00.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.368+0000 7fb9edffb640 1 -- 172.21.15.86:0/2843455033 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fba00046050 con 0x556b2a5dee10 2024-04-21T08:47:00.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.368+0000 7fba0fa28740 10 client.19032 fetch_fsmap learned FSMap version 29 2024-04-21T08:47:00.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.368+0000 7fba0fa28740 1 -- 172.21.15.86:0/2843455033 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556b2a656440 con 0x556b2a5dee10 2024-04-21T08:47:00.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.368+0000 7fb9edffb640 1 -- 172.21.15.86:0/2843455033 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb9f00bf140 con 0x556b2a5dee10 2024-04-21T08:47:00.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.368+0000 7fb9edffb640 1 -- 172.21.15.86:0/2843455033 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fba00028040 con 0x556b2a5dee10 2024-04-21T08:47:00.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.368+0000 7fba0fa28740 10 client.19032 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:47:00.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.368+0000 7fba0fa28740 10 client.19032 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:00.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.368+0000 7fba0fa28740 20 client.19032 populate_metadata read hostname 'smithi086' 2024-04-21T08:47:00.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.368+0000 7fba0fa28740 1 --2- 172.21.15.86:0/2843455033 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x556b2a656b80 0x556b2a658fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.368+0000 7fba0fa28740 4 client.19032 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:47:00.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.368+0000 7fba0fa28740 1 -- 172.21.15.86:0/2843455033 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x556b2a541790 con 0x556b2a656b80 2024-04-21T08:47:00.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.368+0000 7fba0659c640 1 --2- 172.21.15.86:0/2843455033 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x556b2a656b80 0x556b2a658fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.369+0000 7fb9edffb640 10 client.19032 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:00.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.369+0000 7fba0659c640 1 --2- 172.21.15.86:0/2843455033 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x556b2a656b80 0x556b2a658fa0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=39b3fed2496e1ed6 server_cookie=9f991fb0fd7d2b6a in_seq=0 out_seq=0 2024-04-21T08:47:00.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.370+0000 7fb9edffb640 1 -- 172.21.15.86:0/2843455033 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x556b2a541790 con 0x556b2a656b80 2024-04-21T08:47:00.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.370+0000 7fb9edffb640 10 client.19032 handle_command_reply: tid=0 2024-04-21T08:47:00.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.370+0000 7fba0fa28740 1 client.19032 shutdown 2024-04-21T08:47:00.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.370+0000 7fba0fa28740 1 -- 172.21.15.86:0/2843455033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b2a5dee10 msgr2=0x556b2a5e4150 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:00.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.370+0000 7fba0fa28740 1 --2- 172.21.15.86:0/2843455033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b2a5dee10 0x556b2a5e4150 secure :-1 s=READY pgs=1608 cs=0 l=1 rev1=1 crypto rx=0x7fba00009850 tx=0x7fba00004a40 comp rx=0 tx=0).stop 2024-04-21T08:47:00.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.370+0000 7fba0fa28740 1 -- 172.21.15.86:0/2843455033 shutdown_connections 2024-04-21T08:47:00.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.370+0000 7fba0fa28740 1 --2- 172.21.15.86:0/2843455033 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x556b2a656b80 0x556b2a658fa0 crc :-1 s=READY pgs=250 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.370+0000 7fba0fa28740 1 --2- 172.21.15.86:0/2843455033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556b2a5dee10 0x556b2a5e4150 unknown :-1 s=CLOSED pgs=1608 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.370+0000 7fba0fa28740 1 --2- 172.21.15.86:0/2843455033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556b2a5d6290 0x556b2a5e76c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.370+0000 7fba0fa28740 1 --2- 172.21.15.86:0/2843455033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556b2a5d58e0 0x556b2a5e7180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.370+0000 7fba0fa28740 1 -- 172.21.15.86:0/2843455033 >> 172.21.15.86:0/2843455033 conn(0x556b2a5cffb0 msgr2=0x556b2a5ce780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:00.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.370+0000 7fb9edffb640 0 client.19032 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:00.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.370+0000 7fba0fa28740 1 -- 172.21.15.86:0/2843455033 shutdown_connections 2024-04-21T08:47:00.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.370+0000 7fba0fa28740 1 -- 172.21.15.86:0/2843455033 wait complete. 2024-04-21T08:47:00.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.370+0000 7fba0fa28740 20 client.19032 trim_cache size 0 max 16384 2024-04-21T08:47:00.376 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:47:00.376 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-21T08:47:00.376 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "45f7e4fa-a859-43d7-897e-86661f787232", 2024-04-21T08:47:00.376 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-21T08:47:00.376 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:47:00.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.371+0000 7fba0cd52640 1 -- 172.21.15.86:0/2335152824 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb9f40a3d20 msgr2=0x7fb9f40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:00.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.371+0000 7fba0cd52640 1 --2- 172.21.15.86:0/2335152824 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb9f40a3d20 0x7fb9f40a61e0 secure :-1 s=READY pgs=261 cs=0 l=1 rev1=1 crypto rx=0x7fba000070b0 tx=0x7fba000093d0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.371+0000 7fba0cd52640 1 -- 172.21.15.86:0/2335152824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba08106160 msgr2=0x7fba08199b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:00.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.371+0000 7fba0cd52640 1 --2- 172.21.15.86:0/2335152824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba08106160 0x7fba08199b40 secure :-1 s=READY pgs=1604 cs=0 l=1 rev1=1 crypto rx=0x7fb9f0036040 tx=0x7fb9f002c7e0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.371+0000 7fba0cd52640 1 -- 172.21.15.86:0/2335152824 shutdown_connections 2024-04-21T08:47:00.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.371+0000 7fba0cd52640 1 --2- 172.21.15.86:0/2335152824 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb9f40a3d20 0x7fb9f40a61e0 unknown :-1 s=CLOSED pgs=261 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.371+0000 7fba0cd52640 1 --2- 172.21.15.86:0/2335152824 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fba0819a0d0 0x7fba081a4680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.371+0000 7fba0cd52640 1 --2- 172.21.15.86:0/2335152824 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fba08106160 0x7fba08199b40 unknown :-1 s=CLOSED pgs=1604 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.371+0000 7fba0cd52640 1 --2- 172.21.15.86:0/2335152824 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fba08101090 0x7fba08199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.371+0000 7fba0cd52640 1 -- 172.21.15.86:0/2335152824 >> 172.21.15.86:0/2335152824 conn(0x7fba080771d0 msgr2=0x7fba080fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:00.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.371+0000 7fba0cd52640 1 -- 172.21.15.86:0/2335152824 shutdown_connections 2024-04-21T08:47:00.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.371+0000 7fba0cd52640 1 -- 172.21.15.86:0/2335152824 wait complete. 2024-04-21T08:47:00.381 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:47:00.382 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:47:00.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.481+0000 7f68ece26640 1 -- 172.21.15.86:0/3827008960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68e8101a40 msgr2=0x7f68e8105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:00.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.481+0000 7f68ece26640 1 --2- 172.21.15.86:0/3827008960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68e8101a40 0x7f68e8105a80 secure :-1 s=READY pgs=1609 cs=0 l=1 rev1=1 crypto rx=0x7f68d0009900 tx=0x7f68d002bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:00.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.481+0000 7f68ece26640 1 -- 172.21.15.86:0/3827008960 shutdown_connections 2024-04-21T08:47:00.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.481+0000 7f68ece26640 1 --2- 172.21.15.86:0/3827008960 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68e8106160 0x7f68e810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.481+0000 7f68ece26640 1 --2- 172.21.15.86:0/3827008960 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68e8101a40 0x7f68e8105a80 unknown :-1 s=CLOSED pgs=1609 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.481+0000 7f68ece26640 1 --2- 172.21.15.86:0/3827008960 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68e8101090 0x7f68e8101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.481+0000 7f68ece26640 1 -- 172.21.15.86:0/3827008960 >> 172.21.15.86:0/3827008960 conn(0x7f68e80771d0 msgr2=0x7f68e80775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:00.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.481+0000 7f68ece26640 1 -- 172.21.15.86:0/3827008960 shutdown_connections 2024-04-21T08:47:00.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.481+0000 7f68ece26640 1 -- 172.21.15.86:0/3827008960 wait complete. 2024-04-21T08:47:00.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.482+0000 7f68ece26640 1 Processor -- start 2024-04-21T08:47:00.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.482+0000 7f68ece26640 1 -- start start 2024-04-21T08:47:00.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.482+0000 7f68ece26640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68e8101090 0x7f68e8199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.482+0000 7f68ece26640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68e8106160 0x7f68e8199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.482+0000 7f68ece26640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68e819a050 0x7f68e81a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.482+0000 7f68ece26640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f68e81110a0 con 0x7f68e8106160 2024-04-21T08:47:00.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.482+0000 7f68ece26640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f68e8110f20 con 0x7f68e8101090 2024-04-21T08:47:00.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.482+0000 7f68ece26640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f68e8111220 con 0x7f68e819a050 2024-04-21T08:47:00.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.482+0000 7f68dffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68e8106160 0x7f68e8199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.482+0000 7f68dffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68e8106160 0x7f68e8199ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37126/0 (socket says 172.21.15.86:37126) 2024-04-21T08:47:00.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.482+0000 7f68dffff640 1 -- 172.21.15.86:0/4144641717 learned_addr learned my addr 172.21.15.86:0/4144641717 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:00.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.483+0000 7f68e659c640 1 --2- 172.21.15.86:0/4144641717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68e8101090 0x7f68e8199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.483+0000 7f68dffff640 1 -- 172.21.15.86:0/4144641717 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68e819a050 msgr2=0x7f68e81a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:00.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.483+0000 7f68dffff640 1 --2- 172.21.15.86:0/4144641717 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68e819a050 0x7f68e81a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.483+0000 7f68dffff640 1 -- 172.21.15.86:0/4144641717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68e8101090 msgr2=0x7f68e8199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:00.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.483+0000 7f68dffff640 1 --2- 172.21.15.86:0/4144641717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68e8101090 0x7f68e8199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.483+0000 7f68dffff640 1 -- 172.21.15.86:0/4144641717 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f68e81a4dd0 con 0x7f68e8106160 2024-04-21T08:47:00.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.483+0000 7f68e659c640 1 --2- 172.21.15.86:0/4144641717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68e8101090 0x7f68e8199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:00.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.483+0000 7f68dffff640 1 --2- 172.21.15.86:0/4144641717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68e8106160 0x7f68e8199ac0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f68d0036040 tx=0x7f68d002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:00.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.483+0000 7f68df7fe640 1 -- 172.21.15.86:0/4144641717 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d0043020 con 0x7f68e8106160 2024-04-21T08:47:00.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.483+0000 7f68df7fe640 1 -- 172.21.15.86:0/4144641717 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68d002cbe0 con 0x7f68e8106160 2024-04-21T08:47:00.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.483+0000 7f68ece26640 1 -- 172.21.15.86:0/4144641717 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f68e81a5060 con 0x7f68e8106160 2024-04-21T08:47:00.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.483+0000 7f68df7fe640 1 -- 172.21.15.86:0/4144641717 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d0003bf0 con 0x7f68e8106160 2024-04-21T08:47:00.485 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.483+0000 7f68ece26640 1 -- 172.21.15.86:0/4144641717 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f68e81a55b0 con 0x7f68e8106160 2024-04-21T08:47:00.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.485+0000 7f68df7fe640 1 -- 172.21.15.86:0/4144641717 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f68d0046020 con 0x7f68e8106160 2024-04-21T08:47:00.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.486+0000 7f68df7fe640 1 --2- 172.21.15.86:0/4144641717 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f68d40a3d20 0x7f68d40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.486+0000 7f68efa0e740 1 Processor -- start 2024-04-21T08:47:00.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.486+0000 7f68e659c640 1 --2- 172.21.15.86:0/4144641717 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f68d40a3d20 0x7f68d40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.486+0000 7f68df7fe640 1 -- 172.21.15.86:0/4144641717 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f68d00f2dc0 con 0x7f68e8106160 2024-04-21T08:47:00.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.486+0000 7f68efa0e740 1 -- start start 2024-04-21T08:47:00.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.486+0000 7f68efa0e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a181d54580 0x55a181d54960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.486+0000 7f68efa0e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a181d54ea0 0x55a181d5d540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.486+0000 7f68efa0e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a181d5da80 0x55a181d5fe70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.486+0000 7f68efa0e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a181cb91e0 con 0x55a181d5da80 2024-04-21T08:47:00.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.486+0000 7f68efa0e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55a181c56390 con 0x55a181d54580 2024-04-21T08:47:00.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.486+0000 7f68efa0e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a181cb51a0 con 0x55a181d54ea0 2024-04-21T08:47:00.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.486+0000 7f68e659c640 1 --2- 172.21.15.86:0/4144641717 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f68d40a3d20 0x7f68d40a61e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f68d80058c0 tx=0x7f68d8005850 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:00.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.486+0000 7f68e6d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a181d5da80 0x55a181d5fe70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.486+0000 7f68e6d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a181d5da80 0x55a181d5fe70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37142/0 (socket says 172.21.15.86:37142) 2024-04-21T08:47:00.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.486+0000 7f68e6d9d640 1 -- 172.21.15.86:0/2083524391 learned_addr learned my addr 172.21.15.86:0/2083524391 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:00.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.486+0000 7f68e659c640 1 --2- 172.21.15.86:0/2083524391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a181d54580 0x55a181d54960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.486+0000 7f68dffff640 1 --2- 172.21.15.86:0/2083524391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a181d54ea0 0x55a181d5d540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.486+0000 7f68e6d9d640 1 -- 172.21.15.86:0/2083524391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a181d54ea0 msgr2=0x55a181d5d540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:00.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.486+0000 7f68e6d9d640 1 --2- 172.21.15.86:0/2083524391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a181d54ea0 0x55a181d5d540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.486+0000 7f68e6d9d640 1 -- 172.21.15.86:0/2083524391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a181d54580 msgr2=0x55a181d54960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:00.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.486+0000 7f68e6d9d640 1 --2- 172.21.15.86:0/2083524391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a181d54580 0x55a181d54960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.486+0000 7f68e6d9d640 1 -- 172.21.15.86:0/2083524391 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a181a957d0 con 0x55a181d5da80 2024-04-21T08:47:00.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.487+0000 7f68e6d9d640 1 --2- 172.21.15.86:0/2083524391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a181d5da80 0x55a181d5fe70 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f68e0009d50 tx=0x7f68e000d6b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1823d8efd25fdfca server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:00.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.487+0000 7f68dcff9640 1 -- 172.21.15.86:0/2083524391 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e0003970 con 0x55a181d5da80 2024-04-21T08:47:00.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.487+0000 7f68dcff9640 1 -- 172.21.15.86:0/2083524391 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68e0004020 con 0x55a181d5da80 2024-04-21T08:47:00.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.487+0000 7f68dcff9640 1 -- 172.21.15.86:0/2083524391 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68e00106a0 con 0x55a181d5da80 2024-04-21T08:47:00.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.487+0000 7f68efa0e740 1 -- 172.21.15.86:0/2083524391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a181d5da80 msgr2=0x55a181d5fe70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:00.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.487+0000 7f68efa0e740 1 --2- 172.21.15.86:0/2083524391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a181d5da80 0x55a181d5fe70 secure :-1 s=READY pgs=1611 cs=0 l=1 rev1=1 crypto rx=0x7f68e0009d50 tx=0x7f68e000d6b0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.488+0000 7f68efa0e740 1 -- 172.21.15.86:0/2083524391 shutdown_connections 2024-04-21T08:47:00.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.488+0000 7f68efa0e740 1 --2- 172.21.15.86:0/2083524391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a181d5da80 0x55a181d5fe70 unknown :-1 s=CLOSED pgs=1611 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.488+0000 7f68efa0e740 1 --2- 172.21.15.86:0/2083524391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a181d54ea0 0x55a181d5d540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.488+0000 7f68efa0e740 1 --2- 172.21.15.86:0/2083524391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a181d54580 0x55a181d54960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.488+0000 7f68efa0e740 1 -- 172.21.15.86:0/2083524391 >> 172.21.15.86:0/2083524391 conn(0x55a181d4ecb0 msgr2=0x55a181d4f090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:00.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.488+0000 7f68efa0e740 1 -- 172.21.15.86:0/2083524391 shutdown_connections 2024-04-21T08:47:00.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.488+0000 7f68efa0e740 1 -- 172.21.15.86:0/2083524391 wait complete. 2024-04-21T08:47:00.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.488+0000 7f68efa0e740 1 -- start start 2024-04-21T08:47:00.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.488+0000 7f68efa0e740 1 Processor -- start 2024-04-21T08:47:00.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.488+0000 7f68efa0e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a181d54580 0x55a181d5aa90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.488+0000 7f68efa0e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a181d54ea0 0x55a181d5afd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.488+0000 7f68efa0e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a181d5da80 0x55a181d5b510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.488+0000 7f68efa0e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a181d60870 con 0x55a181d54580 2024-04-21T08:47:00.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.488+0000 7f68efa0e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55a181c722f0 con 0x55a181d5da80 2024-04-21T08:47:00.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.488+0000 7f68efa0e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a181d609f0 con 0x55a181d54ea0 2024-04-21T08:47:00.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.488+0000 7f68efa0e740 10 client.0 _ll_register_callbacks cb 0x55a181c7f580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:00.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.488+0000 7f68e659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a181d54580 0x55a181d5aa90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.488+0000 7f68e659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a181d54580 0x55a181d5aa90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37156/0 (socket says 172.21.15.86:37156) 2024-04-21T08:47:00.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.488+0000 7f68e659c640 1 -- 172.21.15.86:0/968744585 learned_addr learned my addr 172.21.15.86:0/968744585 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:00.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.489+0000 7f68e6d9d640 1 --2- 172.21.15.86:0/968744585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a181d5da80 0x55a181d5b510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.489+0000 7f68dffff640 1 --2- 172.21.15.86:0/968744585 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a181d54ea0 0x55a181d5afd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.489+0000 7f68e659c640 1 -- 172.21.15.86:0/968744585 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a181d54ea0 msgr2=0x55a181d5afd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:00.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.489+0000 7f68e659c640 1 --2- 172.21.15.86:0/968744585 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a181d54ea0 0x55a181d5afd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.489+0000 7f68e659c640 1 -- 172.21.15.86:0/968744585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a181d5da80 msgr2=0x55a181d5b510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:00.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.489+0000 7f68e659c640 1 --2- 172.21.15.86:0/968744585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a181d5da80 0x55a181d5b510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.489+0000 7f68e659c640 1 -- 172.21.15.86:0/968744585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a1819cb3f0 con 0x55a181d54580 2024-04-21T08:47:00.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.489+0000 7f68e659c640 1 --2- 172.21.15.86:0/968744585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a181d54580 0x55a181d5aa90 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f68d80116c0 tx=0x7f68d80046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0e31f763ac432ec server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:00.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.489+0000 7f68cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:47:00.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.489+0000 7f68cdffb640 1 -- 172.21.15.86:0/968744585 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d801f900 con 0x55a181d54580 2024-04-21T08:47:00.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.489+0000 7f68cdffb640 1 -- 172.21.15.86:0/968744585 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f68d801faa0 con 0x55a181d54580 2024-04-21T08:47:00.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.489+0000 7f68efa0e740 1 -- 172.21.15.86:0/968744585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68e0004020 con 0x55a181d54580 2024-04-21T08:47:00.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.489+0000 7f68cdffb640 1 -- 172.21.15.86:0/968744585 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f68d8030050 con 0x55a181d54580 2024-04-21T08:47:00.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.489+0000 7f68efa0e740 10 client.19044 fetch_fsmap learned FSMap version 29 2024-04-21T08:47:00.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.489+0000 7f68efa0e740 1 -- 172.21.15.86:0/968744585 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a18198b0f0 con 0x55a181d54580 2024-04-21T08:47:00.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.489+0000 7f68cdffb640 1 -- 172.21.15.86:0/968744585 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68e0004020 con 0x55a181d54580 2024-04-21T08:47:00.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.489+0000 7f68cdffb640 1 -- 172.21.15.86:0/968744585 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f68d80103d0 con 0x55a181d54580 2024-04-21T08:47:00.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.489+0000 7f68efa0e740 10 client.19044 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:47:00.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.489+0000 7f68efa0e740 10 client.19044 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:00.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.489+0000 7f68efa0e740 20 client.19044 populate_metadata read hostname 'smithi086' 2024-04-21T08:47:00.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.489+0000 7f68efa0e740 1 --2- 172.21.15.86:0/968744585 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a181d6dae0 0x55a181d6ff10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.489+0000 7f68efa0e740 4 client.19044 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:00.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.489+0000 7f68efa0e740 1 -- 172.21.15.86:0/968744585 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a181cc6c00 con 0x55a181d6dae0 2024-04-21T08:47:00.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.490+0000 7f68dffff640 1 --2- 172.21.15.86:0/968744585 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a181d6dae0 0x55a181d6ff10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.490+0000 7f68cdffb640 10 client.19044 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:00.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.490+0000 7f68dffff640 1 --2- 172.21.15.86:0/968744585 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a181d6dae0 0x55a181d6ff10 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5b0e0091fc584433 server_cookie=7cb88234f7f88a7f in_seq=0 out_seq=0 2024-04-21T08:47:00.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.491+0000 7f68cdffb640 1 -- 172.21.15.86:0/968744585 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a181cc6c00 con 0x55a181d6dae0 2024-04-21T08:47:00.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.491+0000 7f68cdffb640 10 client.19044 handle_command_reply: tid=0 2024-04-21T08:47:00.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.491+0000 7f68efa0e740 1 client.19044 shutdown 2024-04-21T08:47:00.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.491+0000 7f68efa0e740 1 -- 172.21.15.86:0/968744585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a181d54580 msgr2=0x55a181d5aa90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:00.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.491+0000 7f68efa0e740 1 --2- 172.21.15.86:0/968744585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a181d54580 0x55a181d5aa90 secure :-1 s=READY pgs=1612 cs=0 l=1 rev1=1 crypto rx=0x7f68d80116c0 tx=0x7f68d80046c0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.491+0000 7f68efa0e740 1 -- 172.21.15.86:0/968744585 shutdown_connections 2024-04-21T08:47:00.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.491+0000 7f68efa0e740 1 --2- 172.21.15.86:0/968744585 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a181d6dae0 0x55a181d6ff10 crc :-1 s=READY pgs=251 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.491+0000 7f68efa0e740 1 --2- 172.21.15.86:0/968744585 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a181d5da80 0x55a181d5b510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.491+0000 7f68efa0e740 1 --2- 172.21.15.86:0/968744585 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a181d54ea0 0x55a181d5afd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.491+0000 7f68cdffb640 0 client.19044 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:00.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.491+0000 7f68efa0e740 1 --2- 172.21.15.86:0/968744585 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a181d54580 0x55a181d5aa90 unknown :-1 s=CLOSED pgs=1612 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.491+0000 7f68efa0e740 1 -- 172.21.15.86:0/968744585 >> 172.21.15.86:0/968744585 conn(0x55a181d4ecb0 msgr2=0x55a181d50a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:00.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.491+0000 7f68efa0e740 1 -- 172.21.15.86:0/968744585 shutdown_connections 2024-04-21T08:47:00.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.491+0000 7f68efa0e740 1 -- 172.21.15.86:0/968744585 wait complete. 2024-04-21T08:47:00.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.491+0000 7f68efa0e740 20 client.19044 trim_cache size 0 max 16384 2024-04-21T08:47:00.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.508+0000 7f68efa0e740 1 Processor -- start 2024-04-21T08:47:00.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.508+0000 7f68efa0e740 1 -- start start 2024-04-21T08:47:00.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.508+0000 7f68efa0e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a181d54580 0x55a181d5d1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.508+0000 7f68efa0e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a181d54ea0 0x55a181d59a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.508+0000 7f68efa0e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a181d5da80 0x55a181d69760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.508+0000 7f68efa0e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f68e81af3c0 con 0x55a181d54580 2024-04-21T08:47:00.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.508+0000 7f68efa0e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f68e81b0f00 con 0x55a181d5da80 2024-04-21T08:47:00.516 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.508+0000 7f68efa0e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f68e81afa90 con 0x55a181d54ea0 2024-04-21T08:47:00.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.508+0000 7f68e659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a181d54580 0x55a181d5d1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.508+0000 7f68e659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a181d54580 0x55a181d5d1c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37158/0 (socket says 172.21.15.86:37158) 2024-04-21T08:47:00.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.508+0000 7f68e659c640 1 -- 172.21.15.86:0/2828106661 learned_addr learned my addr 172.21.15.86:0/2828106661 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:00.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.509+0000 7f68e659c640 1 -- 172.21.15.86:0/2828106661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a181d54ea0 msgr2=0x55a181d59a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:00.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.509+0000 7f68e659c640 1 --2- 172.21.15.86:0/2828106661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a181d54ea0 0x55a181d59a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.509+0000 7f68e659c640 1 -- 172.21.15.86:0/2828106661 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a181d5da80 msgr2=0x55a181d69760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:00.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.509+0000 7f68e659c640 1 --2- 172.21.15.86:0/2828106661 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a181d5da80 0x55a181d69760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.509+0000 7f68e659c640 1 -- 172.21.15.86:0/2828106661 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a181d5fe70 con 0x55a181d54580 2024-04-21T08:47:00.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.509+0000 7f68e659c640 1 --2- 172.21.15.86:0/2828106661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a181d54580 0x55a181d5d1c0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f68d8010010 tx=0x7f68d800b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8a627d6266a3ff77 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:00.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.509+0000 7f68cffff640 1 -- 172.21.15.86:0/2828106661 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d8039020 con 0x55a181d54580 2024-04-21T08:47:00.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.509+0000 7f68cffff640 1 -- 172.21.15.86:0/2828106661 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68d80054f0 con 0x55a181d54580 2024-04-21T08:47:00.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.509+0000 7f68cffff640 1 -- 172.21.15.86:0/2828106661 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d8004990 con 0x55a181d54580 2024-04-21T08:47:00.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.509+0000 7f68efa0e740 1 -- 172.21.15.86:0/2828106661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a181d54580 msgr2=0x55a181d5d1c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:00.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.509+0000 7f68efa0e740 1 --2- 172.21.15.86:0/2828106661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a181d54580 0x55a181d5d1c0 secure :-1 s=READY pgs=1613 cs=0 l=1 rev1=1 crypto rx=0x7f68d8010010 tx=0x7f68d800b9b0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.509+0000 7f68efa0e740 1 -- 172.21.15.86:0/2828106661 shutdown_connections 2024-04-21T08:47:00.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.509+0000 7f68efa0e740 1 --2- 172.21.15.86:0/2828106661 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a181d5da80 0x55a181d69760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.510+0000 7f68efa0e740 1 --2- 172.21.15.86:0/2828106661 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a181d54ea0 0x55a181d59a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.510+0000 7f68efa0e740 1 --2- 172.21.15.86:0/2828106661 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a181d54580 0x55a181d5d1c0 unknown :-1 s=CLOSED pgs=1613 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.510+0000 7f68efa0e740 1 -- 172.21.15.86:0/2828106661 >> 172.21.15.86:0/2828106661 conn(0x55a181d4ecb0 msgr2=0x55a181d4dd70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:00.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.510+0000 7f68efa0e740 1 -- 172.21.15.86:0/2828106661 shutdown_connections 2024-04-21T08:47:00.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.510+0000 7f68efa0e740 1 -- 172.21.15.86:0/2828106661 wait complete. 2024-04-21T08:47:00.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.510+0000 7f68efa0e740 1 -- start start 2024-04-21T08:47:00.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.510+0000 7f68efa0e740 1 Processor -- start 2024-04-21T08:47:00.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.510+0000 7f68efa0e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a181d54580 0x55a181d57a50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.510+0000 7f68efa0e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a181d54ea0 0x55a181d57f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.510+0000 7f68efa0e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a181d5da80 0x55a181d584f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.510+0000 7f68efa0e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f68e81b0830 con 0x55a181d54ea0 2024-04-21T08:47:00.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.510+0000 7f68efa0e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f68e81b0160 con 0x55a181d54580 2024-04-21T08:47:00.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.510+0000 7f68efa0e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f68e81ad3c0 con 0x55a181d5da80 2024-04-21T08:47:00.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.510+0000 7f68efa0e740 10 client.0 _ll_register_callbacks cb 0x7f68d8015070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:00.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.510+0000 7f68dffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a181d54ea0 0x55a181d57f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.510+0000 7f68dffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a181d54ea0 0x55a181d57f90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37168/0 (socket says 172.21.15.86:37168) 2024-04-21T08:47:00.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.510+0000 7f68dffff640 1 -- 172.21.15.86:0/2643061442 learned_addr learned my addr 172.21.15.86:0/2643061442 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:00.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.510+0000 7f68e6d9d640 1 --2- 172.21.15.86:0/2643061442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a181d5da80 0x55a181d584f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.510+0000 7f68e659c640 1 --2- 172.21.15.86:0/2643061442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a181d54580 0x55a181d57a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.511+0000 7f68dffff640 1 -- 172.21.15.86:0/2643061442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a181d5da80 msgr2=0x55a181d584f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:00.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.511+0000 7f68dffff640 1 --2- 172.21.15.86:0/2643061442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a181d5da80 0x55a181d584f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.511+0000 7f68dffff640 1 -- 172.21.15.86:0/2643061442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a181d54580 msgr2=0x55a181d57a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:00.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.511+0000 7f68dffff640 1 --2- 172.21.15.86:0/2643061442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a181d54580 0x55a181d57a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.511+0000 7f68dffff640 1 -- 172.21.15.86:0/2643061442 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a181dd3320 con 0x55a181d54ea0 2024-04-21T08:47:00.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.511+0000 7f68e659c640 1 --2- 172.21.15.86:0/2643061442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a181d54580 0x55a181d57a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:00.519 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.511+0000 7f68dffff640 1 --2- 172.21.15.86:0/2643061442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a181d54ea0 0x55a181d57f90 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f68d00bea30 tx=0x7f68d0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=96d66c05d344b200 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:00.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.511+0000 7f68cdffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:47:00.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.511+0000 7f68cdffb640 1 -- 172.21.15.86:0/2643061442 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f68d00c2e50 con 0x55a181d54ea0 2024-04-21T08:47:00.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.511+0000 7f68cdffb640 1 -- 172.21.15.86:0/2643061442 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(98..98 src has 1..98) ==== 9496+0+0 (secure 0 0 0) 0x7f68d00bcda0 con 0x55a181d54ea0 2024-04-21T08:47:00.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.511+0000 7f68efa0e740 1 -- 172.21.15.86:0/2643061442 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f68d80054f0 con 0x55a181d54ea0 2024-04-21T08:47:00.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.511+0000 7f68cdffb640 1 -- 172.21.15.86:0/2643061442 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f68d00c7710 con 0x55a181d54ea0 2024-04-21T08:47:00.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.511+0000 7f68efa0e740 10 client.19050 fetch_fsmap learned FSMap version 29 2024-04-21T08:47:00.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.511+0000 7f68efa0e740 1 -- 172.21.15.86:0/2643061442 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a181dd4630 con 0x55a181d54ea0 2024-04-21T08:47:00.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.511+0000 7f68cdffb640 1 -- 172.21.15.86:0/2643061442 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f68d80054f0 con 0x55a181d54ea0 2024-04-21T08:47:00.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.511+0000 7f68cdffb640 1 -- 172.21.15.86:0/2643061442 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f68d00c7280 con 0x55a181d54ea0 2024-04-21T08:47:00.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.511+0000 7f68efa0e740 10 client.19050 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:47:00.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.511+0000 7f68efa0e740 10 client.19050 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:00.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.511+0000 7f68efa0e740 20 client.19050 populate_metadata read hostname 'smithi086' 2024-04-21T08:47:00.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.511+0000 7f68efa0e740 1 --2- 172.21.15.86:0/2643061442 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a181dd4d00 0x55a181dd70f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:00.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.511+0000 7f68efa0e740 4 client.19050 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:47:00.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.511+0000 7f68efa0e740 1 -- 172.21.15.86:0/2643061442 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a181cc0a70 con 0x55a181dd4d00 2024-04-21T08:47:00.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.512+0000 7f68e659c640 1 --2- 172.21.15.86:0/2643061442 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a181dd4d00 0x55a181dd70f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:00.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.512+0000 7f68cdffb640 10 client.19050 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:00.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.512+0000 7f68e659c640 1 --2- 172.21.15.86:0/2643061442 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a181dd4d00 0x55a181dd70f0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8637452d14625b39 server_cookie=b3cd84e6246ce21b in_seq=0 out_seq=0 2024-04-21T08:47:00.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.512+0000 7f68cdffb640 1 -- 172.21.15.86:0/2643061442 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a181cc0a70 con 0x55a181dd4d00 2024-04-21T08:47:00.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.512+0000 7f68cdffb640 10 client.19050 handle_command_reply: tid=0 2024-04-21T08:47:00.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.512+0000 7f68efa0e740 1 client.19050 shutdown 2024-04-21T08:47:00.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.512+0000 7f68efa0e740 1 -- 172.21.15.86:0/2643061442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a181d54ea0 msgr2=0x55a181d57f90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:00.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.512+0000 7f68efa0e740 1 --2- 172.21.15.86:0/2643061442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a181d54ea0 0x55a181d57f90 secure :-1 s=READY pgs=1614 cs=0 l=1 rev1=1 crypto rx=0x7f68d00bea30 tx=0x7f68d0000bc0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.512+0000 7f68efa0e740 1 -- 172.21.15.86:0/2643061442 shutdown_connections 2024-04-21T08:47:00.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.512+0000 7f68efa0e740 1 --2- 172.21.15.86:0/2643061442 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a181dd4d00 0x55a181dd70f0 crc :-1 s=READY pgs=252 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.512+0000 7f68efa0e740 1 --2- 172.21.15.86:0/2643061442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a181d5da80 0x55a181d584f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.512+0000 7f68efa0e740 1 --2- 172.21.15.86:0/2643061442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a181d54ea0 0x55a181d57f90 unknown :-1 s=CLOSED pgs=1614 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.512+0000 7f68cdffb640 0 client.19050 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:00.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.512+0000 7f68efa0e740 1 --2- 172.21.15.86:0/2643061442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a181d54580 0x55a181d57a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.512+0000 7f68efa0e740 1 -- 172.21.15.86:0/2643061442 >> 172.21.15.86:0/2643061442 conn(0x55a181d4ecb0 msgr2=0x55a181d5aa50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:00.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.512+0000 7f68efa0e740 1 -- 172.21.15.86:0/2643061442 shutdown_connections 2024-04-21T08:47:00.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.512+0000 7f68efa0e740 1 -- 172.21.15.86:0/2643061442 wait complete. 2024-04-21T08:47:00.522 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.512+0000 7f68efa0e740 20 client.19050 trim_cache size 0 max 16384 2024-04-21T08:47:00.522 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:47:00.522 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:47:00.522 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-21T08:47:00.522 INFO:teuthology.orchestra.run.smithi086.stdout: "45f7e4fa-a859-43d7-897e-86661f787232": { 2024-04-21T08:47:00.522 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-21T08:47:00.523 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "45f7e4fa-a859-43d7-897e-86661f787232", 2024-04-21T08:47:00.523 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-21T08:47:00.523 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:47:00.523 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:47:00.523 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:47:00.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.513+0000 7f68ece26640 1 -- 172.21.15.86:0/4144641717 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f68d40a3d20 msgr2=0x7f68d40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:00.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.513+0000 7f68ece26640 1 --2- 172.21.15.86:0/4144641717 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f68d40a3d20 0x7f68d40a61e0 secure :-1 s=READY pgs=262 cs=0 l=1 rev1=1 crypto rx=0x7f68d80058c0 tx=0x7f68d8005850 comp rx=0 tx=0).stop 2024-04-21T08:47:00.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.513+0000 7f68ece26640 1 -- 172.21.15.86:0/4144641717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68e8106160 msgr2=0x7f68e8199ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:00.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.513+0000 7f68ece26640 1 --2- 172.21.15.86:0/4144641717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68e8106160 0x7f68e8199ac0 secure :-1 s=READY pgs=1610 cs=0 l=1 rev1=1 crypto rx=0x7f68d0036040 tx=0x7f68d002c7e0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.513+0000 7f68ece26640 1 -- 172.21.15.86:0/4144641717 shutdown_connections 2024-04-21T08:47:00.524 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.513+0000 7f68ece26640 1 --2- 172.21.15.86:0/4144641717 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f68d40a3d20 0x7f68d40a61e0 unknown :-1 s=CLOSED pgs=262 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.513+0000 7f68ece26640 1 --2- 172.21.15.86:0/4144641717 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f68e819a050 0x7f68e81a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.513+0000 7f68ece26640 1 --2- 172.21.15.86:0/4144641717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f68e8106160 0x7f68e8199ac0 unknown :-1 s=CLOSED pgs=1610 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.513+0000 7f68ece26640 1 --2- 172.21.15.86:0/4144641717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f68e8101090 0x7f68e8199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:00.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.513+0000 7f68ece26640 1 -- 172.21.15.86:0/4144641717 >> 172.21.15.86:0/4144641717 conn(0x7f68e80771d0 msgr2=0x7f68e80fee60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:00.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.513+0000 7f68ece26640 1 -- 172.21.15.86:0/4144641717 shutdown_connections 2024-04-21T08:47:00.525 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:00.513+0000 7f68ece26640 1 -- 172.21.15.86:0/4144641717 wait complete. 2024-04-21T08:47:00.525 INFO:tasks.cephfs.filesystem:scrub status for tag:45f7e4fa-a859-43d7-897e-86661f787232 - {'path': '/', 'tag': '45f7e4fa-a859-43d7-897e-86661f787232', 'options': 'recursive,force'} 2024-04-21T08:47:01.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:00 smithi086 ceph-mon[27939]: scrub queued for path: / 2024-04-21T08:47:01.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:00 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:01.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:00 smithi086 ceph-mon[27939]: scrub summary: active paths [/] 2024-04-21T08:47:01.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:00 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:01.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:00 smithi088 ceph-mon[31259]: scrub queued for path: / 2024-04-21T08:47:01.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:00 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:01.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:00 smithi088 ceph-mon[31259]: scrub summary: active paths [/] 2024-04-21T08:47:01.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:00 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:01.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:00 smithi179 ceph-mon[31829]: scrub queued for path: / 2024-04-21T08:47:01.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:00 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:01.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:00 smithi179 ceph-mon[31829]: scrub summary: active paths [/] 2024-04-21T08:47:01.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:00 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:02.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:01 smithi086 ceph-mon[27939]: pgmap v254: 129 pgs: 129 active+clean; 2.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-21T08:47:02.147 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:01 smithi179 ceph-mon[31829]: pgmap v254: 129 pgs: 129 active+clean; 2.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-21T08:47:02.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:01 smithi088 ceph-mon[31259]: pgmap v254: 129 pgs: 129 active+clean; 2.4 MiB data, 410 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-21T08:47:04.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:03 smithi086 ceph-mon[27939]: pgmap v255: 129 pgs: 129 active+clean; 2.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 105 KiB/s wr, 40 op/s 2024-04-21T08:47:04.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:03 smithi088 ceph-mon[31259]: pgmap v255: 129 pgs: 129 active+clean; 2.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 105 KiB/s wr, 40 op/s 2024-04-21T08:47:04.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:03 smithi179 ceph-mon[31829]: pgmap v255: 129 pgs: 129 active+clean; 2.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 74 KiB/s rd, 105 KiB/s wr, 40 op/s 2024-04-21T08:47:05.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:04 smithi088 ceph-mon[31259]: scrub summary: idle 2024-04-21T08:47:05.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:04 smithi179 ceph-mon[31829]: scrub summary: idle 2024-04-21T08:47:05.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:04 smithi086 ceph-mon[27939]: scrub summary: idle 2024-04-21T08:47:06.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:05 smithi088 ceph-mon[31259]: pgmap v256: 129 pgs: 129 active+clean; 2.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 89 KiB/s wr, 34 op/s 2024-04-21T08:47:06.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:05 smithi088 ceph-mon[31259]: osdmap e99: 12 total, 12 up, 12 in 2024-04-21T08:47:06.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:05 smithi179 ceph-mon[31829]: pgmap v256: 129 pgs: 129 active+clean; 2.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 89 KiB/s wr, 34 op/s 2024-04-21T08:47:06.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:05 smithi179 ceph-mon[31829]: osdmap e99: 12 total, 12 up, 12 in 2024-04-21T08:47:06.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:05 smithi086 ceph-mon[27939]: pgmap v256: 129 pgs: 129 active+clean; 2.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 89 KiB/s wr, 34 op/s 2024-04-21T08:47:06.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:05 smithi086 ceph-mon[27939]: osdmap e99: 12 total, 12 up, 12 in 2024-04-21T08:47:08.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:07 smithi088 ceph-mon[31259]: pgmap v258: 129 pgs: 129 active+clean; 2.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 107 KiB/s wr, 40 op/s 2024-04-21T08:47:08.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:07 smithi179 ceph-mon[31829]: pgmap v258: 129 pgs: 129 active+clean; 2.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 107 KiB/s wr, 40 op/s 2024-04-21T08:47:08.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:07 smithi086 ceph-mon[27939]: pgmap v258: 129 pgs: 129 active+clean; 2.7 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 107 KiB/s wr, 40 op/s 2024-04-21T08:47:10.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:09 smithi088 ceph-mon[31259]: pgmap v259: 129 pgs: 129 active+clean; 2.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 107 KiB/s wr, 40 op/s 2024-04-21T08:47:10.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:09 smithi179 ceph-mon[31829]: pgmap v259: 129 pgs: 129 active+clean; 2.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 107 KiB/s wr, 40 op/s 2024-04-21T08:47:10.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:09 smithi086 ceph-mon[27939]: pgmap v259: 129 pgs: 129 active+clean; 2.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 107 KiB/s wr, 40 op/s 2024-04-21T08:47:11.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:11 smithi086 ceph-mon[27939]: osdmap e100: 12 total, 12 up, 12 in 2024-04-21T08:47:11.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:11 smithi088 ceph-mon[31259]: osdmap e100: 12 total, 12 up, 12 in 2024-04-21T08:47:11.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:11 smithi179 ceph-mon[31829]: osdmap e100: 12 total, 12 up, 12 in 2024-04-21T08:47:12.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:12 smithi086 ceph-mon[27939]: pgmap v261: 129 pgs: 129 active+clean; 2.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 9 op/s 2024-04-21T08:47:12.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:12 smithi088 ceph-mon[31259]: pgmap v261: 129 pgs: 129 active+clean; 2.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 9 op/s 2024-04-21T08:47:12.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:12 smithi179 ceph-mon[31829]: pgmap v261: 129 pgs: 129 active+clean; 2.9 MiB data, 412 MiB used, 1.0 TiB / 1.0 TiB avail; 29 KiB/s rd, 27 KiB/s wr, 9 op/s 2024-04-21T08:47:13.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.241+0000 7f2bf7367640 1 -- 172.21.15.86:0/3621533509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bf01061b0 msgr2=0x7f2bf010dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:13.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.242+0000 7f2bf7367640 1 --2- 172.21.15.86:0/3621533509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bf01061b0 0x7f2bf010dd80 secure :-1 s=READY pgs=1615 cs=0 l=1 rev1=1 crypto rx=0x7f2be8009ca0 tx=0x7f2be802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.242+0000 7f2bf7367640 1 -- 172.21.15.86:0/3621533509 shutdown_connections 2024-04-21T08:47:13.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.242+0000 7f2bf7367640 1 --2- 172.21.15.86:0/3621533509 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bf01061b0 0x7f2bf010dd80 unknown :-1 s=CLOSED pgs=1615 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.242+0000 7f2bf7367640 1 --2- 172.21.15.86:0/3621533509 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bf0101a90 0x7f2bf0105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.242+0000 7f2bf7367640 1 --2- 172.21.15.86:0/3621533509 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bf01010e0 0x7f2bf01014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.242+0000 7f2bf7367640 1 -- 172.21.15.86:0/3621533509 >> 172.21.15.86:0/3621533509 conn(0x7f2bf00771d0 msgr2=0x7f2bf00775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.242+0000 7f2bf7367640 1 -- 172.21.15.86:0/3621533509 shutdown_connections 2024-04-21T08:47:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.242+0000 7f2bf7367640 1 -- 172.21.15.86:0/3621533509 wait complete. 2024-04-21T08:47:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.242+0000 7f2bf7367640 1 Processor -- start 2024-04-21T08:47:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.242+0000 7f2bf7367640 1 -- start start 2024-04-21T08:47:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.242+0000 7f2bf7367640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bf01010e0 0x7f2bf0199460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.242+0000 7f2bf7367640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bf01061b0 0x7f2bf01999a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.242+0000 7f2bf7367640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bf0199f80 0x7f2bf01a44e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.242+0000 7f2bf7367640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf0111120 con 0x7f2bf01061b0 2024-04-21T08:47:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.242+0000 7f2bf7367640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf0110fa0 con 0x7f2bf0199f80 2024-04-21T08:47:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.242+0000 7f2bf7367640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf01112a0 con 0x7f2bf01010e0 2024-04-21T08:47:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.243+0000 7f2bf4902640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bf01061b0 0x7f2bf01999a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.243+0000 7f2bf4902640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bf01061b0 0x7f2bf01999a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34680/0 (socket says 172.21.15.86:34680) 2024-04-21T08:47:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.243+0000 7f2bf4902640 1 -- 172.21.15.86:0/3552926512 learned_addr learned my addr 172.21.15.86:0/3552926512 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.243+0000 7f2bf5103640 1 --2- 172.21.15.86:0/3552926512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bf01010e0 0x7f2bf0199460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:13.244 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.243+0000 7f2bf4902640 1 -- 172.21.15.86:0/3552926512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bf01010e0 msgr2=0x7f2bf0199460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:13.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.243+0000 7f2bf5904640 1 --2- 172.21.15.86:0/3552926512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bf0199f80 0x7f2bf01a44e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:13.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.243+0000 7f2bf4902640 1 --2- 172.21.15.86:0/3552926512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bf01010e0 0x7f2bf0199460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.243+0000 7f2bf4902640 1 -- 172.21.15.86:0/3552926512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bf0199f80 msgr2=0x7f2bf01a44e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:13.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.243+0000 7f2bf4902640 1 --2- 172.21.15.86:0/3552926512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bf0199f80 0x7f2bf01a44e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.243+0000 7f2bf4902640 1 -- 172.21.15.86:0/3552926512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bf01a4d00 con 0x7f2bf01061b0 2024-04-21T08:47:13.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.243+0000 7f2bf4902640 1 --2- 172.21.15.86:0/3552926512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bf01061b0 0x7f2bf01999a0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f2be000e9f0 tx=0x7f2be000eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:13.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.243+0000 7f2be67fc640 1 -- 172.21.15.86:0/3552926512 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be000ccc0 con 0x7f2bf01061b0 2024-04-21T08:47:13.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.243+0000 7f2be67fc640 1 -- 172.21.15.86:0/3552926512 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2be00054d0 con 0x7f2bf01061b0 2024-04-21T08:47:13.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.243+0000 7f2bf7367640 1 -- 172.21.15.86:0/3552926512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bf01a4ff0 con 0x7f2bf01061b0 2024-04-21T08:47:13.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.243+0000 7f2be67fc640 1 -- 172.21.15.86:0/3552926512 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be0010430 con 0x7f2bf01061b0 2024-04-21T08:47:13.245 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.243+0000 7f2bf7367640 1 -- 172.21.15.86:0/3552926512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bf01a5460 con 0x7f2bf01061b0 2024-04-21T08:47:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.246+0000 7f2be67fc640 1 -- 172.21.15.86:0/3552926512 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2be00046c0 con 0x7f2bf01061b0 2024-04-21T08:47:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.246+0000 7f2bfa039740 1 Processor -- start 2024-04-21T08:47:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.246+0000 7f2bfa039740 1 -- start start 2024-04-21T08:47:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.247+0000 7f2bfa039740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558511f538d0 0x558511f52620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.247+0000 7f2bfa039740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558511f52b60 0x558511f52fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.247+0000 7f2be67fc640 1 --2- 172.21.15.86:0/3552926512 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2bdc0a3dc0 0x7f2bdc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.247+0000 7f2bfa039740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558511f5c0e0 0x558511f5e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.247+0000 7f2bfa039740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558511eb78d0 con 0x558511f5c0e0 2024-04-21T08:47:13.251 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.247+0000 7f2bfa039740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x558511e54e80 con 0x558511f538d0 2024-04-21T08:47:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.247+0000 7f2bfa039740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558511eb3890 con 0x558511f52b60 2024-04-21T08:47:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.247+0000 7f2bf4902640 1 --2- 172.21.15.86:0/3552926512 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2bdc0a3dc0 0x7f2bdc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.247+0000 7f2bf5904640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558511f5c0e0 0x558511f5e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.247+0000 7f2bf5904640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558511f5c0e0 0x558511f5e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34684/0 (socket says 172.21.15.86:34684) 2024-04-21T08:47:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.247+0000 7f2bf5904640 1 -- 172.21.15.86:0/3625017298 learned_addr learned my addr 172.21.15.86:0/3625017298 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.247+0000 7f2bf5103640 1 --2- 172.21.15.86:0/3625017298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558511f52b60 0x558511f52fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.247+0000 7f2bf5904640 1 --2- 172.21.15.86:0/3625017298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558511f538d0 0x558511f52620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.247+0000 7f2be67fc640 1 -- 172.21.15.86:0/3552926512 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f2be00d0fd0 con 0x7f2bf01061b0 2024-04-21T08:47:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.247+0000 7f2bf5904640 1 -- 172.21.15.86:0/3625017298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558511f52b60 msgr2=0x558511f52fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.247+0000 7f2bf5904640 1 --2- 172.21.15.86:0/3625017298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558511f52b60 0x558511f52fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.247+0000 7f2bf5904640 1 -- 172.21.15.86:0/3625017298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558511f538d0 msgr2=0x558511f52620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.247+0000 7f2bf5904640 1 --2- 172.21.15.86:0/3625017298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558511f538d0 0x558511f52620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.252 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.247+0000 7f2bf5904640 1 -- 172.21.15.86:0/3625017298 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558511e76320 con 0x558511f5c0e0 2024-04-21T08:47:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.247+0000 7f2bf5103640 1 --2- 172.21.15.86:0/3625017298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558511f52b60 0x558511f52fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:47:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.247+0000 7f2bf4902640 1 --2- 172.21.15.86:0/3552926512 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2bdc0a3dc0 0x7f2bdc0a6280 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2be00a5280 tx=0x7f2be00bd040 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.247+0000 7f2bf5904640 1 --2- 172.21.15.86:0/3625017298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558511f5c0e0 0x558511f5e4d0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f2be8004af0 tx=0x7f2be8004a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2c04ffd9485eb02d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.248+0000 7f2bdb7fe640 1 -- 172.21.15.86:0/3625017298 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be8048070 con 0x558511f5c0e0 2024-04-21T08:47:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.248+0000 7f2bdb7fe640 1 -- 172.21.15.86:0/3625017298 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2be803a040 con 0x558511f5c0e0 2024-04-21T08:47:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.248+0000 7f2bdb7fe640 1 -- 172.21.15.86:0/3625017298 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be8039030 con 0x558511f5c0e0 2024-04-21T08:47:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.248+0000 7f2bfa039740 1 -- 172.21.15.86:0/3625017298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558511f5c0e0 msgr2=0x558511f5e4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.248+0000 7f2bfa039740 1 --2- 172.21.15.86:0/3625017298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558511f5c0e0 0x558511f5e4d0 secure :-1 s=READY pgs=1617 cs=0 l=1 rev1=1 crypto rx=0x7f2be8004af0 tx=0x7f2be8004a60 comp rx=0 tx=0).stop 2024-04-21T08:47:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.248+0000 7f2bfa039740 1 -- 172.21.15.86:0/3625017298 shutdown_connections 2024-04-21T08:47:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.248+0000 7f2bfa039740 1 --2- 172.21.15.86:0/3625017298 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558511f5c0e0 0x558511f5e4d0 unknown :-1 s=CLOSED pgs=1617 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.248+0000 7f2bfa039740 1 --2- 172.21.15.86:0/3625017298 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558511f52b60 0x558511f52fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.248+0000 7f2bfa039740 1 --2- 172.21.15.86:0/3625017298 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558511f538d0 0x558511f52620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.248+0000 7f2bfa039740 1 -- 172.21.15.86:0/3625017298 >> 172.21.15.86:0/3625017298 conn(0x558511f19c90 msgr2=0x558511f4d850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.248+0000 7f2bfa039740 1 -- 172.21.15.86:0/3625017298 shutdown_connections 2024-04-21T08:47:13.253 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.248+0000 7f2bfa039740 1 -- 172.21.15.86:0/3625017298 wait complete. 2024-04-21T08:47:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.249+0000 7f2bfa039740 1 -- start start 2024-04-21T08:47:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.249+0000 7f2bfa039740 1 Processor -- start 2024-04-21T08:47:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.249+0000 7f2bfa039740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558511f52b60 0x558511f5bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.249+0000 7f2bfa039740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558511f538d0 0x558511f58e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.249+0000 7f2bfa039740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558511f5c0e0 0x558511f594a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.249+0000 7f2bfa039740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558511f5edb0 con 0x558511f5c0e0 2024-04-21T08:47:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.249+0000 7f2bfa039740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x558511e3feb0 con 0x558511f538d0 2024-04-21T08:47:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.249+0000 7f2bfa039740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558511f5ef30 con 0x558511f52b60 2024-04-21T08:47:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.249+0000 7f2bfa039740 10 client.0 _ll_register_callbacks cb 0x558511e6cb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.250+0000 7f2bf5904640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558511f5c0e0 0x558511f594a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.250+0000 7f2bf5904640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558511f5c0e0 0x558511f594a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34694/0 (socket says 172.21.15.86:34694) 2024-04-21T08:47:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.250+0000 7f2bf5904640 1 -- 172.21.15.86:0/2423696270 learned_addr learned my addr 172.21.15.86:0/2423696270 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.250+0000 7f2bf5904640 1 --2- 172.21.15.86:0/2423696270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558511f52b60 0x558511f5bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:13.254 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.250+0000 7f2bf5103640 1 --2- 172.21.15.86:0/2423696270 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558511f538d0 0x558511f58e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.250+0000 7f2bf5904640 1 -- 172.21.15.86:0/2423696270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558511f52b60 msgr2=0x558511f5bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.250+0000 7f2bf5904640 1 --2- 172.21.15.86:0/2423696270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558511f52b60 0x558511f5bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.250+0000 7f2bf5904640 1 -- 172.21.15.86:0/2423696270 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558511f538d0 msgr2=0x558511f58e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.250+0000 7f2bf5904640 1 --2- 172.21.15.86:0/2423696270 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558511f538d0 0x558511f58e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.250+0000 7f2bf5904640 1 -- 172.21.15.86:0/2423696270 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558511bca210 con 0x558511f5c0e0 2024-04-21T08:47:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.250+0000 7f2bf5904640 1 --2- 172.21.15.86:0/2423696270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558511f5c0e0 0x558511f594a0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f2be8003cb0 tx=0x7f2be8009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8686f9bc74d1fb86 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.250+0000 7f2bd8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:47:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.250+0000 7f2bd8ff9640 1 -- 172.21.15.86:0/2423696270 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be8048050 con 0x558511f5c0e0 2024-04-21T08:47:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.250+0000 7f2bd8ff9640 1 -- 172.21.15.86:0/2423696270 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f2be8003f60 con 0x558511f5c0e0 2024-04-21T08:47:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.250+0000 7f2bfa039740 1 -- 172.21.15.86:0/2423696270 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2be803a040 con 0x558511f5c0e0 2024-04-21T08:47:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.250+0000 7f2bd8ff9640 1 -- 172.21.15.86:0/2423696270 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2be8039040 con 0x558511f5c0e0 2024-04-21T08:47:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.250+0000 7f2bfa039740 10 client.19062 fetch_fsmap learned FSMap version 29 2024-04-21T08:47:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.250+0000 7f2bfa039740 1 -- 172.21.15.86:0/2423696270 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558511c96620 con 0x558511f5c0e0 2024-04-21T08:47:13.255 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.251+0000 7f2bd8ff9640 1 -- 172.21.15.86:0/2423696270 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2be803a040 con 0x558511f5c0e0 2024-04-21T08:47:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.251+0000 7f2bd8ff9640 1 -- 172.21.15.86:0/2423696270 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2be8034670 con 0x558511f5c0e0 2024-04-21T08:47:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.251+0000 7f2bfa039740 10 client.19062 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:47:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.251+0000 7f2bfa039740 10 client.19062 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.251+0000 7f2bfa039740 20 client.19062 populate_metadata read hostname 'smithi086' 2024-04-21T08:47:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.251+0000 7f2bfa039740 1 --2- 172.21.15.86:0/2423696270 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x558511f6bfd0 0x558511f6e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.251+0000 7f2bfa039740 4 client.19062 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.251+0000 7f2bfa039740 1 -- 172.21.15.86:0/2423696270 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558511ecb880 con 0x558511f6bfd0 2024-04-21T08:47:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.251+0000 7f2bf5103640 1 --2- 172.21.15.86:0/2423696270 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x558511f6bfd0 0x558511f6e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.251+0000 7f2bf5103640 1 --2- 172.21.15.86:0/2423696270 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x558511f6bfd0 0x558511f6e3c0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c3ea00fe60299cde server_cookie=c9a508a53d244e05 in_seq=0 out_seq=0 2024-04-21T08:47:13.256 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.251+0000 7f2bd8ff9640 10 client.19062 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.252+0000 7f2bd8ff9640 1 -- 172.21.15.86:0/2423696270 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558511ecb880 con 0x558511f6bfd0 2024-04-21T08:47:13.257 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.252+0000 7f2bd8ff9640 10 client.19062 handle_command_reply: tid=0 2024-04-21T08:47:13.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.252+0000 7f2bfa039740 1 client.19062 shutdown 2024-04-21T08:47:13.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.252+0000 7f2bfa039740 1 -- 172.21.15.86:0/2423696270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558511f5c0e0 msgr2=0x558511f594a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:13.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.252+0000 7f2bfa039740 1 --2- 172.21.15.86:0/2423696270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558511f5c0e0 0x558511f594a0 secure :-1 s=READY pgs=1618 cs=0 l=1 rev1=1 crypto rx=0x7f2be8003cb0 tx=0x7f2be8009c60 comp rx=0 tx=0).stop 2024-04-21T08:47:13.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.252+0000 7f2bfa039740 1 -- 172.21.15.86:0/2423696270 shutdown_connections 2024-04-21T08:47:13.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.252+0000 7f2bfa039740 1 --2- 172.21.15.86:0/2423696270 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x558511f6bfd0 0x558511f6e3c0 crc :-1 s=READY pgs=253 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.252+0000 7f2bfa039740 1 --2- 172.21.15.86:0/2423696270 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558511f5c0e0 0x558511f594a0 unknown :-1 s=CLOSED pgs=1618 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.252+0000 7f2bfa039740 1 --2- 172.21.15.86:0/2423696270 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558511f538d0 0x558511f58e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.252+0000 7f2bfa039740 1 --2- 172.21.15.86:0/2423696270 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558511f52b60 0x558511f5bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.252+0000 7f2bd8ff9640 0 client.19062 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:13.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.252+0000 7f2bfa039740 1 -- 172.21.15.86:0/2423696270 >> 172.21.15.86:0/2423696270 conn(0x558511f19c90 msgr2=0x558511f4f5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:13.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.252+0000 7f2bfa039740 1 -- 172.21.15.86:0/2423696270 shutdown_connections 2024-04-21T08:47:13.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.252+0000 7f2bfa039740 1 -- 172.21.15.86:0/2423696270 wait complete. 2024-04-21T08:47:13.258 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.253+0000 7f2bfa039740 20 client.19062 trim_cache size 0 max 16384 2024-04-21T08:47:13.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.267+0000 7f2bfa039740 1 Processor -- start 2024-04-21T08:47:13.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.267+0000 7f2bfa039740 1 -- start start 2024-04-21T08:47:13.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.268+0000 7f2bfa039740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558511f52b60 0x558511f5b5c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:13.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.268+0000 7f2bfa039740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558511f538d0 0x558511f5bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:13.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.268+0000 7f2bfa039740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558511f5c0e0 0x558511f5d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:13.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.268+0000 7f2bfa039740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf01af280 con 0x558511f5c0e0 2024-04-21T08:47:13.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.268+0000 7f2bfa039740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf01b0dc0 con 0x558511f52b60 2024-04-21T08:47:13.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.268+0000 7f2bfa039740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf01af950 con 0x558511f538d0 2024-04-21T08:47:13.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.268+0000 7f2bf5904640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558511f5c0e0 0x558511f5d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:13.272 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.268+0000 7f2bf5904640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558511f5c0e0 0x558511f5d4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34708/0 (socket says 172.21.15.86:34708) 2024-04-21T08:47:13.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.268+0000 7f2bf5904640 1 -- 172.21.15.86:0/3140948923 learned_addr learned my addr 172.21.15.86:0/3140948923 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:13.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.268+0000 7f2bf5904640 1 --2- 172.21.15.86:0/3140948923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558511f52b60 0x558511f5b5c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:13.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.268+0000 7f2bf5103640 1 --2- 172.21.15.86:0/3140948923 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558511f538d0 0x558511f5bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:13.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.268+0000 7f2bf5904640 1 -- 172.21.15.86:0/3140948923 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558511f538d0 msgr2=0x558511f5bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:13.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.268+0000 7f2bf5904640 1 --2- 172.21.15.86:0/3140948923 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558511f538d0 0x558511f5bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.268+0000 7f2bf5904640 1 -- 172.21.15.86:0/3140948923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558511f52b60 msgr2=0x558511f5b5c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:13.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.268+0000 7f2bf5904640 1 --2- 172.21.15.86:0/3140948923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558511f52b60 0x558511f5b5c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.268+0000 7f2bf5904640 1 -- 172.21.15.86:0/3140948923 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558511f48f30 con 0x558511f5c0e0 2024-04-21T08:47:13.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.268+0000 7f2bf5103640 1 --2- 172.21.15.86:0/3140948923 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558511f538d0 0x558511f5bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:13.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.268+0000 7f2bf5904640 1 --2- 172.21.15.86:0/3140948923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558511f5c0e0 0x558511f5d4d0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f2be8005fd0 tx=0x7f2be802e040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e85b52b5e5d5ab0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:13.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.268+0000 7f2bdaffd640 1 -- 172.21.15.86:0/3140948923 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be8048070 con 0x558511f5c0e0 2024-04-21T08:47:13.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.268+0000 7f2bdaffd640 1 -- 172.21.15.86:0/3140948923 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2be8060070 con 0x558511f5c0e0 2024-04-21T08:47:13.273 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.268+0000 7f2bdaffd640 1 -- 172.21.15.86:0/3140948923 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be8032e90 con 0x558511f5c0e0 2024-04-21T08:47:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.269+0000 7f2bfa039740 1 -- 172.21.15.86:0/3140948923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558511f5c0e0 msgr2=0x558511f5d4d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.269+0000 7f2bfa039740 1 --2- 172.21.15.86:0/3140948923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558511f5c0e0 0x558511f5d4d0 secure :-1 s=READY pgs=1619 cs=0 l=1 rev1=1 crypto rx=0x7f2be8005fd0 tx=0x7f2be802e040 comp rx=0 tx=0).stop 2024-04-21T08:47:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.269+0000 7f2bfa039740 1 -- 172.21.15.86:0/3140948923 shutdown_connections 2024-04-21T08:47:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.269+0000 7f2bfa039740 1 --2- 172.21.15.86:0/3140948923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558511f5c0e0 0x558511f5d4d0 unknown :-1 s=CLOSED pgs=1619 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.269+0000 7f2bfa039740 1 --2- 172.21.15.86:0/3140948923 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558511f538d0 0x558511f5bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.269+0000 7f2bfa039740 1 --2- 172.21.15.86:0/3140948923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558511f52b60 0x558511f5b5c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.269+0000 7f2bfa039740 1 -- 172.21.15.86:0/3140948923 >> 172.21.15.86:0/3140948923 conn(0x558511f19c90 msgr2=0x558511f54060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.269+0000 7f2bfa039740 1 -- 172.21.15.86:0/3140948923 shutdown_connections 2024-04-21T08:47:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.269+0000 7f2bfa039740 1 -- 172.21.15.86:0/3140948923 wait complete. 2024-04-21T08:47:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.269+0000 7f2bfa039740 1 -- start start 2024-04-21T08:47:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.269+0000 7f2bfa039740 1 Processor -- start 2024-04-21T08:47:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.270+0000 7f2bfa039740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558511f52b60 0x558511f6bfd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.270+0000 7f2bfa039740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558511f538d0 0x558511f6c510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:13.274 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.270+0000 7f2bfa039740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558511f5c0e0 0x558511fd0190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:13.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.270+0000 7f2bfa039740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf01b06f0 con 0x558511f52b60 2024-04-21T08:47:13.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.270+0000 7f2bfa039740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf01b0020 con 0x558511f5c0e0 2024-04-21T08:47:13.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.270+0000 7f2bfa039740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2bf01ad280 con 0x558511f538d0 2024-04-21T08:47:13.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.270+0000 7f2bfa039740 10 client.0 _ll_register_callbacks cb 0x7f2be8046070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:13.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.270+0000 7f2bf5904640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558511f52b60 0x558511f6bfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:13.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.270+0000 7f2bf5904640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558511f52b60 0x558511f6bfd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:34712/0 (socket says 172.21.15.86:34712) 2024-04-21T08:47:13.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.270+0000 7f2bf5904640 1 -- 172.21.15.86:0/75510728 learned_addr learned my addr 172.21.15.86:0/75510728 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:13.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.270+0000 7f2bf5103640 1 --2- 172.21.15.86:0/75510728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558511f538d0 0x558511f6c510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:13.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.270+0000 7f2bf5904640 1 -- 172.21.15.86:0/75510728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558511f538d0 msgr2=0x558511f6c510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:13.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.270+0000 7f2bf5904640 1 --2- 172.21.15.86:0/75510728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558511f538d0 0x558511f6c510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.270+0000 7f2bf5904640 1 -- 172.21.15.86:0/75510728 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558511f5c0e0 msgr2=0x558511fd0190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:13.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.270+0000 7f2bf5904640 1 --2- 172.21.15.86:0/75510728 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558511f5c0e0 0x558511fd0190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.270+0000 7f2bf5904640 1 -- 172.21.15.86:0/75510728 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558511f6cdb0 con 0x558511f52b60 2024-04-21T08:47:13.275 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.270+0000 7f2bf5904640 1 --2- 172.21.15.86:0/75510728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558511f52b60 0x558511f6bfd0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f2be8032460 tx=0x7f2be80327d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d35fb3331276e02 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:13.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.270+0000 7f2bd8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:47:13.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.270+0000 7f2bd8ff9640 1 -- 172.21.15.86:0/75510728 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2be8069070 con 0x558511f52b60 2024-04-21T08:47:13.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.271+0000 7f2bfa039740 1 -- 172.21.15.86:0/75510728 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2be8060070 con 0x558511f52b60 2024-04-21T08:47:13.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.271+0000 7f2bd8ff9640 1 -- 172.21.15.86:0/75510728 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f2be8004160 con 0x558511f52b60 2024-04-21T08:47:13.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.271+0000 7f2bd8ff9640 1 -- 172.21.15.86:0/75510728 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2be8048040 con 0x558511f52b60 2024-04-21T08:47:13.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.271+0000 7f2bfa039740 10 client.19068 fetch_fsmap learned FSMap version 29 2024-04-21T08:47:13.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.271+0000 7f2bfa039740 1 -- 172.21.15.86:0/75510728 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558511fd17b0 con 0x558511f52b60 2024-04-21T08:47:13.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.271+0000 7f2bd8ff9640 1 -- 172.21.15.86:0/75510728 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2be8060070 con 0x558511f52b60 2024-04-21T08:47:13.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.271+0000 7f2bd8ff9640 1 -- 172.21.15.86:0/75510728 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2be8064070 con 0x558511f52b60 2024-04-21T08:47:13.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.271+0000 7f2bfa039740 10 client.19068 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:47:13.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.271+0000 7f2bfa039740 10 client.19068 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:13.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.271+0000 7f2bfa039740 20 client.19068 populate_metadata read hostname 'smithi086' 2024-04-21T08:47:13.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.271+0000 7f2bfa039740 1 --2- 172.21.15.86:0/75510728 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x558511fd1f90 0x558511fd43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:13.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.271+0000 7f2bfa039740 4 client.19068 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:47:13.276 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.271+0000 7f2bfa039740 1 -- 172.21.15.86:0/75510728 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x558511e573d0 con 0x558511fd1f90 2024-04-21T08:47:13.277 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.271+0000 7f2bf5103640 1 --2- 172.21.15.86:0/75510728 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x558511fd1f90 0x558511fd43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:13.277 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.272+0000 7f2bf5103640 1 --2- 172.21.15.86:0/75510728 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x558511fd1f90 0x558511fd43b0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d8a5065f11f9e32c server_cookie=d899b0d1c1ec015f in_seq=0 out_seq=0 2024-04-21T08:47:13.277 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.272+0000 7f2bd8ff9640 10 client.19068 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.286+0000 7f2bd8ff9640 1 -- 172.21.15.86:0/75510728 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x558511e573d0 con 0x558511fd1f90 2024-04-21T08:47:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.286+0000 7f2bd8ff9640 10 client.19068 handle_command_reply: tid=0 2024-04-21T08:47:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.286+0000 7f2bfa039740 1 client.19068 shutdown 2024-04-21T08:47:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.286+0000 7f2bfa039740 1 -- 172.21.15.86:0/75510728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558511f52b60 msgr2=0x558511f6bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.286+0000 7f2bfa039740 1 --2- 172.21.15.86:0/75510728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558511f52b60 0x558511f6bfd0 secure :-1 s=READY pgs=1620 cs=0 l=1 rev1=1 crypto rx=0x7f2be8032460 tx=0x7f2be80327d0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.286 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.286+0000 7f2bfa039740 1 -- 172.21.15.86:0/75510728 shutdown_connections 2024-04-21T08:47:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.286+0000 7f2bfa039740 1 --2- 172.21.15.86:0/75510728 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x558511fd1f90 0x558511fd43b0 crc :-1 s=READY pgs=254 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.286+0000 7f2bfa039740 1 --2- 172.21.15.86:0/75510728 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558511f5c0e0 0x558511fd0190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.286+0000 7f2bfa039740 1 --2- 172.21.15.86:0/75510728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558511f538d0 0x558511f6c510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.286+0000 7f2bd8ff9640 0 client.19068 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.286+0000 7f2bfa039740 1 --2- 172.21.15.86:0/75510728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558511f52b60 0x558511f6bfd0 unknown :-1 s=CLOSED pgs=1620 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.286+0000 7f2bfa039740 1 -- 172.21.15.86:0/75510728 >> 172.21.15.86:0/75510728 conn(0x558511f19c90 msgr2=0x558511f57760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.287+0000 7f2bfa039740 1 -- 172.21.15.86:0/75510728 shutdown_connections 2024-04-21T08:47:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.287+0000 7f2bfa039740 1 -- 172.21.15.86:0/75510728 wait complete. 2024-04-21T08:47:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.287+0000 7f2bfa039740 20 client.19068 trim_cache size 0 max 16384 2024-04-21T08:47:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:47:13.287 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:47:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:47:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:47:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.287+0000 7f2bf7367640 1 -- 172.21.15.86:0/3552926512 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2bdc0a3dc0 msgr2=0x7f2bdc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:13.288 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.287+0000 7f2bf7367640 1 --2- 172.21.15.86:0/3552926512 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2bdc0a3dc0 0x7f2bdc0a6280 secure :-1 s=READY pgs=263 cs=0 l=1 rev1=1 crypto rx=0x7f2be00a5280 tx=0x7f2be00bd040 comp rx=0 tx=0).stop 2024-04-21T08:47:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.287+0000 7f2bf7367640 1 -- 172.21.15.86:0/3552926512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bf01061b0 msgr2=0x7f2bf01999a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.287+0000 7f2bf7367640 1 --2- 172.21.15.86:0/3552926512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bf01061b0 0x7f2bf01999a0 secure :-1 s=READY pgs=1616 cs=0 l=1 rev1=1 crypto rx=0x7f2be000e9f0 tx=0x7f2be000eeb0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.287+0000 7f2bf7367640 1 -- 172.21.15.86:0/3552926512 shutdown_connections 2024-04-21T08:47:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.287+0000 7f2bf7367640 1 --2- 172.21.15.86:0/3552926512 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2bdc0a3dc0 0x7f2bdc0a6280 unknown :-1 s=CLOSED pgs=263 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.287+0000 7f2bf7367640 1 --2- 172.21.15.86:0/3552926512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bf0199f80 0x7f2bf01a44e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.287+0000 7f2bf7367640 1 --2- 172.21.15.86:0/3552926512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bf01061b0 0x7f2bf01999a0 unknown :-1 s=CLOSED pgs=1616 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.287+0000 7f2bf7367640 1 --2- 172.21.15.86:0/3552926512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bf01010e0 0x7f2bf0199460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.287+0000 7f2bf7367640 1 -- 172.21.15.86:0/3552926512 >> 172.21.15.86:0/3552926512 conn(0x7f2bf00771d0 msgr2=0x7f2bf00febb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.287+0000 7f2bf7367640 1 -- 172.21.15.86:0/3552926512 shutdown_connections 2024-04-21T08:47:13.289 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:13.287+0000 7f2bf7367640 1 -- 172.21.15.86:0/3552926512 wait complete. 2024-04-21T08:47:13.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:13 smithi088 ceph-mon[31259]: pgmap v262: 129 pgs: 129 active+clean; 3.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 55 KiB/s wr, 19 op/s 2024-04-21T08:47:13.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:13 smithi179 ceph-mon[31829]: pgmap v262: 129 pgs: 129 active+clean; 3.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 55 KiB/s wr, 19 op/s 2024-04-21T08:47:14.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:13 smithi086 ceph-mon[27939]: pgmap v262: 129 pgs: 129 active+clean; 3.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 55 KiB/s wr, 19 op/s 2024-04-21T08:47:16.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:15 smithi086 ceph-mon[27939]: pgmap v263: 129 pgs: 129 active+clean; 3.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 45 KiB/s wr, 15 op/s 2024-04-21T08:47:16.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:15 smithi088 ceph-mon[31259]: pgmap v263: 129 pgs: 129 active+clean; 3.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 45 KiB/s wr, 15 op/s 2024-04-21T08:47:16.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:15 smithi179 ceph-mon[31829]: pgmap v263: 129 pgs: 129 active+clean; 3.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 45 KiB/s wr, 15 op/s 2024-04-21T08:47:18.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:17 smithi086 ceph-mon[27939]: pgmap v264: 129 pgs: 129 active+clean; 3.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 15 op/s 2024-04-21T08:47:18.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:17 smithi088 ceph-mon[31259]: pgmap v264: 129 pgs: 129 active+clean; 3.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 15 op/s 2024-04-21T08:47:18.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:17 smithi179 ceph-mon[31829]: pgmap v264: 129 pgs: 129 active+clean; 3.1 MiB data, 413 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 44 KiB/s wr, 15 op/s 2024-04-21T08:47:20.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:19 smithi086 ceph-mon[27939]: pgmap v265: 129 pgs: 129 active+clean; 3.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 18 op/s 2024-04-21T08:47:20.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:19 smithi088 ceph-mon[31259]: pgmap v265: 129 pgs: 129 active+clean; 3.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 18 op/s 2024-04-21T08:47:20.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:19 smithi179 ceph-mon[31829]: pgmap v265: 129 pgs: 129 active+clean; 3.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 52 KiB/s wr, 18 op/s 2024-04-21T08:47:22.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:21 smithi086 ceph-mon[27939]: pgmap v266: 129 pgs: 129 active+clean; 3.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-04-21T08:47:22.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:21 smithi088 ceph-mon[31259]: pgmap v266: 129 pgs: 129 active+clean; 3.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-04-21T08:47:22.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:21 smithi179 ceph-mon[31829]: pgmap v266: 129 pgs: 129 active+clean; 3.4 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 18 op/s 2024-04-21T08:47:24.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:23 smithi086 ceph-mon[27939]: pgmap v267: 129 pgs: 129 active+clean; 3.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 61 KiB/s wr, 22 op/s 2024-04-21T08:47:24.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:23 smithi088 ceph-mon[31259]: pgmap v267: 129 pgs: 129 active+clean; 3.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 61 KiB/s wr, 22 op/s 2024-04-21T08:47:24.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:23 smithi179 ceph-mon[31829]: pgmap v267: 129 pgs: 129 active+clean; 3.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 63 KiB/s rd, 61 KiB/s wr, 22 op/s 2024-04-21T08:47:26.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:25 smithi086 ceph-mon[27939]: pgmap v268: 129 pgs: 129 active+clean; 3.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:47:26.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:25 smithi088 ceph-mon[31259]: pgmap v268: 129 pgs: 129 active+clean; 3.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:47:26.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:25 smithi179 ceph-mon[31829]: pgmap v268: 129 pgs: 129 active+clean; 3.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:47:26.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.382+0000 7f341ca93640 1 -- 172.21.15.86:0/1293019630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3418102b70 msgr2=0x7f3418106bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:26.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.382+0000 7f341ca93640 1 --2- 172.21.15.86:0/1293019630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3418102b70 0x7f3418106bb0 secure :-1 s=READY pgs=1621 cs=0 l=1 rev1=1 crypto rx=0x7f3400009900 tx=0x7f340002bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:26.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.382+0000 7f341ca93640 1 -- 172.21.15.86:0/1293019630 shutdown_connections 2024-04-21T08:47:26.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.382+0000 7f341ca93640 1 --2- 172.21.15.86:0/1293019630 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3418107290 0x7f341810ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:26.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.382+0000 7f341ca93640 1 --2- 172.21.15.86:0/1293019630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3418102b70 0x7f3418106bb0 unknown :-1 s=CLOSED pgs=1621 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:26.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.382+0000 7f341ca93640 1 --2- 172.21.15.86:0/1293019630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34181021c0 0x7f34181025a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:26.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.382+0000 7f341ca93640 1 -- 172.21.15.86:0/1293019630 >> 172.21.15.86:0/1293019630 conn(0x7f34180fda20 msgr2=0x7f34180ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:26.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.382+0000 7f341ca93640 1 -- 172.21.15.86:0/1293019630 shutdown_connections 2024-04-21T08:47:26.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.382+0000 7f341ca93640 1 -- 172.21.15.86:0/1293019630 wait complete. 2024-04-21T08:47:26.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.383+0000 7f341ca93640 1 Processor -- start 2024-04-21T08:47:26.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.383+0000 7f341ca93640 1 -- start start 2024-04-21T08:47:26.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.383+0000 7f341ca93640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34181021c0 0x7f3418198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:26.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.383+0000 7f341ca93640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3418107290 0x7f3418198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:26.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.384+0000 7f341ca93640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f341819cf80 0x7f341819d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:26.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.384+0000 7f341ca93640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f34181121d0 con 0x7f3418107290 2024-04-21T08:47:26.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.384+0000 7f341ca93640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3418112050 con 0x7f341819cf80 2024-04-21T08:47:26.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.384+0000 7f341ca93640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3418112350 con 0x7f34181021c0 2024-04-21T08:47:26.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.384+0000 7f340ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3418107290 0x7f3418198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:26.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.384+0000 7f340ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3418107290 0x7f3418198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42670/0 (socket says 172.21.15.86:42670) 2024-04-21T08:47:26.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.384+0000 7f340ffff640 1 -- 172.21.15.86:0/1233942879 learned_addr learned my addr 172.21.15.86:0/1233942879 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:26.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.384+0000 7f3416d9d640 1 --2- 172.21.15.86:0/1233942879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f341819cf80 0x7f341819d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:26.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.384+0000 7f341659c640 1 --2- 172.21.15.86:0/1233942879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34181021c0 0x7f3418198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:26.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.384+0000 7f340ffff640 1 -- 172.21.15.86:0/1233942879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34181021c0 msgr2=0x7f3418198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:26.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.384+0000 7f340ffff640 1 --2- 172.21.15.86:0/1233942879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34181021c0 0x7f3418198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:26.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.384+0000 7f340ffff640 1 -- 172.21.15.86:0/1233942879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f341819cf80 msgr2=0x7f341819d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:26.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.384+0000 7f340ffff640 1 --2- 172.21.15.86:0/1233942879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f341819cf80 0x7f341819d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:26.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.384+0000 7f340ffff640 1 -- 172.21.15.86:0/1233942879 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f341819da70 con 0x7f3418107290 2024-04-21T08:47:26.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.384+0000 7f3416d9d640 1 --2- 172.21.15.86:0/1233942879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f341819cf80 0x7f341819d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:26.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.384+0000 7f341659c640 1 --2- 172.21.15.86:0/1233942879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34181021c0 0x7f3418198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:26.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.384+0000 7f340ffff640 1 --2- 172.21.15.86:0/1233942879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3418107290 0x7f3418198c50 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f3400036040 tx=0x7f34000045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:26.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.384+0000 7f340f7fe640 1 -- 172.21.15.86:0/1233942879 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3400043020 con 0x7f3418107290 2024-04-21T08:47:26.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.384+0000 7f340f7fe640 1 -- 172.21.15.86:0/1233942879 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f340002cb70 con 0x7f3418107290 2024-04-21T08:47:26.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.384+0000 7f341ca93640 1 -- 172.21.15.86:0/1233942879 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f341819dd00 con 0x7f3418107290 2024-04-21T08:47:26.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.384+0000 7f340f7fe640 1 -- 172.21.15.86:0/1233942879 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3400003bf0 con 0x7f3418107290 2024-04-21T08:47:26.387 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.384+0000 7f341ca93640 1 -- 172.21.15.86:0/1233942879 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f34181a5520 con 0x7f3418107290 2024-04-21T08:47:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.387+0000 7f340f7fe640 1 -- 172.21.15.86:0/1233942879 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3400046020 con 0x7f3418107290 2024-04-21T08:47:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.387+0000 7f341f764740 1 Processor -- start 2024-04-21T08:47:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.387+0000 7f341f764740 1 -- start start 2024-04-21T08:47:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.387+0000 7f341f764740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c708278d0 0x556c70826620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.387+0000 7f341f764740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c70826b60 0x556c70826fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.387+0000 7f341f764740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556c708300e0 0x556c708324d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.387+0000 7f341f764740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556c7078b8d0 con 0x556c70826b60 2024-04-21T08:47:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.387+0000 7f341f764740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x556c70728e80 con 0x556c708300e0 2024-04-21T08:47:26.394 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.387+0000 7f341f764740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556c70787890 con 0x556c708278d0 2024-04-21T08:47:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.388+0000 7f340f7fe640 1 --2- 172.21.15.86:0/1233942879 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f34040a3dc0 0x7f34040a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.388+0000 7f341659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c70826b60 0x556c70826fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.388+0000 7f341659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c70826b60 0x556c70826fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42676/0 (socket says 172.21.15.86:42676) 2024-04-21T08:47:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.388+0000 7f341659c640 1 -- 172.21.15.86:0/3562201135 learned_addr learned my addr 172.21.15.86:0/3562201135 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.388+0000 7f3416d9d640 1 --2- 172.21.15.86:0/3562201135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c708278d0 0x556c70826620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.388+0000 7f340ffff640 1 --2- 172.21.15.86:0/3562201135 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556c708300e0 0x556c708324d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.388+0000 7f3416d9d640 1 --2- 172.21.15.86:0/1233942879 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f34040a3dc0 0x7f34040a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.388+0000 7f340f7fe640 1 -- 172.21.15.86:0/1233942879 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f34000f2d30 con 0x7f3418107290 2024-04-21T08:47:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.388+0000 7f341659c640 1 -- 172.21.15.86:0/3562201135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c708278d0 msgr2=0x556c70826620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.388+0000 7f341659c640 1 --2- 172.21.15.86:0/3562201135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c708278d0 0x556c70826620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.388+0000 7f341659c640 1 -- 172.21.15.86:0/3562201135 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556c708300e0 msgr2=0x556c708324d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.388+0000 7f341659c640 1 --2- 172.21.15.86:0/3562201135 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556c708300e0 0x556c708324d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:26.395 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.388+0000 7f341659c640 1 -- 172.21.15.86:0/3562201135 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c7074a320 con 0x556c70826b60 2024-04-21T08:47:26.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.388+0000 7f341659c640 1 --2- 172.21.15.86:0/3562201135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c70826b60 0x556c70826fe0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f340800bd40 tx=0x7f340800f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75d768054a80a336 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:26.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.388+0000 7f3416d9d640 1 --2- 172.21.15.86:0/1233942879 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f34040a3dc0 0x7f34040a6280 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3410009fd0 tx=0x7f3410009500 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:26.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.388+0000 7f340cff9640 1 -- 172.21.15.86:0/3562201135 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3408015070 con 0x556c70826b60 2024-04-21T08:47:26.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.388+0000 7f340cff9640 1 -- 172.21.15.86:0/3562201135 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f340800be30 con 0x556c70826b60 2024-04-21T08:47:26.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.388+0000 7f340cff9640 1 -- 172.21.15.86:0/3562201135 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3408011940 con 0x556c70826b60 2024-04-21T08:47:26.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.389+0000 7f341f764740 1 -- 172.21.15.86:0/3562201135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c70826b60 msgr2=0x556c70826fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:26.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.389+0000 7f341f764740 1 --2- 172.21.15.86:0/3562201135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c70826b60 0x556c70826fe0 secure :-1 s=READY pgs=1623 cs=0 l=1 rev1=1 crypto rx=0x7f340800bd40 tx=0x7f340800f730 comp rx=0 tx=0).stop 2024-04-21T08:47:26.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.389+0000 7f341f764740 1 -- 172.21.15.86:0/3562201135 shutdown_connections 2024-04-21T08:47:26.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.389+0000 7f341f764740 1 --2- 172.21.15.86:0/3562201135 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556c708300e0 0x556c708324d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:26.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.389+0000 7f341f764740 1 --2- 172.21.15.86:0/3562201135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c70826b60 0x556c70826fe0 unknown :-1 s=CLOSED pgs=1623 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:26.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.389+0000 7f341f764740 1 --2- 172.21.15.86:0/3562201135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c708278d0 0x556c70826620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:26.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.389+0000 7f341f764740 1 -- 172.21.15.86:0/3562201135 >> 172.21.15.86:0/3562201135 conn(0x556c707edc90 msgr2=0x556c70821850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:26.396 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.389+0000 7f341f764740 1 -- 172.21.15.86:0/3562201135 shutdown_connections 2024-04-21T08:47:26.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.389+0000 7f341f764740 1 -- 172.21.15.86:0/3562201135 wait complete. 2024-04-21T08:47:26.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.390+0000 7f341f764740 1 -- start start 2024-04-21T08:47:26.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.390+0000 7f341f764740 1 Processor -- start 2024-04-21T08:47:26.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.390+0000 7f341f764740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556c70826b60 0x556c7082fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:26.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.390+0000 7f341f764740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c708278d0 0x556c7082ce10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:26.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.390+0000 7f341f764740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c708300e0 0x556c7082d3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:26.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.390+0000 7f341f764740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556c70832e10 con 0x556c708300e0 2024-04-21T08:47:26.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.390+0000 7f341f764740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x556c70713eb0 con 0x556c70826b60 2024-04-21T08:47:26.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.390+0000 7f341f764740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556c70832f90 con 0x556c708278d0 2024-04-21T08:47:26.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.390+0000 7f341f764740 10 client.0 _ll_register_callbacks cb 0x556c70740b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:26.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.390+0000 7f3416d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c708300e0 0x556c7082d3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:26.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.390+0000 7f3416d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c708300e0 0x556c7082d3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42688/0 (socket says 172.21.15.86:42688) 2024-04-21T08:47:26.397 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.390+0000 7f3416d9d640 1 -- 172.21.15.86:0/98678601 learned_addr learned my addr 172.21.15.86:0/98678601 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:26.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.390+0000 7f340ffff640 1 --2- 172.21.15.86:0/98678601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c708278d0 0x556c7082ce10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:26.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.390+0000 7f341659c640 1 --2- 172.21.15.86:0/98678601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556c70826b60 0x556c7082fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:26.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.390+0000 7f3416d9d640 1 -- 172.21.15.86:0/98678601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c708278d0 msgr2=0x556c7082ce10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:26.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.390+0000 7f3416d9d640 1 --2- 172.21.15.86:0/98678601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c708278d0 0x556c7082ce10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:26.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.390+0000 7f3416d9d640 1 -- 172.21.15.86:0/98678601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556c70826b60 msgr2=0x556c7082fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:26.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.390+0000 7f3416d9d640 1 --2- 172.21.15.86:0/98678601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556c70826b60 0x556c7082fa80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:26.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.390+0000 7f3416d9d640 1 -- 172.21.15.86:0/98678601 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c7049e210 con 0x556c708300e0 2024-04-21T08:47:26.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.390+0000 7f3416d9d640 1 --2- 172.21.15.86:0/98678601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c708300e0 0x556c7082d3b0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f3410012580 tx=0x7f3410004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=562745c88011246d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:26.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.390+0000 7f33fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:47:26.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.391+0000 7f33fdffb640 1 -- 172.21.15.86:0/98678601 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341000fc10 con 0x556c708300e0 2024-04-21T08:47:26.398 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.391+0000 7f33fdffb640 1 -- 172.21.15.86:0/98678601 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f341000fdb0 con 0x556c708300e0 2024-04-21T08:47:26.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.391+0000 7f341f764740 1 -- 172.21.15.86:0/98678601 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f340800be30 con 0x556c708300e0 2024-04-21T08:47:26.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.391+0000 7f33fdffb640 1 -- 172.21.15.86:0/98678601 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3410004b20 con 0x556c708300e0 2024-04-21T08:47:26.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.391+0000 7f341f764740 10 client.19080 fetch_fsmap learned FSMap version 29 2024-04-21T08:47:26.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.391+0000 7f341f764740 1 -- 172.21.15.86:0/98678601 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556c7056a620 con 0x556c708300e0 2024-04-21T08:47:26.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.391+0000 7f33fdffb640 1 -- 172.21.15.86:0/98678601 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f340800be30 con 0x556c708300e0 2024-04-21T08:47:26.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.391+0000 7f33fdffb640 1 -- 172.21.15.86:0/98678601 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3410020530 con 0x556c708300e0 2024-04-21T08:47:26.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.391+0000 7f341f764740 10 client.19080 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:47:26.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.391+0000 7f341f764740 10 client.19080 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:26.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.391+0000 7f341f764740 20 client.19080 populate_metadata read hostname 'smithi086' 2024-04-21T08:47:26.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.391+0000 7f341f764740 1 --2- 172.21.15.86:0/98678601 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x556c708401a0 0x556c708425b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:26.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.391+0000 7f341f764740 4 client.19080 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:26.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.391+0000 7f341f764740 1 -- 172.21.15.86:0/98678601 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556c7079f880 con 0x556c708401a0 2024-04-21T08:47:26.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.392+0000 7f341659c640 1 --2- 172.21.15.86:0/98678601 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x556c708401a0 0x556c708425b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:26.399 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.392+0000 7f341659c640 1 --2- 172.21.15.86:0/98678601 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x556c708401a0 0x556c708425b0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=143f35e5b2da9ce8 server_cookie=8a726f1593ee7645 in_seq=0 out_seq=0 2024-04-21T08:47:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.392+0000 7f33fdffb640 10 client.19080 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.393+0000 7f33fdffb640 1 -- 172.21.15.86:0/98678601 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556c7079f880 con 0x556c708401a0 2024-04-21T08:47:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.393+0000 7f33fdffb640 10 client.19080 handle_command_reply: tid=0 2024-04-21T08:47:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.393+0000 7f341f764740 1 client.19080 shutdown 2024-04-21T08:47:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.393+0000 7f341f764740 1 -- 172.21.15.86:0/98678601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c708300e0 msgr2=0x556c7082d3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.393+0000 7f341f764740 1 --2- 172.21.15.86:0/98678601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c708300e0 0x556c7082d3b0 secure :-1 s=READY pgs=1624 cs=0 l=1 rev1=1 crypto rx=0x7f3410012580 tx=0x7f3410004a40 comp rx=0 tx=0).stop 2024-04-21T08:47:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.393+0000 7f341f764740 1 -- 172.21.15.86:0/98678601 shutdown_connections 2024-04-21T08:47:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.393+0000 7f341f764740 1 --2- 172.21.15.86:0/98678601 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x556c708401a0 0x556c708425b0 crc :-1 s=READY pgs=255 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.393+0000 7f341f764740 1 --2- 172.21.15.86:0/98678601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c708300e0 0x556c7082d3b0 unknown :-1 s=CLOSED pgs=1624 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.393+0000 7f341f764740 1 --2- 172.21.15.86:0/98678601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c708278d0 0x556c7082ce10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.393+0000 7f341f764740 1 --2- 172.21.15.86:0/98678601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556c70826b60 0x556c7082fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.393+0000 7f33fdffb640 0 client.19080 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.393+0000 7f341f764740 1 -- 172.21.15.86:0/98678601 >> 172.21.15.86:0/98678601 conn(0x556c707edc90 msgr2=0x556c708235a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.393+0000 7f341f764740 1 -- 172.21.15.86:0/98678601 shutdown_connections 2024-04-21T08:47:26.400 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.393+0000 7f341f764740 1 -- 172.21.15.86:0/98678601 wait complete. 2024-04-21T08:47:26.401 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.393+0000 7f341f764740 20 client.19080 trim_cache size 0 max 16384 2024-04-21T08:47:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.407+0000 7f341f764740 1 Processor -- start 2024-04-21T08:47:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.407+0000 7f341f764740 1 -- start start 2024-04-21T08:47:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.408+0000 7f341f764740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c70826b60 0x556c7083bd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.408+0000 7f341f764740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556c708278d0 0x556c7082f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.408+0000 7f341f764740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c708300e0 0x556c7082fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.408+0000 7f341f764740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f34181af260 con 0x556c70826b60 2024-04-21T08:47:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.408+0000 7f341f764740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f34181b0da0 con 0x556c708278d0 2024-04-21T08:47:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.408+0000 7f341f764740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f34181af930 con 0x556c708300e0 2024-04-21T08:47:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.408+0000 7f341659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c70826b60 0x556c7083bd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.408+0000 7f341659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c70826b60 0x556c7083bd10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42702/0 (socket says 172.21.15.86:42702) 2024-04-21T08:47:26.412 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.408+0000 7f341659c640 1 -- 172.21.15.86:0/3272519455 learned_addr learned my addr 172.21.15.86:0/3272519455 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.408+0000 7f340ffff640 1 --2- 172.21.15.86:0/3272519455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556c708278d0 0x556c7082f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.408+0000 7f341659c640 1 -- 172.21.15.86:0/3272519455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c708300e0 msgr2=0x556c7082fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.408+0000 7f341659c640 1 --2- 172.21.15.86:0/3272519455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c708300e0 0x556c7082fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.408+0000 7f341659c640 1 -- 172.21.15.86:0/3272519455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556c708278d0 msgr2=0x556c7082f540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.408+0000 7f341659c640 1 --2- 172.21.15.86:0/3272519455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556c708278d0 0x556c7082f540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.408+0000 7f341659c640 1 -- 172.21.15.86:0/3272519455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556c7081cf30 con 0x556c70826b60 2024-04-21T08:47:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.408+0000 7f341659c640 1 --2- 172.21.15.86:0/3272519455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c70826b60 0x556c7083bd10 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f34080114d0 tx=0x7f3408011580 comp rx=0 tx=0).ready entity=mon.0 client_cookie=631e4e6d28e6206c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.408+0000 7f33fffff640 1 -- 172.21.15.86:0/3272519455 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f340802e070 con 0x556c70826b60 2024-04-21T08:47:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.408+0000 7f33fffff640 1 -- 172.21.15.86:0/3272519455 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3408005570 con 0x556c70826b60 2024-04-21T08:47:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.408+0000 7f33fffff640 1 -- 172.21.15.86:0/3272519455 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3408015040 con 0x556c70826b60 2024-04-21T08:47:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.409+0000 7f341f764740 1 -- 172.21.15.86:0/3272519455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c70826b60 msgr2=0x556c7083bd10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.409+0000 7f341f764740 1 --2- 172.21.15.86:0/3272519455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c70826b60 0x556c7083bd10 secure :-1 s=READY pgs=1625 cs=0 l=1 rev1=1 crypto rx=0x7f34080114d0 tx=0x7f3408011580 comp rx=0 tx=0).stop 2024-04-21T08:47:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.409+0000 7f341f764740 1 -- 172.21.15.86:0/3272519455 shutdown_connections 2024-04-21T08:47:26.413 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.409+0000 7f341f764740 1 --2- 172.21.15.86:0/3272519455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c708300e0 0x556c7082fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.409+0000 7f341f764740 1 --2- 172.21.15.86:0/3272519455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556c708278d0 0x556c7082f540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.409+0000 7f341f764740 1 --2- 172.21.15.86:0/3272519455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c70826b60 0x556c7083bd10 unknown :-1 s=CLOSED pgs=1625 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.409+0000 7f341f764740 1 -- 172.21.15.86:0/3272519455 >> 172.21.15.86:0/3272519455 conn(0x556c707edc90 msgr2=0x556c7082b6e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.409+0000 7f341f764740 1 -- 172.21.15.86:0/3272519455 shutdown_connections 2024-04-21T08:47:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.409+0000 7f341f764740 1 -- 172.21.15.86:0/3272519455 wait complete. 2024-04-21T08:47:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.409+0000 7f341f764740 1 -- start start 2024-04-21T08:47:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.409+0000 7f341f764740 1 Processor -- start 2024-04-21T08:47:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.409+0000 7f341f764740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c70826b60 0x556c708384f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.409+0000 7f341f764740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556c708278d0 0x556c7082a100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.409+0000 7f341f764740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c708300e0 0x556c7082a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.410+0000 7f341f764740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f34181b06d0 con 0x556c708300e0 2024-04-21T08:47:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.410+0000 7f341f764740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f34181b0000 con 0x556c708278d0 2024-04-21T08:47:26.414 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.410+0000 7f341f764740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f34181ad260 con 0x556c70826b60 2024-04-21T08:47:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.410+0000 7f341f764740 10 client.0 _ll_register_callbacks cb 0x7f3410009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.410+0000 7f3416d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c708300e0 0x556c7082a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.410+0000 7f3416d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c708300e0 0x556c7082a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42718/0 (socket says 172.21.15.86:42718) 2024-04-21T08:47:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.410+0000 7f3416d9d640 1 -- 172.21.15.86:0/2399868125 learned_addr learned my addr 172.21.15.86:0/2399868125 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.410+0000 7f340ffff640 1 --2- 172.21.15.86:0/2399868125 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556c708278d0 0x556c7082a100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.410+0000 7f341659c640 1 --2- 172.21.15.86:0/2399868125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c70826b60 0x556c708384f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.410+0000 7f3416d9d640 1 -- 172.21.15.86:0/2399868125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c70826b60 msgr2=0x556c708384f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.410+0000 7f3416d9d640 1 --2- 172.21.15.86:0/2399868125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c70826b60 0x556c708384f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.410+0000 7f3416d9d640 1 -- 172.21.15.86:0/2399868125 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556c708278d0 msgr2=0x556c7082a100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.410+0000 7f3416d9d640 1 --2- 172.21.15.86:0/2399868125 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556c708278d0 0x556c7082a100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.410+0000 7f3416d9d640 1 -- 172.21.15.86:0/2399868125 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556c7082ad80 con 0x556c708300e0 2024-04-21T08:47:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.410+0000 7f341659c640 1 --2- 172.21.15.86:0/2399868125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c70826b60 0x556c708384f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.410+0000 7f3416d9d640 1 --2- 172.21.15.86:0/2399868125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c708300e0 0x556c7082a640 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f3410020b10 tx=0x7f341000fe40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dbb92b8a44d1e768 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:26.415 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.410+0000 7f33fdffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:47:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.410+0000 7f33fdffb640 1 -- 172.21.15.86:0/2399868125 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f341000f8e0 con 0x556c708300e0 2024-04-21T08:47:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.410+0000 7f33fdffb640 1 -- 172.21.15.86:0/2399868125 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f341001b070 con 0x556c708300e0 2024-04-21T08:47:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.410+0000 7f341f764740 1 -- 172.21.15.86:0/2399868125 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3408005570 con 0x556c708300e0 2024-04-21T08:47:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.410+0000 7f33fdffb640 1 -- 172.21.15.86:0/2399868125 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3410043050 con 0x556c708300e0 2024-04-21T08:47:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.410+0000 7f341f764740 10 client.19086 fetch_fsmap learned FSMap version 29 2024-04-21T08:47:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.410+0000 7f341f764740 1 -- 172.21.15.86:0/2399868125 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556c708a6730 con 0x556c708300e0 2024-04-21T08:47:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.411+0000 7f33fdffb640 1 -- 172.21.15.86:0/2399868125 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3408005570 con 0x556c708300e0 2024-04-21T08:47:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.411+0000 7f33fdffb640 1 -- 172.21.15.86:0/2399868125 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3410036480 con 0x556c708300e0 2024-04-21T08:47:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.411+0000 7f341f764740 10 client.19086 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:47:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.411+0000 7f341f764740 10 client.19086 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.411+0000 7f341f764740 20 client.19086 populate_metadata read hostname 'smithi086' 2024-04-21T08:47:26.416 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.411+0000 7f341f764740 1 --2- 172.21.15.86:0/2399868125 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x556c708a6dc0 0x556c708a91e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.411+0000 7f341f764740 4 client.19086 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:47:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.411+0000 7f341f764740 1 -- 172.21.15.86:0/2399868125 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x556c7072b3d0 con 0x556c708a6dc0 2024-04-21T08:47:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.411+0000 7f341659c640 1 --2- 172.21.15.86:0/2399868125 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x556c708a6dc0 0x556c708a91e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.411+0000 7f33fdffb640 10 client.19086 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:26.417 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.411+0000 7f341659c640 1 --2- 172.21.15.86:0/2399868125 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x556c708a6dc0 0x556c708a91e0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dfcdc83efef9a28 server_cookie=312076124fce4e8a in_seq=0 out_seq=0 2024-04-21T08:47:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.425+0000 7f33fdffb640 1 -- 172.21.15.86:0/2399868125 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x556c7072b3d0 con 0x556c708a6dc0 2024-04-21T08:47:26.425 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.425+0000 7f33fdffb640 10 client.19086 handle_command_reply: tid=0 2024-04-21T08:47:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.425+0000 7f341f764740 1 client.19086 shutdown 2024-04-21T08:47:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.426+0000 7f341f764740 1 -- 172.21.15.86:0/2399868125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c708300e0 msgr2=0x556c7082a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.426+0000 7f341f764740 1 --2- 172.21.15.86:0/2399868125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c708300e0 0x556c7082a640 secure :-1 s=READY pgs=1626 cs=0 l=1 rev1=1 crypto rx=0x7f3410020b10 tx=0x7f341000fe40 comp rx=0 tx=0).stop 2024-04-21T08:47:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.426+0000 7f341f764740 1 -- 172.21.15.86:0/2399868125 shutdown_connections 2024-04-21T08:47:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.426+0000 7f341f764740 1 --2- 172.21.15.86:0/2399868125 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x556c708a6dc0 0x556c708a91e0 crc :-1 s=READY pgs=256 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.426+0000 7f341f764740 1 --2- 172.21.15.86:0/2399868125 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556c708300e0 0x556c7082a640 unknown :-1 s=CLOSED pgs=1626 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.426+0000 7f341f764740 1 --2- 172.21.15.86:0/2399868125 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556c708278d0 0x556c7082a100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.426+0000 7f341f764740 1 --2- 172.21.15.86:0/2399868125 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556c70826b60 0x556c708384f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.426+0000 7f33fdffb640 0 client.19086 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.426+0000 7f341f764740 1 -- 172.21.15.86:0/2399868125 >> 172.21.15.86:0/2399868125 conn(0x556c707edc90 msgr2=0x556c70820040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.426+0000 7f341f764740 1 -- 172.21.15.86:0/2399868125 shutdown_connections 2024-04-21T08:47:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.426+0000 7f341f764740 1 -- 172.21.15.86:0/2399868125 wait complete. 2024-04-21T08:47:26.426 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.426+0000 7f341f764740 20 client.19086 trim_cache size 0 max 16384 2024-04-21T08:47:26.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:47:26.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:47:26.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:47:26.427 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:47:26.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.426+0000 7f341ca93640 1 -- 172.21.15.86:0/1233942879 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f34040a3dc0 msgr2=0x7f34040a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:26.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.426+0000 7f341ca93640 1 --2- 172.21.15.86:0/1233942879 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f34040a3dc0 0x7f34040a6280 secure :-1 s=READY pgs=264 cs=0 l=1 rev1=1 crypto rx=0x7f3410009fd0 tx=0x7f3410009500 comp rx=0 tx=0).stop 2024-04-21T08:47:26.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.426+0000 7f341ca93640 1 -- 172.21.15.86:0/1233942879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3418107290 msgr2=0x7f3418198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:26.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.426+0000 7f341ca93640 1 --2- 172.21.15.86:0/1233942879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3418107290 0x7f3418198c50 secure :-1 s=READY pgs=1622 cs=0 l=1 rev1=1 crypto rx=0x7f3400036040 tx=0x7f34000045e0 comp rx=0 tx=0).stop 2024-04-21T08:47:26.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.426+0000 7f341ca93640 1 -- 172.21.15.86:0/1233942879 shutdown_connections 2024-04-21T08:47:26.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.426+0000 7f341ca93640 1 --2- 172.21.15.86:0/1233942879 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f34040a3dc0 0x7f34040a6280 unknown :-1 s=CLOSED pgs=264 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:26.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.426+0000 7f341ca93640 1 --2- 172.21.15.86:0/1233942879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f341819cf80 0x7f341819d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:26.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.426+0000 7f341ca93640 1 --2- 172.21.15.86:0/1233942879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3418107290 0x7f3418198c50 unknown :-1 s=CLOSED pgs=1622 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:26.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.426+0000 7f341ca93640 1 --2- 172.21.15.86:0/1233942879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f34181021c0 0x7f3418198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:26.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.426+0000 7f341ca93640 1 -- 172.21.15.86:0/1233942879 >> 172.21.15.86:0/1233942879 conn(0x7f34180fda20 msgr2=0x7f34180ffdc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:26.428 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.426+0000 7f341ca93640 1 -- 172.21.15.86:0/1233942879 shutdown_connections 2024-04-21T08:47:26.429 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:26.426+0000 7f341ca93640 1 -- 172.21.15.86:0/1233942879 wait complete. 2024-04-21T08:47:28.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:27 smithi088 ceph-mon[31259]: pgmap v269: 129 pgs: 129 active+clean; 3.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:47:28.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:27 smithi179 ceph-mon[31829]: pgmap v269: 129 pgs: 129 active+clean; 3.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:47:28.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:27 smithi086 ceph-mon[27939]: pgmap v269: 129 pgs: 129 active+clean; 3.6 MiB data, 414 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:47:30.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:29 smithi088 ceph-mon[31259]: pgmap v270: 129 pgs: 129 active+clean; 3.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 67 KiB/s wr, 24 op/s 2024-04-21T08:47:30.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:29 smithi179 ceph-mon[31829]: pgmap v270: 129 pgs: 129 active+clean; 3.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 67 KiB/s wr, 24 op/s 2024-04-21T08:47:30.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:29 smithi086 ceph-mon[27939]: pgmap v270: 129 pgs: 129 active+clean; 3.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 67 KiB/s wr, 24 op/s 2024-04-21T08:47:30.526 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:47:30.527 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:47:30.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.639+0000 7f8ba0843640 1 -- 172.21.15.86:0/2057721245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b9c06a5b0 msgr2=0x7f8b9c06aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:30.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.639+0000 7f8ba0843640 1 --2- 172.21.15.86:0/2057721245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b9c06a5b0 0x7f8b9c06aa30 secure :-1 s=READY pgs=1627 cs=0 l=1 rev1=1 crypto rx=0x7f8b84009900 tx=0x7f8b8402bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:30.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.639+0000 7f8ba0843640 1 -- 172.21.15.86:0/2057721245 shutdown_connections 2024-04-21T08:47:30.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.639+0000 7f8ba0843640 1 --2- 172.21.15.86:0/2057721245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b9c06af70 0x7f8b9c10edb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.639+0000 7f8ba0843640 1 --2- 172.21.15.86:0/2057721245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b9c06a5b0 0x7f8b9c06aa30 unknown :-1 s=CLOSED pgs=1627 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.639+0000 7f8ba0843640 1 --2- 172.21.15.86:0/2057721245 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b9c106f20 0x7f8b9c107300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.639+0000 7f8ba0843640 1 -- 172.21.15.86:0/2057721245 >> 172.21.15.86:0/2057721245 conn(0x7f8b9c077c40 msgr2=0x7f8b9c078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:30.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.639+0000 7f8ba0843640 1 -- 172.21.15.86:0/2057721245 shutdown_connections 2024-04-21T08:47:30.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.639+0000 7f8ba0843640 1 -- 172.21.15.86:0/2057721245 wait complete. 2024-04-21T08:47:30.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.640+0000 7f8ba0843640 1 Processor -- start 2024-04-21T08:47:30.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.640+0000 7f8ba0843640 1 -- start start 2024-04-21T08:47:30.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.640+0000 7f8ba0843640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b9c06a5b0 0x7f8b9c1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:30.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.640+0000 7f8ba0843640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b9c06af70 0x7f8b9c198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:30.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.640+0000 7f8ba0843640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b9c106f20 0x7f8b9c19cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:30.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.640+0000 7f8ba0843640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9c112170 con 0x7f8b9c06af70 2024-04-21T08:47:30.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.640+0000 7f8ba0843640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9c111ff0 con 0x7f8b9c106f20 2024-04-21T08:47:30.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.640+0000 7f8ba0843640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9c1122f0 con 0x7f8b9c06a5b0 2024-04-21T08:47:30.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.640+0000 7f8b99d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b9c06af70 0x7f8b9c198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:30.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.640+0000 7f8b99d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b9c06af70 0x7f8b9c198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52490/0 (socket says 172.21.15.86:52490) 2024-04-21T08:47:30.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.640+0000 7f8b99d9b640 1 -- 172.21.15.86:0/1583372754 learned_addr learned my addr 172.21.15.86:0/1583372754 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:30.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.640+0000 7f8b9a59c640 1 --2- 172.21.15.86:0/1583372754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b9c06a5b0 0x7f8b9c1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:30.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.640+0000 7f8b9ad9d640 1 --2- 172.21.15.86:0/1583372754 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b9c106f20 0x7f8b9c19cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:30.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.640+0000 7f8b99d9b640 1 -- 172.21.15.86:0/1583372754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b9c06a5b0 msgr2=0x7f8b9c1986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:30.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.640+0000 7f8b99d9b640 1 --2- 172.21.15.86:0/1583372754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b9c06a5b0 0x7f8b9c1986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.640+0000 7f8b99d9b640 1 -- 172.21.15.86:0/1583372754 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b9c106f20 msgr2=0x7f8b9c19cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:30.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.640+0000 7f8b99d9b640 1 --2- 172.21.15.86:0/1583372754 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b9c106f20 0x7f8b9c19cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.640+0000 7f8b99d9b640 1 -- 172.21.15.86:0/1583372754 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8b9c19d670 con 0x7f8b9c06af70 2024-04-21T08:47:30.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.640+0000 7f8b9ad9d640 1 --2- 172.21.15.86:0/1583372754 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b9c106f20 0x7f8b9c19cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:30.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.641+0000 7f8b99d9b640 1 --2- 172.21.15.86:0/1583372754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b9c06af70 0x7f8b9c198c10 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f8b84006010 tx=0x7f8b84004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:30.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.641+0000 7f8b937fe640 1 -- 172.21.15.86:0/1583372754 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b84043020 con 0x7f8b9c06af70 2024-04-21T08:47:30.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.641+0000 7f8ba0843640 1 -- 172.21.15.86:0/1583372754 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8b9c19d900 con 0x7f8b9c06af70 2024-04-21T08:47:30.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.641+0000 7f8b937fe640 1 -- 172.21.15.86:0/1583372754 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b84004440 con 0x7f8b9c06af70 2024-04-21T08:47:30.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.641+0000 7f8b937fe640 1 -- 172.21.15.86:0/1583372754 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b84003b50 con 0x7f8b9c06af70 2024-04-21T08:47:30.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.641+0000 7f8ba0843640 1 -- 172.21.15.86:0/1583372754 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8b9c1a51f0 con 0x7f8b9c06af70 2024-04-21T08:47:30.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.643+0000 7f8b937fe640 1 -- 172.21.15.86:0/1583372754 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8b84046020 con 0x7f8b9c06af70 2024-04-21T08:47:30.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.643+0000 7f8ba3434740 1 Processor -- start 2024-04-21T08:47:30.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.644+0000 7f8ba3434740 1 -- start start 2024-04-21T08:47:30.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.644+0000 7f8b937fe640 1 --2- 172.21.15.86:0/1583372754 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8b880a3d20 0x7f8b880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:30.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.644+0000 7f8ba3434740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603575a3580 0x5603575a3960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:30.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.644+0000 7f8ba3434740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603575a3ea0 0x5603575ac540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:30.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.644+0000 7f8ba3434740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603575aca80 0x5603575aee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:30.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.644+0000 7f8ba3434740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5603575081e0 con 0x5603575a3580 2024-04-21T08:47:30.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.644+0000 7f8ba3434740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5603574a5390 con 0x5603575a3ea0 2024-04-21T08:47:30.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.644+0000 7f8ba3434740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5603575041a0 con 0x5603575aca80 2024-04-21T08:47:30.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.644+0000 7f8b9ad9d640 1 --2- 172.21.15.86:0/1583372754 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8b880a3d20 0x7f8b880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:30.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.644+0000 7f8b9a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603575a3580 0x5603575a3960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:30.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.644+0000 7f8b9a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603575a3580 0x5603575a3960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52492/0 (socket says 172.21.15.86:52492) 2024-04-21T08:47:30.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.644+0000 7f8b9a59c640 1 -- 172.21.15.86:0/3104470870 learned_addr learned my addr 172.21.15.86:0/3104470870 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:30.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.644+0000 7f8b937fe640 1 -- 172.21.15.86:0/1583372754 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f8b840f2650 con 0x7f8b9c06af70 2024-04-21T08:47:30.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.644+0000 7f8b9ad9d640 1 --2- 172.21.15.86:0/3104470870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603575aca80 0x5603575aee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:30.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.644+0000 7f8b99d9b640 1 --2- 172.21.15.86:0/3104470870 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603575a3ea0 0x5603575ac540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:30.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.644+0000 7f8b9a59c640 1 -- 172.21.15.86:0/3104470870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603575aca80 msgr2=0x5603575aee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:30.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.644+0000 7f8b9a59c640 1 --2- 172.21.15.86:0/3104470870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603575aca80 0x5603575aee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.644+0000 7f8b9a59c640 1 -- 172.21.15.86:0/3104470870 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603575a3ea0 msgr2=0x5603575ac540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:30.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.644+0000 7f8b9a59c640 1 --2- 172.21.15.86:0/3104470870 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603575a3ea0 0x5603575ac540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.644+0000 7f8b9a59c640 1 -- 172.21.15.86:0/3104470870 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603572e47d0 con 0x5603575a3580 2024-04-21T08:47:30.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.644+0000 7f8b9ad9d640 1 --2- 172.21.15.86:0/3104470870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603575aca80 0x5603575aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:30.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.644+0000 7f8b99d9b640 1 --2- 172.21.15.86:0/3104470870 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603575a3ea0 0x5603575ac540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:30.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.644+0000 7f8b9a59c640 1 --2- 172.21.15.86:0/3104470870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603575a3580 0x5603575a3960 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0045f0 tx=0x7f8b8c00bbe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ccdc8a59128eeb5a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:30.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.644+0000 7f8b9ad9d640 1 --2- 172.21.15.86:0/1583372754 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8b880a3d20 0x7f8b880a61e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8b94005fd0 tx=0x7f8b94014970 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:30.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.644+0000 7f8b90ff9640 1 -- 172.21.15.86:0/3104470870 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c003d70 con 0x5603575a3580 2024-04-21T08:47:30.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.645+0000 7f8b90ff9640 1 -- 172.21.15.86:0/3104470870 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b8c015070 con 0x5603575a3580 2024-04-21T08:47:30.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.645+0000 7f8b90ff9640 1 -- 172.21.15.86:0/3104470870 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c011640 con 0x5603575a3580 2024-04-21T08:47:30.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.645+0000 7f8ba3434740 1 -- 172.21.15.86:0/3104470870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603575a3580 msgr2=0x5603575a3960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:30.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.645+0000 7f8ba3434740 1 --2- 172.21.15.86:0/3104470870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603575a3580 0x5603575a3960 secure :-1 s=READY pgs=1629 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c0045f0 tx=0x7f8b8c00bbe0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.645+0000 7f8ba3434740 1 -- 172.21.15.86:0/3104470870 shutdown_connections 2024-04-21T08:47:30.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.645+0000 7f8ba3434740 1 --2- 172.21.15.86:0/3104470870 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603575aca80 0x5603575aee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.645+0000 7f8ba3434740 1 --2- 172.21.15.86:0/3104470870 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603575a3ea0 0x5603575ac540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.645+0000 7f8ba3434740 1 --2- 172.21.15.86:0/3104470870 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603575a3580 0x5603575a3960 unknown :-1 s=CLOSED pgs=1629 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.645+0000 7f8ba3434740 1 -- 172.21.15.86:0/3104470870 >> 172.21.15.86:0/3104470870 conn(0x56035759dcb0 msgr2=0x56035759e090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:30.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.645+0000 7f8ba3434740 1 -- 172.21.15.86:0/3104470870 shutdown_connections 2024-04-21T08:47:30.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.645+0000 7f8ba3434740 1 -- 172.21.15.86:0/3104470870 wait complete. 2024-04-21T08:47:30.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.646+0000 7f8ba3434740 1 -- start start 2024-04-21T08:47:30.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.646+0000 7f8ba3434740 1 Processor -- start 2024-04-21T08:47:30.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.646+0000 7f8ba3434740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603575a3580 0x5603575a9a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:30.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.646+0000 7f8ba3434740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603575a3ea0 0x5603575a9f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:30.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.646+0000 7f8ba3434740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603575aca80 0x5603575aa480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:30.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.646+0000 7f8ba3434740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5603575af870 con 0x5603575aca80 2024-04-21T08:47:30.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.646+0000 7f8ba3434740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5603574c12f0 con 0x5603575a3ea0 2024-04-21T08:47:30.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.646+0000 7f8ba3434740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5603575af9f0 con 0x5603575a3580 2024-04-21T08:47:30.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.646+0000 7f8ba3434740 10 client.0 _ll_register_callbacks cb 0x5603574ce580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:30.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.646+0000 7f8b9ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603575aca80 0x5603575aa480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:30.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.646+0000 7f8b9ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603575aca80 0x5603575aa480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52508/0 (socket says 172.21.15.86:52508) 2024-04-21T08:47:30.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.646+0000 7f8b9ad9d640 1 -- 172.21.15.86:0/1803564375 learned_addr learned my addr 172.21.15.86:0/1803564375 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:30.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.646+0000 7f8b9a59c640 1 --2- 172.21.15.86:0/1803564375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603575a3580 0x5603575a9a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:30.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.646+0000 7f8b99d9b640 1 --2- 172.21.15.86:0/1803564375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603575a3ea0 0x5603575a9f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:30.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.646+0000 7f8b9ad9d640 1 -- 172.21.15.86:0/1803564375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603575a3580 msgr2=0x5603575a9a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:30.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.646+0000 7f8b9ad9d640 1 --2- 172.21.15.86:0/1803564375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603575a3580 0x5603575a9a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.646+0000 7f8b9ad9d640 1 -- 172.21.15.86:0/1803564375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603575a3ea0 msgr2=0x5603575a9f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:30.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.646+0000 7f8b9ad9d640 1 --2- 172.21.15.86:0/1803564375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603575a3ea0 0x5603575a9f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.646+0000 7f8b9ad9d640 1 -- 172.21.15.86:0/1803564375 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56035721a3f0 con 0x5603575aca80 2024-04-21T08:47:30.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.647+0000 7f8b9ad9d640 1 --2- 172.21.15.86:0/1803564375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603575aca80 0x5603575aa480 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f8b94015fd0 tx=0x7f8b94004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f994347cff1e99 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:30.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.647+0000 7f8b81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:47:30.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.647+0000 7f8b81ffb640 1 -- 172.21.15.86:0/1803564375 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b94026070 con 0x5603575aca80 2024-04-21T08:47:30.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.647+0000 7f8b81ffb640 1 -- 172.21.15.86:0/1803564375 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f8b9400fdb0 con 0x5603575aca80 2024-04-21T08:47:30.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.647+0000 7f8ba3434740 1 -- 172.21.15.86:0/1803564375 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b8c015070 con 0x5603575aca80 2024-04-21T08:47:30.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.647+0000 7f8b81ffb640 1 -- 172.21.15.86:0/1803564375 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8b94032050 con 0x5603575aca80 2024-04-21T08:47:30.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.647+0000 7f8ba3434740 10 client.19098 fetch_fsmap learned FSMap version 29 2024-04-21T08:47:30.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.647+0000 7f8ba3434740 1 -- 172.21.15.86:0/1803564375 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5603571da0f0 con 0x5603575aca80 2024-04-21T08:47:30.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.647+0000 7f8b81ffb640 1 -- 172.21.15.86:0/1803564375 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b8c015070 con 0x5603575aca80 2024-04-21T08:47:30.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.647+0000 7f8b81ffb640 1 -- 172.21.15.86:0/1803564375 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8b9401e040 con 0x5603575aca80 2024-04-21T08:47:30.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.647+0000 7f8ba3434740 10 client.19098 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:47:30.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.647+0000 7f8ba3434740 10 client.19098 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:30.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.647+0000 7f8ba3434740 20 client.19098 populate_metadata read hostname 'smithi086' 2024-04-21T08:47:30.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.647+0000 7f8ba3434740 1 --2- 172.21.15.86:0/1803564375 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5603575bc8a0 0x5603575bed20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:30.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.647+0000 7f8ba3434740 4 client.19098 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:30.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.647+0000 7f8ba3434740 1 -- 172.21.15.86:0/1803564375 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560357515c00 con 0x5603575bc8a0 2024-04-21T08:47:30.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.648+0000 7f8b9a59c640 1 --2- 172.21.15.86:0/1803564375 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5603575bc8a0 0x5603575bed20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:30.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.648+0000 7f8b81ffb640 10 client.19098 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:30.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.648+0000 7f8b9a59c640 1 --2- 172.21.15.86:0/1803564375 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5603575bc8a0 0x5603575bed20 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a8d756c47fa85cae server_cookie=50e887d360b9d73a in_seq=0 out_seq=0 2024-04-21T08:47:30.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.649+0000 7f8b81ffb640 1 -- 172.21.15.86:0/1803564375 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560357515c00 con 0x5603575bc8a0 2024-04-21T08:47:30.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.649+0000 7f8b81ffb640 10 client.19098 handle_command_reply: tid=0 2024-04-21T08:47:30.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.649+0000 7f8ba3434740 1 client.19098 shutdown 2024-04-21T08:47:30.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.649+0000 7f8ba3434740 1 -- 172.21.15.86:0/1803564375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603575aca80 msgr2=0x5603575aa480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:30.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.649+0000 7f8ba3434740 1 --2- 172.21.15.86:0/1803564375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603575aca80 0x5603575aa480 secure :-1 s=READY pgs=1630 cs=0 l=1 rev1=1 crypto rx=0x7f8b94015fd0 tx=0x7f8b94004ab0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.649+0000 7f8ba3434740 1 -- 172.21.15.86:0/1803564375 shutdown_connections 2024-04-21T08:47:30.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.649+0000 7f8ba3434740 1 --2- 172.21.15.86:0/1803564375 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5603575bc8a0 0x5603575bed20 crc :-1 s=READY pgs=257 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.649+0000 7f8ba3434740 1 --2- 172.21.15.86:0/1803564375 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603575aca80 0x5603575aa480 unknown :-1 s=CLOSED pgs=1630 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.649+0000 7f8ba3434740 1 --2- 172.21.15.86:0/1803564375 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603575a3ea0 0x5603575a9f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.649+0000 7f8b81ffb640 0 client.19098 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:30.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.649+0000 7f8ba3434740 1 --2- 172.21.15.86:0/1803564375 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603575a3580 0x5603575a9a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.649+0000 7f8ba3434740 1 -- 172.21.15.86:0/1803564375 >> 172.21.15.86:0/1803564375 conn(0x56035759dcb0 msgr2=0x56035759fa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:30.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.649+0000 7f8ba3434740 1 -- 172.21.15.86:0/1803564375 shutdown_connections 2024-04-21T08:47:30.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.649+0000 7f8ba3434740 1 -- 172.21.15.86:0/1803564375 wait complete. 2024-04-21T08:47:30.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.649+0000 7f8ba3434740 20 client.19098 trim_cache size 0 max 16384 2024-04-21T08:47:30.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.666+0000 7f8ba3434740 1 Processor -- start 2024-04-21T08:47:30.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.666+0000 7f8ba3434740 1 -- start start 2024-04-21T08:47:30.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.666+0000 7f8ba3434740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603575a3580 0x5603575ac130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:30.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.666+0000 7f8ba3434740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603575a3ea0 0x5603575a8940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:30.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.666+0000 7f8ba3434740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603575aca80 0x5603575b8760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:30.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.666+0000 7f8ba3434740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9c1af280 con 0x5603575a3ea0 2024-04-21T08:47:30.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.666+0000 7f8ba3434740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9c1b0dc0 con 0x5603575a3580 2024-04-21T08:47:30.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.666+0000 7f8ba3434740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9c1af950 con 0x5603575aca80 2024-04-21T08:47:30.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.666+0000 7f8b99d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603575a3ea0 0x5603575a8940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:30.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.666+0000 7f8b99d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603575a3ea0 0x5603575a8940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52512/0 (socket says 172.21.15.86:52512) 2024-04-21T08:47:30.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.666+0000 7f8b99d9b640 1 -- 172.21.15.86:0/247616700 learned_addr learned my addr 172.21.15.86:0/247616700 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:30.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.666+0000 7f8b9a59c640 1 --2- 172.21.15.86:0/247616700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603575a3580 0x5603575ac130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:30.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.666+0000 7f8b9ad9d640 1 --2- 172.21.15.86:0/247616700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603575aca80 0x5603575b8760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:30.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.666+0000 7f8b99d9b640 1 -- 172.21.15.86:0/247616700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603575aca80 msgr2=0x5603575b8760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:30.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.666+0000 7f8b99d9b640 1 --2- 172.21.15.86:0/247616700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603575aca80 0x5603575b8760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.666+0000 7f8b99d9b640 1 -- 172.21.15.86:0/247616700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603575a3580 msgr2=0x5603575ac130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:30.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.666+0000 7f8b99d9b640 1 --2- 172.21.15.86:0/247616700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603575a3580 0x5603575ac130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.671 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.666+0000 7f8b99d9b640 1 -- 172.21.15.86:0/247616700 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5603575a3cd0 con 0x5603575a3ea0 2024-04-21T08:47:30.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.666+0000 7f8b99d9b640 1 --2- 172.21.15.86:0/247616700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603575a3ea0 0x5603575a8940 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f8b840ba790 tx=0x7f8b840be1c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f528ada0aebbd82 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:30.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.667+0000 7f8b83fff640 1 -- 172.21.15.86:0/247616700 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b840be330 con 0x5603575a3ea0 2024-04-21T08:47:30.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.667+0000 7f8b83fff640 1 -- 172.21.15.86:0/247616700 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b840c26e0 con 0x5603575a3ea0 2024-04-21T08:47:30.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.667+0000 7f8b83fff640 1 -- 172.21.15.86:0/247616700 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b840efc50 con 0x5603575a3ea0 2024-04-21T08:47:30.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.667+0000 7f8ba3434740 1 -- 172.21.15.86:0/247616700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603575a3ea0 msgr2=0x5603575a8940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:30.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.667+0000 7f8ba3434740 1 --2- 172.21.15.86:0/247616700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603575a3ea0 0x5603575a8940 secure :-1 s=READY pgs=1631 cs=0 l=1 rev1=1 crypto rx=0x7f8b840ba790 tx=0x7f8b840be1c0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.667+0000 7f8ba3434740 1 -- 172.21.15.86:0/247616700 shutdown_connections 2024-04-21T08:47:30.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.667+0000 7f8ba3434740 1 --2- 172.21.15.86:0/247616700 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603575aca80 0x5603575b8760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.667+0000 7f8ba3434740 1 --2- 172.21.15.86:0/247616700 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603575a3ea0 0x5603575a8940 unknown :-1 s=CLOSED pgs=1631 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.667+0000 7f8ba3434740 1 --2- 172.21.15.86:0/247616700 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603575a3580 0x5603575ac130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.667+0000 7f8ba3434740 1 -- 172.21.15.86:0/247616700 >> 172.21.15.86:0/247616700 conn(0x56035759dcb0 msgr2=0x56035759f970 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:30.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.667+0000 7f8ba3434740 1 -- 172.21.15.86:0/247616700 shutdown_connections 2024-04-21T08:47:30.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.667+0000 7f8ba3434740 1 -- 172.21.15.86:0/247616700 wait complete. 2024-04-21T08:47:30.672 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.667+0000 7f8ba3434740 1 -- start start 2024-04-21T08:47:30.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.668+0000 7f8ba3434740 1 Processor -- start 2024-04-21T08:47:30.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.668+0000 7f8ba3434740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603575a3580 0x5603575a7360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:30.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.668+0000 7f8ba3434740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603575a3ea0 0x5603575a5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:30.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.668+0000 7f8ba3434740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603575aca80 0x5603575a61b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:30.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.668+0000 7f8ba3434740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9c1b06f0 con 0x5603575a3580 2024-04-21T08:47:30.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.668+0000 7f8ba3434740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9c1b0020 con 0x5603575a3ea0 2024-04-21T08:47:30.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.668+0000 7f8ba3434740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8b9c1ad280 con 0x5603575aca80 2024-04-21T08:47:30.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.668+0000 7f8ba3434740 10 client.0 _ll_register_callbacks cb 0x7f8b94009730 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:30.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.668+0000 7f8b9a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603575a3580 0x5603575a7360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:30.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.668+0000 7f8b9a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603575a3580 0x5603575a7360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52514/0 (socket says 172.21.15.86:52514) 2024-04-21T08:47:30.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.668+0000 7f8b9a59c640 1 -- 172.21.15.86:0/3974431643 learned_addr learned my addr 172.21.15.86:0/3974431643 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:30.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.668+0000 7f8b99d9b640 1 --2- 172.21.15.86:0/3974431643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603575a3ea0 0x5603575a5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:30.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.668+0000 7f8b9a59c640 1 -- 172.21.15.86:0/3974431643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603575aca80 msgr2=0x5603575a61b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:30.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.668+0000 7f8b9a59c640 1 --2- 172.21.15.86:0/3974431643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603575aca80 0x5603575a61b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.668+0000 7f8b9a59c640 1 -- 172.21.15.86:0/3974431643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603575a3ea0 msgr2=0x5603575a5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:30.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.668+0000 7f8b9a59c640 1 --2- 172.21.15.86:0/3974431643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603575a3ea0 0x5603575a5c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.668+0000 7f8b9a59c640 1 -- 172.21.15.86:0/3974431643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5603575a6770 con 0x5603575a3580 2024-04-21T08:47:30.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.668+0000 7f8b9a59c640 1 --2- 172.21.15.86:0/3974431643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603575a3580 0x5603575a7360 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c01add0 tx=0x7f8b8c00bee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3cdaf499a29fcf5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:30.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.668+0000 7f8b81ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:47:30.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.668+0000 7f8b81ffb640 1 -- 172.21.15.86:0/3974431643 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8b8c032060 con 0x5603575a3580 2024-04-21T08:47:30.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.668+0000 7f8b81ffb640 1 -- 172.21.15.86:0/3974431643 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f8b8c01a670 con 0x5603575a3580 2024-04-21T08:47:30.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.668+0000 7f8ba3434740 1 -- 172.21.15.86:0/3974431643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8b840c26e0 con 0x5603575a3580 2024-04-21T08:47:30.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.669+0000 7f8b81ffb640 1 -- 172.21.15.86:0/3974431643 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8b8c039050 con 0x5603575a3580 2024-04-21T08:47:30.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.669+0000 7f8ba3434740 10 client.19104 fetch_fsmap learned FSMap version 29 2024-04-21T08:47:30.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.669+0000 7f8ba3434740 1 -- 172.21.15.86:0/3974431643 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560357627cf0 con 0x5603575a3580 2024-04-21T08:47:30.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.669+0000 7f8b81ffb640 1 -- 172.21.15.86:0/3974431643 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8b840c26e0 con 0x5603575a3580 2024-04-21T08:47:30.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.669+0000 7f8b81ffb640 1 -- 172.21.15.86:0/3974431643 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8b8c02c5c0 con 0x5603575a3580 2024-04-21T08:47:30.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.669+0000 7f8ba3434740 10 client.19104 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:47:30.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.669+0000 7f8ba3434740 10 client.19104 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:30.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.669+0000 7f8ba3434740 20 client.19104 populate_metadata read hostname 'smithi086' 2024-04-21T08:47:30.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.669+0000 7f8ba3434740 1 --2- 172.21.15.86:0/3974431643 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5603576281c0 0x56035762a600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:30.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.669+0000 7f8ba3434740 4 client.19104 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:47:30.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.669+0000 7f8ba3434740 1 -- 172.21.15.86:0/3974431643 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x56035750fa70 con 0x5603576281c0 2024-04-21T08:47:30.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.669+0000 7f8b99d9b640 1 --2- 172.21.15.86:0/3974431643 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5603576281c0 0x56035762a600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:30.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.670+0000 7f8b81ffb640 10 client.19104 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:30.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.670+0000 7f8b99d9b640 1 --2- 172.21.15.86:0/3974431643 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5603576281c0 0x56035762a600 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2673cd6bdf980e50 server_cookie=371f958c98a8dbf4 in_seq=0 out_seq=0 2024-04-21T08:47:30.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.670+0000 7f8b81ffb640 1 -- 172.21.15.86:0/3974431643 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x56035750fa70 con 0x5603576281c0 2024-04-21T08:47:30.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.670+0000 7f8b81ffb640 10 client.19104 handle_command_reply: tid=0 2024-04-21T08:47:30.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.670+0000 7f8ba3434740 1 client.19104 shutdown 2024-04-21T08:47:30.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.670+0000 7f8ba3434740 1 -- 172.21.15.86:0/3974431643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603575a3580 msgr2=0x5603575a7360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:30.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.670+0000 7f8ba3434740 1 --2- 172.21.15.86:0/3974431643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603575a3580 0x5603575a7360 secure :-1 s=READY pgs=1632 cs=0 l=1 rev1=1 crypto rx=0x7f8b8c01add0 tx=0x7f8b8c00bee0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.670+0000 7f8ba3434740 1 -- 172.21.15.86:0/3974431643 shutdown_connections 2024-04-21T08:47:30.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.670+0000 7f8ba3434740 1 --2- 172.21.15.86:0/3974431643 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5603576281c0 0x56035762a600 crc :-1 s=READY pgs=258 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.670+0000 7f8ba3434740 1 --2- 172.21.15.86:0/3974431643 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5603575aca80 0x5603575a61b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.670+0000 7f8b81ffb640 0 client.19104 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:30.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.670+0000 7f8ba3434740 1 --2- 172.21.15.86:0/3974431643 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5603575a3ea0 0x5603575a5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.670+0000 7f8ba3434740 1 --2- 172.21.15.86:0/3974431643 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5603575a3580 0x5603575a7360 unknown :-1 s=CLOSED pgs=1632 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.670+0000 7f8ba3434740 1 -- 172.21.15.86:0/3974431643 >> 172.21.15.86:0/3974431643 conn(0x56035759dcb0 msgr2=0x5603575a99c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:30.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.670+0000 7f8ba3434740 1 -- 172.21.15.86:0/3974431643 shutdown_connections 2024-04-21T08:47:30.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.670+0000 7f8ba3434740 1 -- 172.21.15.86:0/3974431643 wait complete. 2024-04-21T08:47:30.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.670+0000 7f8ba3434740 20 client.19104 trim_cache size 0 max 16384 2024-04-21T08:47:30.676 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:47:30.676 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-21T08:47:30.676 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-21T08:47:30.676 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:47:30.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.671+0000 7f8ba0843640 1 -- 172.21.15.86:0/1583372754 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8b880a3d20 msgr2=0x7f8b880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:30.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.671+0000 7f8ba0843640 1 --2- 172.21.15.86:0/1583372754 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8b880a3d20 0x7f8b880a61e0 secure :-1 s=READY pgs=265 cs=0 l=1 rev1=1 crypto rx=0x7f8b94005fd0 tx=0x7f8b94014970 comp rx=0 tx=0).stop 2024-04-21T08:47:30.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.671+0000 7f8ba0843640 1 -- 172.21.15.86:0/1583372754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b9c06af70 msgr2=0x7f8b9c198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:30.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.671+0000 7f8ba0843640 1 --2- 172.21.15.86:0/1583372754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b9c06af70 0x7f8b9c198c10 secure :-1 s=READY pgs=1628 cs=0 l=1 rev1=1 crypto rx=0x7f8b84006010 tx=0x7f8b84004040 comp rx=0 tx=0).stop 2024-04-21T08:47:30.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.671+0000 7f8ba0843640 1 -- 172.21.15.86:0/1583372754 shutdown_connections 2024-04-21T08:47:30.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.671+0000 7f8ba0843640 1 --2- 172.21.15.86:0/1583372754 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8b880a3d20 0x7f8b880a61e0 unknown :-1 s=CLOSED pgs=265 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.671+0000 7f8ba0843640 1 --2- 172.21.15.86:0/1583372754 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8b9c106f20 0x7f8b9c19cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.671+0000 7f8ba0843640 1 --2- 172.21.15.86:0/1583372754 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8b9c06af70 0x7f8b9c198c10 unknown :-1 s=CLOSED pgs=1628 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.671+0000 7f8ba0843640 1 --2- 172.21.15.86:0/1583372754 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8b9c06a5b0 0x7f8b9c1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.671+0000 7f8ba0843640 1 -- 172.21.15.86:0/1583372754 >> 172.21.15.86:0/1583372754 conn(0x7f8b9c077c40 msgr2=0x7f8b9c0fedb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:30.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.671+0000 7f8ba0843640 1 -- 172.21.15.86:0/1583372754 shutdown_connections 2024-04-21T08:47:30.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.671+0000 7f8ba0843640 1 -- 172.21.15.86:0/1583372754 wait complete. 2024-04-21T08:47:30.681 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:47:30.681 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:47:30.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.780+0000 7fa3e59f4640 1 -- 172.21.15.86:0/4043346967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3e006a8b0 msgr2=0x7fa3e006ac90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:30.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.780+0000 7fa3e59f4640 1 --2- 172.21.15.86:0/4043346967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3e006a8b0 0x7fa3e006ac90 secure :-1 s=READY pgs=1633 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0009900 tx=0x7fa3d002bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:30.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.780+0000 7fa3e59f4640 1 -- 172.21.15.86:0/4043346967 shutdown_connections 2024-04-21T08:47:30.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.780+0000 7fa3e59f4640 1 --2- 172.21.15.86:0/4043346967 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3e0107340 0x7fa3e010eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.780+0000 7fa3e59f4640 1 --2- 172.21.15.86:0/4043346967 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3e006b260 0x7fa3e0106c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.780+0000 7fa3e59f4640 1 --2- 172.21.15.86:0/4043346967 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3e006a8b0 0x7fa3e006ac90 unknown :-1 s=CLOSED pgs=1633 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.780+0000 7fa3e59f4640 1 -- 172.21.15.86:0/4043346967 >> 172.21.15.86:0/4043346967 conn(0x7fa3e0077e30 msgr2=0x7fa3e0078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:30.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.780+0000 7fa3e59f4640 1 -- 172.21.15.86:0/4043346967 shutdown_connections 2024-04-21T08:47:30.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.780+0000 7fa3e59f4640 1 -- 172.21.15.86:0/4043346967 wait complete. 2024-04-21T08:47:30.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.781+0000 7fa3e59f4640 1 Processor -- start 2024-04-21T08:47:30.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.781+0000 7fa3e59f4640 1 -- start start 2024-04-21T08:47:30.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.781+0000 7fa3e59f4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3e006a8b0 0x7fa3e019e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:30.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.781+0000 7fa3e59f4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3e0107340 0x7fa3e019e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:30.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.781+0000 7fa3e59f4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3e01985d0 0x7fa3e0198a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:30.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.781+0000 7fa3e59f4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e0112230 con 0x7fa3e01985d0 2024-04-21T08:47:30.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.781+0000 7fa3e59f4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e01120b0 con 0x7fa3e0107340 2024-04-21T08:47:30.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.781+0000 7fa3e59f4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa3e01123b0 con 0x7fa3e006a8b0 2024-04-21T08:47:30.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.781+0000 7fa3df7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3e01985d0 0x7fa3e0198a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:30.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.781+0000 7fa3df7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3e01985d0 0x7fa3e0198a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52522/0 (socket says 172.21.15.86:52522) 2024-04-21T08:47:30.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.781+0000 7fa3df7fe640 1 -- 172.21.15.86:0/3653949454 learned_addr learned my addr 172.21.15.86:0/3653949454 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:30.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.781+0000 7fa3de7fc640 1 --2- 172.21.15.86:0/3653949454 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3e0107340 0x7fa3e019e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:30.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.781+0000 7fa3deffd640 1 --2- 172.21.15.86:0/3653949454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3e006a8b0 0x7fa3e019e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:30.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.781+0000 7fa3df7fe640 1 -- 172.21.15.86:0/3653949454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3e006a8b0 msgr2=0x7fa3e019e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:30.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.781+0000 7fa3df7fe640 1 --2- 172.21.15.86:0/3653949454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3e006a8b0 0x7fa3e019e420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.781+0000 7fa3df7fe640 1 -- 172.21.15.86:0/3653949454 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3e0107340 msgr2=0x7fa3e019e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:30.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.781+0000 7fa3df7fe640 1 --2- 172.21.15.86:0/3653949454 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3e0107340 0x7fa3e019e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:30.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.781+0000 7fa3df7fe640 1 -- 172.21.15.86:0/3653949454 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa3e01990f0 con 0x7fa3e01985d0 2024-04-21T08:47:30.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.782+0000 7fa3df7fe640 1 --2- 172.21.15.86:0/3653949454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3e01985d0 0x7fa3e0198a80 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fa3d800eb10 tx=0x7fa3d800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:30.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.782+0000 7fa3e49f2640 1 -- 172.21.15.86:0/3653949454 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d8004b20 con 0x7fa3e01985d0 2024-04-21T08:47:30.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.782+0000 7fa3e59f4640 1 -- 172.21.15.86:0/3653949454 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa3e01993e0 con 0x7fa3e01985d0 2024-04-21T08:47:30.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.782+0000 7fa3e49f2640 1 -- 172.21.15.86:0/3653949454 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa3d8004cc0 con 0x7fa3e01985d0 2024-04-21T08:47:30.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.782+0000 7fa3e49f2640 1 -- 172.21.15.86:0/3653949454 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa3d8010670 con 0x7fa3e01985d0 2024-04-21T08:47:30.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.782+0000 7fa3e59f4640 1 -- 172.21.15.86:0/3653949454 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa3e01a55e0 con 0x7fa3e01985d0 2024-04-21T08:47:30.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.783+0000 7fa3e59f4640 1 -- 172.21.15.86:0/3653949454 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa3e0199b00 con 0x7fa3e01985d0 2024-04-21T08:47:30.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.784+0000 7fa3e49f2640 1 -- 172.21.15.86:0/3653949454 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa3d8005020 con 0x7fa3e01985d0 2024-04-21T08:47:30.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.784+0000 7fa3e49f2640 1 --2- 172.21.15.86:0/3653949454 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa3c80a3d20 0x7fa3c80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:30.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.784+0000 7fa3deffd640 1 --2- 172.21.15.86:0/3653949454 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa3c80a3d20 0x7fa3c80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:30.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.785+0000 7fa3e49f2640 1 -- 172.21.15.86:0/3653949454 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fa3d80d1180 con 0x7fa3e01985d0 2024-04-21T08:47:30.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.785+0000 7fa3deffd640 1 --2- 172.21.15.86:0/3653949454 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa3c80a3d20 0x7fa3c80a61e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0036040 tx=0x7fa3d0000bc0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:30.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:30.789+0000 7fa3e49f2640 1 -- 172.21.15.86:0/3653949454 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa3d809d450 con 0x7fa3e01985d0 2024-04-21T08:47:31.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.005+0000 7fa3e59f4640 1 -- 172.21.15.86:0/3653949454 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fa3e010a460 con 0x7fa3e01985d0 2024-04-21T08:47:31.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.006+0000 7fa3e49f2640 1 -- 172.21.15.86:0/3653949454 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7fa3d809cd50 con 0x7fa3e01985d0 2024-04-21T08:47:31.006 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:47:31.006 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:38:45.202126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24398,24394,24455,24472]},"id":1}]} 2024-04-21T08:47:31.006 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-21T08:47:31.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.008+0000 7fa3e59f4640 1 -- 172.21.15.86:0/3653949454 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa3c80a3d20 msgr2=0x7fa3c80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.008+0000 7fa3e59f4640 1 --2- 172.21.15.86:0/3653949454 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa3c80a3d20 0x7fa3c80a61e0 secure :-1 s=READY pgs=266 cs=0 l=1 rev1=1 crypto rx=0x7fa3d0036040 tx=0x7fa3d0000bc0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.008+0000 7fa3e59f4640 1 -- 172.21.15.86:0/3653949454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3e01985d0 msgr2=0x7fa3e0198a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.008+0000 7fa3e59f4640 1 --2- 172.21.15.86:0/3653949454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3e01985d0 0x7fa3e0198a80 secure :-1 s=READY pgs=1634 cs=0 l=1 rev1=1 crypto rx=0x7fa3d800eb10 tx=0x7fa3d800c2d0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.008+0000 7fa3e59f4640 1 -- 172.21.15.86:0/3653949454 shutdown_connections 2024-04-21T08:47:31.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.008+0000 7fa3e59f4640 1 --2- 172.21.15.86:0/3653949454 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa3c80a3d20 0x7fa3c80a61e0 unknown :-1 s=CLOSED pgs=266 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.008+0000 7fa3e59f4640 1 --2- 172.21.15.86:0/3653949454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa3e01985d0 0x7fa3e0198a80 unknown :-1 s=CLOSED pgs=1634 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.008+0000 7fa3e59f4640 1 --2- 172.21.15.86:0/3653949454 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa3e0107340 0x7fa3e019e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.008+0000 7fa3e59f4640 1 --2- 172.21.15.86:0/3653949454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa3e006a8b0 0x7fa3e019e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.008+0000 7fa3e59f4640 1 -- 172.21.15.86:0/3653949454 >> 172.21.15.86:0/3653949454 conn(0x7fa3e0077e30 msgr2=0x7fa3e00ff5e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:31.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.008+0000 7fa3e59f4640 1 -- 172.21.15.86:0/3653949454 shutdown_connections 2024-04-21T08:47:31.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.008+0000 7fa3e59f4640 1 -- 172.21.15.86:0/3653949454 wait complete. 2024-04-21T08:47:31.018 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:47:31.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.119+0000 7f8a977e6640 1 -- 172.21.15.86:0/507004596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a90102190 msgr2=0x7f8a90102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.119+0000 7f8a977e6640 1 --2- 172.21.15.86:0/507004596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a90102190 0x7f8a90102570 secure :-1 s=READY pgs=1635 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c009ca0 tx=0x7f8a8c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.119+0000 7f8a977e6640 1 -- 172.21.15.86:0/507004596 shutdown_connections 2024-04-21T08:47:31.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.119+0000 7f8a977e6640 1 --2- 172.21.15.86:0/507004596 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a90107260 0x7f8a9010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.119+0000 7f8a977e6640 1 --2- 172.21.15.86:0/507004596 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a90102b40 0x7f8a90106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.119+0000 7f8a977e6640 1 --2- 172.21.15.86:0/507004596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a90102190 0x7f8a90102570 unknown :-1 s=CLOSED pgs=1635 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.119+0000 7f8a977e6640 1 -- 172.21.15.86:0/507004596 >> 172.21.15.86:0/507004596 conn(0x7f8a900fd990 msgr2=0x7f8a900ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:31.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.119+0000 7f8a977e6640 1 -- 172.21.15.86:0/507004596 shutdown_connections 2024-04-21T08:47:31.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.119+0000 7f8a977e6640 1 -- 172.21.15.86:0/507004596 wait complete. 2024-04-21T08:47:31.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.120+0000 7f8a977e6640 1 Processor -- start 2024-04-21T08:47:31.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.120+0000 7f8a977e6640 1 -- start start 2024-04-21T08:47:31.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.120+0000 7f8a977e6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a90102190 0x7f8a9019e410 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.120+0000 7f8a977e6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a90107260 0x7f8a9019e950 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.120+0000 7f8a977e6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a901985c0 0x7f8a90198a70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.120+0000 7f8a977e6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8a901121c0 con 0x7f8a901985c0 2024-04-21T08:47:31.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.120+0000 7f8a977e6640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8a90112040 con 0x7f8a90107260 2024-04-21T08:47:31.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.120+0000 7f8a977e6640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8a90112340 con 0x7f8a90102190 2024-04-21T08:47:31.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.120+0000 7f8a95d83640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a901985c0 0x7f8a90198a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.120+0000 7f8a95d83640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a901985c0 0x7f8a90198a70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52544/0 (socket says 172.21.15.86:52544) 2024-04-21T08:47:31.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.120+0000 7f8a95d83640 1 -- 172.21.15.86:0/1821338196 learned_addr learned my addr 172.21.15.86:0/1821338196 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:31.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.120+0000 7f8a94d81640 1 --2- 172.21.15.86:0/1821338196 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a90107260 0x7f8a9019e950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.120+0000 7f8a95582640 1 --2- 172.21.15.86:0/1821338196 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a90102190 0x7f8a9019e410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.120+0000 7f8a95d83640 1 -- 172.21.15.86:0/1821338196 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a90102190 msgr2=0x7f8a9019e410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.120+0000 7f8a95d83640 1 --2- 172.21.15.86:0/1821338196 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a90102190 0x7f8a9019e410 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.120+0000 7f8a95d83640 1 -- 172.21.15.86:0/1821338196 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a90107260 msgr2=0x7f8a9019e950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.120+0000 7f8a95d83640 1 --2- 172.21.15.86:0/1821338196 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a90107260 0x7f8a9019e950 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.120+0000 7f8a95d83640 1 -- 172.21.15.86:0/1821338196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8a901990e0 con 0x7f8a901985c0 2024-04-21T08:47:31.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.120+0000 7f8a95582640 1 --2- 172.21.15.86:0/1821338196 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a90102190 0x7f8a9019e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:31.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.120+0000 7f8a95d83640 1 --2- 172.21.15.86:0/1821338196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a901985c0 0x7f8a90198a70 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f8a8800c970 tx=0x7f8a8800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:31.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.120+0000 7f8a867fc640 1 -- 172.21.15.86:0/1821338196 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a88007bb0 con 0x7f8a901985c0 2024-04-21T08:47:31.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.120+0000 7f8a977e6640 1 -- 172.21.15.86:0/1821338196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8a901993d0 con 0x7f8a901985c0 2024-04-21T08:47:31.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.120+0000 7f8a867fc640 1 -- 172.21.15.86:0/1821338196 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a88007d50 con 0x7f8a901985c0 2024-04-21T08:47:31.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.121+0000 7f8a867fc640 1 -- 172.21.15.86:0/1821338196 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a88004e60 con 0x7f8a901985c0 2024-04-21T08:47:31.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.121+0000 7f8a977e6640 1 -- 172.21.15.86:0/1821338196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8a901a55d0 con 0x7f8a901985c0 2024-04-21T08:47:31.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.123+0000 7f8a867fc640 1 -- 172.21.15.86:0/1821338196 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8a88005020 con 0x7f8a901985c0 2024-04-21T08:47:31.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.123+0000 7f8a9a3d4740 1 Processor -- start 2024-04-21T08:47:31.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.123+0000 7f8a9a3d4740 1 -- start start 2024-04-21T08:47:31.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.123+0000 7f8a9a3d4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5651e313f4d0 0x5651e313f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.123+0000 7f8a9a3d4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651e313fe80 0x5651e3148430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.123+0000 7f8a9a3d4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651e3148a00 0x5651e314ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.123+0000 7f8a867fc640 1 --2- 172.21.15.86:0/1821338196 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8a7c0a3d20 0x7f8a7c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.123+0000 7f8a9a3d4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5651e30a41f0 con 0x5651e3148a00 2024-04-21T08:47:31.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.123+0000 7f8a9a3d4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5651e30414b0 con 0x5651e313f4d0 2024-04-21T08:47:31.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.123+0000 7f8a9a3d4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5651e30a01b0 con 0x5651e313fe80 2024-04-21T08:47:31.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.124+0000 7f8a95d83640 1 --2- 172.21.15.86:0/1821338196 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8a7c0a3d20 0x7f8a7c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.124+0000 7f8a94d81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651e3148a00 0x5651e314ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.124+0000 7f8a94d81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651e3148a00 0x5651e314ae50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52546/0 (socket says 172.21.15.86:52546) 2024-04-21T08:47:31.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.124+0000 7f8a94d81640 1 -- 172.21.15.86:0/2443258413 learned_addr learned my addr 172.21.15.86:0/2443258413 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.124+0000 7f8a94d81640 1 --2- 172.21.15.86:0/2443258413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5651e313f4d0 0x5651e313f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.124+0000 7f8a867fc640 1 -- 172.21.15.86:0/1821338196 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f8a880d0800 con 0x7f8a901985c0 2024-04-21T08:47:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.124+0000 7f8a95582640 1 --2- 172.21.15.86:0/2443258413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651e313fe80 0x5651e3148430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.124+0000 7f8a94d81640 1 -- 172.21.15.86:0/2443258413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651e313fe80 msgr2=0x5651e3148430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.124+0000 7f8a94d81640 1 --2- 172.21.15.86:0/2443258413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651e313fe80 0x5651e3148430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.124+0000 7f8a94d81640 1 -- 172.21.15.86:0/2443258413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5651e313f4d0 msgr2=0x5651e313f8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.124+0000 7f8a94d81640 1 --2- 172.21.15.86:0/2443258413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5651e313f4d0 0x5651e313f8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.124+0000 7f8a94d81640 1 -- 172.21.15.86:0/2443258413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651e30331b0 con 0x5651e3148a00 2024-04-21T08:47:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.124+0000 7f8a95d83640 1 --2- 172.21.15.86:0/1821338196 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8a7c0a3d20 0x7f8a7c0a61e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8a880cdc30 tx=0x7f8a8808e040 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.124+0000 7f8a94d81640 1 --2- 172.21.15.86:0/2443258413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651e3148a00 0x5651e314ae50 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f8a80005d00 tx=0x7f8a80005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=240c3b812dfcde90 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.124+0000 7f8a7b7fe640 1 -- 172.21.15.86:0/2443258413 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a80012ca0 con 0x5651e3148a00 2024-04-21T08:47:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.124+0000 7f8a7b7fe640 1 -- 172.21.15.86:0/2443258413 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a80012e40 con 0x5651e3148a00 2024-04-21T08:47:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.124+0000 7f8a7b7fe640 1 -- 172.21.15.86:0/2443258413 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a800049e0 con 0x5651e3148a00 2024-04-21T08:47:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.125+0000 7f8a9a3d4740 1 -- 172.21.15.86:0/2443258413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651e3148a00 msgr2=0x5651e314ae50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.125+0000 7f8a9a3d4740 1 --2- 172.21.15.86:0/2443258413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651e3148a00 0x5651e314ae50 secure :-1 s=READY pgs=1637 cs=0 l=1 rev1=1 crypto rx=0x7f8a80005d00 tx=0x7f8a80005c90 comp rx=0 tx=0).stop 2024-04-21T08:47:31.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.125+0000 7f8a9a3d4740 1 -- 172.21.15.86:0/2443258413 shutdown_connections 2024-04-21T08:47:31.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.125+0000 7f8a9a3d4740 1 --2- 172.21.15.86:0/2443258413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651e3148a00 0x5651e314ae50 unknown :-1 s=CLOSED pgs=1637 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.125+0000 7f8a9a3d4740 1 --2- 172.21.15.86:0/2443258413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651e313fe80 0x5651e3148430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.125+0000 7f8a9a3d4740 1 --2- 172.21.15.86:0/2443258413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5651e313f4d0 0x5651e313f8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.125+0000 7f8a9a3d4740 1 -- 172.21.15.86:0/2443258413 >> 172.21.15.86:0/2443258413 conn(0x5651e3139ba0 msgr2=0x5651e3139f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:31.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.125+0000 7f8a9a3d4740 1 -- 172.21.15.86:0/2443258413 shutdown_connections 2024-04-21T08:47:31.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.125+0000 7f8a9a3d4740 1 -- 172.21.15.86:0/2443258413 wait complete. 2024-04-21T08:47:31.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.125+0000 7f8a9a3d4740 1 -- start start 2024-04-21T08:47:31.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.125+0000 7f8a9a3d4740 1 Processor -- start 2024-04-21T08:47:31.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.126+0000 7f8a9a3d4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651e313f4d0 0x5651e31482f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.126+0000 7f8a9a3d4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5651e313fe80 0x5651e3145680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.126+0000 7f8a9a3d4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651e3148a00 0x5651e3145c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.126+0000 7f8a9a3d4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5651e314b970 con 0x5651e313f4d0 2024-04-21T08:47:31.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.126+0000 7f8a9a3d4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5651e305d500 con 0x5651e313fe80 2024-04-21T08:47:31.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.126+0000 7f8a9a3d4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5651e314baf0 con 0x5651e3148a00 2024-04-21T08:47:31.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.126+0000 7f8a9a3d4740 10 client.0 _ll_register_callbacks cb 0x5651e302bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:31.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.126+0000 7f8a94d81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651e313f4d0 0x5651e31482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.126+0000 7f8a94d81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651e313f4d0 0x5651e31482f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52554/0 (socket says 172.21.15.86:52554) 2024-04-21T08:47:31.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.126+0000 7f8a94d81640 1 -- 172.21.15.86:0/3971513135 learned_addr learned my addr 172.21.15.86:0/3971513135 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:31.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.126+0000 7f8a94d81640 1 -- 172.21.15.86:0/3971513135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651e3148a00 msgr2=0x5651e3145c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.126+0000 7f8a95582640 1 --2- 172.21.15.86:0/3971513135 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5651e313fe80 0x5651e3145680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.126+0000 7f8a94d81640 1 --2- 172.21.15.86:0/3971513135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651e3148a00 0x5651e3145c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.126+0000 7f8a94d81640 1 -- 172.21.15.86:0/3971513135 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5651e313fe80 msgr2=0x5651e3145680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.126+0000 7f8a94d81640 1 --2- 172.21.15.86:0/3971513135 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5651e313fe80 0x5651e3145680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.126+0000 7f8a94d81640 1 -- 172.21.15.86:0/3971513135 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651e2db63f0 con 0x5651e313f4d0 2024-04-21T08:47:31.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.126+0000 7f8a95582640 1 --2- 172.21.15.86:0/3971513135 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5651e313fe80 0x5651e3145680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:31.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.126+0000 7f8a94d81640 1 --2- 172.21.15.86:0/3971513135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651e313f4d0 0x5651e31482f0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f8a80015040 tx=0x7f8a80012ca0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=392bccc2f071b74f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:31.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.126+0000 7f8a78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:47:31.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.126+0000 7f8a78ff9640 1 -- 172.21.15.86:0/3971513135 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a80010940 con 0x5651e313f4d0 2024-04-21T08:47:31.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.127+0000 7f8a78ff9640 1 -- 172.21.15.86:0/3971513135 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f8a80010ae0 con 0x5651e313f4d0 2024-04-21T08:47:31.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.127+0000 7f8a9a3d4740 1 -- 172.21.15.86:0/3971513135 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5651e3043930 con 0x5651e313f4d0 2024-04-21T08:47:31.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.127+0000 7f8a78ff9640 1 -- 172.21.15.86:0/3971513135 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8a80010ce0 con 0x5651e313f4d0 2024-04-21T08:47:31.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.127+0000 7f8a9a3d4740 10 client.19122 fetch_fsmap learned FSMap version 29 2024-04-21T08:47:31.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.127+0000 7f8a9a3d4740 1 -- 172.21.15.86:0/3971513135 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5651e2e807d0 con 0x5651e313f4d0 2024-04-21T08:47:31.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.127+0000 7f8a78ff9640 1 -- 172.21.15.86:0/3971513135 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5651e3043930 con 0x5651e313f4d0 2024-04-21T08:47:31.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.127+0000 7f8a78ff9640 1 -- 172.21.15.86:0/3971513135 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8a8001f6d0 con 0x5651e313f4d0 2024-04-21T08:47:31.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.127+0000 7f8a9a3d4740 10 client.19122 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:47:31.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.127+0000 7f8a9a3d4740 10 client.19122 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:47:31.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.127+0000 7f8a9a3d4740 20 client.19122 populate_metadata read hostname 'smithi086' 2024-04-21T08:47:31.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.127+0000 7f8a9a3d4740 1 --2- 172.21.15.86:0/3971513135 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x5651e3158a80 0x5651e315ae70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.127+0000 7f8a9a3d4740 4 client.19122 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:31.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.127+0000 7f8a9a3d4740 1 -- 172.21.15.86:0/3971513135 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5651e30b1c10 con 0x5651e3158a80 2024-04-21T08:47:31.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.127+0000 7f8a95582640 1 --2- 172.21.15.86:0/3971513135 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x5651e3158a80 0x5651e315ae70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.128+0000 7f8a95582640 1 --2- 172.21.15.86:0/3971513135 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x5651e3158a80 0x5651e315ae70 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=2c15e0026ff4b8be server_cookie=2a13ef175d1cd1c1 in_seq=0 out_seq=0 2024-04-21T08:47:31.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.128+0000 7f8a78ff9640 10 client.19122 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:47:31.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.128+0000 7f8a78ff9640 1 -- 172.21.15.86:0/3971513135 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5651e30b1c10 con 0x5651e3158a80 2024-04-21T08:47:31.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.128+0000 7f8a78ff9640 10 client.19122 handle_command_reply: tid=0 2024-04-21T08:47:31.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.129+0000 7f8a9a3d4740 1 client.19122 shutdown 2024-04-21T08:47:31.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.129+0000 7f8a9a3d4740 1 -- 172.21.15.86:0/3971513135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651e313f4d0 msgr2=0x5651e31482f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.129+0000 7f8a9a3d4740 1 --2- 172.21.15.86:0/3971513135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651e313f4d0 0x5651e31482f0 secure :-1 s=READY pgs=1638 cs=0 l=1 rev1=1 crypto rx=0x7f8a80015040 tx=0x7f8a80012ca0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.129+0000 7f8a9a3d4740 1 -- 172.21.15.86:0/3971513135 shutdown_connections 2024-04-21T08:47:31.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.129+0000 7f8a9a3d4740 1 --2- 172.21.15.86:0/3971513135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651e3148a00 0x5651e3145c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.129+0000 7f8a9a3d4740 1 --2- 172.21.15.86:0/3971513135 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x5651e3158a80 0x5651e315ae70 crc :-1 s=READY pgs=64 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.129+0000 7f8a9a3d4740 1 --2- 172.21.15.86:0/3971513135 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5651e313fe80 0x5651e3145680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.129+0000 7f8a9a3d4740 1 --2- 172.21.15.86:0/3971513135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651e313f4d0 0x5651e31482f0 unknown :-1 s=CLOSED pgs=1638 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.129+0000 7f8a78ff9640 0 client.19122 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:47:31.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.129+0000 7f8a9a3d4740 1 -- 172.21.15.86:0/3971513135 >> 172.21.15.86:0/3971513135 conn(0x5651e3139ba0 msgr2=0x5651e313b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:31.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.129+0000 7f8a9a3d4740 1 -- 172.21.15.86:0/3971513135 shutdown_connections 2024-04-21T08:47:31.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.129+0000 7f8a9a3d4740 1 -- 172.21.15.86:0/3971513135 wait complete. 2024-04-21T08:47:31.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.129+0000 7f8a9a3d4740 20 client.19122 trim_cache size 0 max 16384 2024-04-21T08:47:31.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.142+0000 7f8a9a3d4740 1 Processor -- start 2024-04-21T08:47:31.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.142+0000 7f8a9a3d4740 1 -- start start 2024-04-21T08:47:31.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.142+0000 7f8a9a3d4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5651e313f4d0 0x5651e3147db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.142+0000 7f8a9a3d4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651e313fe80 0x5651e31482f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.142+0000 7f8a9a3d4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651e3148a00 0x5651e31605b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.142+0000 7f8a9a3d4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8a901af450 con 0x5651e313fe80 2024-04-21T08:47:31.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.142+0000 7f8a9a3d4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8a901b0f90 con 0x5651e313f4d0 2024-04-21T08:47:31.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.142+0000 7f8a9a3d4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8a901afb20 con 0x5651e3148a00 2024-04-21T08:47:31.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.142+0000 7f8a95582640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651e313fe80 0x5651e31482f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.142+0000 7f8a95582640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651e313fe80 0x5651e31482f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52568/0 (socket says 172.21.15.86:52568) 2024-04-21T08:47:31.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.142+0000 7f8a95582640 1 -- 172.21.15.86:0/3568737623 learned_addr learned my addr 172.21.15.86:0/3568737623 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:31.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.142+0000 7f8a94d81640 1 --2- 172.21.15.86:0/3568737623 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5651e313f4d0 0x5651e3147db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.143+0000 7f8a94d81640 1 --2- 172.21.15.86:0/3568737623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651e3148a00 0x5651e31605b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.143+0000 7f8a95582640 1 -- 172.21.15.86:0/3568737623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651e3148a00 msgr2=0x5651e31605b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.143+0000 7f8a95582640 1 --2- 172.21.15.86:0/3568737623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651e3148a00 0x5651e31605b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.143+0000 7f8a95582640 1 -- 172.21.15.86:0/3568737623 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5651e313f4d0 msgr2=0x5651e3147db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.143+0000 7f8a95582640 1 --2- 172.21.15.86:0/3568737623 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5651e313f4d0 0x5651e3147db0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.143+0000 7f8a95582640 1 -- 172.21.15.86:0/3568737623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5651e313fc20 con 0x5651e313fe80 2024-04-21T08:47:31.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.143+0000 7f8a95582640 1 --2- 172.21.15.86:0/3568737623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651e313fe80 0x5651e31482f0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c02c6d0 tx=0x7f8a8c0323f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=30a3780e35b0933c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:31.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.143+0000 7f8a7affd640 1 -- 172.21.15.86:0/3568737623 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c039040 con 0x5651e313fe80 2024-04-21T08:47:31.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.143+0000 7f8a7affd640 1 -- 172.21.15.86:0/3568737623 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a8c032970 con 0x5651e313fe80 2024-04-21T08:47:31.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.143+0000 7f8a7affd640 1 -- 172.21.15.86:0/3568737623 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c0044d0 con 0x5651e313fe80 2024-04-21T08:47:31.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.143+0000 7f8a9a3d4740 1 -- 172.21.15.86:0/3568737623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651e313fe80 msgr2=0x5651e31482f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.143+0000 7f8a9a3d4740 1 --2- 172.21.15.86:0/3568737623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651e313fe80 0x5651e31482f0 secure :-1 s=READY pgs=1639 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c02c6d0 tx=0x7f8a8c0323f0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.143+0000 7f8a9a3d4740 1 -- 172.21.15.86:0/3568737623 shutdown_connections 2024-04-21T08:47:31.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.143+0000 7f8a9a3d4740 1 --2- 172.21.15.86:0/3568737623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651e3148a00 0x5651e31605b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.143+0000 7f8a9a3d4740 1 --2- 172.21.15.86:0/3568737623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651e313fe80 0x5651e31482f0 unknown :-1 s=CLOSED pgs=1639 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.143+0000 7f8a9a3d4740 1 --2- 172.21.15.86:0/3568737623 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5651e313f4d0 0x5651e3147db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.143+0000 7f8a9a3d4740 1 -- 172.21.15.86:0/3568737623 >> 172.21.15.86:0/3568737623 conn(0x5651e3139ba0 msgr2=0x5651e3140af0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:31.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.144+0000 7f8a9a3d4740 1 -- 172.21.15.86:0/3568737623 shutdown_connections 2024-04-21T08:47:31.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.144+0000 7f8a9a3d4740 1 -- 172.21.15.86:0/3568737623 wait complete. 2024-04-21T08:47:31.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.144+0000 7f8a9a3d4740 1 -- start start 2024-04-21T08:47:31.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.144+0000 7f8a9a3d4740 1 Processor -- start 2024-04-21T08:47:31.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.144+0000 7f8a9a3d4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651e313f4d0 0x5651e314bd00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.144+0000 7f8a9a3d4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651e313fe80 0x5651e314c240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.144+0000 7f8a9a3d4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5651e3148a00 0x5651e314c780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.144+0000 7f8a9a3d4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8a901b08c0 con 0x5651e313fe80 2024-04-21T08:47:31.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.144+0000 7f8a9a3d4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8a901b01f0 con 0x5651e3148a00 2024-04-21T08:47:31.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.144+0000 7f8a9a3d4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8a901ad450 con 0x5651e313f4d0 2024-04-21T08:47:31.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.144+0000 7f8a9a3d4740 10 client.0 _ll_register_callbacks cb 0x7f8a8000b390 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:31.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.144+0000 7f8a95582640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651e313fe80 0x5651e314c240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.144+0000 7f8a95582640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651e313fe80 0x5651e314c240 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52574/0 (socket says 172.21.15.86:52574) 2024-04-21T08:47:31.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.144+0000 7f8a95582640 1 -- 172.21.15.86:0/262792288 learned_addr learned my addr 172.21.15.86:0/262792288 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:31.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.144+0000 7f8a94d81640 1 --2- 172.21.15.86:0/262792288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5651e3148a00 0x5651e314c780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.145+0000 7f8a94d81640 1 --2- 172.21.15.86:0/262792288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651e313f4d0 0x5651e314bd00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.145+0000 7f8a95582640 1 -- 172.21.15.86:0/262792288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651e313f4d0 msgr2=0x5651e314bd00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.145+0000 7f8a95582640 1 --2- 172.21.15.86:0/262792288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651e313f4d0 0x5651e314bd00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.145+0000 7f8a95582640 1 -- 172.21.15.86:0/262792288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5651e3148a00 msgr2=0x5651e314c780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.145+0000 7f8a95582640 1 --2- 172.21.15.86:0/262792288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5651e3148a00 0x5651e314c780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.145+0000 7f8a95582640 1 -- 172.21.15.86:0/262792288 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5651e31bce40 con 0x5651e313fe80 2024-04-21T08:47:31.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.145+0000 7f8a94d81640 1 --2- 172.21.15.86:0/262792288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651e313f4d0 0x5651e314bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:31.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.145+0000 7f8a95582640 1 --2- 172.21.15.86:0/262792288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651e313fe80 0x5651e314c240 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c009430 tx=0x7f8a8c032eb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b2f2eec20a690ed3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:31.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.145+0000 7f8a78ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:47:31.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.145+0000 7f8a78ff9640 1 -- 172.21.15.86:0/262792288 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8a8c039040 con 0x5651e313fe80 2024-04-21T08:47:31.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.145+0000 7f8a78ff9640 1 -- 172.21.15.86:0/262792288 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f8a8c003970 con 0x5651e313fe80 2024-04-21T08:47:31.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.145+0000 7f8a9a3d4740 1 -- 172.21.15.86:0/262792288 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8a8c032970 con 0x5651e313fe80 2024-04-21T08:47:31.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.145+0000 7f8a78ff9640 1 -- 172.21.15.86:0/262792288 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8a8c003b70 con 0x5651e313fe80 2024-04-21T08:47:31.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.145+0000 7f8a9a3d4740 10 client.19128 fetch_fsmap learned FSMap version 29 2024-04-21T08:47:31.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.145+0000 7f8a9a3d4740 1 -- 172.21.15.86:0/262792288 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5651e31be140 con 0x5651e313fe80 2024-04-21T08:47:31.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.145+0000 7f8a78ff9640 1 -- 172.21.15.86:0/262792288 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8a8c032970 con 0x5651e313fe80 2024-04-21T08:47:31.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.145+0000 7f8a78ff9640 1 -- 172.21.15.86:0/262792288 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8a8c05d030 con 0x5651e313fe80 2024-04-21T08:47:31.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.145+0000 7f8a9a3d4740 10 client.19128 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:47:31.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.145+0000 7f8a9a3d4740 10 client.19128 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:47:31.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.145+0000 7f8a9a3d4740 20 client.19128 populate_metadata read hostname 'smithi086' 2024-04-21T08:47:31.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.145+0000 7f8a9a3d4740 1 --2- 172.21.15.86:0/262792288 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x5651e31be8e0 0x5651e31c0d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.145+0000 7f8a9a3d4740 4 client.19128 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:47:31.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.145+0000 7f8a9a3d4740 1 -- 172.21.15.86:0/262792288 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5651e30aba80 con 0x5651e31be8e0 2024-04-21T08:47:31.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.146+0000 7f8a94d81640 1 --2- 172.21.15.86:0/262792288 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x5651e31be8e0 0x5651e31c0d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.146+0000 7f8a78ff9640 10 client.19128 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:47:31.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.146+0000 7f8a94d81640 1 --2- 172.21.15.86:0/262792288 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x5651e31be8e0 0x5651e31c0d00 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=ef032582b31e6686 server_cookie=49d8f6665eceff7a in_seq=0 out_seq=0 2024-04-21T08:47:31.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.146+0000 7f8a78ff9640 1 -- 172.21.15.86:0/262792288 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5651e30aba80 con 0x5651e31be8e0 2024-04-21T08:47:31.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.146+0000 7f8a78ff9640 10 client.19128 handle_command_reply: tid=0 2024-04-21T08:47:31.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.146+0000 7f8a9a3d4740 1 client.19128 shutdown 2024-04-21T08:47:31.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.146+0000 7f8a9a3d4740 1 -- 172.21.15.86:0/262792288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651e313fe80 msgr2=0x5651e314c240 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.146+0000 7f8a9a3d4740 1 --2- 172.21.15.86:0/262792288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651e313fe80 0x5651e314c240 secure :-1 s=READY pgs=1640 cs=0 l=1 rev1=1 crypto rx=0x7f8a8c009430 tx=0x7f8a8c032eb0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.146+0000 7f8a9a3d4740 1 -- 172.21.15.86:0/262792288 shutdown_connections 2024-04-21T08:47:31.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.146+0000 7f8a9a3d4740 1 --2- 172.21.15.86:0/262792288 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x5651e31be8e0 0x5651e31c0d00 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.146+0000 7f8a9a3d4740 1 --2- 172.21.15.86:0/262792288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5651e3148a00 0x5651e314c780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.146+0000 7f8a9a3d4740 1 --2- 172.21.15.86:0/262792288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5651e313fe80 0x5651e314c240 unknown :-1 s=CLOSED pgs=1640 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.146+0000 7f8a78ff9640 0 client.19128 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:47:31.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.146+0000 7f8a9a3d4740 1 --2- 172.21.15.86:0/262792288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5651e313f4d0 0x5651e314bd00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.146+0000 7f8a9a3d4740 1 -- 172.21.15.86:0/262792288 >> 172.21.15.86:0/262792288 conn(0x5651e3139ba0 msgr2=0x5651e313ae80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:31.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.146+0000 7f8a9a3d4740 1 -- 172.21.15.86:0/262792288 shutdown_connections 2024-04-21T08:47:31.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.146+0000 7f8a9a3d4740 1 -- 172.21.15.86:0/262792288 wait complete. 2024-04-21T08:47:31.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.147+0000 7f8a9a3d4740 20 client.19128 trim_cache size 0 max 16384 2024-04-21T08:47:31.153 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:47:31.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.147+0000 7f8a977e6640 1 -- 172.21.15.86:0/1821338196 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8a7c0a3d20 msgr2=0x7f8a7c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.147+0000 7f8a977e6640 1 --2- 172.21.15.86:0/1821338196 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8a7c0a3d20 0x7f8a7c0a61e0 secure :-1 s=READY pgs=267 cs=0 l=1 rev1=1 crypto rx=0x7f8a880cdc30 tx=0x7f8a8808e040 comp rx=0 tx=0).stop 2024-04-21T08:47:31.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.147+0000 7f8a977e6640 1 -- 172.21.15.86:0/1821338196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a901985c0 msgr2=0x7f8a90198a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.147+0000 7f8a977e6640 1 --2- 172.21.15.86:0/1821338196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a901985c0 0x7f8a90198a70 secure :-1 s=READY pgs=1636 cs=0 l=1 rev1=1 crypto rx=0x7f8a8800c970 tx=0x7f8a8800ce30 comp rx=0 tx=0).stop 2024-04-21T08:47:31.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.147+0000 7f8a977e6640 1 -- 172.21.15.86:0/1821338196 shutdown_connections 2024-04-21T08:47:31.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.147+0000 7f8a977e6640 1 --2- 172.21.15.86:0/1821338196 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8a7c0a3d20 0x7f8a7c0a61e0 unknown :-1 s=CLOSED pgs=267 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.147+0000 7f8a977e6640 1 --2- 172.21.15.86:0/1821338196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8a901985c0 0x7f8a90198a70 unknown :-1 s=CLOSED pgs=1636 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.147+0000 7f8a977e6640 1 --2- 172.21.15.86:0/1821338196 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8a90107260 0x7f8a9019e950 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.147+0000 7f8a977e6640 1 --2- 172.21.15.86:0/1821338196 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8a90102190 0x7f8a9019e410 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.147+0000 7f8a977e6640 1 -- 172.21.15.86:0/1821338196 >> 172.21.15.86:0/1821338196 conn(0x7f8a900fd990 msgr2=0x7f8a900fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:31.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.147+0000 7f8a977e6640 1 -- 172.21.15.86:0/1821338196 shutdown_connections 2024-04-21T08:47:31.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.147+0000 7f8a977e6640 1 -- 172.21.15.86:0/1821338196 wait complete. 2024-04-21T08:47:31.157 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:47:31.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.258+0000 7f3c14a43640 1 -- 172.21.15.86:0/1067652390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c10103af0 msgr2=0x7f3c1010bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.258+0000 7f3c14a43640 1 --2- 172.21.15.86:0/1067652390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c10103af0 0x7f3c1010bfd0 secure :-1 s=READY pgs=1641 cs=0 l=1 rev1=1 crypto rx=0x7f3c04009900 tx=0x7f3c0402bd60 comp rx=0 tx=0).stop 2024-04-21T08:47:31.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.258+0000 7f3c14a43640 1 -- 172.21.15.86:0/1067652390 shutdown_connections 2024-04-21T08:47:31.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.258+0000 7f3c14a43640 1 --2- 172.21.15.86:0/1067652390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3c1010c510 0x7f3c1010e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.258+0000 7f3c14a43640 1 --2- 172.21.15.86:0/1067652390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c10103af0 0x7f3c1010bfd0 unknown :-1 s=CLOSED pgs=1641 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.258+0000 7f3c14a43640 1 --2- 172.21.15.86:0/1067652390 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c101031d0 0x7f3c101035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.258+0000 7f3c14a43640 1 -- 172.21.15.86:0/1067652390 >> 172.21.15.86:0/1067652390 conn(0x7f3c10077f50 msgr2=0x7f3c10078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:31.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.258+0000 7f3c14a43640 1 -- 172.21.15.86:0/1067652390 shutdown_connections 2024-04-21T08:47:31.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.258+0000 7f3c14a43640 1 -- 172.21.15.86:0/1067652390 wait complete. 2024-04-21T08:47:31.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.258+0000 7f3c14a43640 1 Processor -- start 2024-04-21T08:47:31.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.258+0000 7f3c14a43640 1 -- start start 2024-04-21T08:47:31.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.259+0000 7f3c14a43640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c101031d0 0x7f3c10198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.259+0000 7f3c14a43640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3c10103af0 0x7f3c10198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.259+0000 7f3c14a43640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c1010c510 0x7f3c1019d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.259+0000 7f3c14a43640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3c10112180 con 0x7f3c101031d0 2024-04-21T08:47:31.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.259+0000 7f3c14a43640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3c10112000 con 0x7f3c1010c510 2024-04-21T08:47:31.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.259+0000 7f3c14a43640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3c10112300 con 0x7f3c10103af0 2024-04-21T08:47:31.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.259+0000 7f3c0e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c101031d0 0x7f3c10198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.259+0000 7f3c0e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c101031d0 0x7f3c10198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52598/0 (socket says 172.21.15.86:52598) 2024-04-21T08:47:31.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.259+0000 7f3c0e59c640 1 -- 172.21.15.86:0/3922569427 learned_addr learned my addr 172.21.15.86:0/3922569427 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:31.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.259+0000 7f3c0ed9d640 1 --2- 172.21.15.86:0/3922569427 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c1010c510 0x7f3c1019d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.259+0000 7f3c0e59c640 1 -- 172.21.15.86:0/3922569427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3c10103af0 msgr2=0x7f3c10198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.259+0000 7f3c0dd9b640 1 --2- 172.21.15.86:0/3922569427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3c10103af0 0x7f3c10198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.259+0000 7f3c0e59c640 1 --2- 172.21.15.86:0/3922569427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3c10103af0 0x7f3c10198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.259+0000 7f3c0e59c640 1 -- 172.21.15.86:0/3922569427 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c1010c510 msgr2=0x7f3c1019d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.259+0000 7f3c0e59c640 1 --2- 172.21.15.86:0/3922569427 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c1010c510 0x7f3c1019d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.259+0000 7f3c0e59c640 1 -- 172.21.15.86:0/3922569427 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f3c1019d730 con 0x7f3c101031d0 2024-04-21T08:47:31.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.259+0000 7f3c0ed9d640 1 --2- 172.21.15.86:0/3922569427 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c1010c510 0x7f3c1019d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:31.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.259+0000 7f3c0dd9b640 1 --2- 172.21.15.86:0/3922569427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3c10103af0 0x7f3c10198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:31.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.259+0000 7f3c0e59c640 1 --2- 172.21.15.86:0/3922569427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c101031d0 0x7f3c10198740 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f3c0000ebc0 tx=0x7f3c0000c6a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:31.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.259+0000 7f3bff7fe640 1 -- 172.21.15.86:0/3922569427 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c00010630 con 0x7f3c101031d0 2024-04-21T08:47:31.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.259+0000 7f3bff7fe640 1 -- 172.21.15.86:0/3922569427 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c00010c80 con 0x7f3c101031d0 2024-04-21T08:47:31.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.259+0000 7f3c14a43640 1 -- 172.21.15.86:0/3922569427 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3c1019da20 con 0x7f3c101031d0 2024-04-21T08:47:31.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.259+0000 7f3bff7fe640 1 -- 172.21.15.86:0/3922569427 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c00018b60 con 0x7f3c101031d0 2024-04-21T08:47:31.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.259+0000 7f3c14a43640 1 -- 172.21.15.86:0/3922569427 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f3c101a52b0 con 0x7f3c101031d0 2024-04-21T08:47:31.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.261+0000 7f3bff7fe640 1 -- 172.21.15.86:0/3922569427 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3c00018d00 con 0x7f3c101031d0 2024-04-21T08:47:31.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.262+0000 7f3c17637740 1 Processor -- start 2024-04-21T08:47:31.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.262+0000 7f3c17637740 1 -- start start 2024-04-21T08:47:31.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.262+0000 7f3c17637740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446bf704d0 0x56446bf708b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.262+0000 7f3c17637740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56446bf70e80 0x56446bf79430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.262+0000 7f3c17637740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56446bf79a00 0x56446bf7be50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.262+0000 7f3c17637740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56446bed51f0 con 0x56446bf704d0 2024-04-21T08:47:31.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.262+0000 7f3c17637740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x56446be724b0 con 0x56446bf79a00 2024-04-21T08:47:31.268 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.262+0000 7f3c17637740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56446bed11b0 con 0x56446bf70e80 2024-04-21T08:47:31.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.262+0000 7f3c0ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446bf704d0 0x56446bf708b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.262+0000 7f3c0ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446bf704d0 0x56446bf708b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52604/0 (socket says 172.21.15.86:52604) 2024-04-21T08:47:31.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.262+0000 7f3c0ed9d640 1 -- 172.21.15.86:0/882010234 learned_addr learned my addr 172.21.15.86:0/882010234 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:31.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.262+0000 7f3c0dd9b640 1 --2- 172.21.15.86:0/882010234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56446bf79a00 0x56446bf7be50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.262+0000 7f3bff7fe640 1 --2- 172.21.15.86:0/3922569427 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f3bf80a3d20 0x7f3bf80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.262+0000 7f3c0e59c640 1 --2- 172.21.15.86:0/882010234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56446bf70e80 0x56446bf79430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.262+0000 7f3c0ed9d640 1 --2- 172.21.15.86:0/3922569427 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f3bf80a3d20 0x7f3bf80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.263+0000 7f3c0dd9b640 1 -- 172.21.15.86:0/882010234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56446bf70e80 msgr2=0x56446bf79430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.263+0000 7f3c0dd9b640 1 --2- 172.21.15.86:0/882010234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56446bf70e80 0x56446bf79430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.263+0000 7f3c0dd9b640 1 -- 172.21.15.86:0/882010234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446bf704d0 msgr2=0x56446bf708b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.263+0000 7f3c0dd9b640 1 --2- 172.21.15.86:0/882010234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446bf704d0 0x56446bf708b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.263+0000 7f3c0dd9b640 1 -- 172.21.15.86:0/882010234 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56446be641b0 con 0x56446bf79a00 2024-04-21T08:47:31.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.263+0000 7f3c0e59c640 1 --2- 172.21.15.86:0/882010234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56446bf70e80 0x56446bf79430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:31.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.263+0000 7f3bff7fe640 1 -- 172.21.15.86:0/3922569427 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f3c000d1310 con 0x7f3c101031d0 2024-04-21T08:47:31.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.263+0000 7f3c0dd9b640 1 --2- 172.21.15.86:0/882010234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56446bf79a00 0x56446bf7be50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3c04036040 tx=0x7f3c04005e60 comp rx=0 tx=0).ready entity=mon.2 client_cookie=9e80b26dac844bed server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:31.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.263+0000 7f3c0ed9d640 1 --2- 172.21.15.86:0/3922569427 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f3bf80a3d20 0x7f3bf80a61e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3c08005fd0 tx=0x7f3c08004730 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:31.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.263+0000 7f3bfcff9640 1 -- 172.21.15.86:0/882010234 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c04041020 con 0x56446bf79a00 2024-04-21T08:47:31.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.263+0000 7f3bfcff9640 1 -- 172.21.15.86:0/882010234 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c0402c910 con 0x56446bf79a00 2024-04-21T08:47:31.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.263+0000 7f3bfcff9640 1 -- 172.21.15.86:0/882010234 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c04005380 con 0x56446bf79a00 2024-04-21T08:47:31.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.264+0000 7f3c17637740 1 -- 172.21.15.86:0/882010234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56446bf79a00 msgr2=0x56446bf7be50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.264+0000 7f3c17637740 1 --2- 172.21.15.86:0/882010234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56446bf79a00 0x56446bf7be50 secure :-1 s=READY pgs=80 cs=0 l=1 rev1=1 crypto rx=0x7f3c04036040 tx=0x7f3c04005e60 comp rx=0 tx=0).stop 2024-04-21T08:47:31.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.264+0000 7f3c17637740 1 -- 172.21.15.86:0/882010234 shutdown_connections 2024-04-21T08:47:31.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.264+0000 7f3c17637740 1 --2- 172.21.15.86:0/882010234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56446bf79a00 0x56446bf7be50 unknown :-1 s=CLOSED pgs=80 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.264+0000 7f3c17637740 1 --2- 172.21.15.86:0/882010234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56446bf70e80 0x56446bf79430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.264+0000 7f3c17637740 1 --2- 172.21.15.86:0/882010234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446bf704d0 0x56446bf708b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.264+0000 7f3c17637740 1 -- 172.21.15.86:0/882010234 >> 172.21.15.86:0/882010234 conn(0x56446bf6aba0 msgr2=0x56446bf6af80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:31.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.264+0000 7f3c17637740 1 -- 172.21.15.86:0/882010234 shutdown_connections 2024-04-21T08:47:31.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.264+0000 7f3c17637740 1 -- 172.21.15.86:0/882010234 wait complete. 2024-04-21T08:47:31.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.264+0000 7f3c17637740 1 -- start start 2024-04-21T08:47:31.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.264+0000 7f3c17637740 1 Processor -- start 2024-04-21T08:47:31.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.264+0000 7f3c17637740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56446bf704d0 0x56446bf792b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.264+0000 7f3c17637740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56446bf70e80 0x56446bf76640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.264+0000 7f3c17637740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446bf79a00 0x56446bf76b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.264+0000 7f3c17637740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56446bf7c910 con 0x56446bf79a00 2024-04-21T08:47:31.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.264+0000 7f3c17637740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x56446be8e500 con 0x56446bf70e80 2024-04-21T08:47:31.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.264+0000 7f3c17637740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56446bf7ca90 con 0x56446bf704d0 2024-04-21T08:47:31.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.264+0000 7f3c17637740 10 client.0 _ll_register_callbacks cb 0x56446be5cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:31.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.265+0000 7f3c0ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446bf79a00 0x56446bf76b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.265+0000 7f3c0ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446bf79a00 0x56446bf76b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52610/0 (socket says 172.21.15.86:52610) 2024-04-21T08:47:31.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.265+0000 7f3c0ed9d640 1 -- 172.21.15.86:0/1614154794 learned_addr learned my addr 172.21.15.86:0/1614154794 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:31.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.265+0000 7f3c0e59c640 1 --2- 172.21.15.86:0/1614154794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56446bf704d0 0x56446bf792b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.265+0000 7f3c0dd9b640 1 --2- 172.21.15.86:0/1614154794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56446bf70e80 0x56446bf76640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.265+0000 7f3c0ed9d640 1 -- 172.21.15.86:0/1614154794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56446bf704d0 msgr2=0x56446bf792b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.265+0000 7f3c0ed9d640 1 --2- 172.21.15.86:0/1614154794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56446bf704d0 0x56446bf792b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.265+0000 7f3c0ed9d640 1 -- 172.21.15.86:0/1614154794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56446bf70e80 msgr2=0x56446bf76640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.265+0000 7f3c0ed9d640 1 --2- 172.21.15.86:0/1614154794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56446bf70e80 0x56446bf76640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.265+0000 7f3c0ed9d640 1 -- 172.21.15.86:0/1614154794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56446bbe73f0 con 0x56446bf79a00 2024-04-21T08:47:31.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.265+0000 7f3c0ed9d640 1 --2- 172.21.15.86:0/1614154794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446bf79a00 0x56446bf76b80 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f3c08012800 tx=0x7f3c080046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1d55e9adf169f60a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:31.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.265+0000 7f3bf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:47:31.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.265+0000 7f3bf5ffb640 1 -- 172.21.15.86:0/1614154794 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c08024070 con 0x56446bf79a00 2024-04-21T08:47:31.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.265+0000 7f3bf5ffb640 1 -- 172.21.15.86:0/1614154794 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f3c0800fa70 con 0x56446bf79a00 2024-04-21T08:47:31.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.265+0000 7f3c17637740 1 -- 172.21.15.86:0/1614154794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c0402c910 con 0x56446bf79a00 2024-04-21T08:47:31.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.265+0000 7f3bf5ffb640 1 -- 172.21.15.86:0/1614154794 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3c08030050 con 0x56446bf79a00 2024-04-21T08:47:31.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.265+0000 7f3c17637740 10 client.19140 fetch_fsmap learned FSMap version 29 2024-04-21T08:47:31.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.265+0000 7f3c17637740 1 -- 172.21.15.86:0/1614154794 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56446bcb17d0 con 0x56446bf79a00 2024-04-21T08:47:31.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.265+0000 7f3bf5ffb640 1 -- 172.21.15.86:0/1614154794 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c0402c910 con 0x56446bf79a00 2024-04-21T08:47:31.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.266+0000 7f3bf5ffb640 1 -- 172.21.15.86:0/1614154794 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3c0801f070 con 0x56446bf79a00 2024-04-21T08:47:31.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.266+0000 7f3c17637740 10 client.19140 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:47:31.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.266+0000 7f3c17637740 10 client.19140 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:47:31.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.266+0000 7f3c17637740 20 client.19140 populate_metadata read hostname 'smithi086' 2024-04-21T08:47:31.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.266+0000 7f3c17637740 1 --2- 172.21.15.86:0/1614154794 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x56446bf89bf0 0x56446bf8c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.266+0000 7f3c17637740 4 client.19140 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:31.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.266+0000 7f3c17637740 1 -- 172.21.15.86:0/1614154794 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56446be74930 con 0x56446bf89bf0 2024-04-21T08:47:31.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.266+0000 7f3c0e59c640 1 --2- 172.21.15.86:0/1614154794 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x56446bf89bf0 0x56446bf8c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.266+0000 7f3c0e59c640 1 --2- 172.21.15.86:0/1614154794 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x56446bf89bf0 0x56446bf8c050 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=5055ff163a65575c server_cookie=385da9b038ec712 in_seq=0 out_seq=0 2024-04-21T08:47:31.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.266+0000 7f3bf5ffb640 10 client.19140 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:47:31.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.267+0000 7f3bf5ffb640 1 -- 172.21.15.86:0/1614154794 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56446be74930 con 0x56446bf89bf0 2024-04-21T08:47:31.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.267+0000 7f3bf5ffb640 10 client.19140 handle_command_reply: tid=0 2024-04-21T08:47:31.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.267+0000 7f3c17637740 1 client.19140 shutdown 2024-04-21T08:47:31.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.267+0000 7f3c17637740 1 -- 172.21.15.86:0/1614154794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446bf79a00 msgr2=0x56446bf76b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.267+0000 7f3c17637740 1 --2- 172.21.15.86:0/1614154794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446bf79a00 0x56446bf76b80 secure :-1 s=READY pgs=1643 cs=0 l=1 rev1=1 crypto rx=0x7f3c08012800 tx=0x7f3c080046c0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.267+0000 7f3c17637740 1 -- 172.21.15.86:0/1614154794 shutdown_connections 2024-04-21T08:47:31.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.267+0000 7f3c17637740 1 --2- 172.21.15.86:0/1614154794 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x56446bf89bf0 0x56446bf8c050 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.267+0000 7f3c17637740 1 --2- 172.21.15.86:0/1614154794 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446bf79a00 0x56446bf76b80 unknown :-1 s=CLOSED pgs=1643 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.267+0000 7f3bf5ffb640 0 client.19140 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:47:31.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.267+0000 7f3c17637740 1 --2- 172.21.15.86:0/1614154794 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56446bf70e80 0x56446bf76640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.267+0000 7f3c17637740 1 --2- 172.21.15.86:0/1614154794 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56446bf704d0 0x56446bf792b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.267+0000 7f3c17637740 1 -- 172.21.15.86:0/1614154794 >> 172.21.15.86:0/1614154794 conn(0x56446bf6aba0 msgr2=0x56446bf6c8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:31.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.267+0000 7f3c17637740 1 -- 172.21.15.86:0/1614154794 shutdown_connections 2024-04-21T08:47:31.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.267+0000 7f3c17637740 1 -- 172.21.15.86:0/1614154794 wait complete. 2024-04-21T08:47:31.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.267+0000 7f3c17637740 20 client.19140 trim_cache size 0 max 16384 2024-04-21T08:47:31.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.280+0000 7f3c17637740 1 Processor -- start 2024-04-21T08:47:31.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.280+0000 7f3c17637740 1 -- start start 2024-04-21T08:47:31.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.281+0000 7f3c17637740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446bf704d0 0x56446bf857d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.281+0000 7f3c17637740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56446bf70e80 0x56446bf820c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.281+0000 7f3c17637740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56446bf79a00 0x56446bf69a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.281+0000 7f3c17637740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3c101af440 con 0x56446bf704d0 2024-04-21T08:47:31.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.281+0000 7f3c17637740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3c101b0f80 con 0x56446bf79a00 2024-04-21T08:47:31.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.281+0000 7f3c17637740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3c101afb10 con 0x56446bf70e80 2024-04-21T08:47:31.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.281+0000 7f3c0e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446bf704d0 0x56446bf857d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.281+0000 7f3c0e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446bf704d0 0x56446bf857d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52612/0 (socket says 172.21.15.86:52612) 2024-04-21T08:47:31.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.281+0000 7f3c0e59c640 1 -- 172.21.15.86:0/3515375479 learned_addr learned my addr 172.21.15.86:0/3515375479 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:31.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.281+0000 7f3c0e59c640 1 -- 172.21.15.86:0/3515375479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56446bf70e80 msgr2=0x56446bf820c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.281+0000 7f3c0e59c640 1 --2- 172.21.15.86:0/3515375479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56446bf70e80 0x56446bf820c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.281+0000 7f3c0e59c640 1 -- 172.21.15.86:0/3515375479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56446bf79a00 msgr2=0x56446bf69a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.281+0000 7f3c0e59c640 1 --2- 172.21.15.86:0/3515375479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56446bf79a00 0x56446bf69a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.281+0000 7f3c0e59c640 1 -- 172.21.15.86:0/3515375479 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56446bf74b90 con 0x56446bf704d0 2024-04-21T08:47:31.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.281+0000 7f3c0e59c640 1 --2- 172.21.15.86:0/3515375479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446bf704d0 0x56446bf857d0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f3c000992f0 tx=0x7f3c0001f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aa90d7dfec58c2e6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:31.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.281+0000 7f3bf7fff640 1 -- 172.21.15.86:0/3515375479 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c000a1290 con 0x56446bf704d0 2024-04-21T08:47:31.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.281+0000 7f3bf7fff640 1 -- 172.21.15.86:0/3515375479 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c000a5b50 con 0x56446bf704d0 2024-04-21T08:47:31.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.281+0000 7f3bf7fff640 1 -- 172.21.15.86:0/3515375479 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c0009c330 con 0x56446bf704d0 2024-04-21T08:47:31.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.282+0000 7f3c17637740 1 -- 172.21.15.86:0/3515375479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446bf704d0 msgr2=0x56446bf857d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.282+0000 7f3c17637740 1 --2- 172.21.15.86:0/3515375479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446bf704d0 0x56446bf857d0 secure :-1 s=READY pgs=1644 cs=0 l=1 rev1=1 crypto rx=0x7f3c000992f0 tx=0x7f3c0001f000 comp rx=0 tx=0).stop 2024-04-21T08:47:31.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.282+0000 7f3c17637740 1 -- 172.21.15.86:0/3515375479 shutdown_connections 2024-04-21T08:47:31.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.282+0000 7f3c17637740 1 --2- 172.21.15.86:0/3515375479 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56446bf79a00 0x56446bf69a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.282+0000 7f3c17637740 1 --2- 172.21.15.86:0/3515375479 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56446bf70e80 0x56446bf820c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.282+0000 7f3c17637740 1 --2- 172.21.15.86:0/3515375479 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446bf704d0 0x56446bf857d0 unknown :-1 s=CLOSED pgs=1644 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.282+0000 7f3c17637740 1 -- 172.21.15.86:0/3515375479 >> 172.21.15.86:0/3515375479 conn(0x56446bf6aba0 msgr2=0x56446bf69370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:31.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.282+0000 7f3c17637740 1 -- 172.21.15.86:0/3515375479 shutdown_connections 2024-04-21T08:47:31.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.282+0000 7f3c17637740 1 -- 172.21.15.86:0/3515375479 wait complete. 2024-04-21T08:47:31.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.282+0000 7f3c17637740 1 -- start start 2024-04-21T08:47:31.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.282+0000 7f3c17637740 1 Processor -- start 2024-04-21T08:47:31.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.282+0000 7f3c17637740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56446bf704d0 0x56446bf728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.282+0000 7f3c17637740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56446bf70e80 0x56446bf72df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.282+0000 7f3c17637740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446bf79a00 0x56446bf73530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.282+0000 7f3c17637740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3c101b08b0 con 0x56446bf79a00 2024-04-21T08:47:31.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.282+0000 7f3c17637740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3c101b01e0 con 0x56446bf70e80 2024-04-21T08:47:31.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.282+0000 7f3c17637740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3c101ad440 con 0x56446bf704d0 2024-04-21T08:47:31.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.282+0000 7f3c17637740 10 client.0 _ll_register_callbacks cb 0x7f3c08012500 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:31.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.282+0000 7f3c0ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446bf79a00 0x56446bf73530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.283+0000 7f3c0ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446bf79a00 0x56446bf73530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52622/0 (socket says 172.21.15.86:52622) 2024-04-21T08:47:31.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.283+0000 7f3c0ed9d640 1 -- 172.21.15.86:0/2388505256 learned_addr learned my addr 172.21.15.86:0/2388505256 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:31.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.283+0000 7f3c0ed9d640 1 -- 172.21.15.86:0/2388505256 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56446bf704d0 msgr2=0x56446bf728b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.283+0000 7f3c0ed9d640 1 --2- 172.21.15.86:0/2388505256 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56446bf704d0 0x56446bf728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.283+0000 7f3c0dd9b640 1 --2- 172.21.15.86:0/2388505256 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56446bf70e80 0x56446bf72df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.283+0000 7f3c0ed9d640 1 -- 172.21.15.86:0/2388505256 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56446bf70e80 msgr2=0x56446bf72df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.283+0000 7f3c0ed9d640 1 --2- 172.21.15.86:0/2388505256 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56446bf70e80 0x56446bf72df0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.283+0000 7f3c0ed9d640 1 -- 172.21.15.86:0/2388505256 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56446bf73d30 con 0x56446bf79a00 2024-04-21T08:47:31.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.283+0000 7f3c0dd9b640 1 --2- 172.21.15.86:0/2388505256 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56446bf70e80 0x56446bf72df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:31.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.283+0000 7f3c0ed9d640 1 --2- 172.21.15.86:0/2388505256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446bf79a00 0x56446bf73530 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f3c08020e20 tx=0x7f3c080286b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5c77b49ffc097bb2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:31.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.283+0000 7f3bf5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:47:31.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.283+0000 7f3bf5ffb640 1 -- 172.21.15.86:0/2388505256 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3c08024050 con 0x56446bf79a00 2024-04-21T08:47:31.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.283+0000 7f3c17637740 1 -- 172.21.15.86:0/2388505256 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3c000a5b50 con 0x56446bf79a00 2024-04-21T08:47:31.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.283+0000 7f3bf5ffb640 1 -- 172.21.15.86:0/2388505256 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f3c08028aa0 con 0x56446bf79a00 2024-04-21T08:47:31.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.283+0000 7f3bf5ffb640 1 -- 172.21.15.86:0/2388505256 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3c08042050 con 0x56446bf79a00 2024-04-21T08:47:31.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.283+0000 7f3c17637740 10 client.19146 fetch_fsmap learned FSMap version 29 2024-04-21T08:47:31.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.283+0000 7f3c17637740 1 -- 172.21.15.86:0/2388505256 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56446bff48d0 con 0x56446bf79a00 2024-04-21T08:47:31.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.283+0000 7f3bf5ffb640 1 -- 172.21.15.86:0/2388505256 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3c000a5b50 con 0x56446bf79a00 2024-04-21T08:47:31.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.283+0000 7f3bf5ffb640 1 -- 172.21.15.86:0/2388505256 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3c0801f040 con 0x56446bf79a00 2024-04-21T08:47:31.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.284+0000 7f3c17637740 10 client.19146 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:47:31.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.284+0000 7f3c17637740 10 client.19146 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:47:31.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.284+0000 7f3c17637740 20 client.19146 populate_metadata read hostname 'smithi086' 2024-04-21T08:47:31.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.284+0000 7f3c17637740 1 --2- 172.21.15.86:0/2388505256 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x56446bff4f60 0x56446bff7380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.284+0000 7f3c17637740 4 client.19146 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:47:31.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.284+0000 7f3c17637740 1 -- 172.21.15.86:0/2388505256 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56446bee2c10 con 0x56446bff4f60 2024-04-21T08:47:31.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.284+0000 7f3c0e59c640 1 --2- 172.21.15.86:0/2388505256 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x56446bff4f60 0x56446bff7380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.284+0000 7f3bf5ffb640 10 client.19146 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:47:31.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.284+0000 7f3c0e59c640 1 --2- 172.21.15.86:0/2388505256 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x56446bff4f60 0x56446bff7380 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=fab19390d5dbae33 server_cookie=8328b9e040e5acd4 in_seq=0 out_seq=0 2024-04-21T08:47:31.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.284+0000 7f3bf5ffb640 1 -- 172.21.15.86:0/2388505256 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56446bee2c10 con 0x56446bff4f60 2024-04-21T08:47:31.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.284+0000 7f3bf5ffb640 10 client.19146 handle_command_reply: tid=0 2024-04-21T08:47:31.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.284+0000 7f3c17637740 1 client.19146 shutdown 2024-04-21T08:47:31.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.284+0000 7f3c17637740 1 -- 172.21.15.86:0/2388505256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446bf79a00 msgr2=0x56446bf73530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.284+0000 7f3c17637740 1 --2- 172.21.15.86:0/2388505256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446bf79a00 0x56446bf73530 secure :-1 s=READY pgs=1645 cs=0 l=1 rev1=1 crypto rx=0x7f3c08020e20 tx=0x7f3c080286b0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.284+0000 7f3c17637740 1 -- 172.21.15.86:0/2388505256 shutdown_connections 2024-04-21T08:47:31.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.284+0000 7f3c17637740 1 --2- 172.21.15.86:0/2388505256 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x56446bff4f60 0x56446bff7380 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.284+0000 7f3c17637740 1 --2- 172.21.15.86:0/2388505256 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56446bf79a00 0x56446bf73530 unknown :-1 s=CLOSED pgs=1645 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.284+0000 7f3c17637740 1 --2- 172.21.15.86:0/2388505256 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56446bf70e80 0x56446bf72df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.284+0000 7f3c17637740 1 --2- 172.21.15.86:0/2388505256 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56446bf704d0 0x56446bf728b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.284+0000 7f3c17637740 1 -- 172.21.15.86:0/2388505256 >> 172.21.15.86:0/2388505256 conn(0x56446bf6aba0 msgr2=0x56446bf69c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:31.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.284+0000 7f3bf5ffb640 0 client.19146 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:47:31.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.285+0000 7f3c17637740 1 -- 172.21.15.86:0/2388505256 shutdown_connections 2024-04-21T08:47:31.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.285+0000 7f3c17637740 1 -- 172.21.15.86:0/2388505256 wait complete. 2024-04-21T08:47:31.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.285+0000 7f3c17637740 20 client.19146 trim_cache size 0 max 16384 2024-04-21T08:47:31.291 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:47:31.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.285+0000 7f3c14a43640 1 -- 172.21.15.86:0/3922569427 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f3bf80a3d20 msgr2=0x7f3bf80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.285+0000 7f3c14a43640 1 --2- 172.21.15.86:0/3922569427 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f3bf80a3d20 0x7f3bf80a61e0 secure :-1 s=READY pgs=268 cs=0 l=1 rev1=1 crypto rx=0x7f3c08005fd0 tx=0x7f3c08004730 comp rx=0 tx=0).stop 2024-04-21T08:47:31.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.285+0000 7f3c14a43640 1 -- 172.21.15.86:0/3922569427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c101031d0 msgr2=0x7f3c10198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.285+0000 7f3c14a43640 1 --2- 172.21.15.86:0/3922569427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c101031d0 0x7f3c10198740 secure :-1 s=READY pgs=1642 cs=0 l=1 rev1=1 crypto rx=0x7f3c0000ebc0 tx=0x7f3c0000c6a0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.285+0000 7f3c14a43640 1 -- 172.21.15.86:0/3922569427 shutdown_connections 2024-04-21T08:47:31.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.285+0000 7f3c14a43640 1 --2- 172.21.15.86:0/3922569427 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f3bf80a3d20 0x7f3bf80a61e0 unknown :-1 s=CLOSED pgs=268 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.285+0000 7f3c14a43640 1 --2- 172.21.15.86:0/3922569427 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3c1010c510 0x7f3c1019d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.285+0000 7f3c14a43640 1 --2- 172.21.15.86:0/3922569427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3c10103af0 0x7f3c10198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.285+0000 7f3c14a43640 1 --2- 172.21.15.86:0/3922569427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3c101031d0 0x7f3c10198740 unknown :-1 s=CLOSED pgs=1642 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.285+0000 7f3c14a43640 1 -- 172.21.15.86:0/3922569427 >> 172.21.15.86:0/3922569427 conn(0x7f3c10077f50 msgr2=0x7f3c100fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:31.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.285+0000 7f3c14a43640 1 -- 172.21.15.86:0/3922569427 shutdown_connections 2024-04-21T08:47:31.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.285+0000 7f3c14a43640 1 -- 172.21.15.86:0/3922569427 wait complete. 2024-04-21T08:47:31.295 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:47:31.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.395+0000 7f44b81e6640 1 -- 172.21.15.86:0/999914834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44b0102b40 msgr2=0x7f44b0106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.395+0000 7f44b81e6640 1 --2- 172.21.15.86:0/999914834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44b0102b40 0x7f44b0106b80 secure :-1 s=READY pgs=1646 cs=0 l=1 rev1=1 crypto rx=0x7f44a0009900 tx=0x7f44a002bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:31.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.395+0000 7f44b81e6640 1 -- 172.21.15.86:0/999914834 shutdown_connections 2024-04-21T08:47:31.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.395+0000 7f44b81e6640 1 --2- 172.21.15.86:0/999914834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44b0107260 0x7f44b010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.395+0000 7f44b81e6640 1 --2- 172.21.15.86:0/999914834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44b0102b40 0x7f44b0106b80 unknown :-1 s=CLOSED pgs=1646 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.395+0000 7f44b81e6640 1 --2- 172.21.15.86:0/999914834 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44b0102190 0x7f44b0102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.395+0000 7f44b81e6640 1 -- 172.21.15.86:0/999914834 >> 172.21.15.86:0/999914834 conn(0x7f44b00793a0 msgr2=0x7f44b00797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:31.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.395+0000 7f44b81e6640 1 -- 172.21.15.86:0/999914834 shutdown_connections 2024-04-21T08:47:31.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.395+0000 7f44b81e6640 1 -- 172.21.15.86:0/999914834 wait complete. 2024-04-21T08:47:31.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.396+0000 7f44b81e6640 1 Processor -- start 2024-04-21T08:47:31.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.396+0000 7f44b81e6640 1 -- start start 2024-04-21T08:47:31.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.396+0000 7f44b81e6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44b0102190 0x7f44b0198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.396+0000 7f44b81e6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44b0107260 0x7f44b0198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.396+0000 7f44b81e6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44b019cfc0 0x7f44b019d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.396+0000 7f44b81e6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44b0112180 con 0x7f44b0107260 2024-04-21T08:47:31.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.396+0000 7f44b81e6640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f44b0112000 con 0x7f44b0102190 2024-04-21T08:47:31.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.396+0000 7f44b81e6640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f44b0112300 con 0x7f44b019cfc0 2024-04-21T08:47:31.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.396+0000 7f44b5781640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44b0107260 0x7f44b0198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.396+0000 7f44b5781640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44b0107260 0x7f44b0198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52630/0 (socket says 172.21.15.86:52630) 2024-04-21T08:47:31.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.396+0000 7f44b5781640 1 -- 172.21.15.86:0/1720032443 learned_addr learned my addr 172.21.15.86:0/1720032443 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:31.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.396+0000 7f44b5f82640 1 --2- 172.21.15.86:0/1720032443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44b0102190 0x7f44b0198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.396+0000 7f44b6783640 1 --2- 172.21.15.86:0/1720032443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44b019cfc0 0x7f44b019d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.396+0000 7f44b5781640 1 -- 172.21.15.86:0/1720032443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44b019cfc0 msgr2=0x7f44b019d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.396+0000 7f44b5781640 1 --2- 172.21.15.86:0/1720032443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44b019cfc0 0x7f44b019d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.398 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.396+0000 7f44b5781640 1 -- 172.21.15.86:0/1720032443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44b0102190 msgr2=0x7f44b0198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.396+0000 7f44b5781640 1 --2- 172.21.15.86:0/1720032443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44b0102190 0x7f44b0198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.396+0000 7f44b5781640 1 -- 172.21.15.86:0/1720032443 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f44b019dab0 con 0x7f44b0107260 2024-04-21T08:47:31.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.396+0000 7f44b5f82640 1 --2- 172.21.15.86:0/1720032443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44b0102190 0x7f44b0198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:31.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.396+0000 7f44b5781640 1 --2- 172.21.15.86:0/1720032443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44b0107260 0x7f44b0198c40 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f44a0036040 tx=0x7f44a002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:31.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.397+0000 7f44a6ffd640 1 -- 172.21.15.86:0/1720032443 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a0043020 con 0x7f44b0107260 2024-04-21T08:47:31.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.397+0000 7f44a6ffd640 1 -- 172.21.15.86:0/1720032443 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44a002cbe0 con 0x7f44b0107260 2024-04-21T08:47:31.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.397+0000 7f44b81e6640 1 -- 172.21.15.86:0/1720032443 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f44b019dd40 con 0x7f44b0107260 2024-04-21T08:47:31.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.397+0000 7f44a6ffd640 1 -- 172.21.15.86:0/1720032443 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a0003bf0 con 0x7f44b0107260 2024-04-21T08:47:31.399 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.397+0000 7f44b81e6640 1 -- 172.21.15.86:0/1720032443 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f44b01a5560 con 0x7f44b0107260 2024-04-21T08:47:31.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.399+0000 7f44a6ffd640 1 -- 172.21.15.86:0/1720032443 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f44a0046020 con 0x7f44b0107260 2024-04-21T08:47:31.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.399+0000 7f44badd4740 1 Processor -- start 2024-04-21T08:47:31.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.400+0000 7f44badd4740 1 -- start start 2024-04-21T08:47:31.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.400+0000 7f44badd4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ec784c4d0 0x564ec784c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.400+0000 7f44badd4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ec784ce80 0x564ec7855430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.406 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.400+0000 7f44badd4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ec7855a00 0x564ec7857e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.400+0000 7f44badd4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564ec77b11f0 con 0x564ec784ce80 2024-04-21T08:47:31.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.400+0000 7f44badd4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564ec774e4b0 con 0x564ec784c4d0 2024-04-21T08:47:31.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.400+0000 7f44badd4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564ec77ad1b0 con 0x564ec7855a00 2024-04-21T08:47:31.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.400+0000 7f44b5f82640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ec784ce80 0x564ec7855430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.400+0000 7f44b5f82640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ec784ce80 0x564ec7855430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52632/0 (socket says 172.21.15.86:52632) 2024-04-21T08:47:31.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.400+0000 7f44b5f82640 1 -- 172.21.15.86:0/452858523 learned_addr learned my addr 172.21.15.86:0/452858523 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:31.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.400+0000 7f44b5781640 1 --2- 172.21.15.86:0/452858523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ec7855a00 0x564ec7857e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.400+0000 7f44a6ffd640 1 --2- 172.21.15.86:0/1720032443 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f449c0a3d20 0x7f449c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.400+0000 7f44b6783640 1 --2- 172.21.15.86:0/452858523 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ec784c4d0 0x564ec784c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.400+0000 7f44b5f82640 1 -- 172.21.15.86:0/452858523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ec7855a00 msgr2=0x564ec7857e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.400+0000 7f44b5f82640 1 --2- 172.21.15.86:0/452858523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ec7855a00 0x564ec7857e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.400+0000 7f44b5f82640 1 -- 172.21.15.86:0/452858523 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ec784c4d0 msgr2=0x564ec784c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.400+0000 7f44b5f82640 1 --2- 172.21.15.86:0/452858523 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ec784c4d0 0x564ec784c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.400+0000 7f44b5f82640 1 -- 172.21.15.86:0/452858523 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ec77401b0 con 0x564ec784ce80 2024-04-21T08:47:31.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.400+0000 7f44b5781640 1 --2- 172.21.15.86:0/452858523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ec7855a00 0x564ec7857e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:31.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.400+0000 7f44b6783640 1 --2- 172.21.15.86:0/452858523 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ec784c4d0 0x564ec784c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:31.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.400+0000 7f44b6783640 1 --2- 172.21.15.86:0/1720032443 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f449c0a3d20 0x7f449c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.400+0000 7f44a6ffd640 1 -- 172.21.15.86:0/1720032443 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f44a00f2dc0 con 0x7f44b0107260 2024-04-21T08:47:31.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.401+0000 7f44b5f82640 1 --2- 172.21.15.86:0/452858523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ec784ce80 0x564ec7855430 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f44ac009c50 tx=0x7f44ac00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=39071cd31fcf4d6f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:31.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.401+0000 7f449bfff640 1 -- 172.21.15.86:0/452858523 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ac016070 con 0x564ec784ce80 2024-04-21T08:47:31.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.401+0000 7f44b6783640 1 --2- 172.21.15.86:0/1720032443 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f449c0a3d20 0x7f449c0a61e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f44a800bda0 tx=0x7f44a8004a40 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:31.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.401+0000 7f449bfff640 1 -- 172.21.15.86:0/452858523 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44ac0054d0 con 0x564ec784ce80 2024-04-21T08:47:31.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.401+0000 7f449bfff640 1 -- 172.21.15.86:0/452858523 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ac011470 con 0x564ec784ce80 2024-04-21T08:47:31.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.401+0000 7f44badd4740 1 -- 172.21.15.86:0/452858523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ec784ce80 msgr2=0x564ec7855430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.401+0000 7f44badd4740 1 --2- 172.21.15.86:0/452858523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ec784ce80 0x564ec7855430 secure :-1 s=READY pgs=1648 cs=0 l=1 rev1=1 crypto rx=0x7f44ac009c50 tx=0x7f44ac00e6c0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.402+0000 7f44badd4740 1 -- 172.21.15.86:0/452858523 shutdown_connections 2024-04-21T08:47:31.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.402+0000 7f44badd4740 1 --2- 172.21.15.86:0/452858523 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ec7855a00 0x564ec7857e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.402+0000 7f44badd4740 1 --2- 172.21.15.86:0/452858523 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ec784ce80 0x564ec7855430 unknown :-1 s=CLOSED pgs=1648 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.402+0000 7f44badd4740 1 --2- 172.21.15.86:0/452858523 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ec784c4d0 0x564ec784c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.402+0000 7f44badd4740 1 -- 172.21.15.86:0/452858523 >> 172.21.15.86:0/452858523 conn(0x564ec7846ba0 msgr2=0x564ec7846f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:31.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.402+0000 7f44badd4740 1 -- 172.21.15.86:0/452858523 shutdown_connections 2024-04-21T08:47:31.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.402+0000 7f44badd4740 1 -- 172.21.15.86:0/452858523 wait complete. 2024-04-21T08:47:31.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.402+0000 7f44badd4740 1 -- start start 2024-04-21T08:47:31.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.402+0000 7f44badd4740 1 Processor -- start 2024-04-21T08:47:31.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.402+0000 7f44badd4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ec784c4d0 0x564ec78552f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.402+0000 7f44badd4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ec784ce80 0x564ec7852680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.402+0000 7f44badd4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ec7855a00 0x564ec7852c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.402+0000 7f44badd4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564ec7858970 con 0x564ec784ce80 2024-04-21T08:47:31.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.402+0000 7f44badd4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564ec776a500 con 0x564ec784c4d0 2024-04-21T08:47:31.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.402+0000 7f44badd4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564ec7858af0 con 0x564ec7855a00 2024-04-21T08:47:31.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.402+0000 7f44badd4740 10 client.0 _ll_register_callbacks cb 0x564ec7738b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:31.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.402+0000 7f44b5781640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ec784ce80 0x564ec7852680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.403+0000 7f44b5781640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ec784ce80 0x564ec7852680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52644/0 (socket says 172.21.15.86:52644) 2024-04-21T08:47:31.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.403+0000 7f44b5781640 1 -- 172.21.15.86:0/863621838 learned_addr learned my addr 172.21.15.86:0/863621838 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:31.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.403+0000 7f44b5f82640 1 --2- 172.21.15.86:0/863621838 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ec784c4d0 0x564ec78552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.403+0000 7f44b6783640 1 --2- 172.21.15.86:0/863621838 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ec7855a00 0x564ec7852c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.403+0000 7f44b5781640 1 -- 172.21.15.86:0/863621838 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ec7855a00 msgr2=0x564ec7852c00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.403+0000 7f44b5781640 1 --2- 172.21.15.86:0/863621838 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ec7855a00 0x564ec7852c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.403+0000 7f44b5781640 1 -- 172.21.15.86:0/863621838 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ec784c4d0 msgr2=0x564ec78552f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.403+0000 7f44b5781640 1 --2- 172.21.15.86:0/863621838 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ec784c4d0 0x564ec78552f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.403+0000 7f44b5781640 1 -- 172.21.15.86:0/863621838 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ec74c33f0 con 0x564ec784ce80 2024-04-21T08:47:31.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.403+0000 7f44b5f82640 1 --2- 172.21.15.86:0/863621838 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ec784c4d0 0x564ec78552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:31.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.403+0000 7f44b5781640 1 --2- 172.21.15.86:0/863621838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ec784ce80 0x564ec7852680 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f44a00f02b0 tx=0x7f44a0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=620b14a182c447ff server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:31.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.403+0000 7f44997fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:47:31.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.403+0000 7f44997fa640 1 -- 172.21.15.86:0/863621838 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a00bf1a0 con 0x564ec784ce80 2024-04-21T08:47:31.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.403+0000 7f44997fa640 1 -- 172.21.15.86:0/863621838 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f44a00054d0 con 0x564ec784ce80 2024-04-21T08:47:31.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.403+0000 7f44badd4740 1 -- 172.21.15.86:0/863621838 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44ac0054d0 con 0x564ec784ce80 2024-04-21T08:47:31.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.403+0000 7f44997fa640 1 -- 172.21.15.86:0/863621838 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f44a00beaa0 con 0x564ec784ce80 2024-04-21T08:47:31.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.403+0000 7f44badd4740 10 client.19158 fetch_fsmap learned FSMap version 29 2024-04-21T08:47:31.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.403+0000 7f44badd4740 1 -- 172.21.15.86:0/863621838 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564ec758d7d0 con 0x564ec784ce80 2024-04-21T08:47:31.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.403+0000 7f44997fa640 1 -- 172.21.15.86:0/863621838 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44ac0054d0 con 0x564ec784ce80 2024-04-21T08:47:31.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.404+0000 7f44997fa640 1 -- 172.21.15.86:0/863621838 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f44a00c7280 con 0x564ec784ce80 2024-04-21T08:47:31.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.404+0000 7f44badd4740 10 client.19158 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:47:31.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.404+0000 7f44badd4740 10 client.19158 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:47:31.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.404+0000 7f44badd4740 20 client.19158 populate_metadata read hostname 'smithi086' 2024-04-21T08:47:31.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.404+0000 7f44badd4740 1 --2- 172.21.15.86:0/863621838 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x564ec7865a70 0x564ec7867e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.404+0000 7f44badd4740 4 client.19158 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:31.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.404+0000 7f44badd4740 1 -- 172.21.15.86:0/863621838 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564ec7750930 con 0x564ec7865a70 2024-04-21T08:47:31.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.404+0000 7f44b5f82640 1 --2- 172.21.15.86:0/863621838 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x564ec7865a70 0x564ec7867e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.404+0000 7f44b5f82640 1 --2- 172.21.15.86:0/863621838 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x564ec7865a70 0x564ec7867e80 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=6046f932260a4974 server_cookie=5e8a19057eba73a9 in_seq=0 out_seq=0 2024-04-21T08:47:31.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.404+0000 7f44997fa640 10 client.19158 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:47:31.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.405+0000 7f44997fa640 1 -- 172.21.15.86:0/863621838 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564ec7750930 con 0x564ec7865a70 2024-04-21T08:47:31.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.405+0000 7f44997fa640 10 client.19158 handle_command_reply: tid=0 2024-04-21T08:47:31.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.405+0000 7f44badd4740 1 client.19158 shutdown 2024-04-21T08:47:31.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.405+0000 7f44badd4740 1 -- 172.21.15.86:0/863621838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ec784ce80 msgr2=0x564ec7852680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.405+0000 7f44badd4740 1 --2- 172.21.15.86:0/863621838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ec784ce80 0x564ec7852680 secure :-1 s=READY pgs=1649 cs=0 l=1 rev1=1 crypto rx=0x7f44a00f02b0 tx=0x7f44a0000bc0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.405+0000 7f44badd4740 1 -- 172.21.15.86:0/863621838 shutdown_connections 2024-04-21T08:47:31.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.405+0000 7f44badd4740 1 --2- 172.21.15.86:0/863621838 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x564ec7865a70 0x564ec7867e80 crc :-1 s=READY pgs=65 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.405+0000 7f44badd4740 1 --2- 172.21.15.86:0/863621838 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ec7855a00 0x564ec7852c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.405+0000 7f44997fa640 0 client.19158 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:47:31.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.405+0000 7f44badd4740 1 --2- 172.21.15.86:0/863621838 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ec784ce80 0x564ec7852680 unknown :-1 s=CLOSED pgs=1649 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.405+0000 7f44badd4740 1 --2- 172.21.15.86:0/863621838 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ec784c4d0 0x564ec78552f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.405+0000 7f44badd4740 1 -- 172.21.15.86:0/863621838 >> 172.21.15.86:0/863621838 conn(0x564ec7846ba0 msgr2=0x564ec78488f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:31.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.405+0000 7f44badd4740 1 -- 172.21.15.86:0/863621838 shutdown_connections 2024-04-21T08:47:31.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.405+0000 7f44badd4740 1 -- 172.21.15.86:0/863621838 wait complete. 2024-04-21T08:47:31.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.405+0000 7f44badd4740 20 client.19158 trim_cache size 0 max 16384 2024-04-21T08:47:31.419 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.418+0000 7f44badd4740 1 Processor -- start 2024-04-21T08:47:31.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.418+0000 7f44badd4740 1 -- start start 2024-04-21T08:47:31.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.419+0000 7f44badd4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ec784c4d0 0x564ec7861860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.419+0000 7f44badd4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ec784ce80 0x564ec7854db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.419+0000 7f44badd4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ec7855a00 0x564ec78552f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.419+0000 7f44badd4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44b01af330 con 0x564ec7855a00 2024-04-21T08:47:31.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.419+0000 7f44badd4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f44b01b0e70 con 0x564ec784ce80 2024-04-21T08:47:31.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.419+0000 7f44badd4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f44b01afa00 con 0x564ec784c4d0 2024-04-21T08:47:31.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.419+0000 7f44b6783640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ec7855a00 0x564ec78552f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.419+0000 7f44b6783640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ec7855a00 0x564ec78552f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52648/0 (socket says 172.21.15.86:52648) 2024-04-21T08:47:31.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.419+0000 7f44b6783640 1 -- 172.21.15.86:0/1337935695 learned_addr learned my addr 172.21.15.86:0/1337935695 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:31.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.419+0000 7f44b5f82640 1 --2- 172.21.15.86:0/1337935695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ec784c4d0 0x564ec7861860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.419+0000 7f44b6783640 1 -- 172.21.15.86:0/1337935695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ec784c4d0 msgr2=0x564ec7861860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.419+0000 7f44b6783640 1 --2- 172.21.15.86:0/1337935695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ec784c4d0 0x564ec7861860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.419+0000 7f44b5781640 1 --2- 172.21.15.86:0/1337935695 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ec784ce80 0x564ec7854db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.419+0000 7f44b6783640 1 -- 172.21.15.86:0/1337935695 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ec784ce80 msgr2=0x564ec7854db0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.419+0000 7f44b6783640 1 --2- 172.21.15.86:0/1337935695 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ec784ce80 0x564ec7854db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.419+0000 7f44b6783640 1 -- 172.21.15.86:0/1337935695 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564ec784cc20 con 0x564ec7855a00 2024-04-21T08:47:31.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.419+0000 7f44b6783640 1 --2- 172.21.15.86:0/1337935695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ec7855a00 0x564ec78552f0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f44a801a450 tx=0x7f44a8011000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=138b25f85aa77908 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:31.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.419+0000 7f449b7fe640 1 -- 172.21.15.86:0/1337935695 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a8018430 con 0x564ec7855a00 2024-04-21T08:47:31.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.419+0000 7f449b7fe640 1 -- 172.21.15.86:0/1337935695 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44a8018a80 con 0x564ec7855a00 2024-04-21T08:47:31.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.419+0000 7f449b7fe640 1 -- 172.21.15.86:0/1337935695 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44a800faf0 con 0x564ec7855a00 2024-04-21T08:47:31.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.420+0000 7f44badd4740 1 -- 172.21.15.86:0/1337935695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ec7855a00 msgr2=0x564ec78552f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.420+0000 7f44badd4740 1 --2- 172.21.15.86:0/1337935695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ec7855a00 0x564ec78552f0 secure :-1 s=READY pgs=1650 cs=0 l=1 rev1=1 crypto rx=0x7f44a801a450 tx=0x7f44a8011000 comp rx=0 tx=0).stop 2024-04-21T08:47:31.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.420+0000 7f44badd4740 1 -- 172.21.15.86:0/1337935695 shutdown_connections 2024-04-21T08:47:31.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.420+0000 7f44badd4740 1 --2- 172.21.15.86:0/1337935695 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ec7855a00 0x564ec78552f0 unknown :-1 s=CLOSED pgs=1650 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.420+0000 7f44badd4740 1 --2- 172.21.15.86:0/1337935695 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ec784ce80 0x564ec7854db0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.420+0000 7f44badd4740 1 --2- 172.21.15.86:0/1337935695 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ec784c4d0 0x564ec7861860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.420+0000 7f44badd4740 1 -- 172.21.15.86:0/1337935695 >> 172.21.15.86:0/1337935695 conn(0x564ec7846ba0 msgr2=0x564ec7845c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:31.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.420+0000 7f44badd4740 1 -- 172.21.15.86:0/1337935695 shutdown_connections 2024-04-21T08:47:31.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.420+0000 7f44badd4740 1 -- 172.21.15.86:0/1337935695 wait complete. 2024-04-21T08:47:31.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.420+0000 7f44badd4740 1 -- start start 2024-04-21T08:47:31.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.420+0000 7f44badd4740 1 Processor -- start 2024-04-21T08:47:31.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.421+0000 7f44badd4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ec784c4d0 0x564ec784f6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.421+0000 7f44badd4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ec784ce80 0x564ec784fc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.421+0000 7f44badd4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ec7855a00 0x564ec78cafc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.421+0000 7f44badd4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f44b01b07a0 con 0x564ec784c4d0 2024-04-21T08:47:31.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.421+0000 7f44badd4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f44b01b00d0 con 0x564ec784ce80 2024-04-21T08:47:31.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.421+0000 7f44badd4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f44b01ad330 con 0x564ec7855a00 2024-04-21T08:47:31.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.421+0000 7f44badd4740 10 client.0 _ll_register_callbacks cb 0x7f44a0009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:31.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.421+0000 7f44b5f82640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ec784c4d0 0x564ec784f6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.421+0000 7f44b5f82640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ec784c4d0 0x564ec784f6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52650/0 (socket says 172.21.15.86:52650) 2024-04-21T08:47:31.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.421+0000 7f44b5f82640 1 -- 172.21.15.86:0/1352688656 learned_addr learned my addr 172.21.15.86:0/1352688656 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:31.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.421+0000 7f44b6783640 1 --2- 172.21.15.86:0/1352688656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ec7855a00 0x564ec78cafc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.421+0000 7f44b5781640 1 --2- 172.21.15.86:0/1352688656 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ec784ce80 0x564ec784fc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.421+0000 7f44b5f82640 1 -- 172.21.15.86:0/1352688656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ec7855a00 msgr2=0x564ec78cafc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.421+0000 7f44b5f82640 1 --2- 172.21.15.86:0/1352688656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ec7855a00 0x564ec78cafc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.421+0000 7f44b5f82640 1 -- 172.21.15.86:0/1352688656 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ec784ce80 msgr2=0x564ec784fc10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.421+0000 7f44b5f82640 1 --2- 172.21.15.86:0/1352688656 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ec784ce80 0x564ec784fc10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.421+0000 7f44b5f82640 1 -- 172.21.15.86:0/1352688656 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564ec78504e0 con 0x564ec784c4d0 2024-04-21T08:47:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.421+0000 7f44b5781640 1 --2- 172.21.15.86:0/1352688656 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ec784ce80 0x564ec784fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.421+0000 7f44b5f82640 1 --2- 172.21.15.86:0/1352688656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ec784c4d0 0x564ec784f6d0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f44ac005e90 tx=0x7f44ac0095b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f85d209df2a01891 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.421+0000 7f44997fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:47:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.421+0000 7f44997fa640 1 -- 172.21.15.86:0/1352688656 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f44ac016040 con 0x564ec784c4d0 2024-04-21T08:47:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.421+0000 7f44997fa640 1 -- 172.21.15.86:0/1352688656 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f44ac029310 con 0x564ec784c4d0 2024-04-21T08:47:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.421+0000 7f44badd4740 1 -- 172.21.15.86:0/1352688656 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f44a8018a80 con 0x564ec784c4d0 2024-04-21T08:47:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.422+0000 7f44997fa640 1 -- 172.21.15.86:0/1352688656 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f44ac029510 con 0x564ec784c4d0 2024-04-21T08:47:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.422+0000 7f44badd4740 10 client.19164 fetch_fsmap learned FSMap version 29 2024-04-21T08:47:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.422+0000 7f44badd4740 1 -- 172.21.15.86:0/1352688656 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564ec78cc600 con 0x564ec784c4d0 2024-04-21T08:47:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.422+0000 7f44997fa640 1 -- 172.21.15.86:0/1352688656 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f44a8018a80 con 0x564ec784c4d0 2024-04-21T08:47:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.422+0000 7f44997fa640 1 -- 172.21.15.86:0/1352688656 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f44ac00f040 con 0x564ec784c4d0 2024-04-21T08:47:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.422+0000 7f44badd4740 10 client.19164 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:47:31.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.422+0000 7f44badd4740 10 client.19164 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:47:31.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.422+0000 7f44badd4740 20 client.19164 populate_metadata read hostname 'smithi086' 2024-04-21T08:47:31.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.422+0000 7f44badd4740 1 --2- 172.21.15.86:0/1352688656 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x564ec78ccb90 0x564ec78cefb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.422+0000 7f44badd4740 4 client.19164 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:47:31.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.422+0000 7f44badd4740 1 -- 172.21.15.86:0/1352688656 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564ec77bec10 con 0x564ec78ccb90 2024-04-21T08:47:31.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.422+0000 7f44b5781640 1 --2- 172.21.15.86:0/1352688656 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x564ec78ccb90 0x564ec78cefb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.423+0000 7f44b5781640 1 --2- 172.21.15.86:0/1352688656 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x564ec78ccb90 0x564ec78cefb0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9893aeb72d1acf98 server_cookie=34d2a80110ddccc5 in_seq=0 out_seq=0 2024-04-21T08:47:31.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.423+0000 7f44997fa640 10 client.19164 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:47:31.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.423+0000 7f44997fa640 1 -- 172.21.15.86:0/1352688656 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564ec77bec10 con 0x564ec78ccb90 2024-04-21T08:47:31.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.423+0000 7f44997fa640 10 client.19164 handle_command_reply: tid=0 2024-04-21T08:47:31.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.423+0000 7f44badd4740 1 client.19164 shutdown 2024-04-21T08:47:31.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.423+0000 7f44badd4740 1 -- 172.21.15.86:0/1352688656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ec784c4d0 msgr2=0x564ec784f6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.423+0000 7f44badd4740 1 --2- 172.21.15.86:0/1352688656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ec784c4d0 0x564ec784f6d0 secure :-1 s=READY pgs=1651 cs=0 l=1 rev1=1 crypto rx=0x7f44ac005e90 tx=0x7f44ac0095b0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.423+0000 7f44badd4740 1 -- 172.21.15.86:0/1352688656 shutdown_connections 2024-04-21T08:47:31.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.423+0000 7f44badd4740 1 --2- 172.21.15.86:0/1352688656 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x564ec78ccb90 0x564ec78cefb0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.423+0000 7f44badd4740 1 --2- 172.21.15.86:0/1352688656 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564ec7855a00 0x564ec78cafc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.423+0000 7f44badd4740 1 --2- 172.21.15.86:0/1352688656 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564ec784ce80 0x564ec784fc10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.423+0000 7f44997fa640 0 client.19164 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:47:31.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.423+0000 7f44badd4740 1 --2- 172.21.15.86:0/1352688656 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564ec784c4d0 0x564ec784f6d0 unknown :-1 s=CLOSED pgs=1651 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.423+0000 7f44badd4740 1 -- 172.21.15.86:0/1352688656 >> 172.21.15.86:0/1352688656 conn(0x564ec7846ba0 msgr2=0x564ec784d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:31.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.423+0000 7f44badd4740 1 -- 172.21.15.86:0/1352688656 shutdown_connections 2024-04-21T08:47:31.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.423+0000 7f44badd4740 1 -- 172.21.15.86:0/1352688656 wait complete. 2024-04-21T08:47:31.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.423+0000 7f44badd4740 20 client.19164 trim_cache size 0 max 16384 2024-04-21T08:47:31.430 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:47:31.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.423+0000 7f44b81e6640 1 -- 172.21.15.86:0/1720032443 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f449c0a3d20 msgr2=0x7f449c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.424+0000 7f44b81e6640 1 --2- 172.21.15.86:0/1720032443 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f449c0a3d20 0x7f449c0a61e0 secure :-1 s=READY pgs=269 cs=0 l=1 rev1=1 crypto rx=0x7f44a800bda0 tx=0x7f44a8004a40 comp rx=0 tx=0).stop 2024-04-21T08:47:31.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.424+0000 7f44b81e6640 1 -- 172.21.15.86:0/1720032443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44b0107260 msgr2=0x7f44b0198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.424+0000 7f44b81e6640 1 --2- 172.21.15.86:0/1720032443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44b0107260 0x7f44b0198c40 secure :-1 s=READY pgs=1647 cs=0 l=1 rev1=1 crypto rx=0x7f44a0036040 tx=0x7f44a002c7e0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.424+0000 7f44b81e6640 1 -- 172.21.15.86:0/1720032443 shutdown_connections 2024-04-21T08:47:31.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.424+0000 7f44b81e6640 1 --2- 172.21.15.86:0/1720032443 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f449c0a3d20 0x7f449c0a61e0 unknown :-1 s=CLOSED pgs=269 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.424+0000 7f44b81e6640 1 --2- 172.21.15.86:0/1720032443 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f44b019cfc0 0x7f44b019d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.424+0000 7f44b81e6640 1 --2- 172.21.15.86:0/1720032443 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f44b0107260 0x7f44b0198c40 unknown :-1 s=CLOSED pgs=1647 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.424+0000 7f44b81e6640 1 --2- 172.21.15.86:0/1720032443 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f44b0102190 0x7f44b0198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.424+0000 7f44b81e6640 1 -- 172.21.15.86:0/1720032443 >> 172.21.15.86:0/1720032443 conn(0x7f44b00793a0 msgr2=0x7f44b00fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:31.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.424+0000 7f44b81e6640 1 -- 172.21.15.86:0/1720032443 shutdown_connections 2024-04-21T08:47:31.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.424+0000 7f44b81e6640 1 -- 172.21.15.86:0/1720032443 wait complete. 2024-04-21T08:47:31.433 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:47:31.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.534+0000 7f046159d640 1 -- 172.21.15.86:0/3729598688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f045c102b40 msgr2=0x7f045c106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.534+0000 7f046159d640 1 --2- 172.21.15.86:0/3729598688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f045c102b40 0x7f045c106b80 secure :-1 s=READY pgs=1652 cs=0 l=1 rev1=1 crypto rx=0x7f0450009900 tx=0x7f045002bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:31.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.534+0000 7f046159d640 1 -- 172.21.15.86:0/3729598688 shutdown_connections 2024-04-21T08:47:31.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.534+0000 7f046159d640 1 --2- 172.21.15.86:0/3729598688 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f045c107260 0x7f045c10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.534+0000 7f046159d640 1 --2- 172.21.15.86:0/3729598688 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f045c102b40 0x7f045c106b80 unknown :-1 s=CLOSED pgs=1652 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.534+0000 7f046159d640 1 --2- 172.21.15.86:0/3729598688 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f045c102190 0x7f045c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.534+0000 7f046159d640 1 -- 172.21.15.86:0/3729598688 >> 172.21.15.86:0/3729598688 conn(0x7f045c0793a0 msgr2=0x7f045c0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:31.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.534+0000 7f046159d640 1 -- 172.21.15.86:0/3729598688 shutdown_connections 2024-04-21T08:47:31.534 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.534+0000 7f046159d640 1 -- 172.21.15.86:0/3729598688 wait complete. 2024-04-21T08:47:31.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.535+0000 7f046159d640 1 Processor -- start 2024-04-21T08:47:31.535 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.535+0000 7f046159d640 1 -- start start 2024-04-21T08:47:31.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.535+0000 7f046159d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f045c102190 0x7f045c198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.535+0000 7f046159d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f045c107260 0x7f045c198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.535+0000 7f046159d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f045c19cfc0 0x7f045c19d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.535+0000 7f046159d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f045c112180 con 0x7f045c107260 2024-04-21T08:47:31.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.535+0000 7f046159d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f045c112000 con 0x7f045c19cfc0 2024-04-21T08:47:31.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.535+0000 7f046159d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f045c112300 con 0x7f045c102190 2024-04-21T08:47:31.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.535+0000 7f045a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f045c107260 0x7f045c198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.535+0000 7f045a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f045c107260 0x7f045c198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52682/0 (socket says 172.21.15.86:52682) 2024-04-21T08:47:31.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.535+0000 7f045a7fc640 1 -- 172.21.15.86:0/1558941475 learned_addr learned my addr 172.21.15.86:0/1558941475 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:31.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.535+0000 7f045affd640 1 --2- 172.21.15.86:0/1558941475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f045c102190 0x7f045c198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.537 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.535+0000 7f045b7fe640 1 --2- 172.21.15.86:0/1558941475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f045c19cfc0 0x7f045c19d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.535+0000 7f045a7fc640 1 -- 172.21.15.86:0/1558941475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f045c102190 msgr2=0x7f045c198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.535+0000 7f045a7fc640 1 --2- 172.21.15.86:0/1558941475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f045c102190 0x7f045c198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.535+0000 7f045a7fc640 1 -- 172.21.15.86:0/1558941475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f045c19cfc0 msgr2=0x7f045c19d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.535+0000 7f045a7fc640 1 --2- 172.21.15.86:0/1558941475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f045c19cfc0 0x7f045c19d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.535+0000 7f045a7fc640 1 -- 172.21.15.86:0/1558941475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f045c19dab0 con 0x7f045c107260 2024-04-21T08:47:31.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.535+0000 7f045b7fe640 1 --2- 172.21.15.86:0/1558941475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f045c19cfc0 0x7f045c19d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:31.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.536+0000 7f045a7fc640 1 --2- 172.21.15.86:0/1558941475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f045c107260 0x7f045c198c40 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f0450036040 tx=0x7f045002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:31.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.536+0000 7f0443fff640 1 -- 172.21.15.86:0/1558941475 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0450043020 con 0x7f045c107260 2024-04-21T08:47:31.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.536+0000 7f0443fff640 1 -- 172.21.15.86:0/1558941475 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f045002cbe0 con 0x7f045c107260 2024-04-21T08:47:31.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.536+0000 7f046159d640 1 -- 172.21.15.86:0/1558941475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f045c19dd40 con 0x7f045c107260 2024-04-21T08:47:31.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.536+0000 7f0443fff640 1 -- 172.21.15.86:0/1558941475 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0450003b80 con 0x7f045c107260 2024-04-21T08:47:31.538 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.536+0000 7f046159d640 1 -- 172.21.15.86:0/1558941475 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f045c1a5560 con 0x7f045c107260 2024-04-21T08:47:31.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.538+0000 7f0443fff640 1 -- 172.21.15.86:0/1558941475 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0450046020 con 0x7f045c107260 2024-04-21T08:47:31.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.538+0000 7f046418c740 1 Processor -- start 2024-04-21T08:47:31.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.538+0000 7f046418c740 1 -- start start 2024-04-21T08:47:31.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.538+0000 7f046418c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ef0cb34d0 0x560ef0cb38b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.538+0000 7f046418c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ef0cb3e80 0x560ef0cbc430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.538+0000 7f046418c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560ef0cbca00 0x560ef0cbee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.538+0000 7f046418c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560ef0c181f0 con 0x560ef0cb3e80 2024-04-21T08:47:31.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.538+0000 7f046418c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x560ef0bb54b0 con 0x560ef0cbca00 2024-04-21T08:47:31.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.538+0000 7f046418c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x560ef0c141b0 con 0x560ef0cb34d0 2024-04-21T08:47:31.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.539+0000 7f0443fff640 1 --2- 172.21.15.86:0/1558941475 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f04440a3d20 0x7f04440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.539+0000 7f045affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ef0cb3e80 0x560ef0cbc430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.546 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.539+0000 7f045affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ef0cb3e80 0x560ef0cbc430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52694/0 (socket says 172.21.15.86:52694) 2024-04-21T08:47:31.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.539+0000 7f045b7fe640 1 --2- 172.21.15.86:0/1558941475 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f04440a3d20 0x7f04440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.539+0000 7f045affd640 1 -- 172.21.15.86:0/377016452 learned_addr learned my addr 172.21.15.86:0/377016452 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:31.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.539+0000 7f0443fff640 1 -- 172.21.15.86:0/1558941475 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f04500f2d40 con 0x7f045c107260 2024-04-21T08:47:31.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.539+0000 7f045a7fc640 1 --2- 172.21.15.86:0/377016452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560ef0cbca00 0x560ef0cbee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.539+0000 7f045b7fe640 1 --2- 172.21.15.86:0/377016452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ef0cb34d0 0x560ef0cb38b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.539+0000 7f045affd640 1 -- 172.21.15.86:0/377016452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ef0cb34d0 msgr2=0x560ef0cb38b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.539+0000 7f045affd640 1 --2- 172.21.15.86:0/377016452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ef0cb34d0 0x560ef0cb38b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.539+0000 7f045affd640 1 -- 172.21.15.86:0/377016452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560ef0cbca00 msgr2=0x560ef0cbee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.539+0000 7f045affd640 1 --2- 172.21.15.86:0/377016452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560ef0cbca00 0x560ef0cbee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.539+0000 7f045affd640 1 -- 172.21.15.86:0/377016452 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ef0ba71b0 con 0x560ef0cb3e80 2024-04-21T08:47:31.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.539+0000 7f045b7fe640 1 --2- 172.21.15.86:0/377016452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ef0cb34d0 0x560ef0cb38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:31.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.539+0000 7f045affd640 1 --2- 172.21.15.86:0/377016452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ef0cb3e80 0x560ef0cbc430 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f044c00ad00 tx=0x7f044c00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=daee7e56d39a7b35 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:31.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.539+0000 7f045b7fe640 1 --2- 172.21.15.86:0/1558941475 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f04440a3d20 0x7f04440a61e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0454005c40 tx=0x7f0454009530 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:31.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.539+0000 7f04417fa640 1 -- 172.21.15.86:0/377016452 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c01d070 con 0x560ef0cb3e80 2024-04-21T08:47:31.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.539+0000 7f04417fa640 1 -- 172.21.15.86:0/377016452 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f044c018af0 con 0x560ef0cb3e80 2024-04-21T08:47:31.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.539+0000 7f04417fa640 1 -- 172.21.15.86:0/377016452 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c010b30 con 0x560ef0cb3e80 2024-04-21T08:47:31.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.540+0000 7f046418c740 1 -- 172.21.15.86:0/377016452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ef0cb3e80 msgr2=0x560ef0cbc430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.540+0000 7f046418c740 1 --2- 172.21.15.86:0/377016452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ef0cb3e80 0x560ef0cbc430 secure :-1 s=READY pgs=1654 cs=0 l=1 rev1=1 crypto rx=0x7f044c00ad00 tx=0x7f044c00f790 comp rx=0 tx=0).stop 2024-04-21T08:47:31.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.540+0000 7f046418c740 1 -- 172.21.15.86:0/377016452 shutdown_connections 2024-04-21T08:47:31.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.540+0000 7f046418c740 1 --2- 172.21.15.86:0/377016452 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560ef0cbca00 0x560ef0cbee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.540+0000 7f046418c740 1 --2- 172.21.15.86:0/377016452 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ef0cb3e80 0x560ef0cbc430 unknown :-1 s=CLOSED pgs=1654 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.540+0000 7f046418c740 1 --2- 172.21.15.86:0/377016452 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ef0cb34d0 0x560ef0cb38b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.540+0000 7f046418c740 1 -- 172.21.15.86:0/377016452 >> 172.21.15.86:0/377016452 conn(0x560ef0cadba0 msgr2=0x560ef0cadf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:31.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.540+0000 7f046418c740 1 -- 172.21.15.86:0/377016452 shutdown_connections 2024-04-21T08:47:31.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.540+0000 7f046418c740 1 -- 172.21.15.86:0/377016452 wait complete. 2024-04-21T08:47:31.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.540+0000 7f046418c740 1 -- start start 2024-04-21T08:47:31.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.540+0000 7f046418c740 1 Processor -- start 2024-04-21T08:47:31.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.540+0000 7f046418c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ef0cb34d0 0x560ef0cbc270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.541+0000 7f046418c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560ef0cb3e80 0x560ef0cb9600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.541+0000 7f046418c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ef0cbca00 0x560ef0cb9b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.541+0000 7f046418c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560ef0cbf970 con 0x560ef0cbca00 2024-04-21T08:47:31.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.541+0000 7f046418c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x560ef0bd1500 con 0x560ef0cb3e80 2024-04-21T08:47:31.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.541+0000 7f046418c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x560ef0cbfaf0 con 0x560ef0cb34d0 2024-04-21T08:47:31.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.541+0000 7f046418c740 10 client.0 _ll_register_callbacks cb 0x560ef0b9fb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:31.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.541+0000 7f045b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ef0cbca00 0x560ef0cb9b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.541+0000 7f045b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ef0cbca00 0x560ef0cb9b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52704/0 (socket says 172.21.15.86:52704) 2024-04-21T08:47:31.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.541+0000 7f045b7fe640 1 -- 172.21.15.86:0/3575568051 learned_addr learned my addr 172.21.15.86:0/3575568051 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:31.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.541+0000 7f045a7fc640 1 --2- 172.21.15.86:0/3575568051 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560ef0cb3e80 0x560ef0cb9600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.541+0000 7f045affd640 1 --2- 172.21.15.86:0/3575568051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ef0cb34d0 0x560ef0cbc270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.541+0000 7f045b7fe640 1 -- 172.21.15.86:0/3575568051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ef0cb34d0 msgr2=0x560ef0cbc270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.541+0000 7f045b7fe640 1 --2- 172.21.15.86:0/3575568051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ef0cb34d0 0x560ef0cbc270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.541+0000 7f045b7fe640 1 -- 172.21.15.86:0/3575568051 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560ef0cb3e80 msgr2=0x560ef0cb9600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.541+0000 7f045b7fe640 1 --2- 172.21.15.86:0/3575568051 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560ef0cb3e80 0x560ef0cb9600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.541+0000 7f045b7fe640 1 -- 172.21.15.86:0/3575568051 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ef092a3f0 con 0x560ef0cbca00 2024-04-21T08:47:31.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.541+0000 7f045affd640 1 --2- 172.21.15.86:0/3575568051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ef0cb34d0 0x560ef0cbc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:31.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.541+0000 7f045b7fe640 1 --2- 172.21.15.86:0/3575568051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ef0cbca00 0x560ef0cb9b40 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f0454010980 tx=0x7f0454004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4f6edd2cb13b0c70 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:31.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.541+0000 7f043eff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:47:31.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.541+0000 7f043eff5640 1 -- 172.21.15.86:0/3575568051 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045400fcf0 con 0x560ef0cbca00 2024-04-21T08:47:31.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.541+0000 7f043eff5640 1 -- 172.21.15.86:0/3575568051 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f04540263f0 con 0x560ef0cbca00 2024-04-21T08:47:31.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.541+0000 7f046418c740 1 -- 172.21.15.86:0/3575568051 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f044c018af0 con 0x560ef0cbca00 2024-04-21T08:47:31.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.541+0000 7f043eff5640 1 -- 172.21.15.86:0/3575568051 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f045402e050 con 0x560ef0cbca00 2024-04-21T08:47:31.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.541+0000 7f046418c740 10 client.19176 fetch_fsmap learned FSMap version 29 2024-04-21T08:47:31.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.541+0000 7f046418c740 1 -- 172.21.15.86:0/3575568051 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560ef09f47d0 con 0x560ef0cbca00 2024-04-21T08:47:31.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.542+0000 7f043eff5640 1 -- 172.21.15.86:0/3575568051 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f044c018af0 con 0x560ef0cbca00 2024-04-21T08:47:31.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.542+0000 7f043eff5640 1 -- 172.21.15.86:0/3575568051 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f045401e580 con 0x560ef0cbca00 2024-04-21T08:47:31.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.542+0000 7f046418c740 10 client.19176 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:47:31.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.542+0000 7f046418c740 10 client.19176 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:47:31.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.542+0000 7f046418c740 20 client.19176 populate_metadata read hostname 'smithi086' 2024-04-21T08:47:31.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.542+0000 7f046418c740 1 --2- 172.21.15.86:0/3575568051 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x560ef0cccc40 0x560ef0ccf0c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.542+0000 7f046418c740 4 client.19176 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:31.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.542+0000 7f046418c740 1 -- 172.21.15.86:0/3575568051 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560ef0bb7930 con 0x560ef0cccc40 2024-04-21T08:47:31.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.542+0000 7f045affd640 1 --2- 172.21.15.86:0/3575568051 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x560ef0cccc40 0x560ef0ccf0c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.542+0000 7f043eff5640 10 client.19176 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:47:31.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.542+0000 7f045affd640 1 --2- 172.21.15.86:0/3575568051 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x560ef0cccc40 0x560ef0ccf0c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=8a3ff35a3005bf5e server_cookie=24c4021bedbd991b in_seq=0 out_seq=0 2024-04-21T08:47:31.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.543+0000 7f043eff5640 1 -- 172.21.15.86:0/3575568051 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560ef0bb7930 con 0x560ef0cccc40 2024-04-21T08:47:31.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.543+0000 7f043eff5640 10 client.19176 handle_command_reply: tid=0 2024-04-21T08:47:31.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.543+0000 7f046418c740 1 client.19176 shutdown 2024-04-21T08:47:31.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.543+0000 7f046418c740 1 -- 172.21.15.86:0/3575568051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ef0cbca00 msgr2=0x560ef0cb9b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.543+0000 7f046418c740 1 --2- 172.21.15.86:0/3575568051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ef0cbca00 0x560ef0cb9b40 secure :-1 s=READY pgs=1655 cs=0 l=1 rev1=1 crypto rx=0x7f0454010980 tx=0x7f0454004a40 comp rx=0 tx=0).stop 2024-04-21T08:47:31.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.543+0000 7f046418c740 1 -- 172.21.15.86:0/3575568051 shutdown_connections 2024-04-21T08:47:31.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.543+0000 7f046418c740 1 --2- 172.21.15.86:0/3575568051 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x560ef0cccc40 0x560ef0ccf0c0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.543+0000 7f046418c740 1 --2- 172.21.15.86:0/3575568051 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ef0cbca00 0x560ef0cb9b40 unknown :-1 s=CLOSED pgs=1655 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.543+0000 7f046418c740 1 --2- 172.21.15.86:0/3575568051 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560ef0cb3e80 0x560ef0cb9600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.543+0000 7f043eff5640 0 client.19176 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:47:31.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.543+0000 7f046418c740 1 --2- 172.21.15.86:0/3575568051 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ef0cb34d0 0x560ef0cbc270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.543+0000 7f046418c740 1 -- 172.21.15.86:0/3575568051 >> 172.21.15.86:0/3575568051 conn(0x560ef0cadba0 msgr2=0x560ef0caf8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:31.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.543+0000 7f046418c740 1 -- 172.21.15.86:0/3575568051 shutdown_connections 2024-04-21T08:47:31.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.543+0000 7f046418c740 1 -- 172.21.15.86:0/3575568051 wait complete. 2024-04-21T08:47:31.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.543+0000 7f046418c740 20 client.19176 trim_cache size 0 max 16384 2024-04-21T08:47:31.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.556+0000 7f046418c740 1 Processor -- start 2024-04-21T08:47:31.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.556+0000 7f046418c740 1 -- start start 2024-04-21T08:47:31.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.556+0000 7f046418c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ef0cadba0 0x560ef0cc8890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.556+0000 7f046418c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560ef0cb34d0 0x560ef0caf700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.557+0000 7f046418c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ef0cb3e80 0x560ef0cbbd30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.557+0000 7f046418c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f045c1af360 con 0x560ef0cadba0 2024-04-21T08:47:31.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.557+0000 7f046418c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f045c1b0ea0 con 0x560ef0cb34d0 2024-04-21T08:47:31.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.557+0000 7f046418c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f045c1afa30 con 0x560ef0cb3e80 2024-04-21T08:47:31.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.557+0000 7f045affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ef0cadba0 0x560ef0cc8890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.557+0000 7f045affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ef0cadba0 0x560ef0cc8890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52716/0 (socket says 172.21.15.86:52716) 2024-04-21T08:47:31.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.557+0000 7f045affd640 1 -- 172.21.15.86:0/250046229 learned_addr learned my addr 172.21.15.86:0/250046229 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:31.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.557+0000 7f045a7fc640 1 --2- 172.21.15.86:0/250046229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560ef0cb34d0 0x560ef0caf700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.557+0000 7f045affd640 1 -- 172.21.15.86:0/250046229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ef0cb3e80 msgr2=0x560ef0cbbd30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.557+0000 7f045affd640 1 --2- 172.21.15.86:0/250046229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ef0cb3e80 0x560ef0cbbd30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.557+0000 7f045affd640 1 -- 172.21.15.86:0/250046229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560ef0cb34d0 msgr2=0x560ef0caf700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.557+0000 7f045affd640 1 --2- 172.21.15.86:0/250046229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560ef0cb34d0 0x560ef0caf700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.557+0000 7f045affd640 1 -- 172.21.15.86:0/250046229 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560ef0ccb790 con 0x560ef0cadba0 2024-04-21T08:47:31.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.557+0000 7f045affd640 1 --2- 172.21.15.86:0/250046229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ef0cadba0 0x560ef0cc8890 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f044c0106c0 tx=0x7f044c010790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=647f98f9d82b93f3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:31.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.557+0000 7f0440ff9640 1 -- 172.21.15.86:0/250046229 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c02a070 con 0x560ef0cadba0 2024-04-21T08:47:31.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.557+0000 7f0440ff9640 1 -- 172.21.15.86:0/250046229 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f044c01e040 con 0x560ef0cadba0 2024-04-21T08:47:31.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.557+0000 7f0440ff9640 1 -- 172.21.15.86:0/250046229 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f044c01d040 con 0x560ef0cadba0 2024-04-21T08:47:31.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.558+0000 7f046418c740 1 -- 172.21.15.86:0/250046229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ef0cadba0 msgr2=0x560ef0cc8890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.558+0000 7f046418c740 1 --2- 172.21.15.86:0/250046229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ef0cadba0 0x560ef0cc8890 secure :-1 s=READY pgs=1656 cs=0 l=1 rev1=1 crypto rx=0x7f044c0106c0 tx=0x7f044c010790 comp rx=0 tx=0).stop 2024-04-21T08:47:31.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.558+0000 7f046418c740 1 -- 172.21.15.86:0/250046229 shutdown_connections 2024-04-21T08:47:31.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.558+0000 7f046418c740 1 --2- 172.21.15.86:0/250046229 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ef0cb3e80 0x560ef0cbbd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.558+0000 7f046418c740 1 --2- 172.21.15.86:0/250046229 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560ef0cb34d0 0x560ef0caf700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.558+0000 7f046418c740 1 --2- 172.21.15.86:0/250046229 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ef0cadba0 0x560ef0cc8890 unknown :-1 s=CLOSED pgs=1656 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.558+0000 7f046418c740 1 -- 172.21.15.86:0/250046229 >> 172.21.15.86:0/250046229 conn(0x560ef0cb9ca0 msgr2=0x560ef0caeff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:31.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.558+0000 7f046418c740 1 -- 172.21.15.86:0/250046229 shutdown_connections 2024-04-21T08:47:31.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.558+0000 7f046418c740 1 -- 172.21.15.86:0/250046229 wait complete. 2024-04-21T08:47:31.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.558+0000 7f046418c740 1 -- start start 2024-04-21T08:47:31.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.558+0000 7f046418c740 1 Processor -- start 2024-04-21T08:47:31.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.558+0000 7f046418c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560ef0cadba0 0x560ef0cc4fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.558+0000 7f046418c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ef0cb34d0 0x560ef0cc0360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.558+0000 7f046418c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ef0cb3e80 0x560ef0cc08a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.558+0000 7f046418c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f045c1b07d0 con 0x560ef0cb34d0 2024-04-21T08:47:31.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.558+0000 7f046418c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f045c1b0100 con 0x560ef0cadba0 2024-04-21T08:47:31.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.558+0000 7f046418c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f045c1ad360 con 0x560ef0cb3e80 2024-04-21T08:47:31.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.558+0000 7f046418c740 10 client.0 _ll_register_callbacks cb 0x7f0454010630 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:31.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.558+0000 7f045a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ef0cb34d0 0x560ef0cc0360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.558+0000 7f045a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ef0cb34d0 0x560ef0cc0360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52724/0 (socket says 172.21.15.86:52724) 2024-04-21T08:47:31.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.558+0000 7f045a7fc640 1 -- 172.21.15.86:0/1297786373 learned_addr learned my addr 172.21.15.86:0/1297786373 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:31.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.558+0000 7f045b7fe640 1 --2- 172.21.15.86:0/1297786373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ef0cb3e80 0x560ef0cc08a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.559+0000 7f045affd640 1 --2- 172.21.15.86:0/1297786373 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560ef0cadba0 0x560ef0cc4fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.559+0000 7f045a7fc640 1 -- 172.21.15.86:0/1297786373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ef0cb3e80 msgr2=0x560ef0cc08a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.559+0000 7f045a7fc640 1 --2- 172.21.15.86:0/1297786373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ef0cb3e80 0x560ef0cc08a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.559+0000 7f045a7fc640 1 -- 172.21.15.86:0/1297786373 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560ef0cadba0 msgr2=0x560ef0cc4fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.559+0000 7f045a7fc640 1 --2- 172.21.15.86:0/1297786373 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560ef0cadba0 0x560ef0cc4fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.559+0000 7f045a7fc640 1 -- 172.21.15.86:0/1297786373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560ef0cc0e10 con 0x560ef0cb34d0 2024-04-21T08:47:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.559+0000 7f045b7fe640 1 --2- 172.21.15.86:0/1297786373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ef0cb3e80 0x560ef0cc08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.559+0000 7f045a7fc640 1 --2- 172.21.15.86:0/1297786373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ef0cb34d0 0x560ef0cc0360 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f0450032040 tx=0x7f0450003680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e2eb718a932c6c6f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.559+0000 7f043eff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:47:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.559+0000 7f043eff5640 1 -- 172.21.15.86:0/1297786373 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f045003c350 con 0x560ef0cb34d0 2024-04-21T08:47:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.559+0000 7f043eff5640 1 -- 172.21.15.86:0/1297786373 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f04500bb090 con 0x560ef0cb34d0 2024-04-21T08:47:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.559+0000 7f046418c740 1 -- 172.21.15.86:0/1297786373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f044c01e040 con 0x560ef0cb34d0 2024-04-21T08:47:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.559+0000 7f043eff5640 1 -- 172.21.15.86:0/1297786373 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f04500bea20 con 0x560ef0cb34d0 2024-04-21T08:47:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.559+0000 7f046418c740 10 client.19182 fetch_fsmap learned FSMap version 29 2024-04-21T08:47:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.559+0000 7f046418c740 1 -- 172.21.15.86:0/1297786373 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560ef0d32fe0 con 0x560ef0cb34d0 2024-04-21T08:47:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.559+0000 7f043eff5640 1 -- 172.21.15.86:0/1297786373 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f044c01e040 con 0x560ef0cb34d0 2024-04-21T08:47:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.559+0000 7f043eff5640 1 -- 172.21.15.86:0/1297786373 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f04500bec80 con 0x560ef0cb34d0 2024-04-21T08:47:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.559+0000 7f046418c740 10 client.19182 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:47:31.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.559+0000 7f046418c740 10 client.19182 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:47:31.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.559+0000 7f046418c740 20 client.19182 populate_metadata read hostname 'smithi086' 2024-04-21T08:47:31.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.559+0000 7f046418c740 1 --2- 172.21.15.86:0/1297786373 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x560ef0cbca00 0x560ef0d355c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.559+0000 7f046418c740 4 client.19182 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:47:31.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.559+0000 7f046418c740 1 -- 172.21.15.86:0/1297786373 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560ef0c25c10 con 0x560ef0cbca00 2024-04-21T08:47:31.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.560+0000 7f045affd640 1 --2- 172.21.15.86:0/1297786373 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x560ef0cbca00 0x560ef0d355c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.562+0000 7f043eff5640 10 client.19182 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:47:31.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.562+0000 7f045affd640 1 --2- 172.21.15.86:0/1297786373 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x560ef0cbca00 0x560ef0d355c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=eceea6241c043793 server_cookie=9a140a06caeee49d in_seq=0 out_seq=0 2024-04-21T08:47:31.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.562+0000 7f043eff5640 1 -- 172.21.15.86:0/1297786373 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560ef0c25c10 con 0x560ef0cbca00 2024-04-21T08:47:31.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.562+0000 7f043eff5640 10 client.19182 handle_command_reply: tid=0 2024-04-21T08:47:31.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.562+0000 7f046418c740 1 client.19182 shutdown 2024-04-21T08:47:31.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.562+0000 7f046418c740 1 -- 172.21.15.86:0/1297786373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ef0cb34d0 msgr2=0x560ef0cc0360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.563+0000 7f046418c740 1 --2- 172.21.15.86:0/1297786373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ef0cb34d0 0x560ef0cc0360 secure :-1 s=READY pgs=1657 cs=0 l=1 rev1=1 crypto rx=0x7f0450032040 tx=0x7f0450003680 comp rx=0 tx=0).stop 2024-04-21T08:47:31.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.563+0000 7f046418c740 1 -- 172.21.15.86:0/1297786373 shutdown_connections 2024-04-21T08:47:31.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.563+0000 7f046418c740 1 --2- 172.21.15.86:0/1297786373 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560ef0cb3e80 0x560ef0cc08a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.563+0000 7f046418c740 1 --2- 172.21.15.86:0/1297786373 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x560ef0cbca00 0x560ef0d355c0 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.563+0000 7f046418c740 1 --2- 172.21.15.86:0/1297786373 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560ef0cb34d0 0x560ef0cc0360 unknown :-1 s=CLOSED pgs=1657 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.563+0000 7f046418c740 1 --2- 172.21.15.86:0/1297786373 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560ef0cadba0 0x560ef0cc4fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.563+0000 7f043eff5640 0 client.19182 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:47:31.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.563+0000 7f046418c740 1 -- 172.21.15.86:0/1297786373 >> 172.21.15.86:0/1297786373 conn(0x560ef0cb9ca0 msgr2=0x560ef0cac370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:31.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.563+0000 7f046418c740 1 -- 172.21.15.86:0/1297786373 shutdown_connections 2024-04-21T08:47:31.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.563+0000 7f046418c740 1 -- 172.21.15.86:0/1297786373 wait complete. 2024-04-21T08:47:31.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.563+0000 7f046418c740 20 client.19182 trim_cache size 0 max 16384 2024-04-21T08:47:31.568 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:47:31.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.563+0000 7f046159d640 1 -- 172.21.15.86:0/1558941475 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f04440a3d20 msgr2=0x7f04440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.563+0000 7f046159d640 1 --2- 172.21.15.86:0/1558941475 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f04440a3d20 0x7f04440a61e0 secure :-1 s=READY pgs=270 cs=0 l=1 rev1=1 crypto rx=0x7f0454005c40 tx=0x7f0454009530 comp rx=0 tx=0).stop 2024-04-21T08:47:31.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.563+0000 7f046159d640 1 -- 172.21.15.86:0/1558941475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f045c107260 msgr2=0x7f045c198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.563+0000 7f046159d640 1 --2- 172.21.15.86:0/1558941475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f045c107260 0x7f045c198c40 secure :-1 s=READY pgs=1653 cs=0 l=1 rev1=1 crypto rx=0x7f0450036040 tx=0x7f045002c7e0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.563+0000 7f046159d640 1 -- 172.21.15.86:0/1558941475 shutdown_connections 2024-04-21T08:47:31.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.563+0000 7f046159d640 1 --2- 172.21.15.86:0/1558941475 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f04440a3d20 0x7f04440a61e0 unknown :-1 s=CLOSED pgs=270 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.563+0000 7f046159d640 1 --2- 172.21.15.86:0/1558941475 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f045c19cfc0 0x7f045c19d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.563+0000 7f046159d640 1 --2- 172.21.15.86:0/1558941475 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f045c107260 0x7f045c198c40 unknown :-1 s=CLOSED pgs=1653 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.563+0000 7f046159d640 1 --2- 172.21.15.86:0/1558941475 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f045c102190 0x7f045c198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.563+0000 7f046159d640 1 -- 172.21.15.86:0/1558941475 >> 172.21.15.86:0/1558941475 conn(0x7f045c0793a0 msgr2=0x7f045c0fff70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:31.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.563+0000 7f046159d640 1 -- 172.21.15.86:0/1558941475 shutdown_connections 2024-04-21T08:47:31.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.563+0000 7f046159d640 1 -- 172.21.15.86:0/1558941475 wait complete. 2024-04-21T08:47:31.573 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:47:31.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.673+0000 7fbb76fb6640 1 -- 172.21.15.86:0/793578106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb7006af10 msgr2=0x7fbb7010edf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.673+0000 7fbb76fb6640 1 --2- 172.21.15.86:0/793578106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb7006af10 0x7fbb7010edf0 secure :-1 s=READY pgs=1658 cs=0 l=1 rev1=1 crypto rx=0x7fbb68009cd0 tx=0x7fbb6802c180 comp rx=0 tx=0).stop 2024-04-21T08:47:31.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.673+0000 7fbb76fb6640 1 -- 172.21.15.86:0/793578106 shutdown_connections 2024-04-21T08:47:31.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.673+0000 7fbb76fb6640 1 --2- 172.21.15.86:0/793578106 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb7006af10 0x7fbb7010edf0 unknown :-1 s=CLOSED pgs=1658 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.673+0000 7fbb76fb6640 1 --2- 172.21.15.86:0/793578106 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb7006a550 0x7fbb7006a9d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.673+0000 7fbb76fb6640 1 --2- 172.21.15.86:0/793578106 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb70108150 0x7fbb70108530 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.673+0000 7fbb76fb6640 1 -- 172.21.15.86:0/793578106 >> 172.21.15.86:0/793578106 conn(0x7fbb70076c50 msgr2=0x7fbb70077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:31.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.673+0000 7fbb76fb6640 1 -- 172.21.15.86:0/793578106 shutdown_connections 2024-04-21T08:47:31.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.673+0000 7fbb76fb6640 1 -- 172.21.15.86:0/793578106 wait complete. 2024-04-21T08:47:31.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.674+0000 7fbb76fb6640 1 Processor -- start 2024-04-21T08:47:31.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.674+0000 7fbb76fb6640 1 -- start start 2024-04-21T08:47:31.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.674+0000 7fbb76fb6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb7006a550 0x7fbb70198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.674+0000 7fbb76fb6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb7006af10 0x7fbb70198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.674+0000 7fbb76fb6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb70108150 0x7fbb7019d020 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.674+0000 7fbb76fb6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbb70112180 con 0x7fbb7006af10 2024-04-21T08:47:31.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.674+0000 7fbb76fb6640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbb70112000 con 0x7fbb7006a550 2024-04-21T08:47:31.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.674+0000 7fbb76fb6640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbb70112300 con 0x7fbb70108150 2024-04-21T08:47:31.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.674+0000 7fbb6ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb7006af10 0x7fbb70198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.674+0000 7fbb6ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb7006af10 0x7fbb70198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52742/0 (socket says 172.21.15.86:52742) 2024-04-21T08:47:31.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.674+0000 7fbb6ffff640 1 -- 172.21.15.86:0/2164230098 learned_addr learned my addr 172.21.15.86:0/2164230098 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:31.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.674+0000 7fbb74d52640 1 --2- 172.21.15.86:0/2164230098 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb7006a550 0x7fbb70198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.674+0000 7fbb75553640 1 --2- 172.21.15.86:0/2164230098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb70108150 0x7fbb7019d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.674+0000 7fbb6ffff640 1 -- 172.21.15.86:0/2164230098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb70108150 msgr2=0x7fbb7019d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.674+0000 7fbb6ffff640 1 --2- 172.21.15.86:0/2164230098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb70108150 0x7fbb7019d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.674+0000 7fbb6ffff640 1 -- 172.21.15.86:0/2164230098 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb7006a550 msgr2=0x7fbb70198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.674+0000 7fbb6ffff640 1 --2- 172.21.15.86:0/2164230098 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb7006a550 0x7fbb70198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.674+0000 7fbb6ffff640 1 -- 172.21.15.86:0/2164230098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb7019d750 con 0x7fbb7006af10 2024-04-21T08:47:31.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.674+0000 7fbb6ffff640 1 --2- 172.21.15.86:0/2164230098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb7006af10 0x7fbb70198ca0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c00c890 tx=0x7fbb5c00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:31.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.675+0000 7fbb6dffb640 1 -- 172.21.15.86:0/2164230098 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c004220 con 0x7fbb7006af10 2024-04-21T08:47:31.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.675+0000 7fbb6dffb640 1 -- 172.21.15.86:0/2164230098 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb5c0043c0 con 0x7fbb7006af10 2024-04-21T08:47:31.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.675+0000 7fbb6dffb640 1 -- 172.21.15.86:0/2164230098 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c004a00 con 0x7fbb7006af10 2024-04-21T08:47:31.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.675+0000 7fbb76fb6640 1 -- 172.21.15.86:0/2164230098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb7019da40 con 0x7fbb7006af10 2024-04-21T08:47:31.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.675+0000 7fbb76fb6640 1 -- 172.21.15.86:0/2164230098 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb701a52d0 con 0x7fbb7006af10 2024-04-21T08:47:31.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.677+0000 7fbb6dffb640 1 -- 172.21.15.86:0/2164230098 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbb5c0079b0 con 0x7fbb7006af10 2024-04-21T08:47:31.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.677+0000 7fbb79bac740 1 Processor -- start 2024-04-21T08:47:31.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.677+0000 7fbb79bac740 1 -- start start 2024-04-21T08:47:31.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.678+0000 7fbb79bac740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4a1ad4d0 0x55ec4a1ad8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.678+0000 7fbb79bac740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4a1ade80 0x55ec4a1b6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.678+0000 7fbb79bac740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec4a1b6a00 0x55ec4a1b8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.678+0000 7fbb79bac740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ec4a1121f0 con 0x55ec4a1ad4d0 2024-04-21T08:47:31.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.678+0000 7fbb79bac740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55ec4a0af4b0 con 0x55ec4a1ade80 2024-04-21T08:47:31.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.678+0000 7fbb79bac740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ec4a10e1b0 con 0x55ec4a1b6a00 2024-04-21T08:47:31.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.678+0000 7fbb6dffb640 1 --2- 172.21.15.86:0/2164230098 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbb600a3da0 0x7fbb600a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.678+0000 7fbb75553640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4a1ad4d0 0x55ec4a1ad8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.678+0000 7fbb75553640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4a1ad4d0 0x55ec4a1ad8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52758/0 (socket says 172.21.15.86:52758) 2024-04-21T08:47:31.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.678+0000 7fbb75553640 1 -- 172.21.15.86:0/1617199072 learned_addr learned my addr 172.21.15.86:0/1617199072 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:31.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.678+0000 7fbb75553640 1 --2- 172.21.15.86:0/2164230098 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbb600a3da0 0x7fbb600a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.678+0000 7fbb6dffb640 1 -- 172.21.15.86:0/2164230098 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fbb5c0d0900 con 0x7fbb7006af10 2024-04-21T08:47:31.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.678+0000 7fbb74d52640 1 --2- 172.21.15.86:0/1617199072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4a1ade80 0x55ec4a1b6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.678+0000 7fbb6ffff640 1 --2- 172.21.15.86:0/1617199072 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec4a1b6a00 0x55ec4a1b8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.678+0000 7fbb75553640 1 -- 172.21.15.86:0/1617199072 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec4a1b6a00 msgr2=0x55ec4a1b8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.678+0000 7fbb75553640 1 --2- 172.21.15.86:0/1617199072 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec4a1b6a00 0x55ec4a1b8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.678+0000 7fbb75553640 1 -- 172.21.15.86:0/1617199072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4a1ade80 msgr2=0x55ec4a1b6430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.678+0000 7fbb75553640 1 --2- 172.21.15.86:0/1617199072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4a1ade80 0x55ec4a1b6430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.678+0000 7fbb75553640 1 -- 172.21.15.86:0/1617199072 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec4a0a11b0 con 0x55ec4a1ad4d0 2024-04-21T08:47:31.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.678+0000 7fbb75553640 1 --2- 172.21.15.86:0/1617199072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4a1ad4d0 0x55ec4a1ad8b0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fbb68009430 tx=0x7fbb68003f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d496d7b123fd56b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:31.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.678+0000 7fbb75553640 1 --2- 172.21.15.86:0/2164230098 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbb600a3da0 0x7fbb600a6260 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fbb6802c6c0 tx=0x7fbb68040000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:31.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.678+0000 7fbb5affd640 1 -- 172.21.15.86:0/1617199072 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb68039040 con 0x55ec4a1ad4d0 2024-04-21T08:47:31.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.678+0000 7fbb5affd640 1 -- 172.21.15.86:0/1617199072 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb68050070 con 0x55ec4a1ad4d0 2024-04-21T08:47:31.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.678+0000 7fbb5affd640 1 -- 172.21.15.86:0/1617199072 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb68034440 con 0x55ec4a1ad4d0 2024-04-21T08:47:31.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.679+0000 7fbb79bac740 1 -- 172.21.15.86:0/1617199072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4a1ad4d0 msgr2=0x55ec4a1ad8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.679+0000 7fbb79bac740 1 --2- 172.21.15.86:0/1617199072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4a1ad4d0 0x55ec4a1ad8b0 secure :-1 s=READY pgs=1660 cs=0 l=1 rev1=1 crypto rx=0x7fbb68009430 tx=0x7fbb68003f50 comp rx=0 tx=0).stop 2024-04-21T08:47:31.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.679+0000 7fbb79bac740 1 -- 172.21.15.86:0/1617199072 shutdown_connections 2024-04-21T08:47:31.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.679+0000 7fbb79bac740 1 --2- 172.21.15.86:0/1617199072 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec4a1b6a00 0x55ec4a1b8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.679+0000 7fbb79bac740 1 --2- 172.21.15.86:0/1617199072 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4a1ade80 0x55ec4a1b6430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.679+0000 7fbb79bac740 1 --2- 172.21.15.86:0/1617199072 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4a1ad4d0 0x55ec4a1ad8b0 unknown :-1 s=CLOSED pgs=1660 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.679+0000 7fbb79bac740 1 -- 172.21.15.86:0/1617199072 >> 172.21.15.86:0/1617199072 conn(0x55ec4a1a7ba0 msgr2=0x55ec4a1a7f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:31.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.679+0000 7fbb79bac740 1 -- 172.21.15.86:0/1617199072 shutdown_connections 2024-04-21T08:47:31.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.679+0000 7fbb79bac740 1 -- 172.21.15.86:0/1617199072 wait complete. 2024-04-21T08:47:31.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.679+0000 7fbb79bac740 1 -- start start 2024-04-21T08:47:31.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.680+0000 7fbb79bac740 1 Processor -- start 2024-04-21T08:47:31.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.680+0000 7fbb79bac740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec4a1ad4d0 0x55ec4a1b6340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.680+0000 7fbb79bac740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4a1ade80 0x55ec4a1b36d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.680+0000 7fbb79bac740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4a1b6a00 0x55ec4a1b3ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.680+0000 7fbb79bac740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ec4a1b9910 con 0x55ec4a1b6a00 2024-04-21T08:47:31.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.680+0000 7fbb79bac740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55ec4a0cb500 con 0x55ec4a1ade80 2024-04-21T08:47:31.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.680+0000 7fbb79bac740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ec4a1b9a90 con 0x55ec4a1ad4d0 2024-04-21T08:47:31.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.680+0000 7fbb79bac740 10 client.0 _ll_register_callbacks cb 0x55ec4a099b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:31.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.680+0000 7fbb75553640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4a1b6a00 0x55ec4a1b3ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.680+0000 7fbb75553640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4a1b6a00 0x55ec4a1b3ce0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52766/0 (socket says 172.21.15.86:52766) 2024-04-21T08:47:31.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.680+0000 7fbb75553640 1 -- 172.21.15.86:0/587539130 learned_addr learned my addr 172.21.15.86:0/587539130 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:31.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.680+0000 7fbb74d52640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec4a1ad4d0 0x55ec4a1b6340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.680+0000 7fbb6ffff640 1 --2- 172.21.15.86:0/587539130 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4a1ade80 0x55ec4a1b36d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.680+0000 7fbb75553640 1 -- 172.21.15.86:0/587539130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec4a1ad4d0 msgr2=0x55ec4a1b6340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.680+0000 7fbb75553640 1 --2- 172.21.15.86:0/587539130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec4a1ad4d0 0x55ec4a1b6340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.680+0000 7fbb75553640 1 -- 172.21.15.86:0/587539130 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4a1ade80 msgr2=0x55ec4a1b36d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.680+0000 7fbb75553640 1 --2- 172.21.15.86:0/587539130 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4a1ade80 0x55ec4a1b36d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.680+0000 7fbb75553640 1 -- 172.21.15.86:0/587539130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec49e243f0 con 0x55ec4a1b6a00 2024-04-21T08:47:31.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.680+0000 7fbb75553640 1 --2- 172.21.15.86:0/587539130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4a1b6a00 0x55ec4a1b3ce0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fbb68005ba0 tx=0x7fbb68034440 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae59e3c9e58ef66c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:31.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.680+0000 7fbb4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:47:31.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.681+0000 7fbb4ffff640 1 -- 172.21.15.86:0/587539130 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb68059060 con 0x55ec4a1b6a00 2024-04-21T08:47:31.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.681+0000 7fbb4ffff640 1 -- 172.21.15.86:0/587539130 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fbb68032560 con 0x55ec4a1b6a00 2024-04-21T08:47:31.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.681+0000 7fbb79bac740 1 -- 172.21.15.86:0/587539130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb68050070 con 0x55ec4a1b6a00 2024-04-21T08:47:31.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.681+0000 7fbb4ffff640 1 -- 172.21.15.86:0/587539130 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbb68060050 con 0x55ec4a1b6a00 2024-04-21T08:47:31.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.681+0000 7fbb79bac740 10 client.19194 fetch_fsmap learned FSMap version 29 2024-04-21T08:47:31.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.681+0000 7fbb79bac740 1 -- 172.21.15.86:0/587539130 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ec49eee7d0 con 0x55ec4a1b6a00 2024-04-21T08:47:31.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.681+0000 7fbb4ffff640 1 -- 172.21.15.86:0/587539130 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb68050070 con 0x55ec4a1b6a00 2024-04-21T08:47:31.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.681+0000 7fbb4ffff640 1 -- 172.21.15.86:0/587539130 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbb68034cd0 con 0x55ec4a1b6a00 2024-04-21T08:47:31.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.681+0000 7fbb79bac740 10 client.19194 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:47:31.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.681+0000 7fbb79bac740 10 client.19194 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:31.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.681+0000 7fbb79bac740 20 client.19194 populate_metadata read hostname 'smithi086' 2024-04-21T08:47:31.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.681+0000 7fbb79bac740 1 --2- 172.21.15.86:0/587539130 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ec4a1c6ac0 0x55ec4a1c8eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.681+0000 7fbb79bac740 4 client.19194 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:31.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.681+0000 7fbb79bac740 1 -- 172.21.15.86:0/587539130 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ec4a0b1930 con 0x55ec4a1c6ac0 2024-04-21T08:47:31.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.681+0000 7fbb74d52640 1 --2- 172.21.15.86:0/587539130 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ec4a1c6ac0 0x55ec4a1c8eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.682+0000 7fbb4ffff640 10 client.19194 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:31.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.682+0000 7fbb74d52640 1 --2- 172.21.15.86:0/587539130 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ec4a1c6ac0 0x55ec4a1c8eb0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7c4bd6ea3ce7e707 server_cookie=d40e20ad06d16303 in_seq=0 out_seq=0 2024-04-21T08:47:31.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.682+0000 7fbb4ffff640 1 -- 172.21.15.86:0/587539130 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ec4a0b1930 con 0x55ec4a1c6ac0 2024-04-21T08:47:31.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.682+0000 7fbb4ffff640 10 client.19194 handle_command_reply: tid=0 2024-04-21T08:47:31.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.682+0000 7fbb79bac740 1 client.19194 shutdown 2024-04-21T08:47:31.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.683+0000 7fbb79bac740 1 -- 172.21.15.86:0/587539130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4a1b6a00 msgr2=0x55ec4a1b3ce0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.683+0000 7fbb79bac740 1 --2- 172.21.15.86:0/587539130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4a1b6a00 0x55ec4a1b3ce0 secure :-1 s=READY pgs=1661 cs=0 l=1 rev1=1 crypto rx=0x7fbb68005ba0 tx=0x7fbb68034440 comp rx=0 tx=0).stop 2024-04-21T08:47:31.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.683+0000 7fbb79bac740 1 -- 172.21.15.86:0/587539130 shutdown_connections 2024-04-21T08:47:31.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.683+0000 7fbb79bac740 1 --2- 172.21.15.86:0/587539130 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ec4a1c6ac0 0x55ec4a1c8eb0 crc :-1 s=READY pgs=259 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.683+0000 7fbb79bac740 1 --2- 172.21.15.86:0/587539130 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4a1b6a00 0x55ec4a1b3ce0 unknown :-1 s=CLOSED pgs=1661 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.683+0000 7fbb79bac740 1 --2- 172.21.15.86:0/587539130 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4a1ade80 0x55ec4a1b36d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.683+0000 7fbb79bac740 1 --2- 172.21.15.86:0/587539130 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec4a1ad4d0 0x55ec4a1b6340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.683+0000 7fbb79bac740 1 -- 172.21.15.86:0/587539130 >> 172.21.15.86:0/587539130 conn(0x55ec4a1a7ba0 msgr2=0x55ec4a1a98f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:31.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.683+0000 7fbb4ffff640 0 client.19194 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:31.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.683+0000 7fbb79bac740 1 -- 172.21.15.86:0/587539130 shutdown_connections 2024-04-21T08:47:31.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.683+0000 7fbb79bac740 1 -- 172.21.15.86:0/587539130 wait complete. 2024-04-21T08:47:31.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.683+0000 7fbb79bac740 20 client.19194 trim_cache size 0 max 16384 2024-04-21T08:47:31.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.696+0000 7fbb79bac740 1 Processor -- start 2024-04-21T08:47:31.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.696+0000 7fbb79bac740 1 -- start start 2024-04-21T08:47:31.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.696+0000 7fbb79bac740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4a1ad4d0 0x55ec4a1a6e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.696+0000 7fbb79bac740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4a1ade80 0x55ec4a1c2830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.696+0000 7fbb79bac740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec4a1b6a00 0x55ec4a1b5e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.696+0000 7fbb79bac740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbb701af3c0 con 0x55ec4a1ade80 2024-04-21T08:47:31.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.696+0000 7fbb79bac740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbb701b0f00 con 0x55ec4a1ad4d0 2024-04-21T08:47:31.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.696+0000 7fbb79bac740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbb701afa90 con 0x55ec4a1b6a00 2024-04-21T08:47:31.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.696+0000 7fbb6ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4a1ade80 0x55ec4a1c2830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.696+0000 7fbb6ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4a1ade80 0x55ec4a1c2830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52780/0 (socket says 172.21.15.86:52780) 2024-04-21T08:47:31.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.696+0000 7fbb6ffff640 1 -- 172.21.15.86:0/1350514071 learned_addr learned my addr 172.21.15.86:0/1350514071 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:31.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.696+0000 7fbb74d52640 1 --2- 172.21.15.86:0/1350514071 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4a1ad4d0 0x55ec4a1a6e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.696+0000 7fbb75553640 1 --2- 172.21.15.86:0/1350514071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec4a1b6a00 0x55ec4a1b5e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.696+0000 7fbb6ffff640 1 -- 172.21.15.86:0/1350514071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec4a1b6a00 msgr2=0x55ec4a1b5e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.696+0000 7fbb6ffff640 1 --2- 172.21.15.86:0/1350514071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec4a1b6a00 0x55ec4a1b5e00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.696+0000 7fbb6ffff640 1 -- 172.21.15.86:0/1350514071 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4a1ad4d0 msgr2=0x55ec4a1a6e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.696+0000 7fbb6ffff640 1 --2- 172.21.15.86:0/1350514071 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4a1ad4d0 0x55ec4a1a6e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.696+0000 7fbb6ffff640 1 -- 172.21.15.86:0/1350514071 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ec4a1b42e0 con 0x55ec4a1ade80 2024-04-21T08:47:31.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.697+0000 7fbb6ffff640 1 --2- 172.21.15.86:0/1350514071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4a1ade80 0x55ec4a1c2830 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c0cdc80 tx=0x7fbb5c00c810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=961602d9a5d484dd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:31.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.697+0000 7fbb5a7fc640 1 -- 172.21.15.86:0/1350514071 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c0ba070 con 0x55ec4a1ade80 2024-04-21T08:47:31.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.697+0000 7fbb5a7fc640 1 -- 172.21.15.86:0/1350514071 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb5c0a0820 con 0x55ec4a1ade80 2024-04-21T08:47:31.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.697+0000 7fbb5a7fc640 1 -- 172.21.15.86:0/1350514071 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5c0d0450 con 0x55ec4a1ade80 2024-04-21T08:47:31.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.697+0000 7fbb79bac740 1 -- 172.21.15.86:0/1350514071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4a1ade80 msgr2=0x55ec4a1c2830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.697+0000 7fbb79bac740 1 --2- 172.21.15.86:0/1350514071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4a1ade80 0x55ec4a1c2830 secure :-1 s=READY pgs=1662 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c0cdc80 tx=0x7fbb5c00c810 comp rx=0 tx=0).stop 2024-04-21T08:47:31.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.697+0000 7fbb79bac740 1 -- 172.21.15.86:0/1350514071 shutdown_connections 2024-04-21T08:47:31.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.697+0000 7fbb79bac740 1 --2- 172.21.15.86:0/1350514071 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec4a1b6a00 0x55ec4a1b5e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.697+0000 7fbb79bac740 1 --2- 172.21.15.86:0/1350514071 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4a1ade80 0x55ec4a1c2830 unknown :-1 s=CLOSED pgs=1662 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.697+0000 7fbb79bac740 1 --2- 172.21.15.86:0/1350514071 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4a1ad4d0 0x55ec4a1a6e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.697+0000 7fbb79bac740 1 -- 172.21.15.86:0/1350514071 >> 172.21.15.86:0/1350514071 conn(0x55ec4a1a7ba0 msgr2=0x55ec4a1b2700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:31.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.697+0000 7fbb79bac740 1 -- 172.21.15.86:0/1350514071 shutdown_connections 2024-04-21T08:47:31.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.697+0000 7fbb79bac740 1 -- 172.21.15.86:0/1350514071 wait complete. 2024-04-21T08:47:31.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.698+0000 7fbb79bac740 1 -- start start 2024-04-21T08:47:31.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.698+0000 7fbb79bac740 1 Processor -- start 2024-04-21T08:47:31.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.698+0000 7fbb79bac740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4a1ad4d0 0x55ec4a1ba870 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.698+0000 7fbb79bac740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4a1ade80 0x55ec4a1badb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.698+0000 7fbb79bac740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec4a1b6a00 0x55ec4a1bb2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.698+0000 7fbb79bac740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbb701b0830 con 0x55ec4a1ad4d0 2024-04-21T08:47:31.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.698+0000 7fbb79bac740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbb701b0160 con 0x55ec4a1ade80 2024-04-21T08:47:31.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.698+0000 7fbb79bac740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbb701ad3c0 con 0x55ec4a1b6a00 2024-04-21T08:47:31.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.698+0000 7fbb79bac740 10 client.0 _ll_register_callbacks cb 0x7fbb68032380 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:31.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.698+0000 7fbb74d52640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4a1ad4d0 0x55ec4a1ba870 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.698+0000 7fbb74d52640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4a1ad4d0 0x55ec4a1ba870 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52792/0 (socket says 172.21.15.86:52792) 2024-04-21T08:47:31.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.698+0000 7fbb74d52640 1 -- 172.21.15.86:0/3130266065 learned_addr learned my addr 172.21.15.86:0/3130266065 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:31.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.698+0000 7fbb6ffff640 1 --2- 172.21.15.86:0/3130266065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4a1ade80 0x55ec4a1badb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.698+0000 7fbb75553640 1 --2- 172.21.15.86:0/3130266065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec4a1b6a00 0x55ec4a1bb2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.698+0000 7fbb74d52640 1 -- 172.21.15.86:0/3130266065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec4a1b6a00 msgr2=0x55ec4a1bb2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.698+0000 7fbb74d52640 1 --2- 172.21.15.86:0/3130266065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec4a1b6a00 0x55ec4a1bb2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.698+0000 7fbb74d52640 1 -- 172.21.15.86:0/3130266065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4a1ade80 msgr2=0x55ec4a1badb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:31.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.698+0000 7fbb74d52640 1 --2- 172.21.15.86:0/3130266065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4a1ade80 0x55ec4a1badb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.698+0000 7fbb74d52640 1 -- 172.21.15.86:0/3130266065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ec4a1ad0b0 con 0x55ec4a1ad4d0 2024-04-21T08:47:31.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.699+0000 7fbb74d52640 1 --2- 172.21.15.86:0/3130266065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4a1ad4d0 0x55ec4a1ba870 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fbb701033d0 tx=0x7fbb6400bc90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2bf862b4504f9614 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:31.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.699+0000 7fbb4ffff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:47:31.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.699+0000 7fbb4ffff640 1 -- 172.21.15.86:0/3130266065 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb64021070 con 0x55ec4a1ad4d0 2024-04-21T08:47:31.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.699+0000 7fbb4ffff640 1 -- 172.21.15.86:0/3130266065 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fbb6401d380 con 0x55ec4a1ad4d0 2024-04-21T08:47:31.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.699+0000 7fbb79bac740 1 -- 172.21.15.86:0/3130266065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb5c0a0820 con 0x55ec4a1ad4d0 2024-04-21T08:47:31.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.699+0000 7fbb4ffff640 1 -- 172.21.15.86:0/3130266065 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbb6402d050 con 0x55ec4a1ad4d0 2024-04-21T08:47:31.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.699+0000 7fbb79bac740 10 client.19200 fetch_fsmap learned FSMap version 29 2024-04-21T08:47:31.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.699+0000 7fbb79bac740 1 -- 172.21.15.86:0/3130266065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ec4a22cfb0 con 0x55ec4a1ad4d0 2024-04-21T08:47:31.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.699+0000 7fbb4ffff640 1 -- 172.21.15.86:0/3130266065 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb5c0a0820 con 0x55ec4a1ad4d0 2024-04-21T08:47:31.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.699+0000 7fbb4ffff640 1 -- 172.21.15.86:0/3130266065 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbb640047a0 con 0x55ec4a1ad4d0 2024-04-21T08:47:31.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.699+0000 7fbb79bac740 10 client.19200 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:47:31.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.699+0000 7fbb79bac740 10 client.19200 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:31.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.699+0000 7fbb79bac740 20 client.19200 populate_metadata read hostname 'smithi086' 2024-04-21T08:47:31.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.699+0000 7fbb79bac740 1 --2- 172.21.15.86:0/3130266065 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ec4a22d710 0x55ec4a22fb30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:31.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.699+0000 7fbb79bac740 4 client.19200 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:47:31.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.699+0000 7fbb79bac740 1 -- 172.21.15.86:0/3130266065 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55ec4a11fc10 con 0x55ec4a22d710 2024-04-21T08:47:31.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.699+0000 7fbb6ffff640 1 --2- 172.21.15.86:0/3130266065 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ec4a22d710 0x55ec4a22fb30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:31.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.700+0000 7fbb4ffff640 10 client.19200 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:31.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.700+0000 7fbb6ffff640 1 --2- 172.21.15.86:0/3130266065 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ec4a22d710 0x55ec4a22fb30 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1dad48d7c05df688 server_cookie=bd6a31f018435081 in_seq=0 out_seq=0 2024-04-21T08:47:31.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.700+0000 7fbb4ffff640 1 -- 172.21.15.86:0/3130266065 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55ec4a11fc10 con 0x55ec4a22d710 2024-04-21T08:47:31.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.700+0000 7fbb4ffff640 10 client.19200 handle_command_reply: tid=0 2024-04-21T08:47:31.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.700+0000 7fbb79bac740 1 client.19200 shutdown 2024-04-21T08:47:31.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.700+0000 7fbb79bac740 1 -- 172.21.15.86:0/3130266065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4a1ad4d0 msgr2=0x55ec4a1ba870 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.700+0000 7fbb79bac740 1 --2- 172.21.15.86:0/3130266065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4a1ad4d0 0x55ec4a1ba870 secure :-1 s=READY pgs=1663 cs=0 l=1 rev1=1 crypto rx=0x7fbb701033d0 tx=0x7fbb6400bc90 comp rx=0 tx=0).stop 2024-04-21T08:47:31.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.700+0000 7fbb79bac740 1 -- 172.21.15.86:0/3130266065 shutdown_connections 2024-04-21T08:47:31.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.700+0000 7fbb79bac740 1 --2- 172.21.15.86:0/3130266065 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ec4a22d710 0x55ec4a22fb30 crc :-1 s=READY pgs=260 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.700+0000 7fbb79bac740 1 --2- 172.21.15.86:0/3130266065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ec4a1b6a00 0x55ec4a1bb2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.700+0000 7fbb4ffff640 0 client.19200 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:31.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.700+0000 7fbb79bac740 1 --2- 172.21.15.86:0/3130266065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ec4a1ade80 0x55ec4a1badb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.700+0000 7fbb79bac740 1 --2- 172.21.15.86:0/3130266065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ec4a1ad4d0 0x55ec4a1ba870 unknown :-1 s=CLOSED pgs=1663 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.700+0000 7fbb79bac740 1 -- 172.21.15.86:0/3130266065 >> 172.21.15.86:0/3130266065 conn(0x55ec4a1a7ba0 msgr2=0x55ec4a1b1e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:31.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.700+0000 7fbb79bac740 1 -- 172.21.15.86:0/3130266065 shutdown_connections 2024-04-21T08:47:31.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.700+0000 7fbb79bac740 1 -- 172.21.15.86:0/3130266065 wait complete. 2024-04-21T08:47:31.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.700+0000 7fbb79bac740 20 client.19200 trim_cache size 0 max 16384 2024-04-21T08:47:31.707 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:47:31.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.701+0000 7fbb76fb6640 1 -- 172.21.15.86:0/2164230098 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbb600a3da0 msgr2=0x7fbb600a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.701+0000 7fbb76fb6640 1 --2- 172.21.15.86:0/2164230098 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbb600a3da0 0x7fbb600a6260 secure :-1 s=READY pgs=271 cs=0 l=1 rev1=1 crypto rx=0x7fbb6802c6c0 tx=0x7fbb68040000 comp rx=0 tx=0).stop 2024-04-21T08:47:31.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.701+0000 7fbb76fb6640 1 -- 172.21.15.86:0/2164230098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb7006af10 msgr2=0x7fbb70198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:31.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.701+0000 7fbb76fb6640 1 --2- 172.21.15.86:0/2164230098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb7006af10 0x7fbb70198ca0 secure :-1 s=READY pgs=1659 cs=0 l=1 rev1=1 crypto rx=0x7fbb5c00c890 tx=0x7fbb5c00cd50 comp rx=0 tx=0).stop 2024-04-21T08:47:31.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.701+0000 7fbb76fb6640 1 -- 172.21.15.86:0/2164230098 shutdown_connections 2024-04-21T08:47:31.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.701+0000 7fbb76fb6640 1 --2- 172.21.15.86:0/2164230098 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbb600a3da0 0x7fbb600a6260 unknown :-1 s=CLOSED pgs=271 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.701+0000 7fbb76fb6640 1 --2- 172.21.15.86:0/2164230098 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb70108150 0x7fbb7019d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.701+0000 7fbb76fb6640 1 --2- 172.21.15.86:0/2164230098 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb7006af10 0x7fbb70198ca0 unknown :-1 s=CLOSED pgs=1659 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.701+0000 7fbb76fb6640 1 --2- 172.21.15.86:0/2164230098 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb7006a550 0x7fbb70198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:31.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.701+0000 7fbb76fb6640 1 -- 172.21.15.86:0/2164230098 >> 172.21.15.86:0/2164230098 conn(0x7fbb70076c50 msgr2=0x7fbb700ffed0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:31.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.701+0000 7fbb76fb6640 1 -- 172.21.15.86:0/2164230098 shutdown_connections 2024-04-21T08:47:31.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:31.701+0000 7fbb76fb6640 1 -- 172.21.15.86:0/2164230098 wait complete. 2024-04-21T08:47:32.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:31 smithi088 ceph-mon[31259]: pgmap v271: 129 pgs: 129 active+clean; 3.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:47:32.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:31 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3653949454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:47:32.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:31 smithi179 ceph-mon[31829]: pgmap v271: 129 pgs: 129 active+clean; 3.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:47:32.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:31 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3653949454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:47:32.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:31 smithi086 ceph-mon[27939]: pgmap v271: 129 pgs: 129 active+clean; 3.8 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:47:32.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:31 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3653949454' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:47:32.711 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:47:32.712 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:47:32.712 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:47:32.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.827+0000 7f5870a06640 1 -- 172.21.15.86:0/434453099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f586c102b90 msgr2=0x7f586c106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:32.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.827+0000 7f5870a06640 1 --2- 172.21.15.86:0/434453099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f586c102b90 0x7f586c106bd0 secure :-1 s=READY pgs=1664 cs=0 l=1 rev1=1 crypto rx=0x7f5854009900 tx=0x7f585402bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:32.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.827+0000 7f5870a06640 1 -- 172.21.15.86:0/434453099 shutdown_connections 2024-04-21T08:47:32.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.827+0000 7f5870a06640 1 --2- 172.21.15.86:0/434453099 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f586c1072b0 0x7f586c10ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.827+0000 7f5870a06640 1 --2- 172.21.15.86:0/434453099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f586c102b90 0x7f586c106bd0 unknown :-1 s=CLOSED pgs=1664 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.827+0000 7f5870a06640 1 --2- 172.21.15.86:0/434453099 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f586c1021e0 0x7f586c1025c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.827+0000 7f5870a06640 1 -- 172.21.15.86:0/434453099 >> 172.21.15.86:0/434453099 conn(0x7f586c0fda60 msgr2=0x7f586c0ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:32.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.827+0000 7f5870a06640 1 -- 172.21.15.86:0/434453099 shutdown_connections 2024-04-21T08:47:32.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.827+0000 7f5870a06640 1 -- 172.21.15.86:0/434453099 wait complete. 2024-04-21T08:47:32.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.828+0000 7f5870a06640 1 Processor -- start 2024-04-21T08:47:32.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.828+0000 7f5870a06640 1 -- start start 2024-04-21T08:47:32.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.828+0000 7f5870a06640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f586c1021e0 0x7f586c1987b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:32.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.828+0000 7f5870a06640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f586c1072b0 0x7f586c198cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:32.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.828+0000 7f5870a06640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f586c19d020 0x7f586c19d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:32.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.828+0000 7f5870a06640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f586c1121f0 con 0x7f586c1072b0 2024-04-21T08:47:32.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.828+0000 7f5870a06640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f586c112070 con 0x7f586c1021e0 2024-04-21T08:47:32.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.828+0000 7f5870a06640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f586c112370 con 0x7f586c19d020 2024-04-21T08:47:32.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.828+0000 7f5863fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f586c1072b0 0x7f586c198cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:32.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.828+0000 7f5863fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f586c1072b0 0x7f586c198cf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52810/0 (socket says 172.21.15.86:52810) 2024-04-21T08:47:32.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.828+0000 7f5863fff640 1 -- 172.21.15.86:0/2601805116 learned_addr learned my addr 172.21.15.86:0/2601805116 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:32.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.828+0000 7f586a59c640 1 --2- 172.21.15.86:0/2601805116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f586c1021e0 0x7f586c1987b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:32.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.828+0000 7f586ad9d640 1 --2- 172.21.15.86:0/2601805116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f586c19d020 0x7f586c19d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:32.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.829+0000 7f5863fff640 1 -- 172.21.15.86:0/2601805116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f586c19d020 msgr2=0x7f586c19d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:32.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.829+0000 7f5863fff640 1 --2- 172.21.15.86:0/2601805116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f586c19d020 0x7f586c19d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.829+0000 7f5863fff640 1 -- 172.21.15.86:0/2601805116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f586c1021e0 msgr2=0x7f586c1987b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:32.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.829+0000 7f5863fff640 1 --2- 172.21.15.86:0/2601805116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f586c1021e0 0x7f586c1987b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.829+0000 7f5863fff640 1 -- 172.21.15.86:0/2601805116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f586c19db10 con 0x7f586c1072b0 2024-04-21T08:47:32.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.829+0000 7f586ad9d640 1 --2- 172.21.15.86:0/2601805116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f586c19d020 0x7f586c19d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:32.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.829+0000 7f5863fff640 1 --2- 172.21.15.86:0/2601805116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f586c1072b0 0x7f586c198cf0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f5854036040 tx=0x7f585402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:32.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.829+0000 7f58637fe640 1 -- 172.21.15.86:0/2601805116 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5854043020 con 0x7f586c1072b0 2024-04-21T08:47:32.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.829+0000 7f5870a06640 1 -- 172.21.15.86:0/2601805116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f586c19dda0 con 0x7f586c1072b0 2024-04-21T08:47:32.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.829+0000 7f58637fe640 1 -- 172.21.15.86:0/2601805116 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f585402cbe0 con 0x7f586c1072b0 2024-04-21T08:47:32.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.829+0000 7f58637fe640 1 -- 172.21.15.86:0/2601805116 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5854003bf0 con 0x7f586c1072b0 2024-04-21T08:47:32.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.829+0000 7f5870a06640 1 -- 172.21.15.86:0/2601805116 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f586c1a55c0 con 0x7f586c1072b0 2024-04-21T08:47:32.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.832+0000 7f58637fe640 1 -- 172.21.15.86:0/2601805116 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5854046020 con 0x7f586c1072b0 2024-04-21T08:47:32.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.832+0000 7f58735f9740 1 Processor -- start 2024-04-21T08:47:32.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.832+0000 7f58735f9740 1 -- start start 2024-04-21T08:47:32.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.832+0000 7f58735f9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d720e3a740 0x55d720e3ab20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:32.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.832+0000 7f58735f9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d720e3b0f0 0x55d720e43c30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:32.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.832+0000 7f58735f9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d720e44250 0x55d720e446d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:32.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.832+0000 7f58735f9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d720d9ef00 con 0x55d720e3a740 2024-04-21T08:47:32.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.832+0000 7f58735f9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d720d3c590 con 0x55d720e3b0f0 2024-04-21T08:47:32.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.832+0000 7f58735f9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d720d9aec0 con 0x55d720e44250 2024-04-21T08:47:32.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.832+0000 7f586ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d720e3a740 0x55d720e3ab20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:32.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.833+0000 7f586ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d720e3a740 0x55d720e3ab20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52822/0 (socket says 172.21.15.86:52822) 2024-04-21T08:47:32.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.833+0000 7f586ad9d640 1 -- 172.21.15.86:0/764887171 learned_addr learned my addr 172.21.15.86:0/764887171 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:32.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.833+0000 7f58637fe640 1 --2- 172.21.15.86:0/2601805116 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f58580a3d20 0x7f58580a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:32.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.833+0000 7f586a59c640 1 --2- 172.21.15.86:0/764887171 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d720e3b0f0 0x55d720e43c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:32.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.833+0000 7f5863fff640 1 --2- 172.21.15.86:0/764887171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d720e44250 0x55d720e446d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:32.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.833+0000 7f586ad9d640 1 -- 172.21.15.86:0/764887171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d720e44250 msgr2=0x55d720e446d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:32.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.833+0000 7f586ad9d640 1 --2- 172.21.15.86:0/764887171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d720e44250 0x55d720e446d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.833+0000 7f586ad9d640 1 -- 172.21.15.86:0/764887171 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d720e3b0f0 msgr2=0x55d720e43c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:32.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.833+0000 7f586ad9d640 1 --2- 172.21.15.86:0/764887171 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d720e3b0f0 0x55d720e43c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.833+0000 7f586ad9d640 1 -- 172.21.15.86:0/764887171 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d720b797e0 con 0x55d720e3a740 2024-04-21T08:47:32.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.833+0000 7f5863fff640 1 --2- 172.21.15.86:0/764887171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d720e44250 0x55d720e446d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:32.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.833+0000 7f58637fe640 1 -- 172.21.15.86:0/2601805116 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f58540f2dc0 con 0x7f586c1072b0 2024-04-21T08:47:32.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.833+0000 7f586ad9d640 1 --2- 172.21.15.86:0/2601805116 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f58580a3d20 0x7f58580a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:32.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.833+0000 7f586ad9d640 1 --2- 172.21.15.86:0/764887171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d720e3a740 0x55d720e3ab20 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f5864005c80 tx=0x7f586400e7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2b0aa37ae2ef2bc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:32.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.833+0000 7f5860ff9640 1 -- 172.21.15.86:0/764887171 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5864022070 con 0x55d720e3a740 2024-04-21T08:47:32.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.833+0000 7f586ad9d640 1 --2- 172.21.15.86:0/2601805116 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f58580a3d20 0x7f58580a61e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f586401f000 tx=0x7f5864020000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:32.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.833+0000 7f5860ff9640 1 -- 172.21.15.86:0/764887171 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5864011c30 con 0x55d720e3a740 2024-04-21T08:47:32.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.833+0000 7f5860ff9640 1 -- 172.21.15.86:0/764887171 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f586401a660 con 0x55d720e3a740 2024-04-21T08:47:32.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.834+0000 7f58735f9740 1 -- 172.21.15.86:0/764887171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d720e3a740 msgr2=0x55d720e3ab20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:32.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.834+0000 7f58735f9740 1 --2- 172.21.15.86:0/764887171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d720e3a740 0x55d720e3ab20 secure :-1 s=READY pgs=1666 cs=0 l=1 rev1=1 crypto rx=0x7f5864005c80 tx=0x7f586400e7e0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.834+0000 7f58735f9740 1 -- 172.21.15.86:0/764887171 shutdown_connections 2024-04-21T08:47:32.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.834+0000 7f58735f9740 1 --2- 172.21.15.86:0/764887171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d720e44250 0x55d720e446d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.834+0000 7f58735f9740 1 --2- 172.21.15.86:0/764887171 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d720e3b0f0 0x55d720e43c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.834+0000 7f58735f9740 1 --2- 172.21.15.86:0/764887171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d720e3a740 0x55d720e3ab20 unknown :-1 s=CLOSED pgs=1666 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.834+0000 7f58735f9740 1 -- 172.21.15.86:0/764887171 >> 172.21.15.86:0/764887171 conn(0x55d720e34fb0 msgr2=0x55d720e35390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.834+0000 7f58735f9740 1 -- 172.21.15.86:0/764887171 shutdown_connections 2024-04-21T08:47:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.834+0000 7f58735f9740 1 -- 172.21.15.86:0/764887171 wait complete. 2024-04-21T08:47:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.834+0000 7f58735f9740 1 -- start start 2024-04-21T08:47:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.835+0000 7f58735f9740 1 Processor -- start 2024-04-21T08:47:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.835+0000 7f58735f9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d720e3a740 0x55d720e47d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.835+0000 7f58735f9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d720e3b0f0 0x55d720e48250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.835+0000 7f58735f9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d720e44250 0x55d720e48790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.835+0000 7f58735f9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d720e4b1d0 con 0x55d720e3a740 2024-04-21T08:47:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.835+0000 7f58735f9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d720d58580 con 0x55d720e3b0f0 2024-04-21T08:47:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.835+0000 7f58735f9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d720e4b350 con 0x55d720e44250 2024-04-21T08:47:32.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.835+0000 7f58735f9740 10 client.0 _ll_register_callbacks cb 0x55d720d5e650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:32.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.835+0000 7f586a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d720e3a740 0x55d720e47d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:32.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.835+0000 7f586a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d720e3a740 0x55d720e47d10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52836/0 (socket says 172.21.15.86:52836) 2024-04-21T08:47:32.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.835+0000 7f586a59c640 1 -- 172.21.15.86:0/3238997756 learned_addr learned my addr 172.21.15.86:0/3238997756 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:32.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.835+0000 7f5863fff640 1 --2- 172.21.15.86:0/3238997756 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d720e3b0f0 0x55d720e48250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:32.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.835+0000 7f586ad9d640 1 --2- 172.21.15.86:0/3238997756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d720e44250 0x55d720e48790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:32.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.835+0000 7f586a59c640 1 -- 172.21.15.86:0/3238997756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d720e44250 msgr2=0x55d720e48790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:32.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.835+0000 7f586a59c640 1 --2- 172.21.15.86:0/3238997756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d720e44250 0x55d720e48790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.835+0000 7f586a59c640 1 -- 172.21.15.86:0/3238997756 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d720e3b0f0 msgr2=0x55d720e48250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:32.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.835+0000 7f586a59c640 1 --2- 172.21.15.86:0/3238997756 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d720e3b0f0 0x55d720e48250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.835+0000 7f586a59c640 1 -- 172.21.15.86:0/3238997756 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d720bd9f70 con 0x55d720e3a740 2024-04-21T08:47:32.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.835+0000 7f586a59c640 1 --2- 172.21.15.86:0/3238997756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d720e3a740 0x55d720e47d10 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f585c00bed0 tx=0x7f585c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=573fe11be8a9c630 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:32.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.835+0000 7f5851ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:47:32.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.835+0000 7f5851ffb640 1 -- 172.21.15.86:0/3238997756 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c012650 con 0x55d720e3a740 2024-04-21T08:47:32.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.835+0000 7f58735f9740 1 -- 172.21.15.86:0/3238997756 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5864011c30 con 0x55d720e3a740 2024-04-21T08:47:32.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.835+0000 7f5851ffb640 1 -- 172.21.15.86:0/3238997756 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f585c0127f0 con 0x55d720e3a740 2024-04-21T08:47:32.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.836+0000 7f5851ffb640 1 -- 172.21.15.86:0/3238997756 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f585c0129f0 con 0x55d720e3a740 2024-04-21T08:47:32.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.836+0000 7f58735f9740 10 client.19212 fetch_fsmap learned FSMap version 29 2024-04-21T08:47:32.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.836+0000 7f58735f9740 1 -- 172.21.15.86:0/3238997756 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d720a71170 con 0x55d720e3a740 2024-04-21T08:47:32.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.836+0000 7f5851ffb640 1 -- 172.21.15.86:0/3238997756 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5864011c30 con 0x55d720e3a740 2024-04-21T08:47:32.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.836+0000 7f5851ffb640 1 -- 172.21.15.86:0/3238997756 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f585c01a3d0 con 0x55d720e3a740 2024-04-21T08:47:32.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.836+0000 7f58735f9740 10 client.19212 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:47:32.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.836+0000 7f58735f9740 10 client.19212 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:32.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.836+0000 7f58735f9740 20 client.19212 populate_metadata read hostname 'smithi086' 2024-04-21T08:47:32.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.836+0000 7f58735f9740 1 --2- 172.21.15.86:0/3238997756 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d720e587f0 0x55d720e5ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:32.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.836+0000 7f58735f9740 4 client.19212 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:32.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.836+0000 7f58735f9740 1 -- 172.21.15.86:0/3238997756 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d720dac920 con 0x55d720e587f0 2024-04-21T08:47:32.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.836+0000 7f5863fff640 1 --2- 172.21.15.86:0/3238997756 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d720e587f0 0x55d720e5ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:32.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.837+0000 7f5863fff640 1 --2- 172.21.15.86:0/3238997756 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d720e587f0 0x55d720e5ac70 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b570181ccec096af server_cookie=1d7b915341161c88 in_seq=0 out_seq=0 2024-04-21T08:47:32.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.837+0000 7f5851ffb640 10 client.19212 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:32.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.838+0000 7f5851ffb640 1 -- 172.21.15.86:0/3238997756 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d720dac920 con 0x55d720e587f0 2024-04-21T08:47:32.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.838+0000 7f5851ffb640 10 client.19212 handle_command_reply: tid=0 2024-04-21T08:47:32.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.838+0000 7f58735f9740 1 client.19212 shutdown 2024-04-21T08:47:32.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.838+0000 7f58735f9740 1 -- 172.21.15.86:0/3238997756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d720e3a740 msgr2=0x55d720e47d10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:32.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.838+0000 7f58735f9740 1 --2- 172.21.15.86:0/3238997756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d720e3a740 0x55d720e47d10 secure :-1 s=READY pgs=1667 cs=0 l=1 rev1=1 crypto rx=0x7f585c00bed0 tx=0x7f585c0117b0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.838+0000 7f58735f9740 1 -- 172.21.15.86:0/3238997756 shutdown_connections 2024-04-21T08:47:32.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.838+0000 7f58735f9740 1 --2- 172.21.15.86:0/3238997756 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d720e587f0 0x55d720e5ac70 crc :-1 s=READY pgs=261 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.838+0000 7f58735f9740 1 --2- 172.21.15.86:0/3238997756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d720e44250 0x55d720e48790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.838+0000 7f5851ffb640 0 client.19212 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:32.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.838+0000 7f58735f9740 1 --2- 172.21.15.86:0/3238997756 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d720e3b0f0 0x55d720e48250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.838+0000 7f58735f9740 1 --2- 172.21.15.86:0/3238997756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d720e3a740 0x55d720e47d10 unknown :-1 s=CLOSED pgs=1667 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.838+0000 7f58735f9740 1 -- 172.21.15.86:0/3238997756 >> 172.21.15.86:0/3238997756 conn(0x55d720e34fb0 msgr2=0x55d720e36d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:32.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.838+0000 7f58735f9740 1 -- 172.21.15.86:0/3238997756 shutdown_connections 2024-04-21T08:47:32.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.838+0000 7f58735f9740 1 -- 172.21.15.86:0/3238997756 wait complete. 2024-04-21T08:47:32.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.838+0000 7f58735f9740 20 client.19212 trim_cache size 0 max 16384 2024-04-21T08:47:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.855+0000 7f58735f9740 1 Processor -- start 2024-04-21T08:47:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.855+0000 7f58735f9740 1 -- start start 2024-04-21T08:47:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.855+0000 7f58735f9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d720e34fb0 0x55d720e4a440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.855+0000 7f58735f9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d720e3a740 0x55d720e54100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.855+0000 7f58735f9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d720e3b0f0 0x55d720e46e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.855+0000 7f58735f9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f586c1ad060 con 0x55d720e34fb0 2024-04-21T08:47:32.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.855+0000 7f58735f9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f586c1aeba0 con 0x55d720e3a740 2024-04-21T08:47:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.855+0000 7f58735f9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f586c1ad730 con 0x55d720e3b0f0 2024-04-21T08:47:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.855+0000 7f586a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d720e34fb0 0x55d720e4a440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.855+0000 7f586a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d720e34fb0 0x55d720e4a440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52846/0 (socket says 172.21.15.86:52846) 2024-04-21T08:47:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.855+0000 7f586a59c640 1 -- 172.21.15.86:0/3221566873 learned_addr learned my addr 172.21.15.86:0/3221566873 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.855+0000 7f5863fff640 1 --2- 172.21.15.86:0/3221566873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d720e3a740 0x55d720e54100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.855+0000 7f586ad9d640 1 --2- 172.21.15.86:0/3221566873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d720e3b0f0 0x55d720e46e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.855+0000 7f586a59c640 1 -- 172.21.15.86:0/3221566873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d720e3b0f0 msgr2=0x55d720e46e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.855+0000 7f586a59c640 1 --2- 172.21.15.86:0/3221566873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d720e3b0f0 0x55d720e46e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.855+0000 7f586a59c640 1 -- 172.21.15.86:0/3221566873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d720e3a740 msgr2=0x55d720e54100 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.855+0000 7f586a59c640 1 --2- 172.21.15.86:0/3221566873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d720e3a740 0x55d720e54100 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.855+0000 7f586a59c640 1 -- 172.21.15.86:0/3221566873 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d720e34ae0 con 0x55d720e34fb0 2024-04-21T08:47:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.855+0000 7f5863fff640 1 --2- 172.21.15.86:0/3221566873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d720e3a740 0x55d720e54100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.856+0000 7f586a59c640 1 --2- 172.21.15.86:0/3221566873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d720e34fb0 0x55d720e4a440 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f585c0058c0 tx=0x7f585c00ba30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff5c7415202466ca server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:32.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.856+0000 7f5853fff640 1 -- 172.21.15.86:0/3221566873 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c02d020 con 0x55d720e34fb0 2024-04-21T08:47:32.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.856+0000 7f5853fff640 1 -- 172.21.15.86:0/3221566873 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f585c01ac80 con 0x55d720e34fb0 2024-04-21T08:47:32.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.856+0000 7f5853fff640 1 -- 172.21.15.86:0/3221566873 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f585c0052a0 con 0x55d720e34fb0 2024-04-21T08:47:32.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.856+0000 7f58735f9740 1 -- 172.21.15.86:0/3221566873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d720e34fb0 msgr2=0x55d720e4a440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:32.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.856+0000 7f58735f9740 1 --2- 172.21.15.86:0/3221566873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d720e34fb0 0x55d720e4a440 secure :-1 s=READY pgs=1668 cs=0 l=1 rev1=1 crypto rx=0x7f585c0058c0 tx=0x7f585c00ba30 comp rx=0 tx=0).stop 2024-04-21T08:47:32.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.856+0000 7f58735f9740 1 -- 172.21.15.86:0/3221566873 shutdown_connections 2024-04-21T08:47:32.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.856+0000 7f58735f9740 1 --2- 172.21.15.86:0/3221566873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d720e3b0f0 0x55d720e46e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.856+0000 7f58735f9740 1 --2- 172.21.15.86:0/3221566873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d720e3a740 0x55d720e54100 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.856+0000 7f58735f9740 1 --2- 172.21.15.86:0/3221566873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d720e34fb0 0x55d720e4a440 unknown :-1 s=CLOSED pgs=1668 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.856+0000 7f58735f9740 1 -- 172.21.15.86:0/3221566873 >> 172.21.15.86:0/3221566873 conn(0x55d720e43d90 msgr2=0x55d720e363c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:32.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.856+0000 7f58735f9740 1 -- 172.21.15.86:0/3221566873 shutdown_connections 2024-04-21T08:47:32.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.856+0000 7f58735f9740 1 -- 172.21.15.86:0/3221566873 wait complete. 2024-04-21T08:47:32.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.857+0000 7f58735f9740 1 -- start start 2024-04-21T08:47:32.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.857+0000 7f58735f9740 1 Processor -- start 2024-04-21T08:47:32.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.857+0000 7f58735f9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d720e34fb0 0x55d720e60130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:32.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.857+0000 7f58735f9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d720e3a740 0x55d720e60670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:32.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.857+0000 7f58735f9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d720e3b0f0 0x55d720ebde00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:32.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.857+0000 7f58735f9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f586c1ae4d0 con 0x55d720e3b0f0 2024-04-21T08:47:32.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.857+0000 7f58735f9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f586c1ade00 con 0x55d720e34fb0 2024-04-21T08:47:32.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.857+0000 7f58735f9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f586c1ab060 con 0x55d720e3a740 2024-04-21T08:47:32.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.857+0000 7f58735f9740 10 client.0 _ll_register_callbacks cb 0x7f585c00b3c0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:32.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.857+0000 7f586ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d720e3b0f0 0x55d720ebde00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:32.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.857+0000 7f586ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d720e3b0f0 0x55d720ebde00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52850/0 (socket says 172.21.15.86:52850) 2024-04-21T08:47:32.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.857+0000 7f586ad9d640 1 -- 172.21.15.86:0/1577009234 learned_addr learned my addr 172.21.15.86:0/1577009234 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:32.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.857+0000 7f5863fff640 1 --2- 172.21.15.86:0/1577009234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d720e3a740 0x55d720e60670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:32.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.857+0000 7f586a59c640 1 --2- 172.21.15.86:0/1577009234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d720e34fb0 0x55d720e60130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:32.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.857+0000 7f586ad9d640 1 -- 172.21.15.86:0/1577009234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d720e3a740 msgr2=0x55d720e60670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:32.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.857+0000 7f586ad9d640 1 --2- 172.21.15.86:0/1577009234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d720e3a740 0x55d720e60670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.857+0000 7f586ad9d640 1 -- 172.21.15.86:0/1577009234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d720e34fb0 msgr2=0x55d720e60130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:32.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.857+0000 7f586ad9d640 1 --2- 172.21.15.86:0/1577009234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d720e34fb0 0x55d720e60130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.857+0000 7f586ad9d640 1 -- 172.21.15.86:0/1577009234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d720ebe340 con 0x55d720e3b0f0 2024-04-21T08:47:32.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.857+0000 7f586a59c640 1 --2- 172.21.15.86:0/1577009234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d720e34fb0 0x55d720e60130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:32.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.857+0000 7f586ad9d640 1 --2- 172.21.15.86:0/1577009234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d720e3b0f0 0x55d720ebde00 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f5864005cc0 tx=0x7f5864011f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8e9a2ba21d154c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:32.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.857+0000 7f5851ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:47:32.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.858+0000 7f5851ffb640 1 -- 172.21.15.86:0/1577009234 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5864036060 con 0x55d720e3b0f0 2024-04-21T08:47:32.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.858+0000 7f5851ffb640 1 -- 172.21.15.86:0/1577009234 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f586401c670 con 0x55d720e3b0f0 2024-04-21T08:47:32.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.858+0000 7f58735f9740 1 -- 172.21.15.86:0/1577009234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f585c01ac80 con 0x55d720e3b0f0 2024-04-21T08:47:32.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.858+0000 7f5851ffb640 1 -- 172.21.15.86:0/1577009234 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f586403d050 con 0x55d720e3b0f0 2024-04-21T08:47:32.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.858+0000 7f58735f9740 10 client.19218 fetch_fsmap learned FSMap version 29 2024-04-21T08:47:32.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.858+0000 7f58735f9740 1 -- 172.21.15.86:0/1577009234 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d720ebf5e0 con 0x55d720e3b0f0 2024-04-21T08:47:32.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.858+0000 7f5851ffb640 1 -- 172.21.15.86:0/1577009234 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f585c01ac80 con 0x55d720e3b0f0 2024-04-21T08:47:32.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.858+0000 7f5851ffb640 1 -- 172.21.15.86:0/1577009234 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5864019040 con 0x55d720e3b0f0 2024-04-21T08:47:32.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.858+0000 7f58735f9740 10 client.19218 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:47:32.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.858+0000 7f58735f9740 10 client.19218 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:32.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.858+0000 7f58735f9740 20 client.19218 populate_metadata read hostname 'smithi086' 2024-04-21T08:47:32.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.858+0000 7f58735f9740 1 --2- 172.21.15.86:0/1577009234 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d720e44250 0x55d720ec1cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:32.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.858+0000 7f58735f9740 4 client.19218 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:47:32.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.858+0000 7f58735f9740 1 -- 172.21.15.86:0/1577009234 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x55d720da6790 con 0x55d720e44250 2024-04-21T08:47:32.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.858+0000 7f586a59c640 1 --2- 172.21.15.86:0/1577009234 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d720e44250 0x55d720ec1cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:32.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.859+0000 7f5851ffb640 10 client.19218 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:32.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.859+0000 7f586a59c640 1 --2- 172.21.15.86:0/1577009234 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d720e44250 0x55d720ec1cc0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b1298fddfaa4f316 server_cookie=3cb59bab7b2883a5 in_seq=0 out_seq=0 2024-04-21T08:47:32.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.860+0000 7f5851ffb640 1 -- 172.21.15.86:0/1577009234 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x55d720da6790 con 0x55d720e44250 2024-04-21T08:47:32.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.860+0000 7f5851ffb640 10 client.19218 handle_command_reply: tid=0 2024-04-21T08:47:32.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.860+0000 7f58735f9740 1 client.19218 shutdown 2024-04-21T08:47:32.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.860+0000 7f58735f9740 1 -- 172.21.15.86:0/1577009234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d720e3b0f0 msgr2=0x55d720ebde00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:32.869 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.860+0000 7f58735f9740 1 --2- 172.21.15.86:0/1577009234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d720e3b0f0 0x55d720ebde00 secure :-1 s=READY pgs=1669 cs=0 l=1 rev1=1 crypto rx=0x7f5864005cc0 tx=0x7f5864011f90 comp rx=0 tx=0).stop 2024-04-21T08:47:32.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.860+0000 7f58735f9740 1 -- 172.21.15.86:0/1577009234 shutdown_connections 2024-04-21T08:47:32.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.860+0000 7f58735f9740 1 --2- 172.21.15.86:0/1577009234 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d720e44250 0x55d720ec1cc0 crc :-1 s=READY pgs=262 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.860+0000 7f58735f9740 1 --2- 172.21.15.86:0/1577009234 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d720e3b0f0 0x55d720ebde00 unknown :-1 s=CLOSED pgs=1669 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.860+0000 7f5851ffb640 0 client.19218 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:32.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.860+0000 7f58735f9740 1 --2- 172.21.15.86:0/1577009234 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d720e3a740 0x55d720e60670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.860+0000 7f58735f9740 1 --2- 172.21.15.86:0/1577009234 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d720e34fb0 0x55d720e60130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.860+0000 7f58735f9740 1 -- 172.21.15.86:0/1577009234 >> 172.21.15.86:0/1577009234 conn(0x55d720e43d90 msgr2=0x55d720e36c50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:32.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.860+0000 7f58735f9740 1 -- 172.21.15.86:0/1577009234 shutdown_connections 2024-04-21T08:47:32.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.860+0000 7f58735f9740 1 -- 172.21.15.86:0/1577009234 wait complete. 2024-04-21T08:47:32.870 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.860+0000 7f58735f9740 20 client.19218 trim_cache size 0 max 16384 2024-04-21T08:47:32.870 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:47:32.871 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-21T08:47:32.871 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "cdfc81f1-8c4e-402a-a0ed-8db2d97548b5", 2024-04-21T08:47:32.871 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-21T08:47:32.871 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:47:32.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.861+0000 7f5870a06640 1 -- 172.21.15.86:0/2601805116 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f58580a3d20 msgr2=0x7f58580a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:32.872 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.861+0000 7f5870a06640 1 --2- 172.21.15.86:0/2601805116 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f58580a3d20 0x7f58580a61e0 secure :-1 s=READY pgs=272 cs=0 l=1 rev1=1 crypto rx=0x7f586401f000 tx=0x7f5864020000 comp rx=0 tx=0).stop 2024-04-21T08:47:32.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.861+0000 7f5870a06640 1 -- 172.21.15.86:0/2601805116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f586c1072b0 msgr2=0x7f586c198cf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:32.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.861+0000 7f5870a06640 1 --2- 172.21.15.86:0/2601805116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f586c1072b0 0x7f586c198cf0 secure :-1 s=READY pgs=1665 cs=0 l=1 rev1=1 crypto rx=0x7f5854036040 tx=0x7f585402c7e0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.861+0000 7f5870a06640 1 -- 172.21.15.86:0/2601805116 shutdown_connections 2024-04-21T08:47:32.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.861+0000 7f5870a06640 1 --2- 172.21.15.86:0/2601805116 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f58580a3d20 0x7f58580a61e0 unknown :-1 s=CLOSED pgs=272 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.861+0000 7f5870a06640 1 --2- 172.21.15.86:0/2601805116 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f586c19d020 0x7f586c19d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.861+0000 7f5870a06640 1 --2- 172.21.15.86:0/2601805116 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f586c1072b0 0x7f586c198cf0 unknown :-1 s=CLOSED pgs=1665 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.861+0000 7f5870a06640 1 --2- 172.21.15.86:0/2601805116 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f586c1021e0 0x7f586c1987b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.861+0000 7f5870a06640 1 -- 172.21.15.86:0/2601805116 >> 172.21.15.86:0/2601805116 conn(0x7f586c0fda60 msgr2=0x7f586c0fee00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:32.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.861+0000 7f5870a06640 1 -- 172.21.15.86:0/2601805116 shutdown_connections 2024-04-21T08:47:32.873 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.861+0000 7f5870a06640 1 -- 172.21.15.86:0/2601805116 wait complete. 2024-04-21T08:47:32.873 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:47:32.874 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:47:32.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.976+0000 7f379a367640 1 -- 172.21.15.86:0/642747179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3794107270 msgr2=0x7f379410ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:32.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.976+0000 7f379a367640 1 --2- 172.21.15.86:0/642747179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3794107270 0x7f379410ee40 secure :-1 s=READY pgs=1670 cs=0 l=1 rev1=1 crypto rx=0x7f378c009ca0 tx=0x7f378c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.976+0000 7f379a367640 1 -- 172.21.15.86:0/642747179 shutdown_connections 2024-04-21T08:47:32.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.976+0000 7f379a367640 1 --2- 172.21.15.86:0/642747179 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3794107270 0x7f379410ee40 unknown :-1 s=CLOSED pgs=1670 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.976+0000 7f379a367640 1 --2- 172.21.15.86:0/642747179 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3794102b50 0x7f3794106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.976+0000 7f379a367640 1 --2- 172.21.15.86:0/642747179 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f37941021a0 0x7f3794102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.976+0000 7f379a367640 1 -- 172.21.15.86:0/642747179 >> 172.21.15.86:0/642747179 conn(0x7f37940793e0 msgr2=0x7f37940797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:32.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.976+0000 7f379a367640 1 -- 172.21.15.86:0/642747179 shutdown_connections 2024-04-21T08:47:32.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.976+0000 7f379a367640 1 -- 172.21.15.86:0/642747179 wait complete. 2024-04-21T08:47:32.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.977+0000 7f379a367640 1 Processor -- start 2024-04-21T08:47:32.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.977+0000 7f379a367640 1 -- start start 2024-04-21T08:47:32.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.977+0000 7f379a367640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37941021a0 0x7f3794074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:32.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.977+0000 7f379a367640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3794107270 0x7f3794074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:32.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.977+0000 7f379a367640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3794074cd0 0x7f37941a68b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:32.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.977+0000 7f379a367640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37941121e0 con 0x7f3794074cd0 2024-04-21T08:47:32.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.977+0000 7f379a367640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3794112060 con 0x7f3794107270 2024-04-21T08:47:32.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.977+0000 7f379a367640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3794112360 con 0x7f37941021a0 2024-04-21T08:47:32.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.977+0000 7f3798904640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3794074cd0 0x7f37941a68b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:32.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.977+0000 7f3798904640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3794074cd0 0x7f37941a68b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52868/0 (socket says 172.21.15.86:52868) 2024-04-21T08:47:32.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.977+0000 7f3798904640 1 -- 172.21.15.86:0/3746525239 learned_addr learned my addr 172.21.15.86:0/3746525239 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:32.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.978+0000 7f37937fe640 1 --2- 172.21.15.86:0/3746525239 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3794107270 0x7f3794074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:32.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.978+0000 7f3793fff640 1 --2- 172.21.15.86:0/3746525239 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37941021a0 0x7f3794074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:32.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.978+0000 7f3798904640 1 -- 172.21.15.86:0/3746525239 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37941021a0 msgr2=0x7f3794074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:32.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.978+0000 7f3798904640 1 --2- 172.21.15.86:0/3746525239 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37941021a0 0x7f3794074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.978+0000 7f3798904640 1 -- 172.21.15.86:0/3746525239 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3794107270 msgr2=0x7f3794074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:32.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.978+0000 7f3798904640 1 --2- 172.21.15.86:0/3746525239 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3794107270 0x7f3794074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.978+0000 7f3798904640 1 -- 172.21.15.86:0/3746525239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f37941a6fb0 con 0x7f3794074cd0 2024-04-21T08:47:32.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.978+0000 7f3793fff640 1 --2- 172.21.15.86:0/3746525239 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37941021a0 0x7f3794074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:47:32.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.978+0000 7f37937fe640 1 --2- 172.21.15.86:0/3746525239 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3794107270 0x7f3794074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:47:32.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.978+0000 7f3798904640 1 --2- 172.21.15.86:0/3746525239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3794074cd0 0x7f37941a68b0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f378c005e30 tx=0x7f378c034760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:32.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.978+0000 7f37917fa640 1 -- 172.21.15.86:0/3746525239 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c039040 con 0x7f3794074cd0 2024-04-21T08:47:32.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.978+0000 7f379a367640 1 -- 172.21.15.86:0/3746525239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f37941a7240 con 0x7f3794074cd0 2024-04-21T08:47:32.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.978+0000 7f37917fa640 1 -- 172.21.15.86:0/3746525239 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f378c044070 con 0x7f3794074cd0 2024-04-21T08:47:32.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.978+0000 7f37917fa640 1 -- 172.21.15.86:0/3746525239 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378c003e60 con 0x7f3794074cd0 2024-04-21T08:47:32.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.978+0000 7f379a367640 1 -- 172.21.15.86:0/3746525239 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f37941a7730 con 0x7f3794074cd0 2024-04-21T08:47:32.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.981+0000 7f379d038740 1 Processor -- start 2024-04-21T08:47:32.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.981+0000 7f37917fa640 1 -- 172.21.15.86:0/3746525239 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f378c0324c0 con 0x7f3794074cd0 2024-04-21T08:47:32.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.981+0000 7f379d038740 1 -- start start 2024-04-21T08:47:32.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.981+0000 7f379d038740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc715580 0x55cdbc715960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:32.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.981+0000 7f379d038740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cdbc715ea0 0x55cdbc71e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:32.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.981+0000 7f379d038740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdbc71ea80 0x55cdbc720e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:32.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.981+0000 7f379d038740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cdbc67a1e0 con 0x55cdbc715580 2024-04-21T08:47:32.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.981+0000 7f379d038740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55cdbc617390 con 0x55cdbc71ea80 2024-04-21T08:47:32.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.981+0000 7f379d038740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55cdbc6761a0 con 0x55cdbc715ea0 2024-04-21T08:47:32.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.981+0000 7f37937fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc715580 0x55cdbc715960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:32.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.981+0000 7f37937fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc715580 0x55cdbc715960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52876/0 (socket says 172.21.15.86:52876) 2024-04-21T08:47:32.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.981+0000 7f37937fe640 1 -- 172.21.15.86:0/2078892240 learned_addr learned my addr 172.21.15.86:0/2078892240 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:32.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.982+0000 7f37917fa640 1 --2- 172.21.15.86:0/3746525239 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f377c0a3d50 0x7f377c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:32.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.982+0000 7f3793fff640 1 --2- 172.21.15.86:0/2078892240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cdbc715ea0 0x55cdbc71e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:32.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.982+0000 7f37937fe640 1 --2- 172.21.15.86:0/2078892240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdbc71ea80 0x55cdbc720e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:32.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.982+0000 7f37937fe640 1 -- 172.21.15.86:0/2078892240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cdbc715ea0 msgr2=0x55cdbc71e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:32.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.982+0000 7f37937fe640 1 --2- 172.21.15.86:0/2078892240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cdbc715ea0 0x55cdbc71e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.982+0000 7f37937fe640 1 -- 172.21.15.86:0/2078892240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdbc71ea80 msgr2=0x55cdbc720e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:32.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.982+0000 7f37937fe640 1 --2- 172.21.15.86:0/2078892240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdbc71ea80 0x55cdbc720e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.982+0000 7f37937fe640 1 -- 172.21.15.86:0/2078892240 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdbc4567d0 con 0x55cdbc715580 2024-04-21T08:47:32.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.982+0000 7f3798904640 1 --2- 172.21.15.86:0/3746525239 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f377c0a3d50 0x7f377c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:32.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.982+0000 7f37917fa640 1 -- 172.21.15.86:0/3746525239 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f378c0f2e50 con 0x7f3794074cd0 2024-04-21T08:47:32.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.982+0000 7f37937fe640 1 --2- 172.21.15.86:0/2078892240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc715580 0x55cdbc715960 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f37880135a0 tx=0x7f378800b690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5abfc9d8dfbbc769 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:32.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.982+0000 7f377a7fc640 1 -- 172.21.15.86:0/2078892240 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788019030 con 0x55cdbc715580 2024-04-21T08:47:32.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.982+0000 7f377a7fc640 1 -- 172.21.15.86:0/2078892240 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37880054d0 con 0x55cdbc715580 2024-04-21T08:47:32.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.982+0000 7f377a7fc640 1 -- 172.21.15.86:0/2078892240 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788004e60 con 0x55cdbc715580 2024-04-21T08:47:32.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.983+0000 7f379d038740 1 -- 172.21.15.86:0/2078892240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc715580 msgr2=0x55cdbc715960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:32.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.983+0000 7f3798904640 1 --2- 172.21.15.86:0/3746525239 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f377c0a3d50 0x7f377c0a6210 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f378c0f07d0 tx=0x7f378c0b3fc0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:32.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.983+0000 7f379d038740 1 --2- 172.21.15.86:0/2078892240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc715580 0x55cdbc715960 secure :-1 s=READY pgs=1672 cs=0 l=1 rev1=1 crypto rx=0x7f37880135a0 tx=0x7f378800b690 comp rx=0 tx=0).stop 2024-04-21T08:47:32.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.983+0000 7f379d038740 1 -- 172.21.15.86:0/2078892240 shutdown_connections 2024-04-21T08:47:32.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.983+0000 7f379d038740 1 --2- 172.21.15.86:0/2078892240 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdbc71ea80 0x55cdbc720e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.983+0000 7f379d038740 1 --2- 172.21.15.86:0/2078892240 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cdbc715ea0 0x55cdbc71e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.983+0000 7f379d038740 1 --2- 172.21.15.86:0/2078892240 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc715580 0x55cdbc715960 unknown :-1 s=CLOSED pgs=1672 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.983+0000 7f379d038740 1 -- 172.21.15.86:0/2078892240 >> 172.21.15.86:0/2078892240 conn(0x55cdbc70fcb0 msgr2=0x55cdbc710090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:32.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.983+0000 7f379d038740 1 -- 172.21.15.86:0/2078892240 shutdown_connections 2024-04-21T08:47:32.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.983+0000 7f379d038740 1 -- 172.21.15.86:0/2078892240 wait complete. 2024-04-21T08:47:32.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.983+0000 7f379d038740 1 -- start start 2024-04-21T08:47:32.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.983+0000 7f379d038740 1 Processor -- start 2024-04-21T08:47:32.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.984+0000 7f379d038740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cdbc715580 0x55cdbc71bb10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:32.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.984+0000 7f379d038740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdbc715ea0 0x55cdbc71c050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:32.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.984+0000 7f379d038740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc71ea80 0x55cdbc71c590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:32.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.984+0000 7f379d038740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cdbc721870 con 0x55cdbc71ea80 2024-04-21T08:47:32.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.984+0000 7f379d038740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55cdbc6332f0 con 0x55cdbc715ea0 2024-04-21T08:47:32.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.984+0000 7f379d038740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55cdbc7219f0 con 0x55cdbc715580 2024-04-21T08:47:32.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.984+0000 7f379d038740 10 client.0 _ll_register_callbacks cb 0x55cdbc640580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:32.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.984+0000 7f37937fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc71ea80 0x55cdbc71c590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:32.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.984+0000 7f37937fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc71ea80 0x55cdbc71c590 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52886/0 (socket says 172.21.15.86:52886) 2024-04-21T08:47:32.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.984+0000 7f37937fe640 1 -- 172.21.15.86:0/2413030633 learned_addr learned my addr 172.21.15.86:0/2413030633 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:32.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.984+0000 7f3793fff640 1 --2- 172.21.15.86:0/2413030633 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdbc715ea0 0x55cdbc71c050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:32.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.984+0000 7f37937fe640 1 -- 172.21.15.86:0/2413030633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cdbc715580 msgr2=0x55cdbc71bb10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:32.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.984+0000 7f37937fe640 1 --2- 172.21.15.86:0/2413030633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cdbc715580 0x55cdbc71bb10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.984+0000 7f37937fe640 1 -- 172.21.15.86:0/2413030633 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdbc715ea0 msgr2=0x55cdbc71c050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:32.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.984+0000 7f37937fe640 1 --2- 172.21.15.86:0/2413030633 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdbc715ea0 0x55cdbc71c050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.984+0000 7f37937fe640 1 -- 172.21.15.86:0/2413030633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdbc38c3f0 con 0x55cdbc71ea80 2024-04-21T08:47:32.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.984+0000 7f37937fe640 1 --2- 172.21.15.86:0/2413030633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc71ea80 0x55cdbc71c590 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f3788004c00 tx=0x7f378800bce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fabb01522bbf6cac server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:32.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.984+0000 7f37737fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:47:32.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.984+0000 7f37737fe640 1 -- 172.21.15.86:0/2413030633 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788019070 con 0x55cdbc71ea80 2024-04-21T08:47:32.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.984+0000 7f37737fe640 1 -- 172.21.15.86:0/2413030633 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f37880048c0 con 0x55cdbc71ea80 2024-04-21T08:47:32.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.984+0000 7f379d038740 1 -- 172.21.15.86:0/2413030633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f37880054d0 con 0x55cdbc71ea80 2024-04-21T08:47:32.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.984+0000 7f37737fe640 1 -- 172.21.15.86:0/2413030633 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3788032050 con 0x55cdbc71ea80 2024-04-21T08:47:32.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.984+0000 7f379d038740 10 client.19230 fetch_fsmap learned FSMap version 29 2024-04-21T08:47:32.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.984+0000 7f379d038740 1 -- 172.21.15.86:0/2413030633 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cdbc34c0f0 con 0x55cdbc71ea80 2024-04-21T08:47:32.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.985+0000 7f37737fe640 1 -- 172.21.15.86:0/2413030633 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f37880054d0 con 0x55cdbc71ea80 2024-04-21T08:47:32.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.985+0000 7f37737fe640 1 -- 172.21.15.86:0/2413030633 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f378802c4e0 con 0x55cdbc71ea80 2024-04-21T08:47:32.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.985+0000 7f379d038740 10 client.19230 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:47:32.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.985+0000 7f379d038740 10 client.19230 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:32.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.985+0000 7f379d038740 20 client.19230 populate_metadata read hostname 'smithi086' 2024-04-21T08:47:32.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.985+0000 7f379d038740 1 --2- 172.21.15.86:0/2413030633 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55cdbc72e960 0x55cdbc730d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:32.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.985+0000 7f379d038740 4 client.19230 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:32.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.985+0000 7f379d038740 1 -- 172.21.15.86:0/2413030633 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cdbc687c00 con 0x55cdbc72e960 2024-04-21T08:47:32.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.985+0000 7f3793fff640 1 --2- 172.21.15.86:0/2413030633 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55cdbc72e960 0x55cdbc730d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:32.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.985+0000 7f37737fe640 10 client.19230 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:32.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.985+0000 7f3793fff640 1 --2- 172.21.15.86:0/2413030633 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55cdbc72e960 0x55cdbc730d50 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e76ffe56b5898809 server_cookie=8dceaeabc1229d13 in_seq=0 out_seq=0 2024-04-21T08:47:32.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.986+0000 7f37737fe640 1 -- 172.21.15.86:0/2413030633 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cdbc687c00 con 0x55cdbc72e960 2024-04-21T08:47:32.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.986+0000 7f37737fe640 10 client.19230 handle_command_reply: tid=0 2024-04-21T08:47:32.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.986+0000 7f379d038740 1 client.19230 shutdown 2024-04-21T08:47:32.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.986+0000 7f379d038740 1 -- 172.21.15.86:0/2413030633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc71ea80 msgr2=0x55cdbc71c590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:32.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.986+0000 7f379d038740 1 --2- 172.21.15.86:0/2413030633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc71ea80 0x55cdbc71c590 secure :-1 s=READY pgs=1673 cs=0 l=1 rev1=1 crypto rx=0x7f3788004c00 tx=0x7f378800bce0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.986+0000 7f379d038740 1 -- 172.21.15.86:0/2413030633 shutdown_connections 2024-04-21T08:47:32.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.986+0000 7f379d038740 1 --2- 172.21.15.86:0/2413030633 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55cdbc72e960 0x55cdbc730d50 crc :-1 s=READY pgs=263 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.986+0000 7f379d038740 1 --2- 172.21.15.86:0/2413030633 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc71ea80 0x55cdbc71c590 unknown :-1 s=CLOSED pgs=1673 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.986+0000 7f379d038740 1 --2- 172.21.15.86:0/2413030633 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdbc715ea0 0x55cdbc71c050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.986+0000 7f37737fe640 0 client.19230 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:32.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.986+0000 7f379d038740 1 --2- 172.21.15.86:0/2413030633 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cdbc715580 0x55cdbc71bb10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:32.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.986+0000 7f379d038740 1 -- 172.21.15.86:0/2413030633 >> 172.21.15.86:0/2413030633 conn(0x55cdbc70fcb0 msgr2=0x55cdbc711a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:32.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.986+0000 7f379d038740 1 -- 172.21.15.86:0/2413030633 shutdown_connections 2024-04-21T08:47:32.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.986+0000 7f379d038740 1 -- 172.21.15.86:0/2413030633 wait complete. 2024-04-21T08:47:32.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:32.987+0000 7f379d038740 20 client.19230 trim_cache size 0 max 16384 2024-04-21T08:47:33.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.004+0000 7f379d038740 1 Processor -- start 2024-04-21T08:47:33.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.004+0000 7f379d038740 1 -- start start 2024-04-21T08:47:33.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.004+0000 7f379d038740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cdbc715580 0x55cdbc71e240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:33.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.004+0000 7f379d038740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdbc715ea0 0x55cdbc72a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:33.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.004+0000 7f379d038740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc71ea80 0x55cdbc71ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:33.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.004+0000 7f379d038740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37941b1560 con 0x55cdbc71ea80 2024-04-21T08:47:33.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.004+0000 7f379d038740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f37941b30a0 con 0x55cdbc715ea0 2024-04-21T08:47:33.010 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.004+0000 7f379d038740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f37941b1c30 con 0x55cdbc715580 2024-04-21T08:47:33.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.004+0000 7f37937fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc71ea80 0x55cdbc71ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:33.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.004+0000 7f37937fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc71ea80 0x55cdbc71ac70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52890/0 (socket says 172.21.15.86:52890) 2024-04-21T08:47:33.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.004+0000 7f37937fe640 1 -- 172.21.15.86:0/3013433531 learned_addr learned my addr 172.21.15.86:0/3013433531 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:33.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.004+0000 7f37937fe640 1 -- 172.21.15.86:0/3013433531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cdbc715580 msgr2=0x55cdbc71e240 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:47:33.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.004+0000 7f37937fe640 1 --2- 172.21.15.86:0/3013433531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cdbc715580 0x55cdbc71e240 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:33.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.004+0000 7f37937fe640 1 -- 172.21.15.86:0/3013433531 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdbc715ea0 msgr2=0x55cdbc72a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:33.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.004+0000 7f3793fff640 1 --2- 172.21.15.86:0/3013433531 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdbc715ea0 0x55cdbc72a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:33.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.004+0000 7f37937fe640 1 --2- 172.21.15.86:0/3013433531 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdbc715ea0 0x55cdbc72a730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:33.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.004+0000 7f37937fe640 1 -- 172.21.15.86:0/3013433531 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cdbc715cd0 con 0x55cdbc71ea80 2024-04-21T08:47:33.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.004+0000 7f37937fe640 1 --2- 172.21.15.86:0/3013433531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc71ea80 0x55cdbc71ac70 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f37880059f0 tx=0x7f378800b550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0345257d545e96f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:33.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.005+0000 7f3779ffb640 1 -- 172.21.15.86:0/3013433531 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788019070 con 0x55cdbc71ea80 2024-04-21T08:47:33.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.005+0000 7f3779ffb640 1 -- 172.21.15.86:0/3013433531 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3788025030 con 0x55cdbc71ea80 2024-04-21T08:47:33.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.005+0000 7f3779ffb640 1 -- 172.21.15.86:0/3013433531 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f378802c0b0 con 0x55cdbc71ea80 2024-04-21T08:47:33.011 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.005+0000 7f379d038740 1 -- 172.21.15.86:0/3013433531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc71ea80 msgr2=0x55cdbc71ac70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:33.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.005+0000 7f379d038740 1 --2- 172.21.15.86:0/3013433531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc71ea80 0x55cdbc71ac70 secure :-1 s=READY pgs=1674 cs=0 l=1 rev1=1 crypto rx=0x7f37880059f0 tx=0x7f378800b550 comp rx=0 tx=0).stop 2024-04-21T08:47:33.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.005+0000 7f379d038740 1 -- 172.21.15.86:0/3013433531 shutdown_connections 2024-04-21T08:47:33.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.005+0000 7f379d038740 1 --2- 172.21.15.86:0/3013433531 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc71ea80 0x55cdbc71ac70 unknown :-1 s=CLOSED pgs=1674 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:33.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.005+0000 7f379d038740 1 --2- 172.21.15.86:0/3013433531 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdbc715ea0 0x55cdbc72a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:33.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.005+0000 7f379d038740 1 --2- 172.21.15.86:0/3013433531 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cdbc715580 0x55cdbc71e240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:33.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.005+0000 7f379d038740 1 -- 172.21.15.86:0/3013433531 >> 172.21.15.86:0/3013433531 conn(0x55cdbc70fcb0 msgr2=0x55cdbc71bad0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:33.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.005+0000 7f379d038740 1 -- 172.21.15.86:0/3013433531 shutdown_connections 2024-04-21T08:47:33.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.005+0000 7f379d038740 1 -- 172.21.15.86:0/3013433531 wait complete. 2024-04-21T08:47:33.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.005+0000 7f379d038740 1 -- start start 2024-04-21T08:47:33.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.006+0000 7f379d038740 1 Processor -- start 2024-04-21T08:47:33.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.006+0000 7f379d038740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdbc715580 0x55cdbc726c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:33.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.006+0000 7f379d038740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cdbc715ea0 0x55cdbc727180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:33.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.006+0000 7f379d038740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc71ea80 0x55cdbc722390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:33.012 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.006+0000 7f379d038740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f37941b29d0 con 0x55cdbc71ea80 2024-04-21T08:47:33.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.006+0000 7f379d038740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f37941b2300 con 0x55cdbc715580 2024-04-21T08:47:33.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.006+0000 7f379d038740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f37941af560 con 0x55cdbc715ea0 2024-04-21T08:47:33.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.006+0000 7f379d038740 10 client.0 _ll_register_callbacks cb 0x7f3788013400 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:33.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.006+0000 7f37937fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc71ea80 0x55cdbc722390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:33.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.006+0000 7f37937fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc71ea80 0x55cdbc722390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52900/0 (socket says 172.21.15.86:52900) 2024-04-21T08:47:33.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.006+0000 7f37937fe640 1 -- 172.21.15.86:0/1177934362 learned_addr learned my addr 172.21.15.86:0/1177934362 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:33.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.006+0000 7f37937fe640 1 --2- 172.21.15.86:0/1177934362 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdbc715580 0x55cdbc726c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:33.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.006+0000 7f3793fff640 1 --2- 172.21.15.86:0/1177934362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cdbc715ea0 0x55cdbc727180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:33.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.006+0000 7f37937fe640 1 -- 172.21.15.86:0/1177934362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cdbc715ea0 msgr2=0x55cdbc727180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:33.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.006+0000 7f37937fe640 1 --2- 172.21.15.86:0/1177934362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cdbc715ea0 0x55cdbc727180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:33.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.006+0000 7f37937fe640 1 -- 172.21.15.86:0/1177934362 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdbc715580 msgr2=0x55cdbc726c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:33.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.006+0000 7f37937fe640 1 --2- 172.21.15.86:0/1177934362 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdbc715580 0x55cdbc726c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:33.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.006+0000 7f37937fe640 1 -- 172.21.15.86:0/1177934362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cdbc722b90 con 0x55cdbc71ea80 2024-04-21T08:47:33.013 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.006+0000 7f37937fe640 1 --2- 172.21.15.86:0/1177934362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc71ea80 0x55cdbc722390 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f378802ca50 tx=0x7f3788021040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4220c892641dfaa1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:33.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.006+0000 7f37737fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:47:33.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.007+0000 7f37737fe640 1 -- 172.21.15.86:0/1177934362 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3788019030 con 0x55cdbc71ea80 2024-04-21T08:47:33.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.007+0000 7f37737fe640 1 -- 172.21.15.86:0/1177934362 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f378802cc50 con 0x55cdbc71ea80 2024-04-21T08:47:33.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.007+0000 7f379d038740 1 -- 172.21.15.86:0/1177934362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3788025030 con 0x55cdbc71ea80 2024-04-21T08:47:33.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.007+0000 7f37737fe640 1 -- 172.21.15.86:0/1177934362 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f378803c040 con 0x55cdbc71ea80 2024-04-21T08:47:33.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.007+0000 7f379d038740 10 client.19236 fetch_fsmap learned FSMap version 29 2024-04-21T08:47:33.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.007+0000 7f379d038740 1 -- 172.21.15.86:0/1177934362 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cdbc794aa0 con 0x55cdbc71ea80 2024-04-21T08:47:33.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.007+0000 7f37737fe640 1 -- 172.21.15.86:0/1177934362 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3788025030 con 0x55cdbc71ea80 2024-04-21T08:47:33.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.007+0000 7f37737fe640 1 -- 172.21.15.86:0/1177934362 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3788042070 con 0x55cdbc71ea80 2024-04-21T08:47:33.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.007+0000 7f379d038740 10 client.19236 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:47:33.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.007+0000 7f379d038740 10 client.19236 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:33.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.007+0000 7f379d038740 20 client.19236 populate_metadata read hostname 'smithi086' 2024-04-21T08:47:33.014 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.007+0000 7f379d038740 1 --2- 172.21.15.86:0/1177934362 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55cdbc7950c0 0x55cdbc7974e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:33.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.007+0000 7f379d038740 4 client.19236 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:47:33.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.007+0000 7f379d038740 1 -- 172.21.15.86:0/1177934362 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55cdbc681a70 con 0x55cdbc7950c0 2024-04-21T08:47:33.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.007+0000 7f3793fff640 1 --2- 172.21.15.86:0/1177934362 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55cdbc7950c0 0x55cdbc7974e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:33.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.008+0000 7f37737fe640 10 client.19236 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:33.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.008+0000 7f3793fff640 1 --2- 172.21.15.86:0/1177934362 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55cdbc7950c0 0x55cdbc7974e0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5e78fa9a3736e7aa server_cookie=bbcf26514a7f95e5 in_seq=0 out_seq=0 2024-04-21T08:47:33.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.008+0000 7f37737fe640 1 -- 172.21.15.86:0/1177934362 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55cdbc681a70 con 0x55cdbc7950c0 2024-04-21T08:47:33.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.008+0000 7f37737fe640 10 client.19236 handle_command_reply: tid=0 2024-04-21T08:47:33.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.008+0000 7f379d038740 1 client.19236 shutdown 2024-04-21T08:47:33.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.008+0000 7f379d038740 1 -- 172.21.15.86:0/1177934362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc71ea80 msgr2=0x55cdbc722390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:33.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.008+0000 7f379d038740 1 --2- 172.21.15.86:0/1177934362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc71ea80 0x55cdbc722390 secure :-1 s=READY pgs=1675 cs=0 l=1 rev1=1 crypto rx=0x7f378802ca50 tx=0x7f3788021040 comp rx=0 tx=0).stop 2024-04-21T08:47:33.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.008+0000 7f379d038740 1 -- 172.21.15.86:0/1177934362 shutdown_connections 2024-04-21T08:47:33.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.008+0000 7f379d038740 1 --2- 172.21.15.86:0/1177934362 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55cdbc7950c0 0x55cdbc7974e0 crc :-1 s=READY pgs=264 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:33.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.008+0000 7f379d038740 1 --2- 172.21.15.86:0/1177934362 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cdbc71ea80 0x55cdbc722390 unknown :-1 s=CLOSED pgs=1675 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:33.015 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.008+0000 7f37737fe640 0 client.19236 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:33.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.008+0000 7f379d038740 1 --2- 172.21.15.86:0/1177934362 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cdbc715ea0 0x55cdbc727180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:33.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.008+0000 7f379d038740 1 --2- 172.21.15.86:0/1177934362 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cdbc715580 0x55cdbc726c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:33.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.008+0000 7f379d038740 1 -- 172.21.15.86:0/1177934362 >> 172.21.15.86:0/1177934362 conn(0x55cdbc70fcb0 msgr2=0x55cdbc71aaf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:33.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.008+0000 7f379d038740 1 -- 172.21.15.86:0/1177934362 shutdown_connections 2024-04-21T08:47:33.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.008+0000 7f379d038740 1 -- 172.21.15.86:0/1177934362 wait complete. 2024-04-21T08:47:33.016 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.008+0000 7f379d038740 20 client.19236 trim_cache size 0 max 16384 2024-04-21T08:47:33.016 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:47:33.016 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:47:33.016 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-21T08:47:33.016 INFO:teuthology.orchestra.run.smithi086.stdout: "cdfc81f1-8c4e-402a-a0ed-8db2d97548b5": { 2024-04-21T08:47:33.017 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-21T08:47:33.017 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "cdfc81f1-8c4e-402a-a0ed-8db2d97548b5", 2024-04-21T08:47:33.017 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-21T08:47:33.017 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:47:33.017 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:47:33.017 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:47:33.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.009+0000 7f379a367640 1 -- 172.21.15.86:0/3746525239 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f377c0a3d50 msgr2=0x7f377c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:33.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.009+0000 7f379a367640 1 --2- 172.21.15.86:0/3746525239 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f377c0a3d50 0x7f377c0a6210 secure :-1 s=READY pgs=273 cs=0 l=1 rev1=1 crypto rx=0x7f378c0f07d0 tx=0x7f378c0b3fc0 comp rx=0 tx=0).stop 2024-04-21T08:47:33.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.009+0000 7f379a367640 1 -- 172.21.15.86:0/3746525239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3794074cd0 msgr2=0x7f37941a68b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:33.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.009+0000 7f379a367640 1 --2- 172.21.15.86:0/3746525239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3794074cd0 0x7f37941a68b0 secure :-1 s=READY pgs=1671 cs=0 l=1 rev1=1 crypto rx=0x7f378c005e30 tx=0x7f378c034760 comp rx=0 tx=0).stop 2024-04-21T08:47:33.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.009+0000 7f379a367640 1 -- 172.21.15.86:0/3746525239 shutdown_connections 2024-04-21T08:47:33.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.009+0000 7f379a367640 1 --2- 172.21.15.86:0/3746525239 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f377c0a3d50 0x7f377c0a6210 unknown :-1 s=CLOSED pgs=273 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:33.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.009+0000 7f379a367640 1 --2- 172.21.15.86:0/3746525239 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3794074cd0 0x7f37941a68b0 unknown :-1 s=CLOSED pgs=1671 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:33.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.009+0000 7f379a367640 1 --2- 172.21.15.86:0/3746525239 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3794107270 0x7f3794074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:33.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.009+0000 7f379a367640 1 --2- 172.21.15.86:0/3746525239 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f37941021a0 0x7f3794074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:33.018 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.009+0000 7f379a367640 1 -- 172.21.15.86:0/3746525239 >> 172.21.15.86:0/3746525239 conn(0x7f37940793e0 msgr2=0x7f37940ffdf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:33.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.009+0000 7f379a367640 1 -- 172.21.15.86:0/3746525239 shutdown_connections 2024-04-21T08:47:33.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:47:33.009+0000 7f379a367640 1 -- 172.21.15.86:0/3746525239 wait complete. 2024-04-21T08:47:33.019 INFO:tasks.cephfs.filesystem:scrub status for tag:cdfc81f1-8c4e-402a-a0ed-8db2d97548b5 - {'path': '/', 'tag': 'cdfc81f1-8c4e-402a-a0ed-8db2d97548b5', 'options': 'recursive,force'} 2024-04-21T08:47:34.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:33 smithi088 ceph-mon[31259]: pgmap v272: 129 pgs: 129 active+clean; 4.0 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-04-21T08:47:34.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:33 smithi088 ceph-mon[31259]: scrub queued for path: / 2024-04-21T08:47:34.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:33 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:34.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:33 smithi088 ceph-mon[31259]: scrub summary: active paths [/] 2024-04-21T08:47:34.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:33 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:34.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:33 smithi179 ceph-mon[31829]: pgmap v272: 129 pgs: 129 active+clean; 4.0 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-04-21T08:47:34.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:33 smithi179 ceph-mon[31829]: scrub queued for path: / 2024-04-21T08:47:34.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:33 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:34.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:33 smithi179 ceph-mon[31829]: scrub summary: active paths [/] 2024-04-21T08:47:34.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:33 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:34.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:33 smithi086 ceph-mon[27939]: pgmap v272: 129 pgs: 129 active+clean; 4.0 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-04-21T08:47:34.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:33 smithi086 ceph-mon[27939]: scrub queued for path: / 2024-04-21T08:47:34.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:33 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:34.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:33 smithi086 ceph-mon[27939]: scrub summary: active paths [/] 2024-04-21T08:47:34.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:33 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:47:35.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:34 smithi088 ceph-mon[31259]: scrub summary: idle 2024-04-21T08:47:35.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:34 smithi179 ceph-mon[31829]: scrub summary: idle 2024-04-21T08:47:35.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:34 smithi086 ceph-mon[27939]: scrub summary: idle 2024-04-21T08:47:36.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:36 smithi086 ceph-mon[27939]: pgmap v273: 129 pgs: 129 active+clean; 4.0 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:47:36.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:36 smithi088 ceph-mon[31259]: pgmap v273: 129 pgs: 129 active+clean; 4.0 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:47:36.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:36 smithi179 ceph-mon[31829]: pgmap v273: 129 pgs: 129 active+clean; 4.0 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:47:38.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:38 smithi086 ceph-mon[27939]: pgmap v274: 129 pgs: 129 active+clean; 4.0 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:47:38.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:38 smithi088 ceph-mon[31259]: pgmap v274: 129 pgs: 129 active+clean; 4.0 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:47:38.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:38 smithi179 ceph-mon[31829]: pgmap v274: 129 pgs: 129 active+clean; 4.0 MiB data, 415 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:47:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.521+0000 7f5de959f640 1 -- 172.21.15.86:0/1376967463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de410c330 msgr2=0x7f5de410e7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.521+0000 7f5de959f640 1 --2- 172.21.15.86:0/1376967463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de410c330 0x7f5de410e7d0 secure :-1 s=READY pgs=1676 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc009ca0 tx=0x7f5ddc02c180 comp rx=0 tx=0).stop 2024-04-21T08:47:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.521+0000 7f5de959f640 1 -- 172.21.15.86:0/1376967463 shutdown_connections 2024-04-21T08:47:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.521+0000 7f5de959f640 1 --2- 172.21.15.86:0/1376967463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de410c330 0x7f5de410e7d0 unknown :-1 s=CLOSED pgs=1676 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.521+0000 7f5de959f640 1 --2- 172.21.15.86:0/1376967463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5de4103830 0x7f5de410bd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.521+0000 7f5de959f640 1 --2- 172.21.15.86:0/1376967463 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5de4102f10 0x7f5de41032f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.521+0000 7f5de959f640 1 -- 172.21.15.86:0/1376967463 >> 172.21.15.86:0/1376967463 conn(0x7f5de40fc770 msgr2=0x7f5de40feb90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.521+0000 7f5de959f640 1 -- 172.21.15.86:0/1376967463 shutdown_connections 2024-04-21T08:47:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.521+0000 7f5de959f640 1 -- 172.21.15.86:0/1376967463 wait complete. 2024-04-21T08:47:39.522 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.522+0000 7f5de959f640 1 Processor -- start 2024-04-21T08:47:39.523 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.522+0000 7f5de959f640 1 -- start start 2024-04-21T08:47:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.522+0000 7f5de959f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5de4102f10 0x7f5de4198560 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.522+0000 7f5de959f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de4103830 0x7f5de4198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.522+0000 7f5de959f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5de410c330 0x7f5de419ce20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.524 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.522+0000 7f5de959f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5de4111fc0 con 0x7f5de4103830 2024-04-21T08:47:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.522+0000 7f5de959f640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5de4111e40 con 0x7f5de4102f10 2024-04-21T08:47:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.522+0000 7f5de959f640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5de4112140 con 0x7f5de410c330 2024-04-21T08:47:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.522+0000 7f5de37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de4103830 0x7f5de4198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.522+0000 7f5de37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de4103830 0x7f5de4198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54350/0 (socket says 172.21.15.86:54350) 2024-04-21T08:47:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.522+0000 7f5de37fe640 1 -- 172.21.15.86:0/4195577436 learned_addr learned my addr 172.21.15.86:0/4195577436 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.522+0000 7f5de3fff640 1 --2- 172.21.15.86:0/4195577436 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5de4102f10 0x7f5de4198560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.522+0000 7f5de8d9e640 1 --2- 172.21.15.86:0/4195577436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5de410c330 0x7f5de419ce20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.523+0000 7f5de37fe640 1 -- 172.21.15.86:0/4195577436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5de410c330 msgr2=0x7f5de419ce20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.523+0000 7f5de37fe640 1 --2- 172.21.15.86:0/4195577436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5de410c330 0x7f5de419ce20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.523+0000 7f5de37fe640 1 -- 172.21.15.86:0/4195577436 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5de4102f10 msgr2=0x7f5de4198560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.523+0000 7f5de37fe640 1 --2- 172.21.15.86:0/4195577436 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5de4102f10 0x7f5de4198560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.523+0000 7f5de37fe640 1 -- 172.21.15.86:0/4195577436 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5de419d550 con 0x7f5de4103830 2024-04-21T08:47:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.523+0000 7f5de37fe640 1 --2- 172.21.15.86:0/4195577436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de4103830 0x7f5de4198aa0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc00d740 tx=0x7f5dcc00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.523+0000 7f5de17fa640 1 -- 172.21.15.86:0/4195577436 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc017070 con 0x7f5de4103830 2024-04-21T08:47:39.525 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.523+0000 7f5de17fa640 1 -- 172.21.15.86:0/4195577436 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dcc00de50 con 0x7f5de4103830 2024-04-21T08:47:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.523+0000 7f5de959f640 1 -- 172.21.15.86:0/4195577436 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5de419d840 con 0x7f5de4103830 2024-04-21T08:47:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.523+0000 7f5de17fa640 1 -- 172.21.15.86:0/4195577436 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc00f620 con 0x7f5de4103830 2024-04-21T08:47:39.526 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.523+0000 7f5de959f640 1 -- 172.21.15.86:0/4195577436 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5de41a50d0 con 0x7f5de4103830 2024-04-21T08:47:39.532 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.525+0000 7f5de17fa640 1 -- 172.21.15.86:0/4195577436 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5dcc009c30 con 0x7f5de4103830 2024-04-21T08:47:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.526+0000 7f5ded3ea740 1 Processor -- start 2024-04-21T08:47:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.526+0000 7f5ded3ea740 1 -- start start 2024-04-21T08:47:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.526+0000 7f5de17fa640 1 --2- 172.21.15.86:0/4195577436 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5dd00a3d10 0x7f5dd00a61d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.526+0000 7f5ded3ea740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606e1e6dac0 0x5606e1e6c810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.526+0000 7f5ded3ea740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5606e1e6cd50 0x5606e1e6d1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.526+0000 7f5ded3ea740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606e1e7a6e0 0x5606e1e7cad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.526+0000 7f5ded3ea740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5606e1dd18d0 con 0x5606e1e7a6e0 2024-04-21T08:47:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.526+0000 7f5ded3ea740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5606e1d6ee80 con 0x5606e1e6cd50 2024-04-21T08:47:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.526+0000 7f5ded3ea740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5606e1dcd890 con 0x5606e1e6dac0 2024-04-21T08:47:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.526+0000 7f5de8d9e640 1 --2- 172.21.15.86:0/4195577436 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5dd00a3d10 0x7f5dd00a61d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.526+0000 7f5de17fa640 1 -- 172.21.15.86:0/4195577436 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f5dcc0d06b0 con 0x7f5de4103830 2024-04-21T08:47:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.526+0000 7f5de8d9e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606e1e7a6e0 0x5606e1e7cad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.526+0000 7f5de37fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5606e1e6cd50 0x5606e1e6d1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.533 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.526+0000 7f5de3fff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606e1e6dac0 0x5606e1e6c810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.526+0000 7f5de37fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5606e1e6cd50 0x5606e1e6d1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.86:55292/0 (socket says 172.21.15.86:55292) 2024-04-21T08:47:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.526+0000 7f5de8d9e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606e1e7a6e0 0x5606e1e7cad0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54364/0 (socket says 172.21.15.86:54364) 2024-04-21T08:47:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.526+0000 7f5de37fe640 1 -- 172.21.15.86:0/984472785 learned_addr learned my addr 172.21.15.86:0/984472785 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.527+0000 7f5de3fff640 1 -- 172.21.15.86:0/984472785 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5606e1e6cd50 msgr2=0x5606e1e6d1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.527+0000 7f5de3fff640 1 --2- 172.21.15.86:0/984472785 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5606e1e6cd50 0x5606e1e6d1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.527+0000 7f5de3fff640 1 -- 172.21.15.86:0/984472785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606e1e7a6e0 msgr2=0x5606e1e7cad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.527+0000 7f5de3fff640 1 --2- 172.21.15.86:0/984472785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606e1e7a6e0 0x5606e1e7cad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.527+0000 7f5de3fff640 1 -- 172.21.15.86:0/984472785 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606e1d90320 con 0x5606e1e6dac0 2024-04-21T08:47:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.527+0000 7f5de37fe640 1 --2- 172.21.15.86:0/984472785 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5606e1e6cd50 0x5606e1e6d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:47:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.527+0000 7f5de8d9e640 1 --2- 172.21.15.86:0/4195577436 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5dd00a3d10 0x7f5dd00a61d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc004af0 tx=0x7f5ddc032340 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.527+0000 7f5de3fff640 1 --2- 172.21.15.86:0/984472785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606e1e6dac0 0x5606e1e6c810 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4004630 tx=0x7f5dd400bb20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8125b1011e7afd4e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.527+0000 7f5dda7fc640 1 -- 172.21.15.86:0/984472785 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4003ad0 con 0x5606e1e6dac0 2024-04-21T08:47:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.527+0000 7f5dda7fc640 1 -- 172.21.15.86:0/984472785 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dd4015070 con 0x5606e1e6dac0 2024-04-21T08:47:39.534 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.527+0000 7f5dda7fc640 1 -- 172.21.15.86:0/984472785 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dd4004b20 con 0x5606e1e6dac0 2024-04-21T08:47:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.528+0000 7f5ded3ea740 1 -- 172.21.15.86:0/984472785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606e1e6dac0 msgr2=0x5606e1e6c810 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.528+0000 7f5ded3ea740 1 --2- 172.21.15.86:0/984472785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606e1e6dac0 0x5606e1e6c810 secure :-1 s=READY pgs=72 cs=0 l=1 rev1=1 crypto rx=0x7f5dd4004630 tx=0x7f5dd400bb20 comp rx=0 tx=0).stop 2024-04-21T08:47:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.528+0000 7f5ded3ea740 1 -- 172.21.15.86:0/984472785 shutdown_connections 2024-04-21T08:47:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.528+0000 7f5ded3ea740 1 --2- 172.21.15.86:0/984472785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606e1e7a6e0 0x5606e1e7cad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.528+0000 7f5ded3ea740 1 --2- 172.21.15.86:0/984472785 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5606e1e6cd50 0x5606e1e6d1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.528+0000 7f5ded3ea740 1 --2- 172.21.15.86:0/984472785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606e1e6dac0 0x5606e1e6c810 unknown :-1 s=CLOSED pgs=72 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.528+0000 7f5ded3ea740 1 -- 172.21.15.86:0/984472785 >> 172.21.15.86:0/984472785 conn(0x5606e1e33c90 msgr2=0x5606e1e67a40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.528+0000 7f5ded3ea740 1 -- 172.21.15.86:0/984472785 shutdown_connections 2024-04-21T08:47:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.528+0000 7f5ded3ea740 1 -- 172.21.15.86:0/984472785 wait complete. 2024-04-21T08:47:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.528+0000 7f5ded3ea740 1 -- start start 2024-04-21T08:47:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.529+0000 7f5ded3ea740 1 Processor -- start 2024-04-21T08:47:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.529+0000 7f5ded3ea740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5606e1e6cd50 0x5606e1e7a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.529+0000 7f5ded3ea740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606e1e6dac0 0x5606e1e773e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.535 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.529+0000 7f5ded3ea740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606e1e7a6e0 0x5606e1e77940 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.529+0000 7f5ded3ea740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5606e1e7d410 con 0x5606e1e6dac0 2024-04-21T08:47:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.529+0000 7f5ded3ea740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5606e1d59eb0 con 0x5606e1e6cd50 2024-04-21T08:47:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.529+0000 7f5ded3ea740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5606e1e7d590 con 0x5606e1e7a6e0 2024-04-21T08:47:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.529+0000 7f5ded3ea740 10 client.0 _ll_register_callbacks cb 0x5606e1d86b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.529+0000 7f5de37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606e1e6dac0 0x5606e1e773e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.529+0000 7f5de37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606e1e6dac0 0x5606e1e773e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54366/0 (socket says 172.21.15.86:54366) 2024-04-21T08:47:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.529+0000 7f5de37fe640 1 -- 172.21.15.86:0/2842607277 learned_addr learned my addr 172.21.15.86:0/2842607277 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.529+0000 7f5de8d9e640 1 --2- 172.21.15.86:0/2842607277 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606e1e7a6e0 0x5606e1e77940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.529+0000 7f5de3fff640 1 --2- 172.21.15.86:0/2842607277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5606e1e6cd50 0x5606e1e7a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.529+0000 7f5de37fe640 1 -- 172.21.15.86:0/2842607277 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606e1e7a6e0 msgr2=0x5606e1e77940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.529+0000 7f5de37fe640 1 --2- 172.21.15.86:0/2842607277 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606e1e7a6e0 0x5606e1e77940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.529+0000 7f5de37fe640 1 -- 172.21.15.86:0/2842607277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5606e1e6cd50 msgr2=0x5606e1e7a050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.529+0000 7f5de37fe640 1 --2- 172.21.15.86:0/2842607277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5606e1e6cd50 0x5606e1e7a050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.536 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.529+0000 7f5de37fe640 1 -- 172.21.15.86:0/2842607277 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606e1ae4210 con 0x5606e1e6dac0 2024-04-21T08:47:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.529+0000 7f5de37fe640 1 --2- 172.21.15.86:0/2842607277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606e1e6dac0 0x5606e1e773e0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc098730 tx=0x7f5dcc09c0a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b55a327e13ebd9a2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.529+0000 7f5dc37fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:47:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.530+0000 7f5dc37fe640 1 -- 172.21.15.86:0/2842607277 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5dcc020040 con 0x5606e1e6dac0 2024-04-21T08:47:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.530+0000 7f5dc37fe640 1 -- 172.21.15.86:0/2842607277 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f5dcc076a40 con 0x5606e1e6dac0 2024-04-21T08:47:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.530+0000 7f5ded3ea740 1 -- 172.21.15.86:0/2842607277 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5dd4015070 con 0x5606e1e6dac0 2024-04-21T08:47:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.530+0000 7f5dc37fe640 1 -- 172.21.15.86:0/2842607277 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5dcc0cde50 con 0x5606e1e6dac0 2024-04-21T08:47:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.530+0000 7f5ded3ea740 10 client.19248 fetch_fsmap learned FSMap version 29 2024-04-21T08:47:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.530+0000 7f5ded3ea740 1 -- 172.21.15.86:0/2842607277 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5606e1bb0620 con 0x5606e1e6dac0 2024-04-21T08:47:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.530+0000 7f5dc37fe640 1 -- 172.21.15.86:0/2842607277 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5dd4015070 con 0x5606e1e6dac0 2024-04-21T08:47:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.530+0000 7f5dc37fe640 1 -- 172.21.15.86:0/2842607277 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5dcc0a01e0 con 0x5606e1e6dac0 2024-04-21T08:47:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.530+0000 7f5ded3ea740 10 client.19248 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:47:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.530+0000 7f5ded3ea740 10 client.19248 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.530+0000 7f5ded3ea740 20 client.19248 populate_metadata read hostname 'smithi086' 2024-04-21T08:47:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.530+0000 7f5ded3ea740 1 --2- 172.21.15.86:0/2842607277 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5606e1e8a660 0x5606e1e8ca90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.537 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.530+0000 7f5ded3ea740 4 client.19248 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.530+0000 7f5ded3ea740 1 -- 172.21.15.86:0/2842607277 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5606e1de5880 con 0x5606e1e8a660 2024-04-21T08:47:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.530+0000 7f5de3fff640 1 --2- 172.21.15.86:0/2842607277 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5606e1e8a660 0x5606e1e8ca90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.531+0000 7f5de3fff640 1 --2- 172.21.15.86:0/2842607277 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5606e1e8a660 0x5606e1e8ca90 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=be9249b9f2064e6f server_cookie=640514d38321ee7a in_seq=0 out_seq=0 2024-04-21T08:47:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.531+0000 7f5dc37fe640 10 client.19248 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.532+0000 7f5dc37fe640 1 -- 172.21.15.86:0/2842607277 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5606e1de5880 con 0x5606e1e8a660 2024-04-21T08:47:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.532+0000 7f5dc37fe640 10 client.19248 handle_command_reply: tid=0 2024-04-21T08:47:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.532+0000 7f5ded3ea740 1 client.19248 shutdown 2024-04-21T08:47:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.532+0000 7f5ded3ea740 1 -- 172.21.15.86:0/2842607277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606e1e6dac0 msgr2=0x5606e1e773e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.532+0000 7f5ded3ea740 1 --2- 172.21.15.86:0/2842607277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606e1e6dac0 0x5606e1e773e0 secure :-1 s=READY pgs=1678 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc098730 tx=0x7f5dcc09c0a0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.532+0000 7f5ded3ea740 1 -- 172.21.15.86:0/2842607277 shutdown_connections 2024-04-21T08:47:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.532+0000 7f5ded3ea740 1 --2- 172.21.15.86:0/2842607277 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5606e1e8a660 0x5606e1e8ca90 crc :-1 s=READY pgs=265 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.532+0000 7f5ded3ea740 1 --2- 172.21.15.86:0/2842607277 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606e1e7a6e0 0x5606e1e77940 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.532+0000 7f5ded3ea740 1 --2- 172.21.15.86:0/2842607277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606e1e6dac0 0x5606e1e773e0 unknown :-1 s=CLOSED pgs=1678 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.538 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.532+0000 7f5dc37fe640 0 client.19248 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.532+0000 7f5ded3ea740 1 --2- 172.21.15.86:0/2842607277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5606e1e6cd50 0x5606e1e7a050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.532+0000 7f5ded3ea740 1 -- 172.21.15.86:0/2842607277 >> 172.21.15.86:0/2842607277 conn(0x5606e1e33c90 msgr2=0x5606e1e69790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.532+0000 7f5ded3ea740 1 -- 172.21.15.86:0/2842607277 shutdown_connections 2024-04-21T08:47:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.532+0000 7f5ded3ea740 1 -- 172.21.15.86:0/2842607277 wait complete. 2024-04-21T08:47:39.539 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.532+0000 7f5ded3ea740 20 client.19248 trim_cache size 0 max 16384 2024-04-21T08:47:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.547+0000 7f5ded3ea740 1 Processor -- start 2024-04-21T08:47:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.547+0000 7f5ded3ea740 1 -- start start 2024-04-21T08:47:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.547+0000 7f5ded3ea740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5606e1e6cd50 0x5606e1e66960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.547+0000 7f5ded3ea740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606e1e6dac0 0x5606e1e79b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.547+0000 7f5ded3ea740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606e1e7a6e0 0x5606e1e7a050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.547+0000 7f5ded3ea740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5de41ad070 con 0x5606e1e7a6e0 2024-04-21T08:47:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.547+0000 7f5ded3ea740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5de41aebb0 con 0x5606e1e6cd50 2024-04-21T08:47:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.547+0000 7f5ded3ea740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5de41ad740 con 0x5606e1e6dac0 2024-04-21T08:47:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.547+0000 7f5de8d9e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606e1e7a6e0 0x5606e1e7a050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.552 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.547+0000 7f5de8d9e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606e1e7a6e0 0x5606e1e7a050 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54376/0 (socket says 172.21.15.86:54376) 2024-04-21T08:47:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.547+0000 7f5de8d9e640 1 -- 172.21.15.86:0/4080907031 learned_addr learned my addr 172.21.15.86:0/4080907031 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.547+0000 7f5de3fff640 1 --2- 172.21.15.86:0/4080907031 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5606e1e6cd50 0x5606e1e66960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.547+0000 7f5de37fe640 1 --2- 172.21.15.86:0/4080907031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606e1e6dac0 0x5606e1e79b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.547+0000 7f5de8d9e640 1 -- 172.21.15.86:0/4080907031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606e1e6dac0 msgr2=0x5606e1e79b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.547+0000 7f5de8d9e640 1 --2- 172.21.15.86:0/4080907031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606e1e6dac0 0x5606e1e79b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.547+0000 7f5de8d9e640 1 -- 172.21.15.86:0/4080907031 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5606e1e6cd50 msgr2=0x5606e1e66960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.547+0000 7f5de8d9e640 1 --2- 172.21.15.86:0/4080907031 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5606e1e6cd50 0x5606e1e66960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.547+0000 7f5de8d9e640 1 -- 172.21.15.86:0/4080907031 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5606e1e6c020 con 0x5606e1e7a6e0 2024-04-21T08:47:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.548+0000 7f5de8d9e640 1 --2- 172.21.15.86:0/4080907031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606e1e7a6e0 0x5606e1e7a050 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc009ca0 tx=0x7f5ddc047000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f9019abfccc2a66 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.548+0000 7f5dd9ffb640 1 -- 172.21.15.86:0/4080907031 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc053070 con 0x5606e1e7a6e0 2024-04-21T08:47:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.548+0000 7f5dd9ffb640 1 -- 172.21.15.86:0/4080907031 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ddc032560 con 0x5606e1e7a6e0 2024-04-21T08:47:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.548+0000 7f5dd9ffb640 1 -- 172.21.15.86:0/4080907031 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc039040 con 0x5606e1e7a6e0 2024-04-21T08:47:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.548+0000 7f5ded3ea740 1 -- 172.21.15.86:0/4080907031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606e1e7a6e0 msgr2=0x5606e1e7a050 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.553 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.548+0000 7f5ded3ea740 1 --2- 172.21.15.86:0/4080907031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606e1e7a6e0 0x5606e1e7a050 secure :-1 s=READY pgs=1679 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc009ca0 tx=0x7f5ddc047000 comp rx=0 tx=0).stop 2024-04-21T08:47:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.548+0000 7f5ded3ea740 1 -- 172.21.15.86:0/4080907031 shutdown_connections 2024-04-21T08:47:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.548+0000 7f5ded3ea740 1 --2- 172.21.15.86:0/4080907031 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606e1e7a6e0 0x5606e1e7a050 unknown :-1 s=CLOSED pgs=1679 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.548+0000 7f5ded3ea740 1 --2- 172.21.15.86:0/4080907031 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606e1e6dac0 0x5606e1e79b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.548+0000 7f5ded3ea740 1 --2- 172.21.15.86:0/4080907031 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5606e1e6cd50 0x5606e1e66960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.548+0000 7f5ded3ea740 1 -- 172.21.15.86:0/4080907031 >> 172.21.15.86:0/4080907031 conn(0x5606e1e33c90 msgr2=0x5606e1e66250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.548+0000 7f5ded3ea740 1 -- 172.21.15.86:0/4080907031 shutdown_connections 2024-04-21T08:47:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.548+0000 7f5ded3ea740 1 -- 172.21.15.86:0/4080907031 wait complete. 2024-04-21T08:47:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.549+0000 7f5ded3ea740 1 -- start start 2024-04-21T08:47:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.549+0000 7f5ded3ea740 1 Processor -- start 2024-04-21T08:47:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.549+0000 7f5ded3ea740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606e1e6cd50 0x5606e1e74450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.549+0000 7f5ded3ea740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5606e1e6dac0 0x5606e1e74990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.549+0000 7f5ded3ea740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606e1e7a6e0 0x5606e1eefc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.549+0000 7f5ded3ea740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5de41ae4e0 con 0x5606e1e7a6e0 2024-04-21T08:47:39.554 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.549+0000 7f5ded3ea740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5de41ade10 con 0x5606e1e6dac0 2024-04-21T08:47:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.549+0000 7f5ded3ea740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5de41ab070 con 0x5606e1e6cd50 2024-04-21T08:47:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.549+0000 7f5ded3ea740 10 client.0 _ll_register_callbacks cb 0x7f5dcc0cd970 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.549+0000 7f5de8d9e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606e1e7a6e0 0x5606e1eefc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.549+0000 7f5de8d9e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606e1e7a6e0 0x5606e1eefc10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:54380/0 (socket says 172.21.15.86:54380) 2024-04-21T08:47:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.549+0000 7f5de8d9e640 1 -- 172.21.15.86:0/4261273625 learned_addr learned my addr 172.21.15.86:0/4261273625 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.549+0000 7f5de37fe640 1 --2- 172.21.15.86:0/4261273625 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5606e1e6dac0 0x5606e1e74990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.550+0000 7f5de3fff640 1 --2- 172.21.15.86:0/4261273625 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606e1e6cd50 0x5606e1e74450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.550+0000 7f5de8d9e640 1 -- 172.21.15.86:0/4261273625 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606e1e6cd50 msgr2=0x5606e1e74450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.550+0000 7f5de8d9e640 1 --2- 172.21.15.86:0/4261273625 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606e1e6cd50 0x5606e1e74450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.550+0000 7f5de8d9e640 1 -- 172.21.15.86:0/4261273625 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5606e1e6dac0 msgr2=0x5606e1e74990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.550+0000 7f5de8d9e640 1 --2- 172.21.15.86:0/4261273625 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5606e1e6dac0 0x5606e1e74990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.550+0000 7f5de8d9e640 1 -- 172.21.15.86:0/4261273625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5606e1ef0150 con 0x5606e1e7a6e0 2024-04-21T08:47:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.550+0000 7f5de8d9e640 1 --2- 172.21.15.86:0/4261273625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606e1e7a6e0 0x5606e1eefc10 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc02c6c0 tx=0x7f5ddc004590 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6f7285c3d1c7da4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:39.555 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.550+0000 7f5dc37fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:47:39.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.550+0000 7f5dc37fe640 1 -- 172.21.15.86:0/4261273625 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ddc053030 con 0x5606e1e7a6e0 2024-04-21T08:47:39.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.550+0000 7f5dc37fe640 1 -- 172.21.15.86:0/4261273625 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f5ddc058460 con 0x5606e1e7a6e0 2024-04-21T08:47:39.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.550+0000 7f5ded3ea740 1 -- 172.21.15.86:0/4261273625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5ddc032560 con 0x5606e1e7a6e0 2024-04-21T08:47:39.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.550+0000 7f5dc37fe640 1 -- 172.21.15.86:0/4261273625 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5ddc061050 con 0x5606e1e7a6e0 2024-04-21T08:47:39.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.550+0000 7f5ded3ea740 10 client.19254 fetch_fsmap learned FSMap version 29 2024-04-21T08:47:39.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.550+0000 7f5ded3ea740 1 -- 172.21.15.86:0/4261273625 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5606e1ef12c0 con 0x5606e1e7a6e0 2024-04-21T08:47:39.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.550+0000 7f5dc37fe640 1 -- 172.21.15.86:0/4261273625 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ddc032560 con 0x5606e1e7a6e0 2024-04-21T08:47:39.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.550+0000 7f5dc37fe640 1 -- 172.21.15.86:0/4261273625 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5ddc03f040 con 0x5606e1e7a6e0 2024-04-21T08:47:39.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.550+0000 7f5ded3ea740 10 client.19254 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:47:39.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.551+0000 7f5ded3ea740 10 client.19254 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:39.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.551+0000 7f5ded3ea740 20 client.19254 populate_metadata read hostname 'smithi086' 2024-04-21T08:47:39.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.551+0000 7f5ded3ea740 1 --2- 172.21.15.86:0/4261273625 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5606e1ef1900 0x5606e1ef3d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:39.556 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.551+0000 7f5ded3ea740 4 client.19254 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:47:39.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.551+0000 7f5ded3ea740 1 -- 172.21.15.86:0/4261273625 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5606e1d713d0 con 0x5606e1ef1900 2024-04-21T08:47:39.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.551+0000 7f5de3fff640 1 --2- 172.21.15.86:0/4261273625 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5606e1ef1900 0x5606e1ef3d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:39.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.551+0000 7f5dc37fe640 10 client.19254 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:39.557 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.551+0000 7f5de3fff640 1 --2- 172.21.15.86:0/4261273625 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5606e1ef1900 0x5606e1ef3d20 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=da5d57d3e2f04fc2 server_cookie=e2dbc7540a8b551e in_seq=0 out_seq=0 2024-04-21T08:47:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.565+0000 7f5dc37fe640 1 -- 172.21.15.86:0/4261273625 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5606e1d713d0 con 0x5606e1ef1900 2024-04-21T08:47:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.565+0000 7f5dc37fe640 10 client.19254 handle_command_reply: tid=0 2024-04-21T08:47:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.565+0000 7f5ded3ea740 1 client.19254 shutdown 2024-04-21T08:47:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.565+0000 7f5ded3ea740 1 -- 172.21.15.86:0/4261273625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606e1e7a6e0 msgr2=0x5606e1eefc10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.565+0000 7f5ded3ea740 1 --2- 172.21.15.86:0/4261273625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606e1e7a6e0 0x5606e1eefc10 secure :-1 s=READY pgs=1680 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc02c6c0 tx=0x7f5ddc004590 comp rx=0 tx=0).stop 2024-04-21T08:47:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.565+0000 7f5ded3ea740 1 -- 172.21.15.86:0/4261273625 shutdown_connections 2024-04-21T08:47:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.565+0000 7f5ded3ea740 1 --2- 172.21.15.86:0/4261273625 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5606e1ef1900 0x5606e1ef3d20 crc :-1 s=READY pgs=266 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.566 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.565+0000 7f5ded3ea740 1 --2- 172.21.15.86:0/4261273625 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5606e1e7a6e0 0x5606e1eefc10 unknown :-1 s=CLOSED pgs=1680 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.565+0000 7f5ded3ea740 1 --2- 172.21.15.86:0/4261273625 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5606e1e6dac0 0x5606e1e74990 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.565+0000 7f5ded3ea740 1 --2- 172.21.15.86:0/4261273625 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5606e1e6cd50 0x5606e1e74450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.565+0000 7f5dc37fe640 0 client.19254 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.565+0000 7f5ded3ea740 1 -- 172.21.15.86:0/4261273625 >> 172.21.15.86:0/4261273625 conn(0x5606e1e33c90 msgr2=0x5606e1e66ae0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.565+0000 7f5ded3ea740 1 -- 172.21.15.86:0/4261273625 shutdown_connections 2024-04-21T08:47:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.565+0000 7f5ded3ea740 1 -- 172.21.15.86:0/4261273625 wait complete. 2024-04-21T08:47:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.566+0000 7f5ded3ea740 20 client.19254 trim_cache size 0 max 16384 2024-04-21T08:47:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:47:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:47:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:47:39.567 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:47:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.566+0000 7f5de959f640 1 -- 172.21.15.86:0/4195577436 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5dd00a3d10 msgr2=0x7f5dd00a61d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.566+0000 7f5de959f640 1 --2- 172.21.15.86:0/4195577436 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5dd00a3d10 0x7f5dd00a61d0 secure :-1 s=READY pgs=274 cs=0 l=1 rev1=1 crypto rx=0x7f5ddc004af0 tx=0x7f5ddc032340 comp rx=0 tx=0).stop 2024-04-21T08:47:39.568 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.566+0000 7f5de959f640 1 -- 172.21.15.86:0/4195577436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de4103830 msgr2=0x7f5de4198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.566+0000 7f5de959f640 1 --2- 172.21.15.86:0/4195577436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de4103830 0x7f5de4198aa0 secure :-1 s=READY pgs=1677 cs=0 l=1 rev1=1 crypto rx=0x7f5dcc00d740 tx=0x7f5dcc00dc00 comp rx=0 tx=0).stop 2024-04-21T08:47:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.566+0000 7f5de959f640 1 -- 172.21.15.86:0/4195577436 shutdown_connections 2024-04-21T08:47:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.566+0000 7f5de959f640 1 --2- 172.21.15.86:0/4195577436 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5dd00a3d10 0x7f5dd00a61d0 unknown :-1 s=CLOSED pgs=274 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.566+0000 7f5de959f640 1 --2- 172.21.15.86:0/4195577436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5de410c330 0x7f5de419ce20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.566+0000 7f5de959f640 1 --2- 172.21.15.86:0/4195577436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5de4103830 0x7f5de4198aa0 unknown :-1 s=CLOSED pgs=1677 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.566+0000 7f5de959f640 1 --2- 172.21.15.86:0/4195577436 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5de4102f10 0x7f5de4198560 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.566+0000 7f5de959f640 1 -- 172.21.15.86:0/4195577436 >> 172.21.15.86:0/4195577436 conn(0x7f5de40fc770 msgr2=0x7f5de40fe070 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.566+0000 7f5de959f640 1 -- 172.21.15.86:0/4195577436 shutdown_connections 2024-04-21T08:47:39.569 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:39.566+0000 7f5de959f640 1 -- 172.21.15.86:0/4195577436 wait complete. 2024-04-21T08:47:39.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:39 smithi088 ceph-mon[31259]: pgmap v275: 129 pgs: 129 active+clean; 4.2 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-21T08:47:39.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:39 smithi179 ceph-mon[31829]: pgmap v275: 129 pgs: 129 active+clean; 4.2 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-21T08:47:40.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:39 smithi086 ceph-mon[27939]: pgmap v275: 129 pgs: 129 active+clean; 4.2 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-21T08:47:42.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:41 smithi086 ceph-mon[27939]: pgmap v276: 129 pgs: 129 active+clean; 4.2 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-04-21T08:47:42.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:41 smithi088 ceph-mon[31259]: pgmap v276: 129 pgs: 129 active+clean; 4.2 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-04-21T08:47:42.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:41 smithi179 ceph-mon[31829]: pgmap v276: 129 pgs: 129 active+clean; 4.2 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 39 KiB/s rd, 35 KiB/s wr, 14 op/s 2024-04-21T08:47:44.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:43 smithi086 ceph-mon[27939]: pgmap v277: 129 pgs: 129 active+clean; 4.4 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 23 op/s 2024-04-21T08:47:44.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:43 smithi088 ceph-mon[31259]: pgmap v277: 129 pgs: 129 active+clean; 4.4 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 23 op/s 2024-04-21T08:47:44.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:43 smithi179 ceph-mon[31829]: pgmap v277: 129 pgs: 129 active+clean; 4.4 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 58 KiB/s wr, 23 op/s 2024-04-21T08:47:46.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:45 smithi086 ceph-mon[27939]: pgmap v278: 129 pgs: 129 active+clean; 4.4 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:47:46.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:45 smithi088 ceph-mon[31259]: pgmap v278: 129 pgs: 129 active+clean; 4.4 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:47:46.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:45 smithi179 ceph-mon[31829]: pgmap v278: 129 pgs: 129 active+clean; 4.4 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:47:48.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:47 smithi088 ceph-mon[31259]: pgmap v279: 129 pgs: 129 active+clean; 4.4 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:47:48.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:47 smithi179 ceph-mon[31829]: pgmap v279: 129 pgs: 129 active+clean; 4.4 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:47:48.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:47 smithi086 ceph-mon[27939]: pgmap v279: 129 pgs: 129 active+clean; 4.4 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:47:48.443 INFO:tasks.workunit.client.0.smithi086.stdout:375/600: open fd = 3 2024-04-21T08:47:48.443 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.443 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.443 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.443 INFO:tasks.workunit.client.0.smithi086.stdout:376/600: open fd = 3 2024-04-21T08:47:48.443 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.443 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.444 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.444 INFO:tasks.workunit.client.0.smithi086.stdout:377/600: open fd = 3 2024-04-21T08:47:48.444 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.444 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.444 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.444 INFO:tasks.workunit.client.0.smithi086.stdout:378/600: open fd = 3 2024-04-21T08:47:48.444 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.444 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.444 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.444 INFO:tasks.workunit.client.0.smithi086.stdout:379/600: open fd = 3 2024-04-21T08:47:48.444 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.444 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.444 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.444 INFO:tasks.workunit.client.0.smithi086.stdout:380/600: open fd = 3 2024-04-21T08:47:48.444 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.445 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.445 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.445 INFO:tasks.workunit.client.0.smithi086.stdout:381/600: open fd = 3 2024-04-21T08:47:48.445 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.445 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.445 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.445 INFO:tasks.workunit.client.0.smithi086.stdout:382/600: open fd = 3 2024-04-21T08:47:48.445 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.445 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.445 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.445 INFO:tasks.workunit.client.0.smithi086.stdout:383/600: open fd = 3 2024-04-21T08:47:48.445 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.445 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.446 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.446 INFO:tasks.workunit.client.0.smithi086.stdout:384/600: open fd = 3 2024-04-21T08:47:48.446 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.446 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.446 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.446 INFO:tasks.workunit.client.0.smithi086.stdout:385/600: open fd = 3 2024-04-21T08:47:48.446 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.446 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.446 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.446 INFO:tasks.workunit.client.0.smithi086.stdout:386/600: open fd = 3 2024-04-21T08:47:48.446 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.446 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.446 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.446 INFO:tasks.workunit.client.0.smithi086.stdout:387/600: open fd = 3 2024-04-21T08:47:48.447 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.447 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.447 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.447 INFO:tasks.workunit.client.0.smithi086.stdout:388/600: open fd = 3 2024-04-21T08:47:48.447 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.447 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.447 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.447 INFO:tasks.workunit.client.0.smithi086.stdout:389/600: open fd = 3 2024-04-21T08:47:48.447 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.447 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.447 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.447 INFO:tasks.workunit.client.0.smithi086.stdout:390/600: open fd = 3 2024-04-21T08:47:48.447 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.447 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.448 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.448 INFO:tasks.workunit.client.0.smithi086.stdout:391/600: open fd = 3 2024-04-21T08:47:48.448 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.448 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.448 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.448 INFO:tasks.workunit.client.0.smithi086.stdout:392/600: open fd = 3 2024-04-21T08:47:48.448 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.448 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.448 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.448 INFO:tasks.workunit.client.0.smithi086.stdout:393/600: open fd = 3 2024-04-21T08:47:48.448 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.448 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.448 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.448 INFO:tasks.workunit.client.0.smithi086.stdout:394/600: open fd = 3 2024-04-21T08:47:48.448 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.449 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.449 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.449 INFO:tasks.workunit.client.0.smithi086.stdout:395/600: open fd = 3 2024-04-21T08:47:48.449 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.449 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.449 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.449 INFO:tasks.workunit.client.0.smithi086.stdout:396/600: open fd = 3 2024-04-21T08:47:48.449 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.449 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.449 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.449 INFO:tasks.workunit.client.0.smithi086.stdout:397/600: open fd = 3 2024-04-21T08:47:48.449 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.449 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.449 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.449 INFO:tasks.workunit.client.0.smithi086.stdout:398/600: open fd = 3 2024-04-21T08:47:48.450 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.450 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.450 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.450 INFO:tasks.workunit.client.0.smithi086.stdout:399/600: open fd = 3 2024-04-21T08:47:48.450 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.450 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.450 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.450 INFO:tasks.workunit.client.0.smithi086.stdout:400/600: open fd = 3 2024-04-21T08:47:48.450 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.450 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.450 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.450 INFO:tasks.workunit.client.0.smithi086.stdout:401/600: open fd = 3 2024-04-21T08:47:48.450 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.451 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.451 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.451 INFO:tasks.workunit.client.0.smithi086.stdout:402/600: open fd = 3 2024-04-21T08:47:48.451 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.451 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.451 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.451 INFO:tasks.workunit.client.0.smithi086.stdout:403/600: open fd = 3 2024-04-21T08:47:48.451 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.451 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.451 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.451 INFO:tasks.workunit.client.0.smithi086.stdout:404/600: open fd = 3 2024-04-21T08:47:48.451 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.451 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.451 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.451 INFO:tasks.workunit.client.0.smithi086.stdout:405/600: open fd = 3 2024-04-21T08:47:48.452 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.452 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.452 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.452 INFO:tasks.workunit.client.0.smithi086.stdout:406/600: open fd = 3 2024-04-21T08:47:48.452 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.452 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.452 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.452 INFO:tasks.workunit.client.0.smithi086.stdout:407/600: open fd = 3 2024-04-21T08:47:48.452 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.452 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.452 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.452 INFO:tasks.workunit.client.0.smithi086.stdout:408/600: open fd = 3 2024-04-21T08:47:48.452 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.452 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.453 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.453 INFO:tasks.workunit.client.0.smithi086.stdout:409/600: open fd = 3 2024-04-21T08:47:48.453 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.453 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.453 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.453 INFO:tasks.workunit.client.0.smithi086.stdout:410/600: open fd = 3 2024-04-21T08:47:48.453 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.453 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.453 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.453 INFO:tasks.workunit.client.0.smithi086.stdout:411/600: open fd = 3 2024-04-21T08:47:48.453 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.453 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.453 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.453 INFO:tasks.workunit.client.0.smithi086.stdout:412/600: open fd = 3 2024-04-21T08:47:48.454 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.454 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.454 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.454 INFO:tasks.workunit.client.0.smithi086.stdout:413/600: open fd = 3 2024-04-21T08:47:48.454 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.454 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.454 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.454 INFO:tasks.workunit.client.0.smithi086.stdout:414/600: open fd = 3 2024-04-21T08:47:48.454 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.454 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.454 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.454 INFO:tasks.workunit.client.0.smithi086.stdout:415/600: open fd = 3 2024-04-21T08:47:48.454 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.455 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.455 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.455 INFO:tasks.workunit.client.0.smithi086.stdout:416/600: open fd = 3 2024-04-21T08:47:48.455 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.455 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.455 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.455 INFO:tasks.workunit.client.0.smithi086.stdout:417/600: open fd = 3 2024-04-21T08:47:48.455 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.455 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.455 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.455 INFO:tasks.workunit.client.0.smithi086.stdout:418/600: open fd = 3 2024-04-21T08:47:48.455 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.455 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.455 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.456 INFO:tasks.workunit.client.0.smithi086.stdout:419/600: open fd = 3 2024-04-21T08:47:48.456 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.456 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.456 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.456 INFO:tasks.workunit.client.0.smithi086.stdout:420/600: open fd = 3 2024-04-21T08:47:48.456 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.456 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.456 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.456 INFO:tasks.workunit.client.0.smithi086.stdout:421/600: open fd = 3 2024-04-21T08:47:48.456 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.456 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.456 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.456 INFO:tasks.workunit.client.0.smithi086.stdout:422/600: open fd = 3 2024-04-21T08:47:48.456 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.456 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.457 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.457 INFO:tasks.workunit.client.0.smithi086.stdout:423/600: open fd = 3 2024-04-21T08:47:48.457 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.457 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.457 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.457 INFO:tasks.workunit.client.0.smithi086.stdout:424/600: open fd = 3 2024-04-21T08:47:48.457 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.457 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.457 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.457 INFO:tasks.workunit.client.0.smithi086.stdout:425/600: open fd = 3 2024-04-21T08:47:48.457 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.457 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.457 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.457 INFO:tasks.workunit.client.0.smithi086.stdout:426/600: open fd = 3 2024-04-21T08:47:48.458 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.458 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.458 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.458 INFO:tasks.workunit.client.0.smithi086.stdout:427/600: open fd = 3 2024-04-21T08:47:48.458 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.458 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.458 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.458 INFO:tasks.workunit.client.0.smithi086.stdout:428/600: open fd = 3 2024-04-21T08:47:48.458 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.458 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.458 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.458 INFO:tasks.workunit.client.0.smithi086.stdout:429/600: open fd = 3 2024-04-21T08:47:48.458 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.458 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.459 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.459 INFO:tasks.workunit.client.0.smithi086.stdout:430/600: open fd = 3 2024-04-21T08:47:48.459 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.459 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.459 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.459 INFO:tasks.workunit.client.0.smithi086.stdout:431/600: open fd = 3 2024-04-21T08:47:48.459 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.459 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.459 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.459 INFO:tasks.workunit.client.0.smithi086.stdout:432/600: open fd = 3 2024-04-21T08:47:48.459 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.459 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.459 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.459 INFO:tasks.workunit.client.0.smithi086.stdout:433/600: open fd = 3 2024-04-21T08:47:48.459 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.459 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.460 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.460 INFO:tasks.workunit.client.0.smithi086.stdout:434/600: open fd = 3 2024-04-21T08:47:48.460 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.460 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.460 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.460 INFO:tasks.workunit.client.0.smithi086.stdout:435/600: open fd = 3 2024-04-21T08:47:48.460 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.460 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.460 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:48.460 INFO:tasks.workunit.client.0.smithi086.stdout:436/600: open fd = 3 2024-04-21T08:47:48.460 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.460 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:47:48.460 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:47:50.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:49 smithi088 ceph-mon[31259]: pgmap v280: 129 pgs: 129 active+clean; 4.6 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-21T08:47:50.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:49 smithi179 ceph-mon[31829]: pgmap v280: 129 pgs: 129 active+clean; 4.6 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-21T08:47:50.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:49 smithi086 ceph-mon[27939]: pgmap v280: 129 pgs: 129 active+clean; 4.6 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-21T08:47:52.167 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:51 smithi086 ceph-mon[27939]: pgmap v281: 129 pgs: 129 active+clean; 4.6 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:47:52.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:51 smithi088 ceph-mon[31259]: pgmap v281: 129 pgs: 129 active+clean; 4.6 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:47:52.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:51 smithi179 ceph-mon[31829]: pgmap v281: 129 pgs: 129 active+clean; 4.6 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:47:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.664+0000 7f6480d23640 1 -- 172.21.15.86:0/763121781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f647c101310 msgr2=0x7f647c1016f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.664+0000 7f6480d23640 1 --2- 172.21.15.86:0/763121781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f647c101310 0x7f647c1016f0 secure :-1 s=READY pgs=1681 cs=0 l=1 rev1=1 crypto rx=0x7f646c009900 tx=0x7f646c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:47:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.664+0000 7f6480d23640 1 -- 172.21.15.86:0/763121781 shutdown_connections 2024-04-21T08:47:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.664+0000 7f6480d23640 1 --2- 172.21.15.86:0/763121781 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f647c106190 0x7f647c10dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.664+0000 7f6480d23640 1 --2- 172.21.15.86:0/763121781 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f647c101cc0 0x7f647c105ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.664+0000 7f6480d23640 1 --2- 172.21.15.86:0/763121781 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f647c101310 0x7f647c1016f0 unknown :-1 s=CLOSED pgs=1681 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.664+0000 7f6480d23640 1 -- 172.21.15.86:0/763121781 >> 172.21.15.86:0/763121781 conn(0x7f647c077cc0 msgr2=0x7f647c0780d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:52.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.664+0000 7f6480d23640 1 -- 172.21.15.86:0/763121781 shutdown_connections 2024-04-21T08:47:52.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.664+0000 7f6480d23640 1 -- 172.21.15.86:0/763121781 wait complete. 2024-04-21T08:47:52.666 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.665+0000 7f6480d23640 1 Processor -- start 2024-04-21T08:47:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.665+0000 7f6480d23640 1 -- start start 2024-04-21T08:47:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.665+0000 7f6480d23640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f647c101310 0x7f647c1a06a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.665+0000 7f6480d23640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f647c106190 0x7f647c1a0be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.665+0000 7f6480d23640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f647c19a760 0x7f647c19abe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.665+0000 7f6480d23640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f647c111120 con 0x7f647c19a760 2024-04-21T08:47:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.665+0000 7f6480d23640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f647c110fa0 con 0x7f647c101310 2024-04-21T08:47:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.665+0000 7f6480d23640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f647c1112a0 con 0x7f647c106190 2024-04-21T08:47:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.665+0000 7f647ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f647c19a760 0x7f647c19abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.666+0000 7f647ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f647c19a760 0x7f647c19abe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38610/0 (socket says 172.21.15.86:38610) 2024-04-21T08:47:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.666+0000 7f647ad9d640 1 -- 172.21.15.86:0/920252528 learned_addr learned my addr 172.21.15.86:0/920252528 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.666+0000 7f647a59c640 1 --2- 172.21.15.86:0/920252528 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f647c101310 0x7f647c1a06a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.666+0000 7f6479d9b640 1 --2- 172.21.15.86:0/920252528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f647c106190 0x7f647c1a0be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:52.667 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.666+0000 7f647ad9d640 1 -- 172.21.15.86:0/920252528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f647c106190 msgr2=0x7f647c1a0be0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.666+0000 7f647ad9d640 1 --2- 172.21.15.86:0/920252528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f647c106190 0x7f647c1a0be0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.666+0000 7f647ad9d640 1 -- 172.21.15.86:0/920252528 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f647c101310 msgr2=0x7f647c1a06a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.666+0000 7f647ad9d640 1 --2- 172.21.15.86:0/920252528 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f647c101310 0x7f647c1a06a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.666+0000 7f647ad9d640 1 -- 172.21.15.86:0/920252528 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f647c19b360 con 0x7f647c19a760 2024-04-21T08:47:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.666+0000 7f647ad9d640 1 --2- 172.21.15.86:0/920252528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f647c19a760 0x7f647c19abe0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f647400cab0 tx=0x7f647400cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.666+0000 7f646b7fe640 1 -- 172.21.15.86:0/920252528 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647400adc0 con 0x7f647c19a760 2024-04-21T08:47:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.666+0000 7f646b7fe640 1 -- 172.21.15.86:0/920252528 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f647400f900 con 0x7f647c19a760 2024-04-21T08:47:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.666+0000 7f6480d23640 1 -- 172.21.15.86:0/920252528 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f647c1a7440 con 0x7f647c19a760 2024-04-21T08:47:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.666+0000 7f646b7fe640 1 -- 172.21.15.86:0/920252528 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64740189a0 con 0x7f647c19a760 2024-04-21T08:47:52.668 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.666+0000 7f6480d23640 1 -- 172.21.15.86:0/920252528 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f647c1a7840 con 0x7f647c19a760 2024-04-21T08:47:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.668+0000 7f646b7fe640 1 -- 172.21.15.86:0/920252528 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f647400f450 con 0x7f647c19a760 2024-04-21T08:47:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.669+0000 7f64839ed740 1 Processor -- start 2024-04-21T08:47:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.669+0000 7f646b7fe640 1 --2- 172.21.15.86:0/920252528 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f64640a3dc0 0x7f64640a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.669+0000 7f6479d9b640 1 --2- 172.21.15.86:0/920252528 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f64640a3dc0 0x7f64640a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.669+0000 7f646b7fe640 1 -- 172.21.15.86:0/920252528 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f64740d0d20 con 0x7f647c19a760 2024-04-21T08:47:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.669+0000 7f64839ed740 1 -- start start 2024-04-21T08:47:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.669+0000 7f64839ed740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b72e5bd8d0 0x55b72e5bc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.669+0000 7f64839ed740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72e5bcb60 0x55b72e5bcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:52.676 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.669+0000 7f64839ed740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b72e5c60e0 0x55b72e5c84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.669+0000 7f64839ed740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b72e5218d0 con 0x55b72e5bcb60 2024-04-21T08:47:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.669+0000 7f64839ed740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b72e4bee80 con 0x55b72e5bd8d0 2024-04-21T08:47:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.669+0000 7f64839ed740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b72e51d890 con 0x55b72e5c60e0 2024-04-21T08:47:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.670+0000 7f6479d9b640 1 --2- 172.21.15.86:0/920252528 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f64640a3dc0 0x7f64640a6280 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f647c19bfb0 tx=0x7f647000a2d0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.670+0000 7f6479d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72e5bcb60 0x55b72e5bcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.670+0000 7f6479d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72e5bcb60 0x55b72e5bcfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38624/0 (socket says 172.21.15.86:38624) 2024-04-21T08:47:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.670+0000 7f6479d9b640 1 -- 172.21.15.86:0/3762382427 learned_addr learned my addr 172.21.15.86:0/3762382427 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.670+0000 7f647a59c640 1 --2- 172.21.15.86:0/3762382427 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b72e5bd8d0 0x55b72e5bc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.670+0000 7f647ad9d640 1 --2- 172.21.15.86:0/3762382427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b72e5c60e0 0x55b72e5c84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.670+0000 7f6479d9b640 1 -- 172.21.15.86:0/3762382427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b72e5c60e0 msgr2=0x55b72e5c84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.670+0000 7f6479d9b640 1 --2- 172.21.15.86:0/3762382427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b72e5c60e0 0x55b72e5c84d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.670+0000 7f6479d9b640 1 -- 172.21.15.86:0/3762382427 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b72e5bd8d0 msgr2=0x55b72e5bc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.670+0000 7f6479d9b640 1 --2- 172.21.15.86:0/3762382427 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b72e5bd8d0 0x55b72e5bc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:52.677 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.670+0000 7f6479d9b640 1 -- 172.21.15.86:0/3762382427 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b72e4e0320 con 0x55b72e5bcb60 2024-04-21T08:47:52.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.670+0000 7f6479d9b640 1 --2- 172.21.15.86:0/3762382427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72e5bcb60 0x55b72e5bcfe0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f647000ff20 tx=0x7f6470012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c24e068e180f4a6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:52.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.670+0000 7f6468ff9640 1 -- 172.21.15.86:0/3762382427 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6470016070 con 0x55b72e5bcb60 2024-04-21T08:47:52.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.670+0000 7f6468ff9640 1 -- 172.21.15.86:0/3762382427 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6470025b20 con 0x55b72e5bcb60 2024-04-21T08:47:52.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.670+0000 7f6468ff9640 1 -- 172.21.15.86:0/3762382427 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f647001cb10 con 0x55b72e5bcb60 2024-04-21T08:47:52.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.671+0000 7f64839ed740 1 -- 172.21.15.86:0/3762382427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72e5bcb60 msgr2=0x55b72e5bcfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:52.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.671+0000 7f64839ed740 1 --2- 172.21.15.86:0/3762382427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72e5bcb60 0x55b72e5bcfe0 secure :-1 s=READY pgs=1683 cs=0 l=1 rev1=1 crypto rx=0x7f647000ff20 tx=0x7f6470012000 comp rx=0 tx=0).stop 2024-04-21T08:47:52.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.671+0000 7f64839ed740 1 -- 172.21.15.86:0/3762382427 shutdown_connections 2024-04-21T08:47:52.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.671+0000 7f64839ed740 1 --2- 172.21.15.86:0/3762382427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b72e5c60e0 0x55b72e5c84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:52.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.671+0000 7f64839ed740 1 --2- 172.21.15.86:0/3762382427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72e5bcb60 0x55b72e5bcfe0 unknown :-1 s=CLOSED pgs=1683 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:52.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.671+0000 7f64839ed740 1 --2- 172.21.15.86:0/3762382427 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b72e5bd8d0 0x55b72e5bc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:52.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.671+0000 7f64839ed740 1 -- 172.21.15.86:0/3762382427 >> 172.21.15.86:0/3762382427 conn(0x55b72e583c90 msgr2=0x55b72e5b7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:52.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.671+0000 7f64839ed740 1 -- 172.21.15.86:0/3762382427 shutdown_connections 2024-04-21T08:47:52.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.671+0000 7f64839ed740 1 -- 172.21.15.86:0/3762382427 wait complete. 2024-04-21T08:47:52.678 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.671+0000 7f64839ed740 1 -- start start 2024-04-21T08:47:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.672+0000 7f64839ed740 1 Processor -- start 2024-04-21T08:47:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.672+0000 7f64839ed740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b72e5bcb60 0x55b72e5c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.672+0000 7f64839ed740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b72e5bd8d0 0x55b72e5c2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.672+0000 7f64839ed740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72e5c60e0 0x55b72e5c33b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.672+0000 7f64839ed740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b72e5c8e10 con 0x55b72e5c60e0 2024-04-21T08:47:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.672+0000 7f64839ed740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b72e4a9eb0 con 0x55b72e5bcb60 2024-04-21T08:47:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.672+0000 7f64839ed740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b72e5c8f90 con 0x55b72e5bd8d0 2024-04-21T08:47:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.672+0000 7f64839ed740 10 client.0 _ll_register_callbacks cb 0x55b72e4d6b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.672+0000 7f647ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72e5c60e0 0x55b72e5c33b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.672+0000 7f647ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72e5c60e0 0x55b72e5c33b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38626/0 (socket says 172.21.15.86:38626) 2024-04-21T08:47:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.672+0000 7f647ad9d640 1 -- 172.21.15.86:0/1519237566 learned_addr learned my addr 172.21.15.86:0/1519237566 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.672+0000 7f647a59c640 1 --2- 172.21.15.86:0/1519237566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b72e5bcb60 0x55b72e5c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.672+0000 7f6479d9b640 1 --2- 172.21.15.86:0/1519237566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b72e5bd8d0 0x55b72e5c2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.672+0000 7f647ad9d640 1 -- 172.21.15.86:0/1519237566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b72e5bd8d0 msgr2=0x55b72e5c2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:52.679 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.672+0000 7f647ad9d640 1 --2- 172.21.15.86:0/1519237566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b72e5bd8d0 0x55b72e5c2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.672+0000 7f647ad9d640 1 -- 172.21.15.86:0/1519237566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b72e5bcb60 msgr2=0x55b72e5c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.672+0000 7f647ad9d640 1 --2- 172.21.15.86:0/1519237566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b72e5bcb60 0x55b72e5c5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.672+0000 7f647ad9d640 1 -- 172.21.15.86:0/1519237566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b72e234210 con 0x55b72e5c60e0 2024-04-21T08:47:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.672+0000 7f647ad9d640 1 --2- 172.21.15.86:0/1519237566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72e5c60e0 0x55b72e5c33b0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f647400a6e0 tx=0x7f647400f740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9841356be1321cf0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.672+0000 7f6461ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:47:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.672+0000 7f6461ffb640 1 -- 172.21.15.86:0/1519237566 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64740a0ca0 con 0x55b72e5c60e0 2024-04-21T08:47:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.673+0000 7f6461ffb640 1 -- 172.21.15.86:0/1519237566 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f6474098f60 con 0x55b72e5c60e0 2024-04-21T08:47:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.673+0000 7f64839ed740 1 -- 172.21.15.86:0/1519237566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6470025b20 con 0x55b72e5c60e0 2024-04-21T08:47:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.673+0000 7f6461ffb640 1 -- 172.21.15.86:0/1519237566 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f64740a5560 con 0x55b72e5c60e0 2024-04-21T08:47:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.673+0000 7f64839ed740 10 client.19266 fetch_fsmap learned FSMap version 29 2024-04-21T08:47:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.673+0000 7f64839ed740 1 -- 172.21.15.86:0/1519237566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b72e300620 con 0x55b72e5c60e0 2024-04-21T08:47:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.673+0000 7f6461ffb640 1 -- 172.21.15.86:0/1519237566 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6470025b20 con 0x55b72e5c60e0 2024-04-21T08:47:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.673+0000 7f6461ffb640 1 -- 172.21.15.86:0/1519237566 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6474020030 con 0x55b72e5c60e0 2024-04-21T08:47:52.680 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.673+0000 7f64839ed740 10 client.19266 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:47:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.673+0000 7f64839ed740 10 client.19266 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.673+0000 7f64839ed740 20 client.19266 populate_metadata read hostname 'smithi086' 2024-04-21T08:47:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.673+0000 7f64839ed740 1 --2- 172.21.15.86:0/1519237566 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b72e5d6100 0x55b72e5d84f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.673+0000 7f64839ed740 4 client.19266 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:47:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.673+0000 7f64839ed740 1 -- 172.21.15.86:0/1519237566 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b72e535880 con 0x55b72e5d6100 2024-04-21T08:47:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.673+0000 7f647a59c640 1 --2- 172.21.15.86:0/1519237566 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b72e5d6100 0x55b72e5d84f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.674+0000 7f6461ffb640 10 client.19266 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.674+0000 7f647a59c640 1 --2- 172.21.15.86:0/1519237566 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b72e5d6100 0x55b72e5d84f0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=23ae2aeaf04f0f2e server_cookie=f61a9532be0c0852 in_seq=0 out_seq=0 2024-04-21T08:47:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.674+0000 7f6461ffb640 1 -- 172.21.15.86:0/1519237566 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b72e535880 con 0x55b72e5d6100 2024-04-21T08:47:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.674+0000 7f6461ffb640 10 client.19266 handle_command_reply: tid=0 2024-04-21T08:47:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.674+0000 7f64839ed740 1 client.19266 shutdown 2024-04-21T08:47:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.675+0000 7f64839ed740 1 -- 172.21.15.86:0/1519237566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72e5c60e0 msgr2=0x55b72e5c33b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.675+0000 7f64839ed740 1 --2- 172.21.15.86:0/1519237566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72e5c60e0 0x55b72e5c33b0 secure :-1 s=READY pgs=1684 cs=0 l=1 rev1=1 crypto rx=0x7f647400a6e0 tx=0x7f647400f740 comp rx=0 tx=0).stop 2024-04-21T08:47:52.681 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.675+0000 7f64839ed740 1 -- 172.21.15.86:0/1519237566 shutdown_connections 2024-04-21T08:47:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.675+0000 7f64839ed740 1 --2- 172.21.15.86:0/1519237566 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b72e5d6100 0x55b72e5d84f0 crc :-1 s=READY pgs=267 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.675+0000 7f64839ed740 1 --2- 172.21.15.86:0/1519237566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72e5c60e0 0x55b72e5c33b0 unknown :-1 s=CLOSED pgs=1684 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.675+0000 7f64839ed740 1 --2- 172.21.15.86:0/1519237566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b72e5bd8d0 0x55b72e5c2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.675+0000 7f6461ffb640 0 client.19266 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.675+0000 7f64839ed740 1 --2- 172.21.15.86:0/1519237566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b72e5bcb60 0x55b72e5c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.675+0000 7f64839ed740 1 -- 172.21.15.86:0/1519237566 >> 172.21.15.86:0/1519237566 conn(0x55b72e583c90 msgr2=0x55b72e5b95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.675+0000 7f64839ed740 1 -- 172.21.15.86:0/1519237566 shutdown_connections 2024-04-21T08:47:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.675+0000 7f64839ed740 1 -- 172.21.15.86:0/1519237566 wait complete. 2024-04-21T08:47:52.682 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.675+0000 7f64839ed740 20 client.19266 trim_cache size 0 max 16384 2024-04-21T08:47:52.690 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.690+0000 7f64839ed740 1 Processor -- start 2024-04-21T08:47:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.690+0000 7f64839ed740 1 -- start start 2024-04-21T08:47:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.690+0000 7f64839ed740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b72e5bcb60 0x55b72e5d1d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.690+0000 7f64839ed740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72e5bd8d0 0x55b72e5c5540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.690+0000 7f64839ed740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b72e5c60e0 0x55b72e5c5a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.690+0000 7f64839ed740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f647c1b1520 con 0x55b72e5bd8d0 2024-04-21T08:47:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.690+0000 7f64839ed740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f647c1b3060 con 0x55b72e5bcb60 2024-04-21T08:47:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.690+0000 7f64839ed740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f647c1b1bf0 con 0x55b72e5c60e0 2024-04-21T08:47:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.690+0000 7f6479d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72e5bd8d0 0x55b72e5c5540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.690+0000 7f647a59c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b72e5bcb60 0x55b72e5d1d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.690+0000 7f6479d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72e5bd8d0 0x55b72e5c5540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38628/0 (socket says 172.21.15.86:38628) 2024-04-21T08:47:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.690+0000 7f6479d9b640 1 -- 172.21.15.86:0/459879819 learned_addr learned my addr 172.21.15.86:0/459879819 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.690+0000 7f647ad9d640 1 --2- 172.21.15.86:0/459879819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b72e5c60e0 0x55b72e5c5a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:52.691 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.691+0000 7f6479d9b640 1 -- 172.21.15.86:0/459879819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b72e5c60e0 msgr2=0x55b72e5c5a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.691+0000 7f6479d9b640 1 --2- 172.21.15.86:0/459879819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b72e5c60e0 0x55b72e5c5a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.691+0000 7f6479d9b640 1 -- 172.21.15.86:0/459879819 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b72e5bcb60 msgr2=0x55b72e5d1d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.691+0000 7f6479d9b640 1 --2- 172.21.15.86:0/459879819 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b72e5bcb60 0x55b72e5d1d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.691+0000 7f6479d9b640 1 -- 172.21.15.86:0/459879819 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b72e5b2f30 con 0x55b72e5bd8d0 2024-04-21T08:47:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.691+0000 7f6479d9b640 1 --2- 172.21.15.86:0/459879819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72e5bd8d0 0x55b72e5c5540 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f6470005d50 tx=0x7f647000a7b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a4d14d453f0d5b32 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.691+0000 7f6463fff640 1 -- 172.21.15.86:0/459879819 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6470034070 con 0x55b72e5bd8d0 2024-04-21T08:47:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.691+0000 7f6463fff640 1 -- 172.21.15.86:0/459879819 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6470004660 con 0x55b72e5bd8d0 2024-04-21T08:47:52.692 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.691+0000 7f6463fff640 1 -- 172.21.15.86:0/459879819 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6470016040 con 0x55b72e5bd8d0 2024-04-21T08:47:52.695 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.692+0000 7f64839ed740 1 -- 172.21.15.86:0/459879819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72e5bd8d0 msgr2=0x55b72e5c5540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.692+0000 7f64839ed740 1 --2- 172.21.15.86:0/459879819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72e5bd8d0 0x55b72e5c5540 secure :-1 s=READY pgs=1685 cs=0 l=1 rev1=1 crypto rx=0x7f6470005d50 tx=0x7f647000a7b0 comp rx=0 tx=0).stop 2024-04-21T08:47:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.692+0000 7f64839ed740 1 -- 172.21.15.86:0/459879819 shutdown_connections 2024-04-21T08:47:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.692+0000 7f64839ed740 1 --2- 172.21.15.86:0/459879819 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b72e5c60e0 0x55b72e5c5a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.692+0000 7f64839ed740 1 --2- 172.21.15.86:0/459879819 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72e5bd8d0 0x55b72e5c5540 unknown :-1 s=CLOSED pgs=1685 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.692+0000 7f64839ed740 1 --2- 172.21.15.86:0/459879819 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b72e5bcb60 0x55b72e5d1d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.692+0000 7f64839ed740 1 -- 172.21.15.86:0/459879819 >> 172.21.15.86:0/459879819 conn(0x55b72e583c90 msgr2=0x55b72e5be060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.692+0000 7f64839ed740 1 -- 172.21.15.86:0/459879819 shutdown_connections 2024-04-21T08:47:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.692+0000 7f64839ed740 1 -- 172.21.15.86:0/459879819 wait complete. 2024-04-21T08:47:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.692+0000 7f64839ed740 1 -- start start 2024-04-21T08:47:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.692+0000 7f64839ed740 1 Processor -- start 2024-04-21T08:47:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.692+0000 7f64839ed740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72e5bcb60 0x55b72e5ce560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.692+0000 7f64839ed740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b72e5bd8d0 0x55b72e5c98d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.692+0000 7f64839ed740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b72e5c60e0 0x55b72e5c9e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.692+0000 7f64839ed740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f647c1b2990 con 0x55b72e5bcb60 2024-04-21T08:47:52.696 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.692+0000 7f64839ed740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f647c1b22c0 con 0x55b72e5bd8d0 2024-04-21T08:47:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.692+0000 7f64839ed740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f647c1af520 con 0x55b72e5c60e0 2024-04-21T08:47:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.692+0000 7f64839ed740 10 client.0 _ll_register_callbacks cb 0x7f6474007650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:47:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.692+0000 7f647a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72e5bcb60 0x55b72e5ce560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.693+0000 7f647a59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72e5bcb60 0x55b72e5ce560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38638/0 (socket says 172.21.15.86:38638) 2024-04-21T08:47:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.693+0000 7f647a59c640 1 -- 172.21.15.86:0/2160455880 learned_addr learned my addr 172.21.15.86:0/2160455880 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:47:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.693+0000 7f6479d9b640 1 --2- 172.21.15.86:0/2160455880 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b72e5bd8d0 0x55b72e5c98d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.693+0000 7f647ad9d640 1 --2- 172.21.15.86:0/2160455880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b72e5c60e0 0x55b72e5c9e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.693+0000 7f647a59c640 1 -- 172.21.15.86:0/2160455880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b72e5c60e0 msgr2=0x55b72e5c9e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.693+0000 7f647a59c640 1 --2- 172.21.15.86:0/2160455880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b72e5c60e0 0x55b72e5c9e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.693+0000 7f647a59c640 1 -- 172.21.15.86:0/2160455880 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b72e5bd8d0 msgr2=0x55b72e5c98d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:47:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.693+0000 7f647a59c640 1 --2- 172.21.15.86:0/2160455880 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b72e5bd8d0 0x55b72e5c98d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.693+0000 7f647a59c640 1 -- 172.21.15.86:0/2160455880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b72e5ca5c0 con 0x55b72e5bcb60 2024-04-21T08:47:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.693+0000 7f647a59c640 1 --2- 172.21.15.86:0/2160455880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72e5bcb60 0x55b72e5ce560 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f646c02c170 tx=0x7f646c0045c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a5148b9d3b6a0d3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:47:52.697 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.693+0000 7f6461ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:47:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.693+0000 7f6461ffb640 1 -- 172.21.15.86:0/2160455880 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f646c04e070 con 0x55b72e5bcb60 2024-04-21T08:47:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.693+0000 7f6461ffb640 1 -- 172.21.15.86:0/2160455880 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f646c02c8e0 con 0x55b72e5bcb60 2024-04-21T08:47:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.693+0000 7f64839ed740 1 -- 172.21.15.86:0/2160455880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f6470004660 con 0x55b72e5bcb60 2024-04-21T08:47:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.693+0000 7f6461ffb640 1 -- 172.21.15.86:0/2160455880 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f646c059050 con 0x55b72e5bcb60 2024-04-21T08:47:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.693+0000 7f64839ed740 10 client.19272 fetch_fsmap learned FSMap version 29 2024-04-21T08:47:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.693+0000 7f64839ed740 1 -- 172.21.15.86:0/2160455880 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b72e63c510 con 0x55b72e5bcb60 2024-04-21T08:47:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.693+0000 7f6461ffb640 1 -- 172.21.15.86:0/2160455880 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6470004660 con 0x55b72e5bcb60 2024-04-21T08:47:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.694+0000 7f6461ffb640 1 -- 172.21.15.86:0/2160455880 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f646c005160 con 0x55b72e5bcb60 2024-04-21T08:47:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.694+0000 7f64839ed740 10 client.19272 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:47:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.694+0000 7f64839ed740 10 client.19272 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:47:52.698 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.694+0000 7f64839ed740 20 client.19272 populate_metadata read hostname 'smithi086' 2024-04-21T08:47:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.694+0000 7f64839ed740 1 --2- 172.21.15.86:0/2160455880 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b72e63cc30 0x55b72e63f050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:47:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.694+0000 7f64839ed740 4 client.19272 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:47:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.694+0000 7f64839ed740 1 -- 172.21.15.86:0/2160455880 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b72e4c13d0 con 0x55b72e63cc30 2024-04-21T08:47:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.694+0000 7f6479d9b640 1 --2- 172.21.15.86:0/2160455880 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b72e63cc30 0x55b72e63f050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:47:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.694+0000 7f6479d9b640 1 --2- 172.21.15.86:0/2160455880 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b72e63cc30 0x55b72e63f050 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6f21277296c6b8d5 server_cookie=8e23092203cf672b in_seq=0 out_seq=0 2024-04-21T08:47:52.699 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.694+0000 7f6461ffb640 10 client.19272 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.708+0000 7f6461ffb640 1 -- 172.21.15.86:0/2160455880 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b72e4c13d0 con 0x55b72e63cc30 2024-04-21T08:47:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.708+0000 7f6461ffb640 10 client.19272 handle_command_reply: tid=0 2024-04-21T08:47:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.708+0000 7f64839ed740 1 client.19272 shutdown 2024-04-21T08:47:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.708+0000 7f64839ed740 1 -- 172.21.15.86:0/2160455880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72e5bcb60 msgr2=0x55b72e5ce560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.708+0000 7f64839ed740 1 --2- 172.21.15.86:0/2160455880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72e5bcb60 0x55b72e5ce560 secure :-1 s=READY pgs=1686 cs=0 l=1 rev1=1 crypto rx=0x7f646c02c170 tx=0x7f646c0045c0 comp rx=0 tx=0).stop 2024-04-21T08:47:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.708+0000 7f64839ed740 1 -- 172.21.15.86:0/2160455880 shutdown_connections 2024-04-21T08:47:52.709 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.708+0000 7f64839ed740 1 --2- 172.21.15.86:0/2160455880 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b72e63cc30 0x55b72e63f050 crc :-1 s=READY pgs=268 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:52.710 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.708+0000 7f64839ed740 1 --2- 172.21.15.86:0/2160455880 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b72e5c60e0 0x55b72e5c9e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:52.710 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.708+0000 7f64839ed740 1 --2- 172.21.15.86:0/2160455880 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b72e5bd8d0 0x55b72e5c98d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:52.710 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.708+0000 7f6461ffb640 0 client.19272 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:47:52.710 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.708+0000 7f64839ed740 1 --2- 172.21.15.86:0/2160455880 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b72e5bcb60 0x55b72e5ce560 unknown :-1 s=CLOSED pgs=1686 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:52.710 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.708+0000 7f64839ed740 1 -- 172.21.15.86:0/2160455880 >> 172.21.15.86:0/2160455880 conn(0x55b72e583c90 msgr2=0x55b72e5c1670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:52.710 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.708+0000 7f64839ed740 1 -- 172.21.15.86:0/2160455880 shutdown_connections 2024-04-21T08:47:52.710 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.709+0000 7f64839ed740 1 -- 172.21.15.86:0/2160455880 wait complete. 2024-04-21T08:47:52.710 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.709+0000 7f64839ed740 20 client.19272 trim_cache size 0 max 16384 2024-04-21T08:47:52.710 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:47:52.710 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:47:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:47:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:47:52.711 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.709+0000 7f6480d23640 1 -- 172.21.15.86:0/920252528 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f64640a3dc0 msgr2=0x7f64640a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.709+0000 7f6480d23640 1 --2- 172.21.15.86:0/920252528 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f64640a3dc0 0x7f64640a6280 secure :-1 s=READY pgs=275 cs=0 l=1 rev1=1 crypto rx=0x7f647c19bfb0 tx=0x7f647000a2d0 comp rx=0 tx=0).stop 2024-04-21T08:47:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.709+0000 7f6480d23640 1 -- 172.21.15.86:0/920252528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f647c19a760 msgr2=0x7f647c19abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:47:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.709+0000 7f6480d23640 1 --2- 172.21.15.86:0/920252528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f647c19a760 0x7f647c19abe0 secure :-1 s=READY pgs=1682 cs=0 l=1 rev1=1 crypto rx=0x7f647400cab0 tx=0x7f647400cf70 comp rx=0 tx=0).stop 2024-04-21T08:47:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.709+0000 7f6480d23640 1 -- 172.21.15.86:0/920252528 shutdown_connections 2024-04-21T08:47:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.709+0000 7f6480d23640 1 --2- 172.21.15.86:0/920252528 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f64640a3dc0 0x7f64640a6280 unknown :-1 s=CLOSED pgs=275 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.709+0000 7f6480d23640 1 --2- 172.21.15.86:0/920252528 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f647c19a760 0x7f647c19abe0 unknown :-1 s=CLOSED pgs=1682 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.709+0000 7f6480d23640 1 --2- 172.21.15.86:0/920252528 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f647c106190 0x7f647c1a0be0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.709+0000 7f6480d23640 1 --2- 172.21.15.86:0/920252528 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f647c101310 0x7f647c1a06a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:47:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.709+0000 7f6480d23640 1 -- 172.21.15.86:0/920252528 >> 172.21.15.86:0/920252528 conn(0x7f647c077cc0 msgr2=0x7f647c107310 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:47:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.709+0000 7f6480d23640 1 -- 172.21.15.86:0/920252528 shutdown_connections 2024-04-21T08:47:52.712 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:47:52.709+0000 7f6480d23640 1 -- 172.21.15.86:0/920252528 wait complete. 2024-04-21T08:47:54.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:53 smithi088 ceph-mon[31259]: pgmap v282: 129 pgs: 129 active+clean; 4.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 65 KiB/s wr, 24 op/s 2024-04-21T08:47:54.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:53 smithi179 ceph-mon[31829]: pgmap v282: 129 pgs: 129 active+clean; 4.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 65 KiB/s wr, 24 op/s 2024-04-21T08:47:54.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:53 smithi086 ceph-mon[27939]: pgmap v282: 129 pgs: 129 active+clean; 4.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 65 KiB/s wr, 24 op/s 2024-04-21T08:47:56.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:55 smithi088 ceph-mon[31259]: pgmap v283: 129 pgs: 129 active+clean; 4.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:47:56.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:55 smithi179 ceph-mon[31829]: pgmap v283: 129 pgs: 129 active+clean; 4.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:47:56.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:55 smithi086 ceph-mon[27939]: pgmap v283: 129 pgs: 129 active+clean; 4.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:47:58.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:57 smithi088 ceph-mon[31259]: pgmap v284: 129 pgs: 129 active+clean; 4.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:47:58.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:57 smithi179 ceph-mon[31829]: pgmap v284: 129 pgs: 129 active+clean; 4.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:47:58.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:57 smithi086 ceph-mon[27939]: pgmap v284: 129 pgs: 129 active+clean; 4.8 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:47:59.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:58 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:47:59.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:58 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:47:59.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:58 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:48:00.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:59 smithi088 ceph-mon[31259]: pgmap v285: 129 pgs: 129 active+clean; 5.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-21T08:48:00.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:59 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:48:00.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:59 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:48:00.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:47:59 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:48:00.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:59 smithi179 ceph-mon[31829]: pgmap v285: 129 pgs: 129 active+clean; 5.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-21T08:48:00.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:59 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:48:00.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:59 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:48:00.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:47:59 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:48:00.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:59 smithi086 ceph-mon[27939]: pgmap v285: 129 pgs: 129 active+clean; 5.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-21T08:48:00.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:59 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:48:00.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:59 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:48:00.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:47:59 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:48:02.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:01 smithi088 ceph-mon[31259]: pgmap v286: 129 pgs: 129 active+clean; 5.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-21T08:48:02.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:01 smithi179 ceph-mon[31829]: pgmap v286: 129 pgs: 129 active+clean; 5.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-21T08:48:02.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:01 smithi086 ceph-mon[27939]: pgmap v286: 129 pgs: 129 active+clean; 5.0 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-21T08:48:03.020 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:48:03.020 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:48:03.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.133+0000 7fafaa1ef640 1 -- 172.21.15.86:0/755413708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafa4101090 msgr2=0x7fafa4101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.133+0000 7fafaa1ef640 1 --2- 172.21.15.86:0/755413708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafa4101090 0x7fafa4101470 secure :-1 s=READY pgs=1687 cs=0 l=1 rev1=1 crypto rx=0x7faf94009900 tx=0x7faf9402bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:03.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.133+0000 7fafaa1ef640 1 -- 172.21.15.86:0/755413708 shutdown_connections 2024-04-21T08:48:03.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.133+0000 7fafaa1ef640 1 --2- 172.21.15.86:0/755413708 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fafa4106160 0x7fafa410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.133+0000 7fafaa1ef640 1 --2- 172.21.15.86:0/755413708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fafa4101a40 0x7fafa4105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.133+0000 7fafaa1ef640 1 --2- 172.21.15.86:0/755413708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafa4101090 0x7fafa4101470 unknown :-1 s=CLOSED pgs=1687 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.133+0000 7fafaa1ef640 1 -- 172.21.15.86:0/755413708 >> 172.21.15.86:0/755413708 conn(0x7fafa40771d0 msgr2=0x7fafa40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:03.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.133+0000 7fafaa1ef640 1 -- 172.21.15.86:0/755413708 shutdown_connections 2024-04-21T08:48:03.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.133+0000 7fafaa1ef640 1 -- 172.21.15.86:0/755413708 wait complete. 2024-04-21T08:48:03.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.134+0000 7fafaa1ef640 1 Processor -- start 2024-04-21T08:48:03.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.134+0000 7fafaa1ef640 1 -- start start 2024-04-21T08:48:03.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.134+0000 7fafaa1ef640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fafa4101090 0x7fafa41995a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.134+0000 7fafaa1ef640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fafa4106160 0x7fafa4199ae0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.134+0000 7fafaa1ef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafa419a070 0x7fafa41a4620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.134+0000 7fafaa1ef640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fafa41110c0 con 0x7fafa419a070 2024-04-21T08:48:03.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.134+0000 7fafaa1ef640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fafa4110f40 con 0x7fafa4101090 2024-04-21T08:48:03.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.134+0000 7fafaa1ef640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fafa4111240 con 0x7fafa4106160 2024-04-21T08:48:03.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.134+0000 7fafa3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafa419a070 0x7fafa41a4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.135+0000 7fafa3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafa419a070 0x7fafa41a4620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48046/0 (socket says 172.21.15.86:48046) 2024-04-21T08:48:03.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.135+0000 7fafa3fff640 1 -- 172.21.15.86:0/4058325170 learned_addr learned my addr 172.21.15.86:0/4058325170 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:03.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.135+0000 7fafa37fe640 1 --2- 172.21.15.86:0/4058325170 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fafa4101090 0x7fafa41995a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.135+0000 7fafa2ffd640 1 --2- 172.21.15.86:0/4058325170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fafa4106160 0x7fafa4199ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.135+0000 7fafa3fff640 1 -- 172.21.15.86:0/4058325170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fafa4106160 msgr2=0x7fafa4199ae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.135+0000 7fafa3fff640 1 --2- 172.21.15.86:0/4058325170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fafa4106160 0x7fafa4199ae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.135+0000 7fafa3fff640 1 -- 172.21.15.86:0/4058325170 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fafa4101090 msgr2=0x7fafa41995a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.135+0000 7fafa3fff640 1 --2- 172.21.15.86:0/4058325170 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fafa4101090 0x7fafa41995a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.135+0000 7fafa3fff640 1 -- 172.21.15.86:0/4058325170 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fafa41a4df0 con 0x7fafa419a070 2024-04-21T08:48:03.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.135+0000 7fafa3fff640 1 --2- 172.21.15.86:0/4058325170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafa419a070 0x7fafa41a4620 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7faf9c009fd0 tx=0x7faf9c00b260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:03.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.135+0000 7fafa0ff9640 1 -- 172.21.15.86:0/4058325170 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c017070 con 0x7fafa419a070 2024-04-21T08:48:03.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.135+0000 7fafa0ff9640 1 -- 172.21.15.86:0/4058325170 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf9c003e20 con 0x7fafa419a070 2024-04-21T08:48:03.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.135+0000 7fafaa1ef640 1 -- 172.21.15.86:0/4058325170 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fafa41a50e0 con 0x7fafa419a070 2024-04-21T08:48:03.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.135+0000 7fafa0ff9640 1 -- 172.21.15.86:0/4058325170 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c010460 con 0x7fafa419a070 2024-04-21T08:48:03.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.135+0000 7fafaa1ef640 1 -- 172.21.15.86:0/4058325170 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fafa41a55e0 con 0x7fafa419a070 2024-04-21T08:48:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.137+0000 7fafa0ff9640 1 -- 172.21.15.86:0/4058325170 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faf9c003970 con 0x7fafa419a070 2024-04-21T08:48:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.138+0000 7fafacddc740 1 Processor -- start 2024-04-21T08:48:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.138+0000 7fafacddc740 1 -- start start 2024-04-21T08:48:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.138+0000 7fafa0ff9640 1 --2- 172.21.15.86:0/4058325170 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7faf8c0a3d20 0x7faf8c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.138+0000 7fafacddc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612cc235580 0x5612cc235960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.138+0000 7fafacddc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cc235ea0 0x5612cc23e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.138+0000 7fafacddc740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5612cc23ea80 0x5612cc240e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.138+0000 7fafacddc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5612cc19a1e0 con 0x5612cc235ea0 2024-04-21T08:48:03.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.138+0000 7fafacddc740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5612cc137390 con 0x5612cc23ea80 2024-04-21T08:48:03.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.138+0000 7fafacddc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5612cc1961a0 con 0x5612cc235580 2024-04-21T08:48:03.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.138+0000 7fafa2ffd640 1 --2- 172.21.15.86:0/4058325170 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7faf8c0a3d20 0x7faf8c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.138+0000 7fafa0ff9640 1 -- 172.21.15.86:0/4058325170 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7faf9c0d0910 con 0x7fafa419a070 2024-04-21T08:48:03.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.138+0000 7fafa2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cc235ea0 0x5612cc23e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.138+0000 7fafa2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cc235ea0 0x5612cc23e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48050/0 (socket says 172.21.15.86:48050) 2024-04-21T08:48:03.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.138+0000 7fafa2ffd640 1 -- 172.21.15.86:0/436846519 learned_addr learned my addr 172.21.15.86:0/436846519 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:03.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.138+0000 7fafa37fe640 1 --2- 172.21.15.86:0/436846519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612cc235580 0x5612cc235960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.138+0000 7fafa3fff640 1 --2- 172.21.15.86:0/436846519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5612cc23ea80 0x5612cc240e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.138+0000 7fafa2ffd640 1 -- 172.21.15.86:0/436846519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612cc235580 msgr2=0x5612cc235960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:03.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.138+0000 7fafa2ffd640 1 --2- 172.21.15.86:0/436846519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612cc235580 0x5612cc235960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.138+0000 7fafa2ffd640 1 -- 172.21.15.86:0/436846519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5612cc23ea80 msgr2=0x5612cc240e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:03.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.138+0000 7fafa2ffd640 1 --2- 172.21.15.86:0/436846519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5612cc23ea80 0x5612cc240e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.138+0000 7fafa2ffd640 1 -- 172.21.15.86:0/436846519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612cbf767d0 con 0x5612cc235ea0 2024-04-21T08:48:03.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.139+0000 7fafa2ffd640 1 --2- 172.21.15.86:0/4058325170 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7faf8c0a3d20 0x7faf8c0a61e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7faf98009fd0 tx=0x7faf98009b00 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:03.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.139+0000 7fafa2ffd640 1 --2- 172.21.15.86:0/436846519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cc235ea0 0x5612cc23e540 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7faf98005a70 tx=0x7faf9801c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c8829064c5c496d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:03.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.139+0000 7faf89ffb640 1 -- 172.21.15.86:0/436846519 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf98024070 con 0x5612cc235ea0 2024-04-21T08:48:03.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.139+0000 7faf89ffb640 1 -- 172.21.15.86:0/436846519 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf98010e30 con 0x5612cc235ea0 2024-04-21T08:48:03.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.139+0000 7faf89ffb640 1 -- 172.21.15.86:0/436846519 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf98020950 con 0x5612cc235ea0 2024-04-21T08:48:03.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.139+0000 7fafacddc740 1 -- 172.21.15.86:0/436846519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cc235ea0 msgr2=0x5612cc23e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.139+0000 7fafacddc740 1 --2- 172.21.15.86:0/436846519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cc235ea0 0x5612cc23e540 secure :-1 s=READY pgs=1689 cs=0 l=1 rev1=1 crypto rx=0x7faf98005a70 tx=0x7faf9801c000 comp rx=0 tx=0).stop 2024-04-21T08:48:03.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.139+0000 7fafacddc740 1 -- 172.21.15.86:0/436846519 shutdown_connections 2024-04-21T08:48:03.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.139+0000 7fafacddc740 1 --2- 172.21.15.86:0/436846519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5612cc23ea80 0x5612cc240e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.146 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.139+0000 7fafacddc740 1 --2- 172.21.15.86:0/436846519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cc235ea0 0x5612cc23e540 unknown :-1 s=CLOSED pgs=1689 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.139+0000 7fafacddc740 1 --2- 172.21.15.86:0/436846519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612cc235580 0x5612cc235960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.139+0000 7fafacddc740 1 -- 172.21.15.86:0/436846519 >> 172.21.15.86:0/436846519 conn(0x5612cc22fcb0 msgr2=0x5612cc230090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:03.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.140+0000 7fafacddc740 1 -- 172.21.15.86:0/436846519 shutdown_connections 2024-04-21T08:48:03.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.140+0000 7fafacddc740 1 -- 172.21.15.86:0/436846519 wait complete. 2024-04-21T08:48:03.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.140+0000 7fafacddc740 1 -- start start 2024-04-21T08:48:03.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.140+0000 7fafacddc740 1 Processor -- start 2024-04-21T08:48:03.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.140+0000 7fafacddc740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5612cc235580 0x5612cc23bb00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.140+0000 7fafacddc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cc235ea0 0x5612cc23c040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.140+0000 7fafacddc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612cc23ea80 0x5612cc23c580 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.140+0000 7fafacddc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5612cc241810 con 0x5612cc235ea0 2024-04-21T08:48:03.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.140+0000 7fafacddc740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5612cc1532f0 con 0x5612cc235580 2024-04-21T08:48:03.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.140+0000 7fafacddc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5612cc241990 con 0x5612cc23ea80 2024-04-21T08:48:03.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.140+0000 7fafacddc740 10 client.0 _ll_register_callbacks cb 0x5612cc160580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:03.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.140+0000 7fafa2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cc235ea0 0x5612cc23c040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.140+0000 7fafa2ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cc235ea0 0x5612cc23c040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48062/0 (socket says 172.21.15.86:48062) 2024-04-21T08:48:03.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.140+0000 7fafa2ffd640 1 -- 172.21.15.86:0/4162522983 learned_addr learned my addr 172.21.15.86:0/4162522983 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:03.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.141+0000 7fafa37fe640 1 --2- 172.21.15.86:0/4162522983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5612cc235580 0x5612cc23bb00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.141+0000 7fafa3fff640 1 --2- 172.21.15.86:0/4162522983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612cc23ea80 0x5612cc23c580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.141+0000 7fafa2ffd640 1 -- 172.21.15.86:0/4162522983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612cc23ea80 msgr2=0x5612cc23c580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:03.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.141+0000 7fafa2ffd640 1 --2- 172.21.15.86:0/4162522983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612cc23ea80 0x5612cc23c580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.141+0000 7fafa2ffd640 1 -- 172.21.15.86:0/4162522983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5612cc235580 msgr2=0x5612cc23bb00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:03.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.141+0000 7fafa2ffd640 1 --2- 172.21.15.86:0/4162522983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5612cc235580 0x5612cc23bb00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.141+0000 7fafa2ffd640 1 -- 172.21.15.86:0/4162522983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612cbeac3f0 con 0x5612cc235ea0 2024-04-21T08:48:03.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.141+0000 7fafa3fff640 1 --2- 172.21.15.86:0/4162522983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612cc23ea80 0x5612cc23c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:03.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.141+0000 7fafa2ffd640 1 --2- 172.21.15.86:0/4162522983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cc235ea0 0x5612cc23c040 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7faf98009250 tx=0x7faf9801fe90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c5075f6dba1ea48 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:03.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.141+0000 7faf82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:03.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.141+0000 7faf82ffd640 1 -- 172.21.15.86:0/4162522983 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9802f070 con 0x5612cc235ea0 2024-04-21T08:48:03.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.141+0000 7faf82ffd640 1 -- 172.21.15.86:0/4162522983 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7faf9801fbc0 con 0x5612cc235ea0 2024-04-21T08:48:03.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.141+0000 7fafacddc740 1 -- 172.21.15.86:0/4162522983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf98010e30 con 0x5612cc235ea0 2024-04-21T08:48:03.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.141+0000 7faf82ffd640 1 -- 172.21.15.86:0/4162522983 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faf9803b050 con 0x5612cc235ea0 2024-04-21T08:48:03.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.141+0000 7fafacddc740 10 client.19284 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:03.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.141+0000 7fafacddc740 1 -- 172.21.15.86:0/4162522983 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5612cbe6c0f0 con 0x5612cc235ea0 2024-04-21T08:48:03.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.141+0000 7faf82ffd640 1 -- 172.21.15.86:0/4162522983 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf98010e30 con 0x5612cc235ea0 2024-04-21T08:48:03.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.142+0000 7faf82ffd640 1 -- 172.21.15.86:0/4162522983 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7faf9801f050 con 0x5612cc235ea0 2024-04-21T08:48:03.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.142+0000 7fafacddc740 10 client.19284 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:03.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.142+0000 7fafacddc740 10 client.19284 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:03.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.142+0000 7fafacddc740 20 client.19284 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:03.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.142+0000 7fafacddc740 1 --2- 172.21.15.86:0/4162522983 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5612cc24e9a0 0x5612cc250d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.142+0000 7fafacddc740 4 client.19284 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:03.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.142+0000 7fafacddc740 1 -- 172.21.15.86:0/4162522983 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5612cc1a7c00 con 0x5612cc24e9a0 2024-04-21T08:48:03.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.142+0000 7fafa37fe640 1 --2- 172.21.15.86:0/4162522983 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5612cc24e9a0 0x5612cc250d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.142+0000 7fafa37fe640 1 --2- 172.21.15.86:0/4162522983 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5612cc24e9a0 0x5612cc250d90 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=95fcf0244a027d72 server_cookie=4b0cafab2f909c02 in_seq=0 out_seq=0 2024-04-21T08:48:03.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.142+0000 7faf82ffd640 10 client.19284 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:03.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.143+0000 7faf82ffd640 1 -- 172.21.15.86:0/4162522983 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5612cc1a7c00 con 0x5612cc24e9a0 2024-04-21T08:48:03.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.143+0000 7faf82ffd640 10 client.19284 handle_command_reply: tid=0 2024-04-21T08:48:03.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.143+0000 7fafacddc740 1 client.19284 shutdown 2024-04-21T08:48:03.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.143+0000 7fafacddc740 1 -- 172.21.15.86:0/4162522983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cc235ea0 msgr2=0x5612cc23c040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.143+0000 7fafacddc740 1 --2- 172.21.15.86:0/4162522983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cc235ea0 0x5612cc23c040 secure :-1 s=READY pgs=1690 cs=0 l=1 rev1=1 crypto rx=0x7faf98009250 tx=0x7faf9801fe90 comp rx=0 tx=0).stop 2024-04-21T08:48:03.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.143+0000 7fafacddc740 1 -- 172.21.15.86:0/4162522983 shutdown_connections 2024-04-21T08:48:03.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.143+0000 7fafacddc740 1 --2- 172.21.15.86:0/4162522983 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5612cc24e9a0 0x5612cc250d90 crc :-1 s=READY pgs=269 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.143+0000 7fafacddc740 1 --2- 172.21.15.86:0/4162522983 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612cc23ea80 0x5612cc23c580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.143+0000 7fafacddc740 1 --2- 172.21.15.86:0/4162522983 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cc235ea0 0x5612cc23c040 unknown :-1 s=CLOSED pgs=1690 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.143+0000 7fafacddc740 1 --2- 172.21.15.86:0/4162522983 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5612cc235580 0x5612cc23bb00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.143+0000 7faf82ffd640 0 client.19284 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:03.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.143+0000 7fafacddc740 1 -- 172.21.15.86:0/4162522983 >> 172.21.15.86:0/4162522983 conn(0x5612cc22fcb0 msgr2=0x5612cc231a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:03.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.143+0000 7fafacddc740 1 -- 172.21.15.86:0/4162522983 shutdown_connections 2024-04-21T08:48:03.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.143+0000 7fafacddc740 1 -- 172.21.15.86:0/4162522983 wait complete. 2024-04-21T08:48:03.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.143+0000 7fafacddc740 20 client.19284 trim_cache size 0 max 16384 2024-04-21T08:48:03.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.160+0000 7fafacddc740 1 Processor -- start 2024-04-21T08:48:03.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.160+0000 7fafacddc740 1 -- start start 2024-04-21T08:48:03.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.160+0000 7fafacddc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612cc235580 0x5612cc23e1c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.160+0000 7fafacddc740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5612cc235ea0 0x5612cc24a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.160+0000 7fafacddc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cc23ea80 0x5612cc256610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.161+0000 7fafacddc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fafa41af2d0 con 0x5612cc23ea80 2024-04-21T08:48:03.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.161+0000 7fafacddc740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fafa41b0e10 con 0x5612cc235ea0 2024-04-21T08:48:03.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.161+0000 7fafacddc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fafa41af9a0 con 0x5612cc235580 2024-04-21T08:48:03.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.161+0000 7fafa3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cc23ea80 0x5612cc256610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.161+0000 7fafa3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cc23ea80 0x5612cc256610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48074/0 (socket says 172.21.15.86:48074) 2024-04-21T08:48:03.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.161+0000 7fafa3fff640 1 -- 172.21.15.86:0/815755993 learned_addr learned my addr 172.21.15.86:0/815755993 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:03.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.161+0000 7fafa2ffd640 1 --2- 172.21.15.86:0/815755993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5612cc235ea0 0x5612cc24a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.161+0000 7fafa3fff640 1 -- 172.21.15.86:0/815755993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612cc235580 msgr2=0x5612cc23e1c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:03.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.161+0000 7fafa3fff640 1 --2- 172.21.15.86:0/815755993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612cc235580 0x5612cc23e1c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.161+0000 7fafa3fff640 1 -- 172.21.15.86:0/815755993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5612cc235ea0 msgr2=0x5612cc24a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:03.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.161+0000 7fafa3fff640 1 --2- 172.21.15.86:0/815755993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5612cc235ea0 0x5612cc24a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.161+0000 7fafa3fff640 1 -- 172.21.15.86:0/815755993 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5612cc251100 con 0x5612cc23ea80 2024-04-21T08:48:03.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.161+0000 7fafa2ffd640 1 --2- 172.21.15.86:0/815755993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5612cc235ea0 0x5612cc24a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:03.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.161+0000 7fafa3fff640 1 --2- 172.21.15.86:0/815755993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cc23ea80 0x5612cc256610 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7faf9c09c1e0 tx=0x7faf9c09be40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5a659cec20adc530 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:03.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.161+0000 7faf897fa640 1 -- 172.21.15.86:0/815755993 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c020040 con 0x5612cc23ea80 2024-04-21T08:48:03.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.161+0000 7faf897fa640 1 -- 172.21.15.86:0/815755993 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf9c09c4e0 con 0x5612cc23ea80 2024-04-21T08:48:03.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.161+0000 7faf897fa640 1 -- 172.21.15.86:0/815755993 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf9c088210 con 0x5612cc23ea80 2024-04-21T08:48:03.167 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.162+0000 7fafacddc740 1 -- 172.21.15.86:0/815755993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cc23ea80 msgr2=0x5612cc256610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.162+0000 7fafacddc740 1 --2- 172.21.15.86:0/815755993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cc23ea80 0x5612cc256610 secure :-1 s=READY pgs=1691 cs=0 l=1 rev1=1 crypto rx=0x7faf9c09c1e0 tx=0x7faf9c09be40 comp rx=0 tx=0).stop 2024-04-21T08:48:03.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.162+0000 7fafacddc740 1 -- 172.21.15.86:0/815755993 shutdown_connections 2024-04-21T08:48:03.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.162+0000 7fafacddc740 1 --2- 172.21.15.86:0/815755993 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cc23ea80 0x5612cc256610 unknown :-1 s=CLOSED pgs=1691 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.162+0000 7fafacddc740 1 --2- 172.21.15.86:0/815755993 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5612cc235ea0 0x5612cc24a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.162+0000 7fafacddc740 1 --2- 172.21.15.86:0/815755993 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612cc235580 0x5612cc23e1c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.162+0000 7fafacddc740 1 -- 172.21.15.86:0/815755993 >> 172.21.15.86:0/815755993 conn(0x5612cc22fcb0 msgr2=0x5612cc23a3d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:03.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.162+0000 7fafacddc740 1 -- 172.21.15.86:0/815755993 shutdown_connections 2024-04-21T08:48:03.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.162+0000 7fafacddc740 1 -- 172.21.15.86:0/815755993 wait complete. 2024-04-21T08:48:03.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.162+0000 7fafacddc740 1 -- start start 2024-04-21T08:48:03.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.162+0000 7fafacddc740 1 Processor -- start 2024-04-21T08:48:03.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.162+0000 7fafacddc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cc235580 0x5612cc246e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.162+0000 7fafacddc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612cc235ea0 0x5612cc238350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.162+0000 7fafacddc740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5612cc23ea80 0x5612cc238890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.162+0000 7fafacddc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fafa41b0740 con 0x5612cc235580 2024-04-21T08:48:03.168 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.162+0000 7fafacddc740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fafa41b0070 con 0x5612cc23ea80 2024-04-21T08:48:03.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.162+0000 7fafacddc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fafa41ad2d0 con 0x5612cc235ea0 2024-04-21T08:48:03.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.163+0000 7fafacddc740 10 client.0 _ll_register_callbacks cb 0x7faf9c00bea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:03.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.163+0000 7fafa37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cc235580 0x5612cc246e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.163+0000 7fafa37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cc235580 0x5612cc246e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48088/0 (socket says 172.21.15.86:48088) 2024-04-21T08:48:03.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.163+0000 7fafa37fe640 1 -- 172.21.15.86:0/311959931 learned_addr learned my addr 172.21.15.86:0/311959931 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:03.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.163+0000 7fafa2ffd640 1 --2- 172.21.15.86:0/311959931 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612cc235ea0 0x5612cc238350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.163+0000 7fafa3fff640 1 --2- 172.21.15.86:0/311959931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5612cc23ea80 0x5612cc238890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.163+0000 7fafa37fe640 1 -- 172.21.15.86:0/311959931 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612cc235ea0 msgr2=0x5612cc238350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:03.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.163+0000 7fafa37fe640 1 --2- 172.21.15.86:0/311959931 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612cc235ea0 0x5612cc238350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.163+0000 7fafa37fe640 1 -- 172.21.15.86:0/311959931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5612cc23ea80 msgr2=0x5612cc238890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:03.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.163+0000 7fafa37fe640 1 --2- 172.21.15.86:0/311959931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5612cc23ea80 0x5612cc238890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.163+0000 7fafa37fe640 1 -- 172.21.15.86:0/311959931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5612cc238e50 con 0x5612cc235580 2024-04-21T08:48:03.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.163+0000 7fafa3fff640 1 --2- 172.21.15.86:0/311959931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5612cc23ea80 0x5612cc238890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:03.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.163+0000 7fafa37fe640 1 --2- 172.21.15.86:0/311959931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cc235580 0x5612cc246e20 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7faf94009900 tx=0x7faf94004350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b3e50bfbb55a3090 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:03.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.163+0000 7faf82ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:03.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.163+0000 7faf82ffd640 1 -- 172.21.15.86:0/311959931 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf94049050 con 0x5612cc235580 2024-04-21T08:48:03.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.163+0000 7faf82ffd640 1 -- 172.21.15.86:0/311959931 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7faf9403a040 con 0x5612cc235580 2024-04-21T08:48:03.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.163+0000 7fafacddc740 1 -- 172.21.15.86:0/311959931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf9c09c4e0 con 0x5612cc235580 2024-04-21T08:48:03.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.163+0000 7faf82ffd640 1 -- 172.21.15.86:0/311959931 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faf94058050 con 0x5612cc235580 2024-04-21T08:48:03.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.163+0000 7fafacddc740 10 client.19290 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:03.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.163+0000 7fafacddc740 1 -- 172.21.15.86:0/311959931 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5612cc2bd400 con 0x5612cc235580 2024-04-21T08:48:03.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.164+0000 7faf82ffd640 1 -- 172.21.15.86:0/311959931 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf9c09c4e0 con 0x5612cc235580 2024-04-21T08:48:03.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.164+0000 7faf82ffd640 1 -- 172.21.15.86:0/311959931 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7faf9402cb10 con 0x5612cc235580 2024-04-21T08:48:03.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.164+0000 7fafacddc740 10 client.19290 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:03.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.164+0000 7fafacddc740 10 client.19290 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:03.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.164+0000 7fafacddc740 20 client.19290 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:03.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.164+0000 7fafacddc740 1 --2- 172.21.15.86:0/311959931 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5612cc2bdb80 0x5612cc2bffa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.164+0000 7fafacddc740 4 client.19290 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:48:03.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.164+0000 7fafacddc740 1 -- 172.21.15.86:0/311959931 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5612cc1a1a70 con 0x5612cc2bdb80 2024-04-21T08:48:03.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.164+0000 7fafa2ffd640 1 --2- 172.21.15.86:0/311959931 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5612cc2bdb80 0x5612cc2bffa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.164+0000 7faf82ffd640 10 client.19290 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:03.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.164+0000 7fafa2ffd640 1 --2- 172.21.15.86:0/311959931 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5612cc2bdb80 0x5612cc2bffa0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=976bf045012929cf server_cookie=ea1ec49a6b83a672 in_seq=0 out_seq=0 2024-04-21T08:48:03.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.164+0000 7faf82ffd640 1 -- 172.21.15.86:0/311959931 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5612cc1a1a70 con 0x5612cc2bdb80 2024-04-21T08:48:03.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.164+0000 7faf82ffd640 10 client.19290 handle_command_reply: tid=0 2024-04-21T08:48:03.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.164+0000 7fafacddc740 1 client.19290 shutdown 2024-04-21T08:48:03.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.165+0000 7fafacddc740 1 -- 172.21.15.86:0/311959931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cc235580 msgr2=0x5612cc246e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.165+0000 7fafacddc740 1 --2- 172.21.15.86:0/311959931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cc235580 0x5612cc246e20 secure :-1 s=READY pgs=1692 cs=0 l=1 rev1=1 crypto rx=0x7faf94009900 tx=0x7faf94004350 comp rx=0 tx=0).stop 2024-04-21T08:48:03.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.165+0000 7fafacddc740 1 -- 172.21.15.86:0/311959931 shutdown_connections 2024-04-21T08:48:03.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.165+0000 7fafacddc740 1 --2- 172.21.15.86:0/311959931 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5612cc2bdb80 0x5612cc2bffa0 crc :-1 s=READY pgs=270 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.165+0000 7fafacddc740 1 --2- 172.21.15.86:0/311959931 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5612cc23ea80 0x5612cc238890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.165+0000 7fafacddc740 1 --2- 172.21.15.86:0/311959931 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5612cc235ea0 0x5612cc238350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.165+0000 7faf82ffd640 0 client.19290 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:03.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.165+0000 7fafacddc740 1 --2- 172.21.15.86:0/311959931 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5612cc235580 0x5612cc246e20 unknown :-1 s=CLOSED pgs=1692 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.165+0000 7fafacddc740 1 -- 172.21.15.86:0/311959931 >> 172.21.15.86:0/311959931 conn(0x5612cc22fcb0 msgr2=0x5612cc2310e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:03.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.165+0000 7fafacddc740 1 -- 172.21.15.86:0/311959931 shutdown_connections 2024-04-21T08:48:03.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.165+0000 7fafacddc740 1 -- 172.21.15.86:0/311959931 wait complete. 2024-04-21T08:48:03.172 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.165+0000 7fafacddc740 20 client.19290 trim_cache size 0 max 16384 2024-04-21T08:48:03.173 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:48:03.173 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-21T08:48:03.173 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-21T08:48:03.173 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:48:03.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.165+0000 7fafaa1ef640 1 -- 172.21.15.86:0/4058325170 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7faf8c0a3d20 msgr2=0x7faf8c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.165+0000 7fafaa1ef640 1 --2- 172.21.15.86:0/4058325170 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7faf8c0a3d20 0x7faf8c0a61e0 secure :-1 s=READY pgs=276 cs=0 l=1 rev1=1 crypto rx=0x7faf98009fd0 tx=0x7faf98009b00 comp rx=0 tx=0).stop 2024-04-21T08:48:03.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.165+0000 7fafaa1ef640 1 -- 172.21.15.86:0/4058325170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafa419a070 msgr2=0x7fafa41a4620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.165+0000 7fafaa1ef640 1 --2- 172.21.15.86:0/4058325170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafa419a070 0x7fafa41a4620 secure :-1 s=READY pgs=1688 cs=0 l=1 rev1=1 crypto rx=0x7faf9c009fd0 tx=0x7faf9c00b260 comp rx=0 tx=0).stop 2024-04-21T08:48:03.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.165+0000 7fafaa1ef640 1 -- 172.21.15.86:0/4058325170 shutdown_connections 2024-04-21T08:48:03.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.165+0000 7fafaa1ef640 1 --2- 172.21.15.86:0/4058325170 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7faf8c0a3d20 0x7faf8c0a61e0 unknown :-1 s=CLOSED pgs=276 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.165+0000 7fafaa1ef640 1 --2- 172.21.15.86:0/4058325170 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fafa419a070 0x7fafa41a4620 unknown :-1 s=CLOSED pgs=1688 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.165+0000 7fafaa1ef640 1 --2- 172.21.15.86:0/4058325170 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fafa4106160 0x7fafa4199ae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.165+0000 7fafaa1ef640 1 --2- 172.21.15.86:0/4058325170 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fafa4101090 0x7fafa41995a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.165+0000 7fafaa1ef640 1 -- 172.21.15.86:0/4058325170 >> 172.21.15.86:0/4058325170 conn(0x7fafa40771d0 msgr2=0x7fafa40fee10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:03.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.166+0000 7fafaa1ef640 1 -- 172.21.15.86:0/4058325170 shutdown_connections 2024-04-21T08:48:03.175 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.166+0000 7fafaa1ef640 1 -- 172.21.15.86:0/4058325170 wait complete. 2024-04-21T08:48:03.176 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:48:03.176 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:48:03.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.275+0000 7fcf796c1640 1 -- 172.21.15.86:0/2088195449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf74106f10 msgr2=0x7fcf741072f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.275+0000 7fcf796c1640 1 --2- 172.21.15.86:0/2088195449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf74106f10 0x7fcf741072f0 secure :-1 s=READY pgs=1693 cs=0 l=1 rev1=1 crypto rx=0x7fcf64009900 tx=0x7fcf6402bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:03.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.275+0000 7fcf796c1640 1 -- 172.21.15.86:0/2088195449 shutdown_connections 2024-04-21T08:48:03.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.275+0000 7fcf796c1640 1 --2- 172.21.15.86:0/2088195449 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf74101ad0 0x7fcf7410dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.275+0000 7fcf796c1640 1 --2- 172.21.15.86:0/2088195449 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf74101130 0x7fcf74101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.275+0000 7fcf796c1640 1 --2- 172.21.15.86:0/2088195449 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf74106f10 0x7fcf741072f0 unknown :-1 s=CLOSED pgs=1693 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.275+0000 7fcf796c1640 1 -- 172.21.15.86:0/2088195449 >> 172.21.15.86:0/2088195449 conn(0x7fcf74077e30 msgr2=0x7fcf74078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:03.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.275+0000 7fcf796c1640 1 -- 172.21.15.86:0/2088195449 shutdown_connections 2024-04-21T08:48:03.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.275+0000 7fcf796c1640 1 -- 172.21.15.86:0/2088195449 wait complete. 2024-04-21T08:48:03.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.276+0000 7fcf796c1640 1 Processor -- start 2024-04-21T08:48:03.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.276+0000 7fcf796c1640 1 -- start start 2024-04-21T08:48:03.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.276+0000 7fcf796c1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf74101130 0x7fcf741995c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.276+0000 7fcf796c1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf74101ad0 0x7fcf74199b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.276+0000 7fcf796c1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf74106f10 0x7fcf741a4640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.276+0000 7fcf796c1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fcf741110c0 con 0x7fcf74106f10 2024-04-21T08:48:03.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.276+0000 7fcf796c1640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fcf74110f40 con 0x7fcf74101130 2024-04-21T08:48:03.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.276+0000 7fcf796c1640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fcf74111240 con 0x7fcf74101ad0 2024-04-21T08:48:03.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.276+0000 7fcf737fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf74106f10 0x7fcf741a4640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.276+0000 7fcf737fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf74106f10 0x7fcf741a4640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48112/0 (socket says 172.21.15.86:48112) 2024-04-21T08:48:03.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.276+0000 7fcf737fe640 1 -- 172.21.15.86:0/3824735196 learned_addr learned my addr 172.21.15.86:0/3824735196 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:03.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.276+0000 7fcf72ffd640 1 --2- 172.21.15.86:0/3824735196 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf74101130 0x7fcf741995c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.276+0000 7fcf6bfff640 1 --2- 172.21.15.86:0/3824735196 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf74101ad0 0x7fcf74199b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.276+0000 7fcf737fe640 1 -- 172.21.15.86:0/3824735196 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf74101ad0 msgr2=0x7fcf74199b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.276+0000 7fcf737fe640 1 --2- 172.21.15.86:0/3824735196 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf74101ad0 0x7fcf74199b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.276+0000 7fcf737fe640 1 -- 172.21.15.86:0/3824735196 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf74101130 msgr2=0x7fcf741995c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.276+0000 7fcf737fe640 1 --2- 172.21.15.86:0/3824735196 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf74101130 0x7fcf741995c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.276+0000 7fcf737fe640 1 -- 172.21.15.86:0/3824735196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fcf741a4b80 con 0x7fcf74106f10 2024-04-21T08:48:03.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.276+0000 7fcf6bfff640 1 --2- 172.21.15.86:0/3824735196 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf74101ad0 0x7fcf74199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:03.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.276+0000 7fcf737fe640 1 --2- 172.21.15.86:0/3824735196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf74106f10 0x7fcf741a4640 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c00cad0 tx=0x7fcf6c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:03.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.277+0000 7fcf70ff9640 1 -- 172.21.15.86:0/3824735196 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c00ae20 con 0x7fcf74106f10 2024-04-21T08:48:03.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.277+0000 7fcf70ff9640 1 -- 172.21.15.86:0/3824735196 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fcf6c005020 con 0x7fcf74106f10 2024-04-21T08:48:03.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.277+0000 7fcf796c1640 1 -- 172.21.15.86:0/3824735196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fcf741a4d50 con 0x7fcf74106f10 2024-04-21T08:48:03.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.277+0000 7fcf70ff9640 1 -- 172.21.15.86:0/3824735196 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fcf6c00f6b0 con 0x7fcf74106f10 2024-04-21T08:48:03.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.277+0000 7fcf796c1640 1 -- 172.21.15.86:0/3824735196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fcf741a5250 con 0x7fcf74106f10 2024-04-21T08:48:03.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.278+0000 7fcf796c1640 1 -- 172.21.15.86:0/3824735196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fcf74069230 con 0x7fcf74106f10 2024-04-21T08:48:03.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.279+0000 7fcf70ff9640 1 -- 172.21.15.86:0/3824735196 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fcf6c0051d0 con 0x7fcf74106f10 2024-04-21T08:48:03.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.279+0000 7fcf70ff9640 1 --2- 172.21.15.86:0/3824735196 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fcf600a3d20 0x7fcf600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.279+0000 7fcf72ffd640 1 --2- 172.21.15.86:0/3824735196 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fcf600a3d20 0x7fcf600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.279+0000 7fcf70ff9640 1 -- 172.21.15.86:0/3824735196 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fcf6c0d0c10 con 0x7fcf74106f10 2024-04-21T08:48:03.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.280+0000 7fcf72ffd640 1 --2- 172.21.15.86:0/3824735196 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fcf600a3d20 0x7fcf600a61e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcf64006010 tx=0x7fcf64005e40 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:03.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.284+0000 7fcf70ff9640 1 -- 172.21.15.86:0/3824735196 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fcf6c014030 con 0x7fcf74106f10 2024-04-21T08:48:03.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.498+0000 7fcf796c1640 1 -- 172.21.15.86:0/3824735196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fcf74109350 con 0x7fcf74106f10 2024-04-21T08:48:03.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.498+0000 7fcf70ff9640 1 -- 172.21.15.86:0/3824735196 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7fcf6c09cee0 con 0x7fcf74106f10 2024-04-21T08:48:03.498 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:48:03.498 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:38:45.202126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24398,24394,24455,24472]},"id":1}]} 2024-04-21T08:48:03.498 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-21T08:48:03.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.500+0000 7fcf796c1640 1 -- 172.21.15.86:0/3824735196 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fcf600a3d20 msgr2=0x7fcf600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.500+0000 7fcf796c1640 1 --2- 172.21.15.86:0/3824735196 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fcf600a3d20 0x7fcf600a61e0 secure :-1 s=READY pgs=277 cs=0 l=1 rev1=1 crypto rx=0x7fcf64006010 tx=0x7fcf64005e40 comp rx=0 tx=0).stop 2024-04-21T08:48:03.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.500+0000 7fcf796c1640 1 -- 172.21.15.86:0/3824735196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf74106f10 msgr2=0x7fcf741a4640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.500+0000 7fcf796c1640 1 --2- 172.21.15.86:0/3824735196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf74106f10 0x7fcf741a4640 secure :-1 s=READY pgs=1694 cs=0 l=1 rev1=1 crypto rx=0x7fcf6c00cad0 tx=0x7fcf6c00cf90 comp rx=0 tx=0).stop 2024-04-21T08:48:03.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.500+0000 7fcf796c1640 1 -- 172.21.15.86:0/3824735196 shutdown_connections 2024-04-21T08:48:03.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.500+0000 7fcf796c1640 1 --2- 172.21.15.86:0/3824735196 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fcf600a3d20 0x7fcf600a61e0 unknown :-1 s=CLOSED pgs=277 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.500+0000 7fcf796c1640 1 --2- 172.21.15.86:0/3824735196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fcf74106f10 0x7fcf741a4640 unknown :-1 s=CLOSED pgs=1694 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.500+0000 7fcf796c1640 1 --2- 172.21.15.86:0/3824735196 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fcf74101ad0 0x7fcf74199b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.500+0000 7fcf796c1640 1 --2- 172.21.15.86:0/3824735196 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fcf74101130 0x7fcf741995c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.500+0000 7fcf796c1640 1 -- 172.21.15.86:0/3824735196 >> 172.21.15.86:0/3824735196 conn(0x7fcf74077e30 msgr2=0x7fcf740ffee0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:03.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.501+0000 7fcf796c1640 1 -- 172.21.15.86:0/3824735196 shutdown_connections 2024-04-21T08:48:03.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.501+0000 7fcf796c1640 1 -- 172.21.15.86:0/3824735196 wait complete. 2024-04-21T08:48:03.511 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:48:03.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.613+0000 7f2aba3a4640 1 -- 172.21.15.86:0/643611457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ab41021e0 msgr2=0x7f2ab41025c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.613+0000 7f2aba3a4640 1 --2- 172.21.15.86:0/643611457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ab41021e0 0x7f2ab41025c0 secure :-1 s=READY pgs=1695 cs=0 l=1 rev1=1 crypto rx=0x7f2aa4009900 tx=0x7f2aa402bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:03.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.613+0000 7f2aba3a4640 1 -- 172.21.15.86:0/643611457 shutdown_connections 2024-04-21T08:48:03.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.613+0000 7f2aba3a4640 1 --2- 172.21.15.86:0/643611457 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ab41072b0 0x7f2ab410ee80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.613+0000 7f2aba3a4640 1 --2- 172.21.15.86:0/643611457 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ab4102b90 0x7f2ab4106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.613+0000 7f2aba3a4640 1 --2- 172.21.15.86:0/643611457 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ab41021e0 0x7f2ab41025c0 unknown :-1 s=CLOSED pgs=1695 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.613+0000 7f2aba3a4640 1 -- 172.21.15.86:0/643611457 >> 172.21.15.86:0/643611457 conn(0x7f2ab40fda60 msgr2=0x7f2ab40ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:03.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.613+0000 7f2aba3a4640 1 -- 172.21.15.86:0/643611457 shutdown_connections 2024-04-21T08:48:03.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.613+0000 7f2aba3a4640 1 -- 172.21.15.86:0/643611457 wait complete. 2024-04-21T08:48:03.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.613+0000 7f2aba3a4640 1 Processor -- start 2024-04-21T08:48:03.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.613+0000 7f2aba3a4640 1 -- start start 2024-04-21T08:48:03.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.614+0000 7f2aba3a4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ab41021e0 0x7f2ab419e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.614+0000 7f2aba3a4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ab41072b0 0x7f2ab419e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.614+0000 7f2aba3a4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ab4198600 0x7f2ab4198ab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.614+0000 7f2aba3a4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab4112240 con 0x7f2ab41072b0 2024-04-21T08:48:03.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.614+0000 7f2aba3a4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab41120c0 con 0x7f2ab4198600 2024-04-21T08:48:03.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.614+0000 7f2aba3a4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab41123c0 con 0x7f2ab41021e0 2024-04-21T08:48:03.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.614+0000 7f2ab37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ab41072b0 0x7f2ab419e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.614+0000 7f2ab37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ab41072b0 0x7f2ab419e940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48142/0 (socket says 172.21.15.86:48142) 2024-04-21T08:48:03.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.614+0000 7f2ab37fe640 1 -- 172.21.15.86:0/2951139094 learned_addr learned my addr 172.21.15.86:0/2951139094 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:03.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.614+0000 7f2ab3fff640 1 --2- 172.21.15.86:0/2951139094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ab41021e0 0x7f2ab419e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.614+0000 7f2ab8941640 1 --2- 172.21.15.86:0/2951139094 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ab4198600 0x7f2ab4198ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.614+0000 7f2ab37fe640 1 -- 172.21.15.86:0/2951139094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ab41021e0 msgr2=0x7f2ab419e400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.614+0000 7f2ab37fe640 1 --2- 172.21.15.86:0/2951139094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ab41021e0 0x7f2ab419e400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.614+0000 7f2ab37fe640 1 -- 172.21.15.86:0/2951139094 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ab4198600 msgr2=0x7f2ab4198ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.614+0000 7f2ab37fe640 1 --2- 172.21.15.86:0/2951139094 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ab4198600 0x7f2ab4198ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.614+0000 7f2ab37fe640 1 -- 172.21.15.86:0/2951139094 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2ab4199120 con 0x7f2ab41072b0 2024-04-21T08:48:03.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.614+0000 7f2ab8941640 1 --2- 172.21.15.86:0/2951139094 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ab4198600 0x7f2ab4198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:03.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.614+0000 7f2ab37fe640 1 --2- 172.21.15.86:0/2951139094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ab41072b0 0x7f2ab419e940 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f2aa800c970 tx=0x7f2aa800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:03.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.615+0000 7f2ab17fa640 1 -- 172.21.15.86:0/2951139094 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8014070 con 0x7f2ab41072b0 2024-04-21T08:48:03.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.615+0000 7f2aba3a4640 1 -- 172.21.15.86:0/2951139094 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2ab4199410 con 0x7f2ab41072b0 2024-04-21T08:48:03.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.615+0000 7f2ab17fa640 1 -- 172.21.15.86:0/2951139094 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2aa8004440 con 0x7f2ab41072b0 2024-04-21T08:48:03.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.615+0000 7f2ab17fa640 1 -- 172.21.15.86:0/2951139094 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa8003e50 con 0x7f2ab41072b0 2024-04-21T08:48:03.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.615+0000 7f2aba3a4640 1 -- 172.21.15.86:0/2951139094 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2ab41a5660 con 0x7f2ab41072b0 2024-04-21T08:48:03.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.617+0000 7f2ab17fa640 1 -- 172.21.15.86:0/2951139094 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2aa8017070 con 0x7f2ab41072b0 2024-04-21T08:48:03.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.617+0000 7f2abcf92740 1 Processor -- start 2024-04-21T08:48:03.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.617+0000 7f2abcf92740 1 -- start start 2024-04-21T08:48:03.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.617+0000 7f2abcf92740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56364f46d4d0 0x56364f46d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.617+0000 7f2abcf92740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56364f46de80 0x56364f476430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.617+0000 7f2abcf92740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56364f476a00 0x56364f478e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.617+0000 7f2abcf92740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56364f3d21f0 con 0x56364f476a00 2024-04-21T08:48:03.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.617+0000 7f2abcf92740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x56364f36f4b0 con 0x56364f46de80 2024-04-21T08:48:03.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.617+0000 7f2abcf92740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56364f3ce1b0 con 0x56364f46d4d0 2024-04-21T08:48:03.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.618+0000 7f2ab17fa640 1 --2- 172.21.15.86:0/2951139094 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2a9c0a3ec0 0x7f2a9c0a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.618+0000 7f2ab37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56364f476a00 0x56364f478e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.618+0000 7f2ab37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56364f476a00 0x56364f478e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48158/0 (socket says 172.21.15.86:48158) 2024-04-21T08:48:03.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.618+0000 7f2ab37fe640 1 -- 172.21.15.86:0/4159333028 learned_addr learned my addr 172.21.15.86:0/4159333028 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:03.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.618+0000 7f2ab8941640 1 --2- 172.21.15.86:0/2951139094 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2a9c0a3ec0 0x7f2a9c0a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.618+0000 7f2ab3fff640 1 --2- 172.21.15.86:0/4159333028 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56364f46de80 0x56364f476430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.618+0000 7f2ab8941640 1 --2- 172.21.15.86:0/4159333028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56364f46d4d0 0x56364f46d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.618+0000 7f2ab17fa640 1 -- 172.21.15.86:0/2951139094 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f2aa80d0780 con 0x7f2ab41072b0 2024-04-21T08:48:03.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.618+0000 7f2ab37fe640 1 -- 172.21.15.86:0/4159333028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56364f46d4d0 msgr2=0x56364f46d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:03.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.618+0000 7f2ab37fe640 1 --2- 172.21.15.86:0/4159333028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56364f46d4d0 0x56364f46d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.618+0000 7f2ab37fe640 1 -- 172.21.15.86:0/4159333028 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56364f46de80 msgr2=0x56364f476430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:03.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.618+0000 7f2ab37fe640 1 --2- 172.21.15.86:0/4159333028 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56364f46de80 0x56364f476430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.618+0000 7f2ab37fe640 1 -- 172.21.15.86:0/4159333028 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56364f3611b0 con 0x56364f476a00 2024-04-21T08:48:03.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.618+0000 7f2ab8941640 1 --2- 172.21.15.86:0/2951139094 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2a9c0a3ec0 0x7f2a9c0a6380 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2aac00afd0 tx=0x7f2aac012a10 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:03.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.618+0000 7f2ab37fe640 1 --2- 172.21.15.86:0/4159333028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56364f476a00 0x56364f478e50 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f2aa80cda70 tx=0x7f2aa80038b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3cd7913ae5a1edc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:03.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.618+0000 7f2a9a7fc640 1 -- 172.21.15.86:0/4159333028 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa809c350 con 0x56364f476a00 2024-04-21T08:48:03.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.618+0000 7f2a9a7fc640 1 -- 172.21.15.86:0/4159333028 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2aa80a0700 con 0x56364f476a00 2024-04-21T08:48:03.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.618+0000 7f2a9a7fc640 1 -- 172.21.15.86:0/4159333028 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa80989c0 con 0x56364f476a00 2024-04-21T08:48:03.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.619+0000 7f2abcf92740 1 -- 172.21.15.86:0/4159333028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56364f476a00 msgr2=0x56364f478e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.619+0000 7f2abcf92740 1 --2- 172.21.15.86:0/4159333028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56364f476a00 0x56364f478e50 secure :-1 s=READY pgs=1697 cs=0 l=1 rev1=1 crypto rx=0x7f2aa80cda70 tx=0x7f2aa80038b0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.619+0000 7f2abcf92740 1 -- 172.21.15.86:0/4159333028 shutdown_connections 2024-04-21T08:48:03.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.619+0000 7f2abcf92740 1 --2- 172.21.15.86:0/4159333028 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56364f476a00 0x56364f478e50 unknown :-1 s=CLOSED pgs=1697 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.619+0000 7f2abcf92740 1 --2- 172.21.15.86:0/4159333028 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56364f46de80 0x56364f476430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.619+0000 7f2abcf92740 1 --2- 172.21.15.86:0/4159333028 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56364f46d4d0 0x56364f46d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.619+0000 7f2abcf92740 1 -- 172.21.15.86:0/4159333028 >> 172.21.15.86:0/4159333028 conn(0x56364f467ba0 msgr2=0x56364f467f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:03.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.619+0000 7f2abcf92740 1 -- 172.21.15.86:0/4159333028 shutdown_connections 2024-04-21T08:48:03.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.619+0000 7f2abcf92740 1 -- 172.21.15.86:0/4159333028 wait complete. 2024-04-21T08:48:03.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.619+0000 7f2abcf92740 1 -- start start 2024-04-21T08:48:03.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.620+0000 7f2abcf92740 1 Processor -- start 2024-04-21T08:48:03.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.620+0000 7f2abcf92740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56364f46d4d0 0x56364f476340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.620+0000 7f2abcf92740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56364f46de80 0x56364f4736d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.620+0000 7f2abcf92740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56364f476a00 0x56364f473ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.620+0000 7f2abcf92740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56364f479970 con 0x56364f46de80 2024-04-21T08:48:03.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.620+0000 7f2abcf92740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x56364f38b500 con 0x56364f476a00 2024-04-21T08:48:03.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.620+0000 7f2abcf92740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56364f479af0 con 0x56364f46d4d0 2024-04-21T08:48:03.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.620+0000 7f2abcf92740 10 client.0 _ll_register_callbacks cb 0x56364f359b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:03.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.620+0000 7f2ab37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56364f46de80 0x56364f4736d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.620+0000 7f2ab37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56364f46de80 0x56364f4736d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48170/0 (socket says 172.21.15.86:48170) 2024-04-21T08:48:03.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.620+0000 7f2ab37fe640 1 -- 172.21.15.86:0/868234795 learned_addr learned my addr 172.21.15.86:0/868234795 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:03.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.620+0000 7f2ab3fff640 1 --2- 172.21.15.86:0/868234795 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56364f46d4d0 0x56364f476340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.620+0000 7f2ab8941640 1 --2- 172.21.15.86:0/868234795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56364f476a00 0x56364f473ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.620+0000 7f2ab37fe640 1 -- 172.21.15.86:0/868234795 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56364f46d4d0 msgr2=0x56364f476340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:03.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.620+0000 7f2ab37fe640 1 --2- 172.21.15.86:0/868234795 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56364f46d4d0 0x56364f476340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.620+0000 7f2ab37fe640 1 -- 172.21.15.86:0/868234795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56364f476a00 msgr2=0x56364f473ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:03.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.620+0000 7f2ab37fe640 1 --2- 172.21.15.86:0/868234795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56364f476a00 0x56364f473ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.620+0000 7f2ab37fe640 1 -- 172.21.15.86:0/868234795 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56364f0e43f0 con 0x56364f46de80 2024-04-21T08:48:03.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.620+0000 7f2ab37fe640 1 --2- 172.21.15.86:0/868234795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56364f46de80 0x56364f4736d0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8098d20 tx=0x7f2aa8098bf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9ed0f8b957b14092 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:03.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.620+0000 7f2a937fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:03.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.620+0000 7f2a937fe640 1 -- 172.21.15.86:0/868234795 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa80985f0 con 0x56364f46de80 2024-04-21T08:48:03.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.621+0000 7f2a937fe640 1 -- 172.21.15.86:0/868234795 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f2aa80d0390 con 0x56364f46de80 2024-04-21T08:48:03.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.621+0000 7f2abcf92740 1 -- 172.21.15.86:0/868234795 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2aa80a0700 con 0x56364f46de80 2024-04-21T08:48:03.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.621+0000 7f2a937fe640 1 -- 172.21.15.86:0/868234795 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2aa80d05d0 con 0x56364f46de80 2024-04-21T08:48:03.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.621+0000 7f2abcf92740 10 client.19308 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:03.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.621+0000 7f2abcf92740 1 -- 172.21.15.86:0/868234795 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56364f1ae7d0 con 0x56364f46de80 2024-04-21T08:48:03.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.621+0000 7f2a937fe640 1 -- 172.21.15.86:0/868234795 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2aa80a0700 con 0x56364f46de80 2024-04-21T08:48:03.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.621+0000 7f2a937fe640 1 -- 172.21.15.86:0/868234795 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2aa80d0070 con 0x56364f46de80 2024-04-21T08:48:03.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.621+0000 7f2abcf92740 10 client.19308 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:03.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.621+0000 7f2abcf92740 10 client.19308 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:48:03.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.621+0000 7f2abcf92740 20 client.19308 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:03.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.621+0000 7f2abcf92740 1 --2- 172.21.15.86:0/868234795 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x56364f486ac0 0x56364f488eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.621+0000 7f2abcf92740 4 client.19308 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:03.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.621+0000 7f2abcf92740 1 -- 172.21.15.86:0/868234795 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56364f371930 con 0x56364f486ac0 2024-04-21T08:48:03.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.621+0000 7f2ab3fff640 1 --2- 172.21.15.86:0/868234795 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x56364f486ac0 0x56364f488eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.622+0000 7f2ab3fff640 1 --2- 172.21.15.86:0/868234795 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x56364f486ac0 0x56364f488eb0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=8cc708fb8b23a725 server_cookie=6baa4663413eb84c in_seq=0 out_seq=0 2024-04-21T08:48:03.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.622+0000 7f2a937fe640 10 client.19308 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:48:03.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.622+0000 7f2a937fe640 1 -- 172.21.15.86:0/868234795 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56364f371930 con 0x56364f486ac0 2024-04-21T08:48:03.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.622+0000 7f2a937fe640 10 client.19308 handle_command_reply: tid=0 2024-04-21T08:48:03.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.623+0000 7f2abcf92740 1 client.19308 shutdown 2024-04-21T08:48:03.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.623+0000 7f2abcf92740 1 -- 172.21.15.86:0/868234795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56364f46de80 msgr2=0x56364f4736d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.623+0000 7f2abcf92740 1 --2- 172.21.15.86:0/868234795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56364f46de80 0x56364f4736d0 secure :-1 s=READY pgs=1698 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8098d20 tx=0x7f2aa8098bf0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.623+0000 7f2abcf92740 1 -- 172.21.15.86:0/868234795 shutdown_connections 2024-04-21T08:48:03.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.623+0000 7f2abcf92740 1 --2- 172.21.15.86:0/868234795 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x56364f486ac0 0x56364f488eb0 crc :-1 s=READY pgs=66 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.623+0000 7f2abcf92740 1 --2- 172.21.15.86:0/868234795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56364f476a00 0x56364f473ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.623+0000 7f2a937fe640 0 client.19308 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:48:03.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.623+0000 7f2abcf92740 1 --2- 172.21.15.86:0/868234795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56364f46de80 0x56364f4736d0 unknown :-1 s=CLOSED pgs=1698 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.623+0000 7f2abcf92740 1 --2- 172.21.15.86:0/868234795 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56364f46d4d0 0x56364f476340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.623+0000 7f2abcf92740 1 -- 172.21.15.86:0/868234795 >> 172.21.15.86:0/868234795 conn(0x56364f467ba0 msgr2=0x56364f4698f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:03.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.623+0000 7f2abcf92740 1 -- 172.21.15.86:0/868234795 shutdown_connections 2024-04-21T08:48:03.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.623+0000 7f2abcf92740 1 -- 172.21.15.86:0/868234795 wait complete. 2024-04-21T08:48:03.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.623+0000 7f2abcf92740 20 client.19308 trim_cache size 0 max 16384 2024-04-21T08:48:03.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.636+0000 7f2abcf92740 1 Processor -- start 2024-04-21T08:48:03.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.636+0000 7f2abcf92740 1 -- start start 2024-04-21T08:48:03.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.636+0000 7f2abcf92740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56364f46d4d0 0x56364f475e00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.636+0000 7f2abcf92740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56364f46de80 0x56364f476340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.636+0000 7f2abcf92740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56364f476a00 0x56364f48bed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.636+0000 7f2abcf92740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab41af480 con 0x56364f46de80 2024-04-21T08:48:03.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.636+0000 7f2abcf92740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab41b0fc0 con 0x56364f46d4d0 2024-04-21T08:48:03.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.636+0000 7f2abcf92740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab41afb50 con 0x56364f476a00 2024-04-21T08:48:03.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.636+0000 7f2ab37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56364f46de80 0x56364f476340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.636+0000 7f2ab37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56364f46de80 0x56364f476340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48178/0 (socket says 172.21.15.86:48178) 2024-04-21T08:48:03.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.636+0000 7f2ab37fe640 1 -- 172.21.15.86:0/496706499 learned_addr learned my addr 172.21.15.86:0/496706499 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:03.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.636+0000 7f2ab3fff640 1 --2- 172.21.15.86:0/496706499 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56364f46d4d0 0x56364f475e00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.636+0000 7f2ab37fe640 1 -- 172.21.15.86:0/496706499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56364f476a00 msgr2=0x56364f48bed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:03.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.636+0000 7f2ab37fe640 1 --2- 172.21.15.86:0/496706499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56364f476a00 0x56364f48bed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.636+0000 7f2ab37fe640 1 -- 172.21.15.86:0/496706499 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56364f46d4d0 msgr2=0x56364f475e00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:03.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.636+0000 7f2ab37fe640 1 --2- 172.21.15.86:0/496706499 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56364f46d4d0 0x56364f475e00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.636+0000 7f2ab37fe640 1 -- 172.21.15.86:0/496706499 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56364f46dc20 con 0x56364f46de80 2024-04-21T08:48:03.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.637+0000 7f2ab37fe640 1 --2- 172.21.15.86:0/496706499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56364f46de80 0x56364f476340 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f2aa80d0b30 tx=0x7f2aa809ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea477d4a4dcad7d6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:03.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.637+0000 7f2a99ffb640 1 -- 172.21.15.86:0/496706499 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa80a0700 con 0x56364f46de80 2024-04-21T08:48:03.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.637+0000 7f2a99ffb640 1 -- 172.21.15.86:0/496706499 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2aa80a4fc0 con 0x56364f46de80 2024-04-21T08:48:03.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.637+0000 7f2a99ffb640 1 -- 172.21.15.86:0/496706499 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa80d0360 con 0x56364f46de80 2024-04-21T08:48:03.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.637+0000 7f2abcf92740 1 -- 172.21.15.86:0/496706499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56364f46de80 msgr2=0x56364f476340 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.637+0000 7f2abcf92740 1 --2- 172.21.15.86:0/496706499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56364f46de80 0x56364f476340 secure :-1 s=READY pgs=1699 cs=0 l=1 rev1=1 crypto rx=0x7f2aa80d0b30 tx=0x7f2aa809ca70 comp rx=0 tx=0).stop 2024-04-21T08:48:03.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.637+0000 7f2abcf92740 1 -- 172.21.15.86:0/496706499 shutdown_connections 2024-04-21T08:48:03.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.637+0000 7f2abcf92740 1 --2- 172.21.15.86:0/496706499 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56364f476a00 0x56364f48bed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.637+0000 7f2abcf92740 1 --2- 172.21.15.86:0/496706499 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56364f46de80 0x56364f476340 unknown :-1 s=CLOSED pgs=1699 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.637+0000 7f2abcf92740 1 --2- 172.21.15.86:0/496706499 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56364f46d4d0 0x56364f475e00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.637+0000 7f2abcf92740 1 -- 172.21.15.86:0/496706499 >> 172.21.15.86:0/496706499 conn(0x56364f467ba0 msgr2=0x56364f466c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:03.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.637+0000 7f2abcf92740 1 -- 172.21.15.86:0/496706499 shutdown_connections 2024-04-21T08:48:03.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.637+0000 7f2abcf92740 1 -- 172.21.15.86:0/496706499 wait complete. 2024-04-21T08:48:03.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.638+0000 7f2abcf92740 1 -- start start 2024-04-21T08:48:03.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.638+0000 7f2abcf92740 1 Processor -- start 2024-04-21T08:48:03.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.638+0000 7f2abcf92740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56364f46d4d0 0x56364f48bf20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.638+0000 7f2abcf92740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56364f46de80 0x56364f48c460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.638+0000 7f2abcf92740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56364f476a00 0x56364f4894b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.638+0000 7f2abcf92740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab41b08f0 con 0x56364f46de80 2024-04-21T08:48:03.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.638+0000 7f2abcf92740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab41b0220 con 0x56364f46d4d0 2024-04-21T08:48:03.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.638+0000 7f2abcf92740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2ab41ad480 con 0x56364f476a00 2024-04-21T08:48:03.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.638+0000 7f2abcf92740 10 client.0 _ll_register_callbacks cb 0x7f2aa809cb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:03.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.638+0000 7f2ab37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56364f46de80 0x56364f48c460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.638+0000 7f2ab37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56364f46de80 0x56364f48c460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48182/0 (socket says 172.21.15.86:48182) 2024-04-21T08:48:03.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.638+0000 7f2ab37fe640 1 -- 172.21.15.86:0/982810129 learned_addr learned my addr 172.21.15.86:0/982810129 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:03.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.638+0000 7f2ab3fff640 1 --2- 172.21.15.86:0/982810129 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56364f46d4d0 0x56364f48bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.638+0000 7f2ab8941640 1 --2- 172.21.15.86:0/982810129 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56364f476a00 0x56364f4894b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.638+0000 7f2ab37fe640 1 -- 172.21.15.86:0/982810129 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56364f476a00 msgr2=0x56364f4894b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:03.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.638+0000 7f2ab37fe640 1 --2- 172.21.15.86:0/982810129 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56364f476a00 0x56364f4894b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.638+0000 7f2ab37fe640 1 -- 172.21.15.86:0/982810129 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56364f46d4d0 msgr2=0x56364f48bf20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:03.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.638+0000 7f2ab37fe640 1 --2- 172.21.15.86:0/982810129 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56364f46d4d0 0x56364f48bf20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.638+0000 7f2ab37fe640 1 -- 172.21.15.86:0/982810129 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56364f489cb0 con 0x56364f46de80 2024-04-21T08:48:03.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.638+0000 7f2ab8941640 1 --2- 172.21.15.86:0/982810129 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56364f476a00 0x56364f4894b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:03.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.639+0000 7f2ab37fe640 1 --2- 172.21.15.86:0/982810129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56364f46de80 0x56364f48c460 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8098d20 tx=0x7f2aa8003820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdb8c847e5ea9334 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:03.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.639+0000 7f2a937fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:03.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.639+0000 7f2a937fe640 1 -- 172.21.15.86:0/982810129 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2aa809c350 con 0x56364f46de80 2024-04-21T08:48:03.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.639+0000 7f2a937fe640 1 -- 172.21.15.86:0/982810129 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f2aa80d0570 con 0x56364f46de80 2024-04-21T08:48:03.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.639+0000 7f2abcf92740 1 -- 172.21.15.86:0/982810129 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2aa80a4fc0 con 0x56364f46de80 2024-04-21T08:48:03.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.639+0000 7f2a937fe640 1 -- 172.21.15.86:0/982810129 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2aa807c050 con 0x56364f46de80 2024-04-21T08:48:03.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.639+0000 7f2abcf92740 10 client.19314 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:03.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.639+0000 7f2abcf92740 1 -- 172.21.15.86:0/982810129 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56364f489cb0 con 0x56364f46de80 2024-04-21T08:48:03.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.639+0000 7f2a937fe640 1 -- 172.21.15.86:0/982810129 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2aa80a4fc0 con 0x56364f46de80 2024-04-21T08:48:03.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.639+0000 7f2a937fe640 1 -- 172.21.15.86:0/982810129 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2aa80d0040 con 0x56364f46de80 2024-04-21T08:48:03.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.639+0000 7f2abcf92740 10 client.19314 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:03.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.639+0000 7f2abcf92740 10 client.19314 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:48:03.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.639+0000 7f2abcf92740 20 client.19314 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:03.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.639+0000 7f2abcf92740 1 --2- 172.21.15.86:0/982810129 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x56364f4f1090 0x56364f4f3480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.639+0000 7f2abcf92740 4 client.19314 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:48:03.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.639+0000 7f2abcf92740 1 -- 172.21.15.86:0/982810129 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56364f3dfc10 con 0x56364f4f1090 2024-04-21T08:48:03.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.639+0000 7f2ab3fff640 1 --2- 172.21.15.86:0/982810129 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x56364f4f1090 0x56364f4f3480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.640+0000 7f2a937fe640 10 client.19314 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:48:03.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.640+0000 7f2ab3fff640 1 --2- 172.21.15.86:0/982810129 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x56364f4f1090 0x56364f4f3480 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=366c5b9d2e34cf27 server_cookie=de8a91a33c983886 in_seq=0 out_seq=0 2024-04-21T08:48:03.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.640+0000 7f2a937fe640 1 -- 172.21.15.86:0/982810129 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56364f3dfc10 con 0x56364f4f1090 2024-04-21T08:48:03.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.640+0000 7f2a937fe640 10 client.19314 handle_command_reply: tid=0 2024-04-21T08:48:03.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.640+0000 7f2abcf92740 1 client.19314 shutdown 2024-04-21T08:48:03.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.640+0000 7f2abcf92740 1 -- 172.21.15.86:0/982810129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56364f46de80 msgr2=0x56364f48c460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.640+0000 7f2abcf92740 1 --2- 172.21.15.86:0/982810129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56364f46de80 0x56364f48c460 secure :-1 s=READY pgs=1700 cs=0 l=1 rev1=1 crypto rx=0x7f2aa8098d20 tx=0x7f2aa8003820 comp rx=0 tx=0).stop 2024-04-21T08:48:03.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.640+0000 7f2abcf92740 1 -- 172.21.15.86:0/982810129 shutdown_connections 2024-04-21T08:48:03.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.640+0000 7f2abcf92740 1 --2- 172.21.15.86:0/982810129 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56364f476a00 0x56364f4894b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.640+0000 7f2abcf92740 1 --2- 172.21.15.86:0/982810129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56364f46de80 0x56364f48c460 unknown :-1 s=CLOSED pgs=1700 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.640+0000 7f2abcf92740 1 --2- 172.21.15.86:0/982810129 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x56364f4f1090 0x56364f4f3480 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.640+0000 7f2abcf92740 1 --2- 172.21.15.86:0/982810129 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56364f46d4d0 0x56364f48bf20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.640+0000 7f2a937fe640 0 client.19314 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:48:03.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.640+0000 7f2abcf92740 1 -- 172.21.15.86:0/982810129 >> 172.21.15.86:0/982810129 conn(0x56364f467ba0 msgr2=0x56364f46e380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:03.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.640+0000 7f2abcf92740 1 -- 172.21.15.86:0/982810129 shutdown_connections 2024-04-21T08:48:03.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.640+0000 7f2abcf92740 1 -- 172.21.15.86:0/982810129 wait complete. 2024-04-21T08:48:03.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.640+0000 7f2abcf92740 20 client.19314 trim_cache size 0 max 16384 2024-04-21T08:48:03.647 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:48:03.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.641+0000 7f2aba3a4640 1 -- 172.21.15.86:0/2951139094 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2a9c0a3ec0 msgr2=0x7f2a9c0a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.641+0000 7f2aba3a4640 1 --2- 172.21.15.86:0/2951139094 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2a9c0a3ec0 0x7f2a9c0a6380 secure :-1 s=READY pgs=278 cs=0 l=1 rev1=1 crypto rx=0x7f2aac00afd0 tx=0x7f2aac012a10 comp rx=0 tx=0).stop 2024-04-21T08:48:03.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.641+0000 7f2aba3a4640 1 -- 172.21.15.86:0/2951139094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ab41072b0 msgr2=0x7f2ab419e940 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.641+0000 7f2aba3a4640 1 --2- 172.21.15.86:0/2951139094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ab41072b0 0x7f2ab419e940 secure :-1 s=READY pgs=1696 cs=0 l=1 rev1=1 crypto rx=0x7f2aa800c970 tx=0x7f2aa800ce30 comp rx=0 tx=0).stop 2024-04-21T08:48:03.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.641+0000 7f2aba3a4640 1 -- 172.21.15.86:0/2951139094 shutdown_connections 2024-04-21T08:48:03.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.641+0000 7f2aba3a4640 1 --2- 172.21.15.86:0/2951139094 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2a9c0a3ec0 0x7f2a9c0a6380 unknown :-1 s=CLOSED pgs=278 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.641+0000 7f2aba3a4640 1 --2- 172.21.15.86:0/2951139094 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2ab4198600 0x7f2ab4198ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.641+0000 7f2aba3a4640 1 --2- 172.21.15.86:0/2951139094 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2ab41072b0 0x7f2ab419e940 unknown :-1 s=CLOSED pgs=1696 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.641+0000 7f2aba3a4640 1 --2- 172.21.15.86:0/2951139094 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2ab41021e0 0x7f2ab419e400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.641+0000 7f2aba3a4640 1 -- 172.21.15.86:0/2951139094 >> 172.21.15.86:0/2951139094 conn(0x7f2ab40fda60 msgr2=0x7f2ab40fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:03.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.641+0000 7f2aba3a4640 1 -- 172.21.15.86:0/2951139094 shutdown_connections 2024-04-21T08:48:03.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.641+0000 7f2aba3a4640 1 -- 172.21.15.86:0/2951139094 wait complete. 2024-04-21T08:48:03.651 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:48:03.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.752+0000 7f79e6a31640 1 -- 172.21.15.86:0/3745219779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e0102b40 msgr2=0x7f79e0106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.752+0000 7f79e6a31640 1 --2- 172.21.15.86:0/3745219779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e0102b40 0x7f79e0106b80 secure :-1 s=READY pgs=1701 cs=0 l=1 rev1=1 crypto rx=0x7f79d0009900 tx=0x7f79d002bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:03.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.752+0000 7f79e6a31640 1 -- 172.21.15.86:0/3745219779 shutdown_connections 2024-04-21T08:48:03.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.752+0000 7f79e6a31640 1 --2- 172.21.15.86:0/3745219779 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79e0107260 0x7f79e010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.752+0000 7f79e6a31640 1 --2- 172.21.15.86:0/3745219779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e0102b40 0x7f79e0106b80 unknown :-1 s=CLOSED pgs=1701 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.752+0000 7f79e6a31640 1 --2- 172.21.15.86:0/3745219779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79e0102190 0x7f79e0102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.752+0000 7f79e6a31640 1 -- 172.21.15.86:0/3745219779 >> 172.21.15.86:0/3745219779 conn(0x7f79e00fd990 msgr2=0x7f79e00ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:03.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.752+0000 7f79e6a31640 1 -- 172.21.15.86:0/3745219779 shutdown_connections 2024-04-21T08:48:03.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.752+0000 7f79e6a31640 1 -- 172.21.15.86:0/3745219779 wait complete. 2024-04-21T08:48:03.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.752+0000 7f79e6a31640 1 Processor -- start 2024-04-21T08:48:03.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.752+0000 7f79e6a31640 1 -- start start 2024-04-21T08:48:03.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.753+0000 7f79e6a31640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79e0102190 0x7f79e019e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.753+0000 7f79e6a31640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e0107260 0x7f79e019e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.753+0000 7f79e6a31640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79e01985a0 0x7f79e0198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.753+0000 7f79e6a31640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f79e01121a0 con 0x7f79e0107260 2024-04-21T08:48:03.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.753+0000 7f79e6a31640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f79e0112020 con 0x7f79e0102190 2024-04-21T08:48:03.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.753+0000 7f79e6a31640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f79e0112320 con 0x7f79e01985a0 2024-04-21T08:48:03.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.753+0000 7f79df7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e0107260 0x7f79e019e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.753+0000 7f79df7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e0107260 0x7f79e019e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48202/0 (socket says 172.21.15.86:48202) 2024-04-21T08:48:03.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.753+0000 7f79df7fe640 1 -- 172.21.15.86:0/1578920769 learned_addr learned my addr 172.21.15.86:0/1578920769 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:03.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.753+0000 7f79dffff640 1 --2- 172.21.15.86:0/1578920769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79e0102190 0x7f79e019e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.753+0000 7f79e4fce640 1 --2- 172.21.15.86:0/1578920769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79e01985a0 0x7f79e0198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.753+0000 7f79df7fe640 1 -- 172.21.15.86:0/1578920769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79e01985a0 msgr2=0x7f79e0198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.753+0000 7f79df7fe640 1 --2- 172.21.15.86:0/1578920769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79e01985a0 0x7f79e0198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.753+0000 7f79df7fe640 1 -- 172.21.15.86:0/1578920769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79e0102190 msgr2=0x7f79e019e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.753+0000 7f79df7fe640 1 --2- 172.21.15.86:0/1578920769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79e0102190 0x7f79e019e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.753+0000 7f79df7fe640 1 -- 172.21.15.86:0/1578920769 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f79e01990c0 con 0x7f79e0107260 2024-04-21T08:48:03.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.753+0000 7f79e4fce640 1 --2- 172.21.15.86:0/1578920769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79e01985a0 0x7f79e0198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:03.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.753+0000 7f79df7fe640 1 --2- 172.21.15.86:0/1578920769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e0107260 0x7f79e019e930 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f79d0036040 tx=0x7f79d002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:03.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.753+0000 7f79dd7fa640 1 -- 172.21.15.86:0/1578920769 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d0043020 con 0x7f79e0107260 2024-04-21T08:48:03.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.753+0000 7f79e6a31640 1 -- 172.21.15.86:0/1578920769 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f79e0199350 con 0x7f79e0107260 2024-04-21T08:48:03.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.754+0000 7f79dd7fa640 1 -- 172.21.15.86:0/1578920769 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79d002cbe0 con 0x7f79e0107260 2024-04-21T08:48:03.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.754+0000 7f79dd7fa640 1 -- 172.21.15.86:0/1578920769 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d0003bf0 con 0x7f79e0107260 2024-04-21T08:48:03.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.754+0000 7f79e6a31640 1 -- 172.21.15.86:0/1578920769 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f79e01a55f0 con 0x7f79e0107260 2024-04-21T08:48:03.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.756+0000 7f79dd7fa640 1 -- 172.21.15.86:0/1578920769 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f79d0046020 con 0x7f79e0107260 2024-04-21T08:48:03.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.756+0000 7f79e9628740 1 Processor -- start 2024-04-21T08:48:03.763 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.756+0000 7f79e9628740 1 -- start start 2024-04-21T08:48:03.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.756+0000 7f79e9628740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0900a4d0 0x563d0900a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.756+0000 7f79e9628740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d0900ae80 0x563d09013430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.756+0000 7f79e9628740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d09013a00 0x563d09015e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.756+0000 7f79e9628740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563d08f6f1f0 con 0x563d0900a4d0 2024-04-21T08:48:03.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.756+0000 7f79e9628740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x563d08f0c4b0 con 0x563d09013a00 2024-04-21T08:48:03.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.756+0000 7f79e9628740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563d08f6b1b0 con 0x563d0900ae80 2024-04-21T08:48:03.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.756+0000 7f79dd7fa640 1 --2- 172.21.15.86:0/1578920769 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f79cc0a3da0 0x7f79cc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.756+0000 7f79e4fce640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0900a4d0 0x563d0900a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.757+0000 7f79e4fce640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0900a4d0 0x563d0900a8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48206/0 (socket says 172.21.15.86:48206) 2024-04-21T08:48:03.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.757+0000 7f79e4fce640 1 -- 172.21.15.86:0/1488601863 learned_addr learned my addr 172.21.15.86:0/1488601863 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:03.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.757+0000 7f79dd7fa640 1 -- 172.21.15.86:0/1578920769 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f79d00f2c60 con 0x7f79e0107260 2024-04-21T08:48:03.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.757+0000 7f79e4fce640 1 --2- 172.21.15.86:0/1578920769 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f79cc0a3da0 0x7f79cc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.757+0000 7f79dffff640 1 --2- 172.21.15.86:0/1488601863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d0900ae80 0x563d09013430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.757+0000 7f79df7fe640 1 --2- 172.21.15.86:0/1488601863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d09013a00 0x563d09015e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.757+0000 7f79e4fce640 1 -- 172.21.15.86:0/1488601863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d0900ae80 msgr2=0x563d09013430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:03.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.757+0000 7f79e4fce640 1 --2- 172.21.15.86:0/1488601863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d0900ae80 0x563d09013430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.757+0000 7f79e4fce640 1 -- 172.21.15.86:0/1488601863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d09013a00 msgr2=0x563d09015e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:03.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.757+0000 7f79e4fce640 1 --2- 172.21.15.86:0/1488601863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d09013a00 0x563d09015e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.757+0000 7f79e4fce640 1 -- 172.21.15.86:0/1488601863 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d08efe1b0 con 0x563d0900a4d0 2024-04-21T08:48:03.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.757+0000 7f79e4fce640 1 --2- 172.21.15.86:0/1488601863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0900a4d0 0x563d0900a8b0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f79d80092f0 tx=0x7f79d8011750 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ac94cae206d7098b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:03.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.757+0000 7f79e4fce640 1 --2- 172.21.15.86:0/1578920769 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f79cc0a3da0 0x7f79cc0a6260 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f79d8011c50 tx=0x7f79d8016000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:03.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.757+0000 7f79d67fc640 1 -- 172.21.15.86:0/1488601863 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d8022070 con 0x563d0900a4d0 2024-04-21T08:48:03.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.757+0000 7f79d67fc640 1 -- 172.21.15.86:0/1488601863 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79d8009bc0 con 0x563d0900a4d0 2024-04-21T08:48:03.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.757+0000 7f79d67fc640 1 -- 172.21.15.86:0/1488601863 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d80052d0 con 0x563d0900a4d0 2024-04-21T08:48:03.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.758+0000 7f79e9628740 1 -- 172.21.15.86:0/1488601863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0900a4d0 msgr2=0x563d0900a8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.758+0000 7f79e9628740 1 --2- 172.21.15.86:0/1488601863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0900a4d0 0x563d0900a8b0 secure :-1 s=READY pgs=1703 cs=0 l=1 rev1=1 crypto rx=0x7f79d80092f0 tx=0x7f79d8011750 comp rx=0 tx=0).stop 2024-04-21T08:48:03.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.758+0000 7f79e9628740 1 -- 172.21.15.86:0/1488601863 shutdown_connections 2024-04-21T08:48:03.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.758+0000 7f79e9628740 1 --2- 172.21.15.86:0/1488601863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d09013a00 0x563d09015e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.758+0000 7f79e9628740 1 --2- 172.21.15.86:0/1488601863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d0900ae80 0x563d09013430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.758+0000 7f79e9628740 1 --2- 172.21.15.86:0/1488601863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0900a4d0 0x563d0900a8b0 unknown :-1 s=CLOSED pgs=1703 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.758+0000 7f79e9628740 1 -- 172.21.15.86:0/1488601863 >> 172.21.15.86:0/1488601863 conn(0x563d09004ba0 msgr2=0x563d09004f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:03.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.758+0000 7f79e9628740 1 -- 172.21.15.86:0/1488601863 shutdown_connections 2024-04-21T08:48:03.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.758+0000 7f79e9628740 1 -- 172.21.15.86:0/1488601863 wait complete. 2024-04-21T08:48:03.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.758+0000 7f79e9628740 1 -- start start 2024-04-21T08:48:03.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.759+0000 7f79e9628740 1 Processor -- start 2024-04-21T08:48:03.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.759+0000 7f79e9628740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d0900a4d0 0x563d090133a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.759+0000 7f79e9628740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0900ae80 0x563d09010730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.759+0000 7f79e9628740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d09013a00 0x563d09010e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.759+0000 7f79e9628740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563d09016910 con 0x563d0900ae80 2024-04-21T08:48:03.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.759+0000 7f79e9628740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x563d08f28500 con 0x563d09013a00 2024-04-21T08:48:03.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.759+0000 7f79e9628740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563d09016a90 con 0x563d0900a4d0 2024-04-21T08:48:03.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.759+0000 7f79e9628740 10 client.0 _ll_register_callbacks cb 0x563d08ef6b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:03.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.759+0000 7f79df7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0900ae80 0x563d09010730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.759+0000 7f79df7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0900ae80 0x563d09010730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48220/0 (socket says 172.21.15.86:48220) 2024-04-21T08:48:03.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.759+0000 7f79df7fe640 1 -- 172.21.15.86:0/3867684767 learned_addr learned my addr 172.21.15.86:0/3867684767 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:03.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.759+0000 7f79dffff640 1 --2- 172.21.15.86:0/3867684767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d0900a4d0 0x563d090133a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.759+0000 7f79df7fe640 1 -- 172.21.15.86:0/3867684767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d0900a4d0 msgr2=0x563d090133a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:03.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.759+0000 7f79df7fe640 1 --2- 172.21.15.86:0/3867684767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d0900a4d0 0x563d090133a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.759+0000 7f79df7fe640 1 -- 172.21.15.86:0/3867684767 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d09013a00 msgr2=0x563d09010e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:03.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.759+0000 7f79e4fce640 1 --2- 172.21.15.86:0/3867684767 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d09013a00 0x563d09010e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.759+0000 7f79df7fe640 1 --2- 172.21.15.86:0/3867684767 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d09013a00 0x563d09010e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.759+0000 7f79df7fe640 1 -- 172.21.15.86:0/3867684767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d08c813f0 con 0x563d0900ae80 2024-04-21T08:48:03.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.759+0000 7f79e4fce640 1 --2- 172.21.15.86:0/3867684767 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d09013a00 0x563d09010e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:03.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.759+0000 7f79df7fe640 1 --2- 172.21.15.86:0/3867684767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0900ae80 0x563d09010730 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f79d00bada0 tx=0x7f79d0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d95b23e9f7bc8f15 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:03.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.759+0000 7f79bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:03.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.759+0000 7f79bf7fe640 1 -- 172.21.15.86:0/3867684767 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d00be940 con 0x563d0900ae80 2024-04-21T08:48:03.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.759+0000 7f79bf7fe640 1 -- 172.21.15.86:0/3867684767 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f79d00bafb0 con 0x563d0900ae80 2024-04-21T08:48:03.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.759+0000 7f79e9628740 1 -- 172.21.15.86:0/3867684767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79d8009bc0 con 0x563d0900ae80 2024-04-21T08:48:03.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.760+0000 7f79bf7fe640 1 -- 172.21.15.86:0/3867684767 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f79d00c2cf0 con 0x563d0900ae80 2024-04-21T08:48:03.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.760+0000 7f79e9628740 10 client.19326 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:03.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.760+0000 7f79e9628740 1 -- 172.21.15.86:0/3867684767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563d08d4b7d0 con 0x563d0900ae80 2024-04-21T08:48:03.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.760+0000 7f79bf7fe640 1 -- 172.21.15.86:0/3867684767 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79d8009bc0 con 0x563d0900ae80 2024-04-21T08:48:03.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.760+0000 7f79bf7fe640 1 -- 172.21.15.86:0/3867684767 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f79d00a90b0 con 0x563d0900ae80 2024-04-21T08:48:03.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.760+0000 7f79e9628740 10 client.19326 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:03.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.760+0000 7f79e9628740 10 client.19326 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:48:03.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.760+0000 7f79e9628740 20 client.19326 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:03.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.760+0000 7f79e9628740 1 --2- 172.21.15.86:0/3867684767 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x563d09023b70 0x563d09025f60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.760+0000 7f79e9628740 4 client.19326 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:03.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.760+0000 7f79e9628740 1 -- 172.21.15.86:0/3867684767 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d08f0e930 con 0x563d09023b70 2024-04-21T08:48:03.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.760+0000 7f79dffff640 1 --2- 172.21.15.86:0/3867684767 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x563d09023b70 0x563d09025f60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.761+0000 7f79bf7fe640 10 client.19326 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:48:03.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.761+0000 7f79dffff640 1 --2- 172.21.15.86:0/3867684767 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x563d09023b70 0x563d09025f60 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=440579f0f6fcdf09 server_cookie=9f25ac94c83690dc in_seq=0 out_seq=0 2024-04-21T08:48:03.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.761+0000 7f79bf7fe640 1 -- 172.21.15.86:0/3867684767 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563d08f0e930 con 0x563d09023b70 2024-04-21T08:48:03.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.761+0000 7f79bf7fe640 10 client.19326 handle_command_reply: tid=0 2024-04-21T08:48:03.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.761+0000 7f79e9628740 1 client.19326 shutdown 2024-04-21T08:48:03.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.762+0000 7f79e9628740 1 -- 172.21.15.86:0/3867684767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0900ae80 msgr2=0x563d09010730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.762+0000 7f79e9628740 1 --2- 172.21.15.86:0/3867684767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0900ae80 0x563d09010730 secure :-1 s=READY pgs=1704 cs=0 l=1 rev1=1 crypto rx=0x7f79d00bada0 tx=0x7f79d0000bc0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.762+0000 7f79e9628740 1 -- 172.21.15.86:0/3867684767 shutdown_connections 2024-04-21T08:48:03.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.762+0000 7f79e9628740 1 --2- 172.21.15.86:0/3867684767 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x563d09023b70 0x563d09025f60 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.762+0000 7f79e9628740 1 --2- 172.21.15.86:0/3867684767 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d09013a00 0x563d09010e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.762+0000 7f79e9628740 1 --2- 172.21.15.86:0/3867684767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0900ae80 0x563d09010730 unknown :-1 s=CLOSED pgs=1704 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.762+0000 7f79bf7fe640 0 client.19326 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:48:03.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.762+0000 7f79e9628740 1 --2- 172.21.15.86:0/3867684767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d0900a4d0 0x563d090133a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.762+0000 7f79e9628740 1 -- 172.21.15.86:0/3867684767 >> 172.21.15.86:0/3867684767 conn(0x563d09004ba0 msgr2=0x563d090068f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:03.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.762+0000 7f79e9628740 1 -- 172.21.15.86:0/3867684767 shutdown_connections 2024-04-21T08:48:03.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.762+0000 7f79e9628740 1 -- 172.21.15.86:0/3867684767 wait complete. 2024-04-21T08:48:03.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.762+0000 7f79e9628740 20 client.19326 trim_cache size 0 max 16384 2024-04-21T08:48:03.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.775+0000 7f79e9628740 1 Processor -- start 2024-04-21T08:48:03.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.775+0000 7f79e9628740 1 -- start start 2024-04-21T08:48:03.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.775+0000 7f79e9628740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d0900a4d0 0x563d0901f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.775+0000 7f79e9628740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0900ae80 0x563d09003c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.775+0000 7f79e9628740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d09013a00 0x563d0902b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.775+0000 7f79e9628740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f79e01af3c0 con 0x563d0900ae80 2024-04-21T08:48:03.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.775+0000 7f79e9628740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f79e01b0f00 con 0x563d0900a4d0 2024-04-21T08:48:03.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.775+0000 7f79e9628740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f79e01afa90 con 0x563d09013a00 2024-04-21T08:48:03.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.775+0000 7f79df7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0900ae80 0x563d09003c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.775+0000 7f79df7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0900ae80 0x563d09003c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48226/0 (socket says 172.21.15.86:48226) 2024-04-21T08:48:03.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.775+0000 7f79df7fe640 1 -- 172.21.15.86:0/1927189801 learned_addr learned my addr 172.21.15.86:0/1927189801 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:03.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.775+0000 7f79df7fe640 1 -- 172.21.15.86:0/1927189801 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d09013a00 msgr2=0x563d0902b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:03.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.775+0000 7f79df7fe640 1 --2- 172.21.15.86:0/1927189801 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d09013a00 0x563d0902b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.775+0000 7f79df7fe640 1 -- 172.21.15.86:0/1927189801 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d0900a4d0 msgr2=0x563d0901f7d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:03.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.775+0000 7f79dffff640 1 --2- 172.21.15.86:0/1927189801 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d0900a4d0 0x563d0901f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.775+0000 7f79df7fe640 1 --2- 172.21.15.86:0/1927189801 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d0900a4d0 0x563d0901f7d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.775+0000 7f79df7fe640 1 -- 172.21.15.86:0/1927189801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d0900ec80 con 0x563d0900ae80 2024-04-21T08:48:03.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.775+0000 7f79dffff640 1 --2- 172.21.15.86:0/1927189801 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d0900a4d0 0x563d0901f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:03.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.775+0000 7f79df7fe640 1 --2- 172.21.15.86:0/1927189801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0900ae80 0x563d09003c00 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f79d00d3a40 tx=0x7f79d002c450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=56ff2c5a3976817f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:03.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.775+0000 7f79d5ffb640 1 -- 172.21.15.86:0/1927189801 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d00db070 con 0x563d0900ae80 2024-04-21T08:48:03.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.776+0000 7f79d5ffb640 1 -- 172.21.15.86:0/1927189801 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79d00be940 con 0x563d0900ae80 2024-04-21T08:48:03.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.776+0000 7f79d5ffb640 1 -- 172.21.15.86:0/1927189801 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79d00c7120 con 0x563d0900ae80 2024-04-21T08:48:03.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.776+0000 7f79e9628740 1 -- 172.21.15.86:0/1927189801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0900ae80 msgr2=0x563d09003c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.776+0000 7f79e9628740 1 --2- 172.21.15.86:0/1927189801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0900ae80 0x563d09003c00 secure :-1 s=READY pgs=1705 cs=0 l=1 rev1=1 crypto rx=0x7f79d00d3a40 tx=0x7f79d002c450 comp rx=0 tx=0).stop 2024-04-21T08:48:03.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.776+0000 7f79e9628740 1 -- 172.21.15.86:0/1927189801 shutdown_connections 2024-04-21T08:48:03.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.776+0000 7f79e9628740 1 --2- 172.21.15.86:0/1927189801 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d09013a00 0x563d0902b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.776+0000 7f79e9628740 1 --2- 172.21.15.86:0/1927189801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0900ae80 0x563d09003c00 unknown :-1 s=CLOSED pgs=1705 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.776+0000 7f79e9628740 1 --2- 172.21.15.86:0/1927189801 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d0900a4d0 0x563d0901f7d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.776+0000 7f79e9628740 1 -- 172.21.15.86:0/1927189801 >> 172.21.15.86:0/1927189801 conn(0x563d09004ba0 msgr2=0x563d09004f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:03.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.776+0000 7f79e9628740 1 -- 172.21.15.86:0/1927189801 shutdown_connections 2024-04-21T08:48:03.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.776+0000 7f79e9628740 1 -- 172.21.15.86:0/1927189801 wait complete. 2024-04-21T08:48:03.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.776+0000 7f79e9628740 1 -- start start 2024-04-21T08:48:03.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.777+0000 7f79e9628740 1 Processor -- start 2024-04-21T08:48:03.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.777+0000 7f79e9628740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0900a4d0 0x563d09017dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.777+0000 7f79e9628740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d0900ae80 0x563d09018300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.777+0000 7f79e9628740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d09013a00 0x563d090890a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.777+0000 7f79e9628740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f79e01b0830 con 0x563d0900a4d0 2024-04-21T08:48:03.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.777+0000 7f79e9628740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f79e01b0160 con 0x563d09013a00 2024-04-21T08:48:03.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.777+0000 7f79e9628740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f79e01ad3c0 con 0x563d0900ae80 2024-04-21T08:48:03.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.777+0000 7f79e9628740 10 client.0 _ll_register_callbacks cb 0x7f79d00bac90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:03.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.777+0000 7f79dffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0900a4d0 0x563d09017dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.777+0000 7f79dffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0900a4d0 0x563d09017dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48230/0 (socket says 172.21.15.86:48230) 2024-04-21T08:48:03.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.777+0000 7f79dffff640 1 -- 172.21.15.86:0/1394525302 learned_addr learned my addr 172.21.15.86:0/1394525302 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:03.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.777+0000 7f79df7fe640 1 --2- 172.21.15.86:0/1394525302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d0900ae80 0x563d09018300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.777+0000 7f79dffff640 1 -- 172.21.15.86:0/1394525302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d0900ae80 msgr2=0x563d09018300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:03.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.777+0000 7f79dffff640 1 --2- 172.21.15.86:0/1394525302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d0900ae80 0x563d09018300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.777+0000 7f79dffff640 1 -- 172.21.15.86:0/1394525302 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d09013a00 msgr2=0x563d090890a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:03.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.777+0000 7f79e4fce640 1 --2- 172.21.15.86:0/1394525302 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d09013a00 0x563d090890a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.777+0000 7f79dffff640 1 --2- 172.21.15.86:0/1394525302 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d09013a00 0x563d090890a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.777+0000 7f79dffff640 1 -- 172.21.15.86:0/1394525302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d090895e0 con 0x563d0900a4d0 2024-04-21T08:48:03.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.777+0000 7f79df7fe640 1 --2- 172.21.15.86:0/1394525302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d0900ae80 0x563d09018300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:03.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.777+0000 7f79dffff640 1 --2- 172.21.15.86:0/1394525302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0900a4d0 0x563d09017dc0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x563d0900b280 tx=0x7f79c8010ad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1544ef246c4d2585 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:03.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.777+0000 7f79bf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:03.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.778+0000 7f79bf7fe640 1 -- 172.21.15.86:0/1394525302 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f79c8004220 con 0x563d0900a4d0 2024-04-21T08:48:03.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.778+0000 7f79bf7fe640 1 -- 172.21.15.86:0/1394525302 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f79c80043c0 con 0x563d0900a4d0 2024-04-21T08:48:03.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.778+0000 7f79e9628740 1 -- 172.21.15.86:0/1394525302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f79d00be940 con 0x563d0900a4d0 2024-04-21T08:48:03.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.778+0000 7f79bf7fe640 1 -- 172.21.15.86:0/1394525302 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f79c802f050 con 0x563d0900a4d0 2024-04-21T08:48:03.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.778+0000 7f79e9628740 10 client.19332 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:03.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.778+0000 7f79e9628740 1 -- 172.21.15.86:0/1394525302 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563d0908a810 con 0x563d0900a4d0 2024-04-21T08:48:03.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.778+0000 7f79bf7fe640 1 -- 172.21.15.86:0/1394525302 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f79d00be940 con 0x563d0900a4d0 2024-04-21T08:48:03.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.778+0000 7f79bf7fe640 1 -- 172.21.15.86:0/1394525302 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f79c8003c90 con 0x563d0900a4d0 2024-04-21T08:48:03.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.778+0000 7f79e9628740 10 client.19332 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:03.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.778+0000 7f79e9628740 10 client.19332 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:48:03.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.778+0000 7f79e9628740 20 client.19332 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:03.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.778+0000 7f79e9628740 1 --2- 172.21.15.86:0/1394525302 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x563d0908af40 0x563d0908d380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.778+0000 7f79e9628740 4 client.19332 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:48:03.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.778+0000 7f79e9628740 1 -- 172.21.15.86:0/1394525302 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563d08f7cc10 con 0x563d0908af40 2024-04-21T08:48:03.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.778+0000 7f79df7fe640 1 --2- 172.21.15.86:0/1394525302 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x563d0908af40 0x563d0908d380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.779+0000 7f79bf7fe640 10 client.19332 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:48:03.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.779+0000 7f79df7fe640 1 --2- 172.21.15.86:0/1394525302 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x563d0908af40 0x563d0908d380 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=97f677f58ca3b4e2 server_cookie=26fd64d4efd8cfdd in_seq=0 out_seq=0 2024-04-21T08:48:03.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.779+0000 7f79bf7fe640 1 -- 172.21.15.86:0/1394525302 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563d08f7cc10 con 0x563d0908af40 2024-04-21T08:48:03.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.779+0000 7f79bf7fe640 10 client.19332 handle_command_reply: tid=0 2024-04-21T08:48:03.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.779+0000 7f79e9628740 1 client.19332 shutdown 2024-04-21T08:48:03.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.779+0000 7f79e9628740 1 -- 172.21.15.86:0/1394525302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0900a4d0 msgr2=0x563d09017dc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.779+0000 7f79e9628740 1 --2- 172.21.15.86:0/1394525302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0900a4d0 0x563d09017dc0 secure :-1 s=READY pgs=1706 cs=0 l=1 rev1=1 crypto rx=0x563d0900b280 tx=0x7f79c8010ad0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.779+0000 7f79e9628740 1 -- 172.21.15.86:0/1394525302 shutdown_connections 2024-04-21T08:48:03.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.779+0000 7f79e9628740 1 --2- 172.21.15.86:0/1394525302 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x563d0908af40 0x563d0908d380 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.779+0000 7f79e9628740 1 --2- 172.21.15.86:0/1394525302 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d09013a00 0x563d090890a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.779+0000 7f79bf7fe640 0 client.19332 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:48:03.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.779+0000 7f79e9628740 1 --2- 172.21.15.86:0/1394525302 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d0900ae80 0x563d09018300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.779+0000 7f79e9628740 1 --2- 172.21.15.86:0/1394525302 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d0900a4d0 0x563d09017dc0 unknown :-1 s=CLOSED pgs=1706 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.779+0000 7f79e9628740 1 -- 172.21.15.86:0/1394525302 >> 172.21.15.86:0/1394525302 conn(0x563d09004ba0 msgr2=0x563d09006800 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:03.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.779+0000 7f79e9628740 1 -- 172.21.15.86:0/1394525302 shutdown_connections 2024-04-21T08:48:03.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.779+0000 7f79e9628740 1 -- 172.21.15.86:0/1394525302 wait complete. 2024-04-21T08:48:03.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.779+0000 7f79e9628740 20 client.19332 trim_cache size 0 max 16384 2024-04-21T08:48:03.786 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:48:03.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.780+0000 7f79e6a31640 1 -- 172.21.15.86:0/1578920769 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f79cc0a3da0 msgr2=0x7f79cc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.780+0000 7f79e6a31640 1 --2- 172.21.15.86:0/1578920769 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f79cc0a3da0 0x7f79cc0a6260 secure :-1 s=READY pgs=279 cs=0 l=1 rev1=1 crypto rx=0x7f79d8011c50 tx=0x7f79d8016000 comp rx=0 tx=0).stop 2024-04-21T08:48:03.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.780+0000 7f79e6a31640 1 -- 172.21.15.86:0/1578920769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e0107260 msgr2=0x7f79e019e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.780+0000 7f79e6a31640 1 --2- 172.21.15.86:0/1578920769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e0107260 0x7f79e019e930 secure :-1 s=READY pgs=1702 cs=0 l=1 rev1=1 crypto rx=0x7f79d0036040 tx=0x7f79d002c7e0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.780+0000 7f79e6a31640 1 -- 172.21.15.86:0/1578920769 shutdown_connections 2024-04-21T08:48:03.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.780+0000 7f79e6a31640 1 --2- 172.21.15.86:0/1578920769 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f79cc0a3da0 0x7f79cc0a6260 unknown :-1 s=CLOSED pgs=279 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.780+0000 7f79e6a31640 1 --2- 172.21.15.86:0/1578920769 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f79e01985a0 0x7f79e0198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.780+0000 7f79e6a31640 1 --2- 172.21.15.86:0/1578920769 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f79e0107260 0x7f79e019e930 unknown :-1 s=CLOSED pgs=1702 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.780+0000 7f79e6a31640 1 --2- 172.21.15.86:0/1578920769 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f79e0102190 0x7f79e019e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.780+0000 7f79e6a31640 1 -- 172.21.15.86:0/1578920769 >> 172.21.15.86:0/1578920769 conn(0x7f79e00fd990 msgr2=0x7f79e00fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:03.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.780+0000 7f79e6a31640 1 -- 172.21.15.86:0/1578920769 shutdown_connections 2024-04-21T08:48:03.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.780+0000 7f79e6a31640 1 -- 172.21.15.86:0/1578920769 wait complete. 2024-04-21T08:48:03.790 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:48:03.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.890+0000 7fec2bf2e640 1 -- 172.21.15.86:0/2736810464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec24103b00 msgr2=0x7fec2410bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.890+0000 7fec2bf2e640 1 --2- 172.21.15.86:0/2736810464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec24103b00 0x7fec2410bfe0 secure :-1 s=READY pgs=1707 cs=0 l=1 rev1=1 crypto rx=0x7fec14009900 tx=0x7fec1402bd60 comp rx=0 tx=0).stop 2024-04-21T08:48:03.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.890+0000 7fec2bf2e640 1 -- 172.21.15.86:0/2736810464 shutdown_connections 2024-04-21T08:48:03.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.890+0000 7fec2bf2e640 1 --2- 172.21.15.86:0/2736810464 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec2410c520 0x7fec2410e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.890+0000 7fec2bf2e640 1 --2- 172.21.15.86:0/2736810464 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec24103b00 0x7fec2410bfe0 unknown :-1 s=CLOSED pgs=1707 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.890+0000 7fec2bf2e640 1 --2- 172.21.15.86:0/2736810464 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec241031e0 0x7fec241035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.890+0000 7fec2bf2e640 1 -- 172.21.15.86:0/2736810464 >> 172.21.15.86:0/2736810464 conn(0x7fec240771d0 msgr2=0x7fec240775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:03.892 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.891+0000 7fec2bf2e640 1 -- 172.21.15.86:0/2736810464 shutdown_connections 2024-04-21T08:48:03.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.891+0000 7fec2bf2e640 1 -- 172.21.15.86:0/2736810464 wait complete. 2024-04-21T08:48:03.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.891+0000 7fec2bf2e640 1 Processor -- start 2024-04-21T08:48:03.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.891+0000 7fec2bf2e640 1 -- start start 2024-04-21T08:48:03.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.891+0000 7fec2bf2e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec241031e0 0x7fec24198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.891+0000 7fec2bf2e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec24103b00 0x7fec24198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.891+0000 7fec2bf2e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec2410c520 0x7fec2419cfd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.891+0000 7fec2bf2e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fec241121b0 con 0x7fec24103b00 2024-04-21T08:48:03.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.891+0000 7fec2bf2e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fec24112030 con 0x7fec241031e0 2024-04-21T08:48:03.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.891+0000 7fec2bf2e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fec24112330 con 0x7fec2410c520 2024-04-21T08:48:03.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.891+0000 7fec294c9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec24103b00 0x7fec24198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.891+0000 7fec294c9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec24103b00 0x7fec24198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48246/0 (socket says 172.21.15.86:48246) 2024-04-21T08:48:03.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.891+0000 7fec294c9640 1 -- 172.21.15.86:0/4076621114 learned_addr learned my addr 172.21.15.86:0/4076621114 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:03.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.891+0000 7fec2a4cb640 1 --2- 172.21.15.86:0/4076621114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec2410c520 0x7fec2419cfd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.893 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.891+0000 7fec29cca640 1 --2- 172.21.15.86:0/4076621114 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec241031e0 0x7fec24198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.892+0000 7fec294c9640 1 -- 172.21.15.86:0/4076621114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec2410c520 msgr2=0x7fec2419cfd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.892+0000 7fec294c9640 1 --2- 172.21.15.86:0/4076621114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec2410c520 0x7fec2419cfd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.892+0000 7fec294c9640 1 -- 172.21.15.86:0/4076621114 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec241031e0 msgr2=0x7fec24198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.892+0000 7fec294c9640 1 --2- 172.21.15.86:0/4076621114 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec241031e0 0x7fec24198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.892+0000 7fec294c9640 1 -- 172.21.15.86:0/4076621114 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fec2419d700 con 0x7fec24103b00 2024-04-21T08:48:03.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.892+0000 7fec294c9640 1 --2- 172.21.15.86:0/4076621114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec24103b00 0x7fec24198c50 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fec14005940 tx=0x7fec1402d920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:03.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.892+0000 7fec1affd640 1 -- 172.21.15.86:0/4076621114 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec14043020 con 0x7fec24103b00 2024-04-21T08:48:03.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.892+0000 7fec2bf2e640 1 -- 172.21.15.86:0/4076621114 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fec2419d990 con 0x7fec24103b00 2024-04-21T08:48:03.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.892+0000 7fec1affd640 1 -- 172.21.15.86:0/4076621114 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec1402dd20 con 0x7fec24103b00 2024-04-21T08:48:03.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.892+0000 7fec1affd640 1 -- 172.21.15.86:0/4076621114 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec14003b80 con 0x7fec24103b00 2024-04-21T08:48:03.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.892+0000 7fec2bf2e640 1 -- 172.21.15.86:0/4076621114 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fec241a5280 con 0x7fec24103b00 2024-04-21T08:48:03.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.894+0000 7fec1affd640 1 -- 172.21.15.86:0/4076621114 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fec14046020 con 0x7fec24103b00 2024-04-21T08:48:03.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.894+0000 7fec2ebf4740 1 Processor -- start 2024-04-21T08:48:03.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.894+0000 7fec2ebf4740 1 -- start start 2024-04-21T08:48:03.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.895+0000 7fec2ebf4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5645c3e634d0 0x5645c3e638b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.895+0000 7fec2ebf4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645c3e63e80 0x5645c3e6c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.895+0000 7fec2ebf4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645c3e6ca00 0x5645c3e6ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.895+0000 7fec2ebf4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5645c3dc81f0 con 0x5645c3e6ca00 2024-04-21T08:48:03.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.895+0000 7fec2ebf4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5645c3d654b0 con 0x5645c3e634d0 2024-04-21T08:48:03.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.895+0000 7fec2ebf4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5645c3dc41b0 con 0x5645c3e63e80 2024-04-21T08:48:03.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.895+0000 7fec1affd640 1 --2- 172.21.15.86:0/4076621114 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fec100a3d20 0x7fec100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.895+0000 7fec294c9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645c3e6ca00 0x5645c3e6ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.895+0000 7fec294c9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645c3e6ca00 0x5645c3e6ee50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48262/0 (socket says 172.21.15.86:48262) 2024-04-21T08:48:03.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.895+0000 7fec294c9640 1 -- 172.21.15.86:0/2313659245 learned_addr learned my addr 172.21.15.86:0/2313659245 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:03.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.895+0000 7fec2a4cb640 1 --2- 172.21.15.86:0/4076621114 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fec100a3d20 0x7fec100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.895+0000 7fec29cca640 1 --2- 172.21.15.86:0/2313659245 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645c3e63e80 0x5645c3e6c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.895+0000 7fec2a4cb640 1 --2- 172.21.15.86:0/2313659245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5645c3e634d0 0x5645c3e638b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.895+0000 7fec1affd640 1 -- 172.21.15.86:0/4076621114 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fec140f2db0 con 0x7fec24103b00 2024-04-21T08:48:03.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.895+0000 7fec294c9640 1 -- 172.21.15.86:0/2313659245 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645c3e63e80 msgr2=0x5645c3e6c430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:03.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.895+0000 7fec294c9640 1 --2- 172.21.15.86:0/2313659245 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645c3e63e80 0x5645c3e6c430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.895+0000 7fec294c9640 1 -- 172.21.15.86:0/2313659245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5645c3e634d0 msgr2=0x5645c3e638b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:03.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.895+0000 7fec294c9640 1 --2- 172.21.15.86:0/2313659245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5645c3e634d0 0x5645c3e638b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.895+0000 7fec294c9640 1 -- 172.21.15.86:0/2313659245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645c3d571b0 con 0x5645c3e6ca00 2024-04-21T08:48:03.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.895+0000 7fec29cca640 1 --2- 172.21.15.86:0/2313659245 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645c3e63e80 0x5645c3e6c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:03.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.895+0000 7fec2a4cb640 1 --2- 172.21.15.86:0/4076621114 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fec100a3d20 0x7fec100a61e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fec1c007fd0 tx=0x7fec1c007c60 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:03.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.895+0000 7fec294c9640 1 --2- 172.21.15.86:0/2313659245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645c3e6ca00 0x5645c3e6ee50 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fec140bb460 tx=0x7fec140035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3495dc56e312ef63 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:03.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.896+0000 7fec0ffff640 1 -- 172.21.15.86:0/2313659245 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec140c3fd0 con 0x5645c3e6ca00 2024-04-21T08:48:03.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.896+0000 7fec0ffff640 1 -- 172.21.15.86:0/2313659245 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec140bf190 con 0x5645c3e6ca00 2024-04-21T08:48:03.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.896+0000 7fec0ffff640 1 -- 172.21.15.86:0/2313659245 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec140becf0 con 0x5645c3e6ca00 2024-04-21T08:48:03.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.896+0000 7fec2ebf4740 1 -- 172.21.15.86:0/2313659245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645c3e6ca00 msgr2=0x5645c3e6ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.896+0000 7fec2ebf4740 1 --2- 172.21.15.86:0/2313659245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645c3e6ca00 0x5645c3e6ee50 secure :-1 s=READY pgs=1709 cs=0 l=1 rev1=1 crypto rx=0x7fec140bb460 tx=0x7fec140035f0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.897+0000 7fec2ebf4740 1 -- 172.21.15.86:0/2313659245 shutdown_connections 2024-04-21T08:48:03.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.897+0000 7fec2ebf4740 1 --2- 172.21.15.86:0/2313659245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645c3e6ca00 0x5645c3e6ee50 unknown :-1 s=CLOSED pgs=1709 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.897+0000 7fec2ebf4740 1 --2- 172.21.15.86:0/2313659245 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645c3e63e80 0x5645c3e6c430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.897+0000 7fec2ebf4740 1 --2- 172.21.15.86:0/2313659245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5645c3e634d0 0x5645c3e638b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.897+0000 7fec2ebf4740 1 -- 172.21.15.86:0/2313659245 >> 172.21.15.86:0/2313659245 conn(0x5645c3e5dba0 msgr2=0x5645c3e5df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:03.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.897+0000 7fec2ebf4740 1 -- 172.21.15.86:0/2313659245 shutdown_connections 2024-04-21T08:48:03.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.897+0000 7fec2ebf4740 1 -- 172.21.15.86:0/2313659245 wait complete. 2024-04-21T08:48:03.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.897+0000 7fec2ebf4740 1 -- start start 2024-04-21T08:48:03.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.897+0000 7fec2ebf4740 1 Processor -- start 2024-04-21T08:48:03.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.897+0000 7fec2ebf4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5645c3e634d0 0x5645c3e6c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.897+0000 7fec2ebf4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645c3e63e80 0x5645c3e696f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.897+0000 7fec2ebf4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645c3e6ca00 0x5645c3e69d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.897+0000 7fec2ebf4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5645c3e6f730 con 0x5645c3e6ca00 2024-04-21T08:48:03.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.897+0000 7fec2ebf4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5645c3d81500 con 0x5645c3e634d0 2024-04-21T08:48:03.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.897+0000 7fec2ebf4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5645c3e6f8b0 con 0x5645c3e63e80 2024-04-21T08:48:03.904 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.897+0000 7fec2ebf4740 10 client.0 _ll_register_callbacks cb 0x5645c3d4fb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:03.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.898+0000 7fec2a4cb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645c3e6ca00 0x5645c3e69d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.898+0000 7fec29cca640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5645c3e634d0 0x5645c3e6c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.898+0000 7fec29cca640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5645c3e634d0 0x5645c3e6c360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.86:52924/0 (socket says 172.21.15.86:52924) 2024-04-21T08:48:03.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.898+0000 7fec2a4cb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645c3e6ca00 0x5645c3e69d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48274/0 (socket says 172.21.15.86:48274) 2024-04-21T08:48:03.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.898+0000 7fec29cca640 1 -- 172.21.15.86:0/883336448 learned_addr learned my addr 172.21.15.86:0/883336448 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:03.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.898+0000 7fec294c9640 1 --2- 172.21.15.86:0/883336448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645c3e63e80 0x5645c3e696f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.898+0000 7fec2a4cb640 1 -- 172.21.15.86:0/883336448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645c3e63e80 msgr2=0x5645c3e696f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:03.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.898+0000 7fec2a4cb640 1 --2- 172.21.15.86:0/883336448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645c3e63e80 0x5645c3e696f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.898+0000 7fec2a4cb640 1 -- 172.21.15.86:0/883336448 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5645c3e634d0 msgr2=0x5645c3e6c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:03.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.898+0000 7fec2a4cb640 1 --2- 172.21.15.86:0/883336448 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5645c3e634d0 0x5645c3e6c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.898+0000 7fec2a4cb640 1 -- 172.21.15.86:0/883336448 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645c3ada3f0 con 0x5645c3e6ca00 2024-04-21T08:48:03.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.898+0000 7fec2a4cb640 1 --2- 172.21.15.86:0/883336448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645c3e6ca00 0x5645c3e69d00 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fec1c0107a0 tx=0x7fec1c01a000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5141ed97b9a48e3d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:03.905 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.898+0000 7fec0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:03.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.899+0000 7fec0d7fa640 1 -- 172.21.15.86:0/883336448 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec1c024030 con 0x5645c3e6ca00 2024-04-21T08:48:03.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.900+0000 7fec0d7fa640 1 -- 172.21.15.86:0/883336448 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fec1c010cc0 con 0x5645c3e6ca00 2024-04-21T08:48:03.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.900+0000 7fec2ebf4740 1 -- 172.21.15.86:0/883336448 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec140bf190 con 0x5645c3e6ca00 2024-04-21T08:48:03.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.900+0000 7fec0d7fa640 1 -- 172.21.15.86:0/883336448 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fec1c030050 con 0x5645c3e6ca00 2024-04-21T08:48:03.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.900+0000 7fec2ebf4740 10 client.19344 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:03.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.900+0000 7fec2ebf4740 1 -- 172.21.15.86:0/883336448 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5645c3ba47d0 con 0x5645c3e6ca00 2024-04-21T08:48:03.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.900+0000 7fec0d7fa640 1 -- 172.21.15.86:0/883336448 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec140bf190 con 0x5645c3e6ca00 2024-04-21T08:48:03.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.900+0000 7fec0d7fa640 1 -- 172.21.15.86:0/883336448 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fec1c0053e0 con 0x5645c3e6ca00 2024-04-21T08:48:03.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.900+0000 7fec2ebf4740 10 client.19344 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:03.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.900+0000 7fec2ebf4740 10 client.19344 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:48:03.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.900+0000 7fec2ebf4740 20 client.19344 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:03.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.900+0000 7fec2ebf4740 1 --2- 172.21.15.86:0/883336448 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5645c3e7c920 0x5645c3e7ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.900+0000 7fec2ebf4740 4 client.19344 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:03.906 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.900+0000 7fec2ebf4740 1 -- 172.21.15.86:0/883336448 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5645c3d67930 con 0x5645c3e7c920 2024-04-21T08:48:03.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.900+0000 7fec29cca640 1 --2- 172.21.15.86:0/883336448 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5645c3e7c920 0x5645c3e7ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.901+0000 7fec29cca640 1 --2- 172.21.15.86:0/883336448 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5645c3e7c920 0x5645c3e7ed80 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=87314a1f4d2fa02d server_cookie=50c98c31fd0175eb in_seq=0 out_seq=0 2024-04-21T08:48:03.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.901+0000 7fec0d7fa640 10 client.19344 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:48:03.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.901+0000 7fec0d7fa640 1 -- 172.21.15.86:0/883336448 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5645c3d67930 con 0x5645c3e7c920 2024-04-21T08:48:03.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.901+0000 7fec0d7fa640 10 client.19344 handle_command_reply: tid=0 2024-04-21T08:48:03.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.901+0000 7fec2ebf4740 1 client.19344 shutdown 2024-04-21T08:48:03.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.902+0000 7fec2ebf4740 1 -- 172.21.15.86:0/883336448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645c3e6ca00 msgr2=0x5645c3e69d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.902+0000 7fec2ebf4740 1 --2- 172.21.15.86:0/883336448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645c3e6ca00 0x5645c3e69d00 secure :-1 s=READY pgs=1710 cs=0 l=1 rev1=1 crypto rx=0x7fec1c0107a0 tx=0x7fec1c01a000 comp rx=0 tx=0).stop 2024-04-21T08:48:03.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.902+0000 7fec2ebf4740 1 -- 172.21.15.86:0/883336448 shutdown_connections 2024-04-21T08:48:03.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.902+0000 7fec2ebf4740 1 --2- 172.21.15.86:0/883336448 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5645c3e7c920 0x5645c3e7ed80 crc :-1 s=READY pgs=67 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.902+0000 7fec2ebf4740 1 --2- 172.21.15.86:0/883336448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645c3e6ca00 0x5645c3e69d00 unknown :-1 s=CLOSED pgs=1710 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.902+0000 7fec0d7fa640 0 client.19344 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:48:03.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.902+0000 7fec2ebf4740 1 --2- 172.21.15.86:0/883336448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645c3e63e80 0x5645c3e696f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.902+0000 7fec2ebf4740 1 --2- 172.21.15.86:0/883336448 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5645c3e634d0 0x5645c3e6c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.907 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.902+0000 7fec2ebf4740 1 -- 172.21.15.86:0/883336448 >> 172.21.15.86:0/883336448 conn(0x5645c3e5dba0 msgr2=0x5645c3e5f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:03.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.902+0000 7fec2ebf4740 1 -- 172.21.15.86:0/883336448 shutdown_connections 2024-04-21T08:48:03.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.902+0000 7fec2ebf4740 1 -- 172.21.15.86:0/883336448 wait complete. 2024-04-21T08:48:03.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.902+0000 7fec2ebf4740 20 client.19344 trim_cache size 0 max 16384 2024-04-21T08:48:03.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.915+0000 7fec2ebf4740 1 Processor -- start 2024-04-21T08:48:03.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.915+0000 7fec2ebf4740 1 -- start start 2024-04-21T08:48:03.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.915+0000 7fec2ebf4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645c3e634d0 0x5645c3e78560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.915+0000 7fec2ebf4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645c3e63e80 0x5645c3e6be20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.915+0000 7fec2ebf4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5645c3e6ca00 0x5645c3e6c360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.915+0000 7fec2ebf4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fec241af330 con 0x5645c3e634d0 2024-04-21T08:48:03.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.915+0000 7fec2ebf4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fec241b0e70 con 0x5645c3e6ca00 2024-04-21T08:48:03.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.915+0000 7fec2ebf4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fec241afa00 con 0x5645c3e63e80 2024-04-21T08:48:03.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.915+0000 7fec29cca640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645c3e634d0 0x5645c3e78560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.915+0000 7fec29cca640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645c3e634d0 0x5645c3e78560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48290/0 (socket says 172.21.15.86:48290) 2024-04-21T08:48:03.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.915+0000 7fec29cca640 1 -- 172.21.15.86:0/3906662966 learned_addr learned my addr 172.21.15.86:0/3906662966 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:03.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.915+0000 7fec2a4cb640 1 --2- 172.21.15.86:0/3906662966 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5645c3e6ca00 0x5645c3e6c360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.915+0000 7fec294c9640 1 --2- 172.21.15.86:0/3906662966 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645c3e63e80 0x5645c3e6be20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.915+0000 7fec29cca640 1 -- 172.21.15.86:0/3906662966 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645c3e63e80 msgr2=0x5645c3e6be20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:03.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.915+0000 7fec29cca640 1 --2- 172.21.15.86:0/3906662966 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645c3e63e80 0x5645c3e6be20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.916+0000 7fec29cca640 1 -- 172.21.15.86:0/3906662966 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5645c3e6ca00 msgr2=0x5645c3e6c360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:03.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.916+0000 7fec29cca640 1 --2- 172.21.15.86:0/3906662966 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5645c3e6ca00 0x5645c3e6c360 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.916+0000 7fec29cca640 1 -- 172.21.15.86:0/3906662966 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5645c3e6f1c0 con 0x5645c3e634d0 2024-04-21T08:48:03.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.916+0000 7fec2a4cb640 1 --2- 172.21.15.86:0/3906662966 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5645c3e6ca00 0x5645c3e6c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:03.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.916+0000 7fec29cca640 1 --2- 172.21.15.86:0/3906662966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645c3e634d0 0x5645c3e78560 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fec2000bbe0 tx=0x7fec200138f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=380c518d014b768c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:03.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.916+0000 7fec0f7fe640 1 -- 172.21.15.86:0/3906662966 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec2001d430 con 0x5645c3e634d0 2024-04-21T08:48:03.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.916+0000 7fec0f7fe640 1 -- 172.21.15.86:0/3906662966 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec20005020 con 0x5645c3e634d0 2024-04-21T08:48:03.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.916+0000 7fec0f7fe640 1 -- 172.21.15.86:0/3906662966 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec20004c60 con 0x5645c3e634d0 2024-04-21T08:48:03.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.916+0000 7fec2ebf4740 1 -- 172.21.15.86:0/3906662966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645c3e634d0 msgr2=0x5645c3e78560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.916+0000 7fec2ebf4740 1 --2- 172.21.15.86:0/3906662966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645c3e634d0 0x5645c3e78560 secure :-1 s=READY pgs=1711 cs=0 l=1 rev1=1 crypto rx=0x7fec2000bbe0 tx=0x7fec200138f0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.916+0000 7fec2ebf4740 1 -- 172.21.15.86:0/3906662966 shutdown_connections 2024-04-21T08:48:03.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.916+0000 7fec2ebf4740 1 --2- 172.21.15.86:0/3906662966 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5645c3e6ca00 0x5645c3e6c360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.916+0000 7fec2ebf4740 1 --2- 172.21.15.86:0/3906662966 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645c3e63e80 0x5645c3e6be20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.917+0000 7fec2ebf4740 1 --2- 172.21.15.86:0/3906662966 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645c3e634d0 0x5645c3e78560 unknown :-1 s=CLOSED pgs=1711 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.917+0000 7fec2ebf4740 1 -- 172.21.15.86:0/3906662966 >> 172.21.15.86:0/3906662966 conn(0x5645c3e5dba0 msgr2=0x5645c3e5cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:03.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.917+0000 7fec2ebf4740 1 -- 172.21.15.86:0/3906662966 shutdown_connections 2024-04-21T08:48:03.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.917+0000 7fec2ebf4740 1 -- 172.21.15.86:0/3906662966 wait complete. 2024-04-21T08:48:03.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.917+0000 7fec2ebf4740 1 -- start start 2024-04-21T08:48:03.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.917+0000 7fec2ebf4740 1 Processor -- start 2024-04-21T08:48:03.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.917+0000 7fec2ebf4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5645c3e634d0 0x5645c3e74d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.917+0000 7fec2ebf4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645c3e63e80 0x5645c3e66060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.917+0000 7fec2ebf4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645c3e6ca00 0x5645c3e665a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.917+0000 7fec2ebf4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fec241b07a0 con 0x5645c3e63e80 2024-04-21T08:48:03.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.917+0000 7fec2ebf4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fec241b00d0 con 0x5645c3e634d0 2024-04-21T08:48:03.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.917+0000 7fec2ebf4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fec241ad330 con 0x5645c3e6ca00 2024-04-21T08:48:03.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.917+0000 7fec2ebf4740 10 client.0 _ll_register_callbacks cb 0x7fec1c013310 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:03.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.917+0000 7fec294c9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645c3e63e80 0x5645c3e66060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.917+0000 7fec294c9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645c3e63e80 0x5645c3e66060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48294/0 (socket says 172.21.15.86:48294) 2024-04-21T08:48:03.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.917+0000 7fec294c9640 1 -- 172.21.15.86:0/3956946685 learned_addr learned my addr 172.21.15.86:0/3956946685 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:03.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.918+0000 7fec2a4cb640 1 --2- 172.21.15.86:0/3956946685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645c3e6ca00 0x5645c3e665a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.918+0000 7fec29cca640 1 --2- 172.21.15.86:0/3956946685 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5645c3e634d0 0x5645c3e74d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.918+0000 7fec294c9640 1 -- 172.21.15.86:0/3956946685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645c3e6ca00 msgr2=0x5645c3e665a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:03.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.918+0000 7fec294c9640 1 --2- 172.21.15.86:0/3956946685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645c3e6ca00 0x5645c3e665a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.918+0000 7fec294c9640 1 -- 172.21.15.86:0/3956946685 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5645c3e634d0 msgr2=0x5645c3e74d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:03.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.918+0000 7fec294c9640 1 --2- 172.21.15.86:0/3956946685 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5645c3e634d0 0x5645c3e74d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.918+0000 7fec294c9640 1 -- 172.21.15.86:0/3956946685 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5645c3e66ce0 con 0x5645c3e63e80 2024-04-21T08:48:03.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.918+0000 7fec294c9640 1 --2- 172.21.15.86:0/3956946685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645c3e63e80 0x5645c3e66060 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fec140bea20 tx=0x7fec140092d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf89bb0578874576 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:03.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.918+0000 7fec0d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:03.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.918+0000 7fec0d7fa640 1 -- 172.21.15.86:0/3956946685 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fec140be3f0 con 0x5645c3e63e80 2024-04-21T08:48:03.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.918+0000 7fec0d7fa640 1 -- 172.21.15.86:0/3956946685 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7fec14003d20 con 0x5645c3e63e80 2024-04-21T08:48:03.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.918+0000 7fec2ebf4740 1 -- 172.21.15.86:0/3956946685 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fec20005020 con 0x5645c3e63e80 2024-04-21T08:48:03.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.918+0000 7fec0d7fa640 1 -- 172.21.15.86:0/3956946685 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fec1403c350 con 0x5645c3e63e80 2024-04-21T08:48:03.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.918+0000 7fec2ebf4740 10 client.19350 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:03.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.918+0000 7fec2ebf4740 1 -- 172.21.15.86:0/3956946685 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5645c3ee2e40 con 0x5645c3e63e80 2024-04-21T08:48:03.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.918+0000 7fec0d7fa640 1 -- 172.21.15.86:0/3956946685 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fec20005020 con 0x5645c3e63e80 2024-04-21T08:48:03.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.918+0000 7fec0d7fa640 1 -- 172.21.15.86:0/3956946685 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fec140c4210 con 0x5645c3e63e80 2024-04-21T08:48:03.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.918+0000 7fec2ebf4740 10 client.19350 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:03.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.918+0000 7fec2ebf4740 10 client.19350 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:48:03.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.918+0000 7fec2ebf4740 20 client.19350 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:03.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.918+0000 7fec2ebf4740 1 --2- 172.21.15.86:0/3956946685 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5645c3ee33d0 0x5645c3ee57f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:03.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.918+0000 7fec2ebf4740 4 client.19350 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:48:03.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.918+0000 7fec2ebf4740 1 -- 172.21.15.86:0/3956946685 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5645c3dd5c10 con 0x5645c3ee33d0 2024-04-21T08:48:03.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.919+0000 7fec29cca640 1 --2- 172.21.15.86:0/3956946685 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5645c3ee33d0 0x5645c3ee57f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:03.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.919+0000 7fec29cca640 1 --2- 172.21.15.86:0/3956946685 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5645c3ee33d0 0x5645c3ee57f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=801833d0e984fab4 server_cookie=2a90d1dc69412e33 in_seq=0 out_seq=0 2024-04-21T08:48:03.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.919+0000 7fec0d7fa640 10 client.19350 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:48:03.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.919+0000 7fec0d7fa640 1 -- 172.21.15.86:0/3956946685 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5645c3dd5c10 con 0x5645c3ee33d0 2024-04-21T08:48:03.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.919+0000 7fec0d7fa640 10 client.19350 handle_command_reply: tid=0 2024-04-21T08:48:03.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.919+0000 7fec2ebf4740 1 client.19350 shutdown 2024-04-21T08:48:03.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.919+0000 7fec2ebf4740 1 -- 172.21.15.86:0/3956946685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645c3e63e80 msgr2=0x5645c3e66060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.919+0000 7fec2ebf4740 1 --2- 172.21.15.86:0/3956946685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645c3e63e80 0x5645c3e66060 secure :-1 s=READY pgs=1712 cs=0 l=1 rev1=1 crypto rx=0x7fec140bea20 tx=0x7fec140092d0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.919+0000 7fec2ebf4740 1 -- 172.21.15.86:0/3956946685 shutdown_connections 2024-04-21T08:48:03.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.919+0000 7fec2ebf4740 1 --2- 172.21.15.86:0/3956946685 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5645c3ee33d0 0x5645c3ee57f0 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.919+0000 7fec2ebf4740 1 --2- 172.21.15.86:0/3956946685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5645c3e6ca00 0x5645c3e665a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.919+0000 7fec2ebf4740 1 --2- 172.21.15.86:0/3956946685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5645c3e63e80 0x5645c3e66060 unknown :-1 s=CLOSED pgs=1712 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.919+0000 7fec2ebf4740 1 --2- 172.21.15.86:0/3956946685 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5645c3e634d0 0x5645c3e74d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.919+0000 7fec0d7fa640 0 client.19350 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:48:03.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.919+0000 7fec2ebf4740 1 -- 172.21.15.86:0/3956946685 >> 172.21.15.86:0/3956946685 conn(0x5645c3e5dba0 msgr2=0x5645c3e5cc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:03.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.919+0000 7fec2ebf4740 1 -- 172.21.15.86:0/3956946685 shutdown_connections 2024-04-21T08:48:03.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.920+0000 7fec2ebf4740 1 -- 172.21.15.86:0/3956946685 wait complete. 2024-04-21T08:48:03.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.920+0000 7fec2ebf4740 20 client.19350 trim_cache size 0 max 16384 2024-04-21T08:48:03.926 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:48:03.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.920+0000 7fec2bf2e640 1 -- 172.21.15.86:0/4076621114 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fec100a3d20 msgr2=0x7fec100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.920+0000 7fec2bf2e640 1 --2- 172.21.15.86:0/4076621114 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fec100a3d20 0x7fec100a61e0 secure :-1 s=READY pgs=280 cs=0 l=1 rev1=1 crypto rx=0x7fec1c007fd0 tx=0x7fec1c007c60 comp rx=0 tx=0).stop 2024-04-21T08:48:03.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.920+0000 7fec2bf2e640 1 -- 172.21.15.86:0/4076621114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec24103b00 msgr2=0x7fec24198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:03.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.920+0000 7fec2bf2e640 1 --2- 172.21.15.86:0/4076621114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec24103b00 0x7fec24198c50 secure :-1 s=READY pgs=1708 cs=0 l=1 rev1=1 crypto rx=0x7fec14005940 tx=0x7fec1402d920 comp rx=0 tx=0).stop 2024-04-21T08:48:03.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.920+0000 7fec2bf2e640 1 -- 172.21.15.86:0/4076621114 shutdown_connections 2024-04-21T08:48:03.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.920+0000 7fec2bf2e640 1 --2- 172.21.15.86:0/4076621114 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fec100a3d20 0x7fec100a61e0 unknown :-1 s=CLOSED pgs=280 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.920+0000 7fec2bf2e640 1 --2- 172.21.15.86:0/4076621114 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fec2410c520 0x7fec2419cfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.920+0000 7fec2bf2e640 1 --2- 172.21.15.86:0/4076621114 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fec24103b00 0x7fec24198c50 unknown :-1 s=CLOSED pgs=1708 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.927 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.920+0000 7fec2bf2e640 1 --2- 172.21.15.86:0/4076621114 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fec241031e0 0x7fec24198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:03.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.920+0000 7fec2bf2e640 1 -- 172.21.15.86:0/4076621114 >> 172.21.15.86:0/4076621114 conn(0x7fec240771d0 msgr2=0x7fec240fdda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:03.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.920+0000 7fec2bf2e640 1 -- 172.21.15.86:0/4076621114 shutdown_connections 2024-04-21T08:48:03.928 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:03.920+0000 7fec2bf2e640 1 -- 172.21.15.86:0/4076621114 wait complete. 2024-04-21T08:48:03.930 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:48:04.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.030+0000 7faa3932b640 1 -- 172.21.15.86:0/785447802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa3410c520 msgr2=0x7faa3410e9c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:04.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.030+0000 7faa3932b640 1 --2- 172.21.15.86:0/785447802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa3410c520 0x7faa3410e9c0 secure :-1 s=READY pgs=1713 cs=0 l=1 rev1=1 crypto rx=0x7faa2c009ca0 tx=0x7faa2c02c230 comp rx=0 tx=0).stop 2024-04-21T08:48:04.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.030+0000 7faa3932b640 1 -- 172.21.15.86:0/785447802 shutdown_connections 2024-04-21T08:48:04.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.030+0000 7faa3932b640 1 --2- 172.21.15.86:0/785447802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa3410c520 0x7faa3410e9c0 unknown :-1 s=CLOSED pgs=1713 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.030+0000 7faa3932b640 1 --2- 172.21.15.86:0/785447802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa34103b00 0x7faa3410bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.030+0000 7faa3932b640 1 --2- 172.21.15.86:0/785447802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa341031e0 0x7faa341035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.030+0000 7faa3932b640 1 -- 172.21.15.86:0/785447802 >> 172.21.15.86:0/785447802 conn(0x7faa340771d0 msgr2=0x7faa340775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:04.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.030+0000 7faa3932b640 1 -- 172.21.15.86:0/785447802 shutdown_connections 2024-04-21T08:48:04.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.030+0000 7faa3932b640 1 -- 172.21.15.86:0/785447802 wait complete. 2024-04-21T08:48:04.032 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.030+0000 7faa3932b640 1 Processor -- start 2024-04-21T08:48:04.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.031+0000 7faa3932b640 1 -- start start 2024-04-21T08:48:04.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.031+0000 7faa3932b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa341031e0 0x7faa34198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:04.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.031+0000 7faa3932b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa34103b00 0x7faa34198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:04.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.031+0000 7faa3932b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa3410c520 0x7faa3419d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:04.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.031+0000 7faa3932b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa341121b0 con 0x7faa3410c520 2024-04-21T08:48:04.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.031+0000 7faa3932b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faa34112030 con 0x7faa34103b00 2024-04-21T08:48:04.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.031+0000 7faa3932b640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faa34112330 con 0x7faa341031e0 2024-04-21T08:48:04.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.031+0000 7faa337fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa3410c520 0x7faa3419d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:04.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.031+0000 7faa337fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa3410c520 0x7faa3419d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48312/0 (socket says 172.21.15.86:48312) 2024-04-21T08:48:04.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.031+0000 7faa337fe640 1 -- 172.21.15.86:0/1593401390 learned_addr learned my addr 172.21.15.86:0/1593401390 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:04.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.031+0000 7faa327fc640 1 --2- 172.21.15.86:0/1593401390 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa34103b00 0x7faa34198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:04.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.031+0000 7faa32ffd640 1 --2- 172.21.15.86:0/1593401390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa341031e0 0x7faa34198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:04.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.031+0000 7faa337fe640 1 -- 172.21.15.86:0/1593401390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa341031e0 msgr2=0x7faa34198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:04.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.031+0000 7faa337fe640 1 --2- 172.21.15.86:0/1593401390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa341031e0 0x7faa34198750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.033 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.031+0000 7faa337fe640 1 -- 172.21.15.86:0/1593401390 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa34103b00 msgr2=0x7faa34198c90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:04.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.031+0000 7faa337fe640 1 --2- 172.21.15.86:0/1593401390 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa34103b00 0x7faa34198c90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.031+0000 7faa337fe640 1 -- 172.21.15.86:0/1593401390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa3419d740 con 0x7faa3410c520 2024-04-21T08:48:04.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.031+0000 7faa32ffd640 1 --2- 172.21.15.86:0/1593401390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa341031e0 0x7faa34198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:04.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.031+0000 7faa337fe640 1 --2- 172.21.15.86:0/1593401390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa3410c520 0x7faa3419d010 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7faa2c02c840 tx=0x7faa2c035ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:04.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.031+0000 7faa1bfff640 1 -- 172.21.15.86:0/1593401390 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2c039040 con 0x7faa3410c520 2024-04-21T08:48:04.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.031+0000 7faa1bfff640 1 -- 172.21.15.86:0/1593401390 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa2c044070 con 0x7faa3410c520 2024-04-21T08:48:04.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.031+0000 7faa3932b640 1 -- 172.21.15.86:0/1593401390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa3419d9d0 con 0x7faa3410c520 2024-04-21T08:48:04.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.031+0000 7faa1bfff640 1 -- 172.21.15.86:0/1593401390 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2c032880 con 0x7faa3410c520 2024-04-21T08:48:04.034 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.032+0000 7faa3932b640 1 -- 172.21.15.86:0/1593401390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa341a52c0 con 0x7faa3410c520 2024-04-21T08:48:04.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.034+0000 7faa1bfff640 1 -- 172.21.15.86:0/1593401390 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faa2c032a20 con 0x7faa3410c520 2024-04-21T08:48:04.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.034+0000 7faa3bffa740 1 Processor -- start 2024-04-21T08:48:04.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.034+0000 7faa3bffa740 1 -- start start 2024-04-21T08:48:04.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.034+0000 7faa3bffa740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5584e7bf14d0 0x5584e7bf18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:04.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.034+0000 7faa3bffa740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584e7bf1e80 0x5584e7bfa430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:04.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.034+0000 7faa3bffa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584e7bfaa00 0x5584e7bfce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:04.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.034+0000 7faa3bffa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5584e7b561f0 con 0x5584e7bfaa00 2024-04-21T08:48:04.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.034+0000 7faa3bffa740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5584e7af34b0 con 0x5584e7bf14d0 2024-04-21T08:48:04.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.034+0000 7faa3bffa740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5584e7b521b0 con 0x5584e7bf1e80 2024-04-21T08:48:04.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.035+0000 7faa1bfff640 1 --2- 172.21.15.86:0/1593401390 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7faa1c0a3d20 0x7faa1c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:04.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.035+0000 7faa327fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584e7bfaa00 0x5584e7bfce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:04.041 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.035+0000 7faa327fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584e7bfaa00 0x5584e7bfce50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48324/0 (socket says 172.21.15.86:48324) 2024-04-21T08:48:04.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.035+0000 7faa327fc640 1 -- 172.21.15.86:0/3484278050 learned_addr learned my addr 172.21.15.86:0/3484278050 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:04.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.035+0000 7faa327fc640 1 --2- 172.21.15.86:0/3484278050 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5584e7bf14d0 0x5584e7bf18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:04.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.035+0000 7faa337fe640 1 --2- 172.21.15.86:0/1593401390 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7faa1c0a3d20 0x7faa1c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:04.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.035+0000 7faa32ffd640 1 --2- 172.21.15.86:0/3484278050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584e7bf1e80 0x5584e7bfa430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:04.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.035+0000 7faa327fc640 1 -- 172.21.15.86:0/3484278050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584e7bf1e80 msgr2=0x5584e7bfa430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:04.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.035+0000 7faa1bfff640 1 -- 172.21.15.86:0/1593401390 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7faa2c0f2fb0 con 0x7faa3410c520 2024-04-21T08:48:04.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.035+0000 7faa327fc640 1 --2- 172.21.15.86:0/3484278050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584e7bf1e80 0x5584e7bfa430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.035+0000 7faa327fc640 1 -- 172.21.15.86:0/3484278050 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5584e7bf14d0 msgr2=0x5584e7bf18b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:04.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.035+0000 7faa327fc640 1 --2- 172.21.15.86:0/3484278050 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5584e7bf14d0 0x5584e7bf18b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.042 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.035+0000 7faa327fc640 1 -- 172.21.15.86:0/3484278050 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584e7ae51b0 con 0x5584e7bfaa00 2024-04-21T08:48:04.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.035+0000 7faa327fc640 1 --2- 172.21.15.86:0/3484278050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584e7bfaa00 0x5584e7bfce50 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7faa28012720 tx=0x7faa2800b6e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b751fce5f21f95b7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:04.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.035+0000 7faa337fe640 1 --2- 172.21.15.86:0/1593401390 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7faa1c0a3d20 0x7faa1c0a61e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7faa2c0bb600 tx=0x7faa2c0a9760 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:04.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.035+0000 7faa197fa640 1 -- 172.21.15.86:0/3484278050 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa28012ca0 con 0x5584e7bfaa00 2024-04-21T08:48:04.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.035+0000 7faa197fa640 1 -- 172.21.15.86:0/3484278050 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa28012e40 con 0x5584e7bfaa00 2024-04-21T08:48:04.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.035+0000 7faa197fa640 1 -- 172.21.15.86:0/3484278050 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa28004a00 con 0x5584e7bfaa00 2024-04-21T08:48:04.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.036+0000 7faa3bffa740 1 -- 172.21.15.86:0/3484278050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584e7bfaa00 msgr2=0x5584e7bfce50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:04.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.036+0000 7faa3bffa740 1 --2- 172.21.15.86:0/3484278050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584e7bfaa00 0x5584e7bfce50 secure :-1 s=READY pgs=1715 cs=0 l=1 rev1=1 crypto rx=0x7faa28012720 tx=0x7faa2800b6e0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.036+0000 7faa3bffa740 1 -- 172.21.15.86:0/3484278050 shutdown_connections 2024-04-21T08:48:04.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.036+0000 7faa3bffa740 1 --2- 172.21.15.86:0/3484278050 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584e7bfaa00 0x5584e7bfce50 unknown :-1 s=CLOSED pgs=1715 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.036+0000 7faa3bffa740 1 --2- 172.21.15.86:0/3484278050 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584e7bf1e80 0x5584e7bfa430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.036+0000 7faa3bffa740 1 --2- 172.21.15.86:0/3484278050 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5584e7bf14d0 0x5584e7bf18b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.036+0000 7faa3bffa740 1 -- 172.21.15.86:0/3484278050 >> 172.21.15.86:0/3484278050 conn(0x5584e7bebba0 msgr2=0x5584e7bebf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:04.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.036+0000 7faa3bffa740 1 -- 172.21.15.86:0/3484278050 shutdown_connections 2024-04-21T08:48:04.043 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.036+0000 7faa3bffa740 1 -- 172.21.15.86:0/3484278050 wait complete. 2024-04-21T08:48:04.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.036+0000 7faa3bffa740 1 -- start start 2024-04-21T08:48:04.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.036+0000 7faa3bffa740 1 Processor -- start 2024-04-21T08:48:04.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.036+0000 7faa3bffa740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5584e7bf14d0 0x5584e7bfa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:04.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.036+0000 7faa3bffa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584e7bf1e80 0x5584e7bf7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:04.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.036+0000 7faa3bffa740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584e7bfaa00 0x5584e7bf7b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:04.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.037+0000 7faa3bffa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5584e7bfd970 con 0x5584e7bf1e80 2024-04-21T08:48:04.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.037+0000 7faa3bffa740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5584e7b0f500 con 0x5584e7bf14d0 2024-04-21T08:48:04.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.037+0000 7faa3bffa740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5584e7bfdaf0 con 0x5584e7bfaa00 2024-04-21T08:48:04.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.037+0000 7faa3bffa740 10 client.0 _ll_register_callbacks cb 0x5584e7addb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:04.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.037+0000 7faa32ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584e7bf1e80 0x5584e7bf7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:04.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.037+0000 7faa32ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584e7bf1e80 0x5584e7bf7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48340/0 (socket says 172.21.15.86:48340) 2024-04-21T08:48:04.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.037+0000 7faa32ffd640 1 -- 172.21.15.86:0/19021525 learned_addr learned my addr 172.21.15.86:0/19021525 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:04.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.037+0000 7faa327fc640 1 --2- 172.21.15.86:0/19021525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5584e7bf14d0 0x5584e7bfa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:04.044 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.037+0000 7faa32ffd640 1 -- 172.21.15.86:0/19021525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584e7bfaa00 msgr2=0x5584e7bf7b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:04.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.037+0000 7faa327fc640 1 --2- 172.21.15.86:0/19021525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584e7bfaa00 0x5584e7bf7b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:04.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.037+0000 7faa32ffd640 1 --2- 172.21.15.86:0/19021525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584e7bfaa00 0x5584e7bf7b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.037+0000 7faa32ffd640 1 -- 172.21.15.86:0/19021525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5584e7bf14d0 msgr2=0x5584e7bfa2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:04.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.037+0000 7faa32ffd640 1 --2- 172.21.15.86:0/19021525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5584e7bf14d0 0x5584e7bfa2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.037+0000 7faa32ffd640 1 -- 172.21.15.86:0/19021525 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584e78683f0 con 0x5584e7bf1e80 2024-04-21T08:48:04.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.037+0000 7faa32ffd640 1 --2- 172.21.15.86:0/19021525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584e7bf1e80 0x5584e7bf7640 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7faa24013790 tx=0x7faa24013c50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=225624ac01ab443d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:04.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.037+0000 7faa16ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:04.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.037+0000 7faa16ff5640 1 -- 172.21.15.86:0/19021525 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa24011dd0 con 0x5584e7bf1e80 2024-04-21T08:48:04.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.037+0000 7faa16ff5640 1 -- 172.21.15.86:0/19021525 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7faa24004030 con 0x5584e7bf1e80 2024-04-21T08:48:04.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.037+0000 7faa3bffa740 1 -- 172.21.15.86:0/19021525 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x5584e7af5930 con 0x5584e7bf1e80 2024-04-21T08:48:04.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.037+0000 7faa16ff5640 1 -- 172.21.15.86:0/19021525 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faa24004230 con 0x5584e7bf1e80 2024-04-21T08:48:04.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.037+0000 7faa3bffa740 10 client.19362 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:04.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.037+0000 7faa3bffa740 1 -- 172.21.15.86:0/19021525 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5584e79327d0 con 0x5584e7bf1e80 2024-04-21T08:48:04.045 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.038+0000 7faa16ff5640 1 -- 172.21.15.86:0/19021525 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x5584e7af5930 con 0x5584e7bf1e80 2024-04-21T08:48:04.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.038+0000 7faa16ff5640 1 -- 172.21.15.86:0/19021525 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7faa2401a070 con 0x5584e7bf1e80 2024-04-21T08:48:04.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.038+0000 7faa3bffa740 10 client.19362 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:04.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.038+0000 7faa3bffa740 10 client.19362 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:48:04.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.038+0000 7faa3bffa740 20 client.19362 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:04.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.038+0000 7faa3bffa740 1 --2- 172.21.15.86:0/19021525 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5584e7c0ad10 0x5584e7c0d150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:04.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.038+0000 7faa3bffa740 4 client.19362 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:04.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.038+0000 7faa3bffa740 1 -- 172.21.15.86:0/19021525 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5584e7b63c10 con 0x5584e7c0ad10 2024-04-21T08:48:04.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.038+0000 7faa327fc640 1 --2- 172.21.15.86:0/19021525 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5584e7c0ad10 0x5584e7c0d150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:04.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.038+0000 7faa327fc640 1 --2- 172.21.15.86:0/19021525 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5584e7c0ad10 0x5584e7c0d150 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b70c9b4bdd1f90ec server_cookie=3c5a113388d71497 in_seq=0 out_seq=0 2024-04-21T08:48:04.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.038+0000 7faa16ff5640 10 client.19362 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:48:04.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.039+0000 7faa16ff5640 1 -- 172.21.15.86:0/19021525 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5584e7b63c10 con 0x5584e7c0ad10 2024-04-21T08:48:04.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.039+0000 7faa16ff5640 10 client.19362 handle_command_reply: tid=0 2024-04-21T08:48:04.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.039+0000 7faa3bffa740 1 client.19362 shutdown 2024-04-21T08:48:04.046 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.039+0000 7faa3bffa740 1 -- 172.21.15.86:0/19021525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584e7bf1e80 msgr2=0x5584e7bf7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:04.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.039+0000 7faa3bffa740 1 --2- 172.21.15.86:0/19021525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584e7bf1e80 0x5584e7bf7640 secure :-1 s=READY pgs=1716 cs=0 l=1 rev1=1 crypto rx=0x7faa24013790 tx=0x7faa24013c50 comp rx=0 tx=0).stop 2024-04-21T08:48:04.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.039+0000 7faa3bffa740 1 -- 172.21.15.86:0/19021525 shutdown_connections 2024-04-21T08:48:04.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.039+0000 7faa3bffa740 1 --2- 172.21.15.86:0/19021525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584e7bfaa00 0x5584e7bf7b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.039+0000 7faa3bffa740 1 --2- 172.21.15.86:0/19021525 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5584e7c0ad10 0x5584e7c0d150 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.039+0000 7faa3bffa740 1 --2- 172.21.15.86:0/19021525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584e7bf1e80 0x5584e7bf7640 unknown :-1 s=CLOSED pgs=1716 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.039+0000 7faa3bffa740 1 --2- 172.21.15.86:0/19021525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5584e7bf14d0 0x5584e7bfa2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.039+0000 7faa16ff5640 0 client.19362 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:48:04.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.039+0000 7faa3bffa740 1 -- 172.21.15.86:0/19021525 >> 172.21.15.86:0/19021525 conn(0x5584e7bebba0 msgr2=0x5584e7bed8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:04.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.039+0000 7faa3bffa740 1 -- 172.21.15.86:0/19021525 shutdown_connections 2024-04-21T08:48:04.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.039+0000 7faa3bffa740 1 -- 172.21.15.86:0/19021525 wait complete. 2024-04-21T08:48:04.047 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.039+0000 7faa3bffa740 20 client.19362 trim_cache size 0 max 16384 2024-04-21T08:48:04.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.052+0000 7faa3bffa740 1 Processor -- start 2024-04-21T08:48:04.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.052+0000 7faa3bffa740 1 -- start start 2024-04-21T08:48:04.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.053+0000 7faa3bffa740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584e7bf14d0 0x5584e7c06890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:04.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.053+0000 7faa3bffa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584e7bf1e80 0x5584e7bf9d70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:04.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.053+0000 7faa3bffa740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5584e7bfaa00 0x5584e7bfa2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:04.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.053+0000 7faa3bffa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa341af410 con 0x5584e7bf1e80 2024-04-21T08:48:04.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.053+0000 7faa3bffa740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faa341b0f50 con 0x5584e7bfaa00 2024-04-21T08:48:04.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.053+0000 7faa3bffa740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faa341afae0 con 0x5584e7bf14d0 2024-04-21T08:48:04.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.053+0000 7faa32ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584e7bf1e80 0x5584e7bf9d70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:04.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.053+0000 7faa32ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584e7bf1e80 0x5584e7bf9d70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48342/0 (socket says 172.21.15.86:48342) 2024-04-21T08:48:04.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.053+0000 7faa32ffd640 1 -- 172.21.15.86:0/3658016554 learned_addr learned my addr 172.21.15.86:0/3658016554 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:04.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.053+0000 7faa327fc640 1 --2- 172.21.15.86:0/3658016554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584e7bf14d0 0x5584e7c06890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:04.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.053+0000 7faa327fc640 1 --2- 172.21.15.86:0/3658016554 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5584e7bfaa00 0x5584e7bfa2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:04.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.053+0000 7faa32ffd640 1 -- 172.21.15.86:0/3658016554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584e7bf14d0 msgr2=0x5584e7c06890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:04.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.053+0000 7faa32ffd640 1 --2- 172.21.15.86:0/3658016554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584e7bf14d0 0x5584e7c06890 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.053+0000 7faa32ffd640 1 -- 172.21.15.86:0/3658016554 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5584e7bfaa00 msgr2=0x5584e7bfa2b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:04.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.053+0000 7faa32ffd640 1 --2- 172.21.15.86:0/3658016554 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5584e7bfaa00 0x5584e7bfa2b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.053+0000 7faa32ffd640 1 -- 172.21.15.86:0/3658016554 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5584e7bf3270 con 0x5584e7bf1e80 2024-04-21T08:48:04.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.053+0000 7faa327fc640 1 --2- 172.21.15.86:0/3658016554 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5584e7bfaa00 0x5584e7bfa2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:04.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.053+0000 7faa32ffd640 1 --2- 172.21.15.86:0/3658016554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584e7bf1e80 0x5584e7bf9d70 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7faa2400f5f0 tx=0x7faa2400f450 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7e1aa7e417a03f26 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:04.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.053+0000 7faa18ff9640 1 -- 172.21.15.86:0/3658016554 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2402e020 con 0x5584e7bf1e80 2024-04-21T08:48:04.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.053+0000 7faa18ff9640 1 -- 172.21.15.86:0/3658016554 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa2400f860 con 0x5584e7bf1e80 2024-04-21T08:48:04.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.053+0000 7faa18ff9640 1 -- 172.21.15.86:0/3658016554 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2402ab90 con 0x5584e7bf1e80 2024-04-21T08:48:04.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.054+0000 7faa3bffa740 1 -- 172.21.15.86:0/3658016554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584e7bf1e80 msgr2=0x5584e7bf9d70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:04.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.054+0000 7faa3bffa740 1 --2- 172.21.15.86:0/3658016554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584e7bf1e80 0x5584e7bf9d70 secure :-1 s=READY pgs=1717 cs=0 l=1 rev1=1 crypto rx=0x7faa2400f5f0 tx=0x7faa2400f450 comp rx=0 tx=0).stop 2024-04-21T08:48:04.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.054+0000 7faa3bffa740 1 -- 172.21.15.86:0/3658016554 shutdown_connections 2024-04-21T08:48:04.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.054+0000 7faa3bffa740 1 --2- 172.21.15.86:0/3658016554 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5584e7bfaa00 0x5584e7bfa2b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.054+0000 7faa3bffa740 1 --2- 172.21.15.86:0/3658016554 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584e7bf1e80 0x5584e7bf9d70 unknown :-1 s=CLOSED pgs=1717 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.054+0000 7faa3bffa740 1 --2- 172.21.15.86:0/3658016554 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584e7bf14d0 0x5584e7c06890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.054+0000 7faa3bffa740 1 -- 172.21.15.86:0/3658016554 >> 172.21.15.86:0/3658016554 conn(0x5584e7bebba0 msgr2=0x5584e7beac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:04.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.054+0000 7faa3bffa740 1 -- 172.21.15.86:0/3658016554 shutdown_connections 2024-04-21T08:48:04.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.054+0000 7faa3bffa740 1 -- 172.21.15.86:0/3658016554 wait complete. 2024-04-21T08:48:04.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.054+0000 7faa3bffa740 1 -- start start 2024-04-21T08:48:04.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.054+0000 7faa3bffa740 1 Processor -- start 2024-04-21T08:48:04.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.054+0000 7faa3bffa740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5584e7bf14d0 0x5584e7c02f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:04.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.054+0000 7faa3bffa740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584e7bf1e80 0x5584e7bf3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:04.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.054+0000 7faa3bffa740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584e7bfaa00 0x5584e7bf43b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:04.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.054+0000 7faa3bffa740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa341b0880 con 0x5584e7bf1e80 2024-04-21T08:48:04.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.054+0000 7faa3bffa740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faa341b01b0 con 0x5584e7bf14d0 2024-04-21T08:48:04.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.054+0000 7faa3bffa740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faa341ad410 con 0x5584e7bfaa00 2024-04-21T08:48:04.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.054+0000 7faa3bffa740 10 client.0 _ll_register_callbacks cb 0x7faa24009d80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:04.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.054+0000 7faa32ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584e7bf1e80 0x5584e7bf3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:04.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.055+0000 7faa32ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584e7bf1e80 0x5584e7bf3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48352/0 (socket says 172.21.15.86:48352) 2024-04-21T08:48:04.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.055+0000 7faa32ffd640 1 -- 172.21.15.86:0/769416188 learned_addr learned my addr 172.21.15.86:0/769416188 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:04.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.055+0000 7faa327fc640 1 --2- 172.21.15.86:0/769416188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584e7bfaa00 0x5584e7bf43b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:04.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.055+0000 7faa32ffd640 1 -- 172.21.15.86:0/769416188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584e7bfaa00 msgr2=0x5584e7bf43b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:04.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.055+0000 7faa32ffd640 1 --2- 172.21.15.86:0/769416188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584e7bfaa00 0x5584e7bf43b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.055+0000 7faa32ffd640 1 -- 172.21.15.86:0/769416188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5584e7bf14d0 msgr2=0x5584e7c02f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:04.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.055+0000 7faa32ffd640 1 --2- 172.21.15.86:0/769416188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5584e7bf14d0 0x5584e7c02f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.055+0000 7faa32ffd640 1 -- 172.21.15.86:0/769416188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5584e7bf4920 con 0x5584e7bf1e80 2024-04-21T08:48:04.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.055+0000 7faa32ffd640 1 --2- 172.21.15.86:0/769416188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584e7bf1e80 0x5584e7bf3e70 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7faa24009660 tx=0x7faa24011860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86d477f8b2d12d2d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:04.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.055+0000 7faa16ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:04.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.055+0000 7faa16ff5640 1 -- 172.21.15.86:0/769416188 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa2402e030 con 0x5584e7bf1e80 2024-04-21T08:48:04.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.055+0000 7faa16ff5640 1 -- 172.21.15.86:0/769416188 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7faa2402a680 con 0x5584e7bf1e80 2024-04-21T08:48:04.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.055+0000 7faa3bffa740 1 -- 172.21.15.86:0/769416188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa2400f860 con 0x5584e7bf1e80 2024-04-21T08:48:04.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.055+0000 7faa16ff5640 1 -- 172.21.15.86:0/769416188 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faa2400fe60 con 0x5584e7bf1e80 2024-04-21T08:48:04.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.055+0000 7faa3bffa740 10 client.19368 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:04.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.055+0000 7faa3bffa740 1 -- 172.21.15.86:0/769416188 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5584e7c70f80 con 0x5584e7bf1e80 2024-04-21T08:48:04.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.055+0000 7faa16ff5640 1 -- 172.21.15.86:0/769416188 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa2400f860 con 0x5584e7bf1e80 2024-04-21T08:48:04.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.055+0000 7faa16ff5640 1 -- 172.21.15.86:0/769416188 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7faa2401a040 con 0x5584e7bf1e80 2024-04-21T08:48:04.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.055+0000 7faa3bffa740 10 client.19368 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:04.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.055+0000 7faa3bffa740 10 client.19368 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:48:04.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.055+0000 7faa3bffa740 20 client.19368 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:04.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.055+0000 7faa3bffa740 1 --2- 172.21.15.86:0/769416188 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5584e7c715a0 0x5584e7c739c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:04.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.055+0000 7faa3bffa740 4 client.19368 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:48:04.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.055+0000 7faa3bffa740 1 -- 172.21.15.86:0/769416188 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5584e7b5da80 con 0x5584e7c715a0 2024-04-21T08:48:04.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.056+0000 7faa327fc640 1 --2- 172.21.15.86:0/769416188 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5584e7c715a0 0x5584e7c739c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:04.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.056+0000 7faa16ff5640 10 client.19368 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:48:04.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.056+0000 7faa327fc640 1 --2- 172.21.15.86:0/769416188 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5584e7c715a0 0x5584e7c739c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b289867cfb959b75 server_cookie=d60982a53c9de061 in_seq=0 out_seq=0 2024-04-21T08:48:04.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.056+0000 7faa16ff5640 1 -- 172.21.15.86:0/769416188 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5584e7b5da80 con 0x5584e7c715a0 2024-04-21T08:48:04.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.056+0000 7faa16ff5640 10 client.19368 handle_command_reply: tid=0 2024-04-21T08:48:04.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.056+0000 7faa3bffa740 1 client.19368 shutdown 2024-04-21T08:48:04.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.056+0000 7faa3bffa740 1 -- 172.21.15.86:0/769416188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584e7bf1e80 msgr2=0x5584e7bf3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:04.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.056+0000 7faa3bffa740 1 --2- 172.21.15.86:0/769416188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584e7bf1e80 0x5584e7bf3e70 secure :-1 s=READY pgs=1718 cs=0 l=1 rev1=1 crypto rx=0x7faa24009660 tx=0x7faa24011860 comp rx=0 tx=0).stop 2024-04-21T08:48:04.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.056+0000 7faa3bffa740 1 -- 172.21.15.86:0/769416188 shutdown_connections 2024-04-21T08:48:04.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.056+0000 7faa3bffa740 1 --2- 172.21.15.86:0/769416188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5584e7bfaa00 0x5584e7bf43b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.056+0000 7faa3bffa740 1 --2- 172.21.15.86:0/769416188 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5584e7c715a0 0x5584e7c739c0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.056+0000 7faa3bffa740 1 --2- 172.21.15.86:0/769416188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5584e7bf1e80 0x5584e7bf3e70 unknown :-1 s=CLOSED pgs=1718 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.056+0000 7faa16ff5640 0 client.19368 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:48:04.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.056+0000 7faa3bffa740 1 --2- 172.21.15.86:0/769416188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5584e7bf14d0 0x5584e7c02f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.056+0000 7faa3bffa740 1 -- 172.21.15.86:0/769416188 >> 172.21.15.86:0/769416188 conn(0x5584e7bebba0 msgr2=0x5584e7bf2260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:04.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.056+0000 7faa3bffa740 1 -- 172.21.15.86:0/769416188 shutdown_connections 2024-04-21T08:48:04.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.057+0000 7faa3bffa740 1 -- 172.21.15.86:0/769416188 wait complete. 2024-04-21T08:48:04.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.057+0000 7faa3bffa740 20 client.19368 trim_cache size 0 max 16384 2024-04-21T08:48:04.063 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:48:04.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.057+0000 7faa3932b640 1 -- 172.21.15.86:0/1593401390 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7faa1c0a3d20 msgr2=0x7faa1c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:04.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.057+0000 7faa3932b640 1 --2- 172.21.15.86:0/1593401390 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7faa1c0a3d20 0x7faa1c0a61e0 secure :-1 s=READY pgs=281 cs=0 l=1 rev1=1 crypto rx=0x7faa2c0bb600 tx=0x7faa2c0a9760 comp rx=0 tx=0).stop 2024-04-21T08:48:04.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.057+0000 7faa3932b640 1 -- 172.21.15.86:0/1593401390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa3410c520 msgr2=0x7faa3419d010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:04.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.057+0000 7faa3932b640 1 --2- 172.21.15.86:0/1593401390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa3410c520 0x7faa3419d010 secure :-1 s=READY pgs=1714 cs=0 l=1 rev1=1 crypto rx=0x7faa2c02c840 tx=0x7faa2c035ea0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.057+0000 7faa3932b640 1 -- 172.21.15.86:0/1593401390 shutdown_connections 2024-04-21T08:48:04.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.057+0000 7faa3932b640 1 --2- 172.21.15.86:0/1593401390 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7faa1c0a3d20 0x7faa1c0a61e0 unknown :-1 s=CLOSED pgs=281 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.057+0000 7faa3932b640 1 --2- 172.21.15.86:0/1593401390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa3410c520 0x7faa3419d010 unknown :-1 s=CLOSED pgs=1714 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.064 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.057+0000 7faa3932b640 1 --2- 172.21.15.86:0/1593401390 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa34103b00 0x7faa34198c90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.057+0000 7faa3932b640 1 --2- 172.21.15.86:0/1593401390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa341031e0 0x7faa34198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.057+0000 7faa3932b640 1 -- 172.21.15.86:0/1593401390 >> 172.21.15.86:0/1593401390 conn(0x7faa340771d0 msgr2=0x7faa340ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:04.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.057+0000 7faa3932b640 1 -- 172.21.15.86:0/1593401390 shutdown_connections 2024-04-21T08:48:04.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.057+0000 7faa3932b640 1 -- 172.21.15.86:0/1593401390 wait complete. 2024-04-21T08:48:04.066 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:03 smithi086 ceph-mon[27939]: pgmap v287: 129 pgs: 129 active+clean; 5.2 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 105 KiB/s wr, 39 op/s 2024-04-21T08:48:04.066 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:03 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3824735196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:48:04.067 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:48:04.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.167+0000 7f9453596640 1 -- 172.21.15.86:0/1664072222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f944c1010a0 msgr2=0x7f944c101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:04.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.167+0000 7f9453596640 1 --2- 172.21.15.86:0/1664072222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f944c1010a0 0x7f944c101480 secure :-1 s=READY pgs=1719 cs=0 l=1 rev1=1 crypto rx=0x7f9448009ca0 tx=0x7f944802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.167+0000 7f9453596640 1 -- 172.21.15.86:0/1664072222 shutdown_connections 2024-04-21T08:48:04.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.167+0000 7f9453596640 1 --2- 172.21.15.86:0/1664072222 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f944c106170 0x7f944c10dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.167+0000 7f9453596640 1 --2- 172.21.15.86:0/1664072222 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f944c101a50 0x7f944c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.169 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.167+0000 7f9453596640 1 --2- 172.21.15.86:0/1664072222 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f944c1010a0 0x7f944c101480 unknown :-1 s=CLOSED pgs=1719 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.167+0000 7f9453596640 1 -- 172.21.15.86:0/1664072222 >> 172.21.15.86:0/1664072222 conn(0x7f944c077eb0 msgr2=0x7f944c0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:04.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.167+0000 7f9453596640 1 -- 172.21.15.86:0/1664072222 shutdown_connections 2024-04-21T08:48:04.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.167+0000 7f9453596640 1 -- 172.21.15.86:0/1664072222 wait complete. 2024-04-21T08:48:04.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.168+0000 7f9453596640 1 Processor -- start 2024-04-21T08:48:04.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.168+0000 7f9453596640 1 -- start start 2024-04-21T08:48:04.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.168+0000 7f9453596640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f944c1010a0 0x7f944c19e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:04.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.168+0000 7f9453596640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f944c106170 0x7f944c19e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:04.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.168+0000 7f9453596640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f944c1985a0 0x7f944c198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:04.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.168+0000 7f9453596640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f944c1110d0 con 0x7f944c1010a0 2024-04-21T08:48:04.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.168+0000 7f9453596640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f944c110f50 con 0x7f944c1985a0 2024-04-21T08:48:04.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.168+0000 7f9453596640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f944c111250 con 0x7f944c106170 2024-04-21T08:48:04.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.168+0000 7f9451332640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f944c1010a0 0x7f944c19e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:04.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.168+0000 7f9451332640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f944c1010a0 0x7f944c19e3f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48358/0 (socket says 172.21.15.86:48358) 2024-04-21T08:48:04.170 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.168+0000 7f9451332640 1 -- 172.21.15.86:0/2466291164 learned_addr learned my addr 172.21.15.86:0/2466291164 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:04.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.168+0000 7f9451b33640 1 --2- 172.21.15.86:0/2466291164 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f944c1985a0 0x7f944c198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:04.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.168+0000 7f9450b31640 1 --2- 172.21.15.86:0/2466291164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f944c106170 0x7f944c19e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:04.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.168+0000 7f9451332640 1 -- 172.21.15.86:0/2466291164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f944c106170 msgr2=0x7f944c19e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:04.171 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.168+0000 7f9451332640 1 --2- 172.21.15.86:0/2466291164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f944c106170 0x7f944c19e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.168+0000 7f9451332640 1 -- 172.21.15.86:0/2466291164 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f944c1985a0 msgr2=0x7f944c198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:04.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.168+0000 7f9451332640 1 --2- 172.21.15.86:0/2466291164 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f944c1985a0 0x7f944c198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.168+0000 7f9451332640 1 -- 172.21.15.86:0/2466291164 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f944c1990c0 con 0x7f944c1010a0 2024-04-21T08:48:04.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.168+0000 7f9450b31640 1 --2- 172.21.15.86:0/2466291164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f944c106170 0x7f944c19e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:04.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.168+0000 7f9451b33640 1 --2- 172.21.15.86:0/2466291164 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f944c1985a0 0x7f944c198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:04.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.169+0000 7f9451332640 1 --2- 172.21.15.86:0/2466291164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f944c1010a0 0x7f944c19e3f0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f9448005e30 tx=0x7f9448034780 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:04.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.169+0000 7f94427fc640 1 -- 172.21.15.86:0/2466291164 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9448039040 con 0x7f944c1010a0 2024-04-21T08:48:04.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.169+0000 7f94427fc640 1 -- 172.21.15.86:0/2466291164 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9448044070 con 0x7f944c1010a0 2024-04-21T08:48:04.173 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.169+0000 7f9453596640 1 -- 172.21.15.86:0/2466291164 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f944c199350 con 0x7f944c1010a0 2024-04-21T08:48:04.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.169+0000 7f94427fc640 1 -- 172.21.15.86:0/2466291164 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9448003e60 con 0x7f944c1010a0 2024-04-21T08:48:04.174 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.169+0000 7f9453596640 1 -- 172.21.15.86:0/2466291164 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f944c1a55a0 con 0x7f944c1010a0 2024-04-21T08:48:04.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.171+0000 7f94427fc640 1 -- 172.21.15.86:0/2466291164 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f94480324c0 con 0x7f944c1010a0 2024-04-21T08:48:04.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.172+0000 7f9456186740 1 Processor -- start 2024-04-21T08:48:04.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.172+0000 7f9456186740 1 -- start start 2024-04-21T08:48:04.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.172+0000 7f9456186740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557c66c04d0 0x5557c66c08b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:04.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.172+0000 7f9456186740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557c66c0e80 0x5557c66c9430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:04.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.172+0000 7f9456186740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c66c9a00 0x5557c66cbe50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:04.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.172+0000 7f9456186740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5557c66251f0 con 0x5557c66c9a00 2024-04-21T08:48:04.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.172+0000 7f9456186740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5557c65c24b0 con 0x5557c66c04d0 2024-04-21T08:48:04.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.172+0000 7f9456186740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5557c66211b0 con 0x5557c66c0e80 2024-04-21T08:48:04.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.172+0000 7f9450b31640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c66c9a00 0x5557c66cbe50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:04.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.172+0000 7f9451b33640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557c66c04d0 0x5557c66c08b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:04.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.172+0000 7f9450b31640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c66c9a00 0x5557c66cbe50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48368/0 (socket says 172.21.15.86:48368) 2024-04-21T08:48:04.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.172+0000 7f9450b31640 1 -- 172.21.15.86:0/2795256917 learned_addr learned my addr 172.21.15.86:0/2795256917 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:04.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.172+0000 7f9451332640 1 --2- 172.21.15.86:0/2795256917 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557c66c0e80 0x5557c66c9430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:04.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.172+0000 7f94427fc640 1 --2- 172.21.15.86:0/2466291164 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f94380a3da0 0x7f94380a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:04.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.172+0000 7f9451b33640 1 --2- 172.21.15.86:0/2466291164 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f94380a3da0 0x7f94380a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:04.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.172+0000 7f9450b31640 1 -- 172.21.15.86:0/2795256917 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557c66c0e80 msgr2=0x5557c66c9430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:04.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.172+0000 7f9450b31640 1 --2- 172.21.15.86:0/2795256917 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557c66c0e80 0x5557c66c9430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.172+0000 7f9450b31640 1 -- 172.21.15.86:0/2795256917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557c66c04d0 msgr2=0x5557c66c08b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:04.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.172+0000 7f9450b31640 1 --2- 172.21.15.86:0/2795256917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557c66c04d0 0x5557c66c08b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.172+0000 7f9450b31640 1 -- 172.21.15.86:0/2795256917 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557c65b41b0 con 0x5557c66c9a00 2024-04-21T08:48:04.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.172+0000 7f9451b33640 1 --2- 172.21.15.86:0/2795256917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557c66c04d0 0x5557c66c08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:04.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.172+0000 7f94427fc640 1 -- 172.21.15.86:0/2466291164 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f94480f2e60 con 0x7f944c1010a0 2024-04-21T08:48:04.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.173+0000 7f9450b31640 1 --2- 172.21.15.86:0/2795256917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c66c9a00 0x5557c66cbe50 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f943c009bc0 tx=0x7f943c00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5f2229d57b49c065 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:04.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.173+0000 7f94377fe640 1 -- 172.21.15.86:0/2795256917 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943c016070 con 0x5557c66c9a00 2024-04-21T08:48:04.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.173+0000 7f9451b33640 1 --2- 172.21.15.86:0/2466291164 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f94380a3da0 0x7f94380a6260 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9444005fd0 tx=0x7f9444004730 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:04.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.173+0000 7f94377fe640 1 -- 172.21.15.86:0/2795256917 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f943c011920 con 0x5557c66c9a00 2024-04-21T08:48:04.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.173+0000 7f94377fe640 1 -- 172.21.15.86:0/2795256917 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f943c010870 con 0x5557c66c9a00 2024-04-21T08:48:04.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.173+0000 7f9456186740 1 -- 172.21.15.86:0/2795256917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c66c9a00 msgr2=0x5557c66cbe50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:04.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.173+0000 7f9456186740 1 --2- 172.21.15.86:0/2795256917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c66c9a00 0x5557c66cbe50 secure :-1 s=READY pgs=1721 cs=0 l=1 rev1=1 crypto rx=0x7f943c009bc0 tx=0x7f943c00e6c0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.173+0000 7f9456186740 1 -- 172.21.15.86:0/2795256917 shutdown_connections 2024-04-21T08:48:04.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.173+0000 7f9456186740 1 --2- 172.21.15.86:0/2795256917 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c66c9a00 0x5557c66cbe50 unknown :-1 s=CLOSED pgs=1721 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.173+0000 7f9456186740 1 --2- 172.21.15.86:0/2795256917 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557c66c0e80 0x5557c66c9430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.173+0000 7f9456186740 1 --2- 172.21.15.86:0/2795256917 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557c66c04d0 0x5557c66c08b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.173+0000 7f9456186740 1 -- 172.21.15.86:0/2795256917 >> 172.21.15.86:0/2795256917 conn(0x5557c66baba0 msgr2=0x5557c66baf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:04.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.174+0000 7f9456186740 1 -- 172.21.15.86:0/2795256917 shutdown_connections 2024-04-21T08:48:04.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.174+0000 7f9456186740 1 -- 172.21.15.86:0/2795256917 wait complete. 2024-04-21T08:48:04.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.174+0000 7f9456186740 1 -- start start 2024-04-21T08:48:04.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.174+0000 7f9456186740 1 Processor -- start 2024-04-21T08:48:04.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.174+0000 7f9456186740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557c66c04d0 0x5557c66c92e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:04.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.174+0000 7f9456186740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557c66c0e80 0x5557c66c6670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:04.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.174+0000 7f9456186740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c66c9a00 0x5557c66c6bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:04.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.174+0000 7f9456186740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5557c66cc970 con 0x5557c66c9a00 2024-04-21T08:48:04.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.174+0000 7f9456186740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5557c65de500 con 0x5557c66c0e80 2024-04-21T08:48:04.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.174+0000 7f9456186740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5557c66ccaf0 con 0x5557c66c04d0 2024-04-21T08:48:04.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.174+0000 7f9456186740 10 client.0 _ll_register_callbacks cb 0x5557c65acb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:04.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.174+0000 7f9451b33640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c66c9a00 0x5557c66c6bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:04.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.174+0000 7f9451b33640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c66c9a00 0x5557c66c6bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48372/0 (socket says 172.21.15.86:48372) 2024-04-21T08:48:04.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.174+0000 7f9451b33640 1 -- 172.21.15.86:0/3885239748 learned_addr learned my addr 172.21.15.86:0/3885239748 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:04.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.175+0000 7f9450b31640 1 --2- 172.21.15.86:0/3885239748 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557c66c0e80 0x5557c66c6670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:04.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.175+0000 7f9451332640 1 --2- 172.21.15.86:0/3885239748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557c66c04d0 0x5557c66c92e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:04.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.175+0000 7f9451b33640 1 -- 172.21.15.86:0/3885239748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557c66c04d0 msgr2=0x5557c66c92e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:04.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.175+0000 7f9451b33640 1 --2- 172.21.15.86:0/3885239748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557c66c04d0 0x5557c66c92e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.175+0000 7f9451b33640 1 -- 172.21.15.86:0/3885239748 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557c66c0e80 msgr2=0x5557c66c6670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:04.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.175+0000 7f9451b33640 1 --2- 172.21.15.86:0/3885239748 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557c66c0e80 0x5557c66c6670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.175+0000 7f9451b33640 1 -- 172.21.15.86:0/3885239748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557c63373f0 con 0x5557c66c9a00 2024-04-21T08:48:04.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.175+0000 7f9451332640 1 --2- 172.21.15.86:0/3885239748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557c66c04d0 0x5557c66c92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:04.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.175+0000 7f9450b31640 1 --2- 172.21.15.86:0/3885239748 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557c66c0e80 0x5557c66c6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:04.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.175+0000 7f9451b33640 1 --2- 172.21.15.86:0/3885239748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c66c9a00 0x5557c66c6bd0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f94440132c0 tx=0x7f94440046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=981585787987a5e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:04.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.175+0000 7f9434ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:04.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.175+0000 7f9434ff9640 1 -- 172.21.15.86:0/3885239748 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9444019070 con 0x5557c66c9a00 2024-04-21T08:48:04.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.175+0000 7f9434ff9640 1 -- 172.21.15.86:0/3885239748 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f94440109d0 con 0x5557c66c9a00 2024-04-21T08:48:04.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.175+0000 7f9456186740 1 -- 172.21.15.86:0/3885239748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f943c011920 con 0x5557c66c9a00 2024-04-21T08:48:04.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.175+0000 7f9434ff9640 1 -- 172.21.15.86:0/3885239748 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9444010bd0 con 0x5557c66c9a00 2024-04-21T08:48:04.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.175+0000 7f9456186740 10 client.19380 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:04.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.175+0000 7f9456186740 1 -- 172.21.15.86:0/3885239748 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5557c64017d0 con 0x5557c66c9a00 2024-04-21T08:48:04.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.175+0000 7f9434ff9640 1 -- 172.21.15.86:0/3885239748 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f943c011920 con 0x5557c66c9a00 2024-04-21T08:48:04.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.176+0000 7f9434ff9640 1 -- 172.21.15.86:0/3885239748 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9444028400 con 0x5557c66c9a00 2024-04-21T08:48:04.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.176+0000 7f9456186740 10 client.19380 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:04.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.176+0000 7f9456186740 10 client.19380 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:04.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.176+0000 7f9456186740 20 client.19380 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:04.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.176+0000 7f9456186740 1 --2- 172.21.15.86:0/3885239748 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5557c66d9c30 0x5557c66dc0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:04.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.176+0000 7f9456186740 4 client.19380 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:04.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.176+0000 7f9456186740 1 -- 172.21.15.86:0/3885239748 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5557c65c4930 con 0x5557c66d9c30 2024-04-21T08:48:04.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.176+0000 7f9451332640 1 --2- 172.21.15.86:0/3885239748 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5557c66d9c30 0x5557c66dc0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:04.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.176+0000 7f9451332640 1 --2- 172.21.15.86:0/3885239748 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5557c66d9c30 0x5557c66dc0b0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1fac144fb6233144 server_cookie=9f77333797ace54e in_seq=0 out_seq=0 2024-04-21T08:48:04.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.176+0000 7f9434ff9640 10 client.19380 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:04.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.177+0000 7f9434ff9640 1 -- 172.21.15.86:0/3885239748 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5557c65c4930 con 0x5557c66d9c30 2024-04-21T08:48:04.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.177+0000 7f9434ff9640 10 client.19380 handle_command_reply: tid=0 2024-04-21T08:48:04.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.177+0000 7f9456186740 1 client.19380 shutdown 2024-04-21T08:48:04.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.177+0000 7f9456186740 1 -- 172.21.15.86:0/3885239748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c66c9a00 msgr2=0x5557c66c6bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:04.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.177+0000 7f9456186740 1 --2- 172.21.15.86:0/3885239748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c66c9a00 0x5557c66c6bd0 secure :-1 s=READY pgs=1722 cs=0 l=1 rev1=1 crypto rx=0x7f94440132c0 tx=0x7f94440046c0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.177+0000 7f9456186740 1 -- 172.21.15.86:0/3885239748 shutdown_connections 2024-04-21T08:48:04.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.177+0000 7f9456186740 1 --2- 172.21.15.86:0/3885239748 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5557c66d9c30 0x5557c66dc0b0 crc :-1 s=READY pgs=271 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.177+0000 7f9456186740 1 --2- 172.21.15.86:0/3885239748 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c66c9a00 0x5557c66c6bd0 unknown :-1 s=CLOSED pgs=1722 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.177+0000 7f9456186740 1 --2- 172.21.15.86:0/3885239748 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557c66c0e80 0x5557c66c6670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.177+0000 7f9434ff9640 0 client.19380 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:04.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.177+0000 7f9456186740 1 --2- 172.21.15.86:0/3885239748 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557c66c04d0 0x5557c66c92e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.177+0000 7f9456186740 1 -- 172.21.15.86:0/3885239748 >> 172.21.15.86:0/3885239748 conn(0x5557c66baba0 msgr2=0x5557c66bc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:04.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.177+0000 7f9456186740 1 -- 172.21.15.86:0/3885239748 shutdown_connections 2024-04-21T08:48:04.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.177+0000 7f9456186740 1 -- 172.21.15.86:0/3885239748 wait complete. 2024-04-21T08:48:04.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.177+0000 7f9456186740 20 client.19380 trim_cache size 0 max 16384 2024-04-21T08:48:04.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.190+0000 7f9456186740 1 Processor -- start 2024-04-21T08:48:04.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.190+0000 7f9456186740 1 -- start start 2024-04-21T08:48:04.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.190+0000 7f9456186740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557c66c04d0 0x5557c66d5830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:04.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.191+0000 7f9456186740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557c66c0e80 0x5557c66b9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:04.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.191+0000 7f9456186740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c66c9a00 0x5557c66c8da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:04.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.191+0000 7f9456186740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f944c1af350 con 0x5557c66c9a00 2024-04-21T08:48:04.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.191+0000 7f9456186740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f944c1b0e90 con 0x5557c66c04d0 2024-04-21T08:48:04.195 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.191+0000 7f9456186740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f944c1afa20 con 0x5557c66c0e80 2024-04-21T08:48:04.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.191+0000 7f9451b33640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c66c9a00 0x5557c66c8da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:04.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.191+0000 7f9451b33640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c66c9a00 0x5557c66c8da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48386/0 (socket says 172.21.15.86:48386) 2024-04-21T08:48:04.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.191+0000 7f9451b33640 1 -- 172.21.15.86:0/2210424441 learned_addr learned my addr 172.21.15.86:0/2210424441 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:04.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.191+0000 7f9451332640 1 --2- 172.21.15.86:0/2210424441 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557c66c04d0 0x5557c66d5830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:04.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.191+0000 7f9451b33640 1 -- 172.21.15.86:0/2210424441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557c66c0e80 msgr2=0x5557c66b9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:04.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.191+0000 7f9450b31640 1 --2- 172.21.15.86:0/2210424441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557c66c0e80 0x5557c66b9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:04.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.191+0000 7f9451b33640 1 --2- 172.21.15.86:0/2210424441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557c66c0e80 0x5557c66b9a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.191+0000 7f9451b33640 1 -- 172.21.15.86:0/2210424441 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557c66c04d0 msgr2=0x5557c66d5830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:04.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.191+0000 7f9451b33640 1 --2- 172.21.15.86:0/2210424441 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557c66c04d0 0x5557c66d5830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.191+0000 7f9451b33640 1 -- 172.21.15.86:0/2210424441 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5557c66bf7f0 con 0x5557c66c9a00 2024-04-21T08:48:04.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.191+0000 7f9451332640 1 --2- 172.21.15.86:0/2210424441 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557c66c04d0 0x5557c66d5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:04.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.191+0000 7f9451b33640 1 --2- 172.21.15.86:0/2210424441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c66c9a00 0x5557c66c8da0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f94440287f0 tx=0x7f944400d630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=397d17872133e1f0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:04.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.191+0000 7f9436ffd640 1 -- 172.21.15.86:0/2210424441 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9444019070 con 0x5557c66c9a00 2024-04-21T08:48:04.196 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.191+0000 7f9436ffd640 1 -- 172.21.15.86:0/2210424441 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f944403b070 con 0x5557c66c9a00 2024-04-21T08:48:04.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.191+0000 7f9436ffd640 1 -- 172.21.15.86:0/2210424441 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f94440132f0 con 0x5557c66c9a00 2024-04-21T08:48:04.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.192+0000 7f9456186740 1 -- 172.21.15.86:0/2210424441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c66c9a00 msgr2=0x5557c66c8da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:04.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.192+0000 7f9456186740 1 --2- 172.21.15.86:0/2210424441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c66c9a00 0x5557c66c8da0 secure :-1 s=READY pgs=1723 cs=0 l=1 rev1=1 crypto rx=0x7f94440287f0 tx=0x7f944400d630 comp rx=0 tx=0).stop 2024-04-21T08:48:04.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.192+0000 7f9456186740 1 -- 172.21.15.86:0/2210424441 shutdown_connections 2024-04-21T08:48:04.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.192+0000 7f9456186740 1 --2- 172.21.15.86:0/2210424441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c66c9a00 0x5557c66c8da0 unknown :-1 s=CLOSED pgs=1723 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.192+0000 7f9456186740 1 --2- 172.21.15.86:0/2210424441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557c66c0e80 0x5557c66b9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.192+0000 7f9456186740 1 --2- 172.21.15.86:0/2210424441 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557c66c04d0 0x5557c66d5830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.192+0000 7f9456186740 1 -- 172.21.15.86:0/2210424441 >> 172.21.15.86:0/2210424441 conn(0x5557c66baba0 msgr2=0x5557c66b9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:04.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.192+0000 7f9456186740 1 -- 172.21.15.86:0/2210424441 shutdown_connections 2024-04-21T08:48:04.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.192+0000 7f9456186740 1 -- 172.21.15.86:0/2210424441 wait complete. 2024-04-21T08:48:04.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.192+0000 7f9456186740 1 -- start start 2024-04-21T08:48:04.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.192+0000 7f9456186740 1 Processor -- start 2024-04-21T08:48:04.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.193+0000 7f9456186740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557c66c04d0 0x5557c66cd850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:04.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.193+0000 7f9456186740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557c66c0e80 0x5557c66cdd90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:04.197 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.193+0000 7f9456186740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c66c9a00 0x5557c66ce2d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:04.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.193+0000 7f9456186740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f944c1b07c0 con 0x5557c66c9a00 2024-04-21T08:48:04.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.193+0000 7f9456186740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f944c1b00f0 con 0x5557c66c0e80 2024-04-21T08:48:04.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.193+0000 7f9456186740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f944c1ad350 con 0x5557c66c04d0 2024-04-21T08:48:04.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.193+0000 7f9456186740 10 client.0 _ll_register_callbacks cb 0x7f9444013df0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:04.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.193+0000 7f9451b33640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c66c9a00 0x5557c66ce2d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:04.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.193+0000 7f9451b33640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c66c9a00 0x5557c66ce2d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48400/0 (socket says 172.21.15.86:48400) 2024-04-21T08:48:04.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.193+0000 7f9451b33640 1 -- 172.21.15.86:0/3045927608 learned_addr learned my addr 172.21.15.86:0/3045927608 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:04.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.193+0000 7f9451332640 1 --2- 172.21.15.86:0/3045927608 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557c66c04d0 0x5557c66cd850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:04.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.193+0000 7f9450b31640 1 --2- 172.21.15.86:0/3045927608 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557c66c0e80 0x5557c66cdd90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:04.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.193+0000 7f9451b33640 1 -- 172.21.15.86:0/3045927608 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557c66c04d0 msgr2=0x5557c66cd850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:04.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.193+0000 7f9451b33640 1 --2- 172.21.15.86:0/3045927608 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557c66c04d0 0x5557c66cd850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.193+0000 7f9451b33640 1 -- 172.21.15.86:0/3045927608 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557c66c0e80 msgr2=0x5557c66cdd90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:04.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.193+0000 7f9451b33640 1 --2- 172.21.15.86:0/3045927608 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557c66c0e80 0x5557c66cdd90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.198 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.193+0000 7f9451b33640 1 -- 172.21.15.86:0/3045927608 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5557c673f2f0 con 0x5557c66c9a00 2024-04-21T08:48:04.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.193+0000 7f9451332640 1 --2- 172.21.15.86:0/3045927608 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557c66c04d0 0x5557c66cd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:04.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.193+0000 7f9450b31640 1 --2- 172.21.15.86:0/3045927608 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557c66c0e80 0x5557c66cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:04.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.193+0000 7f9451b33640 1 --2- 172.21.15.86:0/3045927608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c66c9a00 0x5557c66ce2d0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f9444010a50 tx=0x7f9444010a80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f1290043b11b964 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:04.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.193+0000 7f9434ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:04.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.193+0000 7f9434ff9640 1 -- 172.21.15.86:0/3045927608 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9444019070 con 0x5557c66c9a00 2024-04-21T08:48:04.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.193+0000 7f9434ff9640 1 -- 172.21.15.86:0/3045927608 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(100..100 src has 1..100) ==== 9496+0+0 (secure 0 0 0) 0x7f94440134d0 con 0x5557c66c9a00 2024-04-21T08:48:04.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.193+0000 7f9456186740 1 -- 172.21.15.86:0/3045927608 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f944403b070 con 0x5557c66c9a00 2024-04-21T08:48:04.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.193+0000 7f9434ff9640 1 -- 172.21.15.86:0/3045927608 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9444010d70 con 0x5557c66c9a00 2024-04-21T08:48:04.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.193+0000 7f9456186740 10 client.19386 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:04.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.193+0000 7f9456186740 1 -- 172.21.15.86:0/3045927608 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5557c67405d0 con 0x5557c66c9a00 2024-04-21T08:48:04.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.194+0000 7f9434ff9640 1 -- 172.21.15.86:0/3045927608 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f944403b070 con 0x5557c66c9a00 2024-04-21T08:48:04.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.194+0000 7f9434ff9640 1 -- 172.21.15.86:0/3045927608 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f944403f450 con 0x5557c66c9a00 2024-04-21T08:48:04.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.194+0000 7f9456186740 10 client.19386 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:04.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.194+0000 7f9456186740 10 client.19386 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:04.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.194+0000 7f9456186740 20 client.19386 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:04.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.194+0000 7f9456186740 1 --2- 172.21.15.86:0/3045927608 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5557c6740bd0 0x5557c6742ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:04.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.194+0000 7f9456186740 4 client.19386 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:48:04.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.194+0000 7f9456186740 1 -- 172.21.15.86:0/3045927608 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5557c6632c10 con 0x5557c6740bd0 2024-04-21T08:48:04.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.194+0000 7f9451332640 1 --2- 172.21.15.86:0/3045927608 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5557c6740bd0 0x5557c6742ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:04.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.194+0000 7f9434ff9640 10 client.19386 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:04.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.194+0000 7f9451332640 1 --2- 172.21.15.86:0/3045927608 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5557c6740bd0 0x5557c6742ff0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=43fd5040c74b807b server_cookie=2a7d219dd05ffee8 in_seq=0 out_seq=0 2024-04-21T08:48:04.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.195+0000 7f9434ff9640 1 -- 172.21.15.86:0/3045927608 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5557c6632c10 con 0x5557c6740bd0 2024-04-21T08:48:04.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.195+0000 7f9434ff9640 10 client.19386 handle_command_reply: tid=0 2024-04-21T08:48:04.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.195+0000 7f9456186740 1 client.19386 shutdown 2024-04-21T08:48:04.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.195+0000 7f9456186740 1 -- 172.21.15.86:0/3045927608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c66c9a00 msgr2=0x5557c66ce2d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:04.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.195+0000 7f9456186740 1 --2- 172.21.15.86:0/3045927608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c66c9a00 0x5557c66ce2d0 secure :-1 s=READY pgs=1724 cs=0 l=1 rev1=1 crypto rx=0x7f9444010a50 tx=0x7f9444010a80 comp rx=0 tx=0).stop 2024-04-21T08:48:04.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.195+0000 7f9456186740 1 -- 172.21.15.86:0/3045927608 shutdown_connections 2024-04-21T08:48:04.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.195+0000 7f9456186740 1 --2- 172.21.15.86:0/3045927608 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5557c6740bd0 0x5557c6742ff0 crc :-1 s=READY pgs=272 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.195+0000 7f9456186740 1 --2- 172.21.15.86:0/3045927608 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5557c66c9a00 0x5557c66ce2d0 unknown :-1 s=CLOSED pgs=1724 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.195+0000 7f9434ff9640 0 client.19386 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:04.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.195+0000 7f9456186740 1 --2- 172.21.15.86:0/3045927608 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5557c66c0e80 0x5557c66cdd90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.195+0000 7f9456186740 1 --2- 172.21.15.86:0/3045927608 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5557c66c04d0 0x5557c66cd850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.195+0000 7f9456186740 1 -- 172.21.15.86:0/3045927608 >> 172.21.15.86:0/3045927608 conn(0x5557c66baba0 msgr2=0x5557c66b9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:04.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.195+0000 7f9456186740 1 -- 172.21.15.86:0/3045927608 shutdown_connections 2024-04-21T08:48:04.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.195+0000 7f9456186740 1 -- 172.21.15.86:0/3045927608 wait complete. 2024-04-21T08:48:04.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.195+0000 7f9456186740 20 client.19386 trim_cache size 0 max 16384 2024-04-21T08:48:04.201 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:48:04.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.195+0000 7f9453596640 1 -- 172.21.15.86:0/2466291164 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f94380a3da0 msgr2=0x7f94380a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:04.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.195+0000 7f9453596640 1 --2- 172.21.15.86:0/2466291164 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f94380a3da0 0x7f94380a6260 secure :-1 s=READY pgs=282 cs=0 l=1 rev1=1 crypto rx=0x7f9444005fd0 tx=0x7f9444004730 comp rx=0 tx=0).stop 2024-04-21T08:48:04.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.195+0000 7f9453596640 1 -- 172.21.15.86:0/2466291164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f944c1010a0 msgr2=0x7f944c19e3f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:04.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.195+0000 7f9453596640 1 --2- 172.21.15.86:0/2466291164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f944c1010a0 0x7f944c19e3f0 secure :-1 s=READY pgs=1720 cs=0 l=1 rev1=1 crypto rx=0x7f9448005e30 tx=0x7f9448034780 comp rx=0 tx=0).stop 2024-04-21T08:48:04.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.195+0000 7f9453596640 1 -- 172.21.15.86:0/2466291164 shutdown_connections 2024-04-21T08:48:04.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.195+0000 7f9453596640 1 --2- 172.21.15.86:0/2466291164 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f94380a3da0 0x7f94380a6260 unknown :-1 s=CLOSED pgs=282 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.195+0000 7f9453596640 1 --2- 172.21.15.86:0/2466291164 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f944c1985a0 0x7f944c198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.195+0000 7f9453596640 1 --2- 172.21.15.86:0/2466291164 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f944c106170 0x7f944c19e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.195+0000 7f9453596640 1 --2- 172.21.15.86:0/2466291164 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f944c1010a0 0x7f944c19e3f0 unknown :-1 s=CLOSED pgs=1720 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:04.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.195+0000 7f9453596640 1 -- 172.21.15.86:0/2466291164 >> 172.21.15.86:0/2466291164 conn(0x7f944c077eb0 msgr2=0x7f944c06a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:04.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.195+0000 7f9453596640 1 -- 172.21.15.86:0/2466291164 shutdown_connections 2024-04-21T08:48:04.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:04.195+0000 7f9453596640 1 -- 172.21.15.86:0/2466291164 wait complete. 2024-04-21T08:48:04.213 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:03 smithi088 ceph-mon[31259]: pgmap v287: 129 pgs: 129 active+clean; 5.2 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 105 KiB/s wr, 39 op/s 2024-04-21T08:48:04.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:03 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3824735196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:48:04.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:03 smithi179 ceph-mon[31829]: pgmap v287: 129 pgs: 129 active+clean; 5.2 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 71 KiB/s rd, 105 KiB/s wr, 39 op/s 2024-04-21T08:48:04.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:03 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3824735196' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:48:05.206 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:48:05.206 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:48:05.206 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:48:05.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.312+0000 7ffa38d52640 1 -- 172.21.15.86:0/2462914424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa34101a50 msgr2=0x7ffa34105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:05.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.312+0000 7ffa38d52640 1 --2- 172.21.15.86:0/2462914424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa34101a50 0x7ffa34105a90 secure :-1 s=READY pgs=1725 cs=0 l=1 rev1=1 crypto rx=0x7ffa1c009900 tx=0x7ffa1c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:05.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.312+0000 7ffa38d52640 1 -- 172.21.15.86:0/2462914424 shutdown_connections 2024-04-21T08:48:05.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.312+0000 7ffa38d52640 1 --2- 172.21.15.86:0/2462914424 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa34106170 0x7ffa3410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.312+0000 7ffa38d52640 1 --2- 172.21.15.86:0/2462914424 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa34101a50 0x7ffa34105a90 unknown :-1 s=CLOSED pgs=1725 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.312+0000 7ffa38d52640 1 --2- 172.21.15.86:0/2462914424 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa341010a0 0x7ffa34101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.312+0000 7ffa38d52640 1 -- 172.21.15.86:0/2462914424 >> 172.21.15.86:0/2462914424 conn(0x7ffa34077eb0 msgr2=0x7ffa340782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:05.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.312+0000 7ffa38d52640 1 -- 172.21.15.86:0/2462914424 shutdown_connections 2024-04-21T08:48:05.312 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.312+0000 7ffa38d52640 1 -- 172.21.15.86:0/2462914424 wait complete. 2024-04-21T08:48:05.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.313+0000 7ffa38d52640 1 Processor -- start 2024-04-21T08:48:05.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.313+0000 7ffa38d52640 1 -- start start 2024-04-21T08:48:05.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.313+0000 7ffa38d52640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa341010a0 0x7ffa34199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.313+0000 7ffa38d52640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa34106170 0x7ffa34199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.313+0000 7ffa38d52640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa3419a0e0 0x7ffa341a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.313+0000 7ffa38d52640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffa34111110 con 0x7ffa341010a0 2024-04-21T08:48:05.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.313+0000 7ffa38d52640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ffa34110f90 con 0x7ffa34106170 2024-04-21T08:48:05.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.313+0000 7ffa38d52640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ffa34111290 con 0x7ffa3419a0e0 2024-04-21T08:48:05.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.313+0000 7ffa3259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa341010a0 0x7ffa34199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.313+0000 7ffa3259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa341010a0 0x7ffa34199610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48428/0 (socket says 172.21.15.86:48428) 2024-04-21T08:48:05.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.313+0000 7ffa3259c640 1 -- 172.21.15.86:0/2854567247 learned_addr learned my addr 172.21.15.86:0/2854567247 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:05.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.313+0000 7ffa2bfff640 1 --2- 172.21.15.86:0/2854567247 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa34106170 0x7ffa34199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.313+0000 7ffa3259c640 1 -- 172.21.15.86:0/2854567247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa3419a0e0 msgr2=0x7ffa341a4690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:05.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.313+0000 7ffa3259c640 1 --2- 172.21.15.86:0/2854567247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa3419a0e0 0x7ffa341a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.313+0000 7ffa3259c640 1 -- 172.21.15.86:0/2854567247 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa34106170 msgr2=0x7ffa34199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:05.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.313+0000 7ffa3259c640 1 --2- 172.21.15.86:0/2854567247 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa34106170 0x7ffa34199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.315 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.313+0000 7ffa3259c640 1 -- 172.21.15.86:0/2854567247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa341a4e60 con 0x7ffa341010a0 2024-04-21T08:48:05.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.314+0000 7ffa3259c640 1 --2- 172.21.15.86:0/2854567247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa341010a0 0x7ffa34199610 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7ffa2400b4a0 tx=0x7ffa2400b960 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:05.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.314+0000 7ffa2b7fe640 1 -- 172.21.15.86:0/2854567247 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa24014070 con 0x7ffa341010a0 2024-04-21T08:48:05.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.314+0000 7ffa38d52640 1 -- 172.21.15.86:0/2854567247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa341a5150 con 0x7ffa341010a0 2024-04-21T08:48:05.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.314+0000 7ffa2b7fe640 1 -- 172.21.15.86:0/2854567247 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa24010ac0 con 0x7ffa341010a0 2024-04-21T08:48:05.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.314+0000 7ffa2b7fe640 1 -- 172.21.15.86:0/2854567247 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa24017c30 con 0x7ffa341010a0 2024-04-21T08:48:05.316 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.314+0000 7ffa38d52640 1 -- 172.21.15.86:0/2854567247 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa341a56f0 con 0x7ffa341010a0 2024-04-21T08:48:05.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.316+0000 7ffa2b7fe640 1 -- 172.21.15.86:0/2854567247 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ffa24017070 con 0x7ffa341010a0 2024-04-21T08:48:05.322 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.316+0000 7ffa3ba28740 1 Processor -- start 2024-04-21T08:48:05.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.317+0000 7ffa3ba28740 1 -- start start 2024-04-21T08:48:05.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.317+0000 7ffa3ba28740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647a25fd8e0 0x5647a25fdcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.317+0000 7ffa3ba28740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647a25fe290 0x5647a2606840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.317+0000 7ffa3ba28740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5647a2606e10 0x5647a2609260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.317+0000 7ffa3ba28740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5647a2561f00 con 0x5647a25fe290 2024-04-21T08:48:05.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.317+0000 7ffa3ba28740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5647a24ff590 con 0x5647a2606e10 2024-04-21T08:48:05.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.317+0000 7ffa3ba28740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5647a255dec0 con 0x5647a25fd8e0 2024-04-21T08:48:05.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.317+0000 7ffa3259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647a25fe290 0x5647a2606840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.317+0000 7ffa3259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647a25fe290 0x5647a2606840 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48438/0 (socket says 172.21.15.86:48438) 2024-04-21T08:48:05.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.317+0000 7ffa3259c640 1 -- 172.21.15.86:0/4210164217 learned_addr learned my addr 172.21.15.86:0/4210164217 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:05.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.317+0000 7ffa2bfff640 1 --2- 172.21.15.86:0/4210164217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5647a2606e10 0x5647a2609260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.317+0000 7ffa32d9d640 1 --2- 172.21.15.86:0/4210164217 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647a25fd8e0 0x5647a25fdcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.317+0000 7ffa2b7fe640 1 --2- 172.21.15.86:0/2854567247 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ffa200a3d20 0x7ffa200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.317+0000 7ffa3259c640 1 -- 172.21.15.86:0/4210164217 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647a25fd8e0 msgr2=0x5647a25fdcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:05.323 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.317+0000 7ffa32d9d640 1 --2- 172.21.15.86:0/2854567247 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ffa200a3d20 0x7ffa200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.317+0000 7ffa3259c640 1 --2- 172.21.15.86:0/4210164217 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647a25fd8e0 0x5647a25fdcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.317+0000 7ffa3259c640 1 -- 172.21.15.86:0/4210164217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5647a2606e10 msgr2=0x5647a2609260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:05.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.317+0000 7ffa3259c640 1 --2- 172.21.15.86:0/4210164217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5647a2606e10 0x5647a2609260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.317+0000 7ffa3259c640 1 -- 172.21.15.86:0/4210164217 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647a233c7e0 con 0x5647a25fe290 2024-04-21T08:48:05.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.317+0000 7ffa2b7fe640 1 -- 172.21.15.86:0/2854567247 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9584+0+0 (secure 0 0 0) 0x7ffa240d0300 con 0x7ffa341010a0 2024-04-21T08:48:05.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.318+0000 7ffa3259c640 1 --2- 172.21.15.86:0/4210164217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647a25fe290 0x5647a2606840 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7ffa240cd670 tx=0x7ffa24000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb3deaa92efe1349 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:05.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.318+0000 7ffa32d9d640 1 --2- 172.21.15.86:0/2854567247 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ffa200a3d20 0x7ffa200a61e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c009270 tx=0x7ffa2c009200 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:05.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.318+0000 7ffa28ff9640 1 -- 172.21.15.86:0/4210164217 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa24017360 con 0x5647a25fe290 2024-04-21T08:48:05.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.318+0000 7ffa28ff9640 1 -- 172.21.15.86:0/4210164217 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa2409c650 con 0x5647a25fe290 2024-04-21T08:48:05.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.318+0000 7ffa28ff9640 1 -- 172.21.15.86:0/4210164217 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa240985c0 con 0x5647a25fe290 2024-04-21T08:48:05.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.318+0000 7ffa3ba28740 1 -- 172.21.15.86:0/4210164217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647a25fe290 msgr2=0x5647a2606840 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:05.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.318+0000 7ffa3ba28740 1 --2- 172.21.15.86:0/4210164217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647a25fe290 0x5647a2606840 secure :-1 s=READY pgs=1727 cs=0 l=1 rev1=1 crypto rx=0x7ffa240cd670 tx=0x7ffa24000bc0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.318+0000 7ffa3ba28740 1 -- 172.21.15.86:0/4210164217 shutdown_connections 2024-04-21T08:48:05.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.318+0000 7ffa3ba28740 1 --2- 172.21.15.86:0/4210164217 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5647a2606e10 0x5647a2609260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.324 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.318+0000 7ffa3ba28740 1 --2- 172.21.15.86:0/4210164217 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647a25fe290 0x5647a2606840 unknown :-1 s=CLOSED pgs=1727 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.318+0000 7ffa3ba28740 1 --2- 172.21.15.86:0/4210164217 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647a25fd8e0 0x5647a25fdcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.318+0000 7ffa3ba28740 1 -- 172.21.15.86:0/4210164217 >> 172.21.15.86:0/4210164217 conn(0x5647a25f7fb0 msgr2=0x5647a25f8390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:05.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.319+0000 7ffa3ba28740 1 -- 172.21.15.86:0/4210164217 shutdown_connections 2024-04-21T08:48:05.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.319+0000 7ffa3ba28740 1 -- 172.21.15.86:0/4210164217 wait complete. 2024-04-21T08:48:05.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.319+0000 7ffa3ba28740 1 -- start start 2024-04-21T08:48:05.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.319+0000 7ffa3ba28740 1 Processor -- start 2024-04-21T08:48:05.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.319+0000 7ffa3ba28740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5647a25fd8e0 0x5647a2603d80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.319+0000 7ffa3ba28740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647a25fe290 0x5647a26042c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.319+0000 7ffa3ba28740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647a2606e10 0x5647a2604800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.319+0000 7ffa3ba28740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5647a2609d80 con 0x5647a2606e10 2024-04-21T08:48:05.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.319+0000 7ffa3ba28740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5647a251b580 con 0x5647a25fd8e0 2024-04-21T08:48:05.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.319+0000 7ffa3ba28740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5647a2609f00 con 0x5647a25fe290 2024-04-21T08:48:05.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.319+0000 7ffa3ba28740 10 client.0 _ll_register_callbacks cb 0x5647a2521650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:05.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.319+0000 7ffa32d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647a2606e10 0x5647a2604800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.325 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.319+0000 7ffa32d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647a2606e10 0x5647a2604800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48452/0 (socket says 172.21.15.86:48452) 2024-04-21T08:48:05.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.319+0000 7ffa32d9d640 1 -- 172.21.15.86:0/1412751117 learned_addr learned my addr 172.21.15.86:0/1412751117 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:05.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.319+0000 7ffa3259c640 1 --2- 172.21.15.86:0/1412751117 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5647a25fd8e0 0x5647a2603d80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.319+0000 7ffa32d9d640 1 -- 172.21.15.86:0/1412751117 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647a25fe290 msgr2=0x5647a26042c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:05.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.320+0000 7ffa32d9d640 1 --2- 172.21.15.86:0/1412751117 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647a25fe290 0x5647a26042c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.320+0000 7ffa32d9d640 1 -- 172.21.15.86:0/1412751117 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5647a25fd8e0 msgr2=0x5647a2603d80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:05.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.320+0000 7ffa32d9d640 1 --2- 172.21.15.86:0/1412751117 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5647a25fd8e0 0x5647a2603d80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.320+0000 7ffa32d9d640 1 -- 172.21.15.86:0/1412751117 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647a239cf70 con 0x5647a2606e10 2024-04-21T08:48:05.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.320+0000 7ffa3259c640 1 --2- 172.21.15.86:0/1412751117 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5647a25fd8e0 0x5647a2603d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:05.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.320+0000 7ffa32d9d640 1 --2- 172.21.15.86:0/1412751117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647a2606e10 0x5647a2604800 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c003db0 tx=0x7ffa2c004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9aed84cbd39f316e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:05.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.320+0000 7ffa19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:05.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.320+0000 7ffa19ffb640 1 -- 172.21.15.86:0/1412751117 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c01c050 con 0x5647a2606e10 2024-04-21T08:48:05.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.320+0000 7ffa19ffb640 1 -- 172.21.15.86:0/1412751117 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9584+0+0 (secure 0 0 0) 0x7ffa2c0047b0 con 0x5647a2606e10 2024-04-21T08:48:05.326 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.320+0000 7ffa3ba28740 1 -- 172.21.15.86:0/1412751117 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa2409c650 con 0x5647a2606e10 2024-04-21T08:48:05.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.320+0000 7ffa19ffb640 1 -- 172.21.15.86:0/1412751117 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffa2c02f050 con 0x5647a2606e10 2024-04-21T08:48:05.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.320+0000 7ffa3ba28740 10 client.19398 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:05.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.320+0000 7ffa3ba28740 1 -- 172.21.15.86:0/1412751117 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5647a2234170 con 0x5647a2606e10 2024-04-21T08:48:05.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.320+0000 7ffa19ffb640 1 -- 172.21.15.86:0/1412751117 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa2409c650 con 0x5647a2606e10 2024-04-21T08:48:05.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.320+0000 7ffa19ffb640 1 -- 172.21.15.86:0/1412751117 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ffa2c00f420 con 0x5647a2606e10 2024-04-21T08:48:05.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.320+0000 7ffa3ba28740 10 client.19398 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:05.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.320+0000 7ffa3ba28740 10 client.19398 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:05.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.320+0000 7ffa3ba28740 20 client.19398 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:05.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.320+0000 7ffa3ba28740 1 --2- 172.21.15.86:0/1412751117 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5647a2617380 0x5647a2619810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.320+0000 7ffa3ba28740 4 client.19398 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:05.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.320+0000 7ffa3ba28740 1 -- 172.21.15.86:0/1412751117 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5647a256f920 con 0x5647a2617380 2024-04-21T08:48:05.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.321+0000 7ffa3259c640 1 --2- 172.21.15.86:0/1412751117 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5647a2617380 0x5647a2619810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.321+0000 7ffa19ffb640 10 client.19398 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:05.327 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.321+0000 7ffa3259c640 1 --2- 172.21.15.86:0/1412751117 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5647a2617380 0x5647a2619810 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e5a6ac172c414159 server_cookie=af104ed2730f411c in_seq=0 out_seq=0 2024-04-21T08:48:05.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.322+0000 7ffa19ffb640 1 -- 172.21.15.86:0/1412751117 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5647a256f920 con 0x5647a2617380 2024-04-21T08:48:05.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.322+0000 7ffa19ffb640 10 client.19398 handle_command_reply: tid=0 2024-04-21T08:48:05.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.322+0000 7ffa3ba28740 1 client.19398 shutdown 2024-04-21T08:48:05.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.322+0000 7ffa3ba28740 1 -- 172.21.15.86:0/1412751117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647a2606e10 msgr2=0x5647a2604800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:05.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.322+0000 7ffa3ba28740 1 --2- 172.21.15.86:0/1412751117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647a2606e10 0x5647a2604800 secure :-1 s=READY pgs=1728 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c003db0 tx=0x7ffa2c004a40 comp rx=0 tx=0).stop 2024-04-21T08:48:05.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.322+0000 7ffa3ba28740 1 -- 172.21.15.86:0/1412751117 shutdown_connections 2024-04-21T08:48:05.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.322+0000 7ffa3ba28740 1 --2- 172.21.15.86:0/1412751117 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5647a2617380 0x5647a2619810 crc :-1 s=READY pgs=273 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.322+0000 7ffa3ba28740 1 --2- 172.21.15.86:0/1412751117 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647a2606e10 0x5647a2604800 unknown :-1 s=CLOSED pgs=1728 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.322+0000 7ffa3ba28740 1 --2- 172.21.15.86:0/1412751117 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647a25fe290 0x5647a26042c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.322+0000 7ffa19ffb640 0 client.19398 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:05.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.322+0000 7ffa3ba28740 1 --2- 172.21.15.86:0/1412751117 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5647a25fd8e0 0x5647a2603d80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.322+0000 7ffa3ba28740 1 -- 172.21.15.86:0/1412751117 >> 172.21.15.86:0/1412751117 conn(0x5647a25f7fb0 msgr2=0x5647a25f9d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:05.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.322+0000 7ffa3ba28740 1 -- 172.21.15.86:0/1412751117 shutdown_connections 2024-04-21T08:48:05.328 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.322+0000 7ffa3ba28740 1 -- 172.21.15.86:0/1412751117 wait complete. 2024-04-21T08:48:05.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.323+0000 7ffa3ba28740 20 client.19398 trim_cache size 0 max 16384 2024-04-21T08:48:05.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.340+0000 7ffa3ba28740 1 Processor -- start 2024-04-21T08:48:05.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.340+0000 7ffa3ba28740 1 -- start start 2024-04-21T08:48:05.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.340+0000 7ffa3ba28740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5647a25fd8e0 0x5647a26064b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.340+0000 7ffa3ba28740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647a25fe290 0x5647a2612c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.340+0000 7ffa3ba28740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647a2606e10 0x5647a2604450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.340+0000 7ffa3ba28740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffa341af620 con 0x5647a2606e10 2024-04-21T08:48:05.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.340+0000 7ffa3ba28740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ffa341b1160 con 0x5647a25fd8e0 2024-04-21T08:48:05.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.340+0000 7ffa3ba28740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ffa341afcf0 con 0x5647a25fe290 2024-04-21T08:48:05.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.340+0000 7ffa32d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647a2606e10 0x5647a2604450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.340+0000 7ffa32d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647a2606e10 0x5647a2604450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48454/0 (socket says 172.21.15.86:48454) 2024-04-21T08:48:05.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.340+0000 7ffa32d9d640 1 -- 172.21.15.86:0/2976680480 learned_addr learned my addr 172.21.15.86:0/2976680480 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:05.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.340+0000 7ffa2bfff640 1 --2- 172.21.15.86:0/2976680480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647a25fe290 0x5647a2612c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.340+0000 7ffa3259c640 1 --2- 172.21.15.86:0/2976680480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5647a25fd8e0 0x5647a26064b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.340+0000 7ffa32d9d640 1 -- 172.21.15.86:0/2976680480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647a25fe290 msgr2=0x5647a2612c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:05.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.340+0000 7ffa32d9d640 1 --2- 172.21.15.86:0/2976680480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647a25fe290 0x5647a2612c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.340+0000 7ffa32d9d640 1 -- 172.21.15.86:0/2976680480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5647a25fd8e0 msgr2=0x5647a26064b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:05.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.340+0000 7ffa32d9d640 1 --2- 172.21.15.86:0/2976680480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5647a25fd8e0 0x5647a26064b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.340+0000 7ffa32d9d640 1 -- 172.21.15.86:0/2976680480 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5647a2602d30 con 0x5647a2606e10 2024-04-21T08:48:05.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.340+0000 7ffa3259c640 1 --2- 172.21.15.86:0/2976680480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5647a25fd8e0 0x5647a26064b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:05.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.340+0000 7ffa32d9d640 1 --2- 172.21.15.86:0/2976680480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647a2606e10 0x5647a2604450 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c012850 tx=0x7ffa2c012f20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4131f05f667161ee server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:05.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.340+0000 7ffa1bfff640 1 -- 172.21.15.86:0/2976680480 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c037030 con 0x5647a2606e10 2024-04-21T08:48:05.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.340+0000 7ffa1bfff640 1 -- 172.21.15.86:0/2976680480 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa2c026070 con 0x5647a2606e10 2024-04-21T08:48:05.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.340+0000 7ffa1bfff640 1 -- 172.21.15.86:0/2976680480 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2c01c050 con 0x5647a2606e10 2024-04-21T08:48:05.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.341+0000 7ffa3ba28740 1 -- 172.21.15.86:0/2976680480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647a2606e10 msgr2=0x5647a2604450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:05.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.341+0000 7ffa3ba28740 1 --2- 172.21.15.86:0/2976680480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647a2606e10 0x5647a2604450 secure :-1 s=READY pgs=1729 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c012850 tx=0x7ffa2c012f20 comp rx=0 tx=0).stop 2024-04-21T08:48:05.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.341+0000 7ffa3ba28740 1 -- 172.21.15.86:0/2976680480 shutdown_connections 2024-04-21T08:48:05.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.341+0000 7ffa3ba28740 1 --2- 172.21.15.86:0/2976680480 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647a2606e10 0x5647a2604450 unknown :-1 s=CLOSED pgs=1729 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.341+0000 7ffa3ba28740 1 --2- 172.21.15.86:0/2976680480 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647a25fe290 0x5647a2612c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.341+0000 7ffa3ba28740 1 --2- 172.21.15.86:0/2976680480 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5647a25fd8e0 0x5647a26064b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.341+0000 7ffa3ba28740 1 -- 172.21.15.86:0/2976680480 >> 172.21.15.86:0/2976680480 conn(0x5647a25f7fb0 msgr2=0x5647a2603d40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:05.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.341+0000 7ffa3ba28740 1 -- 172.21.15.86:0/2976680480 shutdown_connections 2024-04-21T08:48:05.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.341+0000 7ffa3ba28740 1 -- 172.21.15.86:0/2976680480 wait complete. 2024-04-21T08:48:05.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.341+0000 7ffa3ba28740 1 -- start start 2024-04-21T08:48:05.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.341+0000 7ffa3ba28740 1 Processor -- start 2024-04-21T08:48:05.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.342+0000 7ffa3ba28740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647a25fd8e0 0x5647a2601120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.342+0000 7ffa3ba28740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5647a25fe290 0x5647a2601660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.342+0000 7ffa3ba28740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647a2606e10 0x5647a267c950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.342+0000 7ffa3ba28740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffa341b0a90 con 0x5647a25fd8e0 2024-04-21T08:48:05.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.342+0000 7ffa3ba28740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ffa341b03c0 con 0x5647a25fe290 2024-04-21T08:48:05.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.342+0000 7ffa3ba28740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ffa341ad600 con 0x5647a2606e10 2024-04-21T08:48:05.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.342+0000 7ffa3ba28740 10 client.0 _ll_register_callbacks cb 0x7ffa2c009d40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:05.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.342+0000 7ffa3259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647a25fd8e0 0x5647a2601120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.342+0000 7ffa3259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647a25fd8e0 0x5647a2601120 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48458/0 (socket says 172.21.15.86:48458) 2024-04-21T08:48:05.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.342+0000 7ffa3259c640 1 -- 172.21.15.86:0/1874385815 learned_addr learned my addr 172.21.15.86:0/1874385815 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:05.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.342+0000 7ffa3259c640 1 -- 172.21.15.86:0/1874385815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647a2606e10 msgr2=0x5647a267c950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:05.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.342+0000 7ffa3259c640 1 --2- 172.21.15.86:0/1874385815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647a2606e10 0x5647a267c950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.342+0000 7ffa3259c640 1 -- 172.21.15.86:0/1874385815 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5647a25fe290 msgr2=0x5647a2601660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:05.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.342+0000 7ffa2bfff640 1 --2- 172.21.15.86:0/1874385815 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5647a25fe290 0x5647a2601660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.342+0000 7ffa3259c640 1 --2- 172.21.15.86:0/1874385815 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5647a25fe290 0x5647a2601660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.342+0000 7ffa3259c640 1 -- 172.21.15.86:0/1874385815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5647a267ce90 con 0x5647a25fd8e0 2024-04-21T08:48:05.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.342+0000 7ffa3259c640 1 --2- 172.21.15.86:0/1874385815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647a25fd8e0 0x5647a2601120 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7ffa2409bee0 tx=0x7ffa2409a3d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1bd12e8795476980 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:05.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.342+0000 7ffa19ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:05.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.342+0000 7ffa19ffb640 1 -- 172.21.15.86:0/1874385815 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa2401e040 con 0x5647a25fd8e0 2024-04-21T08:48:05.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.342+0000 7ffa19ffb640 1 -- 172.21.15.86:0/1874385815 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9584+0+0 (secure 0 0 0) 0x7ffa240985c0 con 0x5647a25fd8e0 2024-04-21T08:48:05.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.342+0000 7ffa3ba28740 1 -- 172.21.15.86:0/1874385815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ffa2c026070 con 0x5647a25fd8e0 2024-04-21T08:48:05.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.342+0000 7ffa19ffb640 1 -- 172.21.15.86:0/1874385815 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ffa2409c650 con 0x5647a25fd8e0 2024-04-21T08:48:05.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.342+0000 7ffa3ba28740 10 client.19404 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:05.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.342+0000 7ffa3ba28740 1 -- 172.21.15.86:0/1874385815 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5647a267e0e0 con 0x5647a25fd8e0 2024-04-21T08:48:05.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.342+0000 7ffa19ffb640 1 -- 172.21.15.86:0/1874385815 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa2c026070 con 0x5647a25fd8e0 2024-04-21T08:48:05.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.343+0000 7ffa19ffb640 1 -- 172.21.15.86:0/1874385815 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ffa240cd870 con 0x5647a25fd8e0 2024-04-21T08:48:05.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.343+0000 7ffa3ba28740 10 client.19404 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:05.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.343+0000 7ffa3ba28740 10 client.19404 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:05.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.343+0000 7ffa3ba28740 20 client.19404 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:05.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.343+0000 7ffa3ba28740 1 --2- 172.21.15.86:0/1874385815 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5647a267e370 0x5647a2680760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.343+0000 7ffa3ba28740 4 client.19404 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:48:05.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.343+0000 7ffa3ba28740 1 -- 172.21.15.86:0/1874385815 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5647a2569790 con 0x5647a267e370 2024-04-21T08:48:05.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.343+0000 7ffa2bfff640 1 --2- 172.21.15.86:0/1874385815 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5647a267e370 0x5647a2680760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.343+0000 7ffa19ffb640 10 client.19404 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:05.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.343+0000 7ffa2bfff640 1 --2- 172.21.15.86:0/1874385815 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5647a267e370 0x5647a2680760 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e0611d070df0b151 server_cookie=e967e2a4a105729a in_seq=0 out_seq=0 2024-04-21T08:48:05.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.344+0000 7ffa19ffb640 1 -- 172.21.15.86:0/1874385815 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5647a2569790 con 0x5647a267e370 2024-04-21T08:48:05.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.345+0000 7ffa19ffb640 10 client.19404 handle_command_reply: tid=0 2024-04-21T08:48:05.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.345+0000 7ffa3ba28740 1 client.19404 shutdown 2024-04-21T08:48:05.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.345+0000 7ffa3ba28740 1 -- 172.21.15.86:0/1874385815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647a25fd8e0 msgr2=0x5647a2601120 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:05.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.345+0000 7ffa3ba28740 1 --2- 172.21.15.86:0/1874385815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647a25fd8e0 0x5647a2601120 secure :-1 s=READY pgs=1730 cs=0 l=1 rev1=1 crypto rx=0x7ffa2409bee0 tx=0x7ffa2409a3d0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.345+0000 7ffa3ba28740 1 -- 172.21.15.86:0/1874385815 shutdown_connections 2024-04-21T08:48:05.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.345+0000 7ffa3ba28740 1 --2- 172.21.15.86:0/1874385815 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5647a267e370 0x5647a2680760 crc :-1 s=READY pgs=274 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.345+0000 7ffa3ba28740 1 --2- 172.21.15.86:0/1874385815 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5647a2606e10 0x5647a267c950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.345+0000 7ffa3ba28740 1 --2- 172.21.15.86:0/1874385815 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5647a25fe290 0x5647a2601660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.345+0000 7ffa3ba28740 1 --2- 172.21.15.86:0/1874385815 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5647a25fd8e0 0x5647a2601120 unknown :-1 s=CLOSED pgs=1730 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.345+0000 7ffa3ba28740 1 -- 172.21.15.86:0/1874385815 >> 172.21.15.86:0/1874385815 conn(0x5647a25f7fb0 msgr2=0x5647a25f9b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:05.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.345+0000 7ffa19ffb640 0 client.19404 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:05.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.345+0000 7ffa3ba28740 1 -- 172.21.15.86:0/1874385815 shutdown_connections 2024-04-21T08:48:05.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.345+0000 7ffa3ba28740 1 -- 172.21.15.86:0/1874385815 wait complete. 2024-04-21T08:48:05.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.345+0000 7ffa3ba28740 20 client.19404 trim_cache size 0 max 16384 2024-04-21T08:48:05.350 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:48:05.350 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-21T08:48:05.350 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "e4f14c95-6c86-408f-a297-e8d639deadba", 2024-04-21T08:48:05.350 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-21T08:48:05.350 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:48:05.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.345+0000 7ffa38d52640 1 -- 172.21.15.86:0/2854567247 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ffa200a3d20 msgr2=0x7ffa200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:05.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.345+0000 7ffa38d52640 1 --2- 172.21.15.86:0/2854567247 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ffa200a3d20 0x7ffa200a61e0 secure :-1 s=READY pgs=283 cs=0 l=1 rev1=1 crypto rx=0x7ffa2c009270 tx=0x7ffa2c009200 comp rx=0 tx=0).stop 2024-04-21T08:48:05.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.345+0000 7ffa38d52640 1 -- 172.21.15.86:0/2854567247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa341010a0 msgr2=0x7ffa34199610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:05.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.345+0000 7ffa38d52640 1 --2- 172.21.15.86:0/2854567247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa341010a0 0x7ffa34199610 secure :-1 s=READY pgs=1726 cs=0 l=1 rev1=1 crypto rx=0x7ffa2400b4a0 tx=0x7ffa2400b960 comp rx=0 tx=0).stop 2024-04-21T08:48:05.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.345+0000 7ffa38d52640 1 -- 172.21.15.86:0/2854567247 shutdown_connections 2024-04-21T08:48:05.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.345+0000 7ffa38d52640 1 --2- 172.21.15.86:0/2854567247 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ffa200a3d20 0x7ffa200a61e0 unknown :-1 s=CLOSED pgs=283 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.345+0000 7ffa38d52640 1 --2- 172.21.15.86:0/2854567247 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa3419a0e0 0x7ffa341a4690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.345+0000 7ffa38d52640 1 --2- 172.21.15.86:0/2854567247 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa34106170 0x7ffa34199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.345+0000 7ffa38d52640 1 --2- 172.21.15.86:0/2854567247 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa341010a0 0x7ffa34199610 unknown :-1 s=CLOSED pgs=1726 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.345+0000 7ffa38d52640 1 -- 172.21.15.86:0/2854567247 >> 172.21.15.86:0/2854567247 conn(0x7ffa34077eb0 msgr2=0x7ffa340ff0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:05.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.346+0000 7ffa38d52640 1 -- 172.21.15.86:0/2854567247 shutdown_connections 2024-04-21T08:48:05.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.346+0000 7ffa38d52640 1 -- 172.21.15.86:0/2854567247 wait complete. 2024-04-21T08:48:05.356 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:48:05.356 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:48:05.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.456+0000 7fbb65f59640 1 -- 172.21.15.86:0/1712149578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb6006a5d0 msgr2=0x7fbb6006aa30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:05.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.456+0000 7fbb65f59640 1 --2- 172.21.15.86:0/1712149578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb6006a5d0 0x7fbb6006aa30 secure :-1 s=READY pgs=1731 cs=0 l=1 rev1=1 crypto rx=0x7fbb54009900 tx=0x7fbb5402bcd0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.456+0000 7fbb65f59640 1 -- 172.21.15.86:0/1712149578 shutdown_connections 2024-04-21T08:48:05.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.456+0000 7fbb65f59640 1 --2- 172.21.15.86:0/1712149578 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb6006af70 0x7fbb6010ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.456+0000 7fbb65f59640 1 --2- 172.21.15.86:0/1712149578 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb6006a5d0 0x7fbb6006aa30 unknown :-1 s=CLOSED pgs=1731 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.456+0000 7fbb65f59640 1 --2- 172.21.15.86:0/1712149578 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb60108020 0x7fbb60108400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.456+0000 7fbb65f59640 1 -- 172.21.15.86:0/1712149578 >> 172.21.15.86:0/1712149578 conn(0x7fbb60076c50 msgr2=0x7fbb60077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:05.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.456+0000 7fbb65f59640 1 -- 172.21.15.86:0/1712149578 shutdown_connections 2024-04-21T08:48:05.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.456+0000 7fbb65f59640 1 -- 172.21.15.86:0/1712149578 wait complete. 2024-04-21T08:48:05.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.457+0000 7fbb65f59640 1 Processor -- start 2024-04-21T08:48:05.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.457+0000 7fbb65f59640 1 -- start start 2024-04-21T08:48:05.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.457+0000 7fbb65f59640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb6006a5d0 0x7fbb60198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.457+0000 7fbb65f59640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb6006af70 0x7fbb60198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.457+0000 7fbb65f59640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb60108020 0x7fbb6019cf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.457+0000 7fbb65f59640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbb60112160 con 0x7fbb6006a5d0 2024-04-21T08:48:05.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.457+0000 7fbb65f59640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbb60111fe0 con 0x7fbb6006af70 2024-04-21T08:48:05.459 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.457+0000 7fbb65f59640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbb601122e0 con 0x7fbb60108020 2024-04-21T08:48:05.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.457+0000 7fbb5f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb6006a5d0 0x7fbb60198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.457+0000 7fbb5f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb6006a5d0 0x7fbb60198680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48466/0 (socket says 172.21.15.86:48466) 2024-04-21T08:48:05.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.457+0000 7fbb5f7fe640 1 -- 172.21.15.86:0/3764009627 learned_addr learned my addr 172.21.15.86:0/3764009627 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:05.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.457+0000 7fbb5effd640 1 --2- 172.21.15.86:0/3764009627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb6006af70 0x7fbb60198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.457+0000 7fbb5ffff640 1 --2- 172.21.15.86:0/3764009627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb60108020 0x7fbb6019cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.458+0000 7fbb5f7fe640 1 -- 172.21.15.86:0/3764009627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb60108020 msgr2=0x7fbb6019cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:05.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.458+0000 7fbb5f7fe640 1 --2- 172.21.15.86:0/3764009627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb60108020 0x7fbb6019cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.458+0000 7fbb5f7fe640 1 -- 172.21.15.86:0/3764009627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb6006af70 msgr2=0x7fbb60198bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:05.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.458+0000 7fbb5f7fe640 1 --2- 172.21.15.86:0/3764009627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb6006af70 0x7fbb60198bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.458+0000 7fbb5f7fe640 1 -- 172.21.15.86:0/3764009627 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbb6019d670 con 0x7fbb6006a5d0 2024-04-21T08:48:05.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.458+0000 7fbb5f7fe640 1 --2- 172.21.15.86:0/3764009627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb6006a5d0 0x7fbb60198680 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fbb5000c920 tx=0x7fbb5000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:05.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.458+0000 7fbb5cff9640 1 -- 172.21.15.86:0/3764009627 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb50007c70 con 0x7fbb6006a5d0 2024-04-21T08:48:05.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.458+0000 7fbb65f59640 1 -- 172.21.15.86:0/3764009627 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbb6019d960 con 0x7fbb6006a5d0 2024-04-21T08:48:05.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.458+0000 7fbb5cff9640 1 -- 172.21.15.86:0/3764009627 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb50007e10 con 0x7fbb6006a5d0 2024-04-21T08:48:05.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.458+0000 7fbb5cff9640 1 -- 172.21.15.86:0/3764009627 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb50018820 con 0x7fbb6006a5d0 2024-04-21T08:48:05.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.458+0000 7fbb65f59640 1 -- 172.21.15.86:0/3764009627 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbb601a51f0 con 0x7fbb6006a5d0 2024-04-21T08:48:05.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.460+0000 7fbb5cff9640 1 -- 172.21.15.86:0/3764009627 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbb5000f450 con 0x7fbb6006a5d0 2024-04-21T08:48:05.467 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.460+0000 7fbb68c1f740 1 Processor -- start 2024-04-21T08:48:05.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.460+0000 7fbb68c1f740 1 -- start start 2024-04-21T08:48:05.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.461+0000 7fbb68c1f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5c8406580 0x55b5c8406960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.461+0000 7fbb68c1f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5c8406ea0 0x55b5c840f540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.461+0000 7fbb68c1f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5c840fa80 0x55b5c8411e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.461+0000 7fbb68c1f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b5c836b1e0 con 0x55b5c8406ea0 2024-04-21T08:48:05.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.461+0000 7fbb68c1f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b5c8308390 con 0x55b5c8406580 2024-04-21T08:48:05.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.461+0000 7fbb68c1f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b5c83671a0 con 0x55b5c840fa80 2024-04-21T08:48:05.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.461+0000 7fbb5f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5c8406ea0 0x55b5c840f540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.461+0000 7fbb5f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5c8406ea0 0x55b5c840f540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48468/0 (socket says 172.21.15.86:48468) 2024-04-21T08:48:05.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.461+0000 7fbb5f7fe640 1 -- 172.21.15.86:0/3670010350 learned_addr learned my addr 172.21.15.86:0/3670010350 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:05.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.461+0000 7fbb5cff9640 1 --2- 172.21.15.86:0/3764009627 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbb480a3d20 0x7fbb480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.461+0000 7fbb5ffff640 1 --2- 172.21.15.86:0/3670010350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5c8406580 0x55b5c8406960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.461+0000 7fbb5effd640 1 --2- 172.21.15.86:0/3670010350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5c840fa80 0x55b5c8411e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.461+0000 7fbb5ffff640 1 --2- 172.21.15.86:0/3764009627 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbb480a3d20 0x7fbb480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.461+0000 7fbb5f7fe640 1 -- 172.21.15.86:0/3670010350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5c840fa80 msgr2=0x55b5c8411e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:05.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.461+0000 7fbb5f7fe640 1 --2- 172.21.15.86:0/3670010350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5c840fa80 0x55b5c8411e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.461+0000 7fbb5f7fe640 1 -- 172.21.15.86:0/3670010350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5c8406580 msgr2=0x55b5c8406960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:05.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.461+0000 7fbb5f7fe640 1 --2- 172.21.15.86:0/3670010350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5c8406580 0x55b5c8406960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.461+0000 7fbb5f7fe640 1 -- 172.21.15.86:0/3670010350 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5c81477d0 con 0x55b5c8406ea0 2024-04-21T08:48:05.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.461+0000 7fbb5ffff640 1 --2- 172.21.15.86:0/3670010350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5c8406580 0x55b5c8406960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:05.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.461+0000 7fbb5cff9640 1 -- 172.21.15.86:0/3764009627 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9584+0+0 (secure 0 0 0) 0x7fbb500d0c60 con 0x7fbb6006a5d0 2024-04-21T08:48:05.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.461+0000 7fbb5f7fe640 1 --2- 172.21.15.86:0/3670010350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5c8406ea0 0x55b5c840f540 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fbb500cdec0 tx=0x7fbb50020000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70d7b45effb822 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:05.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.461+0000 7fbb5ffff640 1 --2- 172.21.15.86:0/3764009627 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbb480a3d20 0x7fbb480a61e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbb58009fd0 tx=0x7fbb58009500 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:05.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.461+0000 7fbb45ffb640 1 -- 172.21.15.86:0/3670010350 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5009cea0 con 0x55b5c8406ea0 2024-04-21T08:48:05.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.462+0000 7fbb45ffb640 1 -- 172.21.15.86:0/3670010350 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb500a5410 con 0x55b5c8406ea0 2024-04-21T08:48:05.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.462+0000 7fbb45ffb640 1 -- 172.21.15.86:0/3670010350 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb500d4330 con 0x55b5c8406ea0 2024-04-21T08:48:05.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.462+0000 7fbb68c1f740 1 -- 172.21.15.86:0/3670010350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5c8406ea0 msgr2=0x55b5c840f540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:05.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.462+0000 7fbb68c1f740 1 --2- 172.21.15.86:0/3670010350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5c8406ea0 0x55b5c840f540 secure :-1 s=READY pgs=1733 cs=0 l=1 rev1=1 crypto rx=0x7fbb500cdec0 tx=0x7fbb50020000 comp rx=0 tx=0).stop 2024-04-21T08:48:05.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.462+0000 7fbb68c1f740 1 -- 172.21.15.86:0/3670010350 shutdown_connections 2024-04-21T08:48:05.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.462+0000 7fbb68c1f740 1 --2- 172.21.15.86:0/3670010350 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5c840fa80 0x55b5c8411e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.462+0000 7fbb68c1f740 1 --2- 172.21.15.86:0/3670010350 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5c8406ea0 0x55b5c840f540 unknown :-1 s=CLOSED pgs=1733 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.462+0000 7fbb68c1f740 1 --2- 172.21.15.86:0/3670010350 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5c8406580 0x55b5c8406960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.462+0000 7fbb68c1f740 1 -- 172.21.15.86:0/3670010350 >> 172.21.15.86:0/3670010350 conn(0x55b5c8400cb0 msgr2=0x55b5c8401090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:05.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.462+0000 7fbb68c1f740 1 -- 172.21.15.86:0/3670010350 shutdown_connections 2024-04-21T08:48:05.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.462+0000 7fbb68c1f740 1 -- 172.21.15.86:0/3670010350 wait complete. 2024-04-21T08:48:05.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.463+0000 7fbb68c1f740 1 -- start start 2024-04-21T08:48:05.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.463+0000 7fbb68c1f740 1 Processor -- start 2024-04-21T08:48:05.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.463+0000 7fbb68c1f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5c8406580 0x55b5c840ca00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.463+0000 7fbb68c1f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5c8406ea0 0x55b5c840cf40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.463+0000 7fbb68c1f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5c840fa80 0x55b5c840d480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.463+0000 7fbb68c1f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b5c8412810 con 0x55b5c8406580 2024-04-21T08:48:05.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.463+0000 7fbb68c1f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b5c83242f0 con 0x55b5c8406ea0 2024-04-21T08:48:05.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.463+0000 7fbb68c1f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b5c8412990 con 0x55b5c840fa80 2024-04-21T08:48:05.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.463+0000 7fbb68c1f740 10 client.0 _ll_register_callbacks cb 0x55b5c8331580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:05.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.463+0000 7fbb5f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5c8406580 0x55b5c840ca00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.463+0000 7fbb5f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5c8406580 0x55b5c840ca00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48472/0 (socket says 172.21.15.86:48472) 2024-04-21T08:48:05.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.463+0000 7fbb5f7fe640 1 -- 172.21.15.86:0/3563100829 learned_addr learned my addr 172.21.15.86:0/3563100829 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:05.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.463+0000 7fbb5effd640 1 --2- 172.21.15.86:0/3563100829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5c8406ea0 0x55b5c840cf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.463+0000 7fbb5ffff640 1 --2- 172.21.15.86:0/3563100829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5c840fa80 0x55b5c840d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.463+0000 7fbb5f7fe640 1 -- 172.21.15.86:0/3563100829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5c840fa80 msgr2=0x55b5c840d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:05.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.463+0000 7fbb5f7fe640 1 --2- 172.21.15.86:0/3563100829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5c840fa80 0x55b5c840d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.463+0000 7fbb5f7fe640 1 -- 172.21.15.86:0/3563100829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5c8406ea0 msgr2=0x55b5c840cf40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:05.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.463+0000 7fbb5f7fe640 1 --2- 172.21.15.86:0/3563100829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5c8406ea0 0x55b5c840cf40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.463+0000 7fbb5f7fe640 1 -- 172.21.15.86:0/3563100829 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5c807d3f0 con 0x55b5c8406580 2024-04-21T08:48:05.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.463+0000 7fbb5effd640 1 --2- 172.21.15.86:0/3563100829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5c8406ea0 0x55b5c840cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:05.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.464+0000 7fbb5f7fe640 1 --2- 172.21.15.86:0/3563100829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5c8406580 0x55b5c840ca00 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fbb5009c730 tx=0x7fbb50098410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc2bef38e0201faf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:05.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.464+0000 7fbb3effd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:05.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.464+0000 7fbb3effd640 1 -- 172.21.15.86:0/3563100829 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb5009cea0 con 0x55b5c8406580 2024-04-21T08:48:05.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.464+0000 7fbb3effd640 1 -- 172.21.15.86:0/3563100829 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9584+0+0 (secure 0 0 0) 0x7fbb5000fa90 con 0x55b5c8406580 2024-04-21T08:48:05.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.464+0000 7fbb68c1f740 1 -- 172.21.15.86:0/3563100829 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb500a5410 con 0x55b5c8406580 2024-04-21T08:48:05.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.464+0000 7fbb3effd640 1 -- 172.21.15.86:0/3563100829 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbb500ba050 con 0x55b5c8406580 2024-04-21T08:48:05.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.464+0000 7fbb68c1f740 10 client.19416 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:05.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.464+0000 7fbb68c1f740 1 -- 172.21.15.86:0/3563100829 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b5c803d0f0 con 0x55b5c8406580 2024-04-21T08:48:05.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.464+0000 7fbb3effd640 1 -- 172.21.15.86:0/3563100829 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb500a5410 con 0x55b5c8406580 2024-04-21T08:48:05.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.464+0000 7fbb3effd640 1 -- 172.21.15.86:0/3563100829 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbb50085ce0 con 0x55b5c8406580 2024-04-21T08:48:05.472 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.464+0000 7fbb68c1f740 10 client.19416 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:05.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.464+0000 7fbb68c1f740 10 client.19416 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:05.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.464+0000 7fbb68c1f740 20 client.19416 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:05.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.464+0000 7fbb68c1f740 1 --2- 172.21.15.86:0/3563100829 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b5c841f7e0 0x55b5c8421bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.464+0000 7fbb68c1f740 4 client.19416 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:05.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.464+0000 7fbb68c1f740 1 -- 172.21.15.86:0/3563100829 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5c8378c00 con 0x55b5c841f7e0 2024-04-21T08:48:05.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.464+0000 7fbb5effd640 1 --2- 172.21.15.86:0/3563100829 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b5c841f7e0 0x55b5c8421bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.465+0000 7fbb5effd640 1 --2- 172.21.15.86:0/3563100829 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b5c841f7e0 0x55b5c8421bd0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=fcf017aff989be76 server_cookie=90ac791f13d83fe7 in_seq=0 out_seq=0 2024-04-21T08:48:05.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.465+0000 7fbb3effd640 10 client.19416 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:05.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.466+0000 7fbb3effd640 1 -- 172.21.15.86:0/3563100829 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b5c8378c00 con 0x55b5c841f7e0 2024-04-21T08:48:05.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.466+0000 7fbb3effd640 10 client.19416 handle_command_reply: tid=0 2024-04-21T08:48:05.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.466+0000 7fbb68c1f740 1 client.19416 shutdown 2024-04-21T08:48:05.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.466+0000 7fbb68c1f740 1 -- 172.21.15.86:0/3563100829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5c8406580 msgr2=0x55b5c840ca00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:05.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.466+0000 7fbb68c1f740 1 --2- 172.21.15.86:0/3563100829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5c8406580 0x55b5c840ca00 secure :-1 s=READY pgs=1734 cs=0 l=1 rev1=1 crypto rx=0x7fbb5009c730 tx=0x7fbb50098410 comp rx=0 tx=0).stop 2024-04-21T08:48:05.473 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.466+0000 7fbb68c1f740 1 -- 172.21.15.86:0/3563100829 shutdown_connections 2024-04-21T08:48:05.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.466+0000 7fbb68c1f740 1 --2- 172.21.15.86:0/3563100829 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b5c841f7e0 0x55b5c8421bd0 crc :-1 s=READY pgs=275 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.466+0000 7fbb68c1f740 1 --2- 172.21.15.86:0/3563100829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5c840fa80 0x55b5c840d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.466+0000 7fbb3effd640 0 client.19416 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:05.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.466+0000 7fbb68c1f740 1 --2- 172.21.15.86:0/3563100829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5c8406ea0 0x55b5c840cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.466+0000 7fbb68c1f740 1 --2- 172.21.15.86:0/3563100829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5c8406580 0x55b5c840ca00 unknown :-1 s=CLOSED pgs=1734 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.466+0000 7fbb68c1f740 1 -- 172.21.15.86:0/3563100829 >> 172.21.15.86:0/3563100829 conn(0x55b5c8400cb0 msgr2=0x55b5c8402a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:05.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.466+0000 7fbb68c1f740 1 -- 172.21.15.86:0/3563100829 shutdown_connections 2024-04-21T08:48:05.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.466+0000 7fbb68c1f740 1 -- 172.21.15.86:0/3563100829 wait complete. 2024-04-21T08:48:05.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.466+0000 7fbb68c1f740 20 client.19416 trim_cache size 0 max 16384 2024-04-21T08:48:05.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.483+0000 7fbb68c1f740 1 Processor -- start 2024-04-21T08:48:05.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.483+0000 7fbb68c1f740 1 -- start start 2024-04-21T08:48:05.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.483+0000 7fbb68c1f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5c8406580 0x55b5c840f130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.483+0000 7fbb68c1f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5c8406ea0 0x55b5c83ffbf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.483+0000 7fbb68c1f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5c840fa80 0x55b5c8424bf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.483+0000 7fbb68c1f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbb601af520 con 0x55b5c840fa80 2024-04-21T08:48:05.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.483+0000 7fbb68c1f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbb601b1060 con 0x55b5c8406ea0 2024-04-21T08:48:05.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.483+0000 7fbb68c1f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbb601afbf0 con 0x55b5c8406580 2024-04-21T08:48:05.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.483+0000 7fbb5ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5c840fa80 0x55b5c8424bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.483+0000 7fbb5ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5c840fa80 0x55b5c8424bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48478/0 (socket says 172.21.15.86:48478) 2024-04-21T08:48:05.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.483+0000 7fbb5ffff640 1 -- 172.21.15.86:0/1381087599 learned_addr learned my addr 172.21.15.86:0/1381087599 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:05.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.483+0000 7fbb5effd640 1 --2- 172.21.15.86:0/1381087599 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5c8406ea0 0x55b5c83ffbf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.483+0000 7fbb5f7fe640 1 --2- 172.21.15.86:0/1381087599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5c8406580 0x55b5c840f130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.483+0000 7fbb5ffff640 1 -- 172.21.15.86:0/1381087599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5c8406580 msgr2=0x55b5c840f130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:05.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.483+0000 7fbb5ffff640 1 --2- 172.21.15.86:0/1381087599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5c8406580 0x55b5c840f130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.483+0000 7fbb5ffff640 1 -- 172.21.15.86:0/1381087599 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5c8406ea0 msgr2=0x55b5c83ffbf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:05.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.483+0000 7fbb5ffff640 1 --2- 172.21.15.86:0/1381087599 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5c8406ea0 0x55b5c83ffbf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.483+0000 7fbb5ffff640 1 -- 172.21.15.86:0/1381087599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5c8406cd0 con 0x55b5c840fa80 2024-04-21T08:48:05.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.484+0000 7fbb5ffff640 1 --2- 172.21.15.86:0/1381087599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5c840fa80 0x55b5c8424bf0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fbb58003c90 tx=0x7fbb58005bd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2d01acb6ddde435d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:05.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.484+0000 7fbb457fa640 1 -- 172.21.15.86:0/1381087599 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb58010640 con 0x55b5c840fa80 2024-04-21T08:48:05.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.484+0000 7fbb457fa640 1 -- 172.21.15.86:0/1381087599 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb58010c90 con 0x55b5c840fa80 2024-04-21T08:48:05.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.484+0000 7fbb457fa640 1 -- 172.21.15.86:0/1381087599 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb58020cb0 con 0x55b5c840fa80 2024-04-21T08:48:05.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.484+0000 7fbb68c1f740 1 -- 172.21.15.86:0/1381087599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5c840fa80 msgr2=0x55b5c8424bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:05.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.484+0000 7fbb68c1f740 1 --2- 172.21.15.86:0/1381087599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5c840fa80 0x55b5c8424bf0 secure :-1 s=READY pgs=1735 cs=0 l=1 rev1=1 crypto rx=0x7fbb58003c90 tx=0x7fbb58005bd0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.484+0000 7fbb68c1f740 1 -- 172.21.15.86:0/1381087599 shutdown_connections 2024-04-21T08:48:05.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.484+0000 7fbb68c1f740 1 --2- 172.21.15.86:0/1381087599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5c840fa80 0x55b5c8424bf0 unknown :-1 s=CLOSED pgs=1735 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.484+0000 7fbb68c1f740 1 --2- 172.21.15.86:0/1381087599 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5c8406ea0 0x55b5c83ffbf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.484+0000 7fbb68c1f740 1 --2- 172.21.15.86:0/1381087599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5c8406580 0x55b5c840f130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.484+0000 7fbb68c1f740 1 -- 172.21.15.86:0/1381087599 >> 172.21.15.86:0/1381087599 conn(0x55b5c8400cb0 msgr2=0x55b5c83ff4e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:05.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.484+0000 7fbb68c1f740 1 -- 172.21.15.86:0/1381087599 shutdown_connections 2024-04-21T08:48:05.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.485+0000 7fbb68c1f740 1 -- 172.21.15.86:0/1381087599 wait complete. 2024-04-21T08:48:05.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.485+0000 7fbb68c1f740 1 -- start start 2024-04-21T08:48:05.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.485+0000 7fbb68c1f740 1 Processor -- start 2024-04-21T08:48:05.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.485+0000 7fbb68c1f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5c8406580 0x55b5c8422570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.485+0000 7fbb68c1f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5c8406ea0 0x55b5c8422ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.485+0000 7fbb68c1f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5c840fa80 0x55b5c8422ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.485+0000 7fbb68c1f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbb601b0990 con 0x55b5c8406ea0 2024-04-21T08:48:05.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.485+0000 7fbb68c1f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbb601b02c0 con 0x55b5c8406580 2024-04-21T08:48:05.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.485+0000 7fbb68c1f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbb601ad500 con 0x55b5c840fa80 2024-04-21T08:48:05.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.485+0000 7fbb68c1f740 10 client.0 _ll_register_callbacks cb 0x7fbb5009c100 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:05.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.485+0000 7fbb5effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5c8406ea0 0x55b5c8422ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.485+0000 7fbb5effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5c8406ea0 0x55b5c8422ab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48494/0 (socket says 172.21.15.86:48494) 2024-04-21T08:48:05.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.485+0000 7fbb5effd640 1 -- 172.21.15.86:0/1430023606 learned_addr learned my addr 172.21.15.86:0/1430023606 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:05.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.485+0000 7fbb5f7fe640 1 --2- 172.21.15.86:0/1430023606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5c8406580 0x55b5c8422570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.486+0000 7fbb5effd640 1 -- 172.21.15.86:0/1430023606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5c840fa80 msgr2=0x55b5c8422ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:05.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.486+0000 7fbb5effd640 1 --2- 172.21.15.86:0/1430023606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5c840fa80 0x55b5c8422ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.486+0000 7fbb5effd640 1 -- 172.21.15.86:0/1430023606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5c8406580 msgr2=0x55b5c8422570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:05.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.486+0000 7fbb5effd640 1 --2- 172.21.15.86:0/1430023606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5c8406580 0x55b5c8422570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.486+0000 7fbb5effd640 1 -- 172.21.15.86:0/1430023606 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5c84849c0 con 0x55b5c8406ea0 2024-04-21T08:48:05.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.486+0000 7fbb5effd640 1 --2- 172.21.15.86:0/1430023606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5c8406ea0 0x55b5c8422ab0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fbb54005ec0 tx=0x7fbb54004370 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6f66258df8c03f8f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:05.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.486+0000 7fbb3effd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:05.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.486+0000 7fbb3effd640 1 -- 172.21.15.86:0/1430023606 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbb54048070 con 0x55b5c8406ea0 2024-04-21T08:48:05.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.486+0000 7fbb3effd640 1 -- 172.21.15.86:0/1430023606 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9584+0+0 (secure 0 0 0) 0x7fbb5402ca90 con 0x55b5c8406ea0 2024-04-21T08:48:05.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.486+0000 7fbb68c1f740 1 -- 172.21.15.86:0/1430023606 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbb58010c90 con 0x55b5c8406ea0 2024-04-21T08:48:05.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.486+0000 7fbb3effd640 1 -- 172.21.15.86:0/1430023606 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbb54054050 con 0x55b5c8406ea0 2024-04-21T08:48:05.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.486+0000 7fbb68c1f740 10 client.19422 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:05.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.486+0000 7fbb68c1f740 1 -- 172.21.15.86:0/1430023606 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b5c8485bf0 con 0x55b5c8406ea0 2024-04-21T08:48:05.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.486+0000 7fbb3effd640 1 -- 172.21.15.86:0/1430023606 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbb58010c90 con 0x55b5c8406ea0 2024-04-21T08:48:05.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.486+0000 7fbb3effd640 1 -- 172.21.15.86:0/1430023606 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbb540047c0 con 0x55b5c8406ea0 2024-04-21T08:48:05.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.486+0000 7fbb68c1f740 10 client.19422 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:05.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.486+0000 7fbb68c1f740 10 client.19422 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:05.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.486+0000 7fbb68c1f740 20 client.19422 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:05.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.486+0000 7fbb68c1f740 1 --2- 172.21.15.86:0/1430023606 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b5c84863a0 0x55b5c84887c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.486+0000 7fbb68c1f740 4 client.19422 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:48:05.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.486+0000 7fbb68c1f740 1 -- 172.21.15.86:0/1430023606 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b5c8372a70 con 0x55b5c84863a0 2024-04-21T08:48:05.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.487+0000 7fbb5f7fe640 1 --2- 172.21.15.86:0/1430023606 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b5c84863a0 0x55b5c84887c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.487+0000 7fbb3effd640 10 client.19422 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:05.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.487+0000 7fbb5f7fe640 1 --2- 172.21.15.86:0/1430023606 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b5c84863a0 0x55b5c84887c0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=96e9e05d4d6d19df server_cookie=38905109b3304836 in_seq=0 out_seq=0 2024-04-21T08:48:05.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.487+0000 7fbb3effd640 1 -- 172.21.15.86:0/1430023606 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b5c8372a70 con 0x55b5c84863a0 2024-04-21T08:48:05.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.487+0000 7fbb3effd640 10 client.19422 handle_command_reply: tid=0 2024-04-21T08:48:05.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.487+0000 7fbb68c1f740 1 client.19422 shutdown 2024-04-21T08:48:05.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.487+0000 7fbb68c1f740 1 -- 172.21.15.86:0/1430023606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5c8406ea0 msgr2=0x55b5c8422ab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:05.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.487+0000 7fbb68c1f740 1 --2- 172.21.15.86:0/1430023606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5c8406ea0 0x55b5c8422ab0 secure :-1 s=READY pgs=1736 cs=0 l=1 rev1=1 crypto rx=0x7fbb54005ec0 tx=0x7fbb54004370 comp rx=0 tx=0).stop 2024-04-21T08:48:05.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.487+0000 7fbb68c1f740 1 -- 172.21.15.86:0/1430023606 shutdown_connections 2024-04-21T08:48:05.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.487+0000 7fbb68c1f740 1 --2- 172.21.15.86:0/1430023606 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b5c84863a0 0x55b5c84887c0 crc :-1 s=READY pgs=276 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.487+0000 7fbb68c1f740 1 --2- 172.21.15.86:0/1430023606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b5c840fa80 0x55b5c8422ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.487+0000 7fbb68c1f740 1 --2- 172.21.15.86:0/1430023606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b5c8406ea0 0x55b5c8422ab0 unknown :-1 s=CLOSED pgs=1736 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.487+0000 7fbb3effd640 0 client.19422 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:05.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.487+0000 7fbb68c1f740 1 --2- 172.21.15.86:0/1430023606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5c8406580 0x55b5c8422570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.487+0000 7fbb68c1f740 1 -- 172.21.15.86:0/1430023606 >> 172.21.15.86:0/1430023606 conn(0x55b5c8400cb0 msgr2=0x55b5c840d230 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:05.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.487+0000 7fbb68c1f740 1 -- 172.21.15.86:0/1430023606 shutdown_connections 2024-04-21T08:48:05.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.487+0000 7fbb68c1f740 1 -- 172.21.15.86:0/1430023606 wait complete. 2024-04-21T08:48:05.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.488+0000 7fbb68c1f740 20 client.19422 trim_cache size 0 max 16384 2024-04-21T08:48:05.494 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:48:05.494 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:48:05.494 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-21T08:48:05.494 INFO:teuthology.orchestra.run.smithi086.stdout: "e4f14c95-6c86-408f-a297-e8d639deadba": { 2024-04-21T08:48:05.494 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-21T08:48:05.494 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "e4f14c95-6c86-408f-a297-e8d639deadba", 2024-04-21T08:48:05.495 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-21T08:48:05.495 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:48:05.495 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:48:05.495 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:48:05.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.488+0000 7fbb65f59640 1 -- 172.21.15.86:0/3764009627 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbb480a3d20 msgr2=0x7fbb480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:05.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.488+0000 7fbb65f59640 1 --2- 172.21.15.86:0/3764009627 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbb480a3d20 0x7fbb480a61e0 secure :-1 s=READY pgs=284 cs=0 l=1 rev1=1 crypto rx=0x7fbb58009fd0 tx=0x7fbb58009500 comp rx=0 tx=0).stop 2024-04-21T08:48:05.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.488+0000 7fbb65f59640 1 -- 172.21.15.86:0/3764009627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb6006a5d0 msgr2=0x7fbb60198680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:05.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.488+0000 7fbb65f59640 1 --2- 172.21.15.86:0/3764009627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb6006a5d0 0x7fbb60198680 secure :-1 s=READY pgs=1732 cs=0 l=1 rev1=1 crypto rx=0x7fbb5000c920 tx=0x7fbb5000cde0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.488+0000 7fbb65f59640 1 -- 172.21.15.86:0/3764009627 shutdown_connections 2024-04-21T08:48:05.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.488+0000 7fbb65f59640 1 --2- 172.21.15.86:0/3764009627 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbb480a3d20 0x7fbb480a61e0 unknown :-1 s=CLOSED pgs=284 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.488+0000 7fbb65f59640 1 --2- 172.21.15.86:0/3764009627 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbb60108020 0x7fbb6019cf40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.488+0000 7fbb65f59640 1 --2- 172.21.15.86:0/3764009627 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbb6006af70 0x7fbb60198bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.488+0000 7fbb65f59640 1 --2- 172.21.15.86:0/3764009627 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbb6006a5d0 0x7fbb60198680 unknown :-1 s=CLOSED pgs=1732 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.488+0000 7fbb65f59640 1 -- 172.21.15.86:0/3764009627 >> 172.21.15.86:0/3764009627 conn(0x7fbb60076c50 msgr2=0x7fbb600ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:05.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.488+0000 7fbb65f59640 1 -- 172.21.15.86:0/3764009627 shutdown_connections 2024-04-21T08:48:05.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:05.488+0000 7fbb65f59640 1 -- 172.21.15.86:0/3764009627 wait complete. 2024-04-21T08:48:05.498 INFO:tasks.cephfs.filesystem:scrub status for tag:e4f14c95-6c86-408f-a297-e8d639deadba - {'path': '/', 'tag': 'e4f14c95-6c86-408f-a297-e8d639deadba', 'options': 'recursive,force'} 2024-04-21T08:48:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.796+0000 7fb867481640 1 -- 172.21.15.86:0/3278683220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8601021d0 msgr2=0x7fb8601025b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.796+0000 7fb867481640 1 --2- 172.21.15.86:0/3278683220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8601021d0 0x7fb8601025b0 secure :-1 s=READY pgs=1737 cs=0 l=1 rev1=1 crypto rx=0x7fb854009900 tx=0x7fb85402bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.796+0000 7fb867481640 1 -- 172.21.15.86:0/3278683220 shutdown_connections 2024-04-21T08:48:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.796+0000 7fb867481640 1 --2- 172.21.15.86:0/3278683220 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8601072a0 0x7fb86010ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.796+0000 7fb867481640 1 --2- 172.21.15.86:0/3278683220 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb860102b80 0x7fb860106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.796+0000 7fb867481640 1 --2- 172.21.15.86:0/3278683220 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb8601021d0 0x7fb8601025b0 unknown :-1 s=CLOSED pgs=1737 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.796+0000 7fb867481640 1 -- 172.21.15.86:0/3278683220 >> 172.21.15.86:0/3278683220 conn(0x7fb8600793e0 msgr2=0x7fb8600797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.796+0000 7fb867481640 1 -- 172.21.15.86:0/3278683220 shutdown_connections 2024-04-21T08:48:05.798 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.796+0000 7fb867481640 1 -- 172.21.15.86:0/3278683220 wait complete. 2024-04-21T08:48:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.796+0000 7fb867481640 1 Processor -- start 2024-04-21T08:48:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.797+0000 7fb867481640 1 -- start start 2024-04-21T08:48:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.797+0000 7fb867481640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8601021d0 0x7fb8601985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.797+0000 7fb867481640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8601072a0 0x7fb860198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.797+0000 7fb867481640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb86019cea0 0x7fb86019d350 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.797+0000 7fb867481640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb8601121e0 con 0x7fb86019cea0 2024-04-21T08:48:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.797+0000 7fb867481640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb860112060 con 0x7fb8601072a0 2024-04-21T08:48:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.797+0000 7fb867481640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb860112360 con 0x7fb8601021d0 2024-04-21T08:48:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.797+0000 7fb865a1e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb86019cea0 0x7fb86019d350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.797+0000 7fb865a1e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb86019cea0 0x7fb86019d350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48520/0 (socket says 172.21.15.86:48520) 2024-04-21T08:48:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.797+0000 7fb865a1e640 1 -- 172.21.15.86:0/3723444277 learned_addr learned my addr 172.21.15.86:0/3723444277 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.797+0000 7fb864a1c640 1 --2- 172.21.15.86:0/3723444277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8601072a0 0x7fb860198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.799 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.797+0000 7fb86521d640 1 --2- 172.21.15.86:0/3723444277 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8601021d0 0x7fb8601985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.797+0000 7fb865a1e640 1 -- 172.21.15.86:0/3723444277 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8601021d0 msgr2=0x7fb8601985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.797+0000 7fb865a1e640 1 --2- 172.21.15.86:0/3723444277 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8601021d0 0x7fb8601985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.797+0000 7fb865a1e640 1 -- 172.21.15.86:0/3723444277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8601072a0 msgr2=0x7fb860198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.797+0000 7fb865a1e640 1 --2- 172.21.15.86:0/3723444277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8601072a0 0x7fb860198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.797+0000 7fb865a1e640 1 -- 172.21.15.86:0/3723444277 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb86019d990 con 0x7fb86019cea0 2024-04-21T08:48:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.797+0000 7fb864a1c640 1 --2- 172.21.15.86:0/3723444277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8601072a0 0x7fb860198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.797+0000 7fb865a1e640 1 --2- 172.21.15.86:0/3723444277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb86019cea0 0x7fb86019d350 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fb85800e960 tx=0x7fb85800ee20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.797+0000 7fb8527fc640 1 -- 172.21.15.86:0/3723444277 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb858004d20 con 0x7fb86019cea0 2024-04-21T08:48:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.797+0000 7fb8527fc640 1 -- 172.21.15.86:0/3723444277 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb858005020 con 0x7fb86019cea0 2024-04-21T08:48:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.797+0000 7fb867481640 1 -- 172.21.15.86:0/3723444277 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb86019dc80 con 0x7fb86019cea0 2024-04-21T08:48:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.797+0000 7fb8527fc640 1 -- 172.21.15.86:0/3723444277 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8580116d0 con 0x7fb86019cea0 2024-04-21T08:48:05.800 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.798+0000 7fb867481640 1 -- 172.21.15.86:0/3723444277 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb8601a5420 con 0x7fb86019cea0 2024-04-21T08:48:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.800+0000 7fb8527fc640 1 -- 172.21.15.86:0/3723444277 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb8580051d0 con 0x7fb86019cea0 2024-04-21T08:48:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.801+0000 7fb86a07a740 1 Processor -- start 2024-04-21T08:48:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.801+0000 7fb86a07a740 1 -- start start 2024-04-21T08:48:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.801+0000 7fb86a07a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b38fd8d0 0x55d3b38fc620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.801+0000 7fb86a07a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b38fcb60 0x55d3b38fcfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.801+0000 7fb86a07a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d3b39060e0 0x55d3b39084d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.801+0000 7fb86a07a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d3b38618d0 con 0x55d3b38fcb60 2024-04-21T08:48:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.801+0000 7fb86a07a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d3b37fee80 con 0x55d3b39060e0 2024-04-21T08:48:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.801+0000 7fb86a07a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d3b385d890 con 0x55d3b38fd8d0 2024-04-21T08:48:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.801+0000 7fb8527fc640 1 --2- 172.21.15.86:0/3723444277 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb84c0a3dc0 0x7fb84c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.801+0000 7fb86521d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b38fcb60 0x55d3b38fcfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.801+0000 7fb86521d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b38fcb60 0x55d3b38fcfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48522/0 (socket says 172.21.15.86:48522) 2024-04-21T08:48:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.801+0000 7fb86521d640 1 -- 172.21.15.86:0/343502248 learned_addr learned my addr 172.21.15.86:0/343502248 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.801+0000 7fb865a1e640 1 --2- 172.21.15.86:0/3723444277 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb84c0a3dc0 0x7fb84c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.801+0000 7fb864a1c640 1 --2- 172.21.15.86:0/343502248 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d3b39060e0 0x55d3b39084d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.801+0000 7fb864a1c640 1 --2- 172.21.15.86:0/343502248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b38fd8d0 0x55d3b38fc620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.808 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.801+0000 7fb86521d640 1 -- 172.21.15.86:0/343502248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b38fd8d0 msgr2=0x55d3b38fc620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:05.809 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.801+0000 7fb8527fc640 1 -- 172.21.15.86:0/3723444277 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(101..101 src has 1..101) ==== 9584+0+0 (secure 0 0 0) 0x7fb8580d0de0 con 0x7fb86019cea0 2024-04-21T08:48:05.809 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.801+0000 7fb86521d640 1 --2- 172.21.15.86:0/343502248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b38fd8d0 0x55d3b38fc620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.809 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.801+0000 7fb86521d640 1 -- 172.21.15.86:0/343502248 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d3b39060e0 msgr2=0x55d3b39084d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:05.809 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.801+0000 7fb86521d640 1 --2- 172.21.15.86:0/343502248 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d3b39060e0 0x55d3b39084d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.809 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.801+0000 7fb86521d640 1 -- 172.21.15.86:0/343502248 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3b3820320 con 0x55d3b38fcb60 2024-04-21T08:48:05.809 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.801+0000 7fb86521d640 1 --2- 172.21.15.86:0/343502248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b38fcb60 0x55d3b38fcfe0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fb854009900 tx=0x7fb85402c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a6d9853e0a11f642 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:05.809 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.801+0000 7fb865a1e640 1 --2- 172.21.15.86:0/3723444277 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb84c0a3dc0 0x7fb84c0a6280 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb858099330 tx=0x7fb85806f450 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:05.809 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.802+0000 7fb84b7fe640 1 -- 172.21.15.86:0/343502248 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb854044020 con 0x55d3b38fcb60 2024-04-21T08:48:05.809 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.802+0000 7fb84b7fe640 1 -- 172.21.15.86:0/343502248 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb85402cc90 con 0x55d3b38fcb60 2024-04-21T08:48:05.809 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.802+0000 7fb84b7fe640 1 -- 172.21.15.86:0/343502248 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb8540046c0 con 0x55d3b38fcb60 2024-04-21T08:48:05.809 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.802+0000 7fb86a07a740 1 -- 172.21.15.86:0/343502248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b38fcb60 msgr2=0x55d3b38fcfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.802+0000 7fb86a07a740 1 --2- 172.21.15.86:0/343502248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b38fcb60 0x55d3b38fcfe0 secure :-1 s=READY pgs=1739 cs=0 l=1 rev1=1 crypto rx=0x7fb854009900 tx=0x7fb85402c650 comp rx=0 tx=0).stop 2024-04-21T08:48:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.802+0000 7fb86a07a740 1 -- 172.21.15.86:0/343502248 shutdown_connections 2024-04-21T08:48:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.802+0000 7fb86a07a740 1 --2- 172.21.15.86:0/343502248 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d3b39060e0 0x55d3b39084d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.802+0000 7fb86a07a740 1 --2- 172.21.15.86:0/343502248 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b38fcb60 0x55d3b38fcfe0 unknown :-1 s=CLOSED pgs=1739 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.802+0000 7fb86a07a740 1 --2- 172.21.15.86:0/343502248 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b38fd8d0 0x55d3b38fc620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.802+0000 7fb86a07a740 1 -- 172.21.15.86:0/343502248 >> 172.21.15.86:0/343502248 conn(0x55d3b38c3c90 msgr2=0x55d3b38f7850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.802+0000 7fb86a07a740 1 -- 172.21.15.86:0/343502248 shutdown_connections 2024-04-21T08:48:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.802+0000 7fb86a07a740 1 -- 172.21.15.86:0/343502248 wait complete. 2024-04-21T08:48:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.803+0000 7fb86a07a740 1 -- start start 2024-04-21T08:48:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.803+0000 7fb86a07a740 1 Processor -- start 2024-04-21T08:48:05.810 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.803+0000 7fb86a07a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b38fcb60 0x55d3b39059c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.803+0000 7fb86a07a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b38fd8d0 0x55d3b3902dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.803+0000 7fb86a07a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d3b39060e0 0x55d3b3903300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.803+0000 7fb86a07a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d3b3908db0 con 0x55d3b38fcb60 2024-04-21T08:48:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.803+0000 7fb86a07a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d3b37e9eb0 con 0x55d3b39060e0 2024-04-21T08:48:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.803+0000 7fb86a07a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d3b3908f30 con 0x55d3b38fd8d0 2024-04-21T08:48:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.803+0000 7fb86a07a740 10 client.0 _ll_register_callbacks cb 0x55d3b3816b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.803+0000 7fb864a1c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b38fcb60 0x55d3b39059c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.803+0000 7fb864a1c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b38fcb60 0x55d3b39059c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48534/0 (socket says 172.21.15.86:48534) 2024-04-21T08:48:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.803+0000 7fb864a1c640 1 -- 172.21.15.86:0/4191649753 learned_addr learned my addr 172.21.15.86:0/4191649753 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.804+0000 7fb864a1c640 1 --2- 172.21.15.86:0/4191649753 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d3b39060e0 0x55d3b3903300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.811 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.804+0000 7fb86521d640 1 --2- 172.21.15.86:0/4191649753 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b38fd8d0 0x55d3b3902dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.812 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.804+0000 7fb864a1c640 1 -- 172.21.15.86:0/4191649753 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b38fd8d0 msgr2=0x55d3b3902dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:05.812 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.804+0000 7fb864a1c640 1 --2- 172.21.15.86:0/4191649753 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b38fd8d0 0x55d3b3902dc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.812 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.804+0000 7fb864a1c640 1 -- 172.21.15.86:0/4191649753 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d3b39060e0 msgr2=0x55d3b3903300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:05.812 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.804+0000 7fb864a1c640 1 --2- 172.21.15.86:0/4191649753 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d3b39060e0 0x55d3b3903300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.812 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.804+0000 7fb864a1c640 1 -- 172.21.15.86:0/4191649753 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3b3574210 con 0x55d3b38fcb60 2024-04-21T08:48:05.812 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.804+0000 7fb864a1c640 1 --2- 172.21.15.86:0/4191649753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b38fcb60 0x55d3b39059c0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fb85c00bed0 tx=0x7fb85c015040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3187a7d63c19fd9c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:05.812 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.804+0000 7fb848ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:05.812 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.804+0000 7fb848ff9640 1 -- 172.21.15.86:0/4191649753 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c004260 con 0x55d3b38fcb60 2024-04-21T08:48:05.812 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.804+0000 7fb848ff9640 1 -- 172.21.15.86:0/4191649753 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9584+0+0 (secure 0 0 0) 0x7fb85c004400 con 0x55d3b38fcb60 2024-04-21T08:48:05.812 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.804+0000 7fb86a07a740 1 -- 172.21.15.86:0/4191649753 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb85402cc90 con 0x55d3b38fcb60 2024-04-21T08:48:05.813 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.804+0000 7fb848ff9640 1 -- 172.21.15.86:0/4191649753 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb85c025050 con 0x55d3b38fcb60 2024-04-21T08:48:05.813 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.804+0000 7fb86a07a740 10 client.19434 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:05.813 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.804+0000 7fb86a07a740 1 -- 172.21.15.86:0/4191649753 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d3b3640620 con 0x55d3b38fcb60 2024-04-21T08:48:05.813 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.804+0000 7fb848ff9640 1 -- 172.21.15.86:0/4191649753 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb85402cc90 con 0x55d3b38fcb60 2024-04-21T08:48:05.813 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.804+0000 7fb848ff9640 1 -- 172.21.15.86:0/4191649753 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb85c0049d0 con 0x55d3b38fcb60 2024-04-21T08:48:05.813 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.804+0000 7fb86a07a740 10 client.19434 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:05.813 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.804+0000 7fb86a07a740 10 client.19434 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:05.813 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.804+0000 7fb86a07a740 20 client.19434 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.804+0000 7fb86a07a740 1 --2- 172.21.15.86:0/4191649753 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d3b3916090 0x55d3b39184c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.804+0000 7fb86a07a740 4 client.19434 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.804+0000 7fb86a07a740 1 -- 172.21.15.86:0/4191649753 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3b3875880 con 0x55d3b3916090 2024-04-21T08:48:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.805+0000 7fb86521d640 1 --2- 172.21.15.86:0/4191649753 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d3b3916090 0x55d3b39184c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.805+0000 7fb86521d640 1 --2- 172.21.15.86:0/4191649753 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d3b3916090 0x55d3b39184c0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1b3e36c83604cd0e server_cookie=e2627dca56da74bd in_seq=0 out_seq=0 2024-04-21T08:48:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.805+0000 7fb848ff9640 10 client.19434 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.806+0000 7fb848ff9640 1 -- 172.21.15.86:0/4191649753 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d3b3875880 con 0x55d3b3916090 2024-04-21T08:48:05.814 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.806+0000 7fb848ff9640 10 client.19434 handle_command_reply: tid=0 2024-04-21T08:48:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.806+0000 7fb86a07a740 1 client.19434 shutdown 2024-04-21T08:48:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.806+0000 7fb86a07a740 1 -- 172.21.15.86:0/4191649753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b38fcb60 msgr2=0x55d3b39059c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.806+0000 7fb86a07a740 1 --2- 172.21.15.86:0/4191649753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b38fcb60 0x55d3b39059c0 secure :-1 s=READY pgs=1740 cs=0 l=1 rev1=1 crypto rx=0x7fb85c00bed0 tx=0x7fb85c015040 comp rx=0 tx=0).stop 2024-04-21T08:48:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.806+0000 7fb86a07a740 1 -- 172.21.15.86:0/4191649753 shutdown_connections 2024-04-21T08:48:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.806+0000 7fb86a07a740 1 --2- 172.21.15.86:0/4191649753 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d3b3916090 0x55d3b39184c0 crc :-1 s=READY pgs=277 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.806+0000 7fb86a07a740 1 --2- 172.21.15.86:0/4191649753 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d3b39060e0 0x55d3b3903300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.806+0000 7fb86a07a740 1 --2- 172.21.15.86:0/4191649753 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b38fd8d0 0x55d3b3902dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.815 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.806+0000 7fb848ff9640 0 client.19434 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.806+0000 7fb86a07a740 1 --2- 172.21.15.86:0/4191649753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b38fcb60 0x55d3b39059c0 unknown :-1 s=CLOSED pgs=1740 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.806+0000 7fb86a07a740 1 -- 172.21.15.86:0/4191649753 >> 172.21.15.86:0/4191649753 conn(0x55d3b38c3c90 msgr2=0x55d3b38f95a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.806+0000 7fb86a07a740 1 -- 172.21.15.86:0/4191649753 shutdown_connections 2024-04-21T08:48:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.806+0000 7fb86a07a740 1 -- 172.21.15.86:0/4191649753 wait complete. 2024-04-21T08:48:05.816 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.806+0000 7fb86a07a740 20 client.19434 trim_cache size 0 max 16384 2024-04-21T08:48:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.821+0000 7fb86a07a740 1 Processor -- start 2024-04-21T08:48:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.821+0000 7fb86a07a740 1 -- start start 2024-04-21T08:48:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.821+0000 7fb86a07a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b38c3c90 0x55d3b3911cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.821+0000 7fb86a07a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d3b38fcb60 0x55d3b3902320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.821+0000 7fb86a07a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b38fd8d0 0x55d3b39054f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.821+0000 7fb86a07a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb8601af390 con 0x55d3b38fd8d0 2024-04-21T08:48:05.828 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.821+0000 7fb86a07a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb8601b0ed0 con 0x55d3b38fcb60 2024-04-21T08:48:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.821+0000 7fb86a07a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb8601afa60 con 0x55d3b38c3c90 2024-04-21T08:48:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.821+0000 7fb864a1c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b38fd8d0 0x55d3b39054f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.821+0000 7fb864a1c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b38fd8d0 0x55d3b39054f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48550/0 (socket says 172.21.15.86:48550) 2024-04-21T08:48:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.821+0000 7fb864a1c640 1 -- 172.21.15.86:0/1884374991 learned_addr learned my addr 172.21.15.86:0/1884374991 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.821+0000 7fb864a1c640 1 --2- 172.21.15.86:0/1884374991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b38c3c90 0x55d3b3911cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.821+0000 7fb86521d640 1 --2- 172.21.15.86:0/1884374991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d3b38fcb60 0x55d3b3902320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.821+0000 7fb864a1c640 1 -- 172.21.15.86:0/1884374991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b38c3c90 msgr2=0x55d3b3911cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:05.829 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.821+0000 7fb864a1c640 1 --2- 172.21.15.86:0/1884374991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b38c3c90 0x55d3b3911cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.821+0000 7fb864a1c640 1 -- 172.21.15.86:0/1884374991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d3b38fcb60 msgr2=0x55d3b3902320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.821+0000 7fb864a1c640 1 --2- 172.21.15.86:0/1884374991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d3b38fcb60 0x55d3b3902320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.821+0000 7fb864a1c640 1 -- 172.21.15.86:0/1884374991 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d3b38fbe30 con 0x55d3b38fd8d0 2024-04-21T08:48:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.821+0000 7fb86521d640 1 --2- 172.21.15.86:0/1884374991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d3b38fcb60 0x55d3b3902320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.821+0000 7fb864a1c640 1 --2- 172.21.15.86:0/1884374991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b38fd8d0 0x55d3b39054f0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fb85c010d90 tx=0x7fb85c00b460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=42cdf71534fa7631 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.821+0000 7fb84affd640 1 -- 172.21.15.86:0/1884374991 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c00b780 con 0x55d3b38fd8d0 2024-04-21T08:48:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.821+0000 7fb84affd640 1 -- 172.21.15.86:0/1884374991 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb85c00b920 con 0x55d3b38fd8d0 2024-04-21T08:48:05.830 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.821+0000 7fb84affd640 1 -- 172.21.15.86:0/1884374991 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c003c90 con 0x55d3b38fd8d0 2024-04-21T08:48:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.822+0000 7fb86a07a740 1 -- 172.21.15.86:0/1884374991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b38fd8d0 msgr2=0x55d3b39054f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.822+0000 7fb86a07a740 1 --2- 172.21.15.86:0/1884374991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b38fd8d0 0x55d3b39054f0 secure :-1 s=READY pgs=1741 cs=0 l=1 rev1=1 crypto rx=0x7fb85c010d90 tx=0x7fb85c00b460 comp rx=0 tx=0).stop 2024-04-21T08:48:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.822+0000 7fb86a07a740 1 -- 172.21.15.86:0/1884374991 shutdown_connections 2024-04-21T08:48:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.822+0000 7fb86a07a740 1 --2- 172.21.15.86:0/1884374991 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b38fd8d0 0x55d3b39054f0 unknown :-1 s=CLOSED pgs=1741 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.822+0000 7fb86a07a740 1 --2- 172.21.15.86:0/1884374991 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d3b38fcb60 0x55d3b3902320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.822+0000 7fb86a07a740 1 --2- 172.21.15.86:0/1884374991 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b38c3c90 0x55d3b3911cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.822+0000 7fb86a07a740 1 -- 172.21.15.86:0/1884374991 >> 172.21.15.86:0/1884374991 conn(0x55d3b3903460 msgr2=0x55d3b38fe060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:05.831 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.822+0000 7fb86a07a740 1 -- 172.21.15.86:0/1884374991 shutdown_connections 2024-04-21T08:48:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.822+0000 7fb86a07a740 1 -- 172.21.15.86:0/1884374991 wait complete. 2024-04-21T08:48:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.822+0000 7fb86a07a740 1 -- start start 2024-04-21T08:48:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.822+0000 7fb86a07a740 1 Processor -- start 2024-04-21T08:48:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.823+0000 7fb86a07a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b38c3c90 0x55d3b390e3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.823+0000 7fb86a07a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d3b38fcb60 0x55d3b3900060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.823+0000 7fb86a07a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b38fd8d0 0x55d3b39005a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.823+0000 7fb86a07a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb8601b0800 con 0x55d3b38c3c90 2024-04-21T08:48:05.832 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.823+0000 7fb86a07a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb8601b0130 con 0x55d3b38fcb60 2024-04-21T08:48:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.823+0000 7fb86a07a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb8601ad370 con 0x55d3b38fd8d0 2024-04-21T08:48:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.823+0000 7fb86a07a740 10 client.0 _ll_register_callbacks cb 0x7fb85c013bb0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.823+0000 7fb864a1c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b38c3c90 0x55d3b390e3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.823+0000 7fb864a1c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b38c3c90 0x55d3b390e3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:48560/0 (socket says 172.21.15.86:48560) 2024-04-21T08:48:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.823+0000 7fb864a1c640 1 -- 172.21.15.86:0/1599681873 learned_addr learned my addr 172.21.15.86:0/1599681873 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.823+0000 7fb86521d640 1 --2- 172.21.15.86:0/1599681873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d3b38fcb60 0x55d3b3900060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.823+0000 7fb864a1c640 1 --2- 172.21.15.86:0/1599681873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b38fd8d0 0x55d3b39005a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.833 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.823+0000 7fb864a1c640 1 -- 172.21.15.86:0/1599681873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b38fd8d0 msgr2=0x55d3b39005a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.823+0000 7fb864a1c640 1 --2- 172.21.15.86:0/1599681873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b38fd8d0 0x55d3b39005a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.823+0000 7fb864a1c640 1 -- 172.21.15.86:0/1599681873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d3b38fcb60 msgr2=0x55d3b3900060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.823+0000 7fb864a1c640 1 --2- 172.21.15.86:0/1599681873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d3b38fcb60 0x55d3b3900060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.823+0000 7fb864a1c640 1 -- 172.21.15.86:0/1599681873 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3b3900be0 con 0x55d3b38c3c90 2024-04-21T08:48:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.823+0000 7fb864a1c640 1 --2- 172.21.15.86:0/1599681873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b38c3c90 0x55d3b390e3b0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fb85c004490 tx=0x7fb85c00bad0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f4237907b933e0e0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.823+0000 7fb848ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.823+0000 7fb848ff9640 1 -- 172.21.15.86:0/1599681873 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb85c003c80 con 0x55d3b38c3c90 2024-04-21T08:48:05.834 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.823+0000 7fb848ff9640 1 -- 172.21.15.86:0/1599681873 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(101..101 src has 1..101) ==== 9584+0+0 (secure 0 0 0) 0x7fb85c003e20 con 0x55d3b38c3c90 2024-04-21T08:48:05.835 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.823+0000 7fb86a07a740 1 -- 172.21.15.86:0/1599681873 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb85c00b920 con 0x55d3b38c3c90 2024-04-21T08:48:05.835 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.824+0000 7fb848ff9640 1 -- 172.21.15.86:0/1599681873 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb85c0045d0 con 0x55d3b38c3c90 2024-04-21T08:48:05.835 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.824+0000 7fb86a07a740 10 client.19440 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:05.835 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.824+0000 7fb86a07a740 1 -- 172.21.15.86:0/1599681873 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d3b3907450 con 0x55d3b38c3c90 2024-04-21T08:48:05.835 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.824+0000 7fb848ff9640 1 -- 172.21.15.86:0/1599681873 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb85c00b920 con 0x55d3b38c3c90 2024-04-21T08:48:05.835 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.824+0000 7fb848ff9640 1 -- 172.21.15.86:0/1599681873 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb85c037040 con 0x55d3b38c3c90 2024-04-21T08:48:05.835 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.824+0000 7fb86a07a740 10 client.19440 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:05.835 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.824+0000 7fb86a07a740 10 client.19440 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.824+0000 7fb86a07a740 20 client.19440 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.824+0000 7fb86a07a740 1 --2- 172.21.15.86:0/1599681873 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d3b39060e0 0x55d3b397e980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.824+0000 7fb86a07a740 4 client.19440 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:48:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.824+0000 7fb86a07a740 1 -- 172.21.15.86:0/1599681873 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d3b38013d0 con 0x55d3b39060e0 2024-04-21T08:48:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.824+0000 7fb86521d640 1 --2- 172.21.15.86:0/1599681873 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d3b39060e0 0x55d3b397e980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.825+0000 7fb86521d640 1 --2- 172.21.15.86:0/1599681873 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d3b39060e0 0x55d3b397e980 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=11f2f8ab057bb67e server_cookie=fc6747bccf9448ac in_seq=0 out_seq=0 2024-04-21T08:48:05.836 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.825+0000 7fb848ff9640 10 client.19440 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.838+0000 7fb848ff9640 1 -- 172.21.15.86:0/1599681873 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d3b38013d0 con 0x55d3b39060e0 2024-04-21T08:48:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.838+0000 7fb848ff9640 10 client.19440 handle_command_reply: tid=0 2024-04-21T08:48:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.838+0000 7fb86a07a740 1 client.19440 shutdown 2024-04-21T08:48:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.838+0000 7fb86a07a740 1 -- 172.21.15.86:0/1599681873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b38c3c90 msgr2=0x55d3b390e3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.838+0000 7fb86a07a740 1 --2- 172.21.15.86:0/1599681873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b38c3c90 0x55d3b390e3b0 secure :-1 s=READY pgs=1742 cs=0 l=1 rev1=1 crypto rx=0x7fb85c004490 tx=0x7fb85c00bad0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.839+0000 7fb86a07a740 1 -- 172.21.15.86:0/1599681873 shutdown_connections 2024-04-21T08:48:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.839+0000 7fb86a07a740 1 --2- 172.21.15.86:0/1599681873 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d3b39060e0 0x55d3b397e980 crc :-1 s=READY pgs=278 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.839+0000 7fb86a07a740 1 --2- 172.21.15.86:0/1599681873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d3b38fd8d0 0x55d3b39005a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.839 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.839+0000 7fb848ff9640 0 client.19440 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.839+0000 7fb86a07a740 1 --2- 172.21.15.86:0/1599681873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d3b38fcb60 0x55d3b3900060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.839+0000 7fb86a07a740 1 --2- 172.21.15.86:0/1599681873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d3b38c3c90 0x55d3b390e3b0 unknown :-1 s=CLOSED pgs=1742 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.839+0000 7fb86a07a740 1 -- 172.21.15.86:0/1599681873 >> 172.21.15.86:0/1599681873 conn(0x55d3b3903460 msgr2=0x55d3b3901670 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.839+0000 7fb86a07a740 1 -- 172.21.15.86:0/1599681873 shutdown_connections 2024-04-21T08:48:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.839+0000 7fb86a07a740 1 -- 172.21.15.86:0/1599681873 wait complete. 2024-04-21T08:48:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.839+0000 7fb86a07a740 20 client.19440 trim_cache size 0 max 16384 2024-04-21T08:48:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:48:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:48:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:48:05.840 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:48:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.839+0000 7fb867481640 1 -- 172.21.15.86:0/3723444277 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb84c0a3dc0 msgr2=0x7fb84c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.839+0000 7fb867481640 1 --2- 172.21.15.86:0/3723444277 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb84c0a3dc0 0x7fb84c0a6280 secure :-1 s=READY pgs=285 cs=0 l=1 rev1=1 crypto rx=0x7fb858099330 tx=0x7fb85806f450 comp rx=0 tx=0).stop 2024-04-21T08:48:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.839+0000 7fb867481640 1 -- 172.21.15.86:0/3723444277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb86019cea0 msgr2=0x7fb86019d350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:05.841 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.839+0000 7fb867481640 1 --2- 172.21.15.86:0/3723444277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb86019cea0 0x7fb86019d350 secure :-1 s=READY pgs=1738 cs=0 l=1 rev1=1 crypto rx=0x7fb85800e960 tx=0x7fb85800ee20 comp rx=0 tx=0).stop 2024-04-21T08:48:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.840+0000 7fb867481640 1 -- 172.21.15.86:0/3723444277 shutdown_connections 2024-04-21T08:48:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.840+0000 7fb867481640 1 --2- 172.21.15.86:0/3723444277 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb84c0a3dc0 0x7fb84c0a6280 unknown :-1 s=CLOSED pgs=285 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.840+0000 7fb867481640 1 --2- 172.21.15.86:0/3723444277 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb86019cea0 0x7fb86019d350 unknown :-1 s=CLOSED pgs=1738 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.840+0000 7fb867481640 1 --2- 172.21.15.86:0/3723444277 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb8601072a0 0x7fb860198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.840+0000 7fb867481640 1 --2- 172.21.15.86:0/3723444277 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb8601021d0 0x7fb8601985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.840+0000 7fb867481640 1 -- 172.21.15.86:0/3723444277 >> 172.21.15.86:0/3723444277 conn(0x7fb8600793e0 msgr2=0x7fb8600ffc30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.840+0000 7fb867481640 1 -- 172.21.15.86:0/3723444277 shutdown_connections 2024-04-21T08:48:05.842 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:05.840+0000 7fb867481640 1 -- 172.21.15.86:0/3723444277 wait complete. 2024-04-21T08:48:06.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:05 smithi088 ceph-mon[31259]: pgmap v288: 129 pgs: 129 active+clean; 5.2 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 80 KiB/s wr, 29 op/s 2024-04-21T08:48:06.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:05 smithi088 ceph-mon[31259]: osdmap e101: 12 total, 12 up, 12 in 2024-04-21T08:48:06.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:05 smithi088 ceph-mon[31259]: scrub queued for path: / 2024-04-21T08:48:06.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:05 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:06.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:05 smithi088 ceph-mon[31259]: scrub summary: active paths [/] 2024-04-21T08:48:06.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:05 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:06.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:05 smithi179 ceph-mon[31829]: pgmap v288: 129 pgs: 129 active+clean; 5.2 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 80 KiB/s wr, 29 op/s 2024-04-21T08:48:06.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:05 smithi179 ceph-mon[31829]: osdmap e101: 12 total, 12 up, 12 in 2024-04-21T08:48:06.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:05 smithi179 ceph-mon[31829]: scrub queued for path: / 2024-04-21T08:48:06.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:05 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:06.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:05 smithi179 ceph-mon[31829]: scrub summary: active paths [/] 2024-04-21T08:48:06.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:05 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:06.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:05 smithi086 ceph-mon[27939]: pgmap v288: 129 pgs: 129 active+clean; 5.2 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 80 KiB/s wr, 29 op/s 2024-04-21T08:48:06.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:05 smithi086 ceph-mon[27939]: osdmap e101: 12 total, 12 up, 12 in 2024-04-21T08:48:06.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:05 smithi086 ceph-mon[27939]: scrub queued for path: / 2024-04-21T08:48:06.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:05 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:06.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:05 smithi086 ceph-mon[27939]: scrub summary: active paths [/] 2024-04-21T08:48:06.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:05 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:08.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:07 smithi088 ceph-mon[31259]: pgmap v290: 129 pgs: 129 active+clean; 5.2 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 97 KiB/s wr, 35 op/s 2024-04-21T08:48:08.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:07 smithi179 ceph-mon[31829]: pgmap v290: 129 pgs: 129 active+clean; 5.2 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 97 KiB/s wr, 35 op/s 2024-04-21T08:48:08.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:07 smithi086 ceph-mon[27939]: pgmap v290: 129 pgs: 129 active+clean; 5.2 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 64 KiB/s rd, 97 KiB/s wr, 35 op/s 2024-04-21T08:48:10.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:09 smithi088 ceph-mon[31259]: pgmap v291: 129 pgs: 129 active+clean; 1.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 107 KiB/s wr, 41 op/s 2024-04-21T08:48:10.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:09 smithi088 ceph-mon[31259]: scrub summary: idle 2024-04-21T08:48:10.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:09 smithi179 ceph-mon[31829]: pgmap v291: 129 pgs: 129 active+clean; 1.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 107 KiB/s wr, 41 op/s 2024-04-21T08:48:10.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:09 smithi179 ceph-mon[31829]: scrub summary: idle 2024-04-21T08:48:10.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:09 smithi086 ceph-mon[27939]: pgmap v291: 129 pgs: 129 active+clean; 1.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 68 KiB/s rd, 107 KiB/s wr, 41 op/s 2024-04-21T08:48:10.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:09 smithi086 ceph-mon[27939]: scrub summary: idle 2024-04-21T08:48:11.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:11 smithi086 ceph-mon[27939]: osdmap e102: 12 total, 12 up, 12 in 2024-04-21T08:48:11.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:11 smithi088 ceph-mon[31259]: osdmap e102: 12 total, 12 up, 12 in 2024-04-21T08:48:11.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:11 smithi179 ceph-mon[31829]: osdmap e102: 12 total, 12 up, 12 in 2024-04-21T08:48:12.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:12 smithi086 ceph-mon[27939]: pgmap v293: 129 pgs: 129 active+clean; 1.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 16 op/s 2024-04-21T08:48:12.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:12 smithi088 ceph-mon[31259]: pgmap v293: 129 pgs: 129 active+clean; 1.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 16 op/s 2024-04-21T08:48:12.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:12 smithi179 ceph-mon[31829]: pgmap v293: 129 pgs: 129 active+clean; 1.5 MiB data, 416 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 38 KiB/s wr, 16 op/s 2024-04-21T08:48:13.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:13 smithi086 ceph-mon[27939]: pgmap v294: 129 pgs: 129 active+clean; 1.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 63 KiB/s wr, 25 op/s 2024-04-21T08:48:13.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:13 smithi088 ceph-mon[31259]: pgmap v294: 129 pgs: 129 active+clean; 1.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 63 KiB/s wr, 25 op/s 2024-04-21T08:48:13.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:13 smithi179 ceph-mon[31829]: pgmap v294: 129 pgs: 129 active+clean; 1.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 63 KiB/s wr, 25 op/s 2024-04-21T08:48:16.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:15 smithi088 ceph-mon[31259]: pgmap v295: 129 pgs: 129 active+clean; 1.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 51 KiB/s wr, 20 op/s 2024-04-21T08:48:16.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:15 smithi179 ceph-mon[31829]: pgmap v295: 129 pgs: 129 active+clean; 1.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 51 KiB/s wr, 20 op/s 2024-04-21T08:48:16.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:15 smithi086 ceph-mon[27939]: pgmap v295: 129 pgs: 129 active+clean; 1.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 51 KiB/s wr, 20 op/s 2024-04-21T08:48:18.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:17 smithi088 ceph-mon[31259]: pgmap v296: 129 pgs: 129 active+clean; 1.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 50 KiB/s wr, 20 op/s 2024-04-21T08:48:18.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:17 smithi179 ceph-mon[31829]: pgmap v296: 129 pgs: 129 active+clean; 1.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 50 KiB/s wr, 20 op/s 2024-04-21T08:48:18.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:17 smithi086 ceph-mon[27939]: pgmap v296: 129 pgs: 129 active+clean; 1.7 MiB data, 417 MiB used, 1.0 TiB / 1.0 TiB avail; 52 KiB/s rd, 50 KiB/s wr, 20 op/s 2024-04-21T08:48:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.934+0000 7f11ff293640 1 -- 172.21.15.86:0/281521503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11f81010f0 msgr2=0x7f11f81014d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.934+0000 7f11ff293640 1 --2- 172.21.15.86:0/281521503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11f81010f0 0x7f11f81014d0 secure :-1 s=READY pgs=1743 cs=0 l=1 rev1=1 crypto rx=0x7f11f4009900 tx=0x7f11f402bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.934+0000 7f11ff293640 1 -- 172.21.15.86:0/281521503 shutdown_connections 2024-04-21T08:48:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.934+0000 7f11ff293640 1 --2- 172.21.15.86:0/281521503 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11f8106170 0x7f11f810dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.934+0000 7f11ff293640 1 --2- 172.21.15.86:0/281521503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11f8101aa0 0x7f11f8105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.934+0000 7f11ff293640 1 --2- 172.21.15.86:0/281521503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11f81010f0 0x7f11f81014d0 unknown :-1 s=CLOSED pgs=1743 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.934+0000 7f11ff293640 1 -- 172.21.15.86:0/281521503 >> 172.21.15.86:0/281521503 conn(0x7f11f8077fd0 msgr2=0x7f11f80783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.935+0000 7f11ff293640 1 -- 172.21.15.86:0/281521503 shutdown_connections 2024-04-21T08:48:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.935+0000 7f11ff293640 1 -- 172.21.15.86:0/281521503 wait complete. 2024-04-21T08:48:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.935+0000 7f11ff293640 1 Processor -- start 2024-04-21T08:48:18.937 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.935+0000 7f11ff293640 1 -- start start 2024-04-21T08:48:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.935+0000 7f11ff293640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11f81010f0 0x7f11f8197650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.935+0000 7f11ff293640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11f8106170 0x7f11f8197b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.935+0000 7f11ff293640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11f819bf10 0x7f11f819c3c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.935+0000 7f11ff293640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f11f81110d0 con 0x7f11f8106170 2024-04-21T08:48:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.935+0000 7f11ff293640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f11f8110f50 con 0x7f11f819bf10 2024-04-21T08:48:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.935+0000 7f11ff293640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f11f8111250 con 0x7f11f81010f0 2024-04-21T08:48:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.936+0000 7f11fc82e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11f8106170 0x7f11f8197b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.936+0000 7f11fc82e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11f8106170 0x7f11f8197b90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49684/0 (socket says 172.21.15.86:49684) 2024-04-21T08:48:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.936+0000 7f11fc82e640 1 -- 172.21.15.86:0/908582713 learned_addr learned my addr 172.21.15.86:0/908582713 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.936+0000 7f11fd02f640 1 --2- 172.21.15.86:0/908582713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11f81010f0 0x7f11f8197650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.936+0000 7f11fd830640 1 --2- 172.21.15.86:0/908582713 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11f819bf10 0x7f11f819c3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.936+0000 7f11fc82e640 1 -- 172.21.15.86:0/908582713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11f81010f0 msgr2=0x7f11f8197650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:18.938 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.936+0000 7f11fc82e640 1 --2- 172.21.15.86:0/908582713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11f81010f0 0x7f11f8197650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.936+0000 7f11fc82e640 1 -- 172.21.15.86:0/908582713 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11f819bf10 msgr2=0x7f11f819c3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.936+0000 7f11fc82e640 1 --2- 172.21.15.86:0/908582713 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11f819bf10 0x7f11f819c3c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.936+0000 7f11fc82e640 1 -- 172.21.15.86:0/908582713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f11f819ca00 con 0x7f11f8106170 2024-04-21T08:48:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.936+0000 7f11fd830640 1 --2- 172.21.15.86:0/908582713 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11f819bf10 0x7f11f819c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.936+0000 7f11fc82e640 1 --2- 172.21.15.86:0/908582713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11f8106170 0x7f11f8197b90 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f11e800e9f0 tx=0x7f11e800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.936+0000 7f11ee7fc640 1 -- 172.21.15.86:0/908582713 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e800ccc0 con 0x7f11f8106170 2024-04-21T08:48:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.936+0000 7f11ee7fc640 1 -- 172.21.15.86:0/908582713 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11e80054d0 con 0x7f11f8106170 2024-04-21T08:48:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.936+0000 7f11ff293640 1 -- 172.21.15.86:0/908582713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f11f819ccf0 con 0x7f11f8106170 2024-04-21T08:48:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.936+0000 7f11ee7fc640 1 -- 172.21.15.86:0/908582713 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e8010430 con 0x7f11f8106170 2024-04-21T08:48:18.939 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.936+0000 7f11ff293640 1 -- 172.21.15.86:0/908582713 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f11f81a4490 con 0x7f11f8106170 2024-04-21T08:48:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.939+0000 7f11ee7fc640 1 -- 172.21.15.86:0/908582713 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f11e80046c0 con 0x7f11f8106170 2024-04-21T08:48:18.946 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.939+0000 7f1201f64740 1 Processor -- start 2024-04-21T08:48:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.940+0000 7f1201f64740 1 -- start start 2024-04-21T08:48:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.940+0000 7f1201f64740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b31e2a2ae0 0x55b31e2a1830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.940+0000 7f1201f64740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b31e2a1d70 0x55b31e2a21f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.940+0000 7f1201f64740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b31e2ab2f0 0x55b31e2ad6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.940+0000 7f1201f64740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b31e2048d0 con 0x55b31e2a2ae0 2024-04-21T08:48:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.940+0000 7f1201f64740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b31e1a1e80 con 0x55b31e2a1d70 2024-04-21T08:48:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.940+0000 7f1201f64740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b31e200890 con 0x55b31e2ab2f0 2024-04-21T08:48:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.940+0000 7f11ee7fc640 1 --2- 172.21.15.86:0/908582713 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f11e40a3dc0 0x7f11e40a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.940+0000 7f11fd830640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b31e2a2ae0 0x55b31e2a1830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.940+0000 7f11fd830640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b31e2a2ae0 0x55b31e2a1830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49686/0 (socket says 172.21.15.86:49686) 2024-04-21T08:48:18.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.940+0000 7f11fd830640 1 -- 172.21.15.86:0/1958112696 learned_addr learned my addr 172.21.15.86:0/1958112696 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.940+0000 7f11ee7fc640 1 -- 172.21.15.86:0/908582713 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f11e80d0fd0 con 0x7f11f8106170 2024-04-21T08:48:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.940+0000 7f11fd830640 1 --2- 172.21.15.86:0/908582713 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f11e40a3dc0 0x7f11e40a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.940+0000 7f11fc82e640 1 --2- 172.21.15.86:0/1958112696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b31e2ab2f0 0x55b31e2ad6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.940+0000 7f11fd02f640 1 --2- 172.21.15.86:0/1958112696 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b31e2a1d70 0x55b31e2a21f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.940+0000 7f11fd830640 1 -- 172.21.15.86:0/1958112696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b31e2ab2f0 msgr2=0x55b31e2ad6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.940+0000 7f11fd830640 1 --2- 172.21.15.86:0/1958112696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b31e2ab2f0 0x55b31e2ad6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.940+0000 7f11fd830640 1 -- 172.21.15.86:0/1958112696 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b31e2a1d70 msgr2=0x55b31e2a21f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.940+0000 7f11fd830640 1 --2- 172.21.15.86:0/1958112696 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b31e2a1d70 0x55b31e2a21f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.940+0000 7f11fd830640 1 -- 172.21.15.86:0/1958112696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b31e1c3320 con 0x55b31e2a2ae0 2024-04-21T08:48:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.941+0000 7f11fd830640 1 --2- 172.21.15.86:0/1958112696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b31e2a2ae0 0x55b31e2a1830 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f11f0012d90 tx=0x7f11f00107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a7b78a6ce6e683bc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.941+0000 7f11fd830640 1 --2- 172.21.15.86:0/908582713 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f11e40a3dc0 0x7f11e40a6280 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f11f0010ca0 tx=0x7f11f001d000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.941+0000 7f11e37fe640 1 -- 172.21.15.86:0/1958112696 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f0012580 con 0x55b31e2a2ae0 2024-04-21T08:48:18.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.941+0000 7f11e37fe640 1 -- 172.21.15.86:0/1958112696 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11f0012870 con 0x55b31e2a2ae0 2024-04-21T08:48:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.941+0000 7f11e37fe640 1 -- 172.21.15.86:0/1958112696 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f0005420 con 0x55b31e2a2ae0 2024-04-21T08:48:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.941+0000 7f1201f64740 1 -- 172.21.15.86:0/1958112696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b31e2a2ae0 msgr2=0x55b31e2a1830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.941+0000 7f1201f64740 1 --2- 172.21.15.86:0/1958112696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b31e2a2ae0 0x55b31e2a1830 secure :-1 s=READY pgs=1745 cs=0 l=1 rev1=1 crypto rx=0x7f11f0012d90 tx=0x7f11f00107a0 comp rx=0 tx=0).stop 2024-04-21T08:48:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.941+0000 7f1201f64740 1 -- 172.21.15.86:0/1958112696 shutdown_connections 2024-04-21T08:48:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.941+0000 7f1201f64740 1 --2- 172.21.15.86:0/1958112696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b31e2ab2f0 0x55b31e2ad6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.941+0000 7f1201f64740 1 --2- 172.21.15.86:0/1958112696 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b31e2a1d70 0x55b31e2a21f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.941+0000 7f1201f64740 1 --2- 172.21.15.86:0/1958112696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b31e2a2ae0 0x55b31e2a1830 unknown :-1 s=CLOSED pgs=1745 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.941+0000 7f1201f64740 1 -- 172.21.15.86:0/1958112696 >> 172.21.15.86:0/1958112696 conn(0x55b31e266c90 msgr2=0x55b31e29ca60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.942+0000 7f1201f64740 1 -- 172.21.15.86:0/1958112696 shutdown_connections 2024-04-21T08:48:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.942+0000 7f1201f64740 1 -- 172.21.15.86:0/1958112696 wait complete. 2024-04-21T08:48:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.942+0000 7f1201f64740 1 -- start start 2024-04-21T08:48:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.942+0000 7f1201f64740 1 Processor -- start 2024-04-21T08:48:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.942+0000 7f1201f64740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b31e2a1d70 0x55b31e2aacd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.942+0000 7f1201f64740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b31e2a2ae0 0x55b31e2a80d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:18.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.942+0000 7f1201f64740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b31e2ab2f0 0x55b31e2a8750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.942+0000 7f1201f64740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b31e2adf60 con 0x55b31e2a2ae0 2024-04-21T08:48:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.942+0000 7f1201f64740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b31e18ceb0 con 0x55b31e2ab2f0 2024-04-21T08:48:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.942+0000 7f1201f64740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b31e2ae0e0 con 0x55b31e2a1d70 2024-04-21T08:48:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.942+0000 7f1201f64740 10 client.0 _ll_register_callbacks cb 0x55b31e1b9b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.943+0000 7f11fc82e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b31e2a2ae0 0x55b31e2a80d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.943+0000 7f11fc82e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b31e2a2ae0 0x55b31e2a80d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49694/0 (socket says 172.21.15.86:49694) 2024-04-21T08:48:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.943+0000 7f11fc82e640 1 -- 172.21.15.86:0/474991858 learned_addr learned my addr 172.21.15.86:0/474991858 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.943+0000 7f11fd830640 1 --2- 172.21.15.86:0/474991858 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b31e2ab2f0 0x55b31e2a8750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.943+0000 7f11fd02f640 1 --2- 172.21.15.86:0/474991858 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b31e2a1d70 0x55b31e2aacd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.943+0000 7f11fc82e640 1 -- 172.21.15.86:0/474991858 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b31e2a1d70 msgr2=0x55b31e2aacd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.943+0000 7f11fc82e640 1 --2- 172.21.15.86:0/474991858 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b31e2a1d70 0x55b31e2aacd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.943+0000 7f11fc82e640 1 -- 172.21.15.86:0/474991858 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b31e2ab2f0 msgr2=0x55b31e2a8750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.943+0000 7f11fc82e640 1 --2- 172.21.15.86:0/474991858 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b31e2ab2f0 0x55b31e2a8750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:18.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.943+0000 7f11fc82e640 1 -- 172.21.15.86:0/474991858 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b31df17210 con 0x55b31e2a2ae0 2024-04-21T08:48:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.943+0000 7f11fc82e640 1 --2- 172.21.15.86:0/474991858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b31e2a2ae0 0x55b31e2a80d0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f11e8099570 tx=0x7f11e809c7a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bec17812247d74b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.943+0000 7f11e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.943+0000 7f11e0ff9640 1 -- 172.21.15.86:0/474991858 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11e80b5070 con 0x55b31e2a2ae0 2024-04-21T08:48:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.943+0000 7f11e0ff9640 1 -- 172.21.15.86:0/474991858 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f11e80a5de0 con 0x55b31e2a2ae0 2024-04-21T08:48:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.943+0000 7f1201f64740 1 -- 172.21.15.86:0/474991858 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11f0012870 con 0x55b31e2a2ae0 2024-04-21T08:48:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.943+0000 7f11e0ff9640 1 -- 172.21.15.86:0/474991858 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f11e809d2a0 con 0x55b31e2a2ae0 2024-04-21T08:48:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.943+0000 7f1201f64740 10 client.19452 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.943+0000 7f1201f64740 1 -- 172.21.15.86:0/474991858 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b31dfe3620 con 0x55b31e2a2ae0 2024-04-21T08:48:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.943+0000 7f11e0ff9640 1 -- 172.21.15.86:0/474991858 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11f0012870 con 0x55b31e2a2ae0 2024-04-21T08:48:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.944+0000 7f11e0ff9640 1 -- 172.21.15.86:0/474991858 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f11e80a5380 con 0x55b31e2a2ae0 2024-04-21T08:48:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.944+0000 7f1201f64740 10 client.19452 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.944+0000 7f1201f64740 10 client.19452 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:18.951 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.944+0000 7f1201f64740 20 client.19452 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.944+0000 7f1201f64740 1 --2- 172.21.15.86:0/474991858 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b31e2bb370 0x55b31e2bd760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.944+0000 7f1201f64740 4 client.19452 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.944+0000 7f1201f64740 1 -- 172.21.15.86:0/474991858 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b31e218880 con 0x55b31e2bb370 2024-04-21T08:48:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.944+0000 7f11fd02f640 1 --2- 172.21.15.86:0/474991858 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b31e2bb370 0x55b31e2bd760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.944+0000 7f11fd02f640 1 --2- 172.21.15.86:0/474991858 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b31e2bb370 0x55b31e2bd760 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e4140267ab076481 server_cookie=1283a830391fc42a in_seq=0 out_seq=0 2024-04-21T08:48:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.944+0000 7f11e0ff9640 10 client.19452 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.945+0000 7f11e0ff9640 1 -- 172.21.15.86:0/474991858 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b31e218880 con 0x55b31e2bb370 2024-04-21T08:48:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.945+0000 7f11e0ff9640 10 client.19452 handle_command_reply: tid=0 2024-04-21T08:48:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.945+0000 7f1201f64740 1 client.19452 shutdown 2024-04-21T08:48:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.945+0000 7f1201f64740 1 -- 172.21.15.86:0/474991858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b31e2a2ae0 msgr2=0x55b31e2a80d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.945+0000 7f1201f64740 1 --2- 172.21.15.86:0/474991858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b31e2a2ae0 0x55b31e2a80d0 secure :-1 s=READY pgs=1746 cs=0 l=1 rev1=1 crypto rx=0x7f11e8099570 tx=0x7f11e809c7a0 comp rx=0 tx=0).stop 2024-04-21T08:48:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.945+0000 7f1201f64740 1 -- 172.21.15.86:0/474991858 shutdown_connections 2024-04-21T08:48:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.945+0000 7f1201f64740 1 --2- 172.21.15.86:0/474991858 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b31e2bb370 0x55b31e2bd760 crc :-1 s=READY pgs=279 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:18.952 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.945+0000 7f1201f64740 1 --2- 172.21.15.86:0/474991858 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b31e2ab2f0 0x55b31e2a8750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:18.953 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.945+0000 7f1201f64740 1 --2- 172.21.15.86:0/474991858 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b31e2a2ae0 0x55b31e2a80d0 unknown :-1 s=CLOSED pgs=1746 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:18.953 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.945+0000 7f11e0ff9640 0 client.19452 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:18.953 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.945+0000 7f1201f64740 1 --2- 172.21.15.86:0/474991858 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b31e2a1d70 0x55b31e2aacd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:18.953 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.945+0000 7f1201f64740 1 -- 172.21.15.86:0/474991858 >> 172.21.15.86:0/474991858 conn(0x55b31e266c90 msgr2=0x55b31e29e7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:18.953 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.945+0000 7f1201f64740 1 -- 172.21.15.86:0/474991858 shutdown_connections 2024-04-21T08:48:18.953 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.945+0000 7f1201f64740 1 -- 172.21.15.86:0/474991858 wait complete. 2024-04-21T08:48:18.953 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.945+0000 7f1201f64740 20 client.19452 trim_cache size 0 max 16384 2024-04-21T08:48:18.961 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.960+0000 7f1201f64740 1 Processor -- start 2024-04-21T08:48:18.961 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.960+0000 7f1201f64740 1 -- start start 2024-04-21T08:48:18.961 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.960+0000 7f1201f64740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b31e2a1d70 0x55b31e2b6e60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:18.961 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.960+0000 7f1201f64740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b31e2a2ae0 0x55b31e2a79d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:18.961 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.960+0000 7f1201f64740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b31e2ab2f0 0x55b31e299750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.960+0000 7f1201f64740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f11f81ad040 con 0x55b31e2ab2f0 2024-04-21T08:48:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.960+0000 7f1201f64740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f11f81aeb80 con 0x55b31e2a2ae0 2024-04-21T08:48:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.960+0000 7f1201f64740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f11f81ad710 con 0x55b31e2a1d70 2024-04-21T08:48:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.960+0000 7f11fd830640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b31e2ab2f0 0x55b31e299750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.960+0000 7f11fd830640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b31e2ab2f0 0x55b31e299750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49710/0 (socket says 172.21.15.86:49710) 2024-04-21T08:48:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.960+0000 7f11fd830640 1 -- 172.21.15.86:0/795590172 learned_addr learned my addr 172.21.15.86:0/795590172 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.960+0000 7f11fd02f640 1 --2- 172.21.15.86:0/795590172 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b31e2a1d70 0x55b31e2b6e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.961+0000 7f11fc82e640 1 --2- 172.21.15.86:0/795590172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b31e2a2ae0 0x55b31e2a79d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.961+0000 7f11fd830640 1 -- 172.21.15.86:0/795590172 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b31e2a1d70 msgr2=0x55b31e2b6e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.961+0000 7f11fd830640 1 --2- 172.21.15.86:0/795590172 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b31e2a1d70 0x55b31e2b6e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.961+0000 7f11fd830640 1 -- 172.21.15.86:0/795590172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b31e2a2ae0 msgr2=0x55b31e2a79d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.961+0000 7f11fd830640 1 --2- 172.21.15.86:0/795590172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b31e2a2ae0 0x55b31e2a79d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:18.962 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.961+0000 7f11fd830640 1 -- 172.21.15.86:0/795590172 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b31e2a1040 con 0x55b31e2ab2f0 2024-04-21T08:48:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.961+0000 7f11fd830640 1 --2- 172.21.15.86:0/795590172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b31e2ab2f0 0x55b31e299750 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f11f0028040 tx=0x7f11f0005ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37d9c87c168c246e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.961+0000 7f11e2ffd640 1 -- 172.21.15.86:0/795590172 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f0012580 con 0x55b31e2ab2f0 2024-04-21T08:48:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.961+0000 7f11e2ffd640 1 -- 172.21.15.86:0/795590172 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11f0034070 con 0x55b31e2ab2f0 2024-04-21T08:48:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.961+0000 7f11e2ffd640 1 -- 172.21.15.86:0/795590172 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f0004c20 con 0x55b31e2ab2f0 2024-04-21T08:48:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.962+0000 7f1201f64740 1 -- 172.21.15.86:0/795590172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b31e2ab2f0 msgr2=0x55b31e299750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.962+0000 7f1201f64740 1 --2- 172.21.15.86:0/795590172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b31e2ab2f0 0x55b31e299750 secure :-1 s=READY pgs=1747 cs=0 l=1 rev1=1 crypto rx=0x7f11f0028040 tx=0x7f11f0005ea0 comp rx=0 tx=0).stop 2024-04-21T08:48:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.962+0000 7f1201f64740 1 -- 172.21.15.86:0/795590172 shutdown_connections 2024-04-21T08:48:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.962+0000 7f1201f64740 1 --2- 172.21.15.86:0/795590172 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b31e2ab2f0 0x55b31e299750 unknown :-1 s=CLOSED pgs=1747 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.962+0000 7f1201f64740 1 --2- 172.21.15.86:0/795590172 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b31e2a2ae0 0x55b31e2a79d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.962+0000 7f1201f64740 1 --2- 172.21.15.86:0/795590172 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b31e2a1d70 0x55b31e2b6e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.962+0000 7f1201f64740 1 -- 172.21.15.86:0/795590172 >> 172.21.15.86:0/795590172 conn(0x55b31e266c90 msgr2=0x55b31e299040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:18.963 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.962+0000 7f1201f64740 1 -- 172.21.15.86:0/795590172 shutdown_connections 2024-04-21T08:48:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.962+0000 7f1201f64740 1 -- 172.21.15.86:0/795590172 wait complete. 2024-04-21T08:48:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.962+0000 7f1201f64740 1 -- start start 2024-04-21T08:48:18.964 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.962+0000 7f1201f64740 1 Processor -- start 2024-04-21T08:48:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.962+0000 7f1201f64740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b31e2a1d70 0x55b31e2a5060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.962+0000 7f1201f64740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b31e2a2ae0 0x55b31e2a55a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.962+0000 7f1201f64740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b31e2ab2f0 0x55b31e2a5ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.962+0000 7f1201f64740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f11f81ae4b0 con 0x55b31e2ab2f0 2024-04-21T08:48:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.962+0000 7f1201f64740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f11f81adde0 con 0x55b31e2a1d70 2024-04-21T08:48:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.962+0000 7f1201f64740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f11f81ab040 con 0x55b31e2a2ae0 2024-04-21T08:48:18.967 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.963+0000 7f1201f64740 10 client.0 _ll_register_callbacks cb 0x7f11e800e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.963+0000 7f11fd830640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b31e2ab2f0 0x55b31e2a5ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.963+0000 7f11fd830640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b31e2ab2f0 0x55b31e2a5ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:49718/0 (socket says 172.21.15.86:49718) 2024-04-21T08:48:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.963+0000 7f11fd830640 1 -- 172.21.15.86:0/3063155035 learned_addr learned my addr 172.21.15.86:0/3063155035 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.963+0000 7f11fd02f640 1 --2- 172.21.15.86:0/3063155035 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b31e2a1d70 0x55b31e2a5060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.963+0000 7f11fc82e640 1 --2- 172.21.15.86:0/3063155035 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b31e2a2ae0 0x55b31e2a55a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.963+0000 7f11fd830640 1 -- 172.21.15.86:0/3063155035 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b31e2a2ae0 msgr2=0x55b31e2a55a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.963+0000 7f11fd830640 1 --2- 172.21.15.86:0/3063155035 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b31e2a2ae0 0x55b31e2a55a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.963+0000 7f11fd830640 1 -- 172.21.15.86:0/3063155035 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b31e2a1d70 msgr2=0x55b31e2a5060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.963+0000 7f11fd830640 1 --2- 172.21.15.86:0/3063155035 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b31e2a1d70 0x55b31e2a5060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.963+0000 7f11fd830640 1 -- 172.21.15.86:0/3063155035 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b31e320b10 con 0x55b31e2ab2f0 2024-04-21T08:48:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.963+0000 7f11fd830640 1 --2- 172.21.15.86:0/3063155035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b31e2ab2f0 0x55b31e2a5ae0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f11f0006010 tx=0x7f11f0009740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddc93b661596620 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.963+0000 7f11e0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:18.968 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.963+0000 7f11e0ff9640 1 -- 172.21.15.86:0/3063155035 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11f0012240 con 0x55b31e2ab2f0 2024-04-21T08:48:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.963+0000 7f11e0ff9640 1 -- 172.21.15.86:0/3063155035 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f11f00049c0 con 0x55b31e2ab2f0 2024-04-21T08:48:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.963+0000 7f1201f64740 1 -- 172.21.15.86:0/3063155035 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f11f0034070 con 0x55b31e2ab2f0 2024-04-21T08:48:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.963+0000 7f11e0ff9640 1 -- 172.21.15.86:0/3063155035 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f11f0004bc0 con 0x55b31e2ab2f0 2024-04-21T08:48:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.963+0000 7f1201f64740 10 client.19458 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.963+0000 7f1201f64740 1 -- 172.21.15.86:0/3063155035 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b31e321e00 con 0x55b31e2ab2f0 2024-04-21T08:48:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.964+0000 7f11e0ff9640 1 -- 172.21.15.86:0/3063155035 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f11f0034070 con 0x55b31e2ab2f0 2024-04-21T08:48:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.964+0000 7f11e0ff9640 1 -- 172.21.15.86:0/3063155035 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f11f0005770 con 0x55b31e2ab2f0 2024-04-21T08:48:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.964+0000 7f1201f64740 10 client.19458 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.964+0000 7f1201f64740 10 client.19458 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.964+0000 7f1201f64740 20 client.19458 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.964+0000 7f1201f64740 1 --2- 172.21.15.86:0/3063155035 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b31e3223a0 0x55b31e3247c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.964+0000 7f1201f64740 4 client.19458 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:48:18.969 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.964+0000 7f1201f64740 1 -- 172.21.15.86:0/3063155035 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55b31e1a43d0 con 0x55b31e3223a0 2024-04-21T08:48:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.964+0000 7f11fd02f640 1 --2- 172.21.15.86:0/3063155035 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b31e3223a0 0x55b31e3247c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.964+0000 7f11e0ff9640 10 client.19458 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:18.970 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.964+0000 7f11fd02f640 1 --2- 172.21.15.86:0/3063155035 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b31e3223a0 0x55b31e3247c0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e43c672ee2465b30 server_cookie=cf9ecce1c6c244c9 in_seq=0 out_seq=0 2024-04-21T08:48:18.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.979+0000 7f11e0ff9640 1 -- 172.21.15.86:0/3063155035 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55b31e1a43d0 con 0x55b31e3223a0 2024-04-21T08:48:18.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.979+0000 7f11e0ff9640 10 client.19458 handle_command_reply: tid=0 2024-04-21T08:48:18.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.979+0000 7f1201f64740 1 client.19458 shutdown 2024-04-21T08:48:18.979 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.979+0000 7f1201f64740 1 -- 172.21.15.86:0/3063155035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b31e2ab2f0 msgr2=0x55b31e2a5ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.979+0000 7f1201f64740 1 --2- 172.21.15.86:0/3063155035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b31e2ab2f0 0x55b31e2a5ae0 secure :-1 s=READY pgs=1748 cs=0 l=1 rev1=1 crypto rx=0x7f11f0006010 tx=0x7f11f0009740 comp rx=0 tx=0).stop 2024-04-21T08:48:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.979+0000 7f1201f64740 1 -- 172.21.15.86:0/3063155035 shutdown_connections 2024-04-21T08:48:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.979+0000 7f1201f64740 1 --2- 172.21.15.86:0/3063155035 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b31e3223a0 0x55b31e3247c0 crc :-1 s=READY pgs=280 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.979+0000 7f1201f64740 1 --2- 172.21.15.86:0/3063155035 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b31e2ab2f0 0x55b31e2a5ae0 unknown :-1 s=CLOSED pgs=1748 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.979+0000 7f1201f64740 1 --2- 172.21.15.86:0/3063155035 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b31e2a2ae0 0x55b31e2a55a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.979+0000 7f11e0ff9640 0 client.19458 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.979+0000 7f1201f64740 1 --2- 172.21.15.86:0/3063155035 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b31e2a1d70 0x55b31e2a5060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.979+0000 7f1201f64740 1 -- 172.21.15.86:0/3063155035 >> 172.21.15.86:0/3063155035 conn(0x55b31e266c90 msgr2=0x55b31e2998d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.980+0000 7f1201f64740 1 -- 172.21.15.86:0/3063155035 shutdown_connections 2024-04-21T08:48:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.980+0000 7f1201f64740 1 -- 172.21.15.86:0/3063155035 wait complete. 2024-04-21T08:48:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.980+0000 7f1201f64740 20 client.19458 trim_cache size 0 max 16384 2024-04-21T08:48:18.980 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:48:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:48:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:48:18.981 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:48:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.980+0000 7f11ff293640 1 -- 172.21.15.86:0/908582713 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f11e40a3dc0 msgr2=0x7f11e40a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.980+0000 7f11ff293640 1 --2- 172.21.15.86:0/908582713 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f11e40a3dc0 0x7f11e40a6280 secure :-1 s=READY pgs=286 cs=0 l=1 rev1=1 crypto rx=0x7f11f0010ca0 tx=0x7f11f001d000 comp rx=0 tx=0).stop 2024-04-21T08:48:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.980+0000 7f11ff293640 1 -- 172.21.15.86:0/908582713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11f8106170 msgr2=0x7f11f8197b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.980+0000 7f11ff293640 1 --2- 172.21.15.86:0/908582713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11f8106170 0x7f11f8197b90 secure :-1 s=READY pgs=1744 cs=0 l=1 rev1=1 crypto rx=0x7f11e800e9f0 tx=0x7f11e800eeb0 comp rx=0 tx=0).stop 2024-04-21T08:48:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.980+0000 7f11ff293640 1 -- 172.21.15.86:0/908582713 shutdown_connections 2024-04-21T08:48:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.980+0000 7f11ff293640 1 --2- 172.21.15.86:0/908582713 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f11e40a3dc0 0x7f11e40a6280 unknown :-1 s=CLOSED pgs=286 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.980+0000 7f11ff293640 1 --2- 172.21.15.86:0/908582713 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f11f819bf10 0x7f11f819c3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.980+0000 7f11ff293640 1 --2- 172.21.15.86:0/908582713 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f11f8106170 0x7f11f8197b90 unknown :-1 s=CLOSED pgs=1744 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.980+0000 7f11ff293640 1 --2- 172.21.15.86:0/908582713 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f11f81010f0 0x7f11f8197650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.980+0000 7f11ff293640 1 -- 172.21.15.86:0/908582713 >> 172.21.15.86:0/908582713 conn(0x7f11f8077fd0 msgr2=0x7f11f80ff0e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.980+0000 7f11ff293640 1 -- 172.21.15.86:0/908582713 shutdown_connections 2024-04-21T08:48:18.982 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:18.980+0000 7f11ff293640 1 -- 172.21.15.86:0/908582713 wait complete. 2024-04-21T08:48:20.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:19 smithi088 ceph-mon[31259]: pgmap v297: 129 pgs: 129 active+clean; 1.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:48:20.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:19 smithi179 ceph-mon[31829]: pgmap v297: 129 pgs: 129 active+clean; 1.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:48:20.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:19 smithi086 ceph-mon[27939]: pgmap v297: 129 pgs: 129 active+clean; 1.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:48:21.642 INFO:tasks.quiescer.fs.[cephfs]:Going to quiesce for duration: 33.1 2024-04-21T08:48:21.643 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:48:21.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.753+0000 7ffa4d30e640 1 -- 172.21.15.86:0/2828265099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa48100fc0 msgr2=0x7ffa481013a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:21.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.753+0000 7ffa4d30e640 1 --2- 172.21.15.86:0/2828265099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa48100fc0 0x7ffa481013a0 secure :-1 s=READY pgs=1749 cs=0 l=1 rev1=1 crypto rx=0x7ffa38009930 tx=0x7ffa3802bcd0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.753+0000 7ffa4d30e640 1 -- 172.21.15.86:0/2828265099 shutdown_connections 2024-04-21T08:48:21.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.753+0000 7ffa4d30e640 1 --2- 172.21.15.86:0/2828265099 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa481063c0 0x7ffa48106870 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.753+0000 7ffa4d30e640 1 --2- 172.21.15.86:0/2828265099 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa481018e0 0x7ffa48105da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.753+0000 7ffa4d30e640 1 --2- 172.21.15.86:0/2828265099 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa48100fc0 0x7ffa481013a0 unknown :-1 s=CLOSED pgs=1749 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.753+0000 7ffa4d30e640 1 -- 172.21.15.86:0/2828265099 >> 172.21.15.86:0/2828265099 conn(0x7ffa480771d0 msgr2=0x7ffa480775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:21.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.754+0000 7ffa4d30e640 1 -- 172.21.15.86:0/2828265099 shutdown_connections 2024-04-21T08:48:21.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.754+0000 7ffa4d30e640 1 -- 172.21.15.86:0/2828265099 wait complete. 2024-04-21T08:48:21.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.754+0000 7ffa4d30e640 1 Processor -- start 2024-04-21T08:48:21.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.754+0000 7ffa4d30e640 1 -- start start 2024-04-21T08:48:21.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.754+0000 7ffa4d30e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa48100fc0 0x7ffa481a4920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:21.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.754+0000 7ffa4d30e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa481018e0 0x7ffa481a4e60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:21.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.754+0000 7ffa4d30e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa481063c0 0x7ffa4819eb20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:21.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.754+0000 7ffa4d30e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ffa481187a0 con 0x7ffa481063c0 2024-04-21T08:48:21.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.754+0000 7ffa4d30e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ffa48118620 con 0x7ffa48100fc0 2024-04-21T08:48:21.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.754+0000 7ffa4d30e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ffa48118920 con 0x7ffa481018e0 2024-04-21T08:48:21.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.754+0000 7ffa477fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa481063c0 0x7ffa4819eb20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:21.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.754+0000 7ffa477fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa481063c0 0x7ffa4819eb20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60878/0 (socket says 172.21.15.86:60878) 2024-04-21T08:48:21.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.754+0000 7ffa477fe640 1 -- 172.21.15.86:0/2348542442 learned_addr learned my addr 172.21.15.86:0/2348542442 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:21.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.755+0000 7ffa3ffff640 1 --2- 172.21.15.86:0/2348542442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa481018e0 0x7ffa481a4e60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:21.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.755+0000 7ffa46ffd640 1 --2- 172.21.15.86:0/2348542442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa48100fc0 0x7ffa481a4920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:21.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.755+0000 7ffa477fe640 1 -- 172.21.15.86:0/2348542442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa481018e0 msgr2=0x7ffa481a4e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:21.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.755+0000 7ffa477fe640 1 --2- 172.21.15.86:0/2348542442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa481018e0 0x7ffa481a4e60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.755+0000 7ffa477fe640 1 -- 172.21.15.86:0/2348542442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa48100fc0 msgr2=0x7ffa481a4920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:21.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.755+0000 7ffa477fe640 1 --2- 172.21.15.86:0/2348542442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa48100fc0 0x7ffa481a4920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.755+0000 7ffa477fe640 1 -- 172.21.15.86:0/2348542442 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ffa4819f2a0 con 0x7ffa481063c0 2024-04-21T08:48:21.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.755+0000 7ffa46ffd640 1 --2- 172.21.15.86:0/2348542442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa48100fc0 0x7ffa481a4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:21.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.755+0000 7ffa3ffff640 1 --2- 172.21.15.86:0/2348542442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa481018e0 0x7ffa481a4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:21.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.755+0000 7ffa477fe640 1 --2- 172.21.15.86:0/2348542442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa481063c0 0x7ffa4819eb20 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7ffa4000cce0 tx=0x7ffa4000a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:21.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.755+0000 7ffa44ff9640 1 -- 172.21.15.86:0/2348542442 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa4000f450 con 0x7ffa481063c0 2024-04-21T08:48:21.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.755+0000 7ffa44ff9640 1 -- 172.21.15.86:0/2348542442 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ffa4000faa0 con 0x7ffa481063c0 2024-04-21T08:48:21.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.755+0000 7ffa4d30e640 1 -- 172.21.15.86:0/2348542442 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ffa4819f590 con 0x7ffa481063c0 2024-04-21T08:48:21.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.755+0000 7ffa44ff9640 1 -- 172.21.15.86:0/2348542442 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ffa40018b70 con 0x7ffa481063c0 2024-04-21T08:48:21.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.755+0000 7ffa4d30e640 1 -- 172.21.15.86:0/2348542442 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ffa481ab7e0 con 0x7ffa481063c0 2024-04-21T08:48:21.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.756+0000 7ffa4d30e640 1 -- 172.21.15.86:0/2348542442 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7ffa48069230 con 0x7ffa481063c0 2024-04-21T08:48:21.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.757+0000 7ffa44ff9640 1 -- 172.21.15.86:0/2348542442 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ffa40018d10 con 0x7ffa481063c0 2024-04-21T08:48:21.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.758+0000 7ffa44ff9640 1 --2- 172.21.15.86:0/2348542442 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ffa340a3d20 0x7ffa340a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:21.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.758+0000 7ffa46ffd640 1 --2- 172.21.15.86:0/2348542442 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ffa340a3d20 0x7ffa340a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:21.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.758+0000 7ffa44ff9640 1 -- 172.21.15.86:0/2348542442 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7ffa400d10c0 con 0x7ffa481063c0 2024-04-21T08:48:21.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.758+0000 7ffa46ffd640 1 --2- 172.21.15.86:0/2348542442 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ffa340a3d20 0x7ffa340a61e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ffa38006010 tx=0x7ffa38009610 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:21.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.762+0000 7ffa44ff9640 1 -- 172.21.15.86:0/2348542442 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7ffa40014030 con 0x7ffa481063c0 2024-04-21T08:48:21.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.982+0000 7ffa4d30e640 1 -- 172.21.15.86:0/2348542442 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7ffa4819ffc0 con 0x7ffa481063c0 2024-04-21T08:48:21.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.982+0000 7ffa44ff9640 1 -- 172.21.15.86:0/2348542442 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7ffa4819ffc0 con 0x7ffa481063c0 2024-04-21T08:48:21.983 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:48:21.983 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:38:45.202126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24398,24394,24455,24472]},"id":1}]} 2024-04-21T08:48:21.983 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-21T08:48:21.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.984+0000 7ffa4d30e640 1 -- 172.21.15.86:0/2348542442 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ffa340a3d20 msgr2=0x7ffa340a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:21.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.984+0000 7ffa4d30e640 1 --2- 172.21.15.86:0/2348542442 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ffa340a3d20 0x7ffa340a61e0 secure :-1 s=READY pgs=287 cs=0 l=1 rev1=1 crypto rx=0x7ffa38006010 tx=0x7ffa38009610 comp rx=0 tx=0).stop 2024-04-21T08:48:21.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.984+0000 7ffa4d30e640 1 -- 172.21.15.86:0/2348542442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa481063c0 msgr2=0x7ffa4819eb20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:21.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.984+0000 7ffa4d30e640 1 --2- 172.21.15.86:0/2348542442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa481063c0 0x7ffa4819eb20 secure :-1 s=READY pgs=1750 cs=0 l=1 rev1=1 crypto rx=0x7ffa4000cce0 tx=0x7ffa4000a620 comp rx=0 tx=0).stop 2024-04-21T08:48:21.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.984+0000 7ffa4d30e640 1 -- 172.21.15.86:0/2348542442 shutdown_connections 2024-04-21T08:48:21.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.984+0000 7ffa4d30e640 1 --2- 172.21.15.86:0/2348542442 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ffa340a3d20 0x7ffa340a61e0 unknown :-1 s=CLOSED pgs=287 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.984+0000 7ffa4d30e640 1 --2- 172.21.15.86:0/2348542442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ffa481063c0 0x7ffa4819eb20 unknown :-1 s=CLOSED pgs=1750 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.984+0000 7ffa4d30e640 1 --2- 172.21.15.86:0/2348542442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ffa481018e0 0x7ffa481a4e60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.984+0000 7ffa4d30e640 1 --2- 172.21.15.86:0/2348542442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ffa48100fc0 0x7ffa481a4920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:21.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.984+0000 7ffa4d30e640 1 -- 172.21.15.86:0/2348542442 >> 172.21.15.86:0/2348542442 conn(0x7ffa480771d0 msgr2=0x7ffa48103d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:21.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.984+0000 7ffa4d30e640 1 -- 172.21.15.86:0/2348542442 shutdown_connections 2024-04-21T08:48:21.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:21.984+0000 7ffa4d30e640 1 -- 172.21.15.86:0/2348542442 wait complete. 2024-04-21T08:48:21.994 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:21 smithi086 ceph-mon[27939]: pgmap v298: 129 pgs: 129 active+clean; 1.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:48:21.996 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db / --timeout 30 --expiration 93.1 --await' 2024-04-21T08:48:21.996 DEBUG:teuthology.orchestra.run.smithi086:> timeout 45 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24482 quiesce db / --timeout 30 --expiration 93.1 --await 2024-04-21T08:48:22.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.102+0000 7f40b69b2640 1 -- 172.21.15.86:0/447327720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40b01031e0 msgr2=0x7f40b01035c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.102+0000 7f40b69b2640 1 --2- 172.21.15.86:0/447327720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40b01031e0 0x7f40b01035c0 secure :-1 s=READY pgs=1751 cs=0 l=1 rev1=1 crypto rx=0x7f40a4009900 tx=0x7f40a402bd60 comp rx=0 tx=0).stop 2024-04-21T08:48:22.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.102+0000 7f40b69b2640 1 -- 172.21.15.86:0/447327720 shutdown_connections 2024-04-21T08:48:22.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.102+0000 7f40b69b2640 1 --2- 172.21.15.86:0/447327720 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40b010c520 0x7f40b010e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.102+0000 7f40b69b2640 1 --2- 172.21.15.86:0/447327720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40b0103b00 0x7f40b010bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.102+0000 7f40b69b2640 1 --2- 172.21.15.86:0/447327720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40b01031e0 0x7f40b01035c0 unknown :-1 s=CLOSED pgs=1751 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.102+0000 7f40b69b2640 1 -- 172.21.15.86:0/447327720 >> 172.21.15.86:0/447327720 conn(0x7f40b0077f50 msgr2=0x7f40b0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:22.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.102+0000 7f40b69b2640 1 -- 172.21.15.86:0/447327720 shutdown_connections 2024-04-21T08:48:22.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.102+0000 7f40b69b2640 1 -- 172.21.15.86:0/447327720 wait complete. 2024-04-21T08:48:22.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.103+0000 7f40b69b2640 1 Processor -- start 2024-04-21T08:48:22.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.103+0000 7f40b69b2640 1 -- start start 2024-04-21T08:48:22.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.103+0000 7f40b69b2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40b01031e0 0x7f40b0198750 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.103+0000 7f40b69b2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40b0103b00 0x7f40b0198c90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.103+0000 7f40b69b2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40b010c520 0x7f40b019d010 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.103+0000 7f40b69b2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f40b0112190 con 0x7f40b0103b00 2024-04-21T08:48:22.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.103+0000 7f40b69b2640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f40b0112010 con 0x7f40b010c520 2024-04-21T08:48:22.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.103+0000 7f40b69b2640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f40b0112310 con 0x7f40b01031e0 2024-04-21T08:48:22.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.103+0000 7f40af7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40b0103b00 0x7f40b0198c90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.103+0000 7f40af7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40b0103b00 0x7f40b0198c90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60904/0 (socket says 172.21.15.86:60904) 2024-04-21T08:48:22.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.103+0000 7f40af7fe640 1 -- 172.21.15.86:0/3047054027 learned_addr learned my addr 172.21.15.86:0/3047054027 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:22.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.103+0000 7f40b4f4f640 1 --2- 172.21.15.86:0/3047054027 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40b010c520 0x7f40b019d010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.104+0000 7f40af7fe640 1 -- 172.21.15.86:0/3047054027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40b01031e0 msgr2=0x7f40b0198750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.104+0000 7f40af7fe640 1 --2- 172.21.15.86:0/3047054027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40b01031e0 0x7f40b0198750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.104+0000 7f40af7fe640 1 -- 172.21.15.86:0/3047054027 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40b010c520 msgr2=0x7f40b019d010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.104+0000 7f40af7fe640 1 --2- 172.21.15.86:0/3047054027 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40b010c520 0x7f40b019d010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.104+0000 7f40af7fe640 1 -- 172.21.15.86:0/3047054027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b019d740 con 0x7f40b0103b00 2024-04-21T08:48:22.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.104+0000 7f40af7fe640 1 --2- 172.21.15.86:0/3047054027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40b0103b00 0x7f40b0198c90 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f40a000c920 tx=0x7f40a000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:22.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.104+0000 7f40ad7fa640 1 -- 172.21.15.86:0/3047054027 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a0007c70 con 0x7f40b0103b00 2024-04-21T08:48:22.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.104+0000 7f40b69b2640 1 -- 172.21.15.86:0/3047054027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40b019da30 con 0x7f40b0103b00 2024-04-21T08:48:22.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.104+0000 7f40ad7fa640 1 -- 172.21.15.86:0/3047054027 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40a0007e10 con 0x7f40b0103b00 2024-04-21T08:48:22.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.104+0000 7f40ad7fa640 1 -- 172.21.15.86:0/3047054027 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a0018820 con 0x7f40b0103b00 2024-04-21T08:48:22.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.104+0000 7f40b69b2640 1 -- 172.21.15.86:0/3047054027 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40b01a52c0 con 0x7f40b0103b00 2024-04-21T08:48:22.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.106+0000 7f40ad7fa640 1 -- 172.21.15.86:0/3047054027 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f40a000f450 con 0x7f40b0103b00 2024-04-21T08:48:22.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.106+0000 7f40b95a3740 1 Processor -- start 2024-04-21T08:48:22.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.106+0000 7f40b95a3740 1 -- start start 2024-04-21T08:48:22.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.106+0000 7f40ad7fa640 1 --2- 172.21.15.86:0/3047054027 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f40980a3d20 0x7f40980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.107+0000 7f40b95a3740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2da5c79c0 0x55c2da5c7da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.107+0000 7f40b95a3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2da5c82e0 0x55c2da5d0980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.107+0000 7f40b95a3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2da5d0ec0 0x55c2da5d32b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.107+0000 7f40b95a3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c2da5280b0 con 0x55c2da5d0ec0 2024-04-21T08:48:22.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.107+0000 7f40b95a3740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c2da4d9680 con 0x55c2da5c79c0 2024-04-21T08:48:22.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.107+0000 7f40b95a3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c2da4e5c60 con 0x55c2da5c82e0 2024-04-21T08:48:22.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.107+0000 7f40b4f4f640 1 --2- 172.21.15.86:0/3047054027 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f40980a3d20 0x7f40980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.107+0000 7f40ad7fa640 1 -- 172.21.15.86:0/3047054027 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f40a00d0da0 con 0x7f40b0103b00 2024-04-21T08:48:22.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.107+0000 7f40b4f4f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2da5d0ec0 0x55c2da5d32b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.107+0000 7f40b4f4f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2da5d0ec0 0x55c2da5d32b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60920/0 (socket says 172.21.15.86:60920) 2024-04-21T08:48:22.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.107+0000 7f40b4f4f640 1 -- 172.21.15.86:0/2490155658 learned_addr learned my addr 172.21.15.86:0/2490155658 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:22.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.107+0000 7f40af7fe640 1 --2- 172.21.15.86:0/2490155658 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2da5c82e0 0x55c2da5d0980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.107+0000 7f40affff640 1 --2- 172.21.15.86:0/2490155658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2da5c79c0 0x55c2da5c7da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.107+0000 7f40b4f4f640 1 -- 172.21.15.86:0/2490155658 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2da5c82e0 msgr2=0x55c2da5d0980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.107+0000 7f40b4f4f640 1 --2- 172.21.15.86:0/2490155658 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2da5c82e0 0x55c2da5d0980 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.107+0000 7f40b4f4f640 1 -- 172.21.15.86:0/2490155658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2da5c79c0 msgr2=0x55c2da5c7da0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.107+0000 7f40b4f4f640 1 --2- 172.21.15.86:0/2490155658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2da5c79c0 0x55c2da5c7da0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.107+0000 7f40b4f4f640 1 -- 172.21.15.86:0/2490155658 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2da30ac90 con 0x55c2da5d0ec0 2024-04-21T08:48:22.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.107+0000 7f40affff640 1 --2- 172.21.15.86:0/2490155658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2da5c79c0 0x55c2da5c7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:22.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.107+0000 7f40b4f4f640 1 --2- 172.21.15.86:0/3047054027 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f40980a3d20 0x7f40980a61e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f40a8014a40 tx=0x7f40a8009c70 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:22.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.107+0000 7f40b4f4f640 1 --2- 172.21.15.86:0/2490155658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2da5d0ec0 0x55c2da5d32b0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f40a8011c80 tx=0x7f40a801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12182f641b3e331e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:22.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.107+0000 7f40967fc640 1 -- 172.21.15.86:0/2490155658 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8011d70 con 0x55c2da5d0ec0 2024-04-21T08:48:22.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.107+0000 7f40967fc640 1 -- 172.21.15.86:0/2490155658 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40a8014110 con 0x55c2da5d0ec0 2024-04-21T08:48:22.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.107+0000 7f40967fc640 1 -- 172.21.15.86:0/2490155658 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a8003bd0 con 0x55c2da5d0ec0 2024-04-21T08:48:22.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.108+0000 7f40b95a3740 1 -- 172.21.15.86:0/2490155658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2da5d0ec0 msgr2=0x55c2da5d32b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.108+0000 7f40b95a3740 1 --2- 172.21.15.86:0/2490155658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2da5d0ec0 0x55c2da5d32b0 secure :-1 s=READY pgs=1753 cs=0 l=1 rev1=1 crypto rx=0x7f40a8011c80 tx=0x7f40a801b000 comp rx=0 tx=0).stop 2024-04-21T08:48:22.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.108+0000 7f40b95a3740 1 -- 172.21.15.86:0/2490155658 shutdown_connections 2024-04-21T08:48:22.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.108+0000 7f40b95a3740 1 --2- 172.21.15.86:0/2490155658 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2da5d0ec0 0x55c2da5d32b0 unknown :-1 s=CLOSED pgs=1753 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.108+0000 7f40b95a3740 1 --2- 172.21.15.86:0/2490155658 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2da5c82e0 0x55c2da5d0980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.108+0000 7f40b95a3740 1 --2- 172.21.15.86:0/2490155658 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2da5c79c0 0x55c2da5c7da0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.108+0000 7f40b95a3740 1 -- 172.21.15.86:0/2490155658 >> 172.21.15.86:0/2490155658 conn(0x55c2da5c20f0 msgr2=0x55c2da5c24d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:22.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.108+0000 7f40b95a3740 1 -- 172.21.15.86:0/2490155658 shutdown_connections 2024-04-21T08:48:22.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.108+0000 7f40b95a3740 1 -- 172.21.15.86:0/2490155658 wait complete. 2024-04-21T08:48:22.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.108+0000 7f40b95a3740 1 -- start start 2024-04-21T08:48:22.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.109+0000 7f40b95a3740 1 Processor -- start 2024-04-21T08:48:22.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.109+0000 7f40b95a3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2da5c79c0 0x55c2da5cdf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.109+0000 7f40b95a3740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2da5c82e0 0x55c2da5ce4c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.109+0000 7f40b95a3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2da5d0ec0 0x55c2da5cea00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.109+0000 7f40b95a3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c2da52b400 con 0x55c2da5d0ec0 2024-04-21T08:48:22.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.109+0000 7f40b95a3740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c2da4e7840 con 0x55c2da5c82e0 2024-04-21T08:48:22.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.109+0000 7f40b95a3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c2da530720 con 0x55c2da5c79c0 2024-04-21T08:48:22.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.109+0000 7f40b95a3740 10 client.0 _ll_register_callbacks cb 0x55c2da538ea0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:22.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.109+0000 7f40b4f4f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2da5d0ec0 0x55c2da5cea00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.109+0000 7f40b4f4f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2da5d0ec0 0x55c2da5cea00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60924/0 (socket says 172.21.15.86:60924) 2024-04-21T08:48:22.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.109+0000 7f40b4f4f640 1 -- 172.21.15.86:0/4272714471 learned_addr learned my addr 172.21.15.86:0/4272714471 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:22.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.109+0000 7f40affff640 1 --2- 172.21.15.86:0/4272714471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2da5c79c0 0x55c2da5cdf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.109+0000 7f40af7fe640 1 --2- 172.21.15.86:0/4272714471 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2da5c82e0 0x55c2da5ce4c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.109+0000 7f40b4f4f640 1 -- 172.21.15.86:0/4272714471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2da5c79c0 msgr2=0x55c2da5cdf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.109+0000 7f40b4f4f640 1 --2- 172.21.15.86:0/4272714471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2da5c79c0 0x55c2da5cdf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.109+0000 7f40b4f4f640 1 -- 172.21.15.86:0/4272714471 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2da5c82e0 msgr2=0x55c2da5ce4c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.109+0000 7f40b4f4f640 1 --2- 172.21.15.86:0/4272714471 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2da5c82e0 0x55c2da5ce4c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.109+0000 7f40b4f4f640 1 -- 172.21.15.86:0/4272714471 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2da36b420 con 0x55c2da5d0ec0 2024-04-21T08:48:22.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.109+0000 7f40b4f4f640 1 --2- 172.21.15.86:0/4272714471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2da5d0ec0 0x55c2da5cea00 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f40a8006010 tx=0x7f40a8009ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1991ddf108bcd68b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:22.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.109+0000 7f408f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:22.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.110+0000 7f408f7fe640 1 -- 172.21.15.86:0/4272714471 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a802c070 con 0x55c2da5d0ec0 2024-04-21T08:48:22.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.110+0000 7f408f7fe640 1 -- 172.21.15.86:0/4272714471 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f40a800fe40 con 0x55c2da5d0ec0 2024-04-21T08:48:22.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.110+0000 7f40b95a3740 1 -- 172.21.15.86:0/4272714471 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40a8014110 con 0x55c2da5d0ec0 2024-04-21T08:48:22.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.110+0000 7f408f7fe640 1 -- 172.21.15.86:0/4272714471 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f40a8038050 con 0x55c2da5d0ec0 2024-04-21T08:48:22.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.110+0000 7f40b95a3740 10 client.19476 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:22.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.110+0000 7f40b95a3740 1 -- 172.21.15.86:0/4272714471 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c2da1fe580 con 0x55c2da5d0ec0 2024-04-21T08:48:22.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.110+0000 7f408f7fe640 1 -- 172.21.15.86:0/4272714471 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40a8014110 con 0x55c2da5d0ec0 2024-04-21T08:48:22.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.110+0000 7f408f7fe640 1 -- 172.21.15.86:0/4272714471 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f40a800f050 con 0x55c2da5d0ec0 2024-04-21T08:48:22.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.110+0000 7f40b95a3740 10 client.19476 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:22.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.110+0000 7f40b95a3740 10 client.19476 resolve_mds: validated gid 24482 aka daemon mds.b 2024-04-21T08:48:22.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.110+0000 7f40b95a3740 20 client.19476 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:22.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.110+0000 7f40b95a3740 1 --2- 172.21.15.86:0/4272714471 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c2da5e0b10 0x55c2da5e2f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.110+0000 7f40b95a3740 4 client.19476 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:22.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.110+0000 7f40b95a3740 1 -- 172.21.15.86:0/4272714471 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c2da539b10 con 0x55c2da5e0b10 2024-04-21T08:48:22.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.111+0000 7f40affff640 1 --2- 172.21.15.86:0/4272714471 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c2da5e0b10 0x55c2da5e2f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.111+0000 7f408f7fe640 10 client.19476 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:22.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.111+0000 7f40affff640 1 --2- 172.21.15.86:0/4272714471 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c2da5e0b10 0x55c2da5e2f00 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3da584d604d2aa64 server_cookie=2548eae297467684 in_seq=0 out_seq=0 2024-04-21T08:48:22.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.112+0000 7f408f7fe640 1 -- 172.21.15.86:0/4272714471 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c2da539b10 con 0x55c2da5e0b10 2024-04-21T08:48:22.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.112+0000 7f408f7fe640 10 client.19476 handle_command_reply: tid=0 2024-04-21T08:48:22.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.112+0000 7f40b95a3740 1 client.19476 shutdown 2024-04-21T08:48:22.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.112+0000 7f40b95a3740 1 -- 172.21.15.86:0/4272714471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2da5d0ec0 msgr2=0x55c2da5cea00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.112+0000 7f40b95a3740 1 --2- 172.21.15.86:0/4272714471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2da5d0ec0 0x55c2da5cea00 secure :-1 s=READY pgs=1754 cs=0 l=1 rev1=1 crypto rx=0x7f40a8006010 tx=0x7f40a8009ea0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.112+0000 7f40b95a3740 1 -- 172.21.15.86:0/4272714471 shutdown_connections 2024-04-21T08:48:22.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.112+0000 7f40b95a3740 1 --2- 172.21.15.86:0/4272714471 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c2da5e0b10 0x55c2da5e2f00 crc :-1 s=READY pgs=281 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.112+0000 7f40b95a3740 1 --2- 172.21.15.86:0/4272714471 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2da5d0ec0 0x55c2da5cea00 unknown :-1 s=CLOSED pgs=1754 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.112+0000 7f40b95a3740 1 --2- 172.21.15.86:0/4272714471 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2da5c82e0 0x55c2da5ce4c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.112+0000 7f408f7fe640 0 client.19476 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:22.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.112+0000 7f40b95a3740 1 --2- 172.21.15.86:0/4272714471 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2da5c79c0 0x55c2da5cdf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.112+0000 7f40b95a3740 1 -- 172.21.15.86:0/4272714471 >> 172.21.15.86:0/4272714471 conn(0x55c2da5c20f0 msgr2=0x55c2da5c3e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:22.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.112+0000 7f40b95a3740 1 -- 172.21.15.86:0/4272714471 shutdown_connections 2024-04-21T08:48:22.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.112+0000 7f40b95a3740 1 -- 172.21.15.86:0/4272714471 wait complete. 2024-04-21T08:48:22.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.112+0000 7f40b95a3740 20 client.19476 trim_cache size 0 max 16384 2024-04-21T08:48:22.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.129+0000 7f40b95a3740 1 Processor -- start 2024-04-21T08:48:22.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.129+0000 7f40b95a3740 1 -- start start 2024-04-21T08:48:22.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.129+0000 7f40b95a3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2da5c79c0 0x55c2da5dc860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.129+0000 7f40b95a3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2da5c82e0 0x55c2da5ce650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.129+0000 7f40b95a3740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2da5d0ec0 0x55c2da5e84d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.129+0000 7f40b95a3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01b0ed0 con 0x55c2da5c79c0 2024-04-21T08:48:22.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.129+0000 7f40b95a3740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01afa60 con 0x55c2da5d0ec0 2024-04-21T08:48:22.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.129+0000 7f40b95a3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01b0130 con 0x55c2da5c82e0 2024-04-21T08:48:22.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.129+0000 7f40affff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2da5c79c0 0x55c2da5dc860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.129+0000 7f40affff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2da5c79c0 0x55c2da5dc860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60940/0 (socket says 172.21.15.86:60940) 2024-04-21T08:48:22.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.129+0000 7f40affff640 1 -- 172.21.15.86:0/2091991519 learned_addr learned my addr 172.21.15.86:0/2091991519 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:22.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.129+0000 7f40af7fe640 1 --2- 172.21.15.86:0/2091991519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2da5c82e0 0x55c2da5ce650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.129+0000 7f40affff640 1 -- 172.21.15.86:0/2091991519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2da5c82e0 msgr2=0x55c2da5ce650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.129+0000 7f40affff640 1 --2- 172.21.15.86:0/2091991519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2da5c82e0 0x55c2da5ce650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.129+0000 7f40affff640 1 -- 172.21.15.86:0/2091991519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2da5d0ec0 msgr2=0x55c2da5e84d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.129+0000 7f40affff640 1 --2- 172.21.15.86:0/2091991519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2da5d0ec0 0x55c2da5e84d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.129+0000 7f40affff640 1 -- 172.21.15.86:0/2091991519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c2da5e3270 con 0x55c2da5c79c0 2024-04-21T08:48:22.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.129+0000 7f40af7fe640 1 --2- 172.21.15.86:0/2091991519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2da5c82e0 0x55c2da5ce650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:22.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.129+0000 7f40affff640 1 --2- 172.21.15.86:0/2091991519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2da5c79c0 0x55c2da5dc860 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f40a4036040 tx=0x7f40a4000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=af8602745000d2d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:22.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.129+0000 7f4095ffb640 1 -- 172.21.15.86:0/2091991519 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a40043b0 con 0x55c2da5c79c0 2024-04-21T08:48:22.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.129+0000 7f4095ffb640 1 -- 172.21.15.86:0/2091991519 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40a4048070 con 0x55c2da5c79c0 2024-04-21T08:48:22.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.129+0000 7f4095ffb640 1 -- 172.21.15.86:0/2091991519 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a40434c0 con 0x55c2da5c79c0 2024-04-21T08:48:22.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.130+0000 7f40b95a3740 1 -- 172.21.15.86:0/2091991519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2da5c79c0 msgr2=0x55c2da5dc860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.130+0000 7f40b95a3740 1 --2- 172.21.15.86:0/2091991519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2da5c79c0 0x55c2da5dc860 secure :-1 s=READY pgs=1755 cs=0 l=1 rev1=1 crypto rx=0x7f40a4036040 tx=0x7f40a4000bc0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.130+0000 7f40b95a3740 1 -- 172.21.15.86:0/2091991519 shutdown_connections 2024-04-21T08:48:22.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.130+0000 7f40b95a3740 1 --2- 172.21.15.86:0/2091991519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2da5d0ec0 0x55c2da5e84d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.130+0000 7f40b95a3740 1 --2- 172.21.15.86:0/2091991519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2da5c82e0 0x55c2da5ce650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.130+0000 7f40b95a3740 1 --2- 172.21.15.86:0/2091991519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2da5c79c0 0x55c2da5dc860 unknown :-1 s=CLOSED pgs=1755 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.130+0000 7f40b95a3740 1 -- 172.21.15.86:0/2091991519 >> 172.21.15.86:0/2091991519 conn(0x55c2da5c20f0 msgr2=0x55c2da5cdf40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:22.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.130+0000 7f40b95a3740 1 -- 172.21.15.86:0/2091991519 shutdown_connections 2024-04-21T08:48:22.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.130+0000 7f40b95a3740 1 -- 172.21.15.86:0/2091991519 wait complete. 2024-04-21T08:48:22.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.130+0000 7f40b95a3740 1 -- start start 2024-04-21T08:48:22.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.131+0000 7f40b95a3740 1 Processor -- start 2024-04-21T08:48:22.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.131+0000 7f40b95a3740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2da5c79c0 0x55c2da5d4f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.131+0000 7f40b95a3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2da5c82e0 0x55c2da5d5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.131+0000 7f40b95a3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2da5d0ec0 0x55c2da645d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.131+0000 7f40b95a3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01ad390 con 0x55c2da5c82e0 2024-04-21T08:48:22.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.131+0000 7f40b95a3740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01b0800 con 0x55c2da5c79c0 2024-04-21T08:48:22.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.131+0000 7f40b95a3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01ad9e0 con 0x55c2da5d0ec0 2024-04-21T08:48:22.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.131+0000 7f40b95a3740 10 client.0 _ll_register_callbacks cb 0x7f40a0098c10 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:22.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.131+0000 7f40af7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2da5c82e0 0x55c2da5d5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.131+0000 7f40af7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2da5c82e0 0x55c2da5d5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60956/0 (socket says 172.21.15.86:60956) 2024-04-21T08:48:22.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.131+0000 7f40af7fe640 1 -- 172.21.15.86:0/873763659 learned_addr learned my addr 172.21.15.86:0/873763659 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:22.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.131+0000 7f40b4f4f640 1 --2- 172.21.15.86:0/873763659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2da5d0ec0 0x55c2da645d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.131+0000 7f40af7fe640 1 -- 172.21.15.86:0/873763659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2da5d0ec0 msgr2=0x55c2da645d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.131+0000 7f40af7fe640 1 --2- 172.21.15.86:0/873763659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2da5d0ec0 0x55c2da645d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.131+0000 7f40af7fe640 1 -- 172.21.15.86:0/873763659 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2da5c79c0 msgr2=0x55c2da5d4f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:22.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.131+0000 7f40affff640 1 --2- 172.21.15.86:0/873763659 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2da5c79c0 0x55c2da5d4f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.131+0000 7f40af7fe640 1 --2- 172.21.15.86:0/873763659 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2da5c79c0 0x55c2da5d4f00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.131+0000 7f40af7fe640 1 -- 172.21.15.86:0/873763659 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2da5c8110 con 0x55c2da5c82e0 2024-04-21T08:48:22.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.131+0000 7f40b4f4f640 1 --2- 172.21.15.86:0/873763659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2da5d0ec0 0x55c2da645d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:22.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.131+0000 7f40affff640 1 --2- 172.21.15.86:0/873763659 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2da5c79c0 0x55c2da5d4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:22.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.131+0000 7f40af7fe640 1 --2- 172.21.15.86:0/873763659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2da5c82e0 0x55c2da5d5440 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f40a00ce180 tx=0x7f40a000dee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e63335b7bf4e2bce server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:22.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.131+0000 7f408f7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:22.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.131+0000 7f408f7fe640 1 -- 172.21.15.86:0/873763659 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a00a0d20 con 0x55c2da5c82e0 2024-04-21T08:48:22.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.131+0000 7f408f7fe640 1 -- 172.21.15.86:0/873763659 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f40a000f740 con 0x55c2da5c82e0 2024-04-21T08:48:22.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.131+0000 7f40b95a3740 1 -- 172.21.15.86:0/873763659 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f40a4048070 con 0x55c2da5c82e0 2024-04-21T08:48:22.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.132+0000 7f408f7fe640 1 -- 172.21.15.86:0/873763659 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f40a00a55e0 con 0x55c2da5c82e0 2024-04-21T08:48:22.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.132+0000 7f40b95a3740 10 client.19482 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:22.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.132+0000 7f40b95a3740 1 -- 172.21.15.86:0/873763659 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c2da647470 con 0x55c2da5c82e0 2024-04-21T08:48:22.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.132+0000 7f408f7fe640 1 -- 172.21.15.86:0/873763659 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40a4048070 con 0x55c2da5c82e0 2024-04-21T08:48:22.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.132+0000 7f408f7fe640 1 -- 172.21.15.86:0/873763659 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f40a00870b0 con 0x55c2da5c82e0 2024-04-21T08:48:22.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.132+0000 7f40b95a3740 10 client.19482 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:22.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.132+0000 7f40b95a3740 10 client.19482 resolve_mds: validated gid 24482 aka daemon mds.b 2024-04-21T08:48:22.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.132+0000 7f40b95a3740 20 client.19482 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:22.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.132+0000 7f40b95a3740 1 --2- 172.21.15.86:0/873763659 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c2da647c40 0x55c2da64a060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:22.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.132+0000 7f40b95a3740 4 client.19482 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "roots": ["/"], "timeout": 30.0, "expiration": 93.1, "await": true}] 2024-04-21T08:48:22.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.132+0000 7f40b95a3740 1 -- 172.21.15.86:0/873763659 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "quiesce db", "roots": ["/"], "timeout": 30.0, "expiration": 93.1, "await": true}) -- 0x55c2da533980 con 0x55c2da647c40 2024-04-21T08:48:22.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.132+0000 7f40affff640 1 --2- 172.21.15.86:0/873763659 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c2da647c40 0x55c2da64a060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:22.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.132+0000 7f408f7fe640 10 client.19482 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:22.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.132+0000 7f40affff640 1 --2- 172.21.15.86:0/873763659 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c2da647c40 0x55c2da64a060 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8a44c66ea96e40d0 server_cookie=9502ea6710ccdb2a in_seq=0 out_seq=0 2024-04-21T08:48:22.147 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.146+0000 7f408f7fe640 1 -- 172.21.15.86:0/873763659 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x55c2da533980 con 0x55c2da647c40 2024-04-21T08:48:22.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.146+0000 7f408f7fe640 10 client.19482 handle_command_reply: tid=0 2024-04-21T08:48:22.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.146+0000 7f40b95a3740 1 client.19482 shutdown 2024-04-21T08:48:22.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.147+0000 7f40b95a3740 1 -- 172.21.15.86:0/873763659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2da5c82e0 msgr2=0x55c2da5d5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.147+0000 7f40b95a3740 1 --2- 172.21.15.86:0/873763659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2da5c82e0 0x55c2da5d5440 secure :-1 s=READY pgs=1756 cs=0 l=1 rev1=1 crypto rx=0x7f40a00ce180 tx=0x7f40a000dee0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.147+0000 7f40b95a3740 1 -- 172.21.15.86:0/873763659 shutdown_connections 2024-04-21T08:48:22.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.147+0000 7f40b95a3740 1 --2- 172.21.15.86:0/873763659 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c2da647c40 0x55c2da64a060 crc :-1 s=READY pgs=282 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.147+0000 7f40b95a3740 1 --2- 172.21.15.86:0/873763659 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2da5d0ec0 0x55c2da645d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.147+0000 7f40b95a3740 1 --2- 172.21.15.86:0/873763659 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2da5c82e0 0x55c2da5d5440 unknown :-1 s=CLOSED pgs=1756 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.147+0000 7f40b95a3740 1 --2- 172.21.15.86:0/873763659 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2da5c79c0 0x55c2da5d4f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.147+0000 7f408f7fe640 0 client.19482 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:22.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.147+0000 7f40b95a3740 1 -- 172.21.15.86:0/873763659 >> 172.21.15.86:0/873763659 conn(0x55c2da5c20f0 msgr2=0x55c2da5ce860 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:22.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.147+0000 7f40b95a3740 1 -- 172.21.15.86:0/873763659 shutdown_connections 2024-04-21T08:48:22.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.147+0000 7f40b95a3740 1 -- 172.21.15.86:0/873763659 wait complete. 2024-04-21T08:48:22.148 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.147+0000 7f40b95a3740 20 client.19482 trim_cache size 0 max 16384 2024-04-21T08:48:22.149 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:48:22.149 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 29, 2024-04-21T08:48:22.149 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24482, 2024-04-21T08:48:22.149 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 10, 2024-04-21T08:48:22.149 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-04-21T08:48:22.149 INFO:teuthology.orchestra.run.smithi086.stdout: "bea13946": { 2024-04-21T08:48:22.149 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 10, 2024-04-21T08:48:22.150 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-04-21T08:48:22.150 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-21T08:48:22.150 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-04-21T08:48:22.150 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-21T08:48:22.150 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:48:22.150 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 30.0, 2024-04-21T08:48:22.150 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 93.1, 2024-04-21T08:48:22.150 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-04-21T08:48:22.150 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-04-21T08:48:22.150 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-04-21T08:48:22.150 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-21T08:48:22.150 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "QUIESCED", 2024-04-21T08:48:22.151 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-21T08:48:22.152 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:48:22.152 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:48:22.152 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:48:22.152 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:48:22.152 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:48:22.152 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:48:22.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.147+0000 7f40b69b2640 1 -- 172.21.15.86:0/3047054027 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f40980a3d20 msgr2=0x7f40980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.147+0000 7f40b69b2640 1 --2- 172.21.15.86:0/3047054027 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f40980a3d20 0x7f40980a61e0 secure :-1 s=READY pgs=288 cs=0 l=1 rev1=1 crypto rx=0x7f40a8014a40 tx=0x7f40a8009c70 comp rx=0 tx=0).stop 2024-04-21T08:48:22.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.147+0000 7f40b69b2640 1 -- 172.21.15.86:0/3047054027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40b0103b00 msgr2=0x7f40b0198c90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:22.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.147+0000 7f40b69b2640 1 --2- 172.21.15.86:0/3047054027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40b0103b00 0x7f40b0198c90 secure :-1 s=READY pgs=1752 cs=0 l=1 rev1=1 crypto rx=0x7f40a000c920 tx=0x7f40a000cde0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.147+0000 7f40b69b2640 1 -- 172.21.15.86:0/3047054027 shutdown_connections 2024-04-21T08:48:22.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.147+0000 7f40b69b2640 1 --2- 172.21.15.86:0/3047054027 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f40980a3d20 0x7f40980a61e0 unknown :-1 s=CLOSED pgs=288 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.147+0000 7f40b69b2640 1 --2- 172.21.15.86:0/3047054027 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40b010c520 0x7f40b019d010 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.147+0000 7f40b69b2640 1 --2- 172.21.15.86:0/3047054027 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40b0103b00 0x7f40b0198c90 unknown :-1 s=CLOSED pgs=1752 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.147+0000 7f40b69b2640 1 --2- 172.21.15.86:0/3047054027 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40b01031e0 0x7f40b0198750 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:22.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.147+0000 7f40b69b2640 1 -- 172.21.15.86:0/3047054027 >> 172.21.15.86:0/3047054027 conn(0x7f40b0077f50 msgr2=0x7f40b00ff2d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:22.153 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.147+0000 7f40b69b2640 1 -- 172.21.15.86:0/3047054027 shutdown_connections 2024-04-21T08:48:22.154 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:22.147+0000 7f40b69b2640 1 -- 172.21.15.86:0/3047054027 wait complete. 2024-04-21T08:48:22.158 INFO:tasks.quiescer.fs.[cephfs]:Successfully quiesced, set_id: bea13946, quiesce duration: 33.1 2024-04-21T08:48:22.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:21 smithi088 ceph-mon[31259]: pgmap v298: 129 pgs: 129 active+clean; 1.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:48:22.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:21 smithi179 ceph-mon[31829]: pgmap v298: 129 pgs: 129 active+clean; 1.9 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:48:23.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:22 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2348542442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:48:23.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:22 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2348542442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:48:23.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:22 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2348542442' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:48:24.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:23 smithi088 ceph-mon[31259]: pgmap v299: 129 pgs: 129 active+clean; 2.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-21T08:48:24.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:23 smithi179 ceph-mon[31829]: pgmap v299: 129 pgs: 129 active+clean; 2.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-21T08:48:24.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:23 smithi086 ceph-mon[27939]: pgmap v299: 129 pgs: 129 active+clean; 2.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 56 KiB/s wr, 21 op/s 2024-04-21T08:48:26.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:26 smithi086 ceph-mon[27939]: pgmap v300: 129 pgs: 129 active+clean; 2.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-21T08:48:26.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:26 smithi088 ceph-mon[31259]: pgmap v300: 129 pgs: 129 active+clean; 2.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-21T08:48:26.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:26 smithi179 ceph-mon[31829]: pgmap v300: 129 pgs: 129 active+clean; 2.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-21T08:48:27.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:27 smithi086 ceph-mon[27939]: pgmap v301: 129 pgs: 129 active+clean; 2.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-21T08:48:27.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:27 smithi088 ceph-mon[31259]: pgmap v301: 129 pgs: 129 active+clean; 2.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-21T08:48:27.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:27 smithi179 ceph-mon[31829]: pgmap v301: 129 pgs: 129 active+clean; 2.1 MiB data, 418 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-21T08:48:30.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:29 smithi088 ceph-mon[31259]: pgmap v302: 129 pgs: 129 active+clean; 2.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 KiB/s wr, 20 op/s 2024-04-21T08:48:30.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:29 smithi179 ceph-mon[31829]: pgmap v302: 129 pgs: 129 active+clean; 2.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 KiB/s wr, 20 op/s 2024-04-21T08:48:30.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:29 smithi086 ceph-mon[27939]: pgmap v302: 129 pgs: 129 active+clean; 2.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 41 KiB/s wr, 20 op/s 2024-04-21T08:48:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.070+0000 7fbdecc26640 1 -- 172.21.15.86:0/758698887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde81006c0 msgr2=0x7fbde8100aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.070+0000 7fbdecc26640 1 --2- 172.21.15.86:0/758698887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde81006c0 0x7fbde8100aa0 secure :-1 s=READY pgs=1757 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8009900 tx=0x7fbdd802bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.070+0000 7fbdecc26640 1 -- 172.21.15.86:0/758698887 shutdown_connections 2024-04-21T08:48:32.071 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.070+0000 7fbdecc26640 1 --2- 172.21.15.86:0/758698887 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbde8107330 0x7fbde810eeb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.070+0000 7fbdecc26640 1 --2- 172.21.15.86:0/758698887 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbde8101070 0x7fbde8106c00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.070+0000 7fbdecc26640 1 --2- 172.21.15.86:0/758698887 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde81006c0 0x7fbde8100aa0 unknown :-1 s=CLOSED pgs=1757 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.070+0000 7fbdecc26640 1 -- 172.21.15.86:0/758698887 >> 172.21.15.86:0/758698887 conn(0x7fbde8077390 msgr2=0x7fbde80fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.070+0000 7fbdecc26640 1 -- 172.21.15.86:0/758698887 shutdown_connections 2024-04-21T08:48:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.070+0000 7fbdecc26640 1 -- 172.21.15.86:0/758698887 wait complete. 2024-04-21T08:48:32.072 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.071+0000 7fbdecc26640 1 Processor -- start 2024-04-21T08:48:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.071+0000 7fbdecc26640 1 -- start start 2024-04-21T08:48:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.072+0000 7fbdecc26640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbde81006c0 0x7fbde8101630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.072+0000 7fbdecc26640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde8107330 0x7fbde8101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.072+0000 7fbdecc26640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbde81020b0 0x7fbde81a6850 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:32.074 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.072+0000 7fbdecc26640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbde8112220 con 0x7fbde8107330 2024-04-21T08:48:32.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.072+0000 7fbdecc26640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbde81120a0 con 0x7fbde81020b0 2024-04-21T08:48:32.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.072+0000 7fbdecc26640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbde81123a0 con 0x7fbde81006c0 2024-04-21T08:48:32.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.072+0000 7fbdddd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde8107330 0x7fbde8101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:32.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.072+0000 7fbdddd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde8107330 0x7fbde8101b70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38220/0 (socket says 172.21.15.86:38220) 2024-04-21T08:48:32.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.072+0000 7fbdddd9b640 1 -- 172.21.15.86:0/1489993175 learned_addr learned my addr 172.21.15.86:0/1489993175 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:32.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.072+0000 7fbde659c640 1 --2- 172.21.15.86:0/1489993175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbde81006c0 0x7fbde8101630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:32.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.072+0000 7fbde6d9d640 1 --2- 172.21.15.86:0/1489993175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbde81020b0 0x7fbde81a6850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:32.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.072+0000 7fbdddd9b640 1 -- 172.21.15.86:0/1489993175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbde81006c0 msgr2=0x7fbde8101630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:32.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.072+0000 7fbdddd9b640 1 --2- 172.21.15.86:0/1489993175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbde81006c0 0x7fbde8101630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:32.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.072+0000 7fbdddd9b640 1 -- 172.21.15.86:0/1489993175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbde81020b0 msgr2=0x7fbde81a6850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:32.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.072+0000 7fbdddd9b640 1 --2- 172.21.15.86:0/1489993175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbde81020b0 0x7fbde81a6850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:32.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.072+0000 7fbdddd9b640 1 -- 172.21.15.86:0/1489993175 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbde81a6f50 con 0x7fbde8107330 2024-04-21T08:48:32.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.072+0000 7fbde6d9d640 1 --2- 172.21.15.86:0/1489993175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbde81020b0 0x7fbde81a6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:32.075 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.072+0000 7fbdddd9b640 1 --2- 172.21.15.86:0/1489993175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde8107330 0x7fbde8101b70 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000d950 tx=0x7fbdd000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:32.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.072+0000 7fbddffff640 1 -- 172.21.15.86:0/1489993175 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0017070 con 0x7fbde8107330 2024-04-21T08:48:32.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.072+0000 7fbdecc26640 1 -- 172.21.15.86:0/1489993175 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbde81a7240 con 0x7fbde8107330 2024-04-21T08:48:32.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.072+0000 7fbddffff640 1 -- 172.21.15.86:0/1489993175 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdd0009c30 con 0x7fbde8107330 2024-04-21T08:48:32.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.072+0000 7fbddffff640 1 -- 172.21.15.86:0/1489993175 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd00047a0 con 0x7fbde8107330 2024-04-21T08:48:32.076 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.072+0000 7fbdecc26640 1 -- 172.21.15.86:0/1489993175 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbde81a77e0 con 0x7fbde8107330 2024-04-21T08:48:32.083 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.075+0000 7fbdef80e740 1 Processor -- start 2024-04-21T08:48:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.075+0000 7fbddffff640 1 -- 172.21.15.86:0/1489993175 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbdd0004020 con 0x7fbde8107330 2024-04-21T08:48:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.076+0000 7fbdef80e740 1 -- start start 2024-04-21T08:48:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.076+0000 7fbdef80e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5620f85d58d0 0x5620f85d4620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.076+0000 7fbdef80e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620f85d4b60 0x5620f85d4fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.076+0000 7fbdef80e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5620f85de0e0 0x5620f85e04d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.076+0000 7fbdef80e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5620f85398d0 con 0x5620f85d4b60 2024-04-21T08:48:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.076+0000 7fbdef80e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5620f84d6e80 con 0x5620f85d58d0 2024-04-21T08:48:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.076+0000 7fbdef80e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5620f8535890 con 0x5620f85de0e0 2024-04-21T08:48:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.076+0000 7fbde659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620f85d4b60 0x5620f85d4fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.076+0000 7fbde659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620f85d4b60 0x5620f85d4fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38224/0 (socket says 172.21.15.86:38224) 2024-04-21T08:48:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.076+0000 7fbde659c640 1 -- 172.21.15.86:0/1369875635 learned_addr learned my addr 172.21.15.86:0/1369875635 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.076+0000 7fbde6d9d640 1 --2- 172.21.15.86:0/1369875635 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5620f85d58d0 0x5620f85d4620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:32.084 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.076+0000 7fbdddd9b640 1 --2- 172.21.15.86:0/1369875635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5620f85de0e0 0x5620f85e04d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.076+0000 7fbddffff640 1 --2- 172.21.15.86:0/1489993175 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbdd40a3df0 0x7fbdd40a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.076+0000 7fbde659c640 1 -- 172.21.15.86:0/1369875635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5620f85de0e0 msgr2=0x5620f85e04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.076+0000 7fbde659c640 1 --2- 172.21.15.86:0/1369875635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5620f85de0e0 0x5620f85e04d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.076+0000 7fbde659c640 1 -- 172.21.15.86:0/1369875635 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5620f85d58d0 msgr2=0x5620f85d4620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.076+0000 7fbde659c640 1 --2- 172.21.15.86:0/1369875635 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5620f85d58d0 0x5620f85d4620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.076+0000 7fbde659c640 1 -- 172.21.15.86:0/1369875635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620f84f8320 con 0x5620f85d4b60 2024-04-21T08:48:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.076+0000 7fbde6d9d640 1 --2- 172.21.15.86:0/1369875635 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5620f85d58d0 0x5620f85d4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.076+0000 7fbde6d9d640 1 --2- 172.21.15.86:0/1489993175 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbdd40a3df0 0x7fbdd40a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.076+0000 7fbddffff640 1 -- 172.21.15.86:0/1489993175 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7fbdd00d08e0 con 0x7fbde8107330 2024-04-21T08:48:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.076+0000 7fbde659c640 1 --2- 172.21.15.86:0/1369875635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620f85d4b60 0x5620f85d4fe0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8009900 tx=0x7fbdd802c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6cf8fb5fe89c81c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.077+0000 7fbddcd99640 1 -- 172.21.15.86:0/1369875635 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd8044020 con 0x5620f85d4b60 2024-04-21T08:48:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.077+0000 7fbddcd99640 1 -- 172.21.15.86:0/1369875635 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdd802cc90 con 0x5620f85d4b60 2024-04-21T08:48:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.077+0000 7fbddcd99640 1 -- 172.21.15.86:0/1369875635 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd80046c0 con 0x5620f85d4b60 2024-04-21T08:48:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.077+0000 7fbdef80e740 1 -- 172.21.15.86:0/1369875635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620f85d4b60 msgr2=0x5620f85d4fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:32.085 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.077+0000 7fbde6d9d640 1 --2- 172.21.15.86:0/1489993175 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbdd40a3df0 0x7fbdd40a62b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbde0005cc0 tx=0x7fbde0004a40 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.077+0000 7fbdef80e740 1 --2- 172.21.15.86:0/1369875635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620f85d4b60 0x5620f85d4fe0 secure :-1 s=READY pgs=1759 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8009900 tx=0x7fbdd802c650 comp rx=0 tx=0).stop 2024-04-21T08:48:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.077+0000 7fbdef80e740 1 -- 172.21.15.86:0/1369875635 shutdown_connections 2024-04-21T08:48:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.077+0000 7fbdef80e740 1 --2- 172.21.15.86:0/1369875635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5620f85de0e0 0x5620f85e04d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.077+0000 7fbdef80e740 1 --2- 172.21.15.86:0/1369875635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620f85d4b60 0x5620f85d4fe0 unknown :-1 s=CLOSED pgs=1759 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.077+0000 7fbdef80e740 1 --2- 172.21.15.86:0/1369875635 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5620f85d58d0 0x5620f85d4620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.077+0000 7fbdef80e740 1 -- 172.21.15.86:0/1369875635 >> 172.21.15.86:0/1369875635 conn(0x5620f859bc90 msgr2=0x5620f85cf850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.078+0000 7fbdef80e740 1 -- 172.21.15.86:0/1369875635 shutdown_connections 2024-04-21T08:48:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.078+0000 7fbdef80e740 1 -- 172.21.15.86:0/1369875635 wait complete. 2024-04-21T08:48:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.078+0000 7fbdef80e740 1 -- start start 2024-04-21T08:48:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.078+0000 7fbdef80e740 1 Processor -- start 2024-04-21T08:48:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.078+0000 7fbdef80e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620f85d4b60 0x5620f85dda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.078+0000 7fbdef80e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5620f85d58d0 0x5620f85dae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:32.086 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.078+0000 7fbdef80e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5620f85de0e0 0x5620f85db3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.078+0000 7fbdef80e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5620f85e0db0 con 0x5620f85d4b60 2024-04-21T08:48:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.078+0000 7fbdef80e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5620f84c1eb0 con 0x5620f85d58d0 2024-04-21T08:48:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.078+0000 7fbdef80e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5620f85e0f30 con 0x5620f85de0e0 2024-04-21T08:48:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.078+0000 7fbdef80e740 10 client.0 _ll_register_callbacks cb 0x5620f84eeb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.078+0000 7fbde659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620f85d4b60 0x5620f85dda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.078+0000 7fbde659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620f85d4b60 0x5620f85dda90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38230/0 (socket says 172.21.15.86:38230) 2024-04-21T08:48:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.078+0000 7fbde659c640 1 -- 172.21.15.86:0/4236612065 learned_addr learned my addr 172.21.15.86:0/4236612065 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.078+0000 7fbdddd9b640 1 --2- 172.21.15.86:0/4236612065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5620f85d58d0 0x5620f85dae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.079+0000 7fbde6d9d640 1 --2- 172.21.15.86:0/4236612065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5620f85de0e0 0x5620f85db3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.079+0000 7fbde659c640 1 -- 172.21.15.86:0/4236612065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5620f85de0e0 msgr2=0x5620f85db3c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.079+0000 7fbde659c640 1 --2- 172.21.15.86:0/4236612065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5620f85de0e0 0x5620f85db3c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.079+0000 7fbde659c640 1 -- 172.21.15.86:0/4236612065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5620f85d58d0 msgr2=0x5620f85dae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.079+0000 7fbde659c640 1 --2- 172.21.15.86:0/4236612065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5620f85d58d0 0x5620f85dae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:32.087 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.079+0000 7fbde659c640 1 -- 172.21.15.86:0/4236612065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620f824c210 con 0x5620f85d4b60 2024-04-21T08:48:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.079+0000 7fbde659c640 1 --2- 172.21.15.86:0/4236612065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620f85d4b60 0x5620f85dda90 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8000c00 tx=0x7fbdd802c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6ac192dc6a1cfefd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.079+0000 7fbdcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.079+0000 7fbdcdffb640 1 -- 172.21.15.86:0/4236612065 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd8044030 con 0x5620f85d4b60 2024-04-21T08:48:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.079+0000 7fbdcdffb640 1 -- 172.21.15.86:0/4236612065 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7fbdd80037f0 con 0x5620f85d4b60 2024-04-21T08:48:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.079+0000 7fbdef80e740 1 -- 172.21.15.86:0/4236612065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdd802cc90 con 0x5620f85d4b60 2024-04-21T08:48:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.079+0000 7fbdcdffb640 1 -- 172.21.15.86:0/4236612065 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbdd8057050 con 0x5620f85d4b60 2024-04-21T08:48:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.079+0000 7fbdef80e740 10 client.19494 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.079+0000 7fbdef80e740 1 -- 172.21.15.86:0/4236612065 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5620f8318620 con 0x5620f85d4b60 2024-04-21T08:48:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.079+0000 7fbdcdffb640 1 -- 172.21.15.86:0/4236612065 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdd802cc90 con 0x5620f85d4b60 2024-04-21T08:48:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.079+0000 7fbdcdffb640 1 -- 172.21.15.86:0/4236612065 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbdd80409e0 con 0x5620f85d4b60 2024-04-21T08:48:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.080+0000 7fbdef80e740 10 client.19494 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.080+0000 7fbdef80e740 10 client.19494 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.080+0000 7fbdef80e740 20 client.19494 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.080+0000 7fbdef80e740 1 --2- 172.21.15.86:0/4236612065 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5620f85edf20 0x5620f85f0310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:32.088 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.080+0000 7fbdef80e740 4 client.19494 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.080+0000 7fbdef80e740 1 -- 172.21.15.86:0/4236612065 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5620f854d880 con 0x5620f85edf20 2024-04-21T08:48:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.080+0000 7fbdddd9b640 1 --2- 172.21.15.86:0/4236612065 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5620f85edf20 0x5620f85f0310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.080+0000 7fbdcdffb640 10 client.19494 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.080+0000 7fbdddd9b640 1 --2- 172.21.15.86:0/4236612065 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5620f85edf20 0x5620f85f0310 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8970f29c86ce8f43 server_cookie=b7b1c5759170fe14 in_seq=0 out_seq=0 2024-04-21T08:48:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.081+0000 7fbdcdffb640 1 -- 172.21.15.86:0/4236612065 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5620f854d880 con 0x5620f85edf20 2024-04-21T08:48:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.081+0000 7fbdcdffb640 10 client.19494 handle_command_reply: tid=0 2024-04-21T08:48:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.081+0000 7fbdef80e740 1 client.19494 shutdown 2024-04-21T08:48:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.081+0000 7fbdef80e740 1 -- 172.21.15.86:0/4236612065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620f85d4b60 msgr2=0x5620f85dda90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.081+0000 7fbdef80e740 1 --2- 172.21.15.86:0/4236612065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620f85d4b60 0x5620f85dda90 secure :-1 s=READY pgs=1760 cs=0 l=1 rev1=1 crypto rx=0x7fbdd8000c00 tx=0x7fbdd802c740 comp rx=0 tx=0).stop 2024-04-21T08:48:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.081+0000 7fbdef80e740 1 -- 172.21.15.86:0/4236612065 shutdown_connections 2024-04-21T08:48:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.081+0000 7fbdef80e740 1 --2- 172.21.15.86:0/4236612065 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5620f85edf20 0x5620f85f0310 crc :-1 s=READY pgs=283 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.081+0000 7fbdef80e740 1 --2- 172.21.15.86:0/4236612065 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5620f85de0e0 0x5620f85db3c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.081+0000 7fbdef80e740 1 --2- 172.21.15.86:0/4236612065 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5620f85d58d0 0x5620f85dae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.081+0000 7fbdef80e740 1 --2- 172.21.15.86:0/4236612065 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620f85d4b60 0x5620f85dda90 unknown :-1 s=CLOSED pgs=1760 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:32.089 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.081+0000 7fbdcdffb640 0 client.19494 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.081+0000 7fbdef80e740 1 -- 172.21.15.86:0/4236612065 >> 172.21.15.86:0/4236612065 conn(0x5620f859bc90 msgr2=0x5620f85d15a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.081+0000 7fbdef80e740 1 -- 172.21.15.86:0/4236612065 shutdown_connections 2024-04-21T08:48:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.081+0000 7fbdef80e740 1 -- 172.21.15.86:0/4236612065 wait complete. 2024-04-21T08:48:32.090 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.081+0000 7fbdef80e740 20 client.19494 trim_cache size 0 max 16384 2024-04-21T08:48:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.096+0000 7fbdef80e740 1 Processor -- start 2024-04-21T08:48:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.096+0000 7fbdef80e740 1 -- start start 2024-04-21T08:48:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.096+0000 7fbdef80e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5620f85d4b60 0x5620f85e9cb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.096+0000 7fbdef80e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620f85d58d0 0x5620f85dd550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.096+0000 7fbdef80e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5620f85de0e0 0x5620f85dda90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:32.101 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.096+0000 7fbdef80e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbde81b14b0 con 0x5620f85d58d0 2024-04-21T08:48:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.096+0000 7fbdef80e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbde81b2ff0 con 0x5620f85de0e0 2024-04-21T08:48:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.096+0000 7fbdef80e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbde81b1b80 con 0x5620f85d4b60 2024-04-21T08:48:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.096+0000 7fbdddd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620f85d58d0 0x5620f85dd550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.096+0000 7fbdddd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620f85d58d0 0x5620f85dd550 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38232/0 (socket says 172.21.15.86:38232) 2024-04-21T08:48:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.096+0000 7fbdddd9b640 1 -- 172.21.15.86:0/1277704664 learned_addr learned my addr 172.21.15.86:0/1277704664 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.096+0000 7fbde659c640 1 --2- 172.21.15.86:0/1277704664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5620f85d4b60 0x5620f85e9cb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.096+0000 7fbdddd9b640 1 -- 172.21.15.86:0/1277704664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5620f85d4b60 msgr2=0x5620f85e9cb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.096+0000 7fbdddd9b640 1 --2- 172.21.15.86:0/1277704664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5620f85d4b60 0x5620f85e9cb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.096+0000 7fbdddd9b640 1 -- 172.21.15.86:0/1277704664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5620f85de0e0 msgr2=0x5620f85dda90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.096+0000 7fbde6d9d640 1 --2- 172.21.15.86:0/1277704664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5620f85de0e0 0x5620f85dda90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.096+0000 7fbdddd9b640 1 --2- 172.21.15.86:0/1277704664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5620f85de0e0 0x5620f85dda90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:32.102 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.096+0000 7fbdddd9b640 1 -- 172.21.15.86:0/1277704664 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5620f85d3e30 con 0x5620f85d58d0 2024-04-21T08:48:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.096+0000 7fbde659c640 1 --2- 172.21.15.86:0/1277704664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5620f85d4b60 0x5620f85e9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.096+0000 7fbde6d9d640 1 --2- 172.21.15.86:0/1277704664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5620f85de0e0 0x5620f85dda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.097+0000 7fbdddd9b640 1 --2- 172.21.15.86:0/1277704664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620f85d58d0 0x5620f85dd550 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fbdd00cdcc0 tx=0x7fbdd00d2e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cb1de48d029fb30 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.097+0000 7fbdcffff640 1 -- 172.21.15.86:0/1277704664 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0009de0 con 0x5620f85d58d0 2024-04-21T08:48:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.097+0000 7fbdcffff640 1 -- 172.21.15.86:0/1277704664 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdd00a0860 con 0x5620f85d58d0 2024-04-21T08:48:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.097+0000 7fbdcffff640 1 -- 172.21.15.86:0/1277704664 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbdd0098b20 con 0x5620f85d58d0 2024-04-21T08:48:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.097+0000 7fbdef80e740 1 -- 172.21.15.86:0/1277704664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620f85d58d0 msgr2=0x5620f85dd550 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.097+0000 7fbdef80e740 1 --2- 172.21.15.86:0/1277704664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620f85d58d0 0x5620f85dd550 secure :-1 s=READY pgs=1761 cs=0 l=1 rev1=1 crypto rx=0x7fbdd00cdcc0 tx=0x7fbdd00d2e60 comp rx=0 tx=0).stop 2024-04-21T08:48:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.097+0000 7fbdef80e740 1 -- 172.21.15.86:0/1277704664 shutdown_connections 2024-04-21T08:48:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.097+0000 7fbdef80e740 1 --2- 172.21.15.86:0/1277704664 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5620f85de0e0 0x5620f85dda90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.097+0000 7fbdef80e740 1 --2- 172.21.15.86:0/1277704664 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620f85d58d0 0x5620f85dd550 unknown :-1 s=CLOSED pgs=1761 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.097+0000 7fbdef80e740 1 --2- 172.21.15.86:0/1277704664 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5620f85d4b60 0x5620f85e9cb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:32.103 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.097+0000 7fbdef80e740 1 -- 172.21.15.86:0/1277704664 >> 172.21.15.86:0/1277704664 conn(0x5620f859bc90 msgr2=0x5620f85cf380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.097+0000 7fbdef80e740 1 -- 172.21.15.86:0/1277704664 shutdown_connections 2024-04-21T08:48:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.097+0000 7fbdef80e740 1 -- 172.21.15.86:0/1277704664 wait complete. 2024-04-21T08:48:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.098+0000 7fbdef80e740 1 -- start start 2024-04-21T08:48:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.098+0000 7fbdef80e740 1 Processor -- start 2024-04-21T08:48:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.098+0000 7fbdef80e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5620f85d4b60 0x5620f85e65e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.098+0000 7fbdef80e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5620f85d58d0 0x5620f85deac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.098+0000 7fbdef80e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620f85de0e0 0x5620f85df040 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.098+0000 7fbdef80e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbde81b2920 con 0x5620f85de0e0 2024-04-21T08:48:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.098+0000 7fbdef80e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbde81b2250 con 0x5620f85d4b60 2024-04-21T08:48:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.098+0000 7fbdef80e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbde81af4b0 con 0x5620f85d58d0 2024-04-21T08:48:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.098+0000 7fbdef80e740 10 client.0 _ll_register_callbacks cb 0x7fbdd8005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.098+0000 7fbde6d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620f85de0e0 0x5620f85df040 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.098+0000 7fbde6d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620f85de0e0 0x5620f85df040 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38240/0 (socket says 172.21.15.86:38240) 2024-04-21T08:48:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.098+0000 7fbde6d9d640 1 -- 172.21.15.86:0/1194454780 learned_addr learned my addr 172.21.15.86:0/1194454780 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:32.104 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.098+0000 7fbdddd9b640 1 --2- 172.21.15.86:0/1194454780 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5620f85d58d0 0x5620f85deac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.098+0000 7fbde659c640 1 --2- 172.21.15.86:0/1194454780 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5620f85d4b60 0x5620f85e65e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.098+0000 7fbde6d9d640 1 -- 172.21.15.86:0/1194454780 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5620f85d58d0 msgr2=0x5620f85deac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.098+0000 7fbde6d9d640 1 --2- 172.21.15.86:0/1194454780 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5620f85d58d0 0x5620f85deac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.098+0000 7fbde6d9d640 1 -- 172.21.15.86:0/1194454780 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5620f85d4b60 msgr2=0x5620f85e65e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.098+0000 7fbde6d9d640 1 --2- 172.21.15.86:0/1194454780 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5620f85d4b60 0x5620f85e65e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.098+0000 7fbde6d9d640 1 -- 172.21.15.86:0/1194454780 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5620f85df8c0 con 0x5620f85de0e0 2024-04-21T08:48:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.098+0000 7fbde659c640 1 --2- 172.21.15.86:0/1194454780 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5620f85d4b60 0x5620f85e65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.098+0000 7fbde6d9d640 1 --2- 172.21.15.86:0/1194454780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620f85de0e0 0x5620f85df040 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fbde001c9b0 tx=0x7fbde00084c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=de126c649ece787a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.098+0000 7fbdcdffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.099+0000 7fbdcdffb640 1 -- 172.21.15.86:0/1194454780 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbde0011940 con 0x5620f85de0e0 2024-04-21T08:48:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.099+0000 7fbdcdffb640 1 -- 172.21.15.86:0/1194454780 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7fbde0011ae0 con 0x5620f85de0e0 2024-04-21T08:48:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.099+0000 7fbdef80e740 1 -- 172.21.15.86:0/1194454780 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbdd00a0860 con 0x5620f85de0e0 2024-04-21T08:48:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.099+0000 7fbdcdffb640 1 -- 172.21.15.86:0/1194454780 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbde002f050 con 0x5620f85de0e0 2024-04-21T08:48:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.099+0000 7fbdef80e740 10 client.19500 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:32.105 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.099+0000 7fbdef80e740 1 -- 172.21.15.86:0/1194454780 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5620f8654240 con 0x5620f85de0e0 2024-04-21T08:48:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.099+0000 7fbdcdffb640 1 -- 172.21.15.86:0/1194454780 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbdd00a0860 con 0x5620f85de0e0 2024-04-21T08:48:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.099+0000 7fbdcdffb640 1 -- 172.21.15.86:0/1194454780 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbde0026420 con 0x5620f85de0e0 2024-04-21T08:48:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.099+0000 7fbdef80e740 10 client.19500 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.099+0000 7fbdef80e740 10 client.19500 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.099+0000 7fbdef80e740 20 client.19500 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.099+0000 7fbdef80e740 1 --2- 172.21.15.86:0/1194454780 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5620f8654a20 0x5620f8656e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.099+0000 7fbdef80e740 4 client.19500 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:48:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.099+0000 7fbdef80e740 1 -- 172.21.15.86:0/1194454780 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5620f84d93d0 con 0x5620f8654a20 2024-04-21T08:48:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.099+0000 7fbde659c640 1 --2- 172.21.15.86:0/1194454780 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5620f8654a20 0x5620f8656e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.100+0000 7fbdcdffb640 10 client.19500 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:32.106 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.100+0000 7fbde659c640 1 --2- 172.21.15.86:0/1194454780 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5620f8654a20 0x5620f8656e40 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=bad6b2125f31bdb0 server_cookie=d08f6c7e2dfd3cd7 in_seq=0 out_seq=0 2024-04-21T08:48:32.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.113+0000 7fbdcdffb640 1 -- 172.21.15.86:0/1194454780 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5620f84d93d0 con 0x5620f8654a20 2024-04-21T08:48:32.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.113+0000 7fbdcdffb640 10 client.19500 handle_command_reply: tid=0 2024-04-21T08:48:32.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.113+0000 7fbdef80e740 1 client.19500 shutdown 2024-04-21T08:48:32.114 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.113+0000 7fbdef80e740 1 -- 172.21.15.86:0/1194454780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620f85de0e0 msgr2=0x5620f85df040 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:32.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.113+0000 7fbdef80e740 1 --2- 172.21.15.86:0/1194454780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620f85de0e0 0x5620f85df040 secure :-1 s=READY pgs=1762 cs=0 l=1 rev1=1 crypto rx=0x7fbde001c9b0 tx=0x7fbde00084c0 comp rx=0 tx=0).stop 2024-04-21T08:48:32.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.113+0000 7fbdef80e740 1 -- 172.21.15.86:0/1194454780 shutdown_connections 2024-04-21T08:48:32.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.113+0000 7fbdef80e740 1 --2- 172.21.15.86:0/1194454780 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5620f8654a20 0x5620f8656e40 crc :-1 s=READY pgs=284 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:32.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.113+0000 7fbdef80e740 1 --2- 172.21.15.86:0/1194454780 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5620f85de0e0 0x5620f85df040 unknown :-1 s=CLOSED pgs=1762 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:32.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.113+0000 7fbdef80e740 1 --2- 172.21.15.86:0/1194454780 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5620f85d58d0 0x5620f85deac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:32.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.113+0000 7fbdcdffb640 0 client.19500 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:32.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.113+0000 7fbdef80e740 1 --2- 172.21.15.86:0/1194454780 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5620f85d4b60 0x5620f85e65e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:32.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.113+0000 7fbdef80e740 1 -- 172.21.15.86:0/1194454780 >> 172.21.15.86:0/1194454780 conn(0x5620f859bc90 msgr2=0x5620f85d9480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:32.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.113+0000 7fbdef80e740 1 -- 172.21.15.86:0/1194454780 shutdown_connections 2024-04-21T08:48:32.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.114+0000 7fbdef80e740 1 -- 172.21.15.86:0/1194454780 wait complete. 2024-04-21T08:48:32.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.114+0000 7fbdef80e740 20 client.19500 trim_cache size 0 max 16384 2024-04-21T08:48:32.115 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:48:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:48:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:48:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:48:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.114+0000 7fbdecc26640 1 -- 172.21.15.86:0/1489993175 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbdd40a3df0 msgr2=0x7fbdd40a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:32.116 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.114+0000 7fbdecc26640 1 --2- 172.21.15.86:0/1489993175 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbdd40a3df0 0x7fbdd40a62b0 secure :-1 s=READY pgs=289 cs=0 l=1 rev1=1 crypto rx=0x7fbde0005cc0 tx=0x7fbde0004a40 comp rx=0 tx=0).stop 2024-04-21T08:48:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.114+0000 7fbdecc26640 1 -- 172.21.15.86:0/1489993175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde8107330 msgr2=0x7fbde8101b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.114+0000 7fbdecc26640 1 --2- 172.21.15.86:0/1489993175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde8107330 0x7fbde8101b70 secure :-1 s=READY pgs=1758 cs=0 l=1 rev1=1 crypto rx=0x7fbdd000d950 tx=0x7fbdd000de10 comp rx=0 tx=0).stop 2024-04-21T08:48:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.114+0000 7fbdecc26640 1 -- 172.21.15.86:0/1489993175 shutdown_connections 2024-04-21T08:48:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.114+0000 7fbdecc26640 1 --2- 172.21.15.86:0/1489993175 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbdd40a3df0 0x7fbdd40a62b0 unknown :-1 s=CLOSED pgs=289 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.114+0000 7fbdecc26640 1 --2- 172.21.15.86:0/1489993175 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbde81020b0 0x7fbde81a6850 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.114+0000 7fbdecc26640 1 --2- 172.21.15.86:0/1489993175 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbde8107330 0x7fbde8101b70 unknown :-1 s=CLOSED pgs=1758 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.114+0000 7fbdecc26640 1 --2- 172.21.15.86:0/1489993175 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbde81006c0 0x7fbde8101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.114+0000 7fbdecc26640 1 -- 172.21.15.86:0/1489993175 >> 172.21.15.86:0/1489993175 conn(0x7fbde8077390 msgr2=0x7fbde80fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.114+0000 7fbdecc26640 1 -- 172.21.15.86:0/1489993175 shutdown_connections 2024-04-21T08:48:32.117 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:32.114+0000 7fbdecc26640 1 -- 172.21.15.86:0/1489993175 wait complete. 2024-04-21T08:48:32.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:31 smithi088 ceph-mon[31259]: pgmap v303: 129 pgs: 129 active+clean; 2.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-04-21T08:48:32.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:31 smithi179 ceph-mon[31829]: pgmap v303: 129 pgs: 129 active+clean; 2.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-04-21T08:48:32.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:31 smithi086 ceph-mon[27939]: pgmap v303: 129 pgs: 129 active+clean; 2.1 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 27 KiB/s rd, 23 KiB/s wr, 13 op/s 2024-04-21T08:48:34.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:33 smithi088 ceph-mon[31259]: pgmap v304: 129 pgs: 129 active+clean; 2.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 20 op/s 2024-04-21T08:48:34.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:33 smithi179 ceph-mon[31829]: pgmap v304: 129 pgs: 129 active+clean; 2.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 20 op/s 2024-04-21T08:48:34.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:33 smithi086 ceph-mon[27939]: pgmap v304: 129 pgs: 129 active+clean; 2.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 31 KiB/s wr, 20 op/s 2024-04-21T08:48:35.499 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:48:35.499 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:48:35.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.612+0000 7f1161d8f640 1 -- 172.21.15.86:0/2545317630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f115c1003e0 msgr2=0x7f115c100820 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:35.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.612+0000 7f1161d8f640 1 --2- 172.21.15.86:0/2545317630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f115c1003e0 0x7f115c100820 secure :-1 s=READY pgs=1763 cs=0 l=1 rev1=1 crypto rx=0x7f1150009930 tx=0x7f115002bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:35.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.612+0000 7f1161d8f640 1 -- 172.21.15.86:0/2545317630 shutdown_connections 2024-04-21T08:48:35.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.612+0000 7f1161d8f640 1 --2- 172.21.15.86:0/2545317630 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f115c100d60 0x7f115c10ee00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.612+0000 7f1161d8f640 1 --2- 172.21.15.86:0/2545317630 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f115c1003e0 0x7f115c100820 unknown :-1 s=CLOSED pgs=1763 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.612 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.612+0000 7f1161d8f640 1 --2- 172.21.15.86:0/2545317630 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f115c107fd0 0x7f115c1083b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.612+0000 7f1161d8f640 1 -- 172.21.15.86:0/2545317630 >> 172.21.15.86:0/2545317630 conn(0x7f115c077360 msgr2=0x7f115c0fe6f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:35.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.612+0000 7f1161d8f640 1 -- 172.21.15.86:0/2545317630 shutdown_connections 2024-04-21T08:48:35.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.612+0000 7f1161d8f640 1 -- 172.21.15.86:0/2545317630 wait complete. 2024-04-21T08:48:35.613 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.613+0000 7f1161d8f640 1 Processor -- start 2024-04-21T08:48:35.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.613+0000 7f1161d8f640 1 -- start start 2024-04-21T08:48:35.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.613+0000 7f1161d8f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f115c1003e0 0x7f115c19e430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:35.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.613+0000 7f1161d8f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f115c100d60 0x7f115c19e970 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:35.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.613+0000 7f1161d8f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f115c107fd0 0x7f115c1985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:35.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.613+0000 7f1161d8f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f115c112190 con 0x7f115c107fd0 2024-04-21T08:48:35.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.613+0000 7f1161d8f640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f115c112010 con 0x7f115c1003e0 2024-04-21T08:48:35.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.613+0000 7f1161d8f640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f115c112310 con 0x7f115c100d60 2024-04-21T08:48:35.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.613+0000 7f115bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f115c107fd0 0x7f115c1985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:35.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.613+0000 7f115bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f115c107fd0 0x7f115c1985e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38262/0 (socket says 172.21.15.86:38262) 2024-04-21T08:48:35.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.614+0000 7f115bfff640 1 -- 172.21.15.86:0/1225716105 learned_addr learned my addr 172.21.15.86:0/1225716105 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:35.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.614+0000 7f115affd640 1 --2- 172.21.15.86:0/1225716105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f115c100d60 0x7f115c19e970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:35.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.614+0000 7f115b7fe640 1 --2- 172.21.15.86:0/1225716105 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f115c1003e0 0x7f115c19e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:35.615 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.614+0000 7f115bfff640 1 -- 172.21.15.86:0/1225716105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f115c100d60 msgr2=0x7f115c19e970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:35.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.614+0000 7f115bfff640 1 --2- 172.21.15.86:0/1225716105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f115c100d60 0x7f115c19e970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.614+0000 7f115bfff640 1 -- 172.21.15.86:0/1225716105 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f115c1003e0 msgr2=0x7f115c19e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:35.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.614+0000 7f115bfff640 1 --2- 172.21.15.86:0/1225716105 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f115c1003e0 0x7f115c19e430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.614+0000 7f115bfff640 1 -- 172.21.15.86:0/1225716105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f115c198d60 con 0x7f115c107fd0 2024-04-21T08:48:35.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.614+0000 7f115bfff640 1 --2- 172.21.15.86:0/1225716105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f115c107fd0 0x7f115c1985e0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f115400cad0 tx=0x7f115400cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:35.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.614+0000 7f1158ff9640 1 -- 172.21.15.86:0/1225716105 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115400ae20 con 0x7f115c107fd0 2024-04-21T08:48:35.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.614+0000 7f1161d8f640 1 -- 172.21.15.86:0/1225716105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f115c199050 con 0x7f115c107fd0 2024-04-21T08:48:35.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.614+0000 7f1158ff9640 1 -- 172.21.15.86:0/1225716105 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1154005020 con 0x7f115c107fd0 2024-04-21T08:48:35.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.614+0000 7f1158ff9640 1 -- 172.21.15.86:0/1225716105 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115400f6b0 con 0x7f115c107fd0 2024-04-21T08:48:35.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.614+0000 7f1161d8f640 1 -- 172.21.15.86:0/1225716105 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f115c1a52f0 con 0x7f115c107fd0 2024-04-21T08:48:35.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.616+0000 7f1158ff9640 1 -- 172.21.15.86:0/1225716105 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f11540051d0 con 0x7f115c107fd0 2024-04-21T08:48:35.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.617+0000 7f1164980740 1 Processor -- start 2024-04-21T08:48:35.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.617+0000 7f1164980740 1 -- start start 2024-04-21T08:48:35.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.617+0000 7f1158ff9640 1 --2- 172.21.15.86:0/1225716105 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f11440a3d20 0x7f11440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:35.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.617+0000 7f1164980740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d15ad2b0 0x5576d15ad690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:35.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.617+0000 7f1164980740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d15adbd0 0x5576d15ba950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:35.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.617+0000 7f1164980740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d15bae90 0x5576d15bd280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:35.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.617+0000 7f1164980740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5576d15121e0 con 0x5576d15adbd0 2024-04-21T08:48:35.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.617+0000 7f1164980740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5576d14af390 con 0x5576d15ad2b0 2024-04-21T08:48:35.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.617+0000 7f1164980740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5576d150e1a0 con 0x5576d15bae90 2024-04-21T08:48:35.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.617+0000 7f115affd640 1 --2- 172.21.15.86:0/1225716105 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f11440a3d20 0x7f11440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:35.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.617+0000 7f1158ff9640 1 -- 172.21.15.86:0/1225716105 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f11540d0d50 con 0x7f115c107fd0 2024-04-21T08:48:35.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.617+0000 7f115affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d15adbd0 0x5576d15ba950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:35.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.617+0000 7f115affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d15adbd0 0x5576d15ba950 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38272/0 (socket says 172.21.15.86:38272) 2024-04-21T08:48:35.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.617+0000 7f115affd640 1 -- 172.21.15.86:0/3385477360 learned_addr learned my addr 172.21.15.86:0/3385477360 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:35.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.617+0000 7f115bfff640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d15bae90 0x5576d15bd280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:35.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.617+0000 7f115b7fe640 1 --2- 172.21.15.86:0/3385477360 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d15ad2b0 0x5576d15ad690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:35.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.617+0000 7f115affd640 1 -- 172.21.15.86:0/3385477360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d15bae90 msgr2=0x5576d15bd280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:35.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.617+0000 7f115affd640 1 --2- 172.21.15.86:0/3385477360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d15bae90 0x5576d15bd280 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.617+0000 7f115affd640 1 -- 172.21.15.86:0/3385477360 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d15ad2b0 msgr2=0x5576d15ad690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:35.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.617+0000 7f115affd640 1 --2- 172.21.15.86:0/3385477360 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d15ad2b0 0x5576d15ad690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.617+0000 7f115affd640 1 -- 172.21.15.86:0/3385477360 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576d12ee7d0 con 0x5576d15adbd0 2024-04-21T08:48:35.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.618+0000 7f115affd640 1 --2- 172.21.15.86:0/3385477360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d15adbd0 0x5576d15ba950 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f1150036040 tx=0x7f115002ca60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=654f4bbcdcaf0a26 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:35.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.618+0000 7f115affd640 1 --2- 172.21.15.86:0/1225716105 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f11440a3d20 0x7f11440a61e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1150009930 tx=0x7f1150039000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:35.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.618+0000 7f1141ffb640 1 -- 172.21.15.86:0/3385477360 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11500044c0 con 0x5576d15adbd0 2024-04-21T08:48:35.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.618+0000 7f1141ffb640 1 -- 172.21.15.86:0/3385477360 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1150050070 con 0x5576d15adbd0 2024-04-21T08:48:35.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.618+0000 7f1141ffb640 1 -- 172.21.15.86:0/3385477360 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f11500035f0 con 0x5576d15adbd0 2024-04-21T08:48:35.624 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.618+0000 7f1164980740 1 -- 172.21.15.86:0/3385477360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d15adbd0 msgr2=0x5576d15ba950 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:35.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.618+0000 7f1164980740 1 --2- 172.21.15.86:0/3385477360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d15adbd0 0x5576d15ba950 secure :-1 s=READY pgs=1765 cs=0 l=1 rev1=1 crypto rx=0x7f1150036040 tx=0x7f115002ca60 comp rx=0 tx=0).stop 2024-04-21T08:48:35.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.619+0000 7f1164980740 1 -- 172.21.15.86:0/3385477360 shutdown_connections 2024-04-21T08:48:35.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.619+0000 7f1164980740 1 --2- 172.21.15.86:0/3385477360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d15bae90 0x5576d15bd280 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.619+0000 7f1164980740 1 --2- 172.21.15.86:0/3385477360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d15adbd0 0x5576d15ba950 unknown :-1 s=CLOSED pgs=1765 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.619+0000 7f1164980740 1 --2- 172.21.15.86:0/3385477360 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d15ad2b0 0x5576d15ad690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.619+0000 7f1164980740 1 -- 172.21.15.86:0/3385477360 >> 172.21.15.86:0/3385477360 conn(0x5576d15a7cb0 msgr2=0x5576d15a8090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:35.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.619+0000 7f1164980740 1 -- 172.21.15.86:0/3385477360 shutdown_connections 2024-04-21T08:48:35.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.619+0000 7f1164980740 1 -- 172.21.15.86:0/3385477360 wait complete. 2024-04-21T08:48:35.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.619+0000 7f1164980740 1 -- start start 2024-04-21T08:48:35.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.619+0000 7f1164980740 1 Processor -- start 2024-04-21T08:48:35.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.619+0000 7f1164980740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d15ad2b0 0x5576d15b7f10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:35.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.619+0000 7f1164980740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d15adbd0 0x5576d15b8450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:35.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.619+0000 7f1164980740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d15bae90 0x5576d15b8990 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:35.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.619+0000 7f1164980740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5576d15bdbf0 con 0x5576d15bae90 2024-04-21T08:48:35.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.619+0000 7f1164980740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5576d14cb2f0 con 0x5576d15adbd0 2024-04-21T08:48:35.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.619+0000 7f1164980740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5576d15bdd70 con 0x5576d15ad2b0 2024-04-21T08:48:35.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.619+0000 7f1164980740 10 client.0 _ll_register_callbacks cb 0x5576d14d8580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:35.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.619+0000 7f115bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d15bae90 0x5576d15b8990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:35.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.620+0000 7f115bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d15bae90 0x5576d15b8990 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38278/0 (socket says 172.21.15.86:38278) 2024-04-21T08:48:35.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.620+0000 7f115bfff640 1 -- 172.21.15.86:0/2785512512 learned_addr learned my addr 172.21.15.86:0/2785512512 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:35.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.620+0000 7f115b7fe640 1 --2- 172.21.15.86:0/2785512512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d15ad2b0 0x5576d15b7f10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:35.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.620+0000 7f115affd640 1 --2- 172.21.15.86:0/2785512512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d15adbd0 0x5576d15b8450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:35.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.620+0000 7f115bfff640 1 -- 172.21.15.86:0/2785512512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d15ad2b0 msgr2=0x5576d15b7f10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:35.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.620+0000 7f115bfff640 1 --2- 172.21.15.86:0/2785512512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d15ad2b0 0x5576d15b7f10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.620+0000 7f115bfff640 1 -- 172.21.15.86:0/2785512512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d15adbd0 msgr2=0x5576d15b8450 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:35.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.620+0000 7f115bfff640 1 --2- 172.21.15.86:0/2785512512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d15adbd0 0x5576d15b8450 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.620+0000 7f115bfff640 1 -- 172.21.15.86:0/2785512512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576d12243f0 con 0x5576d15bae90 2024-04-21T08:48:35.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.620+0000 7f115affd640 1 --2- 172.21.15.86:0/2785512512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d15adbd0 0x5576d15b8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.620+0000 7f115bfff640 1 --2- 172.21.15.86:0/2785512512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d15bae90 0x5576d15b8990 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f115409c620 tx=0x7f115407a060 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb147b870f259db4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.620+0000 7f113affd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.620+0000 7f113affd640 1 -- 172.21.15.86:0/2785512512 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115401d070 con 0x5576d15bae90 2024-04-21T08:48:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.620+0000 7f113affd640 1 -- 172.21.15.86:0/2785512512 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f1154098f90 con 0x5576d15bae90 2024-04-21T08:48:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.620+0000 7f1164980740 1 -- 172.21.15.86:0/2785512512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1150050070 con 0x5576d15bae90 2024-04-21T08:48:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.620+0000 7f113affd640 1 -- 172.21.15.86:0/2785512512 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1154085e60 con 0x5576d15bae90 2024-04-21T08:48:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.620+0000 7f1164980740 10 client.19512 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.620+0000 7f1164980740 1 -- 172.21.15.86:0/2785512512 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5576d11e40f0 con 0x5576d15bae90 2024-04-21T08:48:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.620+0000 7f113affd640 1 -- 172.21.15.86:0/2785512512 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1150050070 con 0x5576d15bae90 2024-04-21T08:48:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.621+0000 7f113affd640 1 -- 172.21.15.86:0/2785512512 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f11540054f0 con 0x5576d15bae90 2024-04-21T08:48:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.621+0000 7f1164980740 10 client.19512 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.621+0000 7f1164980740 10 client.19512 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.621+0000 7f1164980740 20 client.19512 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.621+0000 7f1164980740 1 --2- 172.21.15.86:0/2785512512 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5576d15caed0 0x5576d15cd330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:35.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.621+0000 7f1164980740 4 client.19512 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:35.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.621+0000 7f1164980740 1 -- 172.21.15.86:0/2785512512 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5576d151fc00 con 0x5576d15caed0 2024-04-21T08:48:35.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.621+0000 7f115b7fe640 1 --2- 172.21.15.86:0/2785512512 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5576d15caed0 0x5576d15cd330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:35.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.621+0000 7f113affd640 10 client.19512 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:35.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.621+0000 7f115b7fe640 1 --2- 172.21.15.86:0/2785512512 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5576d15caed0 0x5576d15cd330 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ae8c06bf9bf0b216 server_cookie=75b6d71a96ee7e26 in_seq=0 out_seq=0 2024-04-21T08:48:35.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.622+0000 7f113affd640 1 -- 172.21.15.86:0/2785512512 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5576d151fc00 con 0x5576d15caed0 2024-04-21T08:48:35.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.622+0000 7f113affd640 10 client.19512 handle_command_reply: tid=0 2024-04-21T08:48:35.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.622+0000 7f1164980740 1 client.19512 shutdown 2024-04-21T08:48:35.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.622+0000 7f1164980740 1 -- 172.21.15.86:0/2785512512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d15bae90 msgr2=0x5576d15b8990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:35.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.622+0000 7f1164980740 1 --2- 172.21.15.86:0/2785512512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d15bae90 0x5576d15b8990 secure :-1 s=READY pgs=1766 cs=0 l=1 rev1=1 crypto rx=0x7f115409c620 tx=0x7f115407a060 comp rx=0 tx=0).stop 2024-04-21T08:48:35.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.622+0000 7f1164980740 1 -- 172.21.15.86:0/2785512512 shutdown_connections 2024-04-21T08:48:35.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.622+0000 7f1164980740 1 --2- 172.21.15.86:0/2785512512 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5576d15caed0 0x5576d15cd330 crc :-1 s=READY pgs=285 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.622+0000 7f1164980740 1 --2- 172.21.15.86:0/2785512512 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d15bae90 0x5576d15b8990 unknown :-1 s=CLOSED pgs=1766 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.622+0000 7f1164980740 1 --2- 172.21.15.86:0/2785512512 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d15adbd0 0x5576d15b8450 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.622+0000 7f113affd640 0 client.19512 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:35.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.622+0000 7f1164980740 1 --2- 172.21.15.86:0/2785512512 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d15ad2b0 0x5576d15b7f10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.622+0000 7f1164980740 1 -- 172.21.15.86:0/2785512512 >> 172.21.15.86:0/2785512512 conn(0x5576d15a7cb0 msgr2=0x5576d15a9a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:35.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.622+0000 7f1164980740 1 -- 172.21.15.86:0/2785512512 shutdown_connections 2024-04-21T08:48:35.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.622+0000 7f1164980740 1 -- 172.21.15.86:0/2785512512 wait complete. 2024-04-21T08:48:35.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.623+0000 7f1164980740 20 client.19512 trim_cache size 0 max 16384 2024-04-21T08:48:35.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.640+0000 7f1164980740 1 Processor -- start 2024-04-21T08:48:35.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.640+0000 7f1164980740 1 -- start start 2024-04-21T08:48:35.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.640+0000 7f1164980740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d15ad2b0 0x5576d15ba640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:35.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.640+0000 7f1164980740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d15adbd0 0x5576d15b6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:35.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.640+0000 7f1164980740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d15bae90 0x5576d15c6b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:35.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.640+0000 7f1164980740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f115c1ad0f0 con 0x5576d15bae90 2024-04-21T08:48:35.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.640+0000 7f1164980740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f115c1aec30 con 0x5576d15ad2b0 2024-04-21T08:48:35.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.640+0000 7f1164980740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f115c1ad7c0 con 0x5576d15adbd0 2024-04-21T08:48:35.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.640+0000 7f115bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d15bae90 0x5576d15c6b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:35.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.640+0000 7f115bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d15bae90 0x5576d15c6b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38292/0 (socket says 172.21.15.86:38292) 2024-04-21T08:48:35.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.640+0000 7f115bfff640 1 -- 172.21.15.86:0/3381050719 learned_addr learned my addr 172.21.15.86:0/3381050719 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:35.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.640+0000 7f115bfff640 1 -- 172.21.15.86:0/3381050719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d15adbd0 msgr2=0x5576d15b6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:35.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.640+0000 7f115bfff640 1 --2- 172.21.15.86:0/3381050719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d15adbd0 0x5576d15b6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.640+0000 7f115b7fe640 1 --2- 172.21.15.86:0/3381050719 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d15ad2b0 0x5576d15ba640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:35.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.640+0000 7f115bfff640 1 -- 172.21.15.86:0/3381050719 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d15ad2b0 msgr2=0x5576d15ba640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:35.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.640+0000 7f115bfff640 1 --2- 172.21.15.86:0/3381050719 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d15ad2b0 0x5576d15ba640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.640+0000 7f115bfff640 1 -- 172.21.15.86:0/3381050719 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5576d15ada00 con 0x5576d15bae90 2024-04-21T08:48:35.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.640+0000 7f115bfff640 1 --2- 172.21.15.86:0/3381050719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d15bae90 0x5576d15c6b20 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f1154098d80 tx=0x7f115409ae30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=937b17e39ed0b515 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:35.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.640+0000 7f11417fa640 1 -- 172.21.15.86:0/3381050719 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f115401d070 con 0x5576d15bae90 2024-04-21T08:48:35.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.641+0000 7f11417fa640 1 -- 172.21.15.86:0/3381050719 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f115409d020 con 0x5576d15bae90 2024-04-21T08:48:35.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.641+0000 7f11417fa640 1 -- 172.21.15.86:0/3381050719 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1154098f90 con 0x5576d15bae90 2024-04-21T08:48:35.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.641+0000 7f1164980740 1 -- 172.21.15.86:0/3381050719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d15bae90 msgr2=0x5576d15c6b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:35.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.641+0000 7f1164980740 1 --2- 172.21.15.86:0/3381050719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d15bae90 0x5576d15c6b20 secure :-1 s=READY pgs=1767 cs=0 l=1 rev1=1 crypto rx=0x7f1154098d80 tx=0x7f115409ae30 comp rx=0 tx=0).stop 2024-04-21T08:48:35.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.641+0000 7f1164980740 1 -- 172.21.15.86:0/3381050719 shutdown_connections 2024-04-21T08:48:35.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.641+0000 7f1164980740 1 --2- 172.21.15.86:0/3381050719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d15bae90 0x5576d15c6b20 unknown :-1 s=CLOSED pgs=1767 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.641+0000 7f1164980740 1 --2- 172.21.15.86:0/3381050719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d15adbd0 0x5576d15b6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.641+0000 7f1164980740 1 --2- 172.21.15.86:0/3381050719 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d15ad2b0 0x5576d15ba640 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.641+0000 7f1164980740 1 -- 172.21.15.86:0/3381050719 >> 172.21.15.86:0/3381050719 conn(0x5576d15a7cb0 msgr2=0x5576d15a9900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:35.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.641+0000 7f1164980740 1 -- 172.21.15.86:0/3381050719 shutdown_connections 2024-04-21T08:48:35.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.641+0000 7f1164980740 1 -- 172.21.15.86:0/3381050719 wait complete. 2024-04-21T08:48:35.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.641+0000 7f1164980740 1 -- start start 2024-04-21T08:48:35.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.642+0000 7f1164980740 1 Processor -- start 2024-04-21T08:48:35.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.642+0000 7f1164980740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d15ad2b0 0x5576d15b4f90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:35.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.642+0000 7f1164980740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d15adbd0 0x5576d15b54d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:35.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.642+0000 7f1164980740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d15bae90 0x5576d1630470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:35.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.642+0000 7f1164980740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f115c1ae560 con 0x5576d15adbd0 2024-04-21T08:48:35.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.642+0000 7f1164980740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f115c1ade90 con 0x5576d15ad2b0 2024-04-21T08:48:35.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.642+0000 7f1164980740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f115c1ab0f0 con 0x5576d15bae90 2024-04-21T08:48:35.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.642+0000 7f1164980740 10 client.0 _ll_register_callbacks cb 0x7f115409c280 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:35.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.642+0000 7f115affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d15adbd0 0x5576d15b54d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:35.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.642+0000 7f115affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d15adbd0 0x5576d15b54d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38302/0 (socket says 172.21.15.86:38302) 2024-04-21T08:48:35.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.642+0000 7f115affd640 1 -- 172.21.15.86:0/1761967150 learned_addr learned my addr 172.21.15.86:0/1761967150 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:35.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.642+0000 7f115bfff640 1 --2- 172.21.15.86:0/1761967150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d15bae90 0x5576d1630470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:35.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.642+0000 7f115b7fe640 1 --2- 172.21.15.86:0/1761967150 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d15ad2b0 0x5576d15b4f90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:35.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.642+0000 7f115affd640 1 -- 172.21.15.86:0/1761967150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d15bae90 msgr2=0x5576d1630470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:35.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.642+0000 7f115affd640 1 --2- 172.21.15.86:0/1761967150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d15bae90 0x5576d1630470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.642+0000 7f115affd640 1 -- 172.21.15.86:0/1761967150 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d15ad2b0 msgr2=0x5576d15b4f90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:35.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.642+0000 7f115affd640 1 --2- 172.21.15.86:0/1761967150 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d15ad2b0 0x5576d15b4f90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.642+0000 7f115affd640 1 -- 172.21.15.86:0/1761967150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5576d16309b0 con 0x5576d15adbd0 2024-04-21T08:48:35.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.642+0000 7f115b7fe640 1 --2- 172.21.15.86:0/1761967150 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d15ad2b0 0x5576d15b4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:35.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.642+0000 7f115affd640 1 --2- 172.21.15.86:0/1761967150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d15adbd0 0x5576d15b54d0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x5576d15ae490 tx=0x7f1150005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6d1c98a7ac638f15 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:35.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.642+0000 7f113affd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:35.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.643+0000 7f113affd640 1 -- 172.21.15.86:0/1761967150 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1150058070 con 0x5576d15adbd0 2024-04-21T08:48:35.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.643+0000 7f113affd640 1 -- 172.21.15.86:0/1761967150 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f1150004ad0 con 0x5576d15adbd0 2024-04-21T08:48:35.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.643+0000 7f1164980740 1 -- 172.21.15.86:0/1761967150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f115409d020 con 0x5576d15adbd0 2024-04-21T08:48:35.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.643+0000 7f113affd640 1 -- 172.21.15.86:0/1761967150 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f11500044c0 con 0x5576d15adbd0 2024-04-21T08:48:35.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.643+0000 7f1164980740 10 client.19518 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:35.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.643+0000 7f1164980740 1 -- 172.21.15.86:0/1761967150 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5576d1631b70 con 0x5576d15adbd0 2024-04-21T08:48:35.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.643+0000 7f113affd640 1 -- 172.21.15.86:0/1761967150 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f115409d020 con 0x5576d15adbd0 2024-04-21T08:48:35.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.643+0000 7f113affd640 1 -- 172.21.15.86:0/1761967150 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f115004a030 con 0x5576d15adbd0 2024-04-21T08:48:35.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.643+0000 7f1164980740 10 client.19518 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:35.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.643+0000 7f1164980740 10 client.19518 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:35.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.643+0000 7f1164980740 20 client.19518 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:35.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.643+0000 7f1164980740 1 --2- 172.21.15.86:0/1761967150 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5576d1631f80 0x5576d1634370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:35.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.643+0000 7f1164980740 4 client.19518 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:48:35.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.643+0000 7f1164980740 1 -- 172.21.15.86:0/1761967150 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5576d1519a70 con 0x5576d1631f80 2024-04-21T08:48:35.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.643+0000 7f115b7fe640 1 --2- 172.21.15.86:0/1761967150 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5576d1631f80 0x5576d1634370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:35.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.644+0000 7f113affd640 10 client.19518 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:35.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.644+0000 7f115b7fe640 1 --2- 172.21.15.86:0/1761967150 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5576d1631f80 0x5576d1634370 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=32e112af619fecb8 server_cookie=df30df41066282dd in_seq=0 out_seq=0 2024-04-21T08:48:35.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.644+0000 7f113affd640 1 -- 172.21.15.86:0/1761967150 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x5576d1519a70 con 0x5576d1631f80 2024-04-21T08:48:35.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.644+0000 7f113affd640 10 client.19518 handle_command_reply: tid=0 2024-04-21T08:48:35.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.644+0000 7f1164980740 1 client.19518 shutdown 2024-04-21T08:48:35.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.644+0000 7f1164980740 1 -- 172.21.15.86:0/1761967150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d15adbd0 msgr2=0x5576d15b54d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:35.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.644+0000 7f1164980740 1 --2- 172.21.15.86:0/1761967150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d15adbd0 0x5576d15b54d0 secure :-1 s=READY pgs=1768 cs=0 l=1 rev1=1 crypto rx=0x5576d15ae490 tx=0x7f1150005640 comp rx=0 tx=0).stop 2024-04-21T08:48:35.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.644+0000 7f1164980740 1 -- 172.21.15.86:0/1761967150 shutdown_connections 2024-04-21T08:48:35.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.644+0000 7f1164980740 1 --2- 172.21.15.86:0/1761967150 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5576d1631f80 0x5576d1634370 crc :-1 s=READY pgs=286 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.644+0000 7f1164980740 1 --2- 172.21.15.86:0/1761967150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5576d15bae90 0x5576d1630470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.644+0000 7f1164980740 1 --2- 172.21.15.86:0/1761967150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5576d15adbd0 0x5576d15b54d0 unknown :-1 s=CLOSED pgs=1768 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.644+0000 7f1164980740 1 --2- 172.21.15.86:0/1761967150 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5576d15ad2b0 0x5576d15b4f90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.644+0000 7f113affd640 0 client.19518 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:35.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.644+0000 7f1164980740 1 -- 172.21.15.86:0/1761967150 >> 172.21.15.86:0/1761967150 conn(0x5576d15a7cb0 msgr2=0x5576d15a64e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:35.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.644+0000 7f1164980740 1 -- 172.21.15.86:0/1761967150 shutdown_connections 2024-04-21T08:48:35.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.644+0000 7f1164980740 1 -- 172.21.15.86:0/1761967150 wait complete. 2024-04-21T08:48:35.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.644+0000 7f1164980740 20 client.19518 trim_cache size 0 max 16384 2024-04-21T08:48:35.651 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:48:35.651 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-21T08:48:35.651 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-21T08:48:35.651 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:48:35.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.645+0000 7f1161d8f640 1 -- 172.21.15.86:0/1225716105 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f11440a3d20 msgr2=0x7f11440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:35.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.645+0000 7f1161d8f640 1 --2- 172.21.15.86:0/1225716105 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f11440a3d20 0x7f11440a61e0 secure :-1 s=READY pgs=290 cs=0 l=1 rev1=1 crypto rx=0x7f1150009930 tx=0x7f1150039000 comp rx=0 tx=0).stop 2024-04-21T08:48:35.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.645+0000 7f1161d8f640 1 -- 172.21.15.86:0/1225716105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f115c107fd0 msgr2=0x7f115c1985e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:35.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.645+0000 7f1161d8f640 1 --2- 172.21.15.86:0/1225716105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f115c107fd0 0x7f115c1985e0 secure :-1 s=READY pgs=1764 cs=0 l=1 rev1=1 crypto rx=0x7f115400cad0 tx=0x7f115400cf90 comp rx=0 tx=0).stop 2024-04-21T08:48:35.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.645+0000 7f1161d8f640 1 -- 172.21.15.86:0/1225716105 shutdown_connections 2024-04-21T08:48:35.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.645+0000 7f1161d8f640 1 --2- 172.21.15.86:0/1225716105 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f11440a3d20 0x7f11440a61e0 unknown :-1 s=CLOSED pgs=290 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.645+0000 7f1161d8f640 1 --2- 172.21.15.86:0/1225716105 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f115c107fd0 0x7f115c1985e0 unknown :-1 s=CLOSED pgs=1764 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.645+0000 7f1161d8f640 1 --2- 172.21.15.86:0/1225716105 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f115c100d60 0x7f115c19e970 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.645+0000 7f1161d8f640 1 --2- 172.21.15.86:0/1225716105 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f115c1003e0 0x7f115c19e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.645+0000 7f1161d8f640 1 -- 172.21.15.86:0/1225716105 >> 172.21.15.86:0/1225716105 conn(0x7f115c077360 msgr2=0x7f115c0fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:35.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.645+0000 7f1161d8f640 1 -- 172.21.15.86:0/1225716105 shutdown_connections 2024-04-21T08:48:35.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.645+0000 7f1161d8f640 1 -- 172.21.15.86:0/1225716105 wait complete. 2024-04-21T08:48:35.655 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:48:35.655 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:48:35.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.753+0000 7f4b92b96640 1 -- 172.21.15.86:0/2875859978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b8c069a80 msgr2=0x7f4b8c10eda0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:35.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.753+0000 7f4b92b96640 1 --2- 172.21.15.86:0/2875859978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b8c069a80 0x7f4b8c10eda0 secure :-1 s=READY pgs=1769 cs=0 l=1 rev1=1 crypto rx=0x7f4b74009900 tx=0x7f4b7402bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:35.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.753+0000 7f4b92b96640 1 -- 172.21.15.86:0/2875859978 shutdown_connections 2024-04-21T08:48:35.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.753+0000 7f4b92b96640 1 --2- 172.21.15.86:0/2875859978 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b8c069a80 0x7f4b8c10eda0 unknown :-1 s=CLOSED pgs=1769 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.753+0000 7f4b92b96640 1 --2- 172.21.15.86:0/2875859978 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b8c0690e0 0x7f4b8c069540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.753+0000 7f4b92b96640 1 --2- 172.21.15.86:0/2875859978 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b8c107fe0 0x7f4b8c1083c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.753+0000 7f4b92b96640 1 -- 172.21.15.86:0/2875859978 >> 172.21.15.86:0/2875859978 conn(0x7f4b8c077e30 msgr2=0x7f4b8c078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:35.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.754+0000 7f4b92b96640 1 -- 172.21.15.86:0/2875859978 shutdown_connections 2024-04-21T08:48:35.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.754+0000 7f4b92b96640 1 -- 172.21.15.86:0/2875859978 wait complete. 2024-04-21T08:48:35.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.754+0000 7f4b92b96640 1 Processor -- start 2024-04-21T08:48:35.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.754+0000 7f4b92b96640 1 -- start start 2024-04-21T08:48:35.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.754+0000 7f4b92b96640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b8c0690e0 0x7f4b8c198690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:35.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.754+0000 7f4b92b96640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b8c069a80 0x7f4b8c198bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:35.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.754+0000 7f4b92b96640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b8c107fe0 0x7f4b8c19cf50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:35.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.754+0000 7f4b92b96640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8c112150 con 0x7f4b8c107fe0 2024-04-21T08:48:35.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.754+0000 7f4b92b96640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8c111fd0 con 0x7f4b8c0690e0 2024-04-21T08:48:35.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.754+0000 7f4b92b96640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4b8c1122d0 con 0x7f4b8c069a80 2024-04-21T08:48:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.754+0000 7f4b91133640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b8c107fe0 0x7f4b8c19cf50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.754+0000 7f4b91133640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b8c107fe0 0x7f4b8c19cf50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38320/0 (socket says 172.21.15.86:38320) 2024-04-21T08:48:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.754+0000 7f4b91133640 1 -- 172.21.15.86:0/2142491191 learned_addr learned my addr 172.21.15.86:0/2142491191 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.754+0000 7f4b90932640 1 --2- 172.21.15.86:0/2142491191 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b8c0690e0 0x7f4b8c198690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.755+0000 7f4b8bfff640 1 --2- 172.21.15.86:0/2142491191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b8c069a80 0x7f4b8c198bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.755+0000 7f4b91133640 1 -- 172.21.15.86:0/2142491191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b8c069a80 msgr2=0x7f4b8c198bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.755+0000 7f4b91133640 1 --2- 172.21.15.86:0/2142491191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b8c069a80 0x7f4b8c198bd0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.755+0000 7f4b91133640 1 -- 172.21.15.86:0/2142491191 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b8c0690e0 msgr2=0x7f4b8c198690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.755+0000 7f4b91133640 1 --2- 172.21.15.86:0/2142491191 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b8c0690e0 0x7f4b8c198690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.755+0000 7f4b91133640 1 -- 172.21.15.86:0/2142491191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4b8c19d680 con 0x7f4b8c107fe0 2024-04-21T08:48:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.755+0000 7f4b91133640 1 --2- 172.21.15.86:0/2142491191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b8c107fe0 0x7f4b8c19cf50 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f4b74009230 tx=0x7f4b74004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.755+0000 7f4b89ffb640 1 -- 172.21.15.86:0/2142491191 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b74043020 con 0x7f4b8c107fe0 2024-04-21T08:48:35.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.755+0000 7f4b92b96640 1 -- 172.21.15.86:0/2142491191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4b8c19d910 con 0x7f4b8c107fe0 2024-04-21T08:48:35.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.755+0000 7f4b89ffb640 1 -- 172.21.15.86:0/2142491191 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4b74004440 con 0x7f4b8c107fe0 2024-04-21T08:48:35.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.755+0000 7f4b89ffb640 1 -- 172.21.15.86:0/2142491191 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4b74003b30 con 0x7f4b8c107fe0 2024-04-21T08:48:35.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.755+0000 7f4b92b96640 1 -- 172.21.15.86:0/2142491191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4b8c1a5200 con 0x7f4b8c107fe0 2024-04-21T08:48:35.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.756+0000 7f4b92b96640 1 -- 172.21.15.86:0/2142491191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4b8c068de0 con 0x7f4b8c107fe0 2024-04-21T08:48:35.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.757+0000 7f4b89ffb640 1 -- 172.21.15.86:0/2142491191 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4b74046020 con 0x7f4b8c107fe0 2024-04-21T08:48:35.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.758+0000 7f4b89ffb640 1 --2- 172.21.15.86:0/2142491191 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4b780a3d20 0x7f4b780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:35.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.758+0000 7f4b90932640 1 --2- 172.21.15.86:0/2142491191 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4b780a3d20 0x7f4b780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:35.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.758+0000 7f4b89ffb640 1 -- 172.21.15.86:0/2142491191 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f4b740f2630 con 0x7f4b8c107fe0 2024-04-21T08:48:35.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.758+0000 7f4b90932640 1 --2- 172.21.15.86:0/2142491191 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4b780a3d20 0x7f4b780a61e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4b84005d00 tx=0x7f4b8400b410 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:35.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.763+0000 7f4b89ffb640 1 -- 172.21.15.86:0/2142491191 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f4b7404d070 con 0x7f4b8c107fe0 2024-04-21T08:48:35.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.978+0000 7f4b92b96640 1 -- 172.21.15.86:0/2142491191 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f4b8c10a450 con 0x7f4b8c107fe0 2024-04-21T08:48:35.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.979+0000 7f4b89ffb640 1 -- 172.21.15.86:0/2142491191 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7f4b740bea10 con 0x7f4b8c107fe0 2024-04-21T08:48:35.979 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:48:35.979 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:38:45.202126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24398,24394,24455,24472]},"id":1}]} 2024-04-21T08:48:35.979 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-21T08:48:35.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.981+0000 7f4b92b96640 1 -- 172.21.15.86:0/2142491191 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4b780a3d20 msgr2=0x7f4b780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:35.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.981+0000 7f4b92b96640 1 --2- 172.21.15.86:0/2142491191 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4b780a3d20 0x7f4b780a61e0 secure :-1 s=READY pgs=291 cs=0 l=1 rev1=1 crypto rx=0x7f4b84005d00 tx=0x7f4b8400b410 comp rx=0 tx=0).stop 2024-04-21T08:48:35.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.981+0000 7f4b92b96640 1 -- 172.21.15.86:0/2142491191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b8c107fe0 msgr2=0x7f4b8c19cf50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:35.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.981+0000 7f4b92b96640 1 --2- 172.21.15.86:0/2142491191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b8c107fe0 0x7f4b8c19cf50 secure :-1 s=READY pgs=1770 cs=0 l=1 rev1=1 crypto rx=0x7f4b74009230 tx=0x7f4b74004040 comp rx=0 tx=0).stop 2024-04-21T08:48:35.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.981+0000 7f4b92b96640 1 -- 172.21.15.86:0/2142491191 shutdown_connections 2024-04-21T08:48:35.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.981+0000 7f4b92b96640 1 --2- 172.21.15.86:0/2142491191 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4b780a3d20 0x7f4b780a61e0 unknown :-1 s=CLOSED pgs=291 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.981+0000 7f4b92b96640 1 --2- 172.21.15.86:0/2142491191 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4b8c107fe0 0x7f4b8c19cf50 unknown :-1 s=CLOSED pgs=1770 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.981+0000 7f4b92b96640 1 --2- 172.21.15.86:0/2142491191 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4b8c069a80 0x7f4b8c198bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.981+0000 7f4b92b96640 1 --2- 172.21.15.86:0/2142491191 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4b8c0690e0 0x7f4b8c198690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:35.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.981+0000 7f4b92b96640 1 -- 172.21.15.86:0/2142491191 >> 172.21.15.86:0/2142491191 conn(0x7f4b8c077e30 msgr2=0x7f4b8c0ffec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:35.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.981+0000 7f4b92b96640 1 -- 172.21.15.86:0/2142491191 shutdown_connections 2024-04-21T08:48:35.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:35.981+0000 7f4b92b96640 1 -- 172.21.15.86:0/2142491191 wait complete. 2024-04-21T08:48:35.991 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:48:36.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.091+0000 7f4db5759640 1 -- 172.21.15.86:0/1422989959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4db010c510 msgr2=0x7f4db010e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.091+0000 7f4db5759640 1 --2- 172.21.15.86:0/1422989959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4db010c510 0x7f4db010e9b0 secure :-1 s=READY pgs=1771 cs=0 l=1 rev1=1 crypto rx=0x7f4da8009900 tx=0x7f4da802bd60 comp rx=0 tx=0).stop 2024-04-21T08:48:36.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.092+0000 7f4db5759640 1 -- 172.21.15.86:0/1422989959 shutdown_connections 2024-04-21T08:48:36.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.092+0000 7f4db5759640 1 --2- 172.21.15.86:0/1422989959 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4db010c510 0x7f4db010e9b0 unknown :-1 s=CLOSED pgs=1771 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.093 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.092+0000 7f4db5759640 1 --2- 172.21.15.86:0/1422989959 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4db0103af0 0x7f4db010bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.092+0000 7f4db5759640 1 --2- 172.21.15.86:0/1422989959 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4db01031d0 0x7f4db01035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.092+0000 7f4db5759640 1 -- 172.21.15.86:0/1422989959 >> 172.21.15.86:0/1422989959 conn(0x7f4db0077f50 msgr2=0x7f4db0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:36.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.092+0000 7f4db5759640 1 -- 172.21.15.86:0/1422989959 shutdown_connections 2024-04-21T08:48:36.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.092+0000 7f4db5759640 1 -- 172.21.15.86:0/1422989959 wait complete. 2024-04-21T08:48:36.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.092+0000 7f4db5759640 1 Processor -- start 2024-04-21T08:48:36.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.092+0000 7f4db5759640 1 -- start start 2024-04-21T08:48:36.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.092+0000 7f4db5759640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4db01031d0 0x7f4db0198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.092+0000 7f4db5759640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4db0103af0 0x7f4db0198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.093+0000 7f4db5759640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4db010c510 0x7f4db019cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.093+0000 7f4db5759640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4db01121a0 con 0x7f4db0103af0 2024-04-21T08:48:36.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.093+0000 7f4db5759640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4db0112020 con 0x7f4db01031d0 2024-04-21T08:48:36.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.093+0000 7f4db5759640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4db0112320 con 0x7f4db010c510 2024-04-21T08:48:36.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.093+0000 7f4dae7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4db0103af0 0x7f4db0198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.093+0000 7f4dae7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4db0103af0 0x7f4db0198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38332/0 (socket says 172.21.15.86:38332) 2024-04-21T08:48:36.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.093+0000 7f4dae7fc640 1 -- 172.21.15.86:0/4164575267 learned_addr learned my addr 172.21.15.86:0/4164575267 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:36.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.093+0000 7f4daeffd640 1 --2- 172.21.15.86:0/4164575267 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4db01031d0 0x7f4db0198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.093+0000 7f4dae7fc640 1 -- 172.21.15.86:0/4164575267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4db010c510 msgr2=0x7f4db019cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.093+0000 7f4daf7fe640 1 --2- 172.21.15.86:0/4164575267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4db010c510 0x7f4db019cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.093+0000 7f4dae7fc640 1 --2- 172.21.15.86:0/4164575267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4db010c510 0x7f4db019cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.093+0000 7f4dae7fc640 1 -- 172.21.15.86:0/4164575267 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4db01031d0 msgr2=0x7f4db0198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.093+0000 7f4dae7fc640 1 --2- 172.21.15.86:0/4164575267 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4db01031d0 0x7f4db0198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.093+0000 7f4dae7fc640 1 -- 172.21.15.86:0/4164575267 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4db019d6f0 con 0x7f4db0103af0 2024-04-21T08:48:36.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.093+0000 7f4daeffd640 1 --2- 172.21.15.86:0/4164575267 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4db01031d0 0x7f4db0198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:36.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.093+0000 7f4dae7fc640 1 --2- 172.21.15.86:0/4164575267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4db0103af0 0x7f4db0198c40 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f4da400d950 tx=0x7f4da400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:36.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.093+0000 7f4d97fff640 1 -- 172.21.15.86:0/4164575267 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da4017070 con 0x7f4db0103af0 2024-04-21T08:48:36.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.093+0000 7f4db5759640 1 -- 172.21.15.86:0/4164575267 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f4db019d9e0 con 0x7f4db0103af0 2024-04-21T08:48:36.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.093+0000 7f4d97fff640 1 -- 172.21.15.86:0/4164575267 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4da4009c30 con 0x7f4db0103af0 2024-04-21T08:48:36.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.093+0000 7f4d97fff640 1 -- 172.21.15.86:0/4164575267 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da400f620 con 0x7f4db0103af0 2024-04-21T08:48:36.096 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.093+0000 7f4db5759640 1 -- 172.21.15.86:0/4164575267 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f4db01a5270 con 0x7f4db0103af0 2024-04-21T08:48:36.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.095+0000 7f4d97fff640 1 -- 172.21.15.86:0/4164575267 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4da4010460 con 0x7f4db0103af0 2024-04-21T08:48:36.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.096+0000 7f4db8420740 1 Processor -- start 2024-04-21T08:48:36.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.096+0000 7f4db8420740 1 -- start start 2024-04-21T08:48:36.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.096+0000 7f4db8420740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558e9743d4d0 0x558e9743d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.096+0000 7f4db8420740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e9743de80 0x558e97446430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.096+0000 7f4db8420740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e97446a00 0x558e97448e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.096+0000 7f4db8420740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558e973a21f0 con 0x558e97446a00 2024-04-21T08:48:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.096+0000 7f4db8420740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x558e9733f4b0 con 0x558e9743d4d0 2024-04-21T08:48:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.096+0000 7f4db8420740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558e9739e1b0 con 0x558e9743de80 2024-04-21T08:48:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.096+0000 7f4dae7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e97446a00 0x558e97448e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.096+0000 7f4dae7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e97446a00 0x558e97448e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38340/0 (socket says 172.21.15.86:38340) 2024-04-21T08:48:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.096+0000 7f4daeffd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e9743de80 0x558e97446430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.096+0000 7f4dae7fc640 1 -- 172.21.15.86:0/1551381319 learned_addr learned my addr 172.21.15.86:0/1551381319 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.096+0000 7f4d97fff640 1 --2- 172.21.15.86:0/4164575267 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4d980a3d20 0x7f4d980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.096+0000 7f4daf7fe640 1 --2- 172.21.15.86:0/1551381319 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558e9743d4d0 0x558e9743d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.096+0000 7f4dae7fc640 1 -- 172.21.15.86:0/1551381319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e9743de80 msgr2=0x558e97446430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.096+0000 7f4dae7fc640 1 --2- 172.21.15.86:0/1551381319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e9743de80 0x558e97446430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.096+0000 7f4dae7fc640 1 -- 172.21.15.86:0/1551381319 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558e9743d4d0 msgr2=0x558e9743d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.096+0000 7f4dae7fc640 1 --2- 172.21.15.86:0/1551381319 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558e9743d4d0 0x558e9743d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.096+0000 7f4dae7fc640 1 -- 172.21.15.86:0/1551381319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e973311b0 con 0x558e97446a00 2024-04-21T08:48:36.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.096+0000 7f4daf7fe640 1 --2- 172.21.15.86:0/4164575267 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4d980a3d20 0x7f4d980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.096+0000 7f4d97fff640 1 -- 172.21.15.86:0/4164575267 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f4da40d0830 con 0x7f4db0103af0 2024-04-21T08:48:36.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.097+0000 7f4dae7fc640 1 --2- 172.21.15.86:0/1551381319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e97446a00 0x558e97448e50 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f4da40cdb20 tx=0x7f4da4066a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bca6595d27d37c3a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:36.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.097+0000 7f4d957fa640 1 -- 172.21.15.86:0/1551381319 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da409bf40 con 0x558e97446a00 2024-04-21T08:48:36.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.097+0000 7f4d957fa640 1 -- 172.21.15.86:0/1551381319 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4da4066f10 con 0x558e97446a00 2024-04-21T08:48:36.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.097+0000 7f4d957fa640 1 -- 172.21.15.86:0/1551381319 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da409bd60 con 0x558e97446a00 2024-04-21T08:48:36.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.097+0000 7f4db8420740 1 -- 172.21.15.86:0/1551381319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e97446a00 msgr2=0x558e97448e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.097+0000 7f4daf7fe640 1 --2- 172.21.15.86:0/4164575267 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4d980a3d20 0x7f4d980a61e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4da8005940 tx=0x7f4da8009260 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:36.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.097+0000 7f4db8420740 1 --2- 172.21.15.86:0/1551381319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e97446a00 0x558e97448e50 secure :-1 s=READY pgs=1773 cs=0 l=1 rev1=1 crypto rx=0x7f4da40cdb20 tx=0x7f4da4066a00 comp rx=0 tx=0).stop 2024-04-21T08:48:36.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.097+0000 7f4db8420740 1 -- 172.21.15.86:0/1551381319 shutdown_connections 2024-04-21T08:48:36.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.097+0000 7f4db8420740 1 --2- 172.21.15.86:0/1551381319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e97446a00 0x558e97448e50 unknown :-1 s=CLOSED pgs=1773 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.097+0000 7f4db8420740 1 --2- 172.21.15.86:0/1551381319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e9743de80 0x558e97446430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.097+0000 7f4db8420740 1 --2- 172.21.15.86:0/1551381319 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558e9743d4d0 0x558e9743d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.097+0000 7f4db8420740 1 -- 172.21.15.86:0/1551381319 >> 172.21.15.86:0/1551381319 conn(0x558e97437ba0 msgr2=0x558e97437f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:36.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.097+0000 7f4db8420740 1 -- 172.21.15.86:0/1551381319 shutdown_connections 2024-04-21T08:48:36.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.097+0000 7f4db8420740 1 -- 172.21.15.86:0/1551381319 wait complete. 2024-04-21T08:48:36.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.098+0000 7f4db8420740 1 -- start start 2024-04-21T08:48:36.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.098+0000 7f4db8420740 1 Processor -- start 2024-04-21T08:48:36.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.098+0000 7f4db8420740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558e9743d4d0 0x558e97446300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.098+0000 7f4db8420740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e9743de80 0x558e97443690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.098+0000 7f4db8420740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e97446a00 0x558e97443c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.098+0000 7f4db8420740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558e97449910 con 0x558e97446a00 2024-04-21T08:48:36.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.098+0000 7f4db8420740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x558e9735b500 con 0x558e9743d4d0 2024-04-21T08:48:36.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.098+0000 7f4db8420740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558e97449a90 con 0x558e9743de80 2024-04-21T08:48:36.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.098+0000 7f4db8420740 10 client.0 _ll_register_callbacks cb 0x558e97329b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:36.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.098+0000 7f4daf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e97446a00 0x558e97443c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.098+0000 7f4daf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e97446a00 0x558e97443c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38354/0 (socket says 172.21.15.86:38354) 2024-04-21T08:48:36.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.098+0000 7f4daf7fe640 1 -- 172.21.15.86:0/3890974330 learned_addr learned my addr 172.21.15.86:0/3890974330 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:36.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.098+0000 7f4dae7fc640 1 --2- 172.21.15.86:0/3890974330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e9743de80 0x558e97443690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.098+0000 7f4daeffd640 1 --2- 172.21.15.86:0/3890974330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558e9743d4d0 0x558e97446300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.098+0000 7f4daf7fe640 1 -- 172.21.15.86:0/3890974330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e9743de80 msgr2=0x558e97443690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.098+0000 7f4daf7fe640 1 --2- 172.21.15.86:0/3890974330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e9743de80 0x558e97443690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.098+0000 7f4daf7fe640 1 -- 172.21.15.86:0/3890974330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558e9743d4d0 msgr2=0x558e97446300 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.098+0000 7f4daf7fe640 1 --2- 172.21.15.86:0/3890974330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558e9743d4d0 0x558e97446300 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.098+0000 7f4daf7fe640 1 -- 172.21.15.86:0/3890974330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e970b43f0 con 0x558e97446a00 2024-04-21T08:48:36.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.099+0000 7f4daf7fe640 1 --2- 172.21.15.86:0/3890974330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e97446a00 0x558e97443c10 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f4da8035000 tx=0x7f4da80058d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=854e31da8c1f8cf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.099+0000 7f4d92ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.099+0000 7f4d92ff5640 1 -- 172.21.15.86:0/3890974330 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da8005510 con 0x558e97446a00 2024-04-21T08:48:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.099+0000 7f4d92ff5640 1 -- 172.21.15.86:0/3890974330 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f4da80046c0 con 0x558e97446a00 2024-04-21T08:48:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.099+0000 7f4db8420740 1 -- 172.21.15.86:0/3890974330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4da4066f10 con 0x558e97446a00 2024-04-21T08:48:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.099+0000 7f4d92ff5640 1 -- 172.21.15.86:0/3890974330 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4da8058050 con 0x558e97446a00 2024-04-21T08:48:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.099+0000 7f4db8420740 10 client.19536 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.099+0000 7f4db8420740 1 -- 172.21.15.86:0/3890974330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558e9717e7d0 con 0x558e97446a00 2024-04-21T08:48:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.099+0000 7f4d92ff5640 1 -- 172.21.15.86:0/3890974330 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4da4066f10 con 0x558e97446a00 2024-04-21T08:48:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.099+0000 7f4d92ff5640 1 -- 172.21.15.86:0/3890974330 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4da8053030 con 0x558e97446a00 2024-04-21T08:48:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.099+0000 7f4db8420740 10 client.19536 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.099+0000 7f4db8420740 10 client.19536 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:48:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.099+0000 7f4db8420740 20 client.19536 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.099+0000 7f4db8420740 1 --2- 172.21.15.86:0/3890974330 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x558e97456ce0 0x558e97459160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.107 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.099+0000 7f4db8420740 4 client.19536 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:36.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.099+0000 7f4db8420740 1 -- 172.21.15.86:0/3890974330 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558e97341930 con 0x558e97456ce0 2024-04-21T08:48:36.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.100+0000 7f4daeffd640 1 --2- 172.21.15.86:0/3890974330 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x558e97456ce0 0x558e97459160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.100+0000 7f4daeffd640 1 --2- 172.21.15.86:0/3890974330 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x558e97456ce0 0x558e97459160 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=49be06630e33d3db server_cookie=74ed71b04dd1eeb4 in_seq=0 out_seq=0 2024-04-21T08:48:36.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.100+0000 7f4d92ff5640 10 client.19536 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:48:36.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.101+0000 7f4d92ff5640 1 -- 172.21.15.86:0/3890974330 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558e97341930 con 0x558e97456ce0 2024-04-21T08:48:36.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.101+0000 7f4d92ff5640 10 client.19536 handle_command_reply: tid=0 2024-04-21T08:48:36.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.101+0000 7f4db8420740 1 client.19536 shutdown 2024-04-21T08:48:36.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.101+0000 7f4db8420740 1 -- 172.21.15.86:0/3890974330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e97446a00 msgr2=0x558e97443c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.101+0000 7f4db8420740 1 --2- 172.21.15.86:0/3890974330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e97446a00 0x558e97443c10 secure :-1 s=READY pgs=1774 cs=0 l=1 rev1=1 crypto rx=0x7f4da8035000 tx=0x7f4da80058d0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.101+0000 7f4db8420740 1 -- 172.21.15.86:0/3890974330 shutdown_connections 2024-04-21T08:48:36.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.101+0000 7f4db8420740 1 --2- 172.21.15.86:0/3890974330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e97446a00 0x558e97443c10 unknown :-1 s=CLOSED pgs=1774 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.101+0000 7f4db8420740 1 --2- 172.21.15.86:0/3890974330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e9743de80 0x558e97443690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.101+0000 7f4db8420740 1 --2- 172.21.15.86:0/3890974330 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x558e97456ce0 0x558e97459160 crc :-1 s=READY pgs=68 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.108 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.101+0000 7f4db8420740 1 --2- 172.21.15.86:0/3890974330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558e9743d4d0 0x558e97446300 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.101+0000 7f4d92ff5640 0 client.19536 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:48:36.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.101+0000 7f4db8420740 1 -- 172.21.15.86:0/3890974330 >> 172.21.15.86:0/3890974330 conn(0x558e97437ba0 msgr2=0x558e974398f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:36.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.101+0000 7f4db8420740 1 -- 172.21.15.86:0/3890974330 shutdown_connections 2024-04-21T08:48:36.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.101+0000 7f4db8420740 1 -- 172.21.15.86:0/3890974330 wait complete. 2024-04-21T08:48:36.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.101+0000 7f4db8420740 20 client.19536 trim_cache size 0 max 16384 2024-04-21T08:48:36.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.114+0000 7f4db8420740 1 Processor -- start 2024-04-21T08:48:36.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.114+0000 7f4db8420740 1 -- start start 2024-04-21T08:48:36.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.114+0000 7f4db8420740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e9743d4d0 0x558e97452830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.114+0000 7f4db8420740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9743de80 0x558e97438fb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.114+0000 7f4db8420740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558e97446a00 0x558e97445dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.114+0000 7f4db8420740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4db01af2d0 con 0x558e9743de80 2024-04-21T08:48:36.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.114+0000 7f4db8420740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4db01b0e10 con 0x558e97446a00 2024-04-21T08:48:36.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.114+0000 7f4db8420740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4db01af9a0 con 0x558e9743d4d0 2024-04-21T08:48:36.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.115+0000 7f4dae7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9743de80 0x558e97438fb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.115+0000 7f4dae7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9743de80 0x558e97438fb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38370/0 (socket says 172.21.15.86:38370) 2024-04-21T08:48:36.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.115+0000 7f4dae7fc640 1 -- 172.21.15.86:0/488361333 learned_addr learned my addr 172.21.15.86:0/488361333 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:36.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.115+0000 7f4daf7fe640 1 --2- 172.21.15.86:0/488361333 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558e97446a00 0x558e97445dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.115+0000 7f4daeffd640 1 --2- 172.21.15.86:0/488361333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e9743d4d0 0x558e97452830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.115+0000 7f4dae7fc640 1 -- 172.21.15.86:0/488361333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e9743d4d0 msgr2=0x558e97452830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.115+0000 7f4dae7fc640 1 --2- 172.21.15.86:0/488361333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e9743d4d0 0x558e97452830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.115+0000 7f4dae7fc640 1 -- 172.21.15.86:0/488361333 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558e97446a00 msgr2=0x558e97445dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.115+0000 7f4dae7fc640 1 --2- 172.21.15.86:0/488361333 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558e97446a00 0x558e97445dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.115+0000 7f4dae7fc640 1 -- 172.21.15.86:0/488361333 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558e97441be0 con 0x558e9743de80 2024-04-21T08:48:36.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.115+0000 7f4daeffd640 1 --2- 172.21.15.86:0/488361333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e9743d4d0 0x558e97452830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:36.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.115+0000 7f4dae7fc640 1 --2- 172.21.15.86:0/488361333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9743de80 0x558e97438fb0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f4da4005fd0 tx=0x7f4da4010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a810f193c08708b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.115+0000 7f4d94ff9640 1 -- 172.21.15.86:0/488361333 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da4020070 con 0x558e9743de80 2024-04-21T08:48:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.115+0000 7f4d94ff9640 1 -- 172.21.15.86:0/488361333 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4da409cb00 con 0x558e9743de80 2024-04-21T08:48:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.115+0000 7f4d94ff9640 1 -- 172.21.15.86:0/488361333 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da40a07b0 con 0x558e9743de80 2024-04-21T08:48:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.116+0000 7f4db8420740 1 -- 172.21.15.86:0/488361333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9743de80 msgr2=0x558e97438fb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.116+0000 7f4db8420740 1 --2- 172.21.15.86:0/488361333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9743de80 0x558e97438fb0 secure :-1 s=READY pgs=1775 cs=0 l=1 rev1=1 crypto rx=0x7f4da4005fd0 tx=0x7f4da4010cb0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.116+0000 7f4db8420740 1 -- 172.21.15.86:0/488361333 shutdown_connections 2024-04-21T08:48:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.116+0000 7f4db8420740 1 --2- 172.21.15.86:0/488361333 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558e97446a00 0x558e97445dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.116+0000 7f4db8420740 1 --2- 172.21.15.86:0/488361333 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9743de80 0x558e97438fb0 unknown :-1 s=CLOSED pgs=1775 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.116+0000 7f4db8420740 1 --2- 172.21.15.86:0/488361333 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e9743d4d0 0x558e97452830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.116+0000 7f4db8420740 1 -- 172.21.15.86:0/488361333 >> 172.21.15.86:0/488361333 conn(0x558e97437ba0 msgr2=0x558e974426c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.116+0000 7f4db8420740 1 -- 172.21.15.86:0/488361333 shutdown_connections 2024-04-21T08:48:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.116+0000 7f4db8420740 1 -- 172.21.15.86:0/488361333 wait complete. 2024-04-21T08:48:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.116+0000 7f4db8420740 1 -- start start 2024-04-21T08:48:36.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.116+0000 7f4db8420740 1 Processor -- start 2024-04-21T08:48:36.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.116+0000 7f4db8420740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9743d4d0 0x558e9744efc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.116+0000 7f4db8420740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e9743de80 0x558e9744a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.116+0000 7f4db8420740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558e97446a00 0x558e9744a8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.116+0000 7f4db8420740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4db01b0740 con 0x558e9743d4d0 2024-04-21T08:48:36.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.116+0000 7f4db8420740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4db01b0070 con 0x558e97446a00 2024-04-21T08:48:36.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.116+0000 7f4db8420740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4db01ad2d0 con 0x558e9743de80 2024-04-21T08:48:36.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.116+0000 7f4db8420740 10 client.0 _ll_register_callbacks cb 0x7f4da8004450 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:36.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.116+0000 7f4daeffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9743d4d0 0x558e9744efc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.116+0000 7f4daeffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9743d4d0 0x558e9744efc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38380/0 (socket says 172.21.15.86:38380) 2024-04-21T08:48:36.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.116+0000 7f4daeffd640 1 -- 172.21.15.86:0/3368570802 learned_addr learned my addr 172.21.15.86:0/3368570802 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:36.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.117+0000 7f4dae7fc640 1 --2- 172.21.15.86:0/3368570802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e9743de80 0x558e9744a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.117+0000 7f4daf7fe640 1 --2- 172.21.15.86:0/3368570802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558e97446a00 0x558e9744a8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.117+0000 7f4daeffd640 1 -- 172.21.15.86:0/3368570802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e9743de80 msgr2=0x558e9744a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.117+0000 7f4daeffd640 1 --2- 172.21.15.86:0/3368570802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e9743de80 0x558e9744a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.117+0000 7f4daeffd640 1 -- 172.21.15.86:0/3368570802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558e97446a00 msgr2=0x558e9744a8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.117+0000 7f4daeffd640 1 --2- 172.21.15.86:0/3368570802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558e97446a00 0x558e9744a8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.117+0000 7f4daeffd640 1 -- 172.21.15.86:0/3368570802 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558e9744af60 con 0x558e9743d4d0 2024-04-21T08:48:36.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.117+0000 7f4daf7fe640 1 --2- 172.21.15.86:0/3368570802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558e97446a00 0x558e9744a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:36.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.117+0000 7f4daeffd640 1 --2- 172.21.15.86:0/3368570802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9743d4d0 0x558e9744efc0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x558e9743cfa0 tx=0x7f4da0013b30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b12c8535ff0be689 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:36.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.117+0000 7f4d92ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:36.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.117+0000 7f4d92ff5640 1 -- 172.21.15.86:0/3368570802 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4da0020850 con 0x558e9743d4d0 2024-04-21T08:48:36.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.117+0000 7f4d92ff5640 1 -- 172.21.15.86:0/3368570802 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f4da0004b20 con 0x558e9743d4d0 2024-04-21T08:48:36.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.117+0000 7f4db8420740 1 -- 172.21.15.86:0/3368570802 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4da409cb00 con 0x558e9743d4d0 2024-04-21T08:48:36.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.117+0000 7f4d92ff5640 1 -- 172.21.15.86:0/3368570802 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4da002f050 con 0x558e9743d4d0 2024-04-21T08:48:36.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.117+0000 7f4db8420740 10 client.19542 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:36.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.117+0000 7f4db8420740 1 -- 172.21.15.86:0/3368570802 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558e974bd120 con 0x558e9743d4d0 2024-04-21T08:48:36.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.117+0000 7f4d92ff5640 1 -- 172.21.15.86:0/3368570802 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4da409cb00 con 0x558e9743d4d0 2024-04-21T08:48:36.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.117+0000 7f4d92ff5640 1 -- 172.21.15.86:0/3368570802 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f4da000f040 con 0x558e9743d4d0 2024-04-21T08:48:36.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.117+0000 7f4db8420740 10 client.19542 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:36.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.117+0000 7f4db8420740 10 client.19542 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:48:36.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.117+0000 7f4db8420740 20 client.19542 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:36.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.117+0000 7f4db8420740 1 --2- 172.21.15.86:0/3368570802 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x558e974bd8a0 0x558e974bfcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.117+0000 7f4db8420740 4 client.19542 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:48:36.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.117+0000 7f4db8420740 1 -- 172.21.15.86:0/3368570802 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558e973afc10 con 0x558e974bd8a0 2024-04-21T08:48:36.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.118+0000 7f4dae7fc640 1 --2- 172.21.15.86:0/3368570802 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x558e974bd8a0 0x558e974bfcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.118+0000 7f4d92ff5640 10 client.19542 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:48:36.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.118+0000 7f4dae7fc640 1 --2- 172.21.15.86:0/3368570802 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x558e974bd8a0 0x558e974bfcc0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5af8f3c830b7755f server_cookie=e4f88358db0ffe15 in_seq=0 out_seq=0 2024-04-21T08:48:36.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.118+0000 7f4d92ff5640 1 -- 172.21.15.86:0/3368570802 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558e973afc10 con 0x558e974bd8a0 2024-04-21T08:48:36.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.118+0000 7f4d92ff5640 10 client.19542 handle_command_reply: tid=0 2024-04-21T08:48:36.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.118+0000 7f4db8420740 1 client.19542 shutdown 2024-04-21T08:48:36.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.118+0000 7f4db8420740 1 -- 172.21.15.86:0/3368570802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9743d4d0 msgr2=0x558e9744efc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.118+0000 7f4db8420740 1 --2- 172.21.15.86:0/3368570802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9743d4d0 0x558e9744efc0 secure :-1 s=READY pgs=1776 cs=0 l=1 rev1=1 crypto rx=0x558e9743cfa0 tx=0x7f4da0013b30 comp rx=0 tx=0).stop 2024-04-21T08:48:36.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.118+0000 7f4db8420740 1 -- 172.21.15.86:0/3368570802 shutdown_connections 2024-04-21T08:48:36.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.118+0000 7f4db8420740 1 --2- 172.21.15.86:0/3368570802 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x558e974bd8a0 0x558e974bfcc0 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.118+0000 7f4db8420740 1 --2- 172.21.15.86:0/3368570802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558e97446a00 0x558e9744a8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.118+0000 7f4db8420740 1 --2- 172.21.15.86:0/3368570802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558e9743de80 0x558e9744a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.118+0000 7f4db8420740 1 --2- 172.21.15.86:0/3368570802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558e9743d4d0 0x558e9744efc0 unknown :-1 s=CLOSED pgs=1776 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.118+0000 7f4d92ff5640 0 client.19542 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:48:36.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.118+0000 7f4db8420740 1 -- 172.21.15.86:0/3368570802 >> 172.21.15.86:0/3368570802 conn(0x558e97437ba0 msgr2=0x558e9743e2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:36.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.118+0000 7f4db8420740 1 -- 172.21.15.86:0/3368570802 shutdown_connections 2024-04-21T08:48:36.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.118+0000 7f4db8420740 1 -- 172.21.15.86:0/3368570802 wait complete. 2024-04-21T08:48:36.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.119+0000 7f4db8420740 20 client.19542 trim_cache size 0 max 16384 2024-04-21T08:48:36.126 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:48:36.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.119+0000 7f4db5759640 1 -- 172.21.15.86:0/4164575267 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4d980a3d20 msgr2=0x7f4d980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.119+0000 7f4db5759640 1 --2- 172.21.15.86:0/4164575267 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4d980a3d20 0x7f4d980a61e0 secure :-1 s=READY pgs=292 cs=0 l=1 rev1=1 crypto rx=0x7f4da8005940 tx=0x7f4da8009260 comp rx=0 tx=0).stop 2024-04-21T08:48:36.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.119+0000 7f4db5759640 1 -- 172.21.15.86:0/4164575267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4db0103af0 msgr2=0x7f4db0198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.119+0000 7f4db5759640 1 --2- 172.21.15.86:0/4164575267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4db0103af0 0x7f4db0198c40 secure :-1 s=READY pgs=1772 cs=0 l=1 rev1=1 crypto rx=0x7f4da400d950 tx=0x7f4da400de10 comp rx=0 tx=0).stop 2024-04-21T08:48:36.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.119+0000 7f4db5759640 1 -- 172.21.15.86:0/4164575267 shutdown_connections 2024-04-21T08:48:36.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.119+0000 7f4db5759640 1 --2- 172.21.15.86:0/4164575267 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f4d980a3d20 0x7f4d980a61e0 unknown :-1 s=CLOSED pgs=292 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.119+0000 7f4db5759640 1 --2- 172.21.15.86:0/4164575267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4db010c510 0x7f4db019cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.119+0000 7f4db5759640 1 --2- 172.21.15.86:0/4164575267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4db0103af0 0x7f4db0198c40 unknown :-1 s=CLOSED pgs=1772 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.119+0000 7f4db5759640 1 --2- 172.21.15.86:0/4164575267 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4db01031d0 0x7f4db0198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.119+0000 7f4db5759640 1 -- 172.21.15.86:0/4164575267 >> 172.21.15.86:0/4164575267 conn(0x7f4db0077f50 msgr2=0x7f4db00fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:36.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.119+0000 7f4db5759640 1 -- 172.21.15.86:0/4164575267 shutdown_connections 2024-04-21T08:48:36.127 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.119+0000 7f4db5759640 1 -- 172.21.15.86:0/4164575267 wait complete. 2024-04-21T08:48:36.129 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:48:36.148 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:35 smithi086 ceph-mon[27939]: pgmap v305: 129 pgs: 129 active+clean; 2.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 KiB/s wr, 11 op/s 2024-04-21T08:48:36.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:35 smithi088 ceph-mon[31259]: pgmap v305: 129 pgs: 129 active+clean; 2.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 KiB/s wr, 11 op/s 2024-04-21T08:48:36.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.229+0000 7f605595f640 1 -- 172.21.15.86:0/2592537861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f605006b230 msgr2=0x7f6050106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.229+0000 7f605595f640 1 --2- 172.21.15.86:0/2592537861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f605006b230 0x7f6050106b90 secure :-1 s=READY pgs=1777 cs=0 l=1 rev1=1 crypto rx=0x7f6040009900 tx=0x7f604002bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:36.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.229+0000 7f605595f640 1 -- 172.21.15.86:0/2592537861 shutdown_connections 2024-04-21T08:48:36.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.229+0000 7f605595f640 1 --2- 172.21.15.86:0/2592537861 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6050107270 0x7f605010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.229+0000 7f605595f640 1 --2- 172.21.15.86:0/2592537861 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f605006b230 0x7f6050106b90 unknown :-1 s=CLOSED pgs=1777 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.229+0000 7f605595f640 1 --2- 172.21.15.86:0/2592537861 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f605006a880 0x7f605006ac60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.229+0000 7f605595f640 1 -- 172.21.15.86:0/2592537861 >> 172.21.15.86:0/2592537861 conn(0x7f6050077eb0 msgr2=0x7f60500782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:36.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.229+0000 7f605595f640 1 -- 172.21.15.86:0/2592537861 shutdown_connections 2024-04-21T08:48:36.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.229+0000 7f605595f640 1 -- 172.21.15.86:0/2592537861 wait complete. 2024-04-21T08:48:36.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.229+0000 7f605595f640 1 Processor -- start 2024-04-21T08:48:36.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.230+0000 7f605595f640 1 -- start start 2024-04-21T08:48:36.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.230+0000 7f605595f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f605006a880 0x7f605019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.230+0000 7f605595f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6050107270 0x7f605019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.230+0000 7f605595f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6050198590 0x7f6050198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.230+0000 7f605595f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60501121b0 con 0x7f6050198590 2024-04-21T08:48:36.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.230+0000 7f605595f640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6050112030 con 0x7f6050107270 2024-04-21T08:48:36.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.230+0000 7f605595f640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6050112330 con 0x7f605006a880 2024-04-21T08:48:36.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.230+0000 7f604f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6050198590 0x7f6050198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.230+0000 7f604f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6050198590 0x7f6050198a40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38398/0 (socket says 172.21.15.86:38398) 2024-04-21T08:48:36.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.230+0000 7f604f7fe640 1 -- 172.21.15.86:0/1067108896 learned_addr learned my addr 172.21.15.86:0/1067108896 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:36.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.230+0000 7f604e7fc640 1 --2- 172.21.15.86:0/1067108896 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6050107270 0x7f605019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.230+0000 7f604effd640 1 --2- 172.21.15.86:0/1067108896 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f605006a880 0x7f605019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.230+0000 7f604f7fe640 1 -- 172.21.15.86:0/1067108896 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f605006a880 msgr2=0x7f605019e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.230+0000 7f604f7fe640 1 --2- 172.21.15.86:0/1067108896 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f605006a880 0x7f605019e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.230+0000 7f604f7fe640 1 -- 172.21.15.86:0/1067108896 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6050107270 msgr2=0x7f605019e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.230+0000 7f604f7fe640 1 --2- 172.21.15.86:0/1067108896 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6050107270 0x7f605019e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.230+0000 7f604f7fe640 1 -- 172.21.15.86:0/1067108896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f60501990b0 con 0x7f6050198590 2024-04-21T08:48:36.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.230+0000 7f604effd640 1 --2- 172.21.15.86:0/1067108896 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f605006a880 0x7f605019e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:36.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.230+0000 7f604f7fe640 1 --2- 172.21.15.86:0/1067108896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6050198590 0x7f6050198a40 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f604800eaa0 tx=0x7f604800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:36.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.230+0000 7f605495d640 1 -- 172.21.15.86:0/1067108896 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604800cdb0 con 0x7f6050198590 2024-04-21T08:48:36.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.230+0000 7f605495d640 1 -- 172.21.15.86:0/1067108896 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6048005020 con 0x7f6050198590 2024-04-21T08:48:36.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.230+0000 7f605595f640 1 -- 172.21.15.86:0/1067108896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f60501993a0 con 0x7f6050198590 2024-04-21T08:48:36.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.230+0000 7f605495d640 1 -- 172.21.15.86:0/1067108896 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60480116d0 con 0x7f6050198590 2024-04-21T08:48:36.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.230+0000 7f605595f640 1 -- 172.21.15.86:0/1067108896 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60501a55f0 con 0x7f6050198590 2024-04-21T08:48:36.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.233+0000 7f605495d640 1 -- 172.21.15.86:0/1067108896 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f60480051d0 con 0x7f6050198590 2024-04-21T08:48:36.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.233+0000 7f605862a740 1 Processor -- start 2024-04-21T08:48:36.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.233+0000 7f605862a740 1 -- start start 2024-04-21T08:48:36.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.233+0000 7f605862a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c796d4d0 0x55d9c796d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.233+0000 7f605862a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d9c796de80 0x55d9c7976430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.233+0000 7f605862a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9c7976a00 0x55d9c7978e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.233+0000 7f605862a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d9c78d21f0 con 0x55d9c796d4d0 2024-04-21T08:48:36.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.233+0000 7f605862a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d9c786f4b0 con 0x55d9c796de80 2024-04-21T08:48:36.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.233+0000 7f605862a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d9c78ce1b0 con 0x55d9c7976a00 2024-04-21T08:48:36.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.233+0000 7f605495d640 1 --2- 172.21.15.86:0/1067108896 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f603c0a3d20 0x7f603c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.233+0000 7f604e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c796d4d0 0x55d9c796d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.233+0000 7f604e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c796d4d0 0x55d9c796d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38402/0 (socket says 172.21.15.86:38402) 2024-04-21T08:48:36.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.233+0000 7f604e7fc640 1 -- 172.21.15.86:0/2675875202 learned_addr learned my addr 172.21.15.86:0/2675875202 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:36.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.233+0000 7f604effd640 1 --2- 172.21.15.86:0/2675875202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d9c796de80 0x55d9c7976430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.233+0000 7f604f7fe640 1 --2- 172.21.15.86:0/1067108896 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f603c0a3d20 0x7f603c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.234+0000 7f605495d640 1 -- 172.21.15.86:0/1067108896 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f60480d0dc0 con 0x7f6050198590 2024-04-21T08:48:36.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.234+0000 7f604e7fc640 1 -- 172.21.15.86:0/2675875202 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9c7976a00 msgr2=0x55d9c7978e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.234+0000 7f604e7fc640 1 --2- 172.21.15.86:0/2675875202 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9c7976a00 0x55d9c7978e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.234+0000 7f604e7fc640 1 -- 172.21.15.86:0/2675875202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d9c796de80 msgr2=0x55d9c7976430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.234+0000 7f604e7fc640 1 --2- 172.21.15.86:0/2675875202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d9c796de80 0x55d9c7976430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.234+0000 7f604e7fc640 1 -- 172.21.15.86:0/2675875202 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9c78611b0 con 0x55d9c796d4d0 2024-04-21T08:48:36.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.234+0000 7f604effd640 1 --2- 172.21.15.86:0/2675875202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d9c796de80 0x55d9c7976430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:36.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.234+0000 7f604f7fe640 1 --2- 172.21.15.86:0/1067108896 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f603c0a3d20 0x7f603c0a61e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f60480ce1f0 tx=0x7f60480bd040 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:36.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.234+0000 7f604e7fc640 1 --2- 172.21.15.86:0/2675875202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c796d4d0 0x55d9c796d8b0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f6040000c00 tx=0x7f604002c5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b719cc2420ee3769 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:36.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.234+0000 7f6045ffb640 1 -- 172.21.15.86:0/2675875202 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6040043070 con 0x55d9c796d4d0 2024-04-21T08:48:36.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.234+0000 7f6045ffb640 1 -- 172.21.15.86:0/2675875202 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f604002cc90 con 0x55d9c796d4d0 2024-04-21T08:48:36.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.234+0000 7f6045ffb640 1 -- 172.21.15.86:0/2675875202 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60400046c0 con 0x55d9c796d4d0 2024-04-21T08:48:36.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.235+0000 7f605862a740 1 -- 172.21.15.86:0/2675875202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c796d4d0 msgr2=0x55d9c796d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.235+0000 7f605862a740 1 --2- 172.21.15.86:0/2675875202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c796d4d0 0x55d9c796d8b0 secure :-1 s=READY pgs=1779 cs=0 l=1 rev1=1 crypto rx=0x7f6040000c00 tx=0x7f604002c5d0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.235+0000 7f605862a740 1 -- 172.21.15.86:0/2675875202 shutdown_connections 2024-04-21T08:48:36.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.235+0000 7f605862a740 1 --2- 172.21.15.86:0/2675875202 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9c7976a00 0x55d9c7978e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.235+0000 7f605862a740 1 --2- 172.21.15.86:0/2675875202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d9c796de80 0x55d9c7976430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.235+0000 7f605862a740 1 --2- 172.21.15.86:0/2675875202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c796d4d0 0x55d9c796d8b0 unknown :-1 s=CLOSED pgs=1779 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.235+0000 7f605862a740 1 -- 172.21.15.86:0/2675875202 >> 172.21.15.86:0/2675875202 conn(0x55d9c7967ba0 msgr2=0x55d9c7967f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:36.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.235+0000 7f605862a740 1 -- 172.21.15.86:0/2675875202 shutdown_connections 2024-04-21T08:48:36.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.235+0000 7f605862a740 1 -- 172.21.15.86:0/2675875202 wait complete. 2024-04-21T08:48:36.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.235+0000 7f605862a740 1 -- start start 2024-04-21T08:48:36.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.235+0000 7f605862a740 1 Processor -- start 2024-04-21T08:48:36.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.236+0000 7f605862a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c796d4d0 0x55d9c7976330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.236+0000 7f605862a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d9c796de80 0x55d9c79736c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.236+0000 7f605862a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9c7976a00 0x55d9c7973cd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.236+0000 7f605862a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d9c7979910 con 0x55d9c796d4d0 2024-04-21T08:48:36.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.236+0000 7f605862a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d9c788b500 con 0x55d9c796de80 2024-04-21T08:48:36.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.236+0000 7f605862a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d9c7979a90 con 0x55d9c7976a00 2024-04-21T08:48:36.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.236+0000 7f605862a740 10 client.0 _ll_register_callbacks cb 0x55d9c7859b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:36.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.236+0000 7f604e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c796d4d0 0x55d9c7976330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.236+0000 7f604e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c796d4d0 0x55d9c7976330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38404/0 (socket says 172.21.15.86:38404) 2024-04-21T08:48:36.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.236+0000 7f604e7fc640 1 -- 172.21.15.86:0/1823752812 learned_addr learned my addr 172.21.15.86:0/1823752812 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:36.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.236+0000 7f604effd640 1 --2- 172.21.15.86:0/1823752812 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d9c796de80 0x55d9c79736c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.236+0000 7f604e7fc640 1 -- 172.21.15.86:0/1823752812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9c7976a00 msgr2=0x55d9c7973cd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.236+0000 7f604e7fc640 1 --2- 172.21.15.86:0/1823752812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9c7976a00 0x55d9c7973cd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.236+0000 7f604e7fc640 1 -- 172.21.15.86:0/1823752812 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d9c796de80 msgr2=0x55d9c79736c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.236+0000 7f604e7fc640 1 --2- 172.21.15.86:0/1823752812 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d9c796de80 0x55d9c79736c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.236+0000 7f604e7fc640 1 -- 172.21.15.86:0/1823752812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9c75e43f0 con 0x55d9c796d4d0 2024-04-21T08:48:36.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.236+0000 7f604e7fc640 1 --2- 172.21.15.86:0/1823752812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c796d4d0 0x55d9c7976330 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f6040036040 tx=0x7f604002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86b7a3066983c91c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:36.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.236+0000 7f602effd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:36.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.236+0000 7f602effd640 1 -- 172.21.15.86:0/1823752812 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604004f030 con 0x55d9c796d4d0 2024-04-21T08:48:36.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.236+0000 7f602effd640 1 -- 172.21.15.86:0/1823752812 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f60400035f0 con 0x55d9c796d4d0 2024-04-21T08:48:36.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.236+0000 7f605862a740 1 -- 172.21.15.86:0/1823752812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f604002cc90 con 0x55d9c796d4d0 2024-04-21T08:48:36.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.236+0000 7f602effd640 1 -- 172.21.15.86:0/1823752812 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f6040043040 con 0x55d9c796d4d0 2024-04-21T08:48:36.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.236+0000 7f605862a740 10 client.19554 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:36.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.236+0000 7f605862a740 1 -- 172.21.15.86:0/1823752812 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d9c76ae7d0 con 0x55d9c796d4d0 2024-04-21T08:48:36.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.237+0000 7f602effd640 1 -- 172.21.15.86:0/1823752812 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f604002cc90 con 0x55d9c796d4d0 2024-04-21T08:48:36.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.237+0000 7f602effd640 1 -- 172.21.15.86:0/1823752812 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6040048780 con 0x55d9c796d4d0 2024-04-21T08:48:36.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.237+0000 7f605862a740 10 client.19554 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:36.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.237+0000 7f605862a740 10 client.19554 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:48:36.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.237+0000 7f605862a740 20 client.19554 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:36.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.237+0000 7f605862a740 1 --2- 172.21.15.86:0/1823752812 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55d9c7986a90 0x55d9c7988e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.237+0000 7f605862a740 4 client.19554 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:36.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.237+0000 7f605862a740 1 -- 172.21.15.86:0/1823752812 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d9c7871930 con 0x55d9c7986a90 2024-04-21T08:48:36.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.237+0000 7f604effd640 1 --2- 172.21.15.86:0/1823752812 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55d9c7986a90 0x55d9c7988e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.237+0000 7f604effd640 1 --2- 172.21.15.86:0/1823752812 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55d9c7986a90 0x55d9c7988e80 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=df724e4c151f3d4e server_cookie=7efa3fefc31015de in_seq=0 out_seq=0 2024-04-21T08:48:36.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.237+0000 7f602effd640 10 client.19554 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:48:36.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.238+0000 7f602effd640 1 -- 172.21.15.86:0/1823752812 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d9c7871930 con 0x55d9c7986a90 2024-04-21T08:48:36.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.238+0000 7f602effd640 10 client.19554 handle_command_reply: tid=0 2024-04-21T08:48:36.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.238+0000 7f605862a740 1 client.19554 shutdown 2024-04-21T08:48:36.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.238+0000 7f605862a740 1 -- 172.21.15.86:0/1823752812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c796d4d0 msgr2=0x55d9c7976330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.238+0000 7f605862a740 1 --2- 172.21.15.86:0/1823752812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c796d4d0 0x55d9c7976330 secure :-1 s=READY pgs=1780 cs=0 l=1 rev1=1 crypto rx=0x7f6040036040 tx=0x7f604002bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:36.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.238+0000 7f605862a740 1 -- 172.21.15.86:0/1823752812 shutdown_connections 2024-04-21T08:48:36.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.238+0000 7f605862a740 1 --2- 172.21.15.86:0/1823752812 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55d9c7986a90 0x55d9c7988e80 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.238+0000 7f605862a740 1 --2- 172.21.15.86:0/1823752812 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9c7976a00 0x55d9c7973cd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.238+0000 7f605862a740 1 --2- 172.21.15.86:0/1823752812 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d9c796de80 0x55d9c79736c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.238+0000 7f602effd640 0 client.19554 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:48:36.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.238+0000 7f605862a740 1 --2- 172.21.15.86:0/1823752812 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c796d4d0 0x55d9c7976330 unknown :-1 s=CLOSED pgs=1780 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.238+0000 7f605862a740 1 -- 172.21.15.86:0/1823752812 >> 172.21.15.86:0/1823752812 conn(0x55d9c7967ba0 msgr2=0x55d9c79698f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:36.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.238+0000 7f605862a740 1 -- 172.21.15.86:0/1823752812 shutdown_connections 2024-04-21T08:48:36.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.238+0000 7f605862a740 1 -- 172.21.15.86:0/1823752812 wait complete. 2024-04-21T08:48:36.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.238+0000 7f605862a740 20 client.19554 trim_cache size 0 max 16384 2024-04-21T08:48:36.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:35 smithi179 ceph-mon[31829]: pgmap v305: 129 pgs: 129 active+clean; 2.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 KiB/s wr, 11 op/s 2024-04-21T08:48:36.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.251+0000 7f605862a740 1 Processor -- start 2024-04-21T08:48:36.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.252+0000 7f605862a740 1 -- start start 2024-04-21T08:48:36.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.252+0000 7f605862a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c796d4d0 0x55d9c7966e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.252+0000 7f605862a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d9c796de80 0x55d9c798c290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.252+0000 7f605862a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9c7976a00 0x55d9c7975df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.252+0000 7f605862a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60501af400 con 0x55d9c796d4d0 2024-04-21T08:48:36.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.252+0000 7f605862a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f60501b0f40 con 0x55d9c796de80 2024-04-21T08:48:36.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.252+0000 7f605862a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f60501afad0 con 0x55d9c7976a00 2024-04-21T08:48:36.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.252+0000 7f604e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c796d4d0 0x55d9c7966e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.252+0000 7f604e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c796d4d0 0x55d9c7966e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38412/0 (socket says 172.21.15.86:38412) 2024-04-21T08:48:36.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.252+0000 7f604e7fc640 1 -- 172.21.15.86:0/1152948228 learned_addr learned my addr 172.21.15.86:0/1152948228 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:36.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.252+0000 7f604e7fc640 1 -- 172.21.15.86:0/1152948228 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9c7976a00 msgr2=0x55d9c7975df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.252+0000 7f604e7fc640 1 --2- 172.21.15.86:0/1152948228 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9c7976a00 0x55d9c7975df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.252+0000 7f604e7fc640 1 -- 172.21.15.86:0/1152948228 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d9c796de80 msgr2=0x55d9c798c290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.252+0000 7f604e7fc640 1 --2- 172.21.15.86:0/1152948228 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d9c796de80 0x55d9c798c290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.252+0000 7f604e7fc640 1 -- 172.21.15.86:0/1152948228 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d9c796dc20 con 0x55d9c796d4d0 2024-04-21T08:48:36.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.252+0000 7f604e7fc640 1 --2- 172.21.15.86:0/1152948228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c796d4d0 0x55d9c7966e40 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f6040000c00 tx=0x7f60400058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6809465651ae09d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:36.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.252+0000 7f60457fa640 1 -- 172.21.15.86:0/1152948228 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604004f030 con 0x55d9c796d4d0 2024-04-21T08:48:36.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.252+0000 7f60457fa640 1 -- 172.21.15.86:0/1152948228 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f604002cc90 con 0x55d9c796d4d0 2024-04-21T08:48:36.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.252+0000 7f60457fa640 1 -- 172.21.15.86:0/1152948228 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6040056620 con 0x55d9c796d4d0 2024-04-21T08:48:36.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.253+0000 7f605862a740 1 -- 172.21.15.86:0/1152948228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c796d4d0 msgr2=0x55d9c7966e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.253+0000 7f605862a740 1 --2- 172.21.15.86:0/1152948228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c796d4d0 0x55d9c7966e40 secure :-1 s=READY pgs=1781 cs=0 l=1 rev1=1 crypto rx=0x7f6040000c00 tx=0x7f60400058b0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.253+0000 7f605862a740 1 -- 172.21.15.86:0/1152948228 shutdown_connections 2024-04-21T08:48:36.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.253+0000 7f605862a740 1 --2- 172.21.15.86:0/1152948228 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9c7976a00 0x55d9c7975df0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.253+0000 7f605862a740 1 --2- 172.21.15.86:0/1152948228 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d9c796de80 0x55d9c798c290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.253+0000 7f605862a740 1 --2- 172.21.15.86:0/1152948228 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c796d4d0 0x55d9c7966e40 unknown :-1 s=CLOSED pgs=1781 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.253+0000 7f605862a740 1 -- 172.21.15.86:0/1152948228 >> 172.21.15.86:0/1152948228 conn(0x55d9c7967ba0 msgr2=0x55d9c79825a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:36.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.253+0000 7f605862a740 1 -- 172.21.15.86:0/1152948228 shutdown_connections 2024-04-21T08:48:36.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.253+0000 7f605862a740 1 -- 172.21.15.86:0/1152948228 wait complete. 2024-04-21T08:48:36.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.253+0000 7f605862a740 1 -- start start 2024-04-21T08:48:36.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.254+0000 7f605862a740 1 Processor -- start 2024-04-21T08:48:36.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.254+0000 7f605862a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d9c796d4d0 0x55d9c79e94d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.254+0000 7f605862a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9c796de80 0x55d9c79e9a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.254+0000 7f605862a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c7976a00 0x55d9c79e38e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.254+0000 7f605862a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60501b0870 con 0x55d9c7976a00 2024-04-21T08:48:36.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.254+0000 7f605862a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f60501b01a0 con 0x55d9c796d4d0 2024-04-21T08:48:36.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.254+0000 7f605862a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f60501ad400 con 0x55d9c796de80 2024-04-21T08:48:36.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.254+0000 7f605862a740 10 client.0 _ll_register_callbacks cb 0x7f6040005de0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:36.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.254+0000 7f604e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c7976a00 0x55d9c79e38e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.254+0000 7f604e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c7976a00 0x55d9c79e38e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38422/0 (socket says 172.21.15.86:38422) 2024-04-21T08:48:36.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.254+0000 7f604e7fc640 1 -- 172.21.15.86:0/3680052802 learned_addr learned my addr 172.21.15.86:0/3680052802 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:36.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.254+0000 7f604effd640 1 --2- 172.21.15.86:0/3680052802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9c796de80 0x55d9c79e9a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.254+0000 7f604e7fc640 1 --2- 172.21.15.86:0/3680052802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d9c796d4d0 0x55d9c79e94d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.254+0000 7f604e7fc640 1 -- 172.21.15.86:0/3680052802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9c796de80 msgr2=0x55d9c79e9a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.254+0000 7f604e7fc640 1 --2- 172.21.15.86:0/3680052802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9c796de80 0x55d9c79e9a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.254+0000 7f604e7fc640 1 -- 172.21.15.86:0/3680052802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d9c796d4d0 msgr2=0x55d9c79e94d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.254+0000 7f604e7fc640 1 --2- 172.21.15.86:0/3680052802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d9c796d4d0 0x55d9c79e94d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.254+0000 7f604e7fc640 1 -- 172.21.15.86:0/3680052802 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d9c79e3fd0 con 0x55d9c7976a00 2024-04-21T08:48:36.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.254+0000 7f604e7fc640 1 --2- 172.21.15.86:0/3680052802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c7976a00 0x55d9c79e38e0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f60400054d0 tx=0x7f6040005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9800ebc468ec1bc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:36.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.254+0000 7f602effd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:36.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.255+0000 7f602effd640 1 -- 172.21.15.86:0/3680052802 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604005e070 con 0x55d9c7976a00 2024-04-21T08:48:36.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.255+0000 7f602effd640 1 -- 172.21.15.86:0/3680052802 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f60400563b0 con 0x55d9c7976a00 2024-04-21T08:48:36.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.255+0000 7f605862a740 1 -- 172.21.15.86:0/3680052802 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f604002cc90 con 0x55d9c7976a00 2024-04-21T08:48:36.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.255+0000 7f602effd640 1 -- 172.21.15.86:0/3680052802 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f604004f040 con 0x55d9c7976a00 2024-04-21T08:48:36.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.255+0000 7f605862a740 10 client.19560 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:36.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.255+0000 7f605862a740 1 -- 172.21.15.86:0/3680052802 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d9c79f0cd0 con 0x55d9c7976a00 2024-04-21T08:48:36.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.255+0000 7f602effd640 1 -- 172.21.15.86:0/3680052802 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f604002cc90 con 0x55d9c7976a00 2024-04-21T08:48:36.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.255+0000 7f602effd640 1 -- 172.21.15.86:0/3680052802 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6040064720 con 0x55d9c7976a00 2024-04-21T08:48:36.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.255+0000 7f605862a740 10 client.19560 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:36.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.255+0000 7f605862a740 10 client.19560 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:48:36.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.255+0000 7f605862a740 20 client.19560 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:36.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.255+0000 7f605862a740 1 --2- 172.21.15.86:0/3680052802 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55d9c79f1230 0x55d9c79f3650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.255+0000 7f605862a740 4 client.19560 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:48:36.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.255+0000 7f605862a740 1 -- 172.21.15.86:0/3680052802 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d9c78dfc10 con 0x55d9c79f1230 2024-04-21T08:48:36.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.255+0000 7f604effd640 1 --2- 172.21.15.86:0/3680052802 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55d9c79f1230 0x55d9c79f3650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.256+0000 7f604effd640 1 --2- 172.21.15.86:0/3680052802 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55d9c79f1230 0x55d9c79f3650 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=18d48f14f2a97631 server_cookie=cb6287bdb5f49663 in_seq=0 out_seq=0 2024-04-21T08:48:36.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.256+0000 7f602effd640 10 client.19560 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:48:36.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.256+0000 7f602effd640 1 -- 172.21.15.86:0/3680052802 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d9c78dfc10 con 0x55d9c79f1230 2024-04-21T08:48:36.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.256+0000 7f602effd640 10 client.19560 handle_command_reply: tid=0 2024-04-21T08:48:36.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.256+0000 7f605862a740 1 client.19560 shutdown 2024-04-21T08:48:36.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.256+0000 7f605862a740 1 -- 172.21.15.86:0/3680052802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c7976a00 msgr2=0x55d9c79e38e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.256+0000 7f605862a740 1 --2- 172.21.15.86:0/3680052802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c7976a00 0x55d9c79e38e0 secure :-1 s=READY pgs=1782 cs=0 l=1 rev1=1 crypto rx=0x7f60400054d0 tx=0x7f6040005fd0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.256+0000 7f605862a740 1 -- 172.21.15.86:0/3680052802 shutdown_connections 2024-04-21T08:48:36.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.256+0000 7f605862a740 1 --2- 172.21.15.86:0/3680052802 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55d9c79f1230 0x55d9c79f3650 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.256+0000 7f605862a740 1 --2- 172.21.15.86:0/3680052802 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d9c7976a00 0x55d9c79e38e0 unknown :-1 s=CLOSED pgs=1782 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.256+0000 7f605862a740 1 --2- 172.21.15.86:0/3680052802 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d9c796de80 0x55d9c79e9a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.256+0000 7f605862a740 1 --2- 172.21.15.86:0/3680052802 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d9c796d4d0 0x55d9c79e94d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.256+0000 7f602effd640 0 client.19560 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:48:36.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.256+0000 7f605862a740 1 -- 172.21.15.86:0/3680052802 >> 172.21.15.86:0/3680052802 conn(0x55d9c7967ba0 msgr2=0x55d9c7981d10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:36.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.256+0000 7f605862a740 1 -- 172.21.15.86:0/3680052802 shutdown_connections 2024-04-21T08:48:36.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.256+0000 7f605862a740 1 -- 172.21.15.86:0/3680052802 wait complete. 2024-04-21T08:48:36.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.256+0000 7f605862a740 20 client.19560 trim_cache size 0 max 16384 2024-04-21T08:48:36.263 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:48:36.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.257+0000 7f605595f640 1 -- 172.21.15.86:0/1067108896 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f603c0a3d20 msgr2=0x7f603c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.257+0000 7f605595f640 1 --2- 172.21.15.86:0/1067108896 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f603c0a3d20 0x7f603c0a61e0 secure :-1 s=READY pgs=293 cs=0 l=1 rev1=1 crypto rx=0x7f60480ce1f0 tx=0x7f60480bd040 comp rx=0 tx=0).stop 2024-04-21T08:48:36.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.257+0000 7f605595f640 1 -- 172.21.15.86:0/1067108896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6050198590 msgr2=0x7f6050198a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.257+0000 7f605595f640 1 --2- 172.21.15.86:0/1067108896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6050198590 0x7f6050198a40 secure :-1 s=READY pgs=1778 cs=0 l=1 rev1=1 crypto rx=0x7f604800eaa0 tx=0x7f604800ef60 comp rx=0 tx=0).stop 2024-04-21T08:48:36.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.257+0000 7f605595f640 1 -- 172.21.15.86:0/1067108896 shutdown_connections 2024-04-21T08:48:36.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.257+0000 7f605595f640 1 --2- 172.21.15.86:0/1067108896 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f603c0a3d20 0x7f603c0a61e0 unknown :-1 s=CLOSED pgs=293 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.257+0000 7f605595f640 1 --2- 172.21.15.86:0/1067108896 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6050198590 0x7f6050198a40 unknown :-1 s=CLOSED pgs=1778 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.257+0000 7f605595f640 1 --2- 172.21.15.86:0/1067108896 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6050107270 0x7f605019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.257+0000 7f605595f640 1 --2- 172.21.15.86:0/1067108896 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f605006a880 0x7f605019e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.257+0000 7f605595f640 1 -- 172.21.15.86:0/1067108896 >> 172.21.15.86:0/1067108896 conn(0x7f6050077eb0 msgr2=0x7f6050108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:36.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.257+0000 7f605595f640 1 -- 172.21.15.86:0/1067108896 shutdown_connections 2024-04-21T08:48:36.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.257+0000 7f605595f640 1 -- 172.21.15.86:0/1067108896 wait complete. 2024-04-21T08:48:36.267 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:48:36.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.409+0000 7fecc46d5640 1 -- 172.21.15.86:0/1214883720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecbc06b1f0 msgr2=0x7fecbc106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.409+0000 7fecc46d5640 1 --2- 172.21.15.86:0/1214883720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecbc06b1f0 0x7fecbc106b90 secure :-1 s=READY pgs=1783 cs=0 l=1 rev1=1 crypto rx=0x7fecac009900 tx=0x7fecac02bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:36.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.409+0000 7fecc46d5640 1 -- 172.21.15.86:0/1214883720 shutdown_connections 2024-04-21T08:48:36.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.409+0000 7fecc46d5640 1 --2- 172.21.15.86:0/1214883720 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecbc107270 0x7fecbc10ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.409+0000 7fecc46d5640 1 --2- 172.21.15.86:0/1214883720 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecbc06b1f0 0x7fecbc106b90 unknown :-1 s=CLOSED pgs=1783 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.409+0000 7fecc46d5640 1 --2- 172.21.15.86:0/1214883720 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fecbc06a840 0x7fecbc06ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.409+0000 7fecc46d5640 1 -- 172.21.15.86:0/1214883720 >> 172.21.15.86:0/1214883720 conn(0x7fecbc077eb0 msgr2=0x7fecbc0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:36.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.409+0000 7fecc46d5640 1 -- 172.21.15.86:0/1214883720 shutdown_connections 2024-04-21T08:48:36.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.409+0000 7fecc46d5640 1 -- 172.21.15.86:0/1214883720 wait complete. 2024-04-21T08:48:36.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.410+0000 7fecc46d5640 1 Processor -- start 2024-04-21T08:48:36.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.410+0000 7fecc46d5640 1 -- start start 2024-04-21T08:48:36.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.410+0000 7fecc46d5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecbc06a840 0x7fecbc198730 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.410+0000 7fecc46d5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fecbc107270 0x7fecbc198c70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.410+0000 7fecc46d5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecbc19cff0 0x7fecbc19d4a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.410+0000 7fecc46d5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc112190 con 0x7fecbc06a840 2024-04-21T08:48:36.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.410+0000 7fecc46d5640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc112010 con 0x7fecbc107270 2024-04-21T08:48:36.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.410+0000 7fecc46d5640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc112310 con 0x7fecbc19cff0 2024-04-21T08:48:36.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.410+0000 7fecc2471640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecbc06a840 0x7fecbc198730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.410+0000 7fecc2471640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecbc06a840 0x7fecbc198730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38440/0 (socket says 172.21.15.86:38440) 2024-04-21T08:48:36.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.410+0000 7fecc2471640 1 -- 172.21.15.86:0/887521849 learned_addr learned my addr 172.21.15.86:0/887521849 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:36.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.410+0000 7fecc2c72640 1 --2- 172.21.15.86:0/887521849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecbc19cff0 0x7fecbc19d4a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.410+0000 7fecc1c70640 1 --2- 172.21.15.86:0/887521849 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fecbc107270 0x7fecbc198c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.410+0000 7fecc2471640 1 -- 172.21.15.86:0/887521849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecbc19cff0 msgr2=0x7fecbc19d4a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.410+0000 7fecc2471640 1 --2- 172.21.15.86:0/887521849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecbc19cff0 0x7fecbc19d4a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.410+0000 7fecc2471640 1 -- 172.21.15.86:0/887521849 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fecbc107270 msgr2=0x7fecbc198c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.410+0000 7fecc2471640 1 --2- 172.21.15.86:0/887521849 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fecbc107270 0x7fecbc198c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.410+0000 7fecc2471640 1 -- 172.21.15.86:0/887521849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fecbc19dae0 con 0x7fecbc06a840 2024-04-21T08:48:36.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.410+0000 7fecc2471640 1 --2- 172.21.15.86:0/887521849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecbc06a840 0x7fecbc198730 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fecb800c890 tx=0x7fecb800cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:36.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.410+0000 7fecb37fe640 1 -- 172.21.15.86:0/887521849 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8004220 con 0x7fecbc06a840 2024-04-21T08:48:36.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.411+0000 7fecb37fe640 1 -- 172.21.15.86:0/887521849 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecb80043c0 con 0x7fecbc06a840 2024-04-21T08:48:36.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.411+0000 7fecc46d5640 1 -- 172.21.15.86:0/887521849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fecbc19ddd0 con 0x7fecbc06a840 2024-04-21T08:48:36.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.411+0000 7fecb37fe640 1 -- 172.21.15.86:0/887521849 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb8004a00 con 0x7fecbc06a840 2024-04-21T08:48:36.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.411+0000 7fecc46d5640 1 -- 172.21.15.86:0/887521849 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fecbc1a5590 con 0x7fecbc06a840 2024-04-21T08:48:36.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.413+0000 7fecb37fe640 1 -- 172.21.15.86:0/887521849 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fecb80079b0 con 0x7fecbc06a840 2024-04-21T08:48:36.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.413+0000 7fecc739b740 1 Processor -- start 2024-04-21T08:48:36.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.413+0000 7fecc739b740 1 -- start start 2024-04-21T08:48:36.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.413+0000 7fecc739b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590efb6f4d0 0x5590efb6f8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.413+0000 7fecc739b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590efb6fe80 0x5590efb78430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.414+0000 7fecc739b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5590efb78a00 0x5590efb7ae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.414+0000 7fecc739b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5590efad41f0 con 0x5590efb6f4d0 2024-04-21T08:48:36.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.414+0000 7fecc739b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5590efa714b0 con 0x5590efb78a00 2024-04-21T08:48:36.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.414+0000 7fecc739b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5590efad01b0 con 0x5590efb6fe80 2024-04-21T08:48:36.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.414+0000 7fecb37fe640 1 --2- 172.21.15.86:0/887521849 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7feca80a3d20 0x7feca80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.414+0000 7fecc2c72640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590efb6f4d0 0x5590efb6f8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.414+0000 7fecc2c72640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590efb6f4d0 0x5590efb6f8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38452/0 (socket says 172.21.15.86:38452) 2024-04-21T08:48:36.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.414+0000 7fecc2c72640 1 -- 172.21.15.86:0/2698578156 learned_addr learned my addr 172.21.15.86:0/2698578156 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:36.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.414+0000 7fecc2c72640 1 --2- 172.21.15.86:0/887521849 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7feca80a3d20 0x7feca80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.414+0000 7fecb37fe640 1 -- 172.21.15.86:0/887521849 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7fecb80d09d0 con 0x7fecbc06a840 2024-04-21T08:48:36.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.414+0000 7fecc2471640 1 --2- 172.21.15.86:0/2698578156 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590efb6fe80 0x5590efb78430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.414+0000 7fecc1c70640 1 --2- 172.21.15.86:0/2698578156 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5590efb78a00 0x5590efb7ae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.414+0000 7fecc2c72640 1 -- 172.21.15.86:0/2698578156 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590efb6fe80 msgr2=0x5590efb78430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.414+0000 7fecc2c72640 1 --2- 172.21.15.86:0/2698578156 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590efb6fe80 0x5590efb78430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.414+0000 7fecc2c72640 1 -- 172.21.15.86:0/2698578156 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5590efb78a00 msgr2=0x5590efb7ae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.414+0000 7fecc2c72640 1 --2- 172.21.15.86:0/2698578156 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5590efb78a00 0x5590efb7ae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.414+0000 7fecc2c72640 1 -- 172.21.15.86:0/2698578156 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590efa631b0 con 0x5590efb6f4d0 2024-04-21T08:48:36.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.414+0000 7fecc1c70640 1 --2- 172.21.15.86:0/2698578156 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5590efb78a00 0x5590efb7ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:36.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.414+0000 7fecc2c72640 1 --2- 172.21.15.86:0/2698578156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590efb6f4d0 0x5590efb6f8b0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fecb4014eb0 tx=0x7fecb40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ec39aadccb1dfdae server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:36.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.414+0000 7fecc2c72640 1 --2- 172.21.15.86:0/887521849 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7feca80a3d20 0x7feca80a61e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fecb4011cb0 tx=0x7fecb4016000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:36.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.414+0000 7fecb0ff9640 1 -- 172.21.15.86:0/2698578156 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb4009c20 con 0x5590efb6f4d0 2024-04-21T08:48:36.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.414+0000 7fecb0ff9640 1 -- 172.21.15.86:0/2698578156 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecb4014810 con 0x5590efb6f4d0 2024-04-21T08:48:36.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.415+0000 7fecb0ff9640 1 -- 172.21.15.86:0/2698578156 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb4003bd0 con 0x5590efb6f4d0 2024-04-21T08:48:36.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.415+0000 7fecc739b740 1 -- 172.21.15.86:0/2698578156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590efb6f4d0 msgr2=0x5590efb6f8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.415+0000 7fecc739b740 1 --2- 172.21.15.86:0/2698578156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590efb6f4d0 0x5590efb6f8b0 secure :-1 s=READY pgs=1785 cs=0 l=1 rev1=1 crypto rx=0x7fecb4014eb0 tx=0x7fecb40117b0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.415+0000 7fecc739b740 1 -- 172.21.15.86:0/2698578156 shutdown_connections 2024-04-21T08:48:36.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.415+0000 7fecc739b740 1 --2- 172.21.15.86:0/2698578156 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5590efb78a00 0x5590efb7ae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.415+0000 7fecc739b740 1 --2- 172.21.15.86:0/2698578156 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590efb6fe80 0x5590efb78430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.415+0000 7fecc739b740 1 --2- 172.21.15.86:0/2698578156 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590efb6f4d0 0x5590efb6f8b0 unknown :-1 s=CLOSED pgs=1785 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.415+0000 7fecc739b740 1 -- 172.21.15.86:0/2698578156 >> 172.21.15.86:0/2698578156 conn(0x5590efb69ba0 msgr2=0x5590efb69f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:36.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.415+0000 7fecc739b740 1 -- 172.21.15.86:0/2698578156 shutdown_connections 2024-04-21T08:48:36.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.415+0000 7fecc739b740 1 -- 172.21.15.86:0/2698578156 wait complete. 2024-04-21T08:48:36.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.416+0000 7fecc739b740 1 -- start start 2024-04-21T08:48:36.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.416+0000 7fecc739b740 1 Processor -- start 2024-04-21T08:48:36.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.416+0000 7fecc739b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590efb6f4d0 0x5590efb78380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.416+0000 7fecc739b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5590efb6fe80 0x5590efb75710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.416+0000 7fecc739b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590efb78a00 0x5590efb75d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.416+0000 7fecc739b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5590efb7b910 con 0x5590efb78a00 2024-04-21T08:48:36.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.416+0000 7fecc739b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5590efa8d500 con 0x5590efb6fe80 2024-04-21T08:48:36.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.416+0000 7fecc739b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5590efb7ba90 con 0x5590efb6f4d0 2024-04-21T08:48:36.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.416+0000 7fecc739b740 10 client.0 _ll_register_callbacks cb 0x5590efa5bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:36.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.416+0000 7fecc2c72640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590efb78a00 0x5590efb75d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.416+0000 7fecc2c72640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590efb78a00 0x5590efb75d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38466/0 (socket says 172.21.15.86:38466) 2024-04-21T08:48:36.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.416+0000 7fecc2c72640 1 -- 172.21.15.86:0/2971205057 learned_addr learned my addr 172.21.15.86:0/2971205057 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:36.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.416+0000 7fecc2471640 1 --2- 172.21.15.86:0/2971205057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590efb6f4d0 0x5590efb78380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.416+0000 7fecc2c72640 1 -- 172.21.15.86:0/2971205057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590efb6f4d0 msgr2=0x5590efb78380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.416+0000 7fecc2c72640 1 --2- 172.21.15.86:0/2971205057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590efb6f4d0 0x5590efb78380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.416+0000 7fecc2c72640 1 -- 172.21.15.86:0/2971205057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5590efb6fe80 msgr2=0x5590efb75710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.416+0000 7fecc1c70640 1 --2- 172.21.15.86:0/2971205057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5590efb6fe80 0x5590efb75710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.416+0000 7fecc2c72640 1 --2- 172.21.15.86:0/2971205057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5590efb6fe80 0x5590efb75710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.416+0000 7fecc2c72640 1 -- 172.21.15.86:0/2971205057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590ef7e63f0 con 0x5590efb78a00 2024-04-21T08:48:36.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.417+0000 7fecc2c72640 1 --2- 172.21.15.86:0/2971205057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590efb78a00 0x5590efb75d20 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fecb40144c0 tx=0x7fecb4009330 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d59738629f941f8d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:36.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.417+0000 7feca5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:36.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.417+0000 7feca5ffb640 1 -- 172.21.15.86:0/2971205057 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb402e070 con 0x5590efb78a00 2024-04-21T08:48:36.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.417+0000 7feca5ffb640 1 -- 172.21.15.86:0/2971205057 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7fecb4003b80 con 0x5590efb78a00 2024-04-21T08:48:36.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.417+0000 7fecc739b740 1 -- 172.21.15.86:0/2971205057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecb4014810 con 0x5590efb78a00 2024-04-21T08:48:36.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.417+0000 7feca5ffb640 1 -- 172.21.15.86:0/2971205057 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fecb403a050 con 0x5590efb78a00 2024-04-21T08:48:36.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.417+0000 7fecc739b740 10 client.19572 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:36.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.417+0000 7fecc739b740 1 -- 172.21.15.86:0/2971205057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5590ef8b07d0 con 0x5590efb78a00 2024-04-21T08:48:36.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.417+0000 7feca5ffb640 1 -- 172.21.15.86:0/2971205057 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecb4014810 con 0x5590efb78a00 2024-04-21T08:48:36.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.417+0000 7feca5ffb640 1 -- 172.21.15.86:0/2971205057 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fecb401d070 con 0x5590efb78a00 2024-04-21T08:48:36.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.417+0000 7fecc739b740 10 client.19572 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:36.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.417+0000 7fecc739b740 10 client.19572 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:48:36.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.417+0000 7fecc739b740 20 client.19572 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:36.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.417+0000 7fecc739b740 1 --2- 172.21.15.86:0/2971205057 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5590efb88b80 0x5590efb8af70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.417+0000 7fecc739b740 4 client.19572 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:36.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.417+0000 7fecc739b740 1 -- 172.21.15.86:0/2971205057 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5590efa73930 con 0x5590efb88b80 2024-04-21T08:48:36.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.417+0000 7fecc2471640 1 --2- 172.21.15.86:0/2971205057 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5590efb88b80 0x5590efb8af70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.418+0000 7feca5ffb640 10 client.19572 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:48:36.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.418+0000 7fecc2471640 1 --2- 172.21.15.86:0/2971205057 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5590efb88b80 0x5590efb8af70 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=c7abbe861a270209 server_cookie=d742e8a7f668a5db in_seq=0 out_seq=0 2024-04-21T08:48:36.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.419+0000 7feca5ffb640 1 -- 172.21.15.86:0/2971205057 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5590efa73930 con 0x5590efb88b80 2024-04-21T08:48:36.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.419+0000 7feca5ffb640 10 client.19572 handle_command_reply: tid=0 2024-04-21T08:48:36.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.419+0000 7fecc739b740 1 client.19572 shutdown 2024-04-21T08:48:36.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.419+0000 7fecc739b740 1 -- 172.21.15.86:0/2971205057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590efb78a00 msgr2=0x5590efb75d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.419+0000 7fecc739b740 1 --2- 172.21.15.86:0/2971205057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590efb78a00 0x5590efb75d20 secure :-1 s=READY pgs=1786 cs=0 l=1 rev1=1 crypto rx=0x7fecb40144c0 tx=0x7fecb4009330 comp rx=0 tx=0).stop 2024-04-21T08:48:36.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.419+0000 7fecc739b740 1 -- 172.21.15.86:0/2971205057 shutdown_connections 2024-04-21T08:48:36.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.419+0000 7fecc739b740 1 --2- 172.21.15.86:0/2971205057 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5590efb88b80 0x5590efb8af70 crc :-1 s=READY pgs=69 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.419+0000 7fecc739b740 1 --2- 172.21.15.86:0/2971205057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590efb78a00 0x5590efb75d20 unknown :-1 s=CLOSED pgs=1786 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.419+0000 7fecc739b740 1 --2- 172.21.15.86:0/2971205057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5590efb6fe80 0x5590efb75710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.419+0000 7feca5ffb640 0 client.19572 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:48:36.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.419+0000 7fecc739b740 1 --2- 172.21.15.86:0/2971205057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590efb6f4d0 0x5590efb78380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.419+0000 7fecc739b740 1 -- 172.21.15.86:0/2971205057 >> 172.21.15.86:0/2971205057 conn(0x5590efb69ba0 msgr2=0x5590efb6b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:36.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.419+0000 7fecc739b740 1 -- 172.21.15.86:0/2971205057 shutdown_connections 2024-04-21T08:48:36.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.419+0000 7fecc739b740 1 -- 172.21.15.86:0/2971205057 wait complete. 2024-04-21T08:48:36.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.419+0000 7fecc739b740 20 client.19572 trim_cache size 0 max 16384 2024-04-21T08:48:36.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.432+0000 7fecc739b740 1 Processor -- start 2024-04-21T08:48:36.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.432+0000 7fecc739b740 1 -- start start 2024-04-21T08:48:36.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.432+0000 7fecc739b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5590efb6f4d0 0x5590efb847d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.432+0000 7fecc739b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590efb6fe80 0x5590efb68370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.432+0000 7fecc739b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590efb78a00 0x5590efb77e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.432+0000 7fecc739b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc1af380 con 0x5590efb78a00 2024-04-21T08:48:36.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.432+0000 7fecc739b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc1b0ec0 con 0x5590efb6f4d0 2024-04-21T08:48:36.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.432+0000 7fecc739b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc1afa50 con 0x5590efb6fe80 2024-04-21T08:48:36.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.433+0000 7fecc2c72640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590efb78a00 0x5590efb77e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.433+0000 7fecc2c72640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590efb78a00 0x5590efb77e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38472/0 (socket says 172.21.15.86:38472) 2024-04-21T08:48:36.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.433+0000 7fecc2c72640 1 -- 172.21.15.86:0/2293334795 learned_addr learned my addr 172.21.15.86:0/2293334795 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:36.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.433+0000 7fecc2471640 1 --2- 172.21.15.86:0/2293334795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5590efb6f4d0 0x5590efb847d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.433+0000 7fecc1c70640 1 --2- 172.21.15.86:0/2293334795 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590efb6fe80 0x5590efb68370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.433+0000 7fecc2c72640 1 -- 172.21.15.86:0/2293334795 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590efb6fe80 msgr2=0x5590efb68370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.433+0000 7fecc2c72640 1 --2- 172.21.15.86:0/2293334795 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590efb6fe80 0x5590efb68370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.433+0000 7fecc2c72640 1 -- 172.21.15.86:0/2293334795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5590efb6f4d0 msgr2=0x5590efb847d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.433+0000 7fecc2c72640 1 --2- 172.21.15.86:0/2293334795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5590efb6f4d0 0x5590efb847d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.433+0000 7fecc2c72640 1 -- 172.21.15.86:0/2293334795 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5590efb65470 con 0x5590efb78a00 2024-04-21T08:48:36.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.433+0000 7fecc2471640 1 --2- 172.21.15.86:0/2293334795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5590efb6f4d0 0x5590efb847d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:36.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.433+0000 7fecc1c70640 1 --2- 172.21.15.86:0/2293334795 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590efb6fe80 0x5590efb68370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:36.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.433+0000 7fecc2c72640 1 --2- 172.21.15.86:0/2293334795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590efb78a00 0x5590efb77e40 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fecb4006010 tx=0x7fecb40117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=375db85b766ed90a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:36.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.433+0000 7feca7fff640 1 -- 172.21.15.86:0/2293334795 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb4037020 con 0x5590efb78a00 2024-04-21T08:48:36.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.433+0000 7feca7fff640 1 -- 172.21.15.86:0/2293334795 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecb4005bd0 con 0x5590efb78a00 2024-04-21T08:48:36.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.433+0000 7feca7fff640 1 -- 172.21.15.86:0/2293334795 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb402e040 con 0x5590efb78a00 2024-04-21T08:48:36.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.434+0000 7fecc739b740 1 -- 172.21.15.86:0/2293334795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590efb78a00 msgr2=0x5590efb77e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.434+0000 7fecc739b740 1 --2- 172.21.15.86:0/2293334795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590efb78a00 0x5590efb77e40 secure :-1 s=READY pgs=1787 cs=0 l=1 rev1=1 crypto rx=0x7fecb4006010 tx=0x7fecb40117b0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.434+0000 7fecc739b740 1 -- 172.21.15.86:0/2293334795 shutdown_connections 2024-04-21T08:48:36.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.434+0000 7fecc739b740 1 --2- 172.21.15.86:0/2293334795 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590efb78a00 0x5590efb77e40 unknown :-1 s=CLOSED pgs=1787 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.434+0000 7fecc739b740 1 --2- 172.21.15.86:0/2293334795 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590efb6fe80 0x5590efb68370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.434+0000 7fecc739b740 1 --2- 172.21.15.86:0/2293334795 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5590efb6f4d0 0x5590efb847d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.434+0000 7fecc739b740 1 -- 172.21.15.86:0/2293334795 >> 172.21.15.86:0/2293334795 conn(0x5590efb69ba0 msgr2=0x5590efb70c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:36.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.434+0000 7fecc739b740 1 -- 172.21.15.86:0/2293334795 shutdown_connections 2024-04-21T08:48:36.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.434+0000 7fecc739b740 1 -- 172.21.15.86:0/2293334795 wait complete. 2024-04-21T08:48:36.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.434+0000 7fecc739b740 1 -- start start 2024-04-21T08:48:36.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.434+0000 7fecc739b740 1 Processor -- start 2024-04-21T08:48:36.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.434+0000 7fecc739b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5590efb6f4d0 0x5590efb7c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.434+0000 7fecc739b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590efb6fe80 0x5590efb7cd70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.434+0000 7fecc739b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590efb78a00 0x5590efb7d2b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.434+0000 7fecc739b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc1b07f0 con 0x5590efb78a00 2024-04-21T08:48:36.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.434+0000 7fecc739b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc1b0120 con 0x5590efb6f4d0 2024-04-21T08:48:36.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.434+0000 7fecc739b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fecbc1ad380 con 0x5590efb6fe80 2024-04-21T08:48:36.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.434+0000 7fecc739b740 10 client.0 _ll_register_callbacks cb 0x7fecb4009750 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:36.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.434+0000 7fecc2c72640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590efb78a00 0x5590efb7d2b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.434+0000 7fecc2c72640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590efb78a00 0x5590efb7d2b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38482/0 (socket says 172.21.15.86:38482) 2024-04-21T08:48:36.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.434+0000 7fecc2c72640 1 -- 172.21.15.86:0/3028466300 learned_addr learned my addr 172.21.15.86:0/3028466300 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:36.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.435+0000 7fecc1c70640 1 --2- 172.21.15.86:0/3028466300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590efb6fe80 0x5590efb7cd70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.435+0000 7fecc2471640 1 --2- 172.21.15.86:0/3028466300 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5590efb6f4d0 0x5590efb7c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.435+0000 7fecc2c72640 1 -- 172.21.15.86:0/3028466300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590efb6fe80 msgr2=0x5590efb7cd70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.435+0000 7fecc2c72640 1 --2- 172.21.15.86:0/3028466300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590efb6fe80 0x5590efb7cd70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.435+0000 7fecc2c72640 1 -- 172.21.15.86:0/3028466300 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5590efb6f4d0 msgr2=0x5590efb7c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.435+0000 7fecc2c72640 1 --2- 172.21.15.86:0/3028466300 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5590efb6f4d0 0x5590efb7c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.435+0000 7fecc2c72640 1 -- 172.21.15.86:0/3028466300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5590efbedf40 con 0x5590efb78a00 2024-04-21T08:48:36.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.435+0000 7fecc2471640 1 --2- 172.21.15.86:0/3028466300 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5590efb6f4d0 0x5590efb7c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:36.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.435+0000 7fecc2c72640 1 --2- 172.21.15.86:0/3028466300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590efb78a00 0x5590efb7d2b0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fecb4014540 tx=0x7fecb4014880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c81449cc37b3fcf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:36.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.435+0000 7feca5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:36.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.435+0000 7feca5ffb640 1 -- 172.21.15.86:0/3028466300 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fecb4037030 con 0x5590efb78a00 2024-04-21T08:48:36.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.435+0000 7feca5ffb640 1 -- 172.21.15.86:0/3028466300 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7fecb4025070 con 0x5590efb78a00 2024-04-21T08:48:36.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.435+0000 7fecc739b740 1 -- 172.21.15.86:0/3028466300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fecb4005bd0 con 0x5590efb78a00 2024-04-21T08:48:36.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.435+0000 7feca5ffb640 1 -- 172.21.15.86:0/3028466300 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fecb404d050 con 0x5590efb78a00 2024-04-21T08:48:36.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.435+0000 7fecc739b740 10 client.19578 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:36.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.435+0000 7fecc739b740 1 -- 172.21.15.86:0/3028466300 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5590efbef190 con 0x5590efb78a00 2024-04-21T08:48:36.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.435+0000 7feca5ffb640 1 -- 172.21.15.86:0/3028466300 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fecb4005bd0 con 0x5590efb78a00 2024-04-21T08:48:36.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.435+0000 7feca5ffb640 1 -- 172.21.15.86:0/3028466300 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fecb401d040 con 0x5590efb78a00 2024-04-21T08:48:36.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.435+0000 7fecc739b740 10 client.19578 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:36.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.435+0000 7fecc739b740 10 client.19578 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:48:36.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.435+0000 7fecc739b740 20 client.19578 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:36.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.435+0000 7fecc739b740 1 --2- 172.21.15.86:0/3028466300 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5590efbef910 0x5590efbf1d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.435+0000 7fecc739b740 4 client.19578 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:48:36.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.435+0000 7fecc739b740 1 -- 172.21.15.86:0/3028466300 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5590efb69220 con 0x5590efbef910 2024-04-21T08:48:36.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.436+0000 7fecc2471640 1 --2- 172.21.15.86:0/3028466300 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5590efbef910 0x5590efbf1d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.436+0000 7feca5ffb640 10 client.19578 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:48:36.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.436+0000 7fecc2471640 1 --2- 172.21.15.86:0/3028466300 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5590efbef910 0x5590efbf1d30 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=b967eacc45334ffc server_cookie=1419faaf9aca9b80 in_seq=0 out_seq=0 2024-04-21T08:48:36.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.436+0000 7feca5ffb640 1 -- 172.21.15.86:0/3028466300 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5590efb69220 con 0x5590efbef910 2024-04-21T08:48:36.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.436+0000 7feca5ffb640 10 client.19578 handle_command_reply: tid=0 2024-04-21T08:48:36.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.436+0000 7fecc739b740 1 client.19578 shutdown 2024-04-21T08:48:36.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.436+0000 7fecc739b740 1 -- 172.21.15.86:0/3028466300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590efb78a00 msgr2=0x5590efb7d2b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.436+0000 7fecc739b740 1 --2- 172.21.15.86:0/3028466300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590efb78a00 0x5590efb7d2b0 secure :-1 s=READY pgs=1788 cs=0 l=1 rev1=1 crypto rx=0x7fecb4014540 tx=0x7fecb4014880 comp rx=0 tx=0).stop 2024-04-21T08:48:36.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.437+0000 7fecc739b740 1 -- 172.21.15.86:0/3028466300 shutdown_connections 2024-04-21T08:48:36.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.437+0000 7fecc739b740 1 --2- 172.21.15.86:0/3028466300 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x5590efbef910 0x5590efbf1d30 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.437+0000 7fecc739b740 1 --2- 172.21.15.86:0/3028466300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5590efb78a00 0x5590efb7d2b0 unknown :-1 s=CLOSED pgs=1788 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.437+0000 7fecc739b740 1 --2- 172.21.15.86:0/3028466300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5590efb6fe80 0x5590efb7cd70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.437+0000 7feca5ffb640 0 client.19578 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:48:36.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.437+0000 7fecc739b740 1 --2- 172.21.15.86:0/3028466300 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5590efb6f4d0 0x5590efb7c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.437+0000 7fecc739b740 1 -- 172.21.15.86:0/3028466300 >> 172.21.15.86:0/3028466300 conn(0x5590efb69ba0 msgr2=0x5590efb6af70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:36.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.437+0000 7fecc739b740 1 -- 172.21.15.86:0/3028466300 shutdown_connections 2024-04-21T08:48:36.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.437+0000 7fecc739b740 1 -- 172.21.15.86:0/3028466300 wait complete. 2024-04-21T08:48:36.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.437+0000 7fecc739b740 20 client.19578 trim_cache size 0 max 16384 2024-04-21T08:48:36.444 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:48:36.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.437+0000 7fecc46d5640 1 -- 172.21.15.86:0/887521849 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7feca80a3d20 msgr2=0x7feca80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.437+0000 7fecc46d5640 1 --2- 172.21.15.86:0/887521849 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7feca80a3d20 0x7feca80a61e0 secure :-1 s=READY pgs=294 cs=0 l=1 rev1=1 crypto rx=0x7fecb4011cb0 tx=0x7fecb4016000 comp rx=0 tx=0).stop 2024-04-21T08:48:36.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.437+0000 7fecc46d5640 1 -- 172.21.15.86:0/887521849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecbc06a840 msgr2=0x7fecbc198730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.437+0000 7fecc46d5640 1 --2- 172.21.15.86:0/887521849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecbc06a840 0x7fecbc198730 secure :-1 s=READY pgs=1784 cs=0 l=1 rev1=1 crypto rx=0x7fecb800c890 tx=0x7fecb800cd50 comp rx=0 tx=0).stop 2024-04-21T08:48:36.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.437+0000 7fecc46d5640 1 -- 172.21.15.86:0/887521849 shutdown_connections 2024-04-21T08:48:36.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.437+0000 7fecc46d5640 1 --2- 172.21.15.86:0/887521849 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7feca80a3d20 0x7feca80a61e0 unknown :-1 s=CLOSED pgs=294 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.437+0000 7fecc46d5640 1 --2- 172.21.15.86:0/887521849 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fecbc19cff0 0x7fecbc19d4a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.437+0000 7fecc46d5640 1 --2- 172.21.15.86:0/887521849 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fecbc107270 0x7fecbc198c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.437+0000 7fecc46d5640 1 --2- 172.21.15.86:0/887521849 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fecbc06a840 0x7fecbc198730 unknown :-1 s=CLOSED pgs=1784 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.437+0000 7fecc46d5640 1 -- 172.21.15.86:0/887521849 >> 172.21.15.86:0/887521849 conn(0x7fecbc077eb0 msgr2=0x7fecbc108440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:36.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.437+0000 7fecc46d5640 1 -- 172.21.15.86:0/887521849 shutdown_connections 2024-04-21T08:48:36.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.437+0000 7fecc46d5640 1 -- 172.21.15.86:0/887521849 wait complete. 2024-04-21T08:48:36.447 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:48:36.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.547+0000 7f2d7759d640 1 -- 172.21.15.86:0/2888406619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d701010a0 msgr2=0x7f2d70101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.547+0000 7f2d7759d640 1 --2- 172.21.15.86:0/2888406619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d701010a0 0x7f2d70101480 secure :-1 s=READY pgs=1789 cs=0 l=1 rev1=1 crypto rx=0x7f2d64009900 tx=0x7f2d6402bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:36.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.547+0000 7f2d7759d640 1 -- 172.21.15.86:0/2888406619 shutdown_connections 2024-04-21T08:48:36.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.547+0000 7f2d7759d640 1 --2- 172.21.15.86:0/2888406619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d70106170 0x7f2d7010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.547+0000 7f2d7759d640 1 --2- 172.21.15.86:0/2888406619 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d70101a50 0x7f2d70105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.547+0000 7f2d7759d640 1 --2- 172.21.15.86:0/2888406619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d701010a0 0x7f2d70101480 unknown :-1 s=CLOSED pgs=1789 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.547+0000 7f2d7759d640 1 -- 172.21.15.86:0/2888406619 >> 172.21.15.86:0/2888406619 conn(0x7f2d700771d0 msgr2=0x7f2d700775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:36.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.548+0000 7f2d7759d640 1 -- 172.21.15.86:0/2888406619 shutdown_connections 2024-04-21T08:48:36.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.548+0000 7f2d7759d640 1 -- 172.21.15.86:0/2888406619 wait complete. 2024-04-21T08:48:36.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.548+0000 7f2d7759d640 1 Processor -- start 2024-04-21T08:48:36.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.548+0000 7f2d7759d640 1 -- start start 2024-04-21T08:48:36.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.548+0000 7f2d7759d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d701010a0 0x7f2d70199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.548+0000 7f2d7759d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d70106170 0x7f2d70199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.548+0000 7f2d7759d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d7019a050 0x7f2d701a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.548+0000 7f2d7759d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2d701110e0 con 0x7f2d70106170 2024-04-21T08:48:36.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.548+0000 7f2d7759d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2d70110f60 con 0x7f2d7019a050 2024-04-21T08:48:36.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.548+0000 7f2d7759d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2d70111260 con 0x7f2d701010a0 2024-04-21T08:48:36.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.548+0000 7f2d74b38640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d70106170 0x7f2d70199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.548+0000 7f2d74b38640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d70106170 0x7f2d70199ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38490/0 (socket says 172.21.15.86:38490) 2024-04-21T08:48:36.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.548+0000 7f2d74b38640 1 -- 172.21.15.86:0/3153689138 learned_addr learned my addr 172.21.15.86:0/3153689138 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:36.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.548+0000 7f2d75339640 1 --2- 172.21.15.86:0/3153689138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d701010a0 0x7f2d70199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.549+0000 7f2d75b3a640 1 --2- 172.21.15.86:0/3153689138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d7019a050 0x7f2d701a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.549+0000 7f2d74b38640 1 -- 172.21.15.86:0/3153689138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d701010a0 msgr2=0x7f2d70199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.549+0000 7f2d74b38640 1 --2- 172.21.15.86:0/3153689138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d701010a0 0x7f2d70199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.549+0000 7f2d74b38640 1 -- 172.21.15.86:0/3153689138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d7019a050 msgr2=0x7f2d701a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.549+0000 7f2d74b38640 1 --2- 172.21.15.86:0/3153689138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d7019a050 0x7f2d701a4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.549+0000 7f2d74b38640 1 -- 172.21.15.86:0/3153689138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2d701a4e00 con 0x7f2d70106170 2024-04-21T08:48:36.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.549+0000 7f2d75b3a640 1 --2- 172.21.15.86:0/3153689138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d7019a050 0x7f2d701a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:36.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.549+0000 7f2d74b38640 1 --2- 172.21.15.86:0/3153689138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d70106170 0x7f2d70199ac0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c00c970 tx=0x7f2d6c00ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:36.551 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.549+0000 7f2d627fc640 1 -- 172.21.15.86:0/3153689138 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c007bb0 con 0x7f2d70106170 2024-04-21T08:48:36.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.549+0000 7f2d627fc640 1 -- 172.21.15.86:0/3153689138 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d6c007d50 con 0x7f2d70106170 2024-04-21T08:48:36.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.549+0000 7f2d7759d640 1 -- 172.21.15.86:0/3153689138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2d701a50f0 con 0x7f2d70106170 2024-04-21T08:48:36.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.549+0000 7f2d627fc640 1 -- 172.21.15.86:0/3153689138 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d6c00f450 con 0x7f2d70106170 2024-04-21T08:48:36.552 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.549+0000 7f2d7759d640 1 -- 172.21.15.86:0/3153689138 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2d701a5690 con 0x7f2d70106170 2024-04-21T08:48:36.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.551+0000 7f2d627fc640 1 -- 172.21.15.86:0/3153689138 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2d6c004f20 con 0x7f2d70106170 2024-04-21T08:48:36.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.552+0000 7f2d7a18c740 1 Processor -- start 2024-04-21T08:48:36.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.552+0000 7f2d7a18c740 1 -- start start 2024-04-21T08:48:36.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.552+0000 7f2d7a18c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557d6c0464d0 0x557d6c0468b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.552+0000 7f2d7a18c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6c046e80 0x557d6c04f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.552+0000 7f2d7a18c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d6c04fa00 0x557d6c051e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.552+0000 7f2d7a18c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557d6bfab1f0 con 0x557d6c046e80 2024-04-21T08:48:36.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.552+0000 7f2d7a18c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x557d6bf484b0 con 0x557d6c0464d0 2024-04-21T08:48:36.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.552+0000 7f2d7a18c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x557d6bfa71b0 con 0x557d6c04fa00 2024-04-21T08:48:36.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.552+0000 7f2d627fc640 1 --2- 172.21.15.86:0/3153689138 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2d5c0a3da0 0x7f2d5c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.552+0000 7f2d75339640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6c046e80 0x557d6c04f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.552+0000 7f2d75b3a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557d6c0464d0 0x557d6c0468b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.552+0000 7f2d75b3a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557d6c0464d0 0x557d6c0468b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.86:48610/0 (socket says 172.21.15.86:48610) 2024-04-21T08:48:36.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.552+0000 7f2d75339640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6c046e80 0x557d6c04f430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38494/0 (socket says 172.21.15.86:38494) 2024-04-21T08:48:36.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.552+0000 7f2d75b3a640 1 -- 172.21.15.86:0/573804188 learned_addr learned my addr 172.21.15.86:0/573804188 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:36.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.552+0000 7f2d75b3a640 1 --2- 172.21.15.86:0/3153689138 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2d5c0a3da0 0x7f2d5c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.552+0000 7f2d74b38640 1 --2- 172.21.15.86:0/573804188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d6c04fa00 0x557d6c051e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.552+0000 7f2d627fc640 1 -- 172.21.15.86:0/3153689138 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f2d6c0d0be0 con 0x7f2d70106170 2024-04-21T08:48:36.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.552+0000 7f2d75b3a640 1 -- 172.21.15.86:0/573804188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d6c04fa00 msgr2=0x557d6c051e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.552+0000 7f2d75b3a640 1 --2- 172.21.15.86:0/573804188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d6c04fa00 0x557d6c051e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.552+0000 7f2d75b3a640 1 -- 172.21.15.86:0/573804188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6c046e80 msgr2=0x557d6c04f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.552+0000 7f2d75b3a640 1 --2- 172.21.15.86:0/573804188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6c046e80 0x557d6c04f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.552+0000 7f2d75b3a640 1 -- 172.21.15.86:0/573804188 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d6bf3a1b0 con 0x557d6c0464d0 2024-04-21T08:48:36.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.552+0000 7f2d75339640 1 --2- 172.21.15.86:0/573804188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6c046e80 0x557d6c04f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:36.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.553+0000 7f2d75b3a640 1 --2- 172.21.15.86:0/573804188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557d6c0464d0 0x557d6c0468b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2d680092f0 tx=0x7f2d68011750 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fdd269888ad77319 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:36.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.553+0000 7f2d75b3a640 1 --2- 172.21.15.86:0/3153689138 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2d5c0a3da0 0x7f2d5c0a6260 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2d68011c50 tx=0x7f2d68016000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:36.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.553+0000 7f2d5b7fe640 1 -- 172.21.15.86:0/573804188 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d68022070 con 0x557d6c0464d0 2024-04-21T08:48:36.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.553+0000 7f2d5b7fe640 1 -- 172.21.15.86:0/573804188 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d68009bc0 con 0x557d6c0464d0 2024-04-21T08:48:36.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.553+0000 7f2d5b7fe640 1 -- 172.21.15.86:0/573804188 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d680052f0 con 0x557d6c0464d0 2024-04-21T08:48:36.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.554+0000 7f2d7a18c740 1 -- 172.21.15.86:0/573804188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557d6c0464d0 msgr2=0x557d6c0468b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.554+0000 7f2d7a18c740 1 --2- 172.21.15.86:0/573804188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557d6c0464d0 0x557d6c0468b0 secure :-1 s=READY pgs=81 cs=0 l=1 rev1=1 crypto rx=0x7f2d680092f0 tx=0x7f2d68011750 comp rx=0 tx=0).stop 2024-04-21T08:48:36.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.554+0000 7f2d7a18c740 1 -- 172.21.15.86:0/573804188 shutdown_connections 2024-04-21T08:48:36.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.554+0000 7f2d7a18c740 1 --2- 172.21.15.86:0/573804188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d6c04fa00 0x557d6c051e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.554+0000 7f2d7a18c740 1 --2- 172.21.15.86:0/573804188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6c046e80 0x557d6c04f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.554+0000 7f2d7a18c740 1 --2- 172.21.15.86:0/573804188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557d6c0464d0 0x557d6c0468b0 unknown :-1 s=CLOSED pgs=81 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.554+0000 7f2d7a18c740 1 -- 172.21.15.86:0/573804188 >> 172.21.15.86:0/573804188 conn(0x557d6c040ba0 msgr2=0x557d6c040f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:36.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.554+0000 7f2d7a18c740 1 -- 172.21.15.86:0/573804188 shutdown_connections 2024-04-21T08:48:36.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.554+0000 7f2d7a18c740 1 -- 172.21.15.86:0/573804188 wait complete. 2024-04-21T08:48:36.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.554+0000 7f2d7a18c740 1 -- start start 2024-04-21T08:48:36.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.554+0000 7f2d7a18c740 1 Processor -- start 2024-04-21T08:48:36.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.554+0000 7f2d7a18c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557d6c0464d0 0x557d6c04f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.554+0000 7f2d7a18c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d6c046e80 0x557d6c04c740 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.555+0000 7f2d7a18c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6c04fa00 0x557d6c04ce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.555+0000 7f2d7a18c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x557d6c052910 con 0x557d6c04fa00 2024-04-21T08:48:36.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.555+0000 7f2d7a18c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x557d6bf64500 con 0x557d6c0464d0 2024-04-21T08:48:36.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.555+0000 7f2d7a18c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x557d6c052a90 con 0x557d6c046e80 2024-04-21T08:48:36.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.555+0000 7f2d7a18c740 10 client.0 _ll_register_callbacks cb 0x557d6bf32b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:36.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.555+0000 7f2d75b3a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6c04fa00 0x557d6c04ce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.555+0000 7f2d75b3a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6c04fa00 0x557d6c04ce80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38506/0 (socket says 172.21.15.86:38506) 2024-04-21T08:48:36.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.555+0000 7f2d75b3a640 1 -- 172.21.15.86:0/702060344 learned_addr learned my addr 172.21.15.86:0/702060344 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:36.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.555+0000 7f2d75339640 1 --2- 172.21.15.86:0/702060344 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557d6c0464d0 0x557d6c04f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.555+0000 7f2d74b38640 1 --2- 172.21.15.86:0/702060344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d6c046e80 0x557d6c04c740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.555+0000 7f2d75b3a640 1 -- 172.21.15.86:0/702060344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d6c046e80 msgr2=0x557d6c04c740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.555+0000 7f2d75b3a640 1 --2- 172.21.15.86:0/702060344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d6c046e80 0x557d6c04c740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.555+0000 7f2d75b3a640 1 -- 172.21.15.86:0/702060344 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557d6c0464d0 msgr2=0x557d6c04f3b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.555+0000 7f2d75b3a640 1 --2- 172.21.15.86:0/702060344 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557d6c0464d0 0x557d6c04f3b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.555+0000 7f2d75b3a640 1 -- 172.21.15.86:0/702060344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d6bcbd3f0 con 0x557d6c04fa00 2024-04-21T08:48:36.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.555+0000 7f2d75339640 1 --2- 172.21.15.86:0/702060344 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557d6c0464d0 0x557d6c04f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:36.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.555+0000 7f2d74b38640 1 --2- 172.21.15.86:0/702060344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d6c046e80 0x557d6c04c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:36.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.555+0000 7f2d75b3a640 1 --2- 172.21.15.86:0/702060344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6c04fa00 0x557d6c04ce80 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f2d68009240 tx=0x7f2d6800cd70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81306f1b20dd3edd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:36.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.555+0000 7f2d58ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:36.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.555+0000 7f2d58ff9640 1 -- 172.21.15.86:0/702060344 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d68033060 con 0x557d6c04fa00 2024-04-21T08:48:36.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.555+0000 7f2d58ff9640 1 -- 172.21.15.86:0/702060344 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f2d68014260 con 0x557d6c04fa00 2024-04-21T08:48:36.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.555+0000 7f2d7a18c740 1 -- 172.21.15.86:0/702060344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d68009bc0 con 0x557d6c04fa00 2024-04-21T08:48:36.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.555+0000 7f2d58ff9640 1 -- 172.21.15.86:0/702060344 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2d6803a050 con 0x557d6c04fa00 2024-04-21T08:48:36.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.555+0000 7f2d7a18c740 10 client.19590 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:36.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.555+0000 7f2d7a18c740 1 -- 172.21.15.86:0/702060344 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557d6bd877d0 con 0x557d6c04fa00 2024-04-21T08:48:36.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.556+0000 7f2d58ff9640 1 -- 172.21.15.86:0/702060344 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d68009bc0 con 0x557d6c04fa00 2024-04-21T08:48:36.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.556+0000 7f2d58ff9640 1 -- 172.21.15.86:0/702060344 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2d68005660 con 0x557d6c04fa00 2024-04-21T08:48:36.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.556+0000 7f2d7a18c740 10 client.19590 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:36.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.556+0000 7f2d7a18c740 10 client.19590 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:48:36.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.556+0000 7f2d7a18c740 20 client.19590 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:36.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.556+0000 7f2d7a18c740 1 --2- 172.21.15.86:0/702060344 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x557d6c05fc30 0x557d6c062020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.556+0000 7f2d7a18c740 4 client.19590 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:36.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.556+0000 7f2d7a18c740 1 -- 172.21.15.86:0/702060344 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x557d6bf4a930 con 0x557d6c05fc30 2024-04-21T08:48:36.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.556+0000 7f2d75339640 1 --2- 172.21.15.86:0/702060344 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x557d6c05fc30 0x557d6c062020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.556+0000 7f2d58ff9640 10 client.19590 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:48:36.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.556+0000 7f2d75339640 1 --2- 172.21.15.86:0/702060344 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x557d6c05fc30 0x557d6c062020 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=b8521006c5ef0148 server_cookie=8b761523bcf5a15b in_seq=0 out_seq=0 2024-04-21T08:48:36.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.557+0000 7f2d58ff9640 1 -- 172.21.15.86:0/702060344 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x557d6bf4a930 con 0x557d6c05fc30 2024-04-21T08:48:36.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.557+0000 7f2d58ff9640 10 client.19590 handle_command_reply: tid=0 2024-04-21T08:48:36.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.557+0000 7f2d7a18c740 1 client.19590 shutdown 2024-04-21T08:48:36.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.557+0000 7f2d7a18c740 1 -- 172.21.15.86:0/702060344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6c04fa00 msgr2=0x557d6c04ce80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.557+0000 7f2d7a18c740 1 --2- 172.21.15.86:0/702060344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6c04fa00 0x557d6c04ce80 secure :-1 s=READY pgs=1791 cs=0 l=1 rev1=1 crypto rx=0x7f2d68009240 tx=0x7f2d6800cd70 comp rx=0 tx=0).stop 2024-04-21T08:48:36.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.557+0000 7f2d7a18c740 1 -- 172.21.15.86:0/702060344 shutdown_connections 2024-04-21T08:48:36.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.557+0000 7f2d7a18c740 1 --2- 172.21.15.86:0/702060344 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x557d6c05fc30 0x557d6c062020 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.557+0000 7f2d7a18c740 1 --2- 172.21.15.86:0/702060344 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6c04fa00 0x557d6c04ce80 unknown :-1 s=CLOSED pgs=1791 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.557+0000 7f2d58ff9640 0 client.19590 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:48:36.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.557+0000 7f2d7a18c740 1 --2- 172.21.15.86:0/702060344 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d6c046e80 0x557d6c04c740 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.557+0000 7f2d7a18c740 1 --2- 172.21.15.86:0/702060344 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557d6c0464d0 0x557d6c04f3b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.557+0000 7f2d7a18c740 1 -- 172.21.15.86:0/702060344 >> 172.21.15.86:0/702060344 conn(0x557d6c040ba0 msgr2=0x557d6c0428f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:36.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.557+0000 7f2d7a18c740 1 -- 172.21.15.86:0/702060344 shutdown_connections 2024-04-21T08:48:36.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.557+0000 7f2d7a18c740 1 -- 172.21.15.86:0/702060344 wait complete. 2024-04-21T08:48:36.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.558+0000 7f2d7a18c740 20 client.19590 trim_cache size 0 max 16384 2024-04-21T08:48:36.575 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.571+0000 7f2d7a18c740 1 Processor -- start 2024-04-21T08:48:36.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.571+0000 7f2d7a18c740 1 -- start start 2024-04-21T08:48:36.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.571+0000 7f2d7a18c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6c0464d0 0x557d6c05b7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.571+0000 7f2d7a18c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d6c046e80 0x557d6c03fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.571+0000 7f2d7a18c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557d6c04fa00 0x557d6c0677b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.571+0000 7f2d7a18c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2d701af410 con 0x557d6c0464d0 2024-04-21T08:48:36.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.571+0000 7f2d7a18c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2d701b0f50 con 0x557d6c04fa00 2024-04-21T08:48:36.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.571+0000 7f2d7a18c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2d701afae0 con 0x557d6c046e80 2024-04-21T08:48:36.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.571+0000 7f2d75339640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6c0464d0 0x557d6c05b7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.571+0000 7f2d75339640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6c0464d0 0x557d6c05b7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38512/0 (socket says 172.21.15.86:38512) 2024-04-21T08:48:36.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.571+0000 7f2d75339640 1 -- 172.21.15.86:0/4020997017 learned_addr learned my addr 172.21.15.86:0/4020997017 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:36.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.571+0000 7f2d75339640 1 -- 172.21.15.86:0/4020997017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d6c046e80 msgr2=0x557d6c03fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.571+0000 7f2d74b38640 1 --2- 172.21.15.86:0/4020997017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d6c046e80 0x557d6c03fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.571+0000 7f2d75b3a640 1 --2- 172.21.15.86:0/4020997017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557d6c04fa00 0x557d6c0677b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.571+0000 7f2d75339640 1 --2- 172.21.15.86:0/4020997017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d6c046e80 0x557d6c03fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.571+0000 7f2d75339640 1 -- 172.21.15.86:0/4020997017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557d6c04fa00 msgr2=0x557d6c0677b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.571+0000 7f2d75339640 1 --2- 172.21.15.86:0/4020997017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557d6c04fa00 0x557d6c0677b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.571+0000 7f2d75339640 1 -- 172.21.15.86:0/4020997017 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x557d6c04ac90 con 0x557d6c0464d0 2024-04-21T08:48:36.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.571+0000 7f2d75b3a640 1 --2- 172.21.15.86:0/4020997017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557d6c04fa00 0x557d6c0677b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:36.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.571+0000 7f2d75339640 1 --2- 172.21.15.86:0/4020997017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6c0464d0 0x557d6c05b7d0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f2d64009900 tx=0x7f2d64009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9e63b8fa507e15ec server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:36.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.571+0000 7f2d5affd640 1 -- 172.21.15.86:0/4020997017 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d64004460 con 0x557d6c0464d0 2024-04-21T08:48:36.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.571+0000 7f2d5affd640 1 -- 172.21.15.86:0/4020997017 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d6402cc90 con 0x557d6c0464d0 2024-04-21T08:48:36.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.571+0000 7f2d5affd640 1 -- 172.21.15.86:0/4020997017 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d640047b0 con 0x557d6c0464d0 2024-04-21T08:48:36.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.572+0000 7f2d7a18c740 1 -- 172.21.15.86:0/4020997017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6c0464d0 msgr2=0x557d6c05b7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.572+0000 7f2d7a18c740 1 --2- 172.21.15.86:0/4020997017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6c0464d0 0x557d6c05b7d0 secure :-1 s=READY pgs=1792 cs=0 l=1 rev1=1 crypto rx=0x7f2d64009900 tx=0x7f2d64009560 comp rx=0 tx=0).stop 2024-04-21T08:48:36.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.572+0000 7f2d7a18c740 1 -- 172.21.15.86:0/4020997017 shutdown_connections 2024-04-21T08:48:36.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.572+0000 7f2d7a18c740 1 --2- 172.21.15.86:0/4020997017 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557d6c04fa00 0x557d6c0677b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.572+0000 7f2d7a18c740 1 --2- 172.21.15.86:0/4020997017 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d6c046e80 0x557d6c03fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.572+0000 7f2d7a18c740 1 --2- 172.21.15.86:0/4020997017 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6c0464d0 0x557d6c05b7d0 unknown :-1 s=CLOSED pgs=1792 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.572+0000 7f2d7a18c740 1 -- 172.21.15.86:0/4020997017 >> 172.21.15.86:0/4020997017 conn(0x557d6c040ba0 msgr2=0x557d6c03f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:36.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.572+0000 7f2d7a18c740 1 -- 172.21.15.86:0/4020997017 shutdown_connections 2024-04-21T08:48:36.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.572+0000 7f2d7a18c740 1 -- 172.21.15.86:0/4020997017 wait complete. 2024-04-21T08:48:36.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.572+0000 7f2d7a18c740 1 -- start start 2024-04-21T08:48:36.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.572+0000 7f2d7a18c740 1 Processor -- start 2024-04-21T08:48:36.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.573+0000 7f2d7a18c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6c0464d0 0x557d6c053640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.573+0000 7f2d7a18c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557d6c046e80 0x557d6c053b80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.573+0000 7f2d7a18c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d6c04fa00 0x557d6c0540c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.573+0000 7f2d7a18c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2d701b0880 con 0x557d6c0464d0 2024-04-21T08:48:36.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.573+0000 7f2d7a18c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2d701b01b0 con 0x557d6c046e80 2024-04-21T08:48:36.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.573+0000 7f2d7a18c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2d701ad410 con 0x557d6c04fa00 2024-04-21T08:48:36.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.573+0000 7f2d7a18c740 10 client.0 _ll_register_callbacks cb 0x7f2d68014bf0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:36.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.573+0000 7f2d75339640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6c0464d0 0x557d6c053640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.573+0000 7f2d75339640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6c0464d0 0x557d6c053640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38522/0 (socket says 172.21.15.86:38522) 2024-04-21T08:48:36.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.573+0000 7f2d75339640 1 -- 172.21.15.86:0/3190875826 learned_addr learned my addr 172.21.15.86:0/3190875826 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:36.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.573+0000 7f2d75b3a640 1 --2- 172.21.15.86:0/3190875826 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d6c04fa00 0x557d6c0540c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.573+0000 7f2d75339640 1 -- 172.21.15.86:0/3190875826 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d6c04fa00 msgr2=0x557d6c0540c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.573+0000 7f2d75339640 1 --2- 172.21.15.86:0/3190875826 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d6c04fa00 0x557d6c0540c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.573+0000 7f2d75339640 1 -- 172.21.15.86:0/3190875826 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557d6c046e80 msgr2=0x557d6c053b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.573+0000 7f2d75339640 1 --2- 172.21.15.86:0/3190875826 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557d6c046e80 0x557d6c053b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.573+0000 7f2d75339640 1 -- 172.21.15.86:0/3190875826 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x557d6c0c5020 con 0x557d6c0464d0 2024-04-21T08:48:36.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.573+0000 7f2d75339640 1 --2- 172.21.15.86:0/3190875826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6c0464d0 0x557d6c053640 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f2d6404c040 tx=0x7f2d64005640 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea131e467208961e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:36.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.573+0000 7f2d58ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:36.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.573+0000 7f2d58ff9640 1 -- 172.21.15.86:0/3190875826 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2d640037f0 con 0x557d6c0464d0 2024-04-21T08:48:36.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.573+0000 7f2d58ff9640 1 -- 172.21.15.86:0/3190875826 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f2d64003990 con 0x557d6c0464d0 2024-04-21T08:48:36.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.573+0000 7f2d7a18c740 1 -- 172.21.15.86:0/3190875826 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f2d6402cc90 con 0x557d6c0464d0 2024-04-21T08:48:36.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.574+0000 7f2d58ff9640 1 -- 172.21.15.86:0/3190875826 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2d6402cab0 con 0x557d6c0464d0 2024-04-21T08:48:36.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.574+0000 7f2d7a18c740 10 client.19596 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:36.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.574+0000 7f2d7a18c740 1 -- 172.21.15.86:0/3190875826 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x557d6c0c6170 con 0x557d6c0464d0 2024-04-21T08:48:36.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.574+0000 7f2d58ff9640 1 -- 172.21.15.86:0/3190875826 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2d6402cc90 con 0x557d6c0464d0 2024-04-21T08:48:36.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.574+0000 7f2d58ff9640 1 -- 172.21.15.86:0/3190875826 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2d64034be0 con 0x557d6c0464d0 2024-04-21T08:48:36.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.574+0000 7f2d7a18c740 10 client.19596 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:36.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.574+0000 7f2d7a18c740 10 client.19596 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:48:36.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.574+0000 7f2d7a18c740 20 client.19596 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:36.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.574+0000 7f2d7a18c740 1 --2- 172.21.15.86:0/3190875826 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x557d6c0c6850 0x557d6c0c8c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.574+0000 7f2d7a18c740 4 client.19596 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:48:36.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.574+0000 7f2d7a18c740 1 -- 172.21.15.86:0/3190875826 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x557d6bfb8c10 con 0x557d6c0c6850 2024-04-21T08:48:36.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.574+0000 7f2d74b38640 1 --2- 172.21.15.86:0/3190875826 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x557d6c0c6850 0x557d6c0c8c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.575+0000 7f2d74b38640 1 --2- 172.21.15.86:0/3190875826 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x557d6c0c6850 0x557d6c0c8c70 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=7ad7286c9be33cc2 server_cookie=85b01220be2e3391 in_seq=0 out_seq=0 2024-04-21T08:48:36.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.575+0000 7f2d58ff9640 10 client.19596 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:48:36.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.575+0000 7f2d58ff9640 1 -- 172.21.15.86:0/3190875826 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x557d6bfb8c10 con 0x557d6c0c6850 2024-04-21T08:48:36.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.575+0000 7f2d58ff9640 10 client.19596 handle_command_reply: tid=0 2024-04-21T08:48:36.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.575+0000 7f2d7a18c740 1 client.19596 shutdown 2024-04-21T08:48:36.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.575+0000 7f2d7a18c740 1 -- 172.21.15.86:0/3190875826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6c0464d0 msgr2=0x557d6c053640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.575+0000 7f2d7a18c740 1 --2- 172.21.15.86:0/3190875826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6c0464d0 0x557d6c053640 secure :-1 s=READY pgs=1793 cs=0 l=1 rev1=1 crypto rx=0x7f2d6404c040 tx=0x7f2d64005640 comp rx=0 tx=0).stop 2024-04-21T08:48:36.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.575+0000 7f2d7a18c740 1 -- 172.21.15.86:0/3190875826 shutdown_connections 2024-04-21T08:48:36.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.575+0000 7f2d7a18c740 1 --2- 172.21.15.86:0/3190875826 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x557d6c04fa00 0x557d6c0540c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.575+0000 7f2d7a18c740 1 --2- 172.21.15.86:0/3190875826 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x557d6c046e80 0x557d6c053b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.575+0000 7f2d7a18c740 1 --2- 172.21.15.86:0/3190875826 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x557d6c0c6850 0x557d6c0c8c70 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.575+0000 7f2d7a18c740 1 --2- 172.21.15.86:0/3190875826 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x557d6c0464d0 0x557d6c053640 unknown :-1 s=CLOSED pgs=1793 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.575+0000 7f2d58ff9640 0 client.19596 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:48:36.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.575+0000 7f2d7a18c740 1 -- 172.21.15.86:0/3190875826 >> 172.21.15.86:0/3190875826 conn(0x557d6c040ba0 msgr2=0x557d6c03fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:36.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.575+0000 7f2d7a18c740 1 -- 172.21.15.86:0/3190875826 shutdown_connections 2024-04-21T08:48:36.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.575+0000 7f2d7a18c740 1 -- 172.21.15.86:0/3190875826 wait complete. 2024-04-21T08:48:36.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.575+0000 7f2d7a18c740 20 client.19596 trim_cache size 0 max 16384 2024-04-21T08:48:36.582 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:48:36.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.576+0000 7f2d7759d640 1 -- 172.21.15.86:0/3153689138 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2d5c0a3da0 msgr2=0x7f2d5c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.576+0000 7f2d7759d640 1 --2- 172.21.15.86:0/3153689138 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2d5c0a3da0 0x7f2d5c0a6260 secure :-1 s=READY pgs=295 cs=0 l=1 rev1=1 crypto rx=0x7f2d68011c50 tx=0x7f2d68016000 comp rx=0 tx=0).stop 2024-04-21T08:48:36.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.576+0000 7f2d7759d640 1 -- 172.21.15.86:0/3153689138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d70106170 msgr2=0x7f2d70199ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.576+0000 7f2d7759d640 1 --2- 172.21.15.86:0/3153689138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d70106170 0x7f2d70199ac0 secure :-1 s=READY pgs=1790 cs=0 l=1 rev1=1 crypto rx=0x7f2d6c00c970 tx=0x7f2d6c00ce30 comp rx=0 tx=0).stop 2024-04-21T08:48:36.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.576+0000 7f2d7759d640 1 -- 172.21.15.86:0/3153689138 shutdown_connections 2024-04-21T08:48:36.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.576+0000 7f2d7759d640 1 --2- 172.21.15.86:0/3153689138 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2d5c0a3da0 0x7f2d5c0a6260 unknown :-1 s=CLOSED pgs=295 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.576+0000 7f2d7759d640 1 --2- 172.21.15.86:0/3153689138 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2d7019a050 0x7f2d701a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.576+0000 7f2d7759d640 1 --2- 172.21.15.86:0/3153689138 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2d70106170 0x7f2d70199ac0 unknown :-1 s=CLOSED pgs=1790 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.576+0000 7f2d7759d640 1 --2- 172.21.15.86:0/3153689138 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2d701010a0 0x7f2d70199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.576+0000 7f2d7759d640 1 -- 172.21.15.86:0/3153689138 >> 172.21.15.86:0/3153689138 conn(0x7f2d700771d0 msgr2=0x7f2d700fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:36.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.576+0000 7f2d7759d640 1 -- 172.21.15.86:0/3153689138 shutdown_connections 2024-04-21T08:48:36.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.576+0000 7f2d7759d640 1 -- 172.21.15.86:0/3153689138 wait complete. 2024-04-21T08:48:36.586 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:48:36.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.687+0000 7f604dae4640 1 -- 172.21.15.86:0/3838157628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6048101090 msgr2=0x7f6048101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.687+0000 7f604dae4640 1 --2- 172.21.15.86:0/3838157628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6048101090 0x7f6048101470 secure :-1 s=READY pgs=1794 cs=0 l=1 rev1=1 crypto rx=0x7f603c009900 tx=0x7f603c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:36.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.687+0000 7f604dae4640 1 -- 172.21.15.86:0/3838157628 shutdown_connections 2024-04-21T08:48:36.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.687+0000 7f604dae4640 1 --2- 172.21.15.86:0/3838157628 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6048106160 0x7f604810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.687+0000 7f604dae4640 1 --2- 172.21.15.86:0/3838157628 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6048101a40 0x7f6048105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.687+0000 7f604dae4640 1 --2- 172.21.15.86:0/3838157628 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6048101090 0x7f6048101470 unknown :-1 s=CLOSED pgs=1794 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.687+0000 7f604dae4640 1 -- 172.21.15.86:0/3838157628 >> 172.21.15.86:0/3838157628 conn(0x7f6048077f50 msgr2=0x7f6048078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:36.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.687+0000 7f604dae4640 1 -- 172.21.15.86:0/3838157628 shutdown_connections 2024-04-21T08:48:36.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.687+0000 7f604dae4640 1 -- 172.21.15.86:0/3838157628 wait complete. 2024-04-21T08:48:36.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.687+0000 7f604dae4640 1 Processor -- start 2024-04-21T08:48:36.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.688+0000 7f604dae4640 1 -- start start 2024-04-21T08:48:36.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.688+0000 7f604dae4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6048101090 0x7f60481975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.688+0000 7f604dae4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6048106160 0x7f6048197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.688+0000 7f604dae4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f604819be90 0x7f604819c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.688+0000 7f604dae4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60481110d0 con 0x7f6048106160 2024-04-21T08:48:36.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.688+0000 7f604dae4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6048110f50 con 0x7f6048101090 2024-04-21T08:48:36.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.688+0000 7f604dae4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6048111250 con 0x7f604819be90 2024-04-21T08:48:36.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.688+0000 7f6046ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6048106160 0x7f6048197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.688+0000 7f6046ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6048106160 0x7f6048197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38540/0 (socket says 172.21.15.86:38540) 2024-04-21T08:48:36.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.688+0000 7f6046ffd640 1 -- 172.21.15.86:0/2400549998 learned_addr learned my addr 172.21.15.86:0/2400549998 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:36.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.688+0000 7f60477fe640 1 --2- 172.21.15.86:0/2400549998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6048101090 0x7f60481975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.688+0000 7f6047fff640 1 --2- 172.21.15.86:0/2400549998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f604819be90 0x7f604819c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.688+0000 7f6046ffd640 1 -- 172.21.15.86:0/2400549998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f604819be90 msgr2=0x7f604819c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.688+0000 7f6046ffd640 1 --2- 172.21.15.86:0/2400549998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f604819be90 0x7f604819c340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.688+0000 7f6046ffd640 1 -- 172.21.15.86:0/2400549998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6048101090 msgr2=0x7f60481975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.688+0000 7f6046ffd640 1 --2- 172.21.15.86:0/2400549998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6048101090 0x7f60481975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.688+0000 7f6046ffd640 1 -- 172.21.15.86:0/2400549998 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f604819c980 con 0x7f6048106160 2024-04-21T08:48:36.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.688+0000 7f6046ffd640 1 --2- 172.21.15.86:0/2400549998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6048106160 0x7f6048197b10 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f603800c970 tx=0x7f603800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:36.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.688+0000 7f6044ff9640 1 -- 172.21.15.86:0/2400549998 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6038014070 con 0x7f6048106160 2024-04-21T08:48:36.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.688+0000 7f604dae4640 1 -- 172.21.15.86:0/2400549998 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f604819cc70 con 0x7f6048106160 2024-04-21T08:48:36.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.688+0000 7f6044ff9640 1 -- 172.21.15.86:0/2400549998 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6038004440 con 0x7f6048106160 2024-04-21T08:48:36.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.689+0000 7f6044ff9640 1 -- 172.21.15.86:0/2400549998 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6038003e50 con 0x7f6048106160 2024-04-21T08:48:36.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.689+0000 7f604dae4640 1 -- 172.21.15.86:0/2400549998 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f60481a4460 con 0x7f6048106160 2024-04-21T08:48:36.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.691+0000 7f60507b1740 1 Processor -- start 2024-04-21T08:48:36.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.691+0000 7f6044ff9640 1 -- 172.21.15.86:0/2400549998 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f6038017070 con 0x7f6048106160 2024-04-21T08:48:36.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.691+0000 7f60507b1740 1 -- start start 2024-04-21T08:48:36.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.692+0000 7f60507b1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55adf687f370 0x55adf687f750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.692+0000 7f60507b1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adf687fd20 0x55adf688a640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.692+0000 7f60507b1740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55adf688ac10 0x55adf688d020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.692+0000 7f60507b1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55adf67e41f0 con 0x55adf687fd20 2024-04-21T08:48:36.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.692+0000 7f60507b1740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55adf67814b0 con 0x55adf688ac10 2024-04-21T08:48:36.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.692+0000 7f60507b1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55adf67e01b0 con 0x55adf687f370 2024-04-21T08:48:36.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.692+0000 7f60477fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adf687fd20 0x55adf688a640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.692+0000 7f60477fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adf687fd20 0x55adf688a640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38542/0 (socket says 172.21.15.86:38542) 2024-04-21T08:48:36.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.692+0000 7f60477fe640 1 -- 172.21.15.86:0/432345367 learned_addr learned my addr 172.21.15.86:0/432345367 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:36.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.692+0000 7f6047fff640 1 --2- 172.21.15.86:0/432345367 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55adf687f370 0x55adf687f750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.692+0000 7f60477fe640 1 -- 172.21.15.86:0/432345367 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55adf687f370 msgr2=0x55adf687f750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.692+0000 7f60477fe640 1 --2- 172.21.15.86:0/432345367 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55adf687f370 0x55adf687f750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.692+0000 7f6046ffd640 1 --2- 172.21.15.86:0/432345367 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55adf688ac10 0x55adf688d020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.692+0000 7f60477fe640 1 -- 172.21.15.86:0/432345367 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55adf688ac10 msgr2=0x55adf688d020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.692+0000 7f60477fe640 1 --2- 172.21.15.86:0/432345367 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55adf688ac10 0x55adf688d020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.692+0000 7f6044ff9640 1 --2- 172.21.15.86:0/2400549998 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f60300a3ec0 0x7f60300a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.692+0000 7f60477fe640 1 -- 172.21.15.86:0/432345367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adf67731b0 con 0x55adf687fd20 2024-04-21T08:48:36.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.692+0000 7f6047fff640 1 --2- 172.21.15.86:0/2400549998 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f60300a3ec0 0x7f60300a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.692+0000 7f60477fe640 1 --2- 172.21.15.86:0/432345367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adf687fd20 0x55adf688a640 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f603c009900 tx=0x7f603c0040f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=46069aaa9f0f4336 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:36.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.692+0000 7f6044ff9640 1 -- 172.21.15.86:0/2400549998 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f60380d0a50 con 0x7f6048106160 2024-04-21T08:48:36.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.692+0000 7f602dffb640 1 -- 172.21.15.86:0/432345367 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603c045020 con 0x55adf687fd20 2024-04-21T08:48:36.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.693+0000 7f602dffb640 1 -- 172.21.15.86:0/432345367 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f603c02cc90 con 0x55adf687fd20 2024-04-21T08:48:36.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.693+0000 7f602dffb640 1 -- 172.21.15.86:0/432345367 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603c0046c0 con 0x55adf687fd20 2024-04-21T08:48:36.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.693+0000 7f60507b1740 1 -- 172.21.15.86:0/432345367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adf687fd20 msgr2=0x55adf688a640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.693+0000 7f6047fff640 1 --2- 172.21.15.86:0/2400549998 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f60300a3ec0 0x7f60300a6380 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f604000bd70 tx=0x7f604000b300 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:36.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.693+0000 7f60507b1740 1 --2- 172.21.15.86:0/432345367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adf687fd20 0x55adf688a640 secure :-1 s=READY pgs=1796 cs=0 l=1 rev1=1 crypto rx=0x7f603c009900 tx=0x7f603c0040f0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.693+0000 7f60507b1740 1 -- 172.21.15.86:0/432345367 shutdown_connections 2024-04-21T08:48:36.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.693+0000 7f60507b1740 1 --2- 172.21.15.86:0/432345367 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55adf688ac10 0x55adf688d020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.693+0000 7f60507b1740 1 --2- 172.21.15.86:0/432345367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adf687fd20 0x55adf688a640 unknown :-1 s=CLOSED pgs=1796 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.693+0000 7f60507b1740 1 --2- 172.21.15.86:0/432345367 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55adf687f370 0x55adf687f750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.693+0000 7f60507b1740 1 -- 172.21.15.86:0/432345367 >> 172.21.15.86:0/432345367 conn(0x55adf6879ba0 msgr2=0x55adf6879f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:36.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.693+0000 7f60507b1740 1 -- 172.21.15.86:0/432345367 shutdown_connections 2024-04-21T08:48:36.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.693+0000 7f60507b1740 1 -- 172.21.15.86:0/432345367 wait complete. 2024-04-21T08:48:36.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.694+0000 7f60507b1740 1 -- start start 2024-04-21T08:48:36.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.694+0000 7f60507b1740 1 Processor -- start 2024-04-21T08:48:36.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.694+0000 7f60507b1740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55adf687f370 0x55adf688a430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.694+0000 7f60507b1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55adf687fd20 0x55adf68877c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.694+0000 7f60507b1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adf688ac10 0x55adf6887d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.694+0000 7f60507b1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55adf688dae0 con 0x55adf688ac10 2024-04-21T08:48:36.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.694+0000 7f60507b1740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55adf679d500 con 0x55adf687f370 2024-04-21T08:48:36.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.694+0000 7f60507b1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55adf688dc60 con 0x55adf687fd20 2024-04-21T08:48:36.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.694+0000 7f60507b1740 10 client.0 _ll_register_callbacks cb 0x55adf676bb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:36.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.694+0000 7f6047fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adf688ac10 0x55adf6887d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.694+0000 7f6047fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adf688ac10 0x55adf6887d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38546/0 (socket says 172.21.15.86:38546) 2024-04-21T08:48:36.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.694+0000 7f6047fff640 1 -- 172.21.15.86:0/3909180974 learned_addr learned my addr 172.21.15.86:0/3909180974 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:36.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.694+0000 7f60477fe640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55adf687f370 0x55adf688a430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.694+0000 7f6046ffd640 1 --2- 172.21.15.86:0/3909180974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55adf687fd20 0x55adf68877c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.694+0000 7f6047fff640 1 -- 172.21.15.86:0/3909180974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55adf687fd20 msgr2=0x55adf68877c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.694+0000 7f6047fff640 1 --2- 172.21.15.86:0/3909180974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55adf687fd20 0x55adf68877c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.694+0000 7f6047fff640 1 -- 172.21.15.86:0/3909180974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55adf687f370 msgr2=0x55adf688a430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.694+0000 7f6047fff640 1 --2- 172.21.15.86:0/3909180974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55adf687f370 0x55adf688a430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.694+0000 7f6047fff640 1 -- 172.21.15.86:0/3909180974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adf64f63f0 con 0x55adf688ac10 2024-04-21T08:48:36.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.694+0000 7f60477fe640 1 --2- 172.21.15.86:0/3909180974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55adf687f370 0x55adf688a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:36.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.695+0000 7f6047fff640 1 --2- 172.21.15.86:0/3909180974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adf688ac10 0x55adf6887d00 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f6040011e40 tx=0x7f6040004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=60183bbbad93e183 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:36.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.695+0000 7f6026ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:36.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.695+0000 7f6026ffd640 1 -- 172.21.15.86:0/3909180974 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f604001e420 con 0x55adf688ac10 2024-04-21T08:48:36.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.695+0000 7f6026ffd640 1 -- 172.21.15.86:0/3909180974 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f604001e5c0 con 0x55adf688ac10 2024-04-21T08:48:36.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.695+0000 7f60507b1740 1 -- 172.21.15.86:0/3909180974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f603c02cc90 con 0x55adf688ac10 2024-04-21T08:48:36.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.695+0000 7f6026ffd640 1 -- 172.21.15.86:0/3909180974 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f604001e7c0 con 0x55adf688ac10 2024-04-21T08:48:36.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.695+0000 7f60507b1740 10 client.19608 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:36.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.695+0000 7f60507b1740 1 -- 172.21.15.86:0/3909180974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55adf65c07d0 con 0x55adf688ac10 2024-04-21T08:48:36.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.695+0000 7f6026ffd640 1 -- 172.21.15.86:0/3909180974 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f603c02cc90 con 0x55adf688ac10 2024-04-21T08:48:36.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.695+0000 7f6026ffd640 1 -- 172.21.15.86:0/3909180974 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f6040026af0 con 0x55adf688ac10 2024-04-21T08:48:36.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.695+0000 7f60507b1740 10 client.19608 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:36.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.695+0000 7f60507b1740 10 client.19608 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:36.705 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.695+0000 7f60507b1740 20 client.19608 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:36.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.695+0000 7f60507b1740 1 --2- 172.21.15.86:0/3909180974 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55adf689ada0 0x55adf689d210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.695+0000 7f60507b1740 4 client.19608 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:36.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.695+0000 7f60507b1740 1 -- 172.21.15.86:0/3909180974 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55adf6783930 con 0x55adf689ada0 2024-04-21T08:48:36.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.696+0000 7f60477fe640 1 --2- 172.21.15.86:0/3909180974 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55adf689ada0 0x55adf689d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.696+0000 7f60477fe640 1 --2- 172.21.15.86:0/3909180974 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55adf689ada0 0x55adf689d210 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a23fed949d105e0b server_cookie=ba7b0d765f78543a in_seq=0 out_seq=0 2024-04-21T08:48:36.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.696+0000 7f6026ffd640 10 client.19608 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:36.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.697+0000 7f6026ffd640 1 -- 172.21.15.86:0/3909180974 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55adf6783930 con 0x55adf689ada0 2024-04-21T08:48:36.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.697+0000 7f6026ffd640 10 client.19608 handle_command_reply: tid=0 2024-04-21T08:48:36.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.697+0000 7f60507b1740 1 client.19608 shutdown 2024-04-21T08:48:36.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.697+0000 7f60507b1740 1 -- 172.21.15.86:0/3909180974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adf688ac10 msgr2=0x55adf6887d00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.697+0000 7f60507b1740 1 --2- 172.21.15.86:0/3909180974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adf688ac10 0x55adf6887d00 secure :-1 s=READY pgs=1797 cs=0 l=1 rev1=1 crypto rx=0x7f6040011e40 tx=0x7f6040004a40 comp rx=0 tx=0).stop 2024-04-21T08:48:36.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.697+0000 7f60507b1740 1 -- 172.21.15.86:0/3909180974 shutdown_connections 2024-04-21T08:48:36.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.697+0000 7f60507b1740 1 --2- 172.21.15.86:0/3909180974 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55adf689ada0 0x55adf689d210 crc :-1 s=READY pgs=287 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.697+0000 7f60507b1740 1 --2- 172.21.15.86:0/3909180974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adf688ac10 0x55adf6887d00 unknown :-1 s=CLOSED pgs=1797 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.697+0000 7f60507b1740 1 --2- 172.21.15.86:0/3909180974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55adf687fd20 0x55adf68877c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.697+0000 7f6026ffd640 0 client.19608 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:36.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.697+0000 7f60507b1740 1 --2- 172.21.15.86:0/3909180974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55adf687f370 0x55adf688a430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.697+0000 7f60507b1740 1 -- 172.21.15.86:0/3909180974 >> 172.21.15.86:0/3909180974 conn(0x55adf6879ba0 msgr2=0x55adf687b8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:36.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.697+0000 7f60507b1740 1 -- 172.21.15.86:0/3909180974 shutdown_connections 2024-04-21T08:48:36.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.697+0000 7f60507b1740 1 -- 172.21.15.86:0/3909180974 wait complete. 2024-04-21T08:48:36.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.697+0000 7f60507b1740 20 client.19608 trim_cache size 0 max 16384 2024-04-21T08:48:36.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.710+0000 7f60507b1740 1 Processor -- start 2024-04-21T08:48:36.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.710+0000 7f60507b1740 1 -- start start 2024-04-21T08:48:36.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.710+0000 7f60507b1740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55adf687f370 0x55adf6896a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.710+0000 7f60507b1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adf687fd20 0x55adf6886920 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.710+0000 7f60507b1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55adf688ac10 0x55adf6882cf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.710+0000 7f60507b1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60481ad040 con 0x55adf687fd20 2024-04-21T08:48:36.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.710+0000 7f60507b1740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f60481aeb80 con 0x55adf687f370 2024-04-21T08:48:36.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.710+0000 7f60507b1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f60481ad710 con 0x55adf688ac10 2024-04-21T08:48:36.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.710+0000 7f6046ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adf687fd20 0x55adf6886920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.710+0000 7f6046ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adf687fd20 0x55adf6886920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38548/0 (socket says 172.21.15.86:38548) 2024-04-21T08:48:36.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.710+0000 7f6046ffd640 1 -- 172.21.15.86:0/715372403 learned_addr learned my addr 172.21.15.86:0/715372403 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:36.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.710+0000 7f60477fe640 1 --2- 172.21.15.86:0/715372403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55adf687f370 0x55adf6896a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.710+0000 7f6046ffd640 1 -- 172.21.15.86:0/715372403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55adf688ac10 msgr2=0x55adf6882cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.710+0000 7f6046ffd640 1 --2- 172.21.15.86:0/715372403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55adf688ac10 0x55adf6882cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.710+0000 7f6046ffd640 1 -- 172.21.15.86:0/715372403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55adf687f370 msgr2=0x55adf6896a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.710+0000 7f6046ffd640 1 --2- 172.21.15.86:0/715372403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55adf687f370 0x55adf6896a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.710+0000 7f6046ffd640 1 -- 172.21.15.86:0/715372403 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55adf6888300 con 0x55adf687fd20 2024-04-21T08:48:36.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.711+0000 7f6046ffd640 1 --2- 172.21.15.86:0/715372403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adf687fd20 0x55adf6886920 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f60380ce3d0 tx=0x7f603809c160 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8c46b428baa328a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:36.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.711+0000 7f602d7fa640 1 -- 172.21.15.86:0/715372403 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603809c620 con 0x55adf687fd20 2024-04-21T08:48:36.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.711+0000 7f602d7fa640 1 -- 172.21.15.86:0/715372403 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60380a09d0 con 0x55adf687fd20 2024-04-21T08:48:36.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.711+0000 7f602d7fa640 1 -- 172.21.15.86:0/715372403 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f60380ce1c0 con 0x55adf687fd20 2024-04-21T08:48:36.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.711+0000 7f60507b1740 1 -- 172.21.15.86:0/715372403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adf687fd20 msgr2=0x55adf6886920 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.711+0000 7f60507b1740 1 --2- 172.21.15.86:0/715372403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adf687fd20 0x55adf6886920 secure :-1 s=READY pgs=1798 cs=0 l=1 rev1=1 crypto rx=0x7f60380ce3d0 tx=0x7f603809c160 comp rx=0 tx=0).stop 2024-04-21T08:48:36.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.711+0000 7f60507b1740 1 -- 172.21.15.86:0/715372403 shutdown_connections 2024-04-21T08:48:36.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.711+0000 7f60507b1740 1 --2- 172.21.15.86:0/715372403 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55adf688ac10 0x55adf6882cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.711+0000 7f60507b1740 1 --2- 172.21.15.86:0/715372403 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adf687fd20 0x55adf6886920 unknown :-1 s=CLOSED pgs=1798 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.711+0000 7f60507b1740 1 --2- 172.21.15.86:0/715372403 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55adf687f370 0x55adf6896a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.711+0000 7f60507b1740 1 -- 172.21.15.86:0/715372403 >> 172.21.15.86:0/715372403 conn(0x55adf6879ba0 msgr2=0x55adf68825e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:36.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.711+0000 7f60507b1740 1 -- 172.21.15.86:0/715372403 shutdown_connections 2024-04-21T08:48:36.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.711+0000 7f60507b1740 1 -- 172.21.15.86:0/715372403 wait complete. 2024-04-21T08:48:36.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.712+0000 7f60507b1740 1 -- start start 2024-04-21T08:48:36.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.712+0000 7f60507b1740 1 Processor -- start 2024-04-21T08:48:36.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.712+0000 7f60507b1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adf687f370 0x55adf6893180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.712+0000 7f60507b1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55adf687fd20 0x55adf68840d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.712+0000 7f60507b1740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55adf688ac10 0x55adf6884610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.712+0000 7f60507b1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f60481ae4b0 con 0x55adf687f370 2024-04-21T08:48:36.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.712+0000 7f60507b1740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f60481adde0 con 0x55adf688ac10 2024-04-21T08:48:36.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.712+0000 7f60507b1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f60481ab040 con 0x55adf687fd20 2024-04-21T08:48:36.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.712+0000 7f60507b1740 10 client.0 _ll_register_callbacks cb 0x7f60400117e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:36.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.712+0000 7f60477fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adf687f370 0x55adf6893180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.712+0000 7f60477fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adf687f370 0x55adf6893180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38550/0 (socket says 172.21.15.86:38550) 2024-04-21T08:48:36.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.712+0000 7f60477fe640 1 -- 172.21.15.86:0/1126609057 learned_addr learned my addr 172.21.15.86:0/1126609057 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:36.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.712+0000 7f6047fff640 1 --2- 172.21.15.86:0/1126609057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55adf688ac10 0x55adf6884610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.712+0000 7f6046ffd640 1 --2- 172.21.15.86:0/1126609057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55adf687fd20 0x55adf68840d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.712+0000 7f60477fe640 1 -- 172.21.15.86:0/1126609057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55adf687fd20 msgr2=0x55adf68840d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.712+0000 7f60477fe640 1 --2- 172.21.15.86:0/1126609057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55adf687fd20 0x55adf68840d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.712+0000 7f60477fe640 1 -- 172.21.15.86:0/1126609057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55adf688ac10 msgr2=0x55adf6884610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:36.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.712+0000 7f60477fe640 1 --2- 172.21.15.86:0/1126609057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55adf688ac10 0x55adf6884610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.712+0000 7f60477fe640 1 -- 172.21.15.86:0/1126609057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55adf6884c80 con 0x55adf687f370 2024-04-21T08:48:36.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.712+0000 7f6047fff640 1 --2- 172.21.15.86:0/1126609057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55adf688ac10 0x55adf6884610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:36.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.712+0000 7f6046ffd640 1 --2- 172.21.15.86:0/1126609057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55adf687fd20 0x55adf68840d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:36.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.712+0000 7f60477fe640 1 --2- 172.21.15.86:0/1126609057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adf687f370 0x55adf6893180 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f603c005ec0 tx=0x7f603c0053d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=75e4131ac9b617fa server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:36.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.712+0000 7f6026ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:36.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.713+0000 7f6026ffd640 1 -- 172.21.15.86:0/1126609057 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f603c052070 con 0x55adf687f370 2024-04-21T08:48:36.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.713+0000 7f6026ffd640 1 -- 172.21.15.86:0/1126609057 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f603c054310 con 0x55adf687f370 2024-04-21T08:48:36.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.713+0000 7f60507b1740 1 -- 172.21.15.86:0/1126609057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f60380a09d0 con 0x55adf687f370 2024-04-21T08:48:36.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.713+0000 7f6026ffd640 1 -- 172.21.15.86:0/1126609057 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f603c045040 con 0x55adf687f370 2024-04-21T08:48:36.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.713+0000 7f60507b1740 10 client.19614 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:36.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.713+0000 7f60507b1740 1 -- 172.21.15.86:0/1126609057 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55adf69011e0 con 0x55adf687f370 2024-04-21T08:48:36.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.713+0000 7f6026ffd640 1 -- 172.21.15.86:0/1126609057 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f60380a09d0 con 0x55adf687f370 2024-04-21T08:48:36.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.713+0000 7f6026ffd640 1 -- 172.21.15.86:0/1126609057 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f603c0035f0 con 0x55adf687f370 2024-04-21T08:48:36.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.713+0000 7f60507b1740 10 client.19614 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:36.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.713+0000 7f60507b1740 10 client.19614 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:36.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.713+0000 7f60507b1740 20 client.19614 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:36.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.713+0000 7f60507b1740 1 --2- 172.21.15.86:0/1126609057 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55adf6901520 0x55adf6903910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:36.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.713+0000 7f60507b1740 4 client.19614 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:48:36.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.713+0000 7f60507b1740 1 -- 172.21.15.86:0/1126609057 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55adf67f1c10 con 0x55adf6901520 2024-04-21T08:48:36.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.713+0000 7f6046ffd640 1 --2- 172.21.15.86:0/1126609057 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55adf6901520 0x55adf6903910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:36.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.714+0000 7f6026ffd640 10 client.19614 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:36.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.714+0000 7f6046ffd640 1 --2- 172.21.15.86:0/1126609057 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55adf6901520 0x55adf6903910 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1e6e9aef6541b899 server_cookie=4477afe5099c5406 in_seq=0 out_seq=0 2024-04-21T08:48:36.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.714+0000 7f6026ffd640 1 -- 172.21.15.86:0/1126609057 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55adf67f1c10 con 0x55adf6901520 2024-04-21T08:48:36.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.714+0000 7f6026ffd640 10 client.19614 handle_command_reply: tid=0 2024-04-21T08:48:36.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.714+0000 7f60507b1740 1 client.19614 shutdown 2024-04-21T08:48:36.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.714+0000 7f60507b1740 1 -- 172.21.15.86:0/1126609057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adf687f370 msgr2=0x55adf6893180 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.714+0000 7f60507b1740 1 --2- 172.21.15.86:0/1126609057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adf687f370 0x55adf6893180 secure :-1 s=READY pgs=1799 cs=0 l=1 rev1=1 crypto rx=0x7f603c005ec0 tx=0x7f603c0053d0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.714+0000 7f60507b1740 1 -- 172.21.15.86:0/1126609057 shutdown_connections 2024-04-21T08:48:36.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.714+0000 7f60507b1740 1 --2- 172.21.15.86:0/1126609057 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55adf6901520 0x55adf6903910 crc :-1 s=READY pgs=288 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.714+0000 7f60507b1740 1 --2- 172.21.15.86:0/1126609057 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55adf688ac10 0x55adf6884610 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.714+0000 7f60507b1740 1 --2- 172.21.15.86:0/1126609057 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55adf687fd20 0x55adf68840d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.714+0000 7f6026ffd640 0 client.19614 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:36.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.714+0000 7f60507b1740 1 --2- 172.21.15.86:0/1126609057 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55adf687f370 0x55adf6893180 unknown :-1 s=CLOSED pgs=1799 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.714+0000 7f60507b1740 1 -- 172.21.15.86:0/1126609057 >> 172.21.15.86:0/1126609057 conn(0x55adf6879ba0 msgr2=0x55adf6886090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:36.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.714+0000 7f60507b1740 1 -- 172.21.15.86:0/1126609057 shutdown_connections 2024-04-21T08:48:36.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.714+0000 7f60507b1740 1 -- 172.21.15.86:0/1126609057 wait complete. 2024-04-21T08:48:36.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.714+0000 7f60507b1740 20 client.19614 trim_cache size 0 max 16384 2024-04-21T08:48:36.721 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:48:36.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.715+0000 7f604dae4640 1 -- 172.21.15.86:0/2400549998 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f60300a3ec0 msgr2=0x7f60300a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.715+0000 7f604dae4640 1 --2- 172.21.15.86:0/2400549998 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f60300a3ec0 0x7f60300a6380 secure :-1 s=READY pgs=296 cs=0 l=1 rev1=1 crypto rx=0x7f604000bd70 tx=0x7f604000b300 comp rx=0 tx=0).stop 2024-04-21T08:48:36.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.715+0000 7f604dae4640 1 -- 172.21.15.86:0/2400549998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6048106160 msgr2=0x7f6048197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:36.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.715+0000 7f604dae4640 1 --2- 172.21.15.86:0/2400549998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6048106160 0x7f6048197b10 secure :-1 s=READY pgs=1795 cs=0 l=1 rev1=1 crypto rx=0x7f603800c970 tx=0x7f603800ce30 comp rx=0 tx=0).stop 2024-04-21T08:48:36.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.715+0000 7f604dae4640 1 -- 172.21.15.86:0/2400549998 shutdown_connections 2024-04-21T08:48:36.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.715+0000 7f604dae4640 1 --2- 172.21.15.86:0/2400549998 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f60300a3ec0 0x7f60300a6380 unknown :-1 s=CLOSED pgs=296 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.715+0000 7f604dae4640 1 --2- 172.21.15.86:0/2400549998 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f604819be90 0x7f604819c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.715+0000 7f604dae4640 1 --2- 172.21.15.86:0/2400549998 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6048106160 0x7f6048197b10 unknown :-1 s=CLOSED pgs=1795 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.715+0000 7f604dae4640 1 --2- 172.21.15.86:0/2400549998 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f6048101090 0x7f60481975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:36.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.715+0000 7f604dae4640 1 -- 172.21.15.86:0/2400549998 >> 172.21.15.86:0/2400549998 conn(0x7f6048077f50 msgr2=0x7f60480fefb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:36.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.715+0000 7f604dae4640 1 -- 172.21.15.86:0/2400549998 shutdown_connections 2024-04-21T08:48:36.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:36.715+0000 7f604dae4640 1 -- 172.21.15.86:0/2400549998 wait complete. 2024-04-21T08:48:37.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:36 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2142491191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:48:37.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:36 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2142491191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:48:37.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:36 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2142491191' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:48:37.725 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:48:37.726 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:48:37.726 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:48:37.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.841+0000 7fa837fef640 1 -- 172.21.15.86:0/2174823594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa830102b40 msgr2=0x7fa830106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:37.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.841+0000 7fa837fef640 1 --2- 172.21.15.86:0/2174823594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa830102b40 0x7fa830106b80 secure :-1 s=READY pgs=1800 cs=0 l=1 rev1=1 crypto rx=0x7fa82c009900 tx=0x7fa82c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:37.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.841+0000 7fa837fef640 1 -- 172.21.15.86:0/2174823594 shutdown_connections 2024-04-21T08:48:37.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.841+0000 7fa837fef640 1 --2- 172.21.15.86:0/2174823594 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa830107260 0x7fa83010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:37.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.841+0000 7fa837fef640 1 --2- 172.21.15.86:0/2174823594 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa830102b40 0x7fa830106b80 unknown :-1 s=CLOSED pgs=1800 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:37.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.841+0000 7fa837fef640 1 --2- 172.21.15.86:0/2174823594 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa830102190 0x7fa830102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:37.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.841+0000 7fa837fef640 1 -- 172.21.15.86:0/2174823594 >> 172.21.15.86:0/2174823594 conn(0x7fa8300793a0 msgr2=0x7fa8300797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:37.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.841+0000 7fa837fef640 1 -- 172.21.15.86:0/2174823594 shutdown_connections 2024-04-21T08:48:37.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.841+0000 7fa837fef640 1 -- 172.21.15.86:0/2174823594 wait complete. 2024-04-21T08:48:37.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.842+0000 7fa837fef640 1 Processor -- start 2024-04-21T08:48:37.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.842+0000 7fa837fef640 1 -- start start 2024-04-21T08:48:37.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.842+0000 7fa837fef640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa830102190 0x7fa8301986e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:37.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.842+0000 7fa837fef640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa830107260 0x7fa830198c20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:37.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.842+0000 7fa837fef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa83019cfa0 0x7fa83019d450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:37.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.842+0000 7fa837fef640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa830112180 con 0x7fa83019cfa0 2024-04-21T08:48:37.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.842+0000 7fa837fef640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa830112000 con 0x7fa830102190 2024-04-21T08:48:37.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.842+0000 7fa837fef640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa830112300 con 0x7fa830107260 2024-04-21T08:48:37.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.842+0000 7fa83658c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa83019cfa0 0x7fa83019d450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:37.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.842+0000 7fa83658c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa83019cfa0 0x7fa83019d450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38570/0 (socket says 172.21.15.86:38570) 2024-04-21T08:48:37.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.842+0000 7fa83658c640 1 -- 172.21.15.86:0/322276286 learned_addr learned my addr 172.21.15.86:0/322276286 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:37.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.842+0000 7fa83558a640 1 --2- 172.21.15.86:0/322276286 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa830107260 0x7fa830198c20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:37.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.842+0000 7fa835d8b640 1 --2- 172.21.15.86:0/322276286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa830102190 0x7fa8301986e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:37.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.842+0000 7fa83658c640 1 -- 172.21.15.86:0/322276286 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa830107260 msgr2=0x7fa830198c20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:37.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.842+0000 7fa83658c640 1 --2- 172.21.15.86:0/322276286 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa830107260 0x7fa830198c20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:37.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.842+0000 7fa83658c640 1 -- 172.21.15.86:0/322276286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa830102190 msgr2=0x7fa8301986e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:37.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.842+0000 7fa83658c640 1 --2- 172.21.15.86:0/322276286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa830102190 0x7fa8301986e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:37.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.842+0000 7fa83658c640 1 -- 172.21.15.86:0/322276286 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa83019da90 con 0x7fa83019cfa0 2024-04-21T08:48:37.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.842+0000 7fa83558a640 1 --2- 172.21.15.86:0/322276286 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa830107260 0x7fa830198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:37.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.842+0000 7fa83658c640 1 --2- 172.21.15.86:0/322276286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa83019cfa0 0x7fa83019d450 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fa82800cce0 tx=0x7fa82800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:37.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.843+0000 7fa822ffd640 1 -- 172.21.15.86:0/322276286 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82800f450 con 0x7fa83019cfa0 2024-04-21T08:48:37.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.843+0000 7fa837fef640 1 -- 172.21.15.86:0/322276286 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa83019dd80 con 0x7fa83019cfa0 2024-04-21T08:48:37.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.843+0000 7fa822ffd640 1 -- 172.21.15.86:0/322276286 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa82800faa0 con 0x7fa83019cfa0 2024-04-21T08:48:37.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.843+0000 7fa822ffd640 1 -- 172.21.15.86:0/322276286 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa828018b70 con 0x7fa83019cfa0 2024-04-21T08:48:37.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.843+0000 7fa837fef640 1 -- 172.21.15.86:0/322276286 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa8301a5540 con 0x7fa83019cfa0 2024-04-21T08:48:37.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.845+0000 7fa822ffd640 1 -- 172.21.15.86:0/322276286 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa828018d10 con 0x7fa83019cfa0 2024-04-21T08:48:37.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.845+0000 7fa83abd6740 1 Processor -- start 2024-04-21T08:48:37.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.845+0000 7fa83abd6740 1 -- start start 2024-04-21T08:48:37.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.846+0000 7fa83abd6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563d9f4c8e0 0x5563d9f4ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:37.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.846+0000 7fa83abd6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5563d9f4d290 0x5563d9f55840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:37.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.846+0000 7fa83abd6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5563d9f55e10 0x5563d9f58260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:37.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.846+0000 7fa83abd6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5563d9eb0f00 con 0x5563d9f4c8e0 2024-04-21T08:48:37.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.846+0000 7fa83abd6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5563d9e4e590 con 0x5563d9f4d290 2024-04-21T08:48:37.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.846+0000 7fa83abd6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5563d9eacec0 con 0x5563d9f55e10 2024-04-21T08:48:37.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.846+0000 7fa822ffd640 1 --2- 172.21.15.86:0/322276286 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa81c0a3d20 0x7fa81c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:37.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.846+0000 7fa83558a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563d9f4c8e0 0x5563d9f4ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:37.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.846+0000 7fa83558a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563d9f4c8e0 0x5563d9f4ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38574/0 (socket says 172.21.15.86:38574) 2024-04-21T08:48:37.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.846+0000 7fa83558a640 1 -- 172.21.15.86:0/3399751288 learned_addr learned my addr 172.21.15.86:0/3399751288 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:37.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.846+0000 7fa83558a640 1 --2- 172.21.15.86:0/3399751288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5563d9f55e10 0x5563d9f58260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:37.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.846+0000 7fa835d8b640 1 --2- 172.21.15.86:0/3399751288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5563d9f4d290 0x5563d9f55840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:37.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.846+0000 7fa83658c640 1 --2- 172.21.15.86:0/322276286 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa81c0a3d20 0x7fa81c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:37.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.846+0000 7fa822ffd640 1 -- 172.21.15.86:0/322276286 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7fa8280d0fe0 con 0x7fa83019cfa0 2024-04-21T08:48:37.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.846+0000 7fa83558a640 1 -- 172.21.15.86:0/3399751288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5563d9f55e10 msgr2=0x5563d9f58260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:37.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.846+0000 7fa83558a640 1 --2- 172.21.15.86:0/3399751288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5563d9f55e10 0x5563d9f58260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:37.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.846+0000 7fa83558a640 1 -- 172.21.15.86:0/3399751288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5563d9f4d290 msgr2=0x5563d9f55840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:37.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.846+0000 7fa83558a640 1 --2- 172.21.15.86:0/3399751288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5563d9f4d290 0x5563d9f55840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:37.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.846+0000 7fa83558a640 1 -- 172.21.15.86:0/3399751288 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563d9c8b7e0 con 0x5563d9f4c8e0 2024-04-21T08:48:37.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.846+0000 7fa83558a640 1 --2- 172.21.15.86:0/3399751288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563d9f4c8e0 0x5563d9f4ccc0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fa82c000c00 tx=0x7fa82c0058b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0680009d6183518 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:37.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.847+0000 7fa83658c640 1 --2- 172.21.15.86:0/322276286 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa81c0a3d20 0x7fa81c0a61e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa828098ff0 tx=0x7fa828067630 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:37.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.847+0000 7fa81bfff640 1 -- 172.21.15.86:0/3399751288 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82c02ca90 con 0x5563d9f4c8e0 2024-04-21T08:48:37.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.847+0000 7fa81bfff640 1 -- 172.21.15.86:0/3399751288 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa82c02cc30 con 0x5563d9f4c8e0 2024-04-21T08:48:37.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.847+0000 7fa81bfff640 1 -- 172.21.15.86:0/3399751288 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82c0437e0 con 0x5563d9f4c8e0 2024-04-21T08:48:37.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.847+0000 7fa83abd6740 1 -- 172.21.15.86:0/3399751288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563d9f4c8e0 msgr2=0x5563d9f4ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:37.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.847+0000 7fa83abd6740 1 --2- 172.21.15.86:0/3399751288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563d9f4c8e0 0x5563d9f4ccc0 secure :-1 s=READY pgs=1802 cs=0 l=1 rev1=1 crypto rx=0x7fa82c000c00 tx=0x7fa82c0058b0 comp rx=0 tx=0).stop 2024-04-21T08:48:37.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.847+0000 7fa83abd6740 1 -- 172.21.15.86:0/3399751288 shutdown_connections 2024-04-21T08:48:37.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.847+0000 7fa83abd6740 1 --2- 172.21.15.86:0/3399751288 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5563d9f55e10 0x5563d9f58260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:37.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.847+0000 7fa83abd6740 1 --2- 172.21.15.86:0/3399751288 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5563d9f4d290 0x5563d9f55840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:37.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.847+0000 7fa83abd6740 1 --2- 172.21.15.86:0/3399751288 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563d9f4c8e0 0x5563d9f4ccc0 unknown :-1 s=CLOSED pgs=1802 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:37.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.847+0000 7fa83abd6740 1 -- 172.21.15.86:0/3399751288 >> 172.21.15.86:0/3399751288 conn(0x5563d9f46fb0 msgr2=0x5563d9f47390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:37.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.847+0000 7fa83abd6740 1 -- 172.21.15.86:0/3399751288 shutdown_connections 2024-04-21T08:48:37.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.848+0000 7fa83abd6740 1 -- 172.21.15.86:0/3399751288 wait complete. 2024-04-21T08:48:37.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.848+0000 7fa83abd6740 1 -- start start 2024-04-21T08:48:37.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.848+0000 7fa83abd6740 1 Processor -- start 2024-04-21T08:48:37.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.848+0000 7fa83abd6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563d9f4c8e0 0x5563d9f52e20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:37.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.848+0000 7fa83abd6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5563d9f4d290 0x5563d9f53360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:37.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.848+0000 7fa83abd6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5563d9f55e10 0x5563d9f538a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:37.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.848+0000 7fa83abd6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5563d9f58d20 con 0x5563d9f4c8e0 2024-04-21T08:48:37.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.848+0000 7fa83abd6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5563d9e6a580 con 0x5563d9f4d290 2024-04-21T08:48:37.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.848+0000 7fa83abd6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5563d9f58ea0 con 0x5563d9f55e10 2024-04-21T08:48:37.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.848+0000 7fa83abd6740 10 client.0 _ll_register_callbacks cb 0x5563d9e70650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:37.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.848+0000 7fa83558a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563d9f4c8e0 0x5563d9f52e20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:37.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.848+0000 7fa83558a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563d9f4c8e0 0x5563d9f52e20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38582/0 (socket says 172.21.15.86:38582) 2024-04-21T08:48:37.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.848+0000 7fa83558a640 1 -- 172.21.15.86:0/135252701 learned_addr learned my addr 172.21.15.86:0/135252701 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:37.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.849+0000 7fa83558a640 1 --2- 172.21.15.86:0/135252701 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5563d9f55e10 0x5563d9f538a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:37.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.849+0000 7fa835d8b640 1 --2- 172.21.15.86:0/135252701 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5563d9f4d290 0x5563d9f53360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:37.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.849+0000 7fa83558a640 1 -- 172.21.15.86:0/135252701 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5563d9f55e10 msgr2=0x5563d9f538a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:37.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.849+0000 7fa83558a640 1 --2- 172.21.15.86:0/135252701 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5563d9f55e10 0x5563d9f538a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:37.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.849+0000 7fa83558a640 1 -- 172.21.15.86:0/135252701 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5563d9f4d290 msgr2=0x5563d9f53360 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:37.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.849+0000 7fa83558a640 1 --2- 172.21.15.86:0/135252701 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5563d9f4d290 0x5563d9f53360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:37.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.849+0000 7fa83558a640 1 -- 172.21.15.86:0/135252701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563d9cebf70 con 0x5563d9f4c8e0 2024-04-21T08:48:37.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.849+0000 7fa83558a640 1 --2- 172.21.15.86:0/135252701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563d9f4c8e0 0x5563d9f52e20 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fa82c02bc70 tx=0x7fa82c0057f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fdc739fb4e464ba6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:37.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.849+0000 7fa8197fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:37.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.849+0000 7fa8197fa640 1 -- 172.21.15.86:0/135252701 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82c003e40 con 0x5563d9f4c8e0 2024-04-21T08:48:37.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.849+0000 7fa8197fa640 1 -- 172.21.15.86:0/135252701 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7fa82c005020 con 0x5563d9f4c8e0 2024-04-21T08:48:37.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.849+0000 7fa83abd6740 1 -- 172.21.15.86:0/135252701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa82c02cc30 con 0x5563d9f4c8e0 2024-04-21T08:48:37.857 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.849+0000 7fa8197fa640 1 -- 172.21.15.86:0/135252701 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa82c05f050 con 0x5563d9f4c8e0 2024-04-21T08:48:37.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.849+0000 7fa83abd6740 10 client.19626 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:37.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.849+0000 7fa83abd6740 1 -- 172.21.15.86:0/135252701 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5563d9b83170 con 0x5563d9f4c8e0 2024-04-21T08:48:37.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.849+0000 7fa8197fa640 1 -- 172.21.15.86:0/135252701 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa82c02cc30 con 0x5563d9f4c8e0 2024-04-21T08:48:37.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.849+0000 7fa8197fa640 1 -- 172.21.15.86:0/135252701 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa82c040040 con 0x5563d9f4c8e0 2024-04-21T08:48:37.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.850+0000 7fa83abd6740 10 client.19626 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:37.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.850+0000 7fa83abd6740 10 client.19626 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:37.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.850+0000 7fa83abd6740 20 client.19626 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:37.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.850+0000 7fa83abd6740 1 --2- 172.21.15.86:0/135252701 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5563d9f661a0 0x5563d9f68590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:37.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.850+0000 7fa83abd6740 4 client.19626 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:37.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.850+0000 7fa83abd6740 1 -- 172.21.15.86:0/135252701 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5563d9ebe920 con 0x5563d9f661a0 2024-04-21T08:48:37.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.850+0000 7fa835d8b640 1 --2- 172.21.15.86:0/135252701 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5563d9f661a0 0x5563d9f68590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:37.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.850+0000 7fa8197fa640 10 client.19626 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:37.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.850+0000 7fa835d8b640 1 --2- 172.21.15.86:0/135252701 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5563d9f661a0 0x5563d9f68590 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7924cbd658f37403 server_cookie=4580524cc1363d80 in_seq=0 out_seq=0 2024-04-21T08:48:37.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.851+0000 7fa8197fa640 1 -- 172.21.15.86:0/135252701 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5563d9ebe920 con 0x5563d9f661a0 2024-04-21T08:48:37.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.851+0000 7fa8197fa640 10 client.19626 handle_command_reply: tid=0 2024-04-21T08:48:37.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.851+0000 7fa83abd6740 1 client.19626 shutdown 2024-04-21T08:48:37.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.851+0000 7fa83abd6740 1 -- 172.21.15.86:0/135252701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563d9f4c8e0 msgr2=0x5563d9f52e20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:37.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.851+0000 7fa83abd6740 1 --2- 172.21.15.86:0/135252701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563d9f4c8e0 0x5563d9f52e20 secure :-1 s=READY pgs=1803 cs=0 l=1 rev1=1 crypto rx=0x7fa82c02bc70 tx=0x7fa82c0057f0 comp rx=0 tx=0).stop 2024-04-21T08:48:37.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.851+0000 7fa83abd6740 1 -- 172.21.15.86:0/135252701 shutdown_connections 2024-04-21T08:48:37.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.851+0000 7fa83abd6740 1 --2- 172.21.15.86:0/135252701 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5563d9f661a0 0x5563d9f68590 crc :-1 s=READY pgs=289 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:37.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.851+0000 7fa83abd6740 1 --2- 172.21.15.86:0/135252701 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5563d9f55e10 0x5563d9f538a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:37.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.851+0000 7fa83abd6740 1 --2- 172.21.15.86:0/135252701 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5563d9f4d290 0x5563d9f53360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:37.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.851+0000 7fa83abd6740 1 --2- 172.21.15.86:0/135252701 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563d9f4c8e0 0x5563d9f52e20 unknown :-1 s=CLOSED pgs=1803 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:37.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.851+0000 7fa8197fa640 0 client.19626 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:37.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.851+0000 7fa83abd6740 1 -- 172.21.15.86:0/135252701 >> 172.21.15.86:0/135252701 conn(0x5563d9f46fb0 msgr2=0x5563d9f48d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:37.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.851+0000 7fa83abd6740 1 -- 172.21.15.86:0/135252701 shutdown_connections 2024-04-21T08:48:37.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.851+0000 7fa83abd6740 1 -- 172.21.15.86:0/135252701 wait complete. 2024-04-21T08:48:37.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.851+0000 7fa83abd6740 20 client.19626 trim_cache size 0 max 16384 2024-04-21T08:48:37.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.869+0000 7fa83abd6740 1 Processor -- start 2024-04-21T08:48:37.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.869+0000 7fa83abd6740 1 -- start start 2024-04-21T08:48:37.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.869+0000 7fa83abd6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5563d9f4c8e0 0x5563d9f55550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:37.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.869+0000 7fa83abd6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563d9f4d290 0x5563d9f61be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:37.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.869+0000 7fa83abd6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5563d9f55e10 0x5563d9f48af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:37.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.869+0000 7fa83abd6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa8301af260 con 0x5563d9f4d290 2024-04-21T08:48:37.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.869+0000 7fa83abd6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa8301b0da0 con 0x5563d9f55e10 2024-04-21T08:48:37.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.869+0000 7fa83abd6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa8301af930 con 0x5563d9f4c8e0 2024-04-21T08:48:37.874 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.869+0000 7fa835d8b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563d9f4d290 0x5563d9f61be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:37.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.869+0000 7fa835d8b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563d9f4d290 0x5563d9f61be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38592/0 (socket says 172.21.15.86:38592) 2024-04-21T08:48:37.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.869+0000 7fa835d8b640 1 -- 172.21.15.86:0/2589098303 learned_addr learned my addr 172.21.15.86:0/2589098303 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:37.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.869+0000 7fa835d8b640 1 -- 172.21.15.86:0/2589098303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5563d9f4c8e0 msgr2=0x5563d9f55550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:37.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.869+0000 7fa835d8b640 1 --2- 172.21.15.86:0/2589098303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5563d9f4c8e0 0x5563d9f55550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:37.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.869+0000 7fa835d8b640 1 -- 172.21.15.86:0/2589098303 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5563d9f55e10 msgr2=0x5563d9f48af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:37.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.869+0000 7fa835d8b640 1 --2- 172.21.15.86:0/2589098303 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5563d9f55e10 0x5563d9f48af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:37.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.869+0000 7fa835d8b640 1 -- 172.21.15.86:0/2589098303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5563d9f68900 con 0x5563d9f4d290 2024-04-21T08:48:37.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.870+0000 7fa835d8b640 1 --2- 172.21.15.86:0/2589098303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563d9f4d290 0x5563d9f61be0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fa82400b740 tx=0x7fa8240046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea8024f95bfa1f4c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:37.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.870+0000 7fa81b7fe640 1 -- 172.21.15.86:0/2589098303 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa824021030 con 0x5563d9f4d290 2024-04-21T08:48:37.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.870+0000 7fa81b7fe640 1 -- 172.21.15.86:0/2589098303 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa82401d420 con 0x5563d9f4d290 2024-04-21T08:48:37.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.870+0000 7fa81b7fe640 1 -- 172.21.15.86:0/2589098303 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa8240055f0 con 0x5563d9f4d290 2024-04-21T08:48:37.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.870+0000 7fa83abd6740 1 -- 172.21.15.86:0/2589098303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563d9f4d290 msgr2=0x5563d9f61be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:37.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.870+0000 7fa83abd6740 1 --2- 172.21.15.86:0/2589098303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563d9f4d290 0x5563d9f61be0 secure :-1 s=READY pgs=1804 cs=0 l=1 rev1=1 crypto rx=0x7fa82400b740 tx=0x7fa8240046c0 comp rx=0 tx=0).stop 2024-04-21T08:48:37.875 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.870+0000 7fa83abd6740 1 -- 172.21.15.86:0/2589098303 shutdown_connections 2024-04-21T08:48:37.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.870+0000 7fa83abd6740 1 --2- 172.21.15.86:0/2589098303 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5563d9f55e10 0x5563d9f48af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:37.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.870+0000 7fa83abd6740 1 --2- 172.21.15.86:0/2589098303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563d9f4d290 0x5563d9f61be0 unknown :-1 s=CLOSED pgs=1804 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:37.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.870+0000 7fa83abd6740 1 --2- 172.21.15.86:0/2589098303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5563d9f4c8e0 0x5563d9f55550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:37.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.870+0000 7fa83abd6740 1 -- 172.21.15.86:0/2589098303 >> 172.21.15.86:0/2589098303 conn(0x5563d9f46fb0 msgr2=0x5563d9f483e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:37.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.870+0000 7fa83abd6740 1 -- 172.21.15.86:0/2589098303 shutdown_connections 2024-04-21T08:48:37.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.870+0000 7fa83abd6740 1 -- 172.21.15.86:0/2589098303 wait complete. 2024-04-21T08:48:37.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.871+0000 7fa83abd6740 1 -- start start 2024-04-21T08:48:37.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.871+0000 7fa83abd6740 1 Processor -- start 2024-04-21T08:48:37.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.871+0000 7fa83abd6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563d9f4c8e0 0x5563d9f5a6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:37.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.871+0000 7fa83abd6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5563d9f4d290 0x5563d9f5abf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:37.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.871+0000 7fa83abd6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5563d9f55e10 0x5563d9fc4390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:37.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.871+0000 7fa83abd6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa8301b06d0 con 0x5563d9f4c8e0 2024-04-21T08:48:37.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.871+0000 7fa83abd6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa8301b0000 con 0x5563d9f55e10 2024-04-21T08:48:37.876 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.871+0000 7fa83abd6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa8301ad260 con 0x5563d9f4d290 2024-04-21T08:48:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.871+0000 7fa83abd6740 10 client.0 _ll_register_callbacks cb 0x7fa82400b770 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.871+0000 7fa83558a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563d9f4c8e0 0x5563d9f5a6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.871+0000 7fa83558a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563d9f4c8e0 0x5563d9f5a6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38600/0 (socket says 172.21.15.86:38600) 2024-04-21T08:48:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.871+0000 7fa83558a640 1 -- 172.21.15.86:0/4126858249 learned_addr learned my addr 172.21.15.86:0/4126858249 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.872+0000 7fa83558a640 1 --2- 172.21.15.86:0/4126858249 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5563d9f55e10 0x5563d9fc4390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.872+0000 7fa835d8b640 1 --2- 172.21.15.86:0/4126858249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5563d9f4d290 0x5563d9f5abf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.872+0000 7fa83558a640 1 -- 172.21.15.86:0/4126858249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5563d9f4d290 msgr2=0x5563d9f5abf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.872+0000 7fa83558a640 1 --2- 172.21.15.86:0/4126858249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5563d9f4d290 0x5563d9f5abf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.872+0000 7fa83558a640 1 -- 172.21.15.86:0/4126858249 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5563d9f55e10 msgr2=0x5563d9fc4390 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.872+0000 7fa83558a640 1 --2- 172.21.15.86:0/4126858249 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5563d9f55e10 0x5563d9fc4390 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.872+0000 7fa83558a640 1 -- 172.21.15.86:0/4126858249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5563d9fc4b90 con 0x5563d9f4c8e0 2024-04-21T08:48:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.872+0000 7fa83558a640 1 --2- 172.21.15.86:0/4126858249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563d9f4c8e0 0x5563d9f5a6b0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fa82c005d00 tx=0x7fa82c005a20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71e28b31d8b88f84 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.872+0000 7fa8197fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:37.877 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.872+0000 7fa8197fa640 1 -- 172.21.15.86:0/4126858249 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa82c004040 con 0x5563d9f4c8e0 2024-04-21T08:48:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.872+0000 7fa8197fa640 1 -- 172.21.15.86:0/4126858249 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7fa82c003e40 con 0x5563d9f4c8e0 2024-04-21T08:48:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.872+0000 7fa83abd6740 1 -- 172.21.15.86:0/4126858249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa82401d420 con 0x5563d9f4c8e0 2024-04-21T08:48:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.872+0000 7fa8197fa640 1 -- 172.21.15.86:0/4126858249 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa82c009780 con 0x5563d9f4c8e0 2024-04-21T08:48:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.872+0000 7fa83abd6740 10 client.19632 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.872+0000 7fa83abd6740 1 -- 172.21.15.86:0/4126858249 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5563d9fccf90 con 0x5563d9f4c8e0 2024-04-21T08:48:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.872+0000 7fa8197fa640 1 -- 172.21.15.86:0/4126858249 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa82401d420 con 0x5563d9f4c8e0 2024-04-21T08:48:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.872+0000 7fa8197fa640 1 -- 172.21.15.86:0/4126858249 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa82c0050d0 con 0x5563d9f4c8e0 2024-04-21T08:48:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.872+0000 7fa83abd6740 10 client.19632 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.872+0000 7fa83abd6740 10 client.19632 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.872+0000 7fa83abd6740 20 client.19632 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.872+0000 7fa83abd6740 1 --2- 172.21.15.86:0/4126858249 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5563d9fcd590 0x5563d9fcf980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.872+0000 7fa83abd6740 4 client.19632 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:48:37.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.872+0000 7fa83abd6740 1 -- 172.21.15.86:0/4126858249 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5563d9eb8790 con 0x5563d9fcd590 2024-04-21T08:48:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.873+0000 7fa835d8b640 1 --2- 172.21.15.86:0/4126858249 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5563d9fcd590 0x5563d9fcf980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.873+0000 7fa835d8b640 1 --2- 172.21.15.86:0/4126858249 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5563d9fcd590 0x5563d9fcf980 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=22f44bc51d645d19 server_cookie=60d93da329592e0 in_seq=0 out_seq=0 2024-04-21T08:48:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.873+0000 7fa8197fa640 10 client.19632 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.874+0000 7fa8197fa640 1 -- 172.21.15.86:0/4126858249 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5563d9eb8790 con 0x5563d9fcd590 2024-04-21T08:48:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.874+0000 7fa8197fa640 10 client.19632 handle_command_reply: tid=0 2024-04-21T08:48:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.874+0000 7fa83abd6740 1 client.19632 shutdown 2024-04-21T08:48:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.874+0000 7fa83abd6740 1 -- 172.21.15.86:0/4126858249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563d9f4c8e0 msgr2=0x5563d9f5a6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.874+0000 7fa83abd6740 1 --2- 172.21.15.86:0/4126858249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563d9f4c8e0 0x5563d9f5a6b0 secure :-1 s=READY pgs=1805 cs=0 l=1 rev1=1 crypto rx=0x7fa82c005d00 tx=0x7fa82c005a20 comp rx=0 tx=0).stop 2024-04-21T08:48:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.874+0000 7fa83abd6740 1 -- 172.21.15.86:0/4126858249 shutdown_connections 2024-04-21T08:48:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.874+0000 7fa83abd6740 1 --2- 172.21.15.86:0/4126858249 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5563d9fcd590 0x5563d9fcf980 crc :-1 s=READY pgs=290 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.874+0000 7fa83abd6740 1 --2- 172.21.15.86:0/4126858249 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5563d9f55e10 0x5563d9fc4390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.874+0000 7fa83abd6740 1 --2- 172.21.15.86:0/4126858249 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5563d9f4d290 0x5563d9f5abf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.874+0000 7fa8197fa640 0 client.19632 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:37.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.874+0000 7fa83abd6740 1 --2- 172.21.15.86:0/4126858249 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5563d9f4c8e0 0x5563d9f5a6b0 unknown :-1 s=CLOSED pgs=1805 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:37.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.875+0000 7fa83abd6740 1 -- 172.21.15.86:0/4126858249 >> 172.21.15.86:0/4126858249 conn(0x5563d9f46fb0 msgr2=0x5563d9f46010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:37.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.875+0000 7fa83abd6740 1 -- 172.21.15.86:0/4126858249 shutdown_connections 2024-04-21T08:48:37.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.875+0000 7fa83abd6740 1 -- 172.21.15.86:0/4126858249 wait complete. 2024-04-21T08:48:37.880 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.875+0000 7fa83abd6740 20 client.19632 trim_cache size 0 max 16384 2024-04-21T08:48:37.880 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:48:37.880 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-21T08:48:37.880 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "dabe5428-d483-47b1-8c2d-ddee325eb392", 2024-04-21T08:48:37.880 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-21T08:48:37.880 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:48:37.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.875+0000 7fa837fef640 1 -- 172.21.15.86:0/322276286 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa81c0a3d20 msgr2=0x7fa81c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:37.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.875+0000 7fa837fef640 1 --2- 172.21.15.86:0/322276286 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa81c0a3d20 0x7fa81c0a61e0 secure :-1 s=READY pgs=297 cs=0 l=1 rev1=1 crypto rx=0x7fa828098ff0 tx=0x7fa828067630 comp rx=0 tx=0).stop 2024-04-21T08:48:37.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.875+0000 7fa837fef640 1 -- 172.21.15.86:0/322276286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa83019cfa0 msgr2=0x7fa83019d450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:37.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.875+0000 7fa837fef640 1 --2- 172.21.15.86:0/322276286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa83019cfa0 0x7fa83019d450 secure :-1 s=READY pgs=1801 cs=0 l=1 rev1=1 crypto rx=0x7fa82800cce0 tx=0x7fa82800a620 comp rx=0 tx=0).stop 2024-04-21T08:48:37.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.875+0000 7fa837fef640 1 -- 172.21.15.86:0/322276286 shutdown_connections 2024-04-21T08:48:37.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.875+0000 7fa837fef640 1 --2- 172.21.15.86:0/322276286 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa81c0a3d20 0x7fa81c0a61e0 unknown :-1 s=CLOSED pgs=297 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:37.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.875+0000 7fa837fef640 1 --2- 172.21.15.86:0/322276286 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa83019cfa0 0x7fa83019d450 unknown :-1 s=CLOSED pgs=1801 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:37.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.875+0000 7fa837fef640 1 --2- 172.21.15.86:0/322276286 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa830107260 0x7fa830198c20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:37.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.875+0000 7fa837fef640 1 --2- 172.21.15.86:0/322276286 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa830102190 0x7fa8301986e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:37.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.875+0000 7fa837fef640 1 -- 172.21.15.86:0/322276286 >> 172.21.15.86:0/322276286 conn(0x7fa8300793a0 msgr2=0x7fa8300fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:37.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.875+0000 7fa837fef640 1 -- 172.21.15.86:0/322276286 shutdown_connections 2024-04-21T08:48:37.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.875+0000 7fa837fef640 1 -- 172.21.15.86:0/322276286 wait complete. 2024-04-21T08:48:37.885 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:48:37.886 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:48:37.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.987+0000 7f0d94b96640 1 -- 172.21.15.86:0/2941539135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d90108020 msgr2=0x7f0d90108400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:37.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.987+0000 7f0d94b96640 1 --2- 172.21.15.86:0/2941539135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d90108020 0x7f0d90108400 secure :-1 s=READY pgs=1806 cs=0 l=1 rev1=1 crypto rx=0x7f0d80009ca0 tx=0x7f0d8002c180 comp rx=0 tx=0).stop 2024-04-21T08:48:37.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.987+0000 7f0d94b96640 1 -- 172.21.15.86:0/2941539135 shutdown_connections 2024-04-21T08:48:37.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.987+0000 7f0d94b96640 1 --2- 172.21.15.86:0/2941539135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d9006af70 0x7f0d9010ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:37.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.987+0000 7f0d94b96640 1 --2- 172.21.15.86:0/2941539135 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d9006a5d0 0x7f0d9006aa30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:37.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.987+0000 7f0d94b96640 1 --2- 172.21.15.86:0/2941539135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d90108020 0x7f0d90108400 unknown :-1 s=CLOSED pgs=1806 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:37.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.987+0000 7f0d94b96640 1 -- 172.21.15.86:0/2941539135 >> 172.21.15.86:0/2941539135 conn(0x7f0d90076c50 msgr2=0x7f0d90077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:37.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.987+0000 7f0d94b96640 1 -- 172.21.15.86:0/2941539135 shutdown_connections 2024-04-21T08:48:37.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.987+0000 7f0d94b96640 1 -- 172.21.15.86:0/2941539135 wait complete. 2024-04-21T08:48:37.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.988+0000 7f0d94b96640 1 Processor -- start 2024-04-21T08:48:37.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.988+0000 7f0d94b96640 1 -- start start 2024-04-21T08:48:37.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.988+0000 7f0d94b96640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d9006a5d0 0x7f0d90198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:37.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.988+0000 7f0d94b96640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d9006af70 0x7f0d90198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:37.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.988+0000 7f0d94b96640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d90108020 0x7f0d9019cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:37.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.988+0000 7f0d94b96640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d90112160 con 0x7f0d90108020 2024-04-21T08:48:37.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.988+0000 7f0d94b96640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0d90111fe0 con 0x7f0d9006af70 2024-04-21T08:48:37.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.988+0000 7f0d94b96640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0d901122e0 con 0x7f0d9006a5d0 2024-04-21T08:48:37.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.988+0000 7f0d8ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d90108020 0x7f0d9019cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:37.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.988+0000 7f0d8ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d90108020 0x7f0d9019cfe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38612/0 (socket says 172.21.15.86:38612) 2024-04-21T08:48:37.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.988+0000 7f0d8ed9d640 1 -- 172.21.15.86:0/1508013120 learned_addr learned my addr 172.21.15.86:0/1508013120 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:37.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.988+0000 7f0d8dd9b640 1 --2- 172.21.15.86:0/1508013120 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d9006af70 0x7f0d90198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:37.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.988+0000 7f0d8e59c640 1 --2- 172.21.15.86:0/1508013120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d9006a5d0 0x7f0d90198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:37.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.988+0000 7f0d8ed9d640 1 -- 172.21.15.86:0/1508013120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d9006a5d0 msgr2=0x7f0d90198720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:37.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.988+0000 7f0d8ed9d640 1 --2- 172.21.15.86:0/1508013120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d9006a5d0 0x7f0d90198720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:37.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.988+0000 7f0d8ed9d640 1 -- 172.21.15.86:0/1508013120 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d9006af70 msgr2=0x7f0d90198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:37.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.988+0000 7f0d8ed9d640 1 --2- 172.21.15.86:0/1508013120 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d9006af70 0x7f0d90198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:37.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.988+0000 7f0d8ed9d640 1 -- 172.21.15.86:0/1508013120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0d9019d710 con 0x7f0d90108020 2024-04-21T08:48:37.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.989+0000 7f0d8ed9d640 1 --2- 172.21.15.86:0/1508013120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d90108020 0x7f0d9019cfe0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f0d8800d700 tx=0x7f0d8800dbc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:37.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.989+0000 7f0d7f7fe640 1 -- 172.21.15.86:0/1508013120 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d88017030 con 0x7f0d90108020 2024-04-21T08:48:37.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.989+0000 7f0d7f7fe640 1 -- 172.21.15.86:0/1508013120 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d88009820 con 0x7f0d90108020 2024-04-21T08:48:37.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.989+0000 7f0d94b96640 1 -- 172.21.15.86:0/1508013120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0d9019da00 con 0x7f0d90108020 2024-04-21T08:48:37.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.989+0000 7f0d7f7fe640 1 -- 172.21.15.86:0/1508013120 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d88003ac0 con 0x7f0d90108020 2024-04-21T08:48:37.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.989+0000 7f0d94b96640 1 -- 172.21.15.86:0/1508013120 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0d901a5290 con 0x7f0d90108020 2024-04-21T08:48:37.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.991+0000 7f0d7f7fe640 1 -- 172.21.15.86:0/1508013120 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0d8801a070 con 0x7f0d90108020 2024-04-21T08:48:37.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.991+0000 7f0d97788740 1 Processor -- start 2024-04-21T08:48:37.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.992+0000 7f0d97788740 1 -- start start 2024-04-21T08:48:37.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.992+0000 7f0d97788740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757f9c8580 0x55757f9c8960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:37.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.992+0000 7f0d97788740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55757f9c8ea0 0x55757f9d1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:37.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.992+0000 7f0d97788740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55757f9d1a80 0x55757f9d3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:37.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.992+0000 7f0d97788740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55757f92d1e0 con 0x55757f9c8580 2024-04-21T08:48:37.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.992+0000 7f0d97788740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55757f8ca390 con 0x55757f9c8ea0 2024-04-21T08:48:37.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.992+0000 7f0d97788740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55757f9291a0 con 0x55757f9d1a80 2024-04-21T08:48:37.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.992+0000 7f0d8dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757f9c8580 0x55757f9c8960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:37.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.992+0000 7f0d7f7fe640 1 --2- 172.21.15.86:0/1508013120 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0d780a3d20 0x7f0d780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:37.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.992+0000 7f0d8dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757f9c8580 0x55757f9c8960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38614/0 (socket says 172.21.15.86:38614) 2024-04-21T08:48:37.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.992+0000 7f0d8dd9b640 1 -- 172.21.15.86:0/448091020 learned_addr learned my addr 172.21.15.86:0/448091020 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:37.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.992+0000 7f0d8dd9b640 1 --2- 172.21.15.86:0/448091020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55757f9d1a80 0x55757f9d3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:37.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.992+0000 7f0d8e59c640 1 --2- 172.21.15.86:0/448091020 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55757f9c8ea0 0x55757f9d1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:37.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.992+0000 7f0d8ed9d640 1 --2- 172.21.15.86:0/1508013120 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0d780a3d20 0x7f0d780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:38.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.992+0000 7f0d8dd9b640 1 -- 172.21.15.86:0/448091020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55757f9d1a80 msgr2=0x55757f9d3e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:38.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.992+0000 7f0d8dd9b640 1 --2- 172.21.15.86:0/448091020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55757f9d1a80 0x55757f9d3e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:38.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.992+0000 7f0d8dd9b640 1 -- 172.21.15.86:0/448091020 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55757f9c8ea0 msgr2=0x55757f9d1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:38.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.992+0000 7f0d8dd9b640 1 --2- 172.21.15.86:0/448091020 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55757f9c8ea0 0x55757f9d1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:38.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.992+0000 7f0d8dd9b640 1 -- 172.21.15.86:0/448091020 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55757f7097d0 con 0x55757f9c8580 2024-04-21T08:48:38.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.992+0000 7f0d7f7fe640 1 -- 172.21.15.86:0/1508013120 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f0d880d0240 con 0x7f0d90108020 2024-04-21T08:48:38.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.992+0000 7f0d8dd9b640 1 --2- 172.21.15.86:0/448091020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757f9c8580 0x55757f9c8960 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f0d8400bfa0 tx=0x7f0d8400b840 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2553e1c736c8c21c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:38.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.993+0000 7f0d7cff9640 1 -- 172.21.15.86:0/448091020 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d84004030 con 0x55757f9c8580 2024-04-21T08:48:38.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.993+0000 7f0d8ed9d640 1 --2- 172.21.15.86:0/1508013120 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0d780a3d20 0x7f0d780a61e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0d880bafd0 tx=0x7f0d8801f6c0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:38.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.993+0000 7f0d7cff9640 1 -- 172.21.15.86:0/448091020 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d840041d0 con 0x55757f9c8580 2024-04-21T08:48:38.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.993+0000 7f0d7cff9640 1 -- 172.21.15.86:0/448091020 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d84010660 con 0x55757f9c8580 2024-04-21T08:48:38.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.993+0000 7f0d97788740 1 -- 172.21.15.86:0/448091020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757f9c8580 msgr2=0x55757f9c8960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:38.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.993+0000 7f0d97788740 1 --2- 172.21.15.86:0/448091020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757f9c8580 0x55757f9c8960 secure :-1 s=READY pgs=1808 cs=0 l=1 rev1=1 crypto rx=0x7f0d8400bfa0 tx=0x7f0d8400b840 comp rx=0 tx=0).stop 2024-04-21T08:48:38.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.993+0000 7f0d97788740 1 -- 172.21.15.86:0/448091020 shutdown_connections 2024-04-21T08:48:38.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.993+0000 7f0d97788740 1 --2- 172.21.15.86:0/448091020 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55757f9d1a80 0x55757f9d3e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:38.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.993+0000 7f0d97788740 1 --2- 172.21.15.86:0/448091020 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55757f9c8ea0 0x55757f9d1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:38.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.993+0000 7f0d97788740 1 --2- 172.21.15.86:0/448091020 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757f9c8580 0x55757f9c8960 unknown :-1 s=CLOSED pgs=1808 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:38.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.993+0000 7f0d97788740 1 -- 172.21.15.86:0/448091020 >> 172.21.15.86:0/448091020 conn(0x55757f9c2cb0 msgr2=0x55757f9c3090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:38.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.993+0000 7f0d97788740 1 -- 172.21.15.86:0/448091020 shutdown_connections 2024-04-21T08:48:38.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.993+0000 7f0d97788740 1 -- 172.21.15.86:0/448091020 wait complete. 2024-04-21T08:48:38.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.994+0000 7f0d97788740 1 -- start start 2024-04-21T08:48:38.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.994+0000 7f0d97788740 1 Processor -- start 2024-04-21T08:48:38.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.994+0000 7f0d97788740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757f9c8580 0x55757f9ceaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:38.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.994+0000 7f0d97788740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55757f9c8ea0 0x55757f9cf030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:38.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.994+0000 7f0d97788740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55757f9d1a80 0x55757f9cf570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:38.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.994+0000 7f0d97788740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55757f9d4870 con 0x55757f9c8580 2024-04-21T08:48:38.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.994+0000 7f0d97788740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55757f8e62f0 con 0x55757f9c8ea0 2024-04-21T08:48:38.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.994+0000 7f0d97788740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55757f9d49f0 con 0x55757f9d1a80 2024-04-21T08:48:38.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.994+0000 7f0d97788740 10 client.0 _ll_register_callbacks cb 0x55757f8f3580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:38.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.994+0000 7f0d8dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757f9c8580 0x55757f9ceaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:38.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.994+0000 7f0d8dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757f9c8580 0x55757f9ceaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38622/0 (socket says 172.21.15.86:38622) 2024-04-21T08:48:38.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.994+0000 7f0d8dd9b640 1 -- 172.21.15.86:0/1587748872 learned_addr learned my addr 172.21.15.86:0/1587748872 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:38.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.994+0000 7f0d8e59c640 1 --2- 172.21.15.86:0/1587748872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55757f9c8ea0 0x55757f9cf030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:38.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.994+0000 7f0d8dd9b640 1 --2- 172.21.15.86:0/1587748872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55757f9d1a80 0x55757f9cf570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:38.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.994+0000 7f0d8dd9b640 1 -- 172.21.15.86:0/1587748872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55757f9d1a80 msgr2=0x55757f9cf570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:38.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.994+0000 7f0d8dd9b640 1 --2- 172.21.15.86:0/1587748872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55757f9d1a80 0x55757f9cf570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:38.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.994+0000 7f0d8dd9b640 1 -- 172.21.15.86:0/1587748872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55757f9c8ea0 msgr2=0x55757f9cf030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:38.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.994+0000 7f0d8dd9b640 1 --2- 172.21.15.86:0/1587748872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55757f9c8ea0 0x55757f9cf030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:38.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.994+0000 7f0d8dd9b640 1 -- 172.21.15.86:0/1587748872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55757f63f3f0 con 0x55757f9c8580 2024-04-21T08:48:38.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.994+0000 7f0d8e59c640 1 --2- 172.21.15.86:0/1587748872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55757f9c8ea0 0x55757f9cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:38.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.995+0000 7f0d8dd9b640 1 --2- 172.21.15.86:0/1587748872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757f9c8580 0x55757f9ceaf0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f0d84005bc0 tx=0x7f0d84003ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4aa90d5d16b5c710 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:38.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.995+0000 7f0d75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:38.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.995+0000 7f0d75ffb640 1 -- 172.21.15.86:0/1587748872 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d840037f0 con 0x55757f9c8580 2024-04-21T08:48:38.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.995+0000 7f0d75ffb640 1 -- 172.21.15.86:0/1587748872 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f0d84003990 con 0x55757f9c8580 2024-04-21T08:48:38.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.995+0000 7f0d97788740 1 -- 172.21.15.86:0/1587748872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d840041d0 con 0x55757f9c8580 2024-04-21T08:48:38.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.995+0000 7f0d75ffb640 1 -- 172.21.15.86:0/1587748872 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0d84005040 con 0x55757f9c8580 2024-04-21T08:48:38.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.995+0000 7f0d97788740 10 client.19644 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:38.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.995+0000 7f0d97788740 1 -- 172.21.15.86:0/1587748872 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55757f5ff0f0 con 0x55757f9c8580 2024-04-21T08:48:38.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.995+0000 7f0d75ffb640 1 -- 172.21.15.86:0/1587748872 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d840041d0 con 0x55757f9c8580 2024-04-21T08:48:38.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.995+0000 7f0d75ffb640 1 -- 172.21.15.86:0/1587748872 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0d84004b90 con 0x55757f9c8580 2024-04-21T08:48:38.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.995+0000 7f0d97788740 10 client.19644 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:38.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.995+0000 7f0d97788740 10 client.19644 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:38.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.995+0000 7f0d97788740 20 client.19644 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:38.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.995+0000 7f0d97788740 1 --2- 172.21.15.86:0/1587748872 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55757f9e1950 0x55757f9e3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:38.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.995+0000 7f0d97788740 4 client.19644 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:38.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.995+0000 7f0d97788740 1 -- 172.21.15.86:0/1587748872 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55757f93ac00 con 0x55757f9e1950 2024-04-21T08:48:38.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.996+0000 7f0d8e59c640 1 --2- 172.21.15.86:0/1587748872 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55757f9e1950 0x55757f9e3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:38.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.996+0000 7f0d75ffb640 10 client.19644 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:38.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.996+0000 7f0d8e59c640 1 --2- 172.21.15.86:0/1587748872 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55757f9e1950 0x55757f9e3d40 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=798b3a38e10bc465 server_cookie=269d80af42616a7e in_seq=0 out_seq=0 2024-04-21T08:48:38.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.997+0000 7f0d75ffb640 1 -- 172.21.15.86:0/1587748872 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55757f93ac00 con 0x55757f9e1950 2024-04-21T08:48:38.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.997+0000 7f0d75ffb640 10 client.19644 handle_command_reply: tid=0 2024-04-21T08:48:38.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.997+0000 7f0d97788740 1 client.19644 shutdown 2024-04-21T08:48:38.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.997+0000 7f0d97788740 1 -- 172.21.15.86:0/1587748872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757f9c8580 msgr2=0x55757f9ceaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:38.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.997+0000 7f0d97788740 1 --2- 172.21.15.86:0/1587748872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757f9c8580 0x55757f9ceaf0 secure :-1 s=READY pgs=1809 cs=0 l=1 rev1=1 crypto rx=0x7f0d84005bc0 tx=0x7f0d84003ce0 comp rx=0 tx=0).stop 2024-04-21T08:48:38.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.997+0000 7f0d97788740 1 -- 172.21.15.86:0/1587748872 shutdown_connections 2024-04-21T08:48:38.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.997+0000 7f0d97788740 1 --2- 172.21.15.86:0/1587748872 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55757f9e1950 0x55757f9e3d40 crc :-1 s=READY pgs=291 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:38.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.997+0000 7f0d97788740 1 --2- 172.21.15.86:0/1587748872 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55757f9d1a80 0x55757f9cf570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:38.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.997+0000 7f0d97788740 1 --2- 172.21.15.86:0/1587748872 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55757f9c8ea0 0x55757f9cf030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:38.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.997+0000 7f0d75ffb640 0 client.19644 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:38.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.997+0000 7f0d97788740 1 --2- 172.21.15.86:0/1587748872 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757f9c8580 0x55757f9ceaf0 unknown :-1 s=CLOSED pgs=1809 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:38.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.997+0000 7f0d97788740 1 -- 172.21.15.86:0/1587748872 >> 172.21.15.86:0/1587748872 conn(0x55757f9c2cb0 msgr2=0x55757f9c4a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:38.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.997+0000 7f0d97788740 1 -- 172.21.15.86:0/1587748872 shutdown_connections 2024-04-21T08:48:38.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.997+0000 7f0d97788740 1 -- 172.21.15.86:0/1587748872 wait complete. 2024-04-21T08:48:38.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:37.997+0000 7f0d97788740 20 client.19644 trim_cache size 0 max 16384 2024-04-21T08:48:38.019 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.014+0000 7f0d97788740 1 Processor -- start 2024-04-21T08:48:38.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.014+0000 7f0d97788740 1 -- start start 2024-04-21T08:48:38.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.014+0000 7f0d97788740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757f9c8580 0x55757f9d1220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:38.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.014+0000 7f0d97788740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55757f9c8ea0 0x55757f9c4840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:38.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.014+0000 7f0d97788740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55757f9d1a80 0x55757f9ce2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:38.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.014+0000 7f0d97788740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d901af400 con 0x55757f9c8580 2024-04-21T08:48:38.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.014+0000 7f0d97788740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0d901b0f40 con 0x55757f9d1a80 2024-04-21T08:48:38.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.014+0000 7f0d97788740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0d901afad0 con 0x55757f9c8ea0 2024-04-21T08:48:38.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.014+0000 7f0d8dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757f9c8580 0x55757f9d1220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:38.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.014+0000 7f0d8dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757f9c8580 0x55757f9d1220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38630/0 (socket says 172.21.15.86:38630) 2024-04-21T08:48:38.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.014+0000 7f0d8dd9b640 1 -- 172.21.15.86:0/2821264792 learned_addr learned my addr 172.21.15.86:0/2821264792 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:38.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.015+0000 7f0d8e59c640 1 --2- 172.21.15.86:0/2821264792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55757f9c8ea0 0x55757f9c4840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:38.020 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.015+0000 7f0d8dd9b640 1 -- 172.21.15.86:0/2821264792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55757f9c8ea0 msgr2=0x55757f9c4840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:38.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.015+0000 7f0d8dd9b640 1 --2- 172.21.15.86:0/2821264792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55757f9c8ea0 0x55757f9c4840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:38.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.015+0000 7f0d8dd9b640 1 -- 172.21.15.86:0/2821264792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55757f9d1a80 msgr2=0x55757f9ce2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:38.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.015+0000 7f0d8dd9b640 1 --2- 172.21.15.86:0/2821264792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55757f9d1a80 0x55757f9ce2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:38.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.015+0000 7f0d8dd9b640 1 -- 172.21.15.86:0/2821264792 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55757f9c8cd0 con 0x55757f9c8580 2024-04-21T08:48:38.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.015+0000 7f0d8e59c640 1 --2- 172.21.15.86:0/2821264792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55757f9c8ea0 0x55757f9c4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:38.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.015+0000 7f0d8dd9b640 1 --2- 172.21.15.86:0/2821264792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757f9c8580 0x55757f9d1220 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f0d84012570 tx=0x7f0d84004f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17ddc88363b79aa5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:38.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.015+0000 7f0d77fff640 1 -- 172.21.15.86:0/2821264792 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d840122e0 con 0x55757f9c8580 2024-04-21T08:48:38.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.015+0000 7f0d77fff640 1 -- 172.21.15.86:0/2821264792 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d84023030 con 0x55757f9c8580 2024-04-21T08:48:38.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.015+0000 7f0d77fff640 1 -- 172.21.15.86:0/2821264792 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d840035f0 con 0x55757f9c8580 2024-04-21T08:48:38.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.015+0000 7f0d97788740 1 -- 172.21.15.86:0/2821264792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757f9c8580 msgr2=0x55757f9d1220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:38.021 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.015+0000 7f0d97788740 1 --2- 172.21.15.86:0/2821264792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757f9c8580 0x55757f9d1220 secure :-1 s=READY pgs=1810 cs=0 l=1 rev1=1 crypto rx=0x7f0d84012570 tx=0x7f0d84004f90 comp rx=0 tx=0).stop 2024-04-21T08:48:38.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.015+0000 7f0d97788740 1 -- 172.21.15.86:0/2821264792 shutdown_connections 2024-04-21T08:48:38.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.015+0000 7f0d97788740 1 --2- 172.21.15.86:0/2821264792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55757f9d1a80 0x55757f9ce2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:38.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.015+0000 7f0d97788740 1 --2- 172.21.15.86:0/2821264792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55757f9c8ea0 0x55757f9c4840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:38.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.015+0000 7f0d97788740 1 --2- 172.21.15.86:0/2821264792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757f9c8580 0x55757f9d1220 unknown :-1 s=CLOSED pgs=1810 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:38.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.015+0000 7f0d97788740 1 -- 172.21.15.86:0/2821264792 >> 172.21.15.86:0/2821264792 conn(0x55757f9c2cb0 msgr2=0x55757f9c14e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:38.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.015+0000 7f0d97788740 1 -- 172.21.15.86:0/2821264792 shutdown_connections 2024-04-21T08:48:38.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.015+0000 7f0d97788740 1 -- 172.21.15.86:0/2821264792 wait complete. 2024-04-21T08:48:38.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.016+0000 7f0d97788740 1 -- start start 2024-04-21T08:48:38.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.016+0000 7f0d97788740 1 Processor -- start 2024-04-21T08:48:38.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.016+0000 7f0d97788740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55757f9c8580 0x55757f9d5a30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:38.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.016+0000 7f0d97788740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55757f9c8ea0 0x55757f9d5f70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:38.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.016+0000 7f0d97788740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757f9d1a80 0x55757fa46d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:38.022 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.016+0000 7f0d97788740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0d901b0870 con 0x55757f9d1a80 2024-04-21T08:48:38.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.016+0000 7f0d97788740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0d901b01a0 con 0x55757f9c8580 2024-04-21T08:48:38.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.016+0000 7f0d97788740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0d901ad400 con 0x55757f9c8ea0 2024-04-21T08:48:38.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.016+0000 7f0d97788740 10 client.0 _ll_register_callbacks cb 0x7f0d84012480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:38.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.016+0000 7f0d8dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757f9d1a80 0x55757fa46d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:38.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.016+0000 7f0d8dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757f9d1a80 0x55757fa46d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38634/0 (socket says 172.21.15.86:38634) 2024-04-21T08:48:38.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.016+0000 7f0d8dd9b640 1 -- 172.21.15.86:0/1337949892 learned_addr learned my addr 172.21.15.86:0/1337949892 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:38.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.016+0000 7f0d8dd9b640 1 --2- 172.21.15.86:0/1337949892 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55757f9c8580 0x55757f9d5a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:38.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.016+0000 7f0d8e59c640 1 --2- 172.21.15.86:0/1337949892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55757f9c8ea0 0x55757f9d5f70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:38.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.016+0000 7f0d8dd9b640 1 -- 172.21.15.86:0/1337949892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55757f9c8ea0 msgr2=0x55757f9d5f70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:38.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.016+0000 7f0d8dd9b640 1 --2- 172.21.15.86:0/1337949892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55757f9c8ea0 0x55757f9d5f70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:38.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.016+0000 7f0d8dd9b640 1 -- 172.21.15.86:0/1337949892 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55757f9c8580 msgr2=0x55757f9d5a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:38.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.016+0000 7f0d8dd9b640 1 --2- 172.21.15.86:0/1337949892 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55757f9c8580 0x55757f9d5a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:38.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.016+0000 7f0d8dd9b640 1 -- 172.21.15.86:0/1337949892 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55757fa47260 con 0x55757f9d1a80 2024-04-21T08:48:38.023 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.016+0000 7f0d8dd9b640 1 --2- 172.21.15.86:0/1337949892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757f9d1a80 0x55757fa46d20 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f0d8400b4e0 tx=0x7f0d8403c040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ea56da262ec0437e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:38.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.016+0000 7f0d75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:38.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.017+0000 7f0d75ffb640 1 -- 172.21.15.86:0/1337949892 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0d840122e0 con 0x55757f9d1a80 2024-04-21T08:48:38.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.017+0000 7f0d75ffb640 1 -- 172.21.15.86:0/1337949892 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f0d84030110 con 0x55757f9d1a80 2024-04-21T08:48:38.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.017+0000 7f0d97788740 1 -- 172.21.15.86:0/1337949892 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0d84023030 con 0x55757f9d1a80 2024-04-21T08:48:38.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.017+0000 7f0d75ffb640 1 -- 172.21.15.86:0/1337949892 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0d84030310 con 0x55757f9d1a80 2024-04-21T08:48:38.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.017+0000 7f0d97788740 10 client.19650 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:38.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.017+0000 7f0d97788740 1 -- 172.21.15.86:0/1337949892 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55757fa48380 con 0x55757f9d1a80 2024-04-21T08:48:38.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.017+0000 7f0d75ffb640 1 -- 172.21.15.86:0/1337949892 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0d84023030 con 0x55757f9d1a80 2024-04-21T08:48:38.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.017+0000 7f0d75ffb640 1 -- 172.21.15.86:0/1337949892 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0d84005700 con 0x55757f9d1a80 2024-04-21T08:48:38.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.017+0000 7f0d97788740 10 client.19650 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:38.024 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.017+0000 7f0d97788740 10 client.19650 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:38.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.017+0000 7f0d97788740 20 client.19650 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:38.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.017+0000 7f0d97788740 1 --2- 172.21.15.86:0/1337949892 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55757fa48990 0x55757fa4adb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:38.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.017+0000 7f0d97788740 4 client.19650 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:48:38.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.017+0000 7f0d97788740 1 -- 172.21.15.86:0/1337949892 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55757f934a70 con 0x55757fa48990 2024-04-21T08:48:38.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.017+0000 7f0d8e59c640 1 --2- 172.21.15.86:0/1337949892 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55757fa48990 0x55757fa4adb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:38.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.018+0000 7f0d75ffb640 10 client.19650 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:38.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.018+0000 7f0d8e59c640 1 --2- 172.21.15.86:0/1337949892 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55757fa48990 0x55757fa4adb0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=de83ab4432b70409 server_cookie=a8f5df0228333714 in_seq=0 out_seq=0 2024-04-21T08:48:38.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.018+0000 7f0d75ffb640 1 -- 172.21.15.86:0/1337949892 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55757f934a70 con 0x55757fa48990 2024-04-21T08:48:38.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.018+0000 7f0d75ffb640 10 client.19650 handle_command_reply: tid=0 2024-04-21T08:48:38.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.018+0000 7f0d97788740 1 client.19650 shutdown 2024-04-21T08:48:38.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.018+0000 7f0d97788740 1 -- 172.21.15.86:0/1337949892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757f9d1a80 msgr2=0x55757fa46d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:38.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.018+0000 7f0d97788740 1 --2- 172.21.15.86:0/1337949892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757f9d1a80 0x55757fa46d20 secure :-1 s=READY pgs=1811 cs=0 l=1 rev1=1 crypto rx=0x7f0d8400b4e0 tx=0x7f0d8403c040 comp rx=0 tx=0).stop 2024-04-21T08:48:38.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.018+0000 7f0d97788740 1 -- 172.21.15.86:0/1337949892 shutdown_connections 2024-04-21T08:48:38.025 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.018+0000 7f0d97788740 1 --2- 172.21.15.86:0/1337949892 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55757fa48990 0x55757fa4adb0 crc :-1 s=READY pgs=292 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:38.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.018+0000 7f0d97788740 1 --2- 172.21.15.86:0/1337949892 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55757f9d1a80 0x55757fa46d20 unknown :-1 s=CLOSED pgs=1811 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:38.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.018+0000 7f0d75ffb640 0 client.19650 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:38.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.018+0000 7f0d97788740 1 --2- 172.21.15.86:0/1337949892 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55757f9c8ea0 0x55757f9d5f70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:38.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.018+0000 7f0d97788740 1 --2- 172.21.15.86:0/1337949892 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55757f9c8580 0x55757f9d5a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:38.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.018+0000 7f0d97788740 1 -- 172.21.15.86:0/1337949892 >> 172.21.15.86:0/1337949892 conn(0x55757f9c2cb0 msgr2=0x55757f9c1d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:38.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.018+0000 7f0d97788740 1 -- 172.21.15.86:0/1337949892 shutdown_connections 2024-04-21T08:48:38.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.018+0000 7f0d97788740 1 -- 172.21.15.86:0/1337949892 wait complete. 2024-04-21T08:48:38.026 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.018+0000 7f0d97788740 20 client.19650 trim_cache size 0 max 16384 2024-04-21T08:48:38.027 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:48:38.027 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:48:38.027 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-21T08:48:38.027 INFO:teuthology.orchestra.run.smithi086.stdout: "dabe5428-d483-47b1-8c2d-ddee325eb392": { 2024-04-21T08:48:38.027 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-21T08:48:38.027 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "dabe5428-d483-47b1-8c2d-ddee325eb392", 2024-04-21T08:48:38.027 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-21T08:48:38.027 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:48:38.027 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:48:38.028 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:48:38.028 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.019+0000 7f0d94b96640 1 -- 172.21.15.86:0/1508013120 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0d780a3d20 msgr2=0x7f0d780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:38.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.019+0000 7f0d94b96640 1 --2- 172.21.15.86:0/1508013120 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0d780a3d20 0x7f0d780a61e0 secure :-1 s=READY pgs=298 cs=0 l=1 rev1=1 crypto rx=0x7f0d880bafd0 tx=0x7f0d8801f6c0 comp rx=0 tx=0).stop 2024-04-21T08:48:38.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.019+0000 7f0d94b96640 1 -- 172.21.15.86:0/1508013120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d90108020 msgr2=0x7f0d9019cfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:38.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.019+0000 7f0d94b96640 1 --2- 172.21.15.86:0/1508013120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d90108020 0x7f0d9019cfe0 secure :-1 s=READY pgs=1807 cs=0 l=1 rev1=1 crypto rx=0x7f0d8800d700 tx=0x7f0d8800dbc0 comp rx=0 tx=0).stop 2024-04-21T08:48:38.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.019+0000 7f0d94b96640 1 -- 172.21.15.86:0/1508013120 shutdown_connections 2024-04-21T08:48:38.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.019+0000 7f0d94b96640 1 --2- 172.21.15.86:0/1508013120 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0d780a3d20 0x7f0d780a61e0 unknown :-1 s=CLOSED pgs=298 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:38.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.019+0000 7f0d94b96640 1 --2- 172.21.15.86:0/1508013120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0d90108020 0x7f0d9019cfe0 unknown :-1 s=CLOSED pgs=1807 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:38.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.019+0000 7f0d94b96640 1 --2- 172.21.15.86:0/1508013120 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0d9006af70 0x7f0d90198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:38.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.019+0000 7f0d94b96640 1 --2- 172.21.15.86:0/1508013120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0d9006a5d0 0x7f0d90198720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:38.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.019+0000 7f0d94b96640 1 -- 172.21.15.86:0/1508013120 >> 172.21.15.86:0/1508013120 conn(0x7f0d90076c50 msgr2=0x7f0d90100050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:38.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.019+0000 7f0d94b96640 1 -- 172.21.15.86:0/1508013120 shutdown_connections 2024-04-21T08:48:38.029 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:38.019+0000 7f0d94b96640 1 -- 172.21.15.86:0/1508013120 wait complete. 2024-04-21T08:48:38.030 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:37 smithi086 ceph-mon[27939]: pgmap v306: 129 pgs: 129 active+clean; 2.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 KiB/s wr, 11 op/s 2024-04-21T08:48:38.030 INFO:tasks.cephfs.filesystem:scrub status for tag:dabe5428-d483-47b1-8c2d-ddee325eb392 - {'path': '/', 'tag': 'dabe5428-d483-47b1-8c2d-ddee325eb392', 'options': 'recursive,force'} 2024-04-21T08:48:38.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:37 smithi088 ceph-mon[31259]: pgmap v306: 129 pgs: 129 active+clean; 2.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 KiB/s wr, 11 op/s 2024-04-21T08:48:38.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:37 smithi179 ceph-mon[31829]: pgmap v306: 129 pgs: 129 active+clean; 2.2 MiB data, 422 MiB used, 1.0 TiB / 1.0 TiB avail; 11 KiB/s rd, 9.5 KiB/s wr, 11 op/s 2024-04-21T08:48:39.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:38 smithi088 ceph-mon[31259]: scrub queued for path: / 2024-04-21T08:48:39.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:38 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:39.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:38 smithi088 ceph-mon[31259]: scrub summary: active paths [/] 2024-04-21T08:48:39.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:38 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:39.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:38 smithi179 ceph-mon[31829]: scrub queued for path: / 2024-04-21T08:48:39.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:38 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:39.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:38 smithi179 ceph-mon[31829]: scrub summary: active paths [/] 2024-04-21T08:48:39.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:38 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:39.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:38 smithi086 ceph-mon[27939]: scrub queued for path: / 2024-04-21T08:48:39.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:38 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:39.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:38 smithi086 ceph-mon[27939]: scrub summary: active paths [/] 2024-04-21T08:48:39.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:38 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:48:40.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:39 smithi088 ceph-mon[31259]: pgmap v307: 129 pgs: 129 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.5 KiB/s wr, 15 op/s 2024-04-21T08:48:40.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:39 smithi088 ceph-mon[31259]: scrub summary: idle 2024-04-21T08:48:40.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:39 smithi179 ceph-mon[31829]: pgmap v307: 129 pgs: 129 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.5 KiB/s wr, 15 op/s 2024-04-21T08:48:40.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:39 smithi179 ceph-mon[31829]: scrub summary: idle 2024-04-21T08:48:40.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:39 smithi086 ceph-mon[27939]: pgmap v307: 129 pgs: 129 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 9.5 KiB/s wr, 15 op/s 2024-04-21T08:48:40.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:39 smithi086 ceph-mon[27939]: scrub summary: idle 2024-04-21T08:48:42.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:41 smithi088 ceph-mon[31259]: pgmap v308: 129 pgs: 129 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.7 KiB/s wr, 11 op/s 2024-04-21T08:48:42.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:41 smithi179 ceph-mon[31829]: pgmap v308: 129 pgs: 129 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.7 KiB/s wr, 11 op/s 2024-04-21T08:48:42.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:41 smithi086 ceph-mon[27939]: pgmap v308: 129 pgs: 129 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 9.1 KiB/s rd, 7.7 KiB/s wr, 11 op/s 2024-04-21T08:48:44.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:43 smithi088 ceph-mon[31259]: pgmap v309: 129 pgs: 129 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.2 KiB/s wr, 16 op/s 2024-04-21T08:48:44.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:43 smithi179 ceph-mon[31829]: pgmap v309: 129 pgs: 129 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.2 KiB/s wr, 16 op/s 2024-04-21T08:48:44.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:43 smithi086 ceph-mon[27939]: pgmap v309: 129 pgs: 129 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 14 KiB/s rd, 9.2 KiB/s wr, 16 op/s 2024-04-21T08:48:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.209+0000 7fa566a81640 1 -- 172.21.15.86:0/2128135753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa560106f20 msgr2=0x7fa560107300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.209+0000 7fa566a81640 1 --2- 172.21.15.86:0/2128135753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa560106f20 0x7fa560107300 secure :-1 s=READY pgs=1812 cs=0 l=1 rev1=1 crypto rx=0x7fa550009900 tx=0x7fa55002bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.209+0000 7fa566a81640 1 -- 172.21.15.86:0/2128135753 shutdown_connections 2024-04-21T08:48:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.209+0000 7fa566a81640 1 --2- 172.21.15.86:0/2128135753 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa56006b0d0 0x7fa56010ee10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.209+0000 7fa566a81640 1 --2- 172.21.15.86:0/2128135753 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa56006a730 0x7fa56006ab90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.209+0000 7fa566a81640 1 --2- 172.21.15.86:0/2128135753 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa560106f20 0x7fa560107300 unknown :-1 s=CLOSED pgs=1812 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.209+0000 7fa566a81640 1 -- 172.21.15.86:0/2128135753 >> 172.21.15.86:0/2128135753 conn(0x7fa560077c30 msgr2=0x7fa560078040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.209+0000 7fa566a81640 1 -- 172.21.15.86:0/2128135753 shutdown_connections 2024-04-21T08:48:45.209 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.209+0000 7fa566a81640 1 -- 172.21.15.86:0/2128135753 wait complete. 2024-04-21T08:48:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.210+0000 7fa566a81640 1 Processor -- start 2024-04-21T08:48:45.210 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.210+0000 7fa566a81640 1 -- start start 2024-04-21T08:48:45.211 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.210+0000 7fa566a81640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa56006a730 0x7fa560101200 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.210+0000 7fa566a81640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa56006b0d0 0x7fa560101740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.210+0000 7fa566a81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa560106f20 0x7fa560101c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.210+0000 7fa566a81640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa560112200 con 0x7fa560106f20 2024-04-21T08:48:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.210+0000 7fa566a81640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa560112080 con 0x7fa56006a730 2024-04-21T08:48:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.210+0000 7fa566a81640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa560112380 con 0x7fa56006b0d0 2024-04-21T08:48:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.210+0000 7fa56501e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa560106f20 0x7fa560101c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.210+0000 7fa56501e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa560106f20 0x7fa560101c80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37406/0 (socket says 172.21.15.86:37406) 2024-04-21T08:48:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.210+0000 7fa56501e640 1 -- 172.21.15.86:0/2369612718 learned_addr learned my addr 172.21.15.86:0/2369612718 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.210+0000 7fa56481d640 1 --2- 172.21.15.86:0/2369612718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa56006a730 0x7fa560101200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.210+0000 7fa55ffff640 1 --2- 172.21.15.86:0/2369612718 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa56006b0d0 0x7fa560101740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.210+0000 7fa56501e640 1 -- 172.21.15.86:0/2369612718 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa56006b0d0 msgr2=0x7fa560101740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.210+0000 7fa56501e640 1 --2- 172.21.15.86:0/2369612718 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa56006b0d0 0x7fa560101740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.212 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.210+0000 7fa56501e640 1 -- 172.21.15.86:0/2369612718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa56006a730 msgr2=0x7fa560101200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.210+0000 7fa56501e640 1 --2- 172.21.15.86:0/2369612718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa56006a730 0x7fa560101200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.210+0000 7fa56501e640 1 -- 172.21.15.86:0/2369612718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa5601a6b80 con 0x7fa560106f20 2024-04-21T08:48:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.210+0000 7fa55ffff640 1 --2- 172.21.15.86:0/2369612718 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa56006b0d0 0x7fa560101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.211+0000 7fa56501e640 1 --2- 172.21.15.86:0/2369612718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa560106f20 0x7fa560101c80 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fa55800eb10 tx=0x7fa55800c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.211+0000 7fa55dffb640 1 -- 172.21.15.86:0/2369612718 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa558004b20 con 0x7fa560106f20 2024-04-21T08:48:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.211+0000 7fa55dffb640 1 -- 172.21.15.86:0/2369612718 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa558004cc0 con 0x7fa560106f20 2024-04-21T08:48:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.211+0000 7fa566a81640 1 -- 172.21.15.86:0/2369612718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa5601a6e70 con 0x7fa560106f20 2024-04-21T08:48:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.211+0000 7fa55dffb640 1 -- 172.21.15.86:0/2369612718 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa558010670 con 0x7fa560106f20 2024-04-21T08:48:45.213 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.211+0000 7fa566a81640 1 -- 172.21.15.86:0/2369612718 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa5601a7340 con 0x7fa560106f20 2024-04-21T08:48:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.213+0000 7fa55dffb640 1 -- 172.21.15.86:0/2369612718 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa558005000 con 0x7fa560106f20 2024-04-21T08:48:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.214+0000 7fa569671740 1 Processor -- start 2024-04-21T08:48:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.214+0000 7fa569671740 1 -- start start 2024-04-21T08:48:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.214+0000 7fa55dffb640 1 --2- 172.21.15.86:0/2369612718 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa5480a3df0 0x7fa5480a62b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.214+0000 7fa569671740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4e60e18d0 0x55d4e60e0620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.214+0000 7fa55ffff640 1 --2- 172.21.15.86:0/2369612718 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa5480a3df0 0x7fa5480a62b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.214+0000 7fa569671740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4e60e0b60 0x55d4e60e0fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.214+0000 7fa569671740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4e60ea0e0 0x55d4e60ec4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.214+0000 7fa569671740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d4e60458d0 con 0x55d4e60e0b60 2024-04-21T08:48:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.214+0000 7fa569671740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d4e5fe2e80 con 0x55d4e60e18d0 2024-04-21T08:48:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.214+0000 7fa569671740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d4e6041890 con 0x55d4e60ea0e0 2024-04-21T08:48:45.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.214+0000 7fa55dffb640 1 -- 172.21.15.86:0/2369612718 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7fa5580d1160 con 0x7fa560106f20 2024-04-21T08:48:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.214+0000 7fa55ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4e60e0b60 0x55d4e60e0fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.214+0000 7fa55ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4e60e0b60 0x55d4e60e0fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37414/0 (socket says 172.21.15.86:37414) 2024-04-21T08:48:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.214+0000 7fa55ffff640 1 -- 172.21.15.86:0/4183787049 learned_addr learned my addr 172.21.15.86:0/4183787049 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.215+0000 7fa56481d640 1 --2- 172.21.15.86:0/4183787049 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4e60e18d0 0x55d4e60e0620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.215+0000 7fa56501e640 1 --2- 172.21.15.86:0/4183787049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4e60ea0e0 0x55d4e60ec4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.215+0000 7fa55ffff640 1 -- 172.21.15.86:0/4183787049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4e60ea0e0 msgr2=0x55d4e60ec4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.215+0000 7fa55ffff640 1 --2- 172.21.15.86:0/4183787049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4e60ea0e0 0x55d4e60ec4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.215+0000 7fa55ffff640 1 -- 172.21.15.86:0/4183787049 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4e60e18d0 msgr2=0x55d4e60e0620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.215+0000 7fa55ffff640 1 --2- 172.21.15.86:0/4183787049 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4e60e18d0 0x55d4e60e0620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.215+0000 7fa55ffff640 1 -- 172.21.15.86:0/4183787049 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4e6004320 con 0x55d4e60e0b60 2024-04-21T08:48:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.215+0000 7fa55ffff640 1 --2- 172.21.15.86:0/2369612718 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa5480a3df0 0x7fa5480a62b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa55400bfd0 tx=0x7fa55400bc80 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.215+0000 7fa55ffff640 1 --2- 172.21.15.86:0/4183787049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4e60e0b60 0x55d4e60e0fe0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fa554011750 tx=0x7fa55401b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bc3a85965b496f45 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.215+0000 7fa546ffd640 1 -- 172.21.15.86:0/4183787049 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55400fc00 con 0x55d4e60e0b60 2024-04-21T08:48:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.215+0000 7fa546ffd640 1 -- 172.21.15.86:0/4183787049 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa55400fda0 con 0x55d4e60e0b60 2024-04-21T08:48:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.215+0000 7fa546ffd640 1 -- 172.21.15.86:0/4183787049 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5540047a0 con 0x55d4e60e0b60 2024-04-21T08:48:45.221 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.216+0000 7fa569671740 1 -- 172.21.15.86:0/4183787049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4e60e0b60 msgr2=0x55d4e60e0fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.216+0000 7fa569671740 1 --2- 172.21.15.86:0/4183787049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4e60e0b60 0x55d4e60e0fe0 secure :-1 s=READY pgs=1814 cs=0 l=1 rev1=1 crypto rx=0x7fa554011750 tx=0x7fa55401b000 comp rx=0 tx=0).stop 2024-04-21T08:48:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.216+0000 7fa569671740 1 -- 172.21.15.86:0/4183787049 shutdown_connections 2024-04-21T08:48:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.216+0000 7fa569671740 1 --2- 172.21.15.86:0/4183787049 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4e60ea0e0 0x55d4e60ec4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.216+0000 7fa569671740 1 --2- 172.21.15.86:0/4183787049 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4e60e0b60 0x55d4e60e0fe0 unknown :-1 s=CLOSED pgs=1814 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.216+0000 7fa569671740 1 --2- 172.21.15.86:0/4183787049 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4e60e18d0 0x55d4e60e0620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.216+0000 7fa569671740 1 -- 172.21.15.86:0/4183787049 >> 172.21.15.86:0/4183787049 conn(0x55d4e60a7c90 msgr2=0x55d4e60db850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.216+0000 7fa569671740 1 -- 172.21.15.86:0/4183787049 shutdown_connections 2024-04-21T08:48:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.216+0000 7fa569671740 1 -- 172.21.15.86:0/4183787049 wait complete. 2024-04-21T08:48:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.216+0000 7fa569671740 1 -- start start 2024-04-21T08:48:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.216+0000 7fa569671740 1 Processor -- start 2024-04-21T08:48:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.217+0000 7fa569671740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4e60e0b60 0x55d4e60e9aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.217+0000 7fa569671740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4e60e18d0 0x55d4e60e6e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.222 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.217+0000 7fa569671740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4e60ea0e0 0x55d4e60e7440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.217+0000 7fa569671740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d4e60ecdb0 con 0x55d4e60e18d0 2024-04-21T08:48:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.217+0000 7fa569671740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d4e5fcdeb0 con 0x55d4e60e0b60 2024-04-21T08:48:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.217+0000 7fa569671740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d4e60ecf30 con 0x55d4e60ea0e0 2024-04-21T08:48:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.217+0000 7fa569671740 10 client.0 _ll_register_callbacks cb 0x55d4e5ffab40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.217+0000 7fa55ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4e60e18d0 0x55d4e60e6e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.217+0000 7fa55ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4e60e18d0 0x55d4e60e6e30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37424/0 (socket says 172.21.15.86:37424) 2024-04-21T08:48:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.217+0000 7fa55ffff640 1 -- 172.21.15.86:0/1907472947 learned_addr learned my addr 172.21.15.86:0/1907472947 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.217+0000 7fa56501e640 1 --2- 172.21.15.86:0/1907472947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4e60ea0e0 0x55d4e60e7440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.217+0000 7fa56481d640 1 --2- 172.21.15.86:0/1907472947 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4e60e0b60 0x55d4e60e9aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.217+0000 7fa55ffff640 1 -- 172.21.15.86:0/1907472947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4e60ea0e0 msgr2=0x55d4e60e7440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.217+0000 7fa55ffff640 1 --2- 172.21.15.86:0/1907472947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4e60ea0e0 0x55d4e60e7440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.217+0000 7fa55ffff640 1 -- 172.21.15.86:0/1907472947 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4e60e0b60 msgr2=0x55d4e60e9aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.217+0000 7fa55ffff640 1 --2- 172.21.15.86:0/1907472947 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4e60e0b60 0x55d4e60e9aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.217+0000 7fa55ffff640 1 -- 172.21.15.86:0/1907472947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4e5d58210 con 0x55d4e60e18d0 2024-04-21T08:48:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.217+0000 7fa56501e640 1 --2- 172.21.15.86:0/1907472947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4e60ea0e0 0x55d4e60e7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:45.223 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.217+0000 7fa56481d640 1 --2- 172.21.15.86:0/1907472947 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4e60e0b60 0x55d4e60e9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.217+0000 7fa55ffff640 1 --2- 172.21.15.86:0/1907472947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4e60e18d0 0x55d4e60e6e30 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fa554005380 tx=0x7fa5540147b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d453d35f45686264 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.217+0000 7fa53ffff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.218+0000 7fa53ffff640 1 -- 172.21.15.86:0/1907472947 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa554004880 con 0x55d4e60e18d0 2024-04-21T08:48:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.218+0000 7fa53ffff640 1 -- 172.21.15.86:0/1907472947 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7fa554004a20 con 0x55d4e60e18d0 2024-04-21T08:48:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.218+0000 7fa569671740 1 -- 172.21.15.86:0/1907472947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa55400fda0 con 0x55d4e60e18d0 2024-04-21T08:48:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.218+0000 7fa53ffff640 1 -- 172.21.15.86:0/1907472947 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa55403a050 con 0x55d4e60e18d0 2024-04-21T08:48:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.218+0000 7fa569671740 10 client.19662 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.218+0000 7fa569671740 1 -- 172.21.15.86:0/1907472947 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d4e5e24620 con 0x55d4e60e18d0 2024-04-21T08:48:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.218+0000 7fa53ffff640 1 -- 172.21.15.86:0/1907472947 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa55400fda0 con 0x55d4e60e18d0 2024-04-21T08:48:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.218+0000 7fa53ffff640 1 -- 172.21.15.86:0/1907472947 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa55400f050 con 0x55d4e60e18d0 2024-04-21T08:48:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.218+0000 7fa569671740 10 client.19662 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.218+0000 7fa569671740 10 client.19662 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.218+0000 7fa569671740 20 client.19662 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:45.224 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.218+0000 7fa569671740 1 --2- 172.21.15.86:0/1907472947 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d4e60fa050 0x55d4e60fc440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.218+0000 7fa569671740 4 client.19662 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.218+0000 7fa569671740 1 -- 172.21.15.86:0/1907472947 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4e6059880 con 0x55d4e60fa050 2024-04-21T08:48:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.219+0000 7fa56481d640 1 --2- 172.21.15.86:0/1907472947 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d4e60fa050 0x55d4e60fc440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.219+0000 7fa53ffff640 10 client.19662 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.219+0000 7fa56481d640 1 --2- 172.21.15.86:0/1907472947 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d4e60fa050 0x55d4e60fc440 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2041827e58b6d6ce server_cookie=d736ef025232cb78 in_seq=0 out_seq=0 2024-04-21T08:48:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.220+0000 7fa53ffff640 1 -- 172.21.15.86:0/1907472947 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d4e6059880 con 0x55d4e60fa050 2024-04-21T08:48:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.220+0000 7fa53ffff640 10 client.19662 handle_command_reply: tid=0 2024-04-21T08:48:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.220+0000 7fa569671740 1 client.19662 shutdown 2024-04-21T08:48:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.220+0000 7fa569671740 1 -- 172.21.15.86:0/1907472947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4e60e18d0 msgr2=0x55d4e60e6e30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.220+0000 7fa569671740 1 --2- 172.21.15.86:0/1907472947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4e60e18d0 0x55d4e60e6e30 secure :-1 s=READY pgs=1815 cs=0 l=1 rev1=1 crypto rx=0x7fa554005380 tx=0x7fa5540147b0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.220+0000 7fa569671740 1 -- 172.21.15.86:0/1907472947 shutdown_connections 2024-04-21T08:48:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.220+0000 7fa569671740 1 --2- 172.21.15.86:0/1907472947 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d4e60fa050 0x55d4e60fc440 crc :-1 s=READY pgs=293 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.220+0000 7fa569671740 1 --2- 172.21.15.86:0/1907472947 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4e60ea0e0 0x55d4e60e7440 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.225 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.220+0000 7fa569671740 1 --2- 172.21.15.86:0/1907472947 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4e60e18d0 0x55d4e60e6e30 unknown :-1 s=CLOSED pgs=1815 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.220+0000 7fa569671740 1 --2- 172.21.15.86:0/1907472947 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4e60e0b60 0x55d4e60e9aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.220+0000 7fa53ffff640 0 client.19662 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.220+0000 7fa569671740 1 -- 172.21.15.86:0/1907472947 >> 172.21.15.86:0/1907472947 conn(0x55d4e60a7c90 msgr2=0x55d4e60dd5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.220+0000 7fa569671740 1 -- 172.21.15.86:0/1907472947 shutdown_connections 2024-04-21T08:48:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.220+0000 7fa569671740 1 -- 172.21.15.86:0/1907472947 wait complete. 2024-04-21T08:48:45.226 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.220+0000 7fa569671740 20 client.19662 trim_cache size 0 max 16384 2024-04-21T08:48:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.235+0000 7fa569671740 1 Processor -- start 2024-04-21T08:48:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.235+0000 7fa569671740 1 -- start start 2024-04-21T08:48:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.235+0000 7fa569671740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4e60e0b60 0x55d4e60f5d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.235+0000 7fa569671740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4e60e18d0 0x55d4e60e5ce0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.235+0000 7fa569671740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4e60ea0e0 0x55d4e60e9560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.235+0000 7fa569671740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa5601b14c0 con 0x55d4e60ea0e0 2024-04-21T08:48:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.235+0000 7fa569671740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa5601b3000 con 0x55d4e60e18d0 2024-04-21T08:48:45.236 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.235+0000 7fa569671740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa5601b1b90 con 0x55d4e60e0b60 2024-04-21T08:48:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.235+0000 7fa56501e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4e60ea0e0 0x55d4e60e9560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.235+0000 7fa56501e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4e60ea0e0 0x55d4e60e9560 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37432/0 (socket says 172.21.15.86:37432) 2024-04-21T08:48:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.235+0000 7fa56501e640 1 -- 172.21.15.86:0/2925226441 learned_addr learned my addr 172.21.15.86:0/2925226441 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.235+0000 7fa55ffff640 1 --2- 172.21.15.86:0/2925226441 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4e60e18d0 0x55d4e60e5ce0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.235+0000 7fa56481d640 1 --2- 172.21.15.86:0/2925226441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4e60e0b60 0x55d4e60f5d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.235+0000 7fa56501e640 1 -- 172.21.15.86:0/2925226441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4e60e0b60 msgr2=0x55d4e60f5d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.235+0000 7fa56501e640 1 --2- 172.21.15.86:0/2925226441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4e60e0b60 0x55d4e60f5d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.235+0000 7fa56501e640 1 -- 172.21.15.86:0/2925226441 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4e60e18d0 msgr2=0x55d4e60e5ce0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.235+0000 7fa56501e640 1 --2- 172.21.15.86:0/2925226441 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4e60e18d0 0x55d4e60e5ce0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.235+0000 7fa56501e640 1 -- 172.21.15.86:0/2925226441 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4e60dfe30 con 0x55d4e60ea0e0 2024-04-21T08:48:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.235+0000 7fa56501e640 1 --2- 172.21.15.86:0/2925226441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4e60ea0e0 0x55d4e60e9560 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fa55809ca10 tx=0x7fa55800cee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c8166f7d51a00c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.236+0000 7fa5467fc640 1 -- 172.21.15.86:0/2925226441 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa55809c670 con 0x55d4e60ea0e0 2024-04-21T08:48:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.236+0000 7fa5467fc640 1 -- 172.21.15.86:0/2925226441 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa5580b6270 con 0x55d4e60ea0e0 2024-04-21T08:48:45.237 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.236+0000 7fa5467fc640 1 -- 172.21.15.86:0/2925226441 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa558099380 con 0x55d4e60ea0e0 2024-04-21T08:48:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.236+0000 7fa569671740 1 -- 172.21.15.86:0/2925226441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4e60ea0e0 msgr2=0x55d4e60e9560 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.236+0000 7fa569671740 1 --2- 172.21.15.86:0/2925226441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4e60ea0e0 0x55d4e60e9560 secure :-1 s=READY pgs=1816 cs=0 l=1 rev1=1 crypto rx=0x7fa55809ca10 tx=0x7fa55800cee0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.236+0000 7fa569671740 1 -- 172.21.15.86:0/2925226441 shutdown_connections 2024-04-21T08:48:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.236+0000 7fa569671740 1 --2- 172.21.15.86:0/2925226441 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4e60ea0e0 0x55d4e60e9560 unknown :-1 s=CLOSED pgs=1816 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.236+0000 7fa569671740 1 --2- 172.21.15.86:0/2925226441 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4e60e18d0 0x55d4e60e5ce0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.236+0000 7fa569671740 1 --2- 172.21.15.86:0/2925226441 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4e60e0b60 0x55d4e60f5d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.236+0000 7fa569671740 1 -- 172.21.15.86:0/2925226441 >> 172.21.15.86:0/2925226441 conn(0x55d4e60a7c90 msgr2=0x55d4e60e55d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.236+0000 7fa569671740 1 -- 172.21.15.86:0/2925226441 shutdown_connections 2024-04-21T08:48:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.236+0000 7fa569671740 1 -- 172.21.15.86:0/2925226441 wait complete. 2024-04-21T08:48:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.237+0000 7fa569671740 1 -- start start 2024-04-21T08:48:45.238 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.237+0000 7fa569671740 1 Processor -- start 2024-04-21T08:48:45.241 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.237+0000 7fa569671740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4e60e0b60 0x55d4e60f2780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.241 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.237+0000 7fa569671740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4e60e18d0 0x55d4e60e4190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.241 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.237+0000 7fa569671740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4e60ea0e0 0x55d4e60e4880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.237+0000 7fa569671740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa5601b2930 con 0x55d4e60e18d0 2024-04-21T08:48:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.237+0000 7fa569671740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa5601b2260 con 0x55d4e60e0b60 2024-04-21T08:48:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.237+0000 7fa569671740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa5601af4c0 con 0x55d4e60ea0e0 2024-04-21T08:48:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.237+0000 7fa569671740 10 client.0 _ll_register_callbacks cb 0x7fa550009780 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.237+0000 7fa55ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4e60e18d0 0x55d4e60e4190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.237+0000 7fa55ffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4e60e18d0 0x55d4e60e4190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37438/0 (socket says 172.21.15.86:37438) 2024-04-21T08:48:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.237+0000 7fa55ffff640 1 -- 172.21.15.86:0/286854019 learned_addr learned my addr 172.21.15.86:0/286854019 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.237+0000 7fa56481d640 1 --2- 172.21.15.86:0/286854019 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4e60e0b60 0x55d4e60f2780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.237+0000 7fa56501e640 1 --2- 172.21.15.86:0/286854019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4e60ea0e0 0x55d4e60e4880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.237+0000 7fa55ffff640 1 -- 172.21.15.86:0/286854019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4e60ea0e0 msgr2=0x55d4e60e4880 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.237+0000 7fa55ffff640 1 --2- 172.21.15.86:0/286854019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4e60ea0e0 0x55d4e60e4880 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.237+0000 7fa55ffff640 1 -- 172.21.15.86:0/286854019 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4e60e0b60 msgr2=0x55d4e60f2780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:45.242 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.237+0000 7fa55ffff640 1 --2- 172.21.15.86:0/286854019 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4e60e0b60 0x55d4e60f2780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.237+0000 7fa55ffff640 1 -- 172.21.15.86:0/286854019 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4e60d6f30 con 0x55d4e60e18d0 2024-04-21T08:48:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.238+0000 7fa55ffff640 1 --2- 172.21.15.86:0/286854019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4e60e18d0 0x55d4e60e4190 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fa554028490 tx=0x7fa554005e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4fd9c21e834910d2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.238+0000 7fa53ffff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.238+0000 7fa53ffff640 1 -- 172.21.15.86:0/286854019 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa5540053b0 con 0x55d4e60e18d0 2024-04-21T08:48:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.238+0000 7fa53ffff640 1 -- 172.21.15.86:0/286854019 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7fa55400fbd0 con 0x55d4e60e18d0 2024-04-21T08:48:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.238+0000 7fa569671740 1 -- 172.21.15.86:0/286854019 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d4e5fe53d0 con 0x55d4e60e18d0 2024-04-21T08:48:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.238+0000 7fa53ffff640 1 -- 172.21.15.86:0/286854019 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa55404c050 con 0x55d4e60e18d0 2024-04-21T08:48:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.238+0000 7fa569671740 10 client.19668 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.238+0000 7fa569671740 1 -- 172.21.15.86:0/286854019 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d4e6160370 con 0x55d4e60e18d0 2024-04-21T08:48:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.238+0000 7fa53ffff640 1 -- 172.21.15.86:0/286854019 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d4e5fe53d0 con 0x55d4e60e18d0 2024-04-21T08:48:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.238+0000 7fa53ffff640 1 -- 172.21.15.86:0/286854019 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa554033450 con 0x55d4e60e18d0 2024-04-21T08:48:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.238+0000 7fa569671740 10 client.19668 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.238+0000 7fa569671740 10 client.19668 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:45.243 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.238+0000 7fa569671740 20 client.19668 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.238+0000 7fa569671740 1 --2- 172.21.15.86:0/286854019 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d4e6160b00 0x55d4e6162f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.238+0000 7fa569671740 4 client.19668 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:48:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.238+0000 7fa569671740 1 -- 172.21.15.86:0/286854019 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55d4e60532f0 con 0x55d4e6160b00 2024-04-21T08:48:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.239+0000 7fa56481d640 1 --2- 172.21.15.86:0/286854019 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d4e6160b00 0x55d4e6162f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.239+0000 7fa53ffff640 10 client.19668 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:45.244 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.239+0000 7fa56481d640 1 --2- 172.21.15.86:0/286854019 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d4e6160b00 0x55d4e6162f20 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1690c4309a3de051 server_cookie=e7242dc3d3e4043d in_seq=0 out_seq=0 2024-04-21T08:48:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.246+0000 7fa53ffff640 1 -- 172.21.15.86:0/286854019 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55d4e60532f0 con 0x55d4e6160b00 2024-04-21T08:48:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.246+0000 7fa53ffff640 10 client.19668 handle_command_reply: tid=0 2024-04-21T08:48:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.246+0000 7fa569671740 1 client.19668 shutdown 2024-04-21T08:48:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.247+0000 7fa569671740 1 -- 172.21.15.86:0/286854019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4e60e18d0 msgr2=0x55d4e60e4190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.247+0000 7fa569671740 1 --2- 172.21.15.86:0/286854019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4e60e18d0 0x55d4e60e4190 secure :-1 s=READY pgs=1817 cs=0 l=1 rev1=1 crypto rx=0x7fa554028490 tx=0x7fa554005e90 comp rx=0 tx=0).stop 2024-04-21T08:48:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.247+0000 7fa569671740 1 -- 172.21.15.86:0/286854019 shutdown_connections 2024-04-21T08:48:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.247+0000 7fa569671740 1 --2- 172.21.15.86:0/286854019 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d4e6160b00 0x55d4e6162f20 crc :-1 s=READY pgs=294 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.247+0000 7fa569671740 1 --2- 172.21.15.86:0/286854019 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4e60ea0e0 0x55d4e60e4880 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.247+0000 7fa569671740 1 --2- 172.21.15.86:0/286854019 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4e60e18d0 0x55d4e60e4190 unknown :-1 s=CLOSED pgs=1817 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.247+0000 7fa53ffff640 0 client.19668 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.247+0000 7fa569671740 1 --2- 172.21.15.86:0/286854019 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4e60e0b60 0x55d4e60f2780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.247+0000 7fa569671740 1 -- 172.21.15.86:0/286854019 >> 172.21.15.86:0/286854019 conn(0x55d4e60a7c90 msgr2=0x55d4e60e5e60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.247+0000 7fa569671740 1 -- 172.21.15.86:0/286854019 shutdown_connections 2024-04-21T08:48:45.247 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.247+0000 7fa569671740 1 -- 172.21.15.86:0/286854019 wait complete. 2024-04-21T08:48:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.247+0000 7fa569671740 20 client.19668 trim_cache size 0 max 16384 2024-04-21T08:48:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:48:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:48:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:48:45.248 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:48:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.247+0000 7fa566a81640 1 -- 172.21.15.86:0/2369612718 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa5480a3df0 msgr2=0x7fa5480a62b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.247+0000 7fa566a81640 1 --2- 172.21.15.86:0/2369612718 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa5480a3df0 0x7fa5480a62b0 secure :-1 s=READY pgs=299 cs=0 l=1 rev1=1 crypto rx=0x7fa55400bfd0 tx=0x7fa55400bc80 comp rx=0 tx=0).stop 2024-04-21T08:48:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.247+0000 7fa566a81640 1 -- 172.21.15.86:0/2369612718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa560106f20 msgr2=0x7fa560101c80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.247+0000 7fa566a81640 1 --2- 172.21.15.86:0/2369612718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa560106f20 0x7fa560101c80 secure :-1 s=READY pgs=1813 cs=0 l=1 rev1=1 crypto rx=0x7fa55800eb10 tx=0x7fa55800c2d0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.248+0000 7fa566a81640 1 -- 172.21.15.86:0/2369612718 shutdown_connections 2024-04-21T08:48:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.248+0000 7fa566a81640 1 --2- 172.21.15.86:0/2369612718 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa5480a3df0 0x7fa5480a62b0 unknown :-1 s=CLOSED pgs=299 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.248+0000 7fa566a81640 1 --2- 172.21.15.86:0/2369612718 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa560106f20 0x7fa560101c80 unknown :-1 s=CLOSED pgs=1813 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.248+0000 7fa566a81640 1 --2- 172.21.15.86:0/2369612718 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa56006b0d0 0x7fa560101740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.248+0000 7fa566a81640 1 --2- 172.21.15.86:0/2369612718 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa56006a730 0x7fa560101200 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.248+0000 7fa566a81640 1 -- 172.21.15.86:0/2369612718 >> 172.21.15.86:0/2369612718 conn(0x7fa560077c30 msgr2=0x7fa5600fedf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.248+0000 7fa566a81640 1 -- 172.21.15.86:0/2369612718 shutdown_connections 2024-04-21T08:48:45.249 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:45.248+0000 7fa566a81640 1 -- 172.21.15.86:0/2369612718 wait complete. 2024-04-21T08:48:46.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:45 smithi088 ceph-mon[31259]: pgmap v310: 129 pgs: 129 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-04-21T08:48:46.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:45 smithi179 ceph-mon[31829]: pgmap v310: 129 pgs: 129 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-04-21T08:48:46.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:45 smithi086 ceph-mon[27939]: pgmap v310: 129 pgs: 129 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-04-21T08:48:48.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:47 smithi088 ceph-mon[31259]: pgmap v311: 129 pgs: 129 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-04-21T08:48:48.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:47 smithi179 ceph-mon[31829]: pgmap v311: 129 pgs: 129 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-04-21T08:48:48.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:47 smithi086 ceph-mon[27939]: pgmap v311: 129 pgs: 129 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 1.4 KiB/s wr, 9 op/s 2024-04-21T08:48:50.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:49 smithi088 ceph-mon[31259]: pgmap v312: 129 pgs: 129 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 3.3 KiB/s wr, 15 op/s 2024-04-21T08:48:50.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:49 smithi179 ceph-mon[31829]: pgmap v312: 129 pgs: 129 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 3.3 KiB/s wr, 15 op/s 2024-04-21T08:48:50.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:49 smithi086 ceph-mon[27939]: pgmap v312: 129 pgs: 129 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 3.3 KiB/s wr, 15 op/s 2024-04-21T08:48:52.169 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:51 smithi086 ceph-mon[27939]: pgmap v313: 129 pgs: 129 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-04-21T08:48:52.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:51 smithi088 ceph-mon[31259]: pgmap v313: 129 pgs: 129 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-04-21T08:48:52.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:51 smithi179 ceph-mon[31829]: pgmap v313: 129 pgs: 129 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 7.2 KiB/s rd, 3.3 KiB/s wr, 11 op/s 2024-04-21T08:48:54.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:53 smithi088 ceph-mon[31259]: pgmap v314: 129 pgs: 129 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 3.4 KiB/s wr, 15 op/s 2024-04-21T08:48:54.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:53 smithi179 ceph-mon[31829]: pgmap v314: 129 pgs: 129 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 3.4 KiB/s wr, 15 op/s 2024-04-21T08:48:54.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:53 smithi086 ceph-mon[27939]: pgmap v314: 129 pgs: 129 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 9.2 KiB/s rd, 3.4 KiB/s wr, 15 op/s 2024-04-21T08:48:55.259 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:48:55.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.371+0000 7f9a63bc7640 1 -- 172.21.15.86:0/497423615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a5c106f10 msgr2=0x7f9a5c1072f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:55.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.371+0000 7f9a63bc7640 1 --2- 172.21.15.86:0/497423615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a5c106f10 0x7f9a5c1072f0 secure :-1 s=READY pgs=1818 cs=0 l=1 rev1=1 crypto rx=0x7f9a58009900 tx=0x7f9a5802bc30 comp rx=0 tx=0).stop 2024-04-21T08:48:55.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.371+0000 7f9a63bc7640 1 -- 172.21.15.86:0/497423615 shutdown_connections 2024-04-21T08:48:55.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.371+0000 7f9a63bc7640 1 --2- 172.21.15.86:0/497423615 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a5c101ad0 0x7f9a5c10dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.371+0000 7f9a63bc7640 1 --2- 172.21.15.86:0/497423615 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a5c101130 0x7f9a5c101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.371+0000 7f9a63bc7640 1 --2- 172.21.15.86:0/497423615 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a5c106f10 0x7f9a5c1072f0 unknown :-1 s=CLOSED pgs=1818 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.371+0000 7f9a63bc7640 1 -- 172.21.15.86:0/497423615 >> 172.21.15.86:0/497423615 conn(0x7f9a5c077e30 msgr2=0x7f9a5c078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:55.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.371+0000 7f9a63bc7640 1 -- 172.21.15.86:0/497423615 shutdown_connections 2024-04-21T08:48:55.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.371+0000 7f9a63bc7640 1 -- 172.21.15.86:0/497423615 wait complete. 2024-04-21T08:48:55.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.372+0000 7f9a63bc7640 1 Processor -- start 2024-04-21T08:48:55.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.372+0000 7f9a63bc7640 1 -- start start 2024-04-21T08:48:55.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.372+0000 7f9a63bc7640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a5c101130 0x7f9a5c19e470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.372+0000 7f9a63bc7640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a5c101ad0 0x7f9a5c19e9b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.372+0000 7f9a63bc7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a5c106f10 0x7f9a5c198620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.372+0000 7f9a63bc7640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5c112260 con 0x7f9a5c106f10 2024-04-21T08:48:55.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.372+0000 7f9a63bc7640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5c1120e0 con 0x7f9a5c101130 2024-04-21T08:48:55.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.372+0000 7f9a63bc7640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9a5c1123e0 con 0x7f9a5c101ad0 2024-04-21T08:48:55.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.372+0000 7f9a62164640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a5c106f10 0x7f9a5c198620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.372+0000 7f9a62164640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a5c106f10 0x7f9a5c198620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38008/0 (socket says 172.21.15.86:38008) 2024-04-21T08:48:55.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.372+0000 7f9a62164640 1 -- 172.21.15.86:0/694581997 learned_addr learned my addr 172.21.15.86:0/694581997 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:55.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.372+0000 7f9a61963640 1 --2- 172.21.15.86:0/694581997 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a5c101130 0x7f9a5c19e470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.374 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.372+0000 7f9a61162640 1 --2- 172.21.15.86:0/694581997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a5c101ad0 0x7f9a5c19e9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.372+0000 7f9a62164640 1 -- 172.21.15.86:0/694581997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a5c101ad0 msgr2=0x7f9a5c19e9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:55.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.372+0000 7f9a62164640 1 --2- 172.21.15.86:0/694581997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a5c101ad0 0x7f9a5c19e9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.372+0000 7f9a62164640 1 -- 172.21.15.86:0/694581997 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a5c101130 msgr2=0x7f9a5c19e470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:55.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.372+0000 7f9a62164640 1 --2- 172.21.15.86:0/694581997 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a5c101130 0x7f9a5c19e470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.372+0000 7f9a62164640 1 -- 172.21.15.86:0/694581997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a5c198da0 con 0x7f9a5c106f10 2024-04-21T08:48:55.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.372+0000 7f9a62164640 1 --2- 172.21.15.86:0/694581997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a5c106f10 0x7f9a5c198620 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f9a5400eb10 tx=0x7f9a5400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:55.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.373+0000 7f9a52ffd640 1 -- 172.21.15.86:0/694581997 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a54004b20 con 0x7f9a5c106f10 2024-04-21T08:48:55.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.373+0000 7f9a52ffd640 1 -- 172.21.15.86:0/694581997 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a54004cc0 con 0x7f9a5c106f10 2024-04-21T08:48:55.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.373+0000 7f9a63bc7640 1 -- 172.21.15.86:0/694581997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a5c199090 con 0x7f9a5c106f10 2024-04-21T08:48:55.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.373+0000 7f9a52ffd640 1 -- 172.21.15.86:0/694581997 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a54010670 con 0x7f9a5c106f10 2024-04-21T08:48:55.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.373+0000 7f9a63bc7640 1 -- 172.21.15.86:0/694581997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a5c1a5330 con 0x7f9a5c106f10 2024-04-21T08:48:55.375 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.374+0000 7f9a63bc7640 1 -- 172.21.15.86:0/694581997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a5c069230 con 0x7f9a5c106f10 2024-04-21T08:48:55.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.375+0000 7f9a52ffd640 1 -- 172.21.15.86:0/694581997 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9a54005000 con 0x7f9a5c106f10 2024-04-21T08:48:55.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.376+0000 7f9a52ffd640 1 --2- 172.21.15.86:0/694581997 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f9a480a3d20 0x7f9a480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.376+0000 7f9a61963640 1 --2- 172.21.15.86:0/694581997 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f9a480a3d20 0x7f9a480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.376+0000 7f9a52ffd640 1 -- 172.21.15.86:0/694581997 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f9a540d1340 con 0x7f9a5c106f10 2024-04-21T08:48:55.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.376+0000 7f9a61963640 1 --2- 172.21.15.86:0/694581997 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f9a480a3d20 0x7f9a480a61e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9a58006010 tx=0x7f9a58005e40 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:55.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.380+0000 7f9a52ffd640 1 -- 172.21.15.86:0/694581997 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9a54053ff0 con 0x7f9a5c106f10 2024-04-21T08:48:55.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.595+0000 7f9a63bc7640 1 -- 172.21.15.86:0/694581997 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a5c199b50 con 0x7f9a5c106f10 2024-04-21T08:48:55.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.596+0000 7f9a52ffd640 1 -- 172.21.15.86:0/694581997 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7f9a5c199b50 con 0x7f9a5c106f10 2024-04-21T08:48:55.655 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:48:55.655 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:38:45.202126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24398,24394,24455,24472]},"id":1}]} 2024-04-21T08:48:55.656 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-21T08:48:55.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.598+0000 7f9a63bc7640 1 -- 172.21.15.86:0/694581997 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f9a480a3d20 msgr2=0x7f9a480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:55.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.598+0000 7f9a63bc7640 1 --2- 172.21.15.86:0/694581997 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f9a480a3d20 0x7f9a480a61e0 secure :-1 s=READY pgs=300 cs=0 l=1 rev1=1 crypto rx=0x7f9a58006010 tx=0x7f9a58005e40 comp rx=0 tx=0).stop 2024-04-21T08:48:55.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.598+0000 7f9a63bc7640 1 -- 172.21.15.86:0/694581997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a5c106f10 msgr2=0x7f9a5c198620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:55.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.598+0000 7f9a63bc7640 1 --2- 172.21.15.86:0/694581997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a5c106f10 0x7f9a5c198620 secure :-1 s=READY pgs=1819 cs=0 l=1 rev1=1 crypto rx=0x7f9a5400eb10 tx=0x7f9a5400c2d0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.598+0000 7f9a63bc7640 1 -- 172.21.15.86:0/694581997 shutdown_connections 2024-04-21T08:48:55.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.598+0000 7f9a63bc7640 1 --2- 172.21.15.86:0/694581997 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f9a480a3d20 0x7f9a480a61e0 unknown :-1 s=CLOSED pgs=300 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.598+0000 7f9a63bc7640 1 --2- 172.21.15.86:0/694581997 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a5c106f10 0x7f9a5c198620 unknown :-1 s=CLOSED pgs=1819 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.598+0000 7f9a63bc7640 1 --2- 172.21.15.86:0/694581997 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a5c101ad0 0x7f9a5c19e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.598+0000 7f9a63bc7640 1 --2- 172.21.15.86:0/694581997 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a5c101130 0x7f9a5c19e470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.598+0000 7f9a63bc7640 1 -- 172.21.15.86:0/694581997 >> 172.21.15.86:0/694581997 conn(0x7f9a5c077e30 msgr2=0x7f9a5c0ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:55.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.598+0000 7f9a63bc7640 1 -- 172.21.15.86:0/694581997 shutdown_connections 2024-04-21T08:48:55.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.598+0000 7f9a63bc7640 1 -- 172.21.15.86:0/694581997 wait complete. 2024-04-21T08:48:55.658 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --set-id bea13946 --release --await' 2024-04-21T08:48:55.658 DEBUG:teuthology.orchestra.run.smithi086:> timeout 45 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24482 quiesce db --set-id bea13946 --release --await 2024-04-21T08:48:55.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.763+0000 7f85f36a8640 1 -- 172.21.15.86:0/346112792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85ec1031b0 msgr2=0x7f85ec103590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:55.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.763+0000 7f85f36a8640 1 --2- 172.21.15.86:0/346112792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85ec1031b0 0x7f85ec103590 secure :-1 s=READY pgs=1820 cs=0 l=1 rev1=1 crypto rx=0x7f85e4009ca0 tx=0x7f85e402c180 comp rx=0 tx=0).stop 2024-04-21T08:48:55.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.763+0000 7f85f36a8640 1 -- 172.21.15.86:0/346112792 shutdown_connections 2024-04-21T08:48:55.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.763+0000 7f85f36a8640 1 --2- 172.21.15.86:0/346112792 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85ec10c580 0x7f85ec10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.763+0000 7f85f36a8640 1 --2- 172.21.15.86:0/346112792 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85ec103ad0 0x7f85ec10bfb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.763+0000 7f85f36a8640 1 --2- 172.21.15.86:0/346112792 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85ec1031b0 0x7f85ec103590 unknown :-1 s=CLOSED pgs=1820 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.763+0000 7f85f36a8640 1 -- 172.21.15.86:0/346112792 >> 172.21.15.86:0/346112792 conn(0x7f85ec0771d0 msgr2=0x7f85ec0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:55.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.763+0000 7f85f36a8640 1 -- 172.21.15.86:0/346112792 shutdown_connections 2024-04-21T08:48:55.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.763+0000 7f85f36a8640 1 -- 172.21.15.86:0/346112792 wait complete. 2024-04-21T08:48:55.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.764+0000 7f85f36a8640 1 Processor -- start 2024-04-21T08:48:55.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.764+0000 7f85f36a8640 1 -- start start 2024-04-21T08:48:55.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.764+0000 7f85f36a8640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85ec1031b0 0x7f85ec1986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.764+0000 7f85f36a8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85ec103ad0 0x7f85ec198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.764+0000 7f85f36a8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85ec10c580 0x7f85ec19cf90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.764+0000 7f85f36a8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec1121d0 con 0x7f85ec103ad0 2024-04-21T08:48:55.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.764+0000 7f85f36a8640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec112050 con 0x7f85ec10c580 2024-04-21T08:48:55.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.764+0000 7f85f36a8640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec112350 con 0x7f85ec1031b0 2024-04-21T08:48:55.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.764+0000 7f85f0c43640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85ec103ad0 0x7f85ec198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.764+0000 7f85f0c43640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85ec103ad0 0x7f85ec198c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38030/0 (socket says 172.21.15.86:38030) 2024-04-21T08:48:55.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.764+0000 7f85f0c43640 1 -- 172.21.15.86:0/2536931066 learned_addr learned my addr 172.21.15.86:0/2536931066 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:55.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.764+0000 7f85f1444640 1 --2- 172.21.15.86:0/2536931066 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85ec1031b0 0x7f85ec1986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.765+0000 7f85f1c45640 1 --2- 172.21.15.86:0/2536931066 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85ec10c580 0x7f85ec19cf90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.765+0000 7f85f0c43640 1 -- 172.21.15.86:0/2536931066 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85ec1031b0 msgr2=0x7f85ec1986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:55.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.765+0000 7f85f0c43640 1 --2- 172.21.15.86:0/2536931066 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85ec1031b0 0x7f85ec1986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.765+0000 7f85f0c43640 1 -- 172.21.15.86:0/2536931066 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85ec10c580 msgr2=0x7f85ec19cf90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:55.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.765+0000 7f85f0c43640 1 --2- 172.21.15.86:0/2536931066 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85ec10c580 0x7f85ec19cf90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.765+0000 7f85f0c43640 1 -- 172.21.15.86:0/2536931066 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f85ec19d6c0 con 0x7f85ec103ad0 2024-04-21T08:48:55.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.765+0000 7f85f1c45640 1 --2- 172.21.15.86:0/2536931066 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85ec10c580 0x7f85ec19cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:55.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.765+0000 7f85f0c43640 1 --2- 172.21.15.86:0/2536931066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85ec103ad0 0x7f85ec198c10 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f85e000c920 tx=0x7f85e000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:55.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.765+0000 7f85ea7fc640 1 -- 172.21.15.86:0/2536931066 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e0007c70 con 0x7f85ec103ad0 2024-04-21T08:48:55.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.765+0000 7f85ea7fc640 1 -- 172.21.15.86:0/2536931066 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85e0007e10 con 0x7f85ec103ad0 2024-04-21T08:48:55.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.765+0000 7f85f36a8640 1 -- 172.21.15.86:0/2536931066 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f85ec19d9b0 con 0x7f85ec103ad0 2024-04-21T08:48:55.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.765+0000 7f85ea7fc640 1 -- 172.21.15.86:0/2536931066 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e0018820 con 0x7f85ec103ad0 2024-04-21T08:48:55.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.765+0000 7f85f36a8640 1 -- 172.21.15.86:0/2536931066 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f85ec1a5240 con 0x7f85ec103ad0 2024-04-21T08:48:55.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.767+0000 7f85ea7fc640 1 -- 172.21.15.86:0/2536931066 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f85e000f450 con 0x7f85ec103ad0 2024-04-21T08:48:55.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.768+0000 7f85f6379740 1 Processor -- start 2024-04-21T08:48:55.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.768+0000 7f85f6379740 1 -- start start 2024-04-21T08:48:55.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.768+0000 7f85ea7fc640 1 --2- 172.21.15.86:0/2536931066 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f85d40a3d20 0x7f85d40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.768+0000 7f85f6379740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c9a59a310 0x561c9a59a6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.768+0000 7f85f6379740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c9a59ac30 0x561c9a5a32d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.768+0000 7f85f6379740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c9a5a3810 0x561c9a5a5c00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.768+0000 7f85f6379740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561c9a4ff500 con 0x561c9a5a3810 2024-04-21T08:48:55.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.768+0000 7f85f6379740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x561c9a4a56a0 con 0x561c9a59ac30 2024-04-21T08:48:55.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.768+0000 7f85f1c45640 1 --2- 172.21.15.86:0/2536931066 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f85d40a3d20 0x7f85d40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.768+0000 7f85f6379740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561c9a4fb4c0 con 0x561c9a59a310 2024-04-21T08:48:55.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.768+0000 7f85ea7fc640 1 -- 172.21.15.86:0/2536931066 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f85e00d0d60 con 0x7f85ec103ad0 2024-04-21T08:48:55.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.768+0000 7f85f1c45640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c9a5a3810 0x561c9a5a5c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.768+0000 7f85f1444640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c9a59a310 0x561c9a59a6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.768+0000 7f85f1c45640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c9a5a3810 0x561c9a5a5c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38032/0 (socket says 172.21.15.86:38032) 2024-04-21T08:48:55.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.768+0000 7f85f1444640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c9a59a310 0x561c9a59a6f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.86:52020/0 (socket says 172.21.15.86:52020) 2024-04-21T08:48:55.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.768+0000 7f85f1c45640 1 -- 172.21.15.86:0/609899382 learned_addr learned my addr 172.21.15.86:0/609899382 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:55.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.768+0000 7f85f0c43640 1 --2- 172.21.15.86:0/609899382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c9a59ac30 0x561c9a5a32d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.768+0000 7f85f1c45640 1 --2- 172.21.15.86:0/2536931066 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f85d40a3d20 0x7f85d40a61e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f85dc009f90 tx=0x7f85dc0147e0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:55.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.768+0000 7f85f1c45640 1 -- 172.21.15.86:0/609899382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c9a59a310 msgr2=0x561c9a59a6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:55.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.768+0000 7f85f1c45640 1 --2- 172.21.15.86:0/609899382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c9a59a310 0x561c9a59a6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.768+0000 7f85f1c45640 1 -- 172.21.15.86:0/609899382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c9a59ac30 msgr2=0x561c9a5a32d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:55.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.768+0000 7f85f1c45640 1 --2- 172.21.15.86:0/609899382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c9a59ac30 0x561c9a5a32d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.768+0000 7f85f1c45640 1 -- 172.21.15.86:0/609899382 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c9a4c45e0 con 0x561c9a5a3810 2024-04-21T08:48:55.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.768+0000 7f85f1444640 1 --2- 172.21.15.86:0/609899382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c9a59a310 0x561c9a59a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:55.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.768+0000 7f85f0c43640 1 --2- 172.21.15.86:0/609899382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c9a59ac30 0x561c9a5a32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:55.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.769+0000 7f85f1c45640 1 --2- 172.21.15.86:0/609899382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c9a5a3810 0x561c9a5a5c00 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f85dc012690 tx=0x7f85dc01e000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=357df5dbeb395fc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:55.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.769+0000 7f85d37fe640 1 -- 172.21.15.86:0/609899382 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc004030 con 0x561c9a5a3810 2024-04-21T08:48:55.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.769+0000 7f85d37fe640 1 -- 172.21.15.86:0/609899382 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85dc0054d0 con 0x561c9a5a3810 2024-04-21T08:48:55.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.769+0000 7f85d37fe640 1 -- 172.21.15.86:0/609899382 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc004bc0 con 0x561c9a5a3810 2024-04-21T08:48:55.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.769+0000 7f85f6379740 1 -- 172.21.15.86:0/609899382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c9a5a3810 msgr2=0x561c9a5a5c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:55.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.769+0000 7f85f6379740 1 --2- 172.21.15.86:0/609899382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c9a5a3810 0x561c9a5a5c00 secure :-1 s=READY pgs=1822 cs=0 l=1 rev1=1 crypto rx=0x7f85dc012690 tx=0x7f85dc01e000 comp rx=0 tx=0).stop 2024-04-21T08:48:55.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.769+0000 7f85f6379740 1 -- 172.21.15.86:0/609899382 shutdown_connections 2024-04-21T08:48:55.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.769+0000 7f85f6379740 1 --2- 172.21.15.86:0/609899382 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c9a5a3810 0x561c9a5a5c00 unknown :-1 s=CLOSED pgs=1822 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.769+0000 7f85f6379740 1 --2- 172.21.15.86:0/609899382 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c9a59ac30 0x561c9a5a32d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.769+0000 7f85f6379740 1 --2- 172.21.15.86:0/609899382 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c9a59a310 0x561c9a59a6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.769+0000 7f85f6379740 1 -- 172.21.15.86:0/609899382 >> 172.21.15.86:0/609899382 conn(0x561c9a594a40 msgr2=0x561c9a594e20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:55.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.770+0000 7f85f6379740 1 -- 172.21.15.86:0/609899382 shutdown_connections 2024-04-21T08:48:55.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.770+0000 7f85f6379740 1 -- 172.21.15.86:0/609899382 wait complete. 2024-04-21T08:48:55.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.770+0000 7f85f6379740 1 -- start start 2024-04-21T08:48:55.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.770+0000 7f85f6379740 1 Processor -- start 2024-04-21T08:48:55.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.770+0000 7f85f6379740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c9a59a310 0x561c9a5a31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.770+0000 7f85f6379740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c9a59ac30 0x561c9a5a0540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.770+0000 7f85f6379740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c9a5a3810 0x561c9a5a0ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.770+0000 7f85f6379740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561c9a5a65a0 con 0x561c9a5a3810 2024-04-21T08:48:55.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.770+0000 7f85f6379740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x561c9a4b8a60 con 0x561c9a59ac30 2024-04-21T08:48:55.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.770+0000 7f85f6379740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561c9a5a6720 con 0x561c9a59a310 2024-04-21T08:48:55.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.770+0000 7f85f6379740 10 client.0 _ll_register_callbacks cb 0x561c9a211110 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:55.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.770+0000 7f85f1c45640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c9a5a3810 0x561c9a5a0ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.770+0000 7f85f1c45640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c9a5a3810 0x561c9a5a0ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38044/0 (socket says 172.21.15.86:38044) 2024-04-21T08:48:55.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.770+0000 7f85f1c45640 1 -- 172.21.15.86:0/520150075 learned_addr learned my addr 172.21.15.86:0/520150075 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:55.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.771+0000 7f85f0c43640 1 --2- 172.21.15.86:0/520150075 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c9a59ac30 0x561c9a5a0540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.771+0000 7f85f1444640 1 --2- 172.21.15.86:0/520150075 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c9a59a310 0x561c9a5a31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.771+0000 7f85f1c45640 1 -- 172.21.15.86:0/520150075 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c9a59a310 msgr2=0x561c9a5a31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:55.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.771+0000 7f85f1c45640 1 --2- 172.21.15.86:0/520150075 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c9a59a310 0x561c9a5a31b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.771+0000 7f85f1c45640 1 -- 172.21.15.86:0/520150075 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c9a59ac30 msgr2=0x561c9a5a0540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:55.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.771+0000 7f85f1c45640 1 --2- 172.21.15.86:0/520150075 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c9a59ac30 0x561c9a5a0540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.771+0000 7f85f1c45640 1 -- 172.21.15.86:0/520150075 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c9a211920 con 0x561c9a5a3810 2024-04-21T08:48:55.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.771+0000 7f85f1444640 1 --2- 172.21.15.86:0/520150075 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c9a59a310 0x561c9a5a31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:55.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.771+0000 7f85f1c45640 1 --2- 172.21.15.86:0/520150075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c9a5a3810 0x561c9a5a0ae0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f85dc0045c0 tx=0x7f85dc0046c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48aab1c3da4fbe25 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:55.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.771+0000 7f85d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:55.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.771+0000 7f85d0ff9640 1 -- 172.21.15.86:0/520150075 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85dc02e030 con 0x561c9a5a3810 2024-04-21T08:48:55.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.771+0000 7f85f6379740 1 -- 172.21.15.86:0/520150075 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85dc0054d0 con 0x561c9a5a3810 2024-04-21T08:48:55.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.771+0000 7f85d0ff9640 1 -- 172.21.15.86:0/520150075 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f85dc025040 con 0x561c9a5a3810 2024-04-21T08:48:55.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.771+0000 7f85d0ff9640 1 -- 172.21.15.86:0/520150075 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f85dc020710 con 0x561c9a5a3810 2024-04-21T08:48:55.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.771+0000 7f85f6379740 10 client.19686 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:55.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.771+0000 7f85f6379740 1 -- 172.21.15.86:0/520150075 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561c9a2ddd30 con 0x561c9a5a3810 2024-04-21T08:48:55.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.771+0000 7f85d0ff9640 1 -- 172.21.15.86:0/520150075 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85dc0054d0 con 0x561c9a5a3810 2024-04-21T08:48:55.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.772+0000 7f85d0ff9640 1 -- 172.21.15.86:0/520150075 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f85dc00f040 con 0x561c9a5a3810 2024-04-21T08:48:55.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.772+0000 7f85f6379740 10 client.19686 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:55.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.772+0000 7f85f6379740 10 client.19686 resolve_mds: validated gid 24482 aka daemon mds.b 2024-04-21T08:48:55.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.772+0000 7f85f6379740 20 client.19686 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:55.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.772+0000 7f85f6379740 1 --2- 172.21.15.86:0/520150075 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x561c9a5b37c0 0x561c9a5b5bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.772+0000 7f85f6379740 4 client.19686 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:55.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.772+0000 7f85f6379740 1 -- 172.21.15.86:0/520150075 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561c9a4d2a10 con 0x561c9a5b37c0 2024-04-21T08:48:55.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.772+0000 7f85f1444640 1 --2- 172.21.15.86:0/520150075 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x561c9a5b37c0 0x561c9a5b5bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.772+0000 7f85f1444640 1 --2- 172.21.15.86:0/520150075 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x561c9a5b37c0 0x561c9a5b5bb0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d76883dfbfb52c32 server_cookie=dfb7ee98ca46b932 in_seq=0 out_seq=0 2024-04-21T08:48:55.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.772+0000 7f85d0ff9640 10 client.19686 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:55.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.773+0000 7f85d0ff9640 1 -- 172.21.15.86:0/520150075 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561c9a4d2a10 con 0x561c9a5b37c0 2024-04-21T08:48:55.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.773+0000 7f85d0ff9640 10 client.19686 handle_command_reply: tid=0 2024-04-21T08:48:55.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.773+0000 7f85f6379740 1 client.19686 shutdown 2024-04-21T08:48:55.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.773+0000 7f85f6379740 1 -- 172.21.15.86:0/520150075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c9a5a3810 msgr2=0x561c9a5a0ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:55.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.773+0000 7f85f6379740 1 --2- 172.21.15.86:0/520150075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c9a5a3810 0x561c9a5a0ae0 secure :-1 s=READY pgs=1823 cs=0 l=1 rev1=1 crypto rx=0x7f85dc0045c0 tx=0x7f85dc0046c0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.773+0000 7f85f6379740 1 -- 172.21.15.86:0/520150075 shutdown_connections 2024-04-21T08:48:55.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.773+0000 7f85f6379740 1 --2- 172.21.15.86:0/520150075 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x561c9a5b37c0 0x561c9a5b5bb0 crc :-1 s=READY pgs=295 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.773+0000 7f85f6379740 1 --2- 172.21.15.86:0/520150075 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c9a5a3810 0x561c9a5a0ae0 unknown :-1 s=CLOSED pgs=1823 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.773+0000 7f85d0ff9640 0 client.19686 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:55.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.773+0000 7f85f6379740 1 --2- 172.21.15.86:0/520150075 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c9a59ac30 0x561c9a5a0540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.773+0000 7f85f6379740 1 --2- 172.21.15.86:0/520150075 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c9a59a310 0x561c9a5a31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.773+0000 7f85f6379740 1 -- 172.21.15.86:0/520150075 >> 172.21.15.86:0/520150075 conn(0x561c9a594a40 msgr2=0x561c9a596790 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:55.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.773+0000 7f85f6379740 1 -- 172.21.15.86:0/520150075 shutdown_connections 2024-04-21T08:48:55.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.773+0000 7f85f6379740 1 -- 172.21.15.86:0/520150075 wait complete. 2024-04-21T08:48:55.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.773+0000 7f85f6379740 20 client.19686 trim_cache size 0 max 16384 2024-04-21T08:48:55.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.790+0000 7f85f6379740 1 Processor -- start 2024-04-21T08:48:55.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.790+0000 7f85f6379740 1 -- start start 2024-04-21T08:48:55.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.790+0000 7f85f6379740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c9a59a310 0x561c9a5a0730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.790+0000 7f85f6379740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c9a59ac30 0x561c9a5a2c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.790+0000 7f85f6379740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c9a5a3810 0x561c9a5a31b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.790+0000 7f85f6379740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec1af380 con 0x561c9a59ac30 2024-04-21T08:48:55.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.790+0000 7f85f6379740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec1b0ec0 con 0x561c9a59a310 2024-04-21T08:48:55.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.790+0000 7f85f6379740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec1afa50 con 0x561c9a5a3810 2024-04-21T08:48:55.795 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.791+0000 7f85f0c43640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c9a59ac30 0x561c9a5a2c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.791+0000 7f85f0c43640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c9a59ac30 0x561c9a5a2c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38058/0 (socket says 172.21.15.86:38058) 2024-04-21T08:48:55.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.791+0000 7f85f0c43640 1 -- 172.21.15.86:0/975156168 learned_addr learned my addr 172.21.15.86:0/975156168 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:55.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.791+0000 7f85f1444640 1 --2- 172.21.15.86:0/975156168 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c9a59a310 0x561c9a5a0730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.791+0000 7f85f1c45640 1 --2- 172.21.15.86:0/975156168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c9a5a3810 0x561c9a5a31b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.791+0000 7f85f0c43640 1 -- 172.21.15.86:0/975156168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c9a5a3810 msgr2=0x561c9a5a31b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:55.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.791+0000 7f85f0c43640 1 --2- 172.21.15.86:0/975156168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c9a5a3810 0x561c9a5a31b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.791+0000 7f85f0c43640 1 -- 172.21.15.86:0/975156168 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c9a59a310 msgr2=0x561c9a5a0730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:55.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.791+0000 7f85f0c43640 1 --2- 172.21.15.86:0/975156168 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c9a59a310 0x561c9a5a0730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.791+0000 7f85f0c43640 1 -- 172.21.15.86:0/975156168 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561c9a5b6140 con 0x561c9a59ac30 2024-04-21T08:48:55.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.791+0000 7f85f1444640 1 --2- 172.21.15.86:0/975156168 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c9a59a310 0x561c9a5a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:48:55.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.791+0000 7f85f0c43640 1 --2- 172.21.15.86:0/975156168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c9a59ac30 0x561c9a5a2c70 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f85e009c860 tx=0x7f85e00bd040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9a1b6f7696255a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:55.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.791+0000 7f85d2ffd640 1 -- 172.21.15.86:0/975156168 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e00ce300 con 0x561c9a59ac30 2024-04-21T08:48:55.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.791+0000 7f85d2ffd640 1 -- 172.21.15.86:0/975156168 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85e000f740 con 0x561c9a59ac30 2024-04-21T08:48:55.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.791+0000 7f85d2ffd640 1 -- 172.21.15.86:0/975156168 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e00a50b0 con 0x561c9a59ac30 2024-04-21T08:48:55.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.792+0000 7f85f6379740 1 -- 172.21.15.86:0/975156168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c9a59ac30 msgr2=0x561c9a5a2c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:55.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.792+0000 7f85f6379740 1 --2- 172.21.15.86:0/975156168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c9a59ac30 0x561c9a5a2c70 secure :-1 s=READY pgs=1824 cs=0 l=1 rev1=1 crypto rx=0x7f85e009c860 tx=0x7f85e00bd040 comp rx=0 tx=0).stop 2024-04-21T08:48:55.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.792+0000 7f85f6379740 1 -- 172.21.15.86:0/975156168 shutdown_connections 2024-04-21T08:48:55.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.792+0000 7f85f6379740 1 --2- 172.21.15.86:0/975156168 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c9a5a3810 0x561c9a5a31b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.792+0000 7f85f6379740 1 --2- 172.21.15.86:0/975156168 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c9a59ac30 0x561c9a5a2c70 unknown :-1 s=CLOSED pgs=1824 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.792+0000 7f85f6379740 1 --2- 172.21.15.86:0/975156168 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c9a59a310 0x561c9a5a0730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.792+0000 7f85f6379740 1 -- 172.21.15.86:0/975156168 >> 172.21.15.86:0/975156168 conn(0x561c9a594a40 msgr2=0x561c9a593270 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:55.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.792+0000 7f85f6379740 1 -- 172.21.15.86:0/975156168 shutdown_connections 2024-04-21T08:48:55.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.792+0000 7f85f6379740 1 -- 172.21.15.86:0/975156168 wait complete. 2024-04-21T08:48:55.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.792+0000 7f85f6379740 1 -- start start 2024-04-21T08:48:55.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.792+0000 7f85f6379740 1 Processor -- start 2024-04-21T08:48:55.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.792+0000 7f85f6379740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c9a59a310 0x561c9a5abec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.793+0000 7f85f6379740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c9a59ac30 0x561c9a59dca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.793+0000 7f85f6379740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c9a5a3810 0x561c9a5a4950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.793+0000 7f85f6379740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec1b07f0 con 0x561c9a59ac30 2024-04-21T08:48:55.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.793+0000 7f85f6379740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec1b0120 con 0x561c9a5a3810 2024-04-21T08:48:55.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.793+0000 7f85f6379740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f85ec1ad380 con 0x561c9a59a310 2024-04-21T08:48:55.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.793+0000 7f85f6379740 10 client.0 _ll_register_callbacks cb 0x7f85dc005f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:55.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.793+0000 7f85f0c43640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c9a59ac30 0x561c9a59dca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.793+0000 7f85f0c43640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c9a59ac30 0x561c9a59dca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38074/0 (socket says 172.21.15.86:38074) 2024-04-21T08:48:55.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.793+0000 7f85f0c43640 1 -- 172.21.15.86:0/2149423489 learned_addr learned my addr 172.21.15.86:0/2149423489 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:55.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.793+0000 7f85f0c43640 1 -- 172.21.15.86:0/2149423489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c9a59a310 msgr2=0x561c9a5abec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:55.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.793+0000 7f85f0c43640 1 --2- 172.21.15.86:0/2149423489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c9a59a310 0x561c9a5abec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.793+0000 7f85f0c43640 1 -- 172.21.15.86:0/2149423489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c9a5a3810 msgr2=0x561c9a5a4950 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:55.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.793+0000 7f85f0c43640 1 --2- 172.21.15.86:0/2149423489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c9a5a3810 0x561c9a5a4950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.793+0000 7f85f0c43640 1 -- 172.21.15.86:0/2149423489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561c9a5a4e90 con 0x561c9a59ac30 2024-04-21T08:48:55.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.793+0000 7f85f0c43640 1 --2- 172.21.15.86:0/2149423489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c9a59ac30 0x561c9a59dca0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f85e00992a0 tx=0x7f85e009c7d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70b3705de6605105 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:55.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.793+0000 7f85d0ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:55.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.793+0000 7f85d0ff9640 1 -- 172.21.15.86:0/2149423489 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f85e0098f40 con 0x561c9a59ac30 2024-04-21T08:48:55.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.793+0000 7f85d0ff9640 1 -- 172.21.15.86:0/2149423489 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f85e00870b0 con 0x561c9a59ac30 2024-04-21T08:48:55.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.793+0000 7f85f6379740 1 -- 172.21.15.86:0/2149423489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f85e000f740 con 0x561c9a59ac30 2024-04-21T08:48:55.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.793+0000 7f85d0ff9640 1 -- 172.21.15.86:0/2149423489 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f85e00990e0 con 0x561c9a59ac30 2024-04-21T08:48:55.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.793+0000 7f85f6379740 10 client.19692 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:55.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.793+0000 7f85f6379740 1 -- 172.21.15.86:0/2149423489 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561c9a61a180 con 0x561c9a59ac30 2024-04-21T08:48:55.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.793+0000 7f85d0ff9640 1 -- 172.21.15.86:0/2149423489 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f85e000f740 con 0x561c9a59ac30 2024-04-21T08:48:55.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.794+0000 7f85d0ff9640 1 -- 172.21.15.86:0/2149423489 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f85e0098620 con 0x561c9a59ac30 2024-04-21T08:48:55.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.794+0000 7f85f6379740 10 client.19692 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:55.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.794+0000 7f85f6379740 10 client.19692 resolve_mds: validated gid 24482 aka daemon mds.b 2024-04-21T08:48:55.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.794+0000 7f85f6379740 20 client.19692 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:55.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.794+0000 7f85f6379740 1 --2- 172.21.15.86:0/2149423489 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x561c9a61a840 0x561c9a61cc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:55.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.794+0000 7f85f6379740 4 client.19692 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "set_id": "bea13946", "release": true, "await": true}] 2024-04-21T08:48:55.799 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.794+0000 7f85f6379740 1 -- 172.21.15.86:0/2149423489 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "quiesce db", "set_id": "bea13946", "release": true, "await": true}) -- 0x561c9a50cf20 con 0x561c9a61a840 2024-04-21T08:48:55.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.794+0000 7f85f1444640 1 --2- 172.21.15.86:0/2149423489 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x561c9a61a840 0x561c9a61cc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:55.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.794+0000 7f85d0ff9640 10 client.19692 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:55.800 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.794+0000 7f85f1444640 1 --2- 172.21.15.86:0/2149423489 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x561c9a61a840 0x561c9a61cc50 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=9bac2653176239dd server_cookie=7de513731d2d8345 in_seq=0 out_seq=0 2024-04-21T08:48:55.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.797+0000 7f85d0ff9640 1 -- 172.21.15.86:0/2149423489 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+590 (crc 0 0 0) 0x561c9a50cf20 con 0x561c9a61a840 2024-04-21T08:48:55.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.797+0000 7f85d0ff9640 10 client.19692 handle_command_reply: tid=0 2024-04-21T08:48:55.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.797+0000 7f85f6379740 1 client.19692 shutdown 2024-04-21T08:48:55.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.798+0000 7f85f6379740 1 -- 172.21.15.86:0/2149423489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c9a59ac30 msgr2=0x561c9a59dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:55.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.798+0000 7f85f6379740 1 --2- 172.21.15.86:0/2149423489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c9a59ac30 0x561c9a59dca0 secure :-1 s=READY pgs=1825 cs=0 l=1 rev1=1 crypto rx=0x7f85e00992a0 tx=0x7f85e009c7d0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.798+0000 7f85f6379740 1 -- 172.21.15.86:0/2149423489 shutdown_connections 2024-04-21T08:48:55.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.798+0000 7f85f6379740 1 --2- 172.21.15.86:0/2149423489 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x561c9a61a840 0x561c9a61cc50 crc :-1 s=READY pgs=296 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.798+0000 7f85f6379740 1 --2- 172.21.15.86:0/2149423489 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561c9a5a3810 0x561c9a5a4950 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.798+0000 7f85f6379740 1 --2- 172.21.15.86:0/2149423489 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561c9a59ac30 0x561c9a59dca0 unknown :-1 s=CLOSED pgs=1825 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.798+0000 7f85f6379740 1 --2- 172.21.15.86:0/2149423489 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561c9a59a310 0x561c9a5abec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.798+0000 7f85d0ff9640 0 client.19692 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:55.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.798+0000 7f85f6379740 1 -- 172.21.15.86:0/2149423489 >> 172.21.15.86:0/2149423489 conn(0x561c9a594a40 msgr2=0x561c9a593b00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:55.801 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.798+0000 7f85f6379740 1 -- 172.21.15.86:0/2149423489 shutdown_connections 2024-04-21T08:48:55.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.798+0000 7f85f6379740 1 -- 172.21.15.86:0/2149423489 wait complete. 2024-04-21T08:48:55.802 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.798+0000 7f85f6379740 20 client.19692 trim_cache size 0 max 16384 2024-04-21T08:48:55.802 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:48:55.802 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 29, 2024-04-21T08:48:55.802 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24482, 2024-04-21T08:48:55.802 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 12, 2024-04-21T08:48:55.802 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": { 2024-04-21T08:48:55.802 INFO:teuthology.orchestra.run.smithi086.stdout: "bea13946": { 2024-04-21T08:48:55.802 INFO:teuthology.orchestra.run.smithi086.stdout: "version": 12, 2024-04-21T08:48:55.802 INFO:teuthology.orchestra.run.smithi086.stdout: "age_ref": 0.0, 2024-04-21T08:48:55.802 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-21T08:48:55.802 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-04-21T08:48:55.802 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-21T08:48:55.803 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:48:55.803 INFO:teuthology.orchestra.run.smithi086.stdout: "timeout": 30.0, 2024-04-21T08:48:55.803 INFO:teuthology.orchestra.run.smithi086.stdout: "expiration": 93.1, 2024-04-21T08:48:55.803 INFO:teuthology.orchestra.run.smithi086.stdout: "members": { 2024-04-21T08:48:55.803 INFO:teuthology.orchestra.run.smithi086.stdout: "file:/": { 2024-04-21T08:48:55.803 INFO:teuthology.orchestra.run.smithi086.stdout: "excluded": false, 2024-04-21T08:48:55.803 INFO:teuthology.orchestra.run.smithi086.stdout: "state": { 2024-04-21T08:48:55.803 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RELEASED", 2024-04-21T08:48:55.803 INFO:teuthology.orchestra.run.smithi086.stdout: "age": 0.0 2024-04-21T08:48:55.803 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:48:55.803 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:48:55.803 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:48:55.803 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:48:55.803 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:48:55.803 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:48:55.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.798+0000 7f85f36a8640 1 -- 172.21.15.86:0/2536931066 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f85d40a3d20 msgr2=0x7f85d40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:55.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.798+0000 7f85f36a8640 1 --2- 172.21.15.86:0/2536931066 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f85d40a3d20 0x7f85d40a61e0 secure :-1 s=READY pgs=301 cs=0 l=1 rev1=1 crypto rx=0x7f85dc009f90 tx=0x7f85dc0147e0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.798+0000 7f85f36a8640 1 -- 172.21.15.86:0/2536931066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85ec103ad0 msgr2=0x7f85ec198c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:55.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.798+0000 7f85f36a8640 1 --2- 172.21.15.86:0/2536931066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85ec103ad0 0x7f85ec198c10 secure :-1 s=READY pgs=1821 cs=0 l=1 rev1=1 crypto rx=0x7f85e000c920 tx=0x7f85e000cde0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.798+0000 7f85f36a8640 1 -- 172.21.15.86:0/2536931066 shutdown_connections 2024-04-21T08:48:55.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.798+0000 7f85f36a8640 1 --2- 172.21.15.86:0/2536931066 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f85d40a3d20 0x7f85d40a61e0 unknown :-1 s=CLOSED pgs=301 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.798+0000 7f85f36a8640 1 --2- 172.21.15.86:0/2536931066 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f85ec10c580 0x7f85ec19cf90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.798+0000 7f85f36a8640 1 --2- 172.21.15.86:0/2536931066 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f85ec103ad0 0x7f85ec198c10 unknown :-1 s=CLOSED pgs=1821 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.798+0000 7f85f36a8640 1 --2- 172.21.15.86:0/2536931066 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f85ec1031b0 0x7f85ec1986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:55.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.798+0000 7f85f36a8640 1 -- 172.21.15.86:0/2536931066 >> 172.21.15.86:0/2536931066 conn(0x7f85ec0771d0 msgr2=0x7f85ec0fdd10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:55.804 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.799+0000 7f85f36a8640 1 -- 172.21.15.86:0/2536931066 shutdown_connections 2024-04-21T08:48:55.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:48:55.799+0000 7f85f36a8640 1 -- 172.21.15.86:0/2536931066 wait complete. 2024-04-21T08:48:55.809 INFO:tasks.quiescer.fs.[cephfs]:Successfully released set_id: bea13946, seconds elapsed: 34.2 2024-04-21T08:48:56.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:55 smithi088 ceph-mon[31259]: pgmap v315: 129 pgs: 129 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-04-21T08:48:56.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:55 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/694581997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:48:56.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:55 smithi179 ceph-mon[31829]: pgmap v315: 129 pgs: 129 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-04-21T08:48:56.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:55 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/694581997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:48:56.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:55 smithi086 ceph-mon[27939]: pgmap v315: 129 pgs: 129 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-04-21T08:48:56.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:55 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/694581997' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:48:58.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:57 smithi088 ceph-mon[31259]: pgmap v316: 129 pgs: 129 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-04-21T08:48:58.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:57 smithi179 ceph-mon[31829]: pgmap v316: 129 pgs: 129 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-04-21T08:48:58.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.344+0000 7f5e85985640 1 -- 172.21.15.86:0/3701904022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e80103b20 msgr2=0x7f5e8010c000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.344+0000 7f5e85985640 1 --2- 172.21.15.86:0/3701904022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e80103b20 0x7f5e8010c000 secure :-1 s=READY pgs=1826 cs=0 l=1 rev1=1 crypto rx=0x7f5e68009900 tx=0x7f5e6802bcd0 comp rx=0 tx=0).stop 2024-04-21T08:48:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.344+0000 7f5e85985640 1 -- 172.21.15.86:0/3701904022 shutdown_connections 2024-04-21T08:48:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.344+0000 7f5e85985640 1 --2- 172.21.15.86:0/3701904022 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e8010c5d0 0x7f5e8010ea00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.344+0000 7f5e85985640 1 --2- 172.21.15.86:0/3701904022 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e80103b20 0x7f5e8010c000 unknown :-1 s=CLOSED pgs=1826 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.344+0000 7f5e85985640 1 --2- 172.21.15.86:0/3701904022 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e80103200 0x7f5e801035e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.344+0000 7f5e85985640 1 -- 172.21.15.86:0/3701904022 >> 172.21.15.86:0/3701904022 conn(0x7f5e800771d0 msgr2=0x7f5e800775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.344+0000 7f5e85985640 1 -- 172.21.15.86:0/3701904022 shutdown_connections 2024-04-21T08:48:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.344+0000 7f5e85985640 1 -- 172.21.15.86:0/3701904022 wait complete. 2024-04-21T08:48:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.344+0000 7f5e85985640 1 Processor -- start 2024-04-21T08:48:58.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.344+0000 7f5e85985640 1 -- start start 2024-04-21T08:48:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.344+0000 7f5e85985640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e80103200 0x7f5e801986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.344+0000 7f5e85985640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e80103b20 0x7f5e80198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.344+0000 7f5e85985640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e8010c5d0 0x7f5e8019cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.344+0000 7f5e85985640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5e80112210 con 0x7f5e80103200 2024-04-21T08:48:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.344+0000 7f5e85985640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5e80112090 con 0x7f5e8010c5d0 2024-04-21T08:48:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.344+0000 7f5e85985640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5e80112390 con 0x7f5e80103b20 2024-04-21T08:48:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.345+0000 7f5e7effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e80103200 0x7f5e801986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.345+0000 7f5e7effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e80103200 0x7f5e801986f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38088/0 (socket says 172.21.15.86:38088) 2024-04-21T08:48:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.345+0000 7f5e7effd640 1 -- 172.21.15.86:0/453410056 learned_addr learned my addr 172.21.15.86:0/453410056 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.345+0000 7f5e77fff640 1 --2- 172.21.15.86:0/453410056 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e80103b20 0x7f5e80198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.345+0000 7f5e7f7fe640 1 --2- 172.21.15.86:0/453410056 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e8010c5d0 0x7f5e8019cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.345+0000 7f5e7effd640 1 -- 172.21.15.86:0/453410056 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e80103b20 msgr2=0x7f5e80198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.345+0000 7f5e7effd640 1 --2- 172.21.15.86:0/453410056 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e80103b20 0x7f5e80198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:58.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.345+0000 7f5e7effd640 1 -- 172.21.15.86:0/453410056 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e8010c5d0 msgr2=0x7f5e8019cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.345+0000 7f5e7effd640 1 --2- 172.21.15.86:0/453410056 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e8010c5d0 0x7f5e8019cf60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.345+0000 7f5e7effd640 1 -- 172.21.15.86:0/453410056 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e8019d690 con 0x7f5e80103200 2024-04-21T08:48:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.345+0000 7f5e7effd640 1 --2- 172.21.15.86:0/453410056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e80103200 0x7f5e801986f0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f5e7000d950 tx=0x7f5e7000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.345+0000 7f5e84983640 1 -- 172.21.15.86:0/453410056 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e70017070 con 0x7f5e80103200 2024-04-21T08:48:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.345+0000 7f5e84983640 1 -- 172.21.15.86:0/453410056 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e70009c30 con 0x7f5e80103200 2024-04-21T08:48:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.345+0000 7f5e84983640 1 -- 172.21.15.86:0/453410056 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7000f620 con 0x7f5e80103200 2024-04-21T08:48:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.345+0000 7f5e85985640 1 -- 172.21.15.86:0/453410056 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e8019d980 con 0x7f5e80103200 2024-04-21T08:48:58.348 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.345+0000 7f5e85985640 1 -- 172.21.15.86:0/453410056 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e801a5210 con 0x7f5e80103200 2024-04-21T08:48:58.355 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.347+0000 7f5e84983640 1 -- 172.21.15.86:0/453410056 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5e70010460 con 0x7f5e80103200 2024-04-21T08:48:58.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.348+0000 7f5e88572740 1 Processor -- start 2024-04-21T08:48:58.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.348+0000 7f5e88572740 1 -- start start 2024-04-21T08:48:58.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.348+0000 7f5e84983640 1 --2- 172.21.15.86:0/453410056 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5e6c0a3e40 0x7f5e6c0a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.348+0000 7f5e88572740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c12226b8d0 0x55c12226a620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.348+0000 7f5e88572740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c12226ab60 0x55c12226afe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.348+0000 7f5e88572740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1222740e0 0x55c1222764d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.348+0000 7f5e88572740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c1221cf8d0 con 0x55c1222740e0 2024-04-21T08:48:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.348+0000 7f5e88572740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c12216ce80 con 0x55c12226b8d0 2024-04-21T08:48:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.348+0000 7f5e88572740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c1221cb890 con 0x55c12226ab60 2024-04-21T08:48:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.348+0000 7f5e7f7fe640 1 --2- 172.21.15.86:0/453410056 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5e6c0a3e40 0x7f5e6c0a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.348+0000 7f5e84983640 1 -- 172.21.15.86:0/453410056 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f5e700d0860 con 0x7f5e80103200 2024-04-21T08:48:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.348+0000 7f5e7effd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c12226b8d0 0x55c12226a620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.348+0000 7f5e7f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1222740e0 0x55c1222764d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.348+0000 7f5e7effd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c12226b8d0 0x55c12226a620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.88:3300/0 says I am v2:172.21.15.86:57200/0 (socket says 172.21.15.86:57200) 2024-04-21T08:48:58.358 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.348+0000 7f5e7effd640 1 -- 172.21.15.86:0/2010215180 learned_addr learned my addr 172.21.15.86:0/2010215180 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.349+0000 7f5e77fff640 1 --2- 172.21.15.86:0/2010215180 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c12226ab60 0x55c12226afe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.349+0000 7f5e7effd640 1 -- 172.21.15.86:0/2010215180 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c12226ab60 msgr2=0x55c12226afe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.349+0000 7f5e7effd640 1 --2- 172.21.15.86:0/2010215180 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c12226ab60 0x55c12226afe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.349+0000 7f5e7effd640 1 -- 172.21.15.86:0/2010215180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1222740e0 msgr2=0x55c1222764d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.349+0000 7f5e7effd640 1 --2- 172.21.15.86:0/2010215180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1222740e0 0x55c1222764d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.349+0000 7f5e7effd640 1 -- 172.21.15.86:0/2010215180 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c12218e320 con 0x55c12226b8d0 2024-04-21T08:48:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.349+0000 7f5e7f7fe640 1 --2- 172.21.15.86:0/2010215180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1222740e0 0x55c1222764d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:48:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.349+0000 7f5e7f7fe640 1 --2- 172.21.15.86:0/453410056 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5e6c0a3e40 0x7f5e6c0a6300 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5e78005c40 tx=0x7f5e78009c10 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.349+0000 7f5e7effd640 1 --2- 172.21.15.86:0/2010215180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c12226b8d0 0x55c12226a620 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5e70098e00 tx=0x7f5e7009cb30 comp rx=0 tx=0).ready entity=mon.2 client_cookie=c1d11918064be03c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.349+0000 7f5e75ffb640 1 -- 172.21.15.86:0/2010215180 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e70020040 con 0x55c12226b8d0 2024-04-21T08:48:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.349+0000 7f5e75ffb640 1 -- 172.21.15.86:0/2010215180 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e7006c4d0 con 0x55c12226b8d0 2024-04-21T08:48:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.349+0000 7f5e75ffb640 1 -- 172.21.15.86:0/2010215180 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e700cde60 con 0x55c12226b8d0 2024-04-21T08:48:58.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.350+0000 7f5e88572740 1 -- 172.21.15.86:0/2010215180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c12226b8d0 msgr2=0x55c12226a620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:58.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.350+0000 7f5e88572740 1 --2- 172.21.15.86:0/2010215180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c12226b8d0 0x55c12226a620 secure :-1 s=READY pgs=82 cs=0 l=1 rev1=1 crypto rx=0x7f5e70098e00 tx=0x7f5e7009cb30 comp rx=0 tx=0).stop 2024-04-21T08:48:58.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.350+0000 7f5e88572740 1 -- 172.21.15.86:0/2010215180 shutdown_connections 2024-04-21T08:48:58.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.350+0000 7f5e88572740 1 --2- 172.21.15.86:0/2010215180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1222740e0 0x55c1222764d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:58.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.350+0000 7f5e88572740 1 --2- 172.21.15.86:0/2010215180 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c12226ab60 0x55c12226afe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:58.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.350+0000 7f5e88572740 1 --2- 172.21.15.86:0/2010215180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c12226b8d0 0x55c12226a620 unknown :-1 s=CLOSED pgs=82 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:58.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.350+0000 7f5e88572740 1 -- 172.21.15.86:0/2010215180 >> 172.21.15.86:0/2010215180 conn(0x55c122231c90 msgr2=0x55c122265850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:58.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.350+0000 7f5e88572740 1 -- 172.21.15.86:0/2010215180 shutdown_connections 2024-04-21T08:48:58.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.350+0000 7f5e88572740 1 -- 172.21.15.86:0/2010215180 wait complete. 2024-04-21T08:48:58.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.350+0000 7f5e88572740 1 -- start start 2024-04-21T08:48:58.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.351+0000 7f5e88572740 1 Processor -- start 2024-04-21T08:48:58.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.351+0000 7f5e88572740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c12226ab60 0x55c122273a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:58.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.351+0000 7f5e88572740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c12226b8d0 0x55c122270e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:58.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.351+0000 7f5e88572740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1222740e0 0x55c1222713b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.351+0000 7f5e88572740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c122276e10 con 0x55c1222740e0 2024-04-21T08:48:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.351+0000 7f5e88572740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c122157eb0 con 0x55c12226ab60 2024-04-21T08:48:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.351+0000 7f5e88572740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c122276f90 con 0x55c12226b8d0 2024-04-21T08:48:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.351+0000 7f5e88572740 10 client.0 _ll_register_callbacks cb 0x55c122184b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.351+0000 7f5e7f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1222740e0 0x55c1222713b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.351+0000 7f5e7effd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c12226ab60 0x55c122273a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.351+0000 7f5e7f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1222740e0 0x55c1222713b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38100/0 (socket says 172.21.15.86:38100) 2024-04-21T08:48:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.351+0000 7f5e7f7fe640 1 -- 172.21.15.86:0/2596723260 learned_addr learned my addr 172.21.15.86:0/2596723260 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.351+0000 7f5e77fff640 1 --2- 172.21.15.86:0/2596723260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c12226b8d0 0x55c122270e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.351+0000 7f5e7f7fe640 1 -- 172.21.15.86:0/2596723260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c12226b8d0 msgr2=0x55c122270e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.351+0000 7f5e7f7fe640 1 --2- 172.21.15.86:0/2596723260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c12226b8d0 0x55c122270e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:58.361 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.351+0000 7f5e7f7fe640 1 -- 172.21.15.86:0/2596723260 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c12226ab60 msgr2=0x55c122273a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:58.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.351+0000 7f5e7f7fe640 1 --2- 172.21.15.86:0/2596723260 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c12226ab60 0x55c122273a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:58.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.351+0000 7f5e7f7fe640 1 -- 172.21.15.86:0/2596723260 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c121ee2210 con 0x55c1222740e0 2024-04-21T08:48:58.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.352+0000 7f5e7f7fe640 1 --2- 172.21.15.86:0/2596723260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1222740e0 0x55c1222713b0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f5e78011bb0 tx=0x7f5e7801b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f59f4431fff31a89 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:58.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.352+0000 7f5e5effd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:58.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.352+0000 7f5e5effd640 1 -- 172.21.15.86:0/2596723260 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e78026070 con 0x55c1222740e0 2024-04-21T08:48:58.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.352+0000 7f5e5effd640 1 -- 172.21.15.86:0/2596723260 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f5e78004d20 con 0x55c1222740e0 2024-04-21T08:48:58.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.352+0000 7f5e88572740 1 -- 172.21.15.86:0/2596723260 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e7006c4d0 con 0x55c1222740e0 2024-04-21T08:48:58.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.352+0000 7f5e5effd640 1 -- 172.21.15.86:0/2596723260 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5e78031050 con 0x55c1222740e0 2024-04-21T08:48:58.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.352+0000 7f5e88572740 10 client.19704 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:58.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.352+0000 7f5e88572740 1 -- 172.21.15.86:0/2596723260 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c121fae620 con 0x55c1222740e0 2024-04-21T08:48:58.362 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.352+0000 7f5e5effd640 1 -- 172.21.15.86:0/2596723260 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e7006c4d0 con 0x55c1222740e0 2024-04-21T08:48:58.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.352+0000 7f5e5effd640 1 -- 172.21.15.86:0/2596723260 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5e7800f050 con 0x55c1222740e0 2024-04-21T08:48:58.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.352+0000 7f5e88572740 10 client.19704 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:58.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.352+0000 7f5e88572740 10 client.19704 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:58.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.352+0000 7f5e88572740 20 client.19704 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:58.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.352+0000 7f5e88572740 1 --2- 172.21.15.86:0/2596723260 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c122284140 0x55c122286550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:58.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.352+0000 7f5e88572740 4 client.19704 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:48:58.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.352+0000 7f5e88572740 1 -- 172.21.15.86:0/2596723260 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c1221e3880 con 0x55c122284140 2024-04-21T08:48:58.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.353+0000 7f5e7effd640 1 --2- 172.21.15.86:0/2596723260 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c122284140 0x55c122286550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:58.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.353+0000 7f5e7effd640 1 --2- 172.21.15.86:0/2596723260 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c122284140 0x55c122286550 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b04cfe16dc49f9ae server_cookie=a8bb3d1d16683ba6 in_seq=0 out_seq=0 2024-04-21T08:48:58.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.353+0000 7f5e5effd640 10 client.19704 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:58.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.354+0000 7f5e5effd640 1 -- 172.21.15.86:0/2596723260 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c1221e3880 con 0x55c122284140 2024-04-21T08:48:58.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.354+0000 7f5e5effd640 10 client.19704 handle_command_reply: tid=0 2024-04-21T08:48:58.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.354+0000 7f5e88572740 1 client.19704 shutdown 2024-04-21T08:48:58.363 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.354+0000 7f5e88572740 1 -- 172.21.15.86:0/2596723260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1222740e0 msgr2=0x55c1222713b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.354+0000 7f5e88572740 1 --2- 172.21.15.86:0/2596723260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1222740e0 0x55c1222713b0 secure :-1 s=READY pgs=1828 cs=0 l=1 rev1=1 crypto rx=0x7f5e78011bb0 tx=0x7f5e7801b000 comp rx=0 tx=0).stop 2024-04-21T08:48:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.354+0000 7f5e88572740 1 -- 172.21.15.86:0/2596723260 shutdown_connections 2024-04-21T08:48:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.354+0000 7f5e88572740 1 --2- 172.21.15.86:0/2596723260 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c122284140 0x55c122286550 crc :-1 s=READY pgs=297 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.354+0000 7f5e88572740 1 --2- 172.21.15.86:0/2596723260 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1222740e0 0x55c1222713b0 unknown :-1 s=CLOSED pgs=1828 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.354+0000 7f5e88572740 1 --2- 172.21.15.86:0/2596723260 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c12226b8d0 0x55c122270e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.354+0000 7f5e5effd640 0 client.19704 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.354+0000 7f5e88572740 1 --2- 172.21.15.86:0/2596723260 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c12226ab60 0x55c122273a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.354+0000 7f5e88572740 1 -- 172.21.15.86:0/2596723260 >> 172.21.15.86:0/2596723260 conn(0x55c122231c90 msgr2=0x55c1222675a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.354+0000 7f5e88572740 1 -- 172.21.15.86:0/2596723260 shutdown_connections 2024-04-21T08:48:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.354+0000 7f5e88572740 1 -- 172.21.15.86:0/2596723260 wait complete. 2024-04-21T08:48:58.364 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.354+0000 7f5e88572740 20 client.19704 trim_cache size 0 max 16384 2024-04-21T08:48:58.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:57 smithi086 ceph-mon[27939]: pgmap v316: 129 pgs: 129 active+clean; 2.2 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 2.0 KiB/s wr, 10 op/s 2024-04-21T08:48:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.369+0000 7f5e88572740 1 Processor -- start 2024-04-21T08:48:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.369+0000 7f5e88572740 1 -- start start 2024-04-21T08:48:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.369+0000 7f5e88572740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c12226ab60 0x55c12227fd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:58.370 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.369+0000 7f5e88572740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c12226b8d0 0x55c122273540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.369+0000 7f5e88572740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c1222740e0 0x55c122273a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.369+0000 7f5e88572740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5e801af360 con 0x55c12226b8d0 2024-04-21T08:48:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.369+0000 7f5e88572740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5e801b0ea0 con 0x55c1222740e0 2024-04-21T08:48:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.369+0000 7f5e88572740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5e801afa30 con 0x55c12226ab60 2024-04-21T08:48:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.369+0000 7f5e77fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c12226b8d0 0x55c122273540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.369+0000 7f5e77fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c12226b8d0 0x55c122273540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38102/0 (socket says 172.21.15.86:38102) 2024-04-21T08:48:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.369+0000 7f5e77fff640 1 -- 172.21.15.86:0/486296930 learned_addr learned my addr 172.21.15.86:0/486296930 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.370+0000 7f5e7effd640 1 --2- 172.21.15.86:0/486296930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c12226ab60 0x55c12227fd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.370+0000 7f5e7f7fe640 1 --2- 172.21.15.86:0/486296930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c1222740e0 0x55c122273a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.370+0000 7f5e77fff640 1 -- 172.21.15.86:0/486296930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c12226ab60 msgr2=0x55c12227fd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.370+0000 7f5e77fff640 1 --2- 172.21.15.86:0/486296930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c12226ab60 0x55c12227fd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.370+0000 7f5e77fff640 1 -- 172.21.15.86:0/486296930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c1222740e0 msgr2=0x55c122273a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.370+0000 7f5e77fff640 1 --2- 172.21.15.86:0/486296930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c1222740e0 0x55c122273a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.370+0000 7f5e77fff640 1 -- 172.21.15.86:0/486296930 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c122269e30 con 0x55c12226b8d0 2024-04-21T08:48:58.371 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.370+0000 7f5e77fff640 1 --2- 172.21.15.86:0/486296930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c12226b8d0 0x55c122273540 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f5e6802c210 tx=0x7f5e68005d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9182ab75edb0a98 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.370+0000 7f5e757fa640 1 -- 172.21.15.86:0/486296930 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e68004240 con 0x55c12226b8d0 2024-04-21T08:48:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.370+0000 7f5e757fa640 1 -- 172.21.15.86:0/486296930 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e680043e0 con 0x55c12226b8d0 2024-04-21T08:48:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.370+0000 7f5e757fa640 1 -- 172.21.15.86:0/486296930 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e680046c0 con 0x55c12226b8d0 2024-04-21T08:48:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.371+0000 7f5e88572740 1 -- 172.21.15.86:0/486296930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c12226b8d0 msgr2=0x55c122273540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.371+0000 7f5e88572740 1 --2- 172.21.15.86:0/486296930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c12226b8d0 0x55c122273540 secure :-1 s=READY pgs=1829 cs=0 l=1 rev1=1 crypto rx=0x7f5e6802c210 tx=0x7f5e68005d50 comp rx=0 tx=0).stop 2024-04-21T08:48:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.371+0000 7f5e88572740 1 -- 172.21.15.86:0/486296930 shutdown_connections 2024-04-21T08:48:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.371+0000 7f5e88572740 1 --2- 172.21.15.86:0/486296930 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c1222740e0 0x55c122273a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.371+0000 7f5e88572740 1 --2- 172.21.15.86:0/486296930 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c12226b8d0 0x55c122273540 unknown :-1 s=CLOSED pgs=1829 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.371+0000 7f5e88572740 1 --2- 172.21.15.86:0/486296930 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c12226ab60 0x55c12227fd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.371+0000 7f5e88572740 1 -- 172.21.15.86:0/486296930 >> 172.21.15.86:0/486296930 conn(0x55c122231c90 msgr2=0x55c122265380 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.371+0000 7f5e88572740 1 -- 172.21.15.86:0/486296930 shutdown_connections 2024-04-21T08:48:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.371+0000 7f5e88572740 1 -- 172.21.15.86:0/486296930 wait complete. 2024-04-21T08:48:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.371+0000 7f5e88572740 1 -- start start 2024-04-21T08:48:58.372 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.371+0000 7f5e88572740 1 Processor -- start 2024-04-21T08:48:58.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.371+0000 7f5e88572740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c12226ab60 0x55c12227c410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:58.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.371+0000 7f5e88572740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c12226b8d0 0x55c12226d6e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:58.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.371+0000 7f5e88572740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1222740e0 0x55c12226dc20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:58.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.371+0000 7f5e88572740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5e801b07d0 con 0x55c1222740e0 2024-04-21T08:48:58.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.372+0000 7f5e88572740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5e801b0100 con 0x55c12226b8d0 2024-04-21T08:48:58.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.372+0000 7f5e88572740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5e801ad360 con 0x55c12226ab60 2024-04-21T08:48:58.375 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.372+0000 7f5e88572740 10 client.0 _ll_register_callbacks cb 0x7f5e78014f90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:48:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.372+0000 7f5e7f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1222740e0 0x55c12226dc20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.372+0000 7f5e7f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1222740e0 0x55c12226dc20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:38110/0 (socket says 172.21.15.86:38110) 2024-04-21T08:48:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.372+0000 7f5e7f7fe640 1 -- 172.21.15.86:0/1072405487 learned_addr learned my addr 172.21.15.86:0/1072405487 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:48:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.372+0000 7f5e77fff640 1 --2- 172.21.15.86:0/1072405487 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c12226b8d0 0x55c12226d6e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.372+0000 7f5e7f7fe640 1 -- 172.21.15.86:0/1072405487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c12226ab60 msgr2=0x55c12227c410 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.372+0000 7f5e7f7fe640 1 --2- 172.21.15.86:0/1072405487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c12226ab60 0x55c12227c410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.372+0000 7f5e7f7fe640 1 -- 172.21.15.86:0/1072405487 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c12226b8d0 msgr2=0x55c12226d6e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:48:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.372+0000 7f5e7f7fe640 1 --2- 172.21.15.86:0/1072405487 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c12226b8d0 0x55c12226d6e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.372+0000 7f5e7f7fe640 1 -- 172.21.15.86:0/1072405487 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c12226e160 con 0x55c1222740e0 2024-04-21T08:48:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.372+0000 7f5e77fff640 1 --2- 172.21.15.86:0/1072405487 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c12226b8d0 0x55c12226d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:48:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.372+0000 7f5e7f7fe640 1 --2- 172.21.15.86:0/1072405487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1222740e0 0x55c12226dc20 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f5e78014050 tx=0x7f5e78012c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a5998c8d7151c075 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:48:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.372+0000 7f5e5effd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:48:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.372+0000 7f5e5effd640 1 -- 172.21.15.86:0/1072405487 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e7803a060 con 0x55c1222740e0 2024-04-21T08:48:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.372+0000 7f5e5effd640 1 -- 172.21.15.86:0/1072405487 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(102..102 src has 1..102) ==== 9496+0+0 (secure 0 0 0) 0x7f5e78003b80 con 0x55c1222740e0 2024-04-21T08:48:58.376 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.372+0000 7f5e88572740 1 -- 172.21.15.86:0/1072405487 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5e680043e0 con 0x55c1222740e0 2024-04-21T08:48:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.372+0000 7f5e5effd640 1 -- 172.21.15.86:0/1072405487 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5e78041050 con 0x55c1222740e0 2024-04-21T08:48:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.372+0000 7f5e88572740 10 client.19710 fetch_fsmap learned FSMap version 29 2024-04-21T08:48:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.372+0000 7f5e88572740 1 -- 172.21.15.86:0/1072405487 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c1222ea400 con 0x55c1222740e0 2024-04-21T08:48:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.372+0000 7f5e5effd640 1 -- 172.21.15.86:0/1072405487 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e680043e0 con 0x55c1222740e0 2024-04-21T08:48:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.373+0000 7f5e5effd640 1 -- 172.21.15.86:0/1072405487 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5e7800f050 con 0x55c1222740e0 2024-04-21T08:48:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.373+0000 7f5e88572740 10 client.19710 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:48:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.373+0000 7f5e88572740 10 client.19710 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:48:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.373+0000 7f5e88572740 20 client.19710 populate_metadata read hostname 'smithi086' 2024-04-21T08:48:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.373+0000 7f5e88572740 1 --2- 172.21.15.86:0/1072405487 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c1222eaac0 0x55c1222ecf00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:48:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.373+0000 7f5e88572740 4 client.19710 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:48:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.373+0000 7f5e88572740 1 -- 172.21.15.86:0/1072405487 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c12216f3d0 con 0x55c1222eaac0 2024-04-21T08:48:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.373+0000 7f5e7effd640 1 --2- 172.21.15.86:0/1072405487 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c1222eaac0 0x55c1222ecf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:48:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.373+0000 7f5e7effd640 1 --2- 172.21.15.86:0/1072405487 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c1222eaac0 0x55c1222ecf00 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=b681439866f75bfa server_cookie=25de8a5cd321fe61 in_seq=0 out_seq=0 2024-04-21T08:48:58.377 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.373+0000 7f5e5effd640 10 client.19710 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:58.382 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.382+0000 7f5e5effd640 1 -- 172.21.15.86:0/1072405487 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c12216f3d0 con 0x55c1222eaac0 2024-04-21T08:48:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.382+0000 7f5e5effd640 10 client.19710 handle_command_reply: tid=0 2024-04-21T08:48:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.382+0000 7f5e88572740 1 client.19710 shutdown 2024-04-21T08:48:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.383+0000 7f5e88572740 1 -- 172.21.15.86:0/1072405487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1222740e0 msgr2=0x55c12226dc20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.383+0000 7f5e88572740 1 --2- 172.21.15.86:0/1072405487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1222740e0 0x55c12226dc20 secure :-1 s=READY pgs=1830 cs=0 l=1 rev1=1 crypto rx=0x7f5e78014050 tx=0x7f5e78012c90 comp rx=0 tx=0).stop 2024-04-21T08:48:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.383+0000 7f5e88572740 1 -- 172.21.15.86:0/1072405487 shutdown_connections 2024-04-21T08:48:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.383+0000 7f5e88572740 1 --2- 172.21.15.86:0/1072405487 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c1222eaac0 0x55c1222ecf00 crc :-1 s=READY pgs=298 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.383+0000 7f5e88572740 1 --2- 172.21.15.86:0/1072405487 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c1222740e0 0x55c12226dc20 unknown :-1 s=CLOSED pgs=1830 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.383+0000 7f5e88572740 1 --2- 172.21.15.86:0/1072405487 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c12226b8d0 0x55c12226d6e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.383+0000 7f5e5effd640 0 client.19710 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:48:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.383+0000 7f5e88572740 1 --2- 172.21.15.86:0/1072405487 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c12226ab60 0x55c12227c410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.383+0000 7f5e88572740 1 -- 172.21.15.86:0/1072405487 >> 172.21.15.86:0/1072405487 conn(0x55c122231c90 msgr2=0x55c12226f5d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.383+0000 7f5e88572740 1 -- 172.21.15.86:0/1072405487 shutdown_connections 2024-04-21T08:48:58.383 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.383+0000 7f5e88572740 1 -- 172.21.15.86:0/1072405487 wait complete. 2024-04-21T08:48:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.383+0000 7f5e88572740 20 client.19710 trim_cache size 0 max 16384 2024-04-21T08:48:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:48:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:48:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:48:58.384 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:48:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.383+0000 7f5e85985640 1 -- 172.21.15.86:0/453410056 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5e6c0a3e40 msgr2=0x7f5e6c0a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.383+0000 7f5e85985640 1 --2- 172.21.15.86:0/453410056 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5e6c0a3e40 0x7f5e6c0a6300 secure :-1 s=READY pgs=302 cs=0 l=1 rev1=1 crypto rx=0x7f5e78005c40 tx=0x7f5e78009c10 comp rx=0 tx=0).stop 2024-04-21T08:48:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.383+0000 7f5e85985640 1 -- 172.21.15.86:0/453410056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e80103200 msgr2=0x7f5e801986f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:48:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.383+0000 7f5e85985640 1 --2- 172.21.15.86:0/453410056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e80103200 0x7f5e801986f0 secure :-1 s=READY pgs=1827 cs=0 l=1 rev1=1 crypto rx=0x7f5e7000d950 tx=0x7f5e7000de10 comp rx=0 tx=0).stop 2024-04-21T08:48:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.383+0000 7f5e85985640 1 -- 172.21.15.86:0/453410056 shutdown_connections 2024-04-21T08:48:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.383+0000 7f5e85985640 1 --2- 172.21.15.86:0/453410056 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5e6c0a3e40 0x7f5e6c0a6300 unknown :-1 s=CLOSED pgs=302 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.383+0000 7f5e85985640 1 --2- 172.21.15.86:0/453410056 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e8010c5d0 0x7f5e8019cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.383+0000 7f5e85985640 1 --2- 172.21.15.86:0/453410056 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e80103b20 0x7f5e80198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.383+0000 7f5e85985640 1 --2- 172.21.15.86:0/453410056 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e80103200 0x7f5e801986f0 unknown :-1 s=CLOSED pgs=1827 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:48:58.385 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.383+0000 7f5e85985640 1 -- 172.21.15.86:0/453410056 >> 172.21.15.86:0/453410056 conn(0x7f5e800771d0 msgr2=0x7f5e800fdd60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:48:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.384+0000 7f5e85985640 1 -- 172.21.15.86:0/453410056 shutdown_connections 2024-04-21T08:48:58.386 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:48:58.384+0000 7f5e85985640 1 -- 172.21.15.86:0/453410056 wait complete. 2024-04-21T08:49:00.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:59 smithi088 ceph-mon[31259]: pgmap v317: 129 pgs: 129 active+clean; 2.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 KiB/s wr, 14 op/s 2024-04-21T08:49:00.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:48:59 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:49:00.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:59 smithi179 ceph-mon[31829]: pgmap v317: 129 pgs: 129 active+clean; 2.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 KiB/s wr, 14 op/s 2024-04-21T08:49:00.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:48:59 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:49:00.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:59 smithi086 ceph-mon[27939]: pgmap v317: 129 pgs: 129 active+clean; 2.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 13 KiB/s rd, 8.7 KiB/s wr, 14 op/s 2024-04-21T08:49:00.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:48:59 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:49:02.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:02 smithi088 ceph-mon[31259]: pgmap v318: 129 pgs: 129 active+clean; 2.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.7 KiB/s wr, 9 op/s 2024-04-21T08:49:02.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:02 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:49:02.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:02 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:49:02.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:02 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:49:02.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:02 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:49:02.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:02 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:49:02.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:02 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:49:02.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:02 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:02.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:02 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:02.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:02 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:02.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:02 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:02.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:02 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:02.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:02 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:02.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:02 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:02.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:02 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:02.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:02 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:02.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:02 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:02.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:02 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:02.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:02 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:02.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:02 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:49:02.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:02 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:49:02.466 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:02 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:49:02.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:02 smithi179 ceph-mon[31829]: pgmap v318: 129 pgs: 129 active+clean; 2.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.7 KiB/s wr, 9 op/s 2024-04-21T08:49:02.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:02 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:49:02.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:02 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:49:02.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:02 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:49:02.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:02 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:49:02.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:02 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:49:02.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:02 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:49:02.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:02 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:02.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:02 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:02.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:02 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:02.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:02 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:02.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:02 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:02.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:02 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:02.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:02 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:02.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:02 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:02.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:02 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:02.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:02 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:02.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:02 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:02.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:02 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:02.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:02 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:49:02.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:02 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:49:02.498 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:02 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:49:02.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:02 smithi086 ceph-mon[27939]: pgmap v318: 129 pgs: 129 active+clean; 2.3 MiB data, 423 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 6.7 KiB/s wr, 9 op/s 2024-04-21T08:49:02.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:02 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:49:02.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:02 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:49:02.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:02 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:49:02.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:02 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:49:02.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:02 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:49:02.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:02 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:49:02.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:02 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.1", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:02.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:02 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.4", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:02.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:02 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.7", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:02.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:02 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.10", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:02.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:02 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.2", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:02.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:02 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.5", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:02.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:02 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.8", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:02.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:02 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.11", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:02.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:02 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.0", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:02.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:02 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.3", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:02.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:02 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.6", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:02.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:02 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config rm", "who": "osd.9", "name": "osd_memory_target"}]: dispatch 2024-04-21T08:49:02.618 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:02 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:49:02.619 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:02 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:49:02.619 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:02 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:49:03.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:03 smithi088 ceph-mon[31259]: Adjusting osd_memory_target on smithi088 to 176.2M 2024-04-21T08:49:03.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:03 smithi088 ceph-mon[31259]: Unable to set osd_memory_target on smithi088 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-21T08:49:03.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:03 smithi088 ceph-mon[31259]: Adjusting osd_memory_target on smithi179 to 176.2M 2024-04-21T08:49:03.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:03 smithi088 ceph-mon[31259]: Unable to set osd_memory_target on smithi179 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-21T08:49:03.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:03 smithi088 ceph-mon[31259]: Adjusting osd_memory_target on smithi086 to 176.2M 2024-04-21T08:49:03.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:03 smithi088 ceph-mon[31259]: Unable to set osd_memory_target on smithi086 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-21T08:49:03.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:03 smithi179 ceph-mon[31829]: Adjusting osd_memory_target on smithi088 to 176.2M 2024-04-21T08:49:03.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:03 smithi179 ceph-mon[31829]: Unable to set osd_memory_target on smithi088 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-21T08:49:03.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:03 smithi179 ceph-mon[31829]: Adjusting osd_memory_target on smithi179 to 176.2M 2024-04-21T08:49:03.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:03 smithi179 ceph-mon[31829]: Unable to set osd_memory_target on smithi179 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-21T08:49:03.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:03 smithi179 ceph-mon[31829]: Adjusting osd_memory_target on smithi086 to 176.2M 2024-04-21T08:49:03.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:03 smithi179 ceph-mon[31829]: Unable to set osd_memory_target on smithi086 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-21T08:49:03.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:03 smithi086 ceph-mon[27939]: Adjusting osd_memory_target on smithi088 to 176.2M 2024-04-21T08:49:03.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:03 smithi086 ceph-mon[27939]: Unable to set osd_memory_target on smithi088 to 184786022: error parsing value: Value '184786022' is below minimum 939524096 2024-04-21T08:49:03.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:03 smithi086 ceph-mon[27939]: Adjusting osd_memory_target on smithi179 to 176.2M 2024-04-21T08:49:03.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:03 smithi086 ceph-mon[27939]: Unable to set osd_memory_target on smithi179 to 184770969: error parsing value: Value '184770969' is below minimum 939524096 2024-04-21T08:49:03.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:03 smithi086 ceph-mon[27939]: Adjusting osd_memory_target on smithi086 to 176.2M 2024-04-21T08:49:03.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:03 smithi086 ceph-mon[27939]: Unable to set osd_memory_target on smithi086 to 184784588: error parsing value: Value '184784588' is below minimum 939524096 2024-04-21T08:49:04.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:04 smithi179 ceph-mon[31829]: pgmap v319: 129 pgs: 129 active+clean; 2.5 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 72 KiB/s wr, 34 op/s 2024-04-21T08:49:04.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:04 smithi086 ceph-mon[27939]: pgmap v319: 129 pgs: 129 active+clean; 2.5 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 72 KiB/s wr, 34 op/s 2024-04-21T08:49:04.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:04 smithi088 ceph-mon[31259]: pgmap v319: 129 pgs: 129 active+clean; 2.5 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 72 KiB/s wr, 34 op/s 2024-04-21T08:49:06.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:06 smithi179 ceph-mon[31829]: pgmap v320: 129 pgs: 129 active+clean; 2.5 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 72 KiB/s wr, 30 op/s 2024-04-21T08:49:06.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:06 smithi179 ceph-mon[31829]: osdmap e103: 12 total, 12 up, 12 in 2024-04-21T08:49:06.558 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:06 smithi088 ceph-mon[31259]: pgmap v320: 129 pgs: 129 active+clean; 2.5 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 72 KiB/s wr, 30 op/s 2024-04-21T08:49:06.558 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:06 smithi088 ceph-mon[31259]: osdmap e103: 12 total, 12 up, 12 in 2024-04-21T08:49:06.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:06 smithi086 ceph-mon[27939]: pgmap v320: 129 pgs: 129 active+clean; 2.5 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 72 KiB/s wr, 30 op/s 2024-04-21T08:49:06.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:06 smithi086 ceph-mon[27939]: osdmap e103: 12 total, 12 up, 12 in 2024-04-21T08:49:08.032 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:49:08.032 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:49:08.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.147+0000 7f3872ec8640 1 -- 172.21.15.86:0/2113794418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f386c105dd0 msgr2=0x7f386c1061b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.147+0000 7f3872ec8640 1 --2- 172.21.15.86:0/2113794418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f386c105dd0 0x7f386c1061b0 secure :-1 s=READY pgs=1831 cs=0 l=1 rev1=1 crypto rx=0x7f385c009900 tx=0x7f385c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:49:08.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.147+0000 7f3872ec8640 1 -- 172.21.15.86:0/2113794418 shutdown_connections 2024-04-21T08:49:08.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.147+0000 7f3872ec8640 1 --2- 172.21.15.86:0/2113794418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f386c101a40 0x7f386c10dca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.147+0000 7f3872ec8640 1 --2- 172.21.15.86:0/2113794418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f386c1010a0 0x7f386c101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.147+0000 7f3872ec8640 1 --2- 172.21.15.86:0/2113794418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f386c105dd0 0x7f386c1061b0 unknown :-1 s=CLOSED pgs=1831 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.149 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.147+0000 7f3872ec8640 1 -- 172.21.15.86:0/2113794418 >> 172.21.15.86:0/2113794418 conn(0x7f386c077c40 msgr2=0x7f386c078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:08.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.147+0000 7f3872ec8640 1 -- 172.21.15.86:0/2113794418 shutdown_connections 2024-04-21T08:49:08.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.147+0000 7f3872ec8640 1 -- 172.21.15.86:0/2113794418 wait complete. 2024-04-21T08:49:08.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.148+0000 7f3872ec8640 1 Processor -- start 2024-04-21T08:49:08.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.148+0000 7f3872ec8640 1 -- start start 2024-04-21T08:49:08.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.148+0000 7f3872ec8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f386c1010a0 0x7f386c1995b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.148+0000 7f3872ec8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f386c101a40 0x7f386c199af0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.148+0000 7f3872ec8640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f386c105dd0 0x7f386c1a4630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.148+0000 7f3872ec8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f386c111150 con 0x7f386c101a40 2024-04-21T08:49:08.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.148+0000 7f3872ec8640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f386c110fd0 con 0x7f386c1010a0 2024-04-21T08:49:08.150 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.148+0000 7f3872ec8640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f386c1112d0 con 0x7f386c105dd0 2024-04-21T08:49:08.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.148+0000 7f386bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f386c101a40 0x7f386c199af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.148+0000 7f386bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f386c101a40 0x7f386c199af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60890/0 (socket says 172.21.15.86:60890) 2024-04-21T08:49:08.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.148+0000 7f386bfff640 1 -- 172.21.15.86:0/3770427453 learned_addr learned my addr 172.21.15.86:0/3770427453 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:08.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.148+0000 7f3870c64640 1 --2- 172.21.15.86:0/3770427453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f386c1010a0 0x7f386c1995b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.148+0000 7f3871465640 1 --2- 172.21.15.86:0/3770427453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f386c105dd0 0x7f386c1a4630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.149+0000 7f386bfff640 1 -- 172.21.15.86:0/3770427453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f386c105dd0 msgr2=0x7f386c1a4630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.149+0000 7f386bfff640 1 --2- 172.21.15.86:0/3770427453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f386c105dd0 0x7f386c1a4630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.149+0000 7f386bfff640 1 -- 172.21.15.86:0/3770427453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f386c1010a0 msgr2=0x7f386c1995b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.149+0000 7f386bfff640 1 --2- 172.21.15.86:0/3770427453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f386c1010a0 0x7f386c1995b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.149+0000 7f386bfff640 1 -- 172.21.15.86:0/3770427453 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f386c1a4b70 con 0x7f386c101a40 2024-04-21T08:49:08.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.149+0000 7f386bfff640 1 --2- 172.21.15.86:0/3770427453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f386c101a40 0x7f386c199af0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f386000d950 tx=0x7f386000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:08.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.149+0000 7f3869ffb640 1 -- 172.21.15.86:0/3770427453 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3860017070 con 0x7f386c101a40 2024-04-21T08:49:08.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.149+0000 7f3869ffb640 1 -- 172.21.15.86:0/3770427453 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3860009c30 con 0x7f386c101a40 2024-04-21T08:49:08.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.149+0000 7f3872ec8640 1 -- 172.21.15.86:0/3770427453 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f386c1a4d40 con 0x7f386c101a40 2024-04-21T08:49:08.151 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.149+0000 7f3869ffb640 1 -- 172.21.15.86:0/3770427453 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f386000f620 con 0x7f386c101a40 2024-04-21T08:49:08.152 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.149+0000 7f3872ec8640 1 -- 172.21.15.86:0/3770427453 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f386c1a5200 con 0x7f386c101a40 2024-04-21T08:49:08.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.152+0000 7f3869ffb640 1 -- 172.21.15.86:0/3770427453 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f3860010460 con 0x7f386c101a40 2024-04-21T08:49:08.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.152+0000 7f3875b9a740 1 Processor -- start 2024-04-21T08:49:08.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.152+0000 7f3875b9a740 1 -- start start 2024-04-21T08:49:08.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.152+0000 7f3869ffb640 1 --2- 172.21.15.86:0/3770427453 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f38540a3d20 0x7f38540a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.159 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.152+0000 7f3875b9a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d90835580 0x563d90835960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.152+0000 7f3875b9a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d90835ea0 0x563d9083e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.153+0000 7f3875b9a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d9083ea80 0x563d90840e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.153+0000 7f3875b9a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563d9079a1e0 con 0x563d9083ea80 2024-04-21T08:49:08.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.153+0000 7f3875b9a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x563d90737390 con 0x563d90835580 2024-04-21T08:49:08.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.153+0000 7f3875b9a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563d907961a0 con 0x563d90835ea0 2024-04-21T08:49:08.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.153+0000 7f3869ffb640 1 -- 172.21.15.86:0/3770427453 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9584+0+0 (secure 0 0 0) 0x7f38600d07d0 con 0x7f386c101a40 2024-04-21T08:49:08.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.153+0000 7f3871465640 1 --2- 172.21.15.86:0/3770427453 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f38540a3d20 0x7f38540a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.153+0000 7f3871465640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d9083ea80 0x563d90840e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.153+0000 7f3871465640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d9083ea80 0x563d90840e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60898/0 (socket says 172.21.15.86:60898) 2024-04-21T08:49:08.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.153+0000 7f3871465640 1 -- 172.21.15.86:0/3623970004 learned_addr learned my addr 172.21.15.86:0/3623970004 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:08.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.153+0000 7f3870c64640 1 --2- 172.21.15.86:0/3623970004 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d90835580 0x563d90835960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.153+0000 7f386bfff640 1 --2- 172.21.15.86:0/3623970004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d90835ea0 0x563d9083e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.153+0000 7f3871465640 1 -- 172.21.15.86:0/3623970004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d90835ea0 msgr2=0x563d9083e540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.153+0000 7f3871465640 1 --2- 172.21.15.86:0/3623970004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d90835ea0 0x563d9083e540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.160 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.153+0000 7f3871465640 1 -- 172.21.15.86:0/3623970004 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d90835580 msgr2=0x563d90835960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.153+0000 7f3871465640 1 --2- 172.21.15.86:0/3623970004 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d90835580 0x563d90835960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.153+0000 7f3871465640 1 -- 172.21.15.86:0/3623970004 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d905767d0 con 0x563d9083ea80 2024-04-21T08:49:08.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.153+0000 7f386bfff640 1 --2- 172.21.15.86:0/3623970004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d90835ea0 0x563d9083e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:08.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.153+0000 7f3871465640 1 --2- 172.21.15.86:0/3770427453 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f38540a3d20 0x7f38540a61e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3864012c00 tx=0x7f3864009e00 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:08.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.153+0000 7f3871465640 1 --2- 172.21.15.86:0/3623970004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d9083ea80 0x563d90840e70 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f3864010800 tx=0x7f386401c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dd93967492aad098 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:08.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.153+0000 7f3852ffd640 1 -- 172.21.15.86:0/3623970004 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3864024070 con 0x563d9083ea80 2024-04-21T08:49:08.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.153+0000 7f3852ffd640 1 -- 172.21.15.86:0/3623970004 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f386400e820 con 0x563d9083ea80 2024-04-21T08:49:08.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.153+0000 7f3852ffd640 1 -- 172.21.15.86:0/3623970004 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3864005220 con 0x563d9083ea80 2024-04-21T08:49:08.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.154+0000 7f3875b9a740 1 -- 172.21.15.86:0/3623970004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d9083ea80 msgr2=0x563d90840e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.154+0000 7f3875b9a740 1 --2- 172.21.15.86:0/3623970004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d9083ea80 0x563d90840e70 secure :-1 s=READY pgs=1833 cs=0 l=1 rev1=1 crypto rx=0x7f3864010800 tx=0x7f386401c000 comp rx=0 tx=0).stop 2024-04-21T08:49:08.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.154+0000 7f3875b9a740 1 -- 172.21.15.86:0/3623970004 shutdown_connections 2024-04-21T08:49:08.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.154+0000 7f3875b9a740 1 --2- 172.21.15.86:0/3623970004 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d9083ea80 0x563d90840e70 unknown :-1 s=CLOSED pgs=1833 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.154+0000 7f3875b9a740 1 --2- 172.21.15.86:0/3623970004 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d90835ea0 0x563d9083e540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.161 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.154+0000 7f3875b9a740 1 --2- 172.21.15.86:0/3623970004 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d90835580 0x563d90835960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.154+0000 7f3875b9a740 1 -- 172.21.15.86:0/3623970004 >> 172.21.15.86:0/3623970004 conn(0x563d9082fcb0 msgr2=0x563d90830090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:08.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.154+0000 7f3875b9a740 1 -- 172.21.15.86:0/3623970004 shutdown_connections 2024-04-21T08:49:08.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.154+0000 7f3875b9a740 1 -- 172.21.15.86:0/3623970004 wait complete. 2024-04-21T08:49:08.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.154+0000 7f3875b9a740 1 -- start start 2024-04-21T08:49:08.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.154+0000 7f3875b9a740 1 Processor -- start 2024-04-21T08:49:08.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.155+0000 7f3875b9a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d90835580 0x563d9083bad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.155+0000 7f3875b9a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d90835ea0 0x563d9083c010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.155+0000 7f3875b9a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d9083ea80 0x563d9083c550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.155+0000 7f3875b9a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563d90841810 con 0x563d90835ea0 2024-04-21T08:49:08.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.155+0000 7f3875b9a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x563d907532f0 con 0x563d90835580 2024-04-21T08:49:08.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.155+0000 7f3875b9a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563d90841990 con 0x563d9083ea80 2024-04-21T08:49:08.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.155+0000 7f3875b9a740 10 client.0 _ll_register_callbacks cb 0x563d90760580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:08.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.155+0000 7f386bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d90835ea0 0x563d9083c010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.155+0000 7f386bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d90835ea0 0x563d9083c010 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60910/0 (socket says 172.21.15.86:60910) 2024-04-21T08:49:08.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.155+0000 7f386bfff640 1 -- 172.21.15.86:0/2930410863 learned_addr learned my addr 172.21.15.86:0/2930410863 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:08.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.155+0000 7f3870c64640 1 --2- 172.21.15.86:0/2930410863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d90835580 0x563d9083bad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.155+0000 7f3871465640 1 --2- 172.21.15.86:0/2930410863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d9083ea80 0x563d9083c550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.155+0000 7f386bfff640 1 -- 172.21.15.86:0/2930410863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d9083ea80 msgr2=0x563d9083c550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.155+0000 7f386bfff640 1 --2- 172.21.15.86:0/2930410863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d9083ea80 0x563d9083c550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.163 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.155+0000 7f386bfff640 1 -- 172.21.15.86:0/2930410863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d90835580 msgr2=0x563d9083bad0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.155+0000 7f386bfff640 1 --2- 172.21.15.86:0/2930410863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d90835580 0x563d9083bad0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.155+0000 7f386bfff640 1 -- 172.21.15.86:0/2930410863 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d904ac3f0 con 0x563d90835ea0 2024-04-21T08:49:08.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.155+0000 7f3871465640 1 --2- 172.21.15.86:0/2930410863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d9083ea80 0x563d9083c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:08.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.155+0000 7f3870c64640 1 --2- 172.21.15.86:0/2930410863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d90835580 0x563d9083bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:08.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.155+0000 7f386bfff640 1 --2- 172.21.15.86:0/2930410863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d90835ea0 0x563d9083c010 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f3860000c00 tx=0x7f386001aed0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d9e986ac7d6ea360 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:08.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.155+0000 7f384bfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:08.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.156+0000 7f384bfff640 1 -- 172.21.15.86:0/2930410863 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3860020070 con 0x563d90835ea0 2024-04-21T08:49:08.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.156+0000 7f384bfff640 1 -- 172.21.15.86:0/2930410863 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9584+0+0 (secure 0 0 0) 0x7f38600cdd80 con 0x563d90835ea0 2024-04-21T08:49:08.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.156+0000 7f3875b9a740 1 -- 172.21.15.86:0/2930410863 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f386400e820 con 0x563d90835ea0 2024-04-21T08:49:08.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.156+0000 7f384bfff640 1 -- 172.21.15.86:0/2930410863 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f38600cdf80 con 0x563d90835ea0 2024-04-21T08:49:08.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.156+0000 7f3875b9a740 10 client.19722 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:08.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.156+0000 7f3875b9a740 1 -- 172.21.15.86:0/2930410863 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563d9046c0f0 con 0x563d90835ea0 2024-04-21T08:49:08.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.156+0000 7f384bfff640 1 -- 172.21.15.86:0/2930410863 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f386400e820 con 0x563d90835ea0 2024-04-21T08:49:08.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.156+0000 7f384bfff640 1 -- 172.21.15.86:0/2930410863 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f38600a4b30 con 0x563d90835ea0 2024-04-21T08:49:08.164 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.156+0000 7f3875b9a740 10 client.19722 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:08.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.156+0000 7f3875b9a740 10 client.19722 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:08.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.156+0000 7f3875b9a740 20 client.19722 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:08.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.156+0000 7f3875b9a740 1 --2- 172.21.15.86:0/2930410863 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563d9084e9d0 0x563d90850de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.156+0000 7f3875b9a740 4 client.19722 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:08.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.156+0000 7f3875b9a740 1 -- 172.21.15.86:0/2930410863 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563d907a7c00 con 0x563d9084e9d0 2024-04-21T08:49:08.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.156+0000 7f3870c64640 1 --2- 172.21.15.86:0/2930410863 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563d9084e9d0 0x563d90850de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.157+0000 7f384bfff640 10 client.19722 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:08.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.157+0000 7f3870c64640 1 --2- 172.21.15.86:0/2930410863 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563d9084e9d0 0x563d90850de0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cd3e2d1a25113de server_cookie=740dda51a2ba8f58 in_seq=0 out_seq=0 2024-04-21T08:49:08.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.157+0000 7f384bfff640 1 -- 172.21.15.86:0/2930410863 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563d907a7c00 con 0x563d9084e9d0 2024-04-21T08:49:08.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.157+0000 7f384bfff640 10 client.19722 handle_command_reply: tid=0 2024-04-21T08:49:08.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.157+0000 7f3875b9a740 1 client.19722 shutdown 2024-04-21T08:49:08.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.158+0000 7f3875b9a740 1 -- 172.21.15.86:0/2930410863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d90835ea0 msgr2=0x563d9083c010 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.158+0000 7f3875b9a740 1 --2- 172.21.15.86:0/2930410863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d90835ea0 0x563d9083c010 secure :-1 s=READY pgs=1834 cs=0 l=1 rev1=1 crypto rx=0x7f3860000c00 tx=0x7f386001aed0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.158+0000 7f3875b9a740 1 -- 172.21.15.86:0/2930410863 shutdown_connections 2024-04-21T08:49:08.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.158+0000 7f3875b9a740 1 --2- 172.21.15.86:0/2930410863 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563d9084e9d0 0x563d90850de0 crc :-1 s=READY pgs=299 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.158+0000 7f3875b9a740 1 --2- 172.21.15.86:0/2930410863 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d9083ea80 0x563d9083c550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.158+0000 7f3875b9a740 1 --2- 172.21.15.86:0/2930410863 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d90835ea0 0x563d9083c010 unknown :-1 s=CLOSED pgs=1834 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.158+0000 7f384bfff640 0 client.19722 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:08.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.158+0000 7f3875b9a740 1 --2- 172.21.15.86:0/2930410863 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d90835580 0x563d9083bad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.158+0000 7f3875b9a740 1 -- 172.21.15.86:0/2930410863 >> 172.21.15.86:0/2930410863 conn(0x563d9082fcb0 msgr2=0x563d90831a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:08.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.158+0000 7f3875b9a740 1 -- 172.21.15.86:0/2930410863 shutdown_connections 2024-04-21T08:49:08.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.158+0000 7f3875b9a740 1 -- 172.21.15.86:0/2930410863 wait complete. 2024-04-21T08:49:08.166 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.158+0000 7f3875b9a740 20 client.19722 trim_cache size 0 max 16384 2024-04-21T08:49:08.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.175+0000 7f3875b9a740 1 Processor -- start 2024-04-21T08:49:08.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.175+0000 7f3875b9a740 1 -- start start 2024-04-21T08:49:08.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.175+0000 7f3875b9a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d90835580 0x563d9083e200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.175+0000 7f3875b9a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d90835ea0 0x563d9084a720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.175+0000 7f3875b9a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d9083ea80 0x563d908317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.175+0000 7f3875b9a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f386c1af450 con 0x563d90835ea0 2024-04-21T08:49:08.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.175+0000 7f3875b9a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f386c1b0f90 con 0x563d90835580 2024-04-21T08:49:08.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.175+0000 7f3875b9a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f386c1afb20 con 0x563d9083ea80 2024-04-21T08:49:08.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.176+0000 7f386bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d90835ea0 0x563d9084a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.176+0000 7f386bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d90835ea0 0x563d9084a720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60926/0 (socket says 172.21.15.86:60926) 2024-04-21T08:49:08.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.176+0000 7f386bfff640 1 -- 172.21.15.86:0/2367989601 learned_addr learned my addr 172.21.15.86:0/2367989601 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:08.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.176+0000 7f3870c64640 1 --2- 172.21.15.86:0/2367989601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d90835580 0x563d9083e200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.176+0000 7f386bfff640 1 -- 172.21.15.86:0/2367989601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d9083ea80 msgr2=0x563d908317f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.176+0000 7f3871465640 1 --2- 172.21.15.86:0/2367989601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d9083ea80 0x563d908317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.181 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.176+0000 7f386bfff640 1 --2- 172.21.15.86:0/2367989601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d9083ea80 0x563d908317f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.176+0000 7f386bfff640 1 -- 172.21.15.86:0/2367989601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d90835580 msgr2=0x563d9083e200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.176+0000 7f386bfff640 1 --2- 172.21.15.86:0/2367989601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d90835580 0x563d9083e200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.176+0000 7f386bfff640 1 -- 172.21.15.86:0/2367989601 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563d90841750 con 0x563d90835ea0 2024-04-21T08:49:08.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.176+0000 7f3870c64640 1 --2- 172.21.15.86:0/2367989601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d90835580 0x563d9083e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:08.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.176+0000 7f386bfff640 1 --2- 172.21.15.86:0/2367989601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d90835ea0 0x563d9084a720 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f386009c390 tx=0x7f386001b000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8453d4e52660076 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:08.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.176+0000 7f38527fc640 1 -- 172.21.15.86:0/2367989601 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38600bb2e0 con 0x563d90835ea0 2024-04-21T08:49:08.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.176+0000 7f38527fc640 1 -- 172.21.15.86:0/2367989601 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38600a0700 con 0x563d90835ea0 2024-04-21T08:49:08.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.176+0000 7f38527fc640 1 -- 172.21.15.86:0/2367989601 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38600cdd80 con 0x563d90835ea0 2024-04-21T08:49:08.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.177+0000 7f3875b9a740 1 -- 172.21.15.86:0/2367989601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d90835ea0 msgr2=0x563d9084a720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.177+0000 7f3875b9a740 1 --2- 172.21.15.86:0/2367989601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d90835ea0 0x563d9084a720 secure :-1 s=READY pgs=1835 cs=0 l=1 rev1=1 crypto rx=0x7f386009c390 tx=0x7f386001b000 comp rx=0 tx=0).stop 2024-04-21T08:49:08.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.177+0000 7f3875b9a740 1 -- 172.21.15.86:0/2367989601 shutdown_connections 2024-04-21T08:49:08.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.177+0000 7f3875b9a740 1 --2- 172.21.15.86:0/2367989601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d9083ea80 0x563d908317f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.177+0000 7f3875b9a740 1 --2- 172.21.15.86:0/2367989601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d90835ea0 0x563d9084a720 unknown :-1 s=CLOSED pgs=1835 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.177+0000 7f3875b9a740 1 --2- 172.21.15.86:0/2367989601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d90835580 0x563d9083e200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.177+0000 7f3875b9a740 1 -- 172.21.15.86:0/2367989601 >> 172.21.15.86:0/2367989601 conn(0x563d9082fcb0 msgr2=0x563d908310e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:08.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.177+0000 7f3875b9a740 1 -- 172.21.15.86:0/2367989601 shutdown_connections 2024-04-21T08:49:08.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.177+0000 7f3875b9a740 1 -- 172.21.15.86:0/2367989601 wait complete. 2024-04-21T08:49:08.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.177+0000 7f3875b9a740 1 -- start start 2024-04-21T08:49:08.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.177+0000 7f3875b9a740 1 Processor -- start 2024-04-21T08:49:08.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.177+0000 7f3875b9a740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d90835580 0x563d90838f00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.177+0000 7f3875b9a740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d90835ea0 0x563d90839440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.177+0000 7f3875b9a740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d9083ea80 0x563d908b3f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.177+0000 7f3875b9a740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f386c1b08c0 con 0x563d90835ea0 2024-04-21T08:49:08.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.177+0000 7f3875b9a740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f386c1b01f0 con 0x563d9083ea80 2024-04-21T08:49:08.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.177+0000 7f3875b9a740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f386c1ad430 con 0x563d90835580 2024-04-21T08:49:08.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.178+0000 7f3875b9a740 10 client.0 _ll_register_callbacks cb 0x7f386009bf40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:08.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.178+0000 7f386bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d90835ea0 0x563d90839440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.178+0000 7f386bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d90835ea0 0x563d90839440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60934/0 (socket says 172.21.15.86:60934) 2024-04-21T08:49:08.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.178+0000 7f386bfff640 1 -- 172.21.15.86:0/3333556096 learned_addr learned my addr 172.21.15.86:0/3333556096 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:08.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.178+0000 7f3871465640 1 --2- 172.21.15.86:0/3333556096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d9083ea80 0x563d908b3f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.178+0000 7f386bfff640 1 -- 172.21.15.86:0/3333556096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d90835580 msgr2=0x563d90838f00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.178+0000 7f386bfff640 1 --2- 172.21.15.86:0/3333556096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d90835580 0x563d90838f00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.178+0000 7f386bfff640 1 -- 172.21.15.86:0/3333556096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d9083ea80 msgr2=0x563d908b3f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.178+0000 7f386bfff640 1 --2- 172.21.15.86:0/3333556096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d9083ea80 0x563d908b3f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.178+0000 7f386bfff640 1 -- 172.21.15.86:0/3333556096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563d908b4460 con 0x563d90835ea0 2024-04-21T08:49:08.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.178+0000 7f386bfff640 1 --2- 172.21.15.86:0/3333556096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d90835ea0 0x563d90839440 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f3860098d70 tx=0x7f386009bff0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c1c75cb30839b00 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:08.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.178+0000 7f384bfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:08.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.178+0000 7f384bfff640 1 -- 172.21.15.86:0/3333556096 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f38600bb2e0 con 0x563d90835ea0 2024-04-21T08:49:08.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.178+0000 7f384bfff640 1 -- 172.21.15.86:0/3333556096 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9584+0+0 (secure 0 0 0) 0x7f386001a360 con 0x563d90835ea0 2024-04-21T08:49:08.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.178+0000 7f3875b9a740 1 -- 172.21.15.86:0/3333556096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f38600a0700 con 0x563d90835ea0 2024-04-21T08:49:08.184 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.178+0000 7f384bfff640 1 -- 172.21.15.86:0/3333556096 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f38600cdf80 con 0x563d90835ea0 2024-04-21T08:49:08.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.178+0000 7f3875b9a740 10 client.19728 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:08.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.178+0000 7f3875b9a740 1 -- 172.21.15.86:0/3333556096 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563d908b5770 con 0x563d90835ea0 2024-04-21T08:49:08.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.178+0000 7f384bfff640 1 -- 172.21.15.86:0/3333556096 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f38600a0700 con 0x563d90835ea0 2024-04-21T08:49:08.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.179+0000 7f384bfff640 1 -- 172.21.15.86:0/3333556096 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f38600981e0 con 0x563d90835ea0 2024-04-21T08:49:08.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.179+0000 7f3875b9a740 10 client.19728 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:08.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.179+0000 7f3875b9a740 10 client.19728 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:08.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.179+0000 7f3875b9a740 20 client.19728 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:08.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.179+0000 7f3875b9a740 1 --2- 172.21.15.86:0/3333556096 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563d908b5bb0 0x563d908b7fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.179+0000 7f3875b9a740 4 client.19728 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:49:08.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.179+0000 7f3875b9a740 1 -- 172.21.15.86:0/3333556096 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563d907a1a70 con 0x563d908b5bb0 2024-04-21T08:49:08.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.179+0000 7f3870c64640 1 --2- 172.21.15.86:0/3333556096 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563d908b5bb0 0x563d908b7fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.179+0000 7f384bfff640 10 client.19728 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:08.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.179+0000 7f3870c64640 1 --2- 172.21.15.86:0/3333556096 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563d908b5bb0 0x563d908b7fd0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=69cfae5cdc43b2aa server_cookie=2e035a530431477d in_seq=0 out_seq=0 2024-04-21T08:49:08.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.179+0000 7f384bfff640 1 -- 172.21.15.86:0/3333556096 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563d907a1a70 con 0x563d908b5bb0 2024-04-21T08:49:08.185 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.179+0000 7f384bfff640 10 client.19728 handle_command_reply: tid=0 2024-04-21T08:49:08.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.179+0000 7f3875b9a740 1 client.19728 shutdown 2024-04-21T08:49:08.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.180+0000 7f3875b9a740 1 -- 172.21.15.86:0/3333556096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d90835ea0 msgr2=0x563d90839440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.180+0000 7f3875b9a740 1 --2- 172.21.15.86:0/3333556096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d90835ea0 0x563d90839440 secure :-1 s=READY pgs=1836 cs=0 l=1 rev1=1 crypto rx=0x7f3860098d70 tx=0x7f386009bff0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.180+0000 7f3875b9a740 1 -- 172.21.15.86:0/3333556096 shutdown_connections 2024-04-21T08:49:08.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.180+0000 7f3875b9a740 1 --2- 172.21.15.86:0/3333556096 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563d908b5bb0 0x563d908b7fd0 crc :-1 s=READY pgs=300 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.180+0000 7f3875b9a740 1 --2- 172.21.15.86:0/3333556096 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563d9083ea80 0x563d908b3f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.180+0000 7f3875b9a740 1 --2- 172.21.15.86:0/3333556096 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563d90835ea0 0x563d90839440 unknown :-1 s=CLOSED pgs=1836 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.180+0000 7f3875b9a740 1 --2- 172.21.15.86:0/3333556096 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563d90835580 0x563d90838f00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.180+0000 7f384bfff640 0 client.19728 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:08.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.180+0000 7f3875b9a740 1 -- 172.21.15.86:0/3333556096 >> 172.21.15.86:0/3333556096 conn(0x563d9082fcb0 msgr2=0x563d9083ba90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:08.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.180+0000 7f3875b9a740 1 -- 172.21.15.86:0/3333556096 shutdown_connections 2024-04-21T08:49:08.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.180+0000 7f3875b9a740 1 -- 172.21.15.86:0/3333556096 wait complete. 2024-04-21T08:49:08.186 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.180+0000 7f3875b9a740 20 client.19728 trim_cache size 0 max 16384 2024-04-21T08:49:08.187 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:49:08.187 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-21T08:49:08.187 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-21T08:49:08.188 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:49:08.189 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.180+0000 7f3872ec8640 1 -- 172.21.15.86:0/3770427453 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f38540a3d20 msgr2=0x7f38540a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.180+0000 7f3872ec8640 1 --2- 172.21.15.86:0/3770427453 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f38540a3d20 0x7f38540a61e0 secure :-1 s=READY pgs=303 cs=0 l=1 rev1=1 crypto rx=0x7f3864012c00 tx=0x7f3864009e00 comp rx=0 tx=0).stop 2024-04-21T08:49:08.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.180+0000 7f3872ec8640 1 -- 172.21.15.86:0/3770427453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f386c101a40 msgr2=0x7f386c199af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.180+0000 7f3872ec8640 1 --2- 172.21.15.86:0/3770427453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f386c101a40 0x7f386c199af0 secure :-1 s=READY pgs=1832 cs=0 l=1 rev1=1 crypto rx=0x7f386000d950 tx=0x7f386000de10 comp rx=0 tx=0).stop 2024-04-21T08:49:08.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.180+0000 7f3872ec8640 1 -- 172.21.15.86:0/3770427453 shutdown_connections 2024-04-21T08:49:08.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.180+0000 7f3872ec8640 1 --2- 172.21.15.86:0/3770427453 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f38540a3d20 0x7f38540a61e0 unknown :-1 s=CLOSED pgs=303 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.180+0000 7f3872ec8640 1 --2- 172.21.15.86:0/3770427453 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f386c105dd0 0x7f386c1a4630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.180+0000 7f3872ec8640 1 --2- 172.21.15.86:0/3770427453 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f386c101a40 0x7f386c199af0 unknown :-1 s=CLOSED pgs=1832 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.180+0000 7f3872ec8640 1 --2- 172.21.15.86:0/3770427453 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f386c1010a0 0x7f386c1995b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.180+0000 7f3872ec8640 1 -- 172.21.15.86:0/3770427453 >> 172.21.15.86:0/3770427453 conn(0x7f386c077c40 msgr2=0x7f386c0fed90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:08.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.180+0000 7f3872ec8640 1 -- 172.21.15.86:0/3770427453 shutdown_connections 2024-04-21T08:49:08.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.180+0000 7f3872ec8640 1 -- 172.21.15.86:0/3770427453 wait complete. 2024-04-21T08:49:08.192 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:49:08.192 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:49:08.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.293+0000 7fb0f61e6640 1 -- 172.21.15.86:0/2294109120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0f0107f70 msgr2=0x7fb0f0108350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.293+0000 7fb0f61e6640 1 --2- 172.21.15.86:0/2294109120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0f0107f70 0x7fb0f0108350 secure :-1 s=READY pgs=1837 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0009930 tx=0x7fb0e002bc30 comp rx=0 tx=0).stop 2024-04-21T08:49:08.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.293+0000 7fb0f61e6640 1 -- 172.21.15.86:0/2294109120 shutdown_connections 2024-04-21T08:49:08.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.293+0000 7fb0f61e6640 1 --2- 172.21.15.86:0/2294109120 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0f0069a80 0x7fb0f010ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.293+0000 7fb0f61e6640 1 --2- 172.21.15.86:0/2294109120 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0f00690e0 0x7fb0f0069540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.293+0000 7fb0f61e6640 1 --2- 172.21.15.86:0/2294109120 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0f0107f70 0x7fb0f0108350 unknown :-1 s=CLOSED pgs=1837 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.293+0000 7fb0f61e6640 1 -- 172.21.15.86:0/2294109120 >> 172.21.15.86:0/2294109120 conn(0x7fb0f0077e30 msgr2=0x7fb0f0078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:08.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.293+0000 7fb0f61e6640 1 -- 172.21.15.86:0/2294109120 shutdown_connections 2024-04-21T08:49:08.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.293+0000 7fb0f61e6640 1 -- 172.21.15.86:0/2294109120 wait complete. 2024-04-21T08:49:08.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.293+0000 7fb0f61e6640 1 Processor -- start 2024-04-21T08:49:08.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.293+0000 7fb0f61e6640 1 -- start start 2024-04-21T08:49:08.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.293+0000 7fb0f61e6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0f00690e0 0x7fb0f0198580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.293+0000 7fb0f61e6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0f0069a80 0x7fb0f0198ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.293+0000 7fb0f61e6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0f0107f70 0x7fb0f019ce40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.293+0000 7fb0f61e6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f0112180 con 0x7fb0f00690e0 2024-04-21T08:49:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.293+0000 7fb0f61e6640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f0112000 con 0x7fb0f0069a80 2024-04-21T08:49:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.293+0000 7fb0f61e6640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb0f0112300 con 0x7fb0f0107f70 2024-04-21T08:49:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.294+0000 7fb0ef7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0f00690e0 0x7fb0f0198580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.294+0000 7fb0ef7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0f00690e0 0x7fb0f0198580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60958/0 (socket says 172.21.15.86:60958) 2024-04-21T08:49:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.294+0000 7fb0ef7fe640 1 -- 172.21.15.86:0/3127821974 learned_addr learned my addr 172.21.15.86:0/3127821974 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.294+0000 7fb0effff640 1 --2- 172.21.15.86:0/3127821974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0f0107f70 0x7fb0f019ce40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.294+0000 7fb0e7fff640 1 --2- 172.21.15.86:0/3127821974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0f0069a80 0x7fb0f0198ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.294+0000 7fb0ef7fe640 1 -- 172.21.15.86:0/3127821974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0f0107f70 msgr2=0x7fb0f019ce40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.294+0000 7fb0ef7fe640 1 --2- 172.21.15.86:0/3127821974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0f0107f70 0x7fb0f019ce40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.294+0000 7fb0ef7fe640 1 -- 172.21.15.86:0/3127821974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0f0069a80 msgr2=0x7fb0f0198ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.294+0000 7fb0ef7fe640 1 --2- 172.21.15.86:0/3127821974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0f0069a80 0x7fb0f0198ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.294+0000 7fb0ef7fe640 1 -- 172.21.15.86:0/3127821974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb0f019d570 con 0x7fb0f00690e0 2024-04-21T08:49:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.294+0000 7fb0e7fff640 1 --2- 172.21.15.86:0/3127821974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0f0069a80 0x7fb0f0198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.294+0000 7fb0ef7fe640 1 --2- 172.21.15.86:0/3127821974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0f00690e0 0x7fb0f0198580 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0005ec0 tx=0x7fb0e002ca90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:08.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.294+0000 7fb0ed7fa640 1 -- 172.21.15.86:0/3127821974 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0043020 con 0x7fb0f00690e0 2024-04-21T08:49:08.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.294+0000 7fb0ed7fa640 1 -- 172.21.15.86:0/3127821974 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb0e002cc80 con 0x7fb0f00690e0 2024-04-21T08:49:08.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.294+0000 7fb0f61e6640 1 -- 172.21.15.86:0/3127821974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb0f019d800 con 0x7fb0f00690e0 2024-04-21T08:49:08.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.294+0000 7fb0ed7fa640 1 -- 172.21.15.86:0/3127821974 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb0e0003c90 con 0x7fb0f00690e0 2024-04-21T08:49:08.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.294+0000 7fb0f61e6640 1 -- 172.21.15.86:0/3127821974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb0f01a50f0 con 0x7fb0f00690e0 2024-04-21T08:49:08.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.295+0000 7fb0f61e6640 1 -- 172.21.15.86:0/3127821974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fb0f0068de0 con 0x7fb0f00690e0 2024-04-21T08:49:08.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.296+0000 7fb0ed7fa640 1 -- 172.21.15.86:0/3127821974 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb0e0046020 con 0x7fb0f00690e0 2024-04-21T08:49:08.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.297+0000 7fb0ed7fa640 1 --2- 172.21.15.86:0/3127821974 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb0dc0a3d20 0x7fb0dc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.297+0000 7fb0e7fff640 1 --2- 172.21.15.86:0/3127821974 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb0dc0a3d20 0x7fb0dc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.297+0000 7fb0ed7fa640 1 -- 172.21.15.86:0/3127821974 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9584+0+0 (secure 0 0 0) 0x7fb0e00f3230 con 0x7fb0f00690e0 2024-04-21T08:49:08.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.297+0000 7fb0e7fff640 1 --2- 172.21.15.86:0/3127821974 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb0dc0a3d20 0x7fb0dc0a61e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0107680 tx=0x7fb0d8005d90 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:08.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.300+0000 7fb0ed7fa640 1 -- 172.21.15.86:0/3127821974 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fb0e004d020 con 0x7fb0f00690e0 2024-04-21T08:49:08.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:08 smithi086 ceph-mon[27939]: pgmap v322: 129 pgs: 129 active+clean; 2.5 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 87 KiB/s wr, 36 op/s 2024-04-21T08:49:08.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:08 smithi179 ceph-mon[31829]: pgmap v322: 129 pgs: 129 active+clean; 2.5 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 87 KiB/s wr, 36 op/s 2024-04-21T08:49:08.517 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.518+0000 7fb0f61e6640 1 -- 172.21.15.86:0/3127821974 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fb0f01a52c0 con 0x7fb0f00690e0 2024-04-21T08:49:08.518 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.519+0000 7fb0ed7fa640 1 -- 172.21.15.86:0/3127821974 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7fb0e00bf5c0 con 0x7fb0f00690e0 2024-04-21T08:49:08.518 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:49:08.518 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:38:45.202126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24398,24394,24455,24472]},"id":1}]} 2024-04-21T08:49:08.519 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-21T08:49:08.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.521+0000 7fb0f61e6640 1 -- 172.21.15.86:0/3127821974 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb0dc0a3d20 msgr2=0x7fb0dc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.521+0000 7fb0f61e6640 1 --2- 172.21.15.86:0/3127821974 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb0dc0a3d20 0x7fb0dc0a61e0 secure :-1 s=READY pgs=304 cs=0 l=1 rev1=1 crypto rx=0x7fb0f0107680 tx=0x7fb0d8005d90 comp rx=0 tx=0).stop 2024-04-21T08:49:08.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.521+0000 7fb0f61e6640 1 -- 172.21.15.86:0/3127821974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0f00690e0 msgr2=0x7fb0f0198580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.521+0000 7fb0f61e6640 1 --2- 172.21.15.86:0/3127821974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0f00690e0 0x7fb0f0198580 secure :-1 s=READY pgs=1838 cs=0 l=1 rev1=1 crypto rx=0x7fb0e0005ec0 tx=0x7fb0e002ca90 comp rx=0 tx=0).stop 2024-04-21T08:49:08.520 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.521+0000 7fb0f61e6640 1 -- 172.21.15.86:0/3127821974 shutdown_connections 2024-04-21T08:49:08.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.521+0000 7fb0f61e6640 1 --2- 172.21.15.86:0/3127821974 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb0dc0a3d20 0x7fb0dc0a61e0 unknown :-1 s=CLOSED pgs=304 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.521+0000 7fb0f61e6640 1 --2- 172.21.15.86:0/3127821974 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb0f0107f70 0x7fb0f019ce40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.521+0000 7fb0f61e6640 1 --2- 172.21.15.86:0/3127821974 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb0f0069a80 0x7fb0f0198ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.521+0000 7fb0f61e6640 1 --2- 172.21.15.86:0/3127821974 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb0f00690e0 0x7fb0f0198580 unknown :-1 s=CLOSED pgs=1838 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.521+0000 7fb0f61e6640 1 -- 172.21.15.86:0/3127821974 >> 172.21.15.86:0/3127821974 conn(0x7fb0f0077e30 msgr2=0x7fb0f00ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:08.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.521+0000 7fb0f61e6640 1 -- 172.21.15.86:0/3127821974 shutdown_connections 2024-04-21T08:49:08.521 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.521+0000 7fb0f61e6640 1 -- 172.21.15.86:0/3127821974 wait complete. 2024-04-21T08:49:08.531 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:49:08.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.633+0000 7f333c006640 1 -- 172.21.15.86:0/3038467665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3334101a50 msgr2=0x7f3334105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.633+0000 7f333c006640 1 --2- 172.21.15.86:0/3038467665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3334101a50 0x7f3334105a90 secure :-1 s=READY pgs=1839 cs=0 l=1 rev1=1 crypto rx=0x7f3324009900 tx=0x7f332402bc30 comp rx=0 tx=0).stop 2024-04-21T08:49:08.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.633+0000 7f333c006640 1 -- 172.21.15.86:0/3038467665 shutdown_connections 2024-04-21T08:49:08.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.633+0000 7f333c006640 1 --2- 172.21.15.86:0/3038467665 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f3334106170 0x7f333410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.633+0000 7f333c006640 1 --2- 172.21.15.86:0/3038467665 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f3334101a50 0x7f3334105a90 unknown :-1 s=CLOSED pgs=1839 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.633+0000 7f333c006640 1 --2- 172.21.15.86:0/3038467665 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f33341010a0 0x7f3334101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.633+0000 7f333c006640 1 -- 172.21.15.86:0/3038467665 >> 172.21.15.86:0/3038467665 conn(0x7f3334077eb0 msgr2=0x7f33340782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:08.650 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.633+0000 7f333c006640 1 -- 172.21.15.86:0/3038467665 shutdown_connections 2024-04-21T08:49:08.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.633+0000 7f333c006640 1 -- 172.21.15.86:0/3038467665 wait complete. 2024-04-21T08:49:08.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.633+0000 7f333c006640 1 Processor -- start 2024-04-21T08:49:08.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.633+0000 7f333c006640 1 -- start start 2024-04-21T08:49:08.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.633+0000 7f333c006640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33341010a0 0x7f33341986d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.633+0000 7f333c006640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3334106170 0x7f3334198c10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.633+0000 7f333c006640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f333419cf90 0x7f333419d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.634+0000 7f333c006640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f33341121f0 con 0x7f333419cf90 2024-04-21T08:49:08.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.634+0000 7f333c006640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3334112070 con 0x7f33341010a0 2024-04-21T08:49:08.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.634+0000 7f333c006640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3334112370 con 0x7f3334106170 2024-04-21T08:49:08.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.634+0000 7f333a5a3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f333419cf90 0x7f333419d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.634+0000 7f333a5a3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f333419cf90 0x7f333419d440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60968/0 (socket says 172.21.15.86:60968) 2024-04-21T08:49:08.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.634+0000 7f333a5a3640 1 -- 172.21.15.86:0/4075458183 learned_addr learned my addr 172.21.15.86:0/4075458183 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:08.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.634+0000 7f3339da2640 1 --2- 172.21.15.86:0/4075458183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33341010a0 0x7f33341986d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.634+0000 7f33395a1640 1 --2- 172.21.15.86:0/4075458183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3334106170 0x7f3334198c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.634+0000 7f333a5a3640 1 -- 172.21.15.86:0/4075458183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3334106170 msgr2=0x7f3334198c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.634+0000 7f333a5a3640 1 --2- 172.21.15.86:0/4075458183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3334106170 0x7f3334198c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.634+0000 7f333a5a3640 1 -- 172.21.15.86:0/4075458183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33341010a0 msgr2=0x7f33341986d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.653 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.634+0000 7f333a5a3640 1 --2- 172.21.15.86:0/4075458183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33341010a0 0x7f33341986d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.634+0000 7f333a5a3640 1 -- 172.21.15.86:0/4075458183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f333419da80 con 0x7f333419cf90 2024-04-21T08:49:08.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.634+0000 7f33395a1640 1 --2- 172.21.15.86:0/4075458183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3334106170 0x7f3334198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:08.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.634+0000 7f333a5a3640 1 --2- 172.21.15.86:0/4075458183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f333419cf90 0x7f333419d440 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f333000c970 tx=0x7f333000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:08.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.634+0000 7f332affd640 1 -- 172.21.15.86:0/4075458183 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3330007bb0 con 0x7f333419cf90 2024-04-21T08:49:08.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.634+0000 7f332affd640 1 -- 172.21.15.86:0/4075458183 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3330007d50 con 0x7f333419cf90 2024-04-21T08:49:08.654 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.634+0000 7f333c006640 1 -- 172.21.15.86:0/4075458183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f333419dd70 con 0x7f333419cf90 2024-04-21T08:49:08.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.634+0000 7f332affd640 1 -- 172.21.15.86:0/4075458183 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3330018770 con 0x7f333419cf90 2024-04-21T08:49:08.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.634+0000 7f333c006640 1 -- 172.21.15.86:0/4075458183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f33341a5510 con 0x7f333419cf90 2024-04-21T08:49:08.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.637+0000 7f332affd640 1 -- 172.21.15.86:0/4075458183 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f333000f990 con 0x7f333419cf90 2024-04-21T08:49:08.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.637+0000 7f333ebf6740 1 Processor -- start 2024-04-21T08:49:08.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.637+0000 7f333ebf6740 1 -- start start 2024-04-21T08:49:08.655 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.638+0000 7f333ebf6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b340e6d4d0 0x55b340e6d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.638+0000 7f333ebf6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b340e6de80 0x55b340e76430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.638+0000 7f333ebf6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b340e76a00 0x55b340e78e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.638+0000 7f333ebf6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b340dd21f0 con 0x55b340e6d4d0 2024-04-21T08:49:08.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.638+0000 7f333ebf6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b340d6f4b0 con 0x55b340e6de80 2024-04-21T08:49:08.656 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.638+0000 7f333ebf6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b340dce1b0 con 0x55b340e76a00 2024-04-21T08:49:08.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.638+0000 7f33395a1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b340e6d4d0 0x55b340e6d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.638+0000 7f33395a1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b340e6d4d0 0x55b340e6d8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60980/0 (socket says 172.21.15.86:60980) 2024-04-21T08:49:08.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.638+0000 7f33395a1640 1 -- 172.21.15.86:0/3203309135 learned_addr learned my addr 172.21.15.86:0/3203309135 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:08.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.638+0000 7f332affd640 1 --2- 172.21.15.86:0/4075458183 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f33200a3d20 0x7f33200a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.638+0000 7f3339da2640 1 --2- 172.21.15.86:0/3203309135 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b340e6de80 0x55b340e76430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.657 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.638+0000 7f33395a1640 1 --2- 172.21.15.86:0/3203309135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b340e76a00 0x55b340e78e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.638+0000 7f333a5a3640 1 --2- 172.21.15.86:0/4075458183 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f33200a3d20 0x7f33200a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.638+0000 7f33395a1640 1 -- 172.21.15.86:0/3203309135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b340e76a00 msgr2=0x55b340e78e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.638+0000 7f33395a1640 1 --2- 172.21.15.86:0/3203309135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b340e76a00 0x55b340e78e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.638+0000 7f33395a1640 1 -- 172.21.15.86:0/3203309135 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b340e6de80 msgr2=0x55b340e76430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.638+0000 7f33395a1640 1 --2- 172.21.15.86:0/3203309135 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b340e6de80 0x55b340e76430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.658 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.638+0000 7f33395a1640 1 -- 172.21.15.86:0/3203309135 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b340d611b0 con 0x55b340e6d4d0 2024-04-21T08:49:08.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.638+0000 7f332affd640 1 -- 172.21.15.86:0/4075458183 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9584+0+0 (secure 0 0 0) 0x7f33300d0cb0 con 0x7f333419cf90 2024-04-21T08:49:08.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.638+0000 7f33395a1640 1 --2- 172.21.15.86:0/3203309135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b340e6d4d0 0x55b340e6d8b0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f3324005ec0 tx=0x7f3324009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb7685b576b1e828 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:08.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.638+0000 7f333a5a3640 1 --2- 172.21.15.86:0/4075458183 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f33200a3d20 0x7f33200a61e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f333009c530 tx=0x7f333006e270 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:08.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.638+0000 7f331ffff640 1 -- 172.21.15.86:0/3203309135 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332402c9e0 con 0x55b340e6d4d0 2024-04-21T08:49:08.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.638+0000 7f331ffff640 1 -- 172.21.15.86:0/3203309135 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3324047070 con 0x55b340e6d4d0 2024-04-21T08:49:08.659 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.638+0000 7f331ffff640 1 -- 172.21.15.86:0/3203309135 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3324004a40 con 0x55b340e6d4d0 2024-04-21T08:49:08.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.639+0000 7f333ebf6740 1 -- 172.21.15.86:0/3203309135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b340e6d4d0 msgr2=0x55b340e6d8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.639+0000 7f333ebf6740 1 --2- 172.21.15.86:0/3203309135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b340e6d4d0 0x55b340e6d8b0 secure :-1 s=READY pgs=1841 cs=0 l=1 rev1=1 crypto rx=0x7f3324005ec0 tx=0x7f3324009560 comp rx=0 tx=0).stop 2024-04-21T08:49:08.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.639+0000 7f333ebf6740 1 -- 172.21.15.86:0/3203309135 shutdown_connections 2024-04-21T08:49:08.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.639+0000 7f333ebf6740 1 --2- 172.21.15.86:0/3203309135 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b340e76a00 0x55b340e78e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.660 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.639+0000 7f333ebf6740 1 --2- 172.21.15.86:0/3203309135 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b340e6de80 0x55b340e76430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.639+0000 7f333ebf6740 1 --2- 172.21.15.86:0/3203309135 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b340e6d4d0 0x55b340e6d8b0 unknown :-1 s=CLOSED pgs=1841 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.639+0000 7f333ebf6740 1 -- 172.21.15.86:0/3203309135 >> 172.21.15.86:0/3203309135 conn(0x55b340e67ba0 msgr2=0x55b340e67f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:08.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.639+0000 7f333ebf6740 1 -- 172.21.15.86:0/3203309135 shutdown_connections 2024-04-21T08:49:08.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.639+0000 7f333ebf6740 1 -- 172.21.15.86:0/3203309135 wait complete. 2024-04-21T08:49:08.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.640+0000 7f333ebf6740 1 -- start start 2024-04-21T08:49:08.661 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.640+0000 7f333ebf6740 1 Processor -- start 2024-04-21T08:49:08.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.640+0000 7f333ebf6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b340e6d4d0 0x55b340e762d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.640+0000 7f333ebf6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b340e6de80 0x55b340e73660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.640+0000 7f333ebf6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b340e76a00 0x55b340e73bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.640+0000 7f333ebf6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b340e79910 con 0x55b340e76a00 2024-04-21T08:49:08.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.640+0000 7f333ebf6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b340d8b500 con 0x55b340e6d4d0 2024-04-21T08:49:08.662 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.640+0000 7f333ebf6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b340e79a90 con 0x55b340e6de80 2024-04-21T08:49:08.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.640+0000 7f333ebf6740 10 client.0 _ll_register_callbacks cb 0x55b340d59b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:08.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.640+0000 7f33395a1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b340e76a00 0x55b340e73bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.640+0000 7f33395a1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b340e76a00 0x55b340e73bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60984/0 (socket says 172.21.15.86:60984) 2024-04-21T08:49:08.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.640+0000 7f33395a1640 1 -- 172.21.15.86:0/1520178682 learned_addr learned my addr 172.21.15.86:0/1520178682 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:08.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.640+0000 7f33395a1640 1 --2- 172.21.15.86:0/1520178682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b340e6d4d0 0x55b340e762d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.663 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.640+0000 7f3339da2640 1 --2- 172.21.15.86:0/1520178682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b340e6de80 0x55b340e73660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.640+0000 7f33395a1640 1 -- 172.21.15.86:0/1520178682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b340e6de80 msgr2=0x55b340e73660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.640+0000 7f33395a1640 1 --2- 172.21.15.86:0/1520178682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b340e6de80 0x55b340e73660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.640+0000 7f33395a1640 1 -- 172.21.15.86:0/1520178682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b340e6d4d0 msgr2=0x55b340e762d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.640+0000 7f33395a1640 1 --2- 172.21.15.86:0/1520178682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b340e6d4d0 0x55b340e762d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.664 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.640+0000 7f33395a1640 1 -- 172.21.15.86:0/1520178682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b340ae43f0 con 0x55b340e76a00 2024-04-21T08:49:08.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.641+0000 7f33395a1640 1 --2- 172.21.15.86:0/1520178682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b340e76a00 0x55b340e73bc0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f3324005a80 tx=0x7f332402cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a3cfed46c0885f19 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:08.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.641+0000 7f331d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:08.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.641+0000 7f331d7fa640 1 -- 172.21.15.86:0/1520178682 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3324051030 con 0x55b340e76a00 2024-04-21T08:49:08.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.641+0000 7f333ebf6740 1 -- 172.21.15.86:0/1520178682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3324047070 con 0x55b340e76a00 2024-04-21T08:49:08.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.641+0000 7f331d7fa640 1 -- 172.21.15.86:0/1520178682 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9584+0+0 (secure 0 0 0) 0x7f33240047c0 con 0x55b340e76a00 2024-04-21T08:49:08.665 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.641+0000 7f331d7fa640 1 -- 172.21.15.86:0/1520178682 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f3324061050 con 0x55b340e76a00 2024-04-21T08:49:08.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.641+0000 7f333ebf6740 10 client.19746 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:08.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.641+0000 7f333ebf6740 1 -- 172.21.15.86:0/1520178682 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b340bae7d0 con 0x55b340e76a00 2024-04-21T08:49:08.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.641+0000 7f331d7fa640 1 -- 172.21.15.86:0/1520178682 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3324047070 con 0x55b340e76a00 2024-04-21T08:49:08.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.642+0000 7f331d7fa640 1 -- 172.21.15.86:0/1520178682 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f3324003b50 con 0x55b340e76a00 2024-04-21T08:49:08.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.642+0000 7f333ebf6740 10 client.19746 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:08.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.642+0000 7f333ebf6740 10 client.19746 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:49:08.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.642+0000 7f333ebf6740 20 client.19746 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:08.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.642+0000 7f333ebf6740 1 --2- 172.21.15.86:0/1520178682 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55b340e86c80 0x55b340e890b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.642+0000 7f333ebf6740 4 client.19746 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:08.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.642+0000 7f333ebf6740 1 -- 172.21.15.86:0/1520178682 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b340d71930 con 0x55b340e86c80 2024-04-21T08:49:08.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.642+0000 7f3339da2640 1 --2- 172.21.15.86:0/1520178682 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55b340e86c80 0x55b340e890b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.667 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.642+0000 7f3339da2640 1 --2- 172.21.15.86:0/1520178682 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55b340e86c80 0x55b340e890b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=98d092e3e7778a33 server_cookie=e41f75ac8a53057f in_seq=0 out_seq=0 2024-04-21T08:49:08.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.642+0000 7f331d7fa640 10 client.19746 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:49:08.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.643+0000 7f331d7fa640 1 -- 172.21.15.86:0/1520178682 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b340d71930 con 0x55b340e86c80 2024-04-21T08:49:08.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.643+0000 7f331d7fa640 10 client.19746 handle_command_reply: tid=0 2024-04-21T08:49:08.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.643+0000 7f333ebf6740 1 client.19746 shutdown 2024-04-21T08:49:08.668 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.643+0000 7f333ebf6740 1 -- 172.21.15.86:0/1520178682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b340e76a00 msgr2=0x55b340e73bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.643+0000 7f333ebf6740 1 --2- 172.21.15.86:0/1520178682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b340e76a00 0x55b340e73bc0 secure :-1 s=READY pgs=1842 cs=0 l=1 rev1=1 crypto rx=0x7f3324005a80 tx=0x7f332402cd90 comp rx=0 tx=0).stop 2024-04-21T08:49:08.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.643+0000 7f333ebf6740 1 -- 172.21.15.86:0/1520178682 shutdown_connections 2024-04-21T08:49:08.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.643+0000 7f333ebf6740 1 --2- 172.21.15.86:0/1520178682 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b340e76a00 0x55b340e73bc0 unknown :-1 s=CLOSED pgs=1842 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.643+0000 7f333ebf6740 1 --2- 172.21.15.86:0/1520178682 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b340e6de80 0x55b340e73660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.643+0000 7f333ebf6740 1 --2- 172.21.15.86:0/1520178682 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55b340e86c80 0x55b340e890b0 crc :-1 s=READY pgs=70 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.669 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.643+0000 7f333ebf6740 1 --2- 172.21.15.86:0/1520178682 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b340e6d4d0 0x55b340e762d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.643+0000 7f331d7fa640 0 client.19746 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:49:08.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.643+0000 7f333ebf6740 1 -- 172.21.15.86:0/1520178682 >> 172.21.15.86:0/1520178682 conn(0x55b340e67ba0 msgr2=0x55b340e698f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:08.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.643+0000 7f333ebf6740 1 -- 172.21.15.86:0/1520178682 shutdown_connections 2024-04-21T08:49:08.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.643+0000 7f333ebf6740 1 -- 172.21.15.86:0/1520178682 wait complete. 2024-04-21T08:49:08.670 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.644+0000 7f333ebf6740 20 client.19746 trim_cache size 0 max 16384 2024-04-21T08:49:08.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.656+0000 7f333ebf6740 1 Processor -- start 2024-04-21T08:49:08.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.656+0000 7f333ebf6740 1 -- start start 2024-04-21T08:49:08.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.657+0000 7f333ebf6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b340e6d4d0 0x55b340e827d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.657+0000 7f333ebf6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b340e6de80 0x55b340e66a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.657+0000 7f333ebf6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b340e76a00 0x55b340e75d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.657+0000 7f333ebf6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f33341af3a0 con 0x55b340e6d4d0 2024-04-21T08:49:08.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.657+0000 7f333ebf6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f33341b0ee0 con 0x55b340e6de80 2024-04-21T08:49:08.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.657+0000 7f333ebf6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f33341afa70 con 0x55b340e76a00 2024-04-21T08:49:08.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.657+0000 7f33395a1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b340e6d4d0 0x55b340e827d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.657+0000 7f33395a1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b340e6d4d0 0x55b340e827d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60992/0 (socket says 172.21.15.86:60992) 2024-04-21T08:49:08.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.657+0000 7f33395a1640 1 -- 172.21.15.86:0/1014795829 learned_addr learned my addr 172.21.15.86:0/1014795829 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:08.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.657+0000 7f3339da2640 1 --2- 172.21.15.86:0/1014795829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b340e6de80 0x55b340e66a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.657+0000 7f33395a1640 1 -- 172.21.15.86:0/1014795829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b340e76a00 msgr2=0x55b340e75d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.657+0000 7f33395a1640 1 --2- 172.21.15.86:0/1014795829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b340e76a00 0x55b340e75d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.657+0000 7f33395a1640 1 -- 172.21.15.86:0/1014795829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b340e6de80 msgr2=0x55b340e66a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.657+0000 7f33395a1640 1 --2- 172.21.15.86:0/1014795829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b340e6de80 0x55b340e66a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.685 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.657+0000 7f33395a1640 1 -- 172.21.15.86:0/1014795829 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b340e73f30 con 0x55b340e6d4d0 2024-04-21T08:49:08.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.657+0000 7f3339da2640 1 --2- 172.21.15.86:0/1014795829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b340e6de80 0x55b340e66a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:08.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.657+0000 7f33395a1640 1 --2- 172.21.15.86:0/1014795829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b340e6d4d0 0x55b340e827d0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f3324004650 tx=0x7f3324004230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a42014d4dbf17877 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:08.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.657+0000 7f331f7fe640 1 -- 172.21.15.86:0/1014795829 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3324051030 con 0x55b340e6d4d0 2024-04-21T08:49:08.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.657+0000 7f331f7fe640 1 -- 172.21.15.86:0/1014795829 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3324047070 con 0x55b340e6d4d0 2024-04-21T08:49:08.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.657+0000 7f331f7fe640 1 -- 172.21.15.86:0/1014795829 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f3324034c10 con 0x55b340e6d4d0 2024-04-21T08:49:08.686 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.658+0000 7f333ebf6740 1 -- 172.21.15.86:0/1014795829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b340e6d4d0 msgr2=0x55b340e827d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.658+0000 7f333ebf6740 1 --2- 172.21.15.86:0/1014795829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b340e6d4d0 0x55b340e827d0 secure :-1 s=READY pgs=1843 cs=0 l=1 rev1=1 crypto rx=0x7f3324004650 tx=0x7f3324004230 comp rx=0 tx=0).stop 2024-04-21T08:49:08.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.658+0000 7f333ebf6740 1 -- 172.21.15.86:0/1014795829 shutdown_connections 2024-04-21T08:49:08.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.658+0000 7f333ebf6740 1 --2- 172.21.15.86:0/1014795829 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b340e76a00 0x55b340e75d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.658+0000 7f333ebf6740 1 --2- 172.21.15.86:0/1014795829 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b340e6de80 0x55b340e66a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.658+0000 7f333ebf6740 1 --2- 172.21.15.86:0/1014795829 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b340e6d4d0 0x55b340e827d0 unknown :-1 s=CLOSED pgs=1843 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.687 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.658+0000 7f333ebf6740 1 -- 172.21.15.86:0/1014795829 >> 172.21.15.86:0/1014795829 conn(0x55b340e67ba0 msgr2=0x55b340e66370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:08.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.658+0000 7f333ebf6740 1 -- 172.21.15.86:0/1014795829 shutdown_connections 2024-04-21T08:49:08.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.658+0000 7f333ebf6740 1 -- 172.21.15.86:0/1014795829 wait complete. 2024-04-21T08:49:08.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.658+0000 7f333ebf6740 1 -- start start 2024-04-21T08:49:08.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.658+0000 7f333ebf6740 1 Processor -- start 2024-04-21T08:49:08.688 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.659+0000 7f333ebf6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b340e6d4d0 0x55b340e79e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.659+0000 7f333ebf6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b340e6de80 0x55b340e7a3d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.659+0000 7f333ebf6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b340e76a00 0x55b340eeb220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.659+0000 7f333ebf6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f33341b0810 con 0x55b340e6de80 2024-04-21T08:49:08.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.659+0000 7f333ebf6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f33341b0140 con 0x55b340e6d4d0 2024-04-21T08:49:08.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.659+0000 7f333ebf6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f33341ad380 con 0x55b340e76a00 2024-04-21T08:49:08.689 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.659+0000 7f333ebf6740 10 client.0 _ll_register_callbacks cb 0x7f332402ca80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:08.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.659+0000 7f3339da2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b340e6de80 0x55b340e7a3d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.659+0000 7f3339da2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b340e6de80 0x55b340e7a3d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60994/0 (socket says 172.21.15.86:60994) 2024-04-21T08:49:08.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.659+0000 7f3339da2640 1 -- 172.21.15.86:0/4294196377 learned_addr learned my addr 172.21.15.86:0/4294196377 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:08.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.659+0000 7f33395a1640 1 --2- 172.21.15.86:0/4294196377 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b340e6d4d0 0x55b340e79e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.659+0000 7f3339da2640 1 -- 172.21.15.86:0/4294196377 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b340e76a00 msgr2=0x55b340eeb220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.690 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.659+0000 7f33395a1640 1 --2- 172.21.15.86:0/4294196377 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b340e76a00 0x55b340eeb220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.659+0000 7f3339da2640 1 --2- 172.21.15.86:0/4294196377 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b340e76a00 0x55b340eeb220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.659+0000 7f3339da2640 1 -- 172.21.15.86:0/4294196377 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b340e6d4d0 msgr2=0x55b340e79e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.659+0000 7f3339da2640 1 --2- 172.21.15.86:0/4294196377 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b340e6d4d0 0x55b340e79e90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.659+0000 7f3339da2640 1 -- 172.21.15.86:0/4294196377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b340eeb760 con 0x55b340e6de80 2024-04-21T08:49:08.691 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.659+0000 7f33395a1640 1 --2- 172.21.15.86:0/4294196377 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b340e6d4d0 0x55b340e79e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:08.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.659+0000 7f3339da2640 1 --2- 172.21.15.86:0/4294196377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b340e6de80 0x55b340e7a3d0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f332c00a940 tx=0x7f332c011c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c0ab868e7772e52a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:08.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.659+0000 7f331d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:08.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.659+0000 7f331d7fa640 1 -- 172.21.15.86:0/4294196377 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f332c004ab0 con 0x55b340e6de80 2024-04-21T08:49:08.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.659+0000 7f331d7fa640 1 -- 172.21.15.86:0/4294196377 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9584+0+0 (secure 0 0 0) 0x7f332c004c50 con 0x55b340e6de80 2024-04-21T08:49:08.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.659+0000 7f333ebf6740 1 -- 172.21.15.86:0/4294196377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f3324047070 con 0x55b340e6de80 2024-04-21T08:49:08.692 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.659+0000 7f331d7fa640 1 -- 172.21.15.86:0/4294196377 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f332c029050 con 0x55b340e6de80 2024-04-21T08:49:08.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.659+0000 7f333ebf6740 10 client.19752 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:08.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.660+0000 7f333ebf6740 1 -- 172.21.15.86:0/4294196377 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b340eec960 con 0x55b340e6de80 2024-04-21T08:49:08.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.660+0000 7f331d7fa640 1 -- 172.21.15.86:0/4294196377 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f3324047070 con 0x55b340e6de80 2024-04-21T08:49:08.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.660+0000 7f331d7fa640 1 -- 172.21.15.86:0/4294196377 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f332c01a6e0 con 0x55b340e6de80 2024-04-21T08:49:08.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.660+0000 7f333ebf6740 10 client.19752 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:08.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.660+0000 7f333ebf6740 10 client.19752 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:49:08.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.660+0000 7f333ebf6740 20 client.19752 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:08.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.660+0000 7f333ebf6740 1 --2- 172.21.15.86:0/4294196377 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55b340eed090 0x55b340eef4b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.660+0000 7f333ebf6740 4 client.19752 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:49:08.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.660+0000 7f333ebf6740 1 -- 172.21.15.86:0/4294196377 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b340ddfc10 con 0x55b340eed090 2024-04-21T08:49:08.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.660+0000 7f33395a1640 1 --2- 172.21.15.86:0/4294196377 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55b340eed090 0x55b340eef4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.661+0000 7f33395a1640 1 --2- 172.21.15.86:0/4294196377 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55b340eed090 0x55b340eef4b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=7ffa34a30744ad48 server_cookie=105b4d2f72e3f0a9 in_seq=0 out_seq=0 2024-04-21T08:49:08.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.661+0000 7f331d7fa640 10 client.19752 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:49:08.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.661+0000 7f331d7fa640 1 -- 172.21.15.86:0/4294196377 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b340ddfc10 con 0x55b340eed090 2024-04-21T08:49:08.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.661+0000 7f331d7fa640 10 client.19752 handle_command_reply: tid=0 2024-04-21T08:49:08.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.661+0000 7f333ebf6740 1 client.19752 shutdown 2024-04-21T08:49:08.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.661+0000 7f333ebf6740 1 -- 172.21.15.86:0/4294196377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b340e6de80 msgr2=0x55b340e7a3d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.661+0000 7f333ebf6740 1 --2- 172.21.15.86:0/4294196377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b340e6de80 0x55b340e7a3d0 secure :-1 s=READY pgs=1844 cs=0 l=1 rev1=1 crypto rx=0x7f332c00a940 tx=0x7f332c011c40 comp rx=0 tx=0).stop 2024-04-21T08:49:08.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.661+0000 7f333ebf6740 1 -- 172.21.15.86:0/4294196377 shutdown_connections 2024-04-21T08:49:08.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.661+0000 7f333ebf6740 1 --2- 172.21.15.86:0/4294196377 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b340e76a00 0x55b340eeb220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.661+0000 7f333ebf6740 1 --2- 172.21.15.86:0/4294196377 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b340e6de80 0x55b340e7a3d0 unknown :-1 s=CLOSED pgs=1844 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.661+0000 7f333ebf6740 1 --2- 172.21.15.86:0/4294196377 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55b340eed090 0x55b340eef4b0 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.661+0000 7f333ebf6740 1 --2- 172.21.15.86:0/4294196377 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b340e6d4d0 0x55b340e79e90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.661+0000 7f333ebf6740 1 -- 172.21.15.86:0/4294196377 >> 172.21.15.86:0/4294196377 conn(0x55b340e67ba0 msgr2=0x55b340e66c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:08.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.661+0000 7f331d7fa640 0 client.19752 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:49:08.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.661+0000 7f333ebf6740 1 -- 172.21.15.86:0/4294196377 shutdown_connections 2024-04-21T08:49:08.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.661+0000 7f333ebf6740 1 -- 172.21.15.86:0/4294196377 wait complete. 2024-04-21T08:49:08.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.661+0000 7f333ebf6740 20 client.19752 trim_cache size 0 max 16384 2024-04-21T08:49:08.698 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:49:08.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.662+0000 7f333c006640 1 -- 172.21.15.86:0/4075458183 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f33200a3d20 msgr2=0x7f33200a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.662+0000 7f333c006640 1 --2- 172.21.15.86:0/4075458183 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f33200a3d20 0x7f33200a61e0 secure :-1 s=READY pgs=305 cs=0 l=1 rev1=1 crypto rx=0x7f333009c530 tx=0x7f333006e270 comp rx=0 tx=0).stop 2024-04-21T08:49:08.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.662+0000 7f333c006640 1 -- 172.21.15.86:0/4075458183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f333419cf90 msgr2=0x7f333419d440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.662+0000 7f333c006640 1 --2- 172.21.15.86:0/4075458183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f333419cf90 0x7f333419d440 secure :-1 s=READY pgs=1840 cs=0 l=1 rev1=1 crypto rx=0x7f333000c970 tx=0x7f333000ce30 comp rx=0 tx=0).stop 2024-04-21T08:49:08.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.662+0000 7f333c006640 1 -- 172.21.15.86:0/4075458183 shutdown_connections 2024-04-21T08:49:08.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.662+0000 7f333c006640 1 --2- 172.21.15.86:0/4075458183 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f33200a3d20 0x7f33200a61e0 unknown :-1 s=CLOSED pgs=305 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.662+0000 7f333c006640 1 --2- 172.21.15.86:0/4075458183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f333419cf90 0x7f333419d440 unknown :-1 s=CLOSED pgs=1840 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.701 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.662+0000 7f333c006640 1 --2- 172.21.15.86:0/4075458183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f3334106170 0x7f3334198c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.662+0000 7f333c006640 1 --2- 172.21.15.86:0/4075458183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f33341010a0 0x7f33341986d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.662+0000 7f333c006640 1 -- 172.21.15.86:0/4075458183 >> 172.21.15.86:0/4075458183 conn(0x7f3334077eb0 msgr2=0x7f3334100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:08.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.662+0000 7f333c006640 1 -- 172.21.15.86:0/4075458183 shutdown_connections 2024-04-21T08:49:08.702 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.662+0000 7f333c006640 1 -- 172.21.15.86:0/4075458183 wait complete. 2024-04-21T08:49:08.703 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:49:08.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:08 smithi088 ceph-mon[31259]: pgmap v322: 129 pgs: 129 active+clean; 2.5 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 46 KiB/s rd, 87 KiB/s wr, 36 op/s 2024-04-21T08:49:08.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.805+0000 7fc8a1ca5640 1 -- 172.21.15.86:0/1850888143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc89c101b40 msgr2=0x7fc89c10dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.805+0000 7fc8a1ca5640 1 --2- 172.21.15.86:0/1850888143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc89c101b40 0x7fc89c10dc80 secure :-1 s=READY pgs=1845 cs=0 l=1 rev1=1 crypto rx=0x7fc894009cd0 tx=0x7fc89402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.805+0000 7fc8a1ca5640 1 -- 172.21.15.86:0/1850888143 shutdown_connections 2024-04-21T08:49:08.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.805+0000 7fc8a1ca5640 1 --2- 172.21.15.86:0/1850888143 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc89c101b40 0x7fc89c10dc80 unknown :-1 s=CLOSED pgs=1845 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.805+0000 7fc8a1ca5640 1 --2- 172.21.15.86:0/1850888143 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc89c1011a0 0x7fc89c101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.805+0000 7fc8a1ca5640 1 --2- 172.21.15.86:0/1850888143 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc89c107030 0x7fc89c107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.805+0000 7fc8a1ca5640 1 -- 172.21.15.86:0/1850888143 >> 172.21.15.86:0/1850888143 conn(0x7fc89c076c50 msgr2=0x7fc89c077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:08.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.805+0000 7fc8a1ca5640 1 -- 172.21.15.86:0/1850888143 shutdown_connections 2024-04-21T08:49:08.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.805+0000 7fc8a1ca5640 1 -- 172.21.15.86:0/1850888143 wait complete. 2024-04-21T08:49:08.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.806+0000 7fc8a1ca5640 1 Processor -- start 2024-04-21T08:49:08.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.806+0000 7fc8a1ca5640 1 -- start start 2024-04-21T08:49:08.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.806+0000 7fc8a1ca5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc89c1011a0 0x7fc89c197660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.806+0000 7fc8a1ca5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc89c101b40 0x7fc89c197ba0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.806+0000 7fc8a1ca5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc89c107030 0x7fc89c19bf20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.806+0000 7fc8a1ca5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c1110e0 con 0x7fc89c107030 2024-04-21T08:49:08.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.806+0000 7fc8a1ca5640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c110f60 con 0x7fc89c1011a0 2024-04-21T08:49:08.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.806+0000 7fc8a1ca5640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c111260 con 0x7fc89c101b40 2024-04-21T08:49:08.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.806+0000 7fc89bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc89c107030 0x7fc89c19bf20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.806+0000 7fc89bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc89c107030 0x7fc89c19bf20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32780/0 (socket says 172.21.15.86:32780) 2024-04-21T08:49:08.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.806+0000 7fc89bfff640 1 -- 172.21.15.86:0/1532318604 learned_addr learned my addr 172.21.15.86:0/1532318604 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:08.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.806+0000 7fc89b7fe640 1 --2- 172.21.15.86:0/1532318604 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc89c1011a0 0x7fc89c197660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.806+0000 7fc89bfff640 1 -- 172.21.15.86:0/1532318604 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc89c101b40 msgr2=0x7fc89c197ba0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.806+0000 7fc89affd640 1 --2- 172.21.15.86:0/1532318604 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc89c101b40 0x7fc89c197ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.806+0000 7fc89bfff640 1 --2- 172.21.15.86:0/1532318604 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc89c101b40 0x7fc89c197ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.806+0000 7fc89bfff640 1 -- 172.21.15.86:0/1532318604 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc89c1011a0 msgr2=0x7fc89c197660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.806+0000 7fc89bfff640 1 --2- 172.21.15.86:0/1532318604 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc89c1011a0 0x7fc89c197660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.806+0000 7fc89bfff640 1 -- 172.21.15.86:0/1532318604 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc89c19c650 con 0x7fc89c107030 2024-04-21T08:49:08.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.806+0000 7fc89b7fe640 1 --2- 172.21.15.86:0/1532318604 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc89c1011a0 0x7fc89c197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:08.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.807+0000 7fc89bfff640 1 --2- 172.21.15.86:0/1532318604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc89c107030 0x7fc89c19bf20 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fc894033000 tx=0x7fc8940346e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:08.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.807+0000 7fc898ff9640 1 -- 172.21.15.86:0/1532318604 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc894041020 con 0x7fc89c107030 2024-04-21T08:49:08.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.807+0000 7fc898ff9640 1 -- 172.21.15.86:0/1532318604 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc894032970 con 0x7fc89c107030 2024-04-21T08:49:08.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.807+0000 7fc8a1ca5640 1 -- 172.21.15.86:0/1532318604 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc89c19c8e0 con 0x7fc89c107030 2024-04-21T08:49:08.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.807+0000 7fc898ff9640 1 -- 172.21.15.86:0/1532318604 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc894039040 con 0x7fc89c107030 2024-04-21T08:49:08.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.807+0000 7fc8a1ca5640 1 -- 172.21.15.86:0/1532318604 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc89c1a41d0 con 0x7fc89c107030 2024-04-21T08:49:08.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.809+0000 7fc898ff9640 1 -- 172.21.15.86:0/1532318604 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc894044020 con 0x7fc89c107030 2024-04-21T08:49:08.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.809+0000 7fc8a4973740 1 Processor -- start 2024-04-21T08:49:08.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.809+0000 7fc8a4973740 1 -- start start 2024-04-21T08:49:08.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.809+0000 7fc8a4973740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556df7247340 0x556df7247720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.809+0000 7fc8a4973740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556df7247cf0 0x556df724e620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.809+0000 7fc8a4973740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556df724ebf0 0x556df724f070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.809+0000 7fc8a4973740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556df71ac1f0 con 0x556df7247cf0 2024-04-21T08:49:08.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.809+0000 7fc8a4973740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x556df71494b0 con 0x556df724ebf0 2024-04-21T08:49:08.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.809+0000 7fc8a4973740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556df71a81b0 con 0x556df7247340 2024-04-21T08:49:08.816 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.809+0000 7fc898ff9640 1 --2- 172.21.15.86:0/1532318604 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc8840a3d20 0x7fc8840a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.809+0000 7fc89b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556df7247cf0 0x556df724e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.809+0000 7fc89b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556df7247cf0 0x556df724e620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32786/0 (socket says 172.21.15.86:32786) 2024-04-21T08:49:08.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.809+0000 7fc89b7fe640 1 -- 172.21.15.86:0/3323143417 learned_addr learned my addr 172.21.15.86:0/3323143417 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:08.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.810+0000 7fc89bfff640 1 --2- 172.21.15.86:0/1532318604 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc8840a3d20 0x7fc8840a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.810+0000 7fc89affd640 1 --2- 172.21.15.86:0/3323143417 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556df724ebf0 0x556df724f070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.810+0000 7fc89affd640 1 --2- 172.21.15.86:0/3323143417 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556df7247340 0x556df7247720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.810+0000 7fc898ff9640 1 -- 172.21.15.86:0/1532318604 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9584+0+0 (secure 0 0 0) 0x7fc8940f2970 con 0x7fc89c107030 2024-04-21T08:49:08.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.810+0000 7fc89b7fe640 1 -- 172.21.15.86:0/3323143417 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556df7247340 msgr2=0x556df7247720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.810+0000 7fc89b7fe640 1 --2- 172.21.15.86:0/3323143417 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556df7247340 0x556df7247720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.810+0000 7fc89b7fe640 1 -- 172.21.15.86:0/3323143417 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556df724ebf0 msgr2=0x556df724f070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.810+0000 7fc89b7fe640 1 --2- 172.21.15.86:0/3323143417 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556df724ebf0 0x556df724f070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.810+0000 7fc89b7fe640 1 -- 172.21.15.86:0/3323143417 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556df713b1b0 con 0x556df7247cf0 2024-04-21T08:49:08.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.810+0000 7fc89affd640 1 --2- 172.21.15.86:0/3323143417 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556df724ebf0 0x556df724f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:08.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.810+0000 7fc89b7fe640 1 --2- 172.21.15.86:0/3323143417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556df7247cf0 0x556df724e620 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fc88c009c50 tx=0x7fc88c00e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=87b4b11a30ddd642 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:08.817 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.810+0000 7fc89bfff640 1 --2- 172.21.15.86:0/1532318604 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc8840a3d20 0x7fc8840a61e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc8940be590 tx=0x7fc8940a9040 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:08.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.810+0000 7fc881ffb640 1 -- 172.21.15.86:0/3323143417 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88c016070 con 0x556df7247cf0 2024-04-21T08:49:08.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.810+0000 7fc881ffb640 1 -- 172.21.15.86:0/3323143417 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc88c0054d0 con 0x556df7247cf0 2024-04-21T08:49:08.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.810+0000 7fc881ffb640 1 -- 172.21.15.86:0/3323143417 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88c011470 con 0x556df7247cf0 2024-04-21T08:49:08.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.811+0000 7fc8a4973740 1 -- 172.21.15.86:0/3323143417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556df7247cf0 msgr2=0x556df724e620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.811+0000 7fc8a4973740 1 --2- 172.21.15.86:0/3323143417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556df7247cf0 0x556df724e620 secure :-1 s=READY pgs=1847 cs=0 l=1 rev1=1 crypto rx=0x7fc88c009c50 tx=0x7fc88c00e6c0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.811+0000 7fc8a4973740 1 -- 172.21.15.86:0/3323143417 shutdown_connections 2024-04-21T08:49:08.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.811+0000 7fc8a4973740 1 --2- 172.21.15.86:0/3323143417 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556df724ebf0 0x556df724f070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.811+0000 7fc8a4973740 1 --2- 172.21.15.86:0/3323143417 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556df7247cf0 0x556df724e620 unknown :-1 s=CLOSED pgs=1847 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.811+0000 7fc8a4973740 1 --2- 172.21.15.86:0/3323143417 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556df7247340 0x556df7247720 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.811+0000 7fc8a4973740 1 -- 172.21.15.86:0/3323143417 >> 172.21.15.86:0/3323143417 conn(0x556df7241ba0 msgr2=0x556df7241f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:08.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.811+0000 7fc8a4973740 1 -- 172.21.15.86:0/3323143417 shutdown_connections 2024-04-21T08:49:08.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.811+0000 7fc8a4973740 1 -- 172.21.15.86:0/3323143417 wait complete. 2024-04-21T08:49:08.818 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.811+0000 7fc8a4973740 1 -- start start 2024-04-21T08:49:08.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.811+0000 7fc8a4973740 1 Processor -- start 2024-04-21T08:49:08.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.811+0000 7fc8a4973740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556df7247340 0x556df7254f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.812+0000 7fc8a4973740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556df7247cf0 0x556df72522b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.812+0000 7fc8a4973740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556df724ebf0 0x556df72527f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.812+0000 7fc8a4973740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556df7255b90 con 0x556df7247340 2024-04-21T08:49:08.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.812+0000 7fc8a4973740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x556df7165500 con 0x556df724ebf0 2024-04-21T08:49:08.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.812+0000 7fc8a4973740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556df7255d10 con 0x556df7247cf0 2024-04-21T08:49:08.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.812+0000 7fc8a4973740 10 client.0 _ll_register_callbacks cb 0x556df7133b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:08.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.812+0000 7fc89affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556df7247340 0x556df7254f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.812+0000 7fc89affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556df7247340 0x556df7254f20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32800/0 (socket says 172.21.15.86:32800) 2024-04-21T08:49:08.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.812+0000 7fc89affd640 1 -- 172.21.15.86:0/1665239704 learned_addr learned my addr 172.21.15.86:0/1665239704 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:08.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.812+0000 7fc89affd640 1 --2- 172.21.15.86:0/1665239704 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556df724ebf0 0x556df72527f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.812+0000 7fc89b7fe640 1 --2- 172.21.15.86:0/1665239704 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556df7247cf0 0x556df72522b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.812+0000 7fc89affd640 1 -- 172.21.15.86:0/1665239704 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556df7247cf0 msgr2=0x556df72522b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.819 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.812+0000 7fc89affd640 1 --2- 172.21.15.86:0/1665239704 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556df7247cf0 0x556df72522b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.812+0000 7fc89affd640 1 -- 172.21.15.86:0/1665239704 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556df724ebf0 msgr2=0x556df72527f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.812+0000 7fc89affd640 1 --2- 172.21.15.86:0/1665239704 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556df724ebf0 0x556df72527f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.812+0000 7fc89affd640 1 -- 172.21.15.86:0/1665239704 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556df6ebe3f0 con 0x556df7247340 2024-04-21T08:49:08.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.812+0000 7fc89b7fe640 1 --2- 172.21.15.86:0/1665239704 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556df7247cf0 0x556df72522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:08.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.812+0000 7fc89affd640 1 --2- 172.21.15.86:0/1665239704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556df7247340 0x556df7254f20 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fc890013e20 tx=0x7fc89000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6c4bf337ab6b6bdf server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:08.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.812+0000 7fc87affd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:08.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.812+0000 7fc87affd640 1 -- 172.21.15.86:0/1665239704 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc890005020 con 0x556df7247340 2024-04-21T08:49:08.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.812+0000 7fc87affd640 1 -- 172.21.15.86:0/1665239704 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9584+0+0 (secure 0 0 0) 0x7fc8900051c0 con 0x556df7247340 2024-04-21T08:49:08.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.812+0000 7fc8a4973740 1 -- 172.21.15.86:0/1665239704 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc88c0054d0 con 0x556df7247340 2024-04-21T08:49:08.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.812+0000 7fc87affd640 1 -- 172.21.15.86:0/1665239704 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc890027050 con 0x556df7247340 2024-04-21T08:49:08.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.812+0000 7fc8a4973740 10 client.19764 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:08.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.812+0000 7fc8a4973740 1 -- 172.21.15.86:0/1665239704 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556df6f887d0 con 0x556df7247340 2024-04-21T08:49:08.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.813+0000 7fc87affd640 1 -- 172.21.15.86:0/1665239704 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc88c0054d0 con 0x556df7247340 2024-04-21T08:49:08.820 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.813+0000 7fc87affd640 1 -- 172.21.15.86:0/1665239704 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc89001d030 con 0x556df7247340 2024-04-21T08:49:08.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.813+0000 7fc8a4973740 10 client.19764 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:08.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.813+0000 7fc8a4973740 10 client.19764 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:49:08.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.813+0000 7fc8a4973740 20 client.19764 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:08.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.813+0000 7fc8a4973740 1 --2- 172.21.15.86:0/1665239704 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x556df7262dd0 0x556df72651c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.813+0000 7fc8a4973740 4 client.19764 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:08.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.813+0000 7fc8a4973740 1 -- 172.21.15.86:0/1665239704 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556df714b930 con 0x556df7262dd0 2024-04-21T08:49:08.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.813+0000 7fc89b7fe640 1 --2- 172.21.15.86:0/1665239704 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x556df7262dd0 0x556df72651c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.814+0000 7fc89b7fe640 1 --2- 172.21.15.86:0/1665239704 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x556df7262dd0 0x556df72651c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=c1cab4683eecb555 server_cookie=2f73013390dc675b in_seq=0 out_seq=0 2024-04-21T08:49:08.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.814+0000 7fc87affd640 10 client.19764 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:49:08.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.814+0000 7fc87affd640 1 -- 172.21.15.86:0/1665239704 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556df714b930 con 0x556df7262dd0 2024-04-21T08:49:08.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.814+0000 7fc87affd640 10 client.19764 handle_command_reply: tid=0 2024-04-21T08:49:08.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.814+0000 7fc8a4973740 1 client.19764 shutdown 2024-04-21T08:49:08.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.814+0000 7fc8a4973740 1 -- 172.21.15.86:0/1665239704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556df7247340 msgr2=0x556df7254f20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.814+0000 7fc8a4973740 1 --2- 172.21.15.86:0/1665239704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556df7247340 0x556df7254f20 secure :-1 s=READY pgs=1848 cs=0 l=1 rev1=1 crypto rx=0x7fc890013e20 tx=0x7fc89000e720 comp rx=0 tx=0).stop 2024-04-21T08:49:08.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.814+0000 7fc8a4973740 1 -- 172.21.15.86:0/1665239704 shutdown_connections 2024-04-21T08:49:08.821 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.814+0000 7fc8a4973740 1 --2- 172.21.15.86:0/1665239704 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x556df7262dd0 0x556df72651c0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.814+0000 7fc8a4973740 1 --2- 172.21.15.86:0/1665239704 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556df724ebf0 0x556df72527f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.814+0000 7fc8a4973740 1 --2- 172.21.15.86:0/1665239704 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556df7247cf0 0x556df72522b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.814+0000 7fc87affd640 0 client.19764 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:49:08.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.814+0000 7fc8a4973740 1 --2- 172.21.15.86:0/1665239704 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556df7247340 0x556df7254f20 unknown :-1 s=CLOSED pgs=1848 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.814+0000 7fc8a4973740 1 -- 172.21.15.86:0/1665239704 >> 172.21.15.86:0/1665239704 conn(0x556df7241ba0 msgr2=0x556df72438f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:08.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.815+0000 7fc8a4973740 1 -- 172.21.15.86:0/1665239704 shutdown_connections 2024-04-21T08:49:08.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.815+0000 7fc8a4973740 1 -- 172.21.15.86:0/1665239704 wait complete. 2024-04-21T08:49:08.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.815+0000 7fc8a4973740 20 client.19764 trim_cache size 0 max 16384 2024-04-21T08:49:08.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.828+0000 7fc8a4973740 1 Processor -- start 2024-04-21T08:49:08.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.828+0000 7fc8a4973740 1 -- start start 2024-04-21T08:49:08.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.828+0000 7fc8a4973740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556df7247340 0x556df725eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.828+0000 7fc8a4973740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556df7247cf0 0x556df72549e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.828+0000 7fc8a4973740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556df724ebf0 0x556df7254f20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.828+0000 7fc8a4973740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c1ad2e0 con 0x556df7247cf0 2024-04-21T08:49:08.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.828+0000 7fc8a4973740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c1aee20 con 0x556df7247340 2024-04-21T08:49:08.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.828+0000 7fc8a4973740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c1ad9b0 con 0x556df724ebf0 2024-04-21T08:49:08.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.828+0000 7fc89b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556df7247cf0 0x556df72549e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.828+0000 7fc89b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556df7247cf0 0x556df72549e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32814/0 (socket says 172.21.15.86:32814) 2024-04-21T08:49:08.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.828+0000 7fc89b7fe640 1 -- 172.21.15.86:0/2428453303 learned_addr learned my addr 172.21.15.86:0/2428453303 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:08.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.828+0000 7fc89affd640 1 --2- 172.21.15.86:0/2428453303 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556df7247340 0x556df725eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.828+0000 7fc89affd640 1 --2- 172.21.15.86:0/2428453303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556df724ebf0 0x556df7254f20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.828+0000 7fc89b7fe640 1 -- 172.21.15.86:0/2428453303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556df724ebf0 msgr2=0x556df7254f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.828+0000 7fc89b7fe640 1 --2- 172.21.15.86:0/2428453303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556df724ebf0 0x556df7254f20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.828+0000 7fc89b7fe640 1 -- 172.21.15.86:0/2428453303 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556df7247340 msgr2=0x556df725eaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.828+0000 7fc89b7fe640 1 --2- 172.21.15.86:0/2428453303 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556df7247340 0x556df725eaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.828+0000 7fc89b7fe640 1 -- 172.21.15.86:0/2428453303 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556df7247a90 con 0x556df7247cf0 2024-04-21T08:49:08.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.828+0000 7fc89affd640 1 --2- 172.21.15.86:0/2428453303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556df724ebf0 0x556df7254f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:08.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.828+0000 7fc89b7fe640 1 --2- 172.21.15.86:0/2428453303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556df7247cf0 0x556df72549e0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fc88c004a90 tx=0x7fc88c0099a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e86ce246e75fdab server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:08.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.828+0000 7fc8817fa640 1 -- 172.21.15.86:0/2428453303 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88c016040 con 0x556df7247cf0 2024-04-21T08:49:08.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.829+0000 7fc8817fa640 1 -- 172.21.15.86:0/2428453303 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc88c004680 con 0x556df7247cf0 2024-04-21T08:49:08.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.829+0000 7fc8817fa640 1 -- 172.21.15.86:0/2428453303 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc88c003ab0 con 0x556df7247cf0 2024-04-21T08:49:08.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.829+0000 7fc8a4973740 1 -- 172.21.15.86:0/2428453303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556df7247cf0 msgr2=0x556df72549e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.829+0000 7fc8a4973740 1 --2- 172.21.15.86:0/2428453303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556df7247cf0 0x556df72549e0 secure :-1 s=READY pgs=1849 cs=0 l=1 rev1=1 crypto rx=0x7fc88c004a90 tx=0x7fc88c0099a0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.829+0000 7fc8a4973740 1 -- 172.21.15.86:0/2428453303 shutdown_connections 2024-04-21T08:49:08.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.829+0000 7fc8a4973740 1 --2- 172.21.15.86:0/2428453303 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556df724ebf0 0x556df7254f20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.829+0000 7fc8a4973740 1 --2- 172.21.15.86:0/2428453303 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556df7247cf0 0x556df72549e0 unknown :-1 s=CLOSED pgs=1849 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.829+0000 7fc8a4973740 1 --2- 172.21.15.86:0/2428453303 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556df7247340 0x556df725eaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.829+0000 7fc8a4973740 1 -- 172.21.15.86:0/2428453303 >> 172.21.15.86:0/2428453303 conn(0x556df7241ba0 msgr2=0x556df724dff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:08.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.829+0000 7fc8a4973740 1 -- 172.21.15.86:0/2428453303 shutdown_connections 2024-04-21T08:49:08.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.829+0000 7fc8a4973740 1 -- 172.21.15.86:0/2428453303 wait complete. 2024-04-21T08:49:08.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.829+0000 7fc8a4973740 1 -- start start 2024-04-21T08:49:08.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.830+0000 7fc8a4973740 1 Processor -- start 2024-04-21T08:49:08.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.830+0000 7fc8a4973740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556df7247340 0x556df725b000 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.830+0000 7fc8a4973740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556df7247cf0 0x556df725b540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.830+0000 7fc8a4973740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556df724ebf0 0x556df7253960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.830+0000 7fc8a4973740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c1ae750 con 0x556df7247340 2024-04-21T08:49:08.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.830+0000 7fc8a4973740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c1ae080 con 0x556df724ebf0 2024-04-21T08:49:08.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.830+0000 7fc8a4973740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc89c1ab2c0 con 0x556df7247cf0 2024-04-21T08:49:08.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.830+0000 7fc8a4973740 10 client.0 _ll_register_callbacks cb 0x7fc8900139b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:08.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.830+0000 7fc89affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556df7247340 0x556df725b000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.830+0000 7fc89affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556df7247340 0x556df725b000 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32816/0 (socket says 172.21.15.86:32816) 2024-04-21T08:49:08.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.830+0000 7fc89affd640 1 -- 172.21.15.86:0/294740227 learned_addr learned my addr 172.21.15.86:0/294740227 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:08.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.830+0000 7fc89b7fe640 1 --2- 172.21.15.86:0/294740227 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556df7247cf0 0x556df725b540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.830+0000 7fc89affd640 1 -- 172.21.15.86:0/294740227 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556df7247cf0 msgr2=0x556df725b540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.830+0000 7fc89affd640 1 --2- 172.21.15.86:0/294740227 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556df7247cf0 0x556df725b540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.830+0000 7fc89affd640 1 -- 172.21.15.86:0/294740227 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556df724ebf0 msgr2=0x556df7253960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.830+0000 7fc89affd640 1 --2- 172.21.15.86:0/294740227 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556df724ebf0 0x556df7253960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.830+0000 7fc89affd640 1 -- 172.21.15.86:0/294740227 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556df7265530 con 0x556df7247340 2024-04-21T08:49:08.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.830+0000 7fc89b7fe640 1 --2- 172.21.15.86:0/294740227 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556df7247cf0 0x556df725b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:08.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.830+0000 7fc89affd640 1 --2- 172.21.15.86:0/294740227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556df7247340 0x556df725b000 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fc89000e480 tx=0x7fc890010cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b74f2d4781117663 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:08.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.830+0000 7fc87affd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:08.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.830+0000 7fc87affd640 1 -- 172.21.15.86:0/294740227 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc890030060 con 0x556df7247340 2024-04-21T08:49:08.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.831+0000 7fc87affd640 1 -- 172.21.15.86:0/294740227 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9584+0+0 (secure 0 0 0) 0x7fc89001eb90 con 0x556df7247340 2024-04-21T08:49:08.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.831+0000 7fc8a4973740 1 -- 172.21.15.86:0/294740227 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc88c004680 con 0x556df7247340 2024-04-21T08:49:08.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.831+0000 7fc87affd640 1 -- 172.21.15.86:0/294740227 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc890037050 con 0x556df7247340 2024-04-21T08:49:08.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.831+0000 7fc8a4973740 10 client.19770 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:08.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.831+0000 7fc8a4973740 1 -- 172.21.15.86:0/294740227 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556df72c8c90 con 0x556df7247340 2024-04-21T08:49:08.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.831+0000 7fc87affd640 1 -- 172.21.15.86:0/294740227 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc88c004680 con 0x556df7247340 2024-04-21T08:49:08.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.831+0000 7fc87affd640 1 -- 172.21.15.86:0/294740227 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc89001d070 con 0x556df7247340 2024-04-21T08:49:08.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.831+0000 7fc8a4973740 10 client.19770 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:08.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.831+0000 7fc8a4973740 10 client.19770 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:49:08.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.831+0000 7fc8a4973740 20 client.19770 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:08.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.831+0000 7fc8a4973740 1 --2- 172.21.15.86:0/294740227 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x556df72c9440 0x556df72cb860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.831+0000 7fc8a4973740 4 client.19770 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:49:08.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.831+0000 7fc8a4973740 1 -- 172.21.15.86:0/294740227 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556df71b9c10 con 0x556df72c9440 2024-04-21T08:49:08.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.832+0000 7fc89b7fe640 1 --2- 172.21.15.86:0/294740227 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x556df72c9440 0x556df72cb860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.832+0000 7fc87affd640 10 client.19770 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:49:08.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.832+0000 7fc89b7fe640 1 --2- 172.21.15.86:0/294740227 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x556df72c9440 0x556df72cb860 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=d42ba7a1ab69834e server_cookie=2dad22e54ce838fe in_seq=0 out_seq=0 2024-04-21T08:49:08.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.832+0000 7fc87affd640 1 -- 172.21.15.86:0/294740227 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556df71b9c10 con 0x556df72c9440 2024-04-21T08:49:08.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.832+0000 7fc87affd640 10 client.19770 handle_command_reply: tid=0 2024-04-21T08:49:08.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.832+0000 7fc8a4973740 1 client.19770 shutdown 2024-04-21T08:49:08.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.833+0000 7fc8a4973740 1 -- 172.21.15.86:0/294740227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556df7247340 msgr2=0x556df725b000 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.833+0000 7fc8a4973740 1 --2- 172.21.15.86:0/294740227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556df7247340 0x556df725b000 secure :-1 s=READY pgs=1850 cs=0 l=1 rev1=1 crypto rx=0x7fc89000e480 tx=0x7fc890010cb0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.833+0000 7fc8a4973740 1 -- 172.21.15.86:0/294740227 shutdown_connections 2024-04-21T08:49:08.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.833+0000 7fc8a4973740 1 --2- 172.21.15.86:0/294740227 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x556df72c9440 0x556df72cb860 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.833+0000 7fc8a4973740 1 --2- 172.21.15.86:0/294740227 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556df724ebf0 0x556df7253960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.833+0000 7fc8a4973740 1 --2- 172.21.15.86:0/294740227 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556df7247cf0 0x556df725b540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.833+0000 7fc87affd640 0 client.19770 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:49:08.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.833+0000 7fc8a4973740 1 --2- 172.21.15.86:0/294740227 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556df7247340 0x556df725b000 unknown :-1 s=CLOSED pgs=1850 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.833+0000 7fc8a4973740 1 -- 172.21.15.86:0/294740227 >> 172.21.15.86:0/294740227 conn(0x556df7241ba0 msgr2=0x556df7250aa0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:08.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.833+0000 7fc8a4973740 1 -- 172.21.15.86:0/294740227 shutdown_connections 2024-04-21T08:49:08.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.833+0000 7fc8a4973740 1 -- 172.21.15.86:0/294740227 wait complete. 2024-04-21T08:49:08.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.833+0000 7fc8a4973740 20 client.19770 trim_cache size 0 max 16384 2024-04-21T08:49:08.839 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:49:08.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.833+0000 7fc8a1ca5640 1 -- 172.21.15.86:0/1532318604 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc8840a3d20 msgr2=0x7fc8840a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.833+0000 7fc8a1ca5640 1 --2- 172.21.15.86:0/1532318604 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc8840a3d20 0x7fc8840a61e0 secure :-1 s=READY pgs=306 cs=0 l=1 rev1=1 crypto rx=0x7fc8940be590 tx=0x7fc8940a9040 comp rx=0 tx=0).stop 2024-04-21T08:49:08.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.833+0000 7fc8a1ca5640 1 -- 172.21.15.86:0/1532318604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc89c107030 msgr2=0x7fc89c19bf20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.833+0000 7fc8a1ca5640 1 --2- 172.21.15.86:0/1532318604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc89c107030 0x7fc89c19bf20 secure :-1 s=READY pgs=1846 cs=0 l=1 rev1=1 crypto rx=0x7fc894033000 tx=0x7fc8940346e0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.833+0000 7fc8a1ca5640 1 -- 172.21.15.86:0/1532318604 shutdown_connections 2024-04-21T08:49:08.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.833+0000 7fc8a1ca5640 1 --2- 172.21.15.86:0/1532318604 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc8840a3d20 0x7fc8840a61e0 unknown :-1 s=CLOSED pgs=306 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.833+0000 7fc8a1ca5640 1 --2- 172.21.15.86:0/1532318604 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc89c107030 0x7fc89c19bf20 unknown :-1 s=CLOSED pgs=1846 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.833+0000 7fc8a1ca5640 1 --2- 172.21.15.86:0/1532318604 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc89c101b40 0x7fc89c197ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.833+0000 7fc8a1ca5640 1 --2- 172.21.15.86:0/1532318604 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc89c1011a0 0x7fc89c197660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.833+0000 7fc8a1ca5640 1 -- 172.21.15.86:0/1532318604 >> 172.21.15.86:0/1532318604 conn(0x7fc89c076c50 msgr2=0x7fc89c0ffe10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:08.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.834+0000 7fc8a1ca5640 1 -- 172.21.15.86:0/1532318604 shutdown_connections 2024-04-21T08:49:08.841 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.834+0000 7fc8a1ca5640 1 -- 172.21.15.86:0/1532318604 wait complete. 2024-04-21T08:49:08.843 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:49:08.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.946+0000 7f5a2f0eb640 1 -- 172.21.15.86:0/990627707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a2806a7c0 msgr2=0x7f5a2806aba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.946+0000 7f5a2f0eb640 1 --2- 172.21.15.86:0/990627707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a2806a7c0 0x7f5a2806aba0 secure :-1 s=READY pgs=1851 cs=0 l=1 rev1=1 crypto rx=0x7f5a18009900 tx=0x7f5a1802bc30 comp rx=0 tx=0).stop 2024-04-21T08:49:08.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.946+0000 7f5a2f0eb640 1 -- 172.21.15.86:0/990627707 shutdown_connections 2024-04-21T08:49:08.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.946+0000 7f5a2f0eb640 1 --2- 172.21.15.86:0/990627707 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a28107280 0x7f5a2810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.946+0000 7f5a2f0eb640 1 --2- 172.21.15.86:0/990627707 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a2806b170 0x7f5a28106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.946+0000 7f5a2f0eb640 1 --2- 172.21.15.86:0/990627707 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a2806a7c0 0x7f5a2806aba0 unknown :-1 s=CLOSED pgs=1851 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.946+0000 7f5a2f0eb640 1 -- 172.21.15.86:0/990627707 >> 172.21.15.86:0/990627707 conn(0x7f5a28076c50 msgr2=0x7f5a28077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:08.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.947+0000 7f5a2f0eb640 1 -- 172.21.15.86:0/990627707 shutdown_connections 2024-04-21T08:49:08.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.947+0000 7f5a2f0eb640 1 -- 172.21.15.86:0/990627707 wait complete. 2024-04-21T08:49:08.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.947+0000 7f5a2f0eb640 1 Processor -- start 2024-04-21T08:49:08.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.947+0000 7f5a2f0eb640 1 -- start start 2024-04-21T08:49:08.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.947+0000 7f5a2f0eb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a2806a7c0 0x7f5a28198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.947+0000 7f5a2f0eb640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a28107280 0x7f5a28198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.947+0000 7f5a2f0eb640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a2819d000 0x7f5a2819d4b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.947+0000 7f5a2f0eb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5a281121e0 con 0x7f5a2806a7c0 2024-04-21T08:49:08.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.947+0000 7f5a2f0eb640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5a28112060 con 0x7f5a2819d000 2024-04-21T08:49:08.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.947+0000 7f5a2f0eb640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5a28112360 con 0x7f5a28107280 2024-04-21T08:49:08.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.947+0000 7f5a2ce87640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a2806a7c0 0x7f5a28198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.947+0000 7f5a2ce87640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a2806a7c0 0x7f5a28198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32838/0 (socket says 172.21.15.86:32838) 2024-04-21T08:49:08.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.947+0000 7f5a2ce87640 1 -- 172.21.15.86:0/4176931635 learned_addr learned my addr 172.21.15.86:0/4176931635 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:08.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.947+0000 7f5a27fff640 1 --2- 172.21.15.86:0/4176931635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a28107280 0x7f5a28198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.948+0000 7f5a2d688640 1 --2- 172.21.15.86:0/4176931635 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a2819d000 0x7f5a2819d4b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.948+0000 7f5a2ce87640 1 -- 172.21.15.86:0/4176931635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a28107280 msgr2=0x7f5a28198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.948+0000 7f5a2ce87640 1 --2- 172.21.15.86:0/4176931635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a28107280 0x7f5a28198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.948+0000 7f5a2ce87640 1 -- 172.21.15.86:0/4176931635 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a2819d000 msgr2=0x7f5a2819d4b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.948+0000 7f5a2ce87640 1 --2- 172.21.15.86:0/4176931635 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a2819d000 0x7f5a2819d4b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.948+0000 7f5a2ce87640 1 -- 172.21.15.86:0/4176931635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5a2819daf0 con 0x7f5a2806a7c0 2024-04-21T08:49:08.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.948+0000 7f5a27fff640 1 --2- 172.21.15.86:0/4176931635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a28107280 0x7f5a28198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:08.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.948+0000 7f5a2ce87640 1 --2- 172.21.15.86:0/4176931635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a2806a7c0 0x7f5a28198740 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f5a18005940 tx=0x7f5a1802c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:08.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.948+0000 7f5a25ffb640 1 -- 172.21.15.86:0/4176931635 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a18043020 con 0x7f5a2806a7c0 2024-04-21T08:49:08.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.948+0000 7f5a25ffb640 1 -- 172.21.15.86:0/4176931635 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a1802cbe0 con 0x7f5a2806a7c0 2024-04-21T08:49:08.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.948+0000 7f5a2f0eb640 1 -- 172.21.15.86:0/4176931635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5a2819dd80 con 0x7f5a2806a7c0 2024-04-21T08:49:08.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.948+0000 7f5a25ffb640 1 -- 172.21.15.86:0/4176931635 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a18003bf0 con 0x7f5a2806a7c0 2024-04-21T08:49:08.950 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.948+0000 7f5a2f0eb640 1 -- 172.21.15.86:0/4176931635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5a281a55a0 con 0x7f5a2806a7c0 2024-04-21T08:49:08.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.950+0000 7f5a25ffb640 1 -- 172.21.15.86:0/4176931635 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5a18046020 con 0x7f5a2806a7c0 2024-04-21T08:49:08.956 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.951+0000 7f5a31db9740 1 Processor -- start 2024-04-21T08:49:08.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.951+0000 7f5a31db9740 1 -- start start 2024-04-21T08:49:08.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.951+0000 7f5a31db9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556e19086330 0x556e19086710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.951+0000 7f5a31db9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e19086ce0 0x556e1908f820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.951+0000 7f5a31db9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1908fe40 0x556e190902c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.951+0000 7f5a31db9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556e18feb1f0 con 0x556e1908fe40 2024-04-21T08:49:08.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.951+0000 7f5a31db9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x556e18f884b0 con 0x556e19086330 2024-04-21T08:49:08.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.951+0000 7f5a31db9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556e18fe71b0 con 0x556e19086ce0 2024-04-21T08:49:08.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.951+0000 7f5a25ffb640 1 --2- 172.21.15.86:0/4176931635 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5a100a3d20 0x7f5a100a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.951+0000 7f5a27fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1908fe40 0x556e190902c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.951+0000 7f5a27fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1908fe40 0x556e190902c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32846/0 (socket says 172.21.15.86:32846) 2024-04-21T08:49:08.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.951+0000 7f5a27fff640 1 -- 172.21.15.86:0/2671700610 learned_addr learned my addr 172.21.15.86:0/2671700610 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:08.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.951+0000 7f5a2d688640 1 --2- 172.21.15.86:0/4176931635 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5a100a3d20 0x7f5a100a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.951+0000 7f5a25ffb640 1 -- 172.21.15.86:0/4176931635 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9584+0+0 (secure 0 0 0) 0x7f5a180f2e00 con 0x7f5a2806a7c0 2024-04-21T08:49:08.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.951+0000 7f5a2ce87640 1 --2- 172.21.15.86:0/2671700610 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e19086ce0 0x556e1908f820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.951+0000 7f5a2d688640 1 --2- 172.21.15.86:0/2671700610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556e19086330 0x556e19086710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.951+0000 7f5a27fff640 1 -- 172.21.15.86:0/2671700610 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e19086ce0 msgr2=0x556e1908f820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.951+0000 7f5a27fff640 1 --2- 172.21.15.86:0/2671700610 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e19086ce0 0x556e1908f820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.951+0000 7f5a27fff640 1 -- 172.21.15.86:0/2671700610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556e19086330 msgr2=0x556e19086710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.951+0000 7f5a27fff640 1 --2- 172.21.15.86:0/2671700610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556e19086330 0x556e19086710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.951+0000 7f5a27fff640 1 -- 172.21.15.86:0/2671700610 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e18f7a1b0 con 0x556e1908fe40 2024-04-21T08:49:08.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.952+0000 7f5a27fff640 1 --2- 172.21.15.86:0/2671700610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1908fe40 0x556e190902c0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c00bcd0 tx=0x7f5a1c00f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed0f2e984b3f053c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:08.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.952+0000 7f5a2d688640 1 --2- 172.21.15.86:0/4176931635 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5a100a3d20 0x7f5a100a61e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5a20005fd0 tx=0x7f5a20009480 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:08.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.952+0000 7f5a0effd640 1 -- 172.21.15.86:0/2671700610 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c015070 con 0x556e1908fe40 2024-04-21T08:49:08.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.952+0000 7f5a0effd640 1 -- 172.21.15.86:0/2671700610 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a1c01a910 con 0x556e1908fe40 2024-04-21T08:49:08.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.952+0000 7f5a0effd640 1 -- 172.21.15.86:0/2671700610 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c011960 con 0x556e1908fe40 2024-04-21T08:49:08.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.952+0000 7f5a31db9740 1 -- 172.21.15.86:0/2671700610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1908fe40 msgr2=0x556e190902c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.952+0000 7f5a31db9740 1 --2- 172.21.15.86:0/2671700610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1908fe40 0x556e190902c0 secure :-1 s=READY pgs=1853 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c00bcd0 tx=0x7f5a1c00f730 comp rx=0 tx=0).stop 2024-04-21T08:49:08.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.952+0000 7f5a31db9740 1 -- 172.21.15.86:0/2671700610 shutdown_connections 2024-04-21T08:49:08.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.952+0000 7f5a31db9740 1 --2- 172.21.15.86:0/2671700610 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1908fe40 0x556e190902c0 unknown :-1 s=CLOSED pgs=1853 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.952+0000 7f5a31db9740 1 --2- 172.21.15.86:0/2671700610 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e19086ce0 0x556e1908f820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.952+0000 7f5a31db9740 1 --2- 172.21.15.86:0/2671700610 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556e19086330 0x556e19086710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.952+0000 7f5a31db9740 1 -- 172.21.15.86:0/2671700610 >> 172.21.15.86:0/2671700610 conn(0x556e19080ba0 msgr2=0x556e19080f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:08.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.952+0000 7f5a31db9740 1 -- 172.21.15.86:0/2671700610 shutdown_connections 2024-04-21T08:49:08.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.952+0000 7f5a31db9740 1 -- 172.21.15.86:0/2671700610 wait complete. 2024-04-21T08:49:08.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.953+0000 7f5a31db9740 1 -- start start 2024-04-21T08:49:08.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.953+0000 7f5a31db9740 1 Processor -- start 2024-04-21T08:49:08.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.953+0000 7f5a31db9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556e19086330 0x556e19096260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.953+0000 7f5a31db9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e19086ce0 0x556e190935f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.953+0000 7f5a31db9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e1908fe40 0x556e19093b70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.953+0000 7f5a31db9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556e19096dc0 con 0x556e19086ce0 2024-04-21T08:49:08.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.953+0000 7f5a31db9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x556e18fa4500 con 0x556e19086330 2024-04-21T08:49:08.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.953+0000 7f5a31db9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556e19096f40 con 0x556e1908fe40 2024-04-21T08:49:08.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.953+0000 7f5a31db9740 10 client.0 _ll_register_callbacks cb 0x556e18f72b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:08.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.953+0000 7f5a27fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e19086ce0 0x556e190935f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.959 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.953+0000 7f5a27fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e19086ce0 0x556e190935f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32850/0 (socket says 172.21.15.86:32850) 2024-04-21T08:49:08.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.953+0000 7f5a27fff640 1 -- 172.21.15.86:0/4066554461 learned_addr learned my addr 172.21.15.86:0/4066554461 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:08.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.953+0000 7f5a2ce87640 1 --2- 172.21.15.86:0/4066554461 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556e19086330 0x556e19096260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.953+0000 7f5a2d688640 1 --2- 172.21.15.86:0/4066554461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e1908fe40 0x556e19093b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.954+0000 7f5a27fff640 1 -- 172.21.15.86:0/4066554461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e1908fe40 msgr2=0x556e19093b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.954+0000 7f5a27fff640 1 --2- 172.21.15.86:0/4066554461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e1908fe40 0x556e19093b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.954+0000 7f5a27fff640 1 -- 172.21.15.86:0/4066554461 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556e19086330 msgr2=0x556e19096260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.954+0000 7f5a27fff640 1 --2- 172.21.15.86:0/4066554461 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556e19086330 0x556e19096260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.954+0000 7f5a27fff640 1 -- 172.21.15.86:0/4066554461 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e18cfd3f0 con 0x556e19086ce0 2024-04-21T08:49:08.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.954+0000 7f5a27fff640 1 --2- 172.21.15.86:0/4066554461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e19086ce0 0x556e190935f0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c011520 tx=0x7f5a1c01a690 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a85acbf520bb3c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:08.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.954+0000 7f5a07fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:08.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.954+0000 7f5a07fff640 1 -- 172.21.15.86:0/4066554461 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a1c015070 con 0x556e19086ce0 2024-04-21T08:49:08.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.954+0000 7f5a07fff640 1 -- 172.21.15.86:0/4066554461 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9584+0+0 (secure 0 0 0) 0x7f5a1c01acd0 con 0x556e19086ce0 2024-04-21T08:49:08.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.954+0000 7f5a31db9740 1 -- 172.21.15.86:0/4066554461 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a1c01a910 con 0x556e19086ce0 2024-04-21T08:49:08.960 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.954+0000 7f5a07fff640 1 -- 172.21.15.86:0/4066554461 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5a1c005040 con 0x556e19086ce0 2024-04-21T08:49:08.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.954+0000 7f5a31db9740 10 client.19782 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:08.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.954+0000 7f5a31db9740 1 -- 172.21.15.86:0/4066554461 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556e18dc77d0 con 0x556e19086ce0 2024-04-21T08:49:08.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.954+0000 7f5a07fff640 1 -- 172.21.15.86:0/4066554461 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a1c01a910 con 0x556e19086ce0 2024-04-21T08:49:08.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.954+0000 7f5a07fff640 1 -- 172.21.15.86:0/4066554461 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5a1c011c60 con 0x556e19086ce0 2024-04-21T08:49:08.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.954+0000 7f5a31db9740 10 client.19782 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:08.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.954+0000 7f5a31db9740 10 client.19782 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:49:08.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.955+0000 7f5a31db9740 20 client.19782 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:08.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.955+0000 7f5a31db9740 1 --2- 172.21.15.86:0/4066554461 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x556e190a4020 0x556e190a6410 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.955+0000 7f5a31db9740 4 client.19782 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:08.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.955+0000 7f5a31db9740 1 -- 172.21.15.86:0/4066554461 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556e18f8a930 con 0x556e190a4020 2024-04-21T08:49:08.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.955+0000 7f5a2ce87640 1 --2- 172.21.15.86:0/4066554461 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x556e190a4020 0x556e190a6410 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.955+0000 7f5a07fff640 10 client.19782 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:49:08.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.955+0000 7f5a2ce87640 1 --2- 172.21.15.86:0/4066554461 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x556e190a4020 0x556e190a6410 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=2f5237f6c0680497 server_cookie=817413795186dc4e in_seq=0 out_seq=0 2024-04-21T08:49:08.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.956+0000 7f5a07fff640 1 -- 172.21.15.86:0/4066554461 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556e18f8a930 con 0x556e190a4020 2024-04-21T08:49:08.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.956+0000 7f5a07fff640 10 client.19782 handle_command_reply: tid=0 2024-04-21T08:49:08.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.956+0000 7f5a31db9740 1 client.19782 shutdown 2024-04-21T08:49:08.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.956+0000 7f5a31db9740 1 -- 172.21.15.86:0/4066554461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e19086ce0 msgr2=0x556e190935f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.956+0000 7f5a31db9740 1 --2- 172.21.15.86:0/4066554461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e19086ce0 0x556e190935f0 secure :-1 s=READY pgs=1854 cs=0 l=1 rev1=1 crypto rx=0x7f5a1c011520 tx=0x7f5a1c01a690 comp rx=0 tx=0).stop 2024-04-21T08:49:08.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.956+0000 7f5a31db9740 1 -- 172.21.15.86:0/4066554461 shutdown_connections 2024-04-21T08:49:08.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.956+0000 7f5a31db9740 1 --2- 172.21.15.86:0/4066554461 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x556e190a4020 0x556e190a6410 crc :-1 s=READY pgs=71 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.956+0000 7f5a31db9740 1 --2- 172.21.15.86:0/4066554461 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e1908fe40 0x556e19093b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.956+0000 7f5a07fff640 0 client.19782 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:49:08.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.956+0000 7f5a31db9740 1 --2- 172.21.15.86:0/4066554461 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e19086ce0 0x556e190935f0 unknown :-1 s=CLOSED pgs=1854 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.956+0000 7f5a31db9740 1 --2- 172.21.15.86:0/4066554461 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556e19086330 0x556e19096260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.956+0000 7f5a31db9740 1 -- 172.21.15.86:0/4066554461 >> 172.21.15.86:0/4066554461 conn(0x556e19080ba0 msgr2=0x556e190828f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:08.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.956+0000 7f5a31db9740 1 -- 172.21.15.86:0/4066554461 shutdown_connections 2024-04-21T08:49:08.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.956+0000 7f5a31db9740 1 -- 172.21.15.86:0/4066554461 wait complete. 2024-04-21T08:49:08.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.956+0000 7f5a31db9740 20 client.19782 trim_cache size 0 max 16384 2024-04-21T08:49:08.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.970+0000 7f5a31db9740 1 Processor -- start 2024-04-21T08:49:08.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.970+0000 7f5a31db9740 1 -- start start 2024-04-21T08:49:08.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.970+0000 7f5a31db9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e19086330 0x556e19091250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.970+0000 7f5a31db9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e19086ce0 0x556e1909fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.970+0000 7f5a31db9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556e1908fe40 0x556e1907fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.970+0000 7f5a31db9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5a281ad190 con 0x556e19086330 2024-04-21T08:49:08.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.970+0000 7f5a31db9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5a281aecd0 con 0x556e1908fe40 2024-04-21T08:49:08.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.970+0000 7f5a31db9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5a281ad860 con 0x556e19086ce0 2024-04-21T08:49:08.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.970+0000 7f5a2ce87640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e19086330 0x556e19091250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.970+0000 7f5a2ce87640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e19086330 0x556e19091250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32858/0 (socket says 172.21.15.86:32858) 2024-04-21T08:49:08.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.970+0000 7f5a2ce87640 1 -- 172.21.15.86:0/844980818 learned_addr learned my addr 172.21.15.86:0/844980818 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:08.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.970+0000 7f5a27fff640 1 --2- 172.21.15.86:0/844980818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e19086ce0 0x556e1909fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.970+0000 7f5a2ce87640 1 -- 172.21.15.86:0/844980818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e19086ce0 msgr2=0x556e1909fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.970+0000 7f5a2ce87640 1 --2- 172.21.15.86:0/844980818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e19086ce0 0x556e1909fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.970+0000 7f5a2ce87640 1 -- 172.21.15.86:0/844980818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556e1908fe40 msgr2=0x556e1907fa80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.970+0000 7f5a2ce87640 1 --2- 172.21.15.86:0/844980818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556e1908fe40 0x556e1907fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.970+0000 7f5a2ce87640 1 -- 172.21.15.86:0/844980818 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556e1908da20 con 0x556e19086330 2024-04-21T08:49:08.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.970+0000 7f5a27fff640 1 --2- 172.21.15.86:0/844980818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e19086ce0 0x556e1909fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:08.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.970+0000 7f5a2ce87640 1 --2- 172.21.15.86:0/844980818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e19086330 0x556e19091250 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f5a180bea20 tx=0x7f5a180a9040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ed10a56e0b7ae11a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:08.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.970+0000 7f5a0e7fc640 1 -- 172.21.15.86:0/844980818 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a180bf190 con 0x556e19086330 2024-04-21T08:49:08.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.970+0000 7f5a0e7fc640 1 -- 172.21.15.86:0/844980818 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a180bea90 con 0x556e19086330 2024-04-21T08:49:08.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.970+0000 7f5a0e7fc640 1 -- 172.21.15.86:0/844980818 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a180054d0 con 0x556e19086330 2024-04-21T08:49:08.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.970+0000 7f5a31db9740 1 -- 172.21.15.86:0/844980818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e19086330 msgr2=0x556e19091250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.970+0000 7f5a31db9740 1 --2- 172.21.15.86:0/844980818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e19086330 0x556e19091250 secure :-1 s=READY pgs=1855 cs=0 l=1 rev1=1 crypto rx=0x7f5a180bea20 tx=0x7f5a180a9040 comp rx=0 tx=0).stop 2024-04-21T08:49:08.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.970+0000 7f5a31db9740 1 -- 172.21.15.86:0/844980818 shutdown_connections 2024-04-21T08:49:08.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.970+0000 7f5a31db9740 1 --2- 172.21.15.86:0/844980818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556e1908fe40 0x556e1907fa80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.970+0000 7f5a31db9740 1 --2- 172.21.15.86:0/844980818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e19086ce0 0x556e1909fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.970+0000 7f5a31db9740 1 --2- 172.21.15.86:0/844980818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e19086330 0x556e19091250 unknown :-1 s=CLOSED pgs=1855 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.970+0000 7f5a31db9740 1 -- 172.21.15.86:0/844980818 >> 172.21.15.86:0/844980818 conn(0x556e19080ba0 msgr2=0x556e1907f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:08.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.970+0000 7f5a31db9740 1 -- 172.21.15.86:0/844980818 shutdown_connections 2024-04-21T08:49:08.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.970+0000 7f5a31db9740 1 -- 172.21.15.86:0/844980818 wait complete. 2024-04-21T08:49:08.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.970+0000 7f5a31db9740 1 -- start start 2024-04-21T08:49:08.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.970+0000 7f5a31db9740 1 Processor -- start 2024-04-21T08:49:08.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.971+0000 7f5a31db9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e19086330 0x556e19095910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.971+0000 7f5a31db9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556e19086ce0 0x556e19095e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.971+0000 7f5a31db9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1908fe40 0x556e191094b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.971+0000 7f5a31db9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5a281ae600 con 0x556e1908fe40 2024-04-21T08:49:08.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.971+0000 7f5a31db9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5a281adf30 con 0x556e19086ce0 2024-04-21T08:49:08.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.971+0000 7f5a31db9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5a281ab170 con 0x556e19086330 2024-04-21T08:49:08.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.971+0000 7f5a31db9740 10 client.0 _ll_register_callbacks cb 0x7f5a1c00b7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:08.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.971+0000 7f5a2d688640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1908fe40 0x556e191094b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.971+0000 7f5a2d688640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1908fe40 0x556e191094b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32860/0 (socket says 172.21.15.86:32860) 2024-04-21T08:49:08.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.971+0000 7f5a2d688640 1 -- 172.21.15.86:0/1660116000 learned_addr learned my addr 172.21.15.86:0/1660116000 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:08.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.971+0000 7f5a2ce87640 1 --2- 172.21.15.86:0/1660116000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e19086330 0x556e19095910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.971+0000 7f5a27fff640 1 --2- 172.21.15.86:0/1660116000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556e19086ce0 0x556e19095e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.971+0000 7f5a2d688640 1 -- 172.21.15.86:0/1660116000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e19086330 msgr2=0x556e19095910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.971+0000 7f5a2d688640 1 --2- 172.21.15.86:0/1660116000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e19086330 0x556e19095910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.971+0000 7f5a2d688640 1 -- 172.21.15.86:0/1660116000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556e19086ce0 msgr2=0x556e19095e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:08.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.971+0000 7f5a2d688640 1 --2- 172.21.15.86:0/1660116000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556e19086ce0 0x556e19095e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.971+0000 7f5a2d688640 1 -- 172.21.15.86:0/1660116000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556e191099f0 con 0x556e1908fe40 2024-04-21T08:49:08.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.971+0000 7f5a2d688640 1 --2- 172.21.15.86:0/1660116000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1908fe40 0x556e191094b0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f5a20013500 tx=0x7f5a20009cb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1cf634aa5c2168b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:08.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.971+0000 7f5a07fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:08.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.971+0000 7f5a07fff640 1 -- 172.21.15.86:0/1660116000 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5a2002c060 con 0x556e1908fe40 2024-04-21T08:49:08.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.972+0000 7f5a31db9740 1 -- 172.21.15.86:0/1660116000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f5a180bea90 con 0x556e1908fe40 2024-04-21T08:49:08.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.972+0000 7f5a07fff640 1 -- 172.21.15.86:0/1660116000 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9584+0+0 (secure 0 0 0) 0x7f5a20012810 con 0x556e1908fe40 2024-04-21T08:49:08.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.972+0000 7f5a07fff640 1 -- 172.21.15.86:0/1660116000 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f5a20033050 con 0x556e1908fe40 2024-04-21T08:49:08.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.972+0000 7f5a31db9740 10 client.19788 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:08.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.972+0000 7f5a31db9740 1 -- 172.21.15.86:0/1660116000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556e1910acf0 con 0x556e1908fe40 2024-04-21T08:49:08.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.972+0000 7f5a07fff640 1 -- 172.21.15.86:0/1660116000 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5a180bea90 con 0x556e1908fe40 2024-04-21T08:49:08.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.972+0000 7f5a07fff640 1 -- 172.21.15.86:0/1660116000 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f5a2000f040 con 0x556e1908fe40 2024-04-21T08:49:08.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.972+0000 7f5a31db9740 10 client.19788 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:08.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.972+0000 7f5a31db9740 10 client.19788 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:49:08.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.972+0000 7f5a31db9740 20 client.19788 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:08.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.972+0000 7f5a31db9740 1 --2- 172.21.15.86:0/1660116000 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x556e1910b470 0x556e1910d890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:08.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.972+0000 7f5a31db9740 4 client.19788 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:49:08.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.972+0000 7f5a31db9740 1 -- 172.21.15.86:0/1660116000 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556e18ff8c10 con 0x556e1910b470 2024-04-21T08:49:08.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.972+0000 7f5a2ce87640 1 --2- 172.21.15.86:0/1660116000 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x556e1910b470 0x556e1910d890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:08.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.973+0000 7f5a2ce87640 1 --2- 172.21.15.86:0/1660116000 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x556e1910b470 0x556e1910d890 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=9882f3639868ad3e server_cookie=784c7be6698487c5 in_seq=0 out_seq=0 2024-04-21T08:49:08.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.973+0000 7f5a07fff640 10 client.19788 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:49:08.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.973+0000 7f5a07fff640 1 -- 172.21.15.86:0/1660116000 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556e18ff8c10 con 0x556e1910b470 2024-04-21T08:49:08.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.973+0000 7f5a07fff640 10 client.19788 handle_command_reply: tid=0 2024-04-21T08:49:08.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.973+0000 7f5a31db9740 1 client.19788 shutdown 2024-04-21T08:49:08.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.973+0000 7f5a31db9740 1 -- 172.21.15.86:0/1660116000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1908fe40 msgr2=0x556e191094b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.973+0000 7f5a31db9740 1 --2- 172.21.15.86:0/1660116000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1908fe40 0x556e191094b0 secure :-1 s=READY pgs=1856 cs=0 l=1 rev1=1 crypto rx=0x7f5a20013500 tx=0x7f5a20009cb0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.973+0000 7f5a31db9740 1 -- 172.21.15.86:0/1660116000 shutdown_connections 2024-04-21T08:49:08.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.973+0000 7f5a31db9740 1 --2- 172.21.15.86:0/1660116000 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x556e1910b470 0x556e1910d890 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.973+0000 7f5a31db9740 1 --2- 172.21.15.86:0/1660116000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556e1908fe40 0x556e191094b0 unknown :-1 s=CLOSED pgs=1856 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.973+0000 7f5a07fff640 0 client.19788 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:49:08.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.973+0000 7f5a31db9740 1 --2- 172.21.15.86:0/1660116000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556e19086ce0 0x556e19095e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.973+0000 7f5a31db9740 1 --2- 172.21.15.86:0/1660116000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556e19086330 0x556e19095910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.973+0000 7f5a31db9740 1 -- 172.21.15.86:0/1660116000 >> 172.21.15.86:0/1660116000 conn(0x556e19080ba0 msgr2=0x556e1907fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:08.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.973+0000 7f5a31db9740 1 -- 172.21.15.86:0/1660116000 shutdown_connections 2024-04-21T08:49:08.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.973+0000 7f5a31db9740 1 -- 172.21.15.86:0/1660116000 wait complete. 2024-04-21T08:49:08.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.973+0000 7f5a31db9740 20 client.19788 trim_cache size 0 max 16384 2024-04-21T08:49:08.980 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:49:08.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.974+0000 7f5a2f0eb640 1 -- 172.21.15.86:0/4176931635 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5a100a3d20 msgr2=0x7f5a100a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.974+0000 7f5a2f0eb640 1 --2- 172.21.15.86:0/4176931635 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5a100a3d20 0x7f5a100a61e0 secure :-1 s=READY pgs=307 cs=0 l=1 rev1=1 crypto rx=0x7f5a20005fd0 tx=0x7f5a20009480 comp rx=0 tx=0).stop 2024-04-21T08:49:08.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.974+0000 7f5a2f0eb640 1 -- 172.21.15.86:0/4176931635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a2806a7c0 msgr2=0x7f5a28198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:08.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.974+0000 7f5a2f0eb640 1 --2- 172.21.15.86:0/4176931635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a2806a7c0 0x7f5a28198740 secure :-1 s=READY pgs=1852 cs=0 l=1 rev1=1 crypto rx=0x7f5a18005940 tx=0x7f5a1802c7e0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.974+0000 7f5a2f0eb640 1 -- 172.21.15.86:0/4176931635 shutdown_connections 2024-04-21T08:49:08.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.974+0000 7f5a2f0eb640 1 --2- 172.21.15.86:0/4176931635 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5a100a3d20 0x7f5a100a61e0 unknown :-1 s=CLOSED pgs=307 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.974+0000 7f5a2f0eb640 1 --2- 172.21.15.86:0/4176931635 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5a2819d000 0x7f5a2819d4b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.974+0000 7f5a2f0eb640 1 --2- 172.21.15.86:0/4176931635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5a28107280 0x7f5a28198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.974+0000 7f5a2f0eb640 1 --2- 172.21.15.86:0/4176931635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5a2806a7c0 0x7f5a28198740 unknown :-1 s=CLOSED pgs=1852 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:08.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.974+0000 7f5a2f0eb640 1 -- 172.21.15.86:0/4176931635 >> 172.21.15.86:0/4176931635 conn(0x7f5a28076c50 msgr2=0x7f5a280ff500 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:08.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.974+0000 7f5a2f0eb640 1 -- 172.21.15.86:0/4176931635 shutdown_connections 2024-04-21T08:49:08.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:08.974+0000 7f5a2f0eb640 1 -- 172.21.15.86:0/4176931635 wait complete. 2024-04-21T08:49:08.985 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:49:09.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.086+0000 7f64f2077640 1 -- 172.21.15.86:0/667768386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64ec106160 msgr2=0x7f64ec10dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:09.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.086+0000 7f64f2077640 1 --2- 172.21.15.86:0/667768386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64ec106160 0x7f64ec10dd30 secure :-1 s=READY pgs=1857 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009ca0 tx=0x7f64e402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.086+0000 7f64f2077640 1 -- 172.21.15.86:0/667768386 shutdown_connections 2024-04-21T08:49:09.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.086+0000 7f64f2077640 1 --2- 172.21.15.86:0/667768386 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64ec106160 0x7f64ec10dd30 unknown :-1 s=CLOSED pgs=1857 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.086+0000 7f64f2077640 1 --2- 172.21.15.86:0/667768386 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64ec101a40 0x7f64ec105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.086+0000 7f64f2077640 1 --2- 172.21.15.86:0/667768386 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64ec101090 0x7f64ec101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.086+0000 7f64f2077640 1 -- 172.21.15.86:0/667768386 >> 172.21.15.86:0/667768386 conn(0x7f64ec077f50 msgr2=0x7f64ec078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:09.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.086+0000 7f64f2077640 1 -- 172.21.15.86:0/667768386 shutdown_connections 2024-04-21T08:49:09.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.086+0000 7f64f2077640 1 -- 172.21.15.86:0/667768386 wait complete. 2024-04-21T08:49:09.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.086+0000 7f64f2077640 1 Processor -- start 2024-04-21T08:49:09.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.087+0000 7f64f2077640 1 -- start start 2024-04-21T08:49:09.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.087+0000 7f64f2077640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64ec101090 0x7f64ec1975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:09.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.087+0000 7f64f2077640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64ec106160 0x7f64ec197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:09.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.087+0000 7f64f2077640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64ec19be90 0x7f64ec19c340 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:09.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.087+0000 7f64f2077640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec1110b0 con 0x7f64ec101090 2024-04-21T08:49:09.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.087+0000 7f64f2077640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec110f30 con 0x7f64ec106160 2024-04-21T08:49:09.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.087+0000 7f64f2077640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec111230 con 0x7f64ec19be90 2024-04-21T08:49:09.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.087+0000 7f64eb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64ec101090 0x7f64ec1975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:09.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.087+0000 7f64eb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64ec101090 0x7f64ec1975d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32880/0 (socket says 172.21.15.86:32880) 2024-04-21T08:49:09.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.087+0000 7f64eb7fe640 1 -- 172.21.15.86:0/4078359425 learned_addr learned my addr 172.21.15.86:0/4078359425 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:09.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.087+0000 7f64ebfff640 1 --2- 172.21.15.86:0/4078359425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64ec19be90 0x7f64ec19c340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:09.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.087+0000 7f64eaffd640 1 --2- 172.21.15.86:0/4078359425 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64ec106160 0x7f64ec197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:09.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.087+0000 7f64eb7fe640 1 -- 172.21.15.86:0/4078359425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64ec19be90 msgr2=0x7f64ec19c340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:09.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.087+0000 7f64eb7fe640 1 --2- 172.21.15.86:0/4078359425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64ec19be90 0x7f64ec19c340 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.087+0000 7f64eb7fe640 1 -- 172.21.15.86:0/4078359425 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64ec106160 msgr2=0x7f64ec197b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:09.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.087+0000 7f64eb7fe640 1 --2- 172.21.15.86:0/4078359425 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64ec106160 0x7f64ec197b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.087+0000 7f64eb7fe640 1 -- 172.21.15.86:0/4078359425 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f64ec19c980 con 0x7f64ec101090 2024-04-21T08:49:09.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.087+0000 7f64eaffd640 1 --2- 172.21.15.86:0/4078359425 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64ec106160 0x7f64ec197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:09.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.087+0000 7f64eb7fe640 1 --2- 172.21.15.86:0/4078359425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64ec101090 0x7f64ec1975d0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00c890 tx=0x7f64dc00cd50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:09.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.087+0000 7f64e8ff9640 1 -- 172.21.15.86:0/4078359425 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc004220 con 0x7f64ec101090 2024-04-21T08:49:09.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.088+0000 7f64f2077640 1 -- 172.21.15.86:0/4078359425 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f64ec19cc70 con 0x7f64ec101090 2024-04-21T08:49:09.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.088+0000 7f64e8ff9640 1 -- 172.21.15.86:0/4078359425 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64dc0043c0 con 0x7f64ec101090 2024-04-21T08:49:09.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.088+0000 7f64e8ff9640 1 -- 172.21.15.86:0/4078359425 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc004a00 con 0x7f64ec101090 2024-04-21T08:49:09.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.088+0000 7f64f2077640 1 -- 172.21.15.86:0/4078359425 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64ec1a44b0 con 0x7f64ec101090 2024-04-21T08:49:09.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.090+0000 7f64e8ff9640 1 -- 172.21.15.86:0/4078359425 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f64dc0079b0 con 0x7f64ec101090 2024-04-21T08:49:09.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.090+0000 7f64f4c60740 1 Processor -- start 2024-04-21T08:49:09.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.090+0000 7f64f4c60740 1 -- start start 2024-04-21T08:49:09.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.090+0000 7f64f4c60740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cc9646470 0x555cc9646850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:09.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.090+0000 7f64f4c60740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc9646e20 0x555cc9651640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:09.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.090+0000 7f64f4c60740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555cc9651c10 0x555cc9654020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:09.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.090+0000 7f64f4c60740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555cc95ab1f0 con 0x555cc9646e20 2024-04-21T08:49:09.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.090+0000 7f64f4c60740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x555cc95484b0 con 0x555cc9651c10 2024-04-21T08:49:09.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.090+0000 7f64f4c60740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555cc95a71b0 con 0x555cc9646470 2024-04-21T08:49:09.098 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.090+0000 7f64eb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc9646e20 0x555cc9651640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:09.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.091+0000 7f64eb7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc9646e20 0x555cc9651640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32884/0 (socket says 172.21.15.86:32884) 2024-04-21T08:49:09.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.091+0000 7f64eb7fe640 1 -- 172.21.15.86:0/199970263 learned_addr learned my addr 172.21.15.86:0/199970263 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:09.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.091+0000 7f64e8ff9640 1 --2- 172.21.15.86:0/4078359425 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f64d40a3d20 0x7f64d40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:09.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.091+0000 7f64ebfff640 1 --2- 172.21.15.86:0/199970263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cc9646470 0x555cc9646850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:09.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.091+0000 7f64eaffd640 1 --2- 172.21.15.86:0/199970263 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555cc9651c10 0x555cc9654020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:09.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.091+0000 7f64ebfff640 1 --2- 172.21.15.86:0/4078359425 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f64d40a3d20 0x7f64d40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:09.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.091+0000 7f64e8ff9640 1 -- 172.21.15.86:0/4078359425 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9584+0+0 (secure 0 0 0) 0x7f64dc0d0970 con 0x7f64ec101090 2024-04-21T08:49:09.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.091+0000 7f64eb7fe640 1 -- 172.21.15.86:0/199970263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cc9646470 msgr2=0x555cc9646850 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:09.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.091+0000 7f64eb7fe640 1 --2- 172.21.15.86:0/199970263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cc9646470 0x555cc9646850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.091+0000 7f64eb7fe640 1 -- 172.21.15.86:0/199970263 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555cc9651c10 msgr2=0x555cc9654020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:09.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.091+0000 7f64eb7fe640 1 --2- 172.21.15.86:0/199970263 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555cc9651c10 0x555cc9654020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.091+0000 7f64eb7fe640 1 -- 172.21.15.86:0/199970263 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cc953a1b0 con 0x555cc9646e20 2024-04-21T08:49:09.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.091+0000 7f64eaffd640 1 --2- 172.21.15.86:0/199970263 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555cc9651c10 0x555cc9654020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:09.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.091+0000 7f64ebfff640 1 --2- 172.21.15.86:0/199970263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cc9646470 0x555cc9646850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:49:09.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.091+0000 7f64eb7fe640 1 --2- 172.21.15.86:0/199970263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc9646e20 0x555cc9651640 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f64dc09c1f0 tx=0x7f64dc0d3f90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3c9e4a0e85d9136 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:09.099 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.091+0000 7f64d1ffb640 1 -- 172.21.15.86:0/199970263 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc01d070 con 0x555cc9646e20 2024-04-21T08:49:09.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.091+0000 7f64ebfff640 1 --2- 172.21.15.86:0/4078359425 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f64d40a3d20 0x7f64d40a61e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f64e4004af0 tx=0x7f64e402c0f0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:09.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.091+0000 7f64d1ffb640 1 -- 172.21.15.86:0/199970263 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64dc09cbf0 con 0x555cc9646e20 2024-04-21T08:49:09.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.091+0000 7f64d1ffb640 1 -- 172.21.15.86:0/199970263 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64dc098b60 con 0x555cc9646e20 2024-04-21T08:49:09.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.092+0000 7f64f4c60740 1 -- 172.21.15.86:0/199970263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc9646e20 msgr2=0x555cc9651640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:09.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.092+0000 7f64f4c60740 1 --2- 172.21.15.86:0/199970263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc9646e20 0x555cc9651640 secure :-1 s=READY pgs=1859 cs=0 l=1 rev1=1 crypto rx=0x7f64dc09c1f0 tx=0x7f64dc0d3f90 comp rx=0 tx=0).stop 2024-04-21T08:49:09.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.092+0000 7f64f4c60740 1 -- 172.21.15.86:0/199970263 shutdown_connections 2024-04-21T08:49:09.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.092+0000 7f64f4c60740 1 --2- 172.21.15.86:0/199970263 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555cc9651c10 0x555cc9654020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.092+0000 7f64f4c60740 1 --2- 172.21.15.86:0/199970263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc9646e20 0x555cc9651640 unknown :-1 s=CLOSED pgs=1859 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.092+0000 7f64f4c60740 1 --2- 172.21.15.86:0/199970263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cc9646470 0x555cc9646850 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.092+0000 7f64f4c60740 1 -- 172.21.15.86:0/199970263 >> 172.21.15.86:0/199970263 conn(0x555cc9640ba0 msgr2=0x555cc9640f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:09.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.092+0000 7f64f4c60740 1 -- 172.21.15.86:0/199970263 shutdown_connections 2024-04-21T08:49:09.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.092+0000 7f64f4c60740 1 -- 172.21.15.86:0/199970263 wait complete. 2024-04-21T08:49:09.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.092+0000 7f64f4c60740 1 -- start start 2024-04-21T08:49:09.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.093+0000 7f64f4c60740 1 Processor -- start 2024-04-21T08:49:09.100 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.093+0000 7f64f4c60740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cc9646470 0x555cc9651560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:09.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.093+0000 7f64f4c60740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555cc9646e20 0x555cc964e8f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:09.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.093+0000 7f64f4c60740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc9651c10 0x555cc964ef00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:09.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.093+0000 7f64f4c60740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x555cc9654a80 con 0x555cc9651c10 2024-04-21T08:49:09.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.093+0000 7f64f4c60740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x555cc9564500 con 0x555cc9646e20 2024-04-21T08:49:09.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.093+0000 7f64f4c60740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x555cc9654c00 con 0x555cc9646470 2024-04-21T08:49:09.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.093+0000 7f64f4c60740 10 client.0 _ll_register_callbacks cb 0x555cc9532b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:09.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.093+0000 7f64ebfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc9651c10 0x555cc964ef00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:09.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.093+0000 7f64ebfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc9651c10 0x555cc964ef00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32890/0 (socket says 172.21.15.86:32890) 2024-04-21T08:49:09.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.093+0000 7f64ebfff640 1 -- 172.21.15.86:0/2535472413 learned_addr learned my addr 172.21.15.86:0/2535472413 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:09.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.093+0000 7f64eb7fe640 1 --2- 172.21.15.86:0/2535472413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cc9646470 0x555cc9651560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:09.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.093+0000 7f64eaffd640 1 --2- 172.21.15.86:0/2535472413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555cc9646e20 0x555cc964e8f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:09.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.093+0000 7f64ebfff640 1 -- 172.21.15.86:0/2535472413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cc9646470 msgr2=0x555cc9651560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:09.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.093+0000 7f64ebfff640 1 --2- 172.21.15.86:0/2535472413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cc9646470 0x555cc9651560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.093+0000 7f64ebfff640 1 -- 172.21.15.86:0/2535472413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555cc9646e20 msgr2=0x555cc964e8f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:09.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.093+0000 7f64ebfff640 1 --2- 172.21.15.86:0/2535472413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555cc9646e20 0x555cc964e8f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.093+0000 7f64ebfff640 1 -- 172.21.15.86:0/2535472413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cc92bd3f0 con 0x555cc9651c10 2024-04-21T08:49:09.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.093+0000 7f64eaffd640 1 --2- 172.21.15.86:0/2535472413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555cc9646e20 0x555cc964e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:09.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.093+0000 7f64ebfff640 1 --2- 172.21.15.86:0/2535472413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc9651c10 0x555cc964ef00 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f64e4032460 tx=0x7f64e40095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=bff914bbf39f0473 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:09.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.093+0000 7f64caffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:09.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.094+0000 7f64caffd640 1 -- 172.21.15.86:0/2535472413 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4052060 con 0x555cc9651c10 2024-04-21T08:49:09.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.094+0000 7f64caffd640 1 -- 172.21.15.86:0/2535472413 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9584+0+0 (secure 0 0 0) 0x7f64e4004240 con 0x555cc9651c10 2024-04-21T08:49:09.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.094+0000 7f64f4c60740 1 -- 172.21.15.86:0/2535472413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64dc09cbf0 con 0x555cc9651c10 2024-04-21T08:49:09.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.094+0000 7f64caffd640 1 -- 172.21.15.86:0/2535472413 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f64e4059050 con 0x555cc9651c10 2024-04-21T08:49:09.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.094+0000 7f64f4c60740 10 client.19800 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:09.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.094+0000 7f64f4c60740 1 -- 172.21.15.86:0/2535472413 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555cc93877d0 con 0x555cc9651c10 2024-04-21T08:49:09.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.094+0000 7f64caffd640 1 -- 172.21.15.86:0/2535472413 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64dc09cbf0 con 0x555cc9651c10 2024-04-21T08:49:09.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.094+0000 7f64caffd640 1 -- 172.21.15.86:0/2535472413 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f64e403f070 con 0x555cc9651c10 2024-04-21T08:49:09.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.094+0000 7f64f4c60740 10 client.19800 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:09.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.094+0000 7f64f4c60740 10 client.19800 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:49:09.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.094+0000 7f64f4c60740 20 client.19800 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:09.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.094+0000 7f64f4c60740 1 --2- 172.21.15.86:0/2535472413 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x555cc9661da0 0x555cc96641e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:09.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.094+0000 7f64f4c60740 4 client.19800 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:09.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.094+0000 7f64f4c60740 1 -- 172.21.15.86:0/2535472413 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x555cc954a930 con 0x555cc9661da0 2024-04-21T08:49:09.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.094+0000 7f64eb7fe640 1 --2- 172.21.15.86:0/2535472413 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x555cc9661da0 0x555cc96641e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:09.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.095+0000 7f64caffd640 10 client.19800 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:49:09.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.095+0000 7f64eb7fe640 1 --2- 172.21.15.86:0/2535472413 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x555cc9661da0 0x555cc96641e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=832f3668947f85d0 server_cookie=d07e68f574435dd1 in_seq=0 out_seq=0 2024-04-21T08:49:09.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.096+0000 7f64caffd640 1 -- 172.21.15.86:0/2535472413 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x555cc954a930 con 0x555cc9661da0 2024-04-21T08:49:09.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.096+0000 7f64caffd640 10 client.19800 handle_command_reply: tid=0 2024-04-21T08:49:09.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.096+0000 7f64f4c60740 1 client.19800 shutdown 2024-04-21T08:49:09.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.096+0000 7f64f4c60740 1 -- 172.21.15.86:0/2535472413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc9651c10 msgr2=0x555cc964ef00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:09.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.096+0000 7f64f4c60740 1 --2- 172.21.15.86:0/2535472413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc9651c10 0x555cc964ef00 secure :-1 s=READY pgs=1860 cs=0 l=1 rev1=1 crypto rx=0x7f64e4032460 tx=0x7f64e40095c0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.096+0000 7f64f4c60740 1 -- 172.21.15.86:0/2535472413 shutdown_connections 2024-04-21T08:49:09.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.096+0000 7f64f4c60740 1 --2- 172.21.15.86:0/2535472413 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x555cc9661da0 0x555cc96641e0 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.096+0000 7f64f4c60740 1 --2- 172.21.15.86:0/2535472413 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc9651c10 0x555cc964ef00 unknown :-1 s=CLOSED pgs=1860 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.096+0000 7f64caffd640 0 client.19800 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:49:09.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.096+0000 7f64f4c60740 1 --2- 172.21.15.86:0/2535472413 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555cc9646e20 0x555cc964e8f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.096+0000 7f64f4c60740 1 --2- 172.21.15.86:0/2535472413 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cc9646470 0x555cc9651560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.096+0000 7f64f4c60740 1 -- 172.21.15.86:0/2535472413 >> 172.21.15.86:0/2535472413 conn(0x555cc9640ba0 msgr2=0x555cc96428f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:09.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.096+0000 7f64f4c60740 1 -- 172.21.15.86:0/2535472413 shutdown_connections 2024-04-21T08:49:09.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.096+0000 7f64f4c60740 1 -- 172.21.15.86:0/2535472413 wait complete. 2024-04-21T08:49:09.104 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.096+0000 7f64f4c60740 20 client.19800 trim_cache size 0 max 16384 2024-04-21T08:49:09.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.109+0000 7f64f4c60740 1 Processor -- start 2024-04-21T08:49:09.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.109+0000 7f64f4c60740 1 -- start start 2024-04-21T08:49:09.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.109+0000 7f64f4c60740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555cc9646470 0x555cc965d980 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:09.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.109+0000 7f64f4c60740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc9646e20 0x555cc965a270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:09.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.109+0000 7f64f4c60740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cc9651c10 0x555cc9651020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:09.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.109+0000 7f64f4c60740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec1ad370 con 0x555cc9646e20 2024-04-21T08:49:09.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.109+0000 7f64f4c60740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec1aeeb0 con 0x555cc9646470 2024-04-21T08:49:09.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.109+0000 7f64f4c60740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec1ada40 con 0x555cc9651c10 2024-04-21T08:49:09.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.109+0000 7f64eaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc9646e20 0x555cc965a270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:09.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.109+0000 7f64eaffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc9646e20 0x555cc965a270 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32898/0 (socket says 172.21.15.86:32898) 2024-04-21T08:49:09.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.109+0000 7f64eaffd640 1 -- 172.21.15.86:0/318551194 learned_addr learned my addr 172.21.15.86:0/318551194 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:09.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.110+0000 7f64ebfff640 1 --2- 172.21.15.86:0/318551194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cc9651c10 0x555cc9651020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:09.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.110+0000 7f64eb7fe640 1 --2- 172.21.15.86:0/318551194 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555cc9646470 0x555cc965d980 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:09.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.110+0000 7f64eaffd640 1 -- 172.21.15.86:0/318551194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cc9651c10 msgr2=0x555cc9651020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:09.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.110+0000 7f64eaffd640 1 --2- 172.21.15.86:0/318551194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cc9651c10 0x555cc9651020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.110+0000 7f64eaffd640 1 -- 172.21.15.86:0/318551194 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555cc9646470 msgr2=0x555cc965d980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:09.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.110+0000 7f64eaffd640 1 --2- 172.21.15.86:0/318551194 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555cc9646470 0x555cc965d980 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.110+0000 7f64eaffd640 1 -- 172.21.15.86:0/318551194 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x555cc9645560 con 0x555cc9646e20 2024-04-21T08:49:09.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.110+0000 7f64eaffd640 1 --2- 172.21.15.86:0/318551194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc9646e20 0x555cc965a270 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f64ec100990 tx=0x7f64e00127c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64ed345e0e5eecf1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:09.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.110+0000 7f64d17fa640 1 -- 172.21.15.86:0/318551194 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e0004020 con 0x555cc9646e20 2024-04-21T08:49:09.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.110+0000 7f64d17fa640 1 -- 172.21.15.86:0/318551194 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64e00041c0 con 0x555cc9646e20 2024-04-21T08:49:09.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.110+0000 7f64d17fa640 1 -- 172.21.15.86:0/318551194 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e001a6e0 con 0x555cc9646e20 2024-04-21T08:49:09.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.110+0000 7f64f4c60740 1 -- 172.21.15.86:0/318551194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc9646e20 msgr2=0x555cc965a270 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:09.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.110+0000 7f64f4c60740 1 --2- 172.21.15.86:0/318551194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc9646e20 0x555cc965a270 secure :-1 s=READY pgs=1861 cs=0 l=1 rev1=1 crypto rx=0x7f64ec100990 tx=0x7f64e00127c0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.111+0000 7f64f4c60740 1 -- 172.21.15.86:0/318551194 shutdown_connections 2024-04-21T08:49:09.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.111+0000 7f64f4c60740 1 --2- 172.21.15.86:0/318551194 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cc9651c10 0x555cc9651020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.111+0000 7f64f4c60740 1 --2- 172.21.15.86:0/318551194 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc9646e20 0x555cc965a270 unknown :-1 s=CLOSED pgs=1861 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.111+0000 7f64f4c60740 1 --2- 172.21.15.86:0/318551194 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555cc9646470 0x555cc965d980 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.111+0000 7f64f4c60740 1 -- 172.21.15.86:0/318551194 >> 172.21.15.86:0/318551194 conn(0x555cc9640ba0 msgr2=0x555cc963fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:09.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.111+0000 7f64f4c60740 1 -- 172.21.15.86:0/318551194 shutdown_connections 2024-04-21T08:49:09.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.111+0000 7f64f4c60740 1 -- 172.21.15.86:0/318551194 wait complete. 2024-04-21T08:49:09.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.111+0000 7f64f4c60740 1 -- start start 2024-04-21T08:49:09.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.111+0000 7f64f4c60740 1 Processor -- start 2024-04-21T08:49:09.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.111+0000 7f64f4c60740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cc9646470 0x555cc9650960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:09.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.111+0000 7f64f4c60740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555cc9646e20 0x555cc9650ea0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:09.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.111+0000 7f64f4c60740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc9651c10 0x555cc96513e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:09.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.111+0000 7f64f4c60740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec1ae7e0 con 0x555cc9651c10 2024-04-21T08:49:09.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.111+0000 7f64f4c60740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec1ae110 con 0x555cc9646e20 2024-04-21T08:49:09.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.111+0000 7f64f4c60740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f64ec1ab350 con 0x555cc9646470 2024-04-21T08:49:09.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.111+0000 7f64f4c60740 10 client.0 _ll_register_callbacks cb 0x7f64e4032620 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:09.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.111+0000 7f64ebfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc9651c10 0x555cc96513e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:09.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.111+0000 7f64ebfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc9651c10 0x555cc96513e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32906/0 (socket says 172.21.15.86:32906) 2024-04-21T08:49:09.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.111+0000 7f64ebfff640 1 -- 172.21.15.86:0/2815708767 learned_addr learned my addr 172.21.15.86:0/2815708767 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:09.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.112+0000 7f64eaffd640 1 --2- 172.21.15.86:0/2815708767 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555cc9646e20 0x555cc9650ea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:09.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.112+0000 7f64eb7fe640 1 --2- 172.21.15.86:0/2815708767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cc9646470 0x555cc9650960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:09.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.112+0000 7f64ebfff640 1 -- 172.21.15.86:0/2815708767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cc9646470 msgr2=0x555cc9650960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:09.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.112+0000 7f64ebfff640 1 --2- 172.21.15.86:0/2815708767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cc9646470 0x555cc9650960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.112+0000 7f64ebfff640 1 -- 172.21.15.86:0/2815708767 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555cc9646e20 msgr2=0x555cc9650ea0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:09.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.112+0000 7f64ebfff640 1 --2- 172.21.15.86:0/2815708767 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555cc9646e20 0x555cc9650ea0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.112+0000 7f64ebfff640 1 -- 172.21.15.86:0/2815708767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x555cc96c9340 con 0x555cc9651c10 2024-04-21T08:49:09.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.112+0000 7f64eaffd640 1 --2- 172.21.15.86:0/2815708767 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555cc9646e20 0x555cc9650ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:09.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.112+0000 7f64ebfff640 1 --2- 172.21.15.86:0/2815708767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc9651c10 0x555cc96513e0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009bf0 tx=0x7f64e4036fe0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fccc7a4fb5b62092 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:09.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.112+0000 7f64caffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:09.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.112+0000 7f64caffd640 1 -- 172.21.15.86:0/2815708767 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f64e4052040 con 0x555cc9651c10 2024-04-21T08:49:09.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.112+0000 7f64caffd640 1 -- 172.21.15.86:0/2815708767 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9584+0+0 (secure 0 0 0) 0x7f64e4034850 con 0x555cc9651c10 2024-04-21T08:49:09.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.112+0000 7f64f4c60740 1 -- 172.21.15.86:0/2815708767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64e00041c0 con 0x555cc9651c10 2024-04-21T08:49:09.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.112+0000 7f64caffd640 1 -- 172.21.15.86:0/2815708767 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f64e406a050 con 0x555cc9651c10 2024-04-21T08:49:09.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.112+0000 7f64f4c60740 10 client.19806 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:09.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.112+0000 7f64f4c60740 1 -- 172.21.15.86:0/2815708767 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x555cc96ca580 con 0x555cc9651c10 2024-04-21T08:49:09.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.112+0000 7f64caffd640 1 -- 172.21.15.86:0/2815708767 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64e00041c0 con 0x555cc9651c10 2024-04-21T08:49:09.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.112+0000 7f64caffd640 1 -- 172.21.15.86:0/2815708767 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f64e4065030 con 0x555cc9651c10 2024-04-21T08:49:09.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.112+0000 7f64f4c60740 10 client.19806 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:09.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.112+0000 7f64f4c60740 10 client.19806 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:49:09.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.113+0000 7f64f4c60740 20 client.19806 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:09.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.113+0000 7f64f4c60740 1 --2- 172.21.15.86:0/2815708767 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x555cc96cacf0 0x555cc96cd0e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:09.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.113+0000 7f64f4c60740 4 client.19806 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:49:09.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.113+0000 7f64f4c60740 1 -- 172.21.15.86:0/2815708767 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x555cc95b8c10 con 0x555cc96cacf0 2024-04-21T08:49:09.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.113+0000 7f64eb7fe640 1 --2- 172.21.15.86:0/2815708767 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x555cc96cacf0 0x555cc96cd0e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:09.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.113+0000 7f64caffd640 10 client.19806 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:49:09.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.113+0000 7f64eb7fe640 1 --2- 172.21.15.86:0/2815708767 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x555cc96cacf0 0x555cc96cd0e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=35e95901fe007b2d server_cookie=c24ac752eda02d21 in_seq=0 out_seq=0 2024-04-21T08:49:09.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.113+0000 7f64caffd640 1 -- 172.21.15.86:0/2815708767 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x555cc95b8c10 con 0x555cc96cacf0 2024-04-21T08:49:09.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.113+0000 7f64caffd640 10 client.19806 handle_command_reply: tid=0 2024-04-21T08:49:09.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.113+0000 7f64f4c60740 1 client.19806 shutdown 2024-04-21T08:49:09.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.113+0000 7f64f4c60740 1 -- 172.21.15.86:0/2815708767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc9651c10 msgr2=0x555cc96513e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:09.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.113+0000 7f64f4c60740 1 --2- 172.21.15.86:0/2815708767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc9651c10 0x555cc96513e0 secure :-1 s=READY pgs=1862 cs=0 l=1 rev1=1 crypto rx=0x7f64e4009bf0 tx=0x7f64e4036fe0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.113+0000 7f64f4c60740 1 -- 172.21.15.86:0/2815708767 shutdown_connections 2024-04-21T08:49:09.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.113+0000 7f64f4c60740 1 --2- 172.21.15.86:0/2815708767 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x555cc96cacf0 0x555cc96cd0e0 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.113+0000 7f64f4c60740 1 --2- 172.21.15.86:0/2815708767 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x555cc9651c10 0x555cc96513e0 unknown :-1 s=CLOSED pgs=1862 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.113+0000 7f64caffd640 0 client.19806 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:49:09.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.113+0000 7f64f4c60740 1 --2- 172.21.15.86:0/2815708767 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x555cc9646e20 0x555cc9650ea0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.114+0000 7f64f4c60740 1 --2- 172.21.15.86:0/2815708767 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x555cc9646470 0x555cc9650960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.114+0000 7f64f4c60740 1 -- 172.21.15.86:0/2815708767 >> 172.21.15.86:0/2815708767 conn(0x555cc9640ba0 msgr2=0x555cc9641f30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:09.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.114+0000 7f64f4c60740 1 -- 172.21.15.86:0/2815708767 shutdown_connections 2024-04-21T08:49:09.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.114+0000 7f64f4c60740 1 -- 172.21.15.86:0/2815708767 wait complete. 2024-04-21T08:49:09.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.114+0000 7f64f4c60740 20 client.19806 trim_cache size 0 max 16384 2024-04-21T08:49:09.122 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:49:09.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.114+0000 7f64f2077640 1 -- 172.21.15.86:0/4078359425 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f64d40a3d20 msgr2=0x7f64d40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:09.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.114+0000 7f64f2077640 1 --2- 172.21.15.86:0/4078359425 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f64d40a3d20 0x7f64d40a61e0 secure :-1 s=READY pgs=308 cs=0 l=1 rev1=1 crypto rx=0x7f64e4004af0 tx=0x7f64e402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.114+0000 7f64f2077640 1 -- 172.21.15.86:0/4078359425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64ec101090 msgr2=0x7f64ec1975d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:09.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.114+0000 7f64f2077640 1 --2- 172.21.15.86:0/4078359425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64ec101090 0x7f64ec1975d0 secure :-1 s=READY pgs=1858 cs=0 l=1 rev1=1 crypto rx=0x7f64dc00c890 tx=0x7f64dc00cd50 comp rx=0 tx=0).stop 2024-04-21T08:49:09.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.114+0000 7f64f2077640 1 -- 172.21.15.86:0/4078359425 shutdown_connections 2024-04-21T08:49:09.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.114+0000 7f64f2077640 1 --2- 172.21.15.86:0/4078359425 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f64d40a3d20 0x7f64d40a61e0 unknown :-1 s=CLOSED pgs=308 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.114+0000 7f64f2077640 1 --2- 172.21.15.86:0/4078359425 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f64ec19be90 0x7f64ec19c340 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.114+0000 7f64f2077640 1 --2- 172.21.15.86:0/4078359425 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64ec106160 0x7f64ec197b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.114+0000 7f64f2077640 1 --2- 172.21.15.86:0/4078359425 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f64ec101090 0x7f64ec1975d0 unknown :-1 s=CLOSED pgs=1858 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.114+0000 7f64f2077640 1 -- 172.21.15.86:0/4078359425 >> 172.21.15.86:0/4078359425 conn(0x7f64ec077f50 msgr2=0x7f64ec0fefe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:09.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.114+0000 7f64f2077640 1 -- 172.21.15.86:0/4078359425 shutdown_connections 2024-04-21T08:49:09.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.114+0000 7f64f2077640 1 -- 172.21.15.86:0/4078359425 wait complete. 2024-04-21T08:49:09.125 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:49:09.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.227+0000 7f0fc7481640 1 -- 172.21.15.86:0/1447587776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc0106160 msgr2=0x7f0fc010dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:09.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.228+0000 7f0fc7481640 1 --2- 172.21.15.86:0/1447587776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc0106160 0x7f0fc010dd30 secure :-1 s=READY pgs=1863 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8009ca0 tx=0x7f0fb802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.228+0000 7f0fc7481640 1 -- 172.21.15.86:0/1447587776 shutdown_connections 2024-04-21T08:49:09.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.228+0000 7f0fc7481640 1 --2- 172.21.15.86:0/1447587776 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc0106160 0x7f0fc010dd30 unknown :-1 s=CLOSED pgs=1863 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.228+0000 7f0fc7481640 1 --2- 172.21.15.86:0/1447587776 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0fc0101a40 0x7f0fc0105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.228+0000 7f0fc7481640 1 --2- 172.21.15.86:0/1447587776 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0fc0101090 0x7f0fc0101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.228+0000 7f0fc7481640 1 -- 172.21.15.86:0/1447587776 >> 172.21.15.86:0/1447587776 conn(0x7f0fc00771d0 msgr2=0x7f0fc00775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:09.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.228+0000 7f0fc7481640 1 -- 172.21.15.86:0/1447587776 shutdown_connections 2024-04-21T08:49:09.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.228+0000 7f0fc7481640 1 -- 172.21.15.86:0/1447587776 wait complete. 2024-04-21T08:49:09.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.228+0000 7f0fc7481640 1 Processor -- start 2024-04-21T08:49:09.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.228+0000 7f0fc7481640 1 -- start start 2024-04-21T08:49:09.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.228+0000 7f0fc7481640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0fc0101090 0x7f0fc01975d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:09.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.228+0000 7f0fc7481640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc0106160 0x7f0fc0197b10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:09.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.229+0000 7f0fc7481640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0fc019be40 0x7f0fc019c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:09.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.229+0000 7f0fc7481640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc01110b0 con 0x7f0fc0106160 2024-04-21T08:49:09.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.229+0000 7f0fc7481640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc0110f30 con 0x7f0fc019be40 2024-04-21T08:49:09.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.229+0000 7f0fc7481640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc0111230 con 0x7f0fc0101090 2024-04-21T08:49:09.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.229+0000 7f0fc4a1c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc0106160 0x7f0fc0197b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:09.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.229+0000 7f0fc4a1c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc0106160 0x7f0fc0197b10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32928/0 (socket says 172.21.15.86:32928) 2024-04-21T08:49:09.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.229+0000 7f0fc4a1c640 1 -- 172.21.15.86:0/1338431268 learned_addr learned my addr 172.21.15.86:0/1338431268 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:09.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.229+0000 7f0fc521d640 1 --2- 172.21.15.86:0/1338431268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0fc0101090 0x7f0fc01975d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:09.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.229+0000 7f0fc5a1e640 1 --2- 172.21.15.86:0/1338431268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0fc019be40 0x7f0fc019c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:09.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.229+0000 7f0fc4a1c640 1 -- 172.21.15.86:0/1338431268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0fc0101090 msgr2=0x7f0fc01975d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:09.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.229+0000 7f0fc4a1c640 1 --2- 172.21.15.86:0/1338431268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0fc0101090 0x7f0fc01975d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.229+0000 7f0fc4a1c640 1 -- 172.21.15.86:0/1338431268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0fc019be40 msgr2=0x7f0fc019c2f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:09.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.229+0000 7f0fc4a1c640 1 --2- 172.21.15.86:0/1338431268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0fc019be40 0x7f0fc019c2f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.229+0000 7f0fc4a1c640 1 -- 172.21.15.86:0/1338431268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0fc019c930 con 0x7f0fc0106160 2024-04-21T08:49:09.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.229+0000 7f0fc4a1c640 1 --2- 172.21.15.86:0/1338431268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc0106160 0x7f0fc0197b10 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0005fd0 tx=0x7f0fb0009d40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:09.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.229+0000 7f0fb67fc640 1 -- 172.21.15.86:0/1338431268 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb000d020 con 0x7f0fc0106160 2024-04-21T08:49:09.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.229+0000 7f0fb67fc640 1 -- 172.21.15.86:0/1338431268 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fb000d1c0 con 0x7f0fc0106160 2024-04-21T08:49:09.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.229+0000 7f0fc7481640 1 -- 172.21.15.86:0/1338431268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0fc019cc20 con 0x7f0fc0106160 2024-04-21T08:49:09.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.229+0000 7f0fb67fc640 1 -- 172.21.15.86:0/1338431268 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb00035f0 con 0x7f0fc0106160 2024-04-21T08:49:09.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.229+0000 7f0fc7481640 1 -- 172.21.15.86:0/1338431268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0fc006a870 con 0x7f0fc0106160 2024-04-21T08:49:09.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.232+0000 7f0fb67fc640 1 -- 172.21.15.86:0/1338431268 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0fb001f020 con 0x7f0fc0106160 2024-04-21T08:49:09.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.232+0000 7f0fca077740 1 Processor -- start 2024-04-21T08:49:09.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.232+0000 7f0fca077740 1 -- start start 2024-04-21T08:49:09.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.232+0000 7f0fca077740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab3e834d0 0x558ab3e838b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:09.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.232+0000 7f0fca077740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab3e83e80 0x558ab3e8c430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:09.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.232+0000 7f0fca077740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab3e8ca00 0x558ab3e8ee50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:09.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.232+0000 7f0fca077740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558ab3de81f0 con 0x558ab3e834d0 2024-04-21T08:49:09.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.232+0000 7f0fca077740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x558ab3d854b0 con 0x558ab3e83e80 2024-04-21T08:49:09.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.232+0000 7f0fca077740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558ab3de41b0 con 0x558ab3e8ca00 2024-04-21T08:49:09.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.233+0000 7f0fb67fc640 1 --2- 172.21.15.86:0/1338431268 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0fac0a3d20 0x7f0fac0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:09.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.233+0000 7f0fc5a1e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab3e834d0 0x558ab3e838b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.233+0000 7f0fc521d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab3e83e80 0x558ab3e8c430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.233+0000 7f0fc5a1e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab3e834d0 0x558ab3e838b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32934/0 (socket says 172.21.15.86:32934) 2024-04-21T08:49:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.233+0000 7f0fc4a1c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab3e8ca00 0x558ab3e8ee50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.233+0000 7f0fc5a1e640 1 -- 172.21.15.86:0/132933940 learned_addr learned my addr 172.21.15.86:0/132933940 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:09.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.233+0000 7f0fc5a1e640 1 --2- 172.21.15.86:0/1338431268 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0fac0a3d20 0x7f0fac0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:09.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.233+0000 7f0fc521d640 1 -- 172.21.15.86:0/132933940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab3e8ca00 msgr2=0x558ab3e8ee50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:09.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.233+0000 7f0fc521d640 1 --2- 172.21.15.86:0/132933940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab3e8ca00 0x558ab3e8ee50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.233+0000 7f0fc521d640 1 -- 172.21.15.86:0/132933940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab3e834d0 msgr2=0x558ab3e838b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:09.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.233+0000 7f0fc521d640 1 --2- 172.21.15.86:0/132933940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab3e834d0 0x558ab3e838b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.233+0000 7f0fc521d640 1 -- 172.21.15.86:0/132933940 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ab3d771b0 con 0x558ab3e83e80 2024-04-21T08:49:09.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.233+0000 7f0fc5a1e640 1 --2- 172.21.15.86:0/132933940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab3e834d0 0x558ab3e838b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:49:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.233+0000 7f0fb67fc640 1 -- 172.21.15.86:0/1338431268 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9584+0+0 (secure 0 0 0) 0x7f0fb009baa0 con 0x7f0fc0106160 2024-04-21T08:49:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.233+0000 7f0fc521d640 1 --2- 172.21.15.86:0/132933940 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab3e83e80 0x558ab3e8c430 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc00abf0 tx=0x7f0fbc00f790 comp rx=0 tx=0).ready entity=mon.2 client_cookie=8ef346a74f7c8118 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.233+0000 7f0fc5a1e640 1 --2- 172.21.15.86:0/1338431268 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0fac0a3d20 0x7f0fac0a61e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8032460 tx=0x7f0fb802c0f0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.234+0000 7f0fab7fe640 1 -- 172.21.15.86:0/132933940 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc005220 con 0x558ab3e83e80 2024-04-21T08:49:09.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.234+0000 7f0fab7fe640 1 -- 172.21.15.86:0/132933940 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fbc004c50 con 0x558ab3e83e80 2024-04-21T08:49:09.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.234+0000 7f0fab7fe640 1 -- 172.21.15.86:0/132933940 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc0187a0 con 0x558ab3e83e80 2024-04-21T08:49:09.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.235+0000 7f0fca077740 1 -- 172.21.15.86:0/132933940 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab3e83e80 msgr2=0x558ab3e8c430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:09.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.235+0000 7f0fca077740 1 --2- 172.21.15.86:0/132933940 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab3e83e80 0x558ab3e8c430 secure :-1 s=READY pgs=83 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc00abf0 tx=0x7f0fbc00f790 comp rx=0 tx=0).stop 2024-04-21T08:49:09.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.235+0000 7f0fca077740 1 -- 172.21.15.86:0/132933940 shutdown_connections 2024-04-21T08:49:09.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.235+0000 7f0fca077740 1 --2- 172.21.15.86:0/132933940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab3e8ca00 0x558ab3e8ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.235+0000 7f0fca077740 1 --2- 172.21.15.86:0/132933940 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab3e83e80 0x558ab3e8c430 unknown :-1 s=CLOSED pgs=83 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.235+0000 7f0fca077740 1 --2- 172.21.15.86:0/132933940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab3e834d0 0x558ab3e838b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.235+0000 7f0fca077740 1 -- 172.21.15.86:0/132933940 >> 172.21.15.86:0/132933940 conn(0x558ab3e7dba0 msgr2=0x558ab3e7df80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:09.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.235+0000 7f0fca077740 1 -- 172.21.15.86:0/132933940 shutdown_connections 2024-04-21T08:49:09.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.235+0000 7f0fca077740 1 -- 172.21.15.86:0/132933940 wait complete. 2024-04-21T08:49:09.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.235+0000 7f0fca077740 1 -- start start 2024-04-21T08:49:09.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.235+0000 7f0fca077740 1 Processor -- start 2024-04-21T08:49:09.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.236+0000 7f0fca077740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab3e834d0 0x558ab3e8c300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:09.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.236+0000 7f0fca077740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab3e83e80 0x558ab3e89690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:09.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.236+0000 7f0fca077740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab3e8ca00 0x558ab3e89c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:09.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.236+0000 7f0fca077740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558ab3e8f970 con 0x558ab3e834d0 2024-04-21T08:49:09.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.236+0000 7f0fca077740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x558ab3da1500 con 0x558ab3e83e80 2024-04-21T08:49:09.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.236+0000 7f0fca077740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558ab3e8faf0 con 0x558ab3e8ca00 2024-04-21T08:49:09.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.236+0000 7f0fca077740 10 client.0 _ll_register_callbacks cb 0x558ab3d6fb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:09.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.236+0000 7f0fc521d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab3e834d0 0x558ab3e8c300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:09.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.236+0000 7f0fc521d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab3e834d0 0x558ab3e8c300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32950/0 (socket says 172.21.15.86:32950) 2024-04-21T08:49:09.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.236+0000 7f0fc521d640 1 -- 172.21.15.86:0/4264646345 learned_addr learned my addr 172.21.15.86:0/4264646345 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:09.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.236+0000 7f0fc4a1c640 1 --2- 172.21.15.86:0/4264646345 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab3e83e80 0x558ab3e89690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:09.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.236+0000 7f0fc5a1e640 1 --2- 172.21.15.86:0/4264646345 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab3e8ca00 0x558ab3e89c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:09.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.236+0000 7f0fc521d640 1 -- 172.21.15.86:0/4264646345 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab3e8ca00 msgr2=0x558ab3e89c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:09.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.236+0000 7f0fc521d640 1 --2- 172.21.15.86:0/4264646345 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab3e8ca00 0x558ab3e89c10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.236+0000 7f0fc521d640 1 -- 172.21.15.86:0/4264646345 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab3e83e80 msgr2=0x558ab3e89690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:09.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.236+0000 7f0fc521d640 1 --2- 172.21.15.86:0/4264646345 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab3e83e80 0x558ab3e89690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.236+0000 7f0fc521d640 1 -- 172.21.15.86:0/4264646345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ab3afa3f0 con 0x558ab3e834d0 2024-04-21T08:49:09.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.236+0000 7f0fc521d640 1 --2- 172.21.15.86:0/4264646345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab3e834d0 0x558ab3e8c300 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc00abc0 tx=0x7f0fbc005030 comp rx=0 tx=0).ready entity=mon.0 client_cookie=566aba42982b3df6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:09.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.236+0000 7f0fa8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:09.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.237+0000 7f0fa8ff9640 1 -- 172.21.15.86:0/4264646345 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc027070 con 0x558ab3e834d0 2024-04-21T08:49:09.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.237+0000 7f0fa8ff9640 1 -- 172.21.15.86:0/4264646345 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9584+0+0 (secure 0 0 0) 0x7f0fbc005430 con 0x558ab3e834d0 2024-04-21T08:49:09.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.237+0000 7f0fca077740 1 -- 172.21.15.86:0/4264646345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fbc004c50 con 0x558ab3e834d0 2024-04-21T08:49:09.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.237+0000 7f0fa8ff9640 1 -- 172.21.15.86:0/4264646345 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0fbc033050 con 0x558ab3e834d0 2024-04-21T08:49:09.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.237+0000 7f0fca077740 10 client.19818 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:09.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.237+0000 7f0fca077740 1 -- 172.21.15.86:0/4264646345 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558ab3bc47d0 con 0x558ab3e834d0 2024-04-21T08:49:09.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.237+0000 7f0fa8ff9640 1 -- 172.21.15.86:0/4264646345 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fbc004c50 con 0x558ab3e834d0 2024-04-21T08:49:09.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.237+0000 7f0fa8ff9640 1 -- 172.21.15.86:0/4264646345 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0fbc023430 con 0x558ab3e834d0 2024-04-21T08:49:09.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.237+0000 7f0fca077740 10 client.19818 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:09.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.237+0000 7f0fca077740 10 client.19818 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:09.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.237+0000 7f0fca077740 20 client.19818 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:09.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.237+0000 7f0fca077740 1 --2- 172.21.15.86:0/4264646345 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x558ab3e9cb40 0x558ab3e9ef30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:09.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.237+0000 7f0fca077740 4 client.19818 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:09.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.237+0000 7f0fca077740 1 -- 172.21.15.86:0/4264646345 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ab3d87930 con 0x558ab3e9cb40 2024-04-21T08:49:09.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.237+0000 7f0fc4a1c640 1 --2- 172.21.15.86:0/4264646345 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x558ab3e9cb40 0x558ab3e9ef30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:09.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.238+0000 7f0fa8ff9640 10 client.19818 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:09.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.238+0000 7f0fc4a1c640 1 --2- 172.21.15.86:0/4264646345 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x558ab3e9cb40 0x558ab3e9ef30 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=628f66325ec0cdcb server_cookie=b6588bdb852c882e in_seq=0 out_seq=0 2024-04-21T08:49:09.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.239+0000 7f0fa8ff9640 1 -- 172.21.15.86:0/4264646345 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558ab3d87930 con 0x558ab3e9cb40 2024-04-21T08:49:09.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.239+0000 7f0fa8ff9640 10 client.19818 handle_command_reply: tid=0 2024-04-21T08:49:09.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.239+0000 7f0fca077740 1 client.19818 shutdown 2024-04-21T08:49:09.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.239+0000 7f0fca077740 1 -- 172.21.15.86:0/4264646345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab3e834d0 msgr2=0x558ab3e8c300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:09.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.239+0000 7f0fca077740 1 --2- 172.21.15.86:0/4264646345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab3e834d0 0x558ab3e8c300 secure :-1 s=READY pgs=1865 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc00abc0 tx=0x7f0fbc005030 comp rx=0 tx=0).stop 2024-04-21T08:49:09.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.239+0000 7f0fca077740 1 -- 172.21.15.86:0/4264646345 shutdown_connections 2024-04-21T08:49:09.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.239+0000 7f0fca077740 1 --2- 172.21.15.86:0/4264646345 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x558ab3e9cb40 0x558ab3e9ef30 crc :-1 s=READY pgs=301 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.239+0000 7f0fca077740 1 --2- 172.21.15.86:0/4264646345 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab3e8ca00 0x558ab3e89c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.239+0000 7f0fca077740 1 --2- 172.21.15.86:0/4264646345 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab3e83e80 0x558ab3e89690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.239+0000 7f0fa8ff9640 0 client.19818 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:09.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.239+0000 7f0fca077740 1 --2- 172.21.15.86:0/4264646345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab3e834d0 0x558ab3e8c300 unknown :-1 s=CLOSED pgs=1865 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.239+0000 7f0fca077740 1 -- 172.21.15.86:0/4264646345 >> 172.21.15.86:0/4264646345 conn(0x558ab3e7dba0 msgr2=0x558ab3e7f8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:09.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.239+0000 7f0fca077740 1 -- 172.21.15.86:0/4264646345 shutdown_connections 2024-04-21T08:49:09.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.239+0000 7f0fca077740 1 -- 172.21.15.86:0/4264646345 wait complete. 2024-04-21T08:49:09.265 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.239+0000 7f0fca077740 20 client.19818 trim_cache size 0 max 16384 2024-04-21T08:49:09.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.252+0000 7f0fca077740 1 Processor -- start 2024-04-21T08:49:09.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.252+0000 7f0fca077740 1 -- start start 2024-04-21T08:49:09.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.252+0000 7f0fca077740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab3e834d0 0x558ab3e98830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:09.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.252+0000 7f0fca077740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab3e83e80 0x558ab3e7fac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:09.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.252+0000 7f0fca077740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab3e8ca00 0x558ab3e8bdc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:09.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.252+0000 7f0fca077740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc01ad2a0 con 0x558ab3e83e80 2024-04-21T08:49:09.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.252+0000 7f0fca077740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc01aede0 con 0x558ab3e834d0 2024-04-21T08:49:09.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.252+0000 7f0fca077740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc01ad970 con 0x558ab3e8ca00 2024-04-21T08:49:09.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.252+0000 7f0fc4a1c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab3e83e80 0x558ab3e7fac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:09.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.252+0000 7f0fc4a1c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab3e83e80 0x558ab3e7fac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32960/0 (socket says 172.21.15.86:32960) 2024-04-21T08:49:09.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.252+0000 7f0fc4a1c640 1 -- 172.21.15.86:0/2732369245 learned_addr learned my addr 172.21.15.86:0/2732369245 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:09.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.252+0000 7f0fc4a1c640 1 -- 172.21.15.86:0/2732369245 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab3e8ca00 msgr2=0x558ab3e8bdc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:09.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.252+0000 7f0fc521d640 1 --2- 172.21.15.86:0/2732369245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab3e834d0 0x558ab3e98830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:09.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.252+0000 7f0fc5a1e640 1 --2- 172.21.15.86:0/2732369245 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab3e8ca00 0x558ab3e8bdc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:09.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.252+0000 7f0fc4a1c640 1 --2- 172.21.15.86:0/2732369245 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab3e8ca00 0x558ab3e8bdc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.252+0000 7f0fc4a1c640 1 -- 172.21.15.86:0/2732369245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab3e834d0 msgr2=0x558ab3e98830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:09.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.252+0000 7f0fc4a1c640 1 --2- 172.21.15.86:0/2732369245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab3e834d0 0x558ab3e98830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.252+0000 7f0fc4a1c640 1 -- 172.21.15.86:0/2732369245 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ab3e993a0 con 0x558ab3e83e80 2024-04-21T08:49:09.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.252+0000 7f0fc5a1e640 1 --2- 172.21.15.86:0/2732369245 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab3e8ca00 0x558ab3e8bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:09.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.253+0000 7f0fc4a1c640 1 --2- 172.21.15.86:0/2732369245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab3e83e80 0x558ab3e7fac0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f0fb009d700 tx=0x7f0fb00be040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=15e0338e0d30e923 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:09.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.253+0000 7f0faaffd640 1 -- 172.21.15.86:0/2732369245 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb001c030 con 0x558ab3e83e80 2024-04-21T08:49:09.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.253+0000 7f0faaffd640 1 -- 172.21.15.86:0/2732369245 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fb00a6670 con 0x558ab3e83e80 2024-04-21T08:49:09.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.253+0000 7f0faaffd640 1 -- 172.21.15.86:0/2732369245 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fb009dc60 con 0x558ab3e83e80 2024-04-21T08:49:09.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.253+0000 7f0fca077740 1 -- 172.21.15.86:0/2732369245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab3e83e80 msgr2=0x558ab3e7fac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:09.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.253+0000 7f0fca077740 1 --2- 172.21.15.86:0/2732369245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab3e83e80 0x558ab3e7fac0 secure :-1 s=READY pgs=1866 cs=0 l=1 rev1=1 crypto rx=0x7f0fb009d700 tx=0x7f0fb00be040 comp rx=0 tx=0).stop 2024-04-21T08:49:09.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.253+0000 7f0fca077740 1 -- 172.21.15.86:0/2732369245 shutdown_connections 2024-04-21T08:49:09.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.253+0000 7f0fca077740 1 --2- 172.21.15.86:0/2732369245 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab3e8ca00 0x558ab3e8bdc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.253+0000 7f0fca077740 1 --2- 172.21.15.86:0/2732369245 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab3e83e80 0x558ab3e7fac0 unknown :-1 s=CLOSED pgs=1866 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.253+0000 7f0fca077740 1 --2- 172.21.15.86:0/2732369245 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab3e834d0 0x558ab3e98830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.253+0000 7f0fca077740 1 -- 172.21.15.86:0/2732369245 >> 172.21.15.86:0/2732369245 conn(0x558ab3e7dba0 msgr2=0x558ab3e886e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:09.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.253+0000 7f0fca077740 1 -- 172.21.15.86:0/2732369245 shutdown_connections 2024-04-21T08:49:09.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.253+0000 7f0fca077740 1 -- 172.21.15.86:0/2732369245 wait complete. 2024-04-21T08:49:09.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.254+0000 7f0fca077740 1 -- start start 2024-04-21T08:49:09.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.254+0000 7f0fca077740 1 Processor -- start 2024-04-21T08:49:09.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.254+0000 7f0fca077740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab3e834d0 0x558ab3e94db0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:09.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.254+0000 7f0fca077740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab3e83e80 0x558ab3e864d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:09.283 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.254+0000 7f0fca077740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab3e8ca00 0x558ab3e86a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:09.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.254+0000 7f0fca077740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc01ae710 con 0x558ab3e834d0 2024-04-21T08:49:09.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.254+0000 7f0fca077740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc01ae040 con 0x558ab3e83e80 2024-04-21T08:49:09.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.254+0000 7f0fca077740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0fc01ab280 con 0x558ab3e8ca00 2024-04-21T08:49:09.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.254+0000 7f0fca077740 10 client.0 _ll_register_callbacks cb 0x7f0fbc00aa40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:09.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.254+0000 7f0fc521d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab3e834d0 0x558ab3e94db0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:09.284 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.254+0000 7f0fc521d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab3e834d0 0x558ab3e94db0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32964/0 (socket says 172.21.15.86:32964) 2024-04-21T08:49:09.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.254+0000 7f0fc521d640 1 -- 172.21.15.86:0/312566618 learned_addr learned my addr 172.21.15.86:0/312566618 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:09.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.254+0000 7f0fc5a1e640 1 --2- 172.21.15.86:0/312566618 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab3e8ca00 0x558ab3e86a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:09.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.254+0000 7f0fc4a1c640 1 --2- 172.21.15.86:0/312566618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab3e83e80 0x558ab3e864d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:09.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.254+0000 7f0fc521d640 1 -- 172.21.15.86:0/312566618 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab3e8ca00 msgr2=0x558ab3e86a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:09.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.254+0000 7f0fc521d640 1 --2- 172.21.15.86:0/312566618 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab3e8ca00 0x558ab3e86a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.285 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.254+0000 7f0fc521d640 1 -- 172.21.15.86:0/312566618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab3e83e80 msgr2=0x558ab3e864d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:09.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.254+0000 7f0fc521d640 1 --2- 172.21.15.86:0/312566618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab3e83e80 0x558ab3e864d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.254+0000 7f0fc521d640 1 -- 172.21.15.86:0/312566618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ab3e86f50 con 0x558ab3e834d0 2024-04-21T08:49:09.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.255+0000 7f0fc521d640 1 --2- 172.21.15.86:0/312566618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab3e834d0 0x558ab3e94db0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc023880 tx=0x7f0fbc005900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9b620f48982d3108 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:09.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.255+0000 7f0fa8ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:09.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.255+0000 7f0fa8ff9640 1 -- 172.21.15.86:0/312566618 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0fbc03c060 con 0x558ab3e834d0 2024-04-21T08:49:09.286 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.255+0000 7f0fa8ff9640 1 -- 172.21.15.86:0/312566618 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9584+0+0 (secure 0 0 0) 0x7f0fbc00a630 con 0x558ab3e834d0 2024-04-21T08:49:09.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.255+0000 7f0fca077740 1 -- 172.21.15.86:0/312566618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0fb00a6670 con 0x558ab3e834d0 2024-04-21T08:49:09.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.255+0000 7f0fa8ff9640 1 -- 172.21.15.86:0/312566618 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0fbc043050 con 0x558ab3e834d0 2024-04-21T08:49:09.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.255+0000 7f0fca077740 10 client.19824 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:09.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.255+0000 7f0fca077740 1 -- 172.21.15.86:0/312566618 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558ab3e92b20 con 0x558ab3e834d0 2024-04-21T08:49:09.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.255+0000 7f0fa8ff9640 1 -- 172.21.15.86:0/312566618 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0fb00a6670 con 0x558ab3e834d0 2024-04-21T08:49:09.287 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.255+0000 7f0fa8ff9640 1 -- 172.21.15.86:0/312566618 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0fbc004a10 con 0x558ab3e834d0 2024-04-21T08:49:09.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.255+0000 7f0fca077740 10 client.19824 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:09.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.255+0000 7f0fca077740 10 client.19824 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:09.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.255+0000 7f0fca077740 20 client.19824 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:09.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.255+0000 7f0fca077740 1 --2- 172.21.15.86:0/312566618 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x558ab3f0bec0 0x558ab3f0e2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:09.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.255+0000 7f0fca077740 4 client.19824 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:49:09.288 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.255+0000 7f0fca077740 1 -- 172.21.15.86:0/312566618 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558ab3df5c10 con 0x558ab3f0bec0 2024-04-21T08:49:09.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.256+0000 7f0fc4a1c640 1 --2- 172.21.15.86:0/312566618 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x558ab3f0bec0 0x558ab3f0e2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:09.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.256+0000 7f0fc4a1c640 1 --2- 172.21.15.86:0/312566618 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x558ab3f0bec0 0x558ab3f0e2e0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c9147c831ba6659d server_cookie=e9e12998d362d19e in_seq=0 out_seq=0 2024-04-21T08:49:09.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.256+0000 7f0fa8ff9640 10 client.19824 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:09.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.256+0000 7f0fa8ff9640 1 -- 172.21.15.86:0/312566618 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558ab3df5c10 con 0x558ab3f0bec0 2024-04-21T08:49:09.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.256+0000 7f0fa8ff9640 10 client.19824 handle_command_reply: tid=0 2024-04-21T08:49:09.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.256+0000 7f0fca077740 1 client.19824 shutdown 2024-04-21T08:49:09.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.257+0000 7f0fca077740 1 -- 172.21.15.86:0/312566618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab3e834d0 msgr2=0x558ab3e94db0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:09.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.257+0000 7f0fca077740 1 --2- 172.21.15.86:0/312566618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab3e834d0 0x558ab3e94db0 secure :-1 s=READY pgs=1867 cs=0 l=1 rev1=1 crypto rx=0x7f0fbc023880 tx=0x7f0fbc005900 comp rx=0 tx=0).stop 2024-04-21T08:49:09.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.257+0000 7f0fca077740 1 -- 172.21.15.86:0/312566618 shutdown_connections 2024-04-21T08:49:09.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.257+0000 7f0fca077740 1 --2- 172.21.15.86:0/312566618 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x558ab3f0bec0 0x558ab3f0e2e0 crc :-1 s=READY pgs=302 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.290 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.257+0000 7f0fca077740 1 --2- 172.21.15.86:0/312566618 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab3e8ca00 0x558ab3e86a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.257+0000 7f0fa8ff9640 0 client.19824 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:09.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.257+0000 7f0fca077740 1 --2- 172.21.15.86:0/312566618 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab3e83e80 0x558ab3e864d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.257+0000 7f0fca077740 1 --2- 172.21.15.86:0/312566618 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab3e834d0 0x558ab3e94db0 unknown :-1 s=CLOSED pgs=1867 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.257+0000 7f0fca077740 1 -- 172.21.15.86:0/312566618 >> 172.21.15.86:0/312566618 conn(0x558ab3e7dba0 msgr2=0x558ab3e7c370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:09.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.257+0000 7f0fca077740 1 -- 172.21.15.86:0/312566618 shutdown_connections 2024-04-21T08:49:09.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.257+0000 7f0fca077740 1 -- 172.21.15.86:0/312566618 wait complete. 2024-04-21T08:49:09.291 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.257+0000 7f0fca077740 20 client.19824 trim_cache size 0 max 16384 2024-04-21T08:49:09.292 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:49:09.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.257+0000 7f0fc7481640 1 -- 172.21.15.86:0/1338431268 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0fac0a3d20 msgr2=0x7f0fac0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:09.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.257+0000 7f0fc7481640 1 --2- 172.21.15.86:0/1338431268 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0fac0a3d20 0x7f0fac0a61e0 secure :-1 s=READY pgs=309 cs=0 l=1 rev1=1 crypto rx=0x7f0fb8032460 tx=0x7f0fb802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.257+0000 7f0fc7481640 1 -- 172.21.15.86:0/1338431268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc0106160 msgr2=0x7f0fc0197b10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:09.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.257+0000 7f0fc7481640 1 --2- 172.21.15.86:0/1338431268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc0106160 0x7f0fc0197b10 secure :-1 s=READY pgs=1864 cs=0 l=1 rev1=1 crypto rx=0x7f0fb0005fd0 tx=0x7f0fb0009d40 comp rx=0 tx=0).stop 2024-04-21T08:49:09.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.257+0000 7f0fc7481640 1 -- 172.21.15.86:0/1338431268 shutdown_connections 2024-04-21T08:49:09.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.257+0000 7f0fc7481640 1 --2- 172.21.15.86:0/1338431268 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0fac0a3d20 0x7f0fac0a61e0 unknown :-1 s=CLOSED pgs=309 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.257+0000 7f0fc7481640 1 --2- 172.21.15.86:0/1338431268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0fc019be40 0x7f0fc019c2f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.257+0000 7f0fc7481640 1 --2- 172.21.15.86:0/1338431268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0fc0106160 0x7f0fc0197b10 unknown :-1 s=CLOSED pgs=1864 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.257+0000 7f0fc7481640 1 --2- 172.21.15.86:0/1338431268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0fc0101090 0x7f0fc01975d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:09.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.257+0000 7f0fc7481640 1 -- 172.21.15.86:0/1338431268 >> 172.21.15.86:0/1338431268 conn(0x7f0fc00771d0 msgr2=0x7f0fc00fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:09.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.257+0000 7f0fc7481640 1 -- 172.21.15.86:0/1338431268 shutdown_connections 2024-04-21T08:49:09.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:09.257+0000 7f0fc7481640 1 -- 172.21.15.86:0/1338431268 wait complete. 2024-04-21T08:49:09.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:09 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3127821974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:49:09.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:09 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3127821974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:49:09.713 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:09 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3127821974' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:49:10.297 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:49:10.297 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:49:10.297 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:49:10.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.411+0000 7f088f104640 1 -- 172.21.15.86:0/2795528581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0888102190 msgr2=0x7f0888102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:10.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.411+0000 7f088f104640 1 --2- 172.21.15.86:0/2795528581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0888102190 0x7f0888102570 secure :-1 s=READY pgs=1868 cs=0 l=1 rev1=1 crypto rx=0x7f087c009900 tx=0x7f087c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:49:10.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.411+0000 7f088f104640 1 -- 172.21.15.86:0/2795528581 shutdown_connections 2024-04-21T08:49:10.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.411+0000 7f088f104640 1 --2- 172.21.15.86:0/2795528581 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0888107260 0x7f088810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.411+0000 7f088f104640 1 --2- 172.21.15.86:0/2795528581 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0888102b40 0x7f0888106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.411+0000 7f088f104640 1 --2- 172.21.15.86:0/2795528581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0888102190 0x7f0888102570 unknown :-1 s=CLOSED pgs=1868 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.411+0000 7f088f104640 1 -- 172.21.15.86:0/2795528581 >> 172.21.15.86:0/2795528581 conn(0x7f08880793a0 msgr2=0x7f08880797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:10.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.411+0000 7f088f104640 1 -- 172.21.15.86:0/2795528581 shutdown_connections 2024-04-21T08:49:10.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.411+0000 7f088f104640 1 -- 172.21.15.86:0/2795528581 wait complete. 2024-04-21T08:49:10.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.412+0000 7f088f104640 1 Processor -- start 2024-04-21T08:49:10.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.412+0000 7f088f104640 1 -- start start 2024-04-21T08:49:10.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.412+0000 7f088f104640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0888102190 0x7f0888198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.412+0000 7f088f104640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0888107260 0x7f0888198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.412+0000 7f088f104640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f088819d020 0x7f088819d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.412+0000 7f088f104640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f08881121a0 con 0x7f0888107260 2024-04-21T08:49:10.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.412+0000 7f088f104640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0888112020 con 0x7f0888102190 2024-04-21T08:49:10.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.412+0000 7f088f104640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0888112320 con 0x7f088819d020 2024-04-21T08:49:10.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.412+0000 7f0887fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0888107260 0x7f0888198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.412+0000 7f0887fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0888107260 0x7f0888198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32828/0 (socket says 172.21.15.86:32828) 2024-04-21T08:49:10.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.412+0000 7f0887fff640 1 -- 172.21.15.86:0/1327782937 learned_addr learned my addr 172.21.15.86:0/1327782937 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:10.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.412+0000 7f088cea0640 1 --2- 172.21.15.86:0/1327782937 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0888102190 0x7f0888198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.412+0000 7f088d6a1640 1 --2- 172.21.15.86:0/1327782937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f088819d020 0x7f088819d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.412+0000 7f0887fff640 1 -- 172.21.15.86:0/1327782937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f088819d020 msgr2=0x7f088819d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:10.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.412+0000 7f0887fff640 1 --2- 172.21.15.86:0/1327782937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f088819d020 0x7f088819d4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.412+0000 7f0887fff640 1 -- 172.21.15.86:0/1327782937 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0888102190 msgr2=0x7f0888198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:10.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.412+0000 7f0887fff640 1 --2- 172.21.15.86:0/1327782937 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0888102190 0x7f0888198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.412+0000 7f0887fff640 1 -- 172.21.15.86:0/1327782937 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f088819db10 con 0x7f0888107260 2024-04-21T08:49:10.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.412+0000 7f088d6a1640 1 --2- 172.21.15.86:0/1327782937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f088819d020 0x7f088819d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:10.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.413+0000 7f0887fff640 1 --2- 172.21.15.86:0/1327782937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0888107260 0x7f0888198ca0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f0874005fd0 tx=0x7f087400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:10.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.413+0000 7f0885ffb640 1 -- 172.21.15.86:0/1327782937 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0874017070 con 0x7f0888107260 2024-04-21T08:49:10.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.413+0000 7f0885ffb640 1 -- 172.21.15.86:0/1327782937 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0874009c30 con 0x7f0888107260 2024-04-21T08:49:10.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.413+0000 7f088f104640 1 -- 172.21.15.86:0/1327782937 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f088819de00 con 0x7f0888107260 2024-04-21T08:49:10.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.413+0000 7f0885ffb640 1 -- 172.21.15.86:0/1327782937 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0874003d30 con 0x7f0888107260 2024-04-21T08:49:10.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.413+0000 7f088f104640 1 -- 172.21.15.86:0/1327782937 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f08881a55f0 con 0x7f0888107260 2024-04-21T08:49:10.418 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.415+0000 7f0885ffb640 1 -- 172.21.15.86:0/1327782937 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0874003ed0 con 0x7f0888107260 2024-04-21T08:49:10.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.416+0000 7f0885ffb640 1 --2- 172.21.15.86:0/1327782937 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f08780a3d20 0x7f08780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.416+0000 7f0891dd3740 1 Processor -- start 2024-04-21T08:49:10.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.416+0000 7f0891dd3740 1 -- start start 2024-04-21T08:49:10.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.416+0000 7f088cea0640 1 --2- 172.21.15.86:0/1327782937 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f08780a3d20 0x7f08780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.416+0000 7f0885ffb640 1 -- 172.21.15.86:0/1327782937 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9584+0+0 (secure 0 0 0) 0x7f08740d08d0 con 0x7f0888107260 2024-04-21T08:49:10.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.416+0000 7f0891dd3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595073877c0 0x559507387ba0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.416+0000 7f0891dd3740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559507388170 0x559507390ca0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.416+0000 7f0891dd3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559507391270 0x5595073916d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.416+0000 7f0891dd3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5595072ebf00 con 0x5595073877c0 2024-04-21T08:49:10.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.416+0000 7f0891dd3740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x559507289590 con 0x559507388170 2024-04-21T08:49:10.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.416+0000 7f0891dd3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5595072e7ec0 con 0x559507391270 2024-04-21T08:49:10.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.416+0000 7f088cea0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595073877c0 0x559507387ba0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.416+0000 7f088cea0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595073877c0 0x559507387ba0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32842/0 (socket says 172.21.15.86:32842) 2024-04-21T08:49:10.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.416+0000 7f088cea0640 1 -- 172.21.15.86:0/2577314329 learned_addr learned my addr 172.21.15.86:0/2577314329 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:10.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.416+0000 7f0887fff640 1 --2- 172.21.15.86:0/2577314329 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559507388170 0x559507390ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.416+0000 7f088d6a1640 1 --2- 172.21.15.86:0/2577314329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559507391270 0x5595073916d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.417+0000 7f088cea0640 1 --2- 172.21.15.86:0/1327782937 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f08780a3d20 0x7f08780a61e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f087c006010 tx=0x7f087c004040 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:10.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.417+0000 7f088cea0640 1 -- 172.21.15.86:0/2577314329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559507391270 msgr2=0x5595073916d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:10.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.417+0000 7f088cea0640 1 --2- 172.21.15.86:0/2577314329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559507391270 0x5595073916d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.417+0000 7f088cea0640 1 -- 172.21.15.86:0/2577314329 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559507388170 msgr2=0x559507390ca0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:10.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.417+0000 7f088cea0640 1 --2- 172.21.15.86:0/2577314329 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559507388170 0x559507390ca0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.417+0000 7f088cea0640 1 -- 172.21.15.86:0/2577314329 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5595070c67e0 con 0x5595073877c0 2024-04-21T08:49:10.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.417+0000 7f088cea0640 1 --2- 172.21.15.86:0/2577314329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595073877c0 0x559507387ba0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f087c0096c0 tx=0x7f087c046000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8ae571948f742c71 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:10.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.417+0000 7f0872ffd640 1 -- 172.21.15.86:0/2577314329 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087c047070 con 0x5595073877c0 2024-04-21T08:49:10.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.417+0000 7f0872ffd640 1 -- 172.21.15.86:0/2577314329 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f087c050070 con 0x5595073877c0 2024-04-21T08:49:10.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.417+0000 7f0872ffd640 1 -- 172.21.15.86:0/2577314329 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087c004d40 con 0x5595073877c0 2024-04-21T08:49:10.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.418+0000 7f0891dd3740 1 -- 172.21.15.86:0/2577314329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595073877c0 msgr2=0x559507387ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:10.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.418+0000 7f0891dd3740 1 --2- 172.21.15.86:0/2577314329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595073877c0 0x559507387ba0 secure :-1 s=READY pgs=1870 cs=0 l=1 rev1=1 crypto rx=0x7f087c0096c0 tx=0x7f087c046000 comp rx=0 tx=0).stop 2024-04-21T08:49:10.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.418+0000 7f0891dd3740 1 -- 172.21.15.86:0/2577314329 shutdown_connections 2024-04-21T08:49:10.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.418+0000 7f0891dd3740 1 --2- 172.21.15.86:0/2577314329 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559507391270 0x5595073916d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.418+0000 7f0891dd3740 1 --2- 172.21.15.86:0/2577314329 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559507388170 0x559507390ca0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.418+0000 7f0891dd3740 1 --2- 172.21.15.86:0/2577314329 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595073877c0 0x559507387ba0 unknown :-1 s=CLOSED pgs=1870 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.418+0000 7f0891dd3740 1 -- 172.21.15.86:0/2577314329 >> 172.21.15.86:0/2577314329 conn(0x559507381fb0 msgr2=0x559507382390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:10.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.418+0000 7f0891dd3740 1 -- 172.21.15.86:0/2577314329 shutdown_connections 2024-04-21T08:49:10.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.418+0000 7f0891dd3740 1 -- 172.21.15.86:0/2577314329 wait complete. 2024-04-21T08:49:10.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.418+0000 7f0891dd3740 1 -- start start 2024-04-21T08:49:10.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.418+0000 7f0891dd3740 1 Processor -- start 2024-04-21T08:49:10.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.418+0000 7f0891dd3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595073877c0 0x559507394d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.418+0000 7f0891dd3740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559507388170 0x559507395270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.418+0000 7f0891dd3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559507391270 0x5595073957b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.418+0000 7f0891dd3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x559507398150 con 0x5595073877c0 2024-04-21T08:49:10.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.419+0000 7f0891dd3740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5595072a5580 con 0x559507388170 2024-04-21T08:49:10.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.419+0000 7f0891dd3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5595073982d0 con 0x559507391270 2024-04-21T08:49:10.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.419+0000 7f0891dd3740 10 client.0 _ll_register_callbacks cb 0x5595072ab650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:10.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.419+0000 7f088cea0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595073877c0 0x559507394d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.419+0000 7f088cea0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595073877c0 0x559507394d30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32862/0 (socket says 172.21.15.86:32862) 2024-04-21T08:49:10.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.419+0000 7f088cea0640 1 -- 172.21.15.86:0/3495762016 learned_addr learned my addr 172.21.15.86:0/3495762016 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:10.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.419+0000 7f088d6a1640 1 --2- 172.21.15.86:0/3495762016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559507391270 0x5595073957b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.419+0000 7f0887fff640 1 --2- 172.21.15.86:0/3495762016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559507388170 0x559507395270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.419+0000 7f088cea0640 1 -- 172.21.15.86:0/3495762016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559507391270 msgr2=0x5595073957b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:10.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.419+0000 7f088cea0640 1 --2- 172.21.15.86:0/3495762016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559507391270 0x5595073957b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.419+0000 7f088cea0640 1 -- 172.21.15.86:0/3495762016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559507388170 msgr2=0x559507395270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:10.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.419+0000 7f088cea0640 1 --2- 172.21.15.86:0/3495762016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559507388170 0x559507395270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.419+0000 7f088cea0640 1 -- 172.21.15.86:0/3495762016 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x559507126f70 con 0x5595073877c0 2024-04-21T08:49:10.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.419+0000 7f0887fff640 1 --2- 172.21.15.86:0/3495762016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559507388170 0x559507395270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:10.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.419+0000 7f088cea0640 1 --2- 172.21.15.86:0/3495762016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595073877c0 0x559507394d30 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f087c02c170 tx=0x7f087c02c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2be7be76c7746216 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:10.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.419+0000 7f0867fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:10.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.419+0000 7f0867fff640 1 -- 172.21.15.86:0/3495762016 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f087c058020 con 0x5595073877c0 2024-04-21T08:49:10.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.419+0000 7f0867fff640 1 -- 172.21.15.86:0/3495762016 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9584+0+0 (secure 0 0 0) 0x7f087c003a00 con 0x5595073877c0 2024-04-21T08:49:10.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.419+0000 7f0891dd3740 1 -- 172.21.15.86:0/3495762016 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f087c050070 con 0x5595073877c0 2024-04-21T08:49:10.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.419+0000 7f0867fff640 1 -- 172.21.15.86:0/3495762016 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f087c063050 con 0x5595073877c0 2024-04-21T08:49:10.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.419+0000 7f0891dd3740 10 client.19836 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:10.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.419+0000 7f0891dd3740 1 -- 172.21.15.86:0/3495762016 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x559506fbe170 con 0x5595073877c0 2024-04-21T08:49:10.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.420+0000 7f0867fff640 1 -- 172.21.15.86:0/3495762016 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f087c050070 con 0x5595073877c0 2024-04-21T08:49:10.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.420+0000 7f0867fff640 1 -- 172.21.15.86:0/3495762016 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f087c04a070 con 0x5595073877c0 2024-04-21T08:49:10.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.420+0000 7f0891dd3740 10 client.19836 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:10.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.420+0000 7f0891dd3740 10 client.19836 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:10.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.420+0000 7f0891dd3740 20 client.19836 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:10.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.420+0000 7f0891dd3740 1 --2- 172.21.15.86:0/3495762016 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5595073a5560 0x5595073a7950 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.420+0000 7f0891dd3740 4 client.19836 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:10.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.420+0000 7f0891dd3740 1 -- 172.21.15.86:0/3495762016 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5595072f9920 con 0x5595073a5560 2024-04-21T08:49:10.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.420+0000 7f0887fff640 1 --2- 172.21.15.86:0/3495762016 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5595073a5560 0x5595073a7950 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.420+0000 7f0867fff640 10 client.19836 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:10.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.420+0000 7f0887fff640 1 --2- 172.21.15.86:0/3495762016 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5595073a5560 0x5595073a7950 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=696c8439010b66c server_cookie=64ef89d540c78659 in_seq=0 out_seq=0 2024-04-21T08:49:10.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.421+0000 7f0867fff640 1 -- 172.21.15.86:0/3495762016 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5595072f9920 con 0x5595073a5560 2024-04-21T08:49:10.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.421+0000 7f0867fff640 10 client.19836 handle_command_reply: tid=0 2024-04-21T08:49:10.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.421+0000 7f0891dd3740 1 client.19836 shutdown 2024-04-21T08:49:10.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.421+0000 7f0891dd3740 1 -- 172.21.15.86:0/3495762016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595073877c0 msgr2=0x559507394d30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:10.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.421+0000 7f0891dd3740 1 --2- 172.21.15.86:0/3495762016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595073877c0 0x559507394d30 secure :-1 s=READY pgs=1871 cs=0 l=1 rev1=1 crypto rx=0x7f087c02c170 tx=0x7f087c02c710 comp rx=0 tx=0).stop 2024-04-21T08:49:10.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.421+0000 7f0891dd3740 1 -- 172.21.15.86:0/3495762016 shutdown_connections 2024-04-21T08:49:10.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.421+0000 7f0891dd3740 1 --2- 172.21.15.86:0/3495762016 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5595073a5560 0x5595073a7950 crc :-1 s=READY pgs=303 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.421+0000 7f0891dd3740 1 --2- 172.21.15.86:0/3495762016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x559507391270 0x5595073957b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.421+0000 7f0891dd3740 1 --2- 172.21.15.86:0/3495762016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559507388170 0x559507395270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.421+0000 7f0891dd3740 1 --2- 172.21.15.86:0/3495762016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5595073877c0 0x559507394d30 unknown :-1 s=CLOSED pgs=1871 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.421+0000 7f0867fff640 0 client.19836 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:10.430 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.421+0000 7f0891dd3740 1 -- 172.21.15.86:0/3495762016 >> 172.21.15.86:0/3495762016 conn(0x559507381fb0 msgr2=0x559507383d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:10.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.421+0000 7f0891dd3740 1 -- 172.21.15.86:0/3495762016 shutdown_connections 2024-04-21T08:49:10.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.421+0000 7f0891dd3740 1 -- 172.21.15.86:0/3495762016 wait complete. 2024-04-21T08:49:10.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.422+0000 7f0891dd3740 20 client.19836 trim_cache size 0 max 16384 2024-04-21T08:49:10.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.439+0000 7f0891dd3740 1 Processor -- start 2024-04-21T08:49:10.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.439+0000 7f0891dd3740 1 -- start start 2024-04-21T08:49:10.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.439+0000 7f0891dd3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595073877c0 0x559507397460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.439+0000 7f0891dd3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559507388170 0x559507394610 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.439+0000 7f0891dd3740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559507391270 0x559507392660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.439+0000 7f0891dd3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f08881ad2f0 con 0x559507388170 2024-04-21T08:49:10.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.439+0000 7f0891dd3740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f08881aee30 con 0x559507391270 2024-04-21T08:49:10.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.439+0000 7f0891dd3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f08881ad9c0 con 0x5595073877c0 2024-04-21T08:49:10.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.439+0000 7f0887fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559507388170 0x559507394610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.439+0000 7f0887fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559507388170 0x559507394610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32866/0 (socket says 172.21.15.86:32866) 2024-04-21T08:49:10.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.439+0000 7f0887fff640 1 -- 172.21.15.86:0/2373356747 learned_addr learned my addr 172.21.15.86:0/2373356747 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:10.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.440+0000 7f088d6a1640 1 --2- 172.21.15.86:0/2373356747 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559507391270 0x559507392660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.440+0000 7f088cea0640 1 --2- 172.21.15.86:0/2373356747 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595073877c0 0x559507397460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.440+0000 7f0887fff640 1 -- 172.21.15.86:0/2373356747 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595073877c0 msgr2=0x559507397460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:10.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.440+0000 7f0887fff640 1 --2- 172.21.15.86:0/2373356747 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595073877c0 0x559507397460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.440+0000 7f0887fff640 1 -- 172.21.15.86:0/2373356747 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559507391270 msgr2=0x559507392660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:10.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.440+0000 7f0887fff640 1 --2- 172.21.15.86:0/2373356747 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559507391270 0x559507392660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.440+0000 7f0887fff640 1 -- 172.21.15.86:0/2373356747 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x559507388550 con 0x559507388170 2024-04-21T08:49:10.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.440+0000 7f088d6a1640 1 --2- 172.21.15.86:0/2373356747 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559507391270 0x559507392660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:10.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.440+0000 7f0887fff640 1 --2- 172.21.15.86:0/2373356747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559507388170 0x559507394610 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f087409c390 tx=0x7f0874098820 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2f5fb3422a654af8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:10.445 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.440+0000 7f08727fc640 1 -- 172.21.15.86:0/2373356747 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0874085070 con 0x559507388170 2024-04-21T08:49:10.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.440+0000 7f08727fc640 1 -- 172.21.15.86:0/2373356747 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f087409a4f0 con 0x559507388170 2024-04-21T08:49:10.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.440+0000 7f08727fc640 1 -- 172.21.15.86:0/2373356747 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f08740041c0 con 0x559507388170 2024-04-21T08:49:10.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.441+0000 7f0891dd3740 1 -- 172.21.15.86:0/2373356747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559507388170 msgr2=0x559507394610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:10.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.441+0000 7f0891dd3740 1 --2- 172.21.15.86:0/2373356747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559507388170 0x559507394610 secure :-1 s=READY pgs=1872 cs=0 l=1 rev1=1 crypto rx=0x7f087409c390 tx=0x7f0874098820 comp rx=0 tx=0).stop 2024-04-21T08:49:10.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.441+0000 7f0891dd3740 1 -- 172.21.15.86:0/2373356747 shutdown_connections 2024-04-21T08:49:10.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.441+0000 7f0891dd3740 1 --2- 172.21.15.86:0/2373356747 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559507391270 0x559507392660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.441+0000 7f0891dd3740 1 --2- 172.21.15.86:0/2373356747 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559507388170 0x559507394610 unknown :-1 s=CLOSED pgs=1872 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.441+0000 7f0891dd3740 1 --2- 172.21.15.86:0/2373356747 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595073877c0 0x559507397460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.441+0000 7f0891dd3740 1 -- 172.21.15.86:0/2373356747 >> 172.21.15.86:0/2373356747 conn(0x559507381fb0 msgr2=0x559507383c20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:10.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.441+0000 7f0891dd3740 1 -- 172.21.15.86:0/2373356747 shutdown_connections 2024-04-21T08:49:10.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.441+0000 7f0891dd3740 1 -- 172.21.15.86:0/2373356747 wait complete. 2024-04-21T08:49:10.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.441+0000 7f0891dd3740 1 -- start start 2024-04-21T08:49:10.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.441+0000 7f0891dd3740 1 Processor -- start 2024-04-21T08:49:10.446 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.441+0000 7f0891dd3740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595073877c0 0x5595073aaef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.441+0000 7f0891dd3740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559507388170 0x5595073a8280 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.441+0000 7f0891dd3740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559507391270 0x5595073a8970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.441+0000 7f0891dd3740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f08881ae760 con 0x559507388170 2024-04-21T08:49:10.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.441+0000 7f0891dd3740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f08881ae090 con 0x559507391270 2024-04-21T08:49:10.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.441+0000 7f0891dd3740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f08881ab2d0 con 0x5595073877c0 2024-04-21T08:49:10.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.441+0000 7f0891dd3740 10 client.0 _ll_register_callbacks cb 0x7f08740986f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:10.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.441+0000 7f0887fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559507388170 0x5595073a8280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.442+0000 7f0887fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559507388170 0x5595073a8280 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32878/0 (socket says 172.21.15.86:32878) 2024-04-21T08:49:10.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.442+0000 7f0887fff640 1 -- 172.21.15.86:0/3786769551 learned_addr learned my addr 172.21.15.86:0/3786769551 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:10.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.442+0000 7f088d6a1640 1 --2- 172.21.15.86:0/3786769551 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559507391270 0x5595073a8970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.442+0000 7f088cea0640 1 --2- 172.21.15.86:0/3786769551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595073877c0 0x5595073aaef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.442+0000 7f0887fff640 1 -- 172.21.15.86:0/3786769551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595073877c0 msgr2=0x5595073aaef0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:10.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.442+0000 7f0887fff640 1 --2- 172.21.15.86:0/3786769551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595073877c0 0x5595073aaef0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.442+0000 7f0887fff640 1 -- 172.21.15.86:0/3786769551 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559507391270 msgr2=0x5595073a8970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:10.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.442+0000 7f0887fff640 1 --2- 172.21.15.86:0/3786769551 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559507391270 0x5595073a8970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.447 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.442+0000 7f0887fff640 1 -- 172.21.15.86:0/3786769551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55950740a740 con 0x559507388170 2024-04-21T08:49:10.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.442+0000 7f088d6a1640 1 --2- 172.21.15.86:0/3786769551 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559507391270 0x5595073a8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:10.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.442+0000 7f088cea0640 1 --2- 172.21.15.86:0/3786769551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595073877c0 0x5595073aaef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:10.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.442+0000 7f0887fff640 1 --2- 172.21.15.86:0/3786769551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559507388170 0x5595073a8280 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f087409c3e0 tx=0x7f087409c550 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d35c92fd77fd9f12 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:10.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.442+0000 7f0867fff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:10.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.442+0000 7f0867fff640 1 -- 172.21.15.86:0/3786769551 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0874085040 con 0x559507388170 2024-04-21T08:49:10.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.442+0000 7f0867fff640 1 -- 172.21.15.86:0/3786769551 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9584+0+0 (secure 0 0 0) 0x7f08740041c0 con 0x559507388170 2024-04-21T08:49:10.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.442+0000 7f0891dd3740 1 -- 172.21.15.86:0/3786769551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f087409a4f0 con 0x559507388170 2024-04-21T08:49:10.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.442+0000 7f0867fff640 1 -- 172.21.15.86:0/3786769551 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f08740043c0 con 0x559507388170 2024-04-21T08:49:10.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.442+0000 7f0891dd3740 10 client.19842 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:10.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.442+0000 7f0891dd3740 1 -- 172.21.15.86:0/3786769551 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55950740b890 con 0x559507388170 2024-04-21T08:49:10.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.442+0000 7f0867fff640 1 -- 172.21.15.86:0/3786769551 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f087409a4f0 con 0x559507388170 2024-04-21T08:49:10.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.443+0000 7f0867fff640 1 -- 172.21.15.86:0/3786769551 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f087409b8a0 con 0x559507388170 2024-04-21T08:49:10.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.443+0000 7f0891dd3740 10 client.19842 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:10.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.443+0000 7f0891dd3740 10 client.19842 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:10.448 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.443+0000 7f0891dd3740 20 client.19842 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:10.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.443+0000 7f0891dd3740 1 --2- 172.21.15.86:0/3786769551 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55950740c040 0x55950740e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.443+0000 7f0891dd3740 4 client.19842 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:49:10.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.443+0000 7f0891dd3740 1 -- 172.21.15.86:0/3786769551 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x559507391030 con 0x55950740c040 2024-04-21T08:49:10.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.443+0000 7f088cea0640 1 --2- 172.21.15.86:0/3786769551 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55950740c040 0x55950740e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.443+0000 7f0867fff640 10 client.19842 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:10.449 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.443+0000 7f088cea0640 1 --2- 172.21.15.86:0/3786769551 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55950740c040 0x55950740e430 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4020cc1351eb88d server_cookie=24f99fd06cf08cff in_seq=0 out_seq=0 2024-04-21T08:49:10.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.444+0000 7f0867fff640 1 -- 172.21.15.86:0/3786769551 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x559507391030 con 0x55950740c040 2024-04-21T08:49:10.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.444+0000 7f0867fff640 10 client.19842 handle_command_reply: tid=0 2024-04-21T08:49:10.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.444+0000 7f0891dd3740 1 client.19842 shutdown 2024-04-21T08:49:10.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.445+0000 7f0891dd3740 1 -- 172.21.15.86:0/3786769551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559507388170 msgr2=0x5595073a8280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:10.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.445+0000 7f0891dd3740 1 --2- 172.21.15.86:0/3786769551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559507388170 0x5595073a8280 secure :-1 s=READY pgs=1873 cs=0 l=1 rev1=1 crypto rx=0x7f087409c3e0 tx=0x7f087409c550 comp rx=0 tx=0).stop 2024-04-21T08:49:10.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.445+0000 7f0891dd3740 1 -- 172.21.15.86:0/3786769551 shutdown_connections 2024-04-21T08:49:10.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.445+0000 7f0891dd3740 1 --2- 172.21.15.86:0/3786769551 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55950740c040 0x55950740e430 crc :-1 s=READY pgs=304 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.445+0000 7f0891dd3740 1 --2- 172.21.15.86:0/3786769551 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x559507391270 0x5595073a8970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.445+0000 7f0891dd3740 1 --2- 172.21.15.86:0/3786769551 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x559507388170 0x5595073a8280 unknown :-1 s=CLOSED pgs=1873 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.445+0000 7f0891dd3740 1 --2- 172.21.15.86:0/3786769551 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5595073877c0 0x5595073aaef0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.445+0000 7f0891dd3740 1 -- 172.21.15.86:0/3786769551 >> 172.21.15.86:0/3786769551 conn(0x559507381fb0 msgr2=0x559507394c80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:10.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.445+0000 7f0867fff640 0 client.19842 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:10.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.445+0000 7f0891dd3740 1 -- 172.21.15.86:0/3786769551 shutdown_connections 2024-04-21T08:49:10.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.445+0000 7f0891dd3740 1 -- 172.21.15.86:0/3786769551 wait complete. 2024-04-21T08:49:10.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.445+0000 7f0891dd3740 20 client.19842 trim_cache size 0 max 16384 2024-04-21T08:49:10.451 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:49:10.451 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-21T08:49:10.451 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "ff05a4fe-12c8-4fc1-9d0a-792c4b66ab6c", 2024-04-21T08:49:10.451 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-21T08:49:10.451 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:49:10.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.445+0000 7f088f104640 1 -- 172.21.15.86:0/1327782937 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f08780a3d20 msgr2=0x7f08780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:10.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.445+0000 7f088f104640 1 --2- 172.21.15.86:0/1327782937 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f08780a3d20 0x7f08780a61e0 secure :-1 s=READY pgs=310 cs=0 l=1 rev1=1 crypto rx=0x7f087c006010 tx=0x7f087c004040 comp rx=0 tx=0).stop 2024-04-21T08:49:10.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.445+0000 7f088f104640 1 -- 172.21.15.86:0/1327782937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0888107260 msgr2=0x7f0888198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:10.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.445+0000 7f088f104640 1 --2- 172.21.15.86:0/1327782937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0888107260 0x7f0888198ca0 secure :-1 s=READY pgs=1869 cs=0 l=1 rev1=1 crypto rx=0x7f0874005fd0 tx=0x7f087400dde0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.446+0000 7f088f104640 1 -- 172.21.15.86:0/1327782937 shutdown_connections 2024-04-21T08:49:10.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.446+0000 7f088f104640 1 --2- 172.21.15.86:0/1327782937 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f08780a3d20 0x7f08780a61e0 unknown :-1 s=CLOSED pgs=310 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.446+0000 7f088f104640 1 --2- 172.21.15.86:0/1327782937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f088819d020 0x7f088819d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.446+0000 7f088f104640 1 --2- 172.21.15.86:0/1327782937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0888107260 0x7f0888198ca0 unknown :-1 s=CLOSED pgs=1869 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.446+0000 7f088f104640 1 --2- 172.21.15.86:0/1327782937 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0888102190 0x7f0888198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.446+0000 7f088f104640 1 -- 172.21.15.86:0/1327782937 >> 172.21.15.86:0/1327782937 conn(0x7f08880793a0 msgr2=0x7f0888100090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:10.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.446+0000 7f088f104640 1 -- 172.21.15.86:0/1327782937 shutdown_connections 2024-04-21T08:49:10.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.446+0000 7f088f104640 1 -- 172.21.15.86:0/1327782937 wait complete. 2024-04-21T08:49:10.457 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:49:10.457 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:49:10.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:10 smithi179 ceph-mon[31829]: pgmap v323: 129 pgs: 129 active+clean; 2.7 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 102 KiB/s wr, 38 op/s 2024-04-21T08:49:10.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.557+0000 7ff052a81640 1 -- 172.21.15.86:0/1287918076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff04c101a30 msgr2=0x7ff04c105a70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:10.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.557+0000 7ff052a81640 1 --2- 172.21.15.86:0/1287918076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff04c101a30 0x7ff04c105a70 secure :-1 s=READY pgs=1874 cs=0 l=1 rev1=1 crypto rx=0x7ff038009900 tx=0x7ff03802bc30 comp rx=0 tx=0).stop 2024-04-21T08:49:10.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.557+0000 7ff052a81640 1 -- 172.21.15.86:0/1287918076 shutdown_connections 2024-04-21T08:49:10.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.557+0000 7ff052a81640 1 --2- 172.21.15.86:0/1287918076 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff04c106150 0x7ff04c10dd20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.557+0000 7ff052a81640 1 --2- 172.21.15.86:0/1287918076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff04c101a30 0x7ff04c105a70 unknown :-1 s=CLOSED pgs=1874 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.557+0000 7ff052a81640 1 --2- 172.21.15.86:0/1287918076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff04c101080 0x7ff04c101460 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.557+0000 7ff052a81640 1 -- 172.21.15.86:0/1287918076 >> 172.21.15.86:0/1287918076 conn(0x7ff04c078000 msgr2=0x7ff04c078410 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:10.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.557+0000 7ff052a81640 1 -- 172.21.15.86:0/1287918076 shutdown_connections 2024-04-21T08:49:10.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.557+0000 7ff052a81640 1 -- 172.21.15.86:0/1287918076 wait complete. 2024-04-21T08:49:10.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.557+0000 7ff052a81640 1 Processor -- start 2024-04-21T08:49:10.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.557+0000 7ff052a81640 1 -- start start 2024-04-21T08:49:10.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.557+0000 7ff052a81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff04c101080 0x7ff04c197670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.557+0000 7ff052a81640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff04c106150 0x7ff04c197bb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.557+0000 7ff052a81640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff04c19bf30 0x7ff04c19c3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.557+0000 7ff052a81640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff04c111070 con 0x7ff04c101080 2024-04-21T08:49:10.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.557+0000 7ff052a81640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff04c110ef0 con 0x7ff04c19bf30 2024-04-21T08:49:10.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.557+0000 7ff052a81640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff04c1111f0 con 0x7ff04c106150 2024-04-21T08:49:10.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.558+0000 7ff05081d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff04c101080 0x7ff04c197670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.558+0000 7ff05081d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff04c101080 0x7ff04c197670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32908/0 (socket says 172.21.15.86:32908) 2024-04-21T08:49:10.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.558+0000 7ff05081d640 1 -- 172.21.15.86:0/2669655816 learned_addr learned my addr 172.21.15.86:0/2669655816 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:10.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.558+0000 7ff05101e640 1 --2- 172.21.15.86:0/2669655816 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff04c19bf30 0x7ff04c19c3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.558+0000 7ff04bfff640 1 --2- 172.21.15.86:0/2669655816 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff04c106150 0x7ff04c197bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.558+0000 7ff05081d640 1 -- 172.21.15.86:0/2669655816 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff04c106150 msgr2=0x7ff04c197bb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:10.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.558+0000 7ff05081d640 1 --2- 172.21.15.86:0/2669655816 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff04c106150 0x7ff04c197bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.558+0000 7ff05081d640 1 -- 172.21.15.86:0/2669655816 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff04c19bf30 msgr2=0x7ff04c19c3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:10.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.558+0000 7ff05081d640 1 --2- 172.21.15.86:0/2669655816 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff04c19bf30 0x7ff04c19c3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.558+0000 7ff05081d640 1 -- 172.21.15.86:0/2669655816 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff04c19ca20 con 0x7ff04c101080 2024-04-21T08:49:10.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.558+0000 7ff05101e640 1 --2- 172.21.15.86:0/2669655816 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff04c19bf30 0x7ff04c19c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:10.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.558+0000 7ff05081d640 1 --2- 172.21.15.86:0/2669655816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff04c101080 0x7ff04c197670 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7ff040009900 tx=0x7ff040009dc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:10.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.558+0000 7ff049ffb640 1 -- 172.21.15.86:0/2669655816 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04000d030 con 0x7ff04c101080 2024-04-21T08:49:10.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.558+0000 7ff052a81640 1 -- 172.21.15.86:0/2669655816 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff04c19cd10 con 0x7ff04c101080 2024-04-21T08:49:10.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.558+0000 7ff049ffb640 1 -- 172.21.15.86:0/2669655816 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff04000d1d0 con 0x7ff04c101080 2024-04-21T08:49:10.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.558+0000 7ff049ffb640 1 -- 172.21.15.86:0/2669655816 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff040005280 con 0x7ff04c101080 2024-04-21T08:49:10.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.558+0000 7ff052a81640 1 -- 172.21.15.86:0/2669655816 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff04c069450 con 0x7ff04c101080 2024-04-21T08:49:10.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.560+0000 7ff049ffb640 1 -- 172.21.15.86:0/2669655816 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff040005420 con 0x7ff04c101080 2024-04-21T08:49:10.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.561+0000 7ff05567c740 1 Processor -- start 2024-04-21T08:49:10.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.561+0000 7ff05567c740 1 -- start start 2024-04-21T08:49:10.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.561+0000 7ff049ffb640 1 --2- 172.21.15.86:0/2669655816 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff03c0a3d20 0x7ff03c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.561+0000 7ff05567c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4cd95d580 0x55d4cd95d960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.561+0000 7ff05567c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4cd95dea0 0x55d4cd966540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.561+0000 7ff05101e640 1 --2- 172.21.15.86:0/2669655816 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff03c0a3d20 0x7ff03c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.561+0000 7ff049ffb640 1 -- 172.21.15.86:0/2669655816 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(103..103 src has 1..103) ==== 9584+0+0 (secure 0 0 0) 0x7ff0400d2010 con 0x7ff04c101080 2024-04-21T08:49:10.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.561+0000 7ff05567c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4cd966a80 0x55d4cd968e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.561+0000 7ff05567c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d4cd8c21e0 con 0x55d4cd95dea0 2024-04-21T08:49:10.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.561+0000 7ff05567c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d4cd85f390 con 0x55d4cd95d580 2024-04-21T08:49:10.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.561+0000 7ff05567c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d4cd8be1a0 con 0x55d4cd966a80 2024-04-21T08:49:10.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.561+0000 7ff04bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4cd95dea0 0x55d4cd966540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.561+0000 7ff04bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4cd95dea0 0x55d4cd966540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32914/0 (socket says 172.21.15.86:32914) 2024-04-21T08:49:10.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.561+0000 7ff04bfff640 1 -- 172.21.15.86:0/849232923 learned_addr learned my addr 172.21.15.86:0/849232923 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:10.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.561+0000 7ff05081d640 1 --2- 172.21.15.86:0/849232923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4cd95d580 0x55d4cd95d960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.561+0000 7ff04bfff640 1 -- 172.21.15.86:0/849232923 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4cd966a80 msgr2=0x55d4cd968e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:10.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.561+0000 7ff04bfff640 1 --2- 172.21.15.86:0/849232923 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4cd966a80 0x55d4cd968e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.561+0000 7ff04bfff640 1 -- 172.21.15.86:0/849232923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4cd95d580 msgr2=0x55d4cd95d960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:10.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.561+0000 7ff04bfff640 1 --2- 172.21.15.86:0/849232923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4cd95d580 0x55d4cd95d960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.561+0000 7ff04bfff640 1 -- 172.21.15.86:0/849232923 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4cd69e7d0 con 0x55d4cd95dea0 2024-04-21T08:49:10.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.562+0000 7ff05101e640 1 --2- 172.21.15.86:0/2669655816 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff03c0a3d20 0x7ff03c0a61e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff044008c60 tx=0x7ff04400d710 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:10.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.562+0000 7ff04bfff640 1 --2- 172.21.15.86:0/849232923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4cd95dea0 0x55d4cd966540 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7ff038006010 tx=0x7ff038005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=77cfa4ade34c6770 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:10.569 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.562+0000 7ff036ffd640 1 -- 172.21.15.86:0/849232923 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff038043020 con 0x55d4cd95dea0 2024-04-21T08:49:10.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.562+0000 7ff036ffd640 1 -- 172.21.15.86:0/849232923 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff03802c7e0 con 0x55d4cd95dea0 2024-04-21T08:49:10.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.562+0000 7ff036ffd640 1 -- 172.21.15.86:0/849232923 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0380054b0 con 0x55d4cd95dea0 2024-04-21T08:49:10.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.562+0000 7ff05567c740 1 -- 172.21.15.86:0/849232923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4cd95dea0 msgr2=0x55d4cd966540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:10.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.562+0000 7ff05567c740 1 --2- 172.21.15.86:0/849232923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4cd95dea0 0x55d4cd966540 secure :-1 s=READY pgs=1876 cs=0 l=1 rev1=1 crypto rx=0x7ff038006010 tx=0x7ff038005e80 comp rx=0 tx=0).stop 2024-04-21T08:49:10.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.563+0000 7ff05567c740 1 -- 172.21.15.86:0/849232923 shutdown_connections 2024-04-21T08:49:10.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.563+0000 7ff05567c740 1 --2- 172.21.15.86:0/849232923 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4cd966a80 0x55d4cd968e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.563+0000 7ff05567c740 1 --2- 172.21.15.86:0/849232923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4cd95dea0 0x55d4cd966540 unknown :-1 s=CLOSED pgs=1876 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.563+0000 7ff05567c740 1 --2- 172.21.15.86:0/849232923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4cd95d580 0x55d4cd95d960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.563+0000 7ff05567c740 1 -- 172.21.15.86:0/849232923 >> 172.21.15.86:0/849232923 conn(0x55d4cd957cb0 msgr2=0x55d4cd958090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:10.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.563+0000 7ff05567c740 1 -- 172.21.15.86:0/849232923 shutdown_connections 2024-04-21T08:49:10.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.563+0000 7ff05567c740 1 -- 172.21.15.86:0/849232923 wait complete. 2024-04-21T08:49:10.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.563+0000 7ff05567c740 1 -- start start 2024-04-21T08:49:10.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.563+0000 7ff05567c740 1 Processor -- start 2024-04-21T08:49:10.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.563+0000 7ff05567c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4cd95d580 0x55d4cd963a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.570 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.563+0000 7ff05567c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4cd95dea0 0x55d4cd963f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.563+0000 7ff05567c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4cd966a80 0x55d4cd964480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.563+0000 7ff05567c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d4cd96dc20 con 0x55d4cd95d580 2024-04-21T08:49:10.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.563+0000 7ff05567c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d4cd87b2f0 con 0x55d4cd95dea0 2024-04-21T08:49:10.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.563+0000 7ff05567c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d4cd96dda0 con 0x55d4cd966a80 2024-04-21T08:49:10.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.563+0000 7ff05567c740 10 client.0 _ll_register_callbacks cb 0x55d4cd888580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:10.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.563+0000 7ff05081d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4cd95d580 0x55d4cd963a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.563+0000 7ff05081d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4cd95d580 0x55d4cd963a00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32924/0 (socket says 172.21.15.86:32924) 2024-04-21T08:49:10.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.563+0000 7ff05081d640 1 -- 172.21.15.86:0/1243770263 learned_addr learned my addr 172.21.15.86:0/1243770263 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:10.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.564+0000 7ff04bfff640 1 --2- 172.21.15.86:0/1243770263 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4cd95dea0 0x55d4cd963f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.564+0000 7ff05101e640 1 --2- 172.21.15.86:0/1243770263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4cd966a80 0x55d4cd964480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.564+0000 7ff05081d640 1 -- 172.21.15.86:0/1243770263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4cd966a80 msgr2=0x55d4cd964480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:10.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.564+0000 7ff05081d640 1 --2- 172.21.15.86:0/1243770263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4cd966a80 0x55d4cd964480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.564+0000 7ff05081d640 1 -- 172.21.15.86:0/1243770263 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4cd95dea0 msgr2=0x55d4cd963f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:10.571 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.564+0000 7ff05081d640 1 --2- 172.21.15.86:0/1243770263 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4cd95dea0 0x55d4cd963f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.564+0000 7ff05081d640 1 -- 172.21.15.86:0/1243770263 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4cd5d43f0 con 0x55d4cd95d580 2024-04-21T08:49:10.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.564+0000 7ff05101e640 1 --2- 172.21.15.86:0/1243770263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4cd966a80 0x55d4cd964480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:10.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.564+0000 7ff05081d640 1 --2- 172.21.15.86:0/1243770263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4cd95d580 0x55d4cd963a00 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7ff040007590 tx=0x7ff04009e350 comp rx=0 tx=0).ready entity=mon.0 client_cookie=95c8080f27f46684 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:10.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.564+0000 7ff02bfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:10.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.564+0000 7ff02bfff640 1 -- 172.21.15.86:0/1243770263 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff0400b7060 con 0x55d4cd95d580 2024-04-21T08:49:10.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.564+0000 7ff02bfff640 1 -- 172.21.15.86:0/1243770263 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9584+0+0 (secure 0 0 0) 0x7ff04009a200 con 0x55d4cd95d580 2024-04-21T08:49:10.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.564+0000 7ff05567c740 1 -- 172.21.15.86:0/1243770263 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff03802c7e0 con 0x55d4cd95d580 2024-04-21T08:49:10.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.564+0000 7ff02bfff640 1 -- 172.21.15.86:0/1243770263 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff04001e040 con 0x55d4cd95d580 2024-04-21T08:49:10.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.564+0000 7ff05567c740 10 client.19854 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:10.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.564+0000 7ff05567c740 1 -- 172.21.15.86:0/1243770263 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d4cd5940f0 con 0x55d4cd95d580 2024-04-21T08:49:10.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.564+0000 7ff02bfff640 1 -- 172.21.15.86:0/1243770263 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff03802c7e0 con 0x55d4cd95d580 2024-04-21T08:49:10.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.564+0000 7ff02bfff640 1 -- 172.21.15.86:0/1243770263 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff040062b10 con 0x55d4cd95d580 2024-04-21T08:49:10.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.565+0000 7ff05567c740 10 client.19854 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:10.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.565+0000 7ff05567c740 10 client.19854 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:10.572 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.565+0000 7ff05567c740 20 client.19854 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:10.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.565+0000 7ff05567c740 1 --2- 172.21.15.86:0/1243770263 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d4cd97ad90 0x55d4cd97d180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.565+0000 7ff05567c740 4 client.19854 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:10.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.565+0000 7ff05567c740 1 -- 172.21.15.86:0/1243770263 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d4cd8cfc00 con 0x55d4cd97ad90 2024-04-21T08:49:10.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.565+0000 7ff04bfff640 1 --2- 172.21.15.86:0/1243770263 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d4cd97ad90 0x55d4cd97d180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.565+0000 7ff04bfff640 1 --2- 172.21.15.86:0/1243770263 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d4cd97ad90 0x55d4cd97d180 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f0d5cd6805ed613c server_cookie=79ac013fb05de18d in_seq=0 out_seq=0 2024-04-21T08:49:10.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.565+0000 7ff02bfff640 10 client.19854 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:10.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.566+0000 7ff02bfff640 1 -- 172.21.15.86:0/1243770263 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d4cd8cfc00 con 0x55d4cd97ad90 2024-04-21T08:49:10.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.566+0000 7ff02bfff640 10 client.19854 handle_command_reply: tid=0 2024-04-21T08:49:10.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.566+0000 7ff05567c740 1 client.19854 shutdown 2024-04-21T08:49:10.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.566+0000 7ff05567c740 1 -- 172.21.15.86:0/1243770263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4cd95d580 msgr2=0x55d4cd963a00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:10.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.566+0000 7ff05567c740 1 --2- 172.21.15.86:0/1243770263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4cd95d580 0x55d4cd963a00 secure :-1 s=READY pgs=1877 cs=0 l=1 rev1=1 crypto rx=0x7ff040007590 tx=0x7ff04009e350 comp rx=0 tx=0).stop 2024-04-21T08:49:10.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.566+0000 7ff05567c740 1 -- 172.21.15.86:0/1243770263 shutdown_connections 2024-04-21T08:49:10.573 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.566+0000 7ff05567c740 1 --2- 172.21.15.86:0/1243770263 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d4cd97ad90 0x55d4cd97d180 crc :-1 s=READY pgs=305 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.566+0000 7ff05567c740 1 --2- 172.21.15.86:0/1243770263 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4cd966a80 0x55d4cd964480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.566+0000 7ff05567c740 1 --2- 172.21.15.86:0/1243770263 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4cd95dea0 0x55d4cd963f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.566+0000 7ff05567c740 1 --2- 172.21.15.86:0/1243770263 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4cd95d580 0x55d4cd963a00 unknown :-1 s=CLOSED pgs=1877 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.566+0000 7ff02bfff640 0 client.19854 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:10.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.566+0000 7ff05567c740 1 -- 172.21.15.86:0/1243770263 >> 172.21.15.86:0/1243770263 conn(0x55d4cd957cb0 msgr2=0x55d4cd959a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:10.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.566+0000 7ff05567c740 1 -- 172.21.15.86:0/1243770263 shutdown_connections 2024-04-21T08:49:10.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.566+0000 7ff05567c740 1 -- 172.21.15.86:0/1243770263 wait complete. 2024-04-21T08:49:10.574 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.566+0000 7ff05567c740 20 client.19854 trim_cache size 0 max 16384 2024-04-21T08:49:10.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.583+0000 7ff05567c740 1 Processor -- start 2024-04-21T08:49:10.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.583+0000 7ff05567c740 1 -- start start 2024-04-21T08:49:10.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.583+0000 7ff05567c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4cd95d580 0x55d4cd966130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.583+0000 7ff05567c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4cd95dea0 0x55d4cd976b10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.583+0000 7ff05567c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4cd966a80 0x55d4cd959730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.583+0000 7ff05567c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff04c1ad3e0 con 0x55d4cd966a80 2024-04-21T08:49:10.589 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.583+0000 7ff05567c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff04c1aef20 con 0x55d4cd95d580 2024-04-21T08:49:10.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.583+0000 7ff05567c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff04c1adab0 con 0x55d4cd95dea0 2024-04-21T08:49:10.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.584+0000 7ff05101e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4cd966a80 0x55d4cd959730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.584+0000 7ff05101e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4cd966a80 0x55d4cd959730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32936/0 (socket says 172.21.15.86:32936) 2024-04-21T08:49:10.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.584+0000 7ff05101e640 1 -- 172.21.15.86:0/4244794308 learned_addr learned my addr 172.21.15.86:0/4244794308 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:10.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.584+0000 7ff05081d640 1 --2- 172.21.15.86:0/4244794308 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4cd95d580 0x55d4cd966130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.584+0000 7ff04bfff640 1 --2- 172.21.15.86:0/4244794308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4cd95dea0 0x55d4cd976b10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.584+0000 7ff05101e640 1 -- 172.21.15.86:0/4244794308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4cd95dea0 msgr2=0x55d4cd976b10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:10.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.584+0000 7ff05101e640 1 --2- 172.21.15.86:0/4244794308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4cd95dea0 0x55d4cd976b10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.584+0000 7ff05101e640 1 -- 172.21.15.86:0/4244794308 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4cd95d580 msgr2=0x55d4cd966130 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:10.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.584+0000 7ff05101e640 1 --2- 172.21.15.86:0/4244794308 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4cd95d580 0x55d4cd966130 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.584+0000 7ff05101e640 1 -- 172.21.15.86:0/4244794308 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d4cd96dc70 con 0x55d4cd966a80 2024-04-21T08:49:10.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.584+0000 7ff04bfff640 1 --2- 172.21.15.86:0/4244794308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4cd95dea0 0x55d4cd976b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:10.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.584+0000 7ff05101e640 1 --2- 172.21.15.86:0/4244794308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4cd966a80 0x55d4cd959730 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7ff044008fd0 tx=0x7ff044012000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c8be6f3679f867a8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:10.590 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.584+0000 7ff0367fc640 1 -- 172.21.15.86:0/4244794308 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04401e680 con 0x55d4cd966a80 2024-04-21T08:49:10.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.584+0000 7ff0367fc640 1 -- 172.21.15.86:0/4244794308 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff044023070 con 0x55d4cd966a80 2024-04-21T08:49:10.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.584+0000 7ff0367fc640 1 -- 172.21.15.86:0/4244794308 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04400fba0 con 0x55d4cd966a80 2024-04-21T08:49:10.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.585+0000 7ff05567c740 1 -- 172.21.15.86:0/4244794308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4cd966a80 msgr2=0x55d4cd959730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:10.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.585+0000 7ff05567c740 1 --2- 172.21.15.86:0/4244794308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4cd966a80 0x55d4cd959730 secure :-1 s=READY pgs=1878 cs=0 l=1 rev1=1 crypto rx=0x7ff044008fd0 tx=0x7ff044012000 comp rx=0 tx=0).stop 2024-04-21T08:49:10.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.585+0000 7ff05567c740 1 -- 172.21.15.86:0/4244794308 shutdown_connections 2024-04-21T08:49:10.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.585+0000 7ff05567c740 1 --2- 172.21.15.86:0/4244794308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4cd966a80 0x55d4cd959730 unknown :-1 s=CLOSED pgs=1878 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.585+0000 7ff05567c740 1 --2- 172.21.15.86:0/4244794308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4cd95dea0 0x55d4cd976b10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.585+0000 7ff05567c740 1 --2- 172.21.15.86:0/4244794308 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4cd95d580 0x55d4cd966130 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.585+0000 7ff05567c740 1 -- 172.21.15.86:0/4244794308 >> 172.21.15.86:0/4244794308 conn(0x55d4cd957cb0 msgr2=0x55d4cd959020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:10.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.585+0000 7ff05567c740 1 -- 172.21.15.86:0/4244794308 shutdown_connections 2024-04-21T08:49:10.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.585+0000 7ff05567c740 1 -- 172.21.15.86:0/4244794308 wait complete. 2024-04-21T08:49:10.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.585+0000 7ff05567c740 1 -- start start 2024-04-21T08:49:10.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.585+0000 7ff05567c740 1 Processor -- start 2024-04-21T08:49:10.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.586+0000 7ff05567c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4cd95d580 0x55d4cd96eb50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.586+0000 7ff05567c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4cd95dea0 0x55d4cd96f090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.591 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.586+0000 7ff05567c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4cd966a80 0x55d4cd96f5d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.586+0000 7ff05567c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff04c1ae850 con 0x55d4cd966a80 2024-04-21T08:49:10.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.586+0000 7ff05567c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff04c1ae180 con 0x55d4cd95d580 2024-04-21T08:49:10.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.586+0000 7ff05567c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff04c1ab3c0 con 0x55d4cd95dea0 2024-04-21T08:49:10.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.586+0000 7ff05567c740 10 client.0 _ll_register_callbacks cb 0x7ff040099f20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:10.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.586+0000 7ff05101e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4cd966a80 0x55d4cd96f5d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.586+0000 7ff05101e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4cd966a80 0x55d4cd96f5d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32948/0 (socket says 172.21.15.86:32948) 2024-04-21T08:49:10.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.586+0000 7ff05101e640 1 -- 172.21.15.86:0/1033298570 learned_addr learned my addr 172.21.15.86:0/1033298570 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:10.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.586+0000 7ff05081d640 1 --2- 172.21.15.86:0/1033298570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4cd95d580 0x55d4cd96eb50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.586+0000 7ff05101e640 1 -- 172.21.15.86:0/1033298570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4cd95dea0 msgr2=0x55d4cd96f090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:10.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.586+0000 7ff05101e640 1 --2- 172.21.15.86:0/1033298570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4cd95dea0 0x55d4cd96f090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.586+0000 7ff05101e640 1 -- 172.21.15.86:0/1033298570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4cd95d580 msgr2=0x55d4cd96eb50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:10.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.586+0000 7ff05101e640 1 --2- 172.21.15.86:0/1033298570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4cd95d580 0x55d4cd96eb50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.592 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.586+0000 7ff05101e640 1 -- 172.21.15.86:0/1033298570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d4cd9e0370 con 0x55d4cd966a80 2024-04-21T08:49:10.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.586+0000 7ff05081d640 1 --2- 172.21.15.86:0/1033298570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4cd95d580 0x55d4cd96eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:10.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.586+0000 7ff05101e640 1 --2- 172.21.15.86:0/1033298570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4cd966a80 0x55d4cd96f5d0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7ff04400f7c0 tx=0x7ff044010b60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48d8f9da16941856 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:10.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.586+0000 7ff02bfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:10.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.586+0000 7ff02bfff640 1 -- 172.21.15.86:0/1033298570 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff04401e680 con 0x55d4cd966a80 2024-04-21T08:49:10.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.586+0000 7ff02bfff640 1 -- 172.21.15.86:0/1033298570 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(103..103 src has 1..103) ==== 9584+0+0 (secure 0 0 0) 0x7ff04401e820 con 0x55d4cd966a80 2024-04-21T08:49:10.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.586+0000 7ff05567c740 1 -- 172.21.15.86:0/1033298570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff044023070 con 0x55d4cd966a80 2024-04-21T08:49:10.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.586+0000 7ff02bfff640 1 -- 172.21.15.86:0/1033298570 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff04403b050 con 0x55d4cd966a80 2024-04-21T08:49:10.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.586+0000 7ff05567c740 10 client.19860 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:10.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.586+0000 7ff05567c740 1 -- 172.21.15.86:0/1033298570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d4cd9e1660 con 0x55d4cd966a80 2024-04-21T08:49:10.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.587+0000 7ff02bfff640 1 -- 172.21.15.86:0/1033298570 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff044023070 con 0x55d4cd966a80 2024-04-21T08:49:10.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.587+0000 7ff02bfff640 1 -- 172.21.15.86:0/1033298570 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff0440329e0 con 0x55d4cd966a80 2024-04-21T08:49:10.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.587+0000 7ff05567c740 10 client.19860 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:10.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.587+0000 7ff05567c740 10 client.19860 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:10.593 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.587+0000 7ff05567c740 20 client.19860 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:10.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.587+0000 7ff05567c740 1 --2- 172.21.15.86:0/1033298570 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d4cd9e1cd0 0x55d4cd9e40c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:10.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.587+0000 7ff05567c740 4 client.19860 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:49:10.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.587+0000 7ff05567c740 1 -- 172.21.15.86:0/1033298570 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d4cd8c9a70 con 0x55d4cd9e1cd0 2024-04-21T08:49:10.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.587+0000 7ff05081d640 1 --2- 172.21.15.86:0/1033298570 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d4cd9e1cd0 0x55d4cd9e40c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:10.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.587+0000 7ff05081d640 1 --2- 172.21.15.86:0/1033298570 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d4cd9e1cd0 0x55d4cd9e40c0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7260d6981dcf8d81 server_cookie=261ed8874db8566c in_seq=0 out_seq=0 2024-04-21T08:49:10.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.587+0000 7ff02bfff640 10 client.19860 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:10.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.588+0000 7ff02bfff640 1 -- 172.21.15.86:0/1033298570 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55d4cd8c9a70 con 0x55d4cd9e1cd0 2024-04-21T08:49:10.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.588+0000 7ff02bfff640 10 client.19860 handle_command_reply: tid=0 2024-04-21T08:49:10.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.588+0000 7ff05567c740 1 client.19860 shutdown 2024-04-21T08:49:10.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.588+0000 7ff05567c740 1 -- 172.21.15.86:0/1033298570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4cd966a80 msgr2=0x55d4cd96f5d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:10.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.588+0000 7ff05567c740 1 --2- 172.21.15.86:0/1033298570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4cd966a80 0x55d4cd96f5d0 secure :-1 s=READY pgs=1879 cs=0 l=1 rev1=1 crypto rx=0x7ff04400f7c0 tx=0x7ff044010b60 comp rx=0 tx=0).stop 2024-04-21T08:49:10.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.588+0000 7ff05567c740 1 -- 172.21.15.86:0/1033298570 shutdown_connections 2024-04-21T08:49:10.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.588+0000 7ff05567c740 1 --2- 172.21.15.86:0/1033298570 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d4cd9e1cd0 0x55d4cd9e40c0 crc :-1 s=READY pgs=306 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.594 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.588+0000 7ff05567c740 1 --2- 172.21.15.86:0/1033298570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d4cd966a80 0x55d4cd96f5d0 unknown :-1 s=CLOSED pgs=1879 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.588+0000 7ff05567c740 1 --2- 172.21.15.86:0/1033298570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d4cd95dea0 0x55d4cd96f090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.588+0000 7ff05567c740 1 --2- 172.21.15.86:0/1033298570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d4cd95d580 0x55d4cd96eb50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.588+0000 7ff02bfff640 0 client.19860 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:10.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.588+0000 7ff05567c740 1 -- 172.21.15.86:0/1033298570 >> 172.21.15.86:0/1033298570 conn(0x55d4cd957cb0 msgr2=0x55d4cd9639c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:10.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.588+0000 7ff05567c740 1 -- 172.21.15.86:0/1033298570 shutdown_connections 2024-04-21T08:49:10.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.588+0000 7ff05567c740 1 -- 172.21.15.86:0/1033298570 wait complete. 2024-04-21T08:49:10.595 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.588+0000 7ff05567c740 20 client.19860 trim_cache size 0 max 16384 2024-04-21T08:49:10.595 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:49:10.595 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:49:10.595 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-21T08:49:10.596 INFO:teuthology.orchestra.run.smithi086.stdout: "ff05a4fe-12c8-4fc1-9d0a-792c4b66ab6c": { 2024-04-21T08:49:10.596 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-21T08:49:10.596 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "ff05a4fe-12c8-4fc1-9d0a-792c4b66ab6c", 2024-04-21T08:49:10.596 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-21T08:49:10.596 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:49:10.596 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:49:10.596 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:49:10.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.588+0000 7ff052a81640 1 -- 172.21.15.86:0/2669655816 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff03c0a3d20 msgr2=0x7ff03c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:10.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.588+0000 7ff052a81640 1 --2- 172.21.15.86:0/2669655816 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff03c0a3d20 0x7ff03c0a61e0 secure :-1 s=READY pgs=311 cs=0 l=1 rev1=1 crypto rx=0x7ff044008c60 tx=0x7ff04400d710 comp rx=0 tx=0).stop 2024-04-21T08:49:10.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.588+0000 7ff052a81640 1 -- 172.21.15.86:0/2669655816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff04c101080 msgr2=0x7ff04c197670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:10.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.588+0000 7ff052a81640 1 --2- 172.21.15.86:0/2669655816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff04c101080 0x7ff04c197670 secure :-1 s=READY pgs=1875 cs=0 l=1 rev1=1 crypto rx=0x7ff040009900 tx=0x7ff040009dc0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.589+0000 7ff052a81640 1 -- 172.21.15.86:0/2669655816 shutdown_connections 2024-04-21T08:49:10.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.589+0000 7ff052a81640 1 --2- 172.21.15.86:0/2669655816 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff03c0a3d20 0x7ff03c0a61e0 unknown :-1 s=CLOSED pgs=311 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.589+0000 7ff052a81640 1 --2- 172.21.15.86:0/2669655816 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff04c19bf30 0x7ff04c19c3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.589+0000 7ff052a81640 1 --2- 172.21.15.86:0/2669655816 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff04c106150 0x7ff04c197bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.589+0000 7ff052a81640 1 --2- 172.21.15.86:0/2669655816 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff04c101080 0x7ff04c197670 unknown :-1 s=CLOSED pgs=1875 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:10.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.589+0000 7ff052a81640 1 -- 172.21.15.86:0/2669655816 >> 172.21.15.86:0/2669655816 conn(0x7ff04c078000 msgr2=0x7ff04c0ff030 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:10.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.589+0000 7ff052a81640 1 -- 172.21.15.86:0/2669655816 shutdown_connections 2024-04-21T08:49:10.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:10.589+0000 7ff052a81640 1 -- 172.21.15.86:0/2669655816 wait complete. 2024-04-21T08:49:10.598 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:10 smithi086 ceph-mon[27939]: pgmap v323: 129 pgs: 129 active+clean; 2.7 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 102 KiB/s wr, 38 op/s 2024-04-21T08:49:10.599 INFO:tasks.cephfs.filesystem:scrub status for tag:ff05a4fe-12c8-4fc1-9d0a-792c4b66ab6c - {'path': '/', 'tag': 'ff05a4fe-12c8-4fc1-9d0a-792c4b66ab6c', 'options': 'recursive,force'} 2024-04-21T08:49:10.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:10 smithi088 ceph-mon[31259]: pgmap v323: 129 pgs: 129 active+clean; 2.7 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 102 KiB/s wr, 38 op/s 2024-04-21T08:49:11.481 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.479+0000 7fdb94915640 1 -- 172.21.15.86:0/3799402532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb9006a780 msgr2=0x7fdb9006abe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.479+0000 7fdb94915640 1 --2- 172.21.15.86:0/3799402532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb9006a780 0x7fdb9006abe0 secure :-1 s=READY pgs=1880 cs=0 l=1 rev1=1 crypto rx=0x7fdb84009900 tx=0x7fdb8402bcd0 comp rx=0 tx=0).stop 2024-04-21T08:49:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.479+0000 7fdb94915640 1 -- 172.21.15.86:0/3799402532 shutdown_connections 2024-04-21T08:49:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.479+0000 7fdb94915640 1 --2- 172.21.15.86:0/3799402532 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb9006b120 0x7fdb9010edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.479+0000 7fdb94915640 1 --2- 172.21.15.86:0/3799402532 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb9006a780 0x7fdb9006abe0 unknown :-1 s=CLOSED pgs=1880 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.479+0000 7fdb94915640 1 --2- 172.21.15.86:0/3799402532 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb901080e0 0x7fdb901084c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.479+0000 7fdb94915640 1 -- 172.21.15.86:0/3799402532 >> 172.21.15.86:0/3799402532 conn(0x7fdb90076c50 msgr2=0x7fdb90077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.479+0000 7fdb94915640 1 -- 172.21.15.86:0/3799402532 shutdown_connections 2024-04-21T08:49:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.479+0000 7fdb94915640 1 -- 172.21.15.86:0/3799402532 wait complete. 2024-04-21T08:49:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.480+0000 7fdb94915640 1 Processor -- start 2024-04-21T08:49:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.480+0000 7fdb94915640 1 -- start start 2024-04-21T08:49:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.480+0000 7fdb94915640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb9006a780 0x7fdb9019a720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:11.482 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.480+0000 7fdb94915640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb9006b120 0x7fdb9019ac60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.480+0000 7fdb94915640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb901080e0 0x7fdb9019efe0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.480+0000 7fdb94915640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdb90112180 con 0x7fdb9006b120 2024-04-21T08:49:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.480+0000 7fdb94915640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdb90112000 con 0x7fdb901080e0 2024-04-21T08:49:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.480+0000 7fdb94915640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdb90112300 con 0x7fdb9006a780 2024-04-21T08:49:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.480+0000 7fdb8dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb9006b120 0x7fdb9019ac60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.480+0000 7fdb8dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb9006b120 0x7fdb9019ac60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32968/0 (socket says 172.21.15.86:32968) 2024-04-21T08:49:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.480+0000 7fdb8dd9b640 1 -- 172.21.15.86:0/3282005632 learned_addr learned my addr 172.21.15.86:0/3282005632 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.480+0000 7fdb8e59c640 1 --2- 172.21.15.86:0/3282005632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb9006a780 0x7fdb9019a720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.480+0000 7fdb8ed9d640 1 --2- 172.21.15.86:0/3282005632 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb901080e0 0x7fdb9019efe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.480+0000 7fdb8dd9b640 1 -- 172.21.15.86:0/3282005632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb9006a780 msgr2=0x7fdb9019a720 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.480+0000 7fdb8dd9b640 1 --2- 172.21.15.86:0/3282005632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb9006a780 0x7fdb9019a720 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.480+0000 7fdb8dd9b640 1 -- 172.21.15.86:0/3282005632 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb901080e0 msgr2=0x7fdb9019efe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.480+0000 7fdb8dd9b640 1 --2- 172.21.15.86:0/3282005632 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb901080e0 0x7fdb9019efe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:11.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.480+0000 7fdb8dd9b640 1 -- 172.21.15.86:0/3282005632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb9019f710 con 0x7fdb9006b120 2024-04-21T08:49:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.480+0000 7fdb8ed9d640 1 --2- 172.21.15.86:0/3282005632 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb901080e0 0x7fdb9019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.480+0000 7fdb8dd9b640 1 --2- 172.21.15.86:0/3282005632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb9006b120 0x7fdb9019ac60 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fdb84005940 tx=0x7fdb8402d910 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.481+0000 7fdb7f7fe640 1 -- 172.21.15.86:0/3282005632 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb84043020 con 0x7fdb9006b120 2024-04-21T08:49:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.481+0000 7fdb7f7fe640 1 -- 172.21.15.86:0/3282005632 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb8402dd10 con 0x7fdb9006b120 2024-04-21T08:49:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.481+0000 7fdb94915640 1 -- 172.21.15.86:0/3282005632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb9019f9a0 con 0x7fdb9006b120 2024-04-21T08:49:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.481+0000 7fdb7f7fe640 1 -- 172.21.15.86:0/3282005632 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb84003b80 con 0x7fdb9006b120 2024-04-21T08:49:11.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.481+0000 7fdb94915640 1 -- 172.21.15.86:0/3282005632 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb901a7290 con 0x7fdb9006b120 2024-04-21T08:49:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.483+0000 7fdb7f7fe640 1 -- 172.21.15.86:0/3282005632 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdb84046020 con 0x7fdb9006b120 2024-04-21T08:49:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.483+0000 7fdb975e7740 1 Processor -- start 2024-04-21T08:49:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.483+0000 7fdb7f7fe640 1 --2- 172.21.15.86:0/3282005632 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdb780a3dc0 0x7fdb780a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.484+0000 7fdb7f7fe640 1 -- 172.21.15.86:0/3282005632 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7fdb840f2dc0 con 0x7fdb9006b120 2024-04-21T08:49:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.484+0000 7fdb8ed9d640 1 --2- 172.21.15.86:0/3282005632 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdb780a3dc0 0x7fdb780a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.484+0000 7fdb975e7740 1 -- start start 2024-04-21T08:49:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.484+0000 7fdb975e7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2217038d0 0x55c221702620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.484+0000 7fdb975e7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c221702b60 0x55c221702fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.484+0000 7fdb975e7740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c22170c0e0 0x55c22170e4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.484+0000 7fdb975e7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c2216678d0 con 0x55c2217038d0 2024-04-21T08:49:11.491 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.484+0000 7fdb975e7740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c221604e80 con 0x55c22170c0e0 2024-04-21T08:49:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.484+0000 7fdb975e7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c221663890 con 0x55c221702b60 2024-04-21T08:49:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.484+0000 7fdb8e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2217038d0 0x55c221702620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.484+0000 7fdb8e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2217038d0 0x55c221702620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32976/0 (socket says 172.21.15.86:32976) 2024-04-21T08:49:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.484+0000 7fdb8e59c640 1 -- 172.21.15.86:0/584755871 learned_addr learned my addr 172.21.15.86:0/584755871 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.484+0000 7fdb8ed9d640 1 --2- 172.21.15.86:0/3282005632 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdb780a3dc0 0x7fdb780a6280 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdb88005fd0 tx=0x7fdb880082b0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.484+0000 7fdb8dd9b640 1 --2- 172.21.15.86:0/584755871 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c221702b60 0x55c221702fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.484+0000 7fdb8ed9d640 1 --2- 172.21.15.86:0/584755871 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c22170c0e0 0x55c22170e4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.484+0000 7fdb8e59c640 1 -- 172.21.15.86:0/584755871 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c221702b60 msgr2=0x55c221702fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.484+0000 7fdb8e59c640 1 --2- 172.21.15.86:0/584755871 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c221702b60 0x55c221702fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.484+0000 7fdb8e59c640 1 -- 172.21.15.86:0/584755871 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c22170c0e0 msgr2=0x55c22170e4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.484+0000 7fdb8e59c640 1 --2- 172.21.15.86:0/584755871 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c22170c0e0 0x55c22170e4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.484+0000 7fdb8e59c640 1 -- 172.21.15.86:0/584755871 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c221626320 con 0x55c2217038d0 2024-04-21T08:49:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.484+0000 7fdb8dd9b640 1 --2- 172.21.15.86:0/584755871 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c221702b60 0x55c221702fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.485+0000 7fdb8e59c640 1 --2- 172.21.15.86:0/584755871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2217038d0 0x55c221702620 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fdb8000cc60 tx=0x7fdb8000e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=44b8d2807125e7bc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.485+0000 7fdb7cff9640 1 -- 172.21.15.86:0/584755871 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb80014070 con 0x55c2217038d0 2024-04-21T08:49:11.492 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.485+0000 7fdb7cff9640 1 -- 172.21.15.86:0/584755871 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb800047a0 con 0x55c2217038d0 2024-04-21T08:49:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.485+0000 7fdb7cff9640 1 -- 172.21.15.86:0/584755871 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb80010690 con 0x55c2217038d0 2024-04-21T08:49:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.485+0000 7fdb975e7740 1 -- 172.21.15.86:0/584755871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2217038d0 msgr2=0x55c221702620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.485+0000 7fdb975e7740 1 --2- 172.21.15.86:0/584755871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2217038d0 0x55c221702620 secure :-1 s=READY pgs=1882 cs=0 l=1 rev1=1 crypto rx=0x7fdb8000cc60 tx=0x7fdb8000e720 comp rx=0 tx=0).stop 2024-04-21T08:49:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.486+0000 7fdb975e7740 1 -- 172.21.15.86:0/584755871 shutdown_connections 2024-04-21T08:49:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.486+0000 7fdb975e7740 1 --2- 172.21.15.86:0/584755871 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c22170c0e0 0x55c22170e4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.486+0000 7fdb975e7740 1 --2- 172.21.15.86:0/584755871 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c221702b60 0x55c221702fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.486+0000 7fdb975e7740 1 --2- 172.21.15.86:0/584755871 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2217038d0 0x55c221702620 unknown :-1 s=CLOSED pgs=1882 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.486+0000 7fdb975e7740 1 -- 172.21.15.86:0/584755871 >> 172.21.15.86:0/584755871 conn(0x55c2216c9c90 msgr2=0x55c2216fd850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.486+0000 7fdb975e7740 1 -- 172.21.15.86:0/584755871 shutdown_connections 2024-04-21T08:49:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.486+0000 7fdb975e7740 1 -- 172.21.15.86:0/584755871 wait complete. 2024-04-21T08:49:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.486+0000 7fdb975e7740 1 -- start start 2024-04-21T08:49:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.486+0000 7fdb975e7740 1 Processor -- start 2024-04-21T08:49:11.493 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.486+0000 7fdb975e7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c221702b60 0x55c22170b9d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.486+0000 7fdb975e7740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2217038d0 0x55c221708dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.486+0000 7fdb975e7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c22170c0e0 0x55c221709310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.486+0000 7fdb975e7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c22170ee10 con 0x55c221702b60 2024-04-21T08:49:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.486+0000 7fdb975e7740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c2215efeb0 con 0x55c2217038d0 2024-04-21T08:49:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.486+0000 7fdb975e7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c22170ef90 con 0x55c22170c0e0 2024-04-21T08:49:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.486+0000 7fdb975e7740 10 client.0 _ll_register_callbacks cb 0x55c22161cb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.486+0000 7fdb8e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c221702b60 0x55c22170b9d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.486+0000 7fdb8e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c221702b60 0x55c22170b9d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32978/0 (socket says 172.21.15.86:32978) 2024-04-21T08:49:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.486+0000 7fdb8e59c640 1 -- 172.21.15.86:0/1195690914 learned_addr learned my addr 172.21.15.86:0/1195690914 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.486+0000 7fdb8dd9b640 1 --2- 172.21.15.86:0/1195690914 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2217038d0 0x55c221708dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.487+0000 7fdb8e59c640 1 -- 172.21.15.86:0/1195690914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c22170c0e0 msgr2=0x55c221709310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.487+0000 7fdb8ed9d640 1 --2- 172.21.15.86:0/1195690914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c22170c0e0 0x55c221709310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.487+0000 7fdb8e59c640 1 --2- 172.21.15.86:0/1195690914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c22170c0e0 0x55c221709310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.487+0000 7fdb8e59c640 1 -- 172.21.15.86:0/1195690914 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2217038d0 msgr2=0x55c221708dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:11.494 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.487+0000 7fdb8e59c640 1 --2- 172.21.15.86:0/1195690914 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2217038d0 0x55c221708dd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.487+0000 7fdb8e59c640 1 -- 172.21.15.86:0/1195690914 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c22137a210 con 0x55c221702b60 2024-04-21T08:49:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.487+0000 7fdb8dd9b640 1 --2- 172.21.15.86:0/1195690914 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2217038d0 0x55c221708dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.487+0000 7fdb8e59c640 1 --2- 172.21.15.86:0/1195690914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c221702b60 0x55c22170b9d0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7fdb80000c00 tx=0x7fdb80004f50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=14373e129cbbbf3c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.487+0000 7fdb75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.487+0000 7fdb75ffb640 1 -- 172.21.15.86:0/1195690914 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb80014070 con 0x55c221702b60 2024-04-21T08:49:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.487+0000 7fdb75ffb640 1 -- 172.21.15.86:0/1195690914 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7fdb80005350 con 0x55c221702b60 2024-04-21T08:49:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.487+0000 7fdb975e7740 1 -- 172.21.15.86:0/1195690914 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb800047a0 con 0x55c221702b60 2024-04-21T08:49:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.487+0000 7fdb75ffb640 1 -- 172.21.15.86:0/1195690914 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdb8002d050 con 0x55c221702b60 2024-04-21T08:49:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.487+0000 7fdb975e7740 10 client.19872 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.487+0000 7fdb975e7740 1 -- 172.21.15.86:0/1195690914 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c221446620 con 0x55c221702b60 2024-04-21T08:49:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.487+0000 7fdb75ffb640 1 -- 172.21.15.86:0/1195690914 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb800047a0 con 0x55c221702b60 2024-04-21T08:49:11.495 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.487+0000 7fdb75ffb640 1 -- 172.21.15.86:0/1195690914 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdb80022410 con 0x55c221702b60 2024-04-21T08:49:11.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.488+0000 7fdb975e7740 10 client.19872 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:11.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.488+0000 7fdb975e7740 10 client.19872 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:11.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.488+0000 7fdb975e7740 20 client.19872 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:11.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.488+0000 7fdb975e7740 1 --2- 172.21.15.86:0/1195690914 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c22171bfd0 0x55c22171e3c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:11.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.488+0000 7fdb975e7740 4 client.19872 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:11.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.488+0000 7fdb975e7740 1 -- 172.21.15.86:0/1195690914 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c22167b880 con 0x55c22171bfd0 2024-04-21T08:49:11.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.488+0000 7fdb8dd9b640 1 --2- 172.21.15.86:0/1195690914 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c22171bfd0 0x55c22171e3c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:11.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.488+0000 7fdb75ffb640 10 client.19872 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:11.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.488+0000 7fdb8dd9b640 1 --2- 172.21.15.86:0/1195690914 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c22171bfd0 0x55c22171e3c0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ca21f599ce82afb server_cookie=efa6c13a9b18925e in_seq=0 out_seq=0 2024-04-21T08:49:11.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.489+0000 7fdb75ffb640 1 -- 172.21.15.86:0/1195690914 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c22167b880 con 0x55c22171bfd0 2024-04-21T08:49:11.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.489+0000 7fdb75ffb640 10 client.19872 handle_command_reply: tid=0 2024-04-21T08:49:11.496 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.489+0000 7fdb975e7740 1 client.19872 shutdown 2024-04-21T08:49:11.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.489+0000 7fdb975e7740 1 -- 172.21.15.86:0/1195690914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c221702b60 msgr2=0x55c22170b9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:11.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.489+0000 7fdb975e7740 1 --2- 172.21.15.86:0/1195690914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c221702b60 0x55c22170b9d0 secure :-1 s=READY pgs=1883 cs=0 l=1 rev1=1 crypto rx=0x7fdb80000c00 tx=0x7fdb80004f50 comp rx=0 tx=0).stop 2024-04-21T08:49:11.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.489+0000 7fdb975e7740 1 -- 172.21.15.86:0/1195690914 shutdown_connections 2024-04-21T08:49:11.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.489+0000 7fdb975e7740 1 --2- 172.21.15.86:0/1195690914 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c22171bfd0 0x55c22171e3c0 crc :-1 s=READY pgs=307 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:11.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.489+0000 7fdb975e7740 1 --2- 172.21.15.86:0/1195690914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c22170c0e0 0x55c221709310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:11.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.489+0000 7fdb975e7740 1 --2- 172.21.15.86:0/1195690914 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2217038d0 0x55c221708dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:11.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.489+0000 7fdb75ffb640 0 client.19872 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:11.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.489+0000 7fdb975e7740 1 --2- 172.21.15.86:0/1195690914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c221702b60 0x55c22170b9d0 unknown :-1 s=CLOSED pgs=1883 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:11.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.489+0000 7fdb975e7740 1 -- 172.21.15.86:0/1195690914 >> 172.21.15.86:0/1195690914 conn(0x55c2216c9c90 msgr2=0x55c2216ff5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:11.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.489+0000 7fdb975e7740 1 -- 172.21.15.86:0/1195690914 shutdown_connections 2024-04-21T08:49:11.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.489+0000 7fdb975e7740 1 -- 172.21.15.86:0/1195690914 wait complete. 2024-04-21T08:49:11.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.489+0000 7fdb975e7740 20 client.19872 trim_cache size 0 max 16384 2024-04-21T08:49:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.504+0000 7fdb975e7740 1 Processor -- start 2024-04-21T08:49:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.504+0000 7fdb975e7740 1 -- start start 2024-04-21T08:49:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.504+0000 7fdb975e7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2216c9c90 0x55c221717d10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.504+0000 7fdb975e7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c221702b60 0x55c221708310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.504+0000 7fdb975e7740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2217038d0 0x55c22170b500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.504+0000 7fdb975e7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdb901b1460 con 0x55c221702b60 2024-04-21T08:49:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.504+0000 7fdb975e7740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdb901b2fa0 con 0x55c2217038d0 2024-04-21T08:49:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.504+0000 7fdb975e7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdb901b1b30 con 0x55c2216c9c90 2024-04-21T08:49:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.504+0000 7fdb8dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c221702b60 0x55c221708310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.504+0000 7fdb8dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c221702b60 0x55c221708310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:32992/0 (socket says 172.21.15.86:32992) 2024-04-21T08:49:11.510 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.504+0000 7fdb8dd9b640 1 -- 172.21.15.86:0/3212151460 learned_addr learned my addr 172.21.15.86:0/3212151460 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:11.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.505+0000 7fdb8e59c640 1 --2- 172.21.15.86:0/3212151460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2216c9c90 0x55c221717d10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:11.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.505+0000 7fdb8dd9b640 1 -- 172.21.15.86:0/3212151460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2216c9c90 msgr2=0x55c221717d10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:11.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.505+0000 7fdb8ed9d640 1 --2- 172.21.15.86:0/3212151460 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2217038d0 0x55c22170b500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:11.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.505+0000 7fdb8dd9b640 1 --2- 172.21.15.86:0/3212151460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2216c9c90 0x55c221717d10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:11.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.505+0000 7fdb8dd9b640 1 -- 172.21.15.86:0/3212151460 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2217038d0 msgr2=0x55c22170b500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:11.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.505+0000 7fdb8dd9b640 1 --2- 172.21.15.86:0/3212151460 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2217038d0 0x55c22170b500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:11.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.505+0000 7fdb8dd9b640 1 -- 172.21.15.86:0/3212151460 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c22170ef80 con 0x55c221702b60 2024-04-21T08:49:11.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.505+0000 7fdb8e59c640 1 --2- 172.21.15.86:0/3212151460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2216c9c90 0x55c221717d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:11.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.505+0000 7fdb8ed9d640 1 --2- 172.21.15.86:0/3212151460 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2217038d0 0x55c22170b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:11.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.505+0000 7fdb8dd9b640 1 --2- 172.21.15.86:0/3212151460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c221702b60 0x55c221708310 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fdb840f02b0 tx=0x7fdb8404a040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4c0acd62823a1a92 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:11.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.505+0000 7fdb77fff640 1 -- 172.21.15.86:0/3212151460 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb840bcb40 con 0x55c221702b60 2024-04-21T08:49:11.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.505+0000 7fdb77fff640 1 -- 172.21.15.86:0/3212151460 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb840bcda0 con 0x55c221702b60 2024-04-21T08:49:11.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.505+0000 7fdb77fff640 1 -- 172.21.15.86:0/3212151460 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb840bb110 con 0x55c221702b60 2024-04-21T08:49:11.511 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.505+0000 7fdb975e7740 1 -- 172.21.15.86:0/3212151460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c221702b60 msgr2=0x55c221708310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.505+0000 7fdb975e7740 1 --2- 172.21.15.86:0/3212151460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c221702b60 0x55c221708310 secure :-1 s=READY pgs=1884 cs=0 l=1 rev1=1 crypto rx=0x7fdb840f02b0 tx=0x7fdb8404a040 comp rx=0 tx=0).stop 2024-04-21T08:49:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.506+0000 7fdb975e7740 1 -- 172.21.15.86:0/3212151460 shutdown_connections 2024-04-21T08:49:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.506+0000 7fdb975e7740 1 --2- 172.21.15.86:0/3212151460 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2217038d0 0x55c22170b500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.506+0000 7fdb975e7740 1 --2- 172.21.15.86:0/3212151460 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c221702b60 0x55c221708310 unknown :-1 s=CLOSED pgs=1884 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.506+0000 7fdb975e7740 1 --2- 172.21.15.86:0/3212151460 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c2216c9c90 0x55c221717d10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.506+0000 7fdb975e7740 1 -- 172.21.15.86:0/3212151460 >> 172.21.15.86:0/3212151460 conn(0x55c221709470 msgr2=0x55c2217076a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.506+0000 7fdb975e7740 1 -- 172.21.15.86:0/3212151460 shutdown_connections 2024-04-21T08:49:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.506+0000 7fdb975e7740 1 -- 172.21.15.86:0/3212151460 wait complete. 2024-04-21T08:49:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.506+0000 7fdb975e7740 1 -- start start 2024-04-21T08:49:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.506+0000 7fdb975e7740 1 Processor -- start 2024-04-21T08:49:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.506+0000 7fdb975e7740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2216c9c90 0x55c221705a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.506+0000 7fdb975e7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c221702b60 0x55c221705f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.506+0000 7fdb975e7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2217038d0 0x55c221706480 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.506+0000 7fdb975e7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdb901b28d0 con 0x55c2217038d0 2024-04-21T08:49:11.512 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.506+0000 7fdb975e7740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdb901b2200 con 0x55c2216c9c90 2024-04-21T08:49:11.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.506+0000 7fdb975e7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdb901af460 con 0x55c221702b60 2024-04-21T08:49:11.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.506+0000 7fdb975e7740 10 client.0 _ll_register_callbacks cb 0x7fdb8000cab0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:11.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.506+0000 7fdb8ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2217038d0 0x55c221706480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:11.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.506+0000 7fdb8ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2217038d0 0x55c221706480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33000/0 (socket says 172.21.15.86:33000) 2024-04-21T08:49:11.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.506+0000 7fdb8ed9d640 1 -- 172.21.15.86:0/3828665352 learned_addr learned my addr 172.21.15.86:0/3828665352 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:11.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.506+0000 7fdb8e59c640 1 --2- 172.21.15.86:0/3828665352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2216c9c90 0x55c221705a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:11.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.506+0000 7fdb8ed9d640 1 -- 172.21.15.86:0/3828665352 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c221702b60 msgr2=0x55c221705f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:11.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.506+0000 7fdb8ed9d640 1 --2- 172.21.15.86:0/3828665352 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c221702b60 0x55c221705f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:11.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.506+0000 7fdb8ed9d640 1 -- 172.21.15.86:0/3828665352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2216c9c90 msgr2=0x55c221705a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:11.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.506+0000 7fdb8ed9d640 1 --2- 172.21.15.86:0/3828665352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2216c9c90 0x55c221705a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:11.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.507+0000 7fdb8ed9d640 1 -- 172.21.15.86:0/3828665352 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c2216f8f30 con 0x55c2217038d0 2024-04-21T08:49:11.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.507+0000 7fdb8e59c640 1 --2- 172.21.15.86:0/3828665352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2216c9c90 0x55c221705a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:11.513 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.507+0000 7fdb8ed9d640 1 --2- 172.21.15.86:0/3828665352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2217038d0 0x55c221706480 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7fdb8800dba0 tx=0x7fdb88004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ddedbfe34fee5665 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:11.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.507+0000 7fdb75ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:11.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.507+0000 7fdb75ffb640 1 -- 172.21.15.86:0/3828665352 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb8800fbb0 con 0x55c2217038d0 2024-04-21T08:49:11.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.507+0000 7fdb75ffb640 1 -- 172.21.15.86:0/3828665352 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7fdb8800fd50 con 0x55c2217038d0 2024-04-21T08:49:11.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.507+0000 7fdb975e7740 1 -- 172.21.15.86:0/3828665352 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdb840bcda0 con 0x55c2217038d0 2024-04-21T08:49:11.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.507+0000 7fdb75ffb640 1 -- 172.21.15.86:0/3828665352 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdb88025440 con 0x55c2217038d0 2024-04-21T08:49:11.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.507+0000 7fdb975e7740 10 client.19878 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:11.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.507+0000 7fdb975e7740 1 -- 172.21.15.86:0/3828665352 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c221701e30 con 0x55c2217038d0 2024-04-21T08:49:11.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.507+0000 7fdb75ffb640 1 -- 172.21.15.86:0/3828665352 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb840bcda0 con 0x55c2217038d0 2024-04-21T08:49:11.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.507+0000 7fdb75ffb640 1 -- 172.21.15.86:0/3828665352 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdb88023070 con 0x55c2217038d0 2024-04-21T08:49:11.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.507+0000 7fdb975e7740 10 client.19878 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:11.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.507+0000 7fdb975e7740 10 client.19878 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:11.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.507+0000 7fdb975e7740 20 client.19878 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:11.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.507+0000 7fdb975e7740 1 --2- 172.21.15.86:0/3828665352 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c22170c0e0 0x55c221784930 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:11.514 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.507+0000 7fdb975e7740 4 client.19878 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:49:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.507+0000 7fdb975e7740 1 -- 172.21.15.86:0/3828665352 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55c2216073d0 con 0x55c22170c0e0 2024-04-21T08:49:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.508+0000 7fdb8e59c640 1 --2- 172.21.15.86:0/3828665352 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c22170c0e0 0x55c221784930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.508+0000 7fdb8e59c640 1 --2- 172.21.15.86:0/3828665352 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c22170c0e0 0x55c221784930 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8f63d2ec07cc1ad8 server_cookie=4805c435c5136870 in_seq=0 out_seq=0 2024-04-21T08:49:11.515 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.508+0000 7fdb75ffb640 10 client.19878 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.522+0000 7fdb75ffb640 1 -- 172.21.15.86:0/3828665352 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55c2216073d0 con 0x55c22170c0e0 2024-04-21T08:49:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.522+0000 7fdb75ffb640 10 client.19878 handle_command_reply: tid=0 2024-04-21T08:49:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.522+0000 7fdb975e7740 1 client.19878 shutdown 2024-04-21T08:49:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.522+0000 7fdb975e7740 1 -- 172.21.15.86:0/3828665352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2217038d0 msgr2=0x55c221706480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.522+0000 7fdb975e7740 1 --2- 172.21.15.86:0/3828665352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2217038d0 0x55c221706480 secure :-1 s=READY pgs=1885 cs=0 l=1 rev1=1 crypto rx=0x7fdb8800dba0 tx=0x7fdb88004a40 comp rx=0 tx=0).stop 2024-04-21T08:49:11.523 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.522+0000 7fdb975e7740 1 -- 172.21.15.86:0/3828665352 shutdown_connections 2024-04-21T08:49:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.522+0000 7fdb975e7740 1 --2- 172.21.15.86:0/3828665352 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55c22170c0e0 0x55c221784930 crc :-1 s=READY pgs=308 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.522+0000 7fdb975e7740 1 --2- 172.21.15.86:0/3828665352 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c2217038d0 0x55c221706480 unknown :-1 s=CLOSED pgs=1885 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.522+0000 7fdb975e7740 1 --2- 172.21.15.86:0/3828665352 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c221702b60 0x55c221705f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.522+0000 7fdb75ffb640 0 client.19878 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.522+0000 7fdb975e7740 1 --2- 172.21.15.86:0/3828665352 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c2216c9c90 0x55c221705a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.522+0000 7fdb975e7740 1 -- 172.21.15.86:0/3828665352 >> 172.21.15.86:0/3828665352 conn(0x55c221709470 msgr2=0x55c2216fc040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.522+0000 7fdb975e7740 1 -- 172.21.15.86:0/3828665352 shutdown_connections 2024-04-21T08:49:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.522+0000 7fdb975e7740 1 -- 172.21.15.86:0/3828665352 wait complete. 2024-04-21T08:49:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.523+0000 7fdb975e7740 20 client.19878 trim_cache size 0 max 16384 2024-04-21T08:49:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:49:11.524 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:49:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:49:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:49:11.525 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.523+0000 7fdb94915640 1 -- 172.21.15.86:0/3282005632 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdb780a3dc0 msgr2=0x7fdb780a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.523+0000 7fdb94915640 1 --2- 172.21.15.86:0/3282005632 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdb780a3dc0 0x7fdb780a6280 secure :-1 s=READY pgs=312 cs=0 l=1 rev1=1 crypto rx=0x7fdb88005fd0 tx=0x7fdb880082b0 comp rx=0 tx=0).stop 2024-04-21T08:49:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.523+0000 7fdb94915640 1 -- 172.21.15.86:0/3282005632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb9006b120 msgr2=0x7fdb9019ac60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.523+0000 7fdb94915640 1 --2- 172.21.15.86:0/3282005632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb9006b120 0x7fdb9019ac60 secure :-1 s=READY pgs=1881 cs=0 l=1 rev1=1 crypto rx=0x7fdb84005940 tx=0x7fdb8402d910 comp rx=0 tx=0).stop 2024-04-21T08:49:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.523+0000 7fdb94915640 1 -- 172.21.15.86:0/3282005632 shutdown_connections 2024-04-21T08:49:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.523+0000 7fdb94915640 1 --2- 172.21.15.86:0/3282005632 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdb780a3dc0 0x7fdb780a6280 unknown :-1 s=CLOSED pgs=312 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.523+0000 7fdb94915640 1 --2- 172.21.15.86:0/3282005632 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb901080e0 0x7fdb9019efe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.523+0000 7fdb94915640 1 --2- 172.21.15.86:0/3282005632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb9006b120 0x7fdb9019ac60 unknown :-1 s=CLOSED pgs=1881 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.523+0000 7fdb94915640 1 --2- 172.21.15.86:0/3282005632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb9006a780 0x7fdb9019a720 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.523+0000 7fdb94915640 1 -- 172.21.15.86:0/3282005632 >> 172.21.15.86:0/3282005632 conn(0x7fdb90076c50 msgr2=0x7fdb9010d0f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.523+0000 7fdb94915640 1 -- 172.21.15.86:0/3282005632 shutdown_connections 2024-04-21T08:49:11.526 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:11.523+0000 7fdb94915640 1 -- 172.21.15.86:0/3282005632 wait complete. 2024-04-21T08:49:11.866 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:11 smithi086 ceph-mon[27939]: scrub queued for path: / 2024-04-21T08:49:11.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:11 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:11.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:11 smithi086 ceph-mon[27939]: scrub summary: active paths [/] 2024-04-21T08:49:11.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:11 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:11.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:11 smithi086 ceph-mon[27939]: osdmap e104: 12 total, 12 up, 12 in 2024-04-21T08:49:11.867 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:11 smithi086 ceph-mon[27939]: pgmap v325: 129 pgs: 129 active+clean; 2.7 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 9 op/s 2024-04-21T08:49:11.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:11 smithi088 ceph-mon[31259]: scrub queued for path: / 2024-04-21T08:49:11.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:11 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:11.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:11 smithi088 ceph-mon[31259]: scrub summary: active paths [/] 2024-04-21T08:49:11.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:11 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:11.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:11 smithi088 ceph-mon[31259]: osdmap e104: 12 total, 12 up, 12 in 2024-04-21T08:49:11.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:11 smithi088 ceph-mon[31259]: pgmap v325: 129 pgs: 129 active+clean; 2.7 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 9 op/s 2024-04-21T08:49:11.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:11 smithi179 ceph-mon[31829]: scrub queued for path: / 2024-04-21T08:49:11.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:11 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:11.978 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:11 smithi179 ceph-mon[31829]: scrub summary: active paths [/] 2024-04-21T08:49:11.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:11 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:11.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:11 smithi179 ceph-mon[31829]: osdmap e104: 12 total, 12 up, 12 in 2024-04-21T08:49:11.979 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:11 smithi179 ceph-mon[31829]: pgmap v325: 129 pgs: 129 active+clean; 2.7 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 32 KiB/s rd, 29 KiB/s wr, 9 op/s 2024-04-21T08:49:14.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:13 smithi088 ceph-mon[31259]: pgmap v326: 129 pgs: 129 active+clean; 2.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 64 KiB/s wr, 25 op/s 2024-04-21T08:49:14.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:13 smithi179 ceph-mon[31829]: pgmap v326: 129 pgs: 129 active+clean; 2.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 64 KiB/s wr, 25 op/s 2024-04-21T08:49:14.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:13 smithi086 ceph-mon[27939]: pgmap v326: 129 pgs: 129 active+clean; 2.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 67 KiB/s rd, 64 KiB/s wr, 25 op/s 2024-04-21T08:49:15.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:14 smithi088 ceph-mon[31259]: scrub summary: idle 2024-04-21T08:49:15.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:14 smithi179 ceph-mon[31829]: scrub summary: idle 2024-04-21T08:49:15.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:14 smithi086 ceph-mon[27939]: scrub summary: idle 2024-04-21T08:49:16.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:15 smithi088 ceph-mon[31259]: pgmap v327: 129 pgs: 129 active+clean; 2.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 53 KiB/s wr, 21 op/s 2024-04-21T08:49:16.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:15 smithi179 ceph-mon[31829]: pgmap v327: 129 pgs: 129 active+clean; 2.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 53 KiB/s wr, 21 op/s 2024-04-21T08:49:16.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:15 smithi086 ceph-mon[27939]: pgmap v327: 129 pgs: 129 active+clean; 2.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 55 KiB/s rd, 53 KiB/s wr, 21 op/s 2024-04-21T08:49:18.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:17 smithi088 ceph-mon[31259]: pgmap v328: 129 pgs: 129 active+clean; 2.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 51 KiB/s wr, 20 op/s 2024-04-21T08:49:18.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:17 smithi179 ceph-mon[31829]: pgmap v328: 129 pgs: 129 active+clean; 2.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 51 KiB/s wr, 20 op/s 2024-04-21T08:49:18.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:17 smithi086 ceph-mon[27939]: pgmap v328: 129 pgs: 129 active+clean; 2.9 MiB data, 425 MiB used, 1.0 TiB / 1.0 TiB avail; 53 KiB/s rd, 51 KiB/s wr, 20 op/s 2024-04-21T08:49:20.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:19 smithi088 ceph-mon[31259]: pgmap v329: 129 pgs: 129 active+clean; 3.1 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 48 KiB/s wr, 19 op/s 2024-04-21T08:49:20.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:19 smithi179 ceph-mon[31829]: pgmap v329: 129 pgs: 129 active+clean; 3.1 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 48 KiB/s wr, 19 op/s 2024-04-21T08:49:20.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:19 smithi086 ceph-mon[27939]: pgmap v329: 129 pgs: 129 active+clean; 3.1 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 49 KiB/s rd, 48 KiB/s wr, 19 op/s 2024-04-21T08:49:22.169 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:21 smithi086 ceph-mon[27939]: pgmap v330: 129 pgs: 129 active+clean; 3.1 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 47 KiB/s wr, 19 op/s 2024-04-21T08:49:22.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:21 smithi088 ceph-mon[31259]: pgmap v330: 129 pgs: 129 active+clean; 3.1 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 47 KiB/s wr, 19 op/s 2024-04-21T08:49:22.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:21 smithi179 ceph-mon[31829]: pgmap v330: 129 pgs: 129 active+clean; 3.1 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 48 KiB/s rd, 47 KiB/s wr, 19 op/s 2024-04-21T08:49:24.026 INFO:tasks.workunit.client.0.smithi086.stdout:437/600: open fd = 3 2024-04-21T08:49:24.027 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.027 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.027 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.027 INFO:tasks.workunit.client.0.smithi086.stdout:438/600: open fd = 3 2024-04-21T08:49:24.027 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.027 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.027 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.027 INFO:tasks.workunit.client.0.smithi086.stdout:439/600: open fd = 3 2024-04-21T08:49:24.027 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.027 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.027 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.027 INFO:tasks.workunit.client.0.smithi086.stdout:440/600: open fd = 3 2024-04-21T08:49:24.027 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.028 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.028 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.028 INFO:tasks.workunit.client.0.smithi086.stdout:441/600: open fd = 3 2024-04-21T08:49:24.028 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.028 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.028 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.028 INFO:tasks.workunit.client.0.smithi086.stdout:442/600: open fd = 3 2024-04-21T08:49:24.028 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.028 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.028 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.028 INFO:tasks.workunit.client.0.smithi086.stdout:443/600: open fd = 3 2024-04-21T08:49:24.028 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.028 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.028 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.029 INFO:tasks.workunit.client.0.smithi086.stdout:444/600: open fd = 3 2024-04-21T08:49:24.029 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.029 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.029 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.029 INFO:tasks.workunit.client.0.smithi086.stdout:445/600: open fd = 3 2024-04-21T08:49:24.029 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.029 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.029 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.029 INFO:tasks.workunit.client.0.smithi086.stdout:446/600: open fd = 3 2024-04-21T08:49:24.029 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.029 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.029 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.029 INFO:tasks.workunit.client.0.smithi086.stdout:447/600: open fd = 3 2024-04-21T08:49:24.029 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.029 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.030 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.030 INFO:tasks.workunit.client.0.smithi086.stdout:448/600: open fd = 3 2024-04-21T08:49:24.030 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.030 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.030 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.030 INFO:tasks.workunit.client.0.smithi086.stdout:449/600: open fd = 3 2024-04-21T08:49:24.030 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.030 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.030 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.030 INFO:tasks.workunit.client.0.smithi086.stdout:450/600: open fd = 3 2024-04-21T08:49:24.030 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.030 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.030 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.030 INFO:tasks.workunit.client.0.smithi086.stdout:451/600: open fd = 3 2024-04-21T08:49:24.031 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.031 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.031 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.031 INFO:tasks.workunit.client.0.smithi086.stdout:452/600: open fd = 3 2024-04-21T08:49:24.031 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.031 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.031 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.031 INFO:tasks.workunit.client.0.smithi086.stdout:453/600: open fd = 3 2024-04-21T08:49:24.031 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.031 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.031 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.031 INFO:tasks.workunit.client.0.smithi086.stdout:454/600: open fd = 3 2024-04-21T08:49:24.031 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.031 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.032 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.032 INFO:tasks.workunit.client.0.smithi086.stdout:455/600: open fd = 3 2024-04-21T08:49:24.032 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.032 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.032 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.032 INFO:tasks.workunit.client.0.smithi086.stdout:456/600: open fd = 3 2024-04-21T08:49:24.032 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.032 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.032 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.032 INFO:tasks.workunit.client.0.smithi086.stdout:457/600: open fd = 3 2024-04-21T08:49:24.032 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.032 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.032 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.032 INFO:tasks.workunit.client.0.smithi086.stdout:458/600: open fd = 3 2024-04-21T08:49:24.032 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.033 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.033 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.033 INFO:tasks.workunit.client.0.smithi086.stdout:459/600: open fd = 3 2024-04-21T08:49:24.033 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.033 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.033 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.033 INFO:tasks.workunit.client.0.smithi086.stdout:460/600: open fd = 3 2024-04-21T08:49:24.033 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.033 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.033 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.033 INFO:tasks.workunit.client.0.smithi086.stdout:461/600: open fd = 3 2024-04-21T08:49:24.033 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.033 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.033 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.034 INFO:tasks.workunit.client.0.smithi086.stdout:462/600: open fd = 3 2024-04-21T08:49:24.034 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.034 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.034 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.034 INFO:tasks.workunit.client.0.smithi086.stdout:463/600: open fd = 3 2024-04-21T08:49:24.034 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.034 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.034 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.034 INFO:tasks.workunit.client.0.smithi086.stdout:464/600: open fd = 3 2024-04-21T08:49:24.034 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.034 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.034 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.034 INFO:tasks.workunit.client.0.smithi086.stdout:465/600: open fd = 3 2024-04-21T08:49:24.034 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.035 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.035 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.035 INFO:tasks.workunit.client.0.smithi086.stdout:466/600: open fd = 3 2024-04-21T08:49:24.035 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.035 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.035 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.035 INFO:tasks.workunit.client.0.smithi086.stdout:467/600: open fd = 3 2024-04-21T08:49:24.035 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.035 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.035 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.035 INFO:tasks.workunit.client.0.smithi086.stdout:468/600: open fd = 3 2024-04-21T08:49:24.035 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.035 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.035 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.035 INFO:tasks.workunit.client.0.smithi086.stdout:469/600: open fd = 3 2024-04-21T08:49:24.036 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.036 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.036 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.036 INFO:tasks.workunit.client.0.smithi086.stdout:470/600: open fd = 3 2024-04-21T08:49:24.036 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.036 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.036 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.036 INFO:tasks.workunit.client.0.smithi086.stdout:471/600: open fd = 3 2024-04-21T08:49:24.036 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.036 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.036 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.036 INFO:tasks.workunit.client.0.smithi086.stdout:472/600: open fd = 3 2024-04-21T08:49:24.036 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.036 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.037 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.037 INFO:tasks.workunit.client.0.smithi086.stdout:473/600: open fd = 3 2024-04-21T08:49:24.037 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.037 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.037 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.037 INFO:tasks.workunit.client.0.smithi086.stdout:474/600: open fd = 3 2024-04-21T08:49:24.037 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.037 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.037 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.037 INFO:tasks.workunit.client.0.smithi086.stdout:475/600: open fd = 3 2024-04-21T08:49:24.037 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.037 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.037 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.037 INFO:tasks.workunit.client.0.smithi086.stdout:476/600: open fd = 3 2024-04-21T08:49:24.037 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.038 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.038 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.038 INFO:tasks.workunit.client.0.smithi086.stdout:477/600: open fd = 3 2024-04-21T08:49:24.038 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.038 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.038 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.038 INFO:tasks.workunit.client.0.smithi086.stdout:478/600: open fd = 3 2024-04-21T08:49:24.038 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.038 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.038 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.038 INFO:tasks.workunit.client.0.smithi086.stdout:479/600: open fd = 3 2024-04-21T08:49:24.038 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.038 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.039 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.039 INFO:tasks.workunit.client.0.smithi086.stdout:480/600: open fd = 3 2024-04-21T08:49:24.039 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.039 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.039 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.039 INFO:tasks.workunit.client.0.smithi086.stdout:481/600: open fd = 3 2024-04-21T08:49:24.039 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.039 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.039 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.039 INFO:tasks.workunit.client.0.smithi086.stdout:482/600: open fd = 3 2024-04-21T08:49:24.039 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.039 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.039 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.039 INFO:tasks.workunit.client.0.smithi086.stdout:483/600: open fd = 3 2024-04-21T08:49:24.039 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.040 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.040 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.040 INFO:tasks.workunit.client.0.smithi086.stdout:484/600: open fd = 3 2024-04-21T08:49:24.040 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.040 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.040 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.040 INFO:tasks.workunit.client.0.smithi086.stdout:485/600: open fd = 3 2024-04-21T08:49:24.040 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.040 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.040 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.040 INFO:tasks.workunit.client.0.smithi086.stdout:486/600: open fd = 3 2024-04-21T08:49:24.040 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.040 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.040 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.040 INFO:tasks.workunit.client.0.smithi086.stdout:487/600: open fd = 3 2024-04-21T08:49:24.041 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.041 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.041 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.041 INFO:tasks.workunit.client.0.smithi086.stdout:488/600: open fd = 3 2024-04-21T08:49:24.041 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.041 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.041 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.041 INFO:tasks.workunit.client.0.smithi086.stdout:489/600: open fd = 3 2024-04-21T08:49:24.041 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.041 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.041 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.041 INFO:tasks.workunit.client.0.smithi086.stdout:490/600: open fd = 3 2024-04-21T08:49:24.041 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.041 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.041 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.041 INFO:tasks.workunit.client.0.smithi086.stdout:491/600: open fd = 3 2024-04-21T08:49:24.042 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.042 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.042 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.042 INFO:tasks.workunit.client.0.smithi086.stdout:492/600: open fd = 3 2024-04-21T08:49:24.042 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.042 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.042 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.042 INFO:tasks.workunit.client.0.smithi086.stdout:493/600: open fd = 3 2024-04-21T08:49:24.042 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.042 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.042 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.042 INFO:tasks.workunit.client.0.smithi086.stdout:494/600: open fd = 3 2024-04-21T08:49:24.042 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.042 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.042 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.043 INFO:tasks.workunit.client.0.smithi086.stdout:495/600: open fd = 3 2024-04-21T08:49:24.043 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.043 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.043 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.043 INFO:tasks.workunit.client.0.smithi086.stdout:496/600: open fd = 3 2024-04-21T08:49:24.043 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.043 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.043 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.043 INFO:tasks.workunit.client.0.smithi086.stdout:497/600: open fd = 3 2024-04-21T08:49:24.043 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.043 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.043 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.043 INFO:tasks.workunit.client.0.smithi086.stdout:498/600: open fd = 3 2024-04-21T08:49:24.043 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.043 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:49:24.044 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:49:24.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:23 smithi088 ceph-mon[31259]: pgmap v331: 129 pgs: 129 active+clean; 3.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 23 op/s 2024-04-21T08:49:24.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:23 smithi179 ceph-mon[31829]: pgmap v331: 129 pgs: 129 active+clean; 3.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 23 op/s 2024-04-21T08:49:24.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:23 smithi086 ceph-mon[27939]: pgmap v331: 129 pgs: 129 active+clean; 3.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 57 KiB/s wr, 23 op/s 2024-04-21T08:49:24.621 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.619+0000 7fdf8dfce640 1 -- 172.21.15.86:0/2706479953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf881061b0 msgr2=0x7fdf8810dd80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:24.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.619+0000 7fdf8dfce640 1 --2- 172.21.15.86:0/2706479953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf881061b0 0x7fdf8810dd80 secure :-1 s=READY pgs=1886 cs=0 l=1 rev1=1 crypto rx=0x7fdf80009900 tx=0x7fdf8002bc30 comp rx=0 tx=0).stop 2024-04-21T08:49:24.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.620+0000 7fdf8dfce640 1 -- 172.21.15.86:0/2706479953 shutdown_connections 2024-04-21T08:49:24.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.620+0000 7fdf8dfce640 1 --2- 172.21.15.86:0/2706479953 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf881061b0 0x7fdf8810dd80 unknown :-1 s=CLOSED pgs=1886 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.620+0000 7fdf8dfce640 1 --2- 172.21.15.86:0/2706479953 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf88101a90 0x7fdf88105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.620+0000 7fdf8dfce640 1 --2- 172.21.15.86:0/2706479953 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf881010e0 0x7fdf881014c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.620+0000 7fdf8dfce640 1 -- 172.21.15.86:0/2706479953 >> 172.21.15.86:0/2706479953 conn(0x7fdf880771d0 msgr2=0x7fdf880775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:24.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.620+0000 7fdf8dfce640 1 -- 172.21.15.86:0/2706479953 shutdown_connections 2024-04-21T08:49:24.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.620+0000 7fdf8dfce640 1 -- 172.21.15.86:0/2706479953 wait complete. 2024-04-21T08:49:24.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.620+0000 7fdf8dfce640 1 Processor -- start 2024-04-21T08:49:24.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.620+0000 7fdf8dfce640 1 -- start start 2024-04-21T08:49:24.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.621+0000 7fdf8dfce640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf881010e0 0x7fdf88198680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:24.622 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.621+0000 7fdf8dfce640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf881061b0 0x7fdf88198bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:24.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.621+0000 7fdf8dfce640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf8819cf40 0x7fdf8819d3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:24.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.621+0000 7fdf8dfce640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdf88111120 con 0x7fdf881061b0 2024-04-21T08:49:24.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.621+0000 7fdf8dfce640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdf88110fa0 con 0x7fdf8819cf40 2024-04-21T08:49:24.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.621+0000 7fdf8dfce640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdf881112a0 con 0x7fdf881010e0 2024-04-21T08:49:24.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.621+0000 7fdf86ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf881061b0 0x7fdf88198bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:24.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.621+0000 7fdf86ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf881061b0 0x7fdf88198bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60236/0 (socket says 172.21.15.86:60236) 2024-04-21T08:49:24.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.621+0000 7fdf86ffd640 1 -- 172.21.15.86:0/836987648 learned_addr learned my addr 172.21.15.86:0/836987648 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:24.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.621+0000 7fdf877fe640 1 --2- 172.21.15.86:0/836987648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf881010e0 0x7fdf88198680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:24.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.621+0000 7fdf87fff640 1 --2- 172.21.15.86:0/836987648 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf8819cf40 0x7fdf8819d3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:24.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.621+0000 7fdf86ffd640 1 -- 172.21.15.86:0/836987648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf881010e0 msgr2=0x7fdf88198680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:24.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.621+0000 7fdf86ffd640 1 --2- 172.21.15.86:0/836987648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf881010e0 0x7fdf88198680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.621+0000 7fdf86ffd640 1 -- 172.21.15.86:0/836987648 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf8819cf40 msgr2=0x7fdf8819d3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:24.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.621+0000 7fdf86ffd640 1 --2- 172.21.15.86:0/836987648 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf8819cf40 0x7fdf8819d3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.621+0000 7fdf86ffd640 1 -- 172.21.15.86:0/836987648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdf8819da30 con 0x7fdf881061b0 2024-04-21T08:49:24.623 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.621+0000 7fdf87fff640 1 --2- 172.21.15.86:0/836987648 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf8819cf40 0x7fdf8819d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.621+0000 7fdf86ffd640 1 --2- 172.21.15.86:0/836987648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf881061b0 0x7fdf88198bc0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c00d740 tx=0x7fdf7c00dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.621+0000 7fdf84ff9640 1 -- 172.21.15.86:0/836987648 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c017070 con 0x7fdf881061b0 2024-04-21T08:49:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.621+0000 7fdf84ff9640 1 -- 172.21.15.86:0/836987648 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf7c00de50 con 0x7fdf881061b0 2024-04-21T08:49:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.621+0000 7fdf8dfce640 1 -- 172.21.15.86:0/836987648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdf8819dd20 con 0x7fdf881061b0 2024-04-21T08:49:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.621+0000 7fdf84ff9640 1 -- 172.21.15.86:0/836987648 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c004ce0 con 0x7fdf881061b0 2024-04-21T08:49:24.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.621+0000 7fdf8dfce640 1 -- 172.21.15.86:0/836987648 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdf881a54c0 con 0x7fdf881061b0 2024-04-21T08:49:24.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.624+0000 7fdf84ff9640 1 -- 172.21.15.86:0/836987648 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdf7c004e80 con 0x7fdf881061b0 2024-04-21T08:49:24.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.624+0000 7fdf90bc4740 1 Processor -- start 2024-04-21T08:49:24.630 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.624+0000 7fdf90bc4740 1 -- start start 2024-04-21T08:49:24.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.624+0000 7fdf90bc4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac4b0308d0 0x55ac4b02f620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:24.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.624+0000 7fdf90bc4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac4b02fb60 0x55ac4b02ffe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:24.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.624+0000 7fdf90bc4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac4b0390e0 0x55ac4b03b4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:24.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.624+0000 7fdf90bc4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ac4af948d0 con 0x55ac4b0390e0 2024-04-21T08:49:24.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.624+0000 7fdf90bc4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55ac4af31e80 con 0x55ac4b0308d0 2024-04-21T08:49:24.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.624+0000 7fdf90bc4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ac4af90890 con 0x55ac4b02fb60 2024-04-21T08:49:24.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.624+0000 7fdf84ff9640 1 --2- 172.21.15.86:0/836987648 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdf700a3dc0 0x7fdf700a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:24.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.624+0000 7fdf86ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac4b0390e0 0x55ac4b03b4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:24.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.624+0000 7fdf87fff640 1 --2- 172.21.15.86:0/836987648 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdf700a3dc0 0x7fdf700a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:24.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.624+0000 7fdf86ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac4b0390e0 0x55ac4b03b4d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60244/0 (socket says 172.21.15.86:60244) 2024-04-21T08:49:24.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.624+0000 7fdf87fff640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac4b0308d0 0x55ac4b02f620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:24.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.624+0000 7fdf86ffd640 1 -- 172.21.15.86:0/952648504 learned_addr learned my addr 172.21.15.86:0/952648504 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:24.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.624+0000 7fdf877fe640 1 --2- 172.21.15.86:0/952648504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac4b02fb60 0x55ac4b02ffe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:24.631 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.625+0000 7fdf84ff9640 1 -- 172.21.15.86:0/836987648 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7fdf7c0d04f0 con 0x7fdf881061b0 2024-04-21T08:49:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.625+0000 7fdf87fff640 1 -- 172.21.15.86:0/952648504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac4b02fb60 msgr2=0x55ac4b02ffe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.625+0000 7fdf87fff640 1 --2- 172.21.15.86:0/952648504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac4b02fb60 0x55ac4b02ffe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.625+0000 7fdf87fff640 1 -- 172.21.15.86:0/952648504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac4b0390e0 msgr2=0x55ac4b03b4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.625+0000 7fdf87fff640 1 --2- 172.21.15.86:0/952648504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac4b0390e0 0x55ac4b03b4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.625+0000 7fdf87fff640 1 -- 172.21.15.86:0/952648504 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac4af53320 con 0x55ac4b0308d0 2024-04-21T08:49:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.625+0000 7fdf87fff640 1 --2- 172.21.15.86:0/836987648 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdf700a3dc0 0x7fdf700a6280 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fdf80005ab0 tx=0x7fdf8002e000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.625+0000 7fdf87fff640 1 --2- 172.21.15.86:0/952648504 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac4b0308d0 0x55ac4b02f620 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdf80005770 tx=0x7fdf8002c7e0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=b1f396aa16854edd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.625+0000 7fdf6dffb640 1 -- 172.21.15.86:0/952648504 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf80004150 con 0x55ac4b0308d0 2024-04-21T08:49:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.625+0000 7fdf6dffb640 1 -- 172.21.15.86:0/952648504 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf80005020 con 0x55ac4b0308d0 2024-04-21T08:49:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.625+0000 7fdf6dffb640 1 -- 172.21.15.86:0/952648504 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf80004900 con 0x55ac4b0308d0 2024-04-21T08:49:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.626+0000 7fdf90bc4740 1 -- 172.21.15.86:0/952648504 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac4b0308d0 msgr2=0x55ac4b02f620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.626+0000 7fdf90bc4740 1 --2- 172.21.15.86:0/952648504 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac4b0308d0 0x55ac4b02f620 secure :-1 s=READY pgs=84 cs=0 l=1 rev1=1 crypto rx=0x7fdf80005770 tx=0x7fdf8002c7e0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.626+0000 7fdf90bc4740 1 -- 172.21.15.86:0/952648504 shutdown_connections 2024-04-21T08:49:24.632 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.626+0000 7fdf90bc4740 1 --2- 172.21.15.86:0/952648504 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac4b0390e0 0x55ac4b03b4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.626+0000 7fdf90bc4740 1 --2- 172.21.15.86:0/952648504 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac4b02fb60 0x55ac4b02ffe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.626+0000 7fdf90bc4740 1 --2- 172.21.15.86:0/952648504 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac4b0308d0 0x55ac4b02f620 unknown :-1 s=CLOSED pgs=84 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.626+0000 7fdf90bc4740 1 -- 172.21.15.86:0/952648504 >> 172.21.15.86:0/952648504 conn(0x55ac4aff6c90 msgr2=0x55ac4b02a850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.626+0000 7fdf90bc4740 1 -- 172.21.15.86:0/952648504 shutdown_connections 2024-04-21T08:49:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.626+0000 7fdf90bc4740 1 -- 172.21.15.86:0/952648504 wait complete. 2024-04-21T08:49:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.626+0000 7fdf90bc4740 1 -- start start 2024-04-21T08:49:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.627+0000 7fdf90bc4740 1 Processor -- start 2024-04-21T08:49:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.627+0000 7fdf90bc4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac4b02fb60 0x55ac4b038a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.627+0000 7fdf90bc4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac4b0308d0 0x55ac4b035e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.627+0000 7fdf90bc4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac4b0390e0 0x55ac4b0364b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.627+0000 7fdf90bc4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55ac4b03bd50 con 0x55ac4b0308d0 2024-04-21T08:49:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.627+0000 7fdf90bc4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55ac4af1ceb0 con 0x55ac4b02fb60 2024-04-21T08:49:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.627+0000 7fdf90bc4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55ac4b03bed0 con 0x55ac4b0390e0 2024-04-21T08:49:24.633 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.627+0000 7fdf90bc4740 10 client.0 _ll_register_callbacks cb 0x55ac4af49b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.627+0000 7fdf86ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac4b0308d0 0x55ac4b035e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.627+0000 7fdf86ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac4b0308d0 0x55ac4b035e80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60252/0 (socket says 172.21.15.86:60252) 2024-04-21T08:49:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.627+0000 7fdf86ffd640 1 -- 172.21.15.86:0/3155633950 learned_addr learned my addr 172.21.15.86:0/3155633950 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.627+0000 7fdf877fe640 1 --2- 172.21.15.86:0/3155633950 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac4b02fb60 0x55ac4b038a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.627+0000 7fdf87fff640 1 --2- 172.21.15.86:0/3155633950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac4b0390e0 0x55ac4b0364b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.627+0000 7fdf86ffd640 1 -- 172.21.15.86:0/3155633950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac4b0390e0 msgr2=0x55ac4b0364b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.627+0000 7fdf86ffd640 1 --2- 172.21.15.86:0/3155633950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac4b0390e0 0x55ac4b0364b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.627+0000 7fdf86ffd640 1 -- 172.21.15.86:0/3155633950 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac4b02fb60 msgr2=0x55ac4b038a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.627+0000 7fdf86ffd640 1 --2- 172.21.15.86:0/3155633950 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac4b02fb60 0x55ac4b038a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.627+0000 7fdf86ffd640 1 -- 172.21.15.86:0/3155633950 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac4aca7210 con 0x55ac4b0308d0 2024-04-21T08:49:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.627+0000 7fdf86ffd640 1 --2- 172.21.15.86:0/3155633950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac4b0308d0 0x55ac4b035e80 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c098520 tx=0x7fdf7c06b9a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fec8e05b70e3846e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.627+0000 7fdf66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:24.634 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.628+0000 7fdf66ffd640 1 -- 172.21.15.86:0/3155633950 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c020070 con 0x55ac4b0308d0 2024-04-21T08:49:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.628+0000 7fdf66ffd640 1 -- 172.21.15.86:0/3155633950 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7fdf7c0cdaf0 con 0x55ac4b0308d0 2024-04-21T08:49:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.628+0000 7fdf90bc4740 1 -- 172.21.15.86:0/3155633950 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf80005020 con 0x55ac4b0308d0 2024-04-21T08:49:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.628+0000 7fdf66ffd640 1 -- 172.21.15.86:0/3155633950 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdf7c0a0470 con 0x55ac4b0308d0 2024-04-21T08:49:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.628+0000 7fdf90bc4740 10 client.19890 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.628+0000 7fdf90bc4740 1 -- 172.21.15.86:0/3155633950 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ac4ad73620 con 0x55ac4b0308d0 2024-04-21T08:49:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.628+0000 7fdf66ffd640 1 -- 172.21.15.86:0/3155633950 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf80005020 con 0x55ac4b0308d0 2024-04-21T08:49:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.628+0000 7fdf66ffd640 1 -- 172.21.15.86:0/3155633950 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdf7c0a48a0 con 0x55ac4b0308d0 2024-04-21T08:49:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.628+0000 7fdf90bc4740 10 client.19890 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.628+0000 7fdf90bc4740 10 client.19890 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.628+0000 7fdf90bc4740 20 client.19890 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.628+0000 7fdf90bc4740 1 --2- 172.21.15.86:0/3155633950 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ac4b049040 0x55ac4b04b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.628+0000 7fdf90bc4740 4 client.19890 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.628+0000 7fdf90bc4740 1 -- 172.21.15.86:0/3155633950 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55ac4afa8880 con 0x55ac4b049040 2024-04-21T08:49:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.628+0000 7fdf877fe640 1 --2- 172.21.15.86:0/3155633950 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ac4b049040 0x55ac4b04b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:24.635 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.629+0000 7fdf66ffd640 10 client.19890 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.629+0000 7fdf877fe640 1 --2- 172.21.15.86:0/3155633950 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ac4b049040 0x55ac4b04b430 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=cb7d66aa7dcc4e01 server_cookie=b6d0c9b02f92673c in_seq=0 out_seq=0 2024-04-21T08:49:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.629+0000 7fdf66ffd640 1 -- 172.21.15.86:0/3155633950 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55ac4afa8880 con 0x55ac4b049040 2024-04-21T08:49:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.629+0000 7fdf66ffd640 10 client.19890 handle_command_reply: tid=0 2024-04-21T08:49:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.629+0000 7fdf90bc4740 1 client.19890 shutdown 2024-04-21T08:49:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.630+0000 7fdf90bc4740 1 -- 172.21.15.86:0/3155633950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac4b0308d0 msgr2=0x55ac4b035e80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.630+0000 7fdf90bc4740 1 --2- 172.21.15.86:0/3155633950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac4b0308d0 0x55ac4b035e80 secure :-1 s=READY pgs=1888 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c098520 tx=0x7fdf7c06b9a0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.630+0000 7fdf90bc4740 1 -- 172.21.15.86:0/3155633950 shutdown_connections 2024-04-21T08:49:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.630+0000 7fdf90bc4740 1 --2- 172.21.15.86:0/3155633950 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ac4b049040 0x55ac4b04b430 crc :-1 s=READY pgs=309 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.630+0000 7fdf90bc4740 1 --2- 172.21.15.86:0/3155633950 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac4b0390e0 0x55ac4b0364b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.630+0000 7fdf90bc4740 1 --2- 172.21.15.86:0/3155633950 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac4b0308d0 0x55ac4b035e80 unknown :-1 s=CLOSED pgs=1888 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.630+0000 7fdf90bc4740 1 --2- 172.21.15.86:0/3155633950 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac4b02fb60 0x55ac4b038a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.630+0000 7fdf66ffd640 0 client.19890 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.630+0000 7fdf90bc4740 1 -- 172.21.15.86:0/3155633950 >> 172.21.15.86:0/3155633950 conn(0x55ac4aff6c90 msgr2=0x55ac4b02c5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:24.636 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.630+0000 7fdf90bc4740 1 -- 172.21.15.86:0/3155633950 shutdown_connections 2024-04-21T08:49:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.630+0000 7fdf90bc4740 1 -- 172.21.15.86:0/3155633950 wait complete. 2024-04-21T08:49:24.637 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.630+0000 7fdf90bc4740 20 client.19890 trim_cache size 0 max 16384 2024-04-21T08:49:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.645+0000 7fdf90bc4740 1 Processor -- start 2024-04-21T08:49:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.645+0000 7fdf90bc4740 1 -- start start 2024-04-21T08:49:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.645+0000 7fdf90bc4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac4b02fb60 0x55ac4b044c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:24.646 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.645+0000 7fdf90bc4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac4b0308d0 0x55ac4b02a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.645+0000 7fdf90bc4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac4b0390e0 0x55ac4b050d20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.645+0000 7fdf90bc4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdf881af250 con 0x55ac4b0390e0 2024-04-21T08:49:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.645+0000 7fdf90bc4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdf881b0d90 con 0x55ac4b0308d0 2024-04-21T08:49:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.645+0000 7fdf90bc4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdf881af920 con 0x55ac4b02fb60 2024-04-21T08:49:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.645+0000 7fdf87fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac4b0390e0 0x55ac4b050d20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.645+0000 7fdf87fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac4b0390e0 0x55ac4b050d20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60258/0 (socket says 172.21.15.86:60258) 2024-04-21T08:49:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.645+0000 7fdf87fff640 1 -- 172.21.15.86:0/2164931372 learned_addr learned my addr 172.21.15.86:0/2164931372 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.645+0000 7fdf877fe640 1 --2- 172.21.15.86:0/2164931372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac4b02fb60 0x55ac4b044c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.645+0000 7fdf86ffd640 1 --2- 172.21.15.86:0/2164931372 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac4b0308d0 0x55ac4b02a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.645+0000 7fdf87fff640 1 -- 172.21.15.86:0/2164931372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac4b02fb60 msgr2=0x55ac4b044c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.645+0000 7fdf87fff640 1 --2- 172.21.15.86:0/2164931372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac4b02fb60 0x55ac4b044c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.645+0000 7fdf87fff640 1 -- 172.21.15.86:0/2164931372 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac4b0308d0 msgr2=0x55ac4b02a380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.645+0000 7fdf87fff640 1 --2- 172.21.15.86:0/2164931372 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac4b0308d0 0x55ac4b02a380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.647 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.645+0000 7fdf87fff640 1 -- 172.21.15.86:0/2164931372 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55ac4b04ba90 con 0x55ac4b0390e0 2024-04-21T08:49:24.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.646+0000 7fdf87fff640 1 --2- 172.21.15.86:0/2164931372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac4b0390e0 0x55ac4b050d20 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fdf800057c0 tx=0x7fdf8002c2b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7f7620e8cafe8957 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:24.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.646+0000 7fdf6d7fa640 1 -- 172.21.15.86:0/2164931372 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf8005b070 con 0x55ac4b0390e0 2024-04-21T08:49:24.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.646+0000 7fdf6d7fa640 1 -- 172.21.15.86:0/2164931372 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf800046c0 con 0x55ac4b0390e0 2024-04-21T08:49:24.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.646+0000 7fdf6d7fa640 1 -- 172.21.15.86:0/2164931372 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf800567c0 con 0x55ac4b0390e0 2024-04-21T08:49:24.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.646+0000 7fdf90bc4740 1 -- 172.21.15.86:0/2164931372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac4b0390e0 msgr2=0x55ac4b050d20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:24.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.646+0000 7fdf90bc4740 1 --2- 172.21.15.86:0/2164931372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac4b0390e0 0x55ac4b050d20 secure :-1 s=READY pgs=1889 cs=0 l=1 rev1=1 crypto rx=0x7fdf800057c0 tx=0x7fdf8002c2b0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.647+0000 7fdf90bc4740 1 -- 172.21.15.86:0/2164931372 shutdown_connections 2024-04-21T08:49:24.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.647+0000 7fdf90bc4740 1 --2- 172.21.15.86:0/2164931372 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac4b0390e0 0x55ac4b050d20 unknown :-1 s=CLOSED pgs=1889 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.647+0000 7fdf90bc4740 1 --2- 172.21.15.86:0/2164931372 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac4b0308d0 0x55ac4b02a380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.647+0000 7fdf90bc4740 1 --2- 172.21.15.86:0/2164931372 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac4b02fb60 0x55ac4b044c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.647+0000 7fdf90bc4740 1 -- 172.21.15.86:0/2164931372 >> 172.21.15.86:0/2164931372 conn(0x55ac4aff6c90 msgr2=0x55ac4b031060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:24.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.647+0000 7fdf90bc4740 1 -- 172.21.15.86:0/2164931372 shutdown_connections 2024-04-21T08:49:24.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.647+0000 7fdf90bc4740 1 -- 172.21.15.86:0/2164931372 wait complete. 2024-04-21T08:49:24.648 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.647+0000 7fdf90bc4740 1 -- start start 2024-04-21T08:49:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.647+0000 7fdf90bc4740 1 Processor -- start 2024-04-21T08:49:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.648+0000 7fdf90bc4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac4b02fb60 0x55ac4b03cd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.648+0000 7fdf90bc4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac4b0308d0 0x55ac4b03d250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:24.651 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.648+0000 7fdf90bc4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac4b0390e0 0x55ac4b03d790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.648+0000 7fdf90bc4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdf881b06c0 con 0x55ac4b0308d0 2024-04-21T08:49:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.648+0000 7fdf90bc4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdf881afff0 con 0x55ac4b0390e0 2024-04-21T08:49:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.648+0000 7fdf90bc4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdf881ad250 con 0x55ac4b02fb60 2024-04-21T08:49:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.648+0000 7fdf90bc4740 10 client.0 _ll_register_callbacks cb 0x7fdf7c09be90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.648+0000 7fdf86ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac4b0308d0 0x55ac4b03d250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.648+0000 7fdf86ffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac4b0308d0 0x55ac4b03d250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:60262/0 (socket says 172.21.15.86:60262) 2024-04-21T08:49:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.648+0000 7fdf86ffd640 1 -- 172.21.15.86:0/1295747330 learned_addr learned my addr 172.21.15.86:0/1295747330 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.648+0000 7fdf877fe640 1 --2- 172.21.15.86:0/1295747330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac4b02fb60 0x55ac4b03cd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.648+0000 7fdf87fff640 1 --2- 172.21.15.86:0/1295747330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac4b0390e0 0x55ac4b03d790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.648+0000 7fdf86ffd640 1 -- 172.21.15.86:0/1295747330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac4b02fb60 msgr2=0x55ac4b03cd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.648+0000 7fdf86ffd640 1 --2- 172.21.15.86:0/1295747330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac4b02fb60 0x55ac4b03cd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.648+0000 7fdf86ffd640 1 -- 172.21.15.86:0/1295747330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac4b0390e0 msgr2=0x55ac4b03d790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:24.652 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.648+0000 7fdf86ffd640 1 --2- 172.21.15.86:0/1295747330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac4b0390e0 0x55ac4b03d790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.648+0000 7fdf86ffd640 1 -- 172.21.15.86:0/1295747330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55ac4b035c50 con 0x55ac4b0308d0 2024-04-21T08:49:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.648+0000 7fdf86ffd640 1 --2- 172.21.15.86:0/1295747330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac4b0308d0 0x55ac4b03d250 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0cde70 tx=0x7fdf7c0a0380 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78ab1c1270d6e2d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.648+0000 7fdf66ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.648+0000 7fdf66ffd640 1 -- 172.21.15.86:0/1295747330 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdf7c020040 con 0x55ac4b0308d0 2024-04-21T08:49:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.648+0000 7fdf66ffd640 1 -- 172.21.15.86:0/1295747330 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7fdf7c0d0170 con 0x55ac4b0308d0 2024-04-21T08:49:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.648+0000 7fdf90bc4740 1 -- 172.21.15.86:0/1295747330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fdf800046c0 con 0x55ac4b0308d0 2024-04-21T08:49:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.649+0000 7fdf66ffd640 1 -- 172.21.15.86:0/1295747330 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fdf7c09ba20 con 0x55ac4b0308d0 2024-04-21T08:49:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.649+0000 7fdf90bc4740 10 client.19896 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.649+0000 7fdf90bc4740 1 -- 172.21.15.86:0/1295747330 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55ac4b02ee30 con 0x55ac4b0308d0 2024-04-21T08:49:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.649+0000 7fdf66ffd640 1 -- 172.21.15.86:0/1295747330 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdf800046c0 con 0x55ac4b0308d0 2024-04-21T08:49:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.649+0000 7fdf66ffd640 1 -- 172.21.15.86:0/1295747330 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fdf7c0b0b60 con 0x55ac4b0308d0 2024-04-21T08:49:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.649+0000 7fdf90bc4740 10 client.19896 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.649+0000 7fdf90bc4740 10 client.19896 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:24.653 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.649+0000 7fdf90bc4740 20 client.19896 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.649+0000 7fdf90bc4740 1 --2- 172.21.15.86:0/1295747330 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ac4b0afe90 0x55ac4b0b22b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.649+0000 7fdf90bc4740 4 client.19896 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:49:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.649+0000 7fdf90bc4740 1 -- 172.21.15.86:0/1295747330 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x55ac4af343d0 con 0x55ac4b0afe90 2024-04-21T08:49:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.649+0000 7fdf877fe640 1 --2- 172.21.15.86:0/1295747330 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ac4b0afe90 0x55ac4b0b22b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.650+0000 7fdf877fe640 1 --2- 172.21.15.86:0/1295747330 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ac4b0afe90 0x55ac4b0b22b0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ff7d74990cf21914 server_cookie=a8aaaca297afc8ac in_seq=0 out_seq=0 2024-04-21T08:49:24.654 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.650+0000 7fdf66ffd640 10 client.19896 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.662+0000 7fdf66ffd640 1 -- 172.21.15.86:0/1295747330 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x55ac4af343d0 con 0x55ac4b0afe90 2024-04-21T08:49:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.662+0000 7fdf66ffd640 10 client.19896 handle_command_reply: tid=0 2024-04-21T08:49:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.662+0000 7fdf90bc4740 1 client.19896 shutdown 2024-04-21T08:49:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.662+0000 7fdf90bc4740 1 -- 172.21.15.86:0/1295747330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac4b0308d0 msgr2=0x55ac4b03d250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.662+0000 7fdf90bc4740 1 --2- 172.21.15.86:0/1295747330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac4b0308d0 0x55ac4b03d250 secure :-1 s=READY pgs=1890 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c0cde70 tx=0x7fdf7c0a0380 comp rx=0 tx=0).stop 2024-04-21T08:49:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.662+0000 7fdf90bc4740 1 -- 172.21.15.86:0/1295747330 shutdown_connections 2024-04-21T08:49:24.662 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.662+0000 7fdf90bc4740 1 --2- 172.21.15.86:0/1295747330 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55ac4b0afe90 0x55ac4b0b22b0 crc :-1 s=READY pgs=310 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.662+0000 7fdf90bc4740 1 --2- 172.21.15.86:0/1295747330 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55ac4b0390e0 0x55ac4b03d790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.662+0000 7fdf90bc4740 1 --2- 172.21.15.86:0/1295747330 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55ac4b0308d0 0x55ac4b03d250 unknown :-1 s=CLOSED pgs=1890 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.662+0000 7fdf90bc4740 1 --2- 172.21.15.86:0/1295747330 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55ac4b02fb60 0x55ac4b03cd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.662+0000 7fdf66ffd640 0 client.19896 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.662+0000 7fdf90bc4740 1 -- 172.21.15.86:0/1295747330 >> 172.21.15.86:0/1295747330 conn(0x55ac4aff6c90 msgr2=0x55ac4b0345c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.662+0000 7fdf90bc4740 1 -- 172.21.15.86:0/1295747330 shutdown_connections 2024-04-21T08:49:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.662+0000 7fdf90bc4740 1 -- 172.21.15.86:0/1295747330 wait complete. 2024-04-21T08:49:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.662+0000 7fdf90bc4740 20 client.19896 trim_cache size 0 max 16384 2024-04-21T08:49:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:49:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:49:24.663 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:49:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:49:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.663+0000 7fdf8dfce640 1 -- 172.21.15.86:0/836987648 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdf700a3dc0 msgr2=0x7fdf700a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:24.664 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.663+0000 7fdf8dfce640 1 --2- 172.21.15.86:0/836987648 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdf700a3dc0 0x7fdf700a6280 secure :-1 s=READY pgs=313 cs=0 l=1 rev1=1 crypto rx=0x7fdf80005ab0 tx=0x7fdf8002e000 comp rx=0 tx=0).stop 2024-04-21T08:49:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.663+0000 7fdf8dfce640 1 -- 172.21.15.86:0/836987648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf881061b0 msgr2=0x7fdf88198bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.663+0000 7fdf8dfce640 1 --2- 172.21.15.86:0/836987648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf881061b0 0x7fdf88198bc0 secure :-1 s=READY pgs=1887 cs=0 l=1 rev1=1 crypto rx=0x7fdf7c00d740 tx=0x7fdf7c00dc00 comp rx=0 tx=0).stop 2024-04-21T08:49:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.663+0000 7fdf8dfce640 1 -- 172.21.15.86:0/836987648 shutdown_connections 2024-04-21T08:49:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.663+0000 7fdf8dfce640 1 --2- 172.21.15.86:0/836987648 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdf700a3dc0 0x7fdf700a6280 unknown :-1 s=CLOSED pgs=313 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.663+0000 7fdf8dfce640 1 --2- 172.21.15.86:0/836987648 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdf8819cf40 0x7fdf8819d3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.663+0000 7fdf8dfce640 1 --2- 172.21.15.86:0/836987648 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdf881061b0 0x7fdf88198bc0 unknown :-1 s=CLOSED pgs=1887 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.663+0000 7fdf8dfce640 1 --2- 172.21.15.86:0/836987648 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdf881010e0 0x7fdf88198680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.663+0000 7fdf8dfce640 1 -- 172.21.15.86:0/836987648 >> 172.21.15.86:0/836987648 conn(0x7fdf880771d0 msgr2=0x7fdf8806aa20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.663+0000 7fdf8dfce640 1 -- 172.21.15.86:0/836987648 shutdown_connections 2024-04-21T08:49:24.665 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:24.663+0000 7fdf8dfce640 1 -- 172.21.15.86:0/836987648 wait complete. 2024-04-21T08:49:26.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:25 smithi088 ceph-mon[31259]: pgmap v332: 129 pgs: 129 active+clean; 3.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-21T08:49:26.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:25 smithi179 ceph-mon[31829]: pgmap v332: 129 pgs: 129 active+clean; 3.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-21T08:49:26.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:25 smithi086 ceph-mon[27939]: pgmap v332: 129 pgs: 129 active+clean; 3.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-21T08:49:28.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:27 smithi088 ceph-mon[31259]: pgmap v333: 129 pgs: 129 active+clean; 3.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-21T08:49:28.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:27 smithi179 ceph-mon[31829]: pgmap v333: 129 pgs: 129 active+clean; 3.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-21T08:49:28.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:27 smithi086 ceph-mon[27939]: pgmap v333: 129 pgs: 129 active+clean; 3.3 MiB data, 426 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-21T08:49:30.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:29 smithi088 ceph-mon[31259]: pgmap v334: 129 pgs: 129 active+clean; 3.5 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-21T08:49:30.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:29 smithi179 ceph-mon[31829]: pgmap v334: 129 pgs: 129 active+clean; 3.5 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-21T08:49:30.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:29 smithi086 ceph-mon[27939]: pgmap v334: 129 pgs: 129 active+clean; 3.5 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-21T08:49:32.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:31 smithi088 ceph-mon[31259]: pgmap v335: 129 pgs: 129 active+clean; 3.5 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:49:32.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:31 smithi179 ceph-mon[31829]: pgmap v335: 129 pgs: 129 active+clean; 3.5 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:49:32.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:31 smithi086 ceph-mon[27939]: pgmap v335: 129 pgs: 129 active+clean; 3.5 MiB data, 427 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:49:34.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:33 smithi088 ceph-mon[31259]: pgmap v336: 129 pgs: 129 active+clean; 3.7 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-21T08:49:34.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:33 smithi179 ceph-mon[31829]: pgmap v336: 129 pgs: 129 active+clean; 3.7 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-21T08:49:34.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:33 smithi086 ceph-mon[27939]: pgmap v336: 129 pgs: 129 active+clean; 3.7 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-21T08:49:36.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:35 smithi088 ceph-mon[31259]: pgmap v337: 129 pgs: 129 active+clean; 3.7 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:49:36.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:35 smithi179 ceph-mon[31829]: pgmap v337: 129 pgs: 129 active+clean; 3.7 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:49:36.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:35 smithi086 ceph-mon[27939]: pgmap v337: 129 pgs: 129 active+clean; 3.7 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:49:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.760+0000 7f0b130c8640 1 -- 172.21.15.86:0/3285598632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b0c10c550 msgr2=0x7f0b0c10e9d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.760+0000 7f0b130c8640 1 --2- 172.21.15.86:0/3285598632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b0c10c550 0x7f0b0c10e9d0 secure :-1 s=READY pgs=1891 cs=0 l=1 rev1=1 crypto rx=0x7f0b04009ca0 tx=0x7f0b0402c180 comp rx=0 tx=0).stop 2024-04-21T08:49:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.760+0000 7f0b130c8640 1 -- 172.21.15.86:0/3285598632 shutdown_connections 2024-04-21T08:49:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.760+0000 7f0b130c8640 1 --2- 172.21.15.86:0/3285598632 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b0c10c550 0x7f0b0c10e9d0 unknown :-1 s=CLOSED pgs=1891 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.760+0000 7f0b130c8640 1 --2- 172.21.15.86:0/3285598632 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b0c103aa0 0x7f0b0c10bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.760+0000 7f0b130c8640 1 --2- 172.21.15.86:0/3285598632 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b0c103180 0x7f0b0c103560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.760+0000 7f0b130c8640 1 -- 172.21.15.86:0/3285598632 >> 172.21.15.86:0/3285598632 conn(0x7f0b0c077fd0 msgr2=0x7f0b0c0783e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:37.762 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.760+0000 7f0b130c8640 1 -- 172.21.15.86:0/3285598632 shutdown_connections 2024-04-21T08:49:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.760+0000 7f0b130c8640 1 -- 172.21.15.86:0/3285598632 wait complete. 2024-04-21T08:49:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.761+0000 7f0b130c8640 1 Processor -- start 2024-04-21T08:49:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.761+0000 7f0b130c8640 1 -- start start 2024-04-21T08:49:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.761+0000 7f0b130c8640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b0c103180 0x7f0b0c198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.761+0000 7f0b130c8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b0c103aa0 0x7f0b0c198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.761+0000 7f0b130c8640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b0c10c550 0x7f0b0c19cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.761+0000 7f0b130c8640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c1121a0 con 0x7f0b0c103aa0 2024-04-21T08:49:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.761+0000 7f0b130c8640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c112020 con 0x7f0b0c103180 2024-04-21T08:49:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.761+0000 7f0b130c8640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c112320 con 0x7f0b0c10c550 2024-04-21T08:49:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.761+0000 7f0b0bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b0c103aa0 0x7f0b0c198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.761+0000 7f0b0bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b0c103aa0 0x7f0b0c198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46968/0 (socket says 172.21.15.86:46968) 2024-04-21T08:49:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.761+0000 7f0b0bfff640 1 -- 172.21.15.86:0/1961724436 learned_addr learned my addr 172.21.15.86:0/1961724436 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.761+0000 7f0b10e64640 1 --2- 172.21.15.86:0/1961724436 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b0c103180 0x7f0b0c198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.761+0000 7f0b11665640 1 --2- 172.21.15.86:0/1961724436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b0c10c550 0x7f0b0c19cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:37.763 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.761+0000 7f0b0bfff640 1 -- 172.21.15.86:0/1961724436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b0c10c550 msgr2=0x7f0b0c19cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.761+0000 7f0b0bfff640 1 --2- 172.21.15.86:0/1961724436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b0c10c550 0x7f0b0c19cf00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.761+0000 7f0b0bfff640 1 -- 172.21.15.86:0/1961724436 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b0c103180 msgr2=0x7f0b0c198640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.761+0000 7f0b0bfff640 1 --2- 172.21.15.86:0/1961724436 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b0c103180 0x7f0b0c198640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.761+0000 7f0b0bfff640 1 -- 172.21.15.86:0/1961724436 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0b0c19d630 con 0x7f0b0c103aa0 2024-04-21T08:49:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.762+0000 7f0b0bfff640 1 --2- 172.21.15.86:0/1961724436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b0c103aa0 0x7f0b0c198b80 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f0af800d950 tx=0x7f0af800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.762+0000 7f0b09ffb640 1 -- 172.21.15.86:0/1961724436 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af8017030 con 0x7f0b0c103aa0 2024-04-21T08:49:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.762+0000 7f0b09ffb640 1 -- 172.21.15.86:0/1961724436 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0af8004020 con 0x7f0b0c103aa0 2024-04-21T08:49:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.762+0000 7f0b09ffb640 1 -- 172.21.15.86:0/1961724436 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0af80047a0 con 0x7f0b0c103aa0 2024-04-21T08:49:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.762+0000 7f0b130c8640 1 -- 172.21.15.86:0/1961724436 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0b0c19d920 con 0x7f0b0c103aa0 2024-04-21T08:49:37.764 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.762+0000 7f0b130c8640 1 -- 172.21.15.86:0/1961724436 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0b0c1a51b0 con 0x7f0b0c103aa0 2024-04-21T08:49:37.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.764+0000 7f0b09ffb640 1 -- 172.21.15.86:0/1961724436 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0af8009a30 con 0x7f0b0c103aa0 2024-04-21T08:49:37.770 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.765+0000 7f0b15d96740 1 Processor -- start 2024-04-21T08:49:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.765+0000 7f0b15d96740 1 -- start start 2024-04-21T08:49:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.765+0000 7f0b09ffb640 1 --2- 172.21.15.86:0/1961724436 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0afc0a3dc0 0x7f0afc0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.765+0000 7f0b15d96740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56290791d8d0 0x56290791c620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.765+0000 7f0b15d96740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56290791cb60 0x56290791cfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.765+0000 7f0b15d96740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56290792a4f0 0x56290792c8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.765+0000 7f0b15d96740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5629078818d0 con 0x56290792a4f0 2024-04-21T08:49:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.765+0000 7f0b15d96740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x56290781ee80 con 0x56290791cb60 2024-04-21T08:49:37.771 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.765+0000 7f0b15d96740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56290787d890 con 0x56290791d8d0 2024-04-21T08:49:37.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.765+0000 7f0b09ffb640 1 -- 172.21.15.86:0/1961724436 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f0af80d0650 con 0x7f0b0c103aa0 2024-04-21T08:49:37.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.765+0000 7f0b11665640 1 --2- 172.21.15.86:0/1961724436 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0afc0a3dc0 0x7f0afc0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:37.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.765+0000 7f0b11665640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56290792a4f0 0x56290792c8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:37.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.765+0000 7f0b10e64640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56290791d8d0 0x56290791c620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:37.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.765+0000 7f0b11665640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56290792a4f0 0x56290792c8e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46972/0 (socket says 172.21.15.86:46972) 2024-04-21T08:49:37.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.765+0000 7f0b11665640 1 -- 172.21.15.86:0/124326188 learned_addr learned my addr 172.21.15.86:0/124326188 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:37.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.765+0000 7f0b0bfff640 1 --2- 172.21.15.86:0/124326188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56290791cb60 0x56290791cfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:37.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.766+0000 7f0b10e64640 1 -- 172.21.15.86:0/124326188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56290791cb60 msgr2=0x56290791cfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:37.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.766+0000 7f0b10e64640 1 --2- 172.21.15.86:0/124326188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56290791cb60 0x56290791cfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:37.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.766+0000 7f0b10e64640 1 -- 172.21.15.86:0/124326188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56290792a4f0 msgr2=0x56290792c8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:37.772 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.766+0000 7f0b10e64640 1 --2- 172.21.15.86:0/124326188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56290792a4f0 0x56290792c8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:37.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.766+0000 7f0b10e64640 1 -- 172.21.15.86:0/124326188 --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562907840320 con 0x56290791d8d0 2024-04-21T08:49:37.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.766+0000 7f0b11665640 1 --2- 172.21.15.86:0/124326188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56290792a4f0 0x56290792c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:49:37.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.766+0000 7f0b11665640 1 --2- 172.21.15.86:0/1961724436 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0afc0a3dc0 0x7f0afc0a6280 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0b04009a60 tx=0x7f0b040324c0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:37.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.766+0000 7f0b10e64640 1 --2- 172.21.15.86:0/124326188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56290791d8d0 0x56290791c620 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0b00009660 tx=0x7f0b00009b20 comp rx=0 tx=0).ready entity=mon.1 client_cookie=8f58205a880a083d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:37.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.766+0000 7f0af6ffd640 1 -- 172.21.15.86:0/124326188 <== mon.1 v2:172.21.15.179:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b00009d70 con 0x56290791d8d0 2024-04-21T08:49:37.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.766+0000 7f0af6ffd640 1 -- 172.21.15.86:0/124326188 <== mon.1 v2:172.21.15.179:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b0000c830 con 0x56290791d8d0 2024-04-21T08:49:37.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.766+0000 7f0af6ffd640 1 -- 172.21.15.86:0/124326188 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b00017410 con 0x56290791d8d0 2024-04-21T08:49:37.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.767+0000 7f0b15d96740 1 -- 172.21.15.86:0/124326188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56290791d8d0 msgr2=0x56290791c620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:37.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.767+0000 7f0b15d96740 1 --2- 172.21.15.86:0/124326188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56290791d8d0 0x56290791c620 secure :-1 s=READY pgs=73 cs=0 l=1 rev1=1 crypto rx=0x7f0b00009660 tx=0x7f0b00009b20 comp rx=0 tx=0).stop 2024-04-21T08:49:37.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.767+0000 7f0b15d96740 1 -- 172.21.15.86:0/124326188 shutdown_connections 2024-04-21T08:49:37.773 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.767+0000 7f0b15d96740 1 --2- 172.21.15.86:0/124326188 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56290792a4f0 0x56290792c8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:37.774 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.767+0000 7f0b15d96740 1 --2- 172.21.15.86:0/124326188 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56290791cb60 0x56290791cfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:37.774 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.767+0000 7f0b15d96740 1 --2- 172.21.15.86:0/124326188 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56290791d8d0 0x56290791c620 unknown :-1 s=CLOSED pgs=73 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:37.774 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.767+0000 7f0b15d96740 1 -- 172.21.15.86:0/124326188 >> 172.21.15.86:0/124326188 conn(0x5629078e3c90 msgr2=0x562907917850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:37.774 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.767+0000 7f0b15d96740 1 -- 172.21.15.86:0/124326188 shutdown_connections 2024-04-21T08:49:37.774 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.767+0000 7f0b15d96740 1 -- 172.21.15.86:0/124326188 wait complete. 2024-04-21T08:49:37.774 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.767+0000 7f0b15d96740 1 -- start start 2024-04-21T08:49:37.774 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.767+0000 7f0b15d96740 1 Processor -- start 2024-04-21T08:49:37.774 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.768+0000 7f0b15d96740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56290791cb60 0x5629079234c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:37.774 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.768+0000 7f0b15d96740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56290791d8d0 0x562907923a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:37.774 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.768+0000 7f0b15d96740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56290792a4f0 0x562907923f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:37.774 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.768+0000 7f0b15d96740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56290791e2c0 con 0x56290791cb60 2024-04-21T08:49:37.774 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.768+0000 7f0b15d96740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x562907809eb0 con 0x56290791d8d0 2024-04-21T08:49:37.774 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.768+0000 7f0b15d96740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56290791e440 con 0x56290792a4f0 2024-04-21T08:49:37.775 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.768+0000 7f0b15d96740 10 client.0 _ll_register_callbacks cb 0x562907836b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:37.775 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.768+0000 7f0b10e64640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56290791cb60 0x5629079234c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:37.775 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.768+0000 7f0b10e64640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56290791cb60 0x5629079234c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46988/0 (socket says 172.21.15.86:46988) 2024-04-21T08:49:37.775 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.768+0000 7f0b10e64640 1 -- 172.21.15.86:0/1257169040 learned_addr learned my addr 172.21.15.86:0/1257169040 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:37.775 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.768+0000 7f0b0bfff640 1 --2- 172.21.15.86:0/1257169040 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56290791d8d0 0x562907923a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:37.775 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.768+0000 7f0b11665640 1 --2- 172.21.15.86:0/1257169040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56290792a4f0 0x562907923f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:37.775 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.768+0000 7f0b10e64640 1 -- 172.21.15.86:0/1257169040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56290792a4f0 msgr2=0x562907923f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:37.775 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.768+0000 7f0b10e64640 1 --2- 172.21.15.86:0/1257169040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56290792a4f0 0x562907923f40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:37.775 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.768+0000 7f0b10e64640 1 -- 172.21.15.86:0/1257169040 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56290791d8d0 msgr2=0x562907923a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:37.775 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.768+0000 7f0b10e64640 1 --2- 172.21.15.86:0/1257169040 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56290791d8d0 0x562907923a00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:37.775 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.768+0000 7f0b10e64640 1 -- 172.21.15.86:0/1257169040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562907594210 con 0x56290791cb60 2024-04-21T08:49:37.775 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.768+0000 7f0b10e64640 1 --2- 172.21.15.86:0/1257169040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56290791cb60 0x5629079234c0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f0b00017d80 tx=0x7f0b00017ea0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=320049a719c675dc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:37.775 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.768+0000 7f0aebfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:37.775 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.768+0000 7f0aebfff640 1 -- 172.21.15.86:0/1257169040 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0000cd70 con 0x56290791cb60 2024-04-21T08:49:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.768+0000 7f0aebfff640 1 -- 172.21.15.86:0/1257169040 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f0b00004030 con 0x56290791cb60 2024-04-21T08:49:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.768+0000 7f0b15d96740 1 -- 172.21.15.86:0/1257169040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b0000c830 con 0x56290791cb60 2024-04-21T08:49:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.769+0000 7f0aebfff640 1 -- 172.21.15.86:0/1257169040 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0b00034050 con 0x56290791cb60 2024-04-21T08:49:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.769+0000 7f0b15d96740 10 client.19908 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.769+0000 7f0b15d96740 1 -- 172.21.15.86:0/1257169040 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562907660620 con 0x56290791cb60 2024-04-21T08:49:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.769+0000 7f0aebfff640 1 -- 172.21.15.86:0/1257169040 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b0000c830 con 0x56290791cb60 2024-04-21T08:49:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.769+0000 7f0aebfff640 1 -- 172.21.15.86:0/1257169040 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0b00017150 con 0x56290791cb60 2024-04-21T08:49:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.769+0000 7f0b15d96740 10 client.19908 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.769+0000 7f0b15d96740 10 client.19908 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.769+0000 7f0b15d96740 20 client.19908 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.769+0000 7f0b15d96740 1 --2- 172.21.15.86:0/1257169040 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x562907936030 0x562907938420 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.769+0000 7f0b15d96740 4 client.19908 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.769+0000 7f0b15d96740 1 -- 172.21.15.86:0/1257169040 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562907895880 con 0x562907936030 2024-04-21T08:49:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.769+0000 7f0b0bfff640 1 --2- 172.21.15.86:0/1257169040 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x562907936030 0x562907938420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:37.776 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.770+0000 7f0aebfff640 10 client.19908 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:37.777 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.770+0000 7f0b0bfff640 1 --2- 172.21.15.86:0/1257169040 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x562907936030 0x562907938420 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e21d543e5e99cdfe server_cookie=3f23c0648bca1043 in_seq=0 out_seq=0 2024-04-21T08:49:37.777 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.770+0000 7f0aebfff640 1 -- 172.21.15.86:0/1257169040 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562907895880 con 0x562907936030 2024-04-21T08:49:37.777 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.770+0000 7f0aebfff640 10 client.19908 handle_command_reply: tid=0 2024-04-21T08:49:37.777 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.770+0000 7f0b15d96740 1 client.19908 shutdown 2024-04-21T08:49:37.777 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.770+0000 7f0b15d96740 1 -- 172.21.15.86:0/1257169040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56290791cb60 msgr2=0x5629079234c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:37.777 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.770+0000 7f0b15d96740 1 --2- 172.21.15.86:0/1257169040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56290791cb60 0x5629079234c0 secure :-1 s=READY pgs=1893 cs=0 l=1 rev1=1 crypto rx=0x7f0b00017d80 tx=0x7f0b00017ea0 comp rx=0 tx=0).stop 2024-04-21T08:49:37.777 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.770+0000 7f0b15d96740 1 -- 172.21.15.86:0/1257169040 shutdown_connections 2024-04-21T08:49:37.777 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.770+0000 7f0b15d96740 1 --2- 172.21.15.86:0/1257169040 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x562907936030 0x562907938420 crc :-1 s=READY pgs=311 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:37.777 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.770+0000 7f0b15d96740 1 --2- 172.21.15.86:0/1257169040 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56290792a4f0 0x562907923f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:37.777 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.770+0000 7f0b15d96740 1 --2- 172.21.15.86:0/1257169040 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56290791d8d0 0x562907923a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:37.777 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.770+0000 7f0aebfff640 0 client.19908 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:37.777 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.770+0000 7f0b15d96740 1 --2- 172.21.15.86:0/1257169040 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56290791cb60 0x5629079234c0 unknown :-1 s=CLOSED pgs=1893 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:37.777 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.770+0000 7f0b15d96740 1 -- 172.21.15.86:0/1257169040 >> 172.21.15.86:0/1257169040 conn(0x5629078e3c90 msgr2=0x5629079195a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:37.777 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.771+0000 7f0b15d96740 1 -- 172.21.15.86:0/1257169040 shutdown_connections 2024-04-21T08:49:37.778 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.771+0000 7f0b15d96740 1 -- 172.21.15.86:0/1257169040 wait complete. 2024-04-21T08:49:37.778 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.771+0000 7f0b15d96740 20 client.19908 trim_cache size 0 max 16384 2024-04-21T08:49:37.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.785+0000 7f0b15d96740 1 Processor -- start 2024-04-21T08:49:37.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.785+0000 7f0b15d96740 1 -- start start 2024-04-21T08:49:37.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.786+0000 7f0b15d96740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56290791cb60 0x562907929c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:37.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.786+0000 7f0b15d96740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56290791d8d0 0x562907916750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:37.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.786+0000 7f0b15d96740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56290792a4f0 0x56290793dae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:37.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.786+0000 7f0b15d96740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c1ad210 con 0x56290791cb60 2024-04-21T08:49:37.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.786+0000 7f0b15d96740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c1aed50 con 0x56290791d8d0 2024-04-21T08:49:37.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.786+0000 7f0b15d96740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c1ad8e0 con 0x56290792a4f0 2024-04-21T08:49:37.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.786+0000 7f0b10e64640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56290791cb60 0x562907929c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:37.790 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.786+0000 7f0b10e64640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56290791cb60 0x562907929c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46998/0 (socket says 172.21.15.86:46998) 2024-04-21T08:49:37.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.786+0000 7f0b10e64640 1 -- 172.21.15.86:0/3746916601 learned_addr learned my addr 172.21.15.86:0/3746916601 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:37.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.786+0000 7f0b11665640 1 --2- 172.21.15.86:0/3746916601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56290792a4f0 0x56290793dae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:37.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.786+0000 7f0b10e64640 1 -- 172.21.15.86:0/3746916601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56290792a4f0 msgr2=0x56290793dae0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:37.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.786+0000 7f0b10e64640 1 --2- 172.21.15.86:0/3746916601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56290792a4f0 0x56290793dae0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:37.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.786+0000 7f0b10e64640 1 -- 172.21.15.86:0/3746916601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56290791d8d0 msgr2=0x562907916750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:37.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.786+0000 7f0b10e64640 1 --2- 172.21.15.86:0/3746916601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56290791d8d0 0x562907916750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:37.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.786+0000 7f0b10e64640 1 -- 172.21.15.86:0/3746916601 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56290791be30 con 0x56290791cb60 2024-04-21T08:49:37.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.786+0000 7f0b10e64640 1 --2- 172.21.15.86:0/3746916601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56290791cb60 0x562907929c10 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f0b000058c0 tx=0x7f0b000178d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81991e0ad0a45f03 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:37.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.786+0000 7f0af67fc640 1 -- 172.21.15.86:0/3746916601 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b0000cd70 con 0x56290791cb60 2024-04-21T08:49:37.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.786+0000 7f0af67fc640 1 -- 172.21.15.86:0/3746916601 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b00017b10 con 0x56290791cb60 2024-04-21T08:49:37.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.786+0000 7f0af67fc640 1 -- 172.21.15.86:0/3746916601 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b000035f0 con 0x56290791cb60 2024-04-21T08:49:37.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.787+0000 7f0b15d96740 1 -- 172.21.15.86:0/3746916601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56290791cb60 msgr2=0x562907929c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:37.791 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.787+0000 7f0b15d96740 1 --2- 172.21.15.86:0/3746916601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56290791cb60 0x562907929c10 secure :-1 s=READY pgs=1894 cs=0 l=1 rev1=1 crypto rx=0x7f0b000058c0 tx=0x7f0b000178d0 comp rx=0 tx=0).stop 2024-04-21T08:49:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.787+0000 7f0b15d96740 1 -- 172.21.15.86:0/3746916601 shutdown_connections 2024-04-21T08:49:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.787+0000 7f0b15d96740 1 --2- 172.21.15.86:0/3746916601 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56290792a4f0 0x56290793dae0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.787+0000 7f0b15d96740 1 --2- 172.21.15.86:0/3746916601 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56290791d8d0 0x562907916750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.787+0000 7f0b15d96740 1 --2- 172.21.15.86:0/3746916601 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56290791cb60 0x562907929c10 unknown :-1 s=CLOSED pgs=1894 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.787+0000 7f0b15d96740 1 -- 172.21.15.86:0/3746916601 >> 172.21.15.86:0/3746916601 conn(0x5629078e3c90 msgr2=0x562907916040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.787+0000 7f0b15d96740 1 -- 172.21.15.86:0/3746916601 shutdown_connections 2024-04-21T08:49:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.787+0000 7f0b15d96740 1 -- 172.21.15.86:0/3746916601 wait complete. 2024-04-21T08:49:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.787+0000 7f0b15d96740 1 -- start start 2024-04-21T08:49:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.787+0000 7f0b15d96740 1 Processor -- start 2024-04-21T08:49:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.788+0000 7f0b15d96740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56290791cb60 0x5629079323d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.788+0000 7f0b15d96740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56290791d8d0 0x562907932910 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.788+0000 7f0b15d96740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56290792a4f0 0x56290791fe40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.788+0000 7f0b15d96740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c1ae680 con 0x56290791cb60 2024-04-21T08:49:37.792 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.788+0000 7f0b15d96740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c1adfb0 con 0x56290792a4f0 2024-04-21T08:49:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.788+0000 7f0b15d96740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0b0c1ab210 con 0x56290791d8d0 2024-04-21T08:49:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.788+0000 7f0b15d96740 10 client.0 _ll_register_callbacks cb 0x7f0b04037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.788+0000 7f0b10e64640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56290791cb60 0x5629079323d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.788+0000 7f0b10e64640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56290791cb60 0x5629079323d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:47004/0 (socket says 172.21.15.86:47004) 2024-04-21T08:49:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.788+0000 7f0b10e64640 1 -- 172.21.15.86:0/4253028107 learned_addr learned my addr 172.21.15.86:0/4253028107 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.788+0000 7f0b11665640 1 --2- 172.21.15.86:0/4253028107 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56290792a4f0 0x56290791fe40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.788+0000 7f0b0bfff640 1 --2- 172.21.15.86:0/4253028107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56290791d8d0 0x562907932910 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.788+0000 7f0b10e64640 1 -- 172.21.15.86:0/4253028107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56290791d8d0 msgr2=0x562907932910 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.788+0000 7f0b10e64640 1 --2- 172.21.15.86:0/4253028107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56290791d8d0 0x562907932910 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.788+0000 7f0b10e64640 1 -- 172.21.15.86:0/4253028107 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56290792a4f0 msgr2=0x56290791fe40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.788+0000 7f0b10e64640 1 --2- 172.21.15.86:0/4253028107 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56290792a4f0 0x56290791fe40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.788+0000 7f0b10e64640 1 -- 172.21.15.86:0/4253028107 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562907920640 con 0x56290791cb60 2024-04-21T08:49:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.788+0000 7f0b10e64640 1 --2- 172.21.15.86:0/4253028107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56290791cb60 0x5629079323d0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f0b000057a0 tx=0x7f0b00009600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3bebd63280c882dc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:37.793 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.788+0000 7f0aebfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.788+0000 7f0aebfff640 1 -- 172.21.15.86:0/4253028107 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0b000050c0 con 0x56290791cb60 2024-04-21T08:49:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.788+0000 7f0b15d96740 1 -- 172.21.15.86:0/4253028107 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0b00017b10 con 0x56290791cb60 2024-04-21T08:49:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.788+0000 7f0aebfff640 1 -- 172.21.15.86:0/4253028107 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f0b0000c830 con 0x56290791cb60 2024-04-21T08:49:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.789+0000 7f0aebfff640 1 -- 172.21.15.86:0/4253028107 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0b000170d0 con 0x56290791cb60 2024-04-21T08:49:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.789+0000 7f0b15d96740 10 client.19914 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.789+0000 7f0b15d96740 1 -- 172.21.15.86:0/4253028107 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56290799bda0 con 0x56290791cb60 2024-04-21T08:49:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.789+0000 7f0aebfff640 1 -- 172.21.15.86:0/4253028107 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0b00017b10 con 0x56290791cb60 2024-04-21T08:49:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.789+0000 7f0aebfff640 1 -- 172.21.15.86:0/4253028107 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0b0002b480 con 0x56290791cb60 2024-04-21T08:49:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.789+0000 7f0b15d96740 10 client.19914 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.789+0000 7f0b15d96740 10 client.19914 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.789+0000 7f0b15d96740 20 client.19914 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.789+0000 7f0b15d96740 1 --2- 172.21.15.86:0/4253028107 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56290799c3a0 0x56290799e790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.789+0000 7f0b15d96740 4 client.19914 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:49:37.794 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.789+0000 7f0b15d96740 1 -- 172.21.15.86:0/4253028107 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5629078213d0 con 0x56290799c3a0 2024-04-21T08:49:37.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.789+0000 7f0b0bfff640 1 --2- 172.21.15.86:0/4253028107 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56290799c3a0 0x56290799e790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:37.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.790+0000 7f0b0bfff640 1 --2- 172.21.15.86:0/4253028107 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56290799c3a0 0x56290799e790 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1694bdb024103db0 server_cookie=332092287239067 in_seq=0 out_seq=0 2024-04-21T08:49:37.795 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.790+0000 7f0aebfff640 10 client.19914 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.805+0000 7f0aebfff640 1 -- 172.21.15.86:0/4253028107 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5629078213d0 con 0x56290799c3a0 2024-04-21T08:49:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.805+0000 7f0aebfff640 10 client.19914 handle_command_reply: tid=0 2024-04-21T08:49:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.806+0000 7f0b15d96740 1 client.19914 shutdown 2024-04-21T08:49:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.806+0000 7f0b15d96740 1 -- 172.21.15.86:0/4253028107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56290791cb60 msgr2=0x5629079323d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.806+0000 7f0b15d96740 1 --2- 172.21.15.86:0/4253028107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56290791cb60 0x5629079323d0 secure :-1 s=READY pgs=1895 cs=0 l=1 rev1=1 crypto rx=0x7f0b000057a0 tx=0x7f0b00009600 comp rx=0 tx=0).stop 2024-04-21T08:49:37.806 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.806+0000 7f0b15d96740 1 -- 172.21.15.86:0/4253028107 shutdown_connections 2024-04-21T08:49:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.806+0000 7f0b15d96740 1 --2- 172.21.15.86:0/4253028107 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x56290799c3a0 0x56290799e790 crc :-1 s=READY pgs=312 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.806+0000 7f0b15d96740 1 --2- 172.21.15.86:0/4253028107 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56290792a4f0 0x56290791fe40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.806+0000 7f0b15d96740 1 --2- 172.21.15.86:0/4253028107 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56290791d8d0 0x562907932910 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.806+0000 7f0aebfff640 0 client.19914 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.806+0000 7f0b15d96740 1 --2- 172.21.15.86:0/4253028107 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56290791cb60 0x5629079323d0 unknown :-1 s=CLOSED pgs=1895 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.806+0000 7f0b15d96740 1 -- 172.21.15.86:0/4253028107 >> 172.21.15.86:0/4253028107 conn(0x5629078e3c90 msgr2=0x562907927360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.806+0000 7f0b15d96740 1 -- 172.21.15.86:0/4253028107 shutdown_connections 2024-04-21T08:49:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.806+0000 7f0b15d96740 1 -- 172.21.15.86:0/4253028107 wait complete. 2024-04-21T08:49:37.807 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.806+0000 7f0b15d96740 20 client.19914 trim_cache size 0 max 16384 2024-04-21T08:49:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:49:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:49:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:49:37.808 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:49:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.807+0000 7f0b130c8640 1 -- 172.21.15.86:0/1961724436 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0afc0a3dc0 msgr2=0x7f0afc0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.807+0000 7f0b130c8640 1 --2- 172.21.15.86:0/1961724436 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0afc0a3dc0 0x7f0afc0a6280 secure :-1 s=READY pgs=314 cs=0 l=1 rev1=1 crypto rx=0x7f0b04009a60 tx=0x7f0b040324c0 comp rx=0 tx=0).stop 2024-04-21T08:49:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.807+0000 7f0b130c8640 1 -- 172.21.15.86:0/1961724436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b0c103aa0 msgr2=0x7f0b0c198b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.807+0000 7f0b130c8640 1 --2- 172.21.15.86:0/1961724436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b0c103aa0 0x7f0b0c198b80 secure :-1 s=READY pgs=1892 cs=0 l=1 rev1=1 crypto rx=0x7f0af800d950 tx=0x7f0af800de10 comp rx=0 tx=0).stop 2024-04-21T08:49:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.807+0000 7f0b130c8640 1 -- 172.21.15.86:0/1961724436 shutdown_connections 2024-04-21T08:49:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.807+0000 7f0b130c8640 1 --2- 172.21.15.86:0/1961724436 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0afc0a3dc0 0x7f0afc0a6280 unknown :-1 s=CLOSED pgs=314 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.807+0000 7f0b130c8640 1 --2- 172.21.15.86:0/1961724436 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0b0c10c550 0x7f0b0c19cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:37.809 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.807+0000 7f0b130c8640 1 --2- 172.21.15.86:0/1961724436 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0b0c103aa0 0x7f0b0c198b80 unknown :-1 s=CLOSED pgs=1892 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.807+0000 7f0b130c8640 1 --2- 172.21.15.86:0/1961724436 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0b0c103180 0x7f0b0c198640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.807+0000 7f0b130c8640 1 -- 172.21.15.86:0/1961724436 >> 172.21.15.86:0/1961724436 conn(0x7f0b0c077fd0 msgr2=0x7f0b0c0fef30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.807+0000 7f0b130c8640 1 -- 172.21.15.86:0/1961724436 shutdown_connections 2024-04-21T08:49:37.810 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:37.807+0000 7f0b130c8640 1 -- 172.21.15.86:0/1961724436 wait complete. 2024-04-21T08:49:38.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:37 smithi088 ceph-mon[31259]: pgmap v338: 129 pgs: 129 active+clean; 3.7 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:49:38.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:37 smithi179 ceph-mon[31829]: pgmap v338: 129 pgs: 129 active+clean; 3.7 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:49:38.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:37 smithi086 ceph-mon[27939]: pgmap v338: 129 pgs: 129 active+clean; 3.7 MiB data, 428 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:49:40.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:39 smithi088 ceph-mon[31259]: pgmap v339: 129 pgs: 129 active+clean; 3.9 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-21T08:49:40.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:39 smithi179 ceph-mon[31829]: pgmap v339: 129 pgs: 129 active+clean; 3.9 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-21T08:49:40.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:39 smithi086 ceph-mon[27939]: pgmap v339: 129 pgs: 129 active+clean; 3.9 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-21T08:49:40.600 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:49:40.601 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:49:40.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.715+0000 7f80a549a640 1 -- 172.21.15.86:0/1315585259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80a0101070 msgr2=0x7f80a0106ba0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:40.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.715+0000 7f80a549a640 1 --2- 172.21.15.86:0/1315585259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80a0101070 0x7f80a0106ba0 secure :-1 s=READY pgs=1896 cs=0 l=1 rev1=1 crypto rx=0x7f8094009900 tx=0x7f809402bc30 comp rx=0 tx=0).stop 2024-04-21T08:49:40.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.715+0000 7f80a549a640 1 -- 172.21.15.86:0/1315585259 shutdown_connections 2024-04-21T08:49:40.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.715+0000 7f80a549a640 1 --2- 172.21.15.86:0/1315585259 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80a0107280 0x7f80a010ee50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:40.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.715+0000 7f80a549a640 1 --2- 172.21.15.86:0/1315585259 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80a0101070 0x7f80a0106ba0 unknown :-1 s=CLOSED pgs=1896 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:40.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.715+0000 7f80a549a640 1 --2- 172.21.15.86:0/1315585259 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80a01006c0 0x7f80a0100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:40.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.715+0000 7f80a549a640 1 -- 172.21.15.86:0/1315585259 >> 172.21.15.86:0/1315585259 conn(0x7f80a00773a0 msgr2=0x7f80a00fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:40.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.715+0000 7f80a549a640 1 -- 172.21.15.86:0/1315585259 shutdown_connections 2024-04-21T08:49:40.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.715+0000 7f80a549a640 1 -- 172.21.15.86:0/1315585259 wait complete. 2024-04-21T08:49:40.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.715+0000 7f80a549a640 1 Processor -- start 2024-04-21T08:49:40.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.715+0000 7f80a549a640 1 -- start start 2024-04-21T08:49:40.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.716+0000 7f80a549a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80a01006c0 0x7f80a0198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:40.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.716+0000 7f80a549a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80a0107280 0x7f80a0198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:40.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.716+0000 7f80a549a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80a019cf90 0x7f80a019d440 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:40.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.716+0000 7f80a549a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f80a01121c0 con 0x7f80a01006c0 2024-04-21T08:49:40.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.716+0000 7f80a549a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f80a0112040 con 0x7f80a0107280 2024-04-21T08:49:40.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.716+0000 7f80a549a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f80a0112340 con 0x7f80a019cf90 2024-04-21T08:49:40.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.716+0000 7f809effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80a01006c0 0x7f80a0198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:40.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.716+0000 7f809effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80a01006c0 0x7f80a0198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35814/0 (socket says 172.21.15.86:35814) 2024-04-21T08:49:40.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.716+0000 7f809effd640 1 -- 172.21.15.86:0/1041177165 learned_addr learned my addr 172.21.15.86:0/1041177165 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:40.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.716+0000 7f809f7fe640 1 --2- 172.21.15.86:0/1041177165 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80a019cf90 0x7f80a019d440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:40.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.716+0000 7f809e7fc640 1 --2- 172.21.15.86:0/1041177165 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80a0107280 0x7f80a0198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:40.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.716+0000 7f809effd640 1 -- 172.21.15.86:0/1041177165 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80a019cf90 msgr2=0x7f80a019d440 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:40.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.716+0000 7f809effd640 1 --2- 172.21.15.86:0/1041177165 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80a019cf90 0x7f80a019d440 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:40.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.716+0000 7f809effd640 1 -- 172.21.15.86:0/1041177165 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80a0107280 msgr2=0x7f80a0198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:40.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.716+0000 7f809effd640 1 --2- 172.21.15.86:0/1041177165 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80a0107280 0x7f80a0198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:40.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.716+0000 7f809effd640 1 -- 172.21.15.86:0/1041177165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f80a019da80 con 0x7f80a01006c0 2024-04-21T08:49:40.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.716+0000 7f809effd640 1 --2- 172.21.15.86:0/1041177165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80a01006c0 0x7f80a0198720 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f809000c920 tx=0x7f809000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:40.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.716+0000 7f8087fff640 1 -- 172.21.15.86:0/1041177165 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8090007c70 con 0x7f80a01006c0 2024-04-21T08:49:40.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.716+0000 7f80a549a640 1 -- 172.21.15.86:0/1041177165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f80a019dd70 con 0x7f80a01006c0 2024-04-21T08:49:40.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.716+0000 7f8087fff640 1 -- 172.21.15.86:0/1041177165 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8090007e10 con 0x7f80a01006c0 2024-04-21T08:49:40.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.716+0000 7f8087fff640 1 -- 172.21.15.86:0/1041177165 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8090018820 con 0x7f80a01006c0 2024-04-21T08:49:40.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.717+0000 7f80a549a640 1 -- 172.21.15.86:0/1041177165 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f80a01a5530 con 0x7f80a01006c0 2024-04-21T08:49:40.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.719+0000 7f8087fff640 1 -- 172.21.15.86:0/1041177165 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f809000f450 con 0x7f80a01006c0 2024-04-21T08:49:40.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.719+0000 7f80a816b740 1 Processor -- start 2024-04-21T08:49:40.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.719+0000 7f80a816b740 1 -- start start 2024-04-21T08:49:40.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.719+0000 7f80a816b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563efdc65580 0x563efdc65960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:40.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.719+0000 7f80a816b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563efdc65ea0 0x563efdc6e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:40.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.719+0000 7f80a816b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563efdc6ea80 0x563efdc70e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:40.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.719+0000 7f80a816b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563efdbca1e0 con 0x563efdc65ea0 2024-04-21T08:49:40.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.719+0000 7f80a816b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x563efdb67390 con 0x563efdc65580 2024-04-21T08:49:40.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.719+0000 7f80a816b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563efdbc61a0 con 0x563efdc6ea80 2024-04-21T08:49:40.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.719+0000 7f809effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563efdc65ea0 0x563efdc6e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:40.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.719+0000 7f809effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563efdc65ea0 0x563efdc6e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35824/0 (socket says 172.21.15.86:35824) 2024-04-21T08:49:40.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.719+0000 7f809effd640 1 -- 172.21.15.86:0/3003234466 learned_addr learned my addr 172.21.15.86:0/3003234466 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:40.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.719+0000 7f8087fff640 1 --2- 172.21.15.86:0/1041177165 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f80880a3d20 0x7f80880a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:40.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.719+0000 7f809f7fe640 1 --2- 172.21.15.86:0/3003234466 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563efdc65580 0x563efdc65960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:40.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.720+0000 7f809f7fe640 1 --2- 172.21.15.86:0/1041177165 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f80880a3d20 0x7f80880a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:40.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.720+0000 7f809e7fc640 1 --2- 172.21.15.86:0/3003234466 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563efdc6ea80 0x563efdc70e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:40.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.720+0000 7f8087fff640 1 -- 172.21.15.86:0/1041177165 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f80900d0cf0 con 0x7f80a01006c0 2024-04-21T08:49:40.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.720+0000 7f809effd640 1 -- 172.21.15.86:0/3003234466 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563efdc6ea80 msgr2=0x563efdc70e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:40.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.720+0000 7f809effd640 1 --2- 172.21.15.86:0/3003234466 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563efdc6ea80 0x563efdc70e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:40.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.720+0000 7f809effd640 1 -- 172.21.15.86:0/3003234466 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563efdc65580 msgr2=0x563efdc65960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:40.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.720+0000 7f809effd640 1 --2- 172.21.15.86:0/3003234466 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563efdc65580 0x563efdc65960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:40.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.720+0000 7f809effd640 1 -- 172.21.15.86:0/3003234466 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563efd9a67d0 con 0x563efdc65ea0 2024-04-21T08:49:40.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.720+0000 7f809f7fe640 1 --2- 172.21.15.86:0/3003234466 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563efdc65580 0x563efdc65960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:49:40.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.720+0000 7f809effd640 1 --2- 172.21.15.86:0/3003234466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563efdc65ea0 0x563efdc6e540 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f8090099290 tx=0x7f8090098560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc09d3850217953a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:40.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.720+0000 7f809f7fe640 1 --2- 172.21.15.86:0/1041177165 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f80880a3d20 0x7f80880a61e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8098005fd0 tx=0x7f8098004ab0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:40.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.720+0000 7f80857fa640 1 -- 172.21.15.86:0/3003234466 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f809009cfc0 con 0x563efdc65ea0 2024-04-21T08:49:40.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.720+0000 7f80857fa640 1 -- 172.21.15.86:0/3003234466 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f809009c8c0 con 0x563efdc65ea0 2024-04-21T08:49:40.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.720+0000 7f80857fa640 1 -- 172.21.15.86:0/3003234466 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f80900ce2f0 con 0x563efdc65ea0 2024-04-21T08:49:40.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.721+0000 7f80a816b740 1 -- 172.21.15.86:0/3003234466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563efdc65ea0 msgr2=0x563efdc6e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:40.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.721+0000 7f80a816b740 1 --2- 172.21.15.86:0/3003234466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563efdc65ea0 0x563efdc6e540 secure :-1 s=READY pgs=1898 cs=0 l=1 rev1=1 crypto rx=0x7f8090099290 tx=0x7f8090098560 comp rx=0 tx=0).stop 2024-04-21T08:49:40.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.721+0000 7f80a816b740 1 -- 172.21.15.86:0/3003234466 shutdown_connections 2024-04-21T08:49:40.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.721+0000 7f80a816b740 1 --2- 172.21.15.86:0/3003234466 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563efdc6ea80 0x563efdc70e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:40.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.721+0000 7f80a816b740 1 --2- 172.21.15.86:0/3003234466 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563efdc65ea0 0x563efdc6e540 unknown :-1 s=CLOSED pgs=1898 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:40.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.721+0000 7f80a816b740 1 --2- 172.21.15.86:0/3003234466 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563efdc65580 0x563efdc65960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:40.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.721+0000 7f80a816b740 1 -- 172.21.15.86:0/3003234466 >> 172.21.15.86:0/3003234466 conn(0x563efdc5fcb0 msgr2=0x563efdc60090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:40.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.721+0000 7f80a816b740 1 -- 172.21.15.86:0/3003234466 shutdown_connections 2024-04-21T08:49:40.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.721+0000 7f80a816b740 1 -- 172.21.15.86:0/3003234466 wait complete. 2024-04-21T08:49:40.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.721+0000 7f80a816b740 1 -- start start 2024-04-21T08:49:40.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.721+0000 7f80a816b740 1 Processor -- start 2024-04-21T08:49:40.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.722+0000 7f80a816b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563efdc65580 0x563efdc6baa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:40.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.722+0000 7f80a816b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563efdc65ea0 0x563efdc6bfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:40.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.722+0000 7f80a816b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563efdc6ea80 0x563efdc6c520 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:40.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.722+0000 7f80a816b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563efdc71810 con 0x563efdc6ea80 2024-04-21T08:49:40.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.722+0000 7f80a816b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x563efdb832f0 con 0x563efdc65ea0 2024-04-21T08:49:40.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.722+0000 7f80a816b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563efdc71990 con 0x563efdc65580 2024-04-21T08:49:40.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.722+0000 7f80a816b740 10 client.0 _ll_register_callbacks cb 0x563efdb90580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:40.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.722+0000 7f809f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563efdc6ea80 0x563efdc6c520 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:40.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.722+0000 7f809f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563efdc6ea80 0x563efdc6c520 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35840/0 (socket says 172.21.15.86:35840) 2024-04-21T08:49:40.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.722+0000 7f809f7fe640 1 -- 172.21.15.86:0/3199092127 learned_addr learned my addr 172.21.15.86:0/3199092127 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:40.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.722+0000 7f809e7fc640 1 --2- 172.21.15.86:0/3199092127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563efdc65ea0 0x563efdc6bfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:40.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.722+0000 7f809effd640 1 --2- 172.21.15.86:0/3199092127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563efdc65580 0x563efdc6baa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:40.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.722+0000 7f809f7fe640 1 -- 172.21.15.86:0/3199092127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563efdc65580 msgr2=0x563efdc6baa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:40.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.722+0000 7f809f7fe640 1 --2- 172.21.15.86:0/3199092127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563efdc65580 0x563efdc6baa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:40.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.722+0000 7f809f7fe640 1 -- 172.21.15.86:0/3199092127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563efdc65ea0 msgr2=0x563efdc6bfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:40.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.722+0000 7f809f7fe640 1 --2- 172.21.15.86:0/3199092127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563efdc65ea0 0x563efdc6bfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:40.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.722+0000 7f809f7fe640 1 -- 172.21.15.86:0/3199092127 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563efd8dc3f0 con 0x563efdc6ea80 2024-04-21T08:49:40.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.722+0000 7f809effd640 1 --2- 172.21.15.86:0/3199092127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563efdc65580 0x563efdc6baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:40.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.722+0000 7f809f7fe640 1 --2- 172.21.15.86:0/3199092127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563efdc6ea80 0x563efdc6c520 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f809800ce80 tx=0x7f8098004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c85756579a7119c8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:40.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.722+0000 7f8082ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:40.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.722+0000 7f8082ff5640 1 -- 172.21.15.86:0/3199092127 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8098026070 con 0x563efdc6ea80 2024-04-21T08:49:40.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.722+0000 7f8082ff5640 1 -- 172.21.15.86:0/3199092127 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f8098011df0 con 0x563efdc6ea80 2024-04-21T08:49:40.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.722+0000 7f80a816b740 1 -- 172.21.15.86:0/3199092127 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f809009c8c0 con 0x563efdc6ea80 2024-04-21T08:49:40.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.722+0000 7f8082ff5640 1 -- 172.21.15.86:0/3199092127 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8098032050 con 0x563efdc6ea80 2024-04-21T08:49:40.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.722+0000 7f80a816b740 10 client.19926 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:40.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.722+0000 7f80a816b740 1 -- 172.21.15.86:0/3199092127 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563efd89c0f0 con 0x563efdc6ea80 2024-04-21T08:49:40.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.723+0000 7f8082ff5640 1 -- 172.21.15.86:0/3199092127 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f809009c8c0 con 0x563efdc6ea80 2024-04-21T08:49:40.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.723+0000 7f8082ff5640 1 -- 172.21.15.86:0/3199092127 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f809800f040 con 0x563efdc6ea80 2024-04-21T08:49:40.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.723+0000 7f80a816b740 10 client.19926 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:40.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.723+0000 7f80a816b740 10 client.19926 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:40.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.723+0000 7f80a816b740 20 client.19926 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:40.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.723+0000 7f80a816b740 1 --2- 172.21.15.86:0/3199092127 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563efdc7e9f0 0x563efdc80e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:40.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.723+0000 7f80a816b740 4 client.19926 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:40.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.723+0000 7f80a816b740 1 -- 172.21.15.86:0/3199092127 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563efdbd7c00 con 0x563efdc7e9f0 2024-04-21T08:49:40.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.723+0000 7f809effd640 1 --2- 172.21.15.86:0/3199092127 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563efdc7e9f0 0x563efdc80e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:40.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.724+0000 7f8082ff5640 10 client.19926 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:40.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.724+0000 7f809effd640 1 --2- 172.21.15.86:0/3199092127 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563efdc7e9f0 0x563efdc80e90 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a0473031423fb2c2 server_cookie=fdfc5f570fc12c75 in_seq=0 out_seq=0 2024-04-21T08:49:40.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.724+0000 7f8082ff5640 1 -- 172.21.15.86:0/3199092127 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563efdbd7c00 con 0x563efdc7e9f0 2024-04-21T08:49:40.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.724+0000 7f8082ff5640 10 client.19926 handle_command_reply: tid=0 2024-04-21T08:49:40.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.724+0000 7f80a816b740 1 client.19926 shutdown 2024-04-21T08:49:40.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.724+0000 7f80a816b740 1 -- 172.21.15.86:0/3199092127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563efdc6ea80 msgr2=0x563efdc6c520 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:40.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.724+0000 7f80a816b740 1 --2- 172.21.15.86:0/3199092127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563efdc6ea80 0x563efdc6c520 secure :-1 s=READY pgs=1899 cs=0 l=1 rev1=1 crypto rx=0x7f809800ce80 tx=0x7f8098004a40 comp rx=0 tx=0).stop 2024-04-21T08:49:40.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.724+0000 7f80a816b740 1 -- 172.21.15.86:0/3199092127 shutdown_connections 2024-04-21T08:49:40.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.724+0000 7f80a816b740 1 --2- 172.21.15.86:0/3199092127 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563efdc7e9f0 0x563efdc80e90 crc :-1 s=READY pgs=313 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:40.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.724+0000 7f80a816b740 1 --2- 172.21.15.86:0/3199092127 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563efdc6ea80 0x563efdc6c520 unknown :-1 s=CLOSED pgs=1899 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:40.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.724+0000 7f8082ff5640 0 client.19926 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:40.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.724+0000 7f80a816b740 1 --2- 172.21.15.86:0/3199092127 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563efdc65ea0 0x563efdc6bfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:40.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.724+0000 7f80a816b740 1 --2- 172.21.15.86:0/3199092127 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563efdc65580 0x563efdc6baa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:40.731 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.724+0000 7f80a816b740 1 -- 172.21.15.86:0/3199092127 >> 172.21.15.86:0/3199092127 conn(0x563efdc5fcb0 msgr2=0x563efdc61a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:40.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.725+0000 7f80a816b740 1 -- 172.21.15.86:0/3199092127 shutdown_connections 2024-04-21T08:49:40.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.725+0000 7f80a816b740 1 -- 172.21.15.86:0/3199092127 wait complete. 2024-04-21T08:49:40.732 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.725+0000 7f80a816b740 20 client.19926 trim_cache size 0 max 16384 2024-04-21T08:49:40.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.742+0000 7f80a816b740 1 Processor -- start 2024-04-21T08:49:40.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.742+0000 7f80a816b740 1 -- start start 2024-04-21T08:49:40.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.742+0000 7f80a816b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563efdc65580 0x563efdc6e1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:40.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.742+0000 7f80a816b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563efdc65ea0 0x563efdc7a6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:40.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.742+0000 7f80a816b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563efdc6ea80 0x563efdc6ac00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:40.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.742+0000 7f80a816b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f80a01af2e0 con 0x563efdc65ea0 2024-04-21T08:49:40.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.742+0000 7f80a816b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f80a01b0e20 con 0x563efdc65580 2024-04-21T08:49:40.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.742+0000 7f80a816b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f80a01af9b0 con 0x563efdc6ea80 2024-04-21T08:49:40.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.742+0000 7f809e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563efdc65ea0 0x563efdc7a6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:40.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.742+0000 7f809e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563efdc65ea0 0x563efdc7a6d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35846/0 (socket says 172.21.15.86:35846) 2024-04-21T08:49:40.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.742+0000 7f809e7fc640 1 -- 172.21.15.86:0/41552356 learned_addr learned my addr 172.21.15.86:0/41552356 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:40.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.742+0000 7f809effd640 1 --2- 172.21.15.86:0/41552356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563efdc65580 0x563efdc6e1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:40.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.742+0000 7f809e7fc640 1 -- 172.21.15.86:0/41552356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563efdc6ea80 msgr2=0x563efdc6ac00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:40.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.742+0000 7f809f7fe640 1 --2- 172.21.15.86:0/41552356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563efdc6ea80 0x563efdc6ac00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:40.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.742+0000 7f809e7fc640 1 --2- 172.21.15.86:0/41552356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563efdc6ea80 0x563efdc6ac00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:40.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.742+0000 7f809e7fc640 1 -- 172.21.15.86:0/41552356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563efdc65580 msgr2=0x563efdc6e1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:40.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.742+0000 7f809e7fc640 1 --2- 172.21.15.86:0/41552356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563efdc65580 0x563efdc6e1d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:40.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.742+0000 7f809e7fc640 1 -- 172.21.15.86:0/41552356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563efdc70e70 con 0x563efdc65ea0 2024-04-21T08:49:40.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.742+0000 7f809f7fe640 1 --2- 172.21.15.86:0/41552356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563efdc6ea80 0x563efdc6ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:40.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.742+0000 7f809e7fc640 1 --2- 172.21.15.86:0/41552356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563efdc65ea0 0x563efdc7a6d0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f80a00746d0 tx=0x7f809402c680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e0769a4d9341fb48 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:40.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.742+0000 7f8084ff9640 1 -- 172.21.15.86:0/41552356 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8094004340 con 0x563efdc65ea0 2024-04-21T08:49:40.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.743+0000 7f8084ff9640 1 -- 172.21.15.86:0/41552356 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f809402cc90 con 0x563efdc65ea0 2024-04-21T08:49:40.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.743+0000 7f8084ff9640 1 -- 172.21.15.86:0/41552356 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8094004920 con 0x563efdc65ea0 2024-04-21T08:49:40.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.743+0000 7f80a816b740 1 -- 172.21.15.86:0/41552356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563efdc65ea0 msgr2=0x563efdc7a6d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:40.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.743+0000 7f80a816b740 1 --2- 172.21.15.86:0/41552356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563efdc65ea0 0x563efdc7a6d0 secure :-1 s=READY pgs=1900 cs=0 l=1 rev1=1 crypto rx=0x7f80a00746d0 tx=0x7f809402c680 comp rx=0 tx=0).stop 2024-04-21T08:49:40.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.743+0000 7f80a816b740 1 -- 172.21.15.86:0/41552356 shutdown_connections 2024-04-21T08:49:40.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.743+0000 7f80a816b740 1 --2- 172.21.15.86:0/41552356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563efdc6ea80 0x563efdc6ac00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:40.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.743+0000 7f80a816b740 1 --2- 172.21.15.86:0/41552356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563efdc65ea0 0x563efdc7a6d0 unknown :-1 s=CLOSED pgs=1900 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:40.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.743+0000 7f80a816b740 1 --2- 172.21.15.86:0/41552356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563efdc65580 0x563efdc6e1d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:40.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.743+0000 7f80a816b740 1 -- 172.21.15.86:0/41552356 >> 172.21.15.86:0/41552356 conn(0x563efdc5fcb0 msgr2=0x563efdc610e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:40.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.743+0000 7f80a816b740 1 -- 172.21.15.86:0/41552356 shutdown_connections 2024-04-21T08:49:40.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.743+0000 7f80a816b740 1 -- 172.21.15.86:0/41552356 wait complete. 2024-04-21T08:49:40.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.743+0000 7f80a816b740 1 -- start start 2024-04-21T08:49:40.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.744+0000 7f80a816b740 1 Processor -- start 2024-04-21T08:49:40.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.744+0000 7f80a816b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563efdc65580 0x563efdc77030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:40.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.744+0000 7f80a816b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563efdc65ea0 0x563efdc723f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:40.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.744+0000 7f80a816b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563efdc6ea80 0x563efdc72970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:40.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.744+0000 7f80a816b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f80a01b0750 con 0x563efdc65ea0 2024-04-21T08:49:40.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.744+0000 7f80a816b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f80a01b0080 con 0x563efdc6ea80 2024-04-21T08:49:40.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.744+0000 7f80a816b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f80a01ad2e0 con 0x563efdc65580 2024-04-21T08:49:40.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.744+0000 7f80a816b740 10 client.0 _ll_register_callbacks cb 0x7f8098009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:40.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.744+0000 7f809e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563efdc65ea0 0x563efdc723f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:40.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.744+0000 7f809e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563efdc65ea0 0x563efdc723f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35852/0 (socket says 172.21.15.86:35852) 2024-04-21T08:49:40.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.744+0000 7f809e7fc640 1 -- 172.21.15.86:0/1174111928 learned_addr learned my addr 172.21.15.86:0/1174111928 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:40.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.744+0000 7f809effd640 1 --2- 172.21.15.86:0/1174111928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563efdc65580 0x563efdc77030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:40.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.744+0000 7f809e7fc640 1 -- 172.21.15.86:0/1174111928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563efdc65580 msgr2=0x563efdc77030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:40.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.744+0000 7f809f7fe640 1 --2- 172.21.15.86:0/1174111928 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563efdc6ea80 0x563efdc72970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:40.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.744+0000 7f809e7fc640 1 --2- 172.21.15.86:0/1174111928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563efdc65580 0x563efdc77030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:40.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.744+0000 7f809e7fc640 1 -- 172.21.15.86:0/1174111928 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563efdc6ea80 msgr2=0x563efdc72970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:40.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.744+0000 7f809e7fc640 1 --2- 172.21.15.86:0/1174111928 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563efdc6ea80 0x563efdc72970 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:40.750 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.744+0000 7f809e7fc640 1 -- 172.21.15.86:0/1174111928 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563efdc731a0 con 0x563efdc65ea0 2024-04-21T08:49:40.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.744+0000 7f809e7fc640 1 --2- 172.21.15.86:0/1174111928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563efdc65ea0 0x563efdc723f0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f8094036040 tx=0x7f8094004460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee57b5cd47667c85 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:40.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.744+0000 7f8082ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:40.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.744+0000 7f8082ff5640 1 -- 172.21.15.86:0/1174111928 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8094005460 con 0x563efdc65ea0 2024-04-21T08:49:40.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.744+0000 7f8082ff5640 1 -- 172.21.15.86:0/1174111928 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f80940046c0 con 0x563efdc65ea0 2024-04-21T08:49:40.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.744+0000 7f80a816b740 1 -- 172.21.15.86:0/1174111928 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f809402cc90 con 0x563efdc65ea0 2024-04-21T08:49:40.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.745+0000 7f8082ff5640 1 -- 172.21.15.86:0/1174111928 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f809405b050 con 0x563efdc65ea0 2024-04-21T08:49:40.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.745+0000 7f80a816b740 10 client.19932 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:40.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.745+0000 7f80a816b740 1 -- 172.21.15.86:0/1174111928 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563efdce4ff0 con 0x563efdc65ea0 2024-04-21T08:49:40.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.745+0000 7f8082ff5640 1 -- 172.21.15.86:0/1174111928 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f809402cc90 con 0x563efdc65ea0 2024-04-21T08:49:40.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.745+0000 7f8082ff5640 1 -- 172.21.15.86:0/1174111928 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8094003ad0 con 0x563efdc65ea0 2024-04-21T08:49:40.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.745+0000 7f80a816b740 10 client.19932 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:40.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.745+0000 7f80a816b740 10 client.19932 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:40.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.745+0000 7f80a816b740 20 client.19932 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:40.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.745+0000 7f80a816b740 1 --2- 172.21.15.86:0/1174111928 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563efdc7e9f0 0x563efdce7730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:40.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.745+0000 7f80a816b740 4 client.19932 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:49:40.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.745+0000 7f80a816b740 1 -- 172.21.15.86:0/1174111928 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x563efdbd1a70 con 0x563efdc7e9f0 2024-04-21T08:49:40.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.745+0000 7f809effd640 1 --2- 172.21.15.86:0/1174111928 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563efdc7e9f0 0x563efdce7730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:40.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.745+0000 7f809effd640 1 --2- 172.21.15.86:0/1174111928 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563efdc7e9f0 0x563efdce7730 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f5e9a334a410d90e server_cookie=e6640f1f4cacfa6b in_seq=0 out_seq=0 2024-04-21T08:49:40.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.745+0000 7f8082ff5640 10 client.19932 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:40.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.746+0000 7f8082ff5640 1 -- 172.21.15.86:0/1174111928 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x563efdbd1a70 con 0x563efdc7e9f0 2024-04-21T08:49:40.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.746+0000 7f8082ff5640 10 client.19932 handle_command_reply: tid=0 2024-04-21T08:49:40.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.746+0000 7f80a816b740 1 client.19932 shutdown 2024-04-21T08:49:40.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.746+0000 7f80a816b740 1 -- 172.21.15.86:0/1174111928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563efdc65ea0 msgr2=0x563efdc723f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:40.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.746+0000 7f80a816b740 1 --2- 172.21.15.86:0/1174111928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563efdc65ea0 0x563efdc723f0 secure :-1 s=READY pgs=1901 cs=0 l=1 rev1=1 crypto rx=0x7f8094036040 tx=0x7f8094004460 comp rx=0 tx=0).stop 2024-04-21T08:49:40.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.746+0000 7f80a816b740 1 -- 172.21.15.86:0/1174111928 shutdown_connections 2024-04-21T08:49:40.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.746+0000 7f80a816b740 1 --2- 172.21.15.86:0/1174111928 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563efdc7e9f0 0x563efdce7730 crc :-1 s=READY pgs=314 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:40.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.746+0000 7f80a816b740 1 --2- 172.21.15.86:0/1174111928 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563efdc6ea80 0x563efdc72970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:40.752 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.746+0000 7f8082ff5640 0 client.19932 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:40.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.746+0000 7f80a816b740 1 --2- 172.21.15.86:0/1174111928 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563efdc65ea0 0x563efdc723f0 unknown :-1 s=CLOSED pgs=1901 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:40.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.746+0000 7f80a816b740 1 --2- 172.21.15.86:0/1174111928 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563efdc65580 0x563efdc77030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:40.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.746+0000 7f80a816b740 1 -- 172.21.15.86:0/1174111928 >> 172.21.15.86:0/1174111928 conn(0x563efdc5fcb0 msgr2=0x563efdc610e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:40.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.746+0000 7f80a816b740 1 -- 172.21.15.86:0/1174111928 shutdown_connections 2024-04-21T08:49:40.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.746+0000 7f80a816b740 1 -- 172.21.15.86:0/1174111928 wait complete. 2024-04-21T08:49:40.753 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.746+0000 7f80a816b740 20 client.19932 trim_cache size 0 max 16384 2024-04-21T08:49:40.753 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:49:40.753 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-21T08:49:40.753 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-21T08:49:40.753 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:49:40.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.746+0000 7f80a549a640 1 -- 172.21.15.86:0/1041177165 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f80880a3d20 msgr2=0x7f80880a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:40.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.746+0000 7f80a549a640 1 --2- 172.21.15.86:0/1041177165 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f80880a3d20 0x7f80880a61e0 secure :-1 s=READY pgs=315 cs=0 l=1 rev1=1 crypto rx=0x7f8098005fd0 tx=0x7f8098004ab0 comp rx=0 tx=0).stop 2024-04-21T08:49:40.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.746+0000 7f80a549a640 1 -- 172.21.15.86:0/1041177165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80a01006c0 msgr2=0x7f80a0198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:40.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.746+0000 7f80a549a640 1 --2- 172.21.15.86:0/1041177165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80a01006c0 0x7f80a0198720 secure :-1 s=READY pgs=1897 cs=0 l=1 rev1=1 crypto rx=0x7f809000c920 tx=0x7f809000cde0 comp rx=0 tx=0).stop 2024-04-21T08:49:40.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.746+0000 7f80a549a640 1 -- 172.21.15.86:0/1041177165 shutdown_connections 2024-04-21T08:49:40.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.746+0000 7f80a549a640 1 --2- 172.21.15.86:0/1041177165 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f80880a3d20 0x7f80880a61e0 unknown :-1 s=CLOSED pgs=315 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:40.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.746+0000 7f80a549a640 1 --2- 172.21.15.86:0/1041177165 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f80a019cf90 0x7f80a019d440 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:40.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.746+0000 7f80a549a640 1 --2- 172.21.15.86:0/1041177165 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f80a0107280 0x7f80a0198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:40.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.746+0000 7f80a549a640 1 --2- 172.21.15.86:0/1041177165 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f80a01006c0 0x7f80a0198720 unknown :-1 s=CLOSED pgs=1897 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:40.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.746+0000 7f80a549a640 1 -- 172.21.15.86:0/1041177165 >> 172.21.15.86:0/1041177165 conn(0x7f80a00773a0 msgr2=0x7f80a01053b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:40.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.747+0000 7f80a549a640 1 -- 172.21.15.86:0/1041177165 shutdown_connections 2024-04-21T08:49:40.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.747+0000 7f80a549a640 1 -- 172.21.15.86:0/1041177165 wait complete. 2024-04-21T08:49:40.756 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:49:40.757 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:49:40.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.855+0000 7efc9510e640 1 -- 172.21.15.86:0/47993579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc90101b60 msgr2=0x7efc9010dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:40.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.855+0000 7efc9510e640 1 --2- 172.21.15.86:0/47993579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc90101b60 0x7efc9010dc90 secure :-1 s=READY pgs=1902 cs=0 l=1 rev1=1 crypto rx=0x7efc80009930 tx=0x7efc8002bc30 comp rx=0 tx=0).stop 2024-04-21T08:49:40.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.855+0000 7efc9510e640 1 -- 172.21.15.86:0/47993579 shutdown_connections 2024-04-21T08:49:40.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.855+0000 7efc9510e640 1 --2- 172.21.15.86:0/47993579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc90101b60 0x7efc9010dc90 unknown :-1 s=CLOSED pgs=1902 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:40.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.855+0000 7efc9510e640 1 --2- 172.21.15.86:0/47993579 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc901011a0 0x7efc90101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:40.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.855+0000 7efc9510e640 1 --2- 172.21.15.86:0/47993579 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc90105e30 0x7efc90106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:40.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.855+0000 7efc9510e640 1 -- 172.21.15.86:0/47993579 >> 172.21.15.86:0/47993579 conn(0x7efc90077e30 msgr2=0x7efc90078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:40.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.855+0000 7efc9510e640 1 -- 172.21.15.86:0/47993579 shutdown_connections 2024-04-21T08:49:40.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.855+0000 7efc9510e640 1 -- 172.21.15.86:0/47993579 wait complete. 2024-04-21T08:49:40.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.856+0000 7efc9510e640 1 Processor -- start 2024-04-21T08:49:40.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.856+0000 7efc9510e640 1 -- start start 2024-04-21T08:49:40.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.856+0000 7efc9510e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc901011a0 0x7efc90198640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:40.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.856+0000 7efc9510e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc90101b60 0x7efc90198b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:40.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.856+0000 7efc9510e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc90105e30 0x7efc9019cf00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:40.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.856+0000 7efc9510e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efc90112200 con 0x7efc901011a0 2024-04-21T08:49:40.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.856+0000 7efc9510e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efc90112080 con 0x7efc90101b60 2024-04-21T08:49:40.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.856+0000 7efc9510e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7efc90112380 con 0x7efc90105e30 2024-04-21T08:49:40.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.857+0000 7efc8ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc901011a0 0x7efc90198640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:40.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.857+0000 7efc8ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc901011a0 0x7efc90198640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35864/0 (socket says 172.21.15.86:35864) 2024-04-21T08:49:40.858 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.857+0000 7efc8ed9d640 1 -- 172.21.15.86:0/2603560076 learned_addr learned my addr 172.21.15.86:0/2603560076 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:40.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.857+0000 7efc8ed9d640 1 -- 172.21.15.86:0/2603560076 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc90105e30 msgr2=0x7efc9019cf00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:40.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.857+0000 7efc8e59c640 1 --2- 172.21.15.86:0/2603560076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc90101b60 0x7efc90198b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:40.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.857+0000 7efc8ed9d640 1 --2- 172.21.15.86:0/2603560076 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc90105e30 0x7efc9019cf00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:40.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.857+0000 7efc8ed9d640 1 -- 172.21.15.86:0/2603560076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc90101b60 msgr2=0x7efc90198b80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:40.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.857+0000 7efc8ed9d640 1 --2- 172.21.15.86:0/2603560076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc90101b60 0x7efc90198b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:40.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.857+0000 7efc8ed9d640 1 -- 172.21.15.86:0/2603560076 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc9019d630 con 0x7efc901011a0 2024-04-21T08:49:40.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.857+0000 7efc8ed9d640 1 --2- 172.21.15.86:0/2603560076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc901011a0 0x7efc90198640 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7efc8800c920 tx=0x7efc8800cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:40.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.857+0000 7efc77fff640 1 -- 172.21.15.86:0/2603560076 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc88007c70 con 0x7efc901011a0 2024-04-21T08:49:40.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.857+0000 7efc77fff640 1 -- 172.21.15.86:0/2603560076 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc88007e10 con 0x7efc901011a0 2024-04-21T08:49:40.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.857+0000 7efc9510e640 1 -- 172.21.15.86:0/2603560076 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc9019d920 con 0x7efc901011a0 2024-04-21T08:49:40.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.857+0000 7efc77fff640 1 -- 172.21.15.86:0/2603560076 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc88018820 con 0x7efc901011a0 2024-04-21T08:49:40.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.857+0000 7efc9510e640 1 -- 172.21.15.86:0/2603560076 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc901a51b0 con 0x7efc901011a0 2024-04-21T08:49:40.859 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.858+0000 7efc9510e640 1 -- 172.21.15.86:0/2603560076 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7efc90068de0 con 0x7efc901011a0 2024-04-21T08:49:40.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.859+0000 7efc77fff640 1 -- 172.21.15.86:0/2603560076 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efc8800f450 con 0x7efc901011a0 2024-04-21T08:49:40.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.860+0000 7efc77fff640 1 --2- 172.21.15.86:0/2603560076 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efc780a3d20 0x7efc780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:40.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.860+0000 7efc8e59c640 1 --2- 172.21.15.86:0/2603560076 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efc780a3d20 0x7efc780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:40.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.860+0000 7efc77fff640 1 -- 172.21.15.86:0/2603560076 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7efc880d0da0 con 0x7efc901011a0 2024-04-21T08:49:40.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.860+0000 7efc8e59c640 1 --2- 172.21.15.86:0/2603560076 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efc780a3d20 0x7efc780a61e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7efc84005c40 tx=0x7efc84005bd0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:40.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:40.865+0000 7efc77fff640 1 -- 172.21.15.86:0/2603560076 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7efc88014030 con 0x7efc901011a0 2024-04-21T08:49:41.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.084+0000 7efc9510e640 1 -- 172.21.15.86:0/2603560076 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7efc9002d5c0 con 0x7efc901011a0 2024-04-21T08:49:41.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.085+0000 7efc77fff640 1 -- 172.21.15.86:0/2603560076 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7efc8800de20 con 0x7efc901011a0 2024-04-21T08:49:41.084 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:49:41.085 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:38:45.202126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24398,24394,24455,24472]},"id":1}]} 2024-04-21T08:49:41.085 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-21T08:49:41.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.087+0000 7efc9510e640 1 -- 172.21.15.86:0/2603560076 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efc780a3d20 msgr2=0x7efc780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.087+0000 7efc9510e640 1 --2- 172.21.15.86:0/2603560076 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efc780a3d20 0x7efc780a61e0 secure :-1 s=READY pgs=316 cs=0 l=1 rev1=1 crypto rx=0x7efc84005c40 tx=0x7efc84005bd0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.087+0000 7efc9510e640 1 -- 172.21.15.86:0/2603560076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc901011a0 msgr2=0x7efc90198640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.087+0000 7efc9510e640 1 --2- 172.21.15.86:0/2603560076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc901011a0 0x7efc90198640 secure :-1 s=READY pgs=1903 cs=0 l=1 rev1=1 crypto rx=0x7efc8800c920 tx=0x7efc8800cde0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.087+0000 7efc9510e640 1 -- 172.21.15.86:0/2603560076 shutdown_connections 2024-04-21T08:49:41.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.087+0000 7efc9510e640 1 --2- 172.21.15.86:0/2603560076 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efc780a3d20 0x7efc780a61e0 unknown :-1 s=CLOSED pgs=316 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.087+0000 7efc9510e640 1 --2- 172.21.15.86:0/2603560076 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc90105e30 0x7efc9019cf00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.087+0000 7efc9510e640 1 --2- 172.21.15.86:0/2603560076 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc90101b60 0x7efc90198b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.087+0000 7efc9510e640 1 --2- 172.21.15.86:0/2603560076 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc901011a0 0x7efc90198640 unknown :-1 s=CLOSED pgs=1903 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.087+0000 7efc9510e640 1 -- 172.21.15.86:0/2603560076 >> 172.21.15.86:0/2603560076 conn(0x7efc90077e30 msgr2=0x7efc900fede0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:41.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.087+0000 7efc9510e640 1 -- 172.21.15.86:0/2603560076 shutdown_connections 2024-04-21T08:49:41.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.087+0000 7efc9510e640 1 -- 172.21.15.86:0/2603560076 wait complete. 2024-04-21T08:49:41.098 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:49:41.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.199+0000 7f1434dbd640 1 -- 172.21.15.86:0/193645778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1430102b40 msgr2=0x7f1430106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.199+0000 7f1434dbd640 1 --2- 172.21.15.86:0/193645778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1430102b40 0x7f1430106b80 secure :-1 s=READY pgs=1904 cs=0 l=1 rev1=1 crypto rx=0x7f1424009900 tx=0x7f142402bc30 comp rx=0 tx=0).stop 2024-04-21T08:49:41.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.199+0000 7f1434dbd640 1 -- 172.21.15.86:0/193645778 shutdown_connections 2024-04-21T08:49:41.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.199+0000 7f1434dbd640 1 --2- 172.21.15.86:0/193645778 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1430107260 0x7f143010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.199+0000 7f1434dbd640 1 --2- 172.21.15.86:0/193645778 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1430102b40 0x7f1430106b80 unknown :-1 s=CLOSED pgs=1904 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.199+0000 7f1434dbd640 1 --2- 172.21.15.86:0/193645778 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1430102190 0x7f1430102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.199+0000 7f1434dbd640 1 -- 172.21.15.86:0/193645778 >> 172.21.15.86:0/193645778 conn(0x7f14300793a0 msgr2=0x7f14300797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:41.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.199+0000 7f1434dbd640 1 -- 172.21.15.86:0/193645778 shutdown_connections 2024-04-21T08:49:41.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.199+0000 7f1434dbd640 1 -- 172.21.15.86:0/193645778 wait complete. 2024-04-21T08:49:41.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.200+0000 7f1434dbd640 1 Processor -- start 2024-04-21T08:49:41.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.200+0000 7f1434dbd640 1 -- start start 2024-04-21T08:49:41.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.200+0000 7f1434dbd640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1430102190 0x7f143019e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.200+0000 7f1434dbd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1430107260 0x7f143019e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.200+0000 7f1434dbd640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14301985a0 0x7f1430198a50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.200+0000 7f1434dbd640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14301121a0 con 0x7f1430107260 2024-04-21T08:49:41.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.200+0000 7f1434dbd640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1430112020 con 0x7f14301985a0 2024-04-21T08:49:41.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.200+0000 7f1434dbd640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1430112320 con 0x7f1430102190 2024-04-21T08:49:41.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.200+0000 7f142dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1430107260 0x7f143019e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.200+0000 7f142dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1430107260 0x7f143019e930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35892/0 (socket says 172.21.15.86:35892) 2024-04-21T08:49:41.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.200+0000 7f142dd9b640 1 -- 172.21.15.86:0/4222354932 learned_addr learned my addr 172.21.15.86:0/4222354932 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:41.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.200+0000 7f142ed9d640 1 --2- 172.21.15.86:0/4222354932 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14301985a0 0x7f1430198a50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.200+0000 7f142e59c640 1 --2- 172.21.15.86:0/4222354932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1430102190 0x7f143019e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.200+0000 7f142dd9b640 1 -- 172.21.15.86:0/4222354932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1430102190 msgr2=0x7f143019e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.200+0000 7f142dd9b640 1 --2- 172.21.15.86:0/4222354932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1430102190 0x7f143019e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.200+0000 7f142dd9b640 1 -- 172.21.15.86:0/4222354932 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14301985a0 msgr2=0x7f1430198a50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.200+0000 7f142dd9b640 1 --2- 172.21.15.86:0/4222354932 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14301985a0 0x7f1430198a50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.200+0000 7f142dd9b640 1 -- 172.21.15.86:0/4222354932 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f14301990c0 con 0x7f1430107260 2024-04-21T08:49:41.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.201+0000 7f142dd9b640 1 --2- 172.21.15.86:0/4222354932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1430107260 0x7f143019e930 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f1424036040 tx=0x7f142402c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:41.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.201+0000 7f141f7fe640 1 -- 172.21.15.86:0/4222354932 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1424043020 con 0x7f1430107260 2024-04-21T08:49:41.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.201+0000 7f1434dbd640 1 -- 172.21.15.86:0/4222354932 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1430199350 con 0x7f1430107260 2024-04-21T08:49:41.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.201+0000 7f141f7fe640 1 -- 172.21.15.86:0/4222354932 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f142402cbe0 con 0x7f1430107260 2024-04-21T08:49:41.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.201+0000 7f141f7fe640 1 -- 172.21.15.86:0/4222354932 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1424003bf0 con 0x7f1430107260 2024-04-21T08:49:41.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.201+0000 7f1434dbd640 1 -- 172.21.15.86:0/4222354932 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f14301a55a0 con 0x7f1430107260 2024-04-21T08:49:41.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.203+0000 7f141f7fe640 1 -- 172.21.15.86:0/4222354932 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1424046020 con 0x7f1430107260 2024-04-21T08:49:41.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.203+0000 7f14379b4740 1 Processor -- start 2024-04-21T08:49:41.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.204+0000 7f14379b4740 1 -- start start 2024-04-21T08:49:41.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.204+0000 7f14379b4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89160c8e0 0x55b89160ccc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.204+0000 7f14379b4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b89160d290 0x55b891615840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.204+0000 7f14379b4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b891615e10 0x55b891618260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.204+0000 7f14379b4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b89156d1f0 con 0x55b89160c8e0 2024-04-21T08:49:41.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.204+0000 7f14379b4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b89150a4b0 con 0x55b89160d290 2024-04-21T08:49:41.210 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.204+0000 7f14379b4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b8915691b0 con 0x55b891615e10 2024-04-21T08:49:41.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.204+0000 7f142ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89160c8e0 0x55b89160ccc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.204+0000 7f142ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89160c8e0 0x55b89160ccc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35904/0 (socket says 172.21.15.86:35904) 2024-04-21T08:49:41.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.204+0000 7f142ed9d640 1 -- 172.21.15.86:0/2302919033 learned_addr learned my addr 172.21.15.86:0/2302919033 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:41.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.204+0000 7f142e59c640 1 --2- 172.21.15.86:0/2302919033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b89160d290 0x55b891615840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.204+0000 7f142dd9b640 1 --2- 172.21.15.86:0/2302919033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b891615e10 0x55b891618260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.204+0000 7f141f7fe640 1 --2- 172.21.15.86:0/4222354932 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f14180a3d20 0x7f14180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.204+0000 7f142ed9d640 1 -- 172.21.15.86:0/2302919033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b891615e10 msgr2=0x55b891618260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.204+0000 7f142ed9d640 1 --2- 172.21.15.86:0/2302919033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b891615e10 0x55b891618260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.204+0000 7f142ed9d640 1 -- 172.21.15.86:0/2302919033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b89160d290 msgr2=0x55b891615840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.204+0000 7f142ed9d640 1 --2- 172.21.15.86:0/2302919033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b89160d290 0x55b891615840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.204+0000 7f142ed9d640 1 -- 172.21.15.86:0/2302919033 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8914fc1b0 con 0x55b89160c8e0 2024-04-21T08:49:41.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.204+0000 7f142dd9b640 1 --2- 172.21.15.86:0/2302919033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b891615e10 0x55b891618260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:41.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.204+0000 7f142ed9d640 1 --2- 172.21.15.86:0/4222354932 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f14180a3d20 0x7f14180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.204+0000 7f141f7fe640 1 -- 172.21.15.86:0/4222354932 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f14240f2d60 con 0x7f1430107260 2024-04-21T08:49:41.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.204+0000 7f142ed9d640 1 --2- 172.21.15.86:0/2302919033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89160c8e0 0x55b89160ccc0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f1428010440 tx=0x7f1428010900 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ae1d29f040e6af29 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:41.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.205+0000 7f141cff9640 1 -- 172.21.15.86:0/2302919033 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f142801d400 con 0x55b89160c8e0 2024-04-21T08:49:41.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.205+0000 7f141cff9640 1 -- 172.21.15.86:0/2302919033 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1428013520 con 0x55b89160c8e0 2024-04-21T08:49:41.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.205+0000 7f142ed9d640 1 --2- 172.21.15.86:0/4222354932 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f14180a3d20 0x7f14180a61e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1428021000 tx=0x7f142801e000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:41.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.205+0000 7f141cff9640 1 -- 172.21.15.86:0/2302919033 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1428011920 con 0x55b89160c8e0 2024-04-21T08:49:41.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.205+0000 7f14379b4740 1 -- 172.21.15.86:0/2302919033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89160c8e0 msgr2=0x55b89160ccc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.205+0000 7f14379b4740 1 --2- 172.21.15.86:0/2302919033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89160c8e0 0x55b89160ccc0 secure :-1 s=READY pgs=1906 cs=0 l=1 rev1=1 crypto rx=0x7f1428010440 tx=0x7f1428010900 comp rx=0 tx=0).stop 2024-04-21T08:49:41.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.205+0000 7f14379b4740 1 -- 172.21.15.86:0/2302919033 shutdown_connections 2024-04-21T08:49:41.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.205+0000 7f14379b4740 1 --2- 172.21.15.86:0/2302919033 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b891615e10 0x55b891618260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.205+0000 7f14379b4740 1 --2- 172.21.15.86:0/2302919033 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b89160d290 0x55b891615840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.205+0000 7f14379b4740 1 --2- 172.21.15.86:0/2302919033 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89160c8e0 0x55b89160ccc0 unknown :-1 s=CLOSED pgs=1906 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.205+0000 7f14379b4740 1 -- 172.21.15.86:0/2302919033 >> 172.21.15.86:0/2302919033 conn(0x55b891606fb0 msgr2=0x55b891607390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:41.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.205+0000 7f14379b4740 1 -- 172.21.15.86:0/2302919033 shutdown_connections 2024-04-21T08:49:41.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.206+0000 7f14379b4740 1 -- 172.21.15.86:0/2302919033 wait complete. 2024-04-21T08:49:41.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.206+0000 7f14379b4740 1 -- start start 2024-04-21T08:49:41.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.206+0000 7f14379b4740 1 Processor -- start 2024-04-21T08:49:41.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.206+0000 7f14379b4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b89160c8e0 0x55b891615790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.206+0000 7f14379b4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b89160d290 0x55b891612b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.206+0000 7f14379b4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b891615e10 0x55b891613130 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.206+0000 7f14379b4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b891618d80 con 0x55b891615e10 2024-04-21T08:49:41.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.206+0000 7f14379b4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b891526500 con 0x55b89160d290 2024-04-21T08:49:41.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.206+0000 7f14379b4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b891618f00 con 0x55b89160c8e0 2024-04-21T08:49:41.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.206+0000 7f14379b4740 10 client.0 _ll_register_callbacks cb 0x55b8914f4b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:41.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.206+0000 7f142ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b891615e10 0x55b891613130 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.206+0000 7f142ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b891615e10 0x55b891613130 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35912/0 (socket says 172.21.15.86:35912) 2024-04-21T08:49:41.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.206+0000 7f142ed9d640 1 -- 172.21.15.86:0/1352489728 learned_addr learned my addr 172.21.15.86:0/1352489728 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:41.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.206+0000 7f142e59c640 1 --2- 172.21.15.86:0/1352489728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b89160c8e0 0x55b891615790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.206+0000 7f142dd9b640 1 --2- 172.21.15.86:0/1352489728 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b89160d290 0x55b891612b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.206+0000 7f142ed9d640 1 -- 172.21.15.86:0/1352489728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b89160c8e0 msgr2=0x55b891615790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.206+0000 7f142ed9d640 1 --2- 172.21.15.86:0/1352489728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b89160c8e0 0x55b891615790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.206+0000 7f142ed9d640 1 -- 172.21.15.86:0/1352489728 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b89160d290 msgr2=0x55b891612b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.206+0000 7f142ed9d640 1 --2- 172.21.15.86:0/1352489728 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b89160d290 0x55b891612b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.207+0000 7f142ed9d640 1 -- 172.21.15.86:0/1352489728 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b89127f3f0 con 0x55b891615e10 2024-04-21T08:49:41.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.207+0000 7f142ed9d640 1 --2- 172.21.15.86:0/1352489728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b891615e10 0x55b891613130 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f1428011420 tx=0x7f1428011930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fce05a05e6eeed42 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:41.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.207+0000 7f1415ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:41.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.207+0000 7f1415ffb640 1 -- 172.21.15.86:0/1352489728 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1428035060 con 0x55b891615e10 2024-04-21T08:49:41.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.207+0000 7f1415ffb640 1 -- 172.21.15.86:0/1352489728 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f1428011d60 con 0x55b891615e10 2024-04-21T08:49:41.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.207+0000 7f14379b4740 1 -- 172.21.15.86:0/1352489728 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1428013520 con 0x55b891615e10 2024-04-21T08:49:41.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.207+0000 7f1415ffb640 1 -- 172.21.15.86:0/1352489728 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f142803c050 con 0x55b891615e10 2024-04-21T08:49:41.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.207+0000 7f14379b4740 10 client.19950 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:41.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.207+0000 7f14379b4740 1 -- 172.21.15.86:0/1352489728 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b8913497d0 con 0x55b891615e10 2024-04-21T08:49:41.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.207+0000 7f1415ffb640 1 -- 172.21.15.86:0/1352489728 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1428013520 con 0x55b891615e10 2024-04-21T08:49:41.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.207+0000 7f1415ffb640 1 -- 172.21.15.86:0/1352489728 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f142800f040 con 0x55b891615e10 2024-04-21T08:49:41.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.207+0000 7f14379b4740 10 client.19950 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:41.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.207+0000 7f14379b4740 10 client.19950 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:49:41.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.207+0000 7f14379b4740 20 client.19950 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:41.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.207+0000 7f14379b4740 1 --2- 172.21.15.86:0/1352489728 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55b891626040 0x55b891628430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.207+0000 7f14379b4740 4 client.19950 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:41.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.207+0000 7f14379b4740 1 -- 172.21.15.86:0/1352489728 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b89150c930 con 0x55b891626040 2024-04-21T08:49:41.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.208+0000 7f142e59c640 1 --2- 172.21.15.86:0/1352489728 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55b891626040 0x55b891628430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.208+0000 7f1415ffb640 10 client.19950 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:49:41.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.208+0000 7f142e59c640 1 --2- 172.21.15.86:0/1352489728 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55b891626040 0x55b891628430 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=b8a7ea93cd965562 server_cookie=aeeca02da6605645 in_seq=0 out_seq=0 2024-04-21T08:49:41.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.209+0000 7f1415ffb640 1 -- 172.21.15.86:0/1352489728 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b89150c930 con 0x55b891626040 2024-04-21T08:49:41.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.209+0000 7f1415ffb640 10 client.19950 handle_command_reply: tid=0 2024-04-21T08:49:41.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.209+0000 7f14379b4740 1 client.19950 shutdown 2024-04-21T08:49:41.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.209+0000 7f14379b4740 1 -- 172.21.15.86:0/1352489728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b891615e10 msgr2=0x55b891613130 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.209+0000 7f14379b4740 1 --2- 172.21.15.86:0/1352489728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b891615e10 0x55b891613130 secure :-1 s=READY pgs=1907 cs=0 l=1 rev1=1 crypto rx=0x7f1428011420 tx=0x7f1428011930 comp rx=0 tx=0).stop 2024-04-21T08:49:41.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.209+0000 7f14379b4740 1 -- 172.21.15.86:0/1352489728 shutdown_connections 2024-04-21T08:49:41.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.209+0000 7f14379b4740 1 --2- 172.21.15.86:0/1352489728 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b891615e10 0x55b891613130 unknown :-1 s=CLOSED pgs=1907 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.209+0000 7f14379b4740 1 --2- 172.21.15.86:0/1352489728 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55b891626040 0x55b891628430 crc :-1 s=READY pgs=72 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.209+0000 7f14379b4740 1 --2- 172.21.15.86:0/1352489728 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b89160d290 0x55b891612b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.209+0000 7f14379b4740 1 --2- 172.21.15.86:0/1352489728 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b89160c8e0 0x55b891615790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.209+0000 7f1415ffb640 0 client.19950 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:49:41.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.209+0000 7f14379b4740 1 -- 172.21.15.86:0/1352489728 >> 172.21.15.86:0/1352489728 conn(0x55b891606fb0 msgr2=0x55b891608d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:41.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.209+0000 7f14379b4740 1 -- 172.21.15.86:0/1352489728 shutdown_connections 2024-04-21T08:49:41.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.209+0000 7f14379b4740 1 -- 172.21.15.86:0/1352489728 wait complete. 2024-04-21T08:49:41.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.209+0000 7f14379b4740 20 client.19950 trim_cache size 0 max 16384 2024-04-21T08:49:41.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.222+0000 7f14379b4740 1 Processor -- start 2024-04-21T08:49:41.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.222+0000 7f14379b4740 1 -- start start 2024-04-21T08:49:41.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.222+0000 7f14379b4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b89160c8e0 0x55b891621c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.222+0000 7f14379b4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89160d290 0x55b891601370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.222+0000 7f14379b4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b891615e10 0x55b891615250 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.222+0000 7f14379b4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14301ad1f0 con 0x55b89160d290 2024-04-21T08:49:41.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.222+0000 7f14379b4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f14301aed30 con 0x55b89160c8e0 2024-04-21T08:49:41.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.222+0000 7f14379b4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f14301ad8c0 con 0x55b891615e10 2024-04-21T08:49:41.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.222+0000 7f142dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89160d290 0x55b891601370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.222+0000 7f142dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89160d290 0x55b891601370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35916/0 (socket says 172.21.15.86:35916) 2024-04-21T08:49:41.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.222+0000 7f142dd9b640 1 -- 172.21.15.86:0/241459514 learned_addr learned my addr 172.21.15.86:0/241459514 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:41.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.223+0000 7f142dd9b640 1 -- 172.21.15.86:0/241459514 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b891615e10 msgr2=0x55b891615250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.223+0000 7f142e59c640 1 --2- 172.21.15.86:0/241459514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b89160c8e0 0x55b891621c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.223+0000 7f142dd9b640 1 --2- 172.21.15.86:0/241459514 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b891615e10 0x55b891615250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.223+0000 7f142dd9b640 1 -- 172.21.15.86:0/241459514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b89160c8e0 msgr2=0x55b891621c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.223+0000 7f142dd9b640 1 --2- 172.21.15.86:0/241459514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b89160c8e0 0x55b891621c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.223+0000 7f142dd9b640 1 -- 172.21.15.86:0/241459514 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b8915fcfe0 con 0x55b89160d290 2024-04-21T08:49:41.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.223+0000 7f142dd9b640 1 --2- 172.21.15.86:0/241459514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89160d290 0x55b891601370 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f14240bb410 tx=0x7f14240036f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=738b2bf89dbc9e18 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:41.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.223+0000 7f1417fff640 1 -- 172.21.15.86:0/241459514 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14240be3a0 con 0x55b89160d290 2024-04-21T08:49:41.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.223+0000 7f1417fff640 1 -- 172.21.15.86:0/241459514 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14240df0b0 con 0x55b89160d290 2024-04-21T08:49:41.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.223+0000 7f1417fff640 1 -- 172.21.15.86:0/241459514 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f14240054d0 con 0x55b89160d290 2024-04-21T08:49:41.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.224+0000 7f14379b4740 1 -- 172.21.15.86:0/241459514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89160d290 msgr2=0x55b891601370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.224+0000 7f14379b4740 1 --2- 172.21.15.86:0/241459514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89160d290 0x55b891601370 secure :-1 s=READY pgs=1908 cs=0 l=1 rev1=1 crypto rx=0x7f14240bb410 tx=0x7f14240036f0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.224+0000 7f14379b4740 1 -- 172.21.15.86:0/241459514 shutdown_connections 2024-04-21T08:49:41.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.224+0000 7f14379b4740 1 --2- 172.21.15.86:0/241459514 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b891615e10 0x55b891615250 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.224+0000 7f14379b4740 1 --2- 172.21.15.86:0/241459514 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b89160d290 0x55b891601370 unknown :-1 s=CLOSED pgs=1908 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.224+0000 7f14379b4740 1 --2- 172.21.15.86:0/241459514 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b89160c8e0 0x55b891621c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.224+0000 7f14379b4740 1 -- 172.21.15.86:0/241459514 >> 172.21.15.86:0/241459514 conn(0x55b891606fb0 msgr2=0x55b891608b20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:41.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.224+0000 7f14379b4740 1 -- 172.21.15.86:0/241459514 shutdown_connections 2024-04-21T08:49:41.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.224+0000 7f14379b4740 1 -- 172.21.15.86:0/241459514 wait complete. 2024-04-21T08:49:41.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.224+0000 7f14379b4740 1 -- start start 2024-04-21T08:49:41.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.224+0000 7f14379b4740 1 Processor -- start 2024-04-21T08:49:41.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.224+0000 7f14379b4740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b89160c8e0 0x55b89160f8d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.224+0000 7f14379b4740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b89160d290 0x55b89160fe10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.224+0000 7f14379b4740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b891615e10 0x55b891610350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.224+0000 7f14379b4740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f14301ae660 con 0x55b891615e10 2024-04-21T08:49:41.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.224+0000 7f14379b4740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f14301adf90 con 0x55b89160c8e0 2024-04-21T08:49:41.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.224+0000 7f14379b4740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f14301ab1f0 con 0x55b89160d290 2024-04-21T08:49:41.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.224+0000 7f14379b4740 10 client.0 _ll_register_callbacks cb 0x7f142000b3a0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:41.229 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.224+0000 7f142ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b891615e10 0x55b891610350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.224+0000 7f142ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b891615e10 0x55b891610350 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35932/0 (socket says 172.21.15.86:35932) 2024-04-21T08:49:41.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.224+0000 7f142ed9d640 1 -- 172.21.15.86:0/632544814 learned_addr learned my addr 172.21.15.86:0/632544814 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:41.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.224+0000 7f142e59c640 1 --2- 172.21.15.86:0/632544814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b89160c8e0 0x55b89160f8d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.224+0000 7f142ed9d640 1 -- 172.21.15.86:0/632544814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b89160d290 msgr2=0x55b89160fe10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.224+0000 7f142ed9d640 1 --2- 172.21.15.86:0/632544814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b89160d290 0x55b89160fe10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.224+0000 7f142ed9d640 1 -- 172.21.15.86:0/632544814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b89160c8e0 msgr2=0x55b89160f8d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.224+0000 7f142ed9d640 1 --2- 172.21.15.86:0/632544814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b89160c8e0 0x55b89160f8d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.224+0000 7f142ed9d640 1 -- 172.21.15.86:0/632544814 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b89168b4c0 con 0x55b891615e10 2024-04-21T08:49:41.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.225+0000 7f142ed9d640 1 --2- 172.21.15.86:0/632544814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b891615e10 0x55b891610350 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f14280138d0 tx=0x7f1428010e00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b197c266fe5baa21 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:41.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.225+0000 7f1415ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:41.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.225+0000 7f1415ffb640 1 -- 172.21.15.86:0/632544814 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1428035070 con 0x55b891615e10 2024-04-21T08:49:41.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.225+0000 7f1415ffb640 1 -- 172.21.15.86:0/632544814 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f1428013520 con 0x55b891615e10 2024-04-21T08:49:41.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.225+0000 7f14379b4740 1 -- 172.21.15.86:0/632544814 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f14240df0b0 con 0x55b891615e10 2024-04-21T08:49:41.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.225+0000 7f1415ffb640 1 -- 172.21.15.86:0/632544814 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1428049050 con 0x55b891615e10 2024-04-21T08:49:41.230 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.225+0000 7f14379b4740 10 client.19956 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:41.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.225+0000 7f14379b4740 1 -- 172.21.15.86:0/632544814 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b89168c7a0 con 0x55b891615e10 2024-04-21T08:49:41.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.225+0000 7f1415ffb640 1 -- 172.21.15.86:0/632544814 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f14240df0b0 con 0x55b891615e10 2024-04-21T08:49:41.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.225+0000 7f1415ffb640 1 -- 172.21.15.86:0/632544814 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f142800f040 con 0x55b891615e10 2024-04-21T08:49:41.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.225+0000 7f14379b4740 10 client.19956 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:41.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.225+0000 7f14379b4740 10 client.19956 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:49:41.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.225+0000 7f14379b4740 20 client.19956 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:41.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.225+0000 7f14379b4740 1 --2- 172.21.15.86:0/632544814 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55b89168cce0 0x55b89168f100 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.225+0000 7f14379b4740 4 client.19956 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:49:41.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.225+0000 7f14379b4740 1 -- 172.21.15.86:0/632544814 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b891602220 con 0x55b89168cce0 2024-04-21T08:49:41.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.226+0000 7f142e59c640 1 --2- 172.21.15.86:0/632544814 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55b89168cce0 0x55b89168f100 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.226+0000 7f142e59c640 1 --2- 172.21.15.86:0/632544814 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55b89168cce0 0x55b89168f100 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=5622e43440de33a1 server_cookie=99c4069a420d616e in_seq=0 out_seq=0 2024-04-21T08:49:41.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.226+0000 7f1415ffb640 10 client.19956 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:49:41.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.226+0000 7f1415ffb640 1 -- 172.21.15.86:0/632544814 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b891602220 con 0x55b89168cce0 2024-04-21T08:49:41.231 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.226+0000 7f1415ffb640 10 client.19956 handle_command_reply: tid=0 2024-04-21T08:49:41.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.226+0000 7f14379b4740 1 client.19956 shutdown 2024-04-21T08:49:41.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.227+0000 7f14379b4740 1 -- 172.21.15.86:0/632544814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b891615e10 msgr2=0x55b891610350 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.227+0000 7f14379b4740 1 --2- 172.21.15.86:0/632544814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b891615e10 0x55b891610350 secure :-1 s=READY pgs=1909 cs=0 l=1 rev1=1 crypto rx=0x7f14280138d0 tx=0x7f1428010e00 comp rx=0 tx=0).stop 2024-04-21T08:49:41.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.227+0000 7f14379b4740 1 -- 172.21.15.86:0/632544814 shutdown_connections 2024-04-21T08:49:41.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.227+0000 7f14379b4740 1 --2- 172.21.15.86:0/632544814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b891615e10 0x55b891610350 unknown :-1 s=CLOSED pgs=1909 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.227+0000 7f14379b4740 1 --2- 172.21.15.86:0/632544814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b89160d290 0x55b89160fe10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.227+0000 7f14379b4740 1 --2- 172.21.15.86:0/632544814 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55b89168cce0 0x55b89168f100 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.227+0000 7f14379b4740 1 --2- 172.21.15.86:0/632544814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b89160c8e0 0x55b89160f8d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.227+0000 7f1415ffb640 0 client.19956 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:49:41.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.227+0000 7f14379b4740 1 -- 172.21.15.86:0/632544814 >> 172.21.15.86:0/632544814 conn(0x55b891606fb0 msgr2=0x55b89160d6c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:41.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.227+0000 7f14379b4740 1 -- 172.21.15.86:0/632544814 shutdown_connections 2024-04-21T08:49:41.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.227+0000 7f14379b4740 1 -- 172.21.15.86:0/632544814 wait complete. 2024-04-21T08:49:41.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.227+0000 7f14379b4740 20 client.19956 trim_cache size 0 max 16384 2024-04-21T08:49:41.233 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:49:41.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.227+0000 7f1434dbd640 1 -- 172.21.15.86:0/4222354932 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f14180a3d20 msgr2=0x7f14180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.227+0000 7f1434dbd640 1 --2- 172.21.15.86:0/4222354932 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f14180a3d20 0x7f14180a61e0 secure :-1 s=READY pgs=317 cs=0 l=1 rev1=1 crypto rx=0x7f1428021000 tx=0x7f142801e000 comp rx=0 tx=0).stop 2024-04-21T08:49:41.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.227+0000 7f1434dbd640 1 -- 172.21.15.86:0/4222354932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1430107260 msgr2=0x7f143019e930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.227+0000 7f1434dbd640 1 --2- 172.21.15.86:0/4222354932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1430107260 0x7f143019e930 secure :-1 s=READY pgs=1905 cs=0 l=1 rev1=1 crypto rx=0x7f1424036040 tx=0x7f142402c7e0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.227+0000 7f1434dbd640 1 -- 172.21.15.86:0/4222354932 shutdown_connections 2024-04-21T08:49:41.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.227+0000 7f1434dbd640 1 --2- 172.21.15.86:0/4222354932 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f14180a3d20 0x7f14180a61e0 unknown :-1 s=CLOSED pgs=317 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.227+0000 7f1434dbd640 1 --2- 172.21.15.86:0/4222354932 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f14301985a0 0x7f1430198a50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.227+0000 7f1434dbd640 1 --2- 172.21.15.86:0/4222354932 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1430107260 0x7f143019e930 unknown :-1 s=CLOSED pgs=1905 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.227+0000 7f1434dbd640 1 --2- 172.21.15.86:0/4222354932 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1430102190 0x7f143019e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.227+0000 7f1434dbd640 1 -- 172.21.15.86:0/4222354932 >> 172.21.15.86:0/4222354932 conn(0x7f14300793a0 msgr2=0x7f1430100000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:41.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.228+0000 7f1434dbd640 1 -- 172.21.15.86:0/4222354932 shutdown_connections 2024-04-21T08:49:41.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.228+0000 7f1434dbd640 1 -- 172.21.15.86:0/4222354932 wait complete. 2024-04-21T08:49:41.237 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:49:41.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.337+0000 7f720e893640 1 -- 172.21.15.86:0/899190103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7208107260 msgr2=0x7f720810ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.337+0000 7f720e893640 1 --2- 172.21.15.86:0/899190103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7208107260 0x7f720810ee30 secure :-1 s=READY pgs=1910 cs=0 l=1 rev1=1 crypto rx=0x7f7200009ca0 tx=0x7f720002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.338+0000 7f720e893640 1 -- 172.21.15.86:0/899190103 shutdown_connections 2024-04-21T08:49:41.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.338+0000 7f720e893640 1 --2- 172.21.15.86:0/899190103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7208107260 0x7f720810ee30 unknown :-1 s=CLOSED pgs=1910 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.338+0000 7f720e893640 1 --2- 172.21.15.86:0/899190103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7208102b40 0x7f7208106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.338+0000 7f720e893640 1 --2- 172.21.15.86:0/899190103 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7208102190 0x7f7208102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.338+0000 7f720e893640 1 -- 172.21.15.86:0/899190103 >> 172.21.15.86:0/899190103 conn(0x7f72080793a0 msgr2=0x7f72080797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:41.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.338+0000 7f720e893640 1 -- 172.21.15.86:0/899190103 shutdown_connections 2024-04-21T08:49:41.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.338+0000 7f720e893640 1 -- 172.21.15.86:0/899190103 wait complete. 2024-04-21T08:49:41.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.338+0000 7f720e893640 1 Processor -- start 2024-04-21T08:49:41.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.338+0000 7f720e893640 1 -- start start 2024-04-21T08:49:41.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.339+0000 7f720e893640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7208102190 0x7f7208074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.339+0000 7f720e893640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7208107260 0x7f7208074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.339+0000 7f720e893640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7208074cd0 0x7f72081a68e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.339+0000 7f720e893640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72081121b0 con 0x7f7208107260 2024-04-21T08:49:41.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.339+0000 7f720e893640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f7208112030 con 0x7f7208102190 2024-04-21T08:49:41.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.339+0000 7f720e893640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f7208112330 con 0x7f7208074cd0 2024-04-21T08:49:41.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.339+0000 7f72077fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7208107260 0x7f7208074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.339+0000 7f72077fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7208107260 0x7f7208074790 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35956/0 (socket says 172.21.15.86:35956) 2024-04-21T08:49:41.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.339+0000 7f72077fe640 1 -- 172.21.15.86:0/2057865254 learned_addr learned my addr 172.21.15.86:0/2057865254 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:41.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.339+0000 7f7207fff640 1 --2- 172.21.15.86:0/2057865254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7208102190 0x7f7208074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.339+0000 7f720ce30640 1 --2- 172.21.15.86:0/2057865254 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7208074cd0 0x7f72081a68e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.339+0000 7f72077fe640 1 -- 172.21.15.86:0/2057865254 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7208074cd0 msgr2=0x7f72081a68e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.339+0000 7f72077fe640 1 --2- 172.21.15.86:0/2057865254 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7208074cd0 0x7f72081a68e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.339+0000 7f72077fe640 1 -- 172.21.15.86:0/2057865254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7208102190 msgr2=0x7f7208074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.339+0000 7f72077fe640 1 --2- 172.21.15.86:0/2057865254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7208102190 0x7f7208074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.339+0000 7f72077fe640 1 -- 172.21.15.86:0/2057865254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f72081a6fe0 con 0x7f7208107260 2024-04-21T08:49:41.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.339+0000 7f7207fff640 1 --2- 172.21.15.86:0/2057865254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7208102190 0x7f7208074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:49:41.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.339+0000 7f72077fe640 1 --2- 172.21.15.86:0/2057865254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7208107260 0x7f7208074790 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f71f800c970 tx=0x7f71f800ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:41.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.339+0000 7f72057fa640 1 -- 172.21.15.86:0/2057865254 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f8007bb0 con 0x7f7208107260 2024-04-21T08:49:41.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.339+0000 7f720e893640 1 -- 172.21.15.86:0/2057865254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f72081a72d0 con 0x7f7208107260 2024-04-21T08:49:41.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.339+0000 7f72057fa640 1 -- 172.21.15.86:0/2057865254 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71f8007d50 con 0x7f7208107260 2024-04-21T08:49:41.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.339+0000 7f72057fa640 1 -- 172.21.15.86:0/2057865254 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f800f450 con 0x7f7208107260 2024-04-21T08:49:41.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.339+0000 7f720e893640 1 -- 172.21.15.86:0/2057865254 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f72081a7820 con 0x7f7208107260 2024-04-21T08:49:41.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.342+0000 7f72057fa640 1 -- 172.21.15.86:0/2057865254 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f71f8004e80 con 0x7f7208107260 2024-04-21T08:49:41.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.342+0000 7f7211564740 1 Processor -- start 2024-04-21T08:49:41.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.342+0000 7f7211564740 1 -- start start 2024-04-21T08:49:41.348 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.342+0000 7f7211564740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8b77c14d0 0x55c8b77c18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.342+0000 7f7211564740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c8b77c1e80 0x55c8b77ca430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.342+0000 7f7211564740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c8b77caa00 0x55c8b77cce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.342+0000 7f7211564740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c8b77261f0 con 0x55c8b77c14d0 2024-04-21T08:49:41.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.342+0000 7f7211564740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c8b76c34b0 con 0x55c8b77caa00 2024-04-21T08:49:41.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.342+0000 7f7211564740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c8b77221b0 con 0x55c8b77c1e80 2024-04-21T08:49:41.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.342+0000 7f720ce30640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8b77c14d0 0x55c8b77c18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.342+0000 7f720ce30640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8b77c14d0 0x55c8b77c18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35964/0 (socket says 172.21.15.86:35964) 2024-04-21T08:49:41.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.342+0000 7f720ce30640 1 -- 172.21.15.86:0/3415652629 learned_addr learned my addr 172.21.15.86:0/3415652629 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:41.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.342+0000 7f72077fe640 1 --2- 172.21.15.86:0/3415652629 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c8b77caa00 0x55c8b77cce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.342+0000 7f7207fff640 1 --2- 172.21.15.86:0/3415652629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c8b77c1e80 0x55c8b77ca430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.342+0000 7f72057fa640 1 --2- 172.21.15.86:0/2057865254 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f71f00a3d50 0x7f71f00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.342+0000 7f720ce30640 1 -- 172.21.15.86:0/3415652629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c8b77c1e80 msgr2=0x55c8b77ca430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.349 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.342+0000 7f720ce30640 1 --2- 172.21.15.86:0/3415652629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c8b77c1e80 0x55c8b77ca430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.342+0000 7f720ce30640 1 -- 172.21.15.86:0/3415652629 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c8b77caa00 msgr2=0x55c8b77cce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.342+0000 7f720ce30640 1 --2- 172.21.15.86:0/3415652629 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c8b77caa00 0x55c8b77cce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.342+0000 7f720ce30640 1 -- 172.21.15.86:0/3415652629 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8b76b51b0 con 0x55c8b77c14d0 2024-04-21T08:49:41.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.343+0000 7f720ce30640 1 --2- 172.21.15.86:0/2057865254 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f71f00a3d50 0x7f71f00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.343+0000 7f72057fa640 1 -- 172.21.15.86:0/2057865254 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f71f80d0840 con 0x7f7208107260 2024-04-21T08:49:41.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.343+0000 7f720ce30640 1 --2- 172.21.15.86:0/3415652629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8b77c14d0 0x55c8b77c18b0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f7200009ca0 tx=0x7f7200009a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9020bd8e6a791613 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:41.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.343+0000 7f71ee7fc640 1 -- 172.21.15.86:0/3415652629 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7200039040 con 0x55c8b77c14d0 2024-04-21T08:49:41.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.343+0000 7f71ee7fc640 1 -- 172.21.15.86:0/3415652629 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7200003b70 con 0x55c8b77c14d0 2024-04-21T08:49:41.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.343+0000 7f71ee7fc640 1 -- 172.21.15.86:0/3415652629 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f7200034440 con 0x55c8b77c14d0 2024-04-21T08:49:41.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.343+0000 7f7211564740 1 -- 172.21.15.86:0/3415652629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8b77c14d0 msgr2=0x55c8b77c18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.343+0000 7f720ce30640 1 --2- 172.21.15.86:0/2057865254 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f71f00a3d50 0x7f71f00a6210 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f720004b000 tx=0x7f7200044000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:41.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.343+0000 7f7211564740 1 --2- 172.21.15.86:0/3415652629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8b77c14d0 0x55c8b77c18b0 secure :-1 s=READY pgs=1912 cs=0 l=1 rev1=1 crypto rx=0x7f7200009ca0 tx=0x7f7200009a00 comp rx=0 tx=0).stop 2024-04-21T08:49:41.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.344+0000 7f7211564740 1 -- 172.21.15.86:0/3415652629 shutdown_connections 2024-04-21T08:49:41.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.344+0000 7f7211564740 1 --2- 172.21.15.86:0/3415652629 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c8b77caa00 0x55c8b77cce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.344+0000 7f7211564740 1 --2- 172.21.15.86:0/3415652629 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c8b77c1e80 0x55c8b77ca430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.344+0000 7f7211564740 1 --2- 172.21.15.86:0/3415652629 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8b77c14d0 0x55c8b77c18b0 unknown :-1 s=CLOSED pgs=1912 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.344+0000 7f7211564740 1 -- 172.21.15.86:0/3415652629 >> 172.21.15.86:0/3415652629 conn(0x55c8b77bbba0 msgr2=0x55c8b77bbf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:41.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.344+0000 7f7211564740 1 -- 172.21.15.86:0/3415652629 shutdown_connections 2024-04-21T08:49:41.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.344+0000 7f7211564740 1 -- 172.21.15.86:0/3415652629 wait complete. 2024-04-21T08:49:41.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.344+0000 7f7211564740 1 -- start start 2024-04-21T08:49:41.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.344+0000 7f7211564740 1 Processor -- start 2024-04-21T08:49:41.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.344+0000 7f7211564740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8b77c14d0 0x55c8b77ca310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.344+0000 7f7211564740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c8b77c1e80 0x55c8b77c76a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.344+0000 7f7211564740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c8b77caa00 0x55c8b77c7c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.344+0000 7f7211564740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55c8b77cd970 con 0x55c8b77c14d0 2024-04-21T08:49:41.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.344+0000 7f7211564740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55c8b76df500 con 0x55c8b77caa00 2024-04-21T08:49:41.351 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.344+0000 7f7211564740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55c8b77cdaf0 con 0x55c8b77c1e80 2024-04-21T08:49:41.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.344+0000 7f7211564740 10 client.0 _ll_register_callbacks cb 0x55c8b76adb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:41.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.344+0000 7f7207fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8b77c14d0 0x55c8b77ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.344+0000 7f7207fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8b77c14d0 0x55c8b77ca310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35976/0 (socket says 172.21.15.86:35976) 2024-04-21T08:49:41.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.344+0000 7f7207fff640 1 -- 172.21.15.86:0/821567749 learned_addr learned my addr 172.21.15.86:0/821567749 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:41.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.345+0000 7f72077fe640 1 --2- 172.21.15.86:0/821567749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c8b77c1e80 0x55c8b77c76a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.345+0000 7f720ce30640 1 --2- 172.21.15.86:0/821567749 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c8b77caa00 0x55c8b77c7c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.345+0000 7f7207fff640 1 -- 172.21.15.86:0/821567749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c8b77c1e80 msgr2=0x55c8b77c76a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.345+0000 7f7207fff640 1 --2- 172.21.15.86:0/821567749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c8b77c1e80 0x55c8b77c76a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.345+0000 7f7207fff640 1 -- 172.21.15.86:0/821567749 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c8b77caa00 msgr2=0x55c8b77c7c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.345+0000 7f7207fff640 1 --2- 172.21.15.86:0/821567749 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c8b77caa00 0x55c8b77c7c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.345+0000 7f7207fff640 1 -- 172.21.15.86:0/821567749 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8b74383f0 con 0x55c8b77c14d0 2024-04-21T08:49:41.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.345+0000 7f7207fff640 1 --2- 172.21.15.86:0/821567749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8b77c14d0 0x55c8b77ca310 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f71fc012d20 tx=0x7f71fc00f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16d2f2210ae1399a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:41.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.345+0000 7f71e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:41.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.345+0000 7f71e77fe640 1 -- 172.21.15.86:0/821567749 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc012110 con 0x55c8b77c14d0 2024-04-21T08:49:41.352 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.345+0000 7f71e77fe640 1 -- 172.21.15.86:0/821567749 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f71fc004050 con 0x55c8b77c14d0 2024-04-21T08:49:41.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.345+0000 7f7211564740 1 -- 172.21.15.86:0/821567749 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f7200003b70 con 0x55c8b77c14d0 2024-04-21T08:49:41.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.345+0000 7f71e77fe640 1 -- 172.21.15.86:0/821567749 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f71fc025050 con 0x55c8b77c14d0 2024-04-21T08:49:41.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.345+0000 7f7211564740 10 client.19968 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:41.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.345+0000 7f7211564740 1 -- 172.21.15.86:0/821567749 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c8b75027d0 con 0x55c8b77c14d0 2024-04-21T08:49:41.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.345+0000 7f71e77fe640 1 -- 172.21.15.86:0/821567749 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f7200003b70 con 0x55c8b77c14d0 2024-04-21T08:49:41.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.346+0000 7f71e77fe640 1 -- 172.21.15.86:0/821567749 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f71fc010430 con 0x55c8b77c14d0 2024-04-21T08:49:41.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.346+0000 7f7211564740 10 client.19968 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:41.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.346+0000 7f7211564740 10 client.19968 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:49:41.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.346+0000 7f7211564740 20 client.19968 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:41.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.346+0000 7f7211564740 1 --2- 172.21.15.86:0/821567749 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55c8b77dac70 0x55c8b77dd0a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.346+0000 7f7211564740 4 client.19968 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:41.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.346+0000 7f7211564740 1 -- 172.21.15.86:0/821567749 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55c8b76c5930 con 0x55c8b77dac70 2024-04-21T08:49:41.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.346+0000 7f72077fe640 1 --2- 172.21.15.86:0/821567749 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55c8b77dac70 0x55c8b77dd0a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.353 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.346+0000 7f71e77fe640 10 client.19968 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:49:41.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.346+0000 7f72077fe640 1 --2- 172.21.15.86:0/821567749 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55c8b77dac70 0x55c8b77dd0a0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=9ec1486ecbf22b79 server_cookie=507e58c2ebf2ca88 in_seq=0 out_seq=0 2024-04-21T08:49:41.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.347+0000 7f71e77fe640 1 -- 172.21.15.86:0/821567749 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55c8b76c5930 con 0x55c8b77dac70 2024-04-21T08:49:41.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.347+0000 7f71e77fe640 10 client.19968 handle_command_reply: tid=0 2024-04-21T08:49:41.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.347+0000 7f7211564740 1 client.19968 shutdown 2024-04-21T08:49:41.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.347+0000 7f7211564740 1 -- 172.21.15.86:0/821567749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8b77c14d0 msgr2=0x55c8b77ca310 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.347+0000 7f7211564740 1 --2- 172.21.15.86:0/821567749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8b77c14d0 0x55c8b77ca310 secure :-1 s=READY pgs=1913 cs=0 l=1 rev1=1 crypto rx=0x7f71fc012d20 tx=0x7f71fc00f790 comp rx=0 tx=0).stop 2024-04-21T08:49:41.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.347+0000 7f7211564740 1 -- 172.21.15.86:0/821567749 shutdown_connections 2024-04-21T08:49:41.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.347+0000 7f7211564740 1 --2- 172.21.15.86:0/821567749 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55c8b77dac70 0x55c8b77dd0a0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.347+0000 7f7211564740 1 --2- 172.21.15.86:0/821567749 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c8b77caa00 0x55c8b77c7c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.347+0000 7f7211564740 1 --2- 172.21.15.86:0/821567749 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c8b77c1e80 0x55c8b77c76a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.347+0000 7f7211564740 1 --2- 172.21.15.86:0/821567749 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8b77c14d0 0x55c8b77ca310 unknown :-1 s=CLOSED pgs=1913 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.347+0000 7f71e77fe640 0 client.19968 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:49:41.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.347+0000 7f7211564740 1 -- 172.21.15.86:0/821567749 >> 172.21.15.86:0/821567749 conn(0x55c8b77bbba0 msgr2=0x55c8b77bd8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:41.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.347+0000 7f7211564740 1 -- 172.21.15.86:0/821567749 shutdown_connections 2024-04-21T08:49:41.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.347+0000 7f7211564740 1 -- 172.21.15.86:0/821567749 wait complete. 2024-04-21T08:49:41.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.347+0000 7f7211564740 20 client.19968 trim_cache size 0 max 16384 2024-04-21T08:49:41.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.360+0000 7f7211564740 1 Processor -- start 2024-04-21T08:49:41.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.360+0000 7f7211564740 1 -- start start 2024-04-21T08:49:41.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.360+0000 7f7211564740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8b77c14d0 0x55c8b77d6830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.360+0000 7f7211564740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c8b77c1e80 0x55c8b77c9dd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.360+0000 7f7211564740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c8b77caa00 0x55c8b77ca310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.360+0000 7f7211564740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72081b1600 con 0x55c8b77c14d0 2024-04-21T08:49:41.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.360+0000 7f7211564740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f72081b3140 con 0x55c8b77c1e80 2024-04-21T08:49:41.365 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.361+0000 7f7211564740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f72081b1cd0 con 0x55c8b77caa00 2024-04-21T08:49:41.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.361+0000 7f7207fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8b77c14d0 0x55c8b77d6830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.361+0000 7f7207fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8b77c14d0 0x55c8b77d6830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35988/0 (socket says 172.21.15.86:35988) 2024-04-21T08:49:41.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.361+0000 7f7207fff640 1 -- 172.21.15.86:0/331917771 learned_addr learned my addr 172.21.15.86:0/331917771 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:41.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.361+0000 7f7207fff640 1 -- 172.21.15.86:0/331917771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c8b77caa00 msgr2=0x55c8b77ca310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.361+0000 7f7207fff640 1 --2- 172.21.15.86:0/331917771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c8b77caa00 0x55c8b77ca310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.361+0000 7f7207fff640 1 -- 172.21.15.86:0/331917771 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c8b77c1e80 msgr2=0x55c8b77c9dd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.361+0000 7f7207fff640 1 --2- 172.21.15.86:0/331917771 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c8b77c1e80 0x55c8b77c9dd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.361+0000 7f7207fff640 1 -- 172.21.15.86:0/331917771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55c8b77c7fb0 con 0x55c8b77c14d0 2024-04-21T08:49:41.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.361+0000 7f7207fff640 1 --2- 172.21.15.86:0/331917771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8b77c14d0 0x55c8b77d6830 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f71fc000c00 tx=0x7f71fc0123a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2c60f02c6f2488f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:41.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.361+0000 7f71edffb640 1 -- 172.21.15.86:0/331917771 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc02e070 con 0x55c8b77c14d0 2024-04-21T08:49:41.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.361+0000 7f71edffb640 1 -- 172.21.15.86:0/331917771 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71fc005560 con 0x55c8b77c14d0 2024-04-21T08:49:41.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.361+0000 7f71edffb640 1 -- 172.21.15.86:0/331917771 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71fc003850 con 0x55c8b77c14d0 2024-04-21T08:49:41.366 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.362+0000 7f7211564740 1 -- 172.21.15.86:0/331917771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8b77c14d0 msgr2=0x55c8b77d6830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.362+0000 7f7211564740 1 --2- 172.21.15.86:0/331917771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8b77c14d0 0x55c8b77d6830 secure :-1 s=READY pgs=1914 cs=0 l=1 rev1=1 crypto rx=0x7f71fc000c00 tx=0x7f71fc0123a0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.362+0000 7f7211564740 1 -- 172.21.15.86:0/331917771 shutdown_connections 2024-04-21T08:49:41.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.362+0000 7f7211564740 1 --2- 172.21.15.86:0/331917771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c8b77caa00 0x55c8b77ca310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.362+0000 7f7211564740 1 --2- 172.21.15.86:0/331917771 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c8b77c1e80 0x55c8b77c9dd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.362+0000 7f7211564740 1 --2- 172.21.15.86:0/331917771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8b77c14d0 0x55c8b77d6830 unknown :-1 s=CLOSED pgs=1914 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.362+0000 7f7211564740 1 -- 172.21.15.86:0/331917771 >> 172.21.15.86:0/331917771 conn(0x55c8b77bbba0 msgr2=0x55c8b77bac00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:41.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.362+0000 7f7211564740 1 -- 172.21.15.86:0/331917771 shutdown_connections 2024-04-21T08:49:41.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.362+0000 7f7211564740 1 -- 172.21.15.86:0/331917771 wait complete. 2024-04-21T08:49:41.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.362+0000 7f7211564740 1 -- start start 2024-04-21T08:49:41.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.362+0000 7f7211564740 1 Processor -- start 2024-04-21T08:49:41.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.362+0000 7f7211564740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c8b77c14d0 0x55c8b77ce900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.362+0000 7f7211564740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8b77c1e80 0x55c8b77cee40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.362+0000 7f7211564740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c8b77caa00 0x55c8b77cf380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.362+0000 7f7211564740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f72081b2a70 con 0x55c8b77c1e80 2024-04-21T08:49:41.367 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.362+0000 7f7211564740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f72081b23a0 con 0x55c8b77caa00 2024-04-21T08:49:41.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.362+0000 7f7211564740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f72081af600 con 0x55c8b77c14d0 2024-04-21T08:49:41.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.362+0000 7f7211564740 10 client.0 _ll_register_callbacks cb 0x7f71fc012930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:41.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.363+0000 7f72077fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8b77c1e80 0x55c8b77cee40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.363+0000 7f72077fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8b77c1e80 0x55c8b77cee40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35990/0 (socket says 172.21.15.86:35990) 2024-04-21T08:49:41.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.363+0000 7f72077fe640 1 -- 172.21.15.86:0/1118163909 learned_addr learned my addr 172.21.15.86:0/1118163909 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:41.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.363+0000 7f720ce30640 1 --2- 172.21.15.86:0/1118163909 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c8b77caa00 0x55c8b77cf380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.363+0000 7f7207fff640 1 --2- 172.21.15.86:0/1118163909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c8b77c14d0 0x55c8b77ce900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.363+0000 7f72077fe640 1 -- 172.21.15.86:0/1118163909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c8b77c14d0 msgr2=0x55c8b77ce900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.363+0000 7f72077fe640 1 --2- 172.21.15.86:0/1118163909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c8b77c14d0 0x55c8b77ce900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.363+0000 7f72077fe640 1 -- 172.21.15.86:0/1118163909 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c8b77caa00 msgr2=0x55c8b77cf380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.363+0000 7f72077fe640 1 --2- 172.21.15.86:0/1118163909 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c8b77caa00 0x55c8b77cf380 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.363+0000 7f72077fe640 1 -- 172.21.15.86:0/1118163909 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55c8b7840360 con 0x55c8b77c1e80 2024-04-21T08:49:41.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.363+0000 7f720ce30640 1 --2- 172.21.15.86:0/1118163909 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c8b77caa00 0x55c8b77cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:41.368 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.363+0000 7f72077fe640 1 --2- 172.21.15.86:0/1118163909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8b77c1e80 0x55c8b77cee40 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f71f809c3a0 tx=0x7f71f8098760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ba23e2b02b62a4a5 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:41.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.363+0000 7f71e77fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:41.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.363+0000 7f71e77fe640 1 -- 172.21.15.86:0/1118163909 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f71f8085020 con 0x55c8b77c1e80 2024-04-21T08:49:41.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.363+0000 7f71e77fe640 1 -- 172.21.15.86:0/1118163909 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f71f80cde40 con 0x55c8b77c1e80 2024-04-21T08:49:41.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.363+0000 7f7211564740 1 -- 172.21.15.86:0/1118163909 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f71fc005560 con 0x55c8b77c1e80 2024-04-21T08:49:41.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.363+0000 7f71e77fe640 1 -- 172.21.15.86:0/1118163909 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f71f80d6040 con 0x55c8b77c1e80 2024-04-21T08:49:41.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.363+0000 7f7211564740 10 client.19974 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:41.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.363+0000 7f7211564740 1 -- 172.21.15.86:0/1118163909 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55c8b7841620 con 0x55c8b77c1e80 2024-04-21T08:49:41.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.363+0000 7f71e77fe640 1 -- 172.21.15.86:0/1118163909 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f71fc005560 con 0x55c8b77c1e80 2024-04-21T08:49:41.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.364+0000 7f71e77fe640 1 -- 172.21.15.86:0/1118163909 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f71f80bb230 con 0x55c8b77c1e80 2024-04-21T08:49:41.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.364+0000 7f7211564740 10 client.19974 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:41.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.364+0000 7f7211564740 10 client.19974 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:49:41.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.364+0000 7f7211564740 20 client.19974 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:41.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.364+0000 7f7211564740 1 --2- 172.21.15.86:0/1118163909 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55c8b7841df0 0x55c8b78441e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.369 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.364+0000 7f7211564740 4 client.19974 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:49:41.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.364+0000 7f7211564740 1 -- 172.21.15.86:0/1118163909 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55c8b7733c10 con 0x55c8b7841df0 2024-04-21T08:49:41.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.364+0000 7f7207fff640 1 --2- 172.21.15.86:0/1118163909 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55c8b7841df0 0x55c8b78441e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.364+0000 7f7207fff640 1 --2- 172.21.15.86:0/1118163909 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55c8b7841df0 0x55c8b78441e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ce257005bf289608 server_cookie=ee3549f02040f5da in_seq=0 out_seq=0 2024-04-21T08:49:41.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.364+0000 7f71e77fe640 10 client.19974 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:49:41.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.365+0000 7f71e77fe640 1 -- 172.21.15.86:0/1118163909 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55c8b7733c10 con 0x55c8b7841df0 2024-04-21T08:49:41.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.365+0000 7f71e77fe640 10 client.19974 handle_command_reply: tid=0 2024-04-21T08:49:41.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.365+0000 7f7211564740 1 client.19974 shutdown 2024-04-21T08:49:41.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.365+0000 7f7211564740 1 -- 172.21.15.86:0/1118163909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8b77c1e80 msgr2=0x55c8b77cee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.365+0000 7f7211564740 1 --2- 172.21.15.86:0/1118163909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8b77c1e80 0x55c8b77cee40 secure :-1 s=READY pgs=1915 cs=0 l=1 rev1=1 crypto rx=0x7f71f809c3a0 tx=0x7f71f8098760 comp rx=0 tx=0).stop 2024-04-21T08:49:41.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.365+0000 7f7211564740 1 -- 172.21.15.86:0/1118163909 shutdown_connections 2024-04-21T08:49:41.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.365+0000 7f7211564740 1 --2- 172.21.15.86:0/1118163909 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x55c8b7841df0 0x55c8b78441e0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.365+0000 7f7211564740 1 --2- 172.21.15.86:0/1118163909 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55c8b77caa00 0x55c8b77cf380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.365+0000 7f71e77fe640 0 client.19974 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:49:41.370 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.365+0000 7f7211564740 1 --2- 172.21.15.86:0/1118163909 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55c8b77c1e80 0x55c8b77cee40 unknown :-1 s=CLOSED pgs=1915 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.365+0000 7f7211564740 1 --2- 172.21.15.86:0/1118163909 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55c8b77c14d0 0x55c8b77ce900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.365+0000 7f7211564740 1 -- 172.21.15.86:0/1118163909 >> 172.21.15.86:0/1118163909 conn(0x55c8b77bbba0 msgr2=0x55c8b77bce80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:41.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.365+0000 7f7211564740 1 -- 172.21.15.86:0/1118163909 shutdown_connections 2024-04-21T08:49:41.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.365+0000 7f7211564740 1 -- 172.21.15.86:0/1118163909 wait complete. 2024-04-21T08:49:41.371 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.365+0000 7f7211564740 20 client.19974 trim_cache size 0 max 16384 2024-04-21T08:49:41.371 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:49:41.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.366+0000 7f720e893640 1 -- 172.21.15.86:0/2057865254 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f71f00a3d50 msgr2=0x7f71f00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.366+0000 7f720e893640 1 --2- 172.21.15.86:0/2057865254 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f71f00a3d50 0x7f71f00a6210 secure :-1 s=READY pgs=318 cs=0 l=1 rev1=1 crypto rx=0x7f720004b000 tx=0x7f7200044000 comp rx=0 tx=0).stop 2024-04-21T08:49:41.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.366+0000 7f720e893640 1 -- 172.21.15.86:0/2057865254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7208107260 msgr2=0x7f7208074790 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.366+0000 7f720e893640 1 --2- 172.21.15.86:0/2057865254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7208107260 0x7f7208074790 secure :-1 s=READY pgs=1911 cs=0 l=1 rev1=1 crypto rx=0x7f71f800c970 tx=0x7f71f800ce30 comp rx=0 tx=0).stop 2024-04-21T08:49:41.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.366+0000 7f720e893640 1 -- 172.21.15.86:0/2057865254 shutdown_connections 2024-04-21T08:49:41.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.366+0000 7f720e893640 1 --2- 172.21.15.86:0/2057865254 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f71f00a3d50 0x7f71f00a6210 unknown :-1 s=CLOSED pgs=318 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.366+0000 7f720e893640 1 --2- 172.21.15.86:0/2057865254 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f7208074cd0 0x7f72081a68e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.366+0000 7f720e893640 1 --2- 172.21.15.86:0/2057865254 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f7208107260 0x7f7208074790 unknown :-1 s=CLOSED pgs=1911 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.372 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.366+0000 7f720e893640 1 --2- 172.21.15.86:0/2057865254 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f7208102190 0x7f7208074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.366+0000 7f720e893640 1 -- 172.21.15.86:0/2057865254 >> 172.21.15.86:0/2057865254 conn(0x7f72080793a0 msgr2=0x7f72080ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:41.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.366+0000 7f720e893640 1 -- 172.21.15.86:0/2057865254 shutdown_connections 2024-04-21T08:49:41.373 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.366+0000 7f720e893640 1 -- 172.21.15.86:0/2057865254 wait complete. 2024-04-21T08:49:41.376 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:49:41.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.476+0000 7faa22c88640 1 -- 172.21.15.86:0/2332641818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa1c107260 msgr2=0x7faa1c10ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.476+0000 7faa22c88640 1 --2- 172.21.15.86:0/2332641818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa1c107260 0x7faa1c10ee30 secure :-1 s=READY pgs=1916 cs=0 l=1 rev1=1 crypto rx=0x7faa14009ca0 tx=0x7faa1402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.476+0000 7faa22c88640 1 -- 172.21.15.86:0/2332641818 shutdown_connections 2024-04-21T08:49:41.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.476+0000 7faa22c88640 1 --2- 172.21.15.86:0/2332641818 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa1c107260 0x7faa1c10ee30 unknown :-1 s=CLOSED pgs=1916 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.476+0000 7faa22c88640 1 --2- 172.21.15.86:0/2332641818 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa1c102b40 0x7faa1c106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.476+0000 7faa22c88640 1 --2- 172.21.15.86:0/2332641818 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa1c102190 0x7faa1c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.476+0000 7faa22c88640 1 -- 172.21.15.86:0/2332641818 >> 172.21.15.86:0/2332641818 conn(0x7faa1c0793a0 msgr2=0x7faa1c0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:41.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.476+0000 7faa22c88640 1 -- 172.21.15.86:0/2332641818 shutdown_connections 2024-04-21T08:49:41.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.476+0000 7faa22c88640 1 -- 172.21.15.86:0/2332641818 wait complete. 2024-04-21T08:49:41.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.476+0000 7faa22c88640 1 Processor -- start 2024-04-21T08:49:41.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.476+0000 7faa22c88640 1 -- start start 2024-04-21T08:49:41.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.477+0000 7faa22c88640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa1c102190 0x7faa1c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.477+0000 7faa22c88640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa1c107260 0x7faa1c198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.477+0000 7faa22c88640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa1c19cfd0 0x7faa1c19d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.477+0000 7faa22c88640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c1121b0 con 0x7faa1c107260 2024-04-21T08:49:41.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.477+0000 7faa22c88640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c112030 con 0x7faa1c19cfd0 2024-04-21T08:49:41.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.477+0000 7faa22c88640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c112330 con 0x7faa1c102190 2024-04-21T08:49:41.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.477+0000 7faa1bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa1c107260 0x7faa1c198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.477+0000 7faa1bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa1c107260 0x7faa1c198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36018/0 (socket says 172.21.15.86:36018) 2024-04-21T08:49:41.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.477+0000 7faa1bfff640 1 -- 172.21.15.86:0/1694834267 learned_addr learned my addr 172.21.15.86:0/1694834267 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:41.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.477+0000 7faa21225640 1 --2- 172.21.15.86:0/1694834267 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa1c19cfd0 0x7faa1c19d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.477+0000 7faa20a24640 1 --2- 172.21.15.86:0/1694834267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa1c102190 0x7faa1c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.477+0000 7faa1bfff640 1 -- 172.21.15.86:0/1694834267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa1c102190 msgr2=0x7faa1c198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.477+0000 7faa1bfff640 1 --2- 172.21.15.86:0/1694834267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa1c102190 0x7faa1c198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.477+0000 7faa1bfff640 1 -- 172.21.15.86:0/1694834267 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa1c19cfd0 msgr2=0x7faa1c19d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.477+0000 7faa1bfff640 1 --2- 172.21.15.86:0/1694834267 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa1c19cfd0 0x7faa1c19d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.477+0000 7faa1bfff640 1 -- 172.21.15.86:0/1694834267 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faa1c19dac0 con 0x7faa1c107260 2024-04-21T08:49:41.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.477+0000 7faa21225640 1 --2- 172.21.15.86:0/1694834267 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa1c19cfd0 0x7faa1c19d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:41.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.477+0000 7faa20a24640 1 --2- 172.21.15.86:0/1694834267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa1c102190 0x7faa1c198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:41.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.477+0000 7faa1bfff640 1 --2- 172.21.15.86:0/1694834267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa1c107260 0x7faa1c198c50 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7faa0800d770 tx=0x7faa0800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:41.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.477+0000 7faa19ffb640 1 -- 172.21.15.86:0/1694834267 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa08017070 con 0x7faa1c107260 2024-04-21T08:49:41.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.477+0000 7faa22c88640 1 -- 172.21.15.86:0/1694834267 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faa1c19ddb0 con 0x7faa1c107260 2024-04-21T08:49:41.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.477+0000 7faa19ffb640 1 -- 172.21.15.86:0/1694834267 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa08009c30 con 0x7faa1c107260 2024-04-21T08:49:41.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.477+0000 7faa19ffb640 1 -- 172.21.15.86:0/1694834267 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa0800f620 con 0x7faa1c107260 2024-04-21T08:49:41.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.478+0000 7faa22c88640 1 -- 172.21.15.86:0/1694834267 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faa1c1a55a0 con 0x7faa1c107260 2024-04-21T08:49:41.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.480+0000 7faa19ffb640 1 -- 172.21.15.86:0/1694834267 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faa0801a070 con 0x7faa1c107260 2024-04-21T08:49:41.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.480+0000 7faa2587e740 1 Processor -- start 2024-04-21T08:49:41.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.480+0000 7faa2587e740 1 -- start start 2024-04-21T08:49:41.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.480+0000 7faa2587e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d859494d0 0x564d859498b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.480+0000 7faa2587e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d85949e80 0x564d85952430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.480+0000 7faa2587e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d85952a00 0x564d85954e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.480+0000 7faa2587e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564d858ae1f0 con 0x564d85952a00 2024-04-21T08:49:41.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.480+0000 7faa2587e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564d8584b4b0 con 0x564d85949e80 2024-04-21T08:49:41.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.480+0000 7faa2587e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564d858aa1b0 con 0x564d859494d0 2024-04-21T08:49:41.488 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.480+0000 7faa1bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d85952a00 0x564d85954e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.480+0000 7faa1bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d85952a00 0x564d85954e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36024/0 (socket says 172.21.15.86:36024) 2024-04-21T08:49:41.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.480+0000 7faa1bfff640 1 -- 172.21.15.86:0/2847802110 learned_addr learned my addr 172.21.15.86:0/2847802110 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:41.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.480+0000 7faa20a24640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d85949e80 0x564d85952430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.480+0000 7faa21225640 1 --2- 172.21.15.86:0/2847802110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d859494d0 0x564d859498b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.480+0000 7faa19ffb640 1 --2- 172.21.15.86:0/1694834267 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7faa0c0a3d20 0x7faa0c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.481+0000 7faa1bfff640 1 -- 172.21.15.86:0/2847802110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d859494d0 msgr2=0x564d859498b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.481+0000 7faa1bfff640 1 --2- 172.21.15.86:0/2847802110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d859494d0 0x564d859498b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.481+0000 7faa1bfff640 1 -- 172.21.15.86:0/2847802110 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d85949e80 msgr2=0x564d85952430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.481+0000 7faa1bfff640 1 --2- 172.21.15.86:0/2847802110 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d85949e80 0x564d85952430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.481+0000 7faa19ffb640 1 -- 172.21.15.86:0/1694834267 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7faa080d0760 con 0x7faa1c107260 2024-04-21T08:49:41.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.481+0000 7faa1bfff640 1 -- 172.21.15.86:0/2847802110 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d8583d1b0 con 0x564d85952a00 2024-04-21T08:49:41.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.481+0000 7faa20a24640 1 --2- 172.21.15.86:0/2847802110 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d85949e80 0x564d85952430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:49:41.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.481+0000 7faa21225640 1 --2- 172.21.15.86:0/2847802110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d859494d0 0x564d859498b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:41.489 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.481+0000 7faa21225640 1 --2- 172.21.15.86:0/1694834267 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7faa0c0a3d20 0x7faa0c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.481+0000 7faa1bfff640 1 --2- 172.21.15.86:0/2847802110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d85952a00 0x564d85954e50 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7faa080cda50 tx=0x7faa0800df10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cdafcbb86a13b04e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:41.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.481+0000 7faa06ffd640 1 -- 172.21.15.86:0/2847802110 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa08021030 con 0x564d85952a00 2024-04-21T08:49:41.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.481+0000 7faa06ffd640 1 -- 172.21.15.86:0/2847802110 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa08009de0 con 0x564d85952a00 2024-04-21T08:49:41.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.481+0000 7faa06ffd640 1 -- 172.21.15.86:0/2847802110 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa080a06e0 con 0x564d85952a00 2024-04-21T08:49:41.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.482+0000 7faa2587e740 1 -- 172.21.15.86:0/2847802110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d85952a00 msgr2=0x564d85954e50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.482+0000 7faa21225640 1 --2- 172.21.15.86:0/1694834267 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7faa0c0a3d20 0x7faa0c0a61e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7faa14006010 tx=0x7faa1402e040 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:41.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.482+0000 7faa2587e740 1 --2- 172.21.15.86:0/2847802110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d85952a00 0x564d85954e50 secure :-1 s=READY pgs=1918 cs=0 l=1 rev1=1 crypto rx=0x7faa080cda50 tx=0x7faa0800df10 comp rx=0 tx=0).stop 2024-04-21T08:49:41.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.482+0000 7faa2587e740 1 -- 172.21.15.86:0/2847802110 shutdown_connections 2024-04-21T08:49:41.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.482+0000 7faa2587e740 1 --2- 172.21.15.86:0/2847802110 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d85952a00 0x564d85954e50 unknown :-1 s=CLOSED pgs=1918 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.482+0000 7faa2587e740 1 --2- 172.21.15.86:0/2847802110 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d85949e80 0x564d85952430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.482+0000 7faa2587e740 1 --2- 172.21.15.86:0/2847802110 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d859494d0 0x564d859498b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.482+0000 7faa2587e740 1 -- 172.21.15.86:0/2847802110 >> 172.21.15.86:0/2847802110 conn(0x564d85943ba0 msgr2=0x564d85943f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:41.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.482+0000 7faa2587e740 1 -- 172.21.15.86:0/2847802110 shutdown_connections 2024-04-21T08:49:41.490 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.482+0000 7faa2587e740 1 -- 172.21.15.86:0/2847802110 wait complete. 2024-04-21T08:49:41.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.482+0000 7faa2587e740 1 -- start start 2024-04-21T08:49:41.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.482+0000 7faa2587e740 1 Processor -- start 2024-04-21T08:49:41.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.482+0000 7faa2587e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d859494d0 0x564d85952320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.482+0000 7faa2587e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d85949e80 0x564d8594f6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.483+0000 7faa2587e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d85952a00 0x564d8594fc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.483+0000 7faa2587e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564d85955910 con 0x564d85949e80 2024-04-21T08:49:41.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.483+0000 7faa2587e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564d85867500 con 0x564d85952a00 2024-04-21T08:49:41.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.483+0000 7faa2587e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564d85955a90 con 0x564d859494d0 2024-04-21T08:49:41.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.483+0000 7faa2587e740 10 client.0 _ll_register_callbacks cb 0x564d85835b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:41.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.483+0000 7faa1bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d85949e80 0x564d8594f6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.483+0000 7faa1bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d85949e80 0x564d8594f6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36040/0 (socket says 172.21.15.86:36040) 2024-04-21T08:49:41.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.483+0000 7faa1bfff640 1 -- 172.21.15.86:0/1781129128 learned_addr learned my addr 172.21.15.86:0/1781129128 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:41.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.483+0000 7faa21225640 1 --2- 172.21.15.86:0/1781129128 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d85952a00 0x564d8594fc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.483+0000 7faa20a24640 1 --2- 172.21.15.86:0/1781129128 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d859494d0 0x564d85952320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.491 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.483+0000 7faa1bfff640 1 -- 172.21.15.86:0/1781129128 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d859494d0 msgr2=0x564d85952320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.483+0000 7faa1bfff640 1 --2- 172.21.15.86:0/1781129128 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d859494d0 0x564d85952320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.483+0000 7faa1bfff640 1 -- 172.21.15.86:0/1781129128 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d85952a00 msgr2=0x564d8594fc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.483+0000 7faa1bfff640 1 --2- 172.21.15.86:0/1781129128 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d85952a00 0x564d8594fc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.483+0000 7faa1bfff640 1 -- 172.21.15.86:0/1781129128 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d855c03f0 con 0x564d85949e80 2024-04-21T08:49:41.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.483+0000 7faa1bfff640 1 --2- 172.21.15.86:0/1781129128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d85949e80 0x564d8594f6b0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7faa0800dec0 tx=0x7faa080cdf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=37bb78f90b191f71 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:41.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.483+0000 7fa9fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:41.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.483+0000 7fa9fbfff640 1 -- 172.21.15.86:0/1781129128 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa08021040 con 0x564d85949e80 2024-04-21T08:49:41.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.483+0000 7fa9fbfff640 1 -- 172.21.15.86:0/1781129128 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7faa080989a0 con 0x564d85949e80 2024-04-21T08:49:41.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.483+0000 7faa2587e740 1 -- 172.21.15.86:0/1781129128 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa08009de0 con 0x564d85949e80 2024-04-21T08:49:41.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.483+0000 7fa9fbfff640 1 -- 172.21.15.86:0/1781129128 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faa0809be70 con 0x564d85949e80 2024-04-21T08:49:41.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.483+0000 7faa2587e740 10 client.19986 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:41.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.483+0000 7faa2587e740 1 -- 172.21.15.86:0/1781129128 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564d8568a7d0 con 0x564d85949e80 2024-04-21T08:49:41.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.484+0000 7fa9fbfff640 1 -- 172.21.15.86:0/1781129128 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa08009de0 con 0x564d85949e80 2024-04-21T08:49:41.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.484+0000 7fa9fbfff640 1 -- 172.21.15.86:0/1781129128 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7faa080bb120 con 0x564d85949e80 2024-04-21T08:49:41.492 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.484+0000 7faa2587e740 10 client.19986 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:41.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.484+0000 7faa2587e740 10 client.19986 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:49:41.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.484+0000 7faa2587e740 20 client.19986 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:41.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.484+0000 7faa2587e740 1 --2- 172.21.15.86:0/1781129128 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x564d85962a40 0x564d85964e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.484+0000 7faa2587e740 4 client.19986 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:41.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.484+0000 7faa2587e740 1 -- 172.21.15.86:0/1781129128 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564d8584d930 con 0x564d85962a40 2024-04-21T08:49:41.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.484+0000 7faa20a24640 1 --2- 172.21.15.86:0/1781129128 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x564d85962a40 0x564d85964e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.484+0000 7fa9fbfff640 10 client.19986 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:49:41.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.484+0000 7faa20a24640 1 --2- 172.21.15.86:0/1781129128 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x564d85962a40 0x564d85964e30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=670b6ba0a161efd6 server_cookie=2a8a913b345fdafd in_seq=0 out_seq=0 2024-04-21T08:49:41.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.485+0000 7fa9fbfff640 1 -- 172.21.15.86:0/1781129128 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564d8584d930 con 0x564d85962a40 2024-04-21T08:49:41.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.485+0000 7fa9fbfff640 10 client.19986 handle_command_reply: tid=0 2024-04-21T08:49:41.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.485+0000 7faa2587e740 1 client.19986 shutdown 2024-04-21T08:49:41.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.485+0000 7faa2587e740 1 -- 172.21.15.86:0/1781129128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d85949e80 msgr2=0x564d8594f6b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.485+0000 7faa2587e740 1 --2- 172.21.15.86:0/1781129128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d85949e80 0x564d8594f6b0 secure :-1 s=READY pgs=1919 cs=0 l=1 rev1=1 crypto rx=0x7faa0800dec0 tx=0x7faa080cdf90 comp rx=0 tx=0).stop 2024-04-21T08:49:41.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.485+0000 7faa2587e740 1 -- 172.21.15.86:0/1781129128 shutdown_connections 2024-04-21T08:49:41.493 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.485+0000 7faa2587e740 1 --2- 172.21.15.86:0/1781129128 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x564d85962a40 0x564d85964e30 crc :-1 s=READY pgs=73 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.485+0000 7faa2587e740 1 --2- 172.21.15.86:0/1781129128 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d85952a00 0x564d8594fc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.485+0000 7fa9fbfff640 0 client.19986 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:49:41.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.485+0000 7faa2587e740 1 --2- 172.21.15.86:0/1781129128 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d85949e80 0x564d8594f6b0 unknown :-1 s=CLOSED pgs=1919 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.485+0000 7faa2587e740 1 --2- 172.21.15.86:0/1781129128 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d859494d0 0x564d85952320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.485+0000 7faa2587e740 1 -- 172.21.15.86:0/1781129128 >> 172.21.15.86:0/1781129128 conn(0x564d85943ba0 msgr2=0x564d859458f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:41.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.485+0000 7faa2587e740 1 -- 172.21.15.86:0/1781129128 shutdown_connections 2024-04-21T08:49:41.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.485+0000 7faa2587e740 1 -- 172.21.15.86:0/1781129128 wait complete. 2024-04-21T08:49:41.494 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.485+0000 7faa2587e740 20 client.19986 trim_cache size 0 max 16384 2024-04-21T08:49:41.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.498+0000 7faa2587e740 1 Processor -- start 2024-04-21T08:49:41.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.498+0000 7faa2587e740 1 -- start start 2024-04-21T08:49:41.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.499+0000 7faa2587e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d859494d0 0x564d85944ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.499+0000 7faa2587e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d85949e80 0x564d85967e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.499+0000 7faa2587e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d85952a00 0x564d85968390 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.499+0000 7faa2587e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c1af430 con 0x564d85952a00 2024-04-21T08:49:41.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.499+0000 7faa2587e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c1b0f70 con 0x564d859494d0 2024-04-21T08:49:41.503 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.499+0000 7faa2587e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c1afb00 con 0x564d85949e80 2024-04-21T08:49:41.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.499+0000 7faa21225640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d85952a00 0x564d85968390 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.499+0000 7faa21225640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d85952a00 0x564d85968390 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36050/0 (socket says 172.21.15.86:36050) 2024-04-21T08:49:41.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.499+0000 7faa21225640 1 -- 172.21.15.86:0/166097889 learned_addr learned my addr 172.21.15.86:0/166097889 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:41.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.499+0000 7faa20a24640 1 --2- 172.21.15.86:0/166097889 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d859494d0 0x564d85944ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.499+0000 7faa1bfff640 1 --2- 172.21.15.86:0/166097889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d85949e80 0x564d85967e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.499+0000 7faa21225640 1 -- 172.21.15.86:0/166097889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d85949e80 msgr2=0x564d85967e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.499+0000 7faa21225640 1 --2- 172.21.15.86:0/166097889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d85949e80 0x564d85967e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.499+0000 7faa21225640 1 -- 172.21.15.86:0/166097889 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d859494d0 msgr2=0x564d85944ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.499+0000 7faa21225640 1 --2- 172.21.15.86:0/166097889 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d859494d0 0x564d85944ff0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.499+0000 7faa21225640 1 -- 172.21.15.86:0/166097889 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564d85945ea0 con 0x564d85952a00 2024-04-21T08:49:41.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.499+0000 7faa20a24640 1 --2- 172.21.15.86:0/166097889 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d859494d0 0x564d85944ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:41.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.499+0000 7faa1bfff640 1 --2- 172.21.15.86:0/166097889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d85949e80 0x564d85967e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:41.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.499+0000 7faa21225640 1 --2- 172.21.15.86:0/166097889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d85952a00 0x564d85968390 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7faa14036000 tx=0x7faa1403f000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2452c646db640c6f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:41.504 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.499+0000 7faa067fc640 1 -- 172.21.15.86:0/166097889 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa14051070 con 0x564d85952a00 2024-04-21T08:49:41.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.499+0000 7faa067fc640 1 -- 172.21.15.86:0/166097889 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa14044070 con 0x564d85952a00 2024-04-21T08:49:41.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.499+0000 7faa067fc640 1 -- 172.21.15.86:0/166097889 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa14039040 con 0x564d85952a00 2024-04-21T08:49:41.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.500+0000 7faa2587e740 1 -- 172.21.15.86:0/166097889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d85952a00 msgr2=0x564d85968390 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.500+0000 7faa2587e740 1 --2- 172.21.15.86:0/166097889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d85952a00 0x564d85968390 secure :-1 s=READY pgs=1920 cs=0 l=1 rev1=1 crypto rx=0x7faa14036000 tx=0x7faa1403f000 comp rx=0 tx=0).stop 2024-04-21T08:49:41.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.500+0000 7faa2587e740 1 -- 172.21.15.86:0/166097889 shutdown_connections 2024-04-21T08:49:41.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.500+0000 7faa2587e740 1 --2- 172.21.15.86:0/166097889 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d85952a00 0x564d85968390 unknown :-1 s=CLOSED pgs=1920 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.500+0000 7faa2587e740 1 --2- 172.21.15.86:0/166097889 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d85949e80 0x564d85967e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.500+0000 7faa2587e740 1 --2- 172.21.15.86:0/166097889 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d859494d0 0x564d85944ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.500+0000 7faa2587e740 1 -- 172.21.15.86:0/166097889 >> 172.21.15.86:0/166097889 conn(0x564d85943ba0 msgr2=0x564d8595e570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:41.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.500+0000 7faa2587e740 1 -- 172.21.15.86:0/166097889 shutdown_connections 2024-04-21T08:49:41.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.500+0000 7faa2587e740 1 -- 172.21.15.86:0/166097889 wait complete. 2024-04-21T08:49:41.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.500+0000 7faa2587e740 1 -- start start 2024-04-21T08:49:41.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.500+0000 7faa2587e740 1 Processor -- start 2024-04-21T08:49:41.505 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.501+0000 7faa2587e740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d859494d0 0x564d85968310 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.501+0000 7faa2587e740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d85949e80 0x564d859657a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.501+0000 7faa2587e740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d85952a00 0x564d85965d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.501+0000 7faa2587e740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c1b08a0 con 0x564d85949e80 2024-04-21T08:49:41.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.501+0000 7faa2587e740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c1b01d0 con 0x564d859494d0 2024-04-21T08:49:41.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.501+0000 7faa2587e740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faa1c1ad430 con 0x564d85952a00 2024-04-21T08:49:41.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.501+0000 7faa2587e740 10 client.0 _ll_register_callbacks cb 0x7faa080d0a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:41.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.501+0000 7faa1bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d85949e80 0x564d859657a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.501+0000 7faa1bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d85949e80 0x564d859657a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36064/0 (socket says 172.21.15.86:36064) 2024-04-21T08:49:41.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.501+0000 7faa1bfff640 1 -- 172.21.15.86:0/3054901205 learned_addr learned my addr 172.21.15.86:0/3054901205 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:41.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.501+0000 7faa21225640 1 --2- 172.21.15.86:0/3054901205 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d85952a00 0x564d85965d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.501+0000 7faa1bfff640 1 -- 172.21.15.86:0/3054901205 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d85952a00 msgr2=0x564d85965d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.501+0000 7faa20a24640 1 --2- 172.21.15.86:0/3054901205 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d859494d0 0x564d85968310 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.501+0000 7faa1bfff640 1 --2- 172.21.15.86:0/3054901205 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d85952a00 0x564d85965d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.506 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.501+0000 7faa1bfff640 1 -- 172.21.15.86:0/3054901205 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d859494d0 msgr2=0x564d85968310 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.501+0000 7faa1bfff640 1 --2- 172.21.15.86:0/3054901205 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d859494d0 0x564d85968310 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.501+0000 7faa1bfff640 1 -- 172.21.15.86:0/3054901205 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564d85966570 con 0x564d85949e80 2024-04-21T08:49:41.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.501+0000 7faa1bfff640 1 --2- 172.21.15.86:0/3054901205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d85949e80 0x564d859657a0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7faa0809c030 tx=0x7faa0809c230 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4763ec64c4db1260 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:41.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.501+0000 7fa9fbfff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:41.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.502+0000 7fa9fbfff640 1 -- 172.21.15.86:0/3054901205 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faa08021070 con 0x564d85949e80 2024-04-21T08:49:41.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.502+0000 7fa9fbfff640 1 -- 172.21.15.86:0/3054901205 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7faa08098080 con 0x564d85949e80 2024-04-21T08:49:41.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.502+0000 7faa2587e740 1 -- 172.21.15.86:0/3054901205 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faa14044070 con 0x564d85949e80 2024-04-21T08:49:41.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.502+0000 7fa9fbfff640 1 -- 172.21.15.86:0/3054901205 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faa0809bc90 con 0x564d85949e80 2024-04-21T08:49:41.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.502+0000 7faa2587e740 10 client.19992 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:41.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.502+0000 7faa2587e740 1 -- 172.21.15.86:0/3054901205 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564d859c8cf0 con 0x564d85949e80 2024-04-21T08:49:41.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.502+0000 7fa9fbfff640 1 -- 172.21.15.86:0/3054901205 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faa14044070 con 0x564d85949e80 2024-04-21T08:49:41.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.502+0000 7fa9fbfff640 1 -- 172.21.15.86:0/3054901205 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7faa080d0340 con 0x564d85949e80 2024-04-21T08:49:41.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.502+0000 7faa2587e740 10 client.19992 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:41.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.502+0000 7faa2587e740 10 client.19992 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:49:41.507 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.502+0000 7faa2587e740 20 client.19992 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:41.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.502+0000 7faa2587e740 1 --2- 172.21.15.86:0/3054901205 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x564d859c9080 0x564d859cb470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.502+0000 7faa2587e740 4 client.19992 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:49:41.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.502+0000 7faa2587e740 1 -- 172.21.15.86:0/3054901205 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564d858bbc10 con 0x564d859c9080 2024-04-21T08:49:41.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.502+0000 7faa20a24640 1 --2- 172.21.15.86:0/3054901205 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x564d859c9080 0x564d859cb470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.503+0000 7fa9fbfff640 10 client.19992 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:49:41.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.503+0000 7faa20a24640 1 --2- 172.21.15.86:0/3054901205 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x564d859c9080 0x564d859cb470 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=64612f5cbf55b572 server_cookie=9b1264523ca9a1f5 in_seq=0 out_seq=0 2024-04-21T08:49:41.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.503+0000 7fa9fbfff640 1 -- 172.21.15.86:0/3054901205 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564d858bbc10 con 0x564d859c9080 2024-04-21T08:49:41.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.503+0000 7fa9fbfff640 10 client.19992 handle_command_reply: tid=0 2024-04-21T08:49:41.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.503+0000 7faa2587e740 1 client.19992 shutdown 2024-04-21T08:49:41.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.503+0000 7faa2587e740 1 -- 172.21.15.86:0/3054901205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d85949e80 msgr2=0x564d859657a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.503+0000 7faa2587e740 1 --2- 172.21.15.86:0/3054901205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d85949e80 0x564d859657a0 secure :-1 s=READY pgs=1921 cs=0 l=1 rev1=1 crypto rx=0x7faa0809c030 tx=0x7faa0809c230 comp rx=0 tx=0).stop 2024-04-21T08:49:41.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.503+0000 7faa2587e740 1 -- 172.21.15.86:0/3054901205 shutdown_connections 2024-04-21T08:49:41.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.503+0000 7faa2587e740 1 --2- 172.21.15.86:0/3054901205 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x564d859c9080 0x564d859cb470 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.503+0000 7faa2587e740 1 --2- 172.21.15.86:0/3054901205 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564d85952a00 0x564d85965d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.508 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.503+0000 7fa9fbfff640 0 client.19992 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:49:41.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.503+0000 7faa2587e740 1 --2- 172.21.15.86:0/3054901205 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564d85949e80 0x564d859657a0 unknown :-1 s=CLOSED pgs=1921 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.503+0000 7faa2587e740 1 --2- 172.21.15.86:0/3054901205 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564d859494d0 0x564d85968310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.503+0000 7faa2587e740 1 -- 172.21.15.86:0/3054901205 >> 172.21.15.86:0/3054901205 conn(0x564d85943ba0 msgr2=0x564d8594a340 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:41.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.503+0000 7faa2587e740 1 -- 172.21.15.86:0/3054901205 shutdown_connections 2024-04-21T08:49:41.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.503+0000 7faa2587e740 1 -- 172.21.15.86:0/3054901205 wait complete. 2024-04-21T08:49:41.509 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.503+0000 7faa2587e740 20 client.19992 trim_cache size 0 max 16384 2024-04-21T08:49:41.509 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:49:41.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.504+0000 7faa22c88640 1 -- 172.21.15.86:0/1694834267 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7faa0c0a3d20 msgr2=0x7faa0c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.504+0000 7faa22c88640 1 --2- 172.21.15.86:0/1694834267 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7faa0c0a3d20 0x7faa0c0a61e0 secure :-1 s=READY pgs=319 cs=0 l=1 rev1=1 crypto rx=0x7faa14006010 tx=0x7faa1402e040 comp rx=0 tx=0).stop 2024-04-21T08:49:41.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.504+0000 7faa22c88640 1 -- 172.21.15.86:0/1694834267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa1c107260 msgr2=0x7faa1c198c50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.504+0000 7faa22c88640 1 --2- 172.21.15.86:0/1694834267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa1c107260 0x7faa1c198c50 secure :-1 s=READY pgs=1917 cs=0 l=1 rev1=1 crypto rx=0x7faa0800d770 tx=0x7faa0800dc30 comp rx=0 tx=0).stop 2024-04-21T08:49:41.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.504+0000 7faa22c88640 1 -- 172.21.15.86:0/1694834267 shutdown_connections 2024-04-21T08:49:41.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.504+0000 7faa22c88640 1 --2- 172.21.15.86:0/1694834267 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7faa0c0a3d20 0x7faa0c0a61e0 unknown :-1 s=CLOSED pgs=319 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.504+0000 7faa22c88640 1 --2- 172.21.15.86:0/1694834267 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faa1c19cfd0 0x7faa1c19d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.504+0000 7faa22c88640 1 --2- 172.21.15.86:0/1694834267 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faa1c107260 0x7faa1c198c50 unknown :-1 s=CLOSED pgs=1917 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.510 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.504+0000 7faa22c88640 1 --2- 172.21.15.86:0/1694834267 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faa1c102190 0x7faa1c198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.504+0000 7faa22c88640 1 -- 172.21.15.86:0/1694834267 >> 172.21.15.86:0/1694834267 conn(0x7faa1c0793a0 msgr2=0x7faa1c0ffe60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:41.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.504+0000 7faa22c88640 1 -- 172.21.15.86:0/1694834267 shutdown_connections 2024-04-21T08:49:41.511 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.504+0000 7faa22c88640 1 -- 172.21.15.86:0/1694834267 wait complete. 2024-04-21T08:49:41.514 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:49:41.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.614+0000 7f8ed8cc1640 1 -- 172.21.15.86:0/2235668016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ed41010a0 msgr2=0x7f8ed4101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.614+0000 7f8ed8cc1640 1 --2- 172.21.15.86:0/2235668016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ed41010a0 0x7f8ed4101480 secure :-1 s=READY pgs=1922 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4009900 tx=0x7f8ec402bc30 comp rx=0 tx=0).stop 2024-04-21T08:49:41.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.614+0000 7f8ed8cc1640 1 -- 172.21.15.86:0/2235668016 shutdown_connections 2024-04-21T08:49:41.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.614+0000 7f8ed8cc1640 1 --2- 172.21.15.86:0/2235668016 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ed4106170 0x7f8ed410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.614+0000 7f8ed8cc1640 1 --2- 172.21.15.86:0/2235668016 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ed4101a50 0x7f8ed4105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.614+0000 7f8ed8cc1640 1 --2- 172.21.15.86:0/2235668016 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ed41010a0 0x7f8ed4101480 unknown :-1 s=CLOSED pgs=1922 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.614+0000 7f8ed8cc1640 1 -- 172.21.15.86:0/2235668016 >> 172.21.15.86:0/2235668016 conn(0x7f8ed40771d0 msgr2=0x7f8ed40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:41.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.614+0000 7f8ed8cc1640 1 -- 172.21.15.86:0/2235668016 shutdown_connections 2024-04-21T08:49:41.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.615+0000 7f8ed8cc1640 1 -- 172.21.15.86:0/2235668016 wait complete. 2024-04-21T08:49:41.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.615+0000 7f8ed8cc1640 1 Processor -- start 2024-04-21T08:49:41.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.615+0000 7f8ed8cc1640 1 -- start start 2024-04-21T08:49:41.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.615+0000 7f8ed8cc1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ed41010a0 0x7f8ed4102b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.615+0000 7f8ed8cc1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ed4106170 0x7f8ed4103070 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.615+0000 7f8ed8cc1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ed41035b0 0x7f8ed41a69c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.615+0000 7f8ed8cc1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed41110e0 con 0x7f8ed4106170 2024-04-21T08:49:41.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.615+0000 7f8ed8cc1640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed4110f60 con 0x7f8ed41035b0 2024-04-21T08:49:41.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.615+0000 7f8ed8cc1640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed4111260 con 0x7f8ed41010a0 2024-04-21T08:49:41.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.615+0000 7f8ed1d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ed4106170 0x7f8ed4103070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.616+0000 7f8ed1d9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ed4106170 0x7f8ed4103070 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36090/0 (socket says 172.21.15.86:36090) 2024-04-21T08:49:41.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.616+0000 7f8ed1d9b640 1 -- 172.21.15.86:0/421602614 learned_addr learned my addr 172.21.15.86:0/421602614 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:41.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.616+0000 7f8ed259c640 1 --2- 172.21.15.86:0/421602614 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ed41010a0 0x7f8ed4102b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.616+0000 7f8ed2d9d640 1 --2- 172.21.15.86:0/421602614 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ed41035b0 0x7f8ed41a69c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.616+0000 7f8ed1d9b640 1 -- 172.21.15.86:0/421602614 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ed41010a0 msgr2=0x7f8ed4102b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.616+0000 7f8ed1d9b640 1 --2- 172.21.15.86:0/421602614 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ed41010a0 0x7f8ed4102b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.616+0000 7f8ed1d9b640 1 -- 172.21.15.86:0/421602614 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ed41035b0 msgr2=0x7f8ed41a69c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.616+0000 7f8ed1d9b640 1 --2- 172.21.15.86:0/421602614 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ed41035b0 0x7f8ed41a69c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.616+0000 7f8ed1d9b640 1 -- 172.21.15.86:0/421602614 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8ed41a70c0 con 0x7f8ed4106170 2024-04-21T08:49:41.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.616+0000 7f8ed1d9b640 1 --2- 172.21.15.86:0/421602614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ed4106170 0x7f8ed4103070 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f8ec800d770 tx=0x7f8ec800dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:41.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.616+0000 7f8ec37fe640 1 -- 172.21.15.86:0/421602614 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec8017070 con 0x7f8ed4106170 2024-04-21T08:49:41.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.616+0000 7f8ec37fe640 1 -- 172.21.15.86:0/421602614 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ec8009c30 con 0x7f8ed4106170 2024-04-21T08:49:41.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.616+0000 7f8ed8cc1640 1 -- 172.21.15.86:0/421602614 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8ed41a73b0 con 0x7f8ed4106170 2024-04-21T08:49:41.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.616+0000 7f8ec37fe640 1 -- 172.21.15.86:0/421602614 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec800f620 con 0x7f8ed4106170 2024-04-21T08:49:41.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.616+0000 7f8ed8cc1640 1 -- 172.21.15.86:0/421602614 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8ed41a7900 con 0x7f8ed4106170 2024-04-21T08:49:41.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.618+0000 7f8ec37fe640 1 -- 172.21.15.86:0/421602614 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8ec801a070 con 0x7f8ed4106170 2024-04-21T08:49:41.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.618+0000 7f8edb993740 1 Processor -- start 2024-04-21T08:49:41.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.619+0000 7f8edb993740 1 -- start start 2024-04-21T08:49:41.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.619+0000 7f8edb993740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646409924d0 0x5646409928b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.619+0000 7f8edb993740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564640992e80 0x56464099b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.619+0000 7f8edb993740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56464099ba00 0x56464099de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.619+0000 7f8edb993740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5646408f71f0 con 0x5646409924d0 2024-04-21T08:49:41.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.619+0000 7f8edb993740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5646408944b0 con 0x564640992e80 2024-04-21T08:49:41.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.619+0000 7f8edb993740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5646408f31b0 con 0x56464099ba00 2024-04-21T08:49:41.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.619+0000 7f8ed2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646409924d0 0x5646409928b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.626 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.619+0000 7f8ed2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646409924d0 0x5646409928b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36100/0 (socket says 172.21.15.86:36100) 2024-04-21T08:49:41.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.619+0000 7f8ed2d9d640 1 -- 172.21.15.86:0/4025009534 learned_addr learned my addr 172.21.15.86:0/4025009534 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:41.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.619+0000 7f8ed1d9b640 1 --2- 172.21.15.86:0/4025009534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56464099ba00 0x56464099de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.619+0000 7f8ed2d9d640 1 -- 172.21.15.86:0/4025009534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56464099ba00 msgr2=0x56464099de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.619+0000 7f8ed2d9d640 1 --2- 172.21.15.86:0/4025009534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56464099ba00 0x56464099de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.619+0000 7f8ed2d9d640 1 -- 172.21.15.86:0/4025009534 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564640992e80 msgr2=0x56464099b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.619+0000 7f8ed2d9d640 1 --2- 172.21.15.86:0/4025009534 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564640992e80 0x56464099b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.619+0000 7f8ed2d9d640 1 -- 172.21.15.86:0/4025009534 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5646408861b0 con 0x5646409924d0 2024-04-21T08:49:41.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.619+0000 7f8ed1d9b640 1 --2- 172.21.15.86:0/4025009534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56464099ba00 0x56464099de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:41.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.619+0000 7f8ec37fe640 1 --2- 172.21.15.86:0/421602614 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8ebc0a3d50 0x7f8ebc0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.619+0000 7f8ed2d9d640 1 --2- 172.21.15.86:0/4025009534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646409924d0 0x5646409928b0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc008d40 tx=0x7f8ecc00e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fefec6c6157b8d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:41.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.619+0000 7f8ed259c640 1 --2- 172.21.15.86:0/421602614 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8ebc0a3d50 0x7f8ebc0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.619+0000 7f8ec37fe640 1 -- 172.21.15.86:0/421602614 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f8ec80d0780 con 0x7f8ed4106170 2024-04-21T08:49:41.627 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.620+0000 7f8ec0ff9640 1 -- 172.21.15.86:0/4025009534 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc014070 con 0x5646409924d0 2024-04-21T08:49:41.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.620+0000 7f8ec0ff9640 1 -- 172.21.15.86:0/4025009534 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ecc003b70 con 0x5646409924d0 2024-04-21T08:49:41.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.620+0000 7f8ec0ff9640 1 -- 172.21.15.86:0/4025009534 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc010670 con 0x5646409924d0 2024-04-21T08:49:41.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.620+0000 7f8edb993740 1 -- 172.21.15.86:0/4025009534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646409924d0 msgr2=0x5646409928b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.620+0000 7f8ed259c640 1 --2- 172.21.15.86:0/421602614 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8ebc0a3d50 0x7f8ebc0a6210 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x564640991fa0 tx=0x7f8ec4005a20 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:41.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.620+0000 7f8edb993740 1 --2- 172.21.15.86:0/4025009534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646409924d0 0x5646409928b0 secure :-1 s=READY pgs=1924 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc008d40 tx=0x7f8ecc00e720 comp rx=0 tx=0).stop 2024-04-21T08:49:41.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.620+0000 7f8edb993740 1 -- 172.21.15.86:0/4025009534 shutdown_connections 2024-04-21T08:49:41.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.620+0000 7f8edb993740 1 --2- 172.21.15.86:0/4025009534 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56464099ba00 0x56464099de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.620+0000 7f8edb993740 1 --2- 172.21.15.86:0/4025009534 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564640992e80 0x56464099b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.620+0000 7f8edb993740 1 --2- 172.21.15.86:0/4025009534 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646409924d0 0x5646409928b0 unknown :-1 s=CLOSED pgs=1924 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.620+0000 7f8edb993740 1 -- 172.21.15.86:0/4025009534 >> 172.21.15.86:0/4025009534 conn(0x56464098cba0 msgr2=0x56464098cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:41.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.620+0000 7f8edb993740 1 -- 172.21.15.86:0/4025009534 shutdown_connections 2024-04-21T08:49:41.628 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.620+0000 7f8edb993740 1 -- 172.21.15.86:0/4025009534 wait complete. 2024-04-21T08:49:41.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.621+0000 7f8edb993740 1 -- start start 2024-04-21T08:49:41.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.621+0000 7f8edb993740 1 Processor -- start 2024-04-21T08:49:41.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.621+0000 7f8edb993740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646409924d0 0x56464099b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.621+0000 7f8edb993740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564640992e80 0x5646409986f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.621+0000 7f8edb993740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56464099ba00 0x564640998d00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.621+0000 7f8edb993740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56464099e970 con 0x5646409924d0 2024-04-21T08:49:41.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.621+0000 7f8edb993740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5646408b0500 con 0x56464099ba00 2024-04-21T08:49:41.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.621+0000 7f8edb993740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56464099eaf0 con 0x564640992e80 2024-04-21T08:49:41.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.621+0000 7f8edb993740 10 client.0 _ll_register_callbacks cb 0x56464087eb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:41.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.621+0000 7f8ed259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646409924d0 0x56464099b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.621+0000 7f8ed259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646409924d0 0x56464099b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36116/0 (socket says 172.21.15.86:36116) 2024-04-21T08:49:41.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.621+0000 7f8ed259c640 1 -- 172.21.15.86:0/3669806216 learned_addr learned my addr 172.21.15.86:0/3669806216 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:41.629 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.621+0000 7f8ed259c640 1 -- 172.21.15.86:0/3669806216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564640992e80 msgr2=0x5646409986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.621+0000 7f8ed259c640 1 --2- 172.21.15.86:0/3669806216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564640992e80 0x5646409986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.621+0000 7f8ed259c640 1 -- 172.21.15.86:0/3669806216 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56464099ba00 msgr2=0x564640998d00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.621+0000 7f8ed2d9d640 1 --2- 172.21.15.86:0/3669806216 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56464099ba00 0x564640998d00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.621+0000 7f8ed259c640 1 --2- 172.21.15.86:0/3669806216 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56464099ba00 0x564640998d00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.621+0000 7f8ed259c640 1 -- 172.21.15.86:0/3669806216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5646406093f0 con 0x5646409924d0 2024-04-21T08:49:41.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.621+0000 7f8ed259c640 1 --2- 172.21.15.86:0/3669806216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646409924d0 0x56464099b360 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4035000 tx=0x7f8ec402bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4bc7e5a4324ca1bc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:41.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.621+0000 7f8eb9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:41.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.622+0000 7f8eb9ffb640 1 -- 172.21.15.86:0/3669806216 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec404e020 con 0x5646409924d0 2024-04-21T08:49:41.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.622+0000 7f8eb9ffb640 1 -- 172.21.15.86:0/3669806216 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f8ec40048c0 con 0x5646409924d0 2024-04-21T08:49:41.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.622+0000 7f8edb993740 1 -- 172.21.15.86:0/3669806216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ecc003b70 con 0x5646409924d0 2024-04-21T08:49:41.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.622+0000 7f8eb9ffb640 1 -- 172.21.15.86:0/3669806216 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8ec4059050 con 0x5646409924d0 2024-04-21T08:49:41.630 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.622+0000 7f8edb993740 10 client.20004 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:41.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.622+0000 7f8edb993740 1 -- 172.21.15.86:0/3669806216 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5646406d37d0 con 0x5646409924d0 2024-04-21T08:49:41.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.622+0000 7f8eb9ffb640 1 -- 172.21.15.86:0/3669806216 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ecc003b70 con 0x5646409924d0 2024-04-21T08:49:41.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.622+0000 7f8eb9ffb640 1 -- 172.21.15.86:0/3669806216 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8ec4003b40 con 0x5646409924d0 2024-04-21T08:49:41.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.622+0000 7f8edb993740 10 client.20004 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:41.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.622+0000 7f8edb993740 10 client.20004 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:49:41.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.622+0000 7f8edb993740 20 client.20004 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:41.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.622+0000 7f8edb993740 1 --2- 172.21.15.86:0/3669806216 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5646409abc10 0x5646409ae0d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.622+0000 7f8edb993740 4 client.20004 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:41.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.622+0000 7f8edb993740 1 -- 172.21.15.86:0/3669806216 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564640896930 con 0x5646409abc10 2024-04-21T08:49:41.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.623+0000 7f8ed1d9b640 1 --2- 172.21.15.86:0/3669806216 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5646409abc10 0x5646409ae0d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.623+0000 7f8eb9ffb640 10 client.20004 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:49:41.631 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.623+0000 7f8ed1d9b640 1 --2- 172.21.15.86:0/3669806216 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5646409abc10 0x5646409ae0d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=a6dfca3a412edb20 server_cookie=51ad25f2b67969f9 in_seq=0 out_seq=0 2024-04-21T08:49:41.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.624+0000 7f8eb9ffb640 1 -- 172.21.15.86:0/3669806216 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564640896930 con 0x5646409abc10 2024-04-21T08:49:41.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.624+0000 7f8eb9ffb640 10 client.20004 handle_command_reply: tid=0 2024-04-21T08:49:41.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.624+0000 7f8edb993740 1 client.20004 shutdown 2024-04-21T08:49:41.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.624+0000 7f8edb993740 1 -- 172.21.15.86:0/3669806216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646409924d0 msgr2=0x56464099b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.624+0000 7f8edb993740 1 --2- 172.21.15.86:0/3669806216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646409924d0 0x56464099b360 secure :-1 s=READY pgs=1925 cs=0 l=1 rev1=1 crypto rx=0x7f8ec4035000 tx=0x7f8ec402bc30 comp rx=0 tx=0).stop 2024-04-21T08:49:41.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.624+0000 7f8edb993740 1 -- 172.21.15.86:0/3669806216 shutdown_connections 2024-04-21T08:49:41.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.624+0000 7f8edb993740 1 --2- 172.21.15.86:0/3669806216 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56464099ba00 0x564640998d00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.624+0000 7f8edb993740 1 --2- 172.21.15.86:0/3669806216 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564640992e80 0x5646409986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.624+0000 7f8edb993740 1 --2- 172.21.15.86:0/3669806216 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x5646409abc10 0x5646409ae0d0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.624+0000 7f8edb993740 1 --2- 172.21.15.86:0/3669806216 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646409924d0 0x56464099b360 unknown :-1 s=CLOSED pgs=1925 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.624+0000 7f8eb9ffb640 0 client.20004 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:49:41.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.624+0000 7f8edb993740 1 -- 172.21.15.86:0/3669806216 >> 172.21.15.86:0/3669806216 conn(0x56464098cba0 msgr2=0x56464098e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:41.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.624+0000 7f8edb993740 1 -- 172.21.15.86:0/3669806216 shutdown_connections 2024-04-21T08:49:41.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.624+0000 7f8edb993740 1 -- 172.21.15.86:0/3669806216 wait complete. 2024-04-21T08:49:41.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.624+0000 7f8edb993740 20 client.20004 trim_cache size 0 max 16384 2024-04-21T08:49:41.642 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.637+0000 7f8edb993740 1 Processor -- start 2024-04-21T08:49:41.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.637+0000 7f8edb993740 1 -- start start 2024-04-21T08:49:41.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.637+0000 7f8edb993740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646409924d0 0x5646409a7830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.637+0000 7f8edb993740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564640992e80 0x56464099ae20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.637+0000 7f8edb993740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56464099ba00 0x56464099b360 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.637+0000 7f8edb993740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed41b1560 con 0x56464099ba00 2024-04-21T08:49:41.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.637+0000 7f8edb993740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed41b30a0 con 0x564640992e80 2024-04-21T08:49:41.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.637+0000 7f8edb993740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed41b1c30 con 0x5646409924d0 2024-04-21T08:49:41.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.637+0000 7f8ed2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56464099ba00 0x56464099b360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.637+0000 7f8ed2d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56464099ba00 0x56464099b360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36120/0 (socket says 172.21.15.86:36120) 2024-04-21T08:49:41.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.637+0000 7f8ed2d9d640 1 -- 172.21.15.86:0/1932597564 learned_addr learned my addr 172.21.15.86:0/1932597564 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:41.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.637+0000 7f8ed1d9b640 1 --2- 172.21.15.86:0/1932597564 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564640992e80 0x56464099ae20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.643 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.638+0000 7f8ed259c640 1 --2- 172.21.15.86:0/1932597564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646409924d0 0x5646409a7830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.638+0000 7f8ed2d9d640 1 -- 172.21.15.86:0/1932597564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646409924d0 msgr2=0x5646409a7830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.638+0000 7f8ed2d9d640 1 --2- 172.21.15.86:0/1932597564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646409924d0 0x5646409a7830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.638+0000 7f8ed2d9d640 1 -- 172.21.15.86:0/1932597564 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564640992e80 msgr2=0x56464099ae20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.638+0000 7f8ed2d9d640 1 --2- 172.21.15.86:0/1932597564 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564640992e80 0x56464099ae20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.638+0000 7f8ed2d9d640 1 -- 172.21.15.86:0/1932597564 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564640999070 con 0x56464099ba00 2024-04-21T08:49:41.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.638+0000 7f8ed2d9d640 1 --2- 172.21.15.86:0/1932597564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56464099ba00 0x56464099b360 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc0084b0 tx=0x7f8ecc008c40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e90e56d434975679 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:41.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.638+0000 7f8ebbfff640 1 -- 172.21.15.86:0/1932597564 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc025070 con 0x56464099ba00 2024-04-21T08:49:41.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.638+0000 7f8ebbfff640 1 -- 172.21.15.86:0/1932597564 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ecc004d80 con 0x56464099ba00 2024-04-21T08:49:41.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.638+0000 7f8ebbfff640 1 -- 172.21.15.86:0/1932597564 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ecc014050 con 0x56464099ba00 2024-04-21T08:49:41.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.638+0000 7f8edb993740 1 -- 172.21.15.86:0/1932597564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56464099ba00 msgr2=0x56464099b360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.638+0000 7f8edb993740 1 --2- 172.21.15.86:0/1932597564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56464099ba00 0x56464099b360 secure :-1 s=READY pgs=1926 cs=0 l=1 rev1=1 crypto rx=0x7f8ecc0084b0 tx=0x7f8ecc008c40 comp rx=0 tx=0).stop 2024-04-21T08:49:41.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.638+0000 7f8edb993740 1 -- 172.21.15.86:0/1932597564 shutdown_connections 2024-04-21T08:49:41.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.638+0000 7f8edb993740 1 --2- 172.21.15.86:0/1932597564 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56464099ba00 0x56464099b360 unknown :-1 s=CLOSED pgs=1926 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.638+0000 7f8edb993740 1 --2- 172.21.15.86:0/1932597564 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564640992e80 0x56464099ae20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.638+0000 7f8edb993740 1 --2- 172.21.15.86:0/1932597564 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5646409924d0 0x5646409a7830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.639+0000 7f8edb993740 1 -- 172.21.15.86:0/1932597564 >> 172.21.15.86:0/1932597564 conn(0x56464098cba0 msgr2=0x564640993c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:41.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.639+0000 7f8edb993740 1 -- 172.21.15.86:0/1932597564 shutdown_connections 2024-04-21T08:49:41.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.639+0000 7f8edb993740 1 -- 172.21.15.86:0/1932597564 wait complete. 2024-04-21T08:49:41.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.639+0000 7f8edb993740 1 -- start start 2024-04-21T08:49:41.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.639+0000 7f8edb993740 1 Processor -- start 2024-04-21T08:49:41.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.639+0000 7f8edb993740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646409924d0 0x5646409a42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.639+0000 7f8edb993740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564640992e80 0x56464099f690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.639+0000 7f8edb993740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56464099ba00 0x56464099fdd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.645 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.639+0000 7f8edb993740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed41b29d0 con 0x5646409924d0 2024-04-21T08:49:41.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.639+0000 7f8edb993740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed41b2300 con 0x564640992e80 2024-04-21T08:49:41.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.639+0000 7f8edb993740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8ed41af560 con 0x56464099ba00 2024-04-21T08:49:41.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.639+0000 7f8edb993740 10 client.0 _ll_register_callbacks cb 0x7f8ec402c7e0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:41.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.639+0000 7f8ed259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646409924d0 0x5646409a42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.639+0000 7f8ed259c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646409924d0 0x5646409a42d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36132/0 (socket says 172.21.15.86:36132) 2024-04-21T08:49:41.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.639+0000 7f8ed259c640 1 -- 172.21.15.86:0/3763981322 learned_addr learned my addr 172.21.15.86:0/3763981322 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:41.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.639+0000 7f8ed2d9d640 1 --2- 172.21.15.86:0/3763981322 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56464099ba00 0x56464099fdd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.639+0000 7f8ed1d9b640 1 --2- 172.21.15.86:0/3763981322 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564640992e80 0x56464099f690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.639+0000 7f8ed259c640 1 -- 172.21.15.86:0/3763981322 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56464099ba00 msgr2=0x56464099fdd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.639+0000 7f8ed259c640 1 --2- 172.21.15.86:0/3763981322 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56464099ba00 0x56464099fdd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.646 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.639+0000 7f8ed259c640 1 -- 172.21.15.86:0/3763981322 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564640992e80 msgr2=0x56464099f690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.639+0000 7f8ed259c640 1 --2- 172.21.15.86:0/3763981322 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564640992e80 0x56464099f690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.639+0000 7f8ed259c640 1 -- 172.21.15.86:0/3763981322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564640a19a70 con 0x5646409924d0 2024-04-21T08:49:41.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.639+0000 7f8ed2d9d640 1 --2- 172.21.15.86:0/3763981322 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56464099ba00 0x56464099fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:41.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.640+0000 7f8ed259c640 1 --2- 172.21.15.86:0/3763981322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646409924d0 0x5646409a42d0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f8ec402c170 tx=0x7f8ec40038a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=48368ecfcadac726 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:41.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.640+0000 7f8eb9ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:41.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.640+0000 7f8eb9ffb640 1 -- 172.21.15.86:0/3763981322 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8ec4061020 con 0x5646409924d0 2024-04-21T08:49:41.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.640+0000 7f8eb9ffb640 1 -- 172.21.15.86:0/3763981322 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f8ec40046c0 con 0x5646409924d0 2024-04-21T08:49:41.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.640+0000 7f8edb993740 1 -- 172.21.15.86:0/3763981322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8ecc004d80 con 0x5646409924d0 2024-04-21T08:49:41.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.640+0000 7f8eb9ffb640 1 -- 172.21.15.86:0/3763981322 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8ec406c050 con 0x5646409924d0 2024-04-21T08:49:41.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.640+0000 7f8edb993740 10 client.20010 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:41.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.640+0000 7f8edb993740 1 -- 172.21.15.86:0/3763981322 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564640a1ac90 con 0x5646409924d0 2024-04-21T08:49:41.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.640+0000 7f8eb9ffb640 1 -- 172.21.15.86:0/3763981322 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8ecc004d80 con 0x5646409924d0 2024-04-21T08:49:41.647 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.640+0000 7f8eb9ffb640 1 -- 172.21.15.86:0/3763981322 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8ec40409b0 con 0x5646409924d0 2024-04-21T08:49:41.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.640+0000 7f8edb993740 10 client.20010 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:41.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.640+0000 7f8edb993740 10 client.20010 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:49:41.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.640+0000 7f8edb993740 20 client.20010 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:41.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.640+0000 7f8edb993740 1 --2- 172.21.15.86:0/3763981322 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x564640a1b430 0x564640a1d850 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.640+0000 7f8edb993740 4 client.20010 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:49:41.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.640+0000 7f8edb993740 1 -- 172.21.15.86:0/3763981322 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564640904c10 con 0x564640a1b430 2024-04-21T08:49:41.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.641+0000 7f8ed1d9b640 1 --2- 172.21.15.86:0/3763981322 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x564640a1b430 0x564640a1d850 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.641+0000 7f8eb9ffb640 10 client.20010 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:49:41.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.641+0000 7f8ed1d9b640 1 --2- 172.21.15.86:0/3763981322 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x564640a1b430 0x564640a1d850 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=9ee5ccc8063a39fe server_cookie=e0c40288c5f84da5 in_seq=0 out_seq=0 2024-04-21T08:49:41.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.641+0000 7f8eb9ffb640 1 -- 172.21.15.86:0/3763981322 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564640904c10 con 0x564640a1b430 2024-04-21T08:49:41.648 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.641+0000 7f8eb9ffb640 10 client.20010 handle_command_reply: tid=0 2024-04-21T08:49:41.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.641+0000 7f8edb993740 1 client.20010 shutdown 2024-04-21T08:49:41.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.641+0000 7f8edb993740 1 -- 172.21.15.86:0/3763981322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646409924d0 msgr2=0x5646409a42d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.641+0000 7f8edb993740 1 --2- 172.21.15.86:0/3763981322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646409924d0 0x5646409a42d0 secure :-1 s=READY pgs=1927 cs=0 l=1 rev1=1 crypto rx=0x7f8ec402c170 tx=0x7f8ec40038a0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.641+0000 7f8edb993740 1 -- 172.21.15.86:0/3763981322 shutdown_connections 2024-04-21T08:49:41.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.641+0000 7f8edb993740 1 --2- 172.21.15.86:0/3763981322 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56464099ba00 0x56464099fdd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.641+0000 7f8edb993740 1 --2- 172.21.15.86:0/3763981322 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564640992e80 0x56464099f690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.641+0000 7f8edb993740 1 --2- 172.21.15.86:0/3763981322 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x564640a1b430 0x564640a1d850 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.641+0000 7f8edb993740 1 --2- 172.21.15.86:0/3763981322 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5646409924d0 0x5646409a42d0 unknown :-1 s=CLOSED pgs=1927 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.641+0000 7f8eb9ffb640 0 client.20010 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:49:41.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.641+0000 7f8edb993740 1 -- 172.21.15.86:0/3763981322 >> 172.21.15.86:0/3763981322 conn(0x56464098cba0 msgr2=0x5646409932d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:41.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.641+0000 7f8edb993740 1 -- 172.21.15.86:0/3763981322 shutdown_connections 2024-04-21T08:49:41.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.641+0000 7f8edb993740 1 -- 172.21.15.86:0/3763981322 wait complete. 2024-04-21T08:49:41.649 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.641+0000 7f8edb993740 20 client.20010 trim_cache size 0 max 16384 2024-04-21T08:49:41.650 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:49:41.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.642+0000 7f8ed8cc1640 1 -- 172.21.15.86:0/421602614 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8ebc0a3d50 msgr2=0x7f8ebc0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.642+0000 7f8ed8cc1640 1 --2- 172.21.15.86:0/421602614 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8ebc0a3d50 0x7f8ebc0a6210 secure :-1 s=READY pgs=320 cs=0 l=1 rev1=1 crypto rx=0x564640991fa0 tx=0x7f8ec4005a20 comp rx=0 tx=0).stop 2024-04-21T08:49:41.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.642+0000 7f8ed8cc1640 1 -- 172.21.15.86:0/421602614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ed4106170 msgr2=0x7f8ed4103070 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.642+0000 7f8ed8cc1640 1 --2- 172.21.15.86:0/421602614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ed4106170 0x7f8ed4103070 secure :-1 s=READY pgs=1923 cs=0 l=1 rev1=1 crypto rx=0x7f8ec800d770 tx=0x7f8ec800dc30 comp rx=0 tx=0).stop 2024-04-21T08:49:41.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.642+0000 7f8ed8cc1640 1 -- 172.21.15.86:0/421602614 shutdown_connections 2024-04-21T08:49:41.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.642+0000 7f8ed8cc1640 1 --2- 172.21.15.86:0/421602614 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8ebc0a3d50 0x7f8ebc0a6210 unknown :-1 s=CLOSED pgs=320 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.642+0000 7f8ed8cc1640 1 --2- 172.21.15.86:0/421602614 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8ed41035b0 0x7f8ed41a69c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.642+0000 7f8ed8cc1640 1 --2- 172.21.15.86:0/421602614 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8ed4106170 0x7f8ed4103070 unknown :-1 s=CLOSED pgs=1923 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.642+0000 7f8ed8cc1640 1 --2- 172.21.15.86:0/421602614 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8ed41010a0 0x7f8ed4102b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.642+0000 7f8ed8cc1640 1 -- 172.21.15.86:0/421602614 >> 172.21.15.86:0/421602614 conn(0x7f8ed40771d0 msgr2=0x7f8ed4104e70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:41.651 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.642+0000 7f8ed8cc1640 1 -- 172.21.15.86:0/421602614 shutdown_connections 2024-04-21T08:49:41.652 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.642+0000 7f8ed8cc1640 1 -- 172.21.15.86:0/421602614 wait complete. 2024-04-21T08:49:41.653 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:49:41.754 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.753+0000 7f9e4bc3a640 1 -- 172.21.15.86:0/4257287089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e44101110 msgr2=0x7f9e441014f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.753+0000 7f9e4bc3a640 1 --2- 172.21.15.86:0/4257287089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e44101110 0x7f9e441014f0 secure :-1 s=READY pgs=1928 cs=0 l=1 rev1=1 crypto rx=0x7f9e38009900 tx=0x7f9e3802bc30 comp rx=0 tx=0).stop 2024-04-21T08:49:41.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.753+0000 7f9e4bc3a640 1 -- 172.21.15.86:0/4257287089 shutdown_connections 2024-04-21T08:49:41.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.753+0000 7f9e4bc3a640 1 --2- 172.21.15.86:0/4257287089 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e441061e0 0x7f9e4410ddb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.753+0000 7f9e4bc3a640 1 --2- 172.21.15.86:0/4257287089 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e44101ac0 0x7f9e44105b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.753+0000 7f9e4bc3a640 1 --2- 172.21.15.86:0/4257287089 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e44101110 0x7f9e441014f0 unknown :-1 s=CLOSED pgs=1928 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.753+0000 7f9e4bc3a640 1 -- 172.21.15.86:0/4257287089 >> 172.21.15.86:0/4257287089 conn(0x7f9e44077f80 msgr2=0x7f9e44078390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:41.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.753+0000 7f9e4bc3a640 1 -- 172.21.15.86:0/4257287089 shutdown_connections 2024-04-21T08:49:41.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.753+0000 7f9e4bc3a640 1 -- 172.21.15.86:0/4257287089 wait complete. 2024-04-21T08:49:41.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.753+0000 7f9e4bc3a640 1 Processor -- start 2024-04-21T08:49:41.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.753+0000 7f9e4bc3a640 1 -- start start 2024-04-21T08:49:41.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.753+0000 7f9e4bc3a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e44101110 0x7f9e441997b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.753+0000 7f9e4bc3a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e441061e0 0x7f9e44199cf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.753+0000 7f9e4bc3a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e4419a230 0x7f9e441a4830 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.753+0000 7f9e4bc3a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e441111a0 con 0x7f9e4419a230 2024-04-21T08:49:41.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.753+0000 7f9e4bc3a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9e44111020 con 0x7f9e44101110 2024-04-21T08:49:41.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.753+0000 7f9e4bc3a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9e44111320 con 0x7f9e441061e0 2024-04-21T08:49:41.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.754+0000 7f9e4a1d7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e4419a230 0x7f9e441a4830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.754+0000 7f9e499d6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e44101110 0x7f9e441997b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.754+0000 7f9e4a1d7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e4419a230 0x7f9e441a4830 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36156/0 (socket says 172.21.15.86:36156) 2024-04-21T08:49:41.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.754+0000 7f9e4a1d7640 1 -- 172.21.15.86:0/1059431495 learned_addr learned my addr 172.21.15.86:0/1059431495 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:41.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.754+0000 7f9e491d5640 1 --2- 172.21.15.86:0/1059431495 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e441061e0 0x7f9e44199cf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.754+0000 7f9e4a1d7640 1 -- 172.21.15.86:0/1059431495 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e441061e0 msgr2=0x7f9e44199cf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.754+0000 7f9e4a1d7640 1 --2- 172.21.15.86:0/1059431495 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e441061e0 0x7f9e44199cf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.754+0000 7f9e4a1d7640 1 -- 172.21.15.86:0/1059431495 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e44101110 msgr2=0x7f9e441997b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.754+0000 7f9e4a1d7640 1 --2- 172.21.15.86:0/1059431495 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e44101110 0x7f9e441997b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.754+0000 7f9e4a1d7640 1 -- 172.21.15.86:0/1059431495 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9e441a4fb0 con 0x7f9e4419a230 2024-04-21T08:49:41.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.754+0000 7f9e499d6640 1 --2- 172.21.15.86:0/1059431495 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e44101110 0x7f9e441997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:49:41.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.754+0000 7f9e4a1d7640 1 --2- 172.21.15.86:0/1059431495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e4419a230 0x7f9e441a4830 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c00b970 tx=0x7f9e3c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:41.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.754+0000 7f9e36ffd640 1 -- 172.21.15.86:0/1059431495 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c019070 con 0x7f9e4419a230 2024-04-21T08:49:41.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.754+0000 7f9e4bc3a640 1 -- 172.21.15.86:0/1059431495 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9e441a52a0 con 0x7f9e4419a230 2024-04-21T08:49:41.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.754+0000 7f9e36ffd640 1 -- 172.21.15.86:0/1059431495 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e3c014440 con 0x7f9e4419a230 2024-04-21T08:49:41.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.754+0000 7f9e36ffd640 1 -- 172.21.15.86:0/1059431495 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3c0075d0 con 0x7f9e4419a230 2024-04-21T08:49:41.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.754+0000 7f9e4bc3a640 1 -- 172.21.15.86:0/1059431495 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9e441a57a0 con 0x7f9e4419a230 2024-04-21T08:49:41.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.757+0000 7f9e36ffd640 1 -- 172.21.15.86:0/1059431495 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9e3c01c070 con 0x7f9e4419a230 2024-04-21T08:49:41.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.757+0000 7f9e4e820740 1 Processor -- start 2024-04-21T08:49:41.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.757+0000 7f9e4e820740 1 -- start start 2024-04-21T08:49:41.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.757+0000 7f9e4e820740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b57979d4d0 0x55b57979d8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.757+0000 7f9e4e820740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57979de80 0x55b5797a6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.757+0000 7f9e4e820740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5797a6a00 0x55b5797a8e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.757+0000 7f9e4e820740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b5797021f0 con 0x55b57979de80 2024-04-21T08:49:41.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.757+0000 7f9e4e820740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b57969f4b0 con 0x55b5797a6a00 2024-04-21T08:49:41.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.757+0000 7f9e4e820740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b5796fe1b0 con 0x55b57979d4d0 2024-04-21T08:49:41.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.758+0000 7f9e499d6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57979de80 0x55b5797a6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.758+0000 7f9e499d6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57979de80 0x55b5797a6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36166/0 (socket says 172.21.15.86:36166) 2024-04-21T08:49:41.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.758+0000 7f9e499d6640 1 -- 172.21.15.86:0/1408462814 learned_addr learned my addr 172.21.15.86:0/1408462814 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:41.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.758+0000 7f9e36ffd640 1 --2- 172.21.15.86:0/1059431495 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f9e300a3d20 0x7f9e300a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.758+0000 7f9e491d5640 1 --2- 172.21.15.86:0/1408462814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5797a6a00 0x55b5797a8e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.758+0000 7f9e491d5640 1 --2- 172.21.15.86:0/1408462814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b57979d4d0 0x55b57979d8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.758+0000 7f9e4a1d7640 1 --2- 172.21.15.86:0/1059431495 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f9e300a3d20 0x7f9e300a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.758+0000 7f9e499d6640 1 -- 172.21.15.86:0/1408462814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b57979d4d0 msgr2=0x55b57979d8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.758+0000 7f9e499d6640 1 --2- 172.21.15.86:0/1408462814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b57979d4d0 0x55b57979d8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.758+0000 7f9e499d6640 1 -- 172.21.15.86:0/1408462814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5797a6a00 msgr2=0x55b5797a8e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.758+0000 7f9e499d6640 1 --2- 172.21.15.86:0/1408462814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5797a6a00 0x55b5797a8e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.758+0000 7f9e36ffd640 1 -- 172.21.15.86:0/1059431495 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f9e3c0d0050 con 0x7f9e4419a230 2024-04-21T08:49:41.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.758+0000 7f9e499d6640 1 -- 172.21.15.86:0/1408462814 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5796911b0 con 0x55b57979de80 2024-04-21T08:49:41.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.758+0000 7f9e491d5640 1 --2- 172.21.15.86:0/1408462814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5797a6a00 0x55b5797a8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:49:41.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.758+0000 7f9e499d6640 1 --2- 172.21.15.86:0/1408462814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57979de80 0x55b5797a6430 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f9e38009900 tx=0x7f9e3802c730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=38a2b1981fe91cfa server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:41.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.758+0000 7f9e4a1d7640 1 --2- 172.21.15.86:0/1059431495 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f9e300a3d20 0x7f9e300a61e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0cd590 tx=0x7f9e3c066040 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:41.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.758+0000 7f9e2ffff640 1 -- 172.21.15.86:0/1408462814 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e38045070 con 0x55b57979de80 2024-04-21T08:49:41.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.758+0000 7f9e2ffff640 1 -- 172.21.15.86:0/1408462814 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e3802c7e0 con 0x55b57979de80 2024-04-21T08:49:41.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.758+0000 7f9e2ffff640 1 -- 172.21.15.86:0/1408462814 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e38003cd0 con 0x55b57979de80 2024-04-21T08:49:41.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.759+0000 7f9e4e820740 1 -- 172.21.15.86:0/1408462814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57979de80 msgr2=0x55b5797a6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.759+0000 7f9e4e820740 1 --2- 172.21.15.86:0/1408462814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57979de80 0x55b5797a6430 secure :-1 s=READY pgs=1930 cs=0 l=1 rev1=1 crypto rx=0x7f9e38009900 tx=0x7f9e3802c730 comp rx=0 tx=0).stop 2024-04-21T08:49:41.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.759+0000 7f9e4e820740 1 -- 172.21.15.86:0/1408462814 shutdown_connections 2024-04-21T08:49:41.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.759+0000 7f9e4e820740 1 --2- 172.21.15.86:0/1408462814 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5797a6a00 0x55b5797a8e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.759+0000 7f9e4e820740 1 --2- 172.21.15.86:0/1408462814 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57979de80 0x55b5797a6430 unknown :-1 s=CLOSED pgs=1930 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.759+0000 7f9e4e820740 1 --2- 172.21.15.86:0/1408462814 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b57979d4d0 0x55b57979d8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.759+0000 7f9e4e820740 1 -- 172.21.15.86:0/1408462814 >> 172.21.15.86:0/1408462814 conn(0x55b579797ba0 msgr2=0x55b579797f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:41.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.759+0000 7f9e4e820740 1 -- 172.21.15.86:0/1408462814 shutdown_connections 2024-04-21T08:49:41.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.759+0000 7f9e4e820740 1 -- 172.21.15.86:0/1408462814 wait complete. 2024-04-21T08:49:41.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.760+0000 7f9e4e820740 1 -- start start 2024-04-21T08:49:41.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.760+0000 7f9e4e820740 1 Processor -- start 2024-04-21T08:49:41.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.760+0000 7f9e4e820740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b57979d4d0 0x55b5797a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.760+0000 7f9e4e820740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57979de80 0x55b5797a3670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.760+0000 7f9e4e820740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5797a6a00 0x55b5797a3bd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.760+0000 7f9e4e820740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b5797a9910 con 0x55b57979de80 2024-04-21T08:49:41.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.760+0000 7f9e4e820740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b5796bb500 con 0x55b5797a6a00 2024-04-21T08:49:41.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.760+0000 7f9e4e820740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b5797a9a90 con 0x55b57979d4d0 2024-04-21T08:49:41.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.760+0000 7f9e4e820740 10 client.0 _ll_register_callbacks cb 0x55b579689b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:41.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.760+0000 7f9e499d6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57979de80 0x55b5797a3670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.760+0000 7f9e499d6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57979de80 0x55b5797a3670 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36172/0 (socket says 172.21.15.86:36172) 2024-04-21T08:49:41.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.760+0000 7f9e499d6640 1 -- 172.21.15.86:0/3327147895 learned_addr learned my addr 172.21.15.86:0/3327147895 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:41.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.760+0000 7f9e491d5640 1 --2- 172.21.15.86:0/3327147895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b57979d4d0 0x55b5797a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.760+0000 7f9e491d5640 1 --2- 172.21.15.86:0/3327147895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5797a6a00 0x55b5797a3bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.760+0000 7f9e499d6640 1 -- 172.21.15.86:0/3327147895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b57979d4d0 msgr2=0x55b5797a62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.760+0000 7f9e499d6640 1 --2- 172.21.15.86:0/3327147895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b57979d4d0 0x55b5797a62e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.760+0000 7f9e499d6640 1 -- 172.21.15.86:0/3327147895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5797a6a00 msgr2=0x55b5797a3bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.760+0000 7f9e499d6640 1 --2- 172.21.15.86:0/3327147895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5797a6a00 0x55b5797a3bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.760+0000 7f9e499d6640 1 -- 172.21.15.86:0/3327147895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b5794143f0 con 0x55b57979de80 2024-04-21T08:49:41.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.761+0000 7f9e499d6640 1 --2- 172.21.15.86:0/3327147895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57979de80 0x55b5797a3670 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f9e38036040 tx=0x7f9e380043f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ca57162d0d88d00c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:41.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.761+0000 7f9e2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:41.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.761+0000 7f9e2d7fa640 1 -- 172.21.15.86:0/3327147895 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3804e070 con 0x55b57979de80 2024-04-21T08:49:41.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.761+0000 7f9e2d7fa640 1 -- 172.21.15.86:0/3327147895 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f9e3804a450 con 0x55b57979de80 2024-04-21T08:49:41.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.761+0000 7f9e4e820740 1 -- 172.21.15.86:0/3327147895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e3802c7e0 con 0x55b57979de80 2024-04-21T08:49:41.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.761+0000 7f9e2d7fa640 1 -- 172.21.15.86:0/3327147895 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9e38059050 con 0x55b57979de80 2024-04-21T08:49:41.769 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.761+0000 7f9e4e820740 10 client.20022 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:41.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.761+0000 7f9e4e820740 1 -- 172.21.15.86:0/3327147895 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b5794de7d0 con 0x55b57979de80 2024-04-21T08:49:41.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.761+0000 7f9e2d7fa640 1 -- 172.21.15.86:0/3327147895 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e3802c7e0 con 0x55b57979de80 2024-04-21T08:49:41.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.761+0000 7f9e2d7fa640 1 -- 172.21.15.86:0/3327147895 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9e3803f440 con 0x55b57979de80 2024-04-21T08:49:41.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.761+0000 7f9e4e820740 10 client.20022 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:41.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.761+0000 7f9e4e820740 10 client.20022 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:41.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.761+0000 7f9e4e820740 20 client.20022 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:41.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.761+0000 7f9e4e820740 1 --2- 172.21.15.86:0/3327147895 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b5797b6a00 0x55b5797b8df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.761+0000 7f9e4e820740 4 client.20022 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:41.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.761+0000 7f9e4e820740 1 -- 172.21.15.86:0/3327147895 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b5796a1930 con 0x55b5797b6a00 2024-04-21T08:49:41.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.762+0000 7f9e491d5640 1 --2- 172.21.15.86:0/3327147895 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b5797b6a00 0x55b5797b8df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.762+0000 7f9e2d7fa640 10 client.20022 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:41.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.762+0000 7f9e491d5640 1 --2- 172.21.15.86:0/3327147895 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b5797b6a00 0x55b5797b8df0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=88b3c6178257a10f server_cookie=b62a81726889caa5 in_seq=0 out_seq=0 2024-04-21T08:49:41.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.763+0000 7f9e2d7fa640 1 -- 172.21.15.86:0/3327147895 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b5796a1930 con 0x55b5797b6a00 2024-04-21T08:49:41.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.763+0000 7f9e2d7fa640 10 client.20022 handle_command_reply: tid=0 2024-04-21T08:49:41.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.763+0000 7f9e4e820740 1 client.20022 shutdown 2024-04-21T08:49:41.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.763+0000 7f9e4e820740 1 -- 172.21.15.86:0/3327147895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57979de80 msgr2=0x55b5797a3670 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.763+0000 7f9e4e820740 1 --2- 172.21.15.86:0/3327147895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57979de80 0x55b5797a3670 secure :-1 s=READY pgs=1931 cs=0 l=1 rev1=1 crypto rx=0x7f9e38036040 tx=0x7f9e380043f0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.763+0000 7f9e4e820740 1 -- 172.21.15.86:0/3327147895 shutdown_connections 2024-04-21T08:49:41.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.763+0000 7f9e4e820740 1 --2- 172.21.15.86:0/3327147895 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b5797b6a00 0x55b5797b8df0 crc :-1 s=READY pgs=315 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.763+0000 7f9e4e820740 1 --2- 172.21.15.86:0/3327147895 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5797a6a00 0x55b5797a3bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.763+0000 7f9e4e820740 1 --2- 172.21.15.86:0/3327147895 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57979de80 0x55b5797a3670 unknown :-1 s=CLOSED pgs=1931 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.763+0000 7f9e4e820740 1 --2- 172.21.15.86:0/3327147895 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b57979d4d0 0x55b5797a62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.763+0000 7f9e2d7fa640 0 client.20022 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:41.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.763+0000 7f9e4e820740 1 -- 172.21.15.86:0/3327147895 >> 172.21.15.86:0/3327147895 conn(0x55b579797ba0 msgr2=0x55b5797998f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:41.771 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.763+0000 7f9e4e820740 1 -- 172.21.15.86:0/3327147895 shutdown_connections 2024-04-21T08:49:41.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.763+0000 7f9e4e820740 1 -- 172.21.15.86:0/3327147895 wait complete. 2024-04-21T08:49:41.772 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.763+0000 7f9e4e820740 20 client.20022 trim_cache size 0 max 16384 2024-04-21T08:49:41.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.776+0000 7f9e4e820740 1 Processor -- start 2024-04-21T08:49:41.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.776+0000 7f9e4e820740 1 -- start start 2024-04-21T08:49:41.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.776+0000 7f9e4e820740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b57979d4d0 0x55b5797b23f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.776+0000 7f9e4e820740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57979de80 0x55b5797a5da0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.776+0000 7f9e4e820740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5797a6a00 0x55b5797a62e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.776+0000 7f9e4e820740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e441af420 con 0x55b57979de80 2024-04-21T08:49:41.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.776+0000 7f9e4e820740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9e441b0f60 con 0x55b5797a6a00 2024-04-21T08:49:41.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.776+0000 7f9e4e820740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9e441afaf0 con 0x55b57979d4d0 2024-04-21T08:49:41.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.776+0000 7f9e499d6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57979de80 0x55b5797a5da0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.776+0000 7f9e499d6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57979de80 0x55b5797a5da0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36188/0 (socket says 172.21.15.86:36188) 2024-04-21T08:49:41.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.776+0000 7f9e499d6640 1 -- 172.21.15.86:0/1890522279 learned_addr learned my addr 172.21.15.86:0/1890522279 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:41.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.777+0000 7f9e491d5640 1 --2- 172.21.15.86:0/1890522279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b57979d4d0 0x55b5797b23f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.777+0000 7f9e499d6640 1 -- 172.21.15.86:0/1890522279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b57979d4d0 msgr2=0x55b5797b23f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.777+0000 7f9e499d6640 1 --2- 172.21.15.86:0/1890522279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b57979d4d0 0x55b5797b23f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.777+0000 7f9e499d6640 1 -- 172.21.15.86:0/1890522279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5797a6a00 msgr2=0x55b5797a62e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.777+0000 7f9e499d6640 1 --2- 172.21.15.86:0/1890522279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5797a6a00 0x55b5797a62e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.777+0000 7f9e499d6640 1 -- 172.21.15.86:0/1890522279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b5797a3f00 con 0x55b57979de80 2024-04-21T08:49:41.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.777+0000 7f9e491d5640 1 --2- 172.21.15.86:0/1890522279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b57979d4d0 0x55b5797b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:41.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.777+0000 7f9e499d6640 1 --2- 172.21.15.86:0/1890522279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57979de80 0x55b5797a5da0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f9e38005d00 tx=0x7f9e38003950 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c2465a19a8746fa3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:41.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.777+0000 7f9e2f7fe640 1 -- 172.21.15.86:0/1890522279 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e3804e070 con 0x55b57979de80 2024-04-21T08:49:41.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.777+0000 7f9e2f7fe640 1 -- 172.21.15.86:0/1890522279 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e3802c7e0 con 0x55b57979de80 2024-04-21T08:49:41.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.777+0000 7f9e2f7fe640 1 -- 172.21.15.86:0/1890522279 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e38045040 con 0x55b57979de80 2024-04-21T08:49:41.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.777+0000 7f9e4e820740 1 -- 172.21.15.86:0/1890522279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57979de80 msgr2=0x55b5797a5da0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.783 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.777+0000 7f9e4e820740 1 --2- 172.21.15.86:0/1890522279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57979de80 0x55b5797a5da0 secure :-1 s=READY pgs=1932 cs=0 l=1 rev1=1 crypto rx=0x7f9e38005d00 tx=0x7f9e38003950 comp rx=0 tx=0).stop 2024-04-21T08:49:41.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.777+0000 7f9e4e820740 1 -- 172.21.15.86:0/1890522279 shutdown_connections 2024-04-21T08:49:41.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.777+0000 7f9e4e820740 1 --2- 172.21.15.86:0/1890522279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5797a6a00 0x55b5797a62e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.777+0000 7f9e4e820740 1 --2- 172.21.15.86:0/1890522279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57979de80 0x55b5797a5da0 unknown :-1 s=CLOSED pgs=1932 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.777+0000 7f9e4e820740 1 --2- 172.21.15.86:0/1890522279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b57979d4d0 0x55b5797b23f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.777+0000 7f9e4e820740 1 -- 172.21.15.86:0/1890522279 >> 172.21.15.86:0/1890522279 conn(0x55b579797ba0 msgr2=0x55b5797b1ce0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:41.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.778+0000 7f9e4e820740 1 -- 172.21.15.86:0/1890522279 shutdown_connections 2024-04-21T08:49:41.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.778+0000 7f9e4e820740 1 -- 172.21.15.86:0/1890522279 wait complete. 2024-04-21T08:49:41.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.778+0000 7f9e4e820740 1 -- start start 2024-04-21T08:49:41.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.778+0000 7f9e4e820740 1 Processor -- start 2024-04-21T08:49:41.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.778+0000 7f9e4e820740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b57979d4d0 0x55b5797b99b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.778+0000 7f9e4e820740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57979de80 0x55b5797b9ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.784 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.778+0000 7f9e4e820740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5797a6a00 0x55b5797ba430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.778+0000 7f9e4e820740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9e441b0890 con 0x55b57979de80 2024-04-21T08:49:41.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.778+0000 7f9e4e820740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9e441b01c0 con 0x55b5797a6a00 2024-04-21T08:49:41.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.778+0000 7f9e4e820740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9e441ad420 con 0x55b57979d4d0 2024-04-21T08:49:41.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.778+0000 7f9e4e820740 10 client.0 _ll_register_callbacks cb 0x7f9e3802c680 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:41.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.778+0000 7f9e499d6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57979de80 0x55b5797b9ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.778+0000 7f9e499d6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57979de80 0x55b5797b9ef0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36198/0 (socket says 172.21.15.86:36198) 2024-04-21T08:49:41.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.778+0000 7f9e499d6640 1 -- 172.21.15.86:0/538345171 learned_addr learned my addr 172.21.15.86:0/538345171 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:41.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.779+0000 7f9e491d5640 1 --2- 172.21.15.86:0/538345171 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5797a6a00 0x55b5797ba430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.779+0000 7f9e491d5640 1 --2- 172.21.15.86:0/538345171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b57979d4d0 0x55b5797b99b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.779+0000 7f9e499d6640 1 -- 172.21.15.86:0/538345171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b57979d4d0 msgr2=0x55b5797b99b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.779+0000 7f9e499d6640 1 --2- 172.21.15.86:0/538345171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b57979d4d0 0x55b5797b99b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.785 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.779+0000 7f9e499d6640 1 -- 172.21.15.86:0/538345171 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5797a6a00 msgr2=0x55b5797ba430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:41.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.779+0000 7f9e499d6640 1 --2- 172.21.15.86:0/538345171 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5797a6a00 0x55b5797ba430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.779+0000 7f9e499d6640 1 -- 172.21.15.86:0/538345171 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b57981bc90 con 0x55b57979de80 2024-04-21T08:49:41.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.779+0000 7f9e491d5640 1 --2- 172.21.15.86:0/538345171 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5797a6a00 0x55b5797ba430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:41.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.779+0000 7f9e499d6640 1 --2- 172.21.15.86:0/538345171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57979de80 0x55b5797b9ef0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f9e38005d00 tx=0x7f9e38004600 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6731ed9de107fa1f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:41.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.779+0000 7f9e2d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:41.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.779+0000 7f9e2d7fa640 1 -- 172.21.15.86:0/538345171 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9e38056070 con 0x55b57979de80 2024-04-21T08:49:41.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.779+0000 7f9e2d7fa640 1 -- 172.21.15.86:0/538345171 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7f9e38003e70 con 0x55b57979de80 2024-04-21T08:49:41.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.779+0000 7f9e4e820740 1 -- 172.21.15.86:0/538345171 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f9e3802c7e0 con 0x55b57979de80 2024-04-21T08:49:41.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.779+0000 7f9e2d7fa640 1 -- 172.21.15.86:0/538345171 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f9e3804e040 con 0x55b57979de80 2024-04-21T08:49:41.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.779+0000 7f9e4e820740 10 client.20028 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:41.786 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.779+0000 7f9e4e820740 1 -- 172.21.15.86:0/538345171 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b57981cee0 con 0x55b57979de80 2024-04-21T08:49:41.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.779+0000 7f9e2d7fa640 1 -- 172.21.15.86:0/538345171 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9e3802c7e0 con 0x55b57979de80 2024-04-21T08:49:41.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.779+0000 7f9e2d7fa640 1 -- 172.21.15.86:0/538345171 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f9e3804ab40 con 0x55b57979de80 2024-04-21T08:49:41.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.779+0000 7f9e4e820740 10 client.20028 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:41.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.779+0000 7f9e4e820740 10 client.20028 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:41.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.779+0000 7f9e4e820740 20 client.20028 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:41.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.779+0000 7f9e4e820740 1 --2- 172.21.15.86:0/538345171 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b57981d3e0 0x55b57981f800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:41.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.779+0000 7f9e4e820740 4 client.20028 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:49:41.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.779+0000 7f9e4e820740 1 -- 172.21.15.86:0/538345171 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55b57970fc10 con 0x55b57981d3e0 2024-04-21T08:49:41.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.780+0000 7f9e491d5640 1 --2- 172.21.15.86:0/538345171 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b57981d3e0 0x55b57981f800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:41.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.780+0000 7f9e2d7fa640 10 client.20028 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:41.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.780+0000 7f9e491d5640 1 --2- 172.21.15.86:0/538345171 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b57981d3e0 0x55b57981f800 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=e150f15d8ceee75 server_cookie=7e5e234bb347e1d6 in_seq=0 out_seq=0 2024-04-21T08:49:41.787 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.780+0000 7f9e2d7fa640 1 -- 172.21.15.86:0/538345171 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55b57970fc10 con 0x55b57981d3e0 2024-04-21T08:49:41.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.780+0000 7f9e2d7fa640 10 client.20028 handle_command_reply: tid=0 2024-04-21T08:49:41.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.780+0000 7f9e4e820740 1 client.20028 shutdown 2024-04-21T08:49:41.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.780+0000 7f9e4e820740 1 -- 172.21.15.86:0/538345171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57979de80 msgr2=0x55b5797b9ef0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.780+0000 7f9e4e820740 1 --2- 172.21.15.86:0/538345171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57979de80 0x55b5797b9ef0 secure :-1 s=READY pgs=1933 cs=0 l=1 rev1=1 crypto rx=0x7f9e38005d00 tx=0x7f9e38004600 comp rx=0 tx=0).stop 2024-04-21T08:49:41.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.781+0000 7f9e4e820740 1 -- 172.21.15.86:0/538345171 shutdown_connections 2024-04-21T08:49:41.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.781+0000 7f9e4e820740 1 --2- 172.21.15.86:0/538345171 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b57981d3e0 0x55b57981f800 crc :-1 s=READY pgs=316 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.781+0000 7f9e4e820740 1 --2- 172.21.15.86:0/538345171 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b5797a6a00 0x55b5797ba430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.781+0000 7f9e4e820740 1 --2- 172.21.15.86:0/538345171 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b57979de80 0x55b5797b9ef0 unknown :-1 s=CLOSED pgs=1933 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.781+0000 7f9e4e820740 1 --2- 172.21.15.86:0/538345171 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b57979d4d0 0x55b5797b99b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.781+0000 7f9e2d7fa640 0 client.20028 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:41.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.781+0000 7f9e4e820740 1 -- 172.21.15.86:0/538345171 >> 172.21.15.86:0/538345171 conn(0x55b579797ba0 msgr2=0x55b57979ec40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:41.788 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.781+0000 7f9e4e820740 1 -- 172.21.15.86:0/538345171 shutdown_connections 2024-04-21T08:49:41.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.781+0000 7f9e4e820740 1 -- 172.21.15.86:0/538345171 wait complete. 2024-04-21T08:49:41.789 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.781+0000 7f9e4e820740 20 client.20028 trim_cache size 0 max 16384 2024-04-21T08:49:41.789 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:49:41.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.781+0000 7f9e4bc3a640 1 -- 172.21.15.86:0/1059431495 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f9e300a3d20 msgr2=0x7f9e300a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.781+0000 7f9e4bc3a640 1 --2- 172.21.15.86:0/1059431495 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f9e300a3d20 0x7f9e300a61e0 secure :-1 s=READY pgs=321 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c0cd590 tx=0x7f9e3c066040 comp rx=0 tx=0).stop 2024-04-21T08:49:41.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.781+0000 7f9e4bc3a640 1 -- 172.21.15.86:0/1059431495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e4419a230 msgr2=0x7f9e441a4830 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:41.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.781+0000 7f9e4bc3a640 1 --2- 172.21.15.86:0/1059431495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e4419a230 0x7f9e441a4830 secure :-1 s=READY pgs=1929 cs=0 l=1 rev1=1 crypto rx=0x7f9e3c00b970 tx=0x7f9e3c00be30 comp rx=0 tx=0).stop 2024-04-21T08:49:41.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.781+0000 7f9e4bc3a640 1 -- 172.21.15.86:0/1059431495 shutdown_connections 2024-04-21T08:49:41.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.781+0000 7f9e4bc3a640 1 --2- 172.21.15.86:0/1059431495 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f9e300a3d20 0x7f9e300a61e0 unknown :-1 s=CLOSED pgs=321 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.781+0000 7f9e4bc3a640 1 --2- 172.21.15.86:0/1059431495 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9e4419a230 0x7f9e441a4830 unknown :-1 s=CLOSED pgs=1929 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.781+0000 7f9e4bc3a640 1 --2- 172.21.15.86:0/1059431495 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9e441061e0 0x7f9e44199cf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.781+0000 7f9e4bc3a640 1 --2- 172.21.15.86:0/1059431495 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9e44101110 0x7f9e441997b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:41.790 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.781+0000 7f9e4bc3a640 1 -- 172.21.15.86:0/1059431495 >> 172.21.15.86:0/1059431495 conn(0x7f9e44077f80 msgr2=0x7f9e440ff2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:41.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.781+0000 7f9e4bc3a640 1 -- 172.21.15.86:0/1059431495 shutdown_connections 2024-04-21T08:49:41.791 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:41.781+0000 7f9e4bc3a640 1 -- 172.21.15.86:0/1059431495 wait complete. 2024-04-21T08:49:42.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:41 smithi088 ceph-mon[31259]: pgmap v340: 129 pgs: 129 active+clean; 3.9 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-21T08:49:42.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:41 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2603560076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:49:42.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:41 smithi179 ceph-mon[31829]: pgmap v340: 129 pgs: 129 active+clean; 3.9 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-21T08:49:42.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:41 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2603560076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:49:42.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:41 smithi086 ceph-mon[27939]: pgmap v340: 129 pgs: 129 active+clean; 3.9 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-21T08:49:42.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:41 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2603560076' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:49:42.792 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:49:42.792 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:49:42.792 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:49:42.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.908+0000 7fa46174b640 1 -- 172.21.15.86:0/727009725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa45c102b40 msgr2=0x7fa45c106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:42.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.908+0000 7fa46174b640 1 --2- 172.21.15.86:0/727009725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa45c102b40 0x7fa45c106b80 secure :-1 s=READY pgs=1934 cs=0 l=1 rev1=1 crypto rx=0x7fa450009900 tx=0x7fa45002bc30 comp rx=0 tx=0).stop 2024-04-21T08:49:42.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.908+0000 7fa46174b640 1 -- 172.21.15.86:0/727009725 shutdown_connections 2024-04-21T08:49:42.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.908+0000 7fa46174b640 1 --2- 172.21.15.86:0/727009725 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa45c107260 0x7fa45c10ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:42.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.908+0000 7fa46174b640 1 --2- 172.21.15.86:0/727009725 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa45c102b40 0x7fa45c106b80 unknown :-1 s=CLOSED pgs=1934 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:42.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.908+0000 7fa46174b640 1 --2- 172.21.15.86:0/727009725 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa45c102190 0x7fa45c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:42.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.908+0000 7fa46174b640 1 -- 172.21.15.86:0/727009725 >> 172.21.15.86:0/727009725 conn(0x7fa45c0793a0 msgr2=0x7fa45c0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:42.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.908+0000 7fa46174b640 1 -- 172.21.15.86:0/727009725 shutdown_connections 2024-04-21T08:49:42.908 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.908+0000 7fa46174b640 1 -- 172.21.15.86:0/727009725 wait complete. 2024-04-21T08:49:42.909 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.909+0000 7fa46174b640 1 Processor -- start 2024-04-21T08:49:42.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.909+0000 7fa46174b640 1 -- start start 2024-04-21T08:49:42.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.909+0000 7fa46174b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa45c102190 0x7fa45c198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:42.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.909+0000 7fa46174b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa45c107260 0x7fa45c198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:42.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.909+0000 7fa46174b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa45c19cfc0 0x7fa45c19d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:42.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.909+0000 7fa46174b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa45c112180 con 0x7fa45c19cfc0 2024-04-21T08:49:42.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.909+0000 7fa46174b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa45c112000 con 0x7fa45c102190 2024-04-21T08:49:42.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.909+0000 7fa46174b640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa45c112300 con 0x7fa45c107260 2024-04-21T08:49:42.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.909+0000 7fa45b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa45c19cfc0 0x7fa45c19d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:42.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.909+0000 7fa45b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa45c19cfc0 0x7fa45c19d470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36220/0 (socket says 172.21.15.86:36220) 2024-04-21T08:49:42.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.909+0000 7fa45b7fe640 1 -- 172.21.15.86:0/1448860653 learned_addr learned my addr 172.21.15.86:0/1448860653 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:42.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.909+0000 7fa45a7fc640 1 --2- 172.21.15.86:0/1448860653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa45c107260 0x7fa45c198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:42.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.909+0000 7fa45affd640 1 --2- 172.21.15.86:0/1448860653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa45c102190 0x7fa45c198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:42.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.909+0000 7fa45b7fe640 1 -- 172.21.15.86:0/1448860653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa45c107260 msgr2=0x7fa45c198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:42.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.909+0000 7fa45b7fe640 1 --2- 172.21.15.86:0/1448860653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa45c107260 0x7fa45c198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:42.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.909+0000 7fa45b7fe640 1 -- 172.21.15.86:0/1448860653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa45c102190 msgr2=0x7fa45c198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:42.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.909+0000 7fa45b7fe640 1 --2- 172.21.15.86:0/1448860653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa45c102190 0x7fa45c198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:42.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.909+0000 7fa45b7fe640 1 -- 172.21.15.86:0/1448860653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa45c19dab0 con 0x7fa45c19cfc0 2024-04-21T08:49:42.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.909+0000 7fa45a7fc640 1 --2- 172.21.15.86:0/1448860653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa45c107260 0x7fa45c198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:42.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.909+0000 7fa45affd640 1 --2- 172.21.15.86:0/1448860653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa45c102190 0x7fa45c198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:42.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.910+0000 7fa45b7fe640 1 --2- 172.21.15.86:0/1448860653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa45c19cfc0 0x7fa45c19d470 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fa45400cce0 tx=0x7fa45400a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:42.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.910+0000 7fa443fff640 1 -- 172.21.15.86:0/1448860653 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45400f450 con 0x7fa45c19cfc0 2024-04-21T08:49:42.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.910+0000 7fa46174b640 1 -- 172.21.15.86:0/1448860653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa45c19dda0 con 0x7fa45c19cfc0 2024-04-21T08:49:42.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.910+0000 7fa443fff640 1 -- 172.21.15.86:0/1448860653 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa45400faa0 con 0x7fa45c19cfc0 2024-04-21T08:49:42.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.910+0000 7fa443fff640 1 -- 172.21.15.86:0/1448860653 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa454018b70 con 0x7fa45c19cfc0 2024-04-21T08:49:42.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.910+0000 7fa46174b640 1 -- 172.21.15.86:0/1448860653 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa45c1a5510 con 0x7fa45c19cfc0 2024-04-21T08:49:42.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.912+0000 7fa443fff640 1 -- 172.21.15.86:0/1448860653 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa454018d10 con 0x7fa45c19cfc0 2024-04-21T08:49:42.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.912+0000 7fa464412740 1 Processor -- start 2024-04-21T08:49:42.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.912+0000 7fa464412740 1 -- start start 2024-04-21T08:49:42.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.912+0000 7fa464412740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561768e518e0 0x561768e51cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:42.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.912+0000 7fa464412740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561768e52290 0x561768e5a840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:42.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.912+0000 7fa443fff640 1 --2- 172.21.15.86:0/1448860653 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa4440a3d20 0x7fa4440a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:42.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.912+0000 7fa464412740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561768e5ae10 0x561768e5d260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:42.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.912+0000 7fa464412740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561768db5f00 con 0x561768e5ae10 2024-04-21T08:49:42.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.912+0000 7fa464412740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x561768d53590 con 0x561768e52290 2024-04-21T08:49:42.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.912+0000 7fa464412740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561768db1ec0 con 0x561768e518e0 2024-04-21T08:49:42.920 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.913+0000 7fa45a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561768e5ae10 0x561768e5d260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:42.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.913+0000 7fa45b7fe640 1 --2- 172.21.15.86:0/1448860653 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa4440a3d20 0x7fa4440a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:42.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.913+0000 7fa45a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561768e5ae10 0x561768e5d260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36226/0 (socket says 172.21.15.86:36226) 2024-04-21T08:49:42.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.913+0000 7fa45a7fc640 1 -- 172.21.15.86:0/1748698796 learned_addr learned my addr 172.21.15.86:0/1748698796 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:42.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.913+0000 7fa45affd640 1 --2- 172.21.15.86:0/1748698796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561768e52290 0x561768e5a840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:42.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.913+0000 7fa45a7fc640 1 --2- 172.21.15.86:0/1748698796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561768e518e0 0x561768e51cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:42.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.913+0000 7fa443fff640 1 -- 172.21.15.86:0/1448860653 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7fa4540d1060 con 0x7fa45c19cfc0 2024-04-21T08:49:42.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.913+0000 7fa45a7fc640 1 -- 172.21.15.86:0/1748698796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561768e518e0 msgr2=0x561768e51cc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:42.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.913+0000 7fa45a7fc640 1 --2- 172.21.15.86:0/1748698796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561768e518e0 0x561768e51cc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:42.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.913+0000 7fa45a7fc640 1 -- 172.21.15.86:0/1748698796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561768e52290 msgr2=0x561768e5a840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:42.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.913+0000 7fa45a7fc640 1 --2- 172.21.15.86:0/1748698796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561768e52290 0x561768e5a840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:42.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.913+0000 7fa45a7fc640 1 -- 172.21.15.86:0/1748698796 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561768b907e0 con 0x561768e5ae10 2024-04-21T08:49:42.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.913+0000 7fa45affd640 1 --2- 172.21.15.86:0/1748698796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561768e52290 0x561768e5a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:42.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.913+0000 7fa45b7fe640 1 --2- 172.21.15.86:0/1448860653 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa4440a3d20 0x7fa4440a61e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa45409c930 tx=0x7fa45400af30 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:42.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.913+0000 7fa45a7fc640 1 --2- 172.21.15.86:0/1748698796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561768e5ae10 0x561768e5d260 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fa45c06b820 tx=0x7fa450009560 comp rx=0 tx=0).ready entity=mon.0 client_cookie=80d9905c93c55ff3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:42.921 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.913+0000 7fa4417fa640 1 -- 172.21.15.86:0/1748698796 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa45002c9e0 con 0x561768e5ae10 2024-04-21T08:49:42.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.913+0000 7fa4417fa640 1 -- 172.21.15.86:0/1748698796 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa450048070 con 0x561768e5ae10 2024-04-21T08:49:42.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.913+0000 7fa4417fa640 1 -- 172.21.15.86:0/1748698796 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa450004ae0 con 0x561768e5ae10 2024-04-21T08:49:42.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.914+0000 7fa464412740 1 -- 172.21.15.86:0/1748698796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561768e5ae10 msgr2=0x561768e5d260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:42.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.914+0000 7fa464412740 1 --2- 172.21.15.86:0/1748698796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561768e5ae10 0x561768e5d260 secure :-1 s=READY pgs=1936 cs=0 l=1 rev1=1 crypto rx=0x7fa45c06b820 tx=0x7fa450009560 comp rx=0 tx=0).stop 2024-04-21T08:49:42.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.914+0000 7fa464412740 1 -- 172.21.15.86:0/1748698796 shutdown_connections 2024-04-21T08:49:42.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.914+0000 7fa464412740 1 --2- 172.21.15.86:0/1748698796 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561768e5ae10 0x561768e5d260 unknown :-1 s=CLOSED pgs=1936 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:42.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.914+0000 7fa464412740 1 --2- 172.21.15.86:0/1748698796 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561768e52290 0x561768e5a840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:42.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.914+0000 7fa464412740 1 --2- 172.21.15.86:0/1748698796 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561768e518e0 0x561768e51cc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:42.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.914+0000 7fa464412740 1 -- 172.21.15.86:0/1748698796 >> 172.21.15.86:0/1748698796 conn(0x561768e4bfb0 msgr2=0x561768e4c390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:42.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.914+0000 7fa464412740 1 -- 172.21.15.86:0/1748698796 shutdown_connections 2024-04-21T08:49:42.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.914+0000 7fa464412740 1 -- 172.21.15.86:0/1748698796 wait complete. 2024-04-21T08:49:42.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.914+0000 7fa464412740 1 -- start start 2024-04-21T08:49:42.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.914+0000 7fa464412740 1 Processor -- start 2024-04-21T08:49:42.922 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.915+0000 7fa464412740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561768e518e0 0x561768e57de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:42.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.915+0000 7fa464412740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561768e52290 0x561768e58320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:42.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.915+0000 7fa464412740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561768e5ae10 0x561768e58860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:42.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.915+0000 7fa464412740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x561768e5dd20 con 0x561768e52290 2024-04-21T08:49:42.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.915+0000 7fa464412740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x561768d6f580 con 0x561768e5ae10 2024-04-21T08:49:42.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.915+0000 7fa464412740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x561768e5dea0 con 0x561768e518e0 2024-04-21T08:49:42.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.915+0000 7fa464412740 10 client.0 _ll_register_callbacks cb 0x561768d75650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:42.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.915+0000 7fa45affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561768e52290 0x561768e58320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:42.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.915+0000 7fa45affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561768e52290 0x561768e58320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36232/0 (socket says 172.21.15.86:36232) 2024-04-21T08:49:42.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.915+0000 7fa45affd640 1 -- 172.21.15.86:0/1422570563 learned_addr learned my addr 172.21.15.86:0/1422570563 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:42.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.915+0000 7fa45a7fc640 1 --2- 172.21.15.86:0/1422570563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561768e5ae10 0x561768e58860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:42.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.915+0000 7fa45affd640 1 -- 172.21.15.86:0/1422570563 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561768e518e0 msgr2=0x561768e57de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:42.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.915+0000 7fa45affd640 1 --2- 172.21.15.86:0/1422570563 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561768e518e0 0x561768e57de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:42.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.915+0000 7fa45affd640 1 -- 172.21.15.86:0/1422570563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561768e5ae10 msgr2=0x561768e58860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:42.923 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.915+0000 7fa45affd640 1 --2- 172.21.15.86:0/1422570563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561768e5ae10 0x561768e58860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:42.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.915+0000 7fa45affd640 1 -- 172.21.15.86:0/1422570563 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561768bf0f70 con 0x561768e52290 2024-04-21T08:49:42.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.915+0000 7fa45affd640 1 --2- 172.21.15.86:0/1422570563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561768e52290 0x561768e58320 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fa44c00bf00 tx=0x7fa44c0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=70c25d388de497bd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:42.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.915+0000 7fa43eff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:42.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.915+0000 7fa43eff5640 1 -- 172.21.15.86:0/1422570563 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44c004220 con 0x561768e52290 2024-04-21T08:49:42.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.915+0000 7fa43eff5640 1 -- 172.21.15.86:0/1422570563 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7fa44c0043c0 con 0x561768e52290 2024-04-21T08:49:42.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.915+0000 7fa464412740 1 -- 172.21.15.86:0/1422570563 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa450048070 con 0x561768e52290 2024-04-21T08:49:42.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.916+0000 7fa43eff5640 1 -- 172.21.15.86:0/1422570563 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa44c0035f0 con 0x561768e52290 2024-04-21T08:49:42.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.916+0000 7fa464412740 10 client.20040 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:42.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.916+0000 7fa464412740 1 -- 172.21.15.86:0/1422570563 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561768a88170 con 0x561768e52290 2024-04-21T08:49:42.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.916+0000 7fa43eff5640 1 -- 172.21.15.86:0/1422570563 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa450048070 con 0x561768e52290 2024-04-21T08:49:42.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.916+0000 7fa43eff5640 1 -- 172.21.15.86:0/1422570563 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa44c01b9c0 con 0x561768e52290 2024-04-21T08:49:42.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.916+0000 7fa464412740 10 client.20040 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:42.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.916+0000 7fa464412740 10 client.20040 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:42.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.916+0000 7fa464412740 20 client.20040 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:42.924 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.916+0000 7fa464412740 1 --2- 172.21.15.86:0/1422570563 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x561768e6b3e0 0x561768e6d840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:42.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.916+0000 7fa464412740 4 client.20040 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:42.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.916+0000 7fa464412740 1 -- 172.21.15.86:0/1422570563 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x561768dc3920 con 0x561768e6b3e0 2024-04-21T08:49:42.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.916+0000 7fa45a7fc640 1 --2- 172.21.15.86:0/1422570563 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x561768e6b3e0 0x561768e6d840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:42.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.917+0000 7fa43eff5640 10 client.20040 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:42.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.917+0000 7fa45a7fc640 1 --2- 172.21.15.86:0/1422570563 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x561768e6b3e0 0x561768e6d840 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=7b67f2b05d3e3f59 server_cookie=2a1e1b4014a64f00 in_seq=0 out_seq=0 2024-04-21T08:49:42.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.917+0000 7fa43eff5640 1 -- 172.21.15.86:0/1422570563 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x561768dc3920 con 0x561768e6b3e0 2024-04-21T08:49:42.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.917+0000 7fa43eff5640 10 client.20040 handle_command_reply: tid=0 2024-04-21T08:49:42.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.917+0000 7fa464412740 1 client.20040 shutdown 2024-04-21T08:49:42.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.917+0000 7fa464412740 1 -- 172.21.15.86:0/1422570563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561768e52290 msgr2=0x561768e58320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:42.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.917+0000 7fa464412740 1 --2- 172.21.15.86:0/1422570563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561768e52290 0x561768e58320 secure :-1 s=READY pgs=1937 cs=0 l=1 rev1=1 crypto rx=0x7fa44c00bf00 tx=0x7fa44c0117b0 comp rx=0 tx=0).stop 2024-04-21T08:49:42.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.917+0000 7fa464412740 1 -- 172.21.15.86:0/1422570563 shutdown_connections 2024-04-21T08:49:42.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.917+0000 7fa464412740 1 --2- 172.21.15.86:0/1422570563 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x561768e6b3e0 0x561768e6d840 crc :-1 s=READY pgs=317 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:42.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.917+0000 7fa464412740 1 --2- 172.21.15.86:0/1422570563 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561768e5ae10 0x561768e58860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:42.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.917+0000 7fa464412740 1 --2- 172.21.15.86:0/1422570563 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561768e52290 0x561768e58320 unknown :-1 s=CLOSED pgs=1937 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:42.925 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.917+0000 7fa43eff5640 0 client.20040 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:42.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.917+0000 7fa464412740 1 --2- 172.21.15.86:0/1422570563 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561768e518e0 0x561768e57de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:42.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.917+0000 7fa464412740 1 -- 172.21.15.86:0/1422570563 >> 172.21.15.86:0/1422570563 conn(0x561768e4bfb0 msgr2=0x561768e4dd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:42.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.918+0000 7fa464412740 1 -- 172.21.15.86:0/1422570563 shutdown_connections 2024-04-21T08:49:42.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.918+0000 7fa464412740 1 -- 172.21.15.86:0/1422570563 wait complete. 2024-04-21T08:49:42.926 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.918+0000 7fa464412740 20 client.20040 trim_cache size 0 max 16384 2024-04-21T08:49:42.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.936+0000 7fa464412740 1 Processor -- start 2024-04-21T08:49:42.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.936+0000 7fa464412740 1 -- start start 2024-04-21T08:49:42.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.936+0000 7fa464412740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561768e518e0 0x561768e5a510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:42.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.936+0000 7fa464412740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561768e52290 0x561768e67d60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:42.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.936+0000 7fa464412740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561768e5ae10 0x561768e66be0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:42.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.936+0000 7fa464412740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa45c1af210 con 0x561768e5ae10 2024-04-21T08:49:42.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.936+0000 7fa464412740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa45c1b0d50 con 0x561768e518e0 2024-04-21T08:49:42.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.936+0000 7fa464412740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa45c1af8e0 con 0x561768e52290 2024-04-21T08:49:42.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.936+0000 7fa45a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561768e5ae10 0x561768e66be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:42.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.936+0000 7fa45a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561768e5ae10 0x561768e66be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36240/0 (socket says 172.21.15.86:36240) 2024-04-21T08:49:42.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.936+0000 7fa45a7fc640 1 -- 172.21.15.86:0/3600859378 learned_addr learned my addr 172.21.15.86:0/3600859378 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:42.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.936+0000 7fa45affd640 1 --2- 172.21.15.86:0/3600859378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561768e52290 0x561768e67d60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:42.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.936+0000 7fa45a7fc640 1 --2- 172.21.15.86:0/3600859378 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561768e518e0 0x561768e5a510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:42.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.936+0000 7fa45a7fc640 1 -- 172.21.15.86:0/3600859378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561768e52290 msgr2=0x561768e67d60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:42.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.936+0000 7fa45a7fc640 1 --2- 172.21.15.86:0/3600859378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561768e52290 0x561768e67d60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:42.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.936+0000 7fa45a7fc640 1 -- 172.21.15.86:0/3600859378 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561768e518e0 msgr2=0x561768e5a510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:42.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.936+0000 7fa45a7fc640 1 --2- 172.21.15.86:0/3600859378 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561768e518e0 0x561768e5a510 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:42.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.936+0000 7fa45a7fc640 1 -- 172.21.15.86:0/3600859378 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x561768e5d6c0 con 0x561768e5ae10 2024-04-21T08:49:42.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.937+0000 7fa45a7fc640 1 --2- 172.21.15.86:0/3600859378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561768e5ae10 0x561768e66be0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7fa45c06b820 tx=0x7fa45002bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b32e9714a2c9acd server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:42.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.937+0000 7fa440ff9640 1 -- 172.21.15.86:0/3600859378 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa450009780 con 0x561768e5ae10 2024-04-21T08:49:42.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.937+0000 7fa440ff9640 1 -- 172.21.15.86:0/3600859378 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa450048030 con 0x561768e5ae10 2024-04-21T08:49:42.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.937+0000 7fa440ff9640 1 -- 172.21.15.86:0/3600859378 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa450043480 con 0x561768e5ae10 2024-04-21T08:49:42.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.938+0000 7fa464412740 1 -- 172.21.15.86:0/3600859378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561768e5ae10 msgr2=0x561768e66be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:42.942 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.938+0000 7fa464412740 1 --2- 172.21.15.86:0/3600859378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561768e5ae10 0x561768e66be0 secure :-1 s=READY pgs=1938 cs=0 l=1 rev1=1 crypto rx=0x7fa45c06b820 tx=0x7fa45002bc30 comp rx=0 tx=0).stop 2024-04-21T08:49:42.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.938+0000 7fa464412740 1 -- 172.21.15.86:0/3600859378 shutdown_connections 2024-04-21T08:49:42.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.938+0000 7fa464412740 1 --2- 172.21.15.86:0/3600859378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561768e5ae10 0x561768e66be0 unknown :-1 s=CLOSED pgs=1938 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:42.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.938+0000 7fa464412740 1 --2- 172.21.15.86:0/3600859378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561768e52290 0x561768e67d60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:42.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.938+0000 7fa464412740 1 --2- 172.21.15.86:0/3600859378 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561768e518e0 0x561768e5a510 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:42.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.938+0000 7fa464412740 1 -- 172.21.15.86:0/3600859378 >> 172.21.15.86:0/3600859378 conn(0x561768e4bfb0 msgr2=0x561768e4d3c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:42.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.938+0000 7fa464412740 1 -- 172.21.15.86:0/3600859378 shutdown_connections 2024-04-21T08:49:42.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.938+0000 7fa464412740 1 -- 172.21.15.86:0/3600859378 wait complete. 2024-04-21T08:49:42.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.939+0000 7fa464412740 1 -- start start 2024-04-21T08:49:42.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.939+0000 7fa464412740 1 Processor -- start 2024-04-21T08:49:42.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.939+0000 7fa464412740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561768e518e0 0x561768e5ee70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:42.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.939+0000 7fa464412740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561768e52290 0x561768e5f3b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:42.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.939+0000 7fa464412740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561768e5ae10 0x561768ed09c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:42.943 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.939+0000 7fa464412740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa45c1b0680 con 0x561768e52290 2024-04-21T08:49:42.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.939+0000 7fa464412740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa45c1affb0 con 0x561768e518e0 2024-04-21T08:49:42.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.939+0000 7fa464412740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa45c1ad210 con 0x561768e5ae10 2024-04-21T08:49:42.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.939+0000 7fa464412740 10 client.0 _ll_register_callbacks cb 0x7fa44c00b480 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:42.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.939+0000 7fa45affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561768e52290 0x561768e5f3b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:42.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.939+0000 7fa45affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561768e52290 0x561768e5f3b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36252/0 (socket says 172.21.15.86:36252) 2024-04-21T08:49:42.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.939+0000 7fa45affd640 1 -- 172.21.15.86:0/865461230 learned_addr learned my addr 172.21.15.86:0/865461230 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:42.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.939+0000 7fa45a7fc640 1 --2- 172.21.15.86:0/865461230 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561768e518e0 0x561768e5ee70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:42.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.939+0000 7fa45a7fc640 1 --2- 172.21.15.86:0/865461230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561768e5ae10 0x561768ed09c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:42.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.939+0000 7fa45affd640 1 -- 172.21.15.86:0/865461230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561768e5ae10 msgr2=0x561768ed09c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:42.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.939+0000 7fa45affd640 1 --2- 172.21.15.86:0/865461230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561768e5ae10 0x561768ed09c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:42.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.939+0000 7fa45affd640 1 -- 172.21.15.86:0/865461230 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561768e518e0 msgr2=0x561768e5ee70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:42.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.939+0000 7fa45affd640 1 --2- 172.21.15.86:0/865461230 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561768e518e0 0x561768e5ee70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:42.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.939+0000 7fa45affd640 1 -- 172.21.15.86:0/865461230 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x561768e5fc80 con 0x561768e52290 2024-04-21T08:49:42.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.939+0000 7fa45a7fc640 1 --2- 172.21.15.86:0/865461230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561768e5ae10 0x561768ed09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:42.944 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.939+0000 7fa45affd640 1 --2- 172.21.15.86:0/865461230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561768e52290 0x561768e5f3b0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x561768e57640 tx=0x7fa44c0052f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1412b28cdec61a00 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:42.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.939+0000 7fa43eff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:42.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.939+0000 7fa43eff5640 1 -- 172.21.15.86:0/865461230 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa44c01b680 con 0x561768e52290 2024-04-21T08:49:42.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.939+0000 7fa43eff5640 1 -- 172.21.15.86:0/865461230 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7fa44c01b820 con 0x561768e52290 2024-04-21T08:49:42.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.939+0000 7fa464412740 1 -- 172.21.15.86:0/865461230 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa450048030 con 0x561768e52290 2024-04-21T08:49:42.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.940+0000 7fa43eff5640 1 -- 172.21.15.86:0/865461230 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa44c01ba20 con 0x561768e52290 2024-04-21T08:49:42.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.940+0000 7fa464412740 10 client.20046 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:42.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.940+0000 7fa464412740 1 -- 172.21.15.86:0/865461230 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x561768ed2070 con 0x561768e52290 2024-04-21T08:49:42.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.940+0000 7fa43eff5640 1 -- 172.21.15.86:0/865461230 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa450048030 con 0x561768e52290 2024-04-21T08:49:42.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.940+0000 7fa43eff5640 1 -- 172.21.15.86:0/865461230 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa44c0124e0 con 0x561768e52290 2024-04-21T08:49:42.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.940+0000 7fa464412740 10 client.20046 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:42.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.940+0000 7fa464412740 10 client.20046 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:42.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.940+0000 7fa464412740 20 client.20046 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:42.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.940+0000 7fa464412740 1 --2- 172.21.15.86:0/865461230 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x561768ed2820 0x561768ed4c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:42.945 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.940+0000 7fa464412740 4 client.20046 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:49:42.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.940+0000 7fa464412740 1 -- 172.21.15.86:0/865461230 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x561768dbd790 con 0x561768ed2820 2024-04-21T08:49:42.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.940+0000 7fa45a7fc640 1 --2- 172.21.15.86:0/865461230 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x561768ed2820 0x561768ed4c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:42.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.941+0000 7fa43eff5640 10 client.20046 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:42.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.941+0000 7fa45a7fc640 1 --2- 172.21.15.86:0/865461230 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x561768ed2820 0x561768ed4c40 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6b98e43056f87e22 server_cookie=691eca71a4804d61 in_seq=0 out_seq=0 2024-04-21T08:49:42.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.942+0000 7fa43eff5640 1 -- 172.21.15.86:0/865461230 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x561768dbd790 con 0x561768ed2820 2024-04-21T08:49:42.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.942+0000 7fa43eff5640 10 client.20046 handle_command_reply: tid=0 2024-04-21T08:49:42.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.942+0000 7fa464412740 1 client.20046 shutdown 2024-04-21T08:49:42.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.942+0000 7fa464412740 1 -- 172.21.15.86:0/865461230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561768e52290 msgr2=0x561768e5f3b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:42.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.942+0000 7fa464412740 1 --2- 172.21.15.86:0/865461230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561768e52290 0x561768e5f3b0 secure :-1 s=READY pgs=1939 cs=0 l=1 rev1=1 crypto rx=0x561768e57640 tx=0x7fa44c0052f0 comp rx=0 tx=0).stop 2024-04-21T08:49:42.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.942+0000 7fa464412740 1 -- 172.21.15.86:0/865461230 shutdown_connections 2024-04-21T08:49:42.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.942+0000 7fa464412740 1 --2- 172.21.15.86:0/865461230 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x561768ed2820 0x561768ed4c40 crc :-1 s=READY pgs=318 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:42.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.942+0000 7fa464412740 1 --2- 172.21.15.86:0/865461230 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x561768e5ae10 0x561768ed09c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:42.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.942+0000 7fa464412740 1 --2- 172.21.15.86:0/865461230 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x561768e52290 0x561768e5f3b0 unknown :-1 s=CLOSED pgs=1939 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:42.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.942+0000 7fa464412740 1 --2- 172.21.15.86:0/865461230 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x561768e518e0 0x561768e5ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:42.946 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.942+0000 7fa43eff5640 0 client.20046 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:42.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.942+0000 7fa464412740 1 -- 172.21.15.86:0/865461230 >> 172.21.15.86:0/865461230 conn(0x561768e4bfb0 msgr2=0x561768e585c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:42.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.942+0000 7fa464412740 1 -- 172.21.15.86:0/865461230 shutdown_connections 2024-04-21T08:49:42.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.942+0000 7fa464412740 1 -- 172.21.15.86:0/865461230 wait complete. 2024-04-21T08:49:42.947 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.942+0000 7fa464412740 20 client.20046 trim_cache size 0 max 16384 2024-04-21T08:49:42.947 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:49:42.947 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-21T08:49:42.947 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "23cbafa6-5e18-4768-ad86-5bba0143477b", 2024-04-21T08:49:42.947 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-21T08:49:42.947 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:49:42.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.942+0000 7fa46174b640 1 -- 172.21.15.86:0/1448860653 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa4440a3d20 msgr2=0x7fa4440a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:42.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.942+0000 7fa46174b640 1 --2- 172.21.15.86:0/1448860653 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa4440a3d20 0x7fa4440a61e0 secure :-1 s=READY pgs=322 cs=0 l=1 rev1=1 crypto rx=0x7fa45409c930 tx=0x7fa45400af30 comp rx=0 tx=0).stop 2024-04-21T08:49:42.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.942+0000 7fa46174b640 1 -- 172.21.15.86:0/1448860653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa45c19cfc0 msgr2=0x7fa45c19d470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:42.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.942+0000 7fa46174b640 1 --2- 172.21.15.86:0/1448860653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa45c19cfc0 0x7fa45c19d470 secure :-1 s=READY pgs=1935 cs=0 l=1 rev1=1 crypto rx=0x7fa45400cce0 tx=0x7fa45400a620 comp rx=0 tx=0).stop 2024-04-21T08:49:42.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.943+0000 7fa46174b640 1 -- 172.21.15.86:0/1448860653 shutdown_connections 2024-04-21T08:49:42.948 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.943+0000 7fa46174b640 1 --2- 172.21.15.86:0/1448860653 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa4440a3d20 0x7fa4440a61e0 unknown :-1 s=CLOSED pgs=322 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:42.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.943+0000 7fa46174b640 1 --2- 172.21.15.86:0/1448860653 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa45c19cfc0 0x7fa45c19d470 unknown :-1 s=CLOSED pgs=1935 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:42.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.943+0000 7fa46174b640 1 --2- 172.21.15.86:0/1448860653 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa45c107260 0x7fa45c198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:42.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.943+0000 7fa46174b640 1 --2- 172.21.15.86:0/1448860653 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa45c102190 0x7fa45c198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:42.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.943+0000 7fa46174b640 1 -- 172.21.15.86:0/1448860653 >> 172.21.15.86:0/1448860653 conn(0x7fa45c0793a0 msgr2=0x7fa45c0fff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:42.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.943+0000 7fa46174b640 1 -- 172.21.15.86:0/1448860653 shutdown_connections 2024-04-21T08:49:42.949 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:42.943+0000 7fa46174b640 1 -- 172.21.15.86:0/1448860653 wait complete. 2024-04-21T08:49:42.953 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:49:42.953 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:49:43.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.054+0000 7fb5abf52640 1 -- 172.21.15.86:0/3992345431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5a4106160 msgr2=0x7fb5a410dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:43.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.054+0000 7fb5abf52640 1 --2- 172.21.15.86:0/3992345431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5a4106160 0x7fb5a410dd30 secure :-1 s=READY pgs=1940 cs=0 l=1 rev1=1 crypto rx=0x7fb59c009ca0 tx=0x7fb59c02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:49:43.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.054+0000 7fb5abf52640 1 -- 172.21.15.86:0/3992345431 shutdown_connections 2024-04-21T08:49:43.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.054+0000 7fb5abf52640 1 --2- 172.21.15.86:0/3992345431 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5a4106160 0x7fb5a410dd30 unknown :-1 s=CLOSED pgs=1940 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:43.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.054+0000 7fb5abf52640 1 --2- 172.21.15.86:0/3992345431 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5a4101a40 0x7fb5a4105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:43.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.054+0000 7fb5abf52640 1 --2- 172.21.15.86:0/3992345431 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5a4101090 0x7fb5a4101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:43.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.054+0000 7fb5abf52640 1 -- 172.21.15.86:0/3992345431 >> 172.21.15.86:0/3992345431 conn(0x7fb5a40771d0 msgr2=0x7fb5a40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:43.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.054+0000 7fb5abf52640 1 -- 172.21.15.86:0/3992345431 shutdown_connections 2024-04-21T08:49:43.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.054+0000 7fb5abf52640 1 -- 172.21.15.86:0/3992345431 wait complete. 2024-04-21T08:49:43.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.055+0000 7fb5abf52640 1 Processor -- start 2024-04-21T08:49:43.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.055+0000 7fb5abf52640 1 -- start start 2024-04-21T08:49:43.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.055+0000 7fb5abf52640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5a4101090 0x7fb5a4198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:43.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.055+0000 7fb5abf52640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5a4106160 0x7fb5a4198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:43.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.055+0000 7fb5abf52640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5a419cfc0 0x7fb5a419d470 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:43.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.055+0000 7fb5abf52640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a41110d0 con 0x7fb5a4106160 2024-04-21T08:49:43.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.055+0000 7fb5abf52640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a4110f50 con 0x7fb5a419cfc0 2024-04-21T08:49:43.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.055+0000 7fb5abf52640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a4111250 con 0x7fb5a4101090 2024-04-21T08:49:43.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.055+0000 7fb5a94ed640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5a4106160 0x7fb5a4198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:43.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.055+0000 7fb5a94ed640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5a4106160 0x7fb5a4198c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36266/0 (socket says 172.21.15.86:36266) 2024-04-21T08:49:43.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.055+0000 7fb5a94ed640 1 -- 172.21.15.86:0/2943220750 learned_addr learned my addr 172.21.15.86:0/2943220750 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:43.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.055+0000 7fb5aa4ef640 1 --2- 172.21.15.86:0/2943220750 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5a419cfc0 0x7fb5a419d470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:43.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.055+0000 7fb5a9cee640 1 --2- 172.21.15.86:0/2943220750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5a4101090 0x7fb5a4198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:43.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.055+0000 7fb5a94ed640 1 -- 172.21.15.86:0/2943220750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5a4101090 msgr2=0x7fb5a4198700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:43.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.055+0000 7fb5a94ed640 1 --2- 172.21.15.86:0/2943220750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5a4101090 0x7fb5a4198700 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:43.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.055+0000 7fb5a94ed640 1 -- 172.21.15.86:0/2943220750 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5a419cfc0 msgr2=0x7fb5a419d470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:43.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.055+0000 7fb5a94ed640 1 --2- 172.21.15.86:0/2943220750 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5a419cfc0 0x7fb5a419d470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:43.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.055+0000 7fb5a94ed640 1 -- 172.21.15.86:0/2943220750 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fb5a419dab0 con 0x7fb5a4106160 2024-04-21T08:49:43.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.055+0000 7fb5a94ed640 1 --2- 172.21.15.86:0/2943220750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5a4106160 0x7fb5a4198c40 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fb59400e9f0 tx=0x7fb59400eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:43.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.055+0000 7fb59affd640 1 -- 172.21.15.86:0/2943220750 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59400ccc0 con 0x7fb5a4106160 2024-04-21T08:49:43.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.056+0000 7fb59affd640 1 -- 172.21.15.86:0/2943220750 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5940054d0 con 0x7fb5a4106160 2024-04-21T08:49:43.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.056+0000 7fb5abf52640 1 -- 172.21.15.86:0/2943220750 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fb5a419dda0 con 0x7fb5a4106160 2024-04-21T08:49:43.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.056+0000 7fb59affd640 1 -- 172.21.15.86:0/2943220750 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb594010430 con 0x7fb5a4106160 2024-04-21T08:49:43.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.056+0000 7fb5abf52640 1 -- 172.21.15.86:0/2943220750 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fb5a41a5540 con 0x7fb5a4106160 2024-04-21T08:49:43.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.058+0000 7fb59affd640 1 -- 172.21.15.86:0/2943220750 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fb5940046c0 con 0x7fb5a4106160 2024-04-21T08:49:43.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.058+0000 7fb5aec18740 1 Processor -- start 2024-04-21T08:49:43.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.058+0000 7fb5aec18740 1 -- start start 2024-04-21T08:49:43.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.058+0000 7fb5aec18740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a65b6ce580 0x55a65b6ce960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:43.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.058+0000 7fb5aec18740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65b6ceea0 0x55a65b6d7540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:43.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.059+0000 7fb5aec18740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a65b6d7a80 0x55a65b6d9e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:43.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.059+0000 7fb5aec18740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a65b6331e0 con 0x55a65b6ce580 2024-04-21T08:49:43.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.059+0000 7fb5aec18740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55a65b5d0390 con 0x55a65b6d7a80 2024-04-21T08:49:43.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.059+0000 7fb5aec18740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a65b62f1a0 con 0x55a65b6ceea0 2024-04-21T08:49:43.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.059+0000 7fb5aa4ef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a65b6ce580 0x55a65b6ce960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:43.065 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.059+0000 7fb5aa4ef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a65b6ce580 0x55a65b6ce960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36282/0 (socket says 172.21.15.86:36282) 2024-04-21T08:49:43.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.059+0000 7fb5aa4ef640 1 -- 172.21.15.86:0/3622572999 learned_addr learned my addr 172.21.15.86:0/3622572999 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:43.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.059+0000 7fb5a94ed640 1 --2- 172.21.15.86:0/3622572999 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a65b6d7a80 0x55a65b6d9e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:43.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.059+0000 7fb59affd640 1 --2- 172.21.15.86:0/2943220750 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb5900a3d20 0x7fb5900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:43.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.059+0000 7fb5a9cee640 1 --2- 172.21.15.86:0/3622572999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65b6ceea0 0x55a65b6d7540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:43.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.059+0000 7fb5aa4ef640 1 -- 172.21.15.86:0/3622572999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65b6ceea0 msgr2=0x55a65b6d7540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:43.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.059+0000 7fb5aa4ef640 1 --2- 172.21.15.86:0/3622572999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65b6ceea0 0x55a65b6d7540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:43.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.059+0000 7fb5aa4ef640 1 -- 172.21.15.86:0/3622572999 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a65b6d7a80 msgr2=0x55a65b6d9e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:43.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.059+0000 7fb5aa4ef640 1 --2- 172.21.15.86:0/3622572999 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a65b6d7a80 0x55a65b6d9e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:43.066 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.059+0000 7fb5aa4ef640 1 -- 172.21.15.86:0/3622572999 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a65b40f7d0 con 0x55a65b6ce580 2024-04-21T08:49:43.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.059+0000 7fb5a94ed640 1 --2- 172.21.15.86:0/3622572999 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a65b6d7a80 0x55a65b6d9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:43.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.059+0000 7fb5aa4ef640 1 --2- 172.21.15.86:0/2943220750 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb5900a3d20 0x7fb5900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:43.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.059+0000 7fb59affd640 1 -- 172.21.15.86:0/2943220750 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7fb5940d0ff0 con 0x7fb5a4106160 2024-04-21T08:49:43.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.059+0000 7fb5aa4ef640 1 --2- 172.21.15.86:0/3622572999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a65b6ce580 0x55a65b6ce960 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fb59c037040 tx=0x7fb59c005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d1743b5805fa96c6 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:43.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.059+0000 7fb58ffff640 1 -- 172.21.15.86:0/3622572999 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c039040 con 0x55a65b6ce580 2024-04-21T08:49:43.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.059+0000 7fb58ffff640 1 -- 172.21.15.86:0/3622572999 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb59c003e20 con 0x55a65b6ce580 2024-04-21T08:49:43.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.059+0000 7fb58ffff640 1 -- 172.21.15.86:0/3622572999 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c034440 con 0x55a65b6ce580 2024-04-21T08:49:43.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.060+0000 7fb5aa4ef640 1 --2- 172.21.15.86:0/2943220750 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb5900a3d20 0x7fb5900a61e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb59c04c000 tx=0x7fb59c045000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:43.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.060+0000 7fb5aec18740 1 -- 172.21.15.86:0/3622572999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a65b6ce580 msgr2=0x55a65b6ce960 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:43.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.060+0000 7fb5aec18740 1 --2- 172.21.15.86:0/3622572999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a65b6ce580 0x55a65b6ce960 secure :-1 s=READY pgs=1942 cs=0 l=1 rev1=1 crypto rx=0x7fb59c037040 tx=0x7fb59c005f70 comp rx=0 tx=0).stop 2024-04-21T08:49:43.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.060+0000 7fb5aec18740 1 -- 172.21.15.86:0/3622572999 shutdown_connections 2024-04-21T08:49:43.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.060+0000 7fb5aec18740 1 --2- 172.21.15.86:0/3622572999 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a65b6d7a80 0x55a65b6d9e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:43.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.060+0000 7fb5aec18740 1 --2- 172.21.15.86:0/3622572999 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65b6ceea0 0x55a65b6d7540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:43.067 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.060+0000 7fb5aec18740 1 --2- 172.21.15.86:0/3622572999 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a65b6ce580 0x55a65b6ce960 unknown :-1 s=CLOSED pgs=1942 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:43.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.060+0000 7fb5aec18740 1 -- 172.21.15.86:0/3622572999 >> 172.21.15.86:0/3622572999 conn(0x55a65b6c8cb0 msgr2=0x55a65b6c9090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:43.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.060+0000 7fb5aec18740 1 -- 172.21.15.86:0/3622572999 shutdown_connections 2024-04-21T08:49:43.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.060+0000 7fb5aec18740 1 -- 172.21.15.86:0/3622572999 wait complete. 2024-04-21T08:49:43.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.061+0000 7fb5aec18740 1 -- start start 2024-04-21T08:49:43.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.061+0000 7fb5aec18740 1 Processor -- start 2024-04-21T08:49:43.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.061+0000 7fb5aec18740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a65b6ce580 0x55a65b6d4ae0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:43.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.061+0000 7fb5aec18740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a65b6ceea0 0x55a65b6d5020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:43.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.061+0000 7fb5aec18740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65b6d7a80 0x55a65b6d5560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:43.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.061+0000 7fb5aec18740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55a65b6da870 con 0x55a65b6ce580 2024-04-21T08:49:43.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.061+0000 7fb5aec18740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55a65b5ec2f0 con 0x55a65b6ceea0 2024-04-21T08:49:43.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.061+0000 7fb5aec18740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55a65b6da9f0 con 0x55a65b6d7a80 2024-04-21T08:49:43.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.061+0000 7fb5aec18740 10 client.0 _ll_register_callbacks cb 0x55a65b5f9580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:43.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.061+0000 7fb5a9cee640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a65b6ce580 0x55a65b6d4ae0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:43.068 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.061+0000 7fb5a9cee640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a65b6ce580 0x55a65b6d4ae0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36284/0 (socket says 172.21.15.86:36284) 2024-04-21T08:49:43.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.061+0000 7fb5a9cee640 1 -- 172.21.15.86:0/3855131566 learned_addr learned my addr 172.21.15.86:0/3855131566 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:43.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.061+0000 7fb5aa4ef640 1 --2- 172.21.15.86:0/3855131566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65b6d7a80 0x55a65b6d5560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:43.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.061+0000 7fb5a94ed640 1 --2- 172.21.15.86:0/3855131566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a65b6ceea0 0x55a65b6d5020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:43.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.061+0000 7fb5a9cee640 1 -- 172.21.15.86:0/3855131566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65b6d7a80 msgr2=0x55a65b6d5560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:43.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.061+0000 7fb5a9cee640 1 --2- 172.21.15.86:0/3855131566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65b6d7a80 0x55a65b6d5560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:43.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.061+0000 7fb5a9cee640 1 -- 172.21.15.86:0/3855131566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a65b6ceea0 msgr2=0x55a65b6d5020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:43.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.061+0000 7fb5a9cee640 1 --2- 172.21.15.86:0/3855131566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a65b6ceea0 0x55a65b6d5020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:43.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.061+0000 7fb5a9cee640 1 -- 172.21.15.86:0/3855131566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a65b3453f0 con 0x55a65b6ce580 2024-04-21T08:49:43.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.062+0000 7fb5a9cee640 1 --2- 172.21.15.86:0/3855131566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a65b6ce580 0x55a65b6d4ae0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fb5a000dea0 tx=0x7fb5a000df90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9c03dc1412be8701 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:43.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.062+0000 7fb58d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:43.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.062+0000 7fb58d7fa640 1 -- 172.21.15.86:0/3855131566 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a0004220 con 0x55a65b6ce580 2024-04-21T08:49:43.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.062+0000 7fb58d7fa640 1 -- 172.21.15.86:0/3855131566 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7fb5a00043c0 con 0x55a65b6ce580 2024-04-21T08:49:43.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.062+0000 7fb5aec18740 1 -- 172.21.15.86:0/3855131566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb59c003e20 con 0x55a65b6ce580 2024-04-21T08:49:43.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.062+0000 7fb58d7fa640 1 -- 172.21.15.86:0/3855131566 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb5a0023050 con 0x55a65b6ce580 2024-04-21T08:49:43.069 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.062+0000 7fb5aec18740 10 client.20058 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:43.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.062+0000 7fb5aec18740 1 -- 172.21.15.86:0/3855131566 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a65b3050f0 con 0x55a65b6ce580 2024-04-21T08:49:43.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.062+0000 7fb58d7fa640 1 -- 172.21.15.86:0/3855131566 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb59c003e20 con 0x55a65b6ce580 2024-04-21T08:49:43.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.062+0000 7fb58d7fa640 1 -- 172.21.15.86:0/3855131566 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb5a0004cd0 con 0x55a65b6ce580 2024-04-21T08:49:43.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.062+0000 7fb5aec18740 10 client.20058 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:43.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.062+0000 7fb5aec18740 10 client.20058 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:43.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.062+0000 7fb5aec18740 20 client.20058 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:43.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.062+0000 7fb5aec18740 1 --2- 172.21.15.86:0/3855131566 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a65b6e7ba0 0x55a65b6ea060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:43.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.062+0000 7fb5aec18740 4 client.20058 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:43.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.062+0000 7fb5aec18740 1 -- 172.21.15.86:0/3855131566 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55a65b640c00 con 0x55a65b6e7ba0 2024-04-21T08:49:43.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.063+0000 7fb5a94ed640 1 --2- 172.21.15.86:0/3855131566 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a65b6e7ba0 0x55a65b6ea060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:43.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.063+0000 7fb58d7fa640 10 client.20058 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:43.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.063+0000 7fb5a94ed640 1 --2- 172.21.15.86:0/3855131566 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a65b6e7ba0 0x55a65b6ea060 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=62d0a66bd57acb2d server_cookie=2c90eaf30a1ef441 in_seq=0 out_seq=0 2024-04-21T08:49:43.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.063+0000 7fb58d7fa640 1 -- 172.21.15.86:0/3855131566 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55a65b640c00 con 0x55a65b6e7ba0 2024-04-21T08:49:43.070 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.063+0000 7fb58d7fa640 10 client.20058 handle_command_reply: tid=0 2024-04-21T08:49:43.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.064+0000 7fb5aec18740 1 client.20058 shutdown 2024-04-21T08:49:43.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.064+0000 7fb5aec18740 1 -- 172.21.15.86:0/3855131566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a65b6ce580 msgr2=0x55a65b6d4ae0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:43.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.064+0000 7fb5aec18740 1 --2- 172.21.15.86:0/3855131566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a65b6ce580 0x55a65b6d4ae0 secure :-1 s=READY pgs=1943 cs=0 l=1 rev1=1 crypto rx=0x7fb5a000dea0 tx=0x7fb5a000df90 comp rx=0 tx=0).stop 2024-04-21T08:49:43.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.064+0000 7fb5aec18740 1 -- 172.21.15.86:0/3855131566 shutdown_connections 2024-04-21T08:49:43.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.064+0000 7fb5aec18740 1 --2- 172.21.15.86:0/3855131566 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a65b6e7ba0 0x55a65b6ea060 crc :-1 s=READY pgs=319 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:43.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.064+0000 7fb5aec18740 1 --2- 172.21.15.86:0/3855131566 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65b6d7a80 0x55a65b6d5560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:43.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.064+0000 7fb5aec18740 1 --2- 172.21.15.86:0/3855131566 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a65b6ceea0 0x55a65b6d5020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:43.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.064+0000 7fb5aec18740 1 --2- 172.21.15.86:0/3855131566 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a65b6ce580 0x55a65b6d4ae0 unknown :-1 s=CLOSED pgs=1943 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:43.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.064+0000 7fb58d7fa640 0 client.20058 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:43.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.064+0000 7fb5aec18740 1 -- 172.21.15.86:0/3855131566 >> 172.21.15.86:0/3855131566 conn(0x55a65b6c8cb0 msgr2=0x55a65b6caa00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:43.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.064+0000 7fb5aec18740 1 -- 172.21.15.86:0/3855131566 shutdown_connections 2024-04-21T08:49:43.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.064+0000 7fb5aec18740 1 -- 172.21.15.86:0/3855131566 wait complete. 2024-04-21T08:49:43.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.064+0000 7fb5aec18740 20 client.20058 trim_cache size 0 max 16384 2024-04-21T08:49:43.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.081+0000 7fb5aec18740 1 Processor -- start 2024-04-21T08:49:43.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.081+0000 7fb5aec18740 1 -- start start 2024-04-21T08:49:43.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.081+0000 7fb5aec18740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a65b6ce580 0x55a65b6d7210 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:43.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.081+0000 7fb5aec18740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a65b6ceea0 0x55a65b6d3a20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:43.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.081+0000 7fb5aec18740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65b6d7a80 0x55a65b6e3730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:43.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.081+0000 7fb5aec18740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a41af3a0 con 0x55a65b6ce580 2024-04-21T08:49:43.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.081+0000 7fb5aec18740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a41b0ee0 con 0x55a65b6ceea0 2024-04-21T08:49:43.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.081+0000 7fb5aec18740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a41afa70 con 0x55a65b6d7a80 2024-04-21T08:49:43.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.081+0000 7fb5a9cee640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a65b6ce580 0x55a65b6d7210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:43.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.081+0000 7fb5a9cee640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a65b6ce580 0x55a65b6d7210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36300/0 (socket says 172.21.15.86:36300) 2024-04-21T08:49:43.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.081+0000 7fb5a9cee640 1 -- 172.21.15.86:0/3741852279 learned_addr learned my addr 172.21.15.86:0/3741852279 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:43.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.081+0000 7fb5a9cee640 1 -- 172.21.15.86:0/3741852279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65b6d7a80 msgr2=0x55a65b6e3730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:43.086 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.081+0000 7fb5a9cee640 1 --2- 172.21.15.86:0/3741852279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65b6d7a80 0x55a65b6e3730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:43.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.081+0000 7fb5a94ed640 1 --2- 172.21.15.86:0/3741852279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a65b6ceea0 0x55a65b6d3a20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:43.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.081+0000 7fb5a9cee640 1 -- 172.21.15.86:0/3741852279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a65b6ceea0 msgr2=0x55a65b6d3a20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:43.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.081+0000 7fb5a9cee640 1 --2- 172.21.15.86:0/3741852279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a65b6ceea0 0x55a65b6d3a20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:43.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.081+0000 7fb5a9cee640 1 -- 172.21.15.86:0/3741852279 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55a65b6e66f0 con 0x55a65b6ce580 2024-04-21T08:49:43.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.081+0000 7fb5a94ed640 1 --2- 172.21.15.86:0/3741852279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a65b6ceea0 0x55a65b6d3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:43.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.081+0000 7fb5a9cee640 1 --2- 172.21.15.86:0/3741852279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a65b6ce580 0x55a65b6d7210 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0000c00 tx=0x7fb5a000dac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=572995bb79a59ab4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:43.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.082+0000 7fb58f7fe640 1 -- 172.21.15.86:0/3741852279 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a001b680 con 0x55a65b6ce580 2024-04-21T08:49:43.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.082+0000 7fb58f7fe640 1 -- 172.21.15.86:0/3741852279 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5a002c030 con 0x55a65b6ce580 2024-04-21T08:49:43.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.082+0000 7fb58f7fe640 1 -- 172.21.15.86:0/3741852279 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb5a0027790 con 0x55a65b6ce580 2024-04-21T08:49:43.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.082+0000 7fb5aec18740 1 -- 172.21.15.86:0/3741852279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a65b6ce580 msgr2=0x55a65b6d7210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:43.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.082+0000 7fb5aec18740 1 --2- 172.21.15.86:0/3741852279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a65b6ce580 0x55a65b6d7210 secure :-1 s=READY pgs=1944 cs=0 l=1 rev1=1 crypto rx=0x7fb5a0000c00 tx=0x7fb5a000dac0 comp rx=0 tx=0).stop 2024-04-21T08:49:43.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.082+0000 7fb5aec18740 1 -- 172.21.15.86:0/3741852279 shutdown_connections 2024-04-21T08:49:43.087 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.082+0000 7fb5aec18740 1 --2- 172.21.15.86:0/3741852279 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65b6d7a80 0x55a65b6e3730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:43.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.082+0000 7fb5aec18740 1 --2- 172.21.15.86:0/3741852279 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a65b6ceea0 0x55a65b6d3a20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:43.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.082+0000 7fb5aec18740 1 --2- 172.21.15.86:0/3741852279 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a65b6ce580 0x55a65b6d7210 unknown :-1 s=CLOSED pgs=1944 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:43.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.082+0000 7fb5aec18740 1 -- 172.21.15.86:0/3741852279 >> 172.21.15.86:0/3741852279 conn(0x55a65b6c8cb0 msgr2=0x55a65b6c7d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:43.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.082+0000 7fb5aec18740 1 -- 172.21.15.86:0/3741852279 shutdown_connections 2024-04-21T08:49:43.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.082+0000 7fb5aec18740 1 -- 172.21.15.86:0/3741852279 wait complete. 2024-04-21T08:49:43.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.083+0000 7fb5aec18740 1 -- start start 2024-04-21T08:49:43.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.083+0000 7fb5aec18740 1 Processor -- start 2024-04-21T08:49:43.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.083+0000 7fb5aec18740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a65b6ce580 0x55a65b74ba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:43.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.083+0000 7fb5aec18740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65b6ceea0 0x55a65b74bf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:43.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.083+0000 7fb5aec18740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a65b6d7a80 0x55a65b746160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:43.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.083+0000 7fb5aec18740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a41b0810 con 0x55a65b6d7a80 2024-04-21T08:49:43.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.083+0000 7fb5aec18740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a41b0140 con 0x55a65b6ce580 2024-04-21T08:49:43.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.083+0000 7fb5aec18740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fb5a41ad3a0 con 0x55a65b6ceea0 2024-04-21T08:49:43.088 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.083+0000 7fb5aec18740 10 client.0 _ll_register_callbacks cb 0x7fb5a000d420 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:43.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.083+0000 7fb5aa4ef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a65b6d7a80 0x55a65b746160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:43.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.083+0000 7fb5aa4ef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a65b6d7a80 0x55a65b746160 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:36314/0 (socket says 172.21.15.86:36314) 2024-04-21T08:49:43.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.083+0000 7fb5aa4ef640 1 -- 172.21.15.86:0/743850620 learned_addr learned my addr 172.21.15.86:0/743850620 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:43.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.083+0000 7fb5a94ed640 1 --2- 172.21.15.86:0/743850620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65b6ceea0 0x55a65b74bf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:43.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.083+0000 7fb5a9cee640 1 --2- 172.21.15.86:0/743850620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a65b6ce580 0x55a65b74ba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:43.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.083+0000 7fb5aa4ef640 1 -- 172.21.15.86:0/743850620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65b6ceea0 msgr2=0x55a65b74bf80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:43.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.083+0000 7fb5aa4ef640 1 --2- 172.21.15.86:0/743850620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65b6ceea0 0x55a65b74bf80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:43.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.083+0000 7fb5aa4ef640 1 -- 172.21.15.86:0/743850620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a65b6ce580 msgr2=0x55a65b74ba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:43.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.083+0000 7fb5aa4ef640 1 --2- 172.21.15.86:0/743850620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a65b6ce580 0x55a65b74ba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:43.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.083+0000 7fb5aa4ef640 1 -- 172.21.15.86:0/743850620 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55a65b6d9e70 con 0x55a65b6d7a80 2024-04-21T08:49:43.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.083+0000 7fb5a9cee640 1 --2- 172.21.15.86:0/743850620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a65b6ce580 0x55a65b74ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:49:43.089 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.083+0000 7fb5a94ed640 1 --2- 172.21.15.86:0/743850620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65b6ceea0 0x55a65b74bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:43.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.084+0000 7fb5aa4ef640 1 --2- 172.21.15.86:0/743850620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a65b6d7a80 0x55a65b746160 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fb59c037040 tx=0x7fb59c0095c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4dcb0d72e49512a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:43.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.084+0000 7fb58d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:43.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.084+0000 7fb58d7fa640 1 -- 172.21.15.86:0/743850620 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fb59c039040 con 0x55a65b6d7a80 2024-04-21T08:49:43.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.084+0000 7fb58d7fa640 1 -- 172.21.15.86:0/743850620 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7fb59c003b70 con 0x55a65b6d7a80 2024-04-21T08:49:43.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.084+0000 7fb5aec18740 1 -- 172.21.15.86:0/743850620 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fb5a002c030 con 0x55a65b6d7a80 2024-04-21T08:49:43.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.084+0000 7fb58d7fa640 1 -- 172.21.15.86:0/743850620 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fb59c066050 con 0x55a65b6d7a80 2024-04-21T08:49:43.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.084+0000 7fb5aec18740 10 client.20064 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:43.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.084+0000 7fb5aec18740 1 -- 172.21.15.86:0/743850620 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55a65b752f10 con 0x55a65b6d7a80 2024-04-21T08:49:43.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.084+0000 7fb58d7fa640 1 -- 172.21.15.86:0/743850620 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fb5a002c030 con 0x55a65b6d7a80 2024-04-21T08:49:43.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.084+0000 7fb58d7fa640 1 -- 172.21.15.86:0/743850620 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fb59c03f040 con 0x55a65b6d7a80 2024-04-21T08:49:43.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.084+0000 7fb5aec18740 10 client.20064 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:43.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.084+0000 7fb5aec18740 10 client.20064 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:43.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.084+0000 7fb5aec18740 20 client.20064 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:43.090 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.084+0000 7fb5aec18740 1 --2- 172.21.15.86:0/743850620 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a65b7532f0 0x55a65b7556e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:43.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.084+0000 7fb5aec18740 4 client.20064 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:49:43.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.084+0000 7fb5aec18740 1 -- 172.21.15.86:0/743850620 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55a65b63aa70 con 0x55a65b7532f0 2024-04-21T08:49:43.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.085+0000 7fb5a9cee640 1 --2- 172.21.15.86:0/743850620 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a65b7532f0 0x55a65b7556e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:43.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.085+0000 7fb5a9cee640 1 --2- 172.21.15.86:0/743850620 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a65b7532f0 0x55a65b7556e0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2f2e0316aee21bb2 server_cookie=59fe08158192b99f in_seq=0 out_seq=0 2024-04-21T08:49:43.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.085+0000 7fb58d7fa640 10 client.20064 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:43.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.085+0000 7fb58d7fa640 1 -- 172.21.15.86:0/743850620 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55a65b63aa70 con 0x55a65b7532f0 2024-04-21T08:49:43.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.085+0000 7fb58d7fa640 10 client.20064 handle_command_reply: tid=0 2024-04-21T08:49:43.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.085+0000 7fb5aec18740 1 client.20064 shutdown 2024-04-21T08:49:43.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.085+0000 7fb5aec18740 1 -- 172.21.15.86:0/743850620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a65b6d7a80 msgr2=0x55a65b746160 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:43.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.085+0000 7fb5aec18740 1 --2- 172.21.15.86:0/743850620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a65b6d7a80 0x55a65b746160 secure :-1 s=READY pgs=1945 cs=0 l=1 rev1=1 crypto rx=0x7fb59c037040 tx=0x7fb59c0095c0 comp rx=0 tx=0).stop 2024-04-21T08:49:43.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.086+0000 7fb5aec18740 1 -- 172.21.15.86:0/743850620 shutdown_connections 2024-04-21T08:49:43.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.086+0000 7fb5aec18740 1 --2- 172.21.15.86:0/743850620 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55a65b7532f0 0x55a65b7556e0 crc :-1 s=READY pgs=320 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:43.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.086+0000 7fb5aec18740 1 --2- 172.21.15.86:0/743850620 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55a65b6d7a80 0x55a65b746160 unknown :-1 s=CLOSED pgs=1945 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:43.091 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.086+0000 7fb5aec18740 1 --2- 172.21.15.86:0/743850620 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55a65b6ceea0 0x55a65b74bf80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:43.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.086+0000 7fb58d7fa640 0 client.20064 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:43.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.086+0000 7fb5aec18740 1 --2- 172.21.15.86:0/743850620 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55a65b6ce580 0x55a65b74ba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:43.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.086+0000 7fb5aec18740 1 -- 172.21.15.86:0/743850620 >> 172.21.15.86:0/743850620 conn(0x55a65b6c8cb0 msgr2=0x55a65b6c74e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:43.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.086+0000 7fb5aec18740 1 -- 172.21.15.86:0/743850620 shutdown_connections 2024-04-21T08:49:43.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.086+0000 7fb5aec18740 1 -- 172.21.15.86:0/743850620 wait complete. 2024-04-21T08:49:43.092 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.086+0000 7fb5aec18740 20 client.20064 trim_cache size 0 max 16384 2024-04-21T08:49:43.092 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:49:43.092 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:49:43.092 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-21T08:49:43.093 INFO:teuthology.orchestra.run.smithi086.stdout: "23cbafa6-5e18-4768-ad86-5bba0143477b": { 2024-04-21T08:49:43.093 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-21T08:49:43.093 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "23cbafa6-5e18-4768-ad86-5bba0143477b", 2024-04-21T08:49:43.093 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-21T08:49:43.093 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:49:43.093 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:49:43.093 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:49:43.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.086+0000 7fb5abf52640 1 -- 172.21.15.86:0/2943220750 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb5900a3d20 msgr2=0x7fb5900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:43.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.086+0000 7fb5abf52640 1 --2- 172.21.15.86:0/2943220750 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb5900a3d20 0x7fb5900a61e0 secure :-1 s=READY pgs=323 cs=0 l=1 rev1=1 crypto rx=0x7fb59c04c000 tx=0x7fb59c045000 comp rx=0 tx=0).stop 2024-04-21T08:49:43.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.086+0000 7fb5abf52640 1 -- 172.21.15.86:0/2943220750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5a4106160 msgr2=0x7fb5a4198c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:43.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.086+0000 7fb5abf52640 1 --2- 172.21.15.86:0/2943220750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5a4106160 0x7fb5a4198c40 secure :-1 s=READY pgs=1941 cs=0 l=1 rev1=1 crypto rx=0x7fb59400e9f0 tx=0x7fb59400eeb0 comp rx=0 tx=0).stop 2024-04-21T08:49:43.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.086+0000 7fb5abf52640 1 -- 172.21.15.86:0/2943220750 shutdown_connections 2024-04-21T08:49:43.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.086+0000 7fb5abf52640 1 --2- 172.21.15.86:0/2943220750 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fb5900a3d20 0x7fb5900a61e0 unknown :-1 s=CLOSED pgs=323 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:43.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.086+0000 7fb5abf52640 1 --2- 172.21.15.86:0/2943220750 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fb5a419cfc0 0x7fb5a419d470 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:43.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.086+0000 7fb5abf52640 1 --2- 172.21.15.86:0/2943220750 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fb5a4106160 0x7fb5a4198c40 unknown :-1 s=CLOSED pgs=1941 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:43.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.086+0000 7fb5abf52640 1 --2- 172.21.15.86:0/2943220750 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fb5a4101090 0x7fb5a4198700 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:43.094 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.086+0000 7fb5abf52640 1 -- 172.21.15.86:0/2943220750 >> 172.21.15.86:0/2943220750 conn(0x7fb5a40771d0 msgr2=0x7fb5a406a940 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:43.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.086+0000 7fb5abf52640 1 -- 172.21.15.86:0/2943220750 shutdown_connections 2024-04-21T08:49:43.095 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:49:43.086+0000 7fb5abf52640 1 -- 172.21.15.86:0/2943220750 wait complete. 2024-04-21T08:49:43.096 INFO:tasks.cephfs.filesystem:scrub status for tag:23cbafa6-5e18-4768-ad86-5bba0143477b - {'path': '/', 'tag': '23cbafa6-5e18-4768-ad86-5bba0143477b', 'options': 'recursive,force'} 2024-04-21T08:49:44.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:43 smithi088 ceph-mon[31259]: pgmap v341: 129 pgs: 129 active+clean; 4.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-21T08:49:44.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:43 smithi088 ceph-mon[31259]: scrub queued for path: / 2024-04-21T08:49:44.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:43 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:44.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:43 smithi088 ceph-mon[31259]: scrub summary: active paths [/] 2024-04-21T08:49:44.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:43 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:44.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:43 smithi179 ceph-mon[31829]: pgmap v341: 129 pgs: 129 active+clean; 4.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-21T08:49:44.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:43 smithi179 ceph-mon[31829]: scrub queued for path: / 2024-04-21T08:49:44.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:43 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:44.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:43 smithi179 ceph-mon[31829]: scrub summary: active paths [/] 2024-04-21T08:49:44.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:43 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:44.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:43 smithi086 ceph-mon[27939]: pgmap v341: 129 pgs: 129 active+clean; 4.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-21T08:49:44.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:43 smithi086 ceph-mon[27939]: scrub queued for path: / 2024-04-21T08:49:44.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:43 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:44.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:43 smithi086 ceph-mon[27939]: scrub summary: active paths [/] 2024-04-21T08:49:44.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:43 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:49:45.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:44 smithi088 ceph-mon[31259]: scrub summary: idle 2024-04-21T08:49:45.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:44 smithi179 ceph-mon[31829]: scrub summary: idle 2024-04-21T08:49:45.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:44 smithi086 ceph-mon[27939]: scrub summary: idle 2024-04-21T08:49:46.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:45 smithi088 ceph-mon[31259]: pgmap v342: 129 pgs: 129 active+clean; 4.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:49:46.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:45 smithi179 ceph-mon[31829]: pgmap v342: 129 pgs: 129 active+clean; 4.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:49:46.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:45 smithi086 ceph-mon[27939]: pgmap v342: 129 pgs: 129 active+clean; 4.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:49:48.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:47 smithi088 ceph-mon[31259]: pgmap v343: 129 pgs: 129 active+clean; 4.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:49:48.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:47 smithi179 ceph-mon[31829]: pgmap v343: 129 pgs: 129 active+clean; 4.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:49:48.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:47 smithi086 ceph-mon[27939]: pgmap v343: 129 pgs: 129 active+clean; 4.1 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 42 KiB/s wr, 15 op/s 2024-04-21T08:49:50.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:49 smithi088 ceph-mon[31259]: pgmap v344: 129 pgs: 129 active+clean; 4.3 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-04-21T08:49:50.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:49 smithi179 ceph-mon[31829]: pgmap v344: 129 pgs: 129 active+clean; 4.3 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-04-21T08:49:50.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:49 smithi086 ceph-mon[27939]: pgmap v344: 129 pgs: 129 active+clean; 4.3 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 60 KiB/s wr, 23 op/s 2024-04-21T08:49:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.902+0000 7fc9f6715640 1 -- 172.21.15.86:0/3417997581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9f01010e0 msgr2=0x7fc9f01014c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.902+0000 7fc9f6715640 1 --2- 172.21.15.86:0/3417997581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9f01010e0 0x7fc9f01014c0 secure :-1 s=READY pgs=1946 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0009900 tx=0x7fc9e002bc30 comp rx=0 tx=0).stop 2024-04-21T08:49:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.902+0000 7fc9f6715640 1 -- 172.21.15.86:0/3417997581 shutdown_connections 2024-04-21T08:49:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.902+0000 7fc9f6715640 1 --2- 172.21.15.86:0/3417997581 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc9f01061b0 0x7fc9f010dd80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.902+0000 7fc9f6715640 1 --2- 172.21.15.86:0/3417997581 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9f0101a90 0x7fc9f0105ad0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.902+0000 7fc9f6715640 1 --2- 172.21.15.86:0/3417997581 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9f01010e0 0x7fc9f01014c0 unknown :-1 s=CLOSED pgs=1946 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.902+0000 7fc9f6715640 1 -- 172.21.15.86:0/3417997581 >> 172.21.15.86:0/3417997581 conn(0x7fc9f00771d0 msgr2=0x7fc9f00775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:50.904 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.902+0000 7fc9f6715640 1 -- 172.21.15.86:0/3417997581 shutdown_connections 2024-04-21T08:49:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.902+0000 7fc9f6715640 1 -- 172.21.15.86:0/3417997581 wait complete. 2024-04-21T08:49:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.902+0000 7fc9f6715640 1 Processor -- start 2024-04-21T08:49:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.902+0000 7fc9f6715640 1 -- start start 2024-04-21T08:49:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.903+0000 7fc9f6715640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9f01010e0 0x7fc9f0199450 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.903+0000 7fc9f6715640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc9f01061b0 0x7fc9f0199990 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.903+0000 7fc9f6715640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9f0199f20 0x7fc9f01a44d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.903+0000 7fc9f6715640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f01110f0 con 0x7fc9f01010e0 2024-04-21T08:49:50.905 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.903+0000 7fc9f6715640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f0110f70 con 0x7fc9f01061b0 2024-04-21T08:49:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.903+0000 7fc9f6715640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f0111270 con 0x7fc9f0199f20 2024-04-21T08:49:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.903+0000 7fc9effff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9f01010e0 0x7fc9f0199450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.903+0000 7fc9effff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9f01010e0 0x7fc9f0199450 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39186/0 (socket says 172.21.15.86:39186) 2024-04-21T08:49:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.903+0000 7fc9effff640 1 -- 172.21.15.86:0/2746745635 learned_addr learned my addr 172.21.15.86:0/2746745635 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.903+0000 7fc9ef7fe640 1 --2- 172.21.15.86:0/2746745635 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc9f01061b0 0x7fc9f0199990 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.903+0000 7fc9f4cb2640 1 --2- 172.21.15.86:0/2746745635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9f0199f20 0x7fc9f01a44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.903+0000 7fc9effff640 1 -- 172.21.15.86:0/2746745635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9f0199f20 msgr2=0x7fc9f01a44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.903+0000 7fc9effff640 1 --2- 172.21.15.86:0/2746745635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9f0199f20 0x7fc9f01a44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.903+0000 7fc9effff640 1 -- 172.21.15.86:0/2746745635 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc9f01061b0 msgr2=0x7fc9f0199990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.903+0000 7fc9effff640 1 --2- 172.21.15.86:0/2746745635 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc9f01061b0 0x7fc9f0199990 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.903+0000 7fc9effff640 1 -- 172.21.15.86:0/2746745635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc9f01a4ca0 con 0x7fc9f01010e0 2024-04-21T08:49:50.906 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.903+0000 7fc9effff640 1 --2- 172.21.15.86:0/2746745635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9f01010e0 0x7fc9f0199450 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0005940 tx=0x7fc9e002c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.903+0000 7fc9ed7fa640 1 -- 172.21.15.86:0/2746745635 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0043020 con 0x7fc9f01010e0 2024-04-21T08:49:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.904+0000 7fc9f6715640 1 -- 172.21.15.86:0/2746745635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc9f01a4f30 con 0x7fc9f01010e0 2024-04-21T08:49:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.904+0000 7fc9ed7fa640 1 -- 172.21.15.86:0/2746745635 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9e002cbe0 con 0x7fc9f01010e0 2024-04-21T08:49:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.904+0000 7fc9ed7fa640 1 -- 172.21.15.86:0/2746745635 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0003af0 con 0x7fc9f01010e0 2024-04-21T08:49:50.907 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.904+0000 7fc9f6715640 1 -- 172.21.15.86:0/2746745635 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc9f01a53f0 con 0x7fc9f01010e0 2024-04-21T08:49:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.906+0000 7fc9ed7fa640 1 -- 172.21.15.86:0/2746745635 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc9e0046020 con 0x7fc9f01010e0 2024-04-21T08:49:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.906+0000 7fc9f93e1740 1 Processor -- start 2024-04-21T08:49:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.906+0000 7fc9f93e1740 1 -- start start 2024-04-21T08:49:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.906+0000 7fc9ed7fa640 1 --2- 172.21.15.86:0/2746745635 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc9d80a3dc0 0x7fc9d80a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.906+0000 7fc9f93e1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643a19e68d0 0x5643a19e5620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:50.916 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.907+0000 7fc9f93e1740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643a19e5b60 0x5643a19e5fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.907+0000 7fc9f93e1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643a19f34f0 0x5643a19f58e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.907+0000 7fc9f93e1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5643a194a8d0 con 0x5643a19e68d0 2024-04-21T08:49:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.907+0000 7fc9f93e1740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5643a18e7e80 con 0x5643a19e5b60 2024-04-21T08:49:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.907+0000 7fc9f93e1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5643a1946890 con 0x5643a19f34f0 2024-04-21T08:49:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.907+0000 7fc9f4cb2640 1 --2- 172.21.15.86:0/2746745635 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc9d80a3dc0 0x7fc9d80a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.907+0000 7fc9effff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643a19e68d0 0x5643a19e5620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.907+0000 7fc9ed7fa640 1 -- 172.21.15.86:0/2746745635 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7fc9e00f2d10 con 0x7fc9f01010e0 2024-04-21T08:49:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.907+0000 7fc9effff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643a19e68d0 0x5643a19e5620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39188/0 (socket says 172.21.15.86:39188) 2024-04-21T08:49:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.907+0000 7fc9effff640 1 -- 172.21.15.86:0/1424715593 learned_addr learned my addr 172.21.15.86:0/1424715593 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:50.917 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.907+0000 7fc9ef7fe640 1 --2- 172.21.15.86:0/1424715593 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643a19e5b60 0x5643a19e5fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.907+0000 7fc9f4cb2640 1 --2- 172.21.15.86:0/1424715593 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643a19f34f0 0x5643a19f58e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.907+0000 7fc9effff640 1 -- 172.21.15.86:0/1424715593 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643a19f34f0 msgr2=0x5643a19f58e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.907+0000 7fc9effff640 1 --2- 172.21.15.86:0/1424715593 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643a19f34f0 0x5643a19f58e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.907+0000 7fc9effff640 1 -- 172.21.15.86:0/1424715593 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643a19e5b60 msgr2=0x5643a19e5fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.907+0000 7fc9effff640 1 --2- 172.21.15.86:0/1424715593 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643a19e5b60 0x5643a19e5fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.907+0000 7fc9effff640 1 -- 172.21.15.86:0/1424715593 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643a1909320 con 0x5643a19e68d0 2024-04-21T08:49:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.907+0000 7fc9ef7fe640 1 --2- 172.21.15.86:0/1424715593 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643a19e5b60 0x5643a19e5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.907+0000 7fc9f4cb2640 1 --2- 172.21.15.86:0/2746745635 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc9d80a3dc0 0x7fc9d80a6280 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc9e80092f0 tx=0x7fc9e8014a70 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.907+0000 7fc9effff640 1 --2- 172.21.15.86:0/1424715593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643a19e68d0 0x5643a19e5620 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00bae50 tx=0x7fc9e00035f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9496096bba33612d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.907+0000 7fc9d67fc640 1 -- 172.21.15.86:0/1424715593 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e00be350 con 0x5643a19e68d0 2024-04-21T08:49:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.907+0000 7fc9d67fc640 1 -- 172.21.15.86:0/1424715593 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9e003c350 con 0x5643a19e68d0 2024-04-21T08:49:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.907+0000 7fc9d67fc640 1 -- 172.21.15.86:0/1424715593 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e0000ed0 con 0x5643a19e68d0 2024-04-21T08:49:50.918 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.908+0000 7fc9f93e1740 1 -- 172.21.15.86:0/1424715593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643a19e68d0 msgr2=0x5643a19e5620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.908+0000 7fc9f93e1740 1 --2- 172.21.15.86:0/1424715593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643a19e68d0 0x5643a19e5620 secure :-1 s=READY pgs=1948 cs=0 l=1 rev1=1 crypto rx=0x7fc9e00bae50 tx=0x7fc9e00035f0 comp rx=0 tx=0).stop 2024-04-21T08:49:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.908+0000 7fc9f93e1740 1 -- 172.21.15.86:0/1424715593 shutdown_connections 2024-04-21T08:49:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.908+0000 7fc9f93e1740 1 --2- 172.21.15.86:0/1424715593 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643a19f34f0 0x5643a19f58e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.908+0000 7fc9f93e1740 1 --2- 172.21.15.86:0/1424715593 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643a19e5b60 0x5643a19e5fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.908+0000 7fc9f93e1740 1 --2- 172.21.15.86:0/1424715593 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643a19e68d0 0x5643a19e5620 unknown :-1 s=CLOSED pgs=1948 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.908+0000 7fc9f93e1740 1 -- 172.21.15.86:0/1424715593 >> 172.21.15.86:0/1424715593 conn(0x5643a19acc90 msgr2=0x5643a19e0850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.908+0000 7fc9f93e1740 1 -- 172.21.15.86:0/1424715593 shutdown_connections 2024-04-21T08:49:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.908+0000 7fc9f93e1740 1 -- 172.21.15.86:0/1424715593 wait complete. 2024-04-21T08:49:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.908+0000 7fc9f93e1740 1 -- start start 2024-04-21T08:49:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.909+0000 7fc9f93e1740 1 Processor -- start 2024-04-21T08:49:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.909+0000 7fc9f93e1740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643a19e5b60 0x5643a19f2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.909+0000 7fc9f93e1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643a19e68d0 0x5643a19f01a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.909+0000 7fc9f93e1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643a19f34f0 0x5643a19f06e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:50.919 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.909+0000 7fc9f93e1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5643a19f5fe0 con 0x5643a19f34f0 2024-04-21T08:49:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.909+0000 7fc9f93e1740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5643a18d2eb0 con 0x5643a19e5b60 2024-04-21T08:49:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.909+0000 7fc9f93e1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5643a19f6160 con 0x5643a19e68d0 2024-04-21T08:49:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.909+0000 7fc9f93e1740 10 client.0 _ll_register_callbacks cb 0x5643a18ffb40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.909+0000 7fc9f4cb2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643a19f34f0 0x5643a19f06e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.909+0000 7fc9f4cb2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643a19f34f0 0x5643a19f06e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39200/0 (socket says 172.21.15.86:39200) 2024-04-21T08:49:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.909+0000 7fc9f4cb2640 1 -- 172.21.15.86:0/2424508190 learned_addr learned my addr 172.21.15.86:0/2424508190 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.909+0000 7fc9ef7fe640 1 --2- 172.21.15.86:0/2424508190 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643a19e68d0 0x5643a19f01a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.909+0000 7fc9effff640 1 --2- 172.21.15.86:0/2424508190 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643a19e5b60 0x5643a19f2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.909+0000 7fc9f4cb2640 1 -- 172.21.15.86:0/2424508190 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643a19e68d0 msgr2=0x5643a19f01a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.909+0000 7fc9f4cb2640 1 --2- 172.21.15.86:0/2424508190 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643a19e68d0 0x5643a19f01a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.909+0000 7fc9f4cb2640 1 -- 172.21.15.86:0/2424508190 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643a19e5b60 msgr2=0x5643a19f2e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.909+0000 7fc9f4cb2640 1 --2- 172.21.15.86:0/2424508190 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643a19e5b60 0x5643a19f2e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.909+0000 7fc9f4cb2640 1 -- 172.21.15.86:0/2424508190 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643a165d210 con 0x5643a19f34f0 2024-04-21T08:49:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.909+0000 7fc9effff640 1 --2- 172.21.15.86:0/2424508190 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643a19e5b60 0x5643a19f2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:50.920 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.910+0000 7fc9f4cb2640 1 --2- 172.21.15.86:0/2424508190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643a19f34f0 0x5643a19f06e0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8015fd0 tx=0x7fc9e8004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5754d0efa570958e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.910+0000 7fc9cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.910+0000 7fc9cf7fe640 1 -- 172.21.15.86:0/2424508190 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8026070 con 0x5643a19f34f0 2024-04-21T08:49:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.910+0000 7fc9cf7fe640 1 -- 172.21.15.86:0/2424508190 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7fc9e8004d20 con 0x5643a19f34f0 2024-04-21T08:49:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.910+0000 7fc9f93e1740 1 -- 172.21.15.86:0/2424508190 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9e003c350 con 0x5643a19f34f0 2024-04-21T08:49:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.910+0000 7fc9cf7fe640 1 -- 172.21.15.86:0/2424508190 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc9e8031050 con 0x5643a19f34f0 2024-04-21T08:49:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.910+0000 7fc9f93e1740 10 client.20076 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.910+0000 7fc9f93e1740 1 -- 172.21.15.86:0/2424508190 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5643a1729620 con 0x5643a19f34f0 2024-04-21T08:49:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.910+0000 7fc9cf7fe640 1 -- 172.21.15.86:0/2424508190 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9e003c350 con 0x5643a19f34f0 2024-04-21T08:49:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.910+0000 7fc9cf7fe640 1 -- 172.21.15.86:0/2424508190 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc9e800f040 con 0x5643a19f34f0 2024-04-21T08:49:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.910+0000 7fc9f93e1740 10 client.20076 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.910+0000 7fc9f93e1740 10 client.20076 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:50.921 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.910+0000 7fc9f93e1740 20 client.20076 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.910+0000 7fc9f93e1740 1 --2- 172.21.15.86:0/2424508190 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5643a1a031e0 0x5643a1a055d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.910+0000 7fc9f93e1740 4 client.20076 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:49:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.910+0000 7fc9f93e1740 1 -- 172.21.15.86:0/2424508190 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5643a195e880 con 0x5643a1a031e0 2024-04-21T08:49:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.911+0000 7fc9effff640 1 --2- 172.21.15.86:0/2424508190 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5643a1a031e0 0x5643a1a055d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.911+0000 7fc9effff640 1 --2- 172.21.15.86:0/2424508190 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5643a1a031e0 0x5643a1a055d0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=407f27bcb6183abe server_cookie=5290ceb077a16bcf in_seq=0 out_seq=0 2024-04-21T08:49:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.911+0000 7fc9cf7fe640 10 client.20076 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.912+0000 7fc9cf7fe640 1 -- 172.21.15.86:0/2424508190 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5643a195e880 con 0x5643a1a031e0 2024-04-21T08:49:50.922 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.912+0000 7fc9cf7fe640 10 client.20076 handle_command_reply: tid=0 2024-04-21T08:49:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.912+0000 7fc9f93e1740 1 client.20076 shutdown 2024-04-21T08:49:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.912+0000 7fc9f93e1740 1 -- 172.21.15.86:0/2424508190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643a19f34f0 msgr2=0x5643a19f06e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.912+0000 7fc9f93e1740 1 --2- 172.21.15.86:0/2424508190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643a19f34f0 0x5643a19f06e0 secure :-1 s=READY pgs=1949 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8015fd0 tx=0x7fc9e8004ab0 comp rx=0 tx=0).stop 2024-04-21T08:49:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.912+0000 7fc9f93e1740 1 -- 172.21.15.86:0/2424508190 shutdown_connections 2024-04-21T08:49:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.912+0000 7fc9f93e1740 1 --2- 172.21.15.86:0/2424508190 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5643a1a031e0 0x5643a1a055d0 crc :-1 s=READY pgs=321 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.912+0000 7fc9f93e1740 1 --2- 172.21.15.86:0/2424508190 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643a19f34f0 0x5643a19f06e0 unknown :-1 s=CLOSED pgs=1949 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.912+0000 7fc9f93e1740 1 --2- 172.21.15.86:0/2424508190 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643a19e68d0 0x5643a19f01a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.912+0000 7fc9cf7fe640 0 client.20076 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.912+0000 7fc9f93e1740 1 --2- 172.21.15.86:0/2424508190 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643a19e5b60 0x5643a19f2e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.912+0000 7fc9f93e1740 1 -- 172.21.15.86:0/2424508190 >> 172.21.15.86:0/2424508190 conn(0x5643a19acc90 msgr2=0x5643a19e25a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:50.923 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.912+0000 7fc9f93e1740 1 -- 172.21.15.86:0/2424508190 shutdown_connections 2024-04-21T08:49:50.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.912+0000 7fc9f93e1740 1 -- 172.21.15.86:0/2424508190 wait complete. 2024-04-21T08:49:50.924 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.912+0000 7fc9f93e1740 20 client.20076 trim_cache size 0 max 16384 2024-04-21T08:49:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.927+0000 7fc9f93e1740 1 Processor -- start 2024-04-21T08:49:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.927+0000 7fc9f93e1740 1 -- start start 2024-04-21T08:49:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.927+0000 7fc9f93e1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643a19acc90 0x5643a19e50a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.927+0000 7fc9f93e1740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643a19e5b60 0x5643a19f28d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.927+0000 7fc9f93e1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643a19e68d0 0x5643a19f2e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.927+0000 7fc9f93e1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f01ad080 con 0x5643a19e68d0 2024-04-21T08:49:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.927+0000 7fc9f93e1740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f01aebc0 con 0x5643a19e5b60 2024-04-21T08:49:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.927+0000 7fc9f93e1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f01ad750 con 0x5643a19acc90 2024-04-21T08:49:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.927+0000 7fc9f4cb2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643a19e68d0 0x5643a19f2e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.927+0000 7fc9f4cb2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643a19e68d0 0x5643a19f2e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39210/0 (socket says 172.21.15.86:39210) 2024-04-21T08:49:50.931 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.927+0000 7fc9f4cb2640 1 -- 172.21.15.86:0/1723636675 learned_addr learned my addr 172.21.15.86:0/1723636675 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.927+0000 7fc9ef7fe640 1 --2- 172.21.15.86:0/1723636675 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643a19e5b60 0x5643a19f28d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.927+0000 7fc9effff640 1 --2- 172.21.15.86:0/1723636675 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643a19acc90 0x5643a19e50a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.927+0000 7fc9f4cb2640 1 -- 172.21.15.86:0/1723636675 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643a19acc90 msgr2=0x5643a19e50a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.927+0000 7fc9f4cb2640 1 --2- 172.21.15.86:0/1723636675 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643a19acc90 0x5643a19e50a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.927+0000 7fc9f4cb2640 1 -- 172.21.15.86:0/1723636675 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643a19e5b60 msgr2=0x5643a19f28d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.927+0000 7fc9f4cb2640 1 --2- 172.21.15.86:0/1723636675 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643a19e5b60 0x5643a19f28d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.927+0000 7fc9f4cb2640 1 -- 172.21.15.86:0/1723636675 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5643a19e4e30 con 0x5643a19e68d0 2024-04-21T08:49:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.928+0000 7fc9ef7fe640 1 --2- 172.21.15.86:0/1723636675 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643a19e5b60 0x5643a19f28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.928+0000 7fc9f4cb2640 1 --2- 172.21.15.86:0/1723636675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643a19e68d0 0x5643a19f2e10 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8005530 tx=0x7fc9e8014270 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e4fc8bfc6ca66982 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.928+0000 7fc9d5ffb640 1 -- 172.21.15.86:0/1723636675 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e803a070 con 0x5643a19e68d0 2024-04-21T08:49:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.928+0000 7fc9d5ffb640 1 -- 172.21.15.86:0/1723636675 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9e8004b20 con 0x5643a19e68d0 2024-04-21T08:49:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.928+0000 7fc9d5ffb640 1 -- 172.21.15.86:0/1723636675 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e8026050 con 0x5643a19e68d0 2024-04-21T08:49:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.928+0000 7fc9f93e1740 1 -- 172.21.15.86:0/1723636675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643a19e68d0 msgr2=0x5643a19f2e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.928+0000 7fc9f93e1740 1 --2- 172.21.15.86:0/1723636675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643a19e68d0 0x5643a19f2e10 secure :-1 s=READY pgs=1950 cs=0 l=1 rev1=1 crypto rx=0x7fc9e8005530 tx=0x7fc9e8014270 comp rx=0 tx=0).stop 2024-04-21T08:49:50.932 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.928+0000 7fc9f93e1740 1 -- 172.21.15.86:0/1723636675 shutdown_connections 2024-04-21T08:49:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.928+0000 7fc9f93e1740 1 --2- 172.21.15.86:0/1723636675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643a19e68d0 0x5643a19f2e10 unknown :-1 s=CLOSED pgs=1950 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.928+0000 7fc9f93e1740 1 --2- 172.21.15.86:0/1723636675 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643a19e5b60 0x5643a19f28d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.928+0000 7fc9f93e1740 1 --2- 172.21.15.86:0/1723636675 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643a19acc90 0x5643a19e50a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.928+0000 7fc9f93e1740 1 -- 172.21.15.86:0/1723636675 >> 172.21.15.86:0/1723636675 conn(0x5643a19ebb70 msgr2=0x5643a19eea70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.928+0000 7fc9f93e1740 1 -- 172.21.15.86:0/1723636675 shutdown_connections 2024-04-21T08:49:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.928+0000 7fc9f93e1740 1 -- 172.21.15.86:0/1723636675 wait complete. 2024-04-21T08:49:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.929+0000 7fc9f93e1740 1 -- start start 2024-04-21T08:49:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.929+0000 7fc9f93e1740 1 Processor -- start 2024-04-21T08:49:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.929+0000 7fc9f93e1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643a19acc90 0x5643a19fb7c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.929+0000 7fc9f93e1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643a19e5b60 0x5643a19f6b30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.929+0000 7fc9f93e1740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643a19e68d0 0x5643a19f7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.929+0000 7fc9f93e1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f01ae4f0 con 0x5643a19e5b60 2024-04-21T08:49:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.929+0000 7fc9f93e1740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f01ade20 con 0x5643a19e68d0 2024-04-21T08:49:50.933 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.929+0000 7fc9f93e1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc9f01ab080 con 0x5643a19acc90 2024-04-21T08:49:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.929+0000 7fc9f93e1740 10 client.0 _ll_register_callbacks cb 0x7fc9e8014040 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:49:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.929+0000 7fc9ef7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643a19e5b60 0x5643a19f6b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.929+0000 7fc9ef7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643a19e5b60 0x5643a19f6b30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:39212/0 (socket says 172.21.15.86:39212) 2024-04-21T08:49:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.929+0000 7fc9ef7fe640 1 -- 172.21.15.86:0/1871131542 learned_addr learned my addr 172.21.15.86:0/1871131542 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:49:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.929+0000 7fc9effff640 1 --2- 172.21.15.86:0/1871131542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643a19acc90 0x5643a19fb7c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.930+0000 7fc9f4cb2640 1 --2- 172.21.15.86:0/1871131542 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643a19e68d0 0x5643a19f7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.930+0000 7fc9ef7fe640 1 -- 172.21.15.86:0/1871131542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643a19acc90 msgr2=0x5643a19fb7c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:50.934 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.930+0000 7fc9ef7fe640 1 --2- 172.21.15.86:0/1871131542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643a19acc90 0x5643a19fb7c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.930+0000 7fc9ef7fe640 1 -- 172.21.15.86:0/1871131542 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643a19e68d0 msgr2=0x5643a19f7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:49:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.930+0000 7fc9ef7fe640 1 --2- 172.21.15.86:0/1871131542 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643a19e68d0 0x5643a19f7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.930+0000 7fc9ef7fe640 1 -- 172.21.15.86:0/1871131542 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5643a19f78a0 con 0x5643a19e5b60 2024-04-21T08:49:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.930+0000 7fc9f4cb2640 1 --2- 172.21.15.86:0/1871131542 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643a19e68d0 0x5643a19f7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:49:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.930+0000 7fc9ef7fe640 1 --2- 172.21.15.86:0/1871131542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643a19e5b60 0x5643a19f6b30 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fc9f006b2d0 tx=0x7fc9e4011be0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=13a0f3cd8948f7b0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:49:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.930+0000 7fc9cf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:49:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.930+0000 7fc9cf7fe640 1 -- 172.21.15.86:0/1871131542 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc9e4010bf0 con 0x5643a19e5b60 2024-04-21T08:49:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.930+0000 7fc9cf7fe640 1 -- 172.21.15.86:0/1871131542 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7fc9e4010d90 con 0x5643a19e5b60 2024-04-21T08:49:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.930+0000 7fc9f93e1740 1 -- 172.21.15.86:0/1871131542 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc9e8004b20 con 0x5643a19e5b60 2024-04-21T08:49:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.930+0000 7fc9cf7fe640 1 -- 172.21.15.86:0/1871131542 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc9e4029050 con 0x5643a19e5b60 2024-04-21T08:49:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.930+0000 7fc9f93e1740 10 client.20082 fetch_fsmap learned FSMap version 29 2024-04-21T08:49:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.930+0000 7fc9f93e1740 1 -- 172.21.15.86:0/1871131542 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5643a1a69820 con 0x5643a19e5b60 2024-04-21T08:49:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.930+0000 7fc9cf7fe640 1 -- 172.21.15.86:0/1871131542 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc9e8004b20 con 0x5643a19e5b60 2024-04-21T08:49:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.930+0000 7fc9cf7fe640 1 -- 172.21.15.86:0/1871131542 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc9e400e3b0 con 0x5643a19e5b60 2024-04-21T08:49:50.935 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.930+0000 7fc9f93e1740 10 client.20082 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:49:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.930+0000 7fc9f93e1740 10 client.20082 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:49:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.930+0000 7fc9f93e1740 20 client.20082 populate_metadata read hostname 'smithi086' 2024-04-21T08:49:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.930+0000 7fc9f93e1740 1 --2- 172.21.15.86:0/1871131542 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5643a19f34f0 0x5643a1a6c060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:49:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.930+0000 7fc9f93e1740 4 client.20082 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:49:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.931+0000 7fc9f93e1740 1 -- 172.21.15.86:0/1871131542 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5643a18ea3d0 con 0x5643a19f34f0 2024-04-21T08:49:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.931+0000 7fc9effff640 1 --2- 172.21.15.86:0/1871131542 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5643a19f34f0 0x5643a1a6c060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:49:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.931+0000 7fc9cf7fe640 10 client.20082 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:50.936 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.931+0000 7fc9effff640 1 --2- 172.21.15.86:0/1871131542 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5643a19f34f0 0x5643a1a6c060 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a2b0ca507ea460d7 server_cookie=379bb8d9429b77b6 in_seq=0 out_seq=0 2024-04-21T08:49:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.945+0000 7fc9cf7fe640 1 -- 172.21.15.86:0/1871131542 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5643a18ea3d0 con 0x5643a19f34f0 2024-04-21T08:49:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.945+0000 7fc9cf7fe640 10 client.20082 handle_command_reply: tid=0 2024-04-21T08:49:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.945+0000 7fc9f93e1740 1 client.20082 shutdown 2024-04-21T08:49:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.946+0000 7fc9f93e1740 1 -- 172.21.15.86:0/1871131542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643a19e5b60 msgr2=0x5643a19f6b30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.946+0000 7fc9f93e1740 1 --2- 172.21.15.86:0/1871131542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643a19e5b60 0x5643a19f6b30 secure :-1 s=READY pgs=1951 cs=0 l=1 rev1=1 crypto rx=0x7fc9f006b2d0 tx=0x7fc9e4011be0 comp rx=0 tx=0).stop 2024-04-21T08:49:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.946+0000 7fc9f93e1740 1 -- 172.21.15.86:0/1871131542 shutdown_connections 2024-04-21T08:49:50.946 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.946+0000 7fc9f93e1740 1 --2- 172.21.15.86:0/1871131542 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5643a19f34f0 0x5643a1a6c060 crc :-1 s=READY pgs=322 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.946+0000 7fc9f93e1740 1 --2- 172.21.15.86:0/1871131542 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5643a19e68d0 0x5643a19f7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.946+0000 7fc9f93e1740 1 --2- 172.21.15.86:0/1871131542 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5643a19e5b60 0x5643a19f6b30 unknown :-1 s=CLOSED pgs=1951 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.946+0000 7fc9cf7fe640 0 client.20082 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:49:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.946+0000 7fc9f93e1740 1 --2- 172.21.15.86:0/1871131542 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5643a19acc90 0x5643a19fb7c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.946+0000 7fc9f93e1740 1 -- 172.21.15.86:0/1871131542 >> 172.21.15.86:0/1871131542 conn(0x5643a19ebb70 msgr2=0x5643a19ee9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.946+0000 7fc9f93e1740 1 -- 172.21.15.86:0/1871131542 shutdown_connections 2024-04-21T08:49:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.946+0000 7fc9f93e1740 1 -- 172.21.15.86:0/1871131542 wait complete. 2024-04-21T08:49:50.947 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.946+0000 7fc9f93e1740 20 client.20082 trim_cache size 0 max 16384 2024-04-21T08:49:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:49:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:49:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:49:50.948 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:49:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.946+0000 7fc9f6715640 1 -- 172.21.15.86:0/2746745635 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc9d80a3dc0 msgr2=0x7fc9d80a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:50.949 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.946+0000 7fc9f6715640 1 --2- 172.21.15.86:0/2746745635 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc9d80a3dc0 0x7fc9d80a6280 secure :-1 s=READY pgs=324 cs=0 l=1 rev1=1 crypto rx=0x7fc9e80092f0 tx=0x7fc9e8014a70 comp rx=0 tx=0).stop 2024-04-21T08:49:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.946+0000 7fc9f6715640 1 -- 172.21.15.86:0/2746745635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9f01010e0 msgr2=0x7fc9f0199450 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:49:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.946+0000 7fc9f6715640 1 --2- 172.21.15.86:0/2746745635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9f01010e0 0x7fc9f0199450 secure :-1 s=READY pgs=1947 cs=0 l=1 rev1=1 crypto rx=0x7fc9e0005940 tx=0x7fc9e002c7e0 comp rx=0 tx=0).stop 2024-04-21T08:49:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.947+0000 7fc9f6715640 1 -- 172.21.15.86:0/2746745635 shutdown_connections 2024-04-21T08:49:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.947+0000 7fc9f6715640 1 --2- 172.21.15.86:0/2746745635 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc9d80a3dc0 0x7fc9d80a6280 unknown :-1 s=CLOSED pgs=324 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.947+0000 7fc9f6715640 1 --2- 172.21.15.86:0/2746745635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc9f0199f20 0x7fc9f01a44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.947+0000 7fc9f6715640 1 --2- 172.21.15.86:0/2746745635 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc9f01061b0 0x7fc9f0199990 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.947+0000 7fc9f6715640 1 --2- 172.21.15.86:0/2746745635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc9f01010e0 0x7fc9f0199450 unknown :-1 s=CLOSED pgs=1947 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:49:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.947+0000 7fc9f6715640 1 -- 172.21.15.86:0/2746745635 >> 172.21.15.86:0/2746745635 conn(0x7fc9f00771d0 msgr2=0x7fc9f00feb50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:49:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.947+0000 7fc9f6715640 1 -- 172.21.15.86:0/2746745635 shutdown_connections 2024-04-21T08:49:50.950 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:49:50.947+0000 7fc9f6715640 1 -- 172.21.15.86:0/2746745635 wait complete. 2024-04-21T08:49:52.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:51 smithi088 ceph-mon[31259]: pgmap v345: 129 pgs: 129 active+clean; 4.3 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:49:52.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:51 smithi179 ceph-mon[31829]: pgmap v345: 129 pgs: 129 active+clean; 4.3 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:49:52.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:51 smithi086 ceph-mon[27939]: pgmap v345: 129 pgs: 129 active+clean; 4.3 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 43 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:49:54.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:53 smithi088 ceph-mon[31259]: pgmap v346: 129 pgs: 129 active+clean; 4.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 66 KiB/s wr, 25 op/s 2024-04-21T08:49:54.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:53 smithi179 ceph-mon[31829]: pgmap v346: 129 pgs: 129 active+clean; 4.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 66 KiB/s wr, 25 op/s 2024-04-21T08:49:54.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:53 smithi086 ceph-mon[27939]: pgmap v346: 129 pgs: 129 active+clean; 4.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 65 KiB/s rd, 66 KiB/s wr, 25 op/s 2024-04-21T08:49:56.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:55 smithi088 ceph-mon[31259]: pgmap v347: 129 pgs: 129 active+clean; 4.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:49:56.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:55 smithi179 ceph-mon[31829]: pgmap v347: 129 pgs: 129 active+clean; 4.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:49:56.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:55 smithi086 ceph-mon[27939]: pgmap v347: 129 pgs: 129 active+clean; 4.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:49:58.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:57 smithi179 ceph-mon[31829]: pgmap v348: 129 pgs: 129 active+clean; 4.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:49:58.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:57 smithi086 ceph-mon[27939]: pgmap v348: 129 pgs: 129 active+clean; 4.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:49:58.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:57 smithi088 ceph-mon[31259]: pgmap v348: 129 pgs: 129 active+clean; 4.6 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 42 KiB/s wr, 16 op/s 2024-04-21T08:50:00.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:49:59 smithi179 ceph-mon[31829]: pgmap v349: 129 pgs: 129 active+clean; 4.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 23 op/s 2024-04-21T08:50:00.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:49:59 smithi086 ceph-mon[27939]: pgmap v349: 129 pgs: 129 active+clean; 4.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 23 op/s 2024-04-21T08:50:00.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:49:59 smithi088 ceph-mon[31259]: pgmap v349: 129 pgs: 129 active+clean; 4.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 23 op/s 2024-04-21T08:50:01.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:00 smithi179 ceph-mon[31829]: overall HEALTH_OK 2024-04-21T08:50:01.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:00 smithi086 ceph-mon[27939]: overall HEALTH_OK 2024-04-21T08:50:01.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:00 smithi088 ceph-mon[31259]: overall HEALTH_OK 2024-04-21T08:50:02.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:01 smithi179 ceph-mon[31829]: pgmap v350: 129 pgs: 129 active+clean; 4.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-21T08:50:02.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:01 smithi086 ceph-mon[27939]: pgmap v350: 129 pgs: 129 active+clean; 4.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-21T08:50:02.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:01 smithi088 ceph-mon[31259]: pgmap v350: 129 pgs: 129 active+clean; 4.8 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 40 KiB/s wr, 15 op/s 2024-04-21T08:50:03.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:02 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:50:03.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:02 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:50:03.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:02 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:50:03.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:02 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:50:03.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:02 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:50:03.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:02 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:50:03.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:02 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:50:03.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:02 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:50:03.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:02 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:50:03.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:02 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:50:03.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:02 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:50:03.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:02 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:50:03.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:02 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:50:03.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:02 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:50:03.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:02 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:50:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.035+0000 7faf5836d640 1 -- 172.21.15.86:0/2290265149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf501011a0 msgr2=0x7faf50101620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.035+0000 7faf5836d640 1 --2- 172.21.15.86:0/2290265149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf501011a0 0x7faf50101620 secure :-1 s=READY pgs=1952 cs=0 l=1 rev1=1 crypto rx=0x7faf40009900 tx=0x7faf4002bc80 comp rx=0 tx=0).stop 2024-04-21T08:50:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.035+0000 7faf5836d640 1 -- 172.21.15.86:0/2290265149 shutdown_connections 2024-04-21T08:50:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.035+0000 7faf5836d640 1 --2- 172.21.15.86:0/2290265149 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf50101b60 0x7faf5010dc70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.035+0000 7faf5836d640 1 --2- 172.21.15.86:0/2290265149 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf501011a0 0x7faf50101620 unknown :-1 s=CLOSED pgs=1952 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.035+0000 7faf5836d640 1 --2- 172.21.15.86:0/2290265149 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf50106fb0 0x7faf50107390 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.035+0000 7faf5836d640 1 -- 172.21.15.86:0/2290265149 >> 172.21.15.86:0/2290265149 conn(0x7faf50076c50 msgr2=0x7faf50077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:04.038 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.035+0000 7faf5836d640 1 -- 172.21.15.86:0/2290265149 shutdown_connections 2024-04-21T08:50:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.035+0000 7faf5836d640 1 -- 172.21.15.86:0/2290265149 wait complete. 2024-04-21T08:50:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.036+0000 7faf5836d640 1 Processor -- start 2024-04-21T08:50:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.036+0000 7faf5836d640 1 -- start start 2024-04-21T08:50:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.036+0000 7faf5836d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf501011a0 0x7faf501985e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.036+0000 7faf5836d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf50101b60 0x7faf50198b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.036+0000 7faf5836d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf50106fb0 0x7faf5019cea0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.036+0000 7faf5836d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faf501121e0 con 0x7faf50106fb0 2024-04-21T08:50:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.036+0000 7faf5836d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faf50112060 con 0x7faf501011a0 2024-04-21T08:50:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.036+0000 7faf5836d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faf50112360 con 0x7faf50101b60 2024-04-21T08:50:04.039 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.036+0000 7faf5690a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf50106fb0 0x7faf5019cea0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.036+0000 7faf5690a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf50106fb0 0x7faf5019cea0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44614/0 (socket says 172.21.15.86:44614) 2024-04-21T08:50:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.036+0000 7faf5690a640 1 -- 172.21.15.86:0/1048231760 learned_addr learned my addr 172.21.15.86:0/1048231760 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.036+0000 7faf55908640 1 --2- 172.21.15.86:0/1048231760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf50101b60 0x7faf50198b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.036+0000 7faf56109640 1 --2- 172.21.15.86:0/1048231760 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf501011a0 0x7faf501985e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.036+0000 7faf5690a640 1 -- 172.21.15.86:0/1048231760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf50101b60 msgr2=0x7faf50198b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.036+0000 7faf5690a640 1 --2- 172.21.15.86:0/1048231760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf50101b60 0x7faf50198b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.036+0000 7faf5690a640 1 -- 172.21.15.86:0/1048231760 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf501011a0 msgr2=0x7faf501985e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.036+0000 7faf5690a640 1 --2- 172.21.15.86:0/1048231760 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf501011a0 0x7faf501985e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.036+0000 7faf5690a640 1 -- 172.21.15.86:0/1048231760 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7faf5019d5d0 con 0x7faf50106fb0 2024-04-21T08:50:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.036+0000 7faf56109640 1 --2- 172.21.15.86:0/1048231760 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf501011a0 0x7faf501985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.036+0000 7faf5690a640 1 --2- 172.21.15.86:0/1048231760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf50106fb0 0x7faf5019cea0 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7faf4800eaa0 tx=0x7faf4800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.037+0000 7faf477fe640 1 -- 172.21.15.86:0/1048231760 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4800cdb0 con 0x7faf50106fb0 2024-04-21T08:50:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.037+0000 7faf477fe640 1 -- 172.21.15.86:0/1048231760 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf48005020 con 0x7faf50106fb0 2024-04-21T08:50:04.040 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.037+0000 7faf5836d640 1 -- 172.21.15.86:0/1048231760 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7faf5019d8c0 con 0x7faf50106fb0 2024-04-21T08:50:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.037+0000 7faf477fe640 1 -- 172.21.15.86:0/1048231760 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf480116d0 con 0x7faf50106fb0 2024-04-21T08:50:04.041 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.037+0000 7faf5836d640 1 -- 172.21.15.86:0/1048231760 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7faf501a5150 con 0x7faf50106fb0 2024-04-21T08:50:04.049 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.039+0000 7faf477fe640 1 -- 172.21.15.86:0/1048231760 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7faf480051d0 con 0x7faf50106fb0 2024-04-21T08:50:04.049 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.039+0000 7faf5b033740 1 Processor -- start 2024-04-21T08:50:04.049 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.039+0000 7faf5b033740 1 -- start start 2024-04-21T08:50:04.049 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.039+0000 7faf5b033740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589b01288d0 0x5589b0127620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:04.049 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.039+0000 7faf5b033740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589b0127b60 0x5589b0127fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:04.049 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.039+0000 7faf477fe640 1 --2- 172.21.15.86:0/1048231760 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7faf3c0a3dc0 0x7faf3c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:04.049 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.039+0000 7faf5b033740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589b01310e0 0x5589b01334d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:04.049 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.039+0000 7faf5b033740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5589b008c8d0 con 0x5589b01288d0 2024-04-21T08:50:04.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.039+0000 7faf5b033740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5589b0029e80 con 0x5589b0127b60 2024-04-21T08:50:04.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.040+0000 7faf5b033740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5589b0088890 con 0x5589b01310e0 2024-04-21T08:50:04.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.040+0000 7faf55908640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589b01288d0 0x5589b0127620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:04.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.040+0000 7faf55908640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589b01288d0 0x5589b0127620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44622/0 (socket says 172.21.15.86:44622) 2024-04-21T08:50:04.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.040+0000 7faf55908640 1 -- 172.21.15.86:0/3694441946 learned_addr learned my addr 172.21.15.86:0/3694441946 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:04.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.040+0000 7faf5690a640 1 --2- 172.21.15.86:0/3694441946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589b01310e0 0x5589b01334d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:04.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.040+0000 7faf55908640 1 --2- 172.21.15.86:0/1048231760 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7faf3c0a3dc0 0x7faf3c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:04.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.040+0000 7faf56109640 1 --2- 172.21.15.86:0/3694441946 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589b0127b60 0x5589b0127fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:04.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.040+0000 7faf477fe640 1 -- 172.21.15.86:0/1048231760 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7faf480d0e70 con 0x7faf50106fb0 2024-04-21T08:50:04.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.040+0000 7faf55908640 1 -- 172.21.15.86:0/3694441946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589b01310e0 msgr2=0x5589b01334d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:04.050 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.040+0000 7faf55908640 1 --2- 172.21.15.86:0/3694441946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589b01310e0 0x5589b01334d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.040+0000 7faf55908640 1 -- 172.21.15.86:0/3694441946 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589b0127b60 msgr2=0x5589b0127fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.040+0000 7faf55908640 1 --2- 172.21.15.86:0/3694441946 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589b0127b60 0x5589b0127fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.040+0000 7faf55908640 1 -- 172.21.15.86:0/3694441946 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589b004b320 con 0x5589b01288d0 2024-04-21T08:50:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.040+0000 7faf55908640 1 --2- 172.21.15.86:0/3694441946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589b01288d0 0x5589b0127620 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7faf4002c1c0 tx=0x7faf4002c980 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f68d576798a1c18 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.040+0000 7faf55908640 1 --2- 172.21.15.86:0/1048231760 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7faf3c0a3dc0 0x7faf3c0a6280 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7faf40009900 tx=0x7faf40047000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.040+0000 7faf44bfe640 1 -- 172.21.15.86:0/3694441946 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4002c5c0 con 0x5589b01288d0 2024-04-21T08:50:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.040+0000 7faf44bfe640 1 -- 172.21.15.86:0/3694441946 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf40050070 con 0x5589b01288d0 2024-04-21T08:50:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.040+0000 7faf44bfe640 1 -- 172.21.15.86:0/3694441946 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf40003850 con 0x5589b01288d0 2024-04-21T08:50:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.041+0000 7faf5b033740 1 -- 172.21.15.86:0/3694441946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589b01288d0 msgr2=0x5589b0127620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.041+0000 7faf5b033740 1 --2- 172.21.15.86:0/3694441946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589b01288d0 0x5589b0127620 secure :-1 s=READY pgs=1954 cs=0 l=1 rev1=1 crypto rx=0x7faf4002c1c0 tx=0x7faf4002c980 comp rx=0 tx=0).stop 2024-04-21T08:50:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.041+0000 7faf5b033740 1 -- 172.21.15.86:0/3694441946 shutdown_connections 2024-04-21T08:50:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.041+0000 7faf5b033740 1 --2- 172.21.15.86:0/3694441946 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589b01310e0 0x5589b01334d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.041+0000 7faf5b033740 1 --2- 172.21.15.86:0/3694441946 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589b0127b60 0x5589b0127fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.041+0000 7faf5b033740 1 --2- 172.21.15.86:0/3694441946 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589b01288d0 0x5589b0127620 unknown :-1 s=CLOSED pgs=1954 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:04.051 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.041+0000 7faf5b033740 1 -- 172.21.15.86:0/3694441946 >> 172.21.15.86:0/3694441946 conn(0x5589b00eec90 msgr2=0x5589b0122850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.041+0000 7faf5b033740 1 -- 172.21.15.86:0/3694441946 shutdown_connections 2024-04-21T08:50:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.041+0000 7faf5b033740 1 -- 172.21.15.86:0/3694441946 wait complete. 2024-04-21T08:50:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.042+0000 7faf5b033740 1 -- start start 2024-04-21T08:50:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.042+0000 7faf5b033740 1 Processor -- start 2024-04-21T08:50:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.042+0000 7faf5b033740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589b0127b60 0x5589b0130af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.042+0000 7faf5b033740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589b01288d0 0x5589b012de80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.042+0000 7faf5b033740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589b01310e0 0x5589b012e490 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.042+0000 7faf5b033740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5589b0133db0 con 0x5589b0127b60 2024-04-21T08:50:04.052 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.042+0000 7faf5b033740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5589b0014eb0 con 0x5589b01288d0 2024-04-21T08:50:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.042+0000 7faf5b033740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5589b0133f30 con 0x5589b01310e0 2024-04-21T08:50:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.042+0000 7faf56109640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589b0127b60 0x5589b0130af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.042+0000 7faf5b033740 10 client.0 _ll_register_callbacks cb 0x5589b0041b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.042+0000 7faf56109640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589b0127b60 0x5589b0130af0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44636/0 (socket says 172.21.15.86:44636) 2024-04-21T08:50:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.042+0000 7faf56109640 1 -- 172.21.15.86:0/1345435029 learned_addr learned my addr 172.21.15.86:0/1345435029 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.042+0000 7faf55908640 1 --2- 172.21.15.86:0/1345435029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589b01288d0 0x5589b012de80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.042+0000 7faf5690a640 1 --2- 172.21.15.86:0/1345435029 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589b01310e0 0x5589b012e490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.042+0000 7faf56109640 1 -- 172.21.15.86:0/1345435029 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589b01310e0 msgr2=0x5589b012e490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.042+0000 7faf56109640 1 --2- 172.21.15.86:0/1345435029 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589b01310e0 0x5589b012e490 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.042+0000 7faf56109640 1 -- 172.21.15.86:0/1345435029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589b01288d0 msgr2=0x5589b012de80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.042+0000 7faf56109640 1 --2- 172.21.15.86:0/1345435029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589b01288d0 0x5589b012de80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.042+0000 7faf56109640 1 -- 172.21.15.86:0/1345435029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589afd9f210 con 0x5589b0127b60 2024-04-21T08:50:04.053 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.042+0000 7faf55908640 1 --2- 172.21.15.86:0/1345435029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589b01288d0 0x5589b012de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.042+0000 7faf56109640 1 --2- 172.21.15.86:0/1345435029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589b0127b60 0x5589b0130af0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7faf4c010a50 tx=0x7faf4c010f10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2695ae4b3f756d8e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.042+0000 7faf39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.043+0000 7faf39ffb640 1 -- 172.21.15.86:0/1345435029 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4c01d060 con 0x5589b0127b60 2024-04-21T08:50:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.043+0000 7faf39ffb640 1 -- 172.21.15.86:0/1345435029 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7faf4c00ebe0 con 0x5589b0127b60 2024-04-21T08:50:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.043+0000 7faf5b033740 1 -- 172.21.15.86:0/1345435029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf40050070 con 0x5589b0127b60 2024-04-21T08:50:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.043+0000 7faf39ffb640 1 -- 172.21.15.86:0/1345435029 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faf4c024050 con 0x5589b0127b60 2024-04-21T08:50:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.043+0000 7faf5b033740 10 client.20094 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:04.054 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.043+0000 7faf5b033740 1 -- 172.21.15.86:0/1345435029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5589afe6b620 con 0x5589b0127b60 2024-04-21T08:50:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.043+0000 7faf39ffb640 1 -- 172.21.15.86:0/1345435029 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf40050070 con 0x5589b0127b60 2024-04-21T08:50:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.043+0000 7faf39ffb640 1 -- 172.21.15.86:0/1345435029 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7faf4c00d3a0 con 0x5589b0127b60 2024-04-21T08:50:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.043+0000 7faf5b033740 10 client.20094 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.043+0000 7faf5b033740 10 client.20094 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.043+0000 7faf5b033740 20 client.20094 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.043+0000 7faf5b033740 1 --2- 172.21.15.86:0/1345435029 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5589b01411e0 0x5589b01435f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.043+0000 7faf5b033740 4 client.20094 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.043+0000 7faf5b033740 1 -- 172.21.15.86:0/1345435029 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5589b00a0880 con 0x5589b01411e0 2024-04-21T08:50:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.043+0000 7faf55908640 1 --2- 172.21.15.86:0/1345435029 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5589b01411e0 0x5589b01435f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:04.055 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.044+0000 7faf39ffb640 10 client.20094 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.044+0000 7faf55908640 1 --2- 172.21.15.86:0/1345435029 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5589b01411e0 0x5589b01435f0 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=879f59e2bf1139a1 server_cookie=416895926cc63be4 in_seq=0 out_seq=0 2024-04-21T08:50:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.044+0000 7faf39ffb640 1 -- 172.21.15.86:0/1345435029 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5589b00a0880 con 0x5589b01411e0 2024-04-21T08:50:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.044+0000 7faf39ffb640 10 client.20094 handle_command_reply: tid=0 2024-04-21T08:50:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.045+0000 7faf5b033740 1 client.20094 shutdown 2024-04-21T08:50:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.045+0000 7faf5b033740 1 -- 172.21.15.86:0/1345435029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589b0127b60 msgr2=0x5589b0130af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.045+0000 7faf5b033740 1 --2- 172.21.15.86:0/1345435029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589b0127b60 0x5589b0130af0 secure :-1 s=READY pgs=1955 cs=0 l=1 rev1=1 crypto rx=0x7faf4c010a50 tx=0x7faf4c010f10 comp rx=0 tx=0).stop 2024-04-21T08:50:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.045+0000 7faf5b033740 1 -- 172.21.15.86:0/1345435029 shutdown_connections 2024-04-21T08:50:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.045+0000 7faf5b033740 1 --2- 172.21.15.86:0/1345435029 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5589b01411e0 0x5589b01435f0 crc :-1 s=READY pgs=323 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.045+0000 7faf5b033740 1 --2- 172.21.15.86:0/1345435029 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589b01310e0 0x5589b012e490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.045+0000 7faf5b033740 1 --2- 172.21.15.86:0/1345435029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589b01288d0 0x5589b012de80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.045+0000 7faf5b033740 1 --2- 172.21.15.86:0/1345435029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589b0127b60 0x5589b0130af0 unknown :-1 s=CLOSED pgs=1955 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.045+0000 7faf39ffb640 0 client.20094 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.045+0000 7faf5b033740 1 -- 172.21.15.86:0/1345435029 >> 172.21.15.86:0/1345435029 conn(0x5589b00eec90 msgr2=0x5589b01245a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.045+0000 7faf5b033740 1 -- 172.21.15.86:0/1345435029 shutdown_connections 2024-04-21T08:50:04.056 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.045+0000 7faf5b033740 1 -- 172.21.15.86:0/1345435029 wait complete. 2024-04-21T08:50:04.057 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.045+0000 7faf5b033740 20 client.20094 trim_cache size 0 max 16384 2024-04-21T08:50:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.059+0000 7faf5b033740 1 Processor -- start 2024-04-21T08:50:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.059+0000 7faf5b033740 1 -- start start 2024-04-21T08:50:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.060+0000 7faf5b033740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589b0127b60 0x5589b013ccb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.060+0000 7faf5b033740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589b01288d0 0x5589b01295d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.060+0000 7faf5b033740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589b01310e0 0x5589b01305b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.060+0000 7faf5b033740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faf501af2a0 con 0x5589b0127b60 2024-04-21T08:50:04.061 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.060+0000 7faf5b033740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faf501b0de0 con 0x5589b01310e0 2024-04-21T08:50:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.060+0000 7faf5b033740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faf501af970 con 0x5589b01288d0 2024-04-21T08:50:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.060+0000 7faf56109640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589b0127b60 0x5589b013ccb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.060+0000 7faf56109640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589b0127b60 0x5589b013ccb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44640/0 (socket says 172.21.15.86:44640) 2024-04-21T08:50:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.060+0000 7faf56109640 1 -- 172.21.15.86:0/51487334 learned_addr learned my addr 172.21.15.86:0/51487334 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.060+0000 7faf56109640 1 -- 172.21.15.86:0/51487334 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589b01288d0 msgr2=0x5589b01295d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.060+0000 7faf56109640 1 --2- 172.21.15.86:0/51487334 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589b01288d0 0x5589b01295d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.060+0000 7faf56109640 1 -- 172.21.15.86:0/51487334 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589b01310e0 msgr2=0x5589b01305b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.060+0000 7faf56109640 1 --2- 172.21.15.86:0/51487334 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589b01310e0 0x5589b01305b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.060+0000 7faf56109640 1 -- 172.21.15.86:0/51487334 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5589b011e120 con 0x5589b0127b60 2024-04-21T08:50:04.062 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.060+0000 7faf56109640 1 --2- 172.21.15.86:0/51487334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589b0127b60 0x5589b013ccb0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7faf4c010a20 tx=0x7faf4c0106c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=17d1374f71c80590 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.060+0000 7faf3bfff640 1 -- 172.21.15.86:0/51487334 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4c00ebe0 con 0x5589b0127b60 2024-04-21T08:50:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.060+0000 7faf3bfff640 1 -- 172.21.15.86:0/51487334 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf4c02d070 con 0x5589b0127b60 2024-04-21T08:50:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.060+0000 7faf3bfff640 1 -- 172.21.15.86:0/51487334 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf4c005530 con 0x5589b0127b60 2024-04-21T08:50:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.061+0000 7faf5b033740 1 -- 172.21.15.86:0/51487334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589b0127b60 msgr2=0x5589b013ccb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.061+0000 7faf5b033740 1 --2- 172.21.15.86:0/51487334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589b0127b60 0x5589b013ccb0 secure :-1 s=READY pgs=1956 cs=0 l=1 rev1=1 crypto rx=0x7faf4c010a20 tx=0x7faf4c0106c0 comp rx=0 tx=0).stop 2024-04-21T08:50:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.061+0000 7faf5b033740 1 -- 172.21.15.86:0/51487334 shutdown_connections 2024-04-21T08:50:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.061+0000 7faf5b033740 1 --2- 172.21.15.86:0/51487334 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589b01310e0 0x5589b01305b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.061+0000 7faf5b033740 1 --2- 172.21.15.86:0/51487334 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589b01288d0 0x5589b01295d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.061+0000 7faf5b033740 1 --2- 172.21.15.86:0/51487334 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589b0127b60 0x5589b013ccb0 unknown :-1 s=CLOSED pgs=1956 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:04.063 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.061+0000 7faf5b033740 1 -- 172.21.15.86:0/51487334 >> 172.21.15.86:0/51487334 conn(0x5589b00eec90 msgr2=0x5589b0128ec0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.061+0000 7faf5b033740 1 -- 172.21.15.86:0/51487334 shutdown_connections 2024-04-21T08:50:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.061+0000 7faf5b033740 1 -- 172.21.15.86:0/51487334 wait complete. 2024-04-21T08:50:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.062+0000 7faf5b033740 1 -- start start 2024-04-21T08:50:04.066 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.062+0000 7faf5b033740 1 Processor -- start 2024-04-21T08:50:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.062+0000 7faf5b033740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589b0127b60 0x5589b012ae40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.062+0000 7faf5b033740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589b01288d0 0x5589b012b380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.062+0000 7faf5b033740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589b01310e0 0x5589b012b8c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.062+0000 7faf5b033740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7faf501b0710 con 0x5589b01288d0 2024-04-21T08:50:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.062+0000 7faf5b033740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7faf501b0040 con 0x5589b01310e0 2024-04-21T08:50:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.062+0000 7faf5b033740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7faf501ad2a0 con 0x5589b0127b60 2024-04-21T08:50:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.062+0000 7faf5b033740 10 client.0 _ll_register_callbacks cb 0x7faf4c00b6b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.062+0000 7faf55908640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589b01288d0 0x5589b012b380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.062+0000 7faf55908640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589b01288d0 0x5589b012b380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:44654/0 (socket says 172.21.15.86:44654) 2024-04-21T08:50:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.062+0000 7faf55908640 1 -- 172.21.15.86:0/2185190851 learned_addr learned my addr 172.21.15.86:0/2185190851 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.062+0000 7faf56109640 1 --2- 172.21.15.86:0/2185190851 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589b0127b60 0x5589b012ae40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.062+0000 7faf5690a640 1 --2- 172.21.15.86:0/2185190851 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589b01310e0 0x5589b012b8c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:04.067 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.062+0000 7faf55908640 1 -- 172.21.15.86:0/2185190851 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589b0127b60 msgr2=0x5589b012ae40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.062+0000 7faf55908640 1 --2- 172.21.15.86:0/2185190851 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589b0127b60 0x5589b012ae40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.062+0000 7faf55908640 1 -- 172.21.15.86:0/2185190851 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589b01310e0 msgr2=0x5589b012b8c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.062+0000 7faf55908640 1 --2- 172.21.15.86:0/2185190851 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589b01310e0 0x5589b012b8c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.062+0000 7faf55908640 1 -- 172.21.15.86:0/2185190851 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5589b01a6a10 con 0x5589b01288d0 2024-04-21T08:50:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.062+0000 7faf56109640 1 --2- 172.21.15.86:0/2185190851 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589b0127b60 0x5589b012ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.062+0000 7faf55908640 1 --2- 172.21.15.86:0/2185190851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589b01288d0 0x5589b012b380 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7faf40004360 tx=0x7faf40009670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8822369ddbd8b935 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.062+0000 7faf39ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.063+0000 7faf39ffb640 1 -- 172.21.15.86:0/2185190851 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7faf40005410 con 0x5589b01288d0 2024-04-21T08:50:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.063+0000 7faf39ffb640 1 -- 172.21.15.86:0/2185190851 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(104..104 src has 1..104) ==== 9496+0+0 (secure 0 0 0) 0x7faf400055b0 con 0x5589b01288d0 2024-04-21T08:50:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.063+0000 7faf5b033740 1 -- 172.21.15.86:0/2185190851 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7faf4c02d070 con 0x5589b01288d0 2024-04-21T08:50:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.063+0000 7faf39ffb640 1 -- 172.21.15.86:0/2185190851 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7faf400046c0 con 0x5589b01288d0 2024-04-21T08:50:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.063+0000 7faf5b033740 10 client.20100 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:04.068 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.063+0000 7faf5b033740 1 -- 172.21.15.86:0/2185190851 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5589b01a7d20 con 0x5589b01288d0 2024-04-21T08:50:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.063+0000 7faf39ffb640 1 -- 172.21.15.86:0/2185190851 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7faf4c02d070 con 0x5589b01288d0 2024-04-21T08:50:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.063+0000 7faf39ffb640 1 -- 172.21.15.86:0/2185190851 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7faf40040040 con 0x5589b01288d0 2024-04-21T08:50:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.063+0000 7faf5b033740 10 client.20100 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.063+0000 7faf5b033740 10 client.20100 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.063+0000 7faf5b033740 20 client.20100 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.063+0000 7faf5b033740 1 --2- 172.21.15.86:0/2185190851 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5589b01a8060 0x5589b01aa450 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.063+0000 7faf5b033740 4 client.20100 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:50:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.063+0000 7faf5b033740 1 -- 172.21.15.86:0/2185190851 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5589b002c3d0 con 0x5589b01a8060 2024-04-21T08:50:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.063+0000 7faf56109640 1 --2- 172.21.15.86:0/2185190851 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5589b01a8060 0x5589b01aa450 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.064+0000 7faf56109640 1 --2- 172.21.15.86:0/2185190851 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5589b01a8060 0x5589b01aa450 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5db0b990040bba53 server_cookie=545e8679c224aa68 in_seq=0 out_seq=0 2024-04-21T08:50:04.069 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.064+0000 7faf39ffb640 10 client.20100 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:04.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.078+0000 7faf39ffb640 1 -- 172.21.15.86:0/2185190851 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5589b002c3d0 con 0x5589b01a8060 2024-04-21T08:50:04.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.078+0000 7faf39ffb640 10 client.20100 handle_command_reply: tid=0 2024-04-21T08:50:04.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.078+0000 7faf5b033740 1 client.20100 shutdown 2024-04-21T08:50:04.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.078+0000 7faf5b033740 1 -- 172.21.15.86:0/2185190851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589b01288d0 msgr2=0x5589b012b380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:04.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.078+0000 7faf5b033740 1 --2- 172.21.15.86:0/2185190851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589b01288d0 0x5589b012b380 secure :-1 s=READY pgs=1957 cs=0 l=1 rev1=1 crypto rx=0x7faf40004360 tx=0x7faf40009670 comp rx=0 tx=0).stop 2024-04-21T08:50:04.079 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.078+0000 7faf5b033740 1 -- 172.21.15.86:0/2185190851 shutdown_connections 2024-04-21T08:50:04.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.078+0000 7faf5b033740 1 --2- 172.21.15.86:0/2185190851 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5589b01a8060 0x5589b01aa450 crc :-1 s=READY pgs=324 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:04.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.078+0000 7faf5b033740 1 --2- 172.21.15.86:0/2185190851 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5589b01310e0 0x5589b012b8c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:04.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.078+0000 7faf5b033740 1 --2- 172.21.15.86:0/2185190851 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5589b01288d0 0x5589b012b380 unknown :-1 s=CLOSED pgs=1957 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:04.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.078+0000 7faf5b033740 1 --2- 172.21.15.86:0/2185190851 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5589b0127b60 0x5589b012ae40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:04.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.078+0000 7faf39ffb640 0 client.20100 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:04.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.078+0000 7faf5b033740 1 -- 172.21.15.86:0/2185190851 >> 172.21.15.86:0/2185190851 conn(0x5589b00eec90 msgr2=0x5589b0121040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:04.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.078+0000 7faf5b033740 1 -- 172.21.15.86:0/2185190851 shutdown_connections 2024-04-21T08:50:04.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.078+0000 7faf5b033740 1 -- 172.21.15.86:0/2185190851 wait complete. 2024-04-21T08:50:04.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.078+0000 7faf5b033740 20 client.20100 trim_cache size 0 max 16384 2024-04-21T08:50:04.080 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:50:04.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:50:04.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:50:04.081 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:50:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.079+0000 7faf5836d640 1 -- 172.21.15.86:0/1048231760 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7faf3c0a3dc0 msgr2=0x7faf3c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.079+0000 7faf5836d640 1 --2- 172.21.15.86:0/1048231760 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7faf3c0a3dc0 0x7faf3c0a6280 secure :-1 s=READY pgs=325 cs=0 l=1 rev1=1 crypto rx=0x7faf40009900 tx=0x7faf40047000 comp rx=0 tx=0).stop 2024-04-21T08:50:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.079+0000 7faf5836d640 1 -- 172.21.15.86:0/1048231760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf50106fb0 msgr2=0x7faf5019cea0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.079+0000 7faf5836d640 1 --2- 172.21.15.86:0/1048231760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf50106fb0 0x7faf5019cea0 secure :-1 s=READY pgs=1953 cs=0 l=1 rev1=1 crypto rx=0x7faf4800eaa0 tx=0x7faf4800ef60 comp rx=0 tx=0).stop 2024-04-21T08:50:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.079+0000 7faf5836d640 1 -- 172.21.15.86:0/1048231760 shutdown_connections 2024-04-21T08:50:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.079+0000 7faf5836d640 1 --2- 172.21.15.86:0/1048231760 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7faf3c0a3dc0 0x7faf3c0a6280 unknown :-1 s=CLOSED pgs=325 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.079+0000 7faf5836d640 1 --2- 172.21.15.86:0/1048231760 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7faf50106fb0 0x7faf5019cea0 unknown :-1 s=CLOSED pgs=1953 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.079+0000 7faf5836d640 1 --2- 172.21.15.86:0/1048231760 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7faf50101b60 0x7faf50198b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.079+0000 7faf5836d640 1 --2- 172.21.15.86:0/1048231760 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7faf501011a0 0x7faf501985e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:04.082 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.079+0000 7faf5836d640 1 -- 172.21.15.86:0/1048231760 >> 172.21.15.86:0/1048231760 conn(0x7faf50076c50 msgr2=0x7faf500ffd20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:04.083 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.079+0000 7faf5836d640 1 -- 172.21.15.86:0/1048231760 shutdown_connections 2024-04-21T08:50:04.083 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:04.079+0000 7faf5836d640 1 -- 172.21.15.86:0/1048231760 wait complete. 2024-04-21T08:50:04.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:03 smithi179 ceph-mon[31829]: pgmap v351: 129 pgs: 129 active+clean; 5.0 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 105 KiB/s wr, 39 op/s 2024-04-21T08:50:04.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:03 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:50:04.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:03 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:50:04.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:03 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:50:04.366 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:03 smithi088 ceph-mon[31259]: pgmap v351: 129 pgs: 129 active+clean; 5.0 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 105 KiB/s wr, 39 op/s 2024-04-21T08:50:04.366 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:03 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:50:04.366 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:03 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:50:04.366 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:03 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:50:04.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:03 smithi086 ceph-mon[27939]: pgmap v351: 129 pgs: 129 active+clean; 5.0 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 72 KiB/s rd, 105 KiB/s wr, 39 op/s 2024-04-21T08:50:04.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:03 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:50:04.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:03 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:50:04.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:03 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:50:06.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:05 smithi179 ceph-mon[31829]: pgmap v352: 129 pgs: 129 active+clean; 5.0 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 81 KiB/s wr, 30 op/s 2024-04-21T08:50:06.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:05 smithi179 ceph-mon[31829]: osdmap e105: 12 total, 12 up, 12 in 2024-04-21T08:50:06.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:05 smithi086 ceph-mon[27939]: pgmap v352: 129 pgs: 129 active+clean; 5.0 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 81 KiB/s wr, 30 op/s 2024-04-21T08:50:06.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:05 smithi086 ceph-mon[27939]: osdmap e105: 12 total, 12 up, 12 in 2024-04-21T08:50:06.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:05 smithi088 ceph-mon[31259]: pgmap v352: 129 pgs: 129 active+clean; 5.0 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 81 KiB/s wr, 30 op/s 2024-04-21T08:50:06.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:05 smithi088 ceph-mon[31259]: osdmap e105: 12 total, 12 up, 12 in 2024-04-21T08:50:08.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:07 smithi179 ceph-mon[31829]: pgmap v354: 129 pgs: 129 active+clean; 5.0 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 98 KiB/s wr, 36 op/s 2024-04-21T08:50:08.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:07 smithi086 ceph-mon[27939]: pgmap v354: 129 pgs: 129 active+clean; 5.0 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 98 KiB/s wr, 36 op/s 2024-04-21T08:50:08.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:07 smithi088 ceph-mon[31259]: pgmap v354: 129 pgs: 129 active+clean; 5.0 MiB data, 440 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 98 KiB/s wr, 36 op/s 2024-04-21T08:50:10.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:09 smithi179 ceph-mon[31829]: pgmap v355: 129 pgs: 129 active+clean; 5.2 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-21T08:50:10.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:09 smithi086 ceph-mon[27939]: pgmap v355: 129 pgs: 129 active+clean; 5.2 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-21T08:50:10.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:09 smithi088 ceph-mon[31259]: pgmap v355: 129 pgs: 129 active+clean; 5.2 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 66 KiB/s rd, 107 KiB/s wr, 39 op/s 2024-04-21T08:50:11.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:11 smithi088 ceph-mon[31259]: osdmap e106: 12 total, 12 up, 12 in 2024-04-21T08:50:11.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:11 smithi088 ceph-mon[31259]: pgmap v357: 129 pgs: 129 active+clean; 5.2 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 37 KiB/s wr, 13 op/s 2024-04-21T08:50:11.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:11 smithi179 ceph-mon[31829]: osdmap e106: 12 total, 12 up, 12 in 2024-04-21T08:50:11.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:11 smithi179 ceph-mon[31829]: pgmap v357: 129 pgs: 129 active+clean; 5.2 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 37 KiB/s wr, 13 op/s 2024-04-21T08:50:12.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:11 smithi086 ceph-mon[27939]: osdmap e106: 12 total, 12 up, 12 in 2024-04-21T08:50:12.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:11 smithi086 ceph-mon[27939]: pgmap v357: 129 pgs: 129 active+clean; 5.2 MiB data, 444 MiB used, 1.0 TiB / 1.0 TiB avail; 34 KiB/s rd, 37 KiB/s wr, 13 op/s 2024-04-21T08:50:13.097 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:50:13.097 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:50:13.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.211+0000 7fd2fff2b640 1 -- 172.21.15.86:0/3461862540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2f8101090 msgr2=0x7fd2f8101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.211+0000 7fd2fff2b640 1 --2- 172.21.15.86:0/3461862540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2f8101090 0x7fd2f8101470 secure :-1 s=READY pgs=1958 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8009900 tx=0x7fd2e802bc30 comp rx=0 tx=0).stop 2024-04-21T08:50:13.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.211+0000 7fd2fff2b640 1 -- 172.21.15.86:0/3461862540 shutdown_connections 2024-04-21T08:50:13.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.211+0000 7fd2fff2b640 1 --2- 172.21.15.86:0/3461862540 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2f8106160 0x7fd2f810dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.211+0000 7fd2fff2b640 1 --2- 172.21.15.86:0/3461862540 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd2f8101a40 0x7fd2f8105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.211+0000 7fd2fff2b640 1 --2- 172.21.15.86:0/3461862540 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2f8101090 0x7fd2f8101470 unknown :-1 s=CLOSED pgs=1958 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.211+0000 7fd2fff2b640 1 -- 172.21.15.86:0/3461862540 >> 172.21.15.86:0/3461862540 conn(0x7fd2f80771d0 msgr2=0x7fd2f80775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:13.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.211+0000 7fd2fff2b640 1 -- 172.21.15.86:0/3461862540 shutdown_connections 2024-04-21T08:50:13.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.211+0000 7fd2fff2b640 1 -- 172.21.15.86:0/3461862540 wait complete. 2024-04-21T08:50:13.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.211+0000 7fd2fff2b640 1 Processor -- start 2024-04-21T08:50:13.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.211+0000 7fd2fff2b640 1 -- start start 2024-04-21T08:50:13.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.212+0000 7fd2fff2b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2f8101090 0x7fd2f8199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.212+0000 7fd2fff2b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2f8106160 0x7fd2f8199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.212+0000 7fd2fff2b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd2f819a010 0x7fd2f81a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.212+0000 7fd2fff2b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f81110c0 con 0x7fd2f8101090 2024-04-21T08:50:13.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.212+0000 7fd2fff2b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f8110f40 con 0x7fd2f8106160 2024-04-21T08:50:13.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.212+0000 7fd2fff2b640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f8111240 con 0x7fd2f819a010 2024-04-21T08:50:13.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.212+0000 7fd2fdcc7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2f8101090 0x7fd2f8199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.212+0000 7fd2fdcc7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2f8101090 0x7fd2f8199540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55118/0 (socket says 172.21.15.86:55118) 2024-04-21T08:50:13.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.212+0000 7fd2fdcc7640 1 -- 172.21.15.86:0/1331348731 learned_addr learned my addr 172.21.15.86:0/1331348731 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:13.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.212+0000 7fd2fe4c8640 1 --2- 172.21.15.86:0/1331348731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd2f819a010 0x7fd2f81a45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.212+0000 7fd2fd4c6640 1 --2- 172.21.15.86:0/1331348731 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2f8106160 0x7fd2f8199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.212+0000 7fd2fdcc7640 1 -- 172.21.15.86:0/1331348731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd2f819a010 msgr2=0x7fd2f81a45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.212+0000 7fd2fdcc7640 1 --2- 172.21.15.86:0/1331348731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd2f819a010 0x7fd2f81a45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.212+0000 7fd2fdcc7640 1 -- 172.21.15.86:0/1331348731 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2f8106160 msgr2=0x7fd2f8199a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.212+0000 7fd2fdcc7640 1 --2- 172.21.15.86:0/1331348731 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2f8106160 0x7fd2f8199a80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.212+0000 7fd2fdcc7640 1 -- 172.21.15.86:0/1331348731 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2f81a4d90 con 0x7fd2f8101090 2024-04-21T08:50:13.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.212+0000 7fd2fd4c6640 1 --2- 172.21.15.86:0/1331348731 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2f8106160 0x7fd2f8199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:13.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.212+0000 7fd2fdcc7640 1 --2- 172.21.15.86:0/1331348731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2f8101090 0x7fd2f8199540 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8005940 tx=0x7fd2e802c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:13.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.213+0000 7fd2eeffd640 1 -- 172.21.15.86:0/1331348731 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8043020 con 0x7fd2f8101090 2024-04-21T08:50:13.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.213+0000 7fd2eeffd640 1 -- 172.21.15.86:0/1331348731 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2e802cbe0 con 0x7fd2f8101090 2024-04-21T08:50:13.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.213+0000 7fd2fff2b640 1 -- 172.21.15.86:0/1331348731 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2f81a5020 con 0x7fd2f8101090 2024-04-21T08:50:13.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.213+0000 7fd2eeffd640 1 -- 172.21.15.86:0/1331348731 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e8003bf0 con 0x7fd2f8101090 2024-04-21T08:50:13.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.213+0000 7fd2fff2b640 1 -- 172.21.15.86:0/1331348731 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2f81a55c0 con 0x7fd2f8101090 2024-04-21T08:50:13.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.215+0000 7fd2eeffd640 1 -- 172.21.15.86:0/1331348731 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd2e8046020 con 0x7fd2f8101090 2024-04-21T08:50:13.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.215+0000 7fd302bf9740 1 Processor -- start 2024-04-21T08:50:13.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.215+0000 7fd302bf9740 1 -- start start 2024-04-21T08:50:13.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.215+0000 7fd302bf9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55781d498580 0x55781d498960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.215+0000 7fd302bf9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d498ea0 0x55781d4a1540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.215+0000 7fd302bf9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55781d4a1a80 0x55781d4a3e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.215+0000 7fd302bf9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55781d3fd1e0 con 0x55781d4a1a80 2024-04-21T08:50:13.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.215+0000 7fd302bf9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55781d39a390 con 0x55781d498580 2024-04-21T08:50:13.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.215+0000 7fd302bf9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55781d3f91a0 con 0x55781d498ea0 2024-04-21T08:50:13.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.215+0000 7fd2eeffd640 1 --2- 172.21.15.86:0/1331348731 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd2e40a3da0 0x7fd2e40a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.216+0000 7fd2fd4c6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55781d4a1a80 0x55781d4a3e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.216+0000 7fd2fd4c6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55781d4a1a80 0x55781d4a3e70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55124/0 (socket says 172.21.15.86:55124) 2024-04-21T08:50:13.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.216+0000 7fd2fd4c6640 1 -- 172.21.15.86:0/1491152873 learned_addr learned my addr 172.21.15.86:0/1491152873 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:13.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.216+0000 7fd2fe4c8640 1 --2- 172.21.15.86:0/1331348731 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd2e40a3da0 0x7fd2e40a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.216+0000 7fd2fdcc7640 1 --2- 172.21.15.86:0/1491152873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d498ea0 0x55781d4a1540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.216+0000 7fd2eeffd640 1 -- 172.21.15.86:0/1331348731 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7fd2e80f2cd0 con 0x7fd2f8101090 2024-04-21T08:50:13.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.216+0000 7fd2fe4c8640 1 --2- 172.21.15.86:0/1491152873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55781d498580 0x55781d498960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.216+0000 7fd2fd4c6640 1 -- 172.21.15.86:0/1491152873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d498ea0 msgr2=0x55781d4a1540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:13.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.216+0000 7fd2fd4c6640 1 --2- 172.21.15.86:0/1491152873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d498ea0 0x55781d4a1540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.216+0000 7fd2fd4c6640 1 -- 172.21.15.86:0/1491152873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55781d498580 msgr2=0x55781d498960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:13.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.216+0000 7fd2fd4c6640 1 --2- 172.21.15.86:0/1491152873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55781d498580 0x55781d498960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.216+0000 7fd2fdcc7640 1 --2- 172.21.15.86:0/1491152873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d498ea0 0x55781d4a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:13.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.216+0000 7fd2fd4c6640 1 -- 172.21.15.86:0/1491152873 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55781d1d97d0 con 0x55781d4a1a80 2024-04-21T08:50:13.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.216+0000 7fd2fe4c8640 1 --2- 172.21.15.86:0/1331348731 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd2e40a3da0 0x7fd2e40a6260 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd2f00098b0 tx=0x7fd2f0012a90 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:13.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.216+0000 7fd2fd4c6640 1 --2- 172.21.15.86:0/1491152873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55781d4a1a80 0x55781d4a3e70 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fd2f400bce0 tx=0x7fd2f400f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fb4823b38d230046 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:13.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.216+0000 7fd2e3fff640 1 -- 172.21.15.86:0/1491152873 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f4015070 con 0x55781d4a1a80 2024-04-21T08:50:13.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.216+0000 7fd2e3fff640 1 -- 172.21.15.86:0/1491152873 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2f401a910 con 0x55781d4a1a80 2024-04-21T08:50:13.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.216+0000 7fd2e3fff640 1 -- 172.21.15.86:0/1491152873 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f4011860 con 0x55781d4a1a80 2024-04-21T08:50:13.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.217+0000 7fd302bf9740 1 -- 172.21.15.86:0/1491152873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55781d4a1a80 msgr2=0x55781d4a3e70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.217+0000 7fd302bf9740 1 --2- 172.21.15.86:0/1491152873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55781d4a1a80 0x55781d4a3e70 secure :-1 s=READY pgs=1960 cs=0 l=1 rev1=1 crypto rx=0x7fd2f400bce0 tx=0x7fd2f400f730 comp rx=0 tx=0).stop 2024-04-21T08:50:13.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.217+0000 7fd302bf9740 1 -- 172.21.15.86:0/1491152873 shutdown_connections 2024-04-21T08:50:13.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.217+0000 7fd302bf9740 1 --2- 172.21.15.86:0/1491152873 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55781d4a1a80 0x55781d4a3e70 unknown :-1 s=CLOSED pgs=1960 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.217+0000 7fd302bf9740 1 --2- 172.21.15.86:0/1491152873 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d498ea0 0x55781d4a1540 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.217+0000 7fd302bf9740 1 --2- 172.21.15.86:0/1491152873 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55781d498580 0x55781d498960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.217+0000 7fd302bf9740 1 -- 172.21.15.86:0/1491152873 >> 172.21.15.86:0/1491152873 conn(0x55781d492cb0 msgr2=0x55781d493090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:13.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.217+0000 7fd302bf9740 1 -- 172.21.15.86:0/1491152873 shutdown_connections 2024-04-21T08:50:13.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.217+0000 7fd302bf9740 1 -- 172.21.15.86:0/1491152873 wait complete. 2024-04-21T08:50:13.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.217+0000 7fd302bf9740 1 -- start start 2024-04-21T08:50:13.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.218+0000 7fd302bf9740 1 Processor -- start 2024-04-21T08:50:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.218+0000 7fd302bf9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55781d498580 0x55781d49ead0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.218+0000 7fd302bf9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55781d498ea0 0x55781d49f010 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.218+0000 7fd302bf9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d4a1a80 0x55781d49f550 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.218+0000 7fd302bf9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55781d4a4870 con 0x55781d498580 2024-04-21T08:50:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.218+0000 7fd302bf9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55781d3b62f0 con 0x55781d498ea0 2024-04-21T08:50:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.218+0000 7fd302bf9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55781d4a49f0 con 0x55781d4a1a80 2024-04-21T08:50:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.218+0000 7fd302bf9740 10 client.0 _ll_register_callbacks cb 0x55781d3c3580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.218+0000 7fd2fdcc7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55781d498580 0x55781d49ead0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.218+0000 7fd2fdcc7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55781d498580 0x55781d49ead0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55140/0 (socket says 172.21.15.86:55140) 2024-04-21T08:50:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.218+0000 7fd2fdcc7640 1 -- 172.21.15.86:0/3149800064 learned_addr learned my addr 172.21.15.86:0/3149800064 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.218+0000 7fd2fdcc7640 1 -- 172.21.15.86:0/3149800064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d4a1a80 msgr2=0x55781d49f550 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.218+0000 7fd2fe4c8640 1 --2- 172.21.15.86:0/3149800064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d4a1a80 0x55781d49f550 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.218+0000 7fd2fd4c6640 1 --2- 172.21.15.86:0/3149800064 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55781d498ea0 0x55781d49f010 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.218+0000 7fd2fdcc7640 1 --2- 172.21.15.86:0/3149800064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d4a1a80 0x55781d49f550 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.218+0000 7fd2fdcc7640 1 -- 172.21.15.86:0/3149800064 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55781d498ea0 msgr2=0x55781d49f010 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:13.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.218+0000 7fd2fdcc7640 1 --2- 172.21.15.86:0/3149800064 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55781d498ea0 0x55781d49f010 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.218+0000 7fd2fdcc7640 1 -- 172.21.15.86:0/3149800064 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55781d10f3f0 con 0x55781d498580 2024-04-21T08:50:13.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.218+0000 7fd2fd4c6640 1 --2- 172.21.15.86:0/3149800064 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55781d498ea0 0x55781d49f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:13.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.218+0000 7fd2fe4c8640 1 --2- 172.21.15.86:0/3149800064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d4a1a80 0x55781d49f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:13.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.219+0000 7fd2fdcc7640 1 --2- 172.21.15.86:0/3149800064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55781d498580 0x55781d49ead0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7fd2e80bb380 tx=0x7fd2e802cd90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=296a5c4ae5f9bc76 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:13.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.219+0000 7fd2e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:13.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.219+0000 7fd2e17fa640 1 -- 172.21.15.86:0/3149800064 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e80be310 con 0x55781d498580 2024-04-21T08:50:13.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.219+0000 7fd2e17fa640 1 -- 172.21.15.86:0/3149800064 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7fd2e80bb020 con 0x55781d498580 2024-04-21T08:50:13.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.219+0000 7fd302bf9740 1 -- 172.21.15.86:0/3149800064 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2f401a910 con 0x55781d498580 2024-04-21T08:50:13.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.219+0000 7fd2e17fa640 1 -- 172.21.15.86:0/3149800064 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd2e80bf0b0 con 0x55781d498580 2024-04-21T08:50:13.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.219+0000 7fd302bf9740 10 client.20112 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:13.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.219+0000 7fd302bf9740 1 -- 172.21.15.86:0/3149800064 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55781d0cf0f0 con 0x55781d498580 2024-04-21T08:50:13.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.219+0000 7fd2e17fa640 1 -- 172.21.15.86:0/3149800064 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2f401a910 con 0x55781d498580 2024-04-21T08:50:13.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.219+0000 7fd2e17fa640 1 -- 172.21.15.86:0/3149800064 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd2e80dd080 con 0x55781d498580 2024-04-21T08:50:13.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.219+0000 7fd302bf9740 10 client.20112 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:13.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.219+0000 7fd302bf9740 10 client.20112 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:13.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.219+0000 7fd302bf9740 20 client.20112 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:13.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.219+0000 7fd302bf9740 1 --2- 172.21.15.86:0/3149800064 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55781d4b1a80 0x55781d4b3ef0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.219+0000 7fd302bf9740 4 client.20112 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:13.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.219+0000 7fd302bf9740 1 -- 172.21.15.86:0/3149800064 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55781d40ac00 con 0x55781d4b1a80 2024-04-21T08:50:13.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.220+0000 7fd2fd4c6640 1 --2- 172.21.15.86:0/3149800064 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55781d4b1a80 0x55781d4b3ef0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.220+0000 7fd2e17fa640 10 client.20112 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:13.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.220+0000 7fd2fd4c6640 1 --2- 172.21.15.86:0/3149800064 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55781d4b1a80 0x55781d4b3ef0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1d1e0d014b57e674 server_cookie=68df450890df01c6 in_seq=0 out_seq=0 2024-04-21T08:50:13.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.221+0000 7fd2e17fa640 1 -- 172.21.15.86:0/3149800064 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55781d40ac00 con 0x55781d4b1a80 2024-04-21T08:50:13.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.221+0000 7fd2e17fa640 10 client.20112 handle_command_reply: tid=0 2024-04-21T08:50:13.227 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.221+0000 7fd302bf9740 1 client.20112 shutdown 2024-04-21T08:50:13.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.221+0000 7fd302bf9740 1 -- 172.21.15.86:0/3149800064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55781d498580 msgr2=0x55781d49ead0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.221+0000 7fd302bf9740 1 --2- 172.21.15.86:0/3149800064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55781d498580 0x55781d49ead0 secure :-1 s=READY pgs=1961 cs=0 l=1 rev1=1 crypto rx=0x7fd2e80bb380 tx=0x7fd2e802cd90 comp rx=0 tx=0).stop 2024-04-21T08:50:13.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.221+0000 7fd302bf9740 1 -- 172.21.15.86:0/3149800064 shutdown_connections 2024-04-21T08:50:13.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.221+0000 7fd302bf9740 1 --2- 172.21.15.86:0/3149800064 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55781d4b1a80 0x55781d4b3ef0 crc :-1 s=READY pgs=325 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.221+0000 7fd302bf9740 1 --2- 172.21.15.86:0/3149800064 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d4a1a80 0x55781d49f550 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.221+0000 7fd2e17fa640 0 client.20112 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:13.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.221+0000 7fd302bf9740 1 --2- 172.21.15.86:0/3149800064 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55781d498ea0 0x55781d49f010 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.221+0000 7fd302bf9740 1 --2- 172.21.15.86:0/3149800064 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55781d498580 0x55781d49ead0 unknown :-1 s=CLOSED pgs=1961 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.221+0000 7fd302bf9740 1 -- 172.21.15.86:0/3149800064 >> 172.21.15.86:0/3149800064 conn(0x55781d492cb0 msgr2=0x55781d494a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:13.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.221+0000 7fd302bf9740 1 -- 172.21.15.86:0/3149800064 shutdown_connections 2024-04-21T08:50:13.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.221+0000 7fd302bf9740 1 -- 172.21.15.86:0/3149800064 wait complete. 2024-04-21T08:50:13.228 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.221+0000 7fd302bf9740 20 client.20112 trim_cache size 0 max 16384 2024-04-21T08:50:13.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.239+0000 7fd302bf9740 1 Processor -- start 2024-04-21T08:50:13.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.239+0000 7fd302bf9740 1 -- start start 2024-04-21T08:50:13.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.239+0000 7fd302bf9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d498580 0x55781d4a1200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.239+0000 7fd302bf9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55781d498ea0 0x55781d4ad730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.239+0000 7fd302bf9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55781d4a1a80 0x55781d493090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.239+0000 7fd302bf9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f81af2d0 con 0x55781d4a1a80 2024-04-21T08:50:13.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.239+0000 7fd302bf9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f81b0e10 con 0x55781d498ea0 2024-04-21T08:50:13.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.239+0000 7fd302bf9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f81af9a0 con 0x55781d498580 2024-04-21T08:50:13.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.239+0000 7fd2fe4c8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55781d4a1a80 0x55781d493090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.239+0000 7fd2fe4c8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55781d4a1a80 0x55781d493090 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55146/0 (socket says 172.21.15.86:55146) 2024-04-21T08:50:13.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.239+0000 7fd2fe4c8640 1 -- 172.21.15.86:0/1497753103 learned_addr learned my addr 172.21.15.86:0/1497753103 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:13.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.239+0000 7fd2fe4c8640 1 -- 172.21.15.86:0/1497753103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d498580 msgr2=0x55781d4a1200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:13.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.239+0000 7fd2fe4c8640 1 --2- 172.21.15.86:0/1497753103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d498580 0x55781d4a1200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.239+0000 7fd2fe4c8640 1 -- 172.21.15.86:0/1497753103 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55781d498ea0 msgr2=0x55781d4ad730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:13.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.239+0000 7fd2fe4c8640 1 --2- 172.21.15.86:0/1497753103 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55781d498ea0 0x55781d4ad730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.239+0000 7fd2fe4c8640 1 -- 172.21.15.86:0/1497753103 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55781d498cd0 con 0x55781d4a1a80 2024-04-21T08:50:13.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.239+0000 7fd2fe4c8640 1 --2- 172.21.15.86:0/1497753103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55781d4a1a80 0x55781d493090 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fd2f00092f0 tx=0x7fd2f0004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7bf9e00d33e373d1 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:13.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.239+0000 7fd2e37fe640 1 -- 172.21.15.86:0/1497753103 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f0023070 con 0x55781d4a1a80 2024-04-21T08:50:13.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.240+0000 7fd2e37fe640 1 -- 172.21.15.86:0/1497753103 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2f0028460 con 0x55781d4a1a80 2024-04-21T08:50:13.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.240+0000 7fd2e37fe640 1 -- 172.21.15.86:0/1497753103 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2f001f610 con 0x55781d4a1a80 2024-04-21T08:50:13.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.240+0000 7fd302bf9740 1 -- 172.21.15.86:0/1497753103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55781d4a1a80 msgr2=0x55781d493090 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.240+0000 7fd302bf9740 1 --2- 172.21.15.86:0/1497753103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55781d4a1a80 0x55781d493090 secure :-1 s=READY pgs=1962 cs=0 l=1 rev1=1 crypto rx=0x7fd2f00092f0 tx=0x7fd2f0004ab0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.240+0000 7fd302bf9740 1 -- 172.21.15.86:0/1497753103 shutdown_connections 2024-04-21T08:50:13.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.240+0000 7fd302bf9740 1 --2- 172.21.15.86:0/1497753103 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55781d4a1a80 0x55781d493090 unknown :-1 s=CLOSED pgs=1962 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.240+0000 7fd302bf9740 1 --2- 172.21.15.86:0/1497753103 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55781d498ea0 0x55781d4ad730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.240+0000 7fd302bf9740 1 --2- 172.21.15.86:0/1497753103 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d498580 0x55781d4a1200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.240+0000 7fd302bf9740 1 -- 172.21.15.86:0/1497753103 >> 172.21.15.86:0/1497753103 conn(0x55781d492cb0 msgr2=0x55781d4914e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:13.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.240+0000 7fd302bf9740 1 -- 172.21.15.86:0/1497753103 shutdown_connections 2024-04-21T08:50:13.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.240+0000 7fd302bf9740 1 -- 172.21.15.86:0/1497753103 wait complete. 2024-04-21T08:50:13.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.241+0000 7fd302bf9740 1 -- start start 2024-04-21T08:50:13.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.241+0000 7fd302bf9740 1 Processor -- start 2024-04-21T08:50:13.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.241+0000 7fd302bf9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55781d498580 0x55781d49bb40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.241+0000 7fd302bf9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d498ea0 0x55781d49c080 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.241+0000 7fd302bf9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55781d4a1a80 0x55781d517020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.241+0000 7fd302bf9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f81b0740 con 0x55781d498580 2024-04-21T08:50:13.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.241+0000 7fd302bf9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f81b0070 con 0x55781d4a1a80 2024-04-21T08:50:13.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.241+0000 7fd302bf9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd2f81ad2d0 con 0x55781d498ea0 2024-04-21T08:50:13.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.241+0000 7fd302bf9740 10 client.0 _ll_register_callbacks cb 0x7fd2e80035f0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:13.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.241+0000 7fd2fdcc7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55781d498580 0x55781d49bb40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.241+0000 7fd2fdcc7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55781d498580 0x55781d49bb40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55158/0 (socket says 172.21.15.86:55158) 2024-04-21T08:50:13.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.241+0000 7fd2fdcc7640 1 -- 172.21.15.86:0/1442440145 learned_addr learned my addr 172.21.15.86:0/1442440145 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:13.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.241+0000 7fd2fd4c6640 1 --2- 172.21.15.86:0/1442440145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d498ea0 0x55781d49c080 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.241+0000 7fd2fe4c8640 1 --2- 172.21.15.86:0/1442440145 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55781d4a1a80 0x55781d517020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.246 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.241+0000 7fd2fdcc7640 1 -- 172.21.15.86:0/1442440145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d498ea0 msgr2=0x55781d49c080 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:13.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.241+0000 7fd2fdcc7640 1 --2- 172.21.15.86:0/1442440145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d498ea0 0x55781d49c080 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.241+0000 7fd2fdcc7640 1 -- 172.21.15.86:0/1442440145 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55781d4a1a80 msgr2=0x55781d517020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:13.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.241+0000 7fd2fdcc7640 1 --2- 172.21.15.86:0/1442440145 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55781d4a1a80 0x55781d517020 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.241+0000 7fd2fdcc7640 1 -- 172.21.15.86:0/1442440145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55781d517560 con 0x55781d498580 2024-04-21T08:50:13.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.242+0000 7fd2fdcc7640 1 --2- 172.21.15.86:0/1442440145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55781d498580 0x55781d49bb40 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fd2e80bcc60 tx=0x7fd2e8009290 comp rx=0 tx=0).ready entity=mon.0 client_cookie=308a4f554e94644e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:13.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.242+0000 7fd2e17fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:13.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.242+0000 7fd2e17fa640 1 -- 172.21.15.86:0/1442440145 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2e80be310 con 0x55781d498580 2024-04-21T08:50:13.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.242+0000 7fd2e17fa640 1 -- 172.21.15.86:0/1442440145 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7fd2e80ddd70 con 0x55781d498580 2024-04-21T08:50:13.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.242+0000 7fd302bf9740 1 -- 172.21.15.86:0/1442440145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd2f0028460 con 0x55781d498580 2024-04-21T08:50:13.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.242+0000 7fd2e17fa640 1 -- 172.21.15.86:0/1442440145 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd2e80a5050 con 0x55781d498580 2024-04-21T08:50:13.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.242+0000 7fd302bf9740 10 client.20118 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:13.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.242+0000 7fd302bf9740 1 -- 172.21.15.86:0/1442440145 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55781d5186d0 con 0x55781d498580 2024-04-21T08:50:13.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.242+0000 7fd2e17fa640 1 -- 172.21.15.86:0/1442440145 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2f0028460 con 0x55781d498580 2024-04-21T08:50:13.247 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.242+0000 7fd2e17fa640 1 -- 172.21.15.86:0/1442440145 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd2e80bfce0 con 0x55781d498580 2024-04-21T08:50:13.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.242+0000 7fd302bf9740 10 client.20118 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:13.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.242+0000 7fd302bf9740 10 client.20118 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:13.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.242+0000 7fd302bf9740 20 client.20118 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:13.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.242+0000 7fd302bf9740 1 --2- 172.21.15.86:0/1442440145 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55781d518c60 0x55781d51b050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.242+0000 7fd302bf9740 4 client.20118 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:50:13.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.242+0000 7fd302bf9740 1 -- 172.21.15.86:0/1442440145 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55781d404a70 con 0x55781d518c60 2024-04-21T08:50:13.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.243+0000 7fd2fd4c6640 1 --2- 172.21.15.86:0/1442440145 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55781d518c60 0x55781d51b050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.243+0000 7fd2e17fa640 10 client.20118 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:13.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.243+0000 7fd2fd4c6640 1 --2- 172.21.15.86:0/1442440145 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55781d518c60 0x55781d51b050 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=14d444c2248b7d18 server_cookie=31bfdaed0e7d45d6 in_seq=0 out_seq=0 2024-04-21T08:50:13.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.243+0000 7fd2e17fa640 1 -- 172.21.15.86:0/1442440145 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55781d404a70 con 0x55781d518c60 2024-04-21T08:50:13.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.243+0000 7fd2e17fa640 10 client.20118 handle_command_reply: tid=0 2024-04-21T08:50:13.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.243+0000 7fd302bf9740 1 client.20118 shutdown 2024-04-21T08:50:13.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.243+0000 7fd302bf9740 1 -- 172.21.15.86:0/1442440145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55781d498580 msgr2=0x55781d49bb40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.243+0000 7fd302bf9740 1 --2- 172.21.15.86:0/1442440145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55781d498580 0x55781d49bb40 secure :-1 s=READY pgs=1963 cs=0 l=1 rev1=1 crypto rx=0x7fd2e80bcc60 tx=0x7fd2e8009290 comp rx=0 tx=0).stop 2024-04-21T08:50:13.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.243+0000 7fd302bf9740 1 -- 172.21.15.86:0/1442440145 shutdown_connections 2024-04-21T08:50:13.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.243+0000 7fd302bf9740 1 --2- 172.21.15.86:0/1442440145 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55781d518c60 0x55781d51b050 crc :-1 s=READY pgs=326 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.243+0000 7fd302bf9740 1 --2- 172.21.15.86:0/1442440145 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55781d4a1a80 0x55781d517020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.243+0000 7fd2e17fa640 0 client.20118 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:13.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.243+0000 7fd302bf9740 1 --2- 172.21.15.86:0/1442440145 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55781d498ea0 0x55781d49c080 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.243+0000 7fd302bf9740 1 --2- 172.21.15.86:0/1442440145 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55781d498580 0x55781d49bb40 unknown :-1 s=CLOSED pgs=1963 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.243+0000 7fd302bf9740 1 -- 172.21.15.86:0/1442440145 >> 172.21.15.86:0/1442440145 conn(0x55781d492cb0 msgr2=0x55781d491d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:13.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.243+0000 7fd302bf9740 1 -- 172.21.15.86:0/1442440145 shutdown_connections 2024-04-21T08:50:13.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.243+0000 7fd302bf9740 1 -- 172.21.15.86:0/1442440145 wait complete. 2024-04-21T08:50:13.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.243+0000 7fd302bf9740 20 client.20118 trim_cache size 0 max 16384 2024-04-21T08:50:13.249 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:50:13.250 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-21T08:50:13.250 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-21T08:50:13.250 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:50:13.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.244+0000 7fd2fff2b640 1 -- 172.21.15.86:0/1331348731 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd2e40a3da0 msgr2=0x7fd2e40a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.244+0000 7fd2fff2b640 1 --2- 172.21.15.86:0/1331348731 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd2e40a3da0 0x7fd2e40a6260 secure :-1 s=READY pgs=326 cs=0 l=1 rev1=1 crypto rx=0x7fd2f00098b0 tx=0x7fd2f0012a90 comp rx=0 tx=0).stop 2024-04-21T08:50:13.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.244+0000 7fd2fff2b640 1 -- 172.21.15.86:0/1331348731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2f8101090 msgr2=0x7fd2f8199540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.244+0000 7fd2fff2b640 1 --2- 172.21.15.86:0/1331348731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2f8101090 0x7fd2f8199540 secure :-1 s=READY pgs=1959 cs=0 l=1 rev1=1 crypto rx=0x7fd2e8005940 tx=0x7fd2e802c7e0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.244+0000 7fd2fff2b640 1 -- 172.21.15.86:0/1331348731 shutdown_connections 2024-04-21T08:50:13.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.244+0000 7fd2fff2b640 1 --2- 172.21.15.86:0/1331348731 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd2e40a3da0 0x7fd2e40a6260 unknown :-1 s=CLOSED pgs=326 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.244+0000 7fd2fff2b640 1 --2- 172.21.15.86:0/1331348731 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd2f819a010 0x7fd2f81a45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.244+0000 7fd2fff2b640 1 --2- 172.21.15.86:0/1331348731 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd2f8106160 0x7fd2f8199a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.244+0000 7fd2fff2b640 1 --2- 172.21.15.86:0/1331348731 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd2f8101090 0x7fd2f8199540 unknown :-1 s=CLOSED pgs=1959 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.244+0000 7fd2fff2b640 1 -- 172.21.15.86:0/1331348731 >> 172.21.15.86:0/1331348731 conn(0x7fd2f80771d0 msgr2=0x7fd2f80fed70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:13.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.244+0000 7fd2fff2b640 1 -- 172.21.15.86:0/1331348731 shutdown_connections 2024-04-21T08:50:13.251 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.244+0000 7fd2fff2b640 1 -- 172.21.15.86:0/1331348731 wait complete. 2024-04-21T08:50:13.254 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:50:13.254 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:50:13.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.352+0000 7fca8570e640 1 -- 172.21.15.86:0/699109574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca80106180 msgr2=0x7fca8010dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.352+0000 7fca8570e640 1 --2- 172.21.15.86:0/699109574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca80106180 0x7fca8010dd50 secure :-1 s=READY pgs=1964 cs=0 l=1 rev1=1 crypto rx=0x7fca78009ca0 tx=0x7fca7802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.352+0000 7fca8570e640 1 -- 172.21.15.86:0/699109574 shutdown_connections 2024-04-21T08:50:13.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.352+0000 7fca8570e640 1 --2- 172.21.15.86:0/699109574 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca80106180 0x7fca8010dd50 unknown :-1 s=CLOSED pgs=1964 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.354 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.352+0000 7fca8570e640 1 --2- 172.21.15.86:0/699109574 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca80101a60 0x7fca80105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.352+0000 7fca8570e640 1 --2- 172.21.15.86:0/699109574 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca801010b0 0x7fca80101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.352+0000 7fca8570e640 1 -- 172.21.15.86:0/699109574 >> 172.21.15.86:0/699109574 conn(0x7fca800771d0 msgr2=0x7fca800775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:13.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.352+0000 7fca8570e640 1 -- 172.21.15.86:0/699109574 shutdown_connections 2024-04-21T08:50:13.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.352+0000 7fca8570e640 1 -- 172.21.15.86:0/699109574 wait complete. 2024-04-21T08:50:13.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.353+0000 7fca8570e640 1 Processor -- start 2024-04-21T08:50:13.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.353+0000 7fca8570e640 1 -- start start 2024-04-21T08:50:13.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.353+0000 7fca8570e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca801010b0 0x7fca80198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.353+0000 7fca8570e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca80106180 0x7fca80198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.353+0000 7fca8570e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca8019cfb0 0x7fca8019d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.353+0000 7fca8570e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fca80112200 con 0x7fca8019cfb0 2024-04-21T08:50:13.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.353+0000 7fca8570e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fca80112080 con 0x7fca80106180 2024-04-21T08:50:13.355 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.353+0000 7fca8570e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fca80112380 con 0x7fca801010b0 2024-04-21T08:50:13.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.353+0000 7fca7f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca8019cfb0 0x7fca8019d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.353+0000 7fca7f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca8019cfb0 0x7fca8019d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55184/0 (socket says 172.21.15.86:55184) 2024-04-21T08:50:13.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.353+0000 7fca7f7fe640 1 -- 172.21.15.86:0/1274474318 learned_addr learned my addr 172.21.15.86:0/1274474318 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:13.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.354+0000 7fca7effd640 1 --2- 172.21.15.86:0/1274474318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca801010b0 0x7fca80198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.354+0000 7fca7e7fc640 1 --2- 172.21.15.86:0/1274474318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca80106180 0x7fca80198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.354+0000 7fca7f7fe640 1 -- 172.21.15.86:0/1274474318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca801010b0 msgr2=0x7fca80198740 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.354+0000 7fca7f7fe640 1 --2- 172.21.15.86:0/1274474318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca801010b0 0x7fca80198740 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.354+0000 7fca7f7fe640 1 -- 172.21.15.86:0/1274474318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca80106180 msgr2=0x7fca80198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.354+0000 7fca7f7fe640 1 --2- 172.21.15.86:0/1274474318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca80106180 0x7fca80198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.354+0000 7fca7f7fe640 1 -- 172.21.15.86:0/1274474318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fca8019daa0 con 0x7fca8019cfb0 2024-04-21T08:50:13.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.354+0000 7fca7f7fe640 1 --2- 172.21.15.86:0/1274474318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca8019cfb0 0x7fca8019d460 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fca78005e30 tx=0x7fca780346d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:13.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.354+0000 7fca67fff640 1 -- 172.21.15.86:0/1274474318 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca78041020 con 0x7fca8019cfb0 2024-04-21T08:50:13.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.354+0000 7fca8570e640 1 -- 172.21.15.86:0/1274474318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fca8019dd30 con 0x7fca8019cfb0 2024-04-21T08:50:13.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.354+0000 7fca67fff640 1 -- 172.21.15.86:0/1274474318 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fca78032970 con 0x7fca8019cfb0 2024-04-21T08:50:13.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.354+0000 7fca67fff640 1 -- 172.21.15.86:0/1274474318 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fca78039040 con 0x7fca8019cfb0 2024-04-21T08:50:13.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.354+0000 7fca8570e640 1 -- 172.21.15.86:0/1274474318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fca801a5550 con 0x7fca8019cfb0 2024-04-21T08:50:13.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.355+0000 7fca8570e640 1 -- 172.21.15.86:0/1274474318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fca80069230 con 0x7fca8019cfb0 2024-04-21T08:50:13.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.356+0000 7fca67fff640 1 -- 172.21.15.86:0/1274474318 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fca78044020 con 0x7fca8019cfb0 2024-04-21T08:50:13.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.357+0000 7fca67fff640 1 --2- 172.21.15.86:0/1274474318 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fca680a3d20 0x7fca680a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.357+0000 7fca7effd640 1 --2- 172.21.15.86:0/1274474318 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fca680a3d20 0x7fca680a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.357+0000 7fca67fff640 1 -- 172.21.15.86:0/1274474318 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7fca780f2740 con 0x7fca8019cfb0 2024-04-21T08:50:13.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.357+0000 7fca7effd640 1 --2- 172.21.15.86:0/1274474318 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fca680a3d20 0x7fca680a61e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fca70009bd0 tx=0x7fca70005f90 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:13.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.361+0000 7fca67fff640 1 -- 172.21.15.86:0/1274474318 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fca7804b020 con 0x7fca8019cfb0 2024-04-21T08:50:13.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.580+0000 7fca8570e640 1 -- 172.21.15.86:0/1274474318 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fca80109470 con 0x7fca8019cfb0 2024-04-21T08:50:13.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.581+0000 7fca67fff640 1 -- 172.21.15.86:0/1274474318 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7fca780beb20 con 0x7fca8019cfb0 2024-04-21T08:50:13.581 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:50:13.581 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:38:45.202126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24398,24394,24455,24472]},"id":1}]} 2024-04-21T08:50:13.581 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-21T08:50:13.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.583+0000 7fca8570e640 1 -- 172.21.15.86:0/1274474318 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fca680a3d20 msgr2=0x7fca680a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.583+0000 7fca8570e640 1 --2- 172.21.15.86:0/1274474318 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fca680a3d20 0x7fca680a61e0 secure :-1 s=READY pgs=327 cs=0 l=1 rev1=1 crypto rx=0x7fca70009bd0 tx=0x7fca70005f90 comp rx=0 tx=0).stop 2024-04-21T08:50:13.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.583+0000 7fca8570e640 1 -- 172.21.15.86:0/1274474318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca8019cfb0 msgr2=0x7fca8019d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.583+0000 7fca8570e640 1 --2- 172.21.15.86:0/1274474318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca8019cfb0 0x7fca8019d460 secure :-1 s=READY pgs=1965 cs=0 l=1 rev1=1 crypto rx=0x7fca78005e30 tx=0x7fca780346d0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.583+0000 7fca8570e640 1 -- 172.21.15.86:0/1274474318 shutdown_connections 2024-04-21T08:50:13.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.583+0000 7fca8570e640 1 --2- 172.21.15.86:0/1274474318 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fca680a3d20 0x7fca680a61e0 unknown :-1 s=CLOSED pgs=327 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.583+0000 7fca8570e640 1 --2- 172.21.15.86:0/1274474318 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fca8019cfb0 0x7fca8019d460 unknown :-1 s=CLOSED pgs=1965 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.583+0000 7fca8570e640 1 --2- 172.21.15.86:0/1274474318 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fca80106180 0x7fca80198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.583+0000 7fca8570e640 1 --2- 172.21.15.86:0/1274474318 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fca801010b0 0x7fca80198740 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.583+0000 7fca8570e640 1 -- 172.21.15.86:0/1274474318 >> 172.21.15.86:0/1274474318 conn(0x7fca800771d0 msgr2=0x7fca800fed50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:13.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.583+0000 7fca8570e640 1 -- 172.21.15.86:0/1274474318 shutdown_connections 2024-04-21T08:50:13.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.583+0000 7fca8570e640 1 -- 172.21.15.86:0/1274474318 wait complete. 2024-04-21T08:50:13.594 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:50:13.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.694+0000 7f0ac46e4640 1 -- 172.21.15.86:0/4003424791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0abc107270 msgr2=0x7f0abc10ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.694+0000 7f0ac46e4640 1 --2- 172.21.15.86:0/4003424791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0abc107270 0x7f0abc10ee40 secure :-1 s=READY pgs=1966 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4009ca0 tx=0x7f0ab402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.694+0000 7f0ac46e4640 1 -- 172.21.15.86:0/4003424791 shutdown_connections 2024-04-21T08:50:13.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.694+0000 7f0ac46e4640 1 --2- 172.21.15.86:0/4003424791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0abc107270 0x7f0abc10ee40 unknown :-1 s=CLOSED pgs=1966 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.694+0000 7f0ac46e4640 1 --2- 172.21.15.86:0/4003424791 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0abc06b1f0 0x7f0abc106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.694+0000 7f0ac46e4640 1 --2- 172.21.15.86:0/4003424791 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0abc06a840 0x7f0abc06ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.694+0000 7f0ac46e4640 1 -- 172.21.15.86:0/4003424791 >> 172.21.15.86:0/4003424791 conn(0x7f0abc077eb0 msgr2=0x7f0abc0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:13.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.694+0000 7f0ac46e4640 1 -- 172.21.15.86:0/4003424791 shutdown_connections 2024-04-21T08:50:13.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.694+0000 7f0ac46e4640 1 -- 172.21.15.86:0/4003424791 wait complete. 2024-04-21T08:50:13.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.695+0000 7f0ac46e4640 1 Processor -- start 2024-04-21T08:50:13.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.695+0000 7f0ac46e4640 1 -- start start 2024-04-21T08:50:13.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.695+0000 7f0ac46e4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0abc06a840 0x7f0abc1986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.695+0000 7f0ac46e4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0abc107270 0x7f0abc198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.695+0000 7f0ac46e4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0abc19cf70 0x7f0abc19d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.695+0000 7f0ac46e4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc1121d0 con 0x7f0abc06a840 2024-04-21T08:50:13.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.695+0000 7f0ac46e4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc112050 con 0x7f0abc107270 2024-04-21T08:50:13.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.695+0000 7f0ac46e4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc112350 con 0x7f0abc19cf70 2024-04-21T08:50:13.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.695+0000 7f0ac2480640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0abc06a840 0x7f0abc1986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.695+0000 7f0ac2480640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0abc06a840 0x7f0abc1986b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55212/0 (socket says 172.21.15.86:55212) 2024-04-21T08:50:13.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.695+0000 7f0ac2480640 1 -- 172.21.15.86:0/1211578378 learned_addr learned my addr 172.21.15.86:0/1211578378 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:13.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.695+0000 7f0ac2c81640 1 --2- 172.21.15.86:0/1211578378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0abc19cf70 0x7f0abc19d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.695+0000 7f0ac2480640 1 -- 172.21.15.86:0/1211578378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0abc19cf70 msgr2=0x7f0abc19d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.695+0000 7f0ac2480640 1 --2- 172.21.15.86:0/1211578378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0abc19cf70 0x7f0abc19d420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.695+0000 7f0ac1c7f640 1 --2- 172.21.15.86:0/1211578378 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0abc107270 0x7f0abc198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.695+0000 7f0ac2480640 1 -- 172.21.15.86:0/1211578378 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0abc107270 msgr2=0x7f0abc198bf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.695+0000 7f0ac2480640 1 --2- 172.21.15.86:0/1211578378 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0abc107270 0x7f0abc198bf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.695+0000 7f0ac2480640 1 -- 172.21.15.86:0/1211578378 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f0abc19da60 con 0x7f0abc06a840 2024-04-21T08:50:13.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.695+0000 7f0ac1c7f640 1 --2- 172.21.15.86:0/1211578378 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0abc107270 0x7f0abc198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:13.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.695+0000 7f0ac2480640 1 --2- 172.21.15.86:0/1211578378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0abc06a840 0x7f0abc1986b0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000d770 tx=0x7f0ab000dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:13.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.695+0000 7f0aaf7fe640 1 -- 172.21.15.86:0/1211578378 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0017070 con 0x7f0abc06a840 2024-04-21T08:50:13.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.695+0000 7f0aaf7fe640 1 -- 172.21.15.86:0/1211578378 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ab00054d0 con 0x7f0abc06a840 2024-04-21T08:50:13.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.695+0000 7f0ac46e4640 1 -- 172.21.15.86:0/1211578378 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f0abc19dd50 con 0x7f0abc06a840 2024-04-21T08:50:13.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.696+0000 7f0aaf7fe640 1 -- 172.21.15.86:0/1211578378 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab0004c30 con 0x7f0abc06a840 2024-04-21T08:50:13.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.696+0000 7f0ac46e4640 1 -- 172.21.15.86:0/1211578378 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f0abc1a5540 con 0x7f0abc06a840 2024-04-21T08:50:13.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.698+0000 7f0aaf7fe640 1 -- 172.21.15.86:0/1211578378 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f0ab0004e10 con 0x7f0abc06a840 2024-04-21T08:50:13.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.698+0000 7f0ac73b2740 1 Processor -- start 2024-04-21T08:50:13.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.698+0000 7f0ac73b2740 1 -- start start 2024-04-21T08:50:13.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.698+0000 7f0ac73b2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558999f764d0 0x558999f768b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.698+0000 7f0ac73b2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558999f76e80 0x558999f7f430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.698+0000 7f0ac73b2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558999f7fa00 0x558999f81e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.698+0000 7f0ac73b2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558999edb1f0 con 0x558999f764d0 2024-04-21T08:50:13.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.698+0000 7f0ac73b2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x558999e784b0 con 0x558999f7fa00 2024-04-21T08:50:13.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.698+0000 7f0ac73b2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558999ed71b0 con 0x558999f76e80 2024-04-21T08:50:13.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.698+0000 7f0ac2c81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558999f764d0 0x558999f768b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.699+0000 7f0ac2c81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558999f764d0 0x558999f768b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55218/0 (socket says 172.21.15.86:55218) 2024-04-21T08:50:13.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.699+0000 7f0ac2c81640 1 -- 172.21.15.86:0/3823225561 learned_addr learned my addr 172.21.15.86:0/3823225561 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:13.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.699+0000 7f0aaf7fe640 1 --2- 172.21.15.86:0/1211578378 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0aa80a3da0 0x7f0aa80a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.699+0000 7f0ac1c7f640 1 --2- 172.21.15.86:0/3823225561 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558999f7fa00 0x558999f81e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.699+0000 7f0ac2480640 1 --2- 172.21.15.86:0/3823225561 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558999f76e80 0x558999f7f430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.699+0000 7f0ac2c81640 1 --2- 172.21.15.86:0/1211578378 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0aa80a3da0 0x7f0aa80a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.699+0000 7f0ac2c81640 1 -- 172.21.15.86:0/3823225561 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558999f76e80 msgr2=0x558999f7f430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:13.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.699+0000 7f0ac2c81640 1 --2- 172.21.15.86:0/3823225561 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558999f76e80 0x558999f7f430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.699+0000 7f0ac2c81640 1 -- 172.21.15.86:0/3823225561 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558999f7fa00 msgr2=0x558999f81e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:13.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.699+0000 7f0ac2c81640 1 --2- 172.21.15.86:0/3823225561 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558999f7fa00 0x558999f81e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.699+0000 7f0ac2c81640 1 -- 172.21.15.86:0/3823225561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558999e6a1b0 con 0x558999f764d0 2024-04-21T08:50:13.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.699+0000 7f0aaf7fe640 1 -- 172.21.15.86:0/1211578378 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f0ab00d06d0 con 0x7f0abc06a840 2024-04-21T08:50:13.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.699+0000 7f0ac2c81640 1 --2- 172.21.15.86:0/3823225561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558999f764d0 0x558999f768b0 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4004af0 tx=0x7f0ab402cf70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c3a761aeb561e11f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:13.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.699+0000 7f0aacff9640 1 -- 172.21.15.86:0/3823225561 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4039040 con 0x558999f764d0 2024-04-21T08:50:13.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.699+0000 7f0aacff9640 1 -- 172.21.15.86:0/3823225561 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ab4032950 con 0x558999f764d0 2024-04-21T08:50:13.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.699+0000 7f0aacff9640 1 -- 172.21.15.86:0/3823225561 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4034440 con 0x558999f764d0 2024-04-21T08:50:13.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.700+0000 7f0ac73b2740 1 -- 172.21.15.86:0/3823225561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558999f764d0 msgr2=0x558999f768b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.700+0000 7f0ac2c81640 1 --2- 172.21.15.86:0/1211578378 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0aa80a3da0 0x7f0aa80a6260 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0ab404c000 tx=0x7f0ab4048000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:13.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.700+0000 7f0ac73b2740 1 --2- 172.21.15.86:0/3823225561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558999f764d0 0x558999f768b0 secure :-1 s=READY pgs=1968 cs=0 l=1 rev1=1 crypto rx=0x7f0ab4004af0 tx=0x7f0ab402cf70 comp rx=0 tx=0).stop 2024-04-21T08:50:13.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.700+0000 7f0ac73b2740 1 -- 172.21.15.86:0/3823225561 shutdown_connections 2024-04-21T08:50:13.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.700+0000 7f0ac73b2740 1 --2- 172.21.15.86:0/3823225561 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558999f7fa00 0x558999f81e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.700+0000 7f0ac73b2740 1 --2- 172.21.15.86:0/3823225561 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558999f76e80 0x558999f7f430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.700+0000 7f0ac73b2740 1 --2- 172.21.15.86:0/3823225561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558999f764d0 0x558999f768b0 unknown :-1 s=CLOSED pgs=1968 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.700+0000 7f0ac73b2740 1 -- 172.21.15.86:0/3823225561 >> 172.21.15.86:0/3823225561 conn(0x558999f70ba0 msgr2=0x558999f70f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:13.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.700+0000 7f0ac73b2740 1 -- 172.21.15.86:0/3823225561 shutdown_connections 2024-04-21T08:50:13.708 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.700+0000 7f0ac73b2740 1 -- 172.21.15.86:0/3823225561 wait complete. 2024-04-21T08:50:13.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.700+0000 7f0ac73b2740 1 -- start start 2024-04-21T08:50:13.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.700+0000 7f0ac73b2740 1 Processor -- start 2024-04-21T08:50:13.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.701+0000 7f0ac73b2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558999f764d0 0x558999f7f2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.701+0000 7f0ac73b2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558999f76e80 0x558999f7c650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.701+0000 7f0ac73b2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558999f7fa00 0x558999f7cbb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.701+0000 7f0ac73b2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558999f82970 con 0x558999f76e80 2024-04-21T08:50:13.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.701+0000 7f0ac73b2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x558999e94500 con 0x558999f7fa00 2024-04-21T08:50:13.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.701+0000 7f0ac73b2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558999f82af0 con 0x558999f764d0 2024-04-21T08:50:13.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.701+0000 7f0ac73b2740 10 client.0 _ll_register_callbacks cb 0x558999e62b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:13.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.701+0000 7f0ac1c7f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558999f76e80 0x558999f7c650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.701+0000 7f0ac1c7f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558999f76e80 0x558999f7c650 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55222/0 (socket says 172.21.15.86:55222) 2024-04-21T08:50:13.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.701+0000 7f0ac1c7f640 1 -- 172.21.15.86:0/1190681771 learned_addr learned my addr 172.21.15.86:0/1190681771 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:13.709 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.701+0000 7f0ac2480640 1 --2- 172.21.15.86:0/1190681771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558999f764d0 0x558999f7f2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.701+0000 7f0ac1c7f640 1 -- 172.21.15.86:0/1190681771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558999f764d0 msgr2=0x558999f7f2c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:13.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.701+0000 7f0ac1c7f640 1 --2- 172.21.15.86:0/1190681771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558999f764d0 0x558999f7f2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.701+0000 7f0ac1c7f640 1 -- 172.21.15.86:0/1190681771 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558999f7fa00 msgr2=0x558999f7cbb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:13.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.701+0000 7f0ac1c7f640 1 --2- 172.21.15.86:0/1190681771 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558999f7fa00 0x558999f7cbb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.701+0000 7f0ac1c7f640 1 -- 172.21.15.86:0/1190681771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558999bed3f0 con 0x558999f76e80 2024-04-21T08:50:13.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.701+0000 7f0ac1c7f640 1 --2- 172.21.15.86:0/1190681771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558999f76e80 0x558999f7c650 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8009f10 tx=0x7f0ab80117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=19b5a405e170d7c2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:13.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.701+0000 7f0aa5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:13.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.701+0000 7f0aa5ffb640 1 -- 172.21.15.86:0/1190681771 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8012860 con 0x558999f76e80 2024-04-21T08:50:13.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.701+0000 7f0aa5ffb640 1 -- 172.21.15.86:0/1190681771 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f0ab8012a00 con 0x558999f76e80 2024-04-21T08:50:13.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.701+0000 7f0ac73b2740 1 -- 172.21.15.86:0/1190681771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ab4032950 con 0x558999f76e80 2024-04-21T08:50:13.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.701+0000 7f0aa5ffb640 1 -- 172.21.15.86:0/1190681771 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0ab8025050 con 0x558999f76e80 2024-04-21T08:50:13.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.701+0000 7f0ac73b2740 10 client.20136 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:13.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.701+0000 7f0ac73b2740 1 -- 172.21.15.86:0/1190681771 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558999cb77d0 con 0x558999f76e80 2024-04-21T08:50:13.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.702+0000 7f0aa5ffb640 1 -- 172.21.15.86:0/1190681771 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ab4032950 con 0x558999f76e80 2024-04-21T08:50:13.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.702+0000 7f0aa5ffb640 1 -- 172.21.15.86:0/1190681771 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0ab800f040 con 0x558999f76e80 2024-04-21T08:50:13.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.702+0000 7f0ac73b2740 10 client.20136 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:13.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.702+0000 7f0ac73b2740 10 client.20136 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:50:13.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.702+0000 7f0ac73b2740 20 client.20136 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:13.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.702+0000 7f0ac73b2740 1 --2- 172.21.15.86:0/1190681771 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x558999f8fd40 0x558999f921e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.702+0000 7f0ac73b2740 4 client.20136 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:13.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.702+0000 7f0ac73b2740 1 -- 172.21.15.86:0/1190681771 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558999e7a930 con 0x558999f8fd40 2024-04-21T08:50:13.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.702+0000 7f0ac2480640 1 --2- 172.21.15.86:0/1190681771 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x558999f8fd40 0x558999f921e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.702+0000 7f0ac2480640 1 --2- 172.21.15.86:0/1190681771 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x558999f8fd40 0x558999f921e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=af63c6d04196a938 server_cookie=fabd6ec36a73e258 in_seq=0 out_seq=0 2024-04-21T08:50:13.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.702+0000 7f0aa5ffb640 10 client.20136 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:50:13.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.703+0000 7f0aa5ffb640 1 -- 172.21.15.86:0/1190681771 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558999e7a930 con 0x558999f8fd40 2024-04-21T08:50:13.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.703+0000 7f0aa5ffb640 10 client.20136 handle_command_reply: tid=0 2024-04-21T08:50:13.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.703+0000 7f0ac73b2740 1 client.20136 shutdown 2024-04-21T08:50:13.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.703+0000 7f0ac73b2740 1 -- 172.21.15.86:0/1190681771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558999f76e80 msgr2=0x558999f7c650 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.703+0000 7f0ac73b2740 1 --2- 172.21.15.86:0/1190681771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558999f76e80 0x558999f7c650 secure :-1 s=READY pgs=1969 cs=0 l=1 rev1=1 crypto rx=0x7f0ab8009f10 tx=0x7f0ab80117b0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.703+0000 7f0ac73b2740 1 -- 172.21.15.86:0/1190681771 shutdown_connections 2024-04-21T08:50:13.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.703+0000 7f0ac73b2740 1 --2- 172.21.15.86:0/1190681771 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x558999f8fd40 0x558999f921e0 crc :-1 s=READY pgs=74 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.703+0000 7f0ac73b2740 1 --2- 172.21.15.86:0/1190681771 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558999f7fa00 0x558999f7cbb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.703+0000 7f0ac73b2740 1 --2- 172.21.15.86:0/1190681771 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558999f76e80 0x558999f7c650 unknown :-1 s=CLOSED pgs=1969 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.703+0000 7f0aa5ffb640 0 client.20136 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:50:13.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.703+0000 7f0ac73b2740 1 --2- 172.21.15.86:0/1190681771 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558999f764d0 0x558999f7f2c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.703+0000 7f0ac73b2740 1 -- 172.21.15.86:0/1190681771 >> 172.21.15.86:0/1190681771 conn(0x558999f70ba0 msgr2=0x558999f728f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:13.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.703+0000 7f0ac73b2740 1 -- 172.21.15.86:0/1190681771 shutdown_connections 2024-04-21T08:50:13.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.703+0000 7f0ac73b2740 1 -- 172.21.15.86:0/1190681771 wait complete. 2024-04-21T08:50:13.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.704+0000 7f0ac73b2740 20 client.20136 trim_cache size 0 max 16384 2024-04-21T08:50:13.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.716+0000 7f0ac73b2740 1 Processor -- start 2024-04-21T08:50:13.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.716+0000 7f0ac73b2740 1 -- start start 2024-04-21T08:50:13.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.717+0000 7f0ac73b2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558999f764d0 0x558999f8b830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.717+0000 7f0ac73b2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558999f76e80 0x558999f6fa80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.717+0000 7f0ac73b2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558999f7fa00 0x558999f7ed80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.717+0000 7f0ac73b2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc1af3c0 con 0x558999f76e80 2024-04-21T08:50:13.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.717+0000 7f0ac73b2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc1b0f00 con 0x558999f7fa00 2024-04-21T08:50:13.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.717+0000 7f0ac73b2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc1afa90 con 0x558999f764d0 2024-04-21T08:50:13.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.717+0000 7f0ac1c7f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558999f76e80 0x558999f6fa80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.717+0000 7f0ac1c7f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558999f76e80 0x558999f6fa80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55236/0 (socket says 172.21.15.86:55236) 2024-04-21T08:50:13.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.717+0000 7f0ac1c7f640 1 -- 172.21.15.86:0/2662733422 learned_addr learned my addr 172.21.15.86:0/2662733422 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:13.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.717+0000 7f0ac2480640 1 --2- 172.21.15.86:0/2662733422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558999f764d0 0x558999f8b830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.717+0000 7f0ac1c7f640 1 -- 172.21.15.86:0/2662733422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558999f764d0 msgr2=0x558999f8b830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:13.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.717+0000 7f0ac1c7f640 1 --2- 172.21.15.86:0/2662733422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558999f764d0 0x558999f8b830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.717+0000 7f0ac1c7f640 1 -- 172.21.15.86:0/2662733422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558999f7fa00 msgr2=0x558999f7ed80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:13.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.717+0000 7f0ac2c81640 1 --2- 172.21.15.86:0/2662733422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558999f7fa00 0x558999f7ed80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.717+0000 7f0ac1c7f640 1 --2- 172.21.15.86:0/2662733422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558999f7fa00 0x558999f7ed80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.717+0000 7f0ac1c7f640 1 -- 172.21.15.86:0/2662733422 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558999f8e820 con 0x558999f76e80 2024-04-21T08:50:13.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.717+0000 7f0ac1c7f640 1 --2- 172.21.15.86:0/2662733422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558999f76e80 0x558999f6fa80 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f0ab80193a0 tx=0x7f0ab80097c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=575dca67bfb11a47 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:13.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.717+0000 7f0aa7fff640 1 -- 172.21.15.86:0/2662733422 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8019620 con 0x558999f76e80 2024-04-21T08:50:13.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.717+0000 7f0aa7fff640 1 -- 172.21.15.86:0/2662733422 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ab802d070 con 0x558999f76e80 2024-04-21T08:50:13.723 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.717+0000 7f0aa7fff640 1 -- 172.21.15.86:0/2662733422 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab8012df0 con 0x558999f76e80 2024-04-21T08:50:13.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.718+0000 7f0ac73b2740 1 -- 172.21.15.86:0/2662733422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558999f76e80 msgr2=0x558999f6fa80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.718+0000 7f0ac73b2740 1 --2- 172.21.15.86:0/2662733422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558999f76e80 0x558999f6fa80 secure :-1 s=READY pgs=1970 cs=0 l=1 rev1=1 crypto rx=0x7f0ab80193a0 tx=0x7f0ab80097c0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.718+0000 7f0ac73b2740 1 -- 172.21.15.86:0/2662733422 shutdown_connections 2024-04-21T08:50:13.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.718+0000 7f0ac73b2740 1 --2- 172.21.15.86:0/2662733422 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558999f7fa00 0x558999f7ed80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.718+0000 7f0ac73b2740 1 --2- 172.21.15.86:0/2662733422 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558999f76e80 0x558999f6fa80 unknown :-1 s=CLOSED pgs=1970 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.718+0000 7f0ac73b2740 1 --2- 172.21.15.86:0/2662733422 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558999f764d0 0x558999f8b830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.718+0000 7f0ac73b2740 1 -- 172.21.15.86:0/2662733422 >> 172.21.15.86:0/2662733422 conn(0x558999f70ba0 msgr2=0x558999f6f370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:13.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.718+0000 7f0ac73b2740 1 -- 172.21.15.86:0/2662733422 shutdown_connections 2024-04-21T08:50:13.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.718+0000 7f0ac73b2740 1 -- 172.21.15.86:0/2662733422 wait complete. 2024-04-21T08:50:13.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.718+0000 7f0ac73b2740 1 -- start start 2024-04-21T08:50:13.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.718+0000 7f0ac73b2740 1 Processor -- start 2024-04-21T08:50:13.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.718+0000 7f0ac73b2740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558999f764d0 0x558999f79190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.718+0000 7f0ac73b2740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558999f76e80 0x558999f796d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.724 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.718+0000 7f0ac73b2740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558999f7fa00 0x558999f79c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.718+0000 7f0ac73b2740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc1b0830 con 0x558999f7fa00 2024-04-21T08:50:13.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.718+0000 7f0ac73b2740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc1b0160 con 0x558999f764d0 2024-04-21T08:50:13.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.718+0000 7f0ac73b2740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f0abc1ad3c0 con 0x558999f76e80 2024-04-21T08:50:13.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.718+0000 7f0ac73b2740 10 client.0 _ll_register_callbacks cb 0x7f0ab8009b20 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:13.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.719+0000 7f0ac2c81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558999f7fa00 0x558999f79c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.719+0000 7f0ac2c81640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558999f7fa00 0x558999f79c10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55242/0 (socket says 172.21.15.86:55242) 2024-04-21T08:50:13.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.719+0000 7f0ac2c81640 1 -- 172.21.15.86:0/3589129940 learned_addr learned my addr 172.21.15.86:0/3589129940 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:13.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.719+0000 7f0ac2480640 1 --2- 172.21.15.86:0/3589129940 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558999f764d0 0x558999f79190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.719+0000 7f0ac1c7f640 1 --2- 172.21.15.86:0/3589129940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558999f76e80 0x558999f796d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.719+0000 7f0ac2c81640 1 -- 172.21.15.86:0/3589129940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558999f76e80 msgr2=0x558999f796d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:13.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.719+0000 7f0ac2c81640 1 --2- 172.21.15.86:0/3589129940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558999f76e80 0x558999f796d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.719+0000 7f0ac2c81640 1 -- 172.21.15.86:0/3589129940 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558999f764d0 msgr2=0x558999f79190 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:13.725 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.719+0000 7f0ac2c81640 1 --2- 172.21.15.86:0/3589129940 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558999f764d0 0x558999f79190 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.719+0000 7f0ac2c81640 1 -- 172.21.15.86:0/3589129940 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558999ff5320 con 0x558999f7fa00 2024-04-21T08:50:13.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.719+0000 7f0ac2480640 1 --2- 172.21.15.86:0/3589129940 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558999f764d0 0x558999f79190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:13.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.719+0000 7f0ac2c81640 1 --2- 172.21.15.86:0/3589129940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558999f7fa00 0x558999f79c10 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x558999f70510 tx=0x7f0ab4005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3b0346dd3c36945c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:13.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.719+0000 7f0aa5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:13.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.719+0000 7f0aa5ffb640 1 -- 172.21.15.86:0/3589129940 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f0ab4058070 con 0x558999f7fa00 2024-04-21T08:50:13.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.719+0000 7f0aa5ffb640 1 -- 172.21.15.86:0/3589129940 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f0ab4003b70 con 0x558999f7fa00 2024-04-21T08:50:13.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.719+0000 7f0ac73b2740 1 -- 172.21.15.86:0/3589129940 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f0ab802d070 con 0x558999f7fa00 2024-04-21T08:50:13.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.719+0000 7f0aa5ffb640 1 -- 172.21.15.86:0/3589129940 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f0ab4064050 con 0x558999f7fa00 2024-04-21T08:50:13.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.719+0000 7f0ac73b2740 10 client.20142 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:13.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.719+0000 7f0ac73b2740 1 -- 172.21.15.86:0/3589129940 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558999ff6500 con 0x558999f7fa00 2024-04-21T08:50:13.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.719+0000 7f0aa5ffb640 1 -- 172.21.15.86:0/3589129940 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f0ab802d070 con 0x558999f7fa00 2024-04-21T08:50:13.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.720+0000 7f0aa5ffb640 1 -- 172.21.15.86:0/3589129940 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f0ab4034bc0 con 0x558999f7fa00 2024-04-21T08:50:13.726 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.720+0000 7f0ac73b2740 10 client.20142 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:13.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.720+0000 7f0ac73b2740 10 client.20142 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:50:13.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.720+0000 7f0ac73b2740 20 client.20142 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:13.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.720+0000 7f0ac73b2740 1 --2- 172.21.15.86:0/3589129940 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x558999ff6870 0x558999ff8c60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.720+0000 7f0ac73b2740 4 client.20142 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:50:13.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.720+0000 7f0ac73b2740 1 -- 172.21.15.86:0/3589129940 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x558999ee8c10 con 0x558999ff6870 2024-04-21T08:50:13.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.720+0000 7f0ac2480640 1 --2- 172.21.15.86:0/3589129940 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x558999ff6870 0x558999ff8c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.720+0000 7f0ac2480640 1 --2- 172.21.15.86:0/3589129940 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x558999ff6870 0x558999ff8c60 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1b3c7c50d319c001 server_cookie=2026d4e42a2ada2f in_seq=0 out_seq=0 2024-04-21T08:50:13.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.720+0000 7f0aa5ffb640 10 client.20142 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:50:13.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.720+0000 7f0aa5ffb640 1 -- 172.21.15.86:0/3589129940 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x558999ee8c10 con 0x558999ff6870 2024-04-21T08:50:13.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.720+0000 7f0aa5ffb640 10 client.20142 handle_command_reply: tid=0 2024-04-21T08:50:13.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.720+0000 7f0ac73b2740 1 client.20142 shutdown 2024-04-21T08:50:13.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.720+0000 7f0ac73b2740 1 -- 172.21.15.86:0/3589129940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558999f7fa00 msgr2=0x558999f79c10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.727 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.720+0000 7f0ac73b2740 1 --2- 172.21.15.86:0/3589129940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558999f7fa00 0x558999f79c10 secure :-1 s=READY pgs=1971 cs=0 l=1 rev1=1 crypto rx=0x558999f70510 tx=0x7f0ab4005d80 comp rx=0 tx=0).stop 2024-04-21T08:50:13.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.721+0000 7f0ac73b2740 1 -- 172.21.15.86:0/3589129940 shutdown_connections 2024-04-21T08:50:13.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.721+0000 7f0ac73b2740 1 --2- 172.21.15.86:0/3589129940 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558999f7fa00 0x558999f79c10 unknown :-1 s=CLOSED pgs=1971 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.721+0000 7f0ac73b2740 1 --2- 172.21.15.86:0/3589129940 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558999f76e80 0x558999f796d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.721+0000 7f0ac73b2740 1 --2- 172.21.15.86:0/3589129940 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x558999ff6870 0x558999ff8c60 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.721+0000 7f0ac73b2740 1 --2- 172.21.15.86:0/3589129940 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558999f764d0 0x558999f79190 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.721+0000 7f0aa5ffb640 0 client.20142 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:50:13.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.721+0000 7f0ac73b2740 1 -- 172.21.15.86:0/3589129940 >> 172.21.15.86:0/3589129940 conn(0x558999f70ba0 msgr2=0x558999f6fc00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:13.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.721+0000 7f0ac73b2740 1 -- 172.21.15.86:0/3589129940 shutdown_connections 2024-04-21T08:50:13.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.721+0000 7f0ac73b2740 1 -- 172.21.15.86:0/3589129940 wait complete. 2024-04-21T08:50:13.728 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.721+0000 7f0ac73b2740 20 client.20142 trim_cache size 0 max 16384 2024-04-21T08:50:13.728 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:50:13.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.721+0000 7f0ac46e4640 1 -- 172.21.15.86:0/1211578378 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0aa80a3da0 msgr2=0x7f0aa80a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.721+0000 7f0ac46e4640 1 --2- 172.21.15.86:0/1211578378 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0aa80a3da0 0x7f0aa80a6260 secure :-1 s=READY pgs=328 cs=0 l=1 rev1=1 crypto rx=0x7f0ab404c000 tx=0x7f0ab4048000 comp rx=0 tx=0).stop 2024-04-21T08:50:13.729 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.721+0000 7f0ac46e4640 1 -- 172.21.15.86:0/1211578378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0abc06a840 msgr2=0x7f0abc1986b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.721+0000 7f0ac46e4640 1 --2- 172.21.15.86:0/1211578378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0abc06a840 0x7f0abc1986b0 secure :-1 s=READY pgs=1967 cs=0 l=1 rev1=1 crypto rx=0x7f0ab000d770 tx=0x7f0ab000dc30 comp rx=0 tx=0).stop 2024-04-21T08:50:13.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.721+0000 7f0ac46e4640 1 -- 172.21.15.86:0/1211578378 shutdown_connections 2024-04-21T08:50:13.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.721+0000 7f0ac46e4640 1 --2- 172.21.15.86:0/1211578378 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f0aa80a3da0 0x7f0aa80a6260 unknown :-1 s=CLOSED pgs=328 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.721+0000 7f0ac46e4640 1 --2- 172.21.15.86:0/1211578378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f0abc19cf70 0x7f0abc19d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.721+0000 7f0ac46e4640 1 --2- 172.21.15.86:0/1211578378 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f0abc107270 0x7f0abc198bf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.721+0000 7f0ac46e4640 1 --2- 172.21.15.86:0/1211578378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f0abc06a840 0x7f0abc1986b0 unknown :-1 s=CLOSED pgs=1967 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.721+0000 7f0ac46e4640 1 -- 172.21.15.86:0/1211578378 >> 172.21.15.86:0/1211578378 conn(0x7f0abc077eb0 msgr2=0x7f0abc0fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:13.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.721+0000 7f0ac46e4640 1 -- 172.21.15.86:0/1211578378 shutdown_connections 2024-04-21T08:50:13.730 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.721+0000 7f0ac46e4640 1 -- 172.21.15.86:0/1211578378 wait complete. 2024-04-21T08:50:13.731 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:50:13.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.832+0000 7fbfb83a4640 1 -- 172.21.15.86:0/1402984308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfb006a5d0 msgr2=0x7fbfb006a9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.832+0000 7fbfb83a4640 1 --2- 172.21.15.86:0/1402984308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfb006a5d0 0x7fbfb006a9b0 secure :-1 s=READY pgs=1972 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8009ca0 tx=0x7fbfa802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.832+0000 7fbfb83a4640 1 -- 172.21.15.86:0/1402984308 shutdown_connections 2024-04-21T08:50:13.833 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.832+0000 7fbfb83a4640 1 --2- 172.21.15.86:0/1402984308 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfb010e960 0x7fbfb0110d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.832+0000 7fbfb83a4640 1 --2- 172.21.15.86:0/1402984308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbfb006aef0 0x7fbfb010e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.832+0000 7fbfb83a4640 1 --2- 172.21.15.86:0/1402984308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfb006a5d0 0x7fbfb006a9b0 unknown :-1 s=CLOSED pgs=1972 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.832+0000 7fbfb83a4640 1 -- 172.21.15.86:0/1402984308 >> 172.21.15.86:0/1402984308 conn(0x7fbfb0077eb0 msgr2=0x7fbfb00782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:13.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.832+0000 7fbfb83a4640 1 -- 172.21.15.86:0/1402984308 shutdown_connections 2024-04-21T08:50:13.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.832+0000 7fbfb83a4640 1 -- 172.21.15.86:0/1402984308 wait complete. 2024-04-21T08:50:13.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.833+0000 7fbfb83a4640 1 Processor -- start 2024-04-21T08:50:13.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.833+0000 7fbfb83a4640 1 -- start start 2024-04-21T08:50:13.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.833+0000 7fbfb83a4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfb006a5d0 0x7fbfb01a0610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.833+0000 7fbfb83a4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfb006aef0 0x7fbfb01a0b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.833+0000 7fbfb83a4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbfb010e960 0x7fbfb019a780 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.833+0000 7fbfb83a4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb01143e0 con 0x7fbfb006a5d0 2024-04-21T08:50:13.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.833+0000 7fbfb83a4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb0114260 con 0x7fbfb006aef0 2024-04-21T08:50:13.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.833+0000 7fbfb83a4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb0114560 con 0x7fbfb010e960 2024-04-21T08:50:13.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.833+0000 7fbfb6140640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfb006a5d0 0x7fbfb01a0610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.833+0000 7fbfb6140640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfb006a5d0 0x7fbfb01a0610 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55262/0 (socket says 172.21.15.86:55262) 2024-04-21T08:50:13.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.833+0000 7fbfb6140640 1 -- 172.21.15.86:0/1700116600 learned_addr learned my addr 172.21.15.86:0/1700116600 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:13.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.833+0000 7fbfb6941640 1 --2- 172.21.15.86:0/1700116600 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbfb010e960 0x7fbfb019a780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.833+0000 7fbfb6140640 1 -- 172.21.15.86:0/1700116600 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbfb010e960 msgr2=0x7fbfb019a780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.833+0000 7fbfb6140640 1 --2- 172.21.15.86:0/1700116600 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbfb010e960 0x7fbfb019a780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.833+0000 7fbfb6140640 1 -- 172.21.15.86:0/1700116600 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfb006aef0 msgr2=0x7fbfb01a0b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.833+0000 7fbfb6140640 1 --2- 172.21.15.86:0/1700116600 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfb006aef0 0x7fbfb01a0b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.833+0000 7fbfb6140640 1 -- 172.21.15.86:0/1700116600 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbfb019af50 con 0x7fbfb006a5d0 2024-04-21T08:50:13.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.833+0000 7fbfb6140640 1 --2- 172.21.15.86:0/1700116600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfb006a5d0 0x7fbfb01a0610 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8005e30 tx=0x7fbfa80346f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:13.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.833+0000 7fbfa77fe640 1 -- 172.21.15.86:0/1700116600 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa8039040 con 0x7fbfb006a5d0 2024-04-21T08:50:13.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.833+0000 7fbfa77fe640 1 -- 172.21.15.86:0/1700116600 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfa8044070 con 0x7fbfb006a5d0 2024-04-21T08:50:13.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.833+0000 7fbfb83a4640 1 -- 172.21.15.86:0/1700116600 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbfb019b1e0 con 0x7fbfb006a5d0 2024-04-21T08:50:13.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.833+0000 7fbfa77fe640 1 -- 172.21.15.86:0/1700116600 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa80042a0 con 0x7fbfb006a5d0 2024-04-21T08:50:13.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.834+0000 7fbfb83a4640 1 -- 172.21.15.86:0/1700116600 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbfb01a7460 con 0x7fbfb006a5d0 2024-04-21T08:50:13.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.836+0000 7fbfa77fe640 1 -- 172.21.15.86:0/1700116600 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbfa8004440 con 0x7fbfb006a5d0 2024-04-21T08:50:13.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.836+0000 7fbfbaf94740 1 Processor -- start 2024-04-21T08:50:13.842 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.836+0000 7fbfbaf94740 1 -- start start 2024-04-21T08:50:13.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.836+0000 7fbfbaf94740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cfef4d0 0x56162cfef8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.836+0000 7fbfbaf94740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56162cfefe80 0x56162cff8430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.836+0000 7fbfbaf94740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162cff8a00 0x56162cffae50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.836+0000 7fbfbaf94740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56162cf541f0 con 0x56162cfef4d0 2024-04-21T08:50:13.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.836+0000 7fbfbaf94740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x56162cef14b0 con 0x56162cfefe80 2024-04-21T08:50:13.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.836+0000 7fbfbaf94740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56162cf501b0 con 0x56162cff8a00 2024-04-21T08:50:13.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.836+0000 7fbfa77fe640 1 --2- 172.21.15.86:0/1700116600 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbf9c0a3d20 0x7fbf9c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.837+0000 7fbfb6941640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cfef4d0 0x56162cfef8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.837+0000 7fbfb6941640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cfef4d0 0x56162cfef8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55268/0 (socket says 172.21.15.86:55268) 2024-04-21T08:50:13.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.837+0000 7fbfb6941640 1 -- 172.21.15.86:0/3788419307 learned_addr learned my addr 172.21.15.86:0/3788419307 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:13.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.837+0000 7fbfb6941640 1 --2- 172.21.15.86:0/1700116600 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbf9c0a3d20 0x7fbf9c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.837+0000 7fbfa77fe640 1 -- 172.21.15.86:0/1700116600 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7fbfa80f2de0 con 0x7fbfb006a5d0 2024-04-21T08:50:13.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.837+0000 7fbfb593f640 1 --2- 172.21.15.86:0/3788419307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162cff8a00 0x56162cffae50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.843 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.837+0000 7fbfb6140640 1 --2- 172.21.15.86:0/3788419307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56162cfefe80 0x56162cff8430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.837+0000 7fbfb6941640 1 -- 172.21.15.86:0/3788419307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162cff8a00 msgr2=0x56162cffae50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:13.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.837+0000 7fbfb6941640 1 --2- 172.21.15.86:0/3788419307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162cff8a00 0x56162cffae50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.837+0000 7fbfb6941640 1 -- 172.21.15.86:0/3788419307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56162cfefe80 msgr2=0x56162cff8430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:13.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.837+0000 7fbfb6941640 1 --2- 172.21.15.86:0/3788419307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56162cfefe80 0x56162cff8430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.837+0000 7fbfb6941640 1 -- 172.21.15.86:0/3788419307 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56162cee31b0 con 0x56162cfef4d0 2024-04-21T08:50:13.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.837+0000 7fbfb6140640 1 --2- 172.21.15.86:0/3788419307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56162cfefe80 0x56162cff8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:13.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.837+0000 7fbfb6941640 1 --2- 172.21.15.86:0/3788419307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cfef4d0 0x56162cfef8b0 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7fbfac014bb0 tx=0x7fbfac0117b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d84e7c33b041f166 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:13.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.837+0000 7fbfb6941640 1 --2- 172.21.15.86:0/1700116600 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbf9c0a3d20 0x7fbf9c0a61e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbfac011cb0 tx=0x7fbfac016000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:13.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.837+0000 7fbfa4bfe640 1 -- 172.21.15.86:0/3788419307 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac022070 con 0x56162cfef4d0 2024-04-21T08:50:13.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.837+0000 7fbfa4bfe640 1 -- 172.21.15.86:0/3788419307 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfac00c940 con 0x56162cfef4d0 2024-04-21T08:50:13.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.837+0000 7fbfa4bfe640 1 -- 172.21.15.86:0/3788419307 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac004a00 con 0x56162cfef4d0 2024-04-21T08:50:13.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.838+0000 7fbfbaf94740 1 -- 172.21.15.86:0/3788419307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cfef4d0 msgr2=0x56162cfef8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.838+0000 7fbfbaf94740 1 --2- 172.21.15.86:0/3788419307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cfef4d0 0x56162cfef8b0 secure :-1 s=READY pgs=1974 cs=0 l=1 rev1=1 crypto rx=0x7fbfac014bb0 tx=0x7fbfac0117b0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.844 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.838+0000 7fbfbaf94740 1 -- 172.21.15.86:0/3788419307 shutdown_connections 2024-04-21T08:50:13.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.838+0000 7fbfbaf94740 1 --2- 172.21.15.86:0/3788419307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162cff8a00 0x56162cffae50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.838+0000 7fbfbaf94740 1 --2- 172.21.15.86:0/3788419307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56162cfefe80 0x56162cff8430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.838+0000 7fbfbaf94740 1 --2- 172.21.15.86:0/3788419307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cfef4d0 0x56162cfef8b0 unknown :-1 s=CLOSED pgs=1974 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.838+0000 7fbfbaf94740 1 -- 172.21.15.86:0/3788419307 >> 172.21.15.86:0/3788419307 conn(0x56162cfe9ba0 msgr2=0x56162cfe9f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:13.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.838+0000 7fbfbaf94740 1 -- 172.21.15.86:0/3788419307 shutdown_connections 2024-04-21T08:50:13.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.838+0000 7fbfbaf94740 1 -- 172.21.15.86:0/3788419307 wait complete. 2024-04-21T08:50:13.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.838+0000 7fbfbaf94740 1 -- start start 2024-04-21T08:50:13.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.838+0000 7fbfbaf94740 1 Processor -- start 2024-04-21T08:50:13.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.839+0000 7fbfbaf94740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56162cfef4d0 0x56162cff83c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.839+0000 7fbfbaf94740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162cfefe80 0x56162cff5750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.839+0000 7fbfbaf94740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cff8a00 0x56162cff5e90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.839+0000 7fbfbaf94740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56162cffb910 con 0x56162cff8a00 2024-04-21T08:50:13.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.839+0000 7fbfbaf94740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x56162cf0d500 con 0x56162cfef4d0 2024-04-21T08:50:13.845 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.839+0000 7fbfbaf94740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56162cffba90 con 0x56162cfefe80 2024-04-21T08:50:13.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.839+0000 7fbfbaf94740 10 client.0 _ll_register_callbacks cb 0x56162cedbb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:13.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.839+0000 7fbfb6941640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cff8a00 0x56162cff5e90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.839+0000 7fbfb6941640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cff8a00 0x56162cff5e90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55272/0 (socket says 172.21.15.86:55272) 2024-04-21T08:50:13.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.839+0000 7fbfb6941640 1 -- 172.21.15.86:0/2117277891 learned_addr learned my addr 172.21.15.86:0/2117277891 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:13.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.839+0000 7fbfb6140640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56162cfef4d0 0x56162cff83c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.839+0000 7fbfb593f640 1 --2- 172.21.15.86:0/2117277891 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162cfefe80 0x56162cff5750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.839+0000 7fbfb6941640 1 -- 172.21.15.86:0/2117277891 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162cfefe80 msgr2=0x56162cff5750 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:13.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.839+0000 7fbfb6941640 1 --2- 172.21.15.86:0/2117277891 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162cfefe80 0x56162cff5750 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.839+0000 7fbfb6941640 1 -- 172.21.15.86:0/2117277891 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56162cfef4d0 msgr2=0x56162cff83c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:13.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.839+0000 7fbfb6941640 1 --2- 172.21.15.86:0/2117277891 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56162cfef4d0 0x56162cff83c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.839+0000 7fbfb6941640 1 -- 172.21.15.86:0/2117277891 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56162cc663f0 con 0x56162cff8a00 2024-04-21T08:50:13.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.839+0000 7fbfb593f640 1 --2- 172.21.15.86:0/2117277891 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162cfefe80 0x56162cff5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:13.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.839+0000 7fbfb6941640 1 --2- 172.21.15.86:0/2117277891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cff8a00 0x56162cff5e90 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fbfac009d40 tx=0x7fbfac009d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=67cc571711a3a45d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:13.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.839+0000 7fbf99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:13.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.839+0000 7fbf99ffb640 1 -- 172.21.15.86:0/2117277891 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfac022040 con 0x56162cff8a00 2024-04-21T08:50:13.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.839+0000 7fbf99ffb640 1 -- 172.21.15.86:0/2117277891 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7fbfac004c70 con 0x56162cff8a00 2024-04-21T08:50:13.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.839+0000 7fbfbaf94740 1 -- 172.21.15.86:0/2117277891 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfac00c940 con 0x56162cff8a00 2024-04-21T08:50:13.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.840+0000 7fbf99ffb640 1 -- 172.21.15.86:0/2117277891 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbfac03b050 con 0x56162cff8a00 2024-04-21T08:50:13.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.840+0000 7fbfbaf94740 10 client.20154 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:13.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.840+0000 7fbfbaf94740 1 -- 172.21.15.86:0/2117277891 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56162cd307d0 con 0x56162cff8a00 2024-04-21T08:50:13.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.840+0000 7fbf99ffb640 1 -- 172.21.15.86:0/2117277891 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfac00c940 con 0x56162cff8a00 2024-04-21T08:50:13.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.840+0000 7fbf99ffb640 1 -- 172.21.15.86:0/2117277891 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbfac00f6f0 con 0x56162cff8a00 2024-04-21T08:50:13.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.840+0000 7fbfbaf94740 10 client.20154 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:13.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.840+0000 7fbfbaf94740 10 client.20154 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:50:13.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.840+0000 7fbfbaf94740 20 client.20154 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:13.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.840+0000 7fbfbaf94740 1 --2- 172.21.15.86:0/2117277891 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x56162d009110 0x56162d00b530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.840+0000 7fbfbaf94740 4 client.20154 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:13.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.840+0000 7fbfbaf94740 1 -- 172.21.15.86:0/2117277891 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56162cef3930 con 0x56162d009110 2024-04-21T08:50:13.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.840+0000 7fbfb6140640 1 --2- 172.21.15.86:0/2117277891 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x56162d009110 0x56162d00b530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.841+0000 7fbf99ffb640 10 client.20154 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:50:13.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.841+0000 7fbfb6140640 1 --2- 172.21.15.86:0/2117277891 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x56162d009110 0x56162d00b530 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=e076dd02b577e29a server_cookie=9b58ec6968b0bde3 in_seq=0 out_seq=0 2024-04-21T08:50:13.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.841+0000 7fbf99ffb640 1 -- 172.21.15.86:0/2117277891 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56162cef3930 con 0x56162d009110 2024-04-21T08:50:13.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.841+0000 7fbf99ffb640 10 client.20154 handle_command_reply: tid=0 2024-04-21T08:50:13.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.841+0000 7fbfbaf94740 1 client.20154 shutdown 2024-04-21T08:50:13.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.841+0000 7fbfbaf94740 1 -- 172.21.15.86:0/2117277891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cff8a00 msgr2=0x56162cff5e90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.841+0000 7fbfbaf94740 1 --2- 172.21.15.86:0/2117277891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cff8a00 0x56162cff5e90 secure :-1 s=READY pgs=1975 cs=0 l=1 rev1=1 crypto rx=0x7fbfac009d40 tx=0x7fbfac009d70 comp rx=0 tx=0).stop 2024-04-21T08:50:13.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.842+0000 7fbfbaf94740 1 -- 172.21.15.86:0/2117277891 shutdown_connections 2024-04-21T08:50:13.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.842+0000 7fbfbaf94740 1 --2- 172.21.15.86:0/2117277891 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x56162d009110 0x56162d00b530 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.842+0000 7fbfbaf94740 1 --2- 172.21.15.86:0/2117277891 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cff8a00 0x56162cff5e90 unknown :-1 s=CLOSED pgs=1975 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.842+0000 7fbfbaf94740 1 --2- 172.21.15.86:0/2117277891 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162cfefe80 0x56162cff5750 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.842+0000 7fbf99ffb640 0 client.20154 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:50:13.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.842+0000 7fbfbaf94740 1 --2- 172.21.15.86:0/2117277891 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56162cfef4d0 0x56162cff83c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.842+0000 7fbfbaf94740 1 -- 172.21.15.86:0/2117277891 >> 172.21.15.86:0/2117277891 conn(0x56162cfe9ba0 msgr2=0x56162cfeb8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:13.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.842+0000 7fbfbaf94740 1 -- 172.21.15.86:0/2117277891 shutdown_connections 2024-04-21T08:50:13.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.842+0000 7fbfbaf94740 1 -- 172.21.15.86:0/2117277891 wait complete. 2024-04-21T08:50:13.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.842+0000 7fbfbaf94740 20 client.20154 trim_cache size 0 max 16384 2024-04-21T08:50:13.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.855+0000 7fbfbaf94740 1 Processor -- start 2024-04-21T08:50:13.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.855+0000 7fbfbaf94740 1 -- start start 2024-04-21T08:50:13.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.855+0000 7fbfbaf94740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162cfef4d0 0x56162d0047d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.855+0000 7fbfbaf94740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cfefe80 0x56162d010e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.855+0000 7fbfbaf94740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56162cff8a00 0x56162d011380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.860 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.855+0000 7fbfbaf94740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb01b1570 con 0x56162cfefe80 2024-04-21T08:50:13.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.855+0000 7fbfbaf94740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb01b30b0 con 0x56162cff8a00 2024-04-21T08:50:13.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.855+0000 7fbfbaf94740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb01b1c40 con 0x56162cfef4d0 2024-04-21T08:50:13.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.855+0000 7fbfb593f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cfefe80 0x56162d010e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.855+0000 7fbfb593f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cfefe80 0x56162d010e40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55284/0 (socket says 172.21.15.86:55284) 2024-04-21T08:50:13.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.855+0000 7fbfb593f640 1 -- 172.21.15.86:0/916977527 learned_addr learned my addr 172.21.15.86:0/916977527 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:13.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.855+0000 7fbfb6140640 1 --2- 172.21.15.86:0/916977527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162cfef4d0 0x56162d0047d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.855+0000 7fbfb593f640 1 -- 172.21.15.86:0/916977527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162cfef4d0 msgr2=0x56162d0047d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:13.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.855+0000 7fbfb593f640 1 --2- 172.21.15.86:0/916977527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162cfef4d0 0x56162d0047d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.855+0000 7fbfb593f640 1 -- 172.21.15.86:0/916977527 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56162cff8a00 msgr2=0x56162d011380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:13.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.855+0000 7fbfb6941640 1 --2- 172.21.15.86:0/916977527 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56162cff8a00 0x56162d011380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.855+0000 7fbfb593f640 1 --2- 172.21.15.86:0/916977527 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56162cff8a00 0x56162d011380 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.855+0000 7fbfb593f640 1 -- 172.21.15.86:0/916977527 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56162cfe3fe0 con 0x56162cfefe80 2024-04-21T08:50:13.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.855+0000 7fbfb593f640 1 --2- 172.21.15.86:0/916977527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cfefe80 0x56162d010e40 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0009bc0 tx=0x7fbfa0013830 comp rx=0 tx=0).ready entity=mon.0 client_cookie=81f9dc22ede19a56 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:13.861 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.856+0000 7fbf9bfff640 1 -- 172.21.15.86:0/916977527 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa0005020 con 0x56162cfefe80 2024-04-21T08:50:13.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.856+0000 7fbf9bfff640 1 -- 172.21.15.86:0/916977527 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfa00047a0 con 0x56162cfefe80 2024-04-21T08:50:13.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.856+0000 7fbf9bfff640 1 -- 172.21.15.86:0/916977527 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa000f740 con 0x56162cfefe80 2024-04-21T08:50:13.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.856+0000 7fbfbaf94740 1 -- 172.21.15.86:0/916977527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cfefe80 msgr2=0x56162d010e40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.856+0000 7fbfbaf94740 1 --2- 172.21.15.86:0/916977527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cfefe80 0x56162d010e40 secure :-1 s=READY pgs=1976 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0009bc0 tx=0x7fbfa0013830 comp rx=0 tx=0).stop 2024-04-21T08:50:13.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.856+0000 7fbfbaf94740 1 -- 172.21.15.86:0/916977527 shutdown_connections 2024-04-21T08:50:13.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.856+0000 7fbfbaf94740 1 --2- 172.21.15.86:0/916977527 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56162cff8a00 0x56162d011380 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.856+0000 7fbfbaf94740 1 --2- 172.21.15.86:0/916977527 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cfefe80 0x56162d010e40 unknown :-1 s=CLOSED pgs=1976 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.856+0000 7fbfbaf94740 1 --2- 172.21.15.86:0/916977527 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162cfef4d0 0x56162d0047d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.856+0000 7fbfbaf94740 1 -- 172.21.15.86:0/916977527 >> 172.21.15.86:0/916977527 conn(0x56162cfe9ba0 msgr2=0x56162cfe8c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:13.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.856+0000 7fbfbaf94740 1 -- 172.21.15.86:0/916977527 shutdown_connections 2024-04-21T08:50:13.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.856+0000 7fbfbaf94740 1 -- 172.21.15.86:0/916977527 wait complete. 2024-04-21T08:50:13.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.856+0000 7fbfbaf94740 1 -- start start 2024-04-21T08:50:13.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.857+0000 7fbfbaf94740 1 Processor -- start 2024-04-21T08:50:13.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.857+0000 7fbfbaf94740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56162cfef4d0 0x56162cff2670 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.857+0000 7fbfbaf94740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cfefe80 0x56162cff2bb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.857+0000 7fbfbaf94740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162cff8a00 0x56162cff30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.857+0000 7fbfbaf94740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb01b29e0 con 0x56162cfefe80 2024-04-21T08:50:13.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.857+0000 7fbfbaf94740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb01b2310 con 0x56162cfef4d0 2024-04-21T08:50:13.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.857+0000 7fbfbaf94740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbfb01af570 con 0x56162cff8a00 2024-04-21T08:50:13.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.857+0000 7fbfbaf94740 10 client.0 _ll_register_callbacks cb 0x7fbfac005710 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:13.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.857+0000 7fbfb593f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cfefe80 0x56162cff2bb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.857+0000 7fbfb593f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cfefe80 0x56162cff2bb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55300/0 (socket says 172.21.15.86:55300) 2024-04-21T08:50:13.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.857+0000 7fbfb593f640 1 -- 172.21.15.86:0/2530565874 learned_addr learned my addr 172.21.15.86:0/2530565874 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:13.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.857+0000 7fbfb6140640 1 --2- 172.21.15.86:0/2530565874 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56162cfef4d0 0x56162cff2670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.857+0000 7fbfb6941640 1 --2- 172.21.15.86:0/2530565874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162cff8a00 0x56162cff30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.857+0000 7fbfb593f640 1 -- 172.21.15.86:0/2530565874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162cff8a00 msgr2=0x56162cff30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:13.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.857+0000 7fbfb593f640 1 --2- 172.21.15.86:0/2530565874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162cff8a00 0x56162cff30f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.857+0000 7fbfb593f640 1 -- 172.21.15.86:0/2530565874 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56162cfef4d0 msgr2=0x56162cff2670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:13.863 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.857+0000 7fbfb593f640 1 --2- 172.21.15.86:0/2530565874 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56162cfef4d0 0x56162cff2670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.857+0000 7fbfb593f640 1 -- 172.21.15.86:0/2530565874 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56162cffb2b0 con 0x56162cfefe80 2024-04-21T08:50:13.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.857+0000 7fbfb593f640 1 --2- 172.21.15.86:0/2530565874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cfefe80 0x56162cff2bb0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0009b90 tx=0x7fbfa0009ee0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=be05f7b1fb7c7f8f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:13.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.857+0000 7fbf99ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:13.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.857+0000 7fbf99ffb640 1 -- 172.21.15.86:0/2530565874 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbfa0029020 con 0x56162cfefe80 2024-04-21T08:50:13.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.857+0000 7fbf99ffb640 1 -- 172.21.15.86:0/2530565874 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7fbfa0004240 con 0x56162cfefe80 2024-04-21T08:50:13.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.857+0000 7fbfbaf94740 1 -- 172.21.15.86:0/2530565874 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fbfa00047a0 con 0x56162cfefe80 2024-04-21T08:50:13.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.858+0000 7fbf99ffb640 1 -- 172.21.15.86:0/2530565874 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fbfa0034050 con 0x56162cfefe80 2024-04-21T08:50:13.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.858+0000 7fbfbaf94740 10 client.20160 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:13.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.858+0000 7fbfbaf94740 1 -- 172.21.15.86:0/2530565874 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56162d06f9f0 con 0x56162cfefe80 2024-04-21T08:50:13.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.858+0000 7fbf99ffb640 1 -- 172.21.15.86:0/2530565874 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbfa00047a0 con 0x56162cfefe80 2024-04-21T08:50:13.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.858+0000 7fbf99ffb640 1 -- 172.21.15.86:0/2530565874 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fbfa001a070 con 0x56162cfefe80 2024-04-21T08:50:13.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.858+0000 7fbfbaf94740 10 client.20160 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:13.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.858+0000 7fbfbaf94740 10 client.20160 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:50:13.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.858+0000 7fbfbaf94740 20 client.20160 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:13.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.858+0000 7fbfbaf94740 1 --2- 172.21.15.86:0/2530565874 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x56162d070170 0x56162d072590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.858+0000 7fbfbaf94740 4 client.20160 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:50:13.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.858+0000 7fbfbaf94740 1 -- 172.21.15.86:0/2530565874 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56162cf61c10 con 0x56162d070170 2024-04-21T08:50:13.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.858+0000 7fbfb6140640 1 --2- 172.21.15.86:0/2530565874 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x56162d070170 0x56162d072590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.859+0000 7fbf99ffb640 10 client.20160 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:50:13.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.859+0000 7fbfb6140640 1 --2- 172.21.15.86:0/2530565874 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x56162d070170 0x56162d072590 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=a8c2a21de13af0f8 server_cookie=c486409a01e0d497 in_seq=0 out_seq=0 2024-04-21T08:50:13.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.859+0000 7fbf99ffb640 1 -- 172.21.15.86:0/2530565874 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56162cf61c10 con 0x56162d070170 2024-04-21T08:50:13.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.859+0000 7fbf99ffb640 10 client.20160 handle_command_reply: tid=0 2024-04-21T08:50:13.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.859+0000 7fbfbaf94740 1 client.20160 shutdown 2024-04-21T08:50:13.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.859+0000 7fbfbaf94740 1 -- 172.21.15.86:0/2530565874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cfefe80 msgr2=0x56162cff2bb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.859+0000 7fbfbaf94740 1 --2- 172.21.15.86:0/2530565874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cfefe80 0x56162cff2bb0 secure :-1 s=READY pgs=1977 cs=0 l=1 rev1=1 crypto rx=0x7fbfa0009b90 tx=0x7fbfa0009ee0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.859+0000 7fbfbaf94740 1 -- 172.21.15.86:0/2530565874 shutdown_connections 2024-04-21T08:50:13.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.859+0000 7fbfbaf94740 1 --2- 172.21.15.86:0/2530565874 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x56162d070170 0x56162d072590 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.859+0000 7fbfbaf94740 1 --2- 172.21.15.86:0/2530565874 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56162cff8a00 0x56162cff30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.859+0000 7fbf99ffb640 0 client.20160 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:50:13.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.859+0000 7fbfbaf94740 1 --2- 172.21.15.86:0/2530565874 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56162cfefe80 0x56162cff2bb0 unknown :-1 s=CLOSED pgs=1977 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.859+0000 7fbfbaf94740 1 --2- 172.21.15.86:0/2530565874 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56162cfef4d0 0x56162cff2670 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.859+0000 7fbfbaf94740 1 -- 172.21.15.86:0/2530565874 >> 172.21.15.86:0/2530565874 conn(0x56162cfe9ba0 msgr2=0x56162cfeaea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:13.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.859+0000 7fbfbaf94740 1 -- 172.21.15.86:0/2530565874 shutdown_connections 2024-04-21T08:50:13.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.859+0000 7fbfbaf94740 1 -- 172.21.15.86:0/2530565874 wait complete. 2024-04-21T08:50:13.866 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.859+0000 7fbfbaf94740 20 client.20160 trim_cache size 0 max 16384 2024-04-21T08:50:13.866 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:50:13.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.859+0000 7fbfb83a4640 1 -- 172.21.15.86:0/1700116600 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbf9c0a3d20 msgr2=0x7fbf9c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.859+0000 7fbfb83a4640 1 --2- 172.21.15.86:0/1700116600 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbf9c0a3d20 0x7fbf9c0a61e0 secure :-1 s=READY pgs=329 cs=0 l=1 rev1=1 crypto rx=0x7fbfac011cb0 tx=0x7fbfac016000 comp rx=0 tx=0).stop 2024-04-21T08:50:13.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.859+0000 7fbfb83a4640 1 -- 172.21.15.86:0/1700116600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfb006a5d0 msgr2=0x7fbfb01a0610 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.859+0000 7fbfb83a4640 1 --2- 172.21.15.86:0/1700116600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfb006a5d0 0x7fbfb01a0610 secure :-1 s=READY pgs=1973 cs=0 l=1 rev1=1 crypto rx=0x7fbfa8005e30 tx=0x7fbfa80346f0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.860+0000 7fbfb83a4640 1 -- 172.21.15.86:0/1700116600 shutdown_connections 2024-04-21T08:50:13.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.860+0000 7fbfb83a4640 1 --2- 172.21.15.86:0/1700116600 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbf9c0a3d20 0x7fbf9c0a61e0 unknown :-1 s=CLOSED pgs=329 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.860+0000 7fbfb83a4640 1 --2- 172.21.15.86:0/1700116600 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbfb010e960 0x7fbfb019a780 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.860+0000 7fbfb83a4640 1 --2- 172.21.15.86:0/1700116600 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbfb006aef0 0x7fbfb01a0b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.867 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.860+0000 7fbfb83a4640 1 --2- 172.21.15.86:0/1700116600 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbfb006a5d0 0x7fbfb01a0610 unknown :-1 s=CLOSED pgs=1973 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.860+0000 7fbfb83a4640 1 -- 172.21.15.86:0/1700116600 >> 172.21.15.86:0/1700116600 conn(0x7fbfb0077eb0 msgr2=0x7fbfb010ff10 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:13.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.860+0000 7fbfb83a4640 1 -- 172.21.15.86:0/1700116600 shutdown_connections 2024-04-21T08:50:13.868 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.860+0000 7fbfb83a4640 1 -- 172.21.15.86:0/1700116600 wait complete. 2024-04-21T08:50:13.870 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:50:13.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.970+0000 7efc2fdb6640 1 -- 172.21.15.86:0/117570675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc2810c510 msgr2=0x7efc2810e9b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.970+0000 7efc2fdb6640 1 --2- 172.21.15.86:0/117570675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc2810c510 0x7efc2810e9b0 secure :-1 s=READY pgs=1978 cs=0 l=1 rev1=1 crypto rx=0x7efc20009900 tx=0x7efc2002bd60 comp rx=0 tx=0).stop 2024-04-21T08:50:13.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.971+0000 7efc2fdb6640 1 -- 172.21.15.86:0/117570675 shutdown_connections 2024-04-21T08:50:13.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.971+0000 7efc2fdb6640 1 --2- 172.21.15.86:0/117570675 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc2810c510 0x7efc2810e9b0 unknown :-1 s=CLOSED pgs=1978 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.971+0000 7efc2fdb6640 1 --2- 172.21.15.86:0/117570675 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc28103af0 0x7efc2810bfd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.971+0000 7efc2fdb6640 1 --2- 172.21.15.86:0/117570675 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc281031d0 0x7efc281035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.971+0000 7efc2fdb6640 1 -- 172.21.15.86:0/117570675 >> 172.21.15.86:0/117570675 conn(0x7efc28077f50 msgr2=0x7efc28078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:13.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.971+0000 7efc2fdb6640 1 -- 172.21.15.86:0/117570675 shutdown_connections 2024-04-21T08:50:13.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.971+0000 7efc2fdb6640 1 -- 172.21.15.86:0/117570675 wait complete. 2024-04-21T08:50:13.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.971+0000 7efc2fdb6640 1 Processor -- start 2024-04-21T08:50:13.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.971+0000 7efc2fdb6640 1 -- start start 2024-04-21T08:50:13.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.971+0000 7efc2fdb6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc281031d0 0x7efc28198700 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.971+0000 7efc2fdb6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc28103af0 0x7efc28198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.971+0000 7efc2fdb6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc2810c510 0x7efc2819cfc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.971+0000 7efc2fdb6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efc281121a0 con 0x7efc281031d0 2024-04-21T08:50:13.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.971+0000 7efc2fdb6640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efc28112020 con 0x7efc2810c510 2024-04-21T08:50:13.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.971+0000 7efc2fdb6640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7efc28112320 con 0x7efc28103af0 2024-04-21T08:50:13.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.971+0000 7efc2db52640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc281031d0 0x7efc28198700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.971+0000 7efc2db52640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc281031d0 0x7efc28198700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55314/0 (socket says 172.21.15.86:55314) 2024-04-21T08:50:13.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.972+0000 7efc2db52640 1 -- 172.21.15.86:0/542995321 learned_addr learned my addr 172.21.15.86:0/542995321 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:13.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.972+0000 7efc2d351640 1 --2- 172.21.15.86:0/542995321 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc28103af0 0x7efc28198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.972+0000 7efc2db52640 1 -- 172.21.15.86:0/542995321 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc28103af0 msgr2=0x7efc28198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.972+0000 7efc2db52640 1 --2- 172.21.15.86:0/542995321 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc28103af0 0x7efc28198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.972+0000 7efc2e353640 1 --2- 172.21.15.86:0/542995321 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc2810c510 0x7efc2819cfc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.972+0000 7efc2db52640 1 -- 172.21.15.86:0/542995321 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc2810c510 msgr2=0x7efc2819cfc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.972+0000 7efc2db52640 1 --2- 172.21.15.86:0/542995321 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc2810c510 0x7efc2819cfc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.972+0000 7efc2db52640 1 -- 172.21.15.86:0/542995321 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7efc2819d6f0 con 0x7efc281031d0 2024-04-21T08:50:13.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.972+0000 7efc2d351640 1 --2- 172.21.15.86:0/542995321 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc28103af0 0x7efc28198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:13.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.972+0000 7efc2db52640 1 --2- 172.21.15.86:0/542995321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc281031d0 0x7efc28198700 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7efc2400ea80 tx=0x7efc2400ef40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:13.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.972+0000 7efc1effd640 1 -- 172.21.15.86:0/542995321 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2400ce40 con 0x7efc281031d0 2024-04-21T08:50:13.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.972+0000 7efc2fdb6640 1 -- 172.21.15.86:0/542995321 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7efc2819d9e0 con 0x7efc281031d0 2024-04-21T08:50:13.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.972+0000 7efc1effd640 1 -- 172.21.15.86:0/542995321 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc24005020 con 0x7efc281031d0 2024-04-21T08:50:13.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.972+0000 7efc1effd640 1 -- 172.21.15.86:0/542995321 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc240116d0 con 0x7efc281031d0 2024-04-21T08:50:13.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.972+0000 7efc2fdb6640 1 -- 172.21.15.86:0/542995321 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7efc281a5270 con 0x7efc281031d0 2024-04-21T08:50:13.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.974+0000 7efc1effd640 1 -- 172.21.15.86:0/542995321 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7efc240054d0 con 0x7efc281031d0 2024-04-21T08:50:13.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.975+0000 7efc329a9740 1 Processor -- start 2024-04-21T08:50:13.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.975+0000 7efc329a9740 1 -- start start 2024-04-21T08:50:13.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.975+0000 7efc329a9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55afac7e0330 0x55afac7e0710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.975+0000 7efc329a9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afac7e0ce0 0x55afac7e9820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.975+0000 7efc329a9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afac7e9e40 0x55afac7ea2c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.975+0000 7efc329a9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55afac7451f0 con 0x55afac7e9e40 2024-04-21T08:50:13.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.975+0000 7efc329a9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55afac6e24b0 con 0x55afac7e0330 2024-04-21T08:50:13.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.975+0000 7efc329a9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55afac7411b0 con 0x55afac7e0ce0 2024-04-21T08:50:13.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.975+0000 7efc1effd640 1 --2- 172.21.15.86:0/542995321 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efc140a3da0 0x7efc140a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.975+0000 7efc2e353640 1 --2- 172.21.15.86:0/542995321 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efc140a3da0 0x7efc140a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.975+0000 7efc2d351640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afac7e9e40 0x55afac7ea2c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.975+0000 7efc2d351640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afac7e9e40 0x55afac7ea2c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55316/0 (socket says 172.21.15.86:55316) 2024-04-21T08:50:13.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.975+0000 7efc2d351640 1 -- 172.21.15.86:0/4009294152 learned_addr learned my addr 172.21.15.86:0/4009294152 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:13.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.975+0000 7efc2e353640 1 --2- 172.21.15.86:0/4009294152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55afac7e0330 0x55afac7e0710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.975+0000 7efc1effd640 1 -- 172.21.15.86:0/542995321 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7efc240d0fe0 con 0x7efc281031d0 2024-04-21T08:50:13.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.976+0000 7efc2db52640 1 --2- 172.21.15.86:0/4009294152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afac7e0ce0 0x55afac7e9820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.976+0000 7efc2d351640 1 -- 172.21.15.86:0/4009294152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afac7e0ce0 msgr2=0x55afac7e9820 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:13.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.976+0000 7efc2d351640 1 --2- 172.21.15.86:0/4009294152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afac7e0ce0 0x55afac7e9820 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.976+0000 7efc2d351640 1 -- 172.21.15.86:0/4009294152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55afac7e0330 msgr2=0x55afac7e0710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:13.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.976+0000 7efc2d351640 1 --2- 172.21.15.86:0/4009294152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55afac7e0330 0x55afac7e0710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.976+0000 7efc2d351640 1 -- 172.21.15.86:0/4009294152 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afac6d41b0 con 0x55afac7e9e40 2024-04-21T08:50:13.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.976+0000 7efc2e353640 1 --2- 172.21.15.86:0/542995321 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efc140a3da0 0x7efc140a6260 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7efc20005770 tx=0x7efc200040f0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:13.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.976+0000 7efc2d351640 1 --2- 172.21.15.86:0/4009294152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afac7e9e40 0x55afac7ea2c0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7efc18009ba0 tx=0x7efc1800e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b624d31afeae9ed4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:13.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.976+0000 7efc13fff640 1 -- 172.21.15.86:0/4009294152 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc18016070 con 0x55afac7e9e40 2024-04-21T08:50:13.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.976+0000 7efc13fff640 1 -- 172.21.15.86:0/4009294152 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc18011920 con 0x55afac7e9e40 2024-04-21T08:50:13.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.976+0000 7efc13fff640 1 -- 172.21.15.86:0/4009294152 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc180107f0 con 0x55afac7e9e40 2024-04-21T08:50:13.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.977+0000 7efc329a9740 1 -- 172.21.15.86:0/4009294152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afac7e9e40 msgr2=0x55afac7ea2c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.977+0000 7efc329a9740 1 --2- 172.21.15.86:0/4009294152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afac7e9e40 0x55afac7ea2c0 secure :-1 s=READY pgs=1980 cs=0 l=1 rev1=1 crypto rx=0x7efc18009ba0 tx=0x7efc1800e6c0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.977+0000 7efc329a9740 1 -- 172.21.15.86:0/4009294152 shutdown_connections 2024-04-21T08:50:13.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.977+0000 7efc329a9740 1 --2- 172.21.15.86:0/4009294152 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afac7e9e40 0x55afac7ea2c0 unknown :-1 s=CLOSED pgs=1980 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.977+0000 7efc329a9740 1 --2- 172.21.15.86:0/4009294152 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afac7e0ce0 0x55afac7e9820 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.977+0000 7efc329a9740 1 --2- 172.21.15.86:0/4009294152 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55afac7e0330 0x55afac7e0710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.977+0000 7efc329a9740 1 -- 172.21.15.86:0/4009294152 >> 172.21.15.86:0/4009294152 conn(0x55afac7daba0 msgr2=0x55afac7daf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:13.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.977+0000 7efc329a9740 1 -- 172.21.15.86:0/4009294152 shutdown_connections 2024-04-21T08:50:13.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.977+0000 7efc329a9740 1 -- 172.21.15.86:0/4009294152 wait complete. 2024-04-21T08:50:13.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.977+0000 7efc329a9740 1 -- start start 2024-04-21T08:50:13.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.977+0000 7efc329a9740 1 Processor -- start 2024-04-21T08:50:13.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.978+0000 7efc329a9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afac7e0330 0x55afac7f01d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.978+0000 7efc329a9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afac7e0ce0 0x55afac7ed560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.978+0000 7efc329a9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55afac7e9e40 0x55afac7edaa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.978+0000 7efc329a9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55afac7f0dc0 con 0x55afac7e0330 2024-04-21T08:50:13.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.978+0000 7efc329a9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55afac6fe500 con 0x55afac7e9e40 2024-04-21T08:50:13.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.978+0000 7efc329a9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55afac7f0f40 con 0x55afac7e0ce0 2024-04-21T08:50:13.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.978+0000 7efc329a9740 10 client.0 _ll_register_callbacks cb 0x55afac6ccb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:13.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.978+0000 7efc2db52640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afac7e0330 0x55afac7f01d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.978+0000 7efc2db52640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afac7e0330 0x55afac7f01d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55320/0 (socket says 172.21.15.86:55320) 2024-04-21T08:50:13.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.978+0000 7efc2db52640 1 -- 172.21.15.86:0/3021375000 learned_addr learned my addr 172.21.15.86:0/3021375000 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:13.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.978+0000 7efc2d351640 1 --2- 172.21.15.86:0/3021375000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afac7e0ce0 0x55afac7ed560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.978+0000 7efc2e353640 1 --2- 172.21.15.86:0/3021375000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55afac7e9e40 0x55afac7edaa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.978+0000 7efc2db52640 1 -- 172.21.15.86:0/3021375000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afac7e0ce0 msgr2=0x55afac7ed560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:13.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.978+0000 7efc2db52640 1 --2- 172.21.15.86:0/3021375000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afac7e0ce0 0x55afac7ed560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.978+0000 7efc2db52640 1 -- 172.21.15.86:0/3021375000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55afac7e9e40 msgr2=0x55afac7edaa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:13.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.978+0000 7efc2db52640 1 --2- 172.21.15.86:0/3021375000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55afac7e9e40 0x55afac7edaa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.978+0000 7efc2db52640 1 -- 172.21.15.86:0/3021375000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afac4573f0 con 0x55afac7e0330 2024-04-21T08:50:13.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.978+0000 7efc2e353640 1 --2- 172.21.15.86:0/3021375000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55afac7e9e40 0x55afac7edaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:13.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.978+0000 7efc2db52640 1 --2- 172.21.15.86:0/3021375000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afac7e0330 0x55afac7f01d0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7efc24005bc0 tx=0x7efc2409cb50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c665b22e6396674a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:13.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.978+0000 7efc117fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:13.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.978+0000 7efc117fa640 1 -- 172.21.15.86:0/3021375000 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2401f040 con 0x55afac7e0330 2024-04-21T08:50:13.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.978+0000 7efc117fa640 1 -- 172.21.15.86:0/3021375000 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7efc240ce580 con 0x55afac7e0330 2024-04-21T08:50:13.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.978+0000 7efc329a9740 1 -- 172.21.15.86:0/3021375000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc18011920 con 0x55afac7e0330 2024-04-21T08:50:13.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.978+0000 7efc117fa640 1 -- 172.21.15.86:0/3021375000 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efc240b60d0 con 0x55afac7e0330 2024-04-21T08:50:13.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.978+0000 7efc329a9740 10 client.20172 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:13.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.978+0000 7efc329a9740 1 -- 172.21.15.86:0/3021375000 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55afac5217d0 con 0x55afac7e0330 2024-04-21T08:50:13.985 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.979+0000 7efc117fa640 1 -- 172.21.15.86:0/3021375000 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc18011920 con 0x55afac7e0330 2024-04-21T08:50:13.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.979+0000 7efc117fa640 1 -- 172.21.15.86:0/3021375000 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7efc2400f040 con 0x55afac7e0330 2024-04-21T08:50:13.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.979+0000 7efc329a9740 10 client.20172 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:13.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.979+0000 7efc329a9740 10 client.20172 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:50:13.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.979+0000 7efc329a9740 20 client.20172 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:13.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.979+0000 7efc329a9740 1 --2- 172.21.15.86:0/3021375000 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55afac7fdfc0 0x55afac8003f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.979+0000 7efc329a9740 4 client.20172 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:13.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.979+0000 7efc329a9740 1 -- 172.21.15.86:0/3021375000 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55afac6e4930 con 0x55afac7fdfc0 2024-04-21T08:50:13.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.979+0000 7efc2d351640 1 --2- 172.21.15.86:0/3021375000 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55afac7fdfc0 0x55afac8003f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.980+0000 7efc117fa640 10 client.20172 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:50:13.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.980+0000 7efc2d351640 1 --2- 172.21.15.86:0/3021375000 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55afac7fdfc0 0x55afac8003f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=77d9c6381563b75e server_cookie=4246e5656fe2cc2 in_seq=0 out_seq=0 2024-04-21T08:50:13.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.980+0000 7efc117fa640 1 -- 172.21.15.86:0/3021375000 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55afac6e4930 con 0x55afac7fdfc0 2024-04-21T08:50:13.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.980+0000 7efc117fa640 10 client.20172 handle_command_reply: tid=0 2024-04-21T08:50:13.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.980+0000 7efc329a9740 1 client.20172 shutdown 2024-04-21T08:50:13.986 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.980+0000 7efc329a9740 1 -- 172.21.15.86:0/3021375000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afac7e0330 msgr2=0x55afac7f01d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:13.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.980+0000 7efc329a9740 1 --2- 172.21.15.86:0/3021375000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afac7e0330 0x55afac7f01d0 secure :-1 s=READY pgs=1981 cs=0 l=1 rev1=1 crypto rx=0x7efc24005bc0 tx=0x7efc2409cb50 comp rx=0 tx=0).stop 2024-04-21T08:50:13.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.981+0000 7efc329a9740 1 -- 172.21.15.86:0/3021375000 shutdown_connections 2024-04-21T08:50:13.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.981+0000 7efc329a9740 1 --2- 172.21.15.86:0/3021375000 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55afac7fdfc0 0x55afac8003f0 crc :-1 s=READY pgs=75 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.981+0000 7efc329a9740 1 --2- 172.21.15.86:0/3021375000 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55afac7e9e40 0x55afac7edaa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.981+0000 7efc329a9740 1 --2- 172.21.15.86:0/3021375000 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afac7e0ce0 0x55afac7ed560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.981+0000 7efc329a9740 1 --2- 172.21.15.86:0/3021375000 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afac7e0330 0x55afac7f01d0 unknown :-1 s=CLOSED pgs=1981 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.981+0000 7efc117fa640 0 client.20172 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:50:13.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.981+0000 7efc329a9740 1 -- 172.21.15.86:0/3021375000 >> 172.21.15.86:0/3021375000 conn(0x55afac7daba0 msgr2=0x55afac7dc8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:13.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.981+0000 7efc329a9740 1 -- 172.21.15.86:0/3021375000 shutdown_connections 2024-04-21T08:50:13.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.981+0000 7efc329a9740 1 -- 172.21.15.86:0/3021375000 wait complete. 2024-04-21T08:50:13.987 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.981+0000 7efc329a9740 20 client.20172 trim_cache size 0 max 16384 2024-04-21T08:50:13.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.994+0000 7efc329a9740 1 Processor -- start 2024-04-21T08:50:13.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.994+0000 7efc329a9740 1 -- start start 2024-04-21T08:50:13.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.994+0000 7efc329a9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afac7daba0 0x55afac7f9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.994+0000 7efc329a9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55afac7e0330 0x55afac7d9a80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.998 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.994+0000 7efc329a9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afac7e0ce0 0x55afac7efc90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:13.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.994+0000 7efc329a9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efc281ad1e0 con 0x55afac7daba0 2024-04-21T08:50:13.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.994+0000 7efc329a9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efc281aed20 con 0x55afac7e0330 2024-04-21T08:50:13.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.994+0000 7efc329a9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7efc281ad8b0 con 0x55afac7e0ce0 2024-04-21T08:50:13.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.994+0000 7efc2db52640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afac7daba0 0x55afac7f9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.994+0000 7efc2db52640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afac7daba0 0x55afac7f9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55336/0 (socket says 172.21.15.86:55336) 2024-04-21T08:50:13.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.994+0000 7efc2db52640 1 -- 172.21.15.86:0/1650435696 learned_addr learned my addr 172.21.15.86:0/1650435696 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:13.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.994+0000 7efc2e353640 1 --2- 172.21.15.86:0/1650435696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afac7e0ce0 0x55afac7efc90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:13.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.994+0000 7efc2db52640 1 -- 172.21.15.86:0/1650435696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afac7e0ce0 msgr2=0x55afac7efc90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:13.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.994+0000 7efc2db52640 1 --2- 172.21.15.86:0/1650435696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afac7e0ce0 0x55afac7efc90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.994+0000 7efc2db52640 1 -- 172.21.15.86:0/1650435696 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55afac7e0330 msgr2=0x55afac7d9a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:13.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.994+0000 7efc2db52640 1 --2- 172.21.15.86:0/1650435696 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55afac7e0330 0x55afac7d9a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:13.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.994+0000 7efc2db52640 1 -- 172.21.15.86:0/1650435696 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55afac7dfe30 con 0x55afac7daba0 2024-04-21T08:50:13.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.994+0000 7efc2db52640 1 --2- 172.21.15.86:0/1650435696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afac7daba0 0x55afac7f9cc0 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7efc2409c850 tx=0x7efc240a5880 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5d584b2729ffabc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:13.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.994+0000 7efc137fe640 1 -- 172.21.15.86:0/1650435696 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2401f040 con 0x55afac7daba0 2024-04-21T08:50:13.999 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.995+0000 7efc137fe640 1 -- 172.21.15.86:0/1650435696 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc240051d0 con 0x55afac7daba0 2024-04-21T08:50:14.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.995+0000 7efc137fe640 1 -- 172.21.15.86:0/1650435696 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc240a53b0 con 0x55afac7daba0 2024-04-21T08:50:14.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.995+0000 7efc329a9740 1 -- 172.21.15.86:0/1650435696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afac7daba0 msgr2=0x55afac7f9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:14.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.995+0000 7efc329a9740 1 --2- 172.21.15.86:0/1650435696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afac7daba0 0x55afac7f9cc0 secure :-1 s=READY pgs=1982 cs=0 l=1 rev1=1 crypto rx=0x7efc2409c850 tx=0x7efc240a5880 comp rx=0 tx=0).stop 2024-04-21T08:50:14.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.995+0000 7efc329a9740 1 -- 172.21.15.86:0/1650435696 shutdown_connections 2024-04-21T08:50:14.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.995+0000 7efc329a9740 1 --2- 172.21.15.86:0/1650435696 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afac7e0ce0 0x55afac7efc90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.995+0000 7efc329a9740 1 --2- 172.21.15.86:0/1650435696 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55afac7e0330 0x55afac7d9a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.995+0000 7efc329a9740 1 --2- 172.21.15.86:0/1650435696 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afac7daba0 0x55afac7f9cc0 unknown :-1 s=CLOSED pgs=1982 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.995+0000 7efc329a9740 1 -- 172.21.15.86:0/1650435696 >> 172.21.15.86:0/1650435696 conn(0x55afac7edc00 msgr2=0x55afac7d9370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:14.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.995+0000 7efc329a9740 1 -- 172.21.15.86:0/1650435696 shutdown_connections 2024-04-21T08:50:14.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.995+0000 7efc329a9740 1 -- 172.21.15.86:0/1650435696 wait complete. 2024-04-21T08:50:14.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.995+0000 7efc329a9740 1 -- start start 2024-04-21T08:50:14.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.995+0000 7efc329a9740 1 Processor -- start 2024-04-21T08:50:14.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.996+0000 7efc329a9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afac7daba0 0x55afac7f6430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.996+0000 7efc329a9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afac7e0330 0x55afac7eed00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.996+0000 7efc329a9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55afac7e0ce0 0x55afac7ef240 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.996+0000 7efc329a9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7efc281ae650 con 0x55afac7daba0 2024-04-21T08:50:14.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.996+0000 7efc329a9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7efc281adf80 con 0x55afac7e0ce0 2024-04-21T08:50:14.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.996+0000 7efc329a9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7efc281ab1e0 con 0x55afac7e0330 2024-04-21T08:50:14.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.996+0000 7efc329a9740 10 client.0 _ll_register_callbacks cb 0x7efc2409c560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:14.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.996+0000 7efc2db52640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afac7daba0 0x55afac7f6430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:14.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.996+0000 7efc2db52640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afac7daba0 0x55afac7f6430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55340/0 (socket says 172.21.15.86:55340) 2024-04-21T08:50:14.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.996+0000 7efc2db52640 1 -- 172.21.15.86:0/1130876418 learned_addr learned my addr 172.21.15.86:0/1130876418 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:14.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.996+0000 7efc2d351640 1 --2- 172.21.15.86:0/1130876418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afac7e0330 0x55afac7eed00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:14.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.996+0000 7efc2db52640 1 -- 172.21.15.86:0/1130876418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afac7e0330 msgr2=0x55afac7eed00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:14.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.996+0000 7efc2db52640 1 --2- 172.21.15.86:0/1130876418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afac7e0330 0x55afac7eed00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.996+0000 7efc2db52640 1 -- 172.21.15.86:0/1130876418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55afac7e0ce0 msgr2=0x55afac7ef240 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:14.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.996+0000 7efc2db52640 1 --2- 172.21.15.86:0/1130876418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55afac7e0ce0 0x55afac7ef240 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.996+0000 7efc2db52640 1 -- 172.21.15.86:0/1130876418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55afac7ef8e0 con 0x55afac7daba0 2024-04-21T08:50:14.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.996+0000 7efc2db52640 1 --2- 172.21.15.86:0/1130876418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afac7daba0 0x55afac7f6430 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7efc2409b490 tx=0x7efc24099460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a598da3115d8dc37 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:14.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.996+0000 7efc117fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:14.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.996+0000 7efc117fa640 1 -- 172.21.15.86:0/1130876418 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7efc2401f040 con 0x55afac7daba0 2024-04-21T08:50:14.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.996+0000 7efc117fa640 1 -- 172.21.15.86:0/1130876418 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7efc2409bfa0 con 0x55afac7daba0 2024-04-21T08:50:14.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.996+0000 7efc329a9740 1 -- 172.21.15.86:0/1130876418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7efc240051d0 con 0x55afac7daba0 2024-04-21T08:50:14.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.997+0000 7efc117fa640 1 -- 172.21.15.86:0/1130876418 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7efc240a55c0 con 0x55afac7daba0 2024-04-21T08:50:14.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.997+0000 7efc329a9740 10 client.20178 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:14.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.997+0000 7efc329a9740 1 -- 172.21.15.86:0/1130876418 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55afac8643c0 con 0x55afac7daba0 2024-04-21T08:50:14.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.997+0000 7efc117fa640 1 -- 172.21.15.86:0/1130876418 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7efc240051d0 con 0x55afac7daba0 2024-04-21T08:50:14.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.997+0000 7efc117fa640 1 -- 172.21.15.86:0/1130876418 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7efc24061ad0 con 0x55afac7daba0 2024-04-21T08:50:14.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.997+0000 7efc329a9740 10 client.20178 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:14.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.997+0000 7efc329a9740 10 client.20178 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:50:14.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.997+0000 7efc329a9740 20 client.20178 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:14.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.997+0000 7efc329a9740 1 --2- 172.21.15.86:0/1130876418 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55afac7e9e40 0x55afac866800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.997+0000 7efc329a9740 4 client.20178 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:50:14.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.997+0000 7efc329a9740 1 -- 172.21.15.86:0/1130876418 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55afac752c10 con 0x55afac7e9e40 2024-04-21T08:50:14.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.997+0000 7efc2d351640 1 --2- 172.21.15.86:0/1130876418 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55afac7e9e40 0x55afac866800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:14.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.998+0000 7efc117fa640 10 client.20178 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:50:14.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.998+0000 7efc2d351640 1 --2- 172.21.15.86:0/1130876418 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55afac7e9e40 0x55afac866800 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=3a8dbd1ef4c04752 server_cookie=ffa69c042806530 in_seq=0 out_seq=0 2024-04-21T08:50:14.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.998+0000 7efc117fa640 1 -- 172.21.15.86:0/1130876418 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55afac752c10 con 0x55afac7e9e40 2024-04-21T08:50:14.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.998+0000 7efc117fa640 10 client.20178 handle_command_reply: tid=0 2024-04-21T08:50:14.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.998+0000 7efc329a9740 1 client.20178 shutdown 2024-04-21T08:50:14.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.998+0000 7efc329a9740 1 -- 172.21.15.86:0/1130876418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afac7daba0 msgr2=0x55afac7f6430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:14.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.998+0000 7efc329a9740 1 --2- 172.21.15.86:0/1130876418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afac7daba0 0x55afac7f6430 secure :-1 s=READY pgs=1983 cs=0 l=1 rev1=1 crypto rx=0x7efc2409b490 tx=0x7efc24099460 comp rx=0 tx=0).stop 2024-04-21T08:50:14.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.998+0000 7efc329a9740 1 -- 172.21.15.86:0/1130876418 shutdown_connections 2024-04-21T08:50:14.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.998+0000 7efc329a9740 1 --2- 172.21.15.86:0/1130876418 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x55afac7e9e40 0x55afac866800 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.998+0000 7efc329a9740 1 --2- 172.21.15.86:0/1130876418 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55afac7e0ce0 0x55afac7ef240 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.998+0000 7efc117fa640 0 client.20178 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:50:14.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.998+0000 7efc329a9740 1 --2- 172.21.15.86:0/1130876418 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55afac7e0330 0x55afac7eed00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.998+0000 7efc329a9740 1 --2- 172.21.15.86:0/1130876418 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55afac7daba0 0x55afac7f6430 unknown :-1 s=CLOSED pgs=1983 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.998+0000 7efc329a9740 1 -- 172.21.15.86:0/1130876418 >> 172.21.15.86:0/1130876418 conn(0x55afac7edc00 msgr2=0x55afac7d9c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:14.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.998+0000 7efc329a9740 1 -- 172.21.15.86:0/1130876418 shutdown_connections 2024-04-21T08:50:14.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.998+0000 7efc329a9740 1 -- 172.21.15.86:0/1130876418 wait complete. 2024-04-21T08:50:14.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.998+0000 7efc329a9740 20 client.20178 trim_cache size 0 max 16384 2024-04-21T08:50:14.004 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:50:14.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.999+0000 7efc2fdb6640 1 -- 172.21.15.86:0/542995321 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efc140a3da0 msgr2=0x7efc140a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:14.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.999+0000 7efc2fdb6640 1 --2- 172.21.15.86:0/542995321 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efc140a3da0 0x7efc140a6260 secure :-1 s=READY pgs=330 cs=0 l=1 rev1=1 crypto rx=0x7efc20005770 tx=0x7efc200040f0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.999+0000 7efc2fdb6640 1 -- 172.21.15.86:0/542995321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc281031d0 msgr2=0x7efc28198700 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:14.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.999+0000 7efc2fdb6640 1 --2- 172.21.15.86:0/542995321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc281031d0 0x7efc28198700 secure :-1 s=READY pgs=1979 cs=0 l=1 rev1=1 crypto rx=0x7efc2400ea80 tx=0x7efc2400ef40 comp rx=0 tx=0).stop 2024-04-21T08:50:14.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.999+0000 7efc2fdb6640 1 -- 172.21.15.86:0/542995321 shutdown_connections 2024-04-21T08:50:14.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.999+0000 7efc2fdb6640 1 --2- 172.21.15.86:0/542995321 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7efc140a3da0 0x7efc140a6260 unknown :-1 s=CLOSED pgs=330 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.999+0000 7efc2fdb6640 1 --2- 172.21.15.86:0/542995321 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7efc2810c510 0x7efc2819cfc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.999+0000 7efc2fdb6640 1 --2- 172.21.15.86:0/542995321 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7efc28103af0 0x7efc28198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.999+0000 7efc2fdb6640 1 --2- 172.21.15.86:0/542995321 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7efc281031d0 0x7efc28198700 unknown :-1 s=CLOSED pgs=1979 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.999+0000 7efc2fdb6640 1 -- 172.21.15.86:0/542995321 >> 172.21.15.86:0/542995321 conn(0x7efc28077f50 msgr2=0x7efc280fdf70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:14.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.999+0000 7efc2fdb6640 1 -- 172.21.15.86:0/542995321 shutdown_connections 2024-04-21T08:50:14.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:13.999+0000 7efc2fdb6640 1 -- 172.21.15.86:0/542995321 wait complete. 2024-04-21T08:50:14.009 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:50:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.109+0000 7f1fe68f2640 1 -- 172.21.15.86:0/1535955077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe0100af0 msgr2=0x7f1fe0100ed0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.109+0000 7f1fe68f2640 1 --2- 172.21.15.86:0/1535955077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe0100af0 0x7f1fe0100ed0 secure :-1 s=READY pgs=1984 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0009900 tx=0x7f1fd002bc30 comp rx=0 tx=0).stop 2024-04-21T08:50:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.109+0000 7f1fe68f2640 1 -- 172.21.15.86:0/1535955077 shutdown_connections 2024-04-21T08:50:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.109+0000 7f1fe68f2640 1 --2- 172.21.15.86:0/1535955077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1fe01072b0 0x7f1fe010eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.109+0000 7f1fe68f2640 1 --2- 172.21.15.86:0/1535955077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fe01014a0 0x7f1fe0106bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.109+0000 7f1fe68f2640 1 --2- 172.21.15.86:0/1535955077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe0100af0 0x7f1fe0100ed0 unknown :-1 s=CLOSED pgs=1984 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.109 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.109+0000 7f1fe68f2640 1 -- 172.21.15.86:0/1535955077 >> 172.21.15.86:0/1535955077 conn(0x7f1fe00760e0 msgr2=0x7f1fe00764f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:14.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.109+0000 7f1fe68f2640 1 -- 172.21.15.86:0/1535955077 shutdown_connections 2024-04-21T08:50:14.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.109+0000 7f1fe68f2640 1 -- 172.21.15.86:0/1535955077 wait complete. 2024-04-21T08:50:14.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.110+0000 7f1fe68f2640 1 Processor -- start 2024-04-21T08:50:14.110 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.110+0000 7f1fe68f2640 1 -- start start 2024-04-21T08:50:14.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.110+0000 7f1fe68f2640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fe0100af0 0x7f1fe0198760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.110+0000 7f1fe68f2640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe01072b0 0x7f1fe0198ca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.110+0000 7f1fe68f2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1fe019d020 0x7f1fe019d4d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.110+0000 7f1fe68f2640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe0112280 con 0x7f1fe01072b0 2024-04-21T08:50:14.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.110+0000 7f1fe68f2640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe0112100 con 0x7f1fe0100af0 2024-04-21T08:50:14.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.110+0000 7f1fe68f2640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe0112400 con 0x7f1fe019d020 2024-04-21T08:50:14.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.110+0000 7f1fdf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe01072b0 0x7f1fe0198ca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:14.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.110+0000 7f1fdf7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe01072b0 0x7f1fe0198ca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55364/0 (socket says 172.21.15.86:55364) 2024-04-21T08:50:14.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.110+0000 7f1fdf7fe640 1 -- 172.21.15.86:0/3268520646 learned_addr learned my addr 172.21.15.86:0/3268520646 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:14.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.110+0000 7f1fdffff640 1 --2- 172.21.15.86:0/3268520646 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fe0100af0 0x7f1fe0198760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:14.112 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.110+0000 7f1fdf7fe640 1 -- 172.21.15.86:0/3268520646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1fe019d020 msgr2=0x7f1fe019d4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:14.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.110+0000 7f1fdf7fe640 1 --2- 172.21.15.86:0/3268520646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1fe019d020 0x7f1fe019d4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.110+0000 7f1fdf7fe640 1 -- 172.21.15.86:0/3268520646 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fe0100af0 msgr2=0x7f1fe0198760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:14.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.110+0000 7f1fdf7fe640 1 --2- 172.21.15.86:0/3268520646 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fe0100af0 0x7f1fe0198760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.110+0000 7f1fdf7fe640 1 -- 172.21.15.86:0/3268520646 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1fe019db10 con 0x7f1fe01072b0 2024-04-21T08:50:14.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.111+0000 7f1fdf7fe640 1 --2- 172.21.15.86:0/3268520646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe01072b0 0x7f1fe0198ca0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4005fd0 tx=0x7f1fd400dde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:14.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.111+0000 7f1fdd7fa640 1 -- 172.21.15.86:0/3268520646 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd4017070 con 0x7f1fe01072b0 2024-04-21T08:50:14.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.111+0000 7f1fdd7fa640 1 -- 172.21.15.86:0/3268520646 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fd4009c30 con 0x7f1fe01072b0 2024-04-21T08:50:14.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.111+0000 7f1fe68f2640 1 -- 172.21.15.86:0/3268520646 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1fe019de00 con 0x7f1fe01072b0 2024-04-21T08:50:14.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.111+0000 7f1fdd7fa640 1 -- 172.21.15.86:0/3268520646 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd4003d30 con 0x7f1fe01072b0 2024-04-21T08:50:14.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.111+0000 7f1fe68f2640 1 -- 172.21.15.86:0/3268520646 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1fe01a55f0 con 0x7f1fe01072b0 2024-04-21T08:50:14.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.113+0000 7f1fdd7fa640 1 -- 172.21.15.86:0/3268520646 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1fd4003ed0 con 0x7f1fe01072b0 2024-04-21T08:50:14.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.113+0000 7f1fe95be740 1 Processor -- start 2024-04-21T08:50:14.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.113+0000 7f1fe95be740 1 -- start start 2024-04-21T08:50:14.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.113+0000 7f1fe95be740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb3961a4d0 0x55fb3961a8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.113+0000 7f1fe95be740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb3961ae80 0x55fb39623430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.113+0000 7f1fe95be740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb39623a00 0x55fb39625e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.114+0000 7f1fe95be740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fb3957f1f0 con 0x55fb3961ae80 2024-04-21T08:50:14.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.114+0000 7f1fe95be740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55fb3951c4b0 con 0x55fb39623a00 2024-04-21T08:50:14.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.114+0000 7f1fe95be740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55fb3957b1b0 con 0x55fb3961a4d0 2024-04-21T08:50:14.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.114+0000 7f1fdffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb3961ae80 0x55fb39623430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:14.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.114+0000 7f1fdffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb3961ae80 0x55fb39623430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55378/0 (socket says 172.21.15.86:55378) 2024-04-21T08:50:14.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.114+0000 7f1fdffff640 1 -- 172.21.15.86:0/1140730538 learned_addr learned my addr 172.21.15.86:0/1140730538 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:14.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.114+0000 7f1fdd7fa640 1 --2- 172.21.15.86:0/3268520646 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1fc80a3d20 0x7f1fc80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.114+0000 7f1fe4e8f640 1 --2- 172.21.15.86:0/1140730538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb3961a4d0 0x55fb3961a8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:14.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.114+0000 7f1fe4e8f640 1 --2- 172.21.15.86:0/3268520646 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1fc80a3d20 0x7f1fc80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:14.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.114+0000 7f1fdd7fa640 1 -- 172.21.15.86:0/3268520646 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f1fd40d0930 con 0x7f1fe01072b0 2024-04-21T08:50:14.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.114+0000 7f1fdf7fe640 1 --2- 172.21.15.86:0/1140730538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb39623a00 0x55fb39625e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:14.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.114+0000 7f1fdffff640 1 -- 172.21.15.86:0/1140730538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb3961a4d0 msgr2=0x55fb3961a8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:14.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.114+0000 7f1fdffff640 1 --2- 172.21.15.86:0/1140730538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb3961a4d0 0x55fb3961a8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.114+0000 7f1fdffff640 1 -- 172.21.15.86:0/1140730538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb39623a00 msgr2=0x55fb39625e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:14.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.114+0000 7f1fdffff640 1 --2- 172.21.15.86:0/1140730538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb39623a00 0x55fb39625e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.114+0000 7f1fdffff640 1 -- 172.21.15.86:0/1140730538 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb3950e1b0 con 0x55fb3961ae80 2024-04-21T08:50:14.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.114+0000 7f1fe4e8f640 1 --2- 172.21.15.86:0/1140730538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb3961a4d0 0x55fb3961a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:14.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.114+0000 7f1fdf7fe640 1 --2- 172.21.15.86:0/1140730538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb39623a00 0x55fb39625e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:14.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.114+0000 7f1fdffff640 1 --2- 172.21.15.86:0/1140730538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb3961ae80 0x55fb39623430 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0009230 tx=0x7f1fd002c650 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5b7c8221289626a0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:14.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.114+0000 7f1fe4e8f640 1 --2- 172.21.15.86:0/3268520646 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1fc80a3d20 0x7f1fc80a61e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8011440 tx=0x7f1fd8005e90 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:14.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.114+0000 7f1fc67fc640 1 -- 172.21.15.86:0/1140730538 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0044020 con 0x55fb3961ae80 2024-04-21T08:50:14.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.114+0000 7f1fc67fc640 1 -- 172.21.15.86:0/1140730538 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fd002cc90 con 0x55fb3961ae80 2024-04-21T08:50:14.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.114+0000 7f1fc67fc640 1 -- 172.21.15.86:0/1140730538 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd00046c0 con 0x55fb3961ae80 2024-04-21T08:50:14.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.115+0000 7f1fe95be740 1 -- 172.21.15.86:0/1140730538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb3961ae80 msgr2=0x55fb39623430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:14.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.115+0000 7f1fe95be740 1 --2- 172.21.15.86:0/1140730538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb3961ae80 0x55fb39623430 secure :-1 s=READY pgs=1986 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0009230 tx=0x7f1fd002c650 comp rx=0 tx=0).stop 2024-04-21T08:50:14.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.115+0000 7f1fe95be740 1 -- 172.21.15.86:0/1140730538 shutdown_connections 2024-04-21T08:50:14.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.115+0000 7f1fe95be740 1 --2- 172.21.15.86:0/1140730538 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb39623a00 0x55fb39625e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.122 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.115+0000 7f1fe95be740 1 --2- 172.21.15.86:0/1140730538 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb3961ae80 0x55fb39623430 unknown :-1 s=CLOSED pgs=1986 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.115+0000 7f1fe95be740 1 --2- 172.21.15.86:0/1140730538 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb3961a4d0 0x55fb3961a8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.115+0000 7f1fe95be740 1 -- 172.21.15.86:0/1140730538 >> 172.21.15.86:0/1140730538 conn(0x55fb39614ba0 msgr2=0x55fb39614f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:14.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.115+0000 7f1fe95be740 1 -- 172.21.15.86:0/1140730538 shutdown_connections 2024-04-21T08:50:14.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.115+0000 7f1fe95be740 1 -- 172.21.15.86:0/1140730538 wait complete. 2024-04-21T08:50:14.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.115+0000 7f1fe95be740 1 -- start start 2024-04-21T08:50:14.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.116+0000 7f1fe95be740 1 Processor -- start 2024-04-21T08:50:14.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.116+0000 7f1fe95be740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb3961a4d0 0x55fb396232d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.116+0000 7f1fe95be740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb3961ae80 0x55fb39620660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.116+0000 7f1fe95be740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb39623a00 0x55fb39620bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.116+0000 7f1fe95be740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55fb39626910 con 0x55fb39623a00 2024-04-21T08:50:14.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.116+0000 7f1fe95be740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55fb39538500 con 0x55fb3961a4d0 2024-04-21T08:50:14.123 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.116+0000 7f1fe95be740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55fb39626a90 con 0x55fb3961ae80 2024-04-21T08:50:14.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.116+0000 7f1fe95be740 10 client.0 _ll_register_callbacks cb 0x55fb39506b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:14.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.116+0000 7f1fe4e8f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb39623a00 0x55fb39620bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:14.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.116+0000 7f1fe4e8f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb39623a00 0x55fb39620bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55394/0 (socket says 172.21.15.86:55394) 2024-04-21T08:50:14.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.116+0000 7f1fe4e8f640 1 -- 172.21.15.86:0/2415337316 learned_addr learned my addr 172.21.15.86:0/2415337316 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:14.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.116+0000 7f1fdf7fe640 1 --2- 172.21.15.86:0/2415337316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb3961ae80 0x55fb39620660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:14.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.116+0000 7f1fdffff640 1 --2- 172.21.15.86:0/2415337316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb3961a4d0 0x55fb396232d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:14.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.116+0000 7f1fe4e8f640 1 -- 172.21.15.86:0/2415337316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb3961ae80 msgr2=0x55fb39620660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:14.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.116+0000 7f1fe4e8f640 1 --2- 172.21.15.86:0/2415337316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb3961ae80 0x55fb39620660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.116+0000 7f1fe4e8f640 1 -- 172.21.15.86:0/2415337316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb3961a4d0 msgr2=0x55fb396232d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:14.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.116+0000 7f1fe4e8f640 1 --2- 172.21.15.86:0/2415337316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb3961a4d0 0x55fb396232d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.116+0000 7f1fe4e8f640 1 -- 172.21.15.86:0/2415337316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb392913f0 con 0x55fb39623a00 2024-04-21T08:50:14.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.116+0000 7f1fdffff640 1 --2- 172.21.15.86:0/2415337316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb3961a4d0 0x55fb396232d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:14.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.116+0000 7f1fdf7fe640 1 --2- 172.21.15.86:0/2415337316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb3961ae80 0x55fb39620660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:14.124 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.116+0000 7f1fe4e8f640 1 --2- 172.21.15.86:0/2415337316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb39623a00 0x55fb39620bc0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80128e0 tx=0x7f1fd8004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cd14db677450c545 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:14.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.116+0000 7f1fbf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:14.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.117+0000 7f1fbf7fe640 1 -- 172.21.15.86:0/2415337316 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8023070 con 0x55fb39623a00 2024-04-21T08:50:14.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.117+0000 7f1fbf7fe640 1 -- 172.21.15.86:0/2415337316 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f1fd8003b70 con 0x55fb39623a00 2024-04-21T08:50:14.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.117+0000 7f1fe95be740 1 -- 172.21.15.86:0/2415337316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fd002cc90 con 0x55fb39623a00 2024-04-21T08:50:14.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.117+0000 7f1fbf7fe640 1 -- 172.21.15.86:0/2415337316 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1fd802e050 con 0x55fb39623a00 2024-04-21T08:50:14.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.117+0000 7f1fe95be740 10 client.20190 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:14.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.117+0000 7f1fe95be740 1 -- 172.21.15.86:0/2415337316 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fb3935b7d0 con 0x55fb39623a00 2024-04-21T08:50:14.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.117+0000 7f1fbf7fe640 1 -- 172.21.15.86:0/2415337316 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fd002cc90 con 0x55fb39623a00 2024-04-21T08:50:14.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.117+0000 7f1fbf7fe640 1 -- 172.21.15.86:0/2415337316 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1fd800f040 con 0x55fb39623a00 2024-04-21T08:50:14.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.117+0000 7f1fe95be740 10 client.20190 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:14.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.117+0000 7f1fe95be740 10 client.20190 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:50:14.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.117+0000 7f1fe95be740 20 client.20190 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:14.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.117+0000 7f1fe95be740 1 --2- 172.21.15.86:0/2415337316 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55fb39633b30 0x55fb39635fa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.117+0000 7f1fe95be740 4 client.20190 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:14.125 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.117+0000 7f1fe95be740 1 -- 172.21.15.86:0/2415337316 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55fb3951e930 con 0x55fb39633b30 2024-04-21T08:50:14.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.118+0000 7f1fdffff640 1 --2- 172.21.15.86:0/2415337316 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55fb39633b30 0x55fb39635fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:14.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.118+0000 7f1fdffff640 1 --2- 172.21.15.86:0/2415337316 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55fb39633b30 0x55fb39635fa0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=907b533a5f3f5e28 server_cookie=b5351c048eb32ddd in_seq=0 out_seq=0 2024-04-21T08:50:14.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.118+0000 7f1fbf7fe640 10 client.20190 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:50:14.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.119+0000 7f1fbf7fe640 1 -- 172.21.15.86:0/2415337316 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55fb3951e930 con 0x55fb39633b30 2024-04-21T08:50:14.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.119+0000 7f1fbf7fe640 10 client.20190 handle_command_reply: tid=0 2024-04-21T08:50:14.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.119+0000 7f1fe95be740 1 client.20190 shutdown 2024-04-21T08:50:14.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.119+0000 7f1fe95be740 1 -- 172.21.15.86:0/2415337316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb39623a00 msgr2=0x55fb39620bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:14.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.119+0000 7f1fe95be740 1 --2- 172.21.15.86:0/2415337316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb39623a00 0x55fb39620bc0 secure :-1 s=READY pgs=1987 cs=0 l=1 rev1=1 crypto rx=0x7f1fd80128e0 tx=0x7f1fd8004a40 comp rx=0 tx=0).stop 2024-04-21T08:50:14.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.119+0000 7f1fe95be740 1 -- 172.21.15.86:0/2415337316 shutdown_connections 2024-04-21T08:50:14.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.119+0000 7f1fe95be740 1 --2- 172.21.15.86:0/2415337316 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55fb39633b30 0x55fb39635fa0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.119+0000 7f1fe95be740 1 --2- 172.21.15.86:0/2415337316 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb39623a00 0x55fb39620bc0 unknown :-1 s=CLOSED pgs=1987 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.119+0000 7f1fbf7fe640 0 client.20190 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:50:14.126 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.119+0000 7f1fe95be740 1 --2- 172.21.15.86:0/2415337316 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb3961ae80 0x55fb39620660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.119+0000 7f1fe95be740 1 --2- 172.21.15.86:0/2415337316 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb3961a4d0 0x55fb396232d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.119+0000 7f1fe95be740 1 -- 172.21.15.86:0/2415337316 >> 172.21.15.86:0/2415337316 conn(0x55fb39614ba0 msgr2=0x55fb396168f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:14.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.119+0000 7f1fe95be740 1 -- 172.21.15.86:0/2415337316 shutdown_connections 2024-04-21T08:50:14.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.119+0000 7f1fe95be740 1 -- 172.21.15.86:0/2415337316 wait complete. 2024-04-21T08:50:14.129 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.119+0000 7f1fe95be740 20 client.20190 trim_cache size 0 max 16384 2024-04-21T08:50:14.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.132+0000 7f1fe95be740 1 Processor -- start 2024-04-21T08:50:14.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.132+0000 7f1fe95be740 1 -- start start 2024-04-21T08:50:14.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.132+0000 7f1fe95be740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb3961a4d0 0x55fb3962f7d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.132+0000 7f1fe95be740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb3961ae80 0x55fb39616690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.132+0000 7f1fe95be740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb39623a00 0x55fb39622d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.132+0000 7f1fe95be740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe01af340 con 0x55fb3961a4d0 2024-04-21T08:50:14.137 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.132+0000 7f1fe95be740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe01b0e80 con 0x55fb39623a00 2024-04-21T08:50:14.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.132+0000 7f1fe95be740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe01afa10 con 0x55fb3961ae80 2024-04-21T08:50:14.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.132+0000 7f1fdffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb3961a4d0 0x55fb3962f7d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:14.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.132+0000 7f1fdffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb3961a4d0 0x55fb3962f7d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55408/0 (socket says 172.21.15.86:55408) 2024-04-21T08:50:14.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.132+0000 7f1fdffff640 1 -- 172.21.15.86:0/2909101908 learned_addr learned my addr 172.21.15.86:0/2909101908 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:14.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.132+0000 7f1fdf7fe640 1 --2- 172.21.15.86:0/2909101908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb3961ae80 0x55fb39616690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:14.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.132+0000 7f1fdffff640 1 -- 172.21.15.86:0/2909101908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb3961ae80 msgr2=0x55fb39616690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:14.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.132+0000 7f1fdffff640 1 --2- 172.21.15.86:0/2909101908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb3961ae80 0x55fb39616690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.132+0000 7f1fdffff640 1 -- 172.21.15.86:0/2909101908 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb39623a00 msgr2=0x55fb39622d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:14.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.132+0000 7f1fdffff640 1 --2- 172.21.15.86:0/2909101908 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb39623a00 0x55fb39622d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.132+0000 7f1fdffff640 1 -- 172.21.15.86:0/2909101908 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55fb39632730 con 0x55fb3961a4d0 2024-04-21T08:50:14.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.133+0000 7f1fdffff640 1 --2- 172.21.15.86:0/2909101908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb3961a4d0 0x55fb3962f7d0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0009900 tx=0x7f1fd0005500 comp rx=0 tx=0).ready entity=mon.0 client_cookie=dcd52955e9c867de server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:14.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.133+0000 7f1fc5ffb640 1 -- 172.21.15.86:0/2909101908 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0044040 con 0x55fb3961a4d0 2024-04-21T08:50:14.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.133+0000 7f1fc5ffb640 1 -- 172.21.15.86:0/2909101908 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fd002cc60 con 0x55fb3961a4d0 2024-04-21T08:50:14.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.133+0000 7f1fc5ffb640 1 -- 172.21.15.86:0/2909101908 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd0053660 con 0x55fb3961a4d0 2024-04-21T08:50:14.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.133+0000 7f1fe95be740 1 -- 172.21.15.86:0/2909101908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb3961a4d0 msgr2=0x55fb3962f7d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:14.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.133+0000 7f1fe95be740 1 --2- 172.21.15.86:0/2909101908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb3961a4d0 0x55fb3962f7d0 secure :-1 s=READY pgs=1988 cs=0 l=1 rev1=1 crypto rx=0x7f1fd0009900 tx=0x7f1fd0005500 comp rx=0 tx=0).stop 2024-04-21T08:50:14.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.133+0000 7f1fe95be740 1 -- 172.21.15.86:0/2909101908 shutdown_connections 2024-04-21T08:50:14.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.133+0000 7f1fe95be740 1 --2- 172.21.15.86:0/2909101908 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb39623a00 0x55fb39622d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.133+0000 7f1fe95be740 1 --2- 172.21.15.86:0/2909101908 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb3961ae80 0x55fb39616690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.133+0000 7f1fe95be740 1 --2- 172.21.15.86:0/2909101908 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb3961a4d0 0x55fb3962f7d0 unknown :-1 s=CLOSED pgs=1988 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.133+0000 7f1fe95be740 1 -- 172.21.15.86:0/2909101908 >> 172.21.15.86:0/2909101908 conn(0x55fb39614ba0 msgr2=0x55fb39615f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:14.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.133+0000 7f1fe95be740 1 -- 172.21.15.86:0/2909101908 shutdown_connections 2024-04-21T08:50:14.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.133+0000 7f1fe95be740 1 -- 172.21.15.86:0/2909101908 wait complete. 2024-04-21T08:50:14.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.134+0000 7f1fe95be740 1 -- start start 2024-04-21T08:50:14.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.134+0000 7f1fe95be740 1 Processor -- start 2024-04-21T08:50:14.139 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.134+0000 7f1fe95be740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb3961a4d0 0x55fb3961d500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.134+0000 7f1fe95be740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb3961ae80 0x55fb3961da40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.134+0000 7f1fe95be740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb39623a00 0x55fb3961df80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.134+0000 7f1fe95be740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe01b07b0 con 0x55fb39623a00 2024-04-21T08:50:14.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.134+0000 7f1fe95be740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe01b00e0 con 0x55fb3961a4d0 2024-04-21T08:50:14.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.134+0000 7f1fe95be740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1fe01ad340 con 0x55fb3961ae80 2024-04-21T08:50:14.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.134+0000 7f1fe95be740 10 client.0 _ll_register_callbacks cb 0x7f1fd8012590 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:14.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.134+0000 7f1fe4e8f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb39623a00 0x55fb3961df80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:14.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.134+0000 7f1fe4e8f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb39623a00 0x55fb3961df80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55416/0 (socket says 172.21.15.86:55416) 2024-04-21T08:50:14.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.134+0000 7f1fe4e8f640 1 -- 172.21.15.86:0/3887151539 learned_addr learned my addr 172.21.15.86:0/3887151539 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:14.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.134+0000 7f1fdf7fe640 1 --2- 172.21.15.86:0/3887151539 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb3961ae80 0x55fb3961da40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:14.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.134+0000 7f1fdffff640 1 --2- 172.21.15.86:0/3887151539 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb3961a4d0 0x55fb3961d500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:14.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.134+0000 7f1fe4e8f640 1 -- 172.21.15.86:0/3887151539 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb3961ae80 msgr2=0x55fb3961da40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:14.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.134+0000 7f1fe4e8f640 1 --2- 172.21.15.86:0/3887151539 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb3961ae80 0x55fb3961da40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.140 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.134+0000 7f1fe4e8f640 1 -- 172.21.15.86:0/3887151539 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb3961a4d0 msgr2=0x55fb3961d500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:14.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.134+0000 7f1fe4e8f640 1 --2- 172.21.15.86:0/3887151539 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb3961a4d0 0x55fb3961d500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.134+0000 7f1fe4e8f640 1 -- 172.21.15.86:0/3887151539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55fb39699290 con 0x55fb39623a00 2024-04-21T08:50:14.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.135+0000 7f1fe4e8f640 1 --2- 172.21.15.86:0/3887151539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb39623a00 0x55fb3961df80 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8012fd0 tx=0x7f1fd800c930 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a985d4547c5f3ef server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:14.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.135+0000 7f1fbf7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:14.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.135+0000 7f1fbf7fe640 1 -- 172.21.15.86:0/3887151539 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1fd8036070 con 0x55fb39623a00 2024-04-21T08:50:14.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.135+0000 7f1fe95be740 1 -- 172.21.15.86:0/3887151539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f1fd002cc60 con 0x55fb39623a00 2024-04-21T08:50:14.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.135+0000 7f1fbf7fe640 1 -- 172.21.15.86:0/3887151539 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f1fd801d550 con 0x55fb39623a00 2024-04-21T08:50:14.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.135+0000 7f1fbf7fe640 1 -- 172.21.15.86:0/3887151539 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f1fd8042050 con 0x55fb39623a00 2024-04-21T08:50:14.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.135+0000 7f1fe95be740 10 client.20196 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:14.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.135+0000 7f1fe95be740 1 -- 172.21.15.86:0/3887151539 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55fb3969a570 con 0x55fb39623a00 2024-04-21T08:50:14.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.135+0000 7f1fbf7fe640 1 -- 172.21.15.86:0/3887151539 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1fd002cc60 con 0x55fb39623a00 2024-04-21T08:50:14.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.135+0000 7f1fbf7fe640 1 -- 172.21.15.86:0/3887151539 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f1fd8003970 con 0x55fb39623a00 2024-04-21T08:50:14.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.135+0000 7f1fe95be740 10 client.20196 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:14.141 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.135+0000 7f1fe95be740 10 client.20196 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:50:14.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.135+0000 7f1fe95be740 20 client.20196 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:14.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.135+0000 7f1fe95be740 1 --2- 172.21.15.86:0/3887151539 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55fb3969ac90 0x55fb3969d0b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.135+0000 7f1fe95be740 4 client.20196 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:50:14.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.135+0000 7f1fe95be740 1 -- 172.21.15.86:0/3887151539 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55fb3958cc10 con 0x55fb3969ac90 2024-04-21T08:50:14.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.136+0000 7f1fdffff640 1 --2- 172.21.15.86:0/3887151539 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55fb3969ac90 0x55fb3969d0b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:14.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.136+0000 7f1fdffff640 1 --2- 172.21.15.86:0/3887151539 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55fb3969ac90 0x55fb3969d0b0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=683c4192f6fae0a8 server_cookie=cbd9d4d5e4e46973 in_seq=0 out_seq=0 2024-04-21T08:50:14.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.136+0000 7f1fbf7fe640 10 client.20196 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:50:14.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.136+0000 7f1fbf7fe640 1 -- 172.21.15.86:0/3887151539 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55fb3958cc10 con 0x55fb3969ac90 2024-04-21T08:50:14.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.136+0000 7f1fbf7fe640 10 client.20196 handle_command_reply: tid=0 2024-04-21T08:50:14.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.136+0000 7f1fe95be740 1 client.20196 shutdown 2024-04-21T08:50:14.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.136+0000 7f1fe95be740 1 -- 172.21.15.86:0/3887151539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb39623a00 msgr2=0x55fb3961df80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:14.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.136+0000 7f1fe95be740 1 --2- 172.21.15.86:0/3887151539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb39623a00 0x55fb3961df80 secure :-1 s=READY pgs=1989 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8012fd0 tx=0x7f1fd800c930 comp rx=0 tx=0).stop 2024-04-21T08:50:14.142 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.137+0000 7f1fe95be740 1 -- 172.21.15.86:0/3887151539 shutdown_connections 2024-04-21T08:50:14.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.137+0000 7f1fe95be740 1 --2- 172.21.15.86:0/3887151539 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55fb3969ac90 0x55fb3969d0b0 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.137+0000 7f1fe95be740 1 --2- 172.21.15.86:0/3887151539 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55fb39623a00 0x55fb3961df80 unknown :-1 s=CLOSED pgs=1989 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.137+0000 7f1fe95be740 1 --2- 172.21.15.86:0/3887151539 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55fb3961ae80 0x55fb3961da40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.137+0000 7f1fe95be740 1 --2- 172.21.15.86:0/3887151539 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55fb3961a4d0 0x55fb3961d500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.137+0000 7f1fe95be740 1 -- 172.21.15.86:0/3887151539 >> 172.21.15.86:0/3887151539 conn(0x55fb39614ba0 msgr2=0x55fb39613c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:14.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.137+0000 7f1fbf7fe640 0 client.20196 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:50:14.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.137+0000 7f1fe95be740 1 -- 172.21.15.86:0/3887151539 shutdown_connections 2024-04-21T08:50:14.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.137+0000 7f1fe95be740 1 -- 172.21.15.86:0/3887151539 wait complete. 2024-04-21T08:50:14.143 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.137+0000 7f1fe95be740 20 client.20196 trim_cache size 0 max 16384 2024-04-21T08:50:14.143 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:50:14.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.137+0000 7f1fe68f2640 1 -- 172.21.15.86:0/3268520646 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1fc80a3d20 msgr2=0x7f1fc80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:14.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.137+0000 7f1fe68f2640 1 --2- 172.21.15.86:0/3268520646 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1fc80a3d20 0x7f1fc80a61e0 secure :-1 s=READY pgs=331 cs=0 l=1 rev1=1 crypto rx=0x7f1fd8011440 tx=0x7f1fd8005e90 comp rx=0 tx=0).stop 2024-04-21T08:50:14.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.137+0000 7f1fe68f2640 1 -- 172.21.15.86:0/3268520646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe01072b0 msgr2=0x7f1fe0198ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:14.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.137+0000 7f1fe68f2640 1 --2- 172.21.15.86:0/3268520646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe01072b0 0x7f1fe0198ca0 secure :-1 s=READY pgs=1985 cs=0 l=1 rev1=1 crypto rx=0x7f1fd4005fd0 tx=0x7f1fd400dde0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.144 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.137+0000 7f1fe68f2640 1 -- 172.21.15.86:0/3268520646 shutdown_connections 2024-04-21T08:50:14.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.137+0000 7f1fe68f2640 1 --2- 172.21.15.86:0/3268520646 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1fc80a3d20 0x7f1fc80a61e0 unknown :-1 s=CLOSED pgs=331 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.137+0000 7f1fe68f2640 1 --2- 172.21.15.86:0/3268520646 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1fe019d020 0x7f1fe019d4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.137+0000 7f1fe68f2640 1 --2- 172.21.15.86:0/3268520646 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1fe01072b0 0x7f1fe0198ca0 unknown :-1 s=CLOSED pgs=1985 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.137+0000 7f1fe68f2640 1 --2- 172.21.15.86:0/3268520646 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1fe0100af0 0x7f1fe0198760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.137+0000 7f1fe68f2640 1 -- 172.21.15.86:0/3268520646 >> 172.21.15.86:0/3268520646 conn(0x7f1fe00760e0 msgr2=0x7f1fe00fead0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:14.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.137+0000 7f1fe68f2640 1 -- 172.21.15.86:0/3268520646 shutdown_connections 2024-04-21T08:50:14.145 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.137+0000 7f1fe68f2640 1 -- 172.21.15.86:0/3268520646 wait complete. 2024-04-21T08:50:14.146 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:13 smithi086 ceph-mon[27939]: pgmap v358: 129 pgs: 129 active+clean; 5.4 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 63 KiB/s wr, 24 op/s 2024-04-21T08:50:14.146 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:13 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1274474318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:50:14.147 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:50:14.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:13 smithi088 ceph-mon[31259]: pgmap v358: 129 pgs: 129 active+clean; 5.4 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 63 KiB/s wr, 24 op/s 2024-04-21T08:50:14.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:13 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1274474318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:50:14.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.246+0000 7f8e15315640 1 -- 172.21.15.86:0/719317708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e1006a860 msgr2=0x7f8e1006ac40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:14.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.246+0000 7f8e15315640 1 --2- 172.21.15.86:0/719317708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e1006a860 0x7f8e1006ac40 secure :-1 s=READY pgs=1990 cs=0 l=1 rev1=1 crypto rx=0x7f8e00009ca0 tx=0x7f8e0002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.247+0000 7f8e15315640 1 -- 172.21.15.86:0/719317708 shutdown_connections 2024-04-21T08:50:14.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.247+0000 7f8e15315640 1 --2- 172.21.15.86:0/719317708 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8e10107270 0x7f8e1010ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.247+0000 7f8e15315640 1 --2- 172.21.15.86:0/719317708 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e1006b210 0x7f8e10106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.248 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.247+0000 7f8e15315640 1 --2- 172.21.15.86:0/719317708 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e1006a860 0x7f8e1006ac40 unknown :-1 s=CLOSED pgs=1990 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.247+0000 7f8e15315640 1 -- 172.21.15.86:0/719317708 >> 172.21.15.86:0/719317708 conn(0x7f8e10077eb0 msgr2=0x7f8e100782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:14.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.247+0000 7f8e15315640 1 -- 172.21.15.86:0/719317708 shutdown_connections 2024-04-21T08:50:14.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.247+0000 7f8e15315640 1 -- 172.21.15.86:0/719317708 wait complete. 2024-04-21T08:50:14.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.247+0000 7f8e15315640 1 Processor -- start 2024-04-21T08:50:14.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.247+0000 7f8e15315640 1 -- start start 2024-04-21T08:50:14.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.247+0000 7f8e15315640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e1006a860 0x7f8e101986b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.248+0000 7f8e15315640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e10107270 0x7f8e10198bf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.248+0000 7f8e15315640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8e1019cf70 0x7f8e1019d420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.248+0000 7f8e15315640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8e101121b0 con 0x7f8e10107270 2024-04-21T08:50:14.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.248+0000 7f8e15315640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8e10112030 con 0x7f8e1019cf70 2024-04-21T08:50:14.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.248+0000 7f8e15315640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8e10112330 con 0x7f8e1006a860 2024-04-21T08:50:14.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.248+0000 7f8e0e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e10107270 0x7f8e10198bf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:14.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.248+0000 7f8e0e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e10107270 0x7f8e10198bf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55424/0 (socket says 172.21.15.86:55424) 2024-04-21T08:50:14.249 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.248+0000 7f8e0e7fc640 1 -- 172.21.15.86:0/482530074 learned_addr learned my addr 172.21.15.86:0/482530074 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:14.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.248+0000 7f8e0effd640 1 --2- 172.21.15.86:0/482530074 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e1006a860 0x7f8e101986b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:14.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.248+0000 7f8e0f7fe640 1 --2- 172.21.15.86:0/482530074 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8e1019cf70 0x7f8e1019d420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:14.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.248+0000 7f8e0e7fc640 1 -- 172.21.15.86:0/482530074 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e1006a860 msgr2=0x7f8e101986b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:14.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.248+0000 7f8e0e7fc640 1 --2- 172.21.15.86:0/482530074 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e1006a860 0x7f8e101986b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.248+0000 7f8e0e7fc640 1 -- 172.21.15.86:0/482530074 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8e1019cf70 msgr2=0x7f8e1019d420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:14.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.248+0000 7f8e0e7fc640 1 --2- 172.21.15.86:0/482530074 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8e1019cf70 0x7f8e1019d420 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.248+0000 7f8e0e7fc640 1 -- 172.21.15.86:0/482530074 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8e1019da60 con 0x7f8e10107270 2024-04-21T08:50:14.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.248+0000 7f8e0e7fc640 1 --2- 172.21.15.86:0/482530074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e10107270 0x7f8e10198bf0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f8e0400d770 tx=0x7f8e0400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:14.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.248+0000 7f8df7fff640 1 -- 172.21.15.86:0/482530074 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e04017070 con 0x7f8e10107270 2024-04-21T08:50:14.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.248+0000 7f8e15315640 1 -- 172.21.15.86:0/482530074 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8e1019dd50 con 0x7f8e10107270 2024-04-21T08:50:14.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.248+0000 7f8df7fff640 1 -- 172.21.15.86:0/482530074 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e04009c30 con 0x7f8e10107270 2024-04-21T08:50:14.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.248+0000 7f8df7fff640 1 -- 172.21.15.86:0/482530074 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0400f620 con 0x7f8e10107270 2024-04-21T08:50:14.250 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.248+0000 7f8e15315640 1 -- 172.21.15.86:0/482530074 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8e101a5540 con 0x7f8e10107270 2024-04-21T08:50:14.251 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:13 smithi179 ceph-mon[31829]: pgmap v358: 129 pgs: 129 active+clean; 5.4 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 62 KiB/s rd, 63 KiB/s wr, 24 op/s 2024-04-21T08:50:14.251 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:13 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1274474318' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:50:14.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.251+0000 7f8df7fff640 1 -- 172.21.15.86:0/482530074 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8e0401a070 con 0x7f8e10107270 2024-04-21T08:50:14.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.251+0000 7f8e17fe1740 1 Processor -- start 2024-04-21T08:50:14.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.251+0000 7f8e17fe1740 1 -- start start 2024-04-21T08:50:14.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.251+0000 7f8e17fe1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565321724d0 0x5565321728b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.251+0000 7f8e17fe1740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556532172e80 0x55653217b430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.251+0000 7f8e17fe1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55653217ba00 0x55653217de50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.251+0000 7f8e17fe1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5565320d71f0 con 0x5565321724d0 2024-04-21T08:50:14.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.251+0000 7f8e17fe1740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5565320744b0 con 0x556532172e80 2024-04-21T08:50:14.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.251+0000 7f8e17fe1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5565320d31b0 con 0x55653217ba00 2024-04-21T08:50:14.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.251+0000 7f8df7fff640 1 --2- 172.21.15.86:0/482530074 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8df80a3d20 0x7f8df80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.251+0000 7f8e0f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565321724d0 0x5565321728b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:14.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.251+0000 7f8e0f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565321724d0 0x5565321728b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55436/0 (socket says 172.21.15.86:55436) 2024-04-21T08:50:14.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.251+0000 7f8e0f7fe640 1 -- 172.21.15.86:0/3768000271 learned_addr learned my addr 172.21.15.86:0/3768000271 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:14.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.251+0000 7f8e0effd640 1 --2- 172.21.15.86:0/3768000271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556532172e80 0x55653217b430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:14.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.251+0000 7f8e0f7fe640 1 --2- 172.21.15.86:0/482530074 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8df80a3d20 0x7f8df80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:14.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.251+0000 7f8df7fff640 1 -- 172.21.15.86:0/482530074 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f8e040d0740 con 0x7f8e10107270 2024-04-21T08:50:14.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.251+0000 7f8e0e7fc640 1 --2- 172.21.15.86:0/3768000271 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55653217ba00 0x55653217de50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:14.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.252+0000 7f8e0f7fe640 1 -- 172.21.15.86:0/3768000271 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55653217ba00 msgr2=0x55653217de50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:14.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.252+0000 7f8e0f7fe640 1 --2- 172.21.15.86:0/3768000271 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55653217ba00 0x55653217de50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.252+0000 7f8e0f7fe640 1 -- 172.21.15.86:0/3768000271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556532172e80 msgr2=0x55653217b430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:14.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.252+0000 7f8e0f7fe640 1 --2- 172.21.15.86:0/3768000271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556532172e80 0x55653217b430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.252+0000 7f8e0f7fe640 1 -- 172.21.15.86:0/3768000271 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5565320661b0 con 0x5565321724d0 2024-04-21T08:50:14.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.252+0000 7f8e0f7fe640 1 --2- 172.21.15.86:0/3768000271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565321724d0 0x5565321728b0 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f8e080129d0 tx=0x7f8e08012e90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a9085add22ec1575 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:14.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.252+0000 7f8e0f7fe640 1 --2- 172.21.15.86:0/482530074 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8df80a3d20 0x7f8df80a61e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8e0800d720 tx=0x7f8e08014000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:14.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.252+0000 7f8df57fa640 1 -- 172.21.15.86:0/3768000271 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e08020070 con 0x5565321724d0 2024-04-21T08:50:14.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.252+0000 7f8df57fa640 1 -- 172.21.15.86:0/3768000271 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e0800f7c0 con 0x5565321724d0 2024-04-21T08:50:14.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.252+0000 7f8df57fa640 1 -- 172.21.15.86:0/3768000271 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e0801cba0 con 0x5565321724d0 2024-04-21T08:50:14.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.253+0000 7f8e17fe1740 1 -- 172.21.15.86:0/3768000271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565321724d0 msgr2=0x5565321728b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:14.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.253+0000 7f8e17fe1740 1 --2- 172.21.15.86:0/3768000271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565321724d0 0x5565321728b0 secure :-1 s=READY pgs=1992 cs=0 l=1 rev1=1 crypto rx=0x7f8e080129d0 tx=0x7f8e08012e90 comp rx=0 tx=0).stop 2024-04-21T08:50:14.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.253+0000 7f8e17fe1740 1 -- 172.21.15.86:0/3768000271 shutdown_connections 2024-04-21T08:50:14.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.253+0000 7f8e17fe1740 1 --2- 172.21.15.86:0/3768000271 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55653217ba00 0x55653217de50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.253+0000 7f8e17fe1740 1 --2- 172.21.15.86:0/3768000271 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556532172e80 0x55653217b430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.253+0000 7f8e17fe1740 1 --2- 172.21.15.86:0/3768000271 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565321724d0 0x5565321728b0 unknown :-1 s=CLOSED pgs=1992 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.253+0000 7f8e17fe1740 1 -- 172.21.15.86:0/3768000271 >> 172.21.15.86:0/3768000271 conn(0x55653216cba0 msgr2=0x55653216cf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:14.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.253+0000 7f8e17fe1740 1 -- 172.21.15.86:0/3768000271 shutdown_connections 2024-04-21T08:50:14.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.253+0000 7f8e17fe1740 1 -- 172.21.15.86:0/3768000271 wait complete. 2024-04-21T08:50:14.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.253+0000 7f8e17fe1740 1 -- start start 2024-04-21T08:50:14.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.253+0000 7f8e17fe1740 1 Processor -- start 2024-04-21T08:50:14.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.253+0000 7f8e17fe1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565321724d0 0x55653217b400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.253+0000 7f8e17fe1740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556532172e80 0x556532178790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.253+0000 7f8e17fe1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55653217ba00 0x556532178e80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.253+0000 7f8e17fe1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55653217e910 con 0x5565321724d0 2024-04-21T08:50:14.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.253+0000 7f8e17fe1740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x556532090500 con 0x556532172e80 2024-04-21T08:50:14.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.253+0000 7f8e17fe1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55653217ea90 con 0x55653217ba00 2024-04-21T08:50:14.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.253+0000 7f8e17fe1740 10 client.0 _ll_register_callbacks cb 0x55653205eb80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:14.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.253+0000 7f8e0effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565321724d0 0x55653217b400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:14.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.254+0000 7f8e0effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565321724d0 0x55653217b400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55446/0 (socket says 172.21.15.86:55446) 2024-04-21T08:50:14.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.254+0000 7f8e0effd640 1 -- 172.21.15.86:0/1573383215 learned_addr learned my addr 172.21.15.86:0/1573383215 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:14.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.254+0000 7f8e0e7fc640 1 --2- 172.21.15.86:0/1573383215 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556532172e80 0x556532178790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:14.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.254+0000 7f8e0f7fe640 1 --2- 172.21.15.86:0/1573383215 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55653217ba00 0x556532178e80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:14.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.254+0000 7f8e0effd640 1 -- 172.21.15.86:0/1573383215 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55653217ba00 msgr2=0x556532178e80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:14.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.254+0000 7f8e0effd640 1 --2- 172.21.15.86:0/1573383215 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55653217ba00 0x556532178e80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.261 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.254+0000 7f8e0effd640 1 -- 172.21.15.86:0/1573383215 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556532172e80 msgr2=0x556532178790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:14.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.254+0000 7f8e0effd640 1 --2- 172.21.15.86:0/1573383215 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556532172e80 0x556532178790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.254+0000 7f8e0effd640 1 -- 172.21.15.86:0/1573383215 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556531de93f0 con 0x5565321724d0 2024-04-21T08:50:14.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.254+0000 7f8e0effd640 1 --2- 172.21.15.86:0/1573383215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565321724d0 0x55653217b400 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f8e0002c700 tx=0x7f8e00032d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7ca4a3f5477fcd90 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:14.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.254+0000 7f8df2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:14.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.254+0000 7f8df2ff5640 1 -- 172.21.15.86:0/1573383215 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e00044070 con 0x5565321724d0 2024-04-21T08:50:14.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.254+0000 7f8df2ff5640 1 -- 172.21.15.86:0/1573383215 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f8e000328d0 con 0x5565321724d0 2024-04-21T08:50:14.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.254+0000 7f8e17fe1740 1 -- 172.21.15.86:0/1573383215 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e0800f7c0 con 0x5565321724d0 2024-04-21T08:50:14.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.254+0000 7f8df2ff5640 1 -- 172.21.15.86:0/1573383215 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8e00051050 con 0x5565321724d0 2024-04-21T08:50:14.262 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.254+0000 7f8e17fe1740 10 client.20208 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:14.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.254+0000 7f8e17fe1740 1 -- 172.21.15.86:0/1573383215 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556531eb37d0 con 0x5565321724d0 2024-04-21T08:50:14.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.254+0000 7f8df2ff5640 1 -- 172.21.15.86:0/1573383215 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e0800f7c0 con 0x5565321724d0 2024-04-21T08:50:14.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.255+0000 7f8df2ff5640 1 -- 172.21.15.86:0/1573383215 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8e00004680 con 0x5565321724d0 2024-04-21T08:50:14.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.255+0000 7f8e17fe1740 10 client.20208 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:14.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.255+0000 7f8e17fe1740 10 client.20208 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:14.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.255+0000 7f8e17fe1740 20 client.20208 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:14.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.255+0000 7f8e17fe1740 1 --2- 172.21.15.86:0/1573383215 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55653218bce0 0x55653218e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.255+0000 7f8e17fe1740 4 client.20208 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:14.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.255+0000 7f8e17fe1740 1 -- 172.21.15.86:0/1573383215 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556532076930 con 0x55653218bce0 2024-04-21T08:50:14.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.255+0000 7f8e0e7fc640 1 --2- 172.21.15.86:0/1573383215 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55653218bce0 0x55653218e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:14.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.255+0000 7f8df2ff5640 10 client.20208 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:14.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.255+0000 7f8e0e7fc640 1 --2- 172.21.15.86:0/1573383215 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55653218bce0 0x55653218e0f0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=dd8295595a2f6c13 server_cookie=c24de410063b746c in_seq=0 out_seq=0 2024-04-21T08:50:14.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.256+0000 7f8df2ff5640 1 -- 172.21.15.86:0/1573383215 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556532076930 con 0x55653218bce0 2024-04-21T08:50:14.263 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.256+0000 7f8df2ff5640 10 client.20208 handle_command_reply: tid=0 2024-04-21T08:50:14.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.256+0000 7f8e17fe1740 1 client.20208 shutdown 2024-04-21T08:50:14.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.256+0000 7f8e17fe1740 1 -- 172.21.15.86:0/1573383215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565321724d0 msgr2=0x55653217b400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:14.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.256+0000 7f8e17fe1740 1 --2- 172.21.15.86:0/1573383215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565321724d0 0x55653217b400 secure :-1 s=READY pgs=1993 cs=0 l=1 rev1=1 crypto rx=0x7f8e0002c700 tx=0x7f8e00032d00 comp rx=0 tx=0).stop 2024-04-21T08:50:14.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.256+0000 7f8e17fe1740 1 -- 172.21.15.86:0/1573383215 shutdown_connections 2024-04-21T08:50:14.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.256+0000 7f8e17fe1740 1 --2- 172.21.15.86:0/1573383215 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55653218bce0 0x55653218e0f0 crc :-1 s=READY pgs=327 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.256+0000 7f8e17fe1740 1 --2- 172.21.15.86:0/1573383215 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55653217ba00 0x556532178e80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.256+0000 7f8e17fe1740 1 --2- 172.21.15.86:0/1573383215 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556532172e80 0x556532178790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.256+0000 7f8df2ff5640 0 client.20208 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:14.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.256+0000 7f8e17fe1740 1 --2- 172.21.15.86:0/1573383215 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565321724d0 0x55653217b400 unknown :-1 s=CLOSED pgs=1993 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.256+0000 7f8e17fe1740 1 -- 172.21.15.86:0/1573383215 >> 172.21.15.86:0/1573383215 conn(0x55653216cba0 msgr2=0x55653216e8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:14.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.256+0000 7f8e17fe1740 1 -- 172.21.15.86:0/1573383215 shutdown_connections 2024-04-21T08:50:14.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.256+0000 7f8e17fe1740 1 -- 172.21.15.86:0/1573383215 wait complete. 2024-04-21T08:50:14.264 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.256+0000 7f8e17fe1740 20 client.20208 trim_cache size 0 max 16384 2024-04-21T08:50:14.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.269+0000 7f8e17fe1740 1 Processor -- start 2024-04-21T08:50:14.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.269+0000 7f8e17fe1740 1 -- start start 2024-04-21T08:50:14.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.269+0000 7f8e17fe1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565321724d0 0x5565321877d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.269+0000 7f8e17fe1740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556532172e80 0x55653216e6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.269+0000 7f8e17fe1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55653217ba00 0x55653217aec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.269+0000 7f8e17fe1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8e101af220 con 0x5565321724d0 2024-04-21T08:50:14.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.269+0000 7f8e17fe1740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8e101b0d60 con 0x556532172e80 2024-04-21T08:50:14.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.269+0000 7f8e17fe1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8e101af8f0 con 0x55653217ba00 2024-04-21T08:50:14.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.269+0000 7f8e0effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565321724d0 0x5565321877d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:14.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.269+0000 7f8e0effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565321724d0 0x5565321877d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55452/0 (socket says 172.21.15.86:55452) 2024-04-21T08:50:14.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.269+0000 7f8e0effd640 1 -- 172.21.15.86:0/3854032416 learned_addr learned my addr 172.21.15.86:0/3854032416 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:14.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.270+0000 7f8e0e7fc640 1 --2- 172.21.15.86:0/3854032416 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556532172e80 0x55653216e6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:14.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.270+0000 7f8e0effd640 1 -- 172.21.15.86:0/3854032416 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55653217ba00 msgr2=0x55653217aec0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:14.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.270+0000 7f8e0f7fe640 1 --2- 172.21.15.86:0/3854032416 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55653217ba00 0x55653217aec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:14.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.270+0000 7f8e0effd640 1 --2- 172.21.15.86:0/3854032416 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55653217ba00 0x55653217aec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.270+0000 7f8e0effd640 1 -- 172.21.15.86:0/3854032416 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556532172e80 msgr2=0x55653216e6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:14.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.270+0000 7f8e0effd640 1 --2- 172.21.15.86:0/3854032416 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556532172e80 0x55653216e6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.270+0000 7f8e0effd640 1 -- 172.21.15.86:0/3854032416 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55653217e960 con 0x5565321724d0 2024-04-21T08:50:14.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.270+0000 7f8e0e7fc640 1 --2- 172.21.15.86:0/3854032416 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556532172e80 0x55653216e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:14.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.270+0000 7f8e0effd640 1 --2- 172.21.15.86:0/3854032416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565321724d0 0x5565321877d0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f8e00009dc0 tx=0x7f8e00032d70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=275bfa5ff1b62b6c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:14.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.270+0000 7f8df4ff9640 1 -- 172.21.15.86:0/3854032416 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e00044070 con 0x5565321724d0 2024-04-21T08:50:14.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.270+0000 7f8df4ff9640 1 -- 172.21.15.86:0/3854032416 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e000326d0 con 0x5565321724d0 2024-04-21T08:50:14.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.270+0000 7f8df4ff9640 1 -- 172.21.15.86:0/3854032416 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e00039040 con 0x5565321724d0 2024-04-21T08:50:14.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.271+0000 7f8e17fe1740 1 -- 172.21.15.86:0/3854032416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565321724d0 msgr2=0x5565321877d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:14.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.271+0000 7f8e17fe1740 1 --2- 172.21.15.86:0/3854032416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565321724d0 0x5565321877d0 secure :-1 s=READY pgs=1994 cs=0 l=1 rev1=1 crypto rx=0x7f8e00009dc0 tx=0x7f8e00032d70 comp rx=0 tx=0).stop 2024-04-21T08:50:14.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.271+0000 7f8e17fe1740 1 -- 172.21.15.86:0/3854032416 shutdown_connections 2024-04-21T08:50:14.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.271+0000 7f8e17fe1740 1 --2- 172.21.15.86:0/3854032416 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55653217ba00 0x55653217aec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.271+0000 7f8e17fe1740 1 --2- 172.21.15.86:0/3854032416 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556532172e80 0x55653216e6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.271+0000 7f8e17fe1740 1 --2- 172.21.15.86:0/3854032416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5565321724d0 0x5565321877d0 unknown :-1 s=CLOSED pgs=1994 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.271+0000 7f8e17fe1740 1 -- 172.21.15.86:0/3854032416 >> 172.21.15.86:0/3854032416 conn(0x55653216cba0 msgr2=0x55653216dfc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:14.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.271+0000 7f8e17fe1740 1 -- 172.21.15.86:0/3854032416 shutdown_connections 2024-04-21T08:50:14.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.271+0000 7f8e17fe1740 1 -- 172.21.15.86:0/3854032416 wait complete. 2024-04-21T08:50:14.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.271+0000 7f8e17fe1740 1 -- start start 2024-04-21T08:50:14.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.271+0000 7f8e17fe1740 1 Processor -- start 2024-04-21T08:50:14.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.271+0000 7f8e17fe1740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5565321724d0 0x5565321755e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.271+0000 7f8e17fe1740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556532172e80 0x556532175b20 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.271+0000 7f8e17fe1740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55653217ba00 0x556532176060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.271+0000 7f8e17fe1740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8e101b0690 con 0x55653217ba00 2024-04-21T08:50:14.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.271+0000 7f8e17fe1740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8e101affc0 con 0x556532172e80 2024-04-21T08:50:14.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.271+0000 7f8e17fe1740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8e101ad220 con 0x5565321724d0 2024-04-21T08:50:14.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.271+0000 7f8e17fe1740 10 client.0 _ll_register_callbacks cb 0x7f8e00009930 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:14.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.271+0000 7f8e0f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55653217ba00 0x556532176060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:14.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.271+0000 7f8e0f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55653217ba00 0x556532176060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55464/0 (socket says 172.21.15.86:55464) 2024-04-21T08:50:14.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.271+0000 7f8e0f7fe640 1 -- 172.21.15.86:0/279438717 learned_addr learned my addr 172.21.15.86:0/279438717 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:14.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.272+0000 7f8e0effd640 1 --2- 172.21.15.86:0/279438717 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5565321724d0 0x5565321755e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:14.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.272+0000 7f8e0e7fc640 1 --2- 172.21.15.86:0/279438717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556532172e80 0x556532175b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:14.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.272+0000 7f8e0f7fe640 1 -- 172.21.15.86:0/279438717 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5565321724d0 msgr2=0x5565321755e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:14.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.272+0000 7f8e0f7fe640 1 --2- 172.21.15.86:0/279438717 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5565321724d0 0x5565321755e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.272+0000 7f8e0f7fe640 1 -- 172.21.15.86:0/279438717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556532172e80 msgr2=0x556532175b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:14.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.272+0000 7f8e0f7fe640 1 --2- 172.21.15.86:0/279438717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556532172e80 0x556532175b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.272+0000 7f8e0f7fe640 1 -- 172.21.15.86:0/279438717 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5565321f13e0 con 0x55653217ba00 2024-04-21T08:50:14.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.272+0000 7f8e0effd640 1 --2- 172.21.15.86:0/279438717 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5565321724d0 0x5565321755e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:14.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.272+0000 7f8e0f7fe640 1 --2- 172.21.15.86:0/279438717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55653217ba00 0x556532176060 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f8e080129d0 tx=0x7f8e08005f30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4e337af572ce8247 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:14.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.272+0000 7f8df2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:14.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.272+0000 7f8df2ff5640 1 -- 172.21.15.86:0/279438717 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8e08020040 con 0x55653217ba00 2024-04-21T08:50:14.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.272+0000 7f8df2ff5640 1 -- 172.21.15.86:0/279438717 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f8e0800a8d0 con 0x55653217ba00 2024-04-21T08:50:14.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.272+0000 7f8e17fe1740 1 -- 172.21.15.86:0/279438717 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8e000326d0 con 0x55653217ba00 2024-04-21T08:50:14.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.272+0000 7f8df2ff5640 1 -- 172.21.15.86:0/279438717 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8e0803a050 con 0x55653217ba00 2024-04-21T08:50:14.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.272+0000 7f8e17fe1740 10 client.20214 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:14.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.272+0000 7f8e17fe1740 1 -- 172.21.15.86:0/279438717 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5565321f2610 con 0x55653217ba00 2024-04-21T08:50:14.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.272+0000 7f8df2ff5640 1 -- 172.21.15.86:0/279438717 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8e000326d0 con 0x55653217ba00 2024-04-21T08:50:14.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.272+0000 7f8df2ff5640 1 -- 172.21.15.86:0/279438717 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8e0801b030 con 0x55653217ba00 2024-04-21T08:50:14.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.272+0000 7f8e17fe1740 10 client.20214 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:14.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.272+0000 7f8e17fe1740 10 client.20214 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:14.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.272+0000 7f8e17fe1740 20 client.20214 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:14.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.273+0000 7f8e17fe1740 1 --2- 172.21.15.86:0/279438717 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5565321f2b60 0x5565321f4f80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:14.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.273+0000 7f8e17fe1740 4 client.20214 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:50:14.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.273+0000 7f8e17fe1740 1 -- 172.21.15.86:0/279438717 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x5565320e4c10 con 0x5565321f2b60 2024-04-21T08:50:14.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.273+0000 7f8e0effd640 1 --2- 172.21.15.86:0/279438717 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5565321f2b60 0x5565321f4f80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:14.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.273+0000 7f8df2ff5640 10 client.20214 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:14.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.273+0000 7f8e0effd640 1 --2- 172.21.15.86:0/279438717 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5565321f2b60 0x5565321f4f80 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=2244dea725d877b9 server_cookie=8718ae97ae7c5469 in_seq=0 out_seq=0 2024-04-21T08:50:14.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.273+0000 7f8df2ff5640 1 -- 172.21.15.86:0/279438717 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x5565320e4c10 con 0x5565321f2b60 2024-04-21T08:50:14.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.273+0000 7f8df2ff5640 10 client.20214 handle_command_reply: tid=0 2024-04-21T08:50:14.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.273+0000 7f8e17fe1740 1 client.20214 shutdown 2024-04-21T08:50:14.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.274+0000 7f8e17fe1740 1 -- 172.21.15.86:0/279438717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55653217ba00 msgr2=0x556532176060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:14.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.274+0000 7f8e17fe1740 1 --2- 172.21.15.86:0/279438717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55653217ba00 0x556532176060 secure :-1 s=READY pgs=1995 cs=0 l=1 rev1=1 crypto rx=0x7f8e080129d0 tx=0x7f8e08005f30 comp rx=0 tx=0).stop 2024-04-21T08:50:14.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.274+0000 7f8e17fe1740 1 -- 172.21.15.86:0/279438717 shutdown_connections 2024-04-21T08:50:14.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.274+0000 7f8e17fe1740 1 --2- 172.21.15.86:0/279438717 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5565321f2b60 0x5565321f4f80 crc :-1 s=READY pgs=328 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.274+0000 7f8e17fe1740 1 --2- 172.21.15.86:0/279438717 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55653217ba00 0x556532176060 unknown :-1 s=CLOSED pgs=1995 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.274+0000 7f8e17fe1740 1 --2- 172.21.15.86:0/279438717 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556532172e80 0x556532175b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.274+0000 7f8e17fe1740 1 --2- 172.21.15.86:0/279438717 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5565321724d0 0x5565321755e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.274+0000 7f8df2ff5640 0 client.20214 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:14.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.274+0000 7f8e17fe1740 1 -- 172.21.15.86:0/279438717 >> 172.21.15.86:0/279438717 conn(0x55653216cba0 msgr2=0x55653216e850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:14.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.274+0000 7f8e17fe1740 1 -- 172.21.15.86:0/279438717 shutdown_connections 2024-04-21T08:50:14.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.274+0000 7f8e17fe1740 1 -- 172.21.15.86:0/279438717 wait complete. 2024-04-21T08:50:14.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.274+0000 7f8e17fe1740 20 client.20214 trim_cache size 0 max 16384 2024-04-21T08:50:14.280 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:50:14.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.274+0000 7f8e15315640 1 -- 172.21.15.86:0/482530074 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8df80a3d20 msgr2=0x7f8df80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:14.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.274+0000 7f8e15315640 1 --2- 172.21.15.86:0/482530074 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8df80a3d20 0x7f8df80a61e0 secure :-1 s=READY pgs=332 cs=0 l=1 rev1=1 crypto rx=0x7f8e0800d720 tx=0x7f8e08014000 comp rx=0 tx=0).stop 2024-04-21T08:50:14.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.274+0000 7f8e15315640 1 -- 172.21.15.86:0/482530074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e10107270 msgr2=0x7f8e10198bf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:14.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.274+0000 7f8e15315640 1 --2- 172.21.15.86:0/482530074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e10107270 0x7f8e10198bf0 secure :-1 s=READY pgs=1991 cs=0 l=1 rev1=1 crypto rx=0x7f8e0400d770 tx=0x7f8e0400dc30 comp rx=0 tx=0).stop 2024-04-21T08:50:14.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.274+0000 7f8e15315640 1 -- 172.21.15.86:0/482530074 shutdown_connections 2024-04-21T08:50:14.281 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.274+0000 7f8e15315640 1 --2- 172.21.15.86:0/482530074 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8df80a3d20 0x7f8df80a61e0 unknown :-1 s=CLOSED pgs=332 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.274+0000 7f8e15315640 1 --2- 172.21.15.86:0/482530074 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8e1019cf70 0x7f8e1019d420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.274+0000 7f8e15315640 1 --2- 172.21.15.86:0/482530074 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8e10107270 0x7f8e10198bf0 unknown :-1 s=CLOSED pgs=1991 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.274+0000 7f8e15315640 1 --2- 172.21.15.86:0/482530074 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8e1006a860 0x7f8e101986b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:14.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.274+0000 7f8e15315640 1 -- 172.21.15.86:0/482530074 >> 172.21.15.86:0/482530074 conn(0x7f8e10077eb0 msgr2=0x7f8e10100550 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:14.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.274+0000 7f8e15315640 1 -- 172.21.15.86:0/482530074 shutdown_connections 2024-04-21T08:50:14.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:14.274+0000 7f8e15315640 1 -- 172.21.15.86:0/482530074 wait complete. 2024-04-21T08:50:15.285 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:50:15.285 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:50:15.285 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:50:15.401 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.400+0000 7fd35b715640 1 -- 172.21.15.86:0/234316677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd354101a40 msgr2=0x7fd354105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:15.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.400+0000 7fd35b715640 1 --2- 172.21.15.86:0/234316677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd354101a40 0x7fd354105a80 secure :-1 s=READY pgs=1996 cs=0 l=1 rev1=1 crypto rx=0x7fd348009900 tx=0x7fd34802bc30 comp rx=0 tx=0).stop 2024-04-21T08:50:15.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.400+0000 7fd35b715640 1 -- 172.21.15.86:0/234316677 shutdown_connections 2024-04-21T08:50:15.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.400+0000 7fd35b715640 1 --2- 172.21.15.86:0/234316677 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd354106160 0x7fd35410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.400+0000 7fd35b715640 1 --2- 172.21.15.86:0/234316677 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd354101a40 0x7fd354105a80 unknown :-1 s=CLOSED pgs=1996 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.400+0000 7fd35b715640 1 --2- 172.21.15.86:0/234316677 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd354101090 0x7fd354101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.400+0000 7fd35b715640 1 -- 172.21.15.86:0/234316677 >> 172.21.15.86:0/234316677 conn(0x7fd354077f50 msgr2=0x7fd354078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:15.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.400+0000 7fd35b715640 1 -- 172.21.15.86:0/234316677 shutdown_connections 2024-04-21T08:50:15.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.400+0000 7fd35b715640 1 -- 172.21.15.86:0/234316677 wait complete. 2024-04-21T08:50:15.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.400+0000 7fd35b715640 1 Processor -- start 2024-04-21T08:50:15.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.400+0000 7fd35b715640 1 -- start start 2024-04-21T08:50:15.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.401+0000 7fd35b715640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd354101090 0x7fd354102b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:15.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.401+0000 7fd35b715640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd354106160 0x7fd354103060 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:15.402 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.401+0000 7fd35b715640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3541035a0 0x7fd3541a6920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:15.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.401+0000 7fd35b715640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd3541110a0 con 0x7fd354106160 2024-04-21T08:50:15.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.401+0000 7fd35b715640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd354110f20 con 0x7fd354101090 2024-04-21T08:50:15.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.401+0000 7fd35b715640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd354111220 con 0x7fd3541035a0 2024-04-21T08:50:15.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.401+0000 7fd358cb0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd354106160 0x7fd354103060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:15.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.401+0000 7fd358cb0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd354106160 0x7fd354103060 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55488/0 (socket says 172.21.15.86:55488) 2024-04-21T08:50:15.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.401+0000 7fd358cb0640 1 -- 172.21.15.86:0/1474510805 learned_addr learned my addr 172.21.15.86:0/1474510805 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:15.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.401+0000 7fd359cb2640 1 --2- 172.21.15.86:0/1474510805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3541035a0 0x7fd3541a6920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:15.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.401+0000 7fd3594b1640 1 --2- 172.21.15.86:0/1474510805 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd354101090 0x7fd354102b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:15.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.401+0000 7fd358cb0640 1 -- 172.21.15.86:0/1474510805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3541035a0 msgr2=0x7fd3541a6920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:15.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.401+0000 7fd358cb0640 1 --2- 172.21.15.86:0/1474510805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3541035a0 0x7fd3541a6920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.401+0000 7fd358cb0640 1 -- 172.21.15.86:0/1474510805 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd354101090 msgr2=0x7fd354102b20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:15.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.401+0000 7fd358cb0640 1 --2- 172.21.15.86:0/1474510805 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd354101090 0x7fd354102b20 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.403 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.401+0000 7fd358cb0640 1 -- 172.21.15.86:0/1474510805 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd3541a7020 con 0x7fd354106160 2024-04-21T08:50:15.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.401+0000 7fd359cb2640 1 --2- 172.21.15.86:0/1474510805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3541035a0 0x7fd3541a6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:15.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.401+0000 7fd358cb0640 1 --2- 172.21.15.86:0/1474510805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd354106160 0x7fd354103060 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7fd348036040 tx=0x7fd34802c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:15.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.401+0000 7fd3527fc640 1 -- 172.21.15.86:0/1474510805 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd348043020 con 0x7fd354106160 2024-04-21T08:50:15.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.401+0000 7fd35b715640 1 -- 172.21.15.86:0/1474510805 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd3541a72b0 con 0x7fd354106160 2024-04-21T08:50:15.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.401+0000 7fd3527fc640 1 -- 172.21.15.86:0/1474510805 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd34802cbe0 con 0x7fd354106160 2024-04-21T08:50:15.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.401+0000 7fd3527fc640 1 -- 172.21.15.86:0/1474510805 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd348003bf0 con 0x7fd354106160 2024-04-21T08:50:15.404 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.401+0000 7fd35b715640 1 -- 172.21.15.86:0/1474510805 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd3541a7850 con 0x7fd354106160 2024-04-21T08:50:15.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.403+0000 7fd3527fc640 1 -- 172.21.15.86:0/1474510805 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd348046020 con 0x7fd354106160 2024-04-21T08:50:15.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.404+0000 7fd35e3dd740 1 Processor -- start 2024-04-21T08:50:15.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.404+0000 7fd3527fc640 1 --2- 172.21.15.86:0/1474510805 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd33c0a3d50 0x7fd33c0a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:15.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.404+0000 7fd35e3dd740 1 -- start start 2024-04-21T08:50:15.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.404+0000 7fd359cb2640 1 --2- 172.21.15.86:0/1474510805 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd33c0a3d50 0x7fd33c0a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:15.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.404+0000 7fd3527fc640 1 -- 172.21.15.86:0/1474510805 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7fd3480f2de0 con 0x7fd354106160 2024-04-21T08:50:15.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.404+0000 7fd35e3dd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab7f958e0 0x558ab7f95cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:15.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.404+0000 7fd35e3dd740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab7f96290 0x558ab7f9e840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:15.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.405+0000 7fd35e3dd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab7f9ee10 0x558ab7fa1260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:15.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.405+0000 7fd35e3dd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558ab7ef9f00 con 0x558ab7f958e0 2024-04-21T08:50:15.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.405+0000 7fd35e3dd740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x558ab7e97590 con 0x558ab7f96290 2024-04-21T08:50:15.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.405+0000 7fd35e3dd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558ab7ef5ec0 con 0x558ab7f9ee10 2024-04-21T08:50:15.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.405+0000 7fd3594b1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab7f958e0 0x558ab7f95cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:15.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.405+0000 7fd3594b1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab7f958e0 0x558ab7f95cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55490/0 (socket says 172.21.15.86:55490) 2024-04-21T08:50:15.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.405+0000 7fd3594b1640 1 -- 172.21.15.86:0/4189089351 learned_addr learned my addr 172.21.15.86:0/4189089351 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:15.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.405+0000 7fd359cb2640 1 --2- 172.21.15.86:0/1474510805 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd33c0a3d50 0x7fd33c0a6210 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd34c005940 tx=0x7fd34c0058b0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:15.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.405+0000 7fd358cb0640 1 --2- 172.21.15.86:0/4189089351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab7f96290 0x558ab7f9e840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:15.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.405+0000 7fd359cb2640 1 --2- 172.21.15.86:0/4189089351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab7f9ee10 0x558ab7fa1260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:15.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.405+0000 7fd3594b1640 1 -- 172.21.15.86:0/4189089351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab7f9ee10 msgr2=0x558ab7fa1260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:15.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.405+0000 7fd3594b1640 1 --2- 172.21.15.86:0/4189089351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab7f9ee10 0x558ab7fa1260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.405+0000 7fd3594b1640 1 -- 172.21.15.86:0/4189089351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab7f96290 msgr2=0x558ab7f9e840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:15.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.405+0000 7fd3594b1640 1 --2- 172.21.15.86:0/4189089351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab7f96290 0x558ab7f9e840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.405+0000 7fd3594b1640 1 -- 172.21.15.86:0/4189089351 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ab7cd47e0 con 0x558ab7f958e0 2024-04-21T08:50:15.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.405+0000 7fd3594b1640 1 --2- 172.21.15.86:0/4189089351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab7f958e0 0x558ab7f95cc0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7fd344009fc0 tx=0x7fd34400f7f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b514432a363cf0b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:15.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.405+0000 7fd33b7fe640 1 -- 172.21.15.86:0/4189089351 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd344004d20 con 0x558ab7f958e0 2024-04-21T08:50:15.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.405+0000 7fd33b7fe640 1 -- 172.21.15.86:0/4189089351 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3440119b0 con 0x558ab7f958e0 2024-04-21T08:50:15.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.405+0000 7fd33b7fe640 1 -- 172.21.15.86:0/4189089351 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd344005440 con 0x558ab7f958e0 2024-04-21T08:50:15.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.406+0000 7fd35e3dd740 1 -- 172.21.15.86:0/4189089351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab7f958e0 msgr2=0x558ab7f95cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:15.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.406+0000 7fd35e3dd740 1 --2- 172.21.15.86:0/4189089351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab7f958e0 0x558ab7f95cc0 secure :-1 s=READY pgs=1998 cs=0 l=1 rev1=1 crypto rx=0x7fd344009fc0 tx=0x7fd34400f7f0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.406+0000 7fd35e3dd740 1 -- 172.21.15.86:0/4189089351 shutdown_connections 2024-04-21T08:50:15.412 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.406+0000 7fd35e3dd740 1 --2- 172.21.15.86:0/4189089351 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab7f9ee10 0x558ab7fa1260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.406+0000 7fd35e3dd740 1 --2- 172.21.15.86:0/4189089351 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab7f96290 0x558ab7f9e840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.406+0000 7fd35e3dd740 1 --2- 172.21.15.86:0/4189089351 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab7f958e0 0x558ab7f95cc0 unknown :-1 s=CLOSED pgs=1998 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.406+0000 7fd35e3dd740 1 -- 172.21.15.86:0/4189089351 >> 172.21.15.86:0/4189089351 conn(0x558ab7f8ffb0 msgr2=0x558ab7f90390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:15.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.406+0000 7fd35e3dd740 1 -- 172.21.15.86:0/4189089351 shutdown_connections 2024-04-21T08:50:15.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.406+0000 7fd35e3dd740 1 -- 172.21.15.86:0/4189089351 wait complete. 2024-04-21T08:50:15.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.406+0000 7fd35e3dd740 1 -- start start 2024-04-21T08:50:15.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.406+0000 7fd35e3dd740 1 Processor -- start 2024-04-21T08:50:15.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.407+0000 7fd35e3dd740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab7f958e0 0x558ab7f9bdb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:15.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.407+0000 7fd35e3dd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab7f96290 0x558ab7f9c2f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:15.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.407+0000 7fd35e3dd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab7f9ee10 0x558ab7f9c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:15.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.407+0000 7fd35e3dd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x558ab7fa1d80 con 0x558ab7f96290 2024-04-21T08:50:15.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.407+0000 7fd35e3dd740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x558ab7eb3580 con 0x558ab7f958e0 2024-04-21T08:50:15.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.407+0000 7fd35e3dd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x558ab7fa1f00 con 0x558ab7f9ee10 2024-04-21T08:50:15.413 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.407+0000 7fd35e3dd740 10 client.0 _ll_register_callbacks cb 0x558ab7eb9650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:15.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.407+0000 7fd358cb0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab7f96290 0x558ab7f9c2f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:15.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.407+0000 7fd358cb0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab7f96290 0x558ab7f9c2f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55494/0 (socket says 172.21.15.86:55494) 2024-04-21T08:50:15.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.407+0000 7fd358cb0640 1 -- 172.21.15.86:0/3648303644 learned_addr learned my addr 172.21.15.86:0/3648303644 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:15.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.407+0000 7fd359cb2640 1 --2- 172.21.15.86:0/3648303644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab7f9ee10 0x558ab7f9c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:15.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.407+0000 7fd3594b1640 1 --2- 172.21.15.86:0/3648303644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab7f958e0 0x558ab7f9bdb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:15.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.407+0000 7fd358cb0640 1 -- 172.21.15.86:0/3648303644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab7f9ee10 msgr2=0x558ab7f9c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:15.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.407+0000 7fd358cb0640 1 --2- 172.21.15.86:0/3648303644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab7f9ee10 0x558ab7f9c830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.407+0000 7fd358cb0640 1 -- 172.21.15.86:0/3648303644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab7f958e0 msgr2=0x558ab7f9bdb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:15.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.407+0000 7fd358cb0640 1 --2- 172.21.15.86:0/3648303644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab7f958e0 0x558ab7f9bdb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.407+0000 7fd358cb0640 1 -- 172.21.15.86:0/3648303644 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ab7d34f70 con 0x558ab7f96290 2024-04-21T08:50:15.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.407+0000 7fd359cb2640 1 --2- 172.21.15.86:0/3648303644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab7f9ee10 0x558ab7f9c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:15.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.407+0000 7fd358cb0640 1 --2- 172.21.15.86:0/3648303644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab7f96290 0x558ab7f9c2f0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fd3480f0760 tx=0x7fd3480be420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b7be2e9e6ed75efb server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:15.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.407+0000 7fd338ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:15.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.408+0000 7fd338ff9640 1 -- 172.21.15.86:0/3648303644 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd3480beac0 con 0x558ab7f96290 2024-04-21T08:50:15.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.408+0000 7fd338ff9640 1 -- 172.21.15.86:0/3648303644 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7fd3480bcdc0 con 0x558ab7f96290 2024-04-21T08:50:15.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.408+0000 7fd35e3dd740 1 -- 172.21.15.86:0/3648303644 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd3440119b0 con 0x558ab7f96290 2024-04-21T08:50:15.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.408+0000 7fd338ff9640 1 -- 172.21.15.86:0/3648303644 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd3480d8020 con 0x558ab7f96290 2024-04-21T08:50:15.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.408+0000 7fd35e3dd740 10 client.20226 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:15.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.408+0000 7fd35e3dd740 1 -- 172.21.15.86:0/3648303644 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558ab7bcc170 con 0x558ab7f96290 2024-04-21T08:50:15.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.408+0000 7fd338ff9640 1 -- 172.21.15.86:0/3648303644 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd3440119b0 con 0x558ab7f96290 2024-04-21T08:50:15.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.408+0000 7fd338ff9640 1 -- 172.21.15.86:0/3648303644 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd3480c72a0 con 0x558ab7f96290 2024-04-21T08:50:15.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.408+0000 7fd35e3dd740 10 client.20226 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:15.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.408+0000 7fd35e3dd740 10 client.20226 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:15.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.408+0000 7fd35e3dd740 20 client.20226 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:15.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.408+0000 7fd35e3dd740 1 --2- 172.21.15.86:0/3648303644 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x558ab7faf300 0x558ab7fb1760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:15.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.408+0000 7fd35e3dd740 4 client.20226 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:15.415 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.408+0000 7fd35e3dd740 1 -- 172.21.15.86:0/3648303644 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x558ab7f07920 con 0x558ab7faf300 2024-04-21T08:50:15.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.408+0000 7fd3594b1640 1 --2- 172.21.15.86:0/3648303644 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x558ab7faf300 0x558ab7fb1760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:15.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.409+0000 7fd338ff9640 10 client.20226 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:15.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.409+0000 7fd3594b1640 1 --2- 172.21.15.86:0/3648303644 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x558ab7faf300 0x558ab7fb1760 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6aff42a78ce9c4c server_cookie=9efe0d957665dc58 in_seq=0 out_seq=0 2024-04-21T08:50:15.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.409+0000 7fd338ff9640 1 -- 172.21.15.86:0/3648303644 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x558ab7f07920 con 0x558ab7faf300 2024-04-21T08:50:15.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.409+0000 7fd338ff9640 10 client.20226 handle_command_reply: tid=0 2024-04-21T08:50:15.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.410+0000 7fd35e3dd740 1 client.20226 shutdown 2024-04-21T08:50:15.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.410+0000 7fd35e3dd740 1 -- 172.21.15.86:0/3648303644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab7f96290 msgr2=0x558ab7f9c2f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:15.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.410+0000 7fd35e3dd740 1 --2- 172.21.15.86:0/3648303644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab7f96290 0x558ab7f9c2f0 secure :-1 s=READY pgs=1999 cs=0 l=1 rev1=1 crypto rx=0x7fd3480f0760 tx=0x7fd3480be420 comp rx=0 tx=0).stop 2024-04-21T08:50:15.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.410+0000 7fd35e3dd740 1 -- 172.21.15.86:0/3648303644 shutdown_connections 2024-04-21T08:50:15.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.410+0000 7fd35e3dd740 1 --2- 172.21.15.86:0/3648303644 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x558ab7faf300 0x558ab7fb1760 crc :-1 s=READY pgs=329 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.410+0000 7fd35e3dd740 1 --2- 172.21.15.86:0/3648303644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab7f9ee10 0x558ab7f9c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.410+0000 7fd35e3dd740 1 --2- 172.21.15.86:0/3648303644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab7f96290 0x558ab7f9c2f0 unknown :-1 s=CLOSED pgs=1999 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.410+0000 7fd338ff9640 0 client.20226 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:15.416 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.410+0000 7fd35e3dd740 1 --2- 172.21.15.86:0/3648303644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab7f958e0 0x558ab7f9bdb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.410+0000 7fd35e3dd740 1 -- 172.21.15.86:0/3648303644 >> 172.21.15.86:0/3648303644 conn(0x558ab7f8ffb0 msgr2=0x558ab7f91d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:15.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.410+0000 7fd35e3dd740 1 -- 172.21.15.86:0/3648303644 shutdown_connections 2024-04-21T08:50:15.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.410+0000 7fd35e3dd740 1 -- 172.21.15.86:0/3648303644 wait complete. 2024-04-21T08:50:15.417 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.410+0000 7fd35e3dd740 20 client.20226 trim_cache size 0 max 16384 2024-04-21T08:50:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.427+0000 7fd35e3dd740 1 Processor -- start 2024-04-21T08:50:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.427+0000 7fd35e3dd740 1 -- start start 2024-04-21T08:50:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.428+0000 7fd35e3dd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab7f8ffb0 0x558ab7f9e4e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.428+0000 7fd35e3dd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab7f958e0 0x558ab7faac40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.428+0000 7fd35e3dd740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab7f96290 0x558ab7fb7070 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.428+0000 7fd35e3dd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd3541b15c0 con 0x558ab7f8ffb0 2024-04-21T08:50:15.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.428+0000 7fd35e3dd740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd3541b3100 con 0x558ab7f96290 2024-04-21T08:50:15.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.428+0000 7fd35e3dd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd3541b1c90 con 0x558ab7f958e0 2024-04-21T08:50:15.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.428+0000 7fd3594b1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab7f8ffb0 0x558ab7f9e4e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:15.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.428+0000 7fd3594b1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab7f8ffb0 0x558ab7f9e4e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55504/0 (socket says 172.21.15.86:55504) 2024-04-21T08:50:15.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.428+0000 7fd3594b1640 1 -- 172.21.15.86:0/2411877199 learned_addr learned my addr 172.21.15.86:0/2411877199 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:15.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.428+0000 7fd358cb0640 1 --2- 172.21.15.86:0/2411877199 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab7f958e0 0x558ab7faac40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:15.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.428+0000 7fd3594b1640 1 -- 172.21.15.86:0/2411877199 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab7f958e0 msgr2=0x558ab7faac40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:15.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.428+0000 7fd3594b1640 1 --2- 172.21.15.86:0/2411877199 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab7f958e0 0x558ab7faac40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.428+0000 7fd359cb2640 1 --2- 172.21.15.86:0/2411877199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab7f96290 0x558ab7fb7070 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:15.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.428+0000 7fd3594b1640 1 -- 172.21.15.86:0/2411877199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab7f96290 msgr2=0x558ab7fb7070 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:15.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.428+0000 7fd3594b1640 1 --2- 172.21.15.86:0/2411877199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab7f96290 0x558ab7fb7070 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.428+0000 7fd3594b1640 1 -- 172.21.15.86:0/2411877199 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x558ab7f9cd10 con 0x558ab7f8ffb0 2024-04-21T08:50:15.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.428+0000 7fd358cb0640 1 --2- 172.21.15.86:0/2411877199 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab7f958e0 0x558ab7faac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:15.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.428+0000 7fd359cb2640 1 --2- 172.21.15.86:0/2411877199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab7f96290 0x558ab7fb7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:15.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.428+0000 7fd3594b1640 1 --2- 172.21.15.86:0/2411877199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab7f8ffb0 0x558ab7f9e4e0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7fd344009370 tx=0x7fd344005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3f35dc1cd56c9fa server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:15.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.428+0000 7fd33affd640 1 -- 172.21.15.86:0/2411877199 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd344004be0 con 0x558ab7f8ffb0 2024-04-21T08:50:15.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.428+0000 7fd33affd640 1 -- 172.21.15.86:0/2411877199 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd34402e070 con 0x558ab7f8ffb0 2024-04-21T08:50:15.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.428+0000 7fd33affd640 1 -- 172.21.15.86:0/2411877199 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd344029740 con 0x558ab7f8ffb0 2024-04-21T08:50:15.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.429+0000 7fd35e3dd740 1 -- 172.21.15.86:0/2411877199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab7f8ffb0 msgr2=0x558ab7f9e4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:15.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.429+0000 7fd35e3dd740 1 --2- 172.21.15.86:0/2411877199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab7f8ffb0 0x558ab7f9e4e0 secure :-1 s=READY pgs=2000 cs=0 l=1 rev1=1 crypto rx=0x7fd344009370 tx=0x7fd344005fd0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.429+0000 7fd35e3dd740 1 -- 172.21.15.86:0/2411877199 shutdown_connections 2024-04-21T08:50:15.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.429+0000 7fd35e3dd740 1 --2- 172.21.15.86:0/2411877199 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab7f96290 0x558ab7fb7070 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.429+0000 7fd35e3dd740 1 --2- 172.21.15.86:0/2411877199 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab7f958e0 0x558ab7faac40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.429+0000 7fd35e3dd740 1 --2- 172.21.15.86:0/2411877199 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab7f8ffb0 0x558ab7f9e4e0 unknown :-1 s=CLOSED pgs=2000 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.429+0000 7fd35e3dd740 1 -- 172.21.15.86:0/2411877199 >> 172.21.15.86:0/2411877199 conn(0x558ab7f969e0 msgr2=0x558ab7f9c600 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:15.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.429+0000 7fd35e3dd740 1 -- 172.21.15.86:0/2411877199 shutdown_connections 2024-04-21T08:50:15.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.429+0000 7fd35e3dd740 1 -- 172.21.15.86:0/2411877199 wait complete. 2024-04-21T08:50:15.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.429+0000 7fd35e3dd740 1 -- start start 2024-04-21T08:50:15.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.429+0000 7fd35e3dd740 1 Processor -- start 2024-04-21T08:50:15.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.430+0000 7fd35e3dd740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab7f8ffb0 0x558ab7fa7640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:15.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.430+0000 7fd35e3dd740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab7f958e0 0x558ab7fa39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:15.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.430+0000 7fd35e3dd740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab7f96290 0x558ab7fa3fd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:15.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.430+0000 7fd35e3dd740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd3541b2a30 con 0x558ab7f8ffb0 2024-04-21T08:50:15.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.430+0000 7fd35e3dd740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd3541b2360 con 0x558ab7f958e0 2024-04-21T08:50:15.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.430+0000 7fd35e3dd740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd3541af5c0 con 0x558ab7f96290 2024-04-21T08:50:15.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.430+0000 7fd35e3dd740 10 client.0 _ll_register_callbacks cb 0x7fd3480be920 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:15.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.430+0000 7fd3594b1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab7f8ffb0 0x558ab7fa7640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:15.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.430+0000 7fd3594b1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab7f8ffb0 0x558ab7fa7640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55516/0 (socket says 172.21.15.86:55516) 2024-04-21T08:50:15.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.430+0000 7fd3594b1640 1 -- 172.21.15.86:0/3674708398 learned_addr learned my addr 172.21.15.86:0/3674708398 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:15.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.430+0000 7fd359cb2640 1 --2- 172.21.15.86:0/3674708398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab7f96290 0x558ab7fa3fd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:15.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.430+0000 7fd3594b1640 1 -- 172.21.15.86:0/3674708398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab7f96290 msgr2=0x558ab7fa3fd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:15.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.430+0000 7fd3594b1640 1 --2- 172.21.15.86:0/3674708398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab7f96290 0x558ab7fa3fd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.430+0000 7fd358cb0640 1 --2- 172.21.15.86:0/3674708398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab7f958e0 0x558ab7fa39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:15.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.430+0000 7fd3594b1640 1 -- 172.21.15.86:0/3674708398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab7f958e0 msgr2=0x558ab7fa39c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:15.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.430+0000 7fd3594b1640 1 --2- 172.21.15.86:0/3674708398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab7f958e0 0x558ab7fa39c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.430+0000 7fd3594b1640 1 -- 172.21.15.86:0/3674708398 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x558ab7fa4800 con 0x558ab7f8ffb0 2024-04-21T08:50:15.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.430+0000 7fd3594b1640 1 --2- 172.21.15.86:0/3674708398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab7f8ffb0 0x558ab7fa7640 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fd344005a90 tx=0x7fd344005ac0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b90f4fa45430fe11 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:15.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.430+0000 7fd338ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:15.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.430+0000 7fd338ff9640 1 -- 172.21.15.86:0/3674708398 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd344038020 con 0x558ab7f8ffb0 2024-04-21T08:50:15.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.431+0000 7fd338ff9640 1 -- 172.21.15.86:0/3674708398 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7fd344011700 con 0x558ab7f8ffb0 2024-04-21T08:50:15.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.431+0000 7fd35e3dd740 1 -- 172.21.15.86:0/3674708398 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd34402e070 con 0x558ab7f8ffb0 2024-04-21T08:50:15.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.431+0000 7fd338ff9640 1 -- 172.21.15.86:0/3674708398 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd344043050 con 0x558ab7f8ffb0 2024-04-21T08:50:15.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.431+0000 7fd35e3dd740 10 client.20232 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:15.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.431+0000 7fd35e3dd740 1 -- 172.21.15.86:0/3674708398 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x558ab80169d0 con 0x558ab7f8ffb0 2024-04-21T08:50:15.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.431+0000 7fd338ff9640 1 -- 172.21.15.86:0/3674708398 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd34402e070 con 0x558ab7f8ffb0 2024-04-21T08:50:15.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.431+0000 7fd338ff9640 1 -- 172.21.15.86:0/3674708398 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd344026070 con 0x558ab7f8ffb0 2024-04-21T08:50:15.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.431+0000 7fd35e3dd740 10 client.20232 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:15.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.431+0000 7fd35e3dd740 10 client.20232 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:15.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.431+0000 7fd35e3dd740 20 client.20232 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:15.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.431+0000 7fd35e3dd740 1 --2- 172.21.15.86:0/3674708398 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x558ab7f9ee10 0x558ab8018f50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:15.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.431+0000 7fd35e3dd740 4 client.20232 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:50:15.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.431+0000 7fd35e3dd740 1 -- 172.21.15.86:0/3674708398 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x558ab7f01790 con 0x558ab7f9ee10 2024-04-21T08:50:15.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.431+0000 7fd358cb0640 1 --2- 172.21.15.86:0/3674708398 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x558ab7f9ee10 0x558ab8018f50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:15.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.432+0000 7fd358cb0640 1 --2- 172.21.15.86:0/3674708398 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x558ab7f9ee10 0x558ab8018f50 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=188a82be78f1a164 server_cookie=6bb8ff1ace0603e6 in_seq=0 out_seq=0 2024-04-21T08:50:15.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.432+0000 7fd338ff9640 10 client.20232 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:15.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.433+0000 7fd338ff9640 1 -- 172.21.15.86:0/3674708398 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x558ab7f01790 con 0x558ab7f9ee10 2024-04-21T08:50:15.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.433+0000 7fd338ff9640 10 client.20232 handle_command_reply: tid=0 2024-04-21T08:50:15.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.433+0000 7fd35e3dd740 1 client.20232 shutdown 2024-04-21T08:50:15.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.433+0000 7fd35e3dd740 1 -- 172.21.15.86:0/3674708398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab7f8ffb0 msgr2=0x558ab7fa7640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:15.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.433+0000 7fd35e3dd740 1 --2- 172.21.15.86:0/3674708398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab7f8ffb0 0x558ab7fa7640 secure :-1 s=READY pgs=2001 cs=0 l=1 rev1=1 crypto rx=0x7fd344005a90 tx=0x7fd344005ac0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.433+0000 7fd35e3dd740 1 -- 172.21.15.86:0/3674708398 shutdown_connections 2024-04-21T08:50:15.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.433+0000 7fd35e3dd740 1 --2- 172.21.15.86:0/3674708398 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x558ab7f9ee10 0x558ab8018f50 crc :-1 s=READY pgs=330 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.433+0000 7fd35e3dd740 1 --2- 172.21.15.86:0/3674708398 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x558ab7f96290 0x558ab7fa3fd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.437 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.433+0000 7fd35e3dd740 1 --2- 172.21.15.86:0/3674708398 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x558ab7f958e0 0x558ab7fa39c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.433+0000 7fd338ff9640 0 client.20232 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:15.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.433+0000 7fd35e3dd740 1 --2- 172.21.15.86:0/3674708398 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x558ab7f8ffb0 0x558ab7fa7640 unknown :-1 s=CLOSED pgs=2001 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.433+0000 7fd35e3dd740 1 -- 172.21.15.86:0/3674708398 >> 172.21.15.86:0/3674708398 conn(0x558ab7f969e0 msgr2=0x558ab7f91210 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:15.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.433+0000 7fd35e3dd740 1 -- 172.21.15.86:0/3674708398 shutdown_connections 2024-04-21T08:50:15.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.433+0000 7fd35e3dd740 1 -- 172.21.15.86:0/3674708398 wait complete. 2024-04-21T08:50:15.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.433+0000 7fd35e3dd740 20 client.20232 trim_cache size 0 max 16384 2024-04-21T08:50:15.438 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:50:15.438 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-21T08:50:15.438 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "99a77aa5-7273-4b22-bd6d-157cf40464fd", 2024-04-21T08:50:15.438 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-21T08:50:15.439 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:50:15.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.434+0000 7fd35b715640 1 -- 172.21.15.86:0/1474510805 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd33c0a3d50 msgr2=0x7fd33c0a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:15.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.434+0000 7fd35b715640 1 --2- 172.21.15.86:0/1474510805 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd33c0a3d50 0x7fd33c0a6210 secure :-1 s=READY pgs=333 cs=0 l=1 rev1=1 crypto rx=0x7fd34c005940 tx=0x7fd34c0058b0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.434+0000 7fd35b715640 1 -- 172.21.15.86:0/1474510805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd354106160 msgr2=0x7fd354103060 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:15.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.434+0000 7fd35b715640 1 --2- 172.21.15.86:0/1474510805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd354106160 0x7fd354103060 secure :-1 s=READY pgs=1997 cs=0 l=1 rev1=1 crypto rx=0x7fd348036040 tx=0x7fd34802c7e0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.434+0000 7fd35b715640 1 -- 172.21.15.86:0/1474510805 shutdown_connections 2024-04-21T08:50:15.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.434+0000 7fd35b715640 1 --2- 172.21.15.86:0/1474510805 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd33c0a3d50 0x7fd33c0a6210 unknown :-1 s=CLOSED pgs=333 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.434+0000 7fd35b715640 1 --2- 172.21.15.86:0/1474510805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd3541035a0 0x7fd3541a6920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.434+0000 7fd35b715640 1 --2- 172.21.15.86:0/1474510805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd354106160 0x7fd354103060 unknown :-1 s=CLOSED pgs=1997 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.434+0000 7fd35b715640 1 --2- 172.21.15.86:0/1474510805 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd354101090 0x7fd354102b20 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.434+0000 7fd35b715640 1 -- 172.21.15.86:0/1474510805 >> 172.21.15.86:0/1474510805 conn(0x7fd354077f50 msgr2=0x7fd3540694f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:15.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.434+0000 7fd35b715640 1 -- 172.21.15.86:0/1474510805 shutdown_connections 2024-04-21T08:50:15.440 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.434+0000 7fd35b715640 1 -- 172.21.15.86:0/1474510805 wait complete. 2024-04-21T08:50:15.444 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:50:15.444 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:50:15.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.546+0000 7f2481a5c640 1 -- 172.21.15.86:0/4122268685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f247c107280 msgr2=0x7f247c10ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:15.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.546+0000 7f2481a5c640 1 --2- 172.21.15.86:0/4122268685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f247c107280 0x7f247c10ee50 secure :-1 s=READY pgs=2002 cs=0 l=1 rev1=1 crypto rx=0x7f2474009900 tx=0x7f247402bc30 comp rx=0 tx=0).stop 2024-04-21T08:50:15.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.546+0000 7f2481a5c640 1 -- 172.21.15.86:0/4122268685 shutdown_connections 2024-04-21T08:50:15.547 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.546+0000 7f2481a5c640 1 --2- 172.21.15.86:0/4122268685 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f247c107280 0x7f247c10ee50 unknown :-1 s=CLOSED pgs=2002 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.546+0000 7f2481a5c640 1 --2- 172.21.15.86:0/4122268685 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f247c102b60 0x7f247c106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.546+0000 7f2481a5c640 1 --2- 172.21.15.86:0/4122268685 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f247c1021b0 0x7f247c102590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.546+0000 7f2481a5c640 1 -- 172.21.15.86:0/4122268685 >> 172.21.15.86:0/4122268685 conn(0x7f247c0fda10 msgr2=0x7f247c0ffe30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:15.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.546+0000 7f2481a5c640 1 -- 172.21.15.86:0/4122268685 shutdown_connections 2024-04-21T08:50:15.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.546+0000 7f2481a5c640 1 -- 172.21.15.86:0/4122268685 wait complete. 2024-04-21T08:50:15.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.546+0000 7f2481a5c640 1 Processor -- start 2024-04-21T08:50:15.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.546+0000 7f2481a5c640 1 -- start start 2024-04-21T08:50:15.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.547+0000 7f2481a5c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f247c1021b0 0x7f247c198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:15.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.547+0000 7f2481a5c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f247c107280 0x7f247c198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:15.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.547+0000 7f2481a5c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f247c19cfd0 0x7f247c19d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:15.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.547+0000 7f2481a5c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f247c1121d0 con 0x7f247c19cfd0 2024-04-21T08:50:15.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.547+0000 7f2481a5c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f247c112050 con 0x7f247c1021b0 2024-04-21T08:50:15.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.547+0000 7f2481a5c640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f247c112350 con 0x7f247c107280 2024-04-21T08:50:15.548 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.547+0000 7f247b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f247c19cfd0 0x7f247c19d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:15.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.547+0000 7f247b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f247c19cfd0 0x7f247c19d480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55528/0 (socket says 172.21.15.86:55528) 2024-04-21T08:50:15.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.547+0000 7f247b7fe640 1 -- 172.21.15.86:0/1271873805 learned_addr learned my addr 172.21.15.86:0/1271873805 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:15.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.547+0000 7f247affd640 1 --2- 172.21.15.86:0/1271873805 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f247c1021b0 0x7f247c198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:15.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.547+0000 7f247a7fc640 1 --2- 172.21.15.86:0/1271873805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f247c107280 0x7f247c198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:15.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.547+0000 7f247b7fe640 1 -- 172.21.15.86:0/1271873805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f247c107280 msgr2=0x7f247c198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:15.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.547+0000 7f247b7fe640 1 --2- 172.21.15.86:0/1271873805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f247c107280 0x7f247c198c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.547+0000 7f247b7fe640 1 -- 172.21.15.86:0/1271873805 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f247c1021b0 msgr2=0x7f247c198710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:15.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.547+0000 7f247b7fe640 1 --2- 172.21.15.86:0/1271873805 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f247c1021b0 0x7f247c198710 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.547+0000 7f247b7fe640 1 -- 172.21.15.86:0/1271873805 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f247c19dac0 con 0x7f247c19cfd0 2024-04-21T08:50:15.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.547+0000 7f247b7fe640 1 --2- 172.21.15.86:0/1271873805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f247c19cfd0 0x7f247c19d480 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f2474009230 tx=0x7f24740045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:15.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.547+0000 7f2480a5a640 1 -- 172.21.15.86:0/1271873805 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2474043020 con 0x7f247c19cfd0 2024-04-21T08:50:15.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.547+0000 7f2480a5a640 1 -- 172.21.15.86:0/1271873805 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f247402cb70 con 0x7f247c19cfd0 2024-04-21T08:50:15.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.547+0000 7f2481a5c640 1 -- 172.21.15.86:0/1271873805 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f247c19dd50 con 0x7f247c19cfd0 2024-04-21T08:50:15.549 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.547+0000 7f2480a5a640 1 -- 172.21.15.86:0/1271873805 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2474003ac0 con 0x7f247c19cfd0 2024-04-21T08:50:15.550 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.547+0000 7f2481a5c640 1 -- 172.21.15.86:0/1271873805 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f247c1a5570 con 0x7f247c19cfd0 2024-04-21T08:50:15.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.549+0000 7f2480a5a640 1 -- 172.21.15.86:0/1271873805 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2474046020 con 0x7f247c19cfd0 2024-04-21T08:50:15.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.550+0000 7f248464d740 1 Processor -- start 2024-04-21T08:50:15.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.550+0000 7f248464d740 1 -- start start 2024-04-21T08:50:15.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.550+0000 7f2480a5a640 1 --2- 172.21.15.86:0/1271873805 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f24640a3d20 0x7f24640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:15.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.550+0000 7f248464d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572c61eb580 0x5572c61eb960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:15.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.550+0000 7f247a7fc640 1 --2- 172.21.15.86:0/1271873805 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f24640a3d20 0x7f24640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:15.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.550+0000 7f2480a5a640 1 -- 172.21.15.86:0/1271873805 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f24740f2db0 con 0x7f247c19cfd0 2024-04-21T08:50:15.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.550+0000 7f248464d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c61ebea0 0x5572c61f4540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:15.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.550+0000 7f248464d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572c61f4a80 0x5572c61f6e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:15.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.550+0000 7f248464d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5572c61501e0 con 0x5572c61ebea0 2024-04-21T08:50:15.556 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.550+0000 7f248464d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5572c60ed390 con 0x5572c61f4a80 2024-04-21T08:50:15.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.550+0000 7f248464d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5572c614c1a0 con 0x5572c61eb580 2024-04-21T08:50:15.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.550+0000 7f247a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c61ebea0 0x5572c61f4540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:15.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.550+0000 7f247a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c61ebea0 0x5572c61f4540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55530/0 (socket says 172.21.15.86:55530) 2024-04-21T08:50:15.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.550+0000 7f247a7fc640 1 -- 172.21.15.86:0/1731501451 learned_addr learned my addr 172.21.15.86:0/1731501451 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:15.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.550+0000 7f247b7fe640 1 --2- 172.21.15.86:0/1731501451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572c61f4a80 0x5572c61f6e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:15.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.550+0000 7f247affd640 1 --2- 172.21.15.86:0/1731501451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572c61eb580 0x5572c61eb960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:15.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.551+0000 7f247a7fc640 1 --2- 172.21.15.86:0/1271873805 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f24640a3d20 0x7f24640a61e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f247000cf60 tx=0x7f247000cf90 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:15.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.551+0000 7f247a7fc640 1 -- 172.21.15.86:0/1731501451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572c61eb580 msgr2=0x5572c61eb960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:15.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.551+0000 7f247a7fc640 1 --2- 172.21.15.86:0/1731501451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572c61eb580 0x5572c61eb960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.551+0000 7f247a7fc640 1 -- 172.21.15.86:0/1731501451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572c61f4a80 msgr2=0x5572c61f6e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:15.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.551+0000 7f247a7fc640 1 --2- 172.21.15.86:0/1731501451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572c61f4a80 0x5572c61f6e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.551+0000 7f247a7fc640 1 -- 172.21.15.86:0/1731501451 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572c5f2c7d0 con 0x5572c61ebea0 2024-04-21T08:50:15.557 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.551+0000 7f247affd640 1 --2- 172.21.15.86:0/1731501451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572c61eb580 0x5572c61eb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:15.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.551+0000 7f247b7fe640 1 --2- 172.21.15.86:0/1731501451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572c61f4a80 0x5572c61f6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:50:15.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.551+0000 7f247a7fc640 1 --2- 172.21.15.86:0/1731501451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c61ebea0 0x5572c61f4540 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f247000c970 tx=0x7f247001d000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7c1eaa0f91e52030 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:15.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.551+0000 7f2461ffb640 1 -- 172.21.15.86:0/1731501451 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247001e710 con 0x5572c61ebea0 2024-04-21T08:50:15.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.551+0000 7f2461ffb640 1 -- 172.21.15.86:0/1731501451 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f247001ed60 con 0x5572c61ebea0 2024-04-21T08:50:15.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.551+0000 7f2461ffb640 1 -- 172.21.15.86:0/1731501451 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f247001dd80 con 0x5572c61ebea0 2024-04-21T08:50:15.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.551+0000 7f248464d740 1 -- 172.21.15.86:0/1731501451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c61ebea0 msgr2=0x5572c61f4540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:15.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.551+0000 7f248464d740 1 --2- 172.21.15.86:0/1731501451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c61ebea0 0x5572c61f4540 secure :-1 s=READY pgs=2004 cs=0 l=1 rev1=1 crypto rx=0x7f247000c970 tx=0x7f247001d000 comp rx=0 tx=0).stop 2024-04-21T08:50:15.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.551+0000 7f248464d740 1 -- 172.21.15.86:0/1731501451 shutdown_connections 2024-04-21T08:50:15.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.552+0000 7f248464d740 1 --2- 172.21.15.86:0/1731501451 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572c61f4a80 0x5572c61f6e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.552+0000 7f248464d740 1 --2- 172.21.15.86:0/1731501451 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c61ebea0 0x5572c61f4540 unknown :-1 s=CLOSED pgs=2004 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.552+0000 7f248464d740 1 --2- 172.21.15.86:0/1731501451 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572c61eb580 0x5572c61eb960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.552+0000 7f248464d740 1 -- 172.21.15.86:0/1731501451 >> 172.21.15.86:0/1731501451 conn(0x5572c61e5cb0 msgr2=0x5572c61e6090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:15.558 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.552+0000 7f248464d740 1 -- 172.21.15.86:0/1731501451 shutdown_connections 2024-04-21T08:50:15.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.552+0000 7f248464d740 1 -- 172.21.15.86:0/1731501451 wait complete. 2024-04-21T08:50:15.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.552+0000 7f248464d740 1 -- start start 2024-04-21T08:50:15.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.552+0000 7f248464d740 1 Processor -- start 2024-04-21T08:50:15.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.552+0000 7f248464d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572c61eb580 0x5572c61f1ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:15.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.552+0000 7f248464d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572c61ebea0 0x5572c61f1ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:15.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.552+0000 7f248464d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c61f4a80 0x5572c61f2530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:15.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.552+0000 7f248464d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5572c61f7810 con 0x5572c61f4a80 2024-04-21T08:50:15.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.552+0000 7f248464d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5572c61092f0 con 0x5572c61eb580 2024-04-21T08:50:15.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.552+0000 7f248464d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5572c61f7990 con 0x5572c61ebea0 2024-04-21T08:50:15.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.552+0000 7f248464d740 10 client.0 _ll_register_callbacks cb 0x5572c6116580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:15.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.552+0000 7f247b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c61f4a80 0x5572c61f2530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:15.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.552+0000 7f247b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c61f4a80 0x5572c61f2530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55544/0 (socket says 172.21.15.86:55544) 2024-04-21T08:50:15.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.552+0000 7f247b7fe640 1 -- 172.21.15.86:0/2594094356 learned_addr learned my addr 172.21.15.86:0/2594094356 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:15.559 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.552+0000 7f247affd640 1 --2- 172.21.15.86:0/2594094356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572c61eb580 0x5572c61f1ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:15.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.553+0000 7f247a7fc640 1 --2- 172.21.15.86:0/2594094356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572c61ebea0 0x5572c61f1ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:15.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.553+0000 7f247b7fe640 1 -- 172.21.15.86:0/2594094356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572c61ebea0 msgr2=0x5572c61f1ff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:15.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.553+0000 7f247b7fe640 1 --2- 172.21.15.86:0/2594094356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572c61ebea0 0x5572c61f1ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.553+0000 7f247b7fe640 1 -- 172.21.15.86:0/2594094356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572c61eb580 msgr2=0x5572c61f1ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:15.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.553+0000 7f247b7fe640 1 --2- 172.21.15.86:0/2594094356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572c61eb580 0x5572c61f1ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.553+0000 7f247b7fe640 1 -- 172.21.15.86:0/2594094356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572c5e623f0 con 0x5572c61f4a80 2024-04-21T08:50:15.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.553+0000 7f247b7fe640 1 --2- 172.21.15.86:0/2594094356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c61f4a80 0x5572c61f2530 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f2474006010 tx=0x7f2474009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b9daf4a4447db0e2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:15.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.553+0000 7f245affd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:15.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.553+0000 7f245affd640 1 -- 172.21.15.86:0/2594094356 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f24740bea30 con 0x5572c61f4a80 2024-04-21T08:50:15.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.553+0000 7f245affd640 1 -- 172.21.15.86:0/2594094356 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f2474003c60 con 0x5572c61f4a80 2024-04-21T08:50:15.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.553+0000 7f248464d740 1 -- 172.21.15.86:0/2594094356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f247001ed60 con 0x5572c61f4a80 2024-04-21T08:50:15.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.553+0000 7f245affd640 1 -- 172.21.15.86:0/2594094356 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2474003e60 con 0x5572c61f4a80 2024-04-21T08:50:15.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.553+0000 7f248464d740 10 client.20244 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:15.560 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.553+0000 7f248464d740 1 -- 172.21.15.86:0/2594094356 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5572c5e220f0 con 0x5572c61f4a80 2024-04-21T08:50:15.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.553+0000 7f245affd640 1 -- 172.21.15.86:0/2594094356 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f247001ed60 con 0x5572c61f4a80 2024-04-21T08:50:15.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.553+0000 7f245affd640 1 -- 172.21.15.86:0/2594094356 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f24740a90b0 con 0x5572c61f4a80 2024-04-21T08:50:15.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.553+0000 7f248464d740 10 client.20244 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:15.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.553+0000 7f248464d740 10 client.20244 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:15.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.553+0000 7f248464d740 20 client.20244 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:15.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.553+0000 7f248464d740 1 --2- 172.21.15.86:0/2594094356 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5572c62049b0 0x5572c6206df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:15.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.553+0000 7f248464d740 4 client.20244 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:15.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.553+0000 7f248464d740 1 -- 172.21.15.86:0/2594094356 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5572c615dc00 con 0x5572c62049b0 2024-04-21T08:50:15.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.554+0000 7f247affd640 1 --2- 172.21.15.86:0/2594094356 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5572c62049b0 0x5572c6206df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:15.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.554+0000 7f247affd640 1 --2- 172.21.15.86:0/2594094356 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5572c62049b0 0x5572c6206df0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a976f3d0db2ab26 server_cookie=f03bafd6fc3fa1b4 in_seq=0 out_seq=0 2024-04-21T08:50:15.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.554+0000 7f245affd640 10 client.20244 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:15.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.555+0000 7f245affd640 1 -- 172.21.15.86:0/2594094356 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5572c615dc00 con 0x5572c62049b0 2024-04-21T08:50:15.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.555+0000 7f245affd640 10 client.20244 handle_command_reply: tid=0 2024-04-21T08:50:15.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.555+0000 7f248464d740 1 client.20244 shutdown 2024-04-21T08:50:15.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.555+0000 7f248464d740 1 -- 172.21.15.86:0/2594094356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c61f4a80 msgr2=0x5572c61f2530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:15.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.555+0000 7f248464d740 1 --2- 172.21.15.86:0/2594094356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c61f4a80 0x5572c61f2530 secure :-1 s=READY pgs=2005 cs=0 l=1 rev1=1 crypto rx=0x7f2474006010 tx=0x7f2474009260 comp rx=0 tx=0).stop 2024-04-21T08:50:15.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.555+0000 7f248464d740 1 -- 172.21.15.86:0/2594094356 shutdown_connections 2024-04-21T08:50:15.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.555+0000 7f248464d740 1 --2- 172.21.15.86:0/2594094356 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5572c62049b0 0x5572c6206df0 crc :-1 s=READY pgs=331 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.555+0000 7f248464d740 1 --2- 172.21.15.86:0/2594094356 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c61f4a80 0x5572c61f2530 unknown :-1 s=CLOSED pgs=2005 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.555+0000 7f248464d740 1 --2- 172.21.15.86:0/2594094356 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572c61ebea0 0x5572c61f1ff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.555+0000 7f248464d740 1 --2- 172.21.15.86:0/2594094356 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572c61eb580 0x5572c61f1ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.555+0000 7f245affd640 0 client.20244 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:15.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.555+0000 7f248464d740 1 -- 172.21.15.86:0/2594094356 >> 172.21.15.86:0/2594094356 conn(0x5572c61e5cb0 msgr2=0x5572c61e7a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:15.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.555+0000 7f248464d740 1 -- 172.21.15.86:0/2594094356 shutdown_connections 2024-04-21T08:50:15.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.555+0000 7f248464d740 1 -- 172.21.15.86:0/2594094356 wait complete. 2024-04-21T08:50:15.562 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.555+0000 7f248464d740 20 client.20244 trim_cache size 0 max 16384 2024-04-21T08:50:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.572+0000 7f248464d740 1 Processor -- start 2024-04-21T08:50:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.572+0000 7f248464d740 1 -- start start 2024-04-21T08:50:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.572+0000 7f248464d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c61eb580 0x5572c61f41e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.572+0000 7f248464d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572c61ebea0 0x5572c61f0a10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.572+0000 7f248464d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572c61f4a80 0x5572c62006d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.572+0000 7f248464d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f247c1af2d0 con 0x5572c61eb580 2024-04-21T08:50:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.572+0000 7f248464d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f247c1b0e10 con 0x5572c61ebea0 2024-04-21T08:50:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.572+0000 7f248464d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f247c1af9a0 con 0x5572c61f4a80 2024-04-21T08:50:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.573+0000 7f247affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c61eb580 0x5572c61f41e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.573+0000 7f247affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c61eb580 0x5572c61f41e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55552/0 (socket says 172.21.15.86:55552) 2024-04-21T08:50:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.573+0000 7f247affd640 1 -- 172.21.15.86:0/3537498153 learned_addr learned my addr 172.21.15.86:0/3537498153 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:15.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.573+0000 7f247a7fc640 1 --2- 172.21.15.86:0/3537498153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572c61ebea0 0x5572c61f0a10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:15.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.573+0000 7f247affd640 1 -- 172.21.15.86:0/3537498153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572c61f4a80 msgr2=0x5572c62006d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:15.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.573+0000 7f247affd640 1 --2- 172.21.15.86:0/3537498153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572c61f4a80 0x5572c62006d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.573+0000 7f247affd640 1 -- 172.21.15.86:0/3537498153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572c61ebea0 msgr2=0x5572c61f0a10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:15.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.573+0000 7f247affd640 1 --2- 172.21.15.86:0/3537498153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572c61ebea0 0x5572c61f0a10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.573+0000 7f247affd640 1 -- 172.21.15.86:0/3537498153 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5572c61ebcd0 con 0x5572c61eb580 2024-04-21T08:50:15.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.573+0000 7f247affd640 1 --2- 172.21.15.86:0/3537498153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c61eb580 0x5572c61f41e0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x5572c6207600 tx=0x7f246c009680 comp rx=0 tx=0).ready entity=mon.0 client_cookie=12cbf8944f2a2d5c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:15.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.573+0000 7f24617fa640 1 -- 172.21.15.86:0/3537498153 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c01f020 con 0x5572c61eb580 2024-04-21T08:50:15.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.573+0000 7f24617fa640 1 -- 172.21.15.86:0/3537498153 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f246c01b830 con 0x5572c61eb580 2024-04-21T08:50:15.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.573+0000 7f24617fa640 1 -- 172.21.15.86:0/3537498153 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f246c010bd0 con 0x5572c61eb580 2024-04-21T08:50:15.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.574+0000 7f248464d740 1 -- 172.21.15.86:0/3537498153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c61eb580 msgr2=0x5572c61f41e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:15.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.574+0000 7f248464d740 1 --2- 172.21.15.86:0/3537498153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c61eb580 0x5572c61f41e0 secure :-1 s=READY pgs=2006 cs=0 l=1 rev1=1 crypto rx=0x5572c6207600 tx=0x7f246c009680 comp rx=0 tx=0).stop 2024-04-21T08:50:15.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.574+0000 7f248464d740 1 -- 172.21.15.86:0/3537498153 shutdown_connections 2024-04-21T08:50:15.578 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.574+0000 7f248464d740 1 --2- 172.21.15.86:0/3537498153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572c61f4a80 0x5572c62006d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.574+0000 7f248464d740 1 --2- 172.21.15.86:0/3537498153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572c61ebea0 0x5572c61f0a10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.574+0000 7f248464d740 1 --2- 172.21.15.86:0/3537498153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c61eb580 0x5572c61f41e0 unknown :-1 s=CLOSED pgs=2006 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.574+0000 7f248464d740 1 -- 172.21.15.86:0/3537498153 >> 172.21.15.86:0/3537498153 conn(0x5572c61e5cb0 msgr2=0x5572c61e4d70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:15.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.574+0000 7f248464d740 1 -- 172.21.15.86:0/3537498153 shutdown_connections 2024-04-21T08:50:15.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.574+0000 7f248464d740 1 -- 172.21.15.86:0/3537498153 wait complete. 2024-04-21T08:50:15.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.574+0000 7f248464d740 1 -- start start 2024-04-21T08:50:15.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.574+0000 7f248464d740 1 Processor -- start 2024-04-21T08:50:15.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.574+0000 7f248464d740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572c61eb580 0x5572c61eede0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:15.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.574+0000 7f248464d740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c61ebea0 0x5572c61ef320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:15.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.574+0000 7f248464d740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572c61f4a80 0x5572c6269f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:15.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.574+0000 7f248464d740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f247c1b0740 con 0x5572c61ebea0 2024-04-21T08:50:15.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.574+0000 7f248464d740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f247c1b0070 con 0x5572c61f4a80 2024-04-21T08:50:15.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.574+0000 7f248464d740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f247c1ad2d0 con 0x5572c61eb580 2024-04-21T08:50:15.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.574+0000 7f248464d740 10 client.0 _ll_register_callbacks cb 0x7f24740badc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:15.579 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.575+0000 7f247a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c61ebea0 0x5572c61ef320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:15.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.575+0000 7f247a7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c61ebea0 0x5572c61ef320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55554/0 (socket says 172.21.15.86:55554) 2024-04-21T08:50:15.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.575+0000 7f247a7fc640 1 -- 172.21.15.86:0/1718384894 learned_addr learned my addr 172.21.15.86:0/1718384894 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:15.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.575+0000 7f247b7fe640 1 --2- 172.21.15.86:0/1718384894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572c61f4a80 0x5572c6269f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:15.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.575+0000 7f247affd640 1 --2- 172.21.15.86:0/1718384894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572c61eb580 0x5572c61eede0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:15.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.575+0000 7f247a7fc640 1 -- 172.21.15.86:0/1718384894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572c61eb580 msgr2=0x5572c61eede0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:15.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.575+0000 7f247a7fc640 1 --2- 172.21.15.86:0/1718384894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572c61eb580 0x5572c61eede0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.575+0000 7f247a7fc640 1 -- 172.21.15.86:0/1718384894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572c61f4a80 msgr2=0x5572c6269f30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:15.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.575+0000 7f247a7fc640 1 --2- 172.21.15.86:0/1718384894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572c61f4a80 0x5572c6269f30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.575+0000 7f247a7fc640 1 -- 172.21.15.86:0/1718384894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5572c626a470 con 0x5572c61ebea0 2024-04-21T08:50:15.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.575+0000 7f247a7fc640 1 --2- 172.21.15.86:0/1718384894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c61ebea0 0x5572c61ef320 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f2470010ed0 tx=0x7f2470005b40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d81fd2a0c1feb3b8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:15.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.575+0000 7f245affd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:15.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.575+0000 7f245affd640 1 -- 172.21.15.86:0/1718384894 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2470035060 con 0x5572c61ebea0 2024-04-21T08:50:15.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.575+0000 7f245affd640 1 -- 172.21.15.86:0/1718384894 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f2470004870 con 0x5572c61ebea0 2024-04-21T08:50:15.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.575+0000 7f248464d740 1 -- 172.21.15.86:0/1718384894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f246c01b830 con 0x5572c61ebea0 2024-04-21T08:50:15.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.575+0000 7f245affd640 1 -- 172.21.15.86:0/1718384894 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2470004a70 con 0x5572c61ebea0 2024-04-21T08:50:15.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.575+0000 7f248464d740 10 client.20250 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:15.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.575+0000 7f248464d740 1 -- 172.21.15.86:0/1718384894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5572c626b690 con 0x5572c61ebea0 2024-04-21T08:50:15.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.575+0000 7f245affd640 1 -- 172.21.15.86:0/1718384894 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f246c01b830 con 0x5572c61ebea0 2024-04-21T08:50:15.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.576+0000 7f245affd640 1 -- 172.21.15.86:0/1718384894 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f247001ea90 con 0x5572c61ebea0 2024-04-21T08:50:15.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.576+0000 7f248464d740 10 client.20250 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:15.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.576+0000 7f248464d740 10 client.20250 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:15.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.576+0000 7f248464d740 20 client.20250 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:15.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.576+0000 7f248464d740 1 --2- 172.21.15.86:0/1718384894 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5572c626bcd0 0x5572c626e0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:15.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.576+0000 7f248464d740 4 client.20250 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:50:15.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.576+0000 7f248464d740 1 -- 172.21.15.86:0/1718384894 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x5572c6157a70 con 0x5572c626bcd0 2024-04-21T08:50:15.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.576+0000 7f247affd640 1 --2- 172.21.15.86:0/1718384894 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5572c626bcd0 0x5572c626e0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:15.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.576+0000 7f247affd640 1 --2- 172.21.15.86:0/1718384894 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5572c626bcd0 0x5572c626e0f0 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=1aa0c5bb77a877aa server_cookie=58e535897ca83807 in_seq=0 out_seq=0 2024-04-21T08:50:15.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.576+0000 7f245affd640 10 client.20250 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:15.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.577+0000 7f245affd640 1 -- 172.21.15.86:0/1718384894 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x5572c6157a70 con 0x5572c626bcd0 2024-04-21T08:50:15.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.577+0000 7f245affd640 10 client.20250 handle_command_reply: tid=0 2024-04-21T08:50:15.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.577+0000 7f248464d740 1 client.20250 shutdown 2024-04-21T08:50:15.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.577+0000 7f248464d740 1 -- 172.21.15.86:0/1718384894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c61ebea0 msgr2=0x5572c61ef320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:15.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.577+0000 7f248464d740 1 --2- 172.21.15.86:0/1718384894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c61ebea0 0x5572c61ef320 secure :-1 s=READY pgs=2007 cs=0 l=1 rev1=1 crypto rx=0x7f2470010ed0 tx=0x7f2470005b40 comp rx=0 tx=0).stop 2024-04-21T08:50:15.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.577+0000 7f248464d740 1 -- 172.21.15.86:0/1718384894 shutdown_connections 2024-04-21T08:50:15.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.577+0000 7f248464d740 1 --2- 172.21.15.86:0/1718384894 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5572c626bcd0 0x5572c626e0f0 crc :-1 s=READY pgs=332 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.577+0000 7f248464d740 1 --2- 172.21.15.86:0/1718384894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5572c61f4a80 0x5572c6269f30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.577+0000 7f245affd640 0 client.20250 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:15.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.577+0000 7f248464d740 1 --2- 172.21.15.86:0/1718384894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5572c61ebea0 0x5572c61ef320 unknown :-1 s=CLOSED pgs=2007 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.577+0000 7f248464d740 1 --2- 172.21.15.86:0/1718384894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5572c61eb580 0x5572c61eede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.577+0000 7f248464d740 1 -- 172.21.15.86:0/1718384894 >> 172.21.15.86:0/1718384894 conn(0x5572c61e5cb0 msgr2=0x5572c61f1a70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:15.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.577+0000 7f248464d740 1 -- 172.21.15.86:0/1718384894 shutdown_connections 2024-04-21T08:50:15.582 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.577+0000 7f248464d740 1 -- 172.21.15.86:0/1718384894 wait complete. 2024-04-21T08:50:15.583 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.577+0000 7f248464d740 20 client.20250 trim_cache size 0 max 16384 2024-04-21T08:50:15.583 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:50:15.583 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:50:15.583 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-21T08:50:15.583 INFO:teuthology.orchestra.run.smithi086.stdout: "99a77aa5-7273-4b22-bd6d-157cf40464fd": { 2024-04-21T08:50:15.583 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-21T08:50:15.583 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "99a77aa5-7273-4b22-bd6d-157cf40464fd", 2024-04-21T08:50:15.583 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-21T08:50:15.583 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:50:15.583 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:50:15.584 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:50:15.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.577+0000 7f2481a5c640 1 -- 172.21.15.86:0/1271873805 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f24640a3d20 msgr2=0x7f24640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:15.584 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.577+0000 7f2481a5c640 1 --2- 172.21.15.86:0/1271873805 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f24640a3d20 0x7f24640a61e0 secure :-1 s=READY pgs=334 cs=0 l=1 rev1=1 crypto rx=0x7f247000cf60 tx=0x7f247000cf90 comp rx=0 tx=0).stop 2024-04-21T08:50:15.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.577+0000 7f2481a5c640 1 -- 172.21.15.86:0/1271873805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f247c19cfd0 msgr2=0x7f247c19d480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:15.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.577+0000 7f2481a5c640 1 --2- 172.21.15.86:0/1271873805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f247c19cfd0 0x7f247c19d480 secure :-1 s=READY pgs=2003 cs=0 l=1 rev1=1 crypto rx=0x7f2474009230 tx=0x7f24740045e0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.577+0000 7f2481a5c640 1 -- 172.21.15.86:0/1271873805 shutdown_connections 2024-04-21T08:50:15.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.577+0000 7f2481a5c640 1 --2- 172.21.15.86:0/1271873805 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f24640a3d20 0x7f24640a61e0 unknown :-1 s=CLOSED pgs=334 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.577+0000 7f2481a5c640 1 --2- 172.21.15.86:0/1271873805 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f247c19cfd0 0x7f247c19d480 unknown :-1 s=CLOSED pgs=2003 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.577+0000 7f2481a5c640 1 --2- 172.21.15.86:0/1271873805 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f247c107280 0x7f247c198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.577+0000 7f2481a5c640 1 --2- 172.21.15.86:0/1271873805 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f247c1021b0 0x7f247c198710 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:15.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.577+0000 7f2481a5c640 1 -- 172.21.15.86:0/1271873805 >> 172.21.15.86:0/1271873805 conn(0x7f247c0fda10 msgr2=0x7f247c0ffd50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:15.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.578+0000 7f2481a5c640 1 -- 172.21.15.86:0/1271873805 shutdown_connections 2024-04-21T08:50:15.585 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:15.578+0000 7f2481a5c640 1 -- 172.21.15.86:0/1271873805 wait complete. 2024-04-21T08:50:15.587 INFO:tasks.cephfs.filesystem:scrub status for tag:99a77aa5-7273-4b22-bd6d-157cf40464fd - {'path': '/', 'tag': '99a77aa5-7273-4b22-bd6d-157cf40464fd', 'options': 'recursive,force'} 2024-04-21T08:50:16.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:15 smithi088 ceph-mon[31259]: pgmap v359: 129 pgs: 129 active+clean; 5.4 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 19 op/s 2024-04-21T08:50:16.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:15 smithi088 ceph-mon[31259]: scrub queued for path: / 2024-04-21T08:50:16.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:15 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:50:16.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:15 smithi088 ceph-mon[31259]: scrub summary: active paths [/] 2024-04-21T08:50:16.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:15 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:50:16.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:15 smithi179 ceph-mon[31829]: pgmap v359: 129 pgs: 129 active+clean; 5.4 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 19 op/s 2024-04-21T08:50:16.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:15 smithi179 ceph-mon[31829]: scrub queued for path: / 2024-04-21T08:50:16.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:15 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:50:16.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:15 smithi179 ceph-mon[31829]: scrub summary: active paths [/] 2024-04-21T08:50:16.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:15 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:50:16.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:15 smithi086 ceph-mon[27939]: pgmap v359: 129 pgs: 129 active+clean; 5.4 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 51 KiB/s wr, 19 op/s 2024-04-21T08:50:16.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:15 smithi086 ceph-mon[27939]: scrub queued for path: / 2024-04-21T08:50:16.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:15 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:50:16.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:15 smithi086 ceph-mon[27939]: scrub summary: active paths [/] 2024-04-21T08:50:16.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:15 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:50:17.173 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.173+0000 7f5372598640 1 -- 172.21.15.86:0/3206873742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f536c1021c0 msgr2=0x7f536c1025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.173+0000 7f5372598640 1 --2- 172.21.15.86:0/3206873742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f536c1021c0 0x7f536c1025a0 secure :-1 s=READY pgs=2008 cs=0 l=1 rev1=1 crypto rx=0x7f535c009900 tx=0x7f535c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:50:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.173+0000 7f5372598640 1 -- 172.21.15.86:0/3206873742 shutdown_connections 2024-04-21T08:50:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.173+0000 7f5372598640 1 --2- 172.21.15.86:0/3206873742 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f536c107290 0x7f536c10ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.173+0000 7f5372598640 1 --2- 172.21.15.86:0/3206873742 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f536c102b70 0x7f536c106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.173+0000 7f5372598640 1 --2- 172.21.15.86:0/3206873742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f536c1021c0 0x7f536c1025a0 unknown :-1 s=CLOSED pgs=2008 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.174+0000 7f5372598640 1 -- 172.21.15.86:0/3206873742 >> 172.21.15.86:0/3206873742 conn(0x7f536c0fda20 msgr2=0x7f536c0ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.174+0000 7f5372598640 1 -- 172.21.15.86:0/3206873742 shutdown_connections 2024-04-21T08:50:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.174+0000 7f5372598640 1 -- 172.21.15.86:0/3206873742 wait complete. 2024-04-21T08:50:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.174+0000 7f5372598640 1 Processor -- start 2024-04-21T08:50:17.174 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.174+0000 7f5372598640 1 -- start start 2024-04-21T08:50:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.175+0000 7f5372598640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f536c1021c0 0x7f536c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:17.176 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.175+0000 7f5372598640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f536c107290 0x7f536c198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.175+0000 7f5372598640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f536c19cfb0 0x7f536c19d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.175+0000 7f5372598640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f536c1121f0 con 0x7f536c107290 2024-04-21T08:50:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.175+0000 7f5372598640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f536c112070 con 0x7f536c19cfb0 2024-04-21T08:50:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.175+0000 7f5372598640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f536c112370 con 0x7f536c1021c0 2024-04-21T08:50:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.175+0000 7f536b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f536c107290 0x7f536c198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.175+0000 7f536b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f536c107290 0x7f536c198c30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55566/0 (socket says 172.21.15.86:55566) 2024-04-21T08:50:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.175+0000 7f536b7fe640 1 -- 172.21.15.86:0/3712187545 learned_addr learned my addr 172.21.15.86:0/3712187545 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.175+0000 7f536bfff640 1 --2- 172.21.15.86:0/3712187545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f536c1021c0 0x7f536c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.175+0000 7f5370b35640 1 --2- 172.21.15.86:0/3712187545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f536c19cfb0 0x7f536c19d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.175+0000 7f536b7fe640 1 -- 172.21.15.86:0/3712187545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f536c1021c0 msgr2=0x7f536c1986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.175+0000 7f536b7fe640 1 --2- 172.21.15.86:0/3712187545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f536c1021c0 0x7f536c1986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.175+0000 7f536b7fe640 1 -- 172.21.15.86:0/3712187545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f536c19cfb0 msgr2=0x7f536c19d460 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.175+0000 7f536b7fe640 1 --2- 172.21.15.86:0/3712187545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f536c19cfb0 0x7f536c19d460 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:17.177 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.175+0000 7f536b7fe640 1 -- 172.21.15.86:0/3712187545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f536c19daa0 con 0x7f536c107290 2024-04-21T08:50:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.175+0000 7f5370b35640 1 --2- 172.21.15.86:0/3712187545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f536c19cfb0 0x7f536c19d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.175+0000 7f536b7fe640 1 --2- 172.21.15.86:0/3712187545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f536c107290 0x7f536c198c30 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f536000d740 tx=0x7f536000dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.175+0000 7f53697fa640 1 -- 172.21.15.86:0/3712187545 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5360017070 con 0x7f536c107290 2024-04-21T08:50:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.175+0000 7f53697fa640 1 -- 172.21.15.86:0/3712187545 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f536000de50 con 0x7f536c107290 2024-04-21T08:50:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.175+0000 7f5372598640 1 -- 172.21.15.86:0/3712187545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f536c19dd90 con 0x7f536c107290 2024-04-21T08:50:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.175+0000 7f53697fa640 1 -- 172.21.15.86:0/3712187545 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5360004ce0 con 0x7f536c107290 2024-04-21T08:50:17.178 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.176+0000 7f5372598640 1 -- 172.21.15.86:0/3712187545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f536c1a5580 con 0x7f536c107290 2024-04-21T08:50:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.178+0000 7f53697fa640 1 -- 172.21.15.86:0/3712187545 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5360004e80 con 0x7f536c107290 2024-04-21T08:50:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.178+0000 7f5375181740 1 Processor -- start 2024-04-21T08:50:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.178+0000 7f5375181740 1 -- start start 2024-04-21T08:50:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.179+0000 7f5375181740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a02fb98d0 0x562a02fb8620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.179+0000 7f53697fa640 1 --2- 172.21.15.86:0/3712187545 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f53540a3e40 0x7f53540a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.179+0000 7f5375181740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a02fb8b60 0x562a02fb8fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.179+0000 7f5375181740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562a02fc20e0 0x562a02fc44d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.179+0000 7f5375181740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562a02f1d8d0 con 0x562a02fb8b60 2024-04-21T08:50:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.179+0000 7f5375181740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x562a02ebae80 con 0x562a02fc20e0 2024-04-21T08:50:17.185 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.179+0000 7f5375181740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562a02f19890 con 0x562a02fb98d0 2024-04-21T08:50:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.179+0000 7f536bfff640 1 --2- 172.21.15.86:0/3712187545 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f53540a3e40 0x7f53540a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.179+0000 7f536b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a02fb8b60 0x562a02fb8fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.179+0000 7f5370b35640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a02fb98d0 0x562a02fb8620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.179+0000 7f536b7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a02fb8b60 0x562a02fb8fe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55570/0 (socket says 172.21.15.86:55570) 2024-04-21T08:50:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.179+0000 7f536b7fe640 1 -- 172.21.15.86:0/3443763791 learned_addr learned my addr 172.21.15.86:0/3443763791 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.179+0000 7f53697fa640 1 -- 172.21.15.86:0/3712187545 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f53600d0560 con 0x7f536c107290 2024-04-21T08:50:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.179+0000 7f5370b35640 1 --2- 172.21.15.86:0/3443763791 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562a02fc20e0 0x562a02fc44d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.179+0000 7f536b7fe640 1 -- 172.21.15.86:0/3443763791 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a02fb98d0 msgr2=0x562a02fb8620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.179+0000 7f536b7fe640 1 --2- 172.21.15.86:0/3443763791 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a02fb98d0 0x562a02fb8620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.179+0000 7f536b7fe640 1 -- 172.21.15.86:0/3443763791 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562a02fc20e0 msgr2=0x562a02fc44d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.179+0000 7f536b7fe640 1 --2- 172.21.15.86:0/3443763791 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562a02fc20e0 0x562a02fc44d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.179+0000 7f536b7fe640 1 -- 172.21.15.86:0/3443763791 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a02edc320 con 0x562a02fb8b60 2024-04-21T08:50:17.186 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.179+0000 7f5370b35640 1 --2- 172.21.15.86:0/3443763791 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562a02fc20e0 0x562a02fc44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.179+0000 7f536bfff640 1 --2- 172.21.15.86:0/3712187545 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f53540a3e40 0x7f53540a6300 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f535c006010 tx=0x7f535c005e40 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.179+0000 7f536b7fe640 1 --2- 172.21.15.86:0/3443763791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a02fb8b60 0x562a02fb8fe0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f5360098530 tx=0x7f536006b9b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a23572383ce85fc3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.180+0000 7f53527fc640 1 -- 172.21.15.86:0/3443763791 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5360020070 con 0x562a02fb8b60 2024-04-21T08:50:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.180+0000 7f53527fc640 1 -- 172.21.15.86:0/3443763791 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f536009c7d0 con 0x562a02fb8b60 2024-04-21T08:50:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.180+0000 7f53527fc640 1 -- 172.21.15.86:0/3443763791 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f536009a470 con 0x562a02fb8b60 2024-04-21T08:50:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.180+0000 7f5375181740 1 -- 172.21.15.86:0/3443763791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a02fb8b60 msgr2=0x562a02fb8fe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.180+0000 7f5375181740 1 --2- 172.21.15.86:0/3443763791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a02fb8b60 0x562a02fb8fe0 secure :-1 s=READY pgs=2010 cs=0 l=1 rev1=1 crypto rx=0x7f5360098530 tx=0x7f536006b9b0 comp rx=0 tx=0).stop 2024-04-21T08:50:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.180+0000 7f5375181740 1 -- 172.21.15.86:0/3443763791 shutdown_connections 2024-04-21T08:50:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.180+0000 7f5375181740 1 --2- 172.21.15.86:0/3443763791 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562a02fc20e0 0x562a02fc44d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.180+0000 7f5375181740 1 --2- 172.21.15.86:0/3443763791 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a02fb8b60 0x562a02fb8fe0 unknown :-1 s=CLOSED pgs=2010 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.180+0000 7f5375181740 1 --2- 172.21.15.86:0/3443763791 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a02fb98d0 0x562a02fb8620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.180+0000 7f5375181740 1 -- 172.21.15.86:0/3443763791 >> 172.21.15.86:0/3443763791 conn(0x562a02f7fc90 msgr2=0x562a02fb3850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.180+0000 7f5375181740 1 -- 172.21.15.86:0/3443763791 shutdown_connections 2024-04-21T08:50:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.180+0000 7f5375181740 1 -- 172.21.15.86:0/3443763791 wait complete. 2024-04-21T08:50:17.187 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.181+0000 7f5375181740 1 -- start start 2024-04-21T08:50:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.181+0000 7f5375181740 1 Processor -- start 2024-04-21T08:50:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.181+0000 7f5375181740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a02fb8b60 0x562a02fc1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.181+0000 7f5375181740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562a02fb98d0 0x562a02fbeed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.181+0000 7f5375181740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a02fc20e0 0x562a02fbf5f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.181+0000 7f5375181740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x562a02fc4d50 con 0x562a02fb8b60 2024-04-21T08:50:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.181+0000 7f5375181740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x562a02ea5eb0 con 0x562a02fb98d0 2024-04-21T08:50:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.181+0000 7f5375181740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x562a02fc4ed0 con 0x562a02fc20e0 2024-04-21T08:50:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.181+0000 7f5375181740 10 client.0 _ll_register_callbacks cb 0x562a02ed2b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.181+0000 7f536bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a02fb8b60 0x562a02fc1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.181+0000 7f536bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a02fb8b60 0x562a02fc1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55578/0 (socket says 172.21.15.86:55578) 2024-04-21T08:50:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.181+0000 7f536bfff640 1 -- 172.21.15.86:0/3425941015 learned_addr learned my addr 172.21.15.86:0/3425941015 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.182+0000 7f536b7fe640 1 --2- 172.21.15.86:0/3425941015 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562a02fb98d0 0x562a02fbeed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.182+0000 7f5370b35640 1 --2- 172.21.15.86:0/3425941015 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a02fc20e0 0x562a02fbf5f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:17.188 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.182+0000 7f536bfff640 1 -- 172.21.15.86:0/3425941015 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a02fc20e0 msgr2=0x562a02fbf5f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.182+0000 7f536bfff640 1 --2- 172.21.15.86:0/3425941015 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a02fc20e0 0x562a02fbf5f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.182+0000 7f536bfff640 1 -- 172.21.15.86:0/3425941015 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562a02fb98d0 msgr2=0x562a02fbeed0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.182+0000 7f536bfff640 1 --2- 172.21.15.86:0/3425941015 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562a02fb98d0 0x562a02fbeed0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.182+0000 7f536bfff640 1 -- 172.21.15.86:0/3425941015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a02c30210 con 0x562a02fb8b60 2024-04-21T08:50:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.182+0000 7f5370b35640 1 --2- 172.21.15.86:0/3425941015 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a02fc20e0 0x562a02fbf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.182+0000 7f536bfff640 1 --2- 172.21.15.86:0/3425941015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a02fb8b60 0x562a02fc1b40 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f535c0096c0 tx=0x7f535c009260 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cb8c62b0ad1839cc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.182+0000 7f534b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.182+0000 7f534b7fe640 1 -- 172.21.15.86:0/3425941015 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c04f020 con 0x562a02fb8b60 2024-04-21T08:50:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.182+0000 7f534b7fe640 1 -- 172.21.15.86:0/3425941015 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f535c0046c0 con 0x562a02fb8b60 2024-04-21T08:50:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.182+0000 7f5375181740 1 -- 172.21.15.86:0/3425941015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f536009c7d0 con 0x562a02fb8b60 2024-04-21T08:50:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.182+0000 7f534b7fe640 1 -- 172.21.15.86:0/3425941015 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f535c05a050 con 0x562a02fb8b60 2024-04-21T08:50:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.182+0000 7f5375181740 10 client.20262 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.182+0000 7f5375181740 1 -- 172.21.15.86:0/3425941015 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562a02cfc620 con 0x562a02fb8b60 2024-04-21T08:50:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.182+0000 7f534b7fe640 1 -- 172.21.15.86:0/3425941015 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f536009c7d0 con 0x562a02fb8b60 2024-04-21T08:50:17.189 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.183+0000 7f534b7fe640 1 -- 172.21.15.86:0/3425941015 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f535c003af0 con 0x562a02fb8b60 2024-04-21T08:50:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.183+0000 7f5375181740 10 client.20262 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.183+0000 7f5375181740 10 client.20262 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.183+0000 7f5375181740 20 client.20262 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.183+0000 7f5375181740 1 --2- 172.21.15.86:0/3425941015 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x562a02fd2120 0x562a02fd4510 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.183+0000 7f5375181740 4 client.20262 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.183+0000 7f5375181740 1 -- 172.21.15.86:0/3425941015 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x562a02f31880 con 0x562a02fd2120 2024-04-21T08:50:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.183+0000 7f536b7fe640 1 --2- 172.21.15.86:0/3425941015 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x562a02fd2120 0x562a02fd4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.183+0000 7f536b7fe640 1 --2- 172.21.15.86:0/3425941015 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x562a02fd2120 0x562a02fd4510 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8b82fdf95bd67145 server_cookie=b4281520966882e1 in_seq=0 out_seq=0 2024-04-21T08:50:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.183+0000 7f534b7fe640 10 client.20262 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.184+0000 7f534b7fe640 1 -- 172.21.15.86:0/3425941015 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x562a02f31880 con 0x562a02fd2120 2024-04-21T08:50:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.184+0000 7f534b7fe640 10 client.20262 handle_command_reply: tid=0 2024-04-21T08:50:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.184+0000 7f5375181740 1 client.20262 shutdown 2024-04-21T08:50:17.190 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.184+0000 7f5375181740 1 -- 172.21.15.86:0/3425941015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a02fb8b60 msgr2=0x562a02fc1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.184+0000 7f5375181740 1 --2- 172.21.15.86:0/3425941015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a02fb8b60 0x562a02fc1b40 secure :-1 s=READY pgs=2011 cs=0 l=1 rev1=1 crypto rx=0x7f535c0096c0 tx=0x7f535c009260 comp rx=0 tx=0).stop 2024-04-21T08:50:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.184+0000 7f5375181740 1 -- 172.21.15.86:0/3425941015 shutdown_connections 2024-04-21T08:50:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.184+0000 7f5375181740 1 --2- 172.21.15.86:0/3425941015 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x562a02fd2120 0x562a02fd4510 crc :-1 s=READY pgs=333 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.184+0000 7f5375181740 1 --2- 172.21.15.86:0/3425941015 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a02fc20e0 0x562a02fbf5f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.184+0000 7f5375181740 1 --2- 172.21.15.86:0/3425941015 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562a02fb98d0 0x562a02fbeed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.184+0000 7f5375181740 1 --2- 172.21.15.86:0/3425941015 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a02fb8b60 0x562a02fc1b40 unknown :-1 s=CLOSED pgs=2011 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.184+0000 7f534b7fe640 0 client.20262 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.184+0000 7f5375181740 1 -- 172.21.15.86:0/3425941015 >> 172.21.15.86:0/3425941015 conn(0x562a02f7fc90 msgr2=0x562a02fb55a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.184+0000 7f5375181740 1 -- 172.21.15.86:0/3425941015 shutdown_connections 2024-04-21T08:50:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.185+0000 7f5375181740 1 -- 172.21.15.86:0/3425941015 wait complete. 2024-04-21T08:50:17.191 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.185+0000 7f5375181740 20 client.20262 trim_cache size 0 max 16384 2024-04-21T08:50:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.199+0000 7f5375181740 1 Processor -- start 2024-04-21T08:50:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.199+0000 7f5375181740 1 -- start start 2024-04-21T08:50:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.199+0000 7f5375181740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a02fb8b60 0x562a02fcdc80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.199+0000 7f5375181740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562a02fb98d0 0x562a02fc1600 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.199+0000 7f5375181740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a02fc20e0 0x562a02fc1b40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:17.204 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.199+0000 7f5375181740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f536c1af3e0 con 0x562a02fc20e0 2024-04-21T08:50:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.199+0000 7f5375181740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f536c1b0f20 con 0x562a02fb98d0 2024-04-21T08:50:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.199+0000 7f5375181740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f536c1afab0 con 0x562a02fb8b60 2024-04-21T08:50:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.199+0000 7f5370b35640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a02fc20e0 0x562a02fc1b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.200+0000 7f5370b35640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a02fc20e0 0x562a02fc1b40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55584/0 (socket says 172.21.15.86:55584) 2024-04-21T08:50:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.200+0000 7f5370b35640 1 -- 172.21.15.86:0/1989906090 learned_addr learned my addr 172.21.15.86:0/1989906090 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.200+0000 7f536b7fe640 1 --2- 172.21.15.86:0/1989906090 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562a02fb98d0 0x562a02fc1600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.200+0000 7f536bfff640 1 --2- 172.21.15.86:0/1989906090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a02fb8b60 0x562a02fcdc80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.200+0000 7f5370b35640 1 -- 172.21.15.86:0/1989906090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a02fb8b60 msgr2=0x562a02fcdc80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.200+0000 7f5370b35640 1 --2- 172.21.15.86:0/1989906090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a02fb8b60 0x562a02fcdc80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.200+0000 7f5370b35640 1 -- 172.21.15.86:0/1989906090 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562a02fb98d0 msgr2=0x562a02fc1600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.200+0000 7f5370b35640 1 --2- 172.21.15.86:0/1989906090 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562a02fb98d0 0x562a02fc1600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:17.205 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.200+0000 7f5370b35640 1 -- 172.21.15.86:0/1989906090 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x562a02faef30 con 0x562a02fc20e0 2024-04-21T08:50:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.200+0000 7f536b7fe640 1 --2- 172.21.15.86:0/1989906090 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562a02fb98d0 0x562a02fc1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.200+0000 7f5370b35640 1 --2- 172.21.15.86:0/1989906090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a02fc20e0 0x562a02fc1b40 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f5364013700 tx=0x7f5364004a40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=85b1feddbae15935 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.200+0000 7f5351ffb640 1 -- 172.21.15.86:0/1989906090 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5364004d20 con 0x562a02fc20e0 2024-04-21T08:50:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.200+0000 7f5351ffb640 1 -- 172.21.15.86:0/1989906090 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f536401a030 con 0x562a02fc20e0 2024-04-21T08:50:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.200+0000 7f5351ffb640 1 -- 172.21.15.86:0/1989906090 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f53640058e0 con 0x562a02fc20e0 2024-04-21T08:50:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.201+0000 7f5375181740 1 -- 172.21.15.86:0/1989906090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a02fc20e0 msgr2=0x562a02fc1b40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.201+0000 7f5375181740 1 --2- 172.21.15.86:0/1989906090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a02fc20e0 0x562a02fc1b40 secure :-1 s=READY pgs=2012 cs=0 l=1 rev1=1 crypto rx=0x7f5364013700 tx=0x7f5364004a40 comp rx=0 tx=0).stop 2024-04-21T08:50:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.201+0000 7f5375181740 1 -- 172.21.15.86:0/1989906090 shutdown_connections 2024-04-21T08:50:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.201+0000 7f5375181740 1 --2- 172.21.15.86:0/1989906090 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a02fc20e0 0x562a02fc1b40 unknown :-1 s=CLOSED pgs=2012 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.201+0000 7f5375181740 1 --2- 172.21.15.86:0/1989906090 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562a02fb98d0 0x562a02fc1600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.201+0000 7f5375181740 1 --2- 172.21.15.86:0/1989906090 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a02fb8b60 0x562a02fcdc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.201+0000 7f5375181740 1 -- 172.21.15.86:0/1989906090 >> 172.21.15.86:0/1989906090 conn(0x562a02f7fc90 msgr2=0x562a02fba060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.201+0000 7f5375181740 1 -- 172.21.15.86:0/1989906090 shutdown_connections 2024-04-21T08:50:17.206 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.201+0000 7f5375181740 1 -- 172.21.15.86:0/1989906090 wait complete. 2024-04-21T08:50:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.201+0000 7f5375181740 1 -- start start 2024-04-21T08:50:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.201+0000 7f5375181740 1 Processor -- start 2024-04-21T08:50:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.202+0000 7f5375181740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a02fb8b60 0x562a02fca190 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.202+0000 7f5375181740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a02fb98d0 0x562a02fca6d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.202+0000 7f5375181740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562a02fc20e0 0x562a02fbc150 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.202+0000 7f5375181740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f536c1b0850 con 0x562a02fb8b60 2024-04-21T08:50:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.202+0000 7f5375181740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f536c1b0180 con 0x562a02fc20e0 2024-04-21T08:50:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.202+0000 7f5375181740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f536c1ad3e0 con 0x562a02fb98d0 2024-04-21T08:50:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.202+0000 7f5375181740 10 client.0 _ll_register_callbacks cb 0x7f535c02c610 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.202+0000 7f536bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a02fb8b60 0x562a02fca190 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.202+0000 7f536bfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a02fb8b60 0x562a02fca190 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:55588/0 (socket says 172.21.15.86:55588) 2024-04-21T08:50:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.202+0000 7f536bfff640 1 -- 172.21.15.86:0/3733780113 learned_addr learned my addr 172.21.15.86:0/3733780113 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.202+0000 7f5370b35640 1 --2- 172.21.15.86:0/3733780113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562a02fc20e0 0x562a02fbc150 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:17.207 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.202+0000 7f536b7fe640 1 --2- 172.21.15.86:0/3733780113 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a02fb98d0 0x562a02fca6d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.202+0000 7f536bfff640 1 -- 172.21.15.86:0/3733780113 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a02fb98d0 msgr2=0x562a02fca6d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.202+0000 7f536bfff640 1 --2- 172.21.15.86:0/3733780113 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a02fb98d0 0x562a02fca6d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.202+0000 7f536bfff640 1 -- 172.21.15.86:0/3733780113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562a02fc20e0 msgr2=0x562a02fbc150 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.202+0000 7f536bfff640 1 --2- 172.21.15.86:0/3733780113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562a02fc20e0 0x562a02fbc150 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.202+0000 7f536bfff640 1 -- 172.21.15.86:0/3733780113 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x562a02fbc7c0 con 0x562a02fb8b60 2024-04-21T08:50:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.202+0000 7f536b7fe640 1 --2- 172.21.15.86:0/3733780113 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a02fb98d0 0x562a02fca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.202+0000 7f536bfff640 1 --2- 172.21.15.86:0/3733780113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a02fb8b60 0x562a02fca190 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f535c000c00 tx=0x7f535c0417b0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=744c5cf0b1a3f034 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.202+0000 7f534b7fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.202+0000 7f534b7fe640 1 -- 172.21.15.86:0/3733780113 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f535c062020 con 0x562a02fb8b60 2024-04-21T08:50:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.202+0000 7f534b7fe640 1 -- 172.21.15.86:0/3733780113 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f535c041c10 con 0x562a02fb8b60 2024-04-21T08:50:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.202+0000 7f5375181740 1 -- 172.21.15.86:0/3733780113 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f536401a030 con 0x562a02fb8b60 2024-04-21T08:50:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.202+0000 7f534b7fe640 1 -- 172.21.15.86:0/3733780113 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f535c06d050 con 0x562a02fb8b60 2024-04-21T08:50:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.202+0000 7f5375181740 10 client.20268 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:17.208 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.202+0000 7f5375181740 1 -- 172.21.15.86:0/3733780113 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x562a02fc2e30 con 0x562a02fb8b60 2024-04-21T08:50:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.203+0000 7f534b7fe640 1 -- 172.21.15.86:0/3733780113 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f536401a030 con 0x562a02fb8b60 2024-04-21T08:50:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.203+0000 7f534b7fe640 1 -- 172.21.15.86:0/3733780113 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f535c0047e0 con 0x562a02fb8b60 2024-04-21T08:50:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.203+0000 7f5375181740 10 client.20268 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.203+0000 7f5375181740 10 client.20268 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.203+0000 7f5375181740 20 client.20268 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.203+0000 7f5375181740 1 --2- 172.21.15.86:0/3733780113 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x562a030387f0 0x562a0303abe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.203+0000 7f5375181740 4 client.20268 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:50:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.203+0000 7f5375181740 1 -- 172.21.15.86:0/3733780113 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x562a02fc4a20 con 0x562a030387f0 2024-04-21T08:50:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.203+0000 7f536b7fe640 1 --2- 172.21.15.86:0/3733780113 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x562a030387f0 0x562a0303abe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:17.209 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.204+0000 7f536b7fe640 1 --2- 172.21.15.86:0/3733780113 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x562a030387f0 0x562a0303abe0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=d7d59c8a32dd8d6b server_cookie=abc496673fcf7cb in_seq=0 out_seq=0 2024-04-21T08:50:17.210 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.204+0000 7f534b7fe640 10 client.20268 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.217+0000 7f534b7fe640 1 -- 172.21.15.86:0/3733780113 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x562a02fc4a20 con 0x562a030387f0 2024-04-21T08:50:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.217+0000 7f534b7fe640 10 client.20268 handle_command_reply: tid=0 2024-04-21T08:50:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.217+0000 7f5375181740 1 client.20268 shutdown 2024-04-21T08:50:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.217+0000 7f5375181740 1 -- 172.21.15.86:0/3733780113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a02fb8b60 msgr2=0x562a02fca190 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.217+0000 7f5375181740 1 --2- 172.21.15.86:0/3733780113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a02fb8b60 0x562a02fca190 secure :-1 s=READY pgs=2013 cs=0 l=1 rev1=1 crypto rx=0x7f535c000c00 tx=0x7f535c0417b0 comp rx=0 tx=0).stop 2024-04-21T08:50:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.217+0000 7f5375181740 1 -- 172.21.15.86:0/3733780113 shutdown_connections 2024-04-21T08:50:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.217+0000 7f5375181740 1 --2- 172.21.15.86:0/3733780113 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x562a030387f0 0x562a0303abe0 crc :-1 s=READY pgs=334 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:17.217 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.217+0000 7f5375181740 1 --2- 172.21.15.86:0/3733780113 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x562a02fc20e0 0x562a02fbc150 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.217+0000 7f5375181740 1 --2- 172.21.15.86:0/3733780113 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x562a02fb98d0 0x562a02fca6d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.217+0000 7f534b7fe640 0 client.20268 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.217+0000 7f5375181740 1 --2- 172.21.15.86:0/3733780113 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x562a02fb8b60 0x562a02fca190 unknown :-1 s=CLOSED pgs=2013 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.217+0000 7f5375181740 1 -- 172.21.15.86:0/3733780113 >> 172.21.15.86:0/3733780113 conn(0x562a02f7fc90 msgr2=0x562a02fbd7a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.217+0000 7f5375181740 1 -- 172.21.15.86:0/3733780113 shutdown_connections 2024-04-21T08:50:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.217+0000 7f5375181740 1 -- 172.21.15.86:0/3733780113 wait complete. 2024-04-21T08:50:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.217+0000 7f5375181740 20 client.20268 trim_cache size 0 max 16384 2024-04-21T08:50:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:50:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:50:17.218 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:50:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:50:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.218+0000 7f5372598640 1 -- 172.21.15.86:0/3712187545 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f53540a3e40 msgr2=0x7f53540a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:17.219 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.218+0000 7f5372598640 1 --2- 172.21.15.86:0/3712187545 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f53540a3e40 0x7f53540a6300 secure :-1 s=READY pgs=335 cs=0 l=1 rev1=1 crypto rx=0x7f535c006010 tx=0x7f535c005e40 comp rx=0 tx=0).stop 2024-04-21T08:50:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.218+0000 7f5372598640 1 -- 172.21.15.86:0/3712187545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f536c107290 msgr2=0x7f536c198c30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.218+0000 7f5372598640 1 --2- 172.21.15.86:0/3712187545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f536c107290 0x7f536c198c30 secure :-1 s=READY pgs=2009 cs=0 l=1 rev1=1 crypto rx=0x7f536000d740 tx=0x7f536000dc00 comp rx=0 tx=0).stop 2024-04-21T08:50:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.218+0000 7f5372598640 1 -- 172.21.15.86:0/3712187545 shutdown_connections 2024-04-21T08:50:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.218+0000 7f5372598640 1 --2- 172.21.15.86:0/3712187545 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f53540a3e40 0x7f53540a6300 unknown :-1 s=CLOSED pgs=335 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.218+0000 7f5372598640 1 --2- 172.21.15.86:0/3712187545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f536c19cfb0 0x7f536c19d460 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.218+0000 7f5372598640 1 --2- 172.21.15.86:0/3712187545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f536c107290 0x7f536c198c30 unknown :-1 s=CLOSED pgs=2009 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.218+0000 7f5372598640 1 --2- 172.21.15.86:0/3712187545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f536c1021c0 0x7f536c1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.218+0000 7f5372598640 1 -- 172.21.15.86:0/3712187545 >> 172.21.15.86:0/3712187545 conn(0x7f536c0fda20 msgr2=0x7f536c0fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.218+0000 7f5372598640 1 -- 172.21.15.86:0/3712187545 shutdown_connections 2024-04-21T08:50:17.220 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:17.218+0000 7f5372598640 1 -- 172.21.15.86:0/3712187545 wait complete. 2024-04-21T08:50:18.215 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:17 smithi088 ceph-mon[31259]: pgmap v360: 129 pgs: 129 active+clean; 5.4 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 19 op/s 2024-04-21T08:50:18.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:17 smithi179 ceph-mon[31829]: pgmap v360: 129 pgs: 129 active+clean; 5.4 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 19 op/s 2024-04-21T08:50:18.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:17 smithi086 ceph-mon[27939]: pgmap v360: 129 pgs: 129 active+clean; 5.4 MiB data, 445 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 51 KiB/s wr, 19 op/s 2024-04-21T08:50:20.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:19 smithi088 ceph-mon[31259]: pgmap v361: 129 pgs: 129 active+clean; 1.7 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 21 op/s 2024-04-21T08:50:20.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:19 smithi088 ceph-mon[31259]: scrub summary: idle 2024-04-21T08:50:20.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:19 smithi179 ceph-mon[31829]: pgmap v361: 129 pgs: 129 active+clean; 1.7 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 21 op/s 2024-04-21T08:50:20.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:19 smithi179 ceph-mon[31829]: scrub summary: idle 2024-04-21T08:50:20.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:19 smithi086 ceph-mon[27939]: pgmap v361: 129 pgs: 129 active+clean; 1.7 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 51 KiB/s rd, 50 KiB/s wr, 21 op/s 2024-04-21T08:50:20.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:19 smithi086 ceph-mon[27939]: scrub summary: idle 2024-04-21T08:50:22.169 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:21 smithi086 ceph-mon[27939]: pgmap v362: 129 pgs: 129 active+clean; 1.7 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 49 KiB/s wr, 21 op/s 2024-04-21T08:50:22.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:21 smithi088 ceph-mon[31259]: pgmap v362: 129 pgs: 129 active+clean; 1.7 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 49 KiB/s wr, 21 op/s 2024-04-21T08:50:22.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:21 smithi179 ceph-mon[31829]: pgmap v362: 129 pgs: 129 active+clean; 1.7 MiB data, 442 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 49 KiB/s wr, 21 op/s 2024-04-21T08:50:24.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:23 smithi088 ceph-mon[31259]: pgmap v363: 129 pgs: 129 active+clean; 1.9 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 24 op/s 2024-04-21T08:50:24.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:23 smithi179 ceph-mon[31829]: pgmap v363: 129 pgs: 129 active+clean; 1.9 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 24 op/s 2024-04-21T08:50:24.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:23 smithi086 ceph-mon[27939]: pgmap v363: 129 pgs: 129 active+clean; 1.9 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 58 KiB/s wr, 24 op/s 2024-04-21T08:50:26.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:25 smithi088 ceph-mon[31259]: pgmap v364: 129 pgs: 129 active+clean; 1.9 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 KiB/s wr, 17 op/s 2024-04-21T08:50:26.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:25 smithi179 ceph-mon[31829]: pgmap v364: 129 pgs: 129 active+clean; 1.9 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 KiB/s wr, 17 op/s 2024-04-21T08:50:26.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:25 smithi086 ceph-mon[27939]: pgmap v364: 129 pgs: 129 active+clean; 1.9 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 KiB/s wr, 17 op/s 2024-04-21T08:50:26.543 INFO:tasks.workunit.client.0.smithi086.stdout:499/600: open fd = 3 2024-04-21T08:50:26.543 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.543 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.543 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.543 INFO:tasks.workunit.client.0.smithi086.stdout:500/600: open fd = 3 2024-04-21T08:50:26.543 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.543 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.543 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.543 INFO:tasks.workunit.client.0.smithi086.stdout:501/600: open fd = 3 2024-04-21T08:50:26.543 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.543 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.543 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.543 INFO:tasks.workunit.client.0.smithi086.stdout:502/600: open fd = 3 2024-04-21T08:50:26.544 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.544 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.544 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.544 INFO:tasks.workunit.client.0.smithi086.stdout:503/600: open fd = 3 2024-04-21T08:50:26.544 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.544 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.544 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.544 INFO:tasks.workunit.client.0.smithi086.stdout:504/600: open fd = 3 2024-04-21T08:50:26.544 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.544 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.544 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.544 INFO:tasks.workunit.client.0.smithi086.stdout:505/600: open fd = 3 2024-04-21T08:50:26.545 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.545 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.545 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.545 INFO:tasks.workunit.client.0.smithi086.stdout:506/600: open fd = 3 2024-04-21T08:50:26.545 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.545 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.545 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.545 INFO:tasks.workunit.client.0.smithi086.stdout:507/600: open fd = 3 2024-04-21T08:50:26.545 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.545 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.545 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.545 INFO:tasks.workunit.client.0.smithi086.stdout:508/600: open fd = 3 2024-04-21T08:50:26.545 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.546 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.546 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.546 INFO:tasks.workunit.client.0.smithi086.stdout:509/600: open fd = 3 2024-04-21T08:50:26.546 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.546 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.546 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.546 INFO:tasks.workunit.client.0.smithi086.stdout:510/600: open fd = 3 2024-04-21T08:50:26.546 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.546 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.546 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.546 INFO:tasks.workunit.client.0.smithi086.stdout:511/600: open fd = 3 2024-04-21T08:50:26.546 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.546 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.547 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.547 INFO:tasks.workunit.client.0.smithi086.stdout:512/600: open fd = 3 2024-04-21T08:50:26.547 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.547 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.547 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.547 INFO:tasks.workunit.client.0.smithi086.stdout:513/600: open fd = 3 2024-04-21T08:50:26.547 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.547 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.547 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.547 INFO:tasks.workunit.client.0.smithi086.stdout:514/600: open fd = 3 2024-04-21T08:50:26.547 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.547 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.547 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.548 INFO:tasks.workunit.client.0.smithi086.stdout:515/600: open fd = 3 2024-04-21T08:50:26.548 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.548 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.548 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.548 INFO:tasks.workunit.client.0.smithi086.stdout:516/600: open fd = 3 2024-04-21T08:50:26.548 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.548 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.548 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.548 INFO:tasks.workunit.client.0.smithi086.stdout:517/600: open fd = 3 2024-04-21T08:50:26.548 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.548 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.548 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.549 INFO:tasks.workunit.client.0.smithi086.stdout:518/600: open fd = 3 2024-04-21T08:50:26.549 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.549 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.549 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.549 INFO:tasks.workunit.client.0.smithi086.stdout:519/600: open fd = 3 2024-04-21T08:50:26.549 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.549 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.549 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.549 INFO:tasks.workunit.client.0.smithi086.stdout:520/600: open fd = 3 2024-04-21T08:50:26.549 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.549 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.549 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.549 INFO:tasks.workunit.client.0.smithi086.stdout:521/600: open fd = 3 2024-04-21T08:50:26.549 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.550 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.550 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.550 INFO:tasks.workunit.client.0.smithi086.stdout:522/600: open fd = 3 2024-04-21T08:50:26.550 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.550 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.550 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.550 INFO:tasks.workunit.client.0.smithi086.stdout:523/600: open fd = 3 2024-04-21T08:50:26.550 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.550 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.550 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.550 INFO:tasks.workunit.client.0.smithi086.stdout:524/600: open fd = 3 2024-04-21T08:50:26.550 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.550 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.550 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.551 INFO:tasks.workunit.client.0.smithi086.stdout:525/600: open fd = 3 2024-04-21T08:50:26.551 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.551 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.551 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.551 INFO:tasks.workunit.client.0.smithi086.stdout:526/600: open fd = 3 2024-04-21T08:50:26.551 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.551 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.551 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.551 INFO:tasks.workunit.client.0.smithi086.stdout:527/600: open fd = 3 2024-04-21T08:50:26.551 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.551 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.551 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.551 INFO:tasks.workunit.client.0.smithi086.stdout:528/600: open fd = 3 2024-04-21T08:50:26.551 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.551 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.552 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.552 INFO:tasks.workunit.client.0.smithi086.stdout:529/600: open fd = 3 2024-04-21T08:50:26.552 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.552 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.552 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.552 INFO:tasks.workunit.client.0.smithi086.stdout:530/600: open fd = 3 2024-04-21T08:50:26.552 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.552 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.552 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.552 INFO:tasks.workunit.client.0.smithi086.stdout:531/600: open fd = 3 2024-04-21T08:50:26.552 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.552 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.552 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.552 INFO:tasks.workunit.client.0.smithi086.stdout:532/600: open fd = 3 2024-04-21T08:50:26.552 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.553 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.553 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.553 INFO:tasks.workunit.client.0.smithi086.stdout:533/600: open fd = 3 2024-04-21T08:50:26.553 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.553 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.553 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.553 INFO:tasks.workunit.client.0.smithi086.stdout:534/600: open fd = 3 2024-04-21T08:50:26.553 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.553 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.553 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.553 INFO:tasks.workunit.client.0.smithi086.stdout:535/600: open fd = 3 2024-04-21T08:50:26.553 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.553 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.553 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.554 INFO:tasks.workunit.client.0.smithi086.stdout:536/600: open fd = 3 2024-04-21T08:50:26.554 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.554 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.554 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.554 INFO:tasks.workunit.client.0.smithi086.stdout:537/600: open fd = 3 2024-04-21T08:50:26.554 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.554 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.554 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.554 INFO:tasks.workunit.client.0.smithi086.stdout:538/600: open fd = 3 2024-04-21T08:50:26.554 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.554 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.554 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.554 INFO:tasks.workunit.client.0.smithi086.stdout:539/600: open fd = 3 2024-04-21T08:50:26.554 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.554 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.555 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.555 INFO:tasks.workunit.client.0.smithi086.stdout:540/600: open fd = 3 2024-04-21T08:50:26.555 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.555 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.555 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.555 INFO:tasks.workunit.client.0.smithi086.stdout:541/600: open fd = 3 2024-04-21T08:50:26.555 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.555 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.555 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.555 INFO:tasks.workunit.client.0.smithi086.stdout:542/600: open fd = 3 2024-04-21T08:50:26.555 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.555 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.555 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.555 INFO:tasks.workunit.client.0.smithi086.stdout:543/600: open fd = 3 2024-04-21T08:50:26.556 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.556 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.556 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.556 INFO:tasks.workunit.client.0.smithi086.stdout:544/600: open fd = 3 2024-04-21T08:50:26.556 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.556 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.556 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.556 INFO:tasks.workunit.client.0.smithi086.stdout:545/600: open fd = 3 2024-04-21T08:50:26.556 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.556 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.556 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.556 INFO:tasks.workunit.client.0.smithi086.stdout:546/600: open fd = 3 2024-04-21T08:50:26.556 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.557 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.557 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.557 INFO:tasks.workunit.client.0.smithi086.stdout:547/600: open fd = 3 2024-04-21T08:50:26.557 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.557 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.557 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.557 INFO:tasks.workunit.client.0.smithi086.stdout:548/600: open fd = 3 2024-04-21T08:50:26.557 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.557 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.557 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.558 INFO:tasks.workunit.client.0.smithi086.stdout:549/600: open fd = 3 2024-04-21T08:50:26.558 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.558 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.558 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.558 INFO:tasks.workunit.client.0.smithi086.stdout:550/600: open fd = 3 2024-04-21T08:50:26.558 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.558 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.558 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.558 INFO:tasks.workunit.client.0.smithi086.stdout:551/600: open fd = 3 2024-04-21T08:50:26.558 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.558 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.558 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.558 INFO:tasks.workunit.client.0.smithi086.stdout:552/600: open fd = 3 2024-04-21T08:50:26.559 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.559 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.559 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.559 INFO:tasks.workunit.client.0.smithi086.stdout:553/600: open fd = 3 2024-04-21T08:50:26.559 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.559 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.559 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.559 INFO:tasks.workunit.client.0.smithi086.stdout:554/600: open fd = 3 2024-04-21T08:50:26.559 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.559 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.559 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.559 INFO:tasks.workunit.client.0.smithi086.stdout:555/600: open fd = 3 2024-04-21T08:50:26.559 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.559 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.560 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.560 INFO:tasks.workunit.client.0.smithi086.stdout:556/600: open fd = 3 2024-04-21T08:50:26.560 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.560 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.560 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.560 INFO:tasks.workunit.client.0.smithi086.stdout:557/600: open fd = 3 2024-04-21T08:50:26.560 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.560 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.560 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.560 INFO:tasks.workunit.client.0.smithi086.stdout:558/600: open fd = 3 2024-04-21T08:50:26.560 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.560 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.560 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.560 INFO:tasks.workunit.client.0.smithi086.stdout:559/600: open fd = 3 2024-04-21T08:50:26.561 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.561 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.561 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:26.561 INFO:tasks.workunit.client.0.smithi086.stdout:560/600: open fd = 3 2024-04-21T08:50:26.561 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.561 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:50:26.561 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:50:28.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:27 smithi088 ceph-mon[31259]: pgmap v365: 129 pgs: 129 active+clean; 1.9 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 KiB/s wr, 17 op/s 2024-04-21T08:50:28.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:27 smithi179 ceph-mon[31829]: pgmap v365: 129 pgs: 129 active+clean; 1.9 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 KiB/s wr, 17 op/s 2024-04-21T08:50:28.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:27 smithi086 ceph-mon[27939]: pgmap v365: 129 pgs: 129 active+clean; 1.9 MiB data, 434 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 41 KiB/s wr, 17 op/s 2024-04-21T08:50:30.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:29 smithi088 ceph-mon[31259]: pgmap v366: 129 pgs: 129 active+clean; 2.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-21T08:50:30.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:29 smithi179 ceph-mon[31829]: pgmap v366: 129 pgs: 129 active+clean; 2.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-21T08:50:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.313+0000 7fc838414640 1 -- 172.21.15.86:0/1959712901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc830107fc0 msgr2=0x7fc8301083a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.313+0000 7fc838414640 1 --2- 172.21.15.86:0/1959712901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc830107fc0 0x7fc8301083a0 secure :-1 s=READY pgs=2014 cs=0 l=1 rev1=1 crypto rx=0x7fc82c009930 tx=0x7fc82c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:50:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.313+0000 7fc838414640 1 -- 172.21.15.86:0/1959712901 shutdown_connections 2024-04-21T08:50:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.313+0000 7fc838414640 1 --2- 172.21.15.86:0/1959712901 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc83006b160 0x7fc83010edd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.313+0000 7fc838414640 1 --2- 172.21.15.86:0/1959712901 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc83006a7c0 0x7fc83006ac20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.313+0000 7fc838414640 1 --2- 172.21.15.86:0/1959712901 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc830107fc0 0x7fc8301083a0 unknown :-1 s=CLOSED pgs=2014 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.313+0000 7fc838414640 1 -- 172.21.15.86:0/1959712901 >> 172.21.15.86:0/1959712901 conn(0x7fc830077ba0 msgr2=0x7fc830077fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.313+0000 7fc838414640 1 -- 172.21.15.86:0/1959712901 shutdown_connections 2024-04-21T08:50:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.313+0000 7fc838414640 1 -- 172.21.15.86:0/1959712901 wait complete. 2024-04-21T08:50:30.315 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.314+0000 7fc838414640 1 Processor -- start 2024-04-21T08:50:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.314+0000 7fc838414640 1 -- start start 2024-04-21T08:50:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.314+0000 7fc838414640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc83006a7c0 0x7fc8301986a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.314+0000 7fc838414640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc83006b160 0x7fc830198be0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.314+0000 7fc838414640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc830107fc0 0x7fc83019cf60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.314+0000 7fc838414640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8301121a0 con 0x7fc83006b160 2024-04-21T08:50:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.314+0000 7fc838414640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc830112020 con 0x7fc83006a7c0 2024-04-21T08:50:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.314+0000 7fc838414640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc830112320 con 0x7fc830107fc0 2024-04-21T08:50:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.314+0000 7fc8359af640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc83006b160 0x7fc830198be0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.314+0000 7fc8359af640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc83006b160 0x7fc830198be0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33362/0 (socket says 172.21.15.86:33362) 2024-04-21T08:50:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.314+0000 7fc8359af640 1 -- 172.21.15.86:0/3657276944 learned_addr learned my addr 172.21.15.86:0/3657276944 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.314+0000 7fc8369b1640 1 --2- 172.21.15.86:0/3657276944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc830107fc0 0x7fc83019cf60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:30.316 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.314+0000 7fc8361b0640 1 --2- 172.21.15.86:0/3657276944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc83006a7c0 0x7fc8301986a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.314+0000 7fc8359af640 1 -- 172.21.15.86:0/3657276944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc830107fc0 msgr2=0x7fc83019cf60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.314+0000 7fc8359af640 1 --2- 172.21.15.86:0/3657276944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc830107fc0 0x7fc83019cf60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.314+0000 7fc8359af640 1 -- 172.21.15.86:0/3657276944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc83006a7c0 msgr2=0x7fc8301986a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.314+0000 7fc8359af640 1 --2- 172.21.15.86:0/3657276944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc83006a7c0 0x7fc8301986a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.314+0000 7fc8359af640 1 -- 172.21.15.86:0/3657276944 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc83019d690 con 0x7fc83006b160 2024-04-21T08:50:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.314+0000 7fc8361b0640 1 --2- 172.21.15.86:0/3657276944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc83006a7c0 0x7fc8301986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.315+0000 7fc8359af640 1 --2- 172.21.15.86:0/3657276944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc83006b160 0x7fc830198be0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7fc82000e9f0 tx=0x7fc82000edf0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.315+0000 7fc8277fe640 1 -- 172.21.15.86:0/3657276944 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82000cd80 con 0x7fc83006b160 2024-04-21T08:50:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.315+0000 7fc8277fe640 1 -- 172.21.15.86:0/3657276944 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc8200054d0 con 0x7fc83006b160 2024-04-21T08:50:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.315+0000 7fc838414640 1 -- 172.21.15.86:0/3657276944 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc83019d980 con 0x7fc83006b160 2024-04-21T08:50:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.315+0000 7fc8277fe640 1 -- 172.21.15.86:0/3657276944 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc820010690 con 0x7fc83006b160 2024-04-21T08:50:30.317 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.315+0000 7fc838414640 1 -- 172.21.15.86:0/3657276944 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc8301a5210 con 0x7fc83006b160 2024-04-21T08:50:30.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.317+0000 7fc8277fe640 1 -- 172.21.15.86:0/3657276944 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc820004e90 con 0x7fc83006b160 2024-04-21T08:50:30.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.318+0000 7fc8277fe640 1 --2- 172.21.15.86:0/3657276944 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc81c0a3dc0 0x7fc81c0a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:30.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.318+0000 7fc83b006740 1 Processor -- start 2024-04-21T08:50:30.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.318+0000 7fc8361b0640 1 --2- 172.21.15.86:0/3657276944 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc81c0a3dc0 0x7fc81c0a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:30.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.318+0000 7fc8277fe640 1 -- 172.21.15.86:0/3657276944 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7fc8200d1110 con 0x7fc83006b160 2024-04-21T08:50:30.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.318+0000 7fc83b006740 1 -- start start 2024-04-21T08:50:30.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.318+0000 7fc83b006740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fd0b8e8d0 0x563fd0b8d620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:30.324 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.318+0000 7fc83b006740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563fd0b8db60 0x563fd0b8dfe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.318+0000 7fc83b006740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fd0b9b4f0 0x563fd0b9d8e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.318+0000 7fc83b006740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563fd0af28d0 con 0x563fd0b8e8d0 2024-04-21T08:50:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.318+0000 7fc83b006740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x563fd0a8fe80 con 0x563fd0b8db60 2024-04-21T08:50:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.318+0000 7fc83b006740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563fd0aee890 con 0x563fd0b9b4f0 2024-04-21T08:50:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.318+0000 7fc8361b0640 1 --2- 172.21.15.86:0/3657276944 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc81c0a3dc0 0x7fc81c0a6280 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc82c036040 tx=0x7fc82c000bc0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.318+0000 7fc8361b0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fd0b8e8d0 0x563fd0b8d620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.318+0000 7fc8361b0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fd0b8e8d0 0x563fd0b8d620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33372/0 (socket says 172.21.15.86:33372) 2024-04-21T08:50:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.318+0000 7fc8361b0640 1 -- 172.21.15.86:0/4037712029 learned_addr learned my addr 172.21.15.86:0/4037712029 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.319+0000 7fc8359af640 1 --2- 172.21.15.86:0/4037712029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563fd0b8db60 0x563fd0b8dfe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.319+0000 7fc8369b1640 1 --2- 172.21.15.86:0/4037712029 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fd0b9b4f0 0x563fd0b9d8e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.319+0000 7fc8361b0640 1 -- 172.21.15.86:0/4037712029 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fd0b9b4f0 msgr2=0x563fd0b9d8e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.319+0000 7fc8361b0640 1 --2- 172.21.15.86:0/4037712029 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fd0b9b4f0 0x563fd0b9d8e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.319+0000 7fc8361b0640 1 -- 172.21.15.86:0/4037712029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563fd0b8db60 msgr2=0x563fd0b8dfe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:30.325 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.319+0000 7fc8361b0640 1 --2- 172.21.15.86:0/4037712029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563fd0b8db60 0x563fd0b8dfe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.319+0000 7fc8361b0640 1 -- 172.21.15.86:0/4037712029 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fd0ab1320 con 0x563fd0b8e8d0 2024-04-21T08:50:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.319+0000 7fc8359af640 1 --2- 172.21.15.86:0/4037712029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563fd0b8db60 0x563fd0b8dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:50:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.319+0000 7fc8361b0640 1 --2- 172.21.15.86:0/4037712029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fd0b8e8d0 0x563fd0b8d620 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7fc82c035000 tx=0x7fc82c03c000 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9047568635eb49c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.319+0000 7fc824bfe640 1 -- 172.21.15.86:0/4037712029 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c046040 con 0x563fd0b8e8d0 2024-04-21T08:50:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.319+0000 7fc824bfe640 1 -- 172.21.15.86:0/4037712029 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc82c04f070 con 0x563fd0b8e8d0 2024-04-21T08:50:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.319+0000 7fc824bfe640 1 -- 172.21.15.86:0/4037712029 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c0035f0 con 0x563fd0b8e8d0 2024-04-21T08:50:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.320+0000 7fc83b006740 1 -- 172.21.15.86:0/4037712029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fd0b8e8d0 msgr2=0x563fd0b8d620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.320+0000 7fc83b006740 1 --2- 172.21.15.86:0/4037712029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fd0b8e8d0 0x563fd0b8d620 secure :-1 s=READY pgs=2016 cs=0 l=1 rev1=1 crypto rx=0x7fc82c035000 tx=0x7fc82c03c000 comp rx=0 tx=0).stop 2024-04-21T08:50:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.320+0000 7fc83b006740 1 -- 172.21.15.86:0/4037712029 shutdown_connections 2024-04-21T08:50:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.320+0000 7fc83b006740 1 --2- 172.21.15.86:0/4037712029 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fd0b9b4f0 0x563fd0b9d8e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.320+0000 7fc83b006740 1 --2- 172.21.15.86:0/4037712029 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563fd0b8db60 0x563fd0b8dfe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:30.326 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.320+0000 7fc83b006740 1 --2- 172.21.15.86:0/4037712029 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fd0b8e8d0 0x563fd0b8d620 unknown :-1 s=CLOSED pgs=2016 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.320+0000 7fc83b006740 1 -- 172.21.15.86:0/4037712029 >> 172.21.15.86:0/4037712029 conn(0x563fd0b54c90 msgr2=0x563fd0b88850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.320+0000 7fc83b006740 1 -- 172.21.15.86:0/4037712029 shutdown_connections 2024-04-21T08:50:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.320+0000 7fc83b006740 1 -- 172.21.15.86:0/4037712029 wait complete. 2024-04-21T08:50:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.320+0000 7fc83b006740 1 -- start start 2024-04-21T08:50:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.320+0000 7fc83b006740 1 Processor -- start 2024-04-21T08:50:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.321+0000 7fc83b006740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fd0b8db60 0x563fd0b9ae00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.321+0000 7fc83b006740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fd0b8e8d0 0x563fd0b98200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.321+0000 7fc83b006740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563fd0b9b4f0 0x563fd0b98780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.321+0000 7fc83b006740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563fd0b9e1c0 con 0x563fd0b8e8d0 2024-04-21T08:50:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.321+0000 7fc83b006740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x563fd0a7aeb0 con 0x563fd0b9b4f0 2024-04-21T08:50:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.321+0000 7fc83b006740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563fd0b9e340 con 0x563fd0b8db60 2024-04-21T08:50:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.321+0000 7fc83b006740 10 client.0 _ll_register_callbacks cb 0x563fd0aa7b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:30.327 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.321+0000 7fc8359af640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fd0b8e8d0 0x563fd0b98200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.321+0000 7fc8359af640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fd0b8e8d0 0x563fd0b98200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33380/0 (socket says 172.21.15.86:33380) 2024-04-21T08:50:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.321+0000 7fc8359af640 1 -- 172.21.15.86:0/2725643183 learned_addr learned my addr 172.21.15.86:0/2725643183 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.321+0000 7fc8361b0640 1 --2- 172.21.15.86:0/2725643183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fd0b8db60 0x563fd0b9ae00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.321+0000 7fc8369b1640 1 --2- 172.21.15.86:0/2725643183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563fd0b9b4f0 0x563fd0b98780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.321+0000 7fc8359af640 1 -- 172.21.15.86:0/2725643183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fd0b8db60 msgr2=0x563fd0b9ae00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.321+0000 7fc8359af640 1 --2- 172.21.15.86:0/2725643183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fd0b8db60 0x563fd0b9ae00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.321+0000 7fc8359af640 1 -- 172.21.15.86:0/2725643183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563fd0b9b4f0 msgr2=0x563fd0b98780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.321+0000 7fc8359af640 1 --2- 172.21.15.86:0/2725643183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563fd0b9b4f0 0x563fd0b98780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.321+0000 7fc8359af640 1 -- 172.21.15.86:0/2725643183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fd0805210 con 0x563fd0b8e8d0 2024-04-21T08:50:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.321+0000 7fc8359af640 1 --2- 172.21.15.86:0/2725643183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fd0b8e8d0 0x563fd0b98200 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7fc820099140 tx=0x7fc820005240 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59a4b37d31767d54 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.321+0000 7fc819ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.321+0000 7fc819ffb640 1 -- 172.21.15.86:0/2725643183 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc8200b5020 con 0x563fd0b8e8d0 2024-04-21T08:50:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.322+0000 7fc819ffb640 1 -- 172.21.15.86:0/2725643183 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7fc820099350 con 0x563fd0b8e8d0 2024-04-21T08:50:30.328 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.322+0000 7fc83b006740 1 -- 172.21.15.86:0/2725643183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc82c04f070 con 0x563fd0b8e8d0 2024-04-21T08:50:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.322+0000 7fc819ffb640 1 -- 172.21.15.86:0/2725643183 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc8200b6250 con 0x563fd0b8e8d0 2024-04-21T08:50:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.322+0000 7fc83b006740 10 client.20280 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.322+0000 7fc83b006740 1 -- 172.21.15.86:0/2725643183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563fd08d1620 con 0x563fd0b8e8d0 2024-04-21T08:50:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.322+0000 7fc819ffb640 1 -- 172.21.15.86:0/2725643183 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc82c04f070 con 0x563fd0b8e8d0 2024-04-21T08:50:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.322+0000 7fc819ffb640 1 -- 172.21.15.86:0/2725643183 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc8200a54c0 con 0x563fd0b8e8d0 2024-04-21T08:50:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.322+0000 7fc83b006740 10 client.20280 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.322+0000 7fc83b006740 10 client.20280 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.322+0000 7fc83b006740 20 client.20280 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.322+0000 7fc83b006740 1 --2- 172.21.15.86:0/2725643183 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563fd0bab490 0x563fd0bad880 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.322+0000 7fc83b006740 4 client.20280 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.322+0000 7fc83b006740 1 -- 172.21.15.86:0/2725643183 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563fd0b06880 con 0x563fd0bab490 2024-04-21T08:50:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.322+0000 7fc8361b0640 1 --2- 172.21.15.86:0/2725643183 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563fd0bab490 0x563fd0bad880 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.323+0000 7fc819ffb640 10 client.20280 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:30.329 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.323+0000 7fc8361b0640 1 --2- 172.21.15.86:0/2725643183 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563fd0bab490 0x563fd0bad880 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=6d9950bba2a3015d server_cookie=7ba90b08bf8492b1 in_seq=0 out_seq=0 2024-04-21T08:50:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.323+0000 7fc819ffb640 1 -- 172.21.15.86:0/2725643183 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563fd0b06880 con 0x563fd0bab490 2024-04-21T08:50:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.323+0000 7fc819ffb640 10 client.20280 handle_command_reply: tid=0 2024-04-21T08:50:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.323+0000 7fc83b006740 1 client.20280 shutdown 2024-04-21T08:50:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.324+0000 7fc83b006740 1 -- 172.21.15.86:0/2725643183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fd0b8e8d0 msgr2=0x563fd0b98200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.324+0000 7fc83b006740 1 --2- 172.21.15.86:0/2725643183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fd0b8e8d0 0x563fd0b98200 secure :-1 s=READY pgs=2017 cs=0 l=1 rev1=1 crypto rx=0x7fc820099140 tx=0x7fc820005240 comp rx=0 tx=0).stop 2024-04-21T08:50:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.324+0000 7fc83b006740 1 -- 172.21.15.86:0/2725643183 shutdown_connections 2024-04-21T08:50:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.324+0000 7fc83b006740 1 --2- 172.21.15.86:0/2725643183 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563fd0bab490 0x563fd0bad880 crc :-1 s=READY pgs=335 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.324+0000 7fc83b006740 1 --2- 172.21.15.86:0/2725643183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563fd0b9b4f0 0x563fd0b98780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.324+0000 7fc83b006740 1 --2- 172.21.15.86:0/2725643183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fd0b8e8d0 0x563fd0b98200 unknown :-1 s=CLOSED pgs=2017 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.324+0000 7fc83b006740 1 --2- 172.21.15.86:0/2725643183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fd0b8db60 0x563fd0b9ae00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.324+0000 7fc819ffb640 0 client.20280 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.324+0000 7fc83b006740 1 -- 172.21.15.86:0/2725643183 >> 172.21.15.86:0/2725643183 conn(0x563fd0b54c90 msgr2=0x563fd0b8a5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:30.330 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.324+0000 7fc83b006740 1 -- 172.21.15.86:0/2725643183 shutdown_connections 2024-04-21T08:50:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.324+0000 7fc83b006740 1 -- 172.21.15.86:0/2725643183 wait complete. 2024-04-21T08:50:30.331 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.324+0000 7fc83b006740 20 client.20280 trim_cache size 0 max 16384 2024-04-21T08:50:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.339+0000 7fc83b006740 1 Processor -- start 2024-04-21T08:50:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.339+0000 7fc83b006740 1 -- start start 2024-04-21T08:50:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.339+0000 7fc83b006740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fd0b8db60 0x563fd0ba70c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.339+0000 7fc83b006740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fd0b8e8d0 0x563fd0b83760 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.339+0000 7fc83b006740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563fd0b9b4f0 0x563fd0bb30f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.339+0000 7fc83b006740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8301ad140 con 0x563fd0b8db60 2024-04-21T08:50:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.339+0000 7fc83b006740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc8301aec80 con 0x563fd0b9b4f0 2024-04-21T08:50:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.339+0000 7fc83b006740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc8301ad810 con 0x563fd0b8e8d0 2024-04-21T08:50:30.343 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.339+0000 7fc8361b0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fd0b8db60 0x563fd0ba70c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.339+0000 7fc8361b0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fd0b8db60 0x563fd0ba70c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33386/0 (socket says 172.21.15.86:33386) 2024-04-21T08:50:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.339+0000 7fc8361b0640 1 -- 172.21.15.86:0/456544183 learned_addr learned my addr 172.21.15.86:0/456544183 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.339+0000 7fc8361b0640 1 -- 172.21.15.86:0/456544183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fd0b8e8d0 msgr2=0x563fd0b83760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.339+0000 7fc8361b0640 1 --2- 172.21.15.86:0/456544183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fd0b8e8d0 0x563fd0b83760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.339+0000 7fc8361b0640 1 -- 172.21.15.86:0/456544183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563fd0b9b4f0 msgr2=0x563fd0bb30f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.339+0000 7fc8361b0640 1 --2- 172.21.15.86:0/456544183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563fd0b9b4f0 0x563fd0bb30f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.339+0000 7fc8361b0640 1 -- 172.21.15.86:0/456544183 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563fd0b83f30 con 0x563fd0b8db60 2024-04-21T08:50:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.339+0000 7fc8361b0640 1 --2- 172.21.15.86:0/456544183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fd0b8db60 0x563fd0ba70c0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7fc82c02c170 tx=0x7fc82c004b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2fc48965615ca62 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.339+0000 7fc81bfff640 1 -- 172.21.15.86:0/456544183 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c05b070 con 0x563fd0b8db60 2024-04-21T08:50:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.339+0000 7fc81bfff640 1 -- 172.21.15.86:0/456544183 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc82c04f070 con 0x563fd0b8db60 2024-04-21T08:50:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.339+0000 7fc81bfff640 1 -- 172.21.15.86:0/456544183 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc82c040480 con 0x563fd0b8db60 2024-04-21T08:50:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.340+0000 7fc83b006740 1 -- 172.21.15.86:0/456544183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fd0b8db60 msgr2=0x563fd0ba70c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:30.344 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.340+0000 7fc83b006740 1 --2- 172.21.15.86:0/456544183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fd0b8db60 0x563fd0ba70c0 secure :-1 s=READY pgs=2018 cs=0 l=1 rev1=1 crypto rx=0x7fc82c02c170 tx=0x7fc82c004b00 comp rx=0 tx=0).stop 2024-04-21T08:50:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.340+0000 7fc83b006740 1 -- 172.21.15.86:0/456544183 shutdown_connections 2024-04-21T08:50:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.340+0000 7fc83b006740 1 --2- 172.21.15.86:0/456544183 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563fd0b9b4f0 0x563fd0bb30f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.340+0000 7fc83b006740 1 --2- 172.21.15.86:0/456544183 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fd0b8e8d0 0x563fd0b83760 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.340+0000 7fc83b006740 1 --2- 172.21.15.86:0/456544183 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fd0b8db60 0x563fd0ba70c0 unknown :-1 s=CLOSED pgs=2018 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.340+0000 7fc83b006740 1 -- 172.21.15.86:0/456544183 >> 172.21.15.86:0/456544183 conn(0x563fd0b54c90 msgr2=0x563fd0b93490 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.340+0000 7fc83b006740 1 -- 172.21.15.86:0/456544183 shutdown_connections 2024-04-21T08:50:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.340+0000 7fc83b006740 1 -- 172.21.15.86:0/456544183 wait complete. 2024-04-21T08:50:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.340+0000 7fc83b006740 1 -- start start 2024-04-21T08:50:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.340+0000 7fc83b006740 1 Processor -- start 2024-04-21T08:50:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.341+0000 7fc83b006740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fd0b8db60 0x563fd0ba3780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.341+0000 7fc83b006740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563fd0b8e8d0 0x563fd0b9ebb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.341+0000 7fc83b006740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fd0b9b4f0 0x563fd0b9f0f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.341+0000 7fc83b006740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc8301ae5b0 con 0x563fd0b9b4f0 2024-04-21T08:50:30.345 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.341+0000 7fc83b006740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc8301adee0 con 0x563fd0b8e8d0 2024-04-21T08:50:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.341+0000 7fc83b006740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc8301ab140 con 0x563fd0b8db60 2024-04-21T08:50:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.341+0000 7fc83b006740 10 client.0 _ll_register_callbacks cb 0x7fc82000e940 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.341+0000 7fc8369b1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fd0b9b4f0 0x563fd0b9f0f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.341+0000 7fc8369b1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fd0b9b4f0 0x563fd0b9f0f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:33392/0 (socket says 172.21.15.86:33392) 2024-04-21T08:50:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.341+0000 7fc8369b1640 1 -- 172.21.15.86:0/1961745003 learned_addr learned my addr 172.21.15.86:0/1961745003 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.341+0000 7fc8359af640 1 --2- 172.21.15.86:0/1961745003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563fd0b8e8d0 0x563fd0b9ebb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.341+0000 7fc8361b0640 1 --2- 172.21.15.86:0/1961745003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fd0b8db60 0x563fd0ba3780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.341+0000 7fc8369b1640 1 -- 172.21.15.86:0/1961745003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fd0b8db60 msgr2=0x563fd0ba3780 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.341+0000 7fc8369b1640 1 --2- 172.21.15.86:0/1961745003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fd0b8db60 0x563fd0ba3780 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.341+0000 7fc8369b1640 1 -- 172.21.15.86:0/1961745003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563fd0b8e8d0 msgr2=0x563fd0b9ebb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.341+0000 7fc8369b1640 1 --2- 172.21.15.86:0/1961745003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563fd0b8e8d0 0x563fd0b9ebb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.341+0000 7fc8369b1640 1 -- 172.21.15.86:0/1961745003 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563fd0b9f730 con 0x563fd0b9b4f0 2024-04-21T08:50:30.346 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.341+0000 7fc8361b0640 1 --2- 172.21.15.86:0/1961745003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fd0b8db60 0x563fd0ba3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.341+0000 7fc8369b1640 1 --2- 172.21.15.86:0/1961745003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fd0b9b4f0 0x563fd0b9f0f0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7fc828009fd0 tx=0x7fc828004410 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e6dac34d2d309a31 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.341+0000 7fc819ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.341+0000 7fc819ffb640 1 -- 172.21.15.86:0/1961745003 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc828020060 con 0x563fd0b9b4f0 2024-04-21T08:50:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.341+0000 7fc819ffb640 1 -- 172.21.15.86:0/1961745003 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7fc828011a50 con 0x563fd0b9b4f0 2024-04-21T08:50:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.341+0000 7fc83b006740 1 -- 172.21.15.86:0/1961745003 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fc82c04f070 con 0x563fd0b9b4f0 2024-04-21T08:50:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.342+0000 7fc819ffb640 1 -- 172.21.15.86:0/1961745003 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fc828027050 con 0x563fd0b9b4f0 2024-04-21T08:50:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.342+0000 7fc83b006740 10 client.20286 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.342+0000 7fc83b006740 1 -- 172.21.15.86:0/1961745003 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563fd0c117e0 con 0x563fd0b9b4f0 2024-04-21T08:50:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.342+0000 7fc819ffb640 1 -- 172.21.15.86:0/1961745003 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc82c04f070 con 0x563fd0b9b4f0 2024-04-21T08:50:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.342+0000 7fc819ffb640 1 -- 172.21.15.86:0/1961745003 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fc82800f3c0 con 0x563fd0b9b4f0 2024-04-21T08:50:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.342+0000 7fc83b006740 10 client.20286 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.342+0000 7fc83b006740 10 client.20286 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.342+0000 7fc83b006740 20 client.20286 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:30.347 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.342+0000 7fc83b006740 1 --2- 172.21.15.86:0/1961745003 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563fd0c11fc0 0x563fd0c143e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:30.348 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.342+0000 7fc83b006740 4 client.20286 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:50:30.348 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.342+0000 7fc83b006740 1 -- 172.21.15.86:0/1961745003 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563fd0a923d0 con 0x563fd0c11fc0 2024-04-21T08:50:30.348 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.342+0000 7fc8361b0640 1 --2- 172.21.15.86:0/1961745003 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563fd0c11fc0 0x563fd0c143e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:30.348 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.343+0000 7fc8361b0640 1 --2- 172.21.15.86:0/1961745003 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563fd0c11fc0 0x563fd0c143e0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=5354cab87a405ee8 server_cookie=4fd22a329e596292 in_seq=0 out_seq=0 2024-04-21T08:50:30.348 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.343+0000 7fc819ffb640 10 client.20286 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.357+0000 7fc819ffb640 1 -- 172.21.15.86:0/1961745003 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563fd0a923d0 con 0x563fd0c11fc0 2024-04-21T08:50:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.357+0000 7fc819ffb640 10 client.20286 handle_command_reply: tid=0 2024-04-21T08:50:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.357+0000 7fc83b006740 1 client.20286 shutdown 2024-04-21T08:50:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.357+0000 7fc83b006740 1 -- 172.21.15.86:0/1961745003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fd0b9b4f0 msgr2=0x563fd0b9f0f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.357+0000 7fc83b006740 1 --2- 172.21.15.86:0/1961745003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fd0b9b4f0 0x563fd0b9f0f0 secure :-1 s=READY pgs=2019 cs=0 l=1 rev1=1 crypto rx=0x7fc828009fd0 tx=0x7fc828004410 comp rx=0 tx=0).stop 2024-04-21T08:50:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.357+0000 7fc83b006740 1 -- 172.21.15.86:0/1961745003 shutdown_connections 2024-04-21T08:50:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.357+0000 7fc83b006740 1 --2- 172.21.15.86:0/1961745003 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563fd0c11fc0 0x563fd0c143e0 crc :-1 s=READY pgs=336 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:30.357 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.357+0000 7fc83b006740 1 --2- 172.21.15.86:0/1961745003 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563fd0b9b4f0 0x563fd0b9f0f0 unknown :-1 s=CLOSED pgs=2019 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.357+0000 7fc83b006740 1 --2- 172.21.15.86:0/1961745003 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563fd0b8e8d0 0x563fd0b9ebb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.357+0000 7fc83b006740 1 --2- 172.21.15.86:0/1961745003 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563fd0b8db60 0x563fd0ba3780 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.357+0000 7fc819ffb640 0 client.20286 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.357+0000 7fc83b006740 1 -- 172.21.15.86:0/1961745003 >> 172.21.15.86:0/1961745003 conn(0x563fd0b54c90 msgr2=0x563fd0b96960 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.357+0000 7fc83b006740 1 -- 172.21.15.86:0/1961745003 shutdown_connections 2024-04-21T08:50:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.357+0000 7fc83b006740 1 -- 172.21.15.86:0/1961745003 wait complete. 2024-04-21T08:50:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.357+0000 7fc83b006740 20 client.20286 trim_cache size 0 max 16384 2024-04-21T08:50:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:50:30.358 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:50:30.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:50:30.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:50:30.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.358+0000 7fc838414640 1 -- 172.21.15.86:0/3657276944 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc81c0a3dc0 msgr2=0x7fc81c0a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:30.359 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.358+0000 7fc838414640 1 --2- 172.21.15.86:0/3657276944 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc81c0a3dc0 0x7fc81c0a6280 secure :-1 s=READY pgs=336 cs=0 l=1 rev1=1 crypto rx=0x7fc82c036040 tx=0x7fc82c000bc0 comp rx=0 tx=0).stop 2024-04-21T08:50:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.358+0000 7fc838414640 1 -- 172.21.15.86:0/3657276944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc83006b160 msgr2=0x7fc830198be0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.358+0000 7fc838414640 1 --2- 172.21.15.86:0/3657276944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc83006b160 0x7fc830198be0 secure :-1 s=READY pgs=2015 cs=0 l=1 rev1=1 crypto rx=0x7fc82000e9f0 tx=0x7fc82000edf0 comp rx=0 tx=0).stop 2024-04-21T08:50:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.358+0000 7fc838414640 1 -- 172.21.15.86:0/3657276944 shutdown_connections 2024-04-21T08:50:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.358+0000 7fc838414640 1 --2- 172.21.15.86:0/3657276944 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc81c0a3dc0 0x7fc81c0a6280 unknown :-1 s=CLOSED pgs=336 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.358+0000 7fc838414640 1 --2- 172.21.15.86:0/3657276944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc830107fc0 0x7fc83019cf60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.358+0000 7fc838414640 1 --2- 172.21.15.86:0/3657276944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc83006b160 0x7fc830198be0 unknown :-1 s=CLOSED pgs=2015 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.358+0000 7fc838414640 1 --2- 172.21.15.86:0/3657276944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc83006a7c0 0x7fc8301986a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.358+0000 7fc838414640 1 -- 172.21.15.86:0/3657276944 >> 172.21.15.86:0/3657276944 conn(0x7fc830077ba0 msgr2=0x7fc8300ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.358+0000 7fc838414640 1 -- 172.21.15.86:0/3657276944 shutdown_connections 2024-04-21T08:50:30.360 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:30.358+0000 7fc838414640 1 -- 172.21.15.86:0/3657276944 wait complete. 2024-04-21T08:50:30.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:29 smithi086 ceph-mon[27939]: pgmap v366: 129 pgs: 129 active+clean; 2.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 59 KiB/s wr, 23 op/s 2024-04-21T08:50:32.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:31 smithi088 ceph-mon[31259]: pgmap v367: 129 pgs: 129 active+clean; 2.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-21T08:50:32.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:31 smithi179 ceph-mon[31829]: pgmap v367: 129 pgs: 129 active+clean; 2.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-21T08:50:32.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:31 smithi086 ceph-mon[27939]: pgmap v367: 129 pgs: 129 active+clean; 2.1 MiB data, 439 MiB used, 1.0 TiB / 1.0 TiB avail; 37 KiB/s rd, 35 KiB/s wr, 12 op/s 2024-04-21T08:50:34.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:33 smithi088 ceph-mon[31259]: pgmap v368: 129 pgs: 129 active+clean; 2.3 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-21T08:50:34.248 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:33 smithi179 ceph-mon[31829]: pgmap v368: 129 pgs: 129 active+clean; 2.3 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-21T08:50:34.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:33 smithi086 ceph-mon[27939]: pgmap v368: 129 pgs: 129 active+clean; 2.3 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-21T08:50:36.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:35 smithi088 ceph-mon[31259]: pgmap v369: 129 pgs: 129 active+clean; 2.3 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-21T08:50:36.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:35 smithi179 ceph-mon[31829]: pgmap v369: 129 pgs: 129 active+clean; 2.3 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-21T08:50:36.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:35 smithi086 ceph-mon[27939]: pgmap v369: 129 pgs: 129 active+clean; 2.3 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-21T08:50:38.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:37 smithi088 ceph-mon[31259]: pgmap v370: 129 pgs: 129 active+clean; 2.3 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-21T08:50:38.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:37 smithi179 ceph-mon[31829]: pgmap v370: 129 pgs: 129 active+clean; 2.3 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-21T08:50:38.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:37 smithi086 ceph-mon[27939]: pgmap v370: 129 pgs: 129 active+clean; 2.3 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 42 KiB/s rd, 43 KiB/s wr, 15 op/s 2024-04-21T08:50:40.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:39 smithi088 ceph-mon[31259]: pgmap v371: 129 pgs: 129 active+clean; 2.5 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-21T08:50:40.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:39 smithi179 ceph-mon[31829]: pgmap v371: 129 pgs: 129 active+clean; 2.5 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-21T08:50:40.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:39 smithi086 ceph-mon[27939]: pgmap v371: 129 pgs: 129 active+clean; 2.5 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 59 KiB/s wr, 22 op/s 2024-04-21T08:50:42.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:41 smithi088 ceph-mon[31259]: pgmap v372: 129 pgs: 129 active+clean; 2.5 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:50:42.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:41 smithi179 ceph-mon[31829]: pgmap v372: 129 pgs: 129 active+clean; 2.5 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:50:42.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:41 smithi086 ceph-mon[27939]: pgmap v372: 129 pgs: 129 active+clean; 2.5 MiB data, 424 MiB used, 1.0 TiB / 1.0 TiB avail; 41 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:50:43.455 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.454+0000 7f1693352640 1 -- 172.21.15.86:0/630315772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f168c1021c0 msgr2=0x7f168c1025a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.454+0000 7f1693352640 1 --2- 172.21.15.86:0/630315772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f168c1021c0 0x7f168c1025a0 secure :-1 s=READY pgs=2020 cs=0 l=1 rev1=1 crypto rx=0x7f1688009900 tx=0x7f168802bc30 comp rx=0 tx=0).stop 2024-04-21T08:50:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.454+0000 7f1693352640 1 -- 172.21.15.86:0/630315772 shutdown_connections 2024-04-21T08:50:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.454+0000 7f1693352640 1 --2- 172.21.15.86:0/630315772 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f168c107290 0x7f168c10ee60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.454+0000 7f1693352640 1 --2- 172.21.15.86:0/630315772 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f168c102b70 0x7f168c106bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.454+0000 7f1693352640 1 --2- 172.21.15.86:0/630315772 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f168c1021c0 0x7f168c1025a0 unknown :-1 s=CLOSED pgs=2020 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.454+0000 7f1693352640 1 -- 172.21.15.86:0/630315772 >> 172.21.15.86:0/630315772 conn(0x7f168c0fda20 msgr2=0x7f168c0ffe40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.454+0000 7f1693352640 1 -- 172.21.15.86:0/630315772 shutdown_connections 2024-04-21T08:50:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.454+0000 7f1693352640 1 -- 172.21.15.86:0/630315772 wait complete. 2024-04-21T08:50:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.454+0000 7f1693352640 1 Processor -- start 2024-04-21T08:50:43.456 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.454+0000 7f1693352640 1 -- start start 2024-04-21T08:50:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.455+0000 7f1693352640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f168c1021c0 0x7f168c1986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.455+0000 7f1693352640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f168c107290 0x7f168c198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.455+0000 7f1693352640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f168c19cfb0 0x7f168c19d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.455+0000 7f1693352640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f168c1121f0 con 0x7f168c19cfb0 2024-04-21T08:50:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.455+0000 7f1693352640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f168c112070 con 0x7f168c107290 2024-04-21T08:50:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.455+0000 7f1693352640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f168c112370 con 0x7f168c1021c0 2024-04-21T08:50:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.455+0000 7f16918ef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f168c19cfb0 0x7f168c19d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.455+0000 7f16918ef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f168c19cfb0 0x7f168c19d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45696/0 (socket says 172.21.15.86:45696) 2024-04-21T08:50:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.455+0000 7f16918ef640 1 -- 172.21.15.86:0/1021043562 learned_addr learned my addr 172.21.15.86:0/1021043562 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.455+0000 7f16910ee640 1 --2- 172.21.15.86:0/1021043562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f168c1021c0 0x7f168c1986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.455+0000 7f16908ed640 1 --2- 172.21.15.86:0/1021043562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f168c107290 0x7f168c198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.455+0000 7f16918ef640 1 -- 172.21.15.86:0/1021043562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f168c1021c0 msgr2=0x7f168c1986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:43.457 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.455+0000 7f16918ef640 1 --2- 172.21.15.86:0/1021043562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f168c1021c0 0x7f168c1986f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.455+0000 7f16918ef640 1 -- 172.21.15.86:0/1021043562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f168c107290 msgr2=0x7f168c198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.455+0000 7f16918ef640 1 --2- 172.21.15.86:0/1021043562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f168c107290 0x7f168c198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.455+0000 7f16918ef640 1 -- 172.21.15.86:0/1021043562 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f168c19daa0 con 0x7f168c19cfb0 2024-04-21T08:50:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.455+0000 7f16908ed640 1 --2- 172.21.15.86:0/1021043562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f168c107290 0x7f168c198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.455+0000 7f16918ef640 1 --2- 172.21.15.86:0/1021043562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f168c19cfb0 0x7f168c19d460 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f168400eb10 tx=0x7f168400c2d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.455+0000 7f16827fc640 1 -- 172.21.15.86:0/1021043562 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1684004b20 con 0x7f168c19cfb0 2024-04-21T08:50:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.455+0000 7f16827fc640 1 -- 172.21.15.86:0/1021043562 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1684004cc0 con 0x7f168c19cfb0 2024-04-21T08:50:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.455+0000 7f16827fc640 1 -- 172.21.15.86:0/1021043562 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1684010670 con 0x7f168c19cfb0 2024-04-21T08:50:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.455+0000 7f1693352640 1 -- 172.21.15.86:0/1021043562 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f168c19dd90 con 0x7f168c19cfb0 2024-04-21T08:50:43.458 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.456+0000 7f1693352640 1 -- 172.21.15.86:0/1021043562 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f168c1a5530 con 0x7f168c19cfb0 2024-04-21T08:50:43.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.458+0000 7f16827fc640 1 -- 172.21.15.86:0/1021043562 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1684005020 con 0x7f168c19cfb0 2024-04-21T08:50:43.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.458+0000 7f1696025740 1 Processor -- start 2024-04-21T08:50:43.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.458+0000 7f1696025740 1 -- start start 2024-04-21T08:50:43.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.458+0000 7f16827fc640 1 --2- 172.21.15.86:0/1021043562 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f16780a3e40 0x7f16780a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:43.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.458+0000 7f1696025740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e811f48d0 0x563e811f3620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:43.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.459+0000 7f1696025740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e811f3b60 0x563e811f3fe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:43.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.459+0000 7f1696025740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e812014f0 0x563e812038e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:43.465 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.459+0000 7f1696025740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563e811588d0 con 0x563e811f48d0 2024-04-21T08:50:43.466 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.459+0000 7f1696025740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x563e810f5e80 con 0x563e811f3b60 2024-04-21T08:50:43.466 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.459+0000 7f1696025740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563e81154890 con 0x563e812014f0 2024-04-21T08:50:43.466 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.459+0000 7f16910ee640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e811f48d0 0x563e811f3620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:43.466 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.459+0000 7f16908ed640 1 --2- 172.21.15.86:0/1021043562 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f16780a3e40 0x7f16780a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:43.466 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.459+0000 7f16910ee640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e811f48d0 0x563e811f3620 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45706/0 (socket says 172.21.15.86:45706) 2024-04-21T08:50:43.466 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.459+0000 7f16827fc640 1 -- 172.21.15.86:0/1021043562 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f16840d1120 con 0x7f168c19cfb0 2024-04-21T08:50:43.466 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.459+0000 7f16910ee640 1 -- 172.21.15.86:0/1381626691 learned_addr learned my addr 172.21.15.86:0/1381626691 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:43.466 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.459+0000 7f16918ef640 1 --2- 172.21.15.86:0/1381626691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e812014f0 0x563e812038e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:43.466 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.459+0000 7f16908ed640 1 --2- 172.21.15.86:0/1381626691 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e811f3b60 0x563e811f3fe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:43.466 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.459+0000 7f16910ee640 1 -- 172.21.15.86:0/1381626691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e812014f0 msgr2=0x563e812038e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.459+0000 7f16910ee640 1 --2- 172.21.15.86:0/1381626691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e812014f0 0x563e812038e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.459+0000 7f16910ee640 1 -- 172.21.15.86:0/1381626691 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e811f3b60 msgr2=0x563e811f3fe0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.459+0000 7f16910ee640 1 --2- 172.21.15.86:0/1381626691 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e811f3b60 0x563e811f3fe0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.459+0000 7f16910ee640 1 -- 172.21.15.86:0/1381626691 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e81117320 con 0x563e811f48d0 2024-04-21T08:50:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.459+0000 7f16908ed640 1 --2- 172.21.15.86:0/1021043562 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f16780a3e40 0x7f16780a6300 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f168c1087a0 tx=0x7f167c015420 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.459+0000 7f16910ee640 1 --2- 172.21.15.86:0/1381626691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e811f48d0 0x563e811f3620 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f1688009900 tx=0x7f168802c610 comp rx=0 tx=0).ready entity=mon.0 client_cookie=11b362af695aa90a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.459+0000 7f16777fe640 1 -- 172.21.15.86:0/1381626691 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1688045020 con 0x563e811f48d0 2024-04-21T08:50:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.459+0000 7f16777fe640 1 -- 172.21.15.86:0/1381626691 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f168802c7e0 con 0x563e811f48d0 2024-04-21T08:50:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.459+0000 7f16777fe640 1 -- 172.21.15.86:0/1381626691 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16880054e0 con 0x563e811f48d0 2024-04-21T08:50:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.460+0000 7f1696025740 1 -- 172.21.15.86:0/1381626691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e811f48d0 msgr2=0x563e811f3620 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.460+0000 7f1696025740 1 --2- 172.21.15.86:0/1381626691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e811f48d0 0x563e811f3620 secure :-1 s=READY pgs=2022 cs=0 l=1 rev1=1 crypto rx=0x7f1688009900 tx=0x7f168802c610 comp rx=0 tx=0).stop 2024-04-21T08:50:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.460+0000 7f1696025740 1 -- 172.21.15.86:0/1381626691 shutdown_connections 2024-04-21T08:50:43.467 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.460+0000 7f1696025740 1 --2- 172.21.15.86:0/1381626691 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e812014f0 0x563e812038e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.460+0000 7f1696025740 1 --2- 172.21.15.86:0/1381626691 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e811f3b60 0x563e811f3fe0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.460+0000 7f1696025740 1 --2- 172.21.15.86:0/1381626691 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e811f48d0 0x563e811f3620 unknown :-1 s=CLOSED pgs=2022 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.460+0000 7f1696025740 1 -- 172.21.15.86:0/1381626691 >> 172.21.15.86:0/1381626691 conn(0x563e811bac90 msgr2=0x563e811ee850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.460+0000 7f1696025740 1 -- 172.21.15.86:0/1381626691 shutdown_connections 2024-04-21T08:50:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.460+0000 7f1696025740 1 -- 172.21.15.86:0/1381626691 wait complete. 2024-04-21T08:50:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.461+0000 7f1696025740 1 -- start start 2024-04-21T08:50:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.461+0000 7f1696025740 1 Processor -- start 2024-04-21T08:50:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.461+0000 7f1696025740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e811f3b60 0x563e81200e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.461+0000 7f1696025740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e811f48d0 0x563e811fe1d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.461+0000 7f1696025740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e812014f0 0x563e811fe730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.461+0000 7f1696025740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563e812041c0 con 0x563e811f48d0 2024-04-21T08:50:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.461+0000 7f1696025740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x563e810e0eb0 con 0x563e812014f0 2024-04-21T08:50:43.468 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.461+0000 7f1696025740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563e81204340 con 0x563e811f3b60 2024-04-21T08:50:43.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.461+0000 7f1696025740 10 client.0 _ll_register_callbacks cb 0x563e8110db40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:43.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.461+0000 7f16908ed640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e811f48d0 0x563e811fe1d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:43.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.461+0000 7f16908ed640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e811f48d0 0x563e811fe1d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45714/0 (socket says 172.21.15.86:45714) 2024-04-21T08:50:43.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.461+0000 7f16908ed640 1 -- 172.21.15.86:0/1063223397 learned_addr learned my addr 172.21.15.86:0/1063223397 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:43.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.461+0000 7f16910ee640 1 --2- 172.21.15.86:0/1063223397 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e811f3b60 0x563e81200e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:43.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.461+0000 7f16918ef640 1 --2- 172.21.15.86:0/1063223397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e812014f0 0x563e811fe730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:43.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.461+0000 7f16908ed640 1 -- 172.21.15.86:0/1063223397 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e811f3b60 msgr2=0x563e81200e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:43.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.461+0000 7f16908ed640 1 --2- 172.21.15.86:0/1063223397 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e811f3b60 0x563e81200e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:43.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.461+0000 7f16908ed640 1 -- 172.21.15.86:0/1063223397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e812014f0 msgr2=0x563e811fe730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:43.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.461+0000 7f16908ed640 1 --2- 172.21.15.86:0/1063223397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e812014f0 0x563e811fe730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:43.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.462+0000 7f16908ed640 1 -- 172.21.15.86:0/1063223397 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e80e6b210 con 0x563e811f48d0 2024-04-21T08:50:43.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.462+0000 7f16918ef640 1 --2- 172.21.15.86:0/1063223397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e812014f0 0x563e811fe730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:43.469 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.462+0000 7f16908ed640 1 --2- 172.21.15.86:0/1063223397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e811f48d0 0x563e811fe1d0 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f167c009880 tx=0x7f167c004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6661d719a95beb22 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.462+0000 7f1674ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.462+0000 7f1674ff9640 1 -- 172.21.15.86:0/1063223397 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f167c024070 con 0x563e811f48d0 2024-04-21T08:50:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.462+0000 7f1674ff9640 1 -- 172.21.15.86:0/1063223397 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f167c015070 con 0x563e811f48d0 2024-04-21T08:50:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.462+0000 7f1696025740 1 -- 172.21.15.86:0/1063223397 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f168802c7e0 con 0x563e811f48d0 2024-04-21T08:50:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.462+0000 7f1674ff9640 1 -- 172.21.15.86:0/1063223397 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f167c030050 con 0x563e811f48d0 2024-04-21T08:50:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.462+0000 7f1696025740 10 client.20298 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.462+0000 7f1696025740 1 -- 172.21.15.86:0/1063223397 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563e80f37620 con 0x563e811f48d0 2024-04-21T08:50:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.462+0000 7f1674ff9640 1 -- 172.21.15.86:0/1063223397 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f168802c7e0 con 0x563e811f48d0 2024-04-21T08:50:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.462+0000 7f1674ff9640 1 -- 172.21.15.86:0/1063223397 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f167c01f070 con 0x563e811f48d0 2024-04-21T08:50:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.462+0000 7f1696025740 10 client.20298 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.462+0000 7f1696025740 10 client.20298 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.463+0000 7f1696025740 20 client.20298 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:43.470 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.463+0000 7f1696025740 1 --2- 172.21.15.86:0/1063223397 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563e812114b0 0x563e812138c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.463+0000 7f1696025740 4 client.20298 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.463+0000 7f1696025740 1 -- 172.21.15.86:0/1063223397 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563e8116c880 con 0x563e812114b0 2024-04-21T08:50:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.463+0000 7f16910ee640 1 --2- 172.21.15.86:0/1063223397 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563e812114b0 0x563e812138c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.463+0000 7f1674ff9640 10 client.20298 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.463+0000 7f16910ee640 1 --2- 172.21.15.86:0/1063223397 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563e812114b0 0x563e812138c0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=aa2f29bb53ff7ccf server_cookie=b558b664d0fd2f67 in_seq=0 out_seq=0 2024-04-21T08:50:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.464+0000 7f1674ff9640 1 -- 172.21.15.86:0/1063223397 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563e8116c880 con 0x563e812114b0 2024-04-21T08:50:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.464+0000 7f1674ff9640 10 client.20298 handle_command_reply: tid=0 2024-04-21T08:50:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.464+0000 7f1696025740 1 client.20298 shutdown 2024-04-21T08:50:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.464+0000 7f1696025740 1 -- 172.21.15.86:0/1063223397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e811f48d0 msgr2=0x563e811fe1d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.464+0000 7f1696025740 1 --2- 172.21.15.86:0/1063223397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e811f48d0 0x563e811fe1d0 secure :-1 s=READY pgs=2023 cs=0 l=1 rev1=1 crypto rx=0x7f167c009880 tx=0x7f167c004730 comp rx=0 tx=0).stop 2024-04-21T08:50:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.464+0000 7f1696025740 1 -- 172.21.15.86:0/1063223397 shutdown_connections 2024-04-21T08:50:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.464+0000 7f1696025740 1 --2- 172.21.15.86:0/1063223397 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563e812114b0 0x563e812138c0 crc :-1 s=READY pgs=337 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.464+0000 7f1696025740 1 --2- 172.21.15.86:0/1063223397 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e812014f0 0x563e811fe730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.464+0000 7f1674ff9640 0 client.20298 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.464+0000 7f1696025740 1 --2- 172.21.15.86:0/1063223397 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e811f48d0 0x563e811fe1d0 unknown :-1 s=CLOSED pgs=2023 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:43.471 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.464+0000 7f1696025740 1 --2- 172.21.15.86:0/1063223397 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e811f3b60 0x563e81200e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.464+0000 7f1696025740 1 -- 172.21.15.86:0/1063223397 >> 172.21.15.86:0/1063223397 conn(0x563e811bac90 msgr2=0x563e811f05a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.465+0000 7f1696025740 1 -- 172.21.15.86:0/1063223397 shutdown_connections 2024-04-21T08:50:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.465+0000 7f1696025740 1 -- 172.21.15.86:0/1063223397 wait complete. 2024-04-21T08:50:43.472 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.465+0000 7f1696025740 20 client.20298 trim_cache size 0 max 16384 2024-04-21T08:50:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.479+0000 7f1696025740 1 Processor -- start 2024-04-21T08:50:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.479+0000 7f1696025740 1 -- start start 2024-04-21T08:50:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.479+0000 7f1696025740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e811f3b60 0x563e811ed750 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.479+0000 7f1696025740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e811f48d0 0x563e81200900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.479+0000 7f1696025740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e812014f0 0x563e81200e40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.479+0000 7f1696025740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f168c1ad110 con 0x563e811f3b60 2024-04-21T08:50:43.483 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.479+0000 7f1696025740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f168c1aec50 con 0x563e811f48d0 2024-04-21T08:50:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.479+0000 7f1696025740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f168c1ad7e0 con 0x563e812014f0 2024-04-21T08:50:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.480+0000 7f16910ee640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e811f3b60 0x563e811ed750 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.480+0000 7f16910ee640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e811f3b60 0x563e811ed750 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45728/0 (socket says 172.21.15.86:45728) 2024-04-21T08:50:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.480+0000 7f16910ee640 1 -- 172.21.15.86:0/3506010885 learned_addr learned my addr 172.21.15.86:0/3506010885 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.480+0000 7f16908ed640 1 --2- 172.21.15.86:0/3506010885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e811f48d0 0x563e81200900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.480+0000 7f16918ef640 1 --2- 172.21.15.86:0/3506010885 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e812014f0 0x563e81200e40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.480+0000 7f16910ee640 1 -- 172.21.15.86:0/3506010885 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e812014f0 msgr2=0x563e81200e40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.480+0000 7f16910ee640 1 --2- 172.21.15.86:0/3506010885 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e812014f0 0x563e81200e40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.480+0000 7f16910ee640 1 -- 172.21.15.86:0/3506010885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e811f48d0 msgr2=0x563e81200900 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.480+0000 7f16910ee640 1 --2- 172.21.15.86:0/3506010885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e811f48d0 0x563e81200900 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.480+0000 7f16910ee640 1 -- 172.21.15.86:0/3506010885 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563e81213f40 con 0x563e811f3b60 2024-04-21T08:50:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.480+0000 7f16918ef640 1 --2- 172.21.15.86:0/3506010885 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e812014f0 0x563e81200e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:43.484 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.480+0000 7f16908ed640 1 --2- 172.21.15.86:0/3506010885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e811f48d0 0x563e81200900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.480+0000 7f16910ee640 1 --2- 172.21.15.86:0/3506010885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e811f3b60 0x563e811ed750 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f1688005150 tx=0x7f1688005fd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=78a658357f3aa3f7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.480+0000 7f1676ffd640 1 -- 172.21.15.86:0/3506010885 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1688045040 con 0x563e811f3b60 2024-04-21T08:50:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.480+0000 7f1676ffd640 1 -- 172.21.15.86:0/3506010885 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f168802c7e0 con 0x563e811f3b60 2024-04-21T08:50:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.480+0000 7f1676ffd640 1 -- 172.21.15.86:0/3506010885 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f168804f490 con 0x563e811f3b60 2024-04-21T08:50:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.481+0000 7f1696025740 1 -- 172.21.15.86:0/3506010885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e811f3b60 msgr2=0x563e811ed750 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.481+0000 7f1696025740 1 --2- 172.21.15.86:0/3506010885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e811f3b60 0x563e811ed750 secure :-1 s=READY pgs=2024 cs=0 l=1 rev1=1 crypto rx=0x7f1688005150 tx=0x7f1688005fd0 comp rx=0 tx=0).stop 2024-04-21T08:50:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.481+0000 7f1696025740 1 -- 172.21.15.86:0/3506010885 shutdown_connections 2024-04-21T08:50:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.481+0000 7f1696025740 1 --2- 172.21.15.86:0/3506010885 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e812014f0 0x563e81200e40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.481+0000 7f1696025740 1 --2- 172.21.15.86:0/3506010885 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e811f48d0 0x563e81200900 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.481+0000 7f1696025740 1 --2- 172.21.15.86:0/3506010885 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e811f3b60 0x563e811ed750 unknown :-1 s=CLOSED pgs=2024 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.481+0000 7f1696025740 1 -- 172.21.15.86:0/3506010885 >> 172.21.15.86:0/3506010885 conn(0x563e811bac90 msgr2=0x563e811ed040 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.481+0000 7f1696025740 1 -- 172.21.15.86:0/3506010885 shutdown_connections 2024-04-21T08:50:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.481+0000 7f1696025740 1 -- 172.21.15.86:0/3506010885 wait complete. 2024-04-21T08:50:43.485 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.481+0000 7f1696025740 1 -- start start 2024-04-21T08:50:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.481+0000 7f1696025740 1 Processor -- start 2024-04-21T08:50:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.482+0000 7f1696025740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e811f3b60 0x563e811f6260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.482+0000 7f1696025740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e811f48d0 0x563e811f67a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.482+0000 7f1696025740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e812014f0 0x563e812085c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.482+0000 7f1696025740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f168c1ae580 con 0x563e812014f0 2024-04-21T08:50:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.482+0000 7f1696025740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f168c1adeb0 con 0x563e811f3b60 2024-04-21T08:50:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.482+0000 7f1696025740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f168c1ab110 con 0x563e811f48d0 2024-04-21T08:50:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.482+0000 7f1696025740 10 client.0 _ll_register_callbacks cb 0x7f167c009dc0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.482+0000 7f16918ef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e812014f0 0x563e812085c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.482+0000 7f16918ef640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e812014f0 0x563e812085c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45738/0 (socket says 172.21.15.86:45738) 2024-04-21T08:50:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.482+0000 7f16918ef640 1 -- 172.21.15.86:0/2245492140 learned_addr learned my addr 172.21.15.86:0/2245492140 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.482+0000 7f16908ed640 1 --2- 172.21.15.86:0/2245492140 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e811f48d0 0x563e811f67a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:43.486 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.482+0000 7f16910ee640 1 --2- 172.21.15.86:0/2245492140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e811f3b60 0x563e811f6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.482+0000 7f16918ef640 1 -- 172.21.15.86:0/2245492140 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e811f48d0 msgr2=0x563e811f67a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.482+0000 7f16918ef640 1 --2- 172.21.15.86:0/2245492140 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e811f48d0 0x563e811f67a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.482+0000 7f16918ef640 1 -- 172.21.15.86:0/2245492140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e811f3b60 msgr2=0x563e811f6260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.482+0000 7f16918ef640 1 --2- 172.21.15.86:0/2245492140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e811f3b60 0x563e811f6260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.482+0000 7f16918ef640 1 -- 172.21.15.86:0/2245492140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563e811f2e30 con 0x563e812014f0 2024-04-21T08:50:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.482+0000 7f16910ee640 1 --2- 172.21.15.86:0/2245492140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e811f3b60 0x563e811f6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.482+0000 7f16918ef640 1 --2- 172.21.15.86:0/2245492140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e812014f0 0x563e812085c0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f168409c9f0 tx=0x7f168409c8f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fa4a7a0275fa5f18 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.482+0000 7f1674ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.482+0000 7f1674ff9640 1 -- 172.21.15.86:0/2245492140 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f16840b6060 con 0x563e812014f0 2024-04-21T08:50:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.482+0000 7f1674ff9640 1 -- 172.21.15.86:0/2245492140 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f168409aff0 con 0x563e812014f0 2024-04-21T08:50:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.482+0000 7f1696025740 1 -- 172.21.15.86:0/2245492140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f168802c7e0 con 0x563e812014f0 2024-04-21T08:50:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.482+0000 7f1674ff9640 1 -- 172.21.15.86:0/2245492140 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f16840a10a0 con 0x563e812014f0 2024-04-21T08:50:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.482+0000 7f1696025740 10 client.20304 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:43.487 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.482+0000 7f1696025740 1 -- 172.21.15.86:0/2245492140 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563e81209ab0 con 0x563e812014f0 2024-04-21T08:50:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.483+0000 7f1674ff9640 1 -- 172.21.15.86:0/2245492140 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f168802c7e0 con 0x563e812014f0 2024-04-21T08:50:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.483+0000 7f1674ff9640 1 -- 172.21.15.86:0/2245492140 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f16840a54d0 con 0x563e812014f0 2024-04-21T08:50:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.483+0000 7f1696025740 10 client.20304 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.483+0000 7f1696025740 10 client.20304 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.483+0000 7f1696025740 20 client.20304 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.483+0000 7f1696025740 1 --2- 172.21.15.86:0/2245492140 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563e8127c390 0x563e8127e780 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.483+0000 7f1696025740 4 client.20304 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:50:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.483+0000 7f1696025740 1 -- 172.21.15.86:0/2245492140 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x563e810f83d0 con 0x563e8127c390 2024-04-21T08:50:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.483+0000 7f16910ee640 1 --2- 172.21.15.86:0/2245492140 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563e8127c390 0x563e8127e780 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.484+0000 7f1674ff9640 10 client.20304 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:43.488 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.484+0000 7f16910ee640 1 --2- 172.21.15.86:0/2245492140 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563e8127c390 0x563e8127e780 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=73784a5d6ed51637 server_cookie=869d2ef8587ad14f in_seq=0 out_seq=0 2024-04-21T08:50:43.497 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.497+0000 7f1674ff9640 1 -- 172.21.15.86:0/2245492140 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x563e810f83d0 con 0x563e8127c390 2024-04-21T08:50:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.497+0000 7f1674ff9640 10 client.20304 handle_command_reply: tid=0 2024-04-21T08:50:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.497+0000 7f1696025740 1 client.20304 shutdown 2024-04-21T08:50:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.497+0000 7f1696025740 1 -- 172.21.15.86:0/2245492140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e812014f0 msgr2=0x563e812085c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.497+0000 7f1696025740 1 --2- 172.21.15.86:0/2245492140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e812014f0 0x563e812085c0 secure :-1 s=READY pgs=2025 cs=0 l=1 rev1=1 crypto rx=0x7f168409c9f0 tx=0x7f168409c8f0 comp rx=0 tx=0).stop 2024-04-21T08:50:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.498+0000 7f1696025740 1 -- 172.21.15.86:0/2245492140 shutdown_connections 2024-04-21T08:50:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.498+0000 7f1696025740 1 --2- 172.21.15.86:0/2245492140 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563e8127c390 0x563e8127e780 crc :-1 s=READY pgs=338 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.498+0000 7f1696025740 1 --2- 172.21.15.86:0/2245492140 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563e812014f0 0x563e812085c0 unknown :-1 s=CLOSED pgs=2025 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.498+0000 7f1696025740 1 --2- 172.21.15.86:0/2245492140 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563e811f48d0 0x563e811f67a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.498+0000 7f1674ff9640 0 client.20304 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.498+0000 7f1696025740 1 --2- 172.21.15.86:0/2245492140 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563e811f3b60 0x563e811f6260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.498+0000 7f1696025740 1 -- 172.21.15.86:0/2245492140 >> 172.21.15.86:0/2245492140 conn(0x563e811bac90 msgr2=0x563e811ed8d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:43.498 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.498+0000 7f1696025740 1 -- 172.21.15.86:0/2245492140 shutdown_connections 2024-04-21T08:50:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.498+0000 7f1696025740 1 -- 172.21.15.86:0/2245492140 wait complete. 2024-04-21T08:50:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.498+0000 7f1696025740 20 client.20304 trim_cache size 0 max 16384 2024-04-21T08:50:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:50:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:50:43.499 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:50:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:50:43.500 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.498+0000 7f1693352640 1 -- 172.21.15.86:0/1021043562 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f16780a3e40 msgr2=0x7f16780a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.498+0000 7f1693352640 1 --2- 172.21.15.86:0/1021043562 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f16780a3e40 0x7f16780a6300 secure :-1 s=READY pgs=337 cs=0 l=1 rev1=1 crypto rx=0x7f168c1087a0 tx=0x7f167c015420 comp rx=0 tx=0).stop 2024-04-21T08:50:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.498+0000 7f1693352640 1 -- 172.21.15.86:0/1021043562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f168c19cfb0 msgr2=0x7f168c19d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.498+0000 7f1693352640 1 --2- 172.21.15.86:0/1021043562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f168c19cfb0 0x7f168c19d460 secure :-1 s=READY pgs=2021 cs=0 l=1 rev1=1 crypto rx=0x7f168400eb10 tx=0x7f168400c2d0 comp rx=0 tx=0).stop 2024-04-21T08:50:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.498+0000 7f1693352640 1 -- 172.21.15.86:0/1021043562 shutdown_connections 2024-04-21T08:50:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.498+0000 7f1693352640 1 --2- 172.21.15.86:0/1021043562 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f16780a3e40 0x7f16780a6300 unknown :-1 s=CLOSED pgs=337 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.498+0000 7f1693352640 1 --2- 172.21.15.86:0/1021043562 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f168c19cfb0 0x7f168c19d460 unknown :-1 s=CLOSED pgs=2021 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.498+0000 7f1693352640 1 --2- 172.21.15.86:0/1021043562 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f168c107290 0x7f168c198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.498+0000 7f1693352640 1 --2- 172.21.15.86:0/1021043562 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f168c1021c0 0x7f168c1986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.498+0000 7f1693352640 1 -- 172.21.15.86:0/1021043562 >> 172.21.15.86:0/1021043562 conn(0x7f168c0fda20 msgr2=0x7f168c0fee20 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.498+0000 7f1693352640 1 -- 172.21.15.86:0/1021043562 shutdown_connections 2024-04-21T08:50:43.501 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:43.499+0000 7f1693352640 1 -- 172.21.15.86:0/1021043562 wait complete. 2024-04-21T08:50:44.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:43 smithi088 ceph-mon[31259]: pgmap v373: 129 pgs: 129 active+clean; 2.7 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-04-21T08:50:44.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:43 smithi179 ceph-mon[31829]: pgmap v373: 129 pgs: 129 active+clean; 2.7 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-04-21T08:50:44.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:43 smithi086 ceph-mon[27939]: pgmap v373: 129 pgs: 129 active+clean; 2.7 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 59 KiB/s rd, 59 KiB/s wr, 21 op/s 2024-04-21T08:50:45.588 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:50:45.588 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:50:45.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.702+0000 7f8795952640 1 -- 172.21.15.86:0/2362541515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8790106160 msgr2=0x7f879010dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:45.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.702+0000 7f8795952640 1 --2- 172.21.15.86:0/2362541515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8790106160 0x7f879010dd30 secure :-1 s=READY pgs=2026 cs=0 l=1 rev1=1 crypto rx=0x7f8788009ca0 tx=0x7f878802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:50:45.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.702+0000 7f8795952640 1 -- 172.21.15.86:0/2362541515 shutdown_connections 2024-04-21T08:50:45.703 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.702+0000 7f8795952640 1 --2- 172.21.15.86:0/2362541515 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8790106160 0x7f879010dd30 unknown :-1 s=CLOSED pgs=2026 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:45.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.702+0000 7f8795952640 1 --2- 172.21.15.86:0/2362541515 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8790101a40 0x7f8790105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:45.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.703+0000 7f8795952640 1 --2- 172.21.15.86:0/2362541515 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8790101090 0x7f8790101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:45.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.703+0000 7f8795952640 1 -- 172.21.15.86:0/2362541515 >> 172.21.15.86:0/2362541515 conn(0x7f87900771d0 msgr2=0x7f87900775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:45.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.703+0000 7f8795952640 1 -- 172.21.15.86:0/2362541515 shutdown_connections 2024-04-21T08:50:45.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.703+0000 7f8795952640 1 -- 172.21.15.86:0/2362541515 wait complete. 2024-04-21T08:50:45.704 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.703+0000 7f8795952640 1 Processor -- start 2024-04-21T08:50:45.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.703+0000 7f8795952640 1 -- start start 2024-04-21T08:50:45.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.704+0000 7f8795952640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8790101090 0x7f879019e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:45.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.704+0000 7f8795952640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8790106160 0x7f879019e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:45.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.704+0000 7f8795952640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87901985e0 0x7f8790198a90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:45.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.704+0000 7f8795952640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f87901110d0 con 0x7f87901985e0 2024-04-21T08:50:45.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.704+0000 7f8795952640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8790110f50 con 0x7f8790101090 2024-04-21T08:50:45.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.704+0000 7f8795952640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8790111250 con 0x7f8790106160 2024-04-21T08:50:45.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.704+0000 7f878f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87901985e0 0x7f8790198a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:45.706 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.704+0000 7f878f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87901985e0 0x7f8790198a90 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45756/0 (socket says 172.21.15.86:45756) 2024-04-21T08:50:45.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.704+0000 7f878f7fe640 1 -- 172.21.15.86:0/2377446924 learned_addr learned my addr 172.21.15.86:0/2377446924 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:45.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.704+0000 7f878effd640 1 --2- 172.21.15.86:0/2377446924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8790101090 0x7f879019e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:45.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.704+0000 7f878e7fc640 1 --2- 172.21.15.86:0/2377446924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8790106160 0x7f879019e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:45.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.704+0000 7f878f7fe640 1 -- 172.21.15.86:0/2377446924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8790106160 msgr2=0x7f879019e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:45.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.704+0000 7f878f7fe640 1 --2- 172.21.15.86:0/2377446924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8790106160 0x7f879019e920 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:45.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.704+0000 7f878f7fe640 1 -- 172.21.15.86:0/2377446924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8790101090 msgr2=0x7f879019e3e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:45.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.704+0000 7f878f7fe640 1 --2- 172.21.15.86:0/2377446924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8790101090 0x7f879019e3e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:45.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.704+0000 7f878f7fe640 1 -- 172.21.15.86:0/2377446924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8790199100 con 0x7f87901985e0 2024-04-21T08:50:45.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.704+0000 7f878f7fe640 1 --2- 172.21.15.86:0/2377446924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87901985e0 0x7f8790198a90 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f8788005e30 tx=0x7f8788032420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:45.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.704+0000 7f8794950640 1 -- 172.21.15.86:0/2377446924 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8788041020 con 0x7f87901985e0 2024-04-21T08:50:45.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.704+0000 7f8794950640 1 -- 172.21.15.86:0/2377446924 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8788032970 con 0x7f87901985e0 2024-04-21T08:50:45.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.705+0000 7f8794950640 1 -- 172.21.15.86:0/2377446924 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8788039040 con 0x7f87901985e0 2024-04-21T08:50:45.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.705+0000 7f8795952640 1 -- 172.21.15.86:0/2377446924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8790199390 con 0x7f87901985e0 2024-04-21T08:50:45.707 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.705+0000 7f8795952640 1 -- 172.21.15.86:0/2377446924 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8790102c90 con 0x7f87901985e0 2024-04-21T08:50:45.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.707+0000 7f8794950640 1 -- 172.21.15.86:0/2377446924 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8788044020 con 0x7f87901985e0 2024-04-21T08:50:45.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.707+0000 7f8798621740 1 Processor -- start 2024-04-21T08:50:45.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.707+0000 7f8798621740 1 -- start start 2024-04-21T08:50:45.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.707+0000 7f8798621740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55728aed6580 0x55728aed6960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:45.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.707+0000 7f8798621740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55728aed6ea0 0x55728aedf540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.707+0000 7f8798621740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55728aedfa80 0x55728aee1e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.707+0000 7f8798621740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55728ae3b1e0 con 0x55728aed6ea0 2024-04-21T08:50:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.707+0000 7f8798621740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55728add8390 con 0x55728aed6580 2024-04-21T08:50:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.707+0000 7f8798621740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55728ae371a0 con 0x55728aedfa80 2024-04-21T08:50:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.708+0000 7f8794950640 1 --2- 172.21.15.86:0/2377446924 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f87780a3d20 0x7f87780a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.708+0000 7f878effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55728aed6ea0 0x55728aedf540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.708+0000 7f878effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55728aed6ea0 0x55728aedf540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45770/0 (socket says 172.21.15.86:45770) 2024-04-21T08:50:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.708+0000 7f878effd640 1 -- 172.21.15.86:0/2029740764 learned_addr learned my addr 172.21.15.86:0/2029740764 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.708+0000 7f878e7fc640 1 --2- 172.21.15.86:0/2029740764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55728aed6580 0x55728aed6960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.708+0000 7f878f7fe640 1 --2- 172.21.15.86:0/2377446924 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f87780a3d20 0x7f87780a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.708+0000 7f878e7fc640 1 --2- 172.21.15.86:0/2029740764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55728aedfa80 0x55728aee1e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.708+0000 7f8794950640 1 -- 172.21.15.86:0/2377446924 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f87880f2890 con 0x7f87901985e0 2024-04-21T08:50:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.708+0000 7f878effd640 1 -- 172.21.15.86:0/2029740764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55728aedfa80 msgr2=0x55728aee1e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:45.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.708+0000 7f878effd640 1 --2- 172.21.15.86:0/2029740764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55728aedfa80 0x55728aee1e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:45.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.708+0000 7f878effd640 1 -- 172.21.15.86:0/2029740764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55728aed6580 msgr2=0x55728aed6960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:45.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.708+0000 7f878effd640 1 --2- 172.21.15.86:0/2029740764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55728aed6580 0x55728aed6960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:45.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.708+0000 7f878effd640 1 -- 172.21.15.86:0/2029740764 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55728ac177d0 con 0x55728aed6ea0 2024-04-21T08:50:45.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.708+0000 7f878effd640 1 --2- 172.21.15.86:0/2029740764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55728aed6ea0 0x55728aedf540 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f878000ac10 tx=0x7f878000f790 comp rx=0 tx=0).ready entity=mon.0 client_cookie=86368009af483475 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:45.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.708+0000 7f878f7fe640 1 --2- 172.21.15.86:0/2377446924 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f87780a3d20 0x7f87780a61e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f87880efdd0 tx=0x7f878802e040 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:45.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.708+0000 7f8775ffb640 1 -- 172.21.15.86:0/2029740764 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878001d070 con 0x55728aed6ea0 2024-04-21T08:50:45.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.708+0000 7f8775ffb640 1 -- 172.21.15.86:0/2029740764 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8780018af0 con 0x55728aed6ea0 2024-04-21T08:50:45.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.708+0000 7f8775ffb640 1 -- 172.21.15.86:0/2029740764 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8780010a70 con 0x55728aed6ea0 2024-04-21T08:50:45.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.709+0000 7f8798621740 1 -- 172.21.15.86:0/2029740764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55728aed6ea0 msgr2=0x55728aedf540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:45.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.709+0000 7f8798621740 1 --2- 172.21.15.86:0/2029740764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55728aed6ea0 0x55728aedf540 secure :-1 s=READY pgs=2028 cs=0 l=1 rev1=1 crypto rx=0x7f878000ac10 tx=0x7f878000f790 comp rx=0 tx=0).stop 2024-04-21T08:50:45.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.709+0000 7f8798621740 1 -- 172.21.15.86:0/2029740764 shutdown_connections 2024-04-21T08:50:45.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.709+0000 7f8798621740 1 --2- 172.21.15.86:0/2029740764 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55728aedfa80 0x55728aee1e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:45.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.709+0000 7f8798621740 1 --2- 172.21.15.86:0/2029740764 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55728aed6ea0 0x55728aedf540 unknown :-1 s=CLOSED pgs=2028 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:45.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.709+0000 7f8798621740 1 --2- 172.21.15.86:0/2029740764 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55728aed6580 0x55728aed6960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:45.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.709+0000 7f8798621740 1 -- 172.21.15.86:0/2029740764 >> 172.21.15.86:0/2029740764 conn(0x55728aed0cb0 msgr2=0x55728aed1090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:45.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.709+0000 7f8798621740 1 -- 172.21.15.86:0/2029740764 shutdown_connections 2024-04-21T08:50:45.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.709+0000 7f8798621740 1 -- 172.21.15.86:0/2029740764 wait complete. 2024-04-21T08:50:45.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.710+0000 7f8798621740 1 -- start start 2024-04-21T08:50:45.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.710+0000 7f8798621740 1 Processor -- start 2024-04-21T08:50:45.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.710+0000 7f8798621740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55728aed6580 0x55728aedcaf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:45.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.710+0000 7f8798621740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55728aed6ea0 0x55728aedd030 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:45.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.710+0000 7f8798621740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55728aedfa80 0x55728aedd570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:45.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.710+0000 7f8798621740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55728aee2870 con 0x55728aedfa80 2024-04-21T08:50:45.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.710+0000 7f8798621740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55728adf42f0 con 0x55728aed6580 2024-04-21T08:50:45.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.710+0000 7f8798621740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55728aee29f0 con 0x55728aed6ea0 2024-04-21T08:50:45.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.710+0000 7f8798621740 10 client.0 _ll_register_callbacks cb 0x55728ae01580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:45.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.710+0000 7f878e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55728aedfa80 0x55728aedd570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:45.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.710+0000 7f878e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55728aedfa80 0x55728aedd570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45774/0 (socket says 172.21.15.86:45774) 2024-04-21T08:50:45.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.710+0000 7f878e7fc640 1 -- 172.21.15.86:0/904049744 learned_addr learned my addr 172.21.15.86:0/904049744 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:45.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.710+0000 7f878effd640 1 --2- 172.21.15.86:0/904049744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55728aed6ea0 0x55728aedd030 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:45.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.710+0000 7f878e7fc640 1 --2- 172.21.15.86:0/904049744 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55728aed6580 0x55728aedcaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:45.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.711+0000 7f878e7fc640 1 -- 172.21.15.86:0/904049744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55728aed6ea0 msgr2=0x55728aedd030 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:45.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.711+0000 7f878e7fc640 1 --2- 172.21.15.86:0/904049744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55728aed6ea0 0x55728aedd030 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:45.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.711+0000 7f878e7fc640 1 -- 172.21.15.86:0/904049744 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55728aed6580 msgr2=0x55728aedcaf0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:45.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.711+0000 7f878e7fc640 1 --2- 172.21.15.86:0/904049744 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55728aed6580 0x55728aedcaf0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:45.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.711+0000 7f878e7fc640 1 -- 172.21.15.86:0/904049744 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55728ab4d3f0 con 0x55728aedfa80 2024-04-21T08:50:45.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.711+0000 7f878e7fc640 1 --2- 172.21.15.86:0/904049744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55728aedfa80 0x55728aedd570 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f8784012ec0 tx=0x7f8784004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=6e9dffc5d38ffaf2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:45.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.711+0000 7f876effd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:45.719 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.711+0000 7f876effd640 1 -- 172.21.15.86:0/904049744 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878400dde0 con 0x55728aedfa80 2024-04-21T08:50:45.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.711+0000 7f876effd640 1 -- 172.21.15.86:0/904049744 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f8784004020 con 0x55728aedfa80 2024-04-21T08:50:45.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.711+0000 7f8798621740 1 -- 172.21.15.86:0/904049744 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8780018af0 con 0x55728aedfa80 2024-04-21T08:50:45.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.711+0000 7f876effd640 1 -- 172.21.15.86:0/904049744 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8784028050 con 0x55728aedfa80 2024-04-21T08:50:45.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.711+0000 7f8798621740 10 client.20316 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:45.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.711+0000 7f8798621740 1 -- 172.21.15.86:0/904049744 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55728ab0d0f0 con 0x55728aedfa80 2024-04-21T08:50:45.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.711+0000 7f876effd640 1 -- 172.21.15.86:0/904049744 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8780018af0 con 0x55728aedfa80 2024-04-21T08:50:45.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.712+0000 7f876effd640 1 -- 172.21.15.86:0/904049744 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8784003c60 con 0x55728aedfa80 2024-04-21T08:50:45.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.712+0000 7f8798621740 10 client.20316 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:45.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.712+0000 7f8798621740 10 client.20316 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:45.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.712+0000 7f8798621740 20 client.20316 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:45.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.712+0000 7f8798621740 1 --2- 172.21.15.86:0/904049744 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55728aeefa70 0x55728aef1ec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:45.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.712+0000 7f8798621740 4 client.20316 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:45.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.712+0000 7f8798621740 1 -- 172.21.15.86:0/904049744 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55728ae48c00 con 0x55728aeefa70 2024-04-21T08:50:45.720 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.712+0000 7f878effd640 1 --2- 172.21.15.86:0/904049744 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55728aeefa70 0x55728aef1ec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.712+0000 7f876effd640 10 client.20316 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.712+0000 7f878effd640 1 --2- 172.21.15.86:0/904049744 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55728aeefa70 0x55728aef1ec0 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8d50381ae5e8654d server_cookie=c24d5ae73fe3d391 in_seq=0 out_seq=0 2024-04-21T08:50:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.713+0000 7f876effd640 1 -- 172.21.15.86:0/904049744 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55728ae48c00 con 0x55728aeefa70 2024-04-21T08:50:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.713+0000 7f876effd640 10 client.20316 handle_command_reply: tid=0 2024-04-21T08:50:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.713+0000 7f8798621740 1 client.20316 shutdown 2024-04-21T08:50:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.713+0000 7f8798621740 1 -- 172.21.15.86:0/904049744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55728aedfa80 msgr2=0x55728aedd570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.713+0000 7f8798621740 1 --2- 172.21.15.86:0/904049744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55728aedfa80 0x55728aedd570 secure :-1 s=READY pgs=2029 cs=0 l=1 rev1=1 crypto rx=0x7f8784012ec0 tx=0x7f8784004730 comp rx=0 tx=0).stop 2024-04-21T08:50:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.713+0000 7f8798621740 1 -- 172.21.15.86:0/904049744 shutdown_connections 2024-04-21T08:50:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.713+0000 7f8798621740 1 --2- 172.21.15.86:0/904049744 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55728aeefa70 0x55728aef1ec0 crc :-1 s=READY pgs=339 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.713+0000 7f8798621740 1 --2- 172.21.15.86:0/904049744 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55728aedfa80 0x55728aedd570 unknown :-1 s=CLOSED pgs=2029 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.713+0000 7f8798621740 1 --2- 172.21.15.86:0/904049744 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55728aed6ea0 0x55728aedd030 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.713+0000 7f876effd640 0 client.20316 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:45.721 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.713+0000 7f8798621740 1 --2- 172.21.15.86:0/904049744 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55728aed6580 0x55728aedcaf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:45.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.713+0000 7f8798621740 1 -- 172.21.15.86:0/904049744 >> 172.21.15.86:0/904049744 conn(0x55728aed0cb0 msgr2=0x55728aed2a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:45.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.714+0000 7f8798621740 1 -- 172.21.15.86:0/904049744 shutdown_connections 2024-04-21T08:50:45.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.714+0000 7f8798621740 1 -- 172.21.15.86:0/904049744 wait complete. 2024-04-21T08:50:45.722 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.714+0000 7f8798621740 20 client.20316 trim_cache size 0 max 16384 2024-04-21T08:50:45.737 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.731+0000 7f8798621740 1 Processor -- start 2024-04-21T08:50:45.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.731+0000 7f8798621740 1 -- start start 2024-04-21T08:50:45.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.731+0000 7f8798621740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55728aed6580 0x55728aedf220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:45.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.731+0000 7f8798621740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55728aed6ea0 0x55728aeeb730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:45.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.731+0000 7f8798621740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55728aedfa80 0x55728aedbc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:45.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.731+0000 7f8798621740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f87901b1670 con 0x55728aed6ea0 2024-04-21T08:50:45.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.731+0000 7f8798621740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f87901b31b0 con 0x55728aed6580 2024-04-21T08:50:45.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.731+0000 7f8798621740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f87901b1d40 con 0x55728aedfa80 2024-04-21T08:50:45.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.731+0000 7f878effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55728aed6ea0 0x55728aeeb730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:45.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.731+0000 7f878effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55728aed6ea0 0x55728aeeb730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45784/0 (socket says 172.21.15.86:45784) 2024-04-21T08:50:45.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.731+0000 7f878effd640 1 -- 172.21.15.86:0/3926977390 learned_addr learned my addr 172.21.15.86:0/3926977390 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:45.738 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.731+0000 7f878e7fc640 1 --2- 172.21.15.86:0/3926977390 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55728aed6580 0x55728aedf220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:45.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.731+0000 7f878e7fc640 1 --2- 172.21.15.86:0/3926977390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55728aedfa80 0x55728aedbc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:45.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.732+0000 7f878effd640 1 -- 172.21.15.86:0/3926977390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55728aedfa80 msgr2=0x55728aedbc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:45.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.732+0000 7f878effd640 1 --2- 172.21.15.86:0/3926977390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55728aedfa80 0x55728aedbc50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:45.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.732+0000 7f878effd640 1 -- 172.21.15.86:0/3926977390 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55728aed6580 msgr2=0x55728aedf220 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:45.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.732+0000 7f878effd640 1 --2- 172.21.15.86:0/3926977390 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55728aed6580 0x55728aedf220 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:45.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.732+0000 7f878effd640 1 -- 172.21.15.86:0/3926977390 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55728aed6cd0 con 0x55728aed6ea0 2024-04-21T08:50:45.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.732+0000 7f878e7fc640 1 --2- 172.21.15.86:0/3926977390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55728aedfa80 0x55728aedbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:45.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.732+0000 7f878effd640 1 --2- 172.21.15.86:0/3926977390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55728aed6ea0 0x55728aeeb730 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x55728aed6050 tx=0x7f878000ff40 comp rx=0 tx=0).ready entity=mon.0 client_cookie=59c80301cd5c4745 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:45.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.732+0000 7f87757fa640 1 -- 172.21.15.86:0/3926977390 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8780032070 con 0x55728aed6ea0 2024-04-21T08:50:45.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.732+0000 7f87757fa640 1 -- 172.21.15.86:0/3926977390 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f878001e040 con 0x55728aed6ea0 2024-04-21T08:50:45.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.732+0000 7f87757fa640 1 -- 172.21.15.86:0/3926977390 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f878001d040 con 0x55728aed6ea0 2024-04-21T08:50:45.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.732+0000 7f8798621740 1 -- 172.21.15.86:0/3926977390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55728aed6ea0 msgr2=0x55728aeeb730 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:45.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.732+0000 7f8798621740 1 --2- 172.21.15.86:0/3926977390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55728aed6ea0 0x55728aeeb730 secure :-1 s=READY pgs=2030 cs=0 l=1 rev1=1 crypto rx=0x55728aed6050 tx=0x7f878000ff40 comp rx=0 tx=0).stop 2024-04-21T08:50:45.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.732+0000 7f8798621740 1 -- 172.21.15.86:0/3926977390 shutdown_connections 2024-04-21T08:50:45.739 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.732+0000 7f8798621740 1 --2- 172.21.15.86:0/3926977390 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55728aedfa80 0x55728aedbc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:45.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.732+0000 7f8798621740 1 --2- 172.21.15.86:0/3926977390 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55728aed6ea0 0x55728aeeb730 unknown :-1 s=CLOSED pgs=2030 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:45.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.732+0000 7f8798621740 1 --2- 172.21.15.86:0/3926977390 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55728aed6580 0x55728aedf220 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:45.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.732+0000 7f8798621740 1 -- 172.21.15.86:0/3926977390 >> 172.21.15.86:0/3926977390 conn(0x55728aed0cb0 msgr2=0x55728aed20e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:45.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.733+0000 7f8798621740 1 -- 172.21.15.86:0/3926977390 shutdown_connections 2024-04-21T08:50:45.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.733+0000 7f8798621740 1 -- 172.21.15.86:0/3926977390 wait complete. 2024-04-21T08:50:45.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.733+0000 7f8798621740 1 -- start start 2024-04-21T08:50:45.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.733+0000 7f8798621740 1 Processor -- start 2024-04-21T08:50:45.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.733+0000 7f8798621740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55728aed6580 0x55728aee8020 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:45.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.733+0000 7f8798621740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55728aed6ea0 0x55728aee33c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:45.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.733+0000 7f8798621740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55728aedfa80 0x55728aee3900 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:45.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.733+0000 7f8798621740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f87901b2ae0 con 0x55728aedfa80 2024-04-21T08:50:45.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.733+0000 7f8798621740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f87901b2410 con 0x55728aed6ea0 2024-04-21T08:50:45.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.733+0000 7f8798621740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f87901af670 con 0x55728aed6580 2024-04-21T08:50:45.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.733+0000 7f8798621740 10 client.0 _ll_register_callbacks cb 0x7f8784009d00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:45.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.733+0000 7f878e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55728aedfa80 0x55728aee3900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:45.740 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.733+0000 7f878e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55728aedfa80 0x55728aee3900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45798/0 (socket says 172.21.15.86:45798) 2024-04-21T08:50:45.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.733+0000 7f878e7fc640 1 -- 172.21.15.86:0/1480648801 learned_addr learned my addr 172.21.15.86:0/1480648801 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:45.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.733+0000 7f878effd640 1 --2- 172.21.15.86:0/1480648801 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55728aed6ea0 0x55728aee33c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:45.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.734+0000 7f878e7fc640 1 --2- 172.21.15.86:0/1480648801 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55728aed6580 0x55728aee8020 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:45.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.734+0000 7f878e7fc640 1 -- 172.21.15.86:0/1480648801 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55728aed6580 msgr2=0x55728aee8020 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:45.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.734+0000 7f878e7fc640 1 --2- 172.21.15.86:0/1480648801 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55728aed6580 0x55728aee8020 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:45.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.734+0000 7f878e7fc640 1 -- 172.21.15.86:0/1480648801 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55728aed6ea0 msgr2=0x55728aee33c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:45.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.734+0000 7f878e7fc640 1 --2- 172.21.15.86:0/1480648801 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55728aed6ea0 0x55728aee33c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:45.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.734+0000 7f878e7fc640 1 -- 172.21.15.86:0/1480648801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55728aee4100 con 0x55728aedfa80 2024-04-21T08:50:45.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.734+0000 7f878e7fc640 1 --2- 172.21.15.86:0/1480648801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55728aedfa80 0x55728aee3900 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f8784012ec0 tx=0x7f8784016040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=26c5919f4ea12007 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:45.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.734+0000 7f876effd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:45.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.734+0000 7f876effd640 1 -- 172.21.15.86:0/1480648801 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8784034060 con 0x55728aedfa80 2024-04-21T08:50:45.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.734+0000 7f876effd640 1 -- 172.21.15.86:0/1480648801 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f8784004000 con 0x55728aedfa80 2024-04-21T08:50:45.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.734+0000 7f8798621740 1 -- 172.21.15.86:0/1480648801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f878001e040 con 0x55728aedfa80 2024-04-21T08:50:45.741 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.734+0000 7f876effd640 1 -- 172.21.15.86:0/1480648801 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f878403b050 con 0x55728aedfa80 2024-04-21T08:50:45.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.734+0000 7f8798621740 10 client.20322 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:45.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.734+0000 7f8798621740 1 -- 172.21.15.86:0/1480648801 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55728af55fb0 con 0x55728aedfa80 2024-04-21T08:50:45.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.734+0000 7f876effd640 1 -- 172.21.15.86:0/1480648801 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f878001e040 con 0x55728aedfa80 2024-04-21T08:50:45.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.734+0000 7f876effd640 1 -- 172.21.15.86:0/1480648801 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8784005040 con 0x55728aedfa80 2024-04-21T08:50:45.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.734+0000 7f8798621740 10 client.20322 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:45.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.734+0000 7f8798621740 10 client.20322 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:45.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.734+0000 7f8798621740 20 client.20322 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:45.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.734+0000 7f8798621740 1 --2- 172.21.15.86:0/1480648801 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55728af56640 0x55728af58a60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:45.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.734+0000 7f8798621740 4 client.20322 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:50:45.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.734+0000 7f8798621740 1 -- 172.21.15.86:0/1480648801 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55728ae42a70 con 0x55728af56640 2024-04-21T08:50:45.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.735+0000 7f878effd640 1 --2- 172.21.15.86:0/1480648801 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55728af56640 0x55728af58a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:45.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.735+0000 7f876effd640 10 client.20322 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:45.742 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.735+0000 7f878effd640 1 --2- 172.21.15.86:0/1480648801 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55728af56640 0x55728af58a60 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8c9452998926c89d server_cookie=3e2d7735bd25dab0 in_seq=0 out_seq=0 2024-04-21T08:50:45.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.735+0000 7f876effd640 1 -- 172.21.15.86:0/1480648801 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55728ae42a70 con 0x55728af56640 2024-04-21T08:50:45.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.735+0000 7f876effd640 10 client.20322 handle_command_reply: tid=0 2024-04-21T08:50:45.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.735+0000 7f8798621740 1 client.20322 shutdown 2024-04-21T08:50:45.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.736+0000 7f8798621740 1 -- 172.21.15.86:0/1480648801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55728aedfa80 msgr2=0x55728aee3900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:45.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.736+0000 7f8798621740 1 --2- 172.21.15.86:0/1480648801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55728aedfa80 0x55728aee3900 secure :-1 s=READY pgs=2031 cs=0 l=1 rev1=1 crypto rx=0x7f8784012ec0 tx=0x7f8784016040 comp rx=0 tx=0).stop 2024-04-21T08:50:45.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.736+0000 7f8798621740 1 -- 172.21.15.86:0/1480648801 shutdown_connections 2024-04-21T08:50:45.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.736+0000 7f8798621740 1 --2- 172.21.15.86:0/1480648801 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55728af56640 0x55728af58a60 crc :-1 s=READY pgs=340 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:45.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.736+0000 7f8798621740 1 --2- 172.21.15.86:0/1480648801 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55728aedfa80 0x55728aee3900 unknown :-1 s=CLOSED pgs=2031 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:45.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.736+0000 7f876effd640 0 client.20322 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:45.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.736+0000 7f8798621740 1 --2- 172.21.15.86:0/1480648801 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55728aed6ea0 0x55728aee33c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:45.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.736+0000 7f8798621740 1 --2- 172.21.15.86:0/1480648801 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55728aed6580 0x55728aee8020 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:45.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.736+0000 7f8798621740 1 -- 172.21.15.86:0/1480648801 >> 172.21.15.86:0/1480648801 conn(0x55728aed0cb0 msgr2=0x55728aedcab0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:45.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.736+0000 7f8798621740 1 -- 172.21.15.86:0/1480648801 shutdown_connections 2024-04-21T08:50:45.743 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.736+0000 7f8798621740 1 -- 172.21.15.86:0/1480648801 wait complete. 2024-04-21T08:50:45.744 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.736+0000 7f8798621740 20 client.20322 trim_cache size 0 max 16384 2024-04-21T08:50:45.744 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:50:45.744 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-21T08:50:45.744 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-21T08:50:45.744 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:50:45.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.736+0000 7f8795952640 1 -- 172.21.15.86:0/2377446924 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f87780a3d20 msgr2=0x7f87780a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:45.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.736+0000 7f8795952640 1 --2- 172.21.15.86:0/2377446924 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f87780a3d20 0x7f87780a61e0 secure :-1 s=READY pgs=338 cs=0 l=1 rev1=1 crypto rx=0x7f87880efdd0 tx=0x7f878802e040 comp rx=0 tx=0).stop 2024-04-21T08:50:45.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.736+0000 7f8795952640 1 -- 172.21.15.86:0/2377446924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87901985e0 msgr2=0x7f8790198a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:45.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.736+0000 7f8795952640 1 --2- 172.21.15.86:0/2377446924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87901985e0 0x7f8790198a90 secure :-1 s=READY pgs=2027 cs=0 l=1 rev1=1 crypto rx=0x7f8788005e30 tx=0x7f8788032420 comp rx=0 tx=0).stop 2024-04-21T08:50:45.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.736+0000 7f8795952640 1 -- 172.21.15.86:0/2377446924 shutdown_connections 2024-04-21T08:50:45.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.736+0000 7f8795952640 1 --2- 172.21.15.86:0/2377446924 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f87780a3d20 0x7f87780a61e0 unknown :-1 s=CLOSED pgs=338 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:45.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.736+0000 7f8795952640 1 --2- 172.21.15.86:0/2377446924 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f87901985e0 0x7f8790198a90 unknown :-1 s=CLOSED pgs=2027 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:45.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.736+0000 7f8795952640 1 --2- 172.21.15.86:0/2377446924 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8790106160 0x7f879019e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:45.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.736+0000 7f8795952640 1 --2- 172.21.15.86:0/2377446924 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8790101090 0x7f879019e3e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:45.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.736+0000 7f8795952640 1 -- 172.21.15.86:0/2377446924 >> 172.21.15.86:0/2377446924 conn(0x7f87900771d0 msgr2=0x7f8790104e40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:45.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.736+0000 7f8795952640 1 -- 172.21.15.86:0/2377446924 shutdown_connections 2024-04-21T08:50:45.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.736+0000 7f8795952640 1 -- 172.21.15.86:0/2377446924 wait complete. 2024-04-21T08:50:45.747 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:50:45.748 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:50:45.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.845+0000 7f1be4825640 1 -- 172.21.15.86:0/4160530848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1be0101b60 msgr2=0x7f1be010dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:45.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.845+0000 7f1be4825640 1 --2- 172.21.15.86:0/4160530848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1be0101b60 0x7f1be010dc90 secure :-1 s=READY pgs=2032 cs=0 l=1 rev1=1 crypto rx=0x7f1bd8009930 tx=0x7f1bd802bc30 comp rx=0 tx=0).stop 2024-04-21T08:50:45.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.845+0000 7f1be4825640 1 -- 172.21.15.86:0/4160530848 shutdown_connections 2024-04-21T08:50:45.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.845+0000 7f1be4825640 1 --2- 172.21.15.86:0/4160530848 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1be0101b60 0x7f1be010dc90 unknown :-1 s=CLOSED pgs=2032 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:45.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.845+0000 7f1be4825640 1 --2- 172.21.15.86:0/4160530848 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1be01011a0 0x7f1be0101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:45.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.845+0000 7f1be4825640 1 --2- 172.21.15.86:0/4160530848 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be0105e30 0x7f1be0106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:45.846 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.845+0000 7f1be4825640 1 -- 172.21.15.86:0/4160530848 >> 172.21.15.86:0/4160530848 conn(0x7f1be0077e30 msgr2=0x7f1be0078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:45.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.846+0000 7f1be4825640 1 -- 172.21.15.86:0/4160530848 shutdown_connections 2024-04-21T08:50:45.847 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.846+0000 7f1be4825640 1 -- 172.21.15.86:0/4160530848 wait complete. 2024-04-21T08:50:45.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.846+0000 7f1be4825640 1 Processor -- start 2024-04-21T08:50:45.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.846+0000 7f1be4825640 1 -- start start 2024-04-21T08:50:45.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.846+0000 7f1be4825640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1be01011a0 0x7f1be0199540 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:45.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.846+0000 7f1be4825640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1be0101b60 0x7f1be0199a80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:45.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.846+0000 7f1be4825640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be0105e30 0x7f1be01a45c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:45.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.846+0000 7f1be4825640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1be01110e0 con 0x7f1be0101b60 2024-04-21T08:50:45.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.846+0000 7f1be4825640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1be0110f60 con 0x7f1be01011a0 2024-04-21T08:50:45.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.846+0000 7f1be4825640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1be0111260 con 0x7f1be0105e30 2024-04-21T08:50:45.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.847+0000 7f1bddd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1be0101b60 0x7f1be0199a80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:45.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.847+0000 7f1bddd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1be0101b60 0x7f1be0199a80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45822/0 (socket says 172.21.15.86:45822) 2024-04-21T08:50:45.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.847+0000 7f1bddd9b640 1 -- 172.21.15.86:0/1275081894 learned_addr learned my addr 172.21.15.86:0/1275081894 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:45.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.847+0000 7f1bde59c640 1 --2- 172.21.15.86:0/1275081894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1be01011a0 0x7f1be0199540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:45.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.847+0000 7f1bded9d640 1 --2- 172.21.15.86:0/1275081894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be0105e30 0x7f1be01a45c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:45.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.847+0000 7f1bddd9b640 1 -- 172.21.15.86:0/1275081894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be0105e30 msgr2=0x7f1be01a45c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:45.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.847+0000 7f1bddd9b640 1 --2- 172.21.15.86:0/1275081894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be0105e30 0x7f1be01a45c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:45.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.847+0000 7f1bddd9b640 1 -- 172.21.15.86:0/1275081894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1be01011a0 msgr2=0x7f1be0199540 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:45.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.847+0000 7f1bddd9b640 1 --2- 172.21.15.86:0/1275081894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1be01011a0 0x7f1be0199540 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:45.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.847+0000 7f1bddd9b640 1 -- 172.21.15.86:0/1275081894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1be01a4b00 con 0x7f1be0101b60 2024-04-21T08:50:45.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.847+0000 7f1bde59c640 1 --2- 172.21.15.86:0/1275081894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1be01011a0 0x7f1be0199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:45.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.847+0000 7f1bded9d640 1 --2- 172.21.15.86:0/1275081894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be0105e30 0x7f1be01a45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:45.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.847+0000 7f1bddd9b640 1 --2- 172.21.15.86:0/1275081894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1be0101b60 0x7f1be0199a80 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f1bd400d950 tx=0x7f1bd400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:45.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.847+0000 7f1bcf7fe640 1 -- 172.21.15.86:0/1275081894 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd4017070 con 0x7f1be0101b60 2024-04-21T08:50:45.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.847+0000 7f1be4825640 1 -- 172.21.15.86:0/1275081894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1be01a4cd0 con 0x7f1be0101b60 2024-04-21T08:50:45.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.847+0000 7f1bcf7fe640 1 -- 172.21.15.86:0/1275081894 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1bd4009c30 con 0x7f1be0101b60 2024-04-21T08:50:45.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.847+0000 7f1bcf7fe640 1 -- 172.21.15.86:0/1275081894 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1bd400f620 con 0x7f1be0101b60 2024-04-21T08:50:45.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.847+0000 7f1be4825640 1 -- 172.21.15.86:0/1275081894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1be01a51a0 con 0x7f1be0101b60 2024-04-21T08:50:45.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.848+0000 7f1be4825640 1 -- 172.21.15.86:0/1275081894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1be0068de0 con 0x7f1be0101b60 2024-04-21T08:50:45.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.849+0000 7f1bcf7fe640 1 -- 172.21.15.86:0/1275081894 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1bd4005030 con 0x7f1be0101b60 2024-04-21T08:50:45.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.850+0000 7f1bcf7fe640 1 --2- 172.21.15.86:0/1275081894 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1bc80a3d20 0x7f1bc80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:45.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.850+0000 7f1bcf7fe640 1 -- 172.21.15.86:0/1275081894 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f1bd40d08a0 con 0x7f1be0101b60 2024-04-21T08:50:45.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.850+0000 7f1bde59c640 1 --2- 172.21.15.86:0/1275081894 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1bc80a3d20 0x7f1bc80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:45.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.850+0000 7f1bde59c640 1 --2- 172.21.15.86:0/1275081894 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1bc80a3d20 0x7f1bc80a61e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0005fd0 tx=0x7f1bd00093c0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:45.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:45.854+0000 7f1bcf7fe640 1 -- 172.21.15.86:0/1275081894 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1bd409cb70 con 0x7f1be0101b60 2024-04-21T08:50:46.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.074+0000 7f1be4825640 1 -- 172.21.15.86:0/1275081894 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f1be00696e0 con 0x7f1be0101b60 2024-04-21T08:50:46.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.075+0000 7f1bcf7fe640 1 -- 172.21.15.86:0/1275081894 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7f1be00696e0 con 0x7f1be0101b60 2024-04-21T08:50:46.076 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:50:46.076 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:38:45.202126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24398,24394,24455,24472]},"id":1}]} 2024-04-21T08:50:46.076 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-21T08:50:46.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.077+0000 7f1be4825640 1 -- 172.21.15.86:0/1275081894 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1bc80a3d20 msgr2=0x7f1bc80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.077+0000 7f1be4825640 1 --2- 172.21.15.86:0/1275081894 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1bc80a3d20 0x7f1bc80a61e0 secure :-1 s=READY pgs=339 cs=0 l=1 rev1=1 crypto rx=0x7f1bd0005fd0 tx=0x7f1bd00093c0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.077+0000 7f1be4825640 1 -- 172.21.15.86:0/1275081894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1be0101b60 msgr2=0x7f1be0199a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.077+0000 7f1be4825640 1 --2- 172.21.15.86:0/1275081894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1be0101b60 0x7f1be0199a80 secure :-1 s=READY pgs=2033 cs=0 l=1 rev1=1 crypto rx=0x7f1bd400d950 tx=0x7f1bd400de10 comp rx=0 tx=0).stop 2024-04-21T08:50:46.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.077+0000 7f1be4825640 1 -- 172.21.15.86:0/1275081894 shutdown_connections 2024-04-21T08:50:46.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.077+0000 7f1be4825640 1 --2- 172.21.15.86:0/1275081894 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1bc80a3d20 0x7f1bc80a61e0 unknown :-1 s=CLOSED pgs=339 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.077+0000 7f1be4825640 1 --2- 172.21.15.86:0/1275081894 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1be0105e30 0x7f1be01a45c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.077+0000 7f1be4825640 1 --2- 172.21.15.86:0/1275081894 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1be0101b60 0x7f1be0199a80 unknown :-1 s=CLOSED pgs=2033 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.077+0000 7f1be4825640 1 --2- 172.21.15.86:0/1275081894 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1be01011a0 0x7f1be0199540 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.077+0000 7f1be4825640 1 -- 172.21.15.86:0/1275081894 >> 172.21.15.86:0/1275081894 conn(0x7f1be0077e30 msgr2=0x7f1be00fee80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:46.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.078+0000 7f1be4825640 1 -- 172.21.15.86:0/1275081894 shutdown_connections 2024-04-21T08:50:46.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.078+0000 7f1be4825640 1 -- 172.21.15.86:0/1275081894 wait complete. 2024-04-21T08:50:46.089 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:50:46.190 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.188+0000 7fe93a906640 1 -- 172.21.15.86:0/2066084497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe934101a40 msgr2=0x7fe934105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.188+0000 7fe93a906640 1 --2- 172.21.15.86:0/2066084497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe934101a40 0x7fe934105a80 secure :-1 s=READY pgs=2034 cs=0 l=1 rev1=1 crypto rx=0x7fe928009900 tx=0x7fe92802bc30 comp rx=0 tx=0).stop 2024-04-21T08:50:46.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.188+0000 7fe93a906640 1 -- 172.21.15.86:0/2066084497 shutdown_connections 2024-04-21T08:50:46.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.188+0000 7fe93a906640 1 --2- 172.21.15.86:0/2066084497 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe934106160 0x7fe93410dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.188+0000 7fe93a906640 1 --2- 172.21.15.86:0/2066084497 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe934101a40 0x7fe934105a80 unknown :-1 s=CLOSED pgs=2034 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.188+0000 7fe93a906640 1 --2- 172.21.15.86:0/2066084497 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe934101090 0x7fe934101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.188+0000 7fe93a906640 1 -- 172.21.15.86:0/2066084497 >> 172.21.15.86:0/2066084497 conn(0x7fe934077f50 msgr2=0x7fe934078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:46.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.188+0000 7fe93a906640 1 -- 172.21.15.86:0/2066084497 shutdown_connections 2024-04-21T08:50:46.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.188+0000 7fe93a906640 1 -- 172.21.15.86:0/2066084497 wait complete. 2024-04-21T08:50:46.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.189+0000 7fe93a906640 1 Processor -- start 2024-04-21T08:50:46.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.189+0000 7fe93a906640 1 -- start start 2024-04-21T08:50:46.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.189+0000 7fe93a906640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe934101090 0x7fe934199600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.189+0000 7fe93a906640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe934106160 0x7fe934199b40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.191 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.189+0000 7fe93a906640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe93419a0d0 0x7fe9341a4680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.189+0000 7fe93a906640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe934111100 con 0x7fe93419a0d0 2024-04-21T08:50:46.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.189+0000 7fe93a906640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe934110f80 con 0x7fe934106160 2024-04-21T08:50:46.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.189+0000 7fe93a906640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe934111280 con 0x7fe934101090 2024-04-21T08:50:46.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.189+0000 7fe938ea3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe93419a0d0 0x7fe9341a4680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.189+0000 7fe938ea3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe93419a0d0 0x7fe9341a4680 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45848/0 (socket says 172.21.15.86:45848) 2024-04-21T08:50:46.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.189+0000 7fe938ea3640 1 -- 172.21.15.86:0/3803262683 learned_addr learned my addr 172.21.15.86:0/3803262683 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:46.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.189+0000 7fe9337fe640 1 --2- 172.21.15.86:0/3803262683 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe934106160 0x7fe934199b40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.189+0000 7fe933fff640 1 --2- 172.21.15.86:0/3803262683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe934101090 0x7fe934199600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.189+0000 7fe938ea3640 1 -- 172.21.15.86:0/3803262683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe934101090 msgr2=0x7fe934199600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.189+0000 7fe938ea3640 1 --2- 172.21.15.86:0/3803262683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe934101090 0x7fe934199600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.189+0000 7fe938ea3640 1 -- 172.21.15.86:0/3803262683 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe934106160 msgr2=0x7fe934199b40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.189+0000 7fe938ea3640 1 --2- 172.21.15.86:0/3803262683 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe934106160 0x7fe934199b40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.192 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.189+0000 7fe938ea3640 1 -- 172.21.15.86:0/3803262683 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe9341a4e80 con 0x7fe93419a0d0 2024-04-21T08:50:46.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.190+0000 7fe938ea3640 1 --2- 172.21.15.86:0/3803262683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe93419a0d0 0x7fe9341a4680 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7fe92c00b970 tx=0x7fe92c00be30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:46.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.190+0000 7fe9317fa640 1 -- 172.21.15.86:0/3803262683 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c019070 con 0x7fe93419a0d0 2024-04-21T08:50:46.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.190+0000 7fe93a906640 1 -- 172.21.15.86:0/3803262683 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe9341a5170 con 0x7fe93419a0d0 2024-04-21T08:50:46.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.190+0000 7fe9317fa640 1 -- 172.21.15.86:0/3803262683 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe92c014440 con 0x7fe93419a0d0 2024-04-21T08:50:46.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.190+0000 7fe9317fa640 1 -- 172.21.15.86:0/3803262683 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92c0075d0 con 0x7fe93419a0d0 2024-04-21T08:50:46.193 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.190+0000 7fe93a906640 1 -- 172.21.15.86:0/3803262683 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe9341a5690 con 0x7fe93419a0d0 2024-04-21T08:50:46.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.192+0000 7fe9317fa640 1 -- 172.21.15.86:0/3803262683 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe92c01c070 con 0x7fe93419a0d0 2024-04-21T08:50:46.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.192+0000 7fe93d5cc740 1 Processor -- start 2024-04-21T08:50:46.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.192+0000 7fe93d5cc740 1 -- start start 2024-04-21T08:50:46.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.192+0000 7fe9317fa640 1 --2- 172.21.15.86:0/3803262683 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe91c0a3d20 0x7fe91c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.192+0000 7fe93d5cc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34f6654d0 0x55e34f6658b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.192+0000 7fe93d5cc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e34f665e80 0x55e34f66e430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.192+0000 7fe93d5cc740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e34f66ea00 0x55e34f670e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.192+0000 7fe93d5cc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e34f5ca1f0 con 0x55e34f6654d0 2024-04-21T08:50:46.199 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.192+0000 7fe93d5cc740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55e34f5674b0 con 0x55e34f66ea00 2024-04-21T08:50:46.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.192+0000 7fe93d5cc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e34f5c61b0 con 0x55e34f665e80 2024-04-21T08:50:46.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.193+0000 7fe9337fe640 1 --2- 172.21.15.86:0/3803262683 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe91c0a3d20 0x7fe91c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.193+0000 7fe933fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34f6654d0 0x55e34f6658b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.193+0000 7fe933fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34f6654d0 0x55e34f6658b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45864/0 (socket says 172.21.15.86:45864) 2024-04-21T08:50:46.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.193+0000 7fe933fff640 1 -- 172.21.15.86:0/2124181944 learned_addr learned my addr 172.21.15.86:0/2124181944 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:46.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.193+0000 7fe9317fa640 1 -- 172.21.15.86:0/3803262683 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7fe92c0d0060 con 0x7fe93419a0d0 2024-04-21T08:50:46.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.193+0000 7fe938ea3640 1 --2- 172.21.15.86:0/2124181944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e34f66ea00 0x55e34f670e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.193+0000 7fe933fff640 1 -- 172.21.15.86:0/2124181944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e34f665e80 msgr2=0x55e34f66e430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.193+0000 7fe933fff640 1 --2- 172.21.15.86:0/2124181944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e34f665e80 0x55e34f66e430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.193+0000 7fe933fff640 1 -- 172.21.15.86:0/2124181944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e34f66ea00 msgr2=0x55e34f670e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.193+0000 7fe933fff640 1 --2- 172.21.15.86:0/2124181944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e34f66ea00 0x55e34f670e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.193+0000 7fe933fff640 1 -- 172.21.15.86:0/2124181944 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e34f5591b0 con 0x55e34f6654d0 2024-04-21T08:50:46.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.193+0000 7fe938ea3640 1 --2- 172.21.15.86:0/2124181944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e34f66ea00 0x55e34f670e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:46.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.193+0000 7fe9337fe640 1 --2- 172.21.15.86:0/3803262683 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe91c0a3d20 0x7fe91c0a61e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe928009230 tx=0x7fe928009560 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:46.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.193+0000 7fe933fff640 1 --2- 172.21.15.86:0/2124181944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34f6654d0 0x55e34f6658b0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7fe92400bcd0 tx=0x7fe9240107a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c5507094b2dfe34f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:46.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.193+0000 7fe91a7fc640 1 -- 172.21.15.86:0/2124181944 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92401c680 con 0x55e34f6654d0 2024-04-21T08:50:46.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.193+0000 7fe91a7fc640 1 -- 172.21.15.86:0/2124181944 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe92401ccd0 con 0x55e34f6654d0 2024-04-21T08:50:46.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.193+0000 7fe91a7fc640 1 -- 172.21.15.86:0/2124181944 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe924011a50 con 0x55e34f6654d0 2024-04-21T08:50:46.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.194+0000 7fe93d5cc740 1 -- 172.21.15.86:0/2124181944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34f6654d0 msgr2=0x55e34f6658b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.194+0000 7fe93d5cc740 1 --2- 172.21.15.86:0/2124181944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34f6654d0 0x55e34f6658b0 secure :-1 s=READY pgs=2036 cs=0 l=1 rev1=1 crypto rx=0x7fe92400bcd0 tx=0x7fe9240107a0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.194+0000 7fe93d5cc740 1 -- 172.21.15.86:0/2124181944 shutdown_connections 2024-04-21T08:50:46.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.194+0000 7fe93d5cc740 1 --2- 172.21.15.86:0/2124181944 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e34f66ea00 0x55e34f670e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.194+0000 7fe93d5cc740 1 --2- 172.21.15.86:0/2124181944 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e34f665e80 0x55e34f66e430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.194+0000 7fe93d5cc740 1 --2- 172.21.15.86:0/2124181944 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34f6654d0 0x55e34f6658b0 unknown :-1 s=CLOSED pgs=2036 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.194+0000 7fe93d5cc740 1 -- 172.21.15.86:0/2124181944 >> 172.21.15.86:0/2124181944 conn(0x55e34f65fba0 msgr2=0x55e34f65ff80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:46.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.194+0000 7fe93d5cc740 1 -- 172.21.15.86:0/2124181944 shutdown_connections 2024-04-21T08:50:46.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.194+0000 7fe93d5cc740 1 -- 172.21.15.86:0/2124181944 wait complete. 2024-04-21T08:50:46.201 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.194+0000 7fe93d5cc740 1 -- start start 2024-04-21T08:50:46.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.195+0000 7fe93d5cc740 1 Processor -- start 2024-04-21T08:50:46.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.195+0000 7fe93d5cc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34f6654d0 0x55e34f66e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.195+0000 7fe93d5cc740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e34f665e80 0x55e34f66b6b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.195+0000 7fe93d5cc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e34f66ea00 0x55e34f66bc50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.195+0000 7fe93d5cc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55e34f671970 con 0x55e34f6654d0 2024-04-21T08:50:46.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.195+0000 7fe93d5cc740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55e34f583500 con 0x55e34f665e80 2024-04-21T08:50:46.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.195+0000 7fe93d5cc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55e34f671af0 con 0x55e34f66ea00 2024-04-21T08:50:46.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.195+0000 7fe93d5cc740 10 client.0 _ll_register_callbacks cb 0x55e34f551b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:46.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.195+0000 7fe933fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34f6654d0 0x55e34f66e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.195+0000 7fe933fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34f6654d0 0x55e34f66e320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45876/0 (socket says 172.21.15.86:45876) 2024-04-21T08:50:46.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.195+0000 7fe933fff640 1 -- 172.21.15.86:0/395324290 learned_addr learned my addr 172.21.15.86:0/395324290 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:46.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.195+0000 7fe938ea3640 1 --2- 172.21.15.86:0/395324290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e34f66ea00 0x55e34f66bc50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.195+0000 7fe9337fe640 1 --2- 172.21.15.86:0/395324290 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e34f665e80 0x55e34f66b6b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.202 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.195+0000 7fe933fff640 1 -- 172.21.15.86:0/395324290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e34f66ea00 msgr2=0x55e34f66bc50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.195+0000 7fe933fff640 1 --2- 172.21.15.86:0/395324290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e34f66ea00 0x55e34f66bc50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.195+0000 7fe933fff640 1 -- 172.21.15.86:0/395324290 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e34f665e80 msgr2=0x55e34f66b6b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.195+0000 7fe933fff640 1 --2- 172.21.15.86:0/395324290 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e34f665e80 0x55e34f66b6b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.195+0000 7fe933fff640 1 -- 172.21.15.86:0/395324290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e34f2dc3f0 con 0x55e34f6654d0 2024-04-21T08:50:46.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.195+0000 7fe933fff640 1 --2- 172.21.15.86:0/395324290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34f6654d0 0x55e34f66e320 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fe924011670 tx=0x7fe924011a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ff87d88ce1189801 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:46.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.195+0000 7fe9137fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:46.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.195+0000 7fe9137fe640 1 -- 172.21.15.86:0/395324290 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92402b060 con 0x55e34f6654d0 2024-04-21T08:50:46.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.195+0000 7fe9137fe640 1 -- 172.21.15.86:0/395324290 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7fe924011e00 con 0x55e34f6654d0 2024-04-21T08:50:46.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.195+0000 7fe93d5cc740 1 -- 172.21.15.86:0/395324290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe92401ccd0 con 0x55e34f6654d0 2024-04-21T08:50:46.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.196+0000 7fe9137fe640 1 -- 172.21.15.86:0/395324290 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe924032050 con 0x55e34f6654d0 2024-04-21T08:50:46.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.196+0000 7fe93d5cc740 10 client.20340 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:46.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.196+0000 7fe93d5cc740 1 -- 172.21.15.86:0/395324290 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e34f3a67d0 con 0x55e34f6654d0 2024-04-21T08:50:46.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.196+0000 7fe9137fe640 1 -- 172.21.15.86:0/395324290 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe92401ccd0 con 0x55e34f6654d0 2024-04-21T08:50:46.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.196+0000 7fe9137fe640 1 -- 172.21.15.86:0/395324290 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe92401a6b0 con 0x55e34f6654d0 2024-04-21T08:50:46.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.196+0000 7fe93d5cc740 10 client.20340 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:46.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.196+0000 7fe93d5cc740 10 client.20340 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:50:46.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.196+0000 7fe93d5cc740 20 client.20340 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:46.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.196+0000 7fe93d5cc740 1 --2- 172.21.15.86:0/395324290 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55e34f67eac0 0x55e34f680eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.196+0000 7fe93d5cc740 4 client.20340 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:46.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.196+0000 7fe93d5cc740 1 -- 172.21.15.86:0/395324290 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55e34f569930 con 0x55e34f67eac0 2024-04-21T08:50:46.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.196+0000 7fe9337fe640 1 --2- 172.21.15.86:0/395324290 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55e34f67eac0 0x55e34f680eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.197+0000 7fe9337fe640 1 --2- 172.21.15.86:0/395324290 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55e34f67eac0 0x55e34f680eb0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=bd2d64fce9cf775f server_cookie=bff294b21e39252e in_seq=0 out_seq=0 2024-04-21T08:50:46.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.197+0000 7fe9137fe640 10 client.20340 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:50:46.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.197+0000 7fe9137fe640 1 -- 172.21.15.86:0/395324290 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55e34f569930 con 0x55e34f67eac0 2024-04-21T08:50:46.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.197+0000 7fe9137fe640 10 client.20340 handle_command_reply: tid=0 2024-04-21T08:50:46.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.197+0000 7fe93d5cc740 1 client.20340 shutdown 2024-04-21T08:50:46.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.198+0000 7fe93d5cc740 1 -- 172.21.15.86:0/395324290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34f6654d0 msgr2=0x55e34f66e320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.198+0000 7fe93d5cc740 1 --2- 172.21.15.86:0/395324290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34f6654d0 0x55e34f66e320 secure :-1 s=READY pgs=2037 cs=0 l=1 rev1=1 crypto rx=0x7fe924011670 tx=0x7fe924011a00 comp rx=0 tx=0).stop 2024-04-21T08:50:46.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.198+0000 7fe93d5cc740 1 -- 172.21.15.86:0/395324290 shutdown_connections 2024-04-21T08:50:46.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.198+0000 7fe93d5cc740 1 --2- 172.21.15.86:0/395324290 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e34f66ea00 0x55e34f66bc50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.198+0000 7fe93d5cc740 1 --2- 172.21.15.86:0/395324290 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55e34f67eac0 0x55e34f680eb0 crc :-1 s=READY pgs=76 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.198+0000 7fe93d5cc740 1 --2- 172.21.15.86:0/395324290 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e34f665e80 0x55e34f66b6b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.198+0000 7fe93d5cc740 1 --2- 172.21.15.86:0/395324290 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34f6654d0 0x55e34f66e320 unknown :-1 s=CLOSED pgs=2037 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.198+0000 7fe9137fe640 0 client.20340 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:50:46.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.198+0000 7fe93d5cc740 1 -- 172.21.15.86:0/395324290 >> 172.21.15.86:0/395324290 conn(0x55e34f65fba0 msgr2=0x55e34f6618f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:46.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.198+0000 7fe93d5cc740 1 -- 172.21.15.86:0/395324290 shutdown_connections 2024-04-21T08:50:46.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.198+0000 7fe93d5cc740 1 -- 172.21.15.86:0/395324290 wait complete. 2024-04-21T08:50:46.205 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.198+0000 7fe93d5cc740 20 client.20340 trim_cache size 0 max 16384 2024-04-21T08:50:46.216 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:45 smithi088 ceph-mon[31259]: pgmap v374: 129 pgs: 129 active+clean; 2.7 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-21T08:50:46.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.211+0000 7fe93d5cc740 1 Processor -- start 2024-04-21T08:50:46.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.211+0000 7fe93d5cc740 1 -- start start 2024-04-21T08:50:46.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.211+0000 7fe93d5cc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e34f6654d0 0x55e34f67a830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.211+0000 7fe93d5cc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34f665e80 0x55e34f66dde0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.211+0000 7fe93d5cc740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e34f66ea00 0x55e34f66e320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.211+0000 7fe93d5cc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe9341af490 con 0x55e34f665e80 2024-04-21T08:50:46.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.211+0000 7fe93d5cc740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe9341b0fd0 con 0x55e34f66ea00 2024-04-21T08:50:46.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.211+0000 7fe93d5cc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe9341afb60 con 0x55e34f6654d0 2024-04-21T08:50:46.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.211+0000 7fe9337fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34f665e80 0x55e34f66dde0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.218 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.211+0000 7fe9337fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34f665e80 0x55e34f66dde0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45882/0 (socket says 172.21.15.86:45882) 2024-04-21T08:50:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.211+0000 7fe9337fe640 1 -- 172.21.15.86:0/4162893276 learned_addr learned my addr 172.21.15.86:0/4162893276 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.211+0000 7fe933fff640 1 --2- 172.21.15.86:0/4162893276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e34f6654d0 0x55e34f67a830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.211+0000 7fe938ea3640 1 --2- 172.21.15.86:0/4162893276 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e34f66ea00 0x55e34f66e320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.211+0000 7fe9337fe640 1 -- 172.21.15.86:0/4162893276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e34f6654d0 msgr2=0x55e34f67a830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.211+0000 7fe9337fe640 1 --2- 172.21.15.86:0/4162893276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e34f6654d0 0x55e34f67a830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.211+0000 7fe9337fe640 1 -- 172.21.15.86:0/4162893276 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e34f66ea00 msgr2=0x55e34f66e320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.211+0000 7fe9337fe640 1 --2- 172.21.15.86:0/4162893276 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e34f66ea00 0x55e34f66e320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.211+0000 7fe9337fe640 1 -- 172.21.15.86:0/4162893276 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55e34f665c20 con 0x55e34f665e80 2024-04-21T08:50:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.211+0000 7fe938ea3640 1 --2- 172.21.15.86:0/4162893276 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e34f66ea00 0x55e34f66e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.211+0000 7fe933fff640 1 --2- 172.21.15.86:0/4162893276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e34f6654d0 0x55e34f67a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.212+0000 7fe9337fe640 1 --2- 172.21.15.86:0/4162893276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34f665e80 0x55e34f66dde0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fe93406b6e0 tx=0x7fe92802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7be941d32e1e08fc server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.212+0000 7fe919ffb640 1 -- 172.21.15.86:0/4162893276 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe928005040 con 0x55e34f665e80 2024-04-21T08:50:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.212+0000 7fe919ffb640 1 -- 172.21.15.86:0/4162893276 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9280046c0 con 0x55e34f665e80 2024-04-21T08:50:46.219 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.212+0000 7fe919ffb640 1 -- 172.21.15.86:0/4162893276 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe928052170 con 0x55e34f665e80 2024-04-21T08:50:46.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.212+0000 7fe93d5cc740 1 -- 172.21.15.86:0/4162893276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34f665e80 msgr2=0x55e34f66dde0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.212+0000 7fe93d5cc740 1 --2- 172.21.15.86:0/4162893276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34f665e80 0x55e34f66dde0 secure :-1 s=READY pgs=2038 cs=0 l=1 rev1=1 crypto rx=0x7fe93406b6e0 tx=0x7fe92802bc30 comp rx=0 tx=0).stop 2024-04-21T08:50:46.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.212+0000 7fe93d5cc740 1 -- 172.21.15.86:0/4162893276 shutdown_connections 2024-04-21T08:50:46.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.212+0000 7fe93d5cc740 1 --2- 172.21.15.86:0/4162893276 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e34f66ea00 0x55e34f66e320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.212+0000 7fe93d5cc740 1 --2- 172.21.15.86:0/4162893276 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34f665e80 0x55e34f66dde0 unknown :-1 s=CLOSED pgs=2038 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.212+0000 7fe93d5cc740 1 --2- 172.21.15.86:0/4162893276 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e34f6654d0 0x55e34f67a830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.212+0000 7fe93d5cc740 1 -- 172.21.15.86:0/4162893276 >> 172.21.15.86:0/4162893276 conn(0x55e34f65fba0 msgr2=0x55e34f66b480 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:46.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.212+0000 7fe93d5cc740 1 -- 172.21.15.86:0/4162893276 shutdown_connections 2024-04-21T08:50:46.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.212+0000 7fe93d5cc740 1 -- 172.21.15.86:0/4162893276 wait complete. 2024-04-21T08:50:46.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.213+0000 7fe93d5cc740 1 -- start start 2024-04-21T08:50:46.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.213+0000 7fe93d5cc740 1 Processor -- start 2024-04-21T08:50:46.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.213+0000 7fe93d5cc740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34f6654d0 0x55e34f6dd140 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.220 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.213+0000 7fe93d5cc740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e34f665e80 0x55e34f6dd680 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.213+0000 7fe93d5cc740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e34f66ea00 0x55e34f672110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.213+0000 7fe93d5cc740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe9341b0900 con 0x55e34f6654d0 2024-04-21T08:50:46.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.213+0000 7fe93d5cc740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe9341b0230 con 0x55e34f66ea00 2024-04-21T08:50:46.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.213+0000 7fe93d5cc740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe9341ad490 con 0x55e34f665e80 2024-04-21T08:50:46.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.213+0000 7fe93d5cc740 10 client.0 _ll_register_callbacks cb 0x7fe92400b900 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:46.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.213+0000 7fe933fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34f6654d0 0x55e34f6dd140 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.213+0000 7fe933fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34f6654d0 0x55e34f6dd140 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45896/0 (socket says 172.21.15.86:45896) 2024-04-21T08:50:46.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.213+0000 7fe933fff640 1 -- 172.21.15.86:0/3884156719 learned_addr learned my addr 172.21.15.86:0/3884156719 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:46.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.213+0000 7fe938ea3640 1 --2- 172.21.15.86:0/3884156719 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e34f66ea00 0x55e34f672110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.213+0000 7fe9337fe640 1 --2- 172.21.15.86:0/3884156719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e34f665e80 0x55e34f6dd680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.213+0000 7fe933fff640 1 -- 172.21.15.86:0/3884156719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e34f665e80 msgr2=0x55e34f6dd680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.213+0000 7fe933fff640 1 --2- 172.21.15.86:0/3884156719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e34f665e80 0x55e34f6dd680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.213+0000 7fe933fff640 1 -- 172.21.15.86:0/3884156719 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e34f66ea00 msgr2=0x55e34f672110 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.221 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.213+0000 7fe933fff640 1 --2- 172.21.15.86:0/3884156719 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e34f66ea00 0x55e34f672110 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.213+0000 7fe933fff640 1 -- 172.21.15.86:0/3884156719 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55e34f672800 con 0x55e34f6654d0 2024-04-21T08:50:46.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.214+0000 7fe933fff640 1 --2- 172.21.15.86:0/3884156719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34f6654d0 0x55e34f6dd140 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fe92400c010 tx=0x7fe92400b810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=21f059f44d8282a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:46.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.214+0000 7fe9137fe640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:46.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.214+0000 7fe9137fe640 1 -- 172.21.15.86:0/3884156719 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe92403b060 con 0x55e34f6654d0 2024-04-21T08:50:46.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.214+0000 7fe9137fe640 1 -- 172.21.15.86:0/3884156719 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7fe924011e00 con 0x55e34f6654d0 2024-04-21T08:50:46.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.214+0000 7fe93d5cc740 1 -- 172.21.15.86:0/3884156719 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fe9280046c0 con 0x55e34f6654d0 2024-04-21T08:50:46.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.214+0000 7fe9137fe640 1 -- 172.21.15.86:0/3884156719 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fe924042050 con 0x55e34f6654d0 2024-04-21T08:50:46.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.214+0000 7fe93d5cc740 10 client.20346 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:46.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.214+0000 7fe93d5cc740 1 -- 172.21.15.86:0/3884156719 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55e34f6e4900 con 0x55e34f6654d0 2024-04-21T08:50:46.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.214+0000 7fe9137fe640 1 -- 172.21.15.86:0/3884156719 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe9280046c0 con 0x55e34f6654d0 2024-04-21T08:50:46.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.214+0000 7fe9137fe640 1 -- 172.21.15.86:0/3884156719 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fe924005270 con 0x55e34f6654d0 2024-04-21T08:50:46.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.214+0000 7fe93d5cc740 10 client.20346 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:46.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.214+0000 7fe93d5cc740 10 client.20346 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:50:46.222 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.214+0000 7fe93d5cc740 20 client.20346 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:46.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.214+0000 7fe93d5cc740 1 --2- 172.21.15.86:0/3884156719 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55e34f6e5050 0x55e34f6e7470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.214+0000 7fe93d5cc740 4 client.20346 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:50:46.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.214+0000 7fe93d5cc740 1 -- 172.21.15.86:0/3884156719 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55e34f5d7c10 con 0x55e34f6e5050 2024-04-21T08:50:46.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.214+0000 7fe9337fe640 1 --2- 172.21.15.86:0/3884156719 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55e34f6e5050 0x55e34f6e7470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.215+0000 7fe9137fe640 10 client.20346 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:50:46.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.215+0000 7fe9337fe640 1 --2- 172.21.15.86:0/3884156719 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55e34f6e5050 0x55e34f6e7470 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1e7d749602ecd649 server_cookie=f8afe733e62c4242 in_seq=0 out_seq=0 2024-04-21T08:50:46.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.215+0000 7fe9137fe640 1 -- 172.21.15.86:0/3884156719 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55e34f5d7c10 con 0x55e34f6e5050 2024-04-21T08:50:46.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.215+0000 7fe9137fe640 10 client.20346 handle_command_reply: tid=0 2024-04-21T08:50:46.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.215+0000 7fe93d5cc740 1 client.20346 shutdown 2024-04-21T08:50:46.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.215+0000 7fe93d5cc740 1 -- 172.21.15.86:0/3884156719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34f6654d0 msgr2=0x55e34f6dd140 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.215+0000 7fe93d5cc740 1 --2- 172.21.15.86:0/3884156719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34f6654d0 0x55e34f6dd140 secure :-1 s=READY pgs=2039 cs=0 l=1 rev1=1 crypto rx=0x7fe92400c010 tx=0x7fe92400b810 comp rx=0 tx=0).stop 2024-04-21T08:50:46.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.215+0000 7fe93d5cc740 1 -- 172.21.15.86:0/3884156719 shutdown_connections 2024-04-21T08:50:46.223 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.215+0000 7fe93d5cc740 1 --2- 172.21.15.86:0/3884156719 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x55e34f6e5050 0x55e34f6e7470 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.215+0000 7fe93d5cc740 1 --2- 172.21.15.86:0/3884156719 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55e34f66ea00 0x55e34f672110 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.215+0000 7fe93d5cc740 1 --2- 172.21.15.86:0/3884156719 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55e34f665e80 0x55e34f6dd680 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.215+0000 7fe9137fe640 0 client.20346 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:50:46.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.215+0000 7fe93d5cc740 1 --2- 172.21.15.86:0/3884156719 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55e34f6654d0 0x55e34f6dd140 unknown :-1 s=CLOSED pgs=2039 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.215+0000 7fe93d5cc740 1 -- 172.21.15.86:0/3884156719 >> 172.21.15.86:0/3884156719 conn(0x55e34f65fba0 msgr2=0x55e34f666260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:46.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.215+0000 7fe93d5cc740 1 -- 172.21.15.86:0/3884156719 shutdown_connections 2024-04-21T08:50:46.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.215+0000 7fe93d5cc740 1 -- 172.21.15.86:0/3884156719 wait complete. 2024-04-21T08:50:46.224 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.215+0000 7fe93d5cc740 20 client.20346 trim_cache size 0 max 16384 2024-04-21T08:50:46.224 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:50:46.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.216+0000 7fe93a906640 1 -- 172.21.15.86:0/3803262683 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe91c0a3d20 msgr2=0x7fe91c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.216+0000 7fe93a906640 1 --2- 172.21.15.86:0/3803262683 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe91c0a3d20 0x7fe91c0a61e0 secure :-1 s=READY pgs=340 cs=0 l=1 rev1=1 crypto rx=0x7fe928009230 tx=0x7fe928009560 comp rx=0 tx=0).stop 2024-04-21T08:50:46.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.216+0000 7fe93a906640 1 -- 172.21.15.86:0/3803262683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe93419a0d0 msgr2=0x7fe9341a4680 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.216+0000 7fe93a906640 1 --2- 172.21.15.86:0/3803262683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe93419a0d0 0x7fe9341a4680 secure :-1 s=READY pgs=2035 cs=0 l=1 rev1=1 crypto rx=0x7fe92c00b970 tx=0x7fe92c00be30 comp rx=0 tx=0).stop 2024-04-21T08:50:46.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.216+0000 7fe93a906640 1 -- 172.21.15.86:0/3803262683 shutdown_connections 2024-04-21T08:50:46.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.216+0000 7fe93a906640 1 --2- 172.21.15.86:0/3803262683 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe91c0a3d20 0x7fe91c0a61e0 unknown :-1 s=CLOSED pgs=340 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.225 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.216+0000 7fe93a906640 1 --2- 172.21.15.86:0/3803262683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe93419a0d0 0x7fe9341a4680 unknown :-1 s=CLOSED pgs=2035 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.216+0000 7fe93a906640 1 --2- 172.21.15.86:0/3803262683 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe934106160 0x7fe934199b40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.216+0000 7fe93a906640 1 --2- 172.21.15.86:0/3803262683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe934101090 0x7fe934199600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.216+0000 7fe93a906640 1 -- 172.21.15.86:0/3803262683 >> 172.21.15.86:0/3803262683 conn(0x7fe934077f50 msgr2=0x7fe9340ff1e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:46.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.216+0000 7fe93a906640 1 -- 172.21.15.86:0/3803262683 shutdown_connections 2024-04-21T08:50:46.226 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.216+0000 7fe93a906640 1 -- 172.21.15.86:0/3803262683 wait complete. 2024-04-21T08:50:46.226 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:45 smithi086 ceph-mon[27939]: pgmap v374: 129 pgs: 129 active+clean; 2.7 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-21T08:50:46.227 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:50:46.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:45 smithi179 ceph-mon[31829]: pgmap v374: 129 pgs: 129 active+clean; 2.7 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-21T08:50:46.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.327+0000 7f31b3d6d640 1 -- 172.21.15.86:0/3432685338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31ac101b40 msgr2=0x7f31ac10dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.327+0000 7f31b3d6d640 1 --2- 172.21.15.86:0/3432685338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31ac101b40 0x7f31ac10dc80 secure :-1 s=READY pgs=2040 cs=0 l=1 rev1=1 crypto rx=0x7f31a4009cd0 tx=0x7f31a402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.327+0000 7f31b3d6d640 1 -- 172.21.15.86:0/3432685338 shutdown_connections 2024-04-21T08:50:46.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.327+0000 7f31b3d6d640 1 --2- 172.21.15.86:0/3432685338 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31ac101b40 0x7f31ac10dc80 unknown :-1 s=CLOSED pgs=2040 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.327+0000 7f31b3d6d640 1 --2- 172.21.15.86:0/3432685338 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31ac1011a0 0x7f31ac101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.329 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.327+0000 7f31b3d6d640 1 --2- 172.21.15.86:0/3432685338 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31ac107030 0x7f31ac107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.327+0000 7f31b3d6d640 1 -- 172.21.15.86:0/3432685338 >> 172.21.15.86:0/3432685338 conn(0x7f31ac076c50 msgr2=0x7f31ac077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:46.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.327+0000 7f31b3d6d640 1 -- 172.21.15.86:0/3432685338 shutdown_connections 2024-04-21T08:50:46.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.327+0000 7f31b3d6d640 1 -- 172.21.15.86:0/3432685338 wait complete. 2024-04-21T08:50:46.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.327+0000 7f31b3d6d640 1 Processor -- start 2024-04-21T08:50:46.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.327+0000 7f31b3d6d640 1 -- start start 2024-04-21T08:50:46.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.327+0000 7f31b3d6d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31ac1011a0 0x7f31ac199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.327+0000 7f31b3d6d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31ac101b40 0x7f31ac199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.327+0000 7f31b3d6d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31ac107030 0x7f31ac1a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.327+0000 7f31b3d6d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f31ac1110c0 con 0x7f31ac101b40 2024-04-21T08:50:46.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.327+0000 7f31b3d6d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f31ac110f40 con 0x7f31ac107030 2024-04-21T08:50:46.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.327+0000 7f31b3d6d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f31ac111240 con 0x7f31ac1011a0 2024-04-21T08:50:46.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.328+0000 7f31b1308640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31ac101b40 0x7f31ac199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.328+0000 7f31b1308640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31ac101b40 0x7f31ac199ac0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45916/0 (socket says 172.21.15.86:45916) 2024-04-21T08:50:46.330 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.328+0000 7f31b1308640 1 -- 172.21.15.86:0/1078032486 learned_addr learned my addr 172.21.15.86:0/1078032486 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:46.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.328+0000 7f31b1b09640 1 --2- 172.21.15.86:0/1078032486 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31ac1011a0 0x7f31ac199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.328+0000 7f31b230a640 1 --2- 172.21.15.86:0/1078032486 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31ac107030 0x7f31ac1a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.328+0000 7f31b1308640 1 -- 172.21.15.86:0/1078032486 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31ac1011a0 msgr2=0x7f31ac199580 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.328+0000 7f31b1308640 1 --2- 172.21.15.86:0/1078032486 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31ac1011a0 0x7f31ac199580 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.328+0000 7f31b1308640 1 -- 172.21.15.86:0/1078032486 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31ac107030 msgr2=0x7f31ac1a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.328+0000 7f31b1308640 1 --2- 172.21.15.86:0/1078032486 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31ac107030 0x7f31ac1a4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.328+0000 7f31b1308640 1 -- 172.21.15.86:0/1078032486 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f31ac1a4b40 con 0x7f31ac101b40 2024-04-21T08:50:46.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.328+0000 7f31b1308640 1 --2- 172.21.15.86:0/1078032486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31ac101b40 0x7f31ac199ac0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f31a000d950 tx=0x7f31a000de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:46.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.328+0000 7f319effd640 1 -- 172.21.15.86:0/1078032486 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a0017070 con 0x7f31ac101b40 2024-04-21T08:50:46.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.328+0000 7f319effd640 1 -- 172.21.15.86:0/1078032486 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31a0009c30 con 0x7f31ac101b40 2024-04-21T08:50:46.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.328+0000 7f31b3d6d640 1 -- 172.21.15.86:0/1078032486 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f31ac1a4d10 con 0x7f31ac101b40 2024-04-21T08:50:46.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.328+0000 7f319effd640 1 -- 172.21.15.86:0/1078032486 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a000f620 con 0x7f31ac101b40 2024-04-21T08:50:46.331 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.328+0000 7f31b3d6d640 1 -- 172.21.15.86:0/1078032486 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f31ac1a51d0 con 0x7f31ac101b40 2024-04-21T08:50:46.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.331+0000 7f31b6a35740 1 Processor -- start 2024-04-21T08:50:46.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.331+0000 7f319effd640 1 -- 172.21.15.86:0/1078032486 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f31a0010460 con 0x7f31ac101b40 2024-04-21T08:50:46.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.331+0000 7f31b6a35740 1 -- start start 2024-04-21T08:50:46.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.332+0000 7f31b6a35740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564afbbe84d0 0x564afbbe88b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.332+0000 7f31b6a35740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564afbbe8e80 0x564afbbf1430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.332+0000 7f31b6a35740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564afbbf1a00 0x564afbbf3e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.338 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.332+0000 7f31b6a35740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564afbb4d1f0 con 0x564afbbe84d0 2024-04-21T08:50:46.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.332+0000 7f31b6a35740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564afbaea4b0 con 0x564afbbe8e80 2024-04-21T08:50:46.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.332+0000 7f31b6a35740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564afbb491b0 con 0x564afbbf1a00 2024-04-21T08:50:46.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.332+0000 7f31b230a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564afbbe84d0 0x564afbbe88b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.332+0000 7f31b230a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564afbbe84d0 0x564afbbe88b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45924/0 (socket says 172.21.15.86:45924) 2024-04-21T08:50:46.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.332+0000 7f31b230a640 1 -- 172.21.15.86:0/779428005 learned_addr learned my addr 172.21.15.86:0/779428005 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:46.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.332+0000 7f31b1b09640 1 --2- 172.21.15.86:0/779428005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564afbbe8e80 0x564afbbf1430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.332+0000 7f31b1308640 1 --2- 172.21.15.86:0/779428005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564afbbf1a00 0x564afbbf3e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.332+0000 7f319effd640 1 --2- 172.21.15.86:0/1078032486 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f31980a3d20 0x7f31980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.332+0000 7f31b230a640 1 -- 172.21.15.86:0/779428005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564afbbf1a00 msgr2=0x564afbbf3e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.332+0000 7f31b230a640 1 --2- 172.21.15.86:0/779428005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564afbbf1a00 0x564afbbf3e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.332+0000 7f31b230a640 1 -- 172.21.15.86:0/779428005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564afbbe8e80 msgr2=0x564afbbf1430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.332+0000 7f31b230a640 1 --2- 172.21.15.86:0/779428005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564afbbe8e80 0x564afbbf1430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.332+0000 7f31b230a640 1 -- 172.21.15.86:0/779428005 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564afbadc1b0 con 0x564afbbe84d0 2024-04-21T08:50:46.339 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.332+0000 7f31b1b09640 1 --2- 172.21.15.86:0/779428005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564afbbe8e80 0x564afbbf1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:46.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.332+0000 7f31b1308640 1 --2- 172.21.15.86:0/779428005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564afbbf1a00 0x564afbbf3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:46.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.332+0000 7f319effd640 1 -- 172.21.15.86:0/1078032486 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f31a00d0780 con 0x7f31ac101b40 2024-04-21T08:50:46.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.332+0000 7f31b230a640 1 --2- 172.21.15.86:0/1078032486 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f31980a3d20 0x7f31980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.332+0000 7f31b230a640 1 --2- 172.21.15.86:0/779428005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564afbbe84d0 0x564afbbe88b0 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f31a4004af0 tx=0x7f31a4005f70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e9f1fdeb74e6aaae server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:46.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.333+0000 7f3197fff640 1 -- 172.21.15.86:0/779428005 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a4039040 con 0x564afbbe84d0 2024-04-21T08:50:46.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.333+0000 7f3197fff640 1 -- 172.21.15.86:0/779428005 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31a4003e20 con 0x564afbbe84d0 2024-04-21T08:50:46.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.333+0000 7f3197fff640 1 -- 172.21.15.86:0/779428005 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a4004760 con 0x564afbbe84d0 2024-04-21T08:50:46.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.333+0000 7f31b230a640 1 --2- 172.21.15.86:0/1078032486 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f31980a3d20 0x7f31980a61e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f31a404c000 tx=0x7f31a4045000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:46.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.333+0000 7f31b6a35740 1 -- 172.21.15.86:0/779428005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564afbbe84d0 msgr2=0x564afbbe88b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.333+0000 7f31b6a35740 1 --2- 172.21.15.86:0/779428005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564afbbe84d0 0x564afbbe88b0 secure :-1 s=READY pgs=2042 cs=0 l=1 rev1=1 crypto rx=0x7f31a4004af0 tx=0x7f31a4005f70 comp rx=0 tx=0).stop 2024-04-21T08:50:46.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.333+0000 7f31b6a35740 1 -- 172.21.15.86:0/779428005 shutdown_connections 2024-04-21T08:50:46.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.333+0000 7f31b6a35740 1 --2- 172.21.15.86:0/779428005 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564afbbf1a00 0x564afbbf3e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.340 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.333+0000 7f31b6a35740 1 --2- 172.21.15.86:0/779428005 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564afbbe8e80 0x564afbbf1430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.333+0000 7f31b6a35740 1 --2- 172.21.15.86:0/779428005 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564afbbe84d0 0x564afbbe88b0 unknown :-1 s=CLOSED pgs=2042 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.333+0000 7f31b6a35740 1 -- 172.21.15.86:0/779428005 >> 172.21.15.86:0/779428005 conn(0x564afbbe2ba0 msgr2=0x564afbbe2f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:46.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.334+0000 7f31b6a35740 1 -- 172.21.15.86:0/779428005 shutdown_connections 2024-04-21T08:50:46.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.334+0000 7f31b6a35740 1 -- 172.21.15.86:0/779428005 wait complete. 2024-04-21T08:50:46.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.334+0000 7f31b6a35740 1 -- start start 2024-04-21T08:50:46.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.334+0000 7f31b6a35740 1 Processor -- start 2024-04-21T08:50:46.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.334+0000 7f31b6a35740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564afbbe84d0 0x564afbbf1330 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.334+0000 7f31b6a35740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564afbbe8e80 0x564afbbee6c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.334+0000 7f31b6a35740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564afbbf1a00 0x564afbbeecd0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.334+0000 7f31b6a35740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564afbbf4970 con 0x564afbbe84d0 2024-04-21T08:50:46.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.334+0000 7f31b6a35740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564afbb06500 con 0x564afbbf1a00 2024-04-21T08:50:46.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.334+0000 7f31b6a35740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564afbbf4af0 con 0x564afbbe8e80 2024-04-21T08:50:46.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.334+0000 7f31b1b09640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564afbbe84d0 0x564afbbf1330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.341 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.334+0000 7f31b6a35740 10 client.0 _ll_register_callbacks cb 0x564afbad4b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:46.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.334+0000 7f31b1b09640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564afbbe84d0 0x564afbbf1330 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45936/0 (socket says 172.21.15.86:45936) 2024-04-21T08:50:46.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.334+0000 7f31b1b09640 1 -- 172.21.15.86:0/2721683111 learned_addr learned my addr 172.21.15.86:0/2721683111 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:46.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.335+0000 7f31b230a640 1 --2- 172.21.15.86:0/2721683111 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564afbbf1a00 0x564afbbeecd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.335+0000 7f31b1308640 1 --2- 172.21.15.86:0/2721683111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564afbbe8e80 0x564afbbee6c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.335+0000 7f31b1b09640 1 -- 172.21.15.86:0/2721683111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564afbbe8e80 msgr2=0x564afbbee6c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.335+0000 7f31b1b09640 1 --2- 172.21.15.86:0/2721683111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564afbbe8e80 0x564afbbee6c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.335+0000 7f31b1b09640 1 -- 172.21.15.86:0/2721683111 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564afbbf1a00 msgr2=0x564afbbeecd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.335+0000 7f31b1b09640 1 --2- 172.21.15.86:0/2721683111 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564afbbf1a00 0x564afbbeecd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.335+0000 7f31b1b09640 1 -- 172.21.15.86:0/2721683111 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564afb85f3f0 con 0x564afbbe84d0 2024-04-21T08:50:46.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.335+0000 7f31b230a640 1 --2- 172.21.15.86:0/2721683111 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564afbbf1a00 0x564afbbeecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:46.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.335+0000 7f31b1b09640 1 --2- 172.21.15.86:0/2721683111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564afbbe84d0 0x564afbbf1330 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f31a8012700 tx=0x7f31a8012b00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c7751ec7169e179b server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:46.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.335+0000 7f31957fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:46.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.335+0000 7f31957fa640 1 -- 172.21.15.86:0/2721683111 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a8004050 con 0x564afbbe84d0 2024-04-21T08:50:46.342 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.335+0000 7f31957fa640 1 -- 172.21.15.86:0/2721683111 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f31a80041f0 con 0x564afbbe84d0 2024-04-21T08:50:46.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.335+0000 7f31b6a35740 1 -- 172.21.15.86:0/2721683111 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31a4003e20 con 0x564afbbe84d0 2024-04-21T08:50:46.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.335+0000 7f31957fa640 1 -- 172.21.15.86:0/2721683111 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f31a80043f0 con 0x564afbbe84d0 2024-04-21T08:50:46.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.335+0000 7f31b6a35740 10 client.20358 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:46.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.335+0000 7f31b6a35740 1 -- 172.21.15.86:0/2721683111 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564afb9297d0 con 0x564afbbe84d0 2024-04-21T08:50:46.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.335+0000 7f31957fa640 1 -- 172.21.15.86:0/2721683111 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31a4003e20 con 0x564afbbe84d0 2024-04-21T08:50:46.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.335+0000 7f31957fa640 1 -- 172.21.15.86:0/2721683111 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f31a800f830 con 0x564afbbe84d0 2024-04-21T08:50:46.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.335+0000 7f31b6a35740 10 client.20358 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:46.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.335+0000 7f31b6a35740 10 client.20358 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:50:46.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.335+0000 7f31b6a35740 20 client.20358 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:46.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.335+0000 7f31b6a35740 1 --2- 172.21.15.86:0/2721683111 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x564afbc01d10 0x564afbc04180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.336+0000 7f31b6a35740 4 client.20358 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:46.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.336+0000 7f31b6a35740 1 -- 172.21.15.86:0/2721683111 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564afbaec930 con 0x564afbc01d10 2024-04-21T08:50:46.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.336+0000 7f31b1308640 1 --2- 172.21.15.86:0/2721683111 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x564afbc01d10 0x564afbc04180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.336+0000 7f31957fa640 10 client.20358 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:50:46.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.336+0000 7f31b1308640 1 --2- 172.21.15.86:0/2721683111 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x564afbc01d10 0x564afbc04180 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=ef7c77636b16350d server_cookie=878121333f530847 in_seq=0 out_seq=0 2024-04-21T08:50:46.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.337+0000 7f31957fa640 1 -- 172.21.15.86:0/2721683111 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564afbaec930 con 0x564afbc01d10 2024-04-21T08:50:46.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.337+0000 7f31957fa640 10 client.20358 handle_command_reply: tid=0 2024-04-21T08:50:46.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.337+0000 7f31b6a35740 1 client.20358 shutdown 2024-04-21T08:50:46.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.337+0000 7f31b6a35740 1 -- 172.21.15.86:0/2721683111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564afbbe84d0 msgr2=0x564afbbf1330 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.337+0000 7f31b6a35740 1 --2- 172.21.15.86:0/2721683111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564afbbe84d0 0x564afbbf1330 secure :-1 s=READY pgs=2043 cs=0 l=1 rev1=1 crypto rx=0x7f31a8012700 tx=0x7f31a8012b00 comp rx=0 tx=0).stop 2024-04-21T08:50:46.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.337+0000 7f31b6a35740 1 -- 172.21.15.86:0/2721683111 shutdown_connections 2024-04-21T08:50:46.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.337+0000 7f31b6a35740 1 --2- 172.21.15.86:0/2721683111 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x564afbc01d10 0x564afbc04180 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.337+0000 7f31b6a35740 1 --2- 172.21.15.86:0/2721683111 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564afbbf1a00 0x564afbbeecd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.337+0000 7f31b6a35740 1 --2- 172.21.15.86:0/2721683111 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564afbbe8e80 0x564afbbee6c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.337+0000 7f31b6a35740 1 --2- 172.21.15.86:0/2721683111 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564afbbe84d0 0x564afbbf1330 unknown :-1 s=CLOSED pgs=2043 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.337+0000 7f31957fa640 0 client.20358 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:50:46.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.337+0000 7f31b6a35740 1 -- 172.21.15.86:0/2721683111 >> 172.21.15.86:0/2721683111 conn(0x564afbbe2ba0 msgr2=0x564afbbe48f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:46.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.337+0000 7f31b6a35740 1 -- 172.21.15.86:0/2721683111 shutdown_connections 2024-04-21T08:50:46.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.337+0000 7f31b6a35740 1 -- 172.21.15.86:0/2721683111 wait complete. 2024-04-21T08:50:46.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.337+0000 7f31b6a35740 20 client.20358 trim_cache size 0 max 16384 2024-04-21T08:50:46.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.350+0000 7f31b6a35740 1 Processor -- start 2024-04-21T08:50:46.356 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.350+0000 7f31b6a35740 1 -- start start 2024-04-21T08:50:46.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.350+0000 7f31b6a35740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564afbbe84d0 0x564afbbfd860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.350+0000 7f31b6a35740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564afbbe8e80 0x564afbbed6f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.350+0000 7f31b6a35740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564afbbf1a00 0x564afbbf0df0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.350+0000 7f31b6a35740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f31ac1af3e0 con 0x564afbbf1a00 2024-04-21T08:50:46.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.350+0000 7f31b6a35740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f31ac1b0f20 con 0x564afbbe8e80 2024-04-21T08:50:46.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.350+0000 7f31b6a35740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f31ac1afab0 con 0x564afbbe84d0 2024-04-21T08:50:46.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.350+0000 7f31b230a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564afbbf1a00 0x564afbbf0df0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.350+0000 7f31b230a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564afbbf1a00 0x564afbbf0df0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45942/0 (socket says 172.21.15.86:45942) 2024-04-21T08:50:46.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.350+0000 7f31b230a640 1 -- 172.21.15.86:0/1526212616 learned_addr learned my addr 172.21.15.86:0/1526212616 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:46.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.351+0000 7f31b1308640 1 --2- 172.21.15.86:0/1526212616 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564afbbe8e80 0x564afbbed6f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.351+0000 7f31b1b09640 1 --2- 172.21.15.86:0/1526212616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564afbbe84d0 0x564afbbfd860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.351+0000 7f31b230a640 1 -- 172.21.15.86:0/1526212616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564afbbe84d0 msgr2=0x564afbbfd860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.351+0000 7f31b230a640 1 --2- 172.21.15.86:0/1526212616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564afbbe84d0 0x564afbbfd860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.357 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.351+0000 7f31b230a640 1 -- 172.21.15.86:0/1526212616 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564afbbe8e80 msgr2=0x564afbbed6f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.351+0000 7f31b230a640 1 --2- 172.21.15.86:0/1526212616 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564afbbe8e80 0x564afbbed6f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.351+0000 7f31b230a640 1 -- 172.21.15.86:0/1526212616 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564afbbe34e0 con 0x564afbbf1a00 2024-04-21T08:50:46.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.351+0000 7f31b1b09640 1 --2- 172.21.15.86:0/1526212616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564afbbe84d0 0x564afbbfd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:46.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.351+0000 7f31b230a640 1 --2- 172.21.15.86:0/1526212616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564afbbf1a00 0x564afbbf0df0 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f31a4032460 tx=0x7f31a4032fc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9a2d1505a2cc8f6d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:46.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.351+0000 7f31977fe640 1 -- 172.21.15.86:0/1526212616 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a4039040 con 0x564afbbf1a00 2024-04-21T08:50:46.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.351+0000 7f31977fe640 1 -- 172.21.15.86:0/1526212616 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31a4057030 con 0x564afbbf1a00 2024-04-21T08:50:46.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.351+0000 7f31977fe640 1 -- 172.21.15.86:0/1526212616 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a4004750 con 0x564afbbf1a00 2024-04-21T08:50:46.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.352+0000 7f31b6a35740 1 -- 172.21.15.86:0/1526212616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564afbbf1a00 msgr2=0x564afbbf0df0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.352+0000 7f31b6a35740 1 --2- 172.21.15.86:0/1526212616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564afbbf1a00 0x564afbbf0df0 secure :-1 s=READY pgs=2044 cs=0 l=1 rev1=1 crypto rx=0x7f31a4032460 tx=0x7f31a4032fc0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.352+0000 7f31b6a35740 1 -- 172.21.15.86:0/1526212616 shutdown_connections 2024-04-21T08:50:46.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.352+0000 7f31b6a35740 1 --2- 172.21.15.86:0/1526212616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564afbbf1a00 0x564afbbf0df0 unknown :-1 s=CLOSED pgs=2044 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.352+0000 7f31b6a35740 1 --2- 172.21.15.86:0/1526212616 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564afbbe8e80 0x564afbbed6f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.352+0000 7f31b6a35740 1 --2- 172.21.15.86:0/1526212616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564afbbe84d0 0x564afbbfd860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.358 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.352+0000 7f31b6a35740 1 -- 172.21.15.86:0/1526212616 >> 172.21.15.86:0/1526212616 conn(0x564afbbe2ba0 msgr2=0x564afbbe1b40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:46.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.352+0000 7f31b6a35740 1 -- 172.21.15.86:0/1526212616 shutdown_connections 2024-04-21T08:50:46.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.352+0000 7f31b6a35740 1 -- 172.21.15.86:0/1526212616 wait complete. 2024-04-21T08:50:46.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.352+0000 7f31b6a35740 1 -- start start 2024-04-21T08:50:46.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.352+0000 7f31b6a35740 1 Processor -- start 2024-04-21T08:50:46.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.352+0000 7f31b6a35740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564afbbe84d0 0x564afbbeb500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.352+0000 7f31b6a35740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564afbbe8e80 0x564afbbeba40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.352+0000 7f31b6a35740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564afbbf1a00 0x564afbbebf80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.352+0000 7f31b6a35740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f31ac1b0850 con 0x564afbbf1a00 2024-04-21T08:50:46.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.352+0000 7f31b6a35740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f31ac1b0180 con 0x564afbbe84d0 2024-04-21T08:50:46.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.352+0000 7f31b6a35740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f31ac1ad3e0 con 0x564afbbe8e80 2024-04-21T08:50:46.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.352+0000 7f31b6a35740 10 client.0 _ll_register_callbacks cb 0x7f31a8009d30 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:46.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.352+0000 7f31b230a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564afbbf1a00 0x564afbbebf80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.352+0000 7f31b230a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564afbbf1a00 0x564afbbebf80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45956/0 (socket says 172.21.15.86:45956) 2024-04-21T08:50:46.359 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.352+0000 7f31b230a640 1 -- 172.21.15.86:0/4068947440 learned_addr learned my addr 172.21.15.86:0/4068947440 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:46.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.353+0000 7f31b1b09640 1 --2- 172.21.15.86:0/4068947440 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564afbbe84d0 0x564afbbeb500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.353+0000 7f31b1308640 1 --2- 172.21.15.86:0/4068947440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564afbbe8e80 0x564afbbeba40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.353+0000 7f31b230a640 1 -- 172.21.15.86:0/4068947440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564afbbe8e80 msgr2=0x564afbbeba40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.353+0000 7f31b230a640 1 --2- 172.21.15.86:0/4068947440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564afbbe8e80 0x564afbbeba40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.353+0000 7f31b230a640 1 -- 172.21.15.86:0/4068947440 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564afbbe84d0 msgr2=0x564afbbeb500 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.353+0000 7f31b230a640 1 --2- 172.21.15.86:0/4068947440 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564afbbe84d0 0x564afbbeb500 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.353+0000 7f31b230a640 1 -- 172.21.15.86:0/4068947440 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564afbc6b880 con 0x564afbbf1a00 2024-04-21T08:50:46.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.353+0000 7f31b230a640 1 --2- 172.21.15.86:0/4068947440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564afbbf1a00 0x564afbbebf80 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f31a4009600 tx=0x7f31a4009c60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=811da4a47dd41509 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:46.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.353+0000 7f31957fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:46.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.353+0000 7f31957fa640 1 -- 172.21.15.86:0/4068947440 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f31a4060070 con 0x564afbbf1a00 2024-04-21T08:50:46.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.353+0000 7f31957fa640 1 -- 172.21.15.86:0/4068947440 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f31a4003b70 con 0x564afbbf1a00 2024-04-21T08:50:46.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.353+0000 7f31b6a35740 1 -- 172.21.15.86:0/4068947440 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f31a4057030 con 0x564afbbf1a00 2024-04-21T08:50:46.360 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.353+0000 7f31957fa640 1 -- 172.21.15.86:0/4068947440 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f31a406c050 con 0x564afbbf1a00 2024-04-21T08:50:46.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.353+0000 7f31b6a35740 10 client.20364 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:46.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.353+0000 7f31b6a35740 1 -- 172.21.15.86:0/4068947440 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564afbc6cab0 con 0x564afbbf1a00 2024-04-21T08:50:46.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.353+0000 7f31957fa640 1 -- 172.21.15.86:0/4068947440 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f31a4057030 con 0x564afbbf1a00 2024-04-21T08:50:46.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.354+0000 7f31957fa640 1 -- 172.21.15.86:0/4068947440 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f31a403f040 con 0x564afbbf1a00 2024-04-21T08:50:46.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.354+0000 7f31b6a35740 10 client.20364 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:46.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.354+0000 7f31b6a35740 10 client.20364 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:50:46.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.354+0000 7f31b6a35740 20 client.20364 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:46.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.354+0000 7f31b6a35740 1 --2- 172.21.15.86:0/4068947440 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x564afbc6cd30 0x564afbc6f120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.354+0000 7f31b6a35740 4 client.20364 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:50:46.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.354+0000 7f31b6a35740 1 -- 172.21.15.86:0/4068947440 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564afbb5ac10 con 0x564afbc6cd30 2024-04-21T08:50:46.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.354+0000 7f31b1b09640 1 --2- 172.21.15.86:0/4068947440 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x564afbc6cd30 0x564afbc6f120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.354+0000 7f31957fa640 10 client.20364 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:50:46.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.354+0000 7f31b1b09640 1 --2- 172.21.15.86:0/4068947440 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x564afbc6cd30 0x564afbc6f120 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=42e148549d24362 server_cookie=731e8fb16a5f1c23 in_seq=0 out_seq=0 2024-04-21T08:50:46.361 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.354+0000 7f31957fa640 1 -- 172.21.15.86:0/4068947440 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564afbb5ac10 con 0x564afbc6cd30 2024-04-21T08:50:46.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.354+0000 7f31957fa640 10 client.20364 handle_command_reply: tid=0 2024-04-21T08:50:46.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.354+0000 7f31b6a35740 1 client.20364 shutdown 2024-04-21T08:50:46.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.354+0000 7f31b6a35740 1 -- 172.21.15.86:0/4068947440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564afbbf1a00 msgr2=0x564afbbebf80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.354+0000 7f31b6a35740 1 --2- 172.21.15.86:0/4068947440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564afbbf1a00 0x564afbbebf80 secure :-1 s=READY pgs=2045 cs=0 l=1 rev1=1 crypto rx=0x7f31a4009600 tx=0x7f31a4009c60 comp rx=0 tx=0).stop 2024-04-21T08:50:46.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.355+0000 7f31b6a35740 1 -- 172.21.15.86:0/4068947440 shutdown_connections 2024-04-21T08:50:46.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.355+0000 7f31b6a35740 1 --2- 172.21.15.86:0/4068947440 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x564afbc6cd30 0x564afbc6f120 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.355+0000 7f31b6a35740 1 --2- 172.21.15.86:0/4068947440 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564afbbf1a00 0x564afbbebf80 unknown :-1 s=CLOSED pgs=2045 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.355+0000 7f31b6a35740 1 --2- 172.21.15.86:0/4068947440 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564afbbe8e80 0x564afbbeba40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.355+0000 7f31957fa640 0 client.20364 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:50:46.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.355+0000 7f31b6a35740 1 --2- 172.21.15.86:0/4068947440 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564afbbe84d0 0x564afbbeb500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.355+0000 7f31b6a35740 1 -- 172.21.15.86:0/4068947440 >> 172.21.15.86:0/4068947440 conn(0x564afbbe2ba0 msgr2=0x564afbbe4710 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:46.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.355+0000 7f31b6a35740 1 -- 172.21.15.86:0/4068947440 shutdown_connections 2024-04-21T08:50:46.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.355+0000 7f31b6a35740 1 -- 172.21.15.86:0/4068947440 wait complete. 2024-04-21T08:50:46.362 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.355+0000 7f31b6a35740 20 client.20364 trim_cache size 0 max 16384 2024-04-21T08:50:46.363 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:50:46.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.355+0000 7f31b3d6d640 1 -- 172.21.15.86:0/1078032486 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f31980a3d20 msgr2=0x7f31980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.355+0000 7f31b3d6d640 1 --2- 172.21.15.86:0/1078032486 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f31980a3d20 0x7f31980a61e0 secure :-1 s=READY pgs=341 cs=0 l=1 rev1=1 crypto rx=0x7f31a404c000 tx=0x7f31a4045000 comp rx=0 tx=0).stop 2024-04-21T08:50:46.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.355+0000 7f31b3d6d640 1 -- 172.21.15.86:0/1078032486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31ac101b40 msgr2=0x7f31ac199ac0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.355+0000 7f31b3d6d640 1 --2- 172.21.15.86:0/1078032486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31ac101b40 0x7f31ac199ac0 secure :-1 s=READY pgs=2041 cs=0 l=1 rev1=1 crypto rx=0x7f31a000d950 tx=0x7f31a000de10 comp rx=0 tx=0).stop 2024-04-21T08:50:46.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.355+0000 7f31b3d6d640 1 -- 172.21.15.86:0/1078032486 shutdown_connections 2024-04-21T08:50:46.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.355+0000 7f31b3d6d640 1 --2- 172.21.15.86:0/1078032486 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f31980a3d20 0x7f31980a61e0 unknown :-1 s=CLOSED pgs=341 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.355+0000 7f31b3d6d640 1 --2- 172.21.15.86:0/1078032486 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f31ac107030 0x7f31ac1a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.355+0000 7f31b3d6d640 1 --2- 172.21.15.86:0/1078032486 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f31ac101b40 0x7f31ac199ac0 unknown :-1 s=CLOSED pgs=2041 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.355+0000 7f31b3d6d640 1 --2- 172.21.15.86:0/1078032486 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f31ac1011a0 0x7f31ac199580 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.355+0000 7f31b3d6d640 1 -- 172.21.15.86:0/1078032486 >> 172.21.15.86:0/1078032486 conn(0x7f31ac076c50 msgr2=0x7f31ac0fff00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:46.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.355+0000 7f31b3d6d640 1 -- 172.21.15.86:0/1078032486 shutdown_connections 2024-04-21T08:50:46.364 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.355+0000 7f31b3d6d640 1 -- 172.21.15.86:0/1078032486 wait complete. 2024-04-21T08:50:46.366 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:50:46.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.466+0000 7f25648b3640 1 -- 172.21.15.86:0/441889006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2560107030 msgr2=0x7f2560107410 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.466+0000 7f25648b3640 1 --2- 172.21.15.86:0/441889006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2560107030 0x7f2560107410 secure :-1 s=READY pgs=2046 cs=0 l=1 rev1=1 crypto rx=0x7f2558009930 tx=0x7f255802bc30 comp rx=0 tx=0).stop 2024-04-21T08:50:46.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.466+0000 7f25648b3640 1 -- 172.21.15.86:0/441889006 shutdown_connections 2024-04-21T08:50:46.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.466+0000 7f25648b3640 1 --2- 172.21.15.86:0/441889006 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2560101b40 0x7f256010dc80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.466+0000 7f25648b3640 1 --2- 172.21.15.86:0/441889006 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25601011a0 0x7f2560101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.466+0000 7f25648b3640 1 --2- 172.21.15.86:0/441889006 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2560107030 0x7f2560107410 unknown :-1 s=CLOSED pgs=2046 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.466+0000 7f25648b3640 1 -- 172.21.15.86:0/441889006 >> 172.21.15.86:0/441889006 conn(0x7f2560076c50 msgr2=0x7f2560077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:46.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.466+0000 7f25648b3640 1 -- 172.21.15.86:0/441889006 shutdown_connections 2024-04-21T08:50:46.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.466+0000 7f25648b3640 1 -- 172.21.15.86:0/441889006 wait complete. 2024-04-21T08:50:46.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.466+0000 7f25648b3640 1 Processor -- start 2024-04-21T08:50:46.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.467+0000 7f25648b3640 1 -- start start 2024-04-21T08:50:46.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.467+0000 7f25648b3640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25601011a0 0x7f2560197680 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.467+0000 7f25648b3640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2560101b40 0x7f2560197bc0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.467+0000 7f25648b3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2560107030 0x7f256019bf40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.467+0000 7f25648b3640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f25601110e0 con 0x7f2560107030 2024-04-21T08:50:46.469 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.467+0000 7f25648b3640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2560110f60 con 0x7f2560101b40 2024-04-21T08:50:46.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.467+0000 7f25648b3640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2560111260 con 0x7f25601011a0 2024-04-21T08:50:46.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.467+0000 7f255ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2560107030 0x7f256019bf40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.467+0000 7f255ed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2560107030 0x7f256019bf40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45982/0 (socket says 172.21.15.86:45982) 2024-04-21T08:50:46.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.467+0000 7f255ed9d640 1 -- 172.21.15.86:0/2666879053 learned_addr learned my addr 172.21.15.86:0/2666879053 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:46.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.467+0000 7f255dd9b640 1 --2- 172.21.15.86:0/2666879053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2560101b40 0x7f2560197bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.467+0000 7f255e59c640 1 --2- 172.21.15.86:0/2666879053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25601011a0 0x7f2560197680 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.467+0000 7f255ed9d640 1 -- 172.21.15.86:0/2666879053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25601011a0 msgr2=0x7f2560197680 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.467+0000 7f255ed9d640 1 --2- 172.21.15.86:0/2666879053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25601011a0 0x7f2560197680 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.467+0000 7f255ed9d640 1 -- 172.21.15.86:0/2666879053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2560101b40 msgr2=0x7f2560197bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.467+0000 7f255ed9d640 1 --2- 172.21.15.86:0/2666879053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2560101b40 0x7f2560197bc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.467+0000 7f255ed9d640 1 -- 172.21.15.86:0/2666879053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f256019c670 con 0x7f2560107030 2024-04-21T08:50:46.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.467+0000 7f255e59c640 1 --2- 172.21.15.86:0/2666879053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25601011a0 0x7f2560197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:46.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.467+0000 7f255ed9d640 1 --2- 172.21.15.86:0/2666879053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2560107030 0x7f256019bf40 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f255000cad0 tx=0x7f255000cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:46.470 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.468+0000 7f254f7fe640 1 -- 172.21.15.86:0/2666879053 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255000ae20 con 0x7f2560107030 2024-04-21T08:50:46.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.468+0000 7f254f7fe640 1 -- 172.21.15.86:0/2666879053 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2550005020 con 0x7f2560107030 2024-04-21T08:50:46.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.468+0000 7f25648b3640 1 -- 172.21.15.86:0/2666879053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f256019c960 con 0x7f2560107030 2024-04-21T08:50:46.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.468+0000 7f254f7fe640 1 -- 172.21.15.86:0/2666879053 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255000f6b0 con 0x7f2560107030 2024-04-21T08:50:46.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.468+0000 7f25648b3640 1 -- 172.21.15.86:0/2666879053 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f25601a41f0 con 0x7f2560107030 2024-04-21T08:50:46.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.470+0000 7f254f7fe640 1 -- 172.21.15.86:0/2666879053 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f25500051d0 con 0x7f2560107030 2024-04-21T08:50:46.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.470+0000 7f256757f740 1 Processor -- start 2024-04-21T08:50:46.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.471+0000 7f256757f740 1 -- start start 2024-04-21T08:50:46.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.471+0000 7f256757f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560e4d0ff6e0 0x560e4d0ffac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.471+0000 7f256757f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e4d100090 0x560e4d108640 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.471+0000 7f256757f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e4d108c10 0x560e4d10b060 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.471+0000 7f256757f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560e4d0621f0 con 0x560e4d100090 2024-04-21T08:50:46.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.471+0000 7f256757f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x560e4cfff4b0 con 0x560e4d0ff6e0 2024-04-21T08:50:46.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.471+0000 7f256757f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x560e4d05e1b0 con 0x560e4d108c10 2024-04-21T08:50:46.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.471+0000 7f255e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e4d100090 0x560e4d108640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.471+0000 7f255e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e4d100090 0x560e4d108640 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45984/0 (socket says 172.21.15.86:45984) 2024-04-21T08:50:46.478 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.471+0000 7f255e59c640 1 -- 172.21.15.86:0/191412570 learned_addr learned my addr 172.21.15.86:0/191412570 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:46.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.471+0000 7f255dd9b640 1 --2- 172.21.15.86:0/191412570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560e4d0ff6e0 0x560e4d0ffac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.471+0000 7f254f7fe640 1 --2- 172.21.15.86:0/2666879053 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f25480a3d20 0x7f25480a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.471+0000 7f255dd9b640 1 --2- 172.21.15.86:0/191412570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e4d108c10 0x560e4d10b060 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.471+0000 7f255e59c640 1 -- 172.21.15.86:0/191412570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e4d108c10 msgr2=0x560e4d10b060 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.471+0000 7f255e59c640 1 --2- 172.21.15.86:0/191412570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e4d108c10 0x560e4d10b060 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.471+0000 7f255e59c640 1 -- 172.21.15.86:0/191412570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560e4d0ff6e0 msgr2=0x560e4d0ffac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.471+0000 7f255e59c640 1 --2- 172.21.15.86:0/191412570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560e4d0ff6e0 0x560e4d0ffac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.471+0000 7f255e59c640 1 -- 172.21.15.86:0/191412570 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e4cff11b0 con 0x560e4d100090 2024-04-21T08:50:46.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.471+0000 7f255dd9b640 1 --2- 172.21.15.86:0/191412570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e4d108c10 0x560e4d10b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:46.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.471+0000 7f255ed9d640 1 --2- 172.21.15.86:0/2666879053 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f25480a3d20 0x7f25480a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.471+0000 7f254f7fe640 1 -- 172.21.15.86:0/2666879053 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f25500d0c80 con 0x7f2560107030 2024-04-21T08:50:46.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.472+0000 7f255e59c640 1 --2- 172.21.15.86:0/191412570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e4d100090 0x560e4d108640 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f2558000c00 tx=0x7f2558005e60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=27dce3eb8e7c10c3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:46.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.472+0000 7f254cff9640 1 -- 172.21.15.86:0/191412570 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2558044020 con 0x560e4d100090 2024-04-21T08:50:46.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.472+0000 7f254cff9640 1 -- 172.21.15.86:0/191412570 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f255802cc90 con 0x560e4d100090 2024-04-21T08:50:46.479 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.472+0000 7f254cff9640 1 -- 172.21.15.86:0/191412570 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f25580048c0 con 0x560e4d100090 2024-04-21T08:50:46.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.472+0000 7f256757f740 1 -- 172.21.15.86:0/191412570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e4d100090 msgr2=0x560e4d108640 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.472+0000 7f256757f740 1 --2- 172.21.15.86:0/191412570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e4d100090 0x560e4d108640 secure :-1 s=READY pgs=2048 cs=0 l=1 rev1=1 crypto rx=0x7f2558000c00 tx=0x7f2558005e60 comp rx=0 tx=0).stop 2024-04-21T08:50:46.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.472+0000 7f255ed9d640 1 --2- 172.21.15.86:0/2666879053 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f25480a3d20 0x7f25480a61e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f25500ce0b0 tx=0x7f25500673a0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:46.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.472+0000 7f256757f740 1 -- 172.21.15.86:0/191412570 shutdown_connections 2024-04-21T08:50:46.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.472+0000 7f256757f740 1 --2- 172.21.15.86:0/191412570 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e4d108c10 0x560e4d10b060 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.472+0000 7f256757f740 1 --2- 172.21.15.86:0/191412570 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e4d100090 0x560e4d108640 unknown :-1 s=CLOSED pgs=2048 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.472+0000 7f256757f740 1 --2- 172.21.15.86:0/191412570 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560e4d0ff6e0 0x560e4d0ffac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.472+0000 7f256757f740 1 -- 172.21.15.86:0/191412570 >> 172.21.15.86:0/191412570 conn(0x560e4d0f9db0 msgr2=0x560e4d0fa190 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:46.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.473+0000 7f256757f740 1 -- 172.21.15.86:0/191412570 shutdown_connections 2024-04-21T08:50:46.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.473+0000 7f256757f740 1 -- 172.21.15.86:0/191412570 wait complete. 2024-04-21T08:50:46.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.473+0000 7f256757f740 1 -- start start 2024-04-21T08:50:46.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.473+0000 7f256757f740 1 Processor -- start 2024-04-21T08:50:46.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.473+0000 7f256757f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e4d0ff6e0 0x560e4d108470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.473+0000 7f256757f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560e4d100090 0x560e4d105800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.473+0000 7f256757f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e4d108c10 0x560e4d105d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.473+0000 7f256757f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x560e4d10bb20 con 0x560e4d0ff6e0 2024-04-21T08:50:46.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.473+0000 7f256757f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x560e4d01b500 con 0x560e4d100090 2024-04-21T08:50:46.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.473+0000 7f256757f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x560e4d10bca0 con 0x560e4d108c10 2024-04-21T08:50:46.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.473+0000 7f256757f740 10 client.0 _ll_register_callbacks cb 0x560e4cfe9b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:46.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.473+0000 7f255dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e4d0ff6e0 0x560e4d108470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.473+0000 7f255dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e4d0ff6e0 0x560e4d108470 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45992/0 (socket says 172.21.15.86:45992) 2024-04-21T08:50:46.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.473+0000 7f255dd9b640 1 -- 172.21.15.86:0/104270735 learned_addr learned my addr 172.21.15.86:0/104270735 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:46.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.473+0000 7f255e59c640 1 --2- 172.21.15.86:0/104270735 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560e4d100090 0x560e4d105800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.474+0000 7f255dd9b640 1 --2- 172.21.15.86:0/104270735 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e4d108c10 0x560e4d105d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.474+0000 7f255e59c640 1 -- 172.21.15.86:0/104270735 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e4d108c10 msgr2=0x560e4d105d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.474+0000 7f255e59c640 1 --2- 172.21.15.86:0/104270735 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e4d108c10 0x560e4d105d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.474+0000 7f255e59c640 1 -- 172.21.15.86:0/104270735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e4d0ff6e0 msgr2=0x560e4d108470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.474+0000 7f255e59c640 1 --2- 172.21.15.86:0/104270735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e4d0ff6e0 0x560e4d108470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.474+0000 7f255e59c640 1 -- 172.21.15.86:0/104270735 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e4cd743f0 con 0x560e4d100090 2024-04-21T08:50:46.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.474+0000 7f255dd9b640 1 --2- 172.21.15.86:0/104270735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e4d0ff6e0 0x560e4d108470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_done state changed! 2024-04-21T08:50:46.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.474+0000 7f255e59c640 1 --2- 172.21.15.86:0/104270735 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560e4d100090 0x560e4d105800 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x560e4d0fae20 tx=0x7f255802ca90 comp rx=0 tx=0).ready entity=mon.2 client_cookie=fceeea155edc804c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:46.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.474+0000 7f2545ffb640 10 client.0 ms_handle_connect on v2:172.21.15.88:3300/0 2024-04-21T08:50:46.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.474+0000 7f2545ffb640 1 -- 172.21.15.86:0/104270735 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255804f070 con 0x560e4d100090 2024-04-21T08:50:46.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.474+0000 7f256757f740 1 -- 172.21.15.86:0/104270735 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f255802cc90 con 0x560e4d100090 2024-04-21T08:50:46.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.474+0000 7f2545ffb640 1 -- 172.21.15.86:0/104270735 <== mon.2 v2:172.21.15.88:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f2558003a00 con 0x560e4d100090 2024-04-21T08:50:46.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.474+0000 7f2545ffb640 1 -- 172.21.15.86:0/104270735 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f255805b050 con 0x560e4d100090 2024-04-21T08:50:46.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.474+0000 7f256757f740 10 client.28439 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:46.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.474+0000 7f256757f740 1 -- 172.21.15.86:0/104270735 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560e4ce3e7d0 con 0x560e4d100090 2024-04-21T08:50:46.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.474+0000 7f2545ffb640 1 -- 172.21.15.86:0/104270735 <== mon.2 v2:172.21.15.88:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f255802cc90 con 0x560e4d100090 2024-04-21T08:50:46.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.475+0000 7f2545ffb640 1 -- 172.21.15.86:0/104270735 <== mon.2 v2:172.21.15.88:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2558040ab0 con 0x560e4d100090 2024-04-21T08:50:46.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.475+0000 7f256757f740 10 client.28439 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:46.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.475+0000 7f256757f740 10 client.28439 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:50:46.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.475+0000 7f256757f740 20 client.28439 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:46.482 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.475+0000 7f256757f740 1 --2- 172.21.15.86:0/104270735 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x560e4d118b20 0x560e4d11af10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.475+0000 7f256757f740 4 client.28439 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:46.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.475+0000 7f256757f740 1 -- 172.21.15.86:0/104270735 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x560e4d001930 con 0x560e4d118b20 2024-04-21T08:50:46.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.475+0000 7f255dd9b640 1 --2- 172.21.15.86:0/104270735 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x560e4d118b20 0x560e4d11af10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.475+0000 7f2545ffb640 10 client.28439 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:50:46.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.475+0000 7f255dd9b640 1 --2- 172.21.15.86:0/104270735 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x560e4d118b20 0x560e4d11af10 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=5852455fa1498ef2 server_cookie=f5b0d80083ab867 in_seq=0 out_seq=0 2024-04-21T08:50:46.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.476+0000 7f2545ffb640 1 -- 172.21.15.86:0/104270735 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x560e4d001930 con 0x560e4d118b20 2024-04-21T08:50:46.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.476+0000 7f2545ffb640 10 client.28439 handle_command_reply: tid=0 2024-04-21T08:50:46.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.476+0000 7f256757f740 1 client.28439 shutdown 2024-04-21T08:50:46.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.476+0000 7f256757f740 1 -- 172.21.15.86:0/104270735 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560e4d100090 msgr2=0x560e4d105800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.476+0000 7f256757f740 1 --2- 172.21.15.86:0/104270735 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560e4d100090 0x560e4d105800 secure :-1 s=READY pgs=85 cs=0 l=1 rev1=1 crypto rx=0x560e4d0fae20 tx=0x7f255802ca90 comp rx=0 tx=0).stop 2024-04-21T08:50:46.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.476+0000 7f256757f740 1 -- 172.21.15.86:0/104270735 shutdown_connections 2024-04-21T08:50:46.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.476+0000 7f256757f740 1 --2- 172.21.15.86:0/104270735 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x560e4d118b20 0x560e4d11af10 crc :-1 s=READY pgs=77 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.476+0000 7f256757f740 1 --2- 172.21.15.86:0/104270735 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e4d108c10 0x560e4d105d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.476+0000 7f2545ffb640 0 client.28439 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:50:46.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.476+0000 7f256757f740 1 --2- 172.21.15.86:0/104270735 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560e4d100090 0x560e4d105800 unknown :-1 s=CLOSED pgs=85 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.476+0000 7f256757f740 1 --2- 172.21.15.86:0/104270735 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e4d0ff6e0 0x560e4d108470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.476+0000 7f256757f740 1 -- 172.21.15.86:0/104270735 >> 172.21.15.86:0/104270735 conn(0x560e4d0f9db0 msgr2=0x560e4d0fbb00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:46.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.476+0000 7f256757f740 1 -- 172.21.15.86:0/104270735 shutdown_connections 2024-04-21T08:50:46.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.476+0000 7f256757f740 1 -- 172.21.15.86:0/104270735 wait complete. 2024-04-21T08:50:46.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.476+0000 7f256757f740 20 client.28439 trim_cache size 0 max 16384 2024-04-21T08:50:46.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.489+0000 7f256757f740 1 Processor -- start 2024-04-21T08:50:46.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.489+0000 7f256757f740 1 -- start start 2024-04-21T08:50:46.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.489+0000 7f256757f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560e4d0ff6e0 0x560e4d1149e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.489+0000 7f256757f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e4d100090 0x560e4d107f30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.490+0000 7f256757f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e4d108c10 0x560e4d108470 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.490+0000 7f256757f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f25601ad1f0 con 0x560e4d100090 2024-04-21T08:50:46.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.490+0000 7f256757f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f25601aed30 con 0x560e4d0ff6e0 2024-04-21T08:50:46.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.490+0000 7f256757f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f25601ad8c0 con 0x560e4d108c10 2024-04-21T08:50:46.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.490+0000 7f255e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e4d100090 0x560e4d107f30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.490+0000 7f255e59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e4d100090 0x560e4d107f30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:45996/0 (socket says 172.21.15.86:45996) 2024-04-21T08:50:46.495 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.490+0000 7f255e59c640 1 -- 172.21.15.86:0/1361464360 learned_addr learned my addr 172.21.15.86:0/1361464360 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:46.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.490+0000 7f255e59c640 1 -- 172.21.15.86:0/1361464360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e4d108c10 msgr2=0x560e4d108470 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.490+0000 7f255dd9b640 1 --2- 172.21.15.86:0/1361464360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e4d108c10 0x560e4d108470 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.490+0000 7f255e59c640 1 --2- 172.21.15.86:0/1361464360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e4d108c10 0x560e4d108470 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.490+0000 7f255e59c640 1 -- 172.21.15.86:0/1361464360 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560e4d0ff6e0 msgr2=0x560e4d1149e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.490+0000 7f255e59c640 1 --2- 172.21.15.86:0/1361464360 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560e4d0ff6e0 0x560e4d1149e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.490+0000 7f255e59c640 1 -- 172.21.15.86:0/1361464360 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560e4d0ffe30 con 0x560e4d100090 2024-04-21T08:50:46.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.490+0000 7f255dd9b640 1 --2- 172.21.15.86:0/1361464360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e4d108c10 0x560e4d108470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:46.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.490+0000 7f255e59c640 1 --2- 172.21.15.86:0/1361464360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e4d100090 0x560e4d107f30 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f25580054d0 tx=0x7f2558000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c323563c7d5b3ba4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:46.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.490+0000 7f2547fff640 1 -- 172.21.15.86:0/1361464360 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255804f040 con 0x560e4d100090 2024-04-21T08:50:46.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.490+0000 7f2547fff640 1 -- 172.21.15.86:0/1361464360 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25580057f0 con 0x560e4d100090 2024-04-21T08:50:46.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.490+0000 7f2547fff640 1 -- 172.21.15.86:0/1361464360 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2558044040 con 0x560e4d100090 2024-04-21T08:50:46.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.491+0000 7f256757f740 1 -- 172.21.15.86:0/1361464360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e4d100090 msgr2=0x560e4d107f30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.491+0000 7f256757f740 1 --2- 172.21.15.86:0/1361464360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e4d100090 0x560e4d107f30 secure :-1 s=READY pgs=2049 cs=0 l=1 rev1=1 crypto rx=0x7f25580054d0 tx=0x7f2558000bc0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.491+0000 7f256757f740 1 -- 172.21.15.86:0/1361464360 shutdown_connections 2024-04-21T08:50:46.496 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.491+0000 7f256757f740 1 --2- 172.21.15.86:0/1361464360 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e4d108c10 0x560e4d108470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.491+0000 7f256757f740 1 --2- 172.21.15.86:0/1361464360 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e4d100090 0x560e4d107f30 unknown :-1 s=CLOSED pgs=2049 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.491+0000 7f256757f740 1 --2- 172.21.15.86:0/1361464360 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560e4d0ff6e0 0x560e4d1149e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.491+0000 7f256757f740 1 -- 172.21.15.86:0/1361464360 >> 172.21.15.86:0/1361464360 conn(0x560e4d0f9db0 msgr2=0x560e4d0f6c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:46.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.491+0000 7f256757f740 1 -- 172.21.15.86:0/1361464360 shutdown_connections 2024-04-21T08:50:46.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.491+0000 7f256757f740 1 -- 172.21.15.86:0/1361464360 wait complete. 2024-04-21T08:50:46.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.491+0000 7f256757f740 1 -- start start 2024-04-21T08:50:46.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.491+0000 7f256757f740 1 Processor -- start 2024-04-21T08:50:46.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.491+0000 7f256757f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e4d0ff6e0 0x560e4d111200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.491+0000 7f256757f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e4d100090 0x560e4d10c570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.491+0000 7f256757f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560e4d108c10 0x560e4d10cab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.491+0000 7f256757f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f25601ae660 con 0x560e4d0ff6e0 2024-04-21T08:50:46.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.491+0000 7f256757f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f25601adf90 con 0x560e4d108c10 2024-04-21T08:50:46.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.491+0000 7f256757f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f25601ab1f0 con 0x560e4d100090 2024-04-21T08:50:46.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.491+0000 7f256757f740 10 client.0 _ll_register_callbacks cb 0x7f2558004570 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:46.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.491+0000 7f255dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e4d0ff6e0 0x560e4d111200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.497 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.491+0000 7f255dd9b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e4d0ff6e0 0x560e4d111200 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46010/0 (socket says 172.21.15.86:46010) 2024-04-21T08:50:46.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.492+0000 7f255dd9b640 1 -- 172.21.15.86:0/4045094834 learned_addr learned my addr 172.21.15.86:0/4045094834 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:46.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.492+0000 7f255e59c640 1 --2- 172.21.15.86:0/4045094834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e4d100090 0x560e4d10c570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.492+0000 7f255dd9b640 1 -- 172.21.15.86:0/4045094834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e4d100090 msgr2=0x560e4d10c570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.492+0000 7f255dd9b640 1 --2- 172.21.15.86:0/4045094834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e4d100090 0x560e4d10c570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.492+0000 7f255dd9b640 1 -- 172.21.15.86:0/4045094834 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560e4d108c10 msgr2=0x560e4d10cab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.492+0000 7f255dd9b640 1 --2- 172.21.15.86:0/4045094834 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560e4d108c10 0x560e4d10cab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.492+0000 7f255dd9b640 1 -- 172.21.15.86:0/4045094834 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x560e4d10d2b0 con 0x560e4d0ff6e0 2024-04-21T08:50:46.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.492+0000 7f255dd9b640 1 --2- 172.21.15.86:0/4045094834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e4d0ff6e0 0x560e4d111200 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f255400b3f0 tx=0x7f25540039c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7eb099ebfdd20c6a server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:46.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.492+0000 7f2545ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:46.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.492+0000 7f2545ffb640 1 -- 172.21.15.86:0/4045094834 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f255401b040 con 0x560e4d0ff6e0 2024-04-21T08:50:46.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.492+0000 7f2545ffb640 1 -- 172.21.15.86:0/4045094834 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f2554005020 con 0x560e4d0ff6e0 2024-04-21T08:50:46.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.492+0000 7f256757f740 1 -- 172.21.15.86:0/4045094834 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f25580057f0 con 0x560e4d0ff6e0 2024-04-21T08:50:46.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.492+0000 7f2545ffb640 1 -- 172.21.15.86:0/4045094834 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f2554031050 con 0x560e4d0ff6e0 2024-04-21T08:50:46.498 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.492+0000 7f256757f740 10 client.20382 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:46.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.492+0000 7f256757f740 1 -- 172.21.15.86:0/4045094834 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560e4d17ecd0 con 0x560e4d0ff6e0 2024-04-21T08:50:46.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.492+0000 7f2545ffb640 1 -- 172.21.15.86:0/4045094834 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f25580057f0 con 0x560e4d0ff6e0 2024-04-21T08:50:46.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.492+0000 7f2545ffb640 1 -- 172.21.15.86:0/4045094834 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f2554022070 con 0x560e4d0ff6e0 2024-04-21T08:50:46.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.493+0000 7f256757f740 10 client.20382 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:46.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.493+0000 7f256757f740 10 client.20382 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:50:46.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.493+0000 7f256757f740 20 client.20382 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:46.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.493+0000 7f256757f740 1 --2- 172.21.15.86:0/4045094834 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x560e4d17f400 0x560e4d181820 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.493+0000 7f256757f740 4 client.20382 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:50:46.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.493+0000 7f256757f740 1 -- 172.21.15.86:0/4045094834 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x560e4d06fc10 con 0x560e4d17f400 2024-04-21T08:50:46.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.493+0000 7f255e59c640 1 --2- 172.21.15.86:0/4045094834 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x560e4d17f400 0x560e4d181820 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.493+0000 7f255e59c640 1 --2- 172.21.15.86:0/4045094834 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x560e4d17f400 0x560e4d181820 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=e55258346cf41a0a server_cookie=d3b69af024150307 in_seq=0 out_seq=0 2024-04-21T08:50:46.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.493+0000 7f2545ffb640 10 client.20382 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:50:46.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.493+0000 7f2545ffb640 1 -- 172.21.15.86:0/4045094834 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x560e4d06fc10 con 0x560e4d17f400 2024-04-21T08:50:46.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.493+0000 7f2545ffb640 10 client.20382 handle_command_reply: tid=0 2024-04-21T08:50:46.499 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.493+0000 7f256757f740 1 client.20382 shutdown 2024-04-21T08:50:46.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.493+0000 7f256757f740 1 -- 172.21.15.86:0/4045094834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e4d0ff6e0 msgr2=0x560e4d111200 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.493+0000 7f256757f740 1 --2- 172.21.15.86:0/4045094834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e4d0ff6e0 0x560e4d111200 secure :-1 s=READY pgs=2050 cs=0 l=1 rev1=1 crypto rx=0x7f255400b3f0 tx=0x7f25540039c0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.494+0000 7f256757f740 1 -- 172.21.15.86:0/4045094834 shutdown_connections 2024-04-21T08:50:46.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.494+0000 7f256757f740 1 --2- 172.21.15.86:0/4045094834 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x560e4d17f400 0x560e4d181820 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.494+0000 7f256757f740 1 --2- 172.21.15.86:0/4045094834 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560e4d108c10 0x560e4d10cab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.494+0000 7f256757f740 1 --2- 172.21.15.86:0/4045094834 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560e4d100090 0x560e4d10c570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.494+0000 7f256757f740 1 --2- 172.21.15.86:0/4045094834 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560e4d0ff6e0 0x560e4d111200 unknown :-1 s=CLOSED pgs=2050 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.494+0000 7f2545ffb640 0 client.20382 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:50:46.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.494+0000 7f256757f740 1 -- 172.21.15.86:0/4045094834 >> 172.21.15.86:0/4045094834 conn(0x560e4d0f9db0 msgr2=0x560e4d0fb180 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:46.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.494+0000 7f256757f740 1 -- 172.21.15.86:0/4045094834 shutdown_connections 2024-04-21T08:50:46.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.494+0000 7f256757f740 1 -- 172.21.15.86:0/4045094834 wait complete. 2024-04-21T08:50:46.500 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.494+0000 7f256757f740 20 client.20382 trim_cache size 0 max 16384 2024-04-21T08:50:46.500 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:50:46.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.494+0000 7f25648b3640 1 -- 172.21.15.86:0/2666879053 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f25480a3d20 msgr2=0x7f25480a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.494+0000 7f25648b3640 1 --2- 172.21.15.86:0/2666879053 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f25480a3d20 0x7f25480a61e0 secure :-1 s=READY pgs=342 cs=0 l=1 rev1=1 crypto rx=0x7f25500ce0b0 tx=0x7f25500673a0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.494+0000 7f25648b3640 1 -- 172.21.15.86:0/2666879053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2560107030 msgr2=0x7f256019bf40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.494+0000 7f25648b3640 1 --2- 172.21.15.86:0/2666879053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2560107030 0x7f256019bf40 secure :-1 s=READY pgs=2047 cs=0 l=1 rev1=1 crypto rx=0x7f255000cad0 tx=0x7f255000cf90 comp rx=0 tx=0).stop 2024-04-21T08:50:46.501 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.494+0000 7f25648b3640 1 -- 172.21.15.86:0/2666879053 shutdown_connections 2024-04-21T08:50:46.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.494+0000 7f25648b3640 1 --2- 172.21.15.86:0/2666879053 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f25480a3d20 0x7f25480a61e0 unknown :-1 s=CLOSED pgs=342 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.494+0000 7f25648b3640 1 --2- 172.21.15.86:0/2666879053 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2560107030 0x7f256019bf40 unknown :-1 s=CLOSED pgs=2047 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.494+0000 7f25648b3640 1 --2- 172.21.15.86:0/2666879053 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2560101b40 0x7f2560197bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.494+0000 7f25648b3640 1 --2- 172.21.15.86:0/2666879053 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f25601011a0 0x7f2560197680 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.494+0000 7f25648b3640 1 -- 172.21.15.86:0/2666879053 >> 172.21.15.86:0/2666879053 conn(0x7f2560076c50 msgr2=0x7f25600ffff0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:46.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.494+0000 7f25648b3640 1 -- 172.21.15.86:0/2666879053 shutdown_connections 2024-04-21T08:50:46.502 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.494+0000 7f25648b3640 1 -- 172.21.15.86:0/2666879053 wait complete. 2024-04-21T08:50:46.505 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:50:46.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.605+0000 7f06e5426640 1 -- 172.21.15.86:0/3084786552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06e0102190 msgr2=0x7f06e0102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.605+0000 7f06e5426640 1 --2- 172.21.15.86:0/3084786552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06e0102190 0x7f06e0102570 secure :-1 s=READY pgs=2051 cs=0 l=1 rev1=1 crypto rx=0x7f06d0009900 tx=0x7f06d002bc30 comp rx=0 tx=0).stop 2024-04-21T08:50:46.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.605+0000 7f06e5426640 1 -- 172.21.15.86:0/3084786552 shutdown_connections 2024-04-21T08:50:46.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.605+0000 7f06e5426640 1 --2- 172.21.15.86:0/3084786552 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06e0107260 0x7f06e010ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.605+0000 7f06e5426640 1 --2- 172.21.15.86:0/3084786552 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06e0102b40 0x7f06e0106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.605+0000 7f06e5426640 1 --2- 172.21.15.86:0/3084786552 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06e0102190 0x7f06e0102570 unknown :-1 s=CLOSED pgs=2051 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.605+0000 7f06e5426640 1 -- 172.21.15.86:0/3084786552 >> 172.21.15.86:0/3084786552 conn(0x7f06e00fd990 msgr2=0x7f06e00ffdb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:46.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.605+0000 7f06e5426640 1 -- 172.21.15.86:0/3084786552 shutdown_connections 2024-04-21T08:50:46.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.605+0000 7f06e5426640 1 -- 172.21.15.86:0/3084786552 wait complete. 2024-04-21T08:50:46.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.605+0000 7f06e5426640 1 Processor -- start 2024-04-21T08:50:46.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.605+0000 7f06e5426640 1 -- start start 2024-04-21T08:50:46.608 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.606+0000 7f06e5426640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06e0102190 0x7f06e0198720 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.606+0000 7f06e5426640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06e0107260 0x7f06e0198c60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.606+0000 7f06e5426640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06e019cfe0 0x7f06e019d490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.606+0000 7f06e5426640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f06e01121c0 con 0x7f06e0102190 2024-04-21T08:50:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.606+0000 7f06e5426640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f06e0112040 con 0x7f06e019cfe0 2024-04-21T08:50:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.606+0000 7f06e5426640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f06e0112340 con 0x7f06e0107260 2024-04-21T08:50:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.606+0000 7f06deffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06e0102190 0x7f06e0198720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.606+0000 7f06deffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06e0102190 0x7f06e0198720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46020/0 (socket says 172.21.15.86:46020) 2024-04-21T08:50:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.606+0000 7f06deffd640 1 -- 172.21.15.86:0/510303037 learned_addr learned my addr 172.21.15.86:0/510303037 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.606+0000 7f06df7fe640 1 --2- 172.21.15.86:0/510303037 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06e019cfe0 0x7f06e019d490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.606+0000 7f06de7fc640 1 --2- 172.21.15.86:0/510303037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06e0107260 0x7f06e0198c60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.606+0000 7f06deffd640 1 -- 172.21.15.86:0/510303037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06e0107260 msgr2=0x7f06e0198c60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.606+0000 7f06deffd640 1 --2- 172.21.15.86:0/510303037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06e0107260 0x7f06e0198c60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.606+0000 7f06deffd640 1 -- 172.21.15.86:0/510303037 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06e019cfe0 msgr2=0x7f06e019d490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.606+0000 7f06deffd640 1 --2- 172.21.15.86:0/510303037 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06e019cfe0 0x7f06e019d490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.609 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.606+0000 7f06deffd640 1 -- 172.21.15.86:0/510303037 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f06e019dad0 con 0x7f06e0102190 2024-04-21T08:50:46.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.606+0000 7f06deffd640 1 --2- 172.21.15.86:0/510303037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06e0102190 0x7f06e0198720 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f06d0005940 tx=0x7f06d002c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:46.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.606+0000 7f06c7fff640 1 -- 172.21.15.86:0/510303037 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0043020 con 0x7f06e0102190 2024-04-21T08:50:46.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.606+0000 7f06e5426640 1 -- 172.21.15.86:0/510303037 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f06e019dd60 con 0x7f06e0102190 2024-04-21T08:50:46.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.606+0000 7f06c7fff640 1 -- 172.21.15.86:0/510303037 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06d002cc60 con 0x7f06e0102190 2024-04-21T08:50:46.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.606+0000 7f06c7fff640 1 -- 172.21.15.86:0/510303037 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d0003c30 con 0x7f06e0102190 2024-04-21T08:50:46.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.607+0000 7f06e5426640 1 -- 172.21.15.86:0/510303037 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f06e01a5580 con 0x7f06e0102190 2024-04-21T08:50:46.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.609+0000 7f06c7fff640 1 -- 172.21.15.86:0/510303037 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f06d0046020 con 0x7f06e0102190 2024-04-21T08:50:46.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.609+0000 7f06e800f740 1 Processor -- start 2024-04-21T08:50:46.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.609+0000 7f06e800f740 1 -- start start 2024-04-21T08:50:46.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.609+0000 7f06c7fff640 1 --2- 172.21.15.86:0/510303037 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f06c80a3d20 0x7f06c80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.609+0000 7f06e800f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5e18bb4d0 0x55d5e18bb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.609+0000 7f06e800f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5e18bbe80 0x55d5e18c4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.609+0000 7f06e800f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e18c4a00 0x55d5e18c6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.609+0000 7f06e800f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d5e18201f0 con 0x55d5e18bb4d0 2024-04-21T08:50:46.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.609+0000 7f06e800f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d5e17bd4b0 con 0x55d5e18bbe80 2024-04-21T08:50:46.616 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.609+0000 7f06e800f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d5e181c1b0 con 0x55d5e18c4a00 2024-04-21T08:50:46.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.609+0000 7f06deffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5e18bb4d0 0x55d5e18bb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.609+0000 7f06df7fe640 1 --2- 172.21.15.86:0/510303037 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f06c80a3d20 0x7f06c80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.609+0000 7f06de7fc640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5e18bbe80 0x55d5e18c4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.609+0000 7f06deffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5e18bb4d0 0x55d5e18bb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46036/0 (socket says 172.21.15.86:46036) 2024-04-21T08:50:46.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.609+0000 7f06deffd640 1 -- 172.21.15.86:0/1820161635 learned_addr learned my addr 172.21.15.86:0/1820161635 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:46.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.610+0000 7f06c7fff640 1 -- 172.21.15.86:0/510303037 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f06d00f2f70 con 0x7f06e0102190 2024-04-21T08:50:46.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.610+0000 7f06df7fe640 1 --2- 172.21.15.86:0/1820161635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e18c4a00 0x55d5e18c6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.610+0000 7f06de7fc640 1 -- 172.21.15.86:0/1820161635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e18c4a00 msgr2=0x55d5e18c6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.617 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.610+0000 7f06de7fc640 1 --2- 172.21.15.86:0/1820161635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e18c4a00 0x55d5e18c6e50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.610+0000 7f06de7fc640 1 -- 172.21.15.86:0/1820161635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5e18bb4d0 msgr2=0x55d5e18bb8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.610+0000 7f06de7fc640 1 --2- 172.21.15.86:0/1820161635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5e18bb4d0 0x55d5e18bb8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.610+0000 7f06de7fc640 1 -- 172.21.15.86:0/1820161635 --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5e17af1b0 con 0x55d5e18bbe80 2024-04-21T08:50:46.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.610+0000 7f06df7fe640 1 --2- 172.21.15.86:0/510303037 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f06c80a3d20 0x7f06c80a61e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f06d800aeb0 tx=0x7f06d80127c0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:46.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.610+0000 7f06de7fc640 1 --2- 172.21.15.86:0/1820161635 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5e18bbe80 0x55d5e18c4430 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f06d40058c0 tx=0x7f06d400cee0 comp rx=0 tx=0).ready entity=mon.2 client_cookie=d4dc36a0379f56ba server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:46.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.610+0000 7f06c57fa640 1 -- 172.21.15.86:0/1820161635 <== mon.2 v2:172.21.15.88:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d400ee90 con 0x55d5e18bbe80 2024-04-21T08:50:46.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.610+0000 7f06c57fa640 1 -- 172.21.15.86:0/1820161635 <== mon.2 v2:172.21.15.88:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06d40035f0 con 0x55d5e18bbe80 2024-04-21T08:50:46.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.610+0000 7f06c57fa640 1 -- 172.21.15.86:0/1820161635 <== mon.2 v2:172.21.15.88:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d4010670 con 0x55d5e18bbe80 2024-04-21T08:50:46.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.611+0000 7f06e800f740 1 -- 172.21.15.86:0/1820161635 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5e18bbe80 msgr2=0x55d5e18c4430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.611+0000 7f06e800f740 1 --2- 172.21.15.86:0/1820161635 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5e18bbe80 0x55d5e18c4430 secure :-1 s=READY pgs=86 cs=0 l=1 rev1=1 crypto rx=0x7f06d40058c0 tx=0x7f06d400cee0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.611+0000 7f06e800f740 1 -- 172.21.15.86:0/1820161635 shutdown_connections 2024-04-21T08:50:46.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.611+0000 7f06e800f740 1 --2- 172.21.15.86:0/1820161635 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e18c4a00 0x55d5e18c6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.611+0000 7f06e800f740 1 --2- 172.21.15.86:0/1820161635 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5e18bbe80 0x55d5e18c4430 unknown :-1 s=CLOSED pgs=86 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.618 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.611+0000 7f06e800f740 1 --2- 172.21.15.86:0/1820161635 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5e18bb4d0 0x55d5e18bb8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.611+0000 7f06e800f740 1 -- 172.21.15.86:0/1820161635 >> 172.21.15.86:0/1820161635 conn(0x55d5e18b5ba0 msgr2=0x55d5e18b5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.611+0000 7f06e800f740 1 -- 172.21.15.86:0/1820161635 shutdown_connections 2024-04-21T08:50:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.611+0000 7f06e800f740 1 -- 172.21.15.86:0/1820161635 wait complete. 2024-04-21T08:50:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.611+0000 7f06e800f740 1 -- start start 2024-04-21T08:50:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.611+0000 7f06e800f740 1 Processor -- start 2024-04-21T08:50:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.611+0000 7f06e800f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5e18bb4d0 0x55d5e18c4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.611+0000 7f06e800f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e18bbe80 0x55d5e18c16b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.611+0000 7f06e800f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5e18c4a00 0x55d5e18c1c50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.612+0000 7f06e800f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d5e18c7970 con 0x55d5e18bb4d0 2024-04-21T08:50:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.612+0000 7f06e800f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d5e17d9500 con 0x55d5e18c4a00 2024-04-21T08:50:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.612+0000 7f06e800f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d5e18c7af0 con 0x55d5e18bbe80 2024-04-21T08:50:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.612+0000 7f06e800f740 10 client.0 _ll_register_callbacks cb 0x55d5e17a7b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.612+0000 7f06deffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5e18bb4d0 0x55d5e18c4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.619 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.612+0000 7f06deffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5e18bb4d0 0x55d5e18c4320 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46040/0 (socket says 172.21.15.86:46040) 2024-04-21T08:50:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.612+0000 7f06deffd640 1 -- 172.21.15.86:0/322128619 learned_addr learned my addr 172.21.15.86:0/322128619 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.612+0000 7f06df7fe640 1 --2- 172.21.15.86:0/322128619 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5e18c4a00 0x55d5e18c1c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.612+0000 7f06deffd640 1 -- 172.21.15.86:0/322128619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e18bbe80 msgr2=0x55d5e18c16b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.612+0000 7f06deffd640 1 --2- 172.21.15.86:0/322128619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e18bbe80 0x55d5e18c16b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.612+0000 7f06deffd640 1 -- 172.21.15.86:0/322128619 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5e18c4a00 msgr2=0x55d5e18c1c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.612+0000 7f06deffd640 1 --2- 172.21.15.86:0/322128619 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5e18c4a00 0x55d5e18c1c50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.612+0000 7f06deffd640 1 -- 172.21.15.86:0/322128619 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5e15323f0 con 0x55d5e18bb4d0 2024-04-21T08:50:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.612+0000 7f06deffd640 1 --2- 172.21.15.86:0/322128619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5e18bb4d0 0x55d5e18c4320 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f06d00bb060 tx=0x7f06d0003760 comp rx=0 tx=0).ready entity=mon.0 client_cookie=e41cea80bc5f46c4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.612+0000 7f06c2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.612+0000 7f06c2ff5640 1 -- 172.21.15.86:0/322128619 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d00d8060 con 0x55d5e18bb4d0 2024-04-21T08:50:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.612+0000 7f06c2ff5640 1 -- 172.21.15.86:0/322128619 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f06d00bb2c0 con 0x55d5e18bb4d0 2024-04-21T08:50:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.612+0000 7f06e800f740 1 -- 172.21.15.86:0/322128619 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f06d40035f0 con 0x55d5e18bb4d0 2024-04-21T08:50:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.612+0000 7f06c2ff5640 1 -- 172.21.15.86:0/322128619 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f06d00bd400 con 0x55d5e18bb4d0 2024-04-21T08:50:46.620 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.612+0000 7f06e800f740 10 client.20394 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.612+0000 7f06e800f740 1 -- 172.21.15.86:0/322128619 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d5e15fc7d0 con 0x55d5e18bb4d0 2024-04-21T08:50:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.612+0000 7f06c2ff5640 1 -- 172.21.15.86:0/322128619 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06d40035f0 con 0x55d5e18bb4d0 2024-04-21T08:50:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.613+0000 7f06c2ff5640 1 -- 172.21.15.86:0/322128619 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f06d00c6c30 con 0x55d5e18bb4d0 2024-04-21T08:50:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.613+0000 7f06e800f740 10 client.20394 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.613+0000 7f06e800f740 10 client.20394 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:50:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.613+0000 7f06e800f740 20 client.20394 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.613+0000 7f06e800f740 1 --2- 172.21.15.86:0/322128619 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55d5e18d4cc0 0x55d5e18d7180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.613+0000 7f06e800f740 4 client.20394 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.613+0000 7f06e800f740 1 -- 172.21.15.86:0/322128619 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d5e17bf930 con 0x55d5e18d4cc0 2024-04-21T08:50:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.613+0000 7f06de7fc640 1 --2- 172.21.15.86:0/322128619 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55d5e18d4cc0 0x55d5e18d7180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.613+0000 7f06c2ff5640 10 client.20394 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:50:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.613+0000 7f06de7fc640 1 --2- 172.21.15.86:0/322128619 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55d5e18d4cc0 0x55d5e18d7180 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=e4869f86728faced server_cookie=924c8aea753888e5 in_seq=0 out_seq=0 2024-04-21T08:50:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.614+0000 7f06c2ff5640 1 -- 172.21.15.86:0/322128619 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d5e17bf930 con 0x55d5e18d4cc0 2024-04-21T08:50:46.621 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.614+0000 7f06c2ff5640 10 client.20394 handle_command_reply: tid=0 2024-04-21T08:50:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.614+0000 7f06e800f740 1 client.20394 shutdown 2024-04-21T08:50:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.614+0000 7f06e800f740 1 -- 172.21.15.86:0/322128619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5e18bb4d0 msgr2=0x55d5e18c4320 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.614+0000 7f06e800f740 1 --2- 172.21.15.86:0/322128619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5e18bb4d0 0x55d5e18c4320 secure :-1 s=READY pgs=2053 cs=0 l=1 rev1=1 crypto rx=0x7f06d00bb060 tx=0x7f06d0003760 comp rx=0 tx=0).stop 2024-04-21T08:50:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.614+0000 7f06e800f740 1 -- 172.21.15.86:0/322128619 shutdown_connections 2024-04-21T08:50:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.614+0000 7f06e800f740 1 --2- 172.21.15.86:0/322128619 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5e18c4a00 0x55d5e18c1c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.614+0000 7f06e800f740 1 --2- 172.21.15.86:0/322128619 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e18bbe80 0x55d5e18c16b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.614+0000 7f06e800f740 1 --2- 172.21.15.86:0/322128619 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55d5e18d4cc0 0x55d5e18d7180 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.614+0000 7f06e800f740 1 --2- 172.21.15.86:0/322128619 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5e18bb4d0 0x55d5e18c4320 unknown :-1 s=CLOSED pgs=2053 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.614+0000 7f06e800f740 1 -- 172.21.15.86:0/322128619 >> 172.21.15.86:0/322128619 conn(0x55d5e18b5ba0 msgr2=0x55d5e18b78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.614+0000 7f06c2ff5640 0 client.20394 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:50:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.614+0000 7f06e800f740 1 -- 172.21.15.86:0/322128619 shutdown_connections 2024-04-21T08:50:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.614+0000 7f06e800f740 1 -- 172.21.15.86:0/322128619 wait complete. 2024-04-21T08:50:46.622 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.614+0000 7f06e800f740 20 client.20394 trim_cache size 0 max 16384 2024-04-21T08:50:46.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.627+0000 7f06e800f740 1 Processor -- start 2024-04-21T08:50:46.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.627+0000 7f06e800f740 1 -- start start 2024-04-21T08:50:46.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.627+0000 7f06e800f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5e18bb4d0 0x55d5e18d0860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.628+0000 7f06e800f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e18bbe80 0x55d5e18c3de0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.628+0000 7f06e800f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5e18c4a00 0x55d5e18c4320 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.628+0000 7f06e800f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f06e01af220 con 0x55d5e18bb4d0 2024-04-21T08:50:46.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.628+0000 7f06e800f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f06e01b0d60 con 0x55d5e18c4a00 2024-04-21T08:50:46.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.628+0000 7f06e800f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f06e01af8f0 con 0x55d5e18bbe80 2024-04-21T08:50:46.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.628+0000 7f06deffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5e18bb4d0 0x55d5e18d0860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.628+0000 7f06deffd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5e18bb4d0 0x55d5e18d0860 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46044/0 (socket says 172.21.15.86:46044) 2024-04-21T08:50:46.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.628+0000 7f06deffd640 1 -- 172.21.15.86:0/316299007 learned_addr learned my addr 172.21.15.86:0/316299007 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:46.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.628+0000 7f06df7fe640 1 --2- 172.21.15.86:0/316299007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5e18c4a00 0x55d5e18c4320 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.628+0000 7f06deffd640 1 -- 172.21.15.86:0/316299007 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e18bbe80 msgr2=0x55d5e18c3de0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.628+0000 7f06deffd640 1 --2- 172.21.15.86:0/316299007 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e18bbe80 0x55d5e18c3de0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.634 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.628+0000 7f06deffd640 1 -- 172.21.15.86:0/316299007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5e18c4a00 msgr2=0x55d5e18c4320 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.628+0000 7f06deffd640 1 --2- 172.21.15.86:0/316299007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5e18c4a00 0x55d5e18c4320 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.628+0000 7f06deffd640 1 -- 172.21.15.86:0/316299007 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d5e18c0660 con 0x55d5e18bb4d0 2024-04-21T08:50:46.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.628+0000 7f06df7fe640 1 --2- 172.21.15.86:0/316299007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5e18c4a00 0x55d5e18c4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:46.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.628+0000 7f06deffd640 1 --2- 172.21.15.86:0/316299007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5e18bb4d0 0x55d5e18d0860 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f06d00bcf00 tx=0x7f06d0000bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1a9021176e98f44 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:46.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.628+0000 7f06c4ff9640 1 -- 172.21.15.86:0/316299007 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d00c7030 con 0x55d5e18bb4d0 2024-04-21T08:50:46.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.628+0000 7f06c4ff9640 1 -- 172.21.15.86:0/316299007 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f06d0005560 con 0x55d5e18bb4d0 2024-04-21T08:50:46.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.628+0000 7f06c4ff9640 1 -- 172.21.15.86:0/316299007 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d00c7220 con 0x55d5e18bb4d0 2024-04-21T08:50:46.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.629+0000 7f06e800f740 1 -- 172.21.15.86:0/316299007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5e18bb4d0 msgr2=0x55d5e18d0860 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.629+0000 7f06e800f740 1 --2- 172.21.15.86:0/316299007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5e18bb4d0 0x55d5e18d0860 secure :-1 s=READY pgs=2054 cs=0 l=1 rev1=1 crypto rx=0x7f06d00bcf00 tx=0x7f06d0000bc0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.629+0000 7f06e800f740 1 -- 172.21.15.86:0/316299007 shutdown_connections 2024-04-21T08:50:46.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.629+0000 7f06e800f740 1 --2- 172.21.15.86:0/316299007 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5e18c4a00 0x55d5e18c4320 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.629+0000 7f06e800f740 1 --2- 172.21.15.86:0/316299007 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e18bbe80 0x55d5e18c3de0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.629+0000 7f06e800f740 1 --2- 172.21.15.86:0/316299007 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5e18bb4d0 0x55d5e18d0860 unknown :-1 s=CLOSED pgs=2054 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.629+0000 7f06e800f740 1 -- 172.21.15.86:0/316299007 >> 172.21.15.86:0/316299007 conn(0x55d5e18b5ba0 msgr2=0x55d5e18bc2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:46.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.629+0000 7f06e800f740 1 -- 172.21.15.86:0/316299007 shutdown_connections 2024-04-21T08:50:46.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.629+0000 7f06e800f740 1 -- 172.21.15.86:0/316299007 wait complete. 2024-04-21T08:50:46.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.629+0000 7f06e800f740 1 -- start start 2024-04-21T08:50:46.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.629+0000 7f06e800f740 1 Processor -- start 2024-04-21T08:50:46.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.629+0000 7f06e800f740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e18bb4d0 0x55d5e18cd160 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.629+0000 7f06e800f740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5e18bbe80 0x55d5e18be110 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.629+0000 7f06e800f740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5e18c4a00 0x55d5e18be650 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.629+0000 7f06e800f740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f06e01b0690 con 0x55d5e18bbe80 2024-04-21T08:50:46.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.629+0000 7f06e800f740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f06e01affc0 con 0x55d5e18c4a00 2024-04-21T08:50:46.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.629+0000 7f06e800f740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f06e01ad220 con 0x55d5e18bb4d0 2024-04-21T08:50:46.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.629+0000 7f06e800f740 10 client.0 _ll_register_callbacks cb 0x7f06d0009560 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:46.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.629+0000 7f06de7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5e18bbe80 0x55d5e18be110 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.629+0000 7f06de7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5e18bbe80 0x55d5e18be110 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46052/0 (socket says 172.21.15.86:46052) 2024-04-21T08:50:46.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.629+0000 7f06de7fc640 1 -- 172.21.15.86:0/2874618937 learned_addr learned my addr 172.21.15.86:0/2874618937 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:46.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.630+0000 7f06deffd640 1 --2- 172.21.15.86:0/2874618937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e18bb4d0 0x55d5e18cd160 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.630+0000 7f06de7fc640 1 -- 172.21.15.86:0/2874618937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e18bb4d0 msgr2=0x55d5e18cd160 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.630+0000 7f06de7fc640 1 --2- 172.21.15.86:0/2874618937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e18bb4d0 0x55d5e18cd160 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.630+0000 7f06de7fc640 1 -- 172.21.15.86:0/2874618937 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5e18c4a00 msgr2=0x55d5e18be650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.630+0000 7f06de7fc640 1 --2- 172.21.15.86:0/2874618937 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5e18c4a00 0x55d5e18be650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.630+0000 7f06de7fc640 1 -- 172.21.15.86:0/2874618937 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d5e18bbc20 con 0x55d5e18bbe80 2024-04-21T08:50:46.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.630+0000 7f06deffd640 1 --2- 172.21.15.86:0/2874618937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e18bb4d0 0x55d5e18cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:46.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.630+0000 7f06de7fc640 1 --2- 172.21.15.86:0/2874618937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5e18bbe80 0x55d5e18be110 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f06d400c700 tx=0x7f06d4005bb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d48d96c4e517ce76 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:46.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.630+0000 7f06c2ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:46.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.630+0000 7f06c2ff5640 1 -- 172.21.15.86:0/2874618937 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f06d402c060 con 0x55d5e18bbe80 2024-04-21T08:50:46.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.630+0000 7f06c2ff5640 1 -- 172.21.15.86:0/2874618937 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f06d4003f80 con 0x55d5e18bbe80 2024-04-21T08:50:46.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.630+0000 7f06e800f740 1 -- 172.21.15.86:0/2874618937 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x55d5e182dc10 con 0x55d5e18bbe80 2024-04-21T08:50:46.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.630+0000 7f06c2ff5640 1 -- 172.21.15.86:0/2874618937 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f06d4033050 con 0x55d5e18bbe80 2024-04-21T08:50:46.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.630+0000 7f06e800f740 10 client.20400 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:46.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.630+0000 7f06e800f740 1 -- 172.21.15.86:0/2874618937 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d5e193b160 con 0x55d5e18bbe80 2024-04-21T08:50:46.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.630+0000 7f06c2ff5640 1 -- 172.21.15.86:0/2874618937 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x55d5e182dc10 con 0x55d5e18bbe80 2024-04-21T08:50:46.637 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.630+0000 7f06c2ff5640 1 -- 172.21.15.86:0/2874618937 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f06d4024070 con 0x55d5e18bbe80 2024-04-21T08:50:46.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.631+0000 7f06e800f740 10 client.20400 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:46.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.631+0000 7f06e800f740 10 client.20400 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:50:46.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.631+0000 7f06e800f740 20 client.20400 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:46.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.631+0000 7f06e800f740 1 --2- 172.21.15.86:0/2874618937 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55d5e193b6b0 0x55d5e193dad0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.631+0000 7f06e800f740 4 client.20400 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:50:46.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.631+0000 7f06e800f740 1 -- 172.21.15.86:0/2874618937 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55d5e1827a80 con 0x55d5e193b6b0 2024-04-21T08:50:46.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.631+0000 7f06deffd640 1 --2- 172.21.15.86:0/2874618937 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55d5e193b6b0 0x55d5e193dad0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.631+0000 7f06c2ff5640 10 client.20400 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:50:46.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.631+0000 7f06deffd640 1 --2- 172.21.15.86:0/2874618937 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55d5e193b6b0 0x55d5e193dad0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=c16a263b3a4294f0 server_cookie=eee702e74547149c in_seq=0 out_seq=0 2024-04-21T08:50:46.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.632+0000 7f06c2ff5640 1 -- 172.21.15.86:0/2874618937 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55d5e1827a80 con 0x55d5e193b6b0 2024-04-21T08:50:46.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.632+0000 7f06c2ff5640 10 client.20400 handle_command_reply: tid=0 2024-04-21T08:50:46.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.632+0000 7f06e800f740 1 client.20400 shutdown 2024-04-21T08:50:46.638 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.632+0000 7f06e800f740 1 -- 172.21.15.86:0/2874618937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5e18bbe80 msgr2=0x55d5e18be110 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.632+0000 7f06e800f740 1 --2- 172.21.15.86:0/2874618937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5e18bbe80 0x55d5e18be110 secure :-1 s=READY pgs=2055 cs=0 l=1 rev1=1 crypto rx=0x7f06d400c700 tx=0x7f06d4005bb0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.632+0000 7f06e800f740 1 -- 172.21.15.86:0/2874618937 shutdown_connections 2024-04-21T08:50:46.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.632+0000 7f06e800f740 1 --2- 172.21.15.86:0/2874618937 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d5e18c4a00 0x55d5e18be650 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.632+0000 7f06e800f740 1 --2- 172.21.15.86:0/2874618937 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55d5e193b6b0 0x55d5e193dad0 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.632+0000 7f06e800f740 1 --2- 172.21.15.86:0/2874618937 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d5e18bbe80 0x55d5e18be110 unknown :-1 s=CLOSED pgs=2055 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.632+0000 7f06e800f740 1 --2- 172.21.15.86:0/2874618937 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d5e18bb4d0 0x55d5e18cd160 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.632+0000 7f06c2ff5640 0 client.20400 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:50:46.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.632+0000 7f06e800f740 1 -- 172.21.15.86:0/2874618937 >> 172.21.15.86:0/2874618937 conn(0x55d5e18b5ba0 msgr2=0x55d5e18bcb40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:46.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.632+0000 7f06e800f740 1 -- 172.21.15.86:0/2874618937 shutdown_connections 2024-04-21T08:50:46.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.632+0000 7f06e800f740 1 -- 172.21.15.86:0/2874618937 wait complete. 2024-04-21T08:50:46.639 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.632+0000 7f06e800f740 20 client.20400 trim_cache size 0 max 16384 2024-04-21T08:50:46.639 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:50:46.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.632+0000 7f06e5426640 1 -- 172.21.15.86:0/510303037 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f06c80a3d20 msgr2=0x7f06c80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.632+0000 7f06e5426640 1 --2- 172.21.15.86:0/510303037 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f06c80a3d20 0x7f06c80a61e0 secure :-1 s=READY pgs=343 cs=0 l=1 rev1=1 crypto rx=0x7f06d800aeb0 tx=0x7f06d80127c0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.632+0000 7f06e5426640 1 -- 172.21.15.86:0/510303037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06e0102190 msgr2=0x7f06e0198720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.640 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.632+0000 7f06e5426640 1 --2- 172.21.15.86:0/510303037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06e0102190 0x7f06e0198720 secure :-1 s=READY pgs=2052 cs=0 l=1 rev1=1 crypto rx=0x7f06d0005940 tx=0x7f06d002c860 comp rx=0 tx=0).stop 2024-04-21T08:50:46.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.632+0000 7f06e5426640 1 -- 172.21.15.86:0/510303037 shutdown_connections 2024-04-21T08:50:46.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.632+0000 7f06e5426640 1 --2- 172.21.15.86:0/510303037 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f06c80a3d20 0x7f06c80a61e0 unknown :-1 s=CLOSED pgs=343 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.632+0000 7f06e5426640 1 --2- 172.21.15.86:0/510303037 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f06e019cfe0 0x7f06e019d490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.632+0000 7f06e5426640 1 --2- 172.21.15.86:0/510303037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f06e0107260 0x7f06e0198c60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.632+0000 7f06e5426640 1 --2- 172.21.15.86:0/510303037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f06e0102190 0x7f06e0198720 unknown :-1 s=CLOSED pgs=2052 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.632+0000 7f06e5426640 1 -- 172.21.15.86:0/510303037 >> 172.21.15.86:0/510303037 conn(0x7f06e00fd990 msgr2=0x7f06e00fecb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:46.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.633+0000 7f06e5426640 1 -- 172.21.15.86:0/510303037 shutdown_connections 2024-04-21T08:50:46.641 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.633+0000 7f06e5426640 1 -- 172.21.15.86:0/510303037 wait complete. 2024-04-21T08:50:46.643 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:50:46.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.744+0000 7ff37bceb640 1 -- 172.21.15.86:0/1568235162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff374100450 msgr2=0x7ff3741008d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.744+0000 7ff37bceb640 1 --2- 172.21.15.86:0/1568235162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff374100450 0x7ff3741008d0 secure :-1 s=READY pgs=2056 cs=0 l=1 rev1=1 crypto rx=0x7ff364009930 tx=0x7ff36402bc30 comp rx=0 tx=0).stop 2024-04-21T08:50:46.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.744+0000 7ff37bceb640 1 -- 172.21.15.86:0/1568235162 shutdown_connections 2024-04-21T08:50:46.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.744+0000 7ff37bceb640 1 --2- 172.21.15.86:0/1568235162 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff374100e10 0x7ff37410ede0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.744+0000 7ff37bceb640 1 --2- 172.21.15.86:0/1568235162 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff374100450 0x7ff3741008d0 unknown :-1 s=CLOSED pgs=2056 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.744+0000 7ff37bceb640 1 --2- 172.21.15.86:0/1568235162 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff374106f30 0x7ff374107310 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.744+0000 7ff37bceb640 1 -- 172.21.15.86:0/1568235162 >> 172.21.15.86:0/1568235162 conn(0x7ff3740773e0 msgr2=0x7ff3740fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:46.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.744+0000 7ff37bceb640 1 -- 172.21.15.86:0/1568235162 shutdown_connections 2024-04-21T08:50:46.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.744+0000 7ff37bceb640 1 -- 172.21.15.86:0/1568235162 wait complete. 2024-04-21T08:50:46.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.745+0000 7ff37bceb640 1 Processor -- start 2024-04-21T08:50:46.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.745+0000 7ff37bceb640 1 -- start start 2024-04-21T08:50:46.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.745+0000 7ff37bceb640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff374100450 0x7ff374101630 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.745+0000 7ff37bceb640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff374100e10 0x7ff374101b70 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.745+0000 7ff37bceb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff374106f30 0x7ff3741020b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.745+0000 7ff37bceb640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff3741122a0 con 0x7ff374106f30 2024-04-21T08:50:46.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.745+0000 7ff37bceb640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff374112120 con 0x7ff374100450 2024-04-21T08:50:46.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.745+0000 7ff37bceb640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff374112420 con 0x7ff374100e10 2024-04-21T08:50:46.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.745+0000 7ff37a288640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff374106f30 0x7ff3741020b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.745+0000 7ff37a288640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff374106f30 0x7ff3741020b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46074/0 (socket says 172.21.15.86:46074) 2024-04-21T08:50:46.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.745+0000 7ff37a288640 1 -- 172.21.15.86:0/2897770391 learned_addr learned my addr 172.21.15.86:0/2897770391 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:46.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.745+0000 7ff379286640 1 --2- 172.21.15.86:0/2897770391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff374100e10 0x7ff374101b70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.745+0000 7ff379a87640 1 --2- 172.21.15.86:0/2897770391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff374100450 0x7ff374101630 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.745+0000 7ff37a288640 1 -- 172.21.15.86:0/2897770391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff374100e10 msgr2=0x7ff374101b70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.745+0000 7ff37a288640 1 --2- 172.21.15.86:0/2897770391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff374100e10 0x7ff374101b70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.745+0000 7ff37a288640 1 -- 172.21.15.86:0/2897770391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff374100450 msgr2=0x7ff374101630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.745+0000 7ff37a288640 1 --2- 172.21.15.86:0/2897770391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff374100450 0x7ff374101630 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.745+0000 7ff37a288640 1 -- 172.21.15.86:0/2897770391 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7ff3741a6d40 con 0x7ff374106f30 2024-04-21T08:50:46.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.745+0000 7ff37a288640 1 --2- 172.21.15.86:0/2897770391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff374106f30 0x7ff3741020b0 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00cad0 tx=0x7ff36c00cf90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:46.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.746+0000 7ff36affd640 1 -- 172.21.15.86:0/2897770391 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36c00ae20 con 0x7ff374106f30 2024-04-21T08:50:46.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.746+0000 7ff36affd640 1 -- 172.21.15.86:0/2897770391 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff36c005020 con 0x7ff374106f30 2024-04-21T08:50:46.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.746+0000 7ff37bceb640 1 -- 172.21.15.86:0/2897770391 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7ff3741a7030 con 0x7ff374106f30 2024-04-21T08:50:46.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.746+0000 7ff36affd640 1 -- 172.21.15.86:0/2897770391 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff36c00f6b0 con 0x7ff374106f30 2024-04-21T08:50:46.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.746+0000 7ff37bceb640 1 -- 172.21.15.86:0/2897770391 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7ff3741a7500 con 0x7ff374106f30 2024-04-21T08:50:46.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.749+0000 7ff36affd640 1 -- 172.21.15.86:0/2897770391 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7ff36c0051d0 con 0x7ff374106f30 2024-04-21T08:50:46.755 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.749+0000 7ff37e9b9740 1 Processor -- start 2024-04-21T08:50:46.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.749+0000 7ff37e9b9740 1 -- start start 2024-04-21T08:50:46.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.749+0000 7ff37e9b9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563eab42c4d0 0x563eab42c8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.749+0000 7ff37e9b9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563eab42ce80 0x563eab435430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.749+0000 7ff37e9b9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563eab435a00 0x563eab437e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.749+0000 7ff37e9b9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563eab3911f0 con 0x563eab42ce80 2024-04-21T08:50:46.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.749+0000 7ff37e9b9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x563eab32e4b0 con 0x563eab42c4d0 2024-04-21T08:50:46.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.749+0000 7ff37e9b9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563eab38d1b0 con 0x563eab435a00 2024-04-21T08:50:46.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.749+0000 7ff379a87640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563eab42ce80 0x563eab435430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.749+0000 7ff379a87640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563eab42ce80 0x563eab435430 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46090/0 (socket says 172.21.15.86:46090) 2024-04-21T08:50:46.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.749+0000 7ff379a87640 1 -- 172.21.15.86:0/131651622 learned_addr learned my addr 172.21.15.86:0/131651622 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:46.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.749+0000 7ff36affd640 1 --2- 172.21.15.86:0/2897770391 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff3600a3d20 0x7ff3600a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.749+0000 7ff379286640 1 --2- 172.21.15.86:0/131651622 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563eab435a00 0x563eab437e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.749+0000 7ff379286640 1 --2- 172.21.15.86:0/131651622 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563eab42c4d0 0x563eab42c8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.749+0000 7ff37a288640 1 --2- 172.21.15.86:0/2897770391 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff3600a3d20 0x7ff3600a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.756 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.749+0000 7ff379a87640 1 -- 172.21.15.86:0/131651622 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563eab435a00 msgr2=0x563eab437e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.749+0000 7ff379a87640 1 --2- 172.21.15.86:0/131651622 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563eab435a00 0x563eab437e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.749+0000 7ff379a87640 1 -- 172.21.15.86:0/131651622 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563eab42c4d0 msgr2=0x563eab42c8b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.749+0000 7ff379a87640 1 --2- 172.21.15.86:0/131651622 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563eab42c4d0 0x563eab42c8b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.750+0000 7ff379a87640 1 -- 172.21.15.86:0/131651622 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563eab3201b0 con 0x563eab42ce80 2024-04-21T08:50:46.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.750+0000 7ff379286640 1 --2- 172.21.15.86:0/131651622 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563eab435a00 0x563eab437e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:46.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.750+0000 7ff36affd640 1 -- 172.21.15.86:0/2897770391 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7ff36c0d0bf0 con 0x7ff374106f30 2024-04-21T08:50:46.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.750+0000 7ff379a87640 1 --2- 172.21.15.86:0/131651622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563eab42ce80 0x563eab435430 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7ff37000bc60 tx=0x7ff37000f730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d6d5cd1981c4c2ba server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:46.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.750+0000 7ff35ffff640 1 -- 172.21.15.86:0/131651622 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff370015070 con 0x563eab42ce80 2024-04-21T08:50:46.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.750+0000 7ff37a288640 1 --2- 172.21.15.86:0/2897770391 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff3600a3d20 0x7ff3600a61e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff36c0ce020 tx=0x7ff36c019f90 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:46.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.750+0000 7ff35ffff640 1 -- 172.21.15.86:0/131651622 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3700118f0 con 0x563eab42ce80 2024-04-21T08:50:46.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.750+0000 7ff35ffff640 1 -- 172.21.15.86:0/131651622 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff370010790 con 0x563eab42ce80 2024-04-21T08:50:46.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.751+0000 7ff37e9b9740 1 -- 172.21.15.86:0/131651622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563eab42ce80 msgr2=0x563eab435430 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.751+0000 7ff37e9b9740 1 --2- 172.21.15.86:0/131651622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563eab42ce80 0x563eab435430 secure :-1 s=READY pgs=2058 cs=0 l=1 rev1=1 crypto rx=0x7ff37000bc60 tx=0x7ff37000f730 comp rx=0 tx=0).stop 2024-04-21T08:50:46.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.751+0000 7ff37e9b9740 1 -- 172.21.15.86:0/131651622 shutdown_connections 2024-04-21T08:50:46.757 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.751+0000 7ff37e9b9740 1 --2- 172.21.15.86:0/131651622 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563eab435a00 0x563eab437e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.751+0000 7ff37e9b9740 1 --2- 172.21.15.86:0/131651622 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563eab42ce80 0x563eab435430 unknown :-1 s=CLOSED pgs=2058 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.751+0000 7ff37e9b9740 1 --2- 172.21.15.86:0/131651622 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563eab42c4d0 0x563eab42c8b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.751+0000 7ff37e9b9740 1 -- 172.21.15.86:0/131651622 >> 172.21.15.86:0/131651622 conn(0x563eab426ba0 msgr2=0x563eab426f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:46.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.751+0000 7ff37e9b9740 1 -- 172.21.15.86:0/131651622 shutdown_connections 2024-04-21T08:50:46.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.751+0000 7ff37e9b9740 1 -- 172.21.15.86:0/131651622 wait complete. 2024-04-21T08:50:46.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.751+0000 7ff37e9b9740 1 -- start start 2024-04-21T08:50:46.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.751+0000 7ff37e9b9740 1 Processor -- start 2024-04-21T08:50:46.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.751+0000 7ff37e9b9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563eab42c4d0 0x563eab435300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.751+0000 7ff37e9b9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563eab42ce80 0x563eab432690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.752+0000 7ff37e9b9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563eab435a00 0x563eab432c10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.752+0000 7ff37e9b9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x563eab438970 con 0x563eab42c4d0 2024-04-21T08:50:46.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.752+0000 7ff37e9b9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x563eab34a500 con 0x563eab42ce80 2024-04-21T08:50:46.758 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.752+0000 7ff37e9b9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x563eab438af0 con 0x563eab435a00 2024-04-21T08:50:46.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.752+0000 7ff37e9b9740 10 client.0 _ll_register_callbacks cb 0x563eab318b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:46.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.752+0000 7ff379286640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563eab42c4d0 0x563eab435300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.752+0000 7ff379286640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563eab42c4d0 0x563eab435300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46098/0 (socket says 172.21.15.86:46098) 2024-04-21T08:50:46.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.752+0000 7ff379286640 1 -- 172.21.15.86:0/3608490252 learned_addr learned my addr 172.21.15.86:0/3608490252 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:46.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.752+0000 7ff379a87640 1 --2- 172.21.15.86:0/3608490252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563eab42ce80 0x563eab432690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.752+0000 7ff379286640 1 -- 172.21.15.86:0/3608490252 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563eab435a00 msgr2=0x563eab432c10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.752+0000 7ff379286640 1 --2- 172.21.15.86:0/3608490252 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563eab435a00 0x563eab432c10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.752+0000 7ff379286640 1 -- 172.21.15.86:0/3608490252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563eab42ce80 msgr2=0x563eab432690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.752+0000 7ff379286640 1 --2- 172.21.15.86:0/3608490252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563eab42ce80 0x563eab432690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.752+0000 7ff379286640 1 -- 172.21.15.86:0/3608490252 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563eab0a33f0 con 0x563eab42c4d0 2024-04-21T08:50:46.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.752+0000 7ff379a87640 1 --2- 172.21.15.86:0/3608490252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563eab42ce80 0x563eab432690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:46.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.752+0000 7ff379286640 1 --2- 172.21.15.86:0/3608490252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563eab42c4d0 0x563eab435300 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7ff364036040 tx=0x7ff36402c740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a08e48b7386c55d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:46.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.752+0000 7ff35d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:46.759 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.752+0000 7ff35d7fa640 1 -- 172.21.15.86:0/3608490252 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff364048020 con 0x563eab42c4d0 2024-04-21T08:50:46.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.752+0000 7ff35d7fa640 1 -- 172.21.15.86:0/3608490252 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7ff364005030 con 0x563eab42c4d0 2024-04-21T08:50:46.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.752+0000 7ff37e9b9740 1 -- 172.21.15.86:0/3608490252 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3700118f0 con 0x563eab42c4d0 2024-04-21T08:50:46.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.752+0000 7ff35d7fa640 1 -- 172.21.15.86:0/3608490252 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff364053050 con 0x563eab42c4d0 2024-04-21T08:50:46.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.752+0000 7ff37e9b9740 10 client.20412 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:46.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.752+0000 7ff37e9b9740 1 -- 172.21.15.86:0/3608490252 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563eab16d7d0 con 0x563eab42c4d0 2024-04-21T08:50:46.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.753+0000 7ff35d7fa640 1 -- 172.21.15.86:0/3608490252 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3700118f0 con 0x563eab42c4d0 2024-04-21T08:50:46.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.753+0000 7ff35d7fa640 1 -- 172.21.15.86:0/3608490252 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff3640035f0 con 0x563eab42c4d0 2024-04-21T08:50:46.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.753+0000 7ff37e9b9740 10 client.20412 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:46.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.753+0000 7ff37e9b9740 10 client.20412 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:46.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.753+0000 7ff37e9b9740 20 client.20412 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:46.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.753+0000 7ff37e9b9740 1 --2- 172.21.15.86:0/3608490252 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563eab445c50 0x563eab4480f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.753+0000 7ff37e9b9740 4 client.20412 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:46.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.753+0000 7ff37e9b9740 1 -- 172.21.15.86:0/3608490252 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x563eab330930 con 0x563eab445c50 2024-04-21T08:50:46.760 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.753+0000 7ff379a87640 1 --2- 172.21.15.86:0/3608490252 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563eab445c50 0x563eab4480f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.754+0000 7ff379a87640 1 --2- 172.21.15.86:0/3608490252 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563eab445c50 0x563eab4480f0 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=f42d7e71dcdeaf8e server_cookie=e1f628c84809dbba in_seq=0 out_seq=0 2024-04-21T08:50:46.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.754+0000 7ff35d7fa640 10 client.20412 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:46.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.754+0000 7ff35d7fa640 1 -- 172.21.15.86:0/3608490252 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x563eab330930 con 0x563eab445c50 2024-04-21T08:50:46.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.754+0000 7ff35d7fa640 10 client.20412 handle_command_reply: tid=0 2024-04-21T08:50:46.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.754+0000 7ff37e9b9740 1 client.20412 shutdown 2024-04-21T08:50:46.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.754+0000 7ff37e9b9740 1 -- 172.21.15.86:0/3608490252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563eab42c4d0 msgr2=0x563eab435300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.754+0000 7ff37e9b9740 1 --2- 172.21.15.86:0/3608490252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563eab42c4d0 0x563eab435300 secure :-1 s=READY pgs=2059 cs=0 l=1 rev1=1 crypto rx=0x7ff364036040 tx=0x7ff36402c740 comp rx=0 tx=0).stop 2024-04-21T08:50:46.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.754+0000 7ff37e9b9740 1 -- 172.21.15.86:0/3608490252 shutdown_connections 2024-04-21T08:50:46.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.754+0000 7ff37e9b9740 1 --2- 172.21.15.86:0/3608490252 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563eab445c50 0x563eab4480f0 crc :-1 s=READY pgs=341 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.754+0000 7ff37e9b9740 1 --2- 172.21.15.86:0/3608490252 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563eab435a00 0x563eab432c10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.754+0000 7ff35d7fa640 0 client.20412 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:46.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.754+0000 7ff37e9b9740 1 --2- 172.21.15.86:0/3608490252 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563eab42ce80 0x563eab432690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.754+0000 7ff37e9b9740 1 --2- 172.21.15.86:0/3608490252 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563eab42c4d0 0x563eab435300 unknown :-1 s=CLOSED pgs=2059 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.761 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.754+0000 7ff37e9b9740 1 -- 172.21.15.86:0/3608490252 >> 172.21.15.86:0/3608490252 conn(0x563eab426ba0 msgr2=0x563eab4288f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:46.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.755+0000 7ff37e9b9740 1 -- 172.21.15.86:0/3608490252 shutdown_connections 2024-04-21T08:50:46.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.755+0000 7ff37e9b9740 1 -- 172.21.15.86:0/3608490252 wait complete. 2024-04-21T08:50:46.762 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.755+0000 7ff37e9b9740 20 client.20412 trim_cache size 0 max 16384 2024-04-21T08:50:46.773 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.768+0000 7ff37e9b9740 1 Processor -- start 2024-04-21T08:50:46.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.768+0000 7ff37e9b9740 1 -- start start 2024-04-21T08:50:46.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.768+0000 7ff37e9b9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563eab42c4d0 0x563eab441890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.768+0000 7ff37e9b9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563eab42ce80 0x563eab434dc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.768+0000 7ff37e9b9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563eab435a00 0x563eab435300 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.768+0000 7ff37e9b9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff3741b1630 con 0x563eab435a00 2024-04-21T08:50:46.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.768+0000 7ff37e9b9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff3741b3170 con 0x563eab42c4d0 2024-04-21T08:50:46.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.768+0000 7ff37e9b9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff3741b1d00 con 0x563eab42ce80 2024-04-21T08:50:46.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.768+0000 7ff379286640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563eab435a00 0x563eab435300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.768+0000 7ff379286640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563eab435a00 0x563eab435300 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46104/0 (socket says 172.21.15.86:46104) 2024-04-21T08:50:46.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.768+0000 7ff379286640 1 -- 172.21.15.86:0/4172333167 learned_addr learned my addr 172.21.15.86:0/4172333167 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:46.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.768+0000 7ff379a87640 1 --2- 172.21.15.86:0/4172333167 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563eab42ce80 0x563eab434dc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.768+0000 7ff379286640 1 -- 172.21.15.86:0/4172333167 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563eab42ce80 msgr2=0x563eab434dc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.774 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.768+0000 7ff379286640 1 --2- 172.21.15.86:0/4172333167 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563eab42ce80 0x563eab434dc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.768+0000 7ff379286640 1 -- 172.21.15.86:0/4172333167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563eab42c4d0 msgr2=0x563eab441890 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.768+0000 7ff379286640 1 --2- 172.21.15.86:0/4172333167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563eab42c4d0 0x563eab441890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.768+0000 7ff379286640 1 -- 172.21.15.86:0/4172333167 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x563eab42cc20 con 0x563eab435a00 2024-04-21T08:50:46.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.768+0000 7ff379286640 1 --2- 172.21.15.86:0/4172333167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563eab435a00 0x563eab435300 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7ff36402c170 tx=0x7ff364004990 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cf19f67a26a51230 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:46.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.768+0000 7ff35f7fe640 1 -- 172.21.15.86:0/4172333167 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff364048070 con 0x563eab435a00 2024-04-21T08:50:46.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.768+0000 7ff35f7fe640 1 -- 172.21.15.86:0/4172333167 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3640035f0 con 0x563eab435a00 2024-04-21T08:50:46.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.768+0000 7ff35f7fe640 1 -- 172.21.15.86:0/4172333167 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff364042c50 con 0x563eab435a00 2024-04-21T08:50:46.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.769+0000 7ff37e9b9740 1 -- 172.21.15.86:0/4172333167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563eab435a00 msgr2=0x563eab435300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.769+0000 7ff37e9b9740 1 --2- 172.21.15.86:0/4172333167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563eab435a00 0x563eab435300 secure :-1 s=READY pgs=2060 cs=0 l=1 rev1=1 crypto rx=0x7ff36402c170 tx=0x7ff364004990 comp rx=0 tx=0).stop 2024-04-21T08:50:46.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.769+0000 7ff37e9b9740 1 -- 172.21.15.86:0/4172333167 shutdown_connections 2024-04-21T08:50:46.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.769+0000 7ff37e9b9740 1 --2- 172.21.15.86:0/4172333167 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563eab435a00 0x563eab435300 unknown :-1 s=CLOSED pgs=2060 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.769+0000 7ff37e9b9740 1 --2- 172.21.15.86:0/4172333167 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563eab42ce80 0x563eab434dc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.769+0000 7ff37e9b9740 1 --2- 172.21.15.86:0/4172333167 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563eab42c4d0 0x563eab441890 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.775 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.769+0000 7ff37e9b9740 1 -- 172.21.15.86:0/4172333167 >> 172.21.15.86:0/4172333167 conn(0x563eab426ba0 msgr2=0x563eab428880 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:46.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.769+0000 7ff37e9b9740 1 -- 172.21.15.86:0/4172333167 shutdown_connections 2024-04-21T08:50:46.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.769+0000 7ff37e9b9740 1 -- 172.21.15.86:0/4172333167 wait complete. 2024-04-21T08:50:46.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.769+0000 7ff37e9b9740 1 -- start start 2024-04-21T08:50:46.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.769+0000 7ff37e9b9740 1 Processor -- start 2024-04-21T08:50:46.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.770+0000 7ff37e9b9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563eab42c4d0 0x563eab43df90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.770+0000 7ff37e9b9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563eab42ce80 0x563eab42eec0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.770+0000 7ff37e9b9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563eab435a00 0x563eab42f400 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.770+0000 7ff37e9b9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7ff3741b2aa0 con 0x563eab42ce80 2024-04-21T08:50:46.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.770+0000 7ff37e9b9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7ff3741b23d0 con 0x563eab435a00 2024-04-21T08:50:46.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.770+0000 7ff37e9b9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7ff3741af630 con 0x563eab42c4d0 2024-04-21T08:50:46.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.770+0000 7ff37e9b9740 10 client.0 _ll_register_callbacks cb 0x7ff364004460 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:46.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.770+0000 7ff379a87640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563eab42ce80 0x563eab42eec0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.770+0000 7ff379a87640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563eab42ce80 0x563eab42eec0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46114/0 (socket says 172.21.15.86:46114) 2024-04-21T08:50:46.776 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.770+0000 7ff379a87640 1 -- 172.21.15.86:0/3483172406 learned_addr learned my addr 172.21.15.86:0/3483172406 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:46.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.770+0000 7ff379286640 1 --2- 172.21.15.86:0/3483172406 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563eab42c4d0 0x563eab43df90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.770+0000 7ff379a87640 1 -- 172.21.15.86:0/3483172406 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563eab42c4d0 msgr2=0x563eab43df90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.770+0000 7ff379a87640 1 --2- 172.21.15.86:0/3483172406 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563eab42c4d0 0x563eab43df90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.770+0000 7ff379a87640 1 -- 172.21.15.86:0/3483172406 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563eab435a00 msgr2=0x563eab42f400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:46.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.770+0000 7ff379a87640 1 --2- 172.21.15.86:0/3483172406 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563eab435a00 0x563eab42f400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.770+0000 7ff379a87640 1 -- 172.21.15.86:0/3483172406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x563eab42f9f0 con 0x563eab42ce80 2024-04-21T08:50:46.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.770+0000 7ff379286640 1 --2- 172.21.15.86:0/3483172406 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563eab42c4d0 0x563eab43df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:46.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.770+0000 7ff379a87640 1 --2- 172.21.15.86:0/3483172406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563eab42ce80 0x563eab42eec0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x563eab42bfa0 tx=0x7ff370005720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4a3eab7c07df5899 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:46.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.770+0000 7ff35d7fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:46.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.770+0000 7ff35d7fa640 1 -- 172.21.15.86:0/3483172406 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7ff37002b060 con 0x563eab42ce80 2024-04-21T08:50:46.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.770+0000 7ff35d7fa640 1 -- 172.21.15.86:0/3483172406 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7ff370004240 con 0x563eab42ce80 2024-04-21T08:50:46.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.770+0000 7ff37e9b9740 1 -- 172.21.15.86:0/3483172406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7ff3640035f0 con 0x563eab42ce80 2024-04-21T08:50:46.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.771+0000 7ff35d7fa640 1 -- 172.21.15.86:0/3483172406 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7ff370032050 con 0x563eab42ce80 2024-04-21T08:50:46.777 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.771+0000 7ff37e9b9740 10 client.20418 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:46.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.771+0000 7ff37e9b9740 1 -- 172.21.15.86:0/3483172406 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x563eab4ac080 con 0x563eab42ce80 2024-04-21T08:50:46.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.771+0000 7ff35d7fa640 1 -- 172.21.15.86:0/3483172406 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7ff3640035f0 con 0x563eab42ce80 2024-04-21T08:50:46.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.771+0000 7ff35d7fa640 1 -- 172.21.15.86:0/3483172406 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7ff370024070 con 0x563eab42ce80 2024-04-21T08:50:46.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.771+0000 7ff37e9b9740 10 client.20418 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:46.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.771+0000 7ff37e9b9740 10 client.20418 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:46.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.771+0000 7ff37e9b9740 20 client.20418 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:46.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.771+0000 7ff37e9b9740 1 --2- 172.21.15.86:0/3483172406 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563eab4ac5d0 0x563eab4ae9f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:46.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.771+0000 7ff37e9b9740 4 client.20418 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:50:46.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.771+0000 7ff37e9b9740 1 -- 172.21.15.86:0/3483172406 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x563eab39ec10 con 0x563eab4ac5d0 2024-04-21T08:50:46.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.771+0000 7ff379286640 1 --2- 172.21.15.86:0/3483172406 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563eab4ac5d0 0x563eab4ae9f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:46.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.772+0000 7ff35d7fa640 10 client.20418 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:46.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.772+0000 7ff379286640 1 --2- 172.21.15.86:0/3483172406 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563eab4ac5d0 0x563eab4ae9f0 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=efe4ed487d9c78aa server_cookie=52b605da8d9ddc19 in_seq=0 out_seq=0 2024-04-21T08:50:46.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.772+0000 7ff35d7fa640 1 -- 172.21.15.86:0/3483172406 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x563eab39ec10 con 0x563eab4ac5d0 2024-04-21T08:50:46.778 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.772+0000 7ff35d7fa640 10 client.20418 handle_command_reply: tid=0 2024-04-21T08:50:46.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.772+0000 7ff37e9b9740 1 client.20418 shutdown 2024-04-21T08:50:46.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.772+0000 7ff37e9b9740 1 -- 172.21.15.86:0/3483172406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563eab42ce80 msgr2=0x563eab42eec0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.772+0000 7ff37e9b9740 1 --2- 172.21.15.86:0/3483172406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563eab42ce80 0x563eab42eec0 secure :-1 s=READY pgs=2061 cs=0 l=1 rev1=1 crypto rx=0x563eab42bfa0 tx=0x7ff370005720 comp rx=0 tx=0).stop 2024-04-21T08:50:46.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.772+0000 7ff37e9b9740 1 -- 172.21.15.86:0/3483172406 shutdown_connections 2024-04-21T08:50:46.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.772+0000 7ff37e9b9740 1 --2- 172.21.15.86:0/3483172406 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x563eab4ac5d0 0x563eab4ae9f0 crc :-1 s=READY pgs=342 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.772+0000 7ff37e9b9740 1 --2- 172.21.15.86:0/3483172406 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x563eab435a00 0x563eab42f400 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.772+0000 7ff35d7fa640 0 client.20418 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:46.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.772+0000 7ff37e9b9740 1 --2- 172.21.15.86:0/3483172406 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x563eab42ce80 0x563eab42eec0 unknown :-1 s=CLOSED pgs=2061 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.772+0000 7ff37e9b9740 1 --2- 172.21.15.86:0/3483172406 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x563eab42c4d0 0x563eab43df90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.772+0000 7ff37e9b9740 1 -- 172.21.15.86:0/3483172406 >> 172.21.15.86:0/3483172406 conn(0x563eab426ba0 msgr2=0x563eab42d2b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:46.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.772+0000 7ff37e9b9740 1 -- 172.21.15.86:0/3483172406 shutdown_connections 2024-04-21T08:50:46.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.772+0000 7ff37e9b9740 1 -- 172.21.15.86:0/3483172406 wait complete. 2024-04-21T08:50:46.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.772+0000 7ff37e9b9740 20 client.20418 trim_cache size 0 max 16384 2024-04-21T08:50:46.780 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:50:46.780 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.773+0000 7ff37bceb640 1 -- 172.21.15.86:0/2897770391 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff3600a3d20 msgr2=0x7ff3600a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.773+0000 7ff37bceb640 1 --2- 172.21.15.86:0/2897770391 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff3600a3d20 0x7ff3600a61e0 secure :-1 s=READY pgs=344 cs=0 l=1 rev1=1 crypto rx=0x7ff36c0ce020 tx=0x7ff36c019f90 comp rx=0 tx=0).stop 2024-04-21T08:50:46.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.773+0000 7ff37bceb640 1 -- 172.21.15.86:0/2897770391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff374106f30 msgr2=0x7ff3741020b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:46.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.773+0000 7ff37bceb640 1 --2- 172.21.15.86:0/2897770391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff374106f30 0x7ff3741020b0 secure :-1 s=READY pgs=2057 cs=0 l=1 rev1=1 crypto rx=0x7ff36c00cad0 tx=0x7ff36c00cf90 comp rx=0 tx=0).stop 2024-04-21T08:50:46.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.773+0000 7ff37bceb640 1 -- 172.21.15.86:0/2897770391 shutdown_connections 2024-04-21T08:50:46.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.773+0000 7ff37bceb640 1 --2- 172.21.15.86:0/2897770391 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7ff3600a3d20 0x7ff3600a61e0 unknown :-1 s=CLOSED pgs=344 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.773+0000 7ff37bceb640 1 --2- 172.21.15.86:0/2897770391 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7ff374106f30 0x7ff3741020b0 unknown :-1 s=CLOSED pgs=2057 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.773+0000 7ff37bceb640 1 --2- 172.21.15.86:0/2897770391 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7ff374100e10 0x7ff374101b70 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.773+0000 7ff37bceb640 1 --2- 172.21.15.86:0/2897770391 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7ff374100450 0x7ff374101630 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:46.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.773+0000 7ff37bceb640 1 -- 172.21.15.86:0/2897770391 >> 172.21.15.86:0/2897770391 conn(0x7ff3740773e0 msgr2=0x7ff374105160 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:46.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.773+0000 7ff37bceb640 1 -- 172.21.15.86:0/2897770391 shutdown_connections 2024-04-21T08:50:46.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:46.773+0000 7ff37bceb640 1 -- 172.21.15.86:0/2897770391 wait complete. 2024-04-21T08:50:47.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:46 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1275081894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:50:47.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:46 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1275081894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:50:47.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:46 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1275081894' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:50:47.784 INFO:tasks.fwd_scrub.fs.[cephfs]:scrubbing fs: cephfs 2024-04-21T08:50:47.785 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:50:47.785 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 300 ceph --cluster ceph tell mds.1:0 scrub start / force,recursive 2024-04-21T08:50:47.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.898+0000 7f6490750640 1 -- 172.21.15.86:0/1111653634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6488106170 msgr2=0x7f648810dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:47.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.898+0000 7f6490750640 1 --2- 172.21.15.86:0/1111653634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6488106170 0x7f648810dd40 secure :-1 s=READY pgs=2062 cs=0 l=1 rev1=1 crypto rx=0x7f6480009ca0 tx=0x7f648002c100 comp rx=0 tx=0).stop 2024-04-21T08:50:47.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.898+0000 7f6490750640 1 -- 172.21.15.86:0/1111653634 shutdown_connections 2024-04-21T08:50:47.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.898+0000 7f6490750640 1 --2- 172.21.15.86:0/1111653634 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f6488106170 0x7f648810dd40 unknown :-1 s=CLOSED pgs=2062 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:47.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.898+0000 7f6490750640 1 --2- 172.21.15.86:0/1111653634 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6488101a50 0x7f6488105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:47.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.898+0000 7f6490750640 1 --2- 172.21.15.86:0/1111653634 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64881010a0 0x7f6488101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:47.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.898+0000 7f6490750640 1 -- 172.21.15.86:0/1111653634 >> 172.21.15.86:0/1111653634 conn(0x7f6488077eb0 msgr2=0x7f64880782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:47.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.898+0000 7f6490750640 1 -- 172.21.15.86:0/1111653634 shutdown_connections 2024-04-21T08:50:47.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.898+0000 7f6490750640 1 -- 172.21.15.86:0/1111653634 wait complete. 2024-04-21T08:50:47.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.899+0000 7f6490750640 1 Processor -- start 2024-04-21T08:50:47.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.899+0000 7f6490750640 1 -- start start 2024-04-21T08:50:47.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.899+0000 7f6490750640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64881010a0 0x7f64881a0400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:47.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.899+0000 7f6490750640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6488106170 0x7f64881a0940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:47.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.899+0000 7f6490750640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f648819a600 0x7f648819aab0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:47.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.899+0000 7f6490750640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f64881110c0 con 0x7f648819a600 2024-04-21T08:50:47.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.899+0000 7f6490750640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f6488110f40 con 0x7f64881010a0 2024-04-21T08:50:47.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.899+0000 7f6490750640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f6488111240 con 0x7f6488106170 2024-04-21T08:50:47.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.899+0000 7f648eced640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f648819a600 0x7f648819aab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:47.902 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.899+0000 7f648eced640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f648819a600 0x7f648819aab0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46134/0 (socket says 172.21.15.86:46134) 2024-04-21T08:50:47.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.899+0000 7f648eced640 1 -- 172.21.15.86:0/1094836416 learned_addr learned my addr 172.21.15.86:0/1094836416 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:47.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.899+0000 7f648dceb640 1 --2- 172.21.15.86:0/1094836416 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6488106170 0x7f64881a0940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:47.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.899+0000 7f648e4ec640 1 --2- 172.21.15.86:0/1094836416 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64881010a0 0x7f64881a0400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:47.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.899+0000 7f648eced640 1 -- 172.21.15.86:0/1094836416 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6488106170 msgr2=0x7f64881a0940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:47.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.899+0000 7f648eced640 1 --2- 172.21.15.86:0/1094836416 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6488106170 0x7f64881a0940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:47.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.899+0000 7f648eced640 1 -- 172.21.15.86:0/1094836416 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64881010a0 msgr2=0x7f64881a0400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:47.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.899+0000 7f648eced640 1 --2- 172.21.15.86:0/1094836416 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64881010a0 0x7f64881a0400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:47.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.899+0000 7f648eced640 1 -- 172.21.15.86:0/1094836416 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f648819b120 con 0x7f648819a600 2024-04-21T08:50:47.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.900+0000 7f648eced640 1 --2- 172.21.15.86:0/1094836416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f648819a600 0x7f648819aab0 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f64800098d0 tx=0x7f6480035d60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:47.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.900+0000 7f647f7fe640 1 -- 172.21.15.86:0/1094836416 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6480039040 con 0x7f648819a600 2024-04-21T08:50:47.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.900+0000 7f647f7fe640 1 -- 172.21.15.86:0/1094836416 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f6480044070 con 0x7f648819a600 2024-04-21T08:50:47.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.900+0000 7f6490750640 1 -- 172.21.15.86:0/1094836416 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f648819b3b0 con 0x7f648819a600 2024-04-21T08:50:47.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.900+0000 7f647f7fe640 1 -- 172.21.15.86:0/1094836416 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6480004540 con 0x7f648819a600 2024-04-21T08:50:47.903 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.900+0000 7f6490750640 1 -- 172.21.15.86:0/1094836416 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f64881a7550 con 0x7f648819a600 2024-04-21T08:50:47.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.902+0000 7f647f7fe640 1 -- 172.21.15.86:0/1094836416 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f64800046e0 con 0x7f648819a600 2024-04-21T08:50:47.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.902+0000 7f6493419740 1 Processor -- start 2024-04-21T08:50:47.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.902+0000 7f6493419740 1 -- start start 2024-04-21T08:50:47.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.902+0000 7f647f7fe640 1 --2- 172.21.15.86:0/1094836416 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f64740a3d20 0x7f64740a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:47.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.903+0000 7f6493419740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56018373f8e0 0x56018373fcc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:47.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.903+0000 7f6493419740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560183740290 0x560183748840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:47.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.903+0000 7f6493419740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560183748e10 0x56018374b260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:47.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.903+0000 7f6493419740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5601836a3f00 con 0x560183748e10 2024-04-21T08:50:47.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.903+0000 7f6493419740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x560183641590 con 0x56018373f8e0 2024-04-21T08:50:47.911 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.903+0000 7f6493419740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56018369fec0 con 0x560183740290 2024-04-21T08:50:47.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.903+0000 7f648dceb640 1 --2- 172.21.15.86:0/1094836416 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f64740a3d20 0x7f64740a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:47.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.903+0000 7f648eced640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560183748e10 0x56018374b260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:47.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.903+0000 7f648eced640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560183748e10 0x56018374b260 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46146/0 (socket says 172.21.15.86:46146) 2024-04-21T08:50:47.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.903+0000 7f648eced640 1 -- 172.21.15.86:0/2878657644 learned_addr learned my addr 172.21.15.86:0/2878657644 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:47.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.903+0000 7f647f7fe640 1 -- 172.21.15.86:0/1094836416 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f64800f33d0 con 0x7f648819a600 2024-04-21T08:50:47.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.903+0000 7f648e4ec640 1 --2- 172.21.15.86:0/2878657644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56018373f8e0 0x56018373fcc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:47.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.903+0000 7f648dceb640 1 --2- 172.21.15.86:0/2878657644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560183740290 0x560183748840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:47.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.903+0000 7f648eced640 1 -- 172.21.15.86:0/2878657644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560183740290 msgr2=0x560183748840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:47.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.903+0000 7f648eced640 1 --2- 172.21.15.86:0/2878657644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560183740290 0x560183748840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:47.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.903+0000 7f648eced640 1 -- 172.21.15.86:0/2878657644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56018373f8e0 msgr2=0x56018373fcc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:47.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.903+0000 7f648eced640 1 --2- 172.21.15.86:0/2878657644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56018373f8e0 0x56018373fcc0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:47.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.903+0000 7f648eced640 1 -- 172.21.15.86:0/2878657644 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56018347e7e0 con 0x560183748e10 2024-04-21T08:50:47.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.903+0000 7f648dceb640 1 --2- 172.21.15.86:0/2878657644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560183740290 0x560183748840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:47.912 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.903+0000 7f648dceb640 1 --2- 172.21.15.86:0/1094836416 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f64740a3d20 0x7f64740a61e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6478005e10 tx=0x7f64780127c0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:47.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.903+0000 7f648eced640 1 --2- 172.21.15.86:0/2878657644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560183748e10 0x56018374b260 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f64800bf040 tx=0x7f6480004040 comp rx=0 tx=0).ready entity=mon.0 client_cookie=aac42c82bf7e6193 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:47.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.903+0000 7f647cff9640 1 -- 172.21.15.86:0/2878657644 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6480096e90 con 0x560183748e10 2024-04-21T08:50:47.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.903+0000 7f647cff9640 1 -- 172.21.15.86:0/2878657644 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64800bf7b0 con 0x560183748e10 2024-04-21T08:50:47.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.904+0000 7f647cff9640 1 -- 172.21.15.86:0/2878657644 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6480044220 con 0x560183748e10 2024-04-21T08:50:47.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.904+0000 7f6493419740 1 -- 172.21.15.86:0/2878657644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560183748e10 msgr2=0x56018374b260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:47.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.904+0000 7f6493419740 1 --2- 172.21.15.86:0/2878657644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560183748e10 0x56018374b260 secure :-1 s=READY pgs=2064 cs=0 l=1 rev1=1 crypto rx=0x7f64800bf040 tx=0x7f6480004040 comp rx=0 tx=0).stop 2024-04-21T08:50:47.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.904+0000 7f6493419740 1 -- 172.21.15.86:0/2878657644 shutdown_connections 2024-04-21T08:50:47.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.904+0000 7f6493419740 1 --2- 172.21.15.86:0/2878657644 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560183748e10 0x56018374b260 unknown :-1 s=CLOSED pgs=2064 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:47.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.904+0000 7f6493419740 1 --2- 172.21.15.86:0/2878657644 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560183740290 0x560183748840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:47.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.904+0000 7f6493419740 1 --2- 172.21.15.86:0/2878657644 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56018373f8e0 0x56018373fcc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:47.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.904+0000 7f6493419740 1 -- 172.21.15.86:0/2878657644 >> 172.21.15.86:0/2878657644 conn(0x560183739fb0 msgr2=0x56018373a390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:47.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.904+0000 7f6493419740 1 -- 172.21.15.86:0/2878657644 shutdown_connections 2024-04-21T08:50:47.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.904+0000 7f6493419740 1 -- 172.21.15.86:0/2878657644 wait complete. 2024-04-21T08:50:47.913 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.905+0000 7f6493419740 1 -- start start 2024-04-21T08:50:47.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.905+0000 7f6493419740 1 Processor -- start 2024-04-21T08:50:47.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.905+0000 7f6493419740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56018373f8e0 0x560183745d30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:47.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.905+0000 7f6493419740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560183740290 0x560183746270 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:47.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.905+0000 7f6493419740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560183748e10 0x5601837467b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:47.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.905+0000 7f6493419740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56018374bb40 con 0x560183748e10 2024-04-21T08:50:47.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.905+0000 7f6493419740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x56018365d580 con 0x560183740290 2024-04-21T08:50:47.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.905+0000 7f6493419740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56018374bcc0 con 0x56018373f8e0 2024-04-21T08:50:47.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.905+0000 7f6493419740 10 client.0 _ll_register_callbacks cb 0x560183663650 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:47.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.905+0000 7f648eced640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560183748e10 0x5601837467b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:47.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.905+0000 7f648eced640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560183748e10 0x5601837467b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46150/0 (socket says 172.21.15.86:46150) 2024-04-21T08:50:47.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.905+0000 7f648eced640 1 -- 172.21.15.86:0/2830887596 learned_addr learned my addr 172.21.15.86:0/2830887596 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:47.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.905+0000 7f648e4ec640 1 --2- 172.21.15.86:0/2830887596 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56018373f8e0 0x560183745d30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:47.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.905+0000 7f648dceb640 1 --2- 172.21.15.86:0/2830887596 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560183740290 0x560183746270 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:47.914 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.905+0000 7f648eced640 1 -- 172.21.15.86:0/2830887596 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56018373f8e0 msgr2=0x560183745d30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:47.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.905+0000 7f648eced640 1 --2- 172.21.15.86:0/2830887596 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56018373f8e0 0x560183745d30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:47.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.905+0000 7f648eced640 1 -- 172.21.15.86:0/2830887596 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560183740290 msgr2=0x560183746270 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:47.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.905+0000 7f648eced640 1 --2- 172.21.15.86:0/2830887596 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560183740290 0x560183746270 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:47.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.905+0000 7f648eced640 1 -- 172.21.15.86:0/2830887596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601834def70 con 0x560183748e10 2024-04-21T08:50:47.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.905+0000 7f648dceb640 1 --2- 172.21.15.86:0/2830887596 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560183740290 0x560183746270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:47.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.905+0000 7f648eced640 1 --2- 172.21.15.86:0/2830887596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560183748e10 0x5601837467b0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f64800f0d50 tx=0x7f6480009630 comp rx=0 tx=0).ready entity=mon.0 client_cookie=5db860e1f0e10e0f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:47.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.905+0000 7f6471ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:47.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.906+0000 7f6471ffb640 1 -- 172.21.15.86:0/2830887596 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6480044220 con 0x560183748e10 2024-04-21T08:50:47.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.906+0000 7f6471ffb640 1 -- 172.21.15.86:0/2830887596 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f6480032780 con 0x560183748e10 2024-04-21T08:50:47.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.906+0000 7f6493419740 1 -- 172.21.15.86:0/2830887596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f64800bf7b0 con 0x560183748e10 2024-04-21T08:50:47.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.906+0000 7f6471ffb640 1 -- 172.21.15.86:0/2830887596 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f64800bf0b0 con 0x560183748e10 2024-04-21T08:50:47.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.906+0000 7f6493419740 10 client.20430 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:47.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.906+0000 7f6493419740 1 -- 172.21.15.86:0/2830887596 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x560183376170 con 0x560183748e10 2024-04-21T08:50:47.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.906+0000 7f6471ffb640 1 -- 172.21.15.86:0/2830887596 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f64800bf7b0 con 0x560183748e10 2024-04-21T08:50:47.915 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.906+0000 7f6471ffb640 1 -- 172.21.15.86:0/2830887596 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f648003c030 con 0x560183748e10 2024-04-21T08:50:47.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.906+0000 7f6493419740 10 client.20430 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:47.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.906+0000 7f6493419740 10 client.20430 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:47.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.906+0000 7f6493419740 20 client.20430 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:47.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.906+0000 7f6493419740 1 --2- 172.21.15.86:0/2830887596 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x560183758ef0 0x56018375b2e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:47.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.906+0000 7f6493419740 4 client.20430 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:47.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.906+0000 7f6493419740 1 -- 172.21.15.86:0/2830887596 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5601836b1920 con 0x560183758ef0 2024-04-21T08:50:47.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.906+0000 7f648e4ec640 1 --2- 172.21.15.86:0/2830887596 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x560183758ef0 0x56018375b2e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:47.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.907+0000 7f6471ffb640 10 client.20430 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:47.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.907+0000 7f648e4ec640 1 --2- 172.21.15.86:0/2830887596 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x560183758ef0 0x56018375b2e0 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=3dbe1d83220f9525 server_cookie=4759d239168b3f43 in_seq=0 out_seq=0 2024-04-21T08:50:47.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.908+0000 7f6471ffb640 1 -- 172.21.15.86:0/2830887596 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5601836b1920 con 0x560183758ef0 2024-04-21T08:50:47.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.908+0000 7f6471ffb640 10 client.20430 handle_command_reply: tid=0 2024-04-21T08:50:47.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.908+0000 7f6493419740 1 client.20430 shutdown 2024-04-21T08:50:47.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.908+0000 7f6493419740 1 -- 172.21.15.86:0/2830887596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560183748e10 msgr2=0x5601837467b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:47.916 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.908+0000 7f6493419740 1 --2- 172.21.15.86:0/2830887596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560183748e10 0x5601837467b0 secure :-1 s=READY pgs=2065 cs=0 l=1 rev1=1 crypto rx=0x7f64800f0d50 tx=0x7f6480009630 comp rx=0 tx=0).stop 2024-04-21T08:50:47.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.908+0000 7f6493419740 1 -- 172.21.15.86:0/2830887596 shutdown_connections 2024-04-21T08:50:47.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.908+0000 7f6493419740 1 --2- 172.21.15.86:0/2830887596 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x560183758ef0 0x56018375b2e0 crc :-1 s=READY pgs=343 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:47.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.908+0000 7f6493419740 1 --2- 172.21.15.86:0/2830887596 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560183748e10 0x5601837467b0 unknown :-1 s=CLOSED pgs=2065 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:47.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.908+0000 7f6493419740 1 --2- 172.21.15.86:0/2830887596 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560183740290 0x560183746270 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:47.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.908+0000 7f6471ffb640 0 client.20430 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:47.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.908+0000 7f6493419740 1 --2- 172.21.15.86:0/2830887596 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56018373f8e0 0x560183745d30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:47.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.908+0000 7f6493419740 1 -- 172.21.15.86:0/2830887596 >> 172.21.15.86:0/2830887596 conn(0x560183739fb0 msgr2=0x56018373bd00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:47.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.908+0000 7f6493419740 1 -- 172.21.15.86:0/2830887596 shutdown_connections 2024-04-21T08:50:47.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.908+0000 7f6493419740 1 -- 172.21.15.86:0/2830887596 wait complete. 2024-04-21T08:50:47.917 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.908+0000 7f6493419740 20 client.20430 trim_cache size 0 max 16384 2024-04-21T08:50:47.930 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.925+0000 7f6493419740 1 Processor -- start 2024-04-21T08:50:47.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.925+0000 7f6493419740 1 -- start start 2024-04-21T08:50:47.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.926+0000 7f6493419740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56018373f8e0 0x560183754970 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:47.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.926+0000 7f6493419740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560183740290 0x5601837465d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:47.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.926+0000 7f6493419740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560183748e10 0x560183760a70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:47.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.926+0000 7f6493419740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f64881b13f0 con 0x560183740290 2024-04-21T08:50:47.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.926+0000 7f6493419740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f64881b2f30 con 0x56018373f8e0 2024-04-21T08:50:47.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.926+0000 7f6493419740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f64881b1ac0 con 0x560183748e10 2024-04-21T08:50:47.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.926+0000 7f648dceb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560183740290 0x5601837465d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:47.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.926+0000 7f648dceb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560183740290 0x5601837465d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46160/0 (socket says 172.21.15.86:46160) 2024-04-21T08:50:47.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.926+0000 7f648dceb640 1 -- 172.21.15.86:0/3735382941 learned_addr learned my addr 172.21.15.86:0/3735382941 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:47.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.926+0000 7f648e4ec640 1 --2- 172.21.15.86:0/3735382941 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56018373f8e0 0x560183754970 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:47.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.926+0000 7f648eced640 1 --2- 172.21.15.86:0/3735382941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560183748e10 0x560183760a70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:47.931 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.926+0000 7f648dceb640 1 -- 172.21.15.86:0/3735382941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560183748e10 msgr2=0x560183760a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:47.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.926+0000 7f648dceb640 1 --2- 172.21.15.86:0/3735382941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560183748e10 0x560183760a70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:47.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.926+0000 7f648dceb640 1 -- 172.21.15.86:0/3735382941 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56018373f8e0 msgr2=0x560183754970 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:47.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.926+0000 7f648dceb640 1 --2- 172.21.15.86:0/3735382941 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56018373f8e0 0x560183754970 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:47.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.926+0000 7f648dceb640 1 -- 172.21.15.86:0/3735382941 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x560183740030 con 0x560183740290 2024-04-21T08:50:47.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.926+0000 7f648dceb640 1 --2- 172.21.15.86:0/3735382941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560183740290 0x5601837465d0 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f6488078750 tx=0x7f6478005da0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=f2f1d23e78808ce7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:47.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.926+0000 7f6473fff640 1 -- 172.21.15.86:0/3735382941 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6478023070 con 0x560183740290 2024-04-21T08:50:47.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.926+0000 7f6473fff640 1 -- 172.21.15.86:0/3735382941 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f647801fb60 con 0x560183740290 2024-04-21T08:50:47.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.926+0000 7f6473fff640 1 -- 172.21.15.86:0/3735382941 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6478028ad0 con 0x560183740290 2024-04-21T08:50:47.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.927+0000 7f6493419740 1 -- 172.21.15.86:0/3735382941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560183740290 msgr2=0x5601837465d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:47.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.927+0000 7f6493419740 1 --2- 172.21.15.86:0/3735382941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560183740290 0x5601837465d0 secure :-1 s=READY pgs=2066 cs=0 l=1 rev1=1 crypto rx=0x7f6488078750 tx=0x7f6478005da0 comp rx=0 tx=0).stop 2024-04-21T08:50:47.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.927+0000 7f6493419740 1 -- 172.21.15.86:0/3735382941 shutdown_connections 2024-04-21T08:50:47.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.927+0000 7f6493419740 1 --2- 172.21.15.86:0/3735382941 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x560183748e10 0x560183760a70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:47.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.927+0000 7f6493419740 1 --2- 172.21.15.86:0/3735382941 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560183740290 0x5601837465d0 unknown :-1 s=CLOSED pgs=2066 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:47.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.927+0000 7f6493419740 1 --2- 172.21.15.86:0/3735382941 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56018373f8e0 0x560183754970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:47.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.927+0000 7f6493419740 1 -- 172.21.15.86:0/3735382941 >> 172.21.15.86:0/3735382941 conn(0x560183739fb0 msgr2=0x56018373bc90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:47.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.927+0000 7f6493419740 1 -- 172.21.15.86:0/3735382941 shutdown_connections 2024-04-21T08:50:47.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.927+0000 7f6493419740 1 -- 172.21.15.86:0/3735382941 wait complete. 2024-04-21T08:50:47.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.927+0000 7f6493419740 1 -- start start 2024-04-21T08:50:47.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.927+0000 7f6493419740 1 Processor -- start 2024-04-21T08:50:47.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.928+0000 7f6493419740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56018373f8e0 0x560183742c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:47.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.928+0000 7f6493419740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560183740290 0x5601837431b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:47.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.928+0000 7f6493419740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560183748e10 0x5601837436f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:47.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.928+0000 7f6493419740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f64881b2860 con 0x560183740290 2024-04-21T08:50:47.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.928+0000 7f6493419740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f64881b2190 con 0x560183748e10 2024-04-21T08:50:47.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.928+0000 7f6493419740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f64881af3f0 con 0x56018373f8e0 2024-04-21T08:50:47.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.928+0000 7f6493419740 10 client.0 _ll_register_callbacks cb 0x7f648400c350 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:47.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.928+0000 7f648dceb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560183740290 0x5601837431b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:47.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.928+0000 7f648dceb640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560183740290 0x5601837431b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46166/0 (socket says 172.21.15.86:46166) 2024-04-21T08:50:47.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.928+0000 7f648dceb640 1 -- 172.21.15.86:0/2494528455 learned_addr learned my addr 172.21.15.86:0/2494528455 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:47.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.928+0000 7f648eced640 1 --2- 172.21.15.86:0/2494528455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560183748e10 0x5601837436f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:47.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.928+0000 7f648e4ec640 1 --2- 172.21.15.86:0/2494528455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56018373f8e0 0x560183742c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:47.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.928+0000 7f648dceb640 1 -- 172.21.15.86:0/2494528455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56018373f8e0 msgr2=0x560183742c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:47.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.928+0000 7f648dceb640 1 --2- 172.21.15.86:0/2494528455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56018373f8e0 0x560183742c70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:47.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.928+0000 7f648dceb640 1 -- 172.21.15.86:0/2494528455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560183748e10 msgr2=0x5601837436f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:47.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.928+0000 7f648dceb640 1 --2- 172.21.15.86:0/2494528455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560183748e10 0x5601837436f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:47.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.928+0000 7f648dceb640 1 -- 172.21.15.86:0/2494528455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5601837be580 con 0x560183740290 2024-04-21T08:50:47.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.928+0000 7f648eced640 1 --2- 172.21.15.86:0/2494528455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560183748e10 0x5601837436f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:47.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.928+0000 7f648e4ec640 1 --2- 172.21.15.86:0/2494528455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56018373f8e0 0x560183742c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:47.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.928+0000 7f648dceb640 1 --2- 172.21.15.86:0/2494528455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560183740290 0x5601837431b0 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f6478005940 tx=0x7f647800a770 comp rx=0 tx=0).ready entity=mon.0 client_cookie=cc415b407c9b0c7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:47.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.928+0000 7f6471ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:47.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.928+0000 7f6471ffb640 1 -- 172.21.15.86:0/2494528455 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f6478023040 con 0x560183740290 2024-04-21T08:50:47.934 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.928+0000 7f6471ffb640 1 -- 172.21.15.86:0/2494528455 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f6478005240 con 0x560183740290 2024-04-21T08:50:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.929+0000 7f6493419740 1 -- 172.21.15.86:0/2494528455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f647801fb60 con 0x560183740290 2024-04-21T08:50:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.929+0000 7f6471ffb640 1 -- 172.21.15.86:0/2494528455 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f647803e050 con 0x560183740290 2024-04-21T08:50:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.929+0000 7f6493419740 10 client.20436 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.929+0000 7f6493419740 1 -- 172.21.15.86:0/2494528455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5601837bf8a0 con 0x560183740290 2024-04-21T08:50:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.929+0000 7f6471ffb640 1 -- 172.21.15.86:0/2494528455 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f647801fb60 con 0x560183740290 2024-04-21T08:50:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.929+0000 7f6471ffb640 1 -- 172.21.15.86:0/2494528455 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f64780046c0 con 0x560183740290 2024-04-21T08:50:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.929+0000 7f6493419740 10 client.20436 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.929+0000 7f6493419740 10 client.20436 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.929+0000 7f6493419740 20 client.20436 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.929+0000 7f6493419740 1 --2- 172.21.15.86:0/2494528455 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5601837bff90 0x5601837c23b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.929+0000 7f6493419740 4 client.20436 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}] 2024-04-21T08:50:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.929+0000 7f6493419740 1 -- 172.21.15.86:0/2494528455 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub start", "path": "/", "scrubops": ["force", "recursive"]}) -- 0x5601836ab790 con 0x5601837bff90 2024-04-21T08:50:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.929+0000 7f648e4ec640 1 --2- 172.21.15.86:0/2494528455 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5601837bff90 0x5601837c23b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.930+0000 7f6471ffb640 10 client.20436 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:47.935 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.930+0000 7f648e4ec640 1 --2- 172.21.15.86:0/2494528455 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5601837bff90 0x5601837c23b0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=4bccff47cfe80b0 server_cookie=40335d50d79a7338 in_seq=0 out_seq=0 2024-04-21T08:50:47.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.931+0000 7f6471ffb640 1 -- 172.21.15.86:0/2494528455 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+110 (crc 0 0 0) 0x5601836ab790 con 0x5601837bff90 2024-04-21T08:50:47.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.931+0000 7f6471ffb640 10 client.20436 handle_command_reply: tid=0 2024-04-21T08:50:47.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.931+0000 7f6493419740 1 client.20436 shutdown 2024-04-21T08:50:47.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.931+0000 7f6493419740 1 -- 172.21.15.86:0/2494528455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560183740290 msgr2=0x5601837431b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:47.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.931+0000 7f6493419740 1 --2- 172.21.15.86:0/2494528455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560183740290 0x5601837431b0 secure :-1 s=READY pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f6478005940 tx=0x7f647800a770 comp rx=0 tx=0).stop 2024-04-21T08:50:47.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.931+0000 7f6493419740 1 -- 172.21.15.86:0/2494528455 shutdown_connections 2024-04-21T08:50:47.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.931+0000 7f6493419740 1 --2- 172.21.15.86:0/2494528455 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5601837bff90 0x5601837c23b0 crc :-1 s=READY pgs=344 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:47.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.931+0000 7f6493419740 1 --2- 172.21.15.86:0/2494528455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x560183748e10 0x5601837436f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:47.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.931+0000 7f6493419740 1 --2- 172.21.15.86:0/2494528455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x560183740290 0x5601837431b0 secure :-1 s=CLOSED pgs=2067 cs=0 l=1 rev1=1 crypto rx=0x7f6478005940 tx=0x7f647800a770 comp rx=0 tx=0).stop 2024-04-21T08:50:47.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.931+0000 7f6493419740 1 --2- 172.21.15.86:0/2494528455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56018373f8e0 0x560183742c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:47.937 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.931+0000 7f6471ffb640 0 client.20436 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:47.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.931+0000 7f6493419740 1 -- 172.21.15.86:0/2494528455 >> 172.21.15.86:0/2494528455 conn(0x560183739fb0 msgr2=0x560183738780 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:47.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.931+0000 7f6493419740 1 -- 172.21.15.86:0/2494528455 shutdown_connections 2024-04-21T08:50:47.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.931+0000 7f6493419740 1 -- 172.21.15.86:0/2494528455 wait complete. 2024-04-21T08:50:47.938 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.931+0000 7f6493419740 20 client.20436 trim_cache size 0 max 16384 2024-04-21T08:50:47.938 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:50:47.938 INFO:teuthology.orchestra.run.smithi086.stdout: "return_code": 0, 2024-04-21T08:50:47.938 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_tag": "56d6aaab-b821-4766-954f-c4c8c4ff1859", 2024-04-21T08:50:47.938 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": "asynchronous" 2024-04-21T08:50:47.938 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:50:47.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.932+0000 7f6490750640 1 -- 172.21.15.86:0/1094836416 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f64740a3d20 msgr2=0x7f64740a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:47.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.932+0000 7f6490750640 1 --2- 172.21.15.86:0/1094836416 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f64740a3d20 0x7f64740a61e0 secure :-1 s=READY pgs=345 cs=0 l=1 rev1=1 crypto rx=0x7f6478005e10 tx=0x7f64780127c0 comp rx=0 tx=0).stop 2024-04-21T08:50:47.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.932+0000 7f6490750640 1 -- 172.21.15.86:0/1094836416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f648819a600 msgr2=0x7f648819aab0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:47.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.932+0000 7f6490750640 1 --2- 172.21.15.86:0/1094836416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f648819a600 0x7f648819aab0 secure :-1 s=READY pgs=2063 cs=0 l=1 rev1=1 crypto rx=0x7f64800098d0 tx=0x7f6480035d60 comp rx=0 tx=0).stop 2024-04-21T08:50:47.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.932+0000 7f6490750640 1 -- 172.21.15.86:0/1094836416 shutdown_connections 2024-04-21T08:50:47.939 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.932+0000 7f6490750640 1 --2- 172.21.15.86:0/1094836416 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f64740a3d20 0x7f64740a61e0 unknown :-1 s=CLOSED pgs=345 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:47.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.932+0000 7f6490750640 1 --2- 172.21.15.86:0/1094836416 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f648819a600 0x7f648819aab0 unknown :-1 s=CLOSED pgs=2063 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:47.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.932+0000 7f6490750640 1 --2- 172.21.15.86:0/1094836416 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f6488106170 0x7f64881a0940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:47.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.932+0000 7f6490750640 1 --2- 172.21.15.86:0/1094836416 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f64881010a0 0x7f64881a0400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:47.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.932+0000 7f6490750640 1 -- 172.21.15.86:0/1094836416 >> 172.21.15.86:0/1094836416 conn(0x7f6488077eb0 msgr2=0x7f648810cbf0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:47.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.932+0000 7f6490750640 1 -- 172.21.15.86:0/1094836416 shutdown_connections 2024-04-21T08:50:47.940 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:47.932+0000 7f6490750640 1 -- 172.21.15.86:0/1094836416 wait complete. 2024-04-21T08:50:47.943 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:50:47.943 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:50:48.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.043+0000 7fd6e6e06640 1 -- 172.21.15.86:0/2867302345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6e0106180 msgr2=0x7fd6e010dd50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:48.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.043+0000 7fd6e6e06640 1 --2- 172.21.15.86:0/2867302345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6e0106180 0x7fd6e010dd50 secure :-1 s=READY pgs=2068 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8009ca0 tx=0x7fd6d802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:50:48.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.043+0000 7fd6e6e06640 1 -- 172.21.15.86:0/2867302345 shutdown_connections 2024-04-21T08:50:48.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.043+0000 7fd6e6e06640 1 --2- 172.21.15.86:0/2867302345 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6e0106180 0x7fd6e010dd50 unknown :-1 s=CLOSED pgs=2068 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:48.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.043+0000 7fd6e6e06640 1 --2- 172.21.15.86:0/2867302345 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6e0101a60 0x7fd6e0105aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:48.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.043+0000 7fd6e6e06640 1 --2- 172.21.15.86:0/2867302345 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6e01010b0 0x7fd6e0101490 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:48.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.043+0000 7fd6e6e06640 1 -- 172.21.15.86:0/2867302345 >> 172.21.15.86:0/2867302345 conn(0x7fd6e00771d0 msgr2=0x7fd6e00775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:48.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.043+0000 7fd6e6e06640 1 -- 172.21.15.86:0/2867302345 shutdown_connections 2024-04-21T08:50:48.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.043+0000 7fd6e6e06640 1 -- 172.21.15.86:0/2867302345 wait complete. 2024-04-21T08:50:48.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.044+0000 7fd6e6e06640 1 Processor -- start 2024-04-21T08:50:48.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.044+0000 7fd6e6e06640 1 -- start start 2024-04-21T08:50:48.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.044+0000 7fd6e6e06640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6e01010b0 0x7fd6e01994b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:48.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.044+0000 7fd6e6e06640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6e0106180 0x7fd6e01999f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:48.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.044+0000 7fd6e6e06640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6e0199f80 0x7fd6e01a4530 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:48.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.044+0000 7fd6e6e06640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e01110d0 con 0x7fd6e0106180 2024-04-21T08:50:48.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.044+0000 7fd6e6e06640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e0110f50 con 0x7fd6e0199f80 2024-04-21T08:50:48.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.044+0000 7fd6e6e06640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e0111250 con 0x7fd6e01010b0 2024-04-21T08:50:48.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.044+0000 7fd6d7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6e0106180 0x7fd6e01999f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:48.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.044+0000 7fd6d7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6e0106180 0x7fd6e01999f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46172/0 (socket says 172.21.15.86:46172) 2024-04-21T08:50:48.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.044+0000 7fd6d7fff640 1 -- 172.21.15.86:0/1531346283 learned_addr learned my addr 172.21.15.86:0/1531346283 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:48.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.044+0000 7fd6e4ba2640 1 --2- 172.21.15.86:0/1531346283 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6e01010b0 0x7fd6e01994b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:48.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.044+0000 7fd6e53a3640 1 --2- 172.21.15.86:0/1531346283 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6e0199f80 0x7fd6e01a4530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:48.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.044+0000 7fd6d7fff640 1 -- 172.21.15.86:0/1531346283 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6e01010b0 msgr2=0x7fd6e01994b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:48.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.044+0000 7fd6d7fff640 1 --2- 172.21.15.86:0/1531346283 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6e01010b0 0x7fd6e01994b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:48.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.044+0000 7fd6d7fff640 1 -- 172.21.15.86:0/1531346283 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6e0199f80 msgr2=0x7fd6e01a4530 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:48.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.044+0000 7fd6d7fff640 1 --2- 172.21.15.86:0/1531346283 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6e0199f80 0x7fd6e01a4530 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:48.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.044+0000 7fd6d7fff640 1 -- 172.21.15.86:0/1531346283 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd6e01a4d30 con 0x7fd6e0106180 2024-04-21T08:50:48.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.044+0000 7fd6e53a3640 1 --2- 172.21.15.86:0/1531346283 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6e0199f80 0x7fd6e01a4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:48.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.044+0000 7fd6d7fff640 1 --2- 172.21.15.86:0/1531346283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6e0106180 0x7fd6e01999f0 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7fd6d000c970 tx=0x7fd6d000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:48.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.045+0000 7fd6d5ffb640 1 -- 172.21.15.86:0/1531346283 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0014070 con 0x7fd6e0106180 2024-04-21T08:50:48.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.045+0000 7fd6d5ffb640 1 -- 172.21.15.86:0/1531346283 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6d0004440 con 0x7fd6e0106180 2024-04-21T08:50:48.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.045+0000 7fd6e6e06640 1 -- 172.21.15.86:0/1531346283 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd6e01a4fc0 con 0x7fd6e0106180 2024-04-21T08:50:48.056 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.045+0000 7fd6d5ffb640 1 -- 172.21.15.86:0/1531346283 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d0003e50 con 0x7fd6e0106180 2024-04-21T08:50:48.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.045+0000 7fd6e6e06640 1 -- 172.21.15.86:0/1531346283 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd6e01a54e0 con 0x7fd6e0106180 2024-04-21T08:50:48.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.047+0000 7fd6d5ffb640 1 -- 172.21.15.86:0/1531346283 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd6d0017070 con 0x7fd6e0106180 2024-04-21T08:50:48.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.048+0000 7fd6e99f6740 1 Processor -- start 2024-04-21T08:50:48.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.048+0000 7fd6e99f6740 1 -- start start 2024-04-21T08:50:48.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.048+0000 7fd6d5ffb640 1 --2- 172.21.15.86:0/1531346283 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd6c80a3ec0 0x7fd6c80a6380 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:48.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.048+0000 7fd6e99f6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b995505580 0x55b995505960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:48.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.048+0000 7fd6e99f6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b995505ea0 0x55b99550e540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:48.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.048+0000 7fd6e99f6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b99550ea80 0x55b995510e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:48.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.048+0000 7fd6e99f6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b99546a1e0 con 0x55b995505ea0 2024-04-21T08:50:48.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.048+0000 7fd6e99f6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b995407390 con 0x55b99550ea80 2024-04-21T08:50:48.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.048+0000 7fd6e99f6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b9954661a0 con 0x55b995505580 2024-04-21T08:50:48.057 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.048+0000 7fd6e53a3640 1 --2- 172.21.15.86:0/1531346283 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd6c80a3ec0 0x7fd6c80a6380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:48.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.048+0000 7fd6d7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b995505ea0 0x55b99550e540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:48.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.048+0000 7fd6e4ba2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b995505580 0x55b995505960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:48.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.048+0000 7fd6d7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b995505ea0 0x55b99550e540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46176/0 (socket says 172.21.15.86:46176) 2024-04-21T08:50:48.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.048+0000 7fd6d7fff640 1 -- 172.21.15.86:0/3364526485 learned_addr learned my addr 172.21.15.86:0/3364526485 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:48.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.048+0000 7fd6e4ba2640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b995505580 0x55b995505960 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.179:3300/0 says I am v2:172.21.15.86:60530/0 (socket says 172.21.15.86:60530) 2024-04-21T08:50:48.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.048+0000 7fd6d5ffb640 1 -- 172.21.15.86:0/1531346283 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7fd6d00d07c0 con 0x7fd6e0106180 2024-04-21T08:50:48.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.048+0000 7fd6e53a3640 1 --2- 172.21.15.86:0/3364526485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b99550ea80 0x55b995510e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:48.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.048+0000 7fd6d7fff640 1 -- 172.21.15.86:0/3364526485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b995505580 msgr2=0x55b995505960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:48.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.048+0000 7fd6d7fff640 1 --2- 172.21.15.86:0/3364526485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b995505580 0x55b995505960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:48.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.048+0000 7fd6d7fff640 1 -- 172.21.15.86:0/3364526485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b99550ea80 msgr2=0x55b995510e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:48.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.048+0000 7fd6d7fff640 1 --2- 172.21.15.86:0/3364526485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b99550ea80 0x55b995510e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:48.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.048+0000 7fd6d7fff640 1 -- 172.21.15.86:0/3364526485 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b9952467d0 con 0x55b995505ea0 2024-04-21T08:50:48.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.049+0000 7fd6e53a3640 1 --2- 172.21.15.86:0/1531346283 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd6c80a3ec0 0x7fd6c80a6380 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8037040 tx=0x7fd6d8032f30 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:48.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.049+0000 7fd6d7fff640 1 --2- 172.21.15.86:0/3364526485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b995505ea0 0x55b99550e540 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7fd6d009c090 tx=0x7fd6d00145e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7230a55de6b0a48c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:48.058 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.049+0000 7fd6c6ffd640 1 -- 172.21.15.86:0/3364526485 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d009bcf0 con 0x55b995505ea0 2024-04-21T08:50:48.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.049+0000 7fd6c6ffd640 1 -- 172.21.15.86:0/3364526485 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6d009ca90 con 0x55b995505ea0 2024-04-21T08:50:48.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.049+0000 7fd6c6ffd640 1 -- 172.21.15.86:0/3364526485 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d00b2200 con 0x55b995505ea0 2024-04-21T08:50:48.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.049+0000 7fd6e99f6740 1 -- 172.21.15.86:0/3364526485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b995505ea0 msgr2=0x55b99550e540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:48.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.049+0000 7fd6e99f6740 1 --2- 172.21.15.86:0/3364526485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b995505ea0 0x55b99550e540 secure :-1 s=READY pgs=2070 cs=0 l=1 rev1=1 crypto rx=0x7fd6d009c090 tx=0x7fd6d00145e0 comp rx=0 tx=0).stop 2024-04-21T08:50:48.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.049+0000 7fd6e99f6740 1 -- 172.21.15.86:0/3364526485 shutdown_connections 2024-04-21T08:50:48.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.049+0000 7fd6e99f6740 1 --2- 172.21.15.86:0/3364526485 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b99550ea80 0x55b995510e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:48.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.049+0000 7fd6e99f6740 1 --2- 172.21.15.86:0/3364526485 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b995505ea0 0x55b99550e540 unknown :-1 s=CLOSED pgs=2070 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:48.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.049+0000 7fd6e99f6740 1 --2- 172.21.15.86:0/3364526485 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b995505580 0x55b995505960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:48.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.049+0000 7fd6e99f6740 1 -- 172.21.15.86:0/3364526485 >> 172.21.15.86:0/3364526485 conn(0x55b9954ffcb0 msgr2=0x55b995500090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:48.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.049+0000 7fd6e99f6740 1 -- 172.21.15.86:0/3364526485 shutdown_connections 2024-04-21T08:50:48.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.050+0000 7fd6e99f6740 1 -- 172.21.15.86:0/3364526485 wait complete. 2024-04-21T08:50:48.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.050+0000 7fd6e99f6740 1 -- start start 2024-04-21T08:50:48.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.050+0000 7fd6e99f6740 1 Processor -- start 2024-04-21T08:50:48.059 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.050+0000 7fd6e99f6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b995505580 0x55b99550bab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:48.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.050+0000 7fd6e99f6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b995505ea0 0x55b99550bff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:48.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.050+0000 7fd6e99f6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b99550ea80 0x55b99550c530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:48.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.050+0000 7fd6e99f6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b995511870 con 0x55b99550ea80 2024-04-21T08:50:48.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.050+0000 7fd6e99f6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b9954232f0 con 0x55b995505580 2024-04-21T08:50:48.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.050+0000 7fd6e99f6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b9955119f0 con 0x55b995505ea0 2024-04-21T08:50:48.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.050+0000 7fd6e99f6740 10 client.0 _ll_register_callbacks cb 0x55b995430580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:48.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.050+0000 7fd6e53a3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b99550ea80 0x55b99550c530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:48.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.050+0000 7fd6e53a3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b99550ea80 0x55b99550c530 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46188/0 (socket says 172.21.15.86:46188) 2024-04-21T08:50:48.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.050+0000 7fd6e53a3640 1 -- 172.21.15.86:0/474458561 learned_addr learned my addr 172.21.15.86:0/474458561 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:48.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.050+0000 7fd6e4ba2640 1 --2- 172.21.15.86:0/474458561 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b995505580 0x55b99550bab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:48.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.051+0000 7fd6e53a3640 1 -- 172.21.15.86:0/474458561 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b995505ea0 msgr2=0x55b99550bff0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:48.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.051+0000 7fd6d7fff640 1 --2- 172.21.15.86:0/474458561 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b995505ea0 0x55b99550bff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:48.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.051+0000 7fd6e53a3640 1 --2- 172.21.15.86:0/474458561 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b995505ea0 0x55b99550bff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:48.060 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.051+0000 7fd6e53a3640 1 -- 172.21.15.86:0/474458561 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b995505580 msgr2=0x55b99550bab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:48.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.051+0000 7fd6e53a3640 1 --2- 172.21.15.86:0/474458561 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b995505580 0x55b99550bab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:48.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.051+0000 7fd6e53a3640 1 -- 172.21.15.86:0/474458561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b99517c3f0 con 0x55b99550ea80 2024-04-21T08:50:48.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.051+0000 7fd6e53a3640 1 --2- 172.21.15.86:0/474458561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b99550ea80 0x55b99550c530 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8003ba0 tx=0x7fd6d802c0f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=eb043c7f1a323cea server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:48.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.051+0000 7fd6bffff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:48.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.051+0000 7fd6bffff640 1 -- 172.21.15.86:0/474458561 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d8053060 con 0x55b99550ea80 2024-04-21T08:50:48.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.051+0000 7fd6bffff640 1 -- 172.21.15.86:0/474458561 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7fd6d80047b0 con 0x55b99550ea80 2024-04-21T08:50:48.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.051+0000 7fd6e99f6740 1 -- 172.21.15.86:0/474458561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6d009ca90 con 0x55b99550ea80 2024-04-21T08:50:48.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.051+0000 7fd6bffff640 1 -- 172.21.15.86:0/474458561 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd6d805a050 con 0x55b99550ea80 2024-04-21T08:50:48.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.051+0000 7fd6e99f6740 10 client.20448 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:48.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.051+0000 7fd6e99f6740 1 -- 172.21.15.86:0/474458561 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b99513c0f0 con 0x55b99550ea80 2024-04-21T08:50:48.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.051+0000 7fd6bffff640 1 -- 172.21.15.86:0/474458561 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6d009ca90 con 0x55b99550ea80 2024-04-21T08:50:48.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.051+0000 7fd6bffff640 1 -- 172.21.15.86:0/474458561 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd6d804d070 con 0x55b99550ea80 2024-04-21T08:50:48.061 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.051+0000 7fd6e99f6740 10 client.20448 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:48.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.051+0000 7fd6e99f6740 10 client.20448 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:48.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.052+0000 7fd6e99f6740 20 client.20448 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:48.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.052+0000 7fd6e99f6740 1 --2- 172.21.15.86:0/474458561 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b99551ea70 0x55b995520ee0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:48.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.052+0000 7fd6e99f6740 4 client.20448 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:48.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.052+0000 7fd6e99f6740 1 -- 172.21.15.86:0/474458561 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b995477c00 con 0x55b99551ea70 2024-04-21T08:50:48.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.052+0000 7fd6e4ba2640 1 --2- 172.21.15.86:0/474458561 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b99551ea70 0x55b995520ee0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:48.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.052+0000 7fd6bffff640 10 client.20448 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:48.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.052+0000 7fd6e4ba2640 1 --2- 172.21.15.86:0/474458561 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b99551ea70 0x55b995520ee0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=c34fecff5b64578a server_cookie=4b9af0de240c8e6 in_seq=0 out_seq=0 2024-04-21T08:50:48.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.053+0000 7fd6bffff640 1 -- 172.21.15.86:0/474458561 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b995477c00 con 0x55b99551ea70 2024-04-21T08:50:48.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.053+0000 7fd6bffff640 10 client.20448 handle_command_reply: tid=0 2024-04-21T08:50:48.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.053+0000 7fd6e99f6740 1 client.20448 shutdown 2024-04-21T08:50:48.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.053+0000 7fd6e99f6740 1 -- 172.21.15.86:0/474458561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b99550ea80 msgr2=0x55b99550c530 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:48.062 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.053+0000 7fd6e99f6740 1 --2- 172.21.15.86:0/474458561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b99550ea80 0x55b99550c530 secure :-1 s=READY pgs=2071 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8003ba0 tx=0x7fd6d802c0f0 comp rx=0 tx=0).stop 2024-04-21T08:50:48.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.053+0000 7fd6e99f6740 1 -- 172.21.15.86:0/474458561 shutdown_connections 2024-04-21T08:50:48.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.053+0000 7fd6e99f6740 1 --2- 172.21.15.86:0/474458561 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b99551ea70 0x55b995520ee0 crc :-1 s=READY pgs=345 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:48.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.053+0000 7fd6e99f6740 1 --2- 172.21.15.86:0/474458561 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b99550ea80 0x55b99550c530 unknown :-1 s=CLOSED pgs=2071 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:48.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.053+0000 7fd6bffff640 0 client.20448 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:48.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.053+0000 7fd6e99f6740 1 --2- 172.21.15.86:0/474458561 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b995505ea0 0x55b99550bff0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:48.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.053+0000 7fd6e99f6740 1 --2- 172.21.15.86:0/474458561 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b995505580 0x55b99550bab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:48.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.053+0000 7fd6e99f6740 1 -- 172.21.15.86:0/474458561 >> 172.21.15.86:0/474458561 conn(0x55b9954ffcb0 msgr2=0x55b995501a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:48.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.053+0000 7fd6e99f6740 1 -- 172.21.15.86:0/474458561 shutdown_connections 2024-04-21T08:50:48.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.053+0000 7fd6e99f6740 1 -- 172.21.15.86:0/474458561 wait complete. 2024-04-21T08:50:48.063 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.053+0000 7fd6e99f6740 20 client.20448 trim_cache size 0 max 16384 2024-04-21T08:50:48.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.070+0000 7fd6e99f6740 1 Processor -- start 2024-04-21T08:50:48.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.070+0000 7fd6e99f6740 1 -- start start 2024-04-21T08:50:48.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.070+0000 7fd6e99f6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b995505580 0x55b99550e1e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:48.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.070+0000 7fd6e99f6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b995505ea0 0x55b99551a730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:48.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.070+0000 7fd6e99f6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b99550ea80 0x55b9954fed70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:48.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.070+0000 7fd6e99f6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e01af430 con 0x55b99550ea80 2024-04-21T08:50:48.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.070+0000 7fd6e99f6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e01b0f70 con 0x55b995505580 2024-04-21T08:50:48.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.070+0000 7fd6e99f6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e01afb00 con 0x55b995505ea0 2024-04-21T08:50:48.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.070+0000 7fd6e53a3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b99550ea80 0x55b9954fed70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:48.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.070+0000 7fd6e53a3640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b99550ea80 0x55b9954fed70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46200/0 (socket says 172.21.15.86:46200) 2024-04-21T08:50:48.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.070+0000 7fd6e53a3640 1 -- 172.21.15.86:0/2355771319 learned_addr learned my addr 172.21.15.86:0/2355771319 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:48.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.071+0000 7fd6e53a3640 1 -- 172.21.15.86:0/2355771319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b995505ea0 msgr2=0x55b99551a730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:48.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.071+0000 7fd6d7fff640 1 --2- 172.21.15.86:0/2355771319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b995505ea0 0x55b99551a730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:48.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.071+0000 7fd6e4ba2640 1 --2- 172.21.15.86:0/2355771319 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b995505580 0x55b99550e1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:48.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.071+0000 7fd6e53a3640 1 --2- 172.21.15.86:0/2355771319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b995505ea0 0x55b99551a730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:48.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.071+0000 7fd6e53a3640 1 -- 172.21.15.86:0/2355771319 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b995505580 msgr2=0x55b99550e1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:48.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.071+0000 7fd6e53a3640 1 --2- 172.21.15.86:0/2355771319 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b995505580 0x55b99550e1e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:48.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.071+0000 7fd6e53a3640 1 -- 172.21.15.86:0/2355771319 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b995505cd0 con 0x55b99550ea80 2024-04-21T08:50:48.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.071+0000 7fd6d7fff640 1 --2- 172.21.15.86:0/2355771319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b995505ea0 0x55b99551a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:48.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.071+0000 7fd6e53a3640 1 --2- 172.21.15.86:0/2355771319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b99550ea80 0x55b9954fed70 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7fd6d80049d0 tx=0x7fd6d8009b20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=93689fa225f06d93 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:48.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.071+0000 7fd6c67fc640 1 -- 172.21.15.86:0/2355771319 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d8053040 con 0x55b99550ea80 2024-04-21T08:50:48.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.071+0000 7fd6c67fc640 1 -- 172.21.15.86:0/2355771319 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6d804e620 con 0x55b99550ea80 2024-04-21T08:50:48.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.071+0000 7fd6c67fc640 1 -- 172.21.15.86:0/2355771319 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d8039070 con 0x55b99550ea80 2024-04-21T08:50:48.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.072+0000 7fd6e99f6740 1 -- 172.21.15.86:0/2355771319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b99550ea80 msgr2=0x55b9954fed70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:48.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.072+0000 7fd6e99f6740 1 --2- 172.21.15.86:0/2355771319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b99550ea80 0x55b9954fed70 secure :-1 s=READY pgs=2072 cs=0 l=1 rev1=1 crypto rx=0x7fd6d80049d0 tx=0x7fd6d8009b20 comp rx=0 tx=0).stop 2024-04-21T08:50:48.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.072+0000 7fd6e99f6740 1 -- 172.21.15.86:0/2355771319 shutdown_connections 2024-04-21T08:50:48.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.072+0000 7fd6e99f6740 1 --2- 172.21.15.86:0/2355771319 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b99550ea80 0x55b9954fed70 unknown :-1 s=CLOSED pgs=2072 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:48.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.072+0000 7fd6e99f6740 1 --2- 172.21.15.86:0/2355771319 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b995505ea0 0x55b99551a730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:48.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.072+0000 7fd6e99f6740 1 --2- 172.21.15.86:0/2355771319 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b995505580 0x55b99550e1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:48.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.072+0000 7fd6e99f6740 1 -- 172.21.15.86:0/2355771319 >> 172.21.15.86:0/2355771319 conn(0x55b9954ffcb0 msgr2=0x55b99550a9f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:48.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.072+0000 7fd6e99f6740 1 -- 172.21.15.86:0/2355771319 shutdown_connections 2024-04-21T08:50:48.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.072+0000 7fd6e99f6740 1 -- 172.21.15.86:0/2355771319 wait complete. 2024-04-21T08:50:48.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.072+0000 7fd6e99f6740 1 -- start start 2024-04-21T08:50:48.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.072+0000 7fd6e99f6740 1 Processor -- start 2024-04-21T08:50:48.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.072+0000 7fd6e99f6740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b995505580 0x55b995512730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:48.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.072+0000 7fd6e99f6740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b995505ea0 0x55b995512c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:48.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.072+0000 7fd6e99f6740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b99550ea80 0x55b9955131b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:48.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.072+0000 7fd6e99f6740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e01b08a0 con 0x55b995505ea0 2024-04-21T08:50:48.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.072+0000 7fd6e99f6740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e01b01d0 con 0x55b99550ea80 2024-04-21T08:50:48.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.072+0000 7fd6e99f6740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd6e01ad430 con 0x55b995505580 2024-04-21T08:50:48.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.072+0000 7fd6e99f6740 10 client.0 _ll_register_callbacks cb 0x7fd6d8037070 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:48.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.072+0000 7fd6d7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b995505ea0 0x55b995512c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:48.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.072+0000 7fd6d7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b995505ea0 0x55b995512c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:46208/0 (socket says 172.21.15.86:46208) 2024-04-21T08:50:48.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.072+0000 7fd6d7fff640 1 -- 172.21.15.86:0/523980502 learned_addr learned my addr 172.21.15.86:0/523980502 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:48.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.073+0000 7fd6e4ba2640 1 --2- 172.21.15.86:0/523980502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b995505580 0x55b995512730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:48.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.073+0000 7fd6e53a3640 1 --2- 172.21.15.86:0/523980502 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b99550ea80 0x55b9955131b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:48.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.073+0000 7fd6d7fff640 1 -- 172.21.15.86:0/523980502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b995505580 msgr2=0x55b995512730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:48.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.073+0000 7fd6d7fff640 1 --2- 172.21.15.86:0/523980502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b995505580 0x55b995512730 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:48.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.073+0000 7fd6d7fff640 1 -- 172.21.15.86:0/523980502 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b99550ea80 msgr2=0x55b9955131b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:48.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.073+0000 7fd6d7fff640 1 --2- 172.21.15.86:0/523980502 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b99550ea80 0x55b9955131b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:48.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.073+0000 7fd6d7fff640 1 -- 172.21.15.86:0/523980502 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b9955840f0 con 0x55b995505ea0 2024-04-21T08:50:48.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.073+0000 7fd6e53a3640 1 --2- 172.21.15.86:0/523980502 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b99550ea80 0x55b9955131b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:48.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.073+0000 7fd6d7fff640 1 --2- 172.21.15.86:0/523980502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b995505ea0 0x55b995512c70 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7fd6d00987f0 tx=0x7fd6d009a8a0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2a19220507ad12ed server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:48.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.073+0000 7fd6bffff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:48.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.073+0000 7fd6bffff640 1 -- 172.21.15.86:0/523980502 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd6d00af450 con 0x55b995505ea0 2024-04-21T08:50:48.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.073+0000 7fd6bffff640 1 -- 172.21.15.86:0/523980502 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7fd6d0098a00 con 0x55b995505ea0 2024-04-21T08:50:48.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.073+0000 7fd6e99f6740 1 -- 172.21.15.86:0/523980502 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd6d804e620 con 0x55b995505ea0 2024-04-21T08:50:48.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.073+0000 7fd6bffff640 1 -- 172.21.15.86:0/523980502 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd6d00af630 con 0x55b995505ea0 2024-04-21T08:50:48.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.073+0000 7fd6e99f6740 10 client.20454 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:48.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.073+0000 7fd6e99f6740 1 -- 172.21.15.86:0/523980502 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b995585370 con 0x55b995505ea0 2024-04-21T08:50:48.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.073+0000 7fd6bffff640 1 -- 172.21.15.86:0/523980502 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd6d804e620 con 0x55b995505ea0 2024-04-21T08:50:48.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.073+0000 7fd6bffff640 1 -- 172.21.15.86:0/523980502 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd6d00bb150 con 0x55b995505ea0 2024-04-21T08:50:48.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.074+0000 7fd6e99f6740 10 client.20454 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:48.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.074+0000 7fd6e99f6740 10 client.20454 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:48.080 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.074+0000 7fd6e99f6740 20 client.20454 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:48.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.074+0000 7fd6e99f6740 1 --2- 172.21.15.86:0/523980502 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b995585670 0x55b995587a90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:48.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.074+0000 7fd6e99f6740 4 client.20454 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:50:48.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.074+0000 7fd6e99f6740 1 -- 172.21.15.86:0/523980502 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55b995471a70 con 0x55b995585670 2024-04-21T08:50:48.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.074+0000 7fd6e4ba2640 1 --2- 172.21.15.86:0/523980502 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b995585670 0x55b995587a90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:48.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.074+0000 7fd6bffff640 10 client.20454 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:48.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.074+0000 7fd6e4ba2640 1 --2- 172.21.15.86:0/523980502 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b995585670 0x55b995587a90 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=a6d68a8902eadeed server_cookie=dea061efec7d9439 in_seq=0 out_seq=0 2024-04-21T08:50:48.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.074+0000 7fd6bffff640 1 -- 172.21.15.86:0/523980502 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+259 (crc 0 0 0) 0x55b995471a70 con 0x55b995585670 2024-04-21T08:50:48.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.074+0000 7fd6bffff640 10 client.20454 handle_command_reply: tid=0 2024-04-21T08:50:48.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.074+0000 7fd6e99f6740 1 client.20454 shutdown 2024-04-21T08:50:48.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.074+0000 7fd6e99f6740 1 -- 172.21.15.86:0/523980502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b995505ea0 msgr2=0x55b995512c70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:48.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.074+0000 7fd6e99f6740 1 --2- 172.21.15.86:0/523980502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b995505ea0 0x55b995512c70 secure :-1 s=READY pgs=2073 cs=0 l=1 rev1=1 crypto rx=0x7fd6d00987f0 tx=0x7fd6d009a8a0 comp rx=0 tx=0).stop 2024-04-21T08:50:48.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.075+0000 7fd6e99f6740 1 -- 172.21.15.86:0/523980502 shutdown_connections 2024-04-21T08:50:48.081 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.075+0000 7fd6e99f6740 1 --2- 172.21.15.86:0/523980502 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b995585670 0x55b995587a90 crc :-1 s=READY pgs=346 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:48.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.075+0000 7fd6e99f6740 1 --2- 172.21.15.86:0/523980502 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b99550ea80 0x55b9955131b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:48.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.075+0000 7fd6bffff640 0 client.20454 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:48.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.075+0000 7fd6e99f6740 1 --2- 172.21.15.86:0/523980502 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b995505ea0 0x55b995512c70 unknown :-1 s=CLOSED pgs=2073 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:48.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.075+0000 7fd6e99f6740 1 --2- 172.21.15.86:0/523980502 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b995505580 0x55b995512730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:48.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.075+0000 7fd6e99f6740 1 -- 172.21.15.86:0/523980502 >> 172.21.15.86:0/523980502 conn(0x55b9954ffcb0 msgr2=0x55b99550c300 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:48.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.075+0000 7fd6e99f6740 1 -- 172.21.15.86:0/523980502 shutdown_connections 2024-04-21T08:50:48.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.075+0000 7fd6e99f6740 1 -- 172.21.15.86:0/523980502 wait complete. 2024-04-21T08:50:48.082 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.075+0000 7fd6e99f6740 20 client.20454 trim_cache size 0 max 16384 2024-04-21T08:50:48.082 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:50:48.082 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "IDLE (waiting for more scrubs)", 2024-04-21T08:50:48.083 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": { 2024-04-21T08:50:48.083 INFO:teuthology.orchestra.run.smithi086.stdout: "56d6aaab-b821-4766-954f-c4c8c4ff1859": { 2024-04-21T08:50:48.083 INFO:teuthology.orchestra.run.smithi086.stdout: "path": "/", 2024-04-21T08:50:48.083 INFO:teuthology.orchestra.run.smithi086.stdout: "tag": "56d6aaab-b821-4766-954f-c4c8c4ff1859", 2024-04-21T08:50:48.083 INFO:teuthology.orchestra.run.smithi086.stdout: "options": "recursive,force" 2024-04-21T08:50:48.083 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:50:48.083 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:50:48.083 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:50:48.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.075+0000 7fd6e6e06640 1 -- 172.21.15.86:0/1531346283 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd6c80a3ec0 msgr2=0x7fd6c80a6380 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:48.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.075+0000 7fd6e6e06640 1 --2- 172.21.15.86:0/1531346283 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd6c80a3ec0 0x7fd6c80a6380 secure :-1 s=READY pgs=346 cs=0 l=1 rev1=1 crypto rx=0x7fd6d8037040 tx=0x7fd6d8032f30 comp rx=0 tx=0).stop 2024-04-21T08:50:48.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.075+0000 7fd6e6e06640 1 -- 172.21.15.86:0/1531346283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6e0106180 msgr2=0x7fd6e01999f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:48.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.075+0000 7fd6e6e06640 1 --2- 172.21.15.86:0/1531346283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6e0106180 0x7fd6e01999f0 secure :-1 s=READY pgs=2069 cs=0 l=1 rev1=1 crypto rx=0x7fd6d000c970 tx=0x7fd6d000ce30 comp rx=0 tx=0).stop 2024-04-21T08:50:48.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.075+0000 7fd6e6e06640 1 -- 172.21.15.86:0/1531346283 shutdown_connections 2024-04-21T08:50:48.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.075+0000 7fd6e6e06640 1 --2- 172.21.15.86:0/1531346283 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd6c80a3ec0 0x7fd6c80a6380 unknown :-1 s=CLOSED pgs=346 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:48.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.075+0000 7fd6e6e06640 1 --2- 172.21.15.86:0/1531346283 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd6e0199f80 0x7fd6e01a4530 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:48.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.075+0000 7fd6e6e06640 1 --2- 172.21.15.86:0/1531346283 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd6e0106180 0x7fd6e01999f0 unknown :-1 s=CLOSED pgs=2069 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:48.084 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.075+0000 7fd6e6e06640 1 --2- 172.21.15.86:0/1531346283 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd6e01010b0 0x7fd6e01994b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:48.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.075+0000 7fd6e6e06640 1 -- 172.21.15.86:0/1531346283 >> 172.21.15.86:0/1531346283 conn(0x7fd6e00771d0 msgr2=0x7fd6e00feba0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:48.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.075+0000 7fd6e6e06640 1 -- 172.21.15.86:0/1531346283 shutdown_connections 2024-04-21T08:50:48.085 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:50:48.075+0000 7fd6e6e06640 1 -- 172.21.15.86:0/1531346283 wait complete. 2024-04-21T08:50:48.085 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:47 smithi086 ceph-mon[27939]: pgmap v375: 129 pgs: 129 active+clean; 2.7 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-21T08:50:48.086 INFO:tasks.cephfs.filesystem:scrub status for tag:56d6aaab-b821-4766-954f-c4c8c4ff1859 - {'path': '/', 'tag': '56d6aaab-b821-4766-954f-c4c8c4ff1859', 'options': 'recursive,force'} 2024-04-21T08:50:48.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:47 smithi088 ceph-mon[31259]: pgmap v375: 129 pgs: 129 active+clean; 2.7 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-21T08:50:48.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:47 smithi179 ceph-mon[31829]: pgmap v375: 129 pgs: 129 active+clean; 2.7 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 34 KiB/s wr, 12 op/s 2024-04-21T08:50:49.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:48 smithi088 ceph-mon[31259]: scrub queued for path: / 2024-04-21T08:50:49.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:48 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:50:49.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:48 smithi088 ceph-mon[31259]: scrub summary: active paths [/] 2024-04-21T08:50:49.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:48 smithi088 ceph-mon[31259]: scrub summary: idle+waiting paths [/] 2024-04-21T08:50:49.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:48 smithi179 ceph-mon[31829]: scrub queued for path: / 2024-04-21T08:50:49.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:48 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:50:49.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:48 smithi179 ceph-mon[31829]: scrub summary: active paths [/] 2024-04-21T08:50:49.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:48 smithi179 ceph-mon[31829]: scrub summary: idle+waiting paths [/] 2024-04-21T08:50:49.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:48 smithi086 ceph-mon[27939]: scrub queued for path: / 2024-04-21T08:50:49.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:48 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:50:49.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:48 smithi086 ceph-mon[27939]: scrub summary: active paths [/] 2024-04-21T08:50:49.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:48 smithi086 ceph-mon[27939]: scrub summary: idle+waiting paths [/] 2024-04-21T08:50:50.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:49 smithi179 ceph-mon[31829]: pgmap v376: 129 pgs: 129 active+clean; 2.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-21T08:50:50.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:49 smithi179 ceph-mon[31829]: scrub summary: idle 2024-04-21T08:50:50.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:49 smithi086 ceph-mon[27939]: pgmap v376: 129 pgs: 129 active+clean; 2.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-21T08:50:50.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:49 smithi086 ceph-mon[27939]: scrub summary: idle 2024-04-21T08:50:50.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:49 smithi088 ceph-mon[31259]: pgmap v376: 129 pgs: 129 active+clean; 2.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 56 KiB/s rd, 58 KiB/s wr, 21 op/s 2024-04-21T08:50:50.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:49 smithi088 ceph-mon[31259]: scrub summary: idle 2024-04-21T08:50:52.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:51 smithi179 ceph-mon[31829]: pgmap v377: 129 pgs: 129 active+clean; 2.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:50:52.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:51 smithi086 ceph-mon[27939]: pgmap v377: 129 pgs: 129 active+clean; 2.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:50:52.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:51 smithi088 ceph-mon[31259]: pgmap v377: 129 pgs: 129 active+clean; 2.9 MiB data, 429 MiB used, 1.0 TiB / 1.0 TiB avail; 38 KiB/s rd, 41 KiB/s wr, 15 op/s 2024-04-21T08:50:54.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:53 smithi179 ceph-mon[31829]: pgmap v378: 129 pgs: 129 active+clean; 3.1 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 56 KiB/s wr, 23 op/s 2024-04-21T08:50:54.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:53 smithi086 ceph-mon[27939]: pgmap v378: 129 pgs: 129 active+clean; 3.1 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 56 KiB/s wr, 23 op/s 2024-04-21T08:50:54.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:53 smithi088 ceph-mon[31259]: pgmap v378: 129 pgs: 129 active+clean; 3.1 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 58 KiB/s rd, 56 KiB/s wr, 23 op/s 2024-04-21T08:50:56.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:55 smithi179 ceph-mon[31829]: pgmap v379: 129 pgs: 129 active+clean; 3.1 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 16 op/s 2024-04-21T08:50:56.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:55 smithi086 ceph-mon[27939]: pgmap v379: 129 pgs: 129 active+clean; 3.1 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 16 op/s 2024-04-21T08:50:56.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:55 smithi088 ceph-mon[31259]: pgmap v379: 129 pgs: 129 active+clean; 3.1 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 16 op/s 2024-04-21T08:50:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.593+0000 7f8da40f1640 1 -- 172.21.15.86:0/3134420343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d9c102b80 msgr2=0x7f8d9c106bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.593+0000 7f8da40f1640 1 --2- 172.21.15.86:0/3134420343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d9c102b80 0x7f8d9c106bc0 secure :-1 s=READY pgs=2074 cs=0 l=1 rev1=1 crypto rx=0x7f8d98009900 tx=0x7f8d9802bc30 comp rx=0 tx=0).stop 2024-04-21T08:50:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.593+0000 7f8da40f1640 1 -- 172.21.15.86:0/3134420343 shutdown_connections 2024-04-21T08:50:56.596 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.593+0000 7f8da40f1640 1 --2- 172.21.15.86:0/3134420343 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d9c1072a0 0x7f8d9c10ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.593+0000 7f8da40f1640 1 --2- 172.21.15.86:0/3134420343 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d9c102b80 0x7f8d9c106bc0 unknown :-1 s=CLOSED pgs=2074 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.593+0000 7f8da40f1640 1 --2- 172.21.15.86:0/3134420343 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d9c1021d0 0x7f8d9c1025b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.593+0000 7f8da40f1640 1 -- 172.21.15.86:0/3134420343 >> 172.21.15.86:0/3134420343 conn(0x7f8d9c0793e0 msgr2=0x7f8d9c0797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.593+0000 7f8da40f1640 1 -- 172.21.15.86:0/3134420343 shutdown_connections 2024-04-21T08:50:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.593+0000 7f8da40f1640 1 -- 172.21.15.86:0/3134420343 wait complete. 2024-04-21T08:50:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.594+0000 7f8da40f1640 1 Processor -- start 2024-04-21T08:50:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.594+0000 7f8da40f1640 1 -- start start 2024-04-21T08:50:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.594+0000 7f8da40f1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d9c1021d0 0x7f8d9c1986c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.594+0000 7f8da40f1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d9c1072a0 0x7f8d9c198c00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.594+0000 7f8da40f1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d9c19cf80 0x7f8d9c19d430 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.594+0000 7f8da40f1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9c1121c0 con 0x7f8d9c1072a0 2024-04-21T08:50:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.594+0000 7f8da40f1640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9c112040 con 0x7f8d9c19cf80 2024-04-21T08:50:56.597 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.594+0000 7f8da40f1640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9c112340 con 0x7f8d9c1021d0 2024-04-21T08:50:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.594+0000 7f8da168c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d9c1072a0 0x7f8d9c198c00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.594+0000 7f8da168c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d9c1072a0 0x7f8d9c198c00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:51986/0 (socket says 172.21.15.86:51986) 2024-04-21T08:50:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.594+0000 7f8da168c640 1 -- 172.21.15.86:0/1786728860 learned_addr learned my addr 172.21.15.86:0/1786728860 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.594+0000 7f8da268e640 1 --2- 172.21.15.86:0/1786728860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d9c19cf80 0x7f8d9c19d430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.594+0000 7f8da1e8d640 1 --2- 172.21.15.86:0/1786728860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d9c1021d0 0x7f8d9c1986c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.594+0000 7f8da168c640 1 -- 172.21.15.86:0/1786728860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d9c1021d0 msgr2=0x7f8d9c1986c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.594+0000 7f8da168c640 1 --2- 172.21.15.86:0/1786728860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d9c1021d0 0x7f8d9c1986c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.594+0000 7f8da168c640 1 -- 172.21.15.86:0/1786728860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d9c19cf80 msgr2=0x7f8d9c19d430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.594+0000 7f8da168c640 1 --2- 172.21.15.86:0/1786728860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d9c19cf80 0x7f8d9c19d430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.594+0000 7f8da168c640 1 -- 172.21.15.86:0/1786728860 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8d9c19da70 con 0x7f8d9c1072a0 2024-04-21T08:50:56.598 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.594+0000 7f8da1e8d640 1 --2- 172.21.15.86:0/1786728860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d9c1021d0 0x7f8d9c1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.595+0000 7f8da168c640 1 --2- 172.21.15.86:0/1786728860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d9c1072a0 0x7f8d9c198c00 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f8d98036040 tx=0x7f8d980045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.595+0000 7f8d8effd640 1 -- 172.21.15.86:0/1786728860 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d98043020 con 0x7f8d9c1072a0 2024-04-21T08:50:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.595+0000 7f8d8effd640 1 -- 172.21.15.86:0/1786728860 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d9802ca70 con 0x7f8d9c1072a0 2024-04-21T08:50:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.595+0000 7f8da40f1640 1 -- 172.21.15.86:0/1786728860 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8d9c19dd00 con 0x7f8d9c1072a0 2024-04-21T08:50:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.595+0000 7f8d8effd640 1 -- 172.21.15.86:0/1786728860 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d98003ac0 con 0x7f8d9c1072a0 2024-04-21T08:50:56.599 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.595+0000 7f8da40f1640 1 -- 172.21.15.86:0/1786728860 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8d9c1a5520 con 0x7f8d9c1072a0 2024-04-21T08:50:56.606 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.598+0000 7f8d8effd640 1 -- 172.21.15.86:0/1786728860 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8d98046020 con 0x7f8d9c1072a0 2024-04-21T08:50:56.606 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.598+0000 7f8da6db7740 1 Processor -- start 2024-04-21T08:50:56.606 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.598+0000 7f8da6db7740 1 -- start start 2024-04-21T08:50:56.606 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.598+0000 7f8da6db7740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5653eedaf8d0 0x5653eedae620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:56.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.598+0000 7f8da6db7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653eedaeb60 0x5653eedaefe0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:56.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.598+0000 7f8da6db7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653eedb80e0 0x5653eedba4d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:56.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.598+0000 7f8da6db7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5653eed138d0 con 0x5653eedaeb60 2024-04-21T08:50:56.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.598+0000 7f8da6db7740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5653eecb0e80 con 0x5653eedaf8d0 2024-04-21T08:50:56.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.598+0000 7f8da6db7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5653eed0f890 con 0x5653eedb80e0 2024-04-21T08:50:56.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.598+0000 7f8d8effd640 1 --2- 172.21.15.86:0/1786728860 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8d880a3dc0 0x7f8d880a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:56.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.599+0000 7f8da1e8d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653eedaeb60 0x5653eedaefe0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:56.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.599+0000 7f8da1e8d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653eedaeb60 0x5653eedaefe0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52000/0 (socket says 172.21.15.86:52000) 2024-04-21T08:50:56.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.599+0000 7f8da1e8d640 1 -- 172.21.15.86:0/3159048519 learned_addr learned my addr 172.21.15.86:0/3159048519 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:56.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.599+0000 7f8da268e640 1 --2- 172.21.15.86:0/1786728860 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8d880a3dc0 0x7f8d880a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:56.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.599+0000 7f8da268e640 1 --2- 172.21.15.86:0/3159048519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5653eedaf8d0 0x5653eedae620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:56.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.599+0000 7f8da168c640 1 --2- 172.21.15.86:0/3159048519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653eedb80e0 0x5653eedba4d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:56.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.599+0000 7f8d8effd640 1 -- 172.21.15.86:0/1786728860 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f8d980f2c90 con 0x7f8d9c1072a0 2024-04-21T08:50:56.607 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.599+0000 7f8da1e8d640 1 -- 172.21.15.86:0/3159048519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653eedb80e0 msgr2=0x5653eedba4d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:56.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.599+0000 7f8da1e8d640 1 --2- 172.21.15.86:0/3159048519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653eedb80e0 0x5653eedba4d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:56.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.599+0000 7f8da1e8d640 1 -- 172.21.15.86:0/3159048519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5653eedaf8d0 msgr2=0x5653eedae620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:56.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.599+0000 7f8da1e8d640 1 --2- 172.21.15.86:0/3159048519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5653eedaf8d0 0x5653eedae620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:56.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.599+0000 7f8da1e8d640 1 -- 172.21.15.86:0/3159048519 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653eecd2320 con 0x5653eedaeb60 2024-04-21T08:50:56.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.599+0000 7f8da168c640 1 --2- 172.21.15.86:0/3159048519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653eedb80e0 0x5653eedba4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:56.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.599+0000 7f8da268e640 1 --2- 172.21.15.86:0/3159048519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5653eedaf8d0 0x5653eedae620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:56.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.599+0000 7f8da1e8d640 1 --2- 172.21.15.86:0/3159048519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653eedaeb60 0x5653eedaefe0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f8d90009c30 tx=0x7f8d90010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=a387c2c33deaca4c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:56.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.599+0000 7f8da268e640 1 --2- 172.21.15.86:0/1786728860 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8d880a3dc0 0x7f8d880a6280 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8d94009fd0 tx=0x7f8d94009b90 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:56.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.599+0000 7f8d87fff640 1 -- 172.21.15.86:0/3159048519 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d90004030 con 0x5653eedaeb60 2024-04-21T08:50:56.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.599+0000 7f8d87fff640 1 -- 172.21.15.86:0/3159048519 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d900041d0 con 0x5653eedaeb60 2024-04-21T08:50:56.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.599+0000 7f8d87fff640 1 -- 172.21.15.86:0/3159048519 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d900126b0 con 0x5653eedaeb60 2024-04-21T08:50:56.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.600+0000 7f8da6db7740 1 -- 172.21.15.86:0/3159048519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653eedaeb60 msgr2=0x5653eedaefe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:56.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.600+0000 7f8da6db7740 1 --2- 172.21.15.86:0/3159048519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653eedaeb60 0x5653eedaefe0 secure :-1 s=READY pgs=2076 cs=0 l=1 rev1=1 crypto rx=0x7f8d90009c30 tx=0x7f8d90010740 comp rx=0 tx=0).stop 2024-04-21T08:50:56.608 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.600+0000 7f8da6db7740 1 -- 172.21.15.86:0/3159048519 shutdown_connections 2024-04-21T08:50:56.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.600+0000 7f8da6db7740 1 --2- 172.21.15.86:0/3159048519 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653eedb80e0 0x5653eedba4d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:56.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.600+0000 7f8da6db7740 1 --2- 172.21.15.86:0/3159048519 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653eedaeb60 0x5653eedaefe0 unknown :-1 s=CLOSED pgs=2076 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:56.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.600+0000 7f8da6db7740 1 --2- 172.21.15.86:0/3159048519 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5653eedaf8d0 0x5653eedae620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:56.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.600+0000 7f8da6db7740 1 -- 172.21.15.86:0/3159048519 >> 172.21.15.86:0/3159048519 conn(0x5653eed75c90 msgr2=0x5653eeda9850 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:56.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.600+0000 7f8da6db7740 1 -- 172.21.15.86:0/3159048519 shutdown_connections 2024-04-21T08:50:56.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.600+0000 7f8da6db7740 1 -- 172.21.15.86:0/3159048519 wait complete. 2024-04-21T08:50:56.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.600+0000 7f8da6db7740 1 -- start start 2024-04-21T08:50:56.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.601+0000 7f8da6db7740 1 Processor -- start 2024-04-21T08:50:56.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.601+0000 7f8da6db7740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5653eedaeb60 0x5653eedb7aa0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:56.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.601+0000 7f8da6db7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653eedaf8d0 0x5653eedb4e30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:56.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.601+0000 7f8da6db7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653eedb80e0 0x5653eedb5440 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:56.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.601+0000 7f8da6db7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x5653eedbae10 con 0x5653eedb80e0 2024-04-21T08:50:56.609 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.601+0000 7f8da6db7740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x5653eec9beb0 con 0x5653eedaeb60 2024-04-21T08:50:56.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.601+0000 7f8da6db7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x5653eedbaf90 con 0x5653eedaf8d0 2024-04-21T08:50:56.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.601+0000 7f8da6db7740 10 client.0 _ll_register_callbacks cb 0x5653eecc8b40 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:56.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.601+0000 7f8da268e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653eedb80e0 0x5653eedb5440 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:56.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.601+0000 7f8da268e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653eedb80e0 0x5653eedb5440 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52010/0 (socket says 172.21.15.86:52010) 2024-04-21T08:50:56.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.601+0000 7f8da268e640 1 -- 172.21.15.86:0/3735548442 learned_addr learned my addr 172.21.15.86:0/3735548442 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:56.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.601+0000 7f8da1e8d640 1 --2- 172.21.15.86:0/3735548442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5653eedaeb60 0x5653eedb7aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:56.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.601+0000 7f8da168c640 1 --2- 172.21.15.86:0/3735548442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653eedaf8d0 0x5653eedb4e30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:56.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.601+0000 7f8da268e640 1 -- 172.21.15.86:0/3735548442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653eedaf8d0 msgr2=0x5653eedb4e30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:56.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.601+0000 7f8da268e640 1 --2- 172.21.15.86:0/3735548442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653eedaf8d0 0x5653eedb4e30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:56.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.601+0000 7f8da268e640 1 -- 172.21.15.86:0/3735548442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5653eedaeb60 msgr2=0x5653eedb7aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:56.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.601+0000 7f8da268e640 1 --2- 172.21.15.86:0/3735548442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5653eedaeb60 0x5653eedb7aa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:56.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.601+0000 7f8da268e640 1 -- 172.21.15.86:0/3735548442 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653eea26210 con 0x5653eedb80e0 2024-04-21T08:50:56.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.601+0000 7f8da168c640 1 --2- 172.21.15.86:0/3735548442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653eedaf8d0 0x5653eedb4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:50:56.610 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.602+0000 7f8da268e640 1 --2- 172.21.15.86:0/3735548442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653eedb80e0 0x5653eedb5440 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f8d94009b60 tx=0x7f8d94004ab0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b8da58b82a86d9a3 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:56.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.602+0000 7f8d857fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:56.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.602+0000 7f8d857fa640 1 -- 172.21.15.86:0/3735548442 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9402c060 con 0x5653eedb80e0 2024-04-21T08:50:56.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.602+0000 7f8d857fa640 1 -- 172.21.15.86:0/3735548442 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f8d940045c0 con 0x5653eedb80e0 2024-04-21T08:50:56.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.602+0000 7f8da6db7740 1 -- 172.21.15.86:0/3735548442 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d900041d0 con 0x5653eedb80e0 2024-04-21T08:50:56.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.602+0000 7f8d857fa640 1 -- 172.21.15.86:0/3735548442 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8d94033050 con 0x5653eedb80e0 2024-04-21T08:50:56.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.602+0000 7f8da6db7740 10 client.20466 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:56.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.602+0000 7f8da6db7740 1 -- 172.21.15.86:0/3735548442 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5653eeaf2620 con 0x5653eedb80e0 2024-04-21T08:50:56.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.602+0000 7f8d857fa640 1 -- 172.21.15.86:0/3735548442 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d900041d0 con 0x5653eedb80e0 2024-04-21T08:50:56.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.602+0000 7f8d857fa640 1 -- 172.21.15.86:0/3735548442 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8d94026030 con 0x5653eedb80e0 2024-04-21T08:50:56.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.602+0000 7f8da6db7740 10 client.20466 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:56.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.602+0000 7f8da6db7740 10 client.20466 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:56.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.602+0000 7f8da6db7740 20 client.20466 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:56.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.602+0000 7f8da6db7740 1 --2- 172.21.15.86:0/3735548442 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5653eedc8160 0x5653eedca590 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:56.611 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.602+0000 7f8da6db7740 4 client.20466 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:50:56.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.602+0000 7f8da6db7740 1 -- 172.21.15.86:0/3735548442 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x5653eed27880 con 0x5653eedc8160 2024-04-21T08:50:56.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.603+0000 7f8da1e8d640 1 --2- 172.21.15.86:0/3735548442 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5653eedc8160 0x5653eedca590 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:56.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.603+0000 7f8da1e8d640 1 --2- 172.21.15.86:0/3735548442 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5653eedc8160 0x5653eedca590 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=566615f0b6be09e3 server_cookie=3feabc031810e860 in_seq=0 out_seq=0 2024-04-21T08:50:56.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.603+0000 7f8d857fa640 10 client.20466 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:56.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.604+0000 7f8d857fa640 1 -- 172.21.15.86:0/3735548442 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x5653eed27880 con 0x5653eedc8160 2024-04-21T08:50:56.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.604+0000 7f8d857fa640 10 client.20466 handle_command_reply: tid=0 2024-04-21T08:50:56.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.604+0000 7f8da6db7740 1 client.20466 shutdown 2024-04-21T08:50:56.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.604+0000 7f8da6db7740 1 -- 172.21.15.86:0/3735548442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653eedb80e0 msgr2=0x5653eedb5440 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:56.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.604+0000 7f8da6db7740 1 --2- 172.21.15.86:0/3735548442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653eedb80e0 0x5653eedb5440 secure :-1 s=READY pgs=2077 cs=0 l=1 rev1=1 crypto rx=0x7f8d94009b60 tx=0x7f8d94004ab0 comp rx=0 tx=0).stop 2024-04-21T08:50:56.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.604+0000 7f8da6db7740 1 -- 172.21.15.86:0/3735548442 shutdown_connections 2024-04-21T08:50:56.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.604+0000 7f8da6db7740 1 --2- 172.21.15.86:0/3735548442 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5653eedc8160 0x5653eedca590 crc :-1 s=READY pgs=347 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:56.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.604+0000 7f8da6db7740 1 --2- 172.21.15.86:0/3735548442 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653eedb80e0 0x5653eedb5440 unknown :-1 s=CLOSED pgs=2077 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:56.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.604+0000 7f8da6db7740 1 --2- 172.21.15.86:0/3735548442 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653eedaf8d0 0x5653eedb4e30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:56.612 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.604+0000 7f8d857fa640 0 client.20466 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:56.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.604+0000 7f8da6db7740 1 --2- 172.21.15.86:0/3735548442 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5653eedaeb60 0x5653eedb7aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:56.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.604+0000 7f8da6db7740 1 -- 172.21.15.86:0/3735548442 >> 172.21.15.86:0/3735548442 conn(0x5653eed75c90 msgr2=0x5653eedab5a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:56.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.604+0000 7f8da6db7740 1 -- 172.21.15.86:0/3735548442 shutdown_connections 2024-04-21T08:50:56.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.604+0000 7f8da6db7740 1 -- 172.21.15.86:0/3735548442 wait complete. 2024-04-21T08:50:56.613 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.604+0000 7f8da6db7740 20 client.20466 trim_cache size 0 max 16384 2024-04-21T08:50:56.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.619+0000 7f8da6db7740 1 Processor -- start 2024-04-21T08:50:56.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.619+0000 7f8da6db7740 1 -- start start 2024-04-21T08:50:56.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.619+0000 7f8da6db7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653eedaeb60 0x5653eedc3d40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:56.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.619+0000 7f8da6db7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653eedaf8d0 0x5653eedb3e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:56.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.619+0000 7f8da6db7740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5653eedb80e0 0x5653eedb7560 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:56.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.619+0000 7f8da6db7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9c1af360 con 0x5653eedaf8d0 2024-04-21T08:50:56.624 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.619+0000 7f8da6db7740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9c1b0ea0 con 0x5653eedb80e0 2024-04-21T08:50:56.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.619+0000 7f8da6db7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9c1afa30 con 0x5653eedaeb60 2024-04-21T08:50:56.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.619+0000 7f8da168c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653eedaf8d0 0x5653eedb3e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:56.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.619+0000 7f8da168c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653eedaf8d0 0x5653eedb3e10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52012/0 (socket says 172.21.15.86:52012) 2024-04-21T08:50:56.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.619+0000 7f8da168c640 1 -- 172.21.15.86:0/1628833202 learned_addr learned my addr 172.21.15.86:0/1628833202 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:56.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.619+0000 7f8da1e8d640 1 --2- 172.21.15.86:0/1628833202 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653eedaeb60 0x5653eedc3d40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:56.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.619+0000 7f8da268e640 1 --2- 172.21.15.86:0/1628833202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5653eedb80e0 0x5653eedb7560 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:56.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.619+0000 7f8da168c640 1 -- 172.21.15.86:0/1628833202 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653eedaeb60 msgr2=0x5653eedc3d40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:56.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.619+0000 7f8da168c640 1 --2- 172.21.15.86:0/1628833202 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653eedaeb60 0x5653eedc3d40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:56.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.619+0000 7f8da168c640 1 -- 172.21.15.86:0/1628833202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5653eedb80e0 msgr2=0x5653eedb7560 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:56.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.619+0000 7f8da168c640 1 --2- 172.21.15.86:0/1628833202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5653eedb80e0 0x5653eedb7560 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:56.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.619+0000 7f8da168c640 1 -- 172.21.15.86:0/1628833202 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x5653eedc6d00 con 0x5653eedaf8d0 2024-04-21T08:50:56.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.619+0000 7f8da268e640 1 --2- 172.21.15.86:0/1628833202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5653eedb80e0 0x5653eedb7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:50:56.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.619+0000 7f8da168c640 1 --2- 172.21.15.86:0/1628833202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653eedaf8d0 0x5653eedb3e10 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f8d980f0610 tx=0x7f8d9802bc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fe3faa9e12ba4440 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:56.625 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.620+0000 7f8d877fe640 1 -- 172.21.15.86:0/1628833202 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9802cc20 con 0x5653eedaf8d0 2024-04-21T08:50:56.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.620+0000 7f8d877fe640 1 -- 172.21.15.86:0/1628833202 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d980be970 con 0x5653eedaf8d0 2024-04-21T08:50:56.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.620+0000 7f8d877fe640 1 -- 172.21.15.86:0/1628833202 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d980053d0 con 0x5653eedaf8d0 2024-04-21T08:50:56.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.620+0000 7f8da6db7740 1 -- 172.21.15.86:0/1628833202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653eedaf8d0 msgr2=0x5653eedb3e10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:56.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.620+0000 7f8da6db7740 1 --2- 172.21.15.86:0/1628833202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653eedaf8d0 0x5653eedb3e10 secure :-1 s=READY pgs=2078 cs=0 l=1 rev1=1 crypto rx=0x7f8d980f0610 tx=0x7f8d9802bc30 comp rx=0 tx=0).stop 2024-04-21T08:50:56.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.620+0000 7f8da6db7740 1 -- 172.21.15.86:0/1628833202 shutdown_connections 2024-04-21T08:50:56.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.620+0000 7f8da6db7740 1 --2- 172.21.15.86:0/1628833202 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5653eedb80e0 0x5653eedb7560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:56.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.620+0000 7f8da6db7740 1 --2- 172.21.15.86:0/1628833202 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653eedaf8d0 0x5653eedb3e10 unknown :-1 s=CLOSED pgs=2078 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:56.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.620+0000 7f8da6db7740 1 --2- 172.21.15.86:0/1628833202 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653eedaeb60 0x5653eedc3d40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:56.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.620+0000 7f8da6db7740 1 -- 172.21.15.86:0/1628833202 >> 172.21.15.86:0/1628833202 conn(0x5653eed75c90 msgr2=0x5653eedb3700 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:56.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.620+0000 7f8da6db7740 1 -- 172.21.15.86:0/1628833202 shutdown_connections 2024-04-21T08:50:56.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.620+0000 7f8da6db7740 1 -- 172.21.15.86:0/1628833202 wait complete. 2024-04-21T08:50:56.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.621+0000 7f8da6db7740 1 -- start start 2024-04-21T08:50:56.626 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.621+0000 7f8da6db7740 1 Processor -- start 2024-04-21T08:50:56.627 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.621+0000 7f8da6db7740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653eedaeb60 0x5653eedbc2a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:56.627 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.621+0000 7f8da6db7740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5653eedaf8d0 0x5653eedbc7e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:56.627 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.621+0000 7f8da6db7740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653eedb80e0 0x5653eee2d710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:56.627 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.621+0000 7f8da6db7740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9c1b07d0 con 0x5653eedb80e0 2024-04-21T08:50:56.627 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.621+0000 7f8da6db7740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9c1b0100 con 0x5653eedaf8d0 2024-04-21T08:50:56.627 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.621+0000 7f8da6db7740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8d9c1ad360 con 0x5653eedaeb60 2024-04-21T08:50:56.627 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.621+0000 7f8da6db7740 10 client.0 _ll_register_callbacks cb 0x7f8d9c06b7d0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:50:56.627 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.621+0000 7f8da268e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653eedb80e0 0x5653eee2d710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:56.627 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.621+0000 7f8da268e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653eedb80e0 0x5653eee2d710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:52028/0 (socket says 172.21.15.86:52028) 2024-04-21T08:50:56.627 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.621+0000 7f8da268e640 1 -- 172.21.15.86:0/2287280129 learned_addr learned my addr 172.21.15.86:0/2287280129 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:50:56.627 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.621+0000 7f8da168c640 1 --2- 172.21.15.86:0/2287280129 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5653eedaf8d0 0x5653eedbc7e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:56.627 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.621+0000 7f8da1e8d640 1 --2- 172.21.15.86:0/2287280129 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653eedaeb60 0x5653eedbc2a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:56.627 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.621+0000 7f8da268e640 1 -- 172.21.15.86:0/2287280129 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653eedaeb60 msgr2=0x5653eedbc2a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:56.627 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.621+0000 7f8da268e640 1 --2- 172.21.15.86:0/2287280129 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653eedaeb60 0x5653eedbc2a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:56.628 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.621+0000 7f8da268e640 1 -- 172.21.15.86:0/2287280129 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5653eedaf8d0 msgr2=0x5653eedbc7e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:50:56.628 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.621+0000 7f8da268e640 1 --2- 172.21.15.86:0/2287280129 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5653eedaf8d0 0x5653eedbc7e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:56.628 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.621+0000 7f8da268e640 1 -- 172.21.15.86:0/2287280129 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x5653eedade30 con 0x5653eedb80e0 2024-04-21T08:50:56.628 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.622+0000 7f8da268e640 1 --2- 172.21.15.86:0/2287280129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653eedb80e0 0x5653eee2d710 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f8d94000f80 tx=0x7f8d9401e8d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b50a81d8b57e5a9f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:50:56.628 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.622+0000 7f8d857fa640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:50:56.628 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.622+0000 7f8d857fa640 1 -- 172.21.15.86:0/2287280129 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8d9401ed80 con 0x5653eedb80e0 2024-04-21T08:50:56.628 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.622+0000 7f8d857fa640 1 -- 172.21.15.86:0/2287280129 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(106..106 src has 1..106) ==== 9496+0+0 (secure 0 0 0) 0x7f8d94004410 con 0x5653eedb80e0 2024-04-21T08:50:56.628 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.622+0000 7f8da6db7740 1 -- 172.21.15.86:0/2287280129 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f8d980be970 con 0x5653eedb80e0 2024-04-21T08:50:56.628 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.622+0000 7f8d857fa640 1 -- 172.21.15.86:0/2287280129 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f8d94009570 con 0x5653eedb80e0 2024-04-21T08:50:56.628 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.622+0000 7f8da6db7740 10 client.20472 fetch_fsmap learned FSMap version 29 2024-04-21T08:50:56.628 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.622+0000 7f8da6db7740 1 -- 172.21.15.86:0/2287280129 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x5653eee2ec00 con 0x5653eedb80e0 2024-04-21T08:50:56.628 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.622+0000 7f8d857fa640 1 -- 172.21.15.86:0/2287280129 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8d980be970 con 0x5653eedb80e0 2024-04-21T08:50:56.628 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.622+0000 7f8d857fa640 1 -- 172.21.15.86:0/2287280129 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f8d94026040 con 0x5653eedb80e0 2024-04-21T08:50:56.628 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.622+0000 7f8da6db7740 10 client.20472 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:50:56.628 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.622+0000 7f8da6db7740 10 client.20472 resolve_mds: resolved cephfs:0 to role '1:0' aka daemon mds.b 2024-04-21T08:50:56.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.622+0000 7f8da6db7740 20 client.20472 populate_metadata read hostname 'smithi086' 2024-04-21T08:50:56.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.622+0000 7f8da6db7740 1 --2- 172.21.15.86:0/2287280129 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5653eee2f3d0 0x5653eee317f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:50:56.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.622+0000 7f8da6db7740 4 client.20472 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "flush journal"}] 2024-04-21T08:50:56.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.622+0000 7f8da6db7740 1 -- 172.21.15.86:0/2287280129 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "flush journal"}) -- 0x5653eecb33d0 con 0x5653eee2f3d0 2024-04-21T08:50:56.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.623+0000 7f8da1e8d640 1 --2- 172.21.15.86:0/2287280129 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5653eee2f3d0 0x5653eee317f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:50:56.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.623+0000 7f8d857fa640 10 client.20472 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:56.629 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.623+0000 7f8da1e8d640 1 --2- 172.21.15.86:0/2287280129 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5653eee2f3d0 0x5653eee317f0 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=25456a0fcb574082 server_cookie=6bcfaab96e65fcf4 in_seq=0 out_seq=0 2024-04-21T08:50:56.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.637+0000 7f8d857fa640 1 -- 172.21.15.86:0/2287280129 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+44 (crc 0 0 0) 0x5653eecb33d0 con 0x5653eee2f3d0 2024-04-21T08:50:56.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.637+0000 7f8d857fa640 10 client.20472 handle_command_reply: tid=0 2024-04-21T08:50:56.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.637+0000 7f8da6db7740 1 client.20472 shutdown 2024-04-21T08:50:56.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.637+0000 7f8da6db7740 1 -- 172.21.15.86:0/2287280129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653eedb80e0 msgr2=0x5653eee2d710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:56.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.637+0000 7f8da6db7740 1 --2- 172.21.15.86:0/2287280129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653eedb80e0 0x5653eee2d710 secure :-1 s=READY pgs=2079 cs=0 l=1 rev1=1 crypto rx=0x7f8d94000f80 tx=0x7f8d9401e8d0 comp rx=0 tx=0).stop 2024-04-21T08:50:56.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.637+0000 7f8da6db7740 1 -- 172.21.15.86:0/2287280129 shutdown_connections 2024-04-21T08:50:56.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.637+0000 7f8da6db7740 1 --2- 172.21.15.86:0/2287280129 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x5653eee2f3d0 0x5653eee317f0 crc :-1 s=READY pgs=348 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:56.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.637+0000 7f8da6db7740 1 --2- 172.21.15.86:0/2287280129 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x5653eedb80e0 0x5653eee2d710 unknown :-1 s=CLOSED pgs=2079 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:56.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.637+0000 7f8d857fa640 0 client.20472 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:50:56.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.637+0000 7f8da6db7740 1 --2- 172.21.15.86:0/2287280129 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x5653eedaf8d0 0x5653eedbc7e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:56.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.637+0000 7f8da6db7740 1 --2- 172.21.15.86:0/2287280129 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x5653eedaeb60 0x5653eedbc2a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:56.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.637+0000 7f8da6db7740 1 -- 172.21.15.86:0/2287280129 >> 172.21.15.86:0/2287280129 conn(0x5653eed75c90 msgr2=0x5653eedb3f90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:56.638 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.637+0000 7f8da6db7740 1 -- 172.21.15.86:0/2287280129 shutdown_connections 2024-04-21T08:50:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.637+0000 7f8da6db7740 1 -- 172.21.15.86:0/2287280129 wait complete. 2024-04-21T08:50:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.637+0000 7f8da6db7740 20 client.20472 trim_cache size 0 max 16384 2024-04-21T08:50:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:{ 2024-04-21T08:50:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "message": "", 2024-04-21T08:50:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout: "return_code": 0 2024-04-21T08:50:56.639 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stdout:} 2024-04-21T08:50:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.638+0000 7f8da40f1640 1 -- 172.21.15.86:0/1786728860 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8d880a3dc0 msgr2=0x7f8d880a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.638+0000 7f8da40f1640 1 --2- 172.21.15.86:0/1786728860 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8d880a3dc0 0x7f8d880a6280 secure :-1 s=READY pgs=347 cs=0 l=1 rev1=1 crypto rx=0x7f8d94009fd0 tx=0x7f8d94009b90 comp rx=0 tx=0).stop 2024-04-21T08:50:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.638+0000 7f8da40f1640 1 -- 172.21.15.86:0/1786728860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d9c1072a0 msgr2=0x7f8d9c198c00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:50:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.638+0000 7f8da40f1640 1 --2- 172.21.15.86:0/1786728860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d9c1072a0 0x7f8d9c198c00 secure :-1 s=READY pgs=2075 cs=0 l=1 rev1=1 crypto rx=0x7f8d98036040 tx=0x7f8d980045e0 comp rx=0 tx=0).stop 2024-04-21T08:50:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.638+0000 7f8da40f1640 1 -- 172.21.15.86:0/1786728860 shutdown_connections 2024-04-21T08:50:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.638+0000 7f8da40f1640 1 --2- 172.21.15.86:0/1786728860 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8d880a3dc0 0x7f8d880a6280 unknown :-1 s=CLOSED pgs=347 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.638+0000 7f8da40f1640 1 --2- 172.21.15.86:0/1786728860 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8d9c19cf80 0x7f8d9c19d430 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:56.640 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.638+0000 7f8da40f1640 1 --2- 172.21.15.86:0/1786728860 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8d9c1072a0 0x7f8d9c198c00 unknown :-1 s=CLOSED pgs=2075 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.638+0000 7f8da40f1640 1 --2- 172.21.15.86:0/1786728860 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8d9c1021d0 0x7f8d9c1986c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:50:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.638+0000 7f8da40f1640 1 -- 172.21.15.86:0/1786728860 >> 172.21.15.86:0/1786728860 conn(0x7f8d9c0793e0 msgr2=0x7f8d9c0ffe00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:50:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.638+0000 7f8da40f1640 1 -- 172.21.15.86:0/1786728860 shutdown_connections 2024-04-21T08:50:56.641 INFO:teuthology.task.background_exec.ubuntu@smithi086.front.sepia.ceph.com.smithi086.stderr:2024-04-21T08:50:56.638+0000 7f8da40f1640 1 -- 172.21.15.86:0/1786728860 wait complete. 2024-04-21T08:50:58.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:57 smithi179 ceph-mon[31829]: pgmap v380: 129 pgs: 129 active+clean; 3.1 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 16 op/s 2024-04-21T08:50:58.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:57 smithi086 ceph-mon[27939]: pgmap v380: 129 pgs: 129 active+clean; 3.1 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 16 op/s 2024-04-21T08:50:58.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:57 smithi088 ceph-mon[31259]: pgmap v380: 129 pgs: 129 active+clean; 3.1 MiB data, 430 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 38 KiB/s wr, 16 op/s 2024-04-21T08:51:00.248 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:50:59 smithi179 ceph-mon[31829]: pgmap v381: 129 pgs: 129 active+clean; 3.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 63 KiB/s wr, 25 op/s 2024-04-21T08:51:00.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:50:59 smithi086 ceph-mon[27939]: pgmap v381: 129 pgs: 129 active+clean; 3.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 63 KiB/s wr, 25 op/s 2024-04-21T08:51:00.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:50:59 smithi088 ceph-mon[31259]: pgmap v381: 129 pgs: 129 active+clean; 3.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 63 KiB/s wr, 25 op/s 2024-04-21T08:51:02.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:01 smithi179 ceph-mon[31829]: pgmap v382: 129 pgs: 129 active+clean; 3.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-21T08:51:02.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:01 smithi086 ceph-mon[27939]: pgmap v382: 129 pgs: 129 active+clean; 3.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-21T08:51:02.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:01 smithi088 ceph-mon[31259]: pgmap v382: 129 pgs: 129 active+clean; 3.3 MiB data, 431 MiB used, 1.0 TiB / 1.0 TiB avail; 40 KiB/s rd, 39 KiB/s wr, 16 op/s 2024-04-21T08:51:04.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:03 smithi179 ceph-mon[31829]: pgmap v383: 129 pgs: 129 active+clean; 3.5 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 103 KiB/s wr, 39 op/s 2024-04-21T08:51:04.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:03 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:51:04.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:03 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:51:04.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:03 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:51:04.266 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:03 smithi088 ceph-mon[31259]: pgmap v383: 129 pgs: 129 active+clean; 3.5 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 103 KiB/s wr, 39 op/s 2024-04-21T08:51:04.266 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:03 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:51:04.266 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:03 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:51:04.266 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:03 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:51:04.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:03 smithi086 ceph-mon[27939]: pgmap v383: 129 pgs: 129 active+clean; 3.5 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 70 KiB/s rd, 103 KiB/s wr, 39 op/s 2024-04-21T08:51:04.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:03 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config dump", "format": "json"}]: dispatch 2024-04-21T08:51:04.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:03 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:51:04.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:03 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:51:05.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:04 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:51:05.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:04 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:51:05.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:04 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:51:05.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:04 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:51:05.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:04 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:51:05.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:04 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:51:05.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:04 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "config generate-minimal-conf"}]: dispatch 2024-04-21T08:51:05.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:04 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "auth get", "entity": "client.admin"}]: dispatch 2024-04-21T08:51:05.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:04 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' 2024-04-21T08:51:06.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:05 smithi179 ceph-mon[31829]: pgmap v384: 129 pgs: 129 active+clean; 3.5 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 88 KiB/s wr, 32 op/s 2024-04-21T08:51:06.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:05 smithi179 ceph-mon[31829]: osdmap e107: 12 total, 12 up, 12 in 2024-04-21T08:51:06.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:05 smithi086 ceph-mon[27939]: pgmap v384: 129 pgs: 129 active+clean; 3.5 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 88 KiB/s wr, 32 op/s 2024-04-21T08:51:06.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:05 smithi086 ceph-mon[27939]: osdmap e107: 12 total, 12 up, 12 in 2024-04-21T08:51:06.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:05 smithi088 ceph-mon[31259]: pgmap v384: 129 pgs: 129 active+clean; 3.5 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 50 KiB/s rd, 88 KiB/s wr, 32 op/s 2024-04-21T08:51:06.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:05 smithi088 ceph-mon[31259]: osdmap e107: 12 total, 12 up, 12 in 2024-04-21T08:51:06.879 INFO:tasks.workunit.client.0.smithi086.stdout:561/600: open fd = 3 2024-04-21T08:51:06.879 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.879 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.879 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.879 INFO:tasks.workunit.client.0.smithi086.stdout:562/600: open fd = 3 2024-04-21T08:51:06.879 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.879 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.879 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.879 INFO:tasks.workunit.client.0.smithi086.stdout:563/600: open fd = 3 2024-04-21T08:51:06.879 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.880 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.880 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.880 INFO:tasks.workunit.client.0.smithi086.stdout:564/600: open fd = 3 2024-04-21T08:51:06.880 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.880 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.880 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.880 INFO:tasks.workunit.client.0.smithi086.stdout:565/600: open fd = 3 2024-04-21T08:51:06.880 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.880 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.880 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.880 INFO:tasks.workunit.client.0.smithi086.stdout:566/600: open fd = 3 2024-04-21T08:51:06.881 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.881 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.881 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.881 INFO:tasks.workunit.client.0.smithi086.stdout:567/600: open fd = 3 2024-04-21T08:51:06.881 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.881 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.881 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.881 INFO:tasks.workunit.client.0.smithi086.stdout:568/600: open fd = 3 2024-04-21T08:51:06.881 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.882 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.882 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.882 INFO:tasks.workunit.client.0.smithi086.stdout:569/600: open fd = 3 2024-04-21T08:51:06.882 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.882 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.882 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.882 INFO:tasks.workunit.client.0.smithi086.stdout:570/600: open fd = 3 2024-04-21T08:51:06.882 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.882 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.882 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.882 INFO:tasks.workunit.client.0.smithi086.stdout:571/600: open fd = 3 2024-04-21T08:51:06.882 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.882 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.882 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.883 INFO:tasks.workunit.client.0.smithi086.stdout:572/600: open fd = 3 2024-04-21T08:51:06.883 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.883 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.883 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.883 INFO:tasks.workunit.client.0.smithi086.stdout:573/600: open fd = 3 2024-04-21T08:51:06.883 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.883 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.883 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.883 INFO:tasks.workunit.client.0.smithi086.stdout:574/600: open fd = 3 2024-04-21T08:51:06.883 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.883 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.884 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.884 INFO:tasks.workunit.client.0.smithi086.stdout:575/600: open fd = 3 2024-04-21T08:51:06.884 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.884 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.884 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.884 INFO:tasks.workunit.client.0.smithi086.stdout:576/600: open fd = 3 2024-04-21T08:51:06.884 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.884 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.884 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.884 INFO:tasks.workunit.client.0.smithi086.stdout:577/600: open fd = 3 2024-04-21T08:51:06.884 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.884 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.884 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.885 INFO:tasks.workunit.client.0.smithi086.stdout:578/600: open fd = 3 2024-04-21T08:51:06.885 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.885 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.885 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.885 INFO:tasks.workunit.client.0.smithi086.stdout:579/600: open fd = 3 2024-04-21T08:51:06.885 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.885 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.885 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.885 INFO:tasks.workunit.client.0.smithi086.stdout:580/600: open fd = 3 2024-04-21T08:51:06.885 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.885 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.885 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.885 INFO:tasks.workunit.client.0.smithi086.stdout:581/600: open fd = 3 2024-04-21T08:51:06.885 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.885 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.886 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.886 INFO:tasks.workunit.client.0.smithi086.stdout:582/600: open fd = 3 2024-04-21T08:51:06.886 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.886 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.886 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.886 INFO:tasks.workunit.client.0.smithi086.stdout:583/600: open fd = 3 2024-04-21T08:51:06.886 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.886 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.886 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.886 INFO:tasks.workunit.client.0.smithi086.stdout:584/600: open fd = 3 2024-04-21T08:51:06.886 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.886 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.886 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.886 INFO:tasks.workunit.client.0.smithi086.stdout:585/600: open fd = 3 2024-04-21T08:51:06.887 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.887 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.887 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.887 INFO:tasks.workunit.client.0.smithi086.stdout:586/600: open fd = 3 2024-04-21T08:51:06.887 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.887 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.887 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.887 INFO:tasks.workunit.client.0.smithi086.stdout:587/600: open fd = 3 2024-04-21T08:51:06.887 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.887 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.887 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.887 INFO:tasks.workunit.client.0.smithi086.stdout:588/600: open fd = 3 2024-04-21T08:51:06.887 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.888 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.888 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.888 INFO:tasks.workunit.client.0.smithi086.stdout:589/600: open fd = 3 2024-04-21T08:51:06.888 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.888 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.888 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.889 INFO:tasks.workunit.client.0.smithi086.stdout:590/600: open fd = 3 2024-04-21T08:51:06.889 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.889 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.889 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.889 INFO:tasks.workunit.client.0.smithi086.stdout:591/600: open fd = 3 2024-04-21T08:51:06.889 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.889 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.890 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.890 INFO:tasks.workunit.client.0.smithi086.stdout:592/600: open fd = 3 2024-04-21T08:51:06.890 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.890 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.890 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.890 INFO:tasks.workunit.client.0.smithi086.stdout:593/600: open fd = 3 2024-04-21T08:51:06.891 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.891 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.891 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.891 INFO:tasks.workunit.client.0.smithi086.stdout:594/600: open fd = 3 2024-04-21T08:51:06.891 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.891 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.892 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.892 INFO:tasks.workunit.client.0.smithi086.stdout:595/600: open fd = 3 2024-04-21T08:51:06.892 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.892 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.892 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.892 INFO:tasks.workunit.client.0.smithi086.stdout:596/600: open fd = 3 2024-04-21T08:51:06.893 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.893 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.893 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.893 INFO:tasks.workunit.client.0.smithi086.stdout:597/600: open fd = 3 2024-04-21T08:51:06.893 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.893 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.894 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.894 INFO:tasks.workunit.client.0.smithi086.stdout:598/600: open fd = 3 2024-04-21T08:51:06.894 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.894 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.894 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.894 INFO:tasks.workunit.client.0.smithi086.stdout:599/600: open fd = 3 2024-04-21T08:51:06.895 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.895 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.895 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.895 INFO:tasks.workunit.client.0.smithi086.stdout:600/600: open fd = 3 2024-04-21T08:51:06.895 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.895 INFO:tasks.workunit.client.0.smithi086.stdout:write ret = 32 2024-04-21T08:51:06.896 INFO:tasks.workunit.client.0.smithi086.stdout:pread ret = 64 2024-04-21T08:51:06.896 INFO:tasks.workunit.client.0.smithi086.stderr:+ echo OK 2024-04-21T08:51:06.898 INFO:tasks.workunit.client.0.smithi086.stdout:OK 2024-04-21T08:51:06.898 INFO:teuthology.orchestra.run:Running command with timeout 3600 2024-04-21T08:51:06.899 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2024-04-21T08:51:06.941 INFO:tasks.workunit:Stopping ['fs/test_o_trunc.sh'] on client.0... 2024-04-21T08:51:06.941 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2024-04-21T08:51:07.259 DEBUG:teuthology.parallel:result is None 2024-04-21T08:51:07.260 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2024-04-21T08:51:07.293 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2024-04-21T08:51:07.293 DEBUG:teuthology.run_tasks:Unwinding manager quiescer 2024-04-21T08:51:07.344 INFO:tasks.quiescer:joining Quiescers 2024-04-21T08:51:07.345 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:51:07.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.444+0000 7f9a45775640 1 -- 172.21.15.86:0/3314954427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a40106f10 msgr2=0x7f9a401072f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:07.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.444+0000 7f9a45775640 1 --2- 172.21.15.86:0/3314954427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a40106f10 0x7f9a401072f0 secure :-1 s=READY pgs=2080 cs=0 l=1 rev1=1 crypto rx=0x7f9a30009900 tx=0x7f9a3002bc30 comp rx=0 tx=0).stop 2024-04-21T08:51:07.450 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.444+0000 7f9a45775640 1 -- 172.21.15.86:0/3314954427 shutdown_connections 2024-04-21T08:51:07.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.444+0000 7f9a45775640 1 --2- 172.21.15.86:0/3314954427 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a40101ad0 0x7f9a4010dcb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.444+0000 7f9a45775640 1 --2- 172.21.15.86:0/3314954427 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a40101130 0x7f9a40101590 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.444+0000 7f9a45775640 1 --2- 172.21.15.86:0/3314954427 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a40106f10 0x7f9a401072f0 unknown :-1 s=CLOSED pgs=2080 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.444+0000 7f9a45775640 1 -- 172.21.15.86:0/3314954427 >> 172.21.15.86:0/3314954427 conn(0x7f9a40077e30 msgr2=0x7f9a40078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:07.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.444+0000 7f9a45775640 1 -- 172.21.15.86:0/3314954427 shutdown_connections 2024-04-21T08:51:07.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.444+0000 7f9a45775640 1 -- 172.21.15.86:0/3314954427 wait complete. 2024-04-21T08:51:07.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.445+0000 7f9a45775640 1 Processor -- start 2024-04-21T08:51:07.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.445+0000 7f9a45775640 1 -- start start 2024-04-21T08:51:07.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.445+0000 7f9a45775640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a40101130 0x7f9a4019e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:07.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.445+0000 7f9a45775640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a40101ad0 0x7f9a4019e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:07.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.445+0000 7f9a45775640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a40106f10 0x7f9a401985a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:07.451 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.445+0000 7f9a45775640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f9a40112260 con 0x7f9a40106f10 2024-04-21T08:51:07.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.445+0000 7f9a45775640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f9a401120e0 con 0x7f9a40101ad0 2024-04-21T08:51:07.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.445+0000 7f9a45775640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f9a401123e0 con 0x7f9a40101130 2024-04-21T08:51:07.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.445+0000 7f9a3f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a40106f10 0x7f9a401985a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:07.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.445+0000 7f9a3f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a40106f10 0x7f9a401985a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37448/0 (socket says 172.21.15.86:37448) 2024-04-21T08:51:07.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.445+0000 7f9a3f7fe640 1 -- 172.21.15.86:0/781632756 learned_addr learned my addr 172.21.15.86:0/781632756 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:07.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.445+0000 7f9a3effd640 1 --2- 172.21.15.86:0/781632756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a40101130 0x7f9a4019e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:07.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.445+0000 7f9a37fff640 1 --2- 172.21.15.86:0/781632756 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a40101ad0 0x7f9a4019e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:07.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.445+0000 7f9a3f7fe640 1 -- 172.21.15.86:0/781632756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a40101130 msgr2=0x7f9a4019e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:07.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.445+0000 7f9a3f7fe640 1 --2- 172.21.15.86:0/781632756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a40101130 0x7f9a4019e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.445+0000 7f9a3f7fe640 1 -- 172.21.15.86:0/781632756 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a40101ad0 msgr2=0x7f9a4019e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:07.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.445+0000 7f9a3f7fe640 1 --2- 172.21.15.86:0/781632756 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a40101ad0 0x7f9a4019e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.445+0000 7f9a3f7fe640 1 -- 172.21.15.86:0/781632756 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f9a40198d20 con 0x7f9a40106f10 2024-04-21T08:51:07.452 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.445+0000 7f9a37fff640 1 --2- 172.21.15.86:0/781632756 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a40101ad0 0x7f9a4019e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:07.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.445+0000 7f9a3f7fe640 1 --2- 172.21.15.86:0/781632756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a40106f10 0x7f9a401985a0 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f9a3800cce0 tx=0x7f9a3800a620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:07.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.446+0000 7f9a3cff9640 1 -- 172.21.15.86:0/781632756 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a3800f450 con 0x7f9a40106f10 2024-04-21T08:51:07.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.446+0000 7f9a3cff9640 1 -- 172.21.15.86:0/781632756 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f9a3800faa0 con 0x7f9a40106f10 2024-04-21T08:51:07.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.446+0000 7f9a3cff9640 1 -- 172.21.15.86:0/781632756 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f9a38018b70 con 0x7f9a40106f10 2024-04-21T08:51:07.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.446+0000 7f9a45775640 1 -- 172.21.15.86:0/781632756 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f9a40199010 con 0x7f9a40106f10 2024-04-21T08:51:07.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.446+0000 7f9a45775640 1 -- 172.21.15.86:0/781632756 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f9a401a52b0 con 0x7f9a40106f10 2024-04-21T08:51:07.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.447+0000 7f9a45775640 1 -- 172.21.15.86:0/781632756 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f9a40069230 con 0x7f9a40106f10 2024-04-21T08:51:07.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.448+0000 7f9a3cff9640 1 -- 172.21.15.86:0/781632756 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f9a38018d10 con 0x7f9a40106f10 2024-04-21T08:51:07.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.448+0000 7f9a3cff9640 1 --2- 172.21.15.86:0/781632756 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f9a2c0a3d20 0x7f9a2c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:07.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.448+0000 7f9a3effd640 1 --2- 172.21.15.86:0/781632756 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f9a2c0a3d20 0x7f9a2c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:07.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.448+0000 7f9a3cff9640 1 -- 172.21.15.86:0/781632756 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9584+0+0 (secure 0 0 0) 0x7f9a380d1080 con 0x7f9a40106f10 2024-04-21T08:51:07.453 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.449+0000 7f9a3effd640 1 --2- 172.21.15.86:0/781632756 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f9a2c0a3d20 0x7f9a2c0a61e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9a30009230 tx=0x7f9a30005fd0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:07.454 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.453+0000 7f9a3cff9640 1 -- 172.21.15.86:0/781632756 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f9a38014030 con 0x7f9a40106f10 2024-04-21T08:51:07.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.674+0000 7f9a45775640 1 -- 172.21.15.86:0/781632756 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f9a40109380 con 0x7f9a40106f10 2024-04-21T08:51:07.674 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.675+0000 7f9a3cff9640 1 -- 172.21.15.86:0/781632756 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7f9a3809d300 con 0x7f9a40106f10 2024-04-21T08:51:07.675 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:51:07.675 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:38:45.202126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24398,24394,24455,24472]},"id":1}]} 2024-04-21T08:51:07.675 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-21T08:51:07.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.677+0000 7f9a45775640 1 -- 172.21.15.86:0/781632756 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f9a2c0a3d20 msgr2=0x7f9a2c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:07.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.677+0000 7f9a45775640 1 --2- 172.21.15.86:0/781632756 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f9a2c0a3d20 0x7f9a2c0a61e0 secure :-1 s=READY pgs=348 cs=0 l=1 rev1=1 crypto rx=0x7f9a30009230 tx=0x7f9a30005fd0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.677+0000 7f9a45775640 1 -- 172.21.15.86:0/781632756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a40106f10 msgr2=0x7f9a401985a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:07.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.677+0000 7f9a45775640 1 --2- 172.21.15.86:0/781632756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a40106f10 0x7f9a401985a0 secure :-1 s=READY pgs=2081 cs=0 l=1 rev1=1 crypto rx=0x7f9a3800cce0 tx=0x7f9a3800a620 comp rx=0 tx=0).stop 2024-04-21T08:51:07.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.678+0000 7f9a45775640 1 -- 172.21.15.86:0/781632756 shutdown_connections 2024-04-21T08:51:07.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.678+0000 7f9a45775640 1 --2- 172.21.15.86:0/781632756 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f9a2c0a3d20 0x7f9a2c0a61e0 unknown :-1 s=CLOSED pgs=348 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.678+0000 7f9a45775640 1 --2- 172.21.15.86:0/781632756 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f9a40106f10 0x7f9a401985a0 unknown :-1 s=CLOSED pgs=2081 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.678+0000 7f9a45775640 1 --2- 172.21.15.86:0/781632756 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f9a40101ad0 0x7f9a4019e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.678+0000 7f9a45775640 1 --2- 172.21.15.86:0/781632756 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f9a40101130 0x7f9a4019e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.678+0000 7f9a45775640 1 -- 172.21.15.86:0/781632756 >> 172.21.15.86:0/781632756 conn(0x7f9a40077e30 msgr2=0x7f9a400ffda0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:07.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.678+0000 7f9a45775640 1 -- 172.21.15.86:0/781632756 shutdown_connections 2024-04-21T08:51:07.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.678+0000 7f9a45775640 1 -- 172.21.15.86:0/781632756 wait complete. 2024-04-21T08:51:07.689 DEBUG:tasks.quiescer.fs.[cephfs]:Running ceph command: 'tell mds.24482 quiesce db --cancel --all' 2024-04-21T08:51:07.689 DEBUG:teuthology.orchestra.run.smithi086:> timeout 45 sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.24482 quiesce db --cancel --all 2024-04-21T08:51:07.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.794+0000 7f59d6e5c640 1 -- 172.21.15.86:0/1852454942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d0107260 msgr2=0x7f59d010ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:07.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.794+0000 7f59d6e5c640 1 --2- 172.21.15.86:0/1852454942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d0107260 0x7f59d010ee30 secure :-1 s=READY pgs=2082 cs=0 l=1 rev1=1 crypto rx=0x7f59c8009900 tx=0x7f59c802bc30 comp rx=0 tx=0).stop 2024-04-21T08:51:07.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.794+0000 7f59d6e5c640 1 -- 172.21.15.86:0/1852454942 shutdown_connections 2024-04-21T08:51:07.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.794+0000 7f59d6e5c640 1 --2- 172.21.15.86:0/1852454942 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d0107260 0x7f59d010ee30 unknown :-1 s=CLOSED pgs=2082 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.794+0000 7f59d6e5c640 1 --2- 172.21.15.86:0/1852454942 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59d0102b40 0x7f59d0106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.794+0000 7f59d6e5c640 1 --2- 172.21.15.86:0/1852454942 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59d0102190 0x7f59d0102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.796 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.794+0000 7f59d6e5c640 1 -- 172.21.15.86:0/1852454942 >> 172.21.15.86:0/1852454942 conn(0x7f59d00793a0 msgr2=0x7f59d00797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.794+0000 7f59d6e5c640 1 -- 172.21.15.86:0/1852454942 shutdown_connections 2024-04-21T08:51:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.794+0000 7f59d6e5c640 1 -- 172.21.15.86:0/1852454942 wait complete. 2024-04-21T08:51:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.795+0000 7f59d6e5c640 1 Processor -- start 2024-04-21T08:51:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.795+0000 7f59d6e5c640 1 -- start start 2024-04-21T08:51:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.795+0000 7f59d6e5c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59d0102190 0x7f59d0074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.795+0000 7f59d6e5c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59d0107260 0x7f59d0074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.795+0000 7f59d6e5c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d0074cd0 0x7f59d01a6900 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.795+0000 7f59d6e5c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f59d01121d0 con 0x7f59d0074cd0 2024-04-21T08:51:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.795+0000 7f59d6e5c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f59d0112050 con 0x7f59d0102190 2024-04-21T08:51:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.795+0000 7f59d6e5c640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f59d0112350 con 0x7f59d0107260 2024-04-21T08:51:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.795+0000 7f59d53f9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d0074cd0 0x7f59d01a6900 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.795+0000 7f59d53f9640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d0074cd0 0x7f59d01a6900 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37466/0 (socket says 172.21.15.86:37466) 2024-04-21T08:51:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.795+0000 7f59d53f9640 1 -- 172.21.15.86:0/2126079785 learned_addr learned my addr 172.21.15.86:0/2126079785 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.795+0000 7f59d4bf8640 1 --2- 172.21.15.86:0/2126079785 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59d0102190 0x7f59d0074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:07.797 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.795+0000 7f59cffff640 1 --2- 172.21.15.86:0/2126079785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59d0107260 0x7f59d0074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.795+0000 7f59d53f9640 1 -- 172.21.15.86:0/2126079785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59d0107260 msgr2=0x7f59d0074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.795+0000 7f59d53f9640 1 --2- 172.21.15.86:0/2126079785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59d0107260 0x7f59d0074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.795+0000 7f59d53f9640 1 -- 172.21.15.86:0/2126079785 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59d0102190 msgr2=0x7f59d0074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.795+0000 7f59d53f9640 1 --2- 172.21.15.86:0/2126079785 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59d0102190 0x7f59d0074250 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.795+0000 7f59d53f9640 1 -- 172.21.15.86:0/2126079785 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59d01a6fa0 con 0x7f59d0074cd0 2024-04-21T08:51:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.795+0000 7f59d53f9640 1 --2- 172.21.15.86:0/2126079785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d0074cd0 0x7f59d01a6900 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f59c8005940 tx=0x7f59c802c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.795+0000 7f59cdffb640 1 -- 172.21.15.86:0/2126079785 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c8043020 con 0x7f59d0074cd0 2024-04-21T08:51:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.796+0000 7f59cdffb640 1 -- 172.21.15.86:0/2126079785 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59c802cc60 con 0x7f59d0074cd0 2024-04-21T08:51:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.796+0000 7f59d6e5c640 1 -- 172.21.15.86:0/2126079785 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59d01a7230 con 0x7f59d0074cd0 2024-04-21T08:51:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.796+0000 7f59cdffb640 1 -- 172.21.15.86:0/2126079785 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c8003bc0 con 0x7f59d0074cd0 2024-04-21T08:51:07.798 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.796+0000 7f59d6e5c640 1 -- 172.21.15.86:0/2126079785 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59d01a7780 con 0x7f59d0074cd0 2024-04-21T08:51:07.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.798+0000 7f59cdffb640 1 -- 172.21.15.86:0/2126079785 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f59c8046020 con 0x7f59d0074cd0 2024-04-21T08:51:07.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.798+0000 7f59d9a54740 1 Processor -- start 2024-04-21T08:51:07.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.798+0000 7f59d9a54740 1 -- start start 2024-04-21T08:51:07.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.798+0000 7f59d9a54740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b677371d10 0x55b6773720f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:07.805 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.798+0000 7f59d9a54740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b6773726c0 0x55b67737ac70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:07.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.798+0000 7f59d9a54740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b67737b240 0x55b67737d690 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:07.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.798+0000 7f59d9a54740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b6772d7000 con 0x55b677371d10 2024-04-21T08:51:07.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.799+0000 7f59d9a54740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b677273b20 con 0x55b6773726c0 2024-04-21T08:51:07.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.799+0000 7f59d9a54740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b6772d2fc0 con 0x55b67737b240 2024-04-21T08:51:07.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.799+0000 7f59cdffb640 1 --2- 172.21.15.86:0/2126079785 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f59bc0a3d20 0x7f59bc0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:07.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.799+0000 7f59cffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b677371d10 0x55b6773720f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:07.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.799+0000 7f59cffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b677371d10 0x55b6773720f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37480/0 (socket says 172.21.15.86:37480) 2024-04-21T08:51:07.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.799+0000 7f59cffff640 1 -- 172.21.15.86:0/3794765308 learned_addr learned my addr 172.21.15.86:0/3794765308 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:07.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.799+0000 7f59d53f9640 1 --2- 172.21.15.86:0/2126079785 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f59bc0a3d20 0x7f59bc0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:07.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.799+0000 7f59d4bf8640 1 --2- 172.21.15.86:0/3794765308 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b6773726c0 0x55b67737ac70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:07.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.799+0000 7f59cdffb640 1 -- 172.21.15.86:0/2126079785 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9584+0+0 (secure 0 0 0) 0x7f59c80f2ea0 con 0x7f59d0074cd0 2024-04-21T08:51:07.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.799+0000 7f59cffff640 1 -- 172.21.15.86:0/3794765308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b67737b240 msgr2=0x55b67737d690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:07.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.799+0000 7f59cffff640 1 --2- 172.21.15.86:0/3794765308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b67737b240 0x55b67737d690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.806 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.799+0000 7f59cffff640 1 -- 172.21.15.86:0/3794765308 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b6773726c0 msgr2=0x55b67737ac70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:07.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.799+0000 7f59cffff640 1 --2- 172.21.15.86:0/3794765308 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b6773726c0 0x55b67737ac70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.799+0000 7f59cffff640 1 -- 172.21.15.86:0/3794765308 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b67728dea0 con 0x55b677371d10 2024-04-21T08:51:07.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.799+0000 7f59cffff640 1 --2- 172.21.15.86:0/3794765308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b677371d10 0x55b6773720f0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f59c0007eb0 tx=0x7f59c0010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3f42e34603a38a12 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:07.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.799+0000 7f59d53f9640 1 --2- 172.21.15.86:0/2126079785 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f59bc0a3d20 0x7f59bc0a61e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f59c80baf90 tx=0x7f59c808d4e0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:07.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.799+0000 7f59c6ffd640 1 -- 172.21.15.86:0/3794765308 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c0016070 con 0x55b677371d10 2024-04-21T08:51:07.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.799+0000 7f59c6ffd640 1 -- 172.21.15.86:0/3794765308 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59c0011ca0 con 0x55b677371d10 2024-04-21T08:51:07.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.799+0000 7f59c6ffd640 1 -- 172.21.15.86:0/3794765308 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c0012cb0 con 0x55b677371d10 2024-04-21T08:51:07.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.800+0000 7f59d9a54740 1 -- 172.21.15.86:0/3794765308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b677371d10 msgr2=0x55b6773720f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:07.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.800+0000 7f59d9a54740 1 --2- 172.21.15.86:0/3794765308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b677371d10 0x55b6773720f0 secure :-1 s=READY pgs=2084 cs=0 l=1 rev1=1 crypto rx=0x7f59c0007eb0 tx=0x7f59c0010740 comp rx=0 tx=0).stop 2024-04-21T08:51:07.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.800+0000 7f59d9a54740 1 -- 172.21.15.86:0/3794765308 shutdown_connections 2024-04-21T08:51:07.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.800+0000 7f59d9a54740 1 --2- 172.21.15.86:0/3794765308 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b67737b240 0x55b67737d690 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.800+0000 7f59d9a54740 1 --2- 172.21.15.86:0/3794765308 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b6773726c0 0x55b67737ac70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.800+0000 7f59d9a54740 1 --2- 172.21.15.86:0/3794765308 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b677371d10 0x55b6773720f0 unknown :-1 s=CLOSED pgs=2084 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.807 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.800+0000 7f59d9a54740 1 -- 172.21.15.86:0/3794765308 >> 172.21.15.86:0/3794765308 conn(0x55b67736c3e0 msgr2=0x55b67736c7c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:07.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.800+0000 7f59d9a54740 1 -- 172.21.15.86:0/3794765308 shutdown_connections 2024-04-21T08:51:07.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.800+0000 7f59d9a54740 1 -- 172.21.15.86:0/3794765308 wait complete. 2024-04-21T08:51:07.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.800+0000 7f59d9a54740 1 -- start start 2024-04-21T08:51:07.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.800+0000 7f59d9a54740 1 Processor -- start 2024-04-21T08:51:07.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.801+0000 7f59d9a54740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b677371d10 0x55b6773782d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:07.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.801+0000 7f59d9a54740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6773726c0 0x55b677378810 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:07.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.801+0000 7f59d9a54740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b67737b240 0x55b677378d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:07.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.801+0000 7f59d9a54740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55b67737e0c0 con 0x55b67737b240 2024-04-21T08:51:07.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.801+0000 7f59d9a54740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55b677271510 con 0x55b677371d10 2024-04-21T08:51:07.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.801+0000 7f59d9a54740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55b67737e240 con 0x55b6773726c0 2024-04-21T08:51:07.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.801+0000 7f59d9a54740 10 client.0 _ll_register_callbacks cb 0x55b677270f00 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:07.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.801+0000 7f59cffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b67737b240 0x55b677378d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:07.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.801+0000 7f59cffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b67737b240 0x55b677378d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37488/0 (socket says 172.21.15.86:37488) 2024-04-21T08:51:07.808 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.801+0000 7f59cffff640 1 -- 172.21.15.86:0/297664083 learned_addr learned my addr 172.21.15.86:0/297664083 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.801+0000 7f59d4bf8640 1 --2- 172.21.15.86:0/297664083 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6773726c0 0x55b677378810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.801+0000 7f59cffff640 1 -- 172.21.15.86:0/297664083 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6773726c0 msgr2=0x55b677378810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.801+0000 7f59cffff640 1 --2- 172.21.15.86:0/297664083 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6773726c0 0x55b677378810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.801+0000 7f59cffff640 1 -- 172.21.15.86:0/297664083 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b677371d10 msgr2=0x55b6773782d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.801+0000 7f59cffff640 1 --2- 172.21.15.86:0/297664083 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b677371d10 0x55b6773782d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.801+0000 7f59cffff640 1 -- 172.21.15.86:0/297664083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6770b5a20 con 0x55b67737b240 2024-04-21T08:51:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.801+0000 7f59cffff640 1 --2- 172.21.15.86:0/297664083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b67737b240 0x55b677378d50 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f59c0007660 tx=0x7f59c0011bc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9d42a7bb111bdb8e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.801+0000 7f59affff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:51:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.802+0000 7f59affff640 1 -- 172.21.15.86:0/297664083 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c0016070 con 0x55b67737b240 2024-04-21T08:51:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.802+0000 7f59affff640 1 -- 172.21.15.86:0/297664083 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9584+0+0 (secure 0 0 0) 0x7f59c0003b70 con 0x55b67737b240 2024-04-21T08:51:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.802+0000 7f59d9a54740 1 -- 172.21.15.86:0/297664083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59c0011ca0 con 0x55b67737b240 2024-04-21T08:51:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.802+0000 7f59affff640 1 -- 172.21.15.86:0/297664083 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f59c0003d70 con 0x55b67737b240 2024-04-21T08:51:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.802+0000 7f59d9a54740 10 client.20490 fetch_fsmap learned FSMap version 29 2024-04-21T08:51:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.802+0000 7f59d9a54740 1 -- 172.21.15.86:0/297664083 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b6771161b0 con 0x55b67737b240 2024-04-21T08:51:07.809 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.802+0000 7f59affff640 1 -- 172.21.15.86:0/297664083 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59c0011ca0 con 0x55b67737b240 2024-04-21T08:51:07.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.802+0000 7f59affff640 1 -- 172.21.15.86:0/297664083 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f59c0004300 con 0x55b67737b240 2024-04-21T08:51:07.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.802+0000 7f59d9a54740 10 client.20490 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:51:07.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.802+0000 7f59d9a54740 10 client.20490 resolve_mds: validated gid 24482 aka daemon mds.b 2024-04-21T08:51:07.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.802+0000 7f59d9a54740 20 client.20490 populate_metadata read hostname 'smithi086' 2024-04-21T08:51:07.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.802+0000 7f59d9a54740 1 --2- 172.21.15.86:0/297664083 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b67738b6d0 0x55b67738dac0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:07.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.802+0000 7f59d9a54740 4 client.20490 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:51:07.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.802+0000 7f59d9a54740 1 -- 172.21.15.86:0/297664083 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55b6772e4a20 con 0x55b67738b6d0 2024-04-21T08:51:07.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.802+0000 7f59d4bf8640 1 --2- 172.21.15.86:0/297664083 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b67738b6d0 0x55b67738dac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:07.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.803+0000 7f59affff640 10 client.20490 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:51:07.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.803+0000 7f59d4bf8640 1 --2- 172.21.15.86:0/297664083 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b67738b6d0 0x55b67738dac0 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=8e273bcbaaf2c8ae server_cookie=47987de33a80dca7 in_seq=0 out_seq=0 2024-04-21T08:51:07.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.803+0000 7f59affff640 1 -- 172.21.15.86:0/297664083 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55b6772e4a20 con 0x55b67738b6d0 2024-04-21T08:51:07.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.803+0000 7f59affff640 10 client.20490 handle_command_reply: tid=0 2024-04-21T08:51:07.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.803+0000 7f59d9a54740 1 client.20490 shutdown 2024-04-21T08:51:07.810 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.804+0000 7f59d9a54740 1 -- 172.21.15.86:0/297664083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b67737b240 msgr2=0x55b677378d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:07.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.804+0000 7f59d9a54740 1 --2- 172.21.15.86:0/297664083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b67737b240 0x55b677378d50 secure :-1 s=READY pgs=2085 cs=0 l=1 rev1=1 crypto rx=0x7f59c0007660 tx=0x7f59c0011bc0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.804+0000 7f59d9a54740 1 -- 172.21.15.86:0/297664083 shutdown_connections 2024-04-21T08:51:07.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.804+0000 7f59d9a54740 1 --2- 172.21.15.86:0/297664083 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b67738b6d0 0x55b67738dac0 crc :-1 s=READY pgs=349 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.804+0000 7f59d9a54740 1 --2- 172.21.15.86:0/297664083 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b67737b240 0x55b677378d50 unknown :-1 s=CLOSED pgs=2085 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.804+0000 7f59affff640 0 client.20490 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:51:07.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.804+0000 7f59d9a54740 1 --2- 172.21.15.86:0/297664083 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6773726c0 0x55b677378810 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.804+0000 7f59d9a54740 1 --2- 172.21.15.86:0/297664083 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b677371d10 0x55b6773782d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.804+0000 7f59d9a54740 1 -- 172.21.15.86:0/297664083 >> 172.21.15.86:0/297664083 conn(0x55b67736c3e0 msgr2=0x55b67736b680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:07.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.804+0000 7f59d9a54740 1 -- 172.21.15.86:0/297664083 shutdown_connections 2024-04-21T08:51:07.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.804+0000 7f59d9a54740 1 -- 172.21.15.86:0/297664083 wait complete. 2024-04-21T08:51:07.811 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.804+0000 7f59d9a54740 20 client.20490 trim_cache size 0 max 16384 2024-04-21T08:51:07.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.821+0000 7f59d9a54740 1 Processor -- start 2024-04-21T08:51:07.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.821+0000 7f59d9a54740 1 -- start start 2024-04-21T08:51:07.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.821+0000 7f59d9a54740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b67736c3e0 0x55b677377180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:07.826 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.821+0000 7f59d9a54740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b677371d10 0x55b67736b460 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:07.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.821+0000 7f59d9a54740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6773726c0 0x55b677387090 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:07.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.821+0000 7f59d9a54740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f59d01b1760 con 0x55b677371d10 2024-04-21T08:51:07.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.821+0000 7f59d9a54740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f59d01b32a0 con 0x55b67736c3e0 2024-04-21T08:51:07.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.821+0000 7f59d9a54740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f59d01b1e30 con 0x55b6773726c0 2024-04-21T08:51:07.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.821+0000 7f59d4bf8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b677371d10 0x55b67736b460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:07.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.821+0000 7f59d4bf8640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b677371d10 0x55b67736b460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37504/0 (socket says 172.21.15.86:37504) 2024-04-21T08:51:07.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.821+0000 7f59d4bf8640 1 -- 172.21.15.86:0/538508493 learned_addr learned my addr 172.21.15.86:0/538508493 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:07.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.821+0000 7f59cffff640 1 --2- 172.21.15.86:0/538508493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b67736c3e0 0x55b677377180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:07.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.822+0000 7f59d4bf8640 1 -- 172.21.15.86:0/538508493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6773726c0 msgr2=0x55b677387090 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:07.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.822+0000 7f59d4bf8640 1 --2- 172.21.15.86:0/538508493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6773726c0 0x55b677387090 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.822+0000 7f59d4bf8640 1 -- 172.21.15.86:0/538508493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b67736c3e0 msgr2=0x55b677377180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:07.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.822+0000 7f59d4bf8640 1 --2- 172.21.15.86:0/538508493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b67736c3e0 0x55b677377180 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.822+0000 7f59d4bf8640 1 -- 172.21.15.86:0/538508493 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55b676fa9310 con 0x55b677371d10 2024-04-21T08:51:07.827 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.822+0000 7f59d4bf8640 1 --2- 172.21.15.86:0/538508493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b677371d10 0x55b67736b460 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x55b677372ac0 tx=0x7f59b8011810 comp rx=0 tx=0).ready entity=mon.0 client_cookie=16b357dad5cccf3e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:07.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.822+0000 7f59c67fc640 1 -- 172.21.15.86:0/538508493 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b8021030 con 0x55b677371d10 2024-04-21T08:51:07.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.822+0000 7f59c67fc640 1 -- 172.21.15.86:0/538508493 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59b801d8d0 con 0x55b677371d10 2024-04-21T08:51:07.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.822+0000 7f59c67fc640 1 -- 172.21.15.86:0/538508493 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59b8004a00 con 0x55b677371d10 2024-04-21T08:51:07.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.822+0000 7f59d9a54740 1 -- 172.21.15.86:0/538508493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b677371d10 msgr2=0x55b67736b460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:07.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.822+0000 7f59d9a54740 1 --2- 172.21.15.86:0/538508493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b677371d10 0x55b67736b460 secure :-1 s=READY pgs=2086 cs=0 l=1 rev1=1 crypto rx=0x55b677372ac0 tx=0x7f59b8011810 comp rx=0 tx=0).stop 2024-04-21T08:51:07.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.822+0000 7f59d9a54740 1 -- 172.21.15.86:0/538508493 shutdown_connections 2024-04-21T08:51:07.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.822+0000 7f59d9a54740 1 --2- 172.21.15.86:0/538508493 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b6773726c0 0x55b677387090 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.822+0000 7f59d9a54740 1 --2- 172.21.15.86:0/538508493 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b677371d10 0x55b67736b460 unknown :-1 s=CLOSED pgs=2086 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.822+0000 7f59d9a54740 1 --2- 172.21.15.86:0/538508493 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b67736c3e0 0x55b677377180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.822+0000 7f59d9a54740 1 -- 172.21.15.86:0/538508493 >> 172.21.15.86:0/538508493 conn(0x55b67736ba10 msgr2=0x55b67736e0c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:07.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.822+0000 7f59d9a54740 1 -- 172.21.15.86:0/538508493 shutdown_connections 2024-04-21T08:51:07.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.823+0000 7f59d9a54740 1 -- 172.21.15.86:0/538508493 wait complete. 2024-04-21T08:51:07.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.823+0000 7f59d9a54740 1 -- start start 2024-04-21T08:51:07.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.823+0000 7f59d9a54740 1 Processor -- start 2024-04-21T08:51:07.828 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.823+0000 7f59d9a54740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b67736c3e0 0x55b677380570 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:07.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.823+0000 7f59d9a54740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b677371d10 0x55b677380ab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:07.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.823+0000 7f59d9a54740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6773726c0 0x55b677380ff0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:07.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.823+0000 7f59d9a54740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f59d01b2bd0 con 0x55b6773726c0 2024-04-21T08:51:07.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.823+0000 7f59d9a54740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f59d01b2500 con 0x55b67736c3e0 2024-04-21T08:51:07.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.823+0000 7f59d9a54740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f59d01af740 con 0x55b677371d10 2024-04-21T08:51:07.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.823+0000 7f59d9a54740 10 client.0 _ll_register_callbacks cb 0x7f59c0011a80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:07.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.823+0000 7f59cffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6773726c0 0x55b677380ff0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:07.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.823+0000 7f59cffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6773726c0 0x55b677380ff0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37510/0 (socket says 172.21.15.86:37510) 2024-04-21T08:51:07.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.823+0000 7f59cffff640 1 -- 172.21.15.86:0/1293690268 learned_addr learned my addr 172.21.15.86:0/1293690268 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:07.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.823+0000 7f59cffff640 1 --2- 172.21.15.86:0/1293690268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b67736c3e0 0x55b677380570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:07.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.824+0000 7f59d4bf8640 1 --2- 172.21.15.86:0/1293690268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b677371d10 0x55b677380ab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:07.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.824+0000 7f59cffff640 1 -- 172.21.15.86:0/1293690268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b677371d10 msgr2=0x55b677380ab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:07.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.824+0000 7f59cffff640 1 --2- 172.21.15.86:0/1293690268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b677371d10 0x55b677380ab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.829 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.824+0000 7f59cffff640 1 -- 172.21.15.86:0/1293690268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b67736c3e0 msgr2=0x55b677380570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:07.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.824+0000 7f59cffff640 1 --2- 172.21.15.86:0/1293690268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b67736c3e0 0x55b677380570 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.824+0000 7f59cffff640 1 -- 172.21.15.86:0/1293690268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55b6773f1be0 con 0x55b6773726c0 2024-04-21T08:51:07.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.824+0000 7f59d4bf8640 1 --2- 172.21.15.86:0/1293690268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b677371d10 0x55b677380ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:51:07.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.824+0000 7f59cffff640 1 --2- 172.21.15.86:0/1293690268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6773726c0 0x55b677380ff0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f59c0004970 tx=0x7f59c0010740 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d3d4b3ce93820ef7 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:07.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.824+0000 7f59affff640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:51:07.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.824+0000 7f59affff640 1 -- 172.21.15.86:0/1293690268 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59c0032030 con 0x55b6773726c0 2024-04-21T08:51:07.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.824+0000 7f59affff640 1 -- 172.21.15.86:0/1293690268 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(107..107 src has 1..107) ==== 9584+0+0 (secure 0 0 0) 0x7f59c0002030 con 0x55b6773726c0 2024-04-21T08:51:07.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.824+0000 7f59d9a54740 1 -- 172.21.15.86:0/1293690268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f59b801d8d0 con 0x55b6773726c0 2024-04-21T08:51:07.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.824+0000 7f59affff640 1 -- 172.21.15.86:0/1293690268 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f59c003f050 con 0x55b6773726c0 2024-04-21T08:51:07.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.824+0000 7f59d9a54740 10 client.20496 fetch_fsmap learned FSMap version 29 2024-04-21T08:51:07.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.824+0000 7f59d9a54740 1 -- 172.21.15.86:0/1293690268 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55b6773f2e40 con 0x55b6773726c0 2024-04-21T08:51:07.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.824+0000 7f59affff640 1 -- 172.21.15.86:0/1293690268 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f59b801d8d0 con 0x55b6773726c0 2024-04-21T08:51:07.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.824+0000 7f59affff640 1 -- 172.21.15.86:0/1293690268 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f59c001b7d0 con 0x55b6773726c0 2024-04-21T08:51:07.830 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.824+0000 7f59d9a54740 10 client.20496 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:51:07.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.824+0000 7f59d9a54740 10 client.20496 resolve_mds: validated gid 24482 aka daemon mds.b 2024-04-21T08:51:07.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.824+0000 7f59d9a54740 20 client.20496 populate_metadata read hostname 'smithi086' 2024-04-21T08:51:07.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.824+0000 7f59d9a54740 1 --2- 172.21.15.86:0/1293690268 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b67737b240 0x55b6773f5530 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:07.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.824+0000 7f59d9a54740 4 client.20496 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "quiesce db", "cancel": true, "all": true}] 2024-04-21T08:51:07.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.824+0000 7f59d9a54740 1 -- 172.21.15.86:0/1293690268 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "quiesce db", "cancel": true, "all": true}) -- 0x55b6772de890 con 0x55b67737b240 2024-04-21T08:51:07.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.825+0000 7f59d4bf8640 1 --2- 172.21.15.86:0/1293690268 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b67737b240 0x55b6773f5530 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:07.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.825+0000 7f59d4bf8640 1 --2- 172.21.15.86:0/1293690268 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b67737b240 0x55b6773f5530 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=ec1c87fbd9d9072c server_cookie=39a4ae39eec2c931 in_seq=0 out_seq=0 2024-04-21T08:51:07.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.825+0000 7f59affff640 10 client.20496 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:51:07.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.825+0000 7f59affff640 1 -- 172.21.15.86:0/1293690268 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+80 (crc 0 0 0) 0x55b6772de890 con 0x55b67737b240 2024-04-21T08:51:07.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.825+0000 7f59affff640 10 client.20496 handle_command_reply: tid=0 2024-04-21T08:51:07.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.825+0000 7f59d9a54740 1 client.20496 shutdown 2024-04-21T08:51:07.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.826+0000 7f59d9a54740 1 -- 172.21.15.86:0/1293690268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6773726c0 msgr2=0x55b677380ff0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:07.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.826+0000 7f59d9a54740 1 --2- 172.21.15.86:0/1293690268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6773726c0 0x55b677380ff0 secure :-1 s=READY pgs=2087 cs=0 l=1 rev1=1 crypto rx=0x7f59c0004970 tx=0x7f59c0010740 comp rx=0 tx=0).stop 2024-04-21T08:51:07.831 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.826+0000 7f59d9a54740 1 -- 172.21.15.86:0/1293690268 shutdown_connections 2024-04-21T08:51:07.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.826+0000 7f59d9a54740 1 --2- 172.21.15.86:0/1293690268 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55b67737b240 0x55b6773f5530 crc :-1 s=READY pgs=350 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.826+0000 7f59d9a54740 1 --2- 172.21.15.86:0/1293690268 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55b6773726c0 0x55b677380ff0 unknown :-1 s=CLOSED pgs=2087 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.826+0000 7f59d9a54740 1 --2- 172.21.15.86:0/1293690268 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55b677371d10 0x55b677380ab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.826+0000 7f59affff640 0 client.20496 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:51:07.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.826+0000 7f59d9a54740 1 --2- 172.21.15.86:0/1293690268 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55b67736c3e0 0x55b677380570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.826+0000 7f59d9a54740 1 -- 172.21.15.86:0/1293690268 >> 172.21.15.86:0/1293690268 conn(0x55b67736ba10 msgr2=0x55b67736d7b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:07.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.826+0000 7f59d9a54740 1 -- 172.21.15.86:0/1293690268 shutdown_connections 2024-04-21T08:51:07.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.826+0000 7f59d9a54740 1 -- 172.21.15.86:0/1293690268 wait complete. 2024-04-21T08:51:07.832 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.826+0000 7f59d9a54740 20 client.20496 trim_cache size 0 max 16384 2024-04-21T08:51:07.832 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:51:07.833 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 29, 2024-04-21T08:51:07.833 INFO:teuthology.orchestra.run.smithi086.stdout: "leader": 24482, 2024-04-21T08:51:07.833 INFO:teuthology.orchestra.run.smithi086.stdout: "set_version": 12, 2024-04-21T08:51:07.833 INFO:teuthology.orchestra.run.smithi086.stdout: "sets": {} 2024-04-21T08:51:07.833 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:51:07.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.826+0000 7f59d6e5c640 1 -- 172.21.15.86:0/2126079785 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f59bc0a3d20 msgr2=0x7f59bc0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:07.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.826+0000 7f59d6e5c640 1 --2- 172.21.15.86:0/2126079785 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f59bc0a3d20 0x7f59bc0a61e0 secure :-1 s=READY pgs=349 cs=0 l=1 rev1=1 crypto rx=0x7f59c80baf90 tx=0x7f59c808d4e0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.826+0000 7f59d6e5c640 1 -- 172.21.15.86:0/2126079785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d0074cd0 msgr2=0x7f59d01a6900 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:07.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.826+0000 7f59d6e5c640 1 --2- 172.21.15.86:0/2126079785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d0074cd0 0x7f59d01a6900 secure :-1 s=READY pgs=2083 cs=0 l=1 rev1=1 crypto rx=0x7f59c8005940 tx=0x7f59c802c860 comp rx=0 tx=0).stop 2024-04-21T08:51:07.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.826+0000 7f59d6e5c640 1 -- 172.21.15.86:0/2126079785 shutdown_connections 2024-04-21T08:51:07.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.826+0000 7f59d6e5c640 1 --2- 172.21.15.86:0/2126079785 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f59bc0a3d20 0x7f59bc0a61e0 unknown :-1 s=CLOSED pgs=349 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.826+0000 7f59d6e5c640 1 --2- 172.21.15.86:0/2126079785 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f59d0074cd0 0x7f59d01a6900 unknown :-1 s=CLOSED pgs=2083 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.826+0000 7f59d6e5c640 1 --2- 172.21.15.86:0/2126079785 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f59d0107260 0x7f59d0074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.826+0000 7f59d6e5c640 1 --2- 172.21.15.86:0/2126079785 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f59d0102190 0x7f59d0074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:07.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.826+0000 7f59d6e5c640 1 -- 172.21.15.86:0/2126079785 >> 172.21.15.86:0/2126079785 conn(0x7f59d00793a0 msgr2=0x7f59d00ffe80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:07.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.827+0000 7f59d6e5c640 1 -- 172.21.15.86:0/2126079785 shutdown_connections 2024-04-21T08:51:07.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:07.827+0000 7f59d6e5c640 1 -- 172.21.15.86:0/2126079785 wait complete. 2024-04-21T08:51:07.837 INFO:tasks.quiescer:done joining Quiescers 2024-04-21T08:51:07.837 DEBUG:teuthology.run_tasks:Unwinding manager background_exec 2024-04-21T08:51:07.848 INFO:teuthology.task.background_exec:Stopping background command on ubuntu@smithi086.front.sepia.ceph.com 2024-04-21T08:51:07.856 DEBUG:teuthology.run_tasks:Unwinding manager full_sequential_finally 2024-04-21T08:51:07.866 INFO:teuthology.task.full_sequential_finally:In full_sequential_finally, running task exec... 2024-04-21T08:51:07.866 INFO:teuthology.task.exec:Executing custom commands... 2024-04-21T08:51:07.866 INFO:teuthology.task.exec:Running commands on role mon.a host ubuntu@smithi086.front.sepia.ceph.com 2024-04-21T08:51:07.866 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'date +%s > END_TIME' 2024-04-21T08:51:07.921 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'START_TIME=$(cat START_TIME); END_TIME=$(cat END_TIME); DIFF_TIME=$((600-(END_TIME-START_TIME))); if [ "$DIFF_TIME" -gt 0 ]; then sleep "$DIFF_TIME"; fi' 2024-04-21T08:51:07.987 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule status --fs=cephfs --path=/' 2024-04-21T08:51:08.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.127+0000 7f595066d640 1 -- 172.21.15.86:0/3745361311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5948101af0 msgr2=0x7f594810dc60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:08.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.127+0000 7f595066d640 1 --2- 172.21.15.86:0/3745361311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5948101af0 0x7f594810dc60 secure :-1 s=READY pgs=2088 cs=0 l=1 rev1=1 crypto rx=0x7f5940009cd0 tx=0x7f594002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:51:08.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.127+0000 7f595066d640 1 -- 172.21.15.86:0/3745361311 shutdown_connections 2024-04-21T08:51:08.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.127+0000 7f595066d640 1 --2- 172.21.15.86:0/3745361311 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5948101af0 0x7f594810dc60 unknown :-1 s=CLOSED pgs=2088 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:08.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.127+0000 7f595066d640 1 --2- 172.21.15.86:0/3745361311 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5948101130 0x7f59481015b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:08.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.127+0000 7f595066d640 1 --2- 172.21.15.86:0/3745361311 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5948106f70 0x7f5948107350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:08.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.127+0000 7f595066d640 1 -- 172.21.15.86:0/3745361311 >> 172.21.15.86:0/3745361311 conn(0x7f5948077ba0 msgr2=0x7f5948077fb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:08.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.127+0000 7f595066d640 1 -- 172.21.15.86:0/3745361311 shutdown_connections 2024-04-21T08:51:08.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.127+0000 7f595066d640 1 -- 172.21.15.86:0/3745361311 wait complete. 2024-04-21T08:51:08.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.127+0000 7f595066d640 1 Processor -- start 2024-04-21T08:51:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.127+0000 7f595066d640 1 -- start start 2024-04-21T08:51:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.128+0000 7f595066d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5948101130 0x7f5948199400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.128+0000 7f595066d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5948101af0 0x7f5948199940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.128+0000 7f595066d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5948106f70 0x7f59481a4480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.128+0000 7f595066d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5948111080 con 0x7f5948106f70 2024-04-21T08:51:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.128+0000 7f595066d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5948110f00 con 0x7f5948101af0 2024-04-21T08:51:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.128+0000 7f595066d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5948111200 con 0x7f5948101130 2024-04-21T08:51:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.128+0000 7f594ec0a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5948106f70 0x7f59481a4480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.128+0000 7f594ec0a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5948106f70 0x7f59481a4480 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37526/0 (socket says 172.21.15.86:37526) 2024-04-21T08:51:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.128+0000 7f594ec0a640 1 -- 172.21.15.86:0/3867375850 learned_addr learned my addr 172.21.15.86:0/3867375850 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.128+0000 7f594e409640 1 --2- 172.21.15.86:0/3867375850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5948101130 0x7f5948199400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.128+0000 7f594ec0a640 1 -- 172.21.15.86:0/3867375850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5948101130 msgr2=0x7f5948199400 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.128+0000 7f594ec0a640 1 --2- 172.21.15.86:0/3867375850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5948101130 0x7f5948199400 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.128+0000 7f594dc08640 1 --2- 172.21.15.86:0/3867375850 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5948101af0 0x7f5948199940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:08.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.128+0000 7f594ec0a640 1 -- 172.21.15.86:0/3867375850 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5948101af0 msgr2=0x7f5948199940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:08.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.128+0000 7f594ec0a640 1 --2- 172.21.15.86:0/3867375850 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5948101af0 0x7f5948199940 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:08.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.128+0000 7f594ec0a640 1 -- 172.21.15.86:0/3867375850 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f59481a49c0 con 0x7f5948106f70 2024-04-21T08:51:08.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.128+0000 7f594ec0a640 1 --2- 172.21.15.86:0/3867375850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5948106f70 0x7f59481a4480 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f5940037040 tx=0x7f5940035ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:08.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.128+0000 7f593f7fe640 1 -- 172.21.15.86:0/3867375850 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5940039040 con 0x7f5948106f70 2024-04-21T08:51:08.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.128+0000 7f593f7fe640 1 -- 172.21.15.86:0/3867375850 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5940044070 con 0x7f5948106f70 2024-04-21T08:51:08.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.128+0000 7f593f7fe640 1 -- 172.21.15.86:0/3867375850 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f59400042a0 con 0x7f5948106f70 2024-04-21T08:51:08.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.128+0000 7f595066d640 1 -- 172.21.15.86:0/3867375850 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f59481a4b90 con 0x7f5948106f70 2024-04-21T08:51:08.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.128+0000 7f595066d640 1 -- 172.21.15.86:0/3867375850 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f59481a4ff0 con 0x7f5948106f70 2024-04-21T08:51:08.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.129+0000 7f595066d640 1 -- 172.21.15.86:0/3867375850 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f59481025a0 con 0x7f5948106f70 2024-04-21T08:51:08.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.130+0000 7f593f7fe640 1 -- 172.21.15.86:0/3867375850 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5940004440 con 0x7f5948106f70 2024-04-21T08:51:08.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.131+0000 7f593f7fe640 1 --2- 172.21.15.86:0/3867375850 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f59340a3e40 0x7f59340a6300 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:08.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.131+0000 7f593f7fe640 1 -- 172.21.15.86:0/3867375850 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9584+0+0 (secure 0 0 0) 0x7f59400f3700 con 0x7f5948106f70 2024-04-21T08:51:08.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.131+0000 7f594e409640 1 --2- 172.21.15.86:0/3867375850 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f59340a3e40 0x7f59340a6300 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:08.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.131+0000 7f594e409640 1 --2- 172.21.15.86:0/3867375850 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f59340a3e40 0x7f59340a6300 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f594400cac0 tx=0x7f594400c4a0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:08.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.135+0000 7f593f7fe640 1 -- 172.21.15.86:0/3867375850 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f59400bfa90 con 0x7f5948106f70 2024-04-21T08:51:08.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:07 smithi179 ceph-mon[31829]: pgmap v386: 129 pgs: 129 active+clean; 3.5 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 105 KiB/s wr, 38 op/s 2024-04-21T08:51:08.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:07 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/781632756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:51:08.335 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.336+0000 7f595066d640 1 -- 172.21.15.86:0/3867375850 --> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] -- mgr_command(tid 0: {"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}) -- 0x7f5948100370 con 0x7f59340a3e40 2024-04-21T08:51:08.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.344+0000 7f593f7fe640 1 -- 172.21.15.86:0/3867375850 <== mgr.15558 v2:172.21.15.86:6800/2452032403 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+350 (secure 0 0 0) 0x7f5948100370 con 0x7f59340a3e40 2024-04-21T08:51:08.343 INFO:teuthology.orchestra.run.smithi086.stdout:{"fs": "cephfs", "subvol": null, "group": null, "path": "/", "rel_path": "/", "schedule": "1m", "retention": {"m": 6, "h": 3}, "start": "2024-04-21T00:00:00", "created": "2024-04-21T08:38:52", "first": "2024-04-21T08:39:00", "last": "2024-04-21T08:51:00", "last_pruned": "2024-04-21T08:51:00", "created_count": 13, "pruned_count": 7, "active": true} 2024-04-21T08:51:08.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.346+0000 7f595066d640 1 -- 172.21.15.86:0/3867375850 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f59340a3e40 msgr2=0x7f59340a6300 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:08.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.346+0000 7f595066d640 1 --2- 172.21.15.86:0/3867375850 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f59340a3e40 0x7f59340a6300 secure :-1 s=READY pgs=350 cs=0 l=1 rev1=1 crypto rx=0x7f594400cac0 tx=0x7f594400c4a0 comp rx=0 tx=0).stop 2024-04-21T08:51:08.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.346+0000 7f595066d640 1 -- 172.21.15.86:0/3867375850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5948106f70 msgr2=0x7f59481a4480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:08.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.346+0000 7f595066d640 1 --2- 172.21.15.86:0/3867375850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5948106f70 0x7f59481a4480 secure :-1 s=READY pgs=2089 cs=0 l=1 rev1=1 crypto rx=0x7f5940037040 tx=0x7f5940035ce0 comp rx=0 tx=0).stop 2024-04-21T08:51:08.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.346+0000 7f595066d640 1 -- 172.21.15.86:0/3867375850 shutdown_connections 2024-04-21T08:51:08.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.346+0000 7f595066d640 1 --2- 172.21.15.86:0/3867375850 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f59340a3e40 0x7f59340a6300 unknown :-1 s=CLOSED pgs=350 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:08.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.346+0000 7f595066d640 1 --2- 172.21.15.86:0/3867375850 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5948106f70 0x7f59481a4480 unknown :-1 s=CLOSED pgs=2089 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:08.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.346+0000 7f595066d640 1 --2- 172.21.15.86:0/3867375850 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5948101af0 0x7f5948199940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:08.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.346+0000 7f595066d640 1 --2- 172.21.15.86:0/3867375850 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5948101130 0x7f5948199400 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:08.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.346+0000 7f595066d640 1 -- 172.21.15.86:0/3867375850 >> 172.21.15.86:0/3867375850 conn(0x7f5948077ba0 msgr2=0x7f59480ffc60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:08.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.346+0000 7f595066d640 1 -- 172.21.15.86:0/3867375850 shutdown_connections 2024-04-21T08:51:08.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.346+0000 7f595066d640 1 -- 172.21.15.86:0/3867375850 wait complete. 2024-04-21T08:51:08.356 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:07 smithi086 ceph-mon[27939]: pgmap v386: 129 pgs: 129 active+clean; 3.5 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 105 KiB/s wr, 38 op/s 2024-04-21T08:51:08.356 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:07 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/781632756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:51:08.357 DEBUG:teuthology.orchestra.run.smithi086:> sudo TESTDIR=/home/ubuntu/cephtest bash -c 'ceph fs snap-schedule list --fs=cephfs --path=/ --recursive=true' 2024-04-21T08:51:08.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.457+0000 7f2bca1f4640 1 -- 172.21.15.86:0/95495394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bc4107280 msgr2=0x7f2bc410ee50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:08.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.457+0000 7f2bca1f4640 1 --2- 172.21.15.86:0/95495394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bc4107280 0x7f2bc410ee50 secure :-1 s=READY pgs=2090 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc009ca0 tx=0x7f2bbc02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:51:08.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.457+0000 7f2bca1f4640 1 -- 172.21.15.86:0/95495394 shutdown_connections 2024-04-21T08:51:08.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.457+0000 7f2bca1f4640 1 --2- 172.21.15.86:0/95495394 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bc4107280 0x7f2bc410ee50 unknown :-1 s=CLOSED pgs=2090 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:08.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.457+0000 7f2bca1f4640 1 --2- 172.21.15.86:0/95495394 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc4101070 0x7f2bc4106ba0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:08.457 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.457+0000 7f2bca1f4640 1 --2- 172.21.15.86:0/95495394 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bc41006c0 0x7f2bc4100aa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:08.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.457+0000 7f2bca1f4640 1 -- 172.21.15.86:0/95495394 >> 172.21.15.86:0/95495394 conn(0x7f2bc4077390 msgr2=0x7f2bc40fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:08.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.457+0000 7f2bca1f4640 1 -- 172.21.15.86:0/95495394 shutdown_connections 2024-04-21T08:51:08.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.457+0000 7f2bca1f4640 1 -- 172.21.15.86:0/95495394 wait complete. 2024-04-21T08:51:08.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.458+0000 7f2bca1f4640 1 Processor -- start 2024-04-21T08:51:08.458 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.458+0000 7f2bca1f4640 1 -- start start 2024-04-21T08:51:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.458+0000 7f2bca1f4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bc41006c0 0x7f2bc41985f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.458+0000 7f2bca1f4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc4107280 0x7f2bc4198b30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.458+0000 7f2bca1f4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bc419ceb0 0x7f2bc419d360 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.458+0000 7f2bca1f4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc41121d0 con 0x7f2bc419ceb0 2024-04-21T08:51:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.458+0000 7f2bca1f4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc4112050 con 0x7f2bc4107280 2024-04-21T08:51:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.458+0000 7f2bca1f4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2bc4112350 con 0x7f2bc41006c0 2024-04-21T08:51:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.458+0000 7f2bc3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bc419ceb0 0x7f2bc419d360 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:08.460 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.458+0000 7f2bc3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bc419ceb0 0x7f2bc419d360 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37548/0 (socket says 172.21.15.86:37548) 2024-04-21T08:51:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.458+0000 7f2bc3fff640 1 -- 172.21.15.86:0/3044873498 learned_addr learned my addr 172.21.15.86:0/3044873498 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.458+0000 7f2bbaffd640 1 --2- 172.21.15.86:0/3044873498 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc4107280 0x7f2bc4198b30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.458+0000 7f2bc37fe640 1 --2- 172.21.15.86:0/3044873498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bc41006c0 0x7f2bc41985f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.458+0000 7f2bc3fff640 1 -- 172.21.15.86:0/3044873498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bc41006c0 msgr2=0x7f2bc41985f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.458+0000 7f2bc3fff640 1 --2- 172.21.15.86:0/3044873498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bc41006c0 0x7f2bc41985f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.458+0000 7f2bc3fff640 1 -- 172.21.15.86:0/3044873498 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc4107280 msgr2=0x7f2bc4198b30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.458+0000 7f2bc3fff640 1 --2- 172.21.15.86:0/3044873498 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc4107280 0x7f2bc4198b30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.459+0000 7f2bc3fff640 1 -- 172.21.15.86:0/3044873498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f2bc419d9a0 con 0x7f2bc419ceb0 2024-04-21T08:51:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.459+0000 7f2bc3fff640 1 --2- 172.21.15.86:0/3044873498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bc419ceb0 0x7f2bc419d360 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc005e30 tx=0x7f2bbc034670 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.459+0000 7f2bc17fa640 1 -- 172.21.15.86:0/3044873498 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc041020 con 0x7f2bc419ceb0 2024-04-21T08:51:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.459+0000 7f2bc17fa640 1 -- 172.21.15.86:0/3044873498 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f2bbc032970 con 0x7f2bc419ceb0 2024-04-21T08:51:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.459+0000 7f2bc17fa640 1 -- 172.21.15.86:0/3044873498 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2bbc039040 con 0x7f2bc419ceb0 2024-04-21T08:51:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.459+0000 7f2bca1f4640 1 -- 172.21.15.86:0/3044873498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f2bc419dc30 con 0x7f2bc419ceb0 2024-04-21T08:51:08.461 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.459+0000 7f2bca1f4640 1 -- 172.21.15.86:0/3044873498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f2bc41a53f0 con 0x7f2bc419ceb0 2024-04-21T08:51:08.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.460+0000 7f2bca1f4640 1 -- 172.21.15.86:0/3044873498 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f2bc404fce0 con 0x7f2bc419ceb0 2024-04-21T08:51:08.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.461+0000 7f2bc17fa640 1 -- 172.21.15.86:0/3044873498 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2bbc044020 con 0x7f2bc419ceb0 2024-04-21T08:51:08.462 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.461+0000 7f2bc17fa640 1 --2- 172.21.15.86:0/3044873498 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2bb00a3dc0 0x7f2bb00a6280 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:08.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.461+0000 7f2bc37fe640 1 --2- 172.21.15.86:0/3044873498 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2bb00a3dc0 0x7f2bb00a6280 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:08.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.462+0000 7f2bc17fa640 1 -- 172.21.15.86:0/3044873498 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(107..107 src has 1..107) ==== 9584+0+0 (secure 0 0 0) 0x7f2bbc0f28e0 con 0x7f2bc419ceb0 2024-04-21T08:51:08.466 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.462+0000 7f2bc37fe640 1 --2- 172.21.15.86:0/3044873498 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2bb00a3dc0 0x7f2bb00a6280 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4005e10 tx=0x7f2bb4005da0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:08.468 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:07 smithi088 ceph-mon[31259]: pgmap v386: 129 pgs: 129 active+clean; 3.5 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 60 KiB/s rd, 105 KiB/s wr, 38 op/s 2024-04-21T08:51:08.468 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:07 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/781632756' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:51:08.468 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.466+0000 7f2bc17fa640 1 -- 172.21.15.86:0/3044873498 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f2bbc04b020 con 0x7f2bc419ceb0 2024-04-21T08:51:08.666 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.667+0000 7f2bca1f4640 1 -- 172.21.15.86:0/3044873498 --> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] -- mgr_command(tid 0: {"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}) -- 0x7f2bc4101070 con 0x7f2bb00a3dc0 2024-04-21T08:51:08.673 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.674+0000 7f2bc17fa640 1 -- 172.21.15.86:0/3044873498 <== mgr.15558 v2:172.21.15.86:6800/2452032403 1 ==== mgr_command_reply(tid 0: 0 ) ==== 8+0+10 (secure 0 0 0) 0x7f2bc4101070 con 0x7f2bb00a3dc0 2024-04-21T08:51:08.674 INFO:teuthology.orchestra.run.smithi086.stdout:/ 1m 6m3h 2024-04-21T08:51:08.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.676+0000 7f2bca1f4640 1 -- 172.21.15.86:0/3044873498 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2bb00a3dc0 msgr2=0x7f2bb00a6280 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:08.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.676+0000 7f2bca1f4640 1 --2- 172.21.15.86:0/3044873498 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2bb00a3dc0 0x7f2bb00a6280 secure :-1 s=READY pgs=351 cs=0 l=1 rev1=1 crypto rx=0x7f2bb4005e10 tx=0x7f2bb4005da0 comp rx=0 tx=0).stop 2024-04-21T08:51:08.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.676+0000 7f2bca1f4640 1 -- 172.21.15.86:0/3044873498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bc419ceb0 msgr2=0x7f2bc419d360 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:08.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.676+0000 7f2bca1f4640 1 --2- 172.21.15.86:0/3044873498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bc419ceb0 0x7f2bc419d360 secure :-1 s=READY pgs=2091 cs=0 l=1 rev1=1 crypto rx=0x7f2bbc005e30 tx=0x7f2bbc034670 comp rx=0 tx=0).stop 2024-04-21T08:51:08.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.676+0000 7f2bca1f4640 1 -- 172.21.15.86:0/3044873498 shutdown_connections 2024-04-21T08:51:08.675 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.676+0000 7f2bca1f4640 1 --2- 172.21.15.86:0/3044873498 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f2bb00a3dc0 0x7f2bb00a6280 unknown :-1 s=CLOSED pgs=351 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:08.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.676+0000 7f2bca1f4640 1 --2- 172.21.15.86:0/3044873498 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2bc419ceb0 0x7f2bc419d360 unknown :-1 s=CLOSED pgs=2091 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:08.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.676+0000 7f2bca1f4640 1 --2- 172.21.15.86:0/3044873498 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2bc4107280 0x7f2bc4198b30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:08.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.676+0000 7f2bca1f4640 1 --2- 172.21.15.86:0/3044873498 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2bc41006c0 0x7f2bc41985f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:08.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.676+0000 7f2bca1f4640 1 -- 172.21.15.86:0/3044873498 >> 172.21.15.86:0/3044873498 conn(0x7f2bc4077390 msgr2=0x7f2bc4105250 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:08.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.676+0000 7f2bca1f4640 1 -- 172.21.15.86:0/3044873498 shutdown_connections 2024-04-21T08:51:08.676 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:08.676+0000 7f2bca1f4640 1 -- 172.21.15.86:0/3044873498 wait complete. 2024-04-21T08:51:08.686 DEBUG:teuthology.run_tasks:Unwinding manager fwd_scrub 2024-04-21T08:51:08.702 INFO:tasks.fwd_scrub:joining ForwardScrubbers 2024-04-21T08:51:09.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:09 smithi086 ceph-mon[27939]: from='client.20502 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:51:09.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:09 smithi088 ceph-mon[31259]: from='client.20502 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:51:09.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:09 smithi179 ceph-mon[31829]: from='client.20502 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule status", "fs": "cephfs", "path": "/", "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:51:10.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:10 smithi086 ceph-mon[27939]: from='client.20508 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:51:10.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:10 smithi086 ceph-mon[27939]: pgmap v387: 129 pgs: 129 active+clean; 3.7 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 100 KiB/s wr, 35 op/s 2024-04-21T08:51:10.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:10 smithi088 ceph-mon[31259]: from='client.20508 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:51:10.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:10 smithi088 ceph-mon[31259]: pgmap v387: 129 pgs: 129 active+clean; 3.7 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 100 KiB/s wr, 35 op/s 2024-04-21T08:51:10.496 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:10 smithi179 ceph-mon[31829]: from='client.20508 -' entity='client.admin' cmd=[{"prefix": "fs snap-schedule list", "fs": "cephfs", "path": "/", "recursive": true, "target": ["mon-mgr", ""]}]: dispatch 2024-04-21T08:51:10.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:10 smithi179 ceph-mon[31829]: pgmap v387: 129 pgs: 129 active+clean; 3.7 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 61 KiB/s rd, 100 KiB/s wr, 35 op/s 2024-04-21T08:51:11.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:11 smithi088 ceph-mon[31259]: osdmap e108: 12 total, 12 up, 12 in 2024-04-21T08:51:11.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:11 smithi088 ceph-mon[31259]: pgmap v389: 129 pgs: 129 active+clean; 3.7 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 9 op/s 2024-04-21T08:51:11.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:11 smithi179 ceph-mon[31829]: osdmap e108: 12 total, 12 up, 12 in 2024-04-21T08:51:11.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:11 smithi179 ceph-mon[31829]: pgmap v389: 129 pgs: 129 active+clean; 3.7 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 9 op/s 2024-04-21T08:51:12.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:11 smithi086 ceph-mon[27939]: osdmap e108: 12 total, 12 up, 12 in 2024-04-21T08:51:12.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:11 smithi086 ceph-mon[27939]: pgmap v389: 129 pgs: 129 active+clean; 3.7 MiB data, 432 MiB used, 1.0 TiB / 1.0 TiB avail; 31 KiB/s rd, 29 KiB/s wr, 9 op/s 2024-04-21T08:51:14.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:13 smithi088 ceph-mon[31259]: pgmap v390: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 31 KiB/s wr, 20 op/s 2024-04-21T08:51:14.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:13 smithi179 ceph-mon[31829]: pgmap v390: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 31 KiB/s wr, 20 op/s 2024-04-21T08:51:14.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:13 smithi086 ceph-mon[27939]: pgmap v390: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 44 KiB/s rd, 31 KiB/s wr, 20 op/s 2024-04-21T08:51:16.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:15 smithi088 ceph-mon[31259]: pgmap v391: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 25 KiB/s wr, 16 op/s 2024-04-21T08:51:16.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:15 smithi179 ceph-mon[31829]: pgmap v391: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 25 KiB/s wr, 16 op/s 2024-04-21T08:51:16.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:15 smithi086 ceph-mon[27939]: pgmap v391: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 36 KiB/s rd, 25 KiB/s wr, 16 op/s 2024-04-21T08:51:18.087 INFO:teuthology.orchestra.run:Running command with timeout 900 2024-04-21T08:51:18.087 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 900 ceph --cluster ceph tell mds.1:0 scrub status 2024-04-21T08:51:18.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.200+0000 7facecad5640 1 -- 172.21.15.86:0/3077929024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7face81011a0 msgr2=0x7face8101600 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.200+0000 7facecad5640 1 --2- 172.21.15.86:0/3077929024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7face81011a0 0x7face8101600 secure :-1 s=READY pgs=2092 cs=0 l=1 rev1=1 crypto rx=0x7facd0009900 tx=0x7facd002bc30 comp rx=0 tx=0).stop 2024-04-21T08:51:18.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.200+0000 7facecad5640 1 -- 172.21.15.86:0/3077929024 shutdown_connections 2024-04-21T08:51:18.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.200+0000 7facecad5640 1 --2- 172.21.15.86:0/3077929024 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7face8101b40 0x7face810dc40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.200+0000 7facecad5640 1 --2- 172.21.15.86:0/3077929024 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7face81011a0 0x7face8101600 unknown :-1 s=CLOSED pgs=2092 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.200+0000 7facecad5640 1 --2- 172.21.15.86:0/3077929024 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7face8106f90 0x7face8107370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.200+0000 7facecad5640 1 -- 172.21.15.86:0/3077929024 >> 172.21.15.86:0/3077929024 conn(0x7face8076c50 msgr2=0x7face8077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:18.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.200+0000 7facecad5640 1 -- 172.21.15.86:0/3077929024 shutdown_connections 2024-04-21T08:51:18.200 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.200+0000 7facecad5640 1 -- 172.21.15.86:0/3077929024 wait complete. 2024-04-21T08:51:18.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.201+0000 7facecad5640 1 Processor -- start 2024-04-21T08:51:18.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.201+0000 7facecad5640 1 -- start start 2024-04-21T08:51:18.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.201+0000 7facecad5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7face81011a0 0x7face810e760 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.201+0000 7facecad5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7face8101b40 0x7face810eca0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.201+0000 7facecad5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7face8106f90 0x7face810f1e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.201+0000 7facecad5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7face8112240 con 0x7face8101b40 2024-04-21T08:51:18.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.201+0000 7facecad5640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7face81120c0 con 0x7face8106f90 2024-04-21T08:51:18.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.201+0000 7facecad5640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7face81123c0 con 0x7face81011a0 2024-04-21T08:51:18.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.201+0000 7facdffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7face8101b40 0x7face810eca0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.201+0000 7facdffff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7face8101b40 0x7face810eca0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56422/0 (socket says 172.21.15.86:56422) 2024-04-21T08:51:18.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.201+0000 7facdffff640 1 -- 172.21.15.86:0/3449176988 learned_addr learned my addr 172.21.15.86:0/3449176988 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:18.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.201+0000 7face659c640 1 --2- 172.21.15.86:0/3449176988 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7face81011a0 0x7face810e760 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.203 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.201+0000 7face6d9d640 1 --2- 172.21.15.86:0/3449176988 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7face8106f90 0x7face810f1e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.201+0000 7facdffff640 1 -- 172.21.15.86:0/3449176988 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7face81011a0 msgr2=0x7face810e760 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.201+0000 7facdffff640 1 --2- 172.21.15.86:0/3449176988 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7face81011a0 0x7face810e760 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.201+0000 7facdffff640 1 -- 172.21.15.86:0/3449176988 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7face8106f90 msgr2=0x7face810f1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.201+0000 7facdffff640 1 --2- 172.21.15.86:0/3449176988 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7face8106f90 0x7face810f1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.201+0000 7facdffff640 1 -- 172.21.15.86:0/3449176988 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7face81a6c20 con 0x7face8101b40 2024-04-21T08:51:18.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.201+0000 7face6d9d640 1 --2- 172.21.15.86:0/3449176988 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7face8106f90 0x7face810f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:18.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.202+0000 7facdffff640 1 --2- 172.21.15.86:0/3449176988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7face8101b40 0x7face810eca0 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7facd0005940 tx=0x7facd002c860 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:18.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.202+0000 7facdf7fe640 1 -- 172.21.15.86:0/3449176988 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd0043020 con 0x7face8101b40 2024-04-21T08:51:18.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.202+0000 7facdf7fe640 1 -- 172.21.15.86:0/3449176988 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facd002cc60 con 0x7face8101b40 2024-04-21T08:51:18.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.202+0000 7facecad5640 1 -- 172.21.15.86:0/3449176988 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7face81a6eb0 con 0x7face8101b40 2024-04-21T08:51:18.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.202+0000 7facdf7fe640 1 -- 172.21.15.86:0/3449176988 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd0003c30 con 0x7face8101b40 2024-04-21T08:51:18.204 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.202+0000 7facecad5640 1 -- 172.21.15.86:0/3449176988 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7face81a7370 con 0x7face8101b40 2024-04-21T08:51:18.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.204+0000 7facdf7fe640 1 -- 172.21.15.86:0/3449176988 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7facd0046020 con 0x7face8101b40 2024-04-21T08:51:18.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.204+0000 7facef79c740 1 Processor -- start 2024-04-21T08:51:18.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.205+0000 7facef79c740 1 -- start start 2024-04-21T08:51:18.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.205+0000 7facef79c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d337238580 0x55d337238960 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.205+0000 7facef79c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d337238ea0 0x55d337241540 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.205+0000 7facef79c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d337241a80 0x55d337243e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.205+0000 7facef79c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d33719d1e0 con 0x55d337238ea0 2024-04-21T08:51:18.211 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.205+0000 7facef79c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d33713a390 con 0x55d337238580 2024-04-21T08:51:18.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.205+0000 7facef79c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d3371991a0 con 0x55d337241a80 2024-04-21T08:51:18.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.205+0000 7facdf7fe640 1 --2- 172.21.15.86:0/3449176988 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7facd40a3d20 0x7facd40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.205+0000 7face659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d337238ea0 0x55d337241540 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.205+0000 7face659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d337238ea0 0x55d337241540 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56432/0 (socket says 172.21.15.86:56432) 2024-04-21T08:51:18.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.205+0000 7face659c640 1 -- 172.21.15.86:0/4190182173 learned_addr learned my addr 172.21.15.86:0/4190182173 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:18.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.205+0000 7face6d9d640 1 --2- 172.21.15.86:0/3449176988 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7facd40a3d20 0x7facd40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.205+0000 7facdffff640 1 --2- 172.21.15.86:0/4190182173 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d337241a80 0x55d337243e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.205+0000 7face6d9d640 1 --2- 172.21.15.86:0/4190182173 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d337238580 0x55d337238960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.205+0000 7facdf7fe640 1 -- 172.21.15.86:0/3449176988 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7facd00f2fa0 con 0x7face8101b40 2024-04-21T08:51:18.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.205+0000 7face659c640 1 -- 172.21.15.86:0/4190182173 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d337241a80 msgr2=0x55d337243e70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:18.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.205+0000 7face659c640 1 --2- 172.21.15.86:0/4190182173 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d337241a80 0x55d337243e70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.205+0000 7face659c640 1 -- 172.21.15.86:0/4190182173 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d337238580 msgr2=0x55d337238960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:18.212 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.205+0000 7face659c640 1 --2- 172.21.15.86:0/4190182173 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d337238580 0x55d337238960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.205+0000 7face659c640 1 -- 172.21.15.86:0/4190182173 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d336f797d0 con 0x55d337238ea0 2024-04-21T08:51:18.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.205+0000 7facdffff640 1 --2- 172.21.15.86:0/4190182173 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d337241a80 0x55d337243e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:18.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.206+0000 7face659c640 1 --2- 172.21.15.86:0/4190182173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d337238ea0 0x55d337241540 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7facd800cb80 tx=0x7facd800e720 comp rx=0 tx=0).ready entity=mon.0 client_cookie=92cd953e22ce400c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:18.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.206+0000 7face6d9d640 1 --2- 172.21.15.86:0/3449176988 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7facd40a3d20 0x7facd40a61e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7face000afd0 tx=0x7face000ada0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:18.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.206+0000 7facdcff9640 1 -- 172.21.15.86:0/4190182173 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd8010630 con 0x55d337238ea0 2024-04-21T08:51:18.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.206+0000 7facdcff9640 1 -- 172.21.15.86:0/4190182173 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facd8010c80 con 0x55d337238ea0 2024-04-21T08:51:18.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.206+0000 7facdcff9640 1 -- 172.21.15.86:0/4190182173 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd800fa30 con 0x55d337238ea0 2024-04-21T08:51:18.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.206+0000 7facef79c740 1 -- 172.21.15.86:0/4190182173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d337238ea0 msgr2=0x55d337241540 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.206+0000 7facef79c740 1 --2- 172.21.15.86:0/4190182173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d337238ea0 0x55d337241540 secure :-1 s=READY pgs=2094 cs=0 l=1 rev1=1 crypto rx=0x7facd800cb80 tx=0x7facd800e720 comp rx=0 tx=0).stop 2024-04-21T08:51:18.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.206+0000 7facef79c740 1 -- 172.21.15.86:0/4190182173 shutdown_connections 2024-04-21T08:51:18.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.206+0000 7facef79c740 1 --2- 172.21.15.86:0/4190182173 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d337241a80 0x55d337243e70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.207+0000 7facef79c740 1 --2- 172.21.15.86:0/4190182173 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d337238ea0 0x55d337241540 unknown :-1 s=CLOSED pgs=2094 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.207+0000 7facef79c740 1 --2- 172.21.15.86:0/4190182173 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d337238580 0x55d337238960 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.207+0000 7facef79c740 1 -- 172.21.15.86:0/4190182173 >> 172.21.15.86:0/4190182173 conn(0x55d337232cb0 msgr2=0x55d337233090 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:18.213 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.207+0000 7facef79c740 1 -- 172.21.15.86:0/4190182173 shutdown_connections 2024-04-21T08:51:18.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.207+0000 7facef79c740 1 -- 172.21.15.86:0/4190182173 wait complete. 2024-04-21T08:51:18.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.207+0000 7facef79c740 1 -- start start 2024-04-21T08:51:18.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.207+0000 7facef79c740 1 Processor -- start 2024-04-21T08:51:18.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.207+0000 7facef79c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d337238580 0x55d33723ea70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.207+0000 7facef79c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d337238ea0 0x55d33723efb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.207+0000 7facef79c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d337241a80 0x55d33723f4f0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.207+0000 7facef79c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55d337244870 con 0x55d337241a80 2024-04-21T08:51:18.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.207+0000 7facef79c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55d3371562f0 con 0x55d337238ea0 2024-04-21T08:51:18.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.207+0000 7facef79c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55d3372449f0 con 0x55d337238580 2024-04-21T08:51:18.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.207+0000 7facef79c740 10 client.0 _ll_register_callbacks cb 0x55d337163580 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:18.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.207+0000 7face6d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d337241a80 0x55d33723f4f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.207+0000 7face6d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d337241a80 0x55d33723f4f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56442/0 (socket says 172.21.15.86:56442) 2024-04-21T08:51:18.214 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.207+0000 7face6d9d640 1 -- 172.21.15.86:0/2511961606 learned_addr learned my addr 172.21.15.86:0/2511961606 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:18.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.208+0000 7facdffff640 1 --2- 172.21.15.86:0/2511961606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d337238ea0 0x55d33723efb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.208+0000 7face659c640 1 --2- 172.21.15.86:0/2511961606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d337238580 0x55d33723ea70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.208+0000 7face6d9d640 1 -- 172.21.15.86:0/2511961606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d337238580 msgr2=0x55d33723ea70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:18.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.208+0000 7face6d9d640 1 --2- 172.21.15.86:0/2511961606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d337238580 0x55d33723ea70 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.208+0000 7face6d9d640 1 -- 172.21.15.86:0/2511961606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d337238ea0 msgr2=0x55d33723efb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:18.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.208+0000 7face6d9d640 1 --2- 172.21.15.86:0/2511961606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d337238ea0 0x55d33723efb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.208+0000 7face6d9d640 1 -- 172.21.15.86:0/2511961606 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d336eaf3f0 con 0x55d337241a80 2024-04-21T08:51:18.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.208+0000 7face659c640 1 --2- 172.21.15.86:0/2511961606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d337238580 0x55d33723ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:18.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.208+0000 7face6d9d640 1 --2- 172.21.15.86:0/2511961606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d337241a80 0x55d33723f4f0 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7face000a770 tx=0x7face000a460 comp rx=0 tx=0).ready entity=mon.0 client_cookie=64c424d1ff8bca4f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:18.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.208+0000 7faccdffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:51:18.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.208+0000 7faccdffb640 1 -- 172.21.15.86:0/2511961606 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face0024070 con 0x55d337241a80 2024-04-21T08:51:18.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.208+0000 7faccdffb640 1 -- 172.21.15.86:0/2511961606 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7face0004d20 con 0x55d337241a80 2024-04-21T08:51:18.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.208+0000 7facef79c740 1 -- 172.21.15.86:0/2511961606 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facd8010c80 con 0x55d337241a80 2024-04-21T08:51:18.215 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.208+0000 7faccdffb640 1 -- 172.21.15.86:0/2511961606 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7face0003970 con 0x55d337241a80 2024-04-21T08:51:18.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.208+0000 7facef79c740 10 client.20520 fetch_fsmap learned FSMap version 29 2024-04-21T08:51:18.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.208+0000 7facef79c740 1 -- 172.21.15.86:0/2511961606 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d336e6f0f0 con 0x55d337241a80 2024-04-21T08:51:18.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.208+0000 7faccdffb640 1 -- 172.21.15.86:0/2511961606 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facd8010c80 con 0x55d337241a80 2024-04-21T08:51:18.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.209+0000 7faccdffb640 1 -- 172.21.15.86:0/2511961606 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7face0005020 con 0x55d337241a80 2024-04-21T08:51:18.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.209+0000 7facef79c740 10 client.20520 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:51:18.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.209+0000 7facef79c740 10 client.20520 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:51:18.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.209+0000 7facef79c740 20 client.20520 populate_metadata read hostname 'smithi086' 2024-04-21T08:51:18.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.209+0000 7facef79c740 1 --2- 172.21.15.86:0/2511961606 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d337251a20 0x55d337253e70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.209+0000 7facef79c740 4 client.20520 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:51:18.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.209+0000 7facef79c740 1 -- 172.21.15.86:0/2511961606 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55d3371aac00 con 0x55d337251a20 2024-04-21T08:51:18.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.209+0000 7face659c640 1 --2- 172.21.15.86:0/2511961606 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d337251a20 0x55d337253e70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.209+0000 7faccdffb640 10 client.20520 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:51:18.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.209+0000 7face659c640 1 --2- 172.21.15.86:0/2511961606 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d337251a20 0x55d337253e70 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=87d75a33f76b05ea server_cookie=d9628df0c522ca23 in_seq=0 out_seq=0 2024-04-21T08:51:18.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.210+0000 7faccdffb640 1 -- 172.21.15.86:0/2511961606 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55d3371aac00 con 0x55d337251a20 2024-04-21T08:51:18.216 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.210+0000 7faccdffb640 10 client.20520 handle_command_reply: tid=0 2024-04-21T08:51:18.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.210+0000 7facef79c740 1 client.20520 shutdown 2024-04-21T08:51:18.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.210+0000 7facef79c740 1 -- 172.21.15.86:0/2511961606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d337241a80 msgr2=0x55d33723f4f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.210+0000 7facef79c740 1 --2- 172.21.15.86:0/2511961606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d337241a80 0x55d33723f4f0 secure :-1 s=READY pgs=2095 cs=0 l=1 rev1=1 crypto rx=0x7face000a770 tx=0x7face000a460 comp rx=0 tx=0).stop 2024-04-21T08:51:18.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.210+0000 7facef79c740 1 -- 172.21.15.86:0/2511961606 shutdown_connections 2024-04-21T08:51:18.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.210+0000 7facef79c740 1 --2- 172.21.15.86:0/2511961606 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d337251a20 0x55d337253e70 crc :-1 s=READY pgs=351 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.210+0000 7facef79c740 1 --2- 172.21.15.86:0/2511961606 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d337241a80 0x55d33723f4f0 unknown :-1 s=CLOSED pgs=2095 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.210+0000 7facef79c740 1 --2- 172.21.15.86:0/2511961606 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d337238ea0 0x55d33723efb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.210+0000 7faccdffb640 0 client.20520 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:51:18.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.210+0000 7facef79c740 1 --2- 172.21.15.86:0/2511961606 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d337238580 0x55d33723ea70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.210+0000 7facef79c740 1 -- 172.21.15.86:0/2511961606 >> 172.21.15.86:0/2511961606 conn(0x55d337232cb0 msgr2=0x55d337234a00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:18.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.210+0000 7facef79c740 1 -- 172.21.15.86:0/2511961606 shutdown_connections 2024-04-21T08:51:18.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.210+0000 7facef79c740 1 -- 172.21.15.86:0/2511961606 wait complete. 2024-04-21T08:51:18.217 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.210+0000 7facef79c740 20 client.20520 trim_cache size 0 max 16384 2024-04-21T08:51:18.218 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:17 smithi088 ceph-mon[31259]: pgmap v392: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 25 KiB/s wr, 16 op/s 2024-04-21T08:51:18.232 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.228+0000 7facef79c740 1 Processor -- start 2024-04-21T08:51:18.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.228+0000 7facef79c740 1 -- start start 2024-04-21T08:51:18.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.228+0000 7facef79c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d337232cb0 0x55d3372411a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.228+0000 7facef79c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d337238580 0x55d33723d9b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.228+0000 7facef79c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d337238ea0 0x55d33724d730 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.228+0000 7facef79c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7face81b1530 con 0x55d337232cb0 2024-04-21T08:51:18.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.228+0000 7facef79c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7face81b3070 con 0x55d337238ea0 2024-04-21T08:51:18.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.228+0000 7facef79c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7face81b1c00 con 0x55d337238580 2024-04-21T08:51:18.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.228+0000 7face659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d337232cb0 0x55d3372411a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.228+0000 7face659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d337232cb0 0x55d3372411a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56456/0 (socket says 172.21.15.86:56456) 2024-04-21T08:51:18.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.228+0000 7face659c640 1 -- 172.21.15.86:0/2907576599 learned_addr learned my addr 172.21.15.86:0/2907576599 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:18.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.228+0000 7facdffff640 1 --2- 172.21.15.86:0/2907576599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d337238580 0x55d33723d9b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.233 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.228+0000 7face659c640 1 -- 172.21.15.86:0/2907576599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d337238580 msgr2=0x55d33723d9b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:18.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.228+0000 7face659c640 1 --2- 172.21.15.86:0/2907576599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d337238580 0x55d33723d9b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.228+0000 7face659c640 1 -- 172.21.15.86:0/2907576599 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d337238ea0 msgr2=0x55d33724d730 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:18.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.228+0000 7face6d9d640 1 --2- 172.21.15.86:0/2907576599 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d337238ea0 0x55d33724d730 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.228+0000 7face659c640 1 --2- 172.21.15.86:0/2907576599 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d337238ea0 0x55d33724d730 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.228+0000 7face659c640 1 -- 172.21.15.86:0/2907576599 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55d337243e70 con 0x55d337232cb0 2024-04-21T08:51:18.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.228+0000 7facdffff640 1 --2- 172.21.15.86:0/2907576599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d337238580 0x55d33723d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:51:18.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.228+0000 7face659c640 1 --2- 172.21.15.86:0/2907576599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d337232cb0 0x55d3372411a0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7facd8013040 tx=0x7facd800f5d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ceb4b90abaf1410d server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:18.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.229+0000 7faccffff640 1 -- 172.21.15.86:0/2907576599 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd8004220 con 0x55d337232cb0 2024-04-21T08:51:18.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.229+0000 7faccffff640 1 -- 172.21.15.86:0/2907576599 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facd80043c0 con 0x55d337232cb0 2024-04-21T08:51:18.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.229+0000 7faccffff640 1 -- 172.21.15.86:0/2907576599 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7facd800fc90 con 0x55d337232cb0 2024-04-21T08:51:18.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.229+0000 7facef79c740 1 -- 172.21.15.86:0/2907576599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d337232cb0 msgr2=0x55d3372411a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.229+0000 7facef79c740 1 --2- 172.21.15.86:0/2907576599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d337232cb0 0x55d3372411a0 secure :-1 s=READY pgs=2096 cs=0 l=1 rev1=1 crypto rx=0x7facd8013040 tx=0x7facd800f5d0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.234 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.229+0000 7facef79c740 1 -- 172.21.15.86:0/2907576599 shutdown_connections 2024-04-21T08:51:18.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.229+0000 7facef79c740 1 --2- 172.21.15.86:0/2907576599 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d337238ea0 0x55d33724d730 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.229+0000 7facef79c740 1 --2- 172.21.15.86:0/2907576599 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d337238580 0x55d33723d9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.229+0000 7facef79c740 1 --2- 172.21.15.86:0/2907576599 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d337232cb0 0x55d3372411a0 unknown :-1 s=CLOSED pgs=2096 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.229+0000 7facef79c740 1 -- 172.21.15.86:0/2907576599 >> 172.21.15.86:0/2907576599 conn(0x55d33723a440 msgr2=0x55d33723f2c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:18.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.229+0000 7facef79c740 1 -- 172.21.15.86:0/2907576599 shutdown_connections 2024-04-21T08:51:18.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.229+0000 7facef79c740 1 -- 172.21.15.86:0/2907576599 wait complete. 2024-04-21T08:51:18.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.229+0000 7facef79c740 1 -- start start 2024-04-21T08:51:18.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.229+0000 7facef79c740 1 Processor -- start 2024-04-21T08:51:18.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.230+0000 7facef79c740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d337232cb0 0x55d337249a00 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.230+0000 7facef79c740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d337238580 0x55d337249f40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.230+0000 7facef79c740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d337238ea0 0x55d337245eb0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.230+0000 7facef79c740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7face81b29a0 con 0x55d337238ea0 2024-04-21T08:51:18.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.230+0000 7facef79c740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7face81b22d0 con 0x55d337232cb0 2024-04-21T08:51:18.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.230+0000 7facef79c740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7face81af530 con 0x55d337238580 2024-04-21T08:51:18.235 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.230+0000 7facef79c740 10 client.0 _ll_register_callbacks cb 0x7face000a840 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:18.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.230+0000 7face6d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d337238ea0 0x55d337245eb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.230+0000 7face6d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d337238ea0 0x55d337245eb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56462/0 (socket says 172.21.15.86:56462) 2024-04-21T08:51:18.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.230+0000 7face6d9d640 1 -- 172.21.15.86:0/1480098879 learned_addr learned my addr 172.21.15.86:0/1480098879 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:18.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.230+0000 7face6d9d640 1 -- 172.21.15.86:0/1480098879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d337238580 msgr2=0x55d337249f40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:18.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.230+0000 7face6d9d640 1 --2- 172.21.15.86:0/1480098879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d337238580 0x55d337249f40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.230+0000 7face6d9d640 1 -- 172.21.15.86:0/1480098879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d337232cb0 msgr2=0x55d337249a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:18.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.230+0000 7face6d9d640 1 --2- 172.21.15.86:0/1480098879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d337232cb0 0x55d337249a00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.230+0000 7face6d9d640 1 -- 172.21.15.86:0/1480098879 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55d3372466b0 con 0x55d337238ea0 2024-04-21T08:51:18.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.230+0000 7face6d9d640 1 --2- 172.21.15.86:0/1480098879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d337238ea0 0x55d337245eb0 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7face0014010 tx=0x7face0004d50 comp rx=0 tx=0).ready entity=mon.0 client_cookie=71a5087c4d0de64e server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:18.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.230+0000 7faccdffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:51:18.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.230+0000 7faccdffb640 1 -- 172.21.15.86:0/1480098879 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7face0024040 con 0x55d337238ea0 2024-04-21T08:51:18.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.230+0000 7faccdffb640 1 -- 172.21.15.86:0/1480098879 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7face0035420 con 0x55d337238ea0 2024-04-21T08:51:18.236 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.230+0000 7facef79c740 1 -- 172.21.15.86:0/1480098879 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7facd80043c0 con 0x55d337238ea0 2024-04-21T08:51:18.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.230+0000 7faccdffb640 1 -- 172.21.15.86:0/1480098879 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7face0043050 con 0x55d337238ea0 2024-04-21T08:51:18.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.230+0000 7facef79c740 10 client.20526 fetch_fsmap learned FSMap version 29 2024-04-21T08:51:18.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.230+0000 7facef79c740 1 -- 172.21.15.86:0/1480098879 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55d3372c11f0 con 0x55d337238ea0 2024-04-21T08:51:18.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.231+0000 7faccdffb640 1 -- 172.21.15.86:0/1480098879 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7facd80043c0 con 0x55d337238ea0 2024-04-21T08:51:18.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.231+0000 7faccdffb640 1 -- 172.21.15.86:0/1480098879 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7face0035c60 con 0x55d337238ea0 2024-04-21T08:51:18.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.231+0000 7facef79c740 10 client.20526 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:51:18.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.231+0000 7facef79c740 10 client.20526 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:51:18.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.231+0000 7facef79c740 20 client.20526 populate_metadata read hostname 'smithi086' 2024-04-21T08:51:18.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.231+0000 7facef79c740 1 --2- 172.21.15.86:0/1480098879 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d337241a80 0x55d3372c39c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.231+0000 7facef79c740 4 client.20526 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "scrub status"}] 2024-04-21T08:51:18.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.231+0000 7facef79c740 1 -- 172.21.15.86:0/1480098879 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "scrub status"}) -- 0x55d3371a4a70 con 0x55d337241a80 2024-04-21T08:51:18.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.231+0000 7face659c640 1 --2- 172.21.15.86:0/1480098879 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d337241a80 0x55d3372c39c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.231+0000 7faccdffb640 10 client.20526 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:51:18.237 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.231+0000 7face659c640 1 --2- 172.21.15.86:0/1480098879 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d337241a80 0x55d3372c39c0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=59cbf06f50fdf170 server_cookie=9b07786ea2cc07a7 in_seq=0 out_seq=0 2024-04-21T08:51:18.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.232+0000 7faccdffb640 1 -- 172.21.15.86:0/1480098879 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+63 (crc 0 0 0) 0x55d3371a4a70 con 0x55d337241a80 2024-04-21T08:51:18.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.232+0000 7faccdffb640 10 client.20526 handle_command_reply: tid=0 2024-04-21T08:51:18.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.232+0000 7facef79c740 1 client.20526 shutdown 2024-04-21T08:51:18.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.232+0000 7facef79c740 1 -- 172.21.15.86:0/1480098879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d337238ea0 msgr2=0x55d337245eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.232+0000 7facef79c740 1 --2- 172.21.15.86:0/1480098879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d337238ea0 0x55d337245eb0 secure :-1 s=READY pgs=2097 cs=0 l=1 rev1=1 crypto rx=0x7face0014010 tx=0x7face0004d50 comp rx=0 tx=0).stop 2024-04-21T08:51:18.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.232+0000 7facef79c740 1 -- 172.21.15.86:0/1480098879 shutdown_connections 2024-04-21T08:51:18.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.232+0000 7facef79c740 1 --2- 172.21.15.86:0/1480098879 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55d337241a80 0x55d3372c39c0 crc :-1 s=READY pgs=352 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.232+0000 7facef79c740 1 --2- 172.21.15.86:0/1480098879 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55d337238ea0 0x55d337245eb0 unknown :-1 s=CLOSED pgs=2097 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.232+0000 7facef79c740 1 --2- 172.21.15.86:0/1480098879 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55d337238580 0x55d337249f40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.232+0000 7faccdffb640 0 client.20526 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:51:18.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.232+0000 7facef79c740 1 --2- 172.21.15.86:0/1480098879 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55d337232cb0 0x55d337249a00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.232+0000 7facef79c740 1 -- 172.21.15.86:0/1480098879 >> 172.21.15.86:0/1480098879 conn(0x55d33723a440 msgr2=0x55d3372314e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:18.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.232+0000 7facef79c740 1 -- 172.21.15.86:0/1480098879 shutdown_connections 2024-04-21T08:51:18.238 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.232+0000 7facef79c740 1 -- 172.21.15.86:0/1480098879 wait complete. 2024-04-21T08:51:18.239 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.232+0000 7facef79c740 20 client.20526 trim_cache size 0 max 16384 2024-04-21T08:51:18.239 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:51:18.239 INFO:teuthology.orchestra.run.smithi086.stdout: "status": "no active scrubs running", 2024-04-21T08:51:18.239 INFO:teuthology.orchestra.run.smithi086.stdout: "scrubs": {} 2024-04-21T08:51:18.239 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:51:18.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.232+0000 7facecad5640 1 -- 172.21.15.86:0/3449176988 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7facd40a3d20 msgr2=0x7facd40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.232+0000 7facecad5640 1 --2- 172.21.15.86:0/3449176988 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7facd40a3d20 0x7facd40a61e0 secure :-1 s=READY pgs=352 cs=0 l=1 rev1=1 crypto rx=0x7face000afd0 tx=0x7face000ada0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.232+0000 7facecad5640 1 -- 172.21.15.86:0/3449176988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7face8101b40 msgr2=0x7face810eca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.232+0000 7facecad5640 1 --2- 172.21.15.86:0/3449176988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7face8101b40 0x7face810eca0 secure :-1 s=READY pgs=2093 cs=0 l=1 rev1=1 crypto rx=0x7facd0005940 tx=0x7facd002c860 comp rx=0 tx=0).stop 2024-04-21T08:51:18.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.232+0000 7facecad5640 1 -- 172.21.15.86:0/3449176988 shutdown_connections 2024-04-21T08:51:18.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.232+0000 7facecad5640 1 --2- 172.21.15.86:0/3449176988 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7facd40a3d20 0x7facd40a61e0 unknown :-1 s=CLOSED pgs=352 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.232+0000 7facecad5640 1 --2- 172.21.15.86:0/3449176988 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7face8106f90 0x7face810f1e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.232+0000 7facecad5640 1 --2- 172.21.15.86:0/3449176988 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7face8101b40 0x7face810eca0 unknown :-1 s=CLOSED pgs=2093 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.232+0000 7facecad5640 1 --2- 172.21.15.86:0/3449176988 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7face81011a0 0x7face810e760 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.232+0000 7facecad5640 1 -- 172.21.15.86:0/3449176988 >> 172.21.15.86:0/3449176988 conn(0x7face8076c50 msgr2=0x7face8100130 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:18.240 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.233+0000 7facecad5640 1 -- 172.21.15.86:0/3449176988 shutdown_connections 2024-04-21T08:51:18.241 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.233+0000 7facecad5640 1 -- 172.21.15.86:0/3449176988 wait complete. 2024-04-21T08:51:18.242 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:17 smithi086 ceph-mon[27939]: pgmap v392: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 25 KiB/s wr, 16 op/s 2024-04-21T08:51:18.243 INFO:tasks.cephfs.filesystem:all active scrubs completed 2024-04-21T08:51:18.243 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:51:18.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:17 smithi179 ceph-mon[31829]: pgmap v392: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 35 KiB/s rd, 25 KiB/s wr, 16 op/s 2024-04-21T08:51:18.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.341+0000 7fd1fcd32640 1 -- 172.21.15.86:0/3843335679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1f8102b50 msgr2=0x7fd1f8106b90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.341+0000 7fd1fcd32640 1 --2- 172.21.15.86:0/3843335679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1f8102b50 0x7fd1f8106b90 secure :-1 s=READY pgs=2098 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec009900 tx=0x7fd1ec02bc30 comp rx=0 tx=0).stop 2024-04-21T08:51:18.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.341+0000 7fd1fcd32640 1 -- 172.21.15.86:0/3843335679 shutdown_connections 2024-04-21T08:51:18.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.341+0000 7fd1fcd32640 1 --2- 172.21.15.86:0/3843335679 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1f8107270 0x7fd1f810ee40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.341+0000 7fd1fcd32640 1 --2- 172.21.15.86:0/3843335679 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1f8102b50 0x7fd1f8106b90 unknown :-1 s=CLOSED pgs=2098 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.341+0000 7fd1fcd32640 1 --2- 172.21.15.86:0/3843335679 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1f81021a0 0x7fd1f8102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.343 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.341+0000 7fd1fcd32640 1 -- 172.21.15.86:0/3843335679 >> 172.21.15.86:0/3843335679 conn(0x7fd1f80793a0 msgr2=0x7fd1f80797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:18.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.341+0000 7fd1fcd32640 1 -- 172.21.15.86:0/3843335679 shutdown_connections 2024-04-21T08:51:18.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.341+0000 7fd1fcd32640 1 -- 172.21.15.86:0/3843335679 wait complete. 2024-04-21T08:51:18.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.342+0000 7fd1fcd32640 1 Processor -- start 2024-04-21T08:51:18.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.342+0000 7fd1fcd32640 1 -- start start 2024-04-21T08:51:18.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.342+0000 7fd1fcd32640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1f81021a0 0x7fd1f8198710 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.342+0000 7fd1fcd32640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1f8107270 0x7fd1f8198c50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.342+0000 7fd1fcd32640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1f819cfd0 0x7fd1f819d480 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.342+0000 7fd1fcd32640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f81121b0 con 0x7fd1f81021a0 2024-04-21T08:51:18.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.342+0000 7fd1fcd32640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f8112030 con 0x7fd1f819cfd0 2024-04-21T08:51:18.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.342+0000 7fd1fcd32640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd1f8112330 con 0x7fd1f8107270 2024-04-21T08:51:18.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.342+0000 7fd1f659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1f81021a0 0x7fd1f8198710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.342+0000 7fd1f659c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1f81021a0 0x7fd1f8198710 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56476/0 (socket says 172.21.15.86:56476) 2024-04-21T08:51:18.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.342+0000 7fd1f659c640 1 -- 172.21.15.86:0/1648506877 learned_addr learned my addr 172.21.15.86:0/1648506877 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:18.344 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.343+0000 7fd1f6d9d640 1 --2- 172.21.15.86:0/1648506877 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1f819cfd0 0x7fd1f819d480 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.343+0000 7fd1f5d9b640 1 --2- 172.21.15.86:0/1648506877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1f8107270 0x7fd1f8198c50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.343+0000 7fd1f659c640 1 -- 172.21.15.86:0/1648506877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1f8107270 msgr2=0x7fd1f8198c50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.343+0000 7fd1f659c640 1 --2- 172.21.15.86:0/1648506877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1f8107270 0x7fd1f8198c50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.343+0000 7fd1f659c640 1 -- 172.21.15.86:0/1648506877 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1f819cfd0 msgr2=0x7fd1f819d480 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.343+0000 7fd1f659c640 1 --2- 172.21.15.86:0/1648506877 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1f819cfd0 0x7fd1f819d480 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.343+0000 7fd1f659c640 1 -- 172.21.15.86:0/1648506877 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd1f819dac0 con 0x7fd1f81021a0 2024-04-21T08:51:18.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.343+0000 7fd1f6d9d640 1 --2- 172.21.15.86:0/1648506877 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1f819cfd0 0x7fd1f819d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:18.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.343+0000 7fd1f659c640 1 --2- 172.21.15.86:0/1648506877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1f81021a0 0x7fd1f8198710 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7fd1e800e9f0 tx=0x7fd1e800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:18.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.343+0000 7fd1e77fe640 1 -- 172.21.15.86:0/1648506877 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e800cd80 con 0x7fd1f81021a0 2024-04-21T08:51:18.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.343+0000 7fd1fcd32640 1 -- 172.21.15.86:0/1648506877 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd1f819ddb0 con 0x7fd1f81021a0 2024-04-21T08:51:18.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.343+0000 7fd1e77fe640 1 -- 172.21.15.86:0/1648506877 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd1e80108e0 con 0x7fd1f81021a0 2024-04-21T08:51:18.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.343+0000 7fd1e77fe640 1 -- 172.21.15.86:0/1648506877 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd1e8018980 con 0x7fd1f81021a0 2024-04-21T08:51:18.345 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.343+0000 7fd1fcd32640 1 -- 172.21.15.86:0/1648506877 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd1f81a55f0 con 0x7fd1f81021a0 2024-04-21T08:51:18.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.344+0000 7fd1fcd32640 1 -- 172.21.15.86:0/1648506877 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd1f8069230 con 0x7fd1f81021a0 2024-04-21T08:51:18.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.345+0000 7fd1e77fe640 1 -- 172.21.15.86:0/1648506877 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd1e8010430 con 0x7fd1f81021a0 2024-04-21T08:51:18.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.346+0000 7fd1e77fe640 1 --2- 172.21.15.86:0/1648506877 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd1e00a3d20 0x7fd1e00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.346 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.346+0000 7fd1f5d9b640 1 --2- 172.21.15.86:0/1648506877 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd1e00a3d20 0x7fd1e00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.346+0000 7fd1e77fe640 1 -- 172.21.15.86:0/1648506877 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7fd1e80d1010 con 0x7fd1f81021a0 2024-04-21T08:51:18.347 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.346+0000 7fd1f5d9b640 1 --2- 172.21.15.86:0/1648506877 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd1e00a3d20 0x7fd1e00a61e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec036040 tx=0x7fd1ec000bc0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:18.350 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.350+0000 7fd1e77fe640 1 -- 172.21.15.86:0/1648506877 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd1e809d2e0 con 0x7fd1f81021a0 2024-04-21T08:51:18.563 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.564+0000 7fd1fcd32640 1 -- 172.21.15.86:0/1648506877 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fd1f810a4a0 con 0x7fd1f81021a0 2024-04-21T08:51:18.564 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.564+0000 7fd1e77fe640 1 -- 172.21.15.86:0/1648506877 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7fd1e809cbe0 con 0x7fd1f81021a0 2024-04-21T08:51:18.564 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:51:18.564 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:38:45.202126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24398,24394,24455,24472]},"id":1}]} 2024-04-21T08:51:18.565 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-21T08:51:18.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.566+0000 7fd1fcd32640 1 -- 172.21.15.86:0/1648506877 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd1e00a3d20 msgr2=0x7fd1e00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.566+0000 7fd1fcd32640 1 --2- 172.21.15.86:0/1648506877 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd1e00a3d20 0x7fd1e00a61e0 secure :-1 s=READY pgs=353 cs=0 l=1 rev1=1 crypto rx=0x7fd1ec036040 tx=0x7fd1ec000bc0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.566+0000 7fd1fcd32640 1 -- 172.21.15.86:0/1648506877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1f81021a0 msgr2=0x7fd1f8198710 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.566+0000 7fd1fcd32640 1 --2- 172.21.15.86:0/1648506877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1f81021a0 0x7fd1f8198710 secure :-1 s=READY pgs=2099 cs=0 l=1 rev1=1 crypto rx=0x7fd1e800e9f0 tx=0x7fd1e800eeb0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.566+0000 7fd1fcd32640 1 -- 172.21.15.86:0/1648506877 shutdown_connections 2024-04-21T08:51:18.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.566+0000 7fd1fcd32640 1 --2- 172.21.15.86:0/1648506877 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd1e00a3d20 0x7fd1e00a61e0 unknown :-1 s=CLOSED pgs=353 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.566+0000 7fd1fcd32640 1 --2- 172.21.15.86:0/1648506877 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd1f819cfd0 0x7fd1f819d480 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.566+0000 7fd1fcd32640 1 --2- 172.21.15.86:0/1648506877 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd1f8107270 0x7fd1f8198c50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.566+0000 7fd1fcd32640 1 --2- 172.21.15.86:0/1648506877 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd1f81021a0 0x7fd1f8198710 unknown :-1 s=CLOSED pgs=2099 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.566+0000 7fd1fcd32640 1 -- 172.21.15.86:0/1648506877 >> 172.21.15.86:0/1648506877 conn(0x7fd1f80793a0 msgr2=0x7fd1f80fff50 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:18.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.567+0000 7fd1fcd32640 1 -- 172.21.15.86:0/1648506877 shutdown_connections 2024-04-21T08:51:18.567 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.567+0000 7fd1fcd32640 1 -- 172.21.15.86:0/1648506877 wait complete. 2024-04-21T08:51:18.578 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:4 damage ls 2024-04-21T08:51:18.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.680+0000 7f77d4932640 1 -- 172.21.15.86:0/3706134446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77d0101090 msgr2=0x7f77d0101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.680+0000 7f77d4932640 1 --2- 172.21.15.86:0/3706134446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77d0101090 0x7f77d0101470 secure :-1 s=READY pgs=2100 cs=0 l=1 rev1=1 crypto rx=0x7f77c0009900 tx=0x7f77c002bc30 comp rx=0 tx=0).stop 2024-04-21T08:51:18.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.680+0000 7f77d4932640 1 -- 172.21.15.86:0/3706134446 shutdown_connections 2024-04-21T08:51:18.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.680+0000 7f77d4932640 1 --2- 172.21.15.86:0/3706134446 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77d0106160 0x7f77d010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.680+0000 7f77d4932640 1 --2- 172.21.15.86:0/3706134446 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77d0101a40 0x7f77d0105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.682 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.680+0000 7f77d4932640 1 --2- 172.21.15.86:0/3706134446 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77d0101090 0x7f77d0101470 unknown :-1 s=CLOSED pgs=2100 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.680+0000 7f77d4932640 1 -- 172.21.15.86:0/3706134446 >> 172.21.15.86:0/3706134446 conn(0x7f77d0077f50 msgr2=0x7f77d0078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.680+0000 7f77d4932640 1 -- 172.21.15.86:0/3706134446 shutdown_connections 2024-04-21T08:51:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.680+0000 7f77d4932640 1 -- 172.21.15.86:0/3706134446 wait complete. 2024-04-21T08:51:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.681+0000 7f77d4932640 1 Processor -- start 2024-04-21T08:51:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.681+0000 7f77d4932640 1 -- start start 2024-04-21T08:51:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.681+0000 7f77d4932640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77d0101090 0x7f77d01995e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.681+0000 7f77d4932640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77d0106160 0x7f77d0199b20 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.681+0000 7f77d4932640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77d019a0b0 0x7f77d01a4660 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.681+0000 7f77d4932640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f77d01110c0 con 0x7f77d0106160 2024-04-21T08:51:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.681+0000 7f77d4932640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f77d0110f40 con 0x7f77d0101090 2024-04-21T08:51:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.681+0000 7f77d4932640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f77d0111240 con 0x7f77d019a0b0 2024-04-21T08:51:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.681+0000 7f77c7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77d0106160 0x7f77d0199b20 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.681+0000 7f77c7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77d0106160 0x7f77d0199b20 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56488/0 (socket says 172.21.15.86:56488) 2024-04-21T08:51:18.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.681+0000 7f77c7fff640 1 -- 172.21.15.86:0/2453352545 learned_addr learned my addr 172.21.15.86:0/2453352545 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.682+0000 7f77ce59c640 1 --2- 172.21.15.86:0/2453352545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77d0101090 0x7f77d01995e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.682+0000 7f77ced9d640 1 --2- 172.21.15.86:0/2453352545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77d019a0b0 0x7f77d01a4660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.682+0000 7f77c7fff640 1 -- 172.21.15.86:0/2453352545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77d019a0b0 msgr2=0x7f77d01a4660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.682+0000 7f77c7fff640 1 --2- 172.21.15.86:0/2453352545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77d019a0b0 0x7f77d01a4660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.682+0000 7f77c7fff640 1 -- 172.21.15.86:0/2453352545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77d0101090 msgr2=0x7f77d01995e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.682+0000 7f77c7fff640 1 --2- 172.21.15.86:0/2453352545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77d0101090 0x7f77d01995e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.682+0000 7f77c7fff640 1 -- 172.21.15.86:0/2453352545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f77d01a4e30 con 0x7f77d0106160 2024-04-21T08:51:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.682+0000 7f77ce59c640 1 --2- 172.21.15.86:0/2453352545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77d0101090 0x7f77d01995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.682+0000 7f77ced9d640 1 --2- 172.21.15.86:0/2453352545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77d019a0b0 0x7f77d01a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:51:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.682+0000 7f77c7fff640 1 --2- 172.21.15.86:0/2453352545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77d0106160 0x7f77d0199b20 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f77b800d740 tx=0x7f77b800dc00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.682+0000 7f77c77fe640 1 -- 172.21.15.86:0/2453352545 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b8017070 con 0x7f77d0106160 2024-04-21T08:51:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.682+0000 7f77c77fe640 1 -- 172.21.15.86:0/2453352545 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77b800de50 con 0x7f77d0106160 2024-04-21T08:51:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.682+0000 7f77d4932640 1 -- 172.21.15.86:0/2453352545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f77d01a5120 con 0x7f77d0106160 2024-04-21T08:51:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.682+0000 7f77c77fe640 1 -- 172.21.15.86:0/2453352545 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b8004ce0 con 0x7f77d0106160 2024-04-21T08:51:18.684 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.682+0000 7f77d4932640 1 -- 172.21.15.86:0/2453352545 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f77d01a5620 con 0x7f77d0106160 2024-04-21T08:51:18.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.684+0000 7f77c77fe640 1 -- 172.21.15.86:0/2453352545 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f77b8004e80 con 0x7f77d0106160 2024-04-21T08:51:18.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.685+0000 7f77d7603740 1 Processor -- start 2024-04-21T08:51:18.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.685+0000 7f77d7603740 1 -- start start 2024-04-21T08:51:18.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.685+0000 7f77d7603740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e9174d0 0x564a6e9178b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.685+0000 7f77d7603740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a6e917e80 0x564a6e920430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.685+0000 7f77d7603740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a6e920a00 0x564a6e922e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.685+0000 7f77d7603740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564a6e87c1f0 con 0x564a6e9174d0 2024-04-21T08:51:18.693 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.685+0000 7f77d7603740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564a6e8194b0 con 0x564a6e917e80 2024-04-21T08:51:18.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.685+0000 7f77d7603740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564a6e8781b0 con 0x564a6e920a00 2024-04-21T08:51:18.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.685+0000 7f77ced9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e9174d0 0x564a6e9178b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.685+0000 7f77ced9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e9174d0 0x564a6e9178b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56490/0 (socket says 172.21.15.86:56490) 2024-04-21T08:51:18.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.685+0000 7f77ced9d640 1 -- 172.21.15.86:0/3219027455 learned_addr learned my addr 172.21.15.86:0/3219027455 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:18.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.685+0000 7f77c77fe640 1 --2- 172.21.15.86:0/2453352545 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f77bc0a3da0 0x7f77bc0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.685+0000 7f77c7fff640 1 --2- 172.21.15.86:0/3219027455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a6e920a00 0x564a6e922e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.685+0000 7f77ce59c640 1 --2- 172.21.15.86:0/3219027455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a6e917e80 0x564a6e920430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.685+0000 7f77ced9d640 1 --2- 172.21.15.86:0/2453352545 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f77bc0a3da0 0x7f77bc0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.685+0000 7f77ced9d640 1 -- 172.21.15.86:0/3219027455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a6e920a00 msgr2=0x564a6e922e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:18.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.685+0000 7f77ced9d640 1 --2- 172.21.15.86:0/3219027455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a6e920a00 0x564a6e922e50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.685+0000 7f77ced9d640 1 -- 172.21.15.86:0/3219027455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a6e917e80 msgr2=0x564a6e920430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:18.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.685+0000 7f77ced9d640 1 --2- 172.21.15.86:0/3219027455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a6e917e80 0x564a6e920430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.685+0000 7f77c77fe640 1 -- 172.21.15.86:0/2453352545 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7f77b80d0690 con 0x7f77d0106160 2024-04-21T08:51:18.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.686+0000 7f77ced9d640 1 -- 172.21.15.86:0/3219027455 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a6e80b1b0 con 0x564a6e9174d0 2024-04-21T08:51:18.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.686+0000 7f77ced9d640 1 --2- 172.21.15.86:0/3219027455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e9174d0 0x564a6e9178b0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f77c8013910 tx=0x7f77c8013dd0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=206990a6e389a82c server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:18.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.686+0000 7f77c4ff9640 1 -- 172.21.15.86:0/3219027455 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c8004b20 con 0x564a6e9174d0 2024-04-21T08:51:18.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.686+0000 7f77c4ff9640 1 -- 172.21.15.86:0/3219027455 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77c8004cc0 con 0x564a6e9174d0 2024-04-21T08:51:18.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.686+0000 7f77c4ff9640 1 -- 172.21.15.86:0/3219027455 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c80056f0 con 0x564a6e9174d0 2024-04-21T08:51:18.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.686+0000 7f77ced9d640 1 --2- 172.21.15.86:0/2453352545 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f77bc0a3da0 0x7f77bc0a6260 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f77c8020000 tx=0x7f77c801f000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:18.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.687+0000 7f77d7603740 1 -- 172.21.15.86:0/3219027455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e9174d0 msgr2=0x564a6e9178b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.687+0000 7f77d7603740 1 --2- 172.21.15.86:0/3219027455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e9174d0 0x564a6e9178b0 secure :-1 s=READY pgs=2102 cs=0 l=1 rev1=1 crypto rx=0x7f77c8013910 tx=0x7f77c8013dd0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.687+0000 7f77d7603740 1 -- 172.21.15.86:0/3219027455 shutdown_connections 2024-04-21T08:51:18.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.687+0000 7f77d7603740 1 --2- 172.21.15.86:0/3219027455 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a6e920a00 0x564a6e922e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.687+0000 7f77d7603740 1 --2- 172.21.15.86:0/3219027455 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a6e917e80 0x564a6e920430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.687+0000 7f77d7603740 1 --2- 172.21.15.86:0/3219027455 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e9174d0 0x564a6e9178b0 unknown :-1 s=CLOSED pgs=2102 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.687+0000 7f77d7603740 1 -- 172.21.15.86:0/3219027455 >> 172.21.15.86:0/3219027455 conn(0x564a6e911ba0 msgr2=0x564a6e911f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:18.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.687+0000 7f77d7603740 1 -- 172.21.15.86:0/3219027455 shutdown_connections 2024-04-21T08:51:18.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.687+0000 7f77d7603740 1 -- 172.21.15.86:0/3219027455 wait complete. 2024-04-21T08:51:18.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.687+0000 7f77d7603740 1 -- start start 2024-04-21T08:51:18.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.687+0000 7f77d7603740 1 Processor -- start 2024-04-21T08:51:18.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.687+0000 7f77d7603740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e9174d0 0x564a6e920370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.687+0000 7f77d7603740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a6e917e80 0x564a6e91d700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.687+0000 7f77d7603740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a6e920a00 0x564a6e91dd10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.687+0000 7f77d7603740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x564a6e923970 con 0x564a6e9174d0 2024-04-21T08:51:18.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.687+0000 7f77d7603740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x564a6e835500 con 0x564a6e920a00 2024-04-21T08:51:18.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.687+0000 7f77d7603740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x564a6e923af0 con 0x564a6e917e80 2024-04-21T08:51:18.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.687+0000 7f77d7603740 10 client.0 _ll_register_callbacks cb 0x564a6e803b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:18.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.687+0000 7f77ce59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e9174d0 0x564a6e920370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.687+0000 7f77ce59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e9174d0 0x564a6e920370 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56506/0 (socket says 172.21.15.86:56506) 2024-04-21T08:51:18.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.687+0000 7f77ce59c640 1 -- 172.21.15.86:0/4248311671 learned_addr learned my addr 172.21.15.86:0/4248311671 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:18.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.688+0000 7f77ced9d640 1 --2- 172.21.15.86:0/4248311671 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a6e920a00 0x564a6e91dd10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.688+0000 7f77ce59c640 1 -- 172.21.15.86:0/4248311671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a6e917e80 msgr2=0x564a6e91d700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:18.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.688+0000 7f77ce59c640 1 --2- 172.21.15.86:0/4248311671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a6e917e80 0x564a6e91d700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.688+0000 7f77ce59c640 1 -- 172.21.15.86:0/4248311671 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a6e920a00 msgr2=0x564a6e91dd10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:18.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.688+0000 7f77ce59c640 1 --2- 172.21.15.86:0/4248311671 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a6e920a00 0x564a6e91dd10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.688+0000 7f77ce59c640 1 -- 172.21.15.86:0/4248311671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a6e58e3f0 con 0x564a6e9174d0 2024-04-21T08:51:18.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.688+0000 7f77ce59c640 1 --2- 172.21.15.86:0/4248311671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e9174d0 0x564a6e920370 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f77c0009900 tx=0x7f77c002ca70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=4430da0b35b3f4b9 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:18.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.688+0000 7f77b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:51:18.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.688+0000 7f77b5ffb640 1 -- 172.21.15.86:0/4248311671 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c0005450 con 0x564a6e9174d0 2024-04-21T08:51:18.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.688+0000 7f77b5ffb640 1 -- 172.21.15.86:0/4248311671 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7f77c00046c0 con 0x564a6e9174d0 2024-04-21T08:51:18.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.688+0000 7f77d7603740 1 -- 172.21.15.86:0/4248311671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77c8004cc0 con 0x564a6e9174d0 2024-04-21T08:51:18.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.688+0000 7f77b5ffb640 1 -- 172.21.15.86:0/4248311671 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f77c0052050 con 0x564a6e9174d0 2024-04-21T08:51:18.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.688+0000 7f77d7603740 10 client.20544 fetch_fsmap learned FSMap version 29 2024-04-21T08:51:18.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.688+0000 7f77d7603740 1 -- 172.21.15.86:0/4248311671 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564a6e6587d0 con 0x564a6e9174d0 2024-04-21T08:51:18.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.688+0000 7f77b5ffb640 1 -- 172.21.15.86:0/4248311671 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77c8004cc0 con 0x564a6e9174d0 2024-04-21T08:51:18.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.688+0000 7f77b5ffb640 1 -- 172.21.15.86:0/4248311671 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f77c0003680 con 0x564a6e9174d0 2024-04-21T08:51:18.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.688+0000 7f77d7603740 10 client.20544 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:51:18.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.688+0000 7f77d7603740 10 client.20544 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:51:18.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.688+0000 7f77d7603740 20 client.20544 populate_metadata read hostname 'smithi086' 2024-04-21T08:51:18.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.689+0000 7f77d7603740 1 --2- 172.21.15.86:0/4248311671 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x564a6e930c50 0x564a6e9330c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.689+0000 7f77d7603740 4 client.20544 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:51:18.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.689+0000 7f77d7603740 1 -- 172.21.15.86:0/4248311671 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x564a6e81b930 con 0x564a6e930c50 2024-04-21T08:51:18.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.689+0000 7f77c7fff640 1 --2- 172.21.15.86:0/4248311671 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x564a6e930c50 0x564a6e9330c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.689+0000 7f77b5ffb640 10 client.20544 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:51:18.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.689+0000 7f77c7fff640 1 --2- 172.21.15.86:0/4248311671 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x564a6e930c50 0x564a6e9330c0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=1c85c21330fa730c server_cookie=141d10682fcc8dfe in_seq=0 out_seq=0 2024-04-21T08:51:18.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.690+0000 7f77b5ffb640 1 -- 172.21.15.86:0/4248311671 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x564a6e81b930 con 0x564a6e930c50 2024-04-21T08:51:18.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.690+0000 7f77b5ffb640 10 client.20544 handle_command_reply: tid=0 2024-04-21T08:51:18.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.690+0000 7f77d7603740 1 client.20544 shutdown 2024-04-21T08:51:18.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.690+0000 7f77d7603740 1 -- 172.21.15.86:0/4248311671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e9174d0 msgr2=0x564a6e920370 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.690+0000 7f77d7603740 1 --2- 172.21.15.86:0/4248311671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e9174d0 0x564a6e920370 secure :-1 s=READY pgs=2103 cs=0 l=1 rev1=1 crypto rx=0x7f77c0009900 tx=0x7f77c002ca70 comp rx=0 tx=0).stop 2024-04-21T08:51:18.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.690+0000 7f77d7603740 1 -- 172.21.15.86:0/4248311671 shutdown_connections 2024-04-21T08:51:18.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.690+0000 7f77d7603740 1 --2- 172.21.15.86:0/4248311671 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x564a6e930c50 0x564a6e9330c0 crc :-1 s=READY pgs=78 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.699 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.690+0000 7f77d7603740 1 --2- 172.21.15.86:0/4248311671 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a6e920a00 0x564a6e91dd10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.690+0000 7f77d7603740 1 --2- 172.21.15.86:0/4248311671 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a6e917e80 0x564a6e91d700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.690+0000 7f77b5ffb640 0 client.20544 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:51:18.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.690+0000 7f77d7603740 1 --2- 172.21.15.86:0/4248311671 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e9174d0 0x564a6e920370 unknown :-1 s=CLOSED pgs=2103 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.690+0000 7f77d7603740 1 -- 172.21.15.86:0/4248311671 >> 172.21.15.86:0/4248311671 conn(0x564a6e911ba0 msgr2=0x564a6e9138f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:18.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.690+0000 7f77d7603740 1 -- 172.21.15.86:0/4248311671 shutdown_connections 2024-04-21T08:51:18.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.691+0000 7f77d7603740 1 -- 172.21.15.86:0/4248311671 wait complete. 2024-04-21T08:51:18.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.691+0000 7f77d7603740 20 client.20544 trim_cache size 0 max 16384 2024-04-21T08:51:18.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.704+0000 7f77d7603740 1 Processor -- start 2024-04-21T08:51:18.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.704+0000 7f77d7603740 1 -- start start 2024-04-21T08:51:18.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.704+0000 7f77d7603740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a6e9174d0 0x564a6e92c830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.704+0000 7f77d7603740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e917e80 0x564a6e91fe30 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.704+0000 7f77d7603740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a6e920a00 0x564a6e920370 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.704+0000 7f77d7603740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f77d01af3a0 con 0x564a6e917e80 2024-04-21T08:51:18.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.704+0000 7f77d7603740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f77d01b0ee0 con 0x564a6e920a00 2024-04-21T08:51:18.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.704+0000 7f77d7603740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f77d01afa70 con 0x564a6e9174d0 2024-04-21T08:51:18.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.704+0000 7f77c7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e917e80 0x564a6e91fe30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.704+0000 7f77c7fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e917e80 0x564a6e91fe30 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56508/0 (socket says 172.21.15.86:56508) 2024-04-21T08:51:18.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.704+0000 7f77c7fff640 1 -- 172.21.15.86:0/4093173463 learned_addr learned my addr 172.21.15.86:0/4093173463 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:18.710 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.704+0000 7f77c7fff640 1 -- 172.21.15.86:0/4093173463 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a6e9174d0 msgr2=0x564a6e92c830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:18.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.704+0000 7f77ced9d640 1 --2- 172.21.15.86:0/4093173463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a6e920a00 0x564a6e920370 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.704+0000 7f77c7fff640 1 --2- 172.21.15.86:0/4093173463 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a6e9174d0 0x564a6e92c830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.704+0000 7f77c7fff640 1 -- 172.21.15.86:0/4093173463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a6e920a00 msgr2=0x564a6e920370 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:18.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.704+0000 7f77c7fff640 1 --2- 172.21.15.86:0/4093173463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a6e920a00 0x564a6e920370 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.704+0000 7f77c7fff640 1 -- 172.21.15.86:0/4093173463 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x564a6e91e080 con 0x564a6e917e80 2024-04-21T08:51:18.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.704+0000 7f77ced9d640 1 --2- 172.21.15.86:0/4093173463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a6e920a00 0x564a6e920370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:51:18.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.704+0000 7f77c7fff640 1 --2- 172.21.15.86:0/4093173463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e917e80 0x564a6e91fe30 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f77b8098660 tx=0x7f77b80d2a60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b27b7114428ef1b2 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:18.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.704+0000 7f77b7fff640 1 -- 172.21.15.86:0/4093173463 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b80d0070 con 0x564a6e917e80 2024-04-21T08:51:18.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.704+0000 7f77b7fff640 1 -- 172.21.15.86:0/4093173463 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77b809a500 con 0x564a6e917e80 2024-04-21T08:51:18.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.704+0000 7f77b7fff640 1 -- 172.21.15.86:0/4093173463 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77b8020040 con 0x564a6e917e80 2024-04-21T08:51:18.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.705+0000 7f77d7603740 1 -- 172.21.15.86:0/4093173463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e917e80 msgr2=0x564a6e91fe30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.711 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.705+0000 7f77d7603740 1 --2- 172.21.15.86:0/4093173463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e917e80 0x564a6e91fe30 secure :-1 s=READY pgs=2104 cs=0 l=1 rev1=1 crypto rx=0x7f77b8098660 tx=0x7f77b80d2a60 comp rx=0 tx=0).stop 2024-04-21T08:51:18.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.705+0000 7f77d7603740 1 -- 172.21.15.86:0/4093173463 shutdown_connections 2024-04-21T08:51:18.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.705+0000 7f77d7603740 1 --2- 172.21.15.86:0/4093173463 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a6e920a00 0x564a6e920370 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.705+0000 7f77d7603740 1 --2- 172.21.15.86:0/4093173463 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e917e80 0x564a6e91fe30 unknown :-1 s=CLOSED pgs=2104 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.705+0000 7f77d7603740 1 --2- 172.21.15.86:0/4093173463 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a6e9174d0 0x564a6e92c830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.705+0000 7f77d7603740 1 -- 172.21.15.86:0/4093173463 >> 172.21.15.86:0/4093173463 conn(0x564a6e911ba0 msgr2=0x564a6e910c00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:18.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.705+0000 7f77d7603740 1 -- 172.21.15.86:0/4093173463 shutdown_connections 2024-04-21T08:51:18.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.705+0000 7f77d7603740 1 -- 172.21.15.86:0/4093173463 wait complete. 2024-04-21T08:51:18.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.705+0000 7f77d7603740 1 -- start start 2024-04-21T08:51:18.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.705+0000 7f77d7603740 1 Processor -- start 2024-04-21T08:51:18.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.706+0000 7f77d7603740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e9174d0 0x564a6e924b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.706+0000 7f77d7603740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a6e917e80 0x564a6e9250a0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.706+0000 7f77d7603740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a6e920a00 0x564a6e996200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.706+0000 7f77d7603740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f77d01b0810 con 0x564a6e9174d0 2024-04-21T08:51:18.712 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.706+0000 7f77d7603740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f77d01b0140 con 0x564a6e920a00 2024-04-21T08:51:18.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.706+0000 7f77d7603740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f77d01ad3a0 con 0x564a6e917e80 2024-04-21T08:51:18.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.706+0000 7f77d7603740 10 client.0 _ll_register_callbacks cb 0x7f77c0004520 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:18.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.706+0000 7f77ce59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e9174d0 0x564a6e924b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.706+0000 7f77ce59c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e9174d0 0x564a6e924b60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56518/0 (socket says 172.21.15.86:56518) 2024-04-21T08:51:18.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.706+0000 7f77ce59c640 1 -- 172.21.15.86:0/4291995982 learned_addr learned my addr 172.21.15.86:0/4291995982 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:18.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.706+0000 7f77ced9d640 1 --2- 172.21.15.86:0/4291995982 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a6e920a00 0x564a6e996200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.706+0000 7f77c7fff640 1 --2- 172.21.15.86:0/4291995982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a6e917e80 0x564a6e9250a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.706+0000 7f77ce59c640 1 -- 172.21.15.86:0/4291995982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a6e917e80 msgr2=0x564a6e9250a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:18.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.706+0000 7f77ce59c640 1 --2- 172.21.15.86:0/4291995982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a6e917e80 0x564a6e9250a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.706+0000 7f77ce59c640 1 -- 172.21.15.86:0/4291995982 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a6e920a00 msgr2=0x564a6e996200 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:18.713 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.706+0000 7f77ce59c640 1 --2- 172.21.15.86:0/4291995982 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a6e920a00 0x564a6e996200 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.706+0000 7f77ce59c640 1 -- 172.21.15.86:0/4291995982 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x564a6e996740 con 0x564a6e9174d0 2024-04-21T08:51:18.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.706+0000 7f77ce59c640 1 --2- 172.21.15.86:0/4291995982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e9174d0 0x564a6e924b60 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f77c0009850 tx=0x7f77c0005e80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b715c2ad8eebd7a4 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:18.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.706+0000 7f77b5ffb640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:51:18.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.706+0000 7f77b5ffb640 1 -- 172.21.15.86:0/4291995982 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f77c002cc00 con 0x564a6e9174d0 2024-04-21T08:51:18.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.706+0000 7f77b5ffb640 1 -- 172.21.15.86:0/4291995982 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7f77c004f040 con 0x564a6e9174d0 2024-04-21T08:51:18.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.706+0000 7f77d7603740 1 -- 172.21.15.86:0/4291995982 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f77b809a500 con 0x564a6e9174d0 2024-04-21T08:51:18.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.707+0000 7f77b5ffb640 1 -- 172.21.15.86:0/4291995982 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f77c0063050 con 0x564a6e9174d0 2024-04-21T08:51:18.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.707+0000 7f77d7603740 10 client.20550 fetch_fsmap learned FSMap version 29 2024-04-21T08:51:18.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.707+0000 7f77d7603740 1 -- 172.21.15.86:0/4291995982 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x564a6e997810 con 0x564a6e9174d0 2024-04-21T08:51:18.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.707+0000 7f77b5ffb640 1 -- 172.21.15.86:0/4291995982 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f77b809a500 con 0x564a6e9174d0 2024-04-21T08:51:18.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.707+0000 7f77b5ffb640 1 -- 172.21.15.86:0/4291995982 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f77c0004750 con 0x564a6e9174d0 2024-04-21T08:51:18.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.707+0000 7f77d7603740 10 client.20550 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:51:18.714 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.707+0000 7f77d7603740 10 client.20550 resolve_mds: resolved 1:4 to role '1:4' aka daemon mds.l 2024-04-21T08:51:18.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.707+0000 7f77d7603740 20 client.20550 populate_metadata read hostname 'smithi086' 2024-04-21T08:51:18.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.707+0000 7f77d7603740 1 --2- 172.21.15.86:0/4291995982 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x564a6e997f60 0x564a6e99a380 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.707+0000 7f77d7603740 4 client.20550 mds_command: new command op to 24394 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:51:18.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.707+0000 7f77d7603740 1 -- 172.21.15.86:0/4291995982 --> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] -- command(tid 0: {"prefix": "damage ls"}) -- 0x564a6e889c10 con 0x564a6e997f60 2024-04-21T08:51:18.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.707+0000 7f77c7fff640 1 --2- 172.21.15.86:0/4291995982 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x564a6e997f60 0x564a6e99a380 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.708+0000 7f77b5ffb640 10 client.20550 ms_handle_connect on v2:172.21.15.179:6832/3606421532 2024-04-21T08:51:18.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.708+0000 7f77c7fff640 1 --2- 172.21.15.86:0/4291995982 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x564a6e997f60 0x564a6e99a380 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.4 client_cookie=c9ff8611a57b153f server_cookie=7deb1e02b653ff46 in_seq=0 out_seq=0 2024-04-21T08:51:18.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.708+0000 7f77b5ffb640 1 -- 172.21.15.86:0/4291995982 <== mds.4 v2:172.21.15.179:6832/3606421532 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x564a6e889c10 con 0x564a6e997f60 2024-04-21T08:51:18.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.708+0000 7f77b5ffb640 10 client.20550 handle_command_reply: tid=0 2024-04-21T08:51:18.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.708+0000 7f77d7603740 1 client.20550 shutdown 2024-04-21T08:51:18.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.708+0000 7f77d7603740 1 -- 172.21.15.86:0/4291995982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e9174d0 msgr2=0x564a6e924b60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.708+0000 7f77d7603740 1 --2- 172.21.15.86:0/4291995982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e9174d0 0x564a6e924b60 secure :-1 s=READY pgs=2105 cs=0 l=1 rev1=1 crypto rx=0x7f77c0009850 tx=0x7f77c0005e80 comp rx=0 tx=0).stop 2024-04-21T08:51:18.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.708+0000 7f77d7603740 1 -- 172.21.15.86:0/4291995982 shutdown_connections 2024-04-21T08:51:18.715 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.708+0000 7f77d7603740 1 --2- 172.21.15.86:0/4291995982 >> [v2:172.21.15.179:6832/3606421532,v1:172.21.15.179:6833/3606421532] conn(0x564a6e997f60 0x564a6e99a380 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.708+0000 7f77d7603740 1 --2- 172.21.15.86:0/4291995982 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x564a6e920a00 0x564a6e996200 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.708+0000 7f77b5ffb640 0 client.20550 ms_handle_reset on v2:172.21.15.179:6832/3606421532 2024-04-21T08:51:18.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.708+0000 7f77d7603740 1 --2- 172.21.15.86:0/4291995982 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x564a6e917e80 0x564a6e9250a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.708+0000 7f77d7603740 1 --2- 172.21.15.86:0/4291995982 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x564a6e9174d0 0x564a6e924b60 unknown :-1 s=CLOSED pgs=2105 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.708+0000 7f77d7603740 1 -- 172.21.15.86:0/4291995982 >> 172.21.15.86:0/4291995982 conn(0x564a6e911ba0 msgr2=0x564a6e910370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:18.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.708+0000 7f77d7603740 1 -- 172.21.15.86:0/4291995982 shutdown_connections 2024-04-21T08:51:18.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.708+0000 7f77d7603740 1 -- 172.21.15.86:0/4291995982 wait complete. 2024-04-21T08:51:18.716 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.708+0000 7f77d7603740 20 client.20550 trim_cache size 0 max 16384 2024-04-21T08:51:18.716 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:51:18.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.708+0000 7f77d4932640 1 -- 172.21.15.86:0/2453352545 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f77bc0a3da0 msgr2=0x7f77bc0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.717 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.708+0000 7f77d4932640 1 --2- 172.21.15.86:0/2453352545 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f77bc0a3da0 0x7f77bc0a6260 secure :-1 s=READY pgs=354 cs=0 l=1 rev1=1 crypto rx=0x7f77c8020000 tx=0x7f77c801f000 comp rx=0 tx=0).stop 2024-04-21T08:51:18.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.708+0000 7f77d4932640 1 -- 172.21.15.86:0/2453352545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77d0106160 msgr2=0x7f77d0199b20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.708+0000 7f77d4932640 1 --2- 172.21.15.86:0/2453352545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77d0106160 0x7f77d0199b20 secure :-1 s=READY pgs=2101 cs=0 l=1 rev1=1 crypto rx=0x7f77b800d740 tx=0x7f77b800dc00 comp rx=0 tx=0).stop 2024-04-21T08:51:18.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.709+0000 7f77d4932640 1 -- 172.21.15.86:0/2453352545 shutdown_connections 2024-04-21T08:51:18.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.709+0000 7f77d4932640 1 --2- 172.21.15.86:0/2453352545 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f77bc0a3da0 0x7f77bc0a6260 unknown :-1 s=CLOSED pgs=354 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.709+0000 7f77d4932640 1 --2- 172.21.15.86:0/2453352545 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f77d019a0b0 0x7f77d01a4660 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.709+0000 7f77d4932640 1 --2- 172.21.15.86:0/2453352545 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f77d0106160 0x7f77d0199b20 unknown :-1 s=CLOSED pgs=2101 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.709+0000 7f77d4932640 1 --2- 172.21.15.86:0/2453352545 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f77d0101090 0x7f77d01995e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.709+0000 7f77d4932640 1 -- 172.21.15.86:0/2453352545 >> 172.21.15.86:0/2453352545 conn(0x7f77d0077f50 msgr2=0x7f77d00ff1d0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:18.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.709+0000 7f77d4932640 1 -- 172.21.15.86:0/2453352545 shutdown_connections 2024-04-21T08:51:18.718 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.709+0000 7f77d4932640 1 -- 172.21.15.86:0/2453352545 wait complete. 2024-04-21T08:51:18.719 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:3 damage ls 2024-04-21T08:51:18.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.820+0000 7f12d9df4640 1 -- 172.21.15.86:0/2943949378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12d406a550 msgr2=0x7f12d406a990 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.820+0000 7f12d9df4640 1 --2- 172.21.15.86:0/2943949378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12d406a550 0x7f12d406a990 secure :-1 s=READY pgs=2106 cs=0 l=1 rev1=1 crypto rx=0x7f12bc009930 tx=0x7f12bc02bc30 comp rx=0 tx=0).stop 2024-04-21T08:51:18.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.820+0000 7f12d9df4640 1 -- 172.21.15.86:0/2943949378 shutdown_connections 2024-04-21T08:51:18.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.820+0000 7f12d9df4640 1 --2- 172.21.15.86:0/2943949378 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12d406aed0 0x7f12d410edf0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.820+0000 7f12d9df4640 1 --2- 172.21.15.86:0/2943949378 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12d406a550 0x7f12d406a990 unknown :-1 s=CLOSED pgs=2106 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.820+0000 7f12d9df4640 1 --2- 172.21.15.86:0/2943949378 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12d4107fc0 0x7f12d41083a0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.822 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.820+0000 7f12d9df4640 1 -- 172.21.15.86:0/2943949378 >> 172.21.15.86:0/2943949378 conn(0x7f12d4076c50 msgr2=0x7f12d4077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:18.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.820+0000 7f12d9df4640 1 -- 172.21.15.86:0/2943949378 shutdown_connections 2024-04-21T08:51:18.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.820+0000 7f12d9df4640 1 -- 172.21.15.86:0/2943949378 wait complete. 2024-04-21T08:51:18.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.821+0000 7f12d9df4640 1 Processor -- start 2024-04-21T08:51:18.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.821+0000 7f12d9df4640 1 -- start start 2024-04-21T08:51:18.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.821+0000 7f12d9df4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12d406a550 0x7f12d419e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.821+0000 7f12d9df4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12d406aed0 0x7f12d419e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.821+0000 7f12d9df4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12d4107fc0 0x7f12d41985b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.821+0000 7f12d9df4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f12d4112160 con 0x7f12d406a550 2024-04-21T08:51:18.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.821+0000 7f12d9df4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f12d4111fe0 con 0x7f12d406aed0 2024-04-21T08:51:18.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.821+0000 7f12d9df4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f12d41122e0 con 0x7f12d4107fc0 2024-04-21T08:51:18.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.821+0000 7f12d37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12d406a550 0x7f12d419e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.821+0000 7f12d37fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12d406a550 0x7f12d419e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56546/0 (socket says 172.21.15.86:56546) 2024-04-21T08:51:18.823 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.821+0000 7f12d37fe640 1 -- 172.21.15.86:0/2673005307 learned_addr learned my addr 172.21.15.86:0/2673005307 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:18.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.821+0000 7f12cbfff640 1 --2- 172.21.15.86:0/2673005307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12d406aed0 0x7f12d419e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.821+0000 7f12d3fff640 1 --2- 172.21.15.86:0/2673005307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12d4107fc0 0x7f12d41985b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.821+0000 7f12d37fe640 1 -- 172.21.15.86:0/2673005307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12d4107fc0 msgr2=0x7f12d41985b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.821+0000 7f12d37fe640 1 --2- 172.21.15.86:0/2673005307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12d4107fc0 0x7f12d41985b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.821+0000 7f12d37fe640 1 -- 172.21.15.86:0/2673005307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12d406aed0 msgr2=0x7f12d419e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.821+0000 7f12d37fe640 1 --2- 172.21.15.86:0/2673005307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12d406aed0 0x7f12d419e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.821+0000 7f12d37fe640 1 -- 172.21.15.86:0/2673005307 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f12d4198d30 con 0x7f12d406a550 2024-04-21T08:51:18.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.821+0000 7f12d3fff640 1 --2- 172.21.15.86:0/2673005307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12d4107fc0 0x7f12d41985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:18.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.822+0000 7f12d37fe640 1 --2- 172.21.15.86:0/2673005307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12d406a550 0x7f12d419e400 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f12c400d950 tx=0x7f12c400de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:18.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.822+0000 7f12d17fa640 1 -- 172.21.15.86:0/2673005307 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c4017030 con 0x7f12d406a550 2024-04-21T08:51:18.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.822+0000 7f12d9df4640 1 -- 172.21.15.86:0/2673005307 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f12d4199020 con 0x7f12d406a550 2024-04-21T08:51:18.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.822+0000 7f12d17fa640 1 -- 172.21.15.86:0/2673005307 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12c4004020 con 0x7f12d406a550 2024-04-21T08:51:18.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.822+0000 7f12d17fa640 1 -- 172.21.15.86:0/2673005307 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12c40047a0 con 0x7f12d406a550 2024-04-21T08:51:18.824 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.822+0000 7f12d9df4640 1 -- 172.21.15.86:0/2673005307 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f12d41a52c0 con 0x7f12d406a550 2024-04-21T08:51:18.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.824+0000 7f12d17fa640 1 -- 172.21.15.86:0/2673005307 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f12c4009a30 con 0x7f12d406a550 2024-04-21T08:51:18.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.824+0000 7f12dc9e8740 1 Processor -- start 2024-04-21T08:51:18.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.824+0000 7f12dc9e8740 1 -- start start 2024-04-21T08:51:18.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.825+0000 7f12dc9e8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56085c4bb4d0 0x56085c4bb8b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.825+0000 7f12dc9e8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56085c4bbe80 0x56085c4c4430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.825+0000 7f12dc9e8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085c4c4a00 0x56085c4c6e50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.825+0000 7f12dc9e8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56085c4201f0 con 0x56085c4bb4d0 2024-04-21T08:51:18.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.825+0000 7f12dc9e8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x56085c3bd4b0 con 0x56085c4bbe80 2024-04-21T08:51:18.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.825+0000 7f12dc9e8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56085c41c1b0 con 0x56085c4c4a00 2024-04-21T08:51:18.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.825+0000 7f12d3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56085c4bb4d0 0x56085c4bb8b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.834 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.825+0000 7f12d3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56085c4bb4d0 0x56085c4bb8b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56556/0 (socket says 172.21.15.86:56556) 2024-04-21T08:51:18.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.825+0000 7f12d3fff640 1 -- 172.21.15.86:0/1223240410 learned_addr learned my addr 172.21.15.86:0/1223240410 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:18.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.825+0000 7f12d17fa640 1 --2- 172.21.15.86:0/2673005307 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f12c00a3d20 0x7f12c00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.825+0000 7f12d37fe640 1 --2- 172.21.15.86:0/1223240410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56085c4bbe80 0x56085c4c4430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.825+0000 7f12d3fff640 1 -- 172.21.15.86:0/1223240410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085c4c4a00 msgr2=0x56085c4c6e50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:18.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.825+0000 7f12d3fff640 1 --2- 172.21.15.86:0/1223240410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085c4c4a00 0x56085c4c6e50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.825+0000 7f12d3fff640 1 -- 172.21.15.86:0/1223240410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56085c4bbe80 msgr2=0x56085c4c4430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:18.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.825+0000 7f12d3fff640 1 --2- 172.21.15.86:0/1223240410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56085c4bbe80 0x56085c4c4430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.825+0000 7f12d3fff640 1 -- 172.21.15.86:0/1223240410 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56085c3af1b0 con 0x56085c4bb4d0 2024-04-21T08:51:18.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.825+0000 7f12d37fe640 1 --2- 172.21.15.86:0/1223240410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56085c4bbe80 0x56085c4c4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:51:18.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.825+0000 7f12d17fa640 1 -- 172.21.15.86:0/2673005307 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7f12c40d0580 con 0x7f12d406a550 2024-04-21T08:51:18.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.825+0000 7f12d3fff640 1 --2- 172.21.15.86:0/2673005307 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f12c00a3d20 0x7f12c00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.825+0000 7f12d3fff640 1 --2- 172.21.15.86:0/1223240410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56085c4bb4d0 0x56085c4bb8b0 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f12cc010540 tx=0x7f12cc010a00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=fbfded3a47cf5707 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:18.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.826+0000 7f12c9ffb640 1 -- 172.21.15.86:0/1223240410 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12cc01d400 con 0x56085c4bb4d0 2024-04-21T08:51:18.835 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.826+0000 7f12c9ffb640 1 -- 172.21.15.86:0/1223240410 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12cc0134c0 con 0x56085c4bb4d0 2024-04-21T08:51:18.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.826+0000 7f12c9ffb640 1 -- 172.21.15.86:0/1223240410 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12cc0117a0 con 0x56085c4bb4d0 2024-04-21T08:51:18.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.826+0000 7f12dc9e8740 1 -- 172.21.15.86:0/1223240410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56085c4bb4d0 msgr2=0x56085c4bb8b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.826+0000 7f12dc9e8740 1 --2- 172.21.15.86:0/1223240410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56085c4bb4d0 0x56085c4bb8b0 secure :-1 s=READY pgs=2108 cs=0 l=1 rev1=1 crypto rx=0x7f12cc010540 tx=0x7f12cc010a00 comp rx=0 tx=0).stop 2024-04-21T08:51:18.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.826+0000 7f12d3fff640 1 --2- 172.21.15.86:0/2673005307 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f12c00a3d20 0x7f12c00a61e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f12cc021000 tx=0x7f12cc01e000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:18.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.826+0000 7f12dc9e8740 1 -- 172.21.15.86:0/1223240410 shutdown_connections 2024-04-21T08:51:18.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.826+0000 7f12dc9e8740 1 --2- 172.21.15.86:0/1223240410 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085c4c4a00 0x56085c4c6e50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.826+0000 7f12dc9e8740 1 --2- 172.21.15.86:0/1223240410 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56085c4bbe80 0x56085c4c4430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.826+0000 7f12dc9e8740 1 --2- 172.21.15.86:0/1223240410 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56085c4bb4d0 0x56085c4bb8b0 unknown :-1 s=CLOSED pgs=2108 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.826+0000 7f12dc9e8740 1 -- 172.21.15.86:0/1223240410 >> 172.21.15.86:0/1223240410 conn(0x56085c4b5ba0 msgr2=0x56085c4b5f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:18.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.826+0000 7f12dc9e8740 1 -- 172.21.15.86:0/1223240410 shutdown_connections 2024-04-21T08:51:18.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.826+0000 7f12dc9e8740 1 -- 172.21.15.86:0/1223240410 wait complete. 2024-04-21T08:51:18.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.827+0000 7f12dc9e8740 1 -- start start 2024-04-21T08:51:18.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.827+0000 7f12dc9e8740 1 Processor -- start 2024-04-21T08:51:18.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.827+0000 7f12dc9e8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56085c4bb4d0 0x56085c4c42d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.836 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.827+0000 7f12dc9e8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085c4bbe80 0x56085c4c1660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.827+0000 7f12dc9e8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56085c4c4a00 0x56085c4c1bc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.827+0000 7f12dc9e8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x56085c4c7970 con 0x56085c4c4a00 2024-04-21T08:51:18.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.827+0000 7f12dc9e8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x56085c3d9500 con 0x56085c4bb4d0 2024-04-21T08:51:18.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.827+0000 7f12dc9e8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x56085c4c7af0 con 0x56085c4bbe80 2024-04-21T08:51:18.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.827+0000 7f12dc9e8740 10 client.0 _ll_register_callbacks cb 0x56085c3a7b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:18.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.827+0000 7f12d3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56085c4c4a00 0x56085c4c1bc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.827+0000 7f12d3fff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56085c4c4a00 0x56085c4c1bc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56564/0 (socket says 172.21.15.86:56564) 2024-04-21T08:51:18.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.827+0000 7f12d3fff640 1 -- 172.21.15.86:0/3715202101 learned_addr learned my addr 172.21.15.86:0/3715202101 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:18.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.827+0000 7f12cbfff640 1 --2- 172.21.15.86:0/3715202101 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085c4bbe80 0x56085c4c1660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.837 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.827+0000 7f12d37fe640 1 --2- 172.21.15.86:0/3715202101 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56085c4bb4d0 0x56085c4c42d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.827+0000 7f12d3fff640 1 -- 172.21.15.86:0/3715202101 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085c4bbe80 msgr2=0x56085c4c1660 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:18.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.827+0000 7f12d3fff640 1 --2- 172.21.15.86:0/3715202101 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085c4bbe80 0x56085c4c1660 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.827+0000 7f12d3fff640 1 -- 172.21.15.86:0/3715202101 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56085c4bb4d0 msgr2=0x56085c4c42d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:18.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.827+0000 7f12d3fff640 1 --2- 172.21.15.86:0/3715202101 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56085c4bb4d0 0x56085c4c42d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.828+0000 7f12d3fff640 1 -- 172.21.15.86:0/3715202101 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56085c1323f0 con 0x56085c4c4a00 2024-04-21T08:51:18.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.828+0000 7f12d3fff640 1 --2- 172.21.15.86:0/3715202101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56085c4c4a00 0x56085c4c1bc0 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f12cc011420 tx=0x7f12cc011d00 comp rx=0 tx=0).ready entity=mon.0 client_cookie=989a28d9ff79c517 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:18.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.828+0000 7f12b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:51:18.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.828+0000 7f12b2ffd640 1 -- 172.21.15.86:0/3715202101 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12cc034060 con 0x56085c4c4a00 2024-04-21T08:51:18.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.828+0000 7f12b2ffd640 1 -- 172.21.15.86:0/3715202101 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7f12cc004c10 con 0x56085c4c4a00 2024-04-21T08:51:18.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.828+0000 7f12dc9e8740 1 -- 172.21.15.86:0/3715202101 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12cc0134c0 con 0x56085c4c4a00 2024-04-21T08:51:18.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.828+0000 7f12b2ffd640 1 -- 172.21.15.86:0/3715202101 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f12cc03b050 con 0x56085c4c4a00 2024-04-21T08:51:18.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.828+0000 7f12dc9e8740 10 client.20562 fetch_fsmap learned FSMap version 29 2024-04-21T08:51:18.838 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.828+0000 7f12dc9e8740 1 -- 172.21.15.86:0/3715202101 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56085c1fc7d0 con 0x56085c4c4a00 2024-04-21T08:51:18.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.828+0000 7f12b2ffd640 1 -- 172.21.15.86:0/3715202101 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12cc0134c0 con 0x56085c4c4a00 2024-04-21T08:51:18.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.828+0000 7f12b2ffd640 1 -- 172.21.15.86:0/3715202101 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f12cc00f040 con 0x56085c4c4a00 2024-04-21T08:51:18.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.828+0000 7f12dc9e8740 10 client.20562 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:51:18.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.828+0000 7f12dc9e8740 10 client.20562 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:51:18.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.828+0000 7f12dc9e8740 20 client.20562 populate_metadata read hostname 'smithi086' 2024-04-21T08:51:18.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.828+0000 7f12dc9e8740 1 --2- 172.21.15.86:0/3715202101 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x56085c4d4ae0 0x56085c4d6ed0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.828+0000 7f12dc9e8740 4 client.20562 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:51:18.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.828+0000 7f12dc9e8740 1 -- 172.21.15.86:0/3715202101 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x56085c3bf930 con 0x56085c4d4ae0 2024-04-21T08:51:18.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.829+0000 7f12d37fe640 1 --2- 172.21.15.86:0/3715202101 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x56085c4d4ae0 0x56085c4d6ed0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.829+0000 7f12b2ffd640 10 client.20562 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:51:18.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.829+0000 7f12d37fe640 1 --2- 172.21.15.86:0/3715202101 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x56085c4d4ae0 0x56085c4d6ed0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=beff9ea40414989 server_cookie=dd079876c73a6a9c in_seq=0 out_seq=0 2024-04-21T08:51:18.839 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.830+0000 7f12b2ffd640 1 -- 172.21.15.86:0/3715202101 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x56085c3bf930 con 0x56085c4d4ae0 2024-04-21T08:51:18.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.830+0000 7f12b2ffd640 10 client.20562 handle_command_reply: tid=0 2024-04-21T08:51:18.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.830+0000 7f12dc9e8740 1 client.20562 shutdown 2024-04-21T08:51:18.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.830+0000 7f12dc9e8740 1 -- 172.21.15.86:0/3715202101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56085c4c4a00 msgr2=0x56085c4c1bc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.830+0000 7f12dc9e8740 1 --2- 172.21.15.86:0/3715202101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56085c4c4a00 0x56085c4c1bc0 secure :-1 s=READY pgs=2109 cs=0 l=1 rev1=1 crypto rx=0x7f12cc011420 tx=0x7f12cc011d00 comp rx=0 tx=0).stop 2024-04-21T08:51:18.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.830+0000 7f12dc9e8740 1 -- 172.21.15.86:0/3715202101 shutdown_connections 2024-04-21T08:51:18.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.830+0000 7f12dc9e8740 1 --2- 172.21.15.86:0/3715202101 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x56085c4d4ae0 0x56085c4d6ed0 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.830+0000 7f12dc9e8740 1 --2- 172.21.15.86:0/3715202101 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56085c4c4a00 0x56085c4c1bc0 unknown :-1 s=CLOSED pgs=2109 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.830+0000 7f12dc9e8740 1 --2- 172.21.15.86:0/3715202101 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085c4bbe80 0x56085c4c1660 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.830+0000 7f12b2ffd640 0 client.20562 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:51:18.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.830+0000 7f12dc9e8740 1 --2- 172.21.15.86:0/3715202101 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56085c4bb4d0 0x56085c4c42d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.830+0000 7f12dc9e8740 1 -- 172.21.15.86:0/3715202101 >> 172.21.15.86:0/3715202101 conn(0x56085c4b5ba0 msgr2=0x56085c4b78f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:18.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.830+0000 7f12dc9e8740 1 -- 172.21.15.86:0/3715202101 shutdown_connections 2024-04-21T08:51:18.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.830+0000 7f12dc9e8740 1 -- 172.21.15.86:0/3715202101 wait complete. 2024-04-21T08:51:18.840 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.830+0000 7f12dc9e8740 20 client.20562 trim_cache size 0 max 16384 2024-04-21T08:51:18.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.843+0000 7f12dc9e8740 1 Processor -- start 2024-04-21T08:51:18.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.843+0000 7f12dc9e8740 1 -- start start 2024-04-21T08:51:18.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.844+0000 7f12dc9e8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085c4bb4d0 0x56085c4d0830 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.848 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.844+0000 7f12dc9e8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56085c4bbe80 0x56085c4b7800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.844+0000 7f12dc9e8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56085c4c4a00 0x56085c4c3d90 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.844+0000 7f12dc9e8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f12d41af4e0 con 0x56085c4bbe80 2024-04-21T08:51:18.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.844+0000 7f12dc9e8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f12d41b1020 con 0x56085c4c4a00 2024-04-21T08:51:18.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.844+0000 7f12dc9e8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f12d41afbb0 con 0x56085c4bb4d0 2024-04-21T08:51:18.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.844+0000 7f12cbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56085c4bbe80 0x56085c4b7800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.844+0000 7f12cbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56085c4bbe80 0x56085c4b7800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56578/0 (socket says 172.21.15.86:56578) 2024-04-21T08:51:18.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.844+0000 7f12cbfff640 1 -- 172.21.15.86:0/3536020693 learned_addr learned my addr 172.21.15.86:0/3536020693 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:18.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.844+0000 7f12d37fe640 1 --2- 172.21.15.86:0/3536020693 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085c4bb4d0 0x56085c4d0830 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.844+0000 7f12cbfff640 1 -- 172.21.15.86:0/3536020693 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085c4bb4d0 msgr2=0x56085c4d0830 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:18.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.844+0000 7f12cbfff640 1 --2- 172.21.15.86:0/3536020693 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085c4bb4d0 0x56085c4d0830 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.844+0000 7f12cbfff640 1 -- 172.21.15.86:0/3536020693 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56085c4c4a00 msgr2=0x56085c4c3d90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:18.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.844+0000 7f12cbfff640 1 --2- 172.21.15.86:0/3536020693 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56085c4c4a00 0x56085c4c3d90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.844+0000 7f12cbfff640 1 -- 172.21.15.86:0/3536020693 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x56085c4bfbb0 con 0x56085c4bbe80 2024-04-21T08:51:18.849 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.844+0000 7f12cbfff640 1 --2- 172.21.15.86:0/3536020693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56085c4bbe80 0x56085c4b7800 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f12bc036040 tx=0x7f12bc02c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=8afcb562fde071ba server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:18.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.844+0000 7f12c97fa640 1 -- 172.21.15.86:0/3536020693 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc004140 con 0x56085c4bbe80 2024-04-21T08:51:18.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.844+0000 7f12c97fa640 1 -- 172.21.15.86:0/3536020693 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12bc02cc90 con 0x56085c4bbe80 2024-04-21T08:51:18.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.844+0000 7f12c97fa640 1 -- 172.21.15.86:0/3536020693 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc0046c0 con 0x56085c4bbe80 2024-04-21T08:51:18.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.845+0000 7f12dc9e8740 1 -- 172.21.15.86:0/3536020693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56085c4bbe80 msgr2=0x56085c4b7800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.845+0000 7f12dc9e8740 1 --2- 172.21.15.86:0/3536020693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56085c4bbe80 0x56085c4b7800 secure :-1 s=READY pgs=2110 cs=0 l=1 rev1=1 crypto rx=0x7f12bc036040 tx=0x7f12bc02c710 comp rx=0 tx=0).stop 2024-04-21T08:51:18.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.845+0000 7f12dc9e8740 1 -- 172.21.15.86:0/3536020693 shutdown_connections 2024-04-21T08:51:18.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.845+0000 7f12dc9e8740 1 --2- 172.21.15.86:0/3536020693 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56085c4c4a00 0x56085c4c3d90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.845+0000 7f12dc9e8740 1 --2- 172.21.15.86:0/3536020693 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56085c4bbe80 0x56085c4b7800 unknown :-1 s=CLOSED pgs=2110 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.845+0000 7f12dc9e8740 1 --2- 172.21.15.86:0/3536020693 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085c4bb4d0 0x56085c4d0830 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.845+0000 7f12dc9e8740 1 -- 172.21.15.86:0/3536020693 >> 172.21.15.86:0/3536020693 conn(0x56085c4b5ba0 msgr2=0x56085c4bc260 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:18.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.845+0000 7f12dc9e8740 1 -- 172.21.15.86:0/3536020693 shutdown_connections 2024-04-21T08:51:18.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.845+0000 7f12dc9e8740 1 -- 172.21.15.86:0/3536020693 wait complete. 2024-04-21T08:51:18.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.845+0000 7f12dc9e8740 1 -- start start 2024-04-21T08:51:18.850 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.845+0000 7f12dc9e8740 1 Processor -- start 2024-04-21T08:51:18.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.846+0000 7f12dc9e8740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56085c4bb4d0 0x56085c4c8700 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.846+0000 7f12dc9e8740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56085c4bbe80 0x56085c4c8c40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.846+0000 7f12dc9e8740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085c4c4a00 0x56085c4c9180 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.846+0000 7f12dc9e8740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f12d41b0950 con 0x56085c4bbe80 2024-04-21T08:51:18.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.846+0000 7f12dc9e8740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f12d41b0280 con 0x56085c4bb4d0 2024-04-21T08:51:18.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.846+0000 7f12dc9e8740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f12d41ad4e0 con 0x56085c4c4a00 2024-04-21T08:51:18.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.846+0000 7f12dc9e8740 10 client.0 _ll_register_callbacks cb 0x7f12cc009a60 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:18.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.846+0000 7f12cbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56085c4bbe80 0x56085c4c8c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.846+0000 7f12cbfff640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56085c4bbe80 0x56085c4c8c40 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56594/0 (socket says 172.21.15.86:56594) 2024-04-21T08:51:18.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.846+0000 7f12cbfff640 1 -- 172.21.15.86:0/1478520674 learned_addr learned my addr 172.21.15.86:0/1478520674 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:18.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.846+0000 7f12d37fe640 1 --2- 172.21.15.86:0/1478520674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56085c4bb4d0 0x56085c4c8700 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.846+0000 7f12cbfff640 1 -- 172.21.15.86:0/1478520674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085c4c4a00 msgr2=0x56085c4c9180 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:18.851 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.846+0000 7f12d3fff640 1 --2- 172.21.15.86:0/1478520674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085c4c4a00 0x56085c4c9180 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.846+0000 7f12cbfff640 1 --2- 172.21.15.86:0/1478520674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085c4c4a00 0x56085c4c9180 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.846+0000 7f12cbfff640 1 -- 172.21.15.86:0/1478520674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56085c4bb4d0 msgr2=0x56085c4c8700 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:18.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.846+0000 7f12cbfff640 1 --2- 172.21.15.86:0/1478520674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56085c4bb4d0 0x56085c4c8700 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.846+0000 7f12cbfff640 1 -- 172.21.15.86:0/1478520674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x56085c539eb0 con 0x56085c4bbe80 2024-04-21T08:51:18.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.846+0000 7f12cbfff640 1 --2- 172.21.15.86:0/1478520674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56085c4bbe80 0x56085c4c8c40 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f12bc02c170 tx=0x7f12bc02cb20 comp rx=0 tx=0).ready entity=mon.0 client_cookie=db27930399aefc72 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:18.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.846+0000 7f12b2ffd640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:51:18.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.846+0000 7f12b2ffd640 1 -- 172.21.15.86:0/1478520674 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f12bc0037f0 con 0x56085c4bbe80 2024-04-21T08:51:18.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.846+0000 7f12b2ffd640 1 -- 172.21.15.86:0/1478520674 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7f12bc003990 con 0x56085c4bbe80 2024-04-21T08:51:18.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.846+0000 7f12dc9e8740 1 -- 172.21.15.86:0/1478520674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f12bc02cc90 con 0x56085c4bbe80 2024-04-21T08:51:18.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.847+0000 7f12b2ffd640 1 -- 172.21.15.86:0/1478520674 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f12bc004490 con 0x56085c4bbe80 2024-04-21T08:51:18.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.847+0000 7f12dc9e8740 10 client.20568 fetch_fsmap learned FSMap version 29 2024-04-21T08:51:18.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.847+0000 7f12dc9e8740 1 -- 172.21.15.86:0/1478520674 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x56085c53b0f0 con 0x56085c4bbe80 2024-04-21T08:51:18.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.847+0000 7f12b2ffd640 1 -- 172.21.15.86:0/1478520674 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f12bc02cc90 con 0x56085c4bbe80 2024-04-21T08:51:18.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.847+0000 7f12b2ffd640 1 -- 172.21.15.86:0/1478520674 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f12bc034440 con 0x56085c4bbe80 2024-04-21T08:51:18.852 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.847+0000 7f12dc9e8740 10 client.20568 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:51:18.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.847+0000 7f12dc9e8740 10 client.20568 resolve_mds: resolved 1:3 to role '1:3' aka daemon mds.k 2024-04-21T08:51:18.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.847+0000 7f12dc9e8740 20 client.20568 populate_metadata read hostname 'smithi086' 2024-04-21T08:51:18.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.847+0000 7f12dc9e8740 1 --2- 172.21.15.86:0/1478520674 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x56085c53b7f0 0x56085c53dc10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.847+0000 7f12dc9e8740 4 client.20568 mds_command: new command op to 24398 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:51:18.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.847+0000 7f12dc9e8740 1 -- 172.21.15.86:0/1478520674 --> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] -- command(tid 0: {"prefix": "damage ls"}) -- 0x56085c42dc10 con 0x56085c53b7f0 2024-04-21T08:51:18.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.847+0000 7f12d37fe640 1 --2- 172.21.15.86:0/1478520674 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x56085c53b7f0 0x56085c53dc10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.848+0000 7f12d37fe640 1 --2- 172.21.15.86:0/1478520674 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x56085c53b7f0 0x56085c53dc10 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.3 client_cookie=f9fb0264b4cc32c0 server_cookie=9d8a6eca60b8132d in_seq=0 out_seq=0 2024-04-21T08:51:18.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.848+0000 7f12b2ffd640 10 client.20568 ms_handle_connect on v2:172.21.15.88:6832/1872744608 2024-04-21T08:51:18.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.848+0000 7f12b2ffd640 1 -- 172.21.15.86:0/1478520674 <== mds.3 v2:172.21.15.88:6832/1872744608 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x56085c42dc10 con 0x56085c53b7f0 2024-04-21T08:51:18.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.848+0000 7f12b2ffd640 10 client.20568 handle_command_reply: tid=0 2024-04-21T08:51:18.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.848+0000 7f12dc9e8740 1 client.20568 shutdown 2024-04-21T08:51:18.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.848+0000 7f12dc9e8740 1 -- 172.21.15.86:0/1478520674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56085c4bbe80 msgr2=0x56085c4c8c40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.853 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.848+0000 7f12dc9e8740 1 --2- 172.21.15.86:0/1478520674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56085c4bbe80 0x56085c4c8c40 secure :-1 s=READY pgs=2111 cs=0 l=1 rev1=1 crypto rx=0x7f12bc02c170 tx=0x7f12bc02cb20 comp rx=0 tx=0).stop 2024-04-21T08:51:18.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.848+0000 7f12dc9e8740 1 -- 172.21.15.86:0/1478520674 shutdown_connections 2024-04-21T08:51:18.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.848+0000 7f12dc9e8740 1 --2- 172.21.15.86:0/1478520674 >> [v2:172.21.15.88:6832/1872744608,v1:172.21.15.88:6833/1872744608] conn(0x56085c53b7f0 0x56085c53dc10 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.848+0000 7f12dc9e8740 1 --2- 172.21.15.86:0/1478520674 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x56085c4c4a00 0x56085c4c9180 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.848+0000 7f12dc9e8740 1 --2- 172.21.15.86:0/1478520674 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x56085c4bbe80 0x56085c4c8c40 unknown :-1 s=CLOSED pgs=2111 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.848+0000 7f12b2ffd640 0 client.20568 ms_handle_reset on v2:172.21.15.88:6832/1872744608 2024-04-21T08:51:18.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.848+0000 7f12dc9e8740 1 --2- 172.21.15.86:0/1478520674 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x56085c4bb4d0 0x56085c4c8700 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.848+0000 7f12dc9e8740 1 -- 172.21.15.86:0/1478520674 >> 172.21.15.86:0/1478520674 conn(0x56085c4b5ba0 msgr2=0x56085c4b7680 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:18.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.848+0000 7f12dc9e8740 1 -- 172.21.15.86:0/1478520674 shutdown_connections 2024-04-21T08:51:18.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.848+0000 7f12dc9e8740 1 -- 172.21.15.86:0/1478520674 wait complete. 2024-04-21T08:51:18.854 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.848+0000 7f12dc9e8740 20 client.20568 trim_cache size 0 max 16384 2024-04-21T08:51:18.854 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:51:18.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.849+0000 7f12d9df4640 1 -- 172.21.15.86:0/2673005307 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f12c00a3d20 msgr2=0x7f12c00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.855 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.849+0000 7f12d9df4640 1 --2- 172.21.15.86:0/2673005307 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f12c00a3d20 0x7f12c00a61e0 secure :-1 s=READY pgs=355 cs=0 l=1 rev1=1 crypto rx=0x7f12cc021000 tx=0x7f12cc01e000 comp rx=0 tx=0).stop 2024-04-21T08:51:18.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.849+0000 7f12d9df4640 1 -- 172.21.15.86:0/2673005307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12d406a550 msgr2=0x7f12d419e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.849+0000 7f12d9df4640 1 --2- 172.21.15.86:0/2673005307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12d406a550 0x7f12d419e400 secure :-1 s=READY pgs=2107 cs=0 l=1 rev1=1 crypto rx=0x7f12c400d950 tx=0x7f12c400de10 comp rx=0 tx=0).stop 2024-04-21T08:51:18.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.849+0000 7f12d9df4640 1 -- 172.21.15.86:0/2673005307 shutdown_connections 2024-04-21T08:51:18.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.849+0000 7f12d9df4640 1 --2- 172.21.15.86:0/2673005307 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f12c00a3d20 0x7f12c00a61e0 unknown :-1 s=CLOSED pgs=355 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.849+0000 7f12d9df4640 1 --2- 172.21.15.86:0/2673005307 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f12d4107fc0 0x7f12d41985b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.849+0000 7f12d9df4640 1 --2- 172.21.15.86:0/2673005307 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f12d406aed0 0x7f12d419e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.849+0000 7f12d9df4640 1 --2- 172.21.15.86:0/2673005307 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f12d406a550 0x7f12d419e400 unknown :-1 s=CLOSED pgs=2107 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.849+0000 7f12d9df4640 1 -- 172.21.15.86:0/2673005307 >> 172.21.15.86:0/2673005307 conn(0x7f12d4076c50 msgr2=0x7f12d41000a0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:18.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.849+0000 7f12d9df4640 1 -- 172.21.15.86:0/2673005307 shutdown_connections 2024-04-21T08:51:18.856 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.849+0000 7f12d9df4640 1 -- 172.21.15.86:0/2673005307 wait complete. 2024-04-21T08:51:18.859 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:2 damage ls 2024-04-21T08:51:18.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.961+0000 7f42457b6640 1 -- 172.21.15.86:0/1959265454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4240107270 msgr2=0x7f424010ee40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.961 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.961+0000 7f42457b6640 1 --2- 172.21.15.86:0/1959265454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4240107270 0x7f424010ee40 secure :-1 s=READY pgs=2112 cs=0 l=1 rev1=1 crypto rx=0x7f4238009900 tx=0x7f423802bc30 comp rx=0 tx=0).stop 2024-04-21T08:51:18.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.961+0000 7f42457b6640 1 -- 172.21.15.86:0/1959265454 shutdown_connections 2024-04-21T08:51:18.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.961+0000 7f42457b6640 1 --2- 172.21.15.86:0/1959265454 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4240107270 0x7f424010ee40 unknown :-1 s=CLOSED pgs=2112 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.961+0000 7f42457b6640 1 --2- 172.21.15.86:0/1959265454 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f4240102b50 0x7f4240106b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.961+0000 7f42457b6640 1 --2- 172.21.15.86:0/1959265454 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f42401021a0 0x7f4240102580 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.961+0000 7f42457b6640 1 -- 172.21.15.86:0/1959265454 >> 172.21.15.86:0/1959265454 conn(0x7f4240079350 msgr2=0x7f4240079760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:18.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.961+0000 7f42457b6640 1 -- 172.21.15.86:0/1959265454 shutdown_connections 2024-04-21T08:51:18.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.961+0000 7f42457b6640 1 -- 172.21.15.86:0/1959265454 wait complete. 2024-04-21T08:51:18.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.962+0000 7f42457b6640 1 Processor -- start 2024-04-21T08:51:18.962 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.962+0000 7f42457b6640 1 -- start start 2024-04-21T08:51:18.964 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.962+0000 7f42457b6640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42401021a0 0x7f4240074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.962+0000 7f42457b6640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4240107270 0x7f4240074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.962+0000 7f42457b6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4240074cd0 0x7f42401a6930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.962+0000 7f42457b6640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f42401121e0 con 0x7f4240074cd0 2024-04-21T08:51:18.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.962+0000 7f42457b6640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4240112060 con 0x7f4240107270 2024-04-21T08:51:18.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.962+0000 7f42457b6640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4240112360 con 0x7f42401021a0 2024-04-21T08:51:18.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.963+0000 7f423f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4240074cd0 0x7f42401a6930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.963+0000 7f423f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4240074cd0 0x7f42401a6930 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56602/0 (socket says 172.21.15.86:56602) 2024-04-21T08:51:18.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.963+0000 7f423f7fe640 1 -- 172.21.15.86:0/218261914 learned_addr learned my addr 172.21.15.86:0/218261914 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:18.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.963+0000 7f423e7fc640 1 --2- 172.21.15.86:0/218261914 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4240107270 0x7f4240074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.963+0000 7f423effd640 1 --2- 172.21.15.86:0/218261914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42401021a0 0x7f4240074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.963+0000 7f423f7fe640 1 -- 172.21.15.86:0/218261914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42401021a0 msgr2=0x7f4240074250 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.963+0000 7f423f7fe640 1 --2- 172.21.15.86:0/218261914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42401021a0 0x7f4240074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.963+0000 7f423f7fe640 1 -- 172.21.15.86:0/218261914 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4240107270 msgr2=0x7f4240074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.965 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.963+0000 7f423f7fe640 1 --2- 172.21.15.86:0/218261914 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4240107270 0x7f4240074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.963+0000 7f423f7fe640 1 -- 172.21.15.86:0/218261914 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f42401a7030 con 0x7f4240074cd0 2024-04-21T08:51:18.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.963+0000 7f423e7fc640 1 --2- 172.21.15.86:0/218261914 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4240107270 0x7f4240074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:18.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.963+0000 7f423f7fe640 1 --2- 172.21.15.86:0/218261914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4240074cd0 0x7f42401a6930 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f4238009230 tx=0x7f42380045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:18.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.963+0000 7f4227fff640 1 -- 172.21.15.86:0/218261914 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238043020 con 0x7f4240074cd0 2024-04-21T08:51:18.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.963+0000 7f42457b6640 1 -- 172.21.15.86:0/218261914 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f42401a72c0 con 0x7f4240074cd0 2024-04-21T08:51:18.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.963+0000 7f4227fff640 1 -- 172.21.15.86:0/218261914 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f423802cb70 con 0x7f4240074cd0 2024-04-21T08:51:18.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.963+0000 7f4227fff640 1 -- 172.21.15.86:0/218261914 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4238003ac0 con 0x7f4240074cd0 2024-04-21T08:51:18.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.963+0000 7f42457b6640 1 -- 172.21.15.86:0/218261914 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f42401a7810 con 0x7f4240074cd0 2024-04-21T08:51:18.966 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.965+0000 7f4227fff640 1 -- 172.21.15.86:0/218261914 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4238046020 con 0x7f4240074cd0 2024-04-21T08:51:18.972 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.966+0000 7f42483a9740 1 Processor -- start 2024-04-21T08:51:18.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.966+0000 7f42483a9740 1 -- start start 2024-04-21T08:51:18.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.966+0000 7f4227fff640 1 --2- 172.21.15.86:0/218261914 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f42280a3d50 0x7f42280a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.966+0000 7f42483a9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f0d3b14d0 0x556f0d3b18b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.966+0000 7f42483a9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556f0d3b1e80 0x556f0d3ba430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.966+0000 7f42483a9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556f0d3baa00 0x556f0d3bce50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.966+0000 7f42483a9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556f0d3161f0 con 0x556f0d3b14d0 2024-04-21T08:51:18.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.966+0000 7f42483a9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x556f0d2b34b0 con 0x556f0d3baa00 2024-04-21T08:51:18.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.966+0000 7f42483a9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556f0d3121b0 con 0x556f0d3b1e80 2024-04-21T08:51:18.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.966+0000 7f4227fff640 1 -- 172.21.15.86:0/218261914 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7f42380f2db0 con 0x7f4240074cd0 2024-04-21T08:51:18.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.966+0000 7f423e7fc640 1 --2- 172.21.15.86:0/218261914 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f42280a3d50 0x7f42280a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.966+0000 7f423effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f0d3b14d0 0x556f0d3b18b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.966+0000 7f423effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f0d3b14d0 0x556f0d3b18b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56606/0 (socket says 172.21.15.86:56606) 2024-04-21T08:51:18.973 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.966+0000 7f423effd640 1 -- 172.21.15.86:0/2172374367 learned_addr learned my addr 172.21.15.86:0/2172374367 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:18.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.966+0000 7f423e7fc640 1 --2- 172.21.15.86:0/2172374367 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556f0d3b1e80 0x556f0d3ba430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.966+0000 7f423f7fe640 1 --2- 172.21.15.86:0/2172374367 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556f0d3baa00 0x556f0d3bce50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.966+0000 7f423effd640 1 -- 172.21.15.86:0/2172374367 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556f0d3b1e80 msgr2=0x556f0d3ba430 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:18.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.966+0000 7f423effd640 1 --2- 172.21.15.86:0/2172374367 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556f0d3b1e80 0x556f0d3ba430 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.966+0000 7f423effd640 1 -- 172.21.15.86:0/2172374367 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556f0d3baa00 msgr2=0x556f0d3bce50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:18.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.966+0000 7f423effd640 1 --2- 172.21.15.86:0/2172374367 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556f0d3baa00 0x556f0d3bce50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.966+0000 7f423effd640 1 -- 172.21.15.86:0/2172374367 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f0d2a51b0 con 0x556f0d3b14d0 2024-04-21T08:51:18.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.966+0000 7f423f7fe640 1 --2- 172.21.15.86:0/2172374367 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556f0d3baa00 0x556f0d3bce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:51:18.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.967+0000 7f423e7fc640 1 --2- 172.21.15.86:0/218261914 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f42280a3d50 0x7f42280a6210 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4234005940 tx=0x7f4234012810 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:18.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.967+0000 7f423effd640 1 --2- 172.21.15.86:0/2172374367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f0d3b14d0 0x556f0d3b18b0 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f4230009ef0 tx=0x7f423000e6c0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=9016761e24f1de03 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:18.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.967+0000 7f42257fa640 1 -- 172.21.15.86:0/2172374367 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4230016070 con 0x556f0d3b14d0 2024-04-21T08:51:18.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.967+0000 7f42257fa640 1 -- 172.21.15.86:0/2172374367 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4230004d20 con 0x556f0d3b14d0 2024-04-21T08:51:18.974 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.967+0000 7f42257fa640 1 -- 172.21.15.86:0/2172374367 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f42300116d0 con 0x556f0d3b14d0 2024-04-21T08:51:18.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.967+0000 7f42483a9740 1 -- 172.21.15.86:0/2172374367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f0d3b14d0 msgr2=0x556f0d3b18b0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.967+0000 7f42483a9740 1 --2- 172.21.15.86:0/2172374367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f0d3b14d0 0x556f0d3b18b0 secure :-1 s=READY pgs=2114 cs=0 l=1 rev1=1 crypto rx=0x7f4230009ef0 tx=0x7f423000e6c0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.967+0000 7f42483a9740 1 -- 172.21.15.86:0/2172374367 shutdown_connections 2024-04-21T08:51:18.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.967+0000 7f42483a9740 1 --2- 172.21.15.86:0/2172374367 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556f0d3baa00 0x556f0d3bce50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.967+0000 7f42483a9740 1 --2- 172.21.15.86:0/2172374367 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556f0d3b1e80 0x556f0d3ba430 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.967+0000 7f42483a9740 1 --2- 172.21.15.86:0/2172374367 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f0d3b14d0 0x556f0d3b18b0 unknown :-1 s=CLOSED pgs=2114 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.967+0000 7f42483a9740 1 -- 172.21.15.86:0/2172374367 >> 172.21.15.86:0/2172374367 conn(0x556f0d3abba0 msgr2=0x556f0d3abf80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:18.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.968+0000 7f42483a9740 1 -- 172.21.15.86:0/2172374367 shutdown_connections 2024-04-21T08:51:18.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.968+0000 7f42483a9740 1 -- 172.21.15.86:0/2172374367 wait complete. 2024-04-21T08:51:18.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.968+0000 7f42483a9740 1 -- start start 2024-04-21T08:51:18.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.968+0000 7f42483a9740 1 Processor -- start 2024-04-21T08:51:18.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.968+0000 7f42483a9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f0d3b14d0 0x556f0d3ba290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.968+0000 7f42483a9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556f0d3b1e80 0x556f0d3b7620 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.975 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.968+0000 7f42483a9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556f0d3baa00 0x556f0d3b7b60 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.968+0000 7f42483a9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x556f0d3bd970 con 0x556f0d3b14d0 2024-04-21T08:51:18.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.968+0000 7f42483a9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x556f0d2cf500 con 0x556f0d3baa00 2024-04-21T08:51:18.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.968+0000 7f42483a9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x556f0d3bdaf0 con 0x556f0d3b1e80 2024-04-21T08:51:18.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.968+0000 7f42483a9740 10 client.0 _ll_register_callbacks cb 0x556f0d29db80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:18.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.968+0000 7f423effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f0d3b14d0 0x556f0d3ba290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.968+0000 7f423effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f0d3b14d0 0x556f0d3ba290 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56618/0 (socket says 172.21.15.86:56618) 2024-04-21T08:51:18.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.968+0000 7f423effd640 1 -- 172.21.15.86:0/415769623 learned_addr learned my addr 172.21.15.86:0/415769623 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:18.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.968+0000 7f423f7fe640 1 --2- 172.21.15.86:0/415769623 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556f0d3baa00 0x556f0d3b7b60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.968+0000 7f423effd640 1 -- 172.21.15.86:0/415769623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556f0d3b1e80 msgr2=0x556f0d3b7620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:18.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.968+0000 7f423effd640 1 --2- 172.21.15.86:0/415769623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556f0d3b1e80 0x556f0d3b7620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.968+0000 7f423effd640 1 -- 172.21.15.86:0/415769623 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556f0d3baa00 msgr2=0x556f0d3b7b60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:18.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.968+0000 7f423effd640 1 --2- 172.21.15.86:0/415769623 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556f0d3baa00 0x556f0d3b7b60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.968+0000 7f423effd640 1 -- 172.21.15.86:0/415769623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f0d0283f0 con 0x556f0d3b14d0 2024-04-21T08:51:18.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.969+0000 7f423effd640 1 --2- 172.21.15.86:0/415769623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f0d3b14d0 0x556f0d3ba290 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f4230006010 tx=0x7f4230005620 comp rx=0 tx=0).ready entity=mon.0 client_cookie=7a72584be85794 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:18.976 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.969+0000 7f4222ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:51:18.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.969+0000 7f4222ff5640 1 -- 172.21.15.86:0/415769623 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4230028060 con 0x556f0d3b14d0 2024-04-21T08:51:18.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.969+0000 7f4222ff5640 1 -- 172.21.15.86:0/415769623 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7f4230004a40 con 0x556f0d3b14d0 2024-04-21T08:51:18.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.969+0000 7f42483a9740 1 -- 172.21.15.86:0/415769623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4230004d20 con 0x556f0d3b14d0 2024-04-21T08:51:18.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.969+0000 7f4222ff5640 1 -- 172.21.15.86:0/415769623 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f423002f050 con 0x556f0d3b14d0 2024-04-21T08:51:18.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.969+0000 7f42483a9740 10 client.20580 fetch_fsmap learned FSMap version 29 2024-04-21T08:51:18.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.969+0000 7f42483a9740 1 -- 172.21.15.86:0/415769623 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556f0d0f27d0 con 0x556f0d3b14d0 2024-04-21T08:51:18.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.969+0000 7f4222ff5640 1 -- 172.21.15.86:0/415769623 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4230004d20 con 0x556f0d3b14d0 2024-04-21T08:51:18.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.969+0000 7f4222ff5640 1 -- 172.21.15.86:0/415769623 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f423000f040 con 0x556f0d3b14d0 2024-04-21T08:51:18.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.969+0000 7f42483a9740 10 client.20580 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:51:18.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.969+0000 7f42483a9740 10 client.20580 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:51:18.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.969+0000 7f42483a9740 20 client.20580 populate_metadata read hostname 'smithi086' 2024-04-21T08:51:18.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.969+0000 7f42483a9740 1 --2- 172.21.15.86:0/415769623 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x556f0d3caa90 0x556f0d3cce80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.969+0000 7f42483a9740 4 client.20580 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:51:18.977 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.969+0000 7f42483a9740 1 -- 172.21.15.86:0/415769623 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x556f0d2b5930 con 0x556f0d3caa90 2024-04-21T08:51:18.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.970+0000 7f423e7fc640 1 --2- 172.21.15.86:0/415769623 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x556f0d3caa90 0x556f0d3cce80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.970+0000 7f4222ff5640 10 client.20580 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:51:18.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.970+0000 7f423e7fc640 1 --2- 172.21.15.86:0/415769623 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x556f0d3caa90 0x556f0d3cce80 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=720a7565d11adeb5 server_cookie=ab01b6d6cf115a1a in_seq=0 out_seq=0 2024-04-21T08:51:18.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.971+0000 7f4222ff5640 1 -- 172.21.15.86:0/415769623 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x556f0d2b5930 con 0x556f0d3caa90 2024-04-21T08:51:18.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.971+0000 7f4222ff5640 10 client.20580 handle_command_reply: tid=0 2024-04-21T08:51:18.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.971+0000 7f42483a9740 1 client.20580 shutdown 2024-04-21T08:51:18.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.971+0000 7f42483a9740 1 -- 172.21.15.86:0/415769623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f0d3b14d0 msgr2=0x556f0d3ba290 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.971+0000 7f42483a9740 1 --2- 172.21.15.86:0/415769623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f0d3b14d0 0x556f0d3ba290 secure :-1 s=READY pgs=2115 cs=0 l=1 rev1=1 crypto rx=0x7f4230006010 tx=0x7f4230005620 comp rx=0 tx=0).stop 2024-04-21T08:51:18.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.971+0000 7f42483a9740 1 -- 172.21.15.86:0/415769623 shutdown_connections 2024-04-21T08:51:18.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.971+0000 7f42483a9740 1 --2- 172.21.15.86:0/415769623 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x556f0d3caa90 0x556f0d3cce80 crc :-1 s=READY pgs=79 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.971+0000 7f42483a9740 1 --2- 172.21.15.86:0/415769623 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556f0d3baa00 0x556f0d3b7b60 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.971+0000 7f42483a9740 1 --2- 172.21.15.86:0/415769623 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556f0d3b1e80 0x556f0d3b7620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.971+0000 7f4222ff5640 0 client.20580 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:51:18.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.971+0000 7f42483a9740 1 --2- 172.21.15.86:0/415769623 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f0d3b14d0 0x556f0d3ba290 unknown :-1 s=CLOSED pgs=2115 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.978 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.971+0000 7f42483a9740 1 -- 172.21.15.86:0/415769623 >> 172.21.15.86:0/415769623 conn(0x556f0d3abba0 msgr2=0x556f0d3ad8f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:18.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.971+0000 7f42483a9740 1 -- 172.21.15.86:0/415769623 shutdown_connections 2024-04-21T08:51:18.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.971+0000 7f42483a9740 1 -- 172.21.15.86:0/415769623 wait complete. 2024-04-21T08:51:18.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.971+0000 7f42483a9740 20 client.20580 trim_cache size 0 max 16384 2024-04-21T08:51:18.988 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.984+0000 7f42483a9740 1 Processor -- start 2024-04-21T08:51:18.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.984+0000 7f42483a9740 1 -- start start 2024-04-21T08:51:18.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.984+0000 7f42483a9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f0d3b14d0 0x556f0d3b9d50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.984+0000 7f42483a9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556f0d3b1e80 0x556f0d3ba290 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.984+0000 7f42483a9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556f0d3baa00 0x556f0d3d24e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.984+0000 7f42483a9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f42401b1620 con 0x556f0d3b14d0 2024-04-21T08:51:18.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.984+0000 7f42483a9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f42401b3160 con 0x556f0d3baa00 2024-04-21T08:51:18.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.984+0000 7f42483a9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f42401b1cf0 con 0x556f0d3b1e80 2024-04-21T08:51:18.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.984+0000 7f423effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f0d3b14d0 0x556f0d3b9d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.984+0000 7f423effd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f0d3b14d0 0x556f0d3b9d50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56628/0 (socket says 172.21.15.86:56628) 2024-04-21T08:51:18.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.984+0000 7f423effd640 1 -- 172.21.15.86:0/837023147 learned_addr learned my addr 172.21.15.86:0/837023147 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:18.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.984+0000 7f423f7fe640 1 --2- 172.21.15.86:0/837023147 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556f0d3baa00 0x556f0d3d24e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.985+0000 7f423effd640 1 -- 172.21.15.86:0/837023147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556f0d3b1e80 msgr2=0x556f0d3ba290 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:18.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.985+0000 7f423effd640 1 --2- 172.21.15.86:0/837023147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556f0d3b1e80 0x556f0d3ba290 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.989 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.985+0000 7f423effd640 1 -- 172.21.15.86:0/837023147 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556f0d3baa00 msgr2=0x556f0d3d24e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:18.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.985+0000 7f423effd640 1 --2- 172.21.15.86:0/837023147 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556f0d3baa00 0x556f0d3d24e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.985+0000 7f423effd640 1 -- 172.21.15.86:0/837023147 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x556f0d3ac3e0 con 0x556f0d3b14d0 2024-04-21T08:51:18.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.985+0000 7f423f7fe640 1 --2- 172.21.15.86:0/837023147 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556f0d3baa00 0x556f0d3d24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:18.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.985+0000 7f423effd640 1 --2- 172.21.15.86:0/837023147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f0d3b14d0 0x556f0d3b9d50 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f423000e700 tx=0x7f4230005c90 comp rx=0 tx=0).ready entity=mon.0 client_cookie=c13681afbe21a1e8 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:18.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.985+0000 7f4224ff9640 1 -- 172.21.15.86:0/837023147 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4230028070 con 0x556f0d3b14d0 2024-04-21T08:51:18.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.985+0000 7f4224ff9640 1 -- 172.21.15.86:0/837023147 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4230009520 con 0x556f0d3b14d0 2024-04-21T08:51:18.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.985+0000 7f4224ff9640 1 -- 172.21.15.86:0/837023147 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4230016040 con 0x556f0d3b14d0 2024-04-21T08:51:18.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.985+0000 7f42483a9740 1 -- 172.21.15.86:0/837023147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f0d3b14d0 msgr2=0x556f0d3b9d50 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.985+0000 7f42483a9740 1 --2- 172.21.15.86:0/837023147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f0d3b14d0 0x556f0d3b9d50 secure :-1 s=READY pgs=2116 cs=0 l=1 rev1=1 crypto rx=0x7f423000e700 tx=0x7f4230005c90 comp rx=0 tx=0).stop 2024-04-21T08:51:18.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.985+0000 7f42483a9740 1 -- 172.21.15.86:0/837023147 shutdown_connections 2024-04-21T08:51:18.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.985+0000 7f42483a9740 1 --2- 172.21.15.86:0/837023147 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556f0d3baa00 0x556f0d3d24e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.985+0000 7f42483a9740 1 --2- 172.21.15.86:0/837023147 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556f0d3b1e80 0x556f0d3ba290 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.990 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.985+0000 7f42483a9740 1 --2- 172.21.15.86:0/837023147 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f0d3b14d0 0x556f0d3b9d50 unknown :-1 s=CLOSED pgs=2116 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.985+0000 7f42483a9740 1 -- 172.21.15.86:0/837023147 >> 172.21.15.86:0/837023147 conn(0x556f0d3abba0 msgr2=0x556f0d3b2c60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:18.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.986+0000 7f42483a9740 1 -- 172.21.15.86:0/837023147 shutdown_connections 2024-04-21T08:51:18.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.986+0000 7f42483a9740 1 -- 172.21.15.86:0/837023147 wait complete. 2024-04-21T08:51:18.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.986+0000 7f42483a9740 1 -- start start 2024-04-21T08:51:18.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.986+0000 7f42483a9740 1 Processor -- start 2024-04-21T08:51:18.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.986+0000 7f42483a9740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556f0d3b14d0 0x556f0d3bffc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.986+0000 7f42483a9740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f0d3b1e80 0x556f0d3c0500 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.986+0000 7f42483a9740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556f0d3baa00 0x556f0d42ed50 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.986+0000 7f42483a9740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f42401b2a90 con 0x556f0d3b1e80 2024-04-21T08:51:18.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.986+0000 7f42483a9740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f42401b23c0 con 0x556f0d3baa00 2024-04-21T08:51:18.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.986+0000 7f42483a9740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f42401af620 con 0x556f0d3b14d0 2024-04-21T08:51:18.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.986+0000 7f42483a9740 10 client.0 _ll_register_callbacks cb 0x7f42300052b0 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:18.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.986+0000 7f423e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f0d3b1e80 0x556f0d3c0500 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.986+0000 7f423e7fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f0d3b1e80 0x556f0d3c0500 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56632/0 (socket says 172.21.15.86:56632) 2024-04-21T08:51:18.991 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.986+0000 7f423e7fc640 1 -- 172.21.15.86:0/2496288726 learned_addr learned my addr 172.21.15.86:0/2496288726 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:18.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.986+0000 7f423f7fe640 1 --2- 172.21.15.86:0/2496288726 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556f0d3baa00 0x556f0d42ed50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.986+0000 7f423e7fc640 1 -- 172.21.15.86:0/2496288726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556f0d3b14d0 msgr2=0x556f0d3bffc0 unknown :-1 s=STATE_CONNECTING_RE l=0).mark_down 2024-04-21T08:51:18.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.986+0000 7f423e7fc640 1 --2- 172.21.15.86:0/2496288726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556f0d3b14d0 0x556f0d3bffc0 unknown :-1 s=START_CONNECT pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.986+0000 7f423e7fc640 1 -- 172.21.15.86:0/2496288726 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556f0d3baa00 msgr2=0x556f0d42ed50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:18.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.986+0000 7f423e7fc640 1 --2- 172.21.15.86:0/2496288726 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556f0d3baa00 0x556f0d42ed50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.986+0000 7f423e7fc640 1 -- 172.21.15.86:0/2496288726 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x556f0d42f310 con 0x556f0d3b1e80 2024-04-21T08:51:18.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.987+0000 7f423e7fc640 1 --2- 172.21.15.86:0/2496288726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f0d3b1e80 0x556f0d3c0500 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f4240075ce0 tx=0x7f4234004730 comp rx=0 tx=0).ready entity=mon.0 client_cookie=3366c0ad704d32ad server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:18.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.987+0000 7f4222ff5640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:51:18.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.987+0000 7f4222ff5640 1 -- 172.21.15.86:0/2496288726 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f423402e060 con 0x556f0d3b1e80 2024-04-21T08:51:18.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.987+0000 7f4222ff5640 1 -- 172.21.15.86:0/2496288726 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7f4234029680 con 0x556f0d3b1e80 2024-04-21T08:51:18.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.987+0000 7f42483a9740 1 -- 172.21.15.86:0/2496288726 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7f4230009520 con 0x556f0d3b1e80 2024-04-21T08:51:18.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.987+0000 7f4222ff5640 1 -- 172.21.15.86:0/2496288726 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7f4234035050 con 0x556f0d3b1e80 2024-04-21T08:51:18.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.987+0000 7f42483a9740 10 client.20586 fetch_fsmap learned FSMap version 29 2024-04-21T08:51:18.992 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.987+0000 7f42483a9740 1 -- 172.21.15.86:0/2496288726 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x556f0d430680 con 0x556f0d3b1e80 2024-04-21T08:51:18.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.987+0000 7f4222ff5640 1 -- 172.21.15.86:0/2496288726 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f4230009520 con 0x556f0d3b1e80 2024-04-21T08:51:18.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.987+0000 7f4222ff5640 1 -- 172.21.15.86:0/2496288726 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7f423401f860 con 0x556f0d3b1e80 2024-04-21T08:51:18.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.987+0000 7f42483a9740 10 client.20586 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:51:18.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.987+0000 7f42483a9740 10 client.20586 resolve_mds: resolved 1:2 to role '1:2' aka daemon mds.h 2024-04-21T08:51:18.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.987+0000 7f42483a9740 20 client.20586 populate_metadata read hostname 'smithi086' 2024-04-21T08:51:18.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.987+0000 7f42483a9740 1 --2- 172.21.15.86:0/2496288726 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x556f0d430de0 0x556f0d433200 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:18.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.987+0000 7f42483a9740 4 client.20586 mds_command: new command op to 24455 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:51:18.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.987+0000 7f42483a9740 1 -- 172.21.15.86:0/2496288726 --> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] -- command(tid 0: {"prefix": "damage ls"}) -- 0x556f0d323c10 con 0x556f0d430de0 2024-04-21T08:51:18.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.988+0000 7f423effd640 1 --2- 172.21.15.86:0/2496288726 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x556f0d430de0 0x556f0d433200 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:18.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.988+0000 7f4222ff5640 10 client.20586 ms_handle_connect on v2:172.21.15.88:6836/1950568471 2024-04-21T08:51:18.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.988+0000 7f423effd640 1 --2- 172.21.15.86:0/2496288726 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x556f0d430de0 0x556f0d433200 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.2 client_cookie=205eda1e65090b2c server_cookie=b62f7cd0769b8dd6 in_seq=0 out_seq=0 2024-04-21T08:51:18.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.988+0000 7f4222ff5640 1 -- 172.21.15.86:0/2496288726 <== mds.2 v2:172.21.15.88:6836/1950568471 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x556f0d323c10 con 0x556f0d430de0 2024-04-21T08:51:18.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.988+0000 7f4222ff5640 10 client.20586 handle_command_reply: tid=0 2024-04-21T08:51:18.993 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.988+0000 7f42483a9740 1 client.20586 shutdown 2024-04-21T08:51:18.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.988+0000 7f42483a9740 1 -- 172.21.15.86:0/2496288726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f0d3b1e80 msgr2=0x556f0d3c0500 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.988+0000 7f42483a9740 1 --2- 172.21.15.86:0/2496288726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f0d3b1e80 0x556f0d3c0500 secure :-1 s=READY pgs=2117 cs=0 l=1 rev1=1 crypto rx=0x7f4240075ce0 tx=0x7f4234004730 comp rx=0 tx=0).stop 2024-04-21T08:51:18.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.988+0000 7f42483a9740 1 -- 172.21.15.86:0/2496288726 shutdown_connections 2024-04-21T08:51:18.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.988+0000 7f42483a9740 1 --2- 172.21.15.86:0/2496288726 >> [v2:172.21.15.88:6836/1950568471,v1:172.21.15.88:6837/1950568471] conn(0x556f0d430de0 0x556f0d433200 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.988+0000 7f42483a9740 1 --2- 172.21.15.86:0/2496288726 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x556f0d3baa00 0x556f0d42ed50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.988+0000 7f42483a9740 1 --2- 172.21.15.86:0/2496288726 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x556f0d3b1e80 0x556f0d3c0500 unknown :-1 s=CLOSED pgs=2117 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.988+0000 7f42483a9740 1 --2- 172.21.15.86:0/2496288726 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x556f0d3b14d0 0x556f0d3bffc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.988+0000 7f4222ff5640 0 client.20586 ms_handle_reset on v2:172.21.15.88:6836/1950568471 2024-04-21T08:51:18.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.988+0000 7f42483a9740 1 -- 172.21.15.86:0/2496288726 >> 172.21.15.86:0/2496288726 conn(0x556f0d3abba0 msgr2=0x556f0d3acd90 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:18.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.988+0000 7f42483a9740 1 -- 172.21.15.86:0/2496288726 shutdown_connections 2024-04-21T08:51:18.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.988+0000 7f42483a9740 1 -- 172.21.15.86:0/2496288726 wait complete. 2024-04-21T08:51:18.994 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.988+0000 7f42483a9740 20 client.20586 trim_cache size 0 max 16384 2024-04-21T08:51:18.995 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:51:18.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.989+0000 7f42457b6640 1 -- 172.21.15.86:0/218261914 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f42280a3d50 msgr2=0x7f42280a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.995 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.989+0000 7f42457b6640 1 --2- 172.21.15.86:0/218261914 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f42280a3d50 0x7f42280a6210 secure :-1 s=READY pgs=356 cs=0 l=1 rev1=1 crypto rx=0x7f4234005940 tx=0x7f4234012810 comp rx=0 tx=0).stop 2024-04-21T08:51:18.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.989+0000 7f42457b6640 1 -- 172.21.15.86:0/218261914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4240074cd0 msgr2=0x7f42401a6930 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:18.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.989+0000 7f42457b6640 1 --2- 172.21.15.86:0/218261914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4240074cd0 0x7f42401a6930 secure :-1 s=READY pgs=2113 cs=0 l=1 rev1=1 crypto rx=0x7f4238009230 tx=0x7f42380045e0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.989+0000 7f42457b6640 1 -- 172.21.15.86:0/218261914 shutdown_connections 2024-04-21T08:51:18.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.989+0000 7f42457b6640 1 --2- 172.21.15.86:0/218261914 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f42280a3d50 0x7f42280a6210 unknown :-1 s=CLOSED pgs=356 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.989+0000 7f42457b6640 1 --2- 172.21.15.86:0/218261914 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4240074cd0 0x7f42401a6930 unknown :-1 s=CLOSED pgs=2113 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.989+0000 7f42457b6640 1 --2- 172.21.15.86:0/218261914 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f4240107270 0x7f4240074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.989+0000 7f42457b6640 1 --2- 172.21.15.86:0/218261914 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f42401021a0 0x7f4240074250 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:18.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.989+0000 7f42457b6640 1 -- 172.21.15.86:0/218261914 >> 172.21.15.86:0/218261914 conn(0x7f4240079350 msgr2=0x7f4240100140 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:18.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.989+0000 7f42457b6640 1 -- 172.21.15.86:0/218261914 shutdown_connections 2024-04-21T08:51:18.996 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:18.989+0000 7f42457b6640 1 -- 172.21.15.86:0/218261914 wait complete. 2024-04-21T08:51:18.999 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:1 damage ls 2024-04-21T08:51:19.101 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.100+0000 7fa0ab83f640 1 -- 172.21.15.86:0/2479313876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0a4102180 msgr2=0x7fa0a4105ca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:19.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.100+0000 7fa0ab83f640 1 --2- 172.21.15.86:0/2479313876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0a4102180 0x7fa0a4105ca0 secure :-1 s=READY pgs=2118 cs=0 l=1 rev1=1 crypto rx=0x7fa094009900 tx=0x7fa09402bc30 comp rx=0 tx=0).stop 2024-04-21T08:51:19.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.100+0000 7fa0ab83f640 1 -- 172.21.15.86:0/2479313876 shutdown_connections 2024-04-21T08:51:19.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.100+0000 7fa0ab83f640 1 --2- 172.21.15.86:0/2479313876 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0a4106380 0x7fa0a410df50 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.100+0000 7fa0ab83f640 1 --2- 172.21.15.86:0/2479313876 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0a4102180 0x7fa0a4105ca0 unknown :-1 s=CLOSED pgs=2118 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.100+0000 7fa0ab83f640 1 --2- 172.21.15.86:0/2479313876 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0a41017d0 0x7fa0a4101bb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.100+0000 7fa0ab83f640 1 -- 172.21.15.86:0/2479313876 >> 172.21.15.86:0/2479313876 conn(0x7fa0a4077410 msgr2=0x7fa0a40fe760 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:19.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.100+0000 7fa0ab83f640 1 -- 172.21.15.86:0/2479313876 shutdown_connections 2024-04-21T08:51:19.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.100+0000 7fa0ab83f640 1 -- 172.21.15.86:0/2479313876 wait complete. 2024-04-21T08:51:19.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.101+0000 7fa0ab83f640 1 Processor -- start 2024-04-21T08:51:19.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.101+0000 7fa0ab83f640 1 -- start start 2024-04-21T08:51:19.102 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.101+0000 7fa0ab83f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0a41017d0 0x7fa0a419e620 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.101+0000 7fa0ab83f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0a4106380 0x7fa0a419eb60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.101+0000 7fa0ab83f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0a4198790 0x7fa0a4198c40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.101+0000 7fa0ab83f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a41112c0 con 0x7fa0a4106380 2024-04-21T08:51:19.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.101+0000 7fa0ab83f640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a4111140 con 0x7fa0a41017d0 2024-04-21T08:51:19.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.101+0000 7fa0ab83f640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a4111440 con 0x7fa0a4198790 2024-04-21T08:51:19.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.101+0000 7fa0a8dda640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0a4106380 0x7fa0a419eb60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.103 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.101+0000 7fa0a8dda640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0a4106380 0x7fa0a419eb60 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56654/0 (socket says 172.21.15.86:56654) 2024-04-21T08:51:19.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.101+0000 7fa0a8dda640 1 -- 172.21.15.86:0/3787123555 learned_addr learned my addr 172.21.15.86:0/3787123555 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:19.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.101+0000 7fa0a95db640 1 --2- 172.21.15.86:0/3787123555 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0a41017d0 0x7fa0a419e620 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.101+0000 7fa0a9ddc640 1 --2- 172.21.15.86:0/3787123555 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0a4198790 0x7fa0a4198c40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.101+0000 7fa0a8dda640 1 -- 172.21.15.86:0/3787123555 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0a4198790 msgr2=0x7fa0a4198c40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:19.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.101+0000 7fa0a8dda640 1 --2- 172.21.15.86:0/3787123555 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0a4198790 0x7fa0a4198c40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.105 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.101+0000 7fa0a8dda640 1 -- 172.21.15.86:0/3787123555 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0a41017d0 msgr2=0x7fa0a419e620 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:19.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.101+0000 7fa0a8dda640 1 --2- 172.21.15.86:0/3787123555 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0a41017d0 0x7fa0a419e620 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.101+0000 7fa0a8dda640 1 -- 172.21.15.86:0/3787123555 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa0a4199320 con 0x7fa0a4106380 2024-04-21T08:51:19.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.101+0000 7fa0a8dda640 1 --2- 172.21.15.86:0/3787123555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0a4106380 0x7fa0a419eb60 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7fa094036040 tx=0x7fa0940045e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:19.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.102+0000 7fa09a7fc640 1 -- 172.21.15.86:0/3787123555 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa094043020 con 0x7fa0a4106380 2024-04-21T08:51:19.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.102+0000 7fa09a7fc640 1 -- 172.21.15.86:0/3787123555 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa09402cb70 con 0x7fa0a4106380 2024-04-21T08:51:19.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.102+0000 7fa0ab83f640 1 -- 172.21.15.86:0/3787123555 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa0a41995b0 con 0x7fa0a4106380 2024-04-21T08:51:19.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.102+0000 7fa09a7fc640 1 -- 172.21.15.86:0/3787123555 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa094003bf0 con 0x7fa0a4106380 2024-04-21T08:51:19.106 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.102+0000 7fa0ab83f640 1 -- 172.21.15.86:0/3787123555 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa0a41a5790 con 0x7fa0a4106380 2024-04-21T08:51:19.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.104+0000 7fa09a7fc640 1 -- 172.21.15.86:0/3787123555 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa094046020 con 0x7fa0a4106380 2024-04-21T08:51:19.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.104+0000 7fa0ae42b740 1 Processor -- start 2024-04-21T08:51:19.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.104+0000 7fa0ae42b740 1 -- start start 2024-04-21T08:51:19.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.104+0000 7fa0ae42b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff2ef98e0 0x55cff2ef9cc0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.113 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.104+0000 7fa0ae42b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff2efa290 0x55cff2f02840 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.104+0000 7fa0ae42b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff2f02e10 0x55cff2f05260 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.104+0000 7fa0ae42b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cff2e5a1f0 con 0x55cff2ef98e0 2024-04-21T08:51:19.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.104+0000 7fa0ae42b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55cff2df74b0 con 0x55cff2efa290 2024-04-21T08:51:19.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.104+0000 7fa0ae42b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55cff2e561b0 con 0x55cff2f02e10 2024-04-21T08:51:19.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.104+0000 7fa0a9ddc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff2ef98e0 0x55cff2ef9cc0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.104+0000 7fa0a9ddc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff2ef98e0 0x55cff2ef9cc0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56662/0 (socket says 172.21.15.86:56662) 2024-04-21T08:51:19.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.104+0000 7fa0a9ddc640 1 -- 172.21.15.86:0/3199140742 learned_addr learned my addr 172.21.15.86:0/3199140742 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:19.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.105+0000 7fa0a95db640 1 --2- 172.21.15.86:0/3199140742 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff2efa290 0x55cff2f02840 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.105+0000 7fa09a7fc640 1 --2- 172.21.15.86:0/3787123555 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa0900a3d20 0x7fa0900a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.105+0000 7fa0a8dda640 1 --2- 172.21.15.86:0/3199140742 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff2f02e10 0x55cff2f05260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.105+0000 7fa0a9ddc640 1 -- 172.21.15.86:0/3199140742 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff2f02e10 msgr2=0x55cff2f05260 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:19.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.105+0000 7fa0a9ddc640 1 --2- 172.21.15.86:0/3199140742 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff2f02e10 0x55cff2f05260 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.114 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.105+0000 7fa0a9ddc640 1 -- 172.21.15.86:0/3199140742 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff2efa290 msgr2=0x55cff2f02840 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:19.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.105+0000 7fa0a9ddc640 1 --2- 172.21.15.86:0/3199140742 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff2efa290 0x55cff2f02840 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.105+0000 7fa0a9ddc640 1 -- 172.21.15.86:0/3199140742 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cff2de91b0 con 0x55cff2ef98e0 2024-04-21T08:51:19.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.105+0000 7fa0a8dda640 1 --2- 172.21.15.86:0/3199140742 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff2f02e10 0x55cff2f05260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:51:19.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.105+0000 7fa0a9ddc640 1 --2- 172.21.15.86:0/3787123555 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa0900a3d20 0x7fa0900a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.105+0000 7fa09a7fc640 1 -- 172.21.15.86:0/3787123555 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7fa0940f2c00 con 0x7fa0a4106380 2024-04-21T08:51:19.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.105+0000 7fa0a9ddc640 1 --2- 172.21.15.86:0/3199140742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff2ef98e0 0x55cff2ef9cc0 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7fa09c010520 tx=0x7fa09c0109e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=525747d776ff65da server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:19.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.105+0000 7fa08f7fe640 1 -- 172.21.15.86:0/3199140742 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09c019070 con 0x55cff2ef98e0 2024-04-21T08:51:19.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.105+0000 7fa08f7fe640 1 -- 172.21.15.86:0/3199140742 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa09c003e10 con 0x55cff2ef98e0 2024-04-21T08:51:19.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.105+0000 7fa08f7fe640 1 -- 172.21.15.86:0/3199140742 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09c0116a0 con 0x55cff2ef98e0 2024-04-21T08:51:19.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.105+0000 7fa0a9ddc640 1 --2- 172.21.15.86:0/3787123555 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa0900a3d20 0x7fa0900a61e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa09c021000 tx=0x7fa09c01e000 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:19.115 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.106+0000 7fa0ae42b740 1 -- 172.21.15.86:0/3199140742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff2ef98e0 msgr2=0x55cff2ef9cc0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:19.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.106+0000 7fa0ae42b740 1 --2- 172.21.15.86:0/3199140742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff2ef98e0 0x55cff2ef9cc0 secure :-1 s=READY pgs=2120 cs=0 l=1 rev1=1 crypto rx=0x7fa09c010520 tx=0x7fa09c0109e0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.106+0000 7fa0ae42b740 1 -- 172.21.15.86:0/3199140742 shutdown_connections 2024-04-21T08:51:19.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.106+0000 7fa0ae42b740 1 --2- 172.21.15.86:0/3199140742 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff2f02e10 0x55cff2f05260 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.106+0000 7fa0ae42b740 1 --2- 172.21.15.86:0/3199140742 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff2efa290 0x55cff2f02840 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.106+0000 7fa0ae42b740 1 --2- 172.21.15.86:0/3199140742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff2ef98e0 0x55cff2ef9cc0 unknown :-1 s=CLOSED pgs=2120 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.106+0000 7fa0ae42b740 1 -- 172.21.15.86:0/3199140742 >> 172.21.15.86:0/3199140742 conn(0x55cff2ef3fb0 msgr2=0x55cff2ef4390 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:19.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.106+0000 7fa0ae42b740 1 -- 172.21.15.86:0/3199140742 shutdown_connections 2024-04-21T08:51:19.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.106+0000 7fa0ae42b740 1 -- 172.21.15.86:0/3199140742 wait complete. 2024-04-21T08:51:19.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.106+0000 7fa0ae42b740 1 -- start start 2024-04-21T08:51:19.116 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.106+0000 7fa0ae42b740 1 Processor -- start 2024-04-21T08:51:19.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.107+0000 7fa0ae42b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff2ef98e0 0x55cff2f02720 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.107+0000 7fa0ae42b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff2efa290 0x55cff2effab0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.107+0000 7fa0ae42b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff2f02e10 0x55cff2f00050 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.107+0000 7fa0ae42b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55cff2f05d80 con 0x55cff2ef98e0 2024-04-21T08:51:19.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.107+0000 7fa0ae42b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55cff2e13500 con 0x55cff2f02e10 2024-04-21T08:51:19.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.107+0000 7fa0ae42b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55cff2f05f00 con 0x55cff2efa290 2024-04-21T08:51:19.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.107+0000 7fa0ae42b740 10 client.0 _ll_register_callbacks cb 0x55cff2de1b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:19.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.107+0000 7fa0a95db640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff2ef98e0 0x55cff2f02720 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.107+0000 7fa0a95db640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff2ef98e0 0x55cff2f02720 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56668/0 (socket says 172.21.15.86:56668) 2024-04-21T08:51:19.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.107+0000 7fa0a95db640 1 -- 172.21.15.86:0/3949977710 learned_addr learned my addr 172.21.15.86:0/3949977710 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:19.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.107+0000 7fa0a8dda640 1 --2- 172.21.15.86:0/3949977710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff2efa290 0x55cff2effab0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.107+0000 7fa0a9ddc640 1 --2- 172.21.15.86:0/3949977710 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff2f02e10 0x55cff2f00050 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.107+0000 7fa0a95db640 1 -- 172.21.15.86:0/3949977710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff2efa290 msgr2=0x55cff2effab0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:19.117 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.107+0000 7fa0a95db640 1 --2- 172.21.15.86:0/3949977710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff2efa290 0x55cff2effab0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.107+0000 7fa0a95db640 1 -- 172.21.15.86:0/3949977710 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff2f02e10 msgr2=0x55cff2f00050 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:19.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.107+0000 7fa0a95db640 1 --2- 172.21.15.86:0/3949977710 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff2f02e10 0x55cff2f00050 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.107+0000 7fa0a95db640 1 -- 172.21.15.86:0/3949977710 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cff2b6c3f0 con 0x55cff2ef98e0 2024-04-21T08:51:19.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.107+0000 7fa0a8dda640 1 --2- 172.21.15.86:0/3949977710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff2efa290 0x55cff2effab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:19.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.107+0000 7fa0a9ddc640 1 --2- 172.21.15.86:0/3949977710 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff2f02e10 0x55cff2f00050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:51:19.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.107+0000 7fa0a95db640 1 --2- 172.21.15.86:0/3949977710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff2ef98e0 0x55cff2f02720 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0010820 tx=0x7fa0a0010ce0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=2ba60060902e2d77 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:19.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.107+0000 7fa08cff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:51:19.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.107+0000 7fa08cff9640 1 -- 172.21.15.86:0/3949977710 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa0a000fb10 con 0x55cff2ef98e0 2024-04-21T08:51:19.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.107+0000 7fa08cff9640 1 -- 172.21.15.86:0/3949977710 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7fa0a000fcb0 con 0x55cff2ef98e0 2024-04-21T08:51:19.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.107+0000 7fa0ae42b740 1 -- 172.21.15.86:0/3949977710 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa09c003e10 con 0x55cff2ef98e0 2024-04-21T08:51:19.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.108+0000 7fa08cff9640 1 -- 172.21.15.86:0/3949977710 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa0a0024050 con 0x55cff2ef98e0 2024-04-21T08:51:19.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.108+0000 7fa0ae42b740 10 client.20598 fetch_fsmap learned FSMap version 29 2024-04-21T08:51:19.118 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.108+0000 7fa0ae42b740 1 -- 172.21.15.86:0/3949977710 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cff2c367d0 con 0x55cff2ef98e0 2024-04-21T08:51:19.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.108+0000 7fa08cff9640 1 -- 172.21.15.86:0/3949977710 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa09c003e10 con 0x55cff2ef98e0 2024-04-21T08:51:19.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.108+0000 7fa08cff9640 1 -- 172.21.15.86:0/3949977710 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa0a000e570 con 0x55cff2ef98e0 2024-04-21T08:51:19.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.108+0000 7fa0ae42b740 10 client.20598 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:51:19.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.108+0000 7fa0ae42b740 10 client.20598 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:51:19.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.108+0000 7fa0ae42b740 20 client.20598 populate_metadata read hostname 'smithi086' 2024-04-21T08:51:19.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.108+0000 7fa0ae42b740 1 --2- 172.21.15.86:0/3949977710 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55cff2f12fe0 0x55cff2f15430 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.108+0000 7fa0ae42b740 4 client.20598 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:51:19.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.108+0000 7fa0ae42b740 1 -- 172.21.15.86:0/3949977710 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55cff2df9930 con 0x55cff2f12fe0 2024-04-21T08:51:19.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.108+0000 7fa0a8dda640 1 --2- 172.21.15.86:0/3949977710 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55cff2f12fe0 0x55cff2f15430 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.109+0000 7fa0a8dda640 1 --2- 172.21.15.86:0/3949977710 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55cff2f12fe0 0x55cff2f15430 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=d2d6cde1dd759216 server_cookie=a3b7c9f4df7b334c in_seq=0 out_seq=0 2024-04-21T08:51:19.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.109+0000 7fa08cff9640 10 client.20598 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:51:19.119 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.109+0000 7fa08cff9640 1 -- 172.21.15.86:0/3949977710 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55cff2df9930 con 0x55cff2f12fe0 2024-04-21T08:51:19.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.109+0000 7fa08cff9640 10 client.20598 handle_command_reply: tid=0 2024-04-21T08:51:19.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.110+0000 7fa0ae42b740 1 client.20598 shutdown 2024-04-21T08:51:19.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.110+0000 7fa0ae42b740 1 -- 172.21.15.86:0/3949977710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff2ef98e0 msgr2=0x55cff2f02720 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:19.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.110+0000 7fa0ae42b740 1 --2- 172.21.15.86:0/3949977710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff2ef98e0 0x55cff2f02720 secure :-1 s=READY pgs=2121 cs=0 l=1 rev1=1 crypto rx=0x7fa0a0010820 tx=0x7fa0a0010ce0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.110+0000 7fa0ae42b740 1 -- 172.21.15.86:0/3949977710 shutdown_connections 2024-04-21T08:51:19.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.110+0000 7fa0ae42b740 1 --2- 172.21.15.86:0/3949977710 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff2f02e10 0x55cff2f00050 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.110+0000 7fa0ae42b740 1 --2- 172.21.15.86:0/3949977710 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff2efa290 0x55cff2effab0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.110+0000 7fa0ae42b740 1 --2- 172.21.15.86:0/3949977710 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55cff2f12fe0 0x55cff2f15430 crc :-1 s=READY pgs=80 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.110+0000 7fa0ae42b740 1 --2- 172.21.15.86:0/3949977710 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff2ef98e0 0x55cff2f02720 unknown :-1 s=CLOSED pgs=2121 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.110+0000 7fa08cff9640 0 client.20598 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:51:19.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.110+0000 7fa0ae42b740 1 -- 172.21.15.86:0/3949977710 >> 172.21.15.86:0/3949977710 conn(0x55cff2ef3fb0 msgr2=0x55cff2ef5d00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:19.120 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.110+0000 7fa0ae42b740 1 -- 172.21.15.86:0/3949977710 shutdown_connections 2024-04-21T08:51:19.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.110+0000 7fa0ae42b740 1 -- 172.21.15.86:0/3949977710 wait complete. 2024-04-21T08:51:19.121 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.110+0000 7fa0ae42b740 20 client.20598 trim_cache size 0 max 16384 2024-04-21T08:51:19.121 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:18 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1648506877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:51:19.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.123+0000 7fa0ae42b740 1 Processor -- start 2024-04-21T08:51:19.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.123+0000 7fa0ae42b740 1 -- start start 2024-04-21T08:51:19.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.123+0000 7fa0ae42b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff2ef98e0 0x55cff2ef5c70 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.123+0000 7fa0ae42b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff2efa290 0x55cff2eeea80 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.123+0000 7fa0ae42b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff2f02e10 0x55cff2f021e0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.123+0000 7fa0ae42b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a41ad220 con 0x55cff2f02e10 2024-04-21T08:51:19.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.123+0000 7fa0ae42b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a41aed60 con 0x55cff2ef98e0 2024-04-21T08:51:19.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.123+0000 7fa0ae42b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a41ad8f0 con 0x55cff2efa290 2024-04-21T08:51:19.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.124+0000 7fa0a9ddc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff2f02e10 0x55cff2f021e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.124+0000 7fa0a9ddc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff2f02e10 0x55cff2f021e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56672/0 (socket says 172.21.15.86:56672) 2024-04-21T08:51:19.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.124+0000 7fa0a9ddc640 1 -- 172.21.15.86:0/2492123970 learned_addr learned my addr 172.21.15.86:0/2492123970 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:19.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.124+0000 7fa0a8dda640 1 --2- 172.21.15.86:0/2492123970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff2efa290 0x55cff2eeea80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.124+0000 7fa0a9ddc640 1 -- 172.21.15.86:0/2492123970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff2efa290 msgr2=0x55cff2eeea80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:19.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.124+0000 7fa0a95db640 1 --2- 172.21.15.86:0/2492123970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff2ef98e0 0x55cff2ef5c70 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.124+0000 7fa0a9ddc640 1 --2- 172.21.15.86:0/2492123970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff2efa290 0x55cff2eeea80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.124+0000 7fa0a9ddc640 1 -- 172.21.15.86:0/2492123970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff2ef98e0 msgr2=0x55cff2ef5c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:19.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.124+0000 7fa0a9ddc640 1 --2- 172.21.15.86:0/2492123970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff2ef98e0 0x55cff2ef5c70 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.131 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.124+0000 7fa0a9ddc640 1 -- 172.21.15.86:0/2492123970 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55cff2f003c0 con 0x55cff2f02e10 2024-04-21T08:51:19.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.124+0000 7fa0a9ddc640 1 --2- 172.21.15.86:0/2492123970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff2f02e10 0x55cff2f021e0 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7fa09c0104f0 tx=0x7fa09c005b70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=d2956a20d482f909 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:19.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.124+0000 7fa08effd640 1 -- 172.21.15.86:0/2492123970 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09c019070 con 0x55cff2f02e10 2024-04-21T08:51:19.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.124+0000 7fa08effd640 1 -- 172.21.15.86:0/2492123970 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa09c003e10 con 0x55cff2f02e10 2024-04-21T08:51:19.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.124+0000 7fa08effd640 1 -- 172.21.15.86:0/2492123970 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09c013520 con 0x55cff2f02e10 2024-04-21T08:51:19.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.125+0000 7fa0ae42b740 1 -- 172.21.15.86:0/2492123970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff2f02e10 msgr2=0x55cff2f021e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:19.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.125+0000 7fa0ae42b740 1 --2- 172.21.15.86:0/2492123970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff2f02e10 0x55cff2f021e0 secure :-1 s=READY pgs=2122 cs=0 l=1 rev1=1 crypto rx=0x7fa09c0104f0 tx=0x7fa09c005b70 comp rx=0 tx=0).stop 2024-04-21T08:51:19.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.125+0000 7fa0ae42b740 1 -- 172.21.15.86:0/2492123970 shutdown_connections 2024-04-21T08:51:19.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.125+0000 7fa0ae42b740 1 --2- 172.21.15.86:0/2492123970 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff2f02e10 0x55cff2f021e0 unknown :-1 s=CLOSED pgs=2122 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.125+0000 7fa0ae42b740 1 --2- 172.21.15.86:0/2492123970 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff2efa290 0x55cff2eeea80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.125+0000 7fa0ae42b740 1 --2- 172.21.15.86:0/2492123970 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff2ef98e0 0x55cff2ef5c70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.125+0000 7fa0ae42b740 1 -- 172.21.15.86:0/2492123970 >> 172.21.15.86:0/2492123970 conn(0x55cff2ef3fb0 msgr2=0x55cff2eee370 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:19.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.125+0000 7fa0ae42b740 1 -- 172.21.15.86:0/2492123970 shutdown_connections 2024-04-21T08:51:19.132 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.125+0000 7fa0ae42b740 1 -- 172.21.15.86:0/2492123970 wait complete. 2024-04-21T08:51:19.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.125+0000 7fa0ae42b740 1 -- start start 2024-04-21T08:51:19.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.125+0000 7fa0ae42b740 1 Processor -- start 2024-04-21T08:51:19.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.125+0000 7fa0ae42b740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff2ef98e0 0x55cff2f0b340 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.125+0000 7fa0ae42b740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff2efa290 0x55cff2f08120 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.125+0000 7fa0ae42b740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff2f02e10 0x55cff2f08660 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.125+0000 7fa0ae42b740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a41ae690 con 0x55cff2f02e10 2024-04-21T08:51:19.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.125+0000 7fa0ae42b740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a41adfc0 con 0x55cff2efa290 2024-04-21T08:51:19.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.125+0000 7fa0ae42b740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa0a41ab220 con 0x55cff2ef98e0 2024-04-21T08:51:19.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.125+0000 7fa0ae42b740 10 client.0 _ll_register_callbacks cb 0x7fa0a0010430 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:19.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.126+0000 7fa0a9ddc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff2f02e10 0x55cff2f08660 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.126+0000 7fa0a9ddc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff2f02e10 0x55cff2f08660 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56682/0 (socket says 172.21.15.86:56682) 2024-04-21T08:51:19.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.126+0000 7fa0a9ddc640 1 -- 172.21.15.86:0/1649661264 learned_addr learned my addr 172.21.15.86:0/1649661264 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:19.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.126+0000 7fa0a8dda640 1 --2- 172.21.15.86:0/1649661264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff2efa290 0x55cff2f08120 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.133 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.126+0000 7fa0a95db640 1 --2- 172.21.15.86:0/1649661264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff2ef98e0 0x55cff2f0b340 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.126+0000 7fa0a9ddc640 1 -- 172.21.15.86:0/1649661264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff2ef98e0 msgr2=0x55cff2f0b340 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:19.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.126+0000 7fa0a9ddc640 1 --2- 172.21.15.86:0/1649661264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff2ef98e0 0x55cff2f0b340 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.126+0000 7fa0a9ddc640 1 -- 172.21.15.86:0/1649661264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff2efa290 msgr2=0x55cff2f08120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:19.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.126+0000 7fa0a9ddc640 1 --2- 172.21.15.86:0/1649661264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff2efa290 0x55cff2f08120 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.126+0000 7fa0a9ddc640 1 -- 172.21.15.86:0/1649661264 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55cff2f08ba0 con 0x55cff2f02e10 2024-04-21T08:51:19.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.126+0000 7fa0a8dda640 1 --2- 172.21.15.86:0/1649661264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff2efa290 0x55cff2f08120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:19.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.126+0000 7fa0a9ddc640 1 --2- 172.21.15.86:0/1649661264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff2f02e10 0x55cff2f08660 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7fa09c009b90 tx=0x7fa09c009cc0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=541f3fd64eb49464 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:19.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.126+0000 7fa08cff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:51:19.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.126+0000 7fa08cff9640 1 -- 172.21.15.86:0/1649661264 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa09c036070 con 0x55cff2f02e10 2024-04-21T08:51:19.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.126+0000 7fa08cff9640 1 -- 172.21.15.86:0/1649661264 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7fa09c004920 con 0x55cff2f02e10 2024-04-21T08:51:19.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.126+0000 7fa0ae42b740 1 -- 172.21.15.86:0/1649661264 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fa09c003e10 con 0x55cff2f02e10 2024-04-21T08:51:19.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.126+0000 7fa08cff9640 1 -- 172.21.15.86:0/1649661264 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fa09c019040 con 0x55cff2f02e10 2024-04-21T08:51:19.134 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.126+0000 7fa0ae42b740 10 client.20604 fetch_fsmap learned FSMap version 29 2024-04-21T08:51:19.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.126+0000 7fa0ae42b740 1 -- 172.21.15.86:0/1649661264 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55cff2f7a1e0 con 0x55cff2f02e10 2024-04-21T08:51:19.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.126+0000 7fa08cff9640 1 -- 172.21.15.86:0/1649661264 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa09c003e10 con 0x55cff2f02e10 2024-04-21T08:51:19.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.127+0000 7fa08cff9640 1 -- 172.21.15.86:0/1649661264 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fa09c027480 con 0x55cff2f02e10 2024-04-21T08:51:19.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.127+0000 7fa0ae42b740 10 client.20604 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:51:19.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.127+0000 7fa0ae42b740 10 client.20604 resolve_mds: resolved 1:1 to role '1:1' aka daemon mds.i 2024-04-21T08:51:19.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.127+0000 7fa0ae42b740 20 client.20604 populate_metadata read hostname 'smithi086' 2024-04-21T08:51:19.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.127+0000 7fa0ae42b740 1 --2- 172.21.15.86:0/1649661264 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55cff2f7a700 0x55cff2f7caf0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.127+0000 7fa0ae42b740 4 client.20604 mds_command: new command op to 24472 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:51:19.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.127+0000 7fa0ae42b740 1 -- 172.21.15.86:0/1649661264 --> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55cff2e67c10 con 0x55cff2f7a700 2024-04-21T08:51:19.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.127+0000 7fa0a95db640 1 --2- 172.21.15.86:0/1649661264 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55cff2f7a700 0x55cff2f7caf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.127+0000 7fa0a95db640 1 --2- 172.21.15.86:0/1649661264 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55cff2f7a700 0x55cff2f7caf0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.1 client_cookie=bea5772ec7d45155 server_cookie=867eee0bd4830753 in_seq=0 out_seq=0 2024-04-21T08:51:19.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.127+0000 7fa08cff9640 10 client.20604 ms_handle_connect on v2:172.21.15.179:6838/3029310820 2024-04-21T08:51:19.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.128+0000 7fa08cff9640 1 -- 172.21.15.86:0/1649661264 <== mds.1 v2:172.21.15.179:6838/3029310820 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55cff2e67c10 con 0x55cff2f7a700 2024-04-21T08:51:19.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.128+0000 7fa08cff9640 10 client.20604 handle_command_reply: tid=0 2024-04-21T08:51:19.135 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.128+0000 7fa0ae42b740 1 client.20604 shutdown 2024-04-21T08:51:19.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.128+0000 7fa0ae42b740 1 -- 172.21.15.86:0/1649661264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff2f02e10 msgr2=0x55cff2f08660 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:19.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.128+0000 7fa0ae42b740 1 --2- 172.21.15.86:0/1649661264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff2f02e10 0x55cff2f08660 secure :-1 s=READY pgs=2123 cs=0 l=1 rev1=1 crypto rx=0x7fa09c009b90 tx=0x7fa09c009cc0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.128+0000 7fa0ae42b740 1 -- 172.21.15.86:0/1649661264 shutdown_connections 2024-04-21T08:51:19.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.128+0000 7fa0ae42b740 1 --2- 172.21.15.86:0/1649661264 >> [v2:172.21.15.179:6838/3029310820,v1:172.21.15.179:6839/3029310820] conn(0x55cff2f7a700 0x55cff2f7caf0 crc :-1 s=READY pgs=81 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.128+0000 7fa0ae42b740 1 --2- 172.21.15.86:0/1649661264 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55cff2f02e10 0x55cff2f08660 unknown :-1 s=CLOSED pgs=2123 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.128+0000 7fa08cff9640 0 client.20604 ms_handle_reset on v2:172.21.15.179:6838/3029310820 2024-04-21T08:51:19.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.128+0000 7fa0ae42b740 1 --2- 172.21.15.86:0/1649661264 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55cff2efa290 0x55cff2f08120 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.128+0000 7fa0ae42b740 1 --2- 172.21.15.86:0/1649661264 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55cff2ef98e0 0x55cff2f0b340 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.128+0000 7fa0ae42b740 1 -- 172.21.15.86:0/1649661264 >> 172.21.15.86:0/1649661264 conn(0x55cff2ef3fb0 msgr2=0x55cff2eeec00 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:19.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.128+0000 7fa0ae42b740 1 -- 172.21.15.86:0/1649661264 shutdown_connections 2024-04-21T08:51:19.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.128+0000 7fa0ae42b740 1 -- 172.21.15.86:0/1649661264 wait complete. 2024-04-21T08:51:19.136 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.128+0000 7fa0ae42b740 20 client.20604 trim_cache size 0 max 16384 2024-04-21T08:51:19.137 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:51:19.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.128+0000 7fa0ab83f640 1 -- 172.21.15.86:0/3787123555 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa0900a3d20 msgr2=0x7fa0900a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:19.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.128+0000 7fa0ab83f640 1 --2- 172.21.15.86:0/3787123555 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa0900a3d20 0x7fa0900a61e0 secure :-1 s=READY pgs=357 cs=0 l=1 rev1=1 crypto rx=0x7fa09c021000 tx=0x7fa09c01e000 comp rx=0 tx=0).stop 2024-04-21T08:51:19.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.128+0000 7fa0ab83f640 1 -- 172.21.15.86:0/3787123555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0a4106380 msgr2=0x7fa0a419eb60 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:19.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.128+0000 7fa0ab83f640 1 --2- 172.21.15.86:0/3787123555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0a4106380 0x7fa0a419eb60 secure :-1 s=READY pgs=2119 cs=0 l=1 rev1=1 crypto rx=0x7fa094036040 tx=0x7fa0940045e0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.128+0000 7fa0ab83f640 1 -- 172.21.15.86:0/3787123555 shutdown_connections 2024-04-21T08:51:19.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.128+0000 7fa0ab83f640 1 --2- 172.21.15.86:0/3787123555 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa0900a3d20 0x7fa0900a61e0 unknown :-1 s=CLOSED pgs=357 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.128+0000 7fa0ab83f640 1 --2- 172.21.15.86:0/3787123555 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa0a4198790 0x7fa0a4198c40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.128+0000 7fa0ab83f640 1 --2- 172.21.15.86:0/3787123555 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa0a4106380 0x7fa0a419eb60 unknown :-1 s=CLOSED pgs=2119 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.128+0000 7fa0ab83f640 1 --2- 172.21.15.86:0/3787123555 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa0a41017d0 0x7fa0a419e620 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.128+0000 7fa0ab83f640 1 -- 172.21.15.86:0/3787123555 >> 172.21.15.86:0/3787123555 conn(0x7fa0a4077410 msgr2=0x7fa0a4100900 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:19.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.129+0000 7fa0ab83f640 1 -- 172.21.15.86:0/3787123555 shutdown_connections 2024-04-21T08:51:19.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.129+0000 7fa0ab83f640 1 -- 172.21.15.86:0/3787123555 wait complete. 2024-04-21T08:51:19.139 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell mds.1:0 damage ls 2024-04-21T08:51:19.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:18 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1648506877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:51:19.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.240+0000 7fd29f584640 1 -- 172.21.15.86:0/3179242963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd298107260 msgr2=0x7fd29810ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:19.242 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.240+0000 7fd29f584640 1 --2- 172.21.15.86:0/3179242963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd298107260 0x7fd29810ee30 secure :-1 s=READY pgs=2124 cs=0 l=1 rev1=1 crypto rx=0x7fd290009ca0 tx=0x7fd29002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.240+0000 7fd29f584640 1 -- 172.21.15.86:0/3179242963 shutdown_connections 2024-04-21T08:51:19.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.240+0000 7fd29f584640 1 --2- 172.21.15.86:0/3179242963 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd298107260 0x7fd29810ee30 unknown :-1 s=CLOSED pgs=2124 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.240+0000 7fd29f584640 1 --2- 172.21.15.86:0/3179242963 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd298102b40 0x7fd298106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.240+0000 7fd29f584640 1 --2- 172.21.15.86:0/3179242963 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd298102190 0x7fd298102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.240+0000 7fd29f584640 1 -- 172.21.15.86:0/3179242963 >> 172.21.15.86:0/3179242963 conn(0x7fd2980793a0 msgr2=0x7fd2980797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:19.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.240+0000 7fd29f584640 1 -- 172.21.15.86:0/3179242963 shutdown_connections 2024-04-21T08:51:19.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.240+0000 7fd29f584640 1 -- 172.21.15.86:0/3179242963 wait complete. 2024-04-21T08:51:19.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.241+0000 7fd29f584640 1 Processor -- start 2024-04-21T08:51:19.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.241+0000 7fd29f584640 1 -- start start 2024-04-21T08:51:19.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.241+0000 7fd29f584640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd298102190 0x7fd29819e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.241+0000 7fd29f584640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd298107260 0x7fd29819e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.241+0000 7fd29f584640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd298198590 0x7fd298198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.241+0000 7fd29f584640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd2981121d0 con 0x7fd298102190 2024-04-21T08:51:19.243 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.241+0000 7fd29f584640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd298112050 con 0x7fd298198590 2024-04-21T08:51:19.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.241+0000 7fd29f584640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd298112350 con 0x7fd298107260 2024-04-21T08:51:19.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.241+0000 7fd29d320640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd298102190 0x7fd29819e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.241+0000 7fd29d320640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd298102190 0x7fd29819e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56708/0 (socket says 172.21.15.86:56708) 2024-04-21T08:51:19.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.241+0000 7fd29d320640 1 -- 172.21.15.86:0/1799277511 learned_addr learned my addr 172.21.15.86:0/1799277511 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:19.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.241+0000 7fd29cb1f640 1 --2- 172.21.15.86:0/1799277511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd298107260 0x7fd29819e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.241+0000 7fd29db21640 1 --2- 172.21.15.86:0/1799277511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd298198590 0x7fd298198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.241+0000 7fd29d320640 1 -- 172.21.15.86:0/1799277511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd298107260 msgr2=0x7fd29819e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:19.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.241+0000 7fd29d320640 1 --2- 172.21.15.86:0/1799277511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd298107260 0x7fd29819e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.241+0000 7fd29d320640 1 -- 172.21.15.86:0/1799277511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd298198590 msgr2=0x7fd298198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:19.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.241+0000 7fd29d320640 1 --2- 172.21.15.86:0/1799277511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd298198590 0x7fd298198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.241+0000 7fd29d320640 1 -- 172.21.15.86:0/1799277511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd2981990b0 con 0x7fd298102190 2024-04-21T08:51:19.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.242+0000 7fd29d320640 1 --2- 172.21.15.86:0/1799277511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd298102190 0x7fd29819e3e0 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7fd29400d770 tx=0x7fd29400dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:19.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.242+0000 7fd28e7fc640 1 -- 172.21.15.86:0/1799277511 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd294017070 con 0x7fd298102190 2024-04-21T08:51:19.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.242+0000 7fd28e7fc640 1 -- 172.21.15.86:0/1799277511 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd2940054d0 con 0x7fd298102190 2024-04-21T08:51:19.244 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.242+0000 7fd28e7fc640 1 -- 172.21.15.86:0/1799277511 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd294004c30 con 0x7fd298102190 2024-04-21T08:51:19.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.242+0000 7fd29f584640 1 -- 172.21.15.86:0/1799277511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd2981993a0 con 0x7fd298102190 2024-04-21T08:51:19.245 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.242+0000 7fd29f584640 1 -- 172.21.15.86:0/1799277511 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd2981a55a0 con 0x7fd298102190 2024-04-21T08:51:19.251 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:18 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1648506877' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:51:19.252 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.245+0000 7fd28e7fc640 1 -- 172.21.15.86:0/1799277511 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd294004e10 con 0x7fd298102190 2024-04-21T08:51:19.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.245+0000 7fd2a2179740 1 Processor -- start 2024-04-21T08:51:19.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.245+0000 7fd2a2179740 1 -- start start 2024-04-21T08:51:19.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.245+0000 7fd2a2179740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55debbe583b0 0x55debbe58790 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.245+0000 7fd2a2179740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55debbe58d60 0x55debbe61890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.245+0000 7fd2a2179740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55debbe61e60 0x55debbe622c0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.245+0000 7fd2a2179740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55debbdbd1f0 con 0x55debbe58d60 2024-04-21T08:51:19.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.245+0000 7fd2a2179740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55debbd5a4b0 con 0x55debbe583b0 2024-04-21T08:51:19.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.245+0000 7fd2a2179740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55debbdb91b0 con 0x55debbe61e60 2024-04-21T08:51:19.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.245+0000 7fd29d320640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55debbe58d60 0x55debbe61890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.245+0000 7fd29d320640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55debbe58d60 0x55debbe61890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56720/0 (socket says 172.21.15.86:56720) 2024-04-21T08:51:19.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.245+0000 7fd29d320640 1 -- 172.21.15.86:0/3283744346 learned_addr learned my addr 172.21.15.86:0/3283744346 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:19.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.245+0000 7fd29db21640 1 --2- 172.21.15.86:0/3283744346 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55debbe583b0 0x55debbe58790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.253 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.245+0000 7fd28e7fc640 1 --2- 172.21.15.86:0/1799277511 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd2840a3d20 0x7fd2840a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.245+0000 7fd29cb1f640 1 --2- 172.21.15.86:0/3283744346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55debbe61e60 0x55debbe622c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.245+0000 7fd29d320640 1 -- 172.21.15.86:0/3283744346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55debbe61e60 msgr2=0x55debbe622c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:19.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.245+0000 7fd29d320640 1 --2- 172.21.15.86:0/3283744346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55debbe61e60 0x55debbe622c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.245+0000 7fd29d320640 1 -- 172.21.15.86:0/3283744346 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55debbe583b0 msgr2=0x55debbe58790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:19.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.245+0000 7fd29d320640 1 --2- 172.21.15.86:0/3283744346 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55debbe583b0 0x55debbe58790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.246+0000 7fd29d320640 1 -- 172.21.15.86:0/3283744346 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55debbd4c1b0 con 0x55debbe58d60 2024-04-21T08:51:19.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.246+0000 7fd29db21640 1 --2- 172.21.15.86:0/1799277511 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd2840a3d20 0x7fd2840a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.246+0000 7fd28e7fc640 1 -- 172.21.15.86:0/1799277511 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7fd2940d0750 con 0x7fd298102190 2024-04-21T08:51:19.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.246+0000 7fd29d320640 1 --2- 172.21.15.86:0/3283744346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55debbe58d60 0x55debbe61890 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7fd2940cda40 tx=0x7fd294066920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=ee1006be0418f340 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:19.254 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.246+0000 7fd2837fe640 1 -- 172.21.15.86:0/3283744346 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd294020070 con 0x55debbe58d60 2024-04-21T08:51:19.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.246+0000 7fd2837fe640 1 -- 172.21.15.86:0/3283744346 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd294005680 con 0x55debbe58d60 2024-04-21T08:51:19.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.246+0000 7fd2837fe640 1 -- 172.21.15.86:0/3283744346 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd29409ca20 con 0x55debbe58d60 2024-04-21T08:51:19.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.246+0000 7fd29db21640 1 --2- 172.21.15.86:0/1799277511 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd2840a3d20 0x7fd2840a61e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd290005e30 tx=0x7fd29002e040 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:19.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.247+0000 7fd2a2179740 1 -- 172.21.15.86:0/3283744346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55debbe58d60 msgr2=0x55debbe61890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:19.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.247+0000 7fd2a2179740 1 --2- 172.21.15.86:0/3283744346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55debbe58d60 0x55debbe61890 secure :-1 s=READY pgs=2126 cs=0 l=1 rev1=1 crypto rx=0x7fd2940cda40 tx=0x7fd294066920 comp rx=0 tx=0).stop 2024-04-21T08:51:19.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.247+0000 7fd2a2179740 1 -- 172.21.15.86:0/3283744346 shutdown_connections 2024-04-21T08:51:19.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.247+0000 7fd2a2179740 1 --2- 172.21.15.86:0/3283744346 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55debbe61e60 0x55debbe622c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.247+0000 7fd2a2179740 1 --2- 172.21.15.86:0/3283744346 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55debbe58d60 0x55debbe61890 unknown :-1 s=CLOSED pgs=2126 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.247+0000 7fd2a2179740 1 --2- 172.21.15.86:0/3283744346 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55debbe583b0 0x55debbe58790 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.247+0000 7fd2a2179740 1 -- 172.21.15.86:0/3283744346 >> 172.21.15.86:0/3283744346 conn(0x55debbe52ba0 msgr2=0x55debbe52f80 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:19.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.247+0000 7fd2a2179740 1 -- 172.21.15.86:0/3283744346 shutdown_connections 2024-04-21T08:51:19.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.247+0000 7fd2a2179740 1 -- 172.21.15.86:0/3283744346 wait complete. 2024-04-21T08:51:19.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.247+0000 7fd2a2179740 1 -- start start 2024-04-21T08:51:19.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.247+0000 7fd2a2179740 1 Processor -- start 2024-04-21T08:51:19.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.247+0000 7fd2a2179740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55debbe583b0 0x55debbe68220 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.247+0000 7fd2a2179740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55debbe58d60 0x55debbe655b0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.247+0000 7fd2a2179740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55debbe61e60 0x55debbe65af0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.247+0000 7fd2a2179740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x55debbe68d40 con 0x55debbe583b0 2024-04-21T08:51:19.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.247+0000 7fd2a2179740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x55debbd76500 con 0x55debbe61e60 2024-04-21T08:51:19.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.247+0000 7fd2a2179740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x55debbe68ec0 con 0x55debbe58d60 2024-04-21T08:51:19.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.248+0000 7fd2a2179740 10 client.0 _ll_register_callbacks cb 0x55debbd44b80 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:19.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.248+0000 7fd29d320640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55debbe583b0 0x55debbe68220 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.248+0000 7fd29d320640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55debbe583b0 0x55debbe68220 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56732/0 (socket says 172.21.15.86:56732) 2024-04-21T08:51:19.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.248+0000 7fd29d320640 1 -- 172.21.15.86:0/2788792506 learned_addr learned my addr 172.21.15.86:0/2788792506 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:19.256 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.248+0000 7fd29cb1f640 1 --2- 172.21.15.86:0/2788792506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55debbe58d60 0x55debbe655b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.248+0000 7fd29db21640 1 --2- 172.21.15.86:0/2788792506 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55debbe61e60 0x55debbe65af0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.248+0000 7fd29d320640 1 -- 172.21.15.86:0/2788792506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55debbe58d60 msgr2=0x55debbe655b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:19.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.248+0000 7fd29d320640 1 --2- 172.21.15.86:0/2788792506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55debbe58d60 0x55debbe655b0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.248+0000 7fd29d320640 1 -- 172.21.15.86:0/2788792506 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55debbe61e60 msgr2=0x55debbe65af0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:19.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.248+0000 7fd29d320640 1 --2- 172.21.15.86:0/2788792506 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55debbe61e60 0x55debbe65af0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.248+0000 7fd29d320640 1 -- 172.21.15.86:0/2788792506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55debbacf3f0 con 0x55debbe583b0 2024-04-21T08:51:19.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.248+0000 7fd29d320640 1 --2- 172.21.15.86:0/2788792506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55debbe583b0 0x55debbe68220 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7fd2940cdb80 tx=0x7fd29406d0d0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=1fa1d54674eb1bfa server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:19.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.248+0000 7fd280ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:51:19.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.248+0000 7fd280ff9640 1 -- 172.21.15.86:0/2788792506 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd294020070 con 0x55debbe583b0 2024-04-21T08:51:19.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.248+0000 7fd280ff9640 1 -- 172.21.15.86:0/2788792506 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7fd29409a620 con 0x55debbe583b0 2024-04-21T08:51:19.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.248+0000 7fd2a2179740 1 -- 172.21.15.86:0/2788792506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd294005680 con 0x55debbe583b0 2024-04-21T08:51:19.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.248+0000 7fd280ff9640 1 -- 172.21.15.86:0/2788792506 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd29409bc80 con 0x55debbe583b0 2024-04-21T08:51:19.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.248+0000 7fd2a2179740 10 client.20616 fetch_fsmap learned FSMap version 29 2024-04-21T08:51:19.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.248+0000 7fd2a2179740 1 -- 172.21.15.86:0/2788792506 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55debbb997d0 con 0x55debbe583b0 2024-04-21T08:51:19.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.248+0000 7fd280ff9640 1 -- 172.21.15.86:0/2788792506 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd294005680 con 0x55debbe583b0 2024-04-21T08:51:19.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.249+0000 7fd280ff9640 1 -- 172.21.15.86:0/2788792506 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd2940cdd50 con 0x55debbe583b0 2024-04-21T08:51:19.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.249+0000 7fd2a2179740 10 client.20616 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:51:19.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.249+0000 7fd2a2179740 10 client.20616 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:51:19.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.249+0000 7fd2a2179740 20 client.20616 populate_metadata read hostname 'smithi086' 2024-04-21T08:51:19.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.249+0000 7fd2a2179740 1 --2- 172.21.15.86:0/2788792506 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55debbe75de0 0x55debbe781d0 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.249+0000 7fd2a2179740 4 client.20616 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "get_command_descriptions"}] 2024-04-21T08:51:19.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.249+0000 7fd2a2179740 1 -- 172.21.15.86:0/2788792506 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "get_command_descriptions"}) -- 0x55debbd5c930 con 0x55debbe75de0 2024-04-21T08:51:19.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.249+0000 7fd29cb1f640 1 --2- 172.21.15.86:0/2788792506 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55debbe75de0 0x55debbe781d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.249+0000 7fd280ff9640 10 client.20616 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:51:19.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.249+0000 7fd29cb1f640 1 --2- 172.21.15.86:0/2788792506 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55debbe75de0 0x55debbe781d0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=609ec75cadeba197 server_cookie=6d3f97691ede37a8 in_seq=0 out_seq=0 2024-04-21T08:51:19.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.250+0000 7fd280ff9640 1 -- 172.21.15.86:0/2788792506 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+23246 (crc 0 0 0) 0x55debbd5c930 con 0x55debbe75de0 2024-04-21T08:51:19.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.250+0000 7fd280ff9640 10 client.20616 handle_command_reply: tid=0 2024-04-21T08:51:19.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.250+0000 7fd2a2179740 1 client.20616 shutdown 2024-04-21T08:51:19.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.250+0000 7fd2a2179740 1 -- 172.21.15.86:0/2788792506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55debbe583b0 msgr2=0x55debbe68220 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:19.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.250+0000 7fd2a2179740 1 --2- 172.21.15.86:0/2788792506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55debbe583b0 0x55debbe68220 secure :-1 s=READY pgs=2127 cs=0 l=1 rev1=1 crypto rx=0x7fd2940cdb80 tx=0x7fd29406d0d0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.250+0000 7fd2a2179740 1 -- 172.21.15.86:0/2788792506 shutdown_connections 2024-04-21T08:51:19.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.250+0000 7fd2a2179740 1 --2- 172.21.15.86:0/2788792506 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55debbe75de0 0x55debbe781d0 crc :-1 s=READY pgs=353 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.250+0000 7fd2a2179740 1 --2- 172.21.15.86:0/2788792506 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55debbe61e60 0x55debbe65af0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.250+0000 7fd280ff9640 0 client.20616 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:51:19.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.250+0000 7fd2a2179740 1 --2- 172.21.15.86:0/2788792506 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55debbe58d60 0x55debbe655b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.250+0000 7fd2a2179740 1 --2- 172.21.15.86:0/2788792506 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55debbe583b0 0x55debbe68220 unknown :-1 s=CLOSED pgs=2127 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.250+0000 7fd2a2179740 1 -- 172.21.15.86:0/2788792506 >> 172.21.15.86:0/2788792506 conn(0x55debbe52ba0 msgr2=0x55debbe548f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:19.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.250+0000 7fd2a2179740 1 -- 172.21.15.86:0/2788792506 shutdown_connections 2024-04-21T08:51:19.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.250+0000 7fd2a2179740 1 -- 172.21.15.86:0/2788792506 wait complete. 2024-04-21T08:51:19.260 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.250+0000 7fd2a2179740 20 client.20616 trim_cache size 0 max 16384 2024-04-21T08:51:19.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.263+0000 7fd2a2179740 1 Processor -- start 2024-04-21T08:51:19.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.263+0000 7fd2a2179740 1 -- start start 2024-04-21T08:51:19.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.263+0000 7fd2a2179740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55debbe583b0 0x55debbe60800 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.263+0000 7fd2a2179740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55debbe58d60 0x55debbe64710 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.263+0000 7fd2a2179740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55debbe61e60 0x55debbe71860 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.263+0000 7fd2a2179740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd2981ad290 con 0x55debbe583b0 2024-04-21T08:51:19.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.263+0000 7fd2a2179740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd2981aedd0 con 0x55debbe58d60 2024-04-21T08:51:19.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.263+0000 7fd2a2179740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd2981ad960 con 0x55debbe61e60 2024-04-21T08:51:19.269 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.264+0000 7fd29d320640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55debbe583b0 0x55debbe60800 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.264+0000 7fd29d320640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55debbe583b0 0x55debbe60800 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56734/0 (socket says 172.21.15.86:56734) 2024-04-21T08:51:19.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.264+0000 7fd29d320640 1 -- 172.21.15.86:0/972272419 learned_addr learned my addr 172.21.15.86:0/972272419 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:19.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.264+0000 7fd29db21640 1 --2- 172.21.15.86:0/972272419 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55debbe61e60 0x55debbe71860 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.264+0000 7fd29d320640 1 -- 172.21.15.86:0/972272419 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55debbe61e60 msgr2=0x55debbe71860 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:19.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.264+0000 7fd29d320640 1 --2- 172.21.15.86:0/972272419 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55debbe61e60 0x55debbe71860 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.264+0000 7fd29d320640 1 -- 172.21.15.86:0/972272419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55debbe58d60 msgr2=0x55debbe64710 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:19.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.264+0000 7fd29d320640 1 --2- 172.21.15.86:0/972272419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55debbe58d60 0x55debbe64710 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.264+0000 7fd29d320640 1 -- 172.21.15.86:0/972272419 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x55debbe57fe0 con 0x55debbe583b0 2024-04-21T08:51:19.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.264+0000 7fd29d320640 1 --2- 172.21.15.86:0/972272419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55debbe583b0 0x55debbe60800 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7fd29409c020 tx=0x7fd294005b10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b0d70bd12939a9f server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:19.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.264+0000 7fd282ffd640 1 -- 172.21.15.86:0/972272419 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd294020070 con 0x55debbe583b0 2024-04-21T08:51:19.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.264+0000 7fd282ffd640 1 -- 172.21.15.86:0/972272419 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd294005680 con 0x55debbe583b0 2024-04-21T08:51:19.270 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.264+0000 7fd282ffd640 1 -- 172.21.15.86:0/972272419 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd2940a4f90 con 0x55debbe583b0 2024-04-21T08:51:19.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.265+0000 7fd2a2179740 1 -- 172.21.15.86:0/972272419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55debbe583b0 msgr2=0x55debbe60800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:19.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.265+0000 7fd2a2179740 1 --2- 172.21.15.86:0/972272419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55debbe583b0 0x55debbe60800 secure :-1 s=READY pgs=2128 cs=0 l=1 rev1=1 crypto rx=0x7fd29409c020 tx=0x7fd294005b10 comp rx=0 tx=0).stop 2024-04-21T08:51:19.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.265+0000 7fd2a2179740 1 -- 172.21.15.86:0/972272419 shutdown_connections 2024-04-21T08:51:19.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.265+0000 7fd2a2179740 1 --2- 172.21.15.86:0/972272419 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55debbe61e60 0x55debbe71860 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.265+0000 7fd2a2179740 1 --2- 172.21.15.86:0/972272419 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55debbe58d60 0x55debbe64710 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.265+0000 7fd2a2179740 1 --2- 172.21.15.86:0/972272419 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55debbe583b0 0x55debbe60800 unknown :-1 s=CLOSED pgs=2128 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.265+0000 7fd2a2179740 1 -- 172.21.15.86:0/972272419 >> 172.21.15.86:0/972272419 conn(0x55debbe52ba0 msgr2=0x55debbe71150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:19.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.265+0000 7fd2a2179740 1 -- 172.21.15.86:0/972272419 shutdown_connections 2024-04-21T08:51:19.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.265+0000 7fd2a2179740 1 -- 172.21.15.86:0/972272419 wait complete. 2024-04-21T08:51:19.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.265+0000 7fd2a2179740 1 -- start start 2024-04-21T08:51:19.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.265+0000 7fd2a2179740 1 Processor -- start 2024-04-21T08:51:19.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.265+0000 7fd2a2179740 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55debbe583b0 0x55debbe78e10 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.271 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.265+0000 7fd2a2179740 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55debbe58d60 0x55debbe79350 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.265+0000 7fd2a2179740 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55debbe61e60 0x55debbe79890 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.265+0000 7fd2a2179740 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd2981ae700 con 0x55debbe61e60 2024-04-21T08:51:19.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.265+0000 7fd2a2179740 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd2981ae030 con 0x55debbe583b0 2024-04-21T08:51:19.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.265+0000 7fd2a2179740 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd2981ab290 con 0x55debbe58d60 2024-04-21T08:51:19.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.265+0000 7fd2a2179740 10 client.0 _ll_register_callbacks cb 0x7fd294009e90 invalidate_ino_cb 0 invalidate_dentry_cb 0 switch_interrupt_cb 0 remount_cb 0 2024-04-21T08:51:19.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.266+0000 7fd29db21640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55debbe61e60 0x55debbe79890 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.266+0000 7fd29db21640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55debbe61e60 0x55debbe79890 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56742/0 (socket says 172.21.15.86:56742) 2024-04-21T08:51:19.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.266+0000 7fd29db21640 1 -- 172.21.15.86:0/4050599575 learned_addr learned my addr 172.21.15.86:0/4050599575 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:19.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.266+0000 7fd29cb1f640 1 --2- 172.21.15.86:0/4050599575 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55debbe58d60 0x55debbe79350 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.266+0000 7fd29d320640 1 --2- 172.21.15.86:0/4050599575 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55debbe583b0 0x55debbe78e10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.266+0000 7fd29db21640 1 -- 172.21.15.86:0/4050599575 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55debbe58d60 msgr2=0x55debbe79350 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:19.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.266+0000 7fd29db21640 1 --2- 172.21.15.86:0/4050599575 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55debbe58d60 0x55debbe79350 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.266+0000 7fd29db21640 1 -- 172.21.15.86:0/4050599575 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55debbe583b0 msgr2=0x55debbe78e10 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).mark_down 2024-04-21T08:51:19.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.266+0000 7fd29db21640 1 --2- 172.21.15.86:0/4050599575 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55debbe583b0 0x55debbe78e10 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.272 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.266+0000 7fd29db21640 1 -- 172.21.15.86:0/4050599575 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x55debbedb170 con 0x55debbe61e60 2024-04-21T08:51:19.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.266+0000 7fd29d320640 1 --2- 172.21.15.86:0/4050599575 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55debbe583b0 0x55debbe78e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:19.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.266+0000 7fd29db21640 1 --2- 172.21.15.86:0/4050599575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55debbe61e60 0x55debbe79890 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7fd290037040 tx=0x7fd290005d80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=b4dee4bf577ee493 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:19.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.266+0000 7fd280ff9640 10 client.0 ms_handle_connect on v2:172.21.15.86:3300/0 2024-04-21T08:51:19.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.266+0000 7fd280ff9640 1 -- 172.21.15.86:0/4050599575 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd290050070 con 0x55debbe61e60 2024-04-21T08:51:19.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.266+0000 7fd2a2179740 1 -- 172.21.15.86:0/4050599575 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_get_version(what=fsmap handle=1) -- 0x7fd294005680 con 0x55debbe61e60 2024-04-21T08:51:19.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.266+0000 7fd280ff9640 1 -- 172.21.15.86:0/4050599575 <== mon.0 v2:172.21.15.86:3300/0 2 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7fd290003970 con 0x55debbe61e60 2024-04-21T08:51:19.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.266+0000 7fd280ff9640 1 -- 172.21.15.86:0/4050599575 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_get_version_reply(handle=1 version=29) ==== 24+0+0 (secure 0 0 0) 0x7fd290039040 con 0x55debbe61e60 2024-04-21T08:51:19.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.266+0000 7fd2a2179740 10 client.20622 fetch_fsmap learned FSMap version 29 2024-04-21T08:51:19.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.266+0000 7fd2a2179740 1 -- 172.21.15.86:0/4050599575 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,fsmap=29,monmap=4+}) -- 0x55debbedc430 con 0x55debbe61e60 2024-04-21T08:51:19.273 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.266+0000 7fd280ff9640 1 -- 172.21.15.86:0/4050599575 <== mon.0 v2:172.21.15.86:3300/0 4 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd294005680 con 0x55debbe61e60 2024-04-21T08:51:19.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.267+0000 7fd280ff9640 1 -- 172.21.15.86:0/4050599575 <== mon.0 v2:172.21.15.86:3300/0 5 ==== fsmap(e 29) ==== 8362+0+0 (secure 0 0 0) 0x7fd290034440 con 0x55debbe61e60 2024-04-21T08:51:19.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.267+0000 7fd2a2179740 10 client.20622 fetch_fsmap finished waiting for FSMap version 29 2024-04-21T08:51:19.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.267+0000 7fd2a2179740 10 client.20622 resolve_mds: resolved 1:0 to role '1:0' aka daemon mds.b 2024-04-21T08:51:19.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.267+0000 7fd2a2179740 20 client.20622 populate_metadata read hostname 'smithi086' 2024-04-21T08:51:19.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.267+0000 7fd2a2179740 1 --2- 172.21.15.86:0/4050599575 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55debbedcb50 0x55debbedef40 unknown :-1 s=NONE pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.267+0000 7fd2a2179740 4 client.20622 mds_command: new command op to 24482 tid=0 multi_id=0 [{"prefix": "damage ls"}] 2024-04-21T08:51:19.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.267+0000 7fd2a2179740 1 -- 172.21.15.86:0/4050599575 --> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] -- command(tid 0: {"prefix": "damage ls"}) -- 0x55debbdcac10 con 0x55debbedcb50 2024-04-21T08:51:19.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.267+0000 7fd29d320640 1 --2- 172.21.15.86:0/4050599575 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55debbedcb50 0x55debbedef40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.267+0000 7fd280ff9640 10 client.20622 ms_handle_connect on v2:172.21.15.88:6838/3969608142 2024-04-21T08:51:19.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.267+0000 7fd29d320640 1 --2- 172.21.15.86:0/4050599575 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55debbedcb50 0x55debbedef40 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).ready entity=mds.0 client_cookie=15f45352df43e72a server_cookie=b5088a1c332748a0 in_seq=0 out_seq=0 2024-04-21T08:51:19.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.267+0000 7fd280ff9640 1 -- 172.21.15.86:0/4050599575 <== mds.0 v2:172.21.15.88:6838/3969608142 1 ==== command_reply(tid 0: 0 ) ==== 8+0+3 (crc 0 0 0) 0x55debbdcac10 con 0x55debbedcb50 2024-04-21T08:51:19.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.267+0000 7fd280ff9640 10 client.20622 handle_command_reply: tid=0 2024-04-21T08:51:19.274 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.267+0000 7fd2a2179740 1 client.20622 shutdown 2024-04-21T08:51:19.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.267+0000 7fd2a2179740 1 -- 172.21.15.86:0/4050599575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55debbe61e60 msgr2=0x55debbe79890 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:19.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.267+0000 7fd2a2179740 1 --2- 172.21.15.86:0/4050599575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55debbe61e60 0x55debbe79890 secure :-1 s=READY pgs=2129 cs=0 l=1 rev1=1 crypto rx=0x7fd290037040 tx=0x7fd290005d80 comp rx=0 tx=0).stop 2024-04-21T08:51:19.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.268+0000 7fd2a2179740 1 -- 172.21.15.86:0/4050599575 shutdown_connections 2024-04-21T08:51:19.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.268+0000 7fd2a2179740 1 --2- 172.21.15.86:0/4050599575 >> [v2:172.21.15.88:6838/3969608142,v1:172.21.15.88:6839/3969608142] conn(0x55debbedcb50 0x55debbedef40 crc :-1 s=READY pgs=354 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.268+0000 7fd2a2179740 1 --2- 172.21.15.86:0/4050599575 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x55debbe61e60 0x55debbe79890 unknown :-1 s=CLOSED pgs=2129 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.268+0000 7fd280ff9640 0 client.20622 ms_handle_reset on v2:172.21.15.88:6838/3969608142 2024-04-21T08:51:19.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.268+0000 7fd2a2179740 1 --2- 172.21.15.86:0/4050599575 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x55debbe58d60 0x55debbe79350 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.268+0000 7fd2a2179740 1 --2- 172.21.15.86:0/4050599575 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x55debbe583b0 0x55debbe78e10 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.268+0000 7fd2a2179740 1 -- 172.21.15.86:0/4050599575 >> 172.21.15.86:0/4050599575 conn(0x55debbe52ba0 msgr2=0x55debbe719e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:19.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.268+0000 7fd2a2179740 1 -- 172.21.15.86:0/4050599575 shutdown_connections 2024-04-21T08:51:19.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.268+0000 7fd2a2179740 1 -- 172.21.15.86:0/4050599575 wait complete. 2024-04-21T08:51:19.275 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.268+0000 7fd2a2179740 20 client.20622 trim_cache size 0 max 16384 2024-04-21T08:51:19.276 INFO:teuthology.orchestra.run.smithi086.stdout:[] 2024-04-21T08:51:19.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.268+0000 7fd29f584640 1 -- 172.21.15.86:0/1799277511 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd2840a3d20 msgr2=0x7fd2840a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:19.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.268+0000 7fd29f584640 1 --2- 172.21.15.86:0/1799277511 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd2840a3d20 0x7fd2840a61e0 secure :-1 s=READY pgs=358 cs=0 l=1 rev1=1 crypto rx=0x7fd290005e30 tx=0x7fd29002e040 comp rx=0 tx=0).stop 2024-04-21T08:51:19.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.268+0000 7fd29f584640 1 -- 172.21.15.86:0/1799277511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd298102190 msgr2=0x7fd29819e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:19.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.268+0000 7fd29f584640 1 --2- 172.21.15.86:0/1799277511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd298102190 0x7fd29819e3e0 secure :-1 s=READY pgs=2125 cs=0 l=1 rev1=1 crypto rx=0x7fd29400d770 tx=0x7fd29400dc30 comp rx=0 tx=0).stop 2024-04-21T08:51:19.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.268+0000 7fd29f584640 1 -- 172.21.15.86:0/1799277511 shutdown_connections 2024-04-21T08:51:19.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.268+0000 7fd29f584640 1 --2- 172.21.15.86:0/1799277511 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd2840a3d20 0x7fd2840a61e0 unknown :-1 s=CLOSED pgs=358 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.268+0000 7fd29f584640 1 --2- 172.21.15.86:0/1799277511 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd298198590 0x7fd298198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.268+0000 7fd29f584640 1 --2- 172.21.15.86:0/1799277511 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd298107260 0x7fd29819e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.268+0000 7fd29f584640 1 --2- 172.21.15.86:0/1799277511 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd298102190 0x7fd29819e3e0 unknown :-1 s=CLOSED pgs=2125 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.268+0000 7fd29f584640 1 -- 172.21.15.86:0/1799277511 >> 172.21.15.86:0/1799277511 conn(0x7fd2980793a0 msgr2=0x7fd2980ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:19.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.268+0000 7fd29f584640 1 -- 172.21.15.86:0/1799277511 shutdown_connections 2024-04-21T08:51:19.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.268+0000 7fd29f584640 1 -- 172.21.15.86:0/1799277511 wait complete. 2024-04-21T08:51:19.279 ERROR:tasks.fwd_scrub.fs.[cephfs]:exception: Traceback (most recent call last): File "/home/teuthworker/src/github.com_leonid-s-usov_ceph-ceph_4f6bc5273643aaf5660909e85400647a8951c10b/qa/tasks/fwd_scrub.py", line 35, in _run self.do_scrub() File "/home/teuthworker/src/github.com_leonid-s-usov_ceph-ceph_4f6bc5273643aaf5660909e85400647a8951c10b/qa/tasks/fwd_scrub.py", line 49, in do_scrub self.sleep_unless_stopped(self.sleep_between_iterations) File "/home/teuthworker/src/github.com_leonid-s-usov_ceph-ceph_4f6bc5273643aaf5660909e85400647a8951c10b/qa/tasks/thrasher.py", line 43, in sleep_unless_stopped raise self.Stopped() tasks.thrasher.ThrasherGreenlet.Stopped 2024-04-21T08:51:19.279 ERROR:teuthology.run_tasks:Manager failed: fwd_scrub Traceback (most recent call last): File "/home/teuthworker/src/git.ceph.com_teuthology_9481b1d62f50e7d0a4f3dd83adf6945b08d5ff17/teuthology/run_tasks.py", line 154, in run_tasks suppress = manager.__exit__(*exc_info) File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ next(self.gen) File "/home/teuthworker/src/github.com_leonid-s-usov_ceph-ceph_4f6bc5273643aaf5660909e85400647a8951c10b/qa/tasks/fwd_scrub.py", line 155, in task stop_all_fwd_scrubbers(ctx.ceph[config['cluster']].thrashers) File "/home/teuthworker/src/github.com_leonid-s-usov_ceph-ceph_4f6bc5273643aaf5660909e85400647a8951c10b/qa/tasks/fwd_scrub.py", line 90, in stop_all_fwd_scrubbers raise RuntimeError(f"error during scrub thrashing: {thrasher.exception}") RuntimeError: error during scrub thrashing: 2024-04-21T08:51:19.280 DEBUG:teuthology.run_tasks:Unwinding manager check-counter 2024-04-21T08:51:19.291 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:51:19.391 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.389+0000 7f13cad6d640 1 -- 172.21.15.86:0/2660838180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13c4101b60 msgr2=0x7f13c410dc90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:19.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.389+0000 7f13cad6d640 1 --2- 172.21.15.86:0/2660838180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13c4101b60 0x7f13c410dc90 secure :-1 s=READY pgs=2130 cs=0 l=1 rev1=1 crypto rx=0x7f13bc009cd0 tx=0x7f13bc02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.389+0000 7f13cad6d640 1 -- 172.21.15.86:0/2660838180 shutdown_connections 2024-04-21T08:51:19.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.389+0000 7f13cad6d640 1 --2- 172.21.15.86:0/2660838180 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13c4101b60 0x7f13c410dc90 unknown :-1 s=CLOSED pgs=2130 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.389+0000 7f13cad6d640 1 --2- 172.21.15.86:0/2660838180 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13c41011a0 0x7f13c4101620 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.389+0000 7f13cad6d640 1 --2- 172.21.15.86:0/2660838180 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13c4105e30 0x7f13c4106210 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.389+0000 7f13cad6d640 1 -- 172.21.15.86:0/2660838180 >> 172.21.15.86:0/2660838180 conn(0x7f13c4077e30 msgr2=0x7f13c4078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:19.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.390+0000 7f13cad6d640 1 -- 172.21.15.86:0/2660838180 shutdown_connections 2024-04-21T08:51:19.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.390+0000 7f13cad6d640 1 -- 172.21.15.86:0/2660838180 wait complete. 2024-04-21T08:51:19.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.390+0000 7f13cad6d640 1 Processor -- start 2024-04-21T08:51:19.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.390+0000 7f13cad6d640 1 -- start start 2024-04-21T08:51:19.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.390+0000 7f13cad6d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13c41011a0 0x7f13c41972d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.392 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.390+0000 7f13cad6d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13c4101b60 0x7f13c4197810 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.390+0000 7f13cad6d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13c4105e30 0x7f13c4197d50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.390+0000 7f13cad6d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f13c41110c0 con 0x7f13c41011a0 2024-04-21T08:51:19.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.390+0000 7f13cad6d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f13c4110f40 con 0x7f13c4105e30 2024-04-21T08:51:19.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.390+0000 7f13cad6d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f13c4111240 con 0x7f13c4101b60 2024-04-21T08:51:19.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.390+0000 7f13c8b09640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13c41011a0 0x7f13c41972d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.390+0000 7f13c8b09640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13c41011a0 0x7f13c41972d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:56746/0 (socket says 172.21.15.86:56746) 2024-04-21T08:51:19.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.390+0000 7f13c8b09640 1 -- 172.21.15.86:0/1662396913 learned_addr learned my addr 172.21.15.86:0/1662396913 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:19.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.390+0000 7f13c930a640 1 --2- 172.21.15.86:0/1662396913 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13c4105e30 0x7f13c4197d50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.391+0000 7f13c3fff640 1 --2- 172.21.15.86:0/1662396913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13c4101b60 0x7f13c4197810 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.391+0000 7f13c8b09640 1 -- 172.21.15.86:0/1662396913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13c4101b60 msgr2=0x7f13c4197810 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:19.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.391+0000 7f13c8b09640 1 --2- 172.21.15.86:0/1662396913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13c4101b60 0x7f13c4197810 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.391+0000 7f13c8b09640 1 -- 172.21.15.86:0/1662396913 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13c4105e30 msgr2=0x7f13c4197d50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:19.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.391+0000 7f13c8b09640 1 --2- 172.21.15.86:0/1662396913 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13c4105e30 0x7f13c4197d50 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.393 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.391+0000 7f13c8b09640 1 -- 172.21.15.86:0/1662396913 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f13c41a6ab0 con 0x7f13c41011a0 2024-04-21T08:51:19.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.391+0000 7f13c8b09640 1 --2- 172.21.15.86:0/1662396913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13c41011a0 0x7f13c41972d0 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f13b800c950 tx=0x7f13b800ce10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:19.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.391+0000 7f13c1ffb640 1 -- 172.21.15.86:0/1662396913 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b8007c70 con 0x7f13c41011a0 2024-04-21T08:51:19.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.391+0000 7f13cad6d640 1 -- 172.21.15.86:0/1662396913 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f13c41a6da0 con 0x7f13c41011a0 2024-04-21T08:51:19.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.391+0000 7f13c1ffb640 1 -- 172.21.15.86:0/1662396913 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f13b8007e10 con 0x7f13c41011a0 2024-04-21T08:51:19.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.391+0000 7f13c1ffb640 1 -- 172.21.15.86:0/1662396913 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f13b8018800 con 0x7f13c41011a0 2024-04-21T08:51:19.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.391+0000 7f13cad6d640 1 -- 172.21.15.86:0/1662396913 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f13c41a7270 con 0x7f13c41011a0 2024-04-21T08:51:19.394 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.392+0000 7f13cad6d640 1 -- 172.21.15.86:0/1662396913 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f13c4068de0 con 0x7f13c41011a0 2024-04-21T08:51:19.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.393+0000 7f13c1ffb640 1 -- 172.21.15.86:0/1662396913 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f13b80189a0 con 0x7f13c41011a0 2024-04-21T08:51:19.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.394+0000 7f13c1ffb640 1 --2- 172.21.15.86:0/1662396913 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f13b40a3d20 0x7f13b40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.394+0000 7f13c3fff640 1 --2- 172.21.15.86:0/1662396913 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f13b40a3d20 0x7f13b40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.394+0000 7f13c1ffb640 1 -- 172.21.15.86:0/1662396913 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7f13b80d0d10 con 0x7f13c41011a0 2024-04-21T08:51:19.395 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.394+0000 7f13c3fff640 1 --2- 172.21.15.86:0/1662396913 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f13b40a3d20 0x7f13b40a61e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f13b0005fd0 tx=0x7f13b0005850 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:19.397 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.398+0000 7f13c1ffb640 1 -- 172.21.15.86:0/1662396913 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f13b8014030 con 0x7f13c41011a0 2024-04-21T08:51:19.576 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.577+0000 7f13cad6d640 1 -- 172.21.15.86:0/1662396913 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f13c4108dc0 con 0x7f13c41011a0 2024-04-21T08:51:19.577 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.578+0000 7f13c1ffb640 1 -- 172.21.15.86:0/1662396913 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) ==== 74+0+25253 (secure 0 0 0) 0x7f13b8085e20 con 0x7f13c41011a0 2024-04-21T08:51:19.578 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:51:19.578 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":108,"fsid":"f8cec7cc-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:22.687329+0000","modified":"2024-04-21T08:51:10.626236+0000","last_up_change":"2024-04-21T08:35:30.048315+0000","last_in_change":"2024-04-21T08:35:10.099496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:45.279109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:18.300782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:19.574636+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":21,"snap_epoch":107,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:22.905632+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":21,"snap_epoch":107,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89b7601f-0620-4db2-90f1-b45cea173738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6803","nonce":2571516897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6805","nonce":2571516897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6809","nonce":2571516897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6807","nonce":2571516897}]},"public_addr":"172.21.15.86:6803/2571516897","cluster_addr":"172.21.15.86:6805/2571516897","heartbeat_back_addr":"172.21.15.86:6809/2571516897","heartbeat_front_addr":"172.21.15.86:6807/2571516897","state":["exists","up"]},{"osd":1,"uuid":"4891031c-f98c-42bc-bb3b-a420855ecc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6801","nonce":74067182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6803","nonce":74067182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6807","nonce":74067182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6805","nonce":74067182}]},"public_addr":"172.21.15.88:6801/74067182","cluster_addr":"172.21.15.88:6803/74067182","heartbeat_back_addr":"172.21.15.88:6807/74067182","heartbeat_front_addr":"172.21.15.88:6805/74067182","state":["exists","up"]},{"osd":2,"uuid":"60d12aa4-9312-4fa2-8d0b-2b3f42caad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6801","nonce":4137659051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6803","nonce":4137659051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6807","nonce":4137659051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6805","nonce":4137659051}]},"public_addr":"172.21.15.179:6801/4137659051","cluster_addr":"172.21.15.179:6803/4137659051","heartbeat_back_addr":"172.21.15.179:6807/4137659051","heartbeat_front_addr":"172.21.15.179:6805/4137659051","state":["exists","up"]},{"osd":3,"uuid":"404c3158-8cb0-4c02-b48a-f9ee5ff7b784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6811","nonce":2309603148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6813","nonce":2309603148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6817","nonce":2309603148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6815","nonce":2309603148}]},"public_addr":"172.21.15.86:6811/2309603148","cluster_addr":"172.21.15.86:6813/2309603148","heartbeat_back_addr":"172.21.15.86:6817/2309603148","heartbeat_front_addr":"172.21.15.86:6815/2309603148","state":["exists","up"]},{"osd":4,"uuid":"458ff202-d781-4ec7-ac93-0b49f835b503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6809","nonce":1818714217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6811","nonce":1818714217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6815","nonce":1818714217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6813","nonce":1818714217}]},"public_addr":"172.21.15.88:6809/1818714217","cluster_addr":"172.21.15.88:6811/1818714217","heartbeat_back_addr":"172.21.15.88:6815/1818714217","heartbeat_front_addr":"172.21.15.88:6813/1818714217","state":["exists","up"]},{"osd":5,"uuid":"8123cc90-bd8b-4204-bf31-46b34e9f589d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6809","nonce":863840990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6811","nonce":863840990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6815","nonce":863840990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6813","nonce":863840990}]},"public_addr":"172.21.15.179:6809/863840990","cluster_addr":"172.21.15.179:6811/863840990","heartbeat_back_addr":"172.21.15.179:6815/863840990","heartbeat_front_addr":"172.21.15.179:6813/863840990","state":["exists","up"]},{"osd":6,"uuid":"aac9d7c7-659a-4110-903a-a871c086066f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6819","nonce":1224231118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6821","nonce":1224231118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6825","nonce":1224231118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6823","nonce":1224231118}]},"public_addr":"172.21.15.86:6819/1224231118","cluster_addr":"172.21.15.86:6821/1224231118","heartbeat_back_addr":"172.21.15.86:6825/1224231118","heartbeat_front_addr":"172.21.15.86:6823/1224231118","state":["exists","up"]},{"osd":7,"uuid":"df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6817","nonce":136629878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6819","nonce":136629878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6823","nonce":136629878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6821","nonce":136629878}]},"public_addr":"172.21.15.88:6817/136629878","cluster_addr":"172.21.15.88:6819/136629878","heartbeat_back_addr":"172.21.15.88:6823/136629878","heartbeat_front_addr":"172.21.15.88:6821/136629878","state":["exists","up"]},{"osd":8,"uuid":"63e59f4b-0426-46db-92f7-c55dae61b5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6817","nonce":1770418957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6819","nonce":1770418957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6823","nonce":1770418957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6821","nonce":1770418957}]},"public_addr":"172.21.15.179:6817/1770418957","cluster_addr":"172.21.15.179:6819/1770418957","heartbeat_back_addr":"172.21.15.179:6823/1770418957","heartbeat_front_addr":"172.21.15.179:6821/1770418957","state":["exists","up"]},{"osd":9,"uuid":"5841bb36-ed6b-45d8-b857-eeab139c44e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6827","nonce":1108500274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6829","nonce":1108500274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6833","nonce":1108500274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6831","nonce":1108500274}]},"public_addr":"172.21.15.86:6827/1108500274","cluster_addr":"172.21.15.86:6829/1108500274","heartbeat_back_addr":"172.21.15.86:6833/1108500274","heartbeat_front_addr":"172.21.15.86:6831/1108500274","state":["exists","up"]},{"osd":10,"uuid":"f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6825","nonce":1294275353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6827","nonce":1294275353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6831","nonce":1294275353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6829","nonce":1294275353}]},"public_addr":"172.21.15.88:6825/1294275353","cluster_addr":"172.21.15.88:6827/1294275353","heartbeat_back_addr":"172.21.15.88:6831/1294275353","heartbeat_front_addr":"172.21.15.88:6829/1294275353","state":["exists","up"]},{"osd":11,"uuid":"e0d4cc76-e7a6-4a6b-909c-95488ccd5a36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6825","nonce":4007070233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6827","nonce":4007070233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6831","nonce":4007070233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6829","nonce":4007070233}]},"public_addr":"172.21.15.179:6825/4007070233","cluster_addr":"172.21.15.179:6827/4007070233","heartbeat_back_addr":"172.21.15.179:6831/4007070233","heartbeat_front_addr":"172.21.15.179:6829/4007070233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:55.210605+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:18.723090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:42.547979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:06.421622+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:30.858315+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:55.455313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.623369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:46.185111+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.103627+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:37.297545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:02.637431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:28.199292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1068089552":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/4273877219":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/2707594358":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/4060044103":"2024-04-22T08:38:46.631621+0000","172.21.15.86:6800/1364156965":"2024-04-22T08:38:46.631621+0000","172.21.15.86:6800/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/1364156965":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/1412393225":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6800/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/1687852404":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2020707404":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2568130933":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/3952607648":"2024-04-22T08:28:57.384403+0000","172.21.15.86:6801/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/2756903426":"2024-04-22T08:29:23.190001+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":8,"length":1},{"begin":21,"length":1}]},{"pool":4,"snaps":[{"begin":8,"length":1},{"begin":21,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:51:19.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.580+0000 7f13cad6d640 1 -- 172.21.15.86:0/1662396913 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f13b40a3d20 msgr2=0x7f13b40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:19.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.580+0000 7f13cad6d640 1 --2- 172.21.15.86:0/1662396913 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f13b40a3d20 0x7f13b40a61e0 secure :-1 s=READY pgs=359 cs=0 l=1 rev1=1 crypto rx=0x7f13b0005fd0 tx=0x7f13b0005850 comp rx=0 tx=0).stop 2024-04-21T08:51:19.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.580+0000 7f13cad6d640 1 -- 172.21.15.86:0/1662396913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13c41011a0 msgr2=0x7f13c41972d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:19.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.580+0000 7f13cad6d640 1 --2- 172.21.15.86:0/1662396913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13c41011a0 0x7f13c41972d0 secure :-1 s=READY pgs=2131 cs=0 l=1 rev1=1 crypto rx=0x7f13b800c950 tx=0x7f13b800ce10 comp rx=0 tx=0).stop 2024-04-21T08:51:19.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.580+0000 7f13cad6d640 1 -- 172.21.15.86:0/1662396913 shutdown_connections 2024-04-21T08:51:19.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.580+0000 7f13cad6d640 1 --2- 172.21.15.86:0/1662396913 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f13b40a3d20 0x7f13b40a61e0 unknown :-1 s=CLOSED pgs=359 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.580 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.580+0000 7f13cad6d640 1 --2- 172.21.15.86:0/1662396913 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f13c4105e30 0x7f13c4197d50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.580+0000 7f13cad6d640 1 --2- 172.21.15.86:0/1662396913 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f13c4101b60 0x7f13c4197810 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.580+0000 7f13cad6d640 1 --2- 172.21.15.86:0/1662396913 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f13c41011a0 0x7f13c41972d0 unknown :-1 s=CLOSED pgs=2131 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.580+0000 7f13cad6d640 1 -- 172.21.15.86:0/1662396913 >> 172.21.15.86:0/1662396913 conn(0x7f13c4077e30 msgr2=0x7f13c40fed60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:19.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.580+0000 7f13cad6d640 1 -- 172.21.15.86:0/1662396913 shutdown_connections 2024-04-21T08:51:19.581 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.580+0000 7f13cad6d640 1 -- 172.21.15.86:0/1662396913 wait complete. 2024-04-21T08:51:19.591 INFO:tasks.check_counter.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:31:45.279109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:36:18.300782+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:36:19.574636+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 21, 'snap_epoch': 107, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:36:22.905632+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 21, 'snap_epoch': 107, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:51:19.592 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:51:19.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.693+0000 7f5eeb89a640 1 -- 172.21.15.86:0/2682590923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ee41010a0 msgr2=0x7f5ee4101480 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:19.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.693+0000 7f5eeb89a640 1 --2- 172.21.15.86:0/2682590923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ee41010a0 0x7f5ee4101480 secure :-1 s=READY pgs=2132 cs=0 l=1 rev1=1 crypto rx=0x7f5edc009ca0 tx=0x7f5edc02c0f0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.693+0000 7f5eeb89a640 1 -- 172.21.15.86:0/2682590923 shutdown_connections 2024-04-21T08:51:19.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.693+0000 7f5eeb89a640 1 --2- 172.21.15.86:0/2682590923 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ee4106170 0x7f5ee410dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.693+0000 7f5eeb89a640 1 --2- 172.21.15.86:0/2682590923 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ee4101a50 0x7f5ee4105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.693+0000 7f5eeb89a640 1 --2- 172.21.15.86:0/2682590923 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ee41010a0 0x7f5ee4101480 unknown :-1 s=CLOSED pgs=2132 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.693+0000 7f5eeb89a640 1 -- 172.21.15.86:0/2682590923 >> 172.21.15.86:0/2682590923 conn(0x7f5ee40771d0 msgr2=0x7f5ee40775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:19.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.693+0000 7f5eeb89a640 1 -- 172.21.15.86:0/2682590923 shutdown_connections 2024-04-21T08:51:19.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.693+0000 7f5eeb89a640 1 -- 172.21.15.86:0/2682590923 wait complete. 2024-04-21T08:51:19.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.693+0000 7f5eeb89a640 1 Processor -- start 2024-04-21T08:51:19.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.693+0000 7f5eeb89a640 1 -- start start 2024-04-21T08:51:19.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.693+0000 7f5eeb89a640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ee41010a0 0x7f5ee41986f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.694+0000 7f5eeb89a640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ee4106170 0x7f5ee4198c30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.694+0000 7f5eeb89a640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ee419cfb0 0x7f5ee419d460 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.694+0000 7f5eeb89a640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee41121f0 con 0x7f5ee419cfb0 2024-04-21T08:51:19.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.694+0000 7f5eeb89a640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee4112070 con 0x7f5ee4106170 2024-04-21T08:51:19.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.694+0000 7f5eeb89a640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5ee4112370 con 0x7f5ee41010a0 2024-04-21T08:51:19.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.694+0000 7f5ee9e37640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ee419cfb0 0x7f5ee419d460 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.694+0000 7f5ee9e37640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ee419cfb0 0x7f5ee419d460 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37540/0 (socket says 172.21.15.86:37540) 2024-04-21T08:51:19.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.694+0000 7f5ee9e37640 1 -- 172.21.15.86:0/3010018987 learned_addr learned my addr 172.21.15.86:0/3010018987 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:19.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.694+0000 7f5ee8e35640 1 --2- 172.21.15.86:0/3010018987 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ee4106170 0x7f5ee4198c30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.694+0000 7f5ee9e37640 1 -- 172.21.15.86:0/3010018987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ee41010a0 msgr2=0x7f5ee41986f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:19.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.694+0000 7f5ee9e37640 1 --2- 172.21.15.86:0/3010018987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ee41010a0 0x7f5ee41986f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.694+0000 7f5ee9e37640 1 -- 172.21.15.86:0/3010018987 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ee4106170 msgr2=0x7f5ee4198c30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:19.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.694+0000 7f5ee9e37640 1 --2- 172.21.15.86:0/3010018987 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ee4106170 0x7f5ee4198c30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.694+0000 7f5ee9e37640 1 -- 172.21.15.86:0/3010018987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5ee419daa0 con 0x7f5ee419cfb0 2024-04-21T08:51:19.696 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.694+0000 7f5ee9e37640 1 --2- 172.21.15.86:0/3010018987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ee419cfb0 0x7f5ee419d460 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f5ee000c970 tx=0x7f5ee000ce30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:19.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.694+0000 7f5eda7fc640 1 -- 172.21.15.86:0/3010018987 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0007bb0 con 0x7f5ee419cfb0 2024-04-21T08:51:19.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.694+0000 7f5eda7fc640 1 -- 172.21.15.86:0/3010018987 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5ee0007d50 con 0x7f5ee419cfb0 2024-04-21T08:51:19.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.694+0000 7f5eeb89a640 1 -- 172.21.15.86:0/3010018987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5ee419dd90 con 0x7f5ee419cfb0 2024-04-21T08:51:19.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.694+0000 7f5eda7fc640 1 -- 172.21.15.86:0/3010018987 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5ee0018770 con 0x7f5ee419cfb0 2024-04-21T08:51:19.697 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.694+0000 7f5eeb89a640 1 -- 172.21.15.86:0/3010018987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5ee41a5530 con 0x7f5ee419cfb0 2024-04-21T08:51:19.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.695+0000 7f5eeb89a640 1 -- 172.21.15.86:0/3010018987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5ee40691c0 con 0x7f5ee419cfb0 2024-04-21T08:51:19.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.696+0000 7f5eda7fc640 1 -- 172.21.15.86:0/3010018987 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5ee000f990 con 0x7f5ee419cfb0 2024-04-21T08:51:19.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.697+0000 7f5eda7fc640 1 --2- 172.21.15.86:0/3010018987 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5ed00a3d20 0x7f5ed00a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:19.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.697+0000 7f5ee9636640 1 --2- 172.21.15.86:0/3010018987 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5ed00a3d20 0x7f5ed00a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:19.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.697+0000 7f5eda7fc640 1 -- 172.21.15.86:0/3010018987 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7f5ee00d0c60 con 0x7f5ee419cfb0 2024-04-21T08:51:19.698 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.697+0000 7f5ee9636640 1 --2- 172.21.15.86:0/3010018987 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5ed00a3d20 0x7f5ed00a61e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5edc006010 tx=0x7f5edc032340 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:19.700 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.701+0000 7f5eda7fc640 1 -- 172.21.15.86:0/3010018987 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5ee0014030 con 0x7f5ee419cfb0 2024-04-21T08:51:19.878 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.879+0000 7f5eeb89a640 1 -- 172.21.15.86:0/3010018987 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7f5ee4109400 con 0x7f5ee419cfb0 2024-04-21T08:51:19.879 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.879+0000 7f5eda7fc640 1 -- 172.21.15.86:0/3010018987 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v108) ==== 93+0+10 (secure 0 0 0) 0x7f5ee009cf30 con 0x7f5ee419cfb0 2024-04-21T08:51:19.879 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-21T08:51:19.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.881+0000 7f5eeb89a640 1 -- 172.21.15.86:0/3010018987 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5ed00a3d20 msgr2=0x7f5ed00a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:19.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.881+0000 7f5eeb89a640 1 --2- 172.21.15.86:0/3010018987 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5ed00a3d20 0x7f5ed00a61e0 secure :-1 s=READY pgs=360 cs=0 l=1 rev1=1 crypto rx=0x7f5edc006010 tx=0x7f5edc032340 comp rx=0 tx=0).stop 2024-04-21T08:51:19.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.881+0000 7f5eeb89a640 1 -- 172.21.15.86:0/3010018987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ee419cfb0 msgr2=0x7f5ee419d460 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:19.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.881+0000 7f5eeb89a640 1 --2- 172.21.15.86:0/3010018987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ee419cfb0 0x7f5ee419d460 secure :-1 s=READY pgs=2133 cs=0 l=1 rev1=1 crypto rx=0x7f5ee000c970 tx=0x7f5ee000ce30 comp rx=0 tx=0).stop 2024-04-21T08:51:19.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.881+0000 7f5eeb89a640 1 -- 172.21.15.86:0/3010018987 shutdown_connections 2024-04-21T08:51:19.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.881+0000 7f5eeb89a640 1 --2- 172.21.15.86:0/3010018987 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5ed00a3d20 0x7f5ed00a61e0 unknown :-1 s=CLOSED pgs=360 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.881 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.881+0000 7f5eeb89a640 1 --2- 172.21.15.86:0/3010018987 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5ee419cfb0 0x7f5ee419d460 unknown :-1 s=CLOSED pgs=2133 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.881+0000 7f5eeb89a640 1 --2- 172.21.15.86:0/3010018987 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5ee4106170 0x7f5ee4198c30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.881+0000 7f5eeb89a640 1 --2- 172.21.15.86:0/3010018987 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5ee41010a0 0x7f5ee41986f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:19.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.881+0000 7f5eeb89a640 1 -- 172.21.15.86:0/3010018987 >> 172.21.15.86:0/3010018987 conn(0x7f5ee40771d0 msgr2=0x7f5ee40feeb0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:19.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.881+0000 7f5eeb89a640 1 -- 172.21.15.86:0/3010018987 shutdown_connections 2024-04-21T08:51:19.882 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.881+0000 7f5eeb89a640 1 -- 172.21.15.86:0/3010018987 wait complete. 2024-04-21T08:51:19.892 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:51:20.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.995+0000 7f79416f1640 1 -- 172.21.15.86:0/2645646153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f793c107260 msgr2=0x7f793c10ee30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:20.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.995+0000 7f79416f1640 1 --2- 172.21.15.86:0/2645646153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f793c107260 0x7f793c10ee30 secure :-1 s=READY pgs=2134 cs=0 l=1 rev1=1 crypto rx=0x7f7934009ca0 tx=0x7f793402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.995+0000 7f79416f1640 1 -- 172.21.15.86:0/2645646153 shutdown_connections 2024-04-21T08:51:20.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.995+0000 7f79416f1640 1 --2- 172.21.15.86:0/2645646153 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f793c107260 0x7f793c10ee30 unknown :-1 s=CLOSED pgs=2134 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.000 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.995+0000 7f79416f1640 1 --2- 172.21.15.86:0/2645646153 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f793c102b40 0x7f793c106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.995+0000 7f79416f1640 1 --2- 172.21.15.86:0/2645646153 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f793c102190 0x7f793c102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.995+0000 7f79416f1640 1 -- 172.21.15.86:0/2645646153 >> 172.21.15.86:0/2645646153 conn(0x7f793c0793a0 msgr2=0x7f793c0797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:20.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.995+0000 7f79416f1640 1 -- 172.21.15.86:0/2645646153 shutdown_connections 2024-04-21T08:51:20.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.995+0000 7f79416f1640 1 -- 172.21.15.86:0/2645646153 wait complete. 2024-04-21T08:51:20.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.996+0000 7f79416f1640 1 Processor -- start 2024-04-21T08:51:20.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.996+0000 7f79416f1640 1 -- start start 2024-04-21T08:51:20.001 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.996+0000 7f79416f1640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f793c102190 0x7f793c19e3e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:20.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.996+0000 7f79416f1640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f793c107260 0x7f793c19e920 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:20.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.996+0000 7f79416f1640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f793c198590 0x7f793c198a40 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:20.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.996+0000 7f79416f1640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f793c1121d0 con 0x7f793c102190 2024-04-21T08:51:20.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.996+0000 7f79416f1640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f793c112050 con 0x7f793c198590 2024-04-21T08:51:20.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.996+0000 7f79416f1640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f793c112350 con 0x7f793c107260 2024-04-21T08:51:20.002 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.996+0000 7f793affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f793c102190 0x7f793c19e3e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:20.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.996+0000 7f793affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f793c102190 0x7f793c19e3e0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37562/0 (socket says 172.21.15.86:37562) 2024-04-21T08:51:20.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.996+0000 7f793affd640 1 -- 172.21.15.86:0/3403330387 learned_addr learned my addr 172.21.15.86:0/3403330387 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:20.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.996+0000 7f7933fff640 1 --2- 172.21.15.86:0/3403330387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f793c107260 0x7f793c19e920 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:20.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.996+0000 7f793b7fe640 1 --2- 172.21.15.86:0/3403330387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f793c198590 0x7f793c198a40 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:20.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.996+0000 7f793affd640 1 -- 172.21.15.86:0/3403330387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f793c107260 msgr2=0x7f793c19e920 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:20.003 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.996+0000 7f793affd640 1 --2- 172.21.15.86:0/3403330387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f793c107260 0x7f793c19e920 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.996+0000 7f793affd640 1 -- 172.21.15.86:0/3403330387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f793c198590 msgr2=0x7f793c198a40 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:20.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.996+0000 7f793affd640 1 --2- 172.21.15.86:0/3403330387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f793c198590 0x7f793c198a40 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.996+0000 7f793affd640 1 -- 172.21.15.86:0/3403330387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f793c1990b0 con 0x7f793c102190 2024-04-21T08:51:20.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.996+0000 7f793b7fe640 1 --2- 172.21.15.86:0/3403330387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f793c198590 0x7f793c198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:20.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.996+0000 7f7933fff640 1 --2- 172.21.15.86:0/3403330387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f793c107260 0x7f793c19e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:20.004 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.996+0000 7f793affd640 1 --2- 172.21.15.86:0/3403330387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f793c102190 0x7f793c19e3e0 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f792c00d770 tx=0x7f792c00dc30 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:20.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.996+0000 7f7938ff9640 1 -- 172.21.15.86:0/3403330387 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792c017070 con 0x7f793c102190 2024-04-21T08:51:20.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.997+0000 7f7938ff9640 1 -- 172.21.15.86:0/3403330387 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f792c0054d0 con 0x7f793c102190 2024-04-21T08:51:20.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.997+0000 7f79416f1640 1 -- 172.21.15.86:0/3403330387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f793c1993a0 con 0x7f793c102190 2024-04-21T08:51:20.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.997+0000 7f7938ff9640 1 -- 172.21.15.86:0/3403330387 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f792c004c30 con 0x7f793c102190 2024-04-21T08:51:20.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.997+0000 7f79416f1640 1 -- 172.21.15.86:0/3403330387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f793c1a55a0 con 0x7f793c102190 2024-04-21T08:51:20.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.998+0000 7f79416f1640 1 -- 172.21.15.86:0/3403330387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f793c0691c0 con 0x7f793c102190 2024-04-21T08:51:20.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.999+0000 7f7938ff9640 1 -- 172.21.15.86:0/3403330387 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f792c004e10 con 0x7f793c102190 2024-04-21T08:51:20.005 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.999+0000 7f7938ff9640 1 --2- 172.21.15.86:0/3403330387 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f79280a3d20 0x7f79280a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:20.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.999+0000 7f7933fff640 1 --2- 172.21.15.86:0/3403330387 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f79280a3d20 0x7f79280a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:20.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:19.999+0000 7f7938ff9640 1 -- 172.21.15.86:0/3403330387 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7f792c0d0750 con 0x7f793c102190 2024-04-21T08:51:20.006 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.000+0000 7f7933fff640 1 --2- 172.21.15.86:0/3403330387 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f79280a3d20 0x7f79280a61e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7924005fd0 tx=0x7f7924005850 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:20.007 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.004+0000 7f7938ff9640 1 -- 172.21.15.86:0/3403330387 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f792c09ca20 con 0x7f793c102190 2024-04-21T08:51:20.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.180+0000 7f79416f1640 1 -- 172.21.15.86:0/3403330387 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f793c10a520 con 0x7f793c102190 2024-04-21T08:51:20.180 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.181+0000 7f7938ff9640 1 -- 172.21.15.86:0/3403330387 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v108) ==== 104+0+11 (secure 0 0 0) 0x7f792c09c320 con 0x7f793c102190 2024-04-21T08:51:20.180 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-21T08:51:20.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.183+0000 7f79416f1640 1 -- 172.21.15.86:0/3403330387 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f79280a3d20 msgr2=0x7f79280a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:20.182 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.183+0000 7f79416f1640 1 --2- 172.21.15.86:0/3403330387 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f79280a3d20 0x7f79280a61e0 secure :-1 s=READY pgs=361 cs=0 l=1 rev1=1 crypto rx=0x7f7924005fd0 tx=0x7f7924005850 comp rx=0 tx=0).stop 2024-04-21T08:51:20.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.183+0000 7f79416f1640 1 -- 172.21.15.86:0/3403330387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f793c102190 msgr2=0x7f793c19e3e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:20.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.183+0000 7f79416f1640 1 --2- 172.21.15.86:0/3403330387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f793c102190 0x7f793c19e3e0 secure :-1 s=READY pgs=2135 cs=0 l=1 rev1=1 crypto rx=0x7f792c00d770 tx=0x7f792c00dc30 comp rx=0 tx=0).stop 2024-04-21T08:51:20.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.183+0000 7f79416f1640 1 -- 172.21.15.86:0/3403330387 shutdown_connections 2024-04-21T08:51:20.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.183+0000 7f79416f1640 1 --2- 172.21.15.86:0/3403330387 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f79280a3d20 0x7f79280a61e0 unknown :-1 s=CLOSED pgs=361 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.183+0000 7f79416f1640 1 --2- 172.21.15.86:0/3403330387 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f793c198590 0x7f793c198a40 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.183+0000 7f79416f1640 1 --2- 172.21.15.86:0/3403330387 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f793c107260 0x7f793c19e920 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.183+0000 7f79416f1640 1 --2- 172.21.15.86:0/3403330387 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f793c102190 0x7f793c19e3e0 unknown :-1 s=CLOSED pgs=2135 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.183+0000 7f79416f1640 1 -- 172.21.15.86:0/3403330387 >> 172.21.15.86:0/3403330387 conn(0x7f793c0793a0 msgr2=0x7f793c0ffef0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:20.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.183+0000 7f79416f1640 1 -- 172.21.15.86:0/3403330387 shutdown_connections 2024-04-21T08:51:20.183 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.183+0000 7f79416f1640 1 -- 172.21.15.86:0/3403330387 wait complete. 2024-04-21T08:51:20.192 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:19 smithi086 ceph-mon[27939]: pgmap v393: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.9 KiB/s wr, 14 op/s 2024-04-21T08:51:20.192 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:19 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1662396913' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:51:20.193 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:19 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3010018987' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:51:20.194 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:51:20.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:19 smithi088 ceph-mon[31259]: pgmap v393: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.9 KiB/s wr, 14 op/s 2024-04-21T08:51:20.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:19 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1662396913' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:51:20.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:19 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3010018987' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:51:20.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:19 smithi179 ceph-mon[31829]: pgmap v393: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 17 KiB/s rd, 4.9 KiB/s wr, 14 op/s 2024-04-21T08:51:20.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:19 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1662396913' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:51:20.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:19 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3010018987' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:51:20.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.296+0000 7f4629946640 1 -- 172.21.15.86:0/546472742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4624103af0 msgr2=0x7f462410bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:20.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.296+0000 7f4629946640 1 --2- 172.21.15.86:0/546472742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4624103af0 0x7f462410bfd0 secure :-1 s=READY pgs=2136 cs=0 l=1 rev1=1 crypto rx=0x7f4618009900 tx=0x7f461802bd60 comp rx=0 tx=0).stop 2024-04-21T08:51:20.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.296+0000 7f4629946640 1 -- 172.21.15.86:0/546472742 shutdown_connections 2024-04-21T08:51:20.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.296+0000 7f4629946640 1 --2- 172.21.15.86:0/546472742 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f462410c510 0x7f462410e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.296+0000 7f4629946640 1 --2- 172.21.15.86:0/546472742 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4624103af0 0x7f462410bfd0 unknown :-1 s=CLOSED pgs=2136 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.296+0000 7f4629946640 1 --2- 172.21.15.86:0/546472742 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f46241031d0 0x7f46241035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.296+0000 7f4629946640 1 -- 172.21.15.86:0/546472742 >> 172.21.15.86:0/546472742 conn(0x7f4624077f50 msgr2=0x7f4624078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:20.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.296+0000 7f4629946640 1 -- 172.21.15.86:0/546472742 shutdown_connections 2024-04-21T08:51:20.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.296+0000 7f4629946640 1 -- 172.21.15.86:0/546472742 wait complete. 2024-04-21T08:51:20.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.297+0000 7f4629946640 1 Processor -- start 2024-04-21T08:51:20.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.297+0000 7f4629946640 1 -- start start 2024-04-21T08:51:20.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.297+0000 7f4629946640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46241031d0 0x7f462419e5b0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:20.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.297+0000 7f4629946640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4624103af0 0x7f462419eaf0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:20.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.297+0000 7f4629946640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f462410c510 0x7f4624198670 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:20.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.297+0000 7f4629946640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f4624112200 con 0x7f4624103af0 2024-04-21T08:51:20.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.297+0000 7f4629946640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f4624112080 con 0x7f462410c510 2024-04-21T08:51:20.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.297+0000 7f4629946640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f4624112380 con 0x7f46241031d0 2024-04-21T08:51:20.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.297+0000 7f46227fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4624103af0 0x7f462419eaf0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:20.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.297+0000 7f46227fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4624103af0 0x7f462419eaf0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37566/0 (socket says 172.21.15.86:37566) 2024-04-21T08:51:20.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.297+0000 7f46227fc640 1 -- 172.21.15.86:0/1002540037 learned_addr learned my addr 172.21.15.86:0/1002540037 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:20.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.298+0000 7f4622ffd640 1 --2- 172.21.15.86:0/1002540037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46241031d0 0x7f462419e5b0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:20.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.298+0000 7f46227fc640 1 -- 172.21.15.86:0/1002540037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46241031d0 msgr2=0x7f462419e5b0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:20.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.298+0000 7f46237fe640 1 --2- 172.21.15.86:0/1002540037 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f462410c510 0x7f4624198670 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:20.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.298+0000 7f46227fc640 1 --2- 172.21.15.86:0/1002540037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46241031d0 0x7f462419e5b0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.298+0000 7f46227fc640 1 -- 172.21.15.86:0/1002540037 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f462410c510 msgr2=0x7f4624198670 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:20.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.298+0000 7f46227fc640 1 --2- 172.21.15.86:0/1002540037 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f462410c510 0x7f4624198670 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.298+0000 7f46227fc640 1 -- 172.21.15.86:0/1002540037 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f4624198f30 con 0x7f4624103af0 2024-04-21T08:51:20.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.298+0000 7f46237fe640 1 --2- 172.21.15.86:0/1002540037 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f462410c510 0x7f4624198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:51:20.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.298+0000 7f46227fc640 1 --2- 172.21.15.86:0/1002540037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4624103af0 0x7f462419eaf0 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f4618005940 tx=0x7f461802d920 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:20.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.298+0000 7f4628944640 1 -- 172.21.15.86:0/1002540037 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4618043020 con 0x7f4624103af0 2024-04-21T08:51:20.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.298+0000 7f4629946640 1 -- 172.21.15.86:0/1002540037 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f46241991c0 con 0x7f4624103af0 2024-04-21T08:51:20.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.298+0000 7f4628944640 1 -- 172.21.15.86:0/1002540037 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f461802dd20 con 0x7f4624103af0 2024-04-21T08:51:20.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.298+0000 7f4628944640 1 -- 172.21.15.86:0/1002540037 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f4618003b80 con 0x7f4624103af0 2024-04-21T08:51:20.300 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.298+0000 7f4629946640 1 -- 172.21.15.86:0/1002540037 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f46241a5530 con 0x7f4624103af0 2024-04-21T08:51:20.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.299+0000 7f4629946640 1 -- 172.21.15.86:0/1002540037 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f4624199530 con 0x7f4624103af0 2024-04-21T08:51:20.301 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.300+0000 7f4628944640 1 -- 172.21.15.86:0/1002540037 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f4618046020 con 0x7f4624103af0 2024-04-21T08:51:20.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.301+0000 7f4628944640 1 --2- 172.21.15.86:0/1002540037 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f460c0a3d20 0x7f460c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:20.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.301+0000 7f4622ffd640 1 --2- 172.21.15.86:0/1002540037 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f460c0a3d20 0x7f460c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:20.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.301+0000 7f4628944640 1 -- 172.21.15.86:0/1002540037 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7f46180f33d0 con 0x7f4624103af0 2024-04-21T08:51:20.302 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.301+0000 7f4622ffd640 1 --2- 172.21.15.86:0/1002540037 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f460c0a3d20 0x7f460c0a61e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4614005fd0 tx=0x7f4614009470 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:20.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.304+0000 7f4628944640 1 -- 172.21.15.86:0/1002540037 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f461804d020 con 0x7f4624103af0 2024-04-21T08:51:20.480 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.481+0000 7f4629946640 1 -- 172.21.15.86:0/1002540037 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f4624199ba0 con 0x7f4624103af0 2024-04-21T08:51:20.481 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.481+0000 7f4628944640 1 -- 172.21.15.86:0/1002540037 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v108) ==== 100+0+11 (secure 0 0 0) 0x7f461803e450 con 0x7f4624103af0 2024-04-21T08:51:20.481 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-21T08:51:20.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.483+0000 7f4629946640 1 -- 172.21.15.86:0/1002540037 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f460c0a3d20 msgr2=0x7f460c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:20.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.483+0000 7f4629946640 1 --2- 172.21.15.86:0/1002540037 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f460c0a3d20 0x7f460c0a61e0 secure :-1 s=READY pgs=362 cs=0 l=1 rev1=1 crypto rx=0x7f4614005fd0 tx=0x7f4614009470 comp rx=0 tx=0).stop 2024-04-21T08:51:20.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.483+0000 7f4629946640 1 -- 172.21.15.86:0/1002540037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4624103af0 msgr2=0x7f462419eaf0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:20.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.483+0000 7f4629946640 1 --2- 172.21.15.86:0/1002540037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4624103af0 0x7f462419eaf0 secure :-1 s=READY pgs=2137 cs=0 l=1 rev1=1 crypto rx=0x7f4618005940 tx=0x7f461802d920 comp rx=0 tx=0).stop 2024-04-21T08:51:20.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.483+0000 7f4629946640 1 -- 172.21.15.86:0/1002540037 shutdown_connections 2024-04-21T08:51:20.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.483+0000 7f4629946640 1 --2- 172.21.15.86:0/1002540037 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f460c0a3d20 0x7f460c0a61e0 unknown :-1 s=CLOSED pgs=362 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.483+0000 7f4629946640 1 --2- 172.21.15.86:0/1002540037 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f462410c510 0x7f4624198670 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.483+0000 7f4629946640 1 --2- 172.21.15.86:0/1002540037 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f4624103af0 0x7f462419eaf0 unknown :-1 s=CLOSED pgs=2137 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.483+0000 7f4629946640 1 --2- 172.21.15.86:0/1002540037 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f46241031d0 0x7f462419e5b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.483+0000 7f4629946640 1 -- 172.21.15.86:0/1002540037 >> 172.21.15.86:0/1002540037 conn(0x7f4624077f50 msgr2=0x7f46240ff3b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:20.483 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.483+0000 7f4629946640 1 -- 172.21.15.86:0/1002540037 shutdown_connections 2024-04-21T08:51:20.484 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.483+0000 7f4629946640 1 -- 172.21.15.86:0/1002540037 wait complete. 2024-04-21T08:51:20.494 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:51:20.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.595+0000 7fc403b5f640 1 -- 172.21.15.86:0/429533886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3fc102b30 msgr2=0x7fc3fc106b70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:20.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.595+0000 7fc403b5f640 1 --2- 172.21.15.86:0/429533886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3fc102b30 0x7fc3fc106b70 secure :-1 s=READY pgs=2138 cs=0 l=1 rev1=1 crypto rx=0x7fc3f8009900 tx=0x7fc3f802bc30 comp rx=0 tx=0).stop 2024-04-21T08:51:20.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.595+0000 7fc403b5f640 1 -- 172.21.15.86:0/429533886 shutdown_connections 2024-04-21T08:51:20.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.595+0000 7fc403b5f640 1 --2- 172.21.15.86:0/429533886 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3fc107250 0x7fc3fc10ee20 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.595+0000 7fc403b5f640 1 --2- 172.21.15.86:0/429533886 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3fc102b30 0x7fc3fc106b70 unknown :-1 s=CLOSED pgs=2138 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.595+0000 7fc403b5f640 1 --2- 172.21.15.86:0/429533886 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3fc102180 0x7fc3fc102560 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.597 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.595+0000 7fc403b5f640 1 -- 172.21.15.86:0/429533886 >> 172.21.15.86:0/429533886 conn(0x7fc3fc0fd9c0 msgr2=0x7fc3fc0ffde0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:20.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.595+0000 7fc403b5f640 1 -- 172.21.15.86:0/429533886 shutdown_connections 2024-04-21T08:51:20.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.595+0000 7fc403b5f640 1 -- 172.21.15.86:0/429533886 wait complete. 2024-04-21T08:51:20.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.596+0000 7fc403b5f640 1 Processor -- start 2024-04-21T08:51:20.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.596+0000 7fc403b5f640 1 -- start start 2024-04-21T08:51:20.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.596+0000 7fc403b5f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3fc102180 0x7fc3fc19e3f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:20.598 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.596+0000 7fc403b5f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3fc107250 0x7fc3fc19e930 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:20.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.596+0000 7fc403b5f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3fc1985f0 0x7fc3fc198aa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:20.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.596+0000 7fc403b5f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fc3fc112190 con 0x7fc3fc1985f0 2024-04-21T08:51:20.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.596+0000 7fc403b5f640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fc3fc112010 con 0x7fc3fc102180 2024-04-21T08:51:20.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.596+0000 7fc403b5f640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fc3fc112310 con 0x7fc3fc107250 2024-04-21T08:51:20.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.596+0000 7fc4020fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3fc1985f0 0x7fc3fc198aa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:20.599 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.596+0000 7fc4020fc640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3fc1985f0 0x7fc3fc198aa0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37586/0 (socket says 172.21.15.86:37586) 2024-04-21T08:51:20.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.596+0000 7fc4020fc640 1 -- 172.21.15.86:0/3202888662 learned_addr learned my addr 172.21.15.86:0/3202888662 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:20.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.596+0000 7fc4018fb640 1 --2- 172.21.15.86:0/3202888662 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3fc102180 0x7fc3fc19e3f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:20.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.596+0000 7fc4010fa640 1 --2- 172.21.15.86:0/3202888662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3fc107250 0x7fc3fc19e930 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:20.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.596+0000 7fc4020fc640 1 -- 172.21.15.86:0/3202888662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3fc107250 msgr2=0x7fc3fc19e930 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:20.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.596+0000 7fc4020fc640 1 --2- 172.21.15.86:0/3202888662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3fc107250 0x7fc3fc19e930 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.600 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.596+0000 7fc4020fc640 1 -- 172.21.15.86:0/3202888662 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3fc102180 msgr2=0x7fc3fc19e3f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:20.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.596+0000 7fc4020fc640 1 --2- 172.21.15.86:0/3202888662 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3fc102180 0x7fc3fc19e3f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.596+0000 7fc4020fc640 1 -- 172.21.15.86:0/3202888662 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fc3fc199110 con 0x7fc3fc1985f0 2024-04-21T08:51:20.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.596+0000 7fc4018fb640 1 --2- 172.21.15.86:0/3202888662 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3fc102180 0x7fc3fc19e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:20.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.596+0000 7fc4020fc640 1 --2- 172.21.15.86:0/3202888662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3fc1985f0 0x7fc3fc198aa0 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7fc3f400eaa0 tx=0x7fc3f400ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:20.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.597+0000 7fc3f2ffd640 1 -- 172.21.15.86:0/3202888662 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f400cdb0 con 0x7fc3fc1985f0 2024-04-21T08:51:20.601 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.597+0000 7fc3f2ffd640 1 -- 172.21.15.86:0/3202888662 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fc3f4005020 con 0x7fc3fc1985f0 2024-04-21T08:51:20.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.597+0000 7fc3f2ffd640 1 -- 172.21.15.86:0/3202888662 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fc3f40116d0 con 0x7fc3fc1985f0 2024-04-21T08:51:20.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.597+0000 7fc403b5f640 1 -- 172.21.15.86:0/3202888662 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fc3fc199400 con 0x7fc3fc1985f0 2024-04-21T08:51:20.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.597+0000 7fc403b5f640 1 -- 172.21.15.86:0/3202888662 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fc3fc1a55b0 con 0x7fc3fc1985f0 2024-04-21T08:51:20.602 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.598+0000 7fc403b5f640 1 -- 172.21.15.86:0/3202888662 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fc3fc068d70 con 0x7fc3fc1985f0 2024-04-21T08:51:20.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.599+0000 7fc3f2ffd640 1 -- 172.21.15.86:0/3202888662 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fc3f40051d0 con 0x7fc3fc1985f0 2024-04-21T08:51:20.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.599+0000 7fc3f2ffd640 1 --2- 172.21.15.86:0/3202888662 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc3e80a3d20 0x7fc3e80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:20.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.599+0000 7fc4018fb640 1 --2- 172.21.15.86:0/3202888662 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc3e80a3d20 0x7fc3e80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:20.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.599+0000 7fc3f2ffd640 1 -- 172.21.15.86:0/3202888662 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7fc3f40d0e20 con 0x7fc3fc1985f0 2024-04-21T08:51:20.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.600+0000 7fc4018fb640 1 --2- 172.21.15.86:0/3202888662 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc3e80a3d20 0x7fc3e80a61e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec009c20 tx=0x7fc3ec005f90 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:20.604 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.604+0000 7fc3f2ffd640 1 -- 172.21.15.86:0/3202888662 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fc3f4016030 con 0x7fc3fc1985f0 2024-04-21T08:51:20.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.779+0000 7fc403b5f640 1 -- 172.21.15.86:0/3202888662 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7fc3fc10a480 con 0x7fc3fc1985f0 2024-04-21T08:51:20.779 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.780+0000 7fc3f2ffd640 1 -- 172.21.15.86:0/3202888662 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v108) ==== 103+0+11 (secure 0 0 0) 0x7fc3f409d0f0 con 0x7fc3fc1985f0 2024-04-21T08:51:20.780 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-21T08:51:20.781 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.782+0000 7fc403b5f640 1 -- 172.21.15.86:0/3202888662 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc3e80a3d20 msgr2=0x7fc3e80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:20.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.782+0000 7fc403b5f640 1 --2- 172.21.15.86:0/3202888662 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc3e80a3d20 0x7fc3e80a61e0 secure :-1 s=READY pgs=363 cs=0 l=1 rev1=1 crypto rx=0x7fc3ec009c20 tx=0x7fc3ec005f90 comp rx=0 tx=0).stop 2024-04-21T08:51:20.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.782+0000 7fc403b5f640 1 -- 172.21.15.86:0/3202888662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3fc1985f0 msgr2=0x7fc3fc198aa0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:20.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.782+0000 7fc403b5f640 1 --2- 172.21.15.86:0/3202888662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3fc1985f0 0x7fc3fc198aa0 secure :-1 s=READY pgs=2139 cs=0 l=1 rev1=1 crypto rx=0x7fc3f400eaa0 tx=0x7fc3f400ef60 comp rx=0 tx=0).stop 2024-04-21T08:51:20.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.782+0000 7fc403b5f640 1 -- 172.21.15.86:0/3202888662 shutdown_connections 2024-04-21T08:51:20.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.782+0000 7fc403b5f640 1 --2- 172.21.15.86:0/3202888662 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fc3e80a3d20 0x7fc3e80a61e0 unknown :-1 s=CLOSED pgs=363 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.782+0000 7fc403b5f640 1 --2- 172.21.15.86:0/3202888662 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fc3fc1985f0 0x7fc3fc198aa0 unknown :-1 s=CLOSED pgs=2139 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.782+0000 7fc403b5f640 1 --2- 172.21.15.86:0/3202888662 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fc3fc107250 0x7fc3fc19e930 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.782+0000 7fc403b5f640 1 --2- 172.21.15.86:0/3202888662 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fc3fc102180 0x7fc3fc19e3f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.782+0000 7fc403b5f640 1 -- 172.21.15.86:0/3202888662 >> 172.21.15.86:0/3202888662 conn(0x7fc3fc0fd9c0 msgr2=0x7fc3fc0fee40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:20.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.782+0000 7fc403b5f640 1 -- 172.21.15.86:0/3202888662 shutdown_connections 2024-04-21T08:51:20.782 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.782+0000 7fc403b5f640 1 -- 172.21.15.86:0/3202888662 wait complete. 2024-04-21T08:51:20.793 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json-pretty 2024-04-21T08:51:20.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.892+0000 7f1150d32640 1 -- 172.21.15.86:0/4285886525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f114c103b00 msgr2=0x7f114c10bfe0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:20.894 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.892+0000 7f1150d32640 1 --2- 172.21.15.86:0/4285886525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f114c103b00 0x7f114c10bfe0 secure :-1 s=READY pgs=2140 cs=0 l=1 rev1=1 crypto rx=0x7f113c009900 tx=0x7f113c02bd60 comp rx=0 tx=0).stop 2024-04-21T08:51:20.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.893+0000 7f1150d32640 1 -- 172.21.15.86:0/4285886525 shutdown_connections 2024-04-21T08:51:20.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.893+0000 7f1150d32640 1 --2- 172.21.15.86:0/4285886525 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f114c10c520 0x7f114c10e9c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.893+0000 7f1150d32640 1 --2- 172.21.15.86:0/4285886525 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f114c103b00 0x7f114c10bfe0 unknown :-1 s=CLOSED pgs=2140 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.893+0000 7f1150d32640 1 --2- 172.21.15.86:0/4285886525 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f114c1031e0 0x7f114c1035c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.893+0000 7f1150d32640 1 -- 172.21.15.86:0/4285886525 >> 172.21.15.86:0/4285886525 conn(0x7f114c077f50 msgr2=0x7f114c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:20.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.893+0000 7f1150d32640 1 -- 172.21.15.86:0/4285886525 shutdown_connections 2024-04-21T08:51:20.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.893+0000 7f1150d32640 1 -- 172.21.15.86:0/4285886525 wait complete. 2024-04-21T08:51:20.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.893+0000 7f1150d32640 1 Processor -- start 2024-04-21T08:51:20.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.893+0000 7f1150d32640 1 -- start start 2024-04-21T08:51:20.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.894+0000 7f1150d32640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f114c1031e0 0x7f114c19e420 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:20.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.894+0000 7f1150d32640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f114c103b00 0x7f114c19e960 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:20.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.894+0000 7f1150d32640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f114c10c520 0x7f114c1985d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:20.895 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.894+0000 7f1150d32640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f114c112210 con 0x7f114c10c520 2024-04-21T08:51:20.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.894+0000 7f1150d32640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f114c112090 con 0x7f114c1031e0 2024-04-21T08:51:20.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.894+0000 7f1150d32640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f114c112390 con 0x7f114c103b00 2024-04-21T08:51:20.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.894+0000 7f114ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f114c10c520 0x7f114c1985d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:20.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.894+0000 7f114ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f114c10c520 0x7f114c1985d0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:37614/0 (socket says 172.21.15.86:37614) 2024-04-21T08:51:20.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.894+0000 7f114ad9d640 1 -- 172.21.15.86:0/1174056077 learned_addr learned my addr 172.21.15.86:0/1174056077 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:20.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.894+0000 7f114a59c640 1 --2- 172.21.15.86:0/1174056077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f114c1031e0 0x7f114c19e420 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:20.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.894+0000 7f1149d9b640 1 --2- 172.21.15.86:0/1174056077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f114c103b00 0x7f114c19e960 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:20.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.894+0000 7f114ad9d640 1 -- 172.21.15.86:0/1174056077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f114c103b00 msgr2=0x7f114c19e960 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:20.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.894+0000 7f114ad9d640 1 --2- 172.21.15.86:0/1174056077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f114c103b00 0x7f114c19e960 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.894+0000 7f114ad9d640 1 -- 172.21.15.86:0/1174056077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f114c1031e0 msgr2=0x7f114c19e420 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:20.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.894+0000 7f114ad9d640 1 --2- 172.21.15.86:0/1174056077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f114c1031e0 0x7f114c19e420 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:20.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.894+0000 7f114ad9d640 1 -- 172.21.15.86:0/1174056077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f114c198d50 con 0x7f114c10c520 2024-04-21T08:51:20.896 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.894+0000 7f114ad9d640 1 --2- 172.21.15.86:0/1174056077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f114c10c520 0x7f114c1985d0 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f114400cad0 tx=0x7f114400ced0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:20.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.894+0000 7f11437fe640 1 -- 172.21.15.86:0/1174056077 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114400ad60 con 0x7f114c10c520 2024-04-21T08:51:20.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.894+0000 7f11437fe640 1 -- 172.21.15.86:0/1174056077 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1144005020 con 0x7f114c10c520 2024-04-21T08:51:20.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.894+0000 7f1150d32640 1 -- 172.21.15.86:0/1174056077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f114c199040 con 0x7f114c10c520 2024-04-21T08:51:20.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.894+0000 7f11437fe640 1 -- 172.21.15.86:0/1174056077 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f114400f6b0 con 0x7f114c10c520 2024-04-21T08:51:20.897 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.894+0000 7f1150d32640 1 -- 172.21.15.86:0/1174056077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f114c1a52e0 con 0x7f114c10c520 2024-04-21T08:51:20.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.895+0000 7f1150d32640 1 -- 172.21.15.86:0/1174056077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f114c068d70 con 0x7f114c10c520 2024-04-21T08:51:20.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.896+0000 7f11437fe640 1 -- 172.21.15.86:0/1174056077 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f11440051d0 con 0x7f114c10c520 2024-04-21T08:51:20.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.897+0000 7f11437fe640 1 --2- 172.21.15.86:0/1174056077 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f11380a3d20 0x7f11380a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:20.898 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.897+0000 7f114a59c640 1 --2- 172.21.15.86:0/1174056077 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f11380a3d20 0x7f11380a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:20.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.897+0000 7f11437fe640 1 -- 172.21.15.86:0/1174056077 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7f11440d0c60 con 0x7f114c10c520 2024-04-21T08:51:20.899 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.898+0000 7f114a59c640 1 --2- 172.21.15.86:0/1174056077 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f11380a3d20 0x7f11380a61e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1134009990 tx=0x7f1134009110 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:20.901 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:20.902+0000 7f11437fe640 1 -- 172.21.15.86:0/1174056077 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1144014030 con 0x7f114c10c520 2024-04-21T08:51:21.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:20 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3403330387' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:51:21.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:20 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1002540037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:51:21.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:20 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3202888662' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:51:21.130 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:21.131+0000 7f1150d32640 1 -- 172.21.15.86:0/1174056077 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "mgr dump", "format": "json-pretty"} v 0) -- 0x7f114c199b00 con 0x7f114c10c520 2024-04-21T08:51:21.138 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:21.139+0000 7f11437fe640 1 -- 172.21.15.86:0/1174056077 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "mgr dump", "format": "json-pretty"}]=0 v26) ==== 81+0+746063 (secure 0 0 0) 0x7f114c199b00 con 0x7f114c10c520 2024-04-21T08:51:21.149 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:51:21.150 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:51:21.150 INFO:teuthology.orchestra.run.smithi086.stdout: "epoch": 26, 2024-04-21T08:51:21.150 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.150 INFO:teuthology.orchestra.run.smithi086.stdout: "active_gid": 15558, 2024-04-21T08:51:21.150 INFO:teuthology.orchestra.run.smithi086.stdout: "active_name": "x", 2024-04-21T08:51:21.150 INFO:teuthology.orchestra.run.smithi086.stdout: "active_addrs": { 2024-04-21T08:51:21.150 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-04-21T08:51:21.150 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.150 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-04-21T08:51:21.150 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:6800", 2024-04-21T08:51:21.150 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 2452032403 2024-04-21T08:51:21.151 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.151 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.151 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v1", 2024-04-21T08:51:21.151 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:6801", 2024-04-21T08:51:21.151 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 2452032403 2024-04-21T08:51:21.151 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.151 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-21T08:51:21.151 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.151 INFO:teuthology.orchestra.run.smithi086.stdout: "active_addr": "172.21.15.86:6801/2452032403", 2024-04-21T08:51:21.151 INFO:teuthology.orchestra.run.smithi086.stdout: "active_change": "2024-04-21T08:38:46.631905+0000", 2024-04-21T08:51:21.151 INFO:teuthology.orchestra.run.smithi086.stdout: "active_mgr_features": 4540701547738038271, 2024-04-21T08:51:21.151 INFO:teuthology.orchestra.run.smithi086.stdout: "available": true, 2024-04-21T08:51:21.152 INFO:teuthology.orchestra.run.smithi086.stdout: "standbys": [ 2024-04-21T08:51:21.152 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.152 INFO:teuthology.orchestra.run.smithi086.stdout: "gid": 24922, 2024-04-21T08:51:21.152 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "z", 2024-04-21T08:51:21.152 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr_features": 4540701547738038271, 2024-04-21T08:51:21.152 INFO:teuthology.orchestra.run.smithi086.stdout: "available_modules": [ 2024-04-21T08:51:21.152 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.152 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "alerts", 2024-04-21T08:51:21.152 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.152 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.152 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.152 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-21T08:51:21.153 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-21T08:51:21.153 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:21.153 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.153 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.153 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-21T08:51:21.153 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.153 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.153 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.153 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How frequently to reexamine health status", 2024-04-21T08:51:21.153 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.153 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.153 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.154 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.154 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.154 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.154 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.154 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.154 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.154 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.154 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.154 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.154 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.154 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.154 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.154 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.155 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.155 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.155 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.155 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.155 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.155 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.155 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.155 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.155 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.155 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.155 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.155 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.156 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.156 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.156 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.156 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.156 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.156 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.156 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.156 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.156 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.156 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.156 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.156 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.157 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.157 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.157 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.157 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.157 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.157 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.157 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.157 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.157 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.157 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.157 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.157 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.157 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.158 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.158 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.158 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.158 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.158 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.158 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.158 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.158 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.158 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.158 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.159 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.159 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.159 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.159 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.159 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.159 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.159 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.159 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.159 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.159 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.159 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.159 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_destination": { 2024-04-21T08:51:21.160 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_destination", 2024-04-21T08:51:21.160 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.160 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.160 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.160 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.160 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.160 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.160 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.160 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email address to send alerts to", 2024-04-21T08:51:21.160 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.160 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.160 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.161 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.161 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_from_name": { 2024-04-21T08:51:21.161 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_from_name", 2024-04-21T08:51:21.161 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.161 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.161 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.161 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "Ceph", 2024-04-21T08:51:21.161 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.161 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.161 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.161 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email From: name", 2024-04-21T08:51:21.161 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.161 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.162 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.162 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.162 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_host": { 2024-04-21T08:51:21.162 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_host", 2024-04-21T08:51:21.162 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.162 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.162 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.162 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.162 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.162 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.162 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.162 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP server", 2024-04-21T08:51:21.163 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.163 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.163 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.163 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.163 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_password": { 2024-04-21T08:51:21.163 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_password", 2024-04-21T08:51:21.163 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.163 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.163 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.163 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.163 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.163 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.164 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.164 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Password to authenticate with", 2024-04-21T08:51:21.164 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.164 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.164 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.164 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.164 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_port": { 2024-04-21T08:51:21.164 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_port", 2024-04-21T08:51:21.164 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.164 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.164 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.164 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "465", 2024-04-21T08:51:21.164 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.165 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.165 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.165 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP port", 2024-04-21T08:51:21.165 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.165 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.165 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.165 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.165 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_sender": { 2024-04-21T08:51:21.165 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_sender", 2024-04-21T08:51:21.165 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.165 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.165 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.165 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.166 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.166 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.166 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.166 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP envelope sender", 2024-04-21T08:51:21.166 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.166 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.166 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.166 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.166 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_ssl": { 2024-04-21T08:51:21.166 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_ssl", 2024-04-21T08:51:21.167 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.167 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.167 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.167 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.167 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.167 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.167 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.167 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-21T08:51:21.167 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.167 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.167 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.168 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.168 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_user": { 2024-04-21T08:51:21.168 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_user", 2024-04-21T08:51:21.168 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.168 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.168 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.168 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.168 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.168 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.168 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.168 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "User to authenticate as", 2024-04-21T08:51:21.168 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.169 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.169 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.169 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.169 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.169 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.169 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.169 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "balancer", 2024-04-21T08:51:21.169 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.169 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.169 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.169 INFO:teuthology.orchestra.run.smithi086.stdout: "active": { 2024-04-21T08:51:21.169 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "active", 2024-04-21T08:51:21.169 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.170 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.170 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.170 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.170 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.170 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.170 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.170 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically balance PGs across cluster", 2024-04-21T08:51:21.170 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.170 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.170 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.170 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.171 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_time": { 2024-04-21T08:51:21.171 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_time", 2024-04-21T08:51:21.171 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.171 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.171 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.171 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0000", 2024-04-21T08:51:21.171 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.171 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.171 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.172 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "beginning time of day to automatically balance", 2024-04-21T08:51:21.172 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-21T08:51:21.172 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.172 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.172 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.172 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_weekday": { 2024-04-21T08:51:21.172 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_weekday", 2024-04-21T08:51:21.172 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-21T08:51:21.172 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.172 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.172 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-21T08:51:21.172 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-04-21T08:51:21.173 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-04-21T08:51:21.173 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.173 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-21T08:51:21.173 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-21T08:51:21.173 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.173 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.173 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.173 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_max_iterations": { 2024-04-21T08:51:21.173 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_max_iterations", 2024-04-21T08:51:21.173 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-21T08:51:21.173 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.173 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.174 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "25", 2024-04-21T08:51:21.174 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-21T08:51:21.174 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "250", 2024-04-21T08:51:21.174 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.174 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-21T08:51:21.174 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.174 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.174 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.174 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_metrics": { 2024-04-21T08:51:21.174 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_metrics", 2024-04-21T08:51:21.174 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.175 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.175 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.175 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "pgs,objects,bytes", 2024-04-21T08:51:21.175 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.175 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.175 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.175 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-21T08:51:21.175 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-21T08:51:21.175 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.175 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.175 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_step": { 2024-04-21T08:51:21.176 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_step", 2024-04-21T08:51:21.176 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-21T08:51:21.176 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.176 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.176 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.5", 2024-04-21T08:51:21.176 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0.001", 2024-04-21T08:51:21.176 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "0.999", 2024-04-21T08:51:21.176 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.176 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "aggressiveness of optimization", 2024-04-21T08:51:21.176 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-21T08:51:21.176 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.177 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.177 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.177 INFO:teuthology.orchestra.run.smithi086.stdout: "end_time": { 2024-04-21T08:51:21.177 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_time", 2024-04-21T08:51:21.177 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.177 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.177 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.177 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2359", 2024-04-21T08:51:21.177 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.177 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.177 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.177 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ending time of day to automatically balance", 2024-04-21T08:51:21.177 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-21T08:51:21.178 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.178 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.178 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.178 INFO:teuthology.orchestra.run.smithi086.stdout: "end_weekday": { 2024-04-21T08:51:21.178 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_weekday", 2024-04-21T08:51:21.178 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-21T08:51:21.178 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.178 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.178 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-21T08:51:21.178 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-04-21T08:51:21.178 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-04-21T08:51:21.178 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.179 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-21T08:51:21.179 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-21T08:51:21.179 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.179 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.179 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.179 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.179 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.179 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.179 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.179 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.179 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.180 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.180 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.180 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.180 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.180 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.180 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.180 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.180 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.180 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.180 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.180 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.180 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.180 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.181 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.181 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.181 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.181 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.181 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.181 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.181 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.181 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.181 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.181 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.182 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.182 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.182 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.182 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.182 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.182 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.182 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.182 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.182 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.182 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.182 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.182 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.183 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.183 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.183 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.183 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.183 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.183 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.183 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.183 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.183 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.183 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.183 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.183 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.184 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.184 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.184 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.184 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.184 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.184 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.184 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.184 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.184 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.184 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.184 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.184 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.184 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.185 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.185 INFO:teuthology.orchestra.run.smithi086.stdout: "min_score": { 2024-04-21T08:51:21.185 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_score", 2024-04-21T08:51:21.185 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-21T08:51:21.185 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.185 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.185 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-21T08:51:21.185 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.185 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.185 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.185 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-21T08:51:21.186 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.186 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.186 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.186 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-04-21T08:51:21.186 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-04-21T08:51:21.186 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.186 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.186 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.186 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "upmap", 2024-04-21T08:51:21.186 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.186 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.187 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.187 INFO:teuthology.orchestra.run.smithi086.stdout: "crush-compat", 2024-04-21T08:51:21.187 INFO:teuthology.orchestra.run.smithi086.stdout: "none", 2024-04-21T08:51:21.187 INFO:teuthology.orchestra.run.smithi086.stdout: "read", 2024-04-21T08:51:21.187 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap", 2024-04-21T08:51:21.187 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap-read" 2024-04-21T08:51:21.187 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.187 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Balancer mode", 2024-04-21T08:51:21.187 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.187 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.187 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.187 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_ids": { 2024-04-21T08:51:21.188 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_ids", 2024-04-21T08:51:21.188 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.188 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.188 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.188 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.188 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.188 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.188 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.188 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-21T08:51:21.188 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.188 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.188 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.189 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.189 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-21T08:51:21.189 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-21T08:51:21.189 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:21.189 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.189 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.189 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-21T08:51:21.189 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.189 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.189 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.189 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-21T08:51:21.189 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.189 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.190 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.190 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.190 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_deviation": { 2024-04-21T08:51:21.190 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_deviation", 2024-04-21T08:51:21.190 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.190 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.190 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.190 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-21T08:51:21.190 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-21T08:51:21.190 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.190 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.190 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-21T08:51:21.191 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-21T08:51:21.191 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.191 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.191 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_optimizations": { 2024-04-21T08:51:21.191 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_optimizations", 2024-04-21T08:51:21.191 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-21T08:51:21.191 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.191 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.191 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-21T08:51:21.191 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.191 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.191 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.192 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-21T08:51:21.192 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.192 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.192 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.192 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.192 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.192 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.192 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cephadm", 2024-04-21T08:51:21.192 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.192 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.192 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.192 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_down_multiplier": { 2024-04-21T08:51:21.193 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_down_multiplier", 2024-04-21T08:51:21.193 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-21T08:51:21.193 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.193 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.193 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-04-21T08:51:21.193 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.193 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.193 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.193 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-21T08:51:21.193 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.193 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.193 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.194 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_refresh_rate": { 2024-04-21T08:51:21.194 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_refresh_rate", 2024-04-21T08:51:21.194 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:21.194 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.194 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.194 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "20", 2024-04-21T08:51:21.194 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.194 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.194 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.194 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-21T08:51:21.195 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.195 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.195 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.195 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.195 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_starting_port": { 2024-04-21T08:51:21.195 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_starting_port", 2024-04-21T08:51:21.195 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.195 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.195 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.195 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4721", 2024-04-21T08:51:21.195 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.196 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.196 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.196 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-21T08:51:21.196 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.196 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.196 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.196 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_ptrace": { 2024-04-21T08:51:21.196 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_ptrace", 2024-04-21T08:51:21.196 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.196 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.197 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.197 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.197 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.197 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.197 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.197 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-21T08:51:21.197 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-21T08:51:21.197 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.197 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.197 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_interval": { 2024-04-21T08:51:21.197 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_interval", 2024-04-21T08:51:21.198 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:21.198 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.198 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.198 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-21T08:51:21.198 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.198 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.198 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.198 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-21T08:51:21.198 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.198 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.198 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.198 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.199 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_memory_target_ratio": { 2024-04-21T08:51:21.199 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_memory_target_ratio", 2024-04-21T08:51:21.199 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-21T08:51:21.199 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.199 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.199 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.7", 2024-04-21T08:51:21.199 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.199 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.199 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.199 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-21T08:51:21.199 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.200 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.200 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.200 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm_log_destination": { 2024-04-21T08:51:21.200 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cephadm_log_destination", 2024-04-21T08:51:21.200 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.200 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.200 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.200 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.200 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.200 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.201 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.201 INFO:teuthology.orchestra.run.smithi086.stdout: "file", 2024-04-21T08:51:21.201 INFO:teuthology.orchestra.run.smithi086.stdout: "file,syslog", 2024-04-21T08:51:21.201 INFO:teuthology.orchestra.run.smithi086.stdout: "syslog" 2024-04-21T08:51:21.201 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.201 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-21T08:51:21.201 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.201 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.201 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.201 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.201 INFO:teuthology.orchestra.run.smithi086.stdout: "cgroups_split": { 2024-04-21T08:51:21.201 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cgroups_split", 2024-04-21T08:51:21.202 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.202 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.202 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.202 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.202 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.202 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.202 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.202 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-21T08:51:21.202 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.202 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.202 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.203 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.203 INFO:teuthology.orchestra.run.smithi086.stdout: "config_checks_enabled": { 2024-04-21T08:51:21.203 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_checks_enabled", 2024-04-21T08:51:21.203 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.203 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.203 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.203 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.203 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.203 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.203 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.203 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-21T08:51:21.203 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.204 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.204 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.204 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.204 INFO:teuthology.orchestra.run.smithi086.stdout: "config_dashboard": { 2024-04-21T08:51:21.204 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_dashboard", 2024-04-21T08:51:21.204 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.204 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.204 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.204 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.204 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.204 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.204 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.204 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-21T08:51:21.205 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.205 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.205 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.205 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.205 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_alertmanager": { 2024-04-21T08:51:21.205 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_alertmanager", 2024-04-21T08:51:21.205 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.205 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.205 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.205 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-21T08:51:21.205 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.205 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.206 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.206 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-21T08:51:21.206 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.206 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.206 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.206 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.206 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_base": { 2024-04-21T08:51:21.206 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_base", 2024-04-21T08:51:21.206 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.206 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.206 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.207 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-21T08:51:21.207 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.207 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.207 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.207 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Container image name, without the tag", 2024-04-21T08:51:21.207 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.207 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.207 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.207 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.207 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_elasticsearch": { 2024-04-21T08:51:21.207 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_elasticsearch", 2024-04-21T08:51:21.207 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.208 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.208 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.208 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-21T08:51:21.208 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.208 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.208 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.208 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "elasticsearch container image", 2024-04-21T08:51:21.208 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.208 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.208 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.208 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.209 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_grafana": { 2024-04-21T08:51:21.209 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_grafana", 2024-04-21T08:51:21.209 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.209 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.209 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.209 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-21T08:51:21.209 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.209 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.209 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.209 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-21T08:51:21.209 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.209 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.209 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.210 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.210 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_haproxy": { 2024-04-21T08:51:21.210 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_haproxy", 2024-04-21T08:51:21.210 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.210 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.210 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.210 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-21T08:51:21.210 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.210 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.210 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.210 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "HAproxy container image", 2024-04-21T08:51:21.210 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.211 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.211 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.211 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.211 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_agent": { 2024-04-21T08:51:21.211 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_agent", 2024-04-21T08:51:21.211 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.211 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.211 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.211 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-21T08:51:21.211 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.211 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.211 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.211 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger agent container image", 2024-04-21T08:51:21.212 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.212 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.212 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.212 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.212 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_collector": { 2024-04-21T08:51:21.212 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_collector", 2024-04-21T08:51:21.212 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.212 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.212 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.212 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-21T08:51:21.212 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.212 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.212 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.213 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger collector container image", 2024-04-21T08:51:21.213 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.213 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.213 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.213 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.213 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_query": { 2024-04-21T08:51:21.213 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_query", 2024-04-21T08:51:21.213 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.213 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.213 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.213 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-21T08:51:21.213 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.214 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.214 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.214 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger query container image", 2024-04-21T08:51:21.214 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.214 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.214 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.214 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.214 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_keepalived": { 2024-04-21T08:51:21.214 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_keepalived", 2024-04-21T08:51:21.214 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.214 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.214 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.215 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-21T08:51:21.215 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.215 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.215 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.215 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Keepalived container image", 2024-04-21T08:51:21.215 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.215 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.215 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.215 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.215 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_loki": { 2024-04-21T08:51:21.215 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_loki", 2024-04-21T08:51:21.215 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.215 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.216 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.216 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-21T08:51:21.216 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.216 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.216 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.216 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Loki container image", 2024-04-21T08:51:21.216 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.216 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.216 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.216 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.216 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_node_exporter": { 2024-04-21T08:51:21.216 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_node_exporter", 2024-04-21T08:51:21.216 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.217 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.217 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.217 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-21T08:51:21.217 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.217 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.217 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.217 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-21T08:51:21.217 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.217 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.217 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.217 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.217 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_nvmeof": { 2024-04-21T08:51:21.218 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_nvmeof", 2024-04-21T08:51:21.218 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.218 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.218 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.218 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-21T08:51:21.218 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.218 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.218 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.218 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Nvme-of container image", 2024-04-21T08:51:21.218 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.218 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.219 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.219 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.219 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_prometheus": { 2024-04-21T08:51:21.219 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_prometheus", 2024-04-21T08:51:21.219 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.219 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.219 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.219 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-21T08:51:21.219 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.219 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.219 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.219 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-21T08:51:21.219 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.220 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.220 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.220 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.220 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_promtail": { 2024-04-21T08:51:21.220 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_promtail", 2024-04-21T08:51:21.220 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.220 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.220 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.220 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-21T08:51:21.220 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.220 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.220 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.221 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Promtail container image", 2024-04-21T08:51:21.221 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.221 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.221 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.221 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.221 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_samba": { 2024-04-21T08:51:21.221 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_samba", 2024-04-21T08:51:21.221 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.221 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.221 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.221 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-21T08:51:21.221 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.221 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.222 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.222 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Samba/SMB container image", 2024-04-21T08:51:21.222 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.222 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.222 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.222 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.222 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_snmp_gateway": { 2024-04-21T08:51:21.222 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_snmp_gateway", 2024-04-21T08:51:21.222 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.222 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.222 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.223 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-21T08:51:21.223 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.223 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.223 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.223 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SNMP Gateway container image", 2024-04-21T08:51:21.223 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.223 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.223 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.223 INFO:teuthology.orchestra.run.smithi086.stdout: "container_init": { 2024-04-21T08:51:21.223 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_init", 2024-04-21T08:51:21.223 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.224 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.224 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.224 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.224 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.224 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.224 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.224 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Run podman/docker with `--init`", 2024-04-21T08:51:21.224 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.224 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.224 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.225 INFO:teuthology.orchestra.run.smithi086.stdout: "daemon_cache_timeout": { 2024-04-21T08:51:21.225 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "daemon_cache_timeout", 2024-04-21T08:51:21.225 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:21.225 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.225 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.225 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-21T08:51:21.225 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.225 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.225 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.225 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-21T08:51:21.226 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.226 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.226 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.226 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.226 INFO:teuthology.orchestra.run.smithi086.stdout: "default_cephadm_command_timeout": { 2024-04-21T08:51:21.226 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_cephadm_command_timeout", 2024-04-21T08:51:21.226 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.226 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.226 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.226 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "900", 2024-04-21T08:51:21.226 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.227 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.227 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.227 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-21T08:51:21.227 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.227 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.227 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.227 INFO:teuthology.orchestra.run.smithi086.stdout: "default_registry": { 2024-04-21T08:51:21.227 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_registry", 2024-04-21T08:51:21.227 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.227 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.227 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.228 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io", 2024-04-21T08:51:21.228 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.228 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.228 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.228 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-21T08:51:21.228 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.228 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.228 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.228 INFO:teuthology.orchestra.run.smithi086.stdout: "device_cache_timeout": { 2024-04-21T08:51:21.228 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_cache_timeout", 2024-04-21T08:51:21.228 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:21.229 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.229 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.229 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1800", 2024-04-21T08:51:21.229 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.229 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.229 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.229 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache device inventory", 2024-04-21T08:51:21.229 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.229 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.229 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.229 INFO:teuthology.orchestra.run.smithi086.stdout: "device_enhanced_scan": { 2024-04-21T08:51:21.230 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_enhanced_scan", 2024-04-21T08:51:21.230 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.230 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.230 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.230 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.230 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.230 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.230 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.230 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-21T08:51:21.230 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.230 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.231 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.231 INFO:teuthology.orchestra.run.smithi086.stdout: "facts_cache_timeout": { 2024-04-21T08:51:21.231 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "facts_cache_timeout", 2024-04-21T08:51:21.231 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:21.231 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.231 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.231 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-21T08:51:21.231 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.231 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.231 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.232 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache host facts data", 2024-04-21T08:51:21.232 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.232 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.232 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.232 INFO:teuthology.orchestra.run.smithi086.stdout: "grafana_dashboards_path": { 2024-04-21T08:51:21.232 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "grafana_dashboards_path", 2024-04-21T08:51:21.232 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.232 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.232 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.232 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-21T08:51:21.233 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.233 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.233 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.233 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-21T08:51:21.233 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.233 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.233 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.233 INFO:teuthology.orchestra.run.smithi086.stdout: "host_check_interval": { 2024-04-21T08:51:21.233 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "host_check_interval", 2024-04-21T08:51:21.233 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:21.234 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.234 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.234 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-21T08:51:21.234 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.234 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.234 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.234 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to perform a host check", 2024-04-21T08:51:21.234 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.234 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.234 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.235 INFO:teuthology.orchestra.run.smithi086.stdout: "hw_monitoring": { 2024-04-21T08:51:21.235 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hw_monitoring", 2024-04-21T08:51:21.235 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.235 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.235 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.235 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.235 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.235 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.235 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.235 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-21T08:51:21.235 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.236 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.236 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.236 INFO:teuthology.orchestra.run.smithi086.stdout: "inventory_list_all": { 2024-04-21T08:51:21.236 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "inventory_list_all", 2024-04-21T08:51:21.236 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.236 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.236 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.236 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.236 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.236 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.236 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.236 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-21T08:51:21.237 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.237 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.237 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.237 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.237 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.237 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.237 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.237 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.237 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.237 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.237 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.238 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.238 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.238 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.238 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.238 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.238 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.238 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.238 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.238 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.238 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.238 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.238 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.239 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.239 INFO:teuthology.orchestra.run.smithi086.stdout: "log_refresh_metadata": { 2024-04-21T08:51:21.239 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_refresh_metadata", 2024-04-21T08:51:21.239 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.239 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.239 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.239 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.239 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.239 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.239 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.239 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-21T08:51:21.239 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.240 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.240 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.240 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.240 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.240 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.240 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.240 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.240 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.240 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.240 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.240 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.241 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-21T08:51:21.241 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.241 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.241 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.241 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.241 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.241 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.241 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.241 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.241 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.241 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.241 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.242 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.242 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.242 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.242 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.242 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.242 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.242 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.242 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.242 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.242 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.242 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.242 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.243 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.243 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.243 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.243 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.243 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.243 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.243 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.243 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.243 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.243 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.244 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.244 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.244 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.244 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.244 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-21T08:51:21.244 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-21T08:51:21.244 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.244 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.244 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.244 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.244 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.244 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.245 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.245 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-21T08:51:21.245 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.245 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.245 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.245 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.245 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-21T08:51:21.245 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-21T08:51:21.245 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.245 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.245 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.245 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "*", 2024-04-21T08:51:21.245 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.246 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.246 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.246 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-21T08:51:21.246 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.246 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.246 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.246 INFO:teuthology.orchestra.run.smithi086.stdout: "max_count_per_host": { 2024-04-21T08:51:21.246 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_count_per_host", 2024-04-21T08:51:21.246 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.246 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.246 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.247 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-21T08:51:21.247 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.247 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.247 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.247 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of daemons per service per host", 2024-04-21T08:51:21.247 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.247 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.247 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.247 INFO:teuthology.orchestra.run.smithi086.stdout: "max_osd_draining_count": { 2024-04-21T08:51:21.247 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_osd_draining_count", 2024-04-21T08:51:21.247 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.247 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.248 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.248 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-21T08:51:21.248 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.248 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.248 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.248 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-21T08:51:21.248 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.248 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.248 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.248 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.248 INFO:teuthology.orchestra.run.smithi086.stdout: "migration_current": { 2024-04-21T08:51:21.248 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "migration_current", 2024-04-21T08:51:21.248 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.249 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.249 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.249 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.249 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.249 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.249 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.249 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "internal - do not modify", 2024-04-21T08:51:21.249 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.249 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.249 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.249 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.249 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-04-21T08:51:21.250 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-04-21T08:51:21.250 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.250 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.250 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.250 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "root", 2024-04-21T08:51:21.250 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.250 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.250 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.250 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm-package", 2024-04-21T08:51:21.250 INFO:teuthology.orchestra.run.smithi086.stdout: "root" 2024-04-21T08:51:21.251 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.251 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "mode for remote execution of cephadm", 2024-04-21T08:51:21.251 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.251 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.251 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.251 INFO:teuthology.orchestra.run.smithi086.stdout: "oob_default_addr": { 2024-04-21T08:51:21.251 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "oob_default_addr", 2024-04-21T08:51:21.251 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.251 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.251 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.251 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "169.254.1.1", 2024-04-21T08:51:21.251 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.252 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.252 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.252 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-21T08:51:21.252 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.252 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.252 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.252 INFO:teuthology.orchestra.run.smithi086.stdout: "prometheus_alerts_path": { 2024-04-21T08:51:21.252 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus_alerts_path", 2024-04-21T08:51:21.252 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.252 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.253 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.253 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-21T08:51:21.253 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.253 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.253 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.253 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-21T08:51:21.253 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.253 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.253 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.253 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_insecure": { 2024-04-21T08:51:21.253 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_insecure", 2024-04-21T08:51:21.254 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.254 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.254 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.254 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.254 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.254 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.254 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.254 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-21T08:51:21.254 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.254 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.254 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.254 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_password": { 2024-04-21T08:51:21.255 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_password", 2024-04-21T08:51:21.255 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.255 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.255 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.255 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.255 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.255 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.255 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.255 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-21T08:51:21.255 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.255 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.255 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.256 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_url": { 2024-04-21T08:51:21.256 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_url", 2024-04-21T08:51:21.256 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.256 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.256 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.256 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.256 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.256 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.256 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.256 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-21T08:51:21.256 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.256 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.257 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.257 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.257 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_username": { 2024-04-21T08:51:21.257 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_username", 2024-04-21T08:51:21.257 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.257 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.257 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.257 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.257 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.257 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.257 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.257 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-21T08:51:21.258 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.258 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.258 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.258 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.258 INFO:teuthology.orchestra.run.smithi086.stdout: "secure_monitoring_stack": { 2024-04-21T08:51:21.258 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "secure_monitoring_stack", 2024-04-21T08:51:21.258 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.258 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.258 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.258 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.258 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.258 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.259 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.259 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-21T08:51:21.259 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.259 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.259 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.259 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.259 INFO:teuthology.orchestra.run.smithi086.stdout: "service_discovery_port": { 2024-04-21T08:51:21.259 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "service_discovery_port", 2024-04-21T08:51:21.259 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.259 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.259 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.259 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8765", 2024-04-21T08:51:21.259 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.260 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.260 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.260 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "cephadm service discovery port", 2024-04-21T08:51:21.260 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.260 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.260 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.260 INFO:teuthology.orchestra.run.smithi086.stdout: "ssh_config_file": { 2024-04-21T08:51:21.260 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssh_config_file", 2024-04-21T08:51:21.260 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.260 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.260 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.261 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.261 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.261 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.261 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.261 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-21T08:51:21.261 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.261 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.261 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.261 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.261 INFO:teuthology.orchestra.run.smithi086.stdout: "use_agent": { 2024-04-21T08:51:21.261 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_agent", 2024-04-21T08:51:21.261 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.261 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.262 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.262 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.262 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.262 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.262 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.262 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-21T08:51:21.262 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.262 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.262 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.262 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.262 INFO:teuthology.orchestra.run.smithi086.stdout: "use_repo_digest": { 2024-04-21T08:51:21.263 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_repo_digest", 2024-04-21T08:51:21.263 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.263 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.263 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.263 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.263 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.263 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.263 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.263 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-21T08:51:21.263 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.263 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.263 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.264 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.264 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_failed_host_check": { 2024-04-21T08:51:21.264 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_failed_host_check", 2024-04-21T08:51:21.264 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.264 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.264 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.264 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.264 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.264 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.264 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.264 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if the host check fails", 2024-04-21T08:51:21.264 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.265 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.265 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.265 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.265 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_daemons": { 2024-04-21T08:51:21.265 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_daemons", 2024-04-21T08:51:21.265 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.265 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.265 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.265 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.265 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.265 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.265 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.265 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-21T08:51:21.266 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.266 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.266 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.266 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.266 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_hosts": { 2024-04-21T08:51:21.266 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_hosts", 2024-04-21T08:51:21.266 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.266 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.266 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.266 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.267 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.267 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.267 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.267 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-21T08:51:21.267 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.267 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.267 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.267 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.267 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.267 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.267 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.267 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crash", 2024-04-21T08:51:21.268 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.268 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.268 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.268 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.268 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.268 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.268 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.268 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.268 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.268 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.268 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.269 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.269 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.269 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.269 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.269 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.269 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.269 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.269 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.269 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.269 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.269 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.269 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.269 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.270 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.270 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.270 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.270 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.270 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.270 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.270 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.270 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.270 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.270 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.270 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.270 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.270 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.271 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.271 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.271 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.271 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.271 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.271 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.271 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.271 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.271 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.271 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.271 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.271 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.271 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.272 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.272 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.272 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.272 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.272 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.272 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.272 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.272 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.272 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.272 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.272 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.273 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.273 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.273 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.273 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.273 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.273 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.273 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.273 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.273 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.273 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.273 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.273 INFO:teuthology.orchestra.run.smithi086.stdout: "retain_interval": { 2024-04-21T08:51:21.274 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retain_interval", 2024-04-21T08:51:21.274 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:21.274 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.274 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.274 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "31536000", 2024-04-21T08:51:21.274 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.274 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.274 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.274 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-21T08:51:21.274 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.275 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.275 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.275 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.275 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_recent_interval": { 2024-04-21T08:51:21.275 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_recent_interval", 2024-04-21T08:51:21.275 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:21.275 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.275 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.275 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1209600", 2024-04-21T08:51:21.275 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.275 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.275 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.276 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-21T08:51:21.276 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.276 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.276 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.276 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.276 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.276 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.276 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.276 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dashboard", 2024-04-21T08:51:21.276 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.276 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.276 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.276 INFO:teuthology.orchestra.run.smithi086.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-21T08:51:21.277 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-21T08:51:21.277 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.277 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.277 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.277 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-21T08:51:21.277 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.277 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.277 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.277 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.277 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.277 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.277 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.278 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.278 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_HOST": { 2024-04-21T08:51:21.278 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-21T08:51:21.278 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.278 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.278 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.278 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.278 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.278 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.278 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.278 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.279 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.279 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.279 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.279 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.279 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-21T08:51:21.279 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-21T08:51:21.279 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.279 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.279 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.279 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.279 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.279 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.279 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.280 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.280 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.280 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.280 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.280 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.280 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_ENABLED": { 2024-04-21T08:51:21.280 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_ENABLED", 2024-04-21T08:51:21.280 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.280 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.280 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.280 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.280 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.280 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.281 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.281 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.281 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.281 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.281 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.281 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.281 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-21T08:51:21.281 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-21T08:51:21.281 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.281 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.281 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.281 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.281 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.282 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.282 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.282 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.282 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.282 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.282 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.282 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.282 INFO:teuthology.orchestra.run.smithi086.stdout: "ENABLE_BROWSABLE_API": { 2024-04-21T08:51:21.282 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-21T08:51:21.282 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.282 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.282 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.283 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.283 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.283 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.283 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.283 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.283 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.283 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.283 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.283 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.283 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-21T08:51:21.283 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-21T08:51:21.284 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.284 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.284 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.284 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.284 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.284 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.284 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.284 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.284 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.284 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.284 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.284 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.284 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-21T08:51:21.285 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-21T08:51:21.285 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.285 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.285 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.285 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.285 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.285 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.285 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.285 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.285 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.285 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.285 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.286 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.286 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-21T08:51:21.286 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-21T08:51:21.286 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.286 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.286 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.286 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.286 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.286 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.286 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.286 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.286 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.287 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.287 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.287 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.287 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-21T08:51:21.287 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-21T08:51:21.287 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.287 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.287 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.287 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.287 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.288 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.288 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.288 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.288 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.288 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.288 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.288 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.288 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-21T08:51:21.288 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-21T08:51:21.288 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.288 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.288 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.289 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.289 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.289 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.289 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.289 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.289 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.289 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.289 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.289 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.289 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-21T08:51:21.289 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-21T08:51:21.289 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.290 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.290 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.290 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.290 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.290 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.290 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.290 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.290 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.290 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.290 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.290 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.291 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-21T08:51:21.291 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-21T08:51:21.291 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.291 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.291 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.291 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.291 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.291 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.291 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.291 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.291 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.291 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.292 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.292 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.292 INFO:teuthology.orchestra.run.smithi086.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-21T08:51:21.292 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-21T08:51:21.292 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.292 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.292 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.292 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.292 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.292 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.292 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.293 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.293 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.293 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.293 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.293 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.293 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_PASSWORD": { 2024-04-21T08:51:21.293 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-21T08:51:21.293 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.293 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.293 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.293 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-04-21T08:51:21.293 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.293 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.294 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.294 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.294 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.294 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.294 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.294 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.294 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-21T08:51:21.294 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-21T08:51:21.294 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.294 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.294 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.295 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.295 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.295 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.295 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.295 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.295 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.295 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.295 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.295 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.295 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_URL": { 2024-04-21T08:51:21.295 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_URL", 2024-04-21T08:51:21.295 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.295 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.296 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.296 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.296 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.296 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.296 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.296 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.296 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.296 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.296 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.296 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.296 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_USERNAME": { 2024-04-21T08:51:21.296 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-21T08:51:21.297 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.297 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.297 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.297 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-04-21T08:51:21.297 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.297 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.297 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.297 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.297 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.297 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.297 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.297 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.297 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-21T08:51:21.298 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-21T08:51:21.298 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.298 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.298 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.298 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.298 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.298 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.298 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.298 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.298 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.298 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.298 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.299 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.299 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-21T08:51:21.299 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-21T08:51:21.299 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.299 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.299 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.299 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.299 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.299 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.299 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.299 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.299 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.300 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.300 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.300 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.300 INFO:teuthology.orchestra.run.smithi086.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-21T08:51:21.300 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-21T08:51:21.300 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.300 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.300 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.300 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.300 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.300 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.301 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.301 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.301 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.301 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.301 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.301 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.301 INFO:teuthology.orchestra.run.smithi086.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-21T08:51:21.301 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-21T08:51:21.301 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.301 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.301 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.301 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.301 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.302 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.302 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.302 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.302 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.302 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.302 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.302 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.302 INFO:teuthology.orchestra.run.smithi086.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-21T08:51:21.302 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-21T08:51:21.302 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.302 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.302 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.303 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "[]", 2024-04-21T08:51:21.303 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.303 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.303 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.303 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.303 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.303 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.303 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.303 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.303 INFO:teuthology.orchestra.run.smithi086.stdout: "MULTICLUSTER_CONFIG": { 2024-04-21T08:51:21.303 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-21T08:51:21.304 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.304 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.304 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.304 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "{}", 2024-04-21T08:51:21.304 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.304 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.304 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.304 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.304 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.304 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.304 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.304 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.305 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_HOST": { 2024-04-21T08:51:21.305 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-21T08:51:21.305 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.305 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.305 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.305 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.305 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.305 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.305 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.305 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.305 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.305 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.306 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.306 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.306 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-21T08:51:21.306 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-21T08:51:21.306 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.306 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.306 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.306 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.306 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.306 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.306 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.306 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.307 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.307 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.307 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.307 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.307 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-21T08:51:21.307 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-21T08:51:21.307 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.307 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.307 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.307 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.307 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.307 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.308 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.308 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.308 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.308 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.308 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.308 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.308 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-21T08:51:21.308 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-21T08:51:21.308 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.308 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.308 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.308 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.309 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.309 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.309 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.309 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.309 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.309 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.309 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.309 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.309 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-21T08:51:21.309 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-21T08:51:21.309 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.309 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.309 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.310 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.310 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.310 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.310 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.310 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.310 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.310 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.310 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.310 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.310 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-21T08:51:21.310 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-21T08:51:21.310 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.311 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.311 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.311 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.311 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.311 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.311 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.311 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.311 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.311 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.311 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.311 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.311 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-21T08:51:21.312 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-21T08:51:21.312 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.312 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.312 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.312 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.312 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.312 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.312 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.312 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.312 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.313 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.313 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.313 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.313 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-21T08:51:21.313 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-21T08:51:21.313 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.313 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.313 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.313 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.313 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.314 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.314 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.314 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.314 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.314 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.314 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.314 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.314 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-21T08:51:21.314 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-21T08:51:21.314 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.314 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.315 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.315 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.315 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.315 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.315 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.315 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.315 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.315 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.315 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.315 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.315 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_ENABLED": { 2024-04-21T08:51:21.315 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-21T08:51:21.316 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.316 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.316 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.316 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.316 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.316 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.316 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.316 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.316 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.316 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.316 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.317 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.317 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-21T08:51:21.317 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-21T08:51:21.317 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.317 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.317 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.317 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-21T08:51:21.317 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.317 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.317 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.317 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.317 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.317 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.318 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.318 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.318 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-21T08:51:21.318 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-21T08:51:21.318 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.318 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.318 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.318 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-21T08:51:21.318 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.318 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.318 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.319 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.319 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.319 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.319 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.319 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.319 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-21T08:51:21.319 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-21T08:51:21.319 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.319 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.319 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.319 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8", 2024-04-21T08:51:21.319 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.320 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.320 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.320 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.320 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.320 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.320 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.320 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.320 INFO:teuthology.orchestra.run.smithi086.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-21T08:51:21.320 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-21T08:51:21.320 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.320 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.320 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.321 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "45", 2024-04-21T08:51:21.321 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.321 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.321 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.321 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.321 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.321 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.321 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.321 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.321 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ACCESS_KEY": { 2024-04-21T08:51:21.321 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-21T08:51:21.321 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.322 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.322 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.322 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.322 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.322 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.322 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.322 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.322 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.322 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.322 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.322 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.322 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-21T08:51:21.323 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-21T08:51:21.323 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.323 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.323 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.323 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-04-21T08:51:21.323 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.323 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.323 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.323 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.323 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.323 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.323 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.324 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.324 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SECRET_KEY": { 2024-04-21T08:51:21.324 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-21T08:51:21.324 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.324 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.324 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.324 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.324 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.324 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.324 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.324 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.324 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.324 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.325 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.325 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.325 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SSL_VERIFY": { 2024-04-21T08:51:21.325 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-21T08:51:21.325 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.325 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.325 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.325 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.325 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.325 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.325 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.325 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.326 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.326 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.326 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.326 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.326 INFO:teuthology.orchestra.run.smithi086.stdout: "UNSAFE_TLS_v1_2": { 2024-04-21T08:51:21.326 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-21T08:51:21.326 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.326 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.326 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.326 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.326 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.326 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.327 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.327 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.327 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.327 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.327 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.327 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.327 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-21T08:51:21.327 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-21T08:51:21.327 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.327 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.327 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.328 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-21T08:51:21.328 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.328 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.328 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.328 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.328 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.328 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.328 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.328 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.328 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-21T08:51:21.328 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-21T08:51:21.329 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.329 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.329 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.329 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-21T08:51:21.329 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.329 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.329 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.329 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.329 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.329 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.329 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.329 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.329 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-21T08:51:21.330 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-21T08:51:21.330 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.330 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.330 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.330 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-21T08:51:21.330 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.330 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.330 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.330 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.330 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.331 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.331 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.331 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.331 INFO:teuthology.orchestra.run.smithi086.stdout: "cross_origin_url": { 2024-04-21T08:51:21.331 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cross_origin_url", 2024-04-21T08:51:21.331 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.331 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.331 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.331 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.332 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.332 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.332 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.332 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.332 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.332 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.332 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.332 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.332 INFO:teuthology.orchestra.run.smithi086.stdout: "crt_file": { 2024-04-21T08:51:21.332 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crt_file", 2024-04-21T08:51:21.332 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.333 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.333 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.333 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.333 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.333 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.333 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.333 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.333 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.333 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.333 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.333 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.333 INFO:teuthology.orchestra.run.smithi086.stdout: "debug": { 2024-04-21T08:51:21.333 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "debug", 2024-04-21T08:51:21.334 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.334 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.334 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.334 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.334 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.334 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.334 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.334 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable/disable debug options", 2024-04-21T08:51:21.334 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.334 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.334 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.334 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.335 INFO:teuthology.orchestra.run.smithi086.stdout: "jwt_token_ttl": { 2024-04-21T08:51:21.335 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "jwt_token_ttl", 2024-04-21T08:51:21.335 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.335 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.335 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.335 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "28800", 2024-04-21T08:51:21.335 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.335 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.335 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.335 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.335 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.335 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.336 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.336 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.336 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-04-21T08:51:21.336 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-04-21T08:51:21.336 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.336 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.336 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.336 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.336 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.336 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.336 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.336 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.336 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.337 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.337 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.337 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.337 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.337 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.337 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.337 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.337 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.337 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.337 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.337 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.337 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.338 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.338 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.338 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.338 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.338 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.338 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.338 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.338 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.338 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.338 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.338 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.338 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.339 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.339 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.339 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.339 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.339 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.339 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.339 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.339 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.339 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.339 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.339 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.339 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.339 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.340 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.340 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.340 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.340 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.340 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.340 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.340 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.340 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.340 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.340 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.340 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.341 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.341 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.341 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.341 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.341 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.341 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.341 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.341 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.341 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.341 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.341 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.341 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.341 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.342 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.342 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.342 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.342 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.342 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.342 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.342 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.342 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.342 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.342 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.342 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.342 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.343 INFO:teuthology.orchestra.run.smithi086.stdout: "motd": { 2024-04-21T08:51:21.343 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "motd", 2024-04-21T08:51:21.343 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.343 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.343 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.343 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.343 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.343 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.343 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.343 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "The message of the day", 2024-04-21T08:51:21.343 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.343 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.344 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.344 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.344 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect_resolve_ip_addr": { 2024-04-21T08:51:21.344 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "redirect_resolve_ip_addr", 2024-04-21T08:51:21.344 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.344 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.344 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.344 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.344 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.344 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.344 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.344 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.344 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.345 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.345 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.345 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.345 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-04-21T08:51:21.345 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-04-21T08:51:21.345 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.345 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.345 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.345 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-04-21T08:51:21.345 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.345 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.345 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.346 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.346 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.346 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.346 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.346 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.346 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-04-21T08:51:21.346 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-04-21T08:51:21.346 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.346 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.347 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.347 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8080", 2024-04-21T08:51:21.347 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.347 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.347 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.347 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.347 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.347 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.347 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.347 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.347 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-04-21T08:51:21.347 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-04-21T08:51:21.348 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.348 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.348 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.348 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.348 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.348 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.348 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.348 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.348 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.348 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.348 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.349 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.349 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl_server_port": { 2024-04-21T08:51:21.349 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl_server_port", 2024-04-21T08:51:21.349 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.349 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.349 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.349 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8443", 2024-04-21T08:51:21.349 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.349 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.349 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.349 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.350 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.350 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.350 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.350 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.350 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-04-21T08:51:21.350 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-04-21T08:51:21.350 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.350 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.350 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.350 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "redirect", 2024-04-21T08:51:21.350 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.351 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.351 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.351 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.351 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect" 2024-04-21T08:51:21.351 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.351 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.351 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.351 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.351 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.351 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.351 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-04-21T08:51:21.351 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-04-21T08:51:21.351 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.352 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.352 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.352 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-04-21T08:51:21.352 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-04-21T08:51:21.352 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-04-21T08:51:21.352 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.352 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.352 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.352 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.352 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.352 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.353 INFO:teuthology.orchestra.run.smithi086.stdout: "url_prefix": { 2024-04-21T08:51:21.353 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url_prefix", 2024-04-21T08:51:21.353 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.353 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.353 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.353 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.353 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.353 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.353 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.353 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.353 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.354 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.354 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.354 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.354 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.354 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.354 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.354 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "devicehealth", 2024-04-21T08:51:21.354 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.354 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.354 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.355 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_monitoring": { 2024-04-21T08:51:21.355 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_monitoring", 2024-04-21T08:51:21.355 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.355 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.355 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.355 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.355 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.355 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.355 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.355 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "monitor device health metrics", 2024-04-21T08:51:21.355 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.355 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.356 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.356 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.356 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.356 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.356 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.357 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.357 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.357 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.357 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.357 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.357 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.357 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.358 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.358 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.358 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.358 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.358 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.358 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.358 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.358 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.358 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.358 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.358 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.359 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.359 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.359 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.359 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.359 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.359 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.359 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.359 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.359 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.359 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.359 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.360 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.360 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.360 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.360 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.360 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.360 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.360 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.360 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.360 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.360 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.360 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.361 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.361 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.361 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.361 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.361 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.361 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.361 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.361 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.361 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.361 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.361 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.362 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.362 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.362 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.362 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.362 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.362 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.362 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.362 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.362 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.362 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.362 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.363 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.363 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.363 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.363 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.363 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.363 INFO:teuthology.orchestra.run.smithi086.stdout: "mark_out_threshold": { 2024-04-21T08:51:21.363 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mark_out_threshold", 2024-04-21T08:51:21.363 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:21.363 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.363 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.363 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2419200", 2024-04-21T08:51:21.363 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.364 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.364 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.364 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-21T08:51:21.364 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.364 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.364 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.364 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.364 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_name": { 2024-04-21T08:51:21.364 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_name", 2024-04-21T08:51:21.364 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.364 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.364 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.365 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "device_health_metrics", 2024-04-21T08:51:21.365 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.365 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.365 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.365 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-21T08:51:21.365 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.365 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.365 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.365 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.365 INFO:teuthology.orchestra.run.smithi086.stdout: "retention_period": { 2024-04-21T08:51:21.365 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retention_period", 2024-04-21T08:51:21.365 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:21.366 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.366 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.366 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15552000", 2024-04-21T08:51:21.366 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.366 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.366 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.366 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain device health metrics", 2024-04-21T08:51:21.366 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.366 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.366 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.366 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.367 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_frequency": { 2024-04-21T08:51:21.367 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_frequency", 2024-04-21T08:51:21.367 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:21.367 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.367 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.367 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-04-21T08:51:21.367 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.367 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.367 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.367 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-21T08:51:21.367 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.367 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.368 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.368 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.368 INFO:teuthology.orchestra.run.smithi086.stdout: "self_heal": { 2024-04-21T08:51:21.368 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "self_heal", 2024-04-21T08:51:21.368 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.368 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.368 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.368 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.368 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.368 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.368 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.368 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-21T08:51:21.369 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.369 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.369 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.369 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.369 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-21T08:51:21.369 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-21T08:51:21.369 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:21.369 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.369 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.369 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-21T08:51:21.369 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.369 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.369 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.370 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and check device health", 2024-04-21T08:51:21.370 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.370 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.370 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.370 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.370 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_threshold": { 2024-04-21T08:51:21.370 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_threshold", 2024-04-21T08:51:21.370 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:21.370 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.370 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.370 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7257600", 2024-04-21T08:51:21.371 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.371 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.371 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.371 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-21T08:51:21.371 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.371 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.371 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.371 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.371 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.371 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.371 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.371 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "diskprediction_local", 2024-04-21T08:51:21.372 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.372 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.372 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.372 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.372 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.372 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.372 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.372 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.372 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.372 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.372 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.373 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.373 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.373 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.373 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.373 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.373 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.373 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.373 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.373 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.373 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.373 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.373 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.374 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.374 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.374 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.374 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.374 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.374 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.374 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.374 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.374 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.374 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.374 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.374 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.375 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.375 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.375 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.375 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.375 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.375 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.375 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.375 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.375 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.375 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.376 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.376 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.376 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.376 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.376 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.376 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.376 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.376 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.376 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.376 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.376 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.376 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.377 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.377 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.377 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.377 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.377 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.377 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.377 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.377 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.377 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.377 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.377 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.377 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.378 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.378 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.378 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.378 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.378 INFO:teuthology.orchestra.run.smithi086.stdout: "predict_interval": { 2024-04-21T08:51:21.378 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predict_interval", 2024-04-21T08:51:21.378 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.378 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.378 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.378 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-04-21T08:51:21.378 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.378 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.379 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.379 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.379 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.379 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.379 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.379 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.379 INFO:teuthology.orchestra.run.smithi086.stdout: "predictor_model": { 2024-04-21T08:51:21.379 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predictor_model", 2024-04-21T08:51:21.379 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.379 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.379 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.379 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "prophetstor", 2024-04-21T08:51:21.380 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.380 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.380 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.380 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.380 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.380 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.380 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.380 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.380 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-21T08:51:21.380 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-21T08:51:21.380 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.380 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.381 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.381 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-21T08:51:21.381 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.381 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.381 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.381 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.381 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.381 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.381 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.381 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.381 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.382 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.382 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.382 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "influx", 2024-04-21T08:51:21.382 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": false, 2024-04-21T08:51:21.382 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "influxdb python module not found", 2024-04-21T08:51:21.382 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.382 INFO:teuthology.orchestra.run.smithi086.stdout: "batch_size": { 2024-04-21T08:51:21.382 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "batch_size", 2024-04-21T08:51:21.382 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.382 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.383 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.383 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5000", 2024-04-21T08:51:21.383 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.383 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.383 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.383 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-21T08:51:21.383 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.383 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.383 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.383 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.383 INFO:teuthology.orchestra.run.smithi086.stdout: "database": { 2024-04-21T08:51:21.384 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "database", 2024-04-21T08:51:21.384 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.384 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.384 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.384 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "ceph", 2024-04-21T08:51:21.384 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.384 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.384 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.384 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-21T08:51:21.384 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.384 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.384 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.385 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.385 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": { 2024-04-21T08:51:21.385 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hostname", 2024-04-21T08:51:21.385 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.385 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.385 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.385 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.385 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.385 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.385 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.385 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server hostname", 2024-04-21T08:51:21.386 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.386 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.386 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.386 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.386 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-21T08:51:21.386 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-21T08:51:21.386 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:21.386 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.386 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.386 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "30", 2024-04-21T08:51:21.387 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "5", 2024-04-21T08:51:21.387 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.387 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.387 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-21T08:51:21.387 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.387 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.387 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.387 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.387 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.387 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.387 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.387 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.388 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.388 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.388 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.388 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.388 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.388 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.388 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.388 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.388 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.388 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.388 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.389 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.389 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.389 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.389 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.389 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.389 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.389 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.389 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.389 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.389 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.389 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.389 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.389 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.390 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.390 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.390 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.390 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.390 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.390 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.390 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.390 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.390 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.390 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.390 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.390 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.391 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.391 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.391 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.391 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.391 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.391 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.391 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.391 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.391 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.391 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.391 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.391 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.391 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.392 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.392 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.392 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.392 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.392 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.392 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.392 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.392 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.392 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.392 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.392 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.392 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.393 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.393 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.393 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.393 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.393 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.393 INFO:teuthology.orchestra.run.smithi086.stdout: "password": { 2024-04-21T08:51:21.393 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "password", 2024-04-21T08:51:21.393 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.393 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.393 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.393 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.393 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.393 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.394 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.394 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "password of InfluxDB server user", 2024-04-21T08:51:21.394 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.394 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.394 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.394 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.394 INFO:teuthology.orchestra.run.smithi086.stdout: "port": { 2024-04-21T08:51:21.394 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "port", 2024-04-21T08:51:21.394 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.394 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.394 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.394 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8086", 2024-04-21T08:51:21.395 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.395 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.395 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.395 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server port", 2024-04-21T08:51:21.395 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.395 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.395 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.395 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.395 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-04-21T08:51:21.395 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-04-21T08:51:21.395 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.395 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.396 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.396 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "false", 2024-04-21T08:51:21.396 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.396 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.396 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.396 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-21T08:51:21.396 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.396 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.396 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.396 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.396 INFO:teuthology.orchestra.run.smithi086.stdout: "threads": { 2024-04-21T08:51:21.396 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threads", 2024-04-21T08:51:21.397 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.397 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.397 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.397 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-21T08:51:21.397 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-21T08:51:21.397 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "32", 2024-04-21T08:51:21.397 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.397 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-21T08:51:21.397 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.397 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.397 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.397 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.397 INFO:teuthology.orchestra.run.smithi086.stdout: "username": { 2024-04-21T08:51:21.398 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "username", 2024-04-21T08:51:21.398 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.398 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.398 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.398 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.398 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.398 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.398 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.398 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "username of InfluxDB server user", 2024-04-21T08:51:21.398 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.398 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.398 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.399 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.399 INFO:teuthology.orchestra.run.smithi086.stdout: "verify_ssl": { 2024-04-21T08:51:21.399 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "verify_ssl", 2024-04-21T08:51:21.399 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.399 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.399 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.399 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "true", 2024-04-21T08:51:21.399 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.399 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.399 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.399 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-21T08:51:21.400 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.400 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.400 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.400 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.400 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.400 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.400 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.400 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "insights", 2024-04-21T08:51:21.400 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.400 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.400 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.400 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.400 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.401 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.401 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.401 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.401 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.401 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.401 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.401 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.401 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.401 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.401 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.401 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.402 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.402 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.402 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.402 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.402 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.402 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.402 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.402 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.402 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.402 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.402 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.402 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.403 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.403 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.403 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.403 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.403 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.403 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.403 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.403 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.403 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.403 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.403 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.403 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.404 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.404 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.404 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.404 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.404 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.404 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.404 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.404 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.404 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.404 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.404 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.404 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.404 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.405 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.405 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.405 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.405 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.405 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.405 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.405 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.405 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.405 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.405 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.405 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.405 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.406 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.406 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.406 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.406 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.406 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.406 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.406 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.406 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.406 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.406 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.407 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.407 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "iostat", 2024-04-21T08:51:21.407 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.407 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.407 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.407 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.407 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.407 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.407 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.407 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.407 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.408 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.408 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.408 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.408 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.408 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.408 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.408 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.408 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.408 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.408 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.408 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.408 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.408 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.409 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.409 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.409 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.409 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.409 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.409 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.409 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.409 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.409 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.409 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.409 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.410 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.410 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.410 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.410 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.410 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.410 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.410 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.410 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.410 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.410 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.410 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.411 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.411 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.411 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.411 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.411 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.411 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.411 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.411 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.411 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.411 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.411 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.411 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.411 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.412 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.412 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.412 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.412 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.412 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.412 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.412 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.412 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.412 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.412 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.412 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.413 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.413 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.413 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.413 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.413 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.413 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.413 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.413 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.413 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "k8sevents", 2024-04-21T08:51:21.413 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.413 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.413 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.414 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_event_retention_days": { 2024-04-21T08:51:21.414 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ceph_event_retention_days", 2024-04-21T08:51:21.414 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.414 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.414 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.414 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7", 2024-04-21T08:51:21.414 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.414 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.414 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.414 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-21T08:51:21.414 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.414 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.415 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.415 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.415 INFO:teuthology.orchestra.run.smithi086.stdout: "config_check_secs": { 2024-04-21T08:51:21.415 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_check_secs", 2024-04-21T08:51:21.415 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.415 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.415 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.415 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-21T08:51:21.415 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "10", 2024-04-21T08:51:21.415 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.415 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.416 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-21T08:51:21.416 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.416 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.416 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.416 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.416 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.416 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.416 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.416 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.416 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.416 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.417 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.417 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.417 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.417 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.417 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.417 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.417 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.417 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.417 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.417 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.417 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.418 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.418 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.418 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.418 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.418 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.418 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.418 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.418 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.418 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.418 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.418 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.418 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.419 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.419 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.419 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.419 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.419 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.419 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.419 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.419 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.419 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.419 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.419 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.420 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.420 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.420 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.420 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.420 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.420 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.420 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.420 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.420 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.420 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.420 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.420 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.421 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.421 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.421 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.421 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.421 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.421 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.421 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.421 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.421 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.421 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.421 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.421 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.422 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.422 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.422 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.422 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.422 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.422 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.422 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.422 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.422 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.422 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "localpool", 2024-04-21T08:51:21.422 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.422 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.423 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.423 INFO:teuthology.orchestra.run.smithi086.stdout: "failure_domain": { 2024-04-21T08:51:21.423 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "failure_domain", 2024-04-21T08:51:21.423 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.423 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.423 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.423 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "host", 2024-04-21T08:51:21.423 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.423 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.423 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.423 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "failure domain for any created local pool", 2024-04-21T08:51:21.423 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-21T08:51:21.423 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.424 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.424 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.424 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.424 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.424 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.424 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.424 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.424 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.424 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.424 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.424 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.425 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.425 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.425 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.425 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.425 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.425 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.425 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.425 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.425 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.425 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.425 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.426 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.426 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.426 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.426 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.426 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.426 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.426 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.426 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.426 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.426 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.426 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.426 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.427 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.427 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.427 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.427 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.427 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.427 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.427 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.427 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.427 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.427 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.427 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.428 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.428 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.428 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.428 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.428 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.428 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.428 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.428 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.428 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.428 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.428 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.429 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.429 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.429 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.429 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.429 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.429 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.429 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.429 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.429 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.429 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.429 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.429 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.430 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.430 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.430 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.430 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.430 INFO:teuthology.orchestra.run.smithi086.stdout: "min_size": { 2024-04-21T08:51:21.430 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_size", 2024-04-21T08:51:21.430 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.430 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.430 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.430 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.430 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.430 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.431 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.431 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default min_size for any created local pool", 2024-04-21T08:51:21.431 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-21T08:51:21.431 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.431 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.431 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.431 INFO:teuthology.orchestra.run.smithi086.stdout: "num_rep": { 2024-04-21T08:51:21.431 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "num_rep", 2024-04-21T08:51:21.431 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.431 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.431 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.432 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3", 2024-04-21T08:51:21.432 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.432 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.432 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.432 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default replica count for any created local pool", 2024-04-21T08:51:21.432 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.432 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.432 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.432 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.432 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_num": { 2024-04-21T08:51:21.432 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_num", 2024-04-21T08:51:21.432 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.433 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.433 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.433 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "128", 2024-04-21T08:51:21.433 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.433 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.433 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.433 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default pg_num for any created local pool", 2024-04-21T08:51:21.433 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.433 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.433 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.433 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.433 INFO:teuthology.orchestra.run.smithi086.stdout: "prefix": { 2024-04-21T08:51:21.433 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prefix", 2024-04-21T08:51:21.434 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.434 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.434 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.434 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.434 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.434 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.434 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.434 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name prefix for any created local pool", 2024-04-21T08:51:21.434 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.434 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.434 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.434 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.435 INFO:teuthology.orchestra.run.smithi086.stdout: "subtree": { 2024-04-21T08:51:21.435 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "subtree", 2024-04-21T08:51:21.435 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.435 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.435 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.435 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "rack", 2024-04-21T08:51:21.435 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.435 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.435 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.435 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-21T08:51:21.435 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-21T08:51:21.435 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.436 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.436 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.436 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.436 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.436 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.436 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mds_autoscaler", 2024-04-21T08:51:21.436 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.436 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.436 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.436 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.436 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.436 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.436 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.437 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.437 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.437 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.437 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.437 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.437 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.437 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.437 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.437 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.437 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.437 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.437 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.438 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.438 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.438 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.438 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.438 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.438 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.438 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.438 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.438 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.438 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.438 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.438 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.438 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.439 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.439 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.439 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.439 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.439 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.439 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.439 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.439 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.439 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.439 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.439 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.439 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.440 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.440 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.440 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.440 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.440 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.440 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.440 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.440 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.440 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.440 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.440 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.440 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.440 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.441 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.441 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.441 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.441 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.441 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.441 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.441 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.441 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.441 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.441 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.441 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.441 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.442 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.442 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.442 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.442 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.442 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.442 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.442 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.442 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirroring", 2024-04-21T08:51:21.442 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.442 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.442 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.442 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.443 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.443 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.443 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.443 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.443 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.443 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.443 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.443 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.443 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.443 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.443 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.443 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.444 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.444 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.444 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.444 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.444 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.444 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.444 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.444 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.444 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.444 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.444 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.444 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.445 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.445 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.445 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.445 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.445 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.445 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.445 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.445 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.445 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.445 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.445 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.445 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.446 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.446 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.446 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.446 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.446 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.446 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.446 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.446 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.446 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.446 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.446 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.446 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.447 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.447 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.447 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.447 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.447 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.447 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.447 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.447 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.447 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.447 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.447 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.447 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.448 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.448 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.448 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.448 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.448 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.448 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.448 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.448 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.448 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.448 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.448 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.448 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.449 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "nfs", 2024-04-21T08:51:21.449 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.449 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.449 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.449 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.449 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.449 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.449 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.449 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.449 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.449 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.450 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.450 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.450 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.450 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.450 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.450 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.450 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.450 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.450 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.450 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.450 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.450 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.450 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.451 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.451 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.451 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.451 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.451 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.451 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.451 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.451 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.451 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.451 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.451 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.452 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.452 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.452 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.452 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.452 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.452 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.452 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.452 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.452 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.452 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.452 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.452 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.453 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.453 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.453 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.453 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.453 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.453 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.453 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.453 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.453 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.453 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.453 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.453 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.453 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.454 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.454 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.454 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.454 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.454 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.454 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.454 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.454 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.454 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.454 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.454 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.455 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.455 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.455 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.455 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.455 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.455 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.455 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-04-21T08:51:21.455 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.455 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.455 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.455 INFO:teuthology.orchestra.run.smithi086.stdout: "fail_fs": { 2024-04-21T08:51:21.455 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "fail_fs", 2024-04-21T08:51:21.455 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.456 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.456 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.456 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.456 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.456 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.456 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.456 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-21T08:51:21.456 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.456 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.456 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.456 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.456 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.457 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.457 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.457 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.457 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.457 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.457 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.457 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.457 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.457 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.457 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.457 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.458 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.458 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.458 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.458 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.458 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.458 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.458 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.458 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.458 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.458 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.458 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.458 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.459 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.459 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.459 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.459 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.459 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.459 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.459 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.459 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.459 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.459 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.459 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.459 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.460 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.460 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.460 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.460 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.460 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.460 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.460 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.460 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.460 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.460 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.461 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.461 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.461 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.461 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.461 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.461 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.461 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.461 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.461 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.461 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.461 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.461 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.462 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.462 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.462 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.462 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.462 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.462 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.462 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.462 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.462 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.462 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.462 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.463 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.463 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator": { 2024-04-21T08:51:21.463 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-04-21T08:51:21.463 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.463 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.463 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.463 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.463 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.463 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.463 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.463 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm", 2024-04-21T08:51:21.463 INFO:teuthology.orchestra.run.smithi086.stdout: "rook", 2024-04-21T08:51:21.463 INFO:teuthology.orchestra.run.smithi086.stdout: "test_orchestrator" 2024-04-21T08:51:21.464 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.464 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Orchestrator backend", 2024-04-21T08:51:21.464 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.464 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.464 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.464 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.464 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.464 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.464 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.464 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_perf_query", 2024-04-21T08:51:21.464 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.465 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.465 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.465 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.465 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.465 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.465 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.465 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.465 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.465 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.465 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.465 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.465 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.465 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.466 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.466 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.466 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.466 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.466 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.466 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.466 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.466 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.466 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.466 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.467 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.467 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.467 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.467 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.467 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.467 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.467 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.467 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.467 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.467 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.467 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.467 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.468 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.468 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.468 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.468 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.468 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.468 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.468 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.468 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.468 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.468 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.468 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.468 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.468 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.468 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.469 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.469 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.469 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.469 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.469 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.469 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.469 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.469 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.469 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.469 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.469 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.469 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.470 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.470 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.470 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.470 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.470 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.470 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.470 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.470 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.470 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.470 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.470 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.470 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.471 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.471 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.471 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_support", 2024-04-21T08:51:21.471 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.471 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.471 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.471 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.471 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.471 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.471 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.471 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.471 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.471 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.472 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.472 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.472 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.472 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.472 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.472 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.472 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.472 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.472 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.472 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.472 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.472 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.473 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.473 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.473 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.473 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.473 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.473 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.473 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.473 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.473 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.473 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.473 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.473 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.474 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.474 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.474 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.474 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.474 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.474 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.474 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.474 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.474 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.474 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.474 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.474 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.475 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.475 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.475 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.475 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.475 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.475 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.475 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.475 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.475 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.475 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.475 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.475 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.475 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.476 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.476 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.476 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.476 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.476 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.476 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.476 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.476 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.476 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.476 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.476 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.476 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.477 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.477 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.477 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.477 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.477 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.477 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_autoscaler", 2024-04-21T08:51:21.477 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.477 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.477 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.477 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.477 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.477 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.478 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.478 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.478 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.478 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.478 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.478 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.478 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.478 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.478 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.478 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.478 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.478 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.479 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.479 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.479 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.479 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.479 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.479 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.479 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.479 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.479 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.479 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.479 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.479 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.480 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.480 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.480 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.480 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.480 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.480 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.480 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.480 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.480 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.480 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.480 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.480 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.481 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.481 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.481 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.481 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.481 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.481 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.481 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.481 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.481 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.481 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.481 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.481 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.482 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.482 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.482 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.482 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.482 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.482 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.482 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.482 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.482 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.482 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.482 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.482 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.483 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.483 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.483 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.483 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.483 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.483 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.483 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.483 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-21T08:51:21.483 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-21T08:51:21.483 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:21.483 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.483 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.484 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-21T08:51:21.484 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.484 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.484 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.484 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.484 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.484 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.484 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.484 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.484 INFO:teuthology.orchestra.run.smithi086.stdout: "threshold": { 2024-04-21T08:51:21.484 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threshold", 2024-04-21T08:51:21.484 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-21T08:51:21.484 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.485 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.485 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-04-21T08:51:21.485 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1.0", 2024-04-21T08:51:21.485 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.485 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.485 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "scaling threshold", 2024-04-21T08:51:21.485 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-21T08:51:21.485 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.485 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.485 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.485 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.485 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.486 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.486 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "progress", 2024-04-21T08:51:21.486 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.486 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.486 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.486 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_pg_recovery_event": { 2024-04-21T08:51:21.486 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_pg_recovery_event", 2024-04-21T08:51:21.486 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.486 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.486 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.487 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.487 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.487 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.487 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.487 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-21T08:51:21.487 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.487 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.487 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.487 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.487 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-04-21T08:51:21.487 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-04-21T08:51:21.487 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.488 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.488 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.488 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.488 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.488 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.488 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.488 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.488 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.488 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.488 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.488 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.488 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.489 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.489 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.489 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.489 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.489 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.489 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.489 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.489 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.489 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.489 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.489 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.489 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.490 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.490 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.490 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.490 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.490 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.490 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.490 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.490 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.490 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.490 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.490 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.491 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.491 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.491 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.491 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.491 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.491 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.491 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.491 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.491 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.491 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.491 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.491 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.492 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.492 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.492 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.492 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.492 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.492 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.492 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.492 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.492 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.492 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.492 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.492 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.493 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.493 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.493 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.493 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.493 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.493 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.493 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.493 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.493 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.493 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.493 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.493 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.494 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.494 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.494 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.494 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.494 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.494 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.494 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.494 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.494 INFO:teuthology.orchestra.run.smithi086.stdout: "max_completed_events": { 2024-04-21T08:51:21.494 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_completed_events", 2024-04-21T08:51:21.494 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.495 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.495 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.495 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "50", 2024-04-21T08:51:21.495 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.495 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.495 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.495 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "number of past completed events to remember", 2024-04-21T08:51:21.495 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.495 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.495 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.495 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.495 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-21T08:51:21.495 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-21T08:51:21.496 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:21.496 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.496 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.496 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-21T08:51:21.496 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.496 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.496 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.496 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long the module is going to sleep", 2024-04-21T08:51:21.496 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.496 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.496 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.496 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.496 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.497 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.497 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus", 2024-04-21T08:51:21.497 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.497 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.497 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.497 INFO:teuthology.orchestra.run.smithi086.stdout: "cache": { 2024-04-21T08:51:21.497 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cache", 2024-04-21T08:51:21.497 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.497 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.497 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.497 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.498 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.498 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.498 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.498 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.498 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.498 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.498 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.498 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.498 INFO:teuthology.orchestra.run.smithi086.stdout: "exclude_perf_counters": { 2024-04-21T08:51:21.498 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "exclude_perf_counters", 2024-04-21T08:51:21.498 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.498 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.499 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.499 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.499 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.499 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.499 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.499 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-21T08:51:21.499 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-21T08:51:21.499 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.499 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.499 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.500 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.500 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.500 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.500 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.500 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.500 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.500 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.500 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.500 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.500 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.500 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.501 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.501 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.501 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.501 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.501 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.501 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.501 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.501 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.501 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.501 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.501 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.502 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.502 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.502 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.502 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.502 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.502 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.502 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.502 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.502 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.502 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.502 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.502 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.503 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.503 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.503 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.503 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.503 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.503 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.503 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.503 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.503 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.503 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.503 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.503 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.504 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.504 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.504 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.504 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.504 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.504 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.504 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.504 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.504 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.504 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.504 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.505 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.505 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.505 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.505 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.505 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.505 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.505 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.505 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.505 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.505 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.505 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.506 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools": { 2024-04-21T08:51:21.506 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools", 2024-04-21T08:51:21.506 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.506 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.506 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.506 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.506 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.506 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.506 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.506 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.506 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.507 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.507 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.507 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.507 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-21T08:51:21.507 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-21T08:51:21.507 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.507 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.507 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.507 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "300", 2024-04-21T08:51:21.507 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.507 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.507 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.507 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.508 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.508 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.508 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.508 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.508 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_interval": { 2024-04-21T08:51:21.508 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_interval", 2024-04-21T08:51:21.508 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-21T08:51:21.508 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.508 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.508 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15.0", 2024-04-21T08:51:21.508 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.508 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.509 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.509 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.509 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.509 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.509 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.509 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.509 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-04-21T08:51:21.509 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-04-21T08:51:21.509 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.509 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.509 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.509 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-04-21T08:51:21.510 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.510 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.510 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.510 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-21T08:51:21.510 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.510 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.510 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.510 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.510 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-04-21T08:51:21.510 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-04-21T08:51:21.510 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.510 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.511 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.511 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "9283", 2024-04-21T08:51:21.511 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.511 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.511 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.511 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-21T08:51:21.511 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.511 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.511 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.511 INFO:teuthology.orchestra.run.smithi086.stdout: "stale_cache_strategy": { 2024-04-21T08:51:21.511 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stale_cache_strategy", 2024-04-21T08:51:21.512 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.512 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.512 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.512 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "log", 2024-04-21T08:51:21.512 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.512 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.512 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.512 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.512 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.512 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.512 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.512 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-04-21T08:51:21.512 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-04-21T08:51:21.513 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.513 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.513 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.513 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "default", 2024-04-21T08:51:21.513 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.513 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.513 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.513 INFO:teuthology.orchestra.run.smithi086.stdout: "default", 2024-04-21T08:51:21.513 INFO:teuthology.orchestra.run.smithi086.stdout: "error" 2024-04-21T08:51:21.513 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.513 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.513 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.513 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.514 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.514 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.514 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-04-21T08:51:21.514 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-04-21T08:51:21.514 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.514 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.514 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.514 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-04-21T08:51:21.514 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-04-21T08:51:21.514 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-04-21T08:51:21.514 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.514 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.514 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.515 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.515 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.515 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.515 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.515 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.515 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_support", 2024-04-21T08:51:21.515 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.515 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.515 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.515 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.515 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.515 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.516 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.516 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.516 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.516 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.516 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.516 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.516 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.516 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.516 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.516 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.516 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.516 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.517 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.517 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.517 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.517 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.517 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.517 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.517 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.517 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.517 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.517 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.517 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.517 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.517 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.517 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.518 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.518 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.518 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.518 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.518 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.518 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.518 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.518 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.518 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.518 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.518 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.519 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.519 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.519 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.519 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.519 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.519 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.519 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.519 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.519 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.519 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.519 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.520 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.520 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.520 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.520 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.520 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.520 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.520 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.520 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.520 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.520 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.520 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.520 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.521 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.521 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.521 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.521 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.521 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.521 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.521 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_snap_create": { 2024-04-21T08:51:21.521 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_snap_create", 2024-04-21T08:51:21.521 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.521 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.521 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.522 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-21T08:51:21.522 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.522 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.522 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.522 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.522 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.522 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.522 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.522 INFO:teuthology.orchestra.run.smithi086.stdout: "mirror_snapshot_schedule": { 2024-04-21T08:51:21.522 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirror_snapshot_schedule", 2024-04-21T08:51:21.522 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.523 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.523 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.523 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.523 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.523 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.523 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.523 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.523 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.523 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.523 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.523 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.523 INFO:teuthology.orchestra.run.smithi086.stdout: "trash_purge_schedule": { 2024-04-21T08:51:21.524 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "trash_purge_schedule", 2024-04-21T08:51:21.524 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.524 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.524 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.524 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.524 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.524 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.524 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.524 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.524 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.524 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.525 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.525 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.525 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.525 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.525 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.525 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "restful", 2024-04-21T08:51:21.525 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.525 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.525 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.525 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_auth": { 2024-04-21T08:51:21.525 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_auth", 2024-04-21T08:51:21.525 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.525 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.526 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.526 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.526 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.526 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.526 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.526 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.526 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.526 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.526 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.526 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-04-21T08:51:21.527 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-04-21T08:51:21.527 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.527 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.527 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.527 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.527 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.527 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.527 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.527 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.527 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.527 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.527 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.528 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.528 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.528 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.528 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.528 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.528 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.528 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.528 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.528 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.528 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.528 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.529 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.529 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.529 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.529 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.529 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.529 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.529 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.529 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.529 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.529 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.529 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.529 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.530 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.530 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.530 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.530 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.530 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.530 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.530 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.530 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.530 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.530 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.530 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.530 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.530 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.531 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.531 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.531 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.531 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.531 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.531 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.531 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.531 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.531 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.531 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.531 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.531 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.532 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.532 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.532 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.532 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.532 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.532 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.532 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.532 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.532 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.532 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.532 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.532 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.533 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.533 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.533 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.533 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.533 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.533 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.533 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.533 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.533 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.533 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.533 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-04-21T08:51:21.533 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-04-21T08:51:21.534 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.534 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.534 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.534 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.534 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.534 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.534 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.534 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.534 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.534 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.534 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.534 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.534 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-04-21T08:51:21.535 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-04-21T08:51:21.535 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.535 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.535 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.535 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.535 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.535 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.535 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.535 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.535 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.535 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.535 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.536 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.536 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.536 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.536 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.536 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rgw", 2024-04-21T08:51:21.536 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.536 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.536 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.536 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.536 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.536 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.536 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.537 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.537 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.537 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.537 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.537 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.537 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.537 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.537 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.537 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.537 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.537 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.537 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.537 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.538 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.538 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.538 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.538 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.538 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.538 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.538 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.538 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.538 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.538 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.538 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.538 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.539 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.539 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.539 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.539 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.539 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.539 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.539 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.539 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.539 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.539 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.539 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.539 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.540 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.540 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.540 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.540 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.540 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.540 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.540 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.540 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.540 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.540 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.540 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.540 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.541 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.541 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.541 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.541 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.541 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.541 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.541 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.541 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.541 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.541 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.541 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.542 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.542 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.542 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.542 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.542 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.542 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.542 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.542 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.542 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rook", 2024-04-21T08:51:21.542 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.542 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.542 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.543 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.543 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.543 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.543 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.543 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.543 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.543 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.543 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.543 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.543 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.543 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.543 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.543 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.544 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.544 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.544 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.544 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.544 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.544 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.544 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.544 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.544 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.544 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.544 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.544 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.545 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.545 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.545 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.545 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.545 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.545 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.545 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.545 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.545 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.545 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.545 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.546 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.546 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.546 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.546 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.546 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.546 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.546 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.546 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.546 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.546 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.547 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.547 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.547 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.547 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.547 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.547 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.547 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.547 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.547 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.548 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.548 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.548 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.548 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.548 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.548 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.548 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.548 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.548 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.548 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.548 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.548 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.549 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.549 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.549 INFO:teuthology.orchestra.run.smithi086.stdout: "storage_class": { 2024-04-21T08:51:21.549 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "storage_class", 2024-04-21T08:51:21.549 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.549 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.549 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.549 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "local", 2024-04-21T08:51:21.549 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.549 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.549 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.550 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-21T08:51:21.550 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.550 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.550 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.550 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.550 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.550 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.550 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.550 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "selftest", 2024-04-21T08:51:21.550 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.550 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.551 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.551 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.551 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.551 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.551 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.551 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.551 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.551 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.551 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.551 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.551 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.551 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.552 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.552 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.552 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.552 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.552 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.552 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.552 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.552 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.552 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.552 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.552 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.553 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.553 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.553 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.553 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.553 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.553 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.553 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.553 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.553 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.553 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.553 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.553 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.554 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.554 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.554 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.554 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.554 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.554 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.554 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.554 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.554 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.554 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.555 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.555 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.555 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.555 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.555 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.555 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.555 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.555 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.555 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.555 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.555 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.555 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.556 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.556 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.556 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.556 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.556 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.556 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.556 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.556 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.556 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.556 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.556 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.556 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.556 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.557 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.557 INFO:teuthology.orchestra.run.smithi086.stdout: "roption1": { 2024-04-21T08:51:21.557 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption1", 2024-04-21T08:51:21.557 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.557 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.557 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.557 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.557 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.557 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.557 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.557 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.557 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.558 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.558 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.558 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.558 INFO:teuthology.orchestra.run.smithi086.stdout: "roption2": { 2024-04-21T08:51:21.558 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption2", 2024-04-21T08:51:21.558 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.558 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.558 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.558 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "xyz", 2024-04-21T08:51:21.558 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.558 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.559 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.559 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.559 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.559 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.559 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.559 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.559 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption1": { 2024-04-21T08:51:21.559 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption1", 2024-04-21T08:51:21.559 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.559 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.559 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.559 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.559 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.560 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.560 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.560 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.560 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.560 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.560 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.560 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption2": { 2024-04-21T08:51:21.560 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption2", 2024-04-21T08:51:21.560 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.560 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.561 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.561 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.561 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.561 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.561 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.561 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.561 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.561 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.561 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.561 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.561 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption3": { 2024-04-21T08:51:21.561 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption3", 2024-04-21T08:51:21.562 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-21T08:51:21.562 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.562 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.562 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.562 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.562 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.562 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.562 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.562 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.562 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.562 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.563 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.563 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption4": { 2024-04-21T08:51:21.563 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption4", 2024-04-21T08:51:21.563 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.563 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.563 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.563 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.563 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.563 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.563 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.563 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.564 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.564 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.564 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.564 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.564 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption5": { 2024-04-21T08:51:21.564 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption5", 2024-04-21T08:51:21.564 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.564 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.564 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.564 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.564 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.564 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.564 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.565 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.565 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.565 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.565 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.565 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.565 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption6": { 2024-04-21T08:51:21.565 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption6", 2024-04-21T08:51:21.565 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.565 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.565 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.565 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.565 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.566 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.566 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.566 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.566 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.566 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.566 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.566 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.566 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption7": { 2024-04-21T08:51:21.566 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption7", 2024-04-21T08:51:21.566 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.566 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.567 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.567 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.567 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-21T08:51:21.567 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "42", 2024-04-21T08:51:21.567 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.567 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.567 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.567 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.567 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.567 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.567 INFO:teuthology.orchestra.run.smithi086.stdout: "testkey": { 2024-04-21T08:51:21.567 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testkey", 2024-04-21T08:51:21.568 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.568 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.568 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.568 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.568 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.568 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.568 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.568 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.568 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.568 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.569 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.569 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.569 INFO:teuthology.orchestra.run.smithi086.stdout: "testlkey": { 2024-04-21T08:51:21.569 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testlkey", 2024-04-21T08:51:21.569 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.569 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.569 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.569 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.569 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.569 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.569 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.569 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.569 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.570 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.570 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.570 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.570 INFO:teuthology.orchestra.run.smithi086.stdout: "testnewline": { 2024-04-21T08:51:21.570 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testnewline", 2024-04-21T08:51:21.570 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.570 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.570 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.570 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.570 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.570 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.570 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.571 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.571 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.571 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.571 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.571 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.571 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.571 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.571 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.571 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snap_schedule", 2024-04-21T08:51:21.571 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.571 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.571 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.571 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_m_granularity": { 2024-04-21T08:51:21.572 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_m_granularity", 2024-04-21T08:51:21.572 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.572 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.572 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.572 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.572 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.572 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.572 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.572 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow minute scheduled snapshots", 2024-04-21T08:51:21.572 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.572 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.572 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.573 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.573 INFO:teuthology.orchestra.run.smithi086.stdout: "dump_on_update": { 2024-04-21T08:51:21.573 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dump_on_update", 2024-04-21T08:51:21.573 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.573 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.573 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.573 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.573 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.573 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.573 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.573 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "dump database to debug log on update", 2024-04-21T08:51:21.573 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.574 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.574 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.574 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.574 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.574 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.574 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.574 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.574 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.574 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.574 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.574 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.574 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.575 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.575 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.575 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.575 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.575 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.575 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.575 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.575 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.575 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.575 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.575 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.575 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.576 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.576 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.576 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.576 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.576 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.576 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.576 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.576 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.576 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.576 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.576 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.577 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.577 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.577 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.577 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.577 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.577 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.577 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.577 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.577 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.577 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.577 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.577 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.578 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.578 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.578 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.578 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.578 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.578 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.578 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.578 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.578 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.578 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.578 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.579 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.579 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.579 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.579 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.579 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.579 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.579 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.579 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.579 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.579 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.579 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.579 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.580 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.580 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.580 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.580 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.580 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.580 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stats", 2024-04-21T08:51:21.580 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.580 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.580 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.580 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.580 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.580 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.580 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.581 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.581 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.581 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.581 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.581 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.581 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.581 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.581 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.581 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.581 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.581 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.581 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.582 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.582 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.582 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.582 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.582 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.582 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.582 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.582 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.582 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.582 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.582 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.583 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.583 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.583 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.583 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.583 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.583 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.583 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.583 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.583 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.583 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.583 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.583 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.584 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.584 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.584 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.584 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.584 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.584 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.584 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.584 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.584 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.584 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.584 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.585 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.586 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.586 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.586 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.586 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.586 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.586 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.586 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.586 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.586 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "status", 2024-04-21T08:51:21.586 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.586 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.586 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.587 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.587 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.587 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.587 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.587 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.587 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.587 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.587 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.587 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.587 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.587 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.587 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.588 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.588 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.588 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.588 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.588 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.588 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.588 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.588 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.588 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.588 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.588 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.589 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.589 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.589 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.589 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.589 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.589 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.589 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.589 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.589 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.589 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.589 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.589 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.589 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.590 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.590 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.590 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.590 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.590 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.590 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.590 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.590 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.590 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.590 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.591 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.591 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.591 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.591 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.591 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.591 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.591 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.591 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.591 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.591 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.591 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.591 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.592 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.592 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.592 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.592 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.592 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.592 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.592 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.592 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.592 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.592 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.593 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.593 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.593 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.593 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.593 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telegraf", 2024-04-21T08:51:21.593 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.593 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.593 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.593 INFO:teuthology.orchestra.run.smithi086.stdout: "address": { 2024-04-21T08:51:21.593 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "address", 2024-04-21T08:51:21.593 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.594 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.594 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.594 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-21T08:51:21.594 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.594 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.594 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.594 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.594 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.594 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.594 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.594 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.595 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-21T08:51:21.595 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-21T08:51:21.595 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:21.595 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.595 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.595 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15", 2024-04-21T08:51:21.595 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.595 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.595 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.595 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.595 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.595 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.595 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.596 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.596 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.596 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.596 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.596 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.596 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.596 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.596 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.596 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.596 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.596 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.596 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.596 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.597 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.597 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.597 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.597 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.597 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.597 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.597 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.597 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.597 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.597 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.597 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.598 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.598 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.598 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.598 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.598 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.598 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.598 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.598 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.598 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.598 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.598 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.598 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.598 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.599 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.599 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.599 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.599 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.599 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.599 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.599 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.599 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.599 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.599 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.599 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.599 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.599 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.600 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.600 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.600 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.600 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.600 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.600 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.600 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.600 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.600 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.600 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.600 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.601 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.601 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.601 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.601 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.601 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.601 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.601 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.601 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.601 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.601 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.601 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.601 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.601 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telemetry", 2024-04-21T08:51:21.602 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.602 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.602 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.602 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_basic": { 2024-04-21T08:51:21.602 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_basic", 2024-04-21T08:51:21.602 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.602 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.602 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.602 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.602 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.602 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.602 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.603 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-21T08:51:21.603 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.603 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.603 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.603 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.603 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_crash": { 2024-04-21T08:51:21.603 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_crash", 2024-04-21T08:51:21.603 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.603 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.603 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.603 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.603 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.604 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.604 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.604 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-21T08:51:21.604 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.604 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.604 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.604 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.604 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_device": { 2024-04-21T08:51:21.604 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_device", 2024-04-21T08:51:21.604 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.604 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.604 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.604 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.605 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.605 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.605 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.605 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-21T08:51:21.605 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.605 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.605 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.605 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_ident": { 2024-04-21T08:51:21.605 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_ident", 2024-04-21T08:51:21.605 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.605 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.606 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.606 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.606 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.606 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.606 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.606 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-21T08:51:21.606 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.606 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.606 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.606 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.606 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_perf": { 2024-04-21T08:51:21.607 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_perf", 2024-04-21T08:51:21.607 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.607 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.607 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.607 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.607 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.607 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.607 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.607 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-21T08:51:21.607 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.607 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.607 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.608 INFO:teuthology.orchestra.run.smithi086.stdout: "contact": { 2024-04-21T08:51:21.608 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "contact", 2024-04-21T08:51:21.608 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.608 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.608 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.608 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.608 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.608 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.608 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.608 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.608 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.608 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.609 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.609 INFO:teuthology.orchestra.run.smithi086.stdout: "description": { 2024-04-21T08:51:21.609 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "description", 2024-04-21T08:51:21.609 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.609 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.609 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.609 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.609 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.609 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.609 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.609 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.610 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.610 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.610 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.610 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.610 INFO:teuthology.orchestra.run.smithi086.stdout: "device_url": { 2024-04-21T08:51:21.610 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_url", 2024-04-21T08:51:21.610 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.610 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.610 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.610 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-21T08:51:21.610 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.610 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.611 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.611 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.611 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.611 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.611 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.611 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-04-21T08:51:21.611 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-04-21T08:51:21.611 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.611 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.611 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.611 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.611 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.612 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.612 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.612 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.612 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.612 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.612 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.612 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-21T08:51:21.612 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-21T08:51:21.612 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.612 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.612 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.612 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "24", 2024-04-21T08:51:21.613 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "8", 2024-04-21T08:51:21.613 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.613 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.613 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.613 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.613 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.613 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.613 INFO:teuthology.orchestra.run.smithi086.stdout: "last_opt_revision": { 2024-04-21T08:51:21.613 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "last_opt_revision", 2024-04-21T08:51:21.613 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.613 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.613 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.614 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1", 2024-04-21T08:51:21.614 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.614 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.614 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.614 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.614 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.614 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.614 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.614 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard": { 2024-04-21T08:51:21.615 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard", 2024-04-21T08:51:21.615 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.615 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.615 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.615 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.615 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.615 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.615 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.615 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.615 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.615 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.615 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.616 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard_description": { 2024-04-21T08:51:21.616 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard_description", 2024-04-21T08:51:21.616 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.616 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.616 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.616 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.616 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.616 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.616 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.616 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.616 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.616 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.617 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.617 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.617 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.617 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.617 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.617 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.617 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.617 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.617 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.617 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.617 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.617 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.617 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.618 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.618 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.618 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.618 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.618 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.618 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.618 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.618 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.618 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.618 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.618 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.618 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.619 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.619 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.619 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.619 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.619 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.619 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.619 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.619 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.619 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.619 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.619 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.619 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.619 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.620 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.620 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.620 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.620 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.620 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.620 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.620 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.620 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.620 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.620 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.620 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.621 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.621 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.621 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.621 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.621 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.621 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.621 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.621 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.621 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.621 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.622 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.622 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.622 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.622 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.622 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.622 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.622 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.622 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.622 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.622 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.622 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.622 INFO:teuthology.orchestra.run.smithi086.stdout: "organization": { 2024-04-21T08:51:21.623 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "organization", 2024-04-21T08:51:21.623 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.623 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.623 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.623 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.623 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.623 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.623 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.623 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.623 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.623 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.623 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.624 INFO:teuthology.orchestra.run.smithi086.stdout: "proxy": { 2024-04-21T08:51:21.624 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "proxy", 2024-04-21T08:51:21.624 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.624 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.624 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.624 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.624 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.624 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.624 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.624 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.624 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.624 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.625 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.625 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.625 INFO:teuthology.orchestra.run.smithi086.stdout: "url": { 2024-04-21T08:51:21.625 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url", 2024-04-21T08:51:21.625 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.625 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.625 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.625 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-21T08:51:21.625 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.625 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.626 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.626 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.626 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.626 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.626 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.626 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.626 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.626 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.626 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "test_orchestrator", 2024-04-21T08:51:21.627 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.627 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.627 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.627 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.627 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.627 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.627 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.627 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.627 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.627 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.628 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.628 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.628 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.628 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.628 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.628 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.628 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.628 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.628 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.628 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.628 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.628 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.629 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.629 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.629 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.629 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.629 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.629 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.629 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.629 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.629 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.629 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.629 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.630 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.630 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.630 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.630 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.630 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.630 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.630 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.630 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.630 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.630 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.630 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.631 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.631 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.631 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.631 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.631 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.631 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.631 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.631 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.631 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.631 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.631 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.631 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.632 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.632 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.632 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.632 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.632 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.632 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.632 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.632 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.632 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.632 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.632 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.632 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.633 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.633 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.633 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.633 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.633 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.633 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.633 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "volumes", 2024-04-21T08:51:21.633 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.633 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.633 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.633 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.634 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.634 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.634 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.634 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.634 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.634 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.634 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.634 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.634 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.634 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.634 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.634 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.635 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.635 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.635 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.635 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.635 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.635 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.635 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.635 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.635 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.635 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.635 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.636 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.636 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.636 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.636 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.636 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.636 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.636 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.636 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.636 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.636 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.636 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.637 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.637 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.637 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.637 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.637 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.637 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.637 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.637 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.637 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.637 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.637 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.637 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.638 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.638 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.638 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.638 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.638 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.638 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.638 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.638 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.638 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.638 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.638 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.639 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.639 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.639 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.639 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.639 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.639 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.639 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.639 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.639 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.639 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.639 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_clones": { 2024-04-21T08:51:21.640 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_clones", 2024-04-21T08:51:21.640 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.640 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.640 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.640 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4", 2024-04-21T08:51:21.640 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.640 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.640 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.640 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-21T08:51:21.640 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.640 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.640 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.641 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.641 INFO:teuthology.orchestra.run.smithi086.stdout: "periodic_async_work": { 2024-04-21T08:51:21.641 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "periodic_async_work", 2024-04-21T08:51:21.641 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.641 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.641 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.641 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.641 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.641 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.641 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.641 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Periodically check for async work", 2024-04-21T08:51:21.641 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.641 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.642 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.642 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.642 INFO:teuthology.orchestra.run.smithi086.stdout: "snapshot_clone_delay": { 2024-04-21T08:51:21.642 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snapshot_clone_delay", 2024-04-21T08:51:21.642 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.642 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.642 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.642 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-21T08:51:21.642 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.642 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.642 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.643 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-21T08:51:21.643 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.643 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.643 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.643 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.643 INFO:teuthology.orchestra.run.smithi086.stdout: "snapshot_clone_no_wait": { 2024-04-21T08:51:21.643 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snapshot_clone_no_wait", 2024-04-21T08:51:21.643 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.643 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.643 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.643 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.643 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.644 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.644 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.644 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-21T08:51:21.644 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.644 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.644 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.644 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.644 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.644 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.644 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix", 2024-04-21T08:51:21.644 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.644 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.645 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.645 INFO:teuthology.orchestra.run.smithi086.stdout: "discovery_interval": { 2024-04-21T08:51:21.645 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "discovery_interval", 2024-04-21T08:51:21.645 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-21T08:51:21.645 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.645 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.645 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "100", 2024-04-21T08:51:21.645 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.645 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.645 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.645 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.645 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.645 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.646 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.646 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.646 INFO:teuthology.orchestra.run.smithi086.stdout: "identifier": { 2024-04-21T08:51:21.646 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "identifier", 2024-04-21T08:51:21.646 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.646 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.646 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.646 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.646 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.646 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.646 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.646 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.647 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.647 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.647 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.647 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-21T08:51:21.647 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-21T08:51:21.647 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:21.647 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.647 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.647 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-21T08:51:21.647 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.647 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.647 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.648 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.648 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.648 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.648 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.648 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.648 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.648 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.648 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.648 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.648 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.649 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.649 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.649 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.649 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.649 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.649 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.649 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.649 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.649 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.650 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.650 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.650 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.650 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.650 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.650 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.650 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.650 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.650 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.650 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.651 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.651 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.651 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.651 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.651 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.651 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.651 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.651 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.651 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.651 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.651 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.651 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.652 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.652 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.652 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.652 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.652 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.652 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.652 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.652 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.652 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.652 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.652 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.652 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.653 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.653 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.653 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.653 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.653 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.653 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.653 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.653 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.653 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.653 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.653 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.653 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.654 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.654 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.654 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.654 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.654 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.654 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.654 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.654 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.654 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.654 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_host": { 2024-04-21T08:51:21.654 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_host", 2024-04-21T08:51:21.655 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.655 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.655 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.655 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.655 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.655 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.655 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.655 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.655 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.655 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.655 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.655 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.656 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_port": { 2024-04-21T08:51:21.656 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_port", 2024-04-21T08:51:21.656 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.656 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.656 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.656 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10051", 2024-04-21T08:51:21.656 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.656 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.656 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.656 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.656 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.657 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.657 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.657 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.657 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_sender": { 2024-04-21T08:51:21.657 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_sender", 2024-04-21T08:51:21.657 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.657 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.657 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.657 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-21T08:51:21.657 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.657 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.658 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.658 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.658 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.658 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.658 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.658 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.658 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.658 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.658 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-21T08:51:21.658 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.658 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.659 INFO:teuthology.orchestra.run.smithi086.stdout: "gid": 24965, 2024-04-21T08:51:21.659 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "y", 2024-04-21T08:51:21.659 INFO:teuthology.orchestra.run.smithi086.stdout: "mgr_features": 4540701547738038271, 2024-04-21T08:51:21.659 INFO:teuthology.orchestra.run.smithi086.stdout: "available_modules": [ 2024-04-21T08:51:21.659 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.659 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "alerts", 2024-04-21T08:51:21.659 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.659 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.659 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.659 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-21T08:51:21.659 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-21T08:51:21.659 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:21.660 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.660 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.660 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-21T08:51:21.660 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.660 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.660 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.660 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How frequently to reexamine health status", 2024-04-21T08:51:21.660 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.660 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.660 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.660 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.661 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.661 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.661 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.661 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.661 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.661 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.661 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.661 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.661 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.661 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.661 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.661 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.662 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.662 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.662 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.662 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.662 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.662 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.662 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.662 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.662 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.662 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.662 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.663 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.663 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.663 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.663 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.663 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.663 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.663 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.663 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.663 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.663 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.663 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.664 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.664 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.664 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.664 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.664 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.664 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.664 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.664 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.664 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.664 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.665 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.665 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.665 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.665 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.665 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.665 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.665 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.665 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.665 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.665 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.665 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.665 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.666 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.666 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.666 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.666 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.666 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.666 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.666 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.666 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.666 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.666 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.667 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.667 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.667 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_destination": { 2024-04-21T08:51:21.667 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_destination", 2024-04-21T08:51:21.667 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.667 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.667 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.667 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.667 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.667 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.667 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.667 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email address to send alerts to", 2024-04-21T08:51:21.668 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.668 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.668 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.668 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_from_name": { 2024-04-21T08:51:21.668 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_from_name", 2024-04-21T08:51:21.668 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.668 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.668 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.668 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "Ceph", 2024-04-21T08:51:21.668 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.668 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.668 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.669 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email From: name", 2024-04-21T08:51:21.669 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.669 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.669 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.669 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_host": { 2024-04-21T08:51:21.669 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_host", 2024-04-21T08:51:21.669 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.669 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.669 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.669 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.669 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.670 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.670 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.670 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP server", 2024-04-21T08:51:21.670 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.670 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.670 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.670 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.670 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_password": { 2024-04-21T08:51:21.670 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_password", 2024-04-21T08:51:21.670 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.670 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.670 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.671 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.671 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.671 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.671 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.671 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Password to authenticate with", 2024-04-21T08:51:21.671 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.671 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.671 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.671 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_port": { 2024-04-21T08:51:21.672 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_port", 2024-04-21T08:51:21.672 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.672 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.672 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.672 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "465", 2024-04-21T08:51:21.672 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.672 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.672 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.672 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP port", 2024-04-21T08:51:21.672 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.672 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.672 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.673 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.673 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_sender": { 2024-04-21T08:51:21.673 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_sender", 2024-04-21T08:51:21.673 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.673 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.673 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.673 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.673 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.673 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.673 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.673 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP envelope sender", 2024-04-21T08:51:21.673 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.674 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.674 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.674 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_ssl": { 2024-04-21T08:51:21.674 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_ssl", 2024-04-21T08:51:21.674 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.674 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.674 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.674 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.674 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.674 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.674 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.675 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-21T08:51:21.675 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.675 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.675 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.675 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_user": { 2024-04-21T08:51:21.675 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_user", 2024-04-21T08:51:21.675 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.675 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.675 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.675 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.676 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.676 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.676 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.676 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "User to authenticate as", 2024-04-21T08:51:21.676 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.676 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.676 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.676 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.676 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.676 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.676 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.677 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "balancer", 2024-04-21T08:51:21.677 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.677 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.677 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.677 INFO:teuthology.orchestra.run.smithi086.stdout: "active": { 2024-04-21T08:51:21.677 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "active", 2024-04-21T08:51:21.677 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.677 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.677 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.677 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.677 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.677 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.678 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.678 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically balance PGs across cluster", 2024-04-21T08:51:21.678 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.678 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.678 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.678 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.678 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_time": { 2024-04-21T08:51:21.678 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_time", 2024-04-21T08:51:21.678 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.678 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.678 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.679 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0000", 2024-04-21T08:51:21.679 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.679 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.679 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.679 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "beginning time of day to automatically balance", 2024-04-21T08:51:21.679 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-21T08:51:21.679 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.679 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.679 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.679 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_weekday": { 2024-04-21T08:51:21.679 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_weekday", 2024-04-21T08:51:21.679 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-21T08:51:21.680 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.680 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.680 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-21T08:51:21.680 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-04-21T08:51:21.680 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-04-21T08:51:21.680 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.680 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-21T08:51:21.680 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-21T08:51:21.680 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.680 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.680 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_max_iterations": { 2024-04-21T08:51:21.681 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_max_iterations", 2024-04-21T08:51:21.681 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-21T08:51:21.681 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.681 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.681 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "25", 2024-04-21T08:51:21.681 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-21T08:51:21.681 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "250", 2024-04-21T08:51:21.681 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.681 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-21T08:51:21.681 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.682 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.682 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.682 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.682 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_metrics": { 2024-04-21T08:51:21.682 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_metrics", 2024-04-21T08:51:21.682 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.682 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.682 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.682 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "pgs,objects,bytes", 2024-04-21T08:51:21.682 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.682 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.683 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.683 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-21T08:51:21.683 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-21T08:51:21.683 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.683 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.683 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.683 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_step": { 2024-04-21T08:51:21.683 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_step", 2024-04-21T08:51:21.683 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-21T08:51:21.683 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.683 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.683 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.5", 2024-04-21T08:51:21.684 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0.001", 2024-04-21T08:51:21.684 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "0.999", 2024-04-21T08:51:21.684 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.684 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "aggressiveness of optimization", 2024-04-21T08:51:21.684 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-21T08:51:21.684 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.684 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.684 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.684 INFO:teuthology.orchestra.run.smithi086.stdout: "end_time": { 2024-04-21T08:51:21.684 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_time", 2024-04-21T08:51:21.684 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.684 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.684 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.685 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2359", 2024-04-21T08:51:21.685 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.685 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.685 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.685 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ending time of day to automatically balance", 2024-04-21T08:51:21.685 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-21T08:51:21.685 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.685 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.685 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.685 INFO:teuthology.orchestra.run.smithi086.stdout: "end_weekday": { 2024-04-21T08:51:21.685 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_weekday", 2024-04-21T08:51:21.685 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-21T08:51:21.686 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.686 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.686 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-21T08:51:21.686 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-04-21T08:51:21.686 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-04-21T08:51:21.686 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.686 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-21T08:51:21.686 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-21T08:51:21.686 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.686 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.687 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.687 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.687 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.687 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.687 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.687 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.687 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.687 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.687 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.687 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.687 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.687 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.688 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.688 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.688 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.688 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.688 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.688 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.688 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.688 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.688 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.688 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.688 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.689 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.689 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.689 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.689 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.689 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.689 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.689 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.689 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.689 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.689 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.689 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.689 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.690 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.690 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.690 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.690 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.690 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.690 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.690 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.690 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.690 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.690 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.690 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.690 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.691 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.691 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.691 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.691 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.691 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.691 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.691 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.691 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.691 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.691 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.691 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.692 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.692 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.692 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.692 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.692 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.692 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.692 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.692 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.692 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.692 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.692 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.693 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.693 INFO:teuthology.orchestra.run.smithi086.stdout: "min_score": { 2024-04-21T08:51:21.693 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_score", 2024-04-21T08:51:21.693 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-21T08:51:21.693 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.693 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.693 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-21T08:51:21.693 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.693 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.693 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.693 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-21T08:51:21.693 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.694 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.694 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.694 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.694 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-04-21T08:51:21.694 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-04-21T08:51:21.694 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.694 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.694 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.694 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "upmap", 2024-04-21T08:51:21.694 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.695 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.695 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.695 INFO:teuthology.orchestra.run.smithi086.stdout: "crush-compat", 2024-04-21T08:51:21.695 INFO:teuthology.orchestra.run.smithi086.stdout: "none", 2024-04-21T08:51:21.695 INFO:teuthology.orchestra.run.smithi086.stdout: "read", 2024-04-21T08:51:21.695 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap", 2024-04-21T08:51:21.695 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap-read" 2024-04-21T08:51:21.695 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.695 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Balancer mode", 2024-04-21T08:51:21.695 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.695 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.696 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.696 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.696 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_ids": { 2024-04-21T08:51:21.696 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_ids", 2024-04-21T08:51:21.696 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.696 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.696 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.696 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.696 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.696 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.696 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.696 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-21T08:51:21.696 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.697 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.697 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.697 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.697 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-21T08:51:21.697 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-21T08:51:21.697 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:21.697 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.697 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.697 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-21T08:51:21.697 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.698 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.698 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.698 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-21T08:51:21.698 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.698 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.698 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.698 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.698 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_deviation": { 2024-04-21T08:51:21.698 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_deviation", 2024-04-21T08:51:21.698 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.698 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.699 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.699 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-21T08:51:21.699 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-21T08:51:21.699 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.699 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.699 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-21T08:51:21.699 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-21T08:51:21.699 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.699 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.699 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.699 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_optimizations": { 2024-04-21T08:51:21.699 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_optimizations", 2024-04-21T08:51:21.699 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-21T08:51:21.700 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.700 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.700 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-21T08:51:21.700 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.700 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.700 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.700 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-21T08:51:21.700 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.700 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.700 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.700 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.700 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.700 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.701 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.701 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cephadm", 2024-04-21T08:51:21.701 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.701 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.701 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.701 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_down_multiplier": { 2024-04-21T08:51:21.701 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_down_multiplier", 2024-04-21T08:51:21.701 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-21T08:51:21.701 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.701 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.701 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-04-21T08:51:21.701 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.702 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.702 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.702 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-21T08:51:21.702 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.702 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.702 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.702 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.702 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_refresh_rate": { 2024-04-21T08:51:21.702 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_refresh_rate", 2024-04-21T08:51:21.702 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:21.702 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.702 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.703 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "20", 2024-04-21T08:51:21.703 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.703 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.703 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.703 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-21T08:51:21.703 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.703 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.703 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.703 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.703 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_starting_port": { 2024-04-21T08:51:21.704 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_starting_port", 2024-04-21T08:51:21.704 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.704 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.704 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.704 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4721", 2024-04-21T08:51:21.704 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.704 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.704 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.704 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-21T08:51:21.704 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.704 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.704 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.704 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.705 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_ptrace": { 2024-04-21T08:51:21.705 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_ptrace", 2024-04-21T08:51:21.705 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.705 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.705 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.705 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.705 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.705 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.705 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.705 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-21T08:51:21.705 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-21T08:51:21.705 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.706 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.706 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.706 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_interval": { 2024-04-21T08:51:21.706 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_interval", 2024-04-21T08:51:21.706 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:21.706 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.706 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.706 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-21T08:51:21.706 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.706 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.706 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.706 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-21T08:51:21.707 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.707 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.707 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.707 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.707 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_memory_target_ratio": { 2024-04-21T08:51:21.707 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_memory_target_ratio", 2024-04-21T08:51:21.707 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-21T08:51:21.707 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.707 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.707 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.7", 2024-04-21T08:51:21.707 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.707 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.708 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.708 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-21T08:51:21.708 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.708 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.708 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.708 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.708 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm_log_destination": { 2024-04-21T08:51:21.708 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cephadm_log_destination", 2024-04-21T08:51:21.708 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.708 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.708 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.708 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.709 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.709 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.709 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.709 INFO:teuthology.orchestra.run.smithi086.stdout: "file", 2024-04-21T08:51:21.709 INFO:teuthology.orchestra.run.smithi086.stdout: "file,syslog", 2024-04-21T08:51:21.709 INFO:teuthology.orchestra.run.smithi086.stdout: "syslog" 2024-04-21T08:51:21.709 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.709 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-21T08:51:21.709 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.710 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.710 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.710 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.710 INFO:teuthology.orchestra.run.smithi086.stdout: "cgroups_split": { 2024-04-21T08:51:21.710 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cgroups_split", 2024-04-21T08:51:21.710 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.710 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.710 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.710 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.710 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.711 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.711 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.711 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-21T08:51:21.711 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.711 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.711 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.711 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.711 INFO:teuthology.orchestra.run.smithi086.stdout: "config_checks_enabled": { 2024-04-21T08:51:21.711 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_checks_enabled", 2024-04-21T08:51:21.711 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.711 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.712 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.712 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.712 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.712 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.712 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.712 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-21T08:51:21.712 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.712 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.712 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.712 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.712 INFO:teuthology.orchestra.run.smithi086.stdout: "config_dashboard": { 2024-04-21T08:51:21.712 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_dashboard", 2024-04-21T08:51:21.713 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.713 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.713 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.713 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.713 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.713 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.713 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.713 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-21T08:51:21.713 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.713 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.713 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.713 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.713 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_alertmanager": { 2024-04-21T08:51:21.713 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_alertmanager", 2024-04-21T08:51:21.714 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.714 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.714 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.714 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-21T08:51:21.714 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.714 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.714 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.714 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-21T08:51:21.714 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.714 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.714 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.715 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.715 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_base": { 2024-04-21T08:51:21.715 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_base", 2024-04-21T08:51:21.715 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.715 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.715 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.715 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-21T08:51:21.715 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.715 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.715 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.715 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Container image name, without the tag", 2024-04-21T08:51:21.715 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.716 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.716 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.716 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.716 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_elasticsearch": { 2024-04-21T08:51:21.716 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_elasticsearch", 2024-04-21T08:51:21.716 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.716 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.716 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.716 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-21T08:51:21.716 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.716 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.717 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.717 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "elasticsearch container image", 2024-04-21T08:51:21.717 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.717 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.717 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.717 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.717 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_grafana": { 2024-04-21T08:51:21.717 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_grafana", 2024-04-21T08:51:21.717 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.717 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.717 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.717 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-21T08:51:21.717 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.717 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.718 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.718 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-21T08:51:21.718 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.718 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.718 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.718 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.718 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_haproxy": { 2024-04-21T08:51:21.718 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_haproxy", 2024-04-21T08:51:21.718 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.718 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.718 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.719 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-21T08:51:21.719 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.719 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.719 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.719 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "HAproxy container image", 2024-04-21T08:51:21.719 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.719 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.719 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.719 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.719 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_agent": { 2024-04-21T08:51:21.719 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_agent", 2024-04-21T08:51:21.719 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.720 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.720 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.720 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-21T08:51:21.720 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.720 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.720 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.720 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger agent container image", 2024-04-21T08:51:21.720 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.720 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.720 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.720 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.720 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_collector": { 2024-04-21T08:51:21.720 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_collector", 2024-04-21T08:51:21.720 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.721 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.721 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.721 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-21T08:51:21.721 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.721 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.721 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.721 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger collector container image", 2024-04-21T08:51:21.721 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.721 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.721 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.721 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.721 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_query": { 2024-04-21T08:51:21.721 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_query", 2024-04-21T08:51:21.722 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.722 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.722 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.722 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-21T08:51:21.722 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.722 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.722 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.722 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger query container image", 2024-04-21T08:51:21.722 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.722 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.722 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.722 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.723 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_keepalived": { 2024-04-21T08:51:21.723 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_keepalived", 2024-04-21T08:51:21.723 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.723 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.723 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.723 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-21T08:51:21.723 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.723 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.723 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.723 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Keepalived container image", 2024-04-21T08:51:21.723 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.723 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.724 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.724 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.724 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_loki": { 2024-04-21T08:51:21.724 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_loki", 2024-04-21T08:51:21.724 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.724 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.724 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.724 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-21T08:51:21.724 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.725 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.725 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.725 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Loki container image", 2024-04-21T08:51:21.725 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.725 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.725 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.725 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.725 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_node_exporter": { 2024-04-21T08:51:21.725 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_node_exporter", 2024-04-21T08:51:21.725 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.725 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.725 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.726 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-21T08:51:21.726 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.726 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.726 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.726 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-21T08:51:21.726 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.726 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.726 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.726 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.726 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_nvmeof": { 2024-04-21T08:51:21.726 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_nvmeof", 2024-04-21T08:51:21.727 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.727 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.727 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.727 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-21T08:51:21.727 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.727 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.727 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.727 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Nvme-of container image", 2024-04-21T08:51:21.727 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.727 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.727 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.727 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.728 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_prometheus": { 2024-04-21T08:51:21.728 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_prometheus", 2024-04-21T08:51:21.728 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.728 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.728 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.728 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-21T08:51:21.728 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.728 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.728 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.728 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-21T08:51:21.728 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.729 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.729 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.729 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.729 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_promtail": { 2024-04-21T08:51:21.729 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_promtail", 2024-04-21T08:51:21.729 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.729 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.729 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.729 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-21T08:51:21.729 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.729 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.730 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.730 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Promtail container image", 2024-04-21T08:51:21.730 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.730 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.730 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.730 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.730 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_samba": { 2024-04-21T08:51:21.730 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_samba", 2024-04-21T08:51:21.730 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.730 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.730 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.731 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-21T08:51:21.731 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.731 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.731 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.731 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Samba/SMB container image", 2024-04-21T08:51:21.731 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.731 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.731 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.731 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.731 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_snmp_gateway": { 2024-04-21T08:51:21.732 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_snmp_gateway", 2024-04-21T08:51:21.732 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.732 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.732 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.732 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-21T08:51:21.732 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.732 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.732 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.732 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SNMP Gateway container image", 2024-04-21T08:51:21.732 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.732 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.732 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.732 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.733 INFO:teuthology.orchestra.run.smithi086.stdout: "container_init": { 2024-04-21T08:51:21.733 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_init", 2024-04-21T08:51:21.733 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.733 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.733 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.733 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.733 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.733 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.733 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.733 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Run podman/docker with `--init`", 2024-04-21T08:51:21.734 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.734 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.734 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.734 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.734 INFO:teuthology.orchestra.run.smithi086.stdout: "daemon_cache_timeout": { 2024-04-21T08:51:21.734 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "daemon_cache_timeout", 2024-04-21T08:51:21.734 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:21.734 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.734 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.734 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-21T08:51:21.734 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.735 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.735 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.735 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-21T08:51:21.735 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.735 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.735 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.735 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.735 INFO:teuthology.orchestra.run.smithi086.stdout: "default_cephadm_command_timeout": { 2024-04-21T08:51:21.735 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_cephadm_command_timeout", 2024-04-21T08:51:21.735 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.735 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.736 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.736 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "900", 2024-04-21T08:51:21.736 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.736 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.736 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.736 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-21T08:51:21.736 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.736 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.736 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.736 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.736 INFO:teuthology.orchestra.run.smithi086.stdout: "default_registry": { 2024-04-21T08:51:21.736 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_registry", 2024-04-21T08:51:21.737 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.737 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.737 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.737 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io", 2024-04-21T08:51:21.737 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.737 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.737 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.737 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-21T08:51:21.737 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.737 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.737 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.737 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.737 INFO:teuthology.orchestra.run.smithi086.stdout: "device_cache_timeout": { 2024-04-21T08:51:21.738 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_cache_timeout", 2024-04-21T08:51:21.738 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:21.738 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.738 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.738 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1800", 2024-04-21T08:51:21.738 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.738 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.738 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.738 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache device inventory", 2024-04-21T08:51:21.739 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.739 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.739 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.739 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.739 INFO:teuthology.orchestra.run.smithi086.stdout: "device_enhanced_scan": { 2024-04-21T08:51:21.739 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_enhanced_scan", 2024-04-21T08:51:21.739 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.739 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.739 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.739 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.739 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.739 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.739 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.740 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-21T08:51:21.740 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.740 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.740 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.740 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.740 INFO:teuthology.orchestra.run.smithi086.stdout: "facts_cache_timeout": { 2024-04-21T08:51:21.740 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "facts_cache_timeout", 2024-04-21T08:51:21.740 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:21.740 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.740 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.740 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-21T08:51:21.740 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.740 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.741 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.741 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache host facts data", 2024-04-21T08:51:21.741 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.741 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.741 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.741 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.741 INFO:teuthology.orchestra.run.smithi086.stdout: "grafana_dashboards_path": { 2024-04-21T08:51:21.741 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "grafana_dashboards_path", 2024-04-21T08:51:21.741 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.741 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.741 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.741 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-21T08:51:21.742 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.742 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.742 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.742 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-21T08:51:21.742 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.742 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.742 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.742 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.742 INFO:teuthology.orchestra.run.smithi086.stdout: "host_check_interval": { 2024-04-21T08:51:21.742 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "host_check_interval", 2024-04-21T08:51:21.742 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:21.742 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.743 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.743 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-21T08:51:21.743 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.743 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.743 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.743 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to perform a host check", 2024-04-21T08:51:21.743 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.743 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.743 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.743 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.743 INFO:teuthology.orchestra.run.smithi086.stdout: "hw_monitoring": { 2024-04-21T08:51:21.743 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hw_monitoring", 2024-04-21T08:51:21.743 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.744 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.744 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.744 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.744 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.744 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.744 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.744 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-21T08:51:21.744 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.744 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.744 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.744 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.744 INFO:teuthology.orchestra.run.smithi086.stdout: "inventory_list_all": { 2024-04-21T08:51:21.744 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "inventory_list_all", 2024-04-21T08:51:21.745 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.745 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.745 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.745 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.745 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.745 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.745 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.745 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-21T08:51:21.745 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.745 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.745 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.745 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.746 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.746 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.746 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.746 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.746 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.746 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.746 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.746 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.746 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.746 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.746 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.746 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.747 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.747 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.747 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.747 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.747 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.747 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.747 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.747 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.747 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.747 INFO:teuthology.orchestra.run.smithi086.stdout: "log_refresh_metadata": { 2024-04-21T08:51:21.747 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_refresh_metadata", 2024-04-21T08:51:21.747 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.748 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.748 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.748 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.748 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.748 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.748 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.748 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-21T08:51:21.748 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.748 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.748 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.748 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.748 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.749 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.749 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.749 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.749 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.749 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.749 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.749 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.749 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.749 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-21T08:51:21.749 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.749 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.749 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.749 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.750 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.750 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.750 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.750 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.750 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.750 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.750 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.750 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.750 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.750 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.750 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.751 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.751 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.751 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.751 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.751 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.751 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.751 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.751 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.751 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.751 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.751 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.751 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.751 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.752 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.752 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.752 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.752 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.752 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.752 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.752 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.752 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.752 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.752 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.752 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.752 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-21T08:51:21.752 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-21T08:51:21.753 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.753 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.753 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.753 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.753 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.753 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.753 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.753 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-21T08:51:21.753 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.753 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.753 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.753 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.754 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-21T08:51:21.754 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-21T08:51:21.754 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.754 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.754 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.754 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "*", 2024-04-21T08:51:21.754 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.754 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.754 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.754 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-21T08:51:21.754 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.755 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.755 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.755 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.755 INFO:teuthology.orchestra.run.smithi086.stdout: "max_count_per_host": { 2024-04-21T08:51:21.755 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_count_per_host", 2024-04-21T08:51:21.755 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.755 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.755 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.755 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-21T08:51:21.755 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.755 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.755 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.755 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of daemons per service per host", 2024-04-21T08:51:21.756 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.756 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.756 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.756 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.756 INFO:teuthology.orchestra.run.smithi086.stdout: "max_osd_draining_count": { 2024-04-21T08:51:21.756 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_osd_draining_count", 2024-04-21T08:51:21.756 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.756 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.756 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.756 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-21T08:51:21.756 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.756 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.757 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.757 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-21T08:51:21.757 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.757 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.757 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.757 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.757 INFO:teuthology.orchestra.run.smithi086.stdout: "migration_current": { 2024-04-21T08:51:21.757 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "migration_current", 2024-04-21T08:51:21.757 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.757 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.757 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.757 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.757 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.758 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.758 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.758 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "internal - do not modify", 2024-04-21T08:51:21.758 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.758 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.758 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.758 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.758 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-04-21T08:51:21.758 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-04-21T08:51:21.758 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.758 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.758 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.759 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "root", 2024-04-21T08:51:21.759 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.759 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.759 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.759 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm-package", 2024-04-21T08:51:21.759 INFO:teuthology.orchestra.run.smithi086.stdout: "root" 2024-04-21T08:51:21.759 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.759 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "mode for remote execution of cephadm", 2024-04-21T08:51:21.759 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.759 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.759 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.759 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.759 INFO:teuthology.orchestra.run.smithi086.stdout: "oob_default_addr": { 2024-04-21T08:51:21.760 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "oob_default_addr", 2024-04-21T08:51:21.760 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.760 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.760 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.760 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "169.254.1.1", 2024-04-21T08:51:21.760 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.760 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.760 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.760 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-21T08:51:21.760 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.760 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.761 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.761 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.761 INFO:teuthology.orchestra.run.smithi086.stdout: "prometheus_alerts_path": { 2024-04-21T08:51:21.761 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus_alerts_path", 2024-04-21T08:51:21.761 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.761 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.761 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.761 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-21T08:51:21.761 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.761 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.761 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.761 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-21T08:51:21.762 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.762 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.762 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.762 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.762 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_insecure": { 2024-04-21T08:51:21.762 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_insecure", 2024-04-21T08:51:21.762 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.762 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.762 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.762 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.762 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.763 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.763 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.763 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-21T08:51:21.763 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.763 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.763 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.763 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.765 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_password": { 2024-04-21T08:51:21.765 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_password", 2024-04-21T08:51:21.765 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.765 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.765 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.765 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.765 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.766 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.766 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.766 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-21T08:51:21.766 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.766 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.766 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.766 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.766 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_url": { 2024-04-21T08:51:21.766 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_url", 2024-04-21T08:51:21.767 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.767 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.767 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.767 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.767 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.767 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.767 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.767 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-21T08:51:21.767 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.767 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.767 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.768 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.768 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_username": { 2024-04-21T08:51:21.768 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_username", 2024-04-21T08:51:21.768 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.768 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.768 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.768 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.768 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.768 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.768 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.768 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-21T08:51:21.768 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.768 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.769 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.769 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.769 INFO:teuthology.orchestra.run.smithi086.stdout: "secure_monitoring_stack": { 2024-04-21T08:51:21.769 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "secure_monitoring_stack", 2024-04-21T08:51:21.769 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.769 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.769 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.769 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.769 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.769 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.769 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.769 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-21T08:51:21.770 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.770 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.770 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.770 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.770 INFO:teuthology.orchestra.run.smithi086.stdout: "service_discovery_port": { 2024-04-21T08:51:21.770 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "service_discovery_port", 2024-04-21T08:51:21.770 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.770 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.770 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.770 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8765", 2024-04-21T08:51:21.770 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.770 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.771 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.771 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "cephadm service discovery port", 2024-04-21T08:51:21.771 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.771 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.771 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.771 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.771 INFO:teuthology.orchestra.run.smithi086.stdout: "ssh_config_file": { 2024-04-21T08:51:21.771 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssh_config_file", 2024-04-21T08:51:21.771 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.771 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.772 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.772 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.772 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.772 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.772 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.772 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-21T08:51:21.772 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.772 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.772 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.772 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.772 INFO:teuthology.orchestra.run.smithi086.stdout: "use_agent": { 2024-04-21T08:51:21.773 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_agent", 2024-04-21T08:51:21.773 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.773 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.773 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.773 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.773 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.773 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.773 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.773 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-21T08:51:21.773 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.773 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.773 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.773 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.774 INFO:teuthology.orchestra.run.smithi086.stdout: "use_repo_digest": { 2024-04-21T08:51:21.774 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_repo_digest", 2024-04-21T08:51:21.774 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.774 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.774 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.774 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.774 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.774 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.774 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.774 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-21T08:51:21.775 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.775 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.775 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.775 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.775 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_failed_host_check": { 2024-04-21T08:51:21.775 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_failed_host_check", 2024-04-21T08:51:21.775 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.775 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.775 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.775 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.775 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.775 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.776 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.776 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if the host check fails", 2024-04-21T08:51:21.776 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.776 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.776 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.776 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.776 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_daemons": { 2024-04-21T08:51:21.776 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_daemons", 2024-04-21T08:51:21.776 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.776 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.776 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.777 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.777 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.777 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.777 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.777 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-21T08:51:21.777 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.777 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.777 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.777 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.777 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_hosts": { 2024-04-21T08:51:21.778 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_hosts", 2024-04-21T08:51:21.778 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.778 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.778 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.778 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.778 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.778 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.778 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.778 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-21T08:51:21.778 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.778 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.779 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.779 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.779 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.779 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.779 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.779 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crash", 2024-04-21T08:51:21.779 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.779 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.779 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.779 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.779 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.779 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.780 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.780 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.780 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.780 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.780 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.780 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.780 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.780 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.780 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.780 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.780 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.780 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.781 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.781 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.781 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.781 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.781 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.781 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.781 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.781 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.781 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.781 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.781 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.781 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.781 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.782 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.782 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.782 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.782 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.782 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.782 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.782 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.782 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.782 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.782 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.782 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.782 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.783 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.783 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.783 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.783 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.783 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.783 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.783 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.783 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.783 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.783 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.783 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.784 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.784 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.784 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.784 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.784 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.784 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.784 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.784 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.784 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.784 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.784 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.784 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.784 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.785 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.785 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.785 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.785 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.785 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.785 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.785 INFO:teuthology.orchestra.run.smithi086.stdout: "retain_interval": { 2024-04-21T08:51:21.785 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retain_interval", 2024-04-21T08:51:21.785 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:21.785 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.785 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.785 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "31536000", 2024-04-21T08:51:21.786 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.786 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.786 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.786 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-21T08:51:21.786 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.786 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.786 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.786 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.786 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_recent_interval": { 2024-04-21T08:51:21.786 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_recent_interval", 2024-04-21T08:51:21.786 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:21.786 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.787 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.787 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1209600", 2024-04-21T08:51:21.787 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.787 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.787 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.787 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-21T08:51:21.787 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.787 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.787 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.787 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.787 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.787 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.788 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.788 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dashboard", 2024-04-21T08:51:21.788 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.788 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.788 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.788 INFO:teuthology.orchestra.run.smithi086.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-21T08:51:21.788 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-21T08:51:21.788 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.788 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.788 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.788 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-21T08:51:21.788 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.788 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.789 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.789 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.789 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.789 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.789 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.789 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.789 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_HOST": { 2024-04-21T08:51:21.789 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-21T08:51:21.789 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.789 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.789 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.789 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.790 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.790 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.790 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.790 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.790 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.790 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.790 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.790 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.790 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-21T08:51:21.790 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-21T08:51:21.790 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.790 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.791 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.791 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.791 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.791 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.791 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.791 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.791 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.791 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.791 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.791 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.791 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_ENABLED": { 2024-04-21T08:51:21.791 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_ENABLED", 2024-04-21T08:51:21.791 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.792 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.792 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.792 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.792 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.792 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.792 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.792 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.792 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.792 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.792 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.792 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.792 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-21T08:51:21.793 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-21T08:51:21.793 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.793 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.793 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.793 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.793 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.793 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.793 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.793 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.793 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.793 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.793 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.794 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.794 INFO:teuthology.orchestra.run.smithi086.stdout: "ENABLE_BROWSABLE_API": { 2024-04-21T08:51:21.794 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-21T08:51:21.794 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.794 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.794 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.794 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.794 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.794 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.794 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.794 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.794 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.795 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.795 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.795 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.795 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-21T08:51:21.795 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-21T08:51:21.795 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.795 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.795 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.795 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.795 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.795 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.795 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.796 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.796 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.796 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.796 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.796 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.796 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-21T08:51:21.796 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-21T08:51:21.796 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.796 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.796 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.796 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.796 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.797 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.797 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.797 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.797 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.797 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.797 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.797 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.797 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-21T08:51:21.797 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-21T08:51:21.797 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.797 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.797 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.798 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.798 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.798 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.798 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.798 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.798 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.798 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.798 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.798 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.798 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-21T08:51:21.798 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-21T08:51:21.798 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.798 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.799 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.799 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.799 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.799 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.799 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.799 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.799 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.799 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.799 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.799 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.799 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-21T08:51:21.799 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-21T08:51:21.800 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.800 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.800 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.800 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.800 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.800 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.800 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.800 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.800 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.800 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.800 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.801 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.801 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-21T08:51:21.801 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-21T08:51:21.801 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.801 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.801 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.801 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.801 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.801 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.801 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.801 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.801 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.802 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.802 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.802 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.802 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-21T08:51:21.802 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-21T08:51:21.802 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.802 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.802 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.802 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.802 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.802 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.802 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.803 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.803 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.803 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.803 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.803 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.803 INFO:teuthology.orchestra.run.smithi086.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-21T08:51:21.803 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-21T08:51:21.803 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.803 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.803 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.803 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.803 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.804 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.804 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.804 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.804 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.804 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.804 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.804 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.804 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_PASSWORD": { 2024-04-21T08:51:21.804 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-21T08:51:21.804 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.804 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.804 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.804 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-04-21T08:51:21.805 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.805 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.805 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.805 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.805 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.805 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.805 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.805 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.805 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-21T08:51:21.805 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-21T08:51:21.805 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.805 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.806 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.806 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.806 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.806 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.806 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.806 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.806 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.806 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.806 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.806 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.806 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_URL": { 2024-04-21T08:51:21.806 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_URL", 2024-04-21T08:51:21.806 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.807 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.807 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.807 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.807 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.807 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.807 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.807 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.807 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.807 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.807 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.807 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.808 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_USERNAME": { 2024-04-21T08:51:21.808 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-21T08:51:21.808 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.808 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.808 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.808 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-04-21T08:51:21.808 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.808 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.808 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.808 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.808 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.808 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.808 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.809 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.809 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-21T08:51:21.809 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-21T08:51:21.809 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.809 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.809 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.809 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.809 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.809 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.809 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.809 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.810 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.810 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.810 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.810 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.810 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-21T08:51:21.810 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-21T08:51:21.810 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.810 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.810 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.810 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.811 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.811 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.811 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.811 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.811 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.811 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.811 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.811 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.811 INFO:teuthology.orchestra.run.smithi086.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-21T08:51:21.811 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-21T08:51:21.811 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.811 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.812 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.812 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.812 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.812 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.812 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.812 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.812 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.812 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.812 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.812 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.812 INFO:teuthology.orchestra.run.smithi086.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-21T08:51:21.812 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-21T08:51:21.813 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.813 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.813 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.813 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.813 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.813 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.813 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.813 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.813 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.813 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.813 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.813 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.814 INFO:teuthology.orchestra.run.smithi086.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-21T08:51:21.814 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-21T08:51:21.814 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.814 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.814 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.814 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "[]", 2024-04-21T08:51:21.814 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.814 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.814 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.814 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.814 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.815 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.815 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.815 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.815 INFO:teuthology.orchestra.run.smithi086.stdout: "MULTICLUSTER_CONFIG": { 2024-04-21T08:51:21.815 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-21T08:51:21.815 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.815 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.815 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.815 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "{}", 2024-04-21T08:51:21.815 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.815 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.815 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.815 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.816 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.816 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.816 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.816 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.816 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_HOST": { 2024-04-21T08:51:21.816 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-21T08:51:21.816 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.816 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.816 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.816 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.816 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.816 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.817 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.817 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.817 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.817 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.817 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.817 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.817 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-21T08:51:21.817 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-21T08:51:21.817 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.817 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.817 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.817 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.817 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.818 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.818 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.818 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.818 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.818 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.818 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.818 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.818 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-21T08:51:21.818 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-21T08:51:21.818 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.818 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.818 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.819 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.819 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.819 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.819 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.819 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.819 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.819 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.819 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.819 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.819 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-21T08:51:21.819 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-21T08:51:21.819 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.819 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.820 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.820 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.820 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.820 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.820 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.820 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.820 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.820 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.820 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.820 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.820 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-21T08:51:21.820 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-21T08:51:21.821 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.821 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.821 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.821 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.821 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.821 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.821 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.821 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.821 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.821 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.821 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.821 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.821 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-21T08:51:21.822 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-21T08:51:21.822 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.822 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.822 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.822 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.822 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.822 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.822 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.822 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.822 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.822 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.822 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.822 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.823 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-21T08:51:21.823 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-21T08:51:21.823 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.823 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.823 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.823 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.823 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.823 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.823 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.823 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.823 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.824 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.824 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.824 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.824 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-21T08:51:21.824 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-21T08:51:21.824 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.824 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.824 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.824 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.824 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.824 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.824 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.825 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.825 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.825 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.825 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.825 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.825 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-21T08:51:21.825 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-21T08:51:21.825 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.825 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.825 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.825 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.825 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.825 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.826 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.826 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.826 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.826 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.826 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.826 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.826 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_ENABLED": { 2024-04-21T08:51:21.826 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-21T08:51:21.826 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.826 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.826 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.827 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.827 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.827 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.827 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.827 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.827 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.827 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.827 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.827 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.827 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-21T08:51:21.827 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-21T08:51:21.828 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.828 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.828 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.828 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-21T08:51:21.828 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.828 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.828 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.828 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.828 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.828 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.828 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.828 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.829 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-21T08:51:21.829 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-21T08:51:21.829 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.829 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.829 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.829 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-21T08:51:21.829 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.829 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.829 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.829 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.829 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.829 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.829 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.830 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.830 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-21T08:51:21.830 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-21T08:51:21.830 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.830 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.830 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.830 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8", 2024-04-21T08:51:21.830 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.830 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.830 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.831 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.831 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.831 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.831 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.831 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.831 INFO:teuthology.orchestra.run.smithi086.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-21T08:51:21.831 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-21T08:51:21.831 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.831 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.831 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.831 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "45", 2024-04-21T08:51:21.831 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.831 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.832 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.832 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.832 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.832 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.832 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.832 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.832 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ACCESS_KEY": { 2024-04-21T08:51:21.832 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-21T08:51:21.832 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.832 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.832 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.832 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.833 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.833 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.833 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.833 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.833 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.833 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.833 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.833 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.833 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-21T08:51:21.833 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-21T08:51:21.833 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.833 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.834 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.834 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-04-21T08:51:21.834 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.834 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.834 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.834 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.834 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.834 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.834 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.834 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.834 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SECRET_KEY": { 2024-04-21T08:51:21.835 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-21T08:51:21.835 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.835 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.835 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.835 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.835 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.835 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.835 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.835 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.835 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.835 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.835 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.836 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.836 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SSL_VERIFY": { 2024-04-21T08:51:21.836 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-21T08:51:21.836 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.836 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.836 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.836 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.836 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.836 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.836 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.836 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.836 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.837 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.837 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.837 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.837 INFO:teuthology.orchestra.run.smithi086.stdout: "UNSAFE_TLS_v1_2": { 2024-04-21T08:51:21.837 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-21T08:51:21.837 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.837 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.837 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.837 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.837 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.837 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.837 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.837 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.838 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.838 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.838 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.838 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.838 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-21T08:51:21.838 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-21T08:51:21.838 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.838 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.838 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.838 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-21T08:51:21.838 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.839 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.839 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.839 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.839 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.839 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.839 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.839 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.839 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-21T08:51:21.839 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-21T08:51:21.839 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.839 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.839 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.840 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-21T08:51:21.840 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.840 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.840 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.840 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.840 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.840 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.840 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.840 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.840 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-21T08:51:21.840 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-21T08:51:21.840 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.841 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.841 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.841 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-21T08:51:21.841 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.841 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.841 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.841 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.841 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.841 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.841 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.841 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.841 INFO:teuthology.orchestra.run.smithi086.stdout: "cross_origin_url": { 2024-04-21T08:51:21.842 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cross_origin_url", 2024-04-21T08:51:21.842 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.842 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.842 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.842 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.842 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.842 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.842 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.842 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.842 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.842 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.842 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.843 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.843 INFO:teuthology.orchestra.run.smithi086.stdout: "crt_file": { 2024-04-21T08:51:21.843 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crt_file", 2024-04-21T08:51:21.843 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.843 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.843 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.843 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.843 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.843 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.843 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.843 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.844 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.844 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.844 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.844 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.844 INFO:teuthology.orchestra.run.smithi086.stdout: "debug": { 2024-04-21T08:51:21.844 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "debug", 2024-04-21T08:51:21.844 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.844 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.844 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.844 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.844 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.845 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.845 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.845 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable/disable debug options", 2024-04-21T08:51:21.845 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.845 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.845 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.845 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.845 INFO:teuthology.orchestra.run.smithi086.stdout: "jwt_token_ttl": { 2024-04-21T08:51:21.845 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "jwt_token_ttl", 2024-04-21T08:51:21.845 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.845 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.846 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.846 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "28800", 2024-04-21T08:51:21.846 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.846 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.846 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.846 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.846 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.846 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.846 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.846 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.846 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-04-21T08:51:21.846 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-04-21T08:51:21.847 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.847 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.847 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.847 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.847 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.847 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.847 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.847 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.847 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.847 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.847 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.847 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.847 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.848 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.848 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.848 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.848 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.848 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.848 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.848 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.848 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.848 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.848 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.848 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.848 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.849 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.849 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.849 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.849 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.849 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.849 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.849 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.849 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.849 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.849 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.849 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.849 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.850 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.850 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.850 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.850 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.850 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.850 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.850 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.850 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.850 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.850 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.850 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.851 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.851 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.851 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.851 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.851 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.851 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.851 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.851 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.851 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.851 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.851 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.851 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.851 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.852 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.852 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.852 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.852 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.852 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.852 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.852 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.852 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.852 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.852 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.852 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.852 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.853 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.853 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.853 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.853 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.853 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.853 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.853 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.853 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.853 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.853 INFO:teuthology.orchestra.run.smithi086.stdout: "motd": { 2024-04-21T08:51:21.853 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "motd", 2024-04-21T08:51:21.853 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.854 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.854 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.854 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.854 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.854 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.854 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.854 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "The message of the day", 2024-04-21T08:51:21.854 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.854 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.854 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.855 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.855 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect_resolve_ip_addr": { 2024-04-21T08:51:21.855 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "redirect_resolve_ip_addr", 2024-04-21T08:51:21.855 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.855 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.855 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.855 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.855 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.855 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.855 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.855 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.855 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.855 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.856 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.856 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.856 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-04-21T08:51:21.856 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-04-21T08:51:21.856 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.856 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.856 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.856 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-04-21T08:51:21.856 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.856 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.856 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.856 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.857 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.857 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.857 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.857 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.857 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-04-21T08:51:21.857 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-04-21T08:51:21.857 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.857 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.857 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.857 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8080", 2024-04-21T08:51:21.857 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.857 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.858 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.858 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.858 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.858 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.858 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.858 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.858 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-04-21T08:51:21.858 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-04-21T08:51:21.858 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.858 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.858 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.858 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.859 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.859 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.859 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.859 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.859 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.859 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.859 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.859 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.859 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl_server_port": { 2024-04-21T08:51:21.859 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl_server_port", 2024-04-21T08:51:21.859 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.859 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.860 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.860 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8443", 2024-04-21T08:51:21.860 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.860 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.860 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.860 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.860 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.860 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.860 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.860 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.860 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-04-21T08:51:21.860 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-04-21T08:51:21.861 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.861 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.861 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.861 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "redirect", 2024-04-21T08:51:21.861 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.861 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.861 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.861 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.861 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect" 2024-04-21T08:51:21.861 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.861 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.861 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.861 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.862 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.862 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.862 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-04-21T08:51:21.862 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-04-21T08:51:21.862 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.862 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.862 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.862 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-04-21T08:51:21.862 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-04-21T08:51:21.862 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-04-21T08:51:21.862 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.862 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.863 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.863 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.863 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.863 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.863 INFO:teuthology.orchestra.run.smithi086.stdout: "url_prefix": { 2024-04-21T08:51:21.863 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url_prefix", 2024-04-21T08:51:21.863 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.863 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.863 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.863 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.863 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.863 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.864 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.864 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.864 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.864 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.864 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.864 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.864 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.864 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.864 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.864 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "devicehealth", 2024-04-21T08:51:21.864 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.864 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.865 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.865 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_monitoring": { 2024-04-21T08:51:21.865 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_monitoring", 2024-04-21T08:51:21.865 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.865 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.865 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.865 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.865 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.865 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.865 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.865 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "monitor device health metrics", 2024-04-21T08:51:21.865 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.865 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.866 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.866 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.866 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.866 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.866 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.866 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.866 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.866 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.866 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.866 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.866 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.866 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.867 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.867 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.867 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.867 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.867 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.867 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.867 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.867 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.867 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.867 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.867 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.868 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.868 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.868 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.868 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.868 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.868 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.868 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.868 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.868 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.868 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.868 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.868 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.869 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.869 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.869 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.869 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.869 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.869 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.869 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.869 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.869 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.869 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.869 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.870 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.870 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.870 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.870 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.870 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.870 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.870 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.870 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.870 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.870 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.871 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.871 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.871 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.871 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.871 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.871 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.871 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.871 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.871 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.871 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.871 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.871 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.872 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.872 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.872 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.872 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.872 INFO:teuthology.orchestra.run.smithi086.stdout: "mark_out_threshold": { 2024-04-21T08:51:21.872 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mark_out_threshold", 2024-04-21T08:51:21.872 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:21.872 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.872 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.872 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2419200", 2024-04-21T08:51:21.872 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.872 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.873 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.873 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-21T08:51:21.873 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.873 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.873 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.873 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.873 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_name": { 2024-04-21T08:51:21.873 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_name", 2024-04-21T08:51:21.873 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.873 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.873 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.873 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "device_health_metrics", 2024-04-21T08:51:21.874 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.874 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.874 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.874 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-21T08:51:21.874 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.874 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.874 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.874 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.874 INFO:teuthology.orchestra.run.smithi086.stdout: "retention_period": { 2024-04-21T08:51:21.874 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retention_period", 2024-04-21T08:51:21.874 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:21.875 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.875 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.875 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15552000", 2024-04-21T08:51:21.875 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.875 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.875 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.875 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain device health metrics", 2024-04-21T08:51:21.875 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.875 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.875 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.875 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.875 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_frequency": { 2024-04-21T08:51:21.876 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_frequency", 2024-04-21T08:51:21.876 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:21.876 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.876 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.876 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-04-21T08:51:21.876 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.876 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.876 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.876 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-21T08:51:21.876 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.876 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.876 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.877 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.877 INFO:teuthology.orchestra.run.smithi086.stdout: "self_heal": { 2024-04-21T08:51:21.877 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "self_heal", 2024-04-21T08:51:21.877 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.877 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.877 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.877 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.877 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.877 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.877 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.877 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-21T08:51:21.878 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.878 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.878 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.878 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.878 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-21T08:51:21.878 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-21T08:51:21.878 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:21.878 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.878 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.878 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-21T08:51:21.878 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.878 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.879 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.879 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and check device health", 2024-04-21T08:51:21.879 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.879 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.879 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.879 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.879 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_threshold": { 2024-04-21T08:51:21.879 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_threshold", 2024-04-21T08:51:21.879 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:21.879 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.879 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.879 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7257600", 2024-04-21T08:51:21.879 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.880 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.880 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.880 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-21T08:51:21.880 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.880 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.880 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.880 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.880 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.880 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.880 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.880 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "diskprediction_local", 2024-04-21T08:51:21.880 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.881 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.881 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.881 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.881 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.881 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.881 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.881 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.881 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.881 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.881 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.881 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.882 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.882 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.882 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.882 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.882 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.882 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.882 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.882 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.882 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.882 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.883 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.883 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.883 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.883 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.883 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.883 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.883 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.883 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.883 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.883 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.883 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.884 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.884 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.884 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.884 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.884 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.884 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.884 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.884 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.884 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.884 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.884 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.884 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.884 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.885 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.885 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.885 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.885 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.885 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.885 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.885 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.885 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.885 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.885 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.885 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.886 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.886 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.886 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.886 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.886 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.886 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.886 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.886 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.886 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.887 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.887 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.887 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.887 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.887 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.887 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.887 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.887 INFO:teuthology.orchestra.run.smithi086.stdout: "predict_interval": { 2024-04-21T08:51:21.887 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predict_interval", 2024-04-21T08:51:21.887 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.887 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.887 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.888 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-04-21T08:51:21.888 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.888 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.888 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.888 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.888 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.888 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.888 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.888 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.888 INFO:teuthology.orchestra.run.smithi086.stdout: "predictor_model": { 2024-04-21T08:51:21.888 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predictor_model", 2024-04-21T08:51:21.889 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.889 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.889 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.889 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "prophetstor", 2024-04-21T08:51:21.889 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.889 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.889 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.889 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.889 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.889 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.889 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.889 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.890 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-21T08:51:21.890 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-21T08:51:21.890 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.890 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.890 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.890 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-21T08:51:21.890 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.890 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.890 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.890 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.890 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.891 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.891 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.891 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.891 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.891 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.891 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.891 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "influx", 2024-04-21T08:51:21.891 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": false, 2024-04-21T08:51:21.891 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "influxdb python module not found", 2024-04-21T08:51:21.891 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.891 INFO:teuthology.orchestra.run.smithi086.stdout: "batch_size": { 2024-04-21T08:51:21.891 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "batch_size", 2024-04-21T08:51:21.892 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.892 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.892 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.892 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5000", 2024-04-21T08:51:21.892 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.892 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.892 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.892 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-21T08:51:21.892 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.892 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.892 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.893 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.893 INFO:teuthology.orchestra.run.smithi086.stdout: "database": { 2024-04-21T08:51:21.893 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "database", 2024-04-21T08:51:21.893 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.893 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.893 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.893 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "ceph", 2024-04-21T08:51:21.893 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.893 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.893 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.893 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-21T08:51:21.893 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.894 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.894 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.894 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.894 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": { 2024-04-21T08:51:21.894 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hostname", 2024-04-21T08:51:21.894 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.894 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.894 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.894 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.894 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.894 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.895 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.895 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server hostname", 2024-04-21T08:51:21.895 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.895 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.895 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.895 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.895 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-21T08:51:21.895 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-21T08:51:21.895 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:21.895 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.895 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.896 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "30", 2024-04-21T08:51:21.896 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "5", 2024-04-21T08:51:21.896 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.896 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.896 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-21T08:51:21.896 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.896 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.896 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.896 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.896 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.896 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.896 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.897 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.897 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.897 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.897 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.897 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.897 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.897 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.897 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.897 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.897 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.897 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.897 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.897 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.898 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.898 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.898 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.898 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.898 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.898 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.898 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.898 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.898 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.898 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.898 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.899 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.899 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.899 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.899 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.899 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.899 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.899 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.899 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.899 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.899 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.899 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.899 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.899 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.900 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.900 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.900 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.900 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.900 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.900 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.900 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.900 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.900 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.900 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.901 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.901 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.901 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.901 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.901 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.901 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.901 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.901 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.901 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.901 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.901 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.901 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.901 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.902 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.902 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.902 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.902 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.902 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.902 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.902 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.902 INFO:teuthology.orchestra.run.smithi086.stdout: "password": { 2024-04-21T08:51:21.902 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "password", 2024-04-21T08:51:21.902 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.902 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.902 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.903 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.903 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.903 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.903 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.903 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "password of InfluxDB server user", 2024-04-21T08:51:21.903 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.903 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.903 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.903 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.904 INFO:teuthology.orchestra.run.smithi086.stdout: "port": { 2024-04-21T08:51:21.904 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "port", 2024-04-21T08:51:21.904 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.904 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.904 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.904 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8086", 2024-04-21T08:51:21.904 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.904 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.904 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.904 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server port", 2024-04-21T08:51:21.904 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.904 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.904 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.905 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.905 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-04-21T08:51:21.905 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-04-21T08:51:21.905 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.905 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.905 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.905 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "false", 2024-04-21T08:51:21.905 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.905 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.905 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.905 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-21T08:51:21.905 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.906 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.906 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.906 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.906 INFO:teuthology.orchestra.run.smithi086.stdout: "threads": { 2024-04-21T08:51:21.906 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threads", 2024-04-21T08:51:21.906 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.906 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.906 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.906 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-21T08:51:21.906 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-21T08:51:21.906 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "32", 2024-04-21T08:51:21.906 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.907 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-21T08:51:21.907 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.907 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.907 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.907 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.907 INFO:teuthology.orchestra.run.smithi086.stdout: "username": { 2024-04-21T08:51:21.907 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "username", 2024-04-21T08:51:21.907 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.907 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.907 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.907 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.907 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.907 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.907 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.908 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "username of InfluxDB server user", 2024-04-21T08:51:21.908 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.908 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.908 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.908 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.908 INFO:teuthology.orchestra.run.smithi086.stdout: "verify_ssl": { 2024-04-21T08:51:21.908 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "verify_ssl", 2024-04-21T08:51:21.908 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.908 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.908 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.908 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "true", 2024-04-21T08:51:21.908 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.908 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.908 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.909 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-21T08:51:21.909 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.909 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.909 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.909 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.909 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.909 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.909 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.909 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "insights", 2024-04-21T08:51:21.909 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.909 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.909 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.909 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.910 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.910 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.910 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.910 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.910 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.910 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.910 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.910 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.910 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.910 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.910 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.910 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.911 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.911 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.911 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.911 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.911 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.911 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.911 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.911 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.911 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.911 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.911 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.911 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.911 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.911 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.912 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.912 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.912 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.912 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.912 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.912 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.912 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.912 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.912 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.912 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.912 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.912 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.913 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.913 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.913 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.913 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.913 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.913 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.913 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.913 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.913 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.913 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.913 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.913 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.913 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.913 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.914 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.914 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.914 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.914 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.914 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.914 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.914 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.914 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.914 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.914 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.914 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.914 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.915 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.915 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.915 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.915 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.915 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.915 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.915 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.915 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.915 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "iostat", 2024-04-21T08:51:21.915 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.915 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.915 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.915 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.916 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.916 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.916 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.916 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.916 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.916 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.916 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.916 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.916 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.916 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.916 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.916 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.917 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.917 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.917 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.917 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.917 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.917 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.917 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.917 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.917 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.917 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.917 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.917 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.918 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.918 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.918 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.918 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.918 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.918 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.918 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.918 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.918 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.918 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.918 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.918 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.919 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.919 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.919 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.919 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.919 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.919 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.919 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.919 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.919 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.919 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.919 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.920 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.920 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.920 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.920 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.920 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.920 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.920 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.920 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.920 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.920 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.920 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.920 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.920 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.921 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.921 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.921 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.921 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.921 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.921 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.921 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.921 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.921 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.921 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.921 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.922 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.922 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "k8sevents", 2024-04-21T08:51:21.922 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.922 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.922 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.922 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_event_retention_days": { 2024-04-21T08:51:21.922 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ceph_event_retention_days", 2024-04-21T08:51:21.922 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.922 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.922 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.922 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7", 2024-04-21T08:51:21.922 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.923 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.923 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.923 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-21T08:51:21.923 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.923 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.923 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.923 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.923 INFO:teuthology.orchestra.run.smithi086.stdout: "config_check_secs": { 2024-04-21T08:51:21.923 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_check_secs", 2024-04-21T08:51:21.923 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.923 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.923 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.924 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-21T08:51:21.924 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "10", 2024-04-21T08:51:21.924 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.924 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.924 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-21T08:51:21.924 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.924 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.924 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.924 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.924 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.924 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.924 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.925 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.925 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.925 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.925 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.925 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.925 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.925 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.925 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.925 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.925 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.925 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.925 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.926 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.926 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.926 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.926 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.926 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.926 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.926 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.926 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.926 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.926 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.926 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.926 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.927 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.927 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.927 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.927 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.927 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.927 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.927 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.927 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.927 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.927 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.927 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.927 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.928 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.928 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.928 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.928 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.928 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.928 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.928 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.928 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.928 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.928 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.928 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.928 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.928 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.929 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.929 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.929 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.929 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.929 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.929 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.929 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.929 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.929 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.929 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.929 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.929 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.930 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.930 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.930 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.930 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.930 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.930 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.930 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.930 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.930 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "localpool", 2024-04-21T08:51:21.930 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.930 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.930 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.930 INFO:teuthology.orchestra.run.smithi086.stdout: "failure_domain": { 2024-04-21T08:51:21.931 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "failure_domain", 2024-04-21T08:51:21.931 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.931 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.931 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.931 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "host", 2024-04-21T08:51:21.931 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.931 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.931 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.931 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "failure domain for any created local pool", 2024-04-21T08:51:21.931 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-21T08:51:21.931 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.931 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.932 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.932 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.932 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.932 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.932 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.932 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.932 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.932 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.932 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.932 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.932 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.932 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.933 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.933 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.933 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.933 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.933 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.933 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.933 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.933 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.933 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.933 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.934 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.934 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.934 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.934 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.934 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.934 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.934 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.934 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.934 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.934 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.935 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.935 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.935 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.935 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.935 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.935 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.935 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.935 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.935 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.935 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.935 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.935 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.936 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.936 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.936 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.936 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.936 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.936 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.936 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.936 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.936 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.936 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.936 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.936 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.936 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.937 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.937 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.937 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.937 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.937 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.937 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.937 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.937 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.937 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.937 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.937 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.938 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.938 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.938 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.938 INFO:teuthology.orchestra.run.smithi086.stdout: "min_size": { 2024-04-21T08:51:21.938 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_size", 2024-04-21T08:51:21.938 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.938 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.938 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.938 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.938 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.939 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.939 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.939 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default min_size for any created local pool", 2024-04-21T08:51:21.939 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-21T08:51:21.939 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.939 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.939 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.939 INFO:teuthology.orchestra.run.smithi086.stdout: "num_rep": { 2024-04-21T08:51:21.939 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "num_rep", 2024-04-21T08:51:21.939 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.939 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.939 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.939 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3", 2024-04-21T08:51:21.940 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.940 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.940 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.940 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default replica count for any created local pool", 2024-04-21T08:51:21.940 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.940 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.940 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.940 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.940 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_num": { 2024-04-21T08:51:21.940 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_num", 2024-04-21T08:51:21.940 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:21.941 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.941 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.941 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "128", 2024-04-21T08:51:21.941 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.941 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.941 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.941 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default pg_num for any created local pool", 2024-04-21T08:51:21.941 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.941 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.941 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.941 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.941 INFO:teuthology.orchestra.run.smithi086.stdout: "prefix": { 2024-04-21T08:51:21.942 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prefix", 2024-04-21T08:51:21.942 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.942 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.942 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.942 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.942 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.942 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.942 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.942 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name prefix for any created local pool", 2024-04-21T08:51:21.942 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.942 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.942 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.943 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.943 INFO:teuthology.orchestra.run.smithi086.stdout: "subtree": { 2024-04-21T08:51:21.943 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "subtree", 2024-04-21T08:51:21.943 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.943 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.943 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.943 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "rack", 2024-04-21T08:51:21.943 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.943 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.943 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.943 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-21T08:51:21.943 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-21T08:51:21.943 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.944 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.944 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.944 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.944 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.944 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.944 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mds_autoscaler", 2024-04-21T08:51:21.944 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.944 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.944 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.944 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.944 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.944 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.944 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.944 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.945 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.945 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.945 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.945 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.945 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.945 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.945 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.945 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.945 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.945 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.945 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.945 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.946 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.946 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.946 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.946 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.946 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.946 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.946 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.946 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.946 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.946 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.946 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.947 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.947 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.947 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.947 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.947 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.947 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.947 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.947 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.947 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.947 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.947 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.947 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.947 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.948 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.948 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.948 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.948 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.948 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.948 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.948 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.948 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.948 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.948 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.948 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.948 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.948 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.949 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.949 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.949 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.949 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.949 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.949 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.949 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.949 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.949 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.949 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.949 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.949 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.950 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.950 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.950 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.950 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.950 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.950 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.950 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.950 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirroring", 2024-04-21T08:51:21.950 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.950 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.950 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.950 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.950 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.951 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.951 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.951 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.951 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.951 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.951 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.951 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.951 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.951 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.951 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.951 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.951 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.951 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.952 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.952 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.952 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.952 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.952 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.952 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.952 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.952 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.952 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.952 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.952 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.952 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.953 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.953 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.953 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.953 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.953 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.953 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.953 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.953 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.953 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.953 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.953 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.953 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.954 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.954 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.954 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.954 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.954 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.954 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.954 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.954 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.954 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.954 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.954 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.954 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.955 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.955 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.955 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.955 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.955 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.955 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.955 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.955 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.955 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.955 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.955 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.955 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.956 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.956 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.956 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.956 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.956 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.956 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.956 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.956 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.956 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.956 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.956 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "nfs", 2024-04-21T08:51:21.957 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.957 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.957 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.957 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.957 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.957 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.957 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.957 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.957 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.957 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.957 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.957 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.957 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.958 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.958 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.958 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.958 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.958 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.958 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.958 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.958 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.958 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.958 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.958 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.958 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.959 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.959 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.959 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.959 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.959 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.959 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.959 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.959 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.959 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.959 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.959 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.959 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.960 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.960 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.960 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.960 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.960 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.960 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.960 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.960 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.960 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.960 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.960 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.960 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.960 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.961 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.961 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.961 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.961 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.961 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.961 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.961 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.961 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.961 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.961 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.961 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.961 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.962 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.962 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.962 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.962 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.962 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.962 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.962 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.962 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.962 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.962 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.962 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.963 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.963 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.963 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.963 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-04-21T08:51:21.963 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.963 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.963 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.963 INFO:teuthology.orchestra.run.smithi086.stdout: "fail_fs": { 2024-04-21T08:51:21.963 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "fail_fs", 2024-04-21T08:51:21.963 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.963 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.963 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.963 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.964 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.964 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.964 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.964 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-21T08:51:21.964 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.964 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.964 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.964 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.964 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.964 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.964 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.964 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.964 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.965 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.965 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.965 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.965 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.965 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.965 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.965 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.965 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.965 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.965 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.965 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.965 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.966 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.966 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.966 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.966 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.966 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.966 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.966 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.966 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.966 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.966 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.966 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.967 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.967 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.967 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.967 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.967 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.967 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.967 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.967 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.967 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.967 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.967 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.967 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.969 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.969 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.969 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.969 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.969 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.969 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.969 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.969 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.970 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.970 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.970 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.970 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.970 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.970 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.970 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.970 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.970 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.970 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.970 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.970 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.971 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.971 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.971 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.971 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.971 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.971 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.971 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.971 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.971 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.971 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.971 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator": { 2024-04-21T08:51:21.971 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-04-21T08:51:21.972 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.972 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.972 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.972 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.972 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.972 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.972 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.972 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm", 2024-04-21T08:51:21.972 INFO:teuthology.orchestra.run.smithi086.stdout: "rook", 2024-04-21T08:51:21.972 INFO:teuthology.orchestra.run.smithi086.stdout: "test_orchestrator" 2024-04-21T08:51:21.972 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.972 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Orchestrator backend", 2024-04-21T08:51:21.972 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.973 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.973 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.973 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.973 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.973 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.973 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.973 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_perf_query", 2024-04-21T08:51:21.973 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.973 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.973 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.973 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.974 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.974 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.974 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.974 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.974 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.974 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.974 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.974 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.974 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.974 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.974 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.975 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.975 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.975 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.975 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.975 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.975 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.975 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.975 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.975 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.975 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.975 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.975 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.976 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.976 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.976 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.976 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.976 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.976 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.976 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.976 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.976 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.976 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.976 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.976 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.977 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.977 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.977 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.977 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.977 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.977 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.977 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.977 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.977 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.977 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.977 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.977 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.977 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.978 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.978 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.978 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.978 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.978 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.978 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.978 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.978 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.978 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.978 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.978 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.978 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.979 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.979 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.979 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.979 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.979 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.979 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.979 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.979 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.979 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.979 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.979 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.979 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.980 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_support", 2024-04-21T08:51:21.980 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.980 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.980 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.980 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.980 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.980 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.980 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.980 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.980 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.980 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.980 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.980 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.981 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.981 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.981 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.981 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.981 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.981 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.981 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.981 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.981 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.981 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.981 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.981 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.982 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.982 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.982 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.982 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.982 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.982 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.982 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.982 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.982 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.982 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.982 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.982 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.983 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.983 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.983 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.983 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.983 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.983 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.983 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.983 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.983 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.983 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.983 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.983 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.984 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.984 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.984 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.984 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.984 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.984 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.984 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.984 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.984 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.984 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.984 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.984 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.984 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.985 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.985 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.985 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.985 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.985 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.985 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.985 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.985 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.985 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.985 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.985 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.986 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.986 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.986 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.986 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.986 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_autoscaler", 2024-04-21T08:51:21.986 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.986 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.986 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.986 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.986 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.986 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.987 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.987 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.987 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.987 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.987 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.987 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.987 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.987 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.987 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.987 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.987 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.987 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.988 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.988 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.988 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.988 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.988 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.988 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.988 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.988 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.988 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.988 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.988 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.988 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.988 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.989 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.989 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.989 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.989 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.989 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.989 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.989 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.989 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:21.989 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:21.989 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.989 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.989 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.989 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:21.990 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.990 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.990 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.990 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.990 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.990 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.990 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.990 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.990 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.990 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.990 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.990 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.991 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.991 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.991 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.991 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:21.991 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:21.991 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.991 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.991 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.991 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.991 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.991 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.991 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.992 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.992 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.992 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.992 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.992 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.992 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-21T08:51:21.992 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-21T08:51:21.992 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:21.992 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.992 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.992 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-21T08:51:21.992 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.993 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.993 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.993 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.993 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.993 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.993 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.993 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.993 INFO:teuthology.orchestra.run.smithi086.stdout: "threshold": { 2024-04-21T08:51:21.993 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threshold", 2024-04-21T08:51:21.993 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-21T08:51:21.993 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.994 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.994 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-04-21T08:51:21.994 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1.0", 2024-04-21T08:51:21.994 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.994 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.994 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "scaling threshold", 2024-04-21T08:51:21.994 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-21T08:51:21.994 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.994 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.994 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.994 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:21.994 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.995 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:21.995 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "progress", 2024-04-21T08:51:21.995 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:21.995 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:21.995 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:21.995 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_pg_recovery_event": { 2024-04-21T08:51:21.995 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_pg_recovery_event", 2024-04-21T08:51:21.995 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.995 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.995 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.995 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:21.996 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.996 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.996 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.996 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-21T08:51:21.996 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.996 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.996 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.996 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.996 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-04-21T08:51:21.996 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-04-21T08:51:21.996 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.996 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.996 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:21.997 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:21.997 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.997 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.997 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:21.997 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.997 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.997 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.997 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.997 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.997 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:21.997 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:21.997 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:21.998 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:21.998 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:21.998 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:21.998 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:21.998 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:21.998 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:21.998 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:21.998 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:21.998 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:21.998 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:21.998 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:21.998 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:21.999 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:21.999 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:21.999 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:21.999 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:21.999 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:21.999 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:21.999 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:21.999 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:21.999 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:21.999 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.000 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.000 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.000 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.000 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.000 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.000 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.000 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.000 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.000 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.000 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.001 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.001 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.001 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.001 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.001 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.001 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.001 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.001 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.001 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.001 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.001 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.002 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.002 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.002 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.002 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.002 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.002 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.002 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.002 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.002 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.002 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.002 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.002 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.003 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.003 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.003 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.003 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.003 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.003 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.003 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.003 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.003 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.003 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.003 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.004 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.004 INFO:teuthology.orchestra.run.smithi086.stdout: "max_completed_events": { 2024-04-21T08:51:22.004 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_completed_events", 2024-04-21T08:51:22.004 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.004 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.004 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.004 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "50", 2024-04-21T08:51:22.004 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.004 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.004 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.004 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "number of past completed events to remember", 2024-04-21T08:51:22.004 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.005 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.005 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.005 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.005 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-21T08:51:22.005 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-21T08:51:22.005 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:22.005 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.005 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.005 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-21T08:51:22.005 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.005 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.005 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long the module is going to sleep", 2024-04-21T08:51:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus", 2024-04-21T08:51:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.006 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.007 INFO:teuthology.orchestra.run.smithi086.stdout: "cache": { 2024-04-21T08:51:22.007 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cache", 2024-04-21T08:51:22.007 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.007 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.007 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.007 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.007 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.007 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.007 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.007 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.007 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.008 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.008 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.008 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.008 INFO:teuthology.orchestra.run.smithi086.stdout: "exclude_perf_counters": { 2024-04-21T08:51:22.008 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "exclude_perf_counters", 2024-04-21T08:51:22.008 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.008 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.008 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.008 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.008 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.008 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.009 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.009 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-21T08:51:22.009 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-21T08:51:22.009 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.009 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.009 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.009 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.009 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.009 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.009 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.009 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.009 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.009 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.010 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.010 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.010 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.010 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.010 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.010 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.010 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.010 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.010 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.010 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.010 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.011 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.011 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.011 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.011 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.011 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.011 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.011 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.011 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.011 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.011 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.011 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.011 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.012 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.012 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.012 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.012 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.012 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.012 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.012 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.012 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.012 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.012 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.012 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.012 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.013 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.013 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.013 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.013 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.013 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.013 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.013 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.013 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.013 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.013 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.013 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.013 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.013 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.013 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.014 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.014 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.014 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.014 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.014 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.014 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.014 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.014 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.014 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.014 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.014 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.014 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.015 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.015 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.015 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools": { 2024-04-21T08:51:22.015 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools", 2024-04-21T08:51:22.015 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.015 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.015 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.015 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.015 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.015 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.015 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.015 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.015 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.016 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.016 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.016 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.016 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-21T08:51:22.016 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-21T08:51:22.016 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.016 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.016 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.016 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "300", 2024-04-21T08:51:22.016 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.016 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.016 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.017 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.017 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.017 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.017 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.017 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.017 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_interval": { 2024-04-21T08:51:22.017 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_interval", 2024-04-21T08:51:22.017 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-21T08:51:22.017 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.017 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.017 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15.0", 2024-04-21T08:51:22.017 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.018 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.018 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.018 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.018 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.018 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.018 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.018 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.018 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-04-21T08:51:22.018 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-04-21T08:51:22.018 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.018 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.018 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.019 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-04-21T08:51:22.019 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.019 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.019 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.019 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-21T08:51:22.019 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.019 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.019 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.019 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.019 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-04-21T08:51:22.020 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-04-21T08:51:22.020 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.020 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.020 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.020 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "9283", 2024-04-21T08:51:22.020 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.020 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.020 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.020 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-21T08:51:22.020 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.020 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.020 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.021 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.021 INFO:teuthology.orchestra.run.smithi086.stdout: "stale_cache_strategy": { 2024-04-21T08:51:22.021 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stale_cache_strategy", 2024-04-21T08:51:22.021 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.021 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.021 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.021 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "log", 2024-04-21T08:51:22.021 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.021 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.021 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.021 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.021 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.021 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.022 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.022 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.022 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-04-21T08:51:22.022 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-04-21T08:51:22.022 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.022 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.022 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.022 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "default", 2024-04-21T08:51:22.022 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.022 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.023 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.023 INFO:teuthology.orchestra.run.smithi086.stdout: "default", 2024-04-21T08:51:22.023 INFO:teuthology.orchestra.run.smithi086.stdout: "error" 2024-04-21T08:51:22.023 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.023 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.023 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.023 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.023 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.023 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.023 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-04-21T08:51:22.023 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-04-21T08:51:22.023 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.024 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.024 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.024 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-04-21T08:51:22.024 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-04-21T08:51:22.024 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-04-21T08:51:22.024 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.024 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.024 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.024 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.024 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.024 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.024 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.025 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.025 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.025 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_support", 2024-04-21T08:51:22.025 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.025 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.025 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.025 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.025 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.025 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.025 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.025 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.025 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.026 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.026 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.026 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.026 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.026 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.026 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.026 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.026 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.026 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.026 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.026 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.026 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.027 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.027 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.027 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.027 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.027 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.027 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.027 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.027 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.027 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.027 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.027 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.027 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.028 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.028 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.028 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.028 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.028 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.028 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.028 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.028 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.028 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.028 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.028 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.029 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.029 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.029 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.029 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.029 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.029 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.029 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.029 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.029 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.029 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.029 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.029 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.029 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.030 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.030 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.030 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.030 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.030 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.030 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.030 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.030 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.030 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.030 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.030 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.030 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.031 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.031 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.031 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.031 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.031 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_snap_create": { 2024-04-21T08:51:22.031 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_snap_create", 2024-04-21T08:51:22.031 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.031 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.032 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.032 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-21T08:51:22.032 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.032 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.032 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.032 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.032 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.032 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.032 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.032 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.032 INFO:teuthology.orchestra.run.smithi086.stdout: "mirror_snapshot_schedule": { 2024-04-21T08:51:22.033 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirror_snapshot_schedule", 2024-04-21T08:51:22.033 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.033 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.033 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.033 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.033 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.033 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.033 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.033 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.033 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.033 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.034 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.034 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.034 INFO:teuthology.orchestra.run.smithi086.stdout: "trash_purge_schedule": { 2024-04-21T08:51:22.034 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "trash_purge_schedule", 2024-04-21T08:51:22.034 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.034 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.034 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.034 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.034 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.034 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.034 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.034 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.035 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.035 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.035 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.035 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.035 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.035 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.035 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.035 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "restful", 2024-04-21T08:51:22.035 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.035 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.035 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.035 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_auth": { 2024-04-21T08:51:22.035 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_auth", 2024-04-21T08:51:22.036 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.036 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.036 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.036 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.036 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.036 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.036 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.036 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.036 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.036 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.036 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.036 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.037 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-04-21T08:51:22.037 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-04-21T08:51:22.037 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.037 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.037 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.037 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.037 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.037 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.037 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.037 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.037 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.037 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.038 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.038 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.038 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.038 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.038 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.038 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.038 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.038 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.038 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.038 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.039 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.039 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.039 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.039 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.039 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.039 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.039 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.039 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.039 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.039 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.039 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.040 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.040 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.040 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.040 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.040 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.040 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.040 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.040 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.040 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.040 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.040 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.040 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.040 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.041 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.041 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.041 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.041 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.041 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.041 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.041 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.041 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.041 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.041 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.041 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.041 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.042 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.042 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.042 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.042 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.042 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.042 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.042 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.042 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.042 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.042 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.042 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.042 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.043 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.043 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.043 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.043 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.043 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.043 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.043 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.043 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.043 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.043 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.043 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.043 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.044 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.044 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.044 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-04-21T08:51:22.044 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-04-21T08:51:22.044 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.044 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.044 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.044 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.044 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.044 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.044 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.044 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.045 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.045 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.045 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.045 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.045 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-04-21T08:51:22.045 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-04-21T08:51:22.045 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.045 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.045 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.045 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.045 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.045 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.046 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.046 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.046 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.046 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.046 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.046 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.046 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.046 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.046 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.046 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rgw", 2024-04-21T08:51:22.046 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.046 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.046 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.047 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.047 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.047 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.047 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.047 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.047 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.047 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.047 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.047 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.047 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.047 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.047 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.048 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.048 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.048 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.048 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.048 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.048 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.048 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.048 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.048 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.048 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.048 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.049 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.049 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.049 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.049 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.049 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.049 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.049 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.049 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.050 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.050 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.050 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.050 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.050 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.050 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.050 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.050 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.050 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.050 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.050 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.051 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.051 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.051 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.051 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.051 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.051 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.051 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.051 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.051 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.051 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.051 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.052 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.052 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.052 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.052 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.052 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.052 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.052 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.052 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.052 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.052 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.052 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.052 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.053 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.053 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.053 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.053 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.053 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.053 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.053 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.053 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.053 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rook", 2024-04-21T08:51:22.053 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.053 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.053 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.054 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.054 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.054 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.054 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.054 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.054 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.054 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.054 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.054 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.054 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.054 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.055 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.055 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.055 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.055 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.055 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.055 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.055 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.055 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.055 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.055 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.055 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.056 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.056 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.056 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.056 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.056 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.056 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.056 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.056 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.056 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.056 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.056 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.056 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.057 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.057 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.057 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.057 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.057 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.057 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.057 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.057 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.057 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.058 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.058 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.058 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.058 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.058 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.058 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.058 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.058 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.058 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.058 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.058 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.058 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.059 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.059 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.059 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.059 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.059 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.059 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.059 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.059 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.059 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.059 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.059 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.059 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.060 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.060 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.060 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.060 INFO:teuthology.orchestra.run.smithi086.stdout: "storage_class": { 2024-04-21T08:51:22.060 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "storage_class", 2024-04-21T08:51:22.060 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.060 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.060 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.060 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "local", 2024-04-21T08:51:22.060 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.060 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.060 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.061 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-21T08:51:22.061 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.061 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.061 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.061 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.061 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.061 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.061 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.061 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "selftest", 2024-04-21T08:51:22.061 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.061 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.062 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.062 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.062 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.062 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.062 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.062 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.062 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.062 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.062 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.062 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.062 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.063 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.063 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.063 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.063 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.063 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.063 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.063 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.063 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.063 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.063 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.063 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.063 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.064 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.064 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.064 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.064 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.064 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.064 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.064 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.064 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.064 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.064 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.064 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.064 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.065 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.065 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.065 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.065 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.065 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.065 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.065 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.065 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.065 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.065 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.066 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.066 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.066 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.066 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.066 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.066 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.066 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.066 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.066 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.066 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.066 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.066 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.067 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.067 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.067 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.067 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.067 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.067 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.067 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.067 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.067 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.067 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.067 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.068 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.068 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.068 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.068 INFO:teuthology.orchestra.run.smithi086.stdout: "roption1": { 2024-04-21T08:51:22.068 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption1", 2024-04-21T08:51:22.068 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.068 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.068 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.068 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.068 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.068 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.068 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.069 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.069 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.069 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.069 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.069 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.069 INFO:teuthology.orchestra.run.smithi086.stdout: "roption2": { 2024-04-21T08:51:22.069 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption2", 2024-04-21T08:51:22.069 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.069 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.069 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.069 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "xyz", 2024-04-21T08:51:22.069 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.070 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.070 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.070 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.070 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.070 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.070 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.070 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.070 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption1": { 2024-04-21T08:51:22.070 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption1", 2024-04-21T08:51:22.070 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.070 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.070 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.071 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.071 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.071 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.071 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.071 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.071 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.071 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.071 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.071 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.071 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption2": { 2024-04-21T08:51:22.071 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption2", 2024-04-21T08:51:22.071 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.072 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.072 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.072 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.072 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.072 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.072 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.072 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.072 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.072 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.072 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.072 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.073 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption3": { 2024-04-21T08:51:22.073 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption3", 2024-04-21T08:51:22.073 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-21T08:51:22.073 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.073 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.073 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.073 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.073 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.073 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.073 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.073 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.073 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.074 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.074 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.074 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption4": { 2024-04-21T08:51:22.074 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption4", 2024-04-21T08:51:22.074 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.074 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.074 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.074 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.074 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.074 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.074 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.074 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.075 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.075 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.075 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.075 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.075 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption5": { 2024-04-21T08:51:22.075 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption5", 2024-04-21T08:51:22.075 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.075 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.075 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.075 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.075 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.075 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.076 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.076 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.076 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.076 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.076 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.076 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.076 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption6": { 2024-04-21T08:51:22.076 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption6", 2024-04-21T08:51:22.076 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.076 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.076 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.076 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.077 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.077 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.077 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.077 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.077 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.077 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.077 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.077 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.077 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption7": { 2024-04-21T08:51:22.077 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption7", 2024-04-21T08:51:22.077 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.078 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.078 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.078 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.078 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-21T08:51:22.078 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "42", 2024-04-21T08:51:22.078 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.078 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.078 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.078 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.078 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.078 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.078 INFO:teuthology.orchestra.run.smithi086.stdout: "testkey": { 2024-04-21T08:51:22.079 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testkey", 2024-04-21T08:51:22.079 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.079 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.079 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.079 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.079 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.079 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.079 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.079 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.079 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.079 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.079 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.080 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.080 INFO:teuthology.orchestra.run.smithi086.stdout: "testlkey": { 2024-04-21T08:51:22.080 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testlkey", 2024-04-21T08:51:22.080 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.080 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.080 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.080 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.080 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.080 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.080 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.080 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.080 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.081 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.081 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.081 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.081 INFO:teuthology.orchestra.run.smithi086.stdout: "testnewline": { 2024-04-21T08:51:22.081 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testnewline", 2024-04-21T08:51:22.081 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.081 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.081 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.081 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.081 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.082 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.082 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.082 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.082 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.082 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.082 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.082 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.082 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.082 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.082 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.082 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snap_schedule", 2024-04-21T08:51:22.083 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.083 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.083 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.083 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_m_granularity": { 2024-04-21T08:51:22.083 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_m_granularity", 2024-04-21T08:51:22.083 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.083 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.083 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.083 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.083 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.083 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.084 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.084 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow minute scheduled snapshots", 2024-04-21T08:51:22.084 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.084 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.084 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.084 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.084 INFO:teuthology.orchestra.run.smithi086.stdout: "dump_on_update": { 2024-04-21T08:51:22.084 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dump_on_update", 2024-04-21T08:51:22.084 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.084 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.084 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.085 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.085 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.085 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.085 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.085 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "dump database to debug log on update", 2024-04-21T08:51:22.085 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.085 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.085 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.085 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.085 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.085 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.086 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.086 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.086 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.086 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.086 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.086 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.086 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.086 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.086 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.086 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.086 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.086 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.087 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.087 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.087 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.087 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.087 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.087 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.087 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.087 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.087 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.087 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.087 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.088 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.088 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.088 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.088 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.088 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.088 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.088 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.088 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.088 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.088 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.088 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.088 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.089 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.089 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.089 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.089 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.089 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.089 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.089 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.089 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.089 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.089 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.089 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.090 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.090 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.090 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.090 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.090 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.090 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.090 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.090 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.090 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.090 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.090 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.091 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.091 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.091 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.091 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.091 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.091 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.091 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.091 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.091 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.091 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.091 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.091 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.092 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.092 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.092 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stats", 2024-04-21T08:51:22.092 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.092 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.092 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.092 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.092 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.092 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.092 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.092 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.093 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.093 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.093 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.093 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.093 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.093 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.093 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.093 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.093 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.093 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.093 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.093 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.093 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.094 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.094 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.094 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.094 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.094 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.094 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.094 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.094 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.094 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.094 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.094 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.095 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.095 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.095 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.095 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.095 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.095 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.095 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.095 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.095 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.095 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.095 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.096 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.096 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.096 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.096 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.096 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.096 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.096 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.096 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.096 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.096 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.096 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.097 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.097 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.097 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.097 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.097 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.097 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.097 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.097 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.097 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.097 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.097 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.097 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.097 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.098 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.098 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.098 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.098 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.098 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.098 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.098 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.098 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.098 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.098 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "status", 2024-04-21T08:51:22.098 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.098 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.099 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.099 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.099 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.099 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.099 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.099 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.099 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.099 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.099 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.099 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.099 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.099 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.099 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.100 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.100 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.100 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.100 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.100 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.100 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.100 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.100 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.100 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.100 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.100 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.100 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.101 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.101 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.101 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.101 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.101 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.101 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.101 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.101 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.101 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.101 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.101 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.101 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.102 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.102 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.102 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.102 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.102 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.102 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.102 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.102 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.102 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.102 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.102 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.103 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.103 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.103 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.103 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.103 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.103 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.103 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.103 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.103 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.103 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.103 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.103 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.104 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.104 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.104 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.104 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.104 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.104 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.104 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.104 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.104 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.104 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.105 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.105 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.105 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.105 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.105 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telegraf", 2024-04-21T08:51:22.105 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.105 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.105 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.105 INFO:teuthology.orchestra.run.smithi086.stdout: "address": { 2024-04-21T08:51:22.105 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "address", 2024-04-21T08:51:22.105 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.106 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.106 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.106 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-21T08:51:22.106 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.106 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.106 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.106 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.106 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.106 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.106 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.106 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.106 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-21T08:51:22.107 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-21T08:51:22.107 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:22.107 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.107 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.107 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15", 2024-04-21T08:51:22.107 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.107 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.107 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.107 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.107 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.107 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.107 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.108 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.108 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.108 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.108 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.108 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.108 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.108 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.108 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.108 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.108 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.108 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.108 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.108 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.109 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.109 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.109 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.109 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.109 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.109 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.109 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.109 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.109 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.109 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.109 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.109 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.110 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.110 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.110 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.110 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.110 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.110 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.110 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.110 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.110 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.110 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.110 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.110 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.111 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.111 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.111 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.111 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.111 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.111 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.111 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.111 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.111 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.111 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.111 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.111 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.111 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.112 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.112 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.112 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.112 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.112 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.112 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.112 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.112 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.112 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.112 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.112 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.113 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.113 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.113 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.113 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.113 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.113 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.113 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.113 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.113 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.113 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.113 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.113 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.114 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.114 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telemetry", 2024-04-21T08:51:22.114 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.114 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.114 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.114 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_basic": { 2024-04-21T08:51:22.114 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_basic", 2024-04-21T08:51:22.114 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.114 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.114 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.114 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.114 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.114 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.115 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.115 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-21T08:51:22.115 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.115 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.115 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.115 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.115 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_crash": { 2024-04-21T08:51:22.115 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_crash", 2024-04-21T08:51:22.115 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.115 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.115 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.115 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.116 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.116 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.116 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.116 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-21T08:51:22.116 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.116 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.116 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.116 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.116 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_device": { 2024-04-21T08:51:22.116 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_device", 2024-04-21T08:51:22.116 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.117 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.117 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.117 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.117 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.117 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.117 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.117 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-21T08:51:22.117 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.117 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.117 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.117 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.117 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_ident": { 2024-04-21T08:51:22.118 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_ident", 2024-04-21T08:51:22.118 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.118 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.118 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.118 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.118 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.118 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.118 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.118 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-21T08:51:22.118 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.118 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.119 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.119 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.119 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_perf": { 2024-04-21T08:51:22.119 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_perf", 2024-04-21T08:51:22.119 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.119 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.119 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.119 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.119 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.119 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.119 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.120 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-21T08:51:22.120 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.120 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.120 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.120 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.120 INFO:teuthology.orchestra.run.smithi086.stdout: "contact": { 2024-04-21T08:51:22.120 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "contact", 2024-04-21T08:51:22.120 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.120 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.120 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.120 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.120 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.121 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.121 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.121 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.121 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.121 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.121 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.121 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.121 INFO:teuthology.orchestra.run.smithi086.stdout: "description": { 2024-04-21T08:51:22.121 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "description", 2024-04-21T08:51:22.121 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.121 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.121 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.121 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.121 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.122 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.122 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.122 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.122 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.122 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.122 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.122 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.122 INFO:teuthology.orchestra.run.smithi086.stdout: "device_url": { 2024-04-21T08:51:22.122 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_url", 2024-04-21T08:51:22.122 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.122 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.123 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.123 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-21T08:51:22.123 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.123 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.123 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.123 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.123 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.123 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.123 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.123 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.123 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-04-21T08:51:22.123 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-04-21T08:51:22.123 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.123 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.124 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.124 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.124 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.124 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.124 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.124 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.124 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.124 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.124 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.124 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.124 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-21T08:51:22.124 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-21T08:51:22.124 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.125 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.125 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.125 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "24", 2024-04-21T08:51:22.125 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "8", 2024-04-21T08:51:22.125 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.125 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.125 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.125 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.125 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.125 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.125 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.125 INFO:teuthology.orchestra.run.smithi086.stdout: "last_opt_revision": { 2024-04-21T08:51:22.126 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "last_opt_revision", 2024-04-21T08:51:22.126 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.126 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.126 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.126 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1", 2024-04-21T08:51:22.126 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.126 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.126 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.126 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.126 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.126 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.126 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.127 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.127 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard": { 2024-04-21T08:51:22.127 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard", 2024-04-21T08:51:22.127 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.127 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.127 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.127 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.127 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.127 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.127 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.127 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.127 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.127 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.128 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.128 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.128 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard_description": { 2024-04-21T08:51:22.128 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard_description", 2024-04-21T08:51:22.128 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.128 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.128 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.128 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.128 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.128 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.128 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.128 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.129 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.129 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.129 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.129 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.129 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.129 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.129 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.129 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.129 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.129 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.129 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.129 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.129 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.129 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.130 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.130 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.130 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.130 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.130 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.130 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.130 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.130 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.130 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.130 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.130 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.130 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.131 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.131 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.131 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.131 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.131 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.131 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.131 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.131 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.131 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.131 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.131 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.131 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.131 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.132 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.132 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.132 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.132 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.132 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.132 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.132 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.132 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.132 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.132 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.132 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.132 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.133 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.133 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.133 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.133 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.133 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.133 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.133 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.133 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.133 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.133 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.133 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.133 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.134 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.134 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.134 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.134 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.134 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.134 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.134 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.134 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.134 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.134 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.134 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.134 INFO:teuthology.orchestra.run.smithi086.stdout: "organization": { 2024-04-21T08:51:22.135 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "organization", 2024-04-21T08:51:22.135 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.135 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.135 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.135 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.135 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.135 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.135 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.135 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.135 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.135 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.135 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.136 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.136 INFO:teuthology.orchestra.run.smithi086.stdout: "proxy": { 2024-04-21T08:51:22.136 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "proxy", 2024-04-21T08:51:22.136 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.136 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.136 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.136 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.136 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.136 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.136 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.136 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.137 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.137 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.137 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.137 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.137 INFO:teuthology.orchestra.run.smithi086.stdout: "url": { 2024-04-21T08:51:22.137 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url", 2024-04-21T08:51:22.137 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.137 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.137 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.137 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-21T08:51:22.137 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.137 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.138 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.138 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.138 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.138 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.138 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.138 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.138 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.138 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.138 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.138 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "test_orchestrator", 2024-04-21T08:51:22.138 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.139 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.139 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.139 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.139 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.139 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.139 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.139 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.139 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.139 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.139 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.139 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.139 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.140 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.140 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.140 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.140 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.140 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.140 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.140 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.140 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.140 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.140 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.140 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.141 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.141 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.141 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.141 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.141 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.141 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.141 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.141 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.141 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.141 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.141 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.141 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.142 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.142 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.142 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.142 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.142 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.142 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.142 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.142 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.142 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.142 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.142 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.143 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.143 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.143 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.143 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.143 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.143 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.143 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.143 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.143 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.143 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.143 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.143 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.144 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.144 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.144 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.144 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.144 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.144 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.144 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.144 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.144 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.144 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.145 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.145 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.145 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.145 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.145 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.145 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.145 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.145 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "volumes", 2024-04-21T08:51:22.145 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.145 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.146 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.146 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.146 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.146 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.146 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.146 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.146 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.146 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.146 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.146 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.146 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.147 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.147 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.147 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.147 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.147 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.147 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.147 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.147 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.147 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.147 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.148 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.148 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.148 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.148 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.148 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.148 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.148 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.148 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.148 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.148 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.148 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.148 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.149 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.149 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.149 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.149 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.149 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.149 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.149 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.149 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.149 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.149 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.149 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.149 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.149 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.150 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.150 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.150 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.150 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.150 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.150 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.150 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.150 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.150 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.150 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.150 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.150 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.151 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.151 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.151 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.151 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.151 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.151 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.151 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.151 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.151 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.151 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.151 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.151 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.151 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.152 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_clones": { 2024-04-21T08:51:22.152 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_clones", 2024-04-21T08:51:22.152 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.152 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.152 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.152 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4", 2024-04-21T08:51:22.152 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.152 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.152 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.152 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-21T08:51:22.152 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.152 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.153 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.153 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.153 INFO:teuthology.orchestra.run.smithi086.stdout: "periodic_async_work": { 2024-04-21T08:51:22.153 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "periodic_async_work", 2024-04-21T08:51:22.153 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.153 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.153 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.153 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.153 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.153 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.153 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.153 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Periodically check for async work", 2024-04-21T08:51:22.154 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.154 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.154 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.154 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.154 INFO:teuthology.orchestra.run.smithi086.stdout: "snapshot_clone_delay": { 2024-04-21T08:51:22.154 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snapshot_clone_delay", 2024-04-21T08:51:22.154 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.154 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.154 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.154 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-21T08:51:22.154 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.154 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.154 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.155 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-21T08:51:22.155 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.155 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.155 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.155 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.155 INFO:teuthology.orchestra.run.smithi086.stdout: "snapshot_clone_no_wait": { 2024-04-21T08:51:22.155 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snapshot_clone_no_wait", 2024-04-21T08:51:22.155 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.155 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.155 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.155 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.155 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.156 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.156 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.156 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-21T08:51:22.156 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.156 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.156 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.156 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.156 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.156 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.156 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.156 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix", 2024-04-21T08:51:22.156 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.157 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.157 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.157 INFO:teuthology.orchestra.run.smithi086.stdout: "discovery_interval": { 2024-04-21T08:51:22.157 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "discovery_interval", 2024-04-21T08:51:22.157 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-21T08:51:22.157 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.157 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.157 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "100", 2024-04-21T08:51:22.157 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.157 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.157 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.157 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.157 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.157 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.158 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.158 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.158 INFO:teuthology.orchestra.run.smithi086.stdout: "identifier": { 2024-04-21T08:51:22.158 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "identifier", 2024-04-21T08:51:22.158 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.158 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.158 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.158 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.158 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.158 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.158 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.158 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.159 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.159 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.159 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.159 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.159 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-21T08:51:22.159 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-21T08:51:22.159 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:22.159 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.159 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.159 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-21T08:51:22.159 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.159 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.159 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.160 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.160 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.160 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.160 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.160 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.160 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.160 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.160 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.160 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.160 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.160 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.160 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.160 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.161 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.161 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.161 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.161 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.161 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.161 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.161 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.161 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.161 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.161 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.161 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.161 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.162 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.162 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.162 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.162 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.162 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.162 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.162 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.162 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.162 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.162 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.162 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.162 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.163 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.163 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.163 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.163 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.163 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.163 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.163 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.163 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.163 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.163 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.163 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.163 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.164 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.164 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.164 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.164 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.164 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.164 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.164 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.164 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.164 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.164 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.164 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.164 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.164 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.165 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.165 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.165 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.165 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.165 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.165 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.165 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.165 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.165 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.165 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.165 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.165 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.166 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.166 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_host": { 2024-04-21T08:51:22.166 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_host", 2024-04-21T08:51:22.166 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.166 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.166 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.166 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.166 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.166 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.166 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.166 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.166 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.167 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.167 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.167 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.167 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_port": { 2024-04-21T08:51:22.167 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_port", 2024-04-21T08:51:22.167 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.167 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.167 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.167 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10051", 2024-04-21T08:51:22.167 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.167 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.167 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.168 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.168 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.168 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.168 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.168 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.168 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_sender": { 2024-04-21T08:51:22.168 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_sender", 2024-04-21T08:51:22.168 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.168 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.168 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.168 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-21T08:51:22.168 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.169 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.169 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.169 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.169 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.169 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.169 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.169 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.169 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.169 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.169 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-21T08:51:22.169 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.170 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.170 INFO:teuthology.orchestra.run.smithi086.stdout: "modules": [ 2024-04-21T08:51:22.170 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm", 2024-04-21T08:51:22.170 INFO:teuthology.orchestra.run.smithi086.stdout: "dashboard", 2024-04-21T08:51:22.170 INFO:teuthology.orchestra.run.smithi086.stdout: "iostat", 2024-04-21T08:51:22.170 INFO:teuthology.orchestra.run.smithi086.stdout: "nfs", 2024-04-21T08:51:22.170 INFO:teuthology.orchestra.run.smithi086.stdout: "restful", 2024-04-21T08:51:22.170 INFO:teuthology.orchestra.run.smithi086.stdout: "snap_schedule" 2024-04-21T08:51:22.170 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.170 INFO:teuthology.orchestra.run.smithi086.stdout: "available_modules": [ 2024-04-21T08:51:22.170 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.171 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "alerts", 2024-04-21T08:51:22.171 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.171 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.171 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.171 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-21T08:51:22.171 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-21T08:51:22.171 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:22.171 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.171 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.171 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-21T08:51:22.171 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.171 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.172 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.172 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How frequently to reexamine health status", 2024-04-21T08:51:22.172 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.172 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.172 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.172 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.172 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.172 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.172 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.172 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.172 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.172 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.172 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.173 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.173 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.173 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.173 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.173 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.173 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.173 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.173 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.173 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.173 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.173 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.173 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.174 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.174 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.174 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.174 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.174 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.174 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.174 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.174 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.174 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.174 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.174 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.175 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.175 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.175 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.175 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.175 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.175 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.175 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.175 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.175 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.175 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.175 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.175 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.176 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.176 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.176 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.176 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.176 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.176 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.176 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.176 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.176 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.176 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.176 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.176 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.177 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.177 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.177 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.177 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.177 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.177 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.177 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.177 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.177 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.177 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.177 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.177 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.178 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.178 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.178 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.178 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.178 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_destination": { 2024-04-21T08:51:22.178 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_destination", 2024-04-21T08:51:22.178 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.178 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.178 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.178 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.178 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.178 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.179 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.179 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email address to send alerts to", 2024-04-21T08:51:22.179 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.179 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.179 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.179 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.179 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_from_name": { 2024-04-21T08:51:22.179 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_from_name", 2024-04-21T08:51:22.179 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.179 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.179 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.180 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "Ceph", 2024-04-21T08:51:22.180 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.180 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.180 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.180 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Email From: name", 2024-04-21T08:51:22.180 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.180 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.180 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.180 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.180 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_host": { 2024-04-21T08:51:22.180 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_host", 2024-04-21T08:51:22.180 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.180 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.181 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.181 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.181 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.181 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.181 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.181 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP server", 2024-04-21T08:51:22.181 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.181 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.181 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.181 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.181 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_password": { 2024-04-21T08:51:22.182 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_password", 2024-04-21T08:51:22.182 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.182 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.182 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.182 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.182 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.182 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.182 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.182 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Password to authenticate with", 2024-04-21T08:51:22.182 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.182 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.182 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.183 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.183 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_port": { 2024-04-21T08:51:22.183 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_port", 2024-04-21T08:51:22.183 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.183 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.183 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.183 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "465", 2024-04-21T08:51:22.183 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.183 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.183 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.183 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP port", 2024-04-21T08:51:22.183 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.184 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.184 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.184 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.184 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_sender": { 2024-04-21T08:51:22.184 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_sender", 2024-04-21T08:51:22.184 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.184 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.184 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.184 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.184 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.184 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.184 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.184 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SMTP envelope sender", 2024-04-21T08:51:22.185 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.185 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.185 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.185 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.185 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_ssl": { 2024-04-21T08:51:22.185 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_ssl", 2024-04-21T08:51:22.185 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.185 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.185 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.185 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.185 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.186 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.186 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.186 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use SSL to connect to SMTP server", 2024-04-21T08:51:22.186 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.186 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.186 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.186 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.186 INFO:teuthology.orchestra.run.smithi086.stdout: "smtp_user": { 2024-04-21T08:51:22.186 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "smtp_user", 2024-04-21T08:51:22.186 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.186 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.186 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.187 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.187 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.187 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.187 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.187 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "User to authenticate as", 2024-04-21T08:51:22.187 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.187 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.187 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.187 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.187 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.187 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.188 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.188 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "balancer", 2024-04-21T08:51:22.188 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.188 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.188 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.188 INFO:teuthology.orchestra.run.smithi086.stdout: "active": { 2024-04-21T08:51:22.188 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "active", 2024-04-21T08:51:22.188 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.188 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.188 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.188 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.188 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.189 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.189 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.189 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically balance PGs across cluster", 2024-04-21T08:51:22.189 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.189 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.189 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.189 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.189 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_time": { 2024-04-21T08:51:22.189 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_time", 2024-04-21T08:51:22.189 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.189 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.189 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.189 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0000", 2024-04-21T08:51:22.190 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.190 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.190 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.190 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "beginning time of day to automatically balance", 2024-04-21T08:51:22.190 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-21T08:51:22.190 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.190 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.190 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.190 INFO:teuthology.orchestra.run.smithi086.stdout: "begin_weekday": { 2024-04-21T08:51:22.190 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "begin_weekday", 2024-04-21T08:51:22.190 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-21T08:51:22.190 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.191 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.191 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-21T08:51:22.191 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-04-21T08:51:22.191 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-04-21T08:51:22.191 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.191 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to this day of the week or later", 2024-04-21T08:51:22.191 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-21T08:51:22.191 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.191 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.191 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.191 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_max_iterations": { 2024-04-21T08:51:22.191 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_max_iterations", 2024-04-21T08:51:22.192 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-21T08:51:22.192 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.192 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.192 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "25", 2024-04-21T08:51:22.192 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-21T08:51:22.192 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "250", 2024-04-21T08:51:22.192 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.192 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum number of iterations to attempt optimization", 2024-04-21T08:51:22.192 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.192 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.192 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.192 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.192 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_metrics": { 2024-04-21T08:51:22.193 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_metrics", 2024-04-21T08:51:22.193 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.193 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.193 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.193 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "pgs,objects,bytes", 2024-04-21T08:51:22.193 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.193 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.193 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.193 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "metrics with which to calculate OSD utilization", 2024-04-21T08:51:22.193 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.", 2024-04-21T08:51:22.193 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.193 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.194 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.194 INFO:teuthology.orchestra.run.smithi086.stdout: "crush_compat_step": { 2024-04-21T08:51:22.194 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crush_compat_step", 2024-04-21T08:51:22.194 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-21T08:51:22.194 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.194 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.194 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.5", 2024-04-21T08:51:22.194 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0.001", 2024-04-21T08:51:22.194 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "0.999", 2024-04-21T08:51:22.194 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.194 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "aggressiveness of optimization", 2024-04-21T08:51:22.194 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": ".99 is very aggressive, .01 is less aggressive", 2024-04-21T08:51:22.195 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.195 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.195 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.195 INFO:teuthology.orchestra.run.smithi086.stdout: "end_time": { 2024-04-21T08:51:22.195 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_time", 2024-04-21T08:51:22.195 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.195 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.195 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.195 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2359", 2024-04-21T08:51:22.195 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.195 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.196 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.196 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ending time of day to automatically balance", 2024-04-21T08:51:22.196 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "This is a time of day in the format HHMM.", 2024-04-21T08:51:22.196 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.196 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.196 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.196 INFO:teuthology.orchestra.run.smithi086.stdout: "end_weekday": { 2024-04-21T08:51:22.196 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "end_weekday", 2024-04-21T08:51:22.196 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-21T08:51:22.196 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.196 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.196 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-21T08:51:22.197 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "0", 2024-04-21T08:51:22.197 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "6", 2024-04-21T08:51:22.197 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.197 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Restrict automatic balancing to days of the week earlier than this", 2024-04-21T08:51:22.197 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "0 = Sunday, 1 = Monday, etc.", 2024-04-21T08:51:22.197 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.197 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.197 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.197 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.197 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.197 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.198 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.198 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.198 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.198 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.198 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.198 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.198 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.198 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.198 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.198 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.198 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.198 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.199 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.199 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.199 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.199 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.199 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.199 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.199 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.199 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.199 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.199 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.200 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.200 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.200 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.200 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.200 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.200 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.200 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.200 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.200 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.200 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.200 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.201 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.201 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.201 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.201 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.201 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.201 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.201 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.201 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.201 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.201 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.201 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.201 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.202 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.202 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.202 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.202 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.202 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.202 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.202 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.202 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.202 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.202 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.202 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.203 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.203 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.203 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.203 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.203 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.203 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.203 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.203 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.203 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.203 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.203 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.203 INFO:teuthology.orchestra.run.smithi086.stdout: "min_score": { 2024-04-21T08:51:22.204 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_score", 2024-04-21T08:51:22.204 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-21T08:51:22.204 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.204 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.204 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-21T08:51:22.204 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.204 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.204 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.204 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "minimum score, below which no optimization is attempted", 2024-04-21T08:51:22.204 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.204 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.205 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.205 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.205 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-04-21T08:51:22.205 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-04-21T08:51:22.205 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.205 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.205 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.205 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "upmap", 2024-04-21T08:51:22.205 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.205 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.205 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.205 INFO:teuthology.orchestra.run.smithi086.stdout: "crush-compat", 2024-04-21T08:51:22.206 INFO:teuthology.orchestra.run.smithi086.stdout: "none", 2024-04-21T08:51:22.206 INFO:teuthology.orchestra.run.smithi086.stdout: "read", 2024-04-21T08:51:22.206 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap", 2024-04-21T08:51:22.206 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap-read" 2024-04-21T08:51:22.206 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.206 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Balancer mode", 2024-04-21T08:51:22.206 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.206 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.206 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.206 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.206 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_ids": { 2024-04-21T08:51:22.206 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_ids", 2024-04-21T08:51:22.207 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.207 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.207 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.207 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.207 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.207 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.207 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.207 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "pools which the automatic balancing will be limited to", 2024-04-21T08:51:22.207 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.207 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.207 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.207 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.208 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-21T08:51:22.208 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-21T08:51:22.208 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:22.208 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.208 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.208 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-21T08:51:22.208 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.208 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.208 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.208 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and attempt optimization", 2024-04-21T08:51:22.208 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.208 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.208 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.209 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.209 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_deviation": { 2024-04-21T08:51:22.209 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_deviation", 2024-04-21T08:51:22.209 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.209 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.209 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.209 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-21T08:51:22.209 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-21T08:51:22.209 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.209 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.209 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "deviation below which no optimization is attempted", 2024-04-21T08:51:22.210 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "If the number of PGs are within this count then no optimization is attempted", 2024-04-21T08:51:22.210 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.210 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.210 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.210 INFO:teuthology.orchestra.run.smithi086.stdout: "upmap_max_optimizations": { 2024-04-21T08:51:22.210 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "upmap_max_optimizations", 2024-04-21T08:51:22.210 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-21T08:51:22.210 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.210 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.210 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-21T08:51:22.211 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.211 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.211 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.211 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "maximum upmap optimizations to make per attempt", 2024-04-21T08:51:22.211 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.211 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.211 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.211 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.211 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.211 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.211 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.211 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cephadm", 2024-04-21T08:51:22.212 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.212 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.212 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.212 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_down_multiplier": { 2024-04-21T08:51:22.212 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_down_multiplier", 2024-04-21T08:51:22.212 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-21T08:51:22.212 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.212 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.212 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-04-21T08:51:22.212 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.212 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.212 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.213 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Multiplied by agent refresh rate to calculate how long agent must not report before being marked down", 2024-04-21T08:51:22.213 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.213 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.213 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.213 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.213 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_refresh_rate": { 2024-04-21T08:51:22.213 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_refresh_rate", 2024-04-21T08:51:22.213 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:22.213 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.213 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.213 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "20", 2024-04-21T08:51:22.214 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.214 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.214 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.214 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How often agent on each host will try to gather and send metadata", 2024-04-21T08:51:22.214 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.214 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.214 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.214 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.214 INFO:teuthology.orchestra.run.smithi086.stdout: "agent_starting_port": { 2024-04-21T08:51:22.214 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "agent_starting_port", 2024-04-21T08:51:22.214 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.214 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.215 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.215 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4721", 2024-04-21T08:51:22.215 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.215 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.215 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.215 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "First port agent will try to bind to (will also try up to next 1000 subsequent ports if blocked)", 2024-04-21T08:51:22.215 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.215 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.215 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.215 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.215 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_ptrace": { 2024-04-21T08:51:22.215 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_ptrace", 2024-04-21T08:51:22.216 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.216 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.216 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.216 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.216 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.216 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.216 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.216 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow SYS_PTRACE capability on ceph containers", 2024-04-21T08:51:22.216 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this options can allow debugging daemons that encounter problems at runtime.", 2024-04-21T08:51:22.216 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.216 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.216 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.217 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_interval": { 2024-04-21T08:51:22.217 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_interval", 2024-04-21T08:51:22.217 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:22.217 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.217 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.217 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-21T08:51:22.217 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.217 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.217 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.217 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to autotune daemon memory", 2024-04-21T08:51:22.217 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.217 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.218 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.218 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.218 INFO:teuthology.orchestra.run.smithi086.stdout: "autotune_memory_target_ratio": { 2024-04-21T08:51:22.218 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "autotune_memory_target_ratio", 2024-04-21T08:51:22.218 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-21T08:51:22.218 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.218 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.218 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0.7", 2024-04-21T08:51:22.218 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.218 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.218 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.218 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "ratio of total system memory to divide amongst autotuned daemons", 2024-04-21T08:51:22.219 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.219 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.219 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.219 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.219 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm_log_destination": { 2024-04-21T08:51:22.219 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cephadm_log_destination", 2024-04-21T08:51:22.219 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.219 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.219 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.219 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.219 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.219 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.220 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.220 INFO:teuthology.orchestra.run.smithi086.stdout: "file", 2024-04-21T08:51:22.220 INFO:teuthology.orchestra.run.smithi086.stdout: "file,syslog", 2024-04-21T08:51:22.220 INFO:teuthology.orchestra.run.smithi086.stdout: "syslog" 2024-04-21T08:51:22.220 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.220 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Destination for cephadm command's persistent logging", 2024-04-21T08:51:22.220 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.220 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.220 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.220 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.220 INFO:teuthology.orchestra.run.smithi086.stdout: "cgroups_split": { 2024-04-21T08:51:22.220 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cgroups_split", 2024-04-21T08:51:22.221 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.221 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.221 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.221 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.221 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.221 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.221 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.221 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Pass --cgroups=split when cephadm creates containers (currently podman only)", 2024-04-21T08:51:22.221 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.221 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.221 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.221 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.222 INFO:teuthology.orchestra.run.smithi086.stdout: "config_checks_enabled": { 2024-04-21T08:51:22.222 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_checks_enabled", 2024-04-21T08:51:22.222 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.222 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.222 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.222 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.222 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.222 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.222 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.222 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable or disable the cephadm configuration analysis", 2024-04-21T08:51:22.222 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.222 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.223 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.223 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.223 INFO:teuthology.orchestra.run.smithi086.stdout: "config_dashboard": { 2024-04-21T08:51:22.223 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_dashboard", 2024-04-21T08:51:22.223 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.223 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.223 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.223 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.223 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.223 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.223 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.223 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "manage configs like API endpoints in Dashboard.", 2024-04-21T08:51:22.224 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.224 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.224 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.224 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.224 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_alertmanager": { 2024-04-21T08:51:22.224 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_alertmanager", 2024-04-21T08:51:22.224 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.224 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.224 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.224 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/alertmanager:v0.25.0", 2024-04-21T08:51:22.225 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.225 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.225 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.225 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-21T08:51:22.225 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.225 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.225 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.225 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.225 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_base": { 2024-04-21T08:51:22.225 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_base", 2024-04-21T08:51:22.226 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.226 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.226 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.226 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/ceph", 2024-04-21T08:51:22.226 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.226 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.226 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.226 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Container image name, without the tag", 2024-04-21T08:51:22.226 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.226 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.227 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.227 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.227 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_elasticsearch": { 2024-04-21T08:51:22.227 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_elasticsearch", 2024-04-21T08:51:22.227 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.227 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.227 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.228 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/omrizeneva/elasticsearch:6.8.23", 2024-04-21T08:51:22.228 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.228 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.228 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.228 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "elasticsearch container image", 2024-04-21T08:51:22.228 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.228 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.228 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.228 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.228 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_grafana": { 2024-04-21T08:51:22.228 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_grafana", 2024-04-21T08:51:22.229 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.229 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.229 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.229 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/grafana:9.4.12", 2024-04-21T08:51:22.229 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.229 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.229 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.229 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-21T08:51:22.229 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.229 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.229 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.229 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.230 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_haproxy": { 2024-04-21T08:51:22.230 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_haproxy", 2024-04-21T08:51:22.230 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.230 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.230 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.230 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/haproxy:2.3", 2024-04-21T08:51:22.230 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.230 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.230 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.230 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "HAproxy container image", 2024-04-21T08:51:22.230 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.231 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.231 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.231 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.231 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_agent": { 2024-04-21T08:51:22.231 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_agent", 2024-04-21T08:51:22.231 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.231 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.231 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.231 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-agent:1.29", 2024-04-21T08:51:22.231 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.231 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.231 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.232 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger agent container image", 2024-04-21T08:51:22.232 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.232 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.232 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.232 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.232 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_collector": { 2024-04-21T08:51:22.232 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_collector", 2024-04-21T08:51:22.232 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.232 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.232 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.233 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-collector:1.29", 2024-04-21T08:51:22.233 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.233 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.233 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.233 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger collector container image", 2024-04-21T08:51:22.233 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.233 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.233 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.233 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.233 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_jaeger_query": { 2024-04-21T08:51:22.233 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_jaeger_query", 2024-04-21T08:51:22.233 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.234 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.234 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.234 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/jaegertracing/jaeger-query:1.29", 2024-04-21T08:51:22.234 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.234 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.234 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.234 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Jaeger query container image", 2024-04-21T08:51:22.234 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.234 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.234 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.234 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.235 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_keepalived": { 2024-04-21T08:51:22.235 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_keepalived", 2024-04-21T08:51:22.235 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.235 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.235 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.235 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/keepalived:2.2.4", 2024-04-21T08:51:22.235 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.235 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.235 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.235 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Keepalived container image", 2024-04-21T08:51:22.235 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.236 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.236 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.236 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.236 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_loki": { 2024-04-21T08:51:22.236 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_loki", 2024-04-21T08:51:22.236 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.236 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.236 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.236 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/loki:2.4.0", 2024-04-21T08:51:22.236 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.236 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.237 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.237 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Loki container image", 2024-04-21T08:51:22.237 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.237 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.237 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.237 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.237 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_node_exporter": { 2024-04-21T08:51:22.237 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_node_exporter", 2024-04-21T08:51:22.237 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.237 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.237 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.237 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/node-exporter:v1.5.0", 2024-04-21T08:51:22.238 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.238 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.238 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.238 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-21T08:51:22.238 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.238 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.238 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.238 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.238 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_nvmeof": { 2024-04-21T08:51:22.239 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_nvmeof", 2024-04-21T08:51:22.239 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.239 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.239 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.239 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/ceph/nvmeof:1.2.1", 2024-04-21T08:51:22.239 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.239 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.239 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.239 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Nvme-of container image", 2024-04-21T08:51:22.239 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.239 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.240 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.240 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.240 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_prometheus": { 2024-04-21T08:51:22.240 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_prometheus", 2024-04-21T08:51:22.240 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.240 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.240 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.240 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/prometheus/prometheus:v2.43.0", 2024-04-21T08:51:22.240 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.240 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.241 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.241 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Prometheus container image", 2024-04-21T08:51:22.241 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.241 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.241 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.241 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.241 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_promtail": { 2024-04-21T08:51:22.241 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_promtail", 2024-04-21T08:51:22.241 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.241 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.241 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.242 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/grafana/promtail:2.4.0", 2024-04-21T08:51:22.242 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.242 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.242 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.242 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Promtail container image", 2024-04-21T08:51:22.242 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.242 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.242 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.242 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.242 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_samba": { 2024-04-21T08:51:22.242 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_samba", 2024-04-21T08:51:22.243 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.243 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.243 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.243 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "quay.io/samba.org/samba-server:devbuilds-centos-amd64", 2024-04-21T08:51:22.243 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.243 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.243 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.243 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Samba/SMB container image", 2024-04-21T08:51:22.243 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.243 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.243 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.243 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.244 INFO:teuthology.orchestra.run.smithi086.stdout: "container_image_snmp_gateway": { 2024-04-21T08:51:22.244 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_image_snmp_gateway", 2024-04-21T08:51:22.244 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.244 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.244 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.244 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io/maxwo/snmp-notifier:v1.2.1", 2024-04-21T08:51:22.244 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.244 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.244 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.244 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "SNMP Gateway container image", 2024-04-21T08:51:22.244 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.244 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.244 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.245 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.245 INFO:teuthology.orchestra.run.smithi086.stdout: "container_init": { 2024-04-21T08:51:22.245 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "container_init", 2024-04-21T08:51:22.245 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.245 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.245 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.245 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.245 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.245 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.245 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.245 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Run podman/docker with `--init`", 2024-04-21T08:51:22.245 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.246 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.246 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.246 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.246 INFO:teuthology.orchestra.run.smithi086.stdout: "daemon_cache_timeout": { 2024-04-21T08:51:22.246 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "daemon_cache_timeout", 2024-04-21T08:51:22.246 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:22.246 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.246 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.246 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-21T08:51:22.246 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.246 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.247 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.247 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache service (daemon) inventory", 2024-04-21T08:51:22.247 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.247 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.247 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.247 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.247 INFO:teuthology.orchestra.run.smithi086.stdout: "default_cephadm_command_timeout": { 2024-04-21T08:51:22.247 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_cephadm_command_timeout", 2024-04-21T08:51:22.247 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.247 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.247 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.247 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "900", 2024-04-21T08:51:22.248 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.248 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.248 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.248 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default timeout applied to cephadm commands run directly on the host (in seconds)", 2024-04-21T08:51:22.248 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.248 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.248 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.248 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.248 INFO:teuthology.orchestra.run.smithi086.stdout: "default_registry": { 2024-04-21T08:51:22.248 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "default_registry", 2024-04-21T08:51:22.248 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.249 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.249 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.249 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "docker.io", 2024-04-21T08:51:22.249 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.249 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.249 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.249 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Search-registry to which we should normalize unqualified image names. This is not the default registry", 2024-04-21T08:51:22.249 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.249 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.249 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.249 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.249 INFO:teuthology.orchestra.run.smithi086.stdout: "device_cache_timeout": { 2024-04-21T08:51:22.249 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_cache_timeout", 2024-04-21T08:51:22.250 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:22.250 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.250 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.250 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1800", 2024-04-21T08:51:22.250 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.250 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.250 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.250 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache device inventory", 2024-04-21T08:51:22.250 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.250 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.250 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.251 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.251 INFO:teuthology.orchestra.run.smithi086.stdout: "device_enhanced_scan": { 2024-04-21T08:51:22.251 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_enhanced_scan", 2024-04-21T08:51:22.251 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.251 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.251 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.251 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.251 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.251 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.251 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.251 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use libstoragemgmt during device scans", 2024-04-21T08:51:22.251 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.252 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.252 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.252 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.252 INFO:teuthology.orchestra.run.smithi086.stdout: "facts_cache_timeout": { 2024-04-21T08:51:22.252 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "facts_cache_timeout", 2024-04-21T08:51:22.252 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:22.252 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.252 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.252 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-21T08:51:22.252 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.252 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.252 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.253 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "seconds to cache host facts data", 2024-04-21T08:51:22.253 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.253 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.253 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.253 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.253 INFO:teuthology.orchestra.run.smithi086.stdout: "grafana_dashboards_path": { 2024-04-21T08:51:22.253 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "grafana_dashboards_path", 2024-04-21T08:51:22.253 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.253 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.253 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.253 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/etc/grafana/dashboards/ceph-dashboard/", 2024-04-21T08:51:22.253 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.254 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.254 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.254 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "location of dashboards to include in grafana deployments", 2024-04-21T08:51:22.254 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.254 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.254 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.254 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.254 INFO:teuthology.orchestra.run.smithi086.stdout: "host_check_interval": { 2024-04-21T08:51:22.254 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "host_check_interval", 2024-04-21T08:51:22.254 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:22.254 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.255 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.255 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-21T08:51:22.255 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.255 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.255 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.255 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to perform a host check", 2024-04-21T08:51:22.255 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.255 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.255 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.255 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.255 INFO:teuthology.orchestra.run.smithi086.stdout: "hw_monitoring": { 2024-04-21T08:51:22.255 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hw_monitoring", 2024-04-21T08:51:22.256 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.256 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.256 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.256 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.256 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.256 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.256 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.256 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Deploy hw monitoring daemon on every host.", 2024-04-21T08:51:22.256 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.256 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.256 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.256 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.256 INFO:teuthology.orchestra.run.smithi086.stdout: "inventory_list_all": { 2024-04-21T08:51:22.257 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "inventory_list_all", 2024-04-21T08:51:22.257 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.257 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.257 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.257 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.257 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.257 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.257 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.257 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)", 2024-04-21T08:51:22.257 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.257 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.257 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.258 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.258 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.258 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.258 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.258 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.258 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.258 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.258 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.258 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.258 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.258 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.258 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.259 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.259 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.259 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.259 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.259 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.259 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.259 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.259 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.259 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.259 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.259 INFO:teuthology.orchestra.run.smithi086.stdout: "log_refresh_metadata": { 2024-04-21T08:51:22.259 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_refresh_metadata", 2024-04-21T08:51:22.259 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.260 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.260 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.260 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.260 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.260 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.260 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.260 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at debug level", 2024-04-21T08:51:22.260 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.260 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.260 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.260 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.260 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.261 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.261 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.261 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.261 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.261 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.261 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.261 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.261 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.261 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "log to the \"cephadm\" cluster log channel\"", 2024-04-21T08:51:22.261 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.261 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.261 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.262 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.262 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.262 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.262 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.262 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.262 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.262 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.262 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.262 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.262 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.262 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.262 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.263 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.263 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.263 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.263 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.263 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.263 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.263 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.263 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.263 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.263 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.263 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.263 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.263 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.264 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.264 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.264 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.264 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.264 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.264 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.264 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.264 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.264 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.264 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.264 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.264 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf": { 2024-04-21T08:51:22.264 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf", 2024-04-21T08:51:22.265 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.265 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.265 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.265 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.265 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.265 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.265 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.265 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Manage and own /etc/ceph/ceph.conf on the hosts.", 2024-04-21T08:51:22.265 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.265 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.265 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.265 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.266 INFO:teuthology.orchestra.run.smithi086.stdout: "manage_etc_ceph_ceph_conf_hosts": { 2024-04-21T08:51:22.266 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "manage_etc_ceph_ceph_conf_hosts", 2024-04-21T08:51:22.266 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.266 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.266 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.266 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "*", 2024-04-21T08:51:22.266 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.266 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.266 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.266 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf", 2024-04-21T08:51:22.266 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.266 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.267 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.267 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.267 INFO:teuthology.orchestra.run.smithi086.stdout: "max_count_per_host": { 2024-04-21T08:51:22.267 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_count_per_host", 2024-04-21T08:51:22.267 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.267 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.267 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.267 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-21T08:51:22.267 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.267 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.267 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.267 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of daemons per service per host", 2024-04-21T08:51:22.267 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.267 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.268 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.268 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.268 INFO:teuthology.orchestra.run.smithi086.stdout: "max_osd_draining_count": { 2024-04-21T08:51:22.268 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_osd_draining_count", 2024-04-21T08:51:22.268 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.268 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.268 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.268 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-21T08:51:22.268 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.268 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.268 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.268 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "max number of osds that will be drained simultaneously when osds are removed", 2024-04-21T08:51:22.268 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.269 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.269 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.269 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.269 INFO:teuthology.orchestra.run.smithi086.stdout: "migration_current": { 2024-04-21T08:51:22.269 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "migration_current", 2024-04-21T08:51:22.269 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.269 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.269 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.269 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.269 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.269 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.269 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.269 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "internal - do not modify", 2024-04-21T08:51:22.270 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.270 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.270 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.270 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.270 INFO:teuthology.orchestra.run.smithi086.stdout: "mode": { 2024-04-21T08:51:22.270 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mode", 2024-04-21T08:51:22.270 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.270 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.270 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.270 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "root", 2024-04-21T08:51:22.270 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.270 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.271 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.271 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm-package", 2024-04-21T08:51:22.271 INFO:teuthology.orchestra.run.smithi086.stdout: "root" 2024-04-21T08:51:22.271 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.271 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "mode for remote execution of cephadm", 2024-04-21T08:51:22.271 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.271 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.271 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.271 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.271 INFO:teuthology.orchestra.run.smithi086.stdout: "oob_default_addr": { 2024-04-21T08:51:22.271 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "oob_default_addr", 2024-04-21T08:51:22.271 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.272 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.272 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.272 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "169.254.1.1", 2024-04-21T08:51:22.272 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.272 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.272 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.272 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Default address for RedFish API (oob management).", 2024-04-21T08:51:22.272 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.272 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.272 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.272 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.272 INFO:teuthology.orchestra.run.smithi086.stdout: "prometheus_alerts_path": { 2024-04-21T08:51:22.273 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus_alerts_path", 2024-04-21T08:51:22.273 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.273 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.273 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.273 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/etc/prometheus/ceph/ceph_default_alerts.yml", 2024-04-21T08:51:22.273 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.273 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.273 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.273 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "location of alerts to include in prometheus deployments", 2024-04-21T08:51:22.273 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.273 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.273 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.273 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.274 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_insecure": { 2024-04-21T08:51:22.274 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_insecure", 2024-04-21T08:51:22.274 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.274 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.274 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.274 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.274 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.274 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.274 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.274 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry is to be considered insecure (no TLS available). Only for development purposes.", 2024-04-21T08:51:22.274 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.275 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.275 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.275 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.275 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_password": { 2024-04-21T08:51:22.275 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_password", 2024-04-21T08:51:22.275 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.275 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.275 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.275 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.275 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.275 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.275 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.275 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository password. Only used for logging into a registry.", 2024-04-21T08:51:22.276 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.276 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.276 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.276 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.276 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_url": { 2024-04-21T08:51:22.276 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_url", 2024-04-21T08:51:22.276 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.276 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.276 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.276 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.277 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.277 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.277 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.277 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Registry url for login purposes. This is not the default registry", 2024-04-21T08:51:22.277 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.277 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.277 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.277 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.277 INFO:teuthology.orchestra.run.smithi086.stdout: "registry_username": { 2024-04-21T08:51:22.277 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "registry_username", 2024-04-21T08:51:22.277 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.277 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.278 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.278 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.278 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.278 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.278 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.278 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Custom repository username. Only used for logging into a registry.", 2024-04-21T08:51:22.278 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.278 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.278 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.278 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.278 INFO:teuthology.orchestra.run.smithi086.stdout: "secure_monitoring_stack": { 2024-04-21T08:51:22.279 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "secure_monitoring_stack", 2024-04-21T08:51:22.279 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.279 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.279 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.279 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.279 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.279 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.279 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.279 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable TLS security for all the monitoring stack daemons", 2024-04-21T08:51:22.279 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.279 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.280 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.280 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.280 INFO:teuthology.orchestra.run.smithi086.stdout: "service_discovery_port": { 2024-04-21T08:51:22.280 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "service_discovery_port", 2024-04-21T08:51:22.280 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.280 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.280 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.280 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8765", 2024-04-21T08:51:22.280 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.280 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.280 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.281 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "cephadm service discovery port", 2024-04-21T08:51:22.281 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.281 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.281 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.281 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.281 INFO:teuthology.orchestra.run.smithi086.stdout: "ssh_config_file": { 2024-04-21T08:51:22.281 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssh_config_file", 2024-04-21T08:51:22.281 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.281 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.281 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.282 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.282 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.282 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.282 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.282 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "customized SSH config file to connect to managed hosts", 2024-04-21T08:51:22.282 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.282 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.282 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.282 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.282 INFO:teuthology.orchestra.run.smithi086.stdout: "use_agent": { 2024-04-21T08:51:22.283 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_agent", 2024-04-21T08:51:22.283 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.283 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.283 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.283 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.283 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.283 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.283 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.283 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use cephadm agent on each host to gather and send metadata", 2024-04-21T08:51:22.283 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.283 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.283 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.284 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.284 INFO:teuthology.orchestra.run.smithi086.stdout: "use_repo_digest": { 2024-04-21T08:51:22.284 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "use_repo_digest", 2024-04-21T08:51:22.284 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.284 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.284 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.284 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.284 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.284 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.284 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.284 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Automatically convert image tags to image digest. Make sure all daemons use the same image", 2024-04-21T08:51:22.285 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.285 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.285 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.285 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.285 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_failed_host_check": { 2024-04-21T08:51:22.285 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_failed_host_check", 2024-04-21T08:51:22.285 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.285 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.285 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.285 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.285 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.286 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.286 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.286 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if the host check fails", 2024-04-21T08:51:22.286 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.286 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.286 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.286 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.286 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_daemons": { 2024-04-21T08:51:22.286 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_daemons", 2024-04-21T08:51:22.286 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.286 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.286 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.287 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.287 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.287 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.287 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.287 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected that are not managed by cephadm", 2024-04-21T08:51:22.287 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.287 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.287 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.287 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.287 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_on_stray_hosts": { 2024-04-21T08:51:22.287 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_on_stray_hosts", 2024-04-21T08:51:22.287 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.288 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.288 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.288 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.288 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.288 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.288 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.288 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise a health warning if daemons are detected on a host that is not managed by cephadm", 2024-04-21T08:51:22.288 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.288 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.288 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.288 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.288 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.288 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.289 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.289 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crash", 2024-04-21T08:51:22.289 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.289 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.289 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.289 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.289 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.289 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.289 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.289 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.289 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.290 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.290 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.290 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.290 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.290 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.290 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.290 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.290 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.290 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.290 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.290 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.290 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.291 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.291 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.291 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.291 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.291 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.291 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.291 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.291 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.291 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.291 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.291 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.291 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.292 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.292 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.292 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.292 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.292 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.292 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.292 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.292 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.292 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.292 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.292 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.292 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.293 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.293 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.293 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.293 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.293 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.293 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.293 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.293 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.293 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.293 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.293 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.294 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.294 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.294 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.294 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.294 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.294 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.294 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.294 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.294 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.294 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.294 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.294 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.295 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.295 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.295 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.295 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.295 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.295 INFO:teuthology.orchestra.run.smithi086.stdout: "retain_interval": { 2024-04-21T08:51:22.295 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retain_interval", 2024-04-21T08:51:22.295 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:22.295 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.295 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.295 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "31536000", 2024-04-21T08:51:22.295 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.296 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.296 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.296 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain crashes before pruning them", 2024-04-21T08:51:22.296 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.296 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.296 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.296 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.296 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_recent_interval": { 2024-04-21T08:51:22.296 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_recent_interval", 2024-04-21T08:51:22.296 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:22.296 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.296 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.297 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1209600", 2024-04-21T08:51:22.297 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.297 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.297 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.297 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "time interval in which to warn about recent crashes", 2024-04-21T08:51:22.297 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.297 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.297 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.297 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.297 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.297 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.297 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.297 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dashboard", 2024-04-21T08:51:22.298 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.298 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.298 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.298 INFO:teuthology.orchestra.run.smithi086.stdout: "ACCOUNT_LOCKOUT_ATTEMPTS": { 2024-04-21T08:51:22.298 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ACCOUNT_LOCKOUT_ATTEMPTS", 2024-04-21T08:51:22.298 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.298 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.298 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.298 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-21T08:51:22.298 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.299 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.299 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.299 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.299 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.299 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.299 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.299 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.299 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_HOST": { 2024-04-21T08:51:22.299 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_HOST", 2024-04-21T08:51:22.299 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.299 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.299 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.299 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.300 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.300 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.300 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.300 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.300 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.300 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.300 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.300 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.300 INFO:teuthology.orchestra.run.smithi086.stdout: "ALERTMANAGER_API_SSL_VERIFY": { 2024-04-21T08:51:22.300 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ALERTMANAGER_API_SSL_VERIFY", 2024-04-21T08:51:22.300 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.300 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.301 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.301 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.301 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.301 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.301 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.301 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.301 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.301 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.301 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.301 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.302 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_ENABLED": { 2024-04-21T08:51:22.302 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_ENABLED", 2024-04-21T08:51:22.302 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.302 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.302 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.302 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.302 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.302 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.302 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.303 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.303 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.303 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.303 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.303 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.303 INFO:teuthology.orchestra.run.smithi086.stdout: "AUDIT_API_LOG_PAYLOAD": { 2024-04-21T08:51:22.303 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "AUDIT_API_LOG_PAYLOAD", 2024-04-21T08:51:22.303 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.303 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.303 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.304 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.304 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.304 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.304 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.304 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.304 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.304 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.304 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.304 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.304 INFO:teuthology.orchestra.run.smithi086.stdout: "ENABLE_BROWSABLE_API": { 2024-04-21T08:51:22.304 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ENABLE_BROWSABLE_API", 2024-04-21T08:51:22.305 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.305 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.305 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.305 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.305 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.305 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.305 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.305 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.305 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.305 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.305 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.306 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.306 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_CEPHFS": { 2024-04-21T08:51:22.306 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_CEPHFS", 2024-04-21T08:51:22.306 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.306 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.306 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.306 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.306 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.306 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.306 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.307 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.307 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.307 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.307 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.307 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.307 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_DASHBOARD": { 2024-04-21T08:51:22.307 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_DASHBOARD", 2024-04-21T08:51:22.307 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.307 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.307 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.307 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.307 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.308 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.308 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.308 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.308 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.308 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.308 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.308 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.308 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_ISCSI": { 2024-04-21T08:51:22.308 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_ISCSI", 2024-04-21T08:51:22.308 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.308 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.308 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.309 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.309 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.309 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.309 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.309 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.309 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.309 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.309 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.309 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.309 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_MIRRORING": { 2024-04-21T08:51:22.309 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_MIRRORING", 2024-04-21T08:51:22.309 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.310 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.310 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.310 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.310 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.310 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.310 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.310 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.310 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.310 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.310 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.310 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.311 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_NFS": { 2024-04-21T08:51:22.311 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_NFS", 2024-04-21T08:51:22.311 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.311 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.311 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.311 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.311 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.311 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.311 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.311 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.311 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.311 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.312 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.312 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.312 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RBD": { 2024-04-21T08:51:22.312 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RBD", 2024-04-21T08:51:22.312 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.312 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.312 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.312 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.312 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.312 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.312 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.312 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.313 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.313 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.313 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.313 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.313 INFO:teuthology.orchestra.run.smithi086.stdout: "FEATURE_TOGGLE_RGW": { 2024-04-21T08:51:22.313 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "FEATURE_TOGGLE_RGW", 2024-04-21T08:51:22.313 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.313 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.313 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.313 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.313 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.313 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.314 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.314 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.314 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.314 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.314 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.314 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.314 INFO:teuthology.orchestra.run.smithi086.stdout: "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE": { 2024-04-21T08:51:22.314 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE", 2024-04-21T08:51:22.314 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.314 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.314 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.314 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.314 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.315 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.315 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.315 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.315 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.315 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.315 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.315 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.315 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_PASSWORD": { 2024-04-21T08:51:22.315 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_PASSWORD", 2024-04-21T08:51:22.315 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.315 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.315 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.316 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-04-21T08:51:22.316 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.316 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.316 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.316 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.316 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.316 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.316 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.316 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.316 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_SSL_VERIFY": { 2024-04-21T08:51:22.316 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_SSL_VERIFY", 2024-04-21T08:51:22.316 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.317 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.317 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.317 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.317 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.317 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.317 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.317 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.317 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.317 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.317 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.317 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.317 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_URL": { 2024-04-21T08:51:22.317 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_URL", 2024-04-21T08:51:22.318 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.318 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.318 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.318 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.318 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.318 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.318 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.318 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.318 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.318 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.318 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.318 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.319 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_API_USERNAME": { 2024-04-21T08:51:22.319 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_API_USERNAME", 2024-04-21T08:51:22.319 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.319 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.319 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.319 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-04-21T08:51:22.319 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.319 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.319 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.319 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.319 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.319 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.319 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.320 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.320 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_FRONTEND_API_URL": { 2024-04-21T08:51:22.320 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_FRONTEND_API_URL", 2024-04-21T08:51:22.320 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.320 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.320 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.320 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.320 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.320 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.320 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.320 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.320 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.321 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.321 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.321 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.321 INFO:teuthology.orchestra.run.smithi086.stdout: "GRAFANA_UPDATE_DASHBOARDS": { 2024-04-21T08:51:22.321 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "GRAFANA_UPDATE_DASHBOARDS", 2024-04-21T08:51:22.321 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.321 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.321 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.321 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.321 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.321 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.322 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.322 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.322 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.322 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.322 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.322 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.322 INFO:teuthology.orchestra.run.smithi086.stdout: "ISCSI_API_SSL_VERIFICATION": { 2024-04-21T08:51:22.322 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISCSI_API_SSL_VERIFICATION", 2024-04-21T08:51:22.322 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.322 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.322 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.322 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.323 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.323 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.323 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.323 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.323 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.323 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.323 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.323 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.323 INFO:teuthology.orchestra.run.smithi086.stdout: "ISSUE_TRACKER_API_KEY": { 2024-04-21T08:51:22.323 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ISSUE_TRACKER_API_KEY", 2024-04-21T08:51:22.323 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.323 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.324 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.324 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.324 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.324 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.324 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.324 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.324 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.324 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.324 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.324 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.324 INFO:teuthology.orchestra.run.smithi086.stdout: "MANAGED_BY_CLUSTERS": { 2024-04-21T08:51:22.324 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "MANAGED_BY_CLUSTERS", 2024-04-21T08:51:22.324 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.325 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.325 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.325 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "[]", 2024-04-21T08:51:22.325 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.325 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.325 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.325 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.325 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.325 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.325 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.325 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.325 INFO:teuthology.orchestra.run.smithi086.stdout: "MULTICLUSTER_CONFIG": { 2024-04-21T08:51:22.326 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "MULTICLUSTER_CONFIG", 2024-04-21T08:51:22.326 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.326 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.326 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.326 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "{}", 2024-04-21T08:51:22.326 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.326 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.326 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.326 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.326 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.326 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.326 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.327 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.327 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_HOST": { 2024-04-21T08:51:22.327 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_HOST", 2024-04-21T08:51:22.327 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.327 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.327 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.327 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.327 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.327 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.327 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.327 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.327 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.327 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.328 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.328 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.328 INFO:teuthology.orchestra.run.smithi086.stdout: "PROMETHEUS_API_SSL_VERIFY": { 2024-04-21T08:51:22.328 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PROMETHEUS_API_SSL_VERIFY", 2024-04-21T08:51:22.328 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.328 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.328 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.328 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.328 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.328 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.328 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.329 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.329 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.329 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.329 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.329 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.329 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_COMPLEXITY_ENABLED": { 2024-04-21T08:51:22.329 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_COMPLEXITY_ENABLED", 2024-04-21T08:51:22.329 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.329 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.329 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.329 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.329 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.329 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.330 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.330 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.330 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.330 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.330 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.330 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.330 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED": { 2024-04-21T08:51:22.330 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED", 2024-04-21T08:51:22.330 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.330 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.330 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.330 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.331 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.331 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.331 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.331 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.331 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.331 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.331 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.331 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.331 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_LENGTH_ENABLED": { 2024-04-21T08:51:22.331 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_LENGTH_ENABLED", 2024-04-21T08:51:22.331 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.331 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.332 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.332 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.332 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.332 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.332 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.332 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.332 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.332 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.332 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.332 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.332 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_OLDPWD_ENABLED": { 2024-04-21T08:51:22.332 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_OLDPWD_ENABLED", 2024-04-21T08:51:22.332 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.333 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.333 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.333 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.333 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.333 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.333 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.333 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.333 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.333 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.333 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.333 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.333 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED": { 2024-04-21T08:51:22.333 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED", 2024-04-21T08:51:22.334 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.334 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.334 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.334 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.334 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.334 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.334 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.334 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.334 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.334 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.334 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.334 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.335 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED": { 2024-04-21T08:51:22.335 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED", 2024-04-21T08:51:22.335 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.335 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.335 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.335 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.335 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.335 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.335 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.335 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.335 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.335 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.336 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.336 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.336 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_CHECK_USERNAME_ENABLED": { 2024-04-21T08:51:22.336 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_CHECK_USERNAME_ENABLED", 2024-04-21T08:51:22.336 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.336 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.336 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.336 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.336 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.336 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.336 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.336 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.336 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.337 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.337 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.337 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.337 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_ENABLED": { 2024-04-21T08:51:22.337 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_ENABLED", 2024-04-21T08:51:22.337 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.337 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.337 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.337 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.337 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.337 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.337 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.337 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.338 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.338 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.338 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.338 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.338 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_EXCLUSION_LIST": { 2024-04-21T08:51:22.338 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_EXCLUSION_LIST", 2024-04-21T08:51:22.338 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.338 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.338 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.338 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps", 2024-04-21T08:51:22.338 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.338 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.339 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.339 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.339 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.339 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.339 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.339 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.339 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_COMPLEXITY": { 2024-04-21T08:51:22.339 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_COMPLEXITY", 2024-04-21T08:51:22.339 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.339 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.339 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.339 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-21T08:51:22.340 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.340 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.340 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.340 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.340 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.340 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.340 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.340 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.340 INFO:teuthology.orchestra.run.smithi086.stdout: "PWD_POLICY_MIN_LENGTH": { 2024-04-21T08:51:22.340 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "PWD_POLICY_MIN_LENGTH", 2024-04-21T08:51:22.340 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.340 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.341 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.341 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8", 2024-04-21T08:51:22.341 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.341 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.341 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.341 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.341 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.341 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.341 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.341 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.341 INFO:teuthology.orchestra.run.smithi086.stdout: "REST_REQUESTS_TIMEOUT": { 2024-04-21T08:51:22.341 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "REST_REQUESTS_TIMEOUT", 2024-04-21T08:51:22.341 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.342 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.342 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.342 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "45", 2024-04-21T08:51:22.342 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.342 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.342 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.342 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.342 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.342 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.342 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.342 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.342 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ACCESS_KEY": { 2024-04-21T08:51:22.343 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ACCESS_KEY", 2024-04-21T08:51:22.343 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.343 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.343 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.343 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.343 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.343 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.343 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.343 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.343 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.343 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.343 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.343 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.344 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_ADMIN_RESOURCE": { 2024-04-21T08:51:22.344 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_ADMIN_RESOURCE", 2024-04-21T08:51:22.344 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.344 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.344 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.344 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "admin", 2024-04-21T08:51:22.344 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.344 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.344 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.344 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.344 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.344 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.345 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.345 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.345 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SECRET_KEY": { 2024-04-21T08:51:22.345 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SECRET_KEY", 2024-04-21T08:51:22.345 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.345 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.345 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.345 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.345 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.345 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.345 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.345 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.345 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.346 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.346 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.346 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.346 INFO:teuthology.orchestra.run.smithi086.stdout: "RGW_API_SSL_VERIFY": { 2024-04-21T08:51:22.346 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "RGW_API_SSL_VERIFY", 2024-04-21T08:51:22.346 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.346 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.346 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.346 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.346 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.346 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.347 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.347 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.347 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.347 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.347 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.347 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.347 INFO:teuthology.orchestra.run.smithi086.stdout: "UNSAFE_TLS_v1_2": { 2024-04-21T08:51:22.347 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "UNSAFE_TLS_v1_2", 2024-04-21T08:51:22.347 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.347 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.347 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.347 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.348 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.348 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.348 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.348 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.348 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.348 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.348 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.348 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.348 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_SPAN": { 2024-04-21T08:51:22.348 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_SPAN", 2024-04-21T08:51:22.348 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.348 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.348 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.349 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-21T08:51:22.349 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.349 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.349 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.349 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.349 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.349 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.349 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.349 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.349 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_1": { 2024-04-21T08:51:22.349 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_1", 2024-04-21T08:51:22.349 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.350 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.350 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.350 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-21T08:51:22.350 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.350 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.350 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.350 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.350 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.350 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.350 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.350 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.350 INFO:teuthology.orchestra.run.smithi086.stdout: "USER_PWD_EXPIRATION_WARNING_2": { 2024-04-21T08:51:22.351 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "USER_PWD_EXPIRATION_WARNING_2", 2024-04-21T08:51:22.351 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.351 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.351 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.351 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-21T08:51:22.351 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.351 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.351 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.351 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.351 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.351 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.351 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.352 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.352 INFO:teuthology.orchestra.run.smithi086.stdout: "cross_origin_url": { 2024-04-21T08:51:22.352 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cross_origin_url", 2024-04-21T08:51:22.352 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.352 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.352 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.352 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.352 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.352 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.352 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.352 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.352 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.353 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.353 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.353 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.353 INFO:teuthology.orchestra.run.smithi086.stdout: "crt_file": { 2024-04-21T08:51:22.353 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "crt_file", 2024-04-21T08:51:22.353 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.353 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.353 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.353 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.353 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.353 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.353 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.354 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.354 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.354 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.354 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.354 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.354 INFO:teuthology.orchestra.run.smithi086.stdout: "debug": { 2024-04-21T08:51:22.354 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "debug", 2024-04-21T08:51:22.354 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.354 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.354 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.354 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.354 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.355 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.355 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.355 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Enable/disable debug options", 2024-04-21T08:51:22.355 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.355 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.355 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.355 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.355 INFO:teuthology.orchestra.run.smithi086.stdout: "jwt_token_ttl": { 2024-04-21T08:51:22.355 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "jwt_token_ttl", 2024-04-21T08:51:22.355 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.355 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.355 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.356 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "28800", 2024-04-21T08:51:22.356 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.356 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.356 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.356 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.356 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.356 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.356 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.356 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.356 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-04-21T08:51:22.356 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-04-21T08:51:22.357 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.357 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.357 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.357 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.357 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.357 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.357 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.357 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.357 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.357 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.357 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.357 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.357 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.358 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.358 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.358 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.358 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.358 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.358 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.358 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.358 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.358 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.358 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.358 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.358 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.359 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.359 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.359 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.359 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.359 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.359 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.359 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.359 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.359 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.359 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.359 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.359 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.360 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.360 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.360 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.360 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.360 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.360 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.360 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.360 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.360 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.360 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.360 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.360 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.361 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.361 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.361 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.361 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.361 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.361 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.361 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.361 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.361 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.361 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.361 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.361 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.361 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.362 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.362 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.362 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.362 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.362 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.362 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.362 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.362 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.362 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.362 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.362 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.362 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.363 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.363 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.363 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.363 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.363 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.363 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.363 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.363 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.363 INFO:teuthology.orchestra.run.smithi086.stdout: "motd": { 2024-04-21T08:51:22.363 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "motd", 2024-04-21T08:51:22.363 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.364 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.364 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.364 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.364 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.364 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.364 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.364 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "The message of the day", 2024-04-21T08:51:22.364 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.364 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.364 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.364 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.364 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect_resolve_ip_addr": { 2024-04-21T08:51:22.365 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "redirect_resolve_ip_addr", 2024-04-21T08:51:22.365 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.365 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.365 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.365 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.365 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.365 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.365 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.365 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.365 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.365 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.366 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.366 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.366 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-04-21T08:51:22.366 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-04-21T08:51:22.366 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.366 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.366 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.366 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-04-21T08:51:22.366 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.366 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.366 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.366 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.367 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.367 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.367 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.367 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.367 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-04-21T08:51:22.367 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-04-21T08:51:22.367 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.367 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.367 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.367 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8080", 2024-04-21T08:51:22.367 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.367 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.368 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.368 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.368 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.368 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.368 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.368 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.368 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-04-21T08:51:22.368 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-04-21T08:51:22.368 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.368 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.368 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.368 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.368 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.369 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.369 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.369 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.369 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.369 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.369 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.369 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.369 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl_server_port": { 2024-04-21T08:51:22.369 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl_server_port", 2024-04-21T08:51:22.369 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.369 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.370 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.370 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8443", 2024-04-21T08:51:22.370 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.370 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.370 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.370 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.370 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.370 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.370 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.370 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.370 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-04-21T08:51:22.370 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-04-21T08:51:22.371 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.371 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.371 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.371 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "redirect", 2024-04-21T08:51:22.371 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.371 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.371 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.371 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.371 INFO:teuthology.orchestra.run.smithi086.stdout: "redirect" 2024-04-21T08:51:22.371 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.371 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.371 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.371 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.372 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.372 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.372 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-04-21T08:51:22.372 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-04-21T08:51:22.372 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.372 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.372 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.372 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-04-21T08:51:22.372 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-04-21T08:51:22.372 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-04-21T08:51:22.372 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.372 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.373 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.373 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.373 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.373 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.373 INFO:teuthology.orchestra.run.smithi086.stdout: "url_prefix": { 2024-04-21T08:51:22.373 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url_prefix", 2024-04-21T08:51:22.373 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.373 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.373 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.373 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.373 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.373 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.374 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.374 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.374 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.374 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.374 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.374 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.374 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.374 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.374 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.374 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "devicehealth", 2024-04-21T08:51:22.374 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.374 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.375 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.375 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_monitoring": { 2024-04-21T08:51:22.375 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_monitoring", 2024-04-21T08:51:22.375 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.375 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.375 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.375 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.375 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.375 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.375 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.375 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "monitor device health metrics", 2024-04-21T08:51:22.375 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.375 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.376 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.376 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.376 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.376 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.376 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.376 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.376 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.376 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.376 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.376 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.376 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.376 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.377 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.377 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.377 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.377 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.377 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.377 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.377 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.377 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.377 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.377 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.377 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.377 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.377 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.378 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.378 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.378 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.378 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.378 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.378 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.378 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.378 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.378 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.378 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.378 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.378 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.379 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.379 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.379 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.379 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.379 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.379 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.379 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.379 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.379 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.379 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.379 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.380 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.380 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.380 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.380 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.380 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.380 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.380 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.380 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.380 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.380 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.380 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.380 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.381 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.381 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.381 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.381 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.381 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.381 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.381 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.381 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.381 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.381 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.381 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.381 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.381 INFO:teuthology.orchestra.run.smithi086.stdout: "mark_out_threshold": { 2024-04-21T08:51:22.382 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mark_out_threshold", 2024-04-21T08:51:22.382 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:22.382 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.382 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.382 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "2419200", 2024-04-21T08:51:22.382 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.382 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.382 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.382 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "automatically mark OSD if it may fail before this long", 2024-04-21T08:51:22.382 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.382 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.382 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.383 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.383 INFO:teuthology.orchestra.run.smithi086.stdout: "pool_name": { 2024-04-21T08:51:22.383 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pool_name", 2024-04-21T08:51:22.383 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.383 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.383 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.383 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "device_health_metrics", 2024-04-21T08:51:22.383 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.383 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.383 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.383 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name of pool in which to store device health metrics", 2024-04-21T08:51:22.383 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.384 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.384 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.384 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.384 INFO:teuthology.orchestra.run.smithi086.stdout: "retention_period": { 2024-04-21T08:51:22.384 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "retention_period", 2024-04-21T08:51:22.384 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:22.384 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.384 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.384 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15552000", 2024-04-21T08:51:22.384 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.384 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.385 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.385 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long to retain device health metrics", 2024-04-21T08:51:22.385 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.385 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.385 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.385 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.385 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_frequency": { 2024-04-21T08:51:22.385 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_frequency", 2024-04-21T08:51:22.385 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:22.385 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.385 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.386 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-04-21T08:51:22.386 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.386 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.386 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.386 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to scrape device health metrics", 2024-04-21T08:51:22.386 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.386 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.386 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.386 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.386 INFO:teuthology.orchestra.run.smithi086.stdout: "self_heal": { 2024-04-21T08:51:22.386 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "self_heal", 2024-04-21T08:51:22.387 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.387 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.387 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.387 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.387 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.387 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.387 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.387 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "preemptively heal cluster around devices that may fail", 2024-04-21T08:51:22.387 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.387 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.387 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.387 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.387 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-21T08:51:22.388 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-21T08:51:22.388 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:22.388 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.388 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.388 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-21T08:51:22.388 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.388 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.388 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.388 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how frequently to wake up and check device health", 2024-04-21T08:51:22.388 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.388 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.388 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.389 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.389 INFO:teuthology.orchestra.run.smithi086.stdout: "warn_threshold": { 2024-04-21T08:51:22.389 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "warn_threshold", 2024-04-21T08:51:22.389 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:22.389 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.389 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.389 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7257600", 2024-04-21T08:51:22.389 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.389 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.389 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.389 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "raise health warning if OSD may fail before this long", 2024-04-21T08:51:22.389 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.390 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.390 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.390 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.390 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.390 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.390 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.390 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "diskprediction_local", 2024-04-21T08:51:22.390 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.390 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.390 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.390 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.390 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.391 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.391 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.391 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.391 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.391 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.391 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.391 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.391 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.391 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.391 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.391 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.391 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.392 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.392 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.392 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.392 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.392 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.392 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.392 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.392 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.392 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.392 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.392 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.392 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.393 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.393 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.393 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.393 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.393 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.393 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.393 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.393 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.393 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.393 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.393 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.393 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.394 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.394 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.394 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.394 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.394 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.394 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.394 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.394 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.394 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.394 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.394 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.394 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.395 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.395 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.395 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.395 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.395 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.395 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.395 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.395 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.395 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.395 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.395 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.396 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.396 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.396 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.396 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.396 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.396 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.396 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.396 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.396 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.396 INFO:teuthology.orchestra.run.smithi086.stdout: "predict_interval": { 2024-04-21T08:51:22.396 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predict_interval", 2024-04-21T08:51:22.396 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.397 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.397 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.397 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "86400", 2024-04-21T08:51:22.397 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.397 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.397 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.397 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.397 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.397 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.397 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.397 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.397 INFO:teuthology.orchestra.run.smithi086.stdout: "predictor_model": { 2024-04-21T08:51:22.398 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "predictor_model", 2024-04-21T08:51:22.398 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.398 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.398 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.398 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "prophetstor", 2024-04-21T08:51:22.398 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.398 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.398 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.398 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.398 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.398 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.398 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.398 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.399 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-21T08:51:22.399 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-21T08:51:22.399 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.399 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.399 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.399 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "600", 2024-04-21T08:51:22.399 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.399 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.399 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.399 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.399 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.399 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.400 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.400 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.400 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.400 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.400 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.400 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "influx", 2024-04-21T08:51:22.400 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": false, 2024-04-21T08:51:22.400 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "influxdb python module not found", 2024-04-21T08:51:22.400 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.400 INFO:teuthology.orchestra.run.smithi086.stdout: "batch_size": { 2024-04-21T08:51:22.400 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "batch_size", 2024-04-21T08:51:22.400 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.400 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.401 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.401 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5000", 2024-04-21T08:51:22.401 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.401 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.401 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.401 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How big batches of data points should be when sending to InfluxDB.", 2024-04-21T08:51:22.401 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.401 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.401 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.401 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.401 INFO:teuthology.orchestra.run.smithi086.stdout: "database": { 2024-04-21T08:51:22.401 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "database", 2024-04-21T08:51:22.401 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.402 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.402 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.402 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "ceph", 2024-04-21T08:51:22.402 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.402 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.402 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.402 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.", 2024-04-21T08:51:22.402 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.402 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.402 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.402 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.402 INFO:teuthology.orchestra.run.smithi086.stdout: "hostname": { 2024-04-21T08:51:22.403 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "hostname", 2024-04-21T08:51:22.403 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.403 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.403 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.403 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.403 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.403 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.403 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.403 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server hostname", 2024-04-21T08:51:22.403 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.403 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.403 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.404 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.404 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-21T08:51:22.404 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-21T08:51:22.404 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:22.404 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.404 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.404 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "30", 2024-04-21T08:51:22.404 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "5", 2024-04-21T08:51:22.404 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.404 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.404 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Time between reports to InfluxDB. Default 30 seconds.", 2024-04-21T08:51:22.404 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.404 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.405 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.405 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.405 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.405 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.405 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.405 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.405 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.405 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.405 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.405 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.405 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.405 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.406 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.406 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.406 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.406 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.406 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.406 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.406 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.406 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.406 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.406 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.406 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.406 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.407 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.407 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.407 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.407 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.407 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.407 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.407 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.407 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.407 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.407 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.407 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.407 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.408 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.408 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.408 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.408 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.408 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.408 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.408 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.408 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.408 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.408 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.408 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.408 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.408 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.409 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.409 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.409 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.409 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.409 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.409 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.409 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.409 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.409 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.409 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.409 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.410 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.410 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.410 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.410 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.410 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.410 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.410 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.410 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.410 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.410 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.410 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.410 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.411 INFO:teuthology.orchestra.run.smithi086.stdout: "password": { 2024-04-21T08:51:22.411 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "password", 2024-04-21T08:51:22.411 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.411 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.411 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.411 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.411 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.411 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.411 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.411 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "password of InfluxDB server user", 2024-04-21T08:51:22.411 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.411 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.412 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.412 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.412 INFO:teuthology.orchestra.run.smithi086.stdout: "port": { 2024-04-21T08:51:22.412 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "port", 2024-04-21T08:51:22.412 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.412 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.412 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.412 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "8086", 2024-04-21T08:51:22.412 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.412 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.412 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.412 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "InfluxDB server port", 2024-04-21T08:51:22.413 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.413 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.413 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.413 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.413 INFO:teuthology.orchestra.run.smithi086.stdout: "ssl": { 2024-04-21T08:51:22.413 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ssl", 2024-04-21T08:51:22.413 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.413 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.413 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.413 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "false", 2024-04-21T08:51:22.413 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.413 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.413 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.414 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Use https connection for InfluxDB server. Use \"true\" or \"false\".", 2024-04-21T08:51:22.414 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.414 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.414 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.414 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.414 INFO:teuthology.orchestra.run.smithi086.stdout: "threads": { 2024-04-21T08:51:22.414 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threads", 2024-04-21T08:51:22.414 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.414 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.414 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.414 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-21T08:51:22.415 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-21T08:51:22.415 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "32", 2024-04-21T08:51:22.415 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.415 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "How many worker threads should be spawned for sending data to InfluxDB.", 2024-04-21T08:51:22.415 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.415 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.415 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.415 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.415 INFO:teuthology.orchestra.run.smithi086.stdout: "username": { 2024-04-21T08:51:22.415 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "username", 2024-04-21T08:51:22.415 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.415 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.416 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.416 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.416 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.416 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.416 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.416 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "username of InfluxDB server user", 2024-04-21T08:51:22.416 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.416 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.416 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.416 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.416 INFO:teuthology.orchestra.run.smithi086.stdout: "verify_ssl": { 2024-04-21T08:51:22.416 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "verify_ssl", 2024-04-21T08:51:22.417 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.417 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.417 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.417 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "true", 2024-04-21T08:51:22.417 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.417 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.417 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.417 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Verify https cert for InfluxDB server. Use \"true\" or \"false\".", 2024-04-21T08:51:22.417 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.417 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.417 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.417 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.417 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.418 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.418 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.418 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "insights", 2024-04-21T08:51:22.418 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.418 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.418 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.418 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.418 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.418 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.418 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.418 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.418 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.419 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.419 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.419 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.419 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.419 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.419 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.419 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.419 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.419 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.419 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.419 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.419 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.420 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.420 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.420 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.420 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.420 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.420 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.420 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.420 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.420 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.420 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.420 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.420 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.421 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.421 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.421 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.421 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.421 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.421 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.421 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.421 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.421 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.421 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.421 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.421 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.421 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.422 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.422 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.422 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.422 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.422 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.422 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.422 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.422 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.422 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.422 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.422 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.422 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.423 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.423 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.423 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.423 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.423 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.423 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.423 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.423 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.423 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.423 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.423 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.423 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.423 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.424 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.424 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.424 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.424 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.424 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.424 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "iostat", 2024-04-21T08:51:22.424 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.424 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.424 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.424 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.424 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.424 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.425 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.425 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.425 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.425 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.425 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.425 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.425 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.425 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.425 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.425 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.425 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.426 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.426 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.426 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.426 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.426 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.426 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.426 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.426 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.426 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.427 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.427 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.427 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.427 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.427 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.427 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.427 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.427 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.427 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.427 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.427 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.427 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.427 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.428 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.428 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.428 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.428 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.428 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.428 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.428 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.428 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.428 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.428 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.428 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.428 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.428 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.429 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.429 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.429 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.429 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.429 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.429 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.429 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.429 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.429 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.429 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.429 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.430 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.430 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.430 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.430 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.430 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.430 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.430 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.430 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.430 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.430 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.430 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.431 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.431 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.431 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "k8sevents", 2024-04-21T08:51:22.431 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.431 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.431 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.431 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_event_retention_days": { 2024-04-21T08:51:22.431 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "ceph_event_retention_days", 2024-04-21T08:51:22.431 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.431 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.431 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.431 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "7", 2024-04-21T08:51:22.431 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.432 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.432 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.432 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Days to hold ceph event information within local cache", 2024-04-21T08:51:22.432 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.432 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.432 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.432 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.432 INFO:teuthology.orchestra.run.smithi086.stdout: "config_check_secs": { 2024-04-21T08:51:22.432 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "config_check_secs", 2024-04-21T08:51:22.432 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.432 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.432 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.432 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-21T08:51:22.433 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "10", 2024-04-21T08:51:22.433 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.433 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.433 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "interval (secs) to check for cluster configuration changes", 2024-04-21T08:51:22.433 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.433 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.433 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.433 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.433 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.433 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.433 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.433 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.434 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.434 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.434 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.434 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.434 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.434 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.434 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.434 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.434 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.434 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.434 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.434 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.435 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.435 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.435 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.435 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.435 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.435 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.435 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.435 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.435 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.435 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.435 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.435 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.436 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.436 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.436 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.436 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.436 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.436 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.436 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.436 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.436 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.436 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.436 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.436 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.437 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.437 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.437 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.437 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.437 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.437 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.437 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.437 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.437 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.437 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.437 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.437 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.437 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.438 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.438 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.438 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.438 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.438 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.438 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.438 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.438 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.438 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.438 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.438 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.438 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.439 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.439 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.439 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.439 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.439 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.439 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.439 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.439 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.439 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "localpool", 2024-04-21T08:51:22.439 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.439 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.439 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.439 INFO:teuthology.orchestra.run.smithi086.stdout: "failure_domain": { 2024-04-21T08:51:22.440 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "failure_domain", 2024-04-21T08:51:22.440 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.440 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.440 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.440 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "host", 2024-04-21T08:51:22.440 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.440 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.440 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.440 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "failure domain for any created local pool", 2024-04-21T08:51:22.440 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "what failure domain we should separate data replicas across.", 2024-04-21T08:51:22.440 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.440 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.440 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.440 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.441 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.441 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.441 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.441 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.441 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.441 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.441 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.441 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.441 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.441 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.441 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.441 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.441 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.442 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.442 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.442 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.442 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.442 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.442 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.442 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.442 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.442 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.442 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.442 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.442 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.443 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.443 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.443 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.443 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.443 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.443 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.443 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.443 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.443 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.443 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.443 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.443 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.444 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.444 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.444 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.444 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.444 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.444 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.444 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.444 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.444 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.444 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.444 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.444 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.445 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.445 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.445 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.445 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.445 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.445 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.445 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.445 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.445 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.445 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.445 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.445 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.446 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.446 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.446 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.446 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.446 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.446 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.446 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.446 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.446 INFO:teuthology.orchestra.run.smithi086.stdout: "min_size": { 2024-04-21T08:51:22.446 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "min_size", 2024-04-21T08:51:22.446 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.447 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.447 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.447 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.447 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.447 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.447 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.447 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default min_size for any created local pool", 2024-04-21T08:51:22.447 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "value to set min_size to (unchanged from Ceph's default if this option is not set)", 2024-04-21T08:51:22.447 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.447 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.447 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.447 INFO:teuthology.orchestra.run.smithi086.stdout: "num_rep": { 2024-04-21T08:51:22.447 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "num_rep", 2024-04-21T08:51:22.448 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.448 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.448 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.448 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3", 2024-04-21T08:51:22.448 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.448 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.448 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.448 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default replica count for any created local pool", 2024-04-21T08:51:22.448 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.448 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.448 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.448 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.448 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_num": { 2024-04-21T08:51:22.448 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_num", 2024-04-21T08:51:22.449 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.449 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.449 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.449 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "128", 2024-04-21T08:51:22.449 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.449 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.449 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.449 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "default pg_num for any created local pool", 2024-04-21T08:51:22.449 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.449 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.449 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.450 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.450 INFO:teuthology.orchestra.run.smithi086.stdout: "prefix": { 2024-04-21T08:51:22.450 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prefix", 2024-04-21T08:51:22.450 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.450 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.450 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.450 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.450 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.450 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.450 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.450 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "name prefix for any created local pool", 2024-04-21T08:51:22.451 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.451 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.451 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.451 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.451 INFO:teuthology.orchestra.run.smithi086.stdout: "subtree": { 2024-04-21T08:51:22.451 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "subtree", 2024-04-21T08:51:22.451 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.451 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.451 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.451 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "rack", 2024-04-21T08:51:22.451 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.451 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.452 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.452 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "CRUSH level for which to create a local pool", 2024-04-21T08:51:22.452 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "which CRUSH subtree type the module should create a pool for.", 2024-04-21T08:51:22.452 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.452 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.452 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.452 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.452 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.452 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.452 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mds_autoscaler", 2024-04-21T08:51:22.452 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.453 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.453 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.453 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.453 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.453 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.453 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.453 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.453 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.453 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.453 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.453 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.453 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.454 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.454 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.454 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.454 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.454 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.454 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.454 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.454 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.454 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.454 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.454 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.455 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.455 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.455 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.455 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.455 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.455 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.455 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.455 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.455 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.455 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.455 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.455 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.456 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.456 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.456 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.456 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.456 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.456 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.456 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.456 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.456 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.456 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.457 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.457 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.457 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.457 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.457 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.457 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.457 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.457 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.457 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.457 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.457 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.457 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.457 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.458 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.458 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.458 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.458 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.458 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.458 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.458 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.458 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.458 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.458 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.458 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.458 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.459 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.459 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.459 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.459 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.459 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.459 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirroring", 2024-04-21T08:51:22.459 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.459 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.459 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.459 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.459 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.459 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.460 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.460 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.460 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.460 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.460 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.460 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.460 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.460 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.460 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.460 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.460 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.460 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.460 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.461 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.461 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.461 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.461 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.461 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.461 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.461 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.461 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.461 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.461 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.461 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.461 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.462 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.462 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.462 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.462 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.462 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.462 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.462 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.462 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.462 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.462 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.462 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.463 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.463 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.463 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.463 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.463 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.463 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.463 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.463 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.463 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.463 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.463 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.463 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.464 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.464 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.464 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.464 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.464 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.464 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.464 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.464 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.464 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.464 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.464 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.464 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.465 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.465 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.465 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.465 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.465 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.465 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.465 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.465 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.465 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.465 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.465 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "nfs", 2024-04-21T08:51:22.465 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.466 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.466 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.466 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.466 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.466 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.466 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.466 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.466 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.466 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.466 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.466 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.467 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.467 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.467 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.467 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.467 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.467 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.467 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.467 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.467 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.467 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.467 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.467 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.468 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.468 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.468 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.468 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.468 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.468 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.468 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.468 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.468 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.468 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.468 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.469 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.469 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.469 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.469 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.469 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.469 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.469 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.469 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.469 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.469 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.469 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.469 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.470 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.470 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.470 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.470 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.470 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.470 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.470 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.470 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.470 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.470 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.470 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.471 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.471 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.471 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.471 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.471 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.471 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.471 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.471 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.471 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.471 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.471 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.471 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.471 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.472 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.472 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.472 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.472 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.472 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.472 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-04-21T08:51:22.472 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.472 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.472 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.472 INFO:teuthology.orchestra.run.smithi086.stdout: "fail_fs": { 2024-04-21T08:51:22.472 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "fail_fs", 2024-04-21T08:51:22.472 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.473 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.473 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.473 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.473 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.473 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.473 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.473 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Fail filesystem for rapid multi-rank mds upgrade", 2024-04-21T08:51:22.473 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.473 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.473 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.473 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.474 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.474 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.474 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.474 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.474 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.474 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.474 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.474 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.474 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.474 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.474 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.474 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.475 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.475 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.475 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.475 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.475 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.475 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.475 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.475 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.475 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.475 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.475 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.476 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.476 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.476 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.476 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.476 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.476 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.476 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.476 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.476 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.476 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.476 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.476 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.477 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.477 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.477 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.477 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.477 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.477 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.477 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.477 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.477 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.477 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.477 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.477 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.478 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.478 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.478 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.478 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.478 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.478 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.478 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.478 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.478 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.478 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.478 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.478 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.479 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.479 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.479 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.479 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.479 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.479 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.479 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.479 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.479 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.479 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.479 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.480 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator": { 2024-04-21T08:51:22.480 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "orchestrator", 2024-04-21T08:51:22.480 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.480 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.480 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.480 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.480 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.480 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.480 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.480 INFO:teuthology.orchestra.run.smithi086.stdout: "cephadm", 2024-04-21T08:51:22.480 INFO:teuthology.orchestra.run.smithi086.stdout: "rook", 2024-04-21T08:51:22.480 INFO:teuthology.orchestra.run.smithi086.stdout: "test_orchestrator" 2024-04-21T08:51:22.480 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.481 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Orchestrator backend", 2024-04-21T08:51:22.481 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.481 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.481 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.481 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.481 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.481 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.481 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.481 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_perf_query", 2024-04-21T08:51:22.481 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.481 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.481 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.482 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.482 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.482 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.482 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.482 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.482 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.482 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.482 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.482 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.482 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.482 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.483 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.483 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.483 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.483 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.483 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.483 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.483 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.483 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.483 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.483 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.483 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.483 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.483 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.484 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.484 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.484 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.484 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.484 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.484 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.484 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.484 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.484 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.484 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.484 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.484 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.484 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.485 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.485 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.485 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.485 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.485 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.485 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.485 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.485 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.485 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.485 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.485 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.486 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.486 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.486 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.486 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.486 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.486 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.486 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.486 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.486 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.486 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.486 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.486 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.487 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.487 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.487 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.487 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.487 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.487 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.487 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.487 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.487 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.487 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.487 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.487 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.488 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.488 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "osd_support", 2024-04-21T08:51:22.488 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.488 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.488 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.488 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.488 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.488 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.488 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.488 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.488 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.488 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.488 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.489 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.489 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.489 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.489 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.489 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.489 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.489 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.489 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.489 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.489 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.489 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.490 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.490 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.490 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.490 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.490 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.490 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.490 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.490 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.490 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.490 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.490 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.490 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.490 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.491 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.491 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.491 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.491 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.491 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.491 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.491 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.491 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.491 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.491 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.491 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.491 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.492 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.492 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.492 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.492 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.492 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.492 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.492 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.492 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.492 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.492 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.492 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.493 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.493 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.493 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.493 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.493 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.493 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.493 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.493 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.493 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.493 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.493 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.494 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.494 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.494 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.494 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.494 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.494 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.494 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.494 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "pg_autoscaler", 2024-04-21T08:51:22.494 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.494 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.494 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.495 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.495 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.495 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.495 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.495 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.495 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.495 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.495 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.495 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.495 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.495 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.495 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.496 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.496 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.496 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.496 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.496 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.496 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.496 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.496 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.496 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.496 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.496 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.496 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.497 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.497 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.497 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.497 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.497 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.497 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.497 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.497 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.497 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.497 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.497 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.497 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.498 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.498 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.498 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.498 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.498 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.498 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.498 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.498 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.498 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.498 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.499 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.499 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.499 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.499 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.499 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.499 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.499 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.499 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.499 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.499 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.499 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.500 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.500 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.500 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.500 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.500 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.500 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.500 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.500 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.500 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.500 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.500 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.501 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.501 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.501 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-21T08:51:22.501 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-21T08:51:22.501 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:22.501 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.501 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.501 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-21T08:51:22.501 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.501 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.501 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.501 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.502 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.502 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.502 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.502 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.502 INFO:teuthology.orchestra.run.smithi086.stdout: "threshold": { 2024-04-21T08:51:22.502 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "threshold", 2024-04-21T08:51:22.502 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-21T08:51:22.502 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.502 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.502 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "3.0", 2024-04-21T08:51:22.502 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1.0", 2024-04-21T08:51:22.503 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.503 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.503 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "scaling threshold", 2024-04-21T08:51:22.503 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0", 2024-04-21T08:51:22.503 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.503 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.503 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.503 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.503 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.503 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.503 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "progress", 2024-04-21T08:51:22.503 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.504 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.504 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.504 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_pg_recovery_event": { 2024-04-21T08:51:22.504 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_pg_recovery_event", 2024-04-21T08:51:22.504 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.504 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.504 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.504 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.504 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.504 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.504 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.504 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow the module to show pg recovery progress", 2024-04-21T08:51:22.505 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.505 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.505 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.505 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.505 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-04-21T08:51:22.505 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-04-21T08:51:22.505 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.505 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.505 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.505 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.505 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.505 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.506 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.506 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.506 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.506 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.506 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.506 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.506 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.506 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.506 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.506 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.506 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.507 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.507 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.507 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.507 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.507 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.507 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.507 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.507 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.507 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.507 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.507 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.507 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.508 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.508 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.508 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.508 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.508 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.508 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.508 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.508 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.508 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.508 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.508 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.509 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.509 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.509 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.509 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.509 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.509 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.509 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.509 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.509 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.509 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.509 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.509 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.510 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.510 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.510 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.510 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.510 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.510 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.510 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.510 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.510 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.510 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.511 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.511 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.511 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.511 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.511 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.511 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.511 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.511 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.511 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.511 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.511 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.511 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.511 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.512 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.512 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.512 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.512 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.512 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.512 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.512 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.512 INFO:teuthology.orchestra.run.smithi086.stdout: "max_completed_events": { 2024-04-21T08:51:22.512 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_completed_events", 2024-04-21T08:51:22.512 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.512 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.512 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.513 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "50", 2024-04-21T08:51:22.513 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.513 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.513 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.513 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "number of past completed events to remember", 2024-04-21T08:51:22.513 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.513 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.513 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.513 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.513 INFO:teuthology.orchestra.run.smithi086.stdout: "sleep_interval": { 2024-04-21T08:51:22.513 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "sleep_interval", 2024-04-21T08:51:22.513 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:22.514 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.514 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.514 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "5", 2024-04-21T08:51:22.514 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.514 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.514 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.514 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "how long the module is going to sleep", 2024-04-21T08:51:22.514 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.514 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.514 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.514 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.515 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.515 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.515 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.515 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "prometheus", 2024-04-21T08:51:22.515 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.515 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.515 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.515 INFO:teuthology.orchestra.run.smithi086.stdout: "cache": { 2024-04-21T08:51:22.515 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "cache", 2024-04-21T08:51:22.515 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.515 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.516 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.516 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.516 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.516 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.516 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.516 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.516 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.516 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.516 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.516 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.516 INFO:teuthology.orchestra.run.smithi086.stdout: "exclude_perf_counters": { 2024-04-21T08:51:22.517 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "exclude_perf_counters", 2024-04-21T08:51:22.517 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.517 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.517 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.517 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.517 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.517 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.517 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.517 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Do not include perf-counters in the metrics output", 2024-04-21T08:51:22.517 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.", 2024-04-21T08:51:22.517 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.517 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.518 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.518 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.518 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.518 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.518 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.518 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.518 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.518 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.518 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.518 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.518 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.519 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.519 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.519 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.519 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.519 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.519 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.519 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.519 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.519 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.519 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.519 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.519 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.519 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.519 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.520 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.520 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.520 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.520 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.520 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.520 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.520 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.520 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.520 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.520 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.520 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.520 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.520 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.521 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.521 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.521 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.521 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.521 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.521 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.521 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.521 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.521 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.521 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.521 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.522 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.522 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.522 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.522 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.522 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.522 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.522 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.522 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.522 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.522 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.522 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.523 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.523 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.523 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.523 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.523 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.523 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.523 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.523 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.523 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.523 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.523 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.523 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools": { 2024-04-21T08:51:22.523 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools", 2024-04-21T08:51:22.524 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.524 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.524 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.524 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.524 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.524 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.524 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.524 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.524 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.524 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.525 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.525 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.525 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_stats_pools_refresh_interval": { 2024-04-21T08:51:22.525 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_stats_pools_refresh_interval", 2024-04-21T08:51:22.525 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.525 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.525 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.525 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "300", 2024-04-21T08:51:22.525 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.525 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.526 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.526 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.526 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.526 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.526 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.526 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.526 INFO:teuthology.orchestra.run.smithi086.stdout: "scrape_interval": { 2024-04-21T08:51:22.526 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "scrape_interval", 2024-04-21T08:51:22.526 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-21T08:51:22.526 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.526 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.527 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15.0", 2024-04-21T08:51:22.527 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.527 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.527 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.527 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.527 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.527 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.527 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.527 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.527 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-04-21T08:51:22.527 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-04-21T08:51:22.527 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.528 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.528 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.528 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "::", 2024-04-21T08:51:22.528 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.528 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.528 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.528 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the IPv4 or IPv6 address on which the module listens for HTTP requests", 2024-04-21T08:51:22.528 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.528 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.528 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.528 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.528 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-04-21T08:51:22.529 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-04-21T08:51:22.529 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.529 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.529 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.529 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "9283", 2024-04-21T08:51:22.529 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.529 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.529 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.529 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "the port on which the module listens for HTTP requests", 2024-04-21T08:51:22.529 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.529 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.529 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.530 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.530 INFO:teuthology.orchestra.run.smithi086.stdout: "stale_cache_strategy": { 2024-04-21T08:51:22.530 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stale_cache_strategy", 2024-04-21T08:51:22.530 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.530 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.530 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.530 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "log", 2024-04-21T08:51:22.530 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.530 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.530 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.530 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.530 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.531 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.531 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.531 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.531 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_behaviour": { 2024-04-21T08:51:22.531 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_behaviour", 2024-04-21T08:51:22.531 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.531 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.531 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.531 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "default", 2024-04-21T08:51:22.531 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.531 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.531 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.531 INFO:teuthology.orchestra.run.smithi086.stdout: "default", 2024-04-21T08:51:22.532 INFO:teuthology.orchestra.run.smithi086.stdout: "error" 2024-04-21T08:51:22.532 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.532 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.532 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.532 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.532 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.532 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.532 INFO:teuthology.orchestra.run.smithi086.stdout: "standby_error_status_code": { 2024-04-21T08:51:22.532 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "standby_error_status_code", 2024-04-21T08:51:22.532 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.532 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.532 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.533 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "500", 2024-04-21T08:51:22.533 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "400", 2024-04-21T08:51:22.533 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "599", 2024-04-21T08:51:22.533 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.533 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.533 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.533 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.533 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.533 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.533 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.533 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.533 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.533 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_support", 2024-04-21T08:51:22.534 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.534 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.534 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.534 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.534 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.534 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.534 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.534 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.534 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.534 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.534 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.534 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.535 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.535 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.535 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.535 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.535 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.535 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.535 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.535 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.535 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.535 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.535 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.536 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.536 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.536 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.536 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.536 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.536 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.536 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.536 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.536 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.536 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.536 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.536 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.536 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.537 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.537 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.537 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.537 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.537 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.537 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.537 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.537 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.537 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.537 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.537 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.537 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.538 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.538 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.538 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.538 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.538 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.538 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.538 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.538 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.538 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.538 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.538 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.538 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.539 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.539 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.539 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.539 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.539 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.539 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.539 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.539 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.539 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.539 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.539 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.539 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.540 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.540 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_snap_create": { 2024-04-21T08:51:22.540 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_snap_create", 2024-04-21T08:51:22.540 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.540 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.540 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.540 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10", 2024-04-21T08:51:22.540 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.540 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.540 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.540 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.540 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.541 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.541 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.541 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.541 INFO:teuthology.orchestra.run.smithi086.stdout: "mirror_snapshot_schedule": { 2024-04-21T08:51:22.541 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "mirror_snapshot_schedule", 2024-04-21T08:51:22.541 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.541 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.541 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.541 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.541 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.541 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.541 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.541 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.541 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.542 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.542 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.542 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.542 INFO:teuthology.orchestra.run.smithi086.stdout: "trash_purge_schedule": { 2024-04-21T08:51:22.542 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "trash_purge_schedule", 2024-04-21T08:51:22.542 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.542 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.542 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.542 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.542 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.542 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.542 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.543 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.543 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.543 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.543 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.543 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.543 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.543 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.543 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.543 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "restful", 2024-04-21T08:51:22.543 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.543 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.543 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.543 INFO:teuthology.orchestra.run.smithi086.stdout: "enable_auth": { 2024-04-21T08:51:22.543 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enable_auth", 2024-04-21T08:51:22.544 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.544 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.544 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.544 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.544 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.544 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.544 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.544 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.544 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.544 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.544 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.544 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.544 INFO:teuthology.orchestra.run.smithi086.stdout: "key_file": { 2024-04-21T08:51:22.545 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "key_file", 2024-04-21T08:51:22.545 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.545 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.545 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.545 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.545 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.545 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.545 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.545 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.545 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.545 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.545 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.545 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.546 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.546 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.546 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.546 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.546 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.546 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.546 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.546 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.546 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.546 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.546 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.546 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.547 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.547 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.547 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.547 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.547 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.547 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.547 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.547 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.547 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.547 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.547 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.547 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.548 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.548 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.548 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.548 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.548 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.548 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.548 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.548 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.548 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.548 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.548 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.549 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.549 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.549 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.549 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.549 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.549 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.549 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.549 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.549 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.549 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.549 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.549 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.550 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.550 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.550 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.550 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.550 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.550 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.550 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.550 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.550 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.550 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.551 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.551 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.551 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.551 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.551 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.551 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.551 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.551 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.551 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.551 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.551 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.551 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.551 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.552 INFO:teuthology.orchestra.run.smithi086.stdout: "server_addr": { 2024-04-21T08:51:22.552 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_addr", 2024-04-21T08:51:22.552 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.552 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.552 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.552 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.552 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.552 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.552 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.552 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.552 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.553 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.553 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.553 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.553 INFO:teuthology.orchestra.run.smithi086.stdout: "server_port": { 2024-04-21T08:51:22.553 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "server_port", 2024-04-21T08:51:22.553 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.553 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.553 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.553 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.553 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.553 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.553 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.553 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.554 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.554 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.554 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.554 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.554 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.554 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.554 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.554 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rgw", 2024-04-21T08:51:22.554 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.554 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.554 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.555 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.555 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.555 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.555 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.555 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.555 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.555 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.555 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.555 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.555 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.555 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.556 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.556 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.556 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.556 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.556 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.556 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.556 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.556 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.556 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.556 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.556 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.556 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.557 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.557 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.557 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.557 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.557 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.557 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.557 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.557 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.557 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.557 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.558 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.558 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.558 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.558 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.558 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.558 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.558 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.558 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.558 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.558 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.559 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.559 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.559 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.559 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.559 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.559 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.559 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.559 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.559 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.559 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.559 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.560 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.560 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.560 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.560 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.560 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.560 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.560 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.560 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.560 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.560 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.560 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.560 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.561 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.561 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.561 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.561 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.561 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.561 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.561 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.561 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rook", 2024-04-21T08:51:22.561 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.561 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.561 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.561 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.562 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.562 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.562 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.562 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.562 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.562 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.562 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.562 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.562 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.562 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.562 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.563 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.563 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.563 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.563 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.563 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.563 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.563 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.563 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.563 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.563 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.563 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.563 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.564 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.564 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.564 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.564 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.564 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.564 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.564 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.564 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.564 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.564 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.564 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.564 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.565 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.565 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.565 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.565 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.565 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.565 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.565 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.565 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.565 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.565 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.565 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.566 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.566 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.566 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.566 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.566 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.566 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.566 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.566 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.566 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.566 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.566 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.566 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.567 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.567 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.567 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.567 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.567 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.567 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.567 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.567 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.568 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.568 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.568 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.568 INFO:teuthology.orchestra.run.smithi086.stdout: "storage_class": { 2024-04-21T08:51:22.568 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "storage_class", 2024-04-21T08:51:22.568 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.568 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.568 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.568 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "local", 2024-04-21T08:51:22.568 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.568 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.568 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.568 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "storage class name for LSO-discovered PVs", 2024-04-21T08:51:22.569 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.569 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.569 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.569 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.569 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.569 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.569 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.569 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "selftest", 2024-04-21T08:51:22.569 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.569 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.569 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.569 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.570 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.570 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.570 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.570 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.570 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.570 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.570 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.570 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.570 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.570 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.570 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.571 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.571 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.571 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.571 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.571 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.571 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.571 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.571 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.571 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.571 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.571 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.571 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.571 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.571 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.572 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.572 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.572 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.572 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.572 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.572 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.572 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.572 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.572 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.572 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.572 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.572 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.572 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.573 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.573 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.573 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.573 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.573 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.573 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.573 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.573 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.573 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.573 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.573 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.573 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.573 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.574 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.574 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.574 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.574 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.574 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.574 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.574 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.574 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.574 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.574 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.574 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.575 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.575 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.575 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.575 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.575 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.575 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.575 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.575 INFO:teuthology.orchestra.run.smithi086.stdout: "roption1": { 2024-04-21T08:51:22.575 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption1", 2024-04-21T08:51:22.575 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.575 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.575 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.575 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.576 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.576 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.576 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.576 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.576 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.576 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.576 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.576 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.576 INFO:teuthology.orchestra.run.smithi086.stdout: "roption2": { 2024-04-21T08:51:22.576 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "roption2", 2024-04-21T08:51:22.576 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.576 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.577 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.577 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "xyz", 2024-04-21T08:51:22.577 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.577 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.577 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.577 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.577 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.577 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.577 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.577 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.577 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption1": { 2024-04-21T08:51:22.577 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption1", 2024-04-21T08:51:22.578 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.578 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.578 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.578 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.578 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.578 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.578 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.578 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.578 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.578 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.579 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.579 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.579 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption2": { 2024-04-21T08:51:22.579 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption2", 2024-04-21T08:51:22.579 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.579 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.579 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.579 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.579 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.579 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.579 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.579 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.580 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.580 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.580 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.581 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.581 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption3": { 2024-04-21T08:51:22.581 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption3", 2024-04-21T08:51:22.581 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "float", 2024-04-21T08:51:22.581 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.581 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.582 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.582 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.582 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.582 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.582 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.582 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.582 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.582 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.582 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.582 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption4": { 2024-04-21T08:51:22.582 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption4", 2024-04-21T08:51:22.582 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.583 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.583 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.583 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.583 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.583 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.583 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.583 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.583 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.583 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.583 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.583 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.583 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption5": { 2024-04-21T08:51:22.583 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption5", 2024-04-21T08:51:22.583 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.584 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.584 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.584 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.584 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.584 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.584 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.584 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.584 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.584 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.584 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.584 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.584 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption6": { 2024-04-21T08:51:22.585 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption6", 2024-04-21T08:51:22.585 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.585 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.585 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.585 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.585 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.585 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.585 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.585 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.585 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.585 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.586 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.586 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.586 INFO:teuthology.orchestra.run.smithi086.stdout: "rwoption7": { 2024-04-21T08:51:22.586 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rwoption7", 2024-04-21T08:51:22.586 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.586 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.586 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.586 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.586 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "1", 2024-04-21T08:51:22.586 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "42", 2024-04-21T08:51:22.586 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.586 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.587 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.587 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.587 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.587 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.587 INFO:teuthology.orchestra.run.smithi086.stdout: "testkey": { 2024-04-21T08:51:22.587 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testkey", 2024-04-21T08:51:22.588 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.588 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.588 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.588 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.588 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.588 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.588 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.588 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.588 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.588 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.589 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.589 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.589 INFO:teuthology.orchestra.run.smithi086.stdout: "testlkey": { 2024-04-21T08:51:22.589 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testlkey", 2024-04-21T08:51:22.589 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.589 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.589 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.589 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.589 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.589 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.589 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.589 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.589 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.590 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.590 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.590 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.590 INFO:teuthology.orchestra.run.smithi086.stdout: "testnewline": { 2024-04-21T08:51:22.590 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "testnewline", 2024-04-21T08:51:22.590 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.590 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.590 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.590 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.590 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.591 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.591 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.591 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.591 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.591 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.591 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.591 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.591 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.591 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.591 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.591 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snap_schedule", 2024-04-21T08:51:22.591 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.592 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.592 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.592 INFO:teuthology.orchestra.run.smithi086.stdout: "allow_m_granularity": { 2024-04-21T08:51:22.592 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "allow_m_granularity", 2024-04-21T08:51:22.592 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.592 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.592 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.592 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.592 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.592 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.592 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.592 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "allow minute scheduled snapshots", 2024-04-21T08:51:22.593 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.593 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.593 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.593 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.593 INFO:teuthology.orchestra.run.smithi086.stdout: "dump_on_update": { 2024-04-21T08:51:22.593 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "dump_on_update", 2024-04-21T08:51:22.593 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.593 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.593 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.593 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.593 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.593 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.593 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.594 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "dump database to debug log on update", 2024-04-21T08:51:22.594 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.594 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.594 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.594 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.594 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.594 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.594 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.594 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.594 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.594 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.594 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.595 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.595 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.595 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.595 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.595 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.595 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.595 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.595 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.595 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.595 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.595 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.595 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.596 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.596 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.596 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.596 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.596 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.596 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.596 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.596 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.596 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.596 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.596 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.596 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.597 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.597 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.597 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.597 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.597 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.597 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.597 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.597 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.597 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.597 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.597 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.598 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.598 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.598 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.598 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.598 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.598 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.598 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.598 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.598 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.598 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.598 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.599 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.599 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.599 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.599 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.599 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.599 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.599 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.599 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.599 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.599 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.599 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.599 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.600 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.600 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.600 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.600 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.600 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.600 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.600 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.600 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.600 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "stats", 2024-04-21T08:51:22.600 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.600 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.600 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.601 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.601 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.601 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.601 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.601 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.601 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.601 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.601 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.601 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.601 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.601 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.602 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.602 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.602 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.602 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.602 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.602 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.602 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.602 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.602 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.602 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.602 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.602 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.603 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.603 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.603 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.603 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.603 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.603 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.603 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.603 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.603 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.603 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.603 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.603 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.604 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.604 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.604 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.604 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.604 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.604 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.604 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.604 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.604 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.604 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.604 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.604 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.605 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.605 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.605 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.605 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.605 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.605 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.605 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.605 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.605 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.605 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.605 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.605 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.606 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.606 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.606 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.606 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.606 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.606 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.606 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.606 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.606 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.606 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.606 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.607 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.607 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.607 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.607 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "status", 2024-04-21T08:51:22.607 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.607 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.607 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.607 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.607 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.607 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.607 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.607 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.608 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.608 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.608 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.608 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.608 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.608 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.608 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.608 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.608 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.608 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.608 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.608 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.609 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.609 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.609 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.609 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.609 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.609 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.609 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.609 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.609 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.609 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.609 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.610 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.610 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.610 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.610 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.610 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.610 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.610 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.610 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.610 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.610 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.610 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.610 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.611 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.611 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.611 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.611 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.611 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.611 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.611 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.611 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.611 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.611 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.611 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.611 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.612 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.612 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.612 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.612 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.612 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.612 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.612 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.612 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.612 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.612 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.612 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.612 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.613 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.613 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.613 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.613 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.613 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.613 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.613 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telegraf", 2024-04-21T08:51:22.613 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.613 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.613 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.613 INFO:teuthology.orchestra.run.smithi086.stdout: "address": { 2024-04-21T08:51:22.614 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "address", 2024-04-21T08:51:22.614 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.614 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.614 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.614 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "unixgram:///tmp/telegraf.sock", 2024-04-21T08:51:22.614 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.614 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.614 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.614 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.614 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.614 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.615 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.615 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.615 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-21T08:51:22.615 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-21T08:51:22.615 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:22.615 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.615 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.615 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "15", 2024-04-21T08:51:22.615 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.615 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.615 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.616 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.616 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.616 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.616 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.616 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.616 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.616 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.616 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.616 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.616 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.616 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.616 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.617 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.617 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.617 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.617 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.617 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.617 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.617 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.617 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.617 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.617 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.617 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.618 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.618 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.618 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.618 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.618 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.618 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.618 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.618 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.618 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.619 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.619 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.619 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.619 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.619 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.619 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.619 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.619 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.619 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.620 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.620 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.620 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.620 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.620 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.620 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.620 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.620 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.620 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.621 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.621 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.621 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.621 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.621 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.621 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.621 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.621 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.621 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.621 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.621 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.622 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.622 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.622 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.622 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.622 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.622 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.622 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.622 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.622 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.622 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.622 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.623 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.623 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.623 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.623 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.623 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.623 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.623 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "telemetry", 2024-04-21T08:51:22.623 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.623 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.623 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.624 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_basic": { 2024-04-21T08:51:22.624 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_basic", 2024-04-21T08:51:22.624 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.624 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.624 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.624 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.624 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.624 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.624 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.624 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share basic cluster information (size, version)", 2024-04-21T08:51:22.624 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.625 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.625 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.625 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.625 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_crash": { 2024-04-21T08:51:22.625 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_crash", 2024-04-21T08:51:22.625 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.625 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.625 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.625 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.625 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.625 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.625 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.626 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share metadata about Ceph daemon crashes (version, stack straces, etc)", 2024-04-21T08:51:22.626 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.626 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.626 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.626 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_device": { 2024-04-21T08:51:22.626 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_device", 2024-04-21T08:51:22.626 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.626 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.626 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.626 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.627 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.627 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.627 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.627 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)", 2024-04-21T08:51:22.627 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.627 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.627 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.627 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_ident": { 2024-04-21T08:51:22.627 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_ident", 2024-04-21T08:51:22.627 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.627 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.628 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.628 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.628 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.628 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.628 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.628 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share a user-provided description and/or contact email for the cluster", 2024-04-21T08:51:22.628 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.628 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.628 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.628 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.628 INFO:teuthology.orchestra.run.smithi086.stdout: "channel_perf": { 2024-04-21T08:51:22.628 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "channel_perf", 2024-04-21T08:51:22.628 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.629 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.629 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.629 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.629 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.629 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.629 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.629 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Share various performance metrics of a cluster", 2024-04-21T08:51:22.629 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.629 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.629 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.629 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.629 INFO:teuthology.orchestra.run.smithi086.stdout: "contact": { 2024-04-21T08:51:22.629 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "contact", 2024-04-21T08:51:22.629 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.630 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.630 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.630 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.630 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.630 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.630 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.630 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.630 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.630 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.630 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.630 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.630 INFO:teuthology.orchestra.run.smithi086.stdout: "description": { 2024-04-21T08:51:22.631 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "description", 2024-04-21T08:51:22.631 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.631 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.631 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.631 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.631 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.631 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.631 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.631 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.631 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.631 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.631 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.632 INFO:teuthology.orchestra.run.smithi086.stdout: "device_url": { 2024-04-21T08:51:22.632 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "device_url", 2024-04-21T08:51:22.632 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.632 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.632 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.632 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/device", 2024-04-21T08:51:22.632 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.632 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.632 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.632 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.633 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.633 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.633 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.633 INFO:teuthology.orchestra.run.smithi086.stdout: "enabled": { 2024-04-21T08:51:22.633 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "enabled", 2024-04-21T08:51:22.633 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.633 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.633 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.633 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.633 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.633 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.633 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.634 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.634 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.634 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.634 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.634 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.634 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-21T08:51:22.634 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-21T08:51:22.634 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.634 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.634 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.634 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "24", 2024-04-21T08:51:22.634 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "8", 2024-04-21T08:51:22.635 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.635 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.635 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.635 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.635 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.635 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.635 INFO:teuthology.orchestra.run.smithi086.stdout: "last_opt_revision": { 2024-04-21T08:51:22.635 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "last_opt_revision", 2024-04-21T08:51:22.635 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.635 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.635 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.636 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "1", 2024-04-21T08:51:22.636 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.636 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.636 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.636 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.636 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.636 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.636 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.636 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard": { 2024-04-21T08:51:22.636 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard", 2024-04-21T08:51:22.636 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.637 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.637 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.637 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.637 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.637 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.637 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.637 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.637 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.637 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.637 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.637 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.637 INFO:teuthology.orchestra.run.smithi086.stdout: "leaderboard_description": { 2024-04-21T08:51:22.637 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "leaderboard_description", 2024-04-21T08:51:22.638 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.638 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.638 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.638 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.638 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.638 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.638 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.638 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.638 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.638 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.638 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.639 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.639 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.639 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.639 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.639 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.639 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.639 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.639 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.639 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.639 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.639 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.639 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.640 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.640 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.640 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.640 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.640 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.640 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.640 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.640 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.640 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.640 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.640 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.640 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.640 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.641 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.641 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.641 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.641 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.641 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.641 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.641 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.641 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.641 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.641 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.641 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.641 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.641 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.642 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.642 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.642 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.642 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.642 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.642 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.642 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.642 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.642 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.642 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.642 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.643 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.643 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.643 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.643 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.643 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.643 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.643 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.643 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.643 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.643 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.643 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.643 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.644 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.644 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.644 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.644 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.644 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.644 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.644 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.644 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.644 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.644 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.644 INFO:teuthology.orchestra.run.smithi086.stdout: "organization": { 2024-04-21T08:51:22.644 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "organization", 2024-04-21T08:51:22.645 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.645 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.645 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.645 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.645 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.645 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.645 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.645 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.645 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.645 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.645 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.645 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.646 INFO:teuthology.orchestra.run.smithi086.stdout: "proxy": { 2024-04-21T08:51:22.646 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "proxy", 2024-04-21T08:51:22.646 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.646 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.646 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.646 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.646 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.646 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.646 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.646 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.646 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.646 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.647 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.647 INFO:teuthology.orchestra.run.smithi086.stdout: "url": { 2024-04-21T08:51:22.647 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "url", 2024-04-21T08:51:22.647 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.647 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.647 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.647 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "https://telemetry.ceph.com/report", 2024-04-21T08:51:22.647 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.647 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.647 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.647 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.647 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.648 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.648 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.648 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.648 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.648 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.648 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "test_orchestrator", 2024-04-21T08:51:22.648 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.648 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.648 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.648 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.648 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.648 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.649 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.649 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.649 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.649 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.649 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.649 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.649 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.649 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.649 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.649 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.649 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.649 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.649 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.650 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.650 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.650 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.650 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.650 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.650 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.650 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.650 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.650 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.650 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.650 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.651 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.651 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.651 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.651 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.651 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.651 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.651 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.651 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.651 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.651 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.651 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.652 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.652 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.652 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.652 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.652 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.652 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.652 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.652 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.652 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.652 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.653 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.653 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.653 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.653 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.653 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.653 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.653 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.653 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.653 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.653 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.653 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.653 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.654 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.654 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.654 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.654 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.654 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.654 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.654 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.654 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.654 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.654 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.654 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.654 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.655 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.655 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "volumes", 2024-04-21T08:51:22.655 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.655 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.655 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.655 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.655 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.655 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.655 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.655 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.655 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.655 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.655 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.656 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.656 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.656 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.656 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.656 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.656 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.656 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.656 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.656 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.656 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.656 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.656 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.657 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.657 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.657 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.657 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.657 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.657 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.657 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.657 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.657 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.657 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.657 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.657 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.658 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.658 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.658 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.658 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.658 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.658 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.658 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.658 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.658 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.658 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.658 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.658 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.659 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.659 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.659 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.659 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.659 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.659 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.659 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.659 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.659 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.659 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.659 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.659 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.659 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.660 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.660 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.660 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.660 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.660 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.660 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.660 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.660 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.660 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.660 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.660 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.660 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.660 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.660 INFO:teuthology.orchestra.run.smithi086.stdout: "max_concurrent_clones": { 2024-04-21T08:51:22.661 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "max_concurrent_clones", 2024-04-21T08:51:22.661 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.661 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.661 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.661 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "4", 2024-04-21T08:51:22.661 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.661 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.661 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.661 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Number of asynchronous cloner threads", 2024-04-21T08:51:22.661 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.661 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.661 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.661 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.662 INFO:teuthology.orchestra.run.smithi086.stdout: "periodic_async_work": { 2024-04-21T08:51:22.662 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "periodic_async_work", 2024-04-21T08:51:22.662 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.662 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.662 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.662 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.662 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.662 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.662 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.662 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Periodically check for async work", 2024-04-21T08:51:22.662 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.662 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.663 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.663 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.663 INFO:teuthology.orchestra.run.smithi086.stdout: "snapshot_clone_delay": { 2024-04-21T08:51:22.663 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snapshot_clone_delay", 2024-04-21T08:51:22.663 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.663 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.663 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.663 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "0", 2024-04-21T08:51:22.663 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.663 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.663 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.663 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Delay clone begin operation by snapshot_clone_delay seconds", 2024-04-21T08:51:22.663 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.664 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.664 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.664 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.664 INFO:teuthology.orchestra.run.smithi086.stdout: "snapshot_clone_no_wait": { 2024-04-21T08:51:22.664 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snapshot_clone_no_wait", 2024-04-21T08:51:22.664 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.664 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.664 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.664 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "True", 2024-04-21T08:51:22.664 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.664 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.664 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.664 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "Reject subvolume clone request when cloner threads are busy", 2024-04-21T08:51:22.665 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.665 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.665 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.665 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.665 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.665 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.665 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.665 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix", 2024-04-21T08:51:22.665 INFO:teuthology.orchestra.run.smithi086.stdout: "can_run": true, 2024-04-21T08:51:22.665 INFO:teuthology.orchestra.run.smithi086.stdout: "error_string": "", 2024-04-21T08:51:22.665 INFO:teuthology.orchestra.run.smithi086.stdout: "module_options": { 2024-04-21T08:51:22.666 INFO:teuthology.orchestra.run.smithi086.stdout: "discovery_interval": { 2024-04-21T08:51:22.666 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "discovery_interval", 2024-04-21T08:51:22.666 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "uint", 2024-04-21T08:51:22.666 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.666 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.666 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "100", 2024-04-21T08:51:22.666 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.666 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.666 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.666 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.667 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.667 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.667 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.667 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.667 INFO:teuthology.orchestra.run.smithi086.stdout: "identifier": { 2024-04-21T08:51:22.667 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "identifier", 2024-04-21T08:51:22.667 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.667 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.667 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.667 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.667 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.668 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.668 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.668 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.668 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.668 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.668 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.668 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.668 INFO:teuthology.orchestra.run.smithi086.stdout: "interval": { 2024-04-21T08:51:22.668 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "interval", 2024-04-21T08:51:22.668 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "secs", 2024-04-21T08:51:22.668 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.668 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.669 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "60", 2024-04-21T08:51:22.669 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.669 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.669 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.669 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.669 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.669 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.669 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.669 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.669 INFO:teuthology.orchestra.run.smithi086.stdout: "log_level": { 2024-04-21T08:51:22.669 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_level", 2024-04-21T08:51:22.670 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.670 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.670 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.670 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.670 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.670 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.670 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.670 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.670 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.670 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.670 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.670 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.671 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.671 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.671 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.671 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.671 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.671 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.671 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.671 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster": { 2024-04-21T08:51:22.671 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster", 2024-04-21T08:51:22.671 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.671 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.672 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.672 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.672 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.672 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.672 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.672 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.672 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.672 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.672 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.672 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.672 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_cluster_level": { 2024-04-21T08:51:22.672 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_cluster_level", 2024-04-21T08:51:22.672 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.673 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.673 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.673 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "info", 2024-04-21T08:51:22.673 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.673 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.673 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [ 2024-04-21T08:51:22.673 INFO:teuthology.orchestra.run.smithi086.stdout: "", 2024-04-21T08:51:22.673 INFO:teuthology.orchestra.run.smithi086.stdout: "critical", 2024-04-21T08:51:22.673 INFO:teuthology.orchestra.run.smithi086.stdout: "debug", 2024-04-21T08:51:22.673 INFO:teuthology.orchestra.run.smithi086.stdout: "error", 2024-04-21T08:51:22.673 INFO:teuthology.orchestra.run.smithi086.stdout: "info", 2024-04-21T08:51:22.674 INFO:teuthology.orchestra.run.smithi086.stdout: "warning" 2024-04-21T08:51:22.674 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.674 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.674 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.674 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.674 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.674 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.674 INFO:teuthology.orchestra.run.smithi086.stdout: "log_to_file": { 2024-04-21T08:51:22.674 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "log_to_file", 2024-04-21T08:51:22.674 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "bool", 2024-04-21T08:51:22.674 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.674 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 1, 2024-04-21T08:51:22.675 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "False", 2024-04-21T08:51:22.675 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.675 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.675 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.675 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.675 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.675 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.675 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.675 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.675 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_host": { 2024-04-21T08:51:22.676 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_host", 2024-04-21T08:51:22.676 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.676 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.676 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.676 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "", 2024-04-21T08:51:22.676 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.676 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.676 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.676 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.676 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.676 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.676 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.677 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.677 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_port": { 2024-04-21T08:51:22.677 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_port", 2024-04-21T08:51:22.677 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "int", 2024-04-21T08:51:22.677 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.677 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.677 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "10051", 2024-04-21T08:51:22.677 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.677 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.677 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.677 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.678 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.678 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.678 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.678 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.678 INFO:teuthology.orchestra.run.smithi086.stdout: "zabbix_sender": { 2024-04-21T08:51:22.678 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "zabbix_sender", 2024-04-21T08:51:22.678 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "str", 2024-04-21T08:51:22.678 INFO:teuthology.orchestra.run.smithi086.stdout: "level": "advanced", 2024-04-21T08:51:22.678 INFO:teuthology.orchestra.run.smithi086.stdout: "flags": 0, 2024-04-21T08:51:22.678 INFO:teuthology.orchestra.run.smithi086.stdout: "default_value": "/usr/bin/zabbix_sender", 2024-04-21T08:51:22.678 INFO:teuthology.orchestra.run.smithi086.stdout: "min": "", 2024-04-21T08:51:22.679 INFO:teuthology.orchestra.run.smithi086.stdout: "max": "", 2024-04-21T08:51:22.679 INFO:teuthology.orchestra.run.smithi086.stdout: "enum_allowed": [], 2024-04-21T08:51:22.679 INFO:teuthology.orchestra.run.smithi086.stdout: "desc": "", 2024-04-21T08:51:22.679 INFO:teuthology.orchestra.run.smithi086.stdout: "long_desc": "", 2024-04-21T08:51:22.679 INFO:teuthology.orchestra.run.smithi086.stdout: "tags": [], 2024-04-21T08:51:22.679 INFO:teuthology.orchestra.run.smithi086.stdout: "see_also": [] 2024-04-21T08:51:22.679 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.679 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.679 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.679 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.679 INFO:teuthology.orchestra.run.smithi086.stdout: "services": { 2024-04-21T08:51:22.680 INFO:teuthology.orchestra.run.smithi086.stdout: "dashboard": "https://172.21.15.86:8443/" 2024-04-21T08:51:22.680 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.680 INFO:teuthology.orchestra.run.smithi086.stdout: "always_on_modules": { 2024-04-21T08:51:22.680 INFO:teuthology.orchestra.run.smithi086.stdout: "octopus": [ 2024-04-21T08:51:22.680 INFO:teuthology.orchestra.run.smithi086.stdout: "balancer", 2024-04-21T08:51:22.680 INFO:teuthology.orchestra.run.smithi086.stdout: "crash", 2024-04-21T08:51:22.680 INFO:teuthology.orchestra.run.smithi086.stdout: "devicehealth", 2024-04-21T08:51:22.680 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator", 2024-04-21T08:51:22.680 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_autoscaler", 2024-04-21T08:51:22.680 INFO:teuthology.orchestra.run.smithi086.stdout: "progress", 2024-04-21T08:51:22.680 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_support", 2024-04-21T08:51:22.680 INFO:teuthology.orchestra.run.smithi086.stdout: "status", 2024-04-21T08:51:22.681 INFO:teuthology.orchestra.run.smithi086.stdout: "telemetry", 2024-04-21T08:51:22.681 INFO:teuthology.orchestra.run.smithi086.stdout: "volumes" 2024-04-21T08:51:22.681 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.681 INFO:teuthology.orchestra.run.smithi086.stdout: "pacific": [ 2024-04-21T08:51:22.681 INFO:teuthology.orchestra.run.smithi086.stdout: "balancer", 2024-04-21T08:51:22.681 INFO:teuthology.orchestra.run.smithi086.stdout: "crash", 2024-04-21T08:51:22.681 INFO:teuthology.orchestra.run.smithi086.stdout: "devicehealth", 2024-04-21T08:51:22.681 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator", 2024-04-21T08:51:22.681 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_autoscaler", 2024-04-21T08:51:22.681 INFO:teuthology.orchestra.run.smithi086.stdout: "progress", 2024-04-21T08:51:22.682 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_support", 2024-04-21T08:51:22.682 INFO:teuthology.orchestra.run.smithi086.stdout: "status", 2024-04-21T08:51:22.682 INFO:teuthology.orchestra.run.smithi086.stdout: "telemetry", 2024-04-21T08:51:22.682 INFO:teuthology.orchestra.run.smithi086.stdout: "volumes" 2024-04-21T08:51:22.682 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.682 INFO:teuthology.orchestra.run.smithi086.stdout: "quincy": [ 2024-04-21T08:51:22.682 INFO:teuthology.orchestra.run.smithi086.stdout: "balancer", 2024-04-21T08:51:22.682 INFO:teuthology.orchestra.run.smithi086.stdout: "crash", 2024-04-21T08:51:22.682 INFO:teuthology.orchestra.run.smithi086.stdout: "devicehealth", 2024-04-21T08:51:22.682 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator", 2024-04-21T08:51:22.683 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_autoscaler", 2024-04-21T08:51:22.683 INFO:teuthology.orchestra.run.smithi086.stdout: "progress", 2024-04-21T08:51:22.683 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_support", 2024-04-21T08:51:22.683 INFO:teuthology.orchestra.run.smithi086.stdout: "status", 2024-04-21T08:51:22.683 INFO:teuthology.orchestra.run.smithi086.stdout: "telemetry", 2024-04-21T08:51:22.683 INFO:teuthology.orchestra.run.smithi086.stdout: "volumes" 2024-04-21T08:51:22.683 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.683 INFO:teuthology.orchestra.run.smithi086.stdout: "reef": [ 2024-04-21T08:51:22.683 INFO:teuthology.orchestra.run.smithi086.stdout: "balancer", 2024-04-21T08:51:22.683 INFO:teuthology.orchestra.run.smithi086.stdout: "crash", 2024-04-21T08:51:22.684 INFO:teuthology.orchestra.run.smithi086.stdout: "devicehealth", 2024-04-21T08:51:22.684 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator", 2024-04-21T08:51:22.684 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_autoscaler", 2024-04-21T08:51:22.684 INFO:teuthology.orchestra.run.smithi086.stdout: "progress", 2024-04-21T08:51:22.684 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_support", 2024-04-21T08:51:22.684 INFO:teuthology.orchestra.run.smithi086.stdout: "status", 2024-04-21T08:51:22.684 INFO:teuthology.orchestra.run.smithi086.stdout: "telemetry", 2024-04-21T08:51:22.684 INFO:teuthology.orchestra.run.smithi086.stdout: "volumes" 2024-04-21T08:51:22.684 INFO:teuthology.orchestra.run.smithi086.stdout: ], 2024-04-21T08:51:22.684 INFO:teuthology.orchestra.run.smithi086.stdout: "squid": [ 2024-04-21T08:51:22.684 INFO:teuthology.orchestra.run.smithi086.stdout: "balancer", 2024-04-21T08:51:22.684 INFO:teuthology.orchestra.run.smithi086.stdout: "crash", 2024-04-21T08:51:22.685 INFO:teuthology.orchestra.run.smithi086.stdout: "devicehealth", 2024-04-21T08:51:22.685 INFO:teuthology.orchestra.run.smithi086.stdout: "orchestrator", 2024-04-21T08:51:22.685 INFO:teuthology.orchestra.run.smithi086.stdout: "pg_autoscaler", 2024-04-21T08:51:22.685 INFO:teuthology.orchestra.run.smithi086.stdout: "progress", 2024-04-21T08:51:22.685 INFO:teuthology.orchestra.run.smithi086.stdout: "rbd_support", 2024-04-21T08:51:22.685 INFO:teuthology.orchestra.run.smithi086.stdout: "status", 2024-04-21T08:51:22.685 INFO:teuthology.orchestra.run.smithi086.stdout: "telemetry", 2024-04-21T08:51:22.685 INFO:teuthology.orchestra.run.smithi086.stdout: "volumes" 2024-04-21T08:51:22.685 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-21T08:51:22.685 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.685 INFO:teuthology.orchestra.run.smithi086.stdout: "last_failure_osd_epoch": 94, 2024-04-21T08:51:22.685 INFO:teuthology.orchestra.run.smithi086.stdout: "active_clients": [ 2024-04-21T08:51:22.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.686 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "devicehealth", 2024-04-21T08:51:22.686 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-04-21T08:51:22.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.686 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-04-21T08:51:22.686 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:0", 2024-04-21T08:51:22.686 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 3323082001 2024-04-21T08:51:22.686 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.686 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-21T08:51:22.686 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.686 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.686 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "libcephsqlite", 2024-04-21T08:51:22.687 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-04-21T08:51:22.687 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.687 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-04-21T08:51:22.687 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:0", 2024-04-21T08:51:22.687 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 3481930559 2024-04-21T08:51:22.687 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.687 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-21T08:51:22.687 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.687 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.687 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "rbd_support", 2024-04-21T08:51:22.687 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-04-21T08:51:22.687 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.688 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-04-21T08:51:22.688 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:0", 2024-04-21T08:51:22.688 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 3790662963 2024-04-21T08:51:22.688 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.688 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-21T08:51:22.688 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.688 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.688 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snap_schedule", 2024-04-21T08:51:22.688 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-04-21T08:51:22.688 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.688 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-04-21T08:51:22.688 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:0", 2024-04-21T08:51:22.689 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 4202796113 2024-04-21T08:51:22.689 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.689 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-21T08:51:22.689 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.689 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.689 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "snap_schedule", 2024-04-21T08:51:22.689 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-04-21T08:51:22.689 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.689 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-04-21T08:51:22.689 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:0", 2024-04-21T08:51:22.689 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 1784469649 2024-04-21T08:51:22.689 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.690 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-21T08:51:22.690 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:22.690 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.690 INFO:teuthology.orchestra.run.smithi086.stdout: "name": "volumes", 2024-04-21T08:51:22.690 INFO:teuthology.orchestra.run.smithi086.stdout: "addrvec": [ 2024-04-21T08:51:22.690 INFO:teuthology.orchestra.run.smithi086.stdout: { 2024-04-21T08:51:22.690 INFO:teuthology.orchestra.run.smithi086.stdout: "type": "v2", 2024-04-21T08:51:22.690 INFO:teuthology.orchestra.run.smithi086.stdout: "addr": "172.21.15.86:0", 2024-04-21T08:51:22.690 INFO:teuthology.orchestra.run.smithi086.stdout: "nonce": 3655004028 2024-04-21T08:51:22.690 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.690 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-21T08:51:22.691 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:22.691 INFO:teuthology.orchestra.run.smithi086.stdout: ] 2024-04-21T08:51:22.691 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:51:22.693 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:20 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3403330387' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:51:22.693 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:20 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1002540037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:51:22.693 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:20 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3202888662' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:51:22.693 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:21 smithi179 ceph-mon[31829]: pgmap v394: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.8 KiB/s wr, 13 op/s 2024-04-21T08:51:22.693 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:21 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1174056077' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-21T08:51:22.693 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:20 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3403330387' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:51:22.693 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:20 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1002540037' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:51:22.693 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:20 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3202888662' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:51:22.693 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:21 smithi088 ceph-mon[31259]: pgmap v394: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.8 KiB/s wr, 13 op/s 2024-04-21T08:51:22.694 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:21 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1174056077' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-21T08:51:22.694 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:21.144+0000 7f1150d32640 1 -- 172.21.15.86:0/1174056077 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f11380a3d20 msgr2=0x7f11380a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:22.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:21.144+0000 7f1150d32640 1 --2- 172.21.15.86:0/1174056077 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f11380a3d20 0x7f11380a61e0 secure :-1 s=READY pgs=364 cs=0 l=1 rev1=1 crypto rx=0x7f1134009990 tx=0x7f1134009110 comp rx=0 tx=0).stop 2024-04-21T08:51:22.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:21.144+0000 7f1150d32640 1 -- 172.21.15.86:0/1174056077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f114c10c520 msgr2=0x7f114c1985d0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:22.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:21.144+0000 7f1150d32640 1 --2- 172.21.15.86:0/1174056077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f114c10c520 0x7f114c1985d0 secure :-1 s=READY pgs=2141 cs=0 l=1 rev1=1 crypto rx=0x7f114400cad0 tx=0x7f114400ced0 comp rx=0 tx=0).stop 2024-04-21T08:51:22.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:21.144+0000 7f1150d32640 1 -- 172.21.15.86:0/1174056077 shutdown_connections 2024-04-21T08:51:22.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:21.144+0000 7f1150d32640 1 --2- 172.21.15.86:0/1174056077 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f11380a3d20 0x7f11380a61e0 unknown :-1 s=CLOSED pgs=364 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:22.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:21.144+0000 7f1150d32640 1 --2- 172.21.15.86:0/1174056077 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f114c10c520 0x7f114c1985d0 unknown :-1 s=CLOSED pgs=2141 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:22.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:21.144+0000 7f1150d32640 1 --2- 172.21.15.86:0/1174056077 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f114c103b00 0x7f114c19e960 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:22.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:21.144+0000 7f1150d32640 1 --2- 172.21.15.86:0/1174056077 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f114c1031e0 0x7f114c19e420 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:22.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:21.144+0000 7f1150d32640 1 -- 172.21.15.86:0/1174056077 >> 172.21.15.86:0/1174056077 conn(0x7f114c077f50 msgr2=0x7f114c0fe020 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:22.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:21.144+0000 7f1150d32640 1 -- 172.21.15.86:0/1174056077 shutdown_connections 2024-04-21T08:51:22.695 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:21.144+0000 7f1150d32640 1 -- 172.21.15.86:0/1174056077 wait complete. 2024-04-21T08:51:22.736 DEBUG:tasks.check_counter:Getting stats from a 2024-04-21T08:51:22.737 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph daemon mds.a perf dump 2024-04-21T08:51:22.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:21 smithi086 ceph-mon[27939]: pgmap v394: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.8 KiB/s wr, 13 op/s 2024-04-21T08:51:22.737 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:21 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1174056077' entity='client.admin' cmd=[{"prefix": "mgr dump", "format": "json-pretty"}]: dispatch 2024-04-21T08:51:22.989 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:51:24.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:23 smithi088 ceph-mon[31259]: pgmap v395: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.1 KiB/s wr, 15 op/s 2024-04-21T08:51:24.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:23 smithi179 ceph-mon[31829]: pgmap v395: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.1 KiB/s wr, 15 op/s 2024-04-21T08:51:24.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:23 smithi086 ceph-mon[27939]: pgmap v395: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 16 KiB/s rd, 4.1 KiB/s wr, 15 op/s 2024-04-21T08:51:25.005 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:51:25.005 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T08:51:25.005 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 1757, 2024-04-21T08:51:25.005 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 1759, 2024-04-21T08:51:25.006 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 526339, 2024-04-21T08:51:25.006 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 558727, 2024-04-21T08:51:25.006 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 6, 2024-04-21T08:51:25.006 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 2, 2024-04-21T08:51:25.006 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.181337428, 2024-04-21T08:51:25.006 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.058386687, 2024-04-21T08:51:25.006 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.048083635, 2024-04-21T08:51:25.006 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.059016200, 2024-04-21T08:51:25.006 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:25.006 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1757, 2024-04-21T08:51:25.006 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.063737779, 2024-04-21T08:51:25.006 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000036276 2024-04-21T08:51:25.006 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.006 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:25.007 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.007 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.007 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.007 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.007 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-21T08:51:25.007 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 0 2024-04-21T08:51:25.007 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.007 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T08:51:25.007 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 1095, 2024-04-21T08:51:25.007 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 1083, 2024-04-21T08:51:25.007 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 1838992, 2024-04-21T08:51:25.007 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 413896, 2024-04-21T08:51:25.007 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 4, 2024-04-21T08:51:25.008 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 3, 2024-04-21T08:51:25.008 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.138416170, 2024-04-21T08:51:25.008 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.040845769, 2024-04-21T08:51:25.008 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.059895155, 2024-04-21T08:51:25.008 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.040320516, 2024-04-21T08:51:25.008 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:25.008 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1082, 2024-04-21T08:51:25.008 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.044509477, 2024-04-21T08:51:25.008 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000041136 2024-04-21T08:51:25.008 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.008 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:25.008 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.008 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.008 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.009 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.009 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 1638096, 2024-04-21T08:51:25.009 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 154976 2024-04-21T08:51:25.009 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.009 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T08:51:25.009 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 7, 2024-04-21T08:51:25.009 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 190, 2024-04-21T08:51:25.009 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 3580, 2024-04-21T08:51:25.009 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 489502, 2024-04-21T08:51:25.009 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 6, 2024-04-21T08:51:25.009 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 4, 2024-04-21T08:51:25.009 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.015264516, 2024-04-21T08:51:25.009 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.010682386, 2024-04-21T08:51:25.009 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.002307930, 2024-04-21T08:51:25.009 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000016120, 2024-04-21T08:51:25.010 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:25.010 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 188, 2024-04-21T08:51:25.010 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.015175023, 2024-04-21T08:51:25.010 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000080718 2024-04-21T08:51:25.010 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.010 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:25.010 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.010 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.010 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.010 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.010 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-21T08:51:25.010 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 489184 2024-04-21T08:51:25.010 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.011 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-04-21T08:51:25.011 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 1, 2024-04-21T08:51:25.011 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-04-21T08:51:25.011 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.011 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-MDSRank": { 2024-04-21T08:51:25.011 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-04-21T08:51:25.011 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-04-21T08:51:25.011 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 3996, 2024-04-21T08:51:25.011 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.125618226, 2024-04-21T08:51:25.011 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000031435 2024-04-21T08:51:25.011 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:25.011 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.011 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-PurgeQueue": { 2024-04-21T08:51:25.012 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-04-21T08:51:25.012 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-04-21T08:51:25.012 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.012 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.012 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.012 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:25.012 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.012 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": { 2024-04-21T08:51:25.012 INFO:teuthology.orchestra.run.smithi086.stdout: "request": 0, 2024-04-21T08:51:25.012 INFO:teuthology.orchestra.run.smithi086.stdout: "reply": 0, 2024-04-21T08:51:25.012 INFO:teuthology.orchestra.run.smithi086.stdout: "reply_latency": { 2024-04-21T08:51:25.012 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.012 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.012 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.012 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.013 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_reply": 0, 2024-04-21T08:51:25.013 INFO:teuthology.orchestra.run.smithi086.stdout: "forward": 0, 2024-04-21T08:51:25.013 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_complete": 0, 2024-04-21T08:51:25.013 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_keys": 0, 2024-04-21T08:51:25.013 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_commit": 0, 2024-04-21T08:51:25.013 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_split": 0, 2024-04-21T08:51:25.013 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_merge": 0, 2024-04-21T08:51:25.013 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes": 0, 2024-04-21T08:51:25.013 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_top": 0, 2024-04-21T08:51:25.013 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_bottom": 0, 2024-04-21T08:51:25.013 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pin_tail": 0, 2024-04-21T08:51:25.013 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pinned": 0, 2024-04-21T08:51:25.013 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_expired": 0, 2024-04-21T08:51:25.013 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_with_caps": 0, 2024-04-21T08:51:25.013 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": 0, 2024-04-21T08:51:25.014 INFO:teuthology.orchestra.run.smithi086.stdout: "subtrees": 1, 2024-04-21T08:51:25.014 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse": 0, 2024-04-21T08:51:25.014 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_hit": 0, 2024-04-21T08:51:25.014 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_forward": 0, 2024-04-21T08:51:25.014 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_discover": 0, 2024-04-21T08:51:25.014 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_dir_fetch": 0, 2024-04-21T08:51:25.014 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_remote_ino": 0, 2024-04-21T08:51:25.014 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_lock": 0, 2024-04-21T08:51:25.014 INFO:teuthology.orchestra.run.smithi086.stdout: "load_cent": 0, 2024-04-21T08:51:25.014 INFO:teuthology.orchestra.run.smithi086.stdout: "q": 0, 2024-04-21T08:51:25.014 INFO:teuthology.orchestra.run.smithi086.stdout: "exported": 0, 2024-04-21T08:51:25.014 INFO:teuthology.orchestra.run.smithi086.stdout: "exported_inodes": 0, 2024-04-21T08:51:25.014 INFO:teuthology.orchestra.run.smithi086.stdout: "imported": 0, 2024-04-21T08:51:25.015 INFO:teuthology.orchestra.run.smithi086.stdout: "imported_inodes": 0, 2024-04-21T08:51:25.015 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_dir_fetch": 0, 2024-04-21T08:51:25.015 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_backtrace_fetch": 0, 2024-04-21T08:51:25.015 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_peer_discover": 0, 2024-04-21T08:51:25.015 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rfiles": 0, 2024-04-21T08:51:25.015 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rbytes": 0, 2024-04-21T08:51:25.015 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rsnaps": 0, 2024-04-21T08:51:25.015 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_fetch": 0, 2024-04-21T08:51:25.015 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_set_tag": 0, 2024-04-21T08:51:25.015 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T08:51:25.015 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_inotable_repaired": 0, 2024-04-21T08:51:25.015 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_inodes": 0, 2024-04-21T08:51:25.015 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_base_inodes": 0, 2024-04-21T08:51:25.016 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dirfrag_rstats": 0, 2024-04-21T08:51:25.016 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_file_inodes": 0, 2024-04-21T08:51:25.016 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_inode_file_caps": 0, 2024-04-21T08:51:25.016 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_revoke": 0, 2024-04-21T08:51:25.016 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_grant": 0, 2024-04-21T08:51:25.016 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_trunc": 0, 2024-04-21T08:51:25.016 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T08:51:25.016 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T08:51:25.016 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps": 0, 2024-04-21T08:51:25.016 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps_dirty": 0, 2024-04-21T08:51:25.016 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_cap_release": 0, 2024-04-21T08:51:25.016 INFO:teuthology.orchestra.run.smithi086.stdout: "process_request_cap_release": 0 2024-04-21T08:51:25.016 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.016 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_cache": { 2024-04-21T08:51:25.017 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update": 0, 2024-04-21T08:51:25.017 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update_receipt": 0, 2024-04-21T08:51:25.017 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_try_discover": 0, 2024-04-21T08:51:25.017 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_send_discover": 0, 2024-04-21T08:51:25.017 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_handle_discover": 0, 2024-04-21T08:51:25.017 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays": 0, 2024-04-21T08:51:25.017 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_delayed": 0, 2024-04-21T08:51:25.017 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_enqueuing": 0, 2024-04-21T08:51:25.017 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_created": 0, 2024-04-21T08:51:25.017 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_enqueued": 0, 2024-04-21T08:51:25.017 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_reintegrated": 0, 2024-04-21T08:51:25.017 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_migrated": 0, 2024-04-21T08:51:25.017 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_processing": 0, 2024-04-21T08:51:25.018 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_enqueued": 0, 2024-04-21T08:51:25.018 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_prioritized": 0, 2024-04-21T08:51:25.018 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_started": 0, 2024-04-21T08:51:25.018 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_completed": 0, 2024-04-21T08:51:25.018 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_path": 0, 2024-04-21T08:51:25.018 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_inode": 0, 2024-04-21T08:51:25.018 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_enqueue_scrub": 0, 2024-04-21T08:51:25.018 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_exportdir": 0, 2024-04-21T08:51:25.018 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_flush": 0, 2024-04-21T08:51:25.018 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragmentdir": 0, 2024-04-21T08:51:25.018 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragstats": 0, 2024-04-21T08:51:25.018 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_inodestats": 0 2024-04-21T08:51:25.018 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.018 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_log": { 2024-04-21T08:51:25.019 INFO:teuthology.orchestra.run.smithi086.stdout: "evlrg": 0, 2024-04-21T08:51:25.019 INFO:teuthology.orchestra.run.smithi086.stdout: "evadd": 0, 2024-04-21T08:51:25.019 INFO:teuthology.orchestra.run.smithi086.stdout: "evex": 0, 2024-04-21T08:51:25.019 INFO:teuthology.orchestra.run.smithi086.stdout: "evtrm": 0, 2024-04-21T08:51:25.019 INFO:teuthology.orchestra.run.smithi086.stdout: "ev": 4, 2024-04-21T08:51:25.019 INFO:teuthology.orchestra.run.smithi086.stdout: "evexg": 0, 2024-04-21T08:51:25.019 INFO:teuthology.orchestra.run.smithi086.stdout: "evexd": 0, 2024-04-21T08:51:25.019 INFO:teuthology.orchestra.run.smithi086.stdout: "segadd": 0, 2024-04-21T08:51:25.019 INFO:teuthology.orchestra.run.smithi086.stdout: "segex": 0, 2024-04-21T08:51:25.019 INFO:teuthology.orchestra.run.smithi086.stdout: "segtrm": 0, 2024-04-21T08:51:25.019 INFO:teuthology.orchestra.run.smithi086.stdout: "seg": 2, 2024-04-21T08:51:25.019 INFO:teuthology.orchestra.run.smithi086.stdout: "segmjr": 2, 2024-04-21T08:51:25.019 INFO:teuthology.orchestra.run.smithi086.stdout: "segexg": 0, 2024-04-21T08:51:25.020 INFO:teuthology.orchestra.run.smithi086.stdout: "segexd": 0, 2024-04-21T08:51:25.020 INFO:teuthology.orchestra.run.smithi086.stdout: "expos": 4194304, 2024-04-21T08:51:25.020 INFO:teuthology.orchestra.run.smithi086.stdout: "wrpos": 4195424, 2024-04-21T08:51:25.020 INFO:teuthology.orchestra.run.smithi086.stdout: "rdpos": 4195283, 2024-04-21T08:51:25.020 INFO:teuthology.orchestra.run.smithi086.stdout: "jlat": { 2024-04-21T08:51:25.020 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.020 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.020 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.020 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.020 INFO:teuthology.orchestra.run.smithi086.stdout: "replayed": 4 2024-04-21T08:51:25.020 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.020 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_mem": { 2024-04-21T08:51:25.020 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 3, 2024-04-21T08:51:25.020 INFO:teuthology.orchestra.run.smithi086.stdout: "ino+": 3, 2024-04-21T08:51:25.021 INFO:teuthology.orchestra.run.smithi086.stdout: "ino-": 0, 2024-04-21T08:51:25.021 INFO:teuthology.orchestra.run.smithi086.stdout: "dir": 1, 2024-04-21T08:51:25.021 INFO:teuthology.orchestra.run.smithi086.stdout: "dir+": 1, 2024-04-21T08:51:25.021 INFO:teuthology.orchestra.run.smithi086.stdout: "dir-": 0, 2024-04-21T08:51:25.021 INFO:teuthology.orchestra.run.smithi086.stdout: "dn": 0, 2024-04-21T08:51:25.021 INFO:teuthology.orchestra.run.smithi086.stdout: "dn+": 0, 2024-04-21T08:51:25.021 INFO:teuthology.orchestra.run.smithi086.stdout: "dn-": 0, 2024-04-21T08:51:25.021 INFO:teuthology.orchestra.run.smithi086.stdout: "cap": 0, 2024-04-21T08:51:25.021 INFO:teuthology.orchestra.run.smithi086.stdout: "cap+": 0, 2024-04-21T08:51:25.021 INFO:teuthology.orchestra.run.smithi086.stdout: "cap-": 0, 2024-04-21T08:51:25.021 INFO:teuthology.orchestra.run.smithi086.stdout: "rss": 50568, 2024-04-21T08:51:25.021 INFO:teuthology.orchestra.run.smithi086.stdout: "heap": 215416 2024-04-21T08:51:25.021 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.022 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_server": { 2024-04-21T08:51:25.022 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_client_request": 0, 2024-04-21T08:51:25.022 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_server_request": 0, 2024-04-21T08:51:25.022 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_request": 0, 2024-04-21T08:51:25.022 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_session": 0, 2024-04-21T08:51:25.022 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_peer_request": 0, 2024-04-21T08:51:25.022 INFO:teuthology.orchestra.run.smithi086.stdout: "req_create_latency": { 2024-04-21T08:51:25.022 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.022 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.022 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.022 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.022 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getattr_latency": { 2024-04-21T08:51:25.022 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.023 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.023 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.023 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.023 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getfilelock_latency": { 2024-04-21T08:51:25.023 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.023 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.023 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.023 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.023 INFO:teuthology.orchestra.run.smithi086.stdout: "req_link_latency": { 2024-04-21T08:51:25.023 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.023 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.023 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.023 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.023 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookup_latency": { 2024-04-21T08:51:25.024 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.024 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.024 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.024 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.024 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookuphash_latency": { 2024-04-21T08:51:25.024 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.024 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.024 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.024 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.024 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupino_latency": { 2024-04-21T08:51:25.024 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.024 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.024 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.024 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.024 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupname_latency": { 2024-04-21T08:51:25.025 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.025 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.025 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.025 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.025 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupparent_latency": { 2024-04-21T08:51:25.025 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.025 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.025 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.025 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.025 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupsnap_latency": { 2024-04-21T08:51:25.025 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.025 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.025 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.025 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.026 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lssnap_latency": { 2024-04-21T08:51:25.026 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.026 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.026 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.026 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.026 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mkdir_latency": { 2024-04-21T08:51:25.026 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.027 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.027 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.027 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.027 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mknod_latency": { 2024-04-21T08:51:25.027 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.027 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.027 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.027 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.027 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mksnap_latency": { 2024-04-21T08:51:25.027 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.027 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.027 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.028 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.028 INFO:teuthology.orchestra.run.smithi086.stdout: "req_open_latency": { 2024-04-21T08:51:25.028 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.028 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.028 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.028 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.028 INFO:teuthology.orchestra.run.smithi086.stdout: "req_readdir_latency": { 2024-04-21T08:51:25.028 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.028 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.028 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.028 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.028 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rename_latency": { 2024-04-21T08:51:25.028 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.029 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.029 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.029 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.029 INFO:teuthology.orchestra.run.smithi086.stdout: "req_renamesnap_latency": { 2024-04-21T08:51:25.029 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.029 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.029 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.029 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.029 INFO:teuthology.orchestra.run.smithi086.stdout: "req_snapdiff_latency": { 2024-04-21T08:51:25.029 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.029 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.029 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.029 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.029 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmdir_latency": { 2024-04-21T08:51:25.029 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.030 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.030 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.030 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.030 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmsnap_latency": { 2024-04-21T08:51:25.030 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.030 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.030 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.030 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.030 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmxattr_latency": { 2024-04-21T08:51:25.030 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.030 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.030 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.031 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.031 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setattr_latency": { 2024-04-21T08:51:25.031 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.031 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.031 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.031 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.031 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setdirlayout_latency": { 2024-04-21T08:51:25.031 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.031 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.031 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.031 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.031 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setfilelock_latency": { 2024-04-21T08:51:25.031 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.031 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.032 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.033 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.033 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setlayout_latency": { 2024-04-21T08:51:25.033 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.033 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.033 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.033 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.033 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setxattr_latency": { 2024-04-21T08:51:25.033 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.034 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.034 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.034 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.034 INFO:teuthology.orchestra.run.smithi086.stdout: "req_symlink_latency": { 2024-04-21T08:51:25.034 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.034 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.034 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.034 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.034 INFO:teuthology.orchestra.run.smithi086.stdout: "req_unlink_latency": { 2024-04-21T08:51:25.034 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.034 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.034 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.034 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.035 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_revoke_eviction": 0, 2024-04-21T08:51:25.035 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_acquisition_throttle": 0, 2024-04-21T08:51:25.035 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getvxattr_latency": { 2024-04-21T08:51:25.035 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.035 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.035 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.035 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:25.035 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.035 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_sessions": { 2024-04-21T08:51:25.035 INFO:teuthology.orchestra.run.smithi086.stdout: "session_count": 0, 2024-04-21T08:51:25.035 INFO:teuthology.orchestra.run.smithi086.stdout: "session_add": 1, 2024-04-21T08:51:25.035 INFO:teuthology.orchestra.run.smithi086.stdout: "session_remove": 1, 2024-04-21T08:51:25.035 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_open": 1, 2024-04-21T08:51:25.035 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_stale": 0, 2024-04-21T08:51:25.036 INFO:teuthology.orchestra.run.smithi086.stdout: "total_load": 0, 2024-04-21T08:51:25.036 INFO:teuthology.orchestra.run.smithi086.stdout: "average_load": 0, 2024-04-21T08:51:25.036 INFO:teuthology.orchestra.run.smithi086.stdout: "avg_session_uptime": 1095, 2024-04-21T08:51:25.036 INFO:teuthology.orchestra.run.smithi086.stdout: "mdthresh_evicted": 0 2024-04-21T08:51:25.036 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.036 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-04-21T08:51:25.036 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-04-21T08:51:25.036 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-04-21T08:51:25.036 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T08:51:25.036 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-04-21T08:51:25.036 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T08:51:25.036 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-04-21T08:51:25.036 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T08:51:25.037 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T08:51:25.037 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T08:51:25.037 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T08:51:25.037 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T08:51:25.037 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-04-21T08:51:25.037 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T08:51:25.037 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T08:51:25.037 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-04-21T08:51:25.037 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-04-21T08:51:25.037 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-04-21T08:51:25.037 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-04-21T08:51:25.037 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T08:51:25.037 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T08:51:25.037 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T08:51:25.038 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T08:51:25.038 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T08:51:25.038 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-04-21T08:51:25.038 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-04-21T08:51:25.038 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-04-21T08:51:25.038 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T08:51:25.038 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T08:51:25.038 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-04-21T08:51:25.038 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-04-21T08:51:25.038 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-04-21T08:51:25.038 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-04-21T08:51:25.038 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T08:51:25.038 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-04-21T08:51:25.039 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T08:51:25.039 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-04-21T08:51:25.039 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 189371, 2024-04-21T08:51:25.039 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 51, 2024-04-21T08:51:25.039 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-04-21T08:51:25.039 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-04-21T08:51:25.039 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-04-21T08:51:25.039 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-04-21T08:51:25.039 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-04-21T08:51:25.039 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-04-21T08:51:25.039 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-04-21T08:51:25.039 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-04-21T08:51:25.039 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 9512, 2024-04-21T08:51:25.039 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 182, 2024-04-21T08:51:25.039 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T08:51:25.040 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-04-21T08:51:25.040 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-04-21T08:51:25.040 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-04-21T08:51:25.040 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 8512, 2024-04-21T08:51:25.040 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 13, 2024-04-21T08:51:25.040 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-04-21T08:51:25.040 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-04-21T08:51:25.040 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-04-21T08:51:25.040 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-04-21T08:51:25.040 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.040 INFO:teuthology.orchestra.run.smithi086.stdout: "objecter": { 2024-04-21T08:51:25.040 INFO:teuthology.orchestra.run.smithi086.stdout: "op_active": 0, 2024-04-21T08:51:25.040 INFO:teuthology.orchestra.run.smithi086.stdout: "op_laggy": 0, 2024-04-21T08:51:25.040 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send": 2572, 2024-04-21T08:51:25.041 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send_bytes": 21, 2024-04-21T08:51:25.041 INFO:teuthology.orchestra.run.smithi086.stdout: "op_resend": 3, 2024-04-21T08:51:25.041 INFO:teuthology.orchestra.run.smithi086.stdout: "op_reply": 2572, 2024-04-21T08:51:25.041 INFO:teuthology.orchestra.run.smithi086.stdout: "op_latency": { 2024-04-21T08:51:25.041 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 2572, 2024-04-21T08:51:25.041 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 1.570966926, 2024-04-21T08:51:25.041 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000610795 2024-04-21T08:51:25.041 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.041 INFO:teuthology.orchestra.run.smithi086.stdout: "op_inflight": 0, 2024-04-21T08:51:25.041 INFO:teuthology.orchestra.run.smithi086.stdout: "oplen_avg": { 2024-04-21T08:51:25.041 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 2572, 2024-04-21T08:51:25.041 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 2573 2024-04-21T08:51:25.041 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.041 INFO:teuthology.orchestra.run.smithi086.stdout: "op": 2572, 2024-04-21T08:51:25.041 INFO:teuthology.orchestra.run.smithi086.stdout: "op_r": 2572, 2024-04-21T08:51:25.042 INFO:teuthology.orchestra.run.smithi086.stdout: "op_w": 0, 2024-04-21T08:51:25.042 INFO:teuthology.orchestra.run.smithi086.stdout: "op_rmw": 0, 2024-04-21T08:51:25.042 INFO:teuthology.orchestra.run.smithi086.stdout: "op_pg": 0, 2024-04-21T08:51:25.042 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_stat": 1688, 2024-04-21T08:51:25.042 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_create": 0, 2024-04-21T08:51:25.042 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_read": 882, 2024-04-21T08:51:25.042 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_write": 0, 2024-04-21T08:51:25.042 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writefull": 0, 2024-04-21T08:51:25.042 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writesame": 0, 2024-04-21T08:51:25.042 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_append": 0, 2024-04-21T08:51:25.042 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_zero": 0, 2024-04-21T08:51:25.042 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_truncate": 0, 2024-04-21T08:51:25.042 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_delete": 0, 2024-04-21T08:51:25.042 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_mapext": 0, 2024-04-21T08:51:25.043 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_sparse_read": 0, 2024-04-21T08:51:25.043 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_clonerange": 0, 2024-04-21T08:51:25.043 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_getxattr": 1, 2024-04-21T08:51:25.043 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_setxattr": 0, 2024-04-21T08:51:25.043 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_cmpxattr": 0, 2024-04-21T08:51:25.043 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_rmxattr": 0, 2024-04-21T08:51:25.043 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_resetxattrs": 0, 2024-04-21T08:51:25.043 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_call": 0, 2024-04-21T08:51:25.043 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_watch": 0, 2024-04-21T08:51:25.043 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_notify": 0, 2024-04-21T08:51:25.043 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T08:51:25.043 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls": 0, 2024-04-21T08:51:25.043 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls_filter": 0, 2024-04-21T08:51:25.043 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_other": 0, 2024-04-21T08:51:25.044 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_active": 0, 2024-04-21T08:51:25.044 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_send": 0, 2024-04-21T08:51:25.044 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_resend": 0, 2024-04-21T08:51:25.044 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_ping": 0, 2024-04-21T08:51:25.044 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_active": 0, 2024-04-21T08:51:25.044 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_send": 0, 2024-04-21T08:51:25.044 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_resend": 0, 2024-04-21T08:51:25.044 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_active": 0, 2024-04-21T08:51:25.044 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_send": 0, 2024-04-21T08:51:25.044 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_resend": 0, 2024-04-21T08:51:25.044 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_active": 0, 2024-04-21T08:51:25.044 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_send": 0, 2024-04-21T08:51:25.044 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_resend": 0, 2024-04-21T08:51:25.044 INFO:teuthology.orchestra.run.smithi086.stdout: "command_active": 0, 2024-04-21T08:51:25.044 INFO:teuthology.orchestra.run.smithi086.stdout: "command_send": 0, 2024-04-21T08:51:25.045 INFO:teuthology.orchestra.run.smithi086.stdout: "command_resend": 0, 2024-04-21T08:51:25.045 INFO:teuthology.orchestra.run.smithi086.stdout: "map_epoch": 108, 2024-04-21T08:51:25.045 INFO:teuthology.orchestra.run.smithi086.stdout: "map_full": 0, 2024-04-21T08:51:25.045 INFO:teuthology.orchestra.run.smithi086.stdout: "map_inc": 19, 2024-04-21T08:51:25.045 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_sessions": 5, 2024-04-21T08:51:25.045 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_open": 5, 2024-04-21T08:51:25.045 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_close": 0, 2024-04-21T08:51:25.045 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_laggy": 0, 2024-04-21T08:51:25.045 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_wr": 0, 2024-04-21T08:51:25.045 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_rd": 2, 2024-04-21T08:51:25.045 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_del": 0 2024-04-21T08:51:25.045 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.045 INFO:teuthology.orchestra.run.smithi086.stdout: "oft": { 2024-04-21T08:51:25.045 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_objs": 0, 2024-04-21T08:51:25.045 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_kv_pairs": 0, 2024-04-21T08:51:25.046 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_updates": 0, 2024-04-21T08:51:25.046 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_removes": 0 2024-04-21T08:51:25.046 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.046 INFO:teuthology.orchestra.run.smithi086.stdout: "osd-slow-ops": { 2024-04-21T08:51:25.046 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_ops_count": 0 2024-04-21T08:51:25.046 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.046 INFO:teuthology.orchestra.run.smithi086.stdout: "purge_queue": { 2024-04-21T08:51:25.046 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops": 0, 2024-04-21T08:51:25.046 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops_high_water": 0, 2024-04-21T08:51:25.046 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing": 0, 2024-04-21T08:51:25.046 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_high_water": 0, 2024-04-21T08:51:25.046 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed_ops": 0, 2024-04-21T08:51:25.046 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed": 0, 2024-04-21T08:51:25.046 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_item_in_journal": 0 2024-04-21T08:51:25.047 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.047 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T08:51:25.047 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-21T08:51:25.047 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-04-21T08:51:25.047 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-21T08:51:25.047 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 2855, 2024-04-21T08:51:25.047 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 2094585, 2024-04-21T08:51:25.047 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:25.047 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 2855, 2024-04-21T08:51:25.047 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-04-21T08:51:25.047 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-04-21T08:51:25.047 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 2855, 2024-04-21T08:51:25.047 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 2094585, 2024-04-21T08:51:25.047 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-21T08:51:25.048 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.048 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.048 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.048 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:25.048 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.048 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_bytes": { 2024-04-21T08:51:25.048 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-21T08:51:25.048 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-04-21T08:51:25.048 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-21T08:51:25.048 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-04-21T08:51:25.048 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-04-21T08:51:25.048 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:25.048 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:25.048 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 2572, 2024-04-21T08:51:25.049 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 1125, 2024-04-21T08:51:25.049 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 2572, 2024-04-21T08:51:25.049 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 1125, 2024-04-21T08:51:25.049 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-21T08:51:25.049 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.049 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.049 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.049 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:25.049 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.049 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_ops": { 2024-04-21T08:51:25.049 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-21T08:51:25.049 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 1024, 2024-04-21T08:51:25.049 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-21T08:51:25.049 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-04-21T08:51:25.050 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-04-21T08:51:25.050 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:25.050 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:25.050 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 2572, 2024-04-21T08:51:25.050 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 2572, 2024-04-21T08:51:25.050 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 2572, 2024-04-21T08:51:25.050 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 2572, 2024-04-21T08:51:25.050 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-21T08:51:25.050 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.050 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.050 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.050 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:25.050 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.050 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle": { 2024-04-21T08:51:25.051 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-21T08:51:25.051 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-04-21T08:51:25.051 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-21T08:51:25.051 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-04-21T08:51:25.051 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-04-21T08:51:25.051 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:25.051 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:25.051 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-04-21T08:51:25.051 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-04-21T08:51:25.051 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-04-21T08:51:25.051 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-04-21T08:51:25.051 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-21T08:51:25.051 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.051 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.052 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.052 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:25.052 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:25.052 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle-0x5600b49026e0": { 2024-04-21T08:51:25.052 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-21T08:51:25.052 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-04-21T08:51:25.052 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-21T08:51:25.052 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-04-21T08:51:25.052 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-04-21T08:51:25.052 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:25.052 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:25.052 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-04-21T08:51:25.052 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-04-21T08:51:25.052 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-04-21T08:51:25.053 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-04-21T08:51:25.053 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-21T08:51:25.053 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:25.053 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:25.053 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:25.053 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:25.053 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:25.053 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:51:25.801 INFO:tasks.check_counter:Daemon mds.a mds.exported=0 2024-04-21T08:51:25.801 INFO:tasks.check_counter:Daemon mds.a mds.imported=0 2024-04-21T08:51:25.801 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update=0 2024-04-21T08:51:25.801 INFO:tasks.check_counter:Daemon mds.a mds_cache.dir_update_receipt=0 2024-04-21T08:51:25.801 INFO:tasks.check_counter:Daemon mds.a mds.root_rsnaps=0 2024-04-21T08:51:25.801 INFO:tasks.check_counter:Daemon mds.a mds_server.req_mksnap_latency.avgcount=0 2024-04-21T08:51:25.802 INFO:tasks.check_counter:Daemon mds.a mds_server.req_rmsnap_latency.avgcount=0 2024-04-21T08:51:25.802 DEBUG:tasks.check_counter:Getting stats from d 2024-04-21T08:51:25.802 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph daemon mds.d perf dump 2024-04-21T08:51:26.048 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:51:26.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:25 smithi086 ceph-mon[27939]: pgmap v396: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-04-21T08:51:26.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:25 smithi088 ceph-mon[31259]: pgmap v396: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-04-21T08:51:26.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:25 smithi179 ceph-mon[31829]: pgmap v396: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-04-21T08:51:27.922 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:51:27.922 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T08:51:27.923 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 262, 2024-04-21T08:51:27.923 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 251, 2024-04-21T08:51:27.923 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 1601264, 2024-04-21T08:51:27.923 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 154640, 2024-04-21T08:51:27.923 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 4, 2024-04-21T08:51:27.923 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 0, 2024-04-21T08:51:27.923 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.052166053, 2024-04-21T08:51:27.923 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.016734901, 2024-04-21T08:51:27.923 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.038349839, 2024-04-21T08:51:27.923 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.005091630, 2024-04-21T08:51:27.923 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:27.923 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 250, 2024-04-21T08:51:27.923 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.012173653, 2024-04-21T08:51:27.923 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000048694 2024-04-21T08:51:27.923 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:27.924 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:27.924 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:27.924 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:27.924 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:27.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:27.924 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 1601264, 2024-04-21T08:51:27.924 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 154640 2024-04-21T08:51:27.924 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:27.924 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T08:51:27.924 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 6, 2024-04-21T08:51:27.924 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 194, 2024-04-21T08:51:27.924 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 3312, 2024-04-21T08:51:27.924 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 190320, 2024-04-21T08:51:27.924 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 5, 2024-04-21T08:51:27.924 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 3, 2024-04-21T08:51:27.925 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.016748940, 2024-04-21T08:51:27.925 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.011650453, 2024-04-21T08:51:27.925 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.002385755, 2024-04-21T08:51:27.925 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-21T08:51:27.925 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:27.925 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 192, 2024-04-21T08:51:27.925 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.016809081, 2024-04-21T08:51:27.925 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000087547 2024-04-21T08:51:27.925 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:27.925 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:27.925 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:27.925 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:27.925 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:27.925 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:27.925 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-21T08:51:27.926 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 190320 2024-04-21T08:51:27.926 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:27.926 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T08:51:27.926 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 0, 2024-04-21T08:51:27.926 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 2, 2024-04-21T08:51:27.926 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 0, 2024-04-21T08:51:27.926 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 0, 2024-04-21T08:51:27.926 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 2, 2024-04-21T08:51:27.926 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 1, 2024-04-21T08:51:27.926 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.000357340, 2024-04-21T08:51:27.926 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.000000000, 2024-04-21T08:51:27.926 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.000282277, 2024-04-21T08:51:27.926 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-21T08:51:27.926 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:27.926 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:27.927 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:27.927 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:27.927 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:27.927 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:27.927 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:27.927 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:27.927 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:27.927 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:27.927 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-21T08:51:27.927 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 0 2024-04-21T08:51:27.927 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:27.927 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-04-21T08:51:27.927 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 0, 2024-04-21T08:51:27.927 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-04-21T08:51:27.928 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:27.928 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-04-21T08:51:27.928 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-04-21T08:51:27.928 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-04-21T08:51:27.928 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T08:51:27.928 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-04-21T08:51:27.928 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T08:51:27.928 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-04-21T08:51:27.928 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T08:51:27.928 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T08:51:27.928 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T08:51:27.928 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T08:51:27.928 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T08:51:27.928 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-04-21T08:51:27.928 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T08:51:27.928 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T08:51:27.929 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-04-21T08:51:27.929 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-04-21T08:51:27.929 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-04-21T08:51:27.929 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-04-21T08:51:27.929 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T08:51:27.929 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T08:51:27.929 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T08:51:27.929 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T08:51:27.929 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T08:51:27.929 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-04-21T08:51:27.929 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-04-21T08:51:27.929 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-04-21T08:51:27.929 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T08:51:27.929 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T08:51:27.929 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-04-21T08:51:27.930 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-04-21T08:51:27.930 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-04-21T08:51:27.930 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-04-21T08:51:27.930 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T08:51:27.930 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-04-21T08:51:27.930 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T08:51:27.930 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-04-21T08:51:27.930 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 149017, 2024-04-21T08:51:27.930 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 31, 2024-04-21T08:51:27.930 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-04-21T08:51:27.930 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-04-21T08:51:27.930 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-04-21T08:51:27.931 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-04-21T08:51:27.931 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-04-21T08:51:27.931 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-04-21T08:51:27.931 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-04-21T08:51:27.931 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-04-21T08:51:27.931 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 0, 2024-04-21T08:51:27.931 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 0, 2024-04-21T08:51:27.931 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T08:51:27.931 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-04-21T08:51:27.931 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-04-21T08:51:27.931 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-04-21T08:51:27.931 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 928, 2024-04-21T08:51:27.931 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 2, 2024-04-21T08:51:27.931 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-04-21T08:51:27.931 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-04-21T08:51:27.932 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-04-21T08:51:27.932 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-04-21T08:51:27.932 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:27.932 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T08:51:27.932 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-21T08:51:27.932 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-04-21T08:51:27.932 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-21T08:51:27.932 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 264, 2024-04-21T08:51:27.932 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 1564304, 2024-04-21T08:51:27.932 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:27.932 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 264, 2024-04-21T08:51:27.932 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-04-21T08:51:27.932 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-04-21T08:51:27.932 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 264, 2024-04-21T08:51:27.933 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 1564304, 2024-04-21T08:51:27.933 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-21T08:51:27.933 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:27.933 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:27.933 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:27.933 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:27.933 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:27.933 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:51:28.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:27 smithi088 ceph-mon[31259]: pgmap v397: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-04-21T08:51:28.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:27 smithi179 ceph-mon[31829]: pgmap v397: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-04-21T08:51:28.274 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:27 smithi086 ceph-mon[27939]: pgmap v397: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 7.1 KiB/s rd, 3.2 KiB/s wr, 8 op/s 2024-04-21T08:51:28.649 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.d 2024-04-21T08:51:28.649 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.d 2024-04-21T08:51:28.649 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.d 2024-04-21T08:51:28.649 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.d 2024-04-21T08:51:28.649 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.d 2024-04-21T08:51:28.649 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.d 2024-04-21T08:51:28.649 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.d 2024-04-21T08:51:28.649 DEBUG:tasks.check_counter:Getting stats from g 2024-04-21T08:51:28.650 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph daemon mds.g perf dump 2024-04-21T08:51:28.903 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:51:30.084 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:29 smithi086 ceph-mon[27939]: pgmap v398: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2024-04-21T08:51:30.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:29 smithi088 ceph-mon[31259]: pgmap v398: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2024-04-21T08:51:30.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:29 smithi179 ceph-mon[31829]: pgmap v398: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 10 KiB/s rd, 4.2 KiB/s wr, 13 op/s 2024-04-21T08:51:30.606 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:51:30.606 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T08:51:30.606 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 884, 2024-04-21T08:51:30.607 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 885, 2024-04-21T08:51:30.607 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 297067, 2024-04-21T08:51:30.607 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 280795, 2024-04-21T08:51:30.607 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 6, 2024-04-21T08:51:30.607 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 2, 2024-04-21T08:51:30.607 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.099277310, 2024-04-21T08:51:30.607 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.042105726, 2024-04-21T08:51:30.607 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.020349730, 2024-04-21T08:51:30.607 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.026245049, 2024-04-21T08:51:30.607 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:30.607 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 883, 2024-04-21T08:51:30.607 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.048337227, 2024-04-21T08:51:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000054742 2024-04-21T08:51:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-21T08:51:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 0 2024-04-21T08:51:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T08:51:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 1917, 2024-04-21T08:51:30.608 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 1905, 2024-04-21T08:51:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 2055676, 2024-04-21T08:51:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 675925, 2024-04-21T08:51:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 4, 2024-04-21T08:51:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 3, 2024-04-21T08:51:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.154558708, 2024-04-21T08:51:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.046924566, 2024-04-21T08:51:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.064151835, 2024-04-21T08:51:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.045077050, 2024-04-21T08:51:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 1904, 2024-04-21T08:51:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.061685758, 2024-04-21T08:51:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000032397 2024-04-21T08:51:30.609 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.610 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:30.610 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.610 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.610 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.610 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.610 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 1638640, 2024-04-21T08:51:30.610 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 156256 2024-04-21T08:51:30.610 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.610 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T08:51:30.610 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 9, 2024-04-21T08:51:30.610 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 194, 2024-04-21T08:51:30.610 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 4660, 2024-04-21T08:51:30.610 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 493733, 2024-04-21T08:51:30.611 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 7, 2024-04-21T08:51:30.611 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 5, 2024-04-21T08:51:30.611 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.014674993, 2024-04-21T08:51:30.611 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.009780454, 2024-04-21T08:51:30.611 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.002575116, 2024-04-21T08:51:30.611 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000052462, 2024-04-21T08:51:30.611 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:30.611 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 192, 2024-04-21T08:51:30.611 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.014695761, 2024-04-21T08:51:30.611 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000076540 2024-04-21T08:51:30.611 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.611 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:30.611 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.612 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.612 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.612 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-21T08:51:30.612 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 492768 2024-04-21T08:51:30.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.612 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-04-21T08:51:30.612 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 1, 2024-04-21T08:51:30.612 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-04-21T08:51:30.612 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.612 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-MDSRank": { 2024-04-21T08:51:30.612 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-04-21T08:51:30.612 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-04-21T08:51:30.612 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 4252, 2024-04-21T08:51:30.613 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.101931149, 2024-04-21T08:51:30.613 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000023972 2024-04-21T08:51:30.613 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:30.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.613 INFO:teuthology.orchestra.run.smithi086.stdout: "finisher-PurgeQueue": { 2024-04-21T08:51:30.613 INFO:teuthology.orchestra.run.smithi086.stdout: "queue_len": 0, 2024-04-21T08:51:30.613 INFO:teuthology.orchestra.run.smithi086.stdout: "complete_latency": { 2024-04-21T08:51:30.613 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.613 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.613 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.613 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:30.613 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.613 INFO:teuthology.orchestra.run.smithi086.stdout: "mds": { 2024-04-21T08:51:30.614 INFO:teuthology.orchestra.run.smithi086.stdout: "request": 0, 2024-04-21T08:51:30.614 INFO:teuthology.orchestra.run.smithi086.stdout: "reply": 0, 2024-04-21T08:51:30.614 INFO:teuthology.orchestra.run.smithi086.stdout: "reply_latency": { 2024-04-21T08:51:30.614 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.614 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.614 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.614 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.614 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_reply": 0, 2024-04-21T08:51:30.614 INFO:teuthology.orchestra.run.smithi086.stdout: "forward": 0, 2024-04-21T08:51:30.614 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_complete": 0, 2024-04-21T08:51:30.614 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_fetch_keys": 0, 2024-04-21T08:51:30.614 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_commit": 0, 2024-04-21T08:51:30.614 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_split": 0, 2024-04-21T08:51:30.614 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_merge": 0, 2024-04-21T08:51:30.615 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes": 0, 2024-04-21T08:51:30.615 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_top": 0, 2024-04-21T08:51:30.615 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_bottom": 0, 2024-04-21T08:51:30.615 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pin_tail": 0, 2024-04-21T08:51:30.615 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_pinned": 0, 2024-04-21T08:51:30.615 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_expired": 0, 2024-04-21T08:51:30.615 INFO:teuthology.orchestra.run.smithi086.stdout: "inodes_with_caps": 0, 2024-04-21T08:51:30.615 INFO:teuthology.orchestra.run.smithi086.stdout: "caps": 0, 2024-04-21T08:51:30.615 INFO:teuthology.orchestra.run.smithi086.stdout: "subtrees": 1, 2024-04-21T08:51:30.615 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse": 0, 2024-04-21T08:51:30.615 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_hit": 0, 2024-04-21T08:51:30.615 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_forward": 0, 2024-04-21T08:51:30.615 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_discover": 0, 2024-04-21T08:51:30.616 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_dir_fetch": 0, 2024-04-21T08:51:30.616 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_remote_ino": 0, 2024-04-21T08:51:30.616 INFO:teuthology.orchestra.run.smithi086.stdout: "traverse_lock": 0, 2024-04-21T08:51:30.616 INFO:teuthology.orchestra.run.smithi086.stdout: "load_cent": 0, 2024-04-21T08:51:30.616 INFO:teuthology.orchestra.run.smithi086.stdout: "q": 0, 2024-04-21T08:51:30.616 INFO:teuthology.orchestra.run.smithi086.stdout: "exported": 0, 2024-04-21T08:51:30.616 INFO:teuthology.orchestra.run.smithi086.stdout: "exported_inodes": 0, 2024-04-21T08:51:30.616 INFO:teuthology.orchestra.run.smithi086.stdout: "imported": 0, 2024-04-21T08:51:30.616 INFO:teuthology.orchestra.run.smithi086.stdout: "imported_inodes": 0, 2024-04-21T08:51:30.616 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_dir_fetch": 0, 2024-04-21T08:51:30.616 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_backtrace_fetch": 0, 2024-04-21T08:51:30.616 INFO:teuthology.orchestra.run.smithi086.stdout: "openino_peer_discover": 0, 2024-04-21T08:51:30.616 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rfiles": 0, 2024-04-21T08:51:30.616 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rbytes": 0, 2024-04-21T08:51:30.617 INFO:teuthology.orchestra.run.smithi086.stdout: "root_rsnaps": 0, 2024-04-21T08:51:30.617 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_fetch": 0, 2024-04-21T08:51:30.617 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_set_tag": 0, 2024-04-21T08:51:30.617 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T08:51:30.617 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_inotable_repaired": 0, 2024-04-21T08:51:30.617 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_inodes": 0, 2024-04-21T08:51:30.617 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dir_base_inodes": 0, 2024-04-21T08:51:30.617 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_dirfrag_rstats": 0, 2024-04-21T08:51:30.617 INFO:teuthology.orchestra.run.smithi086.stdout: "scrub_file_inodes": 0, 2024-04-21T08:51:30.617 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_inode_file_caps": 0, 2024-04-21T08:51:30.617 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_revoke": 0, 2024-04-21T08:51:30.617 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_grant": 0, 2024-04-21T08:51:30.617 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_trunc": 0, 2024-04-21T08:51:30.617 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T08:51:30.618 INFO:teuthology.orchestra.run.smithi086.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T08:51:30.618 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps": 0, 2024-04-21T08:51:30.618 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_caps_dirty": 0, 2024-04-21T08:51:30.618 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_cap_release": 0, 2024-04-21T08:51:30.618 INFO:teuthology.orchestra.run.smithi086.stdout: "process_request_cap_release": 0 2024-04-21T08:51:30.618 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.618 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_cache": { 2024-04-21T08:51:30.618 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update": 0, 2024-04-21T08:51:30.618 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_update_receipt": 0, 2024-04-21T08:51:30.618 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_try_discover": 0, 2024-04-21T08:51:30.618 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_send_discover": 0, 2024-04-21T08:51:30.618 INFO:teuthology.orchestra.run.smithi086.stdout: "dir_handle_discover": 0, 2024-04-21T08:51:30.618 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays": 0, 2024-04-21T08:51:30.619 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_delayed": 0, 2024-04-21T08:51:30.619 INFO:teuthology.orchestra.run.smithi086.stdout: "num_strays_enqueuing": 0, 2024-04-21T08:51:30.619 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_created": 0, 2024-04-21T08:51:30.619 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_enqueued": 0, 2024-04-21T08:51:30.619 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_reintegrated": 0, 2024-04-21T08:51:30.619 INFO:teuthology.orchestra.run.smithi086.stdout: "strays_migrated": 0, 2024-04-21T08:51:30.619 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_processing": 0, 2024-04-21T08:51:30.619 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_enqueued": 0, 2024-04-21T08:51:30.619 INFO:teuthology.orchestra.run.smithi086.stdout: "num_recovering_prioritized": 0, 2024-04-21T08:51:30.619 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_started": 0, 2024-04-21T08:51:30.619 INFO:teuthology.orchestra.run.smithi086.stdout: "recovery_completed": 0, 2024-04-21T08:51:30.619 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_path": 0, 2024-04-21T08:51:30.619 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_quiesce_inode": 0, 2024-04-21T08:51:30.620 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_enqueue_scrub": 0, 2024-04-21T08:51:30.620 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_exportdir": 0, 2024-04-21T08:51:30.620 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_flush": 0, 2024-04-21T08:51:30.620 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragmentdir": 0, 2024-04-21T08:51:30.620 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_fragstats": 0, 2024-04-21T08:51:30.620 INFO:teuthology.orchestra.run.smithi086.stdout: "ireq_inodestats": 0 2024-04-21T08:51:30.620 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.620 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_log": { 2024-04-21T08:51:30.620 INFO:teuthology.orchestra.run.smithi086.stdout: "evlrg": 0, 2024-04-21T08:51:30.620 INFO:teuthology.orchestra.run.smithi086.stdout: "evadd": 0, 2024-04-21T08:51:30.620 INFO:teuthology.orchestra.run.smithi086.stdout: "evex": 0, 2024-04-21T08:51:30.620 INFO:teuthology.orchestra.run.smithi086.stdout: "evtrm": 0, 2024-04-21T08:51:30.620 INFO:teuthology.orchestra.run.smithi086.stdout: "ev": 3, 2024-04-21T08:51:30.620 INFO:teuthology.orchestra.run.smithi086.stdout: "evexg": 0, 2024-04-21T08:51:30.621 INFO:teuthology.orchestra.run.smithi086.stdout: "evexd": 0, 2024-04-21T08:51:30.621 INFO:teuthology.orchestra.run.smithi086.stdout: "segadd": 0, 2024-04-21T08:51:30.621 INFO:teuthology.orchestra.run.smithi086.stdout: "segex": 0, 2024-04-21T08:51:30.621 INFO:teuthology.orchestra.run.smithi086.stdout: "segtrm": 0, 2024-04-21T08:51:30.621 INFO:teuthology.orchestra.run.smithi086.stdout: "seg": 2, 2024-04-21T08:51:30.621 INFO:teuthology.orchestra.run.smithi086.stdout: "segmjr": 2, 2024-04-21T08:51:30.621 INFO:teuthology.orchestra.run.smithi086.stdout: "segexg": 0, 2024-04-21T08:51:30.621 INFO:teuthology.orchestra.run.smithi086.stdout: "segexd": 0, 2024-04-21T08:51:30.621 INFO:teuthology.orchestra.run.smithi086.stdout: "expos": 4194304, 2024-04-21T08:51:30.621 INFO:teuthology.orchestra.run.smithi086.stdout: "wrpos": 4195200, 2024-04-21T08:51:30.621 INFO:teuthology.orchestra.run.smithi086.stdout: "rdpos": 4194892, 2024-04-21T08:51:30.621 INFO:teuthology.orchestra.run.smithi086.stdout: "jlat": { 2024-04-21T08:51:30.621 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.621 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.622 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.622 INFO:teuthology.orchestra.run.smithi086.stdout: "replayed": 3 2024-04-21T08:51:30.622 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.622 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_mem": { 2024-04-21T08:51:30.622 INFO:teuthology.orchestra.run.smithi086.stdout: "ino": 3, 2024-04-21T08:51:30.622 INFO:teuthology.orchestra.run.smithi086.stdout: "ino+": 3, 2024-04-21T08:51:30.622 INFO:teuthology.orchestra.run.smithi086.stdout: "ino-": 0, 2024-04-21T08:51:30.622 INFO:teuthology.orchestra.run.smithi086.stdout: "dir": 1, 2024-04-21T08:51:30.622 INFO:teuthology.orchestra.run.smithi086.stdout: "dir+": 1, 2024-04-21T08:51:30.622 INFO:teuthology.orchestra.run.smithi086.stdout: "dir-": 0, 2024-04-21T08:51:30.622 INFO:teuthology.orchestra.run.smithi086.stdout: "dn": 0, 2024-04-21T08:51:30.622 INFO:teuthology.orchestra.run.smithi086.stdout: "dn+": 0, 2024-04-21T08:51:30.623 INFO:teuthology.orchestra.run.smithi086.stdout: "dn-": 0, 2024-04-21T08:51:30.623 INFO:teuthology.orchestra.run.smithi086.stdout: "cap": 0, 2024-04-21T08:51:30.623 INFO:teuthology.orchestra.run.smithi086.stdout: "cap+": 0, 2024-04-21T08:51:30.623 INFO:teuthology.orchestra.run.smithi086.stdout: "cap-": 0, 2024-04-21T08:51:30.623 INFO:teuthology.orchestra.run.smithi086.stdout: "rss": 50604, 2024-04-21T08:51:30.623 INFO:teuthology.orchestra.run.smithi086.stdout: "heap": 215416 2024-04-21T08:51:30.623 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.623 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_server": { 2024-04-21T08:51:30.623 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_client_request": 0, 2024-04-21T08:51:30.623 INFO:teuthology.orchestra.run.smithi086.stdout: "dispatch_server_request": 0, 2024-04-21T08:51:30.623 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_request": 0, 2024-04-21T08:51:30.623 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_client_session": 0, 2024-04-21T08:51:30.623 INFO:teuthology.orchestra.run.smithi086.stdout: "handle_peer_request": 0, 2024-04-21T08:51:30.624 INFO:teuthology.orchestra.run.smithi086.stdout: "req_create_latency": { 2024-04-21T08:51:30.624 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.624 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.624 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.624 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getattr_latency": { 2024-04-21T08:51:30.624 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.624 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.624 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.624 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getfilelock_latency": { 2024-04-21T08:51:30.624 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.624 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.624 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.624 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.625 INFO:teuthology.orchestra.run.smithi086.stdout: "req_link_latency": { 2024-04-21T08:51:30.625 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.625 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.625 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.625 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.625 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookup_latency": { 2024-04-21T08:51:30.625 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.625 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.625 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.625 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.625 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookuphash_latency": { 2024-04-21T08:51:30.625 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.625 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.626 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.626 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupino_latency": { 2024-04-21T08:51:30.626 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.626 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.626 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.626 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupname_latency": { 2024-04-21T08:51:30.626 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.626 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.626 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.626 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.626 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupparent_latency": { 2024-04-21T08:51:30.626 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.627 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.627 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.627 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lookupsnap_latency": { 2024-04-21T08:51:30.627 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.627 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.627 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.627 INFO:teuthology.orchestra.run.smithi086.stdout: "req_lssnap_latency": { 2024-04-21T08:51:30.627 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.627 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.627 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.627 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.627 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mkdir_latency": { 2024-04-21T08:51:30.628 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.628 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.628 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.629 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.629 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mknod_latency": { 2024-04-21T08:51:30.629 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.629 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.629 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.629 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.629 INFO:teuthology.orchestra.run.smithi086.stdout: "req_mksnap_latency": { 2024-04-21T08:51:30.629 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.629 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.629 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.629 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.629 INFO:teuthology.orchestra.run.smithi086.stdout: "req_open_latency": { 2024-04-21T08:51:30.629 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.629 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.630 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.630 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.630 INFO:teuthology.orchestra.run.smithi086.stdout: "req_readdir_latency": { 2024-04-21T08:51:30.630 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.630 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.630 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.630 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.630 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rename_latency": { 2024-04-21T08:51:30.630 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.630 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.630 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.630 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.630 INFO:teuthology.orchestra.run.smithi086.stdout: "req_renamesnap_latency": { 2024-04-21T08:51:30.630 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.631 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.631 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.631 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.631 INFO:teuthology.orchestra.run.smithi086.stdout: "req_snapdiff_latency": { 2024-04-21T08:51:30.631 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.631 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.631 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.631 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.631 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmdir_latency": { 2024-04-21T08:51:30.631 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.631 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.631 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.631 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.631 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmsnap_latency": { 2024-04-21T08:51:30.632 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.632 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.632 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.632 INFO:teuthology.orchestra.run.smithi086.stdout: "req_rmxattr_latency": { 2024-04-21T08:51:30.632 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.632 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.632 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.632 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setattr_latency": { 2024-04-21T08:51:30.632 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.632 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.632 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.632 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.633 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setdirlayout_latency": { 2024-04-21T08:51:30.633 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.633 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.633 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.633 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setfilelock_latency": { 2024-04-21T08:51:30.633 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.633 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.633 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.633 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.633 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setlayout_latency": { 2024-04-21T08:51:30.633 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.633 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.633 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.634 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.634 INFO:teuthology.orchestra.run.smithi086.stdout: "req_setxattr_latency": { 2024-04-21T08:51:30.634 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.634 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.634 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.634 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.634 INFO:teuthology.orchestra.run.smithi086.stdout: "req_symlink_latency": { 2024-04-21T08:51:30.634 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.634 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.634 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.634 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.634 INFO:teuthology.orchestra.run.smithi086.stdout: "req_unlink_latency": { 2024-04-21T08:51:30.634 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.634 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.635 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.635 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_revoke_eviction": 0, 2024-04-21T08:51:30.635 INFO:teuthology.orchestra.run.smithi086.stdout: "cap_acquisition_throttle": 0, 2024-04-21T08:51:30.635 INFO:teuthology.orchestra.run.smithi086.stdout: "req_getvxattr_latency": { 2024-04-21T08:51:30.635 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.635 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.635 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.635 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:30.635 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.635 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_sessions": { 2024-04-21T08:51:30.635 INFO:teuthology.orchestra.run.smithi086.stdout: "session_count": 1, 2024-04-21T08:51:30.636 INFO:teuthology.orchestra.run.smithi086.stdout: "session_add": 1, 2024-04-21T08:51:30.636 INFO:teuthology.orchestra.run.smithi086.stdout: "session_remove": 0, 2024-04-21T08:51:30.636 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_open": 1, 2024-04-21T08:51:30.636 INFO:teuthology.orchestra.run.smithi086.stdout: "sessions_stale": 0, 2024-04-21T08:51:30.636 INFO:teuthology.orchestra.run.smithi086.stdout: "total_load": 0, 2024-04-21T08:51:30.636 INFO:teuthology.orchestra.run.smithi086.stdout: "average_load": 0, 2024-04-21T08:51:30.636 INFO:teuthology.orchestra.run.smithi086.stdout: "avg_session_uptime": 1912, 2024-04-21T08:51:30.636 INFO:teuthology.orchestra.run.smithi086.stdout: "mdthresh_evicted": 0 2024-04-21T08:51:30.636 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.636 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-04-21T08:51:30.636 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-04-21T08:51:30.636 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-04-21T08:51:30.637 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T08:51:30.637 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-04-21T08:51:30.637 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T08:51:30.637 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-04-21T08:51:30.637 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T08:51:30.637 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T08:51:30.637 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T08:51:30.637 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T08:51:30.637 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T08:51:30.637 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-04-21T08:51:30.637 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T08:51:30.638 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T08:51:30.638 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-04-21T08:51:30.638 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-04-21T08:51:30.638 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-04-21T08:51:30.638 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-04-21T08:51:30.638 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T08:51:30.638 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T08:51:30.638 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T08:51:30.638 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T08:51:30.638 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T08:51:30.638 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-04-21T08:51:30.638 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-04-21T08:51:30.638 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-04-21T08:51:30.639 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T08:51:30.639 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T08:51:30.639 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-04-21T08:51:30.639 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-04-21T08:51:30.639 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-04-21T08:51:30.639 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-04-21T08:51:30.639 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T08:51:30.639 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-04-21T08:51:30.639 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T08:51:30.639 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-04-21T08:51:30.640 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 197473, 2024-04-21T08:51:30.640 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 55, 2024-04-21T08:51:30.640 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-04-21T08:51:30.640 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-04-21T08:51:30.640 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-04-21T08:51:30.640 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-04-21T08:51:30.640 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-04-21T08:51:30.640 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-04-21T08:51:30.640 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-04-21T08:51:30.640 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-04-21T08:51:30.640 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 9512, 2024-04-21T08:51:30.640 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 182, 2024-04-21T08:51:30.640 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T08:51:30.641 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-04-21T08:51:30.641 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-04-21T08:51:30.641 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-04-21T08:51:30.641 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 8512, 2024-04-21T08:51:30.641 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 13, 2024-04-21T08:51:30.641 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-04-21T08:51:30.641 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-04-21T08:51:30.641 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-04-21T08:51:30.641 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-04-21T08:51:30.641 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.641 INFO:teuthology.orchestra.run.smithi086.stdout: "objecter": { 2024-04-21T08:51:30.641 INFO:teuthology.orchestra.run.smithi086.stdout: "op_active": 0, 2024-04-21T08:51:30.641 INFO:teuthology.orchestra.run.smithi086.stdout: "op_laggy": 0, 2024-04-21T08:51:30.642 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send": 2520, 2024-04-21T08:51:30.642 INFO:teuthology.orchestra.run.smithi086.stdout: "op_send_bytes": 21, 2024-04-21T08:51:30.642 INFO:teuthology.orchestra.run.smithi086.stdout: "op_resend": 3, 2024-04-21T08:51:30.642 INFO:teuthology.orchestra.run.smithi086.stdout: "op_reply": 2520, 2024-04-21T08:51:30.642 INFO:teuthology.orchestra.run.smithi086.stdout: "op_latency": { 2024-04-21T08:51:30.642 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 2520, 2024-04-21T08:51:30.642 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 2.119955355, 2024-04-21T08:51:30.642 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000841252 2024-04-21T08:51:30.642 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.642 INFO:teuthology.orchestra.run.smithi086.stdout: "op_inflight": 0, 2024-04-21T08:51:30.642 INFO:teuthology.orchestra.run.smithi086.stdout: "oplen_avg": { 2024-04-21T08:51:30.642 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 2520, 2024-04-21T08:51:30.642 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 2521 2024-04-21T08:51:30.643 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.643 INFO:teuthology.orchestra.run.smithi086.stdout: "op": 2520, 2024-04-21T08:51:30.643 INFO:teuthology.orchestra.run.smithi086.stdout: "op_r": 2520, 2024-04-21T08:51:30.643 INFO:teuthology.orchestra.run.smithi086.stdout: "op_w": 0, 2024-04-21T08:51:30.643 INFO:teuthology.orchestra.run.smithi086.stdout: "op_rmw": 0, 2024-04-21T08:51:30.643 INFO:teuthology.orchestra.run.smithi086.stdout: "op_pg": 0, 2024-04-21T08:51:30.643 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_stat": 1631, 2024-04-21T08:51:30.643 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_create": 0, 2024-04-21T08:51:30.643 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_read": 887, 2024-04-21T08:51:30.643 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_write": 0, 2024-04-21T08:51:30.643 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writefull": 0, 2024-04-21T08:51:30.643 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_writesame": 0, 2024-04-21T08:51:30.643 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_append": 0, 2024-04-21T08:51:30.644 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_zero": 0, 2024-04-21T08:51:30.644 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_truncate": 0, 2024-04-21T08:51:30.644 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_delete": 0, 2024-04-21T08:51:30.644 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_mapext": 0, 2024-04-21T08:51:30.644 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_sparse_read": 0, 2024-04-21T08:51:30.644 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_clonerange": 0, 2024-04-21T08:51:30.644 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_getxattr": 1, 2024-04-21T08:51:30.644 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_setxattr": 0, 2024-04-21T08:51:30.644 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_cmpxattr": 0, 2024-04-21T08:51:30.644 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_rmxattr": 0, 2024-04-21T08:51:30.644 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_resetxattrs": 0, 2024-04-21T08:51:30.644 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_call": 0, 2024-04-21T08:51:30.644 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_watch": 0, 2024-04-21T08:51:30.644 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_notify": 0, 2024-04-21T08:51:30.645 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T08:51:30.645 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls": 0, 2024-04-21T08:51:30.645 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_pgls_filter": 0, 2024-04-21T08:51:30.645 INFO:teuthology.orchestra.run.smithi086.stdout: "osdop_other": 0, 2024-04-21T08:51:30.645 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_active": 0, 2024-04-21T08:51:30.645 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_send": 0, 2024-04-21T08:51:30.645 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_resend": 0, 2024-04-21T08:51:30.645 INFO:teuthology.orchestra.run.smithi086.stdout: "linger_ping": 0, 2024-04-21T08:51:30.645 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_active": 0, 2024-04-21T08:51:30.645 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_send": 0, 2024-04-21T08:51:30.645 INFO:teuthology.orchestra.run.smithi086.stdout: "poolop_resend": 0, 2024-04-21T08:51:30.645 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_active": 0, 2024-04-21T08:51:30.645 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_send": 0, 2024-04-21T08:51:30.646 INFO:teuthology.orchestra.run.smithi086.stdout: "poolstat_resend": 0, 2024-04-21T08:51:30.646 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_active": 0, 2024-04-21T08:51:30.646 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_send": 0, 2024-04-21T08:51:30.646 INFO:teuthology.orchestra.run.smithi086.stdout: "statfs_resend": 0, 2024-04-21T08:51:30.646 INFO:teuthology.orchestra.run.smithi086.stdout: "command_active": 0, 2024-04-21T08:51:30.646 INFO:teuthology.orchestra.run.smithi086.stdout: "command_send": 0, 2024-04-21T08:51:30.646 INFO:teuthology.orchestra.run.smithi086.stdout: "command_resend": 0, 2024-04-21T08:51:30.646 INFO:teuthology.orchestra.run.smithi086.stdout: "map_epoch": 108, 2024-04-21T08:51:30.646 INFO:teuthology.orchestra.run.smithi086.stdout: "map_full": 0, 2024-04-21T08:51:30.646 INFO:teuthology.orchestra.run.smithi086.stdout: "map_inc": 19, 2024-04-21T08:51:30.646 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_sessions": 6, 2024-04-21T08:51:30.646 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_open": 6, 2024-04-21T08:51:30.646 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_session_close": 0, 2024-04-21T08:51:30.646 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_laggy": 0, 2024-04-21T08:51:30.647 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_wr": 0, 2024-04-21T08:51:30.647 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_rd": 2, 2024-04-21T08:51:30.647 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_del": 0 2024-04-21T08:51:30.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.647 INFO:teuthology.orchestra.run.smithi086.stdout: "oft": { 2024-04-21T08:51:30.647 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_objs": 0, 2024-04-21T08:51:30.647 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_kv_pairs": 0, 2024-04-21T08:51:30.647 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_updates": 0, 2024-04-21T08:51:30.647 INFO:teuthology.orchestra.run.smithi086.stdout: "omap_total_removes": 0 2024-04-21T08:51:30.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.647 INFO:teuthology.orchestra.run.smithi086.stdout: "osd-slow-ops": { 2024-04-21T08:51:30.647 INFO:teuthology.orchestra.run.smithi086.stdout: "slow_ops_count": 0 2024-04-21T08:51:30.647 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.647 INFO:teuthology.orchestra.run.smithi086.stdout: "purge_queue": { 2024-04-21T08:51:30.648 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops": 0, 2024-04-21T08:51:30.648 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_ops_high_water": 0, 2024-04-21T08:51:30.648 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing": 0, 2024-04-21T08:51:30.648 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executing_high_water": 0, 2024-04-21T08:51:30.648 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed_ops": 0, 2024-04-21T08:51:30.648 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_executed": 0, 2024-04-21T08:51:30.648 INFO:teuthology.orchestra.run.smithi086.stdout: "pq_item_in_journal": 0 2024-04-21T08:51:30.648 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.648 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T08:51:30.648 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-21T08:51:30.648 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-04-21T08:51:30.648 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-21T08:51:30.648 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 2806, 2024-04-21T08:51:30.648 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 2087383, 2024-04-21T08:51:30.649 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:30.649 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 2806, 2024-04-21T08:51:30.649 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-04-21T08:51:30.649 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-04-21T08:51:30.649 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 2806, 2024-04-21T08:51:30.649 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 2087383, 2024-04-21T08:51:30.649 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-21T08:51:30.649 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.649 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.649 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.649 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:30.649 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.649 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_bytes": { 2024-04-21T08:51:30.650 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-21T08:51:30.650 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-04-21T08:51:30.650 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-21T08:51:30.650 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-04-21T08:51:30.650 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-04-21T08:51:30.650 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:30.650 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:30.650 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 2520, 2024-04-21T08:51:30.650 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 901, 2024-04-21T08:51:30.650 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 2520, 2024-04-21T08:51:30.650 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 901, 2024-04-21T08:51:30.650 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-21T08:51:30.650 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.650 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.651 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.651 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:30.651 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.651 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-objecter_ops": { 2024-04-21T08:51:30.651 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-21T08:51:30.651 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 1024, 2024-04-21T08:51:30.651 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-21T08:51:30.651 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-04-21T08:51:30.651 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-04-21T08:51:30.651 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:30.651 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:30.651 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 2520, 2024-04-21T08:51:30.651 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 2520, 2024-04-21T08:51:30.651 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 2520, 2024-04-21T08:51:30.652 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 2520, 2024-04-21T08:51:30.652 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-21T08:51:30.652 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.652 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.652 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.652 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:30.652 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.652 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle": { 2024-04-21T08:51:30.652 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-21T08:51:30.652 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-04-21T08:51:30.652 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-21T08:51:30.652 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-04-21T08:51:30.652 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-04-21T08:51:30.652 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:30.653 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:30.653 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-04-21T08:51:30.653 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-04-21T08:51:30.653 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-04-21T08:51:30.653 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-04-21T08:51:30.653 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-21T08:51:30.653 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.653 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.653 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.653 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:30.653 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:30.653 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-write_buf_throttle-0x55fad8af26e0": { 2024-04-21T08:51:30.653 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-21T08:51:30.653 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 3758096384, 2024-04-21T08:51:30.654 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-21T08:51:30.654 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 0, 2024-04-21T08:51:30.654 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 0, 2024-04-21T08:51:30.654 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:30.654 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:30.654 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-04-21T08:51:30.654 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-04-21T08:51:30.654 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 0, 2024-04-21T08:51:30.654 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 0, 2024-04-21T08:51:30.654 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-21T08:51:30.654 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:30.654 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:30.654 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:30.655 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:30.655 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:30.655 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:51:31.438 INFO:tasks.check_counter:Daemon mds.g mds.exported=0 2024-04-21T08:51:31.438 INFO:tasks.check_counter:Daemon mds.g mds.imported=0 2024-04-21T08:51:31.438 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update=0 2024-04-21T08:51:31.439 INFO:tasks.check_counter:Daemon mds.g mds_cache.dir_update_receipt=0 2024-04-21T08:51:31.439 INFO:tasks.check_counter:Daemon mds.g mds.root_rsnaps=0 2024-04-21T08:51:31.439 INFO:tasks.check_counter:Daemon mds.g mds_server.req_mksnap_latency.avgcount=0 2024-04-21T08:51:31.439 INFO:tasks.check_counter:Daemon mds.g mds_server.req_rmsnap_latency.avgcount=0 2024-04-21T08:51:31.439 DEBUG:tasks.check_counter:Getting stats from j 2024-04-21T08:51:31.439 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph daemon mds.j perf dump 2024-04-21T08:51:31.676 INFO:teuthology.orchestra.run.smithi086.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.a/config 2024-04-21T08:51:32.081 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:31 smithi086 ceph-mon[27939]: pgmap v399: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-04-21T08:51:32.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:31 smithi088 ceph-mon[31259]: pgmap v399: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-04-21T08:51:32.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:31 smithi179 ceph-mon[31829]: pgmap v399: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.1 KiB/s wr, 8 op/s 2024-04-21T08:51:33.440 INFO:teuthology.orchestra.run.smithi086.stdout:{ 2024-04-21T08:51:33.440 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T08:51:33.440 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 264, 2024-04-21T08:51:33.440 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 253, 2024-04-21T08:51:33.440 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 1601808, 2024-04-21T08:51:33.440 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 155920, 2024-04-21T08:51:33.441 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 4, 2024-04-21T08:51:33.441 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 0, 2024-04-21T08:51:33.441 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.050953769, 2024-04-21T08:51:33.441 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.017373719, 2024-04-21T08:51:33.441 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.037781081, 2024-04-21T08:51:33.441 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.005126628, 2024-04-21T08:51:33.441 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:33.441 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 252, 2024-04-21T08:51:33.441 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.012613030, 2024-04-21T08:51:33.441 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000050051 2024-04-21T08:51:33.441 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:33.441 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:33.441 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:33.441 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:33.441 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:33.442 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:33.442 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 1601808, 2024-04-21T08:51:33.442 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 155920 2024-04-21T08:51:33.442 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:33.442 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T08:51:33.442 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 6, 2024-04-21T08:51:33.442 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 196, 2024-04-21T08:51:33.442 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 3312, 2024-04-21T08:51:33.442 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 190672, 2024-04-21T08:51:33.442 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 5, 2024-04-21T08:51:33.442 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 3, 2024-04-21T08:51:33.442 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.017744425, 2024-04-21T08:51:33.442 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.012605870, 2024-04-21T08:51:33.442 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.002277240, 2024-04-21T08:51:33.443 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-21T08:51:33.443 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:33.443 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 194, 2024-04-21T08:51:33.443 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.015353354, 2024-04-21T08:51:33.443 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000079141 2024-04-21T08:51:33.443 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:33.443 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:33.443 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:33.443 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:33.443 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:33.443 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:33.443 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 3312, 2024-04-21T08:51:33.443 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 190672 2024-04-21T08:51:33.443 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:33.443 INFO:teuthology.orchestra.run.smithi086.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T08:51:33.444 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_messages": 0, 2024-04-21T08:51:33.444 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages": 2, 2024-04-21T08:51:33.444 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_bytes": 0, 2024-04-21T08:51:33.444 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_bytes": 0, 2024-04-21T08:51:33.444 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_created_connections": 2, 2024-04-21T08:51:33.444 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_active_connections": 1, 2024-04-21T08:51:33.444 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_total_time": 0.000367187, 2024-04-21T08:51:33.444 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_send_time": 0.000000000, 2024-04-21T08:51:33.444 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_recv_time": 0.000311373, 2024-04-21T08:51:33.444 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_running_fast_dispatch_time": 0.000000000, 2024-04-21T08:51:33.444 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:33.444 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:33.444 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:33.444 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:33.445 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:33.445 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:33.445 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:33.445 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:33.445 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:33.445 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:33.445 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-21T08:51:33.445 INFO:teuthology.orchestra.run.smithi086.stdout: "msgr_send_encrypted_bytes": 0 2024-04-21T08:51:33.445 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:33.445 INFO:teuthology.orchestra.run.smithi086.stdout: "cct": { 2024-04-21T08:51:33.445 INFO:teuthology.orchestra.run.smithi086.stdout: "total_workers": 0, 2024-04-21T08:51:33.445 INFO:teuthology.orchestra.run.smithi086.stdout: "unhealthy_workers": 0 2024-04-21T08:51:33.445 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:33.445 INFO:teuthology.orchestra.run.smithi086.stdout: "mempool": { 2024-04-21T08:51:33.445 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_bytes": 0, 2024-04-21T08:51:33.446 INFO:teuthology.orchestra.run.smithi086.stdout: "bloom_filter_items": 0, 2024-04-21T08:51:33.446 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T08:51:33.446 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_alloc_items": 0, 2024-04-21T08:51:33.446 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T08:51:33.446 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_data_items": 0, 2024-04-21T08:51:33.446 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T08:51:33.446 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T08:51:33.446 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T08:51:33.446 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T08:51:33.446 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T08:51:33.446 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_other_items": 0, 2024-04-21T08:51:33.446 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T08:51:33.446 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T08:51:33.446 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_bytes": 0, 2024-04-21T08:51:33.447 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_extent_items": 0, 2024-04-21T08:51:33.447 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_bytes": 0, 2024-04-21T08:51:33.447 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_blob_items": 0, 2024-04-21T08:51:33.447 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T08:51:33.447 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T08:51:33.447 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T08:51:33.447 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T08:51:33.447 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T08:51:33.447 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_fsck_items": 0, 2024-04-21T08:51:33.447 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_bytes": 0, 2024-04-21T08:51:33.447 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_txc_items": 0, 2024-04-21T08:51:33.447 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T08:51:33.447 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T08:51:33.447 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_bytes": 0, 2024-04-21T08:51:33.447 INFO:teuthology.orchestra.run.smithi086.stdout: "bluestore_writing_items": 0, 2024-04-21T08:51:33.448 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_bytes": 0, 2024-04-21T08:51:33.448 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_items": 0, 2024-04-21T08:51:33.448 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T08:51:33.448 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_reader_items": 0, 2024-04-21T08:51:33.448 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T08:51:33.448 INFO:teuthology.orchestra.run.smithi086.stdout: "bluefs_file_writer_items": 0, 2024-04-21T08:51:33.448 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_bytes": 148995, 2024-04-21T08:51:33.448 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_anon_items": 31, 2024-04-21T08:51:33.448 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_bytes": 0, 2024-04-21T08:51:33.448 INFO:teuthology.orchestra.run.smithi086.stdout: "buffer_meta_items": 0, 2024-04-21T08:51:33.448 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_bytes": 0, 2024-04-21T08:51:33.448 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_items": 0, 2024-04-21T08:51:33.448 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_bytes": 0, 2024-04-21T08:51:33.448 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_mapbl_items": 0, 2024-04-21T08:51:33.448 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_bytes": 0, 2024-04-21T08:51:33.448 INFO:teuthology.orchestra.run.smithi086.stdout: "osd_pglog_items": 0, 2024-04-21T08:51:33.449 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_bytes": 0, 2024-04-21T08:51:33.449 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_items": 0, 2024-04-21T08:51:33.449 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T08:51:33.449 INFO:teuthology.orchestra.run.smithi086.stdout: "osdmap_mapping_items": 0, 2024-04-21T08:51:33.449 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_bytes": 0, 2024-04-21T08:51:33.449 INFO:teuthology.orchestra.run.smithi086.stdout: "pgmap_items": 0, 2024-04-21T08:51:33.449 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_bytes": 928, 2024-04-21T08:51:33.449 INFO:teuthology.orchestra.run.smithi086.stdout: "mds_co_items": 2, 2024-04-21T08:51:33.449 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_bytes": 0, 2024-04-21T08:51:33.449 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_1_items": 0, 2024-04-21T08:51:33.449 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_bytes": 0, 2024-04-21T08:51:33.449 INFO:teuthology.orchestra.run.smithi086.stdout: "unittest_2_items": 0 2024-04-21T08:51:33.449 INFO:teuthology.orchestra.run.smithi086.stdout: }, 2024-04-21T08:51:33.449 INFO:teuthology.orchestra.run.smithi086.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T08:51:33.449 INFO:teuthology.orchestra.run.smithi086.stdout: "val": 0, 2024-04-21T08:51:33.449 INFO:teuthology.orchestra.run.smithi086.stdout: "max": 104857600, 2024-04-21T08:51:33.450 INFO:teuthology.orchestra.run.smithi086.stdout: "get_started": 0, 2024-04-21T08:51:33.450 INFO:teuthology.orchestra.run.smithi086.stdout: "get": 266, 2024-04-21T08:51:33.450 INFO:teuthology.orchestra.run.smithi086.stdout: "get_sum": 1564564, 2024-04-21T08:51:33.450 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:33.450 INFO:teuthology.orchestra.run.smithi086.stdout: "get_or_fail_success": 266, 2024-04-21T08:51:33.450 INFO:teuthology.orchestra.run.smithi086.stdout: "take": 0, 2024-04-21T08:51:33.450 INFO:teuthology.orchestra.run.smithi086.stdout: "take_sum": 0, 2024-04-21T08:51:33.450 INFO:teuthology.orchestra.run.smithi086.stdout: "put": 266, 2024-04-21T08:51:33.450 INFO:teuthology.orchestra.run.smithi086.stdout: "put_sum": 1564564, 2024-04-21T08:51:33.450 INFO:teuthology.orchestra.run.smithi086.stdout: "wait": { 2024-04-21T08:51:33.450 INFO:teuthology.orchestra.run.smithi086.stdout: "avgcount": 0, 2024-04-21T08:51:33.450 INFO:teuthology.orchestra.run.smithi086.stdout: "sum": 0.000000000, 2024-04-21T08:51:33.450 INFO:teuthology.orchestra.run.smithi086.stdout: "avgtime": 0.000000000 2024-04-21T08:51:33.450 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:33.451 INFO:teuthology.orchestra.run.smithi086.stdout: } 2024-04-21T08:51:33.451 INFO:teuthology.orchestra.run.smithi086.stdout:} 2024-04-21T08:51:34.199 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:33 smithi086 ceph-mon[27939]: pgmap v400: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-21T08:51:34.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:33 smithi088 ceph-mon[31259]: pgmap v400: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-21T08:51:34.227 WARNING:tasks.check_counter:Counter 'mds.exported' not found on daemon mds.j 2024-04-21T08:51:34.227 WARNING:tasks.check_counter:Counter 'mds.imported' not found on daemon mds.j 2024-04-21T08:51:34.227 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update' not found on daemon mds.j 2024-04-21T08:51:34.227 WARNING:tasks.check_counter:Counter 'mds_cache.dir_update_receipt' not found on daemon mds.j 2024-04-21T08:51:34.227 WARNING:tasks.check_counter:Counter 'mds.root_rsnaps' not found on daemon mds.j 2024-04-21T08:51:34.227 WARNING:tasks.check_counter:Counter 'mds_server.req_mksnap_latency.avgcount' not found on daemon mds.j 2024-04-21T08:51:34.228 WARNING:tasks.check_counter:Counter 'mds_server.req_rmsnap_latency.avgcount' not found on daemon mds.j 2024-04-21T08:51:34.228 DEBUG:tasks.check_counter:Getting stats from b 2024-04-21T08:51:34.228 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph daemon mds.b perf dump 2024-04-21T08:51:34.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:33 smithi179 ceph-mon[31829]: pgmap v400: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 7.3 KiB/s rd, 1.2 KiB/s wr, 12 op/s 2024-04-21T08:51:34.476 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T08:51:36.037 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:35 smithi088 ceph-mon[31259]: pgmap v401: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-21T08:51:36.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:35 smithi086 ceph-mon[27939]: pgmap v401: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-21T08:51:36.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:35 smithi179 ceph-mon[31829]: pgmap v401: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-21T08:51:36.308 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-21T08:51:36.308 INFO:teuthology.orchestra.run.smithi088.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T08:51:36.308 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_messages": 2913, 2024-04-21T08:51:36.308 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages": 2938, 2024-04-21T08:51:36.308 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_bytes": 855601, 2024-04-21T08:51:36.308 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_bytes": 14294859, 2024-04-21T08:51:36.308 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_created_connections": 20, 2024-04-21T08:51:36.308 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_active_connections": 8, 2024-04-21T08:51:36.309 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_total_time": 0.280788111, 2024-04-21T08:51:36.309 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_send_time": 0.098864755, 2024-04-21T08:51:36.309 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_recv_time": 0.064190792, 2024-04-21T08:51:36.309 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_fast_dispatch_time": 0.078111054, 2024-04-21T08:51:36.309 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:36.309 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 2926, 2024-04-21T08:51:36.309 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 1.199250417, 2024-04-21T08:51:36.309 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000409860 2024-04-21T08:51:36.309 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.309 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:36.309 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 270, 2024-04-21T08:51:36.309 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000044363, 2024-04-21T08:51:36.309 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000164 2024-04-21T08:51:36.310 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.310 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_encrypted_bytes": 89408, 2024-04-21T08:51:36.310 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_encrypted_bytes": 397904 2024-04-21T08:51:36.310 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.310 INFO:teuthology.orchestra.run.smithi088.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T08:51:36.310 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_messages": 6156, 2024-04-21T08:51:36.310 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages": 8013, 2024-04-21T08:51:36.310 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_bytes": 2102768, 2024-04-21T08:51:36.310 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_bytes": 10222938, 2024-04-21T08:51:36.310 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_created_connections": 15, 2024-04-21T08:51:36.310 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_active_connections": 6, 2024-04-21T08:51:36.310 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_total_time": 0.735470997, 2024-04-21T08:51:36.310 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_send_time": 0.272916735, 2024-04-21T08:51:36.310 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_recv_time": 0.309862702, 2024-04-21T08:51:36.311 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_fast_dispatch_time": 0.159003568, 2024-04-21T08:51:36.311 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:36.311 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 8017, 2024-04-21T08:51:36.311 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.490667134, 2024-04-21T08:51:36.311 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000061203 2024-04-21T08:51:36.311 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.311 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:36.311 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 10509, 2024-04-21T08:51:36.311 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.002454196, 2024-04-21T08:51:36.311 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000233 2024-04-21T08:51:36.311 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.311 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_encrypted_bytes": 1230912, 2024-04-21T08:51:36.311 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_encrypted_bytes": 1765264 2024-04-21T08:51:36.312 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.312 INFO:teuthology.orchestra.run.smithi088.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T08:51:36.312 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_messages": 3745, 2024-04-21T08:51:36.312 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages": 4508, 2024-04-21T08:51:36.312 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_bytes": 2597285, 2024-04-21T08:51:36.312 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_bytes": 9612593, 2024-04-21T08:51:36.312 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_created_connections": 317, 2024-04-21T08:51:36.312 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_active_connections": 6, 2024-04-21T08:51:36.312 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_total_time": 0.476042909, 2024-04-21T08:51:36.312 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_send_time": 0.151066825, 2024-04-21T08:51:36.312 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_recv_time": 0.254550527, 2024-04-21T08:51:36.312 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_fast_dispatch_time": 0.071593008, 2024-04-21T08:51:36.312 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:36.313 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 4511, 2024-04-21T08:51:36.313 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.426737646, 2024-04-21T08:51:36.313 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000094599 2024-04-21T08:51:36.313 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.313 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:36.313 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 5213, 2024-04-21T08:51:36.313 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.001412305, 2024-04-21T08:51:36.313 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000270 2024-04-21T08:51:36.313 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.313 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_encrypted_bytes": 1741536, 2024-04-21T08:51:36.314 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_encrypted_bytes": 193648 2024-04-21T08:51:36.314 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.314 INFO:teuthology.orchestra.run.smithi088.stdout: "cct": { 2024-04-21T08:51:36.314 INFO:teuthology.orchestra.run.smithi088.stdout: "total_workers": 1, 2024-04-21T08:51:36.314 INFO:teuthology.orchestra.run.smithi088.stdout: "unhealthy_workers": 0 2024-04-21T08:51:36.314 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.314 INFO:teuthology.orchestra.run.smithi088.stdout: "finisher-MDSRank": { 2024-04-21T08:51:36.314 INFO:teuthology.orchestra.run.smithi088.stdout: "queue_len": 0, 2024-04-21T08:51:36.314 INFO:teuthology.orchestra.run.smithi088.stdout: "complete_latency": { 2024-04-21T08:51:36.314 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 6313, 2024-04-21T08:51:36.314 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 2.053005249, 2024-04-21T08:51:36.314 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000325202 2024-04-21T08:51:36.314 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-21T08:51:36.314 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.315 INFO:teuthology.orchestra.run.smithi088.stdout: "finisher-PurgeQueue": { 2024-04-21T08:51:36.315 INFO:teuthology.orchestra.run.smithi088.stdout: "queue_len": 0, 2024-04-21T08:51:36.315 INFO:teuthology.orchestra.run.smithi088.stdout: "complete_latency": { 2024-04-21T08:51:36.315 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 16, 2024-04-21T08:51:36.315 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000290807, 2024-04-21T08:51:36.315 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000018175 2024-04-21T08:51:36.315 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-21T08:51:36.315 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.315 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-04-21T08:51:36.315 INFO:teuthology.orchestra.run.smithi088.stdout: "request": 814, 2024-04-21T08:51:36.315 INFO:teuthology.orchestra.run.smithi088.stdout: "reply": 814, 2024-04-21T08:51:36.315 INFO:teuthology.orchestra.run.smithi088.stdout: "reply_latency": { 2024-04-21T08:51:36.315 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 814, 2024-04-21T08:51:36.315 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 3.553787441, 2024-04-21T08:51:36.316 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.004365832 2024-04-21T08:51:36.316 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.316 INFO:teuthology.orchestra.run.smithi088.stdout: "slow_reply": 0, 2024-04-21T08:51:36.316 INFO:teuthology.orchestra.run.smithi088.stdout: "forward": 0, 2024-04-21T08:51:36.316 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_fetch_complete": 0, 2024-04-21T08:51:36.316 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_fetch_keys": 0, 2024-04-21T08:51:36.316 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_commit": 374, 2024-04-21T08:51:36.316 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_split": 0, 2024-04-21T08:51:36.316 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_merge": 0, 2024-04-21T08:51:36.316 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes": 45, 2024-04-21T08:51:36.316 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_top": 14, 2024-04-21T08:51:36.316 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_bottom": 31, 2024-04-21T08:51:36.316 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_pin_tail": 0, 2024-04-21T08:51:36.316 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_pinned": 24, 2024-04-21T08:51:36.316 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_expired": 0, 2024-04-21T08:51:36.317 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_with_caps": 2, 2024-04-21T08:51:36.317 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": 3, 2024-04-21T08:51:36.317 INFO:teuthology.orchestra.run.smithi088.stdout: "subtrees": 2, 2024-04-21T08:51:36.317 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse": 899, 2024-04-21T08:51:36.317 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_hit": 891, 2024-04-21T08:51:36.317 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_forward": 0, 2024-04-21T08:51:36.317 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_discover": 0, 2024-04-21T08:51:36.317 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_dir_fetch": 0, 2024-04-21T08:51:36.317 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_remote_ino": 0, 2024-04-21T08:51:36.317 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_lock": 0, 2024-04-21T08:51:36.317 INFO:teuthology.orchestra.run.smithi088.stdout: "load_cent": 39, 2024-04-21T08:51:36.317 INFO:teuthology.orchestra.run.smithi088.stdout: "q": 0, 2024-04-21T08:51:36.317 INFO:teuthology.orchestra.run.smithi088.stdout: "exported": 0, 2024-04-21T08:51:36.317 INFO:teuthology.orchestra.run.smithi088.stdout: "exported_inodes": 0, 2024-04-21T08:51:36.317 INFO:teuthology.orchestra.run.smithi088.stdout: "imported": 0, 2024-04-21T08:51:36.318 INFO:teuthology.orchestra.run.smithi088.stdout: "imported_inodes": 0, 2024-04-21T08:51:36.318 INFO:teuthology.orchestra.run.smithi088.stdout: "openino_dir_fetch": 0, 2024-04-21T08:51:36.318 INFO:teuthology.orchestra.run.smithi088.stdout: "openino_backtrace_fetch": 317, 2024-04-21T08:51:36.318 INFO:teuthology.orchestra.run.smithi088.stdout: "openino_peer_discover": 0, 2024-04-21T08:51:36.318 INFO:teuthology.orchestra.run.smithi088.stdout: "root_rfiles": 4, 2024-04-21T08:51:36.318 INFO:teuthology.orchestra.run.smithi088.stdout: "root_rbytes": 234, 2024-04-21T08:51:36.318 INFO:teuthology.orchestra.run.smithi088.stdout: "root_rsnaps": 6, 2024-04-21T08:51:36.318 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_backtrace_fetch": 317, 2024-04-21T08:51:36.318 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_set_tag": 0, 2024-04-21T08:51:36.318 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T08:51:36.318 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_inotable_repaired": 0, 2024-04-21T08:51:36.318 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_dir_inodes": 204, 2024-04-21T08:51:36.318 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_dir_base_inodes": 23, 2024-04-21T08:51:36.318 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_dirfrag_rstats": 181, 2024-04-21T08:51:36.319 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_file_inodes": 113, 2024-04-21T08:51:36.319 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_inode_file_caps": 0, 2024-04-21T08:51:36.319 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_revoke": 629, 2024-04-21T08:51:36.319 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_grant": 623, 2024-04-21T08:51:36.319 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_trunc": 599, 2024-04-21T08:51:36.319 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T08:51:36.319 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T08:51:36.319 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_caps": 698, 2024-04-21T08:51:36.319 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_caps_dirty": 666, 2024-04-21T08:51:36.319 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_cap_release": 2, 2024-04-21T08:51:36.319 INFO:teuthology.orchestra.run.smithi088.stdout: "process_request_cap_release": 624 2024-04-21T08:51:36.319 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.319 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_cache": { 2024-04-21T08:51:36.319 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_update": 21, 2024-04-21T08:51:36.320 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_update_receipt": 0, 2024-04-21T08:51:36.320 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_try_discover": 0, 2024-04-21T08:51:36.320 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_send_discover": 0, 2024-04-21T08:51:36.320 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_handle_discover": 47, 2024-04-21T08:51:36.320 INFO:teuthology.orchestra.run.smithi088.stdout: "num_strays": 2, 2024-04-21T08:51:36.320 INFO:teuthology.orchestra.run.smithi088.stdout: "num_strays_delayed": 0, 2024-04-21T08:51:36.320 INFO:teuthology.orchestra.run.smithi088.stdout: "num_strays_enqueuing": 0, 2024-04-21T08:51:36.320 INFO:teuthology.orchestra.run.smithi088.stdout: "strays_created": 3, 2024-04-21T08:51:36.320 INFO:teuthology.orchestra.run.smithi088.stdout: "strays_enqueued": 1, 2024-04-21T08:51:36.320 INFO:teuthology.orchestra.run.smithi088.stdout: "strays_reintegrated": 0, 2024-04-21T08:51:36.320 INFO:teuthology.orchestra.run.smithi088.stdout: "strays_migrated": 0, 2024-04-21T08:51:36.321 INFO:teuthology.orchestra.run.smithi088.stdout: "num_recovering_processing": 0, 2024-04-21T08:51:36.321 INFO:teuthology.orchestra.run.smithi088.stdout: "num_recovering_enqueued": 0, 2024-04-21T08:51:36.321 INFO:teuthology.orchestra.run.smithi088.stdout: "num_recovering_prioritized": 0, 2024-04-21T08:51:36.321 INFO:teuthology.orchestra.run.smithi088.stdout: "recovery_started": 0, 2024-04-21T08:51:36.321 INFO:teuthology.orchestra.run.smithi088.stdout: "recovery_completed": 0, 2024-04-21T08:51:36.321 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_quiesce_path": 3, 2024-04-21T08:51:36.321 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_quiesce_inode": 42, 2024-04-21T08:51:36.321 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_enqueue_scrub": 23, 2024-04-21T08:51:36.321 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_exportdir": 0, 2024-04-21T08:51:36.321 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_flush": 0, 2024-04-21T08:51:36.321 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_fragmentdir": 0, 2024-04-21T08:51:36.321 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_fragstats": 0, 2024-04-21T08:51:36.321 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_inodestats": 0 2024-04-21T08:51:36.322 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.322 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_log": { 2024-04-21T08:51:36.322 INFO:teuthology.orchestra.run.smithi088.stdout: "evlrg": 0, 2024-04-21T08:51:36.322 INFO:teuthology.orchestra.run.smithi088.stdout: "evadd": 2564, 2024-04-21T08:51:36.322 INFO:teuthology.orchestra.run.smithi088.stdout: "evex": 2499, 2024-04-21T08:51:36.322 INFO:teuthology.orchestra.run.smithi088.stdout: "evtrm": 2499, 2024-04-21T08:51:36.322 INFO:teuthology.orchestra.run.smithi088.stdout: "ev": 65, 2024-04-21T08:51:36.322 INFO:teuthology.orchestra.run.smithi088.stdout: "evexg": 0, 2024-04-21T08:51:36.323 INFO:teuthology.orchestra.run.smithi088.stdout: "evexd": 51, 2024-04-21T08:51:36.323 INFO:teuthology.orchestra.run.smithi088.stdout: "segadd": 62, 2024-04-21T08:51:36.323 INFO:teuthology.orchestra.run.smithi088.stdout: "segex": 61, 2024-04-21T08:51:36.323 INFO:teuthology.orchestra.run.smithi088.stdout: "segtrm": 61, 2024-04-21T08:51:36.323 INFO:teuthology.orchestra.run.smithi088.stdout: "seg": 1, 2024-04-21T08:51:36.323 INFO:teuthology.orchestra.run.smithi088.stdout: "segmjr": 1, 2024-04-21T08:51:36.323 INFO:teuthology.orchestra.run.smithi088.stdout: "segexg": 0, 2024-04-21T08:51:36.323 INFO:teuthology.orchestra.run.smithi088.stdout: "segexd": 1, 2024-04-21T08:51:36.323 INFO:teuthology.orchestra.run.smithi088.stdout: "expos": 27209890, 2024-04-21T08:51:36.323 INFO:teuthology.orchestra.run.smithi088.stdout: "wrpos": 27829357, 2024-04-21T08:51:36.323 INFO:teuthology.orchestra.run.smithi088.stdout: "rdpos": 0, 2024-04-21T08:51:36.323 INFO:teuthology.orchestra.run.smithi088.stdout: "jlat": { 2024-04-21T08:51:36.323 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 2483, 2024-04-21T08:51:36.324 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 5.368970768, 2024-04-21T08:51:36.324 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.002162291 2024-04-21T08:51:36.324 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.324 INFO:teuthology.orchestra.run.smithi088.stdout: "replayed": 0 2024-04-21T08:51:36.324 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.324 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_mem": { 2024-04-21T08:51:36.324 INFO:teuthology.orchestra.run.smithi088.stdout: "ino": 38, 2024-04-21T08:51:36.324 INFO:teuthology.orchestra.run.smithi088.stdout: "ino+": 61, 2024-04-21T08:51:36.324 INFO:teuthology.orchestra.run.smithi088.stdout: "ino-": 23, 2024-04-21T08:51:36.324 INFO:teuthology.orchestra.run.smithi088.stdout: "dir": 20, 2024-04-21T08:51:36.324 INFO:teuthology.orchestra.run.smithi088.stdout: "dir+": 20, 2024-04-21T08:51:36.324 INFO:teuthology.orchestra.run.smithi088.stdout: "dir-": 0, 2024-04-21T08:51:36.324 INFO:teuthology.orchestra.run.smithi088.stdout: "dn": 45, 2024-04-21T08:51:36.324 INFO:teuthology.orchestra.run.smithi088.stdout: "dn+": 45, 2024-04-21T08:51:36.324 INFO:teuthology.orchestra.run.smithi088.stdout: "dn-": 0, 2024-04-21T08:51:36.325 INFO:teuthology.orchestra.run.smithi088.stdout: "cap": 3, 2024-04-21T08:51:36.325 INFO:teuthology.orchestra.run.smithi088.stdout: "cap+": 18, 2024-04-21T08:51:36.325 INFO:teuthology.orchestra.run.smithi088.stdout: "cap-": 15, 2024-04-21T08:51:36.325 INFO:teuthology.orchestra.run.smithi088.stdout: "rss": 61664, 2024-04-21T08:51:36.325 INFO:teuthology.orchestra.run.smithi088.stdout: "heap": 231800 2024-04-21T08:51:36.325 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.325 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_server": { 2024-04-21T08:51:36.325 INFO:teuthology.orchestra.run.smithi088.stdout: "dispatch_client_request": 2072, 2024-04-21T08:51:36.325 INFO:teuthology.orchestra.run.smithi088.stdout: "dispatch_server_request": 58, 2024-04-21T08:51:36.325 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_request": 814, 2024-04-21T08:51:36.325 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_session": 38, 2024-04-21T08:51:36.325 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_peer_request": 116, 2024-04-21T08:51:36.325 INFO:teuthology.orchestra.run.smithi088.stdout: "req_create_latency": { 2024-04-21T08:51:36.325 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 8, 2024-04-21T08:51:36.326 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.005728589, 2024-04-21T08:51:36.326 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000716073 2024-04-21T08:51:36.326 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.326 INFO:teuthology.orchestra.run.smithi088.stdout: "req_getattr_latency": { 2024-04-21T08:51:36.326 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 145, 2024-04-21T08:51:36.326 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.028337277, 2024-04-21T08:51:36.326 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000195429 2024-04-21T08:51:36.326 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.326 INFO:teuthology.orchestra.run.smithi088.stdout: "req_getfilelock_latency": { 2024-04-21T08:51:36.326 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:36.326 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:36.326 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:36.326 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.327 INFO:teuthology.orchestra.run.smithi088.stdout: "req_link_latency": { 2024-04-21T08:51:36.327 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:36.327 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:36.327 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:36.327 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.327 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookup_latency": { 2024-04-21T08:51:36.327 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 2, 2024-04-21T08:51:36.327 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.001379738, 2024-04-21T08:51:36.327 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000689869 2024-04-21T08:51:36.328 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.328 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookuphash_latency": { 2024-04-21T08:51:36.328 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:36.328 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:36.328 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:36.328 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.328 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookupino_latency": { 2024-04-21T08:51:36.328 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 1, 2024-04-21T08:51:36.328 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000160822, 2024-04-21T08:51:36.328 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000160822 2024-04-21T08:51:36.329 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.329 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookupname_latency": { 2024-04-21T08:51:36.329 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:36.329 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:36.329 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:36.329 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.329 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookupparent_latency": { 2024-04-21T08:51:36.329 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:36.329 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:36.329 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:36.329 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.330 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookupsnap_latency": { 2024-04-21T08:51:36.330 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:36.330 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:36.330 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:36.330 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.330 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lssnap_latency": { 2024-04-21T08:51:36.330 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 13, 2024-04-21T08:51:36.330 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.002202733, 2024-04-21T08:51:36.331 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000169441 2024-04-21T08:51:36.331 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.331 INFO:teuthology.orchestra.run.smithi088.stdout: "req_mkdir_latency": { 2024-04-21T08:51:36.331 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 8, 2024-04-21T08:51:36.331 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.011828738, 2024-04-21T08:51:36.331 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.001478592 2024-04-21T08:51:36.331 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.331 INFO:teuthology.orchestra.run.smithi088.stdout: "req_mknod_latency": { 2024-04-21T08:51:36.332 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:36.332 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:36.332 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:36.332 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.332 INFO:teuthology.orchestra.run.smithi088.stdout: "req_mksnap_latency": { 2024-04-21T08:51:36.332 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 13, 2024-04-21T08:51:36.332 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.066946173, 2024-04-21T08:51:36.332 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.005149705 2024-04-21T08:51:36.332 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.332 INFO:teuthology.orchestra.run.smithi088.stdout: "req_open_latency": { 2024-04-21T08:51:36.332 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 1, 2024-04-21T08:51:36.332 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000248915, 2024-04-21T08:51:36.333 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000248915 2024-04-21T08:51:36.333 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.333 INFO:teuthology.orchestra.run.smithi088.stdout: "req_readdir_latency": { 2024-04-21T08:51:36.333 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 2, 2024-04-21T08:51:36.333 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.004454815, 2024-04-21T08:51:36.333 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.002227407 2024-04-21T08:51:36.333 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.333 INFO:teuthology.orchestra.run.smithi088.stdout: "req_rename_latency": { 2024-04-21T08:51:36.333 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 2, 2024-04-21T08:51:36.333 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.002105599, 2024-04-21T08:51:36.333 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.001052799 2024-04-21T08:51:36.333 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.333 INFO:teuthology.orchestra.run.smithi088.stdout: "req_renamesnap_latency": { 2024-04-21T08:51:36.333 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:36.334 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:36.334 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:36.334 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.334 INFO:teuthology.orchestra.run.smithi088.stdout: "req_snapdiff_latency": { 2024-04-21T08:51:36.334 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:36.334 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:36.334 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:36.334 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.334 INFO:teuthology.orchestra.run.smithi088.stdout: "req_rmdir_latency": { 2024-04-21T08:51:36.334 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 2, 2024-04-21T08:51:36.334 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.011616645, 2024-04-21T08:51:36.334 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.005808322 2024-04-21T08:51:36.334 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.335 INFO:teuthology.orchestra.run.smithi088.stdout: "req_rmsnap_latency": { 2024-04-21T08:51:36.335 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 7, 2024-04-21T08:51:36.335 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.031769084, 2024-04-21T08:51:36.335 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.004538440 2024-04-21T08:51:36.335 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.335 INFO:teuthology.orchestra.run.smithi088.stdout: "req_rmxattr_latency": { 2024-04-21T08:51:36.335 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:36.335 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:36.335 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:36.335 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.335 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setattr_latency": { 2024-04-21T08:51:36.335 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 601, 2024-04-21T08:51:36.335 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 3.363039383, 2024-04-21T08:51:36.336 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.005595739 2024-04-21T08:51:36.336 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.336 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setdirlayout_latency": { 2024-04-21T08:51:36.336 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:36.336 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:36.336 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:36.336 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.336 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setfilelock_latency": { 2024-04-21T08:51:36.336 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:36.336 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:36.336 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:36.336 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.336 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setlayout_latency": { 2024-04-21T08:51:36.336 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:36.337 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:36.337 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:36.337 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.337 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setxattr_latency": { 2024-04-21T08:51:36.337 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 8, 2024-04-21T08:51:36.337 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.021011621, 2024-04-21T08:51:36.337 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.002626452 2024-04-21T08:51:36.337 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.337 INFO:teuthology.orchestra.run.smithi088.stdout: "req_symlink_latency": { 2024-04-21T08:51:36.337 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:36.337 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:36.337 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:36.337 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.337 INFO:teuthology.orchestra.run.smithi088.stdout: "req_unlink_latency": { 2024-04-21T08:51:36.337 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 1, 2024-04-21T08:51:36.338 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.002957309, 2024-04-21T08:51:36.338 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.002957309 2024-04-21T08:51:36.338 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.338 INFO:teuthology.orchestra.run.smithi088.stdout: "cap_revoke_eviction": 0, 2024-04-21T08:51:36.338 INFO:teuthology.orchestra.run.smithi088.stdout: "cap_acquisition_throttle": 0, 2024-04-21T08:51:36.338 INFO:teuthology.orchestra.run.smithi088.stdout: "req_getvxattr_latency": { 2024-04-21T08:51:36.338 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:36.338 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:36.338 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:36.338 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-21T08:51:36.338 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.338 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_sessions": { 2024-04-21T08:51:36.338 INFO:teuthology.orchestra.run.smithi088.stdout: "session_count": 2, 2024-04-21T08:51:36.339 INFO:teuthology.orchestra.run.smithi088.stdout: "session_add": 6, 2024-04-21T08:51:36.339 INFO:teuthology.orchestra.run.smithi088.stdout: "session_remove": 4, 2024-04-21T08:51:36.339 INFO:teuthology.orchestra.run.smithi088.stdout: "sessions_open": 2, 2024-04-21T08:51:36.339 INFO:teuthology.orchestra.run.smithi088.stdout: "sessions_stale": 0, 2024-04-21T08:51:36.339 INFO:teuthology.orchestra.run.smithi088.stdout: "total_load": 111, 2024-04-21T08:51:36.339 INFO:teuthology.orchestra.run.smithi088.stdout: "average_load": 55, 2024-04-21T08:51:36.339 INFO:teuthology.orchestra.run.smithi088.stdout: "avg_session_uptime": 762, 2024-04-21T08:51:36.339 INFO:teuthology.orchestra.run.smithi088.stdout: "mdthresh_evicted": 0 2024-04-21T08:51:36.339 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.339 INFO:teuthology.orchestra.run.smithi088.stdout: "mempool": { 2024-04-21T08:51:36.339 INFO:teuthology.orchestra.run.smithi088.stdout: "bloom_filter_bytes": 0, 2024-04-21T08:51:36.339 INFO:teuthology.orchestra.run.smithi088.stdout: "bloom_filter_items": 0, 2024-04-21T08:51:36.339 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T08:51:36.339 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_alloc_items": 0, 2024-04-21T08:51:36.339 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T08:51:36.340 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_data_items": 0, 2024-04-21T08:51:36.340 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T08:51:36.340 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T08:51:36.340 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T08:51:36.340 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T08:51:36.340 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T08:51:36.340 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_other_items": 0, 2024-04-21T08:51:36.340 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T08:51:36.340 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T08:51:36.340 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_extent_bytes": 0, 2024-04-21T08:51:36.340 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_extent_items": 0, 2024-04-21T08:51:36.340 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_blob_bytes": 0, 2024-04-21T08:51:36.340 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_blob_items": 0, 2024-04-21T08:51:36.340 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T08:51:36.341 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T08:51:36.341 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T08:51:36.341 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T08:51:36.341 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T08:51:36.341 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_fsck_items": 0, 2024-04-21T08:51:36.341 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_txc_bytes": 0, 2024-04-21T08:51:36.341 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_txc_items": 0, 2024-04-21T08:51:36.341 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T08:51:36.341 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T08:51:36.341 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_writing_bytes": 0, 2024-04-21T08:51:36.341 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_writing_items": 0, 2024-04-21T08:51:36.341 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_bytes": 0, 2024-04-21T08:51:36.341 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_items": 0, 2024-04-21T08:51:36.341 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T08:51:36.341 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_file_reader_items": 0, 2024-04-21T08:51:36.342 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T08:51:36.342 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_file_writer_items": 0, 2024-04-21T08:51:36.342 INFO:teuthology.orchestra.run.smithi088.stdout: "buffer_anon_bytes": 350966, 2024-04-21T08:51:36.342 INFO:teuthology.orchestra.run.smithi088.stdout: "buffer_anon_items": 119, 2024-04-21T08:51:36.342 INFO:teuthology.orchestra.run.smithi088.stdout: "buffer_meta_bytes": 0, 2024-04-21T08:51:36.342 INFO:teuthology.orchestra.run.smithi088.stdout: "buffer_meta_items": 0, 2024-04-21T08:51:36.342 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_bytes": 0, 2024-04-21T08:51:36.342 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_items": 0, 2024-04-21T08:51:36.342 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_mapbl_bytes": 0, 2024-04-21T08:51:36.342 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_mapbl_items": 0, 2024-04-21T08:51:36.342 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_pglog_bytes": 0, 2024-04-21T08:51:36.342 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_pglog_items": 0, 2024-04-21T08:51:36.342 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_bytes": 9512, 2024-04-21T08:51:36.343 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_items": 182, 2024-04-21T08:51:36.343 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T08:51:36.343 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_mapping_items": 0, 2024-04-21T08:51:36.343 INFO:teuthology.orchestra.run.smithi088.stdout: "pgmap_bytes": 0, 2024-04-21T08:51:36.343 INFO:teuthology.orchestra.run.smithi088.stdout: "pgmap_items": 0, 2024-04-21T08:51:36.343 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_co_bytes": 190589, 2024-04-21T08:51:36.343 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_co_items": 1621, 2024-04-21T08:51:36.343 INFO:teuthology.orchestra.run.smithi088.stdout: "unittest_1_bytes": 0, 2024-04-21T08:51:36.343 INFO:teuthology.orchestra.run.smithi088.stdout: "unittest_1_items": 0, 2024-04-21T08:51:36.343 INFO:teuthology.orchestra.run.smithi088.stdout: "unittest_2_bytes": 0, 2024-04-21T08:51:36.343 INFO:teuthology.orchestra.run.smithi088.stdout: "unittest_2_items": 0 2024-04-21T08:51:36.343 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.343 INFO:teuthology.orchestra.run.smithi088.stdout: "objecter": { 2024-04-21T08:51:36.343 INFO:teuthology.orchestra.run.smithi088.stdout: "op_active": 0, 2024-04-21T08:51:36.344 INFO:teuthology.orchestra.run.smithi088.stdout: "op_laggy": 0, 2024-04-21T08:51:36.344 INFO:teuthology.orchestra.run.smithi088.stdout: "op_send": 4698, 2024-04-21T08:51:36.344 INFO:teuthology.orchestra.run.smithi088.stdout: "op_send_bytes": 24911361, 2024-04-21T08:51:36.344 INFO:teuthology.orchestra.run.smithi088.stdout: "op_resend": 37, 2024-04-21T08:51:36.344 INFO:teuthology.orchestra.run.smithi088.stdout: "op_reply": 4698, 2024-04-21T08:51:36.344 INFO:teuthology.orchestra.run.smithi088.stdout: "op_latency": { 2024-04-21T08:51:36.344 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 4698, 2024-04-21T08:51:36.344 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 28.137413633, 2024-04-21T08:51:36.344 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.005989232 2024-04-21T08:51:36.344 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.344 INFO:teuthology.orchestra.run.smithi088.stdout: "op_inflight": 0, 2024-04-21T08:51:36.344 INFO:teuthology.orchestra.run.smithi088.stdout: "oplen_avg": { 2024-04-21T08:51:36.344 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 4698, 2024-04-21T08:51:36.344 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 5487 2024-04-21T08:51:36.344 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.345 INFO:teuthology.orchestra.run.smithi088.stdout: "op": 4698, 2024-04-21T08:51:36.345 INFO:teuthology.orchestra.run.smithi088.stdout: "op_r": 364, 2024-04-21T08:51:36.345 INFO:teuthology.orchestra.run.smithi088.stdout: "op_w": 4334, 2024-04-21T08:51:36.345 INFO:teuthology.orchestra.run.smithi088.stdout: "op_rmw": 0, 2024-04-21T08:51:36.345 INFO:teuthology.orchestra.run.smithi088.stdout: "op_pg": 0, 2024-04-21T08:51:36.345 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_stat": 354, 2024-04-21T08:51:36.345 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_create": 30, 2024-04-21T08:51:36.345 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_read": 24, 2024-04-21T08:51:36.345 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_write": 2484, 2024-04-21T08:51:36.345 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_writefull": 196, 2024-04-21T08:51:36.345 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_writesame": 0, 2024-04-21T08:51:36.345 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_append": 0, 2024-04-21T08:51:36.345 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_zero": 0, 2024-04-21T08:51:36.345 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_truncate": 0, 2024-04-21T08:51:36.346 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_delete": 524, 2024-04-21T08:51:36.346 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_mapext": 0, 2024-04-21T08:51:36.346 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_sparse_read": 0, 2024-04-21T08:51:36.346 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_clonerange": 0, 2024-04-21T08:51:36.346 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_getxattr": 340, 2024-04-21T08:51:36.346 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_setxattr": 55, 2024-04-21T08:51:36.346 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_cmpxattr": 0, 2024-04-21T08:51:36.346 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_rmxattr": 0, 2024-04-21T08:51:36.346 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_resetxattrs": 0, 2024-04-21T08:51:36.346 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_call": 0, 2024-04-21T08:51:36.346 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_watch": 0, 2024-04-21T08:51:36.346 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_notify": 0, 2024-04-21T08:51:36.346 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T08:51:36.346 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_pgls": 0, 2024-04-21T08:51:36.347 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_pgls_filter": 0, 2024-04-21T08:51:36.347 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_other": 599, 2024-04-21T08:51:36.347 INFO:teuthology.orchestra.run.smithi088.stdout: "linger_active": 0, 2024-04-21T08:51:36.347 INFO:teuthology.orchestra.run.smithi088.stdout: "linger_send": 0, 2024-04-21T08:51:36.347 INFO:teuthology.orchestra.run.smithi088.stdout: "linger_resend": 0, 2024-04-21T08:51:36.347 INFO:teuthology.orchestra.run.smithi088.stdout: "linger_ping": 0, 2024-04-21T08:51:36.347 INFO:teuthology.orchestra.run.smithi088.stdout: "poolop_active": 0, 2024-04-21T08:51:36.347 INFO:teuthology.orchestra.run.smithi088.stdout: "poolop_send": 0, 2024-04-21T08:51:36.347 INFO:teuthology.orchestra.run.smithi088.stdout: "poolop_resend": 0, 2024-04-21T08:51:36.347 INFO:teuthology.orchestra.run.smithi088.stdout: "poolstat_active": 0, 2024-04-21T08:51:36.347 INFO:teuthology.orchestra.run.smithi088.stdout: "poolstat_send": 0, 2024-04-21T08:51:36.347 INFO:teuthology.orchestra.run.smithi088.stdout: "poolstat_resend": 0, 2024-04-21T08:51:36.347 INFO:teuthology.orchestra.run.smithi088.stdout: "statfs_active": 0, 2024-04-21T08:51:36.347 INFO:teuthology.orchestra.run.smithi088.stdout: "statfs_send": 0, 2024-04-21T08:51:36.347 INFO:teuthology.orchestra.run.smithi088.stdout: "statfs_resend": 0, 2024-04-21T08:51:36.348 INFO:teuthology.orchestra.run.smithi088.stdout: "command_active": 0, 2024-04-21T08:51:36.348 INFO:teuthology.orchestra.run.smithi088.stdout: "command_send": 0, 2024-04-21T08:51:36.348 INFO:teuthology.orchestra.run.smithi088.stdout: "command_resend": 0, 2024-04-21T08:51:36.348 INFO:teuthology.orchestra.run.smithi088.stdout: "map_epoch": 108, 2024-04-21T08:51:36.348 INFO:teuthology.orchestra.run.smithi088.stdout: "map_full": 0, 2024-04-21T08:51:36.348 INFO:teuthology.orchestra.run.smithi088.stdout: "map_inc": 24, 2024-04-21T08:51:36.348 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_sessions": 12, 2024-04-21T08:51:36.348 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_session_open": 12, 2024-04-21T08:51:36.348 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_session_close": 0, 2024-04-21T08:51:36.348 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_laggy": 0, 2024-04-21T08:51:36.348 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_wr": 871, 2024-04-21T08:51:36.348 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_rd": 0, 2024-04-21T08:51:36.348 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_del": 10 2024-04-21T08:51:36.348 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.349 INFO:teuthology.orchestra.run.smithi088.stdout: "oft": { 2024-04-21T08:51:36.349 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_total_objs": 1, 2024-04-21T08:51:36.349 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_total_kv_pairs": 0, 2024-04-21T08:51:36.349 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_total_updates": 0, 2024-04-21T08:51:36.349 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_total_removes": 0 2024-04-21T08:51:36.349 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.349 INFO:teuthology.orchestra.run.smithi088.stdout: "osd-slow-ops": { 2024-04-21T08:51:36.349 INFO:teuthology.orchestra.run.smithi088.stdout: "slow_ops_count": 102 2024-04-21T08:51:36.349 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.349 INFO:teuthology.orchestra.run.smithi088.stdout: "purge_queue": { 2024-04-21T08:51:36.349 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executing_ops": 0, 2024-04-21T08:51:36.349 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executing_ops_high_water": 2, 2024-04-21T08:51:36.349 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executing": 0, 2024-04-21T08:51:36.350 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executing_high_water": 1, 2024-04-21T08:51:36.350 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executed_ops": 2, 2024-04-21T08:51:36.350 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executed": 1, 2024-04-21T08:51:36.350 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_item_in_journal": 0 2024-04-21T08:51:36.350 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.350 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T08:51:36.350 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 0, 2024-04-21T08:51:36.350 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 104857600, 2024-04-21T08:51:36.350 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-21T08:51:36.350 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 12810, 2024-04-21T08:51:36.350 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 4229010, 2024-04-21T08:51:36.350 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:36.351 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 12810, 2024-04-21T08:51:36.351 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 0, 2024-04-21T08:51:36.351 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 0, 2024-04-21T08:51:36.351 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 12810, 2024-04-21T08:51:36.351 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 4229010, 2024-04-21T08:51:36.351 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-21T08:51:36.351 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:36.351 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:36.351 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:36.351 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-21T08:51:36.351 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.351 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-objecter_bytes": { 2024-04-21T08:51:36.351 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 0, 2024-04-21T08:51:36.351 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 104857600, 2024-04-21T08:51:36.352 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-21T08:51:36.352 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 0, 2024-04-21T08:51:36.352 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 0, 2024-04-21T08:51:36.352 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:36.352 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:36.352 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 4698, 2024-04-21T08:51:36.352 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 24911518, 2024-04-21T08:51:36.352 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 4698, 2024-04-21T08:51:36.352 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 24911518, 2024-04-21T08:51:36.352 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-21T08:51:36.352 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:36.352 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:36.353 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:36.353 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-21T08:51:36.353 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.353 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-objecter_ops": { 2024-04-21T08:51:36.353 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 0, 2024-04-21T08:51:36.353 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 1024, 2024-04-21T08:51:36.353 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-21T08:51:36.353 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 0, 2024-04-21T08:51:36.353 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 0, 2024-04-21T08:51:36.353 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:36.353 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:36.353 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 4698, 2024-04-21T08:51:36.354 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 4698, 2024-04-21T08:51:36.354 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 4698, 2024-04-21T08:51:36.354 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 4698, 2024-04-21T08:51:36.354 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-21T08:51:36.354 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:36.354 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:36.354 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:36.354 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-21T08:51:36.354 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.354 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-write_buf_throttle": { 2024-04-21T08:51:36.354 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 0, 2024-04-21T08:51:36.354 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 3758096384, 2024-04-21T08:51:36.354 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-21T08:51:36.355 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 1, 2024-04-21T08:51:36.355 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 157, 2024-04-21T08:51:36.355 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:36.355 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 1, 2024-04-21T08:51:36.355 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 0, 2024-04-21T08:51:36.355 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 0, 2024-04-21T08:51:36.355 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 1, 2024-04-21T08:51:36.355 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 157, 2024-04-21T08:51:36.355 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-21T08:51:36.355 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:36.355 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:36.355 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:36.355 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-21T08:51:36.355 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:36.356 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-write_buf_throttle-0x5654a5cfc500": { 2024-04-21T08:51:36.356 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 599, 2024-04-21T08:51:36.356 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 3758096384, 2024-04-21T08:51:36.356 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-21T08:51:36.356 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 2564, 2024-04-21T08:51:36.356 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 23635053, 2024-04-21T08:51:36.356 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:36.356 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 2564, 2024-04-21T08:51:36.356 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 0, 2024-04-21T08:51:36.356 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 0, 2024-04-21T08:51:36.356 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 2483, 2024-04-21T08:51:36.356 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 23634454, 2024-04-21T08:51:36.356 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-21T08:51:36.356 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:36.356 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:36.357 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:36.357 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-21T08:51:36.357 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-21T08:51:36.357 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-21T08:51:37.113 INFO:tasks.check_counter:Daemon mds.b mds.exported=0 2024-04-21T08:51:37.113 INFO:tasks.check_counter:Daemon mds.b mds.imported=0 2024-04-21T08:51:37.113 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update=21 2024-04-21T08:51:37.113 INFO:tasks.check_counter:Daemon mds.b mds_cache.dir_update_receipt=0 2024-04-21T08:51:37.113 INFO:tasks.check_counter:Daemon mds.b mds.root_rsnaps=6 2024-04-21T08:51:37.113 INFO:tasks.check_counter:Daemon mds.b mds_server.req_mksnap_latency.avgcount=13 2024-04-21T08:51:37.114 INFO:tasks.check_counter:Daemon mds.b mds_server.req_rmsnap_latency.avgcount=7 2024-04-21T08:51:37.114 DEBUG:tasks.check_counter:Getting stats from e 2024-04-21T08:51:37.114 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph daemon mds.e perf dump 2024-04-21T08:51:37.358 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T08:51:38.214 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:37 smithi088 ceph-mon[31259]: pgmap v402: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-21T08:51:38.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:37 smithi179 ceph-mon[31829]: pgmap v402: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-21T08:51:38.366 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:37 smithi086 ceph-mon[27939]: pgmap v402: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 5.2 KiB/s rd, 1.2 KiB/s wr, 8 op/s 2024-04-21T08:51:39.141 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-21T08:51:39.141 INFO:teuthology.orchestra.run.smithi088.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T08:51:39.141 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_messages": 901, 2024-04-21T08:51:39.141 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages": 903, 2024-04-21T08:51:39.141 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_bytes": 238805, 2024-04-21T08:51:39.141 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_bytes": 286582, 2024-04-21T08:51:39.141 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_created_connections": 6, 2024-04-21T08:51:39.141 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_active_connections": 2, 2024-04-21T08:51:39.141 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_total_time": 0.083331617, 2024-04-21T08:51:39.142 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_send_time": 0.035339976, 2024-04-21T08:51:39.142 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_recv_time": 0.018666814, 2024-04-21T08:51:39.142 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_fast_dispatch_time": 0.020340202, 2024-04-21T08:51:39.142 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:39.142 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 901, 2024-04-21T08:51:39.142 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.037802943, 2024-04-21T08:51:39.142 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000041956 2024-04-21T08:51:39.142 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.142 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:39.142 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.142 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.142 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.142 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.142 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-21T08:51:39.143 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_encrypted_bytes": 0 2024-04-21T08:51:39.143 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.143 INFO:teuthology.orchestra.run.smithi088.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T08:51:39.143 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_messages": 829, 2024-04-21T08:51:39.143 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages": 1020, 2024-04-21T08:51:39.143 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_bytes": 203784, 2024-04-21T08:51:39.143 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_bytes": 757631, 2024-04-21T08:51:39.143 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_created_connections": 7, 2024-04-21T08:51:39.143 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_active_connections": 5, 2024-04-21T08:51:39.143 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_total_time": 0.088276786, 2024-04-21T08:51:39.143 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_send_time": 0.032587427, 2024-04-21T08:51:39.143 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_recv_time": 0.018340320, 2024-04-21T08:51:39.143 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_fast_dispatch_time": 0.027573711, 2024-04-21T08:51:39.143 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:39.143 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 1017, 2024-04-21T08:51:39.143 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.052762188, 2024-04-21T08:51:39.144 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000051880 2024-04-21T08:51:39.144 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.144 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:39.144 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.144 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.144 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.144 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.144 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-21T08:51:39.144 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_encrypted_bytes": 494640 2024-04-21T08:51:39.144 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.144 INFO:teuthology.orchestra.run.smithi088.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T08:51:39.144 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_messages": 1191, 2024-04-21T08:51:39.144 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages": 1172, 2024-04-21T08:51:39.144 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_bytes": 1968736, 2024-04-21T08:51:39.144 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_bytes": 444518, 2024-04-21T08:51:39.145 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_created_connections": 3, 2024-04-21T08:51:39.145 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_active_connections": 2, 2024-04-21T08:51:39.145 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_total_time": 0.167811418, 2024-04-21T08:51:39.145 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_send_time": 0.052040762, 2024-04-21T08:51:39.145 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_recv_time": 0.069227249, 2024-04-21T08:51:39.145 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_fast_dispatch_time": 0.044332742, 2024-04-21T08:51:39.145 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:39.145 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 1172, 2024-04-21T08:51:39.145 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.066668224, 2024-04-21T08:51:39.145 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000056884 2024-04-21T08:51:39.145 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.145 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:39.145 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.145 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.145 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.146 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.146 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_encrypted_bytes": 1668688, 2024-04-21T08:51:39.146 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_encrypted_bytes": 160544 2024-04-21T08:51:39.146 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.146 INFO:teuthology.orchestra.run.smithi088.stdout: "cct": { 2024-04-21T08:51:39.146 INFO:teuthology.orchestra.run.smithi088.stdout: "total_workers": 1, 2024-04-21T08:51:39.146 INFO:teuthology.orchestra.run.smithi088.stdout: "unhealthy_workers": 0 2024-04-21T08:51:39.146 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.146 INFO:teuthology.orchestra.run.smithi088.stdout: "finisher-MDSRank": { 2024-04-21T08:51:39.146 INFO:teuthology.orchestra.run.smithi088.stdout: "queue_len": 0, 2024-04-21T08:51:39.146 INFO:teuthology.orchestra.run.smithi088.stdout: "complete_latency": { 2024-04-21T08:51:39.146 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 4309, 2024-04-21T08:51:39.146 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.133469610, 2024-04-21T08:51:39.146 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000030974 2024-04-21T08:51:39.147 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-21T08:51:39.147 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.147 INFO:teuthology.orchestra.run.smithi088.stdout: "finisher-PurgeQueue": { 2024-04-21T08:51:39.147 INFO:teuthology.orchestra.run.smithi088.stdout: "queue_len": 0, 2024-04-21T08:51:39.147 INFO:teuthology.orchestra.run.smithi088.stdout: "complete_latency": { 2024-04-21T08:51:39.147 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.147 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.147 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.147 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-21T08:51:39.147 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.147 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-04-21T08:51:39.147 INFO:teuthology.orchestra.run.smithi088.stdout: "request": 0, 2024-04-21T08:51:39.147 INFO:teuthology.orchestra.run.smithi088.stdout: "reply": 0, 2024-04-21T08:51:39.147 INFO:teuthology.orchestra.run.smithi088.stdout: "reply_latency": { 2024-04-21T08:51:39.147 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.148 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.148 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.148 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.148 INFO:teuthology.orchestra.run.smithi088.stdout: "slow_reply": 0, 2024-04-21T08:51:39.148 INFO:teuthology.orchestra.run.smithi088.stdout: "forward": 0, 2024-04-21T08:51:39.148 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_fetch_complete": 0, 2024-04-21T08:51:39.148 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_fetch_keys": 0, 2024-04-21T08:51:39.148 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_commit": 0, 2024-04-21T08:51:39.148 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_split": 0, 2024-04-21T08:51:39.148 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_merge": 0, 2024-04-21T08:51:39.148 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes": 0, 2024-04-21T08:51:39.148 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_top": 0, 2024-04-21T08:51:39.148 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_bottom": 0, 2024-04-21T08:51:39.148 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_pin_tail": 0, 2024-04-21T08:51:39.148 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_pinned": 0, 2024-04-21T08:51:39.149 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_expired": 0, 2024-04-21T08:51:39.149 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_with_caps": 0, 2024-04-21T08:51:39.149 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": 0, 2024-04-21T08:51:39.149 INFO:teuthology.orchestra.run.smithi088.stdout: "subtrees": 1, 2024-04-21T08:51:39.149 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse": 0, 2024-04-21T08:51:39.149 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_hit": 0, 2024-04-21T08:51:39.149 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_forward": 0, 2024-04-21T08:51:39.149 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_discover": 0, 2024-04-21T08:51:39.149 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_dir_fetch": 0, 2024-04-21T08:51:39.149 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_remote_ino": 0, 2024-04-21T08:51:39.149 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_lock": 0, 2024-04-21T08:51:39.149 INFO:teuthology.orchestra.run.smithi088.stdout: "load_cent": 0, 2024-04-21T08:51:39.149 INFO:teuthology.orchestra.run.smithi088.stdout: "q": 0, 2024-04-21T08:51:39.149 INFO:teuthology.orchestra.run.smithi088.stdout: "exported": 0, 2024-04-21T08:51:39.150 INFO:teuthology.orchestra.run.smithi088.stdout: "exported_inodes": 0, 2024-04-21T08:51:39.150 INFO:teuthology.orchestra.run.smithi088.stdout: "imported": 0, 2024-04-21T08:51:39.150 INFO:teuthology.orchestra.run.smithi088.stdout: "imported_inodes": 0, 2024-04-21T08:51:39.150 INFO:teuthology.orchestra.run.smithi088.stdout: "openino_dir_fetch": 0, 2024-04-21T08:51:39.150 INFO:teuthology.orchestra.run.smithi088.stdout: "openino_backtrace_fetch": 0, 2024-04-21T08:51:39.150 INFO:teuthology.orchestra.run.smithi088.stdout: "openino_peer_discover": 0, 2024-04-21T08:51:39.150 INFO:teuthology.orchestra.run.smithi088.stdout: "root_rfiles": 0, 2024-04-21T08:51:39.150 INFO:teuthology.orchestra.run.smithi088.stdout: "root_rbytes": 0, 2024-04-21T08:51:39.150 INFO:teuthology.orchestra.run.smithi088.stdout: "root_rsnaps": 0, 2024-04-21T08:51:39.150 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_backtrace_fetch": 0, 2024-04-21T08:51:39.150 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_set_tag": 0, 2024-04-21T08:51:39.150 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T08:51:39.150 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_inotable_repaired": 0, 2024-04-21T08:51:39.150 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_dir_inodes": 0, 2024-04-21T08:51:39.151 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_dir_base_inodes": 0, 2024-04-21T08:51:39.151 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_dirfrag_rstats": 0, 2024-04-21T08:51:39.151 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_file_inodes": 0, 2024-04-21T08:51:39.151 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_inode_file_caps": 0, 2024-04-21T08:51:39.151 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_revoke": 0, 2024-04-21T08:51:39.151 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_grant": 0, 2024-04-21T08:51:39.151 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_trunc": 0, 2024-04-21T08:51:39.151 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T08:51:39.151 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T08:51:39.151 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_caps": 0, 2024-04-21T08:51:39.151 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_caps_dirty": 0, 2024-04-21T08:51:39.151 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_cap_release": 0, 2024-04-21T08:51:39.151 INFO:teuthology.orchestra.run.smithi088.stdout: "process_request_cap_release": 0 2024-04-21T08:51:39.151 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.151 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_cache": { 2024-04-21T08:51:39.151 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_update": 0, 2024-04-21T08:51:39.152 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_update_receipt": 0, 2024-04-21T08:51:39.152 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_try_discover": 0, 2024-04-21T08:51:39.152 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_send_discover": 0, 2024-04-21T08:51:39.152 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_handle_discover": 0, 2024-04-21T08:51:39.152 INFO:teuthology.orchestra.run.smithi088.stdout: "num_strays": 0, 2024-04-21T08:51:39.152 INFO:teuthology.orchestra.run.smithi088.stdout: "num_strays_delayed": 0, 2024-04-21T08:51:39.152 INFO:teuthology.orchestra.run.smithi088.stdout: "num_strays_enqueuing": 0, 2024-04-21T08:51:39.152 INFO:teuthology.orchestra.run.smithi088.stdout: "strays_created": 0, 2024-04-21T08:51:39.152 INFO:teuthology.orchestra.run.smithi088.stdout: "strays_enqueued": 0, 2024-04-21T08:51:39.152 INFO:teuthology.orchestra.run.smithi088.stdout: "strays_reintegrated": 0, 2024-04-21T08:51:39.152 INFO:teuthology.orchestra.run.smithi088.stdout: "strays_migrated": 0, 2024-04-21T08:51:39.152 INFO:teuthology.orchestra.run.smithi088.stdout: "num_recovering_processing": 0, 2024-04-21T08:51:39.152 INFO:teuthology.orchestra.run.smithi088.stdout: "num_recovering_enqueued": 0, 2024-04-21T08:51:39.152 INFO:teuthology.orchestra.run.smithi088.stdout: "num_recovering_prioritized": 0, 2024-04-21T08:51:39.153 INFO:teuthology.orchestra.run.smithi088.stdout: "recovery_started": 0, 2024-04-21T08:51:39.153 INFO:teuthology.orchestra.run.smithi088.stdout: "recovery_completed": 0, 2024-04-21T08:51:39.153 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_quiesce_path": 0, 2024-04-21T08:51:39.153 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_quiesce_inode": 0, 2024-04-21T08:51:39.153 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_enqueue_scrub": 0, 2024-04-21T08:51:39.153 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_exportdir": 0, 2024-04-21T08:51:39.153 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_flush": 0, 2024-04-21T08:51:39.153 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_fragmentdir": 0, 2024-04-21T08:51:39.153 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_fragstats": 0, 2024-04-21T08:51:39.153 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_inodestats": 0 2024-04-21T08:51:39.153 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.153 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_log": { 2024-04-21T08:51:39.153 INFO:teuthology.orchestra.run.smithi088.stdout: "evlrg": 0, 2024-04-21T08:51:39.153 INFO:teuthology.orchestra.run.smithi088.stdout: "evadd": 0, 2024-04-21T08:51:39.153 INFO:teuthology.orchestra.run.smithi088.stdout: "evex": 0, 2024-04-21T08:51:39.154 INFO:teuthology.orchestra.run.smithi088.stdout: "evtrm": 0, 2024-04-21T08:51:39.154 INFO:teuthology.orchestra.run.smithi088.stdout: "ev": 8, 2024-04-21T08:51:39.154 INFO:teuthology.orchestra.run.smithi088.stdout: "evexg": 0, 2024-04-21T08:51:39.154 INFO:teuthology.orchestra.run.smithi088.stdout: "evexd": 0, 2024-04-21T08:51:39.154 INFO:teuthology.orchestra.run.smithi088.stdout: "segadd": 0, 2024-04-21T08:51:39.154 INFO:teuthology.orchestra.run.smithi088.stdout: "segex": 0, 2024-04-21T08:51:39.154 INFO:teuthology.orchestra.run.smithi088.stdout: "segtrm": 0, 2024-04-21T08:51:39.154 INFO:teuthology.orchestra.run.smithi088.stdout: "seg": 2, 2024-04-21T08:51:39.154 INFO:teuthology.orchestra.run.smithi088.stdout: "segmjr": 2, 2024-04-21T08:51:39.154 INFO:teuthology.orchestra.run.smithi088.stdout: "segexg": 0, 2024-04-21T08:51:39.154 INFO:teuthology.orchestra.run.smithi088.stdout: "segexd": 0, 2024-04-21T08:51:39.154 INFO:teuthology.orchestra.run.smithi088.stdout: "expos": 4194304, 2024-04-21T08:51:39.154 INFO:teuthology.orchestra.run.smithi088.stdout: "wrpos": 4196488, 2024-04-21T08:51:39.154 INFO:teuthology.orchestra.run.smithi088.stdout: "rdpos": 4196347, 2024-04-21T08:51:39.155 INFO:teuthology.orchestra.run.smithi088.stdout: "jlat": { 2024-04-21T08:51:39.155 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.155 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.155 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.155 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.155 INFO:teuthology.orchestra.run.smithi088.stdout: "replayed": 8 2024-04-21T08:51:39.155 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.155 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_mem": { 2024-04-21T08:51:39.155 INFO:teuthology.orchestra.run.smithi088.stdout: "ino": 3, 2024-04-21T08:51:39.155 INFO:teuthology.orchestra.run.smithi088.stdout: "ino+": 3, 2024-04-21T08:51:39.155 INFO:teuthology.orchestra.run.smithi088.stdout: "ino-": 0, 2024-04-21T08:51:39.155 INFO:teuthology.orchestra.run.smithi088.stdout: "dir": 1, 2024-04-21T08:51:39.155 INFO:teuthology.orchestra.run.smithi088.stdout: "dir+": 1, 2024-04-21T08:51:39.155 INFO:teuthology.orchestra.run.smithi088.stdout: "dir-": 0, 2024-04-21T08:51:39.155 INFO:teuthology.orchestra.run.smithi088.stdout: "dn": 0, 2024-04-21T08:51:39.155 INFO:teuthology.orchestra.run.smithi088.stdout: "dn+": 0, 2024-04-21T08:51:39.156 INFO:teuthology.orchestra.run.smithi088.stdout: "dn-": 0, 2024-04-21T08:51:39.156 INFO:teuthology.orchestra.run.smithi088.stdout: "cap": 0, 2024-04-21T08:51:39.156 INFO:teuthology.orchestra.run.smithi088.stdout: "cap+": 0, 2024-04-21T08:51:39.156 INFO:teuthology.orchestra.run.smithi088.stdout: "cap-": 0, 2024-04-21T08:51:39.156 INFO:teuthology.orchestra.run.smithi088.stdout: "rss": 50888, 2024-04-21T08:51:39.156 INFO:teuthology.orchestra.run.smithi088.stdout: "heap": 215416 2024-04-21T08:51:39.156 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.156 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_server": { 2024-04-21T08:51:39.156 INFO:teuthology.orchestra.run.smithi088.stdout: "dispatch_client_request": 0, 2024-04-21T08:51:39.156 INFO:teuthology.orchestra.run.smithi088.stdout: "dispatch_server_request": 0, 2024-04-21T08:51:39.156 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_request": 0, 2024-04-21T08:51:39.156 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_session": 0, 2024-04-21T08:51:39.156 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_peer_request": 0, 2024-04-21T08:51:39.156 INFO:teuthology.orchestra.run.smithi088.stdout: "req_create_latency": { 2024-04-21T08:51:39.156 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.157 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.157 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.157 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.157 INFO:teuthology.orchestra.run.smithi088.stdout: "req_getattr_latency": { 2024-04-21T08:51:39.157 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.157 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.157 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.157 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.157 INFO:teuthology.orchestra.run.smithi088.stdout: "req_getfilelock_latency": { 2024-04-21T08:51:39.157 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.157 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.157 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.157 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.157 INFO:teuthology.orchestra.run.smithi088.stdout: "req_link_latency": { 2024-04-21T08:51:39.157 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.158 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.158 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.158 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.158 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookup_latency": { 2024-04-21T08:51:39.158 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.158 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.158 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.158 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.158 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookuphash_latency": { 2024-04-21T08:51:39.158 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.158 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.158 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.158 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.158 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookupino_latency": { 2024-04-21T08:51:39.158 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.159 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.159 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.159 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.159 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookupname_latency": { 2024-04-21T08:51:39.159 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.159 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.159 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.159 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.159 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookupparent_latency": { 2024-04-21T08:51:39.159 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.159 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.159 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.159 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.159 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookupsnap_latency": { 2024-04-21T08:51:39.159 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.160 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.160 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.160 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.160 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lssnap_latency": { 2024-04-21T08:51:39.160 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.160 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.160 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.160 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.160 INFO:teuthology.orchestra.run.smithi088.stdout: "req_mkdir_latency": { 2024-04-21T08:51:39.160 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.161 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.161 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.161 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.161 INFO:teuthology.orchestra.run.smithi088.stdout: "req_mknod_latency": { 2024-04-21T08:51:39.161 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.161 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.161 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.161 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.161 INFO:teuthology.orchestra.run.smithi088.stdout: "req_mksnap_latency": { 2024-04-21T08:51:39.161 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.161 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.162 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.162 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.162 INFO:teuthology.orchestra.run.smithi088.stdout: "req_open_latency": { 2024-04-21T08:51:39.162 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.162 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.162 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.162 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.162 INFO:teuthology.orchestra.run.smithi088.stdout: "req_readdir_latency": { 2024-04-21T08:51:39.162 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.162 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.162 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.162 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.162 INFO:teuthology.orchestra.run.smithi088.stdout: "req_rename_latency": { 2024-04-21T08:51:39.162 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.163 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.163 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.163 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.163 INFO:teuthology.orchestra.run.smithi088.stdout: "req_renamesnap_latency": { 2024-04-21T08:51:39.163 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.163 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.163 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.163 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.163 INFO:teuthology.orchestra.run.smithi088.stdout: "req_snapdiff_latency": { 2024-04-21T08:51:39.163 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.163 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.163 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.163 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.163 INFO:teuthology.orchestra.run.smithi088.stdout: "req_rmdir_latency": { 2024-04-21T08:51:39.163 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.164 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.164 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.164 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.164 INFO:teuthology.orchestra.run.smithi088.stdout: "req_rmsnap_latency": { 2024-04-21T08:51:39.164 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.164 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.164 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.164 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.164 INFO:teuthology.orchestra.run.smithi088.stdout: "req_rmxattr_latency": { 2024-04-21T08:51:39.164 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.164 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.164 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.164 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.164 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setattr_latency": { 2024-04-21T08:51:39.164 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.165 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.165 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.165 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.165 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setdirlayout_latency": { 2024-04-21T08:51:39.165 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.165 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.165 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.165 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.165 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setfilelock_latency": { 2024-04-21T08:51:39.165 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.165 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.165 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.165 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.165 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setlayout_latency": { 2024-04-21T08:51:39.165 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.166 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.166 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.166 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.166 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setxattr_latency": { 2024-04-21T08:51:39.166 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.166 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.166 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.166 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.166 INFO:teuthology.orchestra.run.smithi088.stdout: "req_symlink_latency": { 2024-04-21T08:51:39.166 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.166 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.166 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.166 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.166 INFO:teuthology.orchestra.run.smithi088.stdout: "req_unlink_latency": { 2024-04-21T08:51:39.167 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.167 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.167 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.167 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.167 INFO:teuthology.orchestra.run.smithi088.stdout: "cap_revoke_eviction": 0, 2024-04-21T08:51:39.167 INFO:teuthology.orchestra.run.smithi088.stdout: "cap_acquisition_throttle": 0, 2024-04-21T08:51:39.167 INFO:teuthology.orchestra.run.smithi088.stdout: "req_getvxattr_latency": { 2024-04-21T08:51:39.167 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.167 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.167 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.167 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-21T08:51:39.167 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.167 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_sessions": { 2024-04-21T08:51:39.167 INFO:teuthology.orchestra.run.smithi088.stdout: "session_count": 0, 2024-04-21T08:51:39.167 INFO:teuthology.orchestra.run.smithi088.stdout: "session_add": 3, 2024-04-21T08:51:39.168 INFO:teuthology.orchestra.run.smithi088.stdout: "session_remove": 3, 2024-04-21T08:51:39.168 INFO:teuthology.orchestra.run.smithi088.stdout: "sessions_open": 2, 2024-04-21T08:51:39.168 INFO:teuthology.orchestra.run.smithi088.stdout: "sessions_stale": 0, 2024-04-21T08:51:39.168 INFO:teuthology.orchestra.run.smithi088.stdout: "total_load": 0, 2024-04-21T08:51:39.168 INFO:teuthology.orchestra.run.smithi088.stdout: "average_load": 0, 2024-04-21T08:51:39.168 INFO:teuthology.orchestra.run.smithi088.stdout: "avg_session_uptime": 1208, 2024-04-21T08:51:39.168 INFO:teuthology.orchestra.run.smithi088.stdout: "mdthresh_evicted": 0 2024-04-21T08:51:39.168 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.168 INFO:teuthology.orchestra.run.smithi088.stdout: "mempool": { 2024-04-21T08:51:39.168 INFO:teuthology.orchestra.run.smithi088.stdout: "bloom_filter_bytes": 0, 2024-04-21T08:51:39.168 INFO:teuthology.orchestra.run.smithi088.stdout: "bloom_filter_items": 0, 2024-04-21T08:51:39.168 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T08:51:39.168 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_alloc_items": 0, 2024-04-21T08:51:39.168 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T08:51:39.168 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_data_items": 0, 2024-04-21T08:51:39.169 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T08:51:39.169 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T08:51:39.169 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T08:51:39.169 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T08:51:39.169 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T08:51:39.169 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_other_items": 0, 2024-04-21T08:51:39.169 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T08:51:39.169 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T08:51:39.169 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_extent_bytes": 0, 2024-04-21T08:51:39.169 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_extent_items": 0, 2024-04-21T08:51:39.169 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_blob_bytes": 0, 2024-04-21T08:51:39.169 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_blob_items": 0, 2024-04-21T08:51:39.169 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T08:51:39.169 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T08:51:39.169 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T08:51:39.170 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T08:51:39.170 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T08:51:39.170 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_fsck_items": 0, 2024-04-21T08:51:39.170 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_txc_bytes": 0, 2024-04-21T08:51:39.170 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_txc_items": 0, 2024-04-21T08:51:39.170 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T08:51:39.170 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T08:51:39.170 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_writing_bytes": 0, 2024-04-21T08:51:39.170 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_writing_items": 0, 2024-04-21T08:51:39.170 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_bytes": 0, 2024-04-21T08:51:39.170 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_items": 0, 2024-04-21T08:51:39.170 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T08:51:39.170 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_file_reader_items": 0, 2024-04-21T08:51:39.170 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T08:51:39.171 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_file_writer_items": 0, 2024-04-21T08:51:39.171 INFO:teuthology.orchestra.run.smithi088.stdout: "buffer_anon_bytes": 189433, 2024-04-21T08:51:39.171 INFO:teuthology.orchestra.run.smithi088.stdout: "buffer_anon_items": 52, 2024-04-21T08:51:39.171 INFO:teuthology.orchestra.run.smithi088.stdout: "buffer_meta_bytes": 0, 2024-04-21T08:51:39.171 INFO:teuthology.orchestra.run.smithi088.stdout: "buffer_meta_items": 0, 2024-04-21T08:51:39.171 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_bytes": 0, 2024-04-21T08:51:39.171 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_items": 0, 2024-04-21T08:51:39.171 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_mapbl_bytes": 0, 2024-04-21T08:51:39.171 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_mapbl_items": 0, 2024-04-21T08:51:39.171 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_pglog_bytes": 0, 2024-04-21T08:51:39.171 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_pglog_items": 0, 2024-04-21T08:51:39.171 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_bytes": 9512, 2024-04-21T08:51:39.171 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_items": 182, 2024-04-21T08:51:39.171 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T08:51:39.171 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_mapping_items": 0, 2024-04-21T08:51:39.172 INFO:teuthology.orchestra.run.smithi088.stdout: "pgmap_bytes": 0, 2024-04-21T08:51:39.172 INFO:teuthology.orchestra.run.smithi088.stdout: "pgmap_items": 0, 2024-04-21T08:51:39.172 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_co_bytes": 8512, 2024-04-21T08:51:39.172 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_co_items": 13, 2024-04-21T08:51:39.172 INFO:teuthology.orchestra.run.smithi088.stdout: "unittest_1_bytes": 0, 2024-04-21T08:51:39.172 INFO:teuthology.orchestra.run.smithi088.stdout: "unittest_1_items": 0, 2024-04-21T08:51:39.172 INFO:teuthology.orchestra.run.smithi088.stdout: "unittest_2_bytes": 0, 2024-04-21T08:51:39.172 INFO:teuthology.orchestra.run.smithi088.stdout: "unittest_2_items": 0 2024-04-21T08:51:39.172 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.172 INFO:teuthology.orchestra.run.smithi088.stdout: "objecter": { 2024-04-21T08:51:39.172 INFO:teuthology.orchestra.run.smithi088.stdout: "op_active": 0, 2024-04-21T08:51:39.172 INFO:teuthology.orchestra.run.smithi088.stdout: "op_laggy": 0, 2024-04-21T08:51:39.172 INFO:teuthology.orchestra.run.smithi088.stdout: "op_send": 2621, 2024-04-21T08:51:39.172 INFO:teuthology.orchestra.run.smithi088.stdout: "op_send_bytes": 21, 2024-04-21T08:51:39.172 INFO:teuthology.orchestra.run.smithi088.stdout: "op_resend": 3, 2024-04-21T08:51:39.173 INFO:teuthology.orchestra.run.smithi088.stdout: "op_reply": 2621, 2024-04-21T08:51:39.173 INFO:teuthology.orchestra.run.smithi088.stdout: "op_latency": { 2024-04-21T08:51:39.173 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 2621, 2024-04-21T08:51:39.173 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 1.283973234, 2024-04-21T08:51:39.173 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000489879 2024-04-21T08:51:39.173 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.173 INFO:teuthology.orchestra.run.smithi088.stdout: "op_inflight": 0, 2024-04-21T08:51:39.173 INFO:teuthology.orchestra.run.smithi088.stdout: "oplen_avg": { 2024-04-21T08:51:39.173 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 2621, 2024-04-21T08:51:39.173 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 2622 2024-04-21T08:51:39.173 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.173 INFO:teuthology.orchestra.run.smithi088.stdout: "op": 2621, 2024-04-21T08:51:39.173 INFO:teuthology.orchestra.run.smithi088.stdout: "op_r": 2621, 2024-04-21T08:51:39.173 INFO:teuthology.orchestra.run.smithi088.stdout: "op_w": 0, 2024-04-21T08:51:39.173 INFO:teuthology.orchestra.run.smithi088.stdout: "op_rmw": 0, 2024-04-21T08:51:39.173 INFO:teuthology.orchestra.run.smithi088.stdout: "op_pg": 0, 2024-04-21T08:51:39.174 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_stat": 1718, 2024-04-21T08:51:39.174 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_create": 0, 2024-04-21T08:51:39.174 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_read": 901, 2024-04-21T08:51:39.174 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_write": 0, 2024-04-21T08:51:39.174 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_writefull": 0, 2024-04-21T08:51:39.174 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_writesame": 0, 2024-04-21T08:51:39.174 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_append": 0, 2024-04-21T08:51:39.174 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_zero": 0, 2024-04-21T08:51:39.174 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_truncate": 0, 2024-04-21T08:51:39.174 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_delete": 0, 2024-04-21T08:51:39.174 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_mapext": 0, 2024-04-21T08:51:39.174 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_sparse_read": 0, 2024-04-21T08:51:39.175 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_clonerange": 0, 2024-04-21T08:51:39.175 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_getxattr": 1, 2024-04-21T08:51:39.175 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_setxattr": 0, 2024-04-21T08:51:39.175 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_cmpxattr": 0, 2024-04-21T08:51:39.175 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_rmxattr": 0, 2024-04-21T08:51:39.175 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_resetxattrs": 0, 2024-04-21T08:51:39.175 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_call": 0, 2024-04-21T08:51:39.175 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_watch": 0, 2024-04-21T08:51:39.175 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_notify": 0, 2024-04-21T08:51:39.175 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T08:51:39.175 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_pgls": 0, 2024-04-21T08:51:39.175 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_pgls_filter": 0, 2024-04-21T08:51:39.175 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_other": 0, 2024-04-21T08:51:39.175 INFO:teuthology.orchestra.run.smithi088.stdout: "linger_active": 0, 2024-04-21T08:51:39.175 INFO:teuthology.orchestra.run.smithi088.stdout: "linger_send": 0, 2024-04-21T08:51:39.176 INFO:teuthology.orchestra.run.smithi088.stdout: "linger_resend": 0, 2024-04-21T08:51:39.176 INFO:teuthology.orchestra.run.smithi088.stdout: "linger_ping": 0, 2024-04-21T08:51:39.176 INFO:teuthology.orchestra.run.smithi088.stdout: "poolop_active": 0, 2024-04-21T08:51:39.176 INFO:teuthology.orchestra.run.smithi088.stdout: "poolop_send": 0, 2024-04-21T08:51:39.176 INFO:teuthology.orchestra.run.smithi088.stdout: "poolop_resend": 0, 2024-04-21T08:51:39.176 INFO:teuthology.orchestra.run.smithi088.stdout: "poolstat_active": 0, 2024-04-21T08:51:39.176 INFO:teuthology.orchestra.run.smithi088.stdout: "poolstat_send": 0, 2024-04-21T08:51:39.176 INFO:teuthology.orchestra.run.smithi088.stdout: "poolstat_resend": 0, 2024-04-21T08:51:39.176 INFO:teuthology.orchestra.run.smithi088.stdout: "statfs_active": 0, 2024-04-21T08:51:39.176 INFO:teuthology.orchestra.run.smithi088.stdout: "statfs_send": 0, 2024-04-21T08:51:39.176 INFO:teuthology.orchestra.run.smithi088.stdout: "statfs_resend": 0, 2024-04-21T08:51:39.176 INFO:teuthology.orchestra.run.smithi088.stdout: "command_active": 0, 2024-04-21T08:51:39.176 INFO:teuthology.orchestra.run.smithi088.stdout: "command_send": 0, 2024-04-21T08:51:39.176 INFO:teuthology.orchestra.run.smithi088.stdout: "command_resend": 0, 2024-04-21T08:51:39.176 INFO:teuthology.orchestra.run.smithi088.stdout: "map_epoch": 108, 2024-04-21T08:51:39.177 INFO:teuthology.orchestra.run.smithi088.stdout: "map_full": 0, 2024-04-21T08:51:39.177 INFO:teuthology.orchestra.run.smithi088.stdout: "map_inc": 19, 2024-04-21T08:51:39.177 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_sessions": 5, 2024-04-21T08:51:39.177 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_session_open": 5, 2024-04-21T08:51:39.177 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_session_close": 0, 2024-04-21T08:51:39.177 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_laggy": 0, 2024-04-21T08:51:39.177 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_wr": 0, 2024-04-21T08:51:39.177 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_rd": 2, 2024-04-21T08:51:39.177 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_del": 0 2024-04-21T08:51:39.177 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.177 INFO:teuthology.orchestra.run.smithi088.stdout: "oft": { 2024-04-21T08:51:39.177 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_total_objs": 0, 2024-04-21T08:51:39.177 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_total_kv_pairs": 0, 2024-04-21T08:51:39.177 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_total_updates": 0, 2024-04-21T08:51:39.177 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_total_removes": 0 2024-04-21T08:51:39.178 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.178 INFO:teuthology.orchestra.run.smithi088.stdout: "osd-slow-ops": { 2024-04-21T08:51:39.178 INFO:teuthology.orchestra.run.smithi088.stdout: "slow_ops_count": 0 2024-04-21T08:51:39.178 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.178 INFO:teuthology.orchestra.run.smithi088.stdout: "purge_queue": { 2024-04-21T08:51:39.178 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executing_ops": 0, 2024-04-21T08:51:39.178 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executing_ops_high_water": 0, 2024-04-21T08:51:39.178 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executing": 0, 2024-04-21T08:51:39.178 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executing_high_water": 0, 2024-04-21T08:51:39.178 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executed_ops": 0, 2024-04-21T08:51:39.179 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executed": 0, 2024-04-21T08:51:39.179 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_item_in_journal": 0 2024-04-21T08:51:39.179 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.179 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T08:51:39.179 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 0, 2024-04-21T08:51:39.179 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 104857600, 2024-04-21T08:51:39.179 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-21T08:51:39.179 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 2917, 2024-04-21T08:51:39.179 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 2130776, 2024-04-21T08:51:39.179 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:39.179 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 2917, 2024-04-21T08:51:39.179 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 0, 2024-04-21T08:51:39.179 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 0, 2024-04-21T08:51:39.179 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 2917, 2024-04-21T08:51:39.179 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 2130776, 2024-04-21T08:51:39.180 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-21T08:51:39.180 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.180 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.180 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.180 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-21T08:51:39.180 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.180 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-objecter_bytes": { 2024-04-21T08:51:39.180 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 0, 2024-04-21T08:51:39.180 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 104857600, 2024-04-21T08:51:39.180 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-21T08:51:39.180 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 0, 2024-04-21T08:51:39.180 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 0, 2024-04-21T08:51:39.180 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:39.180 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:39.180 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 2621, 2024-04-21T08:51:39.181 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 2189, 2024-04-21T08:51:39.181 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 2621, 2024-04-21T08:51:39.181 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 2189, 2024-04-21T08:51:39.181 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-21T08:51:39.181 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.181 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.181 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.181 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-21T08:51:39.181 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.181 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-objecter_ops": { 2024-04-21T08:51:39.181 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 0, 2024-04-21T08:51:39.181 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 1024, 2024-04-21T08:51:39.181 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-21T08:51:39.182 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 0, 2024-04-21T08:51:39.182 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 0, 2024-04-21T08:51:39.182 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:39.182 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:39.182 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 2621, 2024-04-21T08:51:39.182 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 2621, 2024-04-21T08:51:39.182 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 2621, 2024-04-21T08:51:39.182 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 2621, 2024-04-21T08:51:39.182 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-21T08:51:39.182 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.182 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.182 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.182 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-21T08:51:39.182 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.183 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-write_buf_throttle": { 2024-04-21T08:51:39.183 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 0, 2024-04-21T08:51:39.183 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 3758096384, 2024-04-21T08:51:39.183 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-21T08:51:39.183 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 0, 2024-04-21T08:51:39.183 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 0, 2024-04-21T08:51:39.183 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:39.183 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:39.183 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 0, 2024-04-21T08:51:39.183 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 0, 2024-04-21T08:51:39.183 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 0, 2024-04-21T08:51:39.183 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 0, 2024-04-21T08:51:39.183 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-21T08:51:39.183 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.183 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.184 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.184 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-21T08:51:39.184 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:39.184 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-write_buf_throttle-0x562dd397a6e0": { 2024-04-21T08:51:39.184 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 0, 2024-04-21T08:51:39.184 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 3758096384, 2024-04-21T08:51:39.184 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-21T08:51:39.184 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 0, 2024-04-21T08:51:39.184 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 0, 2024-04-21T08:51:39.184 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:39.184 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:39.184 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 0, 2024-04-21T08:51:39.184 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 0, 2024-04-21T08:51:39.184 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 0, 2024-04-21T08:51:39.184 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 0, 2024-04-21T08:51:39.185 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-21T08:51:39.185 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:39.185 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:39.185 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:39.185 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-21T08:51:39.185 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-21T08:51:39.185 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-21T08:51:39.966 INFO:tasks.check_counter:Daemon mds.e mds.exported=0 2024-04-21T08:51:39.966 INFO:tasks.check_counter:Daemon mds.e mds.imported=0 2024-04-21T08:51:39.966 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update=0 2024-04-21T08:51:39.966 INFO:tasks.check_counter:Daemon mds.e mds_cache.dir_update_receipt=0 2024-04-21T08:51:39.967 INFO:tasks.check_counter:Daemon mds.e mds.root_rsnaps=0 2024-04-21T08:51:39.967 INFO:tasks.check_counter:Daemon mds.e mds_server.req_mksnap_latency.avgcount=0 2024-04-21T08:51:39.967 INFO:tasks.check_counter:Daemon mds.e mds_server.req_rmsnap_latency.avgcount=0 2024-04-21T08:51:39.967 DEBUG:tasks.check_counter:Getting stats from h 2024-04-21T08:51:39.967 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph daemon mds.h perf dump 2024-04-21T08:51:40.203 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T08:51:40.213 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:39 smithi088 ceph-mon[31259]: pgmap v403: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-04-21T08:51:40.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:39 smithi179 ceph-mon[31829]: pgmap v403: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-04-21T08:51:40.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:39 smithi086 ceph-mon[27939]: pgmap v403: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 7.7 KiB/s rd, 1.8 KiB/s wr, 13 op/s 2024-04-21T08:51:41.970 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-21T08:51:41.970 INFO:teuthology.orchestra.run.smithi088.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T08:51:41.970 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_messages": 149, 2024-04-21T08:51:41.970 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages": 155, 2024-04-21T08:51:41.970 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_bytes": 40838, 2024-04-21T08:51:41.970 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_bytes": 651290, 2024-04-21T08:51:41.971 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_created_connections": 64, 2024-04-21T08:51:41.971 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_active_connections": 3, 2024-04-21T08:51:41.971 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_total_time": 0.031926178, 2024-04-21T08:51:41.971 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_send_time": 0.004665363, 2024-04-21T08:51:41.971 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_recv_time": 0.021664644, 2024-04-21T08:51:41.971 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_fast_dispatch_time": 0.000607944, 2024-04-21T08:51:41.971 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:41.971 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 153, 2024-04-21T08:51:41.971 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.023927100, 2024-04-21T08:51:41.971 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000156386 2024-04-21T08:51:41.971 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.971 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:41.971 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 186, 2024-04-21T08:51:41.972 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000036876, 2024-04-21T08:51:41.972 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000198 2024-04-21T08:51:41.972 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.972 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_encrypted_bytes": 3072, 2024-04-21T08:51:41.972 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_encrypted_bytes": 4240 2024-04-21T08:51:41.972 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.972 INFO:teuthology.orchestra.run.smithi088.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T08:51:41.972 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_messages": 85, 2024-04-21T08:51:41.972 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages": 124, 2024-04-21T08:51:41.972 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_bytes": 30397, 2024-04-21T08:51:41.972 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_bytes": 170720, 2024-04-21T08:51:41.972 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_created_connections": 11, 2024-04-21T08:51:41.972 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_active_connections": 4, 2024-04-21T08:51:41.973 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_total_time": 0.013738970, 2024-04-21T08:51:41.973 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_send_time": 0.005148314, 2024-04-21T08:51:41.973 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_recv_time": 0.007711101, 2024-04-21T08:51:41.973 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_fast_dispatch_time": 0.000260768, 2024-04-21T08:51:41.973 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:41.973 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 121, 2024-04-21T08:51:41.973 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.055940754, 2024-04-21T08:51:41.973 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000462320 2024-04-21T08:51:41.973 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.973 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:41.973 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 116, 2024-04-21T08:51:41.973 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000115971, 2024-04-21T08:51:41.973 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000999 2024-04-21T08:51:41.974 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.974 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_encrypted_bytes": 2336, 2024-04-21T08:51:41.974 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_encrypted_bytes": 134064 2024-04-21T08:51:41.974 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.974 INFO:teuthology.orchestra.run.smithi088.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T08:51:41.974 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_messages": 1849, 2024-04-21T08:51:41.974 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages": 1559, 2024-04-21T08:51:41.974 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_bytes": 2095997, 2024-04-21T08:51:41.974 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_bytes": 804186, 2024-04-21T08:51:41.974 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_created_connections": 8, 2024-04-21T08:51:41.974 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_active_connections": 5, 2024-04-21T08:51:41.974 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_total_time": 0.197508221, 2024-04-21T08:51:41.974 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_send_time": 0.087087791, 2024-04-21T08:51:41.975 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_recv_time": 0.122085834, 2024-04-21T08:51:41.975 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_fast_dispatch_time": 0.010977025, 2024-04-21T08:51:41.975 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:41.975 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 1559, 2024-04-21T08:51:41.975 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.085500856, 2024-04-21T08:51:41.975 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000054843 2024-04-21T08:51:41.975 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.975 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:41.975 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 2466, 2024-04-21T08:51:41.975 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000625079, 2024-04-21T08:51:41.975 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000253 2024-04-21T08:51:41.975 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.975 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_encrypted_bytes": 1675184, 2024-04-21T08:51:41.976 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_encrypted_bytes": 529424 2024-04-21T08:51:41.976 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.976 INFO:teuthology.orchestra.run.smithi088.stdout: "cct": { 2024-04-21T08:51:41.976 INFO:teuthology.orchestra.run.smithi088.stdout: "total_workers": 1, 2024-04-21T08:51:41.976 INFO:teuthology.orchestra.run.smithi088.stdout: "unhealthy_workers": 0 2024-04-21T08:51:41.976 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.976 INFO:teuthology.orchestra.run.smithi088.stdout: "finisher-MDSRank": { 2024-04-21T08:51:41.976 INFO:teuthology.orchestra.run.smithi088.stdout: "queue_len": 0, 2024-04-21T08:51:41.976 INFO:teuthology.orchestra.run.smithi088.stdout: "complete_latency": { 2024-04-21T08:51:41.976 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 50, 2024-04-21T08:51:41.976 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.002092119, 2024-04-21T08:51:41.976 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000041842 2024-04-21T08:51:41.976 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-21T08:51:41.976 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.977 INFO:teuthology.orchestra.run.smithi088.stdout: "finisher-PurgeQueue": { 2024-04-21T08:51:41.977 INFO:teuthology.orchestra.run.smithi088.stdout: "queue_len": 0, 2024-04-21T08:51:41.977 INFO:teuthology.orchestra.run.smithi088.stdout: "complete_latency": { 2024-04-21T08:51:41.977 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 2, 2024-04-21T08:51:41.977 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000006320, 2024-04-21T08:51:41.977 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000003160 2024-04-21T08:51:41.977 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-21T08:51:41.977 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.977 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-04-21T08:51:41.977 INFO:teuthology.orchestra.run.smithi088.stdout: "request": 8, 2024-04-21T08:51:41.977 INFO:teuthology.orchestra.run.smithi088.stdout: "reply": 6, 2024-04-21T08:51:41.977 INFO:teuthology.orchestra.run.smithi088.stdout: "reply_latency": { 2024-04-21T08:51:41.977 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 6, 2024-04-21T08:51:41.977 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.004245798, 2024-04-21T08:51:41.978 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000707633 2024-04-21T08:51:41.978 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.978 INFO:teuthology.orchestra.run.smithi088.stdout: "slow_reply": 0, 2024-04-21T08:51:41.978 INFO:teuthology.orchestra.run.smithi088.stdout: "forward": 2, 2024-04-21T08:51:41.978 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_fetch_complete": 0, 2024-04-21T08:51:41.978 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_fetch_keys": 0, 2024-04-21T08:51:41.978 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_commit": 11, 2024-04-21T08:51:41.978 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_split": 0, 2024-04-21T08:51:41.978 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_merge": 0, 2024-04-21T08:51:41.978 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes": 16, 2024-04-21T08:51:41.978 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_top": 0, 2024-04-21T08:51:41.978 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_bottom": 16, 2024-04-21T08:51:41.978 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_pin_tail": 0, 2024-04-21T08:51:41.979 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_pinned": 13, 2024-04-21T08:51:41.979 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_expired": 5, 2024-04-21T08:51:41.979 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_with_caps": 0, 2024-04-21T08:51:41.979 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": 0, 2024-04-21T08:51:41.979 INFO:teuthology.orchestra.run.smithi088.stdout: "subtrees": 3, 2024-04-21T08:51:41.979 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse": 35, 2024-04-21T08:51:41.979 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_hit": 18, 2024-04-21T08:51:41.979 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_forward": 2, 2024-04-21T08:51:41.979 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_discover": 10, 2024-04-21T08:51:41.979 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_dir_fetch": 0, 2024-04-21T08:51:41.979 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_remote_ino": 0, 2024-04-21T08:51:41.979 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_lock": 0, 2024-04-21T08:51:41.979 INFO:teuthology.orchestra.run.smithi088.stdout: "load_cent": 0, 2024-04-21T08:51:41.979 INFO:teuthology.orchestra.run.smithi088.stdout: "q": 0, 2024-04-21T08:51:41.979 INFO:teuthology.orchestra.run.smithi088.stdout: "exported": 0, 2024-04-21T08:51:41.980 INFO:teuthology.orchestra.run.smithi088.stdout: "exported_inodes": 0, 2024-04-21T08:51:41.980 INFO:teuthology.orchestra.run.smithi088.stdout: "imported": 0, 2024-04-21T08:51:41.980 INFO:teuthology.orchestra.run.smithi088.stdout: "imported_inodes": 0, 2024-04-21T08:51:41.980 INFO:teuthology.orchestra.run.smithi088.stdout: "openino_dir_fetch": 0, 2024-04-21T08:51:41.980 INFO:teuthology.orchestra.run.smithi088.stdout: "openino_backtrace_fetch": 0, 2024-04-21T08:51:41.980 INFO:teuthology.orchestra.run.smithi088.stdout: "openino_peer_discover": 0, 2024-04-21T08:51:41.980 INFO:teuthology.orchestra.run.smithi088.stdout: "root_rfiles": 5, 2024-04-21T08:51:41.980 INFO:teuthology.orchestra.run.smithi088.stdout: "root_rbytes": 298, 2024-04-21T08:51:41.980 INFO:teuthology.orchestra.run.smithi088.stdout: "root_rsnaps": 6, 2024-04-21T08:51:41.980 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_backtrace_fetch": 0, 2024-04-21T08:51:41.980 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_set_tag": 0, 2024-04-21T08:51:41.980 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T08:51:41.980 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_inotable_repaired": 0, 2024-04-21T08:51:41.980 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_dir_inodes": 0, 2024-04-21T08:51:41.980 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_dir_base_inodes": 0, 2024-04-21T08:51:41.981 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_dirfrag_rstats": 0, 2024-04-21T08:51:41.981 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_file_inodes": 0, 2024-04-21T08:51:41.981 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_inode_file_caps": 0, 2024-04-21T08:51:41.981 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_revoke": 0, 2024-04-21T08:51:41.981 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_grant": 0, 2024-04-21T08:51:41.981 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_trunc": 0, 2024-04-21T08:51:41.981 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T08:51:41.981 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T08:51:41.981 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_caps": 0, 2024-04-21T08:51:41.981 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_caps_dirty": 0, 2024-04-21T08:51:41.981 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_cap_release": 0, 2024-04-21T08:51:41.981 INFO:teuthology.orchestra.run.smithi088.stdout: "process_request_cap_release": 0 2024-04-21T08:51:41.981 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.981 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_cache": { 2024-04-21T08:51:41.982 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_update": 0, 2024-04-21T08:51:41.982 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_update_receipt": 19, 2024-04-21T08:51:41.982 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_try_discover": 14, 2024-04-21T08:51:41.982 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_send_discover": 13, 2024-04-21T08:51:41.982 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_handle_discover": 0, 2024-04-21T08:51:41.982 INFO:teuthology.orchestra.run.smithi088.stdout: "num_strays": 0, 2024-04-21T08:51:41.982 INFO:teuthology.orchestra.run.smithi088.stdout: "num_strays_delayed": 0, 2024-04-21T08:51:41.982 INFO:teuthology.orchestra.run.smithi088.stdout: "num_strays_enqueuing": 0, 2024-04-21T08:51:41.982 INFO:teuthology.orchestra.run.smithi088.stdout: "strays_created": 0, 2024-04-21T08:51:41.982 INFO:teuthology.orchestra.run.smithi088.stdout: "strays_enqueued": 0, 2024-04-21T08:51:41.982 INFO:teuthology.orchestra.run.smithi088.stdout: "strays_reintegrated": 0, 2024-04-21T08:51:41.982 INFO:teuthology.orchestra.run.smithi088.stdout: "strays_migrated": 0, 2024-04-21T08:51:41.982 INFO:teuthology.orchestra.run.smithi088.stdout: "num_recovering_processing": 0, 2024-04-21T08:51:41.983 INFO:teuthology.orchestra.run.smithi088.stdout: "num_recovering_enqueued": 0, 2024-04-21T08:51:41.983 INFO:teuthology.orchestra.run.smithi088.stdout: "num_recovering_prioritized": 0, 2024-04-21T08:51:41.983 INFO:teuthology.orchestra.run.smithi088.stdout: "recovery_started": 0, 2024-04-21T08:51:41.983 INFO:teuthology.orchestra.run.smithi088.stdout: "recovery_completed": 0, 2024-04-21T08:51:41.983 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_quiesce_path": 3, 2024-04-21T08:51:41.983 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_quiesce_inode": 15, 2024-04-21T08:51:41.983 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_enqueue_scrub": 0, 2024-04-21T08:51:41.983 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_exportdir": 0, 2024-04-21T08:51:41.983 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_flush": 0, 2024-04-21T08:51:41.983 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_fragmentdir": 0, 2024-04-21T08:51:41.983 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_fragstats": 0, 2024-04-21T08:51:41.983 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_inodestats": 0 2024-04-21T08:51:41.983 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.983 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_log": { 2024-04-21T08:51:41.984 INFO:teuthology.orchestra.run.smithi088.stdout: "evlrg": 0, 2024-04-21T08:51:41.984 INFO:teuthology.orchestra.run.smithi088.stdout: "evadd": 8, 2024-04-21T08:51:41.984 INFO:teuthology.orchestra.run.smithi088.stdout: "evex": 0, 2024-04-21T08:51:41.984 INFO:teuthology.orchestra.run.smithi088.stdout: "evtrm": 0, 2024-04-21T08:51:41.984 INFO:teuthology.orchestra.run.smithi088.stdout: "ev": 8, 2024-04-21T08:51:41.984 INFO:teuthology.orchestra.run.smithi088.stdout: "evexg": 0, 2024-04-21T08:51:41.984 INFO:teuthology.orchestra.run.smithi088.stdout: "evexd": 0, 2024-04-21T08:51:41.984 INFO:teuthology.orchestra.run.smithi088.stdout: "segadd": 2, 2024-04-21T08:51:41.984 INFO:teuthology.orchestra.run.smithi088.stdout: "segex": 0, 2024-04-21T08:51:41.984 INFO:teuthology.orchestra.run.smithi088.stdout: "segtrm": 0, 2024-04-21T08:51:41.984 INFO:teuthology.orchestra.run.smithi088.stdout: "seg": 2, 2024-04-21T08:51:41.984 INFO:teuthology.orchestra.run.smithi088.stdout: "segmjr": 2, 2024-04-21T08:51:41.984 INFO:teuthology.orchestra.run.smithi088.stdout: "segexg": 0, 2024-04-21T08:51:41.984 INFO:teuthology.orchestra.run.smithi088.stdout: "segexd": 0, 2024-04-21T08:51:41.985 INFO:teuthology.orchestra.run.smithi088.stdout: "expos": 4194304, 2024-04-21T08:51:41.985 INFO:teuthology.orchestra.run.smithi088.stdout: "wrpos": 4196488, 2024-04-21T08:51:41.985 INFO:teuthology.orchestra.run.smithi088.stdout: "rdpos": 0, 2024-04-21T08:51:41.985 INFO:teuthology.orchestra.run.smithi088.stdout: "jlat": { 2024-04-21T08:51:41.985 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 7, 2024-04-21T08:51:41.985 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.017715844, 2024-04-21T08:51:41.985 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.002530834 2024-04-21T08:51:41.985 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.985 INFO:teuthology.orchestra.run.smithi088.stdout: "replayed": 0 2024-04-21T08:51:41.985 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.985 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_mem": { 2024-04-21T08:51:41.985 INFO:teuthology.orchestra.run.smithi088.stdout: "ino": 20, 2024-04-21T08:51:41.985 INFO:teuthology.orchestra.run.smithi088.stdout: "ino+": 21, 2024-04-21T08:51:41.985 INFO:teuthology.orchestra.run.smithi088.stdout: "ino-": 1, 2024-04-21T08:51:41.986 INFO:teuthology.orchestra.run.smithi088.stdout: "dir": 19, 2024-04-21T08:51:41.986 INFO:teuthology.orchestra.run.smithi088.stdout: "dir+": 21, 2024-04-21T08:51:41.986 INFO:teuthology.orchestra.run.smithi088.stdout: "dir-": 2, 2024-04-21T08:51:41.986 INFO:teuthology.orchestra.run.smithi088.stdout: "dn": 16, 2024-04-21T08:51:41.986 INFO:teuthology.orchestra.run.smithi088.stdout: "dn+": 21, 2024-04-21T08:51:41.986 INFO:teuthology.orchestra.run.smithi088.stdout: "dn-": 5, 2024-04-21T08:51:41.986 INFO:teuthology.orchestra.run.smithi088.stdout: "cap": 0, 2024-04-21T08:51:41.986 INFO:teuthology.orchestra.run.smithi088.stdout: "cap+": 4, 2024-04-21T08:51:41.986 INFO:teuthology.orchestra.run.smithi088.stdout: "cap-": 4, 2024-04-21T08:51:41.986 INFO:teuthology.orchestra.run.smithi088.stdout: "rss": 52704, 2024-04-21T08:51:41.986 INFO:teuthology.orchestra.run.smithi088.stdout: "heap": 223608 2024-04-21T08:51:41.986 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.986 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_server": { 2024-04-21T08:51:41.987 INFO:teuthology.orchestra.run.smithi088.stdout: "dispatch_client_request": 13, 2024-04-21T08:51:41.987 INFO:teuthology.orchestra.run.smithi088.stdout: "dispatch_server_request": 0, 2024-04-21T08:51:41.987 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_request": 8, 2024-04-21T08:51:41.987 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_session": 12, 2024-04-21T08:51:41.987 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_peer_request": 16, 2024-04-21T08:51:41.987 INFO:teuthology.orchestra.run.smithi088.stdout: "req_create_latency": { 2024-04-21T08:51:41.987 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:41.987 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:41.987 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:41.987 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.987 INFO:teuthology.orchestra.run.smithi088.stdout: "req_getattr_latency": { 2024-04-21T08:51:41.988 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 2, 2024-04-21T08:51:41.988 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.003341218, 2024-04-21T08:51:41.988 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.001670609 2024-04-21T08:51:41.988 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.988 INFO:teuthology.orchestra.run.smithi088.stdout: "req_getfilelock_latency": { 2024-04-21T08:51:41.988 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:41.988 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:41.988 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:41.988 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.988 INFO:teuthology.orchestra.run.smithi088.stdout: "req_link_latency": { 2024-04-21T08:51:41.988 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:41.989 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:41.989 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:41.989 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.989 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookup_latency": { 2024-04-21T08:51:41.989 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 4, 2024-04-21T08:51:41.989 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000904580, 2024-04-21T08:51:41.989 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000226145 2024-04-21T08:51:41.989 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.989 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookuphash_latency": { 2024-04-21T08:51:41.989 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:41.989 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:41.989 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:41.989 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.990 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookupino_latency": { 2024-04-21T08:51:41.990 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:41.990 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:41.990 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:41.990 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.990 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookupname_latency": { 2024-04-21T08:51:41.990 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:41.990 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:41.990 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:41.990 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.990 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookupparent_latency": { 2024-04-21T08:51:41.990 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:41.990 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:41.990 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:41.991 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.991 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookupsnap_latency": { 2024-04-21T08:51:41.991 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:41.991 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:41.991 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:41.991 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.991 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lssnap_latency": { 2024-04-21T08:51:41.991 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:41.991 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:41.991 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:41.991 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.991 INFO:teuthology.orchestra.run.smithi088.stdout: "req_mkdir_latency": { 2024-04-21T08:51:41.992 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:41.992 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:41.992 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:41.992 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.992 INFO:teuthology.orchestra.run.smithi088.stdout: "req_mknod_latency": { 2024-04-21T08:51:41.992 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:41.993 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:41.993 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:41.993 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.993 INFO:teuthology.orchestra.run.smithi088.stdout: "req_mksnap_latency": { 2024-04-21T08:51:41.993 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:41.993 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:41.993 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:41.993 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.993 INFO:teuthology.orchestra.run.smithi088.stdout: "req_open_latency": { 2024-04-21T08:51:41.993 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:41.993 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:41.993 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:41.993 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.994 INFO:teuthology.orchestra.run.smithi088.stdout: "req_readdir_latency": { 2024-04-21T08:51:41.994 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:41.994 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:41.994 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:41.994 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.994 INFO:teuthology.orchestra.run.smithi088.stdout: "req_rename_latency": { 2024-04-21T08:51:41.994 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:41.994 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:41.994 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:41.994 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.994 INFO:teuthology.orchestra.run.smithi088.stdout: "req_renamesnap_latency": { 2024-04-21T08:51:41.994 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:41.994 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:41.994 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:41.995 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.995 INFO:teuthology.orchestra.run.smithi088.stdout: "req_snapdiff_latency": { 2024-04-21T08:51:41.995 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:41.995 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:41.995 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:41.995 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.995 INFO:teuthology.orchestra.run.smithi088.stdout: "req_rmdir_latency": { 2024-04-21T08:51:41.995 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:41.995 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:41.995 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:41.995 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.995 INFO:teuthology.orchestra.run.smithi088.stdout: "req_rmsnap_latency": { 2024-04-21T08:51:41.995 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:41.995 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:41.996 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:41.996 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.996 INFO:teuthology.orchestra.run.smithi088.stdout: "req_rmxattr_latency": { 2024-04-21T08:51:41.996 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:41.996 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:41.996 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:41.996 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.996 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setattr_latency": { 2024-04-21T08:51:41.996 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:41.996 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:41.996 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:41.996 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.996 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setdirlayout_latency": { 2024-04-21T08:51:41.996 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:41.996 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:41.997 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:41.997 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.997 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setfilelock_latency": { 2024-04-21T08:51:41.997 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:41.997 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:41.997 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:41.997 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.997 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setlayout_latency": { 2024-04-21T08:51:41.997 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:41.997 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:41.997 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:41.997 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.997 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setxattr_latency": { 2024-04-21T08:51:41.997 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:41.998 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:41.998 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:41.998 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.998 INFO:teuthology.orchestra.run.smithi088.stdout: "req_symlink_latency": { 2024-04-21T08:51:41.998 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:41.998 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:41.998 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:41.998 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.998 INFO:teuthology.orchestra.run.smithi088.stdout: "req_unlink_latency": { 2024-04-21T08:51:41.998 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:41.998 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:41.998 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:41.998 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.998 INFO:teuthology.orchestra.run.smithi088.stdout: "cap_revoke_eviction": 0, 2024-04-21T08:51:41.999 INFO:teuthology.orchestra.run.smithi088.stdout: "cap_acquisition_throttle": 0, 2024-04-21T08:51:41.999 INFO:teuthology.orchestra.run.smithi088.stdout: "req_getvxattr_latency": { 2024-04-21T08:51:41.999 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:41.999 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:41.999 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:41.999 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-21T08:51:41.999 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:41.999 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_sessions": { 2024-04-21T08:51:41.999 INFO:teuthology.orchestra.run.smithi088.stdout: "session_count": 0, 2024-04-21T08:51:41.999 INFO:teuthology.orchestra.run.smithi088.stdout: "session_add": 3, 2024-04-21T08:51:41.999 INFO:teuthology.orchestra.run.smithi088.stdout: "session_remove": 3, 2024-04-21T08:51:41.999 INFO:teuthology.orchestra.run.smithi088.stdout: "sessions_open": 0, 2024-04-21T08:51:41.999 INFO:teuthology.orchestra.run.smithi088.stdout: "sessions_stale": 0, 2024-04-21T08:51:41.999 INFO:teuthology.orchestra.run.smithi088.stdout: "total_load": 5, 2024-04-21T08:51:42.000 INFO:teuthology.orchestra.run.smithi088.stdout: "average_load": 2, 2024-04-21T08:51:42.000 INFO:teuthology.orchestra.run.smithi088.stdout: "avg_session_uptime": 86, 2024-04-21T08:51:42.000 INFO:teuthology.orchestra.run.smithi088.stdout: "mdthresh_evicted": 0 2024-04-21T08:51:42.000 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:42.000 INFO:teuthology.orchestra.run.smithi088.stdout: "mempool": { 2024-04-21T08:51:42.000 INFO:teuthology.orchestra.run.smithi088.stdout: "bloom_filter_bytes": 0, 2024-04-21T08:51:42.000 INFO:teuthology.orchestra.run.smithi088.stdout: "bloom_filter_items": 0, 2024-04-21T08:51:42.000 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T08:51:42.000 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_alloc_items": 0, 2024-04-21T08:51:42.000 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T08:51:42.000 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_data_items": 0, 2024-04-21T08:51:42.000 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T08:51:42.000 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T08:51:42.000 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T08:51:42.000 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T08:51:42.001 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T08:51:42.001 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_other_items": 0, 2024-04-21T08:51:42.001 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T08:51:42.001 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T08:51:42.001 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_extent_bytes": 0, 2024-04-21T08:51:42.001 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_extent_items": 0, 2024-04-21T08:51:42.001 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_blob_bytes": 0, 2024-04-21T08:51:42.001 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_blob_items": 0, 2024-04-21T08:51:42.001 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T08:51:42.001 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T08:51:42.001 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T08:51:42.001 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T08:51:42.001 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T08:51:42.001 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_fsck_items": 0, 2024-04-21T08:51:42.001 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_txc_bytes": 0, 2024-04-21T08:51:42.002 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_txc_items": 0, 2024-04-21T08:51:42.002 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T08:51:42.002 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T08:51:42.002 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_writing_bytes": 0, 2024-04-21T08:51:42.002 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_writing_items": 0, 2024-04-21T08:51:42.002 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_bytes": 0, 2024-04-21T08:51:42.002 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_items": 0, 2024-04-21T08:51:42.002 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T08:51:42.002 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_file_reader_items": 0, 2024-04-21T08:51:42.002 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T08:51:42.002 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_file_writer_items": 0, 2024-04-21T08:51:42.002 INFO:teuthology.orchestra.run.smithi088.stdout: "buffer_anon_bytes": 278311, 2024-04-21T08:51:42.002 INFO:teuthology.orchestra.run.smithi088.stdout: "buffer_anon_items": 91, 2024-04-21T08:51:42.002 INFO:teuthology.orchestra.run.smithi088.stdout: "buffer_meta_bytes": 0, 2024-04-21T08:51:42.003 INFO:teuthology.orchestra.run.smithi088.stdout: "buffer_meta_items": 0, 2024-04-21T08:51:42.003 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_bytes": 0, 2024-04-21T08:51:42.003 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_items": 0, 2024-04-21T08:51:42.003 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_mapbl_bytes": 0, 2024-04-21T08:51:42.003 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_mapbl_items": 0, 2024-04-21T08:51:42.003 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_pglog_bytes": 0, 2024-04-21T08:51:42.003 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_pglog_items": 0, 2024-04-21T08:51:42.003 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_bytes": 9512, 2024-04-21T08:51:42.003 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_items": 182, 2024-04-21T08:51:42.003 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T08:51:42.003 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_mapping_items": 0, 2024-04-21T08:51:42.003 INFO:teuthology.orchestra.run.smithi088.stdout: "pgmap_bytes": 0, 2024-04-21T08:51:42.003 INFO:teuthology.orchestra.run.smithi088.stdout: "pgmap_items": 0, 2024-04-21T08:51:42.003 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_co_bytes": 87213, 2024-04-21T08:51:42.004 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_co_items": 350, 2024-04-21T08:51:42.004 INFO:teuthology.orchestra.run.smithi088.stdout: "unittest_1_bytes": 0, 2024-04-21T08:51:42.004 INFO:teuthology.orchestra.run.smithi088.stdout: "unittest_1_items": 0, 2024-04-21T08:51:42.004 INFO:teuthology.orchestra.run.smithi088.stdout: "unittest_2_bytes": 0, 2024-04-21T08:51:42.004 INFO:teuthology.orchestra.run.smithi088.stdout: "unittest_2_items": 0 2024-04-21T08:51:42.004 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:42.004 INFO:teuthology.orchestra.run.smithi088.stdout: "objecter": { 2024-04-21T08:51:42.004 INFO:teuthology.orchestra.run.smithi088.stdout: "op_active": 0, 2024-04-21T08:51:42.004 INFO:teuthology.orchestra.run.smithi088.stdout: "op_laggy": 0, 2024-04-21T08:51:42.004 INFO:teuthology.orchestra.run.smithi088.stdout: "op_send": 46, 2024-04-21T08:51:42.004 INFO:teuthology.orchestra.run.smithi088.stdout: "op_send_bytes": 12721, 2024-04-21T08:51:42.004 INFO:teuthology.orchestra.run.smithi088.stdout: "op_resend": 28, 2024-04-21T08:51:42.004 INFO:teuthology.orchestra.run.smithi088.stdout: "op_reply": 46, 2024-04-21T08:51:42.005 INFO:teuthology.orchestra.run.smithi088.stdout: "op_latency": { 2024-04-21T08:51:42.005 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 46, 2024-04-21T08:51:42.005 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.255994677, 2024-04-21T08:51:42.005 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.005565101 2024-04-21T08:51:42.005 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:42.005 INFO:teuthology.orchestra.run.smithi088.stdout: "op_inflight": 0, 2024-04-21T08:51:42.005 INFO:teuthology.orchestra.run.smithi088.stdout: "oplen_avg": { 2024-04-21T08:51:42.005 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 46, 2024-04-21T08:51:42.005 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 70 2024-04-21T08:51:42.005 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:42.005 INFO:teuthology.orchestra.run.smithi088.stdout: "op": 46, 2024-04-21T08:51:42.005 INFO:teuthology.orchestra.run.smithi088.stdout: "op_r": 0, 2024-04-21T08:51:42.005 INFO:teuthology.orchestra.run.smithi088.stdout: "op_w": 46, 2024-04-21T08:51:42.005 INFO:teuthology.orchestra.run.smithi088.stdout: "op_rmw": 0, 2024-04-21T08:51:42.005 INFO:teuthology.orchestra.run.smithi088.stdout: "op_pg": 0, 2024-04-21T08:51:42.006 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_stat": 0, 2024-04-21T08:51:42.006 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_create": 10, 2024-04-21T08:51:42.006 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_read": 0, 2024-04-21T08:51:42.006 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_write": 7, 2024-04-21T08:51:42.006 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_writefull": 8, 2024-04-21T08:51:42.006 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_writesame": 0, 2024-04-21T08:51:42.006 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_append": 0, 2024-04-21T08:51:42.006 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_zero": 0, 2024-04-21T08:51:42.006 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_truncate": 0, 2024-04-21T08:51:42.006 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_delete": 6, 2024-04-21T08:51:42.006 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_mapext": 0, 2024-04-21T08:51:42.006 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_sparse_read": 0, 2024-04-21T08:51:42.006 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_clonerange": 0, 2024-04-21T08:51:42.006 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_getxattr": 0, 2024-04-21T08:51:42.007 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_setxattr": 20, 2024-04-21T08:51:42.007 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_cmpxattr": 0, 2024-04-21T08:51:42.007 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_rmxattr": 0, 2024-04-21T08:51:42.007 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_resetxattrs": 0, 2024-04-21T08:51:42.007 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_call": 0, 2024-04-21T08:51:42.007 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_watch": 0, 2024-04-21T08:51:42.007 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_notify": 0, 2024-04-21T08:51:42.007 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T08:51:42.007 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_pgls": 0, 2024-04-21T08:51:42.007 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_pgls_filter": 0, 2024-04-21T08:51:42.007 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_other": 0, 2024-04-21T08:51:42.007 INFO:teuthology.orchestra.run.smithi088.stdout: "linger_active": 0, 2024-04-21T08:51:42.007 INFO:teuthology.orchestra.run.smithi088.stdout: "linger_send": 0, 2024-04-21T08:51:42.007 INFO:teuthology.orchestra.run.smithi088.stdout: "linger_resend": 0, 2024-04-21T08:51:42.008 INFO:teuthology.orchestra.run.smithi088.stdout: "linger_ping": 0, 2024-04-21T08:51:42.008 INFO:teuthology.orchestra.run.smithi088.stdout: "poolop_active": 0, 2024-04-21T08:51:42.008 INFO:teuthology.orchestra.run.smithi088.stdout: "poolop_send": 0, 2024-04-21T08:51:42.008 INFO:teuthology.orchestra.run.smithi088.stdout: "poolop_resend": 0, 2024-04-21T08:51:42.008 INFO:teuthology.orchestra.run.smithi088.stdout: "poolstat_active": 0, 2024-04-21T08:51:42.008 INFO:teuthology.orchestra.run.smithi088.stdout: "poolstat_send": 0, 2024-04-21T08:51:42.008 INFO:teuthology.orchestra.run.smithi088.stdout: "poolstat_resend": 0, 2024-04-21T08:51:42.008 INFO:teuthology.orchestra.run.smithi088.stdout: "statfs_active": 0, 2024-04-21T08:51:42.008 INFO:teuthology.orchestra.run.smithi088.stdout: "statfs_send": 0, 2024-04-21T08:51:42.008 INFO:teuthology.orchestra.run.smithi088.stdout: "statfs_resend": 0, 2024-04-21T08:51:42.008 INFO:teuthology.orchestra.run.smithi088.stdout: "command_active": 0, 2024-04-21T08:51:42.008 INFO:teuthology.orchestra.run.smithi088.stdout: "command_send": 0, 2024-04-21T08:51:42.008 INFO:teuthology.orchestra.run.smithi088.stdout: "command_resend": 0, 2024-04-21T08:51:42.008 INFO:teuthology.orchestra.run.smithi088.stdout: "map_epoch": 108, 2024-04-21T08:51:42.008 INFO:teuthology.orchestra.run.smithi088.stdout: "map_full": 0, 2024-04-21T08:51:42.008 INFO:teuthology.orchestra.run.smithi088.stdout: "map_inc": 19, 2024-04-21T08:51:42.009 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_sessions": 9, 2024-04-21T08:51:42.009 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_session_open": 9, 2024-04-21T08:51:42.009 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_session_close": 0, 2024-04-21T08:51:42.009 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_laggy": 0, 2024-04-21T08:51:42.009 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_wr": 17, 2024-04-21T08:51:42.009 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_rd": 0, 2024-04-21T08:51:42.009 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_del": 2 2024-04-21T08:51:42.009 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:42.009 INFO:teuthology.orchestra.run.smithi088.stdout: "oft": { 2024-04-21T08:51:42.009 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_total_objs": 1, 2024-04-21T08:51:42.009 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_total_kv_pairs": 0, 2024-04-21T08:51:42.009 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_total_updates": 0, 2024-04-21T08:51:42.009 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_total_removes": 0 2024-04-21T08:51:42.009 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:42.009 INFO:teuthology.orchestra.run.smithi088.stdout: "osd-slow-ops": { 2024-04-21T08:51:42.010 INFO:teuthology.orchestra.run.smithi088.stdout: "slow_ops_count": 18 2024-04-21T08:51:42.010 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:42.010 INFO:teuthology.orchestra.run.smithi088.stdout: "purge_queue": { 2024-04-21T08:51:42.010 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executing_ops": 0, 2024-04-21T08:51:42.010 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executing_ops_high_water": 0, 2024-04-21T08:51:42.010 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executing": 0, 2024-04-21T08:51:42.010 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executing_high_water": 0, 2024-04-21T08:51:42.010 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executed_ops": 0, 2024-04-21T08:51:42.010 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executed": 0, 2024-04-21T08:51:42.010 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_item_in_journal": 0 2024-04-21T08:51:42.010 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:42.010 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T08:51:42.010 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 0, 2024-04-21T08:51:42.011 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 104857600, 2024-04-21T08:51:42.011 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-21T08:51:42.011 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 2079, 2024-04-21T08:51:42.011 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 1960081, 2024-04-21T08:51:42.011 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:42.011 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 2079, 2024-04-21T08:51:42.011 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 0, 2024-04-21T08:51:42.011 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 0, 2024-04-21T08:51:42.011 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 2079, 2024-04-21T08:51:42.011 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 1960081, 2024-04-21T08:51:42.011 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-21T08:51:42.011 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:42.011 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:42.011 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:42.012 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-21T08:51:42.012 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:42.012 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-objecter_bytes": { 2024-04-21T08:51:42.012 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 0, 2024-04-21T08:51:42.012 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 104857600, 2024-04-21T08:51:42.012 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-21T08:51:42.012 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 0, 2024-04-21T08:51:42.012 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 0, 2024-04-21T08:51:42.012 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:42.012 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:42.012 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 46, 2024-04-21T08:51:42.012 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 12721, 2024-04-21T08:51:42.012 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 46, 2024-04-21T08:51:42.012 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 12721, 2024-04-21T08:51:42.012 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-21T08:51:42.012 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:42.013 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:42.013 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:42.013 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-21T08:51:42.013 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:42.013 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-objecter_ops": { 2024-04-21T08:51:42.013 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 0, 2024-04-21T08:51:42.013 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 1024, 2024-04-21T08:51:42.013 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-21T08:51:42.013 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 0, 2024-04-21T08:51:42.013 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 0, 2024-04-21T08:51:42.013 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:42.013 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:42.013 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 46, 2024-04-21T08:51:42.013 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 46, 2024-04-21T08:51:42.014 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 46, 2024-04-21T08:51:42.014 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 46, 2024-04-21T08:51:42.014 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-21T08:51:42.014 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:42.014 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:42.014 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:42.014 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-21T08:51:42.014 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:42.014 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-write_buf_throttle": { 2024-04-21T08:51:42.014 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 0, 2024-04-21T08:51:42.014 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 3758096384, 2024-04-21T08:51:42.014 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-21T08:51:42.014 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 0, 2024-04-21T08:51:42.014 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 0, 2024-04-21T08:51:42.015 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:42.015 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:42.015 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 0, 2024-04-21T08:51:42.015 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 0, 2024-04-21T08:51:42.015 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 0, 2024-04-21T08:51:42.015 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 0, 2024-04-21T08:51:42.015 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-21T08:51:42.015 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:42.015 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:42.015 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:42.015 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-21T08:51:42.015 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:42.015 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-write_buf_throttle-0x55a9d8e56500": { 2024-04-21T08:51:42.015 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 0, 2024-04-21T08:51:42.015 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 3758096384, 2024-04-21T08:51:42.015 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-21T08:51:42.016 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 8, 2024-04-21T08:51:42.016 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 2184, 2024-04-21T08:51:42.016 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:42.016 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 8, 2024-04-21T08:51:42.016 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 0, 2024-04-21T08:51:42.016 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 0, 2024-04-21T08:51:42.016 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 7, 2024-04-21T08:51:42.016 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 2184, 2024-04-21T08:51:42.016 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-21T08:51:42.016 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:42.016 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:42.016 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:42.016 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-21T08:51:42.016 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-21T08:51:42.016 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-21T08:51:42.017 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:41 smithi088 ceph-mon[31259]: pgmap v404: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-21T08:51:42.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:41 smithi179 ceph-mon[31829]: pgmap v404: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-21T08:51:42.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:41 smithi086 ceph-mon[27939]: pgmap v404: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 4.6 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-21T08:51:42.802 INFO:tasks.check_counter:Daemon mds.h mds.exported=0 2024-04-21T08:51:42.802 INFO:tasks.check_counter:Daemon mds.h mds.imported=0 2024-04-21T08:51:42.802 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update=0 2024-04-21T08:51:42.802 INFO:tasks.check_counter:Daemon mds.h mds_cache.dir_update_receipt=19 2024-04-21T08:51:42.802 INFO:tasks.check_counter:Daemon mds.h mds.root_rsnaps=6 2024-04-21T08:51:42.802 INFO:tasks.check_counter:Daemon mds.h mds_server.req_mksnap_latency.avgcount=0 2024-04-21T08:51:42.802 INFO:tasks.check_counter:Daemon mds.h mds_server.req_rmsnap_latency.avgcount=0 2024-04-21T08:51:42.802 DEBUG:tasks.check_counter:Getting stats from k 2024-04-21T08:51:42.803 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph daemon mds.k perf dump 2024-04-21T08:51:43.039 INFO:teuthology.orchestra.run.smithi088.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.b/config 2024-04-21T08:51:44.183 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:43 smithi088 ceph-mon[31259]: pgmap v405: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 853 B/s wr, 13 op/s 2024-04-21T08:51:44.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:43 smithi179 ceph-mon[31829]: pgmap v405: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 853 B/s wr, 13 op/s 2024-04-21T08:51:44.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:43 smithi086 ceph-mon[27939]: pgmap v405: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 853 B/s wr, 13 op/s 2024-04-21T08:51:44.754 INFO:teuthology.orchestra.run.smithi088.stdout:{ 2024-04-21T08:51:44.754 INFO:teuthology.orchestra.run.smithi088.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T08:51:44.754 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_messages": 141, 2024-04-21T08:51:44.754 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages": 141, 2024-04-21T08:51:44.754 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_bytes": 40787, 2024-04-21T08:51:44.754 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_bytes": 645750, 2024-04-21T08:51:44.754 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_created_connections": 63, 2024-04-21T08:51:44.755 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_active_connections": 2, 2024-04-21T08:51:44.755 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_total_time": 0.030574907, 2024-04-21T08:51:44.755 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_send_time": 0.004842320, 2024-04-21T08:51:44.755 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_recv_time": 0.020470945, 2024-04-21T08:51:44.755 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_fast_dispatch_time": 0.000407055, 2024-04-21T08:51:44.755 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:44.755 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 140, 2024-04-21T08:51:44.755 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.011126680, 2024-04-21T08:51:44.755 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000079476 2024-04-21T08:51:44.755 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.755 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:44.755 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 167, 2024-04-21T08:51:44.755 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000034866, 2024-04-21T08:51:44.755 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000208 2024-04-21T08:51:44.756 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.756 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_encrypted_bytes": 3920, 2024-04-21T08:51:44.756 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_encrypted_bytes": 1184 2024-04-21T08:51:44.756 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.756 INFO:teuthology.orchestra.run.smithi088.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T08:51:44.756 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_messages": 399, 2024-04-21T08:51:44.756 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages": 383, 2024-04-21T08:51:44.756 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_bytes": 1694226, 2024-04-21T08:51:44.756 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_bytes": 196462, 2024-04-21T08:51:44.756 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_created_connections": 7, 2024-04-21T08:51:44.756 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_active_connections": 5, 2024-04-21T08:51:44.756 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_total_time": 0.065036870, 2024-04-21T08:51:44.756 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_send_time": 0.020219426, 2024-04-21T08:51:44.757 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_recv_time": 0.048042433, 2024-04-21T08:51:44.757 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_fast_dispatch_time": 0.006426605, 2024-04-21T08:51:44.757 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:44.757 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 382, 2024-04-21T08:51:44.757 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.060139060, 2024-04-21T08:51:44.757 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000157432 2024-04-21T08:51:44.757 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.757 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:44.757 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 113, 2024-04-21T08:51:44.757 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000027178, 2024-04-21T08:51:44.757 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000240 2024-04-21T08:51:44.757 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.757 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_encrypted_bytes": 1666960, 2024-04-21T08:51:44.757 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_encrypted_bytes": 166816 2024-04-21T08:51:44.758 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.758 INFO:teuthology.orchestra.run.smithi088.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T08:51:44.758 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_messages": 1523, 2024-04-21T08:51:44.758 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages": 1288, 2024-04-21T08:51:44.758 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_bytes": 420928, 2024-04-21T08:51:44.758 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_bytes": 743374, 2024-04-21T08:51:44.758 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_created_connections": 10, 2024-04-21T08:51:44.758 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_active_connections": 5, 2024-04-21T08:51:44.758 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_total_time": 0.146772615, 2024-04-21T08:51:44.758 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_send_time": 0.071401486, 2024-04-21T08:51:44.758 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_recv_time": 0.081179667, 2024-04-21T08:51:44.758 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_running_fast_dispatch_time": 0.004909381, 2024-04-21T08:51:44.758 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:44.759 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 1285, 2024-04-21T08:51:44.759 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.080275455, 2024-04-21T08:51:44.759 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000062471 2024-04-21T08:51:44.759 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.759 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:44.759 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 2490, 2024-04-21T08:51:44.759 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000891980, 2024-04-21T08:51:44.759 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000358 2024-04-21T08:51:44.759 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.759 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-21T08:51:44.759 INFO:teuthology.orchestra.run.smithi088.stdout: "msgr_send_encrypted_bytes": 498400 2024-04-21T08:51:44.760 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.760 INFO:teuthology.orchestra.run.smithi088.stdout: "cct": { 2024-04-21T08:51:44.760 INFO:teuthology.orchestra.run.smithi088.stdout: "total_workers": 1, 2024-04-21T08:51:44.760 INFO:teuthology.orchestra.run.smithi088.stdout: "unhealthy_workers": 0 2024-04-21T08:51:44.760 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.760 INFO:teuthology.orchestra.run.smithi088.stdout: "finisher-MDSRank": { 2024-04-21T08:51:44.760 INFO:teuthology.orchestra.run.smithi088.stdout: "queue_len": 0, 2024-04-21T08:51:44.760 INFO:teuthology.orchestra.run.smithi088.stdout: "complete_latency": { 2024-04-21T08:51:44.760 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 38, 2024-04-21T08:51:44.760 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.004252175, 2024-04-21T08:51:44.760 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000111899 2024-04-21T08:51:44.761 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-21T08:51:44.761 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.761 INFO:teuthology.orchestra.run.smithi088.stdout: "finisher-PurgeQueue": { 2024-04-21T08:51:44.761 INFO:teuthology.orchestra.run.smithi088.stdout: "queue_len": 0, 2024-04-21T08:51:44.761 INFO:teuthology.orchestra.run.smithi088.stdout: "complete_latency": { 2024-04-21T08:51:44.761 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 2, 2024-04-21T08:51:44.761 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000127135, 2024-04-21T08:51:44.761 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000063567 2024-04-21T08:51:44.761 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-21T08:51:44.761 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.761 INFO:teuthology.orchestra.run.smithi088.stdout: "mds": { 2024-04-21T08:51:44.761 INFO:teuthology.orchestra.run.smithi088.stdout: "request": 0, 2024-04-21T08:51:44.761 INFO:teuthology.orchestra.run.smithi088.stdout: "reply": 0, 2024-04-21T08:51:44.761 INFO:teuthology.orchestra.run.smithi088.stdout: "reply_latency": { 2024-04-21T08:51:44.762 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:44.762 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:44.762 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:44.762 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.762 INFO:teuthology.orchestra.run.smithi088.stdout: "slow_reply": 0, 2024-04-21T08:51:44.762 INFO:teuthology.orchestra.run.smithi088.stdout: "forward": 0, 2024-04-21T08:51:44.762 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_fetch_complete": 0, 2024-04-21T08:51:44.762 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_fetch_keys": 0, 2024-04-21T08:51:44.762 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_commit": 11, 2024-04-21T08:51:44.762 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_split": 0, 2024-04-21T08:51:44.762 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_merge": 0, 2024-04-21T08:51:44.762 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes": 16, 2024-04-21T08:51:44.762 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_top": 0, 2024-04-21T08:51:44.762 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_bottom": 16, 2024-04-21T08:51:44.763 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_pin_tail": 0, 2024-04-21T08:51:44.763 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_pinned": 13, 2024-04-21T08:51:44.763 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_expired": 3, 2024-04-21T08:51:44.763 INFO:teuthology.orchestra.run.smithi088.stdout: "inodes_with_caps": 0, 2024-04-21T08:51:44.763 INFO:teuthology.orchestra.run.smithi088.stdout: "caps": 0, 2024-04-21T08:51:44.763 INFO:teuthology.orchestra.run.smithi088.stdout: "subtrees": 3, 2024-04-21T08:51:44.763 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse": 20, 2024-04-21T08:51:44.763 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_hit": 11, 2024-04-21T08:51:44.763 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_forward": 0, 2024-04-21T08:51:44.763 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_discover": 8, 2024-04-21T08:51:44.763 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_dir_fetch": 0, 2024-04-21T08:51:44.763 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_remote_ino": 0, 2024-04-21T08:51:44.763 INFO:teuthology.orchestra.run.smithi088.stdout: "traverse_lock": 0, 2024-04-21T08:51:44.763 INFO:teuthology.orchestra.run.smithi088.stdout: "load_cent": 0, 2024-04-21T08:51:44.764 INFO:teuthology.orchestra.run.smithi088.stdout: "q": 0, 2024-04-21T08:51:44.764 INFO:teuthology.orchestra.run.smithi088.stdout: "exported": 0, 2024-04-21T08:51:44.764 INFO:teuthology.orchestra.run.smithi088.stdout: "exported_inodes": 0, 2024-04-21T08:51:44.764 INFO:teuthology.orchestra.run.smithi088.stdout: "imported": 0, 2024-04-21T08:51:44.764 INFO:teuthology.orchestra.run.smithi088.stdout: "imported_inodes": 0, 2024-04-21T08:51:44.764 INFO:teuthology.orchestra.run.smithi088.stdout: "openino_dir_fetch": 0, 2024-04-21T08:51:44.764 INFO:teuthology.orchestra.run.smithi088.stdout: "openino_backtrace_fetch": 0, 2024-04-21T08:51:44.764 INFO:teuthology.orchestra.run.smithi088.stdout: "openino_peer_discover": 0, 2024-04-21T08:51:44.764 INFO:teuthology.orchestra.run.smithi088.stdout: "root_rfiles": 5, 2024-04-21T08:51:44.764 INFO:teuthology.orchestra.run.smithi088.stdout: "root_rbytes": 298, 2024-04-21T08:51:44.764 INFO:teuthology.orchestra.run.smithi088.stdout: "root_rsnaps": 6, 2024-04-21T08:51:44.764 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_backtrace_fetch": 0, 2024-04-21T08:51:44.764 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_set_tag": 0, 2024-04-21T08:51:44.764 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T08:51:44.764 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_inotable_repaired": 0, 2024-04-21T08:51:44.765 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_dir_inodes": 0, 2024-04-21T08:51:44.765 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_dir_base_inodes": 0, 2024-04-21T08:51:44.765 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_dirfrag_rstats": 0, 2024-04-21T08:51:44.765 INFO:teuthology.orchestra.run.smithi088.stdout: "scrub_file_inodes": 0, 2024-04-21T08:51:44.765 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_inode_file_caps": 0, 2024-04-21T08:51:44.765 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_revoke": 0, 2024-04-21T08:51:44.765 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_grant": 0, 2024-04-21T08:51:44.765 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_trunc": 0, 2024-04-21T08:51:44.765 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T08:51:44.765 INFO:teuthology.orchestra.run.smithi088.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T08:51:44.765 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_caps": 0, 2024-04-21T08:51:44.765 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_caps_dirty": 0, 2024-04-21T08:51:44.765 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_cap_release": 0, 2024-04-21T08:51:44.765 INFO:teuthology.orchestra.run.smithi088.stdout: "process_request_cap_release": 0 2024-04-21T08:51:44.766 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.766 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_cache": { 2024-04-21T08:51:44.766 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_update": 0, 2024-04-21T08:51:44.766 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_update_receipt": 19, 2024-04-21T08:51:44.766 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_try_discover": 14, 2024-04-21T08:51:44.766 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_send_discover": 11, 2024-04-21T08:51:44.766 INFO:teuthology.orchestra.run.smithi088.stdout: "dir_handle_discover": 0, 2024-04-21T08:51:44.766 INFO:teuthology.orchestra.run.smithi088.stdout: "num_strays": 0, 2024-04-21T08:51:44.766 INFO:teuthology.orchestra.run.smithi088.stdout: "num_strays_delayed": 0, 2024-04-21T08:51:44.766 INFO:teuthology.orchestra.run.smithi088.stdout: "num_strays_enqueuing": 0, 2024-04-21T08:51:44.766 INFO:teuthology.orchestra.run.smithi088.stdout: "strays_created": 0, 2024-04-21T08:51:44.766 INFO:teuthology.orchestra.run.smithi088.stdout: "strays_enqueued": 0, 2024-04-21T08:51:44.766 INFO:teuthology.orchestra.run.smithi088.stdout: "strays_reintegrated": 0, 2024-04-21T08:51:44.766 INFO:teuthology.orchestra.run.smithi088.stdout: "strays_migrated": 0, 2024-04-21T08:51:44.767 INFO:teuthology.orchestra.run.smithi088.stdout: "num_recovering_processing": 0, 2024-04-21T08:51:44.767 INFO:teuthology.orchestra.run.smithi088.stdout: "num_recovering_enqueued": 0, 2024-04-21T08:51:44.767 INFO:teuthology.orchestra.run.smithi088.stdout: "num_recovering_prioritized": 0, 2024-04-21T08:51:44.767 INFO:teuthology.orchestra.run.smithi088.stdout: "recovery_started": 0, 2024-04-21T08:51:44.767 INFO:teuthology.orchestra.run.smithi088.stdout: "recovery_completed": 0, 2024-04-21T08:51:44.767 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_quiesce_path": 3, 2024-04-21T08:51:44.767 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_quiesce_inode": 15, 2024-04-21T08:51:44.767 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_enqueue_scrub": 0, 2024-04-21T08:51:44.767 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_exportdir": 0, 2024-04-21T08:51:44.767 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_flush": 0, 2024-04-21T08:51:44.767 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_fragmentdir": 0, 2024-04-21T08:51:44.767 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_fragstats": 0, 2024-04-21T08:51:44.767 INFO:teuthology.orchestra.run.smithi088.stdout: "ireq_inodestats": 0 2024-04-21T08:51:44.767 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.767 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_log": { 2024-04-21T08:51:44.768 INFO:teuthology.orchestra.run.smithi088.stdout: "evlrg": 0, 2024-04-21T08:51:44.768 INFO:teuthology.orchestra.run.smithi088.stdout: "evadd": 4, 2024-04-21T08:51:44.768 INFO:teuthology.orchestra.run.smithi088.stdout: "evex": 0, 2024-04-21T08:51:44.768 INFO:teuthology.orchestra.run.smithi088.stdout: "evtrm": 0, 2024-04-21T08:51:44.768 INFO:teuthology.orchestra.run.smithi088.stdout: "ev": 4, 2024-04-21T08:51:44.768 INFO:teuthology.orchestra.run.smithi088.stdout: "evexg": 0, 2024-04-21T08:51:44.768 INFO:teuthology.orchestra.run.smithi088.stdout: "evexd": 0, 2024-04-21T08:51:44.768 INFO:teuthology.orchestra.run.smithi088.stdout: "segadd": 2, 2024-04-21T08:51:44.768 INFO:teuthology.orchestra.run.smithi088.stdout: "segex": 0, 2024-04-21T08:51:44.768 INFO:teuthology.orchestra.run.smithi088.stdout: "segtrm": 0, 2024-04-21T08:51:44.768 INFO:teuthology.orchestra.run.smithi088.stdout: "seg": 2, 2024-04-21T08:51:44.768 INFO:teuthology.orchestra.run.smithi088.stdout: "segmjr": 2, 2024-04-21T08:51:44.768 INFO:teuthology.orchestra.run.smithi088.stdout: "segexg": 0, 2024-04-21T08:51:44.768 INFO:teuthology.orchestra.run.smithi088.stdout: "segexd": 0, 2024-04-21T08:51:44.769 INFO:teuthology.orchestra.run.smithi088.stdout: "expos": 4194304, 2024-04-21T08:51:44.769 INFO:teuthology.orchestra.run.smithi088.stdout: "wrpos": 4195424, 2024-04-21T08:51:44.769 INFO:teuthology.orchestra.run.smithi088.stdout: "rdpos": 0, 2024-04-21T08:51:44.769 INFO:teuthology.orchestra.run.smithi088.stdout: "jlat": { 2024-04-21T08:51:44.769 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 3, 2024-04-21T08:51:44.769 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.005588004, 2024-04-21T08:51:44.769 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.001862668 2024-04-21T08:51:44.769 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.769 INFO:teuthology.orchestra.run.smithi088.stdout: "replayed": 0 2024-04-21T08:51:44.769 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.769 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_mem": { 2024-04-21T08:51:44.769 INFO:teuthology.orchestra.run.smithi088.stdout: "ino": 20, 2024-04-21T08:51:44.769 INFO:teuthology.orchestra.run.smithi088.stdout: "ino+": 21, 2024-04-21T08:51:44.770 INFO:teuthology.orchestra.run.smithi088.stdout: "ino-": 1, 2024-04-21T08:51:44.770 INFO:teuthology.orchestra.run.smithi088.stdout: "dir": 19, 2024-04-21T08:51:44.770 INFO:teuthology.orchestra.run.smithi088.stdout: "dir+": 21, 2024-04-21T08:51:44.770 INFO:teuthology.orchestra.run.smithi088.stdout: "dir-": 2, 2024-04-21T08:51:44.770 INFO:teuthology.orchestra.run.smithi088.stdout: "dn": 16, 2024-04-21T08:51:44.770 INFO:teuthology.orchestra.run.smithi088.stdout: "dn+": 19, 2024-04-21T08:51:44.770 INFO:teuthology.orchestra.run.smithi088.stdout: "dn-": 3, 2024-04-21T08:51:44.770 INFO:teuthology.orchestra.run.smithi088.stdout: "cap": 0, 2024-04-21T08:51:44.770 INFO:teuthology.orchestra.run.smithi088.stdout: "cap+": 0, 2024-04-21T08:51:44.770 INFO:teuthology.orchestra.run.smithi088.stdout: "cap-": 0, 2024-04-21T08:51:44.770 INFO:teuthology.orchestra.run.smithi088.stdout: "rss": 52848, 2024-04-21T08:51:44.770 INFO:teuthology.orchestra.run.smithi088.stdout: "heap": 223608 2024-04-21T08:51:44.770 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.770 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_server": { 2024-04-21T08:51:44.770 INFO:teuthology.orchestra.run.smithi088.stdout: "dispatch_client_request": 0, 2024-04-21T08:51:44.771 INFO:teuthology.orchestra.run.smithi088.stdout: "dispatch_server_request": 0, 2024-04-21T08:51:44.771 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_request": 0, 2024-04-21T08:51:44.771 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_client_session": 4, 2024-04-21T08:51:44.771 INFO:teuthology.orchestra.run.smithi088.stdout: "handle_peer_request": 15, 2024-04-21T08:51:44.771 INFO:teuthology.orchestra.run.smithi088.stdout: "req_create_latency": { 2024-04-21T08:51:44.771 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:44.771 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:44.771 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:44.771 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.771 INFO:teuthology.orchestra.run.smithi088.stdout: "req_getattr_latency": { 2024-04-21T08:51:44.771 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:44.771 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:44.771 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:44.771 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.772 INFO:teuthology.orchestra.run.smithi088.stdout: "req_getfilelock_latency": { 2024-04-21T08:51:44.772 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:44.772 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:44.772 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:44.772 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.772 INFO:teuthology.orchestra.run.smithi088.stdout: "req_link_latency": { 2024-04-21T08:51:44.772 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:44.772 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:44.772 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:44.772 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.772 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookup_latency": { 2024-04-21T08:51:44.772 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:44.773 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:44.773 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:44.773 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.773 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookuphash_latency": { 2024-04-21T08:51:44.773 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:44.773 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:44.773 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:44.773 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.773 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookupino_latency": { 2024-04-21T08:51:44.773 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:44.773 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:44.773 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:44.773 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.773 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookupname_latency": { 2024-04-21T08:51:44.773 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:44.774 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:44.774 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:44.774 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.774 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookupparent_latency": { 2024-04-21T08:51:44.774 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:44.774 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:44.775 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:44.775 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.775 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lookupsnap_latency": { 2024-04-21T08:51:44.775 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:44.775 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:44.775 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:44.775 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.775 INFO:teuthology.orchestra.run.smithi088.stdout: "req_lssnap_latency": { 2024-04-21T08:51:44.775 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:44.775 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:44.775 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:44.775 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.775 INFO:teuthology.orchestra.run.smithi088.stdout: "req_mkdir_latency": { 2024-04-21T08:51:44.776 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:44.777 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:44.777 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:44.777 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.777 INFO:teuthology.orchestra.run.smithi088.stdout: "req_mknod_latency": { 2024-04-21T08:51:44.777 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:44.777 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:44.777 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:44.777 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.777 INFO:teuthology.orchestra.run.smithi088.stdout: "req_mksnap_latency": { 2024-04-21T08:51:44.777 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:44.777 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:44.777 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:44.778 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.778 INFO:teuthology.orchestra.run.smithi088.stdout: "req_open_latency": { 2024-04-21T08:51:44.778 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:44.778 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:44.778 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:44.778 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.778 INFO:teuthology.orchestra.run.smithi088.stdout: "req_readdir_latency": { 2024-04-21T08:51:44.778 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:44.778 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:44.778 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:44.778 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.778 INFO:teuthology.orchestra.run.smithi088.stdout: "req_rename_latency": { 2024-04-21T08:51:44.779 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:44.779 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:44.779 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:44.779 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.779 INFO:teuthology.orchestra.run.smithi088.stdout: "req_renamesnap_latency": { 2024-04-21T08:51:44.779 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:44.779 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:44.779 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:44.779 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.779 INFO:teuthology.orchestra.run.smithi088.stdout: "req_snapdiff_latency": { 2024-04-21T08:51:44.779 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:44.779 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:44.779 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:44.779 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.780 INFO:teuthology.orchestra.run.smithi088.stdout: "req_rmdir_latency": { 2024-04-21T08:51:44.780 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:44.780 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:44.780 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:44.780 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.780 INFO:teuthology.orchestra.run.smithi088.stdout: "req_rmsnap_latency": { 2024-04-21T08:51:44.780 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:44.780 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:44.780 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:44.780 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.780 INFO:teuthology.orchestra.run.smithi088.stdout: "req_rmxattr_latency": { 2024-04-21T08:51:44.780 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:44.780 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:44.780 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:44.781 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.781 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setattr_latency": { 2024-04-21T08:51:44.781 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:44.781 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:44.781 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:44.781 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.781 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setdirlayout_latency": { 2024-04-21T08:51:44.781 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:44.781 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:44.781 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:44.781 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.781 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setfilelock_latency": { 2024-04-21T08:51:44.781 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:44.781 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:44.782 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:44.782 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.782 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setlayout_latency": { 2024-04-21T08:51:44.782 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:44.782 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:44.782 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:44.782 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.782 INFO:teuthology.orchestra.run.smithi088.stdout: "req_setxattr_latency": { 2024-04-21T08:51:44.782 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:44.782 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:44.782 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:44.782 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.782 INFO:teuthology.orchestra.run.smithi088.stdout: "req_symlink_latency": { 2024-04-21T08:51:44.782 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:44.783 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:44.783 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:44.783 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.783 INFO:teuthology.orchestra.run.smithi088.stdout: "req_unlink_latency": { 2024-04-21T08:51:44.783 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:44.783 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:44.783 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:44.783 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.783 INFO:teuthology.orchestra.run.smithi088.stdout: "cap_revoke_eviction": 0, 2024-04-21T08:51:44.783 INFO:teuthology.orchestra.run.smithi088.stdout: "cap_acquisition_throttle": 0, 2024-04-21T08:51:44.783 INFO:teuthology.orchestra.run.smithi088.stdout: "req_getvxattr_latency": { 2024-04-21T08:51:44.783 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:44.783 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:44.783 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:44.784 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-21T08:51:44.784 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.784 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_sessions": { 2024-04-21T08:51:44.784 INFO:teuthology.orchestra.run.smithi088.stdout: "session_count": 0, 2024-04-21T08:51:44.784 INFO:teuthology.orchestra.run.smithi088.stdout: "session_add": 1, 2024-04-21T08:51:44.784 INFO:teuthology.orchestra.run.smithi088.stdout: "session_remove": 1, 2024-04-21T08:51:44.784 INFO:teuthology.orchestra.run.smithi088.stdout: "sessions_open": 0, 2024-04-21T08:51:44.784 INFO:teuthology.orchestra.run.smithi088.stdout: "sessions_stale": 0, 2024-04-21T08:51:44.784 INFO:teuthology.orchestra.run.smithi088.stdout: "total_load": 0, 2024-04-21T08:51:44.784 INFO:teuthology.orchestra.run.smithi088.stdout: "average_load": 0, 2024-04-21T08:51:44.784 INFO:teuthology.orchestra.run.smithi088.stdout: "avg_session_uptime": 64, 2024-04-21T08:51:44.784 INFO:teuthology.orchestra.run.smithi088.stdout: "mdthresh_evicted": 0 2024-04-21T08:51:44.785 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.785 INFO:teuthology.orchestra.run.smithi088.stdout: "mempool": { 2024-04-21T08:51:44.785 INFO:teuthology.orchestra.run.smithi088.stdout: "bloom_filter_bytes": 0, 2024-04-21T08:51:44.785 INFO:teuthology.orchestra.run.smithi088.stdout: "bloom_filter_items": 0, 2024-04-21T08:51:44.785 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T08:51:44.785 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_alloc_items": 0, 2024-04-21T08:51:44.785 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T08:51:44.785 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_data_items": 0, 2024-04-21T08:51:44.785 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T08:51:44.785 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T08:51:44.785 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T08:51:44.785 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T08:51:44.785 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T08:51:44.786 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_other_items": 0, 2024-04-21T08:51:44.786 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T08:51:44.786 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T08:51:44.786 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_extent_bytes": 0, 2024-04-21T08:51:44.786 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_extent_items": 0, 2024-04-21T08:51:44.786 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_blob_bytes": 0, 2024-04-21T08:51:44.786 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_blob_items": 0, 2024-04-21T08:51:44.786 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T08:51:44.786 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T08:51:44.786 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T08:51:44.786 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T08:51:44.786 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T08:51:44.786 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_fsck_items": 0, 2024-04-21T08:51:44.786 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_txc_bytes": 0, 2024-04-21T08:51:44.787 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_txc_items": 0, 2024-04-21T08:51:44.787 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T08:51:44.787 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T08:51:44.787 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_writing_bytes": 0, 2024-04-21T08:51:44.787 INFO:teuthology.orchestra.run.smithi088.stdout: "bluestore_writing_items": 0, 2024-04-21T08:51:44.787 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_bytes": 0, 2024-04-21T08:51:44.787 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_items": 0, 2024-04-21T08:51:44.787 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T08:51:44.787 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_file_reader_items": 0, 2024-04-21T08:51:44.787 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T08:51:44.787 INFO:teuthology.orchestra.run.smithi088.stdout: "bluefs_file_writer_items": 0, 2024-04-21T08:51:44.787 INFO:teuthology.orchestra.run.smithi088.stdout: "buffer_anon_bytes": 230551, 2024-04-21T08:51:44.787 INFO:teuthology.orchestra.run.smithi088.stdout: "buffer_anon_items": 72, 2024-04-21T08:51:44.787 INFO:teuthology.orchestra.run.smithi088.stdout: "buffer_meta_bytes": 0, 2024-04-21T08:51:44.787 INFO:teuthology.orchestra.run.smithi088.stdout: "buffer_meta_items": 0, 2024-04-21T08:51:44.788 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_bytes": 0, 2024-04-21T08:51:44.788 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_items": 0, 2024-04-21T08:51:44.788 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_mapbl_bytes": 0, 2024-04-21T08:51:44.788 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_mapbl_items": 0, 2024-04-21T08:51:44.788 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_pglog_bytes": 0, 2024-04-21T08:51:44.788 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_pglog_items": 0, 2024-04-21T08:51:44.788 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_bytes": 9512, 2024-04-21T08:51:44.788 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_items": 182, 2024-04-21T08:51:44.788 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T08:51:44.788 INFO:teuthology.orchestra.run.smithi088.stdout: "osdmap_mapping_items": 0, 2024-04-21T08:51:44.788 INFO:teuthology.orchestra.run.smithi088.stdout: "pgmap_bytes": 0, 2024-04-21T08:51:44.788 INFO:teuthology.orchestra.run.smithi088.stdout: "pgmap_items": 0, 2024-04-21T08:51:44.788 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_co_bytes": 87141, 2024-04-21T08:51:44.788 INFO:teuthology.orchestra.run.smithi088.stdout: "mds_co_items": 341, 2024-04-21T08:51:44.788 INFO:teuthology.orchestra.run.smithi088.stdout: "unittest_1_bytes": 0, 2024-04-21T08:51:44.789 INFO:teuthology.orchestra.run.smithi088.stdout: "unittest_1_items": 0, 2024-04-21T08:51:44.789 INFO:teuthology.orchestra.run.smithi088.stdout: "unittest_2_bytes": 0, 2024-04-21T08:51:44.789 INFO:teuthology.orchestra.run.smithi088.stdout: "unittest_2_items": 0 2024-04-21T08:51:44.789 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.789 INFO:teuthology.orchestra.run.smithi088.stdout: "objecter": { 2024-04-21T08:51:44.789 INFO:teuthology.orchestra.run.smithi088.stdout: "op_active": 0, 2024-04-21T08:51:44.789 INFO:teuthology.orchestra.run.smithi088.stdout: "op_laggy": 0, 2024-04-21T08:51:44.789 INFO:teuthology.orchestra.run.smithi088.stdout: "op_send": 38, 2024-04-21T08:51:44.789 INFO:teuthology.orchestra.run.smithi088.stdout: "op_send_bytes": 11332, 2024-04-21T08:51:44.789 INFO:teuthology.orchestra.run.smithi088.stdout: "op_resend": 33, 2024-04-21T08:51:44.789 INFO:teuthology.orchestra.run.smithi088.stdout: "op_reply": 38, 2024-04-21T08:51:44.789 INFO:teuthology.orchestra.run.smithi088.stdout: "op_latency": { 2024-04-21T08:51:44.789 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 38, 2024-04-21T08:51:44.790 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.174996359, 2024-04-21T08:51:44.790 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.004605167 2024-04-21T08:51:44.790 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.790 INFO:teuthology.orchestra.run.smithi088.stdout: "op_inflight": 0, 2024-04-21T08:51:44.790 INFO:teuthology.orchestra.run.smithi088.stdout: "oplen_avg": { 2024-04-21T08:51:44.790 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 38, 2024-04-21T08:51:44.790 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 60 2024-04-21T08:51:44.790 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.790 INFO:teuthology.orchestra.run.smithi088.stdout: "op": 38, 2024-04-21T08:51:44.790 INFO:teuthology.orchestra.run.smithi088.stdout: "op_r": 0, 2024-04-21T08:51:44.790 INFO:teuthology.orchestra.run.smithi088.stdout: "op_w": 38, 2024-04-21T08:51:44.790 INFO:teuthology.orchestra.run.smithi088.stdout: "op_rmw": 0, 2024-04-21T08:51:44.790 INFO:teuthology.orchestra.run.smithi088.stdout: "op_pg": 0, 2024-04-21T08:51:44.790 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_stat": 0, 2024-04-21T08:51:44.791 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_create": 10, 2024-04-21T08:51:44.791 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_read": 0, 2024-04-21T08:51:44.791 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_write": 3, 2024-04-21T08:51:44.791 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_writefull": 6, 2024-04-21T08:51:44.791 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_writesame": 0, 2024-04-21T08:51:44.791 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_append": 0, 2024-04-21T08:51:44.791 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_zero": 0, 2024-04-21T08:51:44.791 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_truncate": 0, 2024-04-21T08:51:44.791 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_delete": 6, 2024-04-21T08:51:44.791 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_mapext": 0, 2024-04-21T08:51:44.791 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_sparse_read": 0, 2024-04-21T08:51:44.791 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_clonerange": 0, 2024-04-21T08:51:44.791 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_getxattr": 0, 2024-04-21T08:51:44.791 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_setxattr": 20, 2024-04-21T08:51:44.791 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_cmpxattr": 0, 2024-04-21T08:51:44.792 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_rmxattr": 0, 2024-04-21T08:51:44.792 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_resetxattrs": 0, 2024-04-21T08:51:44.792 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_call": 0, 2024-04-21T08:51:44.792 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_watch": 0, 2024-04-21T08:51:44.792 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_notify": 0, 2024-04-21T08:51:44.792 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T08:51:44.792 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_pgls": 0, 2024-04-21T08:51:44.792 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_pgls_filter": 0, 2024-04-21T08:51:44.792 INFO:teuthology.orchestra.run.smithi088.stdout: "osdop_other": 0, 2024-04-21T08:51:44.792 INFO:teuthology.orchestra.run.smithi088.stdout: "linger_active": 0, 2024-04-21T08:51:44.792 INFO:teuthology.orchestra.run.smithi088.stdout: "linger_send": 0, 2024-04-21T08:51:44.792 INFO:teuthology.orchestra.run.smithi088.stdout: "linger_resend": 0, 2024-04-21T08:51:44.792 INFO:teuthology.orchestra.run.smithi088.stdout: "linger_ping": 0, 2024-04-21T08:51:44.792 INFO:teuthology.orchestra.run.smithi088.stdout: "poolop_active": 0, 2024-04-21T08:51:44.792 INFO:teuthology.orchestra.run.smithi088.stdout: "poolop_send": 0, 2024-04-21T08:51:44.793 INFO:teuthology.orchestra.run.smithi088.stdout: "poolop_resend": 0, 2024-04-21T08:51:44.793 INFO:teuthology.orchestra.run.smithi088.stdout: "poolstat_active": 0, 2024-04-21T08:51:44.793 INFO:teuthology.orchestra.run.smithi088.stdout: "poolstat_send": 0, 2024-04-21T08:51:44.793 INFO:teuthology.orchestra.run.smithi088.stdout: "poolstat_resend": 0, 2024-04-21T08:51:44.793 INFO:teuthology.orchestra.run.smithi088.stdout: "statfs_active": 0, 2024-04-21T08:51:44.793 INFO:teuthology.orchestra.run.smithi088.stdout: "statfs_send": 0, 2024-04-21T08:51:44.793 INFO:teuthology.orchestra.run.smithi088.stdout: "statfs_resend": 0, 2024-04-21T08:51:44.793 INFO:teuthology.orchestra.run.smithi088.stdout: "command_active": 0, 2024-04-21T08:51:44.793 INFO:teuthology.orchestra.run.smithi088.stdout: "command_send": 0, 2024-04-21T08:51:44.793 INFO:teuthology.orchestra.run.smithi088.stdout: "command_resend": 0, 2024-04-21T08:51:44.793 INFO:teuthology.orchestra.run.smithi088.stdout: "map_epoch": 108, 2024-04-21T08:51:44.793 INFO:teuthology.orchestra.run.smithi088.stdout: "map_full": 0, 2024-04-21T08:51:44.793 INFO:teuthology.orchestra.run.smithi088.stdout: "map_inc": 19, 2024-04-21T08:51:44.793 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_sessions": 8, 2024-04-21T08:51:44.793 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_session_open": 8, 2024-04-21T08:51:44.794 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_session_close": 0, 2024-04-21T08:51:44.794 INFO:teuthology.orchestra.run.smithi088.stdout: "osd_laggy": 0, 2024-04-21T08:51:44.794 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_wr": 14, 2024-04-21T08:51:44.794 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_rd": 0, 2024-04-21T08:51:44.794 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_del": 1 2024-04-21T08:51:44.794 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.794 INFO:teuthology.orchestra.run.smithi088.stdout: "oft": { 2024-04-21T08:51:44.794 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_total_objs": 1, 2024-04-21T08:51:44.794 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_total_kv_pairs": 0, 2024-04-21T08:51:44.794 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_total_updates": 0, 2024-04-21T08:51:44.794 INFO:teuthology.orchestra.run.smithi088.stdout: "omap_total_removes": 0 2024-04-21T08:51:44.794 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.794 INFO:teuthology.orchestra.run.smithi088.stdout: "osd-slow-ops": { 2024-04-21T08:51:44.794 INFO:teuthology.orchestra.run.smithi088.stdout: "slow_ops_count": 18 2024-04-21T08:51:44.795 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.795 INFO:teuthology.orchestra.run.smithi088.stdout: "purge_queue": { 2024-04-21T08:51:44.795 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executing_ops": 0, 2024-04-21T08:51:44.795 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executing_ops_high_water": 0, 2024-04-21T08:51:44.795 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executing": 0, 2024-04-21T08:51:44.795 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executing_high_water": 0, 2024-04-21T08:51:44.795 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executed_ops": 0, 2024-04-21T08:51:44.795 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_executed": 0, 2024-04-21T08:51:44.795 INFO:teuthology.orchestra.run.smithi088.stdout: "pq_item_in_journal": 0 2024-04-21T08:51:44.795 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.795 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T08:51:44.795 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 0, 2024-04-21T08:51:44.795 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 104857600, 2024-04-21T08:51:44.795 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-21T08:51:44.796 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 2059, 2024-04-21T08:51:44.796 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 1950949, 2024-04-21T08:51:44.796 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:44.796 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 2059, 2024-04-21T08:51:44.796 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 0, 2024-04-21T08:51:44.796 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 0, 2024-04-21T08:51:44.796 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 2059, 2024-04-21T08:51:44.796 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 1950949, 2024-04-21T08:51:44.796 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-21T08:51:44.796 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:44.796 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:44.796 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:44.796 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-21T08:51:44.797 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.797 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-objecter_bytes": { 2024-04-21T08:51:44.797 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 0, 2024-04-21T08:51:44.797 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 104857600, 2024-04-21T08:51:44.797 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-21T08:51:44.797 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 0, 2024-04-21T08:51:44.797 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 0, 2024-04-21T08:51:44.797 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:44.797 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:44.797 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 38, 2024-04-21T08:51:44.797 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 11332, 2024-04-21T08:51:44.797 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 38, 2024-04-21T08:51:44.797 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 11332, 2024-04-21T08:51:44.797 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-21T08:51:44.798 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:44.798 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:44.798 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:44.798 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-21T08:51:44.798 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.798 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-objecter_ops": { 2024-04-21T08:51:44.798 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 0, 2024-04-21T08:51:44.798 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 1024, 2024-04-21T08:51:44.798 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-21T08:51:44.798 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 0, 2024-04-21T08:51:44.798 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 0, 2024-04-21T08:51:44.798 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:44.798 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:44.798 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 38, 2024-04-21T08:51:44.799 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 38, 2024-04-21T08:51:44.799 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 38, 2024-04-21T08:51:44.799 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 38, 2024-04-21T08:51:44.799 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-21T08:51:44.799 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:44.799 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:44.799 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:44.799 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-21T08:51:44.799 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.799 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-write_buf_throttle": { 2024-04-21T08:51:44.799 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 0, 2024-04-21T08:51:44.799 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 3758096384, 2024-04-21T08:51:44.799 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-21T08:51:44.799 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 0, 2024-04-21T08:51:44.800 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 0, 2024-04-21T08:51:44.800 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:44.800 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:44.800 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 0, 2024-04-21T08:51:44.800 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 0, 2024-04-21T08:51:44.800 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 0, 2024-04-21T08:51:44.800 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 0, 2024-04-21T08:51:44.800 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-21T08:51:44.800 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:44.800 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:44.800 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:44.800 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-21T08:51:44.800 INFO:teuthology.orchestra.run.smithi088.stdout: }, 2024-04-21T08:51:44.800 INFO:teuthology.orchestra.run.smithi088.stdout: "throttle-write_buf_throttle-0x55af3e5c2500": { 2024-04-21T08:51:44.801 INFO:teuthology.orchestra.run.smithi088.stdout: "val": 0, 2024-04-21T08:51:44.801 INFO:teuthology.orchestra.run.smithi088.stdout: "max": 3758096384, 2024-04-21T08:51:44.801 INFO:teuthology.orchestra.run.smithi088.stdout: "get_started": 0, 2024-04-21T08:51:44.801 INFO:teuthology.orchestra.run.smithi088.stdout: "get": 4, 2024-04-21T08:51:44.801 INFO:teuthology.orchestra.run.smithi088.stdout: "get_sum": 1120, 2024-04-21T08:51:44.801 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:44.801 INFO:teuthology.orchestra.run.smithi088.stdout: "get_or_fail_success": 4, 2024-04-21T08:51:44.801 INFO:teuthology.orchestra.run.smithi088.stdout: "take": 0, 2024-04-21T08:51:44.801 INFO:teuthology.orchestra.run.smithi088.stdout: "take_sum": 0, 2024-04-21T08:51:44.801 INFO:teuthology.orchestra.run.smithi088.stdout: "put": 3, 2024-04-21T08:51:44.801 INFO:teuthology.orchestra.run.smithi088.stdout: "put_sum": 1120, 2024-04-21T08:51:44.801 INFO:teuthology.orchestra.run.smithi088.stdout: "wait": { 2024-04-21T08:51:44.801 INFO:teuthology.orchestra.run.smithi088.stdout: "avgcount": 0, 2024-04-21T08:51:44.802 INFO:teuthology.orchestra.run.smithi088.stdout: "sum": 0.000000000, 2024-04-21T08:51:44.802 INFO:teuthology.orchestra.run.smithi088.stdout: "avgtime": 0.000000000 2024-04-21T08:51:44.802 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-21T08:51:44.802 INFO:teuthology.orchestra.run.smithi088.stdout: } 2024-04-21T08:51:44.802 INFO:teuthology.orchestra.run.smithi088.stdout:} 2024-04-21T08:51:45.568 INFO:tasks.check_counter:Daemon mds.k mds.exported=0 2024-04-21T08:51:45.568 INFO:tasks.check_counter:Daemon mds.k mds.imported=0 2024-04-21T08:51:45.568 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update=0 2024-04-21T08:51:45.568 INFO:tasks.check_counter:Daemon mds.k mds_cache.dir_update_receipt=19 2024-04-21T08:51:45.568 INFO:tasks.check_counter:Daemon mds.k mds.root_rsnaps=6 2024-04-21T08:51:45.568 INFO:tasks.check_counter:Daemon mds.k mds_server.req_mksnap_latency.avgcount=0 2024-04-21T08:51:45.568 INFO:tasks.check_counter:Daemon mds.k mds_server.req_rmsnap_latency.avgcount=0 2024-04-21T08:51:45.568 DEBUG:tasks.check_counter:Getting stats from c 2024-04-21T08:51:45.569 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph daemon mds.c perf dump 2024-04-21T08:51:45.818 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:51:46.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:45 smithi179 ceph-mon[31829]: pgmap v406: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-21T08:51:46.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:45 smithi086 ceph-mon[27939]: pgmap v406: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-21T08:51:46.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:45 smithi088 ceph-mon[31259]: pgmap v406: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-21T08:51:47.526 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-04-21T08:51:47.526 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T08:51:47.526 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 1799, 2024-04-21T08:51:47.526 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 1799, 2024-04-21T08:51:47.527 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 8923175, 2024-04-21T08:51:47.527 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 571268, 2024-04-21T08:51:47.527 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 6, 2024-04-21T08:51:47.527 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 2, 2024-04-21T08:51:47.527 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.199081703, 2024-04-21T08:51:47.527 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.057236317, 2024-04-21T08:51:47.527 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.061347665, 2024-04-21T08:51:47.527 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.063108927, 2024-04-21T08:51:47.527 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:47.527 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1798, 2024-04-21T08:51:47.527 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.077012592, 2024-04-21T08:51:47.527 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000042832 2024-04-21T08:51:47.528 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.528 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:47.528 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.528 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.528 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.528 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.528 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-21T08:51:47.528 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 448 2024-04-21T08:51:47.528 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.528 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T08:51:47.528 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 666, 2024-04-21T08:51:47.528 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 857, 2024-04-21T08:51:47.528 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 8556080, 2024-04-21T08:51:47.528 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 709090, 2024-04-21T08:51:47.528 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 7, 2024-04-21T08:51:47.529 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 5, 2024-04-21T08:51:47.529 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.076637932, 2024-04-21T08:51:47.529 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.027107651, 2024-04-21T08:51:47.529 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.023689599, 2024-04-21T08:51:47.529 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.019012125, 2024-04-21T08:51:47.529 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:47.529 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 854, 2024-04-21T08:51:47.529 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.039319887, 2024-04-21T08:51:47.529 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000046042 2024-04-21T08:51:47.529 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.529 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:47.529 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.529 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.529 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.530 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.530 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-21T08:51:47.530 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 497872 2024-04-21T08:51:47.530 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.530 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T08:51:47.530 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 1185, 2024-04-21T08:51:47.530 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 1168, 2024-04-21T08:51:47.530 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 8746144, 2024-04-21T08:51:47.530 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 443277, 2024-04-21T08:51:47.530 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 4, 2024-04-21T08:51:47.530 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 3, 2024-04-21T08:51:47.530 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.139014355, 2024-04-21T08:51:47.530 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.045708603, 2024-04-21T08:51:47.530 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.061454136, 2024-04-21T08:51:47.531 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.033014826, 2024-04-21T08:51:47.531 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:47.531 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1167, 2024-04-21T08:51:47.531 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.047680449, 2024-04-21T08:51:47.531 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000040857 2024-04-21T08:51:47.531 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.531 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:47.531 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.531 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.531 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.531 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.531 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 1665856, 2024-04-21T08:51:47.531 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 160256 2024-04-21T08:51:47.531 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.531 INFO:teuthology.orchestra.run.smithi179.stdout: "cct": { 2024-04-21T08:51:47.532 INFO:teuthology.orchestra.run.smithi179.stdout: "total_workers": 1, 2024-04-21T08:51:47.532 INFO:teuthology.orchestra.run.smithi179.stdout: "unhealthy_workers": 0 2024-04-21T08:51:47.532 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.532 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-MDSRank": { 2024-04-21T08:51:47.532 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-04-21T08:51:47.532 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-04-21T08:51:47.532 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 5704, 2024-04-21T08:51:47.532 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.162762933, 2024-04-21T08:51:47.532 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000028534 2024-04-21T08:51:47.532 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-04-21T08:51:47.532 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.532 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-PurgeQueue": { 2024-04-21T08:51:47.532 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-04-21T08:51:47.532 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-04-21T08:51:47.532 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.533 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.533 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.533 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-04-21T08:51:47.533 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.533 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": { 2024-04-21T08:51:47.533 INFO:teuthology.orchestra.run.smithi179.stdout: "request": 0, 2024-04-21T08:51:47.533 INFO:teuthology.orchestra.run.smithi179.stdout: "reply": 0, 2024-04-21T08:51:47.533 INFO:teuthology.orchestra.run.smithi179.stdout: "reply_latency": { 2024-04-21T08:51:47.533 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.533 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.533 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.533 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.533 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_reply": 0, 2024-04-21T08:51:47.533 INFO:teuthology.orchestra.run.smithi179.stdout: "forward": 0, 2024-04-21T08:51:47.534 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_complete": 0, 2024-04-21T08:51:47.534 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_keys": 0, 2024-04-21T08:51:47.534 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_commit": 0, 2024-04-21T08:51:47.534 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_split": 0, 2024-04-21T08:51:47.534 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_merge": 0, 2024-04-21T08:51:47.534 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes": 33, 2024-04-21T08:51:47.534 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_top": 13, 2024-04-21T08:51:47.534 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_bottom": 20, 2024-04-21T08:51:47.534 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pin_tail": 0, 2024-04-21T08:51:47.534 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pinned": 15, 2024-04-21T08:51:47.534 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_expired": 0, 2024-04-21T08:51:47.534 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_with_caps": 0, 2024-04-21T08:51:47.534 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": 0, 2024-04-21T08:51:47.534 INFO:teuthology.orchestra.run.smithi179.stdout: "subtrees": 2, 2024-04-21T08:51:47.535 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse": 0, 2024-04-21T08:51:47.535 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_hit": 0, 2024-04-21T08:51:47.535 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_forward": 0, 2024-04-21T08:51:47.535 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_discover": 0, 2024-04-21T08:51:47.535 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_dir_fetch": 0, 2024-04-21T08:51:47.535 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_remote_ino": 0, 2024-04-21T08:51:47.535 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_lock": 0, 2024-04-21T08:51:47.535 INFO:teuthology.orchestra.run.smithi179.stdout: "load_cent": 0, 2024-04-21T08:51:47.535 INFO:teuthology.orchestra.run.smithi179.stdout: "q": 0, 2024-04-21T08:51:47.535 INFO:teuthology.orchestra.run.smithi179.stdout: "exported": 0, 2024-04-21T08:51:47.535 INFO:teuthology.orchestra.run.smithi179.stdout: "exported_inodes": 0, 2024-04-21T08:51:47.535 INFO:teuthology.orchestra.run.smithi179.stdout: "imported": 0, 2024-04-21T08:51:47.535 INFO:teuthology.orchestra.run.smithi179.stdout: "imported_inodes": 0, 2024-04-21T08:51:47.535 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_dir_fetch": 0, 2024-04-21T08:51:47.535 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_backtrace_fetch": 0, 2024-04-21T08:51:47.536 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_peer_discover": 0, 2024-04-21T08:51:47.536 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rfiles": 4, 2024-04-21T08:51:47.536 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rbytes": 234, 2024-04-21T08:51:47.536 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rsnaps": 6, 2024-04-21T08:51:47.536 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_fetch": 0, 2024-04-21T08:51:47.536 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_set_tag": 0, 2024-04-21T08:51:47.536 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T08:51:47.536 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_inotable_repaired": 0, 2024-04-21T08:51:47.536 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_inodes": 0, 2024-04-21T08:51:47.536 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_base_inodes": 0, 2024-04-21T08:51:47.536 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dirfrag_rstats": 0, 2024-04-21T08:51:47.536 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_file_inodes": 0, 2024-04-21T08:51:47.537 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_inode_file_caps": 0, 2024-04-21T08:51:47.537 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_revoke": 0, 2024-04-21T08:51:47.537 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_grant": 0, 2024-04-21T08:51:47.537 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_trunc": 0, 2024-04-21T08:51:47.537 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T08:51:47.537 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T08:51:47.537 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps": 0, 2024-04-21T08:51:47.537 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps_dirty": 0, 2024-04-21T08:51:47.537 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_cap_release": 0, 2024-04-21T08:51:47.537 INFO:teuthology.orchestra.run.smithi179.stdout: "process_request_cap_release": 0 2024-04-21T08:51:47.537 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.537 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_cache": { 2024-04-21T08:51:47.537 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update": 0, 2024-04-21T08:51:47.538 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update_receipt": 0, 2024-04-21T08:51:47.538 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_try_discover": 0, 2024-04-21T08:51:47.538 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_send_discover": 0, 2024-04-21T08:51:47.538 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_handle_discover": 0, 2024-04-21T08:51:47.538 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays": 0, 2024-04-21T08:51:47.538 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_delayed": 0, 2024-04-21T08:51:47.538 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_enqueuing": 0, 2024-04-21T08:51:47.538 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_created": 0, 2024-04-21T08:51:47.538 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_enqueued": 0, 2024-04-21T08:51:47.538 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_reintegrated": 0, 2024-04-21T08:51:47.538 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_migrated": 0, 2024-04-21T08:51:47.538 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_processing": 0, 2024-04-21T08:51:47.538 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_enqueued": 0, 2024-04-21T08:51:47.539 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_prioritized": 0, 2024-04-21T08:51:47.539 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_started": 0, 2024-04-21T08:51:47.539 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_completed": 0, 2024-04-21T08:51:47.539 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_quiesce_path": 0, 2024-04-21T08:51:47.539 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_quiesce_inode": 0, 2024-04-21T08:51:47.539 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_enqueue_scrub": 0, 2024-04-21T08:51:47.539 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_exportdir": 0, 2024-04-21T08:51:47.539 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_flush": 0, 2024-04-21T08:51:47.539 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragmentdir": 0, 2024-04-21T08:51:47.539 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragstats": 0, 2024-04-21T08:51:47.539 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_inodestats": 0 2024-04-21T08:51:47.539 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.539 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_log": { 2024-04-21T08:51:47.539 INFO:teuthology.orchestra.run.smithi179.stdout: "evlrg": 0, 2024-04-21T08:51:47.539 INFO:teuthology.orchestra.run.smithi179.stdout: "evadd": 0, 2024-04-21T08:51:47.540 INFO:teuthology.orchestra.run.smithi179.stdout: "evex": 0, 2024-04-21T08:51:47.540 INFO:teuthology.orchestra.run.smithi179.stdout: "evtrm": 0, 2024-04-21T08:51:47.540 INFO:teuthology.orchestra.run.smithi179.stdout: "ev": 2564, 2024-04-21T08:51:47.540 INFO:teuthology.orchestra.run.smithi179.stdout: "evexg": 0, 2024-04-21T08:51:47.540 INFO:teuthology.orchestra.run.smithi179.stdout: "evexd": 0, 2024-04-21T08:51:47.540 INFO:teuthology.orchestra.run.smithi179.stdout: "segadd": 0, 2024-04-21T08:51:47.540 INFO:teuthology.orchestra.run.smithi179.stdout: "segex": 0, 2024-04-21T08:51:47.540 INFO:teuthology.orchestra.run.smithi179.stdout: "segtrm": 0, 2024-04-21T08:51:47.540 INFO:teuthology.orchestra.run.smithi179.stdout: "seg": 2, 2024-04-21T08:51:47.540 INFO:teuthology.orchestra.run.smithi179.stdout: "segmjr": 57, 2024-04-21T08:51:47.540 INFO:teuthology.orchestra.run.smithi179.stdout: "segexg": 0, 2024-04-21T08:51:47.540 INFO:teuthology.orchestra.run.smithi179.stdout: "segexd": 0, 2024-04-21T08:51:47.540 INFO:teuthology.orchestra.run.smithi179.stdout: "expos": 27209890, 2024-04-21T08:51:47.540 INFO:teuthology.orchestra.run.smithi179.stdout: "wrpos": 27829357, 2024-04-21T08:51:47.540 INFO:teuthology.orchestra.run.smithi179.stdout: "rdpos": 27828758, 2024-04-21T08:51:47.540 INFO:teuthology.orchestra.run.smithi179.stdout: "jlat": { 2024-04-21T08:51:47.541 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.541 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.541 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.541 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.541 INFO:teuthology.orchestra.run.smithi179.stdout: "replayed": 2564 2024-04-21T08:51:47.541 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.541 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_mem": { 2024-04-21T08:51:47.541 INFO:teuthology.orchestra.run.smithi179.stdout: "ino": 28, 2024-04-21T08:51:47.541 INFO:teuthology.orchestra.run.smithi179.stdout: "ino+": 29, 2024-04-21T08:51:47.541 INFO:teuthology.orchestra.run.smithi179.stdout: "ino-": 1, 2024-04-21T08:51:47.541 INFO:teuthology.orchestra.run.smithi179.stdout: "dir": 11, 2024-04-21T08:51:47.541 INFO:teuthology.orchestra.run.smithi179.stdout: "dir+": 11, 2024-04-21T08:51:47.541 INFO:teuthology.orchestra.run.smithi179.stdout: "dir-": 0, 2024-04-21T08:51:47.541 INFO:teuthology.orchestra.run.smithi179.stdout: "dn": 33, 2024-04-21T08:51:47.542 INFO:teuthology.orchestra.run.smithi179.stdout: "dn+": 33, 2024-04-21T08:51:47.542 INFO:teuthology.orchestra.run.smithi179.stdout: "dn-": 0, 2024-04-21T08:51:47.542 INFO:teuthology.orchestra.run.smithi179.stdout: "cap": 0, 2024-04-21T08:51:47.542 INFO:teuthology.orchestra.run.smithi179.stdout: "cap+": 0, 2024-04-21T08:51:47.542 INFO:teuthology.orchestra.run.smithi179.stdout: "cap-": 0, 2024-04-21T08:51:47.542 INFO:teuthology.orchestra.run.smithi179.stdout: "rss": 54208, 2024-04-21T08:51:47.542 INFO:teuthology.orchestra.run.smithi179.stdout: "heap": 215416 2024-04-21T08:51:47.542 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.542 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_server": { 2024-04-21T08:51:47.542 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_client_request": 0, 2024-04-21T08:51:47.542 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_server_request": 0, 2024-04-21T08:51:47.542 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_request": 0, 2024-04-21T08:51:47.542 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_session": 0, 2024-04-21T08:51:47.542 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_peer_request": 0, 2024-04-21T08:51:47.543 INFO:teuthology.orchestra.run.smithi179.stdout: "req_create_latency": { 2024-04-21T08:51:47.543 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.543 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.543 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.543 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.543 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getattr_latency": { 2024-04-21T08:51:47.543 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.543 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.543 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.543 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.543 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getfilelock_latency": { 2024-04-21T08:51:47.543 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.543 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.543 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.543 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.544 INFO:teuthology.orchestra.run.smithi179.stdout: "req_link_latency": { 2024-04-21T08:51:47.544 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.544 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.544 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.544 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.544 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookup_latency": { 2024-04-21T08:51:47.544 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.544 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.544 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.544 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.544 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookuphash_latency": { 2024-04-21T08:51:47.544 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.544 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.544 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.544 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.545 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupino_latency": { 2024-04-21T08:51:47.545 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.545 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.545 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.545 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.545 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupname_latency": { 2024-04-21T08:51:47.545 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.545 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.545 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.545 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.545 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupparent_latency": { 2024-04-21T08:51:47.545 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.545 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.545 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.545 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.545 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupsnap_latency": { 2024-04-21T08:51:47.546 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.546 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.546 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.546 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.546 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lssnap_latency": { 2024-04-21T08:51:47.546 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.546 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.546 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.546 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.546 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mkdir_latency": { 2024-04-21T08:51:47.547 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.547 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.547 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.547 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.547 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mknod_latency": { 2024-04-21T08:51:47.547 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.547 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.547 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.547 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.547 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mksnap_latency": { 2024-04-21T08:51:47.547 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.547 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.548 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.548 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.548 INFO:teuthology.orchestra.run.smithi179.stdout: "req_open_latency": { 2024-04-21T08:51:47.548 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.548 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.548 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.548 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.548 INFO:teuthology.orchestra.run.smithi179.stdout: "req_readdir_latency": { 2024-04-21T08:51:47.548 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.548 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.548 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.548 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.548 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rename_latency": { 2024-04-21T08:51:47.548 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.548 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.548 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.549 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.549 INFO:teuthology.orchestra.run.smithi179.stdout: "req_renamesnap_latency": { 2024-04-21T08:51:47.549 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.549 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.549 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.549 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.549 INFO:teuthology.orchestra.run.smithi179.stdout: "req_snapdiff_latency": { 2024-04-21T08:51:47.549 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.549 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.549 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.549 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.549 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmdir_latency": { 2024-04-21T08:51:47.549 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.549 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.549 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.550 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.550 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmsnap_latency": { 2024-04-21T08:51:47.550 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.550 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.550 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.550 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.550 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmxattr_latency": { 2024-04-21T08:51:47.550 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.550 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.550 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.550 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.550 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setattr_latency": { 2024-04-21T08:51:47.550 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.550 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.551 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.551 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.551 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setdirlayout_latency": { 2024-04-21T08:51:47.551 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.551 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.551 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.551 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.551 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setfilelock_latency": { 2024-04-21T08:51:47.551 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.551 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.551 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.551 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.551 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setlayout_latency": { 2024-04-21T08:51:47.551 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.551 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.551 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.552 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.552 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setxattr_latency": { 2024-04-21T08:51:47.552 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.552 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.552 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.552 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.552 INFO:teuthology.orchestra.run.smithi179.stdout: "req_symlink_latency": { 2024-04-21T08:51:47.552 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.552 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.552 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.552 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.552 INFO:teuthology.orchestra.run.smithi179.stdout: "req_unlink_latency": { 2024-04-21T08:51:47.552 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.552 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.552 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.553 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.553 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_revoke_eviction": 0, 2024-04-21T08:51:47.553 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_acquisition_throttle": 0, 2024-04-21T08:51:47.553 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getvxattr_latency": { 2024-04-21T08:51:47.553 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.553 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.553 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.553 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-04-21T08:51:47.553 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.553 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_sessions": { 2024-04-21T08:51:47.553 INFO:teuthology.orchestra.run.smithi179.stdout: "session_count": 2, 2024-04-21T08:51:47.553 INFO:teuthology.orchestra.run.smithi179.stdout: "session_add": 6, 2024-04-21T08:51:47.553 INFO:teuthology.orchestra.run.smithi179.stdout: "session_remove": 4, 2024-04-21T08:51:47.553 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_open": 4, 2024-04-21T08:51:47.554 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_stale": 0, 2024-04-21T08:51:47.554 INFO:teuthology.orchestra.run.smithi179.stdout: "total_load": 0, 2024-04-21T08:51:47.554 INFO:teuthology.orchestra.run.smithi179.stdout: "average_load": 0, 2024-04-21T08:51:47.554 INFO:teuthology.orchestra.run.smithi179.stdout: "avg_session_uptime": 3075, 2024-04-21T08:51:47.554 INFO:teuthology.orchestra.run.smithi179.stdout: "mdthresh_evicted": 0 2024-04-21T08:51:47.554 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.554 INFO:teuthology.orchestra.run.smithi179.stdout: "mempool": { 2024-04-21T08:51:47.554 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_bytes": 0, 2024-04-21T08:51:47.554 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_items": 0, 2024-04-21T08:51:47.554 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T08:51:47.554 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_items": 0, 2024-04-21T08:51:47.554 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T08:51:47.554 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_items": 0, 2024-04-21T08:51:47.555 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T08:51:47.555 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T08:51:47.555 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T08:51:47.555 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T08:51:47.555 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T08:51:47.555 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_items": 0, 2024-04-21T08:51:47.555 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T08:51:47.555 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T08:51:47.555 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_bytes": 0, 2024-04-21T08:51:47.555 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_items": 0, 2024-04-21T08:51:47.555 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_bytes": 0, 2024-04-21T08:51:47.555 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_items": 0, 2024-04-21T08:51:47.555 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T08:51:47.555 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T08:51:47.556 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T08:51:47.556 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T08:51:47.556 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T08:51:47.556 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_items": 0, 2024-04-21T08:51:47.556 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_bytes": 0, 2024-04-21T08:51:47.556 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_items": 0, 2024-04-21T08:51:47.556 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T08:51:47.556 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T08:51:47.556 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_bytes": 0, 2024-04-21T08:51:47.556 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_items": 0, 2024-04-21T08:51:47.556 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_bytes": 0, 2024-04-21T08:51:47.556 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_items": 0, 2024-04-21T08:51:47.556 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T08:51:47.556 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_items": 0, 2024-04-21T08:51:47.557 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T08:51:47.557 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_items": 0, 2024-04-21T08:51:47.557 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_bytes": 199286, 2024-04-21T08:51:47.557 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_items": 84, 2024-04-21T08:51:47.557 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_bytes": 0, 2024-04-21T08:51:47.557 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_items": 0, 2024-04-21T08:51:47.557 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_bytes": 0, 2024-04-21T08:51:47.557 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_items": 0, 2024-04-21T08:51:47.557 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_bytes": 0, 2024-04-21T08:51:47.557 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_items": 0, 2024-04-21T08:51:47.557 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_bytes": 0, 2024-04-21T08:51:47.557 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_items": 0, 2024-04-21T08:51:47.557 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_bytes": 9512, 2024-04-21T08:51:47.557 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_items": 182, 2024-04-21T08:51:47.557 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T08:51:47.558 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_items": 0, 2024-04-21T08:51:47.558 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_bytes": 0, 2024-04-21T08:51:47.558 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_items": 0, 2024-04-21T08:51:47.558 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_bytes": 140767, 2024-04-21T08:51:47.558 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_items": 1087, 2024-04-21T08:51:47.558 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_bytes": 0, 2024-04-21T08:51:47.558 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_items": 0, 2024-04-21T08:51:47.558 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_bytes": 0, 2024-04-21T08:51:47.558 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_items": 0 2024-04-21T08:51:47.558 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.558 INFO:teuthology.orchestra.run.smithi179.stdout: "objecter": { 2024-04-21T08:51:47.558 INFO:teuthology.orchestra.run.smithi179.stdout: "op_active": 0, 2024-04-21T08:51:47.558 INFO:teuthology.orchestra.run.smithi179.stdout: "op_laggy": 0, 2024-04-21T08:51:47.559 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send": 3349, 2024-04-21T08:51:47.559 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send_bytes": 26, 2024-04-21T08:51:47.559 INFO:teuthology.orchestra.run.smithi179.stdout: "op_resend": 4, 2024-04-21T08:51:47.559 INFO:teuthology.orchestra.run.smithi179.stdout: "op_reply": 3349, 2024-04-21T08:51:47.559 INFO:teuthology.orchestra.run.smithi179.stdout: "op_latency": { 2024-04-21T08:51:47.559 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 3349, 2024-04-21T08:51:47.559 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 1.783962293, 2024-04-21T08:51:47.559 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000532685 2024-04-21T08:51:47.559 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.559 INFO:teuthology.orchestra.run.smithi179.stdout: "op_inflight": 0, 2024-04-21T08:51:47.559 INFO:teuthology.orchestra.run.smithi179.stdout: "oplen_avg": { 2024-04-21T08:51:47.559 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 3349, 2024-04-21T08:51:47.559 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 3350 2024-04-21T08:51:47.559 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.559 INFO:teuthology.orchestra.run.smithi179.stdout: "op": 3349, 2024-04-21T08:51:47.560 INFO:teuthology.orchestra.run.smithi179.stdout: "op_r": 3349, 2024-04-21T08:51:47.560 INFO:teuthology.orchestra.run.smithi179.stdout: "op_w": 0, 2024-04-21T08:51:47.560 INFO:teuthology.orchestra.run.smithi179.stdout: "op_rmw": 0, 2024-04-21T08:51:47.560 INFO:teuthology.orchestra.run.smithi179.stdout: "op_pg": 0, 2024-04-21T08:51:47.560 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_stat": 1798, 2024-04-21T08:51:47.560 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_create": 0, 2024-04-21T08:51:47.560 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_read": 1548, 2024-04-21T08:51:47.560 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_write": 0, 2024-04-21T08:51:47.560 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writefull": 0, 2024-04-21T08:51:47.560 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writesame": 0, 2024-04-21T08:51:47.560 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_append": 0, 2024-04-21T08:51:47.560 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_zero": 0, 2024-04-21T08:51:47.560 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_truncate": 0, 2024-04-21T08:51:47.560 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_delete": 0, 2024-04-21T08:51:47.560 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_mapext": 0, 2024-04-21T08:51:47.561 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_sparse_read": 0, 2024-04-21T08:51:47.561 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_clonerange": 0, 2024-04-21T08:51:47.561 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_getxattr": 2, 2024-04-21T08:51:47.561 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_setxattr": 0, 2024-04-21T08:51:47.561 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_cmpxattr": 0, 2024-04-21T08:51:47.561 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_rmxattr": 0, 2024-04-21T08:51:47.561 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_resetxattrs": 0, 2024-04-21T08:51:47.561 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_call": 0, 2024-04-21T08:51:47.561 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_watch": 0, 2024-04-21T08:51:47.561 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_notify": 0, 2024-04-21T08:51:47.561 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T08:51:47.561 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls": 0, 2024-04-21T08:51:47.561 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls_filter": 0, 2024-04-21T08:51:47.561 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_other": 0, 2024-04-21T08:51:47.561 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_active": 0, 2024-04-21T08:51:47.562 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_send": 0, 2024-04-21T08:51:47.562 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_resend": 0, 2024-04-21T08:51:47.562 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_ping": 0, 2024-04-21T08:51:47.562 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_active": 0, 2024-04-21T08:51:47.562 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_send": 0, 2024-04-21T08:51:47.562 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_resend": 0, 2024-04-21T08:51:47.562 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_active": 0, 2024-04-21T08:51:47.562 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_send": 0, 2024-04-21T08:51:47.562 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_resend": 0, 2024-04-21T08:51:47.562 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_active": 0, 2024-04-21T08:51:47.562 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_send": 0, 2024-04-21T08:51:47.562 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_resend": 0, 2024-04-21T08:51:47.562 INFO:teuthology.orchestra.run.smithi179.stdout: "command_active": 0, 2024-04-21T08:51:47.562 INFO:teuthology.orchestra.run.smithi179.stdout: "command_send": 0, 2024-04-21T08:51:47.563 INFO:teuthology.orchestra.run.smithi179.stdout: "command_resend": 0, 2024-04-21T08:51:47.563 INFO:teuthology.orchestra.run.smithi179.stdout: "map_epoch": 108, 2024-04-21T08:51:47.563 INFO:teuthology.orchestra.run.smithi179.stdout: "map_full": 0, 2024-04-21T08:51:47.563 INFO:teuthology.orchestra.run.smithi179.stdout: "map_inc": 19, 2024-04-21T08:51:47.563 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_sessions": 6, 2024-04-21T08:51:47.563 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_open": 6, 2024-04-21T08:51:47.563 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_close": 0, 2024-04-21T08:51:47.563 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_laggy": 0, 2024-04-21T08:51:47.563 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_wr": 0, 2024-04-21T08:51:47.563 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_rd": 2, 2024-04-21T08:51:47.563 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_del": 0 2024-04-21T08:51:47.563 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.563 INFO:teuthology.orchestra.run.smithi179.stdout: "oft": { 2024-04-21T08:51:47.563 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_objs": 0, 2024-04-21T08:51:47.563 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_kv_pairs": 0, 2024-04-21T08:51:47.564 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_updates": 0, 2024-04-21T08:51:47.564 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_removes": 0 2024-04-21T08:51:47.564 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.564 INFO:teuthology.orchestra.run.smithi179.stdout: "osd-slow-ops": { 2024-04-21T08:51:47.564 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_ops_count": 0 2024-04-21T08:51:47.564 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.564 INFO:teuthology.orchestra.run.smithi179.stdout: "purge_queue": { 2024-04-21T08:51:47.564 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops": 0, 2024-04-21T08:51:47.564 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops_high_water": 0, 2024-04-21T08:51:47.564 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing": 0, 2024-04-21T08:51:47.564 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_high_water": 0, 2024-04-21T08:51:47.564 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed_ops": 0, 2024-04-21T08:51:47.564 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed": 0, 2024-04-21T08:51:47.564 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_item_in_journal": 0 2024-04-21T08:51:47.564 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.565 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T08:51:47.565 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-04-21T08:51:47.565 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-04-21T08:51:47.565 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-04-21T08:51:47.565 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 3646, 2024-04-21T08:51:47.565 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 25879194, 2024-04-21T08:51:47.565 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:47.565 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 3646, 2024-04-21T08:51:47.565 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-04-21T08:51:47.565 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-04-21T08:51:47.565 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 3646, 2024-04-21T08:51:47.565 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 25879194, 2024-04-21T08:51:47.565 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-04-21T08:51:47.565 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.565 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.566 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.566 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-04-21T08:51:47.566 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.566 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_bytes": { 2024-04-21T08:51:47.566 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-04-21T08:51:47.566 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-04-21T08:51:47.566 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-04-21T08:51:47.566 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-04-21T08:51:47.566 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-04-21T08:51:47.566 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:47.566 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:47.566 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 3349, 2024-04-21T08:51:47.566 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 23635063, 2024-04-21T08:51:47.566 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 3349, 2024-04-21T08:51:47.567 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 23635063, 2024-04-21T08:51:47.567 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-04-21T08:51:47.567 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.567 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.567 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.567 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-04-21T08:51:47.567 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.567 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_ops": { 2024-04-21T08:51:47.567 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-04-21T08:51:47.567 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 1024, 2024-04-21T08:51:47.567 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-04-21T08:51:47.567 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-04-21T08:51:47.567 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-04-21T08:51:47.567 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:47.568 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:47.568 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 3349, 2024-04-21T08:51:47.568 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 3349, 2024-04-21T08:51:47.568 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 3349, 2024-04-21T08:51:47.568 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 3349, 2024-04-21T08:51:47.568 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-04-21T08:51:47.568 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.568 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.568 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.568 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-04-21T08:51:47.568 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.568 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle": { 2024-04-21T08:51:47.568 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-04-21T08:51:47.568 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-04-21T08:51:47.569 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-04-21T08:51:47.569 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-04-21T08:51:47.569 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-04-21T08:51:47.569 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:47.569 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:47.569 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-04-21T08:51:47.569 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-04-21T08:51:47.569 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 0, 2024-04-21T08:51:47.569 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 0, 2024-04-21T08:51:47.569 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-04-21T08:51:47.569 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.569 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.569 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.569 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-04-21T08:51:47.570 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:47.570 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle-0x55f0ffb1c6e0": { 2024-04-21T08:51:47.570 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-04-21T08:51:47.570 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-04-21T08:51:47.570 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-04-21T08:51:47.570 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-04-21T08:51:47.570 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-04-21T08:51:47.570 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:47.570 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:47.570 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-04-21T08:51:47.570 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-04-21T08:51:47.570 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 0, 2024-04-21T08:51:47.570 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 0, 2024-04-21T08:51:47.570 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-04-21T08:51:47.571 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:47.571 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:47.571 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:47.571 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-04-21T08:51:47.571 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-04-21T08:51:47.571 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-04-21T08:51:48.198 INFO:tasks.check_counter:Daemon mds.c mds.exported=0 2024-04-21T08:51:48.199 INFO:tasks.check_counter:Daemon mds.c mds.imported=0 2024-04-21T08:51:48.199 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update=0 2024-04-21T08:51:48.199 INFO:tasks.check_counter:Daemon mds.c mds_cache.dir_update_receipt=0 2024-04-21T08:51:48.199 INFO:tasks.check_counter:Daemon mds.c mds.root_rsnaps=6 2024-04-21T08:51:48.199 INFO:tasks.check_counter:Daemon mds.c mds_server.req_mksnap_latency.avgcount=0 2024-04-21T08:51:48.199 INFO:tasks.check_counter:Daemon mds.c mds_server.req_rmsnap_latency.avgcount=0 2024-04-21T08:51:48.199 DEBUG:tasks.check_counter:Getting stats from f 2024-04-21T08:51:48.199 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph daemon mds.f perf dump 2024-04-21T08:51:48.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:47 smithi179 ceph-mon[31829]: pgmap v407: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-21T08:51:48.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:47 smithi086 ceph-mon[27939]: pgmap v407: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-21T08:51:48.450 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:51:48.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:47 smithi088 ceph-mon[31259]: pgmap v407: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 4.7 KiB/s rd, 767 B/s wr, 9 op/s 2024-04-21T08:51:50.193 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:49 smithi179 ceph-mon[31829]: pgmap v408: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 767 B/s wr, 13 op/s 2024-04-21T08:51:50.193 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-04-21T08:51:50.193 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T08:51:50.193 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 1204, 2024-04-21T08:51:50.193 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 1185, 2024-04-21T08:51:50.193 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 1972976, 2024-04-21T08:51:50.193 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 449296, 2024-04-21T08:51:50.193 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 5, 2024-04-21T08:51:50.194 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 1, 2024-04-21T08:51:50.194 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.171514973, 2024-04-21T08:51:50.194 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.052723916, 2024-04-21T08:51:50.194 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.068770244, 2024-04-21T08:51:50.194 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.045913732, 2024-04-21T08:51:50.194 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:50.194 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1185, 2024-04-21T08:51:50.194 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.069995990, 2024-04-21T08:51:50.194 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000059068 2024-04-21T08:51:50.194 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.194 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:50.194 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.194 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.195 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.195 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.195 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 1669232, 2024-04-21T08:51:50.195 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 161824 2024-04-21T08:51:50.195 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.195 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T08:51:50.195 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 4, 2024-04-21T08:51:50.195 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 197, 2024-04-21T08:51:50.195 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 876, 2024-04-21T08:51:50.195 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 500538, 2024-04-21T08:51:50.195 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 6, 2024-04-21T08:51:50.195 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 4, 2024-04-21T08:51:50.195 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.014861162, 2024-04-21T08:51:50.196 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.008399450, 2024-04-21T08:51:50.196 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.001957204, 2024-04-21T08:51:50.196 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.000092931, 2024-04-21T08:51:50.196 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:50.196 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 194, 2024-04-21T08:51:50.196 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.015350155, 2024-04-21T08:51:50.196 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000079124 2024-04-21T08:51:50.196 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.196 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:50.196 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.196 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.196 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.196 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.197 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 320, 2024-04-21T08:51:50.197 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 499840 2024-04-21T08:51:50.197 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.197 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T08:51:50.197 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 1747, 2024-04-21T08:51:50.197 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 1749, 2024-04-21T08:51:50.197 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 446299, 2024-04-21T08:51:50.197 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 555553, 2024-04-21T08:51:50.197 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 5, 2024-04-21T08:51:50.197 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 3, 2024-04-21T08:51:50.197 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.123916093, 2024-04-21T08:51:50.197 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.041028451, 2024-04-21T08:51:50.197 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.030323123, 2024-04-21T08:51:50.197 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.043473582, 2024-04-21T08:51:50.198 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:50.198 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1747, 2024-04-21T08:51:50.198 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.070880925, 2024-04-21T08:51:50.198 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000040572 2024-04-21T08:51:50.198 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.198 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:50.198 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.198 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.198 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.198 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.198 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-21T08:51:50.198 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 0 2024-04-21T08:51:50.198 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.198 INFO:teuthology.orchestra.run.smithi179.stdout: "cct": { 2024-04-21T08:51:50.199 INFO:teuthology.orchestra.run.smithi179.stdout: "total_workers": 1, 2024-04-21T08:51:50.199 INFO:teuthology.orchestra.run.smithi179.stdout: "unhealthy_workers": 0 2024-04-21T08:51:50.199 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.199 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-MDSRank": { 2024-04-21T08:51:50.199 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-04-21T08:51:50.199 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-04-21T08:51:50.199 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 4462, 2024-04-21T08:51:50.199 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.132080824, 2024-04-21T08:51:50.199 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000029601 2024-04-21T08:51:50.199 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-04-21T08:51:50.199 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.199 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-PurgeQueue": { 2024-04-21T08:51:50.199 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-04-21T08:51:50.199 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-04-21T08:51:50.200 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.200 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.200 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.200 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-04-21T08:51:50.200 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.200 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": { 2024-04-21T08:51:50.200 INFO:teuthology.orchestra.run.smithi179.stdout: "request": 0, 2024-04-21T08:51:50.200 INFO:teuthology.orchestra.run.smithi179.stdout: "reply": 0, 2024-04-21T08:51:50.200 INFO:teuthology.orchestra.run.smithi179.stdout: "reply_latency": { 2024-04-21T08:51:50.200 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.200 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.200 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.200 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.201 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_reply": 0, 2024-04-21T08:51:50.201 INFO:teuthology.orchestra.run.smithi179.stdout: "forward": 0, 2024-04-21T08:51:50.201 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_complete": 0, 2024-04-21T08:51:50.201 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_keys": 0, 2024-04-21T08:51:50.201 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_commit": 0, 2024-04-21T08:51:50.201 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_split": 0, 2024-04-21T08:51:50.201 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_merge": 0, 2024-04-21T08:51:50.201 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes": 0, 2024-04-21T08:51:50.201 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_top": 0, 2024-04-21T08:51:50.201 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_bottom": 0, 2024-04-21T08:51:50.201 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pin_tail": 0, 2024-04-21T08:51:50.201 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pinned": 0, 2024-04-21T08:51:50.201 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_expired": 0, 2024-04-21T08:51:50.201 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_with_caps": 0, 2024-04-21T08:51:50.201 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": 0, 2024-04-21T08:51:50.202 INFO:teuthology.orchestra.run.smithi179.stdout: "subtrees": 1, 2024-04-21T08:51:50.202 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse": 0, 2024-04-21T08:51:50.202 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_hit": 0, 2024-04-21T08:51:50.202 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_forward": 0, 2024-04-21T08:51:50.202 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_discover": 0, 2024-04-21T08:51:50.202 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_dir_fetch": 0, 2024-04-21T08:51:50.202 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_remote_ino": 0, 2024-04-21T08:51:50.202 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_lock": 0, 2024-04-21T08:51:50.202 INFO:teuthology.orchestra.run.smithi179.stdout: "load_cent": 0, 2024-04-21T08:51:50.202 INFO:teuthology.orchestra.run.smithi179.stdout: "q": 0, 2024-04-21T08:51:50.202 INFO:teuthology.orchestra.run.smithi179.stdout: "exported": 0, 2024-04-21T08:51:50.202 INFO:teuthology.orchestra.run.smithi179.stdout: "exported_inodes": 0, 2024-04-21T08:51:50.202 INFO:teuthology.orchestra.run.smithi179.stdout: "imported": 0, 2024-04-21T08:51:50.202 INFO:teuthology.orchestra.run.smithi179.stdout: "imported_inodes": 0, 2024-04-21T08:51:50.203 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_dir_fetch": 0, 2024-04-21T08:51:50.203 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_backtrace_fetch": 0, 2024-04-21T08:51:50.203 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_peer_discover": 0, 2024-04-21T08:51:50.203 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rfiles": 0, 2024-04-21T08:51:50.203 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rbytes": 0, 2024-04-21T08:51:50.203 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rsnaps": 0, 2024-04-21T08:51:50.203 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_fetch": 0, 2024-04-21T08:51:50.203 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_set_tag": 0, 2024-04-21T08:51:50.203 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T08:51:50.203 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_inotable_repaired": 0, 2024-04-21T08:51:50.203 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_inodes": 0, 2024-04-21T08:51:50.203 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_base_inodes": 0, 2024-04-21T08:51:50.203 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dirfrag_rstats": 0, 2024-04-21T08:51:50.204 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_file_inodes": 0, 2024-04-21T08:51:50.204 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_inode_file_caps": 0, 2024-04-21T08:51:50.204 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_revoke": 0, 2024-04-21T08:51:50.204 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_grant": 0, 2024-04-21T08:51:50.204 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_trunc": 0, 2024-04-21T08:51:50.204 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T08:51:50.204 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T08:51:50.204 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps": 0, 2024-04-21T08:51:50.204 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps_dirty": 0, 2024-04-21T08:51:50.204 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_cap_release": 0, 2024-04-21T08:51:50.204 INFO:teuthology.orchestra.run.smithi179.stdout: "process_request_cap_release": 0 2024-04-21T08:51:50.204 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.204 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_cache": { 2024-04-21T08:51:50.204 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update": 0, 2024-04-21T08:51:50.205 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update_receipt": 0, 2024-04-21T08:51:50.205 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_try_discover": 0, 2024-04-21T08:51:50.205 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_send_discover": 0, 2024-04-21T08:51:50.205 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_handle_discover": 0, 2024-04-21T08:51:50.205 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays": 0, 2024-04-21T08:51:50.205 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_delayed": 0, 2024-04-21T08:51:50.205 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_enqueuing": 0, 2024-04-21T08:51:50.205 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_created": 0, 2024-04-21T08:51:50.205 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_enqueued": 0, 2024-04-21T08:51:50.205 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_reintegrated": 0, 2024-04-21T08:51:50.205 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_migrated": 0, 2024-04-21T08:51:50.205 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_processing": 0, 2024-04-21T08:51:50.205 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_enqueued": 0, 2024-04-21T08:51:50.205 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_prioritized": 0, 2024-04-21T08:51:50.206 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_started": 0, 2024-04-21T08:51:50.206 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_completed": 0, 2024-04-21T08:51:50.206 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_quiesce_path": 0, 2024-04-21T08:51:50.206 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_quiesce_inode": 0, 2024-04-21T08:51:50.206 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_enqueue_scrub": 0, 2024-04-21T08:51:50.206 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_exportdir": 0, 2024-04-21T08:51:50.206 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_flush": 0, 2024-04-21T08:51:50.206 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragmentdir": 0, 2024-04-21T08:51:50.206 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragstats": 0, 2024-04-21T08:51:50.206 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_inodestats": 0 2024-04-21T08:51:50.206 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.206 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_log": { 2024-04-21T08:51:50.206 INFO:teuthology.orchestra.run.smithi179.stdout: "evlrg": 0, 2024-04-21T08:51:50.207 INFO:teuthology.orchestra.run.smithi179.stdout: "evadd": 0, 2024-04-21T08:51:50.207 INFO:teuthology.orchestra.run.smithi179.stdout: "evex": 0, 2024-04-21T08:51:50.207 INFO:teuthology.orchestra.run.smithi179.stdout: "evtrm": 0, 2024-04-21T08:51:50.207 INFO:teuthology.orchestra.run.smithi179.stdout: "ev": 5, 2024-04-21T08:51:50.207 INFO:teuthology.orchestra.run.smithi179.stdout: "evexg": 0, 2024-04-21T08:51:50.207 INFO:teuthology.orchestra.run.smithi179.stdout: "evexd": 0, 2024-04-21T08:51:50.207 INFO:teuthology.orchestra.run.smithi179.stdout: "segadd": 0, 2024-04-21T08:51:50.207 INFO:teuthology.orchestra.run.smithi179.stdout: "segex": 0, 2024-04-21T08:51:50.207 INFO:teuthology.orchestra.run.smithi179.stdout: "segtrm": 0, 2024-04-21T08:51:50.207 INFO:teuthology.orchestra.run.smithi179.stdout: "seg": 2, 2024-04-21T08:51:50.207 INFO:teuthology.orchestra.run.smithi179.stdout: "segmjr": 2, 2024-04-21T08:51:50.207 INFO:teuthology.orchestra.run.smithi179.stdout: "segexg": 0, 2024-04-21T08:51:50.207 INFO:teuthology.orchestra.run.smithi179.stdout: "segexd": 0, 2024-04-21T08:51:50.207 INFO:teuthology.orchestra.run.smithi179.stdout: "expos": 4194304, 2024-04-21T08:51:50.208 INFO:teuthology.orchestra.run.smithi179.stdout: "wrpos": 4195732, 2024-04-21T08:51:50.208 INFO:teuthology.orchestra.run.smithi179.stdout: "rdpos": 4195424, 2024-04-21T08:51:50.208 INFO:teuthology.orchestra.run.smithi179.stdout: "jlat": { 2024-04-21T08:51:50.208 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.208 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.208 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.208 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.208 INFO:teuthology.orchestra.run.smithi179.stdout: "replayed": 5 2024-04-21T08:51:50.208 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.208 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_mem": { 2024-04-21T08:51:50.208 INFO:teuthology.orchestra.run.smithi179.stdout: "ino": 3, 2024-04-21T08:51:50.208 INFO:teuthology.orchestra.run.smithi179.stdout: "ino+": 3, 2024-04-21T08:51:50.208 INFO:teuthology.orchestra.run.smithi179.stdout: "ino-": 0, 2024-04-21T08:51:50.208 INFO:teuthology.orchestra.run.smithi179.stdout: "dir": 1, 2024-04-21T08:51:50.209 INFO:teuthology.orchestra.run.smithi179.stdout: "dir+": 1, 2024-04-21T08:51:50.209 INFO:teuthology.orchestra.run.smithi179.stdout: "dir-": 0, 2024-04-21T08:51:50.209 INFO:teuthology.orchestra.run.smithi179.stdout: "dn": 0, 2024-04-21T08:51:50.209 INFO:teuthology.orchestra.run.smithi179.stdout: "dn+": 0, 2024-04-21T08:51:50.209 INFO:teuthology.orchestra.run.smithi179.stdout: "dn-": 0, 2024-04-21T08:51:50.209 INFO:teuthology.orchestra.run.smithi179.stdout: "cap": 0, 2024-04-21T08:51:50.209 INFO:teuthology.orchestra.run.smithi179.stdout: "cap+": 0, 2024-04-21T08:51:50.209 INFO:teuthology.orchestra.run.smithi179.stdout: "cap-": 0, 2024-04-21T08:51:50.209 INFO:teuthology.orchestra.run.smithi179.stdout: "rss": 53824, 2024-04-21T08:51:50.209 INFO:teuthology.orchestra.run.smithi179.stdout: "heap": 215416 2024-04-21T08:51:50.209 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.209 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_server": { 2024-04-21T08:51:50.209 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_client_request": 0, 2024-04-21T08:51:50.209 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_server_request": 0, 2024-04-21T08:51:50.210 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_request": 0, 2024-04-21T08:51:50.210 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_session": 0, 2024-04-21T08:51:50.210 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_peer_request": 0, 2024-04-21T08:51:50.210 INFO:teuthology.orchestra.run.smithi179.stdout: "req_create_latency": { 2024-04-21T08:51:50.210 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.210 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.210 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.210 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.210 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getattr_latency": { 2024-04-21T08:51:50.210 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.210 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.210 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.210 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.210 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getfilelock_latency": { 2024-04-21T08:51:50.211 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.211 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.211 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.211 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.211 INFO:teuthology.orchestra.run.smithi179.stdout: "req_link_latency": { 2024-04-21T08:51:50.211 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.211 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.211 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.211 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.211 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookup_latency": { 2024-04-21T08:51:50.211 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.211 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.211 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.211 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.211 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookuphash_latency": { 2024-04-21T08:51:50.212 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.212 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.212 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.212 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.212 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupino_latency": { 2024-04-21T08:51:50.212 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.212 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.212 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.212 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.212 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupname_latency": { 2024-04-21T08:51:50.212 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.212 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.212 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.212 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.213 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupparent_latency": { 2024-04-21T08:51:50.213 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.213 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.213 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.213 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.213 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupsnap_latency": { 2024-04-21T08:51:50.213 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.213 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.213 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.213 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.213 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lssnap_latency": { 2024-04-21T08:51:50.213 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.213 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.213 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.214 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.214 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mkdir_latency": { 2024-04-21T08:51:50.214 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.214 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.214 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.215 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.215 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mknod_latency": { 2024-04-21T08:51:50.215 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.215 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.215 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.215 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.215 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mksnap_latency": { 2024-04-21T08:51:50.215 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.215 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.215 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.215 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.215 INFO:teuthology.orchestra.run.smithi179.stdout: "req_open_latency": { 2024-04-21T08:51:50.215 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.215 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.216 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.216 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.216 INFO:teuthology.orchestra.run.smithi179.stdout: "req_readdir_latency": { 2024-04-21T08:51:50.216 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.216 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.216 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.216 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.216 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rename_latency": { 2024-04-21T08:51:50.216 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.216 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.216 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.216 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.216 INFO:teuthology.orchestra.run.smithi179.stdout: "req_renamesnap_latency": { 2024-04-21T08:51:50.217 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.217 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.217 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.217 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.217 INFO:teuthology.orchestra.run.smithi179.stdout: "req_snapdiff_latency": { 2024-04-21T08:51:50.217 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.217 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.217 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.217 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.217 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmdir_latency": { 2024-04-21T08:51:50.217 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.217 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.217 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.217 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.218 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmsnap_latency": { 2024-04-21T08:51:50.218 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.218 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.218 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.218 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.218 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmxattr_latency": { 2024-04-21T08:51:50.218 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.218 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.218 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.218 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.218 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setattr_latency": { 2024-04-21T08:51:50.218 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.218 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.218 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.219 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.219 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setdirlayout_latency": { 2024-04-21T08:51:50.219 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.219 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.219 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.219 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.219 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setfilelock_latency": { 2024-04-21T08:51:50.219 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.219 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.219 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.219 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.219 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setlayout_latency": { 2024-04-21T08:51:50.219 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.219 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.220 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.220 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.220 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setxattr_latency": { 2024-04-21T08:51:50.220 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.220 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.220 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.220 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.220 INFO:teuthology.orchestra.run.smithi179.stdout: "req_symlink_latency": { 2024-04-21T08:51:50.220 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.220 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.220 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.220 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.220 INFO:teuthology.orchestra.run.smithi179.stdout: "req_unlink_latency": { 2024-04-21T08:51:50.221 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.221 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.221 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.221 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.221 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_revoke_eviction": 0, 2024-04-21T08:51:50.221 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_acquisition_throttle": 0, 2024-04-21T08:51:50.221 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getvxattr_latency": { 2024-04-21T08:51:50.221 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.221 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.221 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.221 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-04-21T08:51:50.221 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.221 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_sessions": { 2024-04-21T08:51:50.221 INFO:teuthology.orchestra.run.smithi179.stdout: "session_count": 1, 2024-04-21T08:51:50.222 INFO:teuthology.orchestra.run.smithi179.stdout: "session_add": 2, 2024-04-21T08:51:50.222 INFO:teuthology.orchestra.run.smithi179.stdout: "session_remove": 1, 2024-04-21T08:51:50.222 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_open": 1, 2024-04-21T08:51:50.222 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_stale": 0, 2024-04-21T08:51:50.222 INFO:teuthology.orchestra.run.smithi179.stdout: "total_load": 0, 2024-04-21T08:51:50.222 INFO:teuthology.orchestra.run.smithi179.stdout: "average_load": 0, 2024-04-21T08:51:50.222 INFO:teuthology.orchestra.run.smithi179.stdout: "avg_session_uptime": 1928, 2024-04-21T08:51:50.222 INFO:teuthology.orchestra.run.smithi179.stdout: "mdthresh_evicted": 0 2024-04-21T08:51:50.222 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.222 INFO:teuthology.orchestra.run.smithi179.stdout: "mempool": { 2024-04-21T08:51:50.222 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_bytes": 0, 2024-04-21T08:51:50.222 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_items": 0, 2024-04-21T08:51:50.222 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T08:51:50.223 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_items": 0, 2024-04-21T08:51:50.223 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T08:51:50.223 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_items": 0, 2024-04-21T08:51:50.223 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T08:51:50.223 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T08:51:50.223 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T08:51:50.223 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T08:51:50.223 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T08:51:50.223 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_items": 0, 2024-04-21T08:51:50.223 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T08:51:50.223 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T08:51:50.223 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_bytes": 0, 2024-04-21T08:51:50.223 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_items": 0, 2024-04-21T08:51:50.223 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_bytes": 0, 2024-04-21T08:51:50.224 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_items": 0, 2024-04-21T08:51:50.224 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T08:51:50.224 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T08:51:50.224 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T08:51:50.224 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T08:51:50.224 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T08:51:50.224 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_items": 0, 2024-04-21T08:51:50.224 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_bytes": 0, 2024-04-21T08:51:50.224 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_items": 0, 2024-04-21T08:51:50.224 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T08:51:50.224 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T08:51:50.224 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_bytes": 0, 2024-04-21T08:51:50.224 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_items": 0, 2024-04-21T08:51:50.224 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_bytes": 0, 2024-04-21T08:51:50.225 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_items": 0, 2024-04-21T08:51:50.225 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T08:51:50.225 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_items": 0, 2024-04-21T08:51:50.225 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T08:51:50.225 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_items": 0, 2024-04-21T08:51:50.225 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_bytes": 181321, 2024-04-21T08:51:50.225 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_items": 47, 2024-04-21T08:51:50.225 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_bytes": 0, 2024-04-21T08:51:50.225 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_items": 0, 2024-04-21T08:51:50.225 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_bytes": 0, 2024-04-21T08:51:50.225 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_items": 0, 2024-04-21T08:51:50.225 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_bytes": 0, 2024-04-21T08:51:50.225 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_items": 0, 2024-04-21T08:51:50.225 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_bytes": 0, 2024-04-21T08:51:50.226 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_items": 0, 2024-04-21T08:51:50.226 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_bytes": 9512, 2024-04-21T08:51:50.226 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_items": 182, 2024-04-21T08:51:50.226 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T08:51:50.226 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_items": 0, 2024-04-21T08:51:50.226 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_bytes": 0, 2024-04-21T08:51:50.226 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_items": 0, 2024-04-21T08:51:50.226 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_bytes": 8512, 2024-04-21T08:51:50.226 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_items": 13, 2024-04-21T08:51:50.226 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_bytes": 0, 2024-04-21T08:51:50.226 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_items": 0, 2024-04-21T08:51:50.226 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_bytes": 0, 2024-04-21T08:51:50.226 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_items": 0 2024-04-21T08:51:50.226 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.227 INFO:teuthology.orchestra.run.smithi179.stdout: "objecter": { 2024-04-21T08:51:50.227 INFO:teuthology.orchestra.run.smithi179.stdout: "op_active": 0, 2024-04-21T08:51:50.227 INFO:teuthology.orchestra.run.smithi179.stdout: "op_laggy": 0, 2024-04-21T08:51:50.227 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send": 2653, 2024-04-21T08:51:50.227 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send_bytes": 21, 2024-04-21T08:51:50.227 INFO:teuthology.orchestra.run.smithi179.stdout: "op_resend": 3, 2024-04-21T08:51:50.227 INFO:teuthology.orchestra.run.smithi179.stdout: "op_reply": 2653, 2024-04-21T08:51:50.227 INFO:teuthology.orchestra.run.smithi179.stdout: "op_latency": { 2024-04-21T08:51:50.227 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 2653, 2024-04-21T08:51:50.227 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 1.391970606, 2024-04-21T08:51:50.227 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000524677 2024-04-21T08:51:50.227 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.227 INFO:teuthology.orchestra.run.smithi179.stdout: "op_inflight": 0, 2024-04-21T08:51:50.227 INFO:teuthology.orchestra.run.smithi179.stdout: "oplen_avg": { 2024-04-21T08:51:50.227 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 2653, 2024-04-21T08:51:50.228 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 2654 2024-04-21T08:51:50.228 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.228 INFO:teuthology.orchestra.run.smithi179.stdout: "op": 2653, 2024-04-21T08:51:50.228 INFO:teuthology.orchestra.run.smithi179.stdout: "op_r": 2653, 2024-04-21T08:51:50.228 INFO:teuthology.orchestra.run.smithi179.stdout: "op_w": 0, 2024-04-21T08:51:50.228 INFO:teuthology.orchestra.run.smithi179.stdout: "op_rmw": 0, 2024-04-21T08:51:50.228 INFO:teuthology.orchestra.run.smithi179.stdout: "op_pg": 0, 2024-04-21T08:51:50.228 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_stat": 1740, 2024-04-21T08:51:50.228 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_create": 0, 2024-04-21T08:51:50.228 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_read": 911, 2024-04-21T08:51:50.228 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_write": 0, 2024-04-21T08:51:50.228 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writefull": 0, 2024-04-21T08:51:50.228 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writesame": 0, 2024-04-21T08:51:50.228 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_append": 0, 2024-04-21T08:51:50.229 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_zero": 0, 2024-04-21T08:51:50.229 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_truncate": 0, 2024-04-21T08:51:50.229 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_delete": 0, 2024-04-21T08:51:50.229 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_mapext": 0, 2024-04-21T08:51:50.229 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_sparse_read": 0, 2024-04-21T08:51:50.229 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_clonerange": 0, 2024-04-21T08:51:50.229 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_getxattr": 1, 2024-04-21T08:51:50.229 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_setxattr": 0, 2024-04-21T08:51:50.229 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_cmpxattr": 0, 2024-04-21T08:51:50.229 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_rmxattr": 0, 2024-04-21T08:51:50.229 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_resetxattrs": 0, 2024-04-21T08:51:50.229 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_call": 0, 2024-04-21T08:51:50.229 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_watch": 0, 2024-04-21T08:51:50.229 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_notify": 0, 2024-04-21T08:51:50.229 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T08:51:50.230 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls": 0, 2024-04-21T08:51:50.230 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls_filter": 0, 2024-04-21T08:51:50.230 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_other": 0, 2024-04-21T08:51:50.230 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_active": 0, 2024-04-21T08:51:50.230 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_send": 0, 2024-04-21T08:51:50.230 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_resend": 0, 2024-04-21T08:51:50.230 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_ping": 0, 2024-04-21T08:51:50.230 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_active": 0, 2024-04-21T08:51:50.230 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_send": 0, 2024-04-21T08:51:50.230 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_resend": 0, 2024-04-21T08:51:50.230 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_active": 0, 2024-04-21T08:51:50.230 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_send": 0, 2024-04-21T08:51:50.231 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_resend": 0, 2024-04-21T08:51:50.231 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_active": 0, 2024-04-21T08:51:50.231 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_send": 0, 2024-04-21T08:51:50.231 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_resend": 0, 2024-04-21T08:51:50.231 INFO:teuthology.orchestra.run.smithi179.stdout: "command_active": 0, 2024-04-21T08:51:50.231 INFO:teuthology.orchestra.run.smithi179.stdout: "command_send": 0, 2024-04-21T08:51:50.231 INFO:teuthology.orchestra.run.smithi179.stdout: "command_resend": 0, 2024-04-21T08:51:50.231 INFO:teuthology.orchestra.run.smithi179.stdout: "map_epoch": 108, 2024-04-21T08:51:50.231 INFO:teuthology.orchestra.run.smithi179.stdout: "map_full": 0, 2024-04-21T08:51:50.231 INFO:teuthology.orchestra.run.smithi179.stdout: "map_inc": 19, 2024-04-21T08:51:50.231 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_sessions": 4, 2024-04-21T08:51:50.231 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_open": 5, 2024-04-21T08:51:50.231 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_close": 1, 2024-04-21T08:51:50.231 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_laggy": 0, 2024-04-21T08:51:50.232 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_wr": 0, 2024-04-21T08:51:50.232 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_rd": 2, 2024-04-21T08:51:50.232 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_del": 0 2024-04-21T08:51:50.232 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.232 INFO:teuthology.orchestra.run.smithi179.stdout: "oft": { 2024-04-21T08:51:50.232 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_objs": 0, 2024-04-21T08:51:50.232 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_kv_pairs": 0, 2024-04-21T08:51:50.232 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_updates": 0, 2024-04-21T08:51:50.232 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_removes": 0 2024-04-21T08:51:50.232 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.232 INFO:teuthology.orchestra.run.smithi179.stdout: "osd-slow-ops": { 2024-04-21T08:51:50.232 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_ops_count": 0 2024-04-21T08:51:50.232 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.232 INFO:teuthology.orchestra.run.smithi179.stdout: "purge_queue": { 2024-04-21T08:51:50.233 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops": 0, 2024-04-21T08:51:50.233 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops_high_water": 0, 2024-04-21T08:51:50.233 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing": 0, 2024-04-21T08:51:50.233 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_high_water": 0, 2024-04-21T08:51:50.233 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed_ops": 0, 2024-04-21T08:51:50.233 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed": 0, 2024-04-21T08:51:50.233 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_item_in_journal": 0 2024-04-21T08:51:50.233 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.233 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T08:51:50.233 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-04-21T08:51:50.233 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-04-21T08:51:50.233 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-04-21T08:51:50.233 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 2951, 2024-04-21T08:51:50.233 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 2136438, 2024-04-21T08:51:50.233 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:50.234 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 2951, 2024-04-21T08:51:50.234 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-04-21T08:51:50.234 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-04-21T08:51:50.234 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 2951, 2024-04-21T08:51:50.234 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 2136438, 2024-04-21T08:51:50.234 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-04-21T08:51:50.234 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.234 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.234 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.234 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-04-21T08:51:50.234 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.234 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_bytes": { 2024-04-21T08:51:50.234 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-04-21T08:51:50.235 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-04-21T08:51:50.235 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-04-21T08:51:50.235 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-04-21T08:51:50.235 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-04-21T08:51:50.235 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:50.235 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:50.235 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 2653, 2024-04-21T08:51:50.235 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 1433, 2024-04-21T08:51:50.235 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 2653, 2024-04-21T08:51:50.235 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 1433, 2024-04-21T08:51:50.235 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-04-21T08:51:50.235 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.235 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.235 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.235 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-04-21T08:51:50.236 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.236 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_ops": { 2024-04-21T08:51:50.236 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-04-21T08:51:50.236 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 1024, 2024-04-21T08:51:50.236 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-04-21T08:51:50.236 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-04-21T08:51:50.236 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-04-21T08:51:50.236 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:50.236 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:50.236 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 2653, 2024-04-21T08:51:50.236 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 2653, 2024-04-21T08:51:50.236 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 2653, 2024-04-21T08:51:50.236 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 2653, 2024-04-21T08:51:50.236 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-04-21T08:51:50.236 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.237 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.237 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.237 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-04-21T08:51:50.237 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.237 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle": { 2024-04-21T08:51:50.237 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-04-21T08:51:50.237 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-04-21T08:51:50.237 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-04-21T08:51:50.237 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-04-21T08:51:50.237 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-04-21T08:51:50.237 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:50.237 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:50.237 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-04-21T08:51:50.238 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-04-21T08:51:50.238 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 0, 2024-04-21T08:51:50.238 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 0, 2024-04-21T08:51:50.238 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-04-21T08:51:50.238 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.238 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.238 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.238 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-04-21T08:51:50.238 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:50.238 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle-0x5620d29626e0": { 2024-04-21T08:51:50.238 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-04-21T08:51:50.238 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-04-21T08:51:50.238 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-04-21T08:51:50.238 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-04-21T08:51:50.239 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-04-21T08:51:50.239 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:50.239 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:50.239 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-04-21T08:51:50.239 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-04-21T08:51:50.239 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 0, 2024-04-21T08:51:50.239 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 0, 2024-04-21T08:51:50.239 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-04-21T08:51:50.239 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:50.239 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:50.239 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:50.239 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-04-21T08:51:50.239 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-04-21T08:51:50.239 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-04-21T08:51:50.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:49 smithi086 ceph-mon[27939]: pgmap v408: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 767 B/s wr, 13 op/s 2024-04-21T08:51:50.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:49 smithi088 ceph-mon[31259]: pgmap v408: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 6.7 KiB/s rd, 767 B/s wr, 13 op/s 2024-04-21T08:51:51.010 INFO:tasks.check_counter:Daemon mds.f mds.exported=0 2024-04-21T08:51:51.010 INFO:tasks.check_counter:Daemon mds.f mds.imported=0 2024-04-21T08:51:51.010 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update=0 2024-04-21T08:51:51.010 INFO:tasks.check_counter:Daemon mds.f mds_cache.dir_update_receipt=0 2024-04-21T08:51:51.010 INFO:tasks.check_counter:Daemon mds.f mds.root_rsnaps=0 2024-04-21T08:51:51.010 INFO:tasks.check_counter:Daemon mds.f mds_server.req_mksnap_latency.avgcount=0 2024-04-21T08:51:51.010 INFO:tasks.check_counter:Daemon mds.f mds_server.req_rmsnap_latency.avgcount=0 2024-04-21T08:51:51.010 DEBUG:tasks.check_counter:Getting stats from i 2024-04-21T08:51:51.010 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph daemon mds.i perf dump 2024-04-21T08:51:51.248 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:51:52.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:51 smithi179 ceph-mon[31829]: pgmap v409: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-21T08:51:52.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:51 smithi086 ceph-mon[27939]: pgmap v409: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-21T08:51:52.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:51 smithi088 ceph-mon[31259]: pgmap v409: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 85 B/s wr, 8 op/s 2024-04-21T08:51:52.979 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-04-21T08:51:52.979 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T08:51:52.979 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 1624, 2024-04-21T08:51:52.979 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 1184, 2024-04-21T08:51:52.979 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 451474, 2024-04-21T08:51:52.979 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 405792, 2024-04-21T08:51:52.980 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 10, 2024-04-21T08:51:52.980 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 4, 2024-04-21T08:51:52.980 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.122400251, 2024-04-21T08:51:52.980 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.047798404, 2024-04-21T08:51:52.980 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.076651712, 2024-04-21T08:51:52.980 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.005716157, 2024-04-21T08:51:52.980 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:52.980 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1184, 2024-04-21T08:51:52.980 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.064323255, 2024-04-21T08:51:52.980 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000054327 2024-04-21T08:51:52.980 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:52.980 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:52.980 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 2634, 2024-04-21T08:51:52.980 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000640348, 2024-04-21T08:51:52.981 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000243 2024-04-21T08:51:52.981 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:52.981 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 0, 2024-04-21T08:51:52.981 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 0 2024-04-21T08:51:52.981 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:52.981 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T08:51:52.981 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 99, 2024-04-21T08:51:52.981 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 309, 2024-04-21T08:51:52.981 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 34527, 2024-04-21T08:51:52.981 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 586674, 2024-04-21T08:51:52.981 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 17, 2024-04-21T08:51:52.981 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 5, 2024-04-21T08:51:52.981 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.033601861, 2024-04-21T08:51:52.981 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.014976661, 2024-04-21T08:51:52.982 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.016132990, 2024-04-21T08:51:52.982 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.000274547, 2024-04-21T08:51:52.982 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:52.982 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 307, 2024-04-21T08:51:52.982 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.028262680, 2024-04-21T08:51:52.982 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000092060 2024-04-21T08:51:52.982 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:52.982 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:52.982 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 170, 2024-04-21T08:51:52.982 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000064961, 2024-04-21T08:51:52.982 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000382 2024-04-21T08:51:52.982 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:52.982 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 7184, 2024-04-21T08:51:52.982 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 510144 2024-04-21T08:51:52.983 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:52.983 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T08:51:52.983 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 384, 2024-04-21T08:51:52.983 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 366, 2024-04-21T08:51:52.983 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 1686520, 2024-04-21T08:51:52.983 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 737594, 2024-04-21T08:51:52.983 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 58, 2024-04-21T08:51:52.983 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 5, 2024-04-21T08:51:52.983 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.073753525, 2024-04-21T08:51:52.983 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.019572234, 2024-04-21T08:51:52.983 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.054170769, 2024-04-21T08:51:52.983 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.006467200, 2024-04-21T08:51:52.983 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:52.983 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 363, 2024-04-21T08:51:52.983 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.024362860, 2024-04-21T08:51:52.984 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000067115 2024-04-21T08:51:52.984 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:52.984 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:52.984 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 48, 2024-04-21T08:51:52.984 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000004783, 2024-04-21T08:51:52.984 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000099 2024-04-21T08:51:52.984 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:52.984 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 1677360, 2024-04-21T08:51:52.984 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 165536 2024-04-21T08:51:52.984 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:52.984 INFO:teuthology.orchestra.run.smithi179.stdout: "cct": { 2024-04-21T08:51:52.984 INFO:teuthology.orchestra.run.smithi179.stdout: "total_workers": 1, 2024-04-21T08:51:52.984 INFO:teuthology.orchestra.run.smithi179.stdout: "unhealthy_workers": 0 2024-04-21T08:51:52.984 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:52.985 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-MDSRank": { 2024-04-21T08:51:52.985 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-04-21T08:51:52.985 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-04-21T08:51:52.985 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 38, 2024-04-21T08:51:52.985 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.001364646, 2024-04-21T08:51:52.985 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000035911 2024-04-21T08:51:52.985 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-04-21T08:51:52.985 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:52.985 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-PurgeQueue": { 2024-04-21T08:51:52.985 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-04-21T08:51:52.985 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-04-21T08:51:52.985 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 2, 2024-04-21T08:51:52.985 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000013848, 2024-04-21T08:51:52.985 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000006924 2024-04-21T08:51:52.985 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-04-21T08:51:52.986 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:52.986 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": { 2024-04-21T08:51:52.986 INFO:teuthology.orchestra.run.smithi179.stdout: "request": 3, 2024-04-21T08:51:52.986 INFO:teuthology.orchestra.run.smithi179.stdout: "reply": 1, 2024-04-21T08:51:52.986 INFO:teuthology.orchestra.run.smithi179.stdout: "reply_latency": { 2024-04-21T08:51:52.986 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1, 2024-04-21T08:51:52.986 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000240627, 2024-04-21T08:51:52.986 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000240627 2024-04-21T08:51:52.986 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:52.986 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_reply": 0, 2024-04-21T08:51:52.986 INFO:teuthology.orchestra.run.smithi179.stdout: "forward": 2, 2024-04-21T08:51:52.986 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_complete": 0, 2024-04-21T08:51:52.986 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_keys": 0, 2024-04-21T08:51:52.986 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_commit": 11, 2024-04-21T08:51:52.987 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_split": 0, 2024-04-21T08:51:52.987 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_merge": 0, 2024-04-21T08:51:52.987 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes": 16, 2024-04-21T08:51:52.987 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_top": 0, 2024-04-21T08:51:52.987 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_bottom": 16, 2024-04-21T08:51:52.987 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pin_tail": 0, 2024-04-21T08:51:52.987 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pinned": 13, 2024-04-21T08:51:52.987 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_expired": 3, 2024-04-21T08:51:52.987 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_with_caps": 0, 2024-04-21T08:51:52.987 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": 0, 2024-04-21T08:51:52.987 INFO:teuthology.orchestra.run.smithi179.stdout: "subtrees": 3, 2024-04-21T08:51:52.987 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse": 26, 2024-04-21T08:51:52.987 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_hit": 12, 2024-04-21T08:51:52.987 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_forward": 1, 2024-04-21T08:51:52.988 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_discover": 11, 2024-04-21T08:51:52.988 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_dir_fetch": 0, 2024-04-21T08:51:52.988 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_remote_ino": 0, 2024-04-21T08:51:52.988 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_lock": 0, 2024-04-21T08:51:52.988 INFO:teuthology.orchestra.run.smithi179.stdout: "load_cent": 2000, 2024-04-21T08:51:52.988 INFO:teuthology.orchestra.run.smithi179.stdout: "q": 2, 2024-04-21T08:51:52.988 INFO:teuthology.orchestra.run.smithi179.stdout: "exported": 0, 2024-04-21T08:51:52.988 INFO:teuthology.orchestra.run.smithi179.stdout: "exported_inodes": 0, 2024-04-21T08:51:52.988 INFO:teuthology.orchestra.run.smithi179.stdout: "imported": 0, 2024-04-21T08:51:52.988 INFO:teuthology.orchestra.run.smithi179.stdout: "imported_inodes": 0, 2024-04-21T08:51:52.988 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_dir_fetch": 0, 2024-04-21T08:51:52.988 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_backtrace_fetch": 0, 2024-04-21T08:51:52.988 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_peer_discover": 0, 2024-04-21T08:51:52.988 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rfiles": 5, 2024-04-21T08:51:52.988 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rbytes": 298, 2024-04-21T08:51:52.989 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rsnaps": 6, 2024-04-21T08:51:52.989 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_fetch": 0, 2024-04-21T08:51:52.989 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_set_tag": 0, 2024-04-21T08:51:52.989 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T08:51:52.989 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_inotable_repaired": 0, 2024-04-21T08:51:52.989 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_inodes": 0, 2024-04-21T08:51:52.989 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_base_inodes": 0, 2024-04-21T08:51:52.989 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dirfrag_rstats": 0, 2024-04-21T08:51:52.989 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_file_inodes": 0, 2024-04-21T08:51:52.989 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_inode_file_caps": 0, 2024-04-21T08:51:52.989 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_revoke": 0, 2024-04-21T08:51:52.989 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_grant": 0, 2024-04-21T08:51:52.989 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_trunc": 0, 2024-04-21T08:51:52.989 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T08:51:52.990 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T08:51:52.990 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps": 0, 2024-04-21T08:51:52.990 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps_dirty": 0, 2024-04-21T08:51:52.990 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_cap_release": 0, 2024-04-21T08:51:52.990 INFO:teuthology.orchestra.run.smithi179.stdout: "process_request_cap_release": 1 2024-04-21T08:51:52.990 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:52.990 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_cache": { 2024-04-21T08:51:52.990 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update": 0, 2024-04-21T08:51:52.990 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update_receipt": 19, 2024-04-21T08:51:52.990 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_try_discover": 15, 2024-04-21T08:51:52.990 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_send_discover": 11, 2024-04-21T08:51:52.990 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_handle_discover": 0, 2024-04-21T08:51:52.990 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays": 0, 2024-04-21T08:51:52.991 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_delayed": 0, 2024-04-21T08:51:52.991 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_enqueuing": 0, 2024-04-21T08:51:52.991 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_created": 0, 2024-04-21T08:51:52.991 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_enqueued": 0, 2024-04-21T08:51:52.991 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_reintegrated": 0, 2024-04-21T08:51:52.991 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_migrated": 0, 2024-04-21T08:51:52.991 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_processing": 0, 2024-04-21T08:51:52.991 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_enqueued": 0, 2024-04-21T08:51:52.991 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_prioritized": 0, 2024-04-21T08:51:52.991 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_started": 0, 2024-04-21T08:51:52.991 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_completed": 0, 2024-04-21T08:51:52.991 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_quiesce_path": 3, 2024-04-21T08:51:52.991 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_quiesce_inode": 15, 2024-04-21T08:51:52.991 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_enqueue_scrub": 0, 2024-04-21T08:51:52.991 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_exportdir": 0, 2024-04-21T08:51:52.992 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_flush": 0, 2024-04-21T08:51:52.992 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragmentdir": 0, 2024-04-21T08:51:52.992 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragstats": 0, 2024-04-21T08:51:52.992 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_inodestats": 0 2024-04-21T08:51:52.992 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:52.992 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_log": { 2024-04-21T08:51:52.992 INFO:teuthology.orchestra.run.smithi179.stdout: "evlrg": 0, 2024-04-21T08:51:52.992 INFO:teuthology.orchestra.run.smithi179.stdout: "evadd": 5, 2024-04-21T08:51:52.992 INFO:teuthology.orchestra.run.smithi179.stdout: "evex": 0, 2024-04-21T08:51:52.992 INFO:teuthology.orchestra.run.smithi179.stdout: "evtrm": 0, 2024-04-21T08:51:52.992 INFO:teuthology.orchestra.run.smithi179.stdout: "ev": 5, 2024-04-21T08:51:52.992 INFO:teuthology.orchestra.run.smithi179.stdout: "evexg": 0, 2024-04-21T08:51:52.992 INFO:teuthology.orchestra.run.smithi179.stdout: "evexd": 0, 2024-04-21T08:51:52.992 INFO:teuthology.orchestra.run.smithi179.stdout: "segadd": 2, 2024-04-21T08:51:52.992 INFO:teuthology.orchestra.run.smithi179.stdout: "segex": 0, 2024-04-21T08:51:52.993 INFO:teuthology.orchestra.run.smithi179.stdout: "segtrm": 0, 2024-04-21T08:51:52.993 INFO:teuthology.orchestra.run.smithi179.stdout: "seg": 2, 2024-04-21T08:51:52.993 INFO:teuthology.orchestra.run.smithi179.stdout: "segmjr": 2, 2024-04-21T08:51:52.993 INFO:teuthology.orchestra.run.smithi179.stdout: "segexg": 0, 2024-04-21T08:51:52.993 INFO:teuthology.orchestra.run.smithi179.stdout: "segexd": 0, 2024-04-21T08:51:52.993 INFO:teuthology.orchestra.run.smithi179.stdout: "expos": 4194304, 2024-04-21T08:51:52.993 INFO:teuthology.orchestra.run.smithi179.stdout: "wrpos": 4195732, 2024-04-21T08:51:52.993 INFO:teuthology.orchestra.run.smithi179.stdout: "rdpos": 0, 2024-04-21T08:51:52.993 INFO:teuthology.orchestra.run.smithi179.stdout: "jlat": { 2024-04-21T08:51:52.993 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 4, 2024-04-21T08:51:52.993 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.007409625, 2024-04-21T08:51:52.993 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.001852406 2024-04-21T08:51:52.993 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:52.993 INFO:teuthology.orchestra.run.smithi179.stdout: "replayed": 0 2024-04-21T08:51:52.994 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:52.994 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_mem": { 2024-04-21T08:51:52.994 INFO:teuthology.orchestra.run.smithi179.stdout: "ino": 20, 2024-04-21T08:51:52.994 INFO:teuthology.orchestra.run.smithi179.stdout: "ino+": 21, 2024-04-21T08:51:52.994 INFO:teuthology.orchestra.run.smithi179.stdout: "ino-": 1, 2024-04-21T08:51:52.994 INFO:teuthology.orchestra.run.smithi179.stdout: "dir": 19, 2024-04-21T08:51:52.994 INFO:teuthology.orchestra.run.smithi179.stdout: "dir+": 21, 2024-04-21T08:51:52.994 INFO:teuthology.orchestra.run.smithi179.stdout: "dir-": 2, 2024-04-21T08:51:52.994 INFO:teuthology.orchestra.run.smithi179.stdout: "dn": 16, 2024-04-21T08:51:52.994 INFO:teuthology.orchestra.run.smithi179.stdout: "dn+": 19, 2024-04-21T08:51:52.994 INFO:teuthology.orchestra.run.smithi179.stdout: "dn-": 3, 2024-04-21T08:51:52.994 INFO:teuthology.orchestra.run.smithi179.stdout: "cap": 0, 2024-04-21T08:51:52.994 INFO:teuthology.orchestra.run.smithi179.stdout: "cap+": 1, 2024-04-21T08:51:52.994 INFO:teuthology.orchestra.run.smithi179.stdout: "cap-": 1, 2024-04-21T08:51:52.995 INFO:teuthology.orchestra.run.smithi179.stdout: "rss": 53788, 2024-04-21T08:51:52.995 INFO:teuthology.orchestra.run.smithi179.stdout: "heap": 223608 2024-04-21T08:51:52.995 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:52.995 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_server": { 2024-04-21T08:51:52.995 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_client_request": 4, 2024-04-21T08:51:52.995 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_server_request": 0, 2024-04-21T08:51:52.995 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_request": 3, 2024-04-21T08:51:52.995 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_session": 15, 2024-04-21T08:51:52.995 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_peer_request": 15, 2024-04-21T08:51:52.995 INFO:teuthology.orchestra.run.smithi179.stdout: "req_create_latency": { 2024-04-21T08:51:52.995 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:52.995 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:52.996 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:52.996 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:52.996 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getattr_latency": { 2024-04-21T08:51:52.996 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1, 2024-04-21T08:51:52.996 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000240627, 2024-04-21T08:51:52.996 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000240627 2024-04-21T08:51:52.996 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:52.996 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getfilelock_latency": { 2024-04-21T08:51:52.996 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:52.996 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:52.996 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:52.996 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:52.996 INFO:teuthology.orchestra.run.smithi179.stdout: "req_link_latency": { 2024-04-21T08:51:52.997 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:52.997 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:52.997 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:52.997 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:52.997 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookup_latency": { 2024-04-21T08:51:52.997 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:52.997 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:52.997 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:52.997 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:52.997 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookuphash_latency": { 2024-04-21T08:51:52.997 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:52.997 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:52.997 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:52.997 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:52.997 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupino_latency": { 2024-04-21T08:51:52.997 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:52.998 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:52.998 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:52.998 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:52.998 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupname_latency": { 2024-04-21T08:51:52.998 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:52.998 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:52.998 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:52.998 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:52.998 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupparent_latency": { 2024-04-21T08:51:52.998 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:52.998 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:52.998 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:52.998 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:52.999 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupsnap_latency": { 2024-04-21T08:51:52.999 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:52.999 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:52.999 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:52.999 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:52.999 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lssnap_latency": { 2024-04-21T08:51:52.999 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:52.999 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:52.999 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:52.999 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:52.999 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mkdir_latency": { 2024-04-21T08:51:53.000 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:53.000 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:53.000 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:53.000 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:53.000 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mknod_latency": { 2024-04-21T08:51:53.000 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:53.000 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:53.000 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:53.000 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:53.000 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mksnap_latency": { 2024-04-21T08:51:53.000 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:53.001 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:53.001 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:53.001 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:53.001 INFO:teuthology.orchestra.run.smithi179.stdout: "req_open_latency": { 2024-04-21T08:51:53.001 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:53.001 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:53.001 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:53.001 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:53.001 INFO:teuthology.orchestra.run.smithi179.stdout: "req_readdir_latency": { 2024-04-21T08:51:53.001 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:53.001 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:53.001 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:53.001 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:53.001 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rename_latency": { 2024-04-21T08:51:53.001 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:53.002 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:53.002 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:53.002 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:53.002 INFO:teuthology.orchestra.run.smithi179.stdout: "req_renamesnap_latency": { 2024-04-21T08:51:53.002 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:53.002 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:53.002 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:53.002 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:53.002 INFO:teuthology.orchestra.run.smithi179.stdout: "req_snapdiff_latency": { 2024-04-21T08:51:53.002 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:53.002 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:53.002 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:53.002 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:53.003 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmdir_latency": { 2024-04-21T08:51:53.003 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:53.003 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:53.003 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:53.003 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:53.003 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmsnap_latency": { 2024-04-21T08:51:53.003 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:53.003 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:53.003 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:53.003 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:53.003 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmxattr_latency": { 2024-04-21T08:51:53.003 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:53.003 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:53.003 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:53.004 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:53.004 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setattr_latency": { 2024-04-21T08:51:53.004 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:53.004 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:53.004 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:53.004 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:53.004 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setdirlayout_latency": { 2024-04-21T08:51:53.004 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:53.004 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:53.004 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:53.004 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:53.004 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setfilelock_latency": { 2024-04-21T08:51:53.004 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:53.004 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:53.005 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:53.005 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:53.005 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setlayout_latency": { 2024-04-21T08:51:53.005 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:53.005 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:53.005 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:53.005 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:53.005 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setxattr_latency": { 2024-04-21T08:51:53.005 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:53.005 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:53.005 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:53.005 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:53.005 INFO:teuthology.orchestra.run.smithi179.stdout: "req_symlink_latency": { 2024-04-21T08:51:53.005 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:53.006 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:53.006 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:53.006 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:53.006 INFO:teuthology.orchestra.run.smithi179.stdout: "req_unlink_latency": { 2024-04-21T08:51:53.006 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:53.006 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:53.006 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:53.006 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:53.006 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_revoke_eviction": 0, 2024-04-21T08:51:53.006 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_acquisition_throttle": 0, 2024-04-21T08:51:53.006 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getvxattr_latency": { 2024-04-21T08:51:53.006 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:53.006 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:53.006 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:53.007 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-04-21T08:51:53.007 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:53.007 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_sessions": { 2024-04-21T08:51:53.007 INFO:teuthology.orchestra.run.smithi179.stdout: "session_count": 1, 2024-04-21T08:51:53.007 INFO:teuthology.orchestra.run.smithi179.stdout: "session_add": 2, 2024-04-21T08:51:53.007 INFO:teuthology.orchestra.run.smithi179.stdout: "session_remove": 1, 2024-04-21T08:51:53.007 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_open": 1, 2024-04-21T08:51:53.007 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_stale": 0, 2024-04-21T08:51:53.007 INFO:teuthology.orchestra.run.smithi179.stdout: "total_load": 1, 2024-04-21T08:51:53.007 INFO:teuthology.orchestra.run.smithi179.stdout: "average_load": 1, 2024-04-21T08:51:53.007 INFO:teuthology.orchestra.run.smithi179.stdout: "avg_session_uptime": 786, 2024-04-21T08:51:53.007 INFO:teuthology.orchestra.run.smithi179.stdout: "mdthresh_evicted": 0 2024-04-21T08:51:53.007 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:53.007 INFO:teuthology.orchestra.run.smithi179.stdout: "mempool": { 2024-04-21T08:51:53.008 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_bytes": 0, 2024-04-21T08:51:53.008 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_items": 0, 2024-04-21T08:51:53.008 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T08:51:53.008 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_items": 0, 2024-04-21T08:51:53.008 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T08:51:53.008 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_items": 0, 2024-04-21T08:51:53.008 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T08:51:53.008 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T08:51:53.008 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T08:51:53.008 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T08:51:53.008 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T08:51:53.008 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_items": 0, 2024-04-21T08:51:53.008 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T08:51:53.008 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T08:51:53.009 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_bytes": 0, 2024-04-21T08:51:53.009 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_items": 0, 2024-04-21T08:51:53.009 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_bytes": 0, 2024-04-21T08:51:53.009 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_items": 0, 2024-04-21T08:51:53.009 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T08:51:53.009 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T08:51:53.009 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T08:51:53.009 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T08:51:53.009 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T08:51:53.009 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_items": 0, 2024-04-21T08:51:53.009 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_bytes": 0, 2024-04-21T08:51:53.009 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_items": 0, 2024-04-21T08:51:53.009 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T08:51:53.009 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T08:51:53.009 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_bytes": 0, 2024-04-21T08:51:53.010 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_items": 0, 2024-04-21T08:51:53.010 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_bytes": 0, 2024-04-21T08:51:53.010 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_items": 0, 2024-04-21T08:51:53.010 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T08:51:53.010 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_items": 0, 2024-04-21T08:51:53.010 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T08:51:53.010 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_items": 0, 2024-04-21T08:51:53.010 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_bytes": 254247, 2024-04-21T08:51:53.010 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_items": 84, 2024-04-21T08:51:53.010 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_bytes": 0, 2024-04-21T08:51:53.010 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_items": 0, 2024-04-21T08:51:53.010 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_bytes": 0, 2024-04-21T08:51:53.010 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_items": 0, 2024-04-21T08:51:53.010 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_bytes": 0, 2024-04-21T08:51:53.011 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_items": 0, 2024-04-21T08:51:53.011 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_bytes": 0, 2024-04-21T08:51:53.011 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_items": 0, 2024-04-21T08:51:53.011 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_bytes": 9512, 2024-04-21T08:51:53.011 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_items": 182, 2024-04-21T08:51:53.011 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T08:51:53.011 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_items": 0, 2024-04-21T08:51:53.011 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_bytes": 0, 2024-04-21T08:51:53.011 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_items": 0, 2024-04-21T08:51:53.011 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_bytes": 87181, 2024-04-21T08:51:53.011 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_items": 346, 2024-04-21T08:51:53.011 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_bytes": 0, 2024-04-21T08:51:53.011 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_items": 0, 2024-04-21T08:51:53.011 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_bytes": 0, 2024-04-21T08:51:53.012 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_items": 0 2024-04-21T08:51:53.012 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:53.012 INFO:teuthology.orchestra.run.smithi179.stdout: "objecter": { 2024-04-21T08:51:53.012 INFO:teuthology.orchestra.run.smithi179.stdout: "op_active": 0, 2024-04-21T08:51:53.012 INFO:teuthology.orchestra.run.smithi179.stdout: "op_laggy": 0, 2024-04-21T08:51:53.012 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send": 40, 2024-04-21T08:51:53.012 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send_bytes": 11730, 2024-04-21T08:51:53.012 INFO:teuthology.orchestra.run.smithi179.stdout: "op_resend": 33, 2024-04-21T08:51:53.012 INFO:teuthology.orchestra.run.smithi179.stdout: "op_reply": 40, 2024-04-21T08:51:53.012 INFO:teuthology.orchestra.run.smithi179.stdout: "op_latency": { 2024-04-21T08:51:53.012 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 40, 2024-04-21T08:51:53.012 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.203995716, 2024-04-21T08:51:53.012 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.005099892 2024-04-21T08:51:53.012 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:53.013 INFO:teuthology.orchestra.run.smithi179.stdout: "op_inflight": 0, 2024-04-21T08:51:53.013 INFO:teuthology.orchestra.run.smithi179.stdout: "oplen_avg": { 2024-04-21T08:51:53.013 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 40, 2024-04-21T08:51:53.013 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 62 2024-04-21T08:51:53.013 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:53.013 INFO:teuthology.orchestra.run.smithi179.stdout: "op": 40, 2024-04-21T08:51:53.013 INFO:teuthology.orchestra.run.smithi179.stdout: "op_r": 0, 2024-04-21T08:51:53.013 INFO:teuthology.orchestra.run.smithi179.stdout: "op_w": 40, 2024-04-21T08:51:53.013 INFO:teuthology.orchestra.run.smithi179.stdout: "op_rmw": 0, 2024-04-21T08:51:53.013 INFO:teuthology.orchestra.run.smithi179.stdout: "op_pg": 0, 2024-04-21T08:51:53.013 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_stat": 0, 2024-04-21T08:51:53.013 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_create": 10, 2024-04-21T08:51:53.013 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_read": 0, 2024-04-21T08:51:53.013 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_write": 4, 2024-04-21T08:51:53.013 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writefull": 7, 2024-04-21T08:51:53.014 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writesame": 0, 2024-04-21T08:51:53.014 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_append": 0, 2024-04-21T08:51:53.014 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_zero": 0, 2024-04-21T08:51:53.014 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_truncate": 0, 2024-04-21T08:51:53.014 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_delete": 6, 2024-04-21T08:51:53.014 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_mapext": 0, 2024-04-21T08:51:53.014 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_sparse_read": 0, 2024-04-21T08:51:53.014 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_clonerange": 0, 2024-04-21T08:51:53.014 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_getxattr": 0, 2024-04-21T08:51:53.014 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_setxattr": 20, 2024-04-21T08:51:53.014 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_cmpxattr": 0, 2024-04-21T08:51:53.014 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_rmxattr": 0, 2024-04-21T08:51:53.014 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_resetxattrs": 0, 2024-04-21T08:51:53.014 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_call": 0, 2024-04-21T08:51:53.015 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_watch": 0, 2024-04-21T08:51:53.015 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_notify": 0, 2024-04-21T08:51:53.015 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T08:51:53.015 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls": 0, 2024-04-21T08:51:53.015 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls_filter": 0, 2024-04-21T08:51:53.015 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_other": 0, 2024-04-21T08:51:53.015 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_active": 0, 2024-04-21T08:51:53.015 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_send": 0, 2024-04-21T08:51:53.015 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_resend": 0, 2024-04-21T08:51:53.015 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_ping": 0, 2024-04-21T08:51:53.015 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_active": 0, 2024-04-21T08:51:53.015 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_send": 0, 2024-04-21T08:51:53.015 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_resend": 0, 2024-04-21T08:51:53.015 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_active": 0, 2024-04-21T08:51:53.015 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_send": 0, 2024-04-21T08:51:53.016 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_resend": 0, 2024-04-21T08:51:53.016 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_active": 0, 2024-04-21T08:51:53.016 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_send": 0, 2024-04-21T08:51:53.016 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_resend": 0, 2024-04-21T08:51:53.016 INFO:teuthology.orchestra.run.smithi179.stdout: "command_active": 0, 2024-04-21T08:51:53.016 INFO:teuthology.orchestra.run.smithi179.stdout: "command_send": 0, 2024-04-21T08:51:53.016 INFO:teuthology.orchestra.run.smithi179.stdout: "command_resend": 0, 2024-04-21T08:51:53.016 INFO:teuthology.orchestra.run.smithi179.stdout: "map_epoch": 108, 2024-04-21T08:51:53.016 INFO:teuthology.orchestra.run.smithi179.stdout: "map_full": 0, 2024-04-21T08:51:53.016 INFO:teuthology.orchestra.run.smithi179.stdout: "map_inc": 19, 2024-04-21T08:51:53.016 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_sessions": 9, 2024-04-21T08:51:53.016 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_open": 13, 2024-04-21T08:51:53.016 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_close": 4, 2024-04-21T08:51:53.016 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_laggy": 0, 2024-04-21T08:51:53.017 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_wr": 14, 2024-04-21T08:51:53.017 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_rd": 0, 2024-04-21T08:51:53.017 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_del": 1 2024-04-21T08:51:53.017 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:53.017 INFO:teuthology.orchestra.run.smithi179.stdout: "oft": { 2024-04-21T08:51:53.017 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_objs": 1, 2024-04-21T08:51:53.017 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_kv_pairs": 0, 2024-04-21T08:51:53.017 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_updates": 0, 2024-04-21T08:51:53.017 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_removes": 0 2024-04-21T08:51:53.017 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:53.017 INFO:teuthology.orchestra.run.smithi179.stdout: "osd-slow-ops": { 2024-04-21T08:51:53.017 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_ops_count": 18 2024-04-21T08:51:53.017 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:53.017 INFO:teuthology.orchestra.run.smithi179.stdout: "purge_queue": { 2024-04-21T08:51:53.017 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops": 0, 2024-04-21T08:51:53.018 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops_high_water": 0, 2024-04-21T08:51:53.018 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing": 0, 2024-04-21T08:51:53.018 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_high_water": 0, 2024-04-21T08:51:53.018 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed_ops": 0, 2024-04-21T08:51:53.018 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed": 0, 2024-04-21T08:51:53.018 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_item_in_journal": 0 2024-04-21T08:51:53.018 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:53.018 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T08:51:53.018 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-04-21T08:51:53.018 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-04-21T08:51:53.018 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-04-21T08:51:53.018 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 2103, 2024-04-21T08:51:53.018 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 1962878, 2024-04-21T08:51:53.018 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:53.019 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 2103, 2024-04-21T08:51:53.019 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-04-21T08:51:53.019 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-04-21T08:51:53.019 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 2103, 2024-04-21T08:51:53.019 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 1962878, 2024-04-21T08:51:53.019 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-04-21T08:51:53.019 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:53.019 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:53.019 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:53.019 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-04-21T08:51:53.019 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:53.019 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_bytes": { 2024-04-21T08:51:53.019 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-04-21T08:51:53.019 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-04-21T08:51:53.020 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-04-21T08:51:53.020 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-04-21T08:51:53.020 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-04-21T08:51:53.020 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:53.020 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:53.020 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 40, 2024-04-21T08:51:53.020 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 11730, 2024-04-21T08:51:53.020 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 40, 2024-04-21T08:51:53.020 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 11730, 2024-04-21T08:51:53.020 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-04-21T08:51:53.020 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:53.020 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:53.020 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:53.020 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-04-21T08:51:53.020 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:53.021 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_ops": { 2024-04-21T08:51:53.021 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-04-21T08:51:53.021 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 1024, 2024-04-21T08:51:53.021 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-04-21T08:51:53.021 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-04-21T08:51:53.021 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-04-21T08:51:53.021 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:53.021 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:53.021 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 40, 2024-04-21T08:51:53.021 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 40, 2024-04-21T08:51:53.021 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 40, 2024-04-21T08:51:53.021 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 40, 2024-04-21T08:51:53.021 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-04-21T08:51:53.021 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:53.021 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:53.022 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:53.022 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-04-21T08:51:53.022 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:53.022 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle": { 2024-04-21T08:51:53.022 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-04-21T08:51:53.022 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-04-21T08:51:53.022 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-04-21T08:51:53.022 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-04-21T08:51:53.022 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-04-21T08:51:53.022 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:53.022 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:53.022 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-04-21T08:51:53.022 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-04-21T08:51:53.022 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 0, 2024-04-21T08:51:53.022 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 0, 2024-04-21T08:51:53.023 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-04-21T08:51:53.023 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:53.023 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:53.023 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:53.023 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-04-21T08:51:53.023 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:53.023 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle-0x5611cd274500": { 2024-04-21T08:51:53.023 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-04-21T08:51:53.023 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-04-21T08:51:53.023 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-04-21T08:51:53.023 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 5, 2024-04-21T08:51:53.023 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 1428, 2024-04-21T08:51:53.023 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:53.023 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 5, 2024-04-21T08:51:53.023 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-04-21T08:51:53.024 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-04-21T08:51:53.024 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 4, 2024-04-21T08:51:53.024 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 1428, 2024-04-21T08:51:53.024 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-04-21T08:51:53.024 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:53.024 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:53.024 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:53.024 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-04-21T08:51:53.024 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-04-21T08:51:53.024 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-04-21T08:51:53.816 INFO:tasks.check_counter:Daemon mds.i mds.exported=0 2024-04-21T08:51:53.816 INFO:tasks.check_counter:Daemon mds.i mds.imported=0 2024-04-21T08:51:53.816 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update=0 2024-04-21T08:51:53.816 INFO:tasks.check_counter:Daemon mds.i mds_cache.dir_update_receipt=19 2024-04-21T08:51:53.816 INFO:tasks.check_counter:Daemon mds.i mds.root_rsnaps=6 2024-04-21T08:51:53.817 INFO:tasks.check_counter:Daemon mds.i mds_server.req_mksnap_latency.avgcount=0 2024-04-21T08:51:53.817 INFO:tasks.check_counter:Daemon mds.i mds_server.req_rmsnap_latency.avgcount=0 2024-04-21T08:51:53.817 DEBUG:tasks.check_counter:Getting stats from l 2024-04-21T08:51:53.817 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm --image quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e shell --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 -- ceph daemon mds.l perf dump 2024-04-21T08:51:54.058 INFO:teuthology.orchestra.run.smithi179.stderr:Inferring config /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/mon.c/config 2024-04-21T08:51:54.179 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:53 smithi179 ceph-mon[31829]: pgmap v410: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-21T08:51:54.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:53 smithi086 ceph-mon[27939]: pgmap v410: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-21T08:51:54.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:53 smithi088 ceph-mon[31259]: pgmap v410: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 6.3 KiB/s rd, 85 B/s wr, 12 op/s 2024-04-21T08:51:55.688 INFO:teuthology.orchestra.run.smithi179.stdout:{ 2024-04-21T08:51:55.688 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-0": { 2024-04-21T08:51:55.688 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 1575, 2024-04-21T08:51:55.688 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 1149, 2024-04-21T08:51:55.688 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 438893, 2024-04-21T08:51:55.688 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 377641, 2024-04-21T08:51:55.689 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 10, 2024-04-21T08:51:55.689 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 4, 2024-04-21T08:51:55.689 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.118736652, 2024-04-21T08:51:55.689 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.045780539, 2024-04-21T08:51:55.689 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.074175470, 2024-04-21T08:51:55.689 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.005810119, 2024-04-21T08:51:55.689 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:55.689 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1150, 2024-04-21T08:51:55.689 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.054972514, 2024-04-21T08:51:55.689 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000047802 2024-04-21T08:51:55.689 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.689 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:55.689 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 2547, 2024-04-21T08:51:55.690 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000535507, 2024-04-21T08:51:55.690 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000210 2024-04-21T08:51:55.690 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.690 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 2992, 2024-04-21T08:51:55.690 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 448 2024-04-21T08:51:55.690 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.690 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-1": { 2024-04-21T08:51:55.690 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 451, 2024-04-21T08:51:55.690 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 437, 2024-04-21T08:51:55.690 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 1692405, 2024-04-21T08:51:55.690 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 813978, 2024-04-21T08:51:55.690 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 63, 2024-04-21T08:51:55.690 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 3, 2024-04-21T08:51:55.690 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.087749507, 2024-04-21T08:51:55.691 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.023697657, 2024-04-21T08:51:55.691 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.064521218, 2024-04-21T08:51:55.691 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.008491692, 2024-04-21T08:51:55.691 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:55.691 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 436, 2024-04-21T08:51:55.691 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.032633000, 2024-04-21T08:51:55.691 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000074846 2024-04-21T08:51:55.691 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.691 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:55.691 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 159, 2024-04-21T08:51:55.691 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000028081, 2024-04-21T08:51:55.691 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000176 2024-04-21T08:51:55.691 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.692 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 1658128, 2024-04-21T08:51:55.692 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 169376 2024-04-21T08:51:55.692 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.692 INFO:teuthology.orchestra.run.smithi179.stdout: "AsyncMessenger::Worker-2": { 2024-04-21T08:51:55.692 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_messages": 31, 2024-04-21T08:51:55.692 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages": 268, 2024-04-21T08:51:55.692 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_bytes": 7285, 2024-04-21T08:51:55.692 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_bytes": 522496, 2024-04-21T08:51:55.692 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_created_connections": 13, 2024-04-21T08:51:55.692 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_active_connections": 5, 2024-04-21T08:51:55.692 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_total_time": 0.026813343, 2024-04-21T08:51:55.692 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_send_time": 0.013110432, 2024-04-21T08:51:55.692 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_recv_time": 0.010366379, 2024-04-21T08:51:55.692 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_running_fast_dispatch_time": 0.000348030, 2024-04-21T08:51:55.692 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_messages_queue_lat": { 2024-04-21T08:51:55.693 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 263, 2024-04-21T08:51:55.693 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.025013230, 2024-04-21T08:51:55.693 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000095107 2024-04-21T08:51:55.693 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.693 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_handle_ack_lat": { 2024-04-21T08:51:55.693 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 59, 2024-04-21T08:51:55.693 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000010992, 2024-04-21T08:51:55.693 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000186 2024-04-21T08:51:55.693 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.693 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_recv_encrypted_bytes": 2912, 2024-04-21T08:51:55.693 INFO:teuthology.orchestra.run.smithi179.stdout: "msgr_send_encrypted_bytes": 514560 2024-04-21T08:51:55.693 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.693 INFO:teuthology.orchestra.run.smithi179.stdout: "cct": { 2024-04-21T08:51:55.693 INFO:teuthology.orchestra.run.smithi179.stdout: "total_workers": 1, 2024-04-21T08:51:55.693 INFO:teuthology.orchestra.run.smithi179.stdout: "unhealthy_workers": 0 2024-04-21T08:51:55.694 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.694 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-MDSRank": { 2024-04-21T08:51:55.694 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-04-21T08:51:55.694 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-04-21T08:51:55.694 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 42, 2024-04-21T08:51:55.694 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.001910965, 2024-04-21T08:51:55.694 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000045499 2024-04-21T08:51:55.694 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-04-21T08:51:55.694 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.694 INFO:teuthology.orchestra.run.smithi179.stdout: "finisher-PurgeQueue": { 2024-04-21T08:51:55.694 INFO:teuthology.orchestra.run.smithi179.stdout: "queue_len": 0, 2024-04-21T08:51:55.694 INFO:teuthology.orchestra.run.smithi179.stdout: "complete_latency": { 2024-04-21T08:51:55.694 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 2, 2024-04-21T08:51:55.694 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000006043, 2024-04-21T08:51:55.695 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000003021 2024-04-21T08:51:55.695 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-04-21T08:51:55.695 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.695 INFO:teuthology.orchestra.run.smithi179.stdout: "mds": { 2024-04-21T08:51:55.695 INFO:teuthology.orchestra.run.smithi179.stdout: "request": 1, 2024-04-21T08:51:55.695 INFO:teuthology.orchestra.run.smithi179.stdout: "reply": 1, 2024-04-21T08:51:55.695 INFO:teuthology.orchestra.run.smithi179.stdout: "reply_latency": { 2024-04-21T08:51:55.695 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1, 2024-04-21T08:51:55.695 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000114106, 2024-04-21T08:51:55.695 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000114106 2024-04-21T08:51:55.695 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.695 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_reply": 0, 2024-04-21T08:51:55.695 INFO:teuthology.orchestra.run.smithi179.stdout: "forward": 0, 2024-04-21T08:51:55.695 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_complete": 0, 2024-04-21T08:51:55.696 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_fetch_keys": 0, 2024-04-21T08:51:55.696 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_commit": 11, 2024-04-21T08:51:55.696 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_split": 0, 2024-04-21T08:51:55.696 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_merge": 0, 2024-04-21T08:51:55.696 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes": 15, 2024-04-21T08:51:55.696 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_top": 0, 2024-04-21T08:51:55.696 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_bottom": 15, 2024-04-21T08:51:55.696 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pin_tail": 0, 2024-04-21T08:51:55.696 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_pinned": 13, 2024-04-21T08:51:55.696 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_expired": 3, 2024-04-21T08:51:55.696 INFO:teuthology.orchestra.run.smithi179.stdout: "inodes_with_caps": 1, 2024-04-21T08:51:55.696 INFO:teuthology.orchestra.run.smithi179.stdout: "caps": 1, 2024-04-21T08:51:55.696 INFO:teuthology.orchestra.run.smithi179.stdout: "subtrees": 3, 2024-04-21T08:51:55.696 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse": 17, 2024-04-21T08:51:55.696 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_hit": 9, 2024-04-21T08:51:55.697 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_forward": 0, 2024-04-21T08:51:55.697 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_discover": 7, 2024-04-21T08:51:55.697 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_dir_fetch": 0, 2024-04-21T08:51:55.697 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_remote_ino": 0, 2024-04-21T08:51:55.697 INFO:teuthology.orchestra.run.smithi179.stdout: "traverse_lock": 0, 2024-04-21T08:51:55.697 INFO:teuthology.orchestra.run.smithi179.stdout: "load_cent": 0, 2024-04-21T08:51:55.697 INFO:teuthology.orchestra.run.smithi179.stdout: "q": 0, 2024-04-21T08:51:55.697 INFO:teuthology.orchestra.run.smithi179.stdout: "exported": 0, 2024-04-21T08:51:55.697 INFO:teuthology.orchestra.run.smithi179.stdout: "exported_inodes": 0, 2024-04-21T08:51:55.697 INFO:teuthology.orchestra.run.smithi179.stdout: "imported": 0, 2024-04-21T08:51:55.697 INFO:teuthology.orchestra.run.smithi179.stdout: "imported_inodes": 0, 2024-04-21T08:51:55.697 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_dir_fetch": 0, 2024-04-21T08:51:55.697 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_backtrace_fetch": 0, 2024-04-21T08:51:55.697 INFO:teuthology.orchestra.run.smithi179.stdout: "openino_peer_discover": 0, 2024-04-21T08:51:55.697 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rfiles": 5, 2024-04-21T08:51:55.697 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rbytes": 298, 2024-04-21T08:51:55.698 INFO:teuthology.orchestra.run.smithi179.stdout: "root_rsnaps": 6, 2024-04-21T08:51:55.698 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_fetch": 0, 2024-04-21T08:51:55.698 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_set_tag": 0, 2024-04-21T08:51:55.698 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_backtrace_repaired": 0, 2024-04-21T08:51:55.698 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_inotable_repaired": 0, 2024-04-21T08:51:55.698 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_inodes": 0, 2024-04-21T08:51:55.698 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dir_base_inodes": 0, 2024-04-21T08:51:55.698 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_dirfrag_rstats": 0, 2024-04-21T08:51:55.698 INFO:teuthology.orchestra.run.smithi179.stdout: "scrub_file_inodes": 0, 2024-04-21T08:51:55.698 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_inode_file_caps": 0, 2024-04-21T08:51:55.698 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_revoke": 1, 2024-04-21T08:51:55.698 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_grant": 0, 2024-04-21T08:51:55.698 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_trunc": 0, 2024-04-21T08:51:55.698 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flushsnap_ack": 0, 2024-04-21T08:51:55.698 INFO:teuthology.orchestra.run.smithi179.stdout: "ceph_cap_op_flush_ack": 0, 2024-04-21T08:51:55.698 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps": 1, 2024-04-21T08:51:55.699 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_caps_dirty": 0, 2024-04-21T08:51:55.699 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_cap_release": 0, 2024-04-21T08:51:55.699 INFO:teuthology.orchestra.run.smithi179.stdout: "process_request_cap_release": 0 2024-04-21T08:51:55.699 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.699 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_cache": { 2024-04-21T08:51:55.699 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update": 0, 2024-04-21T08:51:55.699 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_update_receipt": 14, 2024-04-21T08:51:55.699 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_try_discover": 11, 2024-04-21T08:51:55.699 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_send_discover": 8, 2024-04-21T08:51:55.699 INFO:teuthology.orchestra.run.smithi179.stdout: "dir_handle_discover": 0, 2024-04-21T08:51:55.699 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays": 0, 2024-04-21T08:51:55.699 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_delayed": 0, 2024-04-21T08:51:55.699 INFO:teuthology.orchestra.run.smithi179.stdout: "num_strays_enqueuing": 0, 2024-04-21T08:51:55.699 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_created": 0, 2024-04-21T08:51:55.700 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_enqueued": 0, 2024-04-21T08:51:55.700 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_reintegrated": 0, 2024-04-21T08:51:55.700 INFO:teuthology.orchestra.run.smithi179.stdout: "strays_migrated": 0, 2024-04-21T08:51:55.700 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_processing": 0, 2024-04-21T08:51:55.700 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_enqueued": 0, 2024-04-21T08:51:55.700 INFO:teuthology.orchestra.run.smithi179.stdout: "num_recovering_prioritized": 0, 2024-04-21T08:51:55.700 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_started": 0, 2024-04-21T08:51:55.700 INFO:teuthology.orchestra.run.smithi179.stdout: "recovery_completed": 0, 2024-04-21T08:51:55.700 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_quiesce_path": 3, 2024-04-21T08:51:55.700 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_quiesce_inode": 12, 2024-04-21T08:51:55.700 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_enqueue_scrub": 0, 2024-04-21T08:51:55.700 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_exportdir": 0, 2024-04-21T08:51:55.700 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_flush": 0, 2024-04-21T08:51:55.700 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragmentdir": 0, 2024-04-21T08:51:55.701 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_fragstats": 0, 2024-04-21T08:51:55.701 INFO:teuthology.orchestra.run.smithi179.stdout: "ireq_inodestats": 0 2024-04-21T08:51:55.701 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.701 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_log": { 2024-04-21T08:51:55.701 INFO:teuthology.orchestra.run.smithi179.stdout: "evlrg": 0, 2024-04-21T08:51:55.701 INFO:teuthology.orchestra.run.smithi179.stdout: "evadd": 3, 2024-04-21T08:51:55.701 INFO:teuthology.orchestra.run.smithi179.stdout: "evex": 0, 2024-04-21T08:51:55.701 INFO:teuthology.orchestra.run.smithi179.stdout: "evtrm": 0, 2024-04-21T08:51:55.701 INFO:teuthology.orchestra.run.smithi179.stdout: "ev": 3, 2024-04-21T08:51:55.701 INFO:teuthology.orchestra.run.smithi179.stdout: "evexg": 0, 2024-04-21T08:51:55.701 INFO:teuthology.orchestra.run.smithi179.stdout: "evexd": 0, 2024-04-21T08:51:55.701 INFO:teuthology.orchestra.run.smithi179.stdout: "segadd": 2, 2024-04-21T08:51:55.701 INFO:teuthology.orchestra.run.smithi179.stdout: "segex": 0, 2024-04-21T08:51:55.702 INFO:teuthology.orchestra.run.smithi179.stdout: "segtrm": 0, 2024-04-21T08:51:55.702 INFO:teuthology.orchestra.run.smithi179.stdout: "seg": 2, 2024-04-21T08:51:55.702 INFO:teuthology.orchestra.run.smithi179.stdout: "segmjr": 2, 2024-04-21T08:51:55.702 INFO:teuthology.orchestra.run.smithi179.stdout: "segexg": 0, 2024-04-21T08:51:55.702 INFO:teuthology.orchestra.run.smithi179.stdout: "segexd": 0, 2024-04-21T08:51:55.702 INFO:teuthology.orchestra.run.smithi179.stdout: "expos": 4194304, 2024-04-21T08:51:55.702 INFO:teuthology.orchestra.run.smithi179.stdout: "wrpos": 4195200, 2024-04-21T08:51:55.702 INFO:teuthology.orchestra.run.smithi179.stdout: "rdpos": 0, 2024-04-21T08:51:55.702 INFO:teuthology.orchestra.run.smithi179.stdout: "jlat": { 2024-04-21T08:51:55.702 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 2, 2024-04-21T08:51:55.702 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.003392244, 2024-04-21T08:51:55.702 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.001696122 2024-04-21T08:51:55.702 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.702 INFO:teuthology.orchestra.run.smithi179.stdout: "replayed": 0 2024-04-21T08:51:55.703 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.703 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_mem": { 2024-04-21T08:51:55.703 INFO:teuthology.orchestra.run.smithi179.stdout: "ino": 19, 2024-04-21T08:51:55.703 INFO:teuthology.orchestra.run.smithi179.stdout: "ino+": 20, 2024-04-21T08:51:55.703 INFO:teuthology.orchestra.run.smithi179.stdout: "ino-": 1, 2024-04-21T08:51:55.703 INFO:teuthology.orchestra.run.smithi179.stdout: "dir": 18, 2024-04-21T08:51:55.703 INFO:teuthology.orchestra.run.smithi179.stdout: "dir+": 19, 2024-04-21T08:51:55.703 INFO:teuthology.orchestra.run.smithi179.stdout: "dir-": 1, 2024-04-21T08:51:55.703 INFO:teuthology.orchestra.run.smithi179.stdout: "dn": 15, 2024-04-21T08:51:55.703 INFO:teuthology.orchestra.run.smithi179.stdout: "dn+": 18, 2024-04-21T08:51:55.703 INFO:teuthology.orchestra.run.smithi179.stdout: "dn-": 3, 2024-04-21T08:51:55.703 INFO:teuthology.orchestra.run.smithi179.stdout: "cap": 1, 2024-04-21T08:51:55.703 INFO:teuthology.orchestra.run.smithi179.stdout: "cap+": 1, 2024-04-21T08:51:55.703 INFO:teuthology.orchestra.run.smithi179.stdout: "cap-": 0, 2024-04-21T08:51:55.703 INFO:teuthology.orchestra.run.smithi179.stdout: "rss": 52428, 2024-04-21T08:51:55.703 INFO:teuthology.orchestra.run.smithi179.stdout: "heap": 223608 2024-04-21T08:51:55.704 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.704 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_server": { 2024-04-21T08:51:55.704 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_client_request": 1, 2024-04-21T08:51:55.704 INFO:teuthology.orchestra.run.smithi179.stdout: "dispatch_server_request": 0, 2024-04-21T08:51:55.704 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_request": 1, 2024-04-21T08:51:55.704 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_client_session": 11, 2024-04-21T08:51:55.704 INFO:teuthology.orchestra.run.smithi179.stdout: "handle_peer_request": 12, 2024-04-21T08:51:55.704 INFO:teuthology.orchestra.run.smithi179.stdout: "req_create_latency": { 2024-04-21T08:51:55.704 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:55.704 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:55.704 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:55.704 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.704 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getattr_latency": { 2024-04-21T08:51:55.704 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:55.705 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:55.705 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:55.705 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.705 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getfilelock_latency": { 2024-04-21T08:51:55.705 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:55.705 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:55.705 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:55.705 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.705 INFO:teuthology.orchestra.run.smithi179.stdout: "req_link_latency": { 2024-04-21T08:51:55.705 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:55.705 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:55.705 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:55.705 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.705 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookup_latency": { 2024-04-21T08:51:55.705 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:55.706 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:55.706 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:55.706 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.706 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookuphash_latency": { 2024-04-21T08:51:55.706 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:55.706 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:55.706 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:55.706 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.706 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupino_latency": { 2024-04-21T08:51:55.706 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 1, 2024-04-21T08:51:55.706 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000114106, 2024-04-21T08:51:55.706 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000114106 2024-04-21T08:51:55.706 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.706 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupname_latency": { 2024-04-21T08:51:55.706 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:55.706 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:55.707 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:55.707 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.707 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupparent_latency": { 2024-04-21T08:51:55.707 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:55.707 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:55.707 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:55.707 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.707 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lookupsnap_latency": { 2024-04-21T08:51:55.707 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:55.707 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:55.707 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:55.707 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.707 INFO:teuthology.orchestra.run.smithi179.stdout: "req_lssnap_latency": { 2024-04-21T08:51:55.707 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:55.707 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:55.708 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:55.708 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.708 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mkdir_latency": { 2024-04-21T08:51:55.708 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:55.708 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:55.708 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:55.708 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.708 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mknod_latency": { 2024-04-21T08:51:55.709 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:55.709 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:55.709 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:55.709 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.709 INFO:teuthology.orchestra.run.smithi179.stdout: "req_mksnap_latency": { 2024-04-21T08:51:55.709 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:55.709 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:55.709 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:55.709 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.709 INFO:teuthology.orchestra.run.smithi179.stdout: "req_open_latency": { 2024-04-21T08:51:55.709 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:55.709 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:55.709 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:55.709 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.709 INFO:teuthology.orchestra.run.smithi179.stdout: "req_readdir_latency": { 2024-04-21T08:51:55.710 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:55.710 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:55.710 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:55.710 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.710 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rename_latency": { 2024-04-21T08:51:55.710 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:55.710 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:55.710 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:55.710 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.710 INFO:teuthology.orchestra.run.smithi179.stdout: "req_renamesnap_latency": { 2024-04-21T08:51:55.710 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:55.710 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:55.710 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:55.710 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.710 INFO:teuthology.orchestra.run.smithi179.stdout: "req_snapdiff_latency": { 2024-04-21T08:51:55.711 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:55.711 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:55.711 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:55.711 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.711 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmdir_latency": { 2024-04-21T08:51:55.711 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:55.711 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:55.711 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:55.711 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.711 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmsnap_latency": { 2024-04-21T08:51:55.711 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:55.711 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:55.711 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:55.711 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.712 INFO:teuthology.orchestra.run.smithi179.stdout: "req_rmxattr_latency": { 2024-04-21T08:51:55.712 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:55.712 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:55.712 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:55.712 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.712 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setattr_latency": { 2024-04-21T08:51:55.712 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:55.712 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:55.712 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:55.712 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.712 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setdirlayout_latency": { 2024-04-21T08:51:55.712 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:55.712 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:55.712 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:55.712 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.713 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setfilelock_latency": { 2024-04-21T08:51:55.713 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:55.713 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:55.713 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:55.713 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.713 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setlayout_latency": { 2024-04-21T08:51:55.713 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:55.713 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:55.713 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:55.713 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.713 INFO:teuthology.orchestra.run.smithi179.stdout: "req_setxattr_latency": { 2024-04-21T08:51:55.713 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:55.713 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:55.713 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:55.713 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.714 INFO:teuthology.orchestra.run.smithi179.stdout: "req_symlink_latency": { 2024-04-21T08:51:55.714 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:55.714 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:55.714 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:55.714 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.714 INFO:teuthology.orchestra.run.smithi179.stdout: "req_unlink_latency": { 2024-04-21T08:51:55.714 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:55.714 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:55.714 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:55.714 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.714 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_revoke_eviction": 0, 2024-04-21T08:51:55.714 INFO:teuthology.orchestra.run.smithi179.stdout: "cap_acquisition_throttle": 0, 2024-04-21T08:51:55.714 INFO:teuthology.orchestra.run.smithi179.stdout: "req_getvxattr_latency": { 2024-04-21T08:51:55.714 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:55.714 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:55.714 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:55.715 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-04-21T08:51:55.715 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.715 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_sessions": { 2024-04-21T08:51:55.715 INFO:teuthology.orchestra.run.smithi179.stdout: "session_count": 1, 2024-04-21T08:51:55.715 INFO:teuthology.orchestra.run.smithi179.stdout: "session_add": 1, 2024-04-21T08:51:55.715 INFO:teuthology.orchestra.run.smithi179.stdout: "session_remove": 0, 2024-04-21T08:51:55.715 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_open": 1, 2024-04-21T08:51:55.715 INFO:teuthology.orchestra.run.smithi179.stdout: "sessions_stale": 0, 2024-04-21T08:51:55.715 INFO:teuthology.orchestra.run.smithi179.stdout: "total_load": 1, 2024-04-21T08:51:55.715 INFO:teuthology.orchestra.run.smithi179.stdout: "average_load": 1, 2024-04-21T08:51:55.715 INFO:teuthology.orchestra.run.smithi179.stdout: "avg_session_uptime": 772, 2024-04-21T08:51:55.715 INFO:teuthology.orchestra.run.smithi179.stdout: "mdthresh_evicted": 0 2024-04-21T08:51:55.715 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.715 INFO:teuthology.orchestra.run.smithi179.stdout: "mempool": { 2024-04-21T08:51:55.716 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_bytes": 0, 2024-04-21T08:51:55.716 INFO:teuthology.orchestra.run.smithi179.stdout: "bloom_filter_items": 0, 2024-04-21T08:51:55.716 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_bytes": 0, 2024-04-21T08:51:55.716 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_alloc_items": 0, 2024-04-21T08:51:55.716 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_bytes": 0, 2024-04-21T08:51:55.716 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_data_items": 0, 2024-04-21T08:51:55.716 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_bytes": 0, 2024-04-21T08:51:55.716 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_onode_items": 0, 2024-04-21T08:51:55.716 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_bytes": 0, 2024-04-21T08:51:55.716 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_meta_items": 0, 2024-04-21T08:51:55.716 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_bytes": 0, 2024-04-21T08:51:55.716 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_other_items": 0, 2024-04-21T08:51:55.716 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_bytes": 0, 2024-04-21T08:51:55.716 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_cache_buffer_items": 0, 2024-04-21T08:51:55.716 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_bytes": 0, 2024-04-21T08:51:55.717 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_extent_items": 0, 2024-04-21T08:51:55.717 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_bytes": 0, 2024-04-21T08:51:55.717 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_blob_items": 0, 2024-04-21T08:51:55.717 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_bytes": 0, 2024-04-21T08:51:55.717 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_shared_blob_items": 0, 2024-04-21T08:51:55.717 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_bytes": 0, 2024-04-21T08:51:55.717 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_inline_bl_items": 0, 2024-04-21T08:51:55.717 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_bytes": 0, 2024-04-21T08:51:55.717 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_fsck_items": 0, 2024-04-21T08:51:55.717 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_bytes": 0, 2024-04-21T08:51:55.717 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_txc_items": 0, 2024-04-21T08:51:55.717 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_bytes": 0, 2024-04-21T08:51:55.717 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_deferred_items": 0, 2024-04-21T08:51:55.717 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_bytes": 0, 2024-04-21T08:51:55.717 INFO:teuthology.orchestra.run.smithi179.stdout: "bluestore_writing_items": 0, 2024-04-21T08:51:55.718 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_bytes": 0, 2024-04-21T08:51:55.718 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_items": 0, 2024-04-21T08:51:55.718 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_bytes": 0, 2024-04-21T08:51:55.718 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_reader_items": 0, 2024-04-21T08:51:55.718 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_bytes": 0, 2024-04-21T08:51:55.718 INFO:teuthology.orchestra.run.smithi179.stdout: "bluefs_file_writer_items": 0, 2024-04-21T08:51:55.718 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_bytes": 287159, 2024-04-21T08:51:55.718 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_anon_items": 94, 2024-04-21T08:51:55.718 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_bytes": 0, 2024-04-21T08:51:55.718 INFO:teuthology.orchestra.run.smithi179.stdout: "buffer_meta_items": 0, 2024-04-21T08:51:55.718 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_bytes": 0, 2024-04-21T08:51:55.718 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_items": 0, 2024-04-21T08:51:55.718 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_bytes": 0, 2024-04-21T08:51:55.719 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_mapbl_items": 0, 2024-04-21T08:51:55.719 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_bytes": 0, 2024-04-21T08:51:55.719 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_pglog_items": 0, 2024-04-21T08:51:55.719 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_bytes": 9512, 2024-04-21T08:51:55.719 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_items": 182, 2024-04-21T08:51:55.719 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_bytes": 0, 2024-04-21T08:51:55.719 INFO:teuthology.orchestra.run.smithi179.stdout: "osdmap_mapping_items": 0, 2024-04-21T08:51:55.719 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_bytes": 0, 2024-04-21T08:51:55.719 INFO:teuthology.orchestra.run.smithi179.stdout: "pgmap_items": 0, 2024-04-21T08:51:55.719 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_bytes": 83285, 2024-04-21T08:51:55.719 INFO:teuthology.orchestra.run.smithi179.stdout: "mds_co_items": 326, 2024-04-21T08:51:55.719 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_bytes": 0, 2024-04-21T08:51:55.719 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_1_items": 0, 2024-04-21T08:51:55.719 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_bytes": 0, 2024-04-21T08:51:55.720 INFO:teuthology.orchestra.run.smithi179.stdout: "unittest_2_items": 0 2024-04-21T08:51:55.720 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.720 INFO:teuthology.orchestra.run.smithi179.stdout: "objecter": { 2024-04-21T08:51:55.720 INFO:teuthology.orchestra.run.smithi179.stdout: "op_active": 0, 2024-04-21T08:51:55.720 INFO:teuthology.orchestra.run.smithi179.stdout: "op_laggy": 0, 2024-04-21T08:51:55.720 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send": 38, 2024-04-21T08:51:55.720 INFO:teuthology.orchestra.run.smithi179.stdout: "op_send_bytes": 11319, 2024-04-21T08:51:55.720 INFO:teuthology.orchestra.run.smithi179.stdout: "op_resend": 33, 2024-04-21T08:51:55.720 INFO:teuthology.orchestra.run.smithi179.stdout: "op_reply": 38, 2024-04-21T08:51:55.720 INFO:teuthology.orchestra.run.smithi179.stdout: "op_latency": { 2024-04-21T08:51:55.720 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 38, 2024-04-21T08:51:55.720 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.204995672, 2024-04-21T08:51:55.720 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.005394622 2024-04-21T08:51:55.720 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.720 INFO:teuthology.orchestra.run.smithi179.stdout: "op_inflight": 0, 2024-04-21T08:51:55.721 INFO:teuthology.orchestra.run.smithi179.stdout: "oplen_avg": { 2024-04-21T08:51:55.721 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 38, 2024-04-21T08:51:55.721 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 61 2024-04-21T08:51:55.721 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.721 INFO:teuthology.orchestra.run.smithi179.stdout: "op": 38, 2024-04-21T08:51:55.721 INFO:teuthology.orchestra.run.smithi179.stdout: "op_r": 0, 2024-04-21T08:51:55.721 INFO:teuthology.orchestra.run.smithi179.stdout: "op_w": 38, 2024-04-21T08:51:55.721 INFO:teuthology.orchestra.run.smithi179.stdout: "op_rmw": 0, 2024-04-21T08:51:55.721 INFO:teuthology.orchestra.run.smithi179.stdout: "op_pg": 0, 2024-04-21T08:51:55.721 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_stat": 0, 2024-04-21T08:51:55.721 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_create": 10, 2024-04-21T08:51:55.721 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_read": 0, 2024-04-21T08:51:55.721 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_write": 2, 2024-04-21T08:51:55.721 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writefull": 6, 2024-04-21T08:51:55.721 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_writesame": 0, 2024-04-21T08:51:55.722 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_append": 0, 2024-04-21T08:51:55.722 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_zero": 0, 2024-04-21T08:51:55.722 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_truncate": 0, 2024-04-21T08:51:55.722 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_delete": 6, 2024-04-21T08:51:55.722 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_mapext": 0, 2024-04-21T08:51:55.722 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_sparse_read": 0, 2024-04-21T08:51:55.722 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_clonerange": 0, 2024-04-21T08:51:55.722 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_getxattr": 0, 2024-04-21T08:51:55.722 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_setxattr": 20, 2024-04-21T08:51:55.722 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_cmpxattr": 0, 2024-04-21T08:51:55.722 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_rmxattr": 0, 2024-04-21T08:51:55.722 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_resetxattrs": 0, 2024-04-21T08:51:55.722 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_call": 0, 2024-04-21T08:51:55.722 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_watch": 0, 2024-04-21T08:51:55.723 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_notify": 0, 2024-04-21T08:51:55.723 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_src_cmpxattr": 0, 2024-04-21T08:51:55.723 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls": 0, 2024-04-21T08:51:55.723 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_pgls_filter": 0, 2024-04-21T08:51:55.723 INFO:teuthology.orchestra.run.smithi179.stdout: "osdop_other": 0, 2024-04-21T08:51:55.723 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_active": 0, 2024-04-21T08:51:55.723 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_send": 0, 2024-04-21T08:51:55.723 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_resend": 0, 2024-04-21T08:51:55.723 INFO:teuthology.orchestra.run.smithi179.stdout: "linger_ping": 0, 2024-04-21T08:51:55.723 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_active": 0, 2024-04-21T08:51:55.723 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_send": 0, 2024-04-21T08:51:55.723 INFO:teuthology.orchestra.run.smithi179.stdout: "poolop_resend": 0, 2024-04-21T08:51:55.723 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_active": 0, 2024-04-21T08:51:55.723 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_send": 0, 2024-04-21T08:51:55.723 INFO:teuthology.orchestra.run.smithi179.stdout: "poolstat_resend": 0, 2024-04-21T08:51:55.723 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_active": 0, 2024-04-21T08:51:55.724 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_send": 0, 2024-04-21T08:51:55.724 INFO:teuthology.orchestra.run.smithi179.stdout: "statfs_resend": 0, 2024-04-21T08:51:55.724 INFO:teuthology.orchestra.run.smithi179.stdout: "command_active": 0, 2024-04-21T08:51:55.724 INFO:teuthology.orchestra.run.smithi179.stdout: "command_send": 0, 2024-04-21T08:51:55.724 INFO:teuthology.orchestra.run.smithi179.stdout: "command_resend": 0, 2024-04-21T08:51:55.724 INFO:teuthology.orchestra.run.smithi179.stdout: "map_epoch": 108, 2024-04-21T08:51:55.724 INFO:teuthology.orchestra.run.smithi179.stdout: "map_full": 0, 2024-04-21T08:51:55.724 INFO:teuthology.orchestra.run.smithi179.stdout: "map_inc": 19, 2024-04-21T08:51:55.724 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_sessions": 9, 2024-04-21T08:51:55.724 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_open": 14, 2024-04-21T08:51:55.724 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_session_close": 5, 2024-04-21T08:51:55.724 INFO:teuthology.orchestra.run.smithi179.stdout: "osd_laggy": 0, 2024-04-21T08:51:55.724 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_wr": 16, 2024-04-21T08:51:55.724 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_rd": 0, 2024-04-21T08:51:55.724 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_del": 1 2024-04-21T08:51:55.724 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.725 INFO:teuthology.orchestra.run.smithi179.stdout: "oft": { 2024-04-21T08:51:55.725 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_objs": 1, 2024-04-21T08:51:55.725 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_kv_pairs": 0, 2024-04-21T08:51:55.725 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_updates": 0, 2024-04-21T08:51:55.725 INFO:teuthology.orchestra.run.smithi179.stdout: "omap_total_removes": 0 2024-04-21T08:51:55.725 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.725 INFO:teuthology.orchestra.run.smithi179.stdout: "osd-slow-ops": { 2024-04-21T08:51:55.725 INFO:teuthology.orchestra.run.smithi179.stdout: "slow_ops_count": 15 2024-04-21T08:51:55.725 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.725 INFO:teuthology.orchestra.run.smithi179.stdout: "purge_queue": { 2024-04-21T08:51:55.725 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops": 0, 2024-04-21T08:51:55.725 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_ops_high_water": 0, 2024-04-21T08:51:55.725 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing": 0, 2024-04-21T08:51:55.725 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executing_high_water": 0, 2024-04-21T08:51:55.726 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed_ops": 0, 2024-04-21T08:51:55.726 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_executed": 0, 2024-04-21T08:51:55.726 INFO:teuthology.orchestra.run.smithi179.stdout: "pq_item_in_journal": 0 2024-04-21T08:51:55.726 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.726 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-msgr_dispatch_throttler-mds": { 2024-04-21T08:51:55.726 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-04-21T08:51:55.726 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-04-21T08:51:55.726 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-04-21T08:51:55.726 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 2053, 2024-04-21T08:51:55.726 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 1933634, 2024-04-21T08:51:55.726 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:55.726 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 2053, 2024-04-21T08:51:55.726 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-04-21T08:51:55.726 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-04-21T08:51:55.727 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 2053, 2024-04-21T08:51:55.727 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 1933634, 2024-04-21T08:51:55.727 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-04-21T08:51:55.727 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:55.727 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:55.727 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:55.727 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-04-21T08:51:55.727 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.727 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_bytes": { 2024-04-21T08:51:55.727 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-04-21T08:51:55.727 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 104857600, 2024-04-21T08:51:55.727 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-04-21T08:51:55.727 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-04-21T08:51:55.727 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-04-21T08:51:55.727 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:55.727 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:55.728 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 38, 2024-04-21T08:51:55.728 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 11319, 2024-04-21T08:51:55.728 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 38, 2024-04-21T08:51:55.728 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 11319, 2024-04-21T08:51:55.728 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-04-21T08:51:55.728 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:55.728 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:55.728 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:55.728 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-04-21T08:51:55.728 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.728 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-objecter_ops": { 2024-04-21T08:51:55.728 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-04-21T08:51:55.728 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 1024, 2024-04-21T08:51:55.728 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-04-21T08:51:55.729 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-04-21T08:51:55.729 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-04-21T08:51:55.729 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:55.729 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:55.729 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 38, 2024-04-21T08:51:55.729 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 38, 2024-04-21T08:51:55.729 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 38, 2024-04-21T08:51:55.729 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 38, 2024-04-21T08:51:55.729 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-04-21T08:51:55.729 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:55.729 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:55.729 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:55.729 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-04-21T08:51:55.729 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.730 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle": { 2024-04-21T08:51:55.730 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-04-21T08:51:55.730 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-04-21T08:51:55.730 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-04-21T08:51:55.730 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 0, 2024-04-21T08:51:55.730 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 0, 2024-04-21T08:51:55.730 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:55.730 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 0, 2024-04-21T08:51:55.730 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-04-21T08:51:55.730 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-04-21T08:51:55.730 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 0, 2024-04-21T08:51:55.730 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 0, 2024-04-21T08:51:55.730 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-04-21T08:51:55.730 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:55.731 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:55.731 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:55.731 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-04-21T08:51:55.731 INFO:teuthology.orchestra.run.smithi179.stdout: }, 2024-04-21T08:51:55.731 INFO:teuthology.orchestra.run.smithi179.stdout: "throttle-write_buf_throttle-0x55b6acc66500": { 2024-04-21T08:51:55.731 INFO:teuthology.orchestra.run.smithi179.stdout: "val": 0, 2024-04-21T08:51:55.731 INFO:teuthology.orchestra.run.smithi179.stdout: "max": 3758096384, 2024-04-21T08:51:55.731 INFO:teuthology.orchestra.run.smithi179.stdout: "get_started": 0, 2024-04-21T08:51:55.731 INFO:teuthology.orchestra.run.smithi179.stdout: "get": 3, 2024-04-21T08:51:55.731 INFO:teuthology.orchestra.run.smithi179.stdout: "get_sum": 896, 2024-04-21T08:51:55.731 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_fail": 0, 2024-04-21T08:51:55.731 INFO:teuthology.orchestra.run.smithi179.stdout: "get_or_fail_success": 3, 2024-04-21T08:51:55.731 INFO:teuthology.orchestra.run.smithi179.stdout: "take": 0, 2024-04-21T08:51:55.731 INFO:teuthology.orchestra.run.smithi179.stdout: "take_sum": 0, 2024-04-21T08:51:55.731 INFO:teuthology.orchestra.run.smithi179.stdout: "put": 2, 2024-04-21T08:51:55.732 INFO:teuthology.orchestra.run.smithi179.stdout: "put_sum": 896, 2024-04-21T08:51:55.732 INFO:teuthology.orchestra.run.smithi179.stdout: "wait": { 2024-04-21T08:51:55.732 INFO:teuthology.orchestra.run.smithi179.stdout: "avgcount": 0, 2024-04-21T08:51:55.732 INFO:teuthology.orchestra.run.smithi179.stdout: "sum": 0.000000000, 2024-04-21T08:51:55.732 INFO:teuthology.orchestra.run.smithi179.stdout: "avgtime": 0.000000000 2024-04-21T08:51:55.732 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-04-21T08:51:55.732 INFO:teuthology.orchestra.run.smithi179.stdout: } 2024-04-21T08:51:55.732 INFO:teuthology.orchestra.run.smithi179.stdout:} 2024-04-21T08:51:56.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:55 smithi179 ceph-mon[31829]: pgmap v411: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-21T08:51:56.367 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:55 smithi086 ceph-mon[27939]: pgmap v411: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-21T08:51:56.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:55 smithi088 ceph-mon[31259]: pgmap v411: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-21T08:51:56.514 INFO:tasks.check_counter:Daemon mds.l mds.exported=0 2024-04-21T08:51:56.515 INFO:tasks.check_counter:Daemon mds.l mds.imported=0 2024-04-21T08:51:56.515 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update=0 2024-04-21T08:51:56.515 INFO:tasks.check_counter:Daemon mds.l mds_cache.dir_update_receipt=14 2024-04-21T08:51:56.515 INFO:tasks.check_counter:Daemon mds.l mds.root_rsnaps=6 2024-04-21T08:51:56.515 INFO:tasks.check_counter:Daemon mds.l mds_server.req_mksnap_latency.avgcount=0 2024-04-21T08:51:56.515 INFO:tasks.check_counter:Daemon mds.l mds_server.req_rmsnap_latency.avgcount=0 2024-04-21T08:51:56.515 DEBUG:teuthology.run_tasks:Unwinding manager kclient 2024-04-21T08:51:56.528 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-21T08:51:56.528 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-21T08:51:56.529 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-21T08:51:56.550 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-21T08:51:56.551 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-21T08:51:56.609 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:51:56.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.762+0000 7fdb0e19f640 1 -- 172.21.15.86:0/1282427150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb08107050 msgr2=0x7fdb0810ec20 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:56.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.762+0000 7fdb0e19f640 1 --2- 172.21.15.86:0/1282427150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb08107050 0x7fdb0810ec20 secure :-1 s=READY pgs=2142 cs=0 l=1 rev1=1 crypto rx=0x7fdb00009ca0 tx=0x7fdb0002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:51:56.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.762+0000 7fdb0e19f640 1 -- 172.21.15.86:0/1282427150 shutdown_connections 2024-04-21T08:51:56.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.762+0000 7fdb0e19f640 1 --2- 172.21.15.86:0/1282427150 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb08107050 0x7fdb0810ec20 unknown :-1 s=CLOSED pgs=2142 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:56.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.762+0000 7fdb0e19f640 1 --2- 172.21.15.86:0/1282427150 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb08102930 0x7fdb08106970 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:56.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.762+0000 7fdb0e19f640 1 --2- 172.21.15.86:0/1282427150 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb08101f80 0x7fdb08102360 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:56.764 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.762+0000 7fdb0e19f640 1 -- 172.21.15.86:0/1282427150 >> 172.21.15.86:0/1282427150 conn(0x7fdb080fd820 msgr2=0x7fdb080ffc40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:56.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.762+0000 7fdb0e19f640 1 -- 172.21.15.86:0/1282427150 shutdown_connections 2024-04-21T08:51:56.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.762+0000 7fdb0e19f640 1 -- 172.21.15.86:0/1282427150 wait complete. 2024-04-21T08:51:56.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.763+0000 7fdb0e19f640 1 Processor -- start 2024-04-21T08:51:56.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.763+0000 7fdb0e19f640 1 -- start start 2024-04-21T08:51:56.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.763+0000 7fdb0e19f640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb08101f80 0x7fdb081984f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:56.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.763+0000 7fdb0e19f640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb08107050 0x7fdb08198a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:56.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.763+0000 7fdb0e19f640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb0819cd60 0x7fdb0819d210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:56.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.763+0000 7fdb0e19f640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fdb08111fb0 con 0x7fdb0819cd60 2024-04-21T08:51:56.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.763+0000 7fdb0e19f640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fdb08111e30 con 0x7fdb08107050 2024-04-21T08:51:56.765 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.763+0000 7fdb0e19f640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fdb08112130 con 0x7fdb08101f80 2024-04-21T08:51:56.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.763+0000 7fdb0d99e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb0819cd60 0x7fdb0819d210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:56.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.763+0000 7fdb0d99e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb0819cd60 0x7fdb0819d210 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35864/0 (socket says 172.21.15.86:35864) 2024-04-21T08:51:56.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.763+0000 7fdb0d99e640 1 -- 172.21.15.86:0/668629746 learned_addr learned my addr 172.21.15.86:0/668629746 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:56.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.763+0000 7fdb0c99c640 1 --2- 172.21.15.86:0/668629746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb08107050 0x7fdb08198a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:56.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.764+0000 7fdb0d19d640 1 --2- 172.21.15.86:0/668629746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb08101f80 0x7fdb081984f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:56.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.764+0000 7fdb0d99e640 1 -- 172.21.15.86:0/668629746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb08101f80 msgr2=0x7fdb081984f0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:56.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.764+0000 7fdb0d99e640 1 --2- 172.21.15.86:0/668629746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb08101f80 0x7fdb081984f0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:56.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.764+0000 7fdb0d99e640 1 -- 172.21.15.86:0/668629746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb08107050 msgr2=0x7fdb08198a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:56.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.764+0000 7fdb0d99e640 1 --2- 172.21.15.86:0/668629746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb08107050 0x7fdb08198a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:56.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.764+0000 7fdb0d99e640 1 -- 172.21.15.86:0/668629746 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fdb0819d850 con 0x7fdb0819cd60 2024-04-21T08:51:56.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.764+0000 7fdb0d19d640 1 --2- 172.21.15.86:0/668629746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb08101f80 0x7fdb081984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:51:56.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.764+0000 7fdb0d99e640 1 --2- 172.21.15.86:0/668629746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb0819cd60 0x7fdb0819d210 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7fdb00005e30 tx=0x7fdb00032420 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:56.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.764+0000 7fdafa7fc640 1 -- 172.21.15.86:0/668629746 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb00041020 con 0x7fdb0819cd60 2024-04-21T08:51:56.766 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.764+0000 7fdb0e19f640 1 -- 172.21.15.86:0/668629746 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fdb0819dae0 con 0x7fdb0819cd60 2024-04-21T08:51:56.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.764+0000 7fdafa7fc640 1 -- 172.21.15.86:0/668629746 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fdb00032970 con 0x7fdb0819cd60 2024-04-21T08:51:56.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.764+0000 7fdafa7fc640 1 -- 172.21.15.86:0/668629746 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fdb00039040 con 0x7fdb0819cd60 2024-04-21T08:51:56.767 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.764+0000 7fdb0e19f640 1 -- 172.21.15.86:0/668629746 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fdb081a52a0 con 0x7fdb0819cd60 2024-04-21T08:51:56.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.765+0000 7fdb0e19f640 1 -- 172.21.15.86:0/668629746 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fdb081997f0 con 0x7fdb0819cd60 2024-04-21T08:51:56.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.766+0000 7fdafa7fc640 1 -- 172.21.15.86:0/668629746 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fdb00044020 con 0x7fdb0819cd60 2024-04-21T08:51:56.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.767+0000 7fdafa7fc640 1 --2- 172.21.15.86:0/668629746 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdaf40a3d20 0x7fdaf40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:56.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.767+0000 7fdb0d19d640 1 --2- 172.21.15.86:0/668629746 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdaf40a3d20 0x7fdaf40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:56.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.767+0000 7fdafa7fc640 1 -- 172.21.15.86:0/668629746 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7fdb000f2c40 con 0x7fdb0819cd60 2024-04-21T08:51:56.768 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.767+0000 7fdb0d19d640 1 --2- 172.21.15.86:0/668629746 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdaf40a3d20 0x7fdaf40a61e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fdafc009b50 tx=0x7fdafc005f90 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:56.770 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.771+0000 7fdafa7fc640 1 -- 172.21.15.86:0/668629746 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fdb0004b020 con 0x7fdb0819cd60 2024-04-21T08:51:56.953 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.954+0000 7fdb0e19f640 1 -- 172.21.15.86:0/668629746 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7fdb08102930 con 0x7fdb0819cd60 2024-04-21T08:51:56.955 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.956+0000 7fdafa7fc640 1 -- 172.21.15.86:0/668629746 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) ==== 74+0+25253 (secure 0 0 0) 0x7fdb000bf020 con 0x7fdb0819cd60 2024-04-21T08:51:56.955 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:51:56.955 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":108,"fsid":"f8cec7cc-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:22.687329+0000","modified":"2024-04-21T08:51:10.626236+0000","last_up_change":"2024-04-21T08:35:30.048315+0000","last_in_change":"2024-04-21T08:35:10.099496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:45.279109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:18.300782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:19.574636+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":21,"snap_epoch":107,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:22.905632+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":21,"snap_epoch":107,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89b7601f-0620-4db2-90f1-b45cea173738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6803","nonce":2571516897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6805","nonce":2571516897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6809","nonce":2571516897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6807","nonce":2571516897}]},"public_addr":"172.21.15.86:6803/2571516897","cluster_addr":"172.21.15.86:6805/2571516897","heartbeat_back_addr":"172.21.15.86:6809/2571516897","heartbeat_front_addr":"172.21.15.86:6807/2571516897","state":["exists","up"]},{"osd":1,"uuid":"4891031c-f98c-42bc-bb3b-a420855ecc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6801","nonce":74067182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6803","nonce":74067182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6807","nonce":74067182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6805","nonce":74067182}]},"public_addr":"172.21.15.88:6801/74067182","cluster_addr":"172.21.15.88:6803/74067182","heartbeat_back_addr":"172.21.15.88:6807/74067182","heartbeat_front_addr":"172.21.15.88:6805/74067182","state":["exists","up"]},{"osd":2,"uuid":"60d12aa4-9312-4fa2-8d0b-2b3f42caad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6801","nonce":4137659051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6803","nonce":4137659051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6807","nonce":4137659051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6805","nonce":4137659051}]},"public_addr":"172.21.15.179:6801/4137659051","cluster_addr":"172.21.15.179:6803/4137659051","heartbeat_back_addr":"172.21.15.179:6807/4137659051","heartbeat_front_addr":"172.21.15.179:6805/4137659051","state":["exists","up"]},{"osd":3,"uuid":"404c3158-8cb0-4c02-b48a-f9ee5ff7b784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6811","nonce":2309603148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6813","nonce":2309603148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6817","nonce":2309603148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6815","nonce":2309603148}]},"public_addr":"172.21.15.86:6811/2309603148","cluster_addr":"172.21.15.86:6813/2309603148","heartbeat_back_addr":"172.21.15.86:6817/2309603148","heartbeat_front_addr":"172.21.15.86:6815/2309603148","state":["exists","up"]},{"osd":4,"uuid":"458ff202-d781-4ec7-ac93-0b49f835b503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6809","nonce":1818714217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6811","nonce":1818714217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6815","nonce":1818714217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6813","nonce":1818714217}]},"public_addr":"172.21.15.88:6809/1818714217","cluster_addr":"172.21.15.88:6811/1818714217","heartbeat_back_addr":"172.21.15.88:6815/1818714217","heartbeat_front_addr":"172.21.15.88:6813/1818714217","state":["exists","up"]},{"osd":5,"uuid":"8123cc90-bd8b-4204-bf31-46b34e9f589d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6809","nonce":863840990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6811","nonce":863840990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6815","nonce":863840990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6813","nonce":863840990}]},"public_addr":"172.21.15.179:6809/863840990","cluster_addr":"172.21.15.179:6811/863840990","heartbeat_back_addr":"172.21.15.179:6815/863840990","heartbeat_front_addr":"172.21.15.179:6813/863840990","state":["exists","up"]},{"osd":6,"uuid":"aac9d7c7-659a-4110-903a-a871c086066f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6819","nonce":1224231118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6821","nonce":1224231118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6825","nonce":1224231118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6823","nonce":1224231118}]},"public_addr":"172.21.15.86:6819/1224231118","cluster_addr":"172.21.15.86:6821/1224231118","heartbeat_back_addr":"172.21.15.86:6825/1224231118","heartbeat_front_addr":"172.21.15.86:6823/1224231118","state":["exists","up"]},{"osd":7,"uuid":"df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6817","nonce":136629878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6819","nonce":136629878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6823","nonce":136629878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6821","nonce":136629878}]},"public_addr":"172.21.15.88:6817/136629878","cluster_addr":"172.21.15.88:6819/136629878","heartbeat_back_addr":"172.21.15.88:6823/136629878","heartbeat_front_addr":"172.21.15.88:6821/136629878","state":["exists","up"]},{"osd":8,"uuid":"63e59f4b-0426-46db-92f7-c55dae61b5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6817","nonce":1770418957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6819","nonce":1770418957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6823","nonce":1770418957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6821","nonce":1770418957}]},"public_addr":"172.21.15.179:6817/1770418957","cluster_addr":"172.21.15.179:6819/1770418957","heartbeat_back_addr":"172.21.15.179:6823/1770418957","heartbeat_front_addr":"172.21.15.179:6821/1770418957","state":["exists","up"]},{"osd":9,"uuid":"5841bb36-ed6b-45d8-b857-eeab139c44e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6827","nonce":1108500274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6829","nonce":1108500274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6833","nonce":1108500274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6831","nonce":1108500274}]},"public_addr":"172.21.15.86:6827/1108500274","cluster_addr":"172.21.15.86:6829/1108500274","heartbeat_back_addr":"172.21.15.86:6833/1108500274","heartbeat_front_addr":"172.21.15.86:6831/1108500274","state":["exists","up"]},{"osd":10,"uuid":"f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6825","nonce":1294275353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6827","nonce":1294275353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6831","nonce":1294275353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6829","nonce":1294275353}]},"public_addr":"172.21.15.88:6825/1294275353","cluster_addr":"172.21.15.88:6827/1294275353","heartbeat_back_addr":"172.21.15.88:6831/1294275353","heartbeat_front_addr":"172.21.15.88:6829/1294275353","state":["exists","up"]},{"osd":11,"uuid":"e0d4cc76-e7a6-4a6b-909c-95488ccd5a36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6825","nonce":4007070233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6827","nonce":4007070233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6831","nonce":4007070233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6829","nonce":4007070233}]},"public_addr":"172.21.15.179:6825/4007070233","cluster_addr":"172.21.15.179:6827/4007070233","heartbeat_back_addr":"172.21.15.179:6831/4007070233","heartbeat_front_addr":"172.21.15.179:6829/4007070233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:55.210605+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:18.723090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:42.547979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:06.421622+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:30.858315+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:55.455313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.623369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:46.185111+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.103627+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:37.297545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:02.637431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:28.199292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1068089552":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/4273877219":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/2707594358":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/4060044103":"2024-04-22T08:38:46.631621+0000","172.21.15.86:6800/1364156965":"2024-04-22T08:38:46.631621+0000","172.21.15.86:6800/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/1364156965":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/1412393225":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6800/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/1687852404":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2020707404":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2568130933":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/3952607648":"2024-04-22T08:28:57.384403+0000","172.21.15.86:6801/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/2756903426":"2024-04-22T08:29:23.190001+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":8,"length":1},{"begin":21,"length":1}]},{"pool":4,"snaps":[{"begin":8,"length":1},{"begin":21,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:51:56.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.958+0000 7fdb0e19f640 1 -- 172.21.15.86:0/668629746 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdaf40a3d20 msgr2=0x7fdaf40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:56.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.958+0000 7fdb0e19f640 1 --2- 172.21.15.86:0/668629746 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdaf40a3d20 0x7fdaf40a61e0 secure :-1 s=READY pgs=365 cs=0 l=1 rev1=1 crypto rx=0x7fdafc009b50 tx=0x7fdafc005f90 comp rx=0 tx=0).stop 2024-04-21T08:51:56.957 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.958+0000 7fdb0e19f640 1 -- 172.21.15.86:0/668629746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb0819cd60 msgr2=0x7fdb0819d210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:56.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.958+0000 7fdb0e19f640 1 --2- 172.21.15.86:0/668629746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb0819cd60 0x7fdb0819d210 secure :-1 s=READY pgs=2143 cs=0 l=1 rev1=1 crypto rx=0x7fdb00005e30 tx=0x7fdb00032420 comp rx=0 tx=0).stop 2024-04-21T08:51:56.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.958+0000 7fdb0e19f640 1 -- 172.21.15.86:0/668629746 shutdown_connections 2024-04-21T08:51:56.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.958+0000 7fdb0e19f640 1 --2- 172.21.15.86:0/668629746 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fdaf40a3d20 0x7fdaf40a61e0 unknown :-1 s=CLOSED pgs=365 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:56.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.958+0000 7fdb0e19f640 1 --2- 172.21.15.86:0/668629746 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fdb0819cd60 0x7fdb0819d210 unknown :-1 s=CLOSED pgs=2143 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:56.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.958+0000 7fdb0e19f640 1 --2- 172.21.15.86:0/668629746 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fdb08107050 0x7fdb08198a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:56.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.958+0000 7fdb0e19f640 1 --2- 172.21.15.86:0/668629746 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fdb08101f80 0x7fdb081984f0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:56.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.958+0000 7fdb0e19f640 1 -- 172.21.15.86:0/668629746 >> 172.21.15.86:0/668629746 conn(0x7fdb080fd820 msgr2=0x7fdb080ffbc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:56.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.958+0000 7fdb0e19f640 1 -- 172.21.15.86:0/668629746 shutdown_connections 2024-04-21T08:51:56.958 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:56.958+0000 7fdb0e19f640 1 -- 172.21.15.86:0/668629746 wait complete. 2024-04-21T08:51:56.968 INFO:tasks.cephfs.filesystem.ceph_manager:[{'pool': 1, 'pool_name': '.mgr', 'create_time': '2024-04-21T08:31:45.279109+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 1, 'pg_placement_num': 1, 'pg_placement_num_target': 1, 'pg_num_target': 1, 'pg_num_pending': 1, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '21', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_max': 32, 'pg_num_min': 1}, 'application_metadata': {'mgr': {}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 12, 'score_stable': 12, 'optimal_score': 0.25, 'raw_score_acting': 3, 'raw_score_stable': 3, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 2, 'pool_name': 'cephfs_metadata', 'create_time': '2024-04-21T08:36:18.300782+0000', 'flags': 1, 'flags_names': 'hashpspool', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '84', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 0, 'snap_epoch': 0, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_autoscale_bias': 4, 'pg_num_min': 16, 'recovery_priority': 5}, 'application_metadata': {'cephfs': {'metadata': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.8799999952316284, 'score_stable': 1.8799999952316284, 'optimal_score': 1, 'raw_score_acting': 1.8799999952316284, 'raw_score_stable': 1.8799999952316284, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 3, 'pool_name': 'cephfs_data', 'create_time': '2024-04-21T08:36:19.574636+0000', 'flags': 8193, 'flags_names': 'hashpspool,selfmanaged_snaps', 'type': 1, 'size': 3, 'min_size': 2, 'crush_rule': 0, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 64, 'pg_placement_num': 64, 'pg_placement_num_target': 64, 'pg_num_target': 64, 'pg_num_pending': 64, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 21, 'snap_epoch': 107, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': '', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 0, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.05000000074505806}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}, 'read_balance': {'score_type': 'Fair distribution', 'score_acting': 1.5, 'score_stable': 1.5, 'optimal_score': 1, 'raw_score_acting': 1.5, 'raw_score_stable': 1.5, 'primary_affinity_weighted': 0.9999998807907104, 'average_primary_affinity': 1, 'average_primary_affinity_weighted': 1}}, {'pool': 4, 'pool_name': 'cephfs_data_ec', 'create_time': '2024-04-21T08:36:22.905632+0000', 'flags': 8197, 'flags_names': 'hashpspool,ec_overwrites,selfmanaged_snaps', 'type': 3, 'size': 4, 'min_size': 3, 'crush_rule': 1, 'peering_crush_bucket_count': 0, 'peering_crush_bucket_target': 0, 'peering_crush_bucket_barrier': 0, 'peering_crush_bucket_mandatory_member': 2147483647, 'object_hash': 2, 'pg_autoscale_mode': 'off', 'pg_num': 32, 'pg_placement_num': 32, 'pg_placement_num_target': 32, 'pg_num_target': 32, 'pg_num_pending': 32, 'last_pg_merge_meta': {'source_pgid': '0.0', 'ready_epoch': 0, 'last_epoch_started': 0, 'last_epoch_clean': 0, 'source_version': "0'0", 'target_version': "0'0"}, 'last_change': '107', 'last_force_op_resend': '0', 'last_force_op_resend_prenautilus': '0', 'last_force_op_resend_preluminous': '0', 'auid': 0, 'snap_mode': 'selfmanaged', 'snap_seq': 21, 'snap_epoch': 107, 'pool_snaps': [], 'removed_snaps': '[]', 'quota_max_bytes': 0, 'quota_max_objects': 0, 'tiers': [], 'tier_of': -1, 'read_tier': -1, 'write_tier': -1, 'cache_mode': 'none', 'target_max_bytes': 0, 'target_max_objects': 0, 'cache_target_dirty_ratio_micro': 400000, 'cache_target_dirty_high_ratio_micro': 600000, 'cache_target_full_ratio_micro': 800000, 'cache_min_flush_age': 0, 'cache_min_evict_age': 0, 'erasure_code_profile': 'cephfs_data_ec', 'hit_set_params': {'type': 'none'}, 'hit_set_period': 0, 'hit_set_count': 0, 'use_gmt_hitset': True, 'min_read_recency_for_promote': 0, 'min_write_recency_for_promote': 0, 'hit_set_grade_decay_rate': 0, 'hit_set_search_last_n': 0, 'grade_table': [], 'stripe_width': 8192, 'expected_num_objects': 0, 'fast_read': False, 'options': {'pg_num_min': 64, 'target_size_ratio': 0.8999999761581421}, 'application_metadata': {'cephfs': {'data': 'cephfs'}}}] 2024-04-21T08:51:56.969 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get .mgr pg_num 2024-04-21T08:51:57.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.071+0000 7fd581b7e640 1 -- 172.21.15.86:0/1776893102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd57c106170 msgr2=0x7fd57c10dd40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:57.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.071+0000 7fd581b7e640 1 --2- 172.21.15.86:0/1776893102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd57c106170 0x7fd57c10dd40 secure :-1 s=READY pgs=2144 cs=0 l=1 rev1=1 crypto rx=0x7fd574009ca0 tx=0x7fd57402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:51:57.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.071+0000 7fd581b7e640 1 -- 172.21.15.86:0/1776893102 shutdown_connections 2024-04-21T08:51:57.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.071+0000 7fd581b7e640 1 --2- 172.21.15.86:0/1776893102 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd57c106170 0x7fd57c10dd40 unknown :-1 s=CLOSED pgs=2144 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:57.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.071+0000 7fd581b7e640 1 --2- 172.21.15.86:0/1776893102 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd57c101a50 0x7fd57c105a90 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:57.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.071+0000 7fd581b7e640 1 --2- 172.21.15.86:0/1776893102 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd57c1010a0 0x7fd57c101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:57.071 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.071+0000 7fd581b7e640 1 -- 172.21.15.86:0/1776893102 >> 172.21.15.86:0/1776893102 conn(0x7fd57c077eb0 msgr2=0x7fd57c0782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:57.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.071+0000 7fd581b7e640 1 -- 172.21.15.86:0/1776893102 shutdown_connections 2024-04-21T08:51:57.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.071+0000 7fd581b7e640 1 -- 172.21.15.86:0/1776893102 wait complete. 2024-04-21T08:51:57.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.072+0000 7fd581b7e640 1 Processor -- start 2024-04-21T08:51:57.072 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.072+0000 7fd581b7e640 1 -- start start 2024-04-21T08:51:57.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.072+0000 7fd581b7e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd57c1010a0 0x7fd57c1975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:57.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.072+0000 7fd581b7e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd57c106170 0x7fd57c197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:57.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.072+0000 7fd581b7e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd57c19be80 0x7fd57c19c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:57.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.072+0000 7fd581b7e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fd57c1110c0 con 0x7fd57c106170 2024-04-21T08:51:57.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.072+0000 7fd581b7e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fd57c110f40 con 0x7fd57c1010a0 2024-04-21T08:51:57.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.072+0000 7fd581b7e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fd57c111240 con 0x7fd57c19be80 2024-04-21T08:51:57.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.072+0000 7fd57affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd57c106170 0x7fd57c197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:57.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.072+0000 7fd57affd640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd57c106170 0x7fd57c197b00 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35886/0 (socket says 172.21.15.86:35886) 2024-04-21T08:51:57.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.072+0000 7fd57affd640 1 -- 172.21.15.86:0/3288191530 learned_addr learned my addr 172.21.15.86:0/3288191530 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:57.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.072+0000 7fd57b7fe640 1 --2- 172.21.15.86:0/3288191530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd57c1010a0 0x7fd57c1975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:57.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.072+0000 7fd57bfff640 1 --2- 172.21.15.86:0/3288191530 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd57c19be80 0x7fd57c19c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:57.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.073+0000 7fd57affd640 1 -- 172.21.15.86:0/3288191530 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd57c19be80 msgr2=0x7fd57c19c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:57.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.073+0000 7fd57affd640 1 --2- 172.21.15.86:0/3288191530 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd57c19be80 0x7fd57c19c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:57.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.073+0000 7fd57affd640 1 -- 172.21.15.86:0/3288191530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd57c1010a0 msgr2=0x7fd57c1975c0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:57.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.073+0000 7fd57affd640 1 --2- 172.21.15.86:0/3288191530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd57c1010a0 0x7fd57c1975c0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:57.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.073+0000 7fd57affd640 1 -- 172.21.15.86:0/3288191530 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fd57c19c970 con 0x7fd57c106170 2024-04-21T08:51:57.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.073+0000 7fd57bfff640 1 --2- 172.21.15.86:0/3288191530 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd57c19be80 0x7fd57c19c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:51:57.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.073+0000 7fd57b7fe640 1 --2- 172.21.15.86:0/3288191530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd57c1010a0 0x7fd57c1975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:57.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.073+0000 7fd57affd640 1 --2- 172.21.15.86:0/3288191530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd57c106170 0x7fd57c197b00 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7fd57000e9c0 tx=0x7fd57000ee80 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:57.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.073+0000 7fd578ff9640 1 -- 172.21.15.86:0/3288191530 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd57000cd80 con 0x7fd57c106170 2024-04-21T08:51:57.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.073+0000 7fd578ff9640 1 -- 172.21.15.86:0/3288191530 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fd5700054d0 con 0x7fd57c106170 2024-04-21T08:51:57.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.073+0000 7fd581b7e640 1 -- 172.21.15.86:0/3288191530 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fd57c19cc60 con 0x7fd57c106170 2024-04-21T08:51:57.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.073+0000 7fd578ff9640 1 -- 172.21.15.86:0/3288191530 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fd570010430 con 0x7fd57c106170 2024-04-21T08:51:57.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.073+0000 7fd581b7e640 1 -- 172.21.15.86:0/3288191530 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fd57c1a4450 con 0x7fd57c106170 2024-04-21T08:51:57.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.074+0000 7fd581b7e640 1 -- 172.21.15.86:0/3288191530 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fd57c0691c0 con 0x7fd57c106170 2024-04-21T08:51:57.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.075+0000 7fd578ff9640 1 -- 172.21.15.86:0/3288191530 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fd570004fd0 con 0x7fd57c106170 2024-04-21T08:51:57.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.076+0000 7fd578ff9640 1 --2- 172.21.15.86:0/3288191530 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd5640a3d20 0x7fd5640a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:57.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.076+0000 7fd57b7fe640 1 --2- 172.21.15.86:0/3288191530 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd5640a3d20 0x7fd5640a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:57.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.076+0000 7fd578ff9640 1 -- 172.21.15.86:0/3288191530 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7fd5700d1080 con 0x7fd57c106170 2024-04-21T08:51:57.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.076+0000 7fd57b7fe640 1 --2- 172.21.15.86:0/3288191530 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd5640a3d20 0x7fd5640a61e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd56c005fd0 tx=0x7fd56c00b5f0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:57.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.080+0000 7fd578ff9640 1 -- 172.21.15.86:0/3288191530 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fd57009d350 con 0x7fd57c106170 2024-04-21T08:51:57.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:56 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/668629746' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:51:57.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.254+0000 7fd581b7e640 1 -- 172.21.15.86:0/3288191530 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"} v 0) -- 0x7fd57c101a50 con 0x7fd57c106170 2024-04-21T08:51:57.255 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.255+0000 7fd578ff9640 1 -- 172.21.15.86:0/3288191530 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]=0 v108) ==== 93+0+10 (secure 0 0 0) 0x7fd57009cc50 con 0x7fd57c106170 2024-04-21T08:51:57.255 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 1 2024-04-21T08:51:57.257 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.257+0000 7fd581b7e640 1 -- 172.21.15.86:0/3288191530 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd5640a3d20 msgr2=0x7fd5640a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:57.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.257+0000 7fd581b7e640 1 --2- 172.21.15.86:0/3288191530 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd5640a3d20 0x7fd5640a61e0 secure :-1 s=READY pgs=366 cs=0 l=1 rev1=1 crypto rx=0x7fd56c005fd0 tx=0x7fd56c00b5f0 comp rx=0 tx=0).stop 2024-04-21T08:51:57.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.257+0000 7fd581b7e640 1 -- 172.21.15.86:0/3288191530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd57c106170 msgr2=0x7fd57c197b00 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:57.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.257+0000 7fd581b7e640 1 --2- 172.21.15.86:0/3288191530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd57c106170 0x7fd57c197b00 secure :-1 s=READY pgs=2145 cs=0 l=1 rev1=1 crypto rx=0x7fd57000e9c0 tx=0x7fd57000ee80 comp rx=0 tx=0).stop 2024-04-21T08:51:57.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.257+0000 7fd581b7e640 1 -- 172.21.15.86:0/3288191530 shutdown_connections 2024-04-21T08:51:57.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.257+0000 7fd581b7e640 1 --2- 172.21.15.86:0/3288191530 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fd5640a3d20 0x7fd5640a61e0 unknown :-1 s=CLOSED pgs=366 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:57.258 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.257+0000 7fd581b7e640 1 --2- 172.21.15.86:0/3288191530 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fd57c19be80 0x7fd57c19c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:57.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.257+0000 7fd581b7e640 1 --2- 172.21.15.86:0/3288191530 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fd57c106170 0x7fd57c197b00 unknown :-1 s=CLOSED pgs=2145 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:57.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.257+0000 7fd581b7e640 1 --2- 172.21.15.86:0/3288191530 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fd57c1010a0 0x7fd57c1975c0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:57.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.257+0000 7fd581b7e640 1 -- 172.21.15.86:0/3288191530 >> 172.21.15.86:0/3288191530 conn(0x7fd57c077eb0 msgr2=0x7fd57c100570 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:57.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.257+0000 7fd581b7e640 1 -- 172.21.15.86:0/3288191530 shutdown_connections 2024-04-21T08:51:57.259 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.257+0000 7fd581b7e640 1 -- 172.21.15.86:0/3288191530 wait complete. 2024-04-21T08:51:57.266 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:56 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/668629746' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:51:57.267 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_metadata pg_num 2024-04-21T08:51:57.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.373+0000 7f40b4fa4640 1 -- 172.21.15.86:0/3410396678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40b0101a50 msgr2=0x7f40b0105a90 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:57.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.373+0000 7f40b4fa4640 1 --2- 172.21.15.86:0/3410396678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40b0101a50 0x7f40b0105a90 secure :-1 s=READY pgs=2146 cs=0 l=1 rev1=1 crypto rx=0x7f40a4009ca0 tx=0x7f40a402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:51:57.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.373+0000 7f40b4fa4640 1 -- 172.21.15.86:0/3410396678 shutdown_connections 2024-04-21T08:51:57.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.373+0000 7f40b4fa4640 1 --2- 172.21.15.86:0/3410396678 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40b0106170 0x7f40b010dd40 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:57.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.373+0000 7f40b4fa4640 1 --2- 172.21.15.86:0/3410396678 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40b0101a50 0x7f40b0105a90 unknown :-1 s=CLOSED pgs=2146 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:57.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.373+0000 7f40b4fa4640 1 --2- 172.21.15.86:0/3410396678 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40b01010a0 0x7f40b0101480 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:57.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.373+0000 7f40b4fa4640 1 -- 172.21.15.86:0/3410396678 >> 172.21.15.86:0/3410396678 conn(0x7f40b0077eb0 msgr2=0x7f40b00782c0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:57.376 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.373+0000 7f40b4fa4640 1 -- 172.21.15.86:0/3410396678 shutdown_connections 2024-04-21T08:51:57.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.373+0000 7f40b4fa4640 1 -- 172.21.15.86:0/3410396678 wait complete. 2024-04-21T08:51:57.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.374+0000 7f40b4fa4640 1 Processor -- start 2024-04-21T08:51:57.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.374+0000 7f40b4fa4640 1 -- start start 2024-04-21T08:51:57.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.374+0000 7f40b4fa4640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40b01010a0 0x7f40b0199610 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:57.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.374+0000 7f40b4fa4640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40b0106170 0x7f40b0199b50 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:57.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.374+0000 7f40b4fa4640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40b019a0e0 0x7f40b01a4690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:57.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.374+0000 7f40b4fa4640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f40b01110b0 con 0x7f40b019a0e0 2024-04-21T08:51:57.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.374+0000 7f40b4fa4640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f40b0110f30 con 0x7f40b0106170 2024-04-21T08:51:57.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.374+0000 7f40b4fa4640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f40b0111230 con 0x7f40b01010a0 2024-04-21T08:51:57.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.374+0000 7f40aed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40b019a0e0 0x7f40b01a4690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:57.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.374+0000 7f40aed9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40b019a0e0 0x7f40b01a4690 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35902/0 (socket says 172.21.15.86:35902) 2024-04-21T08:51:57.377 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.374+0000 7f40aed9d640 1 -- 172.21.15.86:0/341359503 learned_addr learned my addr 172.21.15.86:0/341359503 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:57.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.374+0000 7f40ae59c640 1 --2- 172.21.15.86:0/341359503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40b01010a0 0x7f40b0199610 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:57.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.374+0000 7f40add9b640 1 --2- 172.21.15.86:0/341359503 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40b0106170 0x7f40b0199b50 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:57.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.374+0000 7f40aed9d640 1 -- 172.21.15.86:0/341359503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40b01010a0 msgr2=0x7f40b0199610 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:57.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.374+0000 7f40aed9d640 1 --2- 172.21.15.86:0/341359503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40b01010a0 0x7f40b0199610 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:57.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.374+0000 7f40aed9d640 1 -- 172.21.15.86:0/341359503 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40b0106170 msgr2=0x7f40b0199b50 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:57.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.374+0000 7f40aed9d640 1 --2- 172.21.15.86:0/341359503 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40b0106170 0x7f40b0199b50 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:57.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.374+0000 7f40aed9d640 1 -- 172.21.15.86:0/341359503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f40b01a4e60 con 0x7f40b019a0e0 2024-04-21T08:51:57.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.374+0000 7f40ae59c640 1 --2- 172.21.15.86:0/341359503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40b01010a0 0x7f40b0199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:57.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.374+0000 7f40aed9d640 1 --2- 172.21.15.86:0/341359503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40b019a0e0 0x7f40b01a4690 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f40a000d750 tx=0x7f40a000dc10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:57.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.374+0000 7f409f7fe640 1 -- 172.21.15.86:0/341359503 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a0017070 con 0x7f40b019a0e0 2024-04-21T08:51:57.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.375+0000 7f409f7fe640 1 -- 172.21.15.86:0/341359503 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f40a00054d0 con 0x7f40b019a0e0 2024-04-21T08:51:57.378 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.375+0000 7f409f7fe640 1 -- 172.21.15.86:0/341359503 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f40a0004c60 con 0x7f40b019a0e0 2024-04-21T08:51:57.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.375+0000 7f40b4fa4640 1 -- 172.21.15.86:0/341359503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f40b01a5150 con 0x7f40b019a0e0 2024-04-21T08:51:57.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.375+0000 7f40b4fa4640 1 -- 172.21.15.86:0/341359503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f40b01a5650 con 0x7f40b019a0e0 2024-04-21T08:51:57.379 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.376+0000 7f40b4fa4640 1 -- 172.21.15.86:0/341359503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f40b00691c0 con 0x7f40b019a0e0 2024-04-21T08:51:57.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.377+0000 7f409f7fe640 1 -- 172.21.15.86:0/341359503 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f40a0009bd0 con 0x7f40b019a0e0 2024-04-21T08:51:57.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.377+0000 7f409f7fe640 1 --2- 172.21.15.86:0/341359503 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f40980a3d20 0x7f40980a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:57.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.377+0000 7f409f7fe640 1 -- 172.21.15.86:0/341359503 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7f40a00d0a20 con 0x7f40b019a0e0 2024-04-21T08:51:57.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.377+0000 7f40ae59c640 1 --2- 172.21.15.86:0/341359503 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f40980a3d20 0x7f40980a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:57.380 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.378+0000 7f40ae59c640 1 --2- 172.21.15.86:0/341359503 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f40980a3d20 0x7f40980a61e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f40a80058c0 tx=0x7f40a8005850 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:57.381 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.381+0000 7f409f7fe640 1 -- 172.21.15.86:0/341359503 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f40a009ccf0 con 0x7f40b019a0e0 2024-04-21T08:51:57.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:56 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/668629746' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:51:57.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.560+0000 7f40b4fa4640 1 -- 172.21.15.86:0/341359503 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"} v 0) -- 0x7f40b01093c0 con 0x7f40b019a0e0 2024-04-21T08:51:57.561 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.561+0000 7f409f7fe640 1 -- 172.21.15.86:0/341359503 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]=0 v108) ==== 104+0+11 (secure 0 0 0) 0x7f40a009c5f0 con 0x7f40b019a0e0 2024-04-21T08:51:57.562 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-21T08:51:57.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.563+0000 7f40b4fa4640 1 -- 172.21.15.86:0/341359503 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f40980a3d20 msgr2=0x7f40980a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:57.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.563+0000 7f40b4fa4640 1 --2- 172.21.15.86:0/341359503 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f40980a3d20 0x7f40980a61e0 secure :-1 s=READY pgs=367 cs=0 l=1 rev1=1 crypto rx=0x7f40a80058c0 tx=0x7f40a8005850 comp rx=0 tx=0).stop 2024-04-21T08:51:57.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.563+0000 7f40b4fa4640 1 -- 172.21.15.86:0/341359503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40b019a0e0 msgr2=0x7f40b01a4690 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:57.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.563+0000 7f40b4fa4640 1 --2- 172.21.15.86:0/341359503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40b019a0e0 0x7f40b01a4690 secure :-1 s=READY pgs=2147 cs=0 l=1 rev1=1 crypto rx=0x7f40a000d750 tx=0x7f40a000dc10 comp rx=0 tx=0).stop 2024-04-21T08:51:57.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.563+0000 7f40b4fa4640 1 -- 172.21.15.86:0/341359503 shutdown_connections 2024-04-21T08:51:57.565 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.563+0000 7f40b4fa4640 1 --2- 172.21.15.86:0/341359503 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f40980a3d20 0x7f40980a61e0 unknown :-1 s=CLOSED pgs=367 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:57.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.563+0000 7f40b4fa4640 1 --2- 172.21.15.86:0/341359503 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f40b019a0e0 0x7f40b01a4690 unknown :-1 s=CLOSED pgs=2147 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:57.566 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.563+0000 7f40b4fa4640 1 --2- 172.21.15.86:0/341359503 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f40b0106170 0x7f40b0199b50 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:57.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.563+0000 7f40b4fa4640 1 --2- 172.21.15.86:0/341359503 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f40b01010a0 0x7f40b0199610 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:57.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.563+0000 7f40b4fa4640 1 -- 172.21.15.86:0/341359503 >> 172.21.15.86:0/341359503 conn(0x7f40b0077eb0 msgr2=0x7f40b00ff150 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:57.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.563+0000 7f40b4fa4640 1 -- 172.21.15.86:0/341359503 shutdown_connections 2024-04-21T08:51:57.568 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.563+0000 7f40b4fa4640 1 -- 172.21.15.86:0/341359503 wait complete. 2024-04-21T08:51:57.573 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data pg_num 2024-04-21T08:51:57.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.675+0000 7f5154375640 1 -- 172.21.15.86:0/4011383291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f514c1014a0 msgr2=0x7f514c106bd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:57.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.675+0000 7f5154375640 1 --2- 172.21.15.86:0/4011383291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f514c1014a0 0x7f514c106bd0 secure :-1 s=READY pgs=2148 cs=0 l=1 rev1=1 crypto rx=0x7f513c009900 tx=0x7f513c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:51:57.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.675+0000 7f5154375640 1 -- 172.21.15.86:0/4011383291 shutdown_connections 2024-04-21T08:51:57.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.675+0000 7f5154375640 1 --2- 172.21.15.86:0/4011383291 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f514c1072b0 0x7f514c10eec0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:57.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.675+0000 7f5154375640 1 --2- 172.21.15.86:0/4011383291 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f514c1014a0 0x7f514c106bd0 unknown :-1 s=CLOSED pgs=2148 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:57.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.675+0000 7f5154375640 1 --2- 172.21.15.86:0/4011383291 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f514c100af0 0x7f514c100ed0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:57.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.675+0000 7f5154375640 1 -- 172.21.15.86:0/4011383291 >> 172.21.15.86:0/4011383291 conn(0x7f514c0760e0 msgr2=0x7f514c0764f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:57.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.675+0000 7f5154375640 1 -- 172.21.15.86:0/4011383291 shutdown_connections 2024-04-21T08:51:57.677 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.675+0000 7f5154375640 1 -- 172.21.15.86:0/4011383291 wait complete. 2024-04-21T08:51:57.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.676+0000 7f5154375640 1 Processor -- start 2024-04-21T08:51:57.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.676+0000 7f5154375640 1 -- start start 2024-04-21T08:51:57.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.676+0000 7f5154375640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f514c100af0 0x7f514c101a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:57.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.676+0000 7f5154375640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f514c1072b0 0x7f514c101fa0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:57.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.676+0000 7f5154375640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f514c1024e0 0x7f514c1a69a0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:57.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.676+0000 7f5154375640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f514c112210 con 0x7f514c1024e0 2024-04-21T08:51:57.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.676+0000 7f5154375640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f514c112090 con 0x7f514c1072b0 2024-04-21T08:51:57.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.676+0000 7f5154375640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f514c112390 con 0x7f514c100af0 2024-04-21T08:51:57.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.676+0000 7f5152912640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f514c1024e0 0x7f514c1a69a0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:57.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.676+0000 7f5152912640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f514c1024e0 0x7f514c1a69a0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35918/0 (socket says 172.21.15.86:35918) 2024-04-21T08:51:57.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.676+0000 7f5152912640 1 -- 172.21.15.86:0/4244638579 learned_addr learned my addr 172.21.15.86:0/4244638579 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:57.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.676+0000 7f5151910640 1 --2- 172.21.15.86:0/4244638579 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f514c1072b0 0x7f514c101fa0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:57.678 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.676+0000 7f5152111640 1 --2- 172.21.15.86:0/4244638579 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f514c100af0 0x7f514c101a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:57.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.676+0000 7f5152912640 1 -- 172.21.15.86:0/4244638579 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f514c100af0 msgr2=0x7f514c101a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:57.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.676+0000 7f5152912640 1 --2- 172.21.15.86:0/4244638579 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f514c100af0 0x7f514c101a60 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:57.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.676+0000 7f5152912640 1 -- 172.21.15.86:0/4244638579 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f514c1072b0 msgr2=0x7f514c101fa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:57.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.676+0000 7f5152912640 1 --2- 172.21.15.86:0/4244638579 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f514c1072b0 0x7f514c101fa0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:57.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.676+0000 7f5152912640 1 -- 172.21.15.86:0/4244638579 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f514c1a70a0 con 0x7f514c1024e0 2024-04-21T08:51:57.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.676+0000 7f5152912640 1 --2- 172.21.15.86:0/4244638579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f514c1024e0 0x7f514c1a69a0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f514800eaa0 tx=0x7f514800ef60 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:57.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.676+0000 7f51437fe640 1 -- 172.21.15.86:0/4244638579 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f514800cdb0 con 0x7f514c1024e0 2024-04-21T08:51:57.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.676+0000 7f51437fe640 1 -- 172.21.15.86:0/4244638579 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5148005020 con 0x7f514c1024e0 2024-04-21T08:51:57.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.676+0000 7f5154375640 1 -- 172.21.15.86:0/4244638579 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f514c1a7390 con 0x7f514c1024e0 2024-04-21T08:51:57.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.677+0000 7f51437fe640 1 -- 172.21.15.86:0/4244638579 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f51480116d0 con 0x7f514c1024e0 2024-04-21T08:51:57.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.677+0000 7f5154375640 1 -- 172.21.15.86:0/4244638579 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f514c1a7860 con 0x7f514c1024e0 2024-04-21T08:51:57.679 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.678+0000 7f5154375640 1 -- 172.21.15.86:0/4244638579 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f514c0691c0 con 0x7f514c1024e0 2024-04-21T08:51:57.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.679+0000 7f51437fe640 1 -- 172.21.15.86:0/4244638579 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f51480051d0 con 0x7f514c1024e0 2024-04-21T08:51:57.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.679+0000 7f51437fe640 1 --2- 172.21.15.86:0/4244638579 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f51380a3d50 0x7f51380a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:57.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.679+0000 7f5152111640 1 --2- 172.21.15.86:0/4244638579 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f51380a3d50 0x7f51380a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:57.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.679+0000 7f51437fe640 1 -- 172.21.15.86:0/4244638579 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7f51480d0c40 con 0x7f514c1024e0 2024-04-21T08:51:57.680 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.680+0000 7f5152111640 1 --2- 172.21.15.86:0/4244638579 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f51380a3d50 0x7f51380a6210 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f514400bbe0 tx=0x7f514400b390 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:57.683 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.684+0000 7f51437fe640 1 -- 172.21.15.86:0/4244638579 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5148016030 con 0x7f514c1024e0 2024-04-21T08:51:57.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.862+0000 7f5154375640 1 -- 172.21.15.86:0/4244638579 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"} v 0) -- 0x7f514c10a4d0 con 0x7f514c1024e0 2024-04-21T08:51:57.862 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.863+0000 7f51437fe640 1 -- 172.21.15.86:0/4244638579 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]=0 v108) ==== 100+0+11 (secure 0 0 0) 0x7f514809c8c0 con 0x7f514c1024e0 2024-04-21T08:51:57.863 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 64 2024-04-21T08:51:57.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.865+0000 7f5154375640 1 -- 172.21.15.86:0/4244638579 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f51380a3d50 msgr2=0x7f51380a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:57.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.865+0000 7f5154375640 1 --2- 172.21.15.86:0/4244638579 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f51380a3d50 0x7f51380a6210 secure :-1 s=READY pgs=368 cs=0 l=1 rev1=1 crypto rx=0x7f514400bbe0 tx=0x7f514400b390 comp rx=0 tx=0).stop 2024-04-21T08:51:57.864 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.865+0000 7f5154375640 1 -- 172.21.15.86:0/4244638579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f514c1024e0 msgr2=0x7f514c1a69a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:57.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.865+0000 7f5154375640 1 --2- 172.21.15.86:0/4244638579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f514c1024e0 0x7f514c1a69a0 secure :-1 s=READY pgs=2149 cs=0 l=1 rev1=1 crypto rx=0x7f514800eaa0 tx=0x7f514800ef60 comp rx=0 tx=0).stop 2024-04-21T08:51:57.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.865+0000 7f5154375640 1 -- 172.21.15.86:0/4244638579 shutdown_connections 2024-04-21T08:51:57.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.865+0000 7f5154375640 1 --2- 172.21.15.86:0/4244638579 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f51380a3d50 0x7f51380a6210 unknown :-1 s=CLOSED pgs=368 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:57.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.865+0000 7f5154375640 1 --2- 172.21.15.86:0/4244638579 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f514c1024e0 0x7f514c1a69a0 unknown :-1 s=CLOSED pgs=2149 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:57.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.865+0000 7f5154375640 1 --2- 172.21.15.86:0/4244638579 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f514c1072b0 0x7f514c101fa0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:57.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.865+0000 7f5154375640 1 --2- 172.21.15.86:0/4244638579 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f514c100af0 0x7f514c101a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:57.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.865+0000 7f5154375640 1 -- 172.21.15.86:0/4244638579 >> 172.21.15.86:0/4244638579 conn(0x7f514c0760e0 msgr2=0x7f514c0fec30 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:57.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.865+0000 7f5154375640 1 -- 172.21.15.86:0/4244638579 shutdown_connections 2024-04-21T08:51:57.865 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.865+0000 7f5154375640 1 -- 172.21.15.86:0/4244638579 wait complete. 2024-04-21T08:51:57.875 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get cephfs_data_ec pg_num 2024-04-21T08:51:57.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.977+0000 7f1f95ea5640 1 -- 172.21.15.86:0/1485808197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f90101a40 msgr2=0x7f1f90105a80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:57.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.977+0000 7f1f95ea5640 1 --2- 172.21.15.86:0/1485808197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f90101a40 0x7f1f90105a80 secure :-1 s=READY pgs=2150 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009900 tx=0x7f1f7802bc30 comp rx=0 tx=0).stop 2024-04-21T08:51:57.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.977+0000 7f1f95ea5640 1 -- 172.21.15.86:0/1485808197 shutdown_connections 2024-04-21T08:51:57.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.977+0000 7f1f95ea5640 1 --2- 172.21.15.86:0/1485808197 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f90106160 0x7f1f9010dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:57.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.977+0000 7f1f95ea5640 1 --2- 172.21.15.86:0/1485808197 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f90101a40 0x7f1f90105a80 unknown :-1 s=CLOSED pgs=2150 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:57.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.977+0000 7f1f95ea5640 1 --2- 172.21.15.86:0/1485808197 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f90101090 0x7f1f90101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:57.979 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.977+0000 7f1f95ea5640 1 -- 172.21.15.86:0/1485808197 >> 172.21.15.86:0/1485808197 conn(0x7f1f90077f50 msgr2=0x7f1f90078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:57.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.977+0000 7f1f95ea5640 1 -- 172.21.15.86:0/1485808197 shutdown_connections 2024-04-21T08:51:57.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.977+0000 7f1f95ea5640 1 -- 172.21.15.86:0/1485808197 wait complete. 2024-04-21T08:51:57.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.978+0000 7f1f95ea5640 1 Processor -- start 2024-04-21T08:51:57.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.978+0000 7f1f95ea5640 1 -- start start 2024-04-21T08:51:57.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.978+0000 7f1f95ea5640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f90101090 0x7f1f901975c0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:57.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.978+0000 7f1f95ea5640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f90106160 0x7f1f90197b00 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:57.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.978+0000 7f1f95ea5640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f9019be80 0x7f1f9019c330 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:57.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.978+0000 7f1f95ea5640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f1f901110a0 con 0x7f1f90101090 2024-04-21T08:51:57.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.978+0000 7f1f95ea5640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f1f90110f20 con 0x7f1f9019be80 2024-04-21T08:51:57.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.978+0000 7f1f95ea5640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f1f90111220 con 0x7f1f90106160 2024-04-21T08:51:57.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.978+0000 7f1f8f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f90101090 0x7f1f901975c0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:57.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.978+0000 7f1f8f7fe640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f90101090 0x7f1f901975c0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35940/0 (socket says 172.21.15.86:35940) 2024-04-21T08:51:57.980 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.978+0000 7f1f8f7fe640 1 -- 172.21.15.86:0/268369212 learned_addr learned my addr 172.21.15.86:0/268369212 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:57.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.978+0000 7f1f87fff640 1 --2- 172.21.15.86:0/268369212 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f90106160 0x7f1f90197b00 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:57.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.978+0000 7f1f8ffff640 1 --2- 172.21.15.86:0/268369212 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f9019be80 0x7f1f9019c330 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:57.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.978+0000 7f1f8f7fe640 1 -- 172.21.15.86:0/268369212 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f90106160 msgr2=0x7f1f90197b00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:57.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.978+0000 7f1f8f7fe640 1 --2- 172.21.15.86:0/268369212 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f90106160 0x7f1f90197b00 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:57.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.978+0000 7f1f8f7fe640 1 -- 172.21.15.86:0/268369212 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f9019be80 msgr2=0x7f1f9019c330 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:57.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.978+0000 7f1f8f7fe640 1 --2- 172.21.15.86:0/268369212 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f9019be80 0x7f1f9019c330 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:57.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.978+0000 7f1f8f7fe640 1 -- 172.21.15.86:0/268369212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f1f9019c970 con 0x7f1f90101090 2024-04-21T08:51:57.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.978+0000 7f1f8ffff640 1 --2- 172.21.15.86:0/268369212 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f9019be80 0x7f1f9019c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:51:57.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.978+0000 7f1f8f7fe640 1 --2- 172.21.15.86:0/268369212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f90101090 0x7f1f901975c0 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f1f8000c920 tx=0x7f1f8000cde0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:57.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.979+0000 7f1f8d7fa640 1 -- 172.21.15.86:0/268369212 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f80007c70 con 0x7f1f90101090 2024-04-21T08:51:57.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.979+0000 7f1f8d7fa640 1 -- 172.21.15.86:0/268369212 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f1f80007e10 con 0x7f1f90101090 2024-04-21T08:51:57.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.979+0000 7f1f8d7fa640 1 -- 172.21.15.86:0/268369212 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f1f80018820 con 0x7f1f90101090 2024-04-21T08:51:57.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.979+0000 7f1f95ea5640 1 -- 172.21.15.86:0/268369212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f1f9019cc60 con 0x7f1f90101090 2024-04-21T08:51:57.981 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.979+0000 7f1f95ea5640 1 -- 172.21.15.86:0/268369212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f1f901a4420 con 0x7f1f90101090 2024-04-21T08:51:57.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.980+0000 7f1f95ea5640 1 -- 172.21.15.86:0/268369212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f1f90068d70 con 0x7f1f90101090 2024-04-21T08:51:57.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.981+0000 7f1f8d7fa640 1 -- 172.21.15.86:0/268369212 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f1f8000f450 con 0x7f1f90101090 2024-04-21T08:51:57.982 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.981+0000 7f1f8d7fa640 1 --2- 172.21.15.86:0/268369212 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1f7c0a3d20 0x7f1f7c0a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:57.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.981+0000 7f1f8d7fa640 1 -- 172.21.15.86:0/268369212 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7f1f800d0fc0 con 0x7f1f90101090 2024-04-21T08:51:57.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.982+0000 7f1f87fff640 1 --2- 172.21.15.86:0/268369212 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1f7c0a3d20 0x7f1f7c0a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:57.983 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.982+0000 7f1f87fff640 1 --2- 172.21.15.86:0/268369212 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1f7c0a3d20 0x7f1f7c0a61e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009230 tx=0x7f1f78005fd0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:57.984 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:57.985+0000 7f1f8d7fa640 1 -- 172.21.15.86:0/268369212 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f1f80014030 con 0x7f1f90101090 2024-04-21T08:51:58.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.163+0000 7f1f95ea5640 1 -- 172.21.15.86:0/268369212 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"} v 0) -- 0x7f1f90101a40 con 0x7f1f90101090 2024-04-21T08:51:58.162 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.163+0000 7f1f8d7fa640 1 -- 172.21.15.86:0/268369212 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]=0 v108) ==== 103+0+11 (secure 0 0 0) 0x7f1f8009d320 con 0x7f1f90101090 2024-04-21T08:51:58.163 INFO:teuthology.orchestra.run.smithi086.stdout:pg_num: 32 2024-04-21T08:51:58.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.165+0000 7f1f95ea5640 1 -- 172.21.15.86:0/268369212 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1f7c0a3d20 msgr2=0x7f1f7c0a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:58.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.165+0000 7f1f95ea5640 1 --2- 172.21.15.86:0/268369212 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1f7c0a3d20 0x7f1f7c0a61e0 secure :-1 s=READY pgs=369 cs=0 l=1 rev1=1 crypto rx=0x7f1f78009230 tx=0x7f1f78005fd0 comp rx=0 tx=0).stop 2024-04-21T08:51:58.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.165+0000 7f1f95ea5640 1 -- 172.21.15.86:0/268369212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f90101090 msgr2=0x7f1f901975c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:58.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.165+0000 7f1f95ea5640 1 --2- 172.21.15.86:0/268369212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f90101090 0x7f1f901975c0 secure :-1 s=READY pgs=2151 cs=0 l=1 rev1=1 crypto rx=0x7f1f8000c920 tx=0x7f1f8000cde0 comp rx=0 tx=0).stop 2024-04-21T08:51:58.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.165+0000 7f1f95ea5640 1 -- 172.21.15.86:0/268369212 shutdown_connections 2024-04-21T08:51:58.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.165+0000 7f1f95ea5640 1 --2- 172.21.15.86:0/268369212 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f1f7c0a3d20 0x7f1f7c0a61e0 unknown :-1 s=CLOSED pgs=369 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:58.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.165+0000 7f1f95ea5640 1 --2- 172.21.15.86:0/268369212 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f1f9019be80 0x7f1f9019c330 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:58.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.165+0000 7f1f95ea5640 1 --2- 172.21.15.86:0/268369212 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f1f90106160 0x7f1f90197b00 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:58.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.165+0000 7f1f95ea5640 1 --2- 172.21.15.86:0/268369212 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f1f90101090 0x7f1f901975c0 unknown :-1 s=CLOSED pgs=2151 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:58.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.165+0000 7f1f95ea5640 1 -- 172.21.15.86:0/268369212 >> 172.21.15.86:0/268369212 conn(0x7f1f90077f50 msgr2=0x7f1f900ff080 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:58.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.165+0000 7f1f95ea5640 1 -- 172.21.15.86:0/268369212 shutdown_connections 2024-04-21T08:51:58.165 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.165+0000 7f1f95ea5640 1 -- 172.21.15.86:0/268369212 wait complete. 2024-04-21T08:51:58.174 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:57 smithi086 ceph-mon[27939]: pgmap v412: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-21T08:51:58.174 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:57 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/3288191530' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:51:58.174 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:57 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/341359503' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:51:58.174 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:57 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/4244638579' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:51:58.176 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd blocklist ls 2024-04-21T08:51:58.246 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:57 smithi179 ceph-mon[31829]: pgmap v412: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-21T08:51:58.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:57 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/3288191530' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:51:58.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:57 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/341359503' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:51:58.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:57 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/4244638579' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:51:58.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.275+0000 7fa42f52b640 1 -- 172.21.15.86:0/2037243945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa428101b40 msgr2=0x7fa42810dc80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:58.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.275+0000 7fa42f52b640 1 --2- 172.21.15.86:0/2037243945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa428101b40 0x7fa42810dc80 secure :-1 s=READY pgs=2152 cs=0 l=1 rev1=1 crypto rx=0x7fa420009cd0 tx=0x7fa42002c0f0 comp rx=0 tx=0).stop 2024-04-21T08:51:58.276 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.275+0000 7fa42f52b640 1 -- 172.21.15.86:0/2037243945 shutdown_connections 2024-04-21T08:51:58.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.275+0000 7fa42f52b640 1 --2- 172.21.15.86:0/2037243945 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa428101b40 0x7fa42810dc80 unknown :-1 s=CLOSED pgs=2152 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:58.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.275+0000 7fa42f52b640 1 --2- 172.21.15.86:0/2037243945 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa4281011a0 0x7fa428101600 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:58.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.275+0000 7fa42f52b640 1 --2- 172.21.15.86:0/2037243945 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa428107030 0x7fa428107410 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:58.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.275+0000 7fa42f52b640 1 -- 172.21.15.86:0/2037243945 >> 172.21.15.86:0/2037243945 conn(0x7fa428076c50 msgr2=0x7fa428077060 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:58.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.275+0000 7fa42f52b640 1 -- 172.21.15.86:0/2037243945 shutdown_connections 2024-04-21T08:51:58.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.275+0000 7fa42f52b640 1 -- 172.21.15.86:0/2037243945 wait complete. 2024-04-21T08:51:58.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.276+0000 7fa42f52b640 1 Processor -- start 2024-04-21T08:51:58.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.276+0000 7fa42f52b640 1 -- start start 2024-04-21T08:51:58.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.276+0000 7fa42f52b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4281011a0 0x7fa428197640 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:58.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.276+0000 7fa42f52b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa428101b40 0x7fa428197b80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:58.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.276+0000 7fa42f52b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa428107030 0x7fa42819beb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:58.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.276+0000 7fa42f52b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa4281110c0 con 0x7fa428101b40 2024-04-21T08:51:58.277 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.276+0000 7fa42f52b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa428110f40 con 0x7fa428107030 2024-04-21T08:51:58.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.276+0000 7fa42f52b640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa428111240 con 0x7fa4281011a0 2024-04-21T08:51:58.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.276+0000 7fa42cac6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa428101b40 0x7fa428197b80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:58.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.276+0000 7fa42cac6640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa428101b40 0x7fa428197b80 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35960/0 (socket says 172.21.15.86:35960) 2024-04-21T08:51:58.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.276+0000 7fa42cac6640 1 -- 172.21.15.86:0/1875034641 learned_addr learned my addr 172.21.15.86:0/1875034641 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:58.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.276+0000 7fa42dac8640 1 --2- 172.21.15.86:0/1875034641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa428107030 0x7fa42819beb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:58.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.276+0000 7fa42d2c7640 1 --2- 172.21.15.86:0/1875034641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4281011a0 0x7fa428197640 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:58.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.276+0000 7fa42cac6640 1 -- 172.21.15.86:0/1875034641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4281011a0 msgr2=0x7fa428197640 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:58.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.276+0000 7fa42cac6640 1 --2- 172.21.15.86:0/1875034641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4281011a0 0x7fa428197640 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:58.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.276+0000 7fa42cac6640 1 -- 172.21.15.86:0/1875034641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa428107030 msgr2=0x7fa42819beb0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:58.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.276+0000 7fa42cac6640 1 --2- 172.21.15.86:0/1875034641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa428107030 0x7fa42819beb0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:58.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.276+0000 7fa42cac6640 1 -- 172.21.15.86:0/1875034641 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa42819c5e0 con 0x7fa428101b40 2024-04-21T08:51:58.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.276+0000 7fa42cac6640 1 --2- 172.21.15.86:0/1875034641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa428101b40 0x7fa428197b80 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7fa41800d950 tx=0x7fa41800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:58.278 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.277+0000 7fa41e7fc640 1 -- 172.21.15.86:0/1875034641 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa418017070 con 0x7fa428101b40 2024-04-21T08:51:58.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.277+0000 7fa41e7fc640 1 -- 172.21.15.86:0/1875034641 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa418009c30 con 0x7fa428101b40 2024-04-21T08:51:58.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.277+0000 7fa42f52b640 1 -- 172.21.15.86:0/1875034641 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa42819c8d0 con 0x7fa428101b40 2024-04-21T08:51:58.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.277+0000 7fa41e7fc640 1 -- 172.21.15.86:0/1875034641 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa41800f620 con 0x7fa428101b40 2024-04-21T08:51:58.279 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.277+0000 7fa42f52b640 1 -- 172.21.15.86:0/1875034641 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa4281a4160 con 0x7fa428101b40 2024-04-21T08:51:58.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.277+0000 7fa42f52b640 1 -- 172.21.15.86:0/1875034641 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa428102590 con 0x7fa428101b40 2024-04-21T08:51:58.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.279+0000 7fa41e7fc640 1 -- 172.21.15.86:0/1875034641 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa418005030 con 0x7fa428101b40 2024-04-21T08:51:58.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.279+0000 7fa41e7fc640 1 --2- 172.21.15.86:0/1875034641 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa4140a3d20 0x7fa4140a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:58.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.279+0000 7fa42d2c7640 1 --2- 172.21.15.86:0/1875034641 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa4140a3d20 0x7fa4140a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:58.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.279+0000 7fa41e7fc640 1 -- 172.21.15.86:0/1875034641 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7fa4180d0740 con 0x7fa428101b40 2024-04-21T08:51:58.280 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.280+0000 7fa42d2c7640 1 --2- 172.21.15.86:0/1875034641 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa4140a3d20 0x7fa4140a61e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa42400bbe0 tx=0x7fa42400b390 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:58.282 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.283+0000 7fa41e7fc640 1 -- 172.21.15.86:0/1875034641 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa418009de0 con 0x7fa428101b40 2024-04-21T08:51:58.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:57 smithi088 ceph-mon[31259]: pgmap v412: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 4.2 KiB/s rd, 8 op/s 2024-04-21T08:51:58.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:57 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/3288191530' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": ".mgr", "var": "pg_num"}]: dispatch 2024-04-21T08:51:58.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:57 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/341359503' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_metadata", "var": "pg_num"}]: dispatch 2024-04-21T08:51:58.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:57 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/4244638579' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data", "var": "pg_num"}]: dispatch 2024-04-21T08:51:58.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.472+0000 7fa42f52b640 1 -- 172.21.15.86:0/1875034641 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd blocklist ls"} v 0) -- 0x7fa4280691c0 con 0x7fa428101b40 2024-04-21T08:51:58.471 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.472+0000 7fa41e7fc640 1 -- 172.21.15.86:0/1875034641 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd blocklist ls"}]=0 listed 16 entries v108) ==== 81+0+946 (secure 0 0 0) 0x7fa4280691c0 con 0x7fa428101b40 2024-04-21T08:51:58.472 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1068089552 2024-04-22T08:38:46.631621+0000 2024-04-21T08:51:58.472 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/4273877219 2024-04-22T08:38:46.631621+0000 2024-04-21T08:51:58.472 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2707594358 2024-04-22T08:38:46.631621+0000 2024-04-21T08:51:58.472 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/4060044103 2024-04-22T08:38:46.631621+0000 2024-04-21T08:51:58.472 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/1364156965 2024-04-22T08:38:46.631621+0000 2024-04-21T08:51:58.472 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/2143206409 2024-04-22T08:29:23.190001+0000 2024-04-21T08:51:58.472 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/1364156965 2024-04-22T08:38:46.631621+0000 2024-04-21T08:51:58.472 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1412393225 2024-04-22T08:29:23.190001+0000 2024-04-21T08:51:58.472 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6800/2386990413 2024-04-22T08:28:57.384403+0000 2024-04-21T08:51:58.472 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/1687852404 2024-04-22T08:29:23.190001+0000 2024-04-21T08:51:58.472 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/2386990413 2024-04-22T08:28:57.384403+0000 2024-04-21T08:51:58.473 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2020707404 2024-04-22T08:28:57.384403+0000 2024-04-21T08:51:58.473 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2568130933 2024-04-22T08:28:57.384403+0000 2024-04-21T08:51:58.473 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/3952607648 2024-04-22T08:28:57.384403+0000 2024-04-21T08:51:58.473 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:6801/2143206409 2024-04-22T08:29:23.190001+0000 2024-04-21T08:51:58.473 INFO:teuthology.orchestra.run.smithi086.stdout:172.21.15.86:0/2756903426 2024-04-22T08:29:23.190001+0000 2024-04-21T08:51:58.474 INFO:teuthology.orchestra.run.smithi086.stderr:listed 16 entries 2024-04-21T08:51:58.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.474+0000 7fa42f52b640 1 -- 172.21.15.86:0/1875034641 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa4140a3d20 msgr2=0x7fa4140a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:58.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.474+0000 7fa42f52b640 1 --2- 172.21.15.86:0/1875034641 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa4140a3d20 0x7fa4140a61e0 secure :-1 s=READY pgs=370 cs=0 l=1 rev1=1 crypto rx=0x7fa42400bbe0 tx=0x7fa42400b390 comp rx=0 tx=0).stop 2024-04-21T08:51:58.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.474+0000 7fa42f52b640 1 -- 172.21.15.86:0/1875034641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa428101b40 msgr2=0x7fa428197b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:58.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.474+0000 7fa42f52b640 1 --2- 172.21.15.86:0/1875034641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa428101b40 0x7fa428197b80 secure :-1 s=READY pgs=2153 cs=0 l=1 rev1=1 crypto rx=0x7fa41800d950 tx=0x7fa41800de10 comp rx=0 tx=0).stop 2024-04-21T08:51:58.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.474+0000 7fa42f52b640 1 -- 172.21.15.86:0/1875034641 shutdown_connections 2024-04-21T08:51:58.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.474+0000 7fa42f52b640 1 --2- 172.21.15.86:0/1875034641 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa4140a3d20 0x7fa4140a61e0 unknown :-1 s=CLOSED pgs=370 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:58.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.474+0000 7fa42f52b640 1 --2- 172.21.15.86:0/1875034641 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa428107030 0x7fa42819beb0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:58.474 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.474+0000 7fa42f52b640 1 --2- 172.21.15.86:0/1875034641 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa428101b40 0x7fa428197b80 unknown :-1 s=CLOSED pgs=2153 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:58.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.474+0000 7fa42f52b640 1 --2- 172.21.15.86:0/1875034641 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa4281011a0 0x7fa428197640 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:58.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.474+0000 7fa42f52b640 1 -- 172.21.15.86:0/1875034641 >> 172.21.15.86:0/1875034641 conn(0x7fa428076c50 msgr2=0x7fa4280ffea0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:58.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.474+0000 7fa42f52b640 1 -- 172.21.15.86:0/1875034641 shutdown_connections 2024-04-21T08:51:58.475 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:58.474+0000 7fa42f52b640 1 -- 172.21.15.86:0/1875034641 wait complete. 2024-04-21T08:51:58.484 DEBUG:tasks.cephfs.kernel_mount:Unmounting client client.0... 2024-04-21T08:51:58.484 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:51:58.484 DEBUG:teuthology.orchestra.run.smithi086:> sudo umount /home/ubuntu/cephtest/mnt.0 2024-04-21T08:51:58.526 INFO:tasks.cephfs.mount:Cleaning up mount ubuntu@smithi086.front.sepia.ceph.com 2024-04-21T08:51:58.526 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:51:58.526 DEBUG:teuthology.orchestra.run.smithi086:> (cd /home/ubuntu/cephtest && exec rmdir -- /home/ubuntu/cephtest/mnt.0) 2024-04-21T08:51:58.582 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:51:58.582 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-21T08:51:58.637 INFO:teuthology.orchestra.run.smithi086.stdout:ceph-ns--home-ubuntu-cephtest-mnt.0 (id: 0) 2024-04-21T08:51:58.637 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:51:58.637 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip netns delete ceph-ns--home-ubuntu-cephtest-mnt.0 2024-04-21T08:51:58.701 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:51:58.702 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link delete ceph-brx 2024-04-21T08:51:58.794 INFO:tasks.kclient:Unmounting kernel clients... 2024-04-21T08:51:58.794 DEBUG:teuthology.orchestra.run.smithi086:> set -ex 2024-04-21T08:51:58.794 DEBUG:teuthology.orchestra.run.smithi086:> dd if=/proc/self/mounts of=/dev/stdout 2024-04-21T08:51:58.809 DEBUG:tasks.cephfs.mount:not mounted; /proc/self/mounts is: proc /proc proc rw,nosuid,nodev,noexec,relatime 0 0 sysfs /sys sysfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 devtmpfs /dev devtmpfs rw,seclabel,nosuid,size=4096k,nr_inodes=4056106,mode=755,inode64 0 0 securityfs /sys/kernel/security securityfs rw,nosuid,nodev,noexec,relatime 0 0 tmpfs /dev/shm tmpfs rw,seclabel,nosuid,nodev,inode64 0 0 devpts /dev/pts devpts rw,seclabel,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000 0 0 tmpfs /run tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 cgroup2 /sys/fs/cgroup cgroup2 rw,seclabel,nosuid,nodev,noexec,relatime,nsdelegate,memory_recursiveprot 0 0 pstore /sys/fs/pstore pstore rw,seclabel,nosuid,nodev,noexec,relatime 0 0 bpf /sys/fs/bpf bpf rw,nosuid,nodev,noexec,relatime,mode=700 0 0 /dev/sda1 / ext4 rw,seclabel,relatime 0 0 selinuxfs /sys/fs/selinux selinuxfs rw,nosuid,noexec,relatime 0 0 systemd-1 /proc/sys/fs/binfmt_misc autofs rw,relatime,fd=29,pgrp=1,timeout=0,minproto=5,maxproto=5,direct,pipe_ino=17597 0 0 hugetlbfs /dev/hugepages hugetlbfs rw,seclabel,relatime,pagesize=2M 0 0 mqueue /dev/mqueue mqueue rw,seclabel,nosuid,nodev,noexec,relatime 0 0 debugfs /sys/kernel/debug debugfs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 tracefs /sys/kernel/tracing tracefs rw,seclabel,nosuid,nodev,noexec,relatime 0 0 configfs /sys/kernel/config configfs rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysusers.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 fusectl /sys/fs/fuse/connections fusectl rw,nosuid,nodev,noexec,relatime 0 0 none /run/credentials/systemd-sysctl.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup-dev.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 none /run/credentials/systemd-tmpfiles-setup.service ramfs ro,seclabel,nosuid,nodev,noexec,relatime,mode=700 0 0 sunrpc /var/lib/nfs/rpc_pipefs rpc_pipefs rw,relatime 0 0 tmpfs /run/user/1000 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,uid=1000,gid=1316,inode64 0 0 /dev/mapper/vg_nvme-lv_5 /var/lib/ceph xfs rw,seclabel,relatime,attr2,inode64,logbufs=8,logbsize=32k,noquota 0 0 /dev/sda1 /var/lib/containers/storage/overlay ext4 rw,seclabel,relatime 0 0 overlay /var/lib/containers/storage/overlay/e00e735beb8948002023d075774cf7ac1f99ec712d86b4a39c92a687d927bf6e/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/QRS46IWGBKJY6DRJNY6YODQK3R:/var/lib/containers/storage/overlay/l/YN7SUONE3GDJV5PN6NXGMKRZO7,upperdir=/var/lib/containers/storage/overlay/e00e735beb8948002023d075774cf7ac1f99ec712d86b4a39c92a687d927bf6e/diff,workdir=/var/lib/containers/storage/overlay/e00e735beb8948002023d075774cf7ac1f99ec712d86b4a39c92a687d927bf6e/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/34b2fa3cee96150a369891b0e4dd1059fe0cf229b9a42cf92c6156f1b46c4680/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/QRS46IWGBKJY6DRJNY6YODQK3R:/var/lib/containers/storage/overlay/l/YN7SUONE3GDJV5PN6NXGMKRZO7,upperdir=/var/lib/containers/storage/overlay/34b2fa3cee96150a369891b0e4dd1059fe0cf229b9a42cf92c6156f1b46c4680/diff,workdir=/var/lib/containers/storage/overlay/34b2fa3cee96150a369891b0e4dd1059fe0cf229b9a42cf92c6156f1b46c4680/work,metacopy=on,volatile 0 0 tmpfs /run/user/0 tmpfs rw,seclabel,nosuid,nodev,relatime,size=3248844k,nr_inodes=812211,mode=700,inode64 0 0 binfmt_misc /proc/sys/fs/binfmt_misc binfmt_misc rw,nosuid,nodev,noexec,relatime 0 0 overlay /var/lib/containers/storage/overlay/4391a3955d3a1e334c3a1bf82bd5f7a3d87409842a8b6f60be9a5b6e2ce5786d/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/QRS46IWGBKJY6DRJNY6YODQK3R:/var/lib/containers/storage/overlay/l/YN7SUONE3GDJV5PN6NXGMKRZO7,upperdir=/var/lib/containers/storage/overlay/4391a3955d3a1e334c3a1bf82bd5f7a3d87409842a8b6f60be9a5b6e2ce5786d/diff,workdir=/var/lib/containers/storage/overlay/4391a3955d3a1e334c3a1bf82bd5f7a3d87409842a8b6f60be9a5b6e2ce5786d/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/0894c811b296cb4ab5cb5a82f4a9b4119c08624dae243f55bdd5bf30f8eaa241/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/QRS46IWGBKJY6DRJNY6YODQK3R:/var/lib/containers/storage/overlay/l/YN7SUONE3GDJV5PN6NXGMKRZO7,upperdir=/var/lib/containers/storage/overlay/0894c811b296cb4ab5cb5a82f4a9b4119c08624dae243f55bdd5bf30f8eaa241/diff,workdir=/var/lib/containers/storage/overlay/0894c811b296cb4ab5cb5a82f4a9b4119c08624dae243f55bdd5bf30f8eaa241/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/496800ad7894f09324f6d30b2466678790c8e9ce9563cfd2ac1ae2a7ef9a6390/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/QRS46IWGBKJY6DRJNY6YODQK3R:/var/lib/containers/storage/overlay/l/YN7SUONE3GDJV5PN6NXGMKRZO7,upperdir=/var/lib/containers/storage/overlay/496800ad7894f09324f6d30b2466678790c8e9ce9563cfd2ac1ae2a7ef9a6390/diff,workdir=/var/lib/containers/storage/overlay/496800ad7894f09324f6d30b2466678790c8e9ce9563cfd2ac1ae2a7ef9a6390/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/f6e7e96b7a10576c6101c9cb66a2ae7f87b312db016d17e4a44e64470e7c4f90/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",relatime,lowerdir=/var/lib/containers/storage/overlay/l/QRS46IWGBKJY6DRJNY6YODQK3R:/var/lib/containers/storage/overlay/l/YN7SUONE3GDJV5PN6NXGMKRZO7,upperdir=/var/lib/containers/storage/overlay/f6e7e96b7a10576c6101c9cb66a2ae7f87b312db016d17e4a44e64470e7c4f90/diff,workdir=/var/lib/containers/storage/overlay/f6e7e96b7a10576c6101c9cb66a2ae7f87b312db016d17e4a44e64470e7c4f90/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/cc97f79ea9026d54bca6a6e998bcbbcfe6aedc00e04b9ca15d16d22c3ad5a0a9/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/QRS46IWGBKJY6DRJNY6YODQK3R:/var/lib/containers/storage/overlay/l/YN7SUONE3GDJV5PN6NXGMKRZO7,upperdir=/var/lib/containers/storage/overlay/cc97f79ea9026d54bca6a6e998bcbbcfe6aedc00e04b9ca15d16d22c3ad5a0a9/diff,workdir=/var/lib/containers/storage/overlay/cc97f79ea9026d54bca6a6e998bcbbcfe6aedc00e04b9ca15d16d22c3ad5a0a9/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/95616b28bf77279c93c2b1a6fa4347ea4613430a4189d2323a4c61ffe512afbe/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/QRS46IWGBKJY6DRJNY6YODQK3R:/var/lib/containers/storage/overlay/l/YN7SUONE3GDJV5PN6NXGMKRZO7,upperdir=/var/lib/containers/storage/overlay/95616b28bf77279c93c2b1a6fa4347ea4613430a4189d2323a4c61ffe512afbe/diff,workdir=/var/lib/containers/storage/overlay/95616b28bf77279c93c2b1a6fa4347ea4613430a4189d2323a4c61ffe512afbe/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/bd7da644af577f438f424da955fba8b6e073f28bd2952920a24e8f69a81f5bab/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/QRS46IWGBKJY6DRJNY6YODQK3R:/var/lib/containers/storage/overlay/l/YN7SUONE3GDJV5PN6NXGMKRZO7,upperdir=/var/lib/containers/storage/overlay/bd7da644af577f438f424da955fba8b6e073f28bd2952920a24e8f69a81f5bab/diff,workdir=/var/lib/containers/storage/overlay/bd7da644af577f438f424da955fba8b6e073f28bd2952920a24e8f69a81f5bab/work,metacopy=on,volatile 0 0 overlay /var/lib/containers/storage/overlay/fded4a0ffeb938e056b2a3db157b569417c20245771918b428b6f1fc91a79ce4/merged overlay rw,context="system_u:object_r:container_file_t:s0:c1022,c1023",nodev,relatime,lowerdir=/var/lib/containers/storage/overlay/l/QRS46IWGBKJY6DRJNY6YODQK3R:/var/lib/containers/storage/overlay/l/YN7SUONE3GDJV5PN6NXGMKRZO7,upperdir=/var/lib/containers/storage/overlay/fded4a0ffeb938e056b2a3db157b569417c20245771918b428b6f1fc91a79ce4/diff,workdir=/var/lib/containers/storage/overlay/fded4a0ffeb938e056b2a3db157b569417c20245771918b428b6f1fc91a79ce4/work,metacopy=on,volatile 0 0 tmpfs /run/netns tmpfs rw,seclabel,nosuid,nodev,size=6497692k,nr_inodes=819200,mode=755,inode64 0 0 2024-04-21T08:51:58.809 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:51:58.809 DEBUG:teuthology.orchestra.run.smithi086:> ip netns list 2024-04-21T08:51:58.865 INFO:teuthology.orchestra.run:Running command with timeout 300 2024-04-21T08:51:58.865 DEBUG:teuthology.orchestra.run.smithi086:> sudo ip link delete ceph-brx 2024-04-21T08:51:58.931 INFO:teuthology.orchestra.run.smithi086.stderr:Cannot find device "ceph-brx" 2024-04-21T08:51:58.932 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T08:51:58.932 DEBUG:teuthology.run_tasks:Unwinding manager cephadm 2024-04-21T08:51:58.945 INFO:tasks.cephadm:Teardown begin 2024-04-21T08:51:58.945 INFO:tasks.cephfs.filesystem:Destroying file system cephfs and related pools 2024-04-21T08:51:58.946 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:51:59.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.072+0000 7fbbfeb9d640 1 -- 172.21.15.86:0/4131082683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbf810c3d0 msgr2=0x7fbbf810e800 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:59.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.072+0000 7fbbfeb9d640 1 --2- 172.21.15.86:0/4131082683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbf810c3d0 0x7fbbf810e800 secure :-1 s=READY pgs=2154 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0009ca0 tx=0x7fbbf002c180 comp rx=0 tx=0).stop 2024-04-21T08:51:59.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.072+0000 7fbbfeb9d640 1 -- 172.21.15.86:0/4131082683 shutdown_connections 2024-04-21T08:51:59.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.072+0000 7fbbfeb9d640 1 --2- 172.21.15.86:0/4131082683 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbf810c3d0 0x7fbbf810e800 unknown :-1 s=CLOSED pgs=2154 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:59.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.072+0000 7fbbfeb9d640 1 --2- 172.21.15.86:0/4131082683 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbf8103920 0x7fbbf810be00 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:59.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.072+0000 7fbbfeb9d640 1 --2- 172.21.15.86:0/4131082683 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbf8103000 0x7fbbf81033e0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:59.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.072+0000 7fbbfeb9d640 1 -- 172.21.15.86:0/4131082683 >> 172.21.15.86:0/4131082683 conn(0x7fbbf80fc740 msgr2=0x7fbbf80feb60 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:59.074 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.072+0000 7fbbfeb9d640 1 -- 172.21.15.86:0/4131082683 shutdown_connections 2024-04-21T08:51:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.072+0000 7fbbfeb9d640 1 -- 172.21.15.86:0/4131082683 wait complete. 2024-04-21T08:51:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.072+0000 7fbbfeb9d640 1 Processor -- start 2024-04-21T08:51:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.073+0000 7fbbfeb9d640 1 -- start start 2024-04-21T08:51:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.073+0000 7fbbfeb9d640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbf8103000 0x7fbbf8198650 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.073+0000 7fbbfeb9d640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbf8103920 0x7fbbf8198b90 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.073+0000 7fbbfeb9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbf810c3d0 0x7fbbf819cf10 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.073+0000 7fbbfeb9d640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf81120d0 con 0x7fbbf810c3d0 2024-04-21T08:51:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.073+0000 7fbbfeb9d640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf8111f50 con 0x7fbbf8103920 2024-04-21T08:51:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.073+0000 7fbbfeb9d640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fbbf8112250 con 0x7fbbf8103000 2024-04-21T08:51:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.073+0000 7fbbfe39c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbf810c3d0 0x7fbbf819cf10 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.073+0000 7fbbfe39c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbf810c3d0 0x7fbbf819cf10 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35978/0 (socket says 172.21.15.86:35978) 2024-04-21T08:51:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.073+0000 7fbbfe39c640 1 -- 172.21.15.86:0/839405779 learned_addr learned my addr 172.21.15.86:0/839405779 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.073+0000 7fbbfd39a640 1 --2- 172.21.15.86:0/839405779 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbf8103920 0x7fbbf8198b90 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:59.075 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.073+0000 7fbbfdb9b640 1 --2- 172.21.15.86:0/839405779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbf8103000 0x7fbbf8198650 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.073+0000 7fbbfe39c640 1 -- 172.21.15.86:0/839405779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbf8103000 msgr2=0x7fbbf8198650 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.073+0000 7fbbfe39c640 1 --2- 172.21.15.86:0/839405779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbf8103000 0x7fbbf8198650 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.073+0000 7fbbfe39c640 1 -- 172.21.15.86:0/839405779 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbf8103920 msgr2=0x7fbbf8198b90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.073+0000 7fbbfe39c640 1 --2- 172.21.15.86:0/839405779 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbf8103920 0x7fbbf8198b90 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.073+0000 7fbbfe39c640 1 -- 172.21.15.86:0/839405779 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fbbf819d640 con 0x7fbbf810c3d0 2024-04-21T08:51:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.073+0000 7fbbfdb9b640 1 --2- 172.21.15.86:0/839405779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbf8103000 0x7fbbf8198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.073+0000 7fbbfe39c640 1 --2- 172.21.15.86:0/839405779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbf810c3d0 0x7fbbf819cf10 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0037040 tx=0x7fbbf0003ba0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.073+0000 7fbbeeffd640 1 -- 172.21.15.86:0/839405779 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0039040 con 0x7fbbf810c3d0 2024-04-21T08:51:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.073+0000 7fbbeeffd640 1 -- 172.21.15.86:0/839405779 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fbbf0044070 con 0x7fbbf810c3d0 2024-04-21T08:51:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.073+0000 7fbbfeb9d640 1 -- 172.21.15.86:0/839405779 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fbbf819d8d0 con 0x7fbbf810c3d0 2024-04-21T08:51:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.073+0000 7fbbeeffd640 1 -- 172.21.15.86:0/839405779 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fbbf0032a30 con 0x7fbbf810c3d0 2024-04-21T08:51:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.074+0000 7fbbfeb9d640 1 -- 172.21.15.86:0/839405779 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fbbf81a51c0 con 0x7fbbf810c3d0 2024-04-21T08:51:59.076 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.074+0000 7fbbfeb9d640 1 -- 172.21.15.86:0/839405779 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fbbf80690e0 con 0x7fbbf810c3d0 2024-04-21T08:51:59.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.076+0000 7fbbeeffd640 1 -- 172.21.15.86:0/839405779 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fbbf0004610 con 0x7fbbf810c3d0 2024-04-21T08:51:59.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.076+0000 7fbbeeffd640 1 --2- 172.21.15.86:0/839405779 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbbe40a3d20 0x7fbbe40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:59.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.076+0000 7fbbfdb9b640 1 --2- 172.21.15.86:0/839405779 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbbe40a3d20 0x7fbbe40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:59.077 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.076+0000 7fbbeeffd640 1 -- 172.21.15.86:0/839405779 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7fbbf00f38c0 con 0x7fbbf810c3d0 2024-04-21T08:51:59.078 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.077+0000 7fbbfdb9b640 1 --2- 172.21.15.86:0/839405779 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbbe40a3d20 0x7fbbe40a61e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4009930 tx=0x7fbbf4008040 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:59.079 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.079+0000 7fbbeeffd640 1 -- 172.21.15.86:0/839405779 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fbbf003e3a0 con 0x7fbbf810c3d0 2024-04-21T08:51:59.116 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:59 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/268369212' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:51:59.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:51:59 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1875034641' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-21T08:51:59.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:59 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/268369212' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:51:59.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:51:59 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1875034641' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-21T08:51:59.294 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.295+0000 7fbbfeb9d640 1 -- 172.21.15.86:0/839405779 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7fbbf81999a0 con 0x7fbbf810c3d0 2024-04-21T08:51:59.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.295+0000 7fbbeeffd640 1 -- 172.21.15.86:0/839405779 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7fbbf00bfca0 con 0x7fbbf810c3d0 2024-04-21T08:51:59.295 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:51:59.295 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:38:45.202126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24398,24394,24455,24472]},"id":1}]} 2024-04-21T08:51:59.295 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-21T08:51:59.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.297+0000 7fbbfeb9d640 1 -- 172.21.15.86:0/839405779 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbbe40a3d20 msgr2=0x7fbbe40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:59.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.297+0000 7fbbfeb9d640 1 --2- 172.21.15.86:0/839405779 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbbe40a3d20 0x7fbbe40a61e0 secure :-1 s=READY pgs=371 cs=0 l=1 rev1=1 crypto rx=0x7fbbf4009930 tx=0x7fbbf4008040 comp rx=0 tx=0).stop 2024-04-21T08:51:59.297 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.297+0000 7fbbfeb9d640 1 -- 172.21.15.86:0/839405779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbf810c3d0 msgr2=0x7fbbf819cf10 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:59.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.297+0000 7fbbfeb9d640 1 --2- 172.21.15.86:0/839405779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbf810c3d0 0x7fbbf819cf10 secure :-1 s=READY pgs=2155 cs=0 l=1 rev1=1 crypto rx=0x7fbbf0037040 tx=0x7fbbf0003ba0 comp rx=0 tx=0).stop 2024-04-21T08:51:59.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.297+0000 7fbbfeb9d640 1 -- 172.21.15.86:0/839405779 shutdown_connections 2024-04-21T08:51:59.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.297+0000 7fbbfeb9d640 1 --2- 172.21.15.86:0/839405779 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fbbe40a3d20 0x7fbbe40a61e0 unknown :-1 s=CLOSED pgs=371 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:59.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.297+0000 7fbbfeb9d640 1 --2- 172.21.15.86:0/839405779 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fbbf810c3d0 0x7fbbf819cf10 unknown :-1 s=CLOSED pgs=2155 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:59.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.297+0000 7fbbfeb9d640 1 --2- 172.21.15.86:0/839405779 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fbbf8103920 0x7fbbf8198b90 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:59.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.297+0000 7fbbfeb9d640 1 --2- 172.21.15.86:0/839405779 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fbbf8103000 0x7fbbf8198650 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:59.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.297+0000 7fbbfeb9d640 1 -- 172.21.15.86:0/839405779 >> 172.21.15.86:0/839405779 conn(0x7fbbf80fc740 msgr2=0x7fbbf80fe000 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:59.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.298+0000 7fbbfeb9d640 1 -- 172.21.15.86:0/839405779 shutdown_connections 2024-04-21T08:51:59.298 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.298+0000 7fbbfeb9d640 1 -- 172.21.15.86:0/839405779 wait complete. 2024-04-21T08:51:59.308 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs dump --format=json 2024-04-21T08:51:59.407 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.406+0000 7f8c0f175640 1 -- 172.21.15.86:0/1496399448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c08102b40 msgr2=0x7f8c08106b80 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:59.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.406+0000 7f8c0f175640 1 --2- 172.21.15.86:0/1496399448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c08102b40 0x7f8c08106b80 secure :-1 s=READY pgs=2156 cs=0 l=1 rev1=1 crypto rx=0x7f8bf8009900 tx=0x7f8bf802bc30 comp rx=0 tx=0).stop 2024-04-21T08:51:59.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.406+0000 7f8c0f175640 1 -- 172.21.15.86:0/1496399448 shutdown_connections 2024-04-21T08:51:59.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.406+0000 7f8c0f175640 1 --2- 172.21.15.86:0/1496399448 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c08107260 0x7f8c0810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:59.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.406+0000 7f8c0f175640 1 --2- 172.21.15.86:0/1496399448 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c08102b40 0x7f8c08106b80 unknown :-1 s=CLOSED pgs=2156 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:59.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.406+0000 7f8c0f175640 1 --2- 172.21.15.86:0/1496399448 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c08102190 0x7f8c08102570 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:59.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.406+0000 7f8c0f175640 1 -- 172.21.15.86:0/1496399448 >> 172.21.15.86:0/1496399448 conn(0x7f8c080793e0 msgr2=0x7f8c080797f0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:59.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.406+0000 7f8c0f175640 1 -- 172.21.15.86:0/1496399448 shutdown_connections 2024-04-21T08:51:59.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.406+0000 7f8c0f175640 1 -- 172.21.15.86:0/1496399448 wait complete. 2024-04-21T08:51:59.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.406+0000 7f8c0f175640 1 Processor -- start 2024-04-21T08:51:59.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.407+0000 7f8c0f175640 1 -- start start 2024-04-21T08:51:59.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.407+0000 7f8c0f175640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c08102190 0x7f8c081a0690 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:59.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.407+0000 7f8c0f175640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c08107260 0x7f8c081a0bd0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:59.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.407+0000 7f8c0f175640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c0819a800 0x7f8c0819acb0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:59.408 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.407+0000 7f8c0f175640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f8c081121a0 con 0x7f8c0819a800 2024-04-21T08:51:59.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.407+0000 7f8c0f175640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f8c08112020 con 0x7f8c08107260 2024-04-21T08:51:59.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.407+0000 7f8c0f175640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f8c08112320 con 0x7f8c08102190 2024-04-21T08:51:59.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.407+0000 7f8c0d712640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c0819a800 0x7f8c0819acb0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:59.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.407+0000 7f8c0d712640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c0819a800 0x7f8c0819acb0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:35998/0 (socket says 172.21.15.86:35998) 2024-04-21T08:51:59.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.407+0000 7f8c0d712640 1 -- 172.21.15.86:0/2658717841 learned_addr learned my addr 172.21.15.86:0/2658717841 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:59.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.407+0000 7f8c07fff640 1 --2- 172.21.15.86:0/2658717841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c08107260 0x7f8c081a0bd0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:59.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.407+0000 7f8c0cf11640 1 --2- 172.21.15.86:0/2658717841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c08102190 0x7f8c081a0690 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:59.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.407+0000 7f8c0d712640 1 -- 172.21.15.86:0/2658717841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c08102190 msgr2=0x7f8c081a0690 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:59.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.407+0000 7f8c0d712640 1 --2- 172.21.15.86:0/2658717841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c08102190 0x7f8c081a0690 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:59.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.407+0000 7f8c0d712640 1 -- 172.21.15.86:0/2658717841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c08107260 msgr2=0x7f8c081a0bd0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:59.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.407+0000 7f8c0d712640 1 --2- 172.21.15.86:0/2658717841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c08107260 0x7f8c081a0bd0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:59.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.407+0000 7f8c0d712640 1 -- 172.21.15.86:0/2658717841 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f8c0819b390 con 0x7f8c0819a800 2024-04-21T08:51:59.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.407+0000 7f8c07fff640 1 --2- 172.21.15.86:0/2658717841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c08107260 0x7f8c081a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:51:59.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.407+0000 7f8c0d712640 1 --2- 172.21.15.86:0/2658717841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c0819a800 0x7f8c0819acb0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f8bf000e970 tx=0x7f8bf000ed70 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:59.409 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.407+0000 7f8c05ffb640 1 -- 172.21.15.86:0/2658717841 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf000ccc0 con 0x7f8c0819a800 2024-04-21T08:51:59.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.407+0000 7f8c0f175640 1 -- 172.21.15.86:0/2658717841 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f8c0819b680 con 0x7f8c0819a800 2024-04-21T08:51:59.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.407+0000 7f8c05ffb640 1 -- 172.21.15.86:0/2658717841 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f8bf000ce60 con 0x7f8c0819a800 2024-04-21T08:51:59.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.407+0000 7f8c05ffb640 1 -- 172.21.15.86:0/2658717841 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f8bf0010430 con 0x7f8c0819a800 2024-04-21T08:51:59.410 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.408+0000 7f8c0f175640 1 -- 172.21.15.86:0/2658717841 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f8c081a77e0 con 0x7f8c0819a800 2024-04-21T08:51:59.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.409+0000 7f8c0f175640 1 -- 172.21.15.86:0/2658717841 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f8c08103660 con 0x7f8c0819a800 2024-04-21T08:51:59.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.410+0000 7f8c05ffb640 1 -- 172.21.15.86:0/2658717841 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f8bf0004ff0 con 0x7f8c0819a800 2024-04-21T08:51:59.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.410+0000 7f8c05ffb640 1 --2- 172.21.15.86:0/2658717841 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8bf40a3d20 0x7f8bf40a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:59.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.410+0000 7f8c0cf11640 1 --2- 172.21.15.86:0/2658717841 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8bf40a3d20 0x7f8bf40a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:59.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.410+0000 7f8c05ffb640 1 -- 172.21.15.86:0/2658717841 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7f8bf00d10b0 con 0x7f8c0819a800 2024-04-21T08:51:59.411 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.411+0000 7f8c0cf11640 1 --2- 172.21.15.86:0/2658717841 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8bf40a3d20 0x7f8bf40a61e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8c00005fd0 tx=0x7f8c00009500 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:59.414 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.415+0000 7f8c05ffb640 1 -- 172.21.15.86:0/2658717841 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f8c0819b680 con 0x7f8c0819a800 2024-04-21T08:51:59.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:59 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/268369212' entity='client.admin' cmd=[{"prefix": "osd pool get", "pool": "cephfs_data_ec", "var": "pg_num"}]: dispatch 2024-04-21T08:51:59.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:51:59 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1875034641' entity='client.admin' cmd=[{"prefix": "osd blocklist ls"}]: dispatch 2024-04-21T08:51:59.632 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.633+0000 7f8c0f175640 1 -- 172.21.15.86:0/2658717841 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs dump", "format": "json"} v 0) -- 0x7f8c08102be0 con 0x7f8c0819a800 2024-04-21T08:51:59.633 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.633+0000 7f8c05ffb640 1 -- 172.21.15.86:0/2658717841 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs dump", "format": "json"}]=0 dumped fsmap epoch 29 v29) ==== 94+0+11935 (secure 0 0 0) 0x7f8bf009d380 con 0x7f8c0819a800 2024-04-21T08:51:59.633 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:51:59.633 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":29,"default_fscid":1,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"feature_flags":{"enable_multiple":true,"ever_enabled_multiple":true},"standbys":[{"gid":14571,"name":"j","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6835/4288306918","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6834","nonce":4288306918},{"type":"v1","addr":"172.21.15.86:6835","nonce":4288306918}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":4},{"gid":14583,"name":"d","rank":-1,"incarnation":0,"state":"up:standby","state_seq":1,"addr":"172.21.15.86:6837/1247248764","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6836","nonce":1247248764},{"type":"v1","addr":"172.21.15.86:6837","nonce":1247248764}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"epoch":5}],"filesystems":[{"mdsmap":{"epoch":29,"flags":306,"flags_state":{"joinable":true,"allow_snaps":true,"allow_multimds_snaps":true,"allow_standby_replay":true,"refuse_client_session":false,"refuse_standby_for_another_fs":false,"balance_automate":true},"ever_allowed_features":32,"explicitly_allowed_features":32,"created":"2024-04-21T08:36:20.626894+0000","modified":"2024-04-21T08:38:45.202126+0000","tableserver":0,"root":0,"session_timeout":300,"session_autoclose":300,"required_client_features":{},"max_file_size":1099511627776,"max_xattr_size":65536,"last_failure":0,"last_failure_osd_epoch":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}},"max_mds":5,"in":[0,1,2,3,4],"up":{"mds_0":24482,"mds_1":24472,"mds_2":24455,"mds_3":24398,"mds_4":24394},"failed":[],"damaged":[],"stopped":[],"info":{"gid_14640":{"gid":14640,"name":"g","rank":4,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6839/3163123023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6838","nonce":3163123023},{"type":"v1","addr":"172.21.15.86:6839","nonce":3163123023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_14652":{"gid":14652,"name":"a","rank":3,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.86:6841/3537972023","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6840","nonce":3537972023},{"type":"v1","addr":"172.21.15.86:6841","nonce":3537972023}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24394":{"gid":24394,"name":"l","rank":4,"incarnation":26,"state":"up:active","state_seq":17,"addr":"172.21.15.179:6833/3606421532","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6832","nonce":3606421532},{"type":"v1","addr":"172.21.15.179:6833","nonce":3606421532}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24398":{"gid":24398,"name":"k","rank":3,"incarnation":24,"state":"up:active","state_seq":16,"addr":"172.21.15.88:6833/1872744608","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6832","nonce":1872744608},{"type":"v1","addr":"172.21.15.88:6833","nonce":1872744608}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24428":{"gid":24428,"name":"e","rank":2,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.88:6835/454792798","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6834","nonce":454792798},{"type":"v1","addr":"172.21.15.88:6835","nonce":454792798}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24436":{"gid":24436,"name":"f","rank":1,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6835/3343050063","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6834","nonce":3343050063},{"type":"v1","addr":"172.21.15.179:6835","nonce":3343050063}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24448":{"gid":24448,"name":"c","rank":0,"incarnation":0,"state":"up:standby-replay","state_seq":1,"addr":"172.21.15.179:6837/1280529894","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6836","nonce":1280529894},{"type":"v1","addr":"172.21.15.179:6837","nonce":1280529894}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24455":{"gid":24455,"name":"h","rank":2,"incarnation":22,"state":"up:active","state_seq":12,"addr":"172.21.15.88:6837/1950568471","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6836","nonce":1950568471},{"type":"v1","addr":"172.21.15.88:6837","nonce":1950568471}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24472":{"gid":24472,"name":"i","rank":1,"incarnation":20,"state":"up:active","state_seq":11,"addr":"172.21.15.179:6839/3029310820","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6838","nonce":3029310820},{"type":"v1","addr":"172.21.15.179:6839","nonce":3029310820}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}},"gid_24482":{"gid":24482,"name":"b","rank":0,"incarnation":14,"state":"up:active","state_seq":3,"addr":"172.21.15.88:6839/3969608142","addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6838","nonce":3969608142},{"type":"v1","addr":"172.21.15.88:6839","nonce":3969608142}]},"join_fscid":-1,"export_targets":[],"features":4540701547738038271,"flags":0,"compat":{"compat":{},"ro_compat":{},"incompat":{"feature_1":"base v0.20","feature_2":"client writeable ranges","feature_3":"default file layouts on dirs","feature_4":"dir inode in separate object","feature_5":"mds uses versioned encoding","feature_6":"dirfrag is stored in omap","feature_7":"mds uses inline data","feature_8":"no anchor table","feature_9":"file layout v2","feature_10":"snaprealm v2","feature_11":"minor log segments","feature_12":"quiesce subvolumes"}}}},"data_pools":[3,4],"metadata_pool":2,"enabled":true,"fs_name":"cephfs","balancer":"","bal_rank_mask":"-1","standby_count_wanted":0,"qdb_leader":24482,"qdb_cluster":[24482,24398,24394,24455,24472]},"id":1}]} 2024-04-21T08:51:59.633 INFO:teuthology.orchestra.run.smithi086.stderr:dumped fsmap epoch 29 2024-04-21T08:51:59.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.635+0000 7f8c0f175640 1 -- 172.21.15.86:0/2658717841 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8bf40a3d20 msgr2=0x7f8bf40a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:59.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.635+0000 7f8c0f175640 1 --2- 172.21.15.86:0/2658717841 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8bf40a3d20 0x7f8bf40a61e0 secure :-1 s=READY pgs=372 cs=0 l=1 rev1=1 crypto rx=0x7f8c00005fd0 tx=0x7f8c00009500 comp rx=0 tx=0).stop 2024-04-21T08:51:59.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.635+0000 7f8c0f175640 1 -- 172.21.15.86:0/2658717841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c0819a800 msgr2=0x7f8c0819acb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:59.635 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.635+0000 7f8c0f175640 1 --2- 172.21.15.86:0/2658717841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c0819a800 0x7f8c0819acb0 secure :-1 s=READY pgs=2157 cs=0 l=1 rev1=1 crypto rx=0x7f8bf000e970 tx=0x7f8bf000ed70 comp rx=0 tx=0).stop 2024-04-21T08:51:59.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.636+0000 7f8c0f175640 1 -- 172.21.15.86:0/2658717841 shutdown_connections 2024-04-21T08:51:59.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.636+0000 7f8c0f175640 1 --2- 172.21.15.86:0/2658717841 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f8bf40a3d20 0x7f8bf40a61e0 unknown :-1 s=CLOSED pgs=372 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:59.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.636+0000 7f8c0f175640 1 --2- 172.21.15.86:0/2658717841 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f8c0819a800 0x7f8c0819acb0 unknown :-1 s=CLOSED pgs=2157 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:59.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.636+0000 7f8c0f175640 1 --2- 172.21.15.86:0/2658717841 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f8c08107260 0x7f8c081a0bd0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:59.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.636+0000 7f8c0f175640 1 --2- 172.21.15.86:0/2658717841 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f8c08102190 0x7f8c081a0690 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:59.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.636+0000 7f8c0f175640 1 -- 172.21.15.86:0/2658717841 >> 172.21.15.86:0/2658717841 conn(0x7f8c080793e0 msgr2=0x7f8c081000b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:59.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.636+0000 7f8c0f175640 1 -- 172.21.15.86:0/2658717841 shutdown_connections 2024-04-21T08:51:59.636 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.636+0000 7f8c0f175640 1 -- 172.21.15.86:0/2658717841 wait complete. 2024-04-21T08:51:59.646 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2024-04-21T08:51:59.745 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.744+0000 7f385784e640 1 -- 172.21.15.86:0/472252709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f385006a8f0 msgr2=0x7f385006acd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:59.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.744+0000 7f385784e640 1 --2- 172.21.15.86:0/472252709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f385006a8f0 0x7f385006acd0 secure :-1 s=READY pgs=2158 cs=0 l=1 rev1=1 crypto rx=0x7f384c009900 tx=0x7f384c02bc30 comp rx=0 tx=0).stop 2024-04-21T08:51:59.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.744+0000 7f385784e640 1 -- 172.21.15.86:0/472252709 shutdown_connections 2024-04-21T08:51:59.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.744+0000 7f385784e640 1 --2- 172.21.15.86:0/472252709 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38501072f0 0x7f385010ee70 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:59.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.744+0000 7f385784e640 1 --2- 172.21.15.86:0/472252709 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f385006b2a0 0x7f3850106bc0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:59.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.744+0000 7f385784e640 1 --2- 172.21.15.86:0/472252709 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f385006a8f0 0x7f385006acd0 unknown :-1 s=CLOSED pgs=2158 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:59.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.744+0000 7f385784e640 1 -- 172.21.15.86:0/472252709 >> 172.21.15.86:0/472252709 conn(0x7f3850077e30 msgr2=0x7f3850078240 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:59.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.744+0000 7f385784e640 1 -- 172.21.15.86:0/472252709 shutdown_connections 2024-04-21T08:51:59.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.744+0000 7f385784e640 1 -- 172.21.15.86:0/472252709 wait complete. 2024-04-21T08:51:59.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.744+0000 7f385784e640 1 Processor -- start 2024-04-21T08:51:59.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.744+0000 7f385784e640 1 -- start start 2024-04-21T08:51:59.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.744+0000 7f385784e640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f385006a8f0 0x7f385019e400 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:59.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.744+0000 7f385784e640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38501072f0 0x7f385019e940 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:59.746 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.744+0000 7f385784e640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38501985b0 0x7f3850198a60 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:59.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.744+0000 7f385784e640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f3850112210 con 0x7f385006a8f0 2024-04-21T08:51:59.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.744+0000 7f385784e640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f3850112090 con 0x7f38501072f0 2024-04-21T08:51:59.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.744+0000 7f385784e640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f3850112390 con 0x7f38501985b0 2024-04-21T08:51:59.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.745+0000 7f38555ea640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f385006a8f0 0x7f385019e400 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:59.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.745+0000 7f38555ea640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f385006a8f0 0x7f385019e400 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42126/0 (socket says 172.21.15.86:42126) 2024-04-21T08:51:59.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.745+0000 7f38555ea640 1 -- 172.21.15.86:0/1734716541 learned_addr learned my addr 172.21.15.86:0/1734716541 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:51:59.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.745+0000 7f3854de9640 1 --2- 172.21.15.86:0/1734716541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38501072f0 0x7f385019e940 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:59.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.745+0000 7f38555ea640 1 -- 172.21.15.86:0/1734716541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38501985b0 msgr2=0x7f3850198a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:59.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.745+0000 7f38555ea640 1 --2- 172.21.15.86:0/1734716541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38501985b0 0x7f3850198a60 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:59.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.745+0000 7f38555ea640 1 -- 172.21.15.86:0/1734716541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38501072f0 msgr2=0x7f385019e940 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:59.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.745+0000 7f38555ea640 1 --2- 172.21.15.86:0/1734716541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38501072f0 0x7f385019e940 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:59.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.745+0000 7f38555ea640 1 -- 172.21.15.86:0/1734716541 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f38501990d0 con 0x7f385006a8f0 2024-04-21T08:51:59.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.745+0000 7f38555ea640 1 --2- 172.21.15.86:0/1734716541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f385006a8f0 0x7f385019e400 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f384c005940 tx=0x7f384c02c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:59.747 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.745+0000 7f38467fc640 1 -- 172.21.15.86:0/1734716541 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384c043020 con 0x7f385006a8f0 2024-04-21T08:51:59.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.745+0000 7f38467fc640 1 -- 172.21.15.86:0/1734716541 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f384c02cbe0 con 0x7f385006a8f0 2024-04-21T08:51:59.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.745+0000 7f38467fc640 1 -- 172.21.15.86:0/1734716541 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f384c003bf0 con 0x7f385006a8f0 2024-04-21T08:51:59.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.745+0000 7f385784e640 1 -- 172.21.15.86:0/1734716541 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f3850199360 con 0x7f385006a8f0 2024-04-21T08:51:59.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.745+0000 7f385784e640 1 -- 172.21.15.86:0/1734716541 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f38501a5600 con 0x7f385006a8f0 2024-04-21T08:51:59.748 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.746+0000 7f385784e640 1 -- 172.21.15.86:0/1734716541 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f3850069230 con 0x7f385006a8f0 2024-04-21T08:51:59.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.747+0000 7f38467fc640 1 -- 172.21.15.86:0/1734716541 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f384c046020 con 0x7f385006a8f0 2024-04-21T08:51:59.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.748+0000 7f38467fc640 1 --2- 172.21.15.86:0/1734716541 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f383c0a3da0 0x7f383c0a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:51:59.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.748+0000 7f3854de9640 1 --2- 172.21.15.86:0/1734716541 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f383c0a3da0 0x7f383c0a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:51:59.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.748+0000 7f38467fc640 1 -- 172.21.15.86:0/1734716541 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7f384c0f33f0 con 0x7f385006a8f0 2024-04-21T08:51:59.749 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.748+0000 7f3854de9640 1 --2- 172.21.15.86:0/1734716541 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f383c0a3da0 0x7f383c0a6260 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3840005fd0 tx=0x7f3840005850 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:51:59.751 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.751+0000 7f38467fc640 1 -- 172.21.15.86:0/1734716541 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f384c04d020 con 0x7f385006a8f0 2024-04-21T08:51:59.932 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.932+0000 7f385784e640 1 -- 172.21.15.86:0/1734716541 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd dump", "format": "json"} v 0) -- 0x7f385006b2a0 con 0x7f385006a8f0 2024-04-21T08:51:59.933 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.934+0000 7f38467fc640 1 -- 172.21.15.86:0/1734716541 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd dump", "format": "json"}]=0 v108) ==== 74+0+25253 (secure 0 0 0) 0x7f384c0bf7d0 con 0x7f385006a8f0 2024-04-21T08:51:59.934 INFO:teuthology.orchestra.run.smithi086.stdout: 2024-04-21T08:51:59.934 INFO:teuthology.orchestra.run.smithi086.stdout:{"epoch":108,"fsid":"f8cec7cc-ffb8-11ee-bc93-c7b262605968","created":"2024-04-21T08:28:22.687329+0000","modified":"2024-04-21T08:51:10.626236+0000","last_up_change":"2024-04-21T08:35:30.048315+0000","last_in_change":"2024-04-21T08:35:10.099496+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":27,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":12,"require_min_compat_client":"luminous","min_compat_client":"luminous","require_osd_release":"squid","allow_crimson":false,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2024-04-21T08:31:45.279109+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"21","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":12,"score_stable":12,"optimal_score":0.25,"raw_score_acting":3,"raw_score_stable":3,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"cephfs_metadata","create_time":"2024-04-21T08:36:18.300782+0000","flags":1,"flags_names":"hashpspool","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"84","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_autoscale_bias":4,"pg_num_min":16,"recovery_priority":5},"application_metadata":{"cephfs":{"metadata":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":3,"pool_name":"cephfs_data","create_time":"2024-04-21T08:36:19.574636+0000","flags":8193,"flags_names":"hashpspool,selfmanaged_snaps","type":1,"size":3,"min_size":2,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":64,"pg_placement_num":64,"pg_placement_num_target":64,"pg_num_target":64,"pg_num_pending":64,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":21,"snap_epoch":107,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.05000000074505806},"application_metadata":{"cephfs":{"data":"cephfs"}},"read_balance":{"score_type":"Fair distribution","score_acting":1.5,"score_stable":1.5,"optimal_score":1,"raw_score_acting":1.5,"raw_score_stable":1.5,"primary_affinity_weighted":0.99999988079071045,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":4,"pool_name":"cephfs_data_ec","create_time":"2024-04-21T08:36:22.905632+0000","flags":8197,"flags_names":"hashpspool,ec_overwrites,selfmanaged_snaps","type":3,"size":4,"min_size":3,"crush_rule":1,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"object_hash":2,"pg_autoscale_mode":"off","pg_num":32,"pg_placement_num":32,"pg_placement_num_target":32,"pg_num_target":32,"pg_num_pending":32,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"107","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":21,"snap_epoch":107,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"cephfs_data_ec","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":8192,"expected_num_objects":0,"fast_read":false,"options":{"pg_num_min":64,"target_size_ratio":0.89999997615814209},"application_metadata":{"cephfs":{"data":"cephfs"}}}],"osds":[{"osd":0,"uuid":"89b7601f-0620-4db2-90f1-b45cea173738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":8,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6802","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6803","nonce":2571516897}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6804","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6805","nonce":2571516897}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6808","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6809","nonce":2571516897}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6806","nonce":2571516897},{"type":"v1","addr":"172.21.15.86:6807","nonce":2571516897}]},"public_addr":"172.21.15.86:6803/2571516897","cluster_addr":"172.21.15.86:6805/2571516897","heartbeat_back_addr":"172.21.15.86:6809/2571516897","heartbeat_front_addr":"172.21.15.86:6807/2571516897","state":["exists","up"]},{"osd":1,"uuid":"4891031c-f98c-42bc-bb3b-a420855ecc06","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6800","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6801","nonce":74067182}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6802","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6803","nonce":74067182}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6806","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6807","nonce":74067182}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6804","nonce":74067182},{"type":"v1","addr":"172.21.15.88:6805","nonce":74067182}]},"public_addr":"172.21.15.88:6801/74067182","cluster_addr":"172.21.15.88:6803/74067182","heartbeat_back_addr":"172.21.15.88:6807/74067182","heartbeat_front_addr":"172.21.15.88:6805/74067182","state":["exists","up"]},{"osd":2,"uuid":"60d12aa4-9312-4fa2-8d0b-2b3f42caad00","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":18,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6800","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6801","nonce":4137659051}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6802","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6803","nonce":4137659051}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6806","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6807","nonce":4137659051}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6804","nonce":4137659051},{"type":"v1","addr":"172.21.15.179:6805","nonce":4137659051}]},"public_addr":"172.21.15.179:6801/4137659051","cluster_addr":"172.21.15.179:6803/4137659051","heartbeat_back_addr":"172.21.15.179:6807/4137659051","heartbeat_front_addr":"172.21.15.179:6805/4137659051","state":["exists","up"]},{"osd":3,"uuid":"404c3158-8cb0-4c02-b48a-f9ee5ff7b784","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":25,"up_thru":87,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6810","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6811","nonce":2309603148}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6812","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6813","nonce":2309603148}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6816","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6817","nonce":2309603148}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6814","nonce":2309603148},{"type":"v1","addr":"172.21.15.86:6815","nonce":2309603148}]},"public_addr":"172.21.15.86:6811/2309603148","cluster_addr":"172.21.15.86:6813/2309603148","heartbeat_back_addr":"172.21.15.86:6817/2309603148","heartbeat_front_addr":"172.21.15.86:6815/2309603148","state":["exists","up"]},{"osd":4,"uuid":"458ff202-d781-4ec7-ac93-0b49f835b503","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":33,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6808","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6809","nonce":1818714217}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6810","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6811","nonce":1818714217}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6814","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6815","nonce":1818714217}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6812","nonce":1818714217},{"type":"v1","addr":"172.21.15.88:6813","nonce":1818714217}]},"public_addr":"172.21.15.88:6809/1818714217","cluster_addr":"172.21.15.88:6811/1818714217","heartbeat_back_addr":"172.21.15.88:6815/1818714217","heartbeat_front_addr":"172.21.15.88:6813/1818714217","state":["exists","up"]},{"osd":5,"uuid":"8123cc90-bd8b-4204-bf31-46b34e9f589d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":38,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6808","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6809","nonce":863840990}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6810","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6811","nonce":863840990}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6814","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6815","nonce":863840990}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6812","nonce":863840990},{"type":"v1","addr":"172.21.15.179:6813","nonce":863840990}]},"public_addr":"172.21.15.179:6809/863840990","cluster_addr":"172.21.15.179:6811/863840990","heartbeat_back_addr":"172.21.15.179:6815/863840990","heartbeat_front_addr":"172.21.15.179:6813/863840990","state":["exists","up"]},{"osd":6,"uuid":"aac9d7c7-659a-4110-903a-a871c086066f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":46,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6818","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6819","nonce":1224231118}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6820","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6821","nonce":1224231118}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6824","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6825","nonce":1224231118}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6822","nonce":1224231118},{"type":"v1","addr":"172.21.15.86:6823","nonce":1224231118}]},"public_addr":"172.21.15.86:6819/1224231118","cluster_addr":"172.21.15.86:6821/1224231118","heartbeat_back_addr":"172.21.15.86:6825/1224231118","heartbeat_front_addr":"172.21.15.86:6823/1224231118","state":["exists","up"]},{"osd":7,"uuid":"df5d1f4a-bacf-48fc-b35b-cc5bbf5aa9a5","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":51,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6816","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6817","nonce":136629878}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6818","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6819","nonce":136629878}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6822","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6823","nonce":136629878}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6820","nonce":136629878},{"type":"v1","addr":"172.21.15.88:6821","nonce":136629878}]},"public_addr":"172.21.15.88:6817/136629878","cluster_addr":"172.21.15.88:6819/136629878","heartbeat_back_addr":"172.21.15.88:6823/136629878","heartbeat_front_addr":"172.21.15.88:6821/136629878","state":["exists","up"]},{"osd":8,"uuid":"63e59f4b-0426-46db-92f7-c55dae61b5ee","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":59,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6816","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6817","nonce":1770418957}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6818","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6819","nonce":1770418957}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6822","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6823","nonce":1770418957}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6820","nonce":1770418957},{"type":"v1","addr":"172.21.15.179:6821","nonce":1770418957}]},"public_addr":"172.21.15.179:6817/1770418957","cluster_addr":"172.21.15.179:6819/1770418957","heartbeat_back_addr":"172.21.15.179:6823/1770418957","heartbeat_front_addr":"172.21.15.179:6821/1770418957","state":["exists","up"]},{"osd":9,"uuid":"5841bb36-ed6b-45d8-b857-eeab139c44e6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":67,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6826","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6827","nonce":1108500274}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6828","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6829","nonce":1108500274}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6832","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6833","nonce":1108500274}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.86:6830","nonce":1108500274},{"type":"v1","addr":"172.21.15.86:6831","nonce":1108500274}]},"public_addr":"172.21.15.86:6827/1108500274","cluster_addr":"172.21.15.86:6829/1108500274","heartbeat_back_addr":"172.21.15.86:6833/1108500274","heartbeat_front_addr":"172.21.15.86:6831/1108500274","state":["exists","up"]},{"osd":10,"uuid":"f4c74a2e-59b3-4c6e-8eb9-19fc7a0df78c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":72,"up_thru":90,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6824","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6825","nonce":1294275353}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6826","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6827","nonce":1294275353}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6830","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6831","nonce":1294275353}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.88:6828","nonce":1294275353},{"type":"v1","addr":"172.21.15.88:6829","nonce":1294275353}]},"public_addr":"172.21.15.88:6825/1294275353","cluster_addr":"172.21.15.88:6827/1294275353","heartbeat_back_addr":"172.21.15.88:6831/1294275353","heartbeat_front_addr":"172.21.15.88:6829/1294275353","state":["exists","up"]},{"osd":11,"uuid":"e0d4cc76-e7a6-4a6b-909c-95488ccd5a36","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":80,"up_thru":92,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6824","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6825","nonce":4007070233}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6826","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6827","nonce":4007070233}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6830","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6831","nonce":4007070233}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.179:6828","nonce":4007070233},{"type":"v1","addr":"172.21.15.179:6829","nonce":4007070233}]},"public_addr":"172.21.15.179:6825/4007070233","cluster_addr":"172.21.15.179:6827/4007070233","heartbeat_back_addr":"172.21.15.179:6831/4007070233","heartbeat_front_addr":"172.21.15.179:6829/4007070233","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:30:55.210605+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:18.723090+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:31:42.547979+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:06.421622+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:30.858315+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:32:55.455313+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:20.623369+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:33:46.185111+0000","dead_epoch":0},{"osd":8,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:11.103627+0000","dead_epoch":0},{"osd":9,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:34:37.297545+0000","dead_epoch":0},{"osd":10,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:02.637431+0000","dead_epoch":0},{"osd":11,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540701547738038271,"old_weight":0,"last_purged_snaps_scrub":"2024-04-21T08:35:28.199292+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[{"pgid":"2.e","mappings":[{"from":2,"to":4}]},{"pgid":"2.f","mappings":[{"from":8,"to":11}]},{"pgid":"2.10","mappings":[{"from":2,"to":1}]},{"pgid":"2.1f","mappings":[{"from":7,"to":11}]},{"pgid":"3.0","mappings":[{"from":4,"to":8}]},{"pgid":"3.3","mappings":[{"from":4,"to":9}]},{"pgid":"3.e","mappings":[{"from":5,"to":9}]},{"pgid":"3.36","mappings":[{"from":4,"to":9}]},{"pgid":"4.5","mappings":[{"from":9,"to":10}]},{"pgid":"4.14","mappings":[{"from":9,"to":8}]},{"pgid":"4.1a","mappings":[{"from":5,"to":8}]},{"pgid":"4.1f","mappings":[{"from":9,"to":8}]}],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"172.21.15.86:0/1068089552":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/4273877219":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/2707594358":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/4060044103":"2024-04-22T08:38:46.631621+0000","172.21.15.86:6800/1364156965":"2024-04-22T08:38:46.631621+0000","172.21.15.86:6800/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/1364156965":"2024-04-22T08:38:46.631621+0000","172.21.15.86:0/1412393225":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6800/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/1687852404":"2024-04-22T08:29:23.190001+0000","172.21.15.86:6801/2386990413":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2020707404":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/2568130933":"2024-04-22T08:28:57.384403+0000","172.21.15.86:0/3952607648":"2024-04-22T08:28:57.384403+0000","172.21.15.86:6801/2143206409":"2024-04-22T08:29:23.190001+0000","172.21.15.86:0/2756903426":"2024-04-22T08:29:23.190001+0000"},"range_blocklist":{},"erasure_code_profiles":{"cephfs_data_ec":{"crush-device-class":"","crush-failure-domain":"osd","crush-num-failure-domains":"0","crush-osds-per-failure-domain":"0","crush-root":"default","jerasure-per-chunk-alignment":"false","k":"2","m":"2","plugin":"jerasure","technique":"reed_sol_van","w":"8"},"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[{"pool":3,"snaps":[{"begin":8,"length":1},{"begin":21,"length":1}]},{"pool":4,"snaps":[{"begin":8,"length":1},{"begin":21,"length":1}]}],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2024-04-21T08:51:59.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.936+0000 7f385784e640 1 -- 172.21.15.86:0/1734716541 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f383c0a3da0 msgr2=0x7f383c0a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:59.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.936+0000 7f385784e640 1 --2- 172.21.15.86:0/1734716541 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f383c0a3da0 0x7f383c0a6260 secure :-1 s=READY pgs=373 cs=0 l=1 rev1=1 crypto rx=0x7f3840005fd0 tx=0x7f3840005850 comp rx=0 tx=0).stop 2024-04-21T08:51:59.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.936+0000 7f385784e640 1 -- 172.21.15.86:0/1734716541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f385006a8f0 msgr2=0x7f385019e400 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:51:59.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.936+0000 7f385784e640 1 --2- 172.21.15.86:0/1734716541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f385006a8f0 0x7f385019e400 secure :-1 s=READY pgs=2159 cs=0 l=1 rev1=1 crypto rx=0x7f384c005940 tx=0x7f384c02c7e0 comp rx=0 tx=0).stop 2024-04-21T08:51:59.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.936+0000 7f385784e640 1 -- 172.21.15.86:0/1734716541 shutdown_connections 2024-04-21T08:51:59.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.936+0000 7f385784e640 1 --2- 172.21.15.86:0/1734716541 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f383c0a3da0 0x7f383c0a6260 unknown :-1 s=CLOSED pgs=373 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:59.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.936+0000 7f385784e640 1 --2- 172.21.15.86:0/1734716541 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f38501985b0 0x7f3850198a60 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:59.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.936+0000 7f385784e640 1 --2- 172.21.15.86:0/1734716541 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f38501072f0 0x7f385019e940 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:59.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.936+0000 7f385784e640 1 --2- 172.21.15.86:0/1734716541 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f385006a8f0 0x7f385019e400 unknown :-1 s=CLOSED pgs=2159 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:51:59.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.936+0000 7f385784e640 1 -- 172.21.15.86:0/1734716541 >> 172.21.15.86:0/1734716541 conn(0x7f3850077e30 msgr2=0x7f38500ff440 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:51:59.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.936+0000 7f385784e640 1 -- 172.21.15.86:0/1734716541 shutdown_connections 2024-04-21T08:51:59.936 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:51:59.936+0000 7f385784e640 1 -- 172.21.15.86:0/1734716541 wait complete. 2024-04-21T08:51:59.947 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs fail cephfs 2024-04-21T08:52:00.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.048+0000 7fe71414b640 1 -- 172.21.15.86:0/1037133910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe70c103af0 msgr2=0x7fe70c10bfd0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:00.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.048+0000 7fe71414b640 1 --2- 172.21.15.86:0/1037133910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe70c103af0 0x7fe70c10bfd0 secure :-1 s=READY pgs=2160 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc009900 tx=0x7fe6fc02bd60 comp rx=0 tx=0).stop 2024-04-21T08:52:00.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.048+0000 7fe71414b640 1 -- 172.21.15.86:0/1037133910 shutdown_connections 2024-04-21T08:52:00.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.048+0000 7fe71414b640 1 --2- 172.21.15.86:0/1037133910 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe70c10c510 0x7fe70c10e9b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:00.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.048+0000 7fe71414b640 1 --2- 172.21.15.86:0/1037133910 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe70c103af0 0x7fe70c10bfd0 unknown :-1 s=CLOSED pgs=2160 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:00.050 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.048+0000 7fe71414b640 1 --2- 172.21.15.86:0/1037133910 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe70c1031d0 0x7fe70c1035b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:00.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.048+0000 7fe71414b640 1 -- 172.21.15.86:0/1037133910 >> 172.21.15.86:0/1037133910 conn(0x7fe70c077f50 msgr2=0x7fe70c078360 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:00.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.048+0000 7fe71414b640 1 -- 172.21.15.86:0/1037133910 shutdown_connections 2024-04-21T08:52:00.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.048+0000 7fe71414b640 1 -- 172.21.15.86:0/1037133910 wait complete. 2024-04-21T08:52:00.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.048+0000 7fe71414b640 1 Processor -- start 2024-04-21T08:52:00.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.048+0000 7fe71414b640 1 -- start start 2024-04-21T08:52:00.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.049+0000 7fe71414b640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe70c1031d0 0x7fe70c198740 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:00.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.049+0000 7fe71414b640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe70c103af0 0x7fe70c198c80 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:00.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.049+0000 7fe71414b640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe70c10c510 0x7fe70c19d000 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:00.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.049+0000 7fe71414b640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c1121a0 con 0x7fe70c1031d0 2024-04-21T08:52:00.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.049+0000 7fe71414b640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c112020 con 0x7fe70c103af0 2024-04-21T08:52:00.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.049+0000 7fe71414b640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fe70c112320 con 0x7fe70c10c510 2024-04-21T08:52:00.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.049+0000 7fe711ee7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe70c1031d0 0x7fe70c198740 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:00.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.049+0000 7fe711ee7640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe70c1031d0 0x7fe70c198740 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42140/0 (socket says 172.21.15.86:42140) 2024-04-21T08:52:00.051 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.049+0000 7fe711ee7640 1 -- 172.21.15.86:0/4205049208 learned_addr learned my addr 172.21.15.86:0/4205049208 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:52:00.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.049+0000 7fe7126e8640 1 --2- 172.21.15.86:0/4205049208 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe70c10c510 0x7fe70c19d000 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:00.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.049+0000 7fe7116e6640 1 --2- 172.21.15.86:0/4205049208 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe70c103af0 0x7fe70c198c80 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:00.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.049+0000 7fe711ee7640 1 -- 172.21.15.86:0/4205049208 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe70c10c510 msgr2=0x7fe70c19d000 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:00.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.049+0000 7fe711ee7640 1 --2- 172.21.15.86:0/4205049208 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe70c10c510 0x7fe70c19d000 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:00.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.049+0000 7fe711ee7640 1 -- 172.21.15.86:0/4205049208 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe70c103af0 msgr2=0x7fe70c198c80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:00.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.049+0000 7fe711ee7640 1 --2- 172.21.15.86:0/4205049208 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe70c103af0 0x7fe70c198c80 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:00.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.049+0000 7fe711ee7640 1 -- 172.21.15.86:0/4205049208 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fe70c19d730 con 0x7fe70c1031d0 2024-04-21T08:52:00.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.049+0000 7fe7116e6640 1 --2- 172.21.15.86:0/4205049208 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe70c103af0 0x7fe70c198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:52:00.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.049+0000 7fe711ee7640 1 --2- 172.21.15.86:0/4205049208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe70c1031d0 0x7fe70c198740 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7fe70800e9f0 tx=0x7fe70800eeb0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:00.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.049+0000 7fe702ffd640 1 -- 172.21.15.86:0/4205049208 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe70800cd80 con 0x7fe70c1031d0 2024-04-21T08:52:00.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.049+0000 7fe702ffd640 1 -- 172.21.15.86:0/4205049208 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fe7080108e0 con 0x7fe70c1031d0 2024-04-21T08:52:00.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.049+0000 7fe71414b640 1 -- 172.21.15.86:0/4205049208 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fe70c19da20 con 0x7fe70c1031d0 2024-04-21T08:52:00.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.049+0000 7fe702ffd640 1 -- 172.21.15.86:0/4205049208 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fe708018980 con 0x7fe70c1031d0 2024-04-21T08:52:00.052 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.049+0000 7fe71414b640 1 -- 172.21.15.86:0/4205049208 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fe70c1a52b0 con 0x7fe70c1031d0 2024-04-21T08:52:00.053 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.050+0000 7fe71414b640 1 -- 172.21.15.86:0/4205049208 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fe70c068d70 con 0x7fe70c1031d0 2024-04-21T08:52:00.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.051+0000 7fe702ffd640 1 -- 172.21.15.86:0/4205049208 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fe708010430 con 0x7fe70c1031d0 2024-04-21T08:52:00.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.052+0000 7fe702ffd640 1 --2- 172.21.15.86:0/4205049208 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe6f80a3d20 0x7fe6f80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:00.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.052+0000 7fe7116e6640 1 --2- 172.21.15.86:0/4205049208 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe6f80a3d20 0x7fe6f80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:00.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.052+0000 7fe702ffd640 1 -- 172.21.15.86:0/4205049208 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(108..108 src has 1..108) ==== 9496+0+0 (secure 0 0 0) 0x7fe7080d1410 con 0x7fe70c1031d0 2024-04-21T08:52:00.054 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.052+0000 7fe7116e6640 1 --2- 172.21.15.86:0/4205049208 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe6f80a3d20 0x7fe6f80a61e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc036040 tx=0x7fe6fc000bc0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:00.055 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.055+0000 7fe702ffd640 1 -- 172.21.15.86:0/4205049208 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fe70809d6e0 con 0x7fe70c1031d0 2024-04-21T08:52:00.289 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.290+0000 7fe71414b640 1 -- 172.21.15.86:0/4205049208 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs fail", "fs_name": "cephfs"} v 0) -- 0x7fe70c199a90 con 0x7fe70c1031d0 2024-04-21T08:52:00.291 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:00 smithi086 ceph-mon[27939]: pgmap v413: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-21T08:52:00.291 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:00 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/839405779' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:52:00.292 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:00 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/2658717841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:52:00.292 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:00 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1734716541' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:52:00.292 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.292+0000 7fe702ffd640 1 -- 172.21.15.86:0/4205049208 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs fail", "fs_name": "cephfs"}]=0 cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. v30) ==== 161+0+0 (secure 0 0 0) 0x7fe70809cfe0 con 0x7fe70c1031d0 2024-04-21T08:52:00.292 INFO:teuthology.orchestra.run.smithi086.stderr:cephfs marked not joinable; MDS cannot join the cluster. All MDS ranks marked failed. 2024-04-21T08:52:00.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.296+0000 7fe71414b640 1 -- 172.21.15.86:0/4205049208 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe6f80a3d20 msgr2=0x7fe6f80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:00.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.296+0000 7fe71414b640 1 --2- 172.21.15.86:0/4205049208 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe6f80a3d20 0x7fe6f80a61e0 secure :-1 s=READY pgs=374 cs=0 l=1 rev1=1 crypto rx=0x7fe6fc036040 tx=0x7fe6fc000bc0 comp rx=0 tx=0).stop 2024-04-21T08:52:00.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.296+0000 7fe71414b640 1 -- 172.21.15.86:0/4205049208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe70c1031d0 msgr2=0x7fe70c198740 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:00.295 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.296+0000 7fe71414b640 1 --2- 172.21.15.86:0/4205049208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe70c1031d0 0x7fe70c198740 secure :-1 s=READY pgs=2161 cs=0 l=1 rev1=1 crypto rx=0x7fe70800e9f0 tx=0x7fe70800eeb0 comp rx=0 tx=0).stop 2024-04-21T08:52:00.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.296+0000 7fe71414b640 1 -- 172.21.15.86:0/4205049208 shutdown_connections 2024-04-21T08:52:00.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.296+0000 7fe71414b640 1 --2- 172.21.15.86:0/4205049208 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fe6f80a3d20 0x7fe6f80a61e0 unknown :-1 s=CLOSED pgs=374 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:00.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.296+0000 7fe71414b640 1 --2- 172.21.15.86:0/4205049208 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fe70c10c510 0x7fe70c19d000 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:00.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.296+0000 7fe71414b640 1 --2- 172.21.15.86:0/4205049208 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fe70c103af0 0x7fe70c198c80 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:00.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.296+0000 7fe71414b640 1 --2- 172.21.15.86:0/4205049208 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fe70c1031d0 0x7fe70c198740 unknown :-1 s=CLOSED pgs=2161 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:00.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.296+0000 7fe71414b640 1 -- 172.21.15.86:0/4205049208 >> 172.21.15.86:0/4205049208 conn(0x7fe70c077f50 msgr2=0x7fe70c0fe010 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:00.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.296+0000 7fe71414b640 1 -- 172.21.15.86:0/4205049208 shutdown_connections 2024-04-21T08:52:00.296 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.296+0000 7fe71414b640 1 -- 172.21.15.86:0/4205049208 wait complete. 2024-04-21T08:52:00.312 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph fs rm cephfs --yes-i-really-mean-it 2024-04-21T08:52:00.335 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:00 smithi088 ceph-mon[31259]: pgmap v413: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-21T08:52:00.336 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:00 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/839405779' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:52:00.336 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:00 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/2658717841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:52:00.336 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:00 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1734716541' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:52:00.343 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:00 smithi179 ceph-mon[31829]: pgmap v413: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 6.2 KiB/s rd, 12 op/s 2024-04-21T08:52:00.344 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:00 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/839405779' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:52:00.344 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:00 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/2658717841' entity='client.admin' cmd=[{"prefix": "fs dump", "format": "json"}]: dispatch 2024-04-21T08:52:00.344 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:00 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1734716541' entity='client.admin' cmd=[{"prefix": "osd dump", "format": "json"}]: dispatch 2024-04-21T08:52:00.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.418+0000 7f5e30c8c640 1 -- 172.21.15.86:0/273724300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e2c106160 msgr2=0x7f5e2c10dd30 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:00.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.418+0000 7f5e30c8c640 1 --2- 172.21.15.86:0/273724300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e2c106160 0x7f5e2c10dd30 secure :-1 s=READY pgs=2171 cs=0 l=1 rev1=1 crypto rx=0x7f5e24009900 tx=0x7f5e2402bc30 comp rx=0 tx=0).stop 2024-04-21T08:52:00.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.418+0000 7f5e30c8c640 1 -- 172.21.15.86:0/273724300 shutdown_connections 2024-04-21T08:52:00.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.418+0000 7f5e30c8c640 1 --2- 172.21.15.86:0/273724300 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e2c106160 0x7f5e2c10dd30 unknown :-1 s=CLOSED pgs=2171 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:00.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.418+0000 7f5e30c8c640 1 --2- 172.21.15.86:0/273724300 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e2c101a40 0x7f5e2c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:00.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.418+0000 7f5e30c8c640 1 --2- 172.21.15.86:0/273724300 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e2c101090 0x7f5e2c101470 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:00.420 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.418+0000 7f5e30c8c640 1 -- 172.21.15.86:0/273724300 >> 172.21.15.86:0/273724300 conn(0x7f5e2c0771d0 msgr2=0x7f5e2c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:00.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.418+0000 7f5e30c8c640 1 -- 172.21.15.86:0/273724300 shutdown_connections 2024-04-21T08:52:00.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.418+0000 7f5e30c8c640 1 -- 172.21.15.86:0/273724300 wait complete. 2024-04-21T08:52:00.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.419+0000 7f5e30c8c640 1 Processor -- start 2024-04-21T08:52:00.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.419+0000 7f5e30c8c640 1 -- start start 2024-04-21T08:52:00.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.419+0000 7f5e30c8c640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e2c101090 0x7f5e2c199490 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:00.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.419+0000 7f5e30c8c640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e2c106160 0x7f5e2c1999d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:00.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.419+0000 7f5e30c8c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e2c199f60 0x7f5e2c1a4510 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:00.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.419+0000 7f5e30c8c640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c1110d0 con 0x7f5e2c199f60 2024-04-21T08:52:00.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.419+0000 7f5e30c8c640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c110f50 con 0x7f5e2c106160 2024-04-21T08:52:00.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.419+0000 7f5e30c8c640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f5e2c111250 con 0x7f5e2c101090 2024-04-21T08:52:00.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.419+0000 7f5e2ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e2c199f60 0x7f5e2c1a4510 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:00.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.419+0000 7f5e2ad9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e2c199f60 0x7f5e2c1a4510 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42206/0 (socket says 172.21.15.86:42206) 2024-04-21T08:52:00.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.419+0000 7f5e2ad9d640 1 -- 172.21.15.86:0/1298954196 learned_addr learned my addr 172.21.15.86:0/1298954196 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:52:00.421 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.419+0000 7f5e2a59c640 1 --2- 172.21.15.86:0/1298954196 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e2c101090 0x7f5e2c199490 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:00.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.419+0000 7f5e29d9b640 1 --2- 172.21.15.86:0/1298954196 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e2c106160 0x7f5e2c1999d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:00.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.419+0000 7f5e2ad9d640 1 -- 172.21.15.86:0/1298954196 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e2c101090 msgr2=0x7f5e2c199490 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:00.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.419+0000 7f5e2ad9d640 1 --2- 172.21.15.86:0/1298954196 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e2c101090 0x7f5e2c199490 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:00.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.419+0000 7f5e2ad9d640 1 -- 172.21.15.86:0/1298954196 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e2c106160 msgr2=0x7f5e2c1999d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:00.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.419+0000 7f5e2ad9d640 1 --2- 172.21.15.86:0/1298954196 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e2c106160 0x7f5e2c1999d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:00.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.419+0000 7f5e2ad9d640 1 -- 172.21.15.86:0/1298954196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f5e2c1a4ce0 con 0x7f5e2c199f60 2024-04-21T08:52:00.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.419+0000 7f5e29d9b640 1 --2- 172.21.15.86:0/1298954196 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e2c106160 0x7f5e2c1999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:52:00.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.419+0000 7f5e2a59c640 1 --2- 172.21.15.86:0/1298954196 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e2c101090 0x7f5e2c199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:52:00.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.419+0000 7f5e2ad9d640 1 --2- 172.21.15.86:0/1298954196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e2c199f60 0x7f5e2c1a4510 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7f5e24009230 tx=0x7f5e2402c710 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:00.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.420+0000 7f5e237fe640 1 -- 172.21.15.86:0/1298954196 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e24043020 con 0x7f5e2c199f60 2024-04-21T08:52:00.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.420+0000 7f5e237fe640 1 -- 172.21.15.86:0/1298954196 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f5e2402caf0 con 0x7f5e2c199f60 2024-04-21T08:52:00.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.420+0000 7f5e30c8c640 1 -- 172.21.15.86:0/1298954196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f5e2c1a4f70 con 0x7f5e2c199f60 2024-04-21T08:52:00.422 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.420+0000 7f5e237fe640 1 -- 172.21.15.86:0/1298954196 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f5e24003990 con 0x7f5e2c199f60 2024-04-21T08:52:00.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.420+0000 7f5e30c8c640 1 -- 172.21.15.86:0/1298954196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f5e2c1a5490 con 0x7f5e2c199f60 2024-04-21T08:52:00.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.421+0000 7f5e30c8c640 1 -- 172.21.15.86:0/1298954196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f5e2c102560 con 0x7f5e2c199f60 2024-04-21T08:52:00.423 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.422+0000 7f5e237fe640 1 -- 172.21.15.86:0/1298954196 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f5e24046020 con 0x7f5e2c199f60 2024-04-21T08:52:00.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.423+0000 7f5e237fe640 1 --2- 172.21.15.86:0/1298954196 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5e180a3d20 0x7f5e180a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:00.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.423+0000 7f5e2a59c640 1 --2- 172.21.15.86:0/1298954196 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5e180a3d20 0x7f5e180a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:00.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.423+0000 7f5e237fe640 1 -- 172.21.15.86:0/1298954196 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9838+0+0 (secure 0 0 0) 0x7f5e240f3220 con 0x7f5e2c199f60 2024-04-21T08:52:00.424 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.423+0000 7f5e2a59c640 1 --2- 172.21.15.86:0/1298954196 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5e180a3d20 0x7f5e180a61e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c007e70 tx=0x7f5e1c007850 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:00.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.426+0000 7f5e237fe640 1 -- 172.21.15.86:0/1298954196 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f5e2404d020 con 0x7f5e2c199f60 2024-04-21T08:52:00.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:00 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mon-a[27935]: 2024-04-21T08:52:00.291+0000 7ff1b5f29700 -1 log_channel(cluster) log [ERR] : Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-21T08:52:00.617 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 21 08:52:00 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-g[66819]: ignoring --setuser ceph since I am not root 2024-04-21T08:52:00.617 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 21 08:52:00 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-g[66819]: ignoring --setgroup ceph since I am not root 2024-04-21T08:52:00.617 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 21 08:52:00 smithi086 ceph-mds[66823]: -- 172.21.15.86:0/2162552147 <== mon.0 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55d9a59a9a40 con 0x55d9a597cc00 2024-04-21T08:52:00.617 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 21 08:52:00 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-g[66819]: starting mds.g at 2024-04-21T08:52:00.617 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 21 08:52:00 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-a[67178]: ignoring --setuser ceph since I am not root 2024-04-21T08:52:00.618 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 21 08:52:00 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-a[67178]: ignoring --setgroup ceph since I am not root 2024-04-21T08:52:00.618 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 21 08:52:00 smithi086 ceph-mds[67182]: -- 172.21.15.86:0/2635206607 <== mon.0 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e1fe5e5a40 con 0x55e1fe5b8800 2024-04-21T08:52:00.618 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 21 08:52:00 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-a[67178]: starting mds.a at 2024-04-21T08:52:00.644 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:00.645+0000 7f5e30c8c640 1 -- 172.21.15.86:0/1298954196 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true} v 0) -- 0x7f5e2c101a40 con 0x7f5e2c199f60 2024-04-21T08:52:00.714 INFO:journalctl@ceph.mds.k.smithi088.stdout:Apr 21 08:52:00 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-k[60695]: ignoring --setuser ceph since I am not root 2024-04-21T08:52:00.714 INFO:journalctl@ceph.mds.k.smithi088.stdout:Apr 21 08:52:00 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-k[60695]: ignoring --setgroup ceph since I am not root 2024-04-21T08:52:00.714 INFO:journalctl@ceph.mds.k.smithi088.stdout:Apr 21 08:52:00 smithi088 ceph-mds[60699]: -- 172.21.15.88:0/1311783857 <== mon.2 v2:172.21.15.88:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55b817edba40 con 0x55b817eaf000 2024-04-21T08:52:00.714 INFO:journalctl@ceph.mds.k.smithi088.stdout:Apr 21 08:52:00 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-k[60695]: starting mds.k at 2024-04-21T08:52:00.715 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 21 08:52:00 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-b[61651]: ignoring --setuser ceph since I am not root 2024-04-21T08:52:00.715 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 21 08:52:00 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-b[61651]: ignoring --setgroup ceph since I am not root 2024-04-21T08:52:00.715 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 21 08:52:00 smithi088 ceph-mds[61655]: -- 172.21.15.88:0/3612353644 <== mon.0 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55cbaf545a40 con 0x55cbaf518800 2024-04-21T08:52:00.715 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 21 08:52:00 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-b[61651]: starting mds.b at 2024-04-21T08:52:00.715 INFO:journalctl@ceph.mds.e.smithi088.stdout:Apr 21 08:52:00 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-e[61012]: ignoring --setuser ceph since I am not root 2024-04-21T08:52:00.715 INFO:journalctl@ceph.mds.e.smithi088.stdout:Apr 21 08:52:00 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-e[61012]: ignoring --setgroup ceph since I am not root 2024-04-21T08:52:00.715 INFO:journalctl@ceph.mds.e.smithi088.stdout:Apr 21 08:52:00 smithi088 ceph-mds[61016]: -- 172.21.15.88:0/973639490 <== mon.1 v2:172.21.15.179:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x55d7a59d01a0 con 0x55d7a59a6c00 2024-04-21T08:52:00.715 INFO:journalctl@ceph.mds.e.smithi088.stdout:Apr 21 08:52:00 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-e[61012]: starting mds.e at 2024-04-21T08:52:00.716 INFO:journalctl@ceph.mds.h.smithi088.stdout:Apr 21 08:52:00 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-h[61329]: ignoring --setuser ceph since I am not root 2024-04-21T08:52:00.716 INFO:journalctl@ceph.mds.h.smithi088.stdout:Apr 21 08:52:00 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-h[61329]: ignoring --setgroup ceph since I am not root 2024-04-21T08:52:00.716 INFO:journalctl@ceph.mds.h.smithi088.stdout:Apr 21 08:52:00 smithi088 ceph-mds[61333]: -- 172.21.15.88:0/3108921120 <== mon.2 v2:172.21.15.88:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x560f022eda40 con 0x560f022c0800 2024-04-21T08:52:00.716 INFO:journalctl@ceph.mds.h.smithi088.stdout:Apr 21 08:52:00 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-h[61329]: starting mds.h at 2024-04-21T08:52:00.747 INFO:journalctl@ceph.mds.f.smithi179.stdout:Apr 21 08:52:00 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-f[61204]: ignoring --setuser ceph since I am not root 2024-04-21T08:52:00.747 INFO:journalctl@ceph.mds.f.smithi179.stdout:Apr 21 08:52:00 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-f[61204]: ignoring --setgroup ceph since I am not root 2024-04-21T08:52:00.747 INFO:journalctl@ceph.mds.f.smithi179.stdout:Apr 21 08:52:00 smithi179 ceph-mds[61208]: -- 172.21.15.179:0/3634593051 <== mon.2 v2:172.21.15.88:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55e81a9f1a40 con 0x55e81a9c4800 2024-04-21T08:52:00.747 INFO:journalctl@ceph.mds.f.smithi179.stdout:Apr 21 08:52:00 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-f[61204]: starting mds.f at 2024-04-21T08:52:00.747 INFO:journalctl@ceph.mds.l.smithi179.stdout:Apr 21 08:52:00 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-l[60895]: ignoring --setuser ceph since I am not root 2024-04-21T08:52:00.747 INFO:journalctl@ceph.mds.l.smithi179.stdout:Apr 21 08:52:00 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-l[60895]: ignoring --setgroup ceph since I am not root 2024-04-21T08:52:00.747 INFO:journalctl@ceph.mds.l.smithi179.stdout:Apr 21 08:52:00 smithi179 ceph-mds[60899]: -- 172.21.15.179:0/1657346025 <== mon.0 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x55c81d8a3a40 con 0x55c81d876c00 2024-04-21T08:52:00.747 INFO:journalctl@ceph.mds.l.smithi179.stdout:Apr 21 08:52:00 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-l[60895]: starting mds.l at 2024-04-21T08:52:00.748 INFO:journalctl@ceph.mds.i.smithi179.stdout:Apr 21 08:52:00 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-i[61826]: ignoring --setuser ceph since I am not root 2024-04-21T08:52:00.748 INFO:journalctl@ceph.mds.i.smithi179.stdout:Apr 21 08:52:00 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-i[61826]: ignoring --setgroup ceph since I am not root 2024-04-21T08:52:00.748 INFO:journalctl@ceph.mds.i.smithi179.stdout:Apr 21 08:52:00 smithi179 ceph-mds[61830]: -- 172.21.15.179:0/2525088677 <== mon.1 v2:172.21.15.179:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x558e24d0da40 con 0x558e24ce1000 2024-04-21T08:52:00.748 INFO:journalctl@ceph.mds.i.smithi179.stdout:Apr 21 08:52:00 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-i[61826]: starting mds.i at 2024-04-21T08:52:00.748 INFO:journalctl@ceph.mds.c.smithi179.stdout:Apr 21 08:52:00 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-c[61508]: ignoring --setuser ceph since I am not root 2024-04-21T08:52:00.748 INFO:journalctl@ceph.mds.c.smithi179.stdout:Apr 21 08:52:00 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-c[61508]: ignoring --setgroup ceph since I am not root 2024-04-21T08:52:00.748 INFO:journalctl@ceph.mds.c.smithi179.stdout:Apr 21 08:52:00 smithi179 ceph-mds[61512]: -- 172.21.15.179:0/2842657352 <== mon.0 v2:172.21.15.86:3300/0 4 ==== auth_reply(proto 2 0 (0) Success) ==== 194+0+0 (secure 0 0 0) 0x5635ec6e7a40 con 0x5635ec6bac00 2024-04-21T08:52:00.748 INFO:journalctl@ceph.mds.c.smithi179.stdout:Apr 21 08:52:00 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-c[61508]: starting mds.c at 2024-04-21T08:52:01.299 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.300+0000 7f5e237fe640 1 -- 172.21.15.86:0/1298954196 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]=0 v31) ==== 104+0+0 (secure 0 0 0) 0x7f5e240bf4b0 con 0x7f5e2c199f60 2024-04-21T08:52:01.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.303+0000 7f5e30c8c640 1 -- 172.21.15.86:0/1298954196 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5e180a3d20 msgr2=0x7f5e180a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:01.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.303+0000 7f5e30c8c640 1 --2- 172.21.15.86:0/1298954196 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5e180a3d20 0x7f5e180a61e0 secure :-1 s=READY pgs=375 cs=0 l=1 rev1=1 crypto rx=0x7f5e1c007e70 tx=0x7f5e1c007850 comp rx=0 tx=0).stop 2024-04-21T08:52:01.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.303+0000 7f5e30c8c640 1 -- 172.21.15.86:0/1298954196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e2c199f60 msgr2=0x7f5e2c1a4510 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:01.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.303+0000 7f5e30c8c640 1 --2- 172.21.15.86:0/1298954196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e2c199f60 0x7f5e2c1a4510 secure :-1 s=READY pgs=2172 cs=0 l=1 rev1=1 crypto rx=0x7f5e24009230 tx=0x7f5e2402c710 comp rx=0 tx=0).stop 2024-04-21T08:52:01.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.303+0000 7f5e30c8c640 1 -- 172.21.15.86:0/1298954196 shutdown_connections 2024-04-21T08:52:01.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.303+0000 7f5e30c8c640 1 --2- 172.21.15.86:0/1298954196 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f5e180a3d20 0x7f5e180a61e0 unknown :-1 s=CLOSED pgs=375 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:01.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.303+0000 7f5e30c8c640 1 --2- 172.21.15.86:0/1298954196 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f5e2c199f60 0x7f5e2c1a4510 unknown :-1 s=CLOSED pgs=2172 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:01.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.303+0000 7f5e30c8c640 1 --2- 172.21.15.86:0/1298954196 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f5e2c106160 0x7f5e2c1999d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:01.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.303+0000 7f5e30c8c640 1 --2- 172.21.15.86:0/1298954196 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f5e2c101090 0x7f5e2c199490 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:01.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.303+0000 7f5e30c8c640 1 -- 172.21.15.86:0/1298954196 >> 172.21.15.86:0/1298954196 conn(0x7f5e2c0771d0 msgr2=0x7f5e2c0febe0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:01.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.303+0000 7f5e30c8c640 1 -- 172.21.15.86:0/1298954196 shutdown_connections 2024-04-21T08:52:01.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.303+0000 7f5e30c8c640 1 -- 172.21.15.86:0/1298954196 wait complete. 2024-04-21T08:52:01.317 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:01 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/4205049208' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-21T08:52:01.317 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:01 smithi086 ceph-mon[27939]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-21T08:52:01.317 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:01 smithi086 ceph-mon[27939]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-21T08:52:01.317 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:01 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/4205049208' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-21T08:52:01.317 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:01 smithi086 ceph-mon[27939]: osdmap e109: 12 total, 12 up, 12 in 2024-04-21T08:52:01.317 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:01 smithi086 ceph-mon[27939]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-21T08:52:01.317 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:01 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1298954196' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-21T08:52:01.319 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_metadata cephfs_metadata --yes-i-really-really-mean-it 2024-04-21T08:52:01.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.423+0000 7f2213911640 1 -- 172.21.15.86:0/399922505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f220c101090 msgr2=0x7f220c101470 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:01.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.423+0000 7f2213911640 1 --2- 172.21.15.86:0/399922505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f220c101090 0x7f220c101470 secure :-1 s=READY pgs=2173 cs=0 l=1 rev1=1 crypto rx=0x7f2208009900 tx=0x7f220802bc30 comp rx=0 tx=0).stop 2024-04-21T08:52:01.425 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.423+0000 7f2213911640 1 -- 172.21.15.86:0/399922505 shutdown_connections 2024-04-21T08:52:01.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.423+0000 7f2213911640 1 --2- 172.21.15.86:0/399922505 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f220c106160 0x7f220c10dd30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:01.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.423+0000 7f2213911640 1 --2- 172.21.15.86:0/399922505 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f220c101a40 0x7f220c105a80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:01.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.423+0000 7f2213911640 1 --2- 172.21.15.86:0/399922505 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f220c101090 0x7f220c101470 unknown :-1 s=CLOSED pgs=2173 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:01.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.423+0000 7f2213911640 1 -- 172.21.15.86:0/399922505 >> 172.21.15.86:0/399922505 conn(0x7f220c0771d0 msgr2=0x7f220c0775e0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:01.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.423+0000 7f2213911640 1 -- 172.21.15.86:0/399922505 shutdown_connections 2024-04-21T08:52:01.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.423+0000 7f2213911640 1 -- 172.21.15.86:0/399922505 wait complete. 2024-04-21T08:52:01.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.424+0000 7f2213911640 1 Processor -- start 2024-04-21T08:52:01.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.424+0000 7f2213911640 1 -- start start 2024-04-21T08:52:01.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.424+0000 7f2213911640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f220c101090 0x7f220c1994f0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:01.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.424+0000 7f2213911640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f220c106160 0x7f220c199a30 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:01.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.424+0000 7f2213911640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f220c19a010 0x7f220c1a4570 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:01.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.424+0000 7f2213911640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f220c1110d0 con 0x7f220c101090 2024-04-21T08:52:01.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.424+0000 7f2213911640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f220c110f50 con 0x7f220c106160 2024-04-21T08:52:01.426 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.424+0000 7f2213911640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f220c111250 con 0x7f220c19a010 2024-04-21T08:52:01.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.424+0000 7f22116ad640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f220c101090 0x7f220c1994f0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:01.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.424+0000 7f22116ad640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f220c101090 0x7f220c1994f0 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42218/0 (socket says 172.21.15.86:42218) 2024-04-21T08:52:01.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.424+0000 7f22116ad640 1 -- 172.21.15.86:0/1572729287 learned_addr learned my addr 172.21.15.86:0/1572729287 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:52:01.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.424+0000 7f2210eac640 1 --2- 172.21.15.86:0/1572729287 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f220c106160 0x7f220c199a30 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:01.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.425+0000 7f2211eae640 1 --2- 172.21.15.86:0/1572729287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f220c19a010 0x7f220c1a4570 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:01.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.425+0000 7f22116ad640 1 -- 172.21.15.86:0/1572729287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f220c19a010 msgr2=0x7f220c1a4570 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:01.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.425+0000 7f22116ad640 1 --2- 172.21.15.86:0/1572729287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f220c19a010 0x7f220c1a4570 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:01.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.425+0000 7f22116ad640 1 -- 172.21.15.86:0/1572729287 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f220c106160 msgr2=0x7f220c199a30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:01.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.425+0000 7f22116ad640 1 --2- 172.21.15.86:0/1572729287 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f220c106160 0x7f220c199a30 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:01.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.425+0000 7f22116ad640 1 -- 172.21.15.86:0/1572729287 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f220c1a4d90 con 0x7f220c101090 2024-04-21T08:52:01.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.425+0000 7f22116ad640 1 --2- 172.21.15.86:0/1572729287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f220c101090 0x7f220c1994f0 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f2208005940 tx=0x7f220802c5f0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:01.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.425+0000 7f22027fc640 1 -- 172.21.15.86:0/1572729287 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f2208043020 con 0x7f220c101090 2024-04-21T08:52:01.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.425+0000 7f22027fc640 1 -- 172.21.15.86:0/1572729287 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f220802ca70 con 0x7f220c101090 2024-04-21T08:52:01.427 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.425+0000 7f2213911640 1 -- 172.21.15.86:0/1572729287 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f220c1a5020 con 0x7f220c101090 2024-04-21T08:52:01.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.425+0000 7f22027fc640 1 -- 172.21.15.86:0/1572729287 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f22080039f0 con 0x7f220c101090 2024-04-21T08:52:01.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.425+0000 7f2213911640 1 -- 172.21.15.86:0/1572729287 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f220c1a5570 con 0x7f220c101090 2024-04-21T08:52:01.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.426+0000 7f2213911640 1 -- 172.21.15.86:0/1572729287 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f220c102560 con 0x7f220c101090 2024-04-21T08:52:01.428 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.427+0000 7f22027fc640 1 -- 172.21.15.86:0/1572729287 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f2208046020 con 0x7f220c101090 2024-04-21T08:52:01.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.427+0000 7f22027fc640 1 --2- 172.21.15.86:0/1572729287 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f21f80a3d20 0x7f21f80a61e0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:01.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.428+0000 7f22027fc640 1 -- 172.21.15.86:0/1572729287 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(109..109 src has 1..109) ==== 9838+0+0 (secure 0 0 0) 0x7f22080f2d00 con 0x7f220c101090 2024-04-21T08:52:01.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.428+0000 7f2210eac640 1 --2- 172.21.15.86:0/1572729287 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f21f80a3d20 0x7f21f80a61e0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:01.429 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.428+0000 7f2210eac640 1 --2- 172.21.15.86:0/1572729287 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f21f80a3d20 0x7f21f80a61e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f220c06b160 tx=0x7f21fc005f90 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:01.431 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.432+0000 7f22027fc640 1 -- 172.21.15.86:0/1572729287 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f220804d020 con 0x7f220c101090 2024-04-21T08:52:01.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:01 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/4205049208' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-21T08:52:01.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:01 smithi088 ceph-mon[31259]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-21T08:52:01.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:01 smithi088 ceph-mon[31259]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-21T08:52:01.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:01 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/4205049208' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-21T08:52:01.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:01 smithi088 ceph-mon[31259]: osdmap e109: 12 total, 12 up, 12 in 2024-04-21T08:52:01.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:01 smithi088 ceph-mon[31259]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-21T08:52:01.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:01 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1298954196' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-21T08:52:01.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:01 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/4205049208' entity='client.admin' cmd=[{"prefix": "fs fail", "fs_name": "cephfs"}]: dispatch 2024-04-21T08:52:01.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:01 smithi179 ceph-mon[31829]: Health check failed: 1 filesystem is degraded (FS_DEGRADED) 2024-04-21T08:52:01.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:01 smithi179 ceph-mon[31829]: Health check failed: 1 filesystem is offline (MDS_ALL_DOWN) 2024-04-21T08:52:01.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:01 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/4205049208' entity='client.admin' cmd='[{"prefix": "fs fail", "fs_name": "cephfs"}]': finished 2024-04-21T08:52:01.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:01 smithi179 ceph-mon[31829]: osdmap e109: 12 total, 12 up, 12 in 2024-04-21T08:52:01.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:01 smithi179 ceph-mon[31829]: fsmap cephfs:0/5 2 up:standby, 5 failed 2024-04-21T08:52:01.497 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:01 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1298954196' entity='client.admin' cmd=[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]: dispatch 2024-04-21T08:52:01.610 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:01.611+0000 7f2213911640 1 -- 172.21.15.86:0/1572729287 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true} v 0) -- 0x7f220c0691c0 con 0x7f220c101090 2024-04-21T08:52:02.303 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.304+0000 7f22027fc640 1 -- 172.21.15.86:0/1572729287 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_metadata' removed v110) ==== 181+0+0 (secure 0 0 0) 0x7f220802cc20 con 0x7f220c101090 2024-04-21T08:52:02.303 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_metadata' removed 2024-04-21T08:52:02.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.306+0000 7f2213911640 1 -- 172.21.15.86:0/1572729287 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f21f80a3d20 msgr2=0x7f21f80a61e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:02.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.306+0000 7f2213911640 1 --2- 172.21.15.86:0/1572729287 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f21f80a3d20 0x7f21f80a61e0 secure :-1 s=READY pgs=386 cs=0 l=1 rev1=1 crypto rx=0x7f220c06b160 tx=0x7f21fc005f90 comp rx=0 tx=0).stop 2024-04-21T08:52:02.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.306+0000 7f2213911640 1 -- 172.21.15.86:0/1572729287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f220c101090 msgr2=0x7f220c1994f0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:02.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.306+0000 7f2213911640 1 --2- 172.21.15.86:0/1572729287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f220c101090 0x7f220c1994f0 secure :-1 s=READY pgs=2174 cs=0 l=1 rev1=1 crypto rx=0x7f2208005940 tx=0x7f220802c5f0 comp rx=0 tx=0).stop 2024-04-21T08:52:02.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.306+0000 7f2213911640 1 -- 172.21.15.86:0/1572729287 shutdown_connections 2024-04-21T08:52:02.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.306+0000 7f2213911640 1 --2- 172.21.15.86:0/1572729287 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f21f80a3d20 0x7f21f80a61e0 unknown :-1 s=CLOSED pgs=386 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:02.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.306+0000 7f2213911640 1 --2- 172.21.15.86:0/1572729287 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f220c19a010 0x7f220c1a4570 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:02.306 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.306+0000 7f2213911640 1 --2- 172.21.15.86:0/1572729287 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f220c106160 0x7f220c199a30 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:02.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.306+0000 7f2213911640 1 --2- 172.21.15.86:0/1572729287 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f220c101090 0x7f220c1994f0 unknown :-1 s=CLOSED pgs=2174 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:02.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.306+0000 7f2213911640 1 -- 172.21.15.86:0/1572729287 >> 172.21.15.86:0/1572729287 conn(0x7f220c0771d0 msgr2=0x7f220c0fedc0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:02.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.307+0000 7f2213911640 1 -- 172.21.15.86:0/1572729287 shutdown_connections 2024-04-21T08:52:02.307 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.307+0000 7f2213911640 1 -- 172.21.15.86:0/1572729287 wait complete. 2024-04-21T08:52:02.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:02 smithi086 ceph-mon[27939]: pgmap v415: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-21T08:52:02.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:02 smithi086 ceph-mon[27939]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-21T08:52:02.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:02 smithi086 ceph-mon[27939]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-21T08:52:02.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:02 smithi086 ceph-mon[27939]: Cluster is now healthy 2024-04-21T08:52:02.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:02 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1298954196' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-21T08:52:02.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:02 smithi086 ceph-mon[27939]: mds.? [v2:172.21.15.86:6838/2192702706,v1:172.21.15.86:6839/2192702706] up:boot 2024-04-21T08:52:02.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:02 smithi086 ceph-mon[27939]: mds.? [v2:172.21.15.88:6833/68820816,v1:172.21.15.88:6837/68820816] up:boot 2024-04-21T08:52:02.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:02 smithi086 ceph-mon[27939]: mds.? [v2:172.21.15.88:6832/2974472004,v1:172.21.15.88:6836/2974472004] up:boot 2024-04-21T08:52:02.321 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:02 smithi086 ceph-mon[27939]: mds.? [v2:172.21.15.88:6834/2721519575,v1:172.21.15.88:6838/2721519575] up:boot 2024-04-21T08:52:02.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:02 smithi086 ceph-mon[27939]: mds.? [v2:172.21.15.88:6835/26155878,v1:172.21.15.88:6839/26155878] up:boot 2024-04-21T08:52:02.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:02 smithi086 ceph-mon[27939]: mds.? [v2:172.21.15.86:6840/3142661991,v1:172.21.15.86:6841/3142661991] up:boot 2024-04-21T08:52:02.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:02 smithi086 ceph-mon[27939]: mds.? [v2:172.21.15.179:6833/94396179,v1:172.21.15.179:6835/94396179] up:boot 2024-04-21T08:52:02.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:02 smithi086 ceph-mon[27939]: mds.? [v2:172.21.15.179:6832/3896411624,v1:172.21.15.179:6834/3896411624] up:boot 2024-04-21T08:52:02.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:02 smithi086 ceph-mon[27939]: mds.? [v2:172.21.15.179:6836/3597358661,v1:172.21.15.179:6838/3597358661] up:boot 2024-04-21T08:52:02.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:02 smithi086 ceph-mon[27939]: mds.? [v2:172.21.15.179:6837/31361855,v1:172.21.15.179:6839/31361855] up:boot 2024-04-21T08:52:02.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:02 smithi086 ceph-mon[27939]: fsmap 12 up:standby 2024-04-21T08:52:02.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:02 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-21T08:52:02.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:02 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T08:52:02.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:02 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-21T08:52:02.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:02 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-21T08:52:02.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:02 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T08:52:02.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:02 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-21T08:52:02.322 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:02 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T08:52:02.323 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:02 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T08:52:02.323 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:02 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T08:52:02.323 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:02 smithi086 ceph-mon[27939]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T08:52:02.323 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:02 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1572729287' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T08:52:02.324 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data cephfs_data --yes-i-really-really-mean-it 2024-04-21T08:52:02.399 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:02 smithi179 ceph-mon[31829]: pgmap v415: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-21T08:52:02.399 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:02 smithi179 ceph-mon[31829]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-21T08:52:02.399 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:02 smithi179 ceph-mon[31829]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-21T08:52:02.399 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:02 smithi179 ceph-mon[31829]: Cluster is now healthy 2024-04-21T08:52:02.399 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:02 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1298954196' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-21T08:52:02.399 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:02 smithi179 ceph-mon[31829]: mds.? [v2:172.21.15.86:6838/2192702706,v1:172.21.15.86:6839/2192702706] up:boot 2024-04-21T08:52:02.399 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:02 smithi179 ceph-mon[31829]: mds.? [v2:172.21.15.88:6833/68820816,v1:172.21.15.88:6837/68820816] up:boot 2024-04-21T08:52:02.399 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:02 smithi179 ceph-mon[31829]: mds.? [v2:172.21.15.88:6832/2974472004,v1:172.21.15.88:6836/2974472004] up:boot 2024-04-21T08:52:02.399 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:02 smithi179 ceph-mon[31829]: mds.? [v2:172.21.15.88:6834/2721519575,v1:172.21.15.88:6838/2721519575] up:boot 2024-04-21T08:52:02.399 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:02 smithi179 ceph-mon[31829]: mds.? [v2:172.21.15.88:6835/26155878,v1:172.21.15.88:6839/26155878] up:boot 2024-04-21T08:52:02.400 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:02 smithi179 ceph-mon[31829]: mds.? [v2:172.21.15.86:6840/3142661991,v1:172.21.15.86:6841/3142661991] up:boot 2024-04-21T08:52:02.400 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:02 smithi179 ceph-mon[31829]: mds.? [v2:172.21.15.179:6833/94396179,v1:172.21.15.179:6835/94396179] up:boot 2024-04-21T08:52:02.400 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:02 smithi179 ceph-mon[31829]: mds.? [v2:172.21.15.179:6832/3896411624,v1:172.21.15.179:6834/3896411624] up:boot 2024-04-21T08:52:02.400 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:02 smithi179 ceph-mon[31829]: mds.? [v2:172.21.15.179:6836/3597358661,v1:172.21.15.179:6838/3597358661] up:boot 2024-04-21T08:52:02.400 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:02 smithi179 ceph-mon[31829]: mds.? [v2:172.21.15.179:6837/31361855,v1:172.21.15.179:6839/31361855] up:boot 2024-04-21T08:52:02.400 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:02 smithi179 ceph-mon[31829]: fsmap 12 up:standby 2024-04-21T08:52:02.400 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:02 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-21T08:52:02.400 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:02 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T08:52:02.400 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:02 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-21T08:52:02.400 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:02 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-21T08:52:02.400 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:02 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T08:52:02.400 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:02 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-21T08:52:02.400 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:02 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T08:52:02.400 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:02 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T08:52:02.401 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:02 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T08:52:02.401 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:02 smithi179 ceph-mon[31829]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T08:52:02.401 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:02 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1572729287' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T08:52:02.432 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.431+0000 7f240d184640 1 -- 172.21.15.86:0/99495426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2408102190 msgr2=0x7f2408102570 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:02.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.431+0000 7f240d184640 1 --2- 172.21.15.86:0/99495426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2408102190 0x7f2408102570 secure :-1 s=READY pgs=2175 cs=0 l=1 rev1=1 crypto rx=0x7f23f8009900 tx=0x7f23f802bc30 comp rx=0 tx=0).stop 2024-04-21T08:52:02.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.431+0000 7f240d184640 1 -- 172.21.15.86:0/99495426 shutdown_connections 2024-04-21T08:52:02.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.431+0000 7f240d184640 1 --2- 172.21.15.86:0/99495426 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2408107260 0x7f240810ee30 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:02.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.431+0000 7f240d184640 1 --2- 172.21.15.86:0/99495426 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2408102b40 0x7f2408106b80 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:02.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.431+0000 7f240d184640 1 --2- 172.21.15.86:0/99495426 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2408102190 0x7f2408102570 unknown :-1 s=CLOSED pgs=2175 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:02.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.431+0000 7f240d184640 1 -- 172.21.15.86:0/99495426 >> 172.21.15.86:0/99495426 conn(0x7f24080793a0 msgr2=0x7f24080797b0 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:02.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.431+0000 7f240d184640 1 -- 172.21.15.86:0/99495426 shutdown_connections 2024-04-21T08:52:02.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.431+0000 7f240d184640 1 -- 172.21.15.86:0/99495426 wait complete. 2024-04-21T08:52:02.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.432+0000 7f240d184640 1 Processor -- start 2024-04-21T08:52:02.433 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.432+0000 7f240d184640 1 -- start start 2024-04-21T08:52:02.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.432+0000 7f240d184640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2408102190 0x7f2408074250 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:02.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.432+0000 7f240d184640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2408107260 0x7f2408074790 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:02.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.432+0000 7f240d184640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2408074cd0 0x7f24081a68d0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:02.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.432+0000 7f240d184640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7f24081121a0 con 0x7f2408102190 2024-04-21T08:52:02.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.432+0000 7f240d184640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7f2408112020 con 0x7f2408074cd0 2024-04-21T08:52:02.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.432+0000 7f240d184640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7f2408112320 con 0x7f2408107260 2024-04-21T08:52:02.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.432+0000 7f2406d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2408102190 0x7f2408074250 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:02.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.432+0000 7f2406d9d640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2408102190 0x7f2408074250 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42228/0 (socket says 172.21.15.86:42228) 2024-04-21T08:52:02.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.432+0000 7f2406d9d640 1 -- 172.21.15.86:0/1377901705 learned_addr learned my addr 172.21.15.86:0/1377901705 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:52:02.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.432+0000 7f240759e640 1 --2- 172.21.15.86:0/1377901705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2408074cd0 0x7f24081a68d0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:02.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.432+0000 7f240659c640 1 --2- 172.21.15.86:0/1377901705 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2408107260 0x7f2408074790 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:02.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.432+0000 7f2406d9d640 1 -- 172.21.15.86:0/1377901705 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2408107260 msgr2=0x7f2408074790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:02.434 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.432+0000 7f2406d9d640 1 --2- 172.21.15.86:0/1377901705 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2408107260 0x7f2408074790 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:02.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.432+0000 7f2406d9d640 1 -- 172.21.15.86:0/1377901705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2408074cd0 msgr2=0x7f24081a68d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:02.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.432+0000 7f2406d9d640 1 --2- 172.21.15.86:0/1377901705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2408074cd0 0x7f24081a68d0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:02.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.432+0000 7f2406d9d640 1 -- 172.21.15.86:0/1377901705 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7f24081a6fd0 con 0x7f2408102190 2024-04-21T08:52:02.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.432+0000 7f240659c640 1 --2- 172.21.15.86:0/1377901705 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2408107260 0x7f2408074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_auth_reply_more state changed! 2024-04-21T08:52:02.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.433+0000 7f2406d9d640 1 --2- 172.21.15.86:0/1377901705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2408102190 0x7f2408074250 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7f23f8005940 tx=0x7f23f802c7e0 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:02.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.433+0000 7f23effff640 1 -- 172.21.15.86:0/1377901705 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f8043020 con 0x7f2408102190 2024-04-21T08:52:02.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.433+0000 7f240d184640 1 -- 172.21.15.86:0/1377901705 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7f24081a7260 con 0x7f2408102190 2024-04-21T08:52:02.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.433+0000 7f23effff640 1 -- 172.21.15.86:0/1377901705 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7f23f802cbe0 con 0x7f2408102190 2024-04-21T08:52:02.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.433+0000 7f23effff640 1 -- 172.21.15.86:0/1377901705 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7f23f8003bf0 con 0x7f2408102190 2024-04-21T08:52:02.435 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.433+0000 7f240d184640 1 -- 172.21.15.86:0/1377901705 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7f24081a77b0 con 0x7f2408102190 2024-04-21T08:52:02.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.434+0000 7f240d184640 1 -- 172.21.15.86:0/1377901705 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7f24080691c0 con 0x7f2408102190 2024-04-21T08:52:02.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.435+0000 7f23effff640 1 -- 172.21.15.86:0/1377901705 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7f23f8046020 con 0x7f2408102190 2024-04-21T08:52:02.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.435+0000 7f23effff640 1 --2- 172.21.15.86:0/1377901705 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f23f00a3d50 0x7f23f00a6210 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:02.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.435+0000 7f240659c640 1 --2- 172.21.15.86:0/1377901705 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f23f00a3d50 0x7f23f00a6210 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:02.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.435+0000 7f23effff640 1 -- 172.21.15.86:0/1377901705 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(110..110 src has 1..110) ==== 9284+0+0 (secure 0 0 0) 0x7f23f80f3300 con 0x7f2408102190 2024-04-21T08:52:02.436 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.436+0000 7f240659c640 1 --2- 172.21.15.86:0/1377901705 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f23f00a3d50 0x7f23f00a6210 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2408075ce0 tx=0x7f23fc00a5c0 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:02.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.439+0000 7f23effff640 1 -- 172.21.15.86:0/1377901705 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7f23f804d020 con 0x7f2408102190 2024-04-21T08:52:02.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:02 smithi088 ceph-mon[31259]: pgmap v415: 129 pgs: 129 active+clean; 3.8 MiB data, 433 MiB used, 1.0 TiB / 1.0 TiB avail; 5.0 KiB/s rd, 9 op/s 2024-04-21T08:52:02.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:02 smithi088 ceph-mon[31259]: Health check cleared: FS_DEGRADED (was: 1 filesystem is degraded) 2024-04-21T08:52:02.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:02 smithi088 ceph-mon[31259]: Health check cleared: MDS_ALL_DOWN (was: 1 filesystem is offline) 2024-04-21T08:52:02.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:02 smithi088 ceph-mon[31259]: Cluster is now healthy 2024-04-21T08:52:02.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:02 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1298954196' entity='client.admin' cmd='[{"prefix": "fs rm", "fs_name": "cephfs", "yes_i_really_mean_it": true}]': finished 2024-04-21T08:52:02.464 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:02 smithi088 ceph-mon[31259]: mds.? [v2:172.21.15.86:6838/2192702706,v1:172.21.15.86:6839/2192702706] up:boot 2024-04-21T08:52:02.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:02 smithi088 ceph-mon[31259]: mds.? [v2:172.21.15.88:6833/68820816,v1:172.21.15.88:6837/68820816] up:boot 2024-04-21T08:52:02.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:02 smithi088 ceph-mon[31259]: mds.? [v2:172.21.15.88:6832/2974472004,v1:172.21.15.88:6836/2974472004] up:boot 2024-04-21T08:52:02.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:02 smithi088 ceph-mon[31259]: mds.? [v2:172.21.15.88:6834/2721519575,v1:172.21.15.88:6838/2721519575] up:boot 2024-04-21T08:52:02.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:02 smithi088 ceph-mon[31259]: mds.? [v2:172.21.15.88:6835/26155878,v1:172.21.15.88:6839/26155878] up:boot 2024-04-21T08:52:02.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:02 smithi088 ceph-mon[31259]: mds.? [v2:172.21.15.86:6840/3142661991,v1:172.21.15.86:6841/3142661991] up:boot 2024-04-21T08:52:02.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:02 smithi088 ceph-mon[31259]: mds.? [v2:172.21.15.179:6833/94396179,v1:172.21.15.179:6835/94396179] up:boot 2024-04-21T08:52:02.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:02 smithi088 ceph-mon[31259]: mds.? [v2:172.21.15.179:6832/3896411624,v1:172.21.15.179:6834/3896411624] up:boot 2024-04-21T08:52:02.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:02 smithi088 ceph-mon[31259]: mds.? [v2:172.21.15.179:6836/3597358661,v1:172.21.15.179:6838/3597358661] up:boot 2024-04-21T08:52:02.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:02 smithi088 ceph-mon[31259]: mds.? [v2:172.21.15.179:6837/31361855,v1:172.21.15.179:6839/31361855] up:boot 2024-04-21T08:52:02.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:02 smithi088 ceph-mon[31259]: fsmap 12 up:standby 2024-04-21T08:52:02.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:02 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "a"}]: dispatch 2024-04-21T08:52:02.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:02 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "e"}]: dispatch 2024-04-21T08:52:02.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:02 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "k"}]: dispatch 2024-04-21T08:52:02.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:02 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "g"}]: dispatch 2024-04-21T08:52:02.465 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:02 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "i"}]: dispatch 2024-04-21T08:52:02.466 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:02 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "l"}]: dispatch 2024-04-21T08:52:02.466 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:02 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "f"}]: dispatch 2024-04-21T08:52:02.466 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:02 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "h"}]: dispatch 2024-04-21T08:52:02.466 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:02 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "b"}]: dispatch 2024-04-21T08:52:02.466 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:02 smithi088 ceph-mon[31259]: from='mgr.15558 172.21.15.86:0/1063261712' entity='mgr.x' cmd=[{"prefix": "mds metadata", "who": "c"}]: dispatch 2024-04-21T08:52:02.466 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:02 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1572729287' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T08:52:02.625 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:02.625+0000 7f240d184640 1 -- 172.21.15.86:0/1377901705 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true} v 0) -- 0x7f240810a490 con 0x7f2408102190 2024-04-21T08:52:03.309 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.309+0000 7f23effff640 1 -- 172.21.15.86:0/1377901705 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data' removed v111) ==== 169+0+0 (secure 0 0 0) 0x7f23f80bf7b0 con 0x7f2408102190 2024-04-21T08:52:03.309 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data' removed 2024-04-21T08:52:03.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.313+0000 7f240d184640 1 -- 172.21.15.86:0/1377901705 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f23f00a3d50 msgr2=0x7f23f00a6210 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:03.313 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.313+0000 7f240d184640 1 --2- 172.21.15.86:0/1377901705 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f23f00a3d50 0x7f23f00a6210 secure :-1 s=READY pgs=393 cs=0 l=1 rev1=1 crypto rx=0x7f2408075ce0 tx=0x7f23fc00a5c0 comp rx=0 tx=0).stop 2024-04-21T08:52:03.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.313+0000 7f240d184640 1 -- 172.21.15.86:0/1377901705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2408102190 msgr2=0x7f2408074250 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:03.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.313+0000 7f240d184640 1 --2- 172.21.15.86:0/1377901705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2408102190 0x7f2408074250 secure :-1 s=READY pgs=2176 cs=0 l=1 rev1=1 crypto rx=0x7f23f8005940 tx=0x7f23f802c7e0 comp rx=0 tx=0).stop 2024-04-21T08:52:03.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.313+0000 7f240d184640 1 -- 172.21.15.86:0/1377901705 shutdown_connections 2024-04-21T08:52:03.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.313+0000 7f240d184640 1 --2- 172.21.15.86:0/1377901705 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7f23f00a3d50 0x7f23f00a6210 unknown :-1 s=CLOSED pgs=393 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:03.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.314+0000 7f240d184640 1 --2- 172.21.15.86:0/1377901705 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7f2408074cd0 0x7f24081a68d0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:03.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.314+0000 7f240d184640 1 --2- 172.21.15.86:0/1377901705 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7f2408107260 0x7f2408074790 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:03.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.314+0000 7f240d184640 1 --2- 172.21.15.86:0/1377901705 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7f2408102190 0x7f2408074250 unknown :-1 s=CLOSED pgs=2176 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:03.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.314+0000 7f240d184640 1 -- 172.21.15.86:0/1377901705 >> 172.21.15.86:0/1377901705 conn(0x7f24080793a0 msgr2=0x7f24080fff40 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:03.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.314+0000 7f240d184640 1 -- 172.21.15.86:0/1377901705 shutdown_connections 2024-04-21T08:52:03.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.314+0000 7f240d184640 1 -- 172.21.15.86:0/1377901705 wait complete. 2024-04-21T08:52:03.328 DEBUG:teuthology.orchestra.run.smithi086:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm cephfs_data_ec cephfs_data_ec --yes-i-really-really-mean-it 2024-04-21T08:52:03.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.436+0000 7fa2232c0640 1 -- 172.21.15.86:0/4010878616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa21c101a40 msgr2=0x7fa21c10dca0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:03.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.436+0000 7fa2232c0640 1 --2- 172.21.15.86:0/4010878616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa21c101a40 0x7fa21c10dca0 secure :-1 s=READY pgs=2177 cs=0 l=1 rev1=1 crypto rx=0x7fa214009ca0 tx=0x7fa21402c0f0 comp rx=0 tx=0).stop 2024-04-21T08:52:03.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.436+0000 7fa2232c0640 1 -- 172.21.15.86:0/4010878616 shutdown_connections 2024-04-21T08:52:03.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.436+0000 7fa2232c0640 1 --2- 172.21.15.86:0/4010878616 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa21c101a40 0x7fa21c10dca0 unknown :-1 s=CLOSED pgs=2177 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:03.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.436+0000 7fa2232c0640 1 --2- 172.21.15.86:0/4010878616 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa21c1010a0 0x7fa21c101500 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:03.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.436+0000 7fa2232c0640 1 --2- 172.21.15.86:0/4010878616 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa21c105dd0 0x7fa21c1061b0 unknown :-1 s=CLOSED pgs=0 cs=0 l=0 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:03.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.436+0000 7fa2232c0640 1 -- 172.21.15.86:0/4010878616 >> 172.21.15.86:0/4010878616 conn(0x7fa21c077c40 msgr2=0x7fa21c078050 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:03.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.436+0000 7fa2232c0640 1 -- 172.21.15.86:0/4010878616 shutdown_connections 2024-04-21T08:52:03.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.436+0000 7fa2232c0640 1 -- 172.21.15.86:0/4010878616 wait complete. 2024-04-21T08:52:03.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.437+0000 7fa2232c0640 1 Processor -- start 2024-04-21T08:52:03.438 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.437+0000 7fa2232c0640 1 -- start start 2024-04-21T08:52:03.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.437+0000 7fa2232c0640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa21c1010a0 0x7fa21c199580 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:03.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.437+0000 7fa2232c0640 1 --2- >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa21c101a40 0x7fa21c199ac0 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:03.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.437+0000 7fa2232c0640 1 --2- >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa21c105dd0 0x7fa21c1a4600 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:03.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.437+0000 7fa2232c0640 1 -- --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_getmap magic: 0 -- 0x7fa21c111160 con 0x7fa21c1010a0 2024-04-21T08:52:03.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.437+0000 7fa2232c0640 1 -- --> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] -- mon_getmap magic: 0 -- 0x7fa21c110fe0 con 0x7fa21c105dd0 2024-04-21T08:52:03.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.437+0000 7fa2232c0640 1 -- --> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] -- mon_getmap magic: 0 -- 0x7fa21c1112e0 con 0x7fa21c101a40 2024-04-21T08:52:03.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.437+0000 7fa22105c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa21c1010a0 0x7fa21c199580 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:03.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.437+0000 7fa22105c640 1 --2- >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa21c1010a0 0x7fa21c199580 unknown :-1 s=HELLO_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).handle_hello peer v2:172.21.15.86:3300/0 says I am v2:172.21.15.86:42254/0 (socket says 172.21.15.86:42254) 2024-04-21T08:52:03.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.437+0000 7fa22105c640 1 -- 172.21.15.86:0/1371408182 learned_addr learned my addr 172.21.15.86:0/1371408182 (peer_addr_for_me v2:172.21.15.86:0/0) 2024-04-21T08:52:03.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.437+0000 7fa22085b640 1 --2- 172.21.15.86:0/1371408182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa21c101a40 0x7fa21c199ac0 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:03.439 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.437+0000 7fa22185d640 1 --2- 172.21.15.86:0/1371408182 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa21c105dd0 0x7fa21c1a4600 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:03.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.437+0000 7fa22105c640 1 -- 172.21.15.86:0/1371408182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa21c101a40 msgr2=0x7fa21c199ac0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:03.441 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.437+0000 7fa22105c640 1 --2- 172.21.15.86:0/1371408182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa21c101a40 0x7fa21c199ac0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:03.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.437+0000 7fa22105c640 1 -- 172.21.15.86:0/1371408182 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa21c105dd0 msgr2=0x7fa21c1a4600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:03.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.437+0000 7fa22105c640 1 --2- 172.21.15.86:0/1371408182 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa21c105dd0 0x7fa21c1a4600 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:03.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.437+0000 7fa22105c640 1 -- 172.21.15.86:0/1371408182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({config=0+,monmap=0+}) -- 0x7fa21c1a4b40 con 0x7fa21c1010a0 2024-04-21T08:52:03.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.437+0000 7fa22185d640 1 --2- 172.21.15.86:0/1371408182 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa21c105dd0 0x7fa21c1a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).send_auth_request state changed! 2024-04-21T08:52:03.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.437+0000 7fa22105c640 1 --2- 172.21.15.86:0/1371408182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa21c1010a0 0x7fa21c199580 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7fa21800d950 tx=0x7fa21800de10 comp rx=0 tx=0).ready entity=mon.0 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:03.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.437+0000 7fa2127fc640 1 -- 172.21.15.86:0/1371408182 <== mon.0 v2:172.21.15.86:3300/0 1 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa218017030 con 0x7fa21c1010a0 2024-04-21T08:52:03.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.438+0000 7fa2127fc640 1 -- 172.21.15.86:0/1371408182 <== mon.0 v2:172.21.15.86:3300/0 2 ==== config(26 keys) ==== 1101+0+0 (secure 0 0 0) 0x7fa218004020 con 0x7fa21c1010a0 2024-04-21T08:52:03.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.438+0000 7fa2232c0640 1 -- 172.21.15.86:0/1371408182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({mgrmap=0+}) -- 0x7fa21c1a4d10 con 0x7fa21c1010a0 2024-04-21T08:52:03.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.438+0000 7fa2127fc640 1 -- 172.21.15.86:0/1371408182 <== mon.0 v2:172.21.15.86:3300/0 3 ==== mon_map magic: 0 ==== 413+0+0 (secure 0 0 0) 0x7fa2180047a0 con 0x7fa21c1010a0 2024-04-21T08:52:03.442 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.438+0000 7fa2232c0640 1 -- 172.21.15.86:0/1371408182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_subscribe({osdmap=0}) -- 0x7fa21c1a5220 con 0x7fa21c1010a0 2024-04-21T08:52:03.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.439+0000 7fa2232c0640 1 -- 172.21.15.86:0/1371408182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "get_command_descriptions"} v 0) -- 0x7fa21c102540 con 0x7fa21c1010a0 2024-04-21T08:52:03.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.440+0000 7fa2127fc640 1 -- 172.21.15.86:0/1371408182 <== mon.0 v2:172.21.15.86:3300/0 4 ==== mgrmap(e 26) ==== 138777+0+0 (secure 0 0 0) 0x7fa218009a30 con 0x7fa21c1010a0 2024-04-21T08:52:03.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.440+0000 7fa2127fc640 1 --2- 172.21.15.86:0/1371408182 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa2080a3da0 0x7fa2080a6260 unknown :-1 s=NONE pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0).connect 2024-04-21T08:52:03.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.440+0000 7fa22085b640 1 --2- 172.21.15.86:0/1371408182 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa2080a3da0 0x7fa2080a6260 unknown :-1 s=BANNER_CONNECTING pgs=0 cs=0 l=1 rev1=0 crypto rx=0 tx=0 comp rx=0 tx=0)._handle_peer_banner_payload supported=3 required=0 2024-04-21T08:52:03.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.440+0000 7fa2127fc640 1 -- 172.21.15.86:0/1371408182 <== mon.0 v2:172.21.15.86:3300/0 5 ==== osd_map(111..111 src has 1..111) ==== 8754+0+0 (secure 0 0 0) 0x7fa2180d0220 con 0x7fa21c1010a0 2024-04-21T08:52:03.443 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.441+0000 7fa22085b640 1 --2- 172.21.15.86:0/1371408182 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa2080a3da0 0x7fa2080a6260 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa21c06b100 tx=0x7fa20c00b410 comp rx=0 tx=0).ready entity=mgr.15558 client_cookie=0 server_cookie=0 in_seq=0 out_seq=0 2024-04-21T08:52:03.444 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.445+0000 7fa2127fc640 1 -- 172.21.15.86:0/1371408182 <== mon.0 v2:172.21.15.86:3300/0 6 ==== mon_command_ack([{"prefix": "get_command_descriptions"}]=0 v0) ==== 72+0+188370 (secure 0 0 0) 0x7fa21809c7d0 con 0x7fa21c1010a0 2024-04-21T08:52:03.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:03 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1572729287' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-21T08:52:03.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:03 smithi086 ceph-mon[27939]: osdmap e110: 12 total, 12 up, 12 in 2024-04-21T08:52:03.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:03 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1377901705' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T08:52:03.617 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:03 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1572729287' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-21T08:52:03.617 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:03 smithi179 ceph-mon[31829]: osdmap e110: 12 total, 12 up, 12 in 2024-04-21T08:52:03.617 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:03 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1377901705' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T08:52:03.623 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:03.624+0000 7fa2232c0640 1 -- 172.21.15.86:0/1371408182 --> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] -- mon_command({"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true} v 0) -- 0x7fa21c0691c0 con 0x7fa21c1010a0 2024-04-21T08:52:03.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:03 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1572729287' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_metadata", "pool2": "cephfs_metadata", "yes_i_really_really_mean_it": true}]': finished 2024-04-21T08:52:03.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:03 smithi088 ceph-mon[31259]: osdmap e110: 12 total, 12 up, 12 in 2024-04-21T08:52:03.714 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:03 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1377901705' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T08:52:04.310 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:04.311+0000 7fa2127fc640 1 -- 172.21.15.86:0/1371408182 <== mon.0 v2:172.21.15.86:3300/0 7 ==== mon_command_ack([{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]=0 pool 'cephfs_data_ec' removed v112) ==== 178+0+0 (secure 0 0 0) 0x7fa21809c0d0 con 0x7fa21c1010a0 2024-04-21T08:52:04.310 INFO:teuthology.orchestra.run.smithi086.stderr:pool 'cephfs_data_ec' removed 2024-04-21T08:52:04.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:04.314+0000 7fa2232c0640 1 -- 172.21.15.86:0/1371408182 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa2080a3da0 msgr2=0x7fa2080a6260 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:04.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:04.314+0000 7fa2232c0640 1 --2- 172.21.15.86:0/1371408182 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa2080a3da0 0x7fa2080a6260 secure :-1 s=READY pgs=394 cs=0 l=1 rev1=1 crypto rx=0x7fa21c06b100 tx=0x7fa20c00b410 comp rx=0 tx=0).stop 2024-04-21T08:52:04.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:04.314+0000 7fa2232c0640 1 -- 172.21.15.86:0/1371408182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa21c1010a0 msgr2=0x7fa21c199580 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).mark_down 2024-04-21T08:52:04.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:04.314+0000 7fa2232c0640 1 --2- 172.21.15.86:0/1371408182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa21c1010a0 0x7fa21c199580 secure :-1 s=READY pgs=2178 cs=0 l=1 rev1=1 crypto rx=0x7fa21800d950 tx=0x7fa21800de10 comp rx=0 tx=0).stop 2024-04-21T08:52:04.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:04.314+0000 7fa2232c0640 1 -- 172.21.15.86:0/1371408182 shutdown_connections 2024-04-21T08:52:04.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:04.314+0000 7fa2232c0640 1 --2- 172.21.15.86:0/1371408182 >> [v2:172.21.15.86:6800/2452032403,v1:172.21.15.86:6801/2452032403] conn(0x7fa2080a3da0 0x7fa2080a6260 unknown :-1 s=CLOSED pgs=394 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:04.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:04.314+0000 7fa2232c0640 1 --2- 172.21.15.86:0/1371408182 >> [v2:172.21.15.88:3300/0,v1:172.21.15.88:6789/0] conn(0x7fa21c105dd0 0x7fa21c1a4600 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:04.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:04.314+0000 7fa2232c0640 1 --2- 172.21.15.86:0/1371408182 >> [v2:172.21.15.179:3300/0,v1:172.21.15.179:6789/0] conn(0x7fa21c101a40 0x7fa21c199ac0 unknown :-1 s=CLOSED pgs=0 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:04.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:04.314+0000 7fa2232c0640 1 --2- 172.21.15.86:0/1371408182 >> [v2:172.21.15.86:3300/0,v1:172.21.15.86:6789/0] conn(0x7fa21c1010a0 0x7fa21c199580 unknown :-1 s=CLOSED pgs=2178 cs=0 l=1 rev1=1 crypto rx=0 tx=0 comp rx=0 tx=0).stop 2024-04-21T08:52:04.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:04.314+0000 7fa2232c0640 1 -- 172.21.15.86:0/1371408182 >> 172.21.15.86:0/1371408182 conn(0x7fa21c077c40 msgr2=0x7fa21c0fee70 unknown :-1 s=STATE_NONE l=0).mark_down 2024-04-21T08:52:04.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:04.314+0000 7fa2232c0640 1 -- 172.21.15.86:0/1371408182 shutdown_connections 2024-04-21T08:52:04.314 INFO:teuthology.orchestra.run.smithi086.stderr:2024-04-21T08:52:04.315+0000 7fa2232c0640 1 -- 172.21.15.86:0/1371408182 wait complete. 2024-04-21T08:52:04.329 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-21T08:52:04.354 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-21T08:52:04.385 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-21T08:52:04.414 INFO:tasks.cephadm:Cleaning up testdir ceph.* files... 2024-04-21T08:52:04.415 DEBUG:teuthology.orchestra.run.smithi086:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-21T08:52:04.431 DEBUG:teuthology.orchestra.run.smithi088:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-21T08:52:04.447 DEBUG:teuthology.orchestra.run.smithi179:> rm -f /home/ubuntu/cephtest/seed.ceph.conf /home/ubuntu/cephtest/ceph.pub 2024-04-21T08:52:04.470 INFO:tasks.cephadm:Stopping all daemons... 2024-04-21T08:52:04.470 INFO:tasks.cephadm.mon.a:Stopping mon.a... 2024-04-21T08:52:04.471 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mon.a 2024-04-21T08:52:04.615 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:04 smithi088 ceph-mon[31259]: pgmap v417: 97 pgs: 97 active+clean; 577 KiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:52:04.616 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:04 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1377901705' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-21T08:52:04.616 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:04 smithi088 ceph-mon[31259]: osdmap e111: 12 total, 12 up, 12 in 2024-04-21T08:52:04.616 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:04 smithi088 ceph-mon[31259]: from='client.? 172.21.15.86:0/1371408182' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T08:52:04.616 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:04 smithi086 ceph-mon[27939]: pgmap v417: 97 pgs: 97 active+clean; 577 KiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:52:04.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:04 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1377901705' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-21T08:52:04.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:04 smithi086 ceph-mon[27939]: osdmap e111: 12 total, 12 up, 12 in 2024-04-21T08:52:04.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:04 smithi086 ceph-mon[27939]: from='client.? 172.21.15.86:0/1371408182' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T08:52:04.617 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:04 smithi086 systemd[1]: Stopping Ceph mon.a for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:52:04.746 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:04 smithi179 ceph-mon[31829]: pgmap v417: 97 pgs: 97 active+clean; 577 KiB data, 437 MiB used, 1.0 TiB / 1.0 TiB avail 2024-04-21T08:52:04.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:04 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1377901705' entity='client.admin' cmd='[{"prefix": "osd pool rm", "pool": "cephfs_data", "pool2": "cephfs_data", "yes_i_really_really_mean_it": true}]': finished 2024-04-21T08:52:04.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:04 smithi179 ceph-mon[31829]: osdmap e111: 12 total, 12 up, 12 in 2024-04-21T08:52:04.747 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:04 smithi179 ceph-mon[31829]: from='client.? 172.21.15.86:0/1371408182' entity='client.admin' cmd=[{"prefix": "osd pool rm", "pool": "cephfs_data_ec", "pool2": "cephfs_data_ec", "yes_i_really_really_mean_it": true}]: dispatch 2024-04-21T08:52:05.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:04 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mon-a[27935]: 2024-04-21T08:52:04.776+0000 7ff1bb734700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:52:05.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:04 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mon-a[27935]: 2024-04-21T08:52:04.776+0000 7ff1bb734700 -1 mon.a@0(leader) e3 *** Got Signal Terminated *** 2024-04-21T08:52:05.117 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:04 smithi086 podman[104137]: 2024-04-21 08:52:04.918903846 +0000 UTC m=+0.258746778 container died f70ad6da7b69cfc5da26ec15d10a6252045d3822626a62d03d5308d8f5a73090 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mon-a, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416) 2024-04-21T08:52:05.569 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:05 smithi086 podman[104137]: 2024-04-21 08:52:05.327080495 +0000 UTC m=+0.666923426 container cleanup f70ad6da7b69cfc5da26ec15d10a6252045d3822626a62d03d5308d8f5a73090 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mon-a, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:52:05.569 INFO:journalctl@ceph.mon.a.smithi086.stdout:Apr 21 08:52:05 smithi086 bash[104137]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mon-a 2024-04-21T08:52:05.691 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mon.a.service' 2024-04-21T08:52:05.738 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:52:05.738 INFO:tasks.cephadm.mon.a:Stopped mon.a 2024-04-21T08:52:05.738 INFO:tasks.cephadm.mon.c:Stopping mon.b... 2024-04-21T08:52:05.739 DEBUG:teuthology.orchestra.run.smithi088:> sudo systemctl stop ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mon.b 2024-04-21T08:52:06.047 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:05 smithi088 systemd[1]: Stopping Ceph mon.b for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:52:06.414 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:06 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mon-b[31255]: 2024-04-21T08:52:06.046+0000 7f20430af700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:52:06.414 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:06 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mon-b[31255]: 2024-04-21T08:52:06.046+0000 7f20430af700 -1 mon.b@2(peon) e3 *** Got Signal Terminated *** 2024-04-21T08:52:06.415 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:06 smithi088 podman[67458]: 2024-04-21 08:52:06.170742283 +0000 UTC m=+0.241137014 container died 2386b428da3ad25c981b7b09b63b5f6c3723d6ceb7033ddf1a08130eaa8016f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mon-b, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_CLEAN=True) 2024-04-21T08:52:06.707 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:06 smithi088 podman[67458]: 2024-04-21 08:52:06.498361222 +0000 UTC m=+0.568755942 container cleanup 2386b428da3ad25c981b7b09b63b5f6c3723d6ceb7033ddf1a08130eaa8016f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mon-b, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-21T08:52:06.707 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:06 smithi088 bash[67458]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mon-b 2024-04-21T08:52:06.915 DEBUG:teuthology.orchestra.run.smithi088:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mon.b.service' 2024-04-21T08:52:06.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:06 smithi088 podman[67471]: 2024-04-21 08:52:06.708192194 +0000 UTC m=+0.532555554 container remove 2386b428da3ad25c981b7b09b63b5f6c3723d6ceb7033ddf1a08130eaa8016f1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mon-b, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, ceph=True, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:52:06.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:06 smithi088 systemd[1]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mon.b.service: Deactivated successfully. 2024-04-21T08:52:06.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:06 smithi088 systemd[1]: Stopped Ceph mon.b for f8cec7cc-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:52:06.964 INFO:journalctl@ceph.mon.b.smithi088.stdout:Apr 21 08:52:06 smithi088 systemd[1]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mon.b.service: Consumed 13.558s CPU time. 2024-04-21T08:52:06.969 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:52:06.969 INFO:tasks.cephadm.mon.c:Stopped mon.b 2024-04-21T08:52:06.969 INFO:tasks.cephadm.mon.c:Stopping mon.c... 2024-04-21T08:52:06.970 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mon.c 2024-04-21T08:52:07.247 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:07 smithi179 systemd[1]: Stopping Ceph mon.c for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:52:07.692 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:07 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mon-c[31824]: 2024-04-21T08:52:07.266+0000 7f9f2eab2700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mon -n mon.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false --default-mon-cluster-log-to-file=false --default-mon-cluster-log-to-journald=true --default-mon-cluster-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:52:07.692 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:07 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mon-c[31824]: 2024-04-21T08:52:07.266+0000 7f9f2eab2700 -1 mon.c@1(peon) e3 *** Got Signal Terminated *** 2024-04-21T08:52:07.692 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:07 smithi179 podman[68387]: 2024-04-21 08:52:07.410541433 +0000 UTC m=+0.245078763 container died e2e289b587bcba803498048168715310bb280631cd7eb827a7877873c61eec2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mon-c, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True) 2024-04-21T08:52:07.996 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:07 smithi179 podman[68387]: 2024-04-21 08:52:07.784483215 +0000 UTC m=+0.619020536 container cleanup e2e289b587bcba803498048168715310bb280631cd7eb827a7877873c61eec2f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mon-c, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:52:07.997 INFO:journalctl@ceph.mon.c.smithi179.stdout:Apr 21 08:52:07 smithi179 bash[68387]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mon-c 2024-04-21T08:52:08.217 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mon.c.service' 2024-04-21T08:52:08.266 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:52:08.266 INFO:tasks.cephadm.mon.c:Stopped mon.c 2024-04-21T08:52:08.266 INFO:tasks.cephadm.mgr.x:Stopping mgr.x... 2024-04-21T08:52:08.266 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mgr.x 2024-04-21T08:52:08.574 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:52:08 smithi086 systemd[1]: Stopping Ceph mgr.x for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:52:08.867 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:52:08 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x[28169]: 2024-04-21T08:52:08.573+0000 7fafef82d700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mgr -n mgr.x -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:52:08.867 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:52:08 smithi086 podman[104357]: 2024-04-21 08:52:08.604704051 +0000 UTC m=+0.143374746 container died bbba55e9c9c1b85cd1aec736267dea8abbc1129d1a742d308c5dbd82f4799707 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, ceph=True, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS) 2024-04-21T08:52:09.267 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:52:09 smithi086 podman[104357]: 2024-04-21 08:52:09.020911191 +0000 UTC m=+0.559581916 container cleanup bbba55e9c9c1b85cd1aec736267dea8abbc1129d1a742d308c5dbd82f4799707 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph:a630765c1f93c6e61d9b40c4b76437611d5ef29e, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:52:09.267 INFO:journalctl@ceph.mgr.x.smithi086.stdout:Apr 21 08:52:09 smithi086 bash[104357]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-x 2024-04-21T08:52:09.435 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mgr.x.service' 2024-04-21T08:52:09.470 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:52:09.470 INFO:tasks.cephadm.mgr.x:Stopped mgr.x 2024-04-21T08:52:09.471 INFO:tasks.cephadm.mgr.y:Stopping mgr.y... 2024-04-21T08:52:09.471 DEBUG:teuthology.orchestra.run.smithi088:> sudo systemctl stop ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mgr.y 2024-04-21T08:52:09.797 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:52:09 smithi088 systemd[1]: Stopping Ceph mgr.y for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:52:10.086 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:52:09 smithi088 podman[67582]: 2024-04-21 08:52:09.798227555 +0000 UTC m=+0.133811253 container died d163c7a7cdb535423088004942a4c61a4181eb9d02ce0f121e96091379305ad0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416) 2024-04-21T08:52:10.370 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:52:10 smithi088 podman[67582]: 2024-04-21 08:52:10.186347984 +0000 UTC m=+0.521931685 container cleanup d163c7a7cdb535423088004942a4c61a4181eb9d02ce0f121e96091379305ad0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T08:52:10.371 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:52:10 smithi088 bash[67582]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y 2024-04-21T08:52:10.609 DEBUG:teuthology.orchestra.run.smithi088:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mgr.y.service' 2024-04-21T08:52:10.630 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:52:10 smithi088 podman[67596]: 2024-04-21 08:52:10.371482643 +0000 UTC m=+0.569093815 container remove d163c7a7cdb535423088004942a4c61a4181eb9d02ce0f121e96091379305ad0 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-y, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=) 2024-04-21T08:52:10.630 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:52:10 smithi088 systemd[1]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mgr.y.service: Main process exited, code=exited, status=143/n/a 2024-04-21T08:52:10.630 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:52:10 smithi088 systemd[1]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mgr.y.service: Failed with result 'exit-code'. 2024-04-21T08:52:10.630 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:52:10 smithi088 systemd[1]: Stopped Ceph mgr.y for f8cec7cc-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:52:10.630 INFO:journalctl@ceph.mgr.y.smithi088.stdout:Apr 21 08:52:10 smithi088 systemd[1]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mgr.y.service: Consumed 20.859s CPU time. 2024-04-21T08:52:10.642 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:52:10.642 INFO:tasks.cephadm.mgr.y:Stopped mgr.y 2024-04-21T08:52:10.642 INFO:tasks.cephadm.mgr.z:Stopping mgr.z... 2024-04-21T08:52:10.642 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mgr.z 2024-04-21T08:52:10.977 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:52:10 smithi179 systemd[1]: Stopping Ceph mgr.z for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:52:11.246 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:52:10 smithi179 podman[68510]: 2024-04-21 08:52:10.977964194 +0000 UTC m=+0.146735307 container died b71731ad80c807de8d3724df59b92561f5c6ce317d6ad3ebd51923bb1569ee5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z, GIT_BRANCH=HEAD, GIT_CLEAN=True, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS) 2024-04-21T08:52:11.650 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:52:11 smithi179 podman[68510]: 2024-04-21 08:52:11.357856386 +0000 UTC m=+0.526627496 container cleanup b71731ad80c807de8d3724df59b92561f5c6ce317d6ad3ebd51923bb1569ee5e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0) 2024-04-21T08:52:11.650 INFO:journalctl@ceph.mgr.z.smithi179.stdout:Apr 21 08:52:11 smithi179 bash[68510]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mgr-z 2024-04-21T08:52:11.780 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mgr.z.service' 2024-04-21T08:52:11.812 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:52:11.813 INFO:tasks.cephadm.mgr.z:Stopped mgr.z 2024-04-21T08:52:11.813 INFO:tasks.cephadm.osd.0:Stopping osd.0... 2024-04-21T08:52:11.813 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@osd.0 2024-04-21T08:52:12.117 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 21 08:52:11 smithi086 systemd[1]: Stopping Ceph osd.0 for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:52:12.617 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 21 08:52:12 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-0[40513]: 2024-04-21T08:52:12.228+0000 7f99d442a700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.0 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:52:12.617 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 21 08:52:12 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-0[40513]: 2024-04-21T08:52:12.228+0000 7f99d442a700 -1 osd.0 112 *** Got signal Terminated *** 2024-04-21T08:52:12.617 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 21 08:52:12 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-0[40513]: 2024-04-21T08:52:12.228+0000 7f99d442a700 -1 osd.0 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T08:52:17.867 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 21 08:52:17 smithi086 podman[104479]: 2024-04-21 08:52:17.606537744 +0000 UTC m=+5.596762894 container died 6dbe1fac2a654a88bcde5b73d2ae3dcea05a4992215d89f9dd166219acbbabcd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-0, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T08:52:18.271 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 21 08:52:17 smithi086 podman[104479]: 2024-04-21 08:52:17.971366809 +0000 UTC m=+5.961591967 container cleanup 6dbe1fac2a654a88bcde5b73d2ae3dcea05a4992215d89f9dd166219acbbabcd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-0, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-21T08:52:18.272 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 21 08:52:17 smithi086 bash[104479]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-0 2024-04-21T08:52:18.617 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 21 08:52:18 smithi086 podman[104496]: 2024-04-21 08:52:18.272758623 +0000 UTC m=+0.656385829 container remove 6dbe1fac2a654a88bcde5b73d2ae3dcea05a4992215d89f9dd166219acbbabcd (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-0, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-21T08:52:18.968 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 21 08:52:18 smithi086 podman[104586]: 2024-04-21 08:52:18.5365246 +0000 UTC m=+0.017968680 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:52:18.968 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 21 08:52:18 smithi086 podman[104586]: 2024-04-21 08:52:18.660431004 +0000 UTC m=+0.141875104 container create 67417f0ed6a12a9b29ed9967bdf43bc24b1914b412798151dfa78485a8e7a5f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-0-deactivate, GIT_CLEAN=True, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=) 2024-04-21T08:52:18.968 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 21 08:52:18 smithi086 podman[104586]: 2024-04-21 08:52:18.968366908 +0000 UTC m=+0.449810965 container init 67417f0ed6a12a9b29ed9967bdf43bc24b1914b412798151dfa78485a8e7a5f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-0-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-21T08:52:19.261 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 21 08:52:18 smithi086 podman[104586]: 2024-04-21 08:52:18.971619311 +0000 UTC m=+0.453063377 container start 67417f0ed6a12a9b29ed9967bdf43bc24b1914b412798151dfa78485a8e7a5f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-0-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS) 2024-04-21T08:52:19.261 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 21 08:52:19 smithi086 podman[104586]: 2024-04-21 08:52:19.076736254 +0000 UTC m=+0.558180324 container attach 67417f0ed6a12a9b29ed9967bdf43bc24b1914b412798151dfa78485a8e7a5f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-0-deactivate, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-21T08:52:19.617 INFO:journalctl@ceph.osd.0.smithi086.stdout:Apr 21 08:52:19 smithi086 podman[104586]: 2024-04-21 08:52:19.262731301 +0000 UTC m=+0.744175370 container died 67417f0ed6a12a9b29ed9967bdf43bc24b1914b412798151dfa78485a8e7a5f7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-0-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS) 2024-04-21T08:52:19.978 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@osd.0.service' 2024-04-21T08:52:20.017 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:52:20.017 INFO:tasks.cephadm.osd.0:Stopped osd.0 2024-04-21T08:52:20.017 INFO:tasks.cephadm.osd.1:Stopping osd.1... 2024-04-21T08:52:20.017 DEBUG:teuthology.orchestra.run.smithi088:> sudo systemctl stop ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@osd.1 2024-04-21T08:52:20.323 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 21 08:52:20 smithi088 systemd[1]: Stopping Ceph osd.1 for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:52:20.714 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 21 08:52:20 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-1[35842]: 2024-04-21T08:52:20.323+0000 7fe4d2dc0700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.1 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:52:20.714 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 21 08:52:20 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-1[35842]: 2024-04-21T08:52:20.323+0000 7fe4d2dc0700 -1 osd.1 112 *** Got signal Terminated *** 2024-04-21T08:52:20.714 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 21 08:52:20 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-1[35842]: 2024-04-21T08:52:20.323+0000 7fe4d2dc0700 -1 osd.1 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T08:52:26.008 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 21 08:52:25 smithi088 podman[67707]: 2024-04-21 08:52:25.732677513 +0000 UTC m=+5.515939283 container died f8ea0f5e9ef02b48bcdf73d4ebb1fb58069ed509d12500654c29e5792457f712 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-1, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True) 2024-04-21T08:52:26.415 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 21 08:52:26 smithi088 podman[67707]: 2024-04-21 08:52:26.090281307 +0000 UTC m=+5.873543067 container cleanup f8ea0f5e9ef02b48bcdf73d4ebb1fb58069ed509d12500654c29e5792457f712 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-1, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-21T08:52:26.416 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 21 08:52:26 smithi088 bash[67707]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-1 2024-04-21T08:52:26.714 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 21 08:52:26 smithi088 podman[67724]: 2024-04-21 08:52:26.416852417 +0000 UTC m=+0.672156847 container remove f8ea0f5e9ef02b48bcdf73d4ebb1fb58069ed509d12500654c29e5792457f712 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-1, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-21T08:52:27.020 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 21 08:52:26 smithi088 podman[67814]: 2024-04-21 08:52:26.633263636 +0000 UTC m=+0.017297264 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:52:27.020 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 21 08:52:26 smithi088 podman[67814]: 2024-04-21 08:52:26.767830838 +0000 UTC m=+0.151864456 container create c4216c46ab68643ba29c95a83478b2530e62bde02e4de1d090ecbe8697d59e48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-1-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, ceph=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-21T08:52:27.317 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 21 08:52:27 smithi088 podman[67814]: 2024-04-21 08:52:27.020852549 +0000 UTC m=+0.404886167 container init c4216c46ab68643ba29c95a83478b2530e62bde02e4de1d090ecbe8697d59e48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-1-deactivate, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, ceph=True, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:52:27.317 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 21 08:52:27 smithi088 podman[67814]: 2024-04-21 08:52:27.024108508 +0000 UTC m=+0.408142128 container start c4216c46ab68643ba29c95a83478b2530e62bde02e4de1d090ecbe8697d59e48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-1-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:52:27.317 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 21 08:52:27 smithi088 podman[67814]: 2024-04-21 08:52:27.095810598 +0000 UTC m=+0.479844227 container attach c4216c46ab68643ba29c95a83478b2530e62bde02e4de1d090ecbe8697d59e48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-1-deactivate, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:52:27.673 INFO:journalctl@ceph.osd.1.smithi088.stdout:Apr 21 08:52:27 smithi088 podman[67814]: 2024-04-21 08:52:27.3180468 +0000 UTC m=+0.702080422 container died c4216c46ab68643ba29c95a83478b2530e62bde02e4de1d090ecbe8697d59e48 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-1-deactivate, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, ceph=True, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS) 2024-04-21T08:52:27.956 DEBUG:teuthology.orchestra.run.smithi088:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@osd.1.service' 2024-04-21T08:52:27.998 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:52:27.998 INFO:tasks.cephadm.osd.1:Stopped osd.1 2024-04-21T08:52:27.998 INFO:tasks.cephadm.osd.2:Stopping osd.2... 2024-04-21T08:52:27.998 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@osd.2 2024-04-21T08:52:28.312 INFO:journalctl@ceph.osd.2.smithi179.stdout:Apr 21 08:52:28 smithi179 systemd[1]: Stopping Ceph osd.2 for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:52:28.746 INFO:journalctl@ceph.osd.2.smithi179.stdout:Apr 21 08:52:28 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-2[36968]: 2024-04-21T08:52:28.311+0000 7f776c8cd700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.2 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:52:28.747 INFO:journalctl@ceph.osd.2.smithi179.stdout:Apr 21 08:52:28 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-2[36968]: 2024-04-21T08:52:28.311+0000 7f776c8cd700 -1 osd.2 112 *** Got signal Terminated *** 2024-04-21T08:52:28.747 INFO:journalctl@ceph.osd.2.smithi179.stdout:Apr 21 08:52:28 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-2[36968]: 2024-04-21T08:52:28.311+0000 7f776c8cd700 -1 osd.2 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T08:52:33.996 INFO:journalctl@ceph.osd.2.smithi179.stdout:Apr 21 08:52:33 smithi179 podman[68636]: 2024-04-21 08:52:33.739435196 +0000 UTC m=+5.549580902 container died fae7dd6aa9f7efb26ed6216fe4ae8f78e7da8f809a06120b5ab38090d95086a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-2, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:52:34.379 INFO:journalctl@ceph.osd.2.smithi179.stdout:Apr 21 08:52:34 smithi179 podman[68636]: 2024-04-21 08:52:34.121298072 +0000 UTC m=+5.931443767 container cleanup fae7dd6aa9f7efb26ed6216fe4ae8f78e7da8f809a06120b5ab38090d95086a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-2, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:52:34.380 INFO:journalctl@ceph.osd.2.smithi179.stdout:Apr 21 08:52:34 smithi179 bash[68636]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-2 2024-04-21T08:52:34.735 INFO:journalctl@ceph.osd.2.smithi179.stdout:Apr 21 08:52:34 smithi179 podman[68653]: 2024-04-21 08:52:34.380977128 +0000 UTC m=+0.635685898 container remove fae7dd6aa9f7efb26ed6216fe4ae8f78e7da8f809a06120b5ab38090d95086a6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-2, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416) 2024-04-21T08:52:34.997 INFO:journalctl@ceph.osd.2.smithi179.stdout:Apr 21 08:52:34 smithi179 podman[68741]: 2024-04-21 08:52:34.639642438 +0000 UTC m=+0.017897924 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:52:34.997 INFO:journalctl@ceph.osd.2.smithi179.stdout:Apr 21 08:52:34 smithi179 podman[68741]: 2024-04-21 08:52:34.767641241 +0000 UTC m=+0.145896704 container create 35c51237afde53176ffb2f745514b6aaf8c07bdd703aa6c89b58bdab0c40e81e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-2-deactivate, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2) 2024-04-21T08:52:35.314 INFO:journalctl@ceph.osd.2.smithi179.stdout:Apr 21 08:52:35 smithi179 podman[68741]: 2024-04-21 08:52:35.020661038 +0000 UTC m=+0.398916505 container init 35c51237afde53176ffb2f745514b6aaf8c07bdd703aa6c89b58bdab0c40e81e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-2-deactivate, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-21T08:52:35.314 INFO:journalctl@ceph.osd.2.smithi179.stdout:Apr 21 08:52:35 smithi179 podman[68741]: 2024-04-21 08:52:35.024118109 +0000 UTC m=+0.402373572 container start 35c51237afde53176ffb2f745514b6aaf8c07bdd703aa6c89b58bdab0c40e81e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-2-deactivate, io.buildah.version=1.33.5, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T08:52:35.314 INFO:journalctl@ceph.osd.2.smithi179.stdout:Apr 21 08:52:35 smithi179 podman[68741]: 2024-04-21 08:52:35.112493223 +0000 UTC m=+0.490748689 container attach 35c51237afde53176ffb2f745514b6aaf8c07bdd703aa6c89b58bdab0c40e81e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-2-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-21T08:52:35.585 INFO:journalctl@ceph.osd.2.smithi179.stdout:Apr 21 08:52:35 smithi179 podman[68741]: 2024-04-21 08:52:35.315537247 +0000 UTC m=+0.693792727 container died 35c51237afde53176ffb2f745514b6aaf8c07bdd703aa6c89b58bdab0c40e81e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-2-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:52:35.956 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@osd.2.service' 2024-04-21T08:52:35.993 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:52:35.993 INFO:tasks.cephadm.osd.2:Stopped osd.2 2024-04-21T08:52:35.993 INFO:tasks.cephadm.osd.3:Stopping osd.3... 2024-04-21T08:52:35.993 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@osd.3 2024-04-21T08:52:36.307 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 21 08:52:36 smithi086 systemd[1]: Stopping Ceph osd.3 for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:52:36.566 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 21 08:52:36 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-3[47346]: 2024-04-21T08:52:36.307+0000 7f1ada112700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.3 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:52:36.566 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 21 08:52:36 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-3[47346]: 2024-04-21T08:52:36.307+0000 7f1ada112700 -1 osd.3 112 *** Got signal Terminated *** 2024-04-21T08:52:36.566 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 21 08:52:36 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-3[47346]: 2024-04-21T08:52:36.307+0000 7f1ada112700 -1 osd.3 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T08:52:38.117 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 21 08:52:37 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-3[47346]: 2024-04-21T08:52:37.792+0000 7f1ad6706700 -1 osd.3 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:11.984103+0000 front 2024-04-21T08:52:11.984064+0000 (oldest deadline 2024-04-21T08:52:37.784790+0000) 2024-04-21T08:52:39.117 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 21 08:52:38 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-3[47346]: 2024-04-21T08:52:38.782+0000 7f1ad6706700 -1 osd.3 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:11.984103+0000 front 2024-04-21T08:52:11.984064+0000 (oldest deadline 2024-04-21T08:52:37.784790+0000) 2024-04-21T08:52:39.117 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:38 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:38.659+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:52:39.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:52:39 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:52:39.425+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:52:39.867 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 21 08:52:39 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-3[47346]: 2024-04-21T08:52:39.832+0000 7f1ad6706700 -1 osd.3 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:11.984103+0000 front 2024-04-21T08:52:11.984064+0000 (oldest deadline 2024-04-21T08:52:37.784790+0000) 2024-04-21T08:52:39.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:39 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:39.611+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:52:40.746 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:52:40 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:52:40.392+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:52:40.867 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 21 08:52:40 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-3[47346]: 2024-04-21T08:52:40.792+0000 7f1ad6706700 -1 osd.3 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:11.984103+0000 front 2024-04-21T08:52:11.984064+0000 (oldest deadline 2024-04-21T08:52:37.784790+0000) 2024-04-21T08:52:40.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:40 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:40.608+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:52:41.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:52:41 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:52:41.310+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:52:41.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:52:41 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:52:41.356+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:52:41.844 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:41 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:41.581+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:52:41.845 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:52:41 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:52:41.734+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:16.244986+0000 front 2024-04-21T08:52:16.245244+0000 (oldest deadline 2024-04-21T08:52:40.945040+0000) 2024-04-21T08:52:42.117 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 21 08:52:41 smithi086 podman[104693]: 2024-04-21 08:52:41.845427167 +0000 UTC m=+5.657364484 container died ecb378610ce2f4a872da8bedc1bf9c5cb35bf983f9f8b0bf953e763b504ffdf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-3, ceph=True, GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0) 2024-04-21T08:52:42.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:52:41 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:52:41.787+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:14.929128+0000 front 2024-04-21T08:52:14.929188+0000 (oldest deadline 2024-04-21T08:52:40.829047+0000) 2024-04-21T08:52:42.508 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 21 08:52:42 smithi086 podman[104693]: 2024-04-21 08:52:42.241472421 +0000 UTC m=+6.053409738 container cleanup ecb378610ce2f4a872da8bedc1bf9c5cb35bf983f9f8b0bf953e763b504ffdf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-3, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:52:42.509 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 21 08:52:42 smithi086 bash[104693]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-3 2024-04-21T08:52:42.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:52:42 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:52:42.325+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:52:42.746 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:52:42 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:52:42.359+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:52:42.747 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:52:42 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-5[43475]: 2024-04-21T08:52:42.561+0000 7f8db825a700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:16.830259+0000 front 2024-04-21T08:52:16.830124+0000 (oldest deadline 2024-04-21T08:52:42.130317+0000) 2024-04-21T08:52:42.817 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 21 08:52:42 smithi086 podman[104722]: 2024-04-21 08:52:42.509583075 +0000 UTC m=+0.657112301 container remove ecb378610ce2f4a872da8bedc1bf9c5cb35bf983f9f8b0bf953e763b504ffdf6 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-3, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True) 2024-04-21T08:52:42.817 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:52:42 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:52:42.720+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:16.244986+0000 front 2024-04-21T08:52:16.245244+0000 (oldest deadline 2024-04-21T08:52:40.945040+0000) 2024-04-21T08:52:42.817 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:42 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:42.569+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:52:43.104 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 21 08:52:42 smithi086 podman[104812]: 2024-04-21 08:52:42.721031829 +0000 UTC m=+0.018599162 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:52:43.104 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 21 08:52:42 smithi086 podman[104812]: 2024-04-21 08:52:42.852299993 +0000 UTC m=+0.149867327 container create 6815dd7b1c00bc53e3855fedd084de0b301139e62972f30540f9e0a275dae3b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-21T08:52:43.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:52:42 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:52:42.801+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:14.929128+0000 front 2024-04-21T08:52:14.929188+0000 (oldest deadline 2024-04-21T08:52:40.829047+0000) 2024-04-21T08:52:43.367 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 21 08:52:43 smithi086 podman[104812]: 2024-04-21 08:52:43.10509416 +0000 UTC m=+0.402661497 container init 6815dd7b1c00bc53e3855fedd084de0b301139e62972f30540f9e0a275dae3b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-3-deactivate, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux ) 2024-04-21T08:52:43.367 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 21 08:52:43 smithi086 podman[104812]: 2024-04-21 08:52:43.10912934 +0000 UTC m=+0.406696675 container start 6815dd7b1c00bc53e3855fedd084de0b301139e62972f30540f9e0a275dae3b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-3-deactivate, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-21T08:52:43.367 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 21 08:52:43 smithi086 podman[104812]: 2024-04-21 08:52:43.196414652 +0000 UTC m=+0.493981987 container attach 6815dd7b1c00bc53e3855fedd084de0b301139e62972f30540f9e0a275dae3b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-3-deactivate, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0) 2024-04-21T08:52:43.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:52:43 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:52:43.301+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:52:43.736 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:43 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:43.533+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:52:43.737 INFO:journalctl@ceph.osd.3.smithi086.stdout:Apr 21 08:52:43 smithi086 podman[104812]: 2024-04-21 08:52:43.40215267 +0000 UTC m=+0.699720005 container died 6815dd7b1c00bc53e3855fedd084de0b301139e62972f30540f9e0a275dae3b2 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-3-deactivate, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5) 2024-04-21T08:52:43.746 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:52:43 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-5[43475]: 2024-04-21T08:52:43.562+0000 7f8db825a700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:16.830259+0000 front 2024-04-21T08:52:16.830124+0000 (oldest deadline 2024-04-21T08:52:42.130317+0000) 2024-04-21T08:52:43.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:52:43 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:52:43.317+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:52:44.041 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:52:43 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:52:43.737+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:16.244986+0000 front 2024-04-21T08:52:16.245244+0000 (oldest deadline 2024-04-21T08:52:40.945040+0000) 2024-04-21T08:52:44.070 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@osd.3.service' 2024-04-21T08:52:44.112 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:52:44.112 INFO:tasks.cephadm.osd.3:Stopped osd.3 2024-04-21T08:52:44.112 INFO:tasks.cephadm.osd.4:Stopping osd.4... 2024-04-21T08:52:44.112 DEBUG:teuthology.orchestra.run.smithi088:> sudo systemctl stop ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@osd.4 2024-04-21T08:52:44.246 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:52:43 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:52:43.772+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:14.929128+0000 front 2024-04-21T08:52:14.929188+0000 (oldest deadline 2024-04-21T08:52:40.829047+0000) 2024-04-21T08:52:44.408 INFO:journalctl@ceph.osd.4.smithi088.stdout:Apr 21 08:52:44 smithi088 systemd[1]: Stopping Ceph osd.4 for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:52:44.409 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:52:44 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:52:44.347+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:52:44.714 INFO:journalctl@ceph.osd.4.smithi088.stdout:Apr 21 08:52:44 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-4[42528]: 2024-04-21T08:52:44.408+0000 7f608e225700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.4 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:52:44.714 INFO:journalctl@ceph.osd.4.smithi088.stdout:Apr 21 08:52:44 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-4[42528]: 2024-04-21T08:52:44.408+0000 7f608e225700 -1 osd.4 112 *** Got signal Terminated *** 2024-04-21T08:52:44.714 INFO:journalctl@ceph.osd.4.smithi088.stdout:Apr 21 08:52:44 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-4[42528]: 2024-04-21T08:52:44.408+0000 7f608e225700 -1 osd.4 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T08:52:44.746 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:52:44 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:52:44.347+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:52:44.747 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:52:44 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-5[43475]: 2024-04-21T08:52:44.539+0000 7f8db825a700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:16.830259+0000 front 2024-04-21T08:52:16.830124+0000 (oldest deadline 2024-04-21T08:52:42.130317+0000) 2024-04-21T08:52:44.867 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:52:44 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:52:44.733+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:16.244986+0000 front 2024-04-21T08:52:16.245244+0000 (oldest deadline 2024-04-21T08:52:40.945040+0000) 2024-04-21T08:52:44.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:44 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:44.495+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:52:45.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:52:44 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:52:44.784+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:14.929128+0000 front 2024-04-21T08:52:14.929188+0000 (oldest deadline 2024-04-21T08:52:40.829047+0000) 2024-04-21T08:52:45.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:52:45 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:52:45.364+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:52:45.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:52:45 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:52:45.308+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:52:45.747 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:52:45 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-5[43475]: 2024-04-21T08:52:45.515+0000 7f8db825a700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:16.830259+0000 front 2024-04-21T08:52:16.830124+0000 (oldest deadline 2024-04-21T08:52:42.130317+0000) 2024-04-21T08:52:45.867 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:52:45 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:52:45.745+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:16.244986+0000 front 2024-04-21T08:52:16.245244+0000 (oldest deadline 2024-04-21T08:52:40.945040+0000) 2024-04-21T08:52:45.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:45 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:45.528+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:52:46.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:52:45 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:52:45.799+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:14.929128+0000 front 2024-04-21T08:52:14.929188+0000 (oldest deadline 2024-04-21T08:52:40.829047+0000) 2024-04-21T08:52:46.464 INFO:journalctl@ceph.osd.4.smithi088.stdout:Apr 21 08:52:46 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-4[42528]: 2024-04-21T08:52:46.173+0000 7f6086811700 -1 osd.4 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.743733+0000 front 2024-04-21T08:52:23.743949+0000 (oldest deadline 2024-04-21T08:52:46.044030+0000) 2024-04-21T08:52:46.464 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:52:46 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:52:46.370+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:52:46.534 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:52:46 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:52:46.272+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:52:46.809 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:52:46 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-5[43475]: 2024-04-21T08:52:46.534+0000 7f8db825a700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:16.830259+0000 front 2024-04-21T08:52:16.830124+0000 (oldest deadline 2024-04-21T08:52:42.130317+0000) 2024-04-21T08:52:46.867 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:52:46 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:52:46.761+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:16.244986+0000 front 2024-04-21T08:52:16.245244+0000 (oldest deadline 2024-04-21T08:52:40.945040+0000) 2024-04-21T08:52:46.867 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:52:46 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:52:46.761+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:25.046260+0000 front 2024-04-21T08:52:25.046169+0000 (oldest deadline 2024-04-21T08:52:46.746043+0000) 2024-04-21T08:52:46.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:46 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:46.576+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:52:46.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:46 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:46.576+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:52:47.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:52:46 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:52:46.809+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:14.929128+0000 front 2024-04-21T08:52:14.929188+0000 (oldest deadline 2024-04-21T08:52:40.829047+0000) 2024-04-21T08:52:47.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:52:46 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:52:46.809+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.329978+0000 front 2024-04-21T08:52:24.329957+0000 (oldest deadline 2024-04-21T08:52:46.029966+0000) 2024-04-21T08:52:47.464 INFO:journalctl@ceph.osd.4.smithi088.stdout:Apr 21 08:52:47 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-4[42528]: 2024-04-21T08:52:47.207+0000 7f6086811700 -1 osd.4 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.743733+0000 front 2024-04-21T08:52:23.743949+0000 (oldest deadline 2024-04-21T08:52:46.044030+0000) 2024-04-21T08:52:47.464 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:52:47 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:52:47.399+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:52:47.464 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:52:47 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:52:47.399+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:52:47.561 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:52:47 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:52:47.310+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:52:47.849 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:52:47 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-5[43475]: 2024-04-21T08:52:47.561+0000 7f8db825a700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:16.830259+0000 front 2024-04-21T08:52:16.830124+0000 (oldest deadline 2024-04-21T08:52:42.130317+0000) 2024-04-21T08:52:47.867 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:52:47 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:52:47.719+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:16.244986+0000 front 2024-04-21T08:52:16.245244+0000 (oldest deadline 2024-04-21T08:52:40.945040+0000) 2024-04-21T08:52:47.867 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:52:47 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:52:47.719+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:25.046260+0000 front 2024-04-21T08:52:25.046169+0000 (oldest deadline 2024-04-21T08:52:46.746043+0000) 2024-04-21T08:52:47.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:47 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:47.550+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:52:47.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:47 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:47.550+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:52:48.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:52:47 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:52:47.850+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:14.929128+0000 front 2024-04-21T08:52:14.929188+0000 (oldest deadline 2024-04-21T08:52:40.829047+0000) 2024-04-21T08:52:48.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:52:47 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:52:47.850+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.329978+0000 front 2024-04-21T08:52:24.329957+0000 (oldest deadline 2024-04-21T08:52:46.029966+0000) 2024-04-21T08:52:48.714 INFO:journalctl@ceph.osd.4.smithi088.stdout:Apr 21 08:52:48 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-4[42528]: 2024-04-21T08:52:48.252+0000 7f6086811700 -1 osd.4 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.743733+0000 front 2024-04-21T08:52:23.743949+0000 (oldest deadline 2024-04-21T08:52:46.044030+0000) 2024-04-21T08:52:48.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:52:48 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:52:48.387+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:52:48.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:52:48 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:52:48.387+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:52:48.746 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:52:48 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:52:48.313+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:52:48.746 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:52:48 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:52:48.313+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:52:48.747 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:52:48 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-5[43475]: 2024-04-21T08:52:48.544+0000 7f8db825a700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:16.830259+0000 front 2024-04-21T08:52:16.830124+0000 (oldest deadline 2024-04-21T08:52:42.130317+0000) 2024-04-21T08:52:48.747 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:52:48 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-5[43475]: 2024-04-21T08:52:48.544+0000 7f8db825a700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.431373+0000 front 2024-04-21T08:52:24.431301+0000 (oldest deadline 2024-04-21T08:52:48.531377+0000) 2024-04-21T08:52:48.867 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:52:48 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:52:48.765+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:16.244986+0000 front 2024-04-21T08:52:16.245244+0000 (oldest deadline 2024-04-21T08:52:40.945040+0000) 2024-04-21T08:52:48.867 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:52:48 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:52:48.765+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:25.046260+0000 front 2024-04-21T08:52:25.046169+0000 (oldest deadline 2024-04-21T08:52:46.746043+0000) 2024-04-21T08:52:48.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:48 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:48.566+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:52:48.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:48 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:48.566+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:52:49.246 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:52:48 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:52:48.861+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:14.929128+0000 front 2024-04-21T08:52:14.929188+0000 (oldest deadline 2024-04-21T08:52:40.829047+0000) 2024-04-21T08:52:49.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:52:48 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:52:48.861+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.329978+0000 front 2024-04-21T08:52:24.329957+0000 (oldest deadline 2024-04-21T08:52:46.029966+0000) 2024-04-21T08:52:49.714 INFO:journalctl@ceph.osd.4.smithi088.stdout:Apr 21 08:52:49 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-4[42528]: 2024-04-21T08:52:49.262+0000 7f6086811700 -1 osd.4 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.743733+0000 front 2024-04-21T08:52:23.743949+0000 (oldest deadline 2024-04-21T08:52:46.044030+0000) 2024-04-21T08:52:49.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:52:49 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:52:49.406+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:52:49.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:52:49 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:52:49.406+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:52:49.746 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:52:49 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:52:49.288+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:52:49.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:52:49 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:52:49.288+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:52:49.747 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:52:49 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-5[43475]: 2024-04-21T08:52:49.507+0000 7f8db825a700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:16.830259+0000 front 2024-04-21T08:52:16.830124+0000 (oldest deadline 2024-04-21T08:52:42.130317+0000) 2024-04-21T08:52:49.747 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:52:49 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-5[43475]: 2024-04-21T08:52:49.507+0000 7f8db825a700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.431373+0000 front 2024-04-21T08:52:24.431301+0000 (oldest deadline 2024-04-21T08:52:48.531377+0000) 2024-04-21T08:52:49.867 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:52:49 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:52:49.793+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:16.244986+0000 front 2024-04-21T08:52:16.245244+0000 (oldest deadline 2024-04-21T08:52:40.945040+0000) 2024-04-21T08:52:49.867 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:52:49 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:52:49.793+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:25.046260+0000 front 2024-04-21T08:52:25.046169+0000 (oldest deadline 2024-04-21T08:52:46.746043+0000) 2024-04-21T08:52:49.869 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:49 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:49.586+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:52:49.870 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:49 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:49.586+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:52:49.984 INFO:journalctl@ceph.osd.4.smithi088.stdout:Apr 21 08:52:49 smithi088 podman[67917]: 2024-04-21 08:52:49.715882034 +0000 UTC m=+5.410206202 container died f41c09f675bf090b7e90cd08ea1762d0be2f3ef4b53ac6b0df2c61621160f383 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-4, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-21T08:52:50.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:52:49 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:52:49.831+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:14.929128+0000 front 2024-04-21T08:52:14.929188+0000 (oldest deadline 2024-04-21T08:52:40.829047+0000) 2024-04-21T08:52:50.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:52:49 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:52:49.831+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.329978+0000 front 2024-04-21T08:52:24.329957+0000 (oldest deadline 2024-04-21T08:52:46.029966+0000) 2024-04-21T08:52:50.367 INFO:journalctl@ceph.osd.4.smithi088.stdout:Apr 21 08:52:50 smithi088 podman[67917]: 2024-04-21 08:52:50.075620876 +0000 UTC m=+5.769945038 container cleanup f41c09f675bf090b7e90cd08ea1762d0be2f3ef4b53ac6b0df2c61621160f383 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-4, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux ) 2024-04-21T08:52:50.368 INFO:journalctl@ceph.osd.4.smithi088.stdout:Apr 21 08:52:50 smithi088 bash[67917]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-4 2024-04-21T08:52:50.685 INFO:journalctl@ceph.osd.4.smithi088.stdout:Apr 21 08:52:50 smithi088 podman[67934]: 2024-04-21 08:52:50.368550211 +0000 UTC m=+0.652886627 container remove f41c09f675bf090b7e90cd08ea1762d0be2f3ef4b53ac6b0df2c61621160f383 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-4, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_CLEAN=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T08:52:50.686 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:52:50 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:52:50.432+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.388905+0000 front 2024-04-21T08:52:24.388964+0000 (oldest deadline 2024-04-21T08:52:49.689097+0000) 2024-04-21T08:52:50.686 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:52:50 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:52:50.369+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:52:50.686 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:52:50 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:52:50.369+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:52:50.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:52:50 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:52:50.310+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:52:50.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:52:50 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:52:50.310+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:52:50.747 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:52:50 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-5[43475]: 2024-04-21T08:52:50.514+0000 7f8db825a700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:16.830259+0000 front 2024-04-21T08:52:16.830124+0000 (oldest deadline 2024-04-21T08:52:42.130317+0000) 2024-04-21T08:52:50.747 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:52:50 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-5[43475]: 2024-04-21T08:52:50.514+0000 7f8db825a700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.431373+0000 front 2024-04-21T08:52:24.431301+0000 (oldest deadline 2024-04-21T08:52:48.531377+0000) 2024-04-21T08:52:50.946 INFO:journalctl@ceph.osd.4.smithi088.stdout:Apr 21 08:52:50 smithi088 podman[68023]: 2024-04-21 08:52:50.587814781 +0000 UTC m=+0.017786180 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:52:50.947 INFO:journalctl@ceph.osd.4.smithi088.stdout:Apr 21 08:52:50 smithi088 podman[68023]: 2024-04-21 08:52:50.711167567 +0000 UTC m=+0.141138998 container create 6749f444c8e74de9a5a8a19cb77b816cdcfc0874119b31c0a2d44dbbc2d78b08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD) 2024-04-21T08:52:50.947 INFO:journalctl@ceph.osd.4.smithi088.stdout:Apr 21 08:52:50 smithi088 podman[68023]: 2024-04-21 08:52:50.94777613 +0000 UTC m=+0.377747520 container init 6749f444c8e74de9a5a8a19cb77b816cdcfc0874119b31c0a2d44dbbc2d78b08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-4-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS) 2024-04-21T08:52:51.117 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:52:50 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:52:50.811+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:16.244986+0000 front 2024-04-21T08:52:16.245244+0000 (oldest deadline 2024-04-21T08:52:40.945040+0000) 2024-04-21T08:52:51.117 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:52:50 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:52:50.811+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:25.046260+0000 front 2024-04-21T08:52:25.046169+0000 (oldest deadline 2024-04-21T08:52:46.746043+0000) 2024-04-21T08:52:51.117 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:50 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:50.625+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:52:51.117 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:50 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:50.625+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:52:51.214 INFO:journalctl@ceph.osd.4.smithi088.stdout:Apr 21 08:52:50 smithi088 podman[68023]: 2024-04-21 08:52:50.950873392 +0000 UTC m=+0.380844781 container start 6749f444c8e74de9a5a8a19cb77b816cdcfc0874119b31c0a2d44dbbc2d78b08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, ceph=True) 2024-04-21T08:52:51.214 INFO:journalctl@ceph.osd.4.smithi088.stdout:Apr 21 08:52:51 smithi088 podman[68023]: 2024-04-21 08:52:51.005698652 +0000 UTC m=+0.435670042 container attach 6749f444c8e74de9a5a8a19cb77b816cdcfc0874119b31c0a2d44dbbc2d78b08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-4-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, ceph=True, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2) 2024-04-21T08:52:51.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:52:50 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:52:50.842+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:14.929128+0000 front 2024-04-21T08:52:14.929188+0000 (oldest deadline 2024-04-21T08:52:40.829047+0000) 2024-04-21T08:52:51.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:52:50 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:52:50.842+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.329978+0000 front 2024-04-21T08:52:24.329957+0000 (oldest deadline 2024-04-21T08:52:46.029966+0000) 2024-04-21T08:52:51.541 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:52:51 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:52:51.481+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.388905+0000 front 2024-04-21T08:52:24.388964+0000 (oldest deadline 2024-04-21T08:52:49.689097+0000) 2024-04-21T08:52:51.542 INFO:journalctl@ceph.osd.4.smithi088.stdout:Apr 21 08:52:51 smithi088 podman[68023]: 2024-04-21 08:52:51.24349808 +0000 UTC m=+0.673469471 container died 6749f444c8e74de9a5a8a19cb77b816cdcfc0874119b31c0a2d44dbbc2d78b08 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-4-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, ceph=True, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T08:52:51.542 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:52:51 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:52:51.366+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:52:51.542 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:52:51 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:52:51.366+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:52:51.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:52:51 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:52:51.293+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:52:51.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:52:51 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:52:51.293+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:52:51.747 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:52:51 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-5[43475]: 2024-04-21T08:52:51.465+0000 7f8db825a700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:16.830259+0000 front 2024-04-21T08:52:16.830124+0000 (oldest deadline 2024-04-21T08:52:42.130317+0000) 2024-04-21T08:52:51.747 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:52:51 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-5[43475]: 2024-04-21T08:52:51.465+0000 7f8db825a700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.431373+0000 front 2024-04-21T08:52:24.431301+0000 (oldest deadline 2024-04-21T08:52:48.531377+0000) 2024-04-21T08:52:51.859 DEBUG:teuthology.orchestra.run.smithi088:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@osd.4.service' 2024-04-21T08:52:51.867 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:52:51 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:52:51.841+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:16.244986+0000 front 2024-04-21T08:52:16.245244+0000 (oldest deadline 2024-04-21T08:52:40.945040+0000) 2024-04-21T08:52:51.867 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:52:51 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:52:51.841+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:25.046260+0000 front 2024-04-21T08:52:25.046169+0000 (oldest deadline 2024-04-21T08:52:46.746043+0000) 2024-04-21T08:52:51.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:51 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:51.595+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:52:51.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:51 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:51.595+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:52:51.895 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:52:51.895 INFO:tasks.cephadm.osd.4:Stopped osd.4 2024-04-21T08:52:51.895 INFO:tasks.cephadm.osd.5:Stopping osd.5... 2024-04-21T08:52:51.895 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@osd.5 2024-04-21T08:52:52.220 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:52:51 smithi179 systemd[1]: Stopping Ceph osd.5 for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:52:52.220 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:52:51 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:52:51.853+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:14.929128+0000 front 2024-04-21T08:52:14.929188+0000 (oldest deadline 2024-04-21T08:52:40.829047+0000) 2024-04-21T08:52:52.220 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:52:51 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:52:51.853+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.329978+0000 front 2024-04-21T08:52:24.329957+0000 (oldest deadline 2024-04-21T08:52:46.029966+0000) 2024-04-21T08:52:52.474 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:52:52 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:52:52.256+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:52:52.474 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:52:52 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:52:52.256+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:52:52.474 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:52:52 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-5[43475]: 2024-04-21T08:52:52.220+0000 7f8dbfc6e700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.5 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:52:52.475 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:52:52 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-5[43475]: 2024-04-21T08:52:52.220+0000 7f8dbfc6e700 -1 osd.5 112 *** Got signal Terminated *** 2024-04-21T08:52:52.475 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:52:52 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-5[43475]: 2024-04-21T08:52:52.220+0000 7f8dbfc6e700 -1 osd.5 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T08:52:52.475 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:52:52 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-5[43475]: 2024-04-21T08:52:52.475+0000 7f8db825a700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:16.830259+0000 front 2024-04-21T08:52:16.830124+0000 (oldest deadline 2024-04-21T08:52:42.130317+0000) 2024-04-21T08:52:52.475 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:52:52 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-5[43475]: 2024-04-21T08:52:52.475+0000 7f8db825a700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.431373+0000 front 2024-04-21T08:52:24.431301+0000 (oldest deadline 2024-04-21T08:52:48.531377+0000) 2024-04-21T08:52:52.714 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:52:52 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:52:52.503+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.388905+0000 front 2024-04-21T08:52:24.388964+0000 (oldest deadline 2024-04-21T08:52:49.689097+0000) 2024-04-21T08:52:52.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:52:52 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:52:52.318+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:52:52.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:52:52 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:52:52.318+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:52:52.869 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:52 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:52.584+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:52:52.870 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:52 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:52.584+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:52:53.231 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:52:52 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:52:52.866+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:14.929128+0000 front 2024-04-21T08:52:14.929188+0000 (oldest deadline 2024-04-21T08:52:40.829047+0000) 2024-04-21T08:52:53.231 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:52:52 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:52:52.866+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.329978+0000 front 2024-04-21T08:52:24.329957+0000 (oldest deadline 2024-04-21T08:52:46.029966+0000) 2024-04-21T08:52:53.367 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:52:52 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:52:52.869+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:16.244986+0000 front 2024-04-21T08:52:16.245244+0000 (oldest deadline 2024-04-21T08:52:40.945040+0000) 2024-04-21T08:52:53.367 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:52:52 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:52:52.869+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:25.046260+0000 front 2024-04-21T08:52:25.046169+0000 (oldest deadline 2024-04-21T08:52:46.746043+0000) 2024-04-21T08:52:53.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:52:53 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:52:53.232+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:52:53.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:52:53 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:52:53.232+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:52:53.497 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:52:53 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-5[43475]: 2024-04-21T08:52:53.455+0000 7f8db825a700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:16.830259+0000 front 2024-04-21T08:52:16.830124+0000 (oldest deadline 2024-04-21T08:52:42.130317+0000) 2024-04-21T08:52:53.497 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:52:53 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-5[43475]: 2024-04-21T08:52:53.455+0000 7f8db825a700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.431373+0000 front 2024-04-21T08:52:24.431301+0000 (oldest deadline 2024-04-21T08:52:48.531377+0000) 2024-04-21T08:52:53.714 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:52:53 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:52:53.477+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.388905+0000 front 2024-04-21T08:52:24.388964+0000 (oldest deadline 2024-04-21T08:52:49.689097+0000) 2024-04-21T08:52:53.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:52:53 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:52:53.366+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:52:53.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:52:53 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:52:53.366+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:52:53.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:53 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:53.550+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:52:53.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:53 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:53.550+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:52:54.238 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:52:53 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:52:53.859+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:14.929128+0000 front 2024-04-21T08:52:14.929188+0000 (oldest deadline 2024-04-21T08:52:40.829047+0000) 2024-04-21T08:52:54.238 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:52:53 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:52:53.859+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.329978+0000 front 2024-04-21T08:52:24.329957+0000 (oldest deadline 2024-04-21T08:52:46.029966+0000) 2024-04-21T08:52:54.367 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:52:53 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:52:53.873+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:16.244986+0000 front 2024-04-21T08:52:16.245244+0000 (oldest deadline 2024-04-21T08:52:40.945040+0000) 2024-04-21T08:52:54.367 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:52:53 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:52:53.873+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:25.046260+0000 front 2024-04-21T08:52:25.046169+0000 (oldest deadline 2024-04-21T08:52:46.746043+0000) 2024-04-21T08:52:54.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:52:54 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:52:54.238+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:52:54.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:52:54 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:52:54.238+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:52:54.497 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:52:54 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-5[43475]: 2024-04-21T08:52:54.485+0000 7f8db825a700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:16.830259+0000 front 2024-04-21T08:52:16.830124+0000 (oldest deadline 2024-04-21T08:52:42.130317+0000) 2024-04-21T08:52:54.497 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:52:54 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-5[43475]: 2024-04-21T08:52:54.485+0000 7f8db825a700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.431373+0000 front 2024-04-21T08:52:24.431301+0000 (oldest deadline 2024-04-21T08:52:48.531377+0000) 2024-04-21T08:52:54.714 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:52:54 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:52:54.515+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.388905+0000 front 2024-04-21T08:52:24.388964+0000 (oldest deadline 2024-04-21T08:52:49.689097+0000) 2024-04-21T08:52:54.715 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:52:54 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:52:54.409+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:52:54.715 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:52:54 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:52:54.409+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:52:54.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:54 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:54.525+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:52:54.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:54 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:54.525+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:52:55.190 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:52:54 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:52:54.872+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:14.929128+0000 front 2024-04-21T08:52:14.929188+0000 (oldest deadline 2024-04-21T08:52:40.829047+0000) 2024-04-21T08:52:55.190 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:52:54 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:52:54.872+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.329978+0000 front 2024-04-21T08:52:24.329957+0000 (oldest deadline 2024-04-21T08:52:46.029966+0000) 2024-04-21T08:52:55.367 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:52:54 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:52:54.913+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:16.244986+0000 front 2024-04-21T08:52:16.245244+0000 (oldest deadline 2024-04-21T08:52:40.945040+0000) 2024-04-21T08:52:55.367 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:52:54 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:52:54.913+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:25.046260+0000 front 2024-04-21T08:52:25.046169+0000 (oldest deadline 2024-04-21T08:52:46.746043+0000) 2024-04-21T08:52:55.449 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:52:55 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:52:55.190+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:52:55.449 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:52:55 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:52:55.190+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:52:55.714 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:52:55 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:52:55.558+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.388905+0000 front 2024-04-21T08:52:24.388964+0000 (oldest deadline 2024-04-21T08:52:49.689097+0000) 2024-04-21T08:52:55.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:52:55 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:52:55.406+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:52:55.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:52:55 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:52:55.406+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:52:55.746 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:52:55 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-5[43475]: 2024-04-21T08:52:55.450+0000 7f8db825a700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:16.830259+0000 front 2024-04-21T08:52:16.830124+0000 (oldest deadline 2024-04-21T08:52:42.130317+0000) 2024-04-21T08:52:55.747 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:52:55 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-5[43475]: 2024-04-21T08:52:55.450+0000 7f8db825a700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.431373+0000 front 2024-04-21T08:52:24.431301+0000 (oldest deadline 2024-04-21T08:52:48.531377+0000) 2024-04-21T08:52:55.866 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:55 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:55.515+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:52:55.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:55 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:55.515+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:52:55.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:55 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:55.515+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.405223+0000 front 2024-04-21T08:52:30.405168+0000 (oldest deadline 2024-04-21T08:52:55.105127+0000) 2024-04-21T08:52:56.234 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:52:55 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:52:55.856+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:14.929128+0000 front 2024-04-21T08:52:14.929188+0000 (oldest deadline 2024-04-21T08:52:40.829047+0000) 2024-04-21T08:52:56.234 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:52:55 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:52:55.856+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.329978+0000 front 2024-04-21T08:52:24.329957+0000 (oldest deadline 2024-04-21T08:52:46.029966+0000) 2024-04-21T08:52:56.234 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:52:55 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:52:55.856+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:32.431682+0000 front 2024-04-21T08:52:32.431710+0000 (oldest deadline 2024-04-21T08:52:55.232351+0000) 2024-04-21T08:52:56.367 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:52:55 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:52:55.885+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:16.244986+0000 front 2024-04-21T08:52:16.245244+0000 (oldest deadline 2024-04-21T08:52:40.945040+0000) 2024-04-21T08:52:56.367 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:52:55 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:52:55.885+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:25.046260+0000 front 2024-04-21T08:52:25.046169+0000 (oldest deadline 2024-04-21T08:52:46.746043+0000) 2024-04-21T08:52:56.367 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:52:55 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:52:55.885+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.147427+0000 front 2024-04-21T08:52:30.147616+0000 (oldest deadline 2024-04-21T08:52:55.447369+0000) 2024-04-21T08:52:56.496 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:52:56 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:52:56.234+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:52:56.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:52:56 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:52:56.234+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:52:56.497 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:52:56 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-5[43475]: 2024-04-21T08:52:56.460+0000 7f8db825a700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:16.830259+0000 front 2024-04-21T08:52:16.830124+0000 (oldest deadline 2024-04-21T08:52:42.130317+0000) 2024-04-21T08:52:56.497 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:52:56 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-5[43475]: 2024-04-21T08:52:56.460+0000 7f8db825a700 -1 osd.5 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.431373+0000 front 2024-04-21T08:52:24.431301+0000 (oldest deadline 2024-04-21T08:52:48.531377+0000) 2024-04-21T08:52:56.714 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:52:56 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:52:56.594+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.388905+0000 front 2024-04-21T08:52:24.388964+0000 (oldest deadline 2024-04-21T08:52:49.689097+0000) 2024-04-21T08:52:56.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:52:56 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:52:56.450+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:52:56.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:52:56 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:52:56.450+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:52:56.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:56 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:56.500+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:52:56.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:56 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:56.500+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:52:56.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:56 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:56.500+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.405223+0000 front 2024-04-21T08:52:30.405168+0000 (oldest deadline 2024-04-21T08:52:55.105127+0000) 2024-04-21T08:52:57.225 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:52:56 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:52:56.829+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:14.929128+0000 front 2024-04-21T08:52:14.929188+0000 (oldest deadline 2024-04-21T08:52:40.829047+0000) 2024-04-21T08:52:57.225 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:52:56 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:52:56.829+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.329978+0000 front 2024-04-21T08:52:24.329957+0000 (oldest deadline 2024-04-21T08:52:46.029966+0000) 2024-04-21T08:52:57.225 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:52:56 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:52:56.829+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:32.431682+0000 front 2024-04-21T08:52:32.431710+0000 (oldest deadline 2024-04-21T08:52:55.232351+0000) 2024-04-21T08:52:57.367 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:52:56 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:52:56.871+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:16.244986+0000 front 2024-04-21T08:52:16.245244+0000 (oldest deadline 2024-04-21T08:52:40.945040+0000) 2024-04-21T08:52:57.367 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:52:56 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:52:56.871+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:25.046260+0000 front 2024-04-21T08:52:25.046169+0000 (oldest deadline 2024-04-21T08:52:46.746043+0000) 2024-04-21T08:52:57.367 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:52:56 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:52:56.871+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.147427+0000 front 2024-04-21T08:52:30.147616+0000 (oldest deadline 2024-04-21T08:52:55.447369+0000) 2024-04-21T08:52:57.496 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:52:57 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:52:57.225+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:52:57.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:52:57 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:52:57.225+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:52:57.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:52:57 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:52:57.225+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:52:57.714 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:52:57 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:52:57.572+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.388905+0000 front 2024-04-21T08:52:24.388964+0000 (oldest deadline 2024-04-21T08:52:49.689097+0000) 2024-04-21T08:52:57.714 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:52:57 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:52:57.572+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.390238+0000 front 2024-04-21T08:52:31.390386+0000 (oldest deadline 2024-04-21T08:52:57.290277+0000) 2024-04-21T08:52:57.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:52:57 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:52:57.442+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:52:57.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:52:57 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:52:57.442+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:52:57.715 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:52:57 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:52:57.442+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:52:57.856 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:57 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:57.542+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:52:57.857 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:57 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:57.542+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:52:57.857 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:57 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:57.542+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.405223+0000 front 2024-04-21T08:52:30.405168+0000 (oldest deadline 2024-04-21T08:52:55.105127+0000) 2024-04-21T08:52:57.895 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:52:57 smithi179 podman[68859]: 2024-04-21 08:52:57.597633161 +0000 UTC m=+5.500523245 container died 2c39de4ecbad19b267c211f867a619616d0b95db8d7617e6e357db1c9e654743 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, ceph=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:52:57.896 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:52:57 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:52:57.837+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:14.929128+0000 front 2024-04-21T08:52:14.929188+0000 (oldest deadline 2024-04-21T08:52:40.829047+0000) 2024-04-21T08:52:57.896 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:52:57 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:52:57.837+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.329978+0000 front 2024-04-21T08:52:24.329957+0000 (oldest deadline 2024-04-21T08:52:46.029966+0000) 2024-04-21T08:52:57.896 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:52:57 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:52:57.837+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:32.431682+0000 front 2024-04-21T08:52:32.431710+0000 (oldest deadline 2024-04-21T08:52:55.232351+0000) 2024-04-21T08:52:58.117 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:52:57 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:52:57.857+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:16.244986+0000 front 2024-04-21T08:52:16.245244+0000 (oldest deadline 2024-04-21T08:52:40.945040+0000) 2024-04-21T08:52:58.117 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:52:57 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:52:57.857+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:25.046260+0000 front 2024-04-21T08:52:25.046169+0000 (oldest deadline 2024-04-21T08:52:46.746043+0000) 2024-04-21T08:52:58.117 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:52:57 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:52:57.857+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.147427+0000 front 2024-04-21T08:52:30.147616+0000 (oldest deadline 2024-04-21T08:52:55.447369+0000) 2024-04-21T08:52:58.182 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:52:57 smithi179 podman[68859]: 2024-04-21 08:52:57.996240184 +0000 UTC m=+5.899130274 container cleanup 2c39de4ecbad19b267c211f867a619616d0b95db8d7617e6e357db1c9e654743 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-5, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5) 2024-04-21T08:52:58.183 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:52:57 smithi179 bash[68859]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-5 2024-04-21T08:52:58.497 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:52:58 smithi179 podman[68877]: 2024-04-21 08:52:58.25610674 +0000 UTC m=+0.649425223 container remove 2c39de4ecbad19b267c211f867a619616d0b95db8d7617e6e357db1c9e654743 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-5, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux ) 2024-04-21T08:52:58.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:52:58 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:52:58.181+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:52:58.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:52:58 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:52:58.181+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:52:58.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:52:58 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:52:58.181+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:52:58.834 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:52:58 smithi179 podman[68966]: 2024-04-21 08:52:58.472181679 +0000 UTC m=+0.017964584 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:52:58.834 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:52:58 smithi179 podman[68966]: 2024-04-21 08:52:58.598707872 +0000 UTC m=+0.144490815 container create 7a79125fb4db510b1eeed578ea6c289c756580bda51f0b8bbbcacac9822f2de3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-5-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS) 2024-04-21T08:52:58.839 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:58 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:58.529+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:52:58.839 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:58 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:58.529+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:52:58.839 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:58 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:58.529+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.405223+0000 front 2024-04-21T08:52:30.405168+0000 (oldest deadline 2024-04-21T08:52:55.105127+0000) 2024-04-21T08:52:58.964 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:52:58 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:52:58.525+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.388905+0000 front 2024-04-21T08:52:24.388964+0000 (oldest deadline 2024-04-21T08:52:49.689097+0000) 2024-04-21T08:52:58.964 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:52:58 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:52:58.525+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.390238+0000 front 2024-04-21T08:52:31.390386+0000 (oldest deadline 2024-04-21T08:52:57.290277+0000) 2024-04-21T08:52:58.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:52:58 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:52:58.483+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:52:58.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:52:58 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:52:58.483+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:52:58.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:52:58 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:52:58.483+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:52:59.117 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:52:58 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:52:58.840+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:16.244986+0000 front 2024-04-21T08:52:16.245244+0000 (oldest deadline 2024-04-21T08:52:40.945040+0000) 2024-04-21T08:52:59.117 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:52:58 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:52:58.840+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:25.046260+0000 front 2024-04-21T08:52:25.046169+0000 (oldest deadline 2024-04-21T08:52:46.746043+0000) 2024-04-21T08:52:59.117 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:52:58 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:52:58.840+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.147427+0000 front 2024-04-21T08:52:30.147616+0000 (oldest deadline 2024-04-21T08:52:55.447369+0000) 2024-04-21T08:52:59.123 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:52:58 smithi179 podman[68966]: 2024-04-21 08:52:58.83497419 +0000 UTC m=+0.380757094 container init 7a79125fb4db510b1eeed578ea6c289c756580bda51f0b8bbbcacac9822f2de3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-5-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-21T08:52:59.123 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:52:58 smithi179 podman[68966]: 2024-04-21 08:52:58.838341146 +0000 UTC m=+0.384124048 container start 7a79125fb4db510b1eeed578ea6c289c756580bda51f0b8bbbcacac9822f2de3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-5-deactivate, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:52:59.123 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:52:58 smithi179 podman[68966]: 2024-04-21 08:52:58.92622688 +0000 UTC m=+0.472009784 container attach 7a79125fb4db510b1eeed578ea6c289c756580bda51f0b8bbbcacac9822f2de3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-5-deactivate, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, ceph=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS) 2024-04-21T08:52:59.123 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:52:58 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:52:58.861+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:14.929128+0000 front 2024-04-21T08:52:14.929188+0000 (oldest deadline 2024-04-21T08:52:40.829047+0000) 2024-04-21T08:52:59.123 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:52:58 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:52:58.861+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.329978+0000 front 2024-04-21T08:52:24.329957+0000 (oldest deadline 2024-04-21T08:52:46.029966+0000) 2024-04-21T08:52:59.123 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:52:58 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:52:58.861+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:32.431682+0000 front 2024-04-21T08:52:32.431710+0000 (oldest deadline 2024-04-21T08:52:55.232351+0000) 2024-04-21T08:52:59.428 INFO:journalctl@ceph.osd.5.smithi179.stdout:Apr 21 08:52:59 smithi179 podman[68966]: 2024-04-21 08:52:59.124262697 +0000 UTC m=+0.670045595 container died 7a79125fb4db510b1eeed578ea6c289c756580bda51f0b8bbbcacac9822f2de3 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-5-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416) 2024-04-21T08:52:59.428 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:52:59 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:52:59.181+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:52:59.428 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:52:59 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:52:59.181+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:52:59.428 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:52:59 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:52:59.181+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:52:59.714 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:52:59 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:52:59.479+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.388905+0000 front 2024-04-21T08:52:24.388964+0000 (oldest deadline 2024-04-21T08:52:49.689097+0000) 2024-04-21T08:52:59.714 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:52:59 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:52:59.479+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.390238+0000 front 2024-04-21T08:52:31.390386+0000 (oldest deadline 2024-04-21T08:52:57.290277+0000) 2024-04-21T08:52:59.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:52:59 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:52:59.448+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:52:59.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:52:59 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:52:59.448+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:52:59.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:52:59 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:52:59.448+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:52:59.759 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@osd.5.service' 2024-04-21T08:52:59.795 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:59 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:59.493+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:52:59.795 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:59 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:59.493+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:52:59.795 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:52:59 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:52:59.493+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.405223+0000 front 2024-04-21T08:52:30.405168+0000 (oldest deadline 2024-04-21T08:52:55.105127+0000) 2024-04-21T08:52:59.797 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:52:59.797 INFO:tasks.cephadm.osd.5:Stopped osd.5 2024-04-21T08:52:59.797 INFO:tasks.cephadm.osd.6:Stopping osd.6... 2024-04-21T08:52:59.797 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@osd.6 2024-04-21T08:52:59.997 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:52:59 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:52:59.861+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:14.929128+0000 front 2024-04-21T08:52:14.929188+0000 (oldest deadline 2024-04-21T08:52:40.829047+0000) 2024-04-21T08:52:59.997 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:52:59 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:52:59.861+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.329978+0000 front 2024-04-21T08:52:24.329957+0000 (oldest deadline 2024-04-21T08:52:46.029966+0000) 2024-04-21T08:52:59.997 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:52:59 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:52:59.861+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:32.431682+0000 front 2024-04-21T08:52:32.431710+0000 (oldest deadline 2024-04-21T08:52:55.232351+0000) 2024-04-21T08:53:00.110 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:52:59 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:52:59.796+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:16.244986+0000 front 2024-04-21T08:52:16.245244+0000 (oldest deadline 2024-04-21T08:52:40.945040+0000) 2024-04-21T08:53:00.110 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:52:59 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:52:59.796+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:25.046260+0000 front 2024-04-21T08:52:25.046169+0000 (oldest deadline 2024-04-21T08:52:46.746043+0000) 2024-04-21T08:53:00.110 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:52:59 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:52:59.796+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.147427+0000 front 2024-04-21T08:52:30.147616+0000 (oldest deadline 2024-04-21T08:52:55.447369+0000) 2024-04-21T08:53:00.110 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:52:59 smithi086 systemd[1]: Stopping Ceph osd.6 for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:53:00.367 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:53:00 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:53:00.110+0000 7f654e9d1700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.6 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:53:00.367 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:53:00 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:53:00.110+0000 7f654e9d1700 -1 osd.6 112 *** Got signal Terminated *** 2024-04-21T08:53:00.367 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:53:00 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:53:00.110+0000 7f654e9d1700 -1 osd.6 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T08:53:00.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:00 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:00.193+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:00.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:00 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:00.193+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:00.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:00 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:00.193+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:00.714 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:00 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:53:00.506+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.388905+0000 front 2024-04-21T08:52:24.388964+0000 (oldest deadline 2024-04-21T08:52:49.689097+0000) 2024-04-21T08:53:00.714 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:00 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:53:00.506+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.390238+0000 front 2024-04-21T08:52:31.390386+0000 (oldest deadline 2024-04-21T08:52:57.290277+0000) 2024-04-21T08:53:00.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:00 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:00.450+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:53:00.715 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:00 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:00.450+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:53:00.715 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:00 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:00.450+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:53:00.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:00 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:00.543+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:53:00.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:00 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:00.543+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:53:00.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:00 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:00.543+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.405223+0000 front 2024-04-21T08:52:30.405168+0000 (oldest deadline 2024-04-21T08:52:55.105127+0000) 2024-04-21T08:53:00.867 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:53:00 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:53:00.786+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:16.244986+0000 front 2024-04-21T08:52:16.245244+0000 (oldest deadline 2024-04-21T08:52:40.945040+0000) 2024-04-21T08:53:00.867 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:53:00 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:53:00.786+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:25.046260+0000 front 2024-04-21T08:52:25.046169+0000 (oldest deadline 2024-04-21T08:52:46.746043+0000) 2024-04-21T08:53:00.867 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:53:00 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:53:00.786+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.147427+0000 front 2024-04-21T08:52:30.147616+0000 (oldest deadline 2024-04-21T08:52:55.447369+0000) 2024-04-21T08:53:01.229 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:00 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:00.901+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:14.929128+0000 front 2024-04-21T08:52:14.929188+0000 (oldest deadline 2024-04-21T08:52:40.829047+0000) 2024-04-21T08:53:01.230 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:00 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:00.901+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.329978+0000 front 2024-04-21T08:52:24.329957+0000 (oldest deadline 2024-04-21T08:52:46.029966+0000) 2024-04-21T08:53:01.230 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:00 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:00.901+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:32.431682+0000 front 2024-04-21T08:52:32.431710+0000 (oldest deadline 2024-04-21T08:52:55.232351+0000) 2024-04-21T08:53:01.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:01 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:01.230+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:01.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:01 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:01.230+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:01.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:01 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:01.230+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:01.867 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:53:01 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:53:01.744+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:16.244986+0000 front 2024-04-21T08:52:16.245244+0000 (oldest deadline 2024-04-21T08:52:40.945040+0000) 2024-04-21T08:53:01.867 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:53:01 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:53:01.744+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:25.046260+0000 front 2024-04-21T08:52:25.046169+0000 (oldest deadline 2024-04-21T08:52:46.746043+0000) 2024-04-21T08:53:01.867 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:53:01 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:53:01.744+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.147427+0000 front 2024-04-21T08:52:30.147616+0000 (oldest deadline 2024-04-21T08:52:55.447369+0000) 2024-04-21T08:53:01.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:01 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:01.520+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:53:01.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:01 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:01.520+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:53:01.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:01 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:01.520+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.405223+0000 front 2024-04-21T08:52:30.405168+0000 (oldest deadline 2024-04-21T08:52:55.105127+0000) 2024-04-21T08:53:01.964 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:01 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:53:01.532+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.388905+0000 front 2024-04-21T08:52:24.388964+0000 (oldest deadline 2024-04-21T08:52:49.689097+0000) 2024-04-21T08:53:01.964 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:01 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:53:01.532+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.390238+0000 front 2024-04-21T08:52:31.390386+0000 (oldest deadline 2024-04-21T08:52:57.290277+0000) 2024-04-21T08:53:01.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:01 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:01.472+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:53:01.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:01 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:01.472+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:53:01.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:01 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:01.472+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:53:02.246 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:01 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:01.908+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:14.929128+0000 front 2024-04-21T08:52:14.929188+0000 (oldest deadline 2024-04-21T08:52:40.829047+0000) 2024-04-21T08:53:02.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:01 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:01.908+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.329978+0000 front 2024-04-21T08:52:24.329957+0000 (oldest deadline 2024-04-21T08:52:46.029966+0000) 2024-04-21T08:53:02.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:01 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:01.908+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:32.431682+0000 front 2024-04-21T08:52:32.431710+0000 (oldest deadline 2024-04-21T08:52:55.232351+0000) 2024-04-21T08:53:02.714 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:02 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:53:02.520+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.388905+0000 front 2024-04-21T08:52:24.388964+0000 (oldest deadline 2024-04-21T08:52:49.689097+0000) 2024-04-21T08:53:02.714 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:02 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:53:02.520+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.390238+0000 front 2024-04-21T08:52:31.390386+0000 (oldest deadline 2024-04-21T08:52:57.290277+0000) 2024-04-21T08:53:02.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:02 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:02.429+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:53:02.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:02 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:02.429+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:53:02.715 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:02 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:02.429+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:53:02.715 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:02 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:02.429+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:39.228449+0000 front 2024-04-21T08:52:39.228437+0000 (oldest deadline 2024-04-21T08:53:02.029070+0000) 2024-04-21T08:53:02.746 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:02 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:02.254+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:02.746 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:02 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:02.254+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:02.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:02 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:02.254+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:02.758 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:02 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:02.485+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:53:02.758 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:02 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:02.485+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:53:02.758 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:02 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:02.485+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.405223+0000 front 2024-04-21T08:52:30.405168+0000 (oldest deadline 2024-04-21T08:52:55.105127+0000) 2024-04-21T08:53:03.117 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:53:02 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:53:02.759+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:16.244986+0000 front 2024-04-21T08:52:16.245244+0000 (oldest deadline 2024-04-21T08:52:40.945040+0000) 2024-04-21T08:53:03.117 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:53:02 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:53:02.759+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:25.046260+0000 front 2024-04-21T08:52:25.046169+0000 (oldest deadline 2024-04-21T08:52:46.746043+0000) 2024-04-21T08:53:03.117 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:53:02 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:53:02.759+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.147427+0000 front 2024-04-21T08:52:30.147616+0000 (oldest deadline 2024-04-21T08:52:55.447369+0000) 2024-04-21T08:53:03.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:02 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:02.878+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:14.929128+0000 front 2024-04-21T08:52:14.929188+0000 (oldest deadline 2024-04-21T08:52:40.829047+0000) 2024-04-21T08:53:03.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:02 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:02.878+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.329978+0000 front 2024-04-21T08:52:24.329957+0000 (oldest deadline 2024-04-21T08:52:46.029966+0000) 2024-04-21T08:53:03.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:02 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:02.878+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:32.431682+0000 front 2024-04-21T08:52:32.431710+0000 (oldest deadline 2024-04-21T08:52:55.232351+0000) 2024-04-21T08:53:03.520 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:03 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:03.281+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:03.521 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:03 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:03.281+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:03.521 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:03 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:03.281+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:03.714 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:03 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:53:03.492+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.388905+0000 front 2024-04-21T08:52:24.388964+0000 (oldest deadline 2024-04-21T08:52:49.689097+0000) 2024-04-21T08:53:03.714 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:03 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:53:03.492+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.390238+0000 front 2024-04-21T08:52:31.390386+0000 (oldest deadline 2024-04-21T08:52:57.290277+0000) 2024-04-21T08:53:03.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:03 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:03.458+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:53:03.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:03 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:03.458+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:53:03.715 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:03 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:03.458+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:53:03.715 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:03 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:03.458+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:39.228449+0000 front 2024-04-21T08:52:39.228437+0000 (oldest deadline 2024-04-21T08:53:02.029070+0000) 2024-04-21T08:53:03.752 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:03 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:03.446+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:53:03.752 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:03 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:03.446+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:53:03.752 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:03 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:03.446+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.405223+0000 front 2024-04-21T08:52:30.405168+0000 (oldest deadline 2024-04-21T08:52:55.105127+0000) 2024-04-21T08:53:04.117 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:53:03 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:53:03.753+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:16.244986+0000 front 2024-04-21T08:52:16.245244+0000 (oldest deadline 2024-04-21T08:52:40.945040+0000) 2024-04-21T08:53:04.117 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:53:03 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:53:03.753+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:25.046260+0000 front 2024-04-21T08:52:25.046169+0000 (oldest deadline 2024-04-21T08:52:46.746043+0000) 2024-04-21T08:53:04.117 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:53:03 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:53:03.753+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.147427+0000 front 2024-04-21T08:52:30.147616+0000 (oldest deadline 2024-04-21T08:52:55.447369+0000) 2024-04-21T08:53:04.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:03 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:03.893+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:14.929128+0000 front 2024-04-21T08:52:14.929188+0000 (oldest deadline 2024-04-21T08:52:40.829047+0000) 2024-04-21T08:53:04.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:03 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:03.893+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.329978+0000 front 2024-04-21T08:52:24.329957+0000 (oldest deadline 2024-04-21T08:52:46.029966+0000) 2024-04-21T08:53:04.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:03 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:03.893+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:32.431682+0000 front 2024-04-21T08:52:32.431710+0000 (oldest deadline 2024-04-21T08:52:55.232351+0000) 2024-04-21T08:53:04.714 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:04 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:53:04.444+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.388905+0000 front 2024-04-21T08:52:24.388964+0000 (oldest deadline 2024-04-21T08:52:49.689097+0000) 2024-04-21T08:53:04.714 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:04 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:53:04.444+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.390238+0000 front 2024-04-21T08:52:31.390386+0000 (oldest deadline 2024-04-21T08:52:57.290277+0000) 2024-04-21T08:53:04.715 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:04 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:04.444+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:53:04.715 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:04 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:04.444+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:53:04.715 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:04 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:04.444+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:53:04.715 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:04 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:04.444+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:39.228449+0000 front 2024-04-21T08:52:39.228437+0000 (oldest deadline 2024-04-21T08:53:02.029070+0000) 2024-04-21T08:53:04.746 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:04 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:04.310+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:04.746 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:04 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:04.310+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:04.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:04 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:04.310+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:04.750 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:04 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:04.411+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:53:04.750 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:04 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:04.411+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:53:04.750 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:04 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:04.411+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.405223+0000 front 2024-04-21T08:52:30.405168+0000 (oldest deadline 2024-04-21T08:52:55.105127+0000) 2024-04-21T08:53:05.116 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:53:04 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:53:04.751+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:16.244986+0000 front 2024-04-21T08:52:16.245244+0000 (oldest deadline 2024-04-21T08:52:40.945040+0000) 2024-04-21T08:53:05.117 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:53:04 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:53:04.751+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:25.046260+0000 front 2024-04-21T08:52:25.046169+0000 (oldest deadline 2024-04-21T08:52:46.746043+0000) 2024-04-21T08:53:05.117 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:53:04 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6[54036]: 2024-04-21T08:53:04.751+0000 7f654afc5700 -1 osd.6 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.147427+0000 front 2024-04-21T08:52:30.147616+0000 (oldest deadline 2024-04-21T08:52:55.447369+0000) 2024-04-21T08:53:05.246 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:04 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:04.851+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:14.929128+0000 front 2024-04-21T08:52:14.929188+0000 (oldest deadline 2024-04-21T08:52:40.829047+0000) 2024-04-21T08:53:05.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:04 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:04.851+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.329978+0000 front 2024-04-21T08:52:24.329957+0000 (oldest deadline 2024-04-21T08:52:46.029966+0000) 2024-04-21T08:53:05.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:04 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:04.851+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:32.431682+0000 front 2024-04-21T08:52:32.431710+0000 (oldest deadline 2024-04-21T08:52:55.232351+0000) 2024-04-21T08:53:05.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:04 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:04.851+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:38.733223+0000 front 2024-04-21T08:52:38.733348+0000 (oldest deadline 2024-04-21T08:53:04.033231+0000) 2024-04-21T08:53:05.711 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:53:05 smithi086 podman[104918]: 2024-04-21 08:53:05.425401006 +0000 UTC m=+5.441403773 container died b1aae3596c7df4713ea305bc587568655431c558e0a8041bc1516c51d24de328 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-21T08:53:05.711 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:05 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:05.393+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:53:05.711 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:05 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:05.393+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:53:05.711 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:05 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:05.393+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.405223+0000 front 2024-04-21T08:52:30.405168+0000 (oldest deadline 2024-04-21T08:52:55.105127+0000) 2024-04-21T08:53:05.711 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:05 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:05.393+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:40.608170+0000 front 2024-04-21T08:52:40.608213+0000 (oldest deadline 2024-04-21T08:53:05.308338+0000) 2024-04-21T08:53:05.714 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:05 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:53:05.453+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.388905+0000 front 2024-04-21T08:52:24.388964+0000 (oldest deadline 2024-04-21T08:52:49.689097+0000) 2024-04-21T08:53:05.714 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:05 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:53:05.453+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.390238+0000 front 2024-04-21T08:52:31.390386+0000 (oldest deadline 2024-04-21T08:52:57.290277+0000) 2024-04-21T08:53:05.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:05 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:05.411+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:53:05.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:05 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:05.411+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:53:05.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:05 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:05.411+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:53:05.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:05 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:05.411+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:39.228449+0000 front 2024-04-21T08:52:39.228437+0000 (oldest deadline 2024-04-21T08:53:02.029070+0000) 2024-04-21T08:53:05.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:05 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:05.307+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:05.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:05 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:05.307+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:05.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:05 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:05.307+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:06.070 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:53:05 smithi086 podman[104918]: 2024-04-21 08:53:05.81923904 +0000 UTC m=+5.835241817 container cleanup b1aae3596c7df4713ea305bc587568655431c558e0a8041bc1516c51d24de328 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6, org.label-schema.schema-version=1.0, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-21T08:53:06.070 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:53:05 smithi086 bash[104918]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6 2024-04-21T08:53:06.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:05 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:05.816+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:14.929128+0000 front 2024-04-21T08:52:14.929188+0000 (oldest deadline 2024-04-21T08:52:40.829047+0000) 2024-04-21T08:53:06.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:05 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:05.816+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.329978+0000 front 2024-04-21T08:52:24.329957+0000 (oldest deadline 2024-04-21T08:52:46.029966+0000) 2024-04-21T08:53:06.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:05 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:05.816+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:32.431682+0000 front 2024-04-21T08:52:32.431710+0000 (oldest deadline 2024-04-21T08:52:55.232351+0000) 2024-04-21T08:53:06.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:05 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:05.816+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:38.733223+0000 front 2024-04-21T08:52:38.733348+0000 (oldest deadline 2024-04-21T08:53:04.033231+0000) 2024-04-21T08:53:06.367 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:53:06 smithi086 podman[104935]: 2024-04-21 08:53:06.070631121 +0000 UTC m=+0.644353093 container remove b1aae3596c7df4713ea305bc587568655431c558e0a8041bc1516c51d24de328 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_CLEAN=True) 2024-04-21T08:53:06.698 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:53:06 smithi086 podman[105027]: 2024-04-21 08:53:06.285401257 +0000 UTC m=+0.017740632 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:53:06.698 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:53:06 smithi086 podman[105027]: 2024-04-21 08:53:06.429943799 +0000 UTC m=+0.162283154 container create 8504efbca22fc04f59bfe0b08eb552a5fb7ffc18f1f54321a13084a139a1613f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6-deactivate, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-21T08:53:06.698 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:53:06 smithi086 podman[105027]: 2024-04-21 08:53:06.699357117 +0000 UTC m=+0.431696468 container init 8504efbca22fc04f59bfe0b08eb552a5fb7ffc18f1f54321a13084a139a1613f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0) 2024-04-21T08:53:06.699 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:06 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:06.428+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:53:06.699 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:06 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:06.428+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:53:06.699 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:06 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:06.428+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.405223+0000 front 2024-04-21T08:52:30.405168+0000 (oldest deadline 2024-04-21T08:52:55.105127+0000) 2024-04-21T08:53:06.699 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:06 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:06.428+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:40.608170+0000 front 2024-04-21T08:52:40.608213+0000 (oldest deadline 2024-04-21T08:53:05.308338+0000) 2024-04-21T08:53:06.714 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:06 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:53:06.433+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.388905+0000 front 2024-04-21T08:52:24.388964+0000 (oldest deadline 2024-04-21T08:52:49.689097+0000) 2024-04-21T08:53:06.714 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:06 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:53:06.433+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.390238+0000 front 2024-04-21T08:52:31.390386+0000 (oldest deadline 2024-04-21T08:52:57.290277+0000) 2024-04-21T08:53:06.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:06 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:06.432+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:53:06.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:06 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:06.432+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:53:06.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:06 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:06.432+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:53:06.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:06 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:06.432+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:39.228449+0000 front 2024-04-21T08:52:39.228437+0000 (oldest deadline 2024-04-21T08:53:02.029070+0000) 2024-04-21T08:53:06.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:06 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:06.306+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:06.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:06 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:06.306+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:06.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:06 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:06.306+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:06.989 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:53:06 smithi086 podman[105027]: 2024-04-21 08:53:06.702494426 +0000 UTC m=+0.434833781 container start 8504efbca22fc04f59bfe0b08eb552a5fb7ffc18f1f54321a13084a139a1613f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6-deactivate, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:53:06.990 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:53:06 smithi086 podman[105027]: 2024-04-21 08:53:06.799239449 +0000 UTC m=+0.531578803 container attach 8504efbca22fc04f59bfe0b08eb552a5fb7ffc18f1f54321a13084a139a1613f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6-deactivate, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, GIT_BRANCH=HEAD) 2024-04-21T08:53:07.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:06 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:06.839+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:14.929128+0000 front 2024-04-21T08:52:14.929188+0000 (oldest deadline 2024-04-21T08:52:40.829047+0000) 2024-04-21T08:53:07.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:06 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:06.839+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.329978+0000 front 2024-04-21T08:52:24.329957+0000 (oldest deadline 2024-04-21T08:52:46.029966+0000) 2024-04-21T08:53:07.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:06 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:06.839+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:32.431682+0000 front 2024-04-21T08:52:32.431710+0000 (oldest deadline 2024-04-21T08:52:55.232351+0000) 2024-04-21T08:53:07.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:06 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:06.839+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:38.733223+0000 front 2024-04-21T08:52:38.733348+0000 (oldest deadline 2024-04-21T08:53:04.033231+0000) 2024-04-21T08:53:07.263 INFO:journalctl@ceph.osd.6.smithi086.stdout:Apr 21 08:53:06 smithi086 podman[105027]: 2024-04-21 08:53:06.99068549 +0000 UTC m=+0.723024859 container died 8504efbca22fc04f59bfe0b08eb552a5fb7ffc18f1f54321a13084a139a1613f (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-6-deactivate, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:53:07.610 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:07 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:07.471+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:53:07.610 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:07 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:07.471+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:53:07.610 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:07 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:07.471+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.405223+0000 front 2024-04-21T08:52:30.405168+0000 (oldest deadline 2024-04-21T08:52:55.105127+0000) 2024-04-21T08:53:07.611 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:07 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:07.471+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:40.608170+0000 front 2024-04-21T08:52:40.608213+0000 (oldest deadline 2024-04-21T08:53:05.308338+0000) 2024-04-21T08:53:07.630 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@osd.6.service' 2024-04-21T08:53:07.667 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:53:07.667 INFO:tasks.cephadm.osd.6:Stopped osd.6 2024-04-21T08:53:07.668 INFO:tasks.cephadm.osd.7:Stopping osd.7... 2024-04-21T08:53:07.668 DEBUG:teuthology.orchestra.run.smithi088:> sudo systemctl stop ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@osd.7 2024-04-21T08:53:07.693 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:07 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:53:07.412+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.388905+0000 front 2024-04-21T08:52:24.388964+0000 (oldest deadline 2024-04-21T08:52:49.689097+0000) 2024-04-21T08:53:07.693 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:07 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:53:07.412+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.390238+0000 front 2024-04-21T08:52:31.390386+0000 (oldest deadline 2024-04-21T08:52:57.290277+0000) 2024-04-21T08:53:07.693 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:07 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:53:07.412+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:37.290787+0000 front 2024-04-21T08:52:37.290736+0000 (oldest deadline 2024-04-21T08:53:07.291271+0000) 2024-04-21T08:53:07.694 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:07 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:07.479+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:53:07.694 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:07 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:07.479+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:53:07.694 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:07 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:07.479+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:53:07.694 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:07 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:07.479+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:39.228449+0000 front 2024-04-21T08:52:39.228437+0000 (oldest deadline 2024-04-21T08:53:02.029070+0000) 2024-04-21T08:53:07.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:07 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:07.354+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:07.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:07 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:07.354+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:07.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:07 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:07.354+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:07.959 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:07 smithi088 systemd[1]: Stopping Ceph osd.7 for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:53:08.214 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:07 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:53:07.960+0000 7f3aca9ab700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.7 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:53:08.214 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:07 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:53:07.960+0000 7f3aca9ab700 -1 osd.7 112 *** Got signal Terminated *** 2024-04-21T08:53:08.214 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:07 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:53:07.960+0000 7f3aca9ab700 -1 osd.7 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T08:53:08.246 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:07 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:07.800+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:14.929128+0000 front 2024-04-21T08:52:14.929188+0000 (oldest deadline 2024-04-21T08:52:40.829047+0000) 2024-04-21T08:53:08.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:07 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:07.800+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.329978+0000 front 2024-04-21T08:52:24.329957+0000 (oldest deadline 2024-04-21T08:52:46.029966+0000) 2024-04-21T08:53:08.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:07 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:07.800+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:32.431682+0000 front 2024-04-21T08:52:32.431710+0000 (oldest deadline 2024-04-21T08:52:55.232351+0000) 2024-04-21T08:53:08.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:07 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:07.800+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:38.733223+0000 front 2024-04-21T08:52:38.733348+0000 (oldest deadline 2024-04-21T08:53:04.033231+0000) 2024-04-21T08:53:08.714 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:08 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:53:08.401+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.388905+0000 front 2024-04-21T08:52:24.388964+0000 (oldest deadline 2024-04-21T08:52:49.689097+0000) 2024-04-21T08:53:08.714 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:08 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:53:08.401+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.390238+0000 front 2024-04-21T08:52:31.390386+0000 (oldest deadline 2024-04-21T08:52:57.290277+0000) 2024-04-21T08:53:08.714 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:08 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:53:08.401+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:37.290787+0000 front 2024-04-21T08:52:37.290736+0000 (oldest deadline 2024-04-21T08:53:07.291271+0000) 2024-04-21T08:53:08.715 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:08 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:08.506+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:53:08.715 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:08 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:08.506+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:53:08.715 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:08 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:08.506+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:53:08.715 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:08 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:08.506+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:39.228449+0000 front 2024-04-21T08:52:39.228437+0000 (oldest deadline 2024-04-21T08:53:02.029070+0000) 2024-04-21T08:53:08.746 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:08 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:08.363+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:08.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:08 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:08.363+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:08.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:08 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:08.363+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:08.871 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:08 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:08.454+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:53:08.899 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:08 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:08.454+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:53:08.899 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:08 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:08.454+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.405223+0000 front 2024-04-21T08:52:30.405168+0000 (oldest deadline 2024-04-21T08:52:55.105127+0000) 2024-04-21T08:53:08.899 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:08 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:08.454+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:40.608170+0000 front 2024-04-21T08:52:40.608213+0000 (oldest deadline 2024-04-21T08:53:05.308338+0000) 2024-04-21T08:53:09.246 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:08 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:08.785+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:14.929128+0000 front 2024-04-21T08:52:14.929188+0000 (oldest deadline 2024-04-21T08:52:40.829047+0000) 2024-04-21T08:53:09.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:08 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:08.785+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.329978+0000 front 2024-04-21T08:52:24.329957+0000 (oldest deadline 2024-04-21T08:52:46.029966+0000) 2024-04-21T08:53:09.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:08 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:08.785+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:32.431682+0000 front 2024-04-21T08:52:32.431710+0000 (oldest deadline 2024-04-21T08:52:55.232351+0000) 2024-04-21T08:53:09.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:08 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:08.785+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:38.733223+0000 front 2024-04-21T08:52:38.733348+0000 (oldest deadline 2024-04-21T08:53:04.033231+0000) 2024-04-21T08:53:09.714 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:09 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:53:09.394+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.388905+0000 front 2024-04-21T08:52:24.388964+0000 (oldest deadline 2024-04-21T08:52:49.689097+0000) 2024-04-21T08:53:09.714 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:09 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:53:09.394+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.390238+0000 front 2024-04-21T08:52:31.390386+0000 (oldest deadline 2024-04-21T08:52:57.290277+0000) 2024-04-21T08:53:09.714 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:09 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:53:09.394+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:37.290787+0000 front 2024-04-21T08:52:37.290736+0000 (oldest deadline 2024-04-21T08:53:07.291271+0000) 2024-04-21T08:53:09.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:09 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:09.461+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:53:09.715 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:09 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:09.461+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:53:09.715 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:09 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:09.461+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:53:09.715 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:09 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:09.461+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:39.228449+0000 front 2024-04-21T08:52:39.228437+0000 (oldest deadline 2024-04-21T08:53:02.029070+0000) 2024-04-21T08:53:09.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:09 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:09.337+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:09.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:09 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:09.337+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:09.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:09 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:09.337+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:09.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:09 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:09.425+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:53:09.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:09 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:09.425+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:53:09.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:09 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:09.425+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.405223+0000 front 2024-04-21T08:52:30.405168+0000 (oldest deadline 2024-04-21T08:52:55.105127+0000) 2024-04-21T08:53:09.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:09 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:09.425+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:40.608170+0000 front 2024-04-21T08:52:40.608213+0000 (oldest deadline 2024-04-21T08:53:05.308338+0000) 2024-04-21T08:53:10.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:09 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:09.787+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:14.929128+0000 front 2024-04-21T08:52:14.929188+0000 (oldest deadline 2024-04-21T08:52:40.829047+0000) 2024-04-21T08:53:10.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:09 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:09.787+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.329978+0000 front 2024-04-21T08:52:24.329957+0000 (oldest deadline 2024-04-21T08:52:46.029966+0000) 2024-04-21T08:53:10.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:09 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:09.787+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:32.431682+0000 front 2024-04-21T08:52:32.431710+0000 (oldest deadline 2024-04-21T08:52:55.232351+0000) 2024-04-21T08:53:10.248 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:09 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:09.787+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:38.733223+0000 front 2024-04-21T08:52:38.733348+0000 (oldest deadline 2024-04-21T08:53:04.033231+0000) 2024-04-21T08:53:10.714 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:10 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:53:10.430+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.388905+0000 front 2024-04-21T08:52:24.388964+0000 (oldest deadline 2024-04-21T08:52:49.689097+0000) 2024-04-21T08:53:10.714 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:10 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:53:10.430+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.390238+0000 front 2024-04-21T08:52:31.390386+0000 (oldest deadline 2024-04-21T08:52:57.290277+0000) 2024-04-21T08:53:10.714 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:10 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:53:10.430+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:37.290787+0000 front 2024-04-21T08:52:37.290736+0000 (oldest deadline 2024-04-21T08:53:07.291271+0000) 2024-04-21T08:53:10.715 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:10 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:10.479+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:53:10.715 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:10 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:10.479+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:53:10.715 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:10 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:10.479+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:53:10.715 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:10 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:10.479+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:39.228449+0000 front 2024-04-21T08:52:39.228437+0000 (oldest deadline 2024-04-21T08:53:02.029070+0000) 2024-04-21T08:53:10.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:10 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:10.341+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:10.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:10 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:10.341+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:10.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:10 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:10.341+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:10.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:10 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:10.379+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:53:10.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:10 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:10.379+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:53:10.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:10 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:10.379+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.405223+0000 front 2024-04-21T08:52:30.405168+0000 (oldest deadline 2024-04-21T08:52:55.105127+0000) 2024-04-21T08:53:10.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:10 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:10.379+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:40.608170+0000 front 2024-04-21T08:52:40.608213+0000 (oldest deadline 2024-04-21T08:53:05.308338+0000) 2024-04-21T08:53:11.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:10 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:10.789+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:14.929128+0000 front 2024-04-21T08:52:14.929188+0000 (oldest deadline 2024-04-21T08:52:40.829047+0000) 2024-04-21T08:53:11.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:10 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:10.789+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.329978+0000 front 2024-04-21T08:52:24.329957+0000 (oldest deadline 2024-04-21T08:52:46.029966+0000) 2024-04-21T08:53:11.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:10 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:10.789+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:32.431682+0000 front 2024-04-21T08:52:32.431710+0000 (oldest deadline 2024-04-21T08:52:55.232351+0000) 2024-04-21T08:53:11.248 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:10 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:10.789+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:38.733223+0000 front 2024-04-21T08:52:38.733348+0000 (oldest deadline 2024-04-21T08:53:04.033231+0000) 2024-04-21T08:53:11.714 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:11 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:53:11.431+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.388905+0000 front 2024-04-21T08:52:24.388964+0000 (oldest deadline 2024-04-21T08:52:49.689097+0000) 2024-04-21T08:53:11.714 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:11 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:53:11.431+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.390238+0000 front 2024-04-21T08:52:31.390386+0000 (oldest deadline 2024-04-21T08:52:57.290277+0000) 2024-04-21T08:53:11.714 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:11 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:53:11.431+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:37.290787+0000 front 2024-04-21T08:52:37.290736+0000 (oldest deadline 2024-04-21T08:53:07.291271+0000) 2024-04-21T08:53:11.714 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:11 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:53:11.431+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:47.291662+0000 front 2024-04-21T08:52:47.291792+0000 (oldest deadline 2024-04-21T08:53:11.391785+0000) 2024-04-21T08:53:11.715 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:11 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:11.500+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:53:11.715 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:11 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:11.500+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:53:11.715 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:11 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:11.500+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:53:11.715 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:11 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:11.500+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:39.228449+0000 front 2024-04-21T08:52:39.228437+0000 (oldest deadline 2024-04-21T08:53:02.029070+0000) 2024-04-21T08:53:11.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:11 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:11.350+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:11.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:11 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:11.350+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:11.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:11 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:11.350+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:11.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:11 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:11.424+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:53:11.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:11 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:11.424+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:53:11.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:11 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:11.424+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.405223+0000 front 2024-04-21T08:52:30.405168+0000 (oldest deadline 2024-04-21T08:52:55.105127+0000) 2024-04-21T08:53:11.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:11 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:11.424+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:40.608170+0000 front 2024-04-21T08:52:40.608213+0000 (oldest deadline 2024-04-21T08:53:05.308338+0000) 2024-04-21T08:53:12.246 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:11 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:11.795+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:14.929128+0000 front 2024-04-21T08:52:14.929188+0000 (oldest deadline 2024-04-21T08:52:40.829047+0000) 2024-04-21T08:53:12.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:11 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:11.795+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.329978+0000 front 2024-04-21T08:52:24.329957+0000 (oldest deadline 2024-04-21T08:52:46.029966+0000) 2024-04-21T08:53:12.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:11 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:11.795+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:32.431682+0000 front 2024-04-21T08:52:32.431710+0000 (oldest deadline 2024-04-21T08:52:55.232351+0000) 2024-04-21T08:53:12.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:11 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:11.795+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:38.733223+0000 front 2024-04-21T08:52:38.733348+0000 (oldest deadline 2024-04-21T08:53:04.033231+0000) 2024-04-21T08:53:12.714 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:12 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:53:12.434+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.388905+0000 front 2024-04-21T08:52:24.388964+0000 (oldest deadline 2024-04-21T08:52:49.689097+0000) 2024-04-21T08:53:12.714 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:12 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:53:12.434+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.390238+0000 front 2024-04-21T08:52:31.390386+0000 (oldest deadline 2024-04-21T08:52:57.290277+0000) 2024-04-21T08:53:12.714 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:12 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:53:12.434+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:37.290787+0000 front 2024-04-21T08:52:37.290736+0000 (oldest deadline 2024-04-21T08:53:07.291271+0000) 2024-04-21T08:53:12.714 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:12 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7[49390]: 2024-04-21T08:53:12.434+0000 7f3ac6f9f700 -1 osd.7 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:47.291662+0000 front 2024-04-21T08:52:47.291792+0000 (oldest deadline 2024-04-21T08:53:11.391785+0000) 2024-04-21T08:53:12.715 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:12 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:12.523+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:53:12.715 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:12 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:12.523+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:53:12.715 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:12 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:12.523+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:53:12.715 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:12 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:12.523+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:39.228449+0000 front 2024-04-21T08:52:39.228437+0000 (oldest deadline 2024-04-21T08:53:02.029070+0000) 2024-04-21T08:53:12.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:12 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:12.358+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:12.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:12 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:12.358+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:12.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:12 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:12.358+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:12.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:12 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:12.420+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:53:12.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:12 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:12.420+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:53:12.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:12 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:12.420+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.405223+0000 front 2024-04-21T08:52:30.405168+0000 (oldest deadline 2024-04-21T08:52:55.105127+0000) 2024-04-21T08:53:12.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:12 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:12.420+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:40.608170+0000 front 2024-04-21T08:52:40.608213+0000 (oldest deadline 2024-04-21T08:53:05.308338+0000) 2024-04-21T08:53:13.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:12 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:12.834+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:14.929128+0000 front 2024-04-21T08:52:14.929188+0000 (oldest deadline 2024-04-21T08:52:40.829047+0000) 2024-04-21T08:53:13.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:12 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:12.834+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.329978+0000 front 2024-04-21T08:52:24.329957+0000 (oldest deadline 2024-04-21T08:52:46.029966+0000) 2024-04-21T08:53:13.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:12 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:12.834+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:32.431682+0000 front 2024-04-21T08:52:32.431710+0000 (oldest deadline 2024-04-21T08:52:55.232351+0000) 2024-04-21T08:53:13.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:12 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:12.834+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:38.733223+0000 front 2024-04-21T08:52:38.733348+0000 (oldest deadline 2024-04-21T08:53:04.033231+0000) 2024-04-21T08:53:13.602 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:13 smithi088 podman[68128]: 2024-04-21 08:53:13.343479053 +0000 UTC m=+5.486843169 container stop b2654b41ae160942fdb131bc2431ac0dc01bd6ccafa4766ffb185f29ab7e8f36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, CEPH_POINT_RELEASE=) 2024-04-21T08:53:13.602 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:13 smithi088 podman[68128]: 2024-04-21 08:53:13.3714032 +0000 UTC m=+5.514767395 container died b2654b41ae160942fdb131bc2431ac0dc01bd6ccafa4766ffb185f29ab7e8f36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux ) 2024-04-21T08:53:13.603 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:13 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:13.497+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:53:13.603 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:13 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:13.497+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:53:13.603 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:13 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:13.497+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:53:13.603 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:13 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:13.497+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:39.228449+0000 front 2024-04-21T08:52:39.228437+0000 (oldest deadline 2024-04-21T08:53:02.029070+0000) 2024-04-21T08:53:13.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:13 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:13.346+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:13.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:13 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:13.346+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:13.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:13 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:13.346+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:13.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:13 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:13.409+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:53:13.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:13 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:13.409+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:53:13.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:13 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:13.409+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.405223+0000 front 2024-04-21T08:52:30.405168+0000 (oldest deadline 2024-04-21T08:52:55.105127+0000) 2024-04-21T08:53:13.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:13 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:13.409+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:40.608170+0000 front 2024-04-21T08:52:40.608213+0000 (oldest deadline 2024-04-21T08:53:05.308338+0000) 2024-04-21T08:53:13.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:13 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:13.409+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.709851+0000 front 2024-04-21T08:52:48.710000+0000 (oldest deadline 2024-04-21T08:53:13.409648+0000) 2024-04-21T08:53:13.966 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:13 smithi088 podman[68128]: 2024-04-21 08:53:13.685094882 +0000 UTC m=+5.828458995 container cleanup b2654b41ae160942fdb131bc2431ac0dc01bd6ccafa4766ffb185f29ab7e8f36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7, GIT_CLEAN=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-21T08:53:13.966 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:13 smithi088 bash[68128]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7 2024-04-21T08:53:13.967 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:13 smithi088 podman[68157]: 2024-04-21 08:53:13.819812738 +0000 UTC m=+0.470371836 container remove b2654b41ae160942fdb131bc2431ac0dc01bd6ccafa4766ffb185f29ab7e8f36 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:53:14.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:13 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:13.842+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:14.929128+0000 front 2024-04-21T08:52:14.929188+0000 (oldest deadline 2024-04-21T08:52:40.829047+0000) 2024-04-21T08:53:14.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:13 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:13.842+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.329978+0000 front 2024-04-21T08:52:24.329957+0000 (oldest deadline 2024-04-21T08:52:46.029966+0000) 2024-04-21T08:53:14.248 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:13 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:13.842+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:32.431682+0000 front 2024-04-21T08:52:32.431710+0000 (oldest deadline 2024-04-21T08:52:55.232351+0000) 2024-04-21T08:53:14.248 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:13 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:13.842+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:38.733223+0000 front 2024-04-21T08:52:38.733348+0000 (oldest deadline 2024-04-21T08:53:04.033231+0000) 2024-04-21T08:53:14.498 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:14 smithi088 podman[68249]: 2024-04-21 08:53:14.128054381 +0000 UTC m=+0.017619139 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:53:14.498 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:14 smithi088 podman[68249]: 2024-04-21 08:53:14.254100884 +0000 UTC m=+0.143665678 container create 4d59ed551b7793908baf85fd25c20067fc62ab28a656f27bd5b95cf37e85a286 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7-deactivate, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_CLEAN=True, CEPH_POINT_RELEASE=, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:53:14.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:14 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:14.349+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:14.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:14 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:14.349+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:14.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:14 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:14.349+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:14.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:14 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:14.349+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.571785+0000 front 2024-04-21T08:52:48.571730+0000 (oldest deadline 2024-04-21T08:53:13.871627+0000) 2024-04-21T08:53:14.788 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:14 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:14.526+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:53:14.788 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:14 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:14.526+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:53:14.788 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:14 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:14.526+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:53:14.788 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:14 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:14.526+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:39.228449+0000 front 2024-04-21T08:52:39.228437+0000 (oldest deadline 2024-04-21T08:53:02.029070+0000) 2024-04-21T08:53:14.788 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:14 smithi088 podman[68249]: 2024-04-21 08:53:14.49913923 +0000 UTC m=+0.388703979 container init 4d59ed551b7793908baf85fd25c20067fc62ab28a656f27bd5b95cf37e85a286 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7-deactivate, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2) 2024-04-21T08:53:14.788 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:14 smithi088 podman[68249]: 2024-04-21 08:53:14.502569786 +0000 UTC m=+0.392134536 container start 4d59ed551b7793908baf85fd25c20067fc62ab28a656f27bd5b95cf37e85a286 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7-deactivate, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux ) 2024-04-21T08:53:14.788 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:14 smithi088 podman[68249]: 2024-04-21 08:53:14.556459223 +0000 UTC m=+0.446023981 container attach 4d59ed551b7793908baf85fd25c20067fc62ab28a656f27bd5b95cf37e85a286 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7-deactivate, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, GIT_CLEAN=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:53:14.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:14 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:14.377+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:53:14.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:14 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:14.377+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:53:14.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:14 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:14.377+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.405223+0000 front 2024-04-21T08:52:30.405168+0000 (oldest deadline 2024-04-21T08:52:55.105127+0000) 2024-04-21T08:53:14.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:14 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:14.377+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:40.608170+0000 front 2024-04-21T08:52:40.608213+0000 (oldest deadline 2024-04-21T08:53:05.308338+0000) 2024-04-21T08:53:14.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:14 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:14.377+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.709851+0000 front 2024-04-21T08:52:48.710000+0000 (oldest deadline 2024-04-21T08:53:13.409648+0000) 2024-04-21T08:53:15.075 INFO:journalctl@ceph.osd.7.smithi088.stdout:Apr 21 08:53:14 smithi088 podman[68249]: 2024-04-21 08:53:14.789176799 +0000 UTC m=+0.678741556 container died 4d59ed551b7793908baf85fd25c20067fc62ab28a656f27bd5b95cf37e85a286 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-7-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, ceph=True, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5) 2024-04-21T08:53:15.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:14 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:14.801+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:14.929128+0000 front 2024-04-21T08:52:14.929188+0000 (oldest deadline 2024-04-21T08:52:40.829047+0000) 2024-04-21T08:53:15.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:14 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:14.801+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.329978+0000 front 2024-04-21T08:52:24.329957+0000 (oldest deadline 2024-04-21T08:52:46.029966+0000) 2024-04-21T08:53:15.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:14 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:14.801+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:32.431682+0000 front 2024-04-21T08:52:32.431710+0000 (oldest deadline 2024-04-21T08:52:55.232351+0000) 2024-04-21T08:53:15.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:14 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:14.801+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:38.733223+0000 front 2024-04-21T08:52:38.733348+0000 (oldest deadline 2024-04-21T08:53:04.033231+0000) 2024-04-21T08:53:15.410 DEBUG:teuthology.orchestra.run.smithi088:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@osd.7.service' 2024-04-21T08:53:15.487 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:53:15.488 INFO:tasks.cephadm.osd.7:Stopped osd.7 2024-04-21T08:53:15.488 INFO:tasks.cephadm.osd.8:Stopping osd.8... 2024-04-21T08:53:15.488 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@osd.8 2024-04-21T08:53:15.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:15 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:15.561+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:53:15.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:15 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:15.561+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:53:15.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:15 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:15.561+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:53:15.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:15 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:15.561+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:39.228449+0000 front 2024-04-21T08:52:39.228437+0000 (oldest deadline 2024-04-21T08:53:02.029070+0000) 2024-04-21T08:53:15.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:15 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:15.561+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:46.129972+0000 front 2024-04-21T08:52:46.130009+0000 (oldest deadline 2024-04-21T08:53:15.530722+0000) 2024-04-21T08:53:15.746 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:15 smithi179 systemd[1]: Stopping Ceph osd.8 for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:53:15.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:15 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:15.304+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:15.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:15 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:15.304+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:15.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:15 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:15.304+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:15.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:15 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:15.304+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.571785+0000 front 2024-04-21T08:52:48.571730+0000 (oldest deadline 2024-04-21T08:53:13.871627+0000) 2024-04-21T08:53:15.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:15 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:15.401+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:53:15.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:15 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:15.401+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:53:15.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:15 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:15.401+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.405223+0000 front 2024-04-21T08:52:30.405168+0000 (oldest deadline 2024-04-21T08:52:55.105127+0000) 2024-04-21T08:53:15.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:15 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:15.401+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:40.608170+0000 front 2024-04-21T08:52:40.608213+0000 (oldest deadline 2024-04-21T08:53:05.308338+0000) 2024-04-21T08:53:15.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:15 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:15.401+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.709851+0000 front 2024-04-21T08:52:48.710000+0000 (oldest deadline 2024-04-21T08:53:13.409648+0000) 2024-04-21T08:53:16.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:15 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:15.824+0000 7f2202568700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.8 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:53:16.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:15 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:15.824+0000 7f2202568700 -1 osd.8 112 *** Got signal Terminated *** 2024-04-21T08:53:16.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:15 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:15.824+0000 7f2202568700 -1 osd.8 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T08:53:16.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:15 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:15.832+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:14.929128+0000 front 2024-04-21T08:52:14.929188+0000 (oldest deadline 2024-04-21T08:52:40.829047+0000) 2024-04-21T08:53:16.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:15 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:15.832+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.329978+0000 front 2024-04-21T08:52:24.329957+0000 (oldest deadline 2024-04-21T08:52:46.029966+0000) 2024-04-21T08:53:16.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:15 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:15.832+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:32.431682+0000 front 2024-04-21T08:52:32.431710+0000 (oldest deadline 2024-04-21T08:52:55.232351+0000) 2024-04-21T08:53:16.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:15 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:15.832+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:38.733223+0000 front 2024-04-21T08:52:38.733348+0000 (oldest deadline 2024-04-21T08:53:04.033231+0000) 2024-04-21T08:53:16.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:16 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:16.289+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:16.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:16 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:16.289+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:16.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:16 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:16.289+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:16.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:16 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:16.289+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.571785+0000 front 2024-04-21T08:52:48.571730+0000 (oldest deadline 2024-04-21T08:53:13.871627+0000) 2024-04-21T08:53:16.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:16 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:16.451+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:53:16.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:16 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:16.451+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:53:16.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:16 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:16.451+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.405223+0000 front 2024-04-21T08:52:30.405168+0000 (oldest deadline 2024-04-21T08:52:55.105127+0000) 2024-04-21T08:53:16.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:16 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:16.451+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:40.608170+0000 front 2024-04-21T08:52:40.608213+0000 (oldest deadline 2024-04-21T08:53:05.308338+0000) 2024-04-21T08:53:16.867 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:16 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:16.451+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.709851+0000 front 2024-04-21T08:52:48.710000+0000 (oldest deadline 2024-04-21T08:53:13.409648+0000) 2024-04-21T08:53:16.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:16 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:16.581+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:53:16.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:16 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:16.581+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:53:16.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:16 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:16.581+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:53:16.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:16 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:16.581+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:39.228449+0000 front 2024-04-21T08:52:39.228437+0000 (oldest deadline 2024-04-21T08:53:02.029070+0000) 2024-04-21T08:53:16.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:16 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:16.581+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:46.129972+0000 front 2024-04-21T08:52:46.130009+0000 (oldest deadline 2024-04-21T08:53:15.530722+0000) 2024-04-21T08:53:17.246 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:16 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:16.855+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:14.929128+0000 front 2024-04-21T08:52:14.929188+0000 (oldest deadline 2024-04-21T08:52:40.829047+0000) 2024-04-21T08:53:17.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:16 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:16.855+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.329978+0000 front 2024-04-21T08:52:24.329957+0000 (oldest deadline 2024-04-21T08:52:46.029966+0000) 2024-04-21T08:53:17.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:16 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:16.855+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:32.431682+0000 front 2024-04-21T08:52:32.431710+0000 (oldest deadline 2024-04-21T08:52:55.232351+0000) 2024-04-21T08:53:17.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:16 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:16.855+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:38.733223+0000 front 2024-04-21T08:52:38.733348+0000 (oldest deadline 2024-04-21T08:53:04.033231+0000) 2024-04-21T08:53:17.699 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:17 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:17.411+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:53:17.700 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:17 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:17.411+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:53:17.700 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:17 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:17.411+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.405223+0000 front 2024-04-21T08:52:30.405168+0000 (oldest deadline 2024-04-21T08:52:55.105127+0000) 2024-04-21T08:53:17.700 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:17 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:17.411+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:40.608170+0000 front 2024-04-21T08:52:40.608213+0000 (oldest deadline 2024-04-21T08:53:05.308338+0000) 2024-04-21T08:53:17.700 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:17 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:17.411+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.709851+0000 front 2024-04-21T08:52:48.710000+0000 (oldest deadline 2024-04-21T08:53:13.409648+0000) 2024-04-21T08:53:17.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:17 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:17.290+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:17.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:17 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:17.290+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:17.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:17 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:17.290+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:17.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:17 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:17.290+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.571785+0000 front 2024-04-21T08:52:48.571730+0000 (oldest deadline 2024-04-21T08:53:13.871627+0000) 2024-04-21T08:53:17.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:17 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:17.574+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:53:17.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:17 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:17.574+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:53:17.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:17 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:17.574+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:53:17.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:17 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:17.574+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:39.228449+0000 front 2024-04-21T08:52:39.228437+0000 (oldest deadline 2024-04-21T08:53:02.029070+0000) 2024-04-21T08:53:17.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:17 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:17.574+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:46.129972+0000 front 2024-04-21T08:52:46.130009+0000 (oldest deadline 2024-04-21T08:53:15.530722+0000) 2024-04-21T08:53:18.246 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:17 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:17.853+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:14.929128+0000 front 2024-04-21T08:52:14.929188+0000 (oldest deadline 2024-04-21T08:52:40.829047+0000) 2024-04-21T08:53:18.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:17 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:17.853+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.329978+0000 front 2024-04-21T08:52:24.329957+0000 (oldest deadline 2024-04-21T08:52:46.029966+0000) 2024-04-21T08:53:18.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:17 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:17.853+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:32.431682+0000 front 2024-04-21T08:52:32.431710+0000 (oldest deadline 2024-04-21T08:52:55.232351+0000) 2024-04-21T08:53:18.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:17 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:17.853+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:38.733223+0000 front 2024-04-21T08:52:38.733348+0000 (oldest deadline 2024-04-21T08:53:04.033231+0000) 2024-04-21T08:53:18.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:18 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:18.367+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:53:18.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:18 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:18.367+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:53:18.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:18 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:18.367+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.405223+0000 front 2024-04-21T08:52:30.405168+0000 (oldest deadline 2024-04-21T08:52:55.105127+0000) 2024-04-21T08:53:18.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:18 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:18.367+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:40.608170+0000 front 2024-04-21T08:52:40.608213+0000 (oldest deadline 2024-04-21T08:53:05.308338+0000) 2024-04-21T08:53:18.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:18 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:18.367+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.709851+0000 front 2024-04-21T08:52:48.710000+0000 (oldest deadline 2024-04-21T08:53:13.409648+0000) 2024-04-21T08:53:18.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:18 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:18.315+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:18.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:18 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:18.315+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:18.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:18 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:18.315+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:18.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:18 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:18.315+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.571785+0000 front 2024-04-21T08:52:48.571730+0000 (oldest deadline 2024-04-21T08:53:13.871627+0000) 2024-04-21T08:53:18.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:18 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:18.612+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:53:18.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:18 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:18.612+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:53:18.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:18 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:18.612+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:53:18.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:18 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:18.612+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:39.228449+0000 front 2024-04-21T08:52:39.228437+0000 (oldest deadline 2024-04-21T08:53:02.029070+0000) 2024-04-21T08:53:18.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:18 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:18.613+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:46.129972+0000 front 2024-04-21T08:52:46.130009+0000 (oldest deadline 2024-04-21T08:53:15.530722+0000) 2024-04-21T08:53:19.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:18 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:18.847+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:14.929128+0000 front 2024-04-21T08:52:14.929188+0000 (oldest deadline 2024-04-21T08:52:40.829047+0000) 2024-04-21T08:53:19.248 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:18 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:18.847+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.329978+0000 front 2024-04-21T08:52:24.329957+0000 (oldest deadline 2024-04-21T08:52:46.029966+0000) 2024-04-21T08:53:19.248 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:18 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:18.847+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:32.431682+0000 front 2024-04-21T08:52:32.431710+0000 (oldest deadline 2024-04-21T08:52:55.232351+0000) 2024-04-21T08:53:19.248 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:18 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:18.847+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:38.733223+0000 front 2024-04-21T08:52:38.733348+0000 (oldest deadline 2024-04-21T08:53:04.033231+0000) 2024-04-21T08:53:19.248 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:18 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:18.847+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:52.134810+0000 front 2024-04-21T08:52:52.134848+0000 (oldest deadline 2024-04-21T08:53:18.034899+0000) 2024-04-21T08:53:19.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:19 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:19.322+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:53:19.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:19 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:19.322+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:53:19.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:19 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:19.322+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.405223+0000 front 2024-04-21T08:52:30.405168+0000 (oldest deadline 2024-04-21T08:52:55.105127+0000) 2024-04-21T08:53:19.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:19 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:19.322+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:40.608170+0000 front 2024-04-21T08:52:40.608213+0000 (oldest deadline 2024-04-21T08:53:05.308338+0000) 2024-04-21T08:53:19.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:19 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:19.322+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.709851+0000 front 2024-04-21T08:52:48.710000+0000 (oldest deadline 2024-04-21T08:53:13.409648+0000) 2024-04-21T08:53:19.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:19 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:19.322+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:53.410665+0000 front 2024-04-21T08:52:53.410445+0000 (oldest deadline 2024-04-21T08:53:19.310061+0000) 2024-04-21T08:53:19.746 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:19 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:19.335+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:19.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:19 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:19.335+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:19.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:19 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:19.335+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:19.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:19 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:19.335+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.571785+0000 front 2024-04-21T08:52:48.571730+0000 (oldest deadline 2024-04-21T08:53:13.871627+0000) 2024-04-21T08:53:19.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:19 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:19.651+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:53:19.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:19 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:19.651+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:53:19.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:19 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:19.651+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:53:19.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:19 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:19.651+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:39.228449+0000 front 2024-04-21T08:52:39.228437+0000 (oldest deadline 2024-04-21T08:53:02.029070+0000) 2024-04-21T08:53:19.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:19 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:19.651+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:46.129972+0000 front 2024-04-21T08:52:46.130009+0000 (oldest deadline 2024-04-21T08:53:15.530722+0000) 2024-04-21T08:53:20.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:19 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:19.858+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:14.929128+0000 front 2024-04-21T08:52:14.929188+0000 (oldest deadline 2024-04-21T08:52:40.829047+0000) 2024-04-21T08:53:20.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:19 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:19.858+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.329978+0000 front 2024-04-21T08:52:24.329957+0000 (oldest deadline 2024-04-21T08:52:46.029966+0000) 2024-04-21T08:53:20.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:19 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:19.858+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:32.431682+0000 front 2024-04-21T08:52:32.431710+0000 (oldest deadline 2024-04-21T08:52:55.232351+0000) 2024-04-21T08:53:20.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:19 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:19.858+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:38.733223+0000 front 2024-04-21T08:52:38.733348+0000 (oldest deadline 2024-04-21T08:53:04.033231+0000) 2024-04-21T08:53:20.247 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:19 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8[50036]: 2024-04-21T08:53:19.858+0000 7f21feb5c700 -1 osd.8 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:52.134810+0000 front 2024-04-21T08:52:52.134848+0000 (oldest deadline 2024-04-21T08:53:18.034899+0000) 2024-04-21T08:53:20.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:20 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:20.297+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:53:20.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:20 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:20.297+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:53:20.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:20 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:20.297+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.405223+0000 front 2024-04-21T08:52:30.405168+0000 (oldest deadline 2024-04-21T08:52:55.105127+0000) 2024-04-21T08:53:20.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:20 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:20.297+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:40.608170+0000 front 2024-04-21T08:52:40.608213+0000 (oldest deadline 2024-04-21T08:53:05.308338+0000) 2024-04-21T08:53:20.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:20 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:20.297+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.709851+0000 front 2024-04-21T08:52:48.710000+0000 (oldest deadline 2024-04-21T08:53:13.409648+0000) 2024-04-21T08:53:20.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:20 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:20.297+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:53.410665+0000 front 2024-04-21T08:52:53.410445+0000 (oldest deadline 2024-04-21T08:53:19.310061+0000) 2024-04-21T08:53:20.746 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:20 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:20.345+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:20.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:20 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:20.345+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:20.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:20 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:20.345+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:20.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:20 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:20.345+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.571785+0000 front 2024-04-21T08:52:48.571730+0000 (oldest deadline 2024-04-21T08:53:13.871627+0000) 2024-04-21T08:53:20.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:20 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:20.345+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:53.872016+0000 front 2024-04-21T08:52:53.872152+0000 (oldest deadline 2024-04-21T08:53:19.772130+0000) 2024-04-21T08:53:20.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:20 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:20.671+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:53:20.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:20 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:20.671+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:53:20.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:20 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:20.671+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:53:20.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:20 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:20.671+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:39.228449+0000 front 2024-04-21T08:52:39.228437+0000 (oldest deadline 2024-04-21T08:53:02.029070+0000) 2024-04-21T08:53:20.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:20 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:20.671+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:46.129972+0000 front 2024-04-21T08:52:46.130009+0000 (oldest deadline 2024-04-21T08:53:15.530722+0000) 2024-04-21T08:53:21.466 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:21 smithi179 podman[69085]: 2024-04-21 08:53:21.158742696 +0000 UTC m=+5.463851998 container died 7239adc6ea73517253f3e0bd49c683a926af655dd1090670d909e845bca5e53e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, org.label-schema.build-date=20240416) 2024-04-21T08:53:21.466 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:21 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:21.365+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:21.466 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:21 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:21.365+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:21.466 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:21 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:21.365+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:21.466 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:21 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:21.365+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.571785+0000 front 2024-04-21T08:52:48.571730+0000 (oldest deadline 2024-04-21T08:53:13.871627+0000) 2024-04-21T08:53:21.466 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:21 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:21.365+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:53.872016+0000 front 2024-04-21T08:52:53.872152+0000 (oldest deadline 2024-04-21T08:53:19.772130+0000) 2024-04-21T08:53:21.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:21 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:21.298+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:53:21.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:21 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:21.298+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:53:21.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:21 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:21.298+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.405223+0000 front 2024-04-21T08:52:30.405168+0000 (oldest deadline 2024-04-21T08:52:55.105127+0000) 2024-04-21T08:53:21.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:21 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:21.298+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:40.608170+0000 front 2024-04-21T08:52:40.608213+0000 (oldest deadline 2024-04-21T08:53:05.308338+0000) 2024-04-21T08:53:21.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:21 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:21.298+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.709851+0000 front 2024-04-21T08:52:48.710000+0000 (oldest deadline 2024-04-21T08:53:13.409648+0000) 2024-04-21T08:53:21.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:21 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:21.298+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:53.410665+0000 front 2024-04-21T08:52:53.410445+0000 (oldest deadline 2024-04-21T08:53:19.310061+0000) 2024-04-21T08:53:21.747 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:21 smithi179 podman[69085]: 2024-04-21 08:53:21.55761504 +0000 UTC m=+5.862724334 container cleanup 7239adc6ea73517253f3e0bd49c683a926af655dd1090670d909e845bca5e53e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True) 2024-04-21T08:53:21.747 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:21 smithi179 bash[69085]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8 2024-04-21T08:53:21.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:21 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:21.653+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:53:21.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:21 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:21.653+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:53:21.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:21 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:21.653+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:53:21.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:21 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:21.653+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:39.228449+0000 front 2024-04-21T08:52:39.228437+0000 (oldest deadline 2024-04-21T08:53:02.029070+0000) 2024-04-21T08:53:21.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:21 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:21.653+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:46.129972+0000 front 2024-04-21T08:52:46.130009+0000 (oldest deadline 2024-04-21T08:53:15.530722+0000) 2024-04-21T08:53:21.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:21 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:21.653+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:55.531438+0000 front 2024-04-21T08:52:55.531448+0000 (oldest deadline 2024-04-21T08:53:21.331779+0000) 2024-04-21T08:53:22.128 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:21 smithi179 podman[69103]: 2024-04-21 08:53:21.800909161 +0000 UTC m=+0.638180245 container remove 7239adc6ea73517253f3e0bd49c683a926af655dd1090670d909e845bca5e53e (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0) 2024-04-21T08:53:22.454 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:22 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:22.367+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:22.454 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:22 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:22.367+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:22.454 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:22 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:22.367+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:22.454 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:22 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:22.367+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.571785+0000 front 2024-04-21T08:52:48.571730+0000 (oldest deadline 2024-04-21T08:53:13.871627+0000) 2024-04-21T08:53:22.454 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:22 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:22.367+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:53.872016+0000 front 2024-04-21T08:52:53.872152+0000 (oldest deadline 2024-04-21T08:53:19.772130+0000) 2024-04-21T08:53:22.455 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:22 smithi179 podman[69197]: 2024-04-21 08:53:22.032430679 +0000 UTC m=+0.018034264 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:53:22.455 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:22 smithi179 podman[69197]: 2024-04-21 08:53:22.168574559 +0000 UTC m=+0.154178188 container create 1ca5c529a57672cb7b5011b945597f7711deff9f6b75792ad108d4071fca1b42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0) 2024-04-21T08:53:22.455 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:22 smithi179 podman[69197]: 2024-04-21 08:53:22.454920407 +0000 UTC m=+0.440523986 container init 1ca5c529a57672cb7b5011b945597f7711deff9f6b75792ad108d4071fca1b42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:53:22.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:22 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:22.257+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:53:22.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:22 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:22.257+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:53:22.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:22 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:22.257+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.405223+0000 front 2024-04-21T08:52:30.405168+0000 (oldest deadline 2024-04-21T08:52:55.105127+0000) 2024-04-21T08:53:22.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:22 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:22.257+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:40.608170+0000 front 2024-04-21T08:52:40.608213+0000 (oldest deadline 2024-04-21T08:53:05.308338+0000) 2024-04-21T08:53:22.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:22 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:22.257+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.709851+0000 front 2024-04-21T08:52:48.710000+0000 (oldest deadline 2024-04-21T08:53:13.409648+0000) 2024-04-21T08:53:22.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:22 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:22.257+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:53.410665+0000 front 2024-04-21T08:52:53.410445+0000 (oldest deadline 2024-04-21T08:53:19.310061+0000) 2024-04-21T08:53:22.747 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:22 smithi179 podman[69197]: 2024-04-21 08:53:22.458511459 +0000 UTC m=+0.444115036 container start 1ca5c529a57672cb7b5011b945597f7711deff9f6b75792ad108d4071fca1b42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8-deactivate, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, ceph=True, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-21T08:53:22.747 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:22 smithi179 podman[69197]: 2024-04-21 08:53:22.563211995 +0000 UTC m=+0.548815581 container attach 1ca5c529a57672cb7b5011b945597f7711deff9f6b75792ad108d4071fca1b42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8-deactivate, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_BRANCH=HEAD) 2024-04-21T08:53:22.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:22 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:22.662+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:53:22.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:22 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:22.662+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:53:22.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:22 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:22.662+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:53:22.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:22 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:22.662+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:39.228449+0000 front 2024-04-21T08:52:39.228437+0000 (oldest deadline 2024-04-21T08:53:02.029070+0000) 2024-04-21T08:53:22.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:22 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:22.662+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:46.129972+0000 front 2024-04-21T08:52:46.130009+0000 (oldest deadline 2024-04-21T08:53:15.530722+0000) 2024-04-21T08:53:22.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:22 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:22.662+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:55.531438+0000 front 2024-04-21T08:52:55.531448+0000 (oldest deadline 2024-04-21T08:53:21.331779+0000) 2024-04-21T08:53:23.032 INFO:journalctl@ceph.osd.8.smithi179.stdout:Apr 21 08:53:22 smithi179 podman[69197]: 2024-04-21 08:53:22.751739067 +0000 UTC m=+0.737342629 container died 1ca5c529a57672cb7b5011b945597f7711deff9f6b75792ad108d4071fca1b42 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-8-deactivate, GIT_BRANCH=HEAD, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416) 2024-04-21T08:53:23.355 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@osd.8.service' 2024-04-21T08:53:23.393 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:53:23.393 INFO:tasks.cephadm.osd.8:Stopped osd.8 2024-04-21T08:53:23.393 INFO:tasks.cephadm.osd.9:Stopping osd.9... 2024-04-21T08:53:23.394 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@osd.9 2024-04-21T08:53:23.603 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:23 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:23.322+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:23.603 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:23 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:23.322+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:23.603 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:23 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:23.322+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:23.603 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:23 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:23.322+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.571785+0000 front 2024-04-21T08:52:48.571730+0000 (oldest deadline 2024-04-21T08:53:13.871627+0000) 2024-04-21T08:53:23.603 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:23 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:23.322+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:53.872016+0000 front 2024-04-21T08:52:53.872152+0000 (oldest deadline 2024-04-21T08:53:19.772130+0000) 2024-04-21T08:53:23.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:23 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:23.236+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:53:23.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:23 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:23.236+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:53:23.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:23 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:23.236+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.405223+0000 front 2024-04-21T08:52:30.405168+0000 (oldest deadline 2024-04-21T08:52:55.105127+0000) 2024-04-21T08:53:23.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:23 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:23.236+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:40.608170+0000 front 2024-04-21T08:52:40.608213+0000 (oldest deadline 2024-04-21T08:53:05.308338+0000) 2024-04-21T08:53:23.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:23 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:23.236+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.709851+0000 front 2024-04-21T08:52:48.710000+0000 (oldest deadline 2024-04-21T08:53:13.409648+0000) 2024-04-21T08:53:23.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:23 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:23.236+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:53.410665+0000 front 2024-04-21T08:52:53.410445+0000 (oldest deadline 2024-04-21T08:53:19.310061+0000) 2024-04-21T08:53:23.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:23 smithi086 systemd[1]: Stopping Ceph osd.9 for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:53:23.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:23 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:23.645+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:53:23.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:23 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:23.645+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:53:23.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:23 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:23.645+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:53:23.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:23 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:23.645+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:39.228449+0000 front 2024-04-21T08:52:39.228437+0000 (oldest deadline 2024-04-21T08:53:02.029070+0000) 2024-04-21T08:53:23.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:23 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:23.645+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:46.129972+0000 front 2024-04-21T08:52:46.130009+0000 (oldest deadline 2024-04-21T08:53:15.530722+0000) 2024-04-21T08:53:23.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:23 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:23.645+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:55.531438+0000 front 2024-04-21T08:52:55.531448+0000 (oldest deadline 2024-04-21T08:53:21.331779+0000) 2024-04-21T08:53:24.116 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:23 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:23.688+0000 7f3575653700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.9 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:53:24.117 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:23 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:23.688+0000 7f3575653700 -1 osd.9 112 *** Got signal Terminated *** 2024-04-21T08:53:24.117 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:23 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:23.688+0000 7f3575653700 -1 osd.9 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T08:53:24.618 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:24 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:24.235+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:53:24.618 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:24 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:24.235+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:53:24.618 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:24 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:24.235+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.405223+0000 front 2024-04-21T08:52:30.405168+0000 (oldest deadline 2024-04-21T08:52:55.105127+0000) 2024-04-21T08:53:24.618 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:24 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:24.235+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:40.608170+0000 front 2024-04-21T08:52:40.608213+0000 (oldest deadline 2024-04-21T08:53:05.308338+0000) 2024-04-21T08:53:24.618 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:24 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:24.235+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.709851+0000 front 2024-04-21T08:52:48.710000+0000 (oldest deadline 2024-04-21T08:53:13.409648+0000) 2024-04-21T08:53:24.618 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:24 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:24.235+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:53.410665+0000 front 2024-04-21T08:52:53.410445+0000 (oldest deadline 2024-04-21T08:53:19.310061+0000) 2024-04-21T08:53:24.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:24 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:24.345+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:24.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:24 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:24.345+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:24.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:24 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:24.345+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:24.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:24 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:24.345+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.571785+0000 front 2024-04-21T08:52:48.571730+0000 (oldest deadline 2024-04-21T08:53:13.871627+0000) 2024-04-21T08:53:24.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:24 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:24.345+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:53.872016+0000 front 2024-04-21T08:52:53.872152+0000 (oldest deadline 2024-04-21T08:53:19.772130+0000) 2024-04-21T08:53:24.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:24 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:24.627+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:53:24.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:24 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:24.627+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:53:24.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:24 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:24.627+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:53:24.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:24 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:24.627+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:39.228449+0000 front 2024-04-21T08:52:39.228437+0000 (oldest deadline 2024-04-21T08:53:02.029070+0000) 2024-04-21T08:53:24.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:24 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:24.627+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:46.129972+0000 front 2024-04-21T08:52:46.130009+0000 (oldest deadline 2024-04-21T08:53:15.530722+0000) 2024-04-21T08:53:24.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:24 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:24.627+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:55.531438+0000 front 2024-04-21T08:52:55.531448+0000 (oldest deadline 2024-04-21T08:53:21.331779+0000) 2024-04-21T08:53:25.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:25 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:25.200+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:53:25.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:25 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:25.200+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:53:25.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:25 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:25.200+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.405223+0000 front 2024-04-21T08:52:30.405168+0000 (oldest deadline 2024-04-21T08:52:55.105127+0000) 2024-04-21T08:53:25.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:25 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:25.200+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:40.608170+0000 front 2024-04-21T08:52:40.608213+0000 (oldest deadline 2024-04-21T08:53:05.308338+0000) 2024-04-21T08:53:25.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:25 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:25.200+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.709851+0000 front 2024-04-21T08:52:48.710000+0000 (oldest deadline 2024-04-21T08:53:13.409648+0000) 2024-04-21T08:53:25.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:25 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:25.200+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:53.410665+0000 front 2024-04-21T08:52:53.410445+0000 (oldest deadline 2024-04-21T08:53:19.310061+0000) 2024-04-21T08:53:25.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:25 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:25.669+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:53:25.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:25 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:25.669+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:53:25.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:25 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:25.669+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:53:25.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:25 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:25.669+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:39.228449+0000 front 2024-04-21T08:52:39.228437+0000 (oldest deadline 2024-04-21T08:53:02.029070+0000) 2024-04-21T08:53:25.714 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:25 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:25.669+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:46.129972+0000 front 2024-04-21T08:52:46.130009+0000 (oldest deadline 2024-04-21T08:53:15.530722+0000) 2024-04-21T08:53:25.715 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:25 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:25.669+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:55.531438+0000 front 2024-04-21T08:52:55.531448+0000 (oldest deadline 2024-04-21T08:53:21.331779+0000) 2024-04-21T08:53:25.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:25 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:25.326+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:25.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:25 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:25.326+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:25.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:25 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:25.326+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:25.748 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:25 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:25.326+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.571785+0000 front 2024-04-21T08:52:48.571730+0000 (oldest deadline 2024-04-21T08:53:13.871627+0000) 2024-04-21T08:53:25.748 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:25 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:25.326+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:53.872016+0000 front 2024-04-21T08:52:53.872152+0000 (oldest deadline 2024-04-21T08:53:19.772130+0000) 2024-04-21T08:53:26.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:26 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:26.168+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:53:26.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:26 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:26.168+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:53:26.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:26 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:26.168+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.405223+0000 front 2024-04-21T08:52:30.405168+0000 (oldest deadline 2024-04-21T08:52:55.105127+0000) 2024-04-21T08:53:26.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:26 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:26.168+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:40.608170+0000 front 2024-04-21T08:52:40.608213+0000 (oldest deadline 2024-04-21T08:53:05.308338+0000) 2024-04-21T08:53:26.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:26 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:26.168+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.709851+0000 front 2024-04-21T08:52:48.710000+0000 (oldest deadline 2024-04-21T08:53:13.409648+0000) 2024-04-21T08:53:26.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:26 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:26.168+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:53.410665+0000 front 2024-04-21T08:52:53.410445+0000 (oldest deadline 2024-04-21T08:53:19.310061+0000) 2024-04-21T08:53:26.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:26 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:26.366+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:26.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:26 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:26.366+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:26.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:26 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:26.366+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:26.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:26 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:26.366+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.571785+0000 front 2024-04-21T08:52:48.571730+0000 (oldest deadline 2024-04-21T08:53:13.871627+0000) 2024-04-21T08:53:26.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:26 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:26.366+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:53.872016+0000 front 2024-04-21T08:52:53.872152+0000 (oldest deadline 2024-04-21T08:53:19.772130+0000) 2024-04-21T08:53:26.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:26 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:26.642+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:53:26.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:26 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:26.642+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:53:26.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:26 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:26.642+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:53:26.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:26 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:26.642+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:39.228449+0000 front 2024-04-21T08:52:39.228437+0000 (oldest deadline 2024-04-21T08:53:02.029070+0000) 2024-04-21T08:53:26.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:26 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:26.642+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:46.129972+0000 front 2024-04-21T08:52:46.130009+0000 (oldest deadline 2024-04-21T08:53:15.530722+0000) 2024-04-21T08:53:26.965 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:26 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:26.642+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:55.531438+0000 front 2024-04-21T08:52:55.531448+0000 (oldest deadline 2024-04-21T08:53:21.331779+0000) 2024-04-21T08:53:26.965 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:26 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:26.642+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-21T08:53:01.332528+0000 front 2024-04-21T08:53:01.332673+0000 (oldest deadline 2024-04-21T08:53:26.032138+0000) 2024-04-21T08:53:27.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:27 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:27.179+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:53:27.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:27 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:27.179+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:53:27.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:27 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:27.179+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.405223+0000 front 2024-04-21T08:52:30.405168+0000 (oldest deadline 2024-04-21T08:52:55.105127+0000) 2024-04-21T08:53:27.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:27 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:27.179+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:40.608170+0000 front 2024-04-21T08:52:40.608213+0000 (oldest deadline 2024-04-21T08:53:05.308338+0000) 2024-04-21T08:53:27.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:27 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:27.179+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.709851+0000 front 2024-04-21T08:52:48.710000+0000 (oldest deadline 2024-04-21T08:53:13.409648+0000) 2024-04-21T08:53:27.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:27 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:27.179+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:53.410665+0000 front 2024-04-21T08:52:53.410445+0000 (oldest deadline 2024-04-21T08:53:19.310061+0000) 2024-04-21T08:53:27.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:27 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:27.354+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:27.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:27 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:27.354+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:27.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:27 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:27.354+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:27.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:27 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:27.354+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.571785+0000 front 2024-04-21T08:52:48.571730+0000 (oldest deadline 2024-04-21T08:53:13.871627+0000) 2024-04-21T08:53:27.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:27 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:27.354+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:53.872016+0000 front 2024-04-21T08:52:53.872152+0000 (oldest deadline 2024-04-21T08:53:19.772130+0000) 2024-04-21T08:53:27.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:27 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:27.354+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-21T08:53:04.973593+0000 front 2024-04-21T08:53:04.973668+0000 (oldest deadline 2024-04-21T08:53:27.273510+0000) 2024-04-21T08:53:27.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:27 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:27.646+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:53:27.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:27 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:27.646+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:53:27.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:27 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:27.646+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:53:27.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:27 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:27.646+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:39.228449+0000 front 2024-04-21T08:52:39.228437+0000 (oldest deadline 2024-04-21T08:53:02.029070+0000) 2024-04-21T08:53:27.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:27 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:27.646+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:46.129972+0000 front 2024-04-21T08:52:46.130009+0000 (oldest deadline 2024-04-21T08:53:15.530722+0000) 2024-04-21T08:53:27.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:27 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:27.646+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:55.531438+0000 front 2024-04-21T08:52:55.531448+0000 (oldest deadline 2024-04-21T08:53:21.331779+0000) 2024-04-21T08:53:27.965 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:27 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:27.646+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-21T08:53:01.332528+0000 front 2024-04-21T08:53:01.332673+0000 (oldest deadline 2024-04-21T08:53:26.032138+0000) 2024-04-21T08:53:28.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:28 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:28.217+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.302047+0000 front 2024-04-21T08:52:12.302079+0000 (oldest deadline 2024-04-21T08:52:38.202192+0000) 2024-04-21T08:53:28.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:28 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:28.217+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:23.503353+0000 front 2024-04-21T08:52:23.503444+0000 (oldest deadline 2024-04-21T08:52:46.403395+0000) 2024-04-21T08:53:28.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:28 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:28.217+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.405223+0000 front 2024-04-21T08:52:30.405168+0000 (oldest deadline 2024-04-21T08:52:55.105127+0000) 2024-04-21T08:53:28.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:28 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:28.217+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:40.608170+0000 front 2024-04-21T08:52:40.608213+0000 (oldest deadline 2024-04-21T08:53:05.308338+0000) 2024-04-21T08:53:28.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:28 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:28.217+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.709851+0000 front 2024-04-21T08:52:48.710000+0000 (oldest deadline 2024-04-21T08:53:13.409648+0000) 2024-04-21T08:53:28.617 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:28 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9[60780]: 2024-04-21T08:53:28.217+0000 7f356dc3f700 -1 osd.9 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:53.410665+0000 front 2024-04-21T08:52:53.410445+0000 (oldest deadline 2024-04-21T08:53:19.310061+0000) 2024-04-21T08:53:28.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:28 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:28.372+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:28.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:28 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:28.372+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:28.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:28 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:28.372+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:28.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:28 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:28.372+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.571785+0000 front 2024-04-21T08:52:48.571730+0000 (oldest deadline 2024-04-21T08:53:13.871627+0000) 2024-04-21T08:53:28.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:28 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:28.372+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:53.872016+0000 front 2024-04-21T08:52:53.872152+0000 (oldest deadline 2024-04-21T08:53:19.772130+0000) 2024-04-21T08:53:28.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:28 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:28.372+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-21T08:53:04.973593+0000 front 2024-04-21T08:53:04.973668+0000 (oldest deadline 2024-04-21T08:53:27.273510+0000) 2024-04-21T08:53:28.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:28 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:28.604+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:53:28.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:28 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:28.604+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:53:28.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:28 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:28.604+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:53:28.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:28 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:28.604+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:39.228449+0000 front 2024-04-21T08:52:39.228437+0000 (oldest deadline 2024-04-21T08:53:02.029070+0000) 2024-04-21T08:53:28.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:28 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:28.604+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:46.129972+0000 front 2024-04-21T08:52:46.130009+0000 (oldest deadline 2024-04-21T08:53:15.530722+0000) 2024-04-21T08:53:28.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:28 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:28.604+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:55.531438+0000 front 2024-04-21T08:52:55.531448+0000 (oldest deadline 2024-04-21T08:53:21.331779+0000) 2024-04-21T08:53:28.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:28 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:28.604+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-21T08:53:01.332528+0000 front 2024-04-21T08:53:01.332673+0000 (oldest deadline 2024-04-21T08:53:26.032138+0000) 2024-04-21T08:53:29.430 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:29 smithi086 podman[105145]: 2024-04-21 08:53:29.151167136 +0000 UTC m=+5.560732297 container died 7d5034b82b7d8ad4d47a971a7889a6d5ce0b9b01f437c62d6a9e74361b7c8c1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS) 2024-04-21T08:53:29.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:29 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:29.350+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:29.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:29 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:29.350+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:29.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:29 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:29.350+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:29.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:29 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:29.350+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.571785+0000 front 2024-04-21T08:52:48.571730+0000 (oldest deadline 2024-04-21T08:53:13.871627+0000) 2024-04-21T08:53:29.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:29 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:29.350+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:53.872016+0000 front 2024-04-21T08:52:53.872152+0000 (oldest deadline 2024-04-21T08:53:19.772130+0000) 2024-04-21T08:53:29.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:29 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:29.350+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-21T08:53:04.973593+0000 front 2024-04-21T08:53:04.973668+0000 (oldest deadline 2024-04-21T08:53:27.273510+0000) 2024-04-21T08:53:29.779 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:29 smithi086 podman[105145]: 2024-04-21 08:53:29.529819909 +0000 UTC m=+5.939385065 container cleanup 7d5034b82b7d8ad4d47a971a7889a6d5ce0b9b01f437c62d6a9e74361b7c8c1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, ceph=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-21T08:53:29.780 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:29 smithi086 bash[105145]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9 2024-04-21T08:53:29.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:29 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:29.609+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:53:29.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:29 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:29.609+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:53:29.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:29 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:29.609+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:53:29.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:29 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:29.609+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:39.228449+0000 front 2024-04-21T08:52:39.228437+0000 (oldest deadline 2024-04-21T08:53:02.029070+0000) 2024-04-21T08:53:29.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:29 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:29.609+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:46.129972+0000 front 2024-04-21T08:52:46.130009+0000 (oldest deadline 2024-04-21T08:53:15.530722+0000) 2024-04-21T08:53:29.965 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:29 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:29.609+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:55.531438+0000 front 2024-04-21T08:52:55.531448+0000 (oldest deadline 2024-04-21T08:53:21.331779+0000) 2024-04-21T08:53:29.965 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:29 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:29.609+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-21T08:53:01.332528+0000 front 2024-04-21T08:53:01.332673+0000 (oldest deadline 2024-04-21T08:53:26.032138+0000) 2024-04-21T08:53:30.094 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:29 smithi086 podman[105162]: 2024-04-21 08:53:29.780952148 +0000 UTC m=+0.630924457 container remove 7d5034b82b7d8ad4d47a971a7889a6d5ce0b9b01f437c62d6a9e74361b7c8c1b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:53:30.367 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:30 smithi086 podman[105254]: 2024-04-21 08:53:29.998673435 +0000 UTC m=+0.017554779 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:53:30.367 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:30 smithi086 podman[105254]: 2024-04-21 08:53:30.098312301 +0000 UTC m=+0.117193686 container create 64cc86bd5f676435413ac24c11875fe6c5f34ebe0349ab521bd0a87ccf74063d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9-deactivate, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , ceph=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:53:30.367 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:30 smithi086 podman[105254]: 2024-04-21 08:53:30.325436503 +0000 UTC m=+0.344317846 container init 64cc86bd5f676435413ac24c11875fe6c5f34ebe0349ab521bd0a87ccf74063d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, org.label-schema.vendor=CentOS) 2024-04-21T08:53:30.367 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:30 smithi086 podman[105254]: 2024-04-21 08:53:30.328547572 +0000 UTC m=+0.347428915 container start 64cc86bd5f676435413ac24c11875fe6c5f34ebe0349ab521bd0a87ccf74063d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, ceph=True, org.label-schema.vendor=CentOS) 2024-04-21T08:53:30.625 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:30 smithi086 podman[105254]: 2024-04-21 08:53:30.417394937 +0000 UTC m=+0.436276292 container attach 64cc86bd5f676435413ac24c11875fe6c5f34ebe0349ab521bd0a87ccf74063d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9-deactivate, org.label-schema.license=GPLv2, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, GIT_CLEAN=True, GIT_BRANCH=HEAD, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T08:53:30.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:30 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:30.371+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:30.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:30 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:30.371+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:30.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:30 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:30.371+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:30.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:30 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:30.371+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.571785+0000 front 2024-04-21T08:52:48.571730+0000 (oldest deadline 2024-04-21T08:53:13.871627+0000) 2024-04-21T08:53:30.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:30 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:30.371+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:53.872016+0000 front 2024-04-21T08:52:53.872152+0000 (oldest deadline 2024-04-21T08:53:19.772130+0000) 2024-04-21T08:53:30.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:30 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:30.371+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-21T08:53:04.973593+0000 front 2024-04-21T08:53:04.973668+0000 (oldest deadline 2024-04-21T08:53:27.273510+0000) 2024-04-21T08:53:30.911 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:30 smithi086 podman[105254]: 2024-04-21 08:53:30.626598701 +0000 UTC m=+0.645480054 container died 64cc86bd5f676435413ac24c11875fe6c5f34ebe0349ab521bd0a87ccf74063d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9-deactivate, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS) 2024-04-21T08:53:30.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:30 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:30.574+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:53:30.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:30 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:30.574+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:53:30.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:30 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:30.574+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:53:30.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:30 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:30.574+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:39.228449+0000 front 2024-04-21T08:52:39.228437+0000 (oldest deadline 2024-04-21T08:53:02.029070+0000) 2024-04-21T08:53:30.965 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:30 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:30.574+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:46.129972+0000 front 2024-04-21T08:52:46.130009+0000 (oldest deadline 2024-04-21T08:53:15.530722+0000) 2024-04-21T08:53:30.965 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:30 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:30.574+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:55.531438+0000 front 2024-04-21T08:52:55.531448+0000 (oldest deadline 2024-04-21T08:53:21.331779+0000) 2024-04-21T08:53:30.965 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:30 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:30.574+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-21T08:53:01.332528+0000 front 2024-04-21T08:53:01.332673+0000 (oldest deadline 2024-04-21T08:53:26.032138+0000) 2024-04-21T08:53:31.171 INFO:journalctl@ceph.osd.9.smithi086.stdout:Apr 21 08:53:31 smithi086 podman[105280]: 2024-04-21 08:53:31.145305267 +0000 UTC m=+0.513073928 container remove 64cc86bd5f676435413ac24c11875fe6c5f34ebe0349ab521bd0a87ccf74063d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-9-deactivate, GIT_BRANCH=HEAD, GIT_CLEAN=True, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True) 2024-04-21T08:53:31.177 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@osd.9.service' 2024-04-21T08:53:31.215 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:53:31.216 INFO:tasks.cephadm.osd.9:Stopped osd.9 2024-04-21T08:53:31.216 INFO:tasks.cephadm.osd.10:Stopping osd.10... 2024-04-21T08:53:31.216 DEBUG:teuthology.orchestra.run.smithi088:> sudo systemctl stop ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@osd.10 2024-04-21T08:53:31.536 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:31 smithi088 systemd[1]: Stopping Ceph osd.10 for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:53:31.746 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:31 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:31.327+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:31.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:31 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:31.327+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:31.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:31 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:31.327+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:31.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:31 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:31.327+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.571785+0000 front 2024-04-21T08:52:48.571730+0000 (oldest deadline 2024-04-21T08:53:13.871627+0000) 2024-04-21T08:53:31.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:31 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:31.327+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:53.872016+0000 front 2024-04-21T08:52:53.872152+0000 (oldest deadline 2024-04-21T08:53:19.772130+0000) 2024-04-21T08:53:31.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:31 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:31.327+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-21T08:53:04.973593+0000 front 2024-04-21T08:53:04.973668+0000 (oldest deadline 2024-04-21T08:53:27.273510+0000) 2024-04-21T08:53:31.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:31 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:31.536+0000 7fd48166f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.10 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:53:31.965 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:31 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:31.536+0000 7fd48166f700 -1 osd.10 112 *** Got signal Terminated *** 2024-04-21T08:53:31.965 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:31 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:31.536+0000 7fd48166f700 -1 osd.10 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T08:53:31.965 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:31 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:31.592+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:53:31.965 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:31 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:31.592+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:53:31.965 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:31 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:31.592+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:53:31.965 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:31 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:31.592+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:39.228449+0000 front 2024-04-21T08:52:39.228437+0000 (oldest deadline 2024-04-21T08:53:02.029070+0000) 2024-04-21T08:53:31.965 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:31 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:31.592+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:46.129972+0000 front 2024-04-21T08:52:46.130009+0000 (oldest deadline 2024-04-21T08:53:15.530722+0000) 2024-04-21T08:53:31.965 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:31 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:31.592+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:55.531438+0000 front 2024-04-21T08:52:55.531448+0000 (oldest deadline 2024-04-21T08:53:21.331779+0000) 2024-04-21T08:53:31.965 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:31 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:31.592+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-21T08:53:01.332528+0000 front 2024-04-21T08:53:01.332673+0000 (oldest deadline 2024-04-21T08:53:26.032138+0000) 2024-04-21T08:53:32.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:32 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:32.338+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:32.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:32 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:32.338+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:32.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:32 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:32.338+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:32.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:32 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:32.338+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.571785+0000 front 2024-04-21T08:52:48.571730+0000 (oldest deadline 2024-04-21T08:53:13.871627+0000) 2024-04-21T08:53:32.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:32 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:32.338+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:53.872016+0000 front 2024-04-21T08:52:53.872152+0000 (oldest deadline 2024-04-21T08:53:19.772130+0000) 2024-04-21T08:53:32.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:32 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:32.338+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-21T08:53:04.973593+0000 front 2024-04-21T08:53:04.973668+0000 (oldest deadline 2024-04-21T08:53:27.273510+0000) 2024-04-21T08:53:32.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:32 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:32.561+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:53:32.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:32 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:32.561+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:53:32.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:32 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:32.561+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:53:32.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:32 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:32.561+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:39.228449+0000 front 2024-04-21T08:52:39.228437+0000 (oldest deadline 2024-04-21T08:53:02.029070+0000) 2024-04-21T08:53:32.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:32 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:32.561+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:46.129972+0000 front 2024-04-21T08:52:46.130009+0000 (oldest deadline 2024-04-21T08:53:15.530722+0000) 2024-04-21T08:53:32.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:32 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:32.561+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:55.531438+0000 front 2024-04-21T08:52:55.531448+0000 (oldest deadline 2024-04-21T08:53:21.331779+0000) 2024-04-21T08:53:32.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:32 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:32.561+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-21T08:53:01.332528+0000 front 2024-04-21T08:53:01.332673+0000 (oldest deadline 2024-04-21T08:53:26.032138+0000) 2024-04-21T08:53:33.648 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:33 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:33.316+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:33.648 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:33 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:33.316+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:33.648 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:33 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:33.316+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:33.648 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:33 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:33.316+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.571785+0000 front 2024-04-21T08:52:48.571730+0000 (oldest deadline 2024-04-21T08:53:13.871627+0000) 2024-04-21T08:53:33.649 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:33 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:33.316+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:53.872016+0000 front 2024-04-21T08:52:53.872152+0000 (oldest deadline 2024-04-21T08:53:19.772130+0000) 2024-04-21T08:53:33.649 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:33 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:33.316+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-21T08:53:04.973593+0000 front 2024-04-21T08:53:04.973668+0000 (oldest deadline 2024-04-21T08:53:27.273510+0000) 2024-04-21T08:53:33.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:33 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:33.548+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:53:33.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:33 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:33.548+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:53:33.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:33 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:33.548+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:53:33.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:33 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:33.548+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:39.228449+0000 front 2024-04-21T08:52:39.228437+0000 (oldest deadline 2024-04-21T08:53:02.029070+0000) 2024-04-21T08:53:33.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:33 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:33.548+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:46.129972+0000 front 2024-04-21T08:52:46.130009+0000 (oldest deadline 2024-04-21T08:53:15.530722+0000) 2024-04-21T08:53:33.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:33 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:33.548+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:55.531438+0000 front 2024-04-21T08:52:55.531448+0000 (oldest deadline 2024-04-21T08:53:21.331779+0000) 2024-04-21T08:53:33.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:33 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:33.548+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-21T08:53:01.332528+0000 front 2024-04-21T08:53:01.332673+0000 (oldest deadline 2024-04-21T08:53:26.032138+0000) 2024-04-21T08:53:34.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:34 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:34.274+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:34.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:34 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:34.274+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:34.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:34 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:34.274+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:34.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:34 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:34.274+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.571785+0000 front 2024-04-21T08:52:48.571730+0000 (oldest deadline 2024-04-21T08:53:13.871627+0000) 2024-04-21T08:53:34.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:34 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:34.274+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:53.872016+0000 front 2024-04-21T08:52:53.872152+0000 (oldest deadline 2024-04-21T08:53:19.772130+0000) 2024-04-21T08:53:34.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:34 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:34.274+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-21T08:53:04.973593+0000 front 2024-04-21T08:53:04.973668+0000 (oldest deadline 2024-04-21T08:53:27.273510+0000) 2024-04-21T08:53:34.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:34 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:34.274+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6820 osd.7 since back 2024-04-21T08:53:11.675993+0000 front 2024-04-21T08:53:11.675963+0000 (oldest deadline 2024-04-21T08:53:33.375829+0000) 2024-04-21T08:53:34.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:34 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:34.587+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:53:34.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:34 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:34.587+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:53:34.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:34 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:34.587+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:53:34.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:34 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:34.587+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:39.228449+0000 front 2024-04-21T08:52:39.228437+0000 (oldest deadline 2024-04-21T08:53:02.029070+0000) 2024-04-21T08:53:34.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:34 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:34.587+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:46.129972+0000 front 2024-04-21T08:52:46.130009+0000 (oldest deadline 2024-04-21T08:53:15.530722+0000) 2024-04-21T08:53:34.965 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:34 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:34.587+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:55.531438+0000 front 2024-04-21T08:52:55.531448+0000 (oldest deadline 2024-04-21T08:53:21.331779+0000) 2024-04-21T08:53:34.965 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:34 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:34.587+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-21T08:53:01.332528+0000 front 2024-04-21T08:53:01.332673+0000 (oldest deadline 2024-04-21T08:53:26.032138+0000) 2024-04-21T08:53:35.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:35 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:35.274+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:35.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:35 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:35.274+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:35.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:35 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:35.274+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:35.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:35 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:35.274+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.571785+0000 front 2024-04-21T08:52:48.571730+0000 (oldest deadline 2024-04-21T08:53:13.871627+0000) 2024-04-21T08:53:35.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:35 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:35.274+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:53.872016+0000 front 2024-04-21T08:52:53.872152+0000 (oldest deadline 2024-04-21T08:53:19.772130+0000) 2024-04-21T08:53:35.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:35 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:35.274+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-21T08:53:04.973593+0000 front 2024-04-21T08:53:04.973668+0000 (oldest deadline 2024-04-21T08:53:27.273510+0000) 2024-04-21T08:53:35.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:35 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:35.274+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6820 osd.7 since back 2024-04-21T08:53:11.675993+0000 front 2024-04-21T08:53:11.675963+0000 (oldest deadline 2024-04-21T08:53:33.375829+0000) 2024-04-21T08:53:35.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:35 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:35.566+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:53:35.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:35 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:35.566+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:53:35.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:35 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:35.566+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:53:35.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:35 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:35.566+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:39.228449+0000 front 2024-04-21T08:52:39.228437+0000 (oldest deadline 2024-04-21T08:53:02.029070+0000) 2024-04-21T08:53:35.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:35 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:35.566+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:46.129972+0000 front 2024-04-21T08:52:46.130009+0000 (oldest deadline 2024-04-21T08:53:15.530722+0000) 2024-04-21T08:53:35.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:35 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:35.566+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:55.531438+0000 front 2024-04-21T08:52:55.531448+0000 (oldest deadline 2024-04-21T08:53:21.331779+0000) 2024-04-21T08:53:35.965 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:35 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:35.566+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-21T08:53:01.332528+0000 front 2024-04-21T08:53:01.332673+0000 (oldest deadline 2024-04-21T08:53:26.032138+0000) 2024-04-21T08:53:36.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:36 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:36.230+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:36.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:36 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:36.230+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:36.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:36 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:36.230+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:36.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:36 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:36.230+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.571785+0000 front 2024-04-21T08:52:48.571730+0000 (oldest deadline 2024-04-21T08:53:13.871627+0000) 2024-04-21T08:53:36.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:36 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:36.230+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:53.872016+0000 front 2024-04-21T08:52:53.872152+0000 (oldest deadline 2024-04-21T08:53:19.772130+0000) 2024-04-21T08:53:36.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:36 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:36.230+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-21T08:53:04.973593+0000 front 2024-04-21T08:53:04.973668+0000 (oldest deadline 2024-04-21T08:53:27.273510+0000) 2024-04-21T08:53:36.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:36 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:36.230+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6820 osd.7 since back 2024-04-21T08:53:11.675993+0000 front 2024-04-21T08:53:11.675963+0000 (oldest deadline 2024-04-21T08:53:33.375829+0000) 2024-04-21T08:53:36.864 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:36 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:36.534+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:15.424132+0000 front 2024-04-21T08:52:15.424159+0000 (oldest deadline 2024-04-21T08:52:40.724186+0000) 2024-04-21T08:53:36.864 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:36 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:36.534+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:21.225003+0000 front 2024-04-21T08:52:21.224970+0000 (oldest deadline 2024-04-21T08:52:47.125090+0000) 2024-04-21T08:53:36.864 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:36 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:36.534+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:31.627352+0000 front 2024-04-21T08:52:31.627230+0000 (oldest deadline 2024-04-21T08:52:56.927289+0000) 2024-04-21T08:53:36.864 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:36 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:36.534+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6814 osd.3 since back 2024-04-21T08:52:39.228449+0000 front 2024-04-21T08:52:39.228437+0000 (oldest deadline 2024-04-21T08:53:02.029070+0000) 2024-04-21T08:53:36.864 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:36 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:36.534+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:46.129972+0000 front 2024-04-21T08:52:46.130009+0000 (oldest deadline 2024-04-21T08:53:15.530722+0000) 2024-04-21T08:53:36.865 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:36 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:36.534+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:55.531438+0000 front 2024-04-21T08:52:55.531448+0000 (oldest deadline 2024-04-21T08:53:21.331779+0000) 2024-04-21T08:53:36.865 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:36 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10[56406]: 2024-04-21T08:53:36.534+0000 7fd47dc63700 -1 osd.10 112 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-21T08:53:01.332528+0000 front 2024-04-21T08:53:01.332673+0000 (oldest deadline 2024-04-21T08:53:26.032138+0000) 2024-04-21T08:53:37.145 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:36 smithi088 podman[68353]: 2024-04-21 08:53:36.864794037 +0000 UTC m=+5.457039558 container died df5e727d553d42f8b2fd29570ce83833c33d8cbff55cf14d1c09759ee2f707c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, maintainer=Guillaume Abrioux , GIT_CLEAN=True, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0) 2024-04-21T08:53:37.145 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:37 smithi088 podman[68353]: 2024-04-21 08:53:37.146123907 +0000 UTC m=+5.738369433 container cleanup df5e727d553d42f8b2fd29570ce83833c33d8cbff55cf14d1c09759ee2f707c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, ceph=True, org.label-schema.schema-version=1.0) 2024-04-21T08:53:37.464 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:37 smithi088 bash[68353]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10 2024-04-21T08:53:37.464 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:37 smithi088 podman[68371]: 2024-04-21 08:53:37.380518749 +0000 UTC m=+0.516527029 container remove df5e727d553d42f8b2fd29570ce83833c33d8cbff55cf14d1c09759ee2f707c7 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:53:37.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:37 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:37.233+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:37.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:37 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:37.233+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:37.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:37 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:37.233+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:37.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:37 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:37.233+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.571785+0000 front 2024-04-21T08:52:48.571730+0000 (oldest deadline 2024-04-21T08:53:13.871627+0000) 2024-04-21T08:53:37.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:37 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:37.233+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:53.872016+0000 front 2024-04-21T08:52:53.872152+0000 (oldest deadline 2024-04-21T08:53:19.772130+0000) 2024-04-21T08:53:37.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:37 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:37.233+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-21T08:53:04.973593+0000 front 2024-04-21T08:53:04.973668+0000 (oldest deadline 2024-04-21T08:53:27.273510+0000) 2024-04-21T08:53:37.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:37 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:37.233+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6820 osd.7 since back 2024-04-21T08:53:11.675993+0000 front 2024-04-21T08:53:11.675963+0000 (oldest deadline 2024-04-21T08:53:33.375829+0000) 2024-04-21T08:53:37.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:37 smithi088 podman[68461]: 2024-04-21 08:53:37.706528366 +0000 UTC m=+0.108714143 container create dc0e7139a74bf0ae86374bc784cbf48fc6cddd8e4312fac2ad83667f05eae68b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0) 2024-04-21T08:53:37.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:37 smithi088 podman[68461]: 2024-04-21 08:53:37.615423433 +0000 UTC m=+0.017609180 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:53:37.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:37 smithi088 podman[68461]: 2024-04-21 08:53:37.92532895 +0000 UTC m=+0.327514686 container init dc0e7139a74bf0ae86374bc784cbf48fc6cddd8e4312fac2ad83667f05eae68b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10-deactivate, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T08:53:37.964 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:37 smithi088 podman[68461]: 2024-04-21 08:53:37.928361994 +0000 UTC m=+0.330547735 container start dc0e7139a74bf0ae86374bc784cbf48fc6cddd8e4312fac2ad83667f05eae68b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10-deactivate, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux ) 2024-04-21T08:53:38.464 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:38 smithi088 podman[68461]: 2024-04-21 08:53:38.00901909 +0000 UTC m=+0.411204833 container attach dc0e7139a74bf0ae86374bc784cbf48fc6cddd8e4312fac2ad83667f05eae68b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10-deactivate, GIT_CLEAN=True, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, CEPH_POINT_RELEASE=, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2) 2024-04-21T08:53:38.464 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:38 smithi088 podman[68461]: 2024-04-21 08:53:38.215070458 +0000 UTC m=+0.617256194 container died dc0e7139a74bf0ae86374bc784cbf48fc6cddd8e4312fac2ad83667f05eae68b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux ) 2024-04-21T08:53:38.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:38 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:38.238+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:38.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:38 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:38.238+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:38.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:38 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:38.238+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:38.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:38 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:38.238+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.571785+0000 front 2024-04-21T08:52:48.571730+0000 (oldest deadline 2024-04-21T08:53:13.871627+0000) 2024-04-21T08:53:38.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:38 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:38.238+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:53.872016+0000 front 2024-04-21T08:52:53.872152+0000 (oldest deadline 2024-04-21T08:53:19.772130+0000) 2024-04-21T08:53:38.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:38 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:38.238+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-21T08:53:04.973593+0000 front 2024-04-21T08:53:04.973668+0000 (oldest deadline 2024-04-21T08:53:27.273510+0000) 2024-04-21T08:53:38.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:38 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:38.238+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6820 osd.7 since back 2024-04-21T08:53:11.675993+0000 front 2024-04-21T08:53:11.675963+0000 (oldest deadline 2024-04-21T08:53:33.375829+0000) 2024-04-21T08:53:38.716 DEBUG:teuthology.orchestra.run.smithi088:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@osd.10.service' 2024-04-21T08:53:38.741 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:38 smithi088 podman[68487]: 2024-04-21 08:53:38.696079581 +0000 UTC m=+0.476364953 container remove dc0e7139a74bf0ae86374bc784cbf48fc6cddd8e4312fac2ad83667f05eae68b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-10-deactivate, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, org.label-schema.schema-version=1.0) 2024-04-21T08:53:38.741 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:38 smithi088 systemd[1]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@osd.10.service: Deactivated successfully. 2024-04-21T08:53:38.741 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:38 smithi088 systemd[1]: Stopped Ceph osd.10 for f8cec7cc-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:53:38.741 INFO:journalctl@ceph.osd.10.smithi088.stdout:Apr 21 08:53:38 smithi088 systemd[1]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@osd.10.service: Consumed 15.131s CPU time. 2024-04-21T08:53:38.753 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:53:38.753 INFO:tasks.cephadm.osd.10:Stopped osd.10 2024-04-21T08:53:38.753 INFO:tasks.cephadm.osd.11:Stopping osd.11... 2024-04-21T08:53:38.753 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@osd.11 2024-04-21T08:53:39.043 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:38 smithi179 systemd[1]: Stopping Ceph osd.11 for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:53:39.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:39 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:39.043+0000 7f4a21721700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-osd -n osd.11 -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:53:39.537 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:39 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:39.043+0000 7f4a21721700 -1 osd.11 112 *** Got signal Terminated *** 2024-04-21T08:53:39.538 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:39 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:39.043+0000 7f4a21721700 -1 osd.11 112 *** Immediate shutdown (osd_fast_shutdown=true) *** 2024-04-21T08:53:39.538 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:39 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:39.288+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:39.538 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:39 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:39.288+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:39.538 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:39 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:39.288+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:39.538 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:39 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:39.288+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.571785+0000 front 2024-04-21T08:52:48.571730+0000 (oldest deadline 2024-04-21T08:53:13.871627+0000) 2024-04-21T08:53:39.538 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:39 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:39.288+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:53.872016+0000 front 2024-04-21T08:52:53.872152+0000 (oldest deadline 2024-04-21T08:53:19.772130+0000) 2024-04-21T08:53:39.538 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:39 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:39.288+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-21T08:53:04.973593+0000 front 2024-04-21T08:53:04.973668+0000 (oldest deadline 2024-04-21T08:53:27.273510+0000) 2024-04-21T08:53:39.538 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:39 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:39.288+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6820 osd.7 since back 2024-04-21T08:53:11.675993+0000 front 2024-04-21T08:53:11.675963+0000 (oldest deadline 2024-04-21T08:53:33.375829+0000) 2024-04-21T08:53:40.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:40 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:40.277+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:40.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:40 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:40.277+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:40.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:40 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:40.277+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:40.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:40 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:40.277+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.571785+0000 front 2024-04-21T08:52:48.571730+0000 (oldest deadline 2024-04-21T08:53:13.871627+0000) 2024-04-21T08:53:40.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:40 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:40.277+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:53.872016+0000 front 2024-04-21T08:52:53.872152+0000 (oldest deadline 2024-04-21T08:53:19.772130+0000) 2024-04-21T08:53:40.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:40 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:40.277+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-21T08:53:04.973593+0000 front 2024-04-21T08:53:04.973668+0000 (oldest deadline 2024-04-21T08:53:27.273510+0000) 2024-04-21T08:53:40.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:40 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:40.277+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6820 osd.7 since back 2024-04-21T08:53:11.675993+0000 front 2024-04-21T08:53:11.675963+0000 (oldest deadline 2024-04-21T08:53:33.375829+0000) 2024-04-21T08:53:41.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:41 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:41.266+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:41.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:41 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:41.266+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:41.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:41 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:41.266+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:41.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:41 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:41.266+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.571785+0000 front 2024-04-21T08:52:48.571730+0000 (oldest deadline 2024-04-21T08:53:13.871627+0000) 2024-04-21T08:53:41.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:41 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:41.266+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:53.872016+0000 front 2024-04-21T08:52:53.872152+0000 (oldest deadline 2024-04-21T08:53:19.772130+0000) 2024-04-21T08:53:41.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:41 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:41.266+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-21T08:53:04.973593+0000 front 2024-04-21T08:53:04.973668+0000 (oldest deadline 2024-04-21T08:53:27.273510+0000) 2024-04-21T08:53:41.747 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:41 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:41.266+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6820 osd.7 since back 2024-04-21T08:53:11.675993+0000 front 2024-04-21T08:53:11.675963+0000 (oldest deadline 2024-04-21T08:53:33.375829+0000) 2024-04-21T08:53:42.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:42 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:42.242+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:42.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:42 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:42.242+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:42.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:42 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:42.242+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:42.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:42 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:42.242+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.571785+0000 front 2024-04-21T08:52:48.571730+0000 (oldest deadline 2024-04-21T08:53:13.871627+0000) 2024-04-21T08:53:42.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:42 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:42.242+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:53.872016+0000 front 2024-04-21T08:52:53.872152+0000 (oldest deadline 2024-04-21T08:53:19.772130+0000) 2024-04-21T08:53:42.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:42 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:42.242+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-21T08:53:04.973593+0000 front 2024-04-21T08:53:04.973668+0000 (oldest deadline 2024-04-21T08:53:27.273510+0000) 2024-04-21T08:53:42.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:42 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:42.242+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6820 osd.7 since back 2024-04-21T08:53:11.675993+0000 front 2024-04-21T08:53:11.675963+0000 (oldest deadline 2024-04-21T08:53:33.375829+0000) 2024-04-21T08:53:43.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:43 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:43.197+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6806 osd.0 since back 2024-04-21T08:52:12.965835+0000 front 2024-04-21T08:52:12.965783+0000 (oldest deadline 2024-04-21T08:52:38.865928+0000) 2024-04-21T08:53:43.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:43 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:43.197+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6804 osd.1 since back 2024-04-21T08:52:24.667954+0000 front 2024-04-21T08:52:24.668125+0000 (oldest deadline 2024-04-21T08:52:47.567834+0000) 2024-04-21T08:53:43.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:43 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:43.197+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6804 osd.2 since back 2024-04-21T08:52:30.969529+0000 front 2024-04-21T08:52:30.969660+0000 (oldest deadline 2024-04-21T08:52:56.869647+0000) 2024-04-21T08:53:43.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:43 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:43.197+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6812 osd.4 since back 2024-04-21T08:52:48.571785+0000 front 2024-04-21T08:52:48.571730+0000 (oldest deadline 2024-04-21T08:53:13.871627+0000) 2024-04-21T08:53:43.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:43 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:43.197+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.179:6812 osd.5 since back 2024-04-21T08:52:53.872016+0000 front 2024-04-21T08:52:53.872152+0000 (oldest deadline 2024-04-21T08:53:19.772130+0000) 2024-04-21T08:53:43.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:43 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:43.197+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.86:6822 osd.6 since back 2024-04-21T08:53:04.973593+0000 front 2024-04-21T08:53:04.973668+0000 (oldest deadline 2024-04-21T08:53:27.273510+0000) 2024-04-21T08:53:43.497 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:43 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11[56631]: 2024-04-21T08:53:43.197+0000 7f4a19d0d700 -1 osd.11 112 heartbeat_check: no reply from 172.21.15.88:6820 osd.7 since back 2024-04-21T08:53:11.675993+0000 front 2024-04-21T08:53:11.675963+0000 (oldest deadline 2024-04-21T08:53:33.375829+0000) 2024-04-21T08:53:44.744 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:44 smithi179 podman[69314]: 2024-04-21 08:53:44.492614992 +0000 UTC m=+5.545939611 container died 77c37a049b52296a9ea959c724222f2f65898ee96b7230a651b3d01a81b63358 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11, GIT_CLEAN=True, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS) 2024-04-21T08:53:44.996 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:44 smithi179 podman[69314]: 2024-04-21 08:53:44.835449377 +0000 UTC m=+5.888773994 container cleanup 77c37a049b52296a9ea959c724222f2f65898ee96b7230a651b3d01a81b63358 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11, GIT_CLEAN=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS) 2024-04-21T08:53:44.997 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:44 smithi179 bash[69314]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11 2024-04-21T08:53:45.419 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:45 smithi179 podman[69331]: 2024-04-21 08:53:45.003307474 +0000 UTC m=+0.510445239 container remove 77c37a049b52296a9ea959c724222f2f65898ee96b7230a651b3d01a81b63358 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.vendor=CentOS, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2) 2024-04-21T08:53:45.722 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:45 smithi179 podman[69423]: 2024-04-21 08:53:45.420901633 +0000 UTC m=+0.113742971 container create a7f4ec9653047378c496f82da60cd1c0c6163395cd07df668257d6a8acff8e95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, ceph=True, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-21T08:53:45.722 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:45 smithi179 podman[69423]: 2024-04-21 08:53:45.325138259 +0000 UTC m=+0.017979598 image pull b45a994450e2a16d2cd2438b86c3e132853c7dd0b34f48fede9705dbcfa91192 quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda 2024-04-21T08:53:45.722 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:45 smithi179 podman[69423]: 2024-04-21 08:53:45.639872351 +0000 UTC m=+0.332713689 container init a7f4ec9653047378c496f82da60cd1c0c6163395cd07df668257d6a8acff8e95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11-deactivate, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, io.buildah.version=1.33.5, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T08:53:45.722 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:45 smithi179 podman[69423]: 2024-04-21 08:53:45.642978387 +0000 UTC m=+0.335819726 container start a7f4ec9653047378c496f82da60cd1c0c6163395cd07df668257d6a8acff8e95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11-deactivate, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image, RELEASE=wip-lusov-quiescer-a630765, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-21T08:53:45.722 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:45 smithi179 podman[69423]: 2024-04-21 08:53:45.723128914 +0000 UTC m=+0.415970265 container attach a7f4ec9653047378c496f82da60cd1c0c6163395cd07df668257d6a8acff8e95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11-deactivate, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:53:45.997 INFO:journalctl@ceph.osd.11.smithi179.stdout:Apr 21 08:53:45 smithi179 podman[69423]: 2024-04-21 08:53:45.940437254 +0000 UTC m=+0.633278597 container died a7f4ec9653047378c496f82da60cd1c0c6163395cd07df668257d6a8acff8e95 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-osd-11-deactivate, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, GIT_CLEAN=True, ceph=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=) 2024-04-21T08:53:46.525 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@osd.11.service' 2024-04-21T08:53:46.563 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:53:46.564 INFO:tasks.cephadm.osd.11:Stopped osd.11 2024-04-21T08:53:46.564 INFO:tasks.cephadm.mds.a:Stopping mds.a... 2024-04-21T08:53:46.564 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mds.a 2024-04-21T08:53:46.855 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 21 08:53:46 smithi086 systemd[1]: Stopping Ceph mds.a for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:53:47.117 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 21 08:53:46 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-a[67178]: 2024-04-21T08:53:46.855+0000 7fe40b498700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.a -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:53:47.117 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 21 08:53:46 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-a[67178]: 2024-04-21T08:53:46.855+0000 7fe40b498700 -1 mds.a *** got signal Terminated *** 2024-04-21T08:53:48.613 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 21 08:53:48 smithi086 podman[105371]: 2024-04-21 08:53:48.352464634 +0000 UTC m=+1.596705986 container died f9fbd33c53f616b8450d58512bcd779ef660061774504dcff95fba8739070fa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-a, io.buildah.version=1.33.5, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_CLEAN=True, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:53:48.867 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 21 08:53:48 smithi086 podman[105371]: 2024-04-21 08:53:48.713630933 +0000 UTC m=+1.957872294 container cleanup f9fbd33c53f616b8450d58512bcd779ef660061774504dcff95fba8739070fa1 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-a, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, ceph=True, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67) 2024-04-21T08:53:48.867 INFO:journalctl@ceph.mds.a.smithi086.stdout:Apr 21 08:53:48 smithi086 bash[105371]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-a 2024-04-21T08:53:49.145 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mds.a.service' 2024-04-21T08:53:49.179 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:53:49.180 INFO:tasks.cephadm.mds.a:Stopped mds.a 2024-04-21T08:53:49.180 INFO:tasks.cephadm.mds.d:Stopping mds.d... 2024-04-21T08:53:49.180 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mds.d 2024-04-21T08:53:49.617 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 21 08:53:49 smithi086 systemd[1]: Stopping Ceph mds.d for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:53:49.617 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 21 08:53:49 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-d[66221]: 2024-04-21T08:53:49.483+0000 7fb05ec83700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.d -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:53:49.617 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 21 08:53:49 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-d[66221]: 2024-04-21T08:53:49.483+0000 7fb05ec83700 -1 mds.d *** got signal Terminated *** 2024-04-21T08:53:52.183 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 21 08:53:51 smithi086 podman[105492]: 2024-04-21 08:53:51.932130842 +0000 UTC m=+2.538631521 container died ce45a36ff107dec3b97830dc856920f5637f2d298b60e394d0d7277a0177086a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-d, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, org.label-schema.build-date=20240416, GIT_CLEAN=True, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:53:52.567 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 21 08:53:52 smithi086 podman[105492]: 2024-04-21 08:53:52.283659363 +0000 UTC m=+2.890160050 container cleanup ce45a36ff107dec3b97830dc856920f5637f2d298b60e394d0d7277a0177086a (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-d, org.label-schema.vendor=CentOS, ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:53:52.567 INFO:journalctl@ceph.mds.d.smithi086.stdout:Apr 21 08:53:52 smithi086 bash[105492]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-d 2024-04-21T08:53:52.711 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mds.d.service' 2024-04-21T08:53:52.743 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:53:52.743 INFO:tasks.cephadm.mds.d:Stopped mds.d 2024-04-21T08:53:52.743 INFO:tasks.cephadm.mds.g:Stopping mds.g... 2024-04-21T08:53:52.744 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mds.g 2024-04-21T08:53:53.117 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 21 08:53:52 smithi086 systemd[1]: Stopping Ceph mds.g for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:53:53.117 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 21 08:53:53 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-g[66819]: 2024-04-21T08:53:53.044+0000 7f1c9da22700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.g -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:53:53.117 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 21 08:53:53 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-g[66819]: 2024-04-21T08:53:53.044+0000 7f1c9da22700 -1 mds.g *** got signal Terminated *** 2024-04-21T08:53:56.617 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 21 08:53:56 smithi086 podman[105614]: 2024-04-21 08:53:56.365079427 +0000 UTC m=+3.409247542 container died 6d381c165a0f6fe94ee0bde8fc729ef3a4a385dc93c419bf64372ca94c2654da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-g, org.label-schema.license=GPLv2, ceph=True, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, RELEASE=wip-lusov-quiescer-a630765, io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:53:57.102 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 21 08:53:56 smithi086 podman[105614]: 2024-04-21 08:53:56.695599858 +0000 UTC m=+3.739767960 container cleanup 6d381c165a0f6fe94ee0bde8fc729ef3a4a385dc93c419bf64372ca94c2654da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-g, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.build-date=20240416, ceph=True, maintainer=Guillaume Abrioux , CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.license=GPLv2) 2024-04-21T08:53:57.102 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 21 08:53:56 smithi086 bash[105614]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-g 2024-04-21T08:53:57.102 INFO:journalctl@ceph.mds.g.smithi086.stdout:Apr 21 08:53:56 smithi086 podman[105628]: 2024-04-21 08:53:56.888318853 +0000 UTC m=+0.521951602 container remove 6d381c165a0f6fe94ee0bde8fc729ef3a4a385dc93c419bf64372ca94c2654da (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-g, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , GIT_CLEAN=True, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T08:53:57.121 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mds.g.service' 2024-04-21T08:53:57.151 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:53:57.152 INFO:tasks.cephadm.mds.g:Stopped mds.g 2024-04-21T08:53:57.152 INFO:tasks.cephadm.mds.j:Stopping mds.j... 2024-04-21T08:53:57.152 DEBUG:teuthology.orchestra.run.smithi086:> sudo systemctl stop ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mds.j 2024-04-21T08:53:57.473 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 21 08:53:57 smithi086 systemd[1]: Stopping Ceph mds.j for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:53:57.867 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 21 08:53:57 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-j[65871]: 2024-04-21T08:53:57.472+0000 7f614e588700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.j -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:53:57.867 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 21 08:53:57 smithi086 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-j[65871]: 2024-04-21T08:53:57.472+0000 7f614e588700 -1 mds.j *** got signal Terminated *** 2024-04-21T08:54:01.348 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 21 08:54:01 smithi086 podman[105735]: 2024-04-21 08:54:01.069368017 +0000 UTC m=+3.705790277 container died 845db0fd9413f0e3a60cded26de256bff35590c6c6d0a7e0ab1ea092c35c4a85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-j, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, ceph=True, RELEASE=wip-lusov-quiescer-a630765, maintainer=Guillaume Abrioux ) 2024-04-21T08:54:01.600 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 21 08:54:01 smithi086 podman[105735]: 2024-04-21 08:54:01.440493116 +0000 UTC m=+4.076915376 container cleanup 845db0fd9413f0e3a60cded26de256bff35590c6c6d0a7e0ab1ea092c35c4a85 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-j, org.label-schema.license=GPLv2, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , ceph=True, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, GIT_REPO=git@github.com:ceph/ceph-container.git, CEPH_POINT_RELEASE=) 2024-04-21T08:54:01.600 INFO:journalctl@ceph.mds.j.smithi086.stdout:Apr 21 08:54:01 smithi086 bash[105735]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-j 2024-04-21T08:54:01.828 DEBUG:teuthology.orchestra.run.smithi086:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mds.j.service' 2024-04-21T08:54:01.859 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:54:01.859 INFO:tasks.cephadm.mds.j:Stopped mds.j 2024-04-21T08:54:01.859 INFO:tasks.cephadm.mds.b:Stopping mds.b... 2024-04-21T08:54:01.859 DEBUG:teuthology.orchestra.run.smithi088:> sudo systemctl stop ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mds.b 2024-04-21T08:54:02.144 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 21 08:54:01 smithi088 systemd[1]: Stopping Ceph mds.b for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:54:02.410 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 21 08:54:02 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-b[61651]: 2024-04-21T08:54:02.145+0000 7f11ac67c700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.b -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:54:02.411 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 21 08:54:02 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-b[61651]: 2024-04-21T08:54:02.145+0000 7f11ac67c700 -1 mds.b *** got signal Terminated *** 2024-04-21T08:54:04.611 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 21 08:54:04 smithi088 podman[68566]: 2024-04-21 08:54:04.357710798 +0000 UTC m=+2.308546973 container died 6ae9477515466e140501ecf80fc9d1184594bec1ac44afbad2aee80dcc7b2f92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-b, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.license=GPLv2, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, CEPH_POINT_RELEASE=, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T08:54:04.964 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 21 08:54:04 smithi088 podman[68566]: 2024-04-21 08:54:04.711064396 +0000 UTC m=+2.661900578 container cleanup 6ae9477515466e140501ecf80fc9d1184594bec1ac44afbad2aee80dcc7b2f92 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-b, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, GIT_BRANCH=HEAD, ceph=True, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image, maintainer=Guillaume Abrioux , org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS) 2024-04-21T08:54:04.964 INFO:journalctl@ceph.mds.b.smithi088.stdout:Apr 21 08:54:04 smithi088 bash[68566]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-b 2024-04-21T08:54:05.136 DEBUG:teuthology.orchestra.run.smithi088:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mds.b.service' 2024-04-21T08:54:05.170 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:54:05.170 INFO:tasks.cephadm.mds.b:Stopped mds.b 2024-04-21T08:54:05.170 INFO:tasks.cephadm.mds.e:Stopping mds.e... 2024-04-21T08:54:05.170 DEBUG:teuthology.orchestra.run.smithi088:> sudo systemctl stop ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mds.e 2024-04-21T08:54:05.714 INFO:journalctl@ceph.mds.e.smithi088.stdout:Apr 21 08:54:05 smithi088 systemd[1]: Stopping Ceph mds.e for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:54:05.714 INFO:journalctl@ceph.mds.e.smithi088.stdout:Apr 21 08:54:05 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-e[61012]: 2024-04-21T08:54:05.490+0000 7f0afa308700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.e -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:54:05.714 INFO:journalctl@ceph.mds.e.smithi088.stdout:Apr 21 08:54:05 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-e[61012]: 2024-04-21T08:54:05.490+0000 7f0afa308700 -1 mds.e *** got signal Terminated *** 2024-04-21T08:54:08.714 INFO:journalctl@ceph.mds.e.smithi088.stdout:Apr 21 08:54:08 smithi088 podman[68700]: 2024-04-21 08:54:08.35865792 +0000 UTC m=+2.976241810 container died 110f8ccf6dd1697ef6a5d6fa68e2f0261c340962f8ffd1dd29a3a0f794e181df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.license=GPLv2, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.schema-version=1.0, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD) 2024-04-21T08:54:09.074 INFO:journalctl@ceph.mds.e.smithi088.stdout:Apr 21 08:54:08 smithi088 podman[68700]: 2024-04-21 08:54:08.832101225 +0000 UTC m=+3.449685134 container cleanup 110f8ccf6dd1697ef6a5d6fa68e2f0261c340962f8ffd1dd29a3a0f794e181df (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-e, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD, ceph=True, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, GIT_CLEAN=True, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:54:09.074 INFO:journalctl@ceph.mds.e.smithi088.stdout:Apr 21 08:54:08 smithi088 bash[68700]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-e 2024-04-21T08:54:09.206 DEBUG:teuthology.orchestra.run.smithi088:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mds.e.service' 2024-04-21T08:54:09.238 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:54:09.238 INFO:tasks.cephadm.mds.e:Stopped mds.e 2024-04-21T08:54:09.238 INFO:tasks.cephadm.mds.h:Stopping mds.h... 2024-04-21T08:54:09.238 DEBUG:teuthology.orchestra.run.smithi088:> sudo systemctl stop ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mds.h 2024-04-21T08:54:09.464 INFO:journalctl@ceph.mds.h.smithi088.stdout:Apr 21 08:54:09 smithi088 systemd[1]: Stopping Ceph mds.h for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:54:09.964 INFO:journalctl@ceph.mds.h.smithi088.stdout:Apr 21 08:54:09 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-h[61329]: 2024-04-21T08:54:09.552+0000 7fc4f0064700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.h -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:54:09.964 INFO:journalctl@ceph.mds.h.smithi088.stdout:Apr 21 08:54:09 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-h[61329]: 2024-04-21T08:54:09.552+0000 7fc4f0064700 -1 mds.h *** got signal Terminated *** 2024-04-21T08:54:12.644 INFO:journalctl@ceph.mds.h.smithi088.stdout:Apr 21 08:54:12 smithi088 podman[68820]: 2024-04-21 08:54:12.360284367 +0000 UTC m=+2.906884732 container died 708fa1f6099be6c56bdec537b258d5537c11fe9947cdbaba5bb5481d491f843b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-h, io.buildah.version=1.33.5, GIT_CLEAN=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.vendor=CentOS, org.label-schema.schema-version=1.0, ceph=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux ) 2024-04-21T08:54:12.964 INFO:journalctl@ceph.mds.h.smithi088.stdout:Apr 21 08:54:12 smithi088 podman[68820]: 2024-04-21 08:54:12.744374775 +0000 UTC m=+3.290975138 container cleanup 708fa1f6099be6c56bdec537b258d5537c11fe9947cdbaba5bb5481d491f843b (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-h, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, ceph=True, org.label-schema.vendor=CentOS, GIT_CLEAN=True, org.label-schema.build-date=20240416, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, GIT_BRANCH=HEAD) 2024-04-21T08:54:12.964 INFO:journalctl@ceph.mds.h.smithi088.stdout:Apr 21 08:54:12 smithi088 bash[68820]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-h 2024-04-21T08:54:13.121 DEBUG:teuthology.orchestra.run.smithi088:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mds.h.service' 2024-04-21T08:54:13.151 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:54:13.152 INFO:tasks.cephadm.mds.h:Stopped mds.h 2024-04-21T08:54:13.152 INFO:tasks.cephadm.mds.k:Stopping mds.k... 2024-04-21T08:54:13.152 DEBUG:teuthology.orchestra.run.smithi088:> sudo systemctl stop ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mds.k 2024-04-21T08:54:13.408 INFO:journalctl@ceph.mds.k.smithi088.stdout:Apr 21 08:54:13 smithi088 systemd[1]: Stopping Ceph mds.k for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:54:13.714 INFO:journalctl@ceph.mds.k.smithi088.stdout:Apr 21 08:54:13 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-k[60695]: 2024-04-21T08:54:13.472+0000 7fb30c32f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.k -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:54:13.714 INFO:journalctl@ceph.mds.k.smithi088.stdout:Apr 21 08:54:13 smithi088 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-k[60695]: 2024-04-21T08:54:13.472+0000 7fb30c32f700 -1 mds.k *** got signal Terminated *** 2024-04-21T08:54:16.714 INFO:journalctl@ceph.mds.k.smithi088.stdout:Apr 21 08:54:16 smithi088 podman[68942]: 2024-04-21 08:54:16.355629204 +0000 UTC m=+2.987095594 container died e43b96f6ea27643ff10ecd0b4d7487265e4e5ec1b09d63703fd3c1e255ce02ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-k, org.label-schema.build-date=20240416, GIT_REPO=git@github.com:ceph/ceph-container.git, GIT_BRANCH=HEAD, ceph=True, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T08:54:17.214 INFO:journalctl@ceph.mds.k.smithi088.stdout:Apr 21 08:54:17 smithi088 podman[68942]: 2024-04-21 08:54:17.01488959 +0000 UTC m=+3.646355981 container cleanup e43b96f6ea27643ff10ecd0b4d7487265e4e5ec1b09d63703fd3c1e255ce02ad (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-k, org.label-schema.schema-version=1.0, GIT_CLEAN=True, org.label-schema.vendor=CentOS, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.build-date=20240416, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , io.buildah.version=1.33.5, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD) 2024-04-21T08:54:17.215 INFO:journalctl@ceph.mds.k.smithi088.stdout:Apr 21 08:54:17 smithi088 bash[68942]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-k 2024-04-21T08:54:17.443 DEBUG:teuthology.orchestra.run.smithi088:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mds.k.service' 2024-04-21T08:54:17.473 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:54:17.473 INFO:tasks.cephadm.mds.k:Stopped mds.k 2024-04-21T08:54:17.474 INFO:tasks.cephadm.mds.c:Stopping mds.c... 2024-04-21T08:54:17.474 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mds.c 2024-04-21T08:54:17.834 INFO:journalctl@ceph.mds.c.smithi179.stdout:Apr 21 08:54:17 smithi179 systemd[1]: Stopping Ceph mds.c for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:54:18.247 INFO:journalctl@ceph.mds.c.smithi179.stdout:Apr 21 08:54:17 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-c[61508]: 2024-04-21T08:54:17.834+0000 7f1b6dbaa700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.c -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:54:18.247 INFO:journalctl@ceph.mds.c.smithi179.stdout:Apr 21 08:54:17 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-c[61508]: 2024-04-21T08:54:17.834+0000 7f1b6dbaa700 -1 mds.c *** got signal Terminated *** 2024-04-21T08:54:20.626 INFO:journalctl@ceph.mds.c.smithi179.stdout:Apr 21 08:54:20 smithi179 podman[69528]: 2024-04-21 08:54:20.363522912 +0000 UTC m=+2.695162202 container died 0cbbb963e35060cabb9fb22ef7aff4defa9dc893beb7882dceb53e698997eec4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-c, org.label-schema.license=GPLv2, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, org.label-schema.schema-version=1.0, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.build-date=20240416) 2024-04-21T08:54:20.993 INFO:journalctl@ceph.mds.c.smithi179.stdout:Apr 21 08:54:20 smithi179 podman[69528]: 2024-04-21 08:54:20.72640104 +0000 UTC m=+3.058040333 container cleanup 0cbbb963e35060cabb9fb22ef7aff4defa9dc893beb7882dceb53e698997eec4 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-c, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, CEPH_POINT_RELEASE=, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, RELEASE=wip-lusov-quiescer-a630765, ceph=True) 2024-04-21T08:54:20.993 INFO:journalctl@ceph.mds.c.smithi179.stdout:Apr 21 08:54:20 smithi179 bash[69528]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-c 2024-04-21T08:54:21.148 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mds.c.service' 2024-04-21T08:54:21.182 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:54:21.182 INFO:tasks.cephadm.mds.c:Stopped mds.c 2024-04-21T08:54:21.183 INFO:tasks.cephadm.mds.f:Stopping mds.f... 2024-04-21T08:54:21.183 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mds.f 2024-04-21T08:54:21.496 INFO:journalctl@ceph.mds.f.smithi179.stdout:Apr 21 08:54:21 smithi179 systemd[1]: Stopping Ceph mds.f for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:54:21.997 INFO:journalctl@ceph.mds.f.smithi179.stdout:Apr 21 08:54:21 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-f[61204]: 2024-04-21T08:54:21.504+0000 7ff87ba7f700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.f -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:54:21.997 INFO:journalctl@ceph.mds.f.smithi179.stdout:Apr 21 08:54:21 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-f[61204]: 2024-04-21T08:54:21.504+0000 7ff87ba7f700 -1 mds.f *** got signal Terminated *** 2024-04-21T08:54:24.747 INFO:journalctl@ceph.mds.f.smithi179.stdout:Apr 21 08:54:24 smithi179 podman[69661]: 2024-04-21 08:54:24.367301886 +0000 UTC m=+2.968909957 container died 09df6c287e69ceb971ba0df1c95ef1af6ad29207bc52e979b7e28541e439b270 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-f, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, CEPH_POINT_RELEASE=, ceph=True, maintainer=Guillaume Abrioux , org.label-schema.license=GPLv2, io.buildah.version=1.33.5, org.label-schema.name=CentOS Stream 8 Base Image) 2024-04-21T08:54:25.080 INFO:journalctl@ceph.mds.f.smithi179.stdout:Apr 21 08:54:24 smithi179 podman[69661]: 2024-04-21 08:54:24.838728796 +0000 UTC m=+3.440336894 container cleanup 09df6c287e69ceb971ba0df1c95ef1af6ad29207bc52e979b7e28541e439b270 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-f, org.label-schema.schema-version=1.0, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, org.label-schema.build-date=20240416, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.license=GPLv2) 2024-04-21T08:54:25.080 INFO:journalctl@ceph.mds.f.smithi179.stdout:Apr 21 08:54:24 smithi179 bash[69661]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-f 2024-04-21T08:54:25.213 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mds.f.service' 2024-04-21T08:54:25.245 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:54:25.245 INFO:tasks.cephadm.mds.f:Stopped mds.f 2024-04-21T08:54:25.245 INFO:tasks.cephadm.mds.i:Stopping mds.i... 2024-04-21T08:54:25.245 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mds.i 2024-04-21T08:54:25.497 INFO:journalctl@ceph.mds.i.smithi179.stdout:Apr 21 08:54:25 smithi179 systemd[1]: Stopping Ceph mds.i for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:54:25.997 INFO:journalctl@ceph.mds.i.smithi179.stdout:Apr 21 08:54:25 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-i[61826]: 2024-04-21T08:54:25.575+0000 7fe401b25700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.i -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:54:25.997 INFO:journalctl@ceph.mds.i.smithi179.stdout:Apr 21 08:54:25 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-i[61826]: 2024-04-21T08:54:25.575+0000 7fe401b25700 -1 mds.i *** got signal Terminated *** 2024-04-21T08:54:28.747 INFO:journalctl@ceph.mds.i.smithi179.stdout:Apr 21 08:54:28 smithi179 podman[69780]: 2024-04-21 08:54:28.363293993 +0000 UTC m=+2.903800235 container died 52eb9c4c876e38889f661b5004d26dfbc43101596a38f9fb78c738e7b59e00f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-i, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_CLEAN=True, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.license=GPLv2, ceph=True, io.buildah.version=1.33.5, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, org.label-schema.name=CentOS Stream 8 Base Image, org.label-schema.schema-version=1.0, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T08:54:29.201 INFO:journalctl@ceph.mds.i.smithi179.stdout:Apr 21 08:54:28 smithi179 podman[69780]: 2024-04-21 08:54:28.942695997 +0000 UTC m=+3.483202251 container cleanup 52eb9c4c876e38889f661b5004d26dfbc43101596a38f9fb78c738e7b59e00f5 (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-i, org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, org.label-schema.schema-version=1.0, maintainer=Guillaume Abrioux , org.label-schema.build-date=20240416, GIT_BRANCH=HEAD, GIT_REPO=git@github.com:ceph/ceph-container.git, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, ceph=True, io.buildah.version=1.33.5, RELEASE=wip-lusov-quiescer-a630765) 2024-04-21T08:54:29.201 INFO:journalctl@ceph.mds.i.smithi179.stdout:Apr 21 08:54:28 smithi179 bash[69780]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-i 2024-04-21T08:54:29.368 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mds.i.service' 2024-04-21T08:54:29.399 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:54:29.399 INFO:tasks.cephadm.mds.i:Stopped mds.i 2024-04-21T08:54:29.399 INFO:tasks.cephadm.mds.l:Stopping mds.l... 2024-04-21T08:54:29.399 DEBUG:teuthology.orchestra.run.smithi179:> sudo systemctl stop ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mds.l 2024-04-21T08:54:29.729 INFO:journalctl@ceph.mds.l.smithi179.stdout:Apr 21 08:54:29 smithi179 systemd[1]: Stopping Ceph mds.l for f8cec7cc-ffb8-11ee-bc93-c7b262605968... 2024-04-21T08:54:29.997 INFO:journalctl@ceph.mds.l.smithi179.stdout:Apr 21 08:54:29 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-l[60895]: 2024-04-21T08:54:29.729+0000 7f4f2c820700 -1 received signal: Terminated from /run/podman-init -- /usr/bin/ceph-mds -n mds.l -f --setuser ceph --setgroup ceph --default-log-to-file=false --default-log-to-journald=true --default-log-to-stderr=false (PID: 1) UID: 0 2024-04-21T08:54:29.997 INFO:journalctl@ceph.mds.l.smithi179.stdout:Apr 21 08:54:29 smithi179 ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-l[60895]: 2024-04-21T08:54:29.729+0000 7f4f2c820700 -1 mds.l *** got signal Terminated *** 2024-04-21T08:54:32.705 INFO:journalctl@ceph.mds.l.smithi179.stdout:Apr 21 08:54:32 smithi179 podman[69900]: 2024-04-21 08:54:32.368150002 +0000 UTC m=+2.753967748 container died c381a381724c219ebda0071c7b02a48e9d1acde2dafa308bc690793a470ec97d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-l, org.label-schema.license=GPLv2, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.schema-version=1.0, org.label-schema.name=CentOS Stream 8 Base Image, GIT_CLEAN=True, org.label-schema.build-date=20240416, maintainer=Guillaume Abrioux , org.label-schema.vendor=CentOS, CEPH_POINT_RELEASE=, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_REPO=git@github.com:ceph/ceph-container.git, io.buildah.version=1.33.5, ceph=True, GIT_BRANCH=HEAD) 2024-04-21T08:54:32.980 INFO:journalctl@ceph.mds.l.smithi179.stdout:Apr 21 08:54:32 smithi179 podman[69900]: 2024-04-21 08:54:32.788402646 +0000 UTC m=+3.174220382 container cleanup c381a381724c219ebda0071c7b02a48e9d1acde2dafa308bc690793a470ec97d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-l, io.buildah.version=1.33.5, org.label-schema.vendor=CentOS, GIT_CLEAN=True, maintainer=Guillaume Abrioux , ceph=True, org.label-schema.name=CentOS Stream 8 Base Image, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_BRANCH=HEAD, RELEASE=wip-lusov-quiescer-a630765, org.label-schema.build-date=20240416, org.label-schema.license=GPLv2, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, GIT_REPO=git@github.com:ceph/ceph-container.git) 2024-04-21T08:54:32.980 INFO:journalctl@ceph.mds.l.smithi179.stdout:Apr 21 08:54:32 smithi179 bash[69900]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-l 2024-04-21T08:54:33.221 DEBUG:teuthology.orchestra.run.smithi179:> sudo pkill -f 'journalctl -f -n 0 -u ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mds.l.service' 2024-04-21T08:54:33.241 INFO:journalctl@ceph.mds.l.smithi179.stdout:Apr 21 08:54:32 smithi179 podman[69914]: 2024-04-21 08:54:32.98113341 +0000 UTC m=+0.611543909 container remove c381a381724c219ebda0071c7b02a48e9d1acde2dafa308bc690793a470ec97d (image=quay-quay-quay.apps.os.sepia.ceph.com/ceph-ci/ceph@sha256:36ea816f637ab0eb3aba78ac1a5e79acaf4ee67536fe53155c0137fedf936fda, name=ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968-mds-l, GIT_COMMIT=e83daab126629de4b83b70f7819e84da953b5b67, GIT_CLEAN=True, GIT_REPO=git@github.com:ceph/ceph-container.git, RELEASE=wip-lusov-quiescer-a630765, GIT_BRANCH=HEAD, org.label-schema.vendor=CentOS, org.label-schema.build-date=20240416, io.buildah.version=1.33.5, org.label-schema.license=GPLv2, org.label-schema.name=CentOS Stream 8 Base Image, ceph=True, org.label-schema.schema-version=1.0, CEPH_POINT_RELEASE=, maintainer=Guillaume Abrioux ) 2024-04-21T08:54:33.241 INFO:journalctl@ceph.mds.l.smithi179.stdout:Apr 21 08:54:33 smithi179 systemd[1]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mds.l.service: Deactivated successfully. 2024-04-21T08:54:33.241 INFO:journalctl@ceph.mds.l.smithi179.stdout:Apr 21 08:54:33 smithi179 systemd[1]: Stopped Ceph mds.l for f8cec7cc-ffb8-11ee-bc93-c7b262605968. 2024-04-21T08:54:33.241 INFO:journalctl@ceph.mds.l.smithi179.stdout:Apr 21 08:54:33 smithi179 systemd[1]: ceph-f8cec7cc-ffb8-11ee-bc93-c7b262605968@mds.l.service: Consumed 2.296s CPU time. 2024-04-21T08:54:33.251 DEBUG:teuthology.orchestra.run:got remote process result: None 2024-04-21T08:54:33.251 INFO:tasks.cephadm.mds.l:Stopped mds.l 2024-04-21T08:54:33.251 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 --force --keep-logs 2024-04-21T08:54:33.440 INFO:teuthology.orchestra.run.smithi086.stdout:Deleting cluster with fsid: f8cec7cc-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:54:35.862 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 --force --keep-logs 2024-04-21T08:54:36.050 INFO:teuthology.orchestra.run.smithi088.stdout:Deleting cluster with fsid: f8cec7cc-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:54:38.496 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 --force --keep-logs 2024-04-21T08:54:38.685 INFO:teuthology.orchestra.run.smithi179.stdout:Deleting cluster with fsid: f8cec7cc-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:54:40.980 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-21T08:54:41.011 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-21T08:54:41.043 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f /etc/ceph/ceph.conf /etc/ceph/ceph.client.admin.keyring 2024-04-21T08:54:41.068 INFO:tasks.cephadm:Archiving crash dumps... 2024-04-21T08:54:41.070 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666511/remote/smithi086/crash 2024-04-21T08:54:41.071 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/crash -- . 2024-04-21T08:54:41.109 INFO:teuthology.orchestra.run.smithi086.stderr:tar: /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-21T08:54:41.109 INFO:teuthology.orchestra.run.smithi086.stderr:tar: Error is not recoverable: exiting now 2024-04-21T08:54:41.111 DEBUG:teuthology.misc:Transferring archived files from smithi088:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666511/remote/smithi088/crash 2024-04-21T08:54:41.112 DEBUG:teuthology.orchestra.run.smithi088:> sudo tar c -f - -C /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/crash -- . 2024-04-21T08:54:41.301 INFO:teuthology.orchestra.run.smithi088.stderr:tar: /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-21T08:54:41.301 INFO:teuthology.orchestra.run.smithi088.stderr:tar: Error is not recoverable: exiting now 2024-04-21T08:54:41.303 DEBUG:teuthology.misc:Transferring archived files from smithi179:/var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/crash to /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666511/remote/smithi179/crash 2024-04-21T08:54:41.304 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/crash -- . 2024-04-21T08:54:41.341 INFO:teuthology.orchestra.run.smithi179.stderr:tar: /var/lib/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/crash: Cannot open: No such file or directory 2024-04-21T08:54:41.341 INFO:teuthology.orchestra.run.smithi179.stderr:tar: Error is not recoverable: exiting now 2024-04-21T08:54:41.342 INFO:tasks.cephadm:Checking cluster log for badness... 2024-04-21T08:54:41.342 DEBUG:teuthology.orchestra.run.smithi086:> sudo egrep '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph.log | egrep -v '\(MDS_ALL_DOWN\)' | egrep -v '\(MDS_UP_LESS_THAN_MAX\)' | egrep -v FS_DEGRADED | egrep -v FS_INLINE_DATA_DEPRECATED | egrep -v FS_WITH_FAILED_MDS | egrep -v MDS_ALL_DOWN | egrep -v MDS_DAMAGE | egrep -v MDS_DEGRADED | egrep -v MDS_FAILED | egrep -v MDS_INSUFFICIENT_STANDBY | egrep -v MDS_UP_LESS_THAN_MAX | egrep -v 'filesystem is online with fewer MDS than max_mds' | egrep -v POOL_APP_NOT_ENABLED | egrep -v 'do not have an application enabled' | egrep -v 'overall HEALTH_' | egrep -v 'Replacing daemon' | egrep -v 'deprecated feature inline_data' | egrep -v 'overall HEALTH_' | egrep -v '\(OSD_DOWN\)' | egrep -v '\(OSD_' | egrep -v 'but it is still running' | egrep -v 'is not responding' | egrep -v PG_AVAILABILITY | egrep -v PG_DEGRADED | egrep -v 'Reduced data availability' | egrep -v 'slow metadata IO' | egrep -v SLOW_OPS | egrep -v 'slow request' | head -n 1 2024-04-21T08:54:41.379 INFO:tasks.cephadm:Compressing logs... 2024-04-21T08:54:41.379 DEBUG:teuthology.orchestra.run.smithi086:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-21T08:54:41.422 DEBUG:teuthology.orchestra.run.smithi088:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-21T08:54:41.425 DEBUG:teuthology.orchestra.run.smithi179:> time sudo find /var/log/ceph /var/log/rbd-target-api -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2024-04-21T08:54:41.444 INFO:teuthology.orchestra.run.smithi086.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph.log 2024-04-21T08:54:41.444 INFO:teuthology.orchestra.run.smithi086.stderr:: No such file or directory 2024-04-21T08:54:41.446 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph.audit.log 2024-04-21T08:54:41.446 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mds.g.log 2024-04-21T08:54:41.446 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph.log: /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mds.j.log 2024-04-21T08:54:41.446 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mds.g.log: gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-osd.3.log 2024-04-21T08:54:41.449 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-osd.0.log 2024-04-21T08:54:41.449 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mds.j.log: /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mgr.x.log 2024-04-21T08:54:41.449 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mon.a.log 2024-04-21T08:54:41.449 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-osd.6.log 2024-04-21T08:54:41.449 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mon.a.log: 87.2% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph.log.gz 2024-04-21T08:54:41.450 INFO:teuthology.orchestra.run.smithi086.stderr: 90.2% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph.audit.log.gz 2024-04-21T08:54:41.455 INFO:teuthology.orchestra.run.smithi086.stderr: 94.6% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mds.j.log.gz 2024-04-21T08:54:41.458 INFO:teuthology.orchestra.run.smithi088.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph.log 2024-04-21T08:54:41.458 INFO:teuthology.orchestra.run.smithi088.stderr:: No such file or directory 2024-04-21T08:54:41.458 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph.audit.log 2024-04-21T08:54:41.458 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-osd.7.log 2024-04-21T08:54:41.458 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph.log: /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mds.b.log 2024-04-21T08:54:41.459 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-osd.7.log: gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-osd.4.log 2024-04-21T08:54:41.459 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-osd.1.log 2024-04-21T08:54:41.459 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mds.b.log: /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-osd.4.log: gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mgr.y.log 2024-04-21T08:54:41.459 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mds.h.log 2024-04-21T08:54:41.459 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mgr.y.log: gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-osd.10.log 2024-04-21T08:54:41.459 INFO:teuthology.orchestra.run.smithi088.stderr: 87.2% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph.log.gz/var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mds.h.log: 2024-04-21T08:54:41.459 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph.cephadm.log 2024-04-21T08:54:41.459 INFO:teuthology.orchestra.run.smithi088.stderr: 90.3%/var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-osd.10.log: -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph.audit.log.gz 2024-04-21T08:54:41.460 INFO:teuthology.orchestra.run.smithi179.stderr:find: ‘/var/log/rbd-target-api’gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph.log 2024-04-21T08:54:41.461 INFO:teuthology.orchestra.run.smithi179.stderr:: No such file or directory 2024-04-21T08:54:41.461 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-osd.2.log 2024-04-21T08:54:41.461 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mds.f.log 2024-04-21T08:54:41.461 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph.log: /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph.audit.log 2024-04-21T08:54:41.461 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mds.f.log: gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-osd.5.log 2024-04-21T08:54:41.461 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mds.c.log 2024-04-21T08:54:41.461 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-osd.5.log: gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mon.c.log 2024-04-21T08:54:41.461 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mds.c.log: gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mgr.z.log 2024-04-21T08:54:41.461 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mon.c.log: /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mgr.z.log: 87.2% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph.log.gz 2024-04-21T08:54:41.461 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mds.i.log 2024-04-21T08:54:41.461 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph.cephadm.log 2024-04-21T08:54:41.461 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mds.i.log: 90.4% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph.audit.log.gz 2024-04-21T08:54:41.462 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-osd.11.log 2024-04-21T08:54:41.462 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph.cephadm.log: 84.1% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-21T08:54:41.462 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-osd.8.log 2024-04-21T08:54:41.462 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-osd.11.log: 92.1% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mgr.z.log.gz 2024-04-21T08:54:41.462 INFO:teuthology.orchestra.run.smithi179.stderr:gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-volume.log 2024-04-21T08:54:41.463 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph.cephadm.log 2024-04-21T08:54:41.463 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-osd.6.log: gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mds.a.log 2024-04-21T08:54:41.464 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-volume.log 2024-04-21T08:54:41.464 INFO:teuthology.orchestra.run.smithi086.stderr: 84.3% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph.cephadm.log.gz/var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mds.a.log: 2024-04-21T08:54:41.464 INFO:teuthology.orchestra.run.smithi088.stderr: 92.0% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mgr.y.log.gz 2024-04-21T08:54:41.466 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mds.e.log 2024-04-21T08:54:41.466 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph.cephadm.log: gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-volume.log 2024-04-21T08:54:41.466 INFO:teuthology.orchestra.run.smithi088.stderr: 84.2% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph.cephadm.log.gz 2024-04-21T08:54:41.466 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mds.e.log: gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mon.b.log 2024-04-21T08:54:41.473 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-osd.8.log: gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mds.l.log 2024-04-21T08:54:41.475 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mds.d.log 2024-04-21T08:54:41.479 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mds.k.log 2024-04-21T08:54:41.480 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mon.b.log: 91.0% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-volume.log.gz 2024-04-21T08:54:41.486 INFO:teuthology.orchestra.run.smithi088.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-21T08:54:41.490 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-volume.log: 91.0% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-volume.log.gz 2024-04-21T08:54:41.490 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-osd.9.log 2024-04-21T08:54:41.491 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-volume.log: gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-21T08:54:41.496 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mds.d.log: 94.6% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mds.d.log.gz 2024-04-21T08:54:41.496 INFO:teuthology.orchestra.run.smithi086.stderr:gzip -5 --verbose -- /var/log/ceph/cephadm.log 2024-04-21T08:54:41.498 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mds.l.log: 91.0% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-volume.log.gz 2024-04-21T08:54:41.501 INFO:teuthology.orchestra.run.smithi088.stderr:/var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mds.k.log: /var/log/ceph/cephadm.log: 92.1% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-21T08:54:41.513 INFO:teuthology.orchestra.run.smithi179.stderr:/var/log/ceph/cephadm.log: 92.4% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-21T08:54:41.514 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-osd.9.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.log 2024-04-21T08:54:41.520 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/cephadm.log: 94.5% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mds.g.log.gz 2024-04-21T08:54:41.522 INFO:teuthology.orchestra.run.smithi179.stderr: 94.7% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mds.f.log.gz 2024-04-21T08:54:41.523 INFO:teuthology.orchestra.run.smithi086.stderr: 93.3% -- replaced with /var/log/ceph/cephadm.log.gz 2024-04-21T08:54:41.536 INFO:teuthology.orchestra.run.smithi086.stderr:/var/log/ceph/ceph-client.admin.log: 92.6% -- replaced with /var/log/ceph/ceph-client.admin.log.gz 2024-04-21T08:54:41.537 INFO:teuthology.orchestra.run.smithi086.stderr: 94.5% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mds.a.log.gz 2024-04-21T08:54:41.543 INFO:teuthology.orchestra.run.smithi088.stderr: 94.5% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mds.e.log.gz 2024-04-21T08:54:41.554 INFO:teuthology.orchestra.run.smithi179.stderr: 92.6% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mds.i.log.gz 2024-04-21T08:54:41.555 INFO:teuthology.orchestra.run.smithi088.stderr: 92.7% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mds.h.log.gz 2024-04-21T08:54:41.600 INFO:teuthology.orchestra.run.smithi179.stderr: 92.6% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mds.l.log.gz 2024-04-21T08:54:41.608 INFO:teuthology.orchestra.run.smithi088.stderr: 92.8% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mds.k.log.gz 2024-04-21T08:54:41.682 INFO:teuthology.orchestra.run.smithi179.stderr: 94.0% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mds.c.log.gz 2024-04-21T08:54:41.861 INFO:teuthology.orchestra.run.smithi086.stderr: 90.2% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mgr.x.log.gz 2024-04-21T08:54:41.866 INFO:teuthology.orchestra.run.smithi088.stderr: 92.1% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mon.b.log.gz 2024-04-21T08:54:41.878 INFO:teuthology.orchestra.run.smithi179.stderr: 92.2% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mon.c.log.gz 2024-04-21T08:54:42.330 INFO:teuthology.orchestra.run.smithi086.stderr: 92.5% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-osd.0.log.gz 2024-04-21T08:54:42.442 INFO:teuthology.orchestra.run.smithi086.stderr: 90.7% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mon.a.log.gz 2024-04-21T08:54:42.740 INFO:teuthology.orchestra.run.smithi086.stderr: 92.2% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-osd.6.log.gz 2024-04-21T08:54:42.759 INFO:teuthology.orchestra.run.smithi086.stderr: 92.4% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-osd.3.log.gz 2024-04-21T08:54:42.863 INFO:teuthology.orchestra.run.smithi088.stderr: 91.7% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-osd.7.log.gz 2024-04-21T08:54:42.891 INFO:teuthology.orchestra.run.smithi179.stderr: 91.8% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-osd.8.log.gz 2024-04-21T08:54:43.131 INFO:teuthology.orchestra.run.smithi088.stderr: 93.0% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-osd.1.log.gz 2024-04-21T08:54:43.136 INFO:teuthology.orchestra.run.smithi179.stderr: 92.9% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-osd.11.log.gz 2024-04-21T08:54:43.318 INFO:teuthology.orchestra.run.smithi088.stderr: 93.3% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-osd.4.log.gz 2024-04-21T08:54:43.319 INFO:teuthology.orchestra.run.smithi088.stderr: 91.8% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-osd.10.log.gz 2024-04-21T08:54:43.615 INFO:teuthology.orchestra.run.smithi086.stderr: 92.7% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-osd.9.log.gz 2024-04-21T08:54:43.617 INFO:teuthology.orchestra.run.smithi086.stderr: 2024-04-21T08:54:43.617 INFO:teuthology.orchestra.run.smithi086.stderr:real 0m2.182s 2024-04-21T08:54:43.617 INFO:teuthology.orchestra.run.smithi086.stderr:user 0m6.946s 2024-04-21T08:54:43.617 INFO:teuthology.orchestra.run.smithi086.stderr:sys 0m0.255s 2024-04-21T08:54:43.770 INFO:teuthology.orchestra.run.smithi088.stderr: 92.9% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-mds.b.log.gz 2024-04-21T08:54:43.772 INFO:teuthology.orchestra.run.smithi088.stderr: 2024-04-21T08:54:43.772 INFO:teuthology.orchestra.run.smithi088.stderr:real 0m2.333s 2024-04-21T08:54:43.772 INFO:teuthology.orchestra.run.smithi088.stderr:user 0m9.488s 2024-04-21T08:54:43.772 INFO:teuthology.orchestra.run.smithi088.stderr:sys 0m0.328s 2024-04-21T08:54:43.872 INFO:teuthology.orchestra.run.smithi179.stderr: 92.4% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-osd.5.log.gz 2024-04-21T08:54:43.914 INFO:teuthology.orchestra.run.smithi179.stderr: 92.2% -- replaced with /var/log/ceph/f8cec7cc-ffb8-11ee-bc93-c7b262605968/ceph-osd.2.log.gz 2024-04-21T08:54:43.916 INFO:teuthology.orchestra.run.smithi179.stderr: 2024-04-21T08:54:43.916 INFO:teuthology.orchestra.run.smithi179.stderr:real 0m2.477s 2024-04-21T08:54:43.916 INFO:teuthology.orchestra.run.smithi179.stderr:user 0m8.634s 2024-04-21T08:54:43.916 INFO:teuthology.orchestra.run.smithi179.stderr:sys 0m0.307s 2024-04-21T08:54:43.917 INFO:tasks.cephadm:Archiving logs... 2024-04-21T08:54:43.917 DEBUG:teuthology.misc:Transferring archived files from smithi086:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666511/remote/smithi086/log 2024-04-21T08:54:43.918 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-21T08:54:44.964 DEBUG:teuthology.misc:Transferring archived files from smithi088:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666511/remote/smithi088/log 2024-04-21T08:54:44.965 DEBUG:teuthology.orchestra.run.smithi088:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-21T08:54:46.340 DEBUG:teuthology.misc:Transferring archived files from smithi179:/var/log/ceph to /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666511/remote/smithi179/log 2024-04-21T08:54:46.341 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /var/log/ceph -- . 2024-04-21T08:54:47.692 INFO:tasks.cephadm:Removing cluster... 2024-04-21T08:54:47.692 DEBUG:teuthology.orchestra.run.smithi086:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 --force 2024-04-21T08:54:47.881 INFO:teuthology.orchestra.run.smithi086.stdout:Deleting cluster with fsid: f8cec7cc-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:54:48.317 DEBUG:teuthology.orchestra.run.smithi088:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 --force 2024-04-21T08:54:48.506 INFO:teuthology.orchestra.run.smithi088.stdout:Deleting cluster with fsid: f8cec7cc-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:54:49.036 DEBUG:teuthology.orchestra.run.smithi179:> sudo /home/ubuntu/cephtest/cephadm rm-cluster --fsid f8cec7cc-ffb8-11ee-bc93-c7b262605968 --force 2024-04-21T08:54:49.224 INFO:teuthology.orchestra.run.smithi179.stdout:Deleting cluster with fsid: f8cec7cc-ffb8-11ee-bc93-c7b262605968 2024-04-21T08:54:49.489 INFO:tasks.cephadm:Removing cephadm ... 2024-04-21T08:54:49.489 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-21T08:54:49.508 DEBUG:teuthology.orchestra.run.smithi088:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-21T08:54:49.523 DEBUG:teuthology.orchestra.run.smithi179:> rm -rf /home/ubuntu/cephtest/cephadm 2024-04-21T08:54:49.538 INFO:tasks.cephadm:Teardown complete 2024-04-21T08:54:49.538 DEBUG:teuthology.run_tasks:Unwinding manager install 2024-04-21T08:54:49.554 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2024-04-21T08:54:49.554 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-21T08:54:49.557 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-21T08:54:49.566 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2024-04-21T08:54:49.662 DEBUG:teuthology.run_tasks:Unwinding manager clock 2024-04-21T08:54:49.672 INFO:teuthology.task.clock:Checking final clock skew... 2024-04-21T08:54:49.672 DEBUG:teuthology.orchestra.run.smithi086:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-21T08:54:49.674 DEBUG:teuthology.orchestra.run.smithi088:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-21T08:54:49.677 DEBUG:teuthology.orchestra.run.smithi179:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2024-04-21T08:54:49.688 INFO:teuthology.orchestra.run.smithi086.stderr:bash: line 1: ntpq: command not found 2024-04-21T08:54:49.688 INFO:teuthology.orchestra.run.smithi088.stderr:bash: line 1: ntpq: command not found 2024-04-21T08:54:49.690 INFO:teuthology.orchestra.run.smithi179.stderr:bash: line 1: ntpq: command not found 2024-04-21T08:54:49.693 INFO:teuthology.orchestra.run.smithi088.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-21T08:54:49.693 INFO:teuthology.orchestra.run.smithi088.stdout:=============================================================================== 2024-04-21T08:54:49.693 INFO:teuthology.orchestra.run.smithi088.stdout:^+ hv01.front.sepia.ceph.com 2 8 377 145 -16us[-4153ns] +/- 75ms 2024-04-21T08:54:49.693 INFO:teuthology.orchestra.run.smithi088.stdout:^* hv02.front.sepia.ceph.com 3 7 377 15 -412us[ -418us] +/- 62ms 2024-04-21T08:54:49.693 INFO:teuthology.orchestra.run.smithi088.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 14 +522us[ +522us] +/- 82ms 2024-04-21T08:54:49.693 INFO:teuthology.orchestra.run.smithi088.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:54:49.694 INFO:teuthology.orchestra.run.smithi086.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-21T08:54:49.694 INFO:teuthology.orchestra.run.smithi086.stdout:=============================================================================== 2024-04-21T08:54:49.694 INFO:teuthology.orchestra.run.smithi086.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 13 -53us[ -53us] +/- 77ms 2024-04-21T08:54:49.694 INFO:teuthology.orchestra.run.smithi086.stdout:^* hv02.front.sepia.ceph.com 3 7 377 13 -386us[ -400us] +/- 62ms 2024-04-21T08:54:49.694 INFO:teuthology.orchestra.run.smithi086.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 15 +551us[ +537us] +/- 82ms 2024-04-21T08:54:49.694 INFO:teuthology.orchestra.run.smithi086.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:54:49.695 INFO:teuthology.orchestra.run.smithi179.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2024-04-21T08:54:49.695 INFO:teuthology.orchestra.run.smithi179.stdout:=============================================================================== 2024-04-21T08:54:49.695 INFO:teuthology.orchestra.run.smithi179.stdout:^+ hv01.front.sepia.ceph.com 2 6 377 18 -20us[ -24us] +/- 77ms 2024-04-21T08:54:49.696 INFO:teuthology.orchestra.run.smithi179.stdout:^* hv02.front.sepia.ceph.com 3 6 377 15 -397us[ -401us] +/- 62ms 2024-04-21T08:54:49.696 INFO:teuthology.orchestra.run.smithi179.stdout:^+ hv03.front.sepia.ceph.com 3 6 377 15 +568us[ +568us] +/- 82ms 2024-04-21T08:54:49.696 INFO:teuthology.orchestra.run.smithi179.stdout:^? hv04.front.sepia.ceph.com 0 9 0 - +0ns[ +0ns] +/- 0ns 2024-04-21T08:54:49.696 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2024-04-21T08:54:49.707 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2024-04-21T08:54:49.707 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2024-04-21T08:54:49.717 DEBUG:teuthology.orchestra.run.smithi086:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-21T08:54:49.818 DEBUG:teuthology.orchestra.run.smithi088:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-21T08:54:49.883 DEBUG:teuthology.orchestra.run.smithi179:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2024-04-21T08:54:49.947 DEBUG:teuthology.orchestra.run.smithi086:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-21T08:54:49.976 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T08:54:49.977 DEBUG:teuthology.orchestra.run.smithi088:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-21T08:54:50.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T08:54:50.003 DEBUG:teuthology.orchestra.run.smithi179:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e scontext=system_u:system_r:getty_t:s0 2024-04-21T08:54:50.029 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T08:54:50.030 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2024-04-21T08:54:50.042 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2024-04-21T08:54:50.053 INFO:teuthology.task.internal:Duration was 2002.384320 seconds 2024-04-21T08:54:50.054 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2024-04-21T08:54:50.065 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2024-04-21T08:54:50.065 DEBUG:teuthology.orchestra.run.smithi086:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-21T08:54:50.067 DEBUG:teuthology.orchestra.run.smithi088:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-21T08:54:50.070 DEBUG:teuthology.orchestra.run.smithi179:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2024-04-21T08:54:50.103 INFO:teuthology.orchestra.run.smithi088.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-21T08:54:50.103 INFO:teuthology.orchestra.run.smithi086.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-21T08:54:50.105 INFO:teuthology.orchestra.run.smithi179.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2024-04-21T08:54:50.502 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2024-04-21T08:54:50.502 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi086.front.sepia.ceph.com 2024-04-21T08:54:50.502 DEBUG:teuthology.orchestra.run.smithi086:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-21T08:54:50.523 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi088.front.sepia.ceph.com 2024-04-21T08:54:50.523 DEBUG:teuthology.orchestra.run.smithi088:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-21T08:54:50.581 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@smithi179.front.sepia.ceph.com 2024-04-21T08:54:50.581 DEBUG:teuthology.orchestra.run.smithi179:> egrep --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | egrep -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | egrep -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2024-04-21T08:54:50.607 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2024-04-21T08:54:50.607 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-21T08:54:50.609 DEBUG:teuthology.orchestra.run.smithi088:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-21T08:54:50.623 DEBUG:teuthology.orchestra.run.smithi179:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --no-run-if-empty -- gzip -- 2024-04-21T08:54:50.672 INFO:teuthology.task.internal.syslog:Gathering journactl -b0... 2024-04-21T08:54:50.672 DEBUG:teuthology.orchestra.run.smithi086:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-21T08:54:50.677 DEBUG:teuthology.orchestra.run.smithi088:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-21T08:54:50.688 DEBUG:teuthology.orchestra.run.smithi179:> sudo journalctl -b0 | gzip -9 > /home/ubuntu/cephtest/archive/syslog/journalctl-b0.gz 2024-04-21T08:54:51.089 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2024-04-21T08:54:51.102 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2024-04-21T08:54:51.102 DEBUG:teuthology.orchestra.run.smithi086:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-21T08:54:51.125 DEBUG:teuthology.orchestra.run.smithi088:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-21T08:54:51.149 DEBUG:teuthology.orchestra.run.smithi179:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2024-04-21T08:54:51.173 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2024-04-21T08:54:51.186 DEBUG:teuthology.orchestra.run.smithi086:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-21T08:54:51.188 DEBUG:teuthology.orchestra.run.smithi088:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-21T08:54:51.192 DEBUG:teuthology.orchestra.run.smithi179:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2024-04-21T08:54:51.208 INFO:teuthology.orchestra.run.smithi086.stdout:kernel.core_pattern = core 2024-04-21T08:54:51.211 INFO:teuthology.orchestra.run.smithi088.stdout:kernel.core_pattern = core 2024-04-21T08:54:51.235 INFO:teuthology.orchestra.run.smithi179.stdout:kernel.core_pattern = core 2024-04-21T08:54:51.252 DEBUG:teuthology.orchestra.run.smithi086:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-21T08:54:51.274 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T08:54:51.274 DEBUG:teuthology.orchestra.run.smithi088:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-21T08:54:51.287 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T08:54:51.288 DEBUG:teuthology.orchestra.run.smithi179:> test -e /home/ubuntu/cephtest/archive/coredump 2024-04-21T08:54:51.306 DEBUG:teuthology.orchestra.run:got remote process result: 1 2024-04-21T08:54:51.306 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2024-04-21T08:54:51.319 INFO:teuthology.task.internal:Transferring archived files... 2024-04-21T08:54:51.319 DEBUG:teuthology.misc:Transferring archived files from smithi086:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666511/remote/smithi086 2024-04-21T08:54:51.320 DEBUG:teuthology.orchestra.run.smithi086:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-21T08:54:51.361 DEBUG:teuthology.misc:Transferring archived files from smithi088:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666511/remote/smithi088 2024-04-21T08:54:51.361 DEBUG:teuthology.orchestra.run.smithi088:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-21T08:54:51.395 DEBUG:teuthology.misc:Transferring archived files from smithi179:/home/ubuntu/cephtest/archive to /home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666511/remote/smithi179 2024-04-21T08:54:51.396 DEBUG:teuthology.orchestra.run.smithi179:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2024-04-21T08:54:51.431 INFO:teuthology.task.internal:Removing archive directory... 2024-04-21T08:54:51.431 DEBUG:teuthology.orchestra.run.smithi086:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-21T08:54:51.434 DEBUG:teuthology.orchestra.run.smithi088:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-21T08:54:51.436 DEBUG:teuthology.orchestra.run.smithi179:> rm -rf -- /home/ubuntu/cephtest/archive 2024-04-21T08:54:51.479 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2024-04-21T08:54:51.492 INFO:teuthology.task.internal:Not uploading archives. 2024-04-21T08:54:51.492 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2024-04-21T08:54:51.504 INFO:teuthology.task.internal:Tidying up after the test... 2024-04-21T08:54:51.505 DEBUG:teuthology.orchestra.run.smithi086:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-21T08:54:51.507 DEBUG:teuthology.orchestra.run.smithi088:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-21T08:54:51.509 DEBUG:teuthology.orchestra.run.smithi179:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2024-04-21T08:54:51.519 INFO:teuthology.orchestra.run.smithi086.stdout: 262213 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 21 08:54 /home/ubuntu/cephtest 2024-04-21T08:54:51.521 INFO:teuthology.orchestra.run.smithi088.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 21 08:54 /home/ubuntu/cephtest 2024-04-21T08:54:51.533 INFO:teuthology.orchestra.run.smithi179.stdout: 262161 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Apr 21 08:54 /home/ubuntu/cephtest 2024-04-21T08:54:51.535 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2024-04-21T08:54:51.547 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2024-04-21T08:54:51.605 INFO:teuthology.nuke:Checking targets against current locks 2024-04-21T08:54:51.630 DEBUG:teuthology.nuke:shortname: smithi086 2024-04-21T08:54:51.630 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-21T08:54:51.652 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi086.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666511', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-21 08:15:32.418863', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBBYfbXPoZ7D8BJKoi2VUL62N9ZlIfp6xJxHHCbXcCmEINmZC3+9zF50s3gdqfg271tP9z3/iq7Bx5j4Hf0/NLoU='} 2024-04-21T08:54:51.658 DEBUG:teuthology.nuke:shortname: smithi088 2024-04-21T08:54:51.659 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-21T08:54:51.682 DEBUG:teuthology.nuke:shortname: smithi179 2024-04-21T08:54:51.682 INFO:teuthology.task.internal.check_lock:Checking locks... 2024-04-21T08:54:51.685 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi088.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666511', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-21 08:15:32.421991', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI1Hyc0xyjsQLdKgYCVOBgiyqDPLM5T0eiIiym5swZ7i9ORD0a10OvDztobC2iwb4Aa3ggytXWvbSMkahIgj4Ug='} 2024-04-21T08:54:51.695 INFO:teuthology.orchestra.console.smithi086:Power off 2024-04-21T08:54:51.696 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-21T08:54:51.707 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'smithi179.front.sepia.ceph.com', 'description': '/home/teuthworker/archive/leonidus-2024-04-21_08:12:55-fs-wip-lusov-quiescer-distro-default-smithi/7666511', 'up': True, 'machine_type': 'smithi', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9', 'arch': 'x86_64', 'locked': True, 'locked_since': '2024-04-21 08:15:32.420592', 'locked_by': 'scheduled_leonidus@teuthology', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBPnr8CAdOQWzcezeu0TweNh/qMpja8vU18k8zRiz2xPUnqGVrCgFU078WM4ypon/wpPvxgc+yvadBIfb7w8XJTY='} 2024-04-21T08:54:51.720 DEBUG:teuthology.orchestra.console.smithi086:power off output: Chassis Power Control: Down/Off 2024-04-21T08:54:51.720 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T08:54:51.737 INFO:teuthology.orchestra.console.smithi088:Power off 2024-04-21T08:54:51.738 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-21T08:54:51.751 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is on 2024-04-21T08:54:51.760 DEBUG:teuthology.orchestra.console.smithi088:power off output: Chassis Power Control: Down/Off 2024-04-21T08:54:51.760 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T08:54:51.775 INFO:teuthology.orchestra.console.smithi179:Power off 2024-04-21T08:54:51.775 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2024-04-21T08:54:51.785 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is on 2024-04-21T08:54:51.798 DEBUG:teuthology.orchestra.console.smithi179:power off output: Chassis Power Control: Down/Off 2024-04-21T08:54:51.799 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T08:54:51.822 DEBUG:teuthology.orchestra.console.smithi179:check power output: Chassis Power is on 2024-04-21T08:54:55.752 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T08:54:55.787 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T08:54:55.823 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T08:54:55.863 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is on 2024-04-21T08:54:55.896 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is on 2024-04-21T08:54:55.932 DEBUG:teuthology.orchestra.console.smithi179:check power output: Chassis Power is on 2024-04-21T08:54:59.866 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T08:54:59.898 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T08:54:59.933 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T08:54:59.976 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is on 2024-04-21T08:55:00.008 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is on 2024-04-21T08:55:00.043 DEBUG:teuthology.orchestra.console.smithi179:check power output: Chassis Power is on 2024-04-21T08:55:03.978 DEBUG:teuthology.orchestra.console.smithi086:pexpect command: ipmitool -H smithi086.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T08:55:04.008 DEBUG:teuthology.orchestra.console.smithi088:pexpect command: ipmitool -H smithi088.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T08:55:04.045 DEBUG:teuthology.orchestra.console.smithi179:pexpect command: ipmitool -H smithi179.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2024-04-21T08:55:04.089 DEBUG:teuthology.orchestra.console.smithi086:check power output: Chassis Power is off 2024-04-21T08:55:04.118 DEBUG:teuthology.orchestra.console.smithi088:check power output: Chassis Power is off 2024-04-21T08:55:04.154 DEBUG:teuthology.orchestra.console.smithi179:check power output: Chassis Power is off 2024-04-21T08:55:04.190 INFO:teuthology.orchestra.console.smithi086:Power off completed 2024-04-21T08:55:04.218 INFO:teuthology.orchestra.console.smithi088:Power off completed 2024-04-21T08:55:04.255 INFO:teuthology.orchestra.console.smithi179:Power off completed 2024-04-21T08:55:04.360 INFO:teuthology.run:Summary data: description: fs/workload/{0-ubuntu_22.04 begin/{0-install 1-cephadm 2-logrotate 3-modules} clusters/1a11s-mds-1c-client-3node conf/{client mds mgr mon osd} mount/kclient/{base/{mount-syntax/{v1} mount overrides/{distro/stock/{centos_9.stream k-stock} ms-die-on-skipped}} ms_mode/secure wsync/yes} objectstore-ec/bluestore-ec-root omap_limit/10000 overrides/{cephsqlite-timeout frag ignorelist_health ignorelist_wrongly_marked_down osd-asserts pg_health session_timeout} ranks/multi/{balancer/automatic export-check n/5 replication/always} standby-replay tasks/{0-subvolume/{with-quota} 1-check-counter 2-scrub/yes 3-snaps/yes 4-flush/yes 5-quiesce/with-quiesce 6-workunit/fs/test_o_trunc}} duration: 2002.3843202590942 failure_reason: 'error during scrub thrashing: ' flavor: default owner: scheduled_leonidus@teuthology status: fail success: false 2024-04-21T08:55:04.360 DEBUG:teuthology.report:Pushing job info to https://paddles.front.sepia.ceph.com/ 2024-04-21T08:55:04.449 INFO:teuthology.run:FAIL